2024-09-19T06:29:23.292 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T06:29:23.321 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T06:29:23.424 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911754 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911754' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi103.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMrxlK4usRAF1eaRRB+y62T5Yuy20nN1IfNViVxlTnDM7heF/aQYSZkYUpZGHPdhI/TuSnZzM95I5uaiwwIl6IM= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJH+B6REFkylT1CW3gqF/Bn3gzYmbuUt5y+sj51b7C/vpLbfAnpBy2rjGi33VsVDq4Bi/xNmqtvC1i9toqFfKgA= smithi174.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPzDVgBTc35dRCglDINnGrV/dSbNaP8WDP+HJCaKHuUuqVVnW0DqsLqCODUUtmWQNicMCCk96yM0KrGeIasaYng= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T06:29:23.425 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T06:29:23.425 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T06:29:23.426 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T06:29:23.427 INFO:teuthology.task.internal:Checking packages... 2024-09-19T06:29:23.449 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T06:29:23.449 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T06:29:23.449 INFO:teuthology.packaging:ref: None 2024-09-19T06:29:23.449 INFO:teuthology.packaging:tag: None 2024-09-19T06:29:23.450 INFO:teuthology.packaging:branch: squid 2024-09-19T06:29:23.450 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:29:23.450 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T06:29:23.696 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T06:29:23.698 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T06:29:23.705 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T06:29:23.706 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T06:29:23.714 INFO:teuthology.task.internal:Saving configuration 2024-09-19T06:29:23.735 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T06:29:23.743 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T06:29:23.767 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911754', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 06:23:54.625008', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMrxlK4usRAF1eaRRB+y62T5Yuy20nN1IfNViVxlTnDM7heF/aQYSZkYUpZGHPdhI/TuSnZzM95I5uaiwwIl6IM='} 2024-09-19T06:29:23.791 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911754', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 06:23:54.628672', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJH+B6REFkylT1CW3gqF/Bn3gzYmbuUt5y+sj51b7C/vpLbfAnpBy2rjGi33VsVDq4Bi/xNmqtvC1i9toqFfKgA='} 2024-09-19T06:29:23.815 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911754', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 06:23:54.627069', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPzDVgBTc35dRCglDINnGrV/dSbNaP8WDP+HJCaKHuUuqVVnW0DqsLqCODUUtmWQNicMCCk96yM0KrGeIasaYng='} 2024-09-19T06:29:23.816 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T06:29:23.823 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T06:29:23.823 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T06:29:23.823 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T06:29:23.823 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T06:29:23.910 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fee48bcf520>, signals=[15]) 2024-09-19T06:29:23.911 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T06:29:23.918 INFO:teuthology.task.internal:Opening connections... 2024-09-19T06:29:23.918 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T06:29:23.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:29:23.998 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-09-19T06:29:23.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:29:24.074 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2024-09-19T06:29:24.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:29:24.150 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T06:29:24.159 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2024-09-19T06:29:24.177 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2024-09-19T06:29:24.177 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-09-19T06:29:24.235 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-09-19T06:29:24.235 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-09-19T06:29:24.235 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-09-19T06:29:24.235 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-09-19T06:29:24.235 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-09-19T06:29:24.235 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-09-19T06:29:24.235 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T06:29:24.235 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-09-19T06:29:24.235 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-09-19T06:29:24.236 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T06:29:24.236 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-09-19T06:29:24.236 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T06:29:24.236 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T06:29:24.236 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T06:29:24.237 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-09-19T06:29:24.257 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-09-19T06:29:24.276 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-09-19T06:29:24.276 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-09-19T06:29:24.333 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-09-19T06:29:24.333 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-09-19T06:29:24.333 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-09-19T06:29:24.333 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-09-19T06:29:24.334 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-09-19T06:29:24.334 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-09-19T06:29:24.334 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T06:29:24.334 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-09-19T06:29:24.334 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-09-19T06:29:24.334 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T06:29:24.334 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-09-19T06:29:24.334 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T06:29:24.334 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T06:29:24.334 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T06:29:24.334 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-09-19T06:29:24.357 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2024-09-19T06:29:24.376 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2024-09-19T06:29:24.376 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-09-19T06:29:24.434 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2024-09-19T06:29:24.435 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2024-09-19T06:29:24.435 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2024-09-19T06:29:24.435 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2024-09-19T06:29:24.435 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2024-09-19T06:29:24.435 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2024-09-19T06:29:24.435 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T06:29:24.435 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2024-09-19T06:29:24.435 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2024-09-19T06:29:24.435 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T06:29:24.435 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2024-09-19T06:29:24.435 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T06:29:24.435 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T06:29:24.435 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T06:29:24.436 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-09-19T06:29:24.457 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T06:29:24.469 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T06:29:24.476 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T06:29:24.476 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2024-09-19T06:29:24.479 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-09-19T06:29:24.481 DEBUG:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2024-09-19T06:29:24.498 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T06:29:24.505 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T06:29:24.505 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2024-09-19T06:29:24.536 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-09-19T06:29:24.539 DEBUG:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2024-09-19T06:29:24.645 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T06:29:24.724 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T06:29:24.740 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-19T06:29:24.740 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-19T06:29:24.740 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-19T06:29:24.741 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T06:29:24.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:29:24.758 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-09-19T06:29:24.817 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-503.el9.x86_64 2024-09-19T06:29:24.817 INFO:teuthology.task.kernel:Running kernel on smithi103: 5.14.0-503.el9.x86_64 2024-09-19T06:29:24.818 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum install -y kernel 2024-09-19T06:29:27.215 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-19T06:29:32.775 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 20 MB 00:04 2024-09-19T06:29:36.976 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-19T06:29:38.710 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 19 kB 00:00 2024-09-19T06:29:39.580 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-19T06:29:44.978 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-19T06:29:46.893 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T06:29:46.893 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T06:29:47.001 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-19T06:29:47.013 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-19T06:29:47.013 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-09-19T06:29:47.013 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-19T06:29:47.013 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-19T06:29:47.013 INFO:teuthology.orchestra.run.smithi103.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T06:29:47.013 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-19T06:29:47.013 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T06:29:47.013 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T06:29:47.013 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T06:29:47.013 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:29:47.013 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-19T06:29:47.013 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-19T06:29:47.013 INFO:teuthology.orchestra.run.smithi103.stdout:Install 4 Packages 2024-09-19T06:29:47.014 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:29:47.014 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 79 M 2024-09-19T06:29:47.014 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 126 M 2024-09-19T06:29:47.014 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-19T06:29:47.679 INFO:teuthology.orchestra.run.smithi103.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 64 kB/s | 36 kB 00:00 2024-09-19T06:29:48.505 INFO:teuthology.orchestra.run.smithi103.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-19T06:29:49.172 INFO:teuthology.orchestra.run.smithi103.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 17 MB/s | 35 MB 00:02 2024-09-19T06:29:50.080 INFO:teuthology.orchestra.run.smithi103.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 12 MB/s | 29 MB 00:02 2024-09-19T06:29:50.081 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:29:50.081 INFO:teuthology.orchestra.run.smithi103.stdout:Total 26 MB/s | 79 MB 00:03 2024-09-19T06:29:50.765 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-19T06:29:51.120 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-19T06:29:51.120 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-19T06:29:52.072 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-19T06:29:52.073 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-19T06:29:54.535 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-19T06:29:55.170 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T06:29:55.249 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T06:29:58.035 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T06:29:58.177 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T06:30:02.994 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T06:30:03.102 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:30:07.967 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:30:46.561 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:30:46.605 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:30:47.008 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:30:47.008 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T06:30:47.008 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T06:30:47.008 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T06:30:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:30:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:30:47.867 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-19T06:30:47.867 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T06:30:47.868 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T06:30:47.868 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T06:30:47.868 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T06:30:47.868 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:30:47.868 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-19T06:30:48.201 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T06:30:48.284 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T06:30:48.285 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T06:30:48.285 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T06:30:48.285 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T06:30:48.285 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T06:30:48.285 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-19T06:30:48.285 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi103.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T06:30:48.286 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T06:30:48.344 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T06:30:48.344 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T06:30:48.344 DEBUG:teuthology.orchestra.run.smithi103:> sudo rpm -qi grub2-tools 2024-09-19T06:30:48.376 INFO:teuthology.orchestra.run.smithi103.stdout:Name : grub2-tools 2024-09-19T06:30:48.376 INFO:teuthology.orchestra.run.smithi103.stdout:Epoch : 1 2024-09-19T06:30:48.376 INFO:teuthology.orchestra.run.smithi103.stdout:Version : 2.06 2024-09-19T06:30:48.376 INFO:teuthology.orchestra.run.smithi103.stdout:Release : 82.el9 2024-09-19T06:30:48.377 INFO:teuthology.orchestra.run.smithi103.stdout:Architecture: x86_64 2024-09-19T06:30:48.377 INFO:teuthology.orchestra.run.smithi103.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T06:30:48.377 INFO:teuthology.orchestra.run.smithi103.stdout:Group : Unspecified 2024-09-19T06:30:48.377 INFO:teuthology.orchestra.run.smithi103.stdout:Size : 8274670 2024-09-19T06:30:48.377 INFO:teuthology.orchestra.run.smithi103.stdout:License : GPLv3+ 2024-09-19T06:30:48.377 INFO:teuthology.orchestra.run.smithi103.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T06:30:48.377 INFO:teuthology.orchestra.run.smithi103.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T06:30:48.377 INFO:teuthology.orchestra.run.smithi103.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T06:30:48.377 INFO:teuthology.orchestra.run.smithi103.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T06:30:48.377 INFO:teuthology.orchestra.run.smithi103.stdout:Packager : builder@centos.org 2024-09-19T06:30:48.377 INFO:teuthology.orchestra.run.smithi103.stdout:Vendor : CentOS 2024-09-19T06:30:48.377 INFO:teuthology.orchestra.run.smithi103.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T06:30:48.377 INFO:teuthology.orchestra.run.smithi103.stdout:Summary : Support tools for GRUB. 2024-09-19T06:30:48.377 INFO:teuthology.orchestra.run.smithi103.stdout:Description : 2024-09-19T06:30:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:30:48.378 INFO:teuthology.orchestra.run.smithi103.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T06:30:48.378 INFO:teuthology.orchestra.run.smithi103.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T06:30:48.378 INFO:teuthology.orchestra.run.smithi103.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T06:30:48.378 INFO:teuthology.orchestra.run.smithi103.stdout:hardware devices. 2024-09-19T06:30:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:30:48.378 INFO:teuthology.orchestra.run.smithi103.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T06:30:48.379 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T06:30:48.379 INFO:teuthology.task.kernel:Updating grub on smithi103 to boot 5.14.0-508.el9.x86_64 2024-09-19T06:30:48.379 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T06:30:48.987 INFO:teuthology.orchestra.run.smithi103.stderr:Generating grub configuration file ... 2024-09-19T06:30:50.260 INFO:teuthology.orchestra.run.smithi103.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T06:30:50.287 INFO:teuthology.orchestra.run.smithi103.stderr:done 2024-09-19T06:30:50.293 DEBUG:teuthology.orchestra.run.smithi103:> mktemp 2024-09-19T06:30:50.309 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/tmp.wYeBiK2rNF 2024-09-19T06:30:50.310 DEBUG:teuthology.orchestra.run.smithi103:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wYeBiK2rNF 2024-09-19T06:30:50.377 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0666 /tmp/tmp.wYeBiK2rNF 2024-09-19T06:30:50.524 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.wYeBiK2rNF is 6KB 2024-09-19T06:30:50.545 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.wYeBiK2rNF 2024-09-19T06:30:50.561 DEBUG:teuthology.orchestra.run.smithi103:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T06:30:50.626 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T06:30:50.626 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T06:30:50.627 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T06:30:50.627 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T06:30:50.628 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T06:30:50.781 DEBUG:teuthology.orchestra.run.smithi103:> sudo shutdown -r now 2024-09-19T06:31:20.784 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-19T06:31:20.785 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T06:31:20.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:31:39.325 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-09-19T06:31:48.325 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T06:31:48.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:31:51.389 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-09-19T06:32:03.390 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T06:32:03.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:32:06.461 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-09-19T06:32:21.462 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T06:32:21.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:32:53.322 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-09-19T06:32:53.630 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T06:32:53.630 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-19T06:32:53.631 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-09-19T06:32:53.645 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-508.el9.x86_64 2024-09-19T06:32:53.645 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T06:32:53.645 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T06:32:53.645 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T06:32:54.646 DEBUG:teuthology.parallel:result is None 2024-09-19T06:32:54.646 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T06:32:54.654 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T06:32:54.654 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T06:32:54.657 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T06:32:54.660 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T06:32:54.678 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T06:32:54.687 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T06:32:54.700 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T06:32:54.701 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T06:32:54.714 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T06:32:54.718 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T06:32:54.826 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T06:32:54.835 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T06:32:54.835 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T06:32:54.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:32:54.853 DEBUG:teuthology.orchestra.run.smithi174:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T06:32:54.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:32:54.870 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T06:32:54.872 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T06:32:54.895 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T06:32:54.937 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T06:32:54.950 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T06:32:54.953 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T06:32:54.959 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T06:32:54.975 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T06:32:54.980 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T06:32:54.982 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T06:32:54.993 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T06:32:54.994 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T06:32:54.996 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T06:32:54.999 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T06:32:55.028 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T06:32:55.042 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T06:32:55.042 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T06:32:55.065 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T06:32:55.068 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T06:32:55.086 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T06:32:55.162 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T06:32:55.227 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T06:32:55.271 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T06:32:55.312 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:32:55.313 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T06:32:55.379 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T06:32:55.423 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T06:32:55.472 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T06:32:55.561 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T06:32:55.599 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T06:32:55.600 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T06:32:55.666 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T06:32:55.705 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T06:32:55.768 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T06:32:55.842 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T06:32:55.896 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T06:32:55.896 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T06:32:55.965 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2024-09-19T06:32:55.967 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-09-19T06:32:55.969 DEBUG:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2024-09-19T06:32:56.008 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T06:32:56.017 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T06:32:56.044 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T06:32:56.477 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T06:32:56.486 INFO:teuthology.task.internal:Starting timer... 2024-09-19T06:32:56.486 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T06:32:56.500 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T06:32:56.509 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T06:32:56.509 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2024-09-19T06:32:56.553 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2024-09-19T06:32:56.554 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2024-09-19T06:32:56.603 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2024-09-19T06:32:56.605 DEBUG:teuthology.orchestra.run.smithi174:> sudo service auditd rotate 2024-09-19T06:32:56.648 INFO:teuthology.orchestra.run.smithi174.stdout:Rotating logs: 2024-09-19T06:32:56.650 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T06:32:56.650 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2024-09-19T06:32:56.675 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2024-09-19T06:32:56.675 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2024-09-19T06:32:56.697 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2024-09-19T06:32:56.698 DEBUG:teuthology.orchestra.run.smithi174:> /usr/sbin/getenforce 2024-09-19T06:32:56.733 INFO:teuthology.orchestra.run.smithi174.stdout:Permissive 2024-09-19T06:32:56.733 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi103.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi174.front.sepia.ceph.com': 'permissive'} 2024-09-19T06:32:56.733 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T06:32:56.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:32:56.762 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T06:32:56.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:32:56.791 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T06:32:56.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:32:56.822 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T06:32:56.822 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2024-09-19T06:32:56.859 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2024-09-19T06:32:56.891 DEBUG:teuthology.orchestra.run.smithi174:> sudo /usr/sbin/setenforce permissive 2024-09-19T06:32:56.925 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T06:32:56.934 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T06:32:56.946 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T06:32:56.947 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi103.front.sepia.ceph.com,smithi134.front.sepia.ceph.com,smithi174.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T06:36:58.046 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi103.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com'), Remote(name='ubuntu@smithi174.front.sepia.ceph.com')] 2024-09-19T06:36:58.047 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T06:36:58.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:36:58.128 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-09-19T06:36:58.204 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T06:36:58.205 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-19T06:36:58.205 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:36:58.283 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-09-19T06:36:58.373 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-19T06:36:58.374 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-19T06:36:58.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:36:58.450 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-09-19T06:36:58.538 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-19T06:36:58.538 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T06:36:58.549 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T06:36:58.549 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T06:36:58.550 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T06:36:58.552 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T06:36:58.552 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T06:36:58.555 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T06:36:58.555 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T06:36:58.581 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T06:36:58.583 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T06:36:58.596 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T06:36:58.598 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T06:36:58.610 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T06:36:58.622 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2024-09-19T06:36:58.623 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2024-09-19T06:36:58.625 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T06:36:58.633 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2024-09-19T06:36:58.635 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2024-09-19T06:36:58.649 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T06:36:58.649 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T06:36:58.652 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: ntpd: command not found 2024-09-19T06:36:58.662 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T06:36:58.664 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T06:36:58.664 INFO:teuthology.orchestra.run.smithi174.stdout:506 Cannot talk to daemon 2024-09-19T06:36:58.678 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T06:36:58.693 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T06:36:58.717 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-19T06:36:58.720 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T06:36:58.720 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-19T06:36:58.720 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:36:58.720 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:36:58.720 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:36:58.720 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:36:58.721 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-09-19T06:36:58.724 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T06:36:58.724 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-09-19T06:36:58.724 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:36:58.725 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:36:58.725 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:36:58.725 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:36:58.749 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-09-19T06:36:58.752 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T06:36:58.752 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-09-19T06:36:58.753 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:36:58.753 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:36:58.753 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:36:58.753 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:36:58.754 INFO:teuthology.run_tasks:Running task install... 2024-09-19T06:36:58.763 DEBUG:teuthology.task.install:project ceph 2024-09-19T06:36:58.763 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T06:36:58.763 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T06:36:58.763 INFO:teuthology.task.install:Using flavor: default 2024-09-19T06:36:58.768 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T06:36:58.768 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T06:36:58.769 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T06:36:58.769 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:36:58.772 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T06:36:58.772 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:36:58.773 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T06:36:58.773 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:36:58.915 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T06:36:58.915 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T06:36:58.923 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T06:36:58.923 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T06:36:58.931 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T06:36:58.931 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T06:36:59.026 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T06:36:59.026 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:36:59.026 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T06:36:59.032 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T06:36:59.032 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T06:36:59.032 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T06:36:59.040 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T06:36:59.040 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T06:36:59.040 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T06:36:59.052 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T06:36:59.052 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T06:36:59.057 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T06:36:59.058 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T06:36:59.066 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T06:36:59.067 DEBUG:teuthology.orchestra.run.smithi174:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T06:36:59.117 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T06:36:59.123 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T06:36:59.133 DEBUG:teuthology.orchestra.run.smithi174:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T06:36:59.192 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T06:36:59.198 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T06:36:59.210 DEBUG:teuthology.orchestra.run.smithi174:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T06:36:59.256 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2024-09-19T06:36:59.257 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2024-09-19T06:36:59.262 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2024-09-19T06:36:59.263 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2024-09-19T06:36:59.275 INFO:teuthology.orchestra.run.smithi174.stdout:check_obsoletes = 1 2024-09-19T06:36:59.277 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum clean all 2024-09-19T06:36:59.489 INFO:teuthology.orchestra.run.smithi134.stdout:45 files removed 2024-09-19T06:36:59.491 INFO:teuthology.orchestra.run.smithi103.stdout:45 files removed 2024-09-19T06:36:59.502 INFO:teuthology.orchestra.run.smithi174.stdout:45 files removed 2024-09-19T06:36:59.515 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T06:36:59.518 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T06:36:59.529 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T06:37:00.069 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 301 kB/s | 84 kB 00:00 2024-09-19T06:37:00.071 INFO:teuthology.orchestra.run.smithi174.stdout:ceph packages for x86_64 314 kB/s | 84 kB 00:00 2024-09-19T06:37:00.076 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 306 kB/s | 84 kB 00:00 2024-09-19T06:37:00.337 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-19T06:37:00.355 INFO:teuthology.orchestra.run.smithi174.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-19T06:37:00.376 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 46 kB/s | 12 kB 00:00 2024-09-19T06:37:00.595 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-19T06:37:00.639 INFO:teuthology.orchestra.run.smithi174.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-19T06:37:00.652 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-19T06:37:01.193 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.3 MB 00:00 2024-09-19T06:37:01.272 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-19T06:37:02.245 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.3 MB 00:01 2024-09-19T06:37:03.988 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-19T06:37:04.490 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 9.4 MB/s | 20 MB 00:02 2024-09-19T06:37:09.097 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 5.2 MB/s | 6.5 MB 00:01 2024-09-19T06:37:09.832 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.5 MB 00:01 2024-09-19T06:37:10.336 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 2.5 MB/s | 20 MB 00:08 2024-09-19T06:37:10.714 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 19 kB 00:00 2024-09-19T06:37:11.316 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 75 kB/s | 19 kB 00:00 2024-09-19T06:37:11.877 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-19T06:37:12.619 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-19T06:37:16.388 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.5 MB 00:02 2024-09-19T06:37:17.316 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-19T06:37:18.013 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 83 kB/s | 19 kB 00:00 2024-09-19T06:37:18.134 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-19T06:37:19.076 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-19T06:37:19.452 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T06:37:19.452 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T06:37:19.464 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T06:37:19.465 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T06:37:19.466 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T06:37:19.467 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T06:37:19.468 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T06:37:19.471 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T06:37:19.472 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T06:37:19.533 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-19T06:37:19.558 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-19T06:37:19.558 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-19T06:37:19.559 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-19T06:37:19.559 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-19T06:37:19.559 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T06:37:19.559 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T06:37:19.559 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T06:37:19.559 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T06:37:19.559 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T06:37:19.559 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T06:37:19.559 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T06:37:19.559 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T06:37:19.559 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T06:37:19.559 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T06:37:19.560 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T06:37:19.560 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T06:37:19.560 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T06:37:19.560 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T06:37:19.560 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T06:37:19.560 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T06:37:19.560 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T06:37:19.560 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T06:37:19.560 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T06:37:19.560 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T06:37:19.560 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T06:37:19.560 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T06:37:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T06:37:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T06:37:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T06:37:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T06:37:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T06:37:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T06:37:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T06:37:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T06:37:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T06:37:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T06:37:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T06:37:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T06:37:19.561 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T06:37:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T06:37:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T06:37:19.562 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-09-19T06:37:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T06:37:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T06:37:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T06:37:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T06:37:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T06:37:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T06:37:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T06:37:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T06:37:19.562 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-19T06:37:19.562 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T06:37:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T06:37:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T06:37:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T06:37:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T06:37:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T06:37:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T06:37:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T06:37:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T06:37:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T06:37:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T06:37:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T06:37:19.563 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T06:37:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T06:37:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T06:37:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T06:37:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T06:37:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T06:37:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T06:37:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T06:37:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T06:37:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T06:37:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T06:37:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T06:37:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T06:37:19.564 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T06:37:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T06:37:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T06:37:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T06:37:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T06:37:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T06:37:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T06:37:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T06:37:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T06:37:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T06:37:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T06:37:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T06:37:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T06:37:19.565 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T06:37:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T06:37:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T06:37:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T06:37:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T06:37:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T06:37:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T06:37:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T06:37:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T06:37:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T06:37:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T06:37:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T06:37:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T06:37:19.566 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T06:37:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T06:37:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T06:37:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T06:37:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T06:37:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T06:37:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T06:37:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T06:37:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T06:37:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T06:37:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T06:37:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T06:37:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T06:37:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T06:37:19.567 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T06:37:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T06:37:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T06:37:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T06:37:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T06:37:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T06:37:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T06:37:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T06:37:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T06:37:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T06:37:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T06:37:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T06:37:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T06:37:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T06:37:19.568 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T06:37:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T06:37:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T06:37:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T06:37:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T06:37:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T06:37:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T06:37:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T06:37:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T06:37:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T06:37:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T06:37:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T06:37:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T06:37:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T06:37:19.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T06:37:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T06:37:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T06:37:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T06:37:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T06:37:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T06:37:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T06:37:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T06:37:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T06:37:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T06:37:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T06:37:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T06:37:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T06:37:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T06:37:19.570 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T06:37:19.571 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T06:37:19.571 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T06:37:19.571 INFO:teuthology.orchestra.run.smithi134.stdout:Installing weak dependencies: 2024-09-19T06:37:19.571 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T06:37:19.571 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:37:19.571 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-19T06:37:19.571 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-19T06:37:19.571 INFO:teuthology.orchestra.run.smithi134.stdout:Install 148 Packages 2024-09-19T06:37:19.571 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 8 Packages 2024-09-19T06:37:19.571 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:37:19.572 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 226 M 2024-09-19T06:37:19.572 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-19T06:37:19.936 INFO:teuthology.orchestra.run.smithi134.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 48 kB/s | 6.6 kB 00:00 2024-09-19T06:37:20.127 INFO:teuthology.orchestra.run.smithi134.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 4.6 MB/s | 896 kB 00:00 2024-09-19T06:37:20.197 INFO:teuthology.orchestra.run.smithi134.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.1 MB/s | 145 kB 00:00 2024-09-19T06:37:20.269 INFO:teuthology.orchestra.run.smithi174.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T06:37:20.275 INFO:teuthology.orchestra.run.smithi174.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T06:37:20.293 INFO:teuthology.orchestra.run.smithi174.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T06:37:20.294 INFO:teuthology.orchestra.run.smithi174.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T06:37:20.294 INFO:teuthology.orchestra.run.smithi174.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T06:37:20.294 INFO:teuthology.orchestra.run.smithi174.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T06:37:20.295 INFO:teuthology.orchestra.run.smithi174.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T06:37:20.296 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T06:37:20.297 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T06:37:20.351 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-09-19T06:37:20.361 INFO:teuthology.orchestra.run.smithi134.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 9.6 MB/s | 5.4 MB 00:00 2024-09-19T06:37:20.367 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-09-19T06:37:20.367 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-09-19T06:37:20.367 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-09-19T06:37:20.367 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-09-19T06:37:20.367 INFO:teuthology.orchestra.run.smithi174.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T06:37:20.367 INFO:teuthology.orchestra.run.smithi174.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T06:37:20.367 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T06:37:20.367 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T06:37:20.367 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T06:37:20.367 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T06:37:20.367 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T06:37:20.368 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T06:37:20.368 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T06:37:20.368 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T06:37:20.368 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T06:37:20.368 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T06:37:20.368 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T06:37:20.368 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T06:37:20.368 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T06:37:20.368 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T06:37:20.368 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T06:37:20.369 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T06:37:20.369 INFO:teuthology.orchestra.run.smithi174.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T06:37:20.369 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T06:37:20.369 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T06:37:20.369 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T06:37:20.369 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T06:37:20.370 INFO:teuthology.orchestra.run.smithi174.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T06:37:20.370 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T06:37:20.370 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T06:37:20.370 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T06:37:20.370 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T06:37:20.370 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T06:37:20.371 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T06:37:20.371 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T06:37:20.371 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T06:37:20.371 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T06:37:20.371 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T06:37:20.371 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T06:37:20.371 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T06:37:20.372 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T06:37:20.372 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrading: 2024-09-19T06:37:20.372 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T06:37:20.372 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T06:37:20.372 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T06:37:20.372 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T06:37:20.373 INFO:teuthology.orchestra.run.smithi174.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T06:37:20.373 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T06:37:20.373 INFO:teuthology.orchestra.run.smithi174.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T06:37:20.373 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T06:37:20.373 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-09-19T06:37:20.373 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T06:37:20.374 INFO:teuthology.orchestra.run.smithi174.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T06:37:20.374 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T06:37:20.374 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T06:37:20.374 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T06:37:20.374 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T06:37:20.375 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T06:37:20.375 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T06:37:20.375 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T06:37:20.376 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T06:37:20.376 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T06:37:20.376 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T06:37:20.376 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T06:37:20.376 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T06:37:20.376 INFO:teuthology.orchestra.run.smithi174.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T06:37:20.377 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T06:37:20.377 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T06:37:20.377 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T06:37:20.377 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T06:37:20.377 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T06:37:20.378 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T06:37:20.378 INFO:teuthology.orchestra.run.smithi174.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T06:37:20.378 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T06:37:20.378 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T06:37:20.378 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T06:37:20.378 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T06:37:20.378 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T06:37:20.379 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T06:37:20.379 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T06:37:20.379 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T06:37:20.379 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T06:37:20.379 INFO:teuthology.orchestra.run.smithi174.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T06:37:20.379 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T06:37:20.380 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T06:37:20.380 INFO:teuthology.orchestra.run.smithi174.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T06:37:20.380 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T06:37:20.380 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T06:37:20.380 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T06:37:20.381 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T06:37:20.381 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T06:37:20.381 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T06:37:20.381 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T06:37:20.381 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T06:37:20.381 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T06:37:20.381 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T06:37:20.381 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T06:37:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T06:37:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T06:37:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T06:37:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T06:37:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T06:37:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T06:37:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T06:37:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T06:37:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T06:37:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T06:37:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T06:37:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T06:37:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T06:37:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T06:37:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T06:37:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T06:37:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T06:37:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T06:37:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T06:37:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T06:37:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T06:37:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T06:37:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T06:37:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T06:37:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T06:37:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T06:37:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T06:37:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T06:37:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T06:37:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T06:37:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T06:37:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T06:37:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T06:37:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T06:37:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T06:37:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T06:37:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T06:37:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T06:37:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T06:37:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T06:37:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T06:37:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T06:37:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T06:37:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T06:37:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T06:37:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T06:37:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T06:37:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T06:37:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T06:37:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T06:37:20.386 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T06:37:20.386 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T06:37:20.386 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T06:37:20.386 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T06:37:20.386 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T06:37:20.386 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T06:37:20.386 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T06:37:20.386 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T06:37:20.386 INFO:teuthology.orchestra.run.smithi174.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T06:37:20.386 INFO:teuthology.orchestra.run.smithi174.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T06:37:20.386 INFO:teuthology.orchestra.run.smithi174.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T06:37:20.387 INFO:teuthology.orchestra.run.smithi174.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T06:37:20.387 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T06:37:20.387 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T06:37:20.387 INFO:teuthology.orchestra.run.smithi174.stdout:Installing weak dependencies: 2024-09-19T06:37:20.387 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T06:37:20.387 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:37:20.387 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-09-19T06:37:20.387 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-09-19T06:37:20.387 INFO:teuthology.orchestra.run.smithi174.stdout:Install 148 Packages 2024-09-19T06:37:20.388 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade 8 Packages 2024-09-19T06:37:20.388 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:37:20.388 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 226 M 2024-09-19T06:37:20.389 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-09-19T06:37:20.444 INFO:teuthology.orchestra.run.smithi134.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 9.5 MB/s | 2.4 MB 00:00 2024-09-19T06:37:20.519 INFO:teuthology.orchestra.run.smithi134.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 9.6 MB/s | 1.5 MB 00:00 2024-09-19T06:37:20.979 INFO:teuthology.orchestra.run.smithi134.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 8.8 MB/s | 4.7 MB 00:00 2024-09-19T06:37:21.311 INFO:teuthology.orchestra.run.smithi134.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 14 MB/s | 21 MB 00:01 2024-09-19T06:37:21.344 INFO:teuthology.orchestra.run.smithi174.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 54 kB/s | 6.6 kB 00:00 2024-09-19T06:37:21.364 INFO:teuthology.orchestra.run.smithi134.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 507 kB/s | 25 kB 00:00 2024-09-19T06:37:21.553 INFO:teuthology.orchestra.run.smithi134.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 16 MB/s | 17 MB 00:01 2024-09-19T06:37:21.620 INFO:teuthology.orchestra.run.smithi134.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.4 MB/s | 228 kB 00:00 2024-09-19T06:37:21.769 INFO:teuthology.orchestra.run.smithi174.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 2.1 MB/s | 896 kB 00:00 2024-09-19T06:37:21.770 INFO:teuthology.orchestra.run.smithi134.stdout:(12/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 13 MB/s | 11 MB 00:00 2024-09-19T06:37:21.796 INFO:teuthology.orchestra.run.smithi134.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 154 kB/s | 27 kB 00:00 2024-09-19T06:37:21.821 INFO:teuthology.orchestra.run.smithi134.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 637 kB/s | 32 kB 00:00 2024-09-19T06:37:21.836 INFO:teuthology.orchestra.run.smithi174.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.1 MB/s | 145 kB 00:00 2024-09-19T06:37:21.863 INFO:teuthology.orchestra.run.smithi134.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 11 MB/s | 751 kB 00:00 2024-09-19T06:37:21.888 INFO:teuthology.orchestra.run.smithi134.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.4 MB/s | 163 kB 00:00 2024-09-19T06:37:21.913 INFO:teuthology.orchestra.run.smithi134.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T06:37:21.955 INFO:teuthology.orchestra.run.smithi134.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 7.4 MB/s | 506 kB 00:00 2024-09-19T06:37:22.022 INFO:teuthology.orchestra.run.smithi134.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 678 kB/s | 45 kB 00:00 2024-09-19T06:37:22.115 INFO:teuthology.orchestra.run.smithi134.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.4 MB/s | 136 kB 00:00 2024-09-19T06:37:22.190 INFO:teuthology.orchestra.run.smithi134.stdout:(21/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 19 MB/s | 5.3 MB 00:00 2024-09-19T06:37:22.223 INFO:teuthology.orchestra.run.smithi134.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.5 MB/s | 161 kB 00:00 2024-09-19T06:37:22.248 INFO:teuthology.orchestra.run.smithi134.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 5.3 MB/s | 320 kB 00:00 2024-09-19T06:37:22.274 INFO:teuthology.orchestra.run.smithi134.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.9 MB/s | 300 kB 00:00 2024-09-19T06:37:22.299 INFO:teuthology.orchestra.run.smithi134.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T06:37:22.324 INFO:teuthology.orchestra.run.smithi134.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T06:37:22.328 INFO:teuthology.orchestra.run.smithi174.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 4.9 MB/s | 5.4 MB 00:01 2024-09-19T06:37:22.392 INFO:teuthology.orchestra.run.smithi134.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-19T06:37:22.420 INFO:teuthology.orchestra.run.smithi174.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 4.1 MB/s | 2.4 MB 00:00 2024-09-19T06:37:22.450 INFO:teuthology.orchestra.run.smithi134.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 21 MB/s | 3.1 MB 00:00 2024-09-19T06:37:22.475 INFO:teuthology.orchestra.run.smithi134.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 382 kB/s | 32 kB 00:00 2024-09-19T06:37:22.509 INFO:teuthology.orchestra.run.smithi134.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.5 MB/s | 146 kB 00:00 2024-09-19T06:37:22.528 INFO:teuthology.orchestra.run.smithi174.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 7.6 MB/s | 1.5 MB 00:00 2024-09-19T06:37:22.801 INFO:teuthology.orchestra.run.smithi134.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 11 MB/s | 3.6 MB 00:00 2024-09-19T06:37:22.868 INFO:teuthology.orchestra.run.smithi134.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 3.7 MB/s | 249 kB 00:00 2024-09-19T06:37:22.960 INFO:teuthology.orchestra.run.smithi134.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 16 MB/s | 7.4 MB 00:00 2024-09-19T06:37:22.985 INFO:teuthology.orchestra.run.smithi134.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 422 kB/s | 49 kB 00:00 2024-09-19T06:37:23.004 INFO:teuthology.orchestra.run.smithi134.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 395 kB/s | 16 kB 00:00 2024-09-19T06:37:23.052 INFO:teuthology.orchestra.run.smithi134.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 4.2 MB/s | 285 kB 00:00 2024-09-19T06:37:23.094 INFO:teuthology.orchestra.run.smithi134.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 8.1 MB/s | 765 kB 00:00 2024-09-19T06:37:23.111 INFO:teuthology.orchestra.run.smithi134.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 312 kB/s | 18 kB 00:00 2024-09-19T06:37:23.362 INFO:teuthology.orchestra.run.smithi174.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 5.0 MB/s | 4.7 MB 00:00 2024-09-19T06:37:23.911 INFO:teuthology.orchestra.run.smithi134.stdout:(39/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 19 MB/s | 49 MB 00:02 2024-09-19T06:37:23.944 INFO:teuthology.orchestra.run.smithi134.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 94 kB/s | 80 kB 00:00 2024-09-19T06:37:24.111 INFO:teuthology.orchestra.run.smithi134.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 962 kB/s | 191 kB 00:00 2024-09-19T06:37:24.179 INFO:teuthology.orchestra.run.smithi174.stdout:(8/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86_ 10 MB/s | 17 MB 00:01 2024-09-19T06:37:24.212 INFO:teuthology.orchestra.run.smithi174.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 763 kB/s | 25 kB 00:00 2024-09-19T06:37:24.507 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-19T06:37:24.587 INFO:teuthology.orchestra.run.smithi174.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 8.7 MB/s | 11 MB 00:01 2024-09-19T06:37:24.637 INFO:teuthology.orchestra.run.smithi174.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 4.5 MB/s | 228 kB 00:00 2024-09-19T06:37:24.688 INFO:teuthology.orchestra.run.smithi174.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 543 kB/s | 27 kB 00:00 2024-09-19T06:37:24.729 INFO:teuthology.orchestra.run.smithi174.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 765 kB/s | 32 kB 00:00 2024-09-19T06:37:24.980 INFO:teuthology.orchestra.run.smithi174.stdout:(14/156): ceph-common-19.1.1-361.g2f599a46.el9. 5.6 MB/s | 21 MB 00:03 2024-09-19T06:37:25.013 INFO:teuthology.orchestra.run.smithi174.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 2.6 MB/s | 751 kB 00:00 2024-09-19T06:37:25.047 INFO:teuthology.orchestra.run.smithi174.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.4 MB/s | 163 kB 00:00 2024-09-19T06:37:25.064 INFO:teuthology.orchestra.run.smithi174.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T06:37:25.139 INFO:teuthology.orchestra.run.smithi174.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 5.4 MB/s | 506 kB 00:00 2024-09-19T06:37:25.189 INFO:teuthology.orchestra.run.smithi174.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 905 kB/s | 45 kB 00:00 2024-09-19T06:37:25.240 INFO:teuthology.orchestra.run.smithi174.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.7 MB/s | 136 kB 00:00 2024-09-19T06:37:25.290 INFO:teuthology.orchestra.run.smithi174.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 3.2 MB/s | 161 kB 00:00 2024-09-19T06:37:25.357 INFO:teuthology.orchestra.run.smithi174.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 4.7 MB/s | 320 kB 00:00 2024-09-19T06:37:25.424 INFO:teuthology.orchestra.run.smithi174.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 4.4 MB/s | 300 kB 00:00 2024-09-19T06:37:25.474 INFO:teuthology.orchestra.run.smithi174.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T06:37:25.516 INFO:teuthology.orchestra.run.smithi174.stdout:(25/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 2.0 MB/s | 85 kB 00:00 2024-09-19T06:37:25.650 INFO:teuthology.orchestra.run.smithi174.stdout:(26/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 9.1 MB/s | 5.3 MB 00:00 2024-09-19T06:37:25.700 INFO:teuthology.orchestra.run.smithi174.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 3.4 MB/s | 171 kB 00:00 2024-09-19T06:37:25.742 INFO:teuthology.orchestra.run.smithi174.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 771 kB/s | 32 kB 00:00 2024-09-19T06:37:25.792 INFO:teuthology.orchestra.run.smithi174.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.9 MB/s | 146 kB 00:00 2024-09-19T06:37:25.909 INFO:teuthology.orchestra.run.smithi174.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 7.9 MB/s | 3.1 MB 00:00 2024-09-19T06:37:25.919 INFO:teuthology.orchestra.run.smithi134.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 128 kB/s | 253 kB 00:01 2024-09-19T06:37:25.953 INFO:teuthology.orchestra.run.smithi134.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 282 kB/s | 801 kB 00:02 2024-09-19T06:37:25.978 INFO:teuthology.orchestra.run.smithi134.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-19T06:37:26.003 INFO:teuthology.orchestra.run.smithi134.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-19T06:37:26.037 INFO:teuthology.orchestra.run.smithi134.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-19T06:37:26.087 INFO:teuthology.orchestra.run.smithi134.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-19T06:37:26.168 INFO:teuthology.orchestra.run.smithi174.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 9.5 MB/s | 3.6 MB 00:00 2024-09-19T06:37:26.210 INFO:teuthology.orchestra.run.smithi174.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 5.9 MB/s | 249 kB 00:00 2024-09-19T06:37:26.243 INFO:teuthology.orchestra.run.smithi174.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 1.4 MB/s | 49 kB 00:00 2024-09-19T06:37:26.277 INFO:teuthology.orchestra.run.smithi174.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 495 kB/s | 16 kB 00:00 2024-09-19T06:37:26.344 INFO:teuthology.orchestra.run.smithi174.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 4.2 MB/s | 285 kB 00:00 2024-09-19T06:37:26.444 INFO:teuthology.orchestra.run.smithi174.stdout:(36/156): cephadm-19.1.1-361.g2f599a46.el9.noar 7.5 MB/s | 765 kB 00:00 2024-09-19T06:37:26.486 INFO:teuthology.orchestra.run.smithi174.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 441 kB/s | 18 kB 00:00 2024-09-19T06:37:26.686 INFO:teuthology.orchestra.run.smithi174.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 9.5 MB/s | 7.4 MB 00:00 2024-09-19T06:37:26.712 INFO:teuthology.orchestra.run.smithi174.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 354 kB/s | 80 kB 00:00 2024-09-19T06:37:26.725 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T06:37:26.726 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T06:37:26.738 INFO:teuthology.orchestra.run.smithi103.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T06:37:26.739 INFO:teuthology.orchestra.run.smithi103.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T06:37:26.739 INFO:teuthology.orchestra.run.smithi103.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T06:37:26.740 INFO:teuthology.orchestra.run.smithi103.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T06:37:26.742 INFO:teuthology.orchestra.run.smithi103.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T06:37:26.744 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T06:37:26.745 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T06:37:26.808 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-19T06:37:26.822 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-19T06:37:26.822 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-19T06:37:26.823 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-19T06:37:26.823 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-19T06:37:26.823 INFO:teuthology.orchestra.run.smithi103.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T06:37:26.823 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T06:37:26.823 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T06:37:26.823 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T06:37:26.823 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T06:37:26.823 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T06:37:26.823 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T06:37:26.823 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T06:37:26.823 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T06:37:26.823 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T06:37:26.824 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T06:37:26.824 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T06:37:26.824 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T06:37:26.824 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T06:37:26.824 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T06:37:26.824 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T06:37:26.824 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T06:37:26.824 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T06:37:26.824 INFO:teuthology.orchestra.run.smithi103.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T06:37:26.825 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T06:37:26.825 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T06:37:26.825 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T06:37:26.825 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T06:37:26.825 INFO:teuthology.orchestra.run.smithi103.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T06:37:26.825 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T06:37:26.825 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T06:37:26.825 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T06:37:26.825 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T06:37:26.825 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T06:37:26.825 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T06:37:26.825 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T06:37:26.826 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T06:37:26.826 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T06:37:26.826 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T06:37:26.826 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T06:37:26.826 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T06:37:26.826 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T06:37:26.826 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2024-09-19T06:37:26.826 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T06:37:26.826 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T06:37:26.826 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T06:37:26.826 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T06:37:26.826 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T06:37:26.826 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T06:37:26.827 INFO:teuthology.orchestra.run.smithi103.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T06:37:26.827 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T06:37:26.827 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-19T06:37:26.827 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T06:37:26.827 INFO:teuthology.orchestra.run.smithi103.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T06:37:26.827 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T06:37:26.827 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T06:37:26.827 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T06:37:26.827 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T06:37:26.827 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T06:37:26.827 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T06:37:26.827 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T06:37:26.827 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T06:37:26.828 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T06:37:26.828 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T06:37:26.828 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T06:37:26.828 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T06:37:26.828 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T06:37:26.828 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T06:37:26.828 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T06:37:26.828 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T06:37:26.828 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T06:37:26.828 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T06:37:26.828 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T06:37:26.828 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T06:37:26.828 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T06:37:26.829 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T06:37:26.829 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T06:37:26.829 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T06:37:26.829 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T06:37:26.829 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T06:37:26.829 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T06:37:26.829 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T06:37:26.829 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T06:37:26.829 INFO:teuthology.orchestra.run.smithi103.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T06:37:26.829 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T06:37:26.829 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T06:37:26.829 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T06:37:26.829 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T06:37:26.830 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T06:37:26.830 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T06:37:26.830 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T06:37:26.830 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T06:37:26.830 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T06:37:26.830 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T06:37:26.830 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T06:37:26.830 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T06:37:26.830 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T06:37:26.830 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T06:37:26.830 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T06:37:26.830 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T06:37:26.830 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T06:37:26.831 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T06:37:26.831 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T06:37:26.831 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T06:37:26.831 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T06:37:26.831 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T06:37:26.831 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T06:37:26.831 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T06:37:26.831 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T06:37:26.831 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T06:37:26.831 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T06:37:26.831 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T06:37:26.831 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T06:37:26.831 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T06:37:26.831 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T06:37:26.832 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T06:37:26.832 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T06:37:26.832 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T06:37:26.832 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T06:37:26.832 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T06:37:26.832 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T06:37:26.832 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T06:37:26.832 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T06:37:26.832 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T06:37:26.832 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T06:37:26.832 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T06:37:26.832 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T06:37:26.832 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T06:37:26.833 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T06:37:26.833 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T06:37:26.833 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T06:37:26.833 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T06:37:26.833 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T06:37:26.833 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T06:37:26.833 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T06:37:26.833 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T06:37:26.833 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T06:37:26.833 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T06:37:26.833 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T06:37:26.833 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T06:37:26.833 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T06:37:26.834 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T06:37:26.834 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T06:37:26.834 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T06:37:26.834 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T06:37:26.834 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T06:37:26.834 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T06:37:26.834 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T06:37:26.834 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T06:37:26.834 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T06:37:26.834 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T06:37:26.834 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T06:37:26.834 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T06:37:26.834 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T06:37:26.835 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T06:37:26.835 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T06:37:26.835 INFO:teuthology.orchestra.run.smithi103.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T06:37:26.835 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T06:37:26.835 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T06:37:26.835 INFO:teuthology.orchestra.run.smithi103.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T06:37:26.835 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T06:37:26.835 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T06:37:26.835 INFO:teuthology.orchestra.run.smithi103.stdout:Installing weak dependencies: 2024-09-19T06:37:26.835 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T06:37:26.835 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:37:26.836 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-19T06:37:26.836 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-19T06:37:26.836 INFO:teuthology.orchestra.run.smithi103.stdout:Install 148 Packages 2024-09-19T06:37:26.836 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 8 Packages 2024-09-19T06:37:26.836 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:37:26.836 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 226 M 2024-09-19T06:37:26.837 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-19T06:37:26.837 INFO:teuthology.orchestra.run.smithi174.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 5.2 MB/s | 801 kB 00:00 2024-09-19T06:37:26.887 INFO:teuthology.orchestra.run.smithi174.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-09-19T06:37:26.912 INFO:teuthology.orchestra.run.smithi134.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 787 kB/s | 689 kB 00:00 2024-09-19T06:37:26.929 INFO:teuthology.orchestra.run.smithi174.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 881 kB/s | 191 kB 00:00 2024-09-19T06:37:26.979 INFO:teuthology.orchestra.run.smithi174.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-09-19T06:37:27.021 INFO:teuthology.orchestra.run.smithi174.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-09-19T06:37:27.046 INFO:teuthology.orchestra.run.smithi174.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-19T06:37:27.071 INFO:teuthology.orchestra.run.smithi134.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 959 kB/s | 943 kB 00:00 2024-09-19T06:37:27.122 INFO:teuthology.orchestra.run.smithi174.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-19T06:37:27.147 INFO:teuthology.orchestra.run.smithi174.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 995 kB/s | 125 kB 00:00 2024-09-19T06:37:27.289 INFO:teuthology.orchestra.run.smithi174.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-09-19T06:37:27.322 INFO:teuthology.orchestra.run.smithi174.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 5.3 MB/s | 943 kB 00:00 2024-09-19T06:37:27.348 INFO:teuthology.orchestra.run.smithi174.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-19T06:37:27.372 INFO:teuthology.orchestra.run.smithi174.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 445 kB/s | 22 kB 00:00 2024-09-19T06:37:27.397 INFO:teuthology.orchestra.run.smithi174.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.3 MB/s | 313 kB 00:00 2024-09-19T06:37:27.422 INFO:teuthology.orchestra.run.smithi174.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-09-19T06:37:27.604 INFO:teuthology.orchestra.run.smithi134.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 155 kB/s | 107 kB 00:00 2024-09-19T06:37:27.630 INFO:teuthology.orchestra.run.smithi134.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 39 kB/s | 22 kB 00:00 2024-09-19T06:37:27.688 INFO:teuthology.orchestra.run.smithi134.stdout:(52/156): python3-cryptography-36.0.1-4.el9.x86 351 kB/s | 1.2 MB 00:03 2024-09-19T06:37:27.831 INFO:teuthology.orchestra.run.smithi174.stdout:(54/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 13 MB/s | 49 MB 00:03 2024-09-19T06:37:27.856 INFO:teuthology.orchestra.run.smithi174.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 42 kB/s | 18 kB 00:00 2024-09-19T06:37:27.898 INFO:teuthology.orchestra.run.smithi174.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 168 kB/s | 11 kB 00:00 2024-09-19T06:37:27.923 INFO:teuthology.orchestra.run.smithi174.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-19T06:37:27.974 INFO:teuthology.orchestra.run.smithi174.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.2 MB/s | 3.0 MB 00:00 2024-09-19T06:37:27.998 INFO:teuthology.orchestra.run.smithi174.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-09-19T06:37:28.032 INFO:teuthology.orchestra.run.smithi174.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 9.7 MB/s | 581 kB 00:00 2024-09-19T06:37:28.057 INFO:teuthology.orchestra.run.smithi174.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-19T06:37:28.091 INFO:teuthology.orchestra.run.smithi174.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-19T06:37:28.116 INFO:teuthology.orchestra.run.smithi174.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-09-19T06:37:28.142 INFO:teuthology.orchestra.run.smithi174.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-19T06:37:28.167 INFO:teuthology.orchestra.run.smithi174.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-19T06:37:28.180 INFO:teuthology.orchestra.run.smithi134.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-09-19T06:37:28.181 INFO:teuthology.orchestra.run.smithi103.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 20 kB/s | 6.6 kB 00:00 2024-09-19T06:37:28.192 INFO:teuthology.orchestra.run.smithi174.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-19T06:37:28.326 INFO:teuthology.orchestra.run.smithi174.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 27 MB/s | 4.9 MB 00:00 2024-09-19T06:37:28.367 INFO:teuthology.orchestra.run.smithi103.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 4.7 MB/s | 896 kB 00:00 2024-09-19T06:37:28.384 INFO:teuthology.orchestra.run.smithi174.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-19T06:37:28.397 INFO:teuthology.orchestra.run.smithi134.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 396 kB/s | 313 kB 00:00 2024-09-19T06:37:28.426 INFO:teuthology.orchestra.run.smithi103.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.4 MB/s | 145 kB 00:00 2024-09-19T06:37:28.426 INFO:teuthology.orchestra.run.smithi174.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 8.9 MB/s | 910 kB 00:00 2024-09-19T06:37:28.443 INFO:teuthology.orchestra.run.smithi174.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-19T06:37:28.485 INFO:teuthology.orchestra.run.smithi174.stdout:(71/156): postgresql-13.16-1.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-09-19T06:37:28.527 INFO:teuthology.orchestra.run.smithi174.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-19T06:37:28.552 INFO:teuthology.orchestra.run.smithi174.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-19T06:37:28.584 INFO:teuthology.orchestra.run.smithi103.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 7.3 MB/s | 5.4 MB 00:00 2024-09-19T06:37:28.594 INFO:teuthology.orchestra.run.smithi174.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-19T06:37:28.664 INFO:teuthology.orchestra.run.smithi134.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 38 kB/s | 18 kB 00:00 2024-09-19T06:37:28.669 INFO:teuthology.orchestra.run.smithi174.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-09-19T06:37:28.676 INFO:teuthology.orchestra.run.smithi103.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 9.5 MB/s | 2.4 MB 00:00 2024-09-19T06:37:28.694 INFO:teuthology.orchestra.run.smithi174.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-19T06:37:28.720 INFO:teuthology.orchestra.run.smithi174.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-19T06:37:28.735 INFO:teuthology.orchestra.run.smithi103.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 10 MB/s | 1.5 MB 00:00 2024-09-19T06:37:28.736 INFO:teuthology.orchestra.run.smithi174.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 613 kB/s | 26 kB 00:00 2024-09-19T06:37:28.748 INFO:teuthology.orchestra.run.smithi134.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.8 MB/s | 3.0 MB 00:01 2024-09-19T06:37:28.753 INFO:teuthology.orchestra.run.smithi174.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 571 kB/s | 19 kB 00:00 2024-09-19T06:37:28.779 INFO:teuthology.orchestra.run.smithi174.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-09-19T06:37:28.796 INFO:teuthology.orchestra.run.smithi174.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 834 kB/s | 35 kB 00:00 2024-09-19T06:37:28.907 INFO:teuthology.orchestra.run.smithi174.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-19T06:37:28.922 INFO:teuthology.orchestra.run.smithi134.stdout:(57/156): libaio-devel-0.3.111-13.el9.x86_64.rp 21 kB/s | 11 kB 00:00 2024-09-19T06:37:29.005 INFO:teuthology.orchestra.run.smithi174.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 250 kB 00:00 2024-09-19T06:37:29.030 INFO:teuthology.orchestra.run.smithi174.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.8 MB/s | 222 kB 00:00 2024-09-19T06:37:29.047 INFO:teuthology.orchestra.run.smithi174.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-09-19T06:37:29.072 INFO:teuthology.orchestra.run.smithi174.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-19T06:37:29.077 INFO:teuthology.orchestra.run.smithi103.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-19T06:37:29.106 INFO:teuthology.orchestra.run.smithi174.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-19T06:37:29.131 INFO:teuthology.orchestra.run.smithi174.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-19T06:37:29.157 INFO:teuthology.orchestra.run.smithi174.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-19T06:37:29.181 INFO:teuthology.orchestra.run.smithi174.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T06:37:29.206 INFO:teuthology.orchestra.run.smithi174.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T06:37:29.290 INFO:teuthology.orchestra.run.smithi174.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-09-19T06:37:29.315 INFO:teuthology.orchestra.run.smithi174.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-19T06:37:29.341 INFO:teuthology.orchestra.run.smithi174.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-09-19T06:37:29.366 INFO:teuthology.orchestra.run.smithi174.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-19T06:37:29.373 INFO:teuthology.orchestra.run.smithi134.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 72 kB/s | 45 kB 00:00 2024-09-19T06:37:29.400 INFO:teuthology.orchestra.run.smithi174.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 611 kB/s | 36 kB 00:00 2024-09-19T06:37:29.425 INFO:teuthology.orchestra.run.smithi174.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 805 kB/s | 47 kB 00:00 2024-09-19T06:37:29.427 INFO:teuthology.orchestra.run.smithi103.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 13 MB/s | 21 MB 00:01 2024-09-19T06:37:29.450 INFO:teuthology.orchestra.run.smithi174.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-19T06:37:29.461 INFO:teuthology.orchestra.run.smithi103.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 761 kB/s | 25 kB 00:00 2024-09-19T06:37:29.475 INFO:teuthology.orchestra.run.smithi174.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-19T06:37:29.482 INFO:teuthology.orchestra.run.smithi134.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 196 kB/s | 160 kB 00:00 2024-09-19T06:37:29.501 INFO:teuthology.orchestra.run.smithi174.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-19T06:37:29.526 INFO:teuthology.orchestra.run.smithi174.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T06:37:29.551 INFO:teuthology.orchestra.run.smithi174.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-19T06:37:29.576 INFO:teuthology.orchestra.run.smithi174.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 443 kB/s | 22 kB 00:00 2024-09-19T06:37:29.627 INFO:teuthology.orchestra.run.smithi174.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T06:37:29.660 INFO:teuthology.orchestra.run.smithi174.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-19T06:37:29.685 INFO:teuthology.orchestra.run.smithi174.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 777 kB/s | 19 kB 00:00 2024-09-19T06:37:29.747 INFO:teuthology.orchestra.run.smithi103.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 17 MB/s | 17 MB 00:01 2024-09-19T06:37:29.799 INFO:teuthology.orchestra.run.smithi134.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 763 kB/s | 662 kB 00:00 2024-09-19T06:37:29.878 INFO:teuthology.orchestra.run.smithi103.stdout:(11/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 13 MB/s | 11 MB 00:00 2024-09-19T06:37:29.897 INFO:teuthology.orchestra.run.smithi174.stdout:(107/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-19T06:37:29.903 INFO:teuthology.orchestra.run.smithi103.stdout:(12/156): cephfs-mirror-19.1.1-361.g2f599a46.el 1.4 MB/s | 228 kB 00:00 2024-09-19T06:37:29.929 INFO:teuthology.orchestra.run.smithi103.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 539 kB/s | 27 kB 00:00 2024-09-19T06:37:29.936 INFO:teuthology.orchestra.run.smithi174.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-09-19T06:37:29.954 INFO:teuthology.orchestra.run.smithi103.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 630 kB/s | 32 kB 00:00 2024-09-19T06:37:29.969 INFO:teuthology.orchestra.run.smithi174.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 362 kB/s | 26 kB 00:00 2024-09-19T06:37:29.995 INFO:teuthology.orchestra.run.smithi174.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.8 MB/s | 67 kB 00:00 2024-09-19T06:37:29.996 INFO:teuthology.orchestra.run.smithi103.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 11 MB/s | 751 kB 00:00 2024-09-19T06:37:30.020 INFO:teuthology.orchestra.run.smithi174.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 591 kB/s | 49 kB 00:00 2024-09-19T06:37:30.021 INFO:teuthology.orchestra.run.smithi103.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.4 MB/s | 163 kB 00:00 2024-09-19T06:37:30.045 INFO:teuthology.orchestra.run.smithi174.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-19T06:37:30.049 INFO:teuthology.orchestra.run.smithi134.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 43 kB/s | 24 kB 00:00 2024-09-19T06:37:30.055 INFO:teuthology.orchestra.run.smithi103.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.1 MB/s | 127 kB 00:00 2024-09-19T06:37:30.079 INFO:teuthology.orchestra.run.smithi174.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-19T06:37:30.080 INFO:teuthology.orchestra.run.smithi103.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 8.4 MB/s | 506 kB 00:00 2024-09-19T06:37:30.131 INFO:teuthology.orchestra.run.smithi103.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 902 kB/s | 45 kB 00:00 2024-09-19T06:37:30.137 INFO:teuthology.orchestra.run.smithi174.stdout:(114/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-09-19T06:37:30.171 INFO:teuthology.orchestra.run.smithi174.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 472 kB/s | 43 kB 00:00 2024-09-19T06:37:30.205 INFO:teuthology.orchestra.run.smithi174.stdout:(116/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.4 MB/s | 839 kB 00:00 2024-09-19T06:37:30.214 INFO:teuthology.orchestra.run.smithi103.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.6 MB/s | 136 kB 00:00 2024-09-19T06:37:30.230 INFO:teuthology.orchestra.run.smithi174.stdout:(117/156): python3-cachetools-4.2.4-1.el9.noarc 349 kB/s | 32 kB 00:00 2024-09-19T06:37:30.232 INFO:teuthology.orchestra.run.smithi134.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 676 kB/s | 581 kB 00:00 2024-09-19T06:37:30.255 INFO:teuthology.orchestra.run.smithi174.stdout:(118/156): python3-certifi-2023.05.07-4.el9.noa 169 kB/s | 14 kB 00:00 2024-09-19T06:37:30.281 INFO:teuthology.orchestra.run.smithi174.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-09-19T06:37:30.306 INFO:teuthology.orchestra.run.smithi174.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-19T06:37:30.323 INFO:teuthology.orchestra.run.smithi103.stdout:(21/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 20 MB/s | 5.3 MB 00:00 2024-09-19T06:37:30.339 INFO:teuthology.orchestra.run.smithi174.stdout:(121/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-19T06:37:30.348 INFO:teuthology.orchestra.run.smithi103.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.2 MB/s | 161 kB 00:00 2024-09-19T06:37:30.364 INFO:teuthology.orchestra.run.smithi174.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-09-19T06:37:30.374 INFO:teuthology.orchestra.run.smithi103.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 6.2 MB/s | 320 kB 00:00 2024-09-19T06:37:30.381 INFO:teuthology.orchestra.run.smithi174.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-09-19T06:37:30.382 INFO:teuthology.orchestra.run.smithi134.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 81 kB/s | 47 kB 00:00 2024-09-19T06:37:30.399 INFO:teuthology.orchestra.run.smithi103.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.8 MB/s | 300 kB 00:00 2024-09-19T06:37:30.407 INFO:teuthology.orchestra.run.smithi174.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 256 kB/s | 11 kB 00:00 2024-09-19T06:37:30.424 INFO:teuthology.orchestra.run.smithi103.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T06:37:30.432 INFO:teuthology.orchestra.run.smithi174.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-19T06:37:30.449 INFO:teuthology.orchestra.run.smithi174.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 553 kB/s | 23 kB 00:00 2024-09-19T06:37:30.449 INFO:teuthology.orchestra.run.smithi103.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T06:37:30.466 INFO:teuthology.orchestra.run.smithi174.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 578 kB/s | 19 kB 00:00 2024-09-19T06:37:30.491 INFO:teuthology.orchestra.run.smithi174.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 468 kB/s | 20 kB 00:00 2024-09-19T06:37:30.508 INFO:teuthology.orchestra.run.smithi174.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-19T06:37:30.509 INFO:teuthology.orchestra.run.smithi103.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-19T06:37:30.525 INFO:teuthology.orchestra.run.smithi174.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 4.7 MB/s | 79 kB 00:00 2024-09-19T06:37:30.542 INFO:teuthology.orchestra.run.smithi174.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-19T06:37:30.584 INFO:teuthology.orchestra.run.smithi103.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 20 MB/s | 3.1 MB 00:00 2024-09-19T06:37:30.584 INFO:teuthology.orchestra.run.smithi174.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-19T06:37:30.609 INFO:teuthology.orchestra.run.smithi103.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 318 kB/s | 32 kB 00:00 2024-09-19T06:37:30.610 INFO:teuthology.orchestra.run.smithi174.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-09-19T06:37:30.635 INFO:teuthology.orchestra.run.smithi103.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.8 MB/s | 146 kB 00:00 2024-09-19T06:37:30.635 INFO:teuthology.orchestra.run.smithi174.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-19T06:37:30.660 INFO:teuthology.orchestra.run.smithi174.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-19T06:37:30.685 INFO:teuthology.orchestra.run.smithi174.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-19T06:37:30.710 INFO:teuthology.orchestra.run.smithi174.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-19T06:37:30.761 INFO:teuthology.orchestra.run.smithi174.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.5 MB/s | 2.0 MB 00:00 2024-09-19T06:37:30.786 INFO:teuthology.orchestra.run.smithi174.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 786 kB/s | 59 kB 00:00 2024-09-19T06:37:30.819 INFO:teuthology.orchestra.run.smithi174.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-19T06:37:30.844 INFO:teuthology.orchestra.run.smithi174.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 428 kB/s | 36 kB 00:00 2024-09-19T06:37:30.870 INFO:teuthology.orchestra.run.smithi174.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 888 kB/s | 74 kB 00:00 2024-09-19T06:37:30.874 INFO:teuthology.orchestra.run.smithi134.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 354 kB/s | 292 kB 00:00 2024-09-19T06:37:30.895 INFO:teuthology.orchestra.run.smithi174.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T06:37:30.910 INFO:teuthology.orchestra.run.smithi103.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 12 MB/s | 3.6 MB 00:00 2024-09-19T06:37:30.941 INFO:teuthology.orchestra.run.smithi134.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-19T06:37:30.948 INFO:teuthology.orchestra.run.smithi174.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 378 kB/s | 20 kB 00:00 2024-09-19T06:37:30.967 INFO:teuthology.orchestra.run.smithi134.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 257 kB/s | 188 kB 00:00 2024-09-19T06:37:30.979 INFO:teuthology.orchestra.run.smithi174.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-09-19T06:37:31.004 INFO:teuthology.orchestra.run.smithi174.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.4 MB/s | 191 kB 00:00 2024-09-19T06:37:31.010 INFO:teuthology.orchestra.run.smithi103.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 20 MB/s | 7.4 MB 00:00 2024-09-19T06:37:31.029 INFO:teuthology.orchestra.run.smithi174.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-09-19T06:37:31.035 INFO:teuthology.orchestra.run.smithi103.stdout:(33/156): ceph-mgr-modules-core-19.1.1-361.g2f5 1.9 MB/s | 249 kB 00:00 2024-09-19T06:37:31.061 INFO:teuthology.orchestra.run.smithi103.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 982 kB/s | 49 kB 00:00 2024-09-19T06:37:31.086 INFO:teuthology.orchestra.run.smithi103.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 327 kB/s | 16 kB 00:00 2024-09-19T06:37:31.088 INFO:teuthology.orchestra.run.smithi174.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-19T06:37:31.112 INFO:teuthology.orchestra.run.smithi103.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.5 MB/s | 285 kB 00:00 2024-09-19T06:37:31.154 INFO:teuthology.orchestra.run.smithi103.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 437 kB/s | 18 kB 00:00 2024-09-19T06:37:31.171 INFO:teuthology.orchestra.run.smithi174.stdout:(149/156): NetworkManager-1.51.0-1.el9.x86_64.r 28 MB/s | 2.3 MB 00:00 2024-09-19T06:37:31.187 INFO:teuthology.orchestra.run.smithi103.stdout:(38/156): cephadm-19.1.1-361.g2f599a46.el9.noar 7.4 MB/s | 765 kB 00:00 2024-09-19T06:37:31.238 INFO:teuthology.orchestra.run.smithi174.stdout:(150/156): NetworkManager-libnm-1.51.0-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-09-19T06:37:31.254 INFO:teuthology.orchestra.run.smithi103.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 798 kB/s | 80 kB 00:00 2024-09-19T06:37:31.281 INFO:teuthology.orchestra.run.smithi174.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 983 kB/s | 41 kB 00:00 2024-09-19T06:37:31.313 INFO:teuthology.orchestra.run.smithi103.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-19T06:37:31.331 INFO:teuthology.orchestra.run.smithi174.stdout:(152/156): NetworkManager-tui-1.51.0-1.el9.x86_ 4.9 MB/s | 249 kB 00:00 2024-09-19T06:37:31.371 INFO:teuthology.orchestra.run.smithi103.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-19T06:37:31.399 INFO:teuthology.orchestra.run.smithi174.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-09-19T06:37:31.405 INFO:teuthology.orchestra.run.smithi103.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.6 MB/s | 801 kB 00:00 2024-09-19T06:37:31.447 INFO:teuthology.orchestra.run.smithi103.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-19T06:37:31.473 INFO:teuthology.orchestra.run.smithi174.stdout:(154/156): librbd1-19.1.1-361.g2f599a46.el9.x86 7.1 MB/s | 3.2 MB 00:00 2024-09-19T06:37:31.523 INFO:teuthology.orchestra.run.smithi174.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-09-19T06:37:31.656 INFO:teuthology.orchestra.run.smithi174.stdout:(156/156): librados2-19.1.1-361.g2f599a46.el9.x 5.2 MB/s | 3.4 MB 00:00 2024-09-19T06:37:31.662 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:37:31.662 INFO:teuthology.orchestra.run.smithi174.stdout:Total 20 MB/s | 226 MB 00:11 2024-09-19T06:37:31.923 INFO:teuthology.orchestra.run.smithi103.stdout:(44/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 20 MB/s | 49 MB 00:02 2024-09-19T06:37:31.934 INFO:teuthology.orchestra.run.smithi134.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:00 2024-09-19T06:37:31.948 INFO:teuthology.orchestra.run.smithi103.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 270 kB/s | 135 kB 00:00 2024-09-19T06:37:31.981 INFO:teuthology.orchestra.run.smithi103.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 2.0 MB/s | 1.2 MB 00:00 2024-09-19T06:37:32.007 INFO:teuthology.orchestra.run.smithi103.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-19T06:37:32.017 INFO:teuthology.orchestra.run.smithi134.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.3 MB/s | 4.9 MB 00:01 2024-09-19T06:37:32.049 INFO:teuthology.orchestra.run.smithi103.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 994 kB/s | 125 kB 00:00 2024-09-19T06:37:32.092 INFO:teuthology.orchestra.run.smithi134.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-09-19T06:37:32.207 INFO:teuthology.orchestra.run.smithi103.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-19T06:37:32.241 INFO:teuthology.orchestra.run.smithi103.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 556 kB/s | 107 kB 00:00 2024-09-19T06:37:32.266 INFO:teuthology.orchestra.run.smithi103.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 370 kB/s | 22 kB 00:00 2024-09-19T06:37:32.325 INFO:teuthology.orchestra.run.smithi103.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-09-19T06:37:32.359 INFO:teuthology.orchestra.run.smithi103.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-09-19T06:37:32.392 INFO:teuthology.orchestra.run.smithi103.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 260 kB/s | 33 kB 00:00 2024-09-19T06:37:32.418 INFO:teuthology.orchestra.run.smithi103.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-09-19T06:37:32.443 INFO:teuthology.orchestra.run.smithi103.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-19T06:37:32.477 INFO:teuthology.orchestra.run.smithi103.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-19T06:37:32.502 INFO:teuthology.orchestra.run.smithi103.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-09-19T06:37:32.552 INFO:teuthology.orchestra.run.smithi103.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-09-19T06:37:32.586 INFO:teuthology.orchestra.run.smithi103.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-19T06:37:32.611 INFO:teuthology.orchestra.run.smithi103.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 417 kB/s | 24 kB 00:00 2024-09-19T06:37:32.645 INFO:teuthology.orchestra.run.smithi103.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.8 MB/s | 662 kB 00:00 2024-09-19T06:37:32.670 INFO:teuthology.orchestra.run.smithi103.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-09-19T06:37:32.704 INFO:teuthology.orchestra.run.smithi103.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-19T06:37:32.729 INFO:teuthology.orchestra.run.smithi103.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 656 kB/s | 38 kB 00:00 2024-09-19T06:37:32.734 INFO:teuthology.orchestra.run.smithi134.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 192 kB/s | 137 kB 00:00 2024-09-19T06:37:32.755 INFO:teuthology.orchestra.run.smithi103.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-19T06:37:32.851 INFO:teuthology.orchestra.run.smithi134.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 992 kB/s | 910 kB 00:00 2024-09-19T06:37:32.855 INFO:teuthology.orchestra.run.smithi103.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-19T06:37:32.888 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-09-19T06:37:32.930 INFO:teuthology.orchestra.run.smithi103.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-19T06:37:33.006 INFO:teuthology.orchestra.run.smithi103.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 16 MB/s | 4.9 MB 00:00 2024-09-19T06:37:33.031 INFO:teuthology.orchestra.run.smithi103.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.3 MB/s | 137 kB 00:00 2024-09-19T06:37:33.037 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-09-19T06:37:33.037 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-09-19T06:37:33.098 INFO:teuthology.orchestra.run.smithi103.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-19T06:37:33.140 INFO:teuthology.orchestra.run.smithi103.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-19T06:37:33.165 INFO:teuthology.orchestra.run.smithi103.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-19T06:37:33.215 INFO:teuthology.orchestra.run.smithi103.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-19T06:37:33.318 INFO:teuthology.orchestra.run.smithi134.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 4.7 MB/s | 5.8 MB 00:01 2024-09-19T06:37:33.324 INFO:teuthology.orchestra.run.smithi103.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-09-19T06:37:33.349 INFO:teuthology.orchestra.run.smithi103.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-19T06:37:33.383 INFO:teuthology.orchestra.run.smithi103.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-19T06:37:33.408 INFO:teuthology.orchestra.run.smithi103.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 436 kB/s | 26 kB 00:00 2024-09-19T06:37:33.484 INFO:teuthology.orchestra.run.smithi103.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-19T06:37:33.509 INFO:teuthology.orchestra.run.smithi103.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 152 kB/s | 19 kB 00:00 2024-09-19T06:37:33.534 INFO:teuthology.orchestra.run.smithi103.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-19T06:37:33.559 INFO:teuthology.orchestra.run.smithi103.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 460 kB/s | 35 kB 00:00 2024-09-19T06:37:33.593 INFO:teuthology.orchestra.run.smithi103.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-19T06:37:33.618 INFO:teuthology.orchestra.run.smithi103.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-19T06:37:33.626 INFO:teuthology.orchestra.run.smithi134.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 113 kB/s | 87 kB 00:00 2024-09-19T06:37:33.652 INFO:teuthology.orchestra.run.smithi103.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-19T06:37:33.668 INFO:teuthology.orchestra.run.smithi134.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-09-19T06:37:33.677 INFO:teuthology.orchestra.run.smithi103.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 719 kB/s | 42 kB 00:00 2024-09-19T06:37:33.720 INFO:teuthology.orchestra.run.smithi103.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-09-19T06:37:33.745 INFO:teuthology.orchestra.run.smithi103.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-19T06:37:33.812 INFO:teuthology.orchestra.run.smithi103.stdout:(89/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-19T06:37:33.837 INFO:teuthology.orchestra.run.smithi103.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 554 kB/s | 51 kB 00:00 2024-09-19T06:37:33.863 INFO:teuthology.orchestra.run.smithi103.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 279 kB 00:00 2024-09-19T06:37:33.888 INFO:teuthology.orchestra.run.smithi103.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-09-19T06:37:33.913 INFO:teuthology.orchestra.run.smithi103.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-09-19T06:37:33.955 INFO:teuthology.orchestra.run.smithi103.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 661 kB/s | 44 kB 00:00 2024-09-19T06:37:33.989 INFO:teuthology.orchestra.run.smithi103.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-19T06:37:34.014 INFO:teuthology.orchestra.run.smithi103.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-09-19T06:37:34.039 INFO:teuthology.orchestra.run.smithi103.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-19T06:37:34.076 INFO:teuthology.orchestra.run.smithi103.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-19T06:37:34.098 INFO:teuthology.orchestra.run.smithi103.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 985 kB/s | 58 kB 00:00 2024-09-19T06:37:34.132 INFO:teuthology.orchestra.run.smithi103.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-19T06:37:34.157 INFO:teuthology.orchestra.run.smithi103.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-19T06:37:34.199 INFO:teuthology.orchestra.run.smithi103.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 670 kB/s | 45 kB 00:00 2024-09-19T06:37:34.244 INFO:teuthology.orchestra.run.smithi134.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 99 kB/s | 61 kB 00:00 2024-09-19T06:37:34.274 INFO:teuthology.orchestra.run.smithi103.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-09-19T06:37:34.450 INFO:teuthology.orchestra.run.smithi103.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-19T06:37:34.492 INFO:teuthology.orchestra.run.smithi103.stdout:(105/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-09-19T06:37:34.494 INFO:teuthology.orchestra.run.smithi134.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.1 MB/s | 6.0 MB 00:01 2024-09-19T06:37:34.527 INFO:teuthology.orchestra.run.smithi134.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 285 kB/s | 245 kB 00:00 2024-09-19T06:37:34.534 INFO:teuthology.orchestra.run.smithi103.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-09-19T06:37:34.559 INFO:teuthology.orchestra.run.smithi103.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.8 MB/s | 308 kB 00:00 2024-09-19T06:37:34.584 INFO:teuthology.orchestra.run.smithi103.stdout:(108/156): grpc-data-1.46.7-10.el9.noarch.rpm 215 kB/s | 19 kB 00:00 2024-09-19T06:37:34.610 INFO:teuthology.orchestra.run.smithi103.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 516 kB/s | 26 kB 00:00 2024-09-19T06:37:34.635 INFO:teuthology.orchestra.run.smithi103.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 964 kB/s | 49 kB 00:00 2024-09-19T06:37:34.660 INFO:teuthology.orchestra.run.smithi103.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-19T06:37:34.697 INFO:teuthology.orchestra.run.smithi103.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-19T06:37:34.753 INFO:teuthology.orchestra.run.smithi103.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-19T06:37:34.778 INFO:teuthology.orchestra.run.smithi103.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T06:37:34.803 INFO:teuthology.orchestra.run.smithi103.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T06:37:34.828 INFO:teuthology.orchestra.run.smithi103.stdout:(116/156): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-09-19T06:37:34.833 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-09-19T06:37:34.833 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-09-19T06:37:34.912 INFO:teuthology.orchestra.run.smithi103.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 130 kB/s | 14 kB 00:00 2024-09-19T06:37:34.938 INFO:teuthology.orchestra.run.smithi103.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-09-19T06:37:34.994 INFO:teuthology.orchestra.run.smithi103.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.3 MB/s | 358 kB 00:00 2024-09-19T06:37:35.019 INFO:teuthology.orchestra.run.smithi134.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 39 kB/s | 19 kB 00:00 2024-09-19T06:37:35.021 INFO:teuthology.orchestra.run.smithi103.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.6 MB/s | 303 kB 00:00 2024-09-19T06:37:35.044 INFO:teuthology.orchestra.run.smithi134.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 47 kB/s | 26 kB 00:00 2024-09-19T06:37:35.047 INFO:teuthology.orchestra.run.smithi103.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.1 MB/s | 220 kB 00:00 2024-09-19T06:37:35.069 INFO:teuthology.orchestra.run.smithi134.stdout:(80/156): python3-jinja2-2.11.3-6.el9.noarch.rp 301 kB/s | 248 kB 00:00 2024-09-19T06:37:35.072 INFO:teuthology.orchestra.run.smithi103.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-19T06:37:35.089 INFO:teuthology.orchestra.run.smithi103.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 646 kB/s | 11 kB 00:00 2024-09-19T06:37:35.106 INFO:teuthology.orchestra.run.smithi103.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-19T06:37:35.123 INFO:teuthology.orchestra.run.smithi103.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-19T06:37:35.140 INFO:teuthology.orchestra.run.smithi103.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.2 MB/s | 19 kB 00:00 2024-09-19T06:37:35.165 INFO:teuthology.orchestra.run.smithi103.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 788 kB/s | 20 kB 00:00 2024-09-19T06:37:35.241 INFO:teuthology.orchestra.run.smithi103.stdout:(128/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-09-19T06:37:35.266 INFO:teuthology.orchestra.run.smithi103.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T06:37:35.291 INFO:teuthology.orchestra.run.smithi103.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T06:37:35.342 INFO:teuthology.orchestra.run.smithi103.stdout:(131/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.4 MB/s | 2.0 MB 00:00 2024-09-19T06:37:35.367 INFO:teuthology.orchestra.run.smithi103.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 768 kB/s | 58 kB 00:00 2024-09-19T06:37:35.392 INFO:teuthology.orchestra.run.smithi103.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-19T06:37:35.434 INFO:teuthology.orchestra.run.smithi103.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 3.8 MB/s | 1.0 MB 00:00 2024-09-19T06:37:35.460 INFO:teuthology.orchestra.run.smithi103.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-09-19T06:37:35.485 INFO:teuthology.orchestra.run.smithi103.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 565 kB/s | 28 kB 00:00 2024-09-19T06:37:35.510 INFO:teuthology.orchestra.run.smithi103.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-19T06:37:35.544 INFO:teuthology.orchestra.run.smithi103.stdout:(138/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 595 kB/s | 90 kB 00:00 2024-09-19T06:37:35.569 INFO:teuthology.orchestra.run.smithi103.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-19T06:37:35.595 INFO:teuthology.orchestra.run.smithi134.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 63 kB/s | 35 kB 00:00 2024-09-19T06:37:35.611 INFO:teuthology.orchestra.run.smithi103.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 533 kB/s | 36 kB 00:00 2024-09-19T06:37:35.637 INFO:teuthology.orchestra.run.smithi103.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-19T06:37:35.662 INFO:teuthology.orchestra.run.smithi103.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-09-19T06:37:35.687 INFO:teuthology.orchestra.run.smithi103.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T06:37:35.713 INFO:teuthology.orchestra.run.smithi103.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-09-19T06:37:35.737 INFO:teuthology.orchestra.run.smithi134.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 240 kB/s | 172 kB 00:00 2024-09-19T06:37:35.738 INFO:teuthology.orchestra.run.smithi103.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-19T06:37:35.763 INFO:teuthology.orchestra.run.smithi103.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T06:37:35.797 INFO:teuthology.orchestra.run.smithi103.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-19T06:37:35.847 INFO:teuthology.orchestra.run.smithi103.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-19T06:37:36.006 INFO:teuthology.orchestra.run.smithi103.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 14 MB/s | 3.4 MB 00:00 2024-09-19T06:37:36.181 INFO:teuthology.orchestra.run.smithi103.stdout:(150/156): librbd1-19.1.1-361.g2f599a46.el9.x86 8.2 MB/s | 3.2 MB 00:00 2024-09-19T06:37:36.281 INFO:teuthology.orchestra.run.smithi103.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 410 kB/s | 41 kB 00:00 2024-09-19T06:37:36.304 INFO:teuthology.orchestra.run.smithi134.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-09-19T06:37:36.382 INFO:teuthology.orchestra.run.smithi103.stdout:(152/156): NetworkManager-libnm-1.51.0-1.el9.x8 4.8 MB/s | 1.8 MB 00:00 2024-09-19T06:37:36.412 INFO:teuthology.orchestra.run.smithi103.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 1.9 MB/s | 249 kB 00:00 2024-09-19T06:37:36.538 INFO:teuthology.orchestra.run.smithi103.stdout:(154/156): NetworkManager-1.51.0-1.el9.x86_64.r 3.4 MB/s | 2.3 MB 00:00 2024-09-19T06:37:36.546 INFO:teuthology.orchestra.run.smithi134.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 264 kB/s | 250 kB 00:00 2024-09-19T06:37:36.579 INFO:teuthology.orchestra.run.smithi134.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 264 kB/s | 222 kB 00:00 2024-09-19T06:37:36.813 INFO:teuthology.orchestra.run.smithi103.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 3.1 MB/s | 1.4 MB 00:00 2024-09-19T06:37:36.980 INFO:teuthology.orchestra.run.smithi103.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 4.2 MB/s | 2.4 MB 00:00 2024-09-19T06:37:36.986 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:37:36.986 INFO:teuthology.orchestra.run.smithi103.stdout:Total 22 MB/s | 226 MB 00:10 2024-09-19T06:37:37.013 INFO:teuthology.orchestra.run.smithi134.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 109 kB/s | 77 kB 00:00 2024-09-19T06:37:37.130 INFO:teuthology.orchestra.run.smithi134.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 73 kB/s | 42 kB 00:00 2024-09-19T06:37:37.347 INFO:teuthology.orchestra.run.smithi134.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 351 kB/s | 269 kB 00:00 2024-09-19T06:37:37.731 INFO:teuthology.orchestra.run.smithi134.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 221 kB/s | 159 kB 00:00 2024-09-19T06:37:37.898 INFO:teuthology.orchestra.run.smithi134.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 363 kB/s | 279 kB 00:00 2024-09-19T06:37:37.964 INFO:teuthology.orchestra.run.smithi134.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 84 kB/s | 51 kB 00:00 2024-09-19T06:37:38.213 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-19T06:37:38.365 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-19T06:37:38.365 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-19T06:37:38.365 INFO:teuthology.orchestra.run.smithi134.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 85 kB/s | 54 kB 00:00 2024-09-19T06:37:38.582 INFO:teuthology.orchestra.run.smithi134.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-09-19T06:37:38.982 INFO:teuthology.orchestra.run.smithi134.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 72 kB/s | 44 kB 00:00 2024-09-19T06:37:39.315 INFO:teuthology.orchestra.run.smithi134.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-19T06:37:39.357 INFO:teuthology.orchestra.run.smithi134.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 285 kB/s | 221 kB 00:00 2024-09-19T06:37:39.691 INFO:teuthology.orchestra.run.smithi134.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 51 kB/s | 36 kB 00:00 2024-09-19T06:37:39.908 INFO:teuthology.orchestra.run.smithi134.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 81 kB/s | 47 kB 00:00 2024-09-19T06:37:40.141 INFO:teuthology.orchestra.run.smithi134.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 388 kB/s | 304 kB 00:00 2024-09-19T06:37:40.300 INFO:teuthology.orchestra.run.smithi134.stdout:(100/156): uuid-1.6.2-55.el9.x86_64.rpm 95 kB/s | 58 kB 00:00 2024-09-19T06:37:40.594 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-19T06:37:40.594 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-19T06:37:40.625 INFO:teuthology.orchestra.run.smithi134.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 135 kB/s | 96 kB 00:00 2024-09-19T06:37:40.709 INFO:teuthology.orchestra.run.smithi134.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-09-19T06:37:40.776 INFO:teuthology.orchestra.run.smithi134.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 102 kB/s | 64 kB 00:00 2024-09-19T06:37:40.834 INFO:teuthology.orchestra.run.smithi134.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.3 MB/s | 548 kB 00:00 2024-09-19T06:37:40.868 INFO:teuthology.orchestra.run.smithi134.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-19T06:37:40.893 INFO:teuthology.orchestra.run.smithi134.stdout:(106/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 76 kB/s | 45 kB 00:00 2024-09-19T06:37:40.918 INFO:teuthology.orchestra.run.smithi134.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 395 kB/s | 19 kB 00:00 2024-09-19T06:37:40.951 INFO:teuthology.orchestra.run.smithi134.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T06:37:40.985 INFO:teuthology.orchestra.run.smithi134.stdout:(109/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-09-19T06:37:41.010 INFO:teuthology.orchestra.run.smithi134.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 858 kB/s | 49 kB 00:00 2024-09-19T06:37:41.044 INFO:teuthology.orchestra.run.smithi134.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.8 MB/s | 151 kB 00:00 2024-09-19T06:37:41.078 INFO:teuthology.orchestra.run.smithi134.stdout:(112/156): libunwind-1.6.2-1.el9.x86_64.rpm 732 kB/s | 67 kB 00:00 2024-09-19T06:37:41.128 INFO:teuthology.orchestra.run.smithi134.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 10 MB/s | 839 kB 00:00 2024-09-19T06:37:41.141 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-09-19T06:37:41.153 INFO:teuthology.orchestra.run.smithi134.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T06:37:41.187 INFO:teuthology.orchestra.run.smithi134.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-09-19T06:37:41.212 INFO:teuthology.orchestra.run.smithi134.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-09-19T06:37:41.237 INFO:teuthology.orchestra.run.smithi134.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-19T06:37:41.260 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T06:37:41.334 INFO:teuthology.orchestra.run.smithi134.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-09-19T06:37:41.391 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T06:37:41.496 INFO:teuthology.orchestra.run.smithi134.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-09-19T06:37:41.530 INFO:teuthology.orchestra.run.smithi134.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.5 MB/s | 303 kB 00:00 2024-09-19T06:37:41.537 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T06:37:41.556 INFO:teuthology.orchestra.run.smithi134.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-19T06:37:41.578 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T06:37:41.589 INFO:teuthology.orchestra.run.smithi134.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-19T06:37:41.603 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T06:37:41.614 INFO:teuthology.orchestra.run.smithi134.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-19T06:37:41.640 INFO:teuthology.orchestra.run.smithi134.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 711 kB/s | 18 kB 00:00 2024-09-19T06:37:41.655 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T06:37:41.665 INFO:teuthology.orchestra.run.smithi134.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 935 kB/s | 23 kB 00:00 2024-09-19T06:37:41.682 INFO:teuthology.orchestra.run.smithi134.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-19T06:37:41.740 INFO:teuthology.orchestra.run.smithi134.stdout:(127/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.2 MB/s | 4.4 MB 00:00 2024-09-19T06:37:41.757 INFO:teuthology.orchestra.run.smithi134.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-09-19T06:37:41.783 INFO:teuthology.orchestra.run.smithi134.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 2.6 MB/s | 46 kB 00:00 2024-09-19T06:37:41.808 INFO:teuthology.orchestra.run.smithi134.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.4 MB/s | 79 kB 00:00 2024-09-19T06:37:41.858 INFO:teuthology.orchestra.run.smithi134.stdout:(131/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.2 MB/s | 2.0 MB 00:00 2024-09-19T06:37:41.883 INFO:teuthology.orchestra.run.smithi134.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 767 kB/s | 58 kB 00:00 2024-09-19T06:37:41.909 INFO:teuthology.orchestra.run.smithi134.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-09-19T06:37:41.934 INFO:teuthology.orchestra.run.smithi134.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-19T06:37:41.968 INFO:teuthology.orchestra.run.smithi134.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-19T06:37:41.984 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T06:37:41.993 INFO:teuthology.orchestra.run.smithi134.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 483 kB/s | 28 kB 00:00 2024-09-19T06:37:42.018 INFO:teuthology.orchestra.run.smithi134.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-19T06:37:42.043 INFO:teuthology.orchestra.run.smithi134.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T06:37:42.077 INFO:teuthology.orchestra.run.smithi134.stdout:(139/156): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-09-19T06:37:42.095 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:37:42.102 INFO:teuthology.orchestra.run.smithi134.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-09-19T06:37:42.127 INFO:teuthology.orchestra.run.smithi134.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 427 kB/s | 36 kB 00:00 2024-09-19T06:37:42.153 INFO:teuthology.orchestra.run.smithi134.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 985 kB/s | 74 kB 00:00 2024-09-19T06:37:42.186 INFO:teuthology.orchestra.run.smithi134.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-19T06:37:42.211 INFO:teuthology.orchestra.run.smithi134.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-19T06:37:42.217 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:37:42.237 INFO:teuthology.orchestra.run.smithi134.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-19T06:37:42.254 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:37:42.262 INFO:teuthology.orchestra.run.smithi134.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T06:37:42.340 INFO:teuthology.orchestra.run.smithi134.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.2 MB/s | 427 kB 00:00 2024-09-19T06:37:42.346 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:37:42.383 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T06:37:42.388 INFO:teuthology.orchestra.run.smithi134.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-19T06:37:42.531 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T06:37:42.580 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T06:37:42.609 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:37:42.613 INFO:teuthology.orchestra.run.smithi134.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 9.8 MB/s | 3.4 MB 00:00 2024-09-19T06:37:42.663 INFO:teuthology.orchestra.run.smithi134.stdout:(150/156): librbd1-19.1.1-361.g2f599a46.el9.x86 9.8 MB/s | 3.2 MB 00:00 2024-09-19T06:37:42.704 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:37:42.726 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T06:37:42.772 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T06:37:42.780 INFO:teuthology.orchestra.run.smithi134.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 350 kB/s | 41 kB 00:00 2024-09-19T06:37:42.794 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T06:37:42.814 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T06:37:42.855 INFO:teuthology.orchestra.run.smithi134.stdout:(152/156): NetworkManager-1.51.0-1.el9.x86_64.r 5.0 MB/s | 2.3 MB 00:00 2024-09-19T06:37:42.956 INFO:teuthology.orchestra.run.smithi134.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 1.4 MB/s | 249 kB 00:00 2024-09-19T06:37:43.003 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T06:37:43.048 INFO:teuthology.orchestra.run.smithi134.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 7.1 MB/s | 1.4 MB 00:00 2024-09-19T06:37:43.134 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T06:37:43.206 INFO:teuthology.orchestra.run.smithi134.stdout:(155/156): NetworkManager-libnm-1.51.0-1.el9.x8 3.0 MB/s | 1.8 MB 00:00 2024-09-19T06:37:43.266 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T06:37:43.366 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:37:43.423 INFO:teuthology.orchestra.run.smithi134.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 5.1 MB/s | 2.4 MB 00:00 2024-09-19T06:37:43.429 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:37:43.429 INFO:teuthology.orchestra.run.smithi134.stdout:Total 9.5 MB/s | 226 MB 00:23 2024-09-19T06:37:43.443 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:37:43.666 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:37:43.783 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T06:37:43.878 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T06:37:43.915 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T06:37:43.958 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T06:37:44.045 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T06:37:44.091 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T06:37:44.123 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T06:37:44.168 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T06:37:44.200 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T06:37:44.236 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T06:37:44.305 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T06:37:44.358 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T06:37:44.388 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T06:37:44.432 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T06:37:44.509 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T06:37:44.555 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T06:37:44.597 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T06:37:44.626 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T06:37:44.667 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-19T06:37:44.694 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T06:37:44.741 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T06:37:44.815 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T06:37:44.819 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-19T06:37:44.819 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-19T06:37:44.956 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T06:37:45.010 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T06:37:45.537 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T06:37:45.614 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T06:37:45.651 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T06:37:45.694 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T06:37:45.732 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T06:37:45.775 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T06:37:45.931 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T06:37:45.985 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T06:37:46.040 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T06:37:46.112 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T06:37:46.144 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T06:37:46.180 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T06:37:46.227 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T06:37:46.263 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T06:37:46.306 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T06:37:46.348 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T06:37:46.490 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T06:37:46.532 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T06:37:46.573 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T06:37:46.633 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-19T06:37:46.633 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-19T06:37:46.677 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T06:37:46.708 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T06:37:46.748 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T06:37:46.773 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T06:37:46.922 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-19T06:37:46.982 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T06:37:47.026 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T06:37:47.058 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T06:37:47.065 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T06:37:47.118 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T06:37:47.151 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T06:37:47.183 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T06:37:47.185 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T06:37:47.222 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T06:37:47.235 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T06:37:47.250 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T06:37:47.284 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T06:37:47.323 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T06:37:47.355 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T06:37:47.385 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T06:37:47.411 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T06:37:47.444 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T06:37:47.485 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T06:37:47.520 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T06:37:47.606 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T06:37:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T06:37:47.649 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T06:37:47.681 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T06:37:47.761 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:37:47.785 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T06:37:47.863 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T06:37:47.883 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:37:47.921 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:37:48.012 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:37:48.042 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T06:37:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T06:37:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T06:37:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:37:48.430 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:37:48.468 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T06:37:48.531 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T06:37:48.561 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T06:37:48.585 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T06:37:48.586 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T06:37:48.637 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T06:37:48.671 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T06:37:48.768 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T06:37:48.788 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T06:37:48.834 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T06:37:48.888 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T06:37:48.968 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T06:37:48.977 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T06:37:49.015 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T06:37:49.096 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T06:37:49.199 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T06:37:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:37:49.229 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T06:37:49.295 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:37:49.307 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T06:37:49.540 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:37:49.655 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T06:37:49.757 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T06:37:49.792 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T06:37:49.827 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T06:37:49.893 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T06:37:49.950 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T06:37:49.953 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T06:37:49.994 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T06:37:50.038 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T06:37:50.051 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T06:37:50.070 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T06:37:50.106 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T06:37:50.175 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T06:37:50.246 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T06:37:50.277 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T06:37:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T06:37:50.396 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T06:37:50.430 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T06:37:50.474 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T06:37:50.502 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T06:37:50.573 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T06:37:50.625 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T06:37:50.701 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T06:37:50.834 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T06:37:50.887 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T06:37:51.269 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T06:37:51.366 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T06:37:51.393 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T06:37:51.420 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T06:37:51.426 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T06:37:51.508 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T06:37:51.536 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T06:37:51.580 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T06:37:51.617 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T06:37:51.661 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T06:37:51.737 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T06:37:51.781 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T06:37:51.816 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T06:37:51.871 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T06:37:51.871 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T06:37:51.893 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T06:37:51.926 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T06:37:52.007 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T06:37:52.046 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T06:37:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T06:37:52.083 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T06:37:52.130 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T06:37:52.166 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T06:37:52.209 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T06:37:52.251 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T06:37:52.389 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T06:37:52.445 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T06:37:52.486 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T06:37:52.529 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T06:37:52.567 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T06:37:52.590 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T06:37:52.629 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T06:37:52.637 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T06:37:52.660 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T06:37:52.669 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T06:37:52.702 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T06:37:52.727 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T06:37:52.962 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T06:37:53.063 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-19T06:37:53.118 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T06:37:53.151 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T06:37:53.155 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T06:37:53.200 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T06:37:53.240 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T06:37:53.281 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T06:37:53.282 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T06:37:53.313 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T06:37:53.335 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T06:37:53.351 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T06:37:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T06:37:53.386 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T06:37:53.419 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T06:37:53.428 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T06:37:53.462 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T06:37:53.478 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T06:37:53.512 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T06:37:53.600 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T06:37:53.651 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T06:37:53.683 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T06:37:53.785 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T06:37:53.787 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T06:37:53.842 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T06:37:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:37:54.001 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:37:54.029 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:37:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:37:54.159 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T06:37:54.307 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T06:37:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T06:37:54.392 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:37:54.479 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:37:54.501 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T06:37:54.563 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T06:37:54.577 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T06:37:54.585 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T06:37:54.612 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T06:37:54.632 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T06:37:54.665 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T06:37:54.766 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T06:37:54.796 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T06:37:54.829 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T06:37:54.883 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T06:37:54.933 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T06:37:54.964 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:37:54.978 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T06:37:54.997 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:37:55.009 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T06:37:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T06:37:55.159 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:37:55.196 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T06:37:55.224 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T06:37:55.259 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:37:55.311 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T06:37:55.494 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:37:55.546 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:37:55.592 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T06:37:55.617 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T06:37:55.677 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T06:37:55.677 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T06:37:55.677 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T06:37:55.678 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:37:55.710 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T06:37:55.720 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T06:37:55.765 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T06:37:55.800 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T06:37:55.873 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T06:37:55.917 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T06:37:55.949 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T06:37:55.996 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T06:37:56.028 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T06:37:56.055 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T06:37:56.125 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T06:37:56.162 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T06:37:56.186 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T06:37:56.225 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T06:37:56.269 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T06:37:56.282 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T06:37:56.357 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T06:37:56.387 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T06:37:56.430 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T06:37:56.460 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T06:37:56.529 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T06:37:56.574 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T06:37:56.665 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T06:37:56.806 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T06:37:56.859 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T06:37:57.387 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T06:37:57.472 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T06:37:57.509 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T06:37:57.543 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T06:37:57.581 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T06:37:57.595 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T06:37:57.623 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T06:37:57.688 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T06:37:57.716 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T06:37:57.749 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T06:37:57.779 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T06:37:57.834 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T06:37:57.889 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T06:37:57.977 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T06:37:58.009 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T06:37:58.045 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T06:37:58.059 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T06:37:58.092 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T06:37:58.104 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T06:37:58.128 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T06:37:58.170 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T06:37:58.194 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T06:37:58.213 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T06:37:58.225 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T06:37:58.262 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T06:37:58.357 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T06:37:58.406 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T06:37:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T06:37:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T06:37:58.581 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T06:37:58.621 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T06:37:58.646 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T06:37:58.671 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T06:37:58.741 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T06:37:58.745 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T06:37:58.774 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T06:37:58.828 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T06:37:58.859 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T06:37:58.873 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T06:37:58.905 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T06:37:58.933 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T06:37:58.976 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T06:37:59.014 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T06:37:59.046 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T06:37:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T06:37:59.110 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T06:37:59.144 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T06:37:59.185 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T06:37:59.228 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T06:37:59.314 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T06:37:59.366 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T06:37:59.406 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T06:37:59.508 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T06:37:59.565 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T06:38:00.281 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T06:38:00.345 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T06:38:00.387 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T06:38:00.485 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T06:38:00.534 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T06:38:00.587 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T06:38:00.682 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T06:38:00.715 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T06:38:00.899 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T06:38:00.940 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T06:38:01.023 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T06:38:01.232 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:38:01.264 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:38:01.667 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T06:38:01.777 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T06:38:01.823 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:38:01.868 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T06:38:01.964 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T06:38:01.964 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T06:38:01.964 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T06:38:01.964 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:38:02.003 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T06:38:02.995 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T06:38:03.065 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T06:38:03.093 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T06:38:03.126 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T06:38:03.437 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T06:38:03.481 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T06:38:03.563 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T06:38:03.594 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T06:38:03.629 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T06:38:04.110 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T06:38:04.150 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T06:38:04.221 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T06:38:04.260 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T06:38:06.528 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:38:06.565 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:38:07.106 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:38:07.151 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T06:38:07.226 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T06:38:07.226 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T06:38:07.226 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T06:38:07.226 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:38:07.269 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T06:38:07.588 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T06:38:07.588 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /sys 2024-09-19T06:38:07.588 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /proc 2024-09-19T06:38:07.588 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /mnt 2024-09-19T06:38:07.588 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /var/tmp 2024-09-19T06:38:07.588 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /home 2024-09-19T06:38:07.588 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /root 2024-09-19T06:38:07.588 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /tmp 2024-09-19T06:38:07.588 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:38:07.761 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T06:38:07.777 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T06:38:07.778 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:07.778 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T06:38:07.778 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:38:09.297 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T06:38:09.325 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T06:38:09.325 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:09.325 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T06:38:09.325 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T06:38:09.325 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T06:38:09.325 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:38:09.548 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T06:38:09.571 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T06:38:09.571 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:09.571 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T06:38:09.572 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T06:38:09.572 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T06:38:09.572 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:38:10.015 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T06:38:10.039 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T06:38:10.039 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:10.039 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T06:38:10.039 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T06:38:10.039 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T06:38:10.039 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:38:10.145 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T06:38:10.202 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T06:38:10.266 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T06:38:10.461 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T06:38:10.507 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T06:38:10.585 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T06:38:10.673 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T06:38:10.812 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T06:38:10.857 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T06:38:10.889 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T06:38:10.917 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T06:38:11.338 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T06:38:11.365 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T06:38:11.396 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T06:38:11.459 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T06:38:11.488 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T06:38:12.536 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T06:38:12.564 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T06:38:13.528 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T06:38:13.565 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T06:38:13.666 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T06:38:13.827 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T06:38:13.850 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T06:38:13.874 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T06:38:13.874 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:13.875 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T06:38:13.875 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T06:38:13.875 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T06:38:13.875 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:38:13.888 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T06:38:13.889 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2024-09-19T06:38:13.889 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2024-09-19T06:38:13.889 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2024-09-19T06:38:13.889 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2024-09-19T06:38:13.889 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2024-09-19T06:38:13.889 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /root 2024-09-19T06:38:13.889 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2024-09-19T06:38:13.889 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:38:13.926 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T06:38:13.937 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T06:38:14.035 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T06:38:14.069 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T06:38:14.084 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T06:38:14.085 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:14.085 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T06:38:14.085 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:38:14.090 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T06:38:15.018 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T06:38:15.045 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T06:38:15.069 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T06:38:15.069 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:15.069 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T06:38:15.069 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T06:38:15.069 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T06:38:15.069 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:38:15.112 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T06:38:15.135 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T06:38:15.135 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:15.135 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T06:38:15.135 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:38:15.179 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T06:38:15.203 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T06:38:15.203 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:15.203 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T06:38:15.203 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T06:38:15.203 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T06:38:15.203 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:38:15.505 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T06:38:15.528 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T06:38:15.529 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:15.529 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T06:38:15.529 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T06:38:15.529 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T06:38:15.529 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:38:15.548 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T06:38:15.574 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T06:38:15.574 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:15.574 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T06:38:15.574 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T06:38:15.574 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T06:38:15.574 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:38:15.798 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T06:38:15.822 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T06:38:15.822 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:15.822 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T06:38:15.823 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T06:38:15.823 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T06:38:15.823 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:38:16.257 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T06:38:16.281 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T06:38:16.281 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:16.281 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T06:38:16.281 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T06:38:16.282 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T06:38:16.282 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:38:16.396 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T06:38:16.461 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T06:38:16.525 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T06:38:16.711 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T06:38:16.767 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T06:38:16.854 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T06:38:16.951 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T06:38:17.092 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T06:38:17.170 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T06:38:17.208 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T06:38:17.236 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T06:38:17.657 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T06:38:17.685 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T06:38:17.716 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T06:38:17.779 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T06:38:17.817 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T06:38:18.866 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T06:38:18.902 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T06:38:19.197 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T06:38:19.198 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2024-09-19T06:38:19.198 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2024-09-19T06:38:19.198 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2024-09-19T06:38:19.198 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2024-09-19T06:38:19.198 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2024-09-19T06:38:19.198 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /root 2024-09-19T06:38:19.198 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2024-09-19T06:38:19.198 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:38:19.369 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T06:38:19.386 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T06:38:19.387 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:19.387 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T06:38:19.387 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:38:19.861 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T06:38:19.894 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T06:38:19.997 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T06:38:20.083 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T06:38:20.125 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T06:38:20.159 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T06:38:20.165 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T06:38:20.188 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T06:38:20.197 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T06:38:20.212 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T06:38:20.212 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:20.212 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T06:38:20.213 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T06:38:20.213 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T06:38:20.213 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:38:20.225 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T06:38:20.264 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T06:38:20.275 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T06:38:20.374 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T06:38:20.409 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T06:38:20.438 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T06:38:20.454 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T06:38:20.484 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T06:38:20.514 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T06:38:20.560 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T06:38:20.897 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T06:38:20.924 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T06:38:20.924 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:20.925 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T06:38:20.925 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T06:38:20.925 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T06:38:20.925 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:38:21.134 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:38:21.156 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T06:38:21.166 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:38:21.180 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T06:38:21.180 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:21.180 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T06:38:21.180 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T06:38:21.180 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T06:38:21.180 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:38:21.187 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:38:21.365 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T06:38:21.400 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T06:38:21.424 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T06:38:21.424 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:21.424 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T06:38:21.424 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T06:38:21.424 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T06:38:21.424 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:38:21.467 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T06:38:21.490 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T06:38:21.490 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:21.490 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T06:38:21.490 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:38:21.535 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T06:38:21.558 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T06:38:21.558 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:21.558 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T06:38:21.558 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T06:38:21.558 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T06:38:21.558 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:38:21.615 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T06:38:21.639 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T06:38:21.639 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:21.639 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T06:38:21.639 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T06:38:21.639 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T06:38:21.639 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:38:21.745 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T06:38:21.793 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T06:38:21.848 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T06:38:21.861 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T06:38:21.884 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T06:38:21.884 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:21.884 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T06:38:21.884 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T06:38:21.885 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T06:38:21.885 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:38:22.029 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T06:38:22.049 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T06:38:22.082 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T06:38:22.112 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T06:38:22.150 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T06:38:22.152 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T06:38:22.234 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T06:38:22.239 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T06:38:22.264 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T06:38:22.290 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T06:38:22.316 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T06:38:22.316 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T06:38:22.378 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T06:38:22.400 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T06:38:22.400 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T06:38:22.424 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T06:38:22.464 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T06:38:22.492 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T06:38:22.492 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T06:38:22.517 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T06:38:22.522 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:38:22.522 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:38:22.702 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:38:22.702 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T06:38:22.786 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T06:38:22.850 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:38:22.937 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T06:38:22.965 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T06:38:23.005 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T06:38:23.075 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T06:38:23.113 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T06:38:24.165 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T06:38:24.198 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T06:38:25.163 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T06:38:25.190 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T06:38:25.293 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T06:38:25.461 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T06:38:25.500 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T06:38:25.525 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T06:38:25.525 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:25.525 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T06:38:25.525 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T06:38:25.525 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T06:38:25.525 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:38:25.585 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T06:38:25.597 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T06:38:25.704 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T06:38:25.758 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T06:38:26.465 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T06:38:26.506 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T06:38:26.555 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T06:38:26.587 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T06:38:26.623 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T06:38:26.683 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T06:38:26.712 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T06:38:26.735 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T06:38:26.735 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:26.735 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T06:38:26.736 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T06:38:26.736 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T06:38:26.736 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:38:26.779 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T06:38:26.803 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T06:38:26.803 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:26.803 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T06:38:26.803 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:38:26.829 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T06:38:26.855 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T06:38:26.870 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T06:38:26.878 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T06:38:26.878 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:26.878 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T06:38:26.878 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T06:38:26.878 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T06:38:26.878 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:38:26.901 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T06:38:26.940 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T06:38:26.985 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T06:38:27.181 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T06:38:27.204 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T06:38:27.205 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:38:27.205 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T06:38:27.205 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T06:38:27.205 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T06:38:27.205 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:38:27.588 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:38:27.626 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:38:27.647 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:38:28.494 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:38:28.494 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T06:38:28.494 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T06:38:28.494 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T06:38:28.494 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T06:38:28.494 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T06:38:28.495 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T06:38:28.495 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T06:38:28.495 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:38:28.495 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:38:28.495 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T06:38:28.495 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T06:38:28.495 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T06:38:28.495 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:38:28.495 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T06:38:28.495 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T06:38:28.495 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T06:38:28.495 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T06:38:28.496 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T06:38:28.496 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T06:38:28.496 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T06:38:28.496 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T06:38:28.496 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T06:38:28.496 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T06:38:28.496 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T06:38:28.496 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T06:38:28.497 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T06:38:28.497 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T06:38:28.497 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T06:38:28.497 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T06:38:28.497 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T06:38:28.497 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T06:38:28.497 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T06:38:28.497 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T06:38:28.497 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T06:38:28.497 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T06:38:28.497 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T06:38:28.498 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T06:38:28.498 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T06:38:28.498 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T06:38:28.498 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T06:38:28.498 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T06:38:28.498 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T06:38:28.498 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T06:38:28.498 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T06:38:28.498 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T06:38:28.499 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T06:38:28.499 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T06:38:28.499 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T06:38:28.499 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T06:38:28.499 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T06:38:28.499 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T06:38:28.499 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T06:38:28.499 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T06:38:28.500 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T06:38:28.500 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T06:38:28.500 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T06:38:28.500 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T06:38:28.500 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T06:38:28.500 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T06:38:28.500 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T06:38:28.500 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T06:38:28.500 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T06:38:28.501 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T06:38:28.501 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T06:38:28.501 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T06:38:28.501 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T06:38:28.501 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T06:38:28.501 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T06:38:28.501 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T06:38:28.501 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T06:38:28.501 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T06:38:28.501 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T06:38:28.502 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T06:38:28.502 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T06:38:28.502 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T06:38:28.502 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T06:38:28.502 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T06:38:28.502 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T06:38:28.502 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T06:38:28.502 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T06:38:28.502 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T06:38:28.502 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T06:38:28.502 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T06:38:28.502 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T06:38:28.503 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T06:38:28.503 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T06:38:28.503 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T06:38:28.503 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T06:38:28.503 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T06:38:28.503 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T06:38:28.503 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T06:38:28.503 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T06:38:28.503 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T06:38:28.503 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T06:38:28.504 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T06:38:28.504 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T06:38:28.504 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T06:38:28.504 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T06:38:28.504 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T06:38:28.504 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T06:38:28.504 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T06:38:28.504 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T06:38:28.504 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T06:38:28.504 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T06:38:28.504 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T06:38:28.504 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T06:38:28.505 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T06:38:28.505 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T06:38:28.505 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T06:38:28.505 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T06:38:28.505 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T06:38:28.505 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T06:38:28.505 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T06:38:28.505 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T06:38:28.505 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T06:38:28.506 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T06:38:28.506 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T06:38:28.506 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T06:38:28.506 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T06:38:28.506 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T06:38:28.506 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T06:38:28.506 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T06:38:28.506 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T06:38:28.506 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T06:38:28.506 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T06:38:28.506 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T06:38:28.506 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T06:38:28.506 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T06:38:28.507 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T06:38:28.507 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T06:38:28.507 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T06:38:28.507 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T06:38:28.507 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T06:38:28.507 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T06:38:28.507 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T06:38:28.507 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T06:38:28.507 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T06:38:28.507 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T06:38:28.507 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T06:38:28.508 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T06:38:28.508 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T06:38:28.508 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T06:38:28.508 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T06:38:28.508 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T06:38:28.508 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T06:38:28.508 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T06:38:28.508 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T06:38:28.508 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T06:38:28.508 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T06:38:28.509 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T06:38:28.509 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T06:38:28.509 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T06:38:28.509 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T06:38:28.509 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T06:38:28.509 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T06:38:28.509 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T06:38:28.509 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T06:38:28.509 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T06:38:28.509 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T06:38:28.509 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T06:38:28.509 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T06:38:28.509 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T06:38:28.510 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T06:38:28.512 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T06:38:28.590 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T06:38:28.646 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T06:38:28.753 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T06:38:28.793 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T06:38:28.819 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T06:38:28.853 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T06:38:28.853 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T06:38:28.937 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T06:38:28.938 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T06:38:29.022 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T06:38:29.047 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T06:38:29.052 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:38:29.052 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:38:29.249 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:38:29.249 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T06:38:29.333 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T06:38:29.399 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:38:31.227 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:38:31.227 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:38:31.228 INFO:teuthology.orchestra.run.smithi174.stdout:Upgraded: 2024-09-19T06:38:31.228 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T06:38:31.228 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T06:38:31.228 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T06:38:31.228 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T06:38:31.228 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.228 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.228 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T06:38:31.228 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T06:38:31.228 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-09-19T06:38:31.228 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T06:38:31.228 INFO:teuthology.orchestra.run.smithi174.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T06:38:31.229 INFO:teuthology.orchestra.run.smithi174.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T06:38:31.229 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T06:38:31.229 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.229 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.229 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.229 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.229 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:31.229 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.229 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.229 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.229 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:31.229 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:31.229 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:31.230 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:31.230 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:31.230 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.230 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.230 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:31.230 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.230 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.230 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.230 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:31.230 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:31.230 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.230 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.230 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:31.231 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T06:38:31.231 INFO:teuthology.orchestra.run.smithi174.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T06:38:31.231 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T06:38:31.231 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T06:38:31.231 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T06:38:31.231 INFO:teuthology.orchestra.run.smithi174.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T06:38:31.231 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T06:38:31.231 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T06:38:31.231 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T06:38:31.231 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T06:38:31.231 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T06:38:31.231 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.231 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.232 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.232 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T06:38:31.232 INFO:teuthology.orchestra.run.smithi174.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T06:38:31.232 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T06:38:31.232 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T06:38:31.232 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T06:38:31.232 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.232 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.232 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T06:38:31.232 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.232 INFO:teuthology.orchestra.run.smithi174.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T06:38:31.232 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T06:38:31.233 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T06:38:31.233 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T06:38:31.233 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T06:38:31.233 INFO:teuthology.orchestra.run.smithi174.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T06:38:31.233 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T06:38:31.233 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T06:38:31.233 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T06:38:31.233 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T06:38:31.233 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T06:38:31.233 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T06:38:31.233 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T06:38:31.233 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T06:38:31.233 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T06:38:31.234 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T06:38:31.234 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T06:38:31.234 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T06:38:31.234 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T06:38:31.234 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T06:38:31.234 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T06:38:31.234 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T06:38:31.234 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T06:38:31.234 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.234 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.234 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.234 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T06:38:31.235 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T06:38:31.235 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T06:38:31.235 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T06:38:31.235 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T06:38:31.235 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T06:38:31.235 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T06:38:31.235 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T06:38:31.235 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T06:38:31.235 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T06:38:31.235 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T06:38:31.235 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T06:38:31.235 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T06:38:31.235 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T06:38:31.236 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T06:38:31.236 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T06:38:31.236 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T06:38:31.236 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T06:38:31.236 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T06:38:31.236 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T06:38:31.236 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T06:38:31.236 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T06:38:31.236 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T06:38:31.236 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T06:38:31.237 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T06:38:31.237 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T06:38:31.237 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T06:38:31.237 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T06:38:31.237 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T06:38:31.237 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T06:38:31.237 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T06:38:31.237 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T06:38:31.237 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T06:38:31.237 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T06:38:31.237 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T06:38:31.238 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T06:38:31.238 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T06:38:31.238 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T06:38:31.238 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T06:38:31.238 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T06:38:31.238 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.238 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.238 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T06:38:31.238 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T06:38:31.238 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T06:38:31.238 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T06:38:31.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T06:38:31.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T06:38:31.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T06:38:31.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T06:38:31.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T06:38:31.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T06:38:31.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T06:38:31.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T06:38:31.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T06:38:31.240 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T06:38:31.240 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T06:38:31.240 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T06:38:31.240 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T06:38:31.240 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T06:38:31.240 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.240 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.240 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:31.240 INFO:teuthology.orchestra.run.smithi174.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T06:38:31.240 INFO:teuthology.orchestra.run.smithi174.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T06:38:31.240 INFO:teuthology.orchestra.run.smithi174.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T06:38:31.240 INFO:teuthology.orchestra.run.smithi174.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T06:38:31.240 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T06:38:31.241 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T06:38:31.241 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T06:38:31.241 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:38:31.241 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-09-19T06:38:31.697 DEBUG:teuthology.parallel:result is None 2024-09-19T06:38:31.742 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T06:38:31.792 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T06:38:31.833 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T06:38:31.868 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T06:38:31.892 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T06:38:32.073 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T06:38:32.114 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T06:38:32.144 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T06:38:32.174 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T06:38:32.228 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T06:38:32.814 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:38:32.859 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:38:32.880 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:38:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T06:38:33.805 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T06:38:33.846 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T06:38:33.927 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T06:38:33.957 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T06:38:34.000 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T06:38:34.026 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T06:38:34.026 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T06:38:34.118 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T06:38:34.118 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T06:38:34.202 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T06:38:34.227 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T06:38:34.233 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:38:34.233 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:38:34.412 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:38:34.412 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T06:38:34.505 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T06:38:34.568 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:38:35.273 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:38:35.273 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T06:38:35.273 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T06:38:35.273 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T06:38:35.273 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T06:38:35.273 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T06:38:35.273 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T06:38:35.273 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T06:38:35.274 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:38:35.274 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:38:35.274 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T06:38:35.274 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T06:38:35.274 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T06:38:35.274 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:38:35.274 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T06:38:35.274 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T06:38:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T06:38:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T06:38:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T06:38:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T06:38:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T06:38:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T06:38:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T06:38:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T06:38:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T06:38:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T06:38:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T06:38:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T06:38:35.276 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T06:38:35.276 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T06:38:35.276 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T06:38:35.287 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T06:38:35.287 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T06:38:35.287 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T06:38:35.287 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T06:38:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T06:38:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T06:38:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T06:38:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T06:38:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T06:38:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T06:38:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T06:38:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T06:38:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T06:38:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T06:38:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T06:38:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T06:38:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T06:38:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T06:38:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T06:38:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T06:38:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T06:38:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T06:38:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T06:38:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T06:38:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T06:38:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T06:38:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T06:38:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T06:38:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T06:38:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T06:38:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T06:38:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T06:38:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T06:38:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T06:38:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T06:38:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T06:38:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T06:38:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T06:38:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T06:38:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T06:38:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T06:38:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T06:38:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T06:38:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T06:38:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T06:38:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T06:38:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T06:38:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T06:38:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T06:38:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T06:38:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T06:38:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T06:38:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T06:38:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T06:38:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T06:38:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T06:38:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T06:38:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T06:38:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T06:38:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T06:38:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T06:38:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T06:38:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T06:38:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T06:38:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T06:38:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T06:38:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T06:38:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T06:38:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T06:38:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T06:38:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T06:38:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T06:38:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T06:38:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T06:38:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T06:38:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T06:38:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T06:38:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T06:38:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T06:38:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T06:38:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T06:38:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T06:38:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T06:38:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T06:38:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T06:38:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T06:38:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T06:38:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T06:38:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T06:38:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T06:38:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T06:38:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T06:38:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T06:38:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T06:38:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T06:38:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T06:38:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T06:38:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T06:38:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T06:38:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T06:38:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T06:38:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T06:38:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T06:38:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T06:38:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T06:38:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T06:38:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T06:38:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T06:38:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T06:38:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T06:38:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T06:38:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T06:38:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T06:38:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T06:38:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T06:38:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T06:38:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T06:38:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T06:38:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T06:38:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T06:38:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T06:38:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T06:38:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T06:38:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T06:38:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T06:38:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T06:38:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T06:38:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T06:38:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T06:38:35.298 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T06:38:35.298 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T06:38:35.298 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T06:38:35.298 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T06:38:37.798 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:38:37.798 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:38:37.798 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2024-09-19T06:38:37.798 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T06:38:37.798 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T06:38:37.798 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T06:38:37.798 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T06:38:37.798 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.798 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.799 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T06:38:37.799 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T06:38:37.799 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-19T06:38:37.799 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T06:38:37.799 INFO:teuthology.orchestra.run.smithi103.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T06:38:37.799 INFO:teuthology.orchestra.run.smithi103.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T06:38:37.799 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T06:38:37.799 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.799 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.799 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.799 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.800 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:37.800 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.800 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.800 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.800 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:37.800 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:37.800 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:37.800 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:37.800 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:37.800 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.800 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.800 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:37.801 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.801 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.801 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.801 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:37.801 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:37.801 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.801 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.801 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:37.801 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T06:38:37.801 INFO:teuthology.orchestra.run.smithi103.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T06:38:37.802 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T06:38:37.802 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T06:38:37.802 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T06:38:37.802 INFO:teuthology.orchestra.run.smithi103.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T06:38:37.802 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T06:38:37.802 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T06:38:37.802 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T06:38:37.802 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T06:38:37.802 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T06:38:37.802 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.802 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.802 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.803 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T06:38:37.803 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T06:38:37.803 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T06:38:37.803 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T06:38:37.803 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T06:38:37.803 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.803 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.803 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T06:38:37.803 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.803 INFO:teuthology.orchestra.run.smithi103.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T06:38:37.803 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T06:38:37.803 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T06:38:37.803 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T06:38:37.804 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T06:38:37.804 INFO:teuthology.orchestra.run.smithi103.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T06:38:37.804 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T06:38:37.804 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T06:38:37.804 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T06:38:37.804 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T06:38:37.804 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T06:38:37.804 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T06:38:37.804 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T06:38:37.804 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T06:38:37.804 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T06:38:37.805 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T06:38:37.805 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T06:38:37.805 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T06:38:37.805 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T06:38:37.805 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T06:38:37.805 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T06:38:37.805 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T06:38:37.805 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T06:38:37.805 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.806 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.806 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.806 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T06:38:37.806 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T06:38:37.806 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T06:38:37.806 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T06:38:37.806 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T06:38:37.806 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T06:38:37.806 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T06:38:37.806 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T06:38:37.806 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T06:38:37.806 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T06:38:37.806 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T06:38:37.806 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T06:38:37.807 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T06:38:37.807 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T06:38:37.807 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T06:38:37.807 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T06:38:37.807 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T06:38:37.807 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T06:38:37.807 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T06:38:37.807 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T06:38:37.807 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T06:38:37.807 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T06:38:37.807 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T06:38:37.807 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T06:38:37.807 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T06:38:37.807 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T06:38:37.808 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T06:38:37.808 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T06:38:37.808 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T06:38:37.808 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T06:38:37.808 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T06:38:37.808 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T06:38:37.808 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T06:38:37.808 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T06:38:37.808 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T06:38:37.808 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T06:38:37.808 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T06:38:37.808 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T06:38:37.808 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T06:38:37.808 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T06:38:37.809 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.809 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.809 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T06:38:37.809 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T06:38:37.809 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T06:38:37.809 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.809 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T06:38:37.809 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T06:38:37.809 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T06:38:37.809 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T06:38:37.809 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T06:38:37.809 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T06:38:37.809 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T06:38:37.810 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T06:38:37.810 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T06:38:37.810 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T06:38:37.810 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T06:38:37.810 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T06:38:37.810 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T06:38:37.810 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T06:38:37.810 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T06:38:37.810 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.810 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.810 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:37.810 INFO:teuthology.orchestra.run.smithi103.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T06:38:37.810 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T06:38:37.810 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T06:38:37.811 INFO:teuthology.orchestra.run.smithi103.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T06:38:37.811 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T06:38:37.811 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T06:38:37.811 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T06:38:37.811 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:38:37.811 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-19T06:38:38.239 DEBUG:teuthology.parallel:result is None 2024-09-19T06:38:40.219 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:38:40.219 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T06:38:40.219 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T06:38:40.220 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T06:38:40.220 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T06:38:40.220 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T06:38:40.220 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T06:38:40.220 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T06:38:40.220 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:38:40.220 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:38:40.220 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T06:38:40.220 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T06:38:40.220 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T06:38:40.220 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:38:40.220 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T06:38:40.221 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T06:38:40.221 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T06:38:40.221 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T06:38:40.221 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T06:38:40.221 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T06:38:40.221 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T06:38:40.221 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T06:38:40.221 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T06:38:40.221 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T06:38:40.221 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T06:38:40.221 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T06:38:40.221 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T06:38:40.222 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T06:38:40.222 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T06:38:40.222 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T06:38:40.222 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T06:38:40.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T06:38:40.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T06:38:40.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T06:38:40.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T06:38:40.233 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T06:38:40.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T06:38:40.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T06:38:40.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T06:38:40.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T06:38:40.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T06:38:40.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T06:38:40.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T06:38:40.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T06:38:40.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T06:38:40.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T06:38:40.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T06:38:40.234 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T06:38:40.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T06:38:40.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T06:38:40.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T06:38:40.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T06:38:40.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T06:38:40.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T06:38:40.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T06:38:40.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T06:38:40.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T06:38:40.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T06:38:40.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T06:38:40.235 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T06:38:40.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T06:38:40.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T06:38:40.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T06:38:40.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T06:38:40.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T06:38:40.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T06:38:40.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T06:38:40.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T06:38:40.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T06:38:40.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T06:38:40.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T06:38:40.236 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T06:38:40.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T06:38:40.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T06:38:40.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T06:38:40.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T06:38:40.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T06:38:40.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T06:38:40.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T06:38:40.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T06:38:40.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T06:38:40.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T06:38:40.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T06:38:40.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T06:38:40.237 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T06:38:40.238 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T06:38:40.238 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T06:38:40.238 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T06:38:40.238 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T06:38:40.238 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T06:38:40.238 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T06:38:40.238 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T06:38:40.238 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T06:38:40.238 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T06:38:40.238 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T06:38:40.238 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T06:38:40.238 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T06:38:40.238 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T06:38:40.239 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T06:38:40.239 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T06:38:40.239 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T06:38:40.239 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T06:38:40.239 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T06:38:40.239 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T06:38:40.239 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T06:38:40.239 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T06:38:40.239 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T06:38:40.239 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T06:38:40.239 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T06:38:40.239 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T06:38:40.239 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T06:38:40.239 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T06:38:40.240 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T06:38:40.240 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T06:38:40.240 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T06:38:40.240 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T06:38:40.240 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T06:38:40.240 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T06:38:40.240 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T06:38:40.240 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T06:38:40.240 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T06:38:40.240 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T06:38:40.240 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T06:38:40.240 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T06:38:40.240 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T06:38:40.241 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T06:38:40.241 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T06:38:40.241 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T06:38:40.241 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T06:38:40.241 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T06:38:40.241 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T06:38:40.241 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T06:38:40.241 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T06:38:40.241 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T06:38:40.241 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T06:38:40.241 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T06:38:40.241 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T06:38:40.241 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T06:38:40.242 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T06:38:40.242 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T06:38:40.242 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T06:38:40.242 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T06:38:40.242 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T06:38:40.242 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T06:38:40.242 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T06:38:40.242 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T06:38:40.242 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T06:38:40.242 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T06:38:40.242 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T06:38:40.242 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T06:38:40.242 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T06:38:40.243 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T06:38:40.243 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T06:38:40.243 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T06:38:40.243 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T06:38:40.243 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T06:38:40.243 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T06:38:40.243 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T06:38:40.243 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T06:38:40.243 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T06:38:40.243 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T06:38:40.243 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T06:38:40.243 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T06:38:40.243 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T06:38:42.849 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:38:42.849 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:38:42.850 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-09-19T06:38:42.850 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T06:38:42.850 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T06:38:42.850 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T06:38:42.850 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T06:38:42.850 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.850 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.850 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T06:38:42.850 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T06:38:42.850 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-19T06:38:42.850 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T06:38:42.851 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T06:38:42.851 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T06:38:42.851 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T06:38:42.851 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.851 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.851 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.851 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.851 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:42.851 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.851 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.851 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.852 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:42.852 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:42.852 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:42.852 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:42.852 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:42.852 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.852 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.852 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:42.852 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.853 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.853 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.853 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:42.853 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:42.853 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.853 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.853 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:38:42.853 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T06:38:42.853 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T06:38:42.853 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T06:38:42.854 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T06:38:42.854 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T06:38:42.854 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T06:38:42.854 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T06:38:42.854 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T06:38:42.854 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T06:38:42.854 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T06:38:42.854 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T06:38:42.854 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.854 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.854 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.855 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T06:38:42.855 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T06:38:42.855 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T06:38:42.855 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T06:38:42.855 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T06:38:42.855 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.855 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.855 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T06:38:42.855 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.856 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T06:38:42.856 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T06:38:42.856 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T06:38:42.856 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T06:38:42.856 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T06:38:42.856 INFO:teuthology.orchestra.run.smithi134.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T06:38:42.856 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T06:38:42.856 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T06:38:42.856 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T06:38:42.856 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T06:38:42.856 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T06:38:42.856 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T06:38:42.857 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T06:38:42.857 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T06:38:42.857 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T06:38:42.857 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T06:38:42.857 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T06:38:42.857 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T06:38:42.857 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T06:38:42.857 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T06:38:42.857 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T06:38:42.857 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T06:38:42.857 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T06:38:42.858 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.858 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.858 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.858 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T06:38:42.858 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T06:38:42.858 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T06:38:42.858 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T06:38:42.858 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T06:38:42.859 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T06:38:42.859 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T06:38:42.859 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T06:38:42.859 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T06:38:42.859 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T06:38:42.859 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T06:38:42.859 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T06:38:42.859 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T06:38:42.859 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T06:38:42.859 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T06:38:42.860 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T06:38:42.860 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T06:38:42.860 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T06:38:42.860 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T06:38:42.860 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T06:38:42.860 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T06:38:42.860 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T06:38:42.860 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T06:38:42.860 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T06:38:42.860 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T06:38:42.861 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T06:38:42.861 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T06:38:42.861 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T06:38:42.861 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T06:38:42.861 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T06:38:42.861 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T06:38:42.861 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T06:38:42.861 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T06:38:42.861 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T06:38:42.861 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T06:38:42.861 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T06:38:42.862 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T06:38:42.862 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T06:38:42.862 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T06:38:42.862 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T06:38:42.862 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.862 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.862 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T06:38:42.862 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T06:38:42.862 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T06:38:42.862 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.862 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T06:38:42.862 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T06:38:42.863 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T06:38:42.863 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T06:38:42.863 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T06:38:42.863 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T06:38:42.863 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T06:38:42.863 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T06:38:42.863 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T06:38:42.863 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T06:38:42.863 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T06:38:42.863 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T06:38:42.863 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T06:38:42.863 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T06:38:42.864 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T06:38:42.864 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.864 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.864 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:38:42.864 INFO:teuthology.orchestra.run.smithi134.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T06:38:42.864 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T06:38:42.864 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T06:38:42.864 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T06:38:42.864 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T06:38:42.864 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T06:38:42.865 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T06:38:42.865 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:38:42.865 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-19T06:38:43.277 DEBUG:teuthology.parallel:result is None 2024-09-19T06:38:43.278 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:38:43.433 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T06:38:43.459 INFO:teuthology.orchestra.run.smithi103.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T06:38:43.460 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T06:38:43.460 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T06:38:43.462 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:38:43.601 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T06:38:43.624 INFO:teuthology.orchestra.run.smithi134.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T06:38:43.624 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T06:38:43.624 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T06:38:43.626 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:38:43.770 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T06:38:43.797 INFO:teuthology.orchestra.run.smithi174.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T06:38:43.797 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T06:38:43.798 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T06:38:43.799 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T06:38:43.800 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:38:43.800 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T06:38:43.832 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T06:38:43.832 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T06:38:43.861 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T06:38:43.862 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T06:38:43.894 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T06:38:43.895 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:38:43.895 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T06:38:43.923 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T06:38:43.991 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T06:38:43.992 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T06:38:44.018 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T06:38:44.085 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T06:38:44.085 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T06:38:44.116 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T06:38:44.186 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T06:38:44.187 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:38:44.187 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T06:38:44.216 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T06:38:44.285 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T06:38:44.285 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T06:38:44.312 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T06:38:44.378 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T06:38:44.378 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T06:38:44.403 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T06:38:44.469 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T06:38:44.469 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:38:44.469 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T06:38:44.500 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T06:38:44.568 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T06:38:44.568 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T06:38:44.593 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T06:38:44.660 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T06:38:44.660 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T06:38:44.685 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T06:38:44.751 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T06:38:44.845 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-19T06:38:44.846 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:38:44.846 INFO:tasks.cephadm:Cluster fsid is d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:38:44.846 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T06:38:44.846 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.103', 'mon.b': '172.21.15.134', 'mon.c': '172.21.15.174'} 2024-09-19T06:38:44.846 INFO:tasks.cephadm:First mon is mon.a on smithi103 2024-09-19T06:38:44.847 INFO:tasks.cephadm:First mgr is x 2024-09-19T06:38:44.847 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T06:38:44.847 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname $(hostname -s) 2024-09-19T06:38:44.877 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname $(hostname -s) 2024-09-19T06:38:44.905 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname $(hostname -s) 2024-09-19T06:38:44.930 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T06:38:44.931 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:38:45.073 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T06:38:45.213 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:38:45.215 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T06:38:45.215 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T06:38:45.215 DEBUG:teuthology.orchestra.run.smithi103:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T06:38:45.623 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 06:38 /home/ubuntu/cephtest/cephadm 2024-09-19T06:38:45.623 DEBUG:teuthology.orchestra.run.smithi134:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T06:38:46.018 INFO:teuthology.orchestra.run.smithi134.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 06:38 /home/ubuntu/cephtest/cephadm 2024-09-19T06:38:46.018 DEBUG:teuthology.orchestra.run.smithi174:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T06:38:46.390 INFO:teuthology.orchestra.run.smithi174.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 06:38 /home/ubuntu/cephtest/cephadm 2024-09-19T06:38:46.391 DEBUG:teuthology.orchestra.run.smithi103:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T06:38:46.411 DEBUG:teuthology.orchestra.run.smithi134:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T06:38:46.431 DEBUG:teuthology.orchestra.run.smithi174:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T06:38:46.467 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T06:38:46.468 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T06:38:46.471 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T06:38:46.474 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T06:38:47.997 INFO:teuthology.orchestra.run.smithi174.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T06:38:48.110 INFO:teuthology.orchestra.run.smithi134.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T06:38:48.175 INFO:teuthology.orchestra.run.smithi103.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T06:39:22.183 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-19T06:39:22.184 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T06:39:22.184 INFO:teuthology.orchestra.run.smithi174.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T06:39:22.184 INFO:teuthology.orchestra.run.smithi174.stdout: "repo_digests": [ 2024-09-19T06:39:22.184 INFO:teuthology.orchestra.run.smithi174.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T06:39:22.184 INFO:teuthology.orchestra.run.smithi174.stdout: ] 2024-09-19T06:39:22.184 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-19T06:39:32.389 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-19T06:39:32.390 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T06:39:32.390 INFO:teuthology.orchestra.run.smithi134.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T06:39:32.390 INFO:teuthology.orchestra.run.smithi134.stdout: "repo_digests": [ 2024-09-19T06:39:32.390 INFO:teuthology.orchestra.run.smithi134.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T06:39:32.390 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-19T06:39:32.390 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-19T06:39:37.580 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:39:37.581 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T06:39:37.581 INFO:teuthology.orchestra.run.smithi103.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T06:39:37.581 INFO:teuthology.orchestra.run.smithi103.stdout: "repo_digests": [ 2024-09-19T06:39:37.581 INFO:teuthology.orchestra.run.smithi103.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T06:39:37.581 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T06:39:37.581 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:39:37.608 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph 2024-09-19T06:39:37.655 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph 2024-09-19T06:39:37.689 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph 2024-09-19T06:39:37.723 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /etc/ceph 2024-09-19T06:39:37.748 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /etc/ceph 2024-09-19T06:39:37.782 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 777 /etc/ceph 2024-09-19T06:39:37.811 INFO:tasks.cephadm:Writing seed config... 2024-09-19T06:39:37.812 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T06:39:37.813 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T06:39:37.813 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T06:39:37.813 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T06:39:37.813 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T06:39:37.813 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-19T06:39:37.813 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T06:39:37.813 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T06:39:37.813 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T06:39:37.813 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T06:39:37.813 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T06:39:37.813 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T06:39:37.813 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T06:39:37.813 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T06:39:37.814 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T06:39:37.814 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T06:39:37.814 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T06:39:37.814 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-19T06:39:37.814 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T06:39:37.814 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T06:39:37.814 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T06:39:37.814 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T06:39:37.814 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-19T06:39:37.814 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T06:39:37.814 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T06:39:37.814 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T06:39:37.814 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T06:39:37.814 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T06:39:37.815 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T06:39:37.815 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T06:39:37.815 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T06:39:37.815 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T06:39:37.815 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-19T06:39:37.815 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-19T06:39:37.815 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-19T06:39:37.815 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T06:39:37.816 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T06:39:37.816 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-19T06:39:37.816 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-19T06:39:37.816 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T06:39:37.816 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T06:39:37.816 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-19T06:39:37.817 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T06:39:37.817 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T06:39:37.817 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T06:39:37.817 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T06:39:37.817 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T06:39:37.817 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T06:39:37.818 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T06:39:37.818 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-19T06:39:37.819 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:39:37.819 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T06:39:37.835 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d158d934-7651-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T06:39:37.836 DEBUG:teuthology.orchestra.run.smithi103:mon.a> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T06:39:37.878 DEBUG:teuthology.orchestra.run.smithi103:mgr.x> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T06:39:37.921 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T06:39:37.921 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid d158d934-7651-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.103 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:39:38.122 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:39:38.122 INFO:teuthology.orchestra.run.smithi103.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', 'd158d934-7651-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.103', '--skip-admin-label'] 2024-09-19T06:39:38.122 INFO:teuthology.orchestra.run.smithi103.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T06:39:38.123 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying podman|docker is present... 2024-09-19T06:39:38.146 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout 5.2.2 2024-09-19T06:39:38.146 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying lvm2 is present... 2024-09-19T06:39:38.147 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying time synchronization is in place... 2024-09-19T06:39:38.154 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T06:39:38.154 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T06:39:38.160 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T06:39:38.160 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-19T06:39:38.167 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout enabled 2024-09-19T06:39:38.173 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout active 2024-09-19T06:39:38.174 INFO:teuthology.orchestra.run.smithi103.stdout:Unit chronyd.service is enabled and running 2024-09-19T06:39:38.174 INFO:teuthology.orchestra.run.smithi103.stdout:Repeating the final host check... 2024-09-19T06:39:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout 5.2.2 2024-09-19T06:39:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T06:39:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl is present 2024-09-19T06:39:38.196 INFO:teuthology.orchestra.run.smithi103.stdout:lvcreate is present 2024-09-19T06:39:38.202 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T06:39:38.203 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T06:39:38.209 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T06:39:38.209 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-19T06:39:38.215 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout enabled 2024-09-19T06:39:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout active 2024-09-19T06:39:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:Unit chronyd.service is enabled and running 2024-09-19T06:39:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:Host looks OK 2024-09-19T06:39:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:Cluster fsid: d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:39:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:Acquiring lock 139802358273024 on /run/cephadm/d158d934-7651-11ef-bceb-c7b262605968.lock 2024-09-19T06:39:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:Lock 139802358273024 acquired on /run/cephadm/d158d934-7651-11ef-bceb-c7b262605968.lock 2024-09-19T06:39:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying IP 172.21.15.103 port 3300 ... 2024-09-19T06:39:38.223 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying IP 172.21.15.103 port 6789 ... 2024-09-19T06:39:38.224 INFO:teuthology.orchestra.run.smithi103.stdout:Base mon IP(s) is [172.21.15.103:3300, 172.21.15.103:6789], mon addrv is [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-09-19T06:39:38.226 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.103 metric 100 2024-09-19T06:39:38.226 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.103 metric 100 2024-09-19T06:39:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T06:39:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-19T06:39:38.229 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-19T06:39:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T06:39:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T06:39:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T06:39:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-19T06:39:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cc0b/64 scope link noprefixroute 2024-09-19T06:39:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T06:39:38.232 INFO:teuthology.orchestra.run.smithi103.stdout:Mon IP `172.21.15.103` is in CIDR network `172.21.0.0/20` 2024-09-19T06:39:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:Mon IP `172.21.15.103` is in CIDR network `172.21.0.0/20` 2024-09-19T06:39:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T06:39:38.233 INFO:teuthology.orchestra.run.smithi103.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T06:39:38.234 INFO:teuthology.orchestra.run.smithi103.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T06:39:38.717 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T06:39:38.717 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T06:39:38.717 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T06:39:38.717 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T06:39:38.718 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T06:39:38.718 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T06:39:38.718 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T06:39:39.663 INFO:teuthology.orchestra.run.smithi103.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T06:39:39.664 INFO:teuthology.orchestra.run.smithi103.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T06:39:39.664 INFO:teuthology.orchestra.run.smithi103.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T06:39:40.560 INFO:teuthology.orchestra.run.smithi103.stdout:stat: stdout 167 167 2024-09-19T06:39:40.560 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial keys... 2024-09-19T06:39:41.575 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQAsx+tm9zkuNBAAJDfXujFxhuqcXzzaRzJEEg== 2024-09-19T06:39:42.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQAtx+tmx2OZNRAAtwqb5k1MqBp6Dp04OlCfwg== 2024-09-19T06:39:43.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQAux+tmRq5xORAAb5LQuIix/HE5L5e4558F3g== 2024-09-19T06:39:43.546 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial monmap... 2024-09-19T06:39:44.392 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T06:39:44.393 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T06:39:44.393 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:39:44.393 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T06:39:44.393 INFO:teuthology.orchestra.run.smithi103.stdout:monmaptool for a [v2:172.21.15.103:3300,v1:172.21.15.103:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T06:39:44.393 INFO:teuthology.orchestra.run.smithi103.stdout:setting min_mon_release = quincy 2024-09-19T06:39:44.393 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: set fsid to d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:39:44.393 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T06:39:44.393 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:39:44.393 INFO:teuthology.orchestra.run.smithi103.stdout:Creating mon... 2024-09-19T06:39:45.349 INFO:teuthology.orchestra.run.smithi103.stdout:create mon.a on 2024-09-19T06:39:45.561 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T06:39:45.714 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T06:39:45.918 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d158d934-7651-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-d158d934-7651-11ef-bceb-c7b262605968.target. 2024-09-19T06:39:45.918 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d158d934-7651-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-d158d934-7651-11ef-bceb-c7b262605968.target. 2024-09-19T06:39:46.171 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d158d934-7651-11ef-bceb-c7b262605968@mon.a 2024-09-19T06:39:46.172 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d158d934-7651-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-d158d934-7651-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T06:39:46.336 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d158d934-7651-11ef-bceb-c7b262605968.target.wants/ceph-d158d934-7651-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-d158d934-7651-11ef-bceb-c7b262605968@.service. 2024-09-19T06:39:46.890 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T06:39:46.890 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-19T06:39:46.898 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T06:39:46.898 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-19T06:39:46.898 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-19T06:39:46.898 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T06:39:46.899 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mon to start... 2024-09-19T06:39:46.899 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mon... 2024-09-19T06:39:46.959 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:46 smithi103 systemd[1]: Started Ceph mon.a for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:39:47.950 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T06:39:47.950 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout id: d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:39:47.950 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T06:39:47.950 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T06:39:47.951 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout services: 2024-09-19T06:39:47.951 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.558823s) 2024-09-19T06:39:47.951 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T06:39:47.951 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T06:39:47.951 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T06:39:47.951 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout data: 2024-09-19T06:39:47.951 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T06:39:47.951 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T06:39:47.951 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T06:39:47.951 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T06:39:47.951 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T06:39:47.951 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.328+0000 7f6914a4a640 1 Processor -- start 2024-09-19T06:39:47.951 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.328+0000 7f6914a4a640 1 -- start start 2024-09-19T06:39:47.951 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.328+0000 7f6914a4a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69101078d0 0x7f6910107cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:47.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.328+0000 7f6914a4a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69101082a0 con 0x7f69101078d0 2024-09-19T06:39:47.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.329+0000 7f690e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69101078d0 0x7f6910107cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:47.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.329+0000 7f690e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69101078d0 0x7f6910107cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33908/0 (socket says 172.21.15.103:33908) 2024-09-19T06:39:47.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.329+0000 7f690e59c640 1 -- 172.21.15.103:0/1224704013 learned_addr learned my addr 172.21.15.103:0/1224704013 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:39:47.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.329+0000 7f690e59c640 1 -- 172.21.15.103:0/1224704013 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6910108a80 con 0x7f69101078d0 2024-09-19T06:39:47.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.331+0000 7f690e59c640 1 --2- 172.21.15.103:0/1224704013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69101078d0 0x7f6910107cd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6900009820 tx=0x7f690002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9d0c8b56d3c320 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:47.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.331+0000 7f690d59a640 1 -- 172.21.15.103:0/1224704013 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6900037070 con 0x7f69101078d0 2024-09-19T06:39:47.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.331+0000 7f690d59a640 1 -- 172.21.15.103:0/1224704013 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6900032440 con 0x7f69101078d0 2024-09-19T06:39:47.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.332+0000 7f690d59a640 1 -- 172.21.15.103:0/1224704013 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6900032740 con 0x7f69101078d0 2024-09-19T06:39:47.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.332+0000 7f6914a4a640 1 -- 172.21.15.103:0/1224704013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69101078d0 msgr2=0x7f6910107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:47.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.332+0000 7f6914a4a640 1 --2- 172.21.15.103:0/1224704013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69101078d0 0x7f6910107cd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6900009820 tx=0x7f690002a270 comp rx=0 tx=0).stop 2024-09-19T06:39:47.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.332+0000 7f6914a4a640 1 -- 172.21.15.103:0/1224704013 shutdown_connections 2024-09-19T06:39:47.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.332+0000 7f6914a4a640 1 --2- 172.21.15.103:0/1224704013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69101078d0 0x7f6910107cd0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:47.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.332+0000 7f6914a4a640 1 -- 172.21.15.103:0/1224704013 >> 172.21.15.103:0/1224704013 conn(0x7f6910103080 msgr2=0x7f69101054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:47.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.332+0000 7f6914a4a640 1 -- 172.21.15.103:0/1224704013 shutdown_connections 2024-09-19T06:39:47.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.332+0000 7f6914a4a640 1 -- 172.21.15.103:0/1224704013 wait complete. 2024-09-19T06:39:47.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.333+0000 7f6914a4a640 1 Processor -- start 2024-09-19T06:39:47.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.333+0000 7f6914a4a640 1 -- start start 2024-09-19T06:39:47.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.333+0000 7f6914a4a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69101078d0 0x7f691019b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:47.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.333+0000 7f6914a4a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6910109020 con 0x7f69101078d0 2024-09-19T06:39:47.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.333+0000 7f690e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69101078d0 0x7f691019b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:47.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.333+0000 7f690e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69101078d0 0x7f691019b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33918/0 (socket says 172.21.15.103:33918) 2024-09-19T06:39:47.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.333+0000 7f690e59c640 1 -- 172.21.15.103:0/3710064261 learned_addr learned my addr 172.21.15.103:0/3710064261 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:39:47.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.334+0000 7f690e59c640 1 -- 172.21.15.103:0/3710064261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691019b580 con 0x7f69101078d0 2024-09-19T06:39:47.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.334+0000 7f690e59c640 1 --2- 172.21.15.103:0/3710064261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69101078d0 0x7f691019b040 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f69000090b0 tx=0x7f69000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:47.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.334+0000 7f68ff7fe640 1 -- 172.21.15.103:0/3710064261 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6900041070 con 0x7f69101078d0 2024-09-19T06:39:47.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.334+0000 7f68ff7fe640 1 -- 172.21.15.103:0/3710064261 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f690003c7a0 con 0x7f69101078d0 2024-09-19T06:39:47.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.334+0000 7f6914a4a640 1 -- 172.21.15.103:0/3710064261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691019b810 con 0x7f69101078d0 2024-09-19T06:39:47.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.334+0000 7f68ff7fe640 1 -- 172.21.15.103:0/3710064261 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6900037040 con 0x7f69101078d0 2024-09-19T06:39:47.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.334+0000 7f6914a4a640 1 -- 172.21.15.103:0/3710064261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691019e500 con 0x7f69101078d0 2024-09-19T06:39:47.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.334+0000 7f68ff7fe640 1 -- 172.21.15.103:0/3710064261 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6900044070 con 0x7f69101078d0 2024-09-19T06:39:47.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.334+0000 7f68ff7fe640 1 -- 172.21.15.103:0/3710064261 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f690003d3f0 con 0x7f69101078d0 2024-09-19T06:39:47.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.335+0000 7f6914a4a640 1 -- 172.21.15.103:0/3710064261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6910109670 con 0x7f69101078d0 2024-09-19T06:39:47.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.337+0000 7f68ff7fe640 1 -- 172.21.15.103:0/3710064261 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6900046450 con 0x7f69101078d0 2024-09-19T06:39:47.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.396+0000 7f6914a4a640 1 -- 172.21.15.103:0/3710064261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f691019e990 con 0x7f69101078d0 2024-09-19T06:39:47.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.397+0000 7f68ff7fe640 1 -- 172.21.15.103:0/3710064261 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f6900046630 con 0x7f69101078d0 2024-09-19T06:39:47.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.397+0000 7f6914a4a640 1 -- 172.21.15.103:0/3710064261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69101078d0 msgr2=0x7f691019b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:47.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.397+0000 7f6914a4a640 1 --2- 172.21.15.103:0/3710064261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69101078d0 0x7f691019b040 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f69000090b0 tx=0x7f69000046c0 comp rx=0 tx=0).stop 2024-09-19T06:39:47.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.397+0000 7f6914a4a640 1 -- 172.21.15.103:0/3710064261 shutdown_connections 2024-09-19T06:39:47.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.397+0000 7f6914a4a640 1 --2- 172.21.15.103:0/3710064261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69101078d0 0x7f691019b040 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:47.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.397+0000 7f6914a4a640 1 -- 172.21.15.103:0/3710064261 >> 172.21.15.103:0/3710064261 conn(0x7f6910103080 msgr2=0x7f691018f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:47.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.397+0000 7f6914a4a640 1 -- 172.21.15.103:0/3710064261 shutdown_connections 2024-09-19T06:39:47.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:47.397+0000 7f6914a4a640 1 -- 172.21.15.103:0/3710064261 wait complete. 2024-09-19T06:39:47.955 INFO:teuthology.orchestra.run.smithi103.stdout:mon is available 2024-09-19T06:39:47.955 INFO:teuthology.orchestra.run.smithi103.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T06:39:48.051 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:47 smithi103 ceph-mon[30792]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T06:39:48.051 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:47 smithi103 ceph-mon[30792]: monmap epoch 1 2024-09-19T06:39:48.051 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:47 smithi103 ceph-mon[30792]: fsid d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:39:48.051 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:47 smithi103 ceph-mon[30792]: last_changed 2024-09-19T06:39:43.868601+0000 2024-09-19T06:39:48.051 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:47 smithi103 ceph-mon[30792]: created 2024-09-19T06:39:43.868601+0000 2024-09-19T06:39:48.051 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:47 smithi103 ceph-mon[30792]: min_mon_release 19 (squid) 2024-09-19T06:39:48.051 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:47 smithi103 ceph-mon[30792]: election_strategy: 1 2024-09-19T06:39:48.051 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:47 smithi103 ceph-mon[30792]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T06:39:48.051 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:47 smithi103 ceph-mon[30792]: fsmap 2024-09-19T06:39:48.051 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:47 smithi103 ceph-mon[30792]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T06:39:48.052 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:47 smithi103 ceph-mon[30792]: mgrmap e1: no daemons active 2024-09-19T06:39:48.052 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:47 smithi103 ceph-mon[30792]: from='client.? 172.21.15.103:0/3710064261' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T06:39:48.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T06:39:48.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T06:39:48.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout fsid = d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:39:48.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-09-19T06:39:48.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T06:39:48.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T06:39:48.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T06:39:48.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T06:39:48.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T06:39:48.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T06:39:48.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T06:39:48.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T06:39:48.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T06:39:48.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T06:39:48.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T06:39:48.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T06:39:48.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T06:39:48.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T06:39:48.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.310+0000 7f68f534f640 1 Processor -- start 2024-09-19T06:39:48.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.310+0000 7f68f534f640 1 -- start start 2024-09-19T06:39:48.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.310+0000 7f68f534f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f01078d0 0x7f68f0107cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:48.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.310+0000 7f68f534f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f68f01082a0 con 0x7f68f01078d0 2024-09-19T06:39:48.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.310+0000 7f68eeffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f01078d0 0x7f68f0107cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:48.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.311+0000 7f68eeffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f01078d0 0x7f68f0107cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33934/0 (socket says 172.21.15.103:33934) 2024-09-19T06:39:48.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.311+0000 7f68eeffd640 1 -- 172.21.15.103:0/3691562110 learned_addr learned my addr 172.21.15.103:0/3691562110 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:39:48.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.311+0000 7f68eeffd640 1 -- 172.21.15.103:0/3691562110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f0108a80 con 0x7f68f01078d0 2024-09-19T06:39:48.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.312+0000 7f68eeffd640 1 --2- 172.21.15.103:0/3691562110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f01078d0 0x7f68f0107cd0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009820 tx=0x7f68e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931ed8f1cff0ee68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:48.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.312+0000 7f68edffb640 1 -- 172.21.15.103:0/3691562110 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68e0037070 con 0x7f68f01078d0 2024-09-19T06:39:48.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.312+0000 7f68edffb640 1 -- 172.21.15.103:0/3691562110 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68e0032440 con 0x7f68f01078d0 2024-09-19T06:39:48.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.312+0000 7f68edffb640 1 -- 172.21.15.103:0/3691562110 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68e0032740 con 0x7f68f01078d0 2024-09-19T06:39:48.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.312+0000 7f68f534f640 1 -- 172.21.15.103:0/3691562110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f01078d0 msgr2=0x7f68f0107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:48.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.312+0000 7f68f534f640 1 --2- 172.21.15.103:0/3691562110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f01078d0 0x7f68f0107cd0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009820 tx=0x7f68e002a270 comp rx=0 tx=0).stop 2024-09-19T06:39:48.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.312+0000 7f68f534f640 1 -- 172.21.15.103:0/3691562110 shutdown_connections 2024-09-19T06:39:48.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.312+0000 7f68f534f640 1 --2- 172.21.15.103:0/3691562110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f01078d0 0x7f68f0107cd0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:48.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.312+0000 7f68f534f640 1 -- 172.21.15.103:0/3691562110 >> 172.21.15.103:0/3691562110 conn(0x7f68f0103080 msgr2=0x7f68f01054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:48.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.312+0000 7f68f534f640 1 -- 172.21.15.103:0/3691562110 shutdown_connections 2024-09-19T06:39:48.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.312+0000 7f68f534f640 1 -- 172.21.15.103:0/3691562110 wait complete. 2024-09-19T06:39:48.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.313+0000 7f68f534f640 1 Processor -- start 2024-09-19T06:39:48.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.313+0000 7f68f534f640 1 -- start start 2024-09-19T06:39:48.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.313+0000 7f68f534f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f01078d0 0x7f68f007db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:48.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.313+0000 7f68f534f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f68f0109020 con 0x7f68f01078d0 2024-09-19T06:39:48.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.313+0000 7f68eeffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f01078d0 0x7f68f007db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:48.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.313+0000 7f68eeffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f01078d0 0x7f68f007db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33946/0 (socket says 172.21.15.103:33946) 2024-09-19T06:39:48.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.313+0000 7f68eeffd640 1 -- 172.21.15.103:0/516017646 learned_addr learned my addr 172.21.15.103:0/516017646 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:39:48.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.314+0000 7f68eeffd640 1 -- 172.21.15.103:0/516017646 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f007e040 con 0x7f68f01078d0 2024-09-19T06:39:48.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.314+0000 7f68eeffd640 1 --2- 172.21.15.103:0/516017646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f01078d0 0x7f68f007db00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f68e00090b0 tx=0x7f68e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:48.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.314+0000 7f68d7fff640 1 -- 172.21.15.103:0/516017646 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68e0041070 con 0x7f68f01078d0 2024-09-19T06:39:48.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.314+0000 7f68d7fff640 1 -- 172.21.15.103:0/516017646 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68e003c7a0 con 0x7f68f01078d0 2024-09-19T06:39:48.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.314+0000 7f68f534f640 1 -- 172.21.15.103:0/516017646 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f007c230 con 0x7f68f01078d0 2024-09-19T06:39:48.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.314+0000 7f68d7fff640 1 -- 172.21.15.103:0/516017646 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68e0037040 con 0x7f68f01078d0 2024-09-19T06:39:48.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.314+0000 7f68f534f640 1 -- 172.21.15.103:0/516017646 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f007c770 con 0x7f68f01078d0 2024-09-19T06:39:48.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.314+0000 7f68d7fff640 1 -- 172.21.15.103:0/516017646 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f68e0044070 con 0x7f68f01078d0 2024-09-19T06:39:48.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.314+0000 7f68d7fff640 1 -- 172.21.15.103:0/516017646 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f68e003d3f0 con 0x7f68f01078d0 2024-09-19T06:39:48.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.315+0000 7f68f534f640 1 -- 172.21.15.103:0/516017646 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68f0109670 con 0x7f68f01078d0 2024-09-19T06:39:48.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.317+0000 7f68d7fff640 1 -- 172.21.15.103:0/516017646 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f68e0046450 con 0x7f68f01078d0 2024-09-19T06:39:48.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.374+0000 7f68f534f640 1 -- 172.21.15.103:0/516017646 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f68f007c9f0 con 0x7f68f01078d0 2024-09-19T06:39:48.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.377+0000 7f68d7fff640 1 -- 172.21.15.103:0/516017646 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f68e0046630 con 0x7f68f01078d0 2024-09-19T06:39:48.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.378+0000 7f68d7fff640 1 -- 172.21.15.103:0/516017646 <== mon.0 v2:172.21.15.103:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f68e0056070 con 0x7f68f01078d0 2024-09-19T06:39:48.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.379+0000 7f68f534f640 1 -- 172.21.15.103:0/516017646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f01078d0 msgr2=0x7f68f007db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:48.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.379+0000 7f68f534f640 1 --2- 172.21.15.103:0/516017646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f01078d0 0x7f68f007db00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f68e00090b0 tx=0x7f68e00046c0 comp rx=0 tx=0).stop 2024-09-19T06:39:48.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.379+0000 7f68f534f640 1 -- 172.21.15.103:0/516017646 shutdown_connections 2024-09-19T06:39:48.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.379+0000 7f68f534f640 1 --2- 172.21.15.103:0/516017646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68f01078d0 0x7f68f007db00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:48.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.379+0000 7f68f534f640 1 -- 172.21.15.103:0/516017646 >> 172.21.15.103:0/516017646 conn(0x7f68f0103080 msgr2=0x7f68f018f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:48.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.379+0000 7f68f534f640 1 -- 172.21.15.103:0/516017646 shutdown_connections 2024-09-19T06:39:48.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:48.379+0000 7f68f534f640 1 -- 172.21.15.103:0/516017646 wait complete. 2024-09-19T06:39:48.970 INFO:teuthology.orchestra.run.smithi103.stdout:Generating new minimal ceph.conf... 2024-09-19T06:39:49.057 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:48 smithi103 ceph-mon[30792]: from='client.? 172.21.15.103:0/516017646' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T06:39:49.057 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:48 smithi103 ceph-mon[30792]: from='client.? 172.21.15.103:0/516017646' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T06:39:49.980 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:49 smithi103 ceph-mon[30792]: from='client.? 172.21.15.103:0/155083328' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:49.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.351+0000 7fe1c749a640 1 Processor -- start 2024-09-19T06:39:49.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.351+0000 7fe1c749a640 1 -- start start 2024-09-19T06:39:49.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.352+0000 7fe1c749a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0106780 0x7fe1c0106b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:49.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.352+0000 7fe1c749a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c0107150 con 0x7fe1c0106780 2024-09-19T06:39:49.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.352+0000 7fe1c5236640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0106780 0x7fe1c0106b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:49.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.352+0000 7fe1c5236640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0106780 0x7fe1c0106b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33962/0 (socket says 172.21.15.103:33962) 2024-09-19T06:39:49.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.352+0000 7fe1c5236640 1 -- 172.21.15.103:0/122420815 learned_addr learned my addr 172.21.15.103:0/122420815 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:39:49.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.352+0000 7fe1c5236640 1 -- 172.21.15.103:0/122420815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c0107980 con 0x7fe1c0106780 2024-09-19T06:39:49.987 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.353+0000 7fe1c5236640 1 --2- 172.21.15.103:0/122420815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0106780 0x7fe1c0106b80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009820 tx=0x7fe1b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bdfef2d8627295 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:49.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.353+0000 7fe1b3fff640 1 -- 172.21.15.103:0/122420815 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1b4037070 con 0x7fe1c0106780 2024-09-19T06:39:49.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.353+0000 7fe1b3fff640 1 -- 172.21.15.103:0/122420815 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1b4032900 con 0x7fe1c0106780 2024-09-19T06:39:49.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.353+0000 7fe1b3fff640 1 -- 172.21.15.103:0/122420815 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1b4030850 con 0x7fe1c0106780 2024-09-19T06:39:49.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.353+0000 7fe1c749a640 1 -- 172.21.15.103:0/122420815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0106780 msgr2=0x7fe1c0106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:49.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.353+0000 7fe1c749a640 1 --2- 172.21.15.103:0/122420815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0106780 0x7fe1c0106b80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009820 tx=0x7fe1b402a270 comp rx=0 tx=0).stop 2024-09-19T06:39:49.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.354+0000 7fe1c749a640 1 -- 172.21.15.103:0/122420815 shutdown_connections 2024-09-19T06:39:49.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.354+0000 7fe1c749a640 1 --2- 172.21.15.103:0/122420815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0106780 0x7fe1c0106b80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:49.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.354+0000 7fe1c749a640 1 -- 172.21.15.103:0/122420815 >> 172.21.15.103:0/122420815 conn(0x7fe1c0101f10 msgr2=0x7fe1c0104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:49.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.354+0000 7fe1c749a640 1 -- 172.21.15.103:0/122420815 shutdown_connections 2024-09-19T06:39:49.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.354+0000 7fe1c749a640 1 -- 172.21.15.103:0/122420815 wait complete. 2024-09-19T06:39:49.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.354+0000 7fe1c749a640 1 Processor -- start 2024-09-19T06:39:49.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.354+0000 7fe1c749a640 1 -- start start 2024-09-19T06:39:49.988 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.354+0000 7fe1c749a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0106780 0x7fe1c019b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:49.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.354+0000 7fe1c749a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c010ad30 con 0x7fe1c0106780 2024-09-19T06:39:49.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.355+0000 7fe1c5236640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0106780 0x7fe1c019b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:49.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.355+0000 7fe1c5236640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0106780 0x7fe1c019b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33972/0 (socket says 172.21.15.103:33972) 2024-09-19T06:39:49.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.355+0000 7fe1c5236640 1 -- 172.21.15.103:0/155083328 learned_addr learned my addr 172.21.15.103:0/155083328 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:39:49.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.355+0000 7fe1c5236640 1 -- 172.21.15.103:0/155083328 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c019be60 con 0x7fe1c0106780 2024-09-19T06:39:49.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.355+0000 7fe1c5236640 1 --2- 172.21.15.103:0/155083328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0106780 0x7fe1c019b920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009940 tx=0x7fe1b40326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:49.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.355+0000 7fe1b27fc640 1 -- 172.21.15.103:0/155083328 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1b4042070 con 0x7fe1c0106780 2024-09-19T06:39:49.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.355+0000 7fe1b27fc640 1 -- 172.21.15.103:0/155083328 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1b4032be0 con 0x7fe1c0106780 2024-09-19T06:39:49.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.355+0000 7fe1c749a640 1 -- 172.21.15.103:0/155083328 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c019c0f0 con 0x7fe1c0106780 2024-09-19T06:39:49.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.355+0000 7fe1b27fc640 1 -- 172.21.15.103:0/155083328 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1b4037040 con 0x7fe1c0106780 2024-09-19T06:39:49.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.355+0000 7fe1c749a640 1 -- 172.21.15.103:0/155083328 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c019ede0 con 0x7fe1c0106780 2024-09-19T06:39:49.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.355+0000 7fe1b27fc640 1 -- 172.21.15.103:0/155083328 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe1b4045070 con 0x7fe1c0106780 2024-09-19T06:39:49.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.355+0000 7fe1b27fc640 1 -- 172.21.15.103:0/155083328 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe1b4047a80 con 0x7fe1c0106780 2024-09-19T06:39:49.989 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.356+0000 7fe1c749a640 1 -- 172.21.15.103:0/155083328 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c010aeb0 con 0x7fe1c0106780 2024-09-19T06:39:49.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.358+0000 7fe1b27fc640 1 -- 172.21.15.103:0/155083328 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe1b4032d90 con 0x7fe1c0106780 2024-09-19T06:39:49.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.416+0000 7fe1c749a640 1 -- 172.21.15.103:0/155083328 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe1c0195850 con 0x7fe1c0106780 2024-09-19T06:39:49.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.416+0000 7fe1b27fc640 1 -- 172.21.15.103:0/155083328 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fe1b403d430 con 0x7fe1c0106780 2024-09-19T06:39:49.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.417+0000 7fe1c749a640 1 -- 172.21.15.103:0/155083328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0106780 msgr2=0x7fe1c019b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:49.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.417+0000 7fe1c749a640 1 --2- 172.21.15.103:0/155083328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0106780 0x7fe1c019b920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009940 tx=0x7fe1b40326e0 comp rx=0 tx=0).stop 2024-09-19T06:39:49.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.417+0000 7fe1c749a640 1 -- 172.21.15.103:0/155083328 shutdown_connections 2024-09-19T06:39:49.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.417+0000 7fe1c749a640 1 --2- 172.21.15.103:0/155083328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0106780 0x7fe1c019b920 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:49.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.417+0000 7fe1c749a640 1 -- 172.21.15.103:0/155083328 >> 172.21.15.103:0/155083328 conn(0x7fe1c0101f10 msgr2=0x7fe1c0102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:49.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.417+0000 7fe1c749a640 1 -- 172.21.15.103:0/155083328 shutdown_connections 2024-09-19T06:39:49.990 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:49.417+0000 7fe1c749a640 1 -- 172.21.15.103:0/155083328 wait complete. 2024-09-19T06:39:49.991 INFO:teuthology.orchestra.run.smithi103.stdout:Restarting the monitor... 2024-09-19T06:39:50.242 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:50 smithi103 systemd[1]: Stopping Ceph mon.a for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T06:39:50.549 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:50 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-a[30788]: 2024-09-19T06:39:50.239+0000 7fc3ec7a1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:39:50.549 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:50 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-a[30788]: 2024-09-19T06:39:50.239+0000 7fc3ec7a1640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T06:39:50.549 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:50 smithi103 podman[31036]: 2024-09-19 06:39:50.282259833 +0000 UTC m=+0.137769167 container died cfc4dee90e4c92d38dabc9a4dc9b906b90beca22ab69dd5276b07bc9d1343e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-a, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:39:50.549 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:50 smithi103 podman[31036]: 2024-09-19 06:39:50.548840031 +0000 UTC m=+0.404349364 container cleanup cfc4dee90e4c92d38dabc9a4dc9b906b90beca22ab69dd5276b07bc9d1343e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-19T06:39:50.549 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:50 smithi103 bash[31036]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-a 2024-09-19T06:39:50.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:50 smithi103 podman[31047]: 2024-09-19 06:39:50.783206975 +0000 UTC m=+0.505078334 container remove cfc4dee90e4c92d38dabc9a4dc9b906b90beca22ab69dd5276b07bc9d1343e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-19T06:39:51.224 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:50 smithi103 systemd[1]: ceph-d158d934-7651-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T06:39:51.224 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:50 smithi103 systemd[1]: Stopped Ceph mon.a for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:39:51.224 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:50 smithi103 systemd[1]: Starting Ceph mon.a for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T06:39:51.224 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 podman[31139]: 2024-09-19 06:39:51.117475858 +0000 UTC m=+0.111420155 container create 0b2d620a0814f47adb7b84144bc63a8b715f29fc07a1846a6080b944f3fcd578 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T06:39:51.224 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 podman[31139]: 2024-09-19 06:39:51.022175268 +0000 UTC m=+0.016119572 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:39:51.416 INFO:teuthology.orchestra.run.smithi103.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T06:39:51.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 podman[31139]: 2024-09-19 06:39:51.336720008 +0000 UTC m=+0.330664315 container init 0b2d620a0814f47adb7b84144bc63a8b715f29fc07a1846a6080b944f3fcd578 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T06:39:51.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 podman[31139]: 2024-09-19 06:39:51.340040504 +0000 UTC m=+0.333984801 container start 0b2d620a0814f47adb7b84144bc63a8b715f29fc07a1846a6080b944f3fcd578 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-19T06:39:51.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T06:39:51.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T06:39:51.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: pidfile_write: ignore empty --pid-file 2024-09-19T06:39:51.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: load: jerasure load: lrc 2024-09-19T06:39:51.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: RocksDB version: 7.9.2 2024-09-19T06:39:51.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Git sha 0 2024-09-19T06:39:51.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T06:39:51.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: DB SUMMARY 2024-09-19T06:39:51.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: DB Session ID: 0C8H6AUIO9L3PC2FYR0W 2024-09-19T06:39:51.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: CURRENT file: CURRENT 2024-09-19T06:39:51.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T06:39:51.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T06:39:51.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T06:39:51.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106682 ; 2024-09-19T06:39:51.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.error_if_exists: 0 2024-09-19T06:39:51.640 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.create_if_missing: 0 2024-09-19T06:39:51.640 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.paranoid_checks: 1 2024-09-19T06:39:51.640 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T06:39:51.640 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T06:39:51.640 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T06:39:51.640 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.env: 0x5635a11b6940 2024-09-19T06:39:51.640 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T06:39:51.640 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.info_log: 0x5635a2a1f6e0 2024-09-19T06:39:51.640 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T06:39:51.640 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.statistics: (nil) 2024-09-19T06:39:51.640 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.use_fsync: 0 2024-09-19T06:39:51.640 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_log_file_size: 0 2024-09-19T06:39:51.640 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T06:39:51.641 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T06:39:51.641 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T06:39:51.641 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T06:39:51.641 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.allow_fallocate: 1 2024-09-19T06:39:51.641 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T06:39:51.641 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T06:39:51.641 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.use_direct_reads: 0 2024-09-19T06:39:51.641 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T06:39:51.641 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T06:39:51.641 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.db_log_dir: 2024-09-19T06:39:51.641 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.wal_dir: 2024-09-19T06:39:51.641 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T06:39:51.641 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T06:39:51.641 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T06:39:51.642 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T06:39:51.642 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T06:39:51.642 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T06:39:51.642 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T06:39:51.642 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T06:39:51.642 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.write_buffer_manager: 0x5635a2a2edc0 2024-09-19T06:39:51.642 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T06:39:51.642 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T06:39:51.642 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T06:39:51.642 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T06:39:51.642 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T06:39:51.642 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T06:39:51.642 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T06:39:51.642 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T06:39:51.643 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.unordered_write: 0 2024-09-19T06:39:51.643 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T06:39:51.643 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T06:39:51.643 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T06:39:51.643 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T06:39:51.643 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.row_cache: None 2024-09-19T06:39:51.643 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.wal_filter: None 2024-09-19T06:39:51.643 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T06:39:51.644 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T06:39:51.645 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.two_write_queues: 0 2024-09-19T06:39:51.645 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T06:39:51.645 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.wal_compression: 0 2024-09-19T06:39:51.645 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.atomic_flush: 0 2024-09-19T06:39:51.645 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T06:39:51.645 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T06:39:51.645 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T06:39:51.645 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.log_readahead_size: 0 2024-09-19T06:39:51.645 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T06:39:51.645 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T06:39:51.645 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T06:39:51.645 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T06:39:51.645 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T06:39:51.646 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T06:39:51.646 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T06:39:51.646 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_background_jobs: 2 2024-09-19T06:39:51.646 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_background_compactions: -1 2024-09-19T06:39:51.646 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_subcompactions: 1 2024-09-19T06:39:51.646 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T06:39:51.646 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T06:39:51.646 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T06:39:51.646 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T06:39:51.646 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T06:39:51.646 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T06:39:51.646 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T06:39:51.646 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T06:39:51.647 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_open_files: -1 2024-09-19T06:39:51.647 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T06:39:51.647 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T06:39:51.647 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T06:39:51.647 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T06:39:51.647 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_background_flushes: -1 2024-09-19T06:39:51.647 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Compression algorithms supported: 2024-09-19T06:39:51.647 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: kZSTD supported: 0 2024-09-19T06:39:51.647 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: kXpressCompression supported: 0 2024-09-19T06:39:51.647 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: kBZip2Compression supported: 0 2024-09-19T06:39:51.647 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T06:39:51.647 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: kLZ4Compression supported: 1 2024-09-19T06:39:51.647 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: kZlibCompression supported: 1 2024-09-19T06:39:51.647 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T06:39:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: kSnappyCompression supported: 1 2024-09-19T06:39:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T06:39:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T06:39:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T06:39:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T06:39:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T06:39:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.merge_operator: 2024-09-19T06:39:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compaction_filter: None 2024-09-19T06:39:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T06:39:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T06:39:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T06:39:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T06:39:51.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5635a2a1f320) 2024-09-19T06:39:51.649 INFO:journalctl@ceph.mon.a.smithi103.stdout: cache_index_and_filter_blocks: 1 2024-09-19T06:39:51.649 INFO:journalctl@ceph.mon.a.smithi103.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T06:39:51.649 INFO:journalctl@ceph.mon.a.smithi103.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T06:39:51.649 INFO:journalctl@ceph.mon.a.smithi103.stdout: pin_top_level_index_and_filter: 1 2024-09-19T06:39:51.649 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_type: 0 2024-09-19T06:39:51.649 INFO:journalctl@ceph.mon.a.smithi103.stdout: data_block_index_type: 0 2024-09-19T06:39:51.649 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_shortening: 1 2024-09-19T06:39:51.649 INFO:journalctl@ceph.mon.a.smithi103.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T06:39:51.649 INFO:journalctl@ceph.mon.a.smithi103.stdout: checksum: 4 2024-09-19T06:39:51.649 INFO:journalctl@ceph.mon.a.smithi103.stdout: no_block_cache: 0 2024-09-19T06:39:51.649 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache: 0x5635a2a131f0 2024-09-19T06:39:51.649 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_name: BinnedLRUCache 2024-09-19T06:39:51.649 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_options: 2024-09-19T06:39:51.649 INFO:journalctl@ceph.mon.a.smithi103.stdout: capacity : 536870912 2024-09-19T06:39:51.650 INFO:journalctl@ceph.mon.a.smithi103.stdout: num_shard_bits : 4 2024-09-19T06:39:51.650 INFO:journalctl@ceph.mon.a.smithi103.stdout: strict_capacity_limit : 0 2024-09-19T06:39:51.650 INFO:journalctl@ceph.mon.a.smithi103.stdout: high_pri_pool_ratio: 0.000 2024-09-19T06:39:51.650 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_compressed: (nil) 2024-09-19T06:39:51.650 INFO:journalctl@ceph.mon.a.smithi103.stdout: persistent_cache: (nil) 2024-09-19T06:39:51.650 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_size: 4096 2024-09-19T06:39:51.650 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_size_deviation: 10 2024-09-19T06:39:51.650 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_restart_interval: 16 2024-09-19T06:39:51.650 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_block_restart_interval: 1 2024-09-19T06:39:51.650 INFO:journalctl@ceph.mon.a.smithi103.stdout: metadata_block_size: 4096 2024-09-19T06:39:51.650 INFO:journalctl@ceph.mon.a.smithi103.stdout: partition_filters: 0 2024-09-19T06:39:51.650 INFO:journalctl@ceph.mon.a.smithi103.stdout: use_delta_encoding: 1 2024-09-19T06:39:51.650 INFO:journalctl@ceph.mon.a.smithi103.stdout: filter_policy: bloomfilter 2024-09-19T06:39:51.650 INFO:journalctl@ceph.mon.a.smithi103.stdout: whole_key_filtering: 1 2024-09-19T06:39:51.651 INFO:journalctl@ceph.mon.a.smithi103.stdout: verify_compression: 0 2024-09-19T06:39:51.651 INFO:journalctl@ceph.mon.a.smithi103.stdout: read_amp_bytes_per_bit: 0 2024-09-19T06:39:51.651 INFO:journalctl@ceph.mon.a.smithi103.stdout: format_version: 5 2024-09-19T06:39:51.651 INFO:journalctl@ceph.mon.a.smithi103.stdout: enable_index_compression: 1 2024-09-19T06:39:51.651 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_align: 0 2024-09-19T06:39:51.651 INFO:journalctl@ceph.mon.a.smithi103.stdout: max_auto_readahead_size: 262144 2024-09-19T06:39:51.651 INFO:journalctl@ceph.mon.a.smithi103.stdout: prepopulate_block_cache: 0 2024-09-19T06:39:51.651 INFO:journalctl@ceph.mon.a.smithi103.stdout: initial_auto_readahead_size: 8192 2024-09-19T06:39:51.651 INFO:journalctl@ceph.mon.a.smithi103.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T06:39:51.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T06:39:51.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T06:39:51.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compression: NoCompression 2024-09-19T06:39:51.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T06:39:51.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T06:39:51.652 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T06:39:51.652 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.num_levels: 7 2024-09-19T06:39:51.652 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T06:39:51.652 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T06:39:51.652 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T06:39:51.652 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T06:39:51.652 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T06:39:51.652 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T06:39:51.652 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T06:39:51.652 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T06:39:51.652 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T06:39:51.652 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T06:39:51.652 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T06:39:51.653 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T06:39:51.653 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T06:39:51.653 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T06:39:51.653 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T06:39:51.653 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T06:39:51.653 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T06:39:51.653 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T06:39:51.653 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T06:39:51.653 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T06:39:51.653 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T06:39:51.653 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T06:39:51.653 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T06:39:51.653 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T06:39:51.653 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T06:39:51.654 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T06:39:51.654 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T06:39:51.654 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T06:39:51.654 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T06:39:51.654 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T06:39:51.654 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T06:39:51.654 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T06:39:51.654 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T06:39:51.654 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T06:39:51.654 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T06:39:51.654 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T06:39:51.654 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T06:39:51.654 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T06:39:51.654 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T06:39:51.655 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T06:39:51.655 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T06:39:51.655 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T06:39:51.655 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T06:39:51.655 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T06:39:51.655 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T06:39:51.655 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T06:39:51.655 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T06:39:51.655 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T06:39:51.655 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T06:39:51.655 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T06:39:51.655 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T06:39:51.655 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T06:39:51.655 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T06:39:51.656 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T06:39:51.656 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.inplace_update_support: 0 2024-09-19T06:39:51.656 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T06:39:51.656 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T06:39:51.656 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T06:39:51.656 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T06:39:51.656 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.bloom_locality: 0 2024-09-19T06:39:51.656 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.max_successive_merges: 0 2024-09-19T06:39:51.656 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T06:39:51.656 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T06:39:51.656 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T06:39:51.656 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T06:39:51.656 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.ttl: 2592000 2024-09-19T06:39:51.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T06:39:51.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T06:39:51.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T06:39:51.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.enable_blob_files: false 2024-09-19T06:39:51.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.min_blob_size: 0 2024-09-19T06:39:51.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T06:39:51.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T06:39:51.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T06:39:51.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T06:39:51.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T06:39:51.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T06:39:51.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T06:39:51.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T06:39:51.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T06:39:51.658 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T06:39:51.658 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0b2c88f6-91d2-413f-81aa-12453bab1cd4 2024-09-19T06:39:51.658 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726727991366153, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T06:39:51.658 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T06:39:51.658 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726727991367206, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101588, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 334, "table_properties": {"data_size": 99471, "index_size": 309, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14649, "raw_average_key_size": 49, "raw_value_size": 91578, "raw_average_value_size": 312, "num_data_blocks": 12, "num_entries": 293, "num_filter_entries": 293, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726727991, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0b2c88f6-91d2-413f-81aa-12453bab1cd4", "db_session_id": "0C8H6AUIO9L3PC2FYR0W", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T06:39:51.658 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726727991367301, "job": 1, "event": "recovery_finished"} 2024-09-19T06:39:51.658 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T06:39:51.658 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T06:39:51.658 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5635a2a40e00 2024-09-19T06:39:51.658 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: DB pointer 0x5635a2b3c000 2024-09-19T06:39:51.658 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T06:39:51.658 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T06:39:51.658 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** DB Stats ** 2024-09-19T06:39:51.658 INFO:journalctl@ceph.mon.a.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T06:39:51.659 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T06:39:51.659 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T06:39:51.659 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T06:39:51.659 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T06:39:51.659 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T06:39:51.659 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T06:39:51.659 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-19T06:39:51.659 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** Compaction Stats [default] ** 2024-09-19T06:39:51.659 INFO:journalctl@ceph.mon.a.smithi103.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T06:39:51.659 INFO:journalctl@ceph.mon.a.smithi103.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T06:39:51.659 INFO:journalctl@ceph.mon.a.smithi103.stdout: L0 2/0 101.07 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 142.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T06:39:51.659 INFO:journalctl@ceph.mon.a.smithi103.stdout: Sum 2/0 101.07 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 142.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T06:39:51.659 INFO:journalctl@ceph.mon.a.smithi103.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 142.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T06:39:51.660 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-19T06:39:51.660 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** Compaction Stats [default] ** 2024-09-19T06:39:51.660 INFO:journalctl@ceph.mon.a.smithi103.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T06:39:51.660 INFO:journalctl@ceph.mon.a.smithi103.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T06:39:51.660 INFO:journalctl@ceph.mon.a.smithi103.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 142.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T06:39:51.660 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-19T06:39:51.660 INFO:journalctl@ceph.mon.a.smithi103.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T06:39:51.660 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-19T06:39:51.660 INFO:journalctl@ceph.mon.a.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T06:39:51.660 INFO:journalctl@ceph.mon.a.smithi103.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T06:39:51.660 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T06:39:51.660 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T06:39:51.660 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T06:39:51.660 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T06:39:51.661 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative compaction: 0.00 GB write, 29.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T06:39:51.661 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval compaction: 0.00 GB write, 29.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T06:39:51.661 INFO:journalctl@ceph.mon.a.smithi103.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T06:39:51.661 INFO:journalctl@ceph.mon.a.smithi103.stdout: Block cache BinnedLRUCache@0x5635a2a131f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T06:39:51.661 INFO:journalctl@ceph.mon.a.smithi103.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T06:39:51.661 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-19T06:39:51.661 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T06:39:51.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: starting mon.a rank 0 at public addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] at bind addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:39:51.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: mon.a@-1(???) e1 preinit fsid d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:39:51.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: mon.a@-1(???).mds e1 new map 2024-09-19T06:39:51.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: mon.a@-1(???).mds e1 print_map 2024-09-19T06:39:51.661 INFO:journalctl@ceph.mon.a.smithi103.stdout: e1 2024-09-19T06:39:51.661 INFO:journalctl@ceph.mon.a.smithi103.stdout: btime 2024-09-19T06:39:46:840386+0000 2024-09-19T06:39:51.661 INFO:journalctl@ceph.mon.a.smithi103.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T06:39:51.662 INFO:journalctl@ceph.mon.a.smithi103.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T06:39:51.662 INFO:journalctl@ceph.mon.a.smithi103.stdout: legacy client fscid: -1 2024-09-19T06:39:51.662 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-19T06:39:51.662 INFO:journalctl@ceph.mon.a.smithi103.stdout: No filesystems configured 2024-09-19T06:39:51.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T06:39:51.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:39:51.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:39:51.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:39:51.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T06:39:51.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 bash[31139]: 0b2d620a0814f47adb7b84144bc63a8b715f29fc07a1846a6080b944f3fcd578 2024-09-19T06:39:51.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 systemd[1]: Started Ceph mon.a for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:39:51.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T06:39:51.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: monmap epoch 1 2024-09-19T06:39:51.663 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: fsid d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:39:51.663 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: last_changed 2024-09-19T06:39:43.868601+0000 2024-09-19T06:39:51.663 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: created 2024-09-19T06:39:43.868601+0000 2024-09-19T06:39:51.663 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: min_mon_release 19 (squid) 2024-09-19T06:39:51.663 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: election_strategy: 1 2024-09-19T06:39:51.663 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T06:39:51.663 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: fsmap 2024-09-19T06:39:51.663 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T06:39:51.663 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:51 smithi103 ceph-mon[31158]: mgrmap e1: no daemons active 2024-09-19T06:39:54.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:39:54 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/387739168' entity='client.admin' 2024-09-19T06:40:00.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.383+0000 7fd38fd3a640 1 Processor -- start 2024-09-19T06:40:00.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.383+0000 7fd38fd3a640 1 -- start start 2024-09-19T06:40:00.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.383+0000 7fd38fd3a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd388108310 0x7fd38810a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:00.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.383+0000 7fd38fd3a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd38807be20 con 0x7fd388108310 2024-09-19T06:40:00.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.384+0000 7fd38dad6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd388108310 0x7fd38810a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:00.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.384+0000 7fd38dad6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd388108310 0x7fd38810a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33986/0 (socket says 172.21.15.103:33986) 2024-09-19T06:40:00.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.384+0000 7fd38dad6640 1 -- 172.21.15.103:0/1637825479 learned_addr learned my addr 172.21.15.103:0/1637825479 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:00.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.384+0000 7fd38dad6640 1 -- 172.21.15.103:0/1637825479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38810ac40 con 0x7fd388108310 2024-09-19T06:40:00.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.385+0000 7fd38dad6640 1 --2- 172.21.15.103:0/1637825479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd388108310 0x7fd38810a700 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd384009820 tx=0x7fd38402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617300aa84971cfd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:00.125 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.386+0000 7fd38cad4640 1 -- 172.21.15.103:0/1637825479 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd384037070 con 0x7fd388108310 2024-09-19T06:40:00.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.386+0000 7fd38cad4640 1 -- 172.21.15.103:0/1637825479 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd384032900 con 0x7fd388108310 2024-09-19T06:40:00.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.386+0000 7fd38cad4640 1 -- 172.21.15.103:0/1637825479 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd384030850 con 0x7fd388108310 2024-09-19T06:40:00.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.386+0000 7fd38fd3a640 1 -- 172.21.15.103:0/1637825479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd388108310 msgr2=0x7fd38810a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:00.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.386+0000 7fd38fd3a640 1 --2- 172.21.15.103:0/1637825479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd388108310 0x7fd38810a700 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd384009820 tx=0x7fd38402a270 comp rx=0 tx=0).stop 2024-09-19T06:40:00.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.386+0000 7fd38fd3a640 1 -- 172.21.15.103:0/1637825479 shutdown_connections 2024-09-19T06:40:00.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.386+0000 7fd38fd3a640 1 --2- 172.21.15.103:0/1637825479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd388108310 0x7fd38810a700 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:00.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.386+0000 7fd38fd3a640 1 -- 172.21.15.103:0/1637825479 >> 172.21.15.103:0/1637825479 conn(0x7fd388101f10 msgr2=0x7fd388104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:00.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.386+0000 7fd38fd3a640 1 -- 172.21.15.103:0/1637825479 shutdown_connections 2024-09-19T06:40:00.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.387+0000 7fd38fd3a640 1 -- 172.21.15.103:0/1637825479 wait complete. 2024-09-19T06:40:00.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.387+0000 7fd38fd3a640 1 Processor -- start 2024-09-19T06:40:00.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.387+0000 7fd38fd3a640 1 -- start start 2024-09-19T06:40:00.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.387+0000 7fd38fd3a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd388108310 0x7fd38819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:00.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.387+0000 7fd38fd3a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd38810bc10 con 0x7fd388108310 2024-09-19T06:40:00.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.387+0000 7fd38dad6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd388108310 0x7fd38819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:00.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.387+0000 7fd38dad6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd388108310 0x7fd38819b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33994/0 (socket says 172.21.15.103:33994) 2024-09-19T06:40:00.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.387+0000 7fd38dad6640 1 -- 172.21.15.103:0/387739168 learned_addr learned my addr 172.21.15.103:0/387739168 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:00.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.388+0000 7fd38dad6640 1 -- 172.21.15.103:0/387739168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38819bd90 con 0x7fd388108310 2024-09-19T06:40:00.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.388+0000 7fd38dad6640 1 --2- 172.21.15.103:0/387739168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd388108310 0x7fd38819b850 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd384031000 tx=0x7fd3840326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:00.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.388+0000 7fd37effd640 1 -- 172.21.15.103:0/387739168 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd384042070 con 0x7fd388108310 2024-09-19T06:40:00.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.388+0000 7fd37effd640 1 -- 172.21.15.103:0/387739168 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd384032be0 con 0x7fd388108310 2024-09-19T06:40:00.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.388+0000 7fd37effd640 1 -- 172.21.15.103:0/387739168 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd384037040 con 0x7fd388108310 2024-09-19T06:40:00.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.388+0000 7fd38fd3a640 1 -- 172.21.15.103:0/387739168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38819c080 con 0x7fd388108310 2024-09-19T06:40:00.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.388+0000 7fd38fd3a640 1 -- 172.21.15.103:0/387739168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38819ed10 con 0x7fd388108310 2024-09-19T06:40:00.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.388+0000 7fd37effd640 1 -- 172.21.15.103:0/387739168 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd384045070 con 0x7fd388108310 2024-09-19T06:40:00.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.388+0000 7fd37effd640 1 -- 172.21.15.103:0/387739168 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd384047a90 con 0x7fd388108310 2024-09-19T06:40:00.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.389+0000 7fd38fd3a640 1 -- 172.21.15.103:0/387739168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd388108770 con 0x7fd388108310 2024-09-19T06:40:00.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.391+0000 7fd37effd640 1 -- 172.21.15.103:0/387739168 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd38404c030 con 0x7fd388108310 2024-09-19T06:40:00.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.450+0000 7fd38fd3a640 1 -- 172.21.15.103:0/387739168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fd38819ef80 con 0x7fd388108310 2024-09-19T06:40:00.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.452+0000 7fd37effd640 1 -- 172.21.15.103:0/387739168 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fd3840328a0 con 0x7fd388108310 2024-09-19T06:40:00.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.452+0000 7fd38fd3a640 1 -- 172.21.15.103:0/387739168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd388108310 msgr2=0x7fd38819b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:00.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.452+0000 7fd38fd3a640 1 --2- 172.21.15.103:0/387739168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd388108310 0x7fd38819b850 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd384031000 tx=0x7fd3840326e0 comp rx=0 tx=0).stop 2024-09-19T06:40:00.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.452+0000 7fd38fd3a640 1 -- 172.21.15.103:0/387739168 shutdown_connections 2024-09-19T06:40:00.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.452+0000 7fd38fd3a640 1 --2- 172.21.15.103:0/387739168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd388108310 0x7fd38819b850 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:00.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.452+0000 7fd38fd3a640 1 -- 172.21.15.103:0/387739168 >> 172.21.15.103:0/387739168 conn(0x7fd388101f10 msgr2=0x7fd388102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:00.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.452+0000 7fd38fd3a640 1 -- 172.21.15.103:0/387739168 shutdown_connections 2024-09-19T06:40:00.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:39:53.452+0000 7fd38fd3a640 1 -- 172.21.15.103:0/387739168 wait complete. 2024-09-19T06:40:00.128 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T06:40:00.368 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:40:00.368 INFO:teuthology.orchestra.run.smithi103.stdout:Creating mgr... 2024-09-19T06:40:00.369 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T06:40:00.369 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T06:40:01.345 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d158d934-7651-11ef-bceb-c7b262605968@mgr.x 2024-09-19T06:40:01.345 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d158d934-7651-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-d158d934-7651-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T06:40:01.675 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d158d934-7651-11ef-bceb-c7b262605968.target.wants/ceph-d158d934-7651-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-d158d934-7651-11ef-bceb-c7b262605968@.service. 2024-09-19T06:40:02.232 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T06:40:02.232 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-19T06:40:02.239 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T06:40:02.239 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-19T06:40:02.239 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-19T06:40:02.240 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T06:40:02.248 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T06:40:02.248 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-19T06:40:02.255 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T06:40:02.255 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-19T06:40:02.256 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-19T06:40:02.256 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T06:40:02.256 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr to start... 2024-09-19T06:40:02.256 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr... 2024-09-19T06:40:02.260 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:02 smithi103 systemd[1]: Started Ceph mgr.x for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:40:02.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:02 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:02.258+0000 7fdad1c3a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:40:02.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:02 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:02.319+0000 7fdad1c3a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:40:02.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:02 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:02.468+0000 7fdad1c3a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:40:02.793 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:02 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:02.537+0000 7fdad1c3a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:40:02.793 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:02 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:02.687+0000 7fdad1c3a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:40:03.107 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:02 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3345142750' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T06:40:03.107 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:02 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:02.836+0000 7fdad1c3a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:40:03.354 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T06:40:03.354 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:40:03.354 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:40:03.354 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T06:40:03.354 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T06:40:03.354 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T06:40:03.354 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T06:40:03.354 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:03.354 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T06:40:03.354 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T06:40:03.354 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-09-19T06:40:03.354 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:40:03.355 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T06:40:03.355 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T06:40:03.355 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:40:03.355 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-09-19T06:40:03.355 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T06:40:03.355 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:40:03.355 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T06:40:03.355 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T06:40:03.355 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:03.355 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T06:40:03.355 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:40:03.355 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T06:40:03.355 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T06:40:03.355 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T06:40:03.363 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T06:40:03.363 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T06:40:03.363 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T06:40:03.363 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:03.363 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T06:40:03.363 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T06:40:03.363 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T06:40:03.363 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T06:40:03.363 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T06:40:03.364 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T06:40:03.364 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T06:40:03.364 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T06:40:03.364 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T06:40:03.364 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:03.364 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T06:40:03.364 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:40:03.364 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T06:39:46:840386+0000", 2024-09-19T06:40:03.364 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T06:40:03.364 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T06:40:03.364 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:03.364 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T06:40:03.364 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T06:40:03.365 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T06:40:03.365 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T06:40:03.365 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T06:40:03.365 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T06:40:03.365 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T06:40:03.365 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:40:03.365 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:40:03.365 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:03.365 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T06:40:03.365 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:40:03.365 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T06:39:46.841035+0000", 2024-09-19T06:40:03.365 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:40:03.365 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:03.365 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T06:40:03.366 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:40:03.366 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.724+0000 7fca7f8cd640 1 Processor -- start 2024-09-19T06:40:03.366 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.724+0000 7fca7f8cd640 1 -- start start 2024-09-19T06:40:03.366 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.724+0000 7fca7f8cd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca78106780 0x7fca78106b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:03.366 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.724+0000 7fca7f8cd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca78107150 con 0x7fca78106780 2024-09-19T06:40:03.366 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.725+0000 7fca7d669640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca78106780 0x7fca78106b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:03.366 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.725+0000 7fca7d669640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca78106780 0x7fca78106b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34110/0 (socket says 172.21.15.103:34110) 2024-09-19T06:40:03.366 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.725+0000 7fca7d669640 1 -- 172.21.15.103:0/176889298 learned_addr learned my addr 172.21.15.103:0/176889298 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:03.366 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.725+0000 7fca7d669640 1 -- 172.21.15.103:0/176889298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca78107980 con 0x7fca78106780 2024-09-19T06:40:03.366 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.726+0000 7fca7d669640 1 --2- 172.21.15.103:0/176889298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca78106780 0x7fca78106b80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fca74009820 tx=0x7fca7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1e26654c5e6198 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:03.366 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.726+0000 7fca6ffff640 1 -- 172.21.15.103:0/176889298 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca74037070 con 0x7fca78106780 2024-09-19T06:40:03.366 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.726+0000 7fca6ffff640 1 -- 172.21.15.103:0/176889298 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fca74032900 con 0x7fca78106780 2024-09-19T06:40:03.366 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.726+0000 7fca6ffff640 1 -- 172.21.15.103:0/176889298 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca74030850 con 0x7fca78106780 2024-09-19T06:40:03.367 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.726+0000 7fca7f8cd640 1 -- 172.21.15.103:0/176889298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca78106780 msgr2=0x7fca78106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:03.367 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.726+0000 7fca7f8cd640 1 --2- 172.21.15.103:0/176889298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca78106780 0x7fca78106b80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fca74009820 tx=0x7fca7402a270 comp rx=0 tx=0).stop 2024-09-19T06:40:03.367 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.727+0000 7fca7f8cd640 1 -- 172.21.15.103:0/176889298 shutdown_connections 2024-09-19T06:40:03.367 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.727+0000 7fca7f8cd640 1 --2- 172.21.15.103:0/176889298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca78106780 0x7fca78106b80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:03.367 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.727+0000 7fca7f8cd640 1 -- 172.21.15.103:0/176889298 >> 172.21.15.103:0/176889298 conn(0x7fca78101f10 msgr2=0x7fca78104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:03.367 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.727+0000 7fca7f8cd640 1 -- 172.21.15.103:0/176889298 shutdown_connections 2024-09-19T06:40:03.367 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.727+0000 7fca7f8cd640 1 -- 172.21.15.103:0/176889298 wait complete. 2024-09-19T06:40:03.367 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.727+0000 7fca7f8cd640 1 Processor -- start 2024-09-19T06:40:03.367 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.727+0000 7fca7f8cd640 1 -- start start 2024-09-19T06:40:03.367 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.727+0000 7fca7f8cd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca78106780 0x7fca781919b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:03.367 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.728+0000 7fca7f8cd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca7810ad30 con 0x7fca78106780 2024-09-19T06:40:03.367 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.728+0000 7fca7d669640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca78106780 0x7fca781919b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:03.367 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.728+0000 7fca7d669640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca78106780 0x7fca781919b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34116/0 (socket says 172.21.15.103:34116) 2024-09-19T06:40:03.368 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.728+0000 7fca7d669640 1 -- 172.21.15.103:0/3345142750 learned_addr learned my addr 172.21.15.103:0/3345142750 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:03.368 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.728+0000 7fca7d669640 1 -- 172.21.15.103:0/3345142750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca78191ef0 con 0x7fca78106780 2024-09-19T06:40:03.368 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.728+0000 7fca7d669640 1 --2- 172.21.15.103:0/3345142750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca78106780 0x7fca781919b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fca740097f0 tx=0x7fca74030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:03.368 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.728+0000 7fca6e7fc640 1 -- 172.21.15.103:0/3345142750 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca74042070 con 0x7fca78106780 2024-09-19T06:40:03.368 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.728+0000 7fca6e7fc640 1 -- 172.21.15.103:0/3345142750 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fca74032ca0 con 0x7fca78106780 2024-09-19T06:40:03.368 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.728+0000 7fca7f8cd640 1 -- 172.21.15.103:0/3345142750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca781900e0 con 0x7fca78106780 2024-09-19T06:40:03.368 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.728+0000 7fca6e7fc640 1 -- 172.21.15.103:0/3345142750 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca74037040 con 0x7fca78106780 2024-09-19T06:40:03.368 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.728+0000 7fca7f8cd640 1 -- 172.21.15.103:0/3345142750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca78190620 con 0x7fca78106780 2024-09-19T06:40:03.368 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.729+0000 7fca6e7fc640 1 -- 172.21.15.103:0/3345142750 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fca74045070 con 0x7fca78106780 2024-09-19T06:40:03.368 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.729+0000 7fca6e7fc640 1 -- 172.21.15.103:0/3345142750 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fca74047ae0 con 0x7fca78106780 2024-09-19T06:40:03.368 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.729+0000 7fca7f8cd640 1 -- 172.21.15.103:0/3345142750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca7810aeb0 con 0x7fca78106780 2024-09-19T06:40:03.368 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.731+0000 7fca6e7fc640 1 -- 172.21.15.103:0/3345142750 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fca7403da60 con 0x7fca78106780 2024-09-19T06:40:03.368 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.791+0000 7fca7f8cd640 1 -- 172.21.15.103:0/3345142750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fca781952c0 con 0x7fca78106780 2024-09-19T06:40:03.369 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.792+0000 7fca6e7fc640 1 -- 172.21.15.103:0/3345142750 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1292 (secure 0 0 0) 0x7fca7403d430 con 0x7fca78106780 2024-09-19T06:40:03.369 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.794+0000 7fca7f8cd640 1 -- 172.21.15.103:0/3345142750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca78106780 msgr2=0x7fca781919b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:03.369 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.794+0000 7fca7f8cd640 1 --2- 172.21.15.103:0/3345142750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca78106780 0x7fca781919b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fca740097f0 tx=0x7fca74030ec0 comp rx=0 tx=0).stop 2024-09-19T06:40:03.369 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.794+0000 7fca7f8cd640 1 -- 172.21.15.103:0/3345142750 shutdown_connections 2024-09-19T06:40:03.369 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.794+0000 7fca7f8cd640 1 --2- 172.21.15.103:0/3345142750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca78106780 0x7fca781919b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:03.369 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.794+0000 7fca7f8cd640 1 -- 172.21.15.103:0/3345142750 >> 172.21.15.103:0/3345142750 conn(0x7fca78101f10 msgr2=0x7fca78102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:03.369 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.794+0000 7fca7f8cd640 1 -- 172.21.15.103:0/3345142750 shutdown_connections 2024-09-19T06:40:03.369 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:02.794+0000 7fca7f8cd640 1 -- 172.21.15.103:0/3345142750 wait complete. 2024-09-19T06:40:03.369 INFO:teuthology.orchestra.run.smithi103.stdout:mgr not available, waiting (1/15)... 2024-09-19T06:40:03.387 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:03 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:03.156+0000 7fdad1c3a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:40:03.388 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:03 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:03.216+0000 7fdad1c3a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:40:03.888 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:03 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:03.404+0000 7fdad1c3a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:40:04.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:04 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:04.275+0000 7fdad1c3a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:40:04.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:04 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:04.340+0000 7fdad1c3a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:40:05.302 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:05 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:05.047+0000 7fdad1c3a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:40:05.302 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:05 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:05.161+0000 7fdad1c3a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:40:05.302 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:05 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:05.300+0000 7fdad1c3a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:40:05.579 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:05 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:05.362+0000 7fdad1c3a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:40:05.579 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:05 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:05.514+0000 7fdad1c3a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:40:05.851 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:05 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:05.585+0000 7fdad1c3a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:40:05.851 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:05 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:05.646+0000 7fdad1c3a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:40:05.851 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:05 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:05.809+0000 7fdad1c3a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:40:06.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:05 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1581096876' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T06:40:06.139 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:05 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:05.880+0000 7fdad1c3a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:40:06.139 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:05 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:05.946+0000 7fdad1c3a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:40:06.139 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:06.028+0000 7fdad1c3a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:40:06.139 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:06.089+0000 7fdad1c3a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:40:06.397 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:06.148+0000 7fdad1c3a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:40:06.409 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T06:40:06.409 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:40:06.409 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:40:06.409 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T06:40:06.409 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T06:40:06.409 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T06:40:06.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T06:40:06.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:06.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T06:40:06.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T06:40:06.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-09-19T06:40:06.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:40:06.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T06:40:06.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T06:40:06.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:40:06.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-09-19T06:40:06.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T06:40:06.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:40:06.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T06:40:06.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T06:40:06.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:06.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T06:40:06.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:40:06.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T06:40:06.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T06:40:06.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T06:40:06.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T06:40:06.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T06:40:06.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T06:40:06.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:06.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T06:40:06.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T06:40:06.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T06:40:06.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T06:40:06.412 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T06:40:06.412 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T06:40:06.412 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T06:40:06.412 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T06:40:06.412 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T06:40:06.412 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:06.412 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T06:40:06.412 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:40:06.412 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T06:39:46:840386+0000", 2024-09-19T06:40:06.412 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T06:40:06.412 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T06:40:06.412 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:06.412 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T06:40:06.412 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T06:40:06.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T06:40:06.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T06:40:06.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T06:40:06.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T06:40:06.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T06:40:06.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:40:06.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:40:06.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:06.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T06:40:06.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:40:06.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T06:39:46.841035+0000", 2024-09-19T06:40:06.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:40:06.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:06.413 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T06:40:06.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:40:06.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.778+0000 7f7cf2551640 1 Processor -- start 2024-09-19T06:40:06.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.778+0000 7f7cf2551640 1 -- start start 2024-09-19T06:40:06.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.778+0000 7f7cf2551640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec108310 0x7f7cec10a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:06.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.778+0000 7f7cf2551640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec07be20 con 0x7f7cec108310 2024-09-19T06:40:06.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.778+0000 7f7cebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec108310 0x7f7cec10a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:06.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.778+0000 7f7cebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec108310 0x7f7cec10a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34128/0 (socket says 172.21.15.103:34128) 2024-09-19T06:40:06.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.778+0000 7f7cebfff640 1 -- 172.21.15.103:0/76908408 learned_addr learned my addr 172.21.15.103:0/76908408 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:06.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.779+0000 7f7cebfff640 1 -- 172.21.15.103:0/76908408 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec10ac40 con 0x7f7cec108310 2024-09-19T06:40:06.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.779+0000 7f7cebfff640 1 --2- 172.21.15.103:0/76908408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec108310 0x7f7cec10a700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc009820 tx=0x7f7cdc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4318bbbc0d38c3c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:06.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.780+0000 7f7ceaffd640 1 -- 172.21.15.103:0/76908408 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cdc037070 con 0x7f7cec108310 2024-09-19T06:40:06.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.780+0000 7f7ceaffd640 1 -- 172.21.15.103:0/76908408 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7cdc032900 con 0x7f7cec108310 2024-09-19T06:40:06.414 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.780+0000 7f7ceaffd640 1 -- 172.21.15.103:0/76908408 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cdc030850 con 0x7f7cec108310 2024-09-19T06:40:06.415 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.780+0000 7f7cf2551640 1 -- 172.21.15.103:0/76908408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec108310 msgr2=0x7f7cec10a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:06.415 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.780+0000 7f7cf2551640 1 --2- 172.21.15.103:0/76908408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec108310 0x7f7cec10a700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc009820 tx=0x7f7cdc02a270 comp rx=0 tx=0).stop 2024-09-19T06:40:06.415 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.780+0000 7f7cf2551640 1 -- 172.21.15.103:0/76908408 shutdown_connections 2024-09-19T06:40:06.415 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.780+0000 7f7cf2551640 1 --2- 172.21.15.103:0/76908408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec108310 0x7f7cec10a700 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:06.415 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.780+0000 7f7cf2551640 1 -- 172.21.15.103:0/76908408 >> 172.21.15.103:0/76908408 conn(0x7f7cec101f90 msgr2=0x7f7cec1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:06.415 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.780+0000 7f7cf2551640 1 -- 172.21.15.103:0/76908408 shutdown_connections 2024-09-19T06:40:06.415 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.780+0000 7f7cf2551640 1 -- 172.21.15.103:0/76908408 wait complete. 2024-09-19T06:40:06.415 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.781+0000 7f7cf2551640 1 Processor -- start 2024-09-19T06:40:06.415 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.781+0000 7f7cf2551640 1 -- start start 2024-09-19T06:40:06.415 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.781+0000 7f7cf2551640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec108310 0x7f7cec1a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:06.415 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.781+0000 7f7cf2551640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec10bc10 con 0x7f7cec108310 2024-09-19T06:40:06.415 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.781+0000 7f7cebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec108310 0x7f7cec1a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:06.415 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.782+0000 7f7cebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec108310 0x7f7cec1a1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34130/0 (socket says 172.21.15.103:34130) 2024-09-19T06:40:06.415 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.782+0000 7f7cebfff640 1 -- 172.21.15.103:0/1581096876 learned_addr learned my addr 172.21.15.103:0/1581096876 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:06.416 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.782+0000 7f7cebfff640 1 -- 172.21.15.103:0/1581096876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec1a2380 con 0x7f7cec108310 2024-09-19T06:40:06.416 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.782+0000 7f7cebfff640 1 --2- 172.21.15.103:0/1581096876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec108310 0x7f7cec1a1e40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc031000 tx=0x7f7cdc0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:06.416 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.782+0000 7f7ce97fa640 1 -- 172.21.15.103:0/1581096876 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cdc042070 con 0x7f7cec108310 2024-09-19T06:40:06.416 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.782+0000 7f7ce97fa640 1 -- 172.21.15.103:0/1581096876 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7cdc032be0 con 0x7f7cec108310 2024-09-19T06:40:06.416 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.782+0000 7f7ce97fa640 1 -- 172.21.15.103:0/1581096876 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cdc037040 con 0x7f7cec108310 2024-09-19T06:40:06.416 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.782+0000 7f7cf2551640 1 -- 172.21.15.103:0/1581096876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cec1a2670 con 0x7f7cec108310 2024-09-19T06:40:06.416 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.782+0000 7f7cf2551640 1 -- 172.21.15.103:0/1581096876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cec1a5300 con 0x7f7cec108310 2024-09-19T06:40:06.416 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.782+0000 7f7ce97fa640 1 -- 172.21.15.103:0/1581096876 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7cdc045070 con 0x7f7cec108310 2024-09-19T06:40:06.416 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.782+0000 7f7ce97fa640 1 -- 172.21.15.103:0/1581096876 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7cdc047a80 con 0x7f7cec108310 2024-09-19T06:40:06.416 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.783+0000 7f7cf2551640 1 -- 172.21.15.103:0/1581096876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cec108770 con 0x7f7cec108310 2024-09-19T06:40:06.416 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.785+0000 7f7ce97fa640 1 -- 172.21.15.103:0/1581096876 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7cdc04c030 con 0x7f7cec108310 2024-09-19T06:40:06.416 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.845+0000 7f7cf2551640 1 -- 172.21.15.103:0/1581096876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7cec1a5530 con 0x7f7cec108310 2024-09-19T06:40:06.417 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.845+0000 7f7ce97fa640 1 -- 172.21.15.103:0/1581096876 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1292 (secure 0 0 0) 0x7f7cdc0328a0 con 0x7f7cec108310 2024-09-19T06:40:06.417 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.847+0000 7f7cf2551640 1 -- 172.21.15.103:0/1581096876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec108310 msgr2=0x7f7cec1a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:06.417 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.847+0000 7f7cf2551640 1 --2- 172.21.15.103:0/1581096876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec108310 0x7f7cec1a1e40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc031000 tx=0x7f7cdc0326e0 comp rx=0 tx=0).stop 2024-09-19T06:40:06.417 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.847+0000 7f7cf2551640 1 -- 172.21.15.103:0/1581096876 shutdown_connections 2024-09-19T06:40:06.417 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.847+0000 7f7cf2551640 1 --2- 172.21.15.103:0/1581096876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec108310 0x7f7cec1a1e40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:06.417 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.847+0000 7f7cf2551640 1 -- 172.21.15.103:0/1581096876 >> 172.21.15.103:0/1581096876 conn(0x7f7cec101f90 msgr2=0x7f7cec102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:06.417 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.847+0000 7f7cf2551640 1 -- 172.21.15.103:0/1581096876 shutdown_connections 2024-09-19T06:40:06.417 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:05.847+0000 7f7cf2551640 1 -- 172.21.15.103:0/1581096876 wait complete. 2024-09-19T06:40:06.417 INFO:teuthology.orchestra.run.smithi103.stdout:mgr not available, waiting (2/15)... 2024-09-19T06:40:07.388 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:06.933+0000 7fdad1c3a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:40:07.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-mon[31158]: Activating manager daemon x 2024-09-19T06:40:07.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-mon[31158]: mgrmap e2: x(active, starting, since 0.00321373s) 2024-09-19T06:40:07.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-mon[31158]: from='mgr.14100 172.21.15.103:0/4083752652' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:40:07.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-mon[31158]: from='mgr.14100 172.21.15.103:0/4083752652' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:40:07.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-mon[31158]: from='mgr.14100 172.21.15.103:0/4083752652' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:40:07.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-mon[31158]: from='mgr.14100 172.21.15.103:0/4083752652' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:40:07.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-mon[31158]: from='mgr.14100 172.21.15.103:0/4083752652' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T06:40:07.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-mon[31158]: Manager daemon x is now available 2024-09-19T06:40:07.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-mon[31158]: from='mgr.14100 172.21.15.103:0/4083752652' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:40:07.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-mon[31158]: from='mgr.14100 172.21.15.103:0/4083752652' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T06:40:07.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-mon[31158]: from='mgr.14100 172.21.15.103:0/4083752652' entity='mgr.x' 2024-09-19T06:40:07.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-mon[31158]: from='mgr.14100 172.21.15.103:0/4083752652' entity='mgr.x' 2024-09-19T06:40:07.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:06 smithi103 ceph-mon[31158]: from='mgr.14100 172.21.15.103:0/4083752652' entity='mgr.x' 2024-09-19T06:40:09.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:08 smithi103 ceph-mon[31158]: mgrmap e3: x(active, since 1.00728s) 2024-09-19T06:40:09.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T06:40:09.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:40:09.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:40:09.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T06:40:09.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T06:40:09.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T06:40:09.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T06:40:09.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:09.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T06:40:09.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T06:40:09.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-09-19T06:40:09.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:40:09.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T06:40:09.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T06:40:09.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:40:09.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-09-19T06:40:09.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T06:40:09.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:40:09.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T06:40:09.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T06:40:09.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:09.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T06:40:09.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:40:09.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T06:40:09.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T06:40:09.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T06:40:09.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T06:40:09.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T06:40:09.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T06:40:09.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:09.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T06:40:09.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T06:40:09.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T06:40:09.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T06:40:09.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T06:40:09.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T06:40:09.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T06:40:09.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T06:40:09.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T06:40:09.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:09.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T06:40:09.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:40:09.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T06:39:46:840386+0000", 2024-09-19T06:40:09.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T06:40:09.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T06:40:09.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:09.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T06:40:09.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T06:40:09.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T06:40:09.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T06:40:09.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T06:40:09.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T06:40:09.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T06:40:09.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:40:09.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:40:09.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:09.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T06:40:09.825 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:40:09.825 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T06:39:46.841035+0000", 2024-09-19T06:40:09.825 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:40:09.825 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:40:09.825 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T06:40:09.825 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:40:09.825 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.024+0000 7f551aa3b640 1 Processor -- start 2024-09-19T06:40:09.825 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.024+0000 7f551aa3b640 1 -- start start 2024-09-19T06:40:09.825 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.024+0000 7f551aa3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f551407dde0 0x7f551407c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:09.825 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.024+0000 7f551aa3b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f551407c7d0 con 0x7f551407dde0 2024-09-19T06:40:09.825 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.024+0000 7f5513fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f551407dde0 0x7f551407c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:09.825 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.024+0000 7f5513fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f551407dde0 0x7f551407c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40450/0 (socket says 172.21.15.103:40450) 2024-09-19T06:40:09.826 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.024+0000 7f5513fff640 1 -- 172.21.15.103:0/2702776611 learned_addr learned my addr 172.21.15.103:0/2702776611 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:09.826 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.024+0000 7f5513fff640 1 -- 172.21.15.103:0/2702776611 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551407c950 con 0x7f551407dde0 2024-09-19T06:40:09.826 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.025+0000 7f5513fff640 1 --2- 172.21.15.103:0/2702776611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f551407dde0 0x7f551407c290 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5504009820 tx=0x7f550402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4204ddcf12ea5527 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:09.826 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.025+0000 7f5512ffd640 1 -- 172.21.15.103:0/2702776611 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5504037070 con 0x7f551407dde0 2024-09-19T06:40:09.826 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.025+0000 7f5512ffd640 1 -- 172.21.15.103:0/2702776611 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5504032900 con 0x7f551407dde0 2024-09-19T06:40:09.826 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.025+0000 7f5512ffd640 1 -- 172.21.15.103:0/2702776611 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55040308e0 con 0x7f551407dde0 2024-09-19T06:40:09.826 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.026+0000 7f551aa3b640 1 -- 172.21.15.103:0/2702776611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f551407dde0 msgr2=0x7f551407c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:09.826 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.026+0000 7f551aa3b640 1 --2- 172.21.15.103:0/2702776611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f551407dde0 0x7f551407c290 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5504009820 tx=0x7f550402a300 comp rx=0 tx=0).stop 2024-09-19T06:40:09.826 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.026+0000 7f551aa3b640 1 -- 172.21.15.103:0/2702776611 shutdown_connections 2024-09-19T06:40:09.826 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.026+0000 7f551aa3b640 1 --2- 172.21.15.103:0/2702776611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f551407dde0 0x7f551407c290 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:09.826 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.026+0000 7f551aa3b640 1 -- 172.21.15.103:0/2702776611 >> 172.21.15.103:0/2702776611 conn(0x7f5514103080 msgr2=0x7f55141054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:09.826 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.026+0000 7f551aa3b640 1 -- 172.21.15.103:0/2702776611 shutdown_connections 2024-09-19T06:40:09.826 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.026+0000 7f551aa3b640 1 -- 172.21.15.103:0/2702776611 wait complete. 2024-09-19T06:40:09.827 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.027+0000 7f551aa3b640 1 Processor -- start 2024-09-19T06:40:09.827 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.027+0000 7f551aa3b640 1 -- start start 2024-09-19T06:40:09.827 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.027+0000 7f551aa3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f551407dde0 0x7f551419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:09.827 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.027+0000 7f551aa3b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f551410dee0 con 0x7f551407dde0 2024-09-19T06:40:09.827 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.027+0000 7f5513fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f551407dde0 0x7f551419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:09.827 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.027+0000 7f5513fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f551407dde0 0x7f551419dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40462/0 (socket says 172.21.15.103:40462) 2024-09-19T06:40:09.827 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.027+0000 7f5513fff640 1 -- 172.21.15.103:0/1474500352 learned_addr learned my addr 172.21.15.103:0/1474500352 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:09.827 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.027+0000 7f5513fff640 1 -- 172.21.15.103:0/1474500352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551419e010 con 0x7f551407dde0 2024-09-19T06:40:09.827 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.028+0000 7f5513fff640 1 --2- 172.21.15.103:0/1474500352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f551407dde0 0x7f551419dad0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5504030610 tx=0x7f5504032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:09.827 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.028+0000 7f55117fa640 1 -- 172.21.15.103:0/1474500352 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5504037040 con 0x7f551407dde0 2024-09-19T06:40:09.827 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.028+0000 7f55117fa640 1 -- 172.21.15.103:0/1474500352 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5504040070 con 0x7f551407dde0 2024-09-19T06:40:09.827 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.028+0000 7f551aa3b640 1 -- 172.21.15.103:0/1474500352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551419e2a0 con 0x7f551407dde0 2024-09-19T06:40:09.828 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.028+0000 7f55117fa640 1 -- 172.21.15.103:0/1474500352 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f550403a390 con 0x7f551407dde0 2024-09-19T06:40:09.828 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.028+0000 7f551aa3b640 1 -- 172.21.15.103:0/1474500352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55141a0f90 con 0x7f551407dde0 2024-09-19T06:40:09.828 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.029+0000 7f55117fa640 1 -- 172.21.15.103:0/1474500352 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f5504032900 con 0x7f551407dde0 2024-09-19T06:40:09.828 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.029+0000 7f55117fa640 1 --2- 172.21.15.103:0/1474500352 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f54fc039160 0x7f54fc03b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:09.828 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.029+0000 7f551aa3b640 1 -- 172.21.15.103:0/1474500352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f551407c310 con 0x7f551407dde0 2024-09-19T06:40:09.828 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.029+0000 7f55117fa640 1 -- 172.21.15.103:0/1474500352 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f550406c140 con 0x7f551407dde0 2024-09-19T06:40:09.828 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.029+0000 7f55137fe640 1 --2- 172.21.15.103:0/1474500352 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f54fc039160 0x7f54fc03b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:09.828 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.029+0000 7f55137fe640 1 --2- 172.21.15.103:0/1474500352 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f54fc039160 0x7f54fc03b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f55080098c0 tx=0x7f5508006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:09.828 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.034+0000 7f55117fa640 1 -- 172.21.15.103:0/1474500352 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f550406bea0 con 0x7f551407dde0 2024-09-19T06:40:09.828 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.268+0000 7f551aa3b640 1 -- 172.21.15.103:0/1474500352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f55141a1560 con 0x7f551407dde0 2024-09-19T06:40:09.828 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.268+0000 7f55117fa640 1 -- 172.21.15.103:0/1474500352 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f550403b5a0 con 0x7f551407dde0 2024-09-19T06:40:09.828 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.270+0000 7f551aa3b640 1 -- 172.21.15.103:0/1474500352 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f54fc039160 msgr2=0x7f54fc03b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:09.828 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.270+0000 7f551aa3b640 1 --2- 172.21.15.103:0/1474500352 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f54fc039160 0x7f54fc03b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f55080098c0 tx=0x7f5508006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:09.829 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.270+0000 7f551aa3b640 1 -- 172.21.15.103:0/1474500352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f551407dde0 msgr2=0x7f551419dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:09.829 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.270+0000 7f551aa3b640 1 --2- 172.21.15.103:0/1474500352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f551407dde0 0x7f551419dad0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5504030610 tx=0x7f5504032670 comp rx=0 tx=0).stop 2024-09-19T06:40:09.829 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.270+0000 7f551aa3b640 1 -- 172.21.15.103:0/1474500352 shutdown_connections 2024-09-19T06:40:09.829 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.270+0000 7f551aa3b640 1 --2- 172.21.15.103:0/1474500352 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f54fc039160 0x7f54fc03b620 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:09.829 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.270+0000 7f551aa3b640 1 --2- 172.21.15.103:0/1474500352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f551407dde0 0x7f551419dad0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:09.829 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.270+0000 7f551aa3b640 1 -- 172.21.15.103:0/1474500352 >> 172.21.15.103:0/1474500352 conn(0x7f5514103080 msgr2=0x7f5514103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:09.829 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.270+0000 7f551aa3b640 1 -- 172.21.15.103:0/1474500352 shutdown_connections 2024-09-19T06:40:09.829 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:09.270+0000 7f551aa3b640 1 -- 172.21.15.103:0/1474500352 wait complete. 2024-09-19T06:40:09.829 INFO:teuthology.orchestra.run.smithi103.stdout:mgr is available 2024-09-19T06:40:10.071 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:09 smithi103 ceph-mon[31158]: mgrmap e4: x(active, since 2s) 2024-09-19T06:40:10.071 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:09 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1474500352' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T06:40:10.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T06:40:10.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T06:40:10.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout fsid = d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:40:10.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-09-19T06:40:10.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T06:40:10.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T06:40:10.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T06:40:10.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T06:40:10.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T06:40:10.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T06:40:10.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T06:40:10.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T06:40:10.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T06:40:10.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T06:40:10.969 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T06:40:10.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T06:40:10.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T06:40:10.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T06:40:10.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.239+0000 7f2ad6576640 1 Processor -- start 2024-09-19T06:40:10.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.239+0000 7f2ad6576640 1 -- start start 2024-09-19T06:40:10.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.239+0000 7f2ad6576640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0106770 0x7f2ad0106b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:10.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.239+0000 7f2ad6576640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad0107140 con 0x7f2ad0106770 2024-09-19T06:40:10.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.239+0000 7f2acffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0106770 0x7f2ad0106b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:10.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.239+0000 7f2acffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0106770 0x7f2ad0106b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40470/0 (socket says 172.21.15.103:40470) 2024-09-19T06:40:10.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.239+0000 7f2acffff640 1 -- 172.21.15.103:0/1749345191 learned_addr learned my addr 172.21.15.103:0/1749345191 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:10.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.239+0000 7f2acffff640 1 -- 172.21.15.103:0/1749345191 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad0107970 con 0x7f2ad0106770 2024-09-19T06:40:10.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.240+0000 7f2acffff640 1 --2- 172.21.15.103:0/1749345191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0106770 0x7f2ad0106b70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009820 tx=0x7f2ac002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2406b48fdf28f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:10.970 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.240+0000 7f2aceffd640 1 -- 172.21.15.103:0/1749345191 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0037070 con 0x7f2ad0106770 2024-09-19T06:40:10.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.240+0000 7f2aceffd640 1 -- 172.21.15.103:0/1749345191 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2ac0032900 con 0x7f2ad0106770 2024-09-19T06:40:10.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.240+0000 7f2aceffd640 1 -- 172.21.15.103:0/1749345191 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0030850 con 0x7f2ad0106770 2024-09-19T06:40:10.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.241+0000 7f2ad6576640 1 -- 172.21.15.103:0/1749345191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0106770 msgr2=0x7f2ad0106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:10.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.241+0000 7f2ad6576640 1 --2- 172.21.15.103:0/1749345191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0106770 0x7f2ad0106b70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009820 tx=0x7f2ac002a270 comp rx=0 tx=0).stop 2024-09-19T06:40:10.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.241+0000 7f2ad6576640 1 -- 172.21.15.103:0/1749345191 shutdown_connections 2024-09-19T06:40:10.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.241+0000 7f2ad6576640 1 --2- 172.21.15.103:0/1749345191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0106770 0x7f2ad0106b70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:10.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.241+0000 7f2ad6576640 1 -- 172.21.15.103:0/1749345191 >> 172.21.15.103:0/1749345191 conn(0x7f2ad0101f40 msgr2=0x7f2ad0104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:10.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.241+0000 7f2ad6576640 1 -- 172.21.15.103:0/1749345191 shutdown_connections 2024-09-19T06:40:10.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.241+0000 7f2ad6576640 1 -- 172.21.15.103:0/1749345191 wait complete. 2024-09-19T06:40:10.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.241+0000 7f2ad6576640 1 Processor -- start 2024-09-19T06:40:10.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.241+0000 7f2ad6576640 1 -- start start 2024-09-19T06:40:10.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.242+0000 7f2ad6576640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0106770 0x7f2ad019b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:10.971 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.242+0000 7f2ad6576640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad010ad20 con 0x7f2ad0106770 2024-09-19T06:40:10.972 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.242+0000 7f2acffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0106770 0x7f2ad019b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:10.972 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.242+0000 7f2acffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0106770 0x7f2ad019b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40476/0 (socket says 172.21.15.103:40476) 2024-09-19T06:40:10.972 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.242+0000 7f2acffff640 1 -- 172.21.15.103:0/193718743 learned_addr learned my addr 172.21.15.103:0/193718743 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:10.972 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.242+0000 7f2acffff640 1 -- 172.21.15.103:0/193718743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad019bda0 con 0x7f2ad0106770 2024-09-19T06:40:10.972 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.242+0000 7f2acffff640 1 --2- 172.21.15.103:0/193718743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0106770 0x7f2ad019b860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009940 tx=0x7f2ac0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:10.972 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.242+0000 7f2acd7fa640 1 -- 172.21.15.103:0/193718743 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0042070 con 0x7f2ad0106770 2024-09-19T06:40:10.972 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.242+0000 7f2ad6576640 1 -- 172.21.15.103:0/193718743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad019c030 con 0x7f2ad0106770 2024-09-19T06:40:10.972 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.242+0000 7f2acd7fa640 1 -- 172.21.15.103:0/193718743 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2ac00308c0 con 0x7f2ad0106770 2024-09-19T06:40:10.972 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.242+0000 7f2acd7fa640 1 -- 172.21.15.103:0/193718743 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0037040 con 0x7f2ad0106770 2024-09-19T06:40:10.972 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.243+0000 7f2ad6576640 1 -- 172.21.15.103:0/193718743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad019ed20 con 0x7f2ad0106770 2024-09-19T06:40:10.972 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.243+0000 7f2acd7fa640 1 -- 172.21.15.103:0/193718743 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f2ac0045070 con 0x7f2ad0106770 2024-09-19T06:40:10.972 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.243+0000 7f2acd7fa640 1 --2- 172.21.15.103:0/193718743 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f2abc039160 0x7f2abc03b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:10.972 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.243+0000 7f2acd7fa640 1 -- 172.21.15.103:0/193718743 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2ac006c220 con 0x7f2ad0106770 2024-09-19T06:40:10.972 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.243+0000 7f2ad6576640 1 -- 172.21.15.103:0/193718743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad010aea0 con 0x7f2ad0106770 2024-09-19T06:40:10.973 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.244+0000 7f2acf7fe640 1 --2- 172.21.15.103:0/193718743 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f2abc039160 0x7f2abc03b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:10.973 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.244+0000 7f2acf7fe640 1 --2- 172.21.15.103:0/193718743 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f2abc039160 0x7f2abc03b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80098c0 tx=0x7f2ab8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:10.973 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.248+0000 7f2acd7fa640 1 -- 172.21.15.103:0/193718743 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2ac003cd90 con 0x7f2ad0106770 2024-09-19T06:40:10.973 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.416+0000 7f2ad6576640 1 -- 172.21.15.103:0/193718743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f2ad0106bf0 con 0x7f2ad0106770 2024-09-19T06:40:10.973 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.417+0000 7f2acd7fa640 1 -- 172.21.15.103:0/193718743 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f2ac003c870 con 0x7f2ad0106770 2024-09-19T06:40:10.973 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.419+0000 7f2ad6576640 1 -- 172.21.15.103:0/193718743 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f2abc039160 msgr2=0x7f2abc03b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:10.973 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.419+0000 7f2ad6576640 1 --2- 172.21.15.103:0/193718743 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f2abc039160 0x7f2abc03b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80098c0 tx=0x7f2ab8006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:10.973 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.419+0000 7f2ad6576640 1 -- 172.21.15.103:0/193718743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0106770 msgr2=0x7f2ad019b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:10.973 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.419+0000 7f2ad6576640 1 --2- 172.21.15.103:0/193718743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0106770 0x7f2ad019b860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009940 tx=0x7f2ac0032ce0 comp rx=0 tx=0).stop 2024-09-19T06:40:10.973 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.419+0000 7f2ad6576640 1 -- 172.21.15.103:0/193718743 shutdown_connections 2024-09-19T06:40:10.973 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.419+0000 7f2ad6576640 1 --2- 172.21.15.103:0/193718743 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f2abc039160 0x7f2abc03b620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:10.973 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.419+0000 7f2ad6576640 1 --2- 172.21.15.103:0/193718743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ad0106770 0x7f2ad019b860 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:10.973 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.419+0000 7f2ad6576640 1 -- 172.21.15.103:0/193718743 >> 172.21.15.103:0/193718743 conn(0x7f2ad0101f40 msgr2=0x7f2ad0102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:10.974 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.419+0000 7f2ad6576640 1 -- 172.21.15.103:0/193718743 shutdown_connections 2024-09-19T06:40:10.974 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:10.419+0000 7f2ad6576640 1 -- 172.21.15.103:0/193718743 wait complete. 2024-09-19T06:40:10.974 INFO:teuthology.orchestra.run.smithi103.stdout:Enabling cephadm module... 2024-09-19T06:40:11.076 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:10 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/193718743' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T06:40:12.257 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:11 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1576528002' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T06:40:12.258 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:11 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: ignoring --setuser ceph since I am not root 2024-09-19T06:40:12.258 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:11 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: ignoring --setgroup ceph since I am not root 2024-09-19T06:40:12.258 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:12.089+0000 7ff59c71a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:40:12.258 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:12.145+0000 7ff59c71a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:40:12.515 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:12.299+0000 7ff59c71a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:40:12.515 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:12.366+0000 7ff59c71a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:40:12.515 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:12.514+0000 7ff59c71a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:40:12.561 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.350+0000 7f49d8997640 1 Processor -- start 2024-09-19T06:40:12.561 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.350+0000 7f49d8997640 1 -- start start 2024-09-19T06:40:12.561 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.350+0000 7f49d8997640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49d41078d0 0x7f49d4107cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:12.561 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.350+0000 7f49d8997640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f49d41082a0 con 0x7f49d41078d0 2024-09-19T06:40:12.561 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.350+0000 7f49d259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49d41078d0 0x7f49d4107cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:12.561 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.350+0000 7f49d259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49d41078d0 0x7f49d4107cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40488/0 (socket says 172.21.15.103:40488) 2024-09-19T06:40:12.561 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.350+0000 7f49d259c640 1 -- 172.21.15.103:0/3618627591 learned_addr learned my addr 172.21.15.103:0/3618627591 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:12.561 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.351+0000 7f49d259c640 1 -- 172.21.15.103:0/3618627591 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d4108a80 con 0x7f49d41078d0 2024-09-19T06:40:12.561 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.351+0000 7f49d259c640 1 --2- 172.21.15.103:0/3618627591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49d41078d0 0x7f49d4107cd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009820 tx=0x7f49c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf38eb3f66f236b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:12.562 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.351+0000 7f49d159a640 1 -- 172.21.15.103:0/3618627591 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c4037070 con 0x7f49d41078d0 2024-09-19T06:40:12.562 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.352+0000 7f49d159a640 1 -- 172.21.15.103:0/3618627591 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f49c4032900 con 0x7f49d41078d0 2024-09-19T06:40:12.562 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.352+0000 7f49d159a640 1 -- 172.21.15.103:0/3618627591 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c4030850 con 0x7f49d41078d0 2024-09-19T06:40:12.562 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.352+0000 7f49d8997640 1 -- 172.21.15.103:0/3618627591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49d41078d0 msgr2=0x7f49d4107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:12.562 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.352+0000 7f49d8997640 1 --2- 172.21.15.103:0/3618627591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49d41078d0 0x7f49d4107cd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009820 tx=0x7f49c402a270 comp rx=0 tx=0).stop 2024-09-19T06:40:12.562 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.352+0000 7f49d8997640 1 -- 172.21.15.103:0/3618627591 shutdown_connections 2024-09-19T06:40:12.562 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.352+0000 7f49d8997640 1 --2- 172.21.15.103:0/3618627591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49d41078d0 0x7f49d4107cd0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:12.562 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.352+0000 7f49d8997640 1 -- 172.21.15.103:0/3618627591 >> 172.21.15.103:0/3618627591 conn(0x7f49d4103080 msgr2=0x7f49d41054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:12.562 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.352+0000 7f49d8997640 1 -- 172.21.15.103:0/3618627591 shutdown_connections 2024-09-19T06:40:12.562 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.352+0000 7f49d8997640 1 -- 172.21.15.103:0/3618627591 wait complete. 2024-09-19T06:40:12.562 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.353+0000 7f49d8997640 1 Processor -- start 2024-09-19T06:40:12.562 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.353+0000 7f49d8997640 1 -- start start 2024-09-19T06:40:12.563 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.353+0000 7f49d8997640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49d41078d0 0x7f49d419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:12.563 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.353+0000 7f49d8997640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f49d410be30 con 0x7f49d41078d0 2024-09-19T06:40:12.563 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.353+0000 7f49d259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49d41078d0 0x7f49d419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:12.563 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.353+0000 7f49d259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49d41078d0 0x7f49d419b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40492/0 (socket says 172.21.15.103:40492) 2024-09-19T06:40:12.563 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.353+0000 7f49d259c640 1 -- 172.21.15.103:0/1576528002 learned_addr learned my addr 172.21.15.103:0/1576528002 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:12.563 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.354+0000 7f49d259c640 1 -- 172.21.15.103:0/1576528002 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d419be30 con 0x7f49d41078d0 2024-09-19T06:40:12.563 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.354+0000 7f49d259c640 1 --2- 172.21.15.103:0/1576528002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49d41078d0 0x7f49d419b8f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009940 tx=0x7f49c40326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:12.563 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.354+0000 7f49c37fe640 1 -- 172.21.15.103:0/1576528002 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c4042070 con 0x7f49d41078d0 2024-09-19T06:40:12.563 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.354+0000 7f49c37fe640 1 -- 172.21.15.103:0/1576528002 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f49c4032be0 con 0x7f49d41078d0 2024-09-19T06:40:12.563 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.354+0000 7f49d8997640 1 -- 172.21.15.103:0/1576528002 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d419c0c0 con 0x7f49d41078d0 2024-09-19T06:40:12.563 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.354+0000 7f49c37fe640 1 -- 172.21.15.103:0/1576528002 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c4037040 con 0x7f49d41078d0 2024-09-19T06:40:12.563 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.354+0000 7f49d8997640 1 -- 172.21.15.103:0/1576528002 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d419edb0 con 0x7f49d41078d0 2024-09-19T06:40:12.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.355+0000 7f49c37fe640 1 -- 172.21.15.103:0/1576528002 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f49c4045070 con 0x7f49d41078d0 2024-09-19T06:40:12.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.355+0000 7f49c37fe640 1 --2- 172.21.15.103:0/1576528002 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f49bc039160 0x7f49bc03b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:12.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.355+0000 7f49c37fe640 1 -- 172.21.15.103:0/1576528002 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f49c406d5f0 con 0x7f49d41078d0 2024-09-19T06:40:12.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.355+0000 7f49d1d9b640 1 --2- 172.21.15.103:0/1576528002 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f49bc039160 0x7f49bc03b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:12.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.355+0000 7f49d8997640 1 -- 172.21.15.103:0/1576528002 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d410bfb0 con 0x7f49d41078d0 2024-09-19T06:40:12.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.356+0000 7f49d1d9b640 1 --2- 172.21.15.103:0/1576528002 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f49bc039160 0x7f49bc03b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f49c80098c0 tx=0x7f49c8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:12.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.360+0000 7f49c37fe640 1 -- 172.21.15.103:0/1576528002 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f49c404b170 con 0x7f49d41078d0 2024-09-19T06:40:12.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.570+0000 7f49d8997640 1 -- 172.21.15.103:0/1576528002 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f49d419f280 con 0x7f49d41078d0 2024-09-19T06:40:12.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.945+0000 7f49c37fe640 1 -- 172.21.15.103:0/1576528002 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f49c406d010 con 0x7f49d41078d0 2024-09-19T06:40:12.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.947+0000 7f49c37fe640 1 -- 172.21.15.103:0/1576528002 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f49c4042d00 con 0x7f49d41078d0 2024-09-19T06:40:12.564 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.948+0000 7f49d8997640 1 -- 172.21.15.103:0/1576528002 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f49bc039160 msgr2=0x7f49bc03b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:12.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.948+0000 7f49d8997640 1 --2- 172.21.15.103:0/1576528002 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f49bc039160 0x7f49bc03b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f49c80098c0 tx=0x7f49c8006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:12.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.948+0000 7f49d8997640 1 -- 172.21.15.103:0/1576528002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49d41078d0 msgr2=0x7f49d419b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:12.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.948+0000 7f49d8997640 1 --2- 172.21.15.103:0/1576528002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49d41078d0 0x7f49d419b8f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009940 tx=0x7f49c40326e0 comp rx=0 tx=0).stop 2024-09-19T06:40:12.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.948+0000 7f49d8997640 1 -- 172.21.15.103:0/1576528002 shutdown_connections 2024-09-19T06:40:12.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.948+0000 7f49d8997640 1 --2- 172.21.15.103:0/1576528002 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f49bc039160 0x7f49bc03b620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:12.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.948+0000 7f49d8997640 1 --2- 172.21.15.103:0/1576528002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49d41078d0 0x7f49d419b8f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:12.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.948+0000 7f49d8997640 1 -- 172.21.15.103:0/1576528002 >> 172.21.15.103:0/1576528002 conn(0x7f49d4103080 msgr2=0x7f49d4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:12.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.948+0000 7f49d8997640 1 -- 172.21.15.103:0/1576528002 shutdown_connections 2024-09-19T06:40:12.565 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:11.948+0000 7f49d8997640 1 -- 172.21.15.103:0/1576528002 wait complete. 2024-09-19T06:40:12.798 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:12.666+0000 7ff59c71a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:40:13.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:12 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1576528002' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T06:40:13.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:12 smithi103 ceph-mon[31158]: mgrmap e5: x(active, since 5s) 2024-09-19T06:40:13.054 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:12.989+0000 7ff59c71a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:40:13.388 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:13 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:13.052+0000 7ff59c71a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:40:13.388 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:13 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:13.245+0000 7ff59c71a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:40:14.116 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:40:14.116 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T06:40:14.116 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T06:40:14.116 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T06:40:14.116 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T06:40:14.116 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:40:14.116 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.971+0000 7f7773279640 1 Processor -- start 2024-09-19T06:40:14.117 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.971+0000 7f7773279640 1 -- start start 2024-09-19T06:40:14.117 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.971+0000 7f7773279640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f776c106780 0x7f776c106b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:14.117 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.971+0000 7f7773279640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f776c107150 con 0x7f776c106780 2024-09-19T06:40:14.117 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.971+0000 7f7771015640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f776c106780 0x7f776c106b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:14.117 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.971+0000 7f7771015640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f776c106780 0x7f776c106b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40512/0 (socket says 172.21.15.103:40512) 2024-09-19T06:40:14.117 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.971+0000 7f7771015640 1 -- 172.21.15.103:0/2150957027 learned_addr learned my addr 172.21.15.103:0/2150957027 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:14.117 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.972+0000 7f7771015640 1 -- 172.21.15.103:0/2150957027 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c107980 con 0x7f776c106780 2024-09-19T06:40:14.117 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.972+0000 7f7771015640 1 --2- 172.21.15.103:0/2150957027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f776c106780 0x7f776c106b80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f776800bac0 tx=0x7f776802c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e1c293f1d6be6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:14.117 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.973+0000 7f7763fff640 1 -- 172.21.15.103:0/2150957027 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77680326d0 con 0x7f776c106780 2024-09-19T06:40:14.118 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.973+0000 7f7763fff640 1 -- 172.21.15.103:0/2150957027 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7768038070 con 0x7f776c106780 2024-09-19T06:40:14.118 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.973+0000 7f7763fff640 1 -- 172.21.15.103:0/2150957027 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7768033d70 con 0x7f776c106780 2024-09-19T06:40:14.118 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.974+0000 7f7773279640 1 -- 172.21.15.103:0/2150957027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f776c106780 msgr2=0x7f776c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:14.118 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.974+0000 7f7773279640 1 --2- 172.21.15.103:0/2150957027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f776c106780 0x7f776c106b80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f776800bac0 tx=0x7f776802c920 comp rx=0 tx=0).stop 2024-09-19T06:40:14.118 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.974+0000 7f7773279640 1 -- 172.21.15.103:0/2150957027 shutdown_connections 2024-09-19T06:40:14.118 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.974+0000 7f7773279640 1 --2- 172.21.15.103:0/2150957027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f776c106780 0x7f776c106b80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:14.118 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.974+0000 7f7773279640 1 -- 172.21.15.103:0/2150957027 >> 172.21.15.103:0/2150957027 conn(0x7f776c101f10 msgr2=0x7f776c104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:14.118 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.974+0000 7f7773279640 1 -- 172.21.15.103:0/2150957027 shutdown_connections 2024-09-19T06:40:14.118 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.974+0000 7f7773279640 1 -- 172.21.15.103:0/2150957027 wait complete. 2024-09-19T06:40:14.118 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.974+0000 7f7773279640 1 Processor -- start 2024-09-19T06:40:14.118 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.974+0000 7f7773279640 1 -- start start 2024-09-19T06:40:14.118 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.974+0000 7f7773279640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f776c106780 0x7f776c199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:14.118 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.974+0000 7f7773279640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f776c07b0e0 con 0x7f776c106780 2024-09-19T06:40:14.119 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.975+0000 7f7771015640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f776c106780 0x7f776c199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:14.119 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.975+0000 7f7771015640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f776c106780 0x7f776c199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40516/0 (socket says 172.21.15.103:40516) 2024-09-19T06:40:14.119 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.975+0000 7f7771015640 1 -- 172.21.15.103:0/4086221455 learned_addr learned my addr 172.21.15.103:0/4086221455 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:14.119 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.975+0000 7f7771015640 1 -- 172.21.15.103:0/4086221455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c199b90 con 0x7f776c106780 2024-09-19T06:40:14.119 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.975+0000 7f7771015640 1 --2- 172.21.15.103:0/4086221455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f776c106780 0x7f776c199650 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f776800b2c0 tx=0x7f77680327d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:14.119 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.975+0000 7f77627fc640 1 -- 172.21.15.103:0/4086221455 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7768042070 con 0x7f776c106780 2024-09-19T06:40:14.119 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.975+0000 7f77627fc640 1 -- 172.21.15.103:0/4086221455 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7768032b70 con 0x7f776c106780 2024-09-19T06:40:14.119 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.975+0000 7f7773279640 1 -- 172.21.15.103:0/4086221455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776c199e20 con 0x7f776c106780 2024-09-19T06:40:14.119 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.975+0000 7f77627fc640 1 -- 172.21.15.103:0/4086221455 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f776803c580 con 0x7f776c106780 2024-09-19T06:40:14.119 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.975+0000 7f7773279640 1 -- 172.21.15.103:0/4086221455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776c19cb10 con 0x7f776c106780 2024-09-19T06:40:14.119 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.976+0000 7f77627fc640 1 -- 172.21.15.103:0/4086221455 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f776803c880 con 0x7f776c106780 2024-09-19T06:40:14.119 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.976+0000 7f7773279640 1 -- 172.21.15.103:0/4086221455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f776c106b80 con 0x7f776c106780 2024-09-19T06:40:14.119 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.976+0000 7f77627fc640 1 --2- 172.21.15.103:0/4086221455 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f77580391b0 0x7f775803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:14.120 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.976+0000 7f77627fc640 1 -- 172.21.15.103:0/4086221455 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7768030050 con 0x7f776c106780 2024-09-19T06:40:14.120 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.976+0000 7f7770814640 1 -- 172.21.15.103:0/4086221455 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f77580391b0 msgr2=0x7f775803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2003588522 2024-09-19T06:40:14.120 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.976+0000 7f7770814640 1 --2- 172.21.15.103:0/4086221455 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f77580391b0 0x7f775803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:40:14.120 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:12.981+0000 7f77627fc640 1 -- 172.21.15.103:0/4086221455 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f776803cb70 con 0x7f776c106780 2024-09-19T06:40:14.120 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:13.177+0000 7f7770814640 1 -- 172.21.15.103:0/4086221455 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f77580391b0 msgr2=0x7f775803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2003588522 2024-09-19T06:40:14.120 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:13.177+0000 7f7770814640 1 --2- 172.21.15.103:0/4086221455 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f77580391b0 0x7f775803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:40:14.120 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:13.203+0000 7f7773279640 1 -- 172.21.15.103:0/4086221455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f776c192b10 con 0x7f776c106780 2024-09-19T06:40:14.120 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:13.203+0000 7f77627fc640 1 -- 172.21.15.103:0/4086221455 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f7768032d20 con 0x7f776c106780 2024-09-19T06:40:14.120 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:13.205+0000 7f7773279640 1 -- 172.21.15.103:0/4086221455 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f77580391b0 msgr2=0x7f775803b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:40:14.120 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:13.205+0000 7f7773279640 1 --2- 172.21.15.103:0/4086221455 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f77580391b0 0x7f775803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:14.120 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:13.205+0000 7f7773279640 1 -- 172.21.15.103:0/4086221455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f776c106780 msgr2=0x7f776c199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:14.120 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:13.205+0000 7f7773279640 1 --2- 172.21.15.103:0/4086221455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f776c106780 0x7f776c199650 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f776800b2c0 tx=0x7f77680327d0 comp rx=0 tx=0).stop 2024-09-19T06:40:14.120 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:13.205+0000 7f7773279640 1 -- 172.21.15.103:0/4086221455 shutdown_connections 2024-09-19T06:40:14.121 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:13.205+0000 7f7773279640 1 --2- 172.21.15.103:0/4086221455 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f77580391b0 0x7f775803b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:14.121 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:13.205+0000 7f7773279640 1 --2- 172.21.15.103:0/4086221455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f776c106780 0x7f776c199650 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:14.121 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:13.205+0000 7f7773279640 1 -- 172.21.15.103:0/4086221455 >> 172.21.15.103:0/4086221455 conn(0x7f776c101f10 msgr2=0x7f776c18fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:14.121 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:13.205+0000 7f7773279640 1 -- 172.21.15.103:0/4086221455 shutdown_connections 2024-09-19T06:40:14.121 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:13.205+0000 7f7773279640 1 -- 172.21.15.103:0/4086221455 wait complete. 2024-09-19T06:40:14.121 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for the mgr to restart... 2024-09-19T06:40:14.121 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr epoch 5... 2024-09-19T06:40:14.205 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:13 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/4086221455' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T06:40:14.206 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:14.131+0000 7ff59c71a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:40:14.206 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:14.204+0000 7ff59c71a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:40:15.183 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:14.926+0000 7ff59c71a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:40:15.183 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:15.039+0000 7ff59c71a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:40:15.458 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:15.181+0000 7ff59c71a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:40:15.458 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:15.242+0000 7ff59c71a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:40:15.458 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:15.390+0000 7ff59c71a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:40:15.458 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:15.456+0000 7ff59c71a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:40:15.737 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:15.517+0000 7ff59c71a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:40:15.737 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:15.672+0000 7ff59c71a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:40:15.737 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:15.735+0000 7ff59c71a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:40:15.999 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:15.801+0000 7ff59c71a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:40:15.999 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:15.882+0000 7ff59c71a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:40:15.999 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:15.940+0000 7ff59c71a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:40:15.999 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:15.998+0000 7ff59c71a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:40:17.137 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:16.779+0000 7ff59c71a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:40:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-mon[31158]: Active manager daemon x restarted 2024-09-19T06:40:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-mon[31158]: Activating manager daemon x 2024-09-19T06:40:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-mon[31158]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T06:40:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-mon[31158]: mgrmap e6: x(active, starting, since 0.00359353s) 2024-09-19T06:40:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:40:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T06:40:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:40:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:40:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:40:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-mon[31158]: Manager daemon x is now available 2024-09-19T06:40:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' 2024-09-19T06:40:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' 2024-09-19T06:40:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:40:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:40:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:40:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:16 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T06:40:18.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:17 smithi103 ceph-mon[31158]: Found migration_current of "None". Setting to last migration. 2024-09-19T06:40:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:17 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' 2024-09-19T06:40:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:17 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' 2024-09-19T06:40:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:17 smithi103 ceph-mon[31158]: mgrmap e7: x(active, since 1.00607s) 2024-09-19T06:40:18.423 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:18.371+0000 7ff503a74640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:06:40:18] ENGINE Error in HTTPServer.serve 2024-09-19T06:40:18.423 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: Traceback (most recent call last): 2024-09-19T06:40:18.423 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:40:18.423 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: self._connections.run(self.expiration_interval) 2024-09-19T06:40:18.423 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:40:18.423 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: self._run(expiration_interval) 2024-09-19T06:40:18.423 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:40:18.423 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:40:18.423 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:40:18.423 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:40:18.424 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:40:18.424 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: s = self.context.wrap_socket( 2024-09-19T06:40:18.424 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:40:18.424 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: return self.sslsocket_class._create( 2024-09-19T06:40:18.424 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:40:18.424 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: self.do_handshake() 2024-09-19T06:40:18.424 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:40:18.424 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: self._sslobj.do_handshake() 2024-09-19T06:40:18.424 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:40:18.424 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:18.431 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:40:18.431 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T06:40:18.432 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T06:40:18.432 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:40:18.432 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.529+0000 7f7c10f0b640 1 Processor -- start 2024-09-19T06:40:18.432 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.530+0000 7f7c10f0b640 1 -- start start 2024-09-19T06:40:18.432 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.530+0000 7f7c10f0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c106140 0x7f7c0c106540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:18.432 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.530+0000 7f7c10f0b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c07bdc0 con 0x7f7c0c106140 2024-09-19T06:40:18.432 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.530+0000 7f7c0a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c106140 0x7f7c0c106540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:18.432 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.530+0000 7f7c0a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c106140 0x7f7c0c106540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40520/0 (socket says 172.21.15.103:40520) 2024-09-19T06:40:18.432 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.530+0000 7f7c0a59c640 1 -- 172.21.15.103:0/2669768810 learned_addr learned my addr 172.21.15.103:0/2669768810 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:18.432 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.530+0000 7f7c0a59c640 1 -- 172.21.15.103:0/2669768810 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0c106a80 con 0x7f7c0c106140 2024-09-19T06:40:18.432 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.531+0000 7f7c0a59c640 1 --2- 172.21.15.103:0/2669768810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c106140 0x7f7c0c106540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009820 tx=0x7f7bfc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac4364f04858050 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:18.433 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.531+0000 7f7c09d9b640 1 -- 172.21.15.103:0/2669768810 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bfc037070 con 0x7f7c0c106140 2024-09-19T06:40:18.433 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.531+0000 7f7c09d9b640 1 -- 172.21.15.103:0/2669768810 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7bfc032900 con 0x7f7c0c106140 2024-09-19T06:40:18.433 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.531+0000 7f7c09d9b640 1 -- 172.21.15.103:0/2669768810 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bfc030850 con 0x7f7c0c106140 2024-09-19T06:40:18.433 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.532+0000 7f7c10f0b640 1 -- 172.21.15.103:0/2669768810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c106140 msgr2=0x7f7c0c106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:18.433 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.532+0000 7f7c10f0b640 1 --2- 172.21.15.103:0/2669768810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c106140 0x7f7c0c106540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009820 tx=0x7f7bfc02a270 comp rx=0 tx=0).stop 2024-09-19T06:40:18.433 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.532+0000 7f7c10f0b640 1 -- 172.21.15.103:0/2669768810 shutdown_connections 2024-09-19T06:40:18.433 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.532+0000 7f7c10f0b640 1 --2- 172.21.15.103:0/2669768810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c106140 0x7f7c0c106540 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:18.433 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.532+0000 7f7c10f0b640 1 -- 172.21.15.103:0/2669768810 >> 172.21.15.103:0/2669768810 conn(0x7f7c0c101f10 msgr2=0x7f7c0c104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:18.433 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.532+0000 7f7c10f0b640 1 -- 172.21.15.103:0/2669768810 shutdown_connections 2024-09-19T06:40:18.433 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.532+0000 7f7c10f0b640 1 -- 172.21.15.103:0/2669768810 wait complete. 2024-09-19T06:40:18.433 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.533+0000 7f7c10f0b640 1 Processor -- start 2024-09-19T06:40:18.433 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.533+0000 7f7c10f0b640 1 -- start start 2024-09-19T06:40:18.433 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.533+0000 7f7c10f0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c106140 0x7f7c0c19da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:18.434 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.533+0000 7f7c10f0b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c107b80 con 0x7f7c0c106140 2024-09-19T06:40:18.434 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.533+0000 7f7c0a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c106140 0x7f7c0c19da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:18.434 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.533+0000 7f7c0a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c106140 0x7f7c0c19da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40536/0 (socket says 172.21.15.103:40536) 2024-09-19T06:40:18.434 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.533+0000 7f7c0a59c640 1 -- 172.21.15.103:0/2654871925 learned_addr learned my addr 172.21.15.103:0/2654871925 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:18.434 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.533+0000 7f7c0a59c640 1 -- 172.21.15.103:0/2654871925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0c19dfc0 con 0x7f7c0c106140 2024-09-19T06:40:18.434 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.533+0000 7f7c0a59c640 1 --2- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c106140 0x7f7c0c19da80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0097f0 tx=0x7f7bfc030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:18.434 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.534+0000 7f7c037fe640 1 -- 172.21.15.103:0/2654871925 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bfc042070 con 0x7f7c0c106140 2024-09-19T06:40:18.434 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.534+0000 7f7c037fe640 1 -- 172.21.15.103:0/2654871925 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7bfc032ca0 con 0x7f7c0c106140 2024-09-19T06:40:18.434 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.534+0000 7f7c10f0b640 1 -- 172.21.15.103:0/2654871925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0c19e250 con 0x7f7c0c106140 2024-09-19T06:40:18.434 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.534+0000 7f7c037fe640 1 -- 172.21.15.103:0/2654871925 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bfc037040 con 0x7f7c0c106140 2024-09-19T06:40:18.435 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.534+0000 7f7c10f0b640 1 -- 172.21.15.103:0/2654871925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0c1a0f40 con 0x7f7c0c106140 2024-09-19T06:40:18.435 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.535+0000 7f7c037fe640 1 -- 172.21.15.103:0/2654871925 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f7bfc045070 con 0x7f7c0c106140 2024-09-19T06:40:18.435 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.535+0000 7f7c037fe640 1 --2- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f7bf80391b0 0x7f7bf803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:18.435 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.535+0000 7f7c037fe640 1 -- 172.21.15.103:0/2654871925 --> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7bf803bd80 con 0x7f7bf80391b0 2024-09-19T06:40:18.435 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.535+0000 7f7c037fe640 1 -- 172.21.15.103:0/2654871925 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7bfc06c190 con 0x7f7c0c106140 2024-09-19T06:40:18.435 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.535+0000 7f7c03fff640 1 -- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f7bf80391b0 msgr2=0x7f7bf803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2003588522 2024-09-19T06:40:18.435 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.535+0000 7f7c03fff640 1 --2- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f7bf80391b0 0x7f7bf803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:40:18.435 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.736+0000 7f7c03fff640 1 -- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f7bf80391b0 msgr2=0x7f7bf803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2003588522 2024-09-19T06:40:18.435 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:14.736+0000 7f7c03fff640 1 --2- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f7bf80391b0 0x7f7bf803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:40:18.435 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:15.137+0000 7f7c03fff640 1 -- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f7bf80391b0 msgr2=0x7f7bf803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2003588522 2024-09-19T06:40:18.435 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:15.137+0000 7f7c03fff640 1 --2- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f7bf80391b0 0x7f7bf803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T06:40:18.435 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:15.938+0000 7f7c03fff640 1 -- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f7bf80391b0 msgr2=0x7f7bf803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2003588522 2024-09-19T06:40:18.436 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:15.938+0000 7f7c03fff640 1 --2- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f7bf80391b0 0x7f7bf803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T06:40:18.436 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:16.783+0000 7f7c037fe640 1 -- 172.21.15.103:0/2654871925 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f7bfc06ad10 con 0x7f7c0c106140 2024-09-19T06:40:18.436 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:16.783+0000 7f7c037fe640 1 -- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f7bf80391b0 msgr2=0x7f7bf803b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:40:18.436 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:16.783+0000 7f7c037fe640 1 --2- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f7bf80391b0 0x7f7bf803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:18.436 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.783+0000 7f7c037fe640 1 -- 172.21.15.103:0/2654871925 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f7bfc03d430 con 0x7f7c0c106140 2024-09-19T06:40:18.436 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.784+0000 7f7c037fe640 1 --2- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f7bf803cae0 0x7f7bf803eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:18.436 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.784+0000 7f7c037fe640 1 -- 172.21.15.103:0/2654871925 --> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7bf803bd80 con 0x7f7bf803cae0 2024-09-19T06:40:18.436 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.784+0000 7f7c03fff640 1 --2- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f7bf803cae0 0x7f7bf803eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:18.436 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.785+0000 7f7c03fff640 1 --2- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f7bf803cae0 0x7f7bf803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4003980 tx=0x7f7bf4006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:18.436 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.785+0000 7f7c037fe640 1 -- 172.21.15.103:0/2654871925 <== mgr.14118 v2:172.21.15.103:6800/4122140716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f7bf803bd80 con 0x7f7bf803cae0 2024-09-19T06:40:18.436 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.792+0000 7f7c10f0b640 1 -- 172.21.15.103:0/2654871925 --> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7c0c106540 con 0x7f7bf803cae0 2024-09-19T06:40:18.436 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.793+0000 7f7c037fe640 1 -- 172.21.15.103:0/2654871925 <== mgr.14118 v2:172.21.15.103:6800/4122140716 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7c0c106540 con 0x7f7bf803cae0 2024-09-19T06:40:18.436 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.793+0000 7f7c10f0b640 1 -- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f7bf803cae0 msgr2=0x7f7bf803eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:18.437 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.793+0000 7f7c10f0b640 1 --2- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f7bf803cae0 0x7f7bf803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4003980 tx=0x7f7bf4006e10 comp rx=0 tx=0).stop 2024-09-19T06:40:18.437 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.793+0000 7f7c10f0b640 1 -- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c106140 msgr2=0x7f7c0c19da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:18.437 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.793+0000 7f7c10f0b640 1 --2- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c106140 0x7f7c0c19da80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0097f0 tx=0x7f7bfc030ec0 comp rx=0 tx=0).stop 2024-09-19T06:40:18.437 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.793+0000 7f7c10f0b640 1 -- 172.21.15.103:0/2654871925 shutdown_connections 2024-09-19T06:40:18.437 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.793+0000 7f7c10f0b640 1 --2- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f7bf803cae0 0x7f7bf803eed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:18.437 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.793+0000 7f7c10f0b640 1 --2- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:6800/2003588522,v1:172.21.15.103:6801/2003588522] conn(0x7f7bf80391b0 0x7f7bf803b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:18.437 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.793+0000 7f7c10f0b640 1 --2- 172.21.15.103:0/2654871925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c106140 0x7f7c0c19da80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:18.437 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.793+0000 7f7c10f0b640 1 -- 172.21.15.103:0/2654871925 >> 172.21.15.103:0/2654871925 conn(0x7f7c0c101f10 msgr2=0x7f7c0c194740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:18.437 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.793+0000 7f7c10f0b640 1 -- 172.21.15.103:0/2654871925 shutdown_connections 2024-09-19T06:40:18.437 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:17.793+0000 7f7c10f0b640 1 -- 172.21.15.103:0/2654871925 wait complete. 2024-09-19T06:40:18.437 INFO:teuthology.orchestra.run.smithi103.stdout:mgr epoch 5 is available 2024-09-19T06:40:18.437 INFO:teuthology.orchestra.run.smithi103.stdout:Setting orchestrator backend to cephadm... 2024-09-19T06:40:19.051 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-mon[31158]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T06:40:19.051 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-mon[31158]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T06:40:19.052 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:18 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:40:19.636 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.861+0000 7fc49ee07640 1 Processor -- start 2024-09-19T06:40:19.637 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.861+0000 7fc49ee07640 1 -- start start 2024-09-19T06:40:19.637 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.861+0000 7fc49ee07640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc49807dde0 0x7fc49807c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:19.637 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.861+0000 7fc49ee07640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc49807c7d0 con 0x7fc49807dde0 2024-09-19T06:40:19.637 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.862+0000 7fc49cba3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc49807dde0 0x7fc49807c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:19.637 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.862+0000 7fc49cba3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc49807dde0 0x7fc49807c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41078/0 (socket says 172.21.15.103:41078) 2024-09-19T06:40:19.637 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.862+0000 7fc49cba3640 1 -- 172.21.15.103:0/1706329087 learned_addr learned my addr 172.21.15.103:0/1706329087 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:19.637 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.862+0000 7fc49cba3640 1 -- 172.21.15.103:0/1706329087 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49807c950 con 0x7fc49807dde0 2024-09-19T06:40:19.637 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.862+0000 7fc49cba3640 1 --2- 172.21.15.103:0/1706329087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc49807dde0 0x7fc49807c290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009820 tx=0x7fc48c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d056e3f4afc7685e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:19.637 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.863+0000 7fc4977fe640 1 -- 172.21.15.103:0/1706329087 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc48c037070 con 0x7fc49807dde0 2024-09-19T06:40:19.638 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.863+0000 7fc4977fe640 1 -- 172.21.15.103:0/1706329087 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc48c032900 con 0x7fc49807dde0 2024-09-19T06:40:19.638 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.863+0000 7fc4977fe640 1 -- 172.21.15.103:0/1706329087 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc48c0308e0 con 0x7fc49807dde0 2024-09-19T06:40:19.638 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.863+0000 7fc49ee07640 1 -- 172.21.15.103:0/1706329087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc49807dde0 msgr2=0x7fc49807c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:19.638 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.863+0000 7fc49ee07640 1 --2- 172.21.15.103:0/1706329087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc49807dde0 0x7fc49807c290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009820 tx=0x7fc48c02a300 comp rx=0 tx=0).stop 2024-09-19T06:40:19.638 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.863+0000 7fc49ee07640 1 -- 172.21.15.103:0/1706329087 shutdown_connections 2024-09-19T06:40:19.638 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.863+0000 7fc49ee07640 1 --2- 172.21.15.103:0/1706329087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc49807dde0 0x7fc49807c290 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:19.638 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.863+0000 7fc49ee07640 1 -- 172.21.15.103:0/1706329087 >> 172.21.15.103:0/1706329087 conn(0x7fc498103080 msgr2=0x7fc4981054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:19.638 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.863+0000 7fc49ee07640 1 -- 172.21.15.103:0/1706329087 shutdown_connections 2024-09-19T06:40:19.638 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.863+0000 7fc49ee07640 1 -- 172.21.15.103:0/1706329087 wait complete. 2024-09-19T06:40:19.638 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.864+0000 7fc49ee07640 1 Processor -- start 2024-09-19T06:40:19.638 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.864+0000 7fc49ee07640 1 -- start start 2024-09-19T06:40:19.638 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.864+0000 7fc49ee07640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc49807dde0 0x7fc49819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:19.638 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.864+0000 7fc49ee07640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc49810dee0 con 0x7fc49807dde0 2024-09-19T06:40:19.639 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.864+0000 7fc49cba3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc49807dde0 0x7fc49819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:19.639 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.864+0000 7fc49cba3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc49807dde0 0x7fc49819dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41084/0 (socket says 172.21.15.103:41084) 2024-09-19T06:40:19.639 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.864+0000 7fc49cba3640 1 -- 172.21.15.103:0/42434174 learned_addr learned my addr 172.21.15.103:0/42434174 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:19.639 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.864+0000 7fc49cba3640 1 -- 172.21.15.103:0/42434174 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49819e010 con 0x7fc49807dde0 2024-09-19T06:40:19.639 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.864+0000 7fc49cba3640 1 --2- 172.21.15.103:0/42434174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc49807dde0 0x7fc49819dad0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc48c02a820 tx=0x7fc48c030900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:19.639 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.865+0000 7fc495ffb640 1 -- 172.21.15.103:0/42434174 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc48c037040 con 0x7fc49807dde0 2024-09-19T06:40:19.639 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.865+0000 7fc495ffb640 1 -- 172.21.15.103:0/42434174 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc48c040070 con 0x7fc49807dde0 2024-09-19T06:40:19.639 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.865+0000 7fc495ffb640 1 -- 172.21.15.103:0/42434174 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc48c03bdd0 con 0x7fc49807dde0 2024-09-19T06:40:19.640 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.865+0000 7fc49ee07640 1 -- 172.21.15.103:0/42434174 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49819e2a0 con 0x7fc49807dde0 2024-09-19T06:40:19.640 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.865+0000 7fc49ee07640 1 -- 172.21.15.103:0/42434174 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4981a0f90 con 0x7fc49807dde0 2024-09-19T06:40:19.640 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.866+0000 7fc495ffb640 1 -- 172.21.15.103:0/42434174 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fc48c03b5a0 con 0x7fc49807dde0 2024-09-19T06:40:19.640 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.866+0000 7fc49ee07640 1 -- 172.21.15.103:0/42434174 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49807c310 con 0x7fc49807dde0 2024-09-19T06:40:19.640 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.866+0000 7fc495ffb640 1 --2- 172.21.15.103:0/42434174 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fc4880391b0 0x7fc48803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:19.640 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.866+0000 7fc495ffb640 1 -- 172.21.15.103:0/42434174 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc48c06c3c0 con 0x7fc49807dde0 2024-09-19T06:40:19.640 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.866+0000 7fc497fff640 1 --2- 172.21.15.103:0/42434174 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fc4880391b0 0x7fc48803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:19.640 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.866+0000 7fc497fff640 1 --2- 172.21.15.103:0/42434174 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fc4880391b0 0x7fc48803b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4840098c0 tx=0x7fc484006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:19.640 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:18.871+0000 7fc495ffb640 1 -- 172.21.15.103:0/42434174 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc48c0457e0 con 0x7fc49807dde0 2024-09-19T06:40:19.641 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:19.050+0000 7fc49ee07640 1 -- 172.21.15.103:0/42434174 --> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fc498104370 con 0x7fc4880391b0 2024-09-19T06:40:19.641 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:19.055+0000 7fc495ffb640 1 -- 172.21.15.103:0/42434174 <== mgr.14118 v2:172.21.15.103:6800/4122140716 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc498104370 con 0x7fc4880391b0 2024-09-19T06:40:19.641 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:19.057+0000 7fc49ee07640 1 -- 172.21.15.103:0/42434174 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fc4880391b0 msgr2=0x7fc48803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:19.641 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:19.057+0000 7fc49ee07640 1 --2- 172.21.15.103:0/42434174 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fc4880391b0 0x7fc48803b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4840098c0 tx=0x7fc484006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:19.641 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:19.057+0000 7fc49ee07640 1 -- 172.21.15.103:0/42434174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc49807dde0 msgr2=0x7fc49819dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:19.641 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:19.057+0000 7fc49ee07640 1 --2- 172.21.15.103:0/42434174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc49807dde0 0x7fc49819dad0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc48c02a820 tx=0x7fc48c030900 comp rx=0 tx=0).stop 2024-09-19T06:40:19.641 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:19.057+0000 7fc49ee07640 1 -- 172.21.15.103:0/42434174 shutdown_connections 2024-09-19T06:40:19.641 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:19.057+0000 7fc49ee07640 1 --2- 172.21.15.103:0/42434174 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fc4880391b0 0x7fc48803b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:19.641 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:19.057+0000 7fc49ee07640 1 --2- 172.21.15.103:0/42434174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc49807dde0 0x7fc49819dad0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:19.641 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:19.057+0000 7fc49ee07640 1 -- 172.21.15.103:0/42434174 >> 172.21.15.103:0/42434174 conn(0x7fc498103080 msgr2=0x7fc498103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:19.641 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:19.057+0000 7fc49ee07640 1 -- 172.21.15.103:0/42434174 shutdown_connections 2024-09-19T06:40:19.642 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:19.058+0000 7fc49ee07640 1 -- 172.21.15.103:0/42434174 wait complete. 2024-09-19T06:40:19.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:19 smithi103 ceph-mon[31158]: [19/Sep/2024:06:40:18] ENGINE Bus STARTING 2024-09-19T06:40:19.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:19 smithi103 ceph-mon[31158]: [19/Sep/2024:06:40:18] ENGINE Serving on http://172.21.15.103:8765 2024-09-19T06:40:19.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:19 smithi103 ceph-mon[31158]: [19/Sep/2024:06:40:18] ENGINE Serving on https://172.21.15.103:7150 2024-09-19T06:40:19.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:19 smithi103 ceph-mon[31158]: [19/Sep/2024:06:40:18] ENGINE Bus STARTED 2024-09-19T06:40:19.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:19 smithi103 ceph-mon[31158]: [19/Sep/2024:06:40:18] ENGINE Error in HTTPServer.serve 2024-09-19T06:40:19.888 INFO:journalctl@ceph.mon.a.smithi103.stdout: Traceback (most recent call last): 2024-09-19T06:40:19.888 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:40:19.888 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:40:19.888 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:40:19.889 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._run(expiration_interval) 2024-09-19T06:40:19.889 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:40:19.889 INFO:journalctl@ceph.mon.a.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:40:19.889 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:40:19.889 INFO:journalctl@ceph.mon.a.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:40:19.889 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:40:19.889 INFO:journalctl@ceph.mon.a.smithi103.stdout: s = self.context.wrap_socket( 2024-09-19T06:40:19.889 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:40:19.889 INFO:journalctl@ceph.mon.a.smithi103.stdout: return self.sslsocket_class._create( 2024-09-19T06:40:19.889 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:40:19.889 INFO:journalctl@ceph.mon.a.smithi103.stdout: self.do_handshake() 2024-09-19T06:40:19.889 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:40:19.889 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._sslobj.do_handshake() 2024-09-19T06:40:19.889 INFO:journalctl@ceph.mon.a.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:40:19.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:19 smithi103 ceph-mon[31158]: mgrmap e8: x(active, since 2s) 2024-09-19T06:40:19.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:19 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' 2024-09-19T06:40:19.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:19 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:40:20.785 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T06:40:20.785 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.006+0000 7fddc4d7b640 1 Processor -- start 2024-09-19T06:40:20.785 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.006+0000 7fddc4d7b640 1 -- start start 2024-09-19T06:40:20.785 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.006+0000 7fddc4d7b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddc0106780 0x7fddc0106b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:20.785 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.006+0000 7fddc4d7b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fddc0107150 con 0x7fddc0106780 2024-09-19T06:40:20.785 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.007+0000 7fddbe59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddc0106780 0x7fddc0106b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:20.785 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.007+0000 7fddbe59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddc0106780 0x7fddc0106b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41096/0 (socket says 172.21.15.103:41096) 2024-09-19T06:40:20.785 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.007+0000 7fddbe59c640 1 -- 172.21.15.103:0/1621524083 learned_addr learned my addr 172.21.15.103:0/1621524083 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:20.785 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.007+0000 7fddbe59c640 1 -- 172.21.15.103:0/1621524083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc0107980 con 0x7fddc0106780 2024-09-19T06:40:20.785 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.007+0000 7fddbe59c640 1 --2- 172.21.15.103:0/1621524083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddc0106780 0x7fddc0106b80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009e80 tx=0x7fddb002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25714133471a9b72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:20.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.008+0000 7fddbd59a640 1 -- 172.21.15.103:0/1621524083 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddb0036070 con 0x7fddc0106780 2024-09-19T06:40:20.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.008+0000 7fddbd59a640 1 -- 172.21.15.103:0/1621524083 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fddb00318c0 con 0x7fddc0106780 2024-09-19T06:40:20.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.008+0000 7fddbd59a640 1 -- 172.21.15.103:0/1621524083 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddb0007ab0 con 0x7fddc0106780 2024-09-19T06:40:20.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.008+0000 7fddc4d7b640 1 -- 172.21.15.103:0/1621524083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddc0106780 msgr2=0x7fddc0106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:20.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.008+0000 7fddc4d7b640 1 --2- 172.21.15.103:0/1621524083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddc0106780 0x7fddc0106b80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009e80 tx=0x7fddb002a7a0 comp rx=0 tx=0).stop 2024-09-19T06:40:20.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.008+0000 7fddc4d7b640 1 -- 172.21.15.103:0/1621524083 shutdown_connections 2024-09-19T06:40:20.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.008+0000 7fddc4d7b640 1 --2- 172.21.15.103:0/1621524083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddc0106780 0x7fddc0106b80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:20.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.008+0000 7fddc4d7b640 1 -- 172.21.15.103:0/1621524083 >> 172.21.15.103:0/1621524083 conn(0x7fddc0101f10 msgr2=0x7fddc0104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:20.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.008+0000 7fddc4d7b640 1 -- 172.21.15.103:0/1621524083 shutdown_connections 2024-09-19T06:40:20.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.008+0000 7fddc4d7b640 1 -- 172.21.15.103:0/1621524083 wait complete. 2024-09-19T06:40:20.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.009+0000 7fddc4d7b640 1 Processor -- start 2024-09-19T06:40:20.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.009+0000 7fddc4d7b640 1 -- start start 2024-09-19T06:40:20.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.009+0000 7fddc4d7b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddc0106780 0x7fddc019b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:20.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.009+0000 7fddc4d7b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fddc010ad30 con 0x7fddc0106780 2024-09-19T06:40:20.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.009+0000 7fddbe59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddc0106780 0x7fddc019b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:20.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.009+0000 7fddbe59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddc0106780 0x7fddc019b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41102/0 (socket says 172.21.15.103:41102) 2024-09-19T06:40:20.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.009+0000 7fddbe59c640 1 -- 172.21.15.103:0/1392834000 learned_addr learned my addr 172.21.15.103:0/1392834000 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:20.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.010+0000 7fddbe59c640 1 -- 172.21.15.103:0/1392834000 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc019be50 con 0x7fddc0106780 2024-09-19T06:40:20.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.010+0000 7fddbe59c640 1 --2- 172.21.15.103:0/1392834000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddc0106780 0x7fddc019b910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fddb002acc0 tx=0x7fddb0031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:20.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.010+0000 7fddb77fe640 1 -- 172.21.15.103:0/1392834000 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddb0036050 con 0x7fddc0106780 2024-09-19T06:40:20.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.010+0000 7fddb77fe640 1 -- 172.21.15.103:0/1392834000 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fddb0037040 con 0x7fddc0106780 2024-09-19T06:40:20.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.010+0000 7fddc4d7b640 1 -- 172.21.15.103:0/1392834000 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc019c0e0 con 0x7fddc0106780 2024-09-19T06:40:20.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.010+0000 7fddb77fe640 1 -- 172.21.15.103:0/1392834000 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddb003d890 con 0x7fddc0106780 2024-09-19T06:40:20.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.010+0000 7fddc4d7b640 1 -- 172.21.15.103:0/1392834000 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc019edd0 con 0x7fddc0106780 2024-09-19T06:40:20.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.011+0000 7fddb77fe640 1 -- 172.21.15.103:0/1392834000 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fddb0044020 con 0x7fddc0106780 2024-09-19T06:40:20.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.011+0000 7fddb77fe640 1 --2- 172.21.15.103:0/1392834000 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fddac039200 0x7fddac03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:20.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.011+0000 7fddb77fe640 1 -- 172.21.15.103:0/1392834000 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fddb0042020 con 0x7fddc0106780 2024-09-19T06:40:20.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.011+0000 7fddbdd9b640 1 --2- 172.21.15.103:0/1392834000 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fddac039200 0x7fddac03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:20.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.011+0000 7fddc4d7b640 1 -- 172.21.15.103:0/1392834000 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc010aeb0 con 0x7fddc0106780 2024-09-19T06:40:20.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.012+0000 7fddbdd9b640 1 --2- 172.21.15.103:0/1392834000 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fddac039200 0x7fddac03b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdda80098c0 tx=0x7fdda8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:20.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.016+0000 7fddb77fe640 1 -- 172.21.15.103:0/1392834000 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fddb004b020 con 0x7fddc0106780 2024-09-19T06:40:20.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.184+0000 7fddc4d7b640 1 -- 172.21.15.103:0/1392834000 --> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fddc002d900 con 0x7fddac039200 2024-09-19T06:40:20.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.185+0000 7fddb77fe640 1 -- 172.21.15.103:0/1392834000 <== mgr.14118 v2:172.21.15.103:6800/4122140716 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fddc002d900 con 0x7fddac039200 2024-09-19T06:40:20.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.189+0000 7fddc4d7b640 1 -- 172.21.15.103:0/1392834000 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fddac039200 msgr2=0x7fddac03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:20.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.189+0000 7fddc4d7b640 1 --2- 172.21.15.103:0/1392834000 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fddac039200 0x7fddac03b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdda80098c0 tx=0x7fdda8006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:20.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.189+0000 7fddc4d7b640 1 -- 172.21.15.103:0/1392834000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddc0106780 msgr2=0x7fddc019b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:20.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.189+0000 7fddc4d7b640 1 --2- 172.21.15.103:0/1392834000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddc0106780 0x7fddc019b910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fddb002acc0 tx=0x7fddb0031ca0 comp rx=0 tx=0).stop 2024-09-19T06:40:20.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.189+0000 7fddc4d7b640 1 -- 172.21.15.103:0/1392834000 shutdown_connections 2024-09-19T06:40:20.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.189+0000 7fddc4d7b640 1 --2- 172.21.15.103:0/1392834000 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fddac039200 0x7fddac03b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:20.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.189+0000 7fddc4d7b640 1 --2- 172.21.15.103:0/1392834000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddc0106780 0x7fddc019b910 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:20.789 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.189+0000 7fddc4d7b640 1 -- 172.21.15.103:0/1392834000 >> 172.21.15.103:0/1392834000 conn(0x7fddc0101f10 msgr2=0x7fddc0102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:20.789 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.189+0000 7fddc4d7b640 1 -- 172.21.15.103:0/1392834000 shutdown_connections 2024-09-19T06:40:20.789 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:20.189+0000 7fddc4d7b640 1 -- 172.21.15.103:0/1392834000 wait complete. 2024-09-19T06:40:20.789 INFO:teuthology.orchestra.run.smithi103.stdout:Generating ssh key... 2024-09-19T06:40:21.113 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:20 smithi103 ceph-mon[31158]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:21.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: Generating public/private rsa key pair. 2024-09-19T06:40:21.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: Your identification has been saved in /tmp/tmpby083zpx/key 2024-09-19T06:40:21.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: Your public key has been saved in /tmp/tmpby083zpx/key.pub 2024-09-19T06:40:21.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: The key fingerprint is: 2024-09-19T06:40:21.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: SHA256:nRTwYtq4xE3+JaC3BKYcaICvbbLmhOoWRH3KOXNwch8 ceph-d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:40:21.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: The key's randomart image is: 2024-09-19T06:40:21.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: +---[RSA 3072]----+ 2024-09-19T06:40:21.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: |o . ... | 2024-09-19T06:40:21.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: |.o = + E . . | 2024-09-19T06:40:21.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: |..+ X + * o | 2024-09-19T06:40:21.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: | o.B * % = . | 2024-09-19T06:40:21.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: |.o * * S + . | 2024-09-19T06:40:21.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: |+.o . + o o | 2024-09-19T06:40:21.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: |.=. . . . | 2024-09-19T06:40:21.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: |+o | 2024-09-19T06:40:21.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: |*o | 2024-09-19T06:40:21.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: +----[SHA256]-----+ 2024-09-19T06:40:22.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-mon[31158]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:22.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' 2024-09-19T06:40:22.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:21 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' 2024-09-19T06:40:22.199 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.196+0000 7f2957be5640 1 Processor -- start 2024-09-19T06:40:22.200 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.196+0000 7f2957be5640 1 -- start start 2024-09-19T06:40:22.200 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.196+0000 7f2957be5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2950106780 0x7f2950106b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:22.200 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.196+0000 7f2957be5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2950107150 con 0x7f2950106780 2024-09-19T06:40:22.200 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.197+0000 7f2955981640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2950106780 0x7f2950106b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:22.200 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.197+0000 7f2955981640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2950106780 0x7f2950106b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41116/0 (socket says 172.21.15.103:41116) 2024-09-19T06:40:22.200 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.197+0000 7f2955981640 1 -- 172.21.15.103:0/4166654734 learned_addr learned my addr 172.21.15.103:0/4166654734 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:22.200 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.197+0000 7f2955981640 1 -- 172.21.15.103:0/4166654734 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2950107980 con 0x7f2950106780 2024-09-19T06:40:22.200 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.198+0000 7f2955981640 1 --2- 172.21.15.103:0/4166654734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2950106780 0x7f2950106b80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2948009e80 tx=0x7f294802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d564479f7fa6bdfb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:22.200 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.198+0000 7f295497f640 1 -- 172.21.15.103:0/4166654734 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2948036070 con 0x7f2950106780 2024-09-19T06:40:22.200 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.198+0000 7f295497f640 1 -- 172.21.15.103:0/4166654734 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f29480318c0 con 0x7f2950106780 2024-09-19T06:40:22.200 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.198+0000 7f295497f640 1 -- 172.21.15.103:0/4166654734 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2948007ab0 con 0x7f2950106780 2024-09-19T06:40:22.201 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.198+0000 7f2957be5640 1 -- 172.21.15.103:0/4166654734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2950106780 msgr2=0x7f2950106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:22.201 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.199+0000 7f2957be5640 1 --2- 172.21.15.103:0/4166654734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2950106780 0x7f2950106b80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2948009e80 tx=0x7f294802a7a0 comp rx=0 tx=0).stop 2024-09-19T06:40:22.201 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.199+0000 7f2957be5640 1 -- 172.21.15.103:0/4166654734 shutdown_connections 2024-09-19T06:40:22.201 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.199+0000 7f2957be5640 1 --2- 172.21.15.103:0/4166654734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2950106780 0x7f2950106b80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:22.201 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.199+0000 7f2957be5640 1 -- 172.21.15.103:0/4166654734 >> 172.21.15.103:0/4166654734 conn(0x7f2950101f10 msgr2=0x7f2950104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:22.201 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.199+0000 7f2957be5640 1 -- 172.21.15.103:0/4166654734 shutdown_connections 2024-09-19T06:40:22.201 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.199+0000 7f2957be5640 1 -- 172.21.15.103:0/4166654734 wait complete. 2024-09-19T06:40:22.201 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.199+0000 7f2957be5640 1 Processor -- start 2024-09-19T06:40:22.201 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.199+0000 7f2957be5640 1 -- start start 2024-09-19T06:40:22.201 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.200+0000 7f2957be5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2950106780 0x7f295007ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:22.201 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.200+0000 7f2957be5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f295010ad30 con 0x7f2950106780 2024-09-19T06:40:22.201 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.200+0000 7f2955981640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2950106780 0x7f295007ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:22.201 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.200+0000 7f2955981640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2950106780 0x7f295007ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41132/0 (socket says 172.21.15.103:41132) 2024-09-19T06:40:22.202 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.200+0000 7f2955981640 1 -- 172.21.15.103:0/2478509956 learned_addr learned my addr 172.21.15.103:0/2478509956 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:22.202 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.200+0000 7f2955981640 1 -- 172.21.15.103:0/2478509956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295019e630 con 0x7f2950106780 2024-09-19T06:40:22.202 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.200+0000 7f2955981640 1 --2- 172.21.15.103:0/2478509956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2950106780 0x7f295007ba20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2948007ce0 tx=0x7f2948031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:22.202 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.200+0000 7f294effd640 1 -- 172.21.15.103:0/2478509956 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2948036050 con 0x7f2950106780 2024-09-19T06:40:22.202 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.200+0000 7f2957be5640 1 -- 172.21.15.103:0/2478509956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f295019e800 con 0x7f2950106780 2024-09-19T06:40:22.202 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.200+0000 7f294effd640 1 -- 172.21.15.103:0/2478509956 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2948042070 con 0x7f2950106780 2024-09-19T06:40:22.202 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.200+0000 7f294effd640 1 -- 172.21.15.103:0/2478509956 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f294803d8a0 con 0x7f2950106780 2024-09-19T06:40:22.202 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.200+0000 7f2957be5640 1 -- 172.21.15.103:0/2478509956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f295019ec20 con 0x7f2950106780 2024-09-19T06:40:22.202 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.201+0000 7f294effd640 1 -- 172.21.15.103:0/2478509956 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f294803dba0 con 0x7f2950106780 2024-09-19T06:40:22.202 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.201+0000 7f294effd640 1 --2- 172.21.15.103:0/2478509956 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f29380391b0 0x7f293803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:22.202 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.201+0000 7f294effd640 1 -- 172.21.15.103:0/2478509956 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2948007e00 con 0x7f2950106780 2024-09-19T06:40:22.202 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.202+0000 7f2955180640 1 --2- 172.21.15.103:0/2478509956 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f29380391b0 0x7f293803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:22.202 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.202+0000 7f2957be5640 1 -- 172.21.15.103:0/2478509956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f295010aeb0 con 0x7f2950106780 2024-09-19T06:40:22.202 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.202+0000 7f2955180640 1 --2- 172.21.15.103:0/2478509956 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f29380391b0 0x7f293803b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f29440098c0 tx=0x7f2944006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.206+0000 7f294effd640 1 -- 172.21.15.103:0/2478509956 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2948044310 con 0x7f2950106780 2024-09-19T06:40:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.375+0000 7f2957be5640 1 -- 172.21.15.103:0/2478509956 --> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f29501031c0 con 0x7f29380391b0 2024-09-19T06:40:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.465+0000 7f294effd640 1 -- 172.21.15.103:0/2478509956 <== mgr.14118 v2:172.21.15.103:6800/4122140716 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f29501031c0 con 0x7f29380391b0 2024-09-19T06:40:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.467+0000 7f2957be5640 1 -- 172.21.15.103:0/2478509956 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f29380391b0 msgr2=0x7f293803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.467+0000 7f2957be5640 1 --2- 172.21.15.103:0/2478509956 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f29380391b0 0x7f293803b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f29440098c0 tx=0x7f2944006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.467+0000 7f2957be5640 1 -- 172.21.15.103:0/2478509956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2950106780 msgr2=0x7f295007ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.467+0000 7f2957be5640 1 --2- 172.21.15.103:0/2478509956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2950106780 0x7f295007ba20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2948007ce0 tx=0x7f2948031ca0 comp rx=0 tx=0).stop 2024-09-19T06:40:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.467+0000 7f2957be5640 1 -- 172.21.15.103:0/2478509956 shutdown_connections 2024-09-19T06:40:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.467+0000 7f2957be5640 1 --2- 172.21.15.103:0/2478509956 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f29380391b0 0x7f293803b670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.467+0000 7f2957be5640 1 --2- 172.21.15.103:0/2478509956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2950106780 0x7f295007ba20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.467+0000 7f2957be5640 1 -- 172.21.15.103:0/2478509956 >> 172.21.15.103:0/2478509956 conn(0x7f2950101f10 msgr2=0x7f2950102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.468+0000 7f2957be5640 1 -- 172.21.15.103:0/2478509956 shutdown_connections 2024-09-19T06:40:22.203 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:21.468+0000 7f2957be5640 1 -- 172.21.15.103:0/2478509956 wait complete. 2024-09-19T06:40:23.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:22 smithi103 ceph-mon[31158]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:23.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:22 smithi103 ceph-mon[31158]: Generating ssh key... 2024-09-19T06:40:23.349 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCm4oM0Z3ASq1o0tmL88hCDCE+YP5L9i7vnvobQ3kANsjgcObTOw9y8v64SaSp1R4u+iFHxUacfzCZd/G/s8r1lwY0a+HMeKSrS2O96C1okb9/yK5KNIyBwY/1/9NlEFlvJTg2oAPgqUIgDZpM3GkizYxxccEzfvug5uB5ca2hmfnfGfvWlYUnz+LK+RVUxqaUoNWqNCvNBn5WMkAc7ya2LMHbv9Xv5Xny8NRtQPKXqo6o0dzVpYI/5qFMaJZXqOOV48YphqEjN68Re+krWFfWAU2m4XriZjM9DZNJwE/2tQNBVqoKgRCWaybozSeiYCk9VAnwsA/rEtR2buBDJhZ0AxHW3nfpsiMrWGI14kJjKNRK0XQ5JqR4H8H7ZIweSj9aq1oBAikpZAFdX0FlyeCmNYEfi8a8QlkT47jpOhz+YuAAg5/at7F5y58qYYjfCdZg5WEJSswJ32jT3IZKBvFYy7uUiILY/0BwZK8EioF/LfJ/y39zMb+Sb6fAICUZMEeE= ceph-d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:40:23.350 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.589+0000 7ff612431640 1 Processor -- start 2024-09-19T06:40:23.350 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.589+0000 7ff612431640 1 -- start start 2024-09-19T06:40:23.350 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.590+0000 7ff612431640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c106780 0x7ff60c106b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:23.350 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.590+0000 7ff612431640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c107150 con 0x7ff60c106780 2024-09-19T06:40:23.350 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.590+0000 7ff60bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c106780 0x7ff60c106b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:23.350 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.590+0000 7ff60bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c106780 0x7ff60c106b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41144/0 (socket says 172.21.15.103:41144) 2024-09-19T06:40:23.350 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.590+0000 7ff60bfff640 1 -- 172.21.15.103:0/2371276960 learned_addr learned my addr 172.21.15.103:0/2371276960 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:23.350 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.590+0000 7ff60bfff640 1 -- 172.21.15.103:0/2371276960 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c107980 con 0x7ff60c106780 2024-09-19T06:40:23.350 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.591+0000 7ff60bfff640 1 --2- 172.21.15.103:0/2371276960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c106780 0x7ff60c106b80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009820 tx=0x7ff5fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d85357f52e36df1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:23.350 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.591+0000 7ff60affd640 1 -- 172.21.15.103:0/2371276960 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5fc037070 con 0x7ff60c106780 2024-09-19T06:40:23.351 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.591+0000 7ff60affd640 1 -- 172.21.15.103:0/2371276960 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5fc032900 con 0x7ff60c106780 2024-09-19T06:40:23.351 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.591+0000 7ff60affd640 1 -- 172.21.15.103:0/2371276960 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5fc030850 con 0x7ff60c106780 2024-09-19T06:40:23.351 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.592+0000 7ff612431640 1 -- 172.21.15.103:0/2371276960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c106780 msgr2=0x7ff60c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:23.351 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.592+0000 7ff612431640 1 --2- 172.21.15.103:0/2371276960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c106780 0x7ff60c106b80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009820 tx=0x7ff5fc02a270 comp rx=0 tx=0).stop 2024-09-19T06:40:23.351 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.592+0000 7ff612431640 1 -- 172.21.15.103:0/2371276960 shutdown_connections 2024-09-19T06:40:23.351 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.592+0000 7ff612431640 1 --2- 172.21.15.103:0/2371276960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c106780 0x7ff60c106b80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:23.351 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.592+0000 7ff612431640 1 -- 172.21.15.103:0/2371276960 >> 172.21.15.103:0/2371276960 conn(0x7ff60c101f10 msgr2=0x7ff60c104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:23.351 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.592+0000 7ff612431640 1 -- 172.21.15.103:0/2371276960 shutdown_connections 2024-09-19T06:40:23.351 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.592+0000 7ff612431640 1 -- 172.21.15.103:0/2371276960 wait complete. 2024-09-19T06:40:23.351 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.592+0000 7ff612431640 1 Processor -- start 2024-09-19T06:40:23.351 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.592+0000 7ff612431640 1 -- start start 2024-09-19T06:40:23.351 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.593+0000 7ff612431640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c106780 0x7ff60c19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:23.351 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.593+0000 7ff612431640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c10ad30 con 0x7ff60c106780 2024-09-19T06:40:23.352 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.593+0000 7ff60bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c106780 0x7ff60c19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:23.352 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.593+0000 7ff60bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c106780 0x7ff60c19a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41156/0 (socket says 172.21.15.103:41156) 2024-09-19T06:40:23.352 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.593+0000 7ff60bfff640 1 -- 172.21.15.103:0/183012129 learned_addr learned my addr 172.21.15.103:0/183012129 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:23.352 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.593+0000 7ff60bfff640 1 -- 172.21.15.103:0/183012129 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c19ad60 con 0x7ff60c106780 2024-09-19T06:40:23.352 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.593+0000 7ff60bfff640 1 --2- 172.21.15.103:0/183012129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c106780 0x7ff60c19a820 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0097f0 tx=0x7ff5fc030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:23.352 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.593+0000 7ff6097fa640 1 -- 172.21.15.103:0/183012129 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5fc042070 con 0x7ff60c106780 2024-09-19T06:40:23.352 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.593+0000 7ff612431640 1 -- 172.21.15.103:0/183012129 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60c19aff0 con 0x7ff60c106780 2024-09-19T06:40:23.352 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.593+0000 7ff6097fa640 1 -- 172.21.15.103:0/183012129 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5fc032ca0 con 0x7ff60c106780 2024-09-19T06:40:23.352 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.593+0000 7ff6097fa640 1 -- 172.21.15.103:0/183012129 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5fc037040 con 0x7ff60c106780 2024-09-19T06:40:23.352 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.593+0000 7ff612431640 1 -- 172.21.15.103:0/183012129 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff60c19dce0 con 0x7ff60c106780 2024-09-19T06:40:23.352 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.594+0000 7ff6097fa640 1 -- 172.21.15.103:0/183012129 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7ff5fc045070 con 0x7ff60c106780 2024-09-19T06:40:23.352 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.594+0000 7ff612431640 1 -- 172.21.15.103:0/183012129 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60c10aeb0 con 0x7ff60c106780 2024-09-19T06:40:23.353 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.594+0000 7ff6097fa640 1 --2- 172.21.15.103:0/183012129 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7ff5f80391b0 0x7ff5f803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:23.353 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.594+0000 7ff6097fa640 1 -- 172.21.15.103:0/183012129 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff5fc06c240 con 0x7ff60c106780 2024-09-19T06:40:23.353 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.595+0000 7ff60b7fe640 1 --2- 172.21.15.103:0/183012129 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7ff5f80391b0 0x7ff5f803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:23.353 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.595+0000 7ff60b7fe640 1 --2- 172.21.15.103:0/183012129 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7ff5f80391b0 0x7ff5f803b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40098c0 tx=0x7ff5f4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:23.353 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.599+0000 7ff6097fa640 1 -- 172.21.15.103:0/183012129 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff5fc045520 con 0x7ff60c106780 2024-09-19T06:40:23.353 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.766+0000 7ff612431640 1 -- 172.21.15.103:0/183012129 --> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7ff60c1031c0 con 0x7ff5f80391b0 2024-09-19T06:40:23.353 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.767+0000 7ff6097fa640 1 -- 172.21.15.103:0/183012129 <== mgr.14118 v2:172.21.15.103:6800/4122140716 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7ff60c1031c0 con 0x7ff5f80391b0 2024-09-19T06:40:23.353 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.769+0000 7ff612431640 1 -- 172.21.15.103:0/183012129 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7ff5f80391b0 msgr2=0x7ff5f803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:23.353 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.769+0000 7ff612431640 1 --2- 172.21.15.103:0/183012129 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7ff5f80391b0 0x7ff5f803b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40098c0 tx=0x7ff5f4006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:23.353 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.769+0000 7ff612431640 1 -- 172.21.15.103:0/183012129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c106780 msgr2=0x7ff60c19a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:23.353 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.769+0000 7ff612431640 1 --2- 172.21.15.103:0/183012129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c106780 0x7ff60c19a820 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0097f0 tx=0x7ff5fc030ec0 comp rx=0 tx=0).stop 2024-09-19T06:40:23.353 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.769+0000 7ff612431640 1 -- 172.21.15.103:0/183012129 shutdown_connections 2024-09-19T06:40:23.353 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.769+0000 7ff612431640 1 --2- 172.21.15.103:0/183012129 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7ff5f80391b0 0x7ff5f803b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:23.354 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.769+0000 7ff612431640 1 --2- 172.21.15.103:0/183012129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c106780 0x7ff60c19a820 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:23.354 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.769+0000 7ff612431640 1 -- 172.21.15.103:0/183012129 >> 172.21.15.103:0/183012129 conn(0x7ff60c101f10 msgr2=0x7ff60c102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:23.354 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.769+0000 7ff612431640 1 -- 172.21.15.103:0/183012129 shutdown_connections 2024-09-19T06:40:23.354 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:22.769+0000 7ff612431640 1 -- 172.21.15.103:0/183012129 wait complete. 2024-09-19T06:40:23.354 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T06:40:23.354 INFO:teuthology.orchestra.run.smithi103.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T06:40:23.357 INFO:teuthology.orchestra.run.smithi103.stdout:Adding host smithi103... 2024-09-19T06:40:23.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:23 smithi103 ceph-mon[31158]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:25.025 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:24 smithi103 ceph-mon[31158]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi103", "addr": "172.21.15.103", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:26.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:25 smithi103 ceph-mon[31158]: Deploying cephadm binary to smithi103 2024-09-19T06:40:27.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Added host 'smithi103' with addr '172.21.15.103' 2024-09-19T06:40:27.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.799+0000 7fd334a6e640 1 Processor -- start 2024-09-19T06:40:27.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.799+0000 7fd334a6e640 1 -- start start 2024-09-19T06:40:27.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.799+0000 7fd334a6e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3301078b0 0x7fd330107cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:27.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.799+0000 7fd334a6e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd330108280 con 0x7fd3301078b0 2024-09-19T06:40:27.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.799+0000 7fd32e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3301078b0 0x7fd330107cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:27.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.799+0000 7fd32e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3301078b0 0x7fd330107cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41164/0 (socket says 172.21.15.103:41164) 2024-09-19T06:40:27.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.799+0000 7fd32e59c640 1 -- 172.21.15.103:0/3304539781 learned_addr learned my addr 172.21.15.103:0/3304539781 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:27.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.799+0000 7fd32e59c640 1 -- 172.21.15.103:0/3304539781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd330108a60 con 0x7fd3301078b0 2024-09-19T06:40:27.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.800+0000 7fd32e59c640 1 --2- 172.21.15.103:0/3304539781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3301078b0 0x7fd330107cb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd320009820 tx=0x7fd32002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d916c44467073878 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:27.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.800+0000 7fd32d59a640 1 -- 172.21.15.103:0/3304539781 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd320037070 con 0x7fd3301078b0 2024-09-19T06:40:27.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.800+0000 7fd32d59a640 1 -- 172.21.15.103:0/3304539781 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd320032900 con 0x7fd3301078b0 2024-09-19T06:40:27.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.800+0000 7fd32d59a640 1 -- 172.21.15.103:0/3304539781 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd320030850 con 0x7fd3301078b0 2024-09-19T06:40:27.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.801+0000 7fd334a6e640 1 -- 172.21.15.103:0/3304539781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3301078b0 msgr2=0x7fd330107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:27.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.801+0000 7fd334a6e640 1 --2- 172.21.15.103:0/3304539781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3301078b0 0x7fd330107cb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd320009820 tx=0x7fd32002a270 comp rx=0 tx=0).stop 2024-09-19T06:40:27.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.801+0000 7fd334a6e640 1 -- 172.21.15.103:0/3304539781 shutdown_connections 2024-09-19T06:40:27.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.801+0000 7fd334a6e640 1 --2- 172.21.15.103:0/3304539781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3301078b0 0x7fd330107cb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:27.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.801+0000 7fd334a6e640 1 -- 172.21.15.103:0/3304539781 >> 172.21.15.103:0/3304539781 conn(0x7fd330103060 msgr2=0x7fd330105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:27.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.801+0000 7fd334a6e640 1 -- 172.21.15.103:0/3304539781 shutdown_connections 2024-09-19T06:40:27.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.801+0000 7fd334a6e640 1 -- 172.21.15.103:0/3304539781 wait complete. 2024-09-19T06:40:27.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.802+0000 7fd334a6e640 1 Processor -- start 2024-09-19T06:40:27.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.802+0000 7fd334a6e640 1 -- start start 2024-09-19T06:40:27.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.802+0000 7fd334a6e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3301078b0 0x7fd33007db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:27.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.802+0000 7fd334a6e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd33010be10 con 0x7fd3301078b0 2024-09-19T06:40:27.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.802+0000 7fd32e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3301078b0 0x7fd33007db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:27.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.802+0000 7fd32e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3301078b0 0x7fd33007db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41178/0 (socket says 172.21.15.103:41178) 2024-09-19T06:40:27.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.802+0000 7fd32e59c640 1 -- 172.21.15.103:0/800488104 learned_addr learned my addr 172.21.15.103:0/800488104 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:27.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.802+0000 7fd32e59c640 1 -- 172.21.15.103:0/800488104 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33007e040 con 0x7fd3301078b0 2024-09-19T06:40:27.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.803+0000 7fd32e59c640 1 --2- 172.21.15.103:0/800488104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3301078b0 0x7fd33007db00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd3200097f0 tx=0x7fd320030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:27.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.803+0000 7fd31f7fe640 1 -- 172.21.15.103:0/800488104 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd320042070 con 0x7fd3301078b0 2024-09-19T06:40:27.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.803+0000 7fd31f7fe640 1 -- 172.21.15.103:0/800488104 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd320032ca0 con 0x7fd3301078b0 2024-09-19T06:40:27.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.803+0000 7fd334a6e640 1 -- 172.21.15.103:0/800488104 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33007c230 con 0x7fd3301078b0 2024-09-19T06:40:27.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.803+0000 7fd31f7fe640 1 -- 172.21.15.103:0/800488104 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd320037040 con 0x7fd3301078b0 2024-09-19T06:40:27.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.803+0000 7fd334a6e640 1 -- 172.21.15.103:0/800488104 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd33007c770 con 0x7fd3301078b0 2024-09-19T06:40:27.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.804+0000 7fd31f7fe640 1 -- 172.21.15.103:0/800488104 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fd320045070 con 0x7fd3301078b0 2024-09-19T06:40:27.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.804+0000 7fd334a6e640 1 -- 172.21.15.103:0/800488104 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd33010bf90 con 0x7fd3301078b0 2024-09-19T06:40:27.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.804+0000 7fd31f7fe640 1 --2- 172.21.15.103:0/800488104 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fd318039200 0x7fd31803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:27.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.804+0000 7fd31f7fe640 1 -- 172.21.15.103:0/800488104 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd32006c1d0 con 0x7fd3301078b0 2024-09-19T06:40:27.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.804+0000 7fd32dd9b640 1 --2- 172.21.15.103:0/800488104 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fd318039200 0x7fd31803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:27.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.805+0000 7fd32dd9b640 1 --2- 172.21.15.103:0/800488104 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fd318039200 0x7fd31803b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd3240098c0 tx=0x7fd324006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:27.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.809+0000 7fd31f7fe640 1 -- 172.21.15.103:0/800488104 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd320045360 con 0x7fd3301078b0 2024-09-19T06:40:27.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:23.981+0000 7fd334a6e640 1 -- 172.21.15.103:0/800488104 --> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi103", "addr": "172.21.15.103", "target": ["mon-mgr", ""]}) -- 0x7fd33007cff0 con 0x7fd318039200 2024-09-19T06:40:27.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:27.206+0000 7fd31f7fe640 1 -- 172.21.15.103:0/800488104 <== mgr.14118 v2:172.21.15.103:6800/4122140716 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd33007cff0 con 0x7fd318039200 2024-09-19T06:40:27.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:27.209+0000 7fd334a6e640 1 -- 172.21.15.103:0/800488104 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fd318039200 msgr2=0x7fd31803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:27.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:27.209+0000 7fd334a6e640 1 --2- 172.21.15.103:0/800488104 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fd318039200 0x7fd31803b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd3240098c0 tx=0x7fd324006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:27.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:27.209+0000 7fd334a6e640 1 -- 172.21.15.103:0/800488104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3301078b0 msgr2=0x7fd33007db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:27.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:27.209+0000 7fd334a6e640 1 --2- 172.21.15.103:0/800488104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3301078b0 0x7fd33007db00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd3200097f0 tx=0x7fd320030ec0 comp rx=0 tx=0).stop 2024-09-19T06:40:27.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:27.209+0000 7fd334a6e640 1 -- 172.21.15.103:0/800488104 shutdown_connections 2024-09-19T06:40:27.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:27.209+0000 7fd334a6e640 1 --2- 172.21.15.103:0/800488104 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7fd318039200 0x7fd31803b6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:27.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:27.209+0000 7fd334a6e640 1 --2- 172.21.15.103:0/800488104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd3301078b0 0x7fd33007db00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:27.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:27.209+0000 7fd334a6e640 1 -- 172.21.15.103:0/800488104 >> 172.21.15.103:0/800488104 conn(0x7fd330103060 msgr2=0x7fd330103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:27.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:27.209+0000 7fd334a6e640 1 -- 172.21.15.103:0/800488104 shutdown_connections 2024-09-19T06:40:27.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:27.209+0000 7fd334a6e640 1 -- 172.21.15.103:0/800488104 wait complete. 2024-09-19T06:40:27.770 INFO:teuthology.orchestra.run.smithi103.stdout:Deploying unmanaged mon service... 2024-09-19T06:40:28.430 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:28 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' 2024-09-19T06:40:28.430 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:28 smithi103 ceph-mon[31158]: Added host smithi103 2024-09-19T06:40:28.430 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:28 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:40:29.000 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T06:40:29.000 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.240+0000 7f6ba82bc640 1 Processor -- start 2024-09-19T06:40:29.000 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.240+0000 7f6ba82bc640 1 -- start start 2024-09-19T06:40:29.001 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.241+0000 7f6ba82bc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba01078b0 0x7f6ba0107cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:29.001 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.241+0000 7f6ba82bc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba0108280 con 0x7f6ba01078b0 2024-09-19T06:40:29.001 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.241+0000 7f6ba6058640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba01078b0 0x7f6ba0107cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:29.001 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.241+0000 7f6ba6058640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba01078b0 0x7f6ba0107cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51726/0 (socket says 172.21.15.103:51726) 2024-09-19T06:40:29.001 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.241+0000 7f6ba6058640 1 -- 172.21.15.103:0/4127617868 learned_addr learned my addr 172.21.15.103:0/4127617868 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:29.001 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.241+0000 7f6ba6058640 1 -- 172.21.15.103:0/4127617868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba0108a60 con 0x7f6ba01078b0 2024-09-19T06:40:29.001 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.242+0000 7f6ba6058640 1 --2- 172.21.15.103:0/4127617868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba01078b0 0x7f6ba0107cb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009820 tx=0x7f6b9c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b91ef08c59c50b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:29.001 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.242+0000 7f6ba5056640 1 -- 172.21.15.103:0/4127617868 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b9c037070 con 0x7f6ba01078b0 2024-09-19T06:40:29.001 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.242+0000 7f6ba5056640 1 -- 172.21.15.103:0/4127617868 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6b9c032900 con 0x7f6ba01078b0 2024-09-19T06:40:29.001 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.242+0000 7f6ba5056640 1 -- 172.21.15.103:0/4127617868 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b9c030850 con 0x7f6ba01078b0 2024-09-19T06:40:29.001 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.243+0000 7f6ba82bc640 1 -- 172.21.15.103:0/4127617868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba01078b0 msgr2=0x7f6ba0107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:29.001 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.243+0000 7f6ba82bc640 1 --2- 172.21.15.103:0/4127617868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba01078b0 0x7f6ba0107cb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009820 tx=0x7f6b9c02a270 comp rx=0 tx=0).stop 2024-09-19T06:40:29.002 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.243+0000 7f6ba82bc640 1 -- 172.21.15.103:0/4127617868 shutdown_connections 2024-09-19T06:40:29.002 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.243+0000 7f6ba82bc640 1 --2- 172.21.15.103:0/4127617868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba01078b0 0x7f6ba0107cb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.002 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.243+0000 7f6ba82bc640 1 -- 172.21.15.103:0/4127617868 >> 172.21.15.103:0/4127617868 conn(0x7f6ba0103060 msgr2=0x7f6ba0105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:29.002 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.243+0000 7f6ba82bc640 1 -- 172.21.15.103:0/4127617868 shutdown_connections 2024-09-19T06:40:29.002 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.243+0000 7f6ba82bc640 1 -- 172.21.15.103:0/4127617868 wait complete. 2024-09-19T06:40:29.002 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.243+0000 7f6ba82bc640 1 Processor -- start 2024-09-19T06:40:29.002 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.243+0000 7f6ba82bc640 1 -- start start 2024-09-19T06:40:29.002 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.243+0000 7f6ba82bc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba01078b0 0x7f6ba019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:29.002 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.243+0000 7f6ba82bc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba010be10 con 0x7f6ba01078b0 2024-09-19T06:40:29.002 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.244+0000 7f6ba6058640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba01078b0 0x7f6ba019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:29.002 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.244+0000 7f6ba6058640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba01078b0 0x7f6ba019b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51736/0 (socket says 172.21.15.103:51736) 2024-09-19T06:40:29.002 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.244+0000 7f6ba6058640 1 -- 172.21.15.103:0/1677829333 learned_addr learned my addr 172.21.15.103:0/1677829333 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:29.002 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.244+0000 7f6ba6058640 1 -- 172.21.15.103:0/1677829333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba019bdd0 con 0x7f6ba01078b0 2024-09-19T06:40:29.002 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.244+0000 7f6ba6058640 1 --2- 172.21.15.103:0/1677829333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba01078b0 0x7f6ba019b890 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009940 tx=0x7f6b9c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:29.003 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.244+0000 7f6b977fe640 1 -- 172.21.15.103:0/1677829333 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b9c042070 con 0x7f6ba01078b0 2024-09-19T06:40:29.003 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.244+0000 7f6b977fe640 1 -- 172.21.15.103:0/1677829333 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6b9c032be0 con 0x7f6ba01078b0 2024-09-19T06:40:29.003 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.244+0000 7f6ba82bc640 1 -- 172.21.15.103:0/1677829333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba019c060 con 0x7f6ba01078b0 2024-09-19T06:40:29.003 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.244+0000 7f6b977fe640 1 -- 172.21.15.103:0/1677829333 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b9c037040 con 0x7f6ba01078b0 2024-09-19T06:40:29.003 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.244+0000 7f6ba82bc640 1 -- 172.21.15.103:0/1677829333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba019ed50 con 0x7f6ba01078b0 2024-09-19T06:40:29.003 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.245+0000 7f6b977fe640 1 -- 172.21.15.103:0/1677829333 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f6b9c045070 con 0x7f6ba01078b0 2024-09-19T06:40:29.003 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.245+0000 7f6b977fe640 1 --2- 172.21.15.103:0/1677829333 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f6b8c039200 0x7f6b8c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:29.003 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.245+0000 7f6b977fe640 1 -- 172.21.15.103:0/1677829333 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6b9c06c1e0 con 0x7f6ba01078b0 2024-09-19T06:40:29.003 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.245+0000 7f6ba5857640 1 --2- 172.21.15.103:0/1677829333 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f6b8c039200 0x7f6b8c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:29.003 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.245+0000 7f6ba82bc640 1 -- 172.21.15.103:0/1677829333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba010bf90 con 0x7f6ba01078b0 2024-09-19T06:40:29.003 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.246+0000 7f6ba5857640 1 --2- 172.21.15.103:0/1677829333 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f6b8c039200 0x7f6b8c03b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6b900098c0 tx=0x7f6b90006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:29.003 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.250+0000 7f6b977fe640 1 -- 172.21.15.103:0/1677829333 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6b9c045360 con 0x7f6ba01078b0 2024-09-19T06:40:29.003 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.429+0000 7f6ba82bc640 1 -- 172.21.15.103:0/1677829333 --> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6ba01043a0 con 0x7f6b8c039200 2024-09-19T06:40:29.004 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.434+0000 7f6b977fe640 1 -- 172.21.15.103:0/1677829333 <== mgr.14118 v2:172.21.15.103:6800/4122140716 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6ba01043a0 con 0x7f6b8c039200 2024-09-19T06:40:29.004 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.437+0000 7f6ba82bc640 1 -- 172.21.15.103:0/1677829333 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f6b8c039200 msgr2=0x7f6b8c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:29.004 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.437+0000 7f6ba82bc640 1 --2- 172.21.15.103:0/1677829333 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f6b8c039200 0x7f6b8c03b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6b900098c0 tx=0x7f6b90006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.004 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.437+0000 7f6ba82bc640 1 -- 172.21.15.103:0/1677829333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba01078b0 msgr2=0x7f6ba019b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:29.004 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.437+0000 7f6ba82bc640 1 --2- 172.21.15.103:0/1677829333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba01078b0 0x7f6ba019b890 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009940 tx=0x7f6b9c0326e0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.004 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.437+0000 7f6ba82bc640 1 -- 172.21.15.103:0/1677829333 shutdown_connections 2024-09-19T06:40:29.004 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.437+0000 7f6ba82bc640 1 --2- 172.21.15.103:0/1677829333 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f6b8c039200 0x7f6b8c03b6c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.004 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.437+0000 7f6ba82bc640 1 --2- 172.21.15.103:0/1677829333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba01078b0 0x7f6ba019b890 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.004 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.437+0000 7f6ba82bc640 1 -- 172.21.15.103:0/1677829333 >> 172.21.15.103:0/1677829333 conn(0x7f6ba0103060 msgr2=0x7f6ba0103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:29.004 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.437+0000 7f6ba82bc640 1 -- 172.21.15.103:0/1677829333 shutdown_connections 2024-09-19T06:40:29.004 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:28.437+0000 7f6ba82bc640 1 -- 172.21.15.103:0/1677829333 wait complete. 2024-09-19T06:40:29.004 INFO:teuthology.orchestra.run.smithi103.stdout:Deploying unmanaged mgr service... 2024-09-19T06:40:29.574 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:29 smithi103 ceph-mon[31158]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:29.574 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:29 smithi103 ceph-mon[31158]: Saving service mon spec with placement count:5 2024-09-19T06:40:29.574 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:29 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' 2024-09-19T06:40:30.166 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T06:40:30.167 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.382+0000 7f7d0760b640 1 Processor -- start 2024-09-19T06:40:30.167 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.382+0000 7f7d0760b640 1 -- start start 2024-09-19T06:40:30.167 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.382+0000 7f7d0760b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d001078d0 0x7f7d00107cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:30.167 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.382+0000 7f7d0760b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d001082a0 con 0x7f7d001078d0 2024-09-19T06:40:30.167 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.383+0000 7f7d053a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d001078d0 0x7f7d00107cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:30.167 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.383+0000 7f7d053a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d001078d0 0x7f7d00107cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51744/0 (socket says 172.21.15.103:51744) 2024-09-19T06:40:30.167 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.383+0000 7f7d053a7640 1 -- 172.21.15.103:0/3275346798 learned_addr learned my addr 172.21.15.103:0/3275346798 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:30.167 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.383+0000 7f7d053a7640 1 -- 172.21.15.103:0/3275346798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d00108a80 con 0x7f7d001078d0 2024-09-19T06:40:30.167 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.383+0000 7f7d053a7640 1 --2- 172.21.15.103:0/3275346798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d001078d0 0x7f7d00107cd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009820 tx=0x7f7cfc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6485b027f7cb12eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:30.167 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.384+0000 7f7cf7fff640 1 -- 172.21.15.103:0/3275346798 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cfc037070 con 0x7f7d001078d0 2024-09-19T06:40:30.167 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.384+0000 7f7cf7fff640 1 -- 172.21.15.103:0/3275346798 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7cfc032900 con 0x7f7d001078d0 2024-09-19T06:40:30.168 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.384+0000 7f7cf7fff640 1 -- 172.21.15.103:0/3275346798 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cfc030850 con 0x7f7d001078d0 2024-09-19T06:40:30.168 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.384+0000 7f7d0760b640 1 -- 172.21.15.103:0/3275346798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d001078d0 msgr2=0x7f7d00107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:30.168 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.384+0000 7f7d0760b640 1 --2- 172.21.15.103:0/3275346798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d001078d0 0x7f7d00107cd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009820 tx=0x7f7cfc02a270 comp rx=0 tx=0).stop 2024-09-19T06:40:30.168 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.384+0000 7f7d0760b640 1 -- 172.21.15.103:0/3275346798 shutdown_connections 2024-09-19T06:40:30.168 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.384+0000 7f7d0760b640 1 --2- 172.21.15.103:0/3275346798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d001078d0 0x7f7d00107cd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.168 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.384+0000 7f7d0760b640 1 -- 172.21.15.103:0/3275346798 >> 172.21.15.103:0/3275346798 conn(0x7f7d00103080 msgr2=0x7f7d001054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:30.168 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.384+0000 7f7d0760b640 1 -- 172.21.15.103:0/3275346798 shutdown_connections 2024-09-19T06:40:30.168 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.384+0000 7f7d0760b640 1 -- 172.21.15.103:0/3275346798 wait complete. 2024-09-19T06:40:30.168 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.385+0000 7f7d0760b640 1 Processor -- start 2024-09-19T06:40:30.168 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.385+0000 7f7d0760b640 1 -- start start 2024-09-19T06:40:30.168 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.385+0000 7f7d0760b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d001078d0 0x7f7d0007db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:30.168 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.385+0000 7f7d0760b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0010be30 con 0x7f7d001078d0 2024-09-19T06:40:30.168 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.385+0000 7f7d053a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d001078d0 0x7f7d0007db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:30.168 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.385+0000 7f7d053a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d001078d0 0x7f7d0007db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51760/0 (socket says 172.21.15.103:51760) 2024-09-19T06:40:30.169 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.385+0000 7f7d053a7640 1 -- 172.21.15.103:0/674359604 learned_addr learned my addr 172.21.15.103:0/674359604 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:30.169 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.385+0000 7f7d053a7640 1 -- 172.21.15.103:0/674359604 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0007e040 con 0x7f7d001078d0 2024-09-19T06:40:30.169 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.386+0000 7f7d053a7640 1 --2- 172.21.15.103:0/674359604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d001078d0 0x7f7d0007db00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0097f0 tx=0x7f7cfc030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:30.169 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.386+0000 7f7cf67fc640 1 -- 172.21.15.103:0/674359604 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cfc042070 con 0x7f7d001078d0 2024-09-19T06:40:30.169 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.386+0000 7f7cf67fc640 1 -- 172.21.15.103:0/674359604 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7cfc032ca0 con 0x7f7d001078d0 2024-09-19T06:40:30.169 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.386+0000 7f7d0760b640 1 -- 172.21.15.103:0/674359604 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0007c230 con 0x7f7d001078d0 2024-09-19T06:40:30.169 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.386+0000 7f7cf67fc640 1 -- 172.21.15.103:0/674359604 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cfc037040 con 0x7f7d001078d0 2024-09-19T06:40:30.169 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.386+0000 7f7d0760b640 1 -- 172.21.15.103:0/674359604 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0007c770 con 0x7f7d001078d0 2024-09-19T06:40:30.169 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.387+0000 7f7cf67fc640 1 -- 172.21.15.103:0/674359604 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f7cfc045070 con 0x7f7d001078d0 2024-09-19T06:40:30.169 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.387+0000 7f7d0760b640 1 -- 172.21.15.103:0/674359604 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d00107d50 con 0x7f7d001078d0 2024-09-19T06:40:30.169 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.387+0000 7f7cf67fc640 1 --2- 172.21.15.103:0/674359604 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f7cec039200 0x7f7cec03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:30.169 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.387+0000 7f7cf67fc640 1 -- 172.21.15.103:0/674359604 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7cfc06c350 con 0x7f7d001078d0 2024-09-19T06:40:30.170 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.387+0000 7f7d04ba6640 1 --2- 172.21.15.103:0/674359604 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f7cec039200 0x7f7cec03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:30.170 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.387+0000 7f7d04ba6640 1 --2- 172.21.15.103:0/674359604 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f7cec039200 0x7f7cec03b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00098c0 tx=0x7f7cf0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:30.170 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.391+0000 7f7cf67fc640 1 -- 172.21.15.103:0/674359604 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7cfc045360 con 0x7f7d001078d0 2024-09-19T06:40:30.170 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.573+0000 7f7d0760b640 1 -- 172.21.15.103:0/674359604 --> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f7d001043c0 con 0x7f7cec039200 2024-09-19T06:40:30.170 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.577+0000 7f7cf67fc640 1 -- 172.21.15.103:0/674359604 <== mgr.14118 v2:172.21.15.103:6800/4122140716 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7d001043c0 con 0x7f7cec039200 2024-09-19T06:40:30.170 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.579+0000 7f7d0760b640 1 -- 172.21.15.103:0/674359604 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f7cec039200 msgr2=0x7f7cec03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:30.170 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.579+0000 7f7d0760b640 1 --2- 172.21.15.103:0/674359604 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f7cec039200 0x7f7cec03b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00098c0 tx=0x7f7cf0006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.170 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.579+0000 7f7d0760b640 1 -- 172.21.15.103:0/674359604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d001078d0 msgr2=0x7f7d0007db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:30.170 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.579+0000 7f7d0760b640 1 --2- 172.21.15.103:0/674359604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d001078d0 0x7f7d0007db00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0097f0 tx=0x7f7cfc030ec0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.170 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.579+0000 7f7d0760b640 1 -- 172.21.15.103:0/674359604 shutdown_connections 2024-09-19T06:40:30.170 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.579+0000 7f7d0760b640 1 --2- 172.21.15.103:0/674359604 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f7cec039200 0x7f7cec03b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.170 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.579+0000 7f7d0760b640 1 --2- 172.21.15.103:0/674359604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d001078d0 0x7f7d0007db00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.170 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.579+0000 7f7d0760b640 1 -- 172.21.15.103:0/674359604 >> 172.21.15.103:0/674359604 conn(0x7f7d00103080 msgr2=0x7f7d00103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:30.171 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.579+0000 7f7d0760b640 1 -- 172.21.15.103:0/674359604 shutdown_connections 2024-09-19T06:40:30.171 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:29.579+0000 7f7d0760b640 1 -- 172.21.15.103:0/674359604 wait complete. 2024-09-19T06:40:30.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:30 smithi103 ceph-mon[31158]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:30.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:30 smithi103 ceph-mon[31158]: Saving service mgr spec with placement count:2 2024-09-19T06:40:30.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:30 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' 2024-09-19T06:40:32.046 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:31 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3355219771' entity='client.admin' 2024-09-19T06:40:32.046 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:31 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' 2024-09-19T06:40:32.046 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:31 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' 2024-09-19T06:40:32.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.584+0000 7f2c797df640 1 Processor -- start 2024-09-19T06:40:32.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.584+0000 7f2c797df640 1 -- start start 2024-09-19T06:40:32.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.584+0000 7f2c797df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c741083a0 0x7f2c7410a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:32.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.584+0000 7f2c797df640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7407bd80 con 0x7f2c741083a0 2024-09-19T06:40:32.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.584+0000 7f2c72ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c741083a0 0x7f2c7410a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:32.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.584+0000 7f2c72ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c741083a0 0x7f2c7410a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51768/0 (socket says 172.21.15.103:51768) 2024-09-19T06:40:32.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.584+0000 7f2c72ffd640 1 -- 172.21.15.103:0/3184864574 learned_addr learned my addr 172.21.15.103:0/3184864574 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:32.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.584+0000 7f2c72ffd640 1 -- 172.21.15.103:0/3184864574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7410acd0 con 0x7f2c741083a0 2024-09-19T06:40:32.053 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.585+0000 7f2c72ffd640 1 --2- 172.21.15.103:0/3184864574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c741083a0 0x7f2c7410a790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009820 tx=0x7f2c6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae6955c25179a9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:32.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.585+0000 7f2c71ffb640 1 -- 172.21.15.103:0/3184864574 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c64037070 con 0x7f2c741083a0 2024-09-19T06:40:32.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.585+0000 7f2c71ffb640 1 -- 172.21.15.103:0/3184864574 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c64032900 con 0x7f2c741083a0 2024-09-19T06:40:32.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.585+0000 7f2c71ffb640 1 -- 172.21.15.103:0/3184864574 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c64030850 con 0x7f2c741083a0 2024-09-19T06:40:32.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.586+0000 7f2c797df640 1 -- 172.21.15.103:0/3184864574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c741083a0 msgr2=0x7f2c7410a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:32.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.586+0000 7f2c797df640 1 --2- 172.21.15.103:0/3184864574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c741083a0 0x7f2c7410a790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009820 tx=0x7f2c6402a270 comp rx=0 tx=0).stop 2024-09-19T06:40:32.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.586+0000 7f2c797df640 1 -- 172.21.15.103:0/3184864574 shutdown_connections 2024-09-19T06:40:32.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.586+0000 7f2c797df640 1 --2- 172.21.15.103:0/3184864574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c741083a0 0x7f2c7410a790 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:32.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.586+0000 7f2c797df640 1 -- 172.21.15.103:0/3184864574 >> 172.21.15.103:0/3184864574 conn(0x7f2c74101f40 msgr2=0x7f2c74104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:32.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.586+0000 7f2c797df640 1 -- 172.21.15.103:0/3184864574 shutdown_connections 2024-09-19T06:40:32.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.586+0000 7f2c797df640 1 -- 172.21.15.103:0/3184864574 wait complete. 2024-09-19T06:40:32.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.586+0000 7f2c797df640 1 Processor -- start 2024-09-19T06:40:32.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.587+0000 7f2c797df640 1 -- start start 2024-09-19T06:40:32.054 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.587+0000 7f2c797df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c741083a0 0x7f2c7419b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:32.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.587+0000 7f2c797df640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7410bcb0 con 0x7f2c741083a0 2024-09-19T06:40:32.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.587+0000 7f2c72ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c741083a0 0x7f2c7419b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:32.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.587+0000 7f2c72ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c741083a0 0x7f2c7419b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51782/0 (socket says 172.21.15.103:51782) 2024-09-19T06:40:32.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.587+0000 7f2c72ffd640 1 -- 172.21.15.103:0/3355219771 learned_addr learned my addr 172.21.15.103:0/3355219771 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:32.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.587+0000 7f2c72ffd640 1 -- 172.21.15.103:0/3355219771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7419bd40 con 0x7f2c741083a0 2024-09-19T06:40:32.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.587+0000 7f2c72ffd640 1 --2- 172.21.15.103:0/3355219771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c741083a0 0x7f2c7419b800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2c640097f0 tx=0x7f2c64030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:32.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.587+0000 7f2c5bfff640 1 -- 172.21.15.103:0/3355219771 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c64037040 con 0x7f2c741083a0 2024-09-19T06:40:32.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.587+0000 7f2c5bfff640 1 -- 172.21.15.103:0/3355219771 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c64040070 con 0x7f2c741083a0 2024-09-19T06:40:32.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.587+0000 7f2c797df640 1 -- 172.21.15.103:0/3355219771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7419bfd0 con 0x7f2c741083a0 2024-09-19T06:40:32.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.587+0000 7f2c5bfff640 1 -- 172.21.15.103:0/3355219771 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c6403a390 con 0x7f2c741083a0 2024-09-19T06:40:32.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.587+0000 7f2c797df640 1 -- 172.21.15.103:0/3355219771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7419ecc0 con 0x7f2c741083a0 2024-09-19T06:40:32.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.588+0000 7f2c5bfff640 1 -- 172.21.15.103:0/3355219771 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f2c6403a5e0 con 0x7f2c741083a0 2024-09-19T06:40:32.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.589+0000 7f2c797df640 1 -- 172.21.15.103:0/3355219771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c74108800 con 0x7f2c741083a0 2024-09-19T06:40:32.055 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.589+0000 7f2c5bfff640 1 --2- 172.21.15.103:0/3355219771 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f2c5c039200 0x7f2c5c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:32.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.589+0000 7f2c5bfff640 1 -- 172.21.15.103:0/3355219771 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2c6406c8a0 con 0x7f2c741083a0 2024-09-19T06:40:32.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.589+0000 7f2c727fc640 1 --2- 172.21.15.103:0/3355219771 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f2c5c039200 0x7f2c5c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:32.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.589+0000 7f2c727fc640 1 --2- 172.21.15.103:0/3355219771 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f2c5c039200 0x7f2c5c03b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2c680098c0 tx=0x7f2c68006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:32.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.593+0000 7f2c5bfff640 1 -- 172.21.15.103:0/3355219771 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2c6403a9d0 con 0x7f2c741083a0 2024-09-19T06:40:32.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.761+0000 7f2c797df640 1 -- 172.21.15.103:0/3355219771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f2c7419f240 con 0x7f2c741083a0 2024-09-19T06:40:32.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.766+0000 7f2c5bfff640 1 -- 172.21.15.103:0/3355219771 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f2c64045460 con 0x7f2c741083a0 2024-09-19T06:40:32.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.769+0000 7f2c797df640 1 -- 172.21.15.103:0/3355219771 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f2c5c039200 msgr2=0x7f2c5c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:32.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.769+0000 7f2c797df640 1 --2- 172.21.15.103:0/3355219771 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f2c5c039200 0x7f2c5c03b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2c680098c0 tx=0x7f2c68006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:32.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.769+0000 7f2c797df640 1 -- 172.21.15.103:0/3355219771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c741083a0 msgr2=0x7f2c7419b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:32.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.769+0000 7f2c797df640 1 --2- 172.21.15.103:0/3355219771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c741083a0 0x7f2c7419b800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2c640097f0 tx=0x7f2c64030d80 comp rx=0 tx=0).stop 2024-09-19T06:40:32.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.769+0000 7f2c797df640 1 -- 172.21.15.103:0/3355219771 shutdown_connections 2024-09-19T06:40:32.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.769+0000 7f2c797df640 1 --2- 172.21.15.103:0/3355219771 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f2c5c039200 0x7f2c5c03b6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:32.056 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.769+0000 7f2c797df640 1 --2- 172.21.15.103:0/3355219771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c741083a0 0x7f2c7419b800 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:32.057 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.769+0000 7f2c797df640 1 -- 172.21.15.103:0/3355219771 >> 172.21.15.103:0/3355219771 conn(0x7f2c74101f40 msgr2=0x7f2c74102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:32.057 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.769+0000 7f2c797df640 1 -- 172.21.15.103:0/3355219771 shutdown_connections 2024-09-19T06:40:32.057 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:30.769+0000 7f2c797df640 1 -- 172.21.15.103:0/3355219771 wait complete. 2024-09-19T06:40:33.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.461+0000 7f71034f0640 1 Processor -- start 2024-09-19T06:40:33.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.462+0000 7f71034f0640 1 -- start start 2024-09-19T06:40:33.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.462+0000 7f71034f0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70fc07de00 0x7f70fc07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:33.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.462+0000 7f71034f0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc07c880 con 0x7f70fc07de00 2024-09-19T06:40:33.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.462+0000 7f710128c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70fc07de00 0x7f70fc07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:33.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.462+0000 7f710128c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70fc07de00 0x7f70fc07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51794/0 (socket says 172.21.15.103:51794) 2024-09-19T06:40:33.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.462+0000 7f710128c640 1 -- 172.21.15.103:0/1400363631 learned_addr learned my addr 172.21.15.103:0/1400363631 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:33.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.462+0000 7f710128c640 1 -- 172.21.15.103:0/1400363631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70fc07ca00 con 0x7f70fc07de00 2024-09-19T06:40:33.446 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.463+0000 7f710128c640 1 --2- 172.21.15.103:0/1400363631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70fc07de00 0x7f70fc07c2b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009820 tx=0x7f70f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f097178c5cbd7d09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:33.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.463+0000 7f70f3fff640 1 -- 172.21.15.103:0/1400363631 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70f8037070 con 0x7f70fc07de00 2024-09-19T06:40:33.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.463+0000 7f70f3fff640 1 -- 172.21.15.103:0/1400363631 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f8032920 con 0x7f70fc07de00 2024-09-19T06:40:33.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.463+0000 7f70f3fff640 1 -- 172.21.15.103:0/1400363631 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70f8030810 con 0x7f70fc07de00 2024-09-19T06:40:33.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.463+0000 7f71034f0640 1 -- 172.21.15.103:0/1400363631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70fc07de00 msgr2=0x7f70fc07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:33.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.464+0000 7f71034f0640 1 --2- 172.21.15.103:0/1400363631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70fc07de00 0x7f70fc07c2b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009820 tx=0x7f70f802a270 comp rx=0 tx=0).stop 2024-09-19T06:40:33.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.464+0000 7f71034f0640 1 -- 172.21.15.103:0/1400363631 shutdown_connections 2024-09-19T06:40:33.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.464+0000 7f71034f0640 1 --2- 172.21.15.103:0/1400363631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70fc07de00 0x7f70fc07c2b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:33.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.464+0000 7f71034f0640 1 -- 172.21.15.103:0/1400363631 >> 172.21.15.103:0/1400363631 conn(0x7f70fc103060 msgr2=0x7f70fc105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:33.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.464+0000 7f71034f0640 1 -- 172.21.15.103:0/1400363631 shutdown_connections 2024-09-19T06:40:33.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.464+0000 7f71034f0640 1 -- 172.21.15.103:0/1400363631 wait complete. 2024-09-19T06:40:33.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.464+0000 7f71034f0640 1 Processor -- start 2024-09-19T06:40:33.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.464+0000 7f71034f0640 1 -- start start 2024-09-19T06:40:33.447 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.464+0000 7f71034f0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70fc07de00 0x7f70fc19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:33.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.464+0000 7f71034f0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc10df00 con 0x7f70fc07de00 2024-09-19T06:40:33.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.465+0000 7f710128c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70fc07de00 0x7f70fc19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:33.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.465+0000 7f710128c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70fc07de00 0x7f70fc19daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51800/0 (socket says 172.21.15.103:51800) 2024-09-19T06:40:33.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.465+0000 7f710128c640 1 -- 172.21.15.103:0/553796926 learned_addr learned my addr 172.21.15.103:0/553796926 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:33.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.465+0000 7f710128c640 1 -- 172.21.15.103:0/553796926 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70fc19dfe0 con 0x7f70fc07de00 2024-09-19T06:40:33.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.465+0000 7f710128c640 1 --2- 172.21.15.103:0/553796926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70fc07de00 0x7f70fc19daa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f70f8030420 tx=0x7f70f8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:33.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.465+0000 7f70f27fc640 1 -- 172.21.15.103:0/553796926 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70f8037040 con 0x7f70fc07de00 2024-09-19T06:40:33.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.465+0000 7f70f27fc640 1 -- 172.21.15.103:0/553796926 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f8040070 con 0x7f70fc07de00 2024-09-19T06:40:33.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.465+0000 7f70f27fc640 1 -- 172.21.15.103:0/553796926 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70f803a6e0 con 0x7f70fc07de00 2024-09-19T06:40:33.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.465+0000 7f71034f0640 1 -- 172.21.15.103:0/553796926 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70fc19e270 con 0x7f70fc07de00 2024-09-19T06:40:33.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.465+0000 7f71034f0640 1 -- 172.21.15.103:0/553796926 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70fc1a0f60 con 0x7f70fc07de00 2024-09-19T06:40:33.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.466+0000 7f70f27fc640 1 -- 172.21.15.103:0/553796926 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f70f803a990 con 0x7f70fc07de00 2024-09-19T06:40:33.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.466+0000 7f71034f0640 1 -- 172.21.15.103:0/553796926 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70fc07c330 con 0x7f70fc07de00 2024-09-19T06:40:33.448 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.466+0000 7f70f27fc640 1 --2- 172.21.15.103:0/553796926 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f70e8039200 0x7f70e803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:33.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.466+0000 7f70f27fc640 1 -- 172.21.15.103:0/553796926 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f70f806c880 con 0x7f70fc07de00 2024-09-19T06:40:33.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.466+0000 7f7100a8b640 1 --2- 172.21.15.103:0/553796926 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f70e8039200 0x7f70e803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:33.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.467+0000 7f7100a8b640 1 --2- 172.21.15.103:0/553796926 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f70e8039200 0x7f70e803b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f70ec0098c0 tx=0x7f70ec006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:33.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.471+0000 7f70f27fc640 1 -- 172.21.15.103:0/553796926 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f70f803bbd0 con 0x7f70fc07de00 2024-09-19T06:40:33.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.652+0000 7f71034f0640 1 -- 172.21.15.103:0/553796926 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f70fc1a1130 con 0x7f70fc07de00 2024-09-19T06:40:33.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.654+0000 7f70f27fc640 1 -- 172.21.15.103:0/553796926 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f70f803b420 con 0x7f70fc07de00 2024-09-19T06:40:33.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.657+0000 7f71034f0640 1 -- 172.21.15.103:0/553796926 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f70e8039200 msgr2=0x7f70e803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:33.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.657+0000 7f71034f0640 1 --2- 172.21.15.103:0/553796926 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f70e8039200 0x7f70e803b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f70ec0098c0 tx=0x7f70ec006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:33.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.657+0000 7f71034f0640 1 -- 172.21.15.103:0/553796926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70fc07de00 msgr2=0x7f70fc19daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:33.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.657+0000 7f71034f0640 1 --2- 172.21.15.103:0/553796926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70fc07de00 0x7f70fc19daa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f70f8030420 tx=0x7f70f8032670 comp rx=0 tx=0).stop 2024-09-19T06:40:33.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.657+0000 7f71034f0640 1 -- 172.21.15.103:0/553796926 shutdown_connections 2024-09-19T06:40:33.449 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.657+0000 7f71034f0640 1 --2- 172.21.15.103:0/553796926 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f70e8039200 0x7f70e803b6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:33.450 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.657+0000 7f71034f0640 1 --2- 172.21.15.103:0/553796926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70fc07de00 0x7f70fc19daa0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:33.450 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.657+0000 7f71034f0640 1 -- 172.21.15.103:0/553796926 >> 172.21.15.103:0/553796926 conn(0x7f70fc103060 msgr2=0x7f70fc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:33.450 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.657+0000 7f71034f0640 1 -- 172.21.15.103:0/553796926 shutdown_connections 2024-09-19T06:40:33.450 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:32.657+0000 7f71034f0640 1 -- 172.21.15.103:0/553796926 wait complete. 2024-09-19T06:40:33.450 INFO:teuthology.orchestra.run.smithi103.stdout:Enabling the dashboard module... 2024-09-19T06:40:33.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:33 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/553796926' entity='client.admin' 2024-09-19T06:40:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:34 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' 2024-09-19T06:40:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:34 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/197926378' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T06:40:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:34 smithi103 ceph-mon[31158]: from='mgr.14118 172.21.15.103:0/3896824416' entity='mgr.x' 2024-09-19T06:40:35.138 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:34 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: ignoring --setuser ceph since I am not root 2024-09-19T06:40:35.138 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:34 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: ignoring --setgroup ceph since I am not root 2024-09-19T06:40:35.138 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:34 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:34.968+0000 7f7212a3f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:40:35.139 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:35 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:35.030+0000 7f7212a3f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:40:35.543 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:35 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:35.181+0000 7f7212a3f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:40:35.543 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:35 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:35.248+0000 7f7212a3f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:40:35.543 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:35 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:35.400+0000 7f7212a3f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:40:35.543 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:35 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:35.542+0000 7f7212a3f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:40:35.951 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.207+0000 7f8dbfe17640 1 Processor -- start 2024-09-19T06:40:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.207+0000 7f8dbfe17640 1 -- start start 2024-09-19T06:40:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.208+0000 7f8dbfe17640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8db81078b0 0x7f8db8107cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.208+0000 7f8dbfe17640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8db8108280 con 0x7f8db81078b0 2024-09-19T06:40:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.208+0000 7f8dbdbb3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8db81078b0 0x7f8db8107cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.208+0000 7f8dbdbb3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8db81078b0 0x7f8db8107cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51812/0 (socket says 172.21.15.103:51812) 2024-09-19T06:40:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.208+0000 7f8dbdbb3640 1 -- 172.21.15.103:0/888009151 learned_addr learned my addr 172.21.15.103:0/888009151 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.208+0000 7f8dbdbb3640 1 -- 172.21.15.103:0/888009151 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db8108a60 con 0x7f8db81078b0 2024-09-19T06:40:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.209+0000 7f8dbdbb3640 1 --2- 172.21.15.103:0/888009151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8db81078b0 0x7f8db8107cb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8db4009820 tx=0x7f8db402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0792d296525baa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.209+0000 7f8dbcbb1640 1 -- 172.21.15.103:0/888009151 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8db4037070 con 0x7f8db81078b0 2024-09-19T06:40:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.209+0000 7f8dbcbb1640 1 -- 172.21.15.103:0/888009151 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8db4032920 con 0x7f8db81078b0 2024-09-19T06:40:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.209+0000 7f8dbcbb1640 1 -- 172.21.15.103:0/888009151 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8db4030810 con 0x7f8db81078b0 2024-09-19T06:40:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.210+0000 7f8dbfe17640 1 -- 172.21.15.103:0/888009151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8db81078b0 msgr2=0x7f8db8107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.210+0000 7f8dbfe17640 1 --2- 172.21.15.103:0/888009151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8db81078b0 0x7f8db8107cb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8db4009820 tx=0x7f8db402a270 comp rx=0 tx=0).stop 2024-09-19T06:40:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.210+0000 7f8dbfe17640 1 -- 172.21.15.103:0/888009151 shutdown_connections 2024-09-19T06:40:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.210+0000 7f8dbfe17640 1 --2- 172.21.15.103:0/888009151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8db81078b0 0x7f8db8107cb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.210+0000 7f8dbfe17640 1 -- 172.21.15.103:0/888009151 >> 172.21.15.103:0/888009151 conn(0x7f8db8103080 msgr2=0x7f8db81054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.210+0000 7f8dbfe17640 1 -- 172.21.15.103:0/888009151 shutdown_connections 2024-09-19T06:40:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.210+0000 7f8dbfe17640 1 -- 172.21.15.103:0/888009151 wait complete. 2024-09-19T06:40:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.211+0000 7f8dbfe17640 1 Processor -- start 2024-09-19T06:40:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.211+0000 7f8dbfe17640 1 -- start start 2024-09-19T06:40:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.211+0000 7f8dbfe17640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8db81078b0 0x7f8db819b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.211+0000 7f8dbfe17640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8db810be10 con 0x7f8db81078b0 2024-09-19T06:40:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.211+0000 7f8dbdbb3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8db81078b0 0x7f8db819b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.211+0000 7f8dbdbb3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8db81078b0 0x7f8db819b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51824/0 (socket says 172.21.15.103:51824) 2024-09-19T06:40:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.211+0000 7f8dbdbb3640 1 -- 172.21.15.103:0/197926378 learned_addr learned my addr 172.21.15.103:0/197926378 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.211+0000 7f8dbdbb3640 1 -- 172.21.15.103:0/197926378 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db819bdc0 con 0x7f8db81078b0 2024-09-19T06:40:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.211+0000 7f8dbdbb3640 1 --2- 172.21.15.103:0/197926378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8db81078b0 0x7f8db819b880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8db40097f0 tx=0x7f8db40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.212+0000 7f8daeffd640 1 -- 172.21.15.103:0/197926378 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8db4037040 con 0x7f8db81078b0 2024-09-19T06:40:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.212+0000 7f8dbfe17640 1 -- 172.21.15.103:0/197926378 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db819c050 con 0x7f8db81078b0 2024-09-19T06:40:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.212+0000 7f8daeffd640 1 -- 172.21.15.103:0/197926378 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8db4030cc0 con 0x7f8db81078b0 2024-09-19T06:40:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.212+0000 7f8daeffd640 1 -- 172.21.15.103:0/197926378 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8db403a640 con 0x7f8db81078b0 2024-09-19T06:40:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.212+0000 7f8dbfe17640 1 -- 172.21.15.103:0/197926378 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db819ed40 con 0x7f8db81078b0 2024-09-19T06:40:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.213+0000 7f8daeffd640 1 -- 172.21.15.103:0/197926378 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f8db403b3f0 con 0x7f8db81078b0 2024-09-19T06:40:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.213+0000 7f8dbfe17640 1 -- 172.21.15.103:0/197926378 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8db810bf90 con 0x7f8db81078b0 2024-09-19T06:40:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.213+0000 7f8daeffd640 1 --2- 172.21.15.103:0/197926378 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f8da40391b0 0x7f8da403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.213+0000 7f8daeffd640 1 -- 172.21.15.103:0/197926378 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8db406c7d0 con 0x7f8db81078b0 2024-09-19T06:40:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.214+0000 7f8dbd3b2640 1 --2- 172.21.15.103:0/197926378 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f8da40391b0 0x7f8da403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.214+0000 7f8dbd3b2640 1 --2- 172.21.15.103:0/197926378 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f8da40391b0 0x7f8da403b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8da80098c0 tx=0x7f8da8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.218+0000 7f8daeffd640 1 -- 172.21.15.103:0/197926378 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8db4042e10 con 0x7f8db81078b0 2024-09-19T06:40:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.441+0000 7f8dbfe17640 1 -- 172.21.15.103:0/197926378 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f8db8107d30 con 0x7f8db81078b0 2024-09-19T06:40:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.820+0000 7f8daeffd640 1 -- 172.21.15.103:0/197926378 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f8db403ba40 con 0x7f8db81078b0 2024-09-19T06:40:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.822+0000 7f8daeffd640 1 -- 172.21.15.103:0/197926378 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f8db403a940 con 0x7f8db81078b0 2024-09-19T06:40:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.823+0000 7f8dbfe17640 1 -- 172.21.15.103:0/197926378 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f8da40391b0 msgr2=0x7f8da403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.823+0000 7f8dbfe17640 1 --2- 172.21.15.103:0/197926378 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f8da40391b0 0x7f8da403b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8da80098c0 tx=0x7f8da8006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.823+0000 7f8dbfe17640 1 -- 172.21.15.103:0/197926378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8db81078b0 msgr2=0x7f8db819b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.823+0000 7f8dbfe17640 1 --2- 172.21.15.103:0/197926378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8db81078b0 0x7f8db819b880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8db40097f0 tx=0x7f8db40327b0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.823+0000 7f8dbfe17640 1 -- 172.21.15.103:0/197926378 shutdown_connections 2024-09-19T06:40:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.823+0000 7f8dbfe17640 1 --2- 172.21.15.103:0/197926378 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f8da40391b0 0x7f8da403b670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.823+0000 7f8dbfe17640 1 --2- 172.21.15.103:0/197926378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8db81078b0 0x7f8db819b880 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.823+0000 7f8dbfe17640 1 -- 172.21.15.103:0/197926378 >> 172.21.15.103:0/197926378 conn(0x7f8db8103080 msgr2=0x7f8db8103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.823+0000 7f8dbfe17640 1 -- 172.21.15.103:0/197926378 shutdown_connections 2024-09-19T06:40:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:34.823+0000 7f8dbfe17640 1 -- 172.21.15.103:0/197926378 wait complete. 2024-09-19T06:40:36.120 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:35 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/197926378' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T06:40:36.121 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:35 smithi103 ceph-mon[31158]: mgrmap e9: x(active, since 18s) 2024-09-19T06:40:36.121 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:35 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:35.862+0000 7f7212a3f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:40:36.121 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:35 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:35.923+0000 7f7212a3f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:40:36.121 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:36 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:36.119+0000 7f7212a3f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:40:37.278 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:37 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/4215000728' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T06:40:37.278 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:37 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:37.028+0000 7f7212a3f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:40:37.278 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:37 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:37.099+0000 7f7212a3f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:40:37.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:40:37.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T06:40:37.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T06:40:37.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T06:40:37.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T06:40:37.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:40:37.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.758+0000 7f8fb7ef3640 1 Processor -- start 2024-09-19T06:40:37.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.758+0000 7f8fb7ef3640 1 -- start start 2024-09-19T06:40:37.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.758+0000 7f8fb7ef3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fb0106780 0x7f8fb0106b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:37.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.758+0000 7f8fb7ef3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0107150 con 0x7f8fb0106780 2024-09-19T06:40:37.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.759+0000 7f8fb5c8f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fb0106780 0x7f8fb0106b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:37.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.759+0000 7f8fb5c8f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fb0106780 0x7f8fb0106b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51848/0 (socket says 172.21.15.103:51848) 2024-09-19T06:40:37.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.759+0000 7f8fb5c8f640 1 -- 172.21.15.103:0/1136785318 learned_addr learned my addr 172.21.15.103:0/1136785318 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:37.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.759+0000 7f8fb5c8f640 1 -- 172.21.15.103:0/1136785318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb0107980 con 0x7f8fb0106780 2024-09-19T06:40:37.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.760+0000 7f8fb5c8f640 1 --2- 172.21.15.103:0/1136785318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fb0106780 0x7f8fb0106b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009820 tx=0x7f8fa002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5999b0d78e599f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:37.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.760+0000 7f8fb4c8d640 1 -- 172.21.15.103:0/1136785318 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fa0037070 con 0x7f8fb0106780 2024-09-19T06:40:37.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.760+0000 7f8fb4c8d640 1 -- 172.21.15.103:0/1136785318 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa0032920 con 0x7f8fb0106780 2024-09-19T06:40:37.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.760+0000 7f8fb4c8d640 1 -- 172.21.15.103:0/1136785318 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fa0030810 con 0x7f8fb0106780 2024-09-19T06:40:37.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.760+0000 7f8fb7ef3640 1 -- 172.21.15.103:0/1136785318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fb0106780 msgr2=0x7f8fb0106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:37.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.760+0000 7f8fb7ef3640 1 --2- 172.21.15.103:0/1136785318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fb0106780 0x7f8fb0106b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009820 tx=0x7f8fa002a270 comp rx=0 tx=0).stop 2024-09-19T06:40:37.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.761+0000 7f8fb7ef3640 1 -- 172.21.15.103:0/1136785318 shutdown_connections 2024-09-19T06:40:37.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.761+0000 7f8fb7ef3640 1 --2- 172.21.15.103:0/1136785318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fb0106780 0x7f8fb0106b80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:37.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.761+0000 7f8fb7ef3640 1 -- 172.21.15.103:0/1136785318 >> 172.21.15.103:0/1136785318 conn(0x7f8fb0101f10 msgr2=0x7f8fb0104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:37.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.761+0000 7f8fb7ef3640 1 -- 172.21.15.103:0/1136785318 shutdown_connections 2024-09-19T06:40:37.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.761+0000 7f8fb7ef3640 1 -- 172.21.15.103:0/1136785318 wait complete. 2024-09-19T06:40:37.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.761+0000 7f8fb7ef3640 1 Processor -- start 2024-09-19T06:40:37.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.761+0000 7f8fb7ef3640 1 -- start start 2024-09-19T06:40:37.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.761+0000 7f8fb7ef3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fb0106780 0x7f8fb019b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:37.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.761+0000 7f8fb7ef3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb010ad30 con 0x7f8fb0106780 2024-09-19T06:40:37.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.762+0000 7f8fb5c8f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fb0106780 0x7f8fb019b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:37.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.762+0000 7f8fb5c8f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fb0106780 0x7f8fb019b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51858/0 (socket says 172.21.15.103:51858) 2024-09-19T06:40:37.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.762+0000 7f8fb5c8f640 1 -- 172.21.15.103:0/4215000728 learned_addr learned my addr 172.21.15.103:0/4215000728 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:37.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.762+0000 7f8fb5c8f640 1 -- 172.21.15.103:0/4215000728 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb019be50 con 0x7f8fb0106780 2024-09-19T06:40:37.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.762+0000 7f8fb5c8f640 1 --2- 172.21.15.103:0/4215000728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fb0106780 0x7f8fb019b910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00097f0 tx=0x7f8fa00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:37.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.762+0000 7f8fa6ffd640 1 -- 172.21.15.103:0/4215000728 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fa0037040 con 0x7f8fb0106780 2024-09-19T06:40:37.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.762+0000 7f8fb7ef3640 1 -- 172.21.15.103:0/4215000728 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb019c0e0 con 0x7f8fb0106780 2024-09-19T06:40:37.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.762+0000 7f8fa6ffd640 1 -- 172.21.15.103:0/4215000728 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa0030c40 con 0x7f8fb0106780 2024-09-19T06:40:37.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.762+0000 7f8fa6ffd640 1 -- 172.21.15.103:0/4215000728 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fa003a5b0 con 0x7f8fb0106780 2024-09-19T06:40:37.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.762+0000 7f8fb7ef3640 1 -- 172.21.15.103:0/4215000728 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb019edd0 con 0x7f8fb0106780 2024-09-19T06:40:37.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.763+0000 7f8fa6ffd640 1 -- 172.21.15.103:0/4215000728 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f8fa003b3f0 con 0x7f8fb0106780 2024-09-19T06:40:37.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.763+0000 7f8fb7ef3640 1 -- 172.21.15.103:0/4215000728 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb010aeb0 con 0x7f8fb0106780 2024-09-19T06:40:37.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.763+0000 7f8fa6ffd640 1 --2- 172.21.15.103:0/4215000728 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f8f9c039200 0x7f8f9c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:37.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.763+0000 7f8fa6ffd640 1 -- 172.21.15.103:0/4215000728 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8fa006c700 con 0x7f8fb0106780 2024-09-19T06:40:37.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.763+0000 7f8fb548e640 1 -- 172.21.15.103:0/4215000728 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f8f9c039200 msgr2=0x7f8f9c03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/4122140716 2024-09-19T06:40:37.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.763+0000 7f8fb548e640 1 --2- 172.21.15.103:0/4215000728 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f8f9c039200 0x7f8f9c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:40:37.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.768+0000 7f8fa6ffd640 1 -- 172.21.15.103:0/4215000728 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8fa00313a0 con 0x7f8fb0106780 2024-09-19T06:40:37.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.964+0000 7f8fb548e640 1 -- 172.21.15.103:0/4215000728 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f8f9c039200 msgr2=0x7f8f9c03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/4122140716 2024-09-19T06:40:37.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.964+0000 7f8fb548e640 1 --2- 172.21.15.103:0/4215000728 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f8f9c039200 0x7f8f9c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:40:37.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.982+0000 7f8fb7ef3640 1 -- 172.21.15.103:0/4215000728 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f8fb002d900 con 0x7f8fb0106780 2024-09-19T06:40:37.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.983+0000 7f8fa6ffd640 1 -- 172.21.15.103:0/4215000728 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f8fa0041d10 con 0x7f8fb0106780 2024-09-19T06:40:37.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.985+0000 7f8fb7ef3640 1 -- 172.21.15.103:0/4215000728 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f8f9c039200 msgr2=0x7f8f9c03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:40:37.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.985+0000 7f8fb7ef3640 1 --2- 172.21.15.103:0/4215000728 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f8f9c039200 0x7f8f9c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:37.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.985+0000 7f8fb7ef3640 1 -- 172.21.15.103:0/4215000728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fb0106780 msgr2=0x7f8fb019b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:37.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.985+0000 7f8fb7ef3640 1 --2- 172.21.15.103:0/4215000728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fb0106780 0x7f8fb019b910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00097f0 tx=0x7f8fa00327b0 comp rx=0 tx=0).stop 2024-09-19T06:40:37.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.985+0000 7f8fb7ef3640 1 -- 172.21.15.103:0/4215000728 shutdown_connections 2024-09-19T06:40:37.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.985+0000 7f8fb7ef3640 1 --2- 172.21.15.103:0/4215000728 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f8f9c039200 0x7f8f9c03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:37.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.985+0000 7f8fb7ef3640 1 --2- 172.21.15.103:0/4215000728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fb0106780 0x7f8fb019b910 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:37.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.985+0000 7f8fb7ef3640 1 -- 172.21.15.103:0/4215000728 >> 172.21.15.103:0/4215000728 conn(0x7f8fb0101f10 msgr2=0x7f8fb0102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:37.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.985+0000 7f8fb7ef3640 1 -- 172.21.15.103:0/4215000728 shutdown_connections 2024-09-19T06:40:37.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:36.985+0000 7f8fb7ef3640 1 -- 172.21.15.103:0/4215000728 wait complete. 2024-09-19T06:40:37.634 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for the mgr to restart... 2024-09-19T06:40:37.634 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr epoch 9... 2024-09-19T06:40:37.872 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:37 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:37.849+0000 7f7212a3f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:40:38.124 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:37 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:37.972+0000 7f7212a3f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:40:38.388 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:38 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:38.121+0000 7f7212a3f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:40:38.388 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:38 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:38.183+0000 7f7212a3f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:40:38.388 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:38 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:38.337+0000 7f7212a3f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:40:38.689 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:38 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:38.405+0000 7f7212a3f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:40:38.689 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:38 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:38.465+0000 7f7212a3f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:40:38.689 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:38 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:38.621+0000 7f7212a3f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:40:38.689 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:38 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:38.686+0000 7f7212a3f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:40:38.955 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:38 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:38.753+0000 7f7212a3f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:40:38.955 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:38 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:38.835+0000 7f7212a3f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:40:38.955 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:38 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:38.894+0000 7f7212a3f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:40:38.955 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:38 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:38.953+0000 7f7212a3f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:40:40.138 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:39 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:39.736+0000 7f7212a3f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:40:40.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:39 smithi103 ceph-mon[31158]: Active manager daemon x restarted 2024-09-19T06:40:40.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:39 smithi103 ceph-mon[31158]: Activating manager daemon x 2024-09-19T06:40:40.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:39 smithi103 ceph-mon[31158]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T06:40:40.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:39 smithi103 ceph-mon[31158]: mgrmap e10: x(active, starting, since 0.00340366s) 2024-09-19T06:40:40.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:40:40.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T06:40:40.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:40:40.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:40:40.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:40:40.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:39 smithi103 ceph-mon[31158]: Manager daemon x is now available 2024-09-19T06:40:40.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:40:40.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:40:40.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T06:40:41.092 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:40.836+0000 7f7178508640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:06:40:40] ENGINE Error in HTTPServer.serve 2024-09-19T06:40:41.092 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: Traceback (most recent call last): 2024-09-19T06:40:41.092 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:40:41.092 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: self._connections.run(self.expiration_interval) 2024-09-19T06:40:41.092 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:40:41.092 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: self._run(expiration_interval) 2024-09-19T06:40:41.092 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:40:41.092 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:40:41.092 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:40:41.092 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:40:41.092 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:40:41.093 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: s = self.context.wrap_socket( 2024-09-19T06:40:41.093 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:40:41.093 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: return self.sslsocket_class._create( 2024-09-19T06:40:41.093 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:40:41.093 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: self.do_handshake() 2024-09-19T06:40:41.093 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:40:41.093 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: self._sslobj.do_handshake() 2024-09-19T06:40:41.093 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:40:41.094 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:40:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:40:41.673 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:40:41.673 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T06:40:41.674 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T06:40:41.674 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:40:41.674 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.036+0000 7f08dd7fd640 1 Processor -- start 2024-09-19T06:40:41.674 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.036+0000 7f08dd7fd640 1 -- start start 2024-09-19T06:40:41.674 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.036+0000 7f08dd7fd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08d8108310 0x7f08d810a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:41.674 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.036+0000 7f08dd7fd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f08d807be20 con 0x7f08d8108310 2024-09-19T06:40:41.674 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.036+0000 7f08d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08d8108310 0x7f08d810a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:41.674 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.036+0000 7f08d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08d8108310 0x7f08d810a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57308/0 (socket says 172.21.15.103:57308) 2024-09-19T06:40:41.674 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.036+0000 7f08d6ffd640 1 -- 172.21.15.103:0/3195803666 learned_addr learned my addr 172.21.15.103:0/3195803666 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:41.674 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.036+0000 7f08d6ffd640 1 -- 172.21.15.103:0/3195803666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d810ac40 con 0x7f08d8108310 2024-09-19T06:40:41.674 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.037+0000 7f08d6ffd640 1 --2- 172.21.15.103:0/3195803666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08d8108310 0x7f08d810a700 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009820 tx=0x7f08c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665be8de9ec0743 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:41.674 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.037+0000 7f08d67fc640 1 -- 172.21.15.103:0/3195803666 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08c8037070 con 0x7f08d8108310 2024-09-19T06:40:41.675 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.037+0000 7f08d67fc640 1 -- 172.21.15.103:0/3195803666 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08c8032920 con 0x7f08d8108310 2024-09-19T06:40:41.675 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.037+0000 7f08d67fc640 1 -- 172.21.15.103:0/3195803666 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08c8030810 con 0x7f08d8108310 2024-09-19T06:40:41.675 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.038+0000 7f08dd7fd640 1 -- 172.21.15.103:0/3195803666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08d8108310 msgr2=0x7f08d810a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:41.675 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.038+0000 7f08dd7fd640 1 --2- 172.21.15.103:0/3195803666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08d8108310 0x7f08d810a700 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009820 tx=0x7f08c802a270 comp rx=0 tx=0).stop 2024-09-19T06:40:41.675 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.038+0000 7f08dd7fd640 1 -- 172.21.15.103:0/3195803666 shutdown_connections 2024-09-19T06:40:41.675 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.038+0000 7f08dd7fd640 1 --2- 172.21.15.103:0/3195803666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08d8108310 0x7f08d810a700 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:41.675 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.038+0000 7f08dd7fd640 1 -- 172.21.15.103:0/3195803666 >> 172.21.15.103:0/3195803666 conn(0x7f08d8101f90 msgr2=0x7f08d81043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:41.675 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.038+0000 7f08dd7fd640 1 -- 172.21.15.103:0/3195803666 shutdown_connections 2024-09-19T06:40:41.675 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.038+0000 7f08dd7fd640 1 -- 172.21.15.103:0/3195803666 wait complete. 2024-09-19T06:40:41.675 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.039+0000 7f08dd7fd640 1 Processor -- start 2024-09-19T06:40:41.675 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.039+0000 7f08dd7fd640 1 -- start start 2024-09-19T06:40:41.675 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.039+0000 7f08dd7fd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08d8108310 0x7f08d819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:41.675 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.039+0000 7f08dd7fd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f08d810bc60 con 0x7f08d8108310 2024-09-19T06:40:41.675 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.039+0000 7f08d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08d8108310 0x7f08d819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:41.676 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.039+0000 7f08d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08d8108310 0x7f08d819b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57322/0 (socket says 172.21.15.103:57322) 2024-09-19T06:40:41.676 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.039+0000 7f08d6ffd640 1 -- 172.21.15.103:0/303920153 learned_addr learned my addr 172.21.15.103:0/303920153 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:41.676 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.039+0000 7f08d6ffd640 1 -- 172.21.15.103:0/303920153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d819bdb0 con 0x7f08d8108310 2024-09-19T06:40:41.676 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.039+0000 7f08d6ffd640 1 --2- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08d8108310 0x7f08d819b870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f08c80097f0 tx=0x7f08c80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:41.676 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.040+0000 7f08d4ff9640 1 -- 172.21.15.103:0/303920153 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08c8037040 con 0x7f08d8108310 2024-09-19T06:40:41.676 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.040+0000 7f08d4ff9640 1 -- 172.21.15.103:0/303920153 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08c8030c40 con 0x7f08d8108310 2024-09-19T06:40:41.676 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.040+0000 7f08dd7fd640 1 -- 172.21.15.103:0/303920153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d819c040 con 0x7f08d8108310 2024-09-19T06:40:41.676 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.040+0000 7f08d4ff9640 1 -- 172.21.15.103:0/303920153 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08c803a5b0 con 0x7f08d8108310 2024-09-19T06:40:41.676 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.040+0000 7f08dd7fd640 1 -- 172.21.15.103:0/303920153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d819ed30 con 0x7f08d8108310 2024-09-19T06:40:41.676 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.041+0000 7f08d4ff9640 1 -- 172.21.15.103:0/303920153 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f08c803a7a0 con 0x7f08d8108310 2024-09-19T06:40:41.676 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.041+0000 7f08d4ff9640 1 --2- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f08c4039260 0x7f08c403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:41.676 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.041+0000 7f08dd7fd640 1 -- 172.21.15.103:0/303920153 --> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f08d8108770 con 0x7f08c4039260 2024-09-19T06:40:41.677 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.041+0000 7f08d4ff9640 1 -- 172.21.15.103:0/303920153 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f08c806c750 con 0x7f08d8108310 2024-09-19T06:40:41.677 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.041+0000 7f08cffff640 1 -- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f08c4039260 msgr2=0x7f08c403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/4122140716 2024-09-19T06:40:41.677 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.041+0000 7f08cffff640 1 --2- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f08c4039260 0x7f08c403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:40:41.677 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.242+0000 7f08cffff640 1 -- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f08c4039260 msgr2=0x7f08c403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/4122140716 2024-09-19T06:40:41.677 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.242+0000 7f08cffff640 1 --2- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f08c4039260 0x7f08c403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:40:41.677 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.643+0000 7f08cffff640 1 -- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f08c4039260 msgr2=0x7f08c403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/4122140716 2024-09-19T06:40:41.677 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:38.643+0000 7f08cffff640 1 --2- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f08c4039260 0x7f08c403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T06:40:41.677 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:39.444+0000 7f08cffff640 1 -- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f08c4039260 msgr2=0x7f08c403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/4122140716 2024-09-19T06:40:41.677 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:39.444+0000 7f08cffff640 1 --2- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f08c4039260 0x7f08c403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T06:40:41.677 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:39.740+0000 7f08d4ff9640 1 -- 172.21.15.103:0/303920153 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f08c806b200 con 0x7f08d8108310 2024-09-19T06:40:41.677 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:39.740+0000 7f08d4ff9640 1 -- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f08c4039260 msgr2=0x7f08c403b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:40:41.677 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:39.740+0000 7f08d4ff9640 1 --2- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f08c4039260 0x7f08c403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:41.677 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.741+0000 7f08d4ff9640 1 -- 172.21.15.103:0/303920153 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f08c806da40 con 0x7f08d8108310 2024-09-19T06:40:41.678 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.741+0000 7f08d4ff9640 1 --2- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f08c403cb30 0x7f08c403ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:41.678 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.742+0000 7f08d4ff9640 1 -- 172.21.15.103:0/303920153 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f08c8065870 con 0x7f08c403cb30 2024-09-19T06:40:41.678 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.742+0000 7f08cffff640 1 --2- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f08c403cb30 0x7f08c403ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:41.678 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.742+0000 7f08cffff640 1 --2- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f08c403cb30 0x7f08c403ef20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f08c0003980 tx=0x7f08c00092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:41.678 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.743+0000 7f08d4ff9640 1 -- 172.21.15.103:0/303920153 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f08c8065870 con 0x7f08c403cb30 2024-09-19T06:40:41.678 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.750+0000 7f08dd7fd640 1 -- 172.21.15.103:0/303920153 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f08d81032d0 con 0x7f08c403cb30 2024-09-19T06:40:41.678 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.751+0000 7f08d4ff9640 1 -- 172.21.15.103:0/303920153 <== mgr.14150 v2:172.21.15.103:6800/2590139910 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f08d81032d0 con 0x7f08c403cb30 2024-09-19T06:40:41.678 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.751+0000 7f08dd7fd640 1 -- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f08c403cb30 msgr2=0x7f08c403ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:41.678 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.751+0000 7f08dd7fd640 1 --2- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f08c403cb30 0x7f08c403ef20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f08c0003980 tx=0x7f08c00092b0 comp rx=0 tx=0).stop 2024-09-19T06:40:41.678 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.751+0000 7f08dd7fd640 1 -- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08d8108310 msgr2=0x7f08d819b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:41.678 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.751+0000 7f08dd7fd640 1 --2- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08d8108310 0x7f08d819b870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f08c80097f0 tx=0x7f08c80327b0 comp rx=0 tx=0).stop 2024-09-19T06:40:41.678 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.751+0000 7f08dd7fd640 1 -- 172.21.15.103:0/303920153 shutdown_connections 2024-09-19T06:40:41.679 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.751+0000 7f08dd7fd640 1 --2- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f08c403cb30 0x7f08c403ef20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:41.679 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.751+0000 7f08dd7fd640 1 --2- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:6800/4122140716,v1:172.21.15.103:6801/4122140716] conn(0x7f08c4039260 0x7f08c403b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:41.679 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.751+0000 7f08dd7fd640 1 --2- 172.21.15.103:0/303920153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08d8108310 0x7f08d819b870 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:41.679 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.751+0000 7f08dd7fd640 1 -- 172.21.15.103:0/303920153 >> 172.21.15.103:0/303920153 conn(0x7f08d8101f90 msgr2=0x7f08d8102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:41.679 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.751+0000 7f08dd7fd640 1 -- 172.21.15.103:0/303920153 shutdown_connections 2024-09-19T06:40:41.679 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:40.751+0000 7f08dd7fd640 1 -- 172.21.15.103:0/303920153 wait complete. 2024-09-19T06:40:41.679 INFO:teuthology.orchestra.run.smithi103.stdout:mgr epoch 9 is available 2024-09-19T06:40:41.679 INFO:teuthology.orchestra.run.smithi103.stdout:Generating a dashboard self-signed certificate... 2024-09-19T06:40:41.928 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:41 smithi103 ceph-mon[31158]: [19/Sep/2024:06:40:40] ENGINE Bus STARTING 2024-09-19T06:40:41.929 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:41 smithi103 ceph-mon[31158]: [19/Sep/2024:06:40:40] ENGINE Serving on http://172.21.15.103:8765 2024-09-19T06:40:41.929 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:41 smithi103 ceph-mon[31158]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T06:40:41.929 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:41 smithi103 ceph-mon[31158]: mgrmap e11: x(active, since 1.00551s) 2024-09-19T06:40:41.929 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:41 smithi103 ceph-mon[31158]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T06:40:41.929 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:41 smithi103 ceph-mon[31158]: [19/Sep/2024:06:40:40] ENGINE Serving on https://172.21.15.103:7150 2024-09-19T06:40:41.929 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:41 smithi103 ceph-mon[31158]: [19/Sep/2024:06:40:40] ENGINE Bus STARTED 2024-09-19T06:40:41.929 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:41 smithi103 ceph-mon[31158]: [19/Sep/2024:06:40:40] ENGINE Error in HTTPServer.serve 2024-09-19T06:40:41.929 INFO:journalctl@ceph.mon.a.smithi103.stdout: Traceback (most recent call last): 2024-09-19T06:40:41.929 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:40:41.929 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:40:41.929 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:40:41.929 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._run(expiration_interval) 2024-09-19T06:40:41.930 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:40:41.930 INFO:journalctl@ceph.mon.a.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:40:41.930 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:40:41.930 INFO:journalctl@ceph.mon.a.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:40:41.930 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:40:41.930 INFO:journalctl@ceph.mon.a.smithi103.stdout: s = self.context.wrap_socket( 2024-09-19T06:40:41.930 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:40:41.930 INFO:journalctl@ceph.mon.a.smithi103.stdout: return self.sslsocket_class._create( 2024-09-19T06:40:41.930 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:40:41.930 INFO:journalctl@ceph.mon.a.smithi103.stdout: self.do_handshake() 2024-09-19T06:40:41.930 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:40:41.930 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._sslobj.do_handshake() 2024-09-19T06:40:41.930 INFO:journalctl@ceph.mon.a.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:40:41.931 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:41 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:42.896 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T06:40:42.896 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.145+0000 7fbd3f11f640 1 Processor -- start 2024-09-19T06:40:42.896 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.145+0000 7fbd3f11f640 1 -- start start 2024-09-19T06:40:42.897 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.145+0000 7fbd3f11f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd381078b0 0x7fbd38107cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:42.897 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.145+0000 7fbd3f11f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd38108280 con 0x7fbd381078b0 2024-09-19T06:40:42.897 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.145+0000 7fbd3cebb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd381078b0 0x7fbd38107cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:42.897 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.145+0000 7fbd3cebb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd381078b0 0x7fbd38107cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57396/0 (socket says 172.21.15.103:57396) 2024-09-19T06:40:42.897 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.145+0000 7fbd3cebb640 1 -- 172.21.15.103:0/2097505209 learned_addr learned my addr 172.21.15.103:0/2097505209 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:42.897 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.145+0000 7fbd3cebb640 1 -- 172.21.15.103:0/2097505209 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd38108a60 con 0x7fbd381078b0 2024-09-19T06:40:42.897 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.146+0000 7fbd3cebb640 1 --2- 172.21.15.103:0/2097505209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd381078b0 0x7fbd38107cb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009820 tx=0x7fbd2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913a59d256d70edc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:42.897 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.146+0000 7fbd377fe640 1 -- 172.21.15.103:0/2097505209 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2c037070 con 0x7fbd381078b0 2024-09-19T06:40:42.897 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.146+0000 7fbd377fe640 1 -- 172.21.15.103:0/2097505209 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd2c032920 con 0x7fbd381078b0 2024-09-19T06:40:42.897 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.146+0000 7fbd377fe640 1 -- 172.21.15.103:0/2097505209 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2c030810 con 0x7fbd381078b0 2024-09-19T06:40:42.898 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.147+0000 7fbd3f11f640 1 -- 172.21.15.103:0/2097505209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd381078b0 msgr2=0x7fbd38107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:42.898 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.147+0000 7fbd3f11f640 1 --2- 172.21.15.103:0/2097505209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd381078b0 0x7fbd38107cb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009820 tx=0x7fbd2c02a270 comp rx=0 tx=0).stop 2024-09-19T06:40:42.898 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.147+0000 7fbd3f11f640 1 -- 172.21.15.103:0/2097505209 shutdown_connections 2024-09-19T06:40:42.898 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.147+0000 7fbd3f11f640 1 --2- 172.21.15.103:0/2097505209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd381078b0 0x7fbd38107cb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:42.898 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.147+0000 7fbd3f11f640 1 -- 172.21.15.103:0/2097505209 >> 172.21.15.103:0/2097505209 conn(0x7fbd38103060 msgr2=0x7fbd38105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:42.898 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.147+0000 7fbd3f11f640 1 -- 172.21.15.103:0/2097505209 shutdown_connections 2024-09-19T06:40:42.898 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.147+0000 7fbd3f11f640 1 -- 172.21.15.103:0/2097505209 wait complete. 2024-09-19T06:40:42.898 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.148+0000 7fbd3f11f640 1 Processor -- start 2024-09-19T06:40:42.898 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.148+0000 7fbd3f11f640 1 -- start start 2024-09-19T06:40:42.898 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.148+0000 7fbd3f11f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd381078b0 0x7fbd3807db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:42.899 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.148+0000 7fbd3f11f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3810be10 con 0x7fbd381078b0 2024-09-19T06:40:42.899 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.148+0000 7fbd3cebb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd381078b0 0x7fbd3807db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:42.899 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.148+0000 7fbd3cebb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd381078b0 0x7fbd3807db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57412/0 (socket says 172.21.15.103:57412) 2024-09-19T06:40:42.899 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.148+0000 7fbd3cebb640 1 -- 172.21.15.103:0/3690608663 learned_addr learned my addr 172.21.15.103:0/3690608663 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:42.899 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.148+0000 7fbd3cebb640 1 -- 172.21.15.103:0/3690608663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3807e040 con 0x7fbd381078b0 2024-09-19T06:40:42.899 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.148+0000 7fbd3cebb640 1 --2- 172.21.15.103:0/3690608663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd381078b0 0x7fbd3807db00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0097f0 tx=0x7fbd2c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:42.899 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.148+0000 7fbd35ffb640 1 -- 172.21.15.103:0/3690608663 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2c037040 con 0x7fbd381078b0 2024-09-19T06:40:42.899 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.148+0000 7fbd35ffb640 1 -- 172.21.15.103:0/3690608663 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd2c030c40 con 0x7fbd381078b0 2024-09-19T06:40:42.899 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.148+0000 7fbd3f11f640 1 -- 172.21.15.103:0/3690608663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3807c230 con 0x7fbd381078b0 2024-09-19T06:40:42.900 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.148+0000 7fbd35ffb640 1 -- 172.21.15.103:0/3690608663 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2c03a5b0 con 0x7fbd381078b0 2024-09-19T06:40:42.900 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.149+0000 7fbd3f11f640 1 -- 172.21.15.103:0/3690608663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3807c770 con 0x7fbd381078b0 2024-09-19T06:40:42.900 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.149+0000 7fbd35ffb640 1 -- 172.21.15.103:0/3690608663 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fbd2c03a7f0 con 0x7fbd381078b0 2024-09-19T06:40:42.900 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.150+0000 7fbd3f11f640 1 -- 172.21.15.103:0/3690608663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd38107d30 con 0x7fbd381078b0 2024-09-19T06:40:42.900 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.150+0000 7fbd35ffb640 1 --2- 172.21.15.103:0/3690608663 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbd28039200 0x7fbd2803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:42.900 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.150+0000 7fbd35ffb640 1 -- 172.21.15.103:0/3690608663 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbd2c06c7e0 con 0x7fbd381078b0 2024-09-19T06:40:42.900 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.150+0000 7fbd37fff640 1 --2- 172.21.15.103:0/3690608663 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbd28039200 0x7fbd2803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:42.900 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.150+0000 7fbd37fff640 1 --2- 172.21.15.103:0/3690608663 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbd28039200 0x7fbd2803b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbd240098c0 tx=0x7fbd24006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:42.900 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.154+0000 7fbd35ffb640 1 -- 172.21.15.103:0/3690608663 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbd2c030df0 con 0x7fbd381078b0 2024-09-19T06:40:42.900 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.336+0000 7fbd3f11f640 1 -- 172.21.15.103:0/3690608663 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fbd3807d0f0 con 0x7fbd28039200 2024-09-19T06:40:42.900 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.388+0000 7fbd35ffb640 1 -- 172.21.15.103:0/3690608663 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fbd3807d0f0 con 0x7fbd28039200 2024-09-19T06:40:42.901 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.390+0000 7fbd3f11f640 1 -- 172.21.15.103:0/3690608663 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbd28039200 msgr2=0x7fbd2803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:42.901 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.390+0000 7fbd3f11f640 1 --2- 172.21.15.103:0/3690608663 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbd28039200 0x7fbd2803b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbd240098c0 tx=0x7fbd24006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:42.901 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.390+0000 7fbd3f11f640 1 -- 172.21.15.103:0/3690608663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd381078b0 msgr2=0x7fbd3807db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:42.901 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.390+0000 7fbd3f11f640 1 --2- 172.21.15.103:0/3690608663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd381078b0 0x7fbd3807db00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0097f0 tx=0x7fbd2c0327b0 comp rx=0 tx=0).stop 2024-09-19T06:40:42.901 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.390+0000 7fbd3f11f640 1 -- 172.21.15.103:0/3690608663 shutdown_connections 2024-09-19T06:40:42.901 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.390+0000 7fbd3f11f640 1 --2- 172.21.15.103:0/3690608663 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbd28039200 0x7fbd2803b6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:42.901 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.390+0000 7fbd3f11f640 1 --2- 172.21.15.103:0/3690608663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd381078b0 0x7fbd3807db00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:42.901 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.390+0000 7fbd3f11f640 1 -- 172.21.15.103:0/3690608663 >> 172.21.15.103:0/3690608663 conn(0x7fbd38103060 msgr2=0x7fbd38103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:42.901 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.390+0000 7fbd3f11f640 1 -- 172.21.15.103:0/3690608663 shutdown_connections 2024-09-19T06:40:42.901 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:42.390+0000 7fbd3f11f640 1 -- 172.21.15.103:0/3690608663 wait complete. 2024-09-19T06:40:42.901 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial admin user... 2024-09-19T06:40:43.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:43 smithi103 ceph-mon[31158]: mgrmap e12: x(active, since 2s) 2024-09-19T06:40:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:43 smithi103 ceph-mon[31158]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:43 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:43 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:44.387 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$xDk7wYneCiuHRjg5H.Nq6OBso3DHJVhARHnOKTvjVYgTcmWuGd26K", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726728043, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T06:40:44.387 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.314+0000 7f5a5077a640 1 Processor -- start 2024-09-19T06:40:44.387 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.314+0000 7f5a5077a640 1 -- start start 2024-09-19T06:40:44.387 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.314+0000 7f5a5077a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a481078b0 0x7f5a48107cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:44.387 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.314+0000 7f5a5077a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5a48108280 con 0x7f5a481078b0 2024-09-19T06:40:44.387 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.314+0000 7f5a4e516640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a481078b0 0x7f5a48107cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:44.387 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.314+0000 7f5a4e516640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a481078b0 0x7f5a48107cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57424/0 (socket says 172.21.15.103:57424) 2024-09-19T06:40:44.387 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.314+0000 7f5a4e516640 1 -- 172.21.15.103:0/3451948634 learned_addr learned my addr 172.21.15.103:0/3451948634 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:44.388 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.314+0000 7f5a4e516640 1 -- 172.21.15.103:0/3451948634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a48108a60 con 0x7f5a481078b0 2024-09-19T06:40:44.388 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.315+0000 7f5a4e516640 1 --2- 172.21.15.103:0/3451948634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a481078b0 0x7f5a48107cb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009820 tx=0x7f5a4402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbeb041e3521ecf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:44.388 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.315+0000 7f5a4d514640 1 -- 172.21.15.103:0/3451948634 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a44037070 con 0x7f5a481078b0 2024-09-19T06:40:44.388 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.315+0000 7f5a4d514640 1 -- 172.21.15.103:0/3451948634 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a44032920 con 0x7f5a481078b0 2024-09-19T06:40:44.388 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.315+0000 7f5a4d514640 1 -- 172.21.15.103:0/3451948634 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a44030810 con 0x7f5a481078b0 2024-09-19T06:40:44.388 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.316+0000 7f5a5077a640 1 -- 172.21.15.103:0/3451948634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a481078b0 msgr2=0x7f5a48107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:44.388 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.316+0000 7f5a5077a640 1 --2- 172.21.15.103:0/3451948634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a481078b0 0x7f5a48107cb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009820 tx=0x7f5a4402a270 comp rx=0 tx=0).stop 2024-09-19T06:40:44.388 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.316+0000 7f5a5077a640 1 -- 172.21.15.103:0/3451948634 shutdown_connections 2024-09-19T06:40:44.388 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.316+0000 7f5a5077a640 1 --2- 172.21.15.103:0/3451948634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a481078b0 0x7f5a48107cb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.388 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.316+0000 7f5a5077a640 1 -- 172.21.15.103:0/3451948634 >> 172.21.15.103:0/3451948634 conn(0x7f5a48103060 msgr2=0x7f5a48105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:44.388 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.316+0000 7f5a5077a640 1 -- 172.21.15.103:0/3451948634 shutdown_connections 2024-09-19T06:40:44.388 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.316+0000 7f5a5077a640 1 -- 172.21.15.103:0/3451948634 wait complete. 2024-09-19T06:40:44.389 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.316+0000 7f5a5077a640 1 Processor -- start 2024-09-19T06:40:44.389 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.317+0000 7f5a5077a640 1 -- start start 2024-09-19T06:40:44.389 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.317+0000 7f5a5077a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a481078b0 0x7f5a4807db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:44.389 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.317+0000 7f5a5077a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4810be10 con 0x7f5a481078b0 2024-09-19T06:40:44.389 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.317+0000 7f5a4e516640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a481078b0 0x7f5a4807db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:44.389 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.317+0000 7f5a4e516640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a481078b0 0x7f5a4807db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57436/0 (socket says 172.21.15.103:57436) 2024-09-19T06:40:44.389 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.317+0000 7f5a4e516640 1 -- 172.21.15.103:0/3232043588 learned_addr learned my addr 172.21.15.103:0/3232043588 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:44.389 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.317+0000 7f5a4e516640 1 -- 172.21.15.103:0/3232043588 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4807e040 con 0x7f5a481078b0 2024-09-19T06:40:44.389 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.317+0000 7f5a4e516640 1 --2- 172.21.15.103:0/3232043588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a481078b0 0x7f5a4807db00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5a44030560 tx=0x7f5a44032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:44.389 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.317+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/3232043588 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a44037040 con 0x7f5a481078b0 2024-09-19T06:40:44.389 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.317+0000 7f5a5077a640 1 -- 172.21.15.103:0/3232043588 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4807c230 con 0x7f5a481078b0 2024-09-19T06:40:44.389 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.317+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/3232043588 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a44040070 con 0x7f5a481078b0 2024-09-19T06:40:44.389 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.317+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/3232043588 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a4403a6d0 con 0x7f5a481078b0 2024-09-19T06:40:44.390 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.318+0000 7f5a5077a640 1 -- 172.21.15.103:0/3232043588 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4807c770 con 0x7f5a481078b0 2024-09-19T06:40:44.390 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.318+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/3232043588 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f5a4403a940 con 0x7f5a481078b0 2024-09-19T06:40:44.390 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.318+0000 7f5a5077a640 1 -- 172.21.15.103:0/3232043588 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a48107d30 con 0x7f5a481078b0 2024-09-19T06:40:44.390 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.318+0000 7f5a3f7fe640 1 --2- 172.21.15.103:0/3232043588 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5a34039200 0x7f5a3403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:44.390 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.319+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/3232043588 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5a4406ca80 con 0x7f5a481078b0 2024-09-19T06:40:44.390 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.319+0000 7f5a4dd15640 1 --2- 172.21.15.103:0/3232043588 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5a34039200 0x7f5a3403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:44.390 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.319+0000 7f5a4dd15640 1 --2- 172.21.15.103:0/3232043588 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5a34039200 0x7f5a3403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5a380098c0 tx=0x7f5a38006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:44.390 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.323+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/3232043588 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5a44045980 con 0x7f5a481078b0 2024-09-19T06:40:44.390 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.514+0000 7f5a5077a640 1 -- 172.21.15.103:0/3232043588 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f5a4807c9f0 con 0x7f5a34039200 2024-09-19T06:40:44.390 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.743+0000 7f5a3f7fe640 1 -- 172.21.15.103:0/3232043588 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f5a4807c9f0 con 0x7f5a34039200 2024-09-19T06:40:44.390 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.745+0000 7f5a5077a640 1 -- 172.21.15.103:0/3232043588 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5a34039200 msgr2=0x7f5a3403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:44.390 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.745+0000 7f5a5077a640 1 --2- 172.21.15.103:0/3232043588 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5a34039200 0x7f5a3403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5a380098c0 tx=0x7f5a38006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.390 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.745+0000 7f5a5077a640 1 -- 172.21.15.103:0/3232043588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a481078b0 msgr2=0x7f5a4807db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:44.390 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.745+0000 7f5a5077a640 1 --2- 172.21.15.103:0/3232043588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a481078b0 0x7f5a4807db00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5a44030560 tx=0x7f5a44032670 comp rx=0 tx=0).stop 2024-09-19T06:40:44.391 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.745+0000 7f5a5077a640 1 -- 172.21.15.103:0/3232043588 shutdown_connections 2024-09-19T06:40:44.391 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.745+0000 7f5a5077a640 1 --2- 172.21.15.103:0/3232043588 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5a34039200 0x7f5a3403b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.391 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.745+0000 7f5a5077a640 1 --2- 172.21.15.103:0/3232043588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a481078b0 0x7f5a4807db00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.391 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.745+0000 7f5a5077a640 1 -- 172.21.15.103:0/3232043588 >> 172.21.15.103:0/3232043588 conn(0x7f5a48103060 msgr2=0x7f5a48103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:44.391 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.745+0000 7f5a5077a640 1 -- 172.21.15.103:0/3232043588 shutdown_connections 2024-09-19T06:40:44.391 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:43.745+0000 7f5a5077a640 1 -- 172.21.15.103:0/3232043588 wait complete. 2024-09-19T06:40:44.391 INFO:teuthology.orchestra.run.smithi103.stdout:Fetching dashboard port number... 2024-09-19T06:40:44.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:44.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:44 smithi103 ceph-mon[31158]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:44.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:45.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:45 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/508147412' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T06:40:45.880 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T06:40:45.880 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.775+0000 7fa356fbe640 1 Processor -- start 2024-09-19T06:40:45.880 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.775+0000 7fa356fbe640 1 -- start start 2024-09-19T06:40:45.880 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.775+0000 7fa356fbe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa350107840 0x7fa350107c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:45.880 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.775+0000 7fa356fbe640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa350108210 con 0x7fa350107840 2024-09-19T06:40:45.880 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.775+0000 7fa354d5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa350107840 0x7fa350107c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:45.881 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.775+0000 7fa354d5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa350107840 0x7fa350107c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57448/0 (socket says 172.21.15.103:57448) 2024-09-19T06:40:45.881 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.775+0000 7fa354d5a640 1 -- 172.21.15.103:0/1766567944 learned_addr learned my addr 172.21.15.103:0/1766567944 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:45.881 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.775+0000 7fa354d5a640 1 -- 172.21.15.103:0/1766567944 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa350108a40 con 0x7fa350107840 2024-09-19T06:40:45.881 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.776+0000 7fa354d5a640 1 --2- 172.21.15.103:0/1766567944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa350107840 0x7fa350107c40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa344009820 tx=0x7fa34402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93f24f8bc2e9420 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:45.881 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.776+0000 7fa34f7fe640 1 -- 172.21.15.103:0/1766567944 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa344037070 con 0x7fa350107840 2024-09-19T06:40:45.881 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.776+0000 7fa34f7fe640 1 -- 172.21.15.103:0/1766567944 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa344032920 con 0x7fa350107840 2024-09-19T06:40:45.881 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.776+0000 7fa34f7fe640 1 -- 172.21.15.103:0/1766567944 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa344030810 con 0x7fa350107840 2024-09-19T06:40:45.881 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.777+0000 7fa356fbe640 1 -- 172.21.15.103:0/1766567944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa350107840 msgr2=0x7fa350107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:45.881 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.777+0000 7fa356fbe640 1 --2- 172.21.15.103:0/1766567944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa350107840 0x7fa350107c40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa344009820 tx=0x7fa34402a270 comp rx=0 tx=0).stop 2024-09-19T06:40:45.881 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.777+0000 7fa356fbe640 1 -- 172.21.15.103:0/1766567944 shutdown_connections 2024-09-19T06:40:45.881 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.777+0000 7fa356fbe640 1 --2- 172.21.15.103:0/1766567944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa350107840 0x7fa350107c40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:45.882 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.777+0000 7fa356fbe640 1 -- 172.21.15.103:0/1766567944 >> 172.21.15.103:0/1766567944 conn(0x7fa3501030b0 msgr2=0x7fa3501054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:45.882 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.777+0000 7fa356fbe640 1 -- 172.21.15.103:0/1766567944 shutdown_connections 2024-09-19T06:40:45.882 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.777+0000 7fa356fbe640 1 -- 172.21.15.103:0/1766567944 wait complete. 2024-09-19T06:40:45.882 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.777+0000 7fa356fbe640 1 Processor -- start 2024-09-19T06:40:45.882 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.777+0000 7fa356fbe640 1 -- start start 2024-09-19T06:40:45.882 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.778+0000 7fa356fbe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa350107840 0x7fa35019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:45.882 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.778+0000 7fa356fbe640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa35010bdf0 con 0x7fa350107840 2024-09-19T06:40:45.882 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.778+0000 7fa354d5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa350107840 0x7fa35019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:45.882 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.778+0000 7fa354d5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa350107840 0x7fa35019b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57464/0 (socket says 172.21.15.103:57464) 2024-09-19T06:40:45.882 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.778+0000 7fa354d5a640 1 -- 172.21.15.103:0/508147412 learned_addr learned my addr 172.21.15.103:0/508147412 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:45.882 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.778+0000 7fa354d5a640 1 -- 172.21.15.103:0/508147412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35019be70 con 0x7fa350107840 2024-09-19T06:40:45.882 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.778+0000 7fa354d5a640 1 --2- 172.21.15.103:0/508147412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa350107840 0x7fa35019b930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa344030560 tx=0x7fa344032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:45.883 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.778+0000 7fa34dffb640 1 -- 172.21.15.103:0/508147412 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa344037040 con 0x7fa350107840 2024-09-19T06:40:45.883 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.778+0000 7fa34dffb640 1 -- 172.21.15.103:0/508147412 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa344040070 con 0x7fa350107840 2024-09-19T06:40:45.883 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.778+0000 7fa356fbe640 1 -- 172.21.15.103:0/508147412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35019c100 con 0x7fa350107840 2024-09-19T06:40:45.883 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.778+0000 7fa34dffb640 1 -- 172.21.15.103:0/508147412 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa34403a6d0 con 0x7fa350107840 2024-09-19T06:40:45.883 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.779+0000 7fa356fbe640 1 -- 172.21.15.103:0/508147412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35019edf0 con 0x7fa350107840 2024-09-19T06:40:45.883 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.779+0000 7fa34dffb640 1 -- 172.21.15.103:0/508147412 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fa34403b420 con 0x7fa350107840 2024-09-19T06:40:45.883 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.780+0000 7fa356fbe640 1 -- 172.21.15.103:0/508147412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35010bf70 con 0x7fa350107840 2024-09-19T06:40:45.883 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.780+0000 7fa34dffb640 1 --2- 172.21.15.103:0/508147412 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa340039200 0x7fa34003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:45.883 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.780+0000 7fa34dffb640 1 -- 172.21.15.103:0/508147412 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa34406c940 con 0x7fa350107840 2024-09-19T06:40:45.883 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.780+0000 7fa34ffff640 1 --2- 172.21.15.103:0/508147412 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa340039200 0x7fa34003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:45.883 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.780+0000 7fa34ffff640 1 --2- 172.21.15.103:0/508147412 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa340039200 0x7fa34003b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0098c0 tx=0x7fa33c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:45.883 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.784+0000 7fa34dffb640 1 -- 172.21.15.103:0/508147412 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa34404a200 con 0x7fa350107840 2024-09-19T06:40:45.883 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.962+0000 7fa356fbe640 1 -- 172.21.15.103:0/508147412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fa35019efc0 con 0x7fa350107840 2024-09-19T06:40:45.884 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.962+0000 7fa34dffb640 1 -- 172.21.15.103:0/508147412 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fa344031e80 con 0x7fa350107840 2024-09-19T06:40:45.884 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.964+0000 7fa356fbe640 1 -- 172.21.15.103:0/508147412 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa340039200 msgr2=0x7fa34003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:45.884 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.964+0000 7fa356fbe640 1 --2- 172.21.15.103:0/508147412 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa340039200 0x7fa34003b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0098c0 tx=0x7fa33c006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:45.884 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.964+0000 7fa356fbe640 1 -- 172.21.15.103:0/508147412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa350107840 msgr2=0x7fa35019b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:45.884 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.964+0000 7fa356fbe640 1 --2- 172.21.15.103:0/508147412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa350107840 0x7fa35019b930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa344030560 tx=0x7fa344032670 comp rx=0 tx=0).stop 2024-09-19T06:40:45.884 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.964+0000 7fa356fbe640 1 -- 172.21.15.103:0/508147412 shutdown_connections 2024-09-19T06:40:45.884 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.964+0000 7fa356fbe640 1 --2- 172.21.15.103:0/508147412 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa340039200 0x7fa34003b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:45.884 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.964+0000 7fa356fbe640 1 --2- 172.21.15.103:0/508147412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa350107840 0x7fa35019b930 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:45.884 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.964+0000 7fa356fbe640 1 -- 172.21.15.103:0/508147412 >> 172.21.15.103:0/508147412 conn(0x7fa3501030b0 msgr2=0x7fa350103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:45.884 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.964+0000 7fa356fbe640 1 -- 172.21.15.103:0/508147412 shutdown_connections 2024-09-19T06:40:45.884 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:44.964+0000 7fa356fbe640 1 -- 172.21.15.103:0/508147412 wait complete. 2024-09-19T06:40:45.888 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T06:40:45.888 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-19T06:40:45.895 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T06:40:45.895 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-19T06:40:45.896 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-19T06:40:45.896 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T06:40:45.897 INFO:teuthology.orchestra.run.smithi103.stdout:Ceph Dashboard is now available at: 2024-09-19T06:40:45.897 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:40:45.897 INFO:teuthology.orchestra.run.smithi103.stdout: URL: https://smithi103.front.sepia.ceph.com:8443/ 2024-09-19T06:40:45.897 INFO:teuthology.orchestra.run.smithi103.stdout: User: admin 2024-09-19T06:40:45.897 INFO:teuthology.orchestra.run.smithi103.stdout: Password: r33fc82w38 2024-09-19T06:40:45.897 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:40:45.897 INFO:teuthology.orchestra.run.smithi103.stdout:Saving cluster configuration to /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config directory 2024-09-19T06:40:47.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.302+0000 7efc6480e640 1 Processor -- start 2024-09-19T06:40:47.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.302+0000 7efc6480e640 1 -- start start 2024-09-19T06:40:47.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.302+0000 7efc6480e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc60107840 0x7efc60107c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:47.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.302+0000 7efc6480e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efc60108210 con 0x7efc60107840 2024-09-19T06:40:47.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.302+0000 7efc5e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc60107840 0x7efc60107c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:47.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.302+0000 7efc5e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc60107840 0x7efc60107c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57474/0 (socket says 172.21.15.103:57474) 2024-09-19T06:40:47.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.302+0000 7efc5e59c640 1 -- 172.21.15.103:0/23087503 learned_addr learned my addr 172.21.15.103:0/23087503 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:47.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.303+0000 7efc5e59c640 1 -- 172.21.15.103:0/23087503 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc60108a40 con 0x7efc60107840 2024-09-19T06:40:47.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.303+0000 7efc5e59c640 1 --2- 172.21.15.103:0/23087503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc60107840 0x7efc60107c40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efc50009820 tx=0x7efc5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9566cd36ce52a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:47.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.304+0000 7efc5dd9b640 1 -- 172.21.15.103:0/23087503 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc50037070 con 0x7efc60107840 2024-09-19T06:40:47.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.304+0000 7efc5dd9b640 1 -- 172.21.15.103:0/23087503 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc50032920 con 0x7efc60107840 2024-09-19T06:40:47.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.304+0000 7efc5dd9b640 1 -- 172.21.15.103:0/23087503 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc50030810 con 0x7efc60107840 2024-09-19T06:40:47.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.304+0000 7efc6480e640 1 -- 172.21.15.103:0/23087503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc60107840 msgr2=0x7efc60107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:47.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.304+0000 7efc6480e640 1 --2- 172.21.15.103:0/23087503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc60107840 0x7efc60107c40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efc50009820 tx=0x7efc5002a270 comp rx=0 tx=0).stop 2024-09-19T06:40:47.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.304+0000 7efc6480e640 1 -- 172.21.15.103:0/23087503 shutdown_connections 2024-09-19T06:40:47.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.304+0000 7efc6480e640 1 --2- 172.21.15.103:0/23087503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc60107840 0x7efc60107c40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:47.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.304+0000 7efc6480e640 1 -- 172.21.15.103:0/23087503 >> 172.21.15.103:0/23087503 conn(0x7efc601030b0 msgr2=0x7efc601054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:47.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.304+0000 7efc6480e640 1 -- 172.21.15.103:0/23087503 shutdown_connections 2024-09-19T06:40:47.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.304+0000 7efc6480e640 1 -- 172.21.15.103:0/23087503 wait complete. 2024-09-19T06:40:47.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.305+0000 7efc6480e640 1 Processor -- start 2024-09-19T06:40:47.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.305+0000 7efc6480e640 1 -- start start 2024-09-19T06:40:47.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.305+0000 7efc6480e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc60107840 0x7efc6019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:47.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.305+0000 7efc6480e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efc6010bdf0 con 0x7efc60107840 2024-09-19T06:40:47.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.305+0000 7efc5e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc60107840 0x7efc6019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:47.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.305+0000 7efc5e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc60107840 0x7efc6019b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57488/0 (socket says 172.21.15.103:57488) 2024-09-19T06:40:47.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.305+0000 7efc5e59c640 1 -- 172.21.15.103:0/1308594643 learned_addr learned my addr 172.21.15.103:0/1308594643 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:47.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.306+0000 7efc5e59c640 1 -- 172.21.15.103:0/1308594643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6019be70 con 0x7efc60107840 2024-09-19T06:40:47.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.306+0000 7efc5e59c640 1 --2- 172.21.15.103:0/1308594643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc60107840 0x7efc6019b930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efc50030560 tx=0x7efc50032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:47.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.306+0000 7efc577fe640 1 -- 172.21.15.103:0/1308594643 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc50037040 con 0x7efc60107840 2024-09-19T06:40:47.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.306+0000 7efc577fe640 1 -- 172.21.15.103:0/1308594643 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc50040070 con 0x7efc60107840 2024-09-19T06:40:47.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.306+0000 7efc6480e640 1 -- 172.21.15.103:0/1308594643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6019c100 con 0x7efc60107840 2024-09-19T06:40:47.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.306+0000 7efc577fe640 1 -- 172.21.15.103:0/1308594643 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc5003a6d0 con 0x7efc60107840 2024-09-19T06:40:47.245 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.306+0000 7efc6480e640 1 -- 172.21.15.103:0/1308594643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6019edf0 con 0x7efc60107840 2024-09-19T06:40:47.245 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.307+0000 7efc577fe640 1 -- 172.21.15.103:0/1308594643 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7efc5003a940 con 0x7efc60107840 2024-09-19T06:40:47.245 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.307+0000 7efc577fe640 1 --2- 172.21.15.103:0/1308594643 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efc4c039200 0x7efc4c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:47.245 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.307+0000 7efc577fe640 1 -- 172.21.15.103:0/1308594643 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efc5006cbf0 con 0x7efc60107840 2024-09-19T06:40:47.245 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.307+0000 7efc6480e640 1 -- 172.21.15.103:0/1308594643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc6010bf70 con 0x7efc60107840 2024-09-19T06:40:47.245 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.307+0000 7efc57fff640 1 --2- 172.21.15.103:0/1308594643 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efc4c039200 0x7efc4c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:47.245 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.307+0000 7efc57fff640 1 --2- 172.21.15.103:0/1308594643 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efc4c039200 0x7efc4c03b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efc480098c0 tx=0x7efc48006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:47.245 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.312+0000 7efc577fe640 1 -- 172.21.15.103:0/1308594643 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efc50044c70 con 0x7efc60107840 2024-09-19T06:40:47.245 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.380+0000 7efc577fe640 1 -- 172.21.15.103:0/1308594643 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7efc50045980 con 0x7efc60107840 2024-09-19T06:40:47.245 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.567+0000 7efc6480e640 1 -- 172.21.15.103:0/1308594643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7efc60107cc0 con 0x7efc60107840 2024-09-19T06:40:47.245 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.569+0000 7efc577fe640 1 -- 172.21.15.103:0/1308594643 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7efc5006f120 con 0x7efc60107840 2024-09-19T06:40:47.245 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T06:40:47.246 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.571+0000 7efc6480e640 1 -- 172.21.15.103:0/1308594643 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efc4c039200 msgr2=0x7efc4c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:47.246 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.571+0000 7efc6480e640 1 --2- 172.21.15.103:0/1308594643 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efc4c039200 0x7efc4c03b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efc480098c0 tx=0x7efc48006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:47.246 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.571+0000 7efc6480e640 1 -- 172.21.15.103:0/1308594643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc60107840 msgr2=0x7efc6019b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:47.246 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.571+0000 7efc6480e640 1 --2- 172.21.15.103:0/1308594643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc60107840 0x7efc6019b930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efc50030560 tx=0x7efc50032670 comp rx=0 tx=0).stop 2024-09-19T06:40:47.246 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.571+0000 7efc6480e640 1 -- 172.21.15.103:0/1308594643 shutdown_connections 2024-09-19T06:40:47.246 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.571+0000 7efc6480e640 1 --2- 172.21.15.103:0/1308594643 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efc4c039200 0x7efc4c03b6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:47.246 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.571+0000 7efc6480e640 1 --2- 172.21.15.103:0/1308594643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc60107840 0x7efc6019b930 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:47.246 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.571+0000 7efc6480e640 1 -- 172.21.15.103:0/1308594643 >> 172.21.15.103:0/1308594643 conn(0x7efc601030b0 msgr2=0x7efc60103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:47.246 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.571+0000 7efc6480e640 1 -- 172.21.15.103:0/1308594643 shutdown_connections 2024-09-19T06:40:47.246 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T06:40:46.571+0000 7efc6480e640 1 -- 172.21.15.103:0/1308594643 wait complete. 2024-09-19T06:40:47.246 INFO:teuthology.orchestra.run.smithi103.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T06:40:47.246 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:40:47.246 INFO:teuthology.orchestra.run.smithi103.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:40:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:40:47.247 INFO:teuthology.orchestra.run.smithi103.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T06:40:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:40:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T06:40:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:40:47.247 INFO:teuthology.orchestra.run.smithi103.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T06:40:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:40:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: ceph telemetry on 2024-09-19T06:40:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:40:47.247 INFO:teuthology.orchestra.run.smithi103.stdout:For more information see: 2024-09-19T06:40:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:40:47.247 INFO:teuthology.orchestra.run.smithi103.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T06:40:47.248 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:40:47.248 INFO:teuthology.orchestra.run.smithi103.stdout:Bootstrap complete. 2024-09-19T06:40:47.286 INFO:tasks.cephadm:Fetching config... 2024-09-19T06:40:47.286 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:40:47.287 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T06:40:47.304 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T06:40:47.304 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:40:47.304 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T06:40:47.362 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T06:40:47.362 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:40:47.362 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T06:40:47.428 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T06:40:47.428 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:40:47.428 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T06:40:47.483 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T06:40:47.483 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCm4oM0Z3ASq1o0tmL88hCDCE+YP5L9i7vnvobQ3kANsjgcObTOw9y8v64SaSp1R4u+iFHxUacfzCZd/G/s8r1lwY0a+HMeKSrS2O96C1okb9/yK5KNIyBwY/1/9NlEFlvJTg2oAPgqUIgDZpM3GkizYxxccEzfvug5uB5ca2hmfnfGfvWlYUnz+LK+RVUxqaUoNWqNCvNBn5WMkAc7ya2LMHbv9Xv5Xny8NRtQPKXqo6o0dzVpYI/5qFMaJZXqOOV48YphqEjN68Re+krWFfWAU2m4XriZjM9DZNJwE/2tQNBVqoKgRCWaybozSeiYCk9VAnwsA/rEtR2buBDJhZ0AxHW3nfpsiMrWGI14kJjKNRK0XQ5JqR4H8H7ZIweSj9aq1oBAikpZAFdX0FlyeCmNYEfi8a8QlkT47jpOhz+YuAAg5/at7F5y58qYYjfCdZg5WEJSswJ32jT3IZKBvFYy7uUiILY/0BwZK8EioF/LfJ/y39zMb+Sb6fAICUZMEeE= ceph-d158d934-7651-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T06:40:47.531 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:47 smithi103 ceph-mon[31158]: mgrmap e13: x(active, since 6s) 2024-09-19T06:40:47.531 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:47 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1308594643' entity='client.admin' 2024-09-19T06:40:47.559 INFO:teuthology.orchestra.run.smithi103.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCm4oM0Z3ASq1o0tmL88hCDCE+YP5L9i7vnvobQ3kANsjgcObTOw9y8v64SaSp1R4u+iFHxUacfzCZd/G/s8r1lwY0a+HMeKSrS2O96C1okb9/yK5KNIyBwY/1/9NlEFlvJTg2oAPgqUIgDZpM3GkizYxxccEzfvug5uB5ca2hmfnfGfvWlYUnz+LK+RVUxqaUoNWqNCvNBn5WMkAc7ya2LMHbv9Xv5Xny8NRtQPKXqo6o0dzVpYI/5qFMaJZXqOOV48YphqEjN68Re+krWFfWAU2m4XriZjM9DZNJwE/2tQNBVqoKgRCWaybozSeiYCk9VAnwsA/rEtR2buBDJhZ0AxHW3nfpsiMrWGI14kJjKNRK0XQ5JqR4H8H7ZIweSj9aq1oBAikpZAFdX0FlyeCmNYEfi8a8QlkT47jpOhz+YuAAg5/at7F5y58qYYjfCdZg5WEJSswJ32jT3IZKBvFYy7uUiILY/0BwZK8EioF/LfJ/y39zMb+Sb6fAICUZMEeE= ceph-d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:40:47.570 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCm4oM0Z3ASq1o0tmL88hCDCE+YP5L9i7vnvobQ3kANsjgcObTOw9y8v64SaSp1R4u+iFHxUacfzCZd/G/s8r1lwY0a+HMeKSrS2O96C1okb9/yK5KNIyBwY/1/9NlEFlvJTg2oAPgqUIgDZpM3GkizYxxccEzfvug5uB5ca2hmfnfGfvWlYUnz+LK+RVUxqaUoNWqNCvNBn5WMkAc7ya2LMHbv9Xv5Xny8NRtQPKXqo6o0dzVpYI/5qFMaJZXqOOV48YphqEjN68Re+krWFfWAU2m4XriZjM9DZNJwE/2tQNBVqoKgRCWaybozSeiYCk9VAnwsA/rEtR2buBDJhZ0AxHW3nfpsiMrWGI14kJjKNRK0XQ5JqR4H8H7ZIweSj9aq1oBAikpZAFdX0FlyeCmNYEfi8a8QlkT47jpOhz+YuAAg5/at7F5y58qYYjfCdZg5WEJSswJ32jT3IZKBvFYy7uUiILY/0BwZK8EioF/LfJ/y39zMb+Sb6fAICUZMEeE= ceph-d158d934-7651-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T06:40:47.613 INFO:teuthology.orchestra.run.smithi134.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCm4oM0Z3ASq1o0tmL88hCDCE+YP5L9i7vnvobQ3kANsjgcObTOw9y8v64SaSp1R4u+iFHxUacfzCZd/G/s8r1lwY0a+HMeKSrS2O96C1okb9/yK5KNIyBwY/1/9NlEFlvJTg2oAPgqUIgDZpM3GkizYxxccEzfvug5uB5ca2hmfnfGfvWlYUnz+LK+RVUxqaUoNWqNCvNBn5WMkAc7ya2LMHbv9Xv5Xny8NRtQPKXqo6o0dzVpYI/5qFMaJZXqOOV48YphqEjN68Re+krWFfWAU2m4XriZjM9DZNJwE/2tQNBVqoKgRCWaybozSeiYCk9VAnwsA/rEtR2buBDJhZ0AxHW3nfpsiMrWGI14kJjKNRK0XQ5JqR4H8H7ZIweSj9aq1oBAikpZAFdX0FlyeCmNYEfi8a8QlkT47jpOhz+YuAAg5/at7F5y58qYYjfCdZg5WEJSswJ32jT3IZKBvFYy7uUiILY/0BwZK8EioF/LfJ/y39zMb+Sb6fAICUZMEeE= ceph-d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:40:47.628 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCm4oM0Z3ASq1o0tmL88hCDCE+YP5L9i7vnvobQ3kANsjgcObTOw9y8v64SaSp1R4u+iFHxUacfzCZd/G/s8r1lwY0a+HMeKSrS2O96C1okb9/yK5KNIyBwY/1/9NlEFlvJTg2oAPgqUIgDZpM3GkizYxxccEzfvug5uB5ca2hmfnfGfvWlYUnz+LK+RVUxqaUoNWqNCvNBn5WMkAc7ya2LMHbv9Xv5Xny8NRtQPKXqo6o0dzVpYI/5qFMaJZXqOOV48YphqEjN68Re+krWFfWAU2m4XriZjM9DZNJwE/2tQNBVqoKgRCWaybozSeiYCk9VAnwsA/rEtR2buBDJhZ0AxHW3nfpsiMrWGI14kJjKNRK0XQ5JqR4H8H7ZIweSj9aq1oBAikpZAFdX0FlyeCmNYEfi8a8QlkT47jpOhz+YuAAg5/at7F5y58qYYjfCdZg5WEJSswJ32jT3IZKBvFYy7uUiILY/0BwZK8EioF/LfJ/y39zMb+Sb6fAICUZMEeE= ceph-d158d934-7651-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T06:40:47.671 INFO:teuthology.orchestra.run.smithi174.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCm4oM0Z3ASq1o0tmL88hCDCE+YP5L9i7vnvobQ3kANsjgcObTOw9y8v64SaSp1R4u+iFHxUacfzCZd/G/s8r1lwY0a+HMeKSrS2O96C1okb9/yK5KNIyBwY/1/9NlEFlvJTg2oAPgqUIgDZpM3GkizYxxccEzfvug5uB5ca2hmfnfGfvWlYUnz+LK+RVUxqaUoNWqNCvNBn5WMkAc7ya2LMHbv9Xv5Xny8NRtQPKXqo6o0dzVpYI/5qFMaJZXqOOV48YphqEjN68Re+krWFfWAU2m4XriZjM9DZNJwE/2tQNBVqoKgRCWaybozSeiYCk9VAnwsA/rEtR2buBDJhZ0AxHW3nfpsiMrWGI14kJjKNRK0XQ5JqR4H8H7ZIweSj9aq1oBAikpZAFdX0FlyeCmNYEfi8a8QlkT47jpOhz+YuAAg5/at7F5y58qYYjfCdZg5WEJSswJ32jT3IZKBvFYy7uUiILY/0BwZK8EioF/LfJ/y39zMb+Sb6fAICUZMEeE= ceph-d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:40:47.684 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T06:40:47.939 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:48.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.397+0000 7f8ad1089640 1 -- 172.21.15.103:0/2569119203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8acc101640 msgr2=0x7f8acc101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:48.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.397+0000 7f8ad1089640 1 --2- 172.21.15.103:0/2569119203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8acc101640 0x7f8acc101a40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009880 tx=0x7f8abc02a5d0 comp rx=0 tx=0).stop 2024-09-19T06:40:48.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.397+0000 7f8ad1089640 1 -- 172.21.15.103:0/2569119203 shutdown_connections 2024-09-19T06:40:48.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.397+0000 7f8ad1089640 1 --2- 172.21.15.103:0/2569119203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8acc101640 0x7f8acc101a40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:48.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.397+0000 7f8ad1089640 1 -- 172.21.15.103:0/2569119203 >> 172.21.15.103:0/2569119203 conn(0x7f8acc0fcdf0 msgr2=0x7f8acc0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:48.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.397+0000 7f8ad1089640 1 -- 172.21.15.103:0/2569119203 shutdown_connections 2024-09-19T06:40:48.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.397+0000 7f8ad1089640 1 -- 172.21.15.103:0/2569119203 wait complete. 2024-09-19T06:40:48.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.398+0000 7f8ad1089640 1 Processor -- start 2024-09-19T06:40:48.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.398+0000 7f8ad1089640 1 -- start start 2024-09-19T06:40:48.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.398+0000 7f8ad1089640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8acc101640 0x7f8acc196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:48.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.398+0000 7f8ad1089640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc107750 con 0x7f8acc101640 2024-09-19T06:40:48.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.398+0000 7f8acad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8acc101640 0x7f8acc196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:48.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.399+0000 7f8acad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8acc101640 0x7f8acc196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41936/0 (socket says 172.21.15.103:41936) 2024-09-19T06:40:48.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.399+0000 7f8acad9d640 1 -- 172.21.15.103:0/361336786 learned_addr learned my addr 172.21.15.103:0/361336786 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:48.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.399+0000 7f8acad9d640 1 -- 172.21.15.103:0/361336786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc197390 con 0x7f8acc101640 2024-09-19T06:40:48.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.399+0000 7f8acad9d640 1 --2- 172.21.15.103:0/361336786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8acc101640 0x7f8acc196e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009230 tx=0x7f8abc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:48.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.399+0000 7f8ab3fff640 1 -- 172.21.15.103:0/361336786 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8abc03f070 con 0x7f8acc101640 2024-09-19T06:40:48.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.399+0000 7f8ab3fff640 1 -- 172.21.15.103:0/361336786 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8abc003ad0 con 0x7f8acc101640 2024-09-19T06:40:48.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.399+0000 7f8ab3fff640 1 -- 172.21.15.103:0/361336786 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8abc035030 con 0x7f8acc101640 2024-09-19T06:40:48.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.399+0000 7f8ad1089640 1 -- 172.21.15.103:0/361336786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8acc197620 con 0x7f8acc101640 2024-09-19T06:40:48.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.399+0000 7f8ad1089640 1 -- 172.21.15.103:0/361336786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8acc19a310 con 0x7f8acc101640 2024-09-19T06:40:48.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.400+0000 7f8ab3fff640 1 -- 172.21.15.103:0/361336786 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f8abc042070 con 0x7f8acc101640 2024-09-19T06:40:48.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.400+0000 7f8ad1089640 1 -- 172.21.15.103:0/361336786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8acc107ed0 con 0x7f8acc101640 2024-09-19T06:40:48.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.400+0000 7f8ab3fff640 1 --2- 172.21.15.103:0/361336786 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8ab40392a0 0x7f8ab403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:48.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.401+0000 7f8ab3fff640 1 -- 172.21.15.103:0/361336786 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8abc06d060 con 0x7f8acc101640 2024-09-19T06:40:48.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.401+0000 7f8aca59c640 1 --2- 172.21.15.103:0/361336786 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8ab40392a0 0x7f8ab403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:48.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.401+0000 7f8aca59c640 1 --2- 172.21.15.103:0/361336786 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8ab40392a0 0x7f8ab403b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8ac00098c0 tx=0x7f8ac0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:48.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.405+0000 7f8ab3fff640 1 -- 172.21.15.103:0/361336786 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8abc02f300 con 0x7f8acc101640 2024-09-19T06:40:48.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.580+0000 7f8ad1089640 1 -- 172.21.15.103:0/361336786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f8acc02d830 con 0x7f8acc101640 2024-09-19T06:40:48.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.582+0000 7f8ab3fff640 1 -- 172.21.15.103:0/361336786 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f8abc003c80 con 0x7f8acc101640 2024-09-19T06:40:48.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.585+0000 7f8ad1089640 1 -- 172.21.15.103:0/361336786 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8ab40392a0 msgr2=0x7f8ab403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:48.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.585+0000 7f8ad1089640 1 --2- 172.21.15.103:0/361336786 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8ab40392a0 0x7f8ab403b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8ac00098c0 tx=0x7f8ac0006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:48.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.585+0000 7f8ad1089640 1 -- 172.21.15.103:0/361336786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8acc101640 msgr2=0x7f8acc196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:48.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.585+0000 7f8ad1089640 1 --2- 172.21.15.103:0/361336786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8acc101640 0x7f8acc196e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009230 tx=0x7f8abc0048f0 comp rx=0 tx=0).stop 2024-09-19T06:40:48.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.585+0000 7f8ad1089640 1 -- 172.21.15.103:0/361336786 shutdown_connections 2024-09-19T06:40:48.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.585+0000 7f8ad1089640 1 --2- 172.21.15.103:0/361336786 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8ab40392a0 0x7f8ab403b760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:48.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.585+0000 7f8ad1089640 1 --2- 172.21.15.103:0/361336786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8acc101640 0x7f8acc196e50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:48.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.585+0000 7f8ad1089640 1 -- 172.21.15.103:0/361336786 >> 172.21.15.103:0/361336786 conn(0x7f8acc0fcdf0 msgr2=0x7f8acc0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:48.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.585+0000 7f8ad1089640 1 -- 172.21.15.103:0/361336786 shutdown_connections 2024-09-19T06:40:48.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:48.585+0000 7f8ad1089640 1 -- 172.21.15.103:0/361336786 wait complete. 2024-09-19T06:40:49.203 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T06:40:49.204 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T06:40:49.848 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:49 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/361336786' entity='client.admin' 2024-09-19T06:40:50.180 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.664+0000 7f1866596640 1 -- 172.21.15.103:0/2323787417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1860101640 msgr2=0x7f1860101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.664+0000 7f1866596640 1 --2- 172.21.15.103:0/2323787417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1860101640 0x7f1860101a40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1850009880 tx=0x7f185002a5d0 comp rx=0 tx=0).stop 2024-09-19T06:40:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.664+0000 7f1866596640 1 -- 172.21.15.103:0/2323787417 shutdown_connections 2024-09-19T06:40:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.664+0000 7f1866596640 1 --2- 172.21.15.103:0/2323787417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1860101640 0x7f1860101a40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.664+0000 7f1866596640 1 -- 172.21.15.103:0/2323787417 >> 172.21.15.103:0/2323787417 conn(0x7f18600fcdf0 msgr2=0x7f18600ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.664+0000 7f1866596640 1 -- 172.21.15.103:0/2323787417 shutdown_connections 2024-09-19T06:40:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.664+0000 7f1866596640 1 -- 172.21.15.103:0/2323787417 wait complete. 2024-09-19T06:40:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.664+0000 7f1866596640 1 Processor -- start 2024-09-19T06:40:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.664+0000 7f1866596640 1 -- start start 2024-09-19T06:40:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.665+0000 7f1866596640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1860101640 0x7f1860196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.665+0000 7f1866596640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1860107750 con 0x7f1860101640 2024-09-19T06:40:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.665+0000 7f185ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1860101640 0x7f1860196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.665+0000 7f185ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1860101640 0x7f1860196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41960/0 (socket says 172.21.15.103:41960) 2024-09-19T06:40:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.665+0000 7f185ffff640 1 -- 172.21.15.103:0/3486204540 learned_addr learned my addr 172.21.15.103:0/3486204540 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.665+0000 7f185ffff640 1 -- 172.21.15.103:0/3486204540 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1860197370 con 0x7f1860101640 2024-09-19T06:40:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.665+0000 7f185ffff640 1 --2- 172.21.15.103:0/3486204540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1860101640 0x7f1860196e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1850009230 tx=0x7f18500048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.665+0000 7f185dffb640 1 -- 172.21.15.103:0/3486204540 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f185003f070 con 0x7f1860101640 2024-09-19T06:40:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.665+0000 7f185dffb640 1 -- 172.21.15.103:0/3486204540 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1850003ad0 con 0x7f1860101640 2024-09-19T06:40:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.665+0000 7f185dffb640 1 -- 172.21.15.103:0/3486204540 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1850035030 con 0x7f1860101640 2024-09-19T06:40:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.665+0000 7f1866596640 1 -- 172.21.15.103:0/3486204540 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1860197600 con 0x7f1860101640 2024-09-19T06:40:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.666+0000 7f1866596640 1 -- 172.21.15.103:0/3486204540 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186019a2f0 con 0x7f1860101640 2024-09-19T06:40:50.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.666+0000 7f185dffb640 1 -- 172.21.15.103:0/3486204540 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f1850042070 con 0x7f1860101640 2024-09-19T06:40:50.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.667+0000 7f1866596640 1 -- 172.21.15.103:0/3486204540 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1860107ed0 con 0x7f1860101640 2024-09-19T06:40:50.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.667+0000 7f185dffb640 1 --2- 172.21.15.103:0/3486204540 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f184c039300 0x7f184c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:50.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.667+0000 7f185dffb640 1 -- 172.21.15.103:0/3486204540 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f185006d070 con 0x7f1860101640 2024-09-19T06:40:50.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.667+0000 7f1857fff640 1 --2- 172.21.15.103:0/3486204540 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f184c039300 0x7f184c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:50.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.667+0000 7f1857fff640 1 --2- 172.21.15.103:0/3486204540 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f184c039300 0x7f184c03b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f18480098c0 tx=0x7f1848006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:50.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.671+0000 7f185dffb640 1 -- 172.21.15.103:0/3486204540 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f185003a3c0 con 0x7f1860101640 2024-09-19T06:40:50.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:50.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:50.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:40:50.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:50.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:40:50.706 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:50.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.852+0000 7f1866596640 1 -- 172.21.15.103:0/3486204540 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f186002d830 con 0x7f184c039300 2024-09-19T06:40:50.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.856+0000 7f185dffb640 1 -- 172.21.15.103:0/3486204540 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f186002d830 con 0x7f184c039300 2024-09-19T06:40:50.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.858+0000 7f1866596640 1 -- 172.21.15.103:0/3486204540 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f184c039300 msgr2=0x7f184c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:50.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.858+0000 7f1866596640 1 --2- 172.21.15.103:0/3486204540 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f184c039300 0x7f184c03b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f18480098c0 tx=0x7f1848006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:50.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.858+0000 7f1866596640 1 -- 172.21.15.103:0/3486204540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1860101640 msgr2=0x7f1860196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:50.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.858+0000 7f1866596640 1 --2- 172.21.15.103:0/3486204540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1860101640 0x7f1860196e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1850009230 tx=0x7f18500048f0 comp rx=0 tx=0).stop 2024-09-19T06:40:50.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.858+0000 7f1866596640 1 -- 172.21.15.103:0/3486204540 shutdown_connections 2024-09-19T06:40:50.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.858+0000 7f1866596640 1 --2- 172.21.15.103:0/3486204540 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f184c039300 0x7f184c03b7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:50.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.858+0000 7f1866596640 1 --2- 172.21.15.103:0/3486204540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1860101640 0x7f1860196e30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:50.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.858+0000 7f1866596640 1 -- 172.21.15.103:0/3486204540 >> 172.21.15.103:0/3486204540 conn(0x7f18600fcdf0 msgr2=0x7f18600fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:50.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.858+0000 7f1866596640 1 -- 172.21.15.103:0/3486204540 shutdown_connections 2024-09-19T06:40:50.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:50.858+0000 7f1866596640 1 -- 172.21.15.103:0/3486204540 wait complete. 2024-09-19T06:40:51.669 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi134 2024-09-19T06:40:51.669 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T06:40:51.669 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.conf 2024-09-19T06:40:51.688 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T06:40:51.689 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:40:51.746 INFO:tasks.cephadm:Adding host smithi134 to orchestrator... 2024-09-19T06:40:51.746 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch host add smithi134 2024-09-19T06:40:51.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:51 smithi103 ceph-mon[31158]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:51.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:51 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:51.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:51 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:40:51.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:51 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:40:51.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:51 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:40:51.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:51 smithi103 ceph-mon[31158]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-19T06:40:51.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:51 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:51.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:51 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:51.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:51 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:52.169 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.599+0000 7f9a964f8640 1 -- 172.21.15.103:0/2949211227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a90102780 msgr2=0x7f9a90102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.599+0000 7f9a964f8640 1 --2- 172.21.15.103:0/2949211227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a90102780 0x7f9a90102b60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009880 tx=0x7f9a8002a670 comp rx=0 tx=0).stop 2024-09-19T06:40:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.599+0000 7f9a964f8640 1 -- 172.21.15.103:0/2949211227 shutdown_connections 2024-09-19T06:40:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.599+0000 7f9a964f8640 1 --2- 172.21.15.103:0/2949211227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a90102780 0x7f9a90102b60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.599+0000 7f9a964f8640 1 -- 172.21.15.103:0/2949211227 >> 172.21.15.103:0/2949211227 conn(0x7f9a900fc1a0 msgr2=0x7f9a900fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.599+0000 7f9a964f8640 1 -- 172.21.15.103:0/2949211227 shutdown_connections 2024-09-19T06:40:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.599+0000 7f9a964f8640 1 -- 172.21.15.103:0/2949211227 wait complete. 2024-09-19T06:40:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.600+0000 7f9a964f8640 1 Processor -- start 2024-09-19T06:40:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.600+0000 7f9a964f8640 1 -- start start 2024-09-19T06:40:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.600+0000 7f9a964f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a90102780 0x7f9a90197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.600+0000 7f9a964f8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9010bcf0 con 0x7f9a90102780 2024-09-19T06:40:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.600+0000 7f9a8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a90102780 0x7f9a90197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.600+0000 7f9a8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a90102780 0x7f9a90197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41978/0 (socket says 172.21.15.103:41978) 2024-09-19T06:40:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.600+0000 7f9a8ffff640 1 -- 172.21.15.103:0/4054898861 learned_addr learned my addr 172.21.15.103:0/4054898861 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:52.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.601+0000 7f9a8ffff640 1 -- 172.21.15.103:0/4054898861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a90197770 con 0x7f9a90102780 2024-09-19T06:40:52.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.601+0000 7f9a8ffff640 1 --2- 172.21.15.103:0/4054898861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a90102780 0x7f9a90197230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009230 tx=0x7f9a80003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:52.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.601+0000 7f9a8d7fa640 1 -- 172.21.15.103:0/4054898861 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a8003f070 con 0x7f9a90102780 2024-09-19T06:40:52.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.601+0000 7f9a8d7fa640 1 -- 172.21.15.103:0/4054898861 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8003bc80 con 0x7f9a90102780 2024-09-19T06:40:52.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.601+0000 7f9a964f8640 1 -- 172.21.15.103:0/4054898861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a90197a00 con 0x7f9a90102780 2024-09-19T06:40:52.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.601+0000 7f9a8d7fa640 1 -- 172.21.15.103:0/4054898861 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a80035030 con 0x7f9a90102780 2024-09-19T06:40:52.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.601+0000 7f9a964f8640 1 -- 172.21.15.103:0/4054898861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a90197e60 con 0x7f9a90102780 2024-09-19T06:40:52.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.602+0000 7f9a8d7fa640 1 -- 172.21.15.103:0/4054898861 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f9a80042070 con 0x7f9a90102780 2024-09-19T06:40:52.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.602+0000 7f9a8d7fa640 1 --2- 172.21.15.103:0/4054898861 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9a780392a0 0x7f9a7803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:52.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.602+0000 7f9a964f8640 1 -- 172.21.15.103:0/4054898861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a9006a7c0 con 0x7f9a90102780 2024-09-19T06:40:52.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.602+0000 7f9a8d7fa640 1 -- 172.21.15.103:0/4054898861 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9a8006cff0 con 0x7f9a90102780 2024-09-19T06:40:52.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.604+0000 7f9a8f7fe640 1 --2- 172.21.15.103:0/4054898861 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9a780392a0 0x7f9a7803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:52.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.604+0000 7f9a8f7fe640 1 --2- 172.21.15.103:0/4054898861 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9a780392a0 0x7f9a7803b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9a840098c0 tx=0x7f9a84006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:52.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.607+0000 7f9a8d7fa640 1 -- 172.21.15.103:0/4054898861 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9a800314c0 con 0x7f9a90102780 2024-09-19T06:40:52.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:52.777+0000 7f9a964f8640 1 -- 172.21.15.103:0/4054898861 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}) -- 0x7f9a90198590 con 0x7f9a780392a0 2024-09-19T06:40:53.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:52 smithi103 ceph-mon[31158]: Updating smithi103:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:40:53.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:52 smithi103 ceph-mon[31158]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:40:53.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:52 smithi103 ceph-mon[31158]: Updating smithi103:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:40:54.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:53 smithi103 ceph-mon[31158]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:55.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:54 smithi103 ceph-mon[31158]: Deploying cephadm binary to smithi134 2024-09-19T06:40:55.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:55.927+0000 7f9a8d7fa640 1 -- 172.21.15.103:0/4054898861 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9a90198590 con 0x7f9a780392a0 2024-09-19T06:40:55.929 INFO:teuthology.orchestra.run.smithi103.stdout:Added host 'smithi134' with addr '172.21.15.134' 2024-09-19T06:40:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:55.930+0000 7f9a964f8640 1 -- 172.21.15.103:0/4054898861 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9a780392a0 msgr2=0x7f9a7803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:55.930+0000 7f9a964f8640 1 --2- 172.21.15.103:0/4054898861 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9a780392a0 0x7f9a7803b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9a840098c0 tx=0x7f9a84006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:55.930+0000 7f9a964f8640 1 -- 172.21.15.103:0/4054898861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a90102780 msgr2=0x7f9a90197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:55.930+0000 7f9a964f8640 1 --2- 172.21.15.103:0/4054898861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a90102780 0x7f9a90197230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009230 tx=0x7f9a80003d40 comp rx=0 tx=0).stop 2024-09-19T06:40:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:55.930+0000 7f9a964f8640 1 -- 172.21.15.103:0/4054898861 shutdown_connections 2024-09-19T06:40:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:55.930+0000 7f9a964f8640 1 --2- 172.21.15.103:0/4054898861 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9a780392a0 0x7f9a7803b760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:55.930+0000 7f9a964f8640 1 --2- 172.21.15.103:0/4054898861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a90102780 0x7f9a90197230 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:55.930+0000 7f9a964f8640 1 -- 172.21.15.103:0/4054898861 >> 172.21.15.103:0/4054898861 conn(0x7f9a900fc1a0 msgr2=0x7f9a90105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:55.930+0000 7f9a964f8640 1 -- 172.21.15.103:0/4054898861 shutdown_connections 2024-09-19T06:40:55.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:55.930+0000 7f9a964f8640 1 -- 172.21.15.103:0/4054898861 wait complete. 2024-09-19T06:40:56.493 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T06:40:56.734 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:56 smithi103 ceph-mon[31158]: Added host smithi134 2024-09-19T06:40:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:40:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:57.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.254+0000 7fb5e675e640 1 -- 172.21.15.103:0/381197964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5e0106590 msgr2=0x7fb5e0106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.254+0000 7fb5e675e640 1 --2- 172.21.15.103:0/381197964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5e0106590 0x7fb5e0106970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009880 tx=0x7fb5d002a5d0 comp rx=0 tx=0).stop 2024-09-19T06:40:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.254+0000 7fb5e675e640 1 -- 172.21.15.103:0/381197964 shutdown_connections 2024-09-19T06:40:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.254+0000 7fb5e675e640 1 --2- 172.21.15.103:0/381197964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5e0106590 0x7fb5e0106970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.254+0000 7fb5e675e640 1 -- 172.21.15.103:0/381197964 >> 172.21.15.103:0/381197964 conn(0x7fb5e00fc1a0 msgr2=0x7fb5e00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.254+0000 7fb5e675e640 1 -- 172.21.15.103:0/381197964 shutdown_connections 2024-09-19T06:40:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.254+0000 7fb5e675e640 1 -- 172.21.15.103:0/381197964 wait complete. 2024-09-19T06:40:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.255+0000 7fb5e675e640 1 Processor -- start 2024-09-19T06:40:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.255+0000 7fb5e675e640 1 -- start start 2024-09-19T06:40:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.255+0000 7fb5e675e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5e0106590 0x7fb5e00fc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.255+0000 7fb5e675e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e010abc0 con 0x7fb5e0106590 2024-09-19T06:40:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.255+0000 7fb5dffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5e0106590 0x7fb5e00fc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.255+0000 7fb5dffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5e0106590 0x7fb5e00fc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41994/0 (socket says 172.21.15.103:41994) 2024-09-19T06:40:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.255+0000 7fb5dffff640 1 -- 172.21.15.103:0/1830208954 learned_addr learned my addr 172.21.15.103:0/1830208954 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.255+0000 7fb5dffff640 1 -- 172.21.15.103:0/1830208954 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e00fe540 con 0x7fb5e0106590 2024-09-19T06:40:57.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.255+0000 7fb5dffff640 1 --2- 172.21.15.103:0/1830208954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5e0106590 0x7fb5e00fc650 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009230 tx=0x7fb5d0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:57.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.256+0000 7fb5dd7fa640 1 -- 172.21.15.103:0/1830208954 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5d003f070 con 0x7fb5e0106590 2024-09-19T06:40:57.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.256+0000 7fb5dd7fa640 1 -- 172.21.15.103:0/1830208954 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5d003bc80 con 0x7fb5e0106590 2024-09-19T06:40:57.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.256+0000 7fb5e675e640 1 -- 172.21.15.103:0/1830208954 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e00fcb90 con 0x7fb5e0106590 2024-09-19T06:40:57.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.256+0000 7fb5dd7fa640 1 -- 172.21.15.103:0/1830208954 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5d0035030 con 0x7fb5e0106590 2024-09-19T06:40:57.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.256+0000 7fb5e675e640 1 -- 172.21.15.103:0/1830208954 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e00fd050 con 0x7fb5e0106590 2024-09-19T06:40:57.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.257+0000 7fb5dd7fa640 1 -- 172.21.15.103:0/1830208954 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fb5d0042070 con 0x7fb5e0106590 2024-09-19T06:40:57.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.257+0000 7fb5dd7fa640 1 --2- 172.21.15.103:0/1830208954 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb5c80392a0 0x7fb5c803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:57.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.257+0000 7fb5dd7fa640 1 -- 172.21.15.103:0/1830208954 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb5d006cfd0 con 0x7fb5e0106590 2024-09-19T06:40:57.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.257+0000 7fb5e675e640 1 -- 172.21.15.103:0/1830208954 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e0101c00 con 0x7fb5e0106590 2024-09-19T06:40:57.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.257+0000 7fb5df7fe640 1 --2- 172.21.15.103:0/1830208954 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb5c80392a0 0x7fb5c803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:57.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.257+0000 7fb5df7fe640 1 --2- 172.21.15.103:0/1830208954 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb5c80392a0 0x7fb5c803b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40098c0 tx=0x7fb5d4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:57.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.262+0000 7fb5dd7fa640 1 -- 172.21.15.103:0/1830208954 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb5d00372d0 con 0x7fb5e0106590 2024-09-19T06:40:57.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.436+0000 7fb5e675e640 1 -- 172.21.15.103:0/1830208954 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb5e00fdb80 con 0x7fb5c80392a0 2024-09-19T06:40:57.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.437+0000 7fb5dd7fa640 1 -- 172.21.15.103:0/1830208954 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fb5e00fdb80 con 0x7fb5c80392a0 2024-09-19T06:40:57.438 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:40:57.438 INFO:teuthology.orchestra.run.smithi103.stdout:[{"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}] 2024-09-19T06:40:57.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.439+0000 7fb5e675e640 1 -- 172.21.15.103:0/1830208954 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb5c80392a0 msgr2=0x7fb5c803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:57.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.439+0000 7fb5e675e640 1 --2- 172.21.15.103:0/1830208954 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb5c80392a0 0x7fb5c803b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40098c0 tx=0x7fb5d4006db0 comp rx=0 tx=0).stop 2024-09-19T06:40:57.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.439+0000 7fb5e675e640 1 -- 172.21.15.103:0/1830208954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5e0106590 msgr2=0x7fb5e00fc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:57.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.439+0000 7fb5e675e640 1 --2- 172.21.15.103:0/1830208954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5e0106590 0x7fb5e00fc650 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009230 tx=0x7fb5d0003d40 comp rx=0 tx=0).stop 2024-09-19T06:40:57.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.439+0000 7fb5e675e640 1 -- 172.21.15.103:0/1830208954 shutdown_connections 2024-09-19T06:40:57.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.439+0000 7fb5e675e640 1 --2- 172.21.15.103:0/1830208954 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb5c80392a0 0x7fb5c803b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:57.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.439+0000 7fb5e675e640 1 --2- 172.21.15.103:0/1830208954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5e0106590 0x7fb5e00fc650 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:57.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.439+0000 7fb5e675e640 1 -- 172.21.15.103:0/1830208954 >> 172.21.15.103:0/1830208954 conn(0x7fb5e00fc1a0 msgr2=0x7fb5e0069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:57.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.439+0000 7fb5e675e640 1 -- 172.21.15.103:0/1830208954 shutdown_connections 2024-09-19T06:40:57.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:57.439+0000 7fb5e675e640 1 -- 172.21.15.103:0/1830208954 wait complete. 2024-09-19T06:40:58.066 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi174 2024-09-19T06:40:58.066 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T06:40:58.066 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.conf 2024-09-19T06:40:58.086 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T06:40:58.086 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:40:58.144 INFO:tasks.cephadm:Adding host smithi174 to orchestrator... 2024-09-19T06:40:58.144 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch host add smithi174 2024-09-19T06:40:58.387 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:58.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:58 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:40:58.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:40:58 smithi103 ceph-mon[31158]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:40:58.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.795+0000 7f45cd0cc640 1 -- 172.21.15.103:0/3778085434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c8106590 msgr2=0x7f45c8106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:58.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.795+0000 7f45cd0cc640 1 --2- 172.21.15.103:0/3778085434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c8106590 0x7f45c8106970 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009880 tx=0x7f45b802a5d0 comp rx=0 tx=0).stop 2024-09-19T06:40:58.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.795+0000 7f45cd0cc640 1 -- 172.21.15.103:0/3778085434 shutdown_connections 2024-09-19T06:40:58.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.795+0000 7f45cd0cc640 1 --2- 172.21.15.103:0/3778085434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c8106590 0x7f45c8106970 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:58.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.795+0000 7f45cd0cc640 1 -- 172.21.15.103:0/3778085434 >> 172.21.15.103:0/3778085434 conn(0x7f45c80fc1a0 msgr2=0x7f45c80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:58.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.795+0000 7f45cd0cc640 1 -- 172.21.15.103:0/3778085434 shutdown_connections 2024-09-19T06:40:58.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.795+0000 7f45cd0cc640 1 -- 172.21.15.103:0/3778085434 wait complete. 2024-09-19T06:40:58.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.796+0000 7f45cd0cc640 1 Processor -- start 2024-09-19T06:40:58.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.796+0000 7f45cd0cc640 1 -- start start 2024-09-19T06:40:58.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.796+0000 7f45cd0cc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c8106590 0x7f45c8197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:58.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.796+0000 7f45cd0cc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45c810abc0 con 0x7f45c8106590 2024-09-19T06:40:58.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.796+0000 7f45c6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c8106590 0x7f45c8197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:58.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.796+0000 7f45c6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c8106590 0x7f45c8197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45188/0 (socket says 172.21.15.103:45188) 2024-09-19T06:40:58.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.796+0000 7f45c6d9d640 1 -- 172.21.15.103:0/2944449139 learned_addr learned my addr 172.21.15.103:0/2944449139 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:40:58.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.797+0000 7f45c6d9d640 1 -- 172.21.15.103:0/2944449139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c81984a0 con 0x7f45c8106590 2024-09-19T06:40:58.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.797+0000 7f45c6d9d640 1 --2- 172.21.15.103:0/2944449139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c8106590 0x7f45c8197f60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009230 tx=0x7f45b8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:58.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.797+0000 7f45bffff640 1 -- 172.21.15.103:0/2944449139 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45b803f070 con 0x7f45c8106590 2024-09-19T06:40:58.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.797+0000 7f45bffff640 1 -- 172.21.15.103:0/2944449139 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b803bc80 con 0x7f45c8106590 2024-09-19T06:40:58.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.797+0000 7f45cd0cc640 1 -- 172.21.15.103:0/2944449139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c8198730 con 0x7f45c8106590 2024-09-19T06:40:58.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.797+0000 7f45bffff640 1 -- 172.21.15.103:0/2944449139 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45b8035030 con 0x7f45c8106590 2024-09-19T06:40:58.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.797+0000 7f45cd0cc640 1 -- 172.21.15.103:0/2944449139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c8198b90 con 0x7f45c8106590 2024-09-19T06:40:58.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.798+0000 7f45bffff640 1 -- 172.21.15.103:0/2944449139 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f45b8042070 con 0x7f45c8106590 2024-09-19T06:40:58.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.798+0000 7f45cd0cc640 1 -- 172.21.15.103:0/2944449139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c8101c00 con 0x7f45c8106590 2024-09-19T06:40:58.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.798+0000 7f45bffff640 1 --2- 172.21.15.103:0/2944449139 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f45b40392a0 0x7f45b403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:58.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.798+0000 7f45bffff640 1 -- 172.21.15.103:0/2944449139 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f45b806d0d0 con 0x7f45c8106590 2024-09-19T06:40:58.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.798+0000 7f45c659c640 1 --2- 172.21.15.103:0/2944449139 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f45b40392a0 0x7f45b403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:58.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.799+0000 7f45c659c640 1 --2- 172.21.15.103:0/2944449139 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f45b40392a0 0x7f45b403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f45b00098c0 tx=0x7f45b0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:58.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.803+0000 7f45bffff640 1 -- 172.21.15.103:0/2944449139 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f45b804c350 con 0x7f45c8106590 2024-09-19T06:40:58.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:40:58.972+0000 7f45cd0cc640 1 -- 172.21.15.103:0/2944449139 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}) -- 0x7f45c81069f0 con 0x7f45b40392a0 2024-09-19T06:41:01.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:00 smithi103 ceph-mon[31158]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:01.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:00 smithi103 ceph-mon[31158]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:01.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:01.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:01.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:01.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:01.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:01.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:01.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:02.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:01 smithi103 ceph-mon[31158]: Deploying cephadm binary to smithi174 2024-09-19T06:41:02.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:01 smithi103 ceph-mon[31158]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-19T06:41:02.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:01 smithi103 ceph-mon[31158]: Updating smithi134:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:02.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:01 smithi103 ceph-mon[31158]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:41:02.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:01 smithi103 ceph-mon[31158]: Updating smithi134:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:41:02.141 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:01 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:02.141 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:01 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:02.141 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:01 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:02.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:02.215+0000 7f45bffff640 1 -- 172.21.15.103:0/2944449139 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f45c81069f0 con 0x7f45b40392a0 2024-09-19T06:41:02.217 INFO:teuthology.orchestra.run.smithi103.stdout:Added host 'smithi174' with addr '172.21.15.174' 2024-09-19T06:41:02.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:02.217+0000 7f45cd0cc640 1 -- 172.21.15.103:0/2944449139 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f45b40392a0 msgr2=0x7f45b403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:02.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:02.217+0000 7f45cd0cc640 1 --2- 172.21.15.103:0/2944449139 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f45b40392a0 0x7f45b403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f45b00098c0 tx=0x7f45b0006db0 comp rx=0 tx=0).stop 2024-09-19T06:41:02.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:02.217+0000 7f45cd0cc640 1 -- 172.21.15.103:0/2944449139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c8106590 msgr2=0x7f45c8197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:02.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:02.217+0000 7f45cd0cc640 1 --2- 172.21.15.103:0/2944449139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c8106590 0x7f45c8197f60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009230 tx=0x7f45b8003d40 comp rx=0 tx=0).stop 2024-09-19T06:41:02.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:02.217+0000 7f45cd0cc640 1 -- 172.21.15.103:0/2944449139 shutdown_connections 2024-09-19T06:41:02.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:02.217+0000 7f45cd0cc640 1 --2- 172.21.15.103:0/2944449139 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f45b40392a0 0x7f45b403b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:02.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:02.217+0000 7f45cd0cc640 1 --2- 172.21.15.103:0/2944449139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c8106590 0x7f45c8197f60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:02.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:02.217+0000 7f45cd0cc640 1 -- 172.21.15.103:0/2944449139 >> 172.21.15.103:0/2944449139 conn(0x7f45c80fc1a0 msgr2=0x7f45c80fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:02.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:02.218+0000 7f45cd0cc640 1 -- 172.21.15.103:0/2944449139 shutdown_connections 2024-09-19T06:41:02.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:02.218+0000 7f45cd0cc640 1 -- 172.21.15.103:0/2944449139 wait complete. 2024-09-19T06:41:02.770 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:02 smithi103 ceph-mon[31158]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:02.770 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:02 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:02.770 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:02 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:02.770 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:02 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:02.798 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T06:41:03.059 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:03.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.502+0000 7fbf60019640 1 -- 172.21.15.103:0/1199548701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf58106590 msgr2=0x7fbf58106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:03.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.502+0000 7fbf60019640 1 --2- 172.21.15.103:0/1199548701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf58106590 0x7fbf58106970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009880 tx=0x7fbf5402a5d0 comp rx=0 tx=0).stop 2024-09-19T06:41:03.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.502+0000 7fbf60019640 1 -- 172.21.15.103:0/1199548701 shutdown_connections 2024-09-19T06:41:03.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.502+0000 7fbf60019640 1 --2- 172.21.15.103:0/1199548701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf58106590 0x7fbf58106970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:03.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.502+0000 7fbf60019640 1 -- 172.21.15.103:0/1199548701 >> 172.21.15.103:0/1199548701 conn(0x7fbf580fc1a0 msgr2=0x7fbf580fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:03.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.502+0000 7fbf60019640 1 -- 172.21.15.103:0/1199548701 shutdown_connections 2024-09-19T06:41:03.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.502+0000 7fbf60019640 1 -- 172.21.15.103:0/1199548701 wait complete. 2024-09-19T06:41:03.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.502+0000 7fbf60019640 1 Processor -- start 2024-09-19T06:41:03.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.502+0000 7fbf60019640 1 -- start start 2024-09-19T06:41:03.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.503+0000 7fbf60019640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf58106590 0x7fbf58197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:03.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.503+0000 7fbf60019640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5810abc0 con 0x7fbf58106590 2024-09-19T06:41:03.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.503+0000 7fbf5ddb5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf58106590 0x7fbf58197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:03.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.503+0000 7fbf5ddb5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf58106590 0x7fbf58197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45208/0 (socket says 172.21.15.103:45208) 2024-09-19T06:41:03.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.503+0000 7fbf5ddb5640 1 -- 172.21.15.103:0/1530482787 learned_addr learned my addr 172.21.15.103:0/1530482787 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:41:03.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.503+0000 7fbf5ddb5640 1 -- 172.21.15.103:0/1530482787 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf58197660 con 0x7fbf58106590 2024-09-19T06:41:03.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.503+0000 7fbf5ddb5640 1 --2- 172.21.15.103:0/1530482787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf58106590 0x7fbf58197120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009230 tx=0x7fbf54003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:03.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.503+0000 7fbf4affd640 1 -- 172.21.15.103:0/1530482787 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf5403f070 con 0x7fbf58106590 2024-09-19T06:41:03.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.503+0000 7fbf4affd640 1 -- 172.21.15.103:0/1530482787 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf5403bc80 con 0x7fbf58106590 2024-09-19T06:41:03.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.503+0000 7fbf60019640 1 -- 172.21.15.103:0/1530482787 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf581978f0 con 0x7fbf58106590 2024-09-19T06:41:03.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.503+0000 7fbf4affd640 1 -- 172.21.15.103:0/1530482787 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf54035030 con 0x7fbf58106590 2024-09-19T06:41:03.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.503+0000 7fbf60019640 1 -- 172.21.15.103:0/1530482787 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5819b9f0 con 0x7fbf58106590 2024-09-19T06:41:03.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.504+0000 7fbf4affd640 1 -- 172.21.15.103:0/1530482787 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fbf54042070 con 0x7fbf58106590 2024-09-19T06:41:03.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.504+0000 7fbf4affd640 1 --2- 172.21.15.103:0/1530482787 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbf440392a0 0x7fbf4403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:03.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.504+0000 7fbf4affd640 1 -- 172.21.15.103:0/1530482787 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbf5406cfa0 con 0x7fbf58106590 2024-09-19T06:41:03.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.504+0000 7fbf60019640 1 -- 172.21.15.103:0/1530482787 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf58101c00 con 0x7fbf58106590 2024-09-19T06:41:03.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.504+0000 7fbf5d5b4640 1 --2- 172.21.15.103:0/1530482787 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbf440392a0 0x7fbf4403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:03.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.505+0000 7fbf5d5b4640 1 --2- 172.21.15.103:0/1530482787 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbf440392a0 0x7fbf4403b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0098c0 tx=0x7fbf4c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:03.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.509+0000 7fbf4affd640 1 -- 172.21.15.103:0/1530482787 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbf5404c350 con 0x7fbf58106590 2024-09-19T06:41:03.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.686+0000 7fbf60019640 1 -- 172.21.15.103:0/1530482787 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf581069f0 con 0x7fbf440392a0 2024-09-19T06:41:03.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.687+0000 7fbf4affd640 1 -- 172.21.15.103:0/1530482787 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7fbf581069f0 con 0x7fbf440392a0 2024-09-19T06:41:03.688 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:41:03.689 INFO:teuthology.orchestra.run.smithi103.stdout:[{"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}, {"addr": "172.21.15.174", "hostname": "smithi174", "labels": [], "status": ""}] 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.689+0000 7fbf60019640 1 -- 172.21.15.103:0/1530482787 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbf440392a0 msgr2=0x7fbf4403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.689+0000 7fbf60019640 1 --2- 172.21.15.103:0/1530482787 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbf440392a0 0x7fbf4403b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0098c0 tx=0x7fbf4c006db0 comp rx=0 tx=0).stop 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.689+0000 7fbf60019640 1 -- 172.21.15.103:0/1530482787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf58106590 msgr2=0x7fbf58197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.689+0000 7fbf60019640 1 --2- 172.21.15.103:0/1530482787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf58106590 0x7fbf58197120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009230 tx=0x7fbf54003d40 comp rx=0 tx=0).stop 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.689+0000 7fbf60019640 1 -- 172.21.15.103:0/1530482787 shutdown_connections 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.689+0000 7fbf60019640 1 --2- 172.21.15.103:0/1530482787 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbf440392a0 0x7fbf4403b760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.689+0000 7fbf60019640 1 --2- 172.21.15.103:0/1530482787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf58106590 0x7fbf58197120 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.689+0000 7fbf60019640 1 -- 172.21.15.103:0/1530482787 >> 172.21.15.103:0/1530482787 conn(0x7fbf580fc1a0 msgr2=0x7fbf58069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.690+0000 7fbf60019640 1 -- 172.21.15.103:0/1530482787 shutdown_connections 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:03.690+0000 7fbf60019640 1 -- 172.21.15.103:0/1530482787 wait complete. 2024-09-19T06:41:04.034 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:03 smithi103 ceph-mon[31158]: Added host smithi174 2024-09-19T06:41:04.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:03 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:04.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:03 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:04.262 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T06:41:04.263 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T06:41:04.500 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:04.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:04 smithi103 ceph-mon[31158]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:41:04.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:04 smithi103 ceph-mon[31158]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:05.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.002+0000 7fa6ccc09640 1 -- 172.21.15.103:0/420234472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6c8107660 msgr2=0x7fa6c8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:05.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.002+0000 7fa6ccc09640 1 --2- 172.21.15.103:0/420234472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6c8107660 0x7fa6c8107a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009880 tx=0x7fa6b802a5d0 comp rx=0 tx=0).stop 2024-09-19T06:41:05.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.002+0000 7fa6ccc09640 1 -- 172.21.15.103:0/420234472 shutdown_connections 2024-09-19T06:41:05.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.002+0000 7fa6ccc09640 1 --2- 172.21.15.103:0/420234472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6c8107660 0x7fa6c8107a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:05.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.002+0000 7fa6ccc09640 1 -- 172.21.15.103:0/420234472 >> 172.21.15.103:0/420234472 conn(0x7fa6c80fd2b0 msgr2=0x7fa6c80ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:05.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.002+0000 7fa6ccc09640 1 -- 172.21.15.103:0/420234472 shutdown_connections 2024-09-19T06:41:05.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.002+0000 7fa6ccc09640 1 -- 172.21.15.103:0/420234472 wait complete. 2024-09-19T06:41:05.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.003+0000 7fa6ccc09640 1 Processor -- start 2024-09-19T06:41:05.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.003+0000 7fa6ccc09640 1 -- start start 2024-09-19T06:41:05.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.003+0000 7fa6ccc09640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6c8107660 0x7fa6c81970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:05.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.003+0000 7fa6ccc09640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c810bc70 con 0x7fa6c8107660 2024-09-19T06:41:05.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.003+0000 7fa6c659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6c8107660 0x7fa6c81970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:05.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.003+0000 7fa6c659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6c8107660 0x7fa6c81970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45232/0 (socket says 172.21.15.103:45232) 2024-09-19T06:41:05.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.003+0000 7fa6c659c640 1 -- 172.21.15.103:0/2111901819 learned_addr learned my addr 172.21.15.103:0/2111901819 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:41:05.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.004+0000 7fa6c659c640 1 -- 172.21.15.103:0/2111901819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c81975e0 con 0x7fa6c8107660 2024-09-19T06:41:05.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.004+0000 7fa6c659c640 1 --2- 172.21.15.103:0/2111901819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6c8107660 0x7fa6c81970a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009230 tx=0x7fa6b8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:05.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.004+0000 7fa6b77fe640 1 -- 172.21.15.103:0/2111901819 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6b803f070 con 0x7fa6c8107660 2024-09-19T06:41:05.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.004+0000 7fa6b77fe640 1 -- 172.21.15.103:0/2111901819 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6b803bc80 con 0x7fa6c8107660 2024-09-19T06:41:05.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.004+0000 7fa6b77fe640 1 -- 172.21.15.103:0/2111901819 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6b8035030 con 0x7fa6c8107660 2024-09-19T06:41:05.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.004+0000 7fa6ccc09640 1 -- 172.21.15.103:0/2111901819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c81978d0 con 0x7fa6c8107660 2024-09-19T06:41:05.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.004+0000 7fa6ccc09640 1 -- 172.21.15.103:0/2111901819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c819b970 con 0x7fa6c8107660 2024-09-19T06:41:05.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.005+0000 7fa6b77fe640 1 -- 172.21.15.103:0/2111901819 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fa6b8042070 con 0x7fa6c8107660 2024-09-19T06:41:05.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.005+0000 7fa6b77fe640 1 --2- 172.21.15.103:0/2111901819 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa6b00392a0 0x7fa6b003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:05.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.005+0000 7fa6ccc09640 1 -- 172.21.15.103:0/2111901819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c8102d10 con 0x7fa6c8107660 2024-09-19T06:41:05.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.005+0000 7fa6b77fe640 1 -- 172.21.15.103:0/2111901819 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa6b806d070 con 0x7fa6c8107660 2024-09-19T06:41:05.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.005+0000 7fa6c5d9b640 1 --2- 172.21.15.103:0/2111901819 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa6b00392a0 0x7fa6b003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:05.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.006+0000 7fa6c5d9b640 1 --2- 172.21.15.103:0/2111901819 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa6b00392a0 0x7fa6b003b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0098c0 tx=0x7fa6bc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:05.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.010+0000 7fa6b77fe640 1 -- 172.21.15.103:0/2111901819 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa6b80372d0 con 0x7fa6c8107660 2024-09-19T06:41:05.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.179+0000 7fa6ccc09640 1 -- 172.21.15.103:0/2111901819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fa6c8198400 con 0x7fa6c8107660 2024-09-19T06:41:05.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.784+0000 7fa6b77fe640 1 -- 172.21.15.103:0/2111901819 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fa6b8031e20 con 0x7fa6c8107660 2024-09-19T06:41:05.786 INFO:teuthology.orchestra.run.smithi103.stderr:adjusted tunables profile to default 2024-09-19T06:41:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.787+0000 7fa6ccc09640 1 -- 172.21.15.103:0/2111901819 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa6b00392a0 msgr2=0x7fa6b003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.787+0000 7fa6ccc09640 1 --2- 172.21.15.103:0/2111901819 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa6b00392a0 0x7fa6b003b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0098c0 tx=0x7fa6bc006db0 comp rx=0 tx=0).stop 2024-09-19T06:41:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.787+0000 7fa6ccc09640 1 -- 172.21.15.103:0/2111901819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6c8107660 msgr2=0x7fa6c81970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.787+0000 7fa6ccc09640 1 --2- 172.21.15.103:0/2111901819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6c8107660 0x7fa6c81970a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009230 tx=0x7fa6b8003d40 comp rx=0 tx=0).stop 2024-09-19T06:41:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.787+0000 7fa6ccc09640 1 -- 172.21.15.103:0/2111901819 shutdown_connections 2024-09-19T06:41:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.787+0000 7fa6ccc09640 1 --2- 172.21.15.103:0/2111901819 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa6b00392a0 0x7fa6b003b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:05.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.787+0000 7fa6ccc09640 1 --2- 172.21.15.103:0/2111901819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6c8107660 0x7fa6c81970a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:05.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.787+0000 7fa6ccc09640 1 -- 172.21.15.103:0/2111901819 >> 172.21.15.103:0/2111901819 conn(0x7fa6c80fd2b0 msgr2=0x7fa6c80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:05.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.787+0000 7fa6ccc09640 1 -- 172.21.15.103:0/2111901819 shutdown_connections 2024-09-19T06:41:05.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:05.787+0000 7fa6ccc09640 1 -- 172.21.15.103:0/2111901819 wait complete. 2024-09-19T06:41:06.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:05 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2111901819' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T06:41:06.376 INFO:tasks.cephadm:Adding mon.a on smithi103 2024-09-19T06:41:06.376 INFO:tasks.cephadm:Adding mon.b on smithi134 2024-09-19T06:41:06.376 INFO:tasks.cephadm:Adding mon.c on smithi174 2024-09-19T06:41:06.377 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi103:172.21.15.103=a;smithi134:172.21.15.134=b;smithi174:172.21.15.174=c' 2024-09-19T06:41:06.619 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:41:06.664 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:41:07.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:06 smithi103 ceph-mon[31158]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:07.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:06 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2111901819' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T06:41:07.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:06 smithi103 ceph-mon[31158]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:41:07.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:06 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:07.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:06 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:07.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:06 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:07.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:06 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:07.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:06 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:07.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:06 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:07.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:06 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:07.149 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.147+0000 7f2e64683640 1 -- 172.21.15.174:0/1422167949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c101620 msgr2=0x7f2e5c101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:07.149 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.147+0000 7f2e64683640 1 --2- 172.21.15.174:0/1422167949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c101620 0x7f2e5c101a20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009880 tx=0x7f2e5802a5d0 comp rx=0 tx=0).stop 2024-09-19T06:41:07.149 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.147+0000 7f2e64683640 1 -- 172.21.15.174:0/1422167949 shutdown_connections 2024-09-19T06:41:07.149 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.147+0000 7f2e64683640 1 --2- 172.21.15.174:0/1422167949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c101620 0x7f2e5c101a20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:07.149 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.147+0000 7f2e64683640 1 -- 172.21.15.174:0/1422167949 >> 172.21.15.174:0/1422167949 conn(0x7f2e5c0fcdb0 msgr2=0x7f2e5c0ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:07.149 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.147+0000 7f2e64683640 1 -- 172.21.15.174:0/1422167949 shutdown_connections 2024-09-19T06:41:07.149 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.147+0000 7f2e64683640 1 -- 172.21.15.174:0/1422167949 wait complete. 2024-09-19T06:41:07.149 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.147+0000 7f2e64683640 1 Processor -- start 2024-09-19T06:41:07.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.148+0000 7f2e64683640 1 -- start start 2024-09-19T06:41:07.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.148+0000 7f2e64683640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c101620 0x7f2e5c196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:07.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.148+0000 7f2e64683640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c107730 con 0x7f2e5c101620 2024-09-19T06:41:07.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.148+0000 7f2e6241f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c101620 0x7f2e5c196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:07.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.148+0000 7f2e6241f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c101620 0x7f2e5c196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.174:45510/0 (socket says 172.21.15.174:45510) 2024-09-19T06:41:07.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.148+0000 7f2e6241f640 1 -- 172.21.15.174:0/2422914206 learned_addr learned my addr 172.21.15.174:0/2422914206 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T06:41:07.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.148+0000 7f2e6241f640 1 -- 172.21.15.174:0/2422914206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c197270 con 0x7f2e5c101620 2024-09-19T06:41:07.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.148+0000 7f2e6241f640 1 --2- 172.21.15.174:0/2422914206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c101620 0x7f2e5c196d30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2e58004f60 tx=0x7f2e580048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:07.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.149+0000 7f2e537fe640 1 -- 172.21.15.174:0/2422914206 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e5803f070 con 0x7f2e5c101620 2024-09-19T06:41:07.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.149+0000 7f2e537fe640 1 -- 172.21.15.174:0/2422914206 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e58004cc0 con 0x7f2e5c101620 2024-09-19T06:41:07.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.149+0000 7f2e64683640 1 -- 172.21.15.174:0/2422914206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5c197500 con 0x7f2e5c101620 2024-09-19T06:41:07.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.149+0000 7f2e537fe640 1 -- 172.21.15.174:0/2422914206 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e58035030 con 0x7f2e5c101620 2024-09-19T06:41:07.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.149+0000 7f2e64683640 1 -- 172.21.15.174:0/2422914206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e5c19a1f0 con 0x7f2e5c101620 2024-09-19T06:41:07.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.150+0000 7f2e537fe640 1 -- 172.21.15.174:0/2422914206 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f2e58042070 con 0x7f2e5c101620 2024-09-19T06:41:07.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.150+0000 7f2e64683640 1 -- 172.21.15.174:0/2422914206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e5c107eb0 con 0x7f2e5c101620 2024-09-19T06:41:07.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.150+0000 7f2e537fe640 1 --2- 172.21.15.174:0/2422914206 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2e48039300 0x7f2e4803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:07.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.150+0000 7f2e537fe640 1 -- 172.21.15.174:0/2422914206 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2e5806d0d0 con 0x7f2e5c101620 2024-09-19T06:41:07.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.150+0000 7f2e61c1e640 1 --2- 172.21.15.174:0/2422914206 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2e48039300 0x7f2e4803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:07.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.151+0000 7f2e61c1e640 1 --2- 172.21.15.174:0/2422914206 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2e48039300 0x7f2e4803b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0098c0 tx=0x7f2e4c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:07.156 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.155+0000 7f2e537fe640 1 -- 172.21.15.174:0/2422914206 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2e58040260 con 0x7f2e5c101620 2024-09-19T06:41:07.341 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.340+0000 7f2e64683640 1 -- 172.21.15.174:0/2422914206 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi134:172.21.15.134=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}) -- 0x7f2e5c02d8f0 con 0x7f2e48039300 2024-09-19T06:41:07.346 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.345+0000 7f2e537fe640 1 -- 172.21.15.174:0/2422914206 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2e5c02d8f0 con 0x7f2e48039300 2024-09-19T06:41:07.346 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mon update... 2024-09-19T06:41:07.348 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.347+0000 7f2e64683640 1 -- 172.21.15.174:0/2422914206 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2e48039300 msgr2=0x7f2e4803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:07.348 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.347+0000 7f2e64683640 1 --2- 172.21.15.174:0/2422914206 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2e48039300 0x7f2e4803b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0098c0 tx=0x7f2e4c006db0 comp rx=0 tx=0).stop 2024-09-19T06:41:07.348 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.347+0000 7f2e64683640 1 -- 172.21.15.174:0/2422914206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c101620 msgr2=0x7f2e5c196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:07.348 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.347+0000 7f2e64683640 1 --2- 172.21.15.174:0/2422914206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c101620 0x7f2e5c196d30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2e58004f60 tx=0x7f2e580048f0 comp rx=0 tx=0).stop 2024-09-19T06:41:07.348 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.347+0000 7f2e64683640 1 -- 172.21.15.174:0/2422914206 shutdown_connections 2024-09-19T06:41:07.348 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.347+0000 7f2e64683640 1 --2- 172.21.15.174:0/2422914206 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2e48039300 0x7f2e4803b7c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:07.348 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.347+0000 7f2e64683640 1 --2- 172.21.15.174:0/2422914206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c101620 0x7f2e5c196d30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:07.349 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.347+0000 7f2e64683640 1 -- 172.21.15.174:0/2422914206 >> 172.21.15.174:0/2422914206 conn(0x7f2e5c0fcdb0 msgr2=0x7f2e5c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:07.349 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.347+0000 7f2e64683640 1 -- 172.21.15.174:0/2422914206 shutdown_connections 2024-09-19T06:41:07.349 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:07.347+0000 7f2e64683640 1 -- 172.21.15.174:0/2422914206 wait complete. 2024-09-19T06:41:07.952 DEBUG:teuthology.orchestra.run.smithi134:mon.b> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T06:41:07.955 DEBUG:teuthology.orchestra.run.smithi174:mon.c> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T06:41:07.958 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T06:41:07.958 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:41:08.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:07 smithi103 ceph-mon[31158]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T06:41:08.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:07 smithi103 ceph-mon[31158]: Updating smithi174:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:08.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:07 smithi103 ceph-mon[31158]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:41:08.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:07 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:08.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:07 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:08.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:07 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:08.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:07 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:08.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:07 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:08.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:07 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:08.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:07 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:08.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:07 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:08.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:07 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:41:08.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:07 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:09.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:08 smithi103 ceph-mon[31158]: Updating smithi174:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:41:09.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:08 smithi103 ceph-mon[31158]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi134:172.21.15.134=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:09.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:08 smithi103 ceph-mon[31158]: Saving service mon spec with placement smithi103:172.21.15.103=a;smithi134:172.21.15.134=b;smithi174:172.21.15.174=c;count:3 2024-09-19T06:41:09.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:08 smithi103 ceph-mon[31158]: Deploying daemon mon.c on smithi174 2024-09-19T06:41:09.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:08 smithi103 ceph-mon[31158]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:09.840 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:10.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.466+0000 7fe84c3bd640 1 -- 172.21.15.174:0/3410763627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe844100990 msgr2=0x7fe844100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:10.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.466+0000 7fe84c3bd640 1 --2- 172.21.15.174:0/3410763627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe844100990 0x7fe844100d70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe840009880 tx=0x7fe84002a5d0 comp rx=0 tx=0).stop 2024-09-19T06:41:10.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.467+0000 7fe84c3bd640 1 -- 172.21.15.174:0/3410763627 shutdown_connections 2024-09-19T06:41:10.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.467+0000 7fe84c3bd640 1 --2- 172.21.15.174:0/3410763627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe844100990 0x7fe844100d70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:10.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.467+0000 7fe84c3bd640 1 -- 172.21.15.174:0/3410763627 >> 172.21.15.174:0/3410763627 conn(0x7fe844076250 msgr2=0x7fe844076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:10.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.467+0000 7fe84c3bd640 1 -- 172.21.15.174:0/3410763627 shutdown_connections 2024-09-19T06:41:10.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.467+0000 7fe84c3bd640 1 -- 172.21.15.174:0/3410763627 wait complete. 2024-09-19T06:41:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.467+0000 7fe84c3bd640 1 Processor -- start 2024-09-19T06:41:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.467+0000 7fe84c3bd640 1 -- start start 2024-09-19T06:41:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.467+0000 7fe84c3bd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe844100990 0x7fe8441992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.467+0000 7fe84c3bd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe84410ab90 con 0x7fe844100990 2024-09-19T06:41:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.468+0000 7fe84a159640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe844100990 0x7fe8441992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.468+0000 7fe84a159640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe844100990 0x7fe8441992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.174:45534/0 (socket says 172.21.15.174:45534) 2024-09-19T06:41:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.468+0000 7fe84a159640 1 -- 172.21.15.174:0/3780779159 learned_addr learned my addr 172.21.15.174:0/3780779159 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T06:41:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.468+0000 7fe84a159640 1 -- 172.21.15.174:0/3780779159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe844199820 con 0x7fe844100990 2024-09-19T06:41:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.468+0000 7fe84a159640 1 --2- 172.21.15.174:0/3780779159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe844100990 0x7fe8441992e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe840009230 tx=0x7fe840003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.468+0000 7fe83b7fe640 1 -- 172.21.15.174:0/3780779159 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe84003f070 con 0x7fe844100990 2024-09-19T06:41:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.468+0000 7fe84c3bd640 1 -- 172.21.15.174:0/3780779159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe844199b10 con 0x7fe844100990 2024-09-19T06:41:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.468+0000 7fe83b7fe640 1 -- 172.21.15.174:0/3780779159 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe84003bc80 con 0x7fe844100990 2024-09-19T06:41:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.469+0000 7fe83b7fe640 1 -- 172.21.15.174:0/3780779159 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe840035030 con 0x7fe844100990 2024-09-19T06:41:10.501 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.469+0000 7fe84c3bd640 1 -- 172.21.15.174:0/3780779159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe84419dbb0 con 0x7fe844100990 2024-09-19T06:41:10.501 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.470+0000 7fe83b7fe640 1 -- 172.21.15.174:0/3780779159 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fe840042070 con 0x7fe844100990 2024-09-19T06:41:10.501 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.470+0000 7fe84c3bd640 1 -- 172.21.15.174:0/3780779159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe844101c40 con 0x7fe844100990 2024-09-19T06:41:10.501 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.470+0000 7fe83b7fe640 1 --2- 172.21.15.174:0/3780779159 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe8300392a0 0x7fe83003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:10.501 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.470+0000 7fe83b7fe640 1 -- 172.21.15.174:0/3780779159 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe84006cf80 con 0x7fe844100990 2024-09-19T06:41:10.501 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.470+0000 7fe849958640 1 --2- 172.21.15.174:0/3780779159 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe8300392a0 0x7fe83003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:10.501 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.471+0000 7fe849958640 1 --2- 172.21.15.174:0/3780779159 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe8300392a0 0x7fe83003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe8340098c0 tx=0x7fe834006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:10.501 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.475+0000 7fe83b7fe640 1 -- 172.21.15.174:0/3780779159 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe84004c350 con 0x7fe844100990 2024-09-19T06:41:10.733 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.731+0000 7fe84c3bd640 1 -- 172.21.15.174:0/3780779159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe84419a640 con 0x7fe844100990 2024-09-19T06:41:10.733 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.731+0000 7fe83b7fe640 1 -- 172.21.15.174:0/3780779159 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fe8400314c0 con 0x7fe844100990 2024-09-19T06:41:10.734 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:41:10.734 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","modified":"2024-09-19T06:39:43.868601Z","created":"2024-09-19T06:39:43.868601Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:41:10.734 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-09-19T06:41:10.736 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.733+0000 7fe84c3bd640 1 -- 172.21.15.174:0/3780779159 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe8300392a0 msgr2=0x7fe83003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:10.736 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.733+0000 7fe84c3bd640 1 --2- 172.21.15.174:0/3780779159 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe8300392a0 0x7fe83003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe8340098c0 tx=0x7fe834006db0 comp rx=0 tx=0).stop 2024-09-19T06:41:10.736 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.733+0000 7fe84c3bd640 1 -- 172.21.15.174:0/3780779159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe844100990 msgr2=0x7fe8441992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:10.736 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.733+0000 7fe84c3bd640 1 --2- 172.21.15.174:0/3780779159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe844100990 0x7fe8441992e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe840009230 tx=0x7fe840003d40 comp rx=0 tx=0).stop 2024-09-19T06:41:10.736 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.734+0000 7fe84c3bd640 1 -- 172.21.15.174:0/3780779159 shutdown_connections 2024-09-19T06:41:10.736 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.734+0000 7fe84c3bd640 1 --2- 172.21.15.174:0/3780779159 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe8300392a0 0x7fe83003b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:10.736 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.734+0000 7fe84c3bd640 1 --2- 172.21.15.174:0/3780779159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe844100990 0x7fe8441992e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:10.737 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.734+0000 7fe84c3bd640 1 -- 172.21.15.174:0/3780779159 >> 172.21.15.174:0/3780779159 conn(0x7fe844076250 msgr2=0x7fe844069fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:10.737 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.734+0000 7fe84c3bd640 1 -- 172.21.15.174:0/3780779159 shutdown_connections 2024-09-19T06:41:10.737 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:10.734+0000 7fe84c3bd640 1 -- 172.21.15.174:0/3780779159 wait complete. 2024-09-19T06:41:11.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:10 smithi174 systemd[1]: Starting Ceph mon.c for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T06:41:11.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:10 smithi103 ceph-mon[31158]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:11.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:10 smithi103 ceph-mon[31158]: from='client.? 172.21.15.174:0/3780779159' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:41:11.625 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 podman[35618]: 2024-09-19 06:41:11.511301415 +0000 UTC m=+0.096919920 container create 20df604859787554ab824e27d3818b846b735254786065b7a83faf110ddb4891 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-c, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T06:41:11.625 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 podman[35618]: 2024-09-19 06:41:11.430218225 +0000 UTC m=+0.015836738 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:41:12.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 podman[35618]: 2024-09-19 06:41:11.71403862 +0000 UTC m=+0.299657122 container init 20df604859787554ab824e27d3818b846b735254786065b7a83faf110ddb4891 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T06:41:12.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 podman[35618]: 2024-09-19 06:41:11.717512265 +0000 UTC m=+0.303130767 container start 20df604859787554ab824e27d3818b846b735254786065b7a83faf110ddb4891 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-c, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T06:41:12.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T06:41:12.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T06:41:12.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: pidfile_write: ignore empty --pid-file 2024-09-19T06:41:12.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: load: jerasure load: lrc 2024-09-19T06:41:12.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: RocksDB version: 7.9.2 2024-09-19T06:41:12.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Git sha 0 2024-09-19T06:41:12.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T06:41:12.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: DB SUMMARY 2024-09-19T06:41:12.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: DB Session ID: FEYUOAEAG3MFAFX67ZX2 2024-09-19T06:41:12.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: CURRENT file: CURRENT 2024-09-19T06:41:12.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T06:41:12.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-19T06:41:12.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-19T06:41:12.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-19T06:41:12.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.error_if_exists: 0 2024-09-19T06:41:12.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.create_if_missing: 0 2024-09-19T06:41:12.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.paranoid_checks: 1 2024-09-19T06:41:12.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T06:41:12.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T06:41:12.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T06:41:12.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.env: 0x55ee941d7940 2024-09-19T06:41:12.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T06:41:12.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.info_log: 0x55ee95ba76e0 2024-09-19T06:41:12.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T06:41:12.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.statistics: (nil) 2024-09-19T06:41:12.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.use_fsync: 0 2024-09-19T06:41:12.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_log_file_size: 0 2024-09-19T06:41:12.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T06:41:12.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T06:41:12.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T06:41:12.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T06:41:12.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.allow_fallocate: 1 2024-09-19T06:41:12.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T06:41:12.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T06:41:12.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.use_direct_reads: 0 2024-09-19T06:41:12.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T06:41:12.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T06:41:12.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.db_log_dir: 2024-09-19T06:41:12.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.wal_dir: 2024-09-19T06:41:12.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T06:41:12.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T06:41:12.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T06:41:12.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T06:41:12.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T06:41:12.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T06:41:12.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T06:41:12.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T06:41:12.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.write_buffer_manager: 0x55ee95bb6dc0 2024-09-19T06:41:12.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T06:41:12.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T06:41:12.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T06:41:12.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T06:41:12.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T06:41:12.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T06:41:12.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T06:41:12.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T06:41:12.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.unordered_write: 0 2024-09-19T06:41:12.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T06:41:12.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T06:41:12.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T06:41:12.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T06:41:12.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.row_cache: None 2024-09-19T06:41:12.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.wal_filter: None 2024-09-19T06:41:12.090 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T06:41:12.090 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T06:41:12.090 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.two_write_queues: 0 2024-09-19T06:41:12.090 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T06:41:12.090 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.wal_compression: 0 2024-09-19T06:41:12.090 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.atomic_flush: 0 2024-09-19T06:41:12.091 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T06:41:12.091 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T06:41:12.091 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T06:41:12.091 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.log_readahead_size: 0 2024-09-19T06:41:12.091 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T06:41:12.091 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T06:41:12.091 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T06:41:12.091 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T06:41:12.091 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T06:41:12.091 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T06:41:12.091 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T06:41:12.091 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_background_jobs: 2 2024-09-19T06:41:12.091 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_background_compactions: -1 2024-09-19T06:41:12.092 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_subcompactions: 1 2024-09-19T06:41:12.092 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T06:41:12.092 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T06:41:12.092 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T06:41:12.092 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T06:41:12.092 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T06:41:12.092 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T06:41:12.092 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T06:41:12.092 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T06:41:12.092 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_open_files: -1 2024-09-19T06:41:12.092 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T06:41:12.092 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T06:41:12.092 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T06:41:12.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T06:41:12.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_background_flushes: -1 2024-09-19T06:41:12.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Compression algorithms supported: 2024-09-19T06:41:12.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: kZSTD supported: 0 2024-09-19T06:41:12.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: kXpressCompression supported: 0 2024-09-19T06:41:12.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: kBZip2Compression supported: 0 2024-09-19T06:41:12.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T06:41:12.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: kLZ4Compression supported: 1 2024-09-19T06:41:12.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: kZlibCompression supported: 1 2024-09-19T06:41:12.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T06:41:12.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: kSnappyCompression supported: 1 2024-09-19T06:41:12.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T06:41:12.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T06:41:12.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-19T06:41:12.094 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T06:41:12.094 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T06:41:12.094 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.merge_operator: 2024-09-19T06:41:12.094 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compaction_filter: None 2024-09-19T06:41:12.094 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T06:41:12.094 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T06:41:12.094 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T06:41:12.094 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T06:41:12.094 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ee95ba7320) 2024-09-19T06:41:12.094 INFO:journalctl@ceph.mon.c.smithi174.stdout: cache_index_and_filter_blocks: 1 2024-09-19T06:41:12.094 INFO:journalctl@ceph.mon.c.smithi174.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T06:41:12.094 INFO:journalctl@ceph.mon.c.smithi174.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T06:41:12.094 INFO:journalctl@ceph.mon.c.smithi174.stdout: pin_top_level_index_and_filter: 1 2024-09-19T06:41:12.095 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_type: 0 2024-09-19T06:41:12.095 INFO:journalctl@ceph.mon.c.smithi174.stdout: data_block_index_type: 0 2024-09-19T06:41:12.095 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_shortening: 1 2024-09-19T06:41:12.095 INFO:journalctl@ceph.mon.c.smithi174.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T06:41:12.095 INFO:journalctl@ceph.mon.c.smithi174.stdout: checksum: 4 2024-09-19T06:41:12.095 INFO:journalctl@ceph.mon.c.smithi174.stdout: no_block_cache: 0 2024-09-19T06:41:12.095 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache: 0x55ee95b9b1f0 2024-09-19T06:41:12.095 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_name: BinnedLRUCache 2024-09-19T06:41:12.095 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_options: 2024-09-19T06:41:12.095 INFO:journalctl@ceph.mon.c.smithi174.stdout: capacity : 536870912 2024-09-19T06:41:12.095 INFO:journalctl@ceph.mon.c.smithi174.stdout: num_shard_bits : 4 2024-09-19T06:41:12.095 INFO:journalctl@ceph.mon.c.smithi174.stdout: strict_capacity_limit : 0 2024-09-19T06:41:12.095 INFO:journalctl@ceph.mon.c.smithi174.stdout: high_pri_pool_ratio: 0.000 2024-09-19T06:41:12.095 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_compressed: (nil) 2024-09-19T06:41:12.096 INFO:journalctl@ceph.mon.c.smithi174.stdout: persistent_cache: (nil) 2024-09-19T06:41:12.096 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_size: 4096 2024-09-19T06:41:12.096 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_size_deviation: 10 2024-09-19T06:41:12.096 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_restart_interval: 16 2024-09-19T06:41:12.096 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_block_restart_interval: 1 2024-09-19T06:41:12.096 INFO:journalctl@ceph.mon.c.smithi174.stdout: metadata_block_size: 4096 2024-09-19T06:41:12.096 INFO:journalctl@ceph.mon.c.smithi174.stdout: partition_filters: 0 2024-09-19T06:41:12.096 INFO:journalctl@ceph.mon.c.smithi174.stdout: use_delta_encoding: 1 2024-09-19T06:41:12.096 INFO:journalctl@ceph.mon.c.smithi174.stdout: filter_policy: bloomfilter 2024-09-19T06:41:12.096 INFO:journalctl@ceph.mon.c.smithi174.stdout: whole_key_filtering: 1 2024-09-19T06:41:12.096 INFO:journalctl@ceph.mon.c.smithi174.stdout: verify_compression: 0 2024-09-19T06:41:12.096 INFO:journalctl@ceph.mon.c.smithi174.stdout: read_amp_bytes_per_bit: 0 2024-09-19T06:41:12.096 INFO:journalctl@ceph.mon.c.smithi174.stdout: format_version: 5 2024-09-19T06:41:12.097 INFO:journalctl@ceph.mon.c.smithi174.stdout: enable_index_compression: 1 2024-09-19T06:41:12.097 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_align: 0 2024-09-19T06:41:12.097 INFO:journalctl@ceph.mon.c.smithi174.stdout: max_auto_readahead_size: 262144 2024-09-19T06:41:12.097 INFO:journalctl@ceph.mon.c.smithi174.stdout: prepopulate_block_cache: 0 2024-09-19T06:41:12.097 INFO:journalctl@ceph.mon.c.smithi174.stdout: initial_auto_readahead_size: 8192 2024-09-19T06:41:12.097 INFO:journalctl@ceph.mon.c.smithi174.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T06:41:12.097 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T06:41:12.097 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T06:41:12.097 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compression: NoCompression 2024-09-19T06:41:12.097 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T06:41:12.097 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T06:41:12.097 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T06:41:12.097 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.num_levels: 7 2024-09-19T06:41:12.097 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T06:41:12.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T06:41:12.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T06:41:12.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T06:41:12.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T06:41:12.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T06:41:12.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T06:41:12.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T06:41:12.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T06:41:12.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T06:41:12.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T06:41:12.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T06:41:12.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T06:41:12.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T06:41:12.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T06:41:12.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T06:41:12.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T06:41:12.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T06:41:12.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T06:41:12.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T06:41:12.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T06:41:12.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T06:41:12.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T06:41:12.100 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T06:41:12.100 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T06:41:12.100 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T06:41:12.100 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T06:41:12.100 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T06:41:12.100 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T06:41:12.100 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T06:41:12.100 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T06:41:12.100 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T06:41:12.100 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T06:41:12.100 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T06:41:12.100 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T06:41:12.100 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T06:41:12.100 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T06:41:12.101 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T06:41:12.101 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T06:41:12.101 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T06:41:12.101 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T06:41:12.101 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T06:41:12.101 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T06:41:12.101 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T06:41:12.101 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T06:41:12.101 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T06:41:12.101 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T06:41:12.101 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T06:41:12.101 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T06:41:12.101 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T06:41:12.102 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T06:41:12.102 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T06:41:12.102 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T06:41:12.102 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T06:41:12.102 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.inplace_update_support: 0 2024-09-19T06:41:12.102 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T06:41:12.102 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T06:41:12.102 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T06:41:12.102 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T06:41:12.102 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.bloom_locality: 0 2024-09-19T06:41:12.102 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.max_successive_merges: 0 2024-09-19T06:41:12.102 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T06:41:12.102 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T06:41:12.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T06:41:12.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T06:41:12.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.ttl: 2592000 2024-09-19T06:41:12.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T06:41:12.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T06:41:12.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T06:41:12.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.enable_blob_files: false 2024-09-19T06:41:12.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.min_blob_size: 0 2024-09-19T06:41:12.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T06:41:12.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T06:41:12.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T06:41:12.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T06:41:12.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T06:41:12.104 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T06:41:12.104 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T06:41:12.104 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T06:41:12.104 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-19T06:41:12.104 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-19T06:41:12.104 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e0a12409-4f47-4016-b297-ddfc200ac805 2024-09-19T06:41:12.104 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726728071744395, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-19T06:41:12.104 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-19T06:41:12.104 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726728071744805, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726728071, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e0a12409-4f47-4016-b297-ddfc200ac805", "db_session_id": "FEYUOAEAG3MFAFX67ZX2", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-19T06:41:12.104 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726728071744899, "job": 1, "event": "recovery_finished"} 2024-09-19T06:41:12.104 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-19T06:41:12.104 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T06:41:12.104 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ee95bc8e00 2024-09-19T06:41:12.105 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: DB pointer 0x55ee95cc4000 2024-09-19T06:41:12.105 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T06:41:12.105 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T06:41:12.105 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** DB Stats ** 2024-09-19T06:41:12.105 INFO:journalctl@ceph.mon.c.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T06:41:12.105 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T06:41:12.105 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T06:41:12.105 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T06:41:12.105 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T06:41:12.105 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T06:41:12.105 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T06:41:12.105 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T06:41:12.105 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** Compaction Stats [default] ** 2024-09-19T06:41:12.105 INFO:journalctl@ceph.mon.c.smithi174.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T06:41:12.106 INFO:journalctl@ceph.mon.c.smithi174.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T06:41:12.106 INFO:journalctl@ceph.mon.c.smithi174.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T06:41:12.106 INFO:journalctl@ceph.mon.c.smithi174.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T06:41:12.106 INFO:journalctl@ceph.mon.c.smithi174.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T06:41:12.106 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T06:41:12.106 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** Compaction Stats [default] ** 2024-09-19T06:41:12.106 INFO:journalctl@ceph.mon.c.smithi174.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T06:41:12.106 INFO:journalctl@ceph.mon.c.smithi174.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T06:41:12.106 INFO:journalctl@ceph.mon.c.smithi174.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T06:41:12.106 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T06:41:12.106 INFO:journalctl@ceph.mon.c.smithi174.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T06:41:12.106 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T06:41:12.106 INFO:journalctl@ceph.mon.c.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T06:41:12.106 INFO:journalctl@ceph.mon.c.smithi174.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T06:41:12.106 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T06:41:12.107 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T06:41:12.107 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T06:41:12.107 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T06:41:12.107 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T06:41:12.107 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T06:41:12.107 INFO:journalctl@ceph.mon.c.smithi174.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T06:41:12.107 INFO:journalctl@ceph.mon.c.smithi174.stdout: Block cache BinnedLRUCache@0x55ee95b9b1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-19T06:41:12.107 INFO:journalctl@ceph.mon.c.smithi174.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-19T06:41:12.107 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T06:41:12.107 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T06:41:12.107 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-19T06:41:12.107 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: using public_addr v2:172.21.15.174:0/0 -> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2024-09-19T06:41:12.107 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: starting mon.c rank -1 at public addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] at bind addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:41:12.107 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: mon.c@-1(???) e0 preinit fsid d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:41:12.108 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: mon.c@-1(synchronizing).mds e1 new map 2024-09-19T06:41:12.108 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-19T06:41:12.108 INFO:journalctl@ceph.mon.c.smithi174.stdout: e1 2024-09-19T06:41:12.108 INFO:journalctl@ceph.mon.c.smithi174.stdout: btime 2024-09-19T06:39:46:840386+0000 2024-09-19T06:41:12.108 INFO:journalctl@ceph.mon.c.smithi174.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T06:41:12.108 INFO:journalctl@ceph.mon.c.smithi174.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T06:41:12.108 INFO:journalctl@ceph.mon.c.smithi174.stdout: legacy client fscid: -1 2024-09-19T06:41:12.108 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T06:41:12.108 INFO:journalctl@ceph.mon.c.smithi174.stdout: No filesystems configured 2024-09-19T06:41:12.108 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T06:41:12.108 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T06:41:12.108 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T06:41:12.108 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T06:41:12.108 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T06:41:12.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T06:41:12.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T06:41:12.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:41:12.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:41:12.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:41:12.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: Added host smithi174 2024-09-19T06:41:12.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:12.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:12.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:41:12.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:12.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2111901819' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T06:41:12.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:12.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2111901819' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T06:41:12.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:41:12.110 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:12.110 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:12.110 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:12.110 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:12.110 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:12.110 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:12.110 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:12.110 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T06:41:12.110 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: Updating smithi174:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:12.110 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:41:12.110 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:12.110 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:12.110 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:12.110 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:12.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:12.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:12.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:12.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:12.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:41:12.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:12.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: Updating smithi174:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:41:12.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi134:172.21.15.134=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:12.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: Saving service mon spec with placement smithi103:172.21.15.103=a;smithi134:172.21.15.134=b;smithi174:172.21.15.174=c;count:3 2024-09-19T06:41:12.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: Deploying daemon mon.c on smithi174 2024-09-19T06:41:12.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:12.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:12.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: from='client.? 172.21.15.174:0/3780779159' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:41:12.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 ceph-mon[35637]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T06:41:12.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 bash[35618]: 20df604859787554ab824e27d3818b846b735254786065b7a83faf110ddb4891 2024-09-19T06:41:12.112 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:11 smithi174 systemd[1]: Started Ceph mon.c for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:41:12.361 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T06:41:12.361 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:41:12.610 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:41:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:41:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: mon.a calling monitor election 2024-09-19T06:41:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: mon.c calling monitor election 2024-09-19T06:41:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T06:41:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: monmap epoch 2 2024-09-19T06:41:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: fsid d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:41:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: last_changed 2024-09-19T06:41:11.769269+0000 2024-09-19T06:41:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: created 2024-09-19T06:39:43.868601+0000 2024-09-19T06:41:17.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: min_mon_release 19 (squid) 2024-09-19T06:41:17.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: election_strategy: 1 2024-09-19T06:41:17.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T06:41:17.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T06:41:17.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: fsmap 2024-09-19T06:41:17.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:41:17.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: mgrmap e13: x(active, since 37s) 2024-09-19T06:41:17.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: overall HEALTH_OK 2024-09-19T06:41:17.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:17.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:17.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:17.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:41:17.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:16 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:17.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:17.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:41:17.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: mon.a calling monitor election 2024-09-19T06:41:17.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:17.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:17.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:17.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:17.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: mon.c calling monitor election 2024-09-19T06:41:17.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:17.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:17.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:17.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:17.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T06:41:17.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: monmap epoch 2 2024-09-19T06:41:17.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: fsid d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:41:17.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: last_changed 2024-09-19T06:41:11.769269+0000 2024-09-19T06:41:17.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: created 2024-09-19T06:39:43.868601+0000 2024-09-19T06:41:17.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: min_mon_release 19 (squid) 2024-09-19T06:41:17.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: election_strategy: 1 2024-09-19T06:41:17.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T06:41:17.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T06:41:17.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: fsmap 2024-09-19T06:41:17.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:41:17.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: mgrmap e13: x(active, since 37s) 2024-09-19T06:41:17.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: overall HEALTH_OK 2024-09-19T06:41:17.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:17.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:17.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:17.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:41:17.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:16 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:17.444 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.441+0000 7fb66e3ba640 1 -- 172.21.15.174:0/855906271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb650003530 msgr2=0x7fb6500059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:17.444 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.441+0000 7fb66e3ba640 1 --2- 172.21.15.174:0/855906271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb650003530 0x7fb6500059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb6681016a0 tx=0x7fb65802c490 comp rx=0 tx=0).stop 2024-09-19T06:41:17.444 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.441+0000 7fb66e3ba640 1 -- 172.21.15.174:0/855906271 shutdown_connections 2024-09-19T06:41:17.444 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.441+0000 7fb66e3ba640 1 --2- 172.21.15.174:0/855906271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb650003530 0x7fb6500059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.444 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.441+0000 7fb66e3ba640 1 -- 172.21.15.174:0/855906271 >> 172.21.15.174:0/855906271 conn(0x7fb6680fc220 msgr2=0x7fb6680fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:17.444 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.441+0000 7fb66e3ba640 1 -- 172.21.15.174:0/855906271 shutdown_connections 2024-09-19T06:41:17.444 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.441+0000 7fb66e3ba640 1 -- 172.21.15.174:0/855906271 wait complete. 2024-09-19T06:41:17.446 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.442+0000 7fb66e3ba640 1 Processor -- start 2024-09-19T06:41:17.446 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.442+0000 7fb66e3ba640 1 -- start start 2024-09-19T06:41:17.446 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.443+0000 7fb66e3ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb650003530 0x7fb668196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.446 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.443+0000 7fb66e3ba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6681966d0 0x7fb66819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.446 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.443+0000 7fb66e3ba640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb668108670 con 0x7fb650003530 2024-09-19T06:41:17.446 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.443+0000 7fb66e3ba640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681084f0 con 0x7fb6681966d0 2024-09-19T06:41:17.446 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.443+0000 7fb6677fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6681966d0 0x7fb66819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.443+0000 7fb6677fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6681966d0 0x7fb66819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:55240/0 (socket says 172.21.15.174:55240) 2024-09-19T06:41:17.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.443+0000 7fb6677fe640 1 -- 172.21.15.174:0/239769141 learned_addr learned my addr 172.21.15.174:0/239769141 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T06:41:17.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.443+0000 7fb667fff640 1 --2- 172.21.15.174:0/239769141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb650003530 0x7fb668196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.443+0000 7fb6677fe640 1 -- 172.21.15.174:0/239769141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6681966d0 msgr2=0x7fb66819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T06:41:17.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.443+0000 7fb6677fe640 1 -- 172.21.15.174:0/239769141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6681966d0 msgr2=0x7fb66819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T06:41:17.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.443+0000 7fb6677fe640 1 --2- 172.21.15.174:0/239769141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6681966d0 0x7fb66819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T06:41:17.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.443+0000 7fb6677fe640 1 --2- 172.21.15.174:0/239769141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6681966d0 0x7fb66819aa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:41:17.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.443+0000 7fb667fff640 1 -- 172.21.15.174:0/239769141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6681966d0 msgr2=0x7fb66819aa60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:41:17.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.443+0000 7fb667fff640 1 --2- 172.21.15.174:0/239769141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6681966d0 0x7fb66819aa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.443+0000 7fb667fff640 1 -- 172.21.15.174:0/239769141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb668196cf0 con 0x7fb650003530 2024-09-19T06:41:17.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.444+0000 7fb667fff640 1 --2- 172.21.15.174:0/239769141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb650003530 0x7fb668196190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb658005f30 tx=0x7fb658033680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:17.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.444+0000 7fb6657fa640 1 -- 172.21.15.174:0/239769141 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb658038040 con 0x7fb650003530 2024-09-19T06:41:17.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.444+0000 7fb6657fa640 1 -- 172.21.15.174:0/239769141 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb658032e00 con 0x7fb650003530 2024-09-19T06:41:17.448 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.444+0000 7fb66e3ba640 1 -- 172.21.15.174:0/239769141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb66819afa0 con 0x7fb650003530 2024-09-19T06:41:17.448 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.444+0000 7fb6657fa640 1 -- 172.21.15.174:0/239769141 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb658004a00 con 0x7fb650003530 2024-09-19T06:41:17.448 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.444+0000 7fb66e3ba640 1 -- 172.21.15.174:0/239769141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb66819b560 con 0x7fb650003530 2024-09-19T06:41:17.448 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.445+0000 7fb6657fa640 1 -- 172.21.15.174:0/239769141 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fb658043020 con 0x7fb650003530 2024-09-19T06:41:17.449 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.445+0000 7fb6657fa640 1 --2- 172.21.15.174:0/239769141 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb654039090 0x7fb65403b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.449 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.445+0000 7fb66e3ba640 1 -- 172.21.15.174:0/239769141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb668101ca0 con 0x7fb650003530 2024-09-19T06:41:17.449 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.445+0000 7fb6657fa640 1 -- 172.21.15.174:0/239769141 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb65806e8a0 con 0x7fb650003530 2024-09-19T06:41:17.449 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.446+0000 7fb6677fe640 1 --2- 172.21.15.174:0/239769141 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb654039090 0x7fb65403b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.449 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.446+0000 7fb6677fe640 1 --2- 172.21.15.174:0/239769141 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb654039090 0x7fb65403b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb65c006a90 tx=0x7fb65c006a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:17.453 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.451+0000 7fb6657fa640 1 -- 172.21.15.174:0/239769141 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb65804a020 con 0x7fb650003530 2024-09-19T06:41:17.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.684+0000 7fb66e3ba640 1 -- 172.21.15.174:0/239769141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb66819b810 con 0x7fb650003530 2024-09-19T06:41:17.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.685+0000 7fb6657fa640 1 -- 172.21.15.174:0/239769141 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fb65804cbc0 con 0x7fb650003530 2024-09-19T06:41:17.687 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:41:17.687 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":2,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","modified":"2024-09-19T06:41:11.769269Z","created":"2024-09-19T06:39:43.868601Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T06:41:17.687 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 2 2024-09-19T06:41:17.689 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.687+0000 7fb66e3ba640 1 -- 172.21.15.174:0/239769141 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb654039090 msgr2=0x7fb65403b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:17.689 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.687+0000 7fb66e3ba640 1 --2- 172.21.15.174:0/239769141 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb654039090 0x7fb65403b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb65c006a90 tx=0x7fb65c006a20 comp rx=0 tx=0).stop 2024-09-19T06:41:17.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.687+0000 7fb66e3ba640 1 -- 172.21.15.174:0/239769141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb650003530 msgr2=0x7fb668196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:17.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.687+0000 7fb66e3ba640 1 --2- 172.21.15.174:0/239769141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb650003530 0x7fb668196190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb658005f30 tx=0x7fb658033680 comp rx=0 tx=0).stop 2024-09-19T06:41:17.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.687+0000 7fb66e3ba640 1 -- 172.21.15.174:0/239769141 shutdown_connections 2024-09-19T06:41:17.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.687+0000 7fb66e3ba640 1 --2- 172.21.15.174:0/239769141 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb654039090 0x7fb65403b550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.687+0000 7fb66e3ba640 1 --2- 172.21.15.174:0/239769141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6681966d0 0x7fb66819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.687+0000 7fb66e3ba640 1 --2- 172.21.15.174:0/239769141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb650003530 0x7fb668196190 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.687+0000 7fb66e3ba640 1 -- 172.21.15.174:0/239769141 >> 172.21.15.174:0/239769141 conn(0x7fb6680fc220 msgr2=0x7fb6680fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:17.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.687+0000 7fb66e3ba640 1 -- 172.21.15.174:0/239769141 shutdown_connections 2024-09-19T06:41:17.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:17.687+0000 7fb66e3ba640 1 -- 172.21.15.174:0/239769141 wait complete. 2024-09-19T06:41:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:17 smithi174 ceph-mon[35637]: Deploying daemon mon.b on smithi134 2024-09-19T06:41:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:17 smithi174 ceph-mon[35637]: from='client.? 172.21.15.174:0/239769141' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:41:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:17 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:17 smithi103 ceph-mon[31158]: Deploying daemon mon.b on smithi134 2024-09-19T06:41:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:17 smithi103 ceph-mon[31158]: from='client.? 172.21.15.174:0/239769141' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:41:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:17 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:19.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:18 smithi103 ceph-mon[31158]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:19.138 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:41:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:41:18.769+0000 7f71b1748640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-19T06:41:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:18 smithi174 ceph-mon[35637]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:19.350 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T06:41:19.350 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:41:19.601 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:41:20.025 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.017+0000 7f9b67162640 1 -- 172.21.15.174:0/1122300271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b60106630 msgr2=0x7f9b60106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:20.025 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.017+0000 7f9b67162640 1 --2- 172.21.15.174:0/1122300271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b60106630 0x7f9b60106a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009880 tx=0x7f9b5802a5d0 comp rx=0 tx=0).stop 2024-09-19T06:41:20.025 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.017+0000 7f9b67162640 1 -- 172.21.15.174:0/1122300271 shutdown_connections 2024-09-19T06:41:20.025 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.017+0000 7f9b67162640 1 --2- 172.21.15.174:0/1122300271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b60106630 0x7f9b60106a10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:20.025 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.017+0000 7f9b67162640 1 -- 172.21.15.174:0/1122300271 >> 172.21.15.174:0/1122300271 conn(0x7f9b600fc220 msgr2=0x7f9b600fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:20.026 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.017+0000 7f9b67162640 1 -- 172.21.15.174:0/1122300271 shutdown_connections 2024-09-19T06:41:20.026 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.017+0000 7f9b67162640 1 -- 172.21.15.174:0/1122300271 wait complete. 2024-09-19T06:41:20.026 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.018+0000 7f9b67162640 1 Processor -- start 2024-09-19T06:41:20.026 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.018+0000 7f9b67162640 1 -- start start 2024-09-19T06:41:20.026 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.018+0000 7f9b67162640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b60106630 0x7f9b601962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:20.026 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.018+0000 7f9b67162640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b60196820 0x7f9b6019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:20.027 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.018+0000 7f9b67162640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6010ac60 con 0x7f9b60106630 2024-09-19T06:41:20.027 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.018+0000 7f9b67162640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6010aae0 con 0x7f9b60196820 2024-09-19T06:41:20.027 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.018+0000 7f9b5ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b60196820 0x7f9b6019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:20.027 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.018+0000 7f9b64efe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b60106630 0x7f9b601962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:20.027 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.018+0000 7f9b5ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b60196820 0x7f9b6019abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:55256/0 (socket says 172.21.15.174:55256) 2024-09-19T06:41:20.028 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.018+0000 7f9b5ffff640 1 -- 172.21.15.174:0/2773229659 learned_addr learned my addr 172.21.15.174:0/2773229659 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T06:41:20.028 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.018+0000 7f9b5ffff640 1 -- 172.21.15.174:0/2773229659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b60196820 msgr2=0x7f9b6019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T06:41:20.028 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.018+0000 7f9b5ffff640 1 -- 172.21.15.174:0/2773229659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b60196820 msgr2=0x7f9b6019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T06:41:20.028 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.018+0000 7f9b5ffff640 1 --2- 172.21.15.174:0/2773229659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b60196820 0x7f9b6019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T06:41:20.028 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.018+0000 7f9b5ffff640 1 --2- 172.21.15.174:0/2773229659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b60196820 0x7f9b6019abb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:41:20.028 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.019+0000 7f9b64efe640 1 -- 172.21.15.174:0/2773229659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b60196820 msgr2=0x7f9b6019abb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:41:20.029 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.019+0000 7f9b64efe640 1 --2- 172.21.15.174:0/2773229659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b60196820 0x7f9b6019abb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:20.029 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.019+0000 7f9b64efe640 1 -- 172.21.15.174:0/2773229659 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b60196e40 con 0x7f9b60106630 2024-09-19T06:41:20.029 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.019+0000 7f9b64efe640 1 --2- 172.21.15.174:0/2773229659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b60106630 0x7f9b601962e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9b58005ab0 tx=0x7f9b58032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:20.029 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.019+0000 7f9b5dffb640 1 -- 172.21.15.174:0/2773229659 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9b58035070 con 0x7f9b60106630 2024-09-19T06:41:20.029 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.019+0000 7f9b67162640 1 -- 172.21.15.174:0/2773229659 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6019b0f0 con 0x7f9b60106630 2024-09-19T06:41:20.029 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.019+0000 7f9b5dffb640 1 -- 172.21.15.174:0/2773229659 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b58032e40 con 0x7f9b60106630 2024-09-19T06:41:20.030 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.019+0000 7f9b5dffb640 1 -- 172.21.15.174:0/2773229659 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9b5803bc10 con 0x7f9b60106630 2024-09-19T06:41:20.030 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.019+0000 7f9b67162640 1 -- 172.21.15.174:0/2773229659 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b6019b5d0 con 0x7f9b60106630 2024-09-19T06:41:20.030 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.020+0000 7f9b67162640 1 -- 172.21.15.174:0/2773229659 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b60101ca0 con 0x7f9b60106630 2024-09-19T06:41:20.030 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.020+0000 7f9b5dffb640 1 -- 172.21.15.174:0/2773229659 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f9b58042030 con 0x7f9b60106630 2024-09-19T06:41:20.030 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.021+0000 7f9b5dffb640 1 --2- 172.21.15.174:0/2773229659 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9b4c039040 0x7f9b4c03b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:20.030 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.021+0000 7f9b5dffb640 1 -- 172.21.15.174:0/2773229659 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9b58040070 con 0x7f9b60106630 2024-09-19T06:41:20.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.021+0000 7f9b5ffff640 1 --2- 172.21.15.174:0/2773229659 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9b4c039040 0x7f9b4c03b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:20.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.021+0000 7f9b5ffff640 1 --2- 172.21.15.174:0/2773229659 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9b4c039040 0x7f9b4c03b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009100 tx=0x7f9b48008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:20.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.025+0000 7f9b5dffb640 1 -- 172.21.15.174:0/2773229659 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9b5804a260 con 0x7f9b60106630 2024-09-19T06:41:20.254 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.252+0000 7f9b67162640 1 -- 172.21.15.174:0/2773229659 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9b6019b8b0 con 0x7f9b60106630 2024-09-19T06:41:20.255 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.252+0000 7f9b5dffb640 1 -- 172.21.15.174:0/2773229659 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f9b5803bdb0 con 0x7f9b60106630 2024-09-19T06:41:20.255 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:41:20.255 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":2,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","modified":"2024-09-19T06:41:11.769269Z","created":"2024-09-19T06:39:43.868601Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T06:41:20.255 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 2 2024-09-19T06:41:20.257 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.255+0000 7f9b67162640 1 -- 172.21.15.174:0/2773229659 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9b4c039040 msgr2=0x7f9b4c03b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:20.257 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.255+0000 7f9b67162640 1 --2- 172.21.15.174:0/2773229659 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9b4c039040 0x7f9b4c03b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009100 tx=0x7f9b48008040 comp rx=0 tx=0).stop 2024-09-19T06:41:20.257 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.255+0000 7f9b67162640 1 -- 172.21.15.174:0/2773229659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b60106630 msgr2=0x7f9b601962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:20.257 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.255+0000 7f9b67162640 1 --2- 172.21.15.174:0/2773229659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b60106630 0x7f9b601962e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9b58005ab0 tx=0x7f9b58032c50 comp rx=0 tx=0).stop 2024-09-19T06:41:20.258 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.255+0000 7f9b67162640 1 -- 172.21.15.174:0/2773229659 shutdown_connections 2024-09-19T06:41:20.258 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.255+0000 7f9b67162640 1 --2- 172.21.15.174:0/2773229659 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9b4c039040 0x7f9b4c03b500 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:20.258 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.255+0000 7f9b67162640 1 --2- 172.21.15.174:0/2773229659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b60196820 0x7f9b6019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:20.258 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.255+0000 7f9b67162640 1 --2- 172.21.15.174:0/2773229659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b60106630 0x7f9b601962e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:20.258 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.255+0000 7f9b67162640 1 -- 172.21.15.174:0/2773229659 >> 172.21.15.174:0/2773229659 conn(0x7f9b600fc220 msgr2=0x7f9b600fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:20.258 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.255+0000 7f9b67162640 1 -- 172.21.15.174:0/2773229659 shutdown_connections 2024-09-19T06:41:20.258 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:20.255+0000 7f9b67162640 1 -- 172.21.15.174:0/2773229659 wait complete. 2024-09-19T06:41:20.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:20 smithi134 systemd[1]: Started Ceph mon.b for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:41:21.788 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T06:41:21.789 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:41:22.040 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:41:25.643 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.640+0000 7fde0f589640 1 -- 172.21.15.174:0/1514689854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddf4003530 msgr2=0x7fddf40059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:25.643 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.640+0000 7fde0f589640 1 --2- 172.21.15.174:0/1514689854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddf4003530 0x7fddf40059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fde081016d0 tx=0x7fde0402c150 comp rx=0 tx=0).stop 2024-09-19T06:41:25.643 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.640+0000 7fde0f589640 1 -- 172.21.15.174:0/1514689854 shutdown_connections 2024-09-19T06:41:25.643 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.640+0000 7fde0f589640 1 --2- 172.21.15.174:0/1514689854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddf4003530 0x7fddf40059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.643 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.640+0000 7fde0f589640 1 -- 172.21.15.174:0/1514689854 >> 172.21.15.174:0/1514689854 conn(0x7fde080fc250 msgr2=0x7fde080fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:25.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.641+0000 7fde0f589640 1 -- 172.21.15.174:0/1514689854 shutdown_connections 2024-09-19T06:41:25.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.641+0000 7fde0f589640 1 -- 172.21.15.174:0/1514689854 wait complete. 2024-09-19T06:41:25.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.641+0000 7fde0f589640 1 Processor -- start 2024-09-19T06:41:25.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.642+0000 7fde0f589640 1 -- start start 2024-09-19T06:41:25.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.642+0000 7fde0f589640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddf4003530 0x7fde08197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:25.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.642+0000 7fde0f589640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde08197840 0x7fde0819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:25.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.642+0000 7fde0f589640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde0819c110 0x7fde0819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:25.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.642+0000 7fde0f589640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde081086a0 con 0x7fddf4003530 2024-09-19T06:41:25.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.642+0000 7fde0f589640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde08108520 con 0x7fde0819c110 2024-09-19T06:41:25.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.642+0000 7fde0f589640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde08108820 con 0x7fde08197840 2024-09-19T06:41:25.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.642+0000 7fde0cb24640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde08197840 0x7fde0819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:25.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.642+0000 7fde0cb24640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde08197840 0x7fde0819bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40108/0 (socket says 172.21.15.174:40108) 2024-09-19T06:41:25.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.642+0000 7fde0cb24640 1 -- 172.21.15.174:0/1912123462 learned_addr learned my addr 172.21.15.174:0/1912123462 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T06:41:25.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.642+0000 7fde0d325640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddf4003530 0x7fde08197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:25.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.642+0000 7fde0cb24640 1 -- 172.21.15.174:0/1912123462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde08197840 msgr2=0x7fde0819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T06:41:25.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.642+0000 7fde0cb24640 1 -- 172.21.15.174:0/1912123462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde08197840 msgr2=0x7fde0819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T06:41:25.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.642+0000 7fde0cb24640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde08197840 0x7fde0819bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T06:41:25.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.642+0000 7fde0db26640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde0819c110 0x7fde0819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:25.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.643+0000 7fde0cb24640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde08197840 0x7fde0819bbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:41:25.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.643+0000 7fde0db26640 1 -- 172.21.15.174:0/1912123462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde0819c110 msgr2=0x7fde0819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-19T06:41:25.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.643+0000 7fde0db26640 1 -- 172.21.15.174:0/1912123462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde0819c110 msgr2=0x7fde0819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T06:41:25.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.643+0000 7fde0db26640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde0819c110 0x7fde0819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T06:41:25.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.643+0000 7fde0db26640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde0819c110 0x7fde0819c5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:41:25.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.643+0000 7fde0d325640 1 -- 172.21.15.174:0/1912123462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde08197840 msgr2=0x7fde0819bbd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:41:25.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.643+0000 7fde0d325640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde08197840 0x7fde0819bbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.643+0000 7fde0d325640 1 -- 172.21.15.174:0/1912123462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde0819c110 msgr2=0x7fde0819c5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:41:25.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.643+0000 7fde0d325640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde0819c110 0x7fde0819c5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.648 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.643+0000 7fde0d325640 1 -- 172.21.15.174:0/1912123462 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde081a3e80 con 0x7fddf4003530 2024-09-19T06:41:25.648 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.643+0000 7fde0d325640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddf4003530 0x7fde08197300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fde04036000 tx=0x7fde04004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:25.648 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.643+0000 7fddfe7fc640 1 -- 172.21.15.174:0/1912123462 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04032040 con 0x7fddf4003530 2024-09-19T06:41:25.648 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.643+0000 7fddfe7fc640 1 -- 172.21.15.174:0/1912123462 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde04043070 con 0x7fddf4003530 2024-09-19T06:41:25.648 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.643+0000 7fde0f589640 1 -- 172.21.15.174:0/1912123462 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde081a4050 con 0x7fddf4003530 2024-09-19T06:41:25.648 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.643+0000 7fddfe7fc640 1 -- 172.21.15.174:0/1912123462 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04036670 con 0x7fddf4003530 2024-09-19T06:41:25.648 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.644+0000 7fde0f589640 1 -- 172.21.15.174:0/1912123462 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde081a45b0 con 0x7fddf4003530 2024-09-19T06:41:25.649 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.644+0000 7fddfe7fc640 1 -- 172.21.15.174:0/1912123462 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fde04036810 con 0x7fddf4003530 2024-09-19T06:41:25.649 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.645+0000 7fddfe7fc640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fddec039730 0x7fddec03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:25.649 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.645+0000 7fddfe7fc640 1 -- 172.21.15.174:0/1912123462 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fde0406e260 con 0x7fddf4003530 2024-09-19T06:41:25.649 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.645+0000 7fde0cb24640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fddec039730 0x7fddec03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:25.649 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.645+0000 7fde0f589640 1 -- 172.21.15.174:0/1912123462 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde081097d0 con 0x7fddf4003530 2024-09-19T06:41:25.650 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.646+0000 7fde0cb24640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fddec039730 0x7fddec03bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009100 tx=0x7fddf8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:25.652 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.650+0000 7fddfe7fc640 1 -- 172.21.15.174:0/1912123462 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fde04045bd0 con 0x7fddf4003530 2024-09-19T06:41:25.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:41:25.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:25.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:25.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: mon.a calling monitor election 2024-09-19T06:41:25.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: mon.c calling monitor election 2024-09-19T06:41:25.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:25.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:25.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:25.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:25.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:25.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:25.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:25.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T06:41:25.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: monmap epoch 3 2024-09-19T06:41:25.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: fsid d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:41:25.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: last_changed 2024-09-19T06:41:20.522872+0000 2024-09-19T06:41:25.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: created 2024-09-19T06:39:43.868601+0000 2024-09-19T06:41:25.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: min_mon_release 19 (squid) 2024-09-19T06:41:25.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: election_strategy: 1 2024-09-19T06:41:25.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T06:41:25.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T06:41:25.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: 2: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-09-19T06:41:25.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: fsmap 2024-09-19T06:41:25.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:41:25.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: mgrmap e13: x(active, since 45s) 2024-09-19T06:41:25.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: overall HEALTH_OK 2024-09-19T06:41:25.890 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:25.891 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:25.891 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:25.891 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:25.898 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.896+0000 7fde0f589640 1 -- 172.21.15.174:0/1912123462 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fde081a47e0 con 0x7fddf4003530 2024-09-19T06:41:25.898 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.896+0000 7fddfe7fc640 1 -- 172.21.15.174:0/1912123462 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7fde0403d380 con 0x7fddf4003530 2024-09-19T06:41:25.898 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:41:25.898 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":3,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","modified":"2024-09-19T06:41:20.522872Z","created":"2024-09-19T06:39:43.868601Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T06:41:25.899 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 3 2024-09-19T06:41:25.899 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:41:25.899 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:25.899 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:25.899 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: mon.a calling monitor election 2024-09-19T06:41:25.899 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: mon.c calling monitor election 2024-09-19T06:41:25.899 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:25.899 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:25.900 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:25.900 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:25.900 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:25.900 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:25.900 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:25.900 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T06:41:25.900 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: monmap epoch 3 2024-09-19T06:41:25.900 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: fsid d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:41:25.900 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: last_changed 2024-09-19T06:41:20.522872+0000 2024-09-19T06:41:25.900 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: created 2024-09-19T06:39:43.868601+0000 2024-09-19T06:41:25.900 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: min_mon_release 19 (squid) 2024-09-19T06:41:25.900 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: election_strategy: 1 2024-09-19T06:41:25.900 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T06:41:25.900 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T06:41:25.901 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: 2: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-09-19T06:41:25.901 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: fsmap 2024-09-19T06:41:25.901 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:41:25.901 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: mgrmap e13: x(active, since 45s) 2024-09-19T06:41:25.901 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: overall HEALTH_OK 2024-09-19T06:41:25.901 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:25.901 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:25.901 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:25.901 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:25.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.898+0000 7fde0f589640 1 -- 172.21.15.174:0/1912123462 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fddec039730 msgr2=0x7fddec03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:25.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.898+0000 7fde0f589640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fddec039730 0x7fddec03bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009100 tx=0x7fddf8008040 comp rx=0 tx=0).stop 2024-09-19T06:41:25.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.898+0000 7fde0f589640 1 -- 172.21.15.174:0/1912123462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddf4003530 msgr2=0x7fde08197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:25.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.898+0000 7fde0f589640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddf4003530 0x7fde08197300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fde04036000 tx=0x7fde04004d30 comp rx=0 tx=0).stop 2024-09-19T06:41:25.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.898+0000 7fde0f589640 1 -- 172.21.15.174:0/1912123462 shutdown_connections 2024-09-19T06:41:25.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.898+0000 7fde0f589640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde0819c110 0x7fde0819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.898+0000 7fde0f589640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fddec039730 0x7fddec03bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.898+0000 7fde0f589640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde08197840 0x7fde0819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.898+0000 7fde0f589640 1 --2- 172.21.15.174:0/1912123462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddf4003530 0x7fde08197300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.898+0000 7fde0f589640 1 -- 172.21.15.174:0/1912123462 >> 172.21.15.174:0/1912123462 conn(0x7fde080fc250 msgr2=0x7fde0806a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:25.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.899+0000 7fde0f589640 1 -- 172.21.15.174:0/1912123462 shutdown_connections 2024-09-19T06:41:25.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:25.899+0000 7fde0f589640 1 -- 172.21.15.174:0/1912123462 wait complete. 2024-09-19T06:41:26.483 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T06:41:26.483 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T06:41:26.740 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:26.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:26 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:26.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:26 smithi174 ceph-mon[35637]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:26.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:26 smithi174 ceph-mon[35637]: from='client.? 172.21.15.174:0/1912123462' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:41:26.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:26 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:26.861 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:26 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:26.861 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:26 smithi103 ceph-mon[31158]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:26.861 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:26 smithi103 ceph-mon[31158]: from='client.? 172.21.15.174:0/1912123462' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:41:26.861 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:26 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:27.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.175+0000 7fdd66583640 1 -- 172.21.15.103:0/620348276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd60100990 msgr2=0x7fdd60100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:27.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.175+0000 7fdd66583640 1 --2- 172.21.15.103:0/620348276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd60100990 0x7fdd60100d70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009880 tx=0x7fdd5002a5d0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.175+0000 7fdd66583640 1 -- 172.21.15.103:0/620348276 shutdown_connections 2024-09-19T06:41:27.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.175+0000 7fdd66583640 1 --2- 172.21.15.103:0/620348276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd60100990 0x7fdd60100d70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.175+0000 7fdd66583640 1 -- 172.21.15.103:0/620348276 >> 172.21.15.103:0/620348276 conn(0x7fdd60076250 msgr2=0x7fdd60076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:27.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.175+0000 7fdd66583640 1 -- 172.21.15.103:0/620348276 shutdown_connections 2024-09-19T06:41:27.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.175+0000 7fdd66583640 1 -- 172.21.15.103:0/620348276 wait complete. 2024-09-19T06:41:27.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.176+0000 7fdd66583640 1 Processor -- start 2024-09-19T06:41:27.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.176+0000 7fdd66583640 1 -- start start 2024-09-19T06:41:27.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.176+0000 7fdd66583640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd60100990 0x7fdd601981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:27.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.176+0000 7fdd66583640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd60198720 0x7fdd60198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:27.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.176+0000 7fdd66583640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd601a1250 0x7fdd601a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:27.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.176+0000 7fdd66583640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6010ab40 con 0x7fdd60198720 2024-09-19T06:41:27.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.176+0000 7fdd66583640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6010a9c0 con 0x7fdd601a1250 2024-09-19T06:41:27.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.176+0000 7fdd66583640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6010acc0 con 0x7fdd60100990 2024-09-19T06:41:27.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd5f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd60198720 0x7fdd60198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:27.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd5f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd60198720 0x7fdd60198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37564/0 (socket says 172.21.15.103:37564) 2024-09-19T06:41:27.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd5f7fe640 1 -- 172.21.15.103:0/964011333 learned_addr learned my addr 172.21.15.103:0/964011333 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:41:27.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd5ffff640 1 --2- 172.21.15.103:0/964011333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd60100990 0x7fdd601981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:27.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd64b20640 1 --2- 172.21.15.103:0/964011333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd601a1250 0x7fdd601a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:27.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd5f7fe640 1 -- 172.21.15.103:0/964011333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd60100990 msgr2=0x7fdd601981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:27.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd5f7fe640 1 --2- 172.21.15.103:0/964011333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd60100990 0x7fdd601981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd64b20640 1 -- 172.21.15.103:0/964011333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd601a1250 msgr2=0x7fdd601a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-19T06:41:27.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd64b20640 1 -- 172.21.15.103:0/964011333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd601a1250 msgr2=0x7fdd601a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T06:41:27.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd64b20640 1 --2- 172.21.15.103:0/964011333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd601a1250 0x7fdd601a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T06:41:27.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd5f7fe640 1 -- 172.21.15.103:0/964011333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd601a1250 msgr2=0x7fdd601a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:27.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd64b20640 1 --2- 172.21.15.103:0/964011333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd601a1250 0x7fdd601a3640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:41:27.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd5f7fe640 1 --2- 172.21.15.103:0/964011333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd601a1250 0x7fdd601a3640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd5f7fe640 1 -- 172.21.15.103:0/964011333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd601a3de0 con 0x7fdd60198720 2024-09-19T06:41:27.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd5f7fe640 1 --2- 172.21.15.103:0/964011333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd60198720 0x7fdd60198b80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400b520 tx=0x7fdd5400b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:27.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd5d7fa640 1 -- 172.21.15.103:0/964011333 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54004230 con 0x7fdd60198720 2024-09-19T06:41:27.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd5d7fa640 1 -- 172.21.15.103:0/964011333 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd540043d0 con 0x7fdd60198720 2024-09-19T06:41:27.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.177+0000 7fdd5d7fa640 1 -- 172.21.15.103:0/964011333 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54003d60 con 0x7fdd60198720 2024-09-19T06:41:27.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.178+0000 7fdd66583640 1 -- 172.21.15.103:0/964011333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd60196030 con 0x7fdd60198720 2024-09-19T06:41:27.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.178+0000 7fdd66583640 1 -- 172.21.15.103:0/964011333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd60196520 con 0x7fdd60198720 2024-09-19T06:41:27.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.179+0000 7fdd66583640 1 -- 172.21.15.103:0/964011333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd60100df0 con 0x7fdd60198720 2024-09-19T06:41:27.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.179+0000 7fdd5d7fa640 1 -- 172.21.15.103:0/964011333 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fdd5400e070 con 0x7fdd60198720 2024-09-19T06:41:27.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.179+0000 7fdd5d7fa640 1 --2- 172.21.15.103:0/964011333 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdd48039730 0x7fdd4803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:27.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.179+0000 7fdd5d7fa640 1 -- 172.21.15.103:0/964011333 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdd5404bcf0 con 0x7fdd60198720 2024-09-19T06:41:27.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.179+0000 7fdd5ffff640 1 --2- 172.21.15.103:0/964011333 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdd48039730 0x7fdd4803bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:27.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.180+0000 7fdd5ffff640 1 --2- 172.21.15.103:0/964011333 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdd48039730 0x7fdd4803bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009230 tx=0x7fdd50005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:27.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.188+0000 7fdd5d7fa640 1 -- 172.21.15.103:0/964011333 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdd54050070 con 0x7fdd60198720 2024-09-19T06:41:27.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.372+0000 7fdd66583640 1 -- 172.21.15.103:0/964011333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fdd6002d7c0 con 0x7fdd60198720 2024-09-19T06:41:27.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.372+0000 7fdd5d7fa640 1 -- 172.21.15.103:0/964011333 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7fdd54014020 con 0x7fdd60198720 2024-09-19T06:41:27.373 INFO:teuthology.orchestra.run.smithi103.stdout:# minimal ceph.conf for d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:41:27.374 INFO:teuthology.orchestra.run.smithi103.stdout:[global] 2024-09-19T06:41:27.374 INFO:teuthology.orchestra.run.smithi103.stdout: fsid = d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:41:27.374 INFO:teuthology.orchestra.run.smithi103.stdout: mon_host = [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2024-09-19T06:41:27.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.374+0000 7fdd66583640 1 -- 172.21.15.103:0/964011333 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdd48039730 msgr2=0x7fdd4803bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:27.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.374+0000 7fdd66583640 1 --2- 172.21.15.103:0/964011333 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdd48039730 0x7fdd4803bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009230 tx=0x7fdd50005a40 comp rx=0 tx=0).stop 2024-09-19T06:41:27.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.374+0000 7fdd66583640 1 -- 172.21.15.103:0/964011333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd60198720 msgr2=0x7fdd60198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:27.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.374+0000 7fdd66583640 1 --2- 172.21.15.103:0/964011333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd60198720 0x7fdd60198b80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400b520 tx=0x7fdd5400b9e0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.374+0000 7fdd66583640 1 -- 172.21.15.103:0/964011333 shutdown_connections 2024-09-19T06:41:27.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.374+0000 7fdd66583640 1 --2- 172.21.15.103:0/964011333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd601a1250 0x7fdd601a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.374+0000 7fdd66583640 1 --2- 172.21.15.103:0/964011333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd60198720 0x7fdd60198b80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.374+0000 7fdd66583640 1 --2- 172.21.15.103:0/964011333 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdd48039730 0x7fdd4803bbf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.374+0000 7fdd66583640 1 --2- 172.21.15.103:0/964011333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd60100990 0x7fdd601981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.374+0000 7fdd66583640 1 -- 172.21.15.103:0/964011333 >> 172.21.15.103:0/964011333 conn(0x7fdd60076250 msgr2=0x7fdd601020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:27.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.374+0000 7fdd66583640 1 -- 172.21.15.103:0/964011333 shutdown_connections 2024-09-19T06:41:27.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:27.375+0000 7fdd66583640 1 -- 172.21.15.103:0/964011333 wait complete. 2024-09-19T06:41:27.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:41:27.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:27.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:27.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: mon.a calling monitor election 2024-09-19T06:41:27.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: mon.c calling monitor election 2024-09-19T06:41:27.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:27.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:27.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:27.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:27.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:27.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:27.939 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:27.939 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T06:41:27.939 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: monmap epoch 3 2024-09-19T06:41:27.939 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: fsid d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:41:27.939 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: last_changed 2024-09-19T06:41:20.522872+0000 2024-09-19T06:41:27.939 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: created 2024-09-19T06:39:43.868601+0000 2024-09-19T06:41:27.939 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: min_mon_release 19 (squid) 2024-09-19T06:41:27.939 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: election_strategy: 1 2024-09-19T06:41:27.939 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T06:41:27.939 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T06:41:27.939 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: 2: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-09-19T06:41:27.939 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: fsmap 2024-09-19T06:41:27.939 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:41:27.939 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: mgrmap e13: x(active, since 45s) 2024-09-19T06:41:27.940 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: overall HEALTH_OK 2024-09-19T06:41:27.940 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:27.940 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:27.940 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:27.940 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:27.940 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:27.940 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:27.940 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: from='client.? 172.21.15.174:0/1912123462' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:41:27.940 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:27 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:28.029 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T06:41:28.030 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:41:28.030 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T06:41:28.058 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:41:28.058 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:41:28.124 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T06:41:28.124 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T06:41:28.152 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T06:41:28.153 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:41:28.218 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T06:41:28.218 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T06:41:28.245 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T06:41:28.246 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:41:28.312 INFO:tasks.cephadm:Adding mgr.x on smithi103 2024-09-19T06:41:28.312 INFO:tasks.cephadm:Adding mgr.y on smithi134 2024-09-19T06:41:28.312 INFO:tasks.cephadm:Adding mgr.z on smithi174 2024-09-19T06:41:28.313 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi103=x;smithi134=y;smithi174=z' 2024-09-19T06:41:28.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: mon.b calling monitor election 2024-09-19T06:41:28.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-19T06:41:28.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-19T06:41:28.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T06:41:28.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: Updating smithi174:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:28.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: Updating smithi103:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:28.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: Updating smithi134:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:28.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: mon.b calling monitor election 2024-09-19T06:41:28.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: mon.c calling monitor election 2024-09-19T06:41:28.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: mon.a calling monitor election 2024-09-19T06:41:28.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T06:41:28.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: monmap epoch 3 2024-09-19T06:41:28.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: fsid d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:41:28.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: last_changed 2024-09-19T06:41:20.522872+0000 2024-09-19T06:41:28.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: created 2024-09-19T06:39:43.868601+0000 2024-09-19T06:41:28.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: min_mon_release 19 (squid) 2024-09-19T06:41:28.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: election_strategy: 1 2024-09-19T06:41:28.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T06:41:28.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T06:41:28.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: 2: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-09-19T06:41:28.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: fsmap 2024-09-19T06:41:28.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:41:28.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: mgrmap e13: x(active, since 47s) 2024-09-19T06:41:28.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: overall HEALTH_OK 2024-09-19T06:41:28.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:28.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.441 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.441 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.441 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.441 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.441 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.441 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T06:41:28.441 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:41:28.441 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:41:28.441 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:28.441 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:28 smithi134 ceph-mon[35356]: Reconfiguring daemon mon.a on smithi103 2024-09-19T06:41:28.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: mon.b calling monitor election 2024-09-19T06:41:28.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-19T06:41:28.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-19T06:41:28.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T06:41:28.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: Updating smithi174:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:28.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: Updating smithi103:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:28.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: Updating smithi134:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:28.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: mon.b calling monitor election 2024-09-19T06:41:28.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: mon.c calling monitor election 2024-09-19T06:41:28.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: mon.a calling monitor election 2024-09-19T06:41:28.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T06:41:28.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: monmap epoch 3 2024-09-19T06:41:28.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: fsid d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:41:28.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: last_changed 2024-09-19T06:41:20.522872+0000 2024-09-19T06:41:28.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: created 2024-09-19T06:39:43.868601+0000 2024-09-19T06:41:28.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: min_mon_release 19 (squid) 2024-09-19T06:41:28.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: election_strategy: 1 2024-09-19T06:41:28.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T06:41:28.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T06:41:28.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: 2: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-09-19T06:41:28.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: fsmap 2024-09-19T06:41:28.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:41:28.483 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: mgrmap e13: x(active, since 47s) 2024-09-19T06:41:28.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: overall HEALTH_OK 2024-09-19T06:41:28.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:28.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T06:41:28.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:41:28.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:41:28.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:28.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:28 smithi103 ceph-mon[31158]: Reconfiguring daemon mon.a on smithi103 2024-09-19T06:41:28.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: mon.b calling monitor election 2024-09-19T06:41:28.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-19T06:41:28.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-19T06:41:28.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T06:41:28.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: Updating smithi174:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:28.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: Updating smithi103:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:28.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: Updating smithi134:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:28.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: mon.b calling monitor election 2024-09-19T06:41:28.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: mon.c calling monitor election 2024-09-19T06:41:28.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: mon.a calling monitor election 2024-09-19T06:41:28.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T06:41:28.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: monmap epoch 3 2024-09-19T06:41:28.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: fsid d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:41:28.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: last_changed 2024-09-19T06:41:20.522872+0000 2024-09-19T06:41:28.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: created 2024-09-19T06:39:43.868601+0000 2024-09-19T06:41:28.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: min_mon_release 19 (squid) 2024-09-19T06:41:28.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: election_strategy: 1 2024-09-19T06:41:28.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T06:41:28.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T06:41:28.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: 2: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-09-19T06:41:28.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: fsmap 2024-09-19T06:41:28.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:41:28.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: mgrmap e13: x(active, since 47s) 2024-09-19T06:41:28.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: overall HEALTH_OK 2024-09-19T06:41:28.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:28.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:28.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T06:41:28.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:41:28.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:41:28.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:28.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:28 smithi174 ceph-mon[35637]: Reconfiguring daemon mon.a on smithi103 2024-09-19T06:41:28.587 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:41:29.014 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.010+0000 7f6b415de640 1 -- 172.21.15.174:0/1222700515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b3c1005e0 msgr2=0x7f6b3c1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:29.014 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.010+0000 7f6b415de640 1 --2- 172.21.15.174:0/1222700515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b3c1005e0 0x7f6b3c1009e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c009c20 tx=0x7f6b2c02aa90 comp rx=0 tx=0).stop 2024-09-19T06:41:29.014 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.010+0000 7f6b415de640 1 -- 172.21.15.174:0/1222700515 shutdown_connections 2024-09-19T06:41:29.014 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.010+0000 7f6b415de640 1 --2- 172.21.15.174:0/1222700515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b3c1005e0 0x7f6b3c1009e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:29.014 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.010+0000 7f6b415de640 1 -- 172.21.15.174:0/1222700515 >> 172.21.15.174:0/1222700515 conn(0x7f6b3c0fbdb0 msgr2=0x7f6b3c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:29.014 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.010+0000 7f6b415de640 1 -- 172.21.15.174:0/1222700515 shutdown_connections 2024-09-19T06:41:29.014 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.010+0000 7f6b415de640 1 -- 172.21.15.174:0/1222700515 wait complete. 2024-09-19T06:41:29.014 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.011+0000 7f6b415de640 1 Processor -- start 2024-09-19T06:41:29.015 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.011+0000 7f6b415de640 1 -- start start 2024-09-19T06:41:29.015 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.011+0000 7f6b415de640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b3c1005e0 0x7f6b3c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:29.015 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.011+0000 7f6b415de640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b3c19c100 0x7f6b3c19e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:29.015 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.011+0000 7f6b415de640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c19ea30 0x7f6b3c1a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:29.015 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.011+0000 7f6b415de640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c1066f0 con 0x7f6b3c1005e0 2024-09-19T06:41:29.015 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.011+0000 7f6b415de640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c106570 con 0x7f6b3c19c100 2024-09-19T06:41:29.015 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.011+0000 7f6b415de640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c106870 con 0x7f6b3c19ea30 2024-09-19T06:41:29.015 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.011+0000 7f6b3b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c19ea30 0x7f6b3c1a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:29.015 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.011+0000 7f6b3b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c19ea30 0x7f6b3c1a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40122/0 (socket says 172.21.15.174:40122) 2024-09-19T06:41:29.015 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.011+0000 7f6b3b7fe640 1 -- 172.21.15.174:0/1683971121 learned_addr learned my addr 172.21.15.174:0/1683971121 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T06:41:29.015 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.011+0000 7f6b3affd640 1 --2- 172.21.15.174:0/1683971121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b3c1005e0 0x7f6b3c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:29.015 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.012+0000 7f6b3b7fe640 1 -- 172.21.15.174:0/1683971121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b3c19c100 msgr2=0x7f6b3c19e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:29.015 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.012+0000 7f6b3a7fc640 1 --2- 172.21.15.174:0/1683971121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b3c19c100 0x7f6b3c19e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:29.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.012+0000 7f6b3b7fe640 1 --2- 172.21.15.174:0/1683971121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b3c19c100 0x7f6b3c19e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:29.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.012+0000 7f6b3b7fe640 1 -- 172.21.15.174:0/1683971121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b3c1005e0 msgr2=0x7f6b3c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:29.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.012+0000 7f6b3b7fe640 1 --2- 172.21.15.174:0/1683971121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b3c1005e0 0x7f6b3c06a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:29.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.012+0000 7f6b3b7fe640 1 -- 172.21.15.174:0/1683971121 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3c1a1610 con 0x7f6b3c19ea30 2024-09-19T06:41:29.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.012+0000 7f6b3affd640 1 --2- 172.21.15.174:0/1683971121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b3c1005e0 0x7f6b3c06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:41:29.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.012+0000 7f6b3a7fc640 1 --2- 172.21.15.174:0/1683971121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b3c19c100 0x7f6b3c19e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:29.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.012+0000 7f6b3b7fe640 1 --2- 172.21.15.174:0/1683971121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c19ea30 0x7f6b3c1a0e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400b450 tx=0x7f6b3400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:29.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.012+0000 7f6b23fff640 1 -- 172.21.15.174:0/1683971121 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34015020 con 0x7f6b3c19ea30 2024-09-19T06:41:29.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.012+0000 7f6b415de640 1 -- 172.21.15.174:0/1683971121 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3c1a18a0 con 0x7f6b3c19ea30 2024-09-19T06:41:29.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.012+0000 7f6b23fff640 1 -- 172.21.15.174:0/1683971121 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b34011d70 con 0x7f6b3c19ea30 2024-09-19T06:41:29.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.012+0000 7f6b23fff640 1 -- 172.21.15.174:0/1683971121 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3400f340 con 0x7f6b3c19ea30 2024-09-19T06:41:29.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.012+0000 7f6b415de640 1 -- 172.21.15.174:0/1683971121 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b3c1a1e60 con 0x7f6b3c19ea30 2024-09-19T06:41:29.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.013+0000 7f6b23fff640 1 -- 172.21.15.174:0/1683971121 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f6b34018020 con 0x7f6b3c19ea30 2024-09-19T06:41:29.017 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.013+0000 7f6b23fff640 1 --2- 172.21.15.174:0/1683971121 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6b24039760 0x7f6b2403bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:29.017 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.013+0000 7f6b23fff640 1 -- 172.21.15.174:0/1683971121 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6b3404d160 con 0x7f6b3c19ea30 2024-09-19T06:41:29.017 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.013+0000 7f6b415de640 1 -- 172.21.15.174:0/1683971121 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b3c18f4b0 con 0x7f6b3c19ea30 2024-09-19T06:41:29.017 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.014+0000 7f6b3affd640 1 --2- 172.21.15.174:0/1683971121 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6b24039760 0x7f6b2403bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:29.018 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.014+0000 7f6b3affd640 1 --2- 172.21.15.174:0/1683971121 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6b24039760 0x7f6b2403bc20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c005e30 tx=0x7f6b2c03a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:29.019 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.018+0000 7f6b23fff640 1 -- 172.21.15.174:0/1683971121 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6b3401f020 con 0x7f6b3c19ea30 2024-09-19T06:41:29.196 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.194+0000 7f6b415de640 1 -- 172.21.15.174:0/1683971121 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi134=y;smithi174=z", "target": ["mon-mgr", ""]}) -- 0x7f6b3c0626c0 con 0x7f6b24039760 2024-09-19T06:41:29.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.200+0000 7f6b23fff640 1 -- 172.21.15.174:0/1683971121 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6b3c0626c0 con 0x7f6b24039760 2024-09-19T06:41:29.201 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mgr update... 2024-09-19T06:41:29.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.202+0000 7f6b415de640 1 -- 172.21.15.174:0/1683971121 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6b24039760 msgr2=0x7f6b2403bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:29.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.202+0000 7f6b415de640 1 --2- 172.21.15.174:0/1683971121 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6b24039760 0x7f6b2403bc20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c005e30 tx=0x7f6b2c03a040 comp rx=0 tx=0).stop 2024-09-19T06:41:29.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.202+0000 7f6b415de640 1 -- 172.21.15.174:0/1683971121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c19ea30 msgr2=0x7f6b3c1a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:29.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.202+0000 7f6b415de640 1 --2- 172.21.15.174:0/1683971121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c19ea30 0x7f6b3c1a0e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400b450 tx=0x7f6b3400b910 comp rx=0 tx=0).stop 2024-09-19T06:41:29.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.202+0000 7f6b415de640 1 -- 172.21.15.174:0/1683971121 shutdown_connections 2024-09-19T06:41:29.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.202+0000 7f6b415de640 1 --2- 172.21.15.174:0/1683971121 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6b24039760 0x7f6b2403bc20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:29.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.202+0000 7f6b415de640 1 --2- 172.21.15.174:0/1683971121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c19ea30 0x7f6b3c1a0e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:29.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.202+0000 7f6b415de640 1 --2- 172.21.15.174:0/1683971121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b3c19c100 0x7f6b3c19e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:29.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.202+0000 7f6b415de640 1 --2- 172.21.15.174:0/1683971121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b3c1005e0 0x7f6b3c06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:29.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.202+0000 7f6b415de640 1 -- 172.21.15.174:0/1683971121 >> 172.21.15.174:0/1683971121 conn(0x7f6b3c0fbdb0 msgr2=0x7f6b3c0fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:29.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.202+0000 7f6b415de640 1 -- 172.21.15.174:0/1683971121 shutdown_connections 2024-09-19T06:41:29.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:41:29.202+0000 7f6b415de640 1 -- 172.21.15.174:0/1683971121 wait complete. 2024-09-19T06:41:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:29 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:29 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:29.525 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:29 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:29.525 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:29 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:29.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:29 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:29.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:29 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:29.778 DEBUG:teuthology.orchestra.run.smithi134:mgr.y> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T06:41:29.781 DEBUG:teuthology.orchestra.run.smithi174:mgr.z> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T06:41:29.783 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T06:41:29.783 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:41:29.783 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T06:41:29.793 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:41:29 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:41:29.522+0000 7f71b1748640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-19T06:41:29.800 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T06:41:29.800 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2024-09-19T06:41:29.857 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T06:41:29.857 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:41:29.857 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T06:41:29.857 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:41:29.857 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:41:29.857 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-19 06:40:50.499731686 +0000 2024-09-19T06:41:29.857 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-19 06:36:22.175158527 +0000 2024-09-19T06:41:29.857 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-19 06:36:22.175158527 +0000 2024-09-19T06:41:29.857 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-19 06:36:22.175158527 +0000 2024-09-19T06:41:29.858 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T06:41:29.921 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-19T06:41:29.921 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-19T06:41:29.921 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000215314 s, 2.4 MB/s 2024-09-19T06:41:29.922 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T06:41:29.979 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2024-09-19T06:41:30.035 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T06:41:30.035 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:41:30.035 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-19T06:41:30.035 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:41:30.035 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:41:30.035 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-19 06:40:50.532731065 +0000 2024-09-19T06:41:30.035 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-19 06:36:22.497152563 +0000 2024-09-19T06:41:30.036 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-19 06:36:22.497152563 +0000 2024-09-19T06:41:30.036 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-19 06:36:22.497152563 +0000 2024-09-19T06:41:30.036 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T06:41:30.099 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-19T06:41:30.099 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-19T06:41:30.099 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000167595 s, 3.1 MB/s 2024-09-19T06:41:30.100 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T06:41:30.157 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2024-09-19T06:41:30.212 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T06:41:30.212 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:41:30.212 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-19T06:41:30.212 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:41:30.212 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:41:30.212 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-19 06:40:50.565730445 +0000 2024-09-19T06:41:30.213 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-19 06:36:22.824146506 +0000 2024-09-19T06:41:30.213 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-19 06:36:22.824146506 +0000 2024-09-19T06:41:30.213 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-19 06:36:22.824146506 +0000 2024-09-19T06:41:30.213 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T06:41:30.279 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-19T06:41:30.279 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-19T06:41:30.279 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000211186 s, 2.4 MB/s 2024-09-19T06:41:30.279 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T06:41:30.335 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2024-09-19T06:41:30.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:30 smithi103 ceph-mon[31158]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi134=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:30.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:30 smithi103 ceph-mon[31158]: Saving service mgr spec with placement smithi103=x;smithi134=y;smithi174=z;count:3 2024-09-19T06:41:30.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:30 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:30.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:30 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:30.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:30 smithi103 ceph-mon[31158]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T06:41:30.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:30 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:41:30.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:30 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:41:30.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:30 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:30.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:30 smithi103 ceph-mon[31158]: Reconfiguring daemon mgr.x on smithi103 2024-09-19T06:41:30.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:30 smithi103 ceph-mon[31158]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:30.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:30 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:30.390 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T06:41:30.391 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:41:30.391 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T06:41:30.391 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:41:30.391 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:41:30.391 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-19 06:40:50.598729826 +0000 2024-09-19T06:41:30.391 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-19 06:36:23.146140541 +0000 2024-09-19T06:41:30.391 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-19 06:36:23.146140541 +0000 2024-09-19T06:41:30.391 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-19 06:36:23.146140541 +0000 2024-09-19T06:41:30.391 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T06:41:30.455 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-19T06:41:30.455 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-19T06:41:30.455 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000190032 s, 2.7 MB/s 2024-09-19T06:41:30.456 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T06:41:30.512 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T06:41:30.513 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T06:41:30.531 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T06:41:30.531 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-09-19T06:41:30.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:30 smithi174 ceph-mon[35637]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi134=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:30.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:30 smithi174 ceph-mon[35637]: Saving service mgr spec with placement smithi103=x;smithi134=y;smithi174=z;count:3 2024-09-19T06:41:30.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:30 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:30.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:30 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:30.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:30 smithi174 ceph-mon[35637]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T06:41:30.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:30 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:41:30.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:30 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:41:30.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:30 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:30.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:30 smithi174 ceph-mon[35637]: Reconfiguring daemon mgr.x on smithi103 2024-09-19T06:41:30.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:30 smithi174 ceph-mon[35637]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:30.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:30 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:30.589 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T06:41:30.590 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:41:30.590 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-19T06:41:30.590 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:41:30.590 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:41:30.590 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-19 06:41:19.950713550 +0000 2024-09-19T06:41:30.590 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-19 06:36:37.598330185 +0000 2024-09-19T06:41:30.590 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-19 06:36:37.598330185 +0000 2024-09-19T06:41:30.590 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-19 06:36:37.598330185 +0000 2024-09-19T06:41:30.590 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T06:41:30.653 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:30 smithi134 ceph-mon[35356]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi134=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:30.653 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:30 smithi134 ceph-mon[35356]: Saving service mgr spec with placement smithi103=x;smithi134=y;smithi174=z;count:3 2024-09-19T06:41:30.653 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:30 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:30.653 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:30 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:30.654 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:30 smithi134 ceph-mon[35356]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T06:41:30.654 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:30 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:41:30.654 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:30 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:41:30.654 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:30 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:30.654 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:30 smithi134 ceph-mon[35356]: Reconfiguring daemon mgr.x on smithi103 2024-09-19T06:41:30.654 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:30 smithi134 ceph-mon[35356]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:30.654 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:30 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:30.655 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-19T06:41:30.655 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-19T06:41:30.656 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000268769 s, 1.9 MB/s 2024-09-19T06:41:30.657 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T06:41:30.715 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-09-19T06:41:30.770 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T06:41:30.770 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:41:30.770 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-19T06:41:30.770 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:41:30.770 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:41:30.771 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-19 06:41:19.950713550 +0000 2024-09-19T06:41:30.771 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-19 06:36:37.918322605 +0000 2024-09-19T06:41:30.771 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-19 06:36:37.918322605 +0000 2024-09-19T06:41:30.771 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-19 06:36:37.918322605 +0000 2024-09-19T06:41:30.771 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T06:41:30.835 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-19T06:41:30.835 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-19T06:41:30.835 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000173459 s, 3.0 MB/s 2024-09-19T06:41:30.837 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T06:41:30.894 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-09-19T06:41:30.949 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T06:41:30.949 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:41:30.949 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-19T06:41:30.949 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:41:30.949 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:41:30.950 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-19 06:41:19.951713528 +0000 2024-09-19T06:41:30.950 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-19 06:36:38.237315049 +0000 2024-09-19T06:41:30.950 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-19 06:36:38.237315049 +0000 2024-09-19T06:41:30.950 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-19 06:36:38.237315049 +0000 2024-09-19T06:41:30.950 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T06:41:31.012 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-19T06:41:31.013 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-19T06:41:31.013 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000198154 s, 2.6 MB/s 2024-09-19T06:41:31.014 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T06:41:31.070 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-09-19T06:41:31.128 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T06:41:31.128 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:41:31.128 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-19T06:41:31.128 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:41:31.128 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:41:31.128 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-19 06:41:19.951713528 +0000 2024-09-19T06:41:31.128 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-19 06:36:38.560307398 +0000 2024-09-19T06:41:31.128 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-19 06:36:38.560307398 +0000 2024-09-19T06:41:31.128 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-19 06:36:38.560307398 +0000 2024-09-19T06:41:31.129 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T06:41:31.192 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-19T06:41:31.192 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-19T06:41:31.192 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000189305 s, 2.7 MB/s 2024-09-19T06:41:31.193 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T06:41:31.252 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T06:41:31.253 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T06:41:31.272 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T06:41:31.272 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 2024-09-19T06:41:31.331 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T06:41:31.331 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:41:31.331 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T06:41:31.332 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:41:31.332 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:41:31.332 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-19 06:41:10.900961811 +0000 2024-09-19T06:41:31.332 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-19 06:36:35.703651154 +0000 2024-09-19T06:41:31.332 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-19 06:36:35.703651154 +0000 2024-09-19T06:41:31.332 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-19 06:36:35.703651154 +0000 2024-09-19T06:41:31.332 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T06:41:31.399 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-19T06:41:31.399 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-19T06:41:31.399 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000355539 s, 1.4 MB/s 2024-09-19T06:41:31.401 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T06:41:31.459 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 2024-09-19T06:41:31.517 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T06:41:31.517 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:41:31.517 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T06:41:31.517 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:41:31.517 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:41:31.517 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-19 06:41:10.900961811 +0000 2024-09-19T06:41:31.517 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-19 06:36:36.032646925 +0000 2024-09-19T06:41:31.517 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-19 06:36:36.032646925 +0000 2024-09-19T06:41:31.517 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-19 06:36:36.032646925 +0000 2024-09-19T06:41:31.518 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T06:41:31.582 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-19T06:41:31.583 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-19T06:41:31.583 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000201651 s, 2.5 MB/s 2024-09-19T06:41:31.584 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T06:41:31.641 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 2024-09-19T06:41:31.698 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T06:41:31.698 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:41:31.698 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-19T06:41:31.698 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:41:31.698 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:41:31.698 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-19 06:41:10.901961798 +0000 2024-09-19T06:41:31.698 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-19 06:36:36.360642708 +0000 2024-09-19T06:41:31.699 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-19 06:36:36.360642708 +0000 2024-09-19T06:41:31.699 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-19 06:36:36.360642708 +0000 2024-09-19T06:41:31.699 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T06:41:31.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:31 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:31.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:31 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:31.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:31 smithi174 ceph-mon[35637]: Reconfiguring mon.b (monmap changed)... 2024-09-19T06:41:31.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:31 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:41:31.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:31 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:41:31.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:31 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:31.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:31 smithi174 ceph-mon[35637]: Reconfiguring daemon mon.b on smithi134 2024-09-19T06:41:31.762 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-19T06:41:31.762 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-19T06:41:31.762 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000191214 s, 2.7 MB/s 2024-09-19T06:41:31.764 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T06:41:31.820 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 2024-09-19T06:41:31.876 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T06:41:31.876 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:41:31.876 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T06:41:31.876 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:41:31.876 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:41:31.877 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-19 06:41:10.901961798 +0000 2024-09-19T06:41:31.877 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-19 06:36:36.688638491 +0000 2024-09-19T06:41:31.877 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-19 06:36:36.688638491 +0000 2024-09-19T06:41:31.877 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-19 06:36:36.688638491 +0000 2024-09-19T06:41:31.877 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T06:41:31.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:31 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:31.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:31 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:31.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:31 smithi103 ceph-mon[31158]: Reconfiguring mon.b (monmap changed)... 2024-09-19T06:41:31.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:31 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:41:31.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:31 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:41:31.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:31 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:31.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:31 smithi103 ceph-mon[31158]: Reconfiguring daemon mon.b on smithi134 2024-09-19T06:41:31.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:31 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:31.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:31 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:31.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:31 smithi134 ceph-mon[35356]: Reconfiguring mon.b (monmap changed)... 2024-09-19T06:41:31.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:31 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:41:31.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:31 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:41:31.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:31 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:31.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:31 smithi134 ceph-mon[35356]: Reconfiguring daemon mon.b on smithi134 2024-09-19T06:41:31.940 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-19T06:41:31.940 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-19T06:41:31.940 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000219414 s, 2.3 MB/s 2024-09-19T06:41:31.941 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T06:41:31.999 INFO:tasks.cephadm:Deploying osd.0 on smithi103 with /dev/vg_nvme/lv_4... 2024-09-19T06:41:31.999 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T06:41:32.246 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:32.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:32 smithi103 ceph-mon[31158]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:32.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:32 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:32.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:32 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:32.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:32 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:41:32.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:32 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:41:32.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:32 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:32.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:32 smithi174 ceph-mon[35637]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:32.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:32 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:32.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:32 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:32.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:32 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:41:32.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:32 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:41:32.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:32 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:32.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:32 smithi134 ceph-mon[35356]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:32.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:32 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:32.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:32 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:32.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:32 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:41:32.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:32 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:41:32.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:32 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:33.870 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:33 smithi174 ceph-mon[35637]: Reconfiguring mon.c (monmap changed)... 2024-09-19T06:41:33.870 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:33 smithi174 ceph-mon[35637]: Reconfiguring daemon mon.c on smithi174 2024-09-19T06:41:33.870 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:33.871 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:33.871 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:33.871 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:33.871 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:33.871 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:33.871 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:41:33.871 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T06:41:33.871 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:41:33.871 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:33.881 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:33 smithi103 ceph-mon[31158]: Reconfiguring mon.c (monmap changed)... 2024-09-19T06:41:33.881 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:33 smithi103 ceph-mon[31158]: Reconfiguring daemon mon.c on smithi174 2024-09-19T06:41:33.881 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:33.881 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:33.881 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:33.881 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:33.881 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:33.882 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:33.882 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:41:33.882 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T06:41:33.882 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:41:33.882 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:33.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:33 smithi134 ceph-mon[35356]: Reconfiguring mon.c (monmap changed)... 2024-09-19T06:41:33.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:33 smithi134 ceph-mon[35356]: Reconfiguring daemon mon.c on smithi174 2024-09-19T06:41:33.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:33.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:33.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:33.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:33.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:33.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:33.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:41:33.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T06:41:33.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:41:33.939 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:34.524 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:41:34.548 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_4 2024-09-19T06:41:34.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:34 smithi103 ceph-mon[31158]: Deploying daemon mgr.z on smithi174 2024-09-19T06:41:34.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:34 smithi103 ceph-mon[31158]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:34.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:34 smithi174 ceph-mon[35637]: Deploying daemon mgr.z on smithi174 2024-09-19T06:41:34.781 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:34 smithi174 ceph-mon[35637]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:34.791 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:34.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:34 smithi134 ceph-mon[35356]: Deploying daemon mgr.z on smithi174 2024-09-19T06:41:34.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:34 smithi134 ceph-mon[35356]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:35.083 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:34 smithi174 systemd[1]: Starting Ceph mgr.z for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T06:41:35.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.206+0000 7fe76e103640 1 -- 172.21.15.103:0/3168257453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe768101cd0 msgr2=0x7fe768102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:35.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.206+0000 7fe76e103640 1 --2- 172.21.15.103:0/3168257453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe768101cd0 0x7fe768102150 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009900 tx=0x7fe75c02a410 comp rx=0 tx=0).stop 2024-09-19T06:41:35.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.206+0000 7fe76e103640 1 -- 172.21.15.103:0/3168257453 shutdown_connections 2024-09-19T06:41:35.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.206+0000 7fe76e103640 1 --2- 172.21.15.103:0/3168257453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe768102690 0x7fe768108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:35.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.206+0000 7fe76e103640 1 --2- 172.21.15.103:0/3168257453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe768101cd0 0x7fe768102150 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:35.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.206+0000 7fe76e103640 1 --2- 172.21.15.103:0/3168257453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe768100ad0 0x7fe768100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:35.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.206+0000 7fe76e103640 1 -- 172.21.15.103:0/3168257453 >> 172.21.15.103:0/3168257453 conn(0x7fe7680fc280 msgr2=0x7fe7680fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:35.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.206+0000 7fe76e103640 1 -- 172.21.15.103:0/3168257453 shutdown_connections 2024-09-19T06:41:35.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.206+0000 7fe76e103640 1 -- 172.21.15.103:0/3168257453 wait complete. 2024-09-19T06:41:35.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.206+0000 7fe76e103640 1 Processor -- start 2024-09-19T06:41:35.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe76e103640 1 -- start start 2024-09-19T06:41:35.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe76e103640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe768100ad0 0x7fe76819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:35.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe76e103640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe768101cd0 0x7fe76819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:35.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe76e103640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe768102690 0x7fe7681a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:35.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe76e103640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe76810d880 con 0x7fe768101cd0 2024-09-19T06:41:35.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe76e103640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe76810d700 con 0x7fe768100ad0 2024-09-19T06:41:35.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe76e103640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe76810da00 con 0x7fe768102690 2024-09-19T06:41:35.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe766ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe768101cd0 0x7fe76819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:35.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe766ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe768101cd0 0x7fe76819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55986/0 (socket says 172.21.15.103:55986) 2024-09-19T06:41:35.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe766ffd640 1 -- 172.21.15.103:0/2656008883 learned_addr learned my addr 172.21.15.103:0/2656008883 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:41:35.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe7677fe640 1 --2- 172.21.15.103:0/2656008883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe768100ad0 0x7fe76819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:35.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe767fff640 1 --2- 172.21.15.103:0/2656008883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe768102690 0x7fe7681a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:35.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe766ffd640 1 -- 172.21.15.103:0/2656008883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe768102690 msgr2=0x7fe7681a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:35.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe766ffd640 1 --2- 172.21.15.103:0/2656008883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe768102690 0x7fe7681a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:35.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe766ffd640 1 -- 172.21.15.103:0/2656008883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe768100ad0 msgr2=0x7fe76819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:35.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe766ffd640 1 --2- 172.21.15.103:0/2656008883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe768100ad0 0x7fe76819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:35.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe766ffd640 1 -- 172.21.15.103:0/2656008883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7681a1d90 con 0x7fe768101cd0 2024-09-19T06:41:35.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.207+0000 7fe766ffd640 1 --2- 172.21.15.103:0/2656008883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe768101cd0 0x7fe76819a6a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009230 tx=0x7fe75c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:35.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.208+0000 7fe764ff9640 1 -- 172.21.15.103:0/2656008883 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c03f030 con 0x7fe768101cd0 2024-09-19T06:41:35.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.208+0000 7fe764ff9640 1 -- 172.21.15.103:0/2656008883 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe75c003ad0 con 0x7fe768101cd0 2024-09-19T06:41:35.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.208+0000 7fe764ff9640 1 -- 172.21.15.103:0/2656008883 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c035070 con 0x7fe768101cd0 2024-09-19T06:41:35.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.208+0000 7fe76e103640 1 -- 172.21.15.103:0/2656008883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7681a2020 con 0x7fe768101cd0 2024-09-19T06:41:35.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.208+0000 7fe76e103640 1 -- 172.21.15.103:0/2656008883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7681a2500 con 0x7fe768101cd0 2024-09-19T06:41:35.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.209+0000 7fe764ff9640 1 -- 172.21.15.103:0/2656008883 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fe75c042030 con 0x7fe768101cd0 2024-09-19T06:41:35.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.209+0000 7fe76e103640 1 -- 172.21.15.103:0/2656008883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7681a2760 con 0x7fe768101cd0 2024-09-19T06:41:35.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.209+0000 7fe764ff9640 1 --2- 172.21.15.103:0/2656008883 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe750039500 0x7fe75003b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:35.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.209+0000 7fe764ff9640 1 -- 172.21.15.103:0/2656008883 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe75c06cd60 con 0x7fe768101cd0 2024-09-19T06:41:35.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.209+0000 7fe7677fe640 1 --2- 172.21.15.103:0/2656008883 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe750039500 0x7fe75003b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:35.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.210+0000 7fe7677fe640 1 --2- 172.21.15.103:0/2656008883 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe750039500 0x7fe75003b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe768101b30 tx=0x7fe758009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:35.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.214+0000 7fe764ff9640 1 -- 172.21.15.103:0/2656008883 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe75c03a9c0 con 0x7fe768101cd0 2024-09-19T06:41:35.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:35.387+0000 7fe76e103640 1 -- 172.21.15.103:0/2656008883 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe768108be0 con 0x7fe750039500 2024-09-19T06:41:35.431 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:35 smithi174 podman[37423]: 2024-09-19 06:41:35.064057984 +0000 UTC m=+0.015820738 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:41:35.431 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:35 smithi174 podman[37423]: 2024-09-19 06:41:35.195494535 +0000 UTC m=+0.147257326 container create ba463a50c210b5a3eb5502ffae3ad6c9112b6b9af7ce4918f0e75c6dc198168b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T06:41:35.432 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:35 smithi174 podman[37423]: 2024-09-19 06:41:35.431162144 +0000 UTC m=+0.382924889 container init ba463a50c210b5a3eb5502ffae3ad6c9112b6b9af7ce4918f0e75c6dc198168b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-19T06:41:35.768 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:35 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:41:35.768 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:35 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:41:35.768 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:35 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:35.768 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:35 smithi174 podman[37423]: 2024-09-19 06:41:35.433871524 +0000 UTC m=+0.385634270 container start ba463a50c210b5a3eb5502ffae3ad6c9112b6b9af7ce4918f0e75c6dc198168b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:41:35.768 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:35 smithi174 ceph-mgr[37442]: -- 172.21.15.174:0/2759479551 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5621137a30e0 con 0x562113781000 2024-09-19T06:41:35.769 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:35 smithi174 bash[37423]: ba463a50c210b5a3eb5502ffae3ad6c9112b6b9af7ce4918f0e75c6dc198168b 2024-09-19T06:41:35.769 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:35 smithi174 systemd[1]: Started Ceph mgr.z for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:41:35.769 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:35 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:35.554+0000 7fd6a7b6b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:41:35.769 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:35 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:35.618+0000 7fd6a7b6b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:41:35.769 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:35 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:35.766+0000 7fd6a7b6b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:41:35.873 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:35 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:41:35.873 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:35 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:41:35.873 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:35 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:35.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:35 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:41:35.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:35 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:41:35.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:35 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:36.083 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:35 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:35.833+0000 7fd6a7b6b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:41:36.083 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:35 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:35.978+0000 7fd6a7b6b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:41:36.424 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:36 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:36.116+0000 7fd6a7b6b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:41:36.424 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:36 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:36.423+0000 7fd6a7b6b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:41:36.833 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:36 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:36.484+0000 7fd6a7b6b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:41:36.833 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:36 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:36.667+0000 7fd6a7b6b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:41:36.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:36 smithi174 ceph-mon[35637]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:36.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:36 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:36.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:36 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:36.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:36 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:36.834 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:36 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:41:36.834 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:36 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T06:41:36.834 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:36 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:41:36.834 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:36 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:36.834 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:36 smithi174 ceph-mon[35637]: Deploying daemon mgr.y on smithi134 2024-09-19T06:41:36.834 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:36 smithi174 ceph-mon[35637]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:36.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:36 smithi103 ceph-mon[31158]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:36.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:36 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:36.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:36 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:36.875 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:36 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:36.875 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:36 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:41:36.875 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:36 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T06:41:36.875 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:36 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:41:36.875 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:36 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:36.875 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:36 smithi103 ceph-mon[31158]: Deploying daemon mgr.y on smithi134 2024-09-19T06:41:36.875 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:36 smithi103 ceph-mon[31158]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:36.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:36 smithi134 ceph-mon[35356]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:36.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:36 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:36.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:36 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:36.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:36 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:36.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:36 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:41:36.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:36 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T06:41:36.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:36 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:41:36.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:36 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:36.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:36 smithi134 ceph-mon[35356]: Deploying daemon mgr.y on smithi134 2024-09-19T06:41:36.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:36 smithi134 ceph-mon[35356]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:37.625 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:37 smithi134 systemd[1]: Starting Ceph mgr.y for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T06:41:37.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:37 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3537774027' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d45dab3-e434-4783-b2e3-f658c3ba583b"}]: dispatch 2024-09-19T06:41:37.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:37 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3537774027' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d45dab3-e434-4783-b2e3-f658c3ba583b"}]': finished 2024-09-19T06:41:37.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:37 smithi174 ceph-mon[35637]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T06:41:37.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:37 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:41:37.833 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:37 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:37.548+0000 7fd6a7b6b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:41:37.833 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:37 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:37.616+0000 7fd6a7b6b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:41:37.910 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:37 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3537774027' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d45dab3-e434-4783-b2e3-f658c3ba583b"}]: dispatch 2024-09-19T06:41:37.911 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:37 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3537774027' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d45dab3-e434-4783-b2e3-f658c3ba583b"}]': finished 2024-09-19T06:41:37.911 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:37 smithi134 ceph-mon[35356]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T06:41:37.911 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:37 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:41:37.911 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:37 smithi134 podman[36704]: 2024-09-19 06:41:37.624636536 +0000 UTC m=+0.110328531 container create 534d031929765e28c32700d636af6887a042dde41f13a1c081dd0f43567b28e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:41:37.911 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:37 smithi134 podman[36704]: 2024-09-19 06:41:37.53005784 +0000 UTC m=+0.015749838 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:41:37.911 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:37 smithi134 podman[36704]: 2024-09-19 06:41:37.826992943 +0000 UTC m=+0.312684938 container init 534d031929765e28c32700d636af6887a042dde41f13a1c081dd0f43567b28e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T06:41:37.911 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:37 smithi134 podman[36704]: 2024-09-19 06:41:37.829815799 +0000 UTC m=+0.315507795 container start 534d031929765e28c32700d636af6887a042dde41f13a1c081dd0f43567b28e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T06:41:37.911 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:37 smithi134 ceph-mgr[36723]: -- 172.21.15.134:0/3746628735 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ad549b0e0 con 0x561ad5478800 2024-09-19T06:41:37.911 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:37 smithi134 bash[36704]: 534d031929765e28c32700d636af6887a042dde41f13a1c081dd0f43567b28e9 2024-09-19T06:41:37.957 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:37 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3537774027' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d45dab3-e434-4783-b2e3-f658c3ba583b"}]: dispatch 2024-09-19T06:41:37.958 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:37 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3537774027' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d45dab3-e434-4783-b2e3-f658c3ba583b"}]': finished 2024-09-19T06:41:37.958 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:37 smithi103 ceph-mon[31158]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T06:41:37.958 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:37 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:41:38.168 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:37 smithi134 systemd[1]: Started Ceph mgr.y for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:41:38.168 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:37 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:37.951+0000 7f7406478040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:41:38.168 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:38 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:38.012+0000 7f7406478040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:41:38.169 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:38 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:38.167+0000 7f7406478040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:41:38.437 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:38 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:38.236+0000 7f7406478040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:41:38.437 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:38 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:38.393+0000 7f7406478040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:41:38.638 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:38 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:38.355+0000 7fd6a7b6b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:41:38.639 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:38 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:38.492+0000 7fd6a7b6b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:41:38.639 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:38 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:38.637+0000 7fd6a7b6b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:41:38.813 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:38 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:38.549+0000 7f7406478040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:41:38.813 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:38 smithi134 ceph-mon[35356]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:38.813 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:38 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:38.814 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:38 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:38.814 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:38 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:38.814 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:38 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:38.814 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:38 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:38.814 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:38 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1396140125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:41:38.892 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:38 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:38.701+0000 7fd6a7b6b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:41:38.893 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:38 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:38.861+0000 7fd6a7b6b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:41:38.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:38 smithi174 ceph-mon[35637]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:38.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:38 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:38.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:38 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:38.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:38 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:38.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:38 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:38.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:38 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:38.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:38 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1396140125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:41:39.127 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:38 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:38.874+0000 7f7406478040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:41:39.128 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:38 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:38.936+0000 7f7406478040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:41:39.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:38 smithi103 ceph-mon[31158]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:39.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:38 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:39.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:38 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:39.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:38 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:39.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:38 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:39.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:38 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:39.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:38 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1396140125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:41:39.156 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:38 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:38.932+0000 7fd6a7b6b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:41:39.156 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:38 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:38.992+0000 7fd6a7b6b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:41:39.156 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:39.155+0000 7fd6a7b6b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:41:39.437 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:39 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:39.126+0000 7f7406478040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:41:39.444 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:39.228+0000 7fd6a7b6b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:41:39.444 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:39.296+0000 7fd6a7b6b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:41:39.444 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:39.381+0000 7fd6a7b6b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:41:39.444 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:39.443+0000 7fd6a7b6b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:41:39.833 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:39.502+0000 7fd6a7b6b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:41:40.420 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:40.049+0000 7f7406478040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:41:40.420 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:40.117+0000 7f7406478040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:41:40.420 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.420 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.420 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.420 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.420 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:40.420 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:40.421 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.421 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:40.421 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:40.421 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:40.421 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.421 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-mon[35356]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:40.421 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:40.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:40.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:40.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:40.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:40.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:40 smithi174 ceph-mon[35637]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:40.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.584 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:41:40 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:41:40.295+0000 7fd6a7b6b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:41:40.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:40.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:40.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:40.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:40.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:40.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:40.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:40 smithi103 ceph-mon[31158]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:40.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:41.089 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:40.832+0000 7f7406478040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:41:41.089 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:40 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:40.947+0000 7f7406478040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:41:41.090 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:41.088+0000 7f7406478040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:41:41.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:41.181+0000 7fe764ff9640 1 -- 172.21.15.103:0/2656008883 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 14) ==== 92409+0+0 (secure 0 0 0) 0x7fe75c03b460 con 0x7fe768101cd0 2024-09-19T06:41:41.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:41 smithi134 ceph-mon[35356]: Standby manager daemon z started 2024-09-19T06:41:41.367 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:41 smithi134 ceph-mon[35356]: from='mgr.? 172.21.15.174:0/756499360' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T06:41:41.367 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:41 smithi134 ceph-mon[35356]: from='mgr.? 172.21.15.174:0/756499360' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:41:41.367 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:41 smithi134 ceph-mon[35356]: from='mgr.? 172.21.15.174:0/756499360' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T06:41:41.367 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:41 smithi134 ceph-mon[35356]: from='mgr.? 172.21.15.174:0/756499360' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:41:41.367 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:41.149+0000 7f7406478040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:41:41.367 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:41.298+0000 7f7406478040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:41:41.367 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:41.365+0000 7f7406478040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:41:41.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:41 smithi174 ceph-mon[35637]: Standby manager daemon z started 2024-09-19T06:41:41.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:41 smithi174 ceph-mon[35637]: from='mgr.? 172.21.15.174:0/756499360' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T06:41:41.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:41 smithi174 ceph-mon[35637]: from='mgr.? 172.21.15.174:0/756499360' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:41:41.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:41 smithi174 ceph-mon[35637]: from='mgr.? 172.21.15.174:0/756499360' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T06:41:41.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:41 smithi174 ceph-mon[35637]: from='mgr.? 172.21.15.174:0/756499360' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:41:41.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:41 smithi103 ceph-mon[31158]: Standby manager daemon z started 2024-09-19T06:41:41.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:41 smithi103 ceph-mon[31158]: from='mgr.? 172.21.15.174:0/756499360' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T06:41:41.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:41 smithi103 ceph-mon[31158]: from='mgr.? 172.21.15.174:0/756499360' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:41:41.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:41 smithi103 ceph-mon[31158]: from='mgr.? 172.21.15.174:0/756499360' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T06:41:41.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:41 smithi103 ceph-mon[31158]: from='mgr.? 172.21.15.174:0/756499360' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:41:41.646 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:41.424+0000 7f7406478040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:41:41.646 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:41.581+0000 7f7406478040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:41:41.646 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:41.645+0000 7f7406478040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:41:41.909 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:41.710+0000 7f7406478040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:41:41.909 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:41.791+0000 7f7406478040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:41:41.909 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:41.850+0000 7f7406478040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:41:41.909 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:41.908+0000 7f7406478040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:41:42.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:42 smithi134 ceph-mon[35356]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-19T06:41:42.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:42 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T06:41:42.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:42 smithi134 ceph-mon[35356]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:42.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:42 smithi174 ceph-mon[35637]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-19T06:41:42.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:42 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T06:41:42.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:42 smithi174 ceph-mon[35637]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:42.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:42 smithi103 ceph-mon[31158]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-19T06:41:42.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:42 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T06:41:42.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:42 smithi103 ceph-mon[31158]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:42.937 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:41:42 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:41:42.684+0000 7f7406478040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:41:43.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:43.195+0000 7fe764ff9640 1 -- 172.21.15.103:0/2656008883 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7fe75c003cf0 con 0x7fe768101cd0 2024-09-19T06:41:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:43 smithi103 ceph-mon[31158]: Standby manager daemon y started 2024-09-19T06:41:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:43 smithi103 ceph-mon[31158]: from='mgr.? 172.21.15.134:0/403788772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T06:41:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:43 smithi103 ceph-mon[31158]: from='mgr.? 172.21.15.134:0/403788772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:41:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:43 smithi103 ceph-mon[31158]: from='mgr.? 172.21.15.134:0/403788772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T06:41:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:43 smithi103 ceph-mon[31158]: from='mgr.? 172.21.15.134:0/403788772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:41:43.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:43 smithi134 ceph-mon[35356]: Standby manager daemon y started 2024-09-19T06:41:43.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:43 smithi134 ceph-mon[35356]: from='mgr.? 172.21.15.134:0/403788772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T06:41:43.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:43 smithi134 ceph-mon[35356]: from='mgr.? 172.21.15.134:0/403788772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:41:43.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:43 smithi134 ceph-mon[35356]: from='mgr.? 172.21.15.134:0/403788772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T06:41:43.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:43 smithi134 ceph-mon[35356]: from='mgr.? 172.21.15.134:0/403788772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:41:43.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:43 smithi174 ceph-mon[35637]: Standby manager daemon y started 2024-09-19T06:41:43.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:43 smithi174 ceph-mon[35637]: from='mgr.? 172.21.15.134:0/403788772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T06:41:43.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:43 smithi174 ceph-mon[35637]: from='mgr.? 172.21.15.134:0/403788772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:41:43.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:43 smithi174 ceph-mon[35637]: from='mgr.? 172.21.15.134:0/403788772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T06:41:43.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:43 smithi174 ceph-mon[35637]: from='mgr.? 172.21.15.134:0/403788772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:41:44.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:44 smithi103 ceph-mon[31158]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-19T06:41:44.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T06:41:44.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:44 smithi103 ceph-mon[31158]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:44.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:44 smithi174 ceph-mon[35637]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-19T06:41:44.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T06:41:44.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:44 smithi174 ceph-mon[35637]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:44.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:44 smithi134 ceph-mon[35356]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-19T06:41:44.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T06:41:44.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:44 smithi134 ceph-mon[35356]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:46.902 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:46 smithi103 ceph-mon[31158]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:46.902 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:46 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T06:41:46.902 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:46 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:47.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:46 smithi174 ceph-mon[35637]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:47.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:46 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T06:41:47.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:46 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:47.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:46 smithi134 ceph-mon[35356]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:47.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:46 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T06:41:47.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:46 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:47.996 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:47 smithi103 ceph-mon[31158]: Deploying daemon osd.0 on smithi103 2024-09-19T06:41:48.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:47 smithi174 ceph-mon[35637]: Deploying daemon osd.0 on smithi103 2024-09-19T06:41:48.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:47 smithi134 ceph-mon[35356]: Deploying daemon osd.0 on smithi103 2024-09-19T06:41:49.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:48 smithi174 ceph-mon[35637]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:49.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:48 smithi103 ceph-mon[31158]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:49.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:48 smithi134 ceph-mon[35356]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:51.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:50 smithi174 ceph-mon[35637]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:51.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:50 smithi103 ceph-mon[31158]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:51.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:50 smithi134 ceph-mon[35356]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:53.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:52 smithi174 ceph-mon[35637]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:53.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:52 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:53.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:52 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:53.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:52 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:53.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:52 smithi103 ceph-mon[31158]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:53.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:52 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:53.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:52 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:53.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:52 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:53.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:52 smithi134 ceph-mon[35356]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:53.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:52 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:53.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:52 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:53.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:52 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:55.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:54 smithi174 ceph-mon[35637]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:55.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:54 smithi103 ceph-mon[31158]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:55.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:54 smithi134 ceph-mon[35356]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:56.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:56.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:56.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:56.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:56.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:56.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:55 smithi103 ceph-mon[31158]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:56.293 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 0 on host 'smithi103' 2024-09-19T06:41:56.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:56.291+0000 7fe764ff9640 1 -- 172.21.15.103:0/2656008883 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe768108be0 con 0x7fe750039500 2024-09-19T06:41:56.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:56.293+0000 7fe76e103640 1 -- 172.21.15.103:0/2656008883 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe750039500 msgr2=0x7fe75003b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:56.293+0000 7fe76e103640 1 --2- 172.21.15.103:0/2656008883 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe750039500 0x7fe75003b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe768101b30 tx=0x7fe758009500 comp rx=0 tx=0).stop 2024-09-19T06:41:56.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:56.293+0000 7fe76e103640 1 -- 172.21.15.103:0/2656008883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe768101cd0 msgr2=0x7fe76819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:56.293+0000 7fe76e103640 1 --2- 172.21.15.103:0/2656008883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe768101cd0 0x7fe76819a6a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009230 tx=0x7fe75c031b60 comp rx=0 tx=0).stop 2024-09-19T06:41:56.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:56.293+0000 7fe76e103640 1 -- 172.21.15.103:0/2656008883 shutdown_connections 2024-09-19T06:41:56.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:56.293+0000 7fe76e103640 1 --2- 172.21.15.103:0/2656008883 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe750039500 0x7fe75003b9c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:56.293+0000 7fe76e103640 1 --2- 172.21.15.103:0/2656008883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe768102690 0x7fe7681a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:56.293+0000 7fe76e103640 1 --2- 172.21.15.103:0/2656008883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe768101cd0 0x7fe76819a6a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:56.293+0000 7fe76e103640 1 --2- 172.21.15.103:0/2656008883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe768100ad0 0x7fe76819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:56.294+0000 7fe76e103640 1 -- 172.21.15.103:0/2656008883 >> 172.21.15.103:0/2656008883 conn(0x7fe7680fc280 msgr2=0x7fe7681074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:56.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:56.294+0000 7fe76e103640 1 -- 172.21.15.103:0/2656008883 shutdown_connections 2024-09-19T06:41:56.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:41:56.294+0000 7fe76e103640 1 -- 172.21.15.103:0/2656008883 wait complete. 2024-09-19T06:41:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:55 smithi174 ceph-mon[35637]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:56.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:56.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:55 smithi134 ceph-mon[35356]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:56.927 DEBUG:teuthology.orchestra.run.smithi103:osd.0> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T06:41:56.930 INFO:tasks.cephadm:Deploying osd.1 on smithi134 with /dev/vg_nvme/lv_4... 2024-09-19T06:41:56.930 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T06:41:57.151 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:57.151 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:57.151 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:57.185 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:41:57.306 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:56 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:57.306 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:56 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:57.306 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:56 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:57.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:56 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:57.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:56 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:57.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:56 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:41:58.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:57 smithi174 ceph-mon[35637]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:58.232 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:57 smithi103 ceph-mon[31158]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:58.322 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:57 smithi134 ceph-mon[35356]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:41:58.888 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 06:41:58 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-0[44957]: 2024-09-19T06:41:58.695+0000 7f70269e5640 -1 osd.0 0 log_to_monitors true 2024-09-19T06:41:59.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:58 smithi134 ceph-mon[35356]: from='osd.0 [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T06:41:59.270 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:58 smithi103 ceph-mon[31158]: from='osd.0 [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T06:41:59.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:58 smithi174 ceph-mon[35637]: from='osd.0 [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T06:41:59.630 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:41:59.665 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_4 2024-09-19T06:41:59.916 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:42:00.237 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 06:41:59 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-0[44957]: 2024-09-19T06:41:59.986+0000 7f7022145640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T06:42:00.237 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 06:41:59 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-0[44957]: 2024-09-19T06:41:59.990+0000 7f701e77f640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:42:00.237 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:59 smithi103 ceph-mon[31158]: from='osd.0 [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T06:42:00.237 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:59 smithi103 ceph-mon[31158]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T06:42:00.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:59 smithi103 ceph-mon[31158]: from='osd.0 [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T06:42:00.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:59 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:42:00.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:59 smithi103 ceph-mon[31158]: Detected new or changed devices on smithi103 2024-09-19T06:42:00.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:59 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:00.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:59 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:00.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:59 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:42:00.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:59 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-19T06:42:00.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:59 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:00.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:59 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:00.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:59 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:00.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:59 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:00.239 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:41:59 smithi103 ceph-mon[31158]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:42:00.239 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:59 smithi134 ceph-mon[35356]: from='osd.0 [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T06:42:00.239 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:59 smithi134 ceph-mon[35356]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T06:42:00.239 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:59 smithi134 ceph-mon[35356]: from='osd.0 [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T06:42:00.239 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:59 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:42:00.239 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:59 smithi134 ceph-mon[35356]: Detected new or changed devices on smithi103 2024-09-19T06:42:00.239 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:59 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:00.239 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:59 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:00.240 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:59 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:42:00.240 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:59 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-19T06:42:00.240 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:59 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:00.240 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:59 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:00.240 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:59 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:00.240 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:59 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:00.240 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:41:59 smithi134 ceph-mon[35356]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:42:00.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:59 smithi174 ceph-mon[35637]: from='osd.0 [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T06:42:00.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:59 smithi174 ceph-mon[35637]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T06:42:00.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:59 smithi174 ceph-mon[35637]: from='osd.0 [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T06:42:00.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:59 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:42:00.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:59 smithi174 ceph-mon[35637]: Detected new or changed devices on smithi103 2024-09-19T06:42:00.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:59 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:00.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:59 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:00.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:59 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:42:00.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:59 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-19T06:42:00.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:59 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:00.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:59 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:00.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:59 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:00.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:59 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:00.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:41:59 smithi174 ceph-mon[35637]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:42:00.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.321+0000 7f4d5bc32640 1 -- 172.21.15.134:0/4032157743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d54107920 msgr2=0x7f4d54109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:00.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.321+0000 7f4d5bc32640 1 --2- 172.21.15.134:0/4032157743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d54107920 0x7f4d54109d10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009ca0 tx=0x7f4d4c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:42:00.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.321+0000 7f4d5bc32640 1 -- 172.21.15.134:0/4032157743 shutdown_connections 2024-09-19T06:42:00.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.321+0000 7f4d5bc32640 1 --2- 172.21.15.134:0/4032157743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d54107920 0x7f4d54109d10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:00.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.321+0000 7f4d5bc32640 1 --2- 172.21.15.134:0/4032157743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54104ff0 0x7f4d541073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:00.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.321+0000 7f4d5bc32640 1 --2- 172.21.15.134:0/4032157743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d541026c0 0x7f4d54104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:00.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.321+0000 7f4d5bc32640 1 -- 172.21.15.134:0/4032157743 >> 172.21.15.134:0/4032157743 conn(0x7f4d540fc280 msgr2=0x7f4d540fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:00.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.321+0000 7f4d5bc32640 1 -- 172.21.15.134:0/4032157743 shutdown_connections 2024-09-19T06:42:00.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.321+0000 7f4d5bc32640 1 -- 172.21.15.134:0/4032157743 wait complete. 2024-09-19T06:42:00.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d5bc32640 1 Processor -- start 2024-09-19T06:42:00.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d5bc32640 1 -- start start 2024-09-19T06:42:00.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d5bc32640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d541026c0 0x7f4d54197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:00.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d5bc32640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d54104ff0 0x7f4d54198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:00.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d5bc32640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54107920 0x7f4d5419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:00.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d5bc32640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5410d8d0 con 0x7f4d54104ff0 2024-09-19T06:42:00.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d5bc32640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5410d750 con 0x7f4d541026c0 2024-09-19T06:42:00.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d5bc32640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5410da50 con 0x7f4d54107920 2024-09-19T06:42:00.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d599ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d541026c0 0x7f4d54197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:00.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d599ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d541026c0 0x7f4d54197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:55514/0 (socket says 172.21.15.134:55514) 2024-09-19T06:42:00.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d599ce640 1 -- 172.21.15.134:0/491890676 learned_addr learned my addr 172.21.15.134:0/491890676 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-19T06:42:00.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d591cd640 1 --2- 172.21.15.134:0/491890676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d54104ff0 0x7f4d54198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:00.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d5a1cf640 1 --2- 172.21.15.134:0/491890676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54107920 0x7f4d5419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:00.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d599ce640 1 -- 172.21.15.134:0/491890676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54107920 msgr2=0x7f4d5419f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:00.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d599ce640 1 --2- 172.21.15.134:0/491890676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54107920 0x7f4d5419f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:00.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d599ce640 1 -- 172.21.15.134:0/491890676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d54104ff0 msgr2=0x7f4d54198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:00.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d599ce640 1 --2- 172.21.15.134:0/491890676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d54104ff0 0x7f4d54198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:00.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d599ce640 1 -- 172.21.15.134:0/491890676 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5419fb00 con 0x7f4d541026c0 2024-09-19T06:42:00.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.322+0000 7f4d591cd640 1 --2- 172.21.15.134:0/491890676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d54104ff0 0x7f4d54198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:42:00.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.323+0000 7f4d599ce640 1 --2- 172.21.15.134:0/491890676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d541026c0 0x7f4d54197ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400d950 tx=0x7f4d4400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:00.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.323+0000 7f4d4affd640 1 -- 172.21.15.134:0/491890676 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44017070 con 0x7f4d541026c0 2024-09-19T06:42:00.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.323+0000 7f4d4affd640 1 -- 172.21.15.134:0/491890676 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d44009c30 con 0x7f4d541026c0 2024-09-19T06:42:00.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.323+0000 7f4d5bc32640 1 -- 172.21.15.134:0/491890676 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5419fdf0 con 0x7f4d541026c0 2024-09-19T06:42:00.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.323+0000 7f4d4affd640 1 -- 172.21.15.134:0/491890676 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4400f6b0 con 0x7f4d541026c0 2024-09-19T06:42:00.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.323+0000 7f4d5bc32640 1 -- 172.21.15.134:0/491890676 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d541a0240 con 0x7f4d541026c0 2024-09-19T06:42:00.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.324+0000 7f4d5bc32640 1 -- 172.21.15.134:0/491890676 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d540fdd70 con 0x7f4d541026c0 2024-09-19T06:42:00.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.325+0000 7f4d4affd640 1 -- 172.21.15.134:0/491890676 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f4d440109d0 con 0x7f4d541026c0 2024-09-19T06:42:00.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.326+0000 7f4d4affd640 1 --2- 172.21.15.134:0/491890676 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4d400a31e0 0x7f4d400a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:00.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.326+0000 7f4d4affd640 1 -- 172.21.15.134:0/491890676 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f4d440cda30 con 0x7f4d541026c0 2024-09-19T06:42:00.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.326+0000 7f4d591cd640 1 --2- 172.21.15.134:0/491890676 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4d400a31e0 0x7f4d400a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:00.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.327+0000 7f4d591cd640 1 --2- 172.21.15.134:0/491890676 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4d400a31e0 0x7f4d400a56a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4d541993f0 tx=0x7f4d50005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:00.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.331+0000 7f4d4affd640 1 -- 172.21.15.134:0/491890676 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4d44009de0 con 0x7f4d541026c0 2024-09-19T06:42:00.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:00.511+0000 7f4d5bc32640 1 -- 172.21.15.134:0/491890676 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4d54062630 con 0x7f4d400a31e0 2024-09-19T06:42:01.169 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:00 smithi134 ceph-mon[35356]: from='osd.0 [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T06:42:01.170 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:00 smithi134 ceph-mon[35356]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T06:42:01.170 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:00 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:42:01.170 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:00 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:42:01.170 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:00 smithi134 ceph-mon[35356]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:42:01.170 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:00 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:42:01.170 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:00 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:42:01.170 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:00 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:01.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:00 smithi174 ceph-mon[35637]: from='osd.0 [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T06:42:01.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:00 smithi174 ceph-mon[35637]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T06:42:01.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:00 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:42:01.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:00 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:42:01.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:00 smithi174 ceph-mon[35637]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:42:01.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:00 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:42:01.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:00 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:42:01.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:00 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:01.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:00 smithi103 ceph-mon[31158]: from='osd.0 [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T06:42:01.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:00 smithi103 ceph-mon[31158]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T06:42:01.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:42:01.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:42:01.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:00 smithi103 ceph-mon[31158]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:42:01.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:42:01.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:42:01.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:02.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:01 smithi134 ceph-mon[35356]: purged_snaps scrub starts 2024-09-19T06:42:02.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:01 smithi134 ceph-mon[35356]: purged_snaps scrub ok 2024-09-19T06:42:02.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:01 smithi134 ceph-mon[35356]: osd.0 [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740] boot 2024-09-19T06:42:02.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:01 smithi134 ceph-mon[35356]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T06:42:02.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:01 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:42:02.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:01 smithi134 ceph-mon[35356]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:42:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:01 smithi174 ceph-mon[35637]: purged_snaps scrub starts 2024-09-19T06:42:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:01 smithi174 ceph-mon[35637]: purged_snaps scrub ok 2024-09-19T06:42:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:01 smithi174 ceph-mon[35637]: osd.0 [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740] boot 2024-09-19T06:42:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:01 smithi174 ceph-mon[35637]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T06:42:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:01 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:42:02.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:01 smithi174 ceph-mon[35637]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:42:02.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:01 smithi103 ceph-mon[31158]: purged_snaps scrub starts 2024-09-19T06:42:02.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:01 smithi103 ceph-mon[31158]: purged_snaps scrub ok 2024-09-19T06:42:02.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:01 smithi103 ceph-mon[31158]: osd.0 [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740] boot 2024-09-19T06:42:02.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:01 smithi103 ceph-mon[31158]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T06:42:02.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:01 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:42:02.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:01 smithi103 ceph-mon[31158]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:42:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:02 smithi174 ceph-mon[35637]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T06:42:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:02 smithi174 ceph-mon[35637]: from='client.? 172.21.15.134:0/514499462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee0e92bd-15ab-4867-9a18-20abb9f5fc86"}]: dispatch 2024-09-19T06:42:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:02 smithi174 ceph-mon[35637]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee0e92bd-15ab-4867-9a18-20abb9f5fc86"}]: dispatch 2024-09-19T06:42:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:02 smithi174 ceph-mon[35637]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee0e92bd-15ab-4867-9a18-20abb9f5fc86"}]': finished 2024-09-19T06:42:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:02 smithi174 ceph-mon[35637]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T06:42:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:02 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:03.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:02 smithi134 ceph-mon[35356]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T06:42:03.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:02 smithi134 ceph-mon[35356]: from='client.? 172.21.15.134:0/514499462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee0e92bd-15ab-4867-9a18-20abb9f5fc86"}]: dispatch 2024-09-19T06:42:03.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:02 smithi134 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee0e92bd-15ab-4867-9a18-20abb9f5fc86"}]: dispatch 2024-09-19T06:42:03.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:02 smithi134 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee0e92bd-15ab-4867-9a18-20abb9f5fc86"}]': finished 2024-09-19T06:42:03.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:02 smithi134 ceph-mon[35356]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T06:42:03.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:02 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:03.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:02 smithi103 ceph-mon[31158]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T06:42:03.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:02 smithi103 ceph-mon[31158]: from='client.? 172.21.15.134:0/514499462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee0e92bd-15ab-4867-9a18-20abb9f5fc86"}]: dispatch 2024-09-19T06:42:03.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:02 smithi103 ceph-mon[31158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee0e92bd-15ab-4867-9a18-20abb9f5fc86"}]: dispatch 2024-09-19T06:42:03.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:02 smithi103 ceph-mon[31158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee0e92bd-15ab-4867-9a18-20abb9f5fc86"}]': finished 2024-09-19T06:42:03.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:02 smithi103 ceph-mon[31158]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T06:42:03.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:02 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:04.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:04 smithi174 ceph-mon[35637]: from='client.? 172.21.15.134:0/4059841770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:42:04.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:04 smithi174 ceph-mon[35637]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:04.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:04 smithi103 ceph-mon[31158]: from='client.? 172.21.15.134:0/4059841770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:42:04.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:04 smithi103 ceph-mon[31158]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:04.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:04 smithi134 ceph-mon[35356]: from='client.? 172.21.15.134:0/4059841770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:42:04.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:04 smithi134 ceph-mon[35356]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:07.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:06 smithi174 ceph-mon[35637]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:07.090 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:06 smithi103 ceph-mon[31158]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:07.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:06 smithi134 ceph-mon[35356]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:09.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:08 smithi174 ceph-mon[35637]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:09.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:08 smithi103 ceph-mon[31158]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:09.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:08 smithi134 ceph-mon[35356]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:11.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:10 smithi174 ceph-mon[35637]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:11.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:10 smithi103 ceph-mon[31158]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:11.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:10 smithi134 ceph-mon[35356]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:11.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:11 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T06:42:11.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:11 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:12.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T06:42:12.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:12.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:11 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T06:42:12.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:11 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:13.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:12 smithi174 ceph-mon[35637]: Deploying daemon osd.1 on smithi134 2024-09-19T06:42:13.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:12 smithi174 ceph-mon[35637]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:13.117 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:12 smithi134 ceph-mon[35356]: Deploying daemon osd.1 on smithi134 2024-09-19T06:42:13.117 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:12 smithi134 ceph-mon[35356]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:13.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:12 smithi103 ceph-mon[31158]: Deploying daemon osd.1 on smithi134 2024-09-19T06:42:13.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:12 smithi103 ceph-mon[31158]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:15.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:14 smithi174 ceph-mon[35637]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:15.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:14 smithi103 ceph-mon[31158]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:15.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:14 smithi134 ceph-mon[35356]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:16.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:16 smithi134 ceph-mon[35356]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:17.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:16 smithi174 ceph-mon[35637]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:16 smithi103 ceph-mon[31158]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:18.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:17 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:42:18.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:17 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:18.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:17 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:18.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:17 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:42:18.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:17 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:18.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:17 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:17 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:42:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:17 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:17 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:19.050 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:18 smithi134 ceph-mon[35356]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:19.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:18 smithi174 ceph-mon[35637]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:19.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:18 smithi103 ceph-mon[31158]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:20 smithi103 ceph-mon[31158]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:20 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:20 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:20 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:20 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:20 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:21.166 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:20 smithi134 ceph-mon[35356]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:21.166 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:20 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:21.166 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:20 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:21.167 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:20 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:21.167 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:20 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:21.167 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:20 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:21.204 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 1 on host 'smithi134' 2024-09-19T06:42:21.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:21.203+0000 7f4d4affd640 1 -- 172.21.15.134:0/491890676 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4d54062630 con 0x7f4d400a31e0 2024-09-19T06:42:21.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:21.206+0000 7f4d5bc32640 1 -- 172.21.15.134:0/491890676 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4d400a31e0 msgr2=0x7f4d400a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:21.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:21.206+0000 7f4d5bc32640 1 --2- 172.21.15.134:0/491890676 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4d400a31e0 0x7f4d400a56a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4d541993f0 tx=0x7f4d50005d30 comp rx=0 tx=0).stop 2024-09-19T06:42:21.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:21.206+0000 7f4d5bc32640 1 -- 172.21.15.134:0/491890676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d541026c0 msgr2=0x7f4d54197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:21.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:21.206+0000 7f4d5bc32640 1 --2- 172.21.15.134:0/491890676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d541026c0 0x7f4d54197ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400d950 tx=0x7f4d4400de10 comp rx=0 tx=0).stop 2024-09-19T06:42:21.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:21.206+0000 7f4d5bc32640 1 -- 172.21.15.134:0/491890676 shutdown_connections 2024-09-19T06:42:21.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:21.206+0000 7f4d5bc32640 1 --2- 172.21.15.134:0/491890676 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4d400a31e0 0x7f4d400a56a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:21.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:21.206+0000 7f4d5bc32640 1 --2- 172.21.15.134:0/491890676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54107920 0x7f4d5419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:21.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:21.206+0000 7f4d5bc32640 1 --2- 172.21.15.134:0/491890676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d54104ff0 0x7f4d54198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:21.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:21.206+0000 7f4d5bc32640 1 --2- 172.21.15.134:0/491890676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d541026c0 0x7f4d54197ed0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:21.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:21.206+0000 7f4d5bc32640 1 -- 172.21.15.134:0/491890676 >> 172.21.15.134:0/491890676 conn(0x7f4d540fc280 msgr2=0x7f4d54102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:21.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:21.206+0000 7f4d5bc32640 1 -- 172.21.15.134:0/491890676 shutdown_connections 2024-09-19T06:42:21.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:42:21.206+0000 7f4d5bc32640 1 -- 172.21.15.134:0/491890676 wait complete. 2024-09-19T06:42:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:20 smithi174 ceph-mon[35637]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:20 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:20 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:20 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:20 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:20 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:21.787 DEBUG:teuthology.orchestra.run.smithi134:osd.1> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T06:42:21.789 INFO:tasks.cephadm:Deploying osd.2 on smithi174 with /dev/vg_nvme/lv_4... 2024-09-19T06:42:21.790 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T06:42:22.044 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:42:22.073 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:21 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:42:22.073 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:21 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:22.073 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:21 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:22.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:21 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:42:22.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:21 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:22.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:21 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:22.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:21 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:42:22.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:21 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:22.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:21 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:22.891 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:22 smithi134 ceph-mon[35356]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:23.112 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:22 smithi174 ceph-mon[35637]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:23.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:22 smithi103 ceph-mon[31158]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:24.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:23 smithi174 ceph-mon[35637]: from='osd.1 [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T06:42:24.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:23 smithi174 ceph-mon[35637]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T06:42:24.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:23 smithi103 ceph-mon[31158]: from='osd.1 [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T06:42:24.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:23 smithi103 ceph-mon[31158]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T06:42:24.156 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:23 smithi134 ceph-mon[35356]: from='osd.1 [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T06:42:24.156 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:23 smithi134 ceph-mon[35356]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T06:42:24.156 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 06:42:23 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-1[39741]: 2024-09-19T06:42:23.820+0000 7f9936755640 -1 osd.1 0 log_to_monitors true 2024-09-19T06:42:24.423 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:42:24.449 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_4 2024-09-19T06:42:24.704 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:42:25.065 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:24 smithi174 ceph-mon[35637]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:25.065 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:24 smithi174 ceph-mon[35637]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T06:42:25.066 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:24 smithi174 ceph-mon[35637]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T06:42:25.066 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:24 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:25.066 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:24 smithi174 ceph-mon[35637]: from='osd.1 [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:42:25.066 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:24 smithi174 ceph-mon[35637]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:42:25.066 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:24 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:25.066 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:24 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:25.066 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:24 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:42:25.066 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:24 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:25.066 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:24 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:25.066 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:24 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:25.066 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:24 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:24 smithi103 ceph-mon[31158]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:24 smithi103 ceph-mon[31158]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T06:42:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:24 smithi103 ceph-mon[31158]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T06:42:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:24 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:24 smithi103 ceph-mon[31158]: from='osd.1 [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:42:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:24 smithi103 ceph-mon[31158]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:42:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:24 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:24 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:24 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:42:25.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:24 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:25.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:24 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:25.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:24 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:25.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:24 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:25.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.149+0000 7f38b01a2640 1 -- 172.21.15.174:0/2841409647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a8100480 msgr2=0x7f38a8102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:25.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.149+0000 7f38b01a2640 1 --2- 172.21.15.174:0/2841409647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a8100480 0x7f38a8102890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009900 tx=0x7f38a402a410 comp rx=0 tx=0).stop 2024-09-19T06:42:25.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.149+0000 7f38b01a2640 1 -- 172.21.15.174:0/2841409647 shutdown_connections 2024-09-19T06:42:25.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.149+0000 7f38b01a2640 1 --2- 172.21.15.174:0/2841409647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38a8105780 0x7f38a8105c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:25.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.149+0000 7f38b01a2640 1 --2- 172.21.15.174:0/2841409647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38a8102dd0 0x7f38a8105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:25.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.149+0000 7f38b01a2640 1 --2- 172.21.15.174:0/2841409647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a8100480 0x7f38a8102890 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:25.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.149+0000 7f38b01a2640 1 -- 172.21.15.174:0/2841409647 >> 172.21.15.174:0/2841409647 conn(0x7f38a80fc290 msgr2=0x7f38a80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:25.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.150+0000 7f38b01a2640 1 -- 172.21.15.174:0/2841409647 shutdown_connections 2024-09-19T06:42:25.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.150+0000 7f38b01a2640 1 -- 172.21.15.174:0/2841409647 wait complete. 2024-09-19T06:42:25.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.150+0000 7f38b01a2640 1 Processor -- start 2024-09-19T06:42:25.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.150+0000 7f38b01a2640 1 -- start start 2024-09-19T06:42:25.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.150+0000 7f38b01a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a8100480 0x7f38a819e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:25.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.150+0000 7f38b01a2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38a8102dd0 0x7f38a819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:25.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.150+0000 7f38b01a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38a8105780 0x7f38a81a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:25.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.150+0000 7f38b01a2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8113e60 con 0x7f38a8105780 2024-09-19T06:42:25.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.150+0000 7f38b01a2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8113ce0 con 0x7f38a8102dd0 2024-09-19T06:42:25.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.150+0000 7f38b01a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8113fe0 con 0x7f38a8100480 2024-09-19T06:42:25.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.150+0000 7f38adf3e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a8100480 0x7f38a819e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:25.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.150+0000 7f38adf3e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a8100480 0x7f38a819e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:60606/0 (socket says 172.21.15.174:60606) 2024-09-19T06:42:25.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.150+0000 7f38adf3e640 1 -- 172.21.15.174:0/544295643 learned_addr learned my addr 172.21.15.174:0/544295643 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T06:42:25.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.150+0000 7f38ad73d640 1 --2- 172.21.15.174:0/544295643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38a8102dd0 0x7f38a819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:25.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.151+0000 7f38ae73f640 1 --2- 172.21.15.174:0/544295643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38a8105780 0x7f38a81a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:25.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.151+0000 7f38adf3e640 1 -- 172.21.15.174:0/544295643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38a8102dd0 msgr2=0x7f38a819ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:25.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.151+0000 7f38adf3e640 1 --2- 172.21.15.174:0/544295643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38a8102dd0 0x7f38a819ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:25.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.151+0000 7f38adf3e640 1 -- 172.21.15.174:0/544295643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38a8105780 msgr2=0x7f38a81a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:25.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.151+0000 7f38adf3e640 1 --2- 172.21.15.174:0/544295643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38a8105780 0x7f38a81a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:25.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.151+0000 7f38adf3e640 1 -- 172.21.15.174:0/544295643 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a81a6120 con 0x7f38a8100480 2024-09-19T06:42:25.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.151+0000 7f38ae73f640 1 --2- 172.21.15.174:0/544295643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38a8105780 0x7f38a81a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:25.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.151+0000 7f38adf3e640 1 --2- 172.21.15.174:0/544295643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a8100480 0x7f38a819e4f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009230 tx=0x7f38a4004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:25.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.151+0000 7f389effd640 1 -- 172.21.15.174:0/544295643 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a403f030 con 0x7f38a8100480 2024-09-19T06:42:25.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.151+0000 7f389effd640 1 -- 172.21.15.174:0/544295643 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38a4003ce0 con 0x7f38a8100480 2024-09-19T06:42:25.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.151+0000 7f38b01a2640 1 -- 172.21.15.174:0/544295643 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a81a63b0 con 0x7f38a8100480 2024-09-19T06:42:25.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.151+0000 7f389effd640 1 -- 172.21.15.174:0/544295643 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4035070 con 0x7f38a8100480 2024-09-19T06:42:25.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.151+0000 7f38b01a2640 1 -- 172.21.15.174:0/544295643 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a81a6890 con 0x7f38a8100480 2024-09-19T06:42:25.156 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.152+0000 7f38b01a2640 1 -- 172.21.15.174:0/544295643 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a80fdd80 con 0x7f38a8100480 2024-09-19T06:42:25.156 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.154+0000 7f389effd640 1 -- 172.21.15.174:0/544295643 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f38a4042030 con 0x7f38a8100480 2024-09-19T06:42:25.156 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.154+0000 7f389effd640 1 --2- 172.21.15.174:0/544295643 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f38940a31e0 0x7f38940a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:25.156 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.154+0000 7f389effd640 1 -- 172.21.15.174:0/544295643 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f38a40ee8d0 con 0x7f38a8100480 2024-09-19T06:42:25.156 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.154+0000 7f38ad73d640 1 --2- 172.21.15.174:0/544295643 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f38940a31e0 0x7f38940a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:25.156 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.155+0000 7f38ad73d640 1 --2- 172.21.15.174:0/544295643 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f38940a31e0 0x7f38940a56a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38a819fa10 tx=0x7f389800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:25.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.159+0000 7f389effd640 1 -- 172.21.15.174:0/544295643 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f38a40bcf60 con 0x7f38a8100480 2024-09-19T06:42:25.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:24 smithi134 ceph-mon[35356]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:25.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:24 smithi134 ceph-mon[35356]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T06:42:25.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:24 smithi134 ceph-mon[35356]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T06:42:25.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:24 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:25.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:24 smithi134 ceph-mon[35356]: from='osd.1 [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:42:25.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:24 smithi134 ceph-mon[35356]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:42:25.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:24 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:25.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:24 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:25.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:24 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:42:25.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:24 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:25.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:24 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:25.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:24 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:25.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:24 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:25.339 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:25.338+0000 7f38b01a2640 1 -- 172.21.15.174:0/544295643 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f38a8062630 con 0x7f38940a31e0 2024-09-19T06:42:25.937 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 06:42:25 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-1[39741]: 2024-09-19T06:42:25.616+0000 7f9931eb5640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T06:42:25.937 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 06:42:25 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-1[39741]: 2024-09-19T06:42:25.621+0000 7f992a4e7640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:42:25.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:25 smithi134 ceph-mon[35356]: Detected new or changed devices on smithi134 2024-09-19T06:42:25.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:25 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi134 to 17088M 2024-09-19T06:42:25.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:25 smithi134 ceph-mon[35356]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T06:42:25.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:25 smithi134 ceph-mon[35356]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T06:42:25.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:25 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:25.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:25 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:25.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:25 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:42:25.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:25 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:42:25.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:25 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:25.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:25 smithi134 ceph-mon[35356]: from='osd.1 ' entity='osd.1' 2024-09-19T06:42:26.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:25 smithi174 ceph-mon[35637]: Detected new or changed devices on smithi134 2024-09-19T06:42:26.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:25 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi134 to 17088M 2024-09-19T06:42:26.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:25 smithi174 ceph-mon[35637]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T06:42:26.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:25 smithi174 ceph-mon[35637]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T06:42:26.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:26.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:26.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:42:26.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:42:26.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:26.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:25 smithi174 ceph-mon[35637]: from='osd.1 ' entity='osd.1' 2024-09-19T06:42:26.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:25 smithi103 ceph-mon[31158]: Detected new or changed devices on smithi134 2024-09-19T06:42:26.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:25 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi134 to 17088M 2024-09-19T06:42:26.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:25 smithi103 ceph-mon[31158]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T06:42:26.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:25 smithi103 ceph-mon[31158]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T06:42:26.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:26.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:26.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:42:26.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:42:26.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:26.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:25 smithi103 ceph-mon[31158]: from='osd.1 ' entity='osd.1' 2024-09-19T06:42:26.918 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:26 smithi174 ceph-mon[35637]: purged_snaps scrub starts 2024-09-19T06:42:26.918 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:26 smithi174 ceph-mon[35637]: purged_snaps scrub ok 2024-09-19T06:42:26.918 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:26 smithi174 ceph-mon[35637]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:42:26.919 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:26 smithi174 ceph-mon[35637]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:26.919 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:26 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:26.919 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:26 smithi174 ceph-mon[35637]: osd.1 [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195] boot 2024-09-19T06:42:26.919 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:26 smithi174 ceph-mon[35637]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T06:42:26.919 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:26 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:27.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:26 smithi103 ceph-mon[31158]: purged_snaps scrub starts 2024-09-19T06:42:27.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:26 smithi103 ceph-mon[31158]: purged_snaps scrub ok 2024-09-19T06:42:27.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:26 smithi103 ceph-mon[31158]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:42:27.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:26 smithi103 ceph-mon[31158]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:27.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:26 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:27.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:26 smithi103 ceph-mon[31158]: osd.1 [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195] boot 2024-09-19T06:42:27.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:26 smithi103 ceph-mon[31158]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T06:42:27.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:26 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:26 smithi134 ceph-mon[35356]: purged_snaps scrub starts 2024-09-19T06:42:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:26 smithi134 ceph-mon[35356]: purged_snaps scrub ok 2024-09-19T06:42:27.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:26 smithi134 ceph-mon[35356]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:42:27.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:26 smithi134 ceph-mon[35356]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:42:27.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:26 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:27.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:26 smithi134 ceph-mon[35356]: osd.1 [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195] boot 2024-09-19T06:42:27.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:26 smithi134 ceph-mon[35356]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T06:42:27.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:26 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:42:28.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:27 smithi174 ceph-mon[35637]: from='client.? 172.21.15.174:0/801036864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0230caed-025b-46e9-a011-237601d16ac5"}]: dispatch 2024-09-19T06:42:28.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:27 smithi174 ceph-mon[35637]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0230caed-025b-46e9-a011-237601d16ac5"}]: dispatch 2024-09-19T06:42:28.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:27 smithi174 ceph-mon[35637]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0230caed-025b-46e9-a011-237601d16ac5"}]': finished 2024-09-19T06:42:28.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:27 smithi174 ceph-mon[35637]: osdmap e14: 3 total, 2 up, 3 in 2024-09-19T06:42:28.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:27 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:28.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:27 smithi103 ceph-mon[31158]: from='client.? 172.21.15.174:0/801036864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0230caed-025b-46e9-a011-237601d16ac5"}]: dispatch 2024-09-19T06:42:28.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:27 smithi103 ceph-mon[31158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0230caed-025b-46e9-a011-237601d16ac5"}]: dispatch 2024-09-19T06:42:28.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:27 smithi103 ceph-mon[31158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0230caed-025b-46e9-a011-237601d16ac5"}]': finished 2024-09-19T06:42:28.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:27 smithi103 ceph-mon[31158]: osdmap e14: 3 total, 2 up, 3 in 2024-09-19T06:42:28.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:27 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:28.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:27 smithi134 ceph-mon[35356]: from='client.? 172.21.15.174:0/801036864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0230caed-025b-46e9-a011-237601d16ac5"}]: dispatch 2024-09-19T06:42:28.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:27 smithi134 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0230caed-025b-46e9-a011-237601d16ac5"}]: dispatch 2024-09-19T06:42:28.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:27 smithi134 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0230caed-025b-46e9-a011-237601d16ac5"}]': finished 2024-09-19T06:42:28.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:27 smithi134 ceph-mon[35356]: osdmap e14: 3 total, 2 up, 3 in 2024-09-19T06:42:28.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:27 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:28 smithi103 ceph-mon[31158]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:28 smithi103 ceph-mon[31158]: from='client.? 172.21.15.174:0/4168363514' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:42:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:28 smithi134 ceph-mon[35356]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:28 smithi134 ceph-mon[35356]: from='client.? 172.21.15.174:0/4168363514' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:42:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:28 smithi174 ceph-mon[35637]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:28 smithi174 ceph-mon[35637]: from='client.? 172.21.15.174:0/4168363514' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:42:31.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:30 smithi103 ceph-mon[31158]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:31.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:30 smithi134 ceph-mon[35356]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:31.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:30 smithi174 ceph-mon[35637]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:32.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:32 smithi174 ceph-mon[35637]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:32 smithi103 ceph-mon[31158]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:32 smithi134 ceph-mon[35356]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:35.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:34 smithi174 ceph-mon[35637]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:35.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:34 smithi103 ceph-mon[31158]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:34 smithi134 ceph-mon[35356]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:36.944 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:36 smithi174 ceph-mon[35637]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:36.944 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:36 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T06:42:36.944 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:36 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:37.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:36 smithi103 ceph-mon[31158]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:37.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:36 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T06:42:37.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:36 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:36 smithi134 ceph-mon[35356]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:36 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T06:42:37.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:36 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:38.049 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:37 smithi174 ceph-mon[35637]: Deploying daemon osd.2 on smithi174 2024-09-19T06:42:38.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:37 smithi103 ceph-mon[31158]: Deploying daemon osd.2 on smithi174 2024-09-19T06:42:38.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:37 smithi134 ceph-mon[35356]: Deploying daemon osd.2 on smithi174 2024-09-19T06:42:39.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:38 smithi174 ceph-mon[35637]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:39.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:38 smithi103 ceph-mon[31158]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:39.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:38 smithi134 ceph-mon[35356]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:41.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:40 smithi103 ceph-mon[31158]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:41.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:40 smithi134 ceph-mon[35356]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:41.303 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:40 smithi174 ceph-mon[35637]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:43.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:42 smithi103 ceph-mon[31158]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:43.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:42 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:42:43.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:42 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:43.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:42 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:43.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:42 smithi174 ceph-mon[35637]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:43.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:42 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:42:43.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:42 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:43.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:42 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:43.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:42 smithi134 ceph-mon[35356]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:43.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:42 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:42:43.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:42 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:43.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:42 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:45.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:44 smithi103 ceph-mon[31158]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:45.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:45.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:45.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:45.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:45.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:45.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:44 smithi134 ceph-mon[35356]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:45.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:45.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:45.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:45.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:45.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:45.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:44 smithi174 ceph-mon[35637]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:45.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:45.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:45.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:45.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:45.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:46.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:46.300+0000 7f389effd640 1 -- 172.21.15.174:0/544295643 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f38a8062630 con 0x7f38940a31e0 2024-09-19T06:42:46.303 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 2 on host 'smithi174' 2024-09-19T06:42:46.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:46.303+0000 7f38b01a2640 1 -- 172.21.15.174:0/544295643 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f38940a31e0 msgr2=0x7f38940a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:46.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:46.303+0000 7f38b01a2640 1 --2- 172.21.15.174:0/544295643 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f38940a31e0 0x7f38940a56a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38a819fa10 tx=0x7f389800b410 comp rx=0 tx=0).stop 2024-09-19T06:42:46.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:46.303+0000 7f38b01a2640 1 -- 172.21.15.174:0/544295643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a8100480 msgr2=0x7f38a819e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:46.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:46.303+0000 7f38b01a2640 1 --2- 172.21.15.174:0/544295643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a8100480 0x7f38a819e4f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009230 tx=0x7f38a4004360 comp rx=0 tx=0).stop 2024-09-19T06:42:46.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:46.303+0000 7f38b01a2640 1 -- 172.21.15.174:0/544295643 shutdown_connections 2024-09-19T06:42:46.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:46.303+0000 7f38b01a2640 1 --2- 172.21.15.174:0/544295643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38a8105780 0x7f38a81a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:46.303+0000 7f38b01a2640 1 --2- 172.21.15.174:0/544295643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38a8102dd0 0x7f38a819ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:46.303+0000 7f38b01a2640 1 --2- 172.21.15.174:0/544295643 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f38940a31e0 0x7f38940a56a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:46.303+0000 7f38b01a2640 1 --2- 172.21.15.174:0/544295643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a8100480 0x7f38a819e4f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:46.303+0000 7f38b01a2640 1 -- 172.21.15.174:0/544295643 >> 172.21.15.174:0/544295643 conn(0x7f38a80fc290 msgr2=0x7f38a81037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:46.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:46.303+0000 7f38b01a2640 1 -- 172.21.15.174:0/544295643 shutdown_connections 2024-09-19T06:42:46.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:42:46.303+0000 7f38b01a2640 1 -- 172.21.15.174:0/544295643 wait complete. 2024-09-19T06:42:46.925 DEBUG:teuthology.orchestra.run.smithi174:osd.2> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T06:42:46.928 INFO:tasks.cephadm:Deploying osd.3 on smithi103 with /dev/vg_nvme/lv_3... 2024-09-19T06:42:46.929 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T06:42:47.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:46 smithi103 ceph-mon[31158]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:47.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:46 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:42:47.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:46 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:47.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:46 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:47.140 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:46 smithi174 ceph-mon[35637]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:47.140 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:46 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:42:47.140 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:46 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:47.140 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:46 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:47.187 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:47.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:46 smithi134 ceph-mon[35356]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:47.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:46 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:42:47.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:46 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:47.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:46 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:48.887 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 06:42:48 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-2[40637]: 2024-09-19T06:42:48.702+0000 7fc7c7c89640 -1 osd.2 0 log_to_monitors true 2024-09-19T06:42:49.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:48 smithi103 ceph-mon[31158]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:49.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:48 smithi103 ceph-mon[31158]: from='osd.2 [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T06:42:49.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:48 smithi103 ceph-mon[31158]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T06:42:49.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:48 smithi134 ceph-mon[35356]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:49.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:48 smithi134 ceph-mon[35356]: from='osd.2 [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T06:42:49.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:48 smithi134 ceph-mon[35356]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T06:42:49.192 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:48 smithi174 ceph-mon[35637]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:49.193 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:48 smithi174 ceph-mon[35637]: from='osd.2 [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T06:42:49.193 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:48 smithi174 ceph-mon[35637]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T06:42:49.463 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:42:49.490 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_3 2024-09-19T06:42:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:50.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:49 smithi103 ceph-mon[31158]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T06:42:50.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:49 smithi103 ceph-mon[31158]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T06:42:50.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:49 smithi103 ceph-mon[31158]: from='osd.2 [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:42:50.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:49 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:50.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:49 smithi103 ceph-mon[31158]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:42:50.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:49 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:50.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:49 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:50.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:49 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:42:50.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:49 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:50.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:49 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:50.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:49 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:50.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:49 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:50.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:49 smithi134 ceph-mon[35356]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T06:42:50.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:49 smithi134 ceph-mon[35356]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T06:42:50.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:49 smithi134 ceph-mon[35356]: from='osd.2 [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:42:50.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:49 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:50.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:49 smithi134 ceph-mon[35356]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:42:50.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:49 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:50.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:49 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:50.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:49 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:42:50.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:49 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:50.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:49 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:50.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:49 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:50.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:49 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:50.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.215+0000 7ff0bbcca640 1 -- 172.21.15.103:0/409896158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b4101c00 msgr2=0x7ff0b4102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.215+0000 7ff0bbcca640 1 --2- 172.21.15.103:0/409896158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b4101c00 0x7ff0b4102000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009900 tx=0x7ff0b002a410 comp rx=0 tx=0).stop 2024-09-19T06:42:50.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.215+0000 7ff0bbcca640 1 -- 172.21.15.103:0/409896158 shutdown_connections 2024-09-19T06:42:50.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.215+0000 7ff0bbcca640 1 --2- 172.21.15.103:0/409896158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0b41037c0 0x7ff0b410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.215+0000 7ff0bbcca640 1 --2- 172.21.15.103:0/409896158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0b4102e00 0x7ff0b4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.216+0000 7ff0bbcca640 1 --2- 172.21.15.103:0/409896158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b4101c00 0x7ff0b4102000 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.216+0000 7ff0bbcca640 1 -- 172.21.15.103:0/409896158 >> 172.21.15.103:0/409896158 conn(0x7ff0b40fd390 msgr2=0x7ff0b40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:50.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.216+0000 7ff0bbcca640 1 -- 172.21.15.103:0/409896158 shutdown_connections 2024-09-19T06:42:50.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.216+0000 7ff0bbcca640 1 -- 172.21.15.103:0/409896158 wait complete. 2024-09-19T06:42:50.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.216+0000 7ff0bbcca640 1 Processor -- start 2024-09-19T06:42:50.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.216+0000 7ff0bbcca640 1 -- start start 2024-09-19T06:42:50.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.216+0000 7ff0bbcca640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0b4101c00 0x7ff0b4197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.216+0000 7ff0bbcca640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b4102e00 0x7ff0b4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.216+0000 7ff0bbcca640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0b41037c0 0x7ff0b419f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.216+0000 7ff0bbcca640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b410d860 con 0x7ff0b4102e00 2024-09-19T06:42:50.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.216+0000 7ff0bbcca640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b410d6e0 con 0x7ff0b41037c0 2024-09-19T06:42:50.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.216+0000 7ff0bbcca640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b410d9e0 con 0x7ff0b4101c00 2024-09-19T06:42:50.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.216+0000 7ff0b9265640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b4102e00 0x7ff0b4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.217+0000 7ff0b9265640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b4102e00 0x7ff0b4198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35794/0 (socket says 172.21.15.103:35794) 2024-09-19T06:42:50.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.217+0000 7ff0b9265640 1 -- 172.21.15.103:0/44851499 learned_addr learned my addr 172.21.15.103:0/44851499 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:42:50.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.217+0000 7ff0b9a66640 1 --2- 172.21.15.103:0/44851499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0b4101c00 0x7ff0b4197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.217+0000 7ff0ba267640 1 --2- 172.21.15.103:0/44851499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0b41037c0 0x7ff0b419f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.217+0000 7ff0b9265640 1 -- 172.21.15.103:0/44851499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0b4101c00 msgr2=0x7ff0b4197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.217+0000 7ff0b9265640 1 --2- 172.21.15.103:0/44851499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0b4101c00 0x7ff0b4197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.217+0000 7ff0b9265640 1 -- 172.21.15.103:0/44851499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0b41037c0 msgr2=0x7ff0b419f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.217+0000 7ff0b9265640 1 --2- 172.21.15.103:0/44851499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0b41037c0 0x7ff0b419f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.217+0000 7ff0b9265640 1 -- 172.21.15.103:0/44851499 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b419fb70 con 0x7ff0b4102e00 2024-09-19T06:42:50.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.217+0000 7ff0b9a66640 1 --2- 172.21.15.103:0/44851499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0b4101c00 0x7ff0b4197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:50.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.217+0000 7ff0b9265640 1 --2- 172.21.15.103:0/44851499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b4102e00 0x7ff0b4198480 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400d950 tx=0x7ff0a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:50.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.217+0000 7ff0aaffd640 1 -- 172.21.15.103:0/44851499 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4017070 con 0x7ff0b4102e00 2024-09-19T06:42:50.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.217+0000 7ff0aaffd640 1 -- 172.21.15.103:0/44851499 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a4009c30 con 0x7ff0b4102e00 2024-09-19T06:42:50.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.217+0000 7ff0bbcca640 1 -- 172.21.15.103:0/44851499 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b419fe60 con 0x7ff0b4102e00 2024-09-19T06:42:50.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.217+0000 7ff0aaffd640 1 -- 172.21.15.103:0/44851499 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a400f6b0 con 0x7ff0b4102e00 2024-09-19T06:42:50.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.217+0000 7ff0bbcca640 1 -- 172.21.15.103:0/44851499 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b41a0310 con 0x7ff0b4102e00 2024-09-19T06:42:50.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.218+0000 7ff0bbcca640 1 -- 172.21.15.103:0/44851499 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b41090a0 con 0x7ff0b4102e00 2024-09-19T06:42:50.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.220+0000 7ff0aaffd640 1 -- 172.21.15.103:0/44851499 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7ff0a40109d0 con 0x7ff0b4102e00 2024-09-19T06:42:50.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.220+0000 7ff0aaffd640 1 --2- 172.21.15.103:0/44851499 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff0a00a31e0 0x7ff0a00a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.220+0000 7ff0b9a66640 1 --2- 172.21.15.103:0/44851499 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff0a00a31e0 0x7ff0a00a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.220+0000 7ff0aaffd640 1 -- 172.21.15.103:0/44851499 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 2571+0+0 (secure 0 0 0) 0x7ff0a40cdd10 con 0x7ff0b4102e00 2024-09-19T06:42:50.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.221+0000 7ff0b9a66640 1 --2- 172.21.15.103:0/44851499 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff0a00a31e0 0x7ff0a00a56a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0006010 tx=0x7ff0b0032cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:50.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.227+0000 7ff0aaffd640 1 -- 172.21.15.103:0/44851499 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff0a4009de0 con 0x7ff0b4102e00 2024-09-19T06:42:50.333 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 06:42:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-2[40637]: 2024-09-19T06:42:49.898+0000 7fc7c33e9640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T06:42:50.333 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 06:42:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-2[40637]: 2024-09-19T06:42:49.902+0000 7fc7bfa23640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:42:50.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:49 smithi174 ceph-mon[35637]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T06:42:50.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:49 smithi174 ceph-mon[35637]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T06:42:50.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:49 smithi174 ceph-mon[35637]: from='osd.2 [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:42:50.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:49 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:50.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:49 smithi174 ceph-mon[35637]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:42:50.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:49 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:50.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:49 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:50.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:49 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:42:50.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:49 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:50.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:49 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:50.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:49 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:50.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:49 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:50.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:50.410+0000 7ff0bbcca640 1 -- 172.21.15.103:0/44851499 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff0b4062630 con 0x7ff0a00a31e0 2024-09-19T06:42:51.042 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:50 smithi103 ceph-mon[31158]: Detected new or changed devices on smithi174 2024-09-19T06:42:51.042 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:50 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-19T06:42:51.042 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:50 smithi103 ceph-mon[31158]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:51.043 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:50 smithi103 ceph-mon[31158]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T06:42:51.043 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:50 smithi103 ceph-mon[31158]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T06:42:51.043 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:51.043 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:51.043 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:50 smithi103 ceph-mon[31158]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:42:51.043 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:42:51.043 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:42:51.043 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:51.043 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:51.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:50 smithi134 ceph-mon[35356]: Detected new or changed devices on smithi174 2024-09-19T06:42:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:50 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-19T06:42:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:50 smithi134 ceph-mon[35356]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:50 smithi134 ceph-mon[35356]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T06:42:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:50 smithi134 ceph-mon[35356]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T06:42:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:50 smithi134 ceph-mon[35356]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:42:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:42:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:42:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:51.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:50 smithi174 ceph-mon[35637]: Detected new or changed devices on smithi174 2024-09-19T06:42:51.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:50 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-19T06:42:51.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:50 smithi174 ceph-mon[35637]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:51.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:50 smithi174 ceph-mon[35637]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T06:42:51.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:50 smithi174 ceph-mon[35637]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T06:42:51.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:51.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:51.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:50 smithi174 ceph-mon[35637]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:42:51.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:42:51.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:42:51.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:51.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:52.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:52 smithi103 ceph-mon[31158]: purged_snaps scrub starts 2024-09-19T06:42:52.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:52 smithi103 ceph-mon[31158]: purged_snaps scrub ok 2024-09-19T06:42:52.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:52 smithi103 ceph-mon[31158]: osd.2 [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290] boot 2024-09-19T06:42:52.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:52 smithi103 ceph-mon[31158]: osdmap e17: 3 total, 3 up, 3 in 2024-09-19T06:42:52.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:52 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:52.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:52 smithi174 ceph-mon[35637]: purged_snaps scrub starts 2024-09-19T06:42:52.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:52 smithi174 ceph-mon[35637]: purged_snaps scrub ok 2024-09-19T06:42:52.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:52 smithi174 ceph-mon[35637]: osd.2 [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290] boot 2024-09-19T06:42:52.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:52 smithi174 ceph-mon[35637]: osdmap e17: 3 total, 3 up, 3 in 2024-09-19T06:42:52.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:52 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:52.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:52 smithi134 ceph-mon[35356]: purged_snaps scrub starts 2024-09-19T06:42:52.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:52 smithi134 ceph-mon[35356]: purged_snaps scrub ok 2024-09-19T06:42:52.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:52 smithi134 ceph-mon[35356]: osd.2 [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290] boot 2024-09-19T06:42:52.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:52 smithi134 ceph-mon[35356]: osdmap e17: 3 total, 3 up, 3 in 2024-09-19T06:42:52.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:52 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:42:53.058 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 06:42:52 smithi103 sudo[49380]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:42:53.058 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 06:42:52 smithi103 sudo[49380]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:42:53.058 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 06:42:52 smithi103 sudo[49380]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:42:53.058 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 06:42:52 smithi103 sudo[49380]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:42:53.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:53 smithi174 ceph-mon[35637]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:53.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:53 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:42:53.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:53 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T06:42:53.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:53 smithi174 ceph-mon[35637]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T06:42:53.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:53 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:42:53.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:53 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/4092674946' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c54a8399-bf4b-4d70-b207-a13d85f99590"}]: dispatch 2024-09-19T06:42:53.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:53 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T06:42:53.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:53 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/4092674946' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c54a8399-bf4b-4d70-b207-a13d85f99590"}]': finished 2024-09-19T06:42:53.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:53 smithi174 ceph-mon[35637]: osdmap e19: 4 total, 3 up, 4 in 2024-09-19T06:42:53.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:53 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:42:53.388 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 06:42:53 smithi103 sudo[49406]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:42:53.388 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 06:42:53 smithi103 sudo[49406]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:42:53.388 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 06:42:53 smithi103 sudo[49406]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:42:53.388 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 06:42:53 smithi103 sudo[49406]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:42:53.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:53 smithi103 ceph-mon[31158]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:53.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:53 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:42:53.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:53 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T06:42:53.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:53 smithi103 ceph-mon[31158]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T06:42:53.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:53 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:42:53.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:53 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/4092674946' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c54a8399-bf4b-4d70-b207-a13d85f99590"}]: dispatch 2024-09-19T06:42:53.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:53 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T06:42:53.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:53 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/4092674946' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c54a8399-bf4b-4d70-b207-a13d85f99590"}]': finished 2024-09-19T06:42:53.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:53 smithi103 ceph-mon[31158]: osdmap e19: 4 total, 3 up, 4 in 2024-09-19T06:42:53.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:53 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:42:53.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:53 smithi134 ceph-mon[35356]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:42:53.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:53 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:42:53.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:53 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T06:42:53.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:53 smithi134 ceph-mon[35356]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T06:42:53.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:53 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:42:53.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:53 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/4092674946' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c54a8399-bf4b-4d70-b207-a13d85f99590"}]: dispatch 2024-09-19T06:42:53.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:53 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T06:42:53.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:53 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/4092674946' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c54a8399-bf4b-4d70-b207-a13d85f99590"}]': finished 2024-09-19T06:42:53.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:53 smithi134 ceph-mon[35356]: osdmap e19: 4 total, 3 up, 4 in 2024-09-19T06:42:53.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:53 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:42:53.720 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 06:42:53 smithi134 sudo[43703]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:42:53.720 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 06:42:53 smithi134 sudo[43703]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:42:53.720 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 06:42:53 smithi134 sudo[43703]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:42:53.720 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 06:42:53 smithi134 sudo[43703]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:42:54.059 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 06:42:53 smithi134 sudo[43706]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:42:54.059 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 06:42:53 smithi134 sudo[43706]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:42:54.059 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 06:42:53 smithi134 sudo[43706]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:42:54.059 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 06:42:53 smithi134 sudo[43706]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:42:54.292 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 06:42:54 smithi174 sudo[44699]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:42:54.292 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 06:42:54 smithi174 sudo[44699]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:42:54.293 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 06:42:54 smithi174 sudo[44699]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:42:54.293 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 06:42:54 smithi174 sudo[44699]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:42:54.293 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:54 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2725002583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:42:54.293 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:54.294 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:54 smithi174 ceph-mon[35637]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T06:42:54.294 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:42:54.294 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:54 smithi174 ceph-mon[35637]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:42:54.294 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:54.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:54 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2725002583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:42:54.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:54.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:54 smithi103 ceph-mon[31158]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T06:42:54.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:42:54.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:54 smithi103 ceph-mon[31158]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:42:54.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:54.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:54 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2725002583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:42:54.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:54.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:54 smithi134 ceph-mon[35356]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T06:42:54.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:42:54.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:54 smithi134 ceph-mon[35356]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:42:54.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:54.583 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 06:42:54 smithi174 sudo[44702]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:42:54.583 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 06:42:54 smithi174 sudo[44702]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:42:54.583 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 06:42:54 smithi174 sudo[44702]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:42:54.583 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 06:42:54 smithi174 sudo[44702]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:42:54.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:54 smithi103 sudo[49626]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:42:54.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:54 smithi103 sudo[49626]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:42:54.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:54 smithi103 sudo[49626]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:42:54.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:54 smithi103 sudo[49626]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:42:55.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:54 smithi103 sudo[49813]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:42:55.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:54 smithi103 sudo[49813]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:42:55.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:54 smithi103 sudo[49813]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:42:55.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:54 smithi103 sudo[49813]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:42:55.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 sudo[44705]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:42:55.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 sudo[44705]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:42:55.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 sudo[44705]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:42:55.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 sudo[44705]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:42:55.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:42:55.345+0000 7ff0aaffd640 1 -- 172.21.15.103:0/44851499 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff0a40980f0 con 0x7ff0b4102e00 2024-09-19T06:42:55.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 sudo[44708]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:42:55.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 sudo[44708]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:42:55.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 sudo[44708]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:42:55.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 sudo[44708]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:42:55.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 sudo[43709]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:42:55.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 sudo[43709]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:42:55.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 sudo[43709]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:42:55.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 sudo[43709]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:42:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:55 smithi103 ceph-mon[31158]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:42:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:55 smithi103 ceph-mon[31158]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:42:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:42:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:42:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:42:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:42:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:42:56.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:42:56.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 sudo[43712]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:42:56.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 sudo[43712]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:42:56.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 sudo[43712]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:42:56.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 sudo[43712]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:42:56.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:56.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:42:56.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:42:56.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:42:56.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:42:56.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:42:56.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:42:56.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:42:56.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:42:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:42:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 ceph-mon[35637]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:42:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 ceph-mon[35637]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:42:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:42:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:42:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:42:56.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:42:56.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:42:56.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:42:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:56 smithi103 ceph-mon[31158]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:42:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:56 smithi103 ceph-mon[31158]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:42:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:56 smithi103 ceph-mon[31158]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:42:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:56 smithi103 ceph-mon[31158]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T06:42:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:42:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:42:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:42:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:56 smithi103 ceph-mon[31158]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:42:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:56 smithi103 ceph-mon[31158]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:42:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:56 smithi134 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:42:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:56 smithi134 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:42:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:56 smithi134 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:42:57.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:56 smithi134 ceph-mon[35356]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T06:42:57.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:56 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:42:57.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:56 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:42:57.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:56 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:42:57.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:56 smithi134 ceph-mon[35356]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:42:57.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:56 smithi134 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:42:57.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:56 smithi174 ceph-mon[35637]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:42:57.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:56 smithi174 ceph-mon[35637]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:42:57.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:56 smithi174 ceph-mon[35637]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:42:57.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:56 smithi174 ceph-mon[35637]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T06:42:57.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:56 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:42:57.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:56 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:42:57.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:56 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:42:57.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:56 smithi174 ceph-mon[35637]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:42:57.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:56 smithi174 ceph-mon[35637]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:42:58.949 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:42:58 smithi103 ceph-mon[31158]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:42:59.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:42:58 smithi134 ceph-mon[35356]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:42:59.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:42:58 smithi174 ceph-mon[35637]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:01.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:00 smithi103 ceph-mon[31158]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:01.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:00 smithi134 ceph-mon[35356]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:01.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:00 smithi174 ceph-mon[35637]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:02.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:01 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T06:43:02.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:01 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:02.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:01 smithi103 ceph-mon[31158]: Deploying daemon osd.3 on smithi103 2024-09-19T06:43:02.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:01 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T06:43:02.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:01 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:02.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:01 smithi134 ceph-mon[35356]: Deploying daemon osd.3 on smithi103 2024-09-19T06:43:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:01 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T06:43:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:01 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:01 smithi174 ceph-mon[35637]: Deploying daemon osd.3 on smithi103 2024-09-19T06:43:03.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:02 smithi103 ceph-mon[31158]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:03.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:02 smithi134 ceph-mon[35356]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:02 smithi174 ceph-mon[35637]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:04 smithi103 ceph-mon[31158]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:05.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:04 smithi134 ceph-mon[35356]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:05.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:04 smithi174 ceph-mon[35637]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:07.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:06 smithi103 ceph-mon[31158]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:07.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:06 smithi134 ceph-mon[35356]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:07.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:06 smithi174 ceph-mon[35637]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:08.008 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:07 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:08.009 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:07 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:08.009 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:07 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:08.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:07 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:08.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:07 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:08.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:07 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:08.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:07 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:08.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:07 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:08.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:07 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:09.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:08 smithi103 ceph-mon[31158]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:09.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:08 smithi134 ceph-mon[35356]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:09.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:08 smithi174 ceph-mon[35637]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:10.782 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 3 on host 'smithi103' 2024-09-19T06:43:10.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:43:10.780+0000 7ff0aaffd640 1 -- 172.21.15.103:0/44851499 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff0b4062630 con 0x7ff0a00a31e0 2024-09-19T06:43:10.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:43:10.782+0000 7ff0bbcca640 1 -- 172.21.15.103:0/44851499 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff0a00a31e0 msgr2=0x7ff0a00a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:10.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:43:10.782+0000 7ff0bbcca640 1 --2- 172.21.15.103:0/44851499 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff0a00a31e0 0x7ff0a00a56a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0006010 tx=0x7ff0b0032cf0 comp rx=0 tx=0).stop 2024-09-19T06:43:10.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:43:10.782+0000 7ff0bbcca640 1 -- 172.21.15.103:0/44851499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b4102e00 msgr2=0x7ff0b4198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:10.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:43:10.782+0000 7ff0bbcca640 1 --2- 172.21.15.103:0/44851499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b4102e00 0x7ff0b4198480 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400d950 tx=0x7ff0a400de10 comp rx=0 tx=0).stop 2024-09-19T06:43:10.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:43:10.782+0000 7ff0bbcca640 1 -- 172.21.15.103:0/44851499 shutdown_connections 2024-09-19T06:43:10.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:43:10.782+0000 7ff0bbcca640 1 --2- 172.21.15.103:0/44851499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0b41037c0 0x7ff0b419f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:10.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:43:10.782+0000 7ff0bbcca640 1 --2- 172.21.15.103:0/44851499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b4102e00 0x7ff0b4198480 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:10.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:43:10.782+0000 7ff0bbcca640 1 --2- 172.21.15.103:0/44851499 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff0a00a31e0 0x7ff0a00a56a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:10.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:43:10.782+0000 7ff0bbcca640 1 --2- 172.21.15.103:0/44851499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0b4101c00 0x7ff0b4197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:10.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:43:10.782+0000 7ff0bbcca640 1 -- 172.21.15.103:0/44851499 >> 172.21.15.103:0/44851499 conn(0x7ff0b40fd390 msgr2=0x7ff0b4106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:10.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:43:10.782+0000 7ff0bbcca640 1 -- 172.21.15.103:0/44851499 shutdown_connections 2024-09-19T06:43:10.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:43:10.782+0000 7ff0bbcca640 1 -- 172.21.15.103:0/44851499 wait complete. 2024-09-19T06:43:11.077 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:10 smithi103 ceph-mon[31158]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:11.077 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:10 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:11.078 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:10 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:11.078 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:10 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:11.078 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:10 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:11.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:10 smithi134 ceph-mon[35356]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:11.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:10 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:11.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:10 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:11.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:10 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:11.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:10 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:11.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:10 smithi174 ceph-mon[35637]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:11.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:10 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:11.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:10 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:11.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:10 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:11.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:10 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:11.422 DEBUG:teuthology.orchestra.run.smithi103:osd.3> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T06:43:11.425 INFO:tasks.cephadm:Deploying osd.4 on smithi134 with /dev/vg_nvme/lv_3... 2024-09-19T06:43:11.425 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T06:43:11.681 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:43:11.990 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:11 smithi103 podman[52139]: 2024-09-19 06:43:11.558556137 +0000 UTC m=+0.020188415 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:43:11.990 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:11 smithi103 podman[52139]: 2024-09-19 06:43:11.661831198 +0000 UTC m=+0.123463452 container create 1abbfb35f47d67f28a0f9cbf2b98b99fd8d828a7b1e0d028c26788395056a589 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T06:43:11.990 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:11 smithi103 podman[52139]: 2024-09-19 06:43:11.889596746 +0000 UTC m=+0.351229000 container init 1abbfb35f47d67f28a0f9cbf2b98b99fd8d828a7b1e0d028c26788395056a589 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:43:11.991 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:11 smithi103 podman[52139]: 2024-09-19 06:43:11.893173742 +0000 UTC m=+0.354805997 container start 1abbfb35f47d67f28a0f9cbf2b98b99fd8d828a7b1e0d028c26788395056a589 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T06:43:11.991 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:11 smithi103 podman[52139]: 2024-09-19 06:43:11.989606905 +0000 UTC m=+0.451239172 container attach 1abbfb35f47d67f28a0f9cbf2b98b99fd8d828a7b1e0d028c26788395056a589 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T06:43:13.108 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:12 smithi134 ceph-mon[35356]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:13.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:12 smithi103 ceph-mon[31158]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:13.138 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:13 smithi103 podman[52139]: 2024-09-19 06:43:13.015341122 +0000 UTC m=+1.476973377 container died 1abbfb35f47d67f28a0f9cbf2b98b99fd8d828a7b1e0d028c26788395056a589 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T06:43:13.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:12 smithi174 ceph-mon[35637]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:13.804 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:13 smithi103 podman[52300]: 2024-09-19 06:43:13.549980411 +0000 UTC m=+1.458494196 container remove 1abbfb35f47d67f28a0f9cbf2b98b99fd8d828a7b1e0d028c26788395056a589 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T06:43:13.804 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:13 smithi103 systemd[1]: ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.3.service: Failed with result 'exit-code'. 2024-09-19T06:43:13.805 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:13 smithi103 systemd[1]: ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.3.service: Consumed 1.126s CPU time. 2024-09-19T06:43:14.161 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:43:14.186 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_3 2024-09-19T06:43:14.429 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:43:14.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.894+0000 7f0f10a0a640 1 -- 172.21.15.134:0/3177956585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f0c100b10 msgr2=0x7f0f0c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:14.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.894+0000 7f0f10a0a640 1 --2- 172.21.15.134:0/3177956585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f0c100b10 0x7f0f0c100f10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009900 tx=0x7f0efc02a410 comp rx=0 tx=0).stop 2024-09-19T06:43:14.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.894+0000 7f0f10a0a640 1 -- 172.21.15.134:0/3177956585 shutdown_connections 2024-09-19T06:43:14.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.894+0000 7f0f10a0a640 1 --2- 172.21.15.134:0/3177956585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f0c1026d0 0x7f0f0c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:14.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.894+0000 7f0f10a0a640 1 --2- 172.21.15.134:0/3177956585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f0c101d10 0x7f0f0c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:14.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.894+0000 7f0f10a0a640 1 --2- 172.21.15.134:0/3177956585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f0c100b10 0x7f0f0c100f10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:14.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.894+0000 7f0f10a0a640 1 -- 172.21.15.134:0/3177956585 >> 172.21.15.134:0/3177956585 conn(0x7f0f0c0fc2a0 msgr2=0x7f0f0c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:14.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.894+0000 7f0f10a0a640 1 -- 172.21.15.134:0/3177956585 shutdown_connections 2024-09-19T06:43:14.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.894+0000 7f0f10a0a640 1 -- 172.21.15.134:0/3177956585 wait complete. 2024-09-19T06:43:14.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.895+0000 7f0f10a0a640 1 Processor -- start 2024-09-19T06:43:14.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.895+0000 7f0f10a0a640 1 -- start start 2024-09-19T06:43:14.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.895+0000 7f0f10a0a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f0c100b10 0x7f0f0c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:14.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.895+0000 7f0f10a0a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f0c101d10 0x7f0f0c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:14.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.895+0000 7f0f10a0a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f0c1026d0 0x7f0f0c19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:14.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.895+0000 7f0f10a0a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c10c770 con 0x7f0f0c101d10 2024-09-19T06:43:14.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.895+0000 7f0f10a0a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c10c5f0 con 0x7f0f0c1026d0 2024-09-19T06:43:14.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.895+0000 7f0f10a0a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c10c8f0 con 0x7f0f0c100b10 2024-09-19T06:43:14.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.895+0000 7f0f0ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f0c1026d0 0x7f0f0c19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:14.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.895+0000 7f0f0ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f0c1026d0 0x7f0f0c19f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37518/0 (socket says 172.21.15.134:37518) 2024-09-19T06:43:14.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.895+0000 7f0f0ad9d640 1 -- 172.21.15.134:0/2140338253 learned_addr learned my addr 172.21.15.134:0/2140338253 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-19T06:43:14.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.895+0000 7f0f09d9b640 1 --2- 172.21.15.134:0/2140338253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f0c101d10 0x7f0f0c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:14.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.895+0000 7f0f0a59c640 1 --2- 172.21.15.134:0/2140338253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f0c100b10 0x7f0f0c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:14.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.896+0000 7f0f0ad9d640 1 -- 172.21.15.134:0/2140338253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f0c100b10 msgr2=0x7f0f0c197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:14.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.896+0000 7f0f0ad9d640 1 --2- 172.21.15.134:0/2140338253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f0c100b10 0x7f0f0c197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:14.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.896+0000 7f0f0ad9d640 1 -- 172.21.15.134:0/2140338253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f0c101d10 msgr2=0x7f0f0c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:14.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.896+0000 7f0f0ad9d640 1 --2- 172.21.15.134:0/2140338253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f0c101d10 0x7f0f0c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:14.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.896+0000 7f0f0ad9d640 1 -- 172.21.15.134:0/2140338253 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0c19fbb0 con 0x7f0f0c1026d0 2024-09-19T06:43:14.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.896+0000 7f0f0ad9d640 1 --2- 172.21.15.134:0/2140338253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f0c1026d0 0x7f0f0c19f540 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400eaa0 tx=0x7f0f0400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:14.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.896+0000 7f0efb7fe640 1 -- 172.21.15.134:0/2140338253 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0400cdb0 con 0x7f0f0c1026d0 2024-09-19T06:43:14.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.896+0000 7f0f10a0a640 1 -- 172.21.15.134:0/2140338253 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0c19fea0 con 0x7f0f0c1026d0 2024-09-19T06:43:14.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.896+0000 7f0efb7fe640 1 -- 172.21.15.134:0/2140338253 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f04005020 con 0x7f0f0c1026d0 2024-09-19T06:43:14.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.896+0000 7f0efb7fe640 1 -- 172.21.15.134:0/2140338253 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f040116b0 con 0x7f0f0c1026d0 2024-09-19T06:43:14.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.896+0000 7f0f10a0a640 1 -- 172.21.15.134:0/2140338253 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f0c1a0350 con 0x7f0f0c1026d0 2024-09-19T06:43:14.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.897+0000 7f0f10a0a640 1 -- 172.21.15.134:0/2140338253 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f0c1a0610 con 0x7f0f0c1026d0 2024-09-19T06:43:14.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.898+0000 7f0efb7fe640 1 -- 172.21.15.134:0/2140338253 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f0f04005560 con 0x7f0f0c1026d0 2024-09-19T06:43:14.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.899+0000 7f0efb7fe640 1 --2- 172.21.15.134:0/2140338253 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f0ef40a32f0 0x7f0ef40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:14.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.899+0000 7f0efb7fe640 1 -- 172.21.15.134:0/2140338253 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3352+0+0 (secure 0 0 0) 0x7f0f040ce6b0 con 0x7f0f0c1026d0 2024-09-19T06:43:14.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.899+0000 7f0f0a59c640 1 --2- 172.21.15.134:0/2140338253 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f0ef40a32f0 0x7f0ef40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:14.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.899+0000 7f0f0a59c640 1 --2- 172.21.15.134:0/2140338253 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f0ef40a32f0 0x7f0ef40a57b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0efc005ab0 tx=0x7f0efc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:14.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:14.903+0000 7f0efb7fe640 1 -- 172.21.15.134:0/2140338253 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0f04016030 con 0x7f0f0c1026d0 2024-09-19T06:43:15.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:15.076+0000 7f0f10a0a640 1 -- 172.21.15.134:0/2140338253 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0f0c1a07e0 con 0x7f0ef40a32f0 2024-09-19T06:43:15.078 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:14 smithi134 ceph-mon[35356]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:15.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:14 smithi103 ceph-mon[31158]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:15.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:14 smithi174 ceph-mon[35637]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:43:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:43:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:43:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:43:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:43:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:43:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:16.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:43:16.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:43:16.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:16.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:16.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:16.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:43:16.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:43:16.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:16.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:16.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:43:16.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:16.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:16.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:16.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:43:16.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:43:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:43:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:43:16.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:43:16.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:16.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:16.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:43:16.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:16.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:16.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:16.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:43:16.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:17.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:16 smithi134 ceph-mon[35356]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:43:17.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:16 smithi134 ceph-mon[35356]: Detected new or changed devices on smithi103 2024-09-19T06:43:17.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:16 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-19T06:43:17.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:16 smithi134 ceph-mon[35356]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:16 smithi103 ceph-mon[31158]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:43:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:16 smithi103 ceph-mon[31158]: Detected new or changed devices on smithi103 2024-09-19T06:43:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:16 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-19T06:43:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:16 smithi103 ceph-mon[31158]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:17.234 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:16 smithi174 ceph-mon[35637]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:43:17.234 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:16 smithi174 ceph-mon[35637]: Detected new or changed devices on smithi103 2024-09-19T06:43:17.234 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:16 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-19T06:43:17.234 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:16 smithi174 ceph-mon[35637]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:17.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:17 smithi134 ceph-mon[35356]: from='client.? 172.21.15.134:0/1000106187' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c046be6-464a-4ab5-812b-1781121fc317"}]: dispatch 2024-09-19T06:43:17.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:17 smithi134 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c046be6-464a-4ab5-812b-1781121fc317"}]: dispatch 2024-09-19T06:43:17.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:17 smithi134 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c046be6-464a-4ab5-812b-1781121fc317"}]': finished 2024-09-19T06:43:17.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:17 smithi134 ceph-mon[35356]: osdmap e21: 5 total, 3 up, 5 in 2024-09-19T06:43:17.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:17 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:43:17.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:17 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:17 smithi103 ceph-mon[31158]: from='client.? 172.21.15.134:0/1000106187' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c046be6-464a-4ab5-812b-1781121fc317"}]: dispatch 2024-09-19T06:43:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:17 smithi103 ceph-mon[31158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c046be6-464a-4ab5-812b-1781121fc317"}]: dispatch 2024-09-19T06:43:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:17 smithi103 ceph-mon[31158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c046be6-464a-4ab5-812b-1781121fc317"}]': finished 2024-09-19T06:43:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:17 smithi103 ceph-mon[31158]: osdmap e21: 5 total, 3 up, 5 in 2024-09-19T06:43:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:17 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:43:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:17 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:17 smithi174 ceph-mon[35637]: from='client.? 172.21.15.134:0/1000106187' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c046be6-464a-4ab5-812b-1781121fc317"}]: dispatch 2024-09-19T06:43:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:17 smithi174 ceph-mon[35637]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c046be6-464a-4ab5-812b-1781121fc317"}]: dispatch 2024-09-19T06:43:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:17 smithi174 ceph-mon[35637]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c046be6-464a-4ab5-812b-1781121fc317"}]': finished 2024-09-19T06:43:18.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:17 smithi174 ceph-mon[35637]: osdmap e21: 5 total, 3 up, 5 in 2024-09-19T06:43:18.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:17 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:43:18.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:17 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:19.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:18 smithi103 ceph-mon[31158]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:19.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:18 smithi103 ceph-mon[31158]: from='client.? 172.21.15.134:0/1484060787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:43:19.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:18 smithi134 ceph-mon[35356]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:19.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:18 smithi134 ceph-mon[35356]: from='client.? 172.21.15.134:0/1484060787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:43:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:18 smithi174 ceph-mon[35637]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:18 smithi174 ceph-mon[35637]: from='client.? 172.21.15.134:0/1484060787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:43:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:20 smithi103 ceph-mon[31158]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:21.191 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:20 smithi134 ceph-mon[35356]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:20 smithi174 ceph-mon[35637]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:23.132 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:22 smithi134 ceph-mon[35356]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:23.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:22 smithi103 ceph-mon[31158]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:23.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:22 smithi174 ceph-mon[35637]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:24.014 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:23 smithi103 systemd[1]: ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.3.service: Scheduled restart job, restart counter is at 1. 2024-09-19T06:43:24.014 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:23 smithi103 systemd[1]: Stopped Ceph osd.3 for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:43:24.014 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:23 smithi103 systemd[1]: ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.3.service: Consumed 1.126s CPU time. 2024-09-19T06:43:24.014 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:23 smithi103 systemd[1]: Starting Ceph osd.3 for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T06:43:24.014 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:23 smithi103 podman[54970]: 2024-09-19 06:43:23.778274636 +0000 UTC m=+0.016698404 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:43:24.015 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:23 smithi103 podman[54970]: 2024-09-19 06:43:23.883825845 +0000 UTC m=+0.122249599 container create 7e0d91e65316119f9161d365757d60b041d5c99c624289539082616137e56ed3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-activate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T06:43:24.268 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:24 smithi103 podman[54970]: 2024-09-19 06:43:24.137032283 +0000 UTC m=+0.375456046 container init 7e0d91e65316119f9161d365757d60b041d5c99c624289539082616137e56ed3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-19T06:43:24.269 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:24 smithi103 podman[54970]: 2024-09-19 06:43:24.140749448 +0000 UTC m=+0.379173205 container start 7e0d91e65316119f9161d365757d60b041d5c99c624289539082616137e56ed3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-activate, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T06:43:24.269 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:24 smithi103 podman[54970]: 2024-09-19 06:43:24.220216552 +0000 UTC m=+0.458640315 container attach 7e0d91e65316119f9161d365757d60b041d5c99c624289539082616137e56ed3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-activate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T06:43:24.638 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:24 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-activate[54985]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-19T06:43:24.638 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:24 smithi103 bash[54970]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-19T06:43:24.638 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:24 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-activate[54985]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-19T06:43:24.638 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:24 smithi103 bash[54970]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-19T06:43:25.086 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:24 smithi134 ceph-mon[35356]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:24 smithi103 ceph-mon[31158]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:25.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:24 smithi174 ceph-mon[35637]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:25.534 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:25 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-activate[54985]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-19T06:43:25.534 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:25 smithi103 bash[54970]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-19T06:43:25.534 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:25 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-activate[54985]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-19T06:43:25.535 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:25 smithi103 bash[54970]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-19T06:43:25.889 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:25 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-activate[54985]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-19T06:43:25.889 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:25 smithi103 bash[54970]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-19T06:43:25.889 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:25 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-activate[54985]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-19T06:43:25.889 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:25 smithi103 bash[54970]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-19T06:43:25.890 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:25 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-activate[54985]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-3/block 2024-09-19T06:43:25.890 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:25 smithi103 bash[54970]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-3/block 2024-09-19T06:43:25.890 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:25 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-activate[54985]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-19T06:43:25.890 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:25 smithi103 bash[54970]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-19T06:43:25.890 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:25 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-activate[54985]: --> ceph-volume raw activate successful for osd ID: 3 2024-09-19T06:43:25.891 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:25 smithi103 bash[54970]: --> ceph-volume raw activate successful for osd ID: 3 2024-09-19T06:43:25.891 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:25 smithi103 podman[54970]: 2024-09-19 06:43:25.583047563 +0000 UTC m=+1.821471331 container died 7e0d91e65316119f9161d365757d60b041d5c99c624289539082616137e56ed3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T06:43:26.388 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:26 smithi103 podman[55124]: 2024-09-19 06:43:26.19064665 +0000 UTC m=+0.601948456 container remove 7e0d91e65316119f9161d365757d60b041d5c99c624289539082616137e56ed3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-activate, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:43:26.736 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:26 smithi103 podman[55195]: 2024-09-19 06:43:26.308246816 +0000 UTC m=+0.019606315 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:43:26.737 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:26 smithi103 podman[55195]: 2024-09-19 06:43:26.424827548 +0000 UTC m=+0.136187056 container create cb1e7247b99f16d7b93f7f85bd7d1ae52b8a2d81af0a8e40d2f9341db090ff92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T06:43:26.737 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:26 smithi103 podman[55195]: 2024-09-19 06:43:26.62746396 +0000 UTC m=+0.338823464 container init cb1e7247b99f16d7b93f7f85bd7d1ae52b8a2d81af0a8e40d2f9341db090ff92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T06:43:26.737 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:26 smithi103 podman[55195]: 2024-09-19 06:43:26.63073293 +0000 UTC m=+0.342092430 container start cb1e7247b99f16d7b93f7f85bd7d1ae52b8a2d81af0a8e40d2f9341db090ff92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True) 2024-09-19T06:43:26.737 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:26 smithi103 ceph-osd[55215]: -- 172.21.15.103:0/2608662097 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557c9dd69a40 con 0x557c9eb30400 2024-09-19T06:43:26.737 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:26 smithi103 bash[55195]: cb1e7247b99f16d7b93f7f85bd7d1ae52b8a2d81af0a8e40d2f9341db090ff92 2024-09-19T06:43:27.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:26 smithi103 ceph-mon[31158]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:27.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:26 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T06:43:27.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:26 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:27.138 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:26 smithi103 systemd[1]: Started Ceph osd.3 for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:43:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:26 smithi134 ceph-mon[35356]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:26 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T06:43:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:26 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:26 smithi174 ceph-mon[35637]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:26 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T06:43:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:26 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:28.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:27 smithi134 ceph-mon[35356]: Deploying daemon osd.4 on smithi134 2024-09-19T06:43:28.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:27 smithi103 ceph-mon[31158]: Deploying daemon osd.4 on smithi134 2024-09-19T06:43:28.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:27 smithi174 ceph-mon[35637]: Deploying daemon osd.4 on smithi134 2024-09-19T06:43:28.637 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:28 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3[55210]: 2024-09-19T06:43:28.247+0000 7f7148c09640 -1 Falling back to public interface 2024-09-19T06:43:29.157 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:28 smithi134 ceph-mon[35356]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:28 smithi174 ceph-mon[35637]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:29.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:28 smithi103 ceph-mon[31158]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:31.156 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:30 smithi134 ceph-mon[35356]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:31.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:30 smithi174 ceph-mon[35637]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:31.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:30 smithi103 ceph-mon[31158]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:33.138 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:32 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3[55210]: 2024-09-19T06:43:32.747+0000 7f7148c09640 -1 osd.3 0 log_to_monitors true 2024-09-19T06:43:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:32 smithi103 ceph-mon[31158]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:32 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:32 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:32 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:32 smithi103 ceph-mon[31158]: from='osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T06:43:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:32 smithi134 ceph-mon[35356]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:33.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:32 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:33.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:32 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:33.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:32 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:33.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:32 smithi134 ceph-mon[35356]: from='osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T06:43:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:32 smithi174 ceph-mon[35637]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:32 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:32 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:32 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:32 smithi174 ceph-mon[35637]: from='osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T06:43:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:34 smithi174 ceph-mon[35637]: from='osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T06:43:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:34 smithi174 ceph-mon[35637]: osdmap e22: 5 total, 3 up, 5 in 2024-09-19T06:43:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:34 smithi174 ceph-mon[35637]: from='osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T06:43:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:34 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:43:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:34 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:34 smithi174 ceph-mon[35637]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:34.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:34 smithi103 ceph-mon[31158]: from='osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T06:43:34.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:34 smithi103 ceph-mon[31158]: osdmap e22: 5 total, 3 up, 5 in 2024-09-19T06:43:34.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:34 smithi103 ceph-mon[31158]: from='osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T06:43:34.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:34 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:43:34.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:34 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:34.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:34 smithi103 ceph-mon[31158]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:34.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:34 smithi134 ceph-mon[35356]: from='osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T06:43:34.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:34 smithi134 ceph-mon[35356]: osdmap e22: 5 total, 3 up, 5 in 2024-09-19T06:43:34.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:34 smithi134 ceph-mon[35356]: from='osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T06:43:34.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:34 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:43:34.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:34 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:34.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:34 smithi134 ceph-mon[35356]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:43:35.138 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:34 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3[55210]: 2024-09-19T06:43:34.826+0000 7f7144369640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T06:43:35.138 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 06:43:34 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3[55210]: 2024-09-19T06:43:34.836+0000 7f713c99b640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:43:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:35 smithi103 ceph-mon[31158]: from='osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T06:43:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:35 smithi103 ceph-mon[31158]: osdmap e23: 5 total, 3 up, 5 in 2024-09-19T06:43:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:35 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:43:35.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:35 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:35.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:35 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:43:35.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:35 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:35.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:35 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:35.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:35 smithi103 ceph-mon[31158]: from='osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333]' entity='osd.3' 2024-09-19T06:43:35.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:35 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:35.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:35 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:43:35.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:35 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:35.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:35 smithi103 ceph-mon[31158]: osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333] boot 2024-09-19T06:43:35.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:35 smithi103 ceph-mon[31158]: osdmap e24: 5 total, 4 up, 5 in 2024-09-19T06:43:35.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:35 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:43:35.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:35 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:35.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:35 smithi174 ceph-mon[35637]: from='osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T06:43:35.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:35 smithi174 ceph-mon[35637]: osdmap e23: 5 total, 3 up, 5 in 2024-09-19T06:43:35.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:35 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:43:35.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:35 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:35.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:35 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:43:35.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:35 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:35.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:35 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:35.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:35 smithi174 ceph-mon[35637]: from='osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333]' entity='osd.3' 2024-09-19T06:43:35.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:35 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:35.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:35 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:43:35.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:35 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:35.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:35 smithi174 ceph-mon[35637]: osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333] boot 2024-09-19T06:43:35.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:35 smithi174 ceph-mon[35637]: osdmap e24: 5 total, 4 up, 5 in 2024-09-19T06:43:35.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:35 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:43:35.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:35 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:35.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:35 smithi134 ceph-mon[35356]: from='osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T06:43:35.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:35 smithi134 ceph-mon[35356]: osdmap e23: 5 total, 3 up, 5 in 2024-09-19T06:43:35.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:35 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:43:35.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:35 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:35.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:35 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:43:35.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:35 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:35.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:35 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:35.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:35 smithi134 ceph-mon[35356]: from='osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333]' entity='osd.3' 2024-09-19T06:43:35.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:35 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:35.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:35 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:43:35.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:35 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:35.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:35 smithi134 ceph-mon[35356]: osd.3 [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333] boot 2024-09-19T06:43:35.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:35 smithi134 ceph-mon[35356]: osdmap e24: 5 total, 4 up, 5 in 2024-09-19T06:43:35.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:35 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:43:35.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:35 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:35.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:35.913+0000 7f0efb7fe640 1 -- 172.21.15.134:0/2140338253 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0f0c1a07e0 con 0x7f0ef40a32f0 2024-09-19T06:43:35.914 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 4 on host 'smithi134' 2024-09-19T06:43:35.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:35.915+0000 7f0f10a0a640 1 -- 172.21.15.134:0/2140338253 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f0ef40a32f0 msgr2=0x7f0ef40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:35.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:35.915+0000 7f0f10a0a640 1 --2- 172.21.15.134:0/2140338253 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f0ef40a32f0 0x7f0ef40a57b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0efc005ab0 tx=0x7f0efc000bc0 comp rx=0 tx=0).stop 2024-09-19T06:43:35.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:35.915+0000 7f0f10a0a640 1 -- 172.21.15.134:0/2140338253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f0c1026d0 msgr2=0x7f0f0c19f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:35.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:35.915+0000 7f0f10a0a640 1 --2- 172.21.15.134:0/2140338253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f0c1026d0 0x7f0f0c19f540 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400eaa0 tx=0x7f0f0400ef60 comp rx=0 tx=0).stop 2024-09-19T06:43:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:35.916+0000 7f0f10a0a640 1 -- 172.21.15.134:0/2140338253 shutdown_connections 2024-09-19T06:43:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:35.916+0000 7f0f10a0a640 1 --2- 172.21.15.134:0/2140338253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f0c1026d0 0x7f0f0c19f540 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:35.916+0000 7f0f10a0a640 1 --2- 172.21.15.134:0/2140338253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f0c101d10 0x7f0f0c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:35.916+0000 7f0f10a0a640 1 --2- 172.21.15.134:0/2140338253 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f0ef40a32f0 0x7f0ef40a57b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:35.916+0000 7f0f10a0a640 1 --2- 172.21.15.134:0/2140338253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f0c100b10 0x7f0f0c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:35.916+0000 7f0f10a0a640 1 -- 172.21.15.134:0/2140338253 >> 172.21.15.134:0/2140338253 conn(0x7f0f0c0fc2a0 msgr2=0x7f0f0c06a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:35.916+0000 7f0f10a0a640 1 -- 172.21.15.134:0/2140338253 shutdown_connections 2024-09-19T06:43:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:43:35.917+0000 7f0f10a0a640 1 -- 172.21.15.134:0/2140338253 wait complete. 2024-09-19T06:43:36.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:36 smithi134 ceph-mon[35356]: purged_snaps scrub starts 2024-09-19T06:43:36.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:36 smithi134 ceph-mon[35356]: purged_snaps scrub ok 2024-09-19T06:43:36.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:36 smithi134 ceph-mon[35356]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:43:36.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:36 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:36.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:36 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:36.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:36 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:36.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:36 smithi134 ceph-mon[35356]: osdmap e25: 5 total, 4 up, 5 in 2024-09-19T06:43:36.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:36 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:36 smithi174 ceph-mon[35637]: purged_snaps scrub starts 2024-09-19T06:43:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:36 smithi174 ceph-mon[35637]: purged_snaps scrub ok 2024-09-19T06:43:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:36 smithi174 ceph-mon[35637]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:43:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:36 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:36 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:36 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:36 smithi174 ceph-mon[35637]: osdmap e25: 5 total, 4 up, 5 in 2024-09-19T06:43:36.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:36 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:36.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:36 smithi103 ceph-mon[31158]: purged_snaps scrub starts 2024-09-19T06:43:36.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:36 smithi103 ceph-mon[31158]: purged_snaps scrub ok 2024-09-19T06:43:36.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:36 smithi103 ceph-mon[31158]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:43:36.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:36 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:36.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:36 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:36.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:36 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:36.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:36 smithi103 ceph-mon[31158]: osdmap e25: 5 total, 4 up, 5 in 2024-09-19T06:43:36.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:36 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:36.588 DEBUG:teuthology.orchestra.run.smithi134:osd.4> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T06:43:36.590 INFO:tasks.cephadm:Deploying osd.5 on smithi174 with /dev/vg_nvme/lv_3... 2024-09-19T06:43:36.590 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T06:43:36.843 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:43:38.061 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:38 smithi174 ceph-mon[35637]: osdmap e26: 5 total, 4 up, 5 in 2024-09-19T06:43:38.061 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:38 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:38.062 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:38 smithi174 ceph-mon[35637]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:43:38.320 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:38 smithi134 ceph-mon[35356]: osdmap e26: 5 total, 4 up, 5 in 2024-09-19T06:43:38.321 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:38 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:38.321 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:38 smithi134 ceph-mon[35356]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:43:38.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:38 smithi103 ceph-mon[31158]: osdmap e26: 5 total, 4 up, 5 in 2024-09-19T06:43:38.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:38 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:38.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:38 smithi103 ceph-mon[31158]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:43:38.687 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 06:43:38 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4[46370]: 2024-09-19T06:43:38.318+0000 7f85279d3640 -1 osd.4 0 log_to_monitors true 2024-09-19T06:43:39.147 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:43:39.173 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_3 2024-09-19T06:43:39.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:39 smithi174 ceph-mon[35637]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T06:43:39.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:39 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:39.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:39 smithi174 ceph-mon[35637]: from='osd.4 [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T06:43:39.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:39 smithi174 ceph-mon[35637]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T06:43:39.341 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:39 smithi134 ceph-mon[35356]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T06:43:39.341 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:39 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:39.341 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:39 smithi134 ceph-mon[35356]: from='osd.4 [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T06:43:39.341 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:39 smithi134 ceph-mon[35356]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T06:43:39.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:39 smithi103 ceph-mon[31158]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T06:43:39.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:39.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:39 smithi103 ceph-mon[31158]: from='osd.4 [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T06:43:39.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:39 smithi103 ceph-mon[31158]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T06:43:39.430 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:43:39.906 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.898+0000 7f004ad9f640 1 -- 172.21.15.174:0/2924488804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00441037c0 msgr2=0x7f004410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:39.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.898+0000 7f004ad9f640 1 --2- 172.21.15.174:0/2924488804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00441037c0 0x7f004410a050 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f003c009ca0 tx=0x7f003c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.898+0000 7f004ad9f640 1 -- 172.21.15.174:0/2924488804 shutdown_connections 2024-09-19T06:43:39.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.898+0000 7f004ad9f640 1 --2- 172.21.15.174:0/2924488804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00441037c0 0x7f004410a050 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.898+0000 7f004ad9f640 1 --2- 172.21.15.174:0/2924488804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0044102e00 0x7f0044103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.898+0000 7f004ad9f640 1 --2- 172.21.15.174:0/2924488804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0044101c00 0x7f0044102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.898+0000 7f004ad9f640 1 -- 172.21.15.174:0/2924488804 >> 172.21.15.174:0/2924488804 conn(0x7f00440fd390 msgr2=0x7f00440ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:39.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.898+0000 7f004ad9f640 1 -- 172.21.15.174:0/2924488804 shutdown_connections 2024-09-19T06:43:39.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.898+0000 7f004ad9f640 1 -- 172.21.15.174:0/2924488804 wait complete. 2024-09-19T06:43:39.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f004ad9f640 1 Processor -- start 2024-09-19T06:43:39.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f004ad9f640 1 -- start start 2024-09-19T06:43:39.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f004ad9f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0044101c00 0x7f0044197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f004ad9f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0044102e00 0x7f00441983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f004ad9f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00441037c0 0x7f004419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f004ad9f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f004410d860 con 0x7f0044101c00 2024-09-19T06:43:39.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f004ad9f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f004410d6e0 con 0x7f0044102e00 2024-09-19T06:43:39.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f004ad9f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f004410d9e0 con 0x7f00441037c0 2024-09-19T06:43:39.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f004933c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00441037c0 0x7f004419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f004933c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00441037c0 0x7f004419f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:55204/0 (socket says 172.21.15.174:55204) 2024-09-19T06:43:39.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f004933c640 1 -- 172.21.15.174:0/914018485 learned_addr learned my addr 172.21.15.174:0/914018485 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T06:43:39.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f0048b3b640 1 --2- 172.21.15.174:0/914018485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0044101c00 0x7f0044197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f0043fff640 1 --2- 172.21.15.174:0/914018485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0044102e00 0x7f00441983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f0048b3b640 1 -- 172.21.15.174:0/914018485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00441037c0 msgr2=0x7f004419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:39.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f0048b3b640 1 --2- 172.21.15.174:0/914018485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00441037c0 0x7f004419f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f0048b3b640 1 -- 172.21.15.174:0/914018485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0044102e00 msgr2=0x7f00441983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:39.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f0048b3b640 1 --2- 172.21.15.174:0/914018485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0044102e00 0x7f00441983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f0048b3b640 1 -- 172.21.15.174:0/914018485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004419fae0 con 0x7f0044101c00 2024-09-19T06:43:39.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f0043fff640 1 --2- 172.21.15.174:0/914018485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0044102e00 0x7f00441983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:39.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.899+0000 7f0048b3b640 1 --2- 172.21.15.174:0/914018485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0044101c00 0x7f0044197eb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f003800e9c0 tx=0x7f003800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:39.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.900+0000 7f0041ffb640 1 -- 172.21.15.174:0/914018485 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003800ccc0 con 0x7f0044101c00 2024-09-19T06:43:39.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.900+0000 7f004ad9f640 1 -- 172.21.15.174:0/914018485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004419fdd0 con 0x7f0044101c00 2024-09-19T06:43:39.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.900+0000 7f0041ffb640 1 -- 172.21.15.174:0/914018485 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00380035f0 con 0x7f0044101c00 2024-09-19T06:43:39.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.900+0000 7f0041ffb640 1 -- 172.21.15.174:0/914018485 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003800ce60 con 0x7f0044101c00 2024-09-19T06:43:39.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.900+0000 7f004ad9f640 1 -- 172.21.15.174:0/914018485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00441a0280 con 0x7f0044101c00 2024-09-19T06:43:39.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.901+0000 7f004ad9f640 1 -- 172.21.15.174:0/914018485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0044108fd0 con 0x7f0044101c00 2024-09-19T06:43:39.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.902+0000 7f0041ffb640 1 -- 172.21.15.174:0/914018485 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f00380037a0 con 0x7f0044101c00 2024-09-19T06:43:39.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.902+0000 7f0041ffb640 1 --2- 172.21.15.174:0/914018485 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f00340a32f0 0x7f00340a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.903+0000 7f0041ffb640 1 -- 172.21.15.174:0/914018485 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3805+0+0 (secure 0 0 0) 0x7f00380ceb60 con 0x7f0044101c00 2024-09-19T06:43:39.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.903+0000 7f0043fff640 1 --2- 172.21.15.174:0/914018485 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f00340a32f0 0x7f00340a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.904+0000 7f0043fff640 1 --2- 172.21.15.174:0/914018485 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f00340a32f0 0x7f00340a57b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f00441993d0 tx=0x7f0030005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:39.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:39.907+0000 7f0041ffb640 1 -- 172.21.15.174:0/914018485 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f003809c980 con 0x7f0044101c00 2024-09-19T06:43:40.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:40 smithi174 ceph-mon[35637]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T06:43:40.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:40 smithi174 ceph-mon[35637]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T06:43:40.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:40.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:40 smithi174 ceph-mon[35637]: from='osd.4 [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:43:40.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:40 smithi174 ceph-mon[35637]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:43:40.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:40 smithi174 ceph-mon[35637]: Detected new or changed devices on smithi134 2024-09-19T06:43:40.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:40.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:40.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:43:40.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:43:40.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:40 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi134 to 8544M 2024-09-19T06:43:40.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:40.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:40.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:43:40.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:40.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:40 smithi174 ceph-mon[35637]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:43:40.100 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:43:40.100+0000 7f004ad9f640 1 -- 172.21.15.174:0/914018485 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0044062630 con 0x7f00340a32f0 2024-09-19T06:43:40.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:40 smithi103 ceph-mon[31158]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T06:43:40.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:40 smithi103 ceph-mon[31158]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T06:43:40.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:40.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:40 smithi103 ceph-mon[31158]: from='osd.4 [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:43:40.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:40 smithi103 ceph-mon[31158]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:43:40.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:40 smithi103 ceph-mon[31158]: Detected new or changed devices on smithi134 2024-09-19T06:43:40.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:40.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:40.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:43:40.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:43:40.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:40 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi134 to 8544M 2024-09-19T06:43:40.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:40.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:40.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:43:40.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:40.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:40 smithi103 ceph-mon[31158]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:43:40.437 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4[46370]: 2024-09-19T06:43:40.057+0000 7f8523133640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T06:43:40.437 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4[46370]: 2024-09-19T06:43:40.063+0000 7f851f76d640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:43:40.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-mon[35356]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T06:43:40.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-mon[35356]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T06:43:40.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:40.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-mon[35356]: from='osd.4 [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:43:40.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-mon[35356]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:43:40.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-mon[35356]: Detected new or changed devices on smithi134 2024-09-19T06:43:40.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:40.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:40.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:43:40.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:43:40.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi134 to 8544M 2024-09-19T06:43:40.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:40.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:40.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:43:40.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:40.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:40 smithi134 ceph-mon[35356]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:43:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:41 smithi174 ceph-mon[35637]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T06:43:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:41 smithi174 ceph-mon[35637]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T06:43:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:41 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:41 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:41 smithi174 ceph-mon[35637]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:43:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:41 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:43:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:41 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:43:41.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:41 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:41.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:41 smithi103 ceph-mon[31158]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T06:43:41.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:41 smithi103 ceph-mon[31158]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T06:43:41.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:41 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:41.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:41 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:41.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:41 smithi103 ceph-mon[31158]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:43:41.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:41 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:43:41.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:41 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:43:41.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:41 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:41.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:41 smithi134 ceph-mon[35356]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T06:43:41.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:41 smithi134 ceph-mon[35356]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T06:43:41.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:41 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:41.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:41 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:41.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:41 smithi134 ceph-mon[35356]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:43:41.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:41 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:43:41.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:41 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:43:41.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:41 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:42.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:42 smithi174 ceph-mon[35637]: purged_snaps scrub starts 2024-09-19T06:43:42.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:42 smithi174 ceph-mon[35637]: purged_snaps scrub ok 2024-09-19T06:43:42.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:42 smithi174 ceph-mon[35637]: osd.4 [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968] boot 2024-09-19T06:43:42.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:42 smithi174 ceph-mon[35637]: osdmap e30: 5 total, 5 up, 5 in 2024-09-19T06:43:42.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:42 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:42.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:42 smithi174 ceph-mon[35637]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:43:42.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:42 smithi103 ceph-mon[31158]: purged_snaps scrub starts 2024-09-19T06:43:42.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:42 smithi103 ceph-mon[31158]: purged_snaps scrub ok 2024-09-19T06:43:42.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:42 smithi103 ceph-mon[31158]: osd.4 [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968] boot 2024-09-19T06:43:42.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:42 smithi103 ceph-mon[31158]: osdmap e30: 5 total, 5 up, 5 in 2024-09-19T06:43:42.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:42 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:42.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:42 smithi103 ceph-mon[31158]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:43:42.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:42 smithi134 ceph-mon[35356]: purged_snaps scrub starts 2024-09-19T06:43:42.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:42 smithi134 ceph-mon[35356]: purged_snaps scrub ok 2024-09-19T06:43:42.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:42 smithi134 ceph-mon[35356]: osd.4 [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968] boot 2024-09-19T06:43:42.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:42 smithi134 ceph-mon[35356]: osdmap e30: 5 total, 5 up, 5 in 2024-09-19T06:43:42.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:42 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:43:42.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:42 smithi134 ceph-mon[35356]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:43:43.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:43 smithi174 ceph-mon[35637]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T06:43:43.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:43 smithi174 ceph-mon[35637]: from='client.? 172.21.15.174:0/963495720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8622596b-6a06-459c-882d-50fba9e296ed"}]: dispatch 2024-09-19T06:43:43.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:43 smithi174 ceph-mon[35637]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8622596b-6a06-459c-882d-50fba9e296ed"}]: dispatch 2024-09-19T06:43:43.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:43 smithi174 ceph-mon[35637]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8622596b-6a06-459c-882d-50fba9e296ed"}]': finished 2024-09-19T06:43:43.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:43 smithi174 ceph-mon[35637]: osdmap e32: 6 total, 5 up, 6 in 2024-09-19T06:43:43.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:43 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:43:43.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:43 smithi174 ceph-mon[35637]: from='client.? 172.21.15.174:0/624920324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:43:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:43 smithi103 ceph-mon[31158]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T06:43:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:43 smithi103 ceph-mon[31158]: from='client.? 172.21.15.174:0/963495720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8622596b-6a06-459c-882d-50fba9e296ed"}]: dispatch 2024-09-19T06:43:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:43 smithi103 ceph-mon[31158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8622596b-6a06-459c-882d-50fba9e296ed"}]: dispatch 2024-09-19T06:43:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:43 smithi103 ceph-mon[31158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8622596b-6a06-459c-882d-50fba9e296ed"}]': finished 2024-09-19T06:43:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:43 smithi103 ceph-mon[31158]: osdmap e32: 6 total, 5 up, 6 in 2024-09-19T06:43:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:43 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:43:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:43 smithi103 ceph-mon[31158]: from='client.? 172.21.15.174:0/624920324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:43:43.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:43 smithi134 ceph-mon[35356]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T06:43:43.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:43 smithi134 ceph-mon[35356]: from='client.? 172.21.15.174:0/963495720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8622596b-6a06-459c-882d-50fba9e296ed"}]: dispatch 2024-09-19T06:43:43.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:43 smithi134 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8622596b-6a06-459c-882d-50fba9e296ed"}]: dispatch 2024-09-19T06:43:43.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:43 smithi134 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8622596b-6a06-459c-882d-50fba9e296ed"}]': finished 2024-09-19T06:43:43.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:43 smithi134 ceph-mon[35356]: osdmap e32: 6 total, 5 up, 6 in 2024-09-19T06:43:43.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:43 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:43:43.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:43 smithi134 ceph-mon[35356]: from='client.? 172.21.15.174:0/624920324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:43:44.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:44 smithi174 ceph-mon[35637]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T06:43:44.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:44 smithi103 ceph-mon[31158]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T06:43:44.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:44 smithi134 ceph-mon[35356]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T06:43:47.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:46 smithi174 ceph-mon[35637]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-19T06:43:47.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:46 smithi103 ceph-mon[31158]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-19T06:43:47.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:46 smithi134 ceph-mon[35356]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-19T06:43:49.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:48 smithi174 ceph-mon[35637]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T06:43:49.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:48 smithi103 ceph-mon[31158]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T06:43:49.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:48 smithi134 ceph-mon[35356]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T06:43:50.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:50 smithi134 ceph-mon[35356]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:43:50.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:50 smithi174 ceph-mon[35637]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:43:51.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:50 smithi103 ceph-mon[31158]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:43:52.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:51 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T06:43:52.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:51 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:52.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:51 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T06:43:52.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:51 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:52.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:51 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T06:43:52.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:51 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:53.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:52 smithi103 ceph-mon[31158]: Deploying daemon osd.5 on smithi174 2024-09-19T06:43:53.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:52 smithi103 ceph-mon[31158]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:43:53.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:52 smithi134 ceph-mon[35356]: Deploying daemon osd.5 on smithi174 2024-09-19T06:43:53.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:52 smithi134 ceph-mon[35356]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:43:53.311 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:52 smithi174 ceph-mon[35637]: Deploying daemon osd.5 on smithi174 2024-09-19T06:43:53.311 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:52 smithi174 ceph-mon[35637]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:43:55.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:54 smithi103 ceph-mon[31158]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:43:55.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:54 smithi134 ceph-mon[35356]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:43:55.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:54 smithi174 ceph-mon[35637]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:43:57.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:56 smithi174 ceph-mon[35637]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:43:57.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:56 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:57.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:56 smithi103 ceph-mon[31158]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:43:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:56 smithi134 ceph-mon[35356]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:43:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:56 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:58.128 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:57 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:58.128 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:57 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:58.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:57 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:58.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:57 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:58.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:57 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:58.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:57 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:43:59.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:43:58 smithi103 ceph-mon[31158]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:43:59.158 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:43:58 smithi174 ceph-mon[35637]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:43:59.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:43:58 smithi134 ceph-mon[35356]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:44:00.321 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:00 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:00.321 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:00 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:00.321 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:00 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:00.321 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:00 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:00.321 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:00 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:00.321 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:00 smithi174 ceph-mon[35637]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:44:00.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:00.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:00.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:00.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:00.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:00.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:00 smithi103 ceph-mon[31158]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:44:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:00 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:00 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:00 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:00 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:00 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:00 smithi134 ceph-mon[35356]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:44:00.669 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:00.666+0000 7f0041ffb640 1 -- 172.21.15.174:0/914018485 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0044062630 con 0x7f00340a32f0 2024-09-19T06:44:00.669 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 5 on host 'smithi174' 2024-09-19T06:44:00.671 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:00.669+0000 7f004ad9f640 1 -- 172.21.15.174:0/914018485 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f00340a32f0 msgr2=0x7f00340a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:00.671 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:00.669+0000 7f004ad9f640 1 --2- 172.21.15.174:0/914018485 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f00340a32f0 0x7f00340a57b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f00441993d0 tx=0x7f0030005f90 comp rx=0 tx=0).stop 2024-09-19T06:44:00.671 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:00.669+0000 7f004ad9f640 1 -- 172.21.15.174:0/914018485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0044101c00 msgr2=0x7f0044197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:00.671 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:00.669+0000 7f004ad9f640 1 --2- 172.21.15.174:0/914018485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0044101c00 0x7f0044197eb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f003800e9c0 tx=0x7f003800ee80 comp rx=0 tx=0).stop 2024-09-19T06:44:00.671 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:00.669+0000 7f004ad9f640 1 -- 172.21.15.174:0/914018485 shutdown_connections 2024-09-19T06:44:00.671 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:00.669+0000 7f004ad9f640 1 --2- 172.21.15.174:0/914018485 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f00340a32f0 0x7f00340a57b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:00.672 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:00.669+0000 7f004ad9f640 1 --2- 172.21.15.174:0/914018485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00441037c0 0x7f004419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:00.672 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:00.669+0000 7f004ad9f640 1 --2- 172.21.15.174:0/914018485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0044102e00 0x7f00441983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:00.672 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:00.669+0000 7f004ad9f640 1 --2- 172.21.15.174:0/914018485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0044101c00 0x7f0044197eb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:00.672 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:00.669+0000 7f004ad9f640 1 -- 172.21.15.174:0/914018485 >> 172.21.15.174:0/914018485 conn(0x7f00440fd390 msgr2=0x7f00441061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:00.672 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:00.669+0000 7f004ad9f640 1 -- 172.21.15.174:0/914018485 shutdown_connections 2024-09-19T06:44:00.672 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:00.669+0000 7f004ad9f640 1 -- 172.21.15.174:0/914018485 wait complete. 2024-09-19T06:44:01.284 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:01 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:01.284 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:01 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:01.284 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:01 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:01.286 DEBUG:teuthology.orchestra.run.smithi174:osd.5> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T06:44:01.288 INFO:tasks.cephadm:Deploying osd.6 on smithi103 with /dev/vg_nvme/lv_2... 2024-09-19T06:44:01.288 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T06:44:01.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:01 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:01.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:01 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:01.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:01 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:01.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:01 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:01.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:01 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:01.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:01 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:01.549 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:44:02.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:02 smithi103 ceph-mon[31158]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:44:02.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:02 smithi174 ceph-mon[35637]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:44:02.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:02 smithi134 ceph-mon[35356]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:44:03.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 06:44:02 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T06:44:02.936+0000 7fb828a1f640 -1 osd.5 0 log_to_monitors true 2024-09-19T06:44:03.307 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:03 smithi103 ceph-mon[31158]: from='osd.5 [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T06:44:03.307 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:03 smithi103 ceph-mon[31158]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T06:44:03.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:03 smithi134 ceph-mon[35356]: from='osd.5 [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T06:44:03.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:03 smithi134 ceph-mon[35356]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T06:44:03.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:03 smithi174 ceph-mon[35637]: from='osd.5 [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T06:44:03.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:03 smithi174 ceph-mon[35637]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T06:44:04.095 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:44:04.128 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_2 2024-09-19T06:44:04.333 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 06:44:04 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T06:44:04.102+0000 7fb82417f640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T06:44:04.333 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 06:44:04 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T06:44:04.110+0000 7fb8207b9640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:44:04.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:04 smithi174 ceph-mon[35637]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T06:44:04.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:04 smithi174 ceph-mon[35637]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T06:44:04.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:44:04.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:04 smithi174 ceph-mon[35637]: from='osd.5 [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:44:04.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:04 smithi174 ceph-mon[35637]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:44:04.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:04 smithi174 ceph-mon[35637]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:44:04.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:04.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:04.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:04.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:04.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:04.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:04.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:04.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:44:04.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:04 smithi103 ceph-mon[31158]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T06:44:04.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:04 smithi103 ceph-mon[31158]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T06:44:04.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:44:04.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:04 smithi103 ceph-mon[31158]: from='osd.5 [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:44:04.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:04 smithi103 ceph-mon[31158]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:44:04.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:04 smithi103 ceph-mon[31158]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:44:04.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:04.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:04.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:04.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:04.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:04.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:04.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:04.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:04.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:04 smithi134 ceph-mon[35356]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T06:44:04.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:04 smithi134 ceph-mon[35356]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T06:44:04.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:44:04.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:04 smithi134 ceph-mon[35356]: from='osd.5 [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:44:04.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:04 smithi134 ceph-mon[35356]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:44:04.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:04 smithi134 ceph-mon[35356]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:44:04.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:04.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:04.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:04.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:04.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:04.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:04.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:04.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:04.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.871+0000 7fb44c1c7640 1 -- 172.21.15.103:0/1047066667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb444074b40 msgr2=0x7fb444074fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:04.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.871+0000 7fb44c1c7640 1 --2- 172.21.15.103:0/1047066667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb444074b40 0x7fb444074fa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb434009900 tx=0x7fb43402a410 comp rx=0 tx=0).stop 2024-09-19T06:44:04.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.871+0000 7fb44c1c7640 1 -- 172.21.15.103:0/1047066667 shutdown_connections 2024-09-19T06:44:04.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.871+0000 7fb44c1c7640 1 --2- 172.21.15.103:0/1047066667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb444109a70 0x7fb44410bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:04.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.871+0000 7fb44c1c7640 1 --2- 172.21.15.103:0/1047066667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb444074b40 0x7fb444074fa0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:04.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.871+0000 7fb44c1c7640 1 --2- 172.21.15.103:0/1047066667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4440761a0 0x7fb444074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:04.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.871+0000 7fb44c1c7640 1 -- 172.21.15.103:0/1047066667 >> 172.21.15.103:0/1047066667 conn(0x7fb4440fd390 msgr2=0x7fb4440ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:04.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.872+0000 7fb44c1c7640 1 -- 172.21.15.103:0/1047066667 shutdown_connections 2024-09-19T06:44:04.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.872+0000 7fb44c1c7640 1 -- 172.21.15.103:0/1047066667 wait complete. 2024-09-19T06:44:04.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.872+0000 7fb44c1c7640 1 Processor -- start 2024-09-19T06:44:04.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.872+0000 7fb44c1c7640 1 -- start start 2024-09-19T06:44:04.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.872+0000 7fb44c1c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb444074b40 0x7fb44419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:04.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.872+0000 7fb44c1c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4440761a0 0x7fb44419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:04.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.872+0000 7fb44c1c7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb444109a70 0x7fb4441a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:04.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.872+0000 7fb44c1c7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb44410fb50 con 0x7fb4440761a0 2024-09-19T06:44:04.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.872+0000 7fb44c1c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb44410f9d0 con 0x7fb444074b40 2024-09-19T06:44:04.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.872+0000 7fb44c1c7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb44410fcd0 con 0x7fb444109a70 2024-09-19T06:44:04.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.872+0000 7fb449762640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4440761a0 0x7fb44419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:04.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.872+0000 7fb449762640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4440761a0 0x7fb44419a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36644/0 (socket says 172.21.15.103:36644) 2024-09-19T06:44:04.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.872+0000 7fb449762640 1 -- 172.21.15.103:0/3650163982 learned_addr learned my addr 172.21.15.103:0/3650163982 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:44:04.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.873+0000 7fb449f63640 1 --2- 172.21.15.103:0/3650163982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb444074b40 0x7fb44419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:04.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.873+0000 7fb44a764640 1 --2- 172.21.15.103:0/3650163982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb444109a70 0x7fb4441a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:04.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.873+0000 7fb449762640 1 -- 172.21.15.103:0/3650163982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb444109a70 msgr2=0x7fb4441a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:04.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.873+0000 7fb449762640 1 --2- 172.21.15.103:0/3650163982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb444109a70 0x7fb4441a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:04.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.873+0000 7fb449762640 1 -- 172.21.15.103:0/3650163982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb444074b40 msgr2=0x7fb44419a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:04.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.873+0000 7fb449762640 1 --2- 172.21.15.103:0/3650163982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb444074b40 0x7fb44419a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:04.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.873+0000 7fb449762640 1 -- 172.21.15.103:0/3650163982 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4441a1e60 con 0x7fb4440761a0 2024-09-19T06:44:04.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.873+0000 7fb449762640 1 --2- 172.21.15.103:0/3650163982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4440761a0 0x7fb44419a770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb434009230 tx=0x7fb434004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:04.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.873+0000 7fb43affd640 1 -- 172.21.15.103:0/3650163982 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43403f030 con 0x7fb4440761a0 2024-09-19T06:44:04.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.873+0000 7fb43affd640 1 -- 172.21.15.103:0/3650163982 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb434003ce0 con 0x7fb4440761a0 2024-09-19T06:44:04.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.873+0000 7fb43affd640 1 -- 172.21.15.103:0/3650163982 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb434035070 con 0x7fb4440761a0 2024-09-19T06:44:04.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.873+0000 7fb44c1c7640 1 -- 172.21.15.103:0/3650163982 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4441a20f0 con 0x7fb4440761a0 2024-09-19T06:44:04.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.873+0000 7fb44c1c7640 1 -- 172.21.15.103:0/3650163982 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4441a2600 con 0x7fb4440761a0 2024-09-19T06:44:04.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.874+0000 7fb44c1c7640 1 -- 172.21.15.103:0/3650163982 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb444108880 con 0x7fb4440761a0 2024-09-19T06:44:04.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.875+0000 7fb43affd640 1 -- 172.21.15.103:0/3650163982 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb434042030 con 0x7fb4440761a0 2024-09-19T06:44:04.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.876+0000 7fb43affd640 1 --2- 172.21.15.103:0/3650163982 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb4300a3270 0x7fb4300a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:04.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.876+0000 7fb43affd640 1 -- 172.21.15.103:0/3650163982 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4253+0+0 (secure 0 0 0) 0x7fb4340ef130 con 0x7fb4440761a0 2024-09-19T06:44:04.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.876+0000 7fb449f63640 1 --2- 172.21.15.103:0/3650163982 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb4300a3270 0x7fb4300a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:04.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.876+0000 7fb449f63640 1 --2- 172.21.15.103:0/3650163982 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb4300a3270 0x7fb4300a5730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb440009890 tx=0x7fb440008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:04.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:04.880+0000 7fb43affd640 1 -- 172.21.15.103:0/3650163982 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb4340bce30 con 0x7fb4440761a0 2024-09-19T06:44:05.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:05.054+0000 7fb44c1c7640 1 -- 172.21.15.103:0/3650163982 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb444062630 con 0x7fb4300a3270 2024-09-19T06:44:05.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:05 smithi103 ceph-mon[31158]: Detected new or changed devices on smithi174 2024-09-19T06:44:05.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:05 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-19T06:44:05.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:05 smithi103 ceph-mon[31158]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T06:44:05.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:05 smithi103 ceph-mon[31158]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T06:44:05.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:05 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:44:05.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:05 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:44:05.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:05 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:44:05.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:05 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:44:05.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:05 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:05.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:05 smithi103 ceph-mon[31158]: osd.5 [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644] boot 2024-09-19T06:44:05.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:05 smithi103 ceph-mon[31158]: osdmap e35: 6 total, 6 up, 6 in 2024-09-19T06:44:05.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:05 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:44:05.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:05 smithi134 ceph-mon[35356]: Detected new or changed devices on smithi174 2024-09-19T06:44:05.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:05 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-19T06:44:05.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:05 smithi134 ceph-mon[35356]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T06:44:05.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:05 smithi134 ceph-mon[35356]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T06:44:05.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:05 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:44:05.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:05 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:44:05.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:05 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:44:05.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:05 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:44:05.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:05 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:05.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:05 smithi134 ceph-mon[35356]: osd.5 [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644] boot 2024-09-19T06:44:05.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:05 smithi134 ceph-mon[35356]: osdmap e35: 6 total, 6 up, 6 in 2024-09-19T06:44:05.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:05 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:44:05.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:05 smithi174 ceph-mon[35637]: Detected new or changed devices on smithi174 2024-09-19T06:44:05.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:05 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-19T06:44:05.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:05 smithi174 ceph-mon[35637]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T06:44:05.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:05 smithi174 ceph-mon[35637]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T06:44:05.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:05 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:44:05.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:05 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:44:05.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:05 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:44:05.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:05 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:44:05.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:05 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:05.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:05 smithi174 ceph-mon[35637]: osd.5 [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644] boot 2024-09-19T06:44:05.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:05 smithi174 ceph-mon[35637]: osdmap e35: 6 total, 6 up, 6 in 2024-09-19T06:44:05.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:05 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:44:06.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:06 smithi103 ceph-mon[31158]: purged_snaps scrub starts 2024-09-19T06:44:06.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:06 smithi103 ceph-mon[31158]: purged_snaps scrub ok 2024-09-19T06:44:06.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:06 smithi103 ceph-mon[31158]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:44:06.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:06 smithi103 ceph-mon[31158]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:06.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:06 smithi103 ceph-mon[31158]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T06:44:06.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:06 smithi134 ceph-mon[35356]: purged_snaps scrub starts 2024-09-19T06:44:06.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:06 smithi134 ceph-mon[35356]: purged_snaps scrub ok 2024-09-19T06:44:06.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:06 smithi134 ceph-mon[35356]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:44:06.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:06 smithi134 ceph-mon[35356]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:06.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:06 smithi134 ceph-mon[35356]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T06:44:06.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:06 smithi174 ceph-mon[35637]: purged_snaps scrub starts 2024-09-19T06:44:06.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:06 smithi174 ceph-mon[35637]: purged_snaps scrub ok 2024-09-19T06:44:06.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:06 smithi174 ceph-mon[35637]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:44:06.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:06 smithi174 ceph-mon[35637]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:06.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:06 smithi174 ceph-mon[35637]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T06:44:08.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:08 smithi103 ceph-mon[31158]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T06:44:08.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:08 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1980553170' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64d858d6-a8b3-4f65-a187-af9ab1ccee0f"}]: dispatch 2024-09-19T06:44:08.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:08 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1980553170' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64d858d6-a8b3-4f65-a187-af9ab1ccee0f"}]': finished 2024-09-19T06:44:08.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:08 smithi103 ceph-mon[31158]: osdmap e38: 7 total, 6 up, 7 in 2024-09-19T06:44:08.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:08 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:08.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:08 smithi103 ceph-mon[31158]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:08.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:08 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3431267286' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:44:08.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:08 smithi134 ceph-mon[35356]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T06:44:08.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:08 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1980553170' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64d858d6-a8b3-4f65-a187-af9ab1ccee0f"}]: dispatch 2024-09-19T06:44:08.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:08 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1980553170' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64d858d6-a8b3-4f65-a187-af9ab1ccee0f"}]': finished 2024-09-19T06:44:08.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:08 smithi134 ceph-mon[35356]: osdmap e38: 7 total, 6 up, 7 in 2024-09-19T06:44:08.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:08 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:08.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:08 smithi134 ceph-mon[35356]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:08.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:08 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3431267286' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:44:08.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:08 smithi174 ceph-mon[35637]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T06:44:08.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:08 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1980553170' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64d858d6-a8b3-4f65-a187-af9ab1ccee0f"}]: dispatch 2024-09-19T06:44:08.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:08 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1980553170' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64d858d6-a8b3-4f65-a187-af9ab1ccee0f"}]': finished 2024-09-19T06:44:08.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:08 smithi174 ceph-mon[35637]: osdmap e38: 7 total, 6 up, 7 in 2024-09-19T06:44:08.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:08 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:08.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:08 smithi174 ceph-mon[35637]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:08.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:08 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3431267286' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:44:09.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:09 smithi174 ceph-mon[35637]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T06:44:09.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:09 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:09.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:09 smithi103 ceph-mon[31158]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T06:44:09.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:09 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:09.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:09 smithi134 ceph-mon[35356]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T06:44:09.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:09 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:10.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:10 smithi174 ceph-mon[35637]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:10.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:10 smithi103 ceph-mon[31158]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:10.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:10 smithi134 ceph-mon[35356]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:13.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:12 smithi103 ceph-mon[31158]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:13.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:12 smithi134 ceph-mon[35356]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:13.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:12 smithi174 ceph-mon[35637]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:15.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:14 smithi103 ceph-mon[31158]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T06:44:15.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:14 smithi134 ceph-mon[35356]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T06:44:15.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:14 smithi174 ceph-mon[35637]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T06:44:16.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:16 smithi103 ceph-mon[31158]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-19T06:44:16.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:16 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T06:44:16.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:16 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:16.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:16 smithi103 ceph-mon[31158]: Deploying daemon osd.6 on smithi103 2024-09-19T06:44:17.135 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:16 smithi174 ceph-mon[35637]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-19T06:44:17.136 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:16 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T06:44:17.136 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:16 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:17.136 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:16 smithi174 ceph-mon[35637]: Deploying daemon osd.6 on smithi103 2024-09-19T06:44:17.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:16 smithi134 ceph-mon[35356]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-19T06:44:17.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:16 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T06:44:17.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:16 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:17.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:16 smithi134 ceph-mon[35356]: Deploying daemon osd.6 on smithi103 2024-09-19T06:44:18.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:18 smithi103 ceph-mon[31158]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T06:44:19.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:18 smithi134 ceph-mon[35356]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T06:44:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:18 smithi174 ceph-mon[35637]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T06:44:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:20 smithi103 ceph-mon[31158]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T06:44:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:20 smithi134 ceph-mon[35356]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T06:44:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:20 smithi174 ceph-mon[35637]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T06:44:22.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:21 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:22.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:21 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:22.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:21 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:22.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:21 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:22.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:21 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:22.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:21 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:21 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:21 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:21 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:23.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:22 smithi103 ceph-mon[31158]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:44:23.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:22 smithi134 ceph-mon[35356]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:44:23.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:22 smithi174 ceph-mon[35637]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:44:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:24 smithi103 ceph-mon[31158]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:44:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:24 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:24 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:24 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:24 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:24 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:25.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:24 smithi134 ceph-mon[35356]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:44:25.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:24 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:25.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:24 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:25.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:24 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:25.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:24 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:25.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:24 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:25.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:24 smithi174 ceph-mon[35637]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:44:25.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:24 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:25.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:24 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:25.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:24 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:25.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:24 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:25.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:24 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:25.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:25.470+0000 7fb43affd640 1 -- 172.21.15.103:0/3650163982 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb444062630 con 0x7fb4300a3270 2024-09-19T06:44:25.472 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 6 on host 'smithi103' 2024-09-19T06:44:25.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:25.473+0000 7fb44c1c7640 1 -- 172.21.15.103:0/3650163982 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb4300a3270 msgr2=0x7fb4300a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:25.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:25.473+0000 7fb44c1c7640 1 --2- 172.21.15.103:0/3650163982 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb4300a3270 0x7fb4300a5730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb440009890 tx=0x7fb440008040 comp rx=0 tx=0).stop 2024-09-19T06:44:25.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:25.473+0000 7fb44c1c7640 1 -- 172.21.15.103:0/3650163982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4440761a0 msgr2=0x7fb44419a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:25.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:25.473+0000 7fb44c1c7640 1 --2- 172.21.15.103:0/3650163982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4440761a0 0x7fb44419a770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb434009230 tx=0x7fb434004430 comp rx=0 tx=0).stop 2024-09-19T06:44:25.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:25.473+0000 7fb44c1c7640 1 -- 172.21.15.103:0/3650163982 shutdown_connections 2024-09-19T06:44:25.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:25.473+0000 7fb44c1c7640 1 --2- 172.21.15.103:0/3650163982 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb4300a3270 0x7fb4300a5730 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:25.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:25.473+0000 7fb44c1c7640 1 --2- 172.21.15.103:0/3650163982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb444109a70 0x7fb4441a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:25.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:25.473+0000 7fb44c1c7640 1 --2- 172.21.15.103:0/3650163982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4440761a0 0x7fb44419a770 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:25.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:25.473+0000 7fb44c1c7640 1 --2- 172.21.15.103:0/3650163982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb444074b40 0x7fb44419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:25.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:25.473+0000 7fb44c1c7640 1 -- 172.21.15.103:0/3650163982 >> 172.21.15.103:0/3650163982 conn(0x7fb4440fd390 msgr2=0x7fb444105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:25.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:25.473+0000 7fb44c1c7640 1 -- 172.21.15.103:0/3650163982 shutdown_connections 2024-09-19T06:44:25.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:44:25.473+0000 7fb44c1c7640 1 -- 172.21.15.103:0/3650163982 wait complete. 2024-09-19T06:44:26.070 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:26.070 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:26.071 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:26.096 DEBUG:teuthology.orchestra.run.smithi103:osd.6> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T06:44:26.099 INFO:tasks.cephadm:Deploying osd.7 on smithi134 with /dev/vg_nvme/lv_2... 2024-09-19T06:44:26.099 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T06:44:26.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:25 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:26.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:25 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:26.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:25 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:26.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:26.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:26.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:26.358 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:44:27.103 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:26 smithi103 ceph-mon[31158]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:26 smithi134 ceph-mon[35356]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:26 smithi174 ceph-mon[35637]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:28.086 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:27 smithi103 ceph-mon[31158]: from='osd.6 [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T06:44:28.086 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 06:44:27 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T06:44:27.785+0000 7ff8b99c5640 -1 osd.6 0 log_to_monitors true 2024-09-19T06:44:28.150 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:27 smithi134 ceph-mon[35356]: from='osd.6 [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T06:44:28.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:27 smithi174 ceph-mon[35637]: from='osd.6 [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T06:44:28.989 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:44:29.014 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_2 2024-09-19T06:44:29.138 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 06:44:28 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T06:44:28.871+0000 7ff8b5125640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T06:44:29.138 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 06:44:28 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T06:44:28.881+0000 7ff8b175f640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:44:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:28 smithi103 ceph-mon[31158]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:28 smithi103 ceph-mon[31158]: from='osd.6 [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T06:44:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:28 smithi103 ceph-mon[31158]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T06:44:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:28 smithi103 ceph-mon[31158]: from='osd.6 [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T06:44:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:29.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:29.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:29.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:29.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:29.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:29.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:29.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:29.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:28 smithi134 ceph-mon[35356]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:28 smithi134 ceph-mon[35356]: from='osd.6 [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T06:44:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:28 smithi134 ceph-mon[35356]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T06:44:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:28 smithi134 ceph-mon[35356]: from='osd.6 [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T06:44:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:29.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:29.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:29.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:29.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:29.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:29.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:29.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:29.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:29.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:29.302 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:44:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:28 smithi174 ceph-mon[35637]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:28 smithi174 ceph-mon[35637]: from='osd.6 [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T06:44:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:28 smithi174 ceph-mon[35637]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T06:44:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:28 smithi174 ceph-mon[35637]: from='osd.6 [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T06:44:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:29.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:29.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:29.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:29.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:29.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:29.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.752+0000 7f67c2da0640 1 -- 172.21.15.134:0/1992473324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67bc102c40 msgr2=0x7f67bc1030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:29.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.752+0000 7f67c2da0640 1 --2- 172.21.15.134:0/1992473324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67bc102c40 0x7f67bc1030c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009900 tx=0x7f67b002a410 comp rx=0 tx=0).stop 2024-09-19T06:44:29.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.752+0000 7f67c2da0640 1 -- 172.21.15.134:0/1992473324 shutdown_connections 2024-09-19T06:44:29.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.752+0000 7f67c2da0640 1 --2- 172.21.15.134:0/1992473324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67bc103600 0x7f67bc109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:29.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.752+0000 7f67c2da0640 1 --2- 172.21.15.134:0/1992473324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67bc102c40 0x7f67bc1030c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:29.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.752+0000 7f67c2da0640 1 --2- 172.21.15.134:0/1992473324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67bc101a40 0x7f67bc101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:29.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.752+0000 7f67c2da0640 1 -- 172.21.15.134:0/1992473324 >> 172.21.15.134:0/1992473324 conn(0x7f67bc0fd1f0 msgr2=0x7f67bc0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:29.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.752+0000 7f67c2da0640 1 -- 172.21.15.134:0/1992473324 shutdown_connections 2024-09-19T06:44:29.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.752+0000 7f67c2da0640 1 -- 172.21.15.134:0/1992473324 wait complete. 2024-09-19T06:44:29.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.753+0000 7f67c2da0640 1 Processor -- start 2024-09-19T06:44:29.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.753+0000 7f67c2da0640 1 -- start start 2024-09-19T06:44:29.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.753+0000 7f67c2da0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67bc101a40 0x7f67bc197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:29.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.753+0000 7f67c2da0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67bc102c40 0x7f67bc198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:29.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.753+0000 7f67c2da0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67bc103600 0x7f67bc19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:29.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.753+0000 7f67c2da0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc10d720 con 0x7f67bc102c40 2024-09-19T06:44:29.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.753+0000 7f67c2da0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc10d5a0 con 0x7f67bc101a40 2024-09-19T06:44:29.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.753+0000 7f67c2da0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc10d8a0 con 0x7f67bc103600 2024-09-19T06:44:29.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.753+0000 7f67c1d9e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67bc101a40 0x7f67bc197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:29.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.753+0000 7f67c1d9e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67bc101a40 0x7f67bc197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:60050/0 (socket says 172.21.15.134:60050) 2024-09-19T06:44:29.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.753+0000 7f67c1d9e640 1 -- 172.21.15.134:0/3081248607 learned_addr learned my addr 172.21.15.134:0/3081248607 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-19T06:44:29.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.753+0000 7f67c159d640 1 --2- 172.21.15.134:0/3081248607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67bc102c40 0x7f67bc198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:29.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.754+0000 7f67c1d9e640 1 -- 172.21.15.134:0/3081248607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67bc103600 msgr2=0x7f67bc19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:29.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.754+0000 7f67c259f640 1 --2- 172.21.15.134:0/3081248607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67bc103600 0x7f67bc19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:29.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.754+0000 7f67c1d9e640 1 --2- 172.21.15.134:0/3081248607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67bc103600 0x7f67bc19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:29.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.754+0000 7f67c1d9e640 1 -- 172.21.15.134:0/3081248607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67bc102c40 msgr2=0x7f67bc198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:29.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.754+0000 7f67c1d9e640 1 --2- 172.21.15.134:0/3081248607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67bc102c40 0x7f67bc198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:29.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.754+0000 7f67c1d9e640 1 -- 172.21.15.134:0/3081248607 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67bc19f950 con 0x7f67bc101a40 2024-09-19T06:44:29.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.754+0000 7f67c159d640 1 --2- 172.21.15.134:0/3081248607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67bc102c40 0x7f67bc198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:44:29.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.754+0000 7f67c259f640 1 --2- 172.21.15.134:0/3081248607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67bc103600 0x7f67bc19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:44:29.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.754+0000 7f67c1d9e640 1 --2- 172.21.15.134:0/3081248607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67bc101a40 0x7f67bc197d30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f67b800c970 tx=0x7f67b800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:29.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.754+0000 7f67aeffd640 1 -- 172.21.15.134:0/3081248607 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8007c10 con 0x7f67bc101a40 2024-09-19T06:44:29.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.754+0000 7f67c2da0640 1 -- 172.21.15.134:0/3081248607 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67bc19fc40 con 0x7f67bc101a40 2024-09-19T06:44:29.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.754+0000 7f67aeffd640 1 -- 172.21.15.134:0/3081248607 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67b8007db0 con 0x7f67bc101a40 2024-09-19T06:44:29.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.754+0000 7f67aeffd640 1 -- 172.21.15.134:0/3081248607 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b800f8a0 con 0x7f67bc101a40 2024-09-19T06:44:29.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.754+0000 7f67c2da0640 1 -- 172.21.15.134:0/3081248607 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67bc1a0180 con 0x7f67bc101a40 2024-09-19T06:44:29.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.755+0000 7f67c2da0640 1 -- 172.21.15.134:0/3081248607 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67bc108d70 con 0x7f67bc101a40 2024-09-19T06:44:29.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.756+0000 7f67aeffd640 1 -- 172.21.15.134:0/3081248607 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f67b800fb20 con 0x7f67bc101a40 2024-09-19T06:44:29.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.757+0000 7f67aeffd640 1 --2- 172.21.15.134:0/3081248607 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f67a80a3240 0x7f67a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:29.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.757+0000 7f67aeffd640 1 -- 172.21.15.134:0/3081248607 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4685+0+0 (secure 0 0 0) 0x7f67b80cef10 con 0x7f67bc101a40 2024-09-19T06:44:29.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.757+0000 7f67c159d640 1 --2- 172.21.15.134:0/3081248607 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f67a80a3240 0x7f67a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:29.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.757+0000 7f67c159d640 1 --2- 172.21.15.134:0/3081248607 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f67a80a3240 0x7f67a80a5700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f67b0000c00 tx=0x7f67b0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:29.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.761+0000 7f67aeffd640 1 -- 172.21.15.134:0/3081248607 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f67b809caf0 con 0x7f67bc101a40 2024-09-19T06:44:29.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:29 smithi134 ceph-mon[35356]: Detected new or changed devices on smithi103 2024-09-19T06:44:29.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:29 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-19T06:44:29.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:29 smithi134 ceph-mon[35356]: from='osd.6 [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T06:44:29.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:29 smithi134 ceph-mon[35356]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T06:44:29.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:29 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:29.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:29 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:29.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:29.938+0000 7f67c2da0640 1 -- 172.21.15.134:0/3081248607 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f67bc062630 con 0x7f67a80a3240 2024-09-19T06:44:30.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:29 smithi103 ceph-mon[31158]: Detected new or changed devices on smithi103 2024-09-19T06:44:30.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:29 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-19T06:44:30.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:29 smithi103 ceph-mon[31158]: from='osd.6 [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T06:44:30.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:29 smithi103 ceph-mon[31158]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T06:44:30.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:29 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:30.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:29 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:29 smithi174 ceph-mon[35637]: Detected new or changed devices on smithi103 2024-09-19T06:44:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:29 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-19T06:44:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:29 smithi174 ceph-mon[35637]: from='osd.6 [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T06:44:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:29 smithi174 ceph-mon[35637]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T06:44:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:29 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:29 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:30.927 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:30 smithi134 ceph-mon[35356]: purged_snaps scrub starts 2024-09-19T06:44:30.927 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:30 smithi134 ceph-mon[35356]: purged_snaps scrub ok 2024-09-19T06:44:30.927 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:30 smithi134 ceph-mon[35356]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:30.927 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:30 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:30.927 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:30 smithi134 ceph-mon[35356]: osd.6 [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394] boot 2024-09-19T06:44:30.927 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:30 smithi134 ceph-mon[35356]: osdmap e42: 7 total, 7 up, 7 in 2024-09-19T06:44:30.927 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:30 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:30.927 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:30 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:44:30.928 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:30 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:44:30.928 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:30 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:31.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:30 smithi103 ceph-mon[31158]: purged_snaps scrub starts 2024-09-19T06:44:31.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:30 smithi103 ceph-mon[31158]: purged_snaps scrub ok 2024-09-19T06:44:31.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:30 smithi103 ceph-mon[31158]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:31.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:30 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:31.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:30 smithi103 ceph-mon[31158]: osd.6 [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394] boot 2024-09-19T06:44:31.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:30 smithi103 ceph-mon[31158]: osdmap e42: 7 total, 7 up, 7 in 2024-09-19T06:44:31.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:30 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:31.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:30 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:44:31.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:30 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:44:31.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:30 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:31.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:30 smithi174 ceph-mon[35637]: purged_snaps scrub starts 2024-09-19T06:44:31.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:30 smithi174 ceph-mon[35637]: purged_snaps scrub ok 2024-09-19T06:44:31.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:30 smithi174 ceph-mon[35637]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:44:31.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:30 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:31.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:30 smithi174 ceph-mon[35637]: osd.6 [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394] boot 2024-09-19T06:44:31.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:30 smithi174 ceph-mon[35637]: osdmap e42: 7 total, 7 up, 7 in 2024-09-19T06:44:31.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:30 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:44:31.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:30 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:44:31.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:30 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:44:31.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:30 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:31 smithi134 ceph-mon[35356]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:44:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:31 smithi134 ceph-mon[35356]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T06:44:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:31 smithi174 ceph-mon[35637]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:44:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:31 smithi174 ceph-mon[35637]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T06:44:32.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:31 smithi103 ceph-mon[31158]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:44:32.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:31 smithi103 ceph-mon[31158]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T06:44:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:32 smithi134 ceph-mon[35356]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:32 smithi134 ceph-mon[35356]: from='client.? 172.21.15.134:0/3004337953' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d405d1c6-839c-4c92-8261-4adb3163c90f"}]: dispatch 2024-09-19T06:44:33.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:32 smithi134 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d405d1c6-839c-4c92-8261-4adb3163c90f"}]: dispatch 2024-09-19T06:44:33.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:32 smithi134 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d405d1c6-839c-4c92-8261-4adb3163c90f"}]': finished 2024-09-19T06:44:33.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:32 smithi134 ceph-mon[35356]: osdmap e44: 8 total, 7 up, 8 in 2024-09-19T06:44:33.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:32 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:44:33.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:32 smithi134 ceph-mon[35356]: from='client.? 172.21.15.134:0/2721278105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:44:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:32 smithi174 ceph-mon[35637]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:32 smithi174 ceph-mon[35637]: from='client.? 172.21.15.134:0/3004337953' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d405d1c6-839c-4c92-8261-4adb3163c90f"}]: dispatch 2024-09-19T06:44:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:32 smithi174 ceph-mon[35637]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d405d1c6-839c-4c92-8261-4adb3163c90f"}]: dispatch 2024-09-19T06:44:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:32 smithi174 ceph-mon[35637]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d405d1c6-839c-4c92-8261-4adb3163c90f"}]': finished 2024-09-19T06:44:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:32 smithi174 ceph-mon[35637]: osdmap e44: 8 total, 7 up, 8 in 2024-09-19T06:44:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:32 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:44:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:32 smithi174 ceph-mon[35637]: from='client.? 172.21.15.134:0/2721278105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:44:33.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:32 smithi103 ceph-mon[31158]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:33.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:32 smithi103 ceph-mon[31158]: from='client.? 172.21.15.134:0/3004337953' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d405d1c6-839c-4c92-8261-4adb3163c90f"}]: dispatch 2024-09-19T06:44:33.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:32 smithi103 ceph-mon[31158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d405d1c6-839c-4c92-8261-4adb3163c90f"}]: dispatch 2024-09-19T06:44:33.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:32 smithi103 ceph-mon[31158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d405d1c6-839c-4c92-8261-4adb3163c90f"}]': finished 2024-09-19T06:44:33.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:32 smithi103 ceph-mon[31158]: osdmap e44: 8 total, 7 up, 8 in 2024-09-19T06:44:33.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:32 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:44:33.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:32 smithi103 ceph-mon[31158]: from='client.? 172.21.15.134:0/2721278105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:44:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:34 smithi134 ceph-mon[35356]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:35.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:34 smithi174 ceph-mon[35637]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:35.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:34 smithi103 ceph-mon[31158]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:36 smithi134 ceph-mon[35356]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:37.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:36 smithi174 ceph-mon[35637]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:37.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:36 smithi103 ceph-mon[31158]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:39.106 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:38 smithi134 ceph-mon[35356]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:39.190 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:38 smithi103 ceph-mon[31158]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:39.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:38 smithi174 ceph-mon[35637]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:41.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:40 smithi134 ceph-mon[35356]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:40 smithi174 ceph-mon[35637]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:41.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:40 smithi103 ceph-mon[31158]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:42.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:41 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T06:44:42.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:41 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:42.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:41 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T06:44:42.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:41 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:42.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:41 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T06:44:42.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:41 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:43.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:42 smithi134 ceph-mon[35356]: Deploying daemon osd.7 on smithi134 2024-09-19T06:44:43.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:42 smithi134 ceph-mon[35356]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:43.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:42 smithi174 ceph-mon[35637]: Deploying daemon osd.7 on smithi134 2024-09-19T06:44:43.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:42 smithi174 ceph-mon[35637]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:42 smithi103 ceph-mon[31158]: Deploying daemon osd.7 on smithi134 2024-09-19T06:44:43.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:42 smithi103 ceph-mon[31158]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:45.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:44 smithi134 ceph-mon[35356]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:45.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:44 smithi174 ceph-mon[35637]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:45.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:44 smithi103 ceph-mon[31158]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:47.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:46 smithi134 ceph-mon[35356]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:47.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:46 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:47.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:46 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:47.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:46 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:47.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:46 smithi174 ceph-mon[35637]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:47.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:46 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:47.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:46 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:47.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:46 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:47.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:46 smithi103 ceph-mon[31158]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:47.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:46 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:47.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:46 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:47.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:46 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:48.237 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:47 smithi134 ceph-mon[35356]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:48.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:47 smithi174 ceph-mon[35637]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:48.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:47 smithi103 ceph-mon[31158]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:50.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:50.434+0000 7f67aeffd640 1 -- 172.21.15.134:0/3081248607 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f67bc062630 con 0x7f67a80a3240 2024-09-19T06:44:50.437 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 7 on host 'smithi134' 2024-09-19T06:44:50.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:50.437+0000 7f67c2da0640 1 -- 172.21.15.134:0/3081248607 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f67a80a3240 msgr2=0x7f67a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:50.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:50.437+0000 7f67c2da0640 1 --2- 172.21.15.134:0/3081248607 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f67a80a3240 0x7f67a80a5700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f67b0000c00 tx=0x7f67b0031400 comp rx=0 tx=0).stop 2024-09-19T06:44:50.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:50.437+0000 7f67c2da0640 1 -- 172.21.15.134:0/3081248607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67bc101a40 msgr2=0x7f67bc197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:50.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:50.437+0000 7f67c2da0640 1 --2- 172.21.15.134:0/3081248607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67bc101a40 0x7f67bc197d30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f67b800c970 tx=0x7f67b800ce30 comp rx=0 tx=0).stop 2024-09-19T06:44:50.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:50.437+0000 7f67c2da0640 1 -- 172.21.15.134:0/3081248607 shutdown_connections 2024-09-19T06:44:50.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:50.437+0000 7f67c2da0640 1 --2- 172.21.15.134:0/3081248607 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f67a80a3240 0x7f67a80a5700 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:50.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:50.437+0000 7f67c2da0640 1 --2- 172.21.15.134:0/3081248607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67bc103600 0x7f67bc19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:50.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:50.437+0000 7f67c2da0640 1 --2- 172.21.15.134:0/3081248607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67bc102c40 0x7f67bc198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:50.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:50.437+0000 7f67c2da0640 1 --2- 172.21.15.134:0/3081248607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67bc101a40 0x7f67bc197d30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:50.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:50.437+0000 7f67c2da0640 1 -- 172.21.15.134:0/3081248607 >> 172.21.15.134:0/3081248607 conn(0x7f67bc0fd1f0 msgr2=0x7f67bc105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:50.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:50.437+0000 7f67c2da0640 1 -- 172.21.15.134:0/3081248607 shutdown_connections 2024-09-19T06:44:50.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:44:50.437+0000 7f67c2da0640 1 -- 172.21.15.134:0/3081248607 wait complete. 2024-09-19T06:44:50.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:50.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:50.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:50 smithi134 ceph-mon[35356]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:50.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:50.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:50.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:50.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:50.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:50.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:50.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:50.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:50.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:50 smithi174 ceph-mon[35637]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:50.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:50.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:50.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:50.834 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:50.834 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:50.834 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:50.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:50.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:50.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:50 smithi103 ceph-mon[31158]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:50.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:50.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:50.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:50.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:50.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:50.889 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:51.093 DEBUG:teuthology.orchestra.run.smithi134:osd.7> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T06:44:51.095 INFO:tasks.cephadm:Deploying osd.8 on smithi174 with /dev/vg_nvme/lv_2... 2024-09-19T06:44:51.095 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T06:44:51.349 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:44:53.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:52 smithi174 ceph-mon[35637]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:53.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:52 smithi103 ceph-mon[31158]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:53.169 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:52 smithi134 ceph-mon[35356]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:53.169 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 06:44:52 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T06:44:52.873+0000 7f82308d3640 -1 osd.7 0 log_to_monitors true 2024-09-19T06:44:53.866 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:44:53.872 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:53 smithi134 ceph-mon[35356]: from='osd.7 [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T06:44:53.872 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:53 smithi134 ceph-mon[35356]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T06:44:53.893 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_2 2024-09-19T06:44:54.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:53 smithi103 ceph-mon[31158]: from='osd.7 [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T06:44:54.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:53 smithi103 ceph-mon[31158]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T06:44:54.151 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:44:54.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:53 smithi174 ceph-mon[35637]: from='osd.7 [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T06:44:54.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:53 smithi174 ceph-mon[35637]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T06:44:54.634 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.616+0000 7f123b588640 1 -- 172.21.15.174:0/1881002906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12341037c0 msgr2=0x7f123410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:54.634 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.616+0000 7f123b588640 1 --2- 172.21.15.174:0/1881002906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12341037c0 0x7f123410a050 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f122c009ca0 tx=0x7f122c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.634 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.616+0000 7f123b588640 1 -- 172.21.15.174:0/1881002906 shutdown_connections 2024-09-19T06:44:54.634 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.616+0000 7f123b588640 1 --2- 172.21.15.174:0/1881002906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12341037c0 0x7f123410a050 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.634 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.616+0000 7f123b588640 1 --2- 172.21.15.174:0/1881002906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1234102e00 0x7f1234103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.634 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.616+0000 7f123b588640 1 --2- 172.21.15.174:0/1881002906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1234101c00 0x7f1234102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.634 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.616+0000 7f123b588640 1 -- 172.21.15.174:0/1881002906 >> 172.21.15.174:0/1881002906 conn(0x7f12340fd390 msgr2=0x7f12340ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:54.634 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.616+0000 7f123b588640 1 -- 172.21.15.174:0/1881002906 shutdown_connections 2024-09-19T06:44:54.634 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.616+0000 7f123b588640 1 -- 172.21.15.174:0/1881002906 wait complete. 2024-09-19T06:44:54.634 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f123b588640 1 Processor -- start 2024-09-19T06:44:54.635 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f123b588640 1 -- start start 2024-09-19T06:44:54.635 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f123b588640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1234101c00 0x7f1234197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.635 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f123b588640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1234102e00 0x7f1234198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.635 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f123b588640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12341037c0 0x7f123419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.635 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f123b588640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f123410d860 con 0x7f1234102e00 2024-09-19T06:44:54.635 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f123b588640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f123410d6e0 con 0x7f1234101c00 2024-09-19T06:44:54.635 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f123b588640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f123410d9e0 con 0x7f12341037c0 2024-09-19T06:44:54.635 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f1239b25640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12341037c0 0x7f123419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.635 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f1239b25640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12341037c0 0x7f123419f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:39120/0 (socket says 172.21.15.174:39120) 2024-09-19T06:44:54.635 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f1239b25640 1 -- 172.21.15.174:0/557163114 learned_addr learned my addr 172.21.15.174:0/557163114 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T06:44:54.635 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f1239324640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1234101c00 0x7f1234197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.635 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f1238b23640 1 --2- 172.21.15.174:0/557163114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1234102e00 0x7f1234198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.636 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f1239b25640 1 -- 172.21.15.174:0/557163114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1234101c00 msgr2=0x7f1234197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:54.636 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f1239b25640 1 --2- 172.21.15.174:0/557163114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1234101c00 0x7f1234197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.636 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f1239b25640 1 -- 172.21.15.174:0/557163114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1234102e00 msgr2=0x7f1234198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:54.636 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f1239b25640 1 --2- 172.21.15.174:0/557163114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1234102e00 0x7f1234198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.636 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f1239b25640 1 -- 172.21.15.174:0/557163114 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123419fa80 con 0x7f12341037c0 2024-09-19T06:44:54.636 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.617+0000 7f1239b25640 1 --2- 172.21.15.174:0/557163114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12341037c0 0x7f123419f410 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f122c005e30 tx=0x7f122c031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:54.636 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.618+0000 7f122a7fc640 1 -- 172.21.15.174:0/557163114 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c003c30 con 0x7f12341037c0 2024-09-19T06:44:54.636 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.618+0000 7f122a7fc640 1 -- 172.21.15.174:0/557163114 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f122c003dd0 con 0x7f12341037c0 2024-09-19T06:44:54.636 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.618+0000 7f123b588640 1 -- 172.21.15.174:0/557163114 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123419fd10 con 0x7f12341037c0 2024-09-19T06:44:54.636 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.618+0000 7f122a7fc640 1 -- 172.21.15.174:0/557163114 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c02fde0 con 0x7f12341037c0 2024-09-19T06:44:54.636 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.618+0000 7f123b588640 1 -- 172.21.15.174:0/557163114 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12341a0240 con 0x7f12341037c0 2024-09-19T06:44:54.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.619+0000 7f123b588640 1 -- 172.21.15.174:0/557163114 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1234108fb0 con 0x7f12341037c0 2024-09-19T06:44:54.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.620+0000 7f122a7fc640 1 -- 172.21.15.174:0/557163114 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f122c004050 con 0x7f12341037c0 2024-09-19T06:44:54.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.621+0000 7f122a7fc640 1 --2- 172.21.15.174:0/557163114 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f12200a32f0 0x7f12200a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.621+0000 7f122a7fc640 1 -- 172.21.15.174:0/557163114 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5101+0+0 (secure 0 0 0) 0x7f122c0f0000 con 0x7f12341037c0 2024-09-19T06:44:54.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.621+0000 7f1239324640 1 --2- 172.21.15.174:0/557163114 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f12200a32f0 0x7f12200a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.621+0000 7f1239324640 1 --2- 172.21.15.174:0/557163114 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f12200a32f0 0x7f12200a57b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1234102c60 tx=0x7f123000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:54.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.624+0000 7f122a7fc640 1 -- 172.21.15.174:0/557163114 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f122c0bd9b0 con 0x7f12341037c0 2024-09-19T06:44:54.801 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:44:54.800+0000 7f123b588640 1 -- 172.21.15.174:0/557163114 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1234062630 con 0x7f12200a32f0 2024-09-19T06:44:55.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:55.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T06:44:55.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T06:44:55.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:44:55.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: from='osd.7 [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:44:55.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:44:55.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: Detected new or changed devices on smithi134 2024-09-19T06:44:55.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:55.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:55.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:55.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:55.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:55.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:55.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:55.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:55.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:55.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:44:55.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:44:55.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:55.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:55.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T06:44:55.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T06:44:55.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:44:55.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: from='osd.7 [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:44:55.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:44:55.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: Detected new or changed devices on smithi134 2024-09-19T06:44:55.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:55.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:55.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:55.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:55.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:55.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:55.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:55.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:55.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:55.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:44:55.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:44:55.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:55.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:55.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T06:44:55.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T06:44:55.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:44:55.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: from='osd.7 [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:44:55.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:44:55.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: Detected new or changed devices on smithi134 2024-09-19T06:44:55.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:55.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:55.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:55.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:55.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:44:55.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:55.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:55.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:55.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:44:55.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:44:55.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:44:55.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:55.937 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 06:44:55 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T06:44:55.640+0000 7f822c033640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T06:44:55.937 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 06:44:55 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T06:44:55.654+0000 7f8224665640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:44:55.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:55 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi134 to 5696M 2024-09-19T06:44:55.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:55 smithi134 ceph-mon[35356]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:44:55.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:55 smithi134 ceph-mon[35356]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T06:44:55.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:55 smithi134 ceph-mon[35356]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T06:44:55.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:44:55.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:44:55.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:55 smithi134 ceph-mon[35356]: from='osd.7 ' entity='osd.7' 2024-09-19T06:44:55.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:55 smithi134 ceph-mon[35356]: osd.7 [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765] boot 2024-09-19T06:44:55.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:55 smithi134 ceph-mon[35356]: osdmap e47: 8 total, 8 up, 8 in 2024-09-19T06:44:55.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:44:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:55 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi134 to 5696M 2024-09-19T06:44:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:55 smithi103 ceph-mon[31158]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:44:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:55 smithi103 ceph-mon[31158]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T06:44:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:55 smithi103 ceph-mon[31158]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T06:44:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:44:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:44:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:55 smithi103 ceph-mon[31158]: from='osd.7 ' entity='osd.7' 2024-09-19T06:44:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:55 smithi103 ceph-mon[31158]: osd.7 [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765] boot 2024-09-19T06:44:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:55 smithi103 ceph-mon[31158]: osdmap e47: 8 total, 8 up, 8 in 2024-09-19T06:44:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:44:56.139 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:55 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi134 to 5696M 2024-09-19T06:44:56.139 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:55 smithi174 ceph-mon[35637]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:44:56.139 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:55 smithi174 ceph-mon[35637]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T06:44:56.139 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:55 smithi174 ceph-mon[35637]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T06:44:56.139 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:44:56.139 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:44:56.140 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:55 smithi174 ceph-mon[35637]: from='osd.7 ' entity='osd.7' 2024-09-19T06:44:56.140 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:55 smithi174 ceph-mon[35637]: osd.7 [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765] boot 2024-09-19T06:44:56.140 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:55 smithi174 ceph-mon[35637]: osdmap e47: 8 total, 8 up, 8 in 2024-09-19T06:44:56.140 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:44:57.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:56 smithi174 ceph-mon[35637]: purged_snaps scrub starts 2024-09-19T06:44:57.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:56 smithi174 ceph-mon[35637]: purged_snaps scrub ok 2024-09-19T06:44:57.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:56 smithi174 ceph-mon[35637]: pgmap v164: 1 pgs: 1 unknown; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:57.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:56 smithi174 ceph-mon[35637]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T06:44:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:56 smithi103 ceph-mon[31158]: purged_snaps scrub starts 2024-09-19T06:44:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:56 smithi103 ceph-mon[31158]: purged_snaps scrub ok 2024-09-19T06:44:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:56 smithi103 ceph-mon[31158]: pgmap v164: 1 pgs: 1 unknown; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:56 smithi103 ceph-mon[31158]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T06:44:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:56 smithi134 ceph-mon[35356]: purged_snaps scrub starts 2024-09-19T06:44:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:56 smithi134 ceph-mon[35356]: purged_snaps scrub ok 2024-09-19T06:44:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:56 smithi134 ceph-mon[35356]: pgmap v164: 1 pgs: 1 unknown; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:44:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:56 smithi134 ceph-mon[35356]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T06:44:58.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:57 smithi174 ceph-mon[35637]: from='client.? 172.21.15.174:0/159336853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c452cd9-816c-4f77-9368-2cd4bbeee4ff"}]: dispatch 2024-09-19T06:44:58.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:57 smithi174 ceph-mon[35637]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c452cd9-816c-4f77-9368-2cd4bbeee4ff"}]: dispatch 2024-09-19T06:44:58.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:57 smithi174 ceph-mon[35637]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c452cd9-816c-4f77-9368-2cd4bbeee4ff"}]': finished 2024-09-19T06:44:58.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:57 smithi174 ceph-mon[35637]: osdmap e49: 9 total, 8 up, 9 in 2024-09-19T06:44:58.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:57 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:44:58.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:57 smithi174 ceph-mon[35637]: from='client.? 172.21.15.174:0/3450841718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:44:58.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:57 smithi103 ceph-mon[31158]: from='client.? 172.21.15.174:0/159336853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c452cd9-816c-4f77-9368-2cd4bbeee4ff"}]: dispatch 2024-09-19T06:44:58.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:57 smithi103 ceph-mon[31158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c452cd9-816c-4f77-9368-2cd4bbeee4ff"}]: dispatch 2024-09-19T06:44:58.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:57 smithi103 ceph-mon[31158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c452cd9-816c-4f77-9368-2cd4bbeee4ff"}]': finished 2024-09-19T06:44:58.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:57 smithi103 ceph-mon[31158]: osdmap e49: 9 total, 8 up, 9 in 2024-09-19T06:44:58.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:57 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:44:58.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:57 smithi103 ceph-mon[31158]: from='client.? 172.21.15.174:0/3450841718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:44:58.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:57 smithi134 ceph-mon[35356]: from='client.? 172.21.15.174:0/159336853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c452cd9-816c-4f77-9368-2cd4bbeee4ff"}]: dispatch 2024-09-19T06:44:58.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:57 smithi134 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c452cd9-816c-4f77-9368-2cd4bbeee4ff"}]: dispatch 2024-09-19T06:44:58.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:57 smithi134 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c452cd9-816c-4f77-9368-2cd4bbeee4ff"}]': finished 2024-09-19T06:44:58.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:57 smithi134 ceph-mon[35356]: osdmap e49: 9 total, 8 up, 9 in 2024-09-19T06:44:58.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:57 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:44:58.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:57 smithi134 ceph-mon[35356]: from='client.? 172.21.15.174:0/3450841718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:44:59.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:58 smithi103 ceph-mon[31158]: pgmap v168: 1 pgs: 1 unknown; 577 KiB data, 380 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:44:59.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:58 smithi103 ceph-mon[31158]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T06:44:59.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:44:58 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:44:59.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:58 smithi134 ceph-mon[35356]: pgmap v168: 1 pgs: 1 unknown; 577 KiB data, 380 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:44:59.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:58 smithi134 ceph-mon[35356]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T06:44:59.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:44:58 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:44:59.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:58 smithi174 ceph-mon[35637]: pgmap v168: 1 pgs: 1 unknown; 577 KiB data, 380 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:44:59.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:58 smithi174 ceph-mon[35637]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T06:44:59.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:44:58 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:00.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:00 smithi174 ceph-mon[35637]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T06:45:00.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:00 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:00.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:00 smithi174 ceph-mon[35637]: pgmap v171: 1 pgs: 1 unknown; 577 KiB data, 380 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:00.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:00 smithi103 ceph-mon[31158]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T06:45:00.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:00 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:00.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:00 smithi103 ceph-mon[31158]: pgmap v171: 1 pgs: 1 unknown; 577 KiB data, 380 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:00.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:00 smithi134 ceph-mon[35356]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T06:45:00.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:00 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:00.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:00 smithi134 ceph-mon[35356]: pgmap v171: 1 pgs: 1 unknown; 577 KiB data, 380 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:03.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:02 smithi174 ceph-mon[35637]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:03.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:02 smithi103 ceph-mon[31158]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:03.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:02 smithi134 ceph-mon[35356]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:05.075 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:04 smithi174 ceph-mon[35637]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:05.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:04 smithi103 ceph-mon[31158]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:05.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:04 smithi134 ceph-mon[35356]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:06.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:05 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T06:45:06.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:05 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:06.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:05 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T06:45:06.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:05 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:06.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:05 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T06:45:06.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:05 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:07.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:06 smithi174 ceph-mon[35637]: Deploying daemon osd.8 on smithi174 2024-09-19T06:45:07.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:06 smithi174 ceph-mon[35637]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:07.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:06 smithi103 ceph-mon[31158]: Deploying daemon osd.8 on smithi174 2024-09-19T06:45:07.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:06 smithi103 ceph-mon[31158]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:07.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:06 smithi134 ceph-mon[35356]: Deploying daemon osd.8 on smithi174 2024-09-19T06:45:07.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:06 smithi134 ceph-mon[35356]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:08.927 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:08 smithi174 ceph-mon[35637]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:09.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:08 smithi103 ceph-mon[31158]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:09.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:08 smithi134 ceph-mon[35356]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:11.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:10 smithi103 ceph-mon[31158]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:11.176 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:10 smithi174 ceph-mon[35637]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:11.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:10 smithi134 ceph-mon[35356]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:11.921 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:45:11.921 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:11.921 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:12.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:11 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:45:12.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:11 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:12.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:11 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:12.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:11 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:45:12.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:11 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:12.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:11 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:13.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:12 smithi103 ceph-mon[31158]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:13.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:12 smithi134 ceph-mon[35356]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:13.301 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:12 smithi174 ceph-mon[35637]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:14.898 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:45:14.896+0000 7f122a7fc640 1 -- 172.21.15.174:0/557163114 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1234062630 con 0x7f12200a32f0 2024-09-19T06:45:14.898 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 8 on host 'smithi174' 2024-09-19T06:45:14.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:45:14.898+0000 7f123b588640 1 -- 172.21.15.174:0/557163114 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f12200a32f0 msgr2=0x7f12200a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:14.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:45:14.898+0000 7f123b588640 1 --2- 172.21.15.174:0/557163114 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f12200a32f0 0x7f12200a57b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1234102c60 tx=0x7f123000b390 comp rx=0 tx=0).stop 2024-09-19T06:45:14.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:45:14.898+0000 7f123b588640 1 -- 172.21.15.174:0/557163114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12341037c0 msgr2=0x7f123419f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:14.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:45:14.898+0000 7f123b588640 1 --2- 172.21.15.174:0/557163114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12341037c0 0x7f123419f410 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f122c005e30 tx=0x7f122c031ee0 comp rx=0 tx=0).stop 2024-09-19T06:45:14.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:45:14.898+0000 7f123b588640 1 -- 172.21.15.174:0/557163114 shutdown_connections 2024-09-19T06:45:14.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:45:14.898+0000 7f123b588640 1 --2- 172.21.15.174:0/557163114 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f12200a32f0 0x7f12200a57b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:14.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:45:14.898+0000 7f123b588640 1 --2- 172.21.15.174:0/557163114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12341037c0 0x7f123419f410 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:14.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:45:14.898+0000 7f123b588640 1 --2- 172.21.15.174:0/557163114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1234102e00 0x7f1234198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:14.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:45:14.898+0000 7f123b588640 1 --2- 172.21.15.174:0/557163114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1234101c00 0x7f1234197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:14.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:45:14.898+0000 7f123b588640 1 -- 172.21.15.174:0/557163114 >> 172.21.15.174:0/557163114 conn(0x7f12340fd390 msgr2=0x7f1234106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:14.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:45:14.898+0000 7f123b588640 1 -- 172.21.15.174:0/557163114 shutdown_connections 2024-09-19T06:45:14.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:45:14.898+0000 7f123b588640 1 -- 172.21.15.174:0/557163114 wait complete. 2024-09-19T06:45:15.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:14 smithi103 ceph-mon[31158]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:15.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:14 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:15.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:14 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:15.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:14 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:15.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:14 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:45:15.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:14 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:15.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:14 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:45:15.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:14 smithi134 ceph-mon[35356]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:15.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:14 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:15.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:14 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:15.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:14 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:15.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:14 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:45:15.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:14 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:15.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:14 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:45:15.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:14 smithi174 ceph-mon[35637]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:15.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:14 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:15.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:14 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:15.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:14 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:15.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:14 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:45:15.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:14 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:15.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:14 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:45:15.661 DEBUG:teuthology.orchestra.run.smithi174:osd.8> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T06:45:15.664 INFO:tasks.cephadm:Deploying osd.9 on smithi103 with /dev/vg_nvme/lv_1... 2024-09-19T06:45:15.664 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T06:45:15.888 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:15.888 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:15.927 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:45:16.047 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:16.047 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:16.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:16.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:16.983 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:16 smithi174 ceph-mon[35637]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:17.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:16 smithi103 ceph-mon[31158]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:17.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:16 smithi134 ceph-mon[35356]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:17.678 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 06:45:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T06:45:17.541+0000 7f2f346a5640 -1 osd.8 0 log_to_monitors true 2024-09-19T06:45:18.000 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:17 smithi174 ceph-mon[35637]: from='osd.8 [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T06:45:18.000 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:17 smithi174 ceph-mon[35637]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T06:45:18.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:17 smithi103 ceph-mon[31158]: from='osd.8 [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T06:45:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:17 smithi103 ceph-mon[31158]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T06:45:18.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:17 smithi134 ceph-mon[35356]: from='osd.8 [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T06:45:18.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:17 smithi134 ceph-mon[35356]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T06:45:18.609 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:45:18.637 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_1 2024-09-19T06:45:18.897 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:45:19.178 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:18 smithi103 ceph-mon[31158]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:19.178 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:18 smithi103 ceph-mon[31158]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T06:45:19.179 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:18 smithi103 ceph-mon[31158]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T06:45:19.179 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:18 smithi103 ceph-mon[31158]: from='osd.8 [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:45:19.179 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:18 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:19.179 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:18 smithi103 ceph-mon[31158]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:45:19.179 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:18 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:19.179 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:18 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:19.179 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:18 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:19.179 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:18 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:19.179 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:18 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:19.179 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:18 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:19.179 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:18 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:19.179 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:18 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:45:19.179 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:18 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:19.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:18 smithi134 ceph-mon[35356]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:19.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:18 smithi134 ceph-mon[35356]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T06:45:19.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:18 smithi134 ceph-mon[35356]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T06:45:19.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:18 smithi134 ceph-mon[35356]: from='osd.8 [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:45:19.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:18 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:19.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:18 smithi134 ceph-mon[35356]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:45:19.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:18 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:19.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:18 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:19.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:18 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:19.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:18 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:19.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:18 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:19.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:18 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:19.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:18 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:19.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:18 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:45:19.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:18 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:19.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T06:45:18.910+0000 7f2f2fe05640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T06:45:19.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T06:45:18.923+0000 7f2f2c43f640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:45:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-mon[35637]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-mon[35637]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T06:45:19.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-mon[35637]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T06:45:19.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-mon[35637]: from='osd.8 [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:45:19.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:19.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-mon[35637]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:45:19.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:19.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:19.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:19.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:19.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:19.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:19.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:19.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:45:19.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:18 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:19.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.526+0000 7f28a1905640 1 -- 172.21.15.103:0/491528296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f289c1037c0 msgr2=0x7f289c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:19.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.526+0000 7f28a1905640 1 --2- 172.21.15.103:0/491528296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f289c1037c0 0x7f289c10a050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2894009ca0 tx=0x7f289402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:45:19.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.526+0000 7f28a1905640 1 -- 172.21.15.103:0/491528296 shutdown_connections 2024-09-19T06:45:19.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.526+0000 7f28a1905640 1 --2- 172.21.15.103:0/491528296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f289c1037c0 0x7f289c10a050 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:19.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.526+0000 7f28a1905640 1 --2- 172.21.15.103:0/491528296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c102e00 0x7f289c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:19.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.526+0000 7f28a1905640 1 --2- 172.21.15.103:0/491528296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f289c101c00 0x7f289c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:19.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.526+0000 7f28a1905640 1 -- 172.21.15.103:0/491528296 >> 172.21.15.103:0/491528296 conn(0x7f289c0fd390 msgr2=0x7f289c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:19.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.526+0000 7f28a1905640 1 -- 172.21.15.103:0/491528296 shutdown_connections 2024-09-19T06:45:19.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.526+0000 7f28a1905640 1 -- 172.21.15.103:0/491528296 wait complete. 2024-09-19T06:45:19.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.527+0000 7f28a1905640 1 Processor -- start 2024-09-19T06:45:19.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.527+0000 7f28a1905640 1 -- start start 2024-09-19T06:45:19.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.527+0000 7f28a1905640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f289c101c00 0x7f289c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:19.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.527+0000 7f28a1905640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f289c102e00 0x7f289c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:19.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.527+0000 7f28a1905640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c1037c0 0x7f289c19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:19.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.527+0000 7f28a1905640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f289c10d860 con 0x7f289c101c00 2024-09-19T06:45:19.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.527+0000 7f28a1905640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f289c10d6e0 con 0x7f289c1037c0 2024-09-19T06:45:19.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.527+0000 7f28a1905640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f289c10d9e0 con 0x7f289c102e00 2024-09-19T06:45:19.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.527+0000 7f289affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f289c101c00 0x7f289c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:19.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.527+0000 7f289affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f289c101c00 0x7f289c197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48064/0 (socket says 172.21.15.103:48064) 2024-09-19T06:45:19.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.527+0000 7f289affd640 1 -- 172.21.15.103:0/1908213505 learned_addr learned my addr 172.21.15.103:0/1908213505 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:45:19.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.527+0000 7f289a7fc640 1 --2- 172.21.15.103:0/1908213505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f289c102e00 0x7f289c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:19.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.528+0000 7f289b7fe640 1 --2- 172.21.15.103:0/1908213505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c1037c0 0x7f289c19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:19.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.528+0000 7f289a7fc640 1 -- 172.21.15.103:0/1908213505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c1037c0 msgr2=0x7f289c19f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:19.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.528+0000 7f289a7fc640 1 --2- 172.21.15.103:0/1908213505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c1037c0 0x7f289c19f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:19.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.528+0000 7f289a7fc640 1 -- 172.21.15.103:0/1908213505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f289c101c00 msgr2=0x7f289c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:19.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.528+0000 7f289a7fc640 1 --2- 172.21.15.103:0/1908213505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f289c101c00 0x7f289c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:19.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.528+0000 7f289a7fc640 1 -- 172.21.15.103:0/1908213505 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289c19fab0 con 0x7f289c102e00 2024-09-19T06:45:19.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.528+0000 7f289affd640 1 --2- 172.21.15.103:0/1908213505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f289c101c00 0x7f289c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:19.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.528+0000 7f289a7fc640 1 --2- 172.21.15.103:0/1908213505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f289c102e00 0x7f289c1983c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f289000c970 tx=0x7f289000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:19.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.528+0000 7f28a0903640 1 -- 172.21.15.103:0/1908213505 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890007bb0 con 0x7f289c102e00 2024-09-19T06:45:19.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.528+0000 7f28a0903640 1 -- 172.21.15.103:0/1908213505 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2890007d50 con 0x7f289c102e00 2024-09-19T06:45:19.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.528+0000 7f28a1905640 1 -- 172.21.15.103:0/1908213505 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289c19fda0 con 0x7f289c102e00 2024-09-19T06:45:19.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.528+0000 7f28a0903640 1 -- 172.21.15.103:0/1908213505 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890004e60 con 0x7f289c102e00 2024-09-19T06:45:19.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.528+0000 7f28a1905640 1 -- 172.21.15.103:0/1908213505 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f289c1a0250 con 0x7f289c102e00 2024-09-19T06:45:19.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.529+0000 7f28a1905640 1 -- 172.21.15.103:0/1908213505 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f289c109000 con 0x7f289c102e00 2024-09-19T06:45:19.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.530+0000 7f28a0903640 1 -- 172.21.15.103:0/1908213505 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2890005000 con 0x7f289c102e00 2024-09-19T06:45:19.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.531+0000 7f28a0903640 1 --2- 172.21.15.103:0/1908213505 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f28840a3270 0x7f28840a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:19.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.531+0000 7f289affd640 1 --2- 172.21.15.103:0/1908213505 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f28840a3270 0x7f28840a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:19.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.531+0000 7f28a0903640 1 -- 172.21.15.103:0/1908213505 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5549+0+0 (secure 0 0 0) 0x7f28900cebd0 con 0x7f289c102e00 2024-09-19T06:45:19.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.531+0000 7f289affd640 1 --2- 172.21.15.103:0/1908213505 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f28840a3270 0x7f28840a5730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f288c0058c0 tx=0x7f288c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:19.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.534+0000 7f28a0903640 1 -- 172.21.15.103:0/1908213505 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2890014030 con 0x7f289c102e00 2024-09-19T06:45:19.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:19.714+0000 7f28a1905640 1 -- 172.21.15.103:0/1908213505 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f289c062630 con 0x7f28840a3270 2024-09-19T06:45:20.122 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:19 smithi103 ceph-mon[31158]: Detected new or changed devices on smithi174 2024-09-19T06:45:20.122 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:19 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-19T06:45:20.122 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:19 smithi103 ceph-mon[31158]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T06:45:20.122 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:19 smithi103 ceph-mon[31158]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T06:45:20.122 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:19 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:20.122 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:19 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:20.122 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:19 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:45:20.122 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:19 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:45:20.122 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:19 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:19 smithi134 ceph-mon[35356]: Detected new or changed devices on smithi174 2024-09-19T06:45:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:19 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-19T06:45:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:19 smithi134 ceph-mon[35356]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T06:45:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:19 smithi134 ceph-mon[35356]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T06:45:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:19 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:19 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:19 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:45:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:19 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:45:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:19 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:19 smithi174 ceph-mon[35637]: Detected new or changed devices on smithi174 2024-09-19T06:45:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:19 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-19T06:45:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:19 smithi174 ceph-mon[35637]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T06:45:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:19 smithi174 ceph-mon[35637]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T06:45:20.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:19 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:20.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:19 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:20.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:19 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:45:20.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:19 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:45:20.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:19 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:20 smithi134 ceph-mon[35356]: purged_snaps scrub starts 2024-09-19T06:45:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:20 smithi134 ceph-mon[35356]: purged_snaps scrub ok 2024-09-19T06:45:21.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:20 smithi134 ceph-mon[35356]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:45:21.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:20 smithi134 ceph-mon[35356]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:21.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:20 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:21.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:20 smithi134 ceph-mon[35356]: osd.8 [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465] boot 2024-09-19T06:45:21.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:20 smithi134 ceph-mon[35356]: osdmap e54: 9 total, 9 up, 9 in 2024-09-19T06:45:21.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:20 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:21.208 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:20 smithi103 ceph-mon[31158]: purged_snaps scrub starts 2024-09-19T06:45:21.208 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:20 smithi103 ceph-mon[31158]: purged_snaps scrub ok 2024-09-19T06:45:21.208 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:20 smithi103 ceph-mon[31158]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:45:21.208 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:20 smithi103 ceph-mon[31158]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:21.208 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:20 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:21.208 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:20 smithi103 ceph-mon[31158]: osd.8 [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465] boot 2024-09-19T06:45:21.209 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:20 smithi103 ceph-mon[31158]: osdmap e54: 9 total, 9 up, 9 in 2024-09-19T06:45:21.209 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:20 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:20 smithi174 ceph-mon[35637]: purged_snaps scrub starts 2024-09-19T06:45:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:20 smithi174 ceph-mon[35637]: purged_snaps scrub ok 2024-09-19T06:45:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:20 smithi174 ceph-mon[35637]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:45:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:20 smithi174 ceph-mon[35637]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:45:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:20 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:20 smithi174 ceph-mon[35637]: osd.8 [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465] boot 2024-09-19T06:45:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:20 smithi174 ceph-mon[35637]: osdmap e54: 9 total, 9 up, 9 in 2024-09-19T06:45:21.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:20 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:45:22.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:21 smithi103 ceph-mon[31158]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T06:45:22.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:21 smithi103 ceph-mon[31158]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:22.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:21 smithi103 ceph-mon[31158]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T06:45:22.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:21 smithi134 ceph-mon[35356]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T06:45:22.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:21 smithi134 ceph-mon[35356]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:22.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:21 smithi134 ceph-mon[35356]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T06:45:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:21 smithi174 ceph-mon[35637]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T06:45:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:21 smithi174 ceph-mon[35637]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:21 smithi174 ceph-mon[35637]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T06:45:23.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:22 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3488968699' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb3662e0-2d8e-45c4-9119-6d08b35443da"}]: dispatch 2024-09-19T06:45:23.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:22 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3488968699' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb3662e0-2d8e-45c4-9119-6d08b35443da"}]': finished 2024-09-19T06:45:23.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:22 smithi103 ceph-mon[31158]: osdmap e57: 10 total, 9 up, 10 in 2024-09-19T06:45:23.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:22 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:23.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:22 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/254965104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:45:23.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:22 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3488968699' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb3662e0-2d8e-45c4-9119-6d08b35443da"}]: dispatch 2024-09-19T06:45:23.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:22 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3488968699' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb3662e0-2d8e-45c4-9119-6d08b35443da"}]': finished 2024-09-19T06:45:23.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:22 smithi134 ceph-mon[35356]: osdmap e57: 10 total, 9 up, 10 in 2024-09-19T06:45:23.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:22 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:23.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:22 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/254965104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:45:23.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:22 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3488968699' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb3662e0-2d8e-45c4-9119-6d08b35443da"}]: dispatch 2024-09-19T06:45:23.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:22 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3488968699' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb3662e0-2d8e-45c4-9119-6d08b35443da"}]': finished 2024-09-19T06:45:23.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:22 smithi174 ceph-mon[35637]: osdmap e57: 10 total, 9 up, 10 in 2024-09-19T06:45:23.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:22 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:23.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:22 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/254965104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:45:24.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:24 smithi174 ceph-mon[35637]: osdmap e58: 10 total, 9 up, 10 in 2024-09-19T06:45:24.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:24 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:24.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:24 smithi174 ceph-mon[35637]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:24.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:24 smithi103 ceph-mon[31158]: osdmap e58: 10 total, 9 up, 10 in 2024-09-19T06:45:24.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:24 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:24.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:24 smithi103 ceph-mon[31158]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:24.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:24 smithi134 ceph-mon[35356]: osdmap e58: 10 total, 9 up, 10 in 2024-09-19T06:45:24.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:24 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:24.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:24 smithi134 ceph-mon[35356]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:25.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:25 smithi174 ceph-mon[35637]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T06:45:25.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:25 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:25.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:25 smithi103 ceph-mon[31158]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T06:45:25.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:25 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:25.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:25 smithi134 ceph-mon[35356]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T06:45:25.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:25 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:26.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:26 smithi174 ceph-mon[35637]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:26.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:26 smithi103 ceph-mon[31158]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:26.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:26 smithi134 ceph-mon[35356]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:29.115 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:28 smithi103 ceph-mon[31158]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-19T06:45:29.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:28 smithi134 ceph-mon[35356]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-19T06:45:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:28 smithi174 ceph-mon[35637]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-19T06:45:31.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:30 smithi103 ceph-mon[31158]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-19T06:45:31.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:30 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T06:45:31.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:30 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:31.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:30 smithi134 ceph-mon[35356]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-19T06:45:31.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:30 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T06:45:31.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:30 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:31.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:30 smithi174 ceph-mon[35637]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-19T06:45:31.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:30 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T06:45:31.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:30 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:32.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:31 smithi103 ceph-mon[31158]: Deploying daemon osd.9 on smithi103 2024-09-19T06:45:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:31 smithi134 ceph-mon[35356]: Deploying daemon osd.9 on smithi103 2024-09-19T06:45:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:31 smithi174 ceph-mon[35637]: Deploying daemon osd.9 on smithi103 2024-09-19T06:45:32.959 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:32 smithi103 ceph-mon[31158]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-19T06:45:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:32 smithi134 ceph-mon[35356]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-19T06:45:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:32 smithi174 ceph-mon[35637]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-19T06:45:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:34 smithi103 ceph-mon[31158]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T06:45:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:34 smithi134 ceph-mon[35356]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T06:45:35.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:34 smithi174 ceph-mon[35637]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T06:45:37.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:36 smithi103 ceph-mon[31158]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T06:45:37.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:36 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:45:37.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:36 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:37.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:36 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:36 smithi134 ceph-mon[35356]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T06:45:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:36 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:45:37.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:36 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:37.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:36 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:37.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:36 smithi174 ceph-mon[35637]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T06:45:37.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:36 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:45:37.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:36 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:37.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:36 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:39.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:38 smithi103 ceph-mon[31158]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:45:39.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:38 smithi134 ceph-mon[35356]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:45:39.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:38 smithi174 ceph-mon[35637]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:45:39.931 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 9 on host 'smithi103' 2024-09-19T06:45:39.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:39.930+0000 7f28a0903640 1 -- 172.21.15.103:0/1908213505 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f289c062630 con 0x7f28840a3270 2024-09-19T06:45:39.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:39.932+0000 7f28a1905640 1 -- 172.21.15.103:0/1908213505 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f28840a3270 msgr2=0x7f28840a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:39.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:39.932+0000 7f28a1905640 1 --2- 172.21.15.103:0/1908213505 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f28840a3270 0x7f28840a5730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f288c0058c0 tx=0x7f288c005f90 comp rx=0 tx=0).stop 2024-09-19T06:45:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:39.932+0000 7f28a1905640 1 -- 172.21.15.103:0/1908213505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f289c102e00 msgr2=0x7f289c1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:39.932+0000 7f28a1905640 1 --2- 172.21.15.103:0/1908213505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f289c102e00 0x7f289c1983c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f289000c970 tx=0x7f289000ce30 comp rx=0 tx=0).stop 2024-09-19T06:45:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:39.932+0000 7f28a1905640 1 -- 172.21.15.103:0/1908213505 shutdown_connections 2024-09-19T06:45:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:39.932+0000 7f28a1905640 1 --2- 172.21.15.103:0/1908213505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c1037c0 0x7f289c19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:39.932+0000 7f28a1905640 1 --2- 172.21.15.103:0/1908213505 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f28840a3270 0x7f28840a5730 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:39.932+0000 7f28a1905640 1 --2- 172.21.15.103:0/1908213505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f289c102e00 0x7f289c1983c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:39.932+0000 7f28a1905640 1 --2- 172.21.15.103:0/1908213505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f289c101c00 0x7f289c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:39.932+0000 7f28a1905640 1 -- 172.21.15.103:0/1908213505 >> 172.21.15.103:0/1908213505 conn(0x7f289c0fd390 msgr2=0x7f289c1061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:39.933+0000 7f28a1905640 1 -- 172.21.15.103:0/1908213505 shutdown_connections 2024-09-19T06:45:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:45:39.933+0000 7f28a1905640 1 -- 172.21.15.103:0/1908213505 wait complete. 2024-09-19T06:45:40.574 DEBUG:teuthology.orchestra.run.smithi103:osd.9> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T06:45:40.576 INFO:tasks.cephadm:Deploying osd.10 on smithi134 with /dev/vg_nvme/lv_1... 2024-09-19T06:45:40.576 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T06:45:40.837 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:45:41.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:40 smithi103 ceph-mon[31158]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:41.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:41.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:41.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:41.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:41.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:40 smithi134 ceph-mon[35356]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:41.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:41.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:41.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:41.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:40 smithi174 ceph-mon[35637]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:42.990 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:42 smithi103 ceph-mon[31158]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:42.990 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:42 smithi103 ceph-mon[31158]: from='osd.9 [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T06:45:42.990 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 06:45:42 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T06:45:42.742+0000 7f9e440a5640 -1 osd.9 0 log_to_monitors true 2024-09-19T06:45:43.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:42 smithi134 ceph-mon[35356]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:43.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:42 smithi134 ceph-mon[35356]: from='osd.9 [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T06:45:43.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:42 smithi174 ceph-mon[35637]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:43.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:42 smithi174 ceph-mon[35637]: from='osd.9 [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T06:45:43.387 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T06:45:43.413 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_1 2024-09-19T06:45:43.669 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:45:43.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:43 smithi134 ceph-mon[35356]: from='osd.9 [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T06:45:43.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:43 smithi134 ceph-mon[35356]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T06:45:43.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:43 smithi134 ceph-mon[35356]: from='osd.9 [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T06:45:43.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:43 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:43.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:43 smithi134 ceph-mon[35356]: from='osd.9 [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T06:45:44.045 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 06:45:43 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T06:45:43.880+0000 7f9e3f805640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T06:45:44.045 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 06:45:43 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T06:45:43.894+0000 7f9e3be3f640 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:45:44.046 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:43 smithi103 ceph-mon[31158]: from='osd.9 [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T06:45:44.046 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:43 smithi103 ceph-mon[31158]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T06:45:44.046 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:43 smithi103 ceph-mon[31158]: from='osd.9 [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T06:45:44.046 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:43 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:44.046 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:43 smithi103 ceph-mon[31158]: from='osd.9 [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T06:45:44.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.186+0000 7fbdee02b640 1 -- 172.21.15.134:0/3439359320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbde8107960 msgr2=0x7fbde8109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:44.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.186+0000 7fbdee02b640 1 --2- 172.21.15.134:0/3439359320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbde8107960 0x7fbde8109d50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009900 tx=0x7fbde002a410 comp rx=0 tx=0).stop 2024-09-19T06:45:44.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.186+0000 7fbdee02b640 1 -- 172.21.15.134:0/3439359320 shutdown_connections 2024-09-19T06:45:44.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.186+0000 7fbdee02b640 1 --2- 172.21.15.134:0/3439359320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbde8107960 0x7fbde8109d50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.186+0000 7fbdee02b640 1 --2- 172.21.15.134:0/3439359320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbde8105030 0x7fbde8107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.186+0000 7fbdee02b640 1 --2- 172.21.15.134:0/3439359320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbde8102700 0x7fbde8104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.186+0000 7fbdee02b640 1 -- 172.21.15.134:0/3439359320 >> 172.21.15.134:0/3439359320 conn(0x7fbde80fc280 msgr2=0x7fbde80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:44.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.187+0000 7fbdee02b640 1 -- 172.21.15.134:0/3439359320 shutdown_connections 2024-09-19T06:45:44.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.187+0000 7fbdee02b640 1 -- 172.21.15.134:0/3439359320 wait complete. 2024-09-19T06:45:44.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.187+0000 7fbdee02b640 1 Processor -- start 2024-09-19T06:45:44.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.187+0000 7fbdee02b640 1 -- start start 2024-09-19T06:45:44.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.187+0000 7fbdee02b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbde8102700 0x7fbde8197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.187+0000 7fbdee02b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbde8105030 0x7fbde8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.187+0000 7fbdee02b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbde8107960 0x7fbde819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.187+0000 7fbdee02b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbde810d880 con 0x7fbde8107960 2024-09-19T06:45:44.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.187+0000 7fbdee02b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbde810d700 con 0x7fbde8102700 2024-09-19T06:45:44.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.187+0000 7fbdee02b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbde810da00 con 0x7fbde8105030 2024-09-19T06:45:44.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbde77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbde8102700 0x7fbde8197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbde77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbde8102700 0x7fbde8197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:47854/0 (socket says 172.21.15.134:47854) 2024-09-19T06:45:44.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbde77fe640 1 -- 172.21.15.134:0/655192146 learned_addr learned my addr 172.21.15.134:0/655192146 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-19T06:45:44.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbde7fff640 1 --2- 172.21.15.134:0/655192146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbde8107960 0x7fbde819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbde6ffd640 1 --2- 172.21.15.134:0/655192146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbde8105030 0x7fbde8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbde77fe640 1 -- 172.21.15.134:0/655192146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbde8105030 msgr2=0x7fbde8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:44.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbde77fe640 1 --2- 172.21.15.134:0/655192146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbde8105030 0x7fbde8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbde77fe640 1 -- 172.21.15.134:0/655192146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbde8107960 msgr2=0x7fbde819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:44.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbde77fe640 1 --2- 172.21.15.134:0/655192146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbde8107960 0x7fbde819f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbde77fe640 1 -- 172.21.15.134:0/655192146 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde819fb30 con 0x7fbde8102700 2024-09-19T06:45:44.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbde6ffd640 1 --2- 172.21.15.134:0/655192146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbde8105030 0x7fbde8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:45:44.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbde77fe640 1 --2- 172.21.15.134:0/655192146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbde8102700 0x7fbde8197f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800cce0 tx=0x7fbdd800a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:44.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbde4ff9640 1 -- 172.21.15.134:0/655192146 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8004b20 con 0x7fbde8102700 2024-09-19T06:45:44.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbdee02b640 1 -- 172.21.15.134:0/655192146 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde819fe20 con 0x7fbde8102700 2024-09-19T06:45:44.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbde4ff9640 1 -- 172.21.15.134:0/655192146 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdd8004cc0 con 0x7fbde8102700 2024-09-19T06:45:44.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbde4ff9640 1 -- 172.21.15.134:0/655192146 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd800f8c0 con 0x7fbde8102700 2024-09-19T06:45:44.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.188+0000 7fbdee02b640 1 -- 172.21.15.134:0/655192146 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde81a0270 con 0x7fbde8102700 2024-09-19T06:45:44.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.189+0000 7fbdee02b640 1 -- 172.21.15.134:0/655192146 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde80fdd70 con 0x7fbde8102700 2024-09-19T06:45:44.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.190+0000 7fbde4ff9640 1 -- 172.21.15.134:0/655192146 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fbdd800fa60 con 0x7fbde8102700 2024-09-19T06:45:44.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.191+0000 7fbde4ff9640 1 --2- 172.21.15.134:0/655192146 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbdd40a32f0 0x7fbdd40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.191+0000 7fbde4ff9640 1 -- 172.21.15.134:0/655192146 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5981+0+0 (secure 0 0 0) 0x7fbdd80ceff0 con 0x7fbde8102700 2024-09-19T06:45:44.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.191+0000 7fbde6ffd640 1 --2- 172.21.15.134:0/655192146 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbdd40a32f0 0x7fbdd40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.192+0000 7fbde6ffd640 1 --2- 172.21.15.134:0/655192146 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbdd40a32f0 0x7fbdd40a57b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbde8199420 tx=0x7fbdd0005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:44.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.195+0000 7fbde4ff9640 1 -- 172.21.15.134:0/655192146 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbdd8014030 con 0x7fbde8102700 2024-09-19T06:45:44.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:43 smithi174 ceph-mon[35637]: from='osd.9 [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T06:45:44.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:43 smithi174 ceph-mon[35637]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T06:45:44.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:43 smithi174 ceph-mon[35637]: from='osd.9 [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T06:45:44.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:43 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:44.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:43 smithi174 ceph-mon[35637]: from='osd.9 [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T06:45:44.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:45:44.376+0000 7fbdee02b640 1 -- 172.21.15.134:0/655192146 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbde8062630 con 0x7fbdd40a32f0 2024-09-19T06:45:44.936 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:44.936 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T06:45:44.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:44.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:44.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:45:44.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:45:44.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:44.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:44.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:44.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:44.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:44.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:44.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:44.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:44.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:44.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:45:44.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:44.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:45:44.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:44.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:45:44.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:44.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:45.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:45.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T06:45:45.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:45.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:45.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:45:45.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:45:45.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:45.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:45.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:45.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:45.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:45.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:45.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:45.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:45.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:45.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:45:45.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:45.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:45:45.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:45.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:45:45.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:45.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:45.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:45.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T06:45:45.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:45.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:45.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:45:45.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:45:45.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:45.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:45.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:45.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:45.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:45.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:45.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:45:45.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:45.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:45.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:45:45.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:45.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:45:45.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:45.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:45:45.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:45:45.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:46.070 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:45 smithi134 ceph-mon[35356]: purged_snaps scrub starts 2024-09-19T06:45:46.070 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:45 smithi134 ceph-mon[35356]: purged_snaps scrub ok 2024-09-19T06:45:46.070 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:45 smithi134 ceph-mon[35356]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:45:46.070 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:45 smithi134 ceph-mon[35356]: Detected new or changed devices on smithi103 2024-09-19T06:45:46.070 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:45 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-19T06:45:46.071 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:45 smithi134 ceph-mon[35356]: osd.9 [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413] boot 2024-09-19T06:45:46.071 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:45 smithi134 ceph-mon[35356]: osdmap e62: 10 total, 10 up, 10 in 2024-09-19T06:45:46.071 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:45 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:45 smithi174 ceph-mon[35637]: purged_snaps scrub starts 2024-09-19T06:45:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:45 smithi174 ceph-mon[35637]: purged_snaps scrub ok 2024-09-19T06:45:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:45 smithi174 ceph-mon[35637]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:45:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:45 smithi174 ceph-mon[35637]: Detected new or changed devices on smithi103 2024-09-19T06:45:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:45 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-19T06:45:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:45 smithi174 ceph-mon[35637]: osd.9 [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413] boot 2024-09-19T06:45:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:45 smithi174 ceph-mon[35637]: osdmap e62: 10 total, 10 up, 10 in 2024-09-19T06:45:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:45 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:46.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:45 smithi103 ceph-mon[31158]: purged_snaps scrub starts 2024-09-19T06:45:46.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:45 smithi103 ceph-mon[31158]: purged_snaps scrub ok 2024-09-19T06:45:46.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:45 smithi103 ceph-mon[31158]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:45:46.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:45 smithi103 ceph-mon[31158]: Detected new or changed devices on smithi103 2024-09-19T06:45:46.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:45 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-19T06:45:46.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:45 smithi103 ceph-mon[31158]: osd.9 [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413] boot 2024-09-19T06:45:46.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:45 smithi103 ceph-mon[31158]: osdmap e62: 10 total, 10 up, 10 in 2024-09-19T06:45:46.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:45 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:45:47.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:46 smithi134 ceph-mon[35356]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:47.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:46 smithi134 ceph-mon[35356]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T06:45:47.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:46 smithi134 ceph-mon[35356]: from='client.? 172.21.15.134:0/4115300726' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e52c3cbe-a6af-441e-b423-b94f3a2da544"}]: dispatch 2024-09-19T06:45:47.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:46 smithi134 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e52c3cbe-a6af-441e-b423-b94f3a2da544"}]: dispatch 2024-09-19T06:45:47.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:46 smithi134 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e52c3cbe-a6af-441e-b423-b94f3a2da544"}]': finished 2024-09-19T06:45:47.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:46 smithi134 ceph-mon[35356]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T06:45:47.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:46 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:45:47.234 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:46 smithi174 ceph-mon[35637]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:47.235 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:46 smithi174 ceph-mon[35637]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T06:45:47.235 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:46 smithi174 ceph-mon[35637]: from='client.? 172.21.15.134:0/4115300726' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e52c3cbe-a6af-441e-b423-b94f3a2da544"}]: dispatch 2024-09-19T06:45:47.235 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:46 smithi174 ceph-mon[35637]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e52c3cbe-a6af-441e-b423-b94f3a2da544"}]: dispatch 2024-09-19T06:45:47.235 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:46 smithi174 ceph-mon[35637]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e52c3cbe-a6af-441e-b423-b94f3a2da544"}]': finished 2024-09-19T06:45:47.235 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:46 smithi174 ceph-mon[35637]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T06:45:47.236 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:46 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:45:47.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:46 smithi103 ceph-mon[31158]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:45:47.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:46 smithi103 ceph-mon[31158]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T06:45:47.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:46 smithi103 ceph-mon[31158]: from='client.? 172.21.15.134:0/4115300726' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e52c3cbe-a6af-441e-b423-b94f3a2da544"}]: dispatch 2024-09-19T06:45:47.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:46 smithi103 ceph-mon[31158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e52c3cbe-a6af-441e-b423-b94f3a2da544"}]: dispatch 2024-09-19T06:45:47.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:46 smithi103 ceph-mon[31158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e52c3cbe-a6af-441e-b423-b94f3a2da544"}]': finished 2024-09-19T06:45:47.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:46 smithi103 ceph-mon[31158]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T06:45:47.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:46 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:45:48.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:47 smithi174 ceph-mon[35637]: from='client.? 172.21.15.134:0/2279454002' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:45:48.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:47 smithi103 ceph-mon[31158]: from='client.? 172.21.15.134:0/2279454002' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:45:48.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:47 smithi134 ceph-mon[35356]: from='client.? 172.21.15.134:0/2279454002' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:45:49.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:48 smithi174 ceph-mon[35637]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:49.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:48 smithi103 ceph-mon[31158]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:49.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:48 smithi134 ceph-mon[35356]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:50.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:49 smithi174 ceph-mon[35637]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:50.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:49 smithi103 ceph-mon[31158]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:50.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:49 smithi134 ceph-mon[35356]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:53.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:52 smithi103 ceph-mon[31158]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:53.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:52 smithi134 ceph-mon[35356]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:53.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:52 smithi174 ceph-mon[35637]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:54.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:54 smithi134 ceph-mon[35356]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:55.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:54 smithi103 ceph-mon[31158]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:55.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:54 smithi174 ceph-mon[35637]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:56.025 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T06:45:56.025 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:55 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T06:45:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:55 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T06:45:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:55 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:45:57.071 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:56 smithi134 ceph-mon[35356]: Deploying daemon osd.10 on smithi134 2024-09-19T06:45:57.071 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:56 smithi134 ceph-mon[35356]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:57.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:56 smithi103 ceph-mon[31158]: Deploying daemon osd.10 on smithi134 2024-09-19T06:45:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:56 smithi103 ceph-mon[31158]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:57.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:56 smithi174 ceph-mon[35637]: Deploying daemon osd.10 on smithi134 2024-09-19T06:45:57.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:56 smithi174 ceph-mon[35637]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:59.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:45:58 smithi103 ceph-mon[31158]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:59.173 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:45:58 smithi134 ceph-mon[35356]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:45:59.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:45:58 smithi174 ceph-mon[35637]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:01.059 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:00 smithi134 ceph-mon[35356]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:01.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:00 smithi103 ceph-mon[31158]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:01.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:00 smithi174 ceph-mon[35637]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:02.076 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:01 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:02.076 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:01 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:02.076 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:01 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:02.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:01 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:02.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:01 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:02.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:01 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:01 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:01 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:01 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:03.066 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:02 smithi134 ceph-mon[35356]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:03.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:02 smithi103 ceph-mon[31158]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:02 smithi174 ceph-mon[35637]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:04.837 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 10 on host 'smithi134' 2024-09-19T06:46:04.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:46:04.836+0000 7fbde4ff9640 1 -- 172.21.15.134:0/655192146 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbde8062630 con 0x7fbdd40a32f0 2024-09-19T06:46:04.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:46:04.838+0000 7fbdee02b640 1 -- 172.21.15.134:0/655192146 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbdd40a32f0 msgr2=0x7fbdd40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:04.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:46:04.838+0000 7fbdee02b640 1 --2- 172.21.15.134:0/655192146 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbdd40a32f0 0x7fbdd40a57b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbde8199420 tx=0x7fbdd0005e50 comp rx=0 tx=0).stop 2024-09-19T06:46:04.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:46:04.838+0000 7fbdee02b640 1 -- 172.21.15.134:0/655192146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbde8102700 msgr2=0x7fbde8197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:04.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:46:04.838+0000 7fbdee02b640 1 --2- 172.21.15.134:0/655192146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbde8102700 0x7fbde8197f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800cce0 tx=0x7fbdd800a620 comp rx=0 tx=0).stop 2024-09-19T06:46:04.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:46:04.838+0000 7fbdee02b640 1 -- 172.21.15.134:0/655192146 shutdown_connections 2024-09-19T06:46:04.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:46:04.838+0000 7fbdee02b640 1 --2- 172.21.15.134:0/655192146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbde8107960 0x7fbde819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:04.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:46:04.838+0000 7fbdee02b640 1 --2- 172.21.15.134:0/655192146 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fbdd40a32f0 0x7fbdd40a57b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:04.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:46:04.838+0000 7fbdee02b640 1 --2- 172.21.15.134:0/655192146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbde8105030 0x7fbde8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:04.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:46:04.839+0000 7fbdee02b640 1 --2- 172.21.15.134:0/655192146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbde8102700 0x7fbde8197f00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:04.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:46:04.839+0000 7fbdee02b640 1 -- 172.21.15.134:0/655192146 >> 172.21.15.134:0/655192146 conn(0x7fbde80fc280 msgr2=0x7fbde8103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:04.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:46:04.839+0000 7fbdee02b640 1 -- 172.21.15.134:0/655192146 shutdown_connections 2024-09-19T06:46:04.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T06:46:04.839+0000 7fbdee02b640 1 -- 172.21.15.134:0/655192146 wait complete. 2024-09-19T06:46:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:04 smithi103 ceph-mon[31158]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:05.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:04 smithi134 ceph-mon[35356]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:05.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:05.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:05.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:05.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:05.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:05.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:05.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:05.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:05.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:04 smithi174 ceph-mon[35637]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:05.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:05.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:05.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:05.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:05.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:05.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:05.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:05.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:05.471 DEBUG:teuthology.orchestra.run.smithi134:osd.10> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T06:46:05.474 INFO:tasks.cephadm:Deploying osd.11 on smithi174 with /dev/vg_nvme/lv_1... 2024-09-19T06:46:05.474 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T06:46:05.728 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:46:06.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:06 smithi134 ceph-mon[35356]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:07.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:06 smithi103 ceph-mon[31158]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:07.182 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:06 smithi174 ceph-mon[35637]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:07.672 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 06:46:07 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T06:46:07.366+0000 7fef472c3640 -1 osd.10 0 log_to_monitors true 2024-09-19T06:46:07.880 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:07 smithi103 ceph-mon[31158]: from='osd.10 [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:46:07.880 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:07 smithi103 ceph-mon[31158]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:46:07.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:07 smithi134 ceph-mon[35356]: from='osd.10 [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:46:07.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:07 smithi134 ceph-mon[35356]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:46:08.302 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:07 smithi174 ceph-mon[35637]: from='osd.10 [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:46:08.302 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:07 smithi174 ceph-mon[35637]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:46:08.305 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T06:46:08.332 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_1 2024-09-19T06:46:08.589 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:46:09.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.031+0000 7f6f20d1b640 1 -- 172.21.15.174:0/3609438078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f1c100ad0 msgr2=0x7f6f1c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:09.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.031+0000 7f6f20d1b640 1 --2- 172.21.15.174:0/3609438078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f1c100ad0 0x7f6f1c100ed0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009900 tx=0x7f6f1002a410 comp rx=0 tx=0).stop 2024-09-19T06:46:09.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.031+0000 7f6f20d1b640 1 -- 172.21.15.174:0/3609438078 shutdown_connections 2024-09-19T06:46:09.040 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.031+0000 7f6f20d1b640 1 --2- 172.21.15.174:0/3609438078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f1c102690 0x7f6f1c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.040 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.031+0000 7f6f20d1b640 1 --2- 172.21.15.174:0/3609438078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1c101cd0 0x7f6f1c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.040 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.031+0000 7f6f20d1b640 1 --2- 172.21.15.174:0/3609438078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f1c100ad0 0x7f6f1c100ed0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.040 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.031+0000 7f6f20d1b640 1 -- 172.21.15.174:0/3609438078 >> 172.21.15.174:0/3609438078 conn(0x7f6f1c0fc280 msgr2=0x7f6f1c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:09.040 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.031+0000 7f6f20d1b640 1 -- 172.21.15.174:0/3609438078 shutdown_connections 2024-09-19T06:46:09.040 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.031+0000 7f6f20d1b640 1 -- 172.21.15.174:0/3609438078 wait complete. 2024-09-19T06:46:09.040 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f20d1b640 1 Processor -- start 2024-09-19T06:46:09.040 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f20d1b640 1 -- start start 2024-09-19T06:46:09.040 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f20d1b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f1c100ad0 0x7f6f1c104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.040 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f20d1b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f1c101cd0 0x7f6f1c1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.040 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f20d1b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1c102690 0x7f6f1c1035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.040 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f20d1b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c10c7b0 con 0x7f6f1c102690 2024-09-19T06:46:09.041 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f20d1b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c10c630 con 0x7f6f1c101cd0 2024-09-19T06:46:09.041 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f20d1b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c10c930 con 0x7f6f1c100ad0 2024-09-19T06:46:09.041 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f1a59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f1c100ad0 0x7f6f1c104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.041 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f1a59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f1c100ad0 0x7f6f1c104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38864/0 (socket says 172.21.15.174:38864) 2024-09-19T06:46:09.041 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f1a59c640 1 -- 172.21.15.174:0/3298817584 learned_addr learned my addr 172.21.15.174:0/3298817584 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T06:46:09.041 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f1a59c640 1 -- 172.21.15.174:0/3298817584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f1c101cd0 msgr2=0x7f6f1c1030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:09.041 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f19d9b640 1 --2- 172.21.15.174:0/3298817584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f1c101cd0 0x7f6f1c1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.041 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f1ad9d640 1 --2- 172.21.15.174:0/3298817584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1c102690 0x7f6f1c1035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.041 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f1a59c640 1 --2- 172.21.15.174:0/3298817584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f1c101cd0 0x7f6f1c1030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.041 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f1a59c640 1 -- 172.21.15.174:0/3298817584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1c102690 msgr2=0x7f6f1c1035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:09.041 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f1a59c640 1 --2- 172.21.15.174:0/3298817584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1c102690 0x7f6f1c1035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.041 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f1a59c640 1 -- 172.21.15.174:0/3298817584 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c103eb0 con 0x7f6f1c100ad0 2024-09-19T06:46:09.041 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.032+0000 7f6f1a59c640 1 --2- 172.21.15.174:0/3298817584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f1c100ad0 0x7f6f1c104a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6f10031610 tx=0x7f6f100043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:09.041 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.033+0000 7f6f0b7fe640 1 -- 172.21.15.174:0/3298817584 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1003f030 con 0x7f6f1c100ad0 2024-09-19T06:46:09.042 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.033+0000 7f6f0b7fe640 1 -- 172.21.15.174:0/3298817584 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f10003ce0 con 0x7f6f1c100ad0 2024-09-19T06:46:09.042 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.033+0000 7f6f20d1b640 1 -- 172.21.15.174:0/3298817584 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1c1a2030 con 0x7f6f1c100ad0 2024-09-19T06:46:09.042 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.033+0000 7f6f0b7fe640 1 -- 172.21.15.174:0/3298817584 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10035070 con 0x7f6f1c100ad0 2024-09-19T06:46:09.042 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.033+0000 7f6f20d1b640 1 -- 172.21.15.174:0/3298817584 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1c1a25b0 con 0x7f6f1c100ad0 2024-09-19T06:46:09.042 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.034+0000 7f6f20d1b640 1 -- 172.21.15.174:0/3298817584 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f1c1a28a0 con 0x7f6f1c100ad0 2024-09-19T06:46:09.042 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.035+0000 7f6f0b7fe640 1 -- 172.21.15.174:0/3298817584 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f6f10042030 con 0x7f6f1c100ad0 2024-09-19T06:46:09.042 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.036+0000 7f6f0b7fe640 1 --2- 172.21.15.174:0/3298817584 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6f040a32c0 0x7f6f040a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.042 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.036+0000 7f6f0b7fe640 1 -- 172.21.15.174:0/3298817584 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6414+0+0 (secure 0 0 0) 0x7f6f100ef880 con 0x7f6f1c100ad0 2024-09-19T06:46:09.042 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.036+0000 7f6f19d9b640 1 --2- 172.21.15.174:0/3298817584 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6f040a32c0 0x7f6f040a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.042 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.036+0000 7f6f19d9b640 1 --2- 172.21.15.174:0/3298817584 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6f040a32c0 0x7f6f040a5780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c1047e0 tx=0x7f6f0c005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:09.042 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.040+0000 7f6f0b7fe640 1 -- 172.21.15.174:0/3298817584 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6f100bcd10 con 0x7f6f1c100ad0 2024-09-19T06:46:09.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T06:46:08.890+0000 7fef42a23640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T06:46:09.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T06:46:08.903+0000 7fef3f05d640 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:46:09.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-mon[35356]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:09.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-mon[35356]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T06:46:09.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-mon[35356]: from='osd.10 [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:46:09.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-mon[35356]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T06:46:09.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:46:09.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-mon[35356]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:46:09.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:09.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:09.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:09.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:09.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:09.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:09.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:09.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:09.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:09.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:08 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:09.222 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:09.221+0000 7f6f20d1b640 1 -- 172.21.15.174:0/3298817584 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6f1c06a770 con 0x7f6f040a32c0 2024-09-19T06:46:09.223 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:08 smithi174 ceph-mon[35637]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:09.223 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:08 smithi174 ceph-mon[35637]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T06:46:09.223 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:08 smithi174 ceph-mon[35637]: from='osd.10 [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:46:09.223 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:08 smithi174 ceph-mon[35637]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T06:46:09.223 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:08 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:46:09.223 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:08 smithi174 ceph-mon[35637]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:46:09.223 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:08 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:09.223 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:08 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:09.223 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:08 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:09.223 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:08 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:09.223 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:08 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:09.223 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:08 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:09.224 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:08 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:09.224 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:08 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:09.224 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:08 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:09.224 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:08 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:09.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:08 smithi103 ceph-mon[31158]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:09.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:08 smithi103 ceph-mon[31158]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T06:46:09.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:08 smithi103 ceph-mon[31158]: from='osd.10 [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:46:09.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:08 smithi103 ceph-mon[31158]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T06:46:09.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:08 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:46:09.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:08 smithi103 ceph-mon[31158]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T06:46:09.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:08 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:09.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:08 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:09.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:08 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:09.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:08 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:09.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:08 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:09.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:08 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:09.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:08 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:09.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:08 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:09.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:08 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:09.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:08 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:10.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:09 smithi134 ceph-mon[35356]: Detected new or changed devices on smithi134 2024-09-19T06:46:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:09 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi134 to 4272M 2024-09-19T06:46:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:09 smithi134 ceph-mon[35356]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T06:46:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:09 smithi134 ceph-mon[35356]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T06:46:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:09 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:46:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:09 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:46:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:09 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:46:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:09 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:46:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:09 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:10.289 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:09 smithi103 ceph-mon[31158]: Detected new or changed devices on smithi134 2024-09-19T06:46:10.289 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:09 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi134 to 4272M 2024-09-19T06:46:10.289 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:09 smithi103 ceph-mon[31158]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T06:46:10.289 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:09 smithi103 ceph-mon[31158]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T06:46:10.289 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:09 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:46:10.289 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:09 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:46:10.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:09 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:46:10.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:09 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:46:10.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:09 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:10.296 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:09 smithi174 ceph-mon[35637]: Detected new or changed devices on smithi134 2024-09-19T06:46:10.296 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:09 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi134 to 4272M 2024-09-19T06:46:10.296 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:09 smithi174 ceph-mon[35637]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T06:46:10.296 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:09 smithi174 ceph-mon[35637]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T06:46:10.296 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:09 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:46:10.296 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:09 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:46:10.296 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:09 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:46:10.296 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:09 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:46:10.296 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:09 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:10.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:10 smithi134 ceph-mon[35356]: purged_snaps scrub starts 2024-09-19T06:46:10.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:10 smithi134 ceph-mon[35356]: purged_snaps scrub ok 2024-09-19T06:46:10.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:10 smithi134 ceph-mon[35356]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:46:10.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:10 smithi134 ceph-mon[35356]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:10.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:10 smithi134 ceph-mon[35356]: osd.10 [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632] boot 2024-09-19T06:46:10.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:10 smithi134 ceph-mon[35356]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T06:46:10.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:10 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:46:10.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:10 smithi134 ceph-mon[35356]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T06:46:10.949 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:10 smithi174 ceph-mon[35637]: purged_snaps scrub starts 2024-09-19T06:46:10.949 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:10 smithi174 ceph-mon[35637]: purged_snaps scrub ok 2024-09-19T06:46:10.949 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:10 smithi174 ceph-mon[35637]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:46:10.949 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:10 smithi174 ceph-mon[35637]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:10.949 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:10 smithi174 ceph-mon[35637]: osd.10 [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632] boot 2024-09-19T06:46:10.949 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:10 smithi174 ceph-mon[35637]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T06:46:10.949 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:10 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:46:10.950 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:10 smithi174 ceph-mon[35637]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T06:46:11.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:10 smithi103 ceph-mon[31158]: purged_snaps scrub starts 2024-09-19T06:46:11.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:10 smithi103 ceph-mon[31158]: purged_snaps scrub ok 2024-09-19T06:46:11.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:10 smithi103 ceph-mon[31158]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:46:11.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:10 smithi103 ceph-mon[31158]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:46:11.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:10 smithi103 ceph-mon[31158]: osd.10 [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632] boot 2024-09-19T06:46:11.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:10 smithi103 ceph-mon[31158]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T06:46:11.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:10 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:46:11.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:10 smithi103 ceph-mon[31158]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T06:46:12.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:11 smithi134 ceph-mon[35356]: from='client.? 172.21.15.174:0/1806180775' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9040f3f8-3873-4dea-8716-dcda5811c8d1"}]: dispatch 2024-09-19T06:46:12.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:11 smithi134 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9040f3f8-3873-4dea-8716-dcda5811c8d1"}]: dispatch 2024-09-19T06:46:12.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:11 smithi134 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9040f3f8-3873-4dea-8716-dcda5811c8d1"}]': finished 2024-09-19T06:46:12.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:11 smithi134 ceph-mon[35356]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T06:46:12.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:11 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:12.207 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:11 smithi174 ceph-mon[35637]: from='client.? 172.21.15.174:0/1806180775' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9040f3f8-3873-4dea-8716-dcda5811c8d1"}]: dispatch 2024-09-19T06:46:12.208 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:11 smithi174 ceph-mon[35637]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9040f3f8-3873-4dea-8716-dcda5811c8d1"}]: dispatch 2024-09-19T06:46:12.208 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:11 smithi174 ceph-mon[35637]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9040f3f8-3873-4dea-8716-dcda5811c8d1"}]': finished 2024-09-19T06:46:12.208 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:11 smithi174 ceph-mon[35637]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T06:46:12.208 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:11 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:12.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:11 smithi103 ceph-mon[31158]: from='client.? 172.21.15.174:0/1806180775' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9040f3f8-3873-4dea-8716-dcda5811c8d1"}]: dispatch 2024-09-19T06:46:12.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:11 smithi103 ceph-mon[31158]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9040f3f8-3873-4dea-8716-dcda5811c8d1"}]: dispatch 2024-09-19T06:46:12.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:11 smithi103 ceph-mon[31158]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9040f3f8-3873-4dea-8716-dcda5811c8d1"}]': finished 2024-09-19T06:46:12.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:11 smithi103 ceph-mon[31158]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T06:46:12.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:11 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:13.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:12 smithi134 ceph-mon[35356]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:13.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:12 smithi134 ceph-mon[35356]: from='client.? 172.21.15.174:0/2037128137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:46:13.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:12 smithi174 ceph-mon[35637]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:13.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:12 smithi174 ceph-mon[35637]: from='client.? 172.21.15.174:0/2037128137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:46:13.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:12 smithi103 ceph-mon[31158]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:13.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:12 smithi103 ceph-mon[31158]: from='client.? 172.21.15.174:0/2037128137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:46:14.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:13 smithi134 ceph-mon[35356]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T06:46:14.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:13 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:14.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:13 smithi174 ceph-mon[35637]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T06:46:14.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:13 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:14.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:13 smithi103 ceph-mon[31158]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T06:46:14.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:13 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:15.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:14 smithi134 ceph-mon[35356]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:15.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:14 smithi134 ceph-mon[35356]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T06:46:15.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:14 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:15.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:14 smithi174 ceph-mon[35637]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:15.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:14 smithi174 ceph-mon[35637]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T06:46:15.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:14 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:15.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:14 smithi103 ceph-mon[31158]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:15.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:14 smithi103 ceph-mon[31158]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T06:46:15.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:14 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:17.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:16 smithi174 ceph-mon[35637]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:17.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:16 smithi134 ceph-mon[35356]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:17.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:16 smithi103 ceph-mon[31158]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:19.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:18 smithi134 ceph-mon[35356]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-19T06:46:19.197 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:18 smithi174 ceph-mon[35637]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-19T06:46:19.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:18 smithi103 ceph-mon[31158]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-19T06:46:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:20 smithi134 ceph-mon[35356]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:46:21.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:20 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T06:46:21.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:20 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:21.213 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:20 smithi174 ceph-mon[35637]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:46:21.214 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:20 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T06:46:21.214 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:20 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:21.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:20 smithi103 ceph-mon[31158]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:46:21.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:20 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T06:46:21.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:20 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:22.182 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:21 smithi174 ceph-mon[35637]: Deploying daemon osd.11 on smithi174 2024-09-19T06:46:22.182 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:21 smithi174 ceph-mon[35637]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-19T06:46:22.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:21 smithi134 ceph-mon[35356]: Deploying daemon osd.11 on smithi174 2024-09-19T06:46:22.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:21 smithi134 ceph-mon[35356]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-19T06:46:22.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:21 smithi103 ceph-mon[31158]: Deploying daemon osd.11 on smithi174 2024-09-19T06:46:22.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:21 smithi103 ceph-mon[31158]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-19T06:46:25.079 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:24 smithi174 ceph-mon[35637]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T06:46:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:24 smithi103 ceph-mon[31158]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T06:46:25.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:24 smithi134 ceph-mon[35356]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T06:46:27.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:26 smithi174 ceph-mon[35637]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T06:46:27.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:26 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:27.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:26 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:27.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:26 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:27.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:26 smithi103 ceph-mon[31158]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T06:46:27.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:26 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:27.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:26 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:27.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:26 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:26 smithi134 ceph-mon[35356]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T06:46:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:26 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:26 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:26 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:28 smithi103 ceph-mon[31158]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:46:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:28 smithi134 ceph-mon[35356]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:46:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:29.316 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:28 smithi174 ceph-mon[35637]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:46:29.317 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:29.317 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:29.317 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:29.317 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:29.317 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:29.740 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:29.738+0000 7f6f0b7fe640 1 -- 172.21.15.174:0/3298817584 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6f1c06a770 con 0x7f6f040a32c0 2024-09-19T06:46:29.741 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 11 on host 'smithi174' 2024-09-19T06:46:29.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:29.740+0000 7f6f20d1b640 1 -- 172.21.15.174:0/3298817584 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6f040a32c0 msgr2=0x7f6f040a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:29.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:29.740+0000 7f6f20d1b640 1 --2- 172.21.15.174:0/3298817584 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6f040a32c0 0x7f6f040a5780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c1047e0 tx=0x7f6f0c005cc0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:29.740+0000 7f6f20d1b640 1 -- 172.21.15.174:0/3298817584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f1c100ad0 msgr2=0x7f6f1c104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:29.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:29.740+0000 7f6f20d1b640 1 --2- 172.21.15.174:0/3298817584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f1c100ad0 0x7f6f1c104a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6f10031610 tx=0x7f6f100043b0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:29.741+0000 7f6f20d1b640 1 -- 172.21.15.174:0/3298817584 shutdown_connections 2024-09-19T06:46:29.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:29.741+0000 7f6f20d1b640 1 --2- 172.21.15.174:0/3298817584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1c102690 0x7f6f1c1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:29.741+0000 7f6f20d1b640 1 --2- 172.21.15.174:0/3298817584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f1c101cd0 0x7f6f1c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:29.741+0000 7f6f20d1b640 1 --2- 172.21.15.174:0/3298817584 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6f040a32c0 0x7f6f040a5780 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:29.741+0000 7f6f20d1b640 1 --2- 172.21.15.174:0/3298817584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f1c100ad0 0x7f6f1c104a60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:29.741+0000 7f6f20d1b640 1 -- 172.21.15.174:0/3298817584 >> 172.21.15.174:0/3298817584 conn(0x7f6f1c0fc280 msgr2=0x7f6f1c06a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:29.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:29.741+0000 7f6f20d1b640 1 -- 172.21.15.174:0/3298817584 shutdown_connections 2024-09-19T06:46:29.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:29.741+0000 7f6f20d1b640 1 -- 172.21.15.174:0/3298817584 wait complete. 2024-09-19T06:46:30.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:29 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:30.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:29 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:30.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:29 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:30.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:29 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:30.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:29 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:30.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:29 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:30.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:29 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:30.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:29 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:30.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:29 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:30.408 DEBUG:teuthology.orchestra.run.smithi174:osd.11> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T06:46:30.410 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T06:46:30.410 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T06:46:30.671 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:31.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:30 smithi174 ceph-mon[35637]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:31.105 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:30 smithi103 ceph-mon[31158]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:31.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.104+0000 7f337da4e640 1 -- 172.21.15.103:0/2864712180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33781026d0 msgr2=0x7f3378108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:31.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.104+0000 7f337da4e640 1 --2- 172.21.15.103:0/2864712180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33781026d0 0x7f3378108f60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3370009900 tx=0x7f337002a410 comp rx=0 tx=0).stop 2024-09-19T06:46:31.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.104+0000 7f337da4e640 1 -- 172.21.15.103:0/2864712180 shutdown_connections 2024-09-19T06:46:31.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.104+0000 7f337da4e640 1 --2- 172.21.15.103:0/2864712180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33781026d0 0x7f3378108f60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.104+0000 7f337da4e640 1 --2- 172.21.15.103:0/2864712180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3378101d10 0x7f3378102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.104+0000 7f337da4e640 1 --2- 172.21.15.103:0/2864712180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3378100b10 0x7f3378100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.104+0000 7f337da4e640 1 -- 172.21.15.103:0/2864712180 >> 172.21.15.103:0/2864712180 conn(0x7f33780fc2a0 msgr2=0x7f33780fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:31.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.104+0000 7f337da4e640 1 -- 172.21.15.103:0/2864712180 shutdown_connections 2024-09-19T06:46:31.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.104+0000 7f337da4e640 1 -- 172.21.15.103:0/2864712180 wait complete. 2024-09-19T06:46:31.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.104+0000 7f337da4e640 1 Processor -- start 2024-09-19T06:46:31.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.104+0000 7f337da4e640 1 -- start start 2024-09-19T06:46:31.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f337da4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3378100b10 0x7f3378197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:31.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f337da4e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3378101d10 0x7f33781983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:31.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f337da4e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33781026d0 0x7f337819f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:31.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f337da4e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f337810d8e0 con 0x7f33781026d0 2024-09-19T06:46:31.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f337da4e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f337810d760 con 0x7f3378100b10 2024-09-19T06:46:31.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f337da4e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f337810da60 con 0x7f3378101d10 2024-09-19T06:46:31.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f33777fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33781026d0 0x7f337819f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:31.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f33777fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33781026d0 0x7f337819f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48068/0 (socket says 172.21.15.103:48068) 2024-09-19T06:46:31.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f33777fe640 1 -- 172.21.15.103:0/3597573855 learned_addr learned my addr 172.21.15.103:0/3597573855 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:46:31.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f3376ffd640 1 --2- 172.21.15.103:0/3597573855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3378100b10 0x7f3378197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:31.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f33767fc640 1 --2- 172.21.15.103:0/3597573855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3378101d10 0x7f33781983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:31.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f33777fe640 1 -- 172.21.15.103:0/3597573855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3378101d10 msgr2=0x7f33781983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:31.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f33777fe640 1 --2- 172.21.15.103:0/3597573855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3378101d10 0x7f33781983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f33777fe640 1 -- 172.21.15.103:0/3597573855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3378100b10 msgr2=0x7f3378197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:31.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f33777fe640 1 --2- 172.21.15.103:0/3597573855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3378100b10 0x7f3378197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f33777fe640 1 -- 172.21.15.103:0/3597573855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337819fae0 con 0x7f33781026d0 2024-09-19T06:46:31.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.105+0000 7f33777fe640 1 --2- 172.21.15.103:0/3597573855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33781026d0 0x7f337819f470 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f337002a930 tx=0x7f3370032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:31.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.106+0000 7f337ca4c640 1 -- 172.21.15.103:0/3597573855 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337003f030 con 0x7f33781026d0 2024-09-19T06:46:31.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.106+0000 7f337da4e640 1 -- 172.21.15.103:0/3597573855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337819fd70 con 0x7f33781026d0 2024-09-19T06:46:31.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.106+0000 7f337ca4c640 1 -- 172.21.15.103:0/3597573855 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3370032e50 con 0x7f33781026d0 2024-09-19T06:46:31.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.106+0000 7f337ca4c640 1 -- 172.21.15.103:0/3597573855 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370035070 con 0x7f33781026d0 2024-09-19T06:46:31.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.106+0000 7f337da4e640 1 -- 172.21.15.103:0/3597573855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33781a0240 con 0x7f33781026d0 2024-09-19T06:46:31.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.107+0000 7f337da4e640 1 -- 172.21.15.103:0/3597573855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3378107ee0 con 0x7f33781026d0 2024-09-19T06:46:31.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.108+0000 7f337ca4c640 1 -- 172.21.15.103:0/3597573855 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3370042030 con 0x7f33781026d0 2024-09-19T06:46:31.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.108+0000 7f337ca4c640 1 --2- 172.21.15.103:0/3597573855 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f33600a3270 0x7f33600a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:31.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.109+0000 7f3376ffd640 1 --2- 172.21.15.103:0/3597573855 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f33600a3270 0x7f33600a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:31.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.109+0000 7f337ca4c640 1 -- 172.21.15.103:0/3597573855 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6809+0+0 (secure 0 0 0) 0x7f33700ef7d0 con 0x7f33781026d0 2024-09-19T06:46:31.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.109+0000 7f3376ffd640 1 --2- 172.21.15.103:0/3597573855 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f33600a3270 0x7f33600a5730 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3378101b70 tx=0x7f3368009670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:31.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.113+0000 7f337ca4c640 1 -- 172.21.15.103:0/3597573855 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f33700bc480 con 0x7f33781026d0 2024-09-19T06:46:31.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:30 smithi134 ceph-mon[35356]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:31.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.277+0000 7f337da4e640 1 -- 172.21.15.103:0/3597573855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3378062df0 con 0x7f33781026d0 2024-09-19T06:46:31.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.278+0000 7f337ca4c640 1 -- 172.21.15.103:0/3597573855 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f33700c0780 con 0x7f33781026d0 2024-09-19T06:46:31.279 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:46:31.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.280+0000 7f337da4e640 1 -- 172.21.15.103:0/3597573855 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f33600a3270 msgr2=0x7f33600a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:31.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.280+0000 7f337da4e640 1 --2- 172.21.15.103:0/3597573855 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f33600a3270 0x7f33600a5730 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3378101b70 tx=0x7f3368009670 comp rx=0 tx=0).stop 2024-09-19T06:46:31.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.280+0000 7f337da4e640 1 -- 172.21.15.103:0/3597573855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33781026d0 msgr2=0x7f337819f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:31.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.280+0000 7f337da4e640 1 --2- 172.21.15.103:0/3597573855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33781026d0 0x7f337819f470 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f337002a930 tx=0x7f3370032c60 comp rx=0 tx=0).stop 2024-09-19T06:46:31.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.280+0000 7f337da4e640 1 -- 172.21.15.103:0/3597573855 shutdown_connections 2024-09-19T06:46:31.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.280+0000 7f337da4e640 1 --2- 172.21.15.103:0/3597573855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33781026d0 0x7f337819f470 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.280+0000 7f337da4e640 1 --2- 172.21.15.103:0/3597573855 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f33600a3270 0x7f33600a5730 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.280+0000 7f337da4e640 1 --2- 172.21.15.103:0/3597573855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3378101d10 0x7f33781983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.280+0000 7f337da4e640 1 --2- 172.21.15.103:0/3597573855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3378100b10 0x7f3378197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.280+0000 7f337da4e640 1 -- 172.21.15.103:0/3597573855 >> 172.21.15.103:0/3597573855 conn(0x7f33780fc2a0 msgr2=0x7f33781050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:31.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.280+0000 7f337da4e640 1 -- 172.21.15.103:0/3597573855 shutdown_connections 2024-09-19T06:46:31.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:31.280+0000 7f337da4e640 1 -- 172.21.15.103:0/3597573855 wait complete. 2024-09-19T06:46:31.956 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:31 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3597573855' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:46:32.050 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1726728369,"num_in_osds":12,"osd_in_since":1726728371,"num_remapped_pgs":0} 2024-09-19T06:46:32.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:31 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3597573855' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:46:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:31 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3597573855' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:46:32.833 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 06:46:32 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T06:46:32.455+0000 7f694e8ec640 -1 osd.11 0 log_to_monitors true 2024-09-19T06:46:33.052 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T06:46:33.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:32 smithi103 ceph-mon[31158]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:32 smithi103 ceph-mon[31158]: from='osd.11 [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:46:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:32 smithi103 ceph-mon[31158]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:46:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:32 smithi134 ceph-mon[35356]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:33.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:32 smithi134 ceph-mon[35356]: from='osd.11 [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:46:33.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:32 smithi134 ceph-mon[35356]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:46:33.226 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:32 smithi174 ceph-mon[35637]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:33.226 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:32 smithi174 ceph-mon[35637]: from='osd.11 [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:46:33.226 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:32 smithi174 ceph-mon[35637]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:46:33.314 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:33.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.759+0000 7feb3a532640 1 -- 172.21.15.103:0/3153901311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34104f90 msgr2=0x7feb34107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:33.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.759+0000 7feb3a532640 1 --2- 172.21.15.103:0/3153901311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34104f90 0x7feb34107380 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feb28009900 tx=0x7feb2802a410 comp rx=0 tx=0).stop 2024-09-19T06:46:33.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.759+0000 7feb3a532640 1 -- 172.21.15.103:0/3153901311 shutdown_connections 2024-09-19T06:46:33.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.759+0000 7feb3a532640 1 --2- 172.21.15.103:0/3153901311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb341078c0 0x7feb34109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:33.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.759+0000 7feb3a532640 1 --2- 172.21.15.103:0/3153901311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34104f90 0x7feb34107380 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:33.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.759+0000 7feb3a532640 1 --2- 172.21.15.103:0/3153901311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb34102660 0x7feb34104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:33.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.759+0000 7feb3a532640 1 -- 172.21.15.103:0/3153901311 >> 172.21.15.103:0/3153901311 conn(0x7feb340fc280 msgr2=0x7feb340fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:33.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.759+0000 7feb3a532640 1 -- 172.21.15.103:0/3153901311 shutdown_connections 2024-09-19T06:46:33.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.759+0000 7feb3a532640 1 -- 172.21.15.103:0/3153901311 wait complete. 2024-09-19T06:46:33.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb3a532640 1 Processor -- start 2024-09-19T06:46:33.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb3a532640 1 -- start start 2024-09-19T06:46:33.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb3a532640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34102660 0x7feb34101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:33.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb3a532640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb34104f90 0x7feb34100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:33.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb3a532640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb341078c0 0x7feb34100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:33.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb3a532640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb3410d840 con 0x7feb34102660 2024-09-19T06:46:33.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb3a532640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feb3410d6c0 con 0x7feb341078c0 2024-09-19T06:46:33.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb3a532640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb3410d9c0 con 0x7feb34104f90 2024-09-19T06:46:33.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb33fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34102660 0x7feb34101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:33.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb33fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34102660 0x7feb34101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48090/0 (socket says 172.21.15.103:48090) 2024-09-19T06:46:33.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb33fff640 1 -- 172.21.15.103:0/3775881949 learned_addr learned my addr 172.21.15.103:0/3775881949 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:46:33.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb337fe640 1 --2- 172.21.15.103:0/3775881949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb34104f90 0x7feb34100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:33.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb38acf640 1 --2- 172.21.15.103:0/3775881949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb341078c0 0x7feb34100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:33.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb33fff640 1 -- 172.21.15.103:0/3775881949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb34104f90 msgr2=0x7feb34100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:33.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb33fff640 1 --2- 172.21.15.103:0/3775881949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb34104f90 0x7feb34100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:33.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb33fff640 1 -- 172.21.15.103:0/3775881949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb341078c0 msgr2=0x7feb34100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:33.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb33fff640 1 --2- 172.21.15.103:0/3775881949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb341078c0 0x7feb34100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:33.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb33fff640 1 -- 172.21.15.103:0/3775881949 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb34101410 con 0x7feb34102660 2024-09-19T06:46:33.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.760+0000 7feb38acf640 1 --2- 172.21.15.103:0/3775881949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb341078c0 0x7feb34100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:33.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.761+0000 7feb33fff640 1 --2- 172.21.15.103:0/3775881949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34102660 0x7feb34101fc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feb2400d950 tx=0x7feb2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:33.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.761+0000 7feb317fa640 1 -- 172.21.15.103:0/3775881949 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24017030 con 0x7feb34102660 2024-09-19T06:46:33.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.761+0000 7feb3a532640 1 -- 172.21.15.103:0/3775881949 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb341a20b0 con 0x7feb34102660 2024-09-19T06:46:33.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.761+0000 7feb317fa640 1 -- 172.21.15.103:0/3775881949 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb24004020 con 0x7feb34102660 2024-09-19T06:46:33.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.761+0000 7feb317fa640 1 -- 172.21.15.103:0/3775881949 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb240047a0 con 0x7feb34102660 2024-09-19T06:46:33.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.761+0000 7feb3a532640 1 -- 172.21.15.103:0/3775881949 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb341a2610 con 0x7feb34102660 2024-09-19T06:46:33.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.762+0000 7feb3a532640 1 -- 172.21.15.103:0/3775881949 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb340fdd70 con 0x7feb34102660 2024-09-19T06:46:33.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.764+0000 7feb317fa640 1 -- 172.21.15.103:0/3775881949 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7feb240041d0 con 0x7feb34102660 2024-09-19T06:46:33.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.765+0000 7feb317fa640 1 --2- 172.21.15.103:0/3775881949 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feb1c0a3240 0x7feb1c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:33.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.765+0000 7feb317fa640 1 -- 172.21.15.103:0/3775881949 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6831+0+0 (secure 0 0 0) 0x7feb240cf4b0 con 0x7feb34102660 2024-09-19T06:46:33.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.765+0000 7feb337fe640 1 --2- 172.21.15.103:0/3775881949 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feb1c0a3240 0x7feb1c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:33.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.765+0000 7feb337fe640 1 --2- 172.21.15.103:0/3775881949 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feb1c0a3240 0x7feb1c0a5700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feb28005ab0 tx=0x7feb28000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:33.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.768+0000 7feb317fa640 1 -- 172.21.15.103:0/3775881949 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feb2409c700 con 0x7feb34102660 2024-09-19T06:46:33.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.938+0000 7feb3a532640 1 -- 172.21.15.103:0/3775881949 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7feb3402da50 con 0x7feb34102660 2024-09-19T06:46:33.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.938+0000 7feb317fa640 1 -- 172.21.15.103:0/3775881949 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7feb2409c000 con 0x7feb34102660 2024-09-19T06:46:33.940 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:46:33.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.941+0000 7feb3a532640 1 -- 172.21.15.103:0/3775881949 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feb1c0a3240 msgr2=0x7feb1c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:33.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.941+0000 7feb3a532640 1 --2- 172.21.15.103:0/3775881949 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feb1c0a3240 0x7feb1c0a5700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feb28005ab0 tx=0x7feb28000bc0 comp rx=0 tx=0).stop 2024-09-19T06:46:33.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.941+0000 7feb3a532640 1 -- 172.21.15.103:0/3775881949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34102660 msgr2=0x7feb34101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:33.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.941+0000 7feb3a532640 1 --2- 172.21.15.103:0/3775881949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34102660 0x7feb34101fc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feb2400d950 tx=0x7feb2400de10 comp rx=0 tx=0).stop 2024-09-19T06:46:33.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.941+0000 7feb3a532640 1 -- 172.21.15.103:0/3775881949 shutdown_connections 2024-09-19T06:46:33.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.941+0000 7feb3a532640 1 --2- 172.21.15.103:0/3775881949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb341078c0 0x7feb34100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:33.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.941+0000 7feb3a532640 1 --2- 172.21.15.103:0/3775881949 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feb1c0a3240 0x7feb1c0a5700 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:33.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.941+0000 7feb3a532640 1 --2- 172.21.15.103:0/3775881949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb34104f90 0x7feb34100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:33.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.941+0000 7feb3a532640 1 --2- 172.21.15.103:0/3775881949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34102660 0x7feb34101fc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:33.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.941+0000 7feb3a532640 1 -- 172.21.15.103:0/3775881949 >> 172.21.15.103:0/3775881949 conn(0x7feb340fc280 msgr2=0x7feb341030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:33.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.941+0000 7feb3a532640 1 -- 172.21.15.103:0/3775881949 shutdown_connections 2024-09-19T06:46:33.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:33.941+0000 7feb3a532640 1 -- 172.21.15.103:0/3775881949 wait complete. 2024-09-19T06:46:34.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:33 smithi103 ceph-mon[31158]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T06:46:34.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:33 smithi103 ceph-mon[31158]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T06:46:34.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:34.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:33 smithi103 ceph-mon[31158]: from='osd.11 [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:46:34.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:33 smithi103 ceph-mon[31158]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:46:34.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:34.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:34.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:34.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:34.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:34.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:34.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:34.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:34.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:34.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:33 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:34.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:33 smithi134 ceph-mon[35356]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T06:46:34.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:33 smithi134 ceph-mon[35356]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T06:46:34.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:34.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:33 smithi134 ceph-mon[35356]: from='osd.11 [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:46:34.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:33 smithi134 ceph-mon[35356]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:46:34.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:34.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:34.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:34.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:34.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:34.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:34.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:34.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:34.190 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:34.190 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:33 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:33 smithi174 ceph-mon[35637]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T06:46:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:33 smithi174 ceph-mon[35637]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T06:46:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:33 smithi174 ceph-mon[35637]: from='osd.11 [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:46:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:33 smithi174 ceph-mon[35637]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T06:46:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:34.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:34.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:34.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:34.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:46:34.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:34.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:34.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:34.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:33 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:34.808 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1726728369,"num_in_osds":12,"osd_in_since":1726728371,"num_remapped_pgs":0} 2024-09-19T06:46:35.083 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 06:46:34 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T06:46:34.647+0000 7f694a04c640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T06:46:35.083 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 06:46:34 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T06:46:34.664+0000 7f694267e640 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:46:35.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:34 smithi174 ceph-mon[35637]: Detected new or changed devices on smithi174 2024-09-19T06:46:35.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:34 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-19T06:46:35.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:34 smithi174 ceph-mon[35637]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:35.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:34 smithi174 ceph-mon[35637]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T06:46:35.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:34 smithi174 ceph-mon[35637]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T06:46:35.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:34 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:35.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:34 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:35.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:34 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3775881949' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:46:35.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:34 smithi174 ceph-mon[35637]: from='osd.11 ' entity='osd.11' 2024-09-19T06:46:35.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:34 smithi103 ceph-mon[31158]: Detected new or changed devices on smithi174 2024-09-19T06:46:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:34 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-19T06:46:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:34 smithi103 ceph-mon[31158]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:34 smithi103 ceph-mon[31158]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T06:46:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:34 smithi103 ceph-mon[31158]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T06:46:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:34 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:34 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:34 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3775881949' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:46:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:34 smithi103 ceph-mon[31158]: from='osd.11 ' entity='osd.11' 2024-09-19T06:46:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:34 smithi134 ceph-mon[35356]: Detected new or changed devices on smithi174 2024-09-19T06:46:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:34 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-19T06:46:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:34 smithi134 ceph-mon[35356]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:34 smithi134 ceph-mon[35356]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T06:46:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:34 smithi134 ceph-mon[35356]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T06:46:35.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:34 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:35.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:34 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:35.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:34 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3775881949' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:46:35.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:34 smithi134 ceph-mon[35356]: from='osd.11 ' entity='osd.11' 2024-09-19T06:46:35.809 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T06:46:36.069 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:36.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:35 smithi103 ceph-mon[31158]: purged_snaps scrub starts 2024-09-19T06:46:36.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:35 smithi103 ceph-mon[31158]: purged_snaps scrub ok 2024-09-19T06:46:36.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:35 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:36.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:35 smithi103 ceph-mon[31158]: osd.11 [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442] boot 2024-09-19T06:46:36.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:35 smithi103 ceph-mon[31158]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T06:46:36.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:35 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:36.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:35 smithi134 ceph-mon[35356]: purged_snaps scrub starts 2024-09-19T06:46:36.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:35 smithi134 ceph-mon[35356]: purged_snaps scrub ok 2024-09-19T06:46:36.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:35 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:36.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:35 smithi134 ceph-mon[35356]: osd.11 [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442] boot 2024-09-19T06:46:36.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:35 smithi134 ceph-mon[35356]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T06:46:36.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:35 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:35 smithi174 ceph-mon[35637]: purged_snaps scrub starts 2024-09-19T06:46:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:35 smithi174 ceph-mon[35637]: purged_snaps scrub ok 2024-09-19T06:46:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:35 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:35 smithi174 ceph-mon[35637]: osd.11 [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442] boot 2024-09-19T06:46:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:35 smithi174 ceph-mon[35637]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T06:46:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:35 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:46:36.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.500+0000 7fc21ddf7640 1 -- 172.21.15.103:0/1949205148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc218100ad0 msgr2=0x7fc218100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:36.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.500+0000 7fc21ddf7640 1 --2- 172.21.15.103:0/1949205148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc218100ad0 0x7fc218100ed0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc208009900 tx=0x7fc20802a410 comp rx=0 tx=0).stop 2024-09-19T06:46:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.500+0000 7fc21ddf7640 1 -- 172.21.15.103:0/1949205148 shutdown_connections 2024-09-19T06:46:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.500+0000 7fc21ddf7640 1 --2- 172.21.15.103:0/1949205148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc218102690 0x7fc218108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.500+0000 7fc21ddf7640 1 --2- 172.21.15.103:0/1949205148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc218101cd0 0x7fc218102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.500+0000 7fc21ddf7640 1 --2- 172.21.15.103:0/1949205148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc218100ad0 0x7fc218100ed0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.500+0000 7fc21ddf7640 1 -- 172.21.15.103:0/1949205148 >> 172.21.15.103:0/1949205148 conn(0x7fc2180fc280 msgr2=0x7fc2180fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.500+0000 7fc21ddf7640 1 -- 172.21.15.103:0/1949205148 shutdown_connections 2024-09-19T06:46:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.500+0000 7fc21ddf7640 1 -- 172.21.15.103:0/1949205148 wait complete. 2024-09-19T06:46:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc21ddf7640 1 Processor -- start 2024-09-19T06:46:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc21ddf7640 1 -- start start 2024-09-19T06:46:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc21ddf7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc218100ad0 0x7fc21806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc21ddf7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc218101cd0 0x7fc21819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc21ddf7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc218102690 0x7fc21819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc21ddf7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc21810c830 con 0x7fc218102690 2024-09-19T06:46:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc21ddf7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc21810c6b0 con 0x7fc218101cd0 2024-09-19T06:46:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc21ddf7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc21810c9b0 con 0x7fc218100ad0 2024-09-19T06:46:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc217fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc218102690 0x7fc21819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc217fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc218102690 0x7fc21819f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48104/0 (socket says 172.21.15.103:48104) 2024-09-19T06:46:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc217fff640 1 -- 172.21.15.103:0/2986714084 learned_addr learned my addr 172.21.15.103:0/2986714084 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:46:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc20ffff640 1 --2- 172.21.15.103:0/2986714084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc218101cd0 0x7fc21819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc2177fe640 1 --2- 172.21.15.103:0/2986714084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc218100ad0 0x7fc21806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc217fff640 1 -- 172.21.15.103:0/2986714084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc218100ad0 msgr2=0x7fc21806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc217fff640 1 --2- 172.21.15.103:0/2986714084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc218100ad0 0x7fc21806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc217fff640 1 -- 172.21.15.103:0/2986714084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc218101cd0 msgr2=0x7fc21819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc217fff640 1 --2- 172.21.15.103:0/2986714084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc218101cd0 0x7fc21819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc217fff640 1 -- 172.21.15.103:0/2986714084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21819fd30 con 0x7fc218102690 2024-09-19T06:46:36.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.501+0000 7fc217fff640 1 --2- 172.21.15.103:0/2986714084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc218102690 0x7fc21819f420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc21000b970 tx=0x7fc21000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:36.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.502+0000 7fc2157fa640 1 -- 172.21.15.103:0/2986714084 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210019070 con 0x7fc218102690 2024-09-19T06:46:36.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.502+0000 7fc2157fa640 1 -- 172.21.15.103:0/2986714084 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc21000cb40 con 0x7fc218102690 2024-09-19T06:46:36.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.502+0000 7fc21ddf7640 1 -- 172.21.15.103:0/2986714084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc21819ffc0 con 0x7fc218102690 2024-09-19T06:46:36.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.502+0000 7fc2157fa640 1 -- 172.21.15.103:0/2986714084 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210007650 con 0x7fc218102690 2024-09-19T06:46:36.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.502+0000 7fc21ddf7640 1 -- 172.21.15.103:0/2986714084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2181a0500 con 0x7fc218102690 2024-09-19T06:46:36.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.503+0000 7fc21ddf7640 1 -- 172.21.15.103:0/2986714084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc218107f70 con 0x7fc218102690 2024-09-19T06:46:36.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.505+0000 7fc2157fa640 1 -- 172.21.15.103:0/2986714084 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc21001c070 con 0x7fc218102690 2024-09-19T06:46:36.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.505+0000 7fc2157fa640 1 --2- 172.21.15.103:0/2986714084 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc2040a3240 0x7fc2040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.505+0000 7fc2177fe640 1 --2- 172.21.15.103:0/2986714084 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc2040a3240 0x7fc2040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.505+0000 7fc2157fa640 1 -- 172.21.15.103:0/2986714084 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fc2100cef40 con 0x7fc218102690 2024-09-19T06:46:36.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.506+0000 7fc2177fe640 1 --2- 172.21.15.103:0/2986714084 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc2040a3240 0x7fc2040a5700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc208006010 tx=0x7fc208000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:36.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.508+0000 7fc2157fa640 1 -- 172.21.15.103:0/2986714084 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc21009c100 con 0x7fc218102690 2024-09-19T06:46:36.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.677+0000 7fc21ddf7640 1 -- 172.21.15.103:0/2986714084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc21802d9c0 con 0x7fc218102690 2024-09-19T06:46:36.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.677+0000 7fc2157fa640 1 -- 172.21.15.103:0/2986714084 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7fc21009ba00 con 0x7fc218102690 2024-09-19T06:46:36.679 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:46:36.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.679+0000 7fc21ddf7640 1 -- 172.21.15.103:0/2986714084 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc2040a3240 msgr2=0x7fc2040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:36.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.679+0000 7fc21ddf7640 1 --2- 172.21.15.103:0/2986714084 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc2040a3240 0x7fc2040a5700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc208006010 tx=0x7fc208000bc0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.679+0000 7fc21ddf7640 1 -- 172.21.15.103:0/2986714084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc218102690 msgr2=0x7fc21819f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:36.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.679+0000 7fc21ddf7640 1 --2- 172.21.15.103:0/2986714084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc218102690 0x7fc21819f420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc21000b970 tx=0x7fc21000be30 comp rx=0 tx=0).stop 2024-09-19T06:46:36.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.679+0000 7fc21ddf7640 1 -- 172.21.15.103:0/2986714084 shutdown_connections 2024-09-19T06:46:36.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.679+0000 7fc21ddf7640 1 --2- 172.21.15.103:0/2986714084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc218102690 0x7fc21819f420 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.679+0000 7fc21ddf7640 1 --2- 172.21.15.103:0/2986714084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc218101cd0 0x7fc21819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.679+0000 7fc21ddf7640 1 --2- 172.21.15.103:0/2986714084 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc2040a3240 0x7fc2040a5700 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.679+0000 7fc21ddf7640 1 --2- 172.21.15.103:0/2986714084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc218100ad0 0x7fc21806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.679+0000 7fc21ddf7640 1 -- 172.21.15.103:0/2986714084 >> 172.21.15.103:0/2986714084 conn(0x7fc2180fc280 msgr2=0x7fc218105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:36.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.680+0000 7fc21ddf7640 1 -- 172.21.15.103:0/2986714084 shutdown_connections 2024-09-19T06:46:36.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:36.680+0000 7fc21ddf7640 1 -- 172.21.15.103:0/2986714084 wait complete. 2024-09-19T06:46:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:36 smithi134 ceph-mon[35356]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:36 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2986714084' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:46:37.214 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:36 smithi103 ceph-mon[31158]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:37.214 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:36 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2986714084' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:46:37.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:36 smithi174 ceph-mon[35637]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:46:37.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:36 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2986714084' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:46:37.363 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":74,"num_osds":12,"num_up_osds":12,"osd_up_since":1726728395,"num_in_osds":12,"osd_in_since":1726728371,"num_remapped_pgs":0} 2024-09-19T06:46:37.363 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:46:37.604 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:37.988 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:37 smithi103 ceph-mon[31158]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T06:46:38.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.066+0000 7f316163b640 1 -- 172.21.15.103:0/3990233385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f315c101420 msgr2=0x7f315c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:38.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.066+0000 7f316163b640 1 --2- 172.21.15.103:0/3990233385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f315c101420 0x7f315c10d950 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3154009cd0 tx=0x7f315402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.066+0000 7f316163b640 1 -- 172.21.15.103:0/3990233385 shutdown_connections 2024-09-19T06:46:38.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.066+0000 7f316163b640 1 --2- 172.21.15.103:0/3990233385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f315c101420 0x7f315c10d950 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.066+0000 7f316163b640 1 --2- 172.21.15.103:0/3990233385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f315c100a80 0x7f315c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.066+0000 7f316163b640 1 --2- 172.21.15.103:0/3990233385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f315c106a80 0x7f315c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.066+0000 7f316163b640 1 -- 172.21.15.103:0/3990233385 >> 172.21.15.103:0/3990233385 conn(0x7f315c0fc7a0 msgr2=0x7f315c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:38.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.067+0000 7f316163b640 1 -- 172.21.15.103:0/3990233385 shutdown_connections 2024-09-19T06:46:38.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.067+0000 7f316163b640 1 -- 172.21.15.103:0/3990233385 wait complete. 2024-09-19T06:46:38.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.067+0000 7f316163b640 1 Processor -- start 2024-09-19T06:46:38.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.067+0000 7f316163b640 1 -- start start 2024-09-19T06:46:38.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f316163b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f315c100a80 0x7f315c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:38.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f316163b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f315c101420 0x7f315c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:38.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f316163b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f315c106a80 0x7f315c1a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:38.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f316163b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f315c110e80 con 0x7f315c100a80 2024-09-19T06:46:38.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f316163b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f315c110d00 con 0x7f315c106a80 2024-09-19T06:46:38.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f316163b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f315c111000 con 0x7f315c101420 2024-09-19T06:46:38.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f315affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f315c100a80 0x7f315c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:38.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f315affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f315c100a80 0x7f315c1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54040/0 (socket says 172.21.15.103:54040) 2024-09-19T06:46:38.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f315affd640 1 -- 172.21.15.103:0/3042163798 learned_addr learned my addr 172.21.15.103:0/3042163798 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:46:38.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f315b7fe640 1 --2- 172.21.15.103:0/3042163798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f315c106a80 0x7f315c1a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:38.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f315affd640 1 -- 172.21.15.103:0/3042163798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f315c101420 msgr2=0x7f315c1997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:38.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f315a7fc640 1 --2- 172.21.15.103:0/3042163798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f315c101420 0x7f315c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:38.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f315affd640 1 --2- 172.21.15.103:0/3042163798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f315c101420 0x7f315c1997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f315affd640 1 -- 172.21.15.103:0/3042163798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f315c106a80 msgr2=0x7f315c1a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:38.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f315affd640 1 --2- 172.21.15.103:0/3042163798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f315c106a80 0x7f315c1a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f315affd640 1 -- 172.21.15.103:0/3042163798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315c1a4870 con 0x7f315c100a80 2024-09-19T06:46:38.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f315affd640 1 --2- 172.21.15.103:0/3042163798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f315c100a80 0x7f315c1992b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f314c00d950 tx=0x7f314c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:38.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f3143fff640 1 -- 172.21.15.103:0/3042163798 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314c017070 con 0x7f315c100a80 2024-09-19T06:46:38.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f3143fff640 1 -- 172.21.15.103:0/3042163798 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f314c009c30 con 0x7f315c100a80 2024-09-19T06:46:38.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.068+0000 7f316163b640 1 -- 172.21.15.103:0/3042163798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f315c1a4a40 con 0x7f315c100a80 2024-09-19T06:46:38.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.069+0000 7f3143fff640 1 -- 172.21.15.103:0/3042163798 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314c00f6b0 con 0x7f315c100a80 2024-09-19T06:46:38.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.069+0000 7f316163b640 1 -- 172.21.15.103:0/3042163798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f315c1a4f50 con 0x7f315c100a80 2024-09-19T06:46:38.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.069+0000 7f316163b640 1 -- 172.21.15.103:0/3042163798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f315c100ee0 con 0x7f315c100a80 2024-09-19T06:46:38.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.071+0000 7f3143fff640 1 -- 172.21.15.103:0/3042163798 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f314c0109d0 con 0x7f315c100a80 2024-09-19T06:46:38.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.072+0000 7f3143fff640 1 --2- 172.21.15.103:0/3042163798 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f31440a3240 0x7f31440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:38.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.072+0000 7f315a7fc640 1 --2- 172.21.15.103:0/3042163798 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f31440a3240 0x7f31440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:38.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.072+0000 7f3143fff640 1 -- 172.21.15.103:0/3042163798 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f314c0cf0b0 con 0x7f315c100a80 2024-09-19T06:46:38.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.072+0000 7f315a7fc640 1 --2- 172.21.15.103:0/3042163798 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f31440a3240 0x7f31440a5700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f315c06b2e0 tx=0x7f3150005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:38.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.075+0000 7f3143fff640 1 -- 172.21.15.103:0/3042163798 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f314c009de0 con 0x7f315c100a80 2024-09-19T06:46:38.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.241+0000 7f316163b640 1 -- 172.21.15.103:0/3042163798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f315c06ad80 con 0x7f315c100a80 2024-09-19T06:46:38.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.242+0000 7f3143fff640 1 -- 172.21.15.103:0/3042163798 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+18540 (secure 0 0 0) 0x7f314c09c270 con 0x7f315c100a80 2024-09-19T06:46:38.244 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:46:38.244 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":75,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","created":"2024-09-19T06:39:46.840574+0000","modified":"2024-09-19T06:46:36.888108+0000","last_up_change":"2024-09-19T06:46:35.652960+0000","last_in_change":"2024-09-19T06:46:11.630585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:42:52.050881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d45dab3-e434-4783-b2e3-f658c3ba583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6803","nonce":3157697740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6805","nonce":3157697740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6809","nonce":3157697740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6807","nonce":3157697740}]},"public_addr":"172.21.15.103:6803/3157697740","cluster_addr":"172.21.15.103:6805/3157697740","heartbeat_back_addr":"172.21.15.103:6809/3157697740","heartbeat_front_addr":"172.21.15.103:6807/3157697740","state":["exists","up"]},{"osd":1,"uuid":"ee0e92bd-15ab-4867-9a18-20abb9f5fc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6801","nonce":1418301195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6803","nonce":1418301195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6807","nonce":1418301195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6805","nonce":1418301195}]},"public_addr":"172.21.15.134:6801/1418301195","cluster_addr":"172.21.15.134:6803/1418301195","heartbeat_back_addr":"172.21.15.134:6807/1418301195","heartbeat_front_addr":"172.21.15.134:6805/1418301195","state":["exists","up"]},{"osd":2,"uuid":"0230caed-025b-46e9-a011-237601d16ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6801","nonce":2292242290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6803","nonce":2292242290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6807","nonce":2292242290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6805","nonce":2292242290}]},"public_addr":"172.21.15.174:6801/2292242290","cluster_addr":"172.21.15.174:6803/2292242290","heartbeat_back_addr":"172.21.15.174:6807/2292242290","heartbeat_front_addr":"172.21.15.174:6805/2292242290","state":["exists","up"]},{"osd":3,"uuid":"c54a8399-bf4b-4d70-b207-a13d85f99590","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6811","nonce":2698986333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6813","nonce":2698986333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6817","nonce":2698986333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6815","nonce":2698986333}]},"public_addr":"172.21.15.103:6811/2698986333","cluster_addr":"172.21.15.103:6813/2698986333","heartbeat_back_addr":"172.21.15.103:6817/2698986333","heartbeat_front_addr":"172.21.15.103:6815/2698986333","state":["exists","up"]},{"osd":4,"uuid":"7c046be6-464a-4ab5-812b-1781121fc317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6809","nonce":3169891968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6811","nonce":3169891968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6815","nonce":3169891968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6813","nonce":3169891968}]},"public_addr":"172.21.15.134:6809/3169891968","cluster_addr":"172.21.15.134:6811/3169891968","heartbeat_back_addr":"172.21.15.134:6815/3169891968","heartbeat_front_addr":"172.21.15.134:6813/3169891968","state":["exists","up"]},{"osd":5,"uuid":"8622596b-6a06-459c-882d-50fba9e296ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6809","nonce":763749644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6811","nonce":763749644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6815","nonce":763749644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6813","nonce":763749644}]},"public_addr":"172.21.15.174:6809/763749644","cluster_addr":"172.21.15.174:6811/763749644","heartbeat_back_addr":"172.21.15.174:6815/763749644","heartbeat_front_addr":"172.21.15.174:6813/763749644","state":["exists","up"]},{"osd":6,"uuid":"64d858d6-a8b3-4f65-a187-af9ab1ccee0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6819","nonce":1396972394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6821","nonce":1396972394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6825","nonce":1396972394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6823","nonce":1396972394}]},"public_addr":"172.21.15.103:6819/1396972394","cluster_addr":"172.21.15.103:6821/1396972394","heartbeat_back_addr":"172.21.15.103:6825/1396972394","heartbeat_front_addr":"172.21.15.103:6823/1396972394","state":["exists","up"]},{"osd":7,"uuid":"d405d1c6-839c-4c92-8261-4adb3163c90f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6817","nonce":2196366765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6819","nonce":2196366765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6823","nonce":2196366765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6821","nonce":2196366765}]},"public_addr":"172.21.15.134:6817/2196366765","cluster_addr":"172.21.15.134:6819/2196366765","heartbeat_back_addr":"172.21.15.134:6823/2196366765","heartbeat_front_addr":"172.21.15.134:6821/2196366765","state":["exists","up"]},{"osd":8,"uuid":"4c452cd9-816c-4f77-9368-2cd4bbeee4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6817","nonce":189977465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6819","nonce":189977465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6823","nonce":189977465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6821","nonce":189977465}]},"public_addr":"172.21.15.174:6817/189977465","cluster_addr":"172.21.15.174:6819/189977465","heartbeat_back_addr":"172.21.15.174:6823/189977465","heartbeat_front_addr":"172.21.15.174:6821/189977465","state":["exists","up"]},{"osd":9,"uuid":"fb3662e0-2d8e-45c4-9119-6d08b35443da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6827","nonce":3161966413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6829","nonce":3161966413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6833","nonce":3161966413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6831","nonce":3161966413}]},"public_addr":"172.21.15.103:6827/3161966413","cluster_addr":"172.21.15.103:6829/3161966413","heartbeat_back_addr":"172.21.15.103:6833/3161966413","heartbeat_front_addr":"172.21.15.103:6831/3161966413","state":["exists","up"]},{"osd":10,"uuid":"e52c3cbe-a6af-441e-b423-b94f3a2da544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6825","nonce":2532758632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6827","nonce":2532758632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6831","nonce":2532758632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6829","nonce":2532758632}]},"public_addr":"172.21.15.134:6825/2532758632","cluster_addr":"172.21.15.134:6827/2532758632","heartbeat_back_addr":"172.21.15.134:6831/2532758632","heartbeat_front_addr":"172.21.15.134:6829/2532758632","state":["exists","up"]},{"osd":11,"uuid":"9040f3f8-3873-4dea-8716-dcda5811c8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6825","nonce":161916442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6827","nonce":161916442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6831","nonce":161916442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6829","nonce":161916442}]},"public_addr":"172.21.15.174:6825/161916442","cluster_addr":"172.21.15.174:6827/161916442","heartbeat_back_addr":"172.21.15.174:6831/161916442","heartbeat_front_addr":"172.21.15.174:6829/161916442","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:41:59.656854+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:24.797551+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:49.685497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:33.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:39.324983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:03.987310+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:28.747971+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:53.827780+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:18.554450+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:43.705691+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:08.401275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:33.502542+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2867049580":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/3727259725":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/1496153385":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:6800/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1642809745":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/2309624637":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1596229922":"2024-09-20T06:40:16.780890+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:46:38.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.245+0000 7f316163b640 1 -- 172.21.15.103:0/3042163798 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f31440a3240 msgr2=0x7f31440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:38.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.245+0000 7f316163b640 1 --2- 172.21.15.103:0/3042163798 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f31440a3240 0x7f31440a5700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f315c06b2e0 tx=0x7f3150005c60 comp rx=0 tx=0).stop 2024-09-19T06:46:38.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.245+0000 7f316163b640 1 -- 172.21.15.103:0/3042163798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f315c100a80 msgr2=0x7f315c1992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:38.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.245+0000 7f316163b640 1 --2- 172.21.15.103:0/3042163798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f315c100a80 0x7f315c1992b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f314c00d950 tx=0x7f314c00de10 comp rx=0 tx=0).stop 2024-09-19T06:46:38.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.245+0000 7f316163b640 1 -- 172.21.15.103:0/3042163798 shutdown_connections 2024-09-19T06:46:38.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.245+0000 7f316163b640 1 --2- 172.21.15.103:0/3042163798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f315c106a80 0x7f315c1a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.245+0000 7f316163b640 1 --2- 172.21.15.103:0/3042163798 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f31440a3240 0x7f31440a5700 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.245+0000 7f316163b640 1 --2- 172.21.15.103:0/3042163798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f315c101420 0x7f315c1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.245+0000 7f316163b640 1 --2- 172.21.15.103:0/3042163798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f315c100a80 0x7f315c1992b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.245+0000 7f316163b640 1 -- 172.21.15.103:0/3042163798 >> 172.21.15.103:0/3042163798 conn(0x7f315c0fc7a0 msgr2=0x7f315c0fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:38.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.245+0000 7f316163b640 1 -- 172.21.15.103:0/3042163798 shutdown_connections 2024-09-19T06:46:38.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:38.245+0000 7f316163b640 1 -- 172.21.15.103:0/3042163798 wait complete. 2024-09-19T06:46:38.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:37 smithi174 ceph-mon[35637]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T06:46:38.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:37 smithi134 ceph-mon[35356]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T06:46:38.921 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T06:42:52.050881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T06:46:38.921 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T06:46:39.162 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:39.283 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:38 smithi103 ceph-mon[31158]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:39.283 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:38 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3042163798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:46:39.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:38 smithi174 ceph-mon[35637]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:39.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:38 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3042163798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:46:39.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:38 smithi134 ceph-mon[35356]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:39.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:38 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3042163798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:46:39.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.600+0000 7f73b0949640 1 -- 172.21.15.103:0/375554187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73ac1036e0 msgr2=0x7f73ac107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:39.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.600+0000 7f73b0949640 1 --2- 172.21.15.103:0/375554187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73ac1036e0 0x7f73ac107b90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f739c009930 tx=0x7f739c02a420 comp rx=0 tx=0).stop 2024-09-19T06:46:39.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.600+0000 7f73b0949640 1 -- 172.21.15.103:0/375554187 shutdown_connections 2024-09-19T06:46:39.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.600+0000 7f73b0949640 1 --2- 172.21.15.103:0/375554187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ac1082a0 0x7f73ac10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:39.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.600+0000 7f73b0949640 1 --2- 172.21.15.103:0/375554187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73ac1036e0 0x7f73ac107b90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:39.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.600+0000 7f73b0949640 1 --2- 172.21.15.103:0/375554187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73ac102d30 0x7f73ac103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:39.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.600+0000 7f73b0949640 1 -- 172.21.15.103:0/375554187 >> 172.21.15.103:0/375554187 conn(0x7f73ac0fc7d0 msgr2=0x7f73ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:39.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.600+0000 7f73b0949640 1 -- 172.21.15.103:0/375554187 shutdown_connections 2024-09-19T06:46:39.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.600+0000 7f73b0949640 1 -- 172.21.15.103:0/375554187 wait complete. 2024-09-19T06:46:39.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.601+0000 7f73b0949640 1 Processor -- start 2024-09-19T06:46:39.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.601+0000 7f73b0949640 1 -- start start 2024-09-19T06:46:39.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.601+0000 7f73b0949640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73ac102d30 0x7f73ac198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:39.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.601+0000 7f73b0949640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73ac1036e0 0x7f73ac198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:39.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.601+0000 7f73b0949640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ac1082a0 0x7f73ac19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:39.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.601+0000 7f73b0949640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac112060 con 0x7f73ac102d30 2024-09-19T06:46:39.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.601+0000 7f73b0949640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac111ee0 con 0x7f73ac1082a0 2024-09-19T06:46:39.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.601+0000 7f73b0949640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1121e0 con 0x7f73ac1036e0 2024-09-19T06:46:39.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.601+0000 7f73aa59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73ac102d30 0x7f73ac198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:39.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.601+0000 7f73aa59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73ac102d30 0x7f73ac198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54052/0 (socket says 172.21.15.103:54052) 2024-09-19T06:46:39.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.601+0000 7f73aa59c640 1 -- 172.21.15.103:0/4061652323 learned_addr learned my addr 172.21.15.103:0/4061652323 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:46:39.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.602+0000 7f73aad9d640 1 --2- 172.21.15.103:0/4061652323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ac1082a0 0x7f73ac19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:39.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.602+0000 7f73a9d9b640 1 --2- 172.21.15.103:0/4061652323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73ac1036e0 0x7f73ac198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:39.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.602+0000 7f73aa59c640 1 -- 172.21.15.103:0/4061652323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73ac1036e0 msgr2=0x7f73ac198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:39.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.602+0000 7f73aa59c640 1 --2- 172.21.15.103:0/4061652323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73ac1036e0 0x7f73ac198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:39.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.602+0000 7f73aa59c640 1 -- 172.21.15.103:0/4061652323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ac1082a0 msgr2=0x7f73ac19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:39.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.602+0000 7f73aa59c640 1 --2- 172.21.15.103:0/4061652323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ac1082a0 0x7f73ac19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:39.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.602+0000 7f73aa59c640 1 -- 172.21.15.103:0/4061652323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ac19d520 con 0x7f73ac102d30 2024-09-19T06:46:39.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.602+0000 7f73aad9d640 1 --2- 172.21.15.103:0/4061652323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ac1082a0 0x7f73ac19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:39.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.602+0000 7f73aa59c640 1 --2- 172.21.15.103:0/4061652323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73ac102d30 0x7f73ac198570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f739400c920 tx=0x7f739400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:39.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.602+0000 7f73a37fe640 1 -- 172.21.15.103:0/4061652323 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394007c70 con 0x7f73ac102d30 2024-09-19T06:46:39.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.602+0000 7f73a37fe640 1 -- 172.21.15.103:0/4061652323 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7394007e10 con 0x7f73ac102d30 2024-09-19T06:46:39.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.602+0000 7f73b0949640 1 -- 172.21.15.103:0/4061652323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ac19d810 con 0x7f73ac102d30 2024-09-19T06:46:39.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.602+0000 7f73a37fe640 1 -- 172.21.15.103:0/4061652323 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394018890 con 0x7f73ac102d30 2024-09-19T06:46:39.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.602+0000 7f73b0949640 1 -- 172.21.15.103:0/4061652323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ac1a50f0 con 0x7f73ac102d30 2024-09-19T06:46:39.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.603+0000 7f73b0949640 1 -- 172.21.15.103:0/4061652323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ac103110 con 0x7f73ac102d30 2024-09-19T06:46:39.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.604+0000 7f73a37fe640 1 -- 172.21.15.103:0/4061652323 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f7394018a70 con 0x7f73ac102d30 2024-09-19T06:46:39.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.605+0000 7f73a37fe640 1 --2- 172.21.15.103:0/4061652323 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f73980a3240 0x7f73980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:39.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.605+0000 7f73a9d9b640 1 --2- 172.21.15.103:0/4061652323 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f73980a3240 0x7f73980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:39.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.605+0000 7f73a37fe640 1 -- 172.21.15.103:0/4061652323 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f73940cf670 con 0x7f73ac102d30 2024-09-19T06:46:39.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.605+0000 7f73a9d9b640 1 --2- 172.21.15.103:0/4061652323 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f73980a3240 0x7f73980a5700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f739c009230 tx=0x7f739c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:39.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.609+0000 7f73a37fe640 1 -- 172.21.15.103:0/4061652323 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7394014030 con 0x7f73ac102d30 2024-09-19T06:46:39.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.776+0000 7f73b0949640 1 -- 172.21.15.103:0/4061652323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f73ac10a2b0 con 0x7f73ac102d30 2024-09-19T06:46:39.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.777+0000 7f73a37fe640 1 -- 172.21.15.103:0/4061652323 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7f739400de20 con 0x7f73ac102d30 2024-09-19T06:46:39.778 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 1 2024-09-19T06:46:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.779+0000 7f73b0949640 1 -- 172.21.15.103:0/4061652323 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f73980a3240 msgr2=0x7f73980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.779+0000 7f73b0949640 1 --2- 172.21.15.103:0/4061652323 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f73980a3240 0x7f73980a5700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f739c009230 tx=0x7f739c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:46:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.779+0000 7f73b0949640 1 -- 172.21.15.103:0/4061652323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73ac102d30 msgr2=0x7f73ac198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.779+0000 7f73b0949640 1 --2- 172.21.15.103:0/4061652323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73ac102d30 0x7f73ac198570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f739400c920 tx=0x7f739400cde0 comp rx=0 tx=0).stop 2024-09-19T06:46:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.779+0000 7f73b0949640 1 -- 172.21.15.103:0/4061652323 shutdown_connections 2024-09-19T06:46:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.779+0000 7f73b0949640 1 --2- 172.21.15.103:0/4061652323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ac1082a0 0x7f73ac19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.779+0000 7f73b0949640 1 --2- 172.21.15.103:0/4061652323 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f73980a3240 0x7f73980a5700 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.779+0000 7f73b0949640 1 --2- 172.21.15.103:0/4061652323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73ac1036e0 0x7f73ac198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.779+0000 7f73b0949640 1 --2- 172.21.15.103:0/4061652323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73ac102d30 0x7f73ac198570 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:39.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.779+0000 7f73b0949640 1 -- 172.21.15.103:0/4061652323 >> 172.21.15.103:0/4061652323 conn(0x7f73ac0fc7d0 msgr2=0x7f73ac105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:39.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.779+0000 7f73b0949640 1 -- 172.21.15.103:0/4061652323 shutdown_connections 2024-09-19T06:46:39.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:39.779+0000 7f73b0949640 1 -- 172.21.15.103:0/4061652323 wait complete. 2024-09-19T06:46:40.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:39 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/4061652323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:46:40.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:39 smithi174 ceph-mon[35637]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:40.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:39 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/4061652323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:46:40.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:39 smithi103 ceph-mon[31158]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:40.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:39 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/4061652323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:46:40.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:39 smithi134 ceph-mon[35356]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:40.488 INFO:tasks.cephadm:Adding mds.a on smithi103 2024-09-19T06:46:40.489 INFO:tasks.cephadm:Adding mds.d on smithi103 2024-09-19T06:46:40.489 INFO:tasks.cephadm:Adding mds.g on smithi103 2024-09-19T06:46:40.489 INFO:tasks.cephadm:Adding mds.j on smithi103 2024-09-19T06:46:40.489 INFO:tasks.cephadm:Adding mds.b on smithi134 2024-09-19T06:46:40.489 INFO:tasks.cephadm:Adding mds.e on smithi134 2024-09-19T06:46:40.489 INFO:tasks.cephadm:Adding mds.h on smithi134 2024-09-19T06:46:40.490 INFO:tasks.cephadm:Adding mds.k on smithi134 2024-09-19T06:46:40.490 INFO:tasks.cephadm:Adding mds.c on smithi174 2024-09-19T06:46:40.490 INFO:tasks.cephadm:Adding mds.f on smithi174 2024-09-19T06:46:40.490 INFO:tasks.cephadm:Adding mds.i on smithi174 2024-09-19T06:46:40.490 INFO:tasks.cephadm:Adding mds.l on smithi174 2024-09-19T06:46:40.490 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l' 2024-09-19T06:46:40.748 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:46:41.238 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.230+0000 7fd8fbabd640 1 -- 172.21.15.174:0/186948904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f41037b0 msgr2=0x7fd8f410a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:41.238 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.230+0000 7fd8fbabd640 1 --2- 172.21.15.174:0/186948904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f41037b0 0x7fd8f410a040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009ca0 tx=0x7fd8ec02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:46:41.238 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.230+0000 7fd8fbabd640 1 -- 172.21.15.174:0/186948904 shutdown_connections 2024-09-19T06:46:41.238 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.230+0000 7fd8fbabd640 1 --2- 172.21.15.174:0/186948904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f41037b0 0x7fd8f410a040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:41.238 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.230+0000 7fd8fbabd640 1 --2- 172.21.15.174:0/186948904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8f4102df0 0x7fd8f4103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:41.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.230+0000 7fd8fbabd640 1 --2- 172.21.15.174:0/186948904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8f4101bf0 0x7fd8f4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:41.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.230+0000 7fd8fbabd640 1 -- 172.21.15.174:0/186948904 >> 172.21.15.174:0/186948904 conn(0x7fd8f40fd3c0 msgr2=0x7fd8f40ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:41.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.230+0000 7fd8fbabd640 1 -- 172.21.15.174:0/186948904 shutdown_connections 2024-09-19T06:46:41.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.230+0000 7fd8fbabd640 1 -- 172.21.15.174:0/186948904 wait complete. 2024-09-19T06:46:41.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.230+0000 7fd8fbabd640 1 Processor -- start 2024-09-19T06:46:41.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8fbabd640 1 -- start start 2024-09-19T06:46:41.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8fbabd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8f4101bf0 0x7fd8f4075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:41.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8fbabd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f4102df0 0x7fd8f40745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:41.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8fbabd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8f41037b0 0x7fd8f4074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:41.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8fbabd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f410d870 con 0x7fd8f41037b0 2024-09-19T06:46:41.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8fbabd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f410d6f0 con 0x7fd8f4101bf0 2024-09-19T06:46:41.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8fbabd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f410d9f0 con 0x7fd8f4102df0 2024-09-19T06:46:41.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8f9058640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f4102df0 0x7fd8f40745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:41.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8f9058640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f4102df0 0x7fd8f40745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:44326/0 (socket says 172.21.15.174:44326) 2024-09-19T06:46:41.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8f9058640 1 -- 172.21.15.174:0/1678847288 learned_addr learned my addr 172.21.15.174:0/1678847288 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T06:46:41.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8fa05a640 1 --2- 172.21.15.174:0/1678847288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8f41037b0 0x7fd8f4074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:41.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8f9859640 1 --2- 172.21.15.174:0/1678847288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8f4101bf0 0x7fd8f4075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:41.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8f9058640 1 -- 172.21.15.174:0/1678847288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8f4101bf0 msgr2=0x7fd8f4075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:41.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8f9058640 1 --2- 172.21.15.174:0/1678847288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8f4101bf0 0x7fd8f4075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:41.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8f9058640 1 -- 172.21.15.174:0/1678847288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8f41037b0 msgr2=0x7fd8f4074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:41.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8f9058640 1 --2- 172.21.15.174:0/1678847288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8f41037b0 0x7fd8f4074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:41.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8f9058640 1 -- 172.21.15.174:0/1678847288 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f40753e0 con 0x7fd8f4102df0 2024-09-19T06:46:41.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.231+0000 7fd8f9058640 1 --2- 172.21.15.174:0/1678847288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f4102df0 0x7fd8f40745e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000c890 tx=0x7fd8f000cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:41.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.232+0000 7fd8eaffd640 1 -- 172.21.15.174:0/1678847288 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0004220 con 0x7fd8f4102df0 2024-09-19T06:46:41.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.232+0000 7fd8eaffd640 1 -- 172.21.15.174:0/1678847288 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8f00043c0 con 0x7fd8f4102df0 2024-09-19T06:46:41.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.232+0000 7fd8eaffd640 1 -- 172.21.15.174:0/1678847288 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00049e0 con 0x7fd8f4102df0 2024-09-19T06:46:41.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.232+0000 7fd8fbabd640 1 -- 172.21.15.174:0/1678847288 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f41a1ff0 con 0x7fd8f4102df0 2024-09-19T06:46:41.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.232+0000 7fd8fbabd640 1 -- 172.21.15.174:0/1678847288 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f41a2450 con 0x7fd8f4102df0 2024-09-19T06:46:41.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.233+0000 7fd8fbabd640 1 -- 172.21.15.174:0/1678847288 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f4108fc0 con 0x7fd8f4102df0 2024-09-19T06:46:41.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.234+0000 7fd8eaffd640 1 -- 172.21.15.174:0/1678847288 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fd8f00079b0 con 0x7fd8f4102df0 2024-09-19T06:46:41.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.235+0000 7fd8eaffd640 1 --2- 172.21.15.174:0/1678847288 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fd8e00a3240 0x7fd8e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:41.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.235+0000 7fd8eaffd640 1 -- 172.21.15.174:0/1678847288 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fd8f00cf420 con 0x7fd8f4102df0 2024-09-19T06:46:41.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.235+0000 7fd8f9859640 1 --2- 172.21.15.174:0/1678847288 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fd8e00a3240 0x7fd8e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:41.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.236+0000 7fd8f9859640 1 --2- 172.21.15.174:0/1678847288 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fd8e00a3240 0x7fd8e00a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4005fd0 tx=0x7fd8e400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:41.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.239+0000 7fd8eaffd640 1 -- 172.21.15.174:0/1678847288 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd8f0014030 con 0x7fd8f4102df0 2024-09-19T06:46:41.419 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.417+0000 7fd8fbabd640 1 -- 172.21.15.174:0/1678847288 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}) -- 0x7fd8f4062730 con 0x7fd8e00a3240 2024-09-19T06:46:41.426 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.424+0000 7fd8eaffd640 1 -- 172.21.15.174:0/1678847288 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fd8f4062730 con 0x7fd8e00a3240 2024-09-19T06:46:41.426 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mds.all update... 2024-09-19T06:46:41.428 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.426+0000 7fd8fbabd640 1 -- 172.21.15.174:0/1678847288 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fd8e00a3240 msgr2=0x7fd8e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:41.428 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.426+0000 7fd8fbabd640 1 --2- 172.21.15.174:0/1678847288 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fd8e00a3240 0x7fd8e00a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4005fd0 tx=0x7fd8e400b390 comp rx=0 tx=0).stop 2024-09-19T06:46:41.428 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.426+0000 7fd8fbabd640 1 -- 172.21.15.174:0/1678847288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f4102df0 msgr2=0x7fd8f40745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:41.428 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.426+0000 7fd8fbabd640 1 --2- 172.21.15.174:0/1678847288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f4102df0 0x7fd8f40745e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000c890 tx=0x7fd8f000cd50 comp rx=0 tx=0).stop 2024-09-19T06:46:41.428 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.426+0000 7fd8fbabd640 1 -- 172.21.15.174:0/1678847288 shutdown_connections 2024-09-19T06:46:41.429 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.426+0000 7fd8fbabd640 1 --2- 172.21.15.174:0/1678847288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8f41037b0 0x7fd8f4074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:41.429 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.426+0000 7fd8fbabd640 1 --2- 172.21.15.174:0/1678847288 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fd8e00a3240 0x7fd8e00a5700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:41.429 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.426+0000 7fd8fbabd640 1 --2- 172.21.15.174:0/1678847288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f4102df0 0x7fd8f40745e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:41.429 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.426+0000 7fd8fbabd640 1 --2- 172.21.15.174:0/1678847288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8f4101bf0 0x7fd8f4075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:41.429 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.426+0000 7fd8fbabd640 1 -- 172.21.15.174:0/1678847288 >> 172.21.15.174:0/1678847288 conn(0x7fd8f40fd3c0 msgr2=0x7fd8f4106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:41.429 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.426+0000 7fd8fbabd640 1 -- 172.21.15.174:0/1678847288 shutdown_connections 2024-09-19T06:46:41.429 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T06:46:41.426+0000 7fd8fbabd640 1 -- 172.21.15.174:0/1678847288 wait complete. 2024-09-19T06:46:42.102 DEBUG:teuthology.orchestra.run.smithi103:mds.a> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T06:46:42.104 DEBUG:teuthology.orchestra.run.smithi103:mds.d> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T06:46:42.147 DEBUG:teuthology.orchestra.run.smithi103:mds.g> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T06:46:42.189 DEBUG:teuthology.orchestra.run.smithi103:mds.j> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T06:46:42.232 DEBUG:teuthology.orchestra.run.smithi134:mds.b> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T06:46:42.235 DEBUG:teuthology.orchestra.run.smithi134:mds.e> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T06:46:42.279 DEBUG:teuthology.orchestra.run.smithi134:mds.h> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T06:46:42.323 DEBUG:teuthology.orchestra.run.smithi134:mds.k> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T06:46:42.366 DEBUG:teuthology.orchestra.run.smithi174:mds.c> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T06:46:42.368 DEBUG:teuthology.orchestra.run.smithi174:mds.f> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T06:46:42.411 DEBUG:teuthology.orchestra.run.smithi174:mds.i> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T06:46:42.454 DEBUG:teuthology.orchestra.run.smithi174:mds.l> sudo journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T06:46:42.497 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T06:46:42.497 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:46:42.553 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:42 smithi174 ceph-mon[35637]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:46:42.553 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:42 smithi174 ceph-mon[35637]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-19T06:46:42.553 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:42 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:42.553 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:42 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:42.553 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:42 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:42.553 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:42 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:42.553 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:42 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:42.553 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:42 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:42.553 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:42 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:42.553 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:42 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:42.553 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:42 smithi174 ceph-mon[35637]: Deploying daemon mds.l on smithi174 2024-09-19T06:46:42.553 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:42 smithi174 ceph-mon[35637]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:42.681 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:42 smithi103 ceph-mon[31158]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:46:42.681 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:42 smithi103 ceph-mon[31158]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-19T06:46:42.681 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:42 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:42.681 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:42 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:42.681 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:42 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:42.682 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:42 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:42.682 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:42 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:42.682 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:42 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:42.682 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:42 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:42.682 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:42 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:42.682 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:42 smithi103 ceph-mon[31158]: Deploying daemon mds.l on smithi174 2024-09-19T06:46:42.682 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:42 smithi103 ceph-mon[31158]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:42.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:42 smithi134 ceph-mon[35356]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:46:42.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:42 smithi134 ceph-mon[35356]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-19T06:46:42.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:42 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:42.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:42 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:42.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:42 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:42.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:42 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:42.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:42 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:42.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:42 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:42.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:42 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:42.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:42 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:42.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:42 smithi134 ceph-mon[35356]: Deploying daemon mds.l on smithi174 2024-09-19T06:46:42.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:42 smithi134 ceph-mon[35356]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:42.757 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:43.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.227+0000 7fcdc7373640 1 -- 172.21.15.103:0/350534990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdc0100b30 msgr2=0x7fcdc0100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:43.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.227+0000 7fcdc7373640 1 --2- 172.21.15.103:0/350534990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdc0100b30 0x7fcdc0100f10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009900 tx=0x7fcdbc02a470 comp rx=0 tx=0).stop 2024-09-19T06:46:43.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.227+0000 7fcdc7373640 1 -- 172.21.15.103:0/350534990 shutdown_connections 2024-09-19T06:46:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.227+0000 7fcdc7373640 1 --2- 172.21.15.103:0/350534990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc0105f40 0x7fcdc01063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.227+0000 7fcdc7373640 1 --2- 172.21.15.103:0/350534990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdc0101450 0x7fcdc0105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.227+0000 7fcdc7373640 1 --2- 172.21.15.103:0/350534990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdc0100b30 0x7fcdc0100f10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.227+0000 7fcdc7373640 1 -- 172.21.15.103:0/350534990 >> 172.21.15.103:0/350534990 conn(0x7fcdc00fc7a0 msgr2=0x7fcdc00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.227+0000 7fcdc7373640 1 -- 172.21.15.103:0/350534990 shutdown_connections 2024-09-19T06:46:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.228+0000 7fcdc7373640 1 -- 172.21.15.103:0/350534990 wait complete. 2024-09-19T06:46:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.228+0000 7fcdc7373640 1 Processor -- start 2024-09-19T06:46:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.228+0000 7fcdc7373640 1 -- start start 2024-09-19T06:46:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.228+0000 7fcdc7373640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdc0100b30 0x7fcdc01a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.228+0000 7fcdc7373640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdc0101450 0x7fcdc01a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.228+0000 7fcdc7373640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc0105f40 0x7fcdc019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.228+0000 7fcdc7373640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc01185d0 con 0x7fcdc0100b30 2024-09-19T06:46:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.228+0000 7fcdc7373640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc0118450 con 0x7fcdc0101450 2024-09-19T06:46:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.228+0000 7fcdc7373640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc0118750 con 0x7fcdc0105f40 2024-09-19T06:46:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.228+0000 7fcdc510f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdc0100b30 0x7fcdc01a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.228+0000 7fcdc510f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdc0100b30 0x7fcdc01a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54084/0 (socket says 172.21.15.103:54084) 2024-09-19T06:46:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.228+0000 7fcdc510f640 1 -- 172.21.15.103:0/1177620131 learned_addr learned my addr 172.21.15.103:0/1177620131 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:46:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.229+0000 7fcdc490e640 1 --2- 172.21.15.103:0/1177620131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdc0101450 0x7fcdc01a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.229+0000 7fcdc5910640 1 --2- 172.21.15.103:0/1177620131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc0105f40 0x7fcdc019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.229+0000 7fcdc510f640 1 -- 172.21.15.103:0/1177620131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc0105f40 msgr2=0x7fcdc019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.229+0000 7fcdc510f640 1 --2- 172.21.15.103:0/1177620131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc0105f40 0x7fcdc019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.229+0000 7fcdc510f640 1 -- 172.21.15.103:0/1177620131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdc0101450 msgr2=0x7fcdc01a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.229+0000 7fcdc510f640 1 --2- 172.21.15.103:0/1177620131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdc0101450 0x7fcdc01a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.229+0000 7fcdc510f640 1 -- 172.21.15.103:0/1177620131 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc019efb0 con 0x7fcdc0100b30 2024-09-19T06:46:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.229+0000 7fcdc5910640 1 --2- 172.21.15.103:0/1177620131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc0105f40 0x7fcdc019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.229+0000 7fcdc510f640 1 --2- 172.21.15.103:0/1177620131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdc0100b30 0x7fcdc01a46c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc005ab0 tx=0x7fcdbc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.229+0000 7fcdb67fc640 1 -- 172.21.15.103:0/1177620131 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc03f030 con 0x7fcdc0100b30 2024-09-19T06:46:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.229+0000 7fcdb67fc640 1 -- 172.21.15.103:0/1177620131 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdbc003ce0 con 0x7fcdc0100b30 2024-09-19T06:46:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.229+0000 7fcdb67fc640 1 -- 172.21.15.103:0/1177620131 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc035070 con 0x7fcdc0100b30 2024-09-19T06:46:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.229+0000 7fcdc7373640 1 -- 172.21.15.103:0/1177620131 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc019f240 con 0x7fcdc0100b30 2024-09-19T06:46:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.229+0000 7fcdc7373640 1 -- 172.21.15.103:0/1177620131 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc01ab590 con 0x7fcdc0100b30 2024-09-19T06:46:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.230+0000 7fcdc7373640 1 -- 172.21.15.103:0/1177620131 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdc0105920 con 0x7fcdc0100b30 2024-09-19T06:46:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.231+0000 7fcdb67fc640 1 -- 172.21.15.103:0/1177620131 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fcdbc042030 con 0x7fcdc0100b30 2024-09-19T06:46:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.232+0000 7fcdb67fc640 1 --2- 172.21.15.103:0/1177620131 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fcdac0a32c0 0x7fcdac0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.232+0000 7fcdc490e640 1 --2- 172.21.15.103:0/1177620131 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fcdac0a32c0 0x7fcdac0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.232+0000 7fcdb67fc640 1 -- 172.21.15.103:0/1177620131 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fcdbc0efd00 con 0x7fcdc0100b30 2024-09-19T06:46:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.232+0000 7fcdc490e640 1 --2- 172.21.15.103:0/1177620131 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fcdac0a32c0 0x7fcdac0a5780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00ffd90 tx=0x7fcdb000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.236+0000 7fcdb67fc640 1 -- 172.21.15.103:0/1177620131 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcdbc0bcec0 con 0x7fcdc0100b30 2024-09-19T06:46:43.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.439+0000 7fcdc7373640 1 -- 172.21.15.103:0/1177620131 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdc01ab760 con 0x7fcdc0100b30 2024-09-19T06:46:43.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.439+0000 7fcdb67fc640 1 -- 172.21.15.103:0/1177620131 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fcdbc0bc7c0 con 0x7fcdc0100b30 2024-09-19T06:46:43.441 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:46:43.441 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":1,"btime":"2024-09-19T06:39:46:840386+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T06:46:43.441 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 1 2024-09-19T06:46:43.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.441+0000 7fcdc7373640 1 -- 172.21.15.103:0/1177620131 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fcdac0a32c0 msgr2=0x7fcdac0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:43.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.441+0000 7fcdc7373640 1 --2- 172.21.15.103:0/1177620131 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fcdac0a32c0 0x7fcdac0a5780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00ffd90 tx=0x7fcdb000a250 comp rx=0 tx=0).stop 2024-09-19T06:46:43.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.441+0000 7fcdc7373640 1 -- 172.21.15.103:0/1177620131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdc0100b30 msgr2=0x7fcdc01a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:43.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.441+0000 7fcdc7373640 1 --2- 172.21.15.103:0/1177620131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdc0100b30 0x7fcdc01a46c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc005ab0 tx=0x7fcdbc0042f0 comp rx=0 tx=0).stop 2024-09-19T06:46:43.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.442+0000 7fcdc7373640 1 -- 172.21.15.103:0/1177620131 shutdown_connections 2024-09-19T06:46:43.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.442+0000 7fcdc7373640 1 --2- 172.21.15.103:0/1177620131 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fcdac0a32c0 0x7fcdac0a5780 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:43.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.442+0000 7fcdc7373640 1 --2- 172.21.15.103:0/1177620131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc0105f40 0x7fcdc019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:43.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.442+0000 7fcdc7373640 1 --2- 172.21.15.103:0/1177620131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdc0101450 0x7fcdc01a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:43.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.442+0000 7fcdc7373640 1 --2- 172.21.15.103:0/1177620131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdc0100b30 0x7fcdc01a46c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:43.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.442+0000 7fcdc7373640 1 -- 172.21.15.103:0/1177620131 >> 172.21.15.103:0/1177620131 conn(0x7fcdc00fc7a0 msgr2=0x7fcdc01038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:43.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.442+0000 7fcdc7373640 1 -- 172.21.15.103:0/1177620131 shutdown_connections 2024-09-19T06:46:43.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:43.442+0000 7fcdc7373640 1 -- 172.21.15.103:0/1177620131 wait complete. 2024-09-19T06:46:43.810 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 06:46:43 smithi174 systemd[1]: Starting Ceph mds.l for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T06:46:44.081 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:43 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1177620131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:44.081 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 06:46:43 smithi174 podman[65433]: 2024-09-19 06:46:43.851706646 +0000 UTC m=+0.015575802 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:46:44.081 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 06:46:43 smithi174 podman[65433]: 2024-09-19 06:46:43.982783962 +0000 UTC m=+0.146653160 container create ad715ddcc610065aea0786c64bec8447f2156b2275b819a930f34d4262267101 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:46:44.128 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:43 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1177620131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:44.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:43 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1177620131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:44.583 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 06:46:44 smithi174 podman[65433]: 2024-09-19 06:46:44.176901731 +0000 UTC m=+0.340770893 container init ad715ddcc610065aea0786c64bec8447f2156b2275b819a930f34d4262267101 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T06:46:44.583 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 06:46:44 smithi174 podman[65433]: 2024-09-19 06:46:44.179870798 +0000 UTC m=+0.343739954 container start ad715ddcc610065aea0786c64bec8447f2156b2275b819a930f34d4262267101 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-l, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T06:46:44.583 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 06:46:44 smithi174 ceph-mds[65453]: -- 172.21.15.174:0/530445759 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5607fca59860 con 0x5607fca37400 2024-09-19T06:46:44.583 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 06:46:44 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-l[65449]: starting mds.l at 2024-09-19T06:46:44.583 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 06:46:44 smithi174 bash[65433]: ad715ddcc610065aea0786c64bec8447f2156b2275b819a930f34d4262267101 2024-09-19T06:46:44.584 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 06:46:44 smithi174 systemd[1]: Started Ceph mds.l for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:46:45.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:44 smithi134 ceph-mon[35356]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:45.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:44 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.174:6832/2319563070,v1:172.21.15.174:6833/2319563070] up:boot 2024-09-19T06:46:45.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:44 smithi134 ceph-mon[35356]: fsmap 1 up:standby 2024-09-19T06:46:45.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:46:45.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:45.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:45.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:45.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:45.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:45.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:44 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:45.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:44 smithi174 ceph-mon[35637]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:45.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:44 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.174:6832/2319563070,v1:172.21.15.174:6833/2319563070] up:boot 2024-09-19T06:46:45.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:44 smithi174 ceph-mon[35637]: fsmap 1 up:standby 2024-09-19T06:46:45.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:46:45.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:45.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:45.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:45.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:45.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:45.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:44 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:45.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:44 smithi103 ceph-mon[31158]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:45.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:44 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.174:6832/2319563070,v1:172.21.15.174:6833/2319563070] up:boot 2024-09-19T06:46:45.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:44 smithi103 ceph-mon[31158]: fsmap 1 up:standby 2024-09-19T06:46:45.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:46:45.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:45.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:45.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:45.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:45.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:45.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:44 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:46.044 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:45 smithi134 ceph-mon[35356]: Deploying daemon mds.k on smithi134 2024-09-19T06:46:46.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:45 smithi174 ceph-mon[35637]: Deploying daemon mds.k on smithi134 2024-09-19T06:46:46.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:45 smithi103 ceph-mon[31158]: Deploying daemon mds.k on smithi134 2024-09-19T06:46:46.156 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:46:46.412 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:46.432 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 06:46:46 smithi134 systemd[1]: Starting Ceph mds.k for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T06:46:46.687 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 06:46:46 smithi134 podman[64678]: 2024-09-19 06:46:46.334210909 +0000 UTC m=+0.016192353 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:46:46.687 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 06:46:46 smithi134 podman[64678]: 2024-09-19 06:46:46.457034789 +0000 UTC m=+0.139016230 container create d2780a11bcfb42f6317855c11fe289b691ea42a0a1f790adc3845bc3fbbe007d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True) 2024-09-19T06:46:46.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.861+0000 7fa96864a640 1 -- 172.21.15.103:0/3189746167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa960101420 msgr2=0x7fa96010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:46.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.861+0000 7fa96864a640 1 --2- 172.21.15.103:0/3189746167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa960101420 0x7fa96010d950 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa958009cd0 tx=0x7fa95802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:46:46.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.861+0000 7fa96864a640 1 -- 172.21.15.103:0/3189746167 shutdown_connections 2024-09-19T06:46:46.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.861+0000 7fa96864a640 1 --2- 172.21.15.103:0/3189746167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa960101420 0x7fa96010d950 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:46.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.861+0000 7fa96864a640 1 --2- 172.21.15.103:0/3189746167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa960100a80 0x7fa960100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:46.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.861+0000 7fa96864a640 1 --2- 172.21.15.103:0/3189746167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa960106a80 0x7fa960106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:46.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.861+0000 7fa96864a640 1 -- 172.21.15.103:0/3189746167 >> 172.21.15.103:0/3189746167 conn(0x7fa9600fc7a0 msgr2=0x7fa9600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:46.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.861+0000 7fa96864a640 1 -- 172.21.15.103:0/3189746167 shutdown_connections 2024-09-19T06:46:46.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.861+0000 7fa96864a640 1 -- 172.21.15.103:0/3189746167 wait complete. 2024-09-19T06:46:46.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa96864a640 1 Processor -- start 2024-09-19T06:46:46.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa96864a640 1 -- start start 2024-09-19T06:46:46.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa96864a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa960100a80 0x7fa9601992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:46.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa96864a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa960101420 0x7fa9601997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:46.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa96864a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa960106a80 0x7fa9601a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:46.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa96864a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa960110e80 con 0x7fa960101420 2024-09-19T06:46:46.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa96864a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa960110d00 con 0x7fa960106a80 2024-09-19T06:46:46.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa96864a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa960111000 con 0x7fa960100a80 2024-09-19T06:46:46.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa965be5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa960101420 0x7fa9601997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:46.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa965be5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa960101420 0x7fa9601997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54110/0 (socket says 172.21.15.103:54110) 2024-09-19T06:46:46.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa965be5640 1 -- 172.21.15.103:0/2082181463 learned_addr learned my addr 172.21.15.103:0/2082181463 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:46:46.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa966be7640 1 --2- 172.21.15.103:0/2082181463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa960106a80 0x7fa9601a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:46.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa965be5640 1 -- 172.21.15.103:0/2082181463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa960100a80 msgr2=0x7fa9601992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:46.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa965be5640 1 --2- 172.21.15.103:0/2082181463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa960100a80 0x7fa9601992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:46.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa965be5640 1 -- 172.21.15.103:0/2082181463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa960106a80 msgr2=0x7fa9601a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:46.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa965be5640 1 --2- 172.21.15.103:0/2082181463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa960106a80 0x7fa9601a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:46.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.862+0000 7fa965be5640 1 -- 172.21.15.103:0/2082181463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9601a4870 con 0x7fa960101420 2024-09-19T06:46:46.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.863+0000 7fa965be5640 1 --2- 172.21.15.103:0/2082181463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa960101420 0x7fa9601997f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa95000d950 tx=0x7fa95000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:46.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.863+0000 7fa9577fe640 1 -- 172.21.15.103:0/2082181463 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950017030 con 0x7fa960101420 2024-09-19T06:46:46.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.863+0000 7fa9577fe640 1 -- 172.21.15.103:0/2082181463 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa950004020 con 0x7fa960101420 2024-09-19T06:46:46.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.863+0000 7fa96864a640 1 -- 172.21.15.103:0/2082181463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9601a4a40 con 0x7fa960101420 2024-09-19T06:46:46.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.863+0000 7fa9577fe640 1 -- 172.21.15.103:0/2082181463 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9500047a0 con 0x7fa960101420 2024-09-19T06:46:46.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.863+0000 7fa96864a640 1 -- 172.21.15.103:0/2082181463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9601a4f50 con 0x7fa960101420 2024-09-19T06:46:46.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.864+0000 7fa96864a640 1 -- 172.21.15.103:0/2082181463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa960100ee0 con 0x7fa960101420 2024-09-19T06:46:46.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.865+0000 7fa9577fe640 1 -- 172.21.15.103:0/2082181463 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa9500041d0 con 0x7fa960101420 2024-09-19T06:46:46.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.865+0000 7fa9577fe640 1 --2- 172.21.15.103:0/2082181463 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa94c0a3240 0x7fa94c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:46.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.866+0000 7fa9663e6640 1 --2- 172.21.15.103:0/2082181463 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa94c0a3240 0x7fa94c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:46.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.866+0000 7fa9577fe640 1 -- 172.21.15.103:0/2082181463 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fa9500cee50 con 0x7fa960101420 2024-09-19T06:46:46.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.866+0000 7fa9663e6640 1 --2- 172.21.15.103:0/2082181463 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa94c0a3240 0x7fa94c0a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009ad0 tx=0x7fa95c009590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:46.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:46.870+0000 7fa9577fe640 1 -- 172.21.15.103:0/2082181463 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa95009c010 con 0x7fa960101420 2024-09-19T06:46:47.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:46 smithi174 ceph-mon[35637]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:47.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:47.086+0000 7fa96864a640 1 -- 172.21.15.103:0/2082181463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa960108ff0 con 0x7fa960101420 2024-09-19T06:46:47.087 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:46 smithi103 ceph-mon[31158]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:47.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:47.087+0000 7fa9577fe640 1 -- 172.21.15.103:0/2082181463 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7fa95009b910 con 0x7fa960101420 2024-09-19T06:46:47.088 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:46:47.088 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":3,"btime":"2024-09-19T06:46:46:818717+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-19T06:46:47.088 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 3 2024-09-19T06:46:47.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:47.089+0000 7fa96864a640 1 -- 172.21.15.103:0/2082181463 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa94c0a3240 msgr2=0x7fa94c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:47.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:47.089+0000 7fa96864a640 1 --2- 172.21.15.103:0/2082181463 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa94c0a3240 0x7fa94c0a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009ad0 tx=0x7fa95c009590 comp rx=0 tx=0).stop 2024-09-19T06:46:47.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:47.089+0000 7fa96864a640 1 -- 172.21.15.103:0/2082181463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa960101420 msgr2=0x7fa9601997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:47.089+0000 7fa96864a640 1 --2- 172.21.15.103:0/2082181463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa960101420 0x7fa9601997f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa95000d950 tx=0x7fa95000de10 comp rx=0 tx=0).stop 2024-09-19T06:46:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:47.089+0000 7fa96864a640 1 -- 172.21.15.103:0/2082181463 shutdown_connections 2024-09-19T06:46:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:47.089+0000 7fa96864a640 1 --2- 172.21.15.103:0/2082181463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa960106a80 0x7fa9601a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:47.089+0000 7fa96864a640 1 --2- 172.21.15.103:0/2082181463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa960101420 0x7fa9601997f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:47.089+0000 7fa96864a640 1 --2- 172.21.15.103:0/2082181463 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fa94c0a3240 0x7fa94c0a5700 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:47.089+0000 7fa96864a640 1 --2- 172.21.15.103:0/2082181463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa960100a80 0x7fa9601992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:47.089+0000 7fa96864a640 1 -- 172.21.15.103:0/2082181463 >> 172.21.15.103:0/2082181463 conn(0x7fa9600fc7a0 msgr2=0x7fa9600fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:47.089+0000 7fa96864a640 1 -- 172.21.15.103:0/2082181463 shutdown_connections 2024-09-19T06:46:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:47.089+0000 7fa96864a640 1 -- 172.21.15.103:0/2082181463 wait complete. 2024-09-19T06:46:47.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:46 smithi134 ceph-mon[35356]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:47.187 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 06:46:46 smithi134 podman[64678]: 2024-09-19 06:46:46.748250788 +0000 UTC m=+0.430232245 container init d2780a11bcfb42f6317855c11fe289b691ea42a0a1f790adc3845bc3fbbe007d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-k, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T06:46:47.188 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 06:46:46 smithi134 podman[64678]: 2024-09-19 06:46:46.751051088 +0000 UTC m=+0.433032527 container start d2780a11bcfb42f6317855c11fe289b691ea42a0a1f790adc3845bc3fbbe007d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-k, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-19T06:46:47.188 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 06:46:46 smithi134 ceph-mds[64697]: -- 172.21.15.134:0/2827141365 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f77f25860 con 0x556f77f02800 2024-09-19T06:46:47.188 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 06:46:46 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-k[64693]: starting mds.k at 2024-09-19T06:46:47.188 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 06:46:46 smithi134 bash[64678]: d2780a11bcfb42f6317855c11fe289b691ea42a0a1f790adc3845bc3fbbe007d 2024-09-19T06:46:47.188 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 06:46:46 smithi134 systemd[1]: Started Ceph mds.k for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:46:48.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:47 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.134:6832/1556743259,v1:172.21.15.134:6833/1556743259] up:boot 2024-09-19T06:46:48.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:47 smithi174 ceph-mon[35637]: fsmap 2 up:standby 2024-09-19T06:46:48.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:47 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:46:48.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:47 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:48.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:47 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:48.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:47 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:48.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:47 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:48.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:47 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:48.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:47 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:48.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:47 smithi174 ceph-mon[35637]: Deploying daemon mds.j on smithi103 2024-09-19T06:46:48.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:47 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2082181463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:48.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:47 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.134:6832/1556743259,v1:172.21.15.134:6833/1556743259] up:boot 2024-09-19T06:46:48.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:47 smithi103 ceph-mon[31158]: fsmap 2 up:standby 2024-09-19T06:46:48.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:47 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:46:48.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:47 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:48.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:47 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:48.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:47 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:48.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:47 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:48.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:47 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:48.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:47 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:48.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:47 smithi103 ceph-mon[31158]: Deploying daemon mds.j on smithi103 2024-09-19T06:46:48.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:47 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2082181463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:48.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:47 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.134:6832/1556743259,v1:172.21.15.134:6833/1556743259] up:boot 2024-09-19T06:46:48.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:47 smithi134 ceph-mon[35356]: fsmap 2 up:standby 2024-09-19T06:46:48.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:47 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:46:48.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:47 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:48.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:47 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:48.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:47 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:48.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:47 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:48.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:47 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:48.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:47 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:48.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:47 smithi134 ceph-mon[35356]: Deploying daemon mds.j on smithi103 2024-09-19T06:46:48.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:47 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2082181463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:49.045 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:48 smithi103 ceph-mon[31158]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:49.166 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:48 smithi134 ceph-mon[35356]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:49.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:48 smithi174 ceph-mon[35637]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:49.638 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 06:46:49 smithi103 systemd[1]: Starting Ceph mds.j for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T06:46:49.799 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:46:49.985 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 06:46:49 smithi103 podman[70885]: 2024-09-19 06:46:49.730286105 +0000 UTC m=+0.108033150 container create c9305037c62eaa32e907dec30eb55e69431438974a8494104fb9fe535eb04401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T06:46:49.986 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 06:46:49 smithi103 podman[70885]: 2024-09-19 06:46:49.638217088 +0000 UTC m=+0.015964130 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:46:49.986 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 06:46:49 smithi103 podman[70885]: 2024-09-19 06:46:49.958245864 +0000 UTC m=+0.335992914 container init c9305037c62eaa32e907dec30eb55e69431438974a8494104fb9fe535eb04401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T06:46:49.986 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 06:46:49 smithi103 podman[70885]: 2024-09-19 06:46:49.961221316 +0000 UTC m=+0.338968359 container start c9305037c62eaa32e907dec30eb55e69431438974a8494104fb9fe535eb04401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:46:50.140 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:50.262 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 06:46:49 smithi103 ceph-mds[70922]: -- 172.21.15.103:0/8212959 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ec3f57860 con 0x561ec3f35000 2024-09-19T06:46:50.262 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 06:46:49 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-j[70918]: starting mds.j at 2024-09-19T06:46:50.263 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 06:46:50 smithi103 bash[70885]: c9305037c62eaa32e907dec30eb55e69431438974a8494104fb9fe535eb04401 2024-09-19T06:46:50.263 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 06:46:50 smithi103 systemd[1]: Started Ceph mds.j for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:46:50.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.591+0000 7f6bace6b640 1 -- 172.21.15.103:0/2620420328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8074eb0 msgr2=0x7f6ba8112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.591+0000 7f6bace6b640 1 --2- 172.21.15.103:0/2620420328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8074eb0 0x7f6ba8112650 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009900 tx=0x7f6b9002a480 comp rx=0 tx=0).stop 2024-09-19T06:46:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.591+0000 7f6bace6b640 1 -- 172.21.15.103:0/2620420328 shutdown_connections 2024-09-19T06:46:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.591+0000 7f6bace6b640 1 --2- 172.21.15.103:0/2620420328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8112b90 0x7f6ba8114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.591+0000 7f6bace6b640 1 --2- 172.21.15.103:0/2620420328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8074eb0 0x7f6ba8112650 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.591+0000 7f6bace6b640 1 --2- 172.21.15.103:0/2620420328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba8074590 0x7f6ba8074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.591+0000 7f6bace6b640 1 -- 172.21.15.103:0/2620420328 >> 172.21.15.103:0/2620420328 conn(0x7f6ba80fd8c0 msgr2=0x7f6ba80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.591+0000 7f6bace6b640 1 -- 172.21.15.103:0/2620420328 shutdown_connections 2024-09-19T06:46:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.591+0000 7f6bace6b640 1 -- 172.21.15.103:0/2620420328 wait complete. 2024-09-19T06:46:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.592+0000 7f6bace6b640 1 Processor -- start 2024-09-19T06:46:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.592+0000 7f6bace6b640 1 -- start start 2024-09-19T06:46:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.592+0000 7f6bace6b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba8074590 0x7f6ba81a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.592+0000 7f6bace6b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8074eb0 0x7f6ba81a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:50.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.592+0000 7f6bace6b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8112b90 0x7f6ba819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:50.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.592+0000 7f6bace6b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8118650 con 0x7f6ba8112b90 2024-09-19T06:46:50.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.592+0000 7f6bace6b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81184d0 con 0x7f6ba8074590 2024-09-19T06:46:50.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.592+0000 7f6bace6b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81187d0 con 0x7f6ba8074eb0 2024-09-19T06:46:50.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.592+0000 7f6ba6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8112b90 0x7f6ba819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:50.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.592+0000 7f6ba6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8112b90 0x7f6ba819ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41106/0 (socket says 172.21.15.103:41106) 2024-09-19T06:46:50.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.592+0000 7f6ba6d9d640 1 -- 172.21.15.103:0/2009223055 learned_addr learned my addr 172.21.15.103:0/2009223055 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:46:50.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.593+0000 7f6ba659c640 1 --2- 172.21.15.103:0/2009223055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba8074590 0x7f6ba81a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:50.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.593+0000 7f6b9ffff640 1 --2- 172.21.15.103:0/2009223055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8074eb0 0x7f6ba81a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:50.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.593+0000 7f6ba6d9d640 1 -- 172.21.15.103:0/2009223055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8074eb0 msgr2=0x7f6ba81a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:50.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.593+0000 7f6ba6d9d640 1 --2- 172.21.15.103:0/2009223055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8074eb0 0x7f6ba81a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:50.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.593+0000 7f6ba6d9d640 1 -- 172.21.15.103:0/2009223055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba8074590 msgr2=0x7f6ba81a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:50.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.593+0000 7f6ba6d9d640 1 --2- 172.21.15.103:0/2009223055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba8074590 0x7f6ba81a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:50.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.593+0000 7f6ba6d9d640 1 -- 172.21.15.103:0/2009223055 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba819f180 con 0x7f6ba8112b90 2024-09-19T06:46:50.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.593+0000 7f6ba659c640 1 --2- 172.21.15.103:0/2009223055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba8074590 0x7f6ba81a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:50.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.593+0000 7f6ba6d9d640 1 --2- 172.21.15.103:0/2009223055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8112b90 0x7f6ba819ea50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000b9e0 tx=0x7f6ba000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:50.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.593+0000 7f6b9f7fe640 1 -- 172.21.15.103:0/2009223055 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0019070 con 0x7f6ba8112b90 2024-09-19T06:46:50.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.593+0000 7f6b9f7fe640 1 -- 172.21.15.103:0/2009223055 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ba0014440 con 0x7f6ba8112b90 2024-09-19T06:46:50.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.593+0000 7f6bace6b640 1 -- 172.21.15.103:0/2009223055 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba819f470 con 0x7f6ba8112b90 2024-09-19T06:46:50.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.593+0000 7f6b9f7fe640 1 -- 172.21.15.103:0/2009223055 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba00125c0 con 0x7f6ba8112b90 2024-09-19T06:46:50.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.593+0000 7f6bace6b640 1 -- 172.21.15.103:0/2009223055 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba81ab760 con 0x7f6ba8112b90 2024-09-19T06:46:50.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.594+0000 7f6bace6b640 1 -- 172.21.15.103:0/2009223055 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba80749f0 con 0x7f6ba8112b90 2024-09-19T06:46:50.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.595+0000 7f6b9f7fe640 1 -- 172.21.15.103:0/2009223055 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f6ba001c070 con 0x7f6ba8112b90 2024-09-19T06:46:50.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.596+0000 7f6b9f7fe640 1 --2- 172.21.15.103:0/2009223055 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6b940a3240 0x7f6b940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:50.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.596+0000 7f6ba659c640 1 --2- 172.21.15.103:0/2009223055 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6b940a3240 0x7f6b940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:50.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.596+0000 7f6b9f7fe640 1 -- 172.21.15.103:0/2009223055 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f6ba00cea80 con 0x7f6ba8112b90 2024-09-19T06:46:50.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.597+0000 7f6ba659c640 1 --2- 172.21.15.103:0/2009223055 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6b940a3240 0x7f6b940a5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6ba819fb90 tx=0x7f6b98008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:50.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.600+0000 7f6b9f7fe640 1 -- 172.21.15.103:0/2009223055 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6ba009bc40 con 0x7f6ba8112b90 2024-09-19T06:46:50.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.814+0000 7f6bace6b640 1 -- 172.21.15.103:0/2009223055 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ba819ff80 con 0x7f6ba8112b90 2024-09-19T06:46:50.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.814+0000 7f6b9f7fe640 1 -- 172.21.15.103:0/2009223055 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7f6ba819ff80 con 0x7f6ba8112b90 2024-09-19T06:46:50.816 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:46:50.816 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":3,"btime":"2024-09-19T06:46:46:818717+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-19T06:46:50.817 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 3 2024-09-19T06:46:50.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.816+0000 7f6bace6b640 1 -- 172.21.15.103:0/2009223055 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6b940a3240 msgr2=0x7f6b940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:50.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.816+0000 7f6bace6b640 1 --2- 172.21.15.103:0/2009223055 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6b940a3240 0x7f6b940a5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6ba819fb90 tx=0x7f6b98008040 comp rx=0 tx=0).stop 2024-09-19T06:46:50.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.816+0000 7f6bace6b640 1 -- 172.21.15.103:0/2009223055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8112b90 msgr2=0x7f6ba819ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:50.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.816+0000 7f6bace6b640 1 --2- 172.21.15.103:0/2009223055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8112b90 0x7f6ba819ea50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000b9e0 tx=0x7f6ba000bea0 comp rx=0 tx=0).stop 2024-09-19T06:46:50.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.816+0000 7f6bace6b640 1 -- 172.21.15.103:0/2009223055 shutdown_connections 2024-09-19T06:46:50.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.816+0000 7f6bace6b640 1 --2- 172.21.15.103:0/2009223055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8112b90 0x7f6ba819ea50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:50.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.816+0000 7f6bace6b640 1 --2- 172.21.15.103:0/2009223055 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f6b940a3240 0x7f6b940a5700 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:50.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.816+0000 7f6bace6b640 1 --2- 172.21.15.103:0/2009223055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8074eb0 0x7f6ba81a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:50.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.816+0000 7f6bace6b640 1 --2- 172.21.15.103:0/2009223055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba8074590 0x7f6ba81a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:50.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.816+0000 7f6bace6b640 1 -- 172.21.15.103:0/2009223055 >> 172.21.15.103:0/2009223055 conn(0x7f6ba80fd8c0 msgr2=0x7f6ba80ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:50.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.816+0000 7f6bace6b640 1 -- 172.21.15.103:0/2009223055 shutdown_connections 2024-09-19T06:46:50.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:50.816+0000 7f6bace6b640 1 -- 172.21.15.103:0/2009223055 wait complete. 2024-09-19T06:46:51.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:50 smithi103 ceph-mon[31158]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:51.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:51.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:51.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:51.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:51.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:51.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:50 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:51.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:50 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2009223055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:51.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:50 smithi134 ceph-mon[35356]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:51.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:50 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:51.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:50 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2009223055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:51.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:50 smithi174 ceph-mon[35637]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:51.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:51.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:51.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:51.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:51.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:51.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:50 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:51.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:50 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2009223055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:52.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:51 smithi134 ceph-mon[35356]: Deploying daemon mds.d on smithi103 2024-09-19T06:46:52.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:51 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.103:6834/4290108970,v1:172.21.15.103:6835/4290108970] up:boot 2024-09-19T06:46:52.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:51 smithi134 ceph-mon[35356]: fsmap 3 up:standby 2024-09-19T06:46:52.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:51 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:46:52.291 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:51 smithi103 ceph-mon[31158]: Deploying daemon mds.d on smithi103 2024-09-19T06:46:52.291 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:51 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.103:6834/4290108970,v1:172.21.15.103:6835/4290108970] up:boot 2024-09-19T06:46:52.291 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:51 smithi103 ceph-mon[31158]: fsmap 3 up:standby 2024-09-19T06:46:52.292 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:51 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:46:52.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:51 smithi174 ceph-mon[35637]: Deploying daemon mds.d on smithi103 2024-09-19T06:46:52.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:51 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.103:6834/4290108970,v1:172.21.15.103:6835/4290108970] up:boot 2024-09-19T06:46:52.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:51 smithi174 ceph-mon[35637]: fsmap 3 up:standby 2024-09-19T06:46:52.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:51 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:46:53.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:52 smithi103 ceph-mon[31158]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:53.138 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 06:46:53 smithi103 systemd[1]: Starting Ceph mds.d for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T06:46:53.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:52 smithi134 ceph-mon[35356]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:53.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:52 smithi174 ceph-mon[35637]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:53.606 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 06:46:53 smithi103 podman[71301]: 2024-09-19 06:46:53.19684053 +0000 UTC m=+0.016117963 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:46:53.606 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 06:46:53 smithi103 podman[71301]: 2024-09-19 06:46:53.294258678 +0000 UTC m=+0.113536100 container create 940e4279d328c783d88d3829bc064490103ae269ce2eeae0011632537bc6051d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-d, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T06:46:53.606 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 06:46:53 smithi103 podman[71301]: 2024-09-19 06:46:53.513570059 +0000 UTC m=+0.332847480 container init 940e4279d328c783d88d3829bc064490103ae269ce2eeae0011632537bc6051d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-d, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T06:46:53.606 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 06:46:53 smithi103 podman[71301]: 2024-09-19 06:46:53.516522092 +0000 UTC m=+0.335799516 container start 940e4279d328c783d88d3829bc064490103ae269ce2eeae0011632537bc6051d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-19T06:46:53.606 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 06:46:53 smithi103 ceph-mds[71320]: -- 172.21.15.103:0/279184697 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562976859860 con 0x562976837000 2024-09-19T06:46:53.606 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 06:46:53 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-d[71316]: starting mds.d at 2024-09-19T06:46:53.607 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 06:46:53 smithi103 bash[71301]: 940e4279d328c783d88d3829bc064490103ae269ce2eeae0011632537bc6051d 2024-09-19T06:46:53.648 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:46:53.887 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 06:46:53 smithi103 systemd[1]: Started Ceph mds.d for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:46:53.900 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:54.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.396+0000 7efc9eaad640 1 -- 172.21.15.103:0/2455251915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc98100e20 msgr2=0x7efc98101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:54.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.396+0000 7efc9eaad640 1 --2- 172.21.15.103:0/2455251915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc98100e20 0x7efc98101200 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efc90009900 tx=0x7efc9002a480 comp rx=0 tx=0).stop 2024-09-19T06:46:54.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.396+0000 7efc9eaad640 1 -- 172.21.15.103:0/2455251915 shutdown_connections 2024-09-19T06:46:54.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.396+0000 7efc9eaad640 1 --2- 172.21.15.103:0/2455251915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc98102ef0 0x7efc9810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:54.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.396+0000 7efc9eaad640 1 --2- 172.21.15.103:0/2455251915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc98102570 0x7efc981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:54.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.396+0000 7efc9eaad640 1 --2- 172.21.15.103:0/2455251915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc98100e20 0x7efc98101200 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:54.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.396+0000 7efc9eaad640 1 -- 172.21.15.103:0/2455251915 >> 172.21.15.103:0/2455251915 conn(0x7efc980fc7d0 msgr2=0x7efc980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:54.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.396+0000 7efc9eaad640 1 -- 172.21.15.103:0/2455251915 shutdown_connections 2024-09-19T06:46:54.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.396+0000 7efc9eaad640 1 -- 172.21.15.103:0/2455251915 wait complete. 2024-09-19T06:46:54.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.397+0000 7efc9eaad640 1 Processor -- start 2024-09-19T06:46:54.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.397+0000 7efc9eaad640 1 -- start start 2024-09-19T06:46:54.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.397+0000 7efc9eaad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc98100e20 0x7efc98199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:54.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.397+0000 7efc9eaad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc98102570 0x7efc981998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:54.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.397+0000 7efc9eaad640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc98102ef0 0x7efc981a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:54.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.397+0000 7efc9eaad640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efc98110fa0 con 0x7efc98102570 2024-09-19T06:46:54.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.397+0000 7efc9eaad640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc98110e20 con 0x7efc98100e20 2024-09-19T06:46:54.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.397+0000 7efc9eaad640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc98111120 con 0x7efc98102ef0 2024-09-19T06:46:54.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.397+0000 7efc97fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc98102570 0x7efc981998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:54.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.397+0000 7efc97fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc98102570 0x7efc981998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41132/0 (socket says 172.21.15.103:41132) 2024-09-19T06:46:54.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.398+0000 7efc97fff640 1 -- 172.21.15.103:0/2081915621 learned_addr learned my addr 172.21.15.103:0/2081915621 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:46:54.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.398+0000 7efc9c849640 1 --2- 172.21.15.103:0/2081915621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc98100e20 0x7efc98199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:54.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.398+0000 7efc9d04a640 1 --2- 172.21.15.103:0/2081915621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc98102ef0 0x7efc981a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:54.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.398+0000 7efc97fff640 1 -- 172.21.15.103:0/2081915621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc98102ef0 msgr2=0x7efc981a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:54.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.398+0000 7efc97fff640 1 --2- 172.21.15.103:0/2081915621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc98102ef0 0x7efc981a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:54.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.398+0000 7efc97fff640 1 -- 172.21.15.103:0/2081915621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc98100e20 msgr2=0x7efc98199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:54.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.398+0000 7efc97fff640 1 --2- 172.21.15.103:0/2081915621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc98100e20 0x7efc98199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:54.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.398+0000 7efc97fff640 1 -- 172.21.15.103:0/2081915621 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc981a4920 con 0x7efc98102570 2024-09-19T06:46:54.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.398+0000 7efc9d04a640 1 --2- 172.21.15.103:0/2081915621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc98102ef0 0x7efc981a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:54.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.398+0000 7efc97fff640 1 --2- 172.21.15.103:0/2081915621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc98102570 0x7efc981998a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efc8800ba50 tx=0x7efc8800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:54.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.398+0000 7efc95ffb640 1 -- 172.21.15.103:0/2081915621 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88019070 con 0x7efc98102570 2024-09-19T06:46:54.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.398+0000 7efc95ffb640 1 -- 172.21.15.103:0/2081915621 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc8800cbe0 con 0x7efc98102570 2024-09-19T06:46:54.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.398+0000 7efc9eaad640 1 -- 172.21.15.103:0/2081915621 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc981a4af0 con 0x7efc98102570 2024-09-19T06:46:54.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.398+0000 7efc95ffb640 1 -- 172.21.15.103:0/2081915621 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc880076e0 con 0x7efc98102570 2024-09-19T06:46:54.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.398+0000 7efc9eaad640 1 -- 172.21.15.103:0/2081915621 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc981a5030 con 0x7efc98102570 2024-09-19T06:46:54.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.399+0000 7efc9eaad640 1 -- 172.21.15.103:0/2081915621 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc98101200 con 0x7efc98102570 2024-09-19T06:46:54.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.400+0000 7efc95ffb640 1 -- 172.21.15.103:0/2081915621 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7efc8801c070 con 0x7efc98102570 2024-09-19T06:46:54.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.401+0000 7efc95ffb640 1 --2- 172.21.15.103:0/2081915621 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efc840a32c0 0x7efc840a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:54.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.401+0000 7efc9c849640 1 --2- 172.21.15.103:0/2081915621 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efc840a32c0 0x7efc840a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:54.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.401+0000 7efc95ffb640 1 -- 172.21.15.103:0/2081915621 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7efc880ce9c0 con 0x7efc98102570 2024-09-19T06:46:54.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.402+0000 7efc9c849640 1 --2- 172.21.15.103:0/2081915621 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efc840a32c0 0x7efc840a5780 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efc90009230 tx=0x7efc900058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:54.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.405+0000 7efc95ffb640 1 -- 172.21.15.103:0/2081915621 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efc880d0e20 con 0x7efc98102570 2024-09-19T06:46:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.607+0000 7efc9eaad640 1 -- 172.21.15.103:0/2081915621 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc981a5200 con 0x7efc98102570 2024-09-19T06:46:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.607+0000 7efc95ffb640 1 -- 172.21.15.103:0/2081915621 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3919 (secure 0 0 0) 0x7efc8809b530 con 0x7efc98102570 2024-09-19T06:46:54.609 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:46:54.609 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":5,"btime":"2024-09-19T06:46:53:934607+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-19T06:46:54.610 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 5 2024-09-19T06:46:54.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.609+0000 7efc9eaad640 1 -- 172.21.15.103:0/2081915621 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efc840a32c0 msgr2=0x7efc840a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:54.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.609+0000 7efc9eaad640 1 --2- 172.21.15.103:0/2081915621 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efc840a32c0 0x7efc840a5780 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efc90009230 tx=0x7efc900058d0 comp rx=0 tx=0).stop 2024-09-19T06:46:54.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.609+0000 7efc9eaad640 1 -- 172.21.15.103:0/2081915621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc98102570 msgr2=0x7efc981998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:54.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.609+0000 7efc9eaad640 1 --2- 172.21.15.103:0/2081915621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc98102570 0x7efc981998a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efc8800ba50 tx=0x7efc8800bf10 comp rx=0 tx=0).stop 2024-09-19T06:46:54.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.609+0000 7efc9eaad640 1 -- 172.21.15.103:0/2081915621 shutdown_connections 2024-09-19T06:46:54.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.609+0000 7efc9eaad640 1 --2- 172.21.15.103:0/2081915621 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efc840a32c0 0x7efc840a5780 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:54.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.609+0000 7efc9eaad640 1 --2- 172.21.15.103:0/2081915621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc98102ef0 0x7efc981a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:54.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.609+0000 7efc9eaad640 1 --2- 172.21.15.103:0/2081915621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc98102570 0x7efc981998a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:54.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.609+0000 7efc9eaad640 1 --2- 172.21.15.103:0/2081915621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc98100e20 0x7efc98199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:54.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.609+0000 7efc9eaad640 1 -- 172.21.15.103:0/2081915621 >> 172.21.15.103:0/2081915621 conn(0x7efc980fc7d0 msgr2=0x7efc980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:54.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.609+0000 7efc9eaad640 1 -- 172.21.15.103:0/2081915621 shutdown_connections 2024-09-19T06:46:54.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:54.609+0000 7efc9eaad640 1 -- 172.21.15.103:0/2081915621 wait complete. 2024-09-19T06:46:54.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:54.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:54.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:54.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:54.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:54.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:54.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:54 smithi103 ceph-mon[31158]: Deploying daemon mds.e on smithi134 2024-09-19T06:46:54.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:54 smithi103 ceph-mon[31158]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:54.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:54 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.103:6836/323370484,v1:172.21.15.103:6837/323370484] up:boot 2024-09-19T06:46:54.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:54 smithi103 ceph-mon[31158]: fsmap 4 up:standby 2024-09-19T06:46:54.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:54 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:46:54.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:54 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2081915621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:54.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:54.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:54.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:54.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:54.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:54.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:54.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:54 smithi134 ceph-mon[35356]: Deploying daemon mds.e on smithi134 2024-09-19T06:46:54.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:54 smithi134 ceph-mon[35356]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:54.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:54 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.103:6836/323370484,v1:172.21.15.103:6837/323370484] up:boot 2024-09-19T06:46:54.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:54 smithi134 ceph-mon[35356]: fsmap 4 up:standby 2024-09-19T06:46:54.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:54 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:46:54.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:54 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2081915621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:55.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:55.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:55.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:55.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:55.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:55.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:55.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:54 smithi174 ceph-mon[35637]: Deploying daemon mds.e on smithi134 2024-09-19T06:46:55.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:54 smithi174 ceph-mon[35637]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:55.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:54 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.103:6836/323370484,v1:172.21.15.103:6837/323370484] up:boot 2024-09-19T06:46:55.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:54 smithi174 ceph-mon[35637]: fsmap 4 up:standby 2024-09-19T06:46:55.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:54 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:46:55.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:54 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2081915621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:55.937 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 06:46:55 smithi134 systemd[1]: Starting Ceph mds.e for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T06:46:56.210 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 06:46:55 smithi134 podman[64994]: 2024-09-19 06:46:55.850665833 +0000 UTC m=+0.015750495 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:46:56.211 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 06:46:55 smithi134 podman[64994]: 2024-09-19 06:46:55.973523742 +0000 UTC m=+0.138608393 container create 3d782529ff85ae7e8d3ea066b9cfd1a08aba4abe234d4bcb4abd99d627fa71d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T06:46:56.211 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 06:46:56 smithi134 podman[64994]: 2024-09-19 06:46:56.184325389 +0000 UTC m=+0.349410040 container init 3d782529ff85ae7e8d3ea066b9cfd1a08aba4abe234d4bcb4abd99d627fa71d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-09-19T06:46:56.211 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 06:46:56 smithi134 podman[64994]: 2024-09-19 06:46:56.187016005 +0000 UTC m=+0.352100656 container start 3d782529ff85ae7e8d3ea066b9cfd1a08aba4abe234d4bcb4abd99d627fa71d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-e, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T06:46:56.687 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 06:46:56 smithi134 ceph-mds[65013]: -- 172.21.15.134:0/2989114743 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c5b2d91860 con 0x55c5b2d6f000 2024-09-19T06:46:56.687 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 06:46:56 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-e[65009]: starting mds.e at 2024-09-19T06:46:56.687 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 06:46:56 smithi134 bash[64994]: 3d782529ff85ae7e8d3ea066b9cfd1a08aba4abe234d4bcb4abd99d627fa71d5 2024-09-19T06:46:56.687 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 06:46:56 smithi134 systemd[1]: Started Ceph mds.e for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:46:57.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:56 smithi174 ceph-mon[35637]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:57.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:56 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:57.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:56 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:57.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:56 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:57.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:56 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:57.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:56 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:57.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:56 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:56 smithi103 ceph-mon[31158]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:57.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:56 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:56 smithi134 ceph-mon[35356]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:56 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:56 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:56 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:56 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:56 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:57.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:56 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:57.414 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:46:57.671 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:58.026 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:57 smithi174 ceph-mon[35637]: Deploying daemon mds.f on smithi174 2024-09-19T06:46:58.026 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:57 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] up:boot 2024-09-19T06:46:58.026 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:57 smithi174 ceph-mon[35637]: fsmap 5 up:standby 2024-09-19T06:46:58.026 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:57 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:46:58.108 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:57 smithi103 ceph-mon[31158]: Deploying daemon mds.f on smithi174 2024-09-19T06:46:58.108 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:57 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] up:boot 2024-09-19T06:46:58.108 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:57 smithi103 ceph-mon[31158]: fsmap 5 up:standby 2024-09-19T06:46:58.108 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:57 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:46:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.106+0000 7f6301636640 1 -- 172.21.15.103:0/1576437350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62fc100a80 msgr2=0x7f62fc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.106+0000 7f6301636640 1 --2- 172.21.15.103:0/1576437350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62fc100a80 0x7f62fc100ee0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009930 tx=0x7f62e402a420 comp rx=0 tx=0).stop 2024-09-19T06:46:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.106+0000 7f6301636640 1 -- 172.21.15.103:0/1576437350 shutdown_connections 2024-09-19T06:46:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.106+0000 7f6301636640 1 --2- 172.21.15.103:0/1576437350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62fc101420 0x7f62fc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.106+0000 7f6301636640 1 --2- 172.21.15.103:0/1576437350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62fc100a80 0x7f62fc100ee0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.106+0000 7f6301636640 1 --2- 172.21.15.103:0/1576437350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62fc106a80 0x7f62fc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.106+0000 7f6301636640 1 -- 172.21.15.103:0/1576437350 >> 172.21.15.103:0/1576437350 conn(0x7f62fc0fc7a0 msgr2=0x7f62fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.106+0000 7f6301636640 1 -- 172.21.15.103:0/1576437350 shutdown_connections 2024-09-19T06:46:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.106+0000 7f6301636640 1 -- 172.21.15.103:0/1576437350 wait complete. 2024-09-19T06:46:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.107+0000 7f6301636640 1 Processor -- start 2024-09-19T06:46:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.107+0000 7f6301636640 1 -- start start 2024-09-19T06:46:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.107+0000 7f6301636640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62fc100a80 0x7f62fc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.107+0000 7f6301636640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62fc101420 0x7f62fc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.107+0000 7f6301636640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62fc106a80 0x7f62fc19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.107+0000 7f6301636640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc110ea0 con 0x7f62fc100a80 2024-09-19T06:46:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.107+0000 7f6301636640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc110d20 con 0x7f62fc106a80 2024-09-19T06:46:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.107+0000 7f6301636640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc111020 con 0x7f62fc101420 2024-09-19T06:46:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.107+0000 7f62faffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62fc100a80 0x7f62fc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.107+0000 7f62faffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62fc100a80 0x7f62fc198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47484/0 (socket says 172.21.15.103:47484) 2024-09-19T06:46:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.107+0000 7f62faffd640 1 -- 172.21.15.103:0/501227917 learned_addr learned my addr 172.21.15.103:0/501227917 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:46:58.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.107+0000 7f62fa7fc640 1 --2- 172.21.15.103:0/501227917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62fc101420 0x7f62fc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:58.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.107+0000 7f62fb7fe640 1 --2- 172.21.15.103:0/501227917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62fc106a80 0x7f62fc19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:58.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.108+0000 7f62faffd640 1 -- 172.21.15.103:0/501227917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62fc101420 msgr2=0x7f62fc1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:58.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.108+0000 7f62faffd640 1 --2- 172.21.15.103:0/501227917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62fc101420 0x7f62fc1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:58.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.108+0000 7f62faffd640 1 -- 172.21.15.103:0/501227917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62fc106a80 msgr2=0x7f62fc19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:58.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.108+0000 7f62faffd640 1 --2- 172.21.15.103:0/501227917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62fc106a80 0x7f62fc19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:58.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.108+0000 7f62faffd640 1 -- 172.21.15.103:0/501227917 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62fc19d420 con 0x7f62fc100a80 2024-09-19T06:46:58.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.108+0000 7f62fb7fe640 1 --2- 172.21.15.103:0/501227917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62fc106a80 0x7f62fc19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:58.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.108+0000 7f62faffd640 1 --2- 172.21.15.103:0/501227917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62fc100a80 0x7f62fc198470 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f62ec00d950 tx=0x7f62ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:58.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.108+0000 7f62f3fff640 1 -- 172.21.15.103:0/501227917 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec017030 con 0x7f62fc100a80 2024-09-19T06:46:58.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.108+0000 7f62f3fff640 1 -- 172.21.15.103:0/501227917 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62ec004020 con 0x7f62fc100a80 2024-09-19T06:46:58.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.108+0000 7f6301636640 1 -- 172.21.15.103:0/501227917 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62fc19d710 con 0x7f62fc100a80 2024-09-19T06:46:58.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.108+0000 7f62f3fff640 1 -- 172.21.15.103:0/501227917 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec0047a0 con 0x7f62fc100a80 2024-09-19T06:46:58.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.108+0000 7f6301636640 1 -- 172.21.15.103:0/501227917 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62fc1a4ff0 con 0x7f62fc100a80 2024-09-19T06:46:58.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.109+0000 7f6301636640 1 -- 172.21.15.103:0/501227917 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62fc100ee0 con 0x7f62fc100a80 2024-09-19T06:46:58.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.110+0000 7f62f3fff640 1 -- 172.21.15.103:0/501227917 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f62ec0041d0 con 0x7f62fc100a80 2024-09-19T06:46:58.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.111+0000 7f62f3fff640 1 --2- 172.21.15.103:0/501227917 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f62e80a3240 0x7f62e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:58.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.111+0000 7f62f3fff640 1 -- 172.21.15.103:0/501227917 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f62ec0cee30 con 0x7f62fc100a80 2024-09-19T06:46:58.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.111+0000 7f62fa7fc640 1 --2- 172.21.15.103:0/501227917 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f62e80a3240 0x7f62e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:58.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.111+0000 7f62fa7fc640 1 --2- 172.21.15.103:0/501227917 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f62e80a3240 0x7f62e80a5700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009900 tx=0x7f62e40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:58.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.115+0000 7f62f3fff640 1 -- 172.21.15.103:0/501227917 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f62ec09bff0 con 0x7f62fc100a80 2024-09-19T06:46:58.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:57 smithi134 ceph-mon[35356]: Deploying daemon mds.f on smithi174 2024-09-19T06:46:58.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:57 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] up:boot 2024-09-19T06:46:58.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:57 smithi134 ceph-mon[35356]: fsmap 5 up:standby 2024-09-19T06:46:58.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:57 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:46:58.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.316+0000 7f6301636640 1 -- 172.21.15.103:0/501227917 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62fc1997d0 con 0x7f62fc100a80 2024-09-19T06:46:58.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.316+0000 7f62f3fff640 1 -- 172.21.15.103:0/501227917 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4747 (secure 0 0 0) 0x7f62ec0860c0 con 0x7f62fc100a80 2024-09-19T06:46:58.319 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:46:58.319 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":6,"btime":"2024-09-19T06:46:56:864701+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-19T06:46:58.319 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 6 2024-09-19T06:46:58.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.318+0000 7f6301636640 1 -- 172.21.15.103:0/501227917 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f62e80a3240 msgr2=0x7f62e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:58.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.318+0000 7f6301636640 1 --2- 172.21.15.103:0/501227917 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f62e80a3240 0x7f62e80a5700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009900 tx=0x7f62e40058d0 comp rx=0 tx=0).stop 2024-09-19T06:46:58.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.318+0000 7f6301636640 1 -- 172.21.15.103:0/501227917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62fc100a80 msgr2=0x7f62fc198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:58.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.318+0000 7f6301636640 1 --2- 172.21.15.103:0/501227917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62fc100a80 0x7f62fc198470 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f62ec00d950 tx=0x7f62ec00de10 comp rx=0 tx=0).stop 2024-09-19T06:46:58.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.318+0000 7f6301636640 1 -- 172.21.15.103:0/501227917 shutdown_connections 2024-09-19T06:46:58.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.318+0000 7f6301636640 1 --2- 172.21.15.103:0/501227917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62fc106a80 0x7f62fc19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:58.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.318+0000 7f6301636640 1 --2- 172.21.15.103:0/501227917 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f62e80a3240 0x7f62e80a5700 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:58.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.318+0000 7f6301636640 1 --2- 172.21.15.103:0/501227917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62fc101420 0x7f62fc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:58.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.318+0000 7f6301636640 1 --2- 172.21.15.103:0/501227917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62fc100a80 0x7f62fc198470 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:58.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.318+0000 7f6301636640 1 -- 172.21.15.103:0/501227917 >> 172.21.15.103:0/501227917 conn(0x7f62fc0fc7a0 msgr2=0x7f62fc06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:58.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.319+0000 7f6301636640 1 -- 172.21.15.103:0/501227917 shutdown_connections 2024-09-19T06:46:58.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:46:58.319+0000 7f6301636640 1 -- 172.21.15.103:0/501227917 wait complete. 2024-09-19T06:46:58.333 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 06:46:58 smithi174 systemd[1]: Starting Ceph mds.f for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T06:46:58.721 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 06:46:58 smithi174 podman[65733]: 2024-09-19 06:46:58.409914437 +0000 UTC m=+0.094429926 container create 93a672d45a6fbb5f0320b4e806b05000da0580be40faf1d5d2765541e139ad42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:46:58.721 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 06:46:58 smithi174 podman[65733]: 2024-09-19 06:46:58.331268656 +0000 UTC m=+0.015784158 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:46:58.721 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 06:46:58 smithi174 podman[65733]: 2024-09-19 06:46:58.628921078 +0000 UTC m=+0.313436581 container init 93a672d45a6fbb5f0320b4e806b05000da0580be40faf1d5d2765541e139ad42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:46:58.721 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 06:46:58 smithi174 podman[65733]: 2024-09-19 06:46:58.631849035 +0000 UTC m=+0.316364524 container start 93a672d45a6fbb5f0320b4e806b05000da0580be40faf1d5d2765541e139ad42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T06:46:58.721 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 06:46:58 smithi174 ceph-mds[65752]: -- 172.21.15.174:0/1693084747 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d85dc9860 con 0x556d85da7000 2024-09-19T06:46:58.721 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 06:46:58 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-f[65748]: starting mds.f at 2024-09-19T06:46:59.083 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 06:46:58 smithi174 bash[65733]: 93a672d45a6fbb5f0320b4e806b05000da0580be40faf1d5d2765541e139ad42 2024-09-19T06:46:59.083 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 06:46:58 smithi174 systemd[1]: Started Ceph mds.f for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:46:59.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:58 smithi174 ceph-mon[35637]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:59.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:58 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/501227917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:59.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:58 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:59.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:58 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:59.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:58 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:59.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:58 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:59.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:58 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:59.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:58 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:59.106 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:58 smithi103 ceph-mon[31158]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:59.107 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:58 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/501227917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:59.107 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:58 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:59.107 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:58 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:59.107 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:58 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:59.107 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:58 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:59.107 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:58 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:59.107 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:58 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:59.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:58 smithi134 ceph-mon[35356]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:46:59.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:58 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/501227917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:59.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:58 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:59.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:58 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:59.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:58 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:46:59.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:58 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:46:59.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:58 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:46:59.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:58 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:00.135 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:59 smithi174 ceph-mon[35637]: Deploying daemon mds.c on smithi174 2024-09-19T06:47:00.136 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:59 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.174:6834/963732203,v1:172.21.15.174:6835/963732203] up:boot 2024-09-19T06:47:00.136 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:59 smithi174 ceph-mon[35637]: fsmap 6 up:standby 2024-09-19T06:47:00.136 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:46:59 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:47:00.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:59 smithi134 ceph-mon[35356]: Deploying daemon mds.c on smithi174 2024-09-19T06:47:00.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:59 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.174:6834/963732203,v1:172.21.15.174:6835/963732203] up:boot 2024-09-19T06:47:00.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:59 smithi134 ceph-mon[35356]: fsmap 6 up:standby 2024-09-19T06:47:00.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:46:59 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:47:00.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:59 smithi103 ceph-mon[31158]: Deploying daemon mds.c on smithi174 2024-09-19T06:47:00.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:59 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.174:6834/963732203,v1:172.21.15.174:6835/963732203] up:boot 2024-09-19T06:47:00.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:59 smithi103 ceph-mon[31158]: fsmap 6 up:standby 2024-09-19T06:47:00.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:46:59 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:47:00.821 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 06:47:00 smithi174 systemd[1]: Starting Ceph mds.c for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T06:47:01.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:00 smithi174 ceph-mon[35637]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:01.083 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 06:47:00 smithi174 podman[66038]: 2024-09-19 06:47:00.820539368 +0000 UTC m=+0.111157811 container create e81cc5d6af587aee4aa9dd5330ec3c0d1daea70375b09e09c689b42fb9a74e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-c, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:47:01.083 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 06:47:00 smithi174 podman[66038]: 2024-09-19 06:47:00.724829906 +0000 UTC m=+0.015448319 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:47:01.083 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 06:47:01 smithi174 podman[66038]: 2024-09-19 06:47:01.064487412 +0000 UTC m=+0.355105810 container init e81cc5d6af587aee4aa9dd5330ec3c0d1daea70375b09e09c689b42fb9a74e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-c, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T06:47:01.083 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 06:47:01 smithi174 podman[66038]: 2024-09-19 06:47:01.067333413 +0000 UTC m=+0.357951820 container start e81cc5d6af587aee4aa9dd5330ec3c0d1daea70375b09e09c689b42fb9a74e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-c, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:47:01.133 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:47:01.161 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:00 smithi103 ceph-mon[31158]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:01.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:00 smithi134 ceph-mon[35356]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:01.395 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:01.583 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 06:47:01 smithi174 ceph-mds[66058]: -- 172.21.15.174:0/13186775 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9f4ead860 con 0x55f9f4e8a800 2024-09-19T06:47:01.583 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 06:47:01 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-c[66054]: starting mds.c at 2024-09-19T06:47:01.583 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 06:47:01 smithi174 bash[66038]: e81cc5d6af587aee4aa9dd5330ec3c0d1daea70375b09e09c689b42fb9a74e9e 2024-09-19T06:47:01.583 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 06:47:01 smithi174 systemd[1]: Started Ceph mds.c for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:47:01.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.879+0000 7fb1b12e2640 1 -- 172.21.15.103:0/3010624890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1ac106f10 msgr2=0x7fb1ac10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.879+0000 7fb1b12e2640 1 --2- 172.21.15.103:0/3010624890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1ac106f10 0x7fb1ac10ea90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009900 tx=0x7fb1a402a410 comp rx=0 tx=0).stop 2024-09-19T06:47:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.880+0000 7fb1b12e2640 1 -- 172.21.15.103:0/3010624890 shutdown_connections 2024-09-19T06:47:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.880+0000 7fb1b12e2640 1 --2- 172.21.15.103:0/3010624890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1ac106f10 0x7fb1ac10ea90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.880+0000 7fb1b12e2640 1 --2- 172.21.15.103:0/3010624890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1ac102790 0x7fb1ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.880+0000 7fb1b12e2640 1 --2- 172.21.15.103:0/3010624890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1ac101de0 0x7fb1ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.880+0000 7fb1b12e2640 1 -- 172.21.15.103:0/3010624890 >> 172.21.15.103:0/3010624890 conn(0x7fb1ac0fd8c0 msgr2=0x7fb1ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.880+0000 7fb1b12e2640 1 -- 172.21.15.103:0/3010624890 shutdown_connections 2024-09-19T06:47:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.880+0000 7fb1b12e2640 1 -- 172.21.15.103:0/3010624890 wait complete. 2024-09-19T06:47:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.880+0000 7fb1b12e2640 1 Processor -- start 2024-09-19T06:47:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.880+0000 7fb1b12e2640 1 -- start start 2024-09-19T06:47:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1b12e2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1ac101de0 0x7fb1ac1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1b12e2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1ac102790 0x7fb1ac1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1b12e2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1ac106f10 0x7fb1ac19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1b12e2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac111f70 con 0x7fb1ac106f10 2024-09-19T06:47:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1b12e2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac111df0 con 0x7fb1ac102790 2024-09-19T06:47:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1b12e2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac1120f0 con 0x7fb1ac101de0 2024-09-19T06:47:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1ab7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1ac106f10 0x7fb1ac19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1ab7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1ac106f10 0x7fb1ac19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47492/0 (socket says 172.21.15.103:47492) 2024-09-19T06:47:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1ab7fe640 1 -- 172.21.15.103:0/3233306846 learned_addr learned my addr 172.21.15.103:0/3233306846 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1aaffd640 1 --2- 172.21.15.103:0/3233306846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1ac101de0 0x7fb1ac1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1aa7fc640 1 --2- 172.21.15.103:0/3233306846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1ac102790 0x7fb1ac1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1ab7fe640 1 -- 172.21.15.103:0/3233306846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1ac101de0 msgr2=0x7fb1ac1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1ab7fe640 1 --2- 172.21.15.103:0/3233306846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1ac101de0 0x7fb1ac1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1ab7fe640 1 -- 172.21.15.103:0/3233306846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1ac102790 msgr2=0x7fb1ac1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1ab7fe640 1 --2- 172.21.15.103:0/3233306846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1ac102790 0x7fb1ac1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1ab7fe640 1 -- 172.21.15.103:0/3233306846 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ac19d360 con 0x7fb1ac106f10 2024-09-19T06:47:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1ab7fe640 1 --2- 172.21.15.103:0/3233306846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1ac106f10 0x7fb1ac19cc80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4005ab0 tx=0x7fb1a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1a3fff640 1 -- 172.21.15.103:0/3233306846 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a403f030 con 0x7fb1ac106f10 2024-09-19T06:47:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1b12e2640 1 -- 172.21.15.103:0/3233306846 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ac19d5f0 con 0x7fb1ac106f10 2024-09-19T06:47:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1a3fff640 1 -- 172.21.15.103:0/3233306846 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a4003ad0 con 0x7fb1ac106f10 2024-09-19T06:47:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1a3fff640 1 -- 172.21.15.103:0/3233306846 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4035070 con 0x7fb1ac106f10 2024-09-19T06:47:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.881+0000 7fb1b12e2640 1 -- 172.21.15.103:0/3233306846 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ac1a4f30 con 0x7fb1ac106f10 2024-09-19T06:47:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.882+0000 7fb1b12e2640 1 -- 172.21.15.103:0/3233306846 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1ac1032c0 con 0x7fb1ac106f10 2024-09-19T06:47:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.883+0000 7fb1a3fff640 1 -- 172.21.15.103:0/3233306846 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb1a4042030 con 0x7fb1ac106f10 2024-09-19T06:47:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.884+0000 7fb1a3fff640 1 --2- 172.21.15.103:0/3233306846 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb1980a3240 0x7fb1980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.884+0000 7fb1aaffd640 1 --2- 172.21.15.103:0/3233306846 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb1980a3240 0x7fb1980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.884+0000 7fb1a3fff640 1 -- 172.21.15.103:0/3233306846 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fb1a40ef950 con 0x7fb1ac106f10 2024-09-19T06:47:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.885+0000 7fb1aaffd640 1 --2- 172.21.15.103:0/3233306846 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb1980a3240 0x7fb1980a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb19c005fd0 tx=0x7fb19c009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:01.888+0000 7fb1a3fff640 1 -- 172.21.15.103:0/3233306846 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb1a40bc4c0 con 0x7fb1ac106f10 2024-09-19T06:47:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:02.092+0000 7fb1b12e2640 1 -- 172.21.15.103:0/3233306846 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1ac10a0f0 con 0x7fb1ac106f10 2024-09-19T06:47:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:02.093+0000 7fb1a3fff640 1 -- 172.21.15.103:0/3233306846 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6400 (secure 0 0 0) 0x7fb1a40c07c0 con 0x7fb1ac106f10 2024-09-19T06:47:02.095 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:47:02.095 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":8,"btime":"2024-09-19T06:47:01:890374+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T06:47:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 8 2024-09-19T06:47:02.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:02.096+0000 7fb1b12e2640 1 -- 172.21.15.103:0/3233306846 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb1980a3240 msgr2=0x7fb1980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:02.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:02.096+0000 7fb1b12e2640 1 --2- 172.21.15.103:0/3233306846 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb1980a3240 0x7fb1980a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb19c005fd0 tx=0x7fb19c009680 comp rx=0 tx=0).stop 2024-09-19T06:47:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:02.096+0000 7fb1b12e2640 1 -- 172.21.15.103:0/3233306846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1ac106f10 msgr2=0x7fb1ac19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:02.096+0000 7fb1b12e2640 1 --2- 172.21.15.103:0/3233306846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1ac106f10 0x7fb1ac19cc80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4005ab0 tx=0x7fb1a4004270 comp rx=0 tx=0).stop 2024-09-19T06:47:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:02.096+0000 7fb1b12e2640 1 -- 172.21.15.103:0/3233306846 shutdown_connections 2024-09-19T06:47:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:02.096+0000 7fb1b12e2640 1 --2- 172.21.15.103:0/3233306846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1ac106f10 0x7fb1ac19cc80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:02.096+0000 7fb1b12e2640 1 --2- 172.21.15.103:0/3233306846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1ac102790 0x7fb1ac1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:02.096+0000 7fb1b12e2640 1 --2- 172.21.15.103:0/3233306846 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb1980a3240 0x7fb1980a5700 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:02.096+0000 7fb1b12e2640 1 --2- 172.21.15.103:0/3233306846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1ac101de0 0x7fb1ac1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:02.096+0000 7fb1b12e2640 1 -- 172.21.15.103:0/3233306846 >> 172.21.15.103:0/3233306846 conn(0x7fb1ac0fd8c0 msgr2=0x7fb1ac0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:02.096+0000 7fb1b12e2640 1 -- 172.21.15.103:0/3233306846 shutdown_connections 2024-09-19T06:47:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:02.097+0000 7fb1b12e2640 1 -- 172.21.15.103:0/3233306846 wait complete. 2024-09-19T06:47:02.385 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:02 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:02.385 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:02 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:02.386 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:02 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:02.386 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:02 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:47:02.386 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:02 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:47:02.386 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:02 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:02.386 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:02 smithi103 ceph-mon[31158]: Deploying daemon mds.h on smithi134 2024-09-19T06:47:02.386 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:02 smithi103 ceph-mon[31158]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:02.386 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:02 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] up:boot 2024-09-19T06:47:02.386 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:02 smithi103 ceph-mon[31158]: fsmap 7 up:standby 2024-09-19T06:47:02.386 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:02 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:47:02.386 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:02 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3233306846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:02.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:02 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:02.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:02 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:02.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:02 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:02.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:02 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:47:02.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:02 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:47:02.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:02 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:02.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:02 smithi174 ceph-mon[35637]: Deploying daemon mds.h on smithi134 2024-09-19T06:47:02.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:02 smithi174 ceph-mon[35637]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:02.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:02 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] up:boot 2024-09-19T06:47:02.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:02 smithi174 ceph-mon[35637]: fsmap 7 up:standby 2024-09-19T06:47:02.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:02 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:47:02.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:02 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3233306846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:02.680 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:02 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:02.680 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:02 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:02.680 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:02 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:02.680 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:02 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:47:02.680 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:02 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:47:02.680 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:02 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:02.680 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:02 smithi134 ceph-mon[35356]: Deploying daemon mds.h on smithi134 2024-09-19T06:47:02.680 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:02 smithi134 ceph-mon[35356]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:02.680 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:02 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] up:boot 2024-09-19T06:47:02.680 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:02 smithi134 ceph-mon[35356]: fsmap 7 up:standby 2024-09-19T06:47:02.680 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:02 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:47:02.680 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:02 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3233306846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:03.437 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 06:47:03 smithi134 systemd[1]: Starting Ceph mds.h for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T06:47:03.763 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 06:47:03 smithi134 podman[65296]: 2024-09-19 06:47:03.406747179 +0000 UTC m=+0.015642940 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:47:03.763 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 06:47:03 smithi134 podman[65296]: 2024-09-19 06:47:03.535328622 +0000 UTC m=+0.144224380 container create 91cf7bec2287c75bb9d7ea0d6ad6381a33a3e1757e35d4a8a95a4f85e908c8cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T06:47:03.763 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 06:47:03 smithi134 podman[65296]: 2024-09-19 06:47:03.762440389 +0000 UTC m=+0.371336146 container init 91cf7bec2287c75bb9d7ea0d6ad6381a33a3e1757e35d4a8a95a4f85e908c8cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-h, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T06:47:04.188 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 06:47:03 smithi134 podman[65296]: 2024-09-19 06:47:03.765242482 +0000 UTC m=+0.374138249 container start 91cf7bec2287c75bb9d7ea0d6ad6381a33a3e1757e35d4a8a95a4f85e908c8cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-h, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-19T06:47:04.189 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 06:47:03 smithi134 ceph-mds[65315]: -- 172.21.15.134:0/2341491041 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5625357c9860 con 0x5625357a7000 2024-09-19T06:47:04.189 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 06:47:03 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-h[65311]: starting mds.h at 2024-09-19T06:47:04.189 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 06:47:03 smithi134 bash[65296]: 91cf7bec2287c75bb9d7ea0d6ad6381a33a3e1757e35d4a8a95a4f85e908c8cb 2024-09-19T06:47:04.189 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 06:47:03 smithi134 systemd[1]: Started Ceph mds.h for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:47:04.918 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:47:05.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:04 smithi174 ceph-mon[35637]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:05.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:04 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] up:boot 2024-09-19T06:47:05.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:04 smithi174 ceph-mon[35637]: fsmap 8 up:standby 2024-09-19T06:47:05.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:47:05.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:05.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:05.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:05.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:47:05.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:47:05.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:04 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:05.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:04 smithi174 ceph-mon[35637]: Deploying daemon mds.i on smithi174 2024-09-19T06:47:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:04 smithi103 ceph-mon[31158]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:04 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] up:boot 2024-09-19T06:47:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:04 smithi103 ceph-mon[31158]: fsmap 8 up:standby 2024-09-19T06:47:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:47:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:47:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:47:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:04 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:05.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:04 smithi103 ceph-mon[31158]: Deploying daemon mds.i on smithi174 2024-09-19T06:47:05.166 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:05.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:04 smithi134 ceph-mon[35356]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:05.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:04 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] up:boot 2024-09-19T06:47:05.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:04 smithi134 ceph-mon[35356]: fsmap 8 up:standby 2024-09-19T06:47:05.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:47:05.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:05.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:05.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:05.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:47:05.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:47:05.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:04 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:05.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:04 smithi134 ceph-mon[35356]: Deploying daemon mds.i on smithi174 2024-09-19T06:47:05.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.631+0000 7f2f9c827640 1 -- 172.21.15.103:0/2646827321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f98101420 msgr2=0x7f2f9810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:05.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.631+0000 7f2f9c827640 1 --2- 172.21.15.103:0/2646827321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f98101420 0x7f2f9810d950 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009cd0 tx=0x7f2f9002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.631+0000 7f2f9c827640 1 -- 172.21.15.103:0/2646827321 shutdown_connections 2024-09-19T06:47:05.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.631+0000 7f2f9c827640 1 --2- 172.21.15.103:0/2646827321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f98101420 0x7f2f9810d950 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.631+0000 7f2f9c827640 1 --2- 172.21.15.103:0/2646827321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f98100a80 0x7f2f98100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.631+0000 7f2f9c827640 1 --2- 172.21.15.103:0/2646827321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f98106a80 0x7f2f98106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.631+0000 7f2f9c827640 1 -- 172.21.15.103:0/2646827321 >> 172.21.15.103:0/2646827321 conn(0x7f2f980fc7a0 msgr2=0x7f2f980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:05.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.631+0000 7f2f9c827640 1 -- 172.21.15.103:0/2646827321 shutdown_connections 2024-09-19T06:47:05.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.631+0000 7f2f9c827640 1 -- 172.21.15.103:0/2646827321 wait complete. 2024-09-19T06:47:05.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.632+0000 7f2f9c827640 1 Processor -- start 2024-09-19T06:47:05.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.632+0000 7f2f9c827640 1 -- start start 2024-09-19T06:47:05.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.632+0000 7f2f9c827640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f98100a80 0x7f2f9819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.632+0000 7f2f9c827640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f98101420 0x7f2f9819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.632+0000 7f2f9c827640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f98106a80 0x7f2f9819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.632+0000 7f2f9c827640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98110f10 con 0x7f2f98106a80 2024-09-19T06:47:05.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.632+0000 7f2f9c827640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98110d90 con 0x7f2f98100a80 2024-09-19T06:47:05.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.632+0000 7f2f9c827640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98111090 con 0x7f2f98101420 2024-09-19T06:47:05.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.632+0000 7f2f96d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f98106a80 0x7f2f9819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.632+0000 7f2f96d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f98106a80 0x7f2f9819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47516/0 (socket says 172.21.15.103:47516) 2024-09-19T06:47:05.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.632+0000 7f2f96d9d640 1 -- 172.21.15.103:0/2362977473 learned_addr learned my addr 172.21.15.103:0/2362977473 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:05.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.632+0000 7f2f9659c640 1 --2- 172.21.15.103:0/2362977473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f98100a80 0x7f2f9819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.632+0000 7f2f95d9b640 1 --2- 172.21.15.103:0/2362977473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f98101420 0x7f2f9819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.633+0000 7f2f96d9d640 1 -- 172.21.15.103:0/2362977473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f98101420 msgr2=0x7f2f9819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.633+0000 7f2f96d9d640 1 --2- 172.21.15.103:0/2362977473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f98101420 0x7f2f9819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.633+0000 7f2f96d9d640 1 -- 172.21.15.103:0/2362977473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f98100a80 msgr2=0x7f2f9819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.633+0000 7f2f96d9d640 1 --2- 172.21.15.103:0/2362977473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f98100a80 0x7f2f9819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.633+0000 7f2f96d9d640 1 -- 172.21.15.103:0/2362977473 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9819f650 con 0x7f2f98106a80 2024-09-19T06:47:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.633+0000 7f2f95d9b640 1 --2- 172.21.15.103:0/2362977473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f98101420 0x7f2f9819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.633+0000 7f2f9659c640 1 --2- 172.21.15.103:0/2362977473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f98100a80 0x7f2f9819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.633+0000 7f2f96d9d640 1 --2- 172.21.15.103:0/2362977473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f98106a80 0x7f2f9819ef70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2f90033040 tx=0x7f2f90003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.633+0000 7f2f8f7fe640 1 -- 172.21.15.103:0/2362977473 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90004040 con 0x7f2f98106a80 2024-09-19T06:47:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.633+0000 7f2f9c827640 1 -- 172.21.15.103:0/2362977473 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9819f8e0 con 0x7f2f98106a80 2024-09-19T06:47:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.633+0000 7f2f8f7fe640 1 -- 172.21.15.103:0/2362977473 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f900046c0 con 0x7f2f98106a80 2024-09-19T06:47:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.633+0000 7f2f8f7fe640 1 -- 172.21.15.103:0/2362977473 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90046480 con 0x7f2f98106a80 2024-09-19T06:47:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.633+0000 7f2f9c827640 1 -- 172.21.15.103:0/2362977473 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f981a7220 con 0x7f2f98106a80 2024-09-19T06:47:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.634+0000 7f2f9c827640 1 -- 172.21.15.103:0/2362977473 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f98100ee0 con 0x7f2f98106a80 2024-09-19T06:47:05.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.635+0000 7f2f8f7fe640 1 -- 172.21.15.103:0/2362977473 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2f900305a0 con 0x7f2f98106a80 2024-09-19T06:47:05.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.636+0000 7f2f8f7fe640 1 --2- 172.21.15.103:0/2362977473 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2f840a3240 0x7f2f840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.636+0000 7f2f9659c640 1 --2- 172.21.15.103:0/2362977473 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2f840a3240 0x7f2f840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.636+0000 7f2f8f7fe640 1 -- 172.21.15.103:0/2362977473 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f2f9002e030 con 0x7f2f98106a80 2024-09-19T06:47:05.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.636+0000 7f2f9659c640 1 --2- 172.21.15.103:0/2362977473 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2f840a3240 0x7f2f840a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009930 tx=0x7f2f88008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:05.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.639+0000 7f2f8f7fe640 1 -- 172.21.15.103:0/2362977473 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2f9004b060 con 0x7f2f98106a80 2024-09-19T06:47:05.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.838+0000 7f2f9c827640 1 -- 172.21.15.103:0/2362977473 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f981a74c0 con 0x7f2f98106a80 2024-09-19T06:47:05.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.839+0000 7f2f8f7fe640 1 -- 172.21.15.103:0/2362977473 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7228 (secure 0 0 0) 0x7f2f900bdde0 con 0x7f2f98106a80 2024-09-19T06:47:05.840 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:47:05.840 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":9,"btime":"2024-09-19T06:47:03:843529+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-19T06:47:05.840 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 9 2024-09-19T06:47:05.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.841+0000 7f2f9c827640 1 -- 172.21.15.103:0/2362977473 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2f840a3240 msgr2=0x7f2f840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:05.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.841+0000 7f2f9c827640 1 --2- 172.21.15.103:0/2362977473 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2f840a3240 0x7f2f840a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009930 tx=0x7f2f88008040 comp rx=0 tx=0).stop 2024-09-19T06:47:05.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.841+0000 7f2f9c827640 1 -- 172.21.15.103:0/2362977473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f98106a80 msgr2=0x7f2f9819ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:05.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.841+0000 7f2f9c827640 1 --2- 172.21.15.103:0/2362977473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f98106a80 0x7f2f9819ef70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2f90033040 tx=0x7f2f90003970 comp rx=0 tx=0).stop 2024-09-19T06:47:05.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.841+0000 7f2f9c827640 1 -- 172.21.15.103:0/2362977473 shutdown_connections 2024-09-19T06:47:05.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.841+0000 7f2f9c827640 1 --2- 172.21.15.103:0/2362977473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f98106a80 0x7f2f9819ef70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.841+0000 7f2f9c827640 1 --2- 172.21.15.103:0/2362977473 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2f840a3240 0x7f2f840a5700 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.841+0000 7f2f9c827640 1 --2- 172.21.15.103:0/2362977473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f98101420 0x7f2f9819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.841+0000 7f2f9c827640 1 --2- 172.21.15.103:0/2362977473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f98100a80 0x7f2f9819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.841+0000 7f2f9c827640 1 -- 172.21.15.103:0/2362977473 >> 172.21.15.103:0/2362977473 conn(0x7f2f980fc7a0 msgr2=0x7f2f9806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:05.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.841+0000 7f2f9c827640 1 -- 172.21.15.103:0/2362977473 shutdown_connections 2024-09-19T06:47:05.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:05.841+0000 7f2f9c827640 1 -- 172.21.15.103:0/2362977473 wait complete. 2024-09-19T06:47:06.018 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:05 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2362977473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:06.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:05 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2362977473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:06.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:05 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2362977473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:06.333 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 06:47:06 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-i[66358]: starting mds.i at 2024-09-19T06:47:06.833 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 06:47:06 smithi174 bash[66343]: 8fbbb96b41ec47e0fa8dcf6a38651e9d5b582831eaa938c95cafca920b5a04a7 2024-09-19T06:47:06.833 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 06:47:06 smithi174 systemd[1]: Started Ceph mds.i for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:47:07.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:06 smithi103 ceph-mon[31158]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:07.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:06 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:07.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:06 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:07.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:06 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:07.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:06 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:47:07.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:06 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:47:07.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:06 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:07.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:06 smithi134 ceph-mon[35356]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:07.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:06 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:07.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:06 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:07.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:06 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:07.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:06 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:47:07.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:06 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:47:07.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:06 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:07.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:06 smithi174 ceph-mon[35637]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:07.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:06 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:07.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:06 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:07.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:06 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:07.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:06 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:47:07.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:06 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:47:07.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:06 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:08.102 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:07 smithi103 ceph-mon[31158]: Deploying daemon mds.g on smithi103 2024-09-19T06:47:08.103 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:07 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] up:boot 2024-09-19T06:47:08.103 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:07 smithi103 ceph-mon[31158]: fsmap 9 up:standby 2024-09-19T06:47:08.103 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:07 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:47:08.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:07 smithi134 ceph-mon[35356]: Deploying daemon mds.g on smithi103 2024-09-19T06:47:08.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:07 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] up:boot 2024-09-19T06:47:08.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:07 smithi134 ceph-mon[35356]: fsmap 9 up:standby 2024-09-19T06:47:08.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:07 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:47:08.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:07 smithi174 ceph-mon[35637]: Deploying daemon mds.g on smithi103 2024-09-19T06:47:08.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:07 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] up:boot 2024-09-19T06:47:08.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:07 smithi174 ceph-mon[35637]: fsmap 9 up:standby 2024-09-19T06:47:08.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:07 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:47:08.388 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 06:47:08 smithi103 systemd[1]: Starting Ceph mds.g for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T06:47:08.438 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:47:08.753 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 06:47:08 smithi103 podman[72066]: 2024-09-19 06:47:08.405934185 +0000 UTC m=+0.016481207 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:47:08.753 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 06:47:08 smithi103 podman[72066]: 2024-09-19 06:47:08.524778932 +0000 UTC m=+0.135325932 container create e59a59eb9a948822cad8d1e640eb051f7688106335231a110e511e4cfcf1a156 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:47:08.753 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 06:47:08 smithi103 podman[72066]: 2024-09-19 06:47:08.752537132 +0000 UTC m=+0.363084149 container init e59a59eb9a948822cad8d1e640eb051f7688106335231a110e511e4cfcf1a156 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:47:08.922 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:09.044 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:08 smithi103 ceph-mon[31158]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:09.044 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 06:47:08 smithi103 podman[72066]: 2024-09-19 06:47:08.755895723 +0000 UTC m=+0.366442723 container start e59a59eb9a948822cad8d1e640eb051f7688106335231a110e511e4cfcf1a156 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:47:09.044 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 06:47:08 smithi103 ceph-mds[72129]: -- 172.21.15.103:0/4293483722 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562e5273f860 con 0x562e5271c800 2024-09-19T06:47:09.044 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 06:47:08 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-g[72112]: starting mds.g at 2024-09-19T06:47:09.045 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 06:47:08 smithi103 bash[72066]: e59a59eb9a948822cad8d1e640eb051f7688106335231a110e511e4cfcf1a156 2024-09-19T06:47:09.045 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 06:47:08 smithi103 systemd[1]: Started Ceph mds.g for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:47:09.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:08 smithi134 ceph-mon[35356]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:09.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:08 smithi174 ceph-mon[35637]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:09.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.427+0000 7ff1d2d62640 1 -- 172.21.15.103:0/3232680463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc100a80 msgr2=0x7ff1cc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:09.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.427+0000 7ff1d2d62640 1 --2- 172.21.15.103:0/3232680463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc100a80 0x7ff1cc100ee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009930 tx=0x7ff1b802a420 comp rx=0 tx=0).stop 2024-09-19T06:47:09.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.427+0000 7ff1d2d62640 1 -- 172.21.15.103:0/3232680463 shutdown_connections 2024-09-19T06:47:09.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.427+0000 7ff1d2d62640 1 --2- 172.21.15.103:0/3232680463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1cc101420 0x7ff1cc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:09.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.427+0000 7ff1d2d62640 1 --2- 172.21.15.103:0/3232680463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc100a80 0x7ff1cc100ee0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:09.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.427+0000 7ff1d2d62640 1 --2- 172.21.15.103:0/3232680463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1cc106a80 0x7ff1cc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:09.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.427+0000 7ff1d2d62640 1 -- 172.21.15.103:0/3232680463 >> 172.21.15.103:0/3232680463 conn(0x7ff1cc0fc7a0 msgr2=0x7ff1cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:09.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.427+0000 7ff1d2d62640 1 -- 172.21.15.103:0/3232680463 shutdown_connections 2024-09-19T06:47:09.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.427+0000 7ff1d2d62640 1 -- 172.21.15.103:0/3232680463 wait complete. 2024-09-19T06:47:09.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.428+0000 7ff1d2d62640 1 Processor -- start 2024-09-19T06:47:09.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.428+0000 7ff1d2d62640 1 -- start start 2024-09-19T06:47:09.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.428+0000 7ff1d2d62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1cc100a80 0x7ff1cc1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:09.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.428+0000 7ff1d2d62640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc101420 0x7ff1cc1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:09.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.428+0000 7ff1d2d62640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1cc106a80 0x7ff1cc19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:09.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.428+0000 7ff1d2d62640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc110e80 con 0x7ff1cc101420 2024-09-19T06:47:09.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.428+0000 7ff1d2d62640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc110d00 con 0x7ff1cc100a80 2024-09-19T06:47:09.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.428+0000 7ff1d2d62640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc111000 con 0x7ff1cc106a80 2024-09-19T06:47:09.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.428+0000 7ff1cbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc101420 0x7ff1cc1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:09.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.428+0000 7ff1cbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc101420 0x7ff1cc1978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52984/0 (socket says 172.21.15.103:52984) 2024-09-19T06:47:09.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.428+0000 7ff1cbfff640 1 -- 172.21.15.103:0/1094071910 learned_addr learned my addr 172.21.15.103:0/1094071910 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:09.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.428+0000 7ff1d0afe640 1 --2- 172.21.15.103:0/1094071910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1cc100a80 0x7ff1cc1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:09.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.428+0000 7ff1d12ff640 1 --2- 172.21.15.103:0/1094071910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1cc106a80 0x7ff1cc19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:09.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.429+0000 7ff1cbfff640 1 -- 172.21.15.103:0/1094071910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1cc106a80 msgr2=0x7ff1cc19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:09.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.429+0000 7ff1cbfff640 1 --2- 172.21.15.103:0/1094071910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1cc106a80 0x7ff1cc19bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:09.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.429+0000 7ff1cbfff640 1 -- 172.21.15.103:0/1094071910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1cc100a80 msgr2=0x7ff1cc1973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:09.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.429+0000 7ff1cbfff640 1 --2- 172.21.15.103:0/1094071910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1cc100a80 0x7ff1cc1973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:09.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.429+0000 7ff1cbfff640 1 -- 172.21.15.103:0/1094071910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc19c350 con 0x7ff1cc101420 2024-09-19T06:47:09.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.429+0000 7ff1cbfff640 1 --2- 172.21.15.103:0/1094071910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc101420 0x7ff1cc1978e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009230 tx=0x7ff1b8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:09.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.429+0000 7ff1c9ffb640 1 -- 172.21.15.103:0/1094071910 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b803f030 con 0x7ff1cc101420 2024-09-19T06:47:09.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.429+0000 7ff1d2d62640 1 -- 172.21.15.103:0/1094071910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1cc19c5e0 con 0x7ff1cc101420 2024-09-19T06:47:09.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.429+0000 7ff1c9ffb640 1 -- 172.21.15.103:0/1094071910 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1b8004440 con 0x7ff1cc101420 2024-09-19T06:47:09.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.429+0000 7ff1c9ffb640 1 -- 172.21.15.103:0/1094071910 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8035070 con 0x7ff1cc101420 2024-09-19T06:47:09.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.429+0000 7ff1d2d62640 1 -- 172.21.15.103:0/1094071910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1cc1a3f20 con 0x7ff1cc101420 2024-09-19T06:47:09.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.430+0000 7ff1d2d62640 1 -- 172.21.15.103:0/1094071910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1cc1021c0 con 0x7ff1cc101420 2024-09-19T06:47:09.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.431+0000 7ff1c9ffb640 1 -- 172.21.15.103:0/1094071910 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff1b8042030 con 0x7ff1cc101420 2024-09-19T06:47:09.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.432+0000 7ff1c9ffb640 1 --2- 172.21.15.103:0/1094071910 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff1bc0a3240 0x7ff1bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:09.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.432+0000 7ff1d0afe640 1 --2- 172.21.15.103:0/1094071910 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff1bc0a3240 0x7ff1bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:09.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.432+0000 7ff1c9ffb640 1 -- 172.21.15.103:0/1094071910 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7ff1b80ef750 con 0x7ff1cc101420 2024-09-19T06:47:09.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.432+0000 7ff1d0afe640 1 --2- 172.21.15.103:0/1094071910 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff1bc0a3240 0x7ff1bc0a5700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009b50 tx=0x7ff1c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:09.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.436+0000 7ff1c9ffb640 1 -- 172.21.15.103:0/1094071910 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff1b80bc2c0 con 0x7ff1cc101420 2024-09-19T06:47:09.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.646+0000 7ff1d2d62640 1 -- 172.21.15.103:0/1094071910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1cc109020 con 0x7ff1cc101420 2024-09-19T06:47:09.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.647+0000 7ff1c9ffb640 1 -- 172.21.15.103:0/1094071910 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8887 (secure 0 0 0) 0x7ff1b80c05c0 con 0x7ff1cc101420 2024-09-19T06:47:09.648 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:47:09.648 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":11,"btime":"2024-09-19T06:47:08:869086+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-19T06:47:09.648 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 11 2024-09-19T06:47:09.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.649+0000 7ff1d2d62640 1 -- 172.21.15.103:0/1094071910 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff1bc0a3240 msgr2=0x7ff1bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:09.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.649+0000 7ff1d2d62640 1 --2- 172.21.15.103:0/1094071910 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff1bc0a3240 0x7ff1bc0a5700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009b50 tx=0x7ff1c0005f90 comp rx=0 tx=0).stop 2024-09-19T06:47:09.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.649+0000 7ff1d2d62640 1 -- 172.21.15.103:0/1094071910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc101420 msgr2=0x7ff1cc1978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:09.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.649+0000 7ff1d2d62640 1 --2- 172.21.15.103:0/1094071910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc101420 0x7ff1cc1978e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009230 tx=0x7ff1b8004040 comp rx=0 tx=0).stop 2024-09-19T06:47:09.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.649+0000 7ff1d2d62640 1 -- 172.21.15.103:0/1094071910 shutdown_connections 2024-09-19T06:47:09.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.649+0000 7ff1d2d62640 1 --2- 172.21.15.103:0/1094071910 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff1bc0a3240 0x7ff1bc0a5700 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:09.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.649+0000 7ff1d2d62640 1 --2- 172.21.15.103:0/1094071910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1cc106a80 0x7ff1cc19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:09.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.649+0000 7ff1d2d62640 1 --2- 172.21.15.103:0/1094071910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc101420 0x7ff1cc1978e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:09.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.649+0000 7ff1d2d62640 1 --2- 172.21.15.103:0/1094071910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1cc100a80 0x7ff1cc1973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:09.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.649+0000 7ff1d2d62640 1 -- 172.21.15.103:0/1094071910 >> 172.21.15.103:0/1094071910 conn(0x7ff1cc0fc7a0 msgr2=0x7ff1cc0fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:09.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.649+0000 7ff1d2d62640 1 -- 172.21.15.103:0/1094071910 shutdown_connections 2024-09-19T06:47:09.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:09.649+0000 7ff1d2d62640 1 -- 172.21.15.103:0/1094071910 wait complete. 2024-09-19T06:47:09.928 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:09 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.103:6838/3119326077,v1:172.21.15.103:6839/3119326077] up:boot 2024-09-19T06:47:09.928 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:09 smithi103 ceph-mon[31158]: fsmap 10 up:standby 2024-09-19T06:47:09.928 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:09 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:47:09.928 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:09 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:09.929 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:09 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:09.929 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:09 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:09.929 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:09 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:47:09.929 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:09 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:47:09.929 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:09 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:09.929 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:09 smithi103 ceph-mon[31158]: Deploying daemon mds.a on smithi103 2024-09-19T06:47:09.929 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:09 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1094071910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:10.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:09 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.103:6838/3119326077,v1:172.21.15.103:6839/3119326077] up:boot 2024-09-19T06:47:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:09 smithi134 ceph-mon[35356]: fsmap 10 up:standby 2024-09-19T06:47:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:09 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:47:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:09 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:09 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:09 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:09 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:47:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:09 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:47:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:09 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:09 smithi134 ceph-mon[35356]: Deploying daemon mds.a on smithi103 2024-09-19T06:47:10.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:09 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1094071910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:10.336 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:09 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.103:6838/3119326077,v1:172.21.15.103:6839/3119326077] up:boot 2024-09-19T06:47:10.337 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:09 smithi174 ceph-mon[35637]: fsmap 10 up:standby 2024-09-19T06:47:10.337 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:09 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:47:10.337 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:09 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:10.337 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:09 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:10.337 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:09 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:10.337 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:09 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:47:10.337 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:09 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:47:10.337 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:09 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:10.337 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:09 smithi174 ceph-mon[35637]: Deploying daemon mds.a on smithi103 2024-09-19T06:47:10.337 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:09 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1094071910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:11.094 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:10 smithi103 ceph-mon[31158]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:11.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:10 smithi134 ceph-mon[35356]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:11.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:10 smithi174 ceph-mon[35637]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:11.638 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 06:47:11 smithi103 systemd[1]: Starting Ceph mds.a for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T06:47:12.050 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 06:47:11 smithi103 podman[72480]: 2024-09-19 06:47:11.688509321 +0000 UTC m=+0.016368468 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:47:12.050 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 06:47:11 smithi103 podman[72480]: 2024-09-19 06:47:11.787748739 +0000 UTC m=+0.115607915 container create a036b7678ef625c8e1776c40d4a8686e6a5eb294dc7e54c7eeff8c533a7a0ed0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4) 2024-09-19T06:47:12.051 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 06:47:12 smithi103 podman[72480]: 2024-09-19 06:47:12.024293539 +0000 UTC m=+0.352152683 container init a036b7678ef625c8e1776c40d4a8686e6a5eb294dc7e54c7eeff8c533a7a0ed0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-19T06:47:12.051 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 06:47:12 smithi103 podman[72480]: 2024-09-19 06:47:12.027370894 +0000 UTC m=+0.355230038 container start a036b7678ef625c8e1776c40d4a8686e6a5eb294dc7e54c7eeff8c533a7a0ed0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T06:47:12.387 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 06:47:12 smithi103 ceph-mds[72499]: -- 172.21.15.103:0/2729732259 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae8e359860 con 0x55ae8e337400 2024-09-19T06:47:12.388 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 06:47:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-a[72495]: starting mds.a at 2024-09-19T06:47:12.388 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 06:47:12 smithi103 bash[72480]: a036b7678ef625c8e1776c40d4a8686e6a5eb294dc7e54c7eeff8c533a7a0ed0 2024-09-19T06:47:12.388 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 06:47:12 smithi103 systemd[1]: Started Ceph mds.a for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:47:12.628 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:47:12.881 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:13.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:12 smithi134 ceph-mon[35356]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:13.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:12 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:13.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:12 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:13.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:12 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:13.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:12 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:47:13.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:12 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:47:13.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:12 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:13.229 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:12 smithi103 ceph-mon[31158]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:13.229 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:12 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:13.229 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:12 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:13.229 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:12 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:13.229 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:12 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:47:13.229 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:12 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:47:13.229 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:12 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:13.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:12 smithi174 ceph-mon[35637]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:13.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:12 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:13.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:12 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:13.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:12 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:13.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:12 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:47:13.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:12 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:47:13.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:12 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.308+0000 7f706d830640 1 -- 172.21.15.103:0/4236739785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7068102d40 msgr2=0x7f7068103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.308+0000 7f706d830640 1 --2- 172.21.15.103:0/4236739785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7068102d40 0x7f7068103120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7058009900 tx=0x7f705802a480 comp rx=0 tx=0).stop 2024-09-19T06:47:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.309+0000 7f706d830640 1 -- 172.21.15.103:0/4236739785 shutdown_connections 2024-09-19T06:47:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.309+0000 7f706d830640 1 --2- 172.21.15.103:0/4236739785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f706810c170 0x7f706810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.309+0000 7f706d830640 1 --2- 172.21.15.103:0/4236739785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7068103660 0x7f706810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.309+0000 7f706d830640 1 --2- 172.21.15.103:0/4236739785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7068102d40 0x7f7068103120 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.309+0000 7f706d830640 1 -- 172.21.15.103:0/4236739785 >> 172.21.15.103:0/4236739785 conn(0x7f70680fc7a0 msgr2=0x7f70680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.309+0000 7f706d830640 1 -- 172.21.15.103:0/4236739785 shutdown_connections 2024-09-19T06:47:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.309+0000 7f706d830640 1 -- 172.21.15.103:0/4236739785 wait complete. 2024-09-19T06:47:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.309+0000 7f706d830640 1 Processor -- start 2024-09-19T06:47:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.309+0000 7f706d830640 1 -- start start 2024-09-19T06:47:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f706d830640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7068102d40 0x7f7068198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f706d830640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7068103660 0x7f7068198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f706d830640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706810c170 0x7f706819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f706d830640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7068111f70 con 0x7f7068103660 2024-09-19T06:47:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f706d830640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7068111df0 con 0x7f706810c170 2024-09-19T06:47:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f706d830640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70681120f0 con 0x7f7068102d40 2024-09-19T06:47:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f70667fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7068103660 0x7f7068198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f70667fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7068103660 0x7f7068198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53026/0 (socket says 172.21.15.103:53026) 2024-09-19T06:47:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f70667fc640 1 -- 172.21.15.103:0/502891145 learned_addr learned my addr 172.21.15.103:0/502891145 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f70677fe640 1 --2- 172.21.15.103:0/502891145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706810c170 0x7f706819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f7066ffd640 1 --2- 172.21.15.103:0/502891145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7068102d40 0x7f7068198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f70667fc640 1 -- 172.21.15.103:0/502891145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7068102d40 msgr2=0x7f7068198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f70667fc640 1 --2- 172.21.15.103:0/502891145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7068102d40 0x7f7068198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f70667fc640 1 -- 172.21.15.103:0/502891145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706810c170 msgr2=0x7f706819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f70667fc640 1 --2- 172.21.15.103:0/502891145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706810c170 0x7f706819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f70667fc640 1 -- 172.21.15.103:0/502891145 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706819d400 con 0x7f7068103660 2024-09-19T06:47:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f70677fe640 1 --2- 172.21.15.103:0/502891145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706810c170 0x7f706819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f70667fc640 1 --2- 172.21.15.103:0/502891145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7068103660 0x7f7068198990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f705c00d740 tx=0x7f705c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f706c82e640 1 -- 172.21.15.103:0/502891145 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c017070 con 0x7f7068103660 2024-09-19T06:47:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f706c82e640 1 -- 172.21.15.103:0/502891145 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f705c00de50 con 0x7f7068103660 2024-09-19T06:47:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f706d830640 1 -- 172.21.15.103:0/502891145 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f706819d6f0 con 0x7f7068103660 2024-09-19T06:47:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.310+0000 7f706c82e640 1 -- 172.21.15.103:0/502891145 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c00f620 con 0x7f7068103660 2024-09-19T06:47:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.311+0000 7f706d830640 1 -- 172.21.15.103:0/502891145 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70681a4fd0 con 0x7f7068103660 2024-09-19T06:47:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.312+0000 7f706d830640 1 -- 172.21.15.103:0/502891145 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7068103120 con 0x7f7068103660 2024-09-19T06:47:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.313+0000 7f706c82e640 1 -- 172.21.15.103:0/502891145 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f705c009c30 con 0x7f7068103660 2024-09-19T06:47:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.314+0000 7f706c82e640 1 --2- 172.21.15.103:0/502891145 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f70500a3240 0x7f70500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.314+0000 7f7066ffd640 1 --2- 172.21.15.103:0/502891145 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f70500a3240 0x7f70500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.314+0000 7f706c82e640 1 -- 172.21.15.103:0/502891145 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f705c0cf180 con 0x7f7068103660 2024-09-19T06:47:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.314+0000 7f7066ffd640 1 --2- 172.21.15.103:0/502891145 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f70500a3240 0x7f70500a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7058005ab0 tx=0x7f7058005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:13.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.317+0000 7f706c82e640 1 -- 172.21.15.103:0/502891145 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f705c09c2e0 con 0x7f7068103660 2024-09-19T06:47:13.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.515+0000 7f706d830640 1 -- 172.21.15.103:0/502891145 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7068107270 con 0x7f7068103660 2024-09-19T06:47:13.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.516+0000 7f706c82e640 1 -- 172.21.15.103:0/502891145 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9716 (secure 0 0 0) 0x7f705c09bbe0 con 0x7f7068103660 2024-09-19T06:47:13.517 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:47:13.517 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":12,"btime":"2024-09-19T06:47:12:928425+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-19T06:47:13.518 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 12 2024-09-19T06:47:13.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.518+0000 7f706d830640 1 -- 172.21.15.103:0/502891145 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f70500a3240 msgr2=0x7f70500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:13.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.518+0000 7f706d830640 1 --2- 172.21.15.103:0/502891145 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f70500a3240 0x7f70500a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7058005ab0 tx=0x7f7058005a20 comp rx=0 tx=0).stop 2024-09-19T06:47:13.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.518+0000 7f706d830640 1 -- 172.21.15.103:0/502891145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7068103660 msgr2=0x7f7068198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:13.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.518+0000 7f706d830640 1 --2- 172.21.15.103:0/502891145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7068103660 0x7f7068198990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f705c00d740 tx=0x7f705c00dc00 comp rx=0 tx=0).stop 2024-09-19T06:47:13.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.518+0000 7f706d830640 1 -- 172.21.15.103:0/502891145 shutdown_connections 2024-09-19T06:47:13.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.518+0000 7f706d830640 1 --2- 172.21.15.103:0/502891145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706810c170 0x7f706819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:13.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.518+0000 7f706d830640 1 --2- 172.21.15.103:0/502891145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7068103660 0x7f7068198990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:13.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.518+0000 7f706d830640 1 --2- 172.21.15.103:0/502891145 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f70500a3240 0x7f70500a5700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:13.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.518+0000 7f706d830640 1 --2- 172.21.15.103:0/502891145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7068102d40 0x7f7068198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:13.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.518+0000 7f706d830640 1 -- 172.21.15.103:0/502891145 >> 172.21.15.103:0/502891145 conn(0x7f70680fc7a0 msgr2=0x7f70680fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:13.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.518+0000 7f706d830640 1 -- 172.21.15.103:0/502891145 shutdown_connections 2024-09-19T06:47:13.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:13.518+0000 7f706d830640 1 -- 172.21.15.103:0/502891145 wait complete. 2024-09-19T06:47:14.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:13 smithi134 ceph-mon[35356]: Deploying daemon mds.b on smithi134 2024-09-19T06:47:14.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:13 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.103:6840/3771437304,v1:172.21.15.103:6841/3771437304] up:boot 2024-09-19T06:47:14.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:13 smithi134 ceph-mon[35356]: fsmap 11 up:standby 2024-09-19T06:47:14.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:13 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:47:14.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:13 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/502891145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:14.233 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:13 smithi103 ceph-mon[31158]: Deploying daemon mds.b on smithi134 2024-09-19T06:47:14.234 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:13 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.103:6840/3771437304,v1:172.21.15.103:6841/3771437304] up:boot 2024-09-19T06:47:14.234 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:13 smithi103 ceph-mon[31158]: fsmap 11 up:standby 2024-09-19T06:47:14.234 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:13 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:47:14.234 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:13 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/502891145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:14.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:13 smithi174 ceph-mon[35637]: Deploying daemon mds.b on smithi134 2024-09-19T06:47:14.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:13 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.103:6840/3771437304,v1:172.21.15.103:6841/3771437304] up:boot 2024-09-19T06:47:14.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:13 smithi174 ceph-mon[35637]: fsmap 11 up:standby 2024-09-19T06:47:14.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:13 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:47:14.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:13 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/502891145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:14.934 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 06:47:14 smithi134 systemd[1]: Started Ceph mds.b for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T06:47:15.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:14 smithi134 ceph-mon[35356]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:15.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:14 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:15.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:14 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:15.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:14 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:15.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:14 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:15.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:14 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:15.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:14 smithi174 ceph-mon[35637]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:15.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:14 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:15.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:14 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:15.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:14 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:15.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:14 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:15.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:14 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:15.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:14 smithi103 ceph-mon[31158]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:15.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:14 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:15.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:14 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:15.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:14 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:15.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:14 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:15.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:14 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:16.263 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:47:16.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:15 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] up:boot 2024-09-19T06:47:16.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:15 smithi103 ceph-mon[31158]: fsmap 12 up:standby 2024-09-19T06:47:16.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:47:16.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:15 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:47:16.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:15 smithi103 ceph-mon[31158]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:15 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] up:boot 2024-09-19T06:47:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:15 smithi174 ceph-mon[35637]: fsmap 12 up:standby 2024-09-19T06:47:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:47:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:15 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:47:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:15 smithi174 ceph-mon[35637]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:16.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:15 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] up:boot 2024-09-19T06:47:16.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:15 smithi134 ceph-mon[35356]: fsmap 12 up:standby 2024-09-19T06:47:16.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:47:16.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:15 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:47:16.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:15 smithi134 ceph-mon[35356]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:16.549 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.006+0000 7f9f55b19640 1 -- 172.21.15.103:0/2444305818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f50102d30 msgr2=0x7f9f50103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.006+0000 7f9f55b19640 1 --2- 172.21.15.103:0/2444305818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f50102d30 0x7f9f50103110 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009cd0 tx=0x7f9f4002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:47:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.006+0000 7f9f55b19640 1 -- 172.21.15.103:0/2444305818 shutdown_connections 2024-09-19T06:47:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.006+0000 7f9f55b19640 1 --2- 172.21.15.103:0/2444305818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f501082a0 0x7f9f5010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.006+0000 7f9f55b19640 1 --2- 172.21.15.103:0/2444305818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f501036e0 0x7f9f50107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.006+0000 7f9f55b19640 1 --2- 172.21.15.103:0/2444305818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f50102d30 0x7f9f50103110 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.006+0000 7f9f55b19640 1 -- 172.21.15.103:0/2444305818 >> 172.21.15.103:0/2444305818 conn(0x7f9f500fc7d0 msgr2=0x7f9f500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.006+0000 7f9f55b19640 1 -- 172.21.15.103:0/2444305818 shutdown_connections 2024-09-19T06:47:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.006+0000 7f9f55b19640 1 -- 172.21.15.103:0/2444305818 wait complete. 2024-09-19T06:47:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.006+0000 7f9f55b19640 1 Processor -- start 2024-09-19T06:47:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f55b19640 1 -- start start 2024-09-19T06:47:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f55b19640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f50102d30 0x7f9f50100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f55b19640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f501036e0 0x7f9f50100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f55b19640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f501082a0 0x7f9f50101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f55b19640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f501120a0 con 0x7f9f501082a0 2024-09-19T06:47:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f55b19640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9f50111f20 con 0x7f9f501036e0 2024-09-19T06:47:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f55b19640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9f50112220 con 0x7f9f50102d30 2024-09-19T06:47:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f4ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f501082a0 0x7f9f50101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f4ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f501082a0 0x7f9f50101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53048/0 (socket says 172.21.15.103:53048) 2024-09-19T06:47:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f4ffff640 1 -- 172.21.15.103:0/3954431597 learned_addr learned my addr 172.21.15.103:0/3954431597 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f4f7fe640 1 --2- 172.21.15.103:0/3954431597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f50102d30 0x7f9f50100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f4effd640 1 --2- 172.21.15.103:0/3954431597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f501036e0 0x7f9f50100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f4ffff640 1 -- 172.21.15.103:0/3954431597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f50102d30 msgr2=0x7f9f50100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f4ffff640 1 --2- 172.21.15.103:0/3954431597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f50102d30 0x7f9f50100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f4ffff640 1 -- 172.21.15.103:0/3954431597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f501036e0 msgr2=0x7f9f50100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f4ffff640 1 --2- 172.21.15.103:0/3954431597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f501036e0 0x7f9f50100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f4ffff640 1 -- 172.21.15.103:0/3954431597 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f501a6aa0 con 0x7f9f501082a0 2024-09-19T06:47:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.007+0000 7f9f4ffff640 1 --2- 172.21.15.103:0/3954431597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f501082a0 0x7f9f50101410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800d750 tx=0x7f9f4800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.008+0000 7f9f4cff9640 1 -- 172.21.15.103:0/3954431597 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f48017070 con 0x7f9f501082a0 2024-09-19T06:47:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.008+0000 7f9f55b19640 1 -- 172.21.15.103:0/3954431597 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f501a6d90 con 0x7f9f501082a0 2024-09-19T06:47:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.008+0000 7f9f4cff9640 1 -- 172.21.15.103:0/3954431597 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f48005500 con 0x7f9f501082a0 2024-09-19T06:47:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.008+0000 7f9f4cff9640 1 -- 172.21.15.103:0/3954431597 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f48004d90 con 0x7f9f501082a0 2024-09-19T06:47:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.008+0000 7f9f55b19640 1 -- 172.21.15.103:0/3954431597 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f501a72d0 con 0x7f9f501082a0 2024-09-19T06:47:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.009+0000 7f9f55b19640 1 -- 172.21.15.103:0/3954431597 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5006a850 con 0x7f9f501082a0 2024-09-19T06:47:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.010+0000 7f9f4cff9640 1 -- 172.21.15.103:0/3954431597 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9f48010460 con 0x7f9f501082a0 2024-09-19T06:47:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.011+0000 7f9f4cff9640 1 --2- 172.21.15.103:0/3954431597 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9f3c0a3240 0x7f9f3c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.011+0000 7f9f4f7fe640 1 --2- 172.21.15.103:0/3954431597 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9f3c0a3240 0x7f9f3c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.011+0000 7f9f4cff9640 1 -- 172.21.15.103:0/3954431597 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f9f480ced60 con 0x7f9f501082a0 2024-09-19T06:47:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.011+0000 7f9f4f7fe640 1 --2- 172.21.15.103:0/3954431597 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9f3c0a3240 0x7f9f3c0a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9f40006010 tx=0x7f9f40009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.019+0000 7f9f4cff9640 1 -- 172.21.15.103:0/3954431597 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9f4809b8d0 con 0x7f9f501082a0 2024-09-19T06:47:17.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.235+0000 7f9f55b19640 1 -- 172.21.15.103:0/3954431597 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f5010a2b0 con 0x7f9f501082a0 2024-09-19T06:47:17.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.236+0000 7f9f4cff9640 1 -- 172.21.15.103:0/3954431597 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10545 (secure 0 0 0) 0x7f9f4809fbd0 con 0x7f9f501082a0 2024-09-19T06:47:17.237 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:47:17.237 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":13,"btime":"2024-09-19T06:47:14:989595+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-19T06:47:17.238 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 13 2024-09-19T06:47:17.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.238+0000 7f9f55b19640 1 -- 172.21.15.103:0/3954431597 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9f3c0a3240 msgr2=0x7f9f3c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:17.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.238+0000 7f9f55b19640 1 --2- 172.21.15.103:0/3954431597 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9f3c0a3240 0x7f9f3c0a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9f40006010 tx=0x7f9f40009360 comp rx=0 tx=0).stop 2024-09-19T06:47:17.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.238+0000 7f9f55b19640 1 -- 172.21.15.103:0/3954431597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f501082a0 msgr2=0x7f9f50101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:17.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.238+0000 7f9f55b19640 1 --2- 172.21.15.103:0/3954431597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f501082a0 0x7f9f50101410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800d750 tx=0x7f9f4800dc10 comp rx=0 tx=0).stop 2024-09-19T06:47:17.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.238+0000 7f9f55b19640 1 -- 172.21.15.103:0/3954431597 shutdown_connections 2024-09-19T06:47:17.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.238+0000 7f9f55b19640 1 --2- 172.21.15.103:0/3954431597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f501082a0 0x7f9f50101410 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:17.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.238+0000 7f9f55b19640 1 --2- 172.21.15.103:0/3954431597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f501036e0 0x7f9f50100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:17.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.238+0000 7f9f55b19640 1 --2- 172.21.15.103:0/3954431597 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9f3c0a3240 0x7f9f3c0a5700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:17.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.238+0000 7f9f55b19640 1 --2- 172.21.15.103:0/3954431597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f50102d30 0x7f9f50100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:17.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.238+0000 7f9f55b19640 1 -- 172.21.15.103:0/3954431597 >> 172.21.15.103:0/3954431597 conn(0x7f9f500fc7d0 msgr2=0x7f9f50105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:17.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.238+0000 7f9f55b19640 1 -- 172.21.15.103:0/3954431597 shutdown_connections 2024-09-19T06:47:17.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:17.238+0000 7f9f55b19640 1 -- 172.21.15.103:0/3954431597 wait complete. 2024-09-19T06:47:17.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:17 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:17 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:17 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:17 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:17 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:17 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:17 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:17.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:17 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:47:17.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:17 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:17 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:47:17.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:17 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3954431597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:17.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:17 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:17 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:17 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:17 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:17 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:17 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:17 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:17.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:17 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:47:17.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:17 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:17 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:47:17.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:17 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3954431597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:17.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:17 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:17 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:17 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:17 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:17 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:17 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:17 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:17.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:17 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:47:17.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:17 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:17.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:17 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:47:17.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:17 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3954431597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:17.883 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T06:47:18.609 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:18.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:18 smithi134 ceph-mon[35356]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:18.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:18 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:18.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:18 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:18.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:18 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:18.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:18 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:18.733 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:18 smithi103 ceph-mon[31158]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:18.733 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:18 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:18.733 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:18 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:18.733 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:18 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:18.733 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:18 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:18.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:18 smithi174 ceph-mon[35637]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:18.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:18 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:18.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:18 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:18.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:18 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:18.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:18 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.173+0000 7fce9b21e640 1 -- 172.21.15.103:0/2174143900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce94108300 msgr2=0x7fce9410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.174+0000 7fce9b21e640 1 --2- 172.21.15.103:0/2174143900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce94108300 0x7fce9410ec70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009930 tx=0x7fce8c02a420 comp rx=0 tx=0).stop 2024-09-19T06:47:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.174+0000 7fce9b21e640 1 -- 172.21.15.103:0/2174143900 shutdown_connections 2024-09-19T06:47:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.174+0000 7fce9b21e640 1 --2- 172.21.15.103:0/2174143900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce94108300 0x7fce9410ec70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.174+0000 7fce9b21e640 1 --2- 172.21.15.103:0/2174143900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce94103780 0x7fce94107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.174+0000 7fce9b21e640 1 --2- 172.21.15.103:0/2174143900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce94102dd0 0x7fce941031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.174+0000 7fce9b21e640 1 -- 172.21.15.103:0/2174143900 >> 172.21.15.103:0/2174143900 conn(0x7fce940fc7d0 msgr2=0x7fce940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.174+0000 7fce9b21e640 1 -- 172.21.15.103:0/2174143900 shutdown_connections 2024-09-19T06:47:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.174+0000 7fce9b21e640 1 -- 172.21.15.103:0/2174143900 wait complete. 2024-09-19T06:47:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.174+0000 7fce9b21e640 1 Processor -- start 2024-09-19T06:47:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.174+0000 7fce9b21e640 1 -- start start 2024-09-19T06:47:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.174+0000 7fce9b21e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce94102dd0 0x7fce941984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.174+0000 7fce9b21e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce94103780 0x7fce941989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.174+0000 7fce9b21e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce94108300 0x7fce9419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.174+0000 7fce9b21e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fce941120d0 con 0x7fce94102dd0 2024-09-19T06:47:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.174+0000 7fce9b21e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce94111f50 con 0x7fce94108300 2024-09-19T06:47:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.174+0000 7fce9b21e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fce94112250 con 0x7fce94103780 2024-09-19T06:47:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce98fba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce94102dd0 0x7fce941984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce98fba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce94102dd0 0x7fce941984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46440/0 (socket says 172.21.15.103:46440) 2024-09-19T06:47:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce98fba640 1 -- 172.21.15.103:0/1602468536 learned_addr learned my addr 172.21.15.103:0/1602468536 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce997bb640 1 --2- 172.21.15.103:0/1602468536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce94108300 0x7fce9419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce93fff640 1 --2- 172.21.15.103:0/1602468536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce94103780 0x7fce941989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce98fba640 1 -- 172.21.15.103:0/1602468536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce94103780 msgr2=0x7fce941989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce98fba640 1 --2- 172.21.15.103:0/1602468536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce94103780 0x7fce941989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce98fba640 1 -- 172.21.15.103:0/1602468536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce94108300 msgr2=0x7fce9419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce98fba640 1 --2- 172.21.15.103:0/1602468536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce94108300 0x7fce9419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce98fba640 1 -- 172.21.15.103:0/1602468536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9419d460 con 0x7fce94102dd0 2024-09-19T06:47:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce93fff640 1 --2- 172.21.15.103:0/1602468536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce94103780 0x7fce941989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce997bb640 1 --2- 172.21.15.103:0/1602468536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce94108300 0x7fce9419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce98fba640 1 --2- 172.21.15.103:0/1602468536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce94102dd0 0x7fce941984b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fce8400cce0 tx=0x7fce8400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce91ffb640 1 -- 172.21.15.103:0/1602468536 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8400f650 con 0x7fce94102dd0 2024-09-19T06:47:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce9b21e640 1 -- 172.21.15.103:0/1602468536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9419d750 con 0x7fce94102dd0 2024-09-19T06:47:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce91ffb640 1 -- 172.21.15.103:0/1602468536 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce8400fcd0 con 0x7fce94102dd0 2024-09-19T06:47:19.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce91ffb640 1 -- 172.21.15.103:0/1602468536 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84018c70 con 0x7fce94102dd0 2024-09-19T06:47:19.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.175+0000 7fce9b21e640 1 -- 172.21.15.103:0/1602468536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce941a5030 con 0x7fce94102dd0 2024-09-19T06:47:19.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.176+0000 7fce9b21e640 1 -- 172.21.15.103:0/1602468536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce941031b0 con 0x7fce94102dd0 2024-09-19T06:47:19.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.177+0000 7fce91ffb640 1 -- 172.21.15.103:0/1602468536 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fce8400f7f0 con 0x7fce94102dd0 2024-09-19T06:47:19.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.178+0000 7fce91ffb640 1 --2- 172.21.15.103:0/1602468536 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fce800a3240 0x7fce800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:19.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.178+0000 7fce93fff640 1 --2- 172.21.15.103:0/1602468536 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fce800a3240 0x7fce800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:19.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.178+0000 7fce91ffb640 1 -- 172.21.15.103:0/1602468536 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fce840cf9d0 con 0x7fce94102dd0 2024-09-19T06:47:19.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.178+0000 7fce93fff640 1 --2- 172.21.15.103:0/1602468536 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fce800a3240 0x7fce800a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fce940fedb0 tx=0x7fce7c0094b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:19.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.182+0000 7fce91ffb640 1 -- 172.21.15.103:0/1602468536 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fce84014030 con 0x7fce94102dd0 2024-09-19T06:47:19.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.358+0000 7fce9b21e640 1 -- 172.21.15.103:0/1602468536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fce94199810 con 0x7fce94102dd0 2024-09-19T06:47:19.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.358+0000 7fce91ffb640 1 -- 172.21.15.103:0/1602468536 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) ==== 84+0+65 (secure 0 0 0) 0x7fce8409cb90 con 0x7fce94102dd0 2024-09-19T06:47:19.360 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:47:19.360 INFO:teuthology.orchestra.run.smithi103.stdout:[ 2024-09-19T06:47:19.360 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T06:47:19.360 INFO:teuthology.orchestra.run.smithi103.stdout: "poolnum": 1, 2024-09-19T06:47:19.361 INFO:teuthology.orchestra.run.smithi103.stdout: "poolname": ".mgr" 2024-09-19T06:47:19.361 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:47:19.361 INFO:teuthology.orchestra.run.smithi103.stdout:] 2024-09-19T06:47:19.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.361+0000 7fce9b21e640 1 -- 172.21.15.103:0/1602468536 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fce800a3240 msgr2=0x7fce800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:19.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.361+0000 7fce9b21e640 1 --2- 172.21.15.103:0/1602468536 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fce800a3240 0x7fce800a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fce940fedb0 tx=0x7fce7c0094b0 comp rx=0 tx=0).stop 2024-09-19T06:47:19.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.361+0000 7fce9b21e640 1 -- 172.21.15.103:0/1602468536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce94102dd0 msgr2=0x7fce941984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:19.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.361+0000 7fce9b21e640 1 --2- 172.21.15.103:0/1602468536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce94102dd0 0x7fce941984b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fce8400cce0 tx=0x7fce8400a620 comp rx=0 tx=0).stop 2024-09-19T06:47:19.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.361+0000 7fce9b21e640 1 -- 172.21.15.103:0/1602468536 shutdown_connections 2024-09-19T06:47:19.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.361+0000 7fce9b21e640 1 --2- 172.21.15.103:0/1602468536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce94108300 0x7fce9419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:19.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.361+0000 7fce9b21e640 1 --2- 172.21.15.103:0/1602468536 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fce800a3240 0x7fce800a5700 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:19.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.361+0000 7fce9b21e640 1 --2- 172.21.15.103:0/1602468536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce94103780 0x7fce941989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:19.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.361+0000 7fce9b21e640 1 --2- 172.21.15.103:0/1602468536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce94102dd0 0x7fce941984b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:19.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.361+0000 7fce9b21e640 1 -- 172.21.15.103:0/1602468536 >> 172.21.15.103:0/1602468536 conn(0x7fce940fc7d0 msgr2=0x7fce94105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:19.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.361+0000 7fce9b21e640 1 -- 172.21.15.103:0/1602468536 shutdown_connections 2024-09-19T06:47:19.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:19.361+0000 7fce9b21e640 1 -- 172.21.15.103:0/1602468536 wait complete. 2024-09-19T06:47:19.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:19 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1602468536' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T06:47:19.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:19 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1602468536' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T06:47:19.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:19 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1602468536' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T06:47:20.010 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T06:47:20.011 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T06:47:20.269 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:20.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.778+0000 7f3ab8fca640 1 -- 172.21.15.103:0/435054411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4102ef0 msgr2=0x7f3ab410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:20.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.778+0000 7f3ab8fca640 1 --2- 172.21.15.103:0/435054411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4102ef0 0x7f3ab410da40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009ca0 tx=0x7f3aac02a950 comp rx=0 tx=0).stop 2024-09-19T06:47:20.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.779+0000 7f3ab8fca640 1 -- 172.21.15.103:0/435054411 shutdown_connections 2024-09-19T06:47:20.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.779+0000 7f3ab8fca640 1 --2- 172.21.15.103:0/435054411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4102ef0 0x7f3ab410da40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:20.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.779+0000 7f3ab8fca640 1 --2- 172.21.15.103:0/435054411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ab4102570 0x7f3ab41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:20.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.779+0000 7f3ab8fca640 1 --2- 172.21.15.103:0/435054411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab4100e20 0x7f3ab4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:20.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.779+0000 7f3ab8fca640 1 -- 172.21.15.103:0/435054411 >> 172.21.15.103:0/435054411 conn(0x7f3ab40fc7d0 msgr2=0x7f3ab40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:20.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.779+0000 7f3ab8fca640 1 -- 172.21.15.103:0/435054411 shutdown_connections 2024-09-19T06:47:20.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.779+0000 7f3ab8fca640 1 -- 172.21.15.103:0/435054411 wait complete. 2024-09-19T06:47:20.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.779+0000 7f3ab8fca640 1 Processor -- start 2024-09-19T06:47:20.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab8fca640 1 -- start start 2024-09-19T06:47:20.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab8fca640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4100e20 0x7f3ab4197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:20.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab8fca640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab4102570 0x7f3ab4197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:20.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab8fca640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ab4102ef0 0x7f3ab419be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:20.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab8fca640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4111020 con 0x7f3ab4100e20 2024-09-19T06:47:20.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab8fca640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4110ea0 con 0x7f3ab4102570 2024-09-19T06:47:20.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab8fca640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41111a0 con 0x7f3ab4102ef0 2024-09-19T06:47:20.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4100e20 0x7f3ab4197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:20.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4100e20 0x7f3ab4197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46460/0 (socket says 172.21.15.103:46460) 2024-09-19T06:47:20.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab259c640 1 -- 172.21.15.103:0/3426476618 learned_addr learned my addr 172.21.15.103:0/3426476618 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:20.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3aabfff640 1 --2- 172.21.15.103:0/3426476618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab4102570 0x7f3ab4197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:20.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab259c640 1 -- 172.21.15.103:0/3426476618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ab4102ef0 msgr2=0x7f3ab419be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:20.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab2d9d640 1 --2- 172.21.15.103:0/3426476618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ab4102ef0 0x7f3ab419be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:20.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab259c640 1 --2- 172.21.15.103:0/3426476618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ab4102ef0 0x7f3ab419be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:20.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab259c640 1 -- 172.21.15.103:0/3426476618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab4102570 msgr2=0x7f3ab4197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:20.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab259c640 1 --2- 172.21.15.103:0/3426476618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab4102570 0x7f3ab4197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:20.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab259c640 1 -- 172.21.15.103:0/3426476618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab419c4f0 con 0x7f3ab4100e20 2024-09-19T06:47:20.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.780+0000 7f3ab2d9d640 1 --2- 172.21.15.103:0/3426476618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ab4102ef0 0x7f3ab419be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:20.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.781+0000 7f3ab259c640 1 --2- 172.21.15.103:0/3426476618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4100e20 0x7f3ab4197540 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400b4a0 tx=0x7f3aa400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:20.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.781+0000 7f3aab7fe640 1 -- 172.21.15.103:0/3426476618 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4014070 con 0x7f3ab4100e20 2024-09-19T06:47:20.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.781+0000 7f3aab7fe640 1 -- 172.21.15.103:0/3426476618 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa4010af0 con 0x7f3ab4100e20 2024-09-19T06:47:20.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.781+0000 7f3ab8fca640 1 -- 172.21.15.103:0/3426476618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab419c7e0 con 0x7f3ab4100e20 2024-09-19T06:47:20.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.781+0000 7f3aab7fe640 1 -- 172.21.15.103:0/3426476618 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4017d10 con 0x7f3ab4100e20 2024-09-19T06:47:20.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.781+0000 7f3ab8fca640 1 -- 172.21.15.103:0/3426476618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab41a40c0 con 0x7f3ab4100e20 2024-09-19T06:47:20.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.782+0000 7f3ab8fca640 1 -- 172.21.15.103:0/3426476618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab4101280 con 0x7f3ab4100e20 2024-09-19T06:47:20.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.783+0000 7f3aab7fe640 1 -- 172.21.15.103:0/3426476618 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3aa4017070 con 0x7f3ab4100e20 2024-09-19T06:47:20.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.784+0000 7f3aab7fe640 1 --2- 172.21.15.103:0/3426476618 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3aa00a32c0 0x7f3aa00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:20.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.784+0000 7f3aabfff640 1 --2- 172.21.15.103:0/3426476618 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3aa00a32c0 0x7f3aa00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:20.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.784+0000 7f3aab7fe640 1 -- 172.21.15.103:0/3426476618 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f3aa40cec30 con 0x7f3ab4100e20 2024-09-19T06:47:20.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.784+0000 7f3aabfff640 1 --2- 172.21.15.103:0/3426476618 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3aa00a32c0 0x7f3aa00a5780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40ffe30 tx=0x7f3a9c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:20.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.787+0000 7f3aab7fe640 1 -- 172.21.15.103:0/3426476618 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3aa409bdf0 con 0x7f3ab4100e20 2024-09-19T06:47:20.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:20.956+0000 7f3ab8fca640 1 -- 172.21.15.103:0/3426476618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3ab4109090 con 0x7f3ab4100e20 2024-09-19T06:47:20.958 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:20 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:20.958 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:20 smithi103 ceph-mon[31158]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:20.958 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:20 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:20.958 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:20 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:20.958 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:20 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:20.958 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:20 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:47:20.958 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:20 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:21.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:20 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:21.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:20 smithi174 ceph-mon[35637]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:21.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:20 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:21.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:20 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:21.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:20 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:21.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:20 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:47:21.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:20 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:20 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:20 smithi134 ceph-mon[35356]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:20 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:20 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:20 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:20 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:47:21.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:20 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:47:21.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:21.810+0000 7f3aab7fe640 1 -- 172.21.15.103:0/3426476618 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) ==== 138+0+0 (secure 0 0 0) 0x7f3aa409b6f0 con 0x7f3ab4100e20 2024-09-19T06:47:21.811 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_metadata' created 2024-09-19T06:47:21.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:21.813+0000 7f3ab8fca640 1 -- 172.21.15.103:0/3426476618 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3aa00a32c0 msgr2=0x7f3aa00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:21.813+0000 7f3ab8fca640 1 --2- 172.21.15.103:0/3426476618 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3aa00a32c0 0x7f3aa00a5780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40ffe30 tx=0x7f3a9c008040 comp rx=0 tx=0).stop 2024-09-19T06:47:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:21.813+0000 7f3ab8fca640 1 -- 172.21.15.103:0/3426476618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4100e20 msgr2=0x7f3ab4197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:21.813+0000 7f3ab8fca640 1 --2- 172.21.15.103:0/3426476618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4100e20 0x7f3ab4197540 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400b4a0 tx=0x7f3aa400b960 comp rx=0 tx=0).stop 2024-09-19T06:47:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:21.813+0000 7f3ab8fca640 1 -- 172.21.15.103:0/3426476618 shutdown_connections 2024-09-19T06:47:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:21.813+0000 7f3ab8fca640 1 --2- 172.21.15.103:0/3426476618 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3aa00a32c0 0x7f3aa00a5780 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:21.813+0000 7f3ab8fca640 1 --2- 172.21.15.103:0/3426476618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ab4102ef0 0x7f3ab419be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:21.813+0000 7f3ab8fca640 1 --2- 172.21.15.103:0/3426476618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab4102570 0x7f3ab4197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:21.813+0000 7f3ab8fca640 1 --2- 172.21.15.103:0/3426476618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4100e20 0x7f3ab4197540 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:21.813+0000 7f3ab8fca640 1 -- 172.21.15.103:0/3426476618 >> 172.21.15.103:0/3426476618 conn(0x7f3ab40fc7d0 msgr2=0x7f3ab40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:21.813+0000 7f3ab8fca640 1 -- 172.21.15.103:0/3426476618 shutdown_connections 2024-09-19T06:47:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:21.814+0000 7f3ab8fca640 1 -- 172.21.15.103:0/3426476618 wait complete. 2024-09-19T06:47:22.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:21 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3426476618' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T06:47:22.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:21 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3426476618' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T06:47:22.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:21 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3426476618' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T06:47:22.496 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T06:47:22.744 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:23.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:22 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3426476618' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T06:47:23.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:22 smithi174 ceph-mon[35637]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T06:47:23.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:22 smithi174 ceph-mon[35637]: pgmap v267: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:23.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:22 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3426476618' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T06:47:23.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:22 smithi103 ceph-mon[31158]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T06:47:23.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:22 smithi103 ceph-mon[31158]: pgmap v267: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:23.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:22 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3426476618' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T06:47:23.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:22 smithi134 ceph-mon[35356]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T06:47:23.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:22 smithi134 ceph-mon[35356]: pgmap v267: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:23.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.235+0000 7f54577f0640 1 -- 172.21.15.103:0/3304519772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5450106a80 msgr2=0x7f5450106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:23.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.235+0000 7f54577f0640 1 --2- 172.21.15.103:0/3304519772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5450106a80 0x7f5450106e60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f544c009930 tx=0x7f544c02a420 comp rx=0 tx=0).stop 2024-09-19T06:47:23.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.235+0000 7f54577f0640 1 -- 172.21.15.103:0/3304519772 shutdown_connections 2024-09-19T06:47:23.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.235+0000 7f54577f0640 1 --2- 172.21.15.103:0/3304519772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5450101420 0x7f545010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:23.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.235+0000 7f54577f0640 1 --2- 172.21.15.103:0/3304519772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5450100a80 0x7f5450100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:23.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.235+0000 7f54577f0640 1 --2- 172.21.15.103:0/3304519772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5450106a80 0x7f5450106e60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:23.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.235+0000 7f54577f0640 1 -- 172.21.15.103:0/3304519772 >> 172.21.15.103:0/3304519772 conn(0x7f54500fc7a0 msgr2=0x7f54500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:23.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.235+0000 7f54577f0640 1 -- 172.21.15.103:0/3304519772 shutdown_connections 2024-09-19T06:47:23.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.235+0000 7f54577f0640 1 -- 172.21.15.103:0/3304519772 wait complete. 2024-09-19T06:47:23.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.236+0000 7f54577f0640 1 Processor -- start 2024-09-19T06:47:23.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.236+0000 7f54577f0640 1 -- start start 2024-09-19T06:47:23.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.236+0000 7f54577f0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5450100a80 0x7f54501973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:23.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.236+0000 7f54577f0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5450101420 0x7f54501978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:23.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.236+0000 7f54577f0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5450106a80 0x7f545019bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:23.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.236+0000 7f54577f0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5450110e80 con 0x7f5450106a80 2024-09-19T06:47:23.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.236+0000 7f54577f0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5450110d00 con 0x7f5450101420 2024-09-19T06:47:23.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.236+0000 7f54577f0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5450111000 con 0x7f5450100a80 2024-09-19T06:47:23.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.236+0000 7f5455d8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5450106a80 0x7f545019bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:23.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.236+0000 7f5455d8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5450106a80 0x7f545019bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46492/0 (socket says 172.21.15.103:46492) 2024-09-19T06:47:23.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.236+0000 7f5455d8d640 1 -- 172.21.15.103:0/4245736586 learned_addr learned my addr 172.21.15.103:0/4245736586 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:23.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.236+0000 7f5454d8b640 1 --2- 172.21.15.103:0/4245736586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5450101420 0x7f54501978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:23.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.236+0000 7f545558c640 1 --2- 172.21.15.103:0/4245736586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5450100a80 0x7f54501973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:23.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.237+0000 7f5455d8d640 1 -- 172.21.15.103:0/4245736586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5450100a80 msgr2=0x7f54501973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:23.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.237+0000 7f5455d8d640 1 --2- 172.21.15.103:0/4245736586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5450100a80 0x7f54501973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:23.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.237+0000 7f5455d8d640 1 -- 172.21.15.103:0/4245736586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5450101420 msgr2=0x7f54501978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:23.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.237+0000 7f5455d8d640 1 --2- 172.21.15.103:0/4245736586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5450101420 0x7f54501978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:23.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.237+0000 7f5455d8d640 1 -- 172.21.15.103:0/4245736586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545019c360 con 0x7f5450106a80 2024-09-19T06:47:23.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.237+0000 7f5455d8d640 1 --2- 172.21.15.103:0/4245736586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5450106a80 0x7f545019bc80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f544800eaa0 tx=0x7f544800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:23.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.237+0000 7f54467fc640 1 -- 172.21.15.103:0/4245736586 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544800cdb0 con 0x7f5450106a80 2024-09-19T06:47:23.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.237+0000 7f54577f0640 1 -- 172.21.15.103:0/4245736586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545019c650 con 0x7f5450106a80 2024-09-19T06:47:23.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.237+0000 7f54467fc640 1 -- 172.21.15.103:0/4245736586 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5448005020 con 0x7f5450106a80 2024-09-19T06:47:23.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.237+0000 7f54467fc640 1 -- 172.21.15.103:0/4245736586 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54480116b0 con 0x7f5450106a80 2024-09-19T06:47:23.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.237+0000 7f54577f0640 1 -- 172.21.15.103:0/4245736586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54501a3f30 con 0x7f5450106a80 2024-09-19T06:47:23.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.238+0000 7f54577f0640 1 -- 172.21.15.103:0/4245736586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5450100ee0 con 0x7f5450106a80 2024-09-19T06:47:23.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.239+0000 7f54467fc640 1 -- 172.21.15.103:0/4245736586 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f5448005560 con 0x7f5450106a80 2024-09-19T06:47:23.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.240+0000 7f54467fc640 1 --2- 172.21.15.103:0/4245736586 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f543c0a3240 0x7f543c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:23.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.240+0000 7f545558c640 1 --2- 172.21.15.103:0/4245736586 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f543c0a3240 0x7f543c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:23.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.240+0000 7f54467fc640 1 -- 172.21.15.103:0/4245736586 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7497+0+0 (secure 0 0 0) 0x7f54480cfb90 con 0x7f5450106a80 2024-09-19T06:47:23.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.240+0000 7f545558c640 1 --2- 172.21.15.103:0/4245736586 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f543c0a3240 0x7f543c0a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f544c009900 tx=0x7f544c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:23.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.243+0000 7f54467fc640 1 -- 172.21.15.103:0/4245736586 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5448016030 con 0x7f5450106a80 2024-09-19T06:47:23.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.409+0000 7f54577f0640 1 -- 172.21.15.103:0/4245736586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5450108a60 con 0x7f5450106a80 2024-09-19T06:47:23.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.820+0000 7f54467fc640 1 -- 172.21.15.103:0/4245736586 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) ==== 170+0+0 (secure 0 0 0) 0x7f544809c4f0 con 0x7f5450106a80 2024-09-19T06:47:23.821 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_data' created 2024-09-19T06:47:23.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.822+0000 7f54577f0640 1 -- 172.21.15.103:0/4245736586 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f543c0a3240 msgr2=0x7f543c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:23.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.822+0000 7f54577f0640 1 --2- 172.21.15.103:0/4245736586 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f543c0a3240 0x7f543c0a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f544c009900 tx=0x7f544c0058d0 comp rx=0 tx=0).stop 2024-09-19T06:47:23.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.822+0000 7f54577f0640 1 -- 172.21.15.103:0/4245736586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5450106a80 msgr2=0x7f545019bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:23.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.822+0000 7f54577f0640 1 --2- 172.21.15.103:0/4245736586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5450106a80 0x7f545019bc80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f544800eaa0 tx=0x7f544800ef60 comp rx=0 tx=0).stop 2024-09-19T06:47:23.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.822+0000 7f54577f0640 1 -- 172.21.15.103:0/4245736586 shutdown_connections 2024-09-19T06:47:23.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.822+0000 7f54577f0640 1 --2- 172.21.15.103:0/4245736586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5450106a80 0x7f545019bc80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:23.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.822+0000 7f54577f0640 1 --2- 172.21.15.103:0/4245736586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5450101420 0x7f54501978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:23.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.822+0000 7f54577f0640 1 --2- 172.21.15.103:0/4245736586 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f543c0a3240 0x7f543c0a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:23.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.822+0000 7f54577f0640 1 --2- 172.21.15.103:0/4245736586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5450100a80 0x7f54501973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:23.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.822+0000 7f54577f0640 1 -- 172.21.15.103:0/4245736586 >> 172.21.15.103:0/4245736586 conn(0x7f54500fc7a0 msgr2=0x7f54500fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:23.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.822+0000 7f54577f0640 1 -- 172.21.15.103:0/4245736586 shutdown_connections 2024-09-19T06:47:23.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:23.822+0000 7f54577f0640 1 -- 172.21.15.103:0/4245736586 wait complete. 2024-09-19T06:47:24.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:23 smithi174 ceph-mon[35637]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T06:47:24.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:23 smithi174 ceph-mon[35637]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T06:47:24.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:23 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/4245736586' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T06:47:24.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:23 smithi103 ceph-mon[31158]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T06:47:24.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:23 smithi103 ceph-mon[31158]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T06:47:24.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:23 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/4245736586' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T06:47:24.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:23 smithi134 ceph-mon[35356]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T06:47:24.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:23 smithi134 ceph-mon[35356]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T06:47:24.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:23 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/4245736586' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T06:47:24.536 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T06:47:24.795 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:25.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:24 smithi174 ceph-mon[35637]: pgmap v269: 33 pgs: 7 creating+peering, 25 unknown, 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:25.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:24 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/4245736586' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T06:47:25.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:24 smithi174 ceph-mon[35637]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T06:47:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:24 smithi103 ceph-mon[31158]: pgmap v269: 33 pgs: 7 creating+peering, 25 unknown, 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:24 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/4245736586' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T06:47:25.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:24 smithi103 ceph-mon[31158]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T06:47:25.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:24 smithi134 ceph-mon[35356]: pgmap v269: 33 pgs: 7 creating+peering, 25 unknown, 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:25.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:24 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/4245736586' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T06:47:25.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:24 smithi134 ceph-mon[35356]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T06:47:25.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.236+0000 7fb0dbec2640 1 -- 172.21.15.103:0/1829799780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0d41008d0 msgr2=0x7fb0d4100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:25.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.236+0000 7fb0dbec2640 1 --2- 172.21.15.103:0/1829799780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0d41008d0 0x7fb0d4100cb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009900 tx=0x7fb0cc02a410 comp rx=0 tx=0).stop 2024-09-19T06:47:25.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.236+0000 7fb0dbec2640 1 -- 172.21.15.103:0/1829799780 shutdown_connections 2024-09-19T06:47:25.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.236+0000 7fb0dbec2640 1 --2- 172.21.15.103:0/1829799780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d410e480 0x7fb0d4110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:25.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.236+0000 7fb0dbec2640 1 --2- 172.21.15.103:0/1829799780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d4101280 0x7fb0d410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:25.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.236+0000 7fb0dbec2640 1 --2- 172.21.15.103:0/1829799780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0d41008d0 0x7fb0d4100cb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:25.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.236+0000 7fb0dbec2640 1 -- 172.21.15.103:0/1829799780 >> 172.21.15.103:0/1829799780 conn(0x7fb0d40fc7a0 msgr2=0x7fb0d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:25.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.236+0000 7fb0dbec2640 1 -- 172.21.15.103:0/1829799780 shutdown_connections 2024-09-19T06:47:25.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.236+0000 7fb0dbec2640 1 -- 172.21.15.103:0/1829799780 wait complete. 2024-09-19T06:47:25.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0dbec2640 1 Processor -- start 2024-09-19T06:47:25.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0dbec2640 1 -- start start 2024-09-19T06:47:25.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0dbec2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d41008d0 0x7fb0d41a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:25.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0dbec2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0d4101280 0x7fb0d41a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:25.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0dbec2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d410e480 0x7fb0d419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:25.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0dbec2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4114260 con 0x7fb0d4101280 2024-09-19T06:47:25.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0dbec2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41140e0 con 0x7fb0d41008d0 2024-09-19T06:47:25.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0dbec2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41143e0 con 0x7fb0d410e480 2024-09-19T06:47:25.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0d945d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0d4101280 0x7fb0d41a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:25.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0d945d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0d4101280 0x7fb0d41a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46498/0 (socket says 172.21.15.103:46498) 2024-09-19T06:47:25.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0d945d640 1 -- 172.21.15.103:0/112710573 learned_addr learned my addr 172.21.15.103:0/112710573 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:25.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0da45f640 1 --2- 172.21.15.103:0/112710573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d410e480 0x7fb0d419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:25.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0d945d640 1 -- 172.21.15.103:0/112710573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d410e480 msgr2=0x7fb0d419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:25.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0d945d640 1 --2- 172.21.15.103:0/112710573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d410e480 0x7fb0d419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:25.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0d945d640 1 -- 172.21.15.103:0/112710573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d41008d0 msgr2=0x7fb0d41a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:25.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0d9c5e640 1 --2- 172.21.15.103:0/112710573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d41008d0 0x7fb0d41a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:25.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0d945d640 1 --2- 172.21.15.103:0/112710573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d41008d0 0x7fb0d41a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:25.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.237+0000 7fb0d945d640 1 -- 172.21.15.103:0/112710573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d419aee0 con 0x7fb0d4101280 2024-09-19T06:47:25.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.238+0000 7fb0da45f640 1 --2- 172.21.15.103:0/112710573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d410e480 0x7fb0d419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:25.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.238+0000 7fb0d945d640 1 --2- 172.21.15.103:0/112710573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0d4101280 0x7fb0d41a0ae0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800b4a0 tx=0x7fb0c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:25.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.238+0000 7fb0d2ffd640 1 -- 172.21.15.103:0/112710573 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8014070 con 0x7fb0d4101280 2024-09-19T06:47:25.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.238+0000 7fb0d2ffd640 1 -- 172.21.15.103:0/112710573 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0c8010af0 con 0x7fb0d4101280 2024-09-19T06:47:25.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.238+0000 7fb0dbec2640 1 -- 172.21.15.103:0/112710573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d419b1d0 con 0x7fb0d4101280 2024-09-19T06:47:25.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.238+0000 7fb0d2ffd640 1 -- 172.21.15.103:0/112710573 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8017d10 con 0x7fb0d4101280 2024-09-19T06:47:25.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.238+0000 7fb0dbec2640 1 -- 172.21.15.103:0/112710573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d41a7520 con 0x7fb0d4101280 2024-09-19T06:47:25.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.239+0000 7fb0dbec2640 1 -- 172.21.15.103:0/112710573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d406b070 con 0x7fb0d4101280 2024-09-19T06:47:25.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.240+0000 7fb0d2ffd640 1 -- 172.21.15.103:0/112710573 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb0c8017070 con 0x7fb0d4101280 2024-09-19T06:47:25.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.241+0000 7fb0d2ffd640 1 --2- 172.21.15.103:0/112710573 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb0bc0a32c0 0x7fb0bc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:25.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.241+0000 7fb0d9c5e640 1 --2- 172.21.15.103:0/112710573 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb0bc0a32c0 0x7fb0bc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:25.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.241+0000 7fb0d2ffd640 1 -- 172.21.15.103:0/112710573 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7fb0c80cefc0 con 0x7fb0d4101280 2024-09-19T06:47:25.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.241+0000 7fb0d9c5e640 1 --2- 172.21.15.103:0/112710573 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb0bc0a32c0 0x7fb0bc0a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0096c0 tx=0x7fb0cc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:25.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.245+0000 7fb0d2ffd640 1 -- 172.21.15.103:0/112710573 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb0c809bdf0 con 0x7fb0d4101280 2024-09-19T06:47:25.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.444+0000 7fb0dbec2640 1 -- 172.21.15.103:0/112710573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb0d4109610 con 0x7fb0d4101280 2024-09-19T06:47:25.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.836+0000 7fb0d2ffd640 1 -- 172.21.15.103:0/112710573 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fb0c809b6f0 con 0x7fb0d4101280 2024-09-19T06:47:25.838 INFO:teuthology.orchestra.run.smithi103.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T06:47:25.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.839+0000 7fb0dbec2640 1 -- 172.21.15.103:0/112710573 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb0bc0a32c0 msgr2=0x7fb0bc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:25.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.839+0000 7fb0dbec2640 1 --2- 172.21.15.103:0/112710573 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb0bc0a32c0 0x7fb0bc0a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0096c0 tx=0x7fb0cc000bc0 comp rx=0 tx=0).stop 2024-09-19T06:47:25.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.839+0000 7fb0dbec2640 1 -- 172.21.15.103:0/112710573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0d4101280 msgr2=0x7fb0d41a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:25.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.839+0000 7fb0dbec2640 1 --2- 172.21.15.103:0/112710573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0d4101280 0x7fb0d41a0ae0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800b4a0 tx=0x7fb0c800b960 comp rx=0 tx=0).stop 2024-09-19T06:47:25.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.839+0000 7fb0dbec2640 1 -- 172.21.15.103:0/112710573 shutdown_connections 2024-09-19T06:47:25.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.839+0000 7fb0dbec2640 1 --2- 172.21.15.103:0/112710573 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb0bc0a32c0 0x7fb0bc0a5780 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:25.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.839+0000 7fb0dbec2640 1 --2- 172.21.15.103:0/112710573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d410e480 0x7fb0d419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:25.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.839+0000 7fb0dbec2640 1 --2- 172.21.15.103:0/112710573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0d4101280 0x7fb0d41a0ae0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:25.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.839+0000 7fb0dbec2640 1 --2- 172.21.15.103:0/112710573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d41008d0 0x7fb0d41a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:25.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.839+0000 7fb0dbec2640 1 -- 172.21.15.103:0/112710573 >> 172.21.15.103:0/112710573 conn(0x7fb0d40fc7a0 msgr2=0x7fb0d40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:25.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.839+0000 7fb0dbec2640 1 -- 172.21.15.103:0/112710573 shutdown_connections 2024-09-19T06:47:25.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:25.839+0000 7fb0dbec2640 1 -- 172.21.15.103:0/112710573 wait complete. 2024-09-19T06:47:26.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:25 smithi103 ceph-mon[31158]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T06:47:26.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:25 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/112710573' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T06:47:26.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:25 smithi103 ceph-mon[31158]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T06:47:26.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:25 smithi134 ceph-mon[35356]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T06:47:26.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:25 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/112710573' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T06:47:26.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:25 smithi134 ceph-mon[35356]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T06:47:26.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:25 smithi174 ceph-mon[35637]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T06:47:26.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:25 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/112710573' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T06:47:26.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:25 smithi174 ceph-mon[35637]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T06:47:26.487 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:47:26.737 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:27.096 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:26 smithi103 ceph-mon[31158]: pgmap v272: 97 pgs: 7 creating+peering, 62 unknown, 28 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:27.096 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:26 smithi103 ceph-mon[31158]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T06:47:27.096 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:26 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/112710573' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T06:47:27.096 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:26 smithi103 ceph-mon[31158]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T06:47:27.096 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:26 smithi103 ceph-mon[31158]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T06:47:27.096 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:26 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] up:active 2024-09-19T06:47:27.096 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:26 smithi103 ceph-mon[31158]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:47:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:26 smithi134 ceph-mon[35356]: pgmap v272: 97 pgs: 7 creating+peering, 62 unknown, 28 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:26 smithi134 ceph-mon[35356]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T06:47:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:26 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/112710573' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T06:47:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:26 smithi134 ceph-mon[35356]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T06:47:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:26 smithi134 ceph-mon[35356]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T06:47:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:26 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] up:active 2024-09-19T06:47:27.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:26 smithi134 ceph-mon[35356]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:47:27.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.174+0000 7f610062b640 1 -- 172.21.15.103:0/1160496344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60f8102570 msgr2=0x7f60f81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:27.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.174+0000 7f610062b640 1 --2- 172.21.15.103:0/1160496344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60f8102570 0x7f60f81029b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009900 tx=0x7f60e802a480 comp rx=0 tx=0).stop 2024-09-19T06:47:27.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.174+0000 7f610062b640 1 -- 172.21.15.103:0/1160496344 shutdown_connections 2024-09-19T06:47:27.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.174+0000 7f610062b640 1 --2- 172.21.15.103:0/1160496344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f8102ef0 0x7f60f810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:27.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.174+0000 7f610062b640 1 --2- 172.21.15.103:0/1160496344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60f8102570 0x7f60f81029b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:27.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.174+0000 7f610062b640 1 --2- 172.21.15.103:0/1160496344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f8100e20 0x7f60f8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:27.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.174+0000 7f610062b640 1 -- 172.21.15.103:0/1160496344 >> 172.21.15.103:0/1160496344 conn(0x7f60f80fc7d0 msgr2=0x7f60f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:27.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.174+0000 7f610062b640 1 -- 172.21.15.103:0/1160496344 shutdown_connections 2024-09-19T06:47:27.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.174+0000 7f610062b640 1 -- 172.21.15.103:0/1160496344 wait complete. 2024-09-19T06:47:27.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.174+0000 7f610062b640 1 Processor -- start 2024-09-19T06:47:27.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.174+0000 7f610062b640 1 -- start start 2024-09-19T06:47:27.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f610062b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f8100e20 0x7f60f819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:27.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f610062b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60f8102570 0x7f60f819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:27.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f610062b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f8102ef0 0x7f60f819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:27.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f610062b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f60f8110f80 con 0x7f60f8102570 2024-09-19T06:47:27.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f610062b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60f8110e00 con 0x7f60f8102ef0 2024-09-19T06:47:27.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f610062b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60f8111100 con 0x7f60f8100e20 2024-09-19T06:47:27.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f60fdbc6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60f8102570 0x7f60f819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:27.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f60fdbc6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60f8102570 0x7f60f819ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46508/0 (socket says 172.21.15.103:46508) 2024-09-19T06:47:27.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f60fdbc6640 1 -- 172.21.15.103:0/865580805 learned_addr learned my addr 172.21.15.103:0/865580805 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:27.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f60fe3c7640 1 --2- 172.21.15.103:0/865580805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f8100e20 0x7f60f819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:27.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f60fdbc6640 1 -- 172.21.15.103:0/865580805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f8100e20 msgr2=0x7f60f819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:27.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f60fdbc6640 1 --2- 172.21.15.103:0/865580805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f8100e20 0x7f60f819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:27.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f60fdbc6640 1 -- 172.21.15.103:0/865580805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f8102ef0 msgr2=0x7f60f819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:27.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f60febc8640 1 --2- 172.21.15.103:0/865580805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f8102ef0 0x7f60f819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:27.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f60fdbc6640 1 --2- 172.21.15.103:0/865580805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f8102ef0 0x7f60f819efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:27.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f60fdbc6640 1 -- 172.21.15.103:0/865580805 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f819f6b0 con 0x7f60f8102570 2024-09-19T06:47:27.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f60fe3c7640 1 --2- 172.21.15.103:0/865580805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f8100e20 0x7f60f819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:27.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f60fdbc6640 1 --2- 172.21.15.103:0/865580805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60f8102570 0x7f60f819ac40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f60e8006010 tx=0x7f60e8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:27.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f60ef7fe640 1 -- 172.21.15.103:0/865580805 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e803f070 con 0x7f60f8102570 2024-09-19T06:47:27.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f60ef7fe640 1 -- 172.21.15.103:0/865580805 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e8031e30 con 0x7f60f8102570 2024-09-19T06:47:27.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f610062b640 1 -- 172.21.15.103:0/865580805 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f819f940 con 0x7f60f8102570 2024-09-19T06:47:27.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f60ef7fe640 1 -- 172.21.15.103:0/865580805 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8035030 con 0x7f60f8102570 2024-09-19T06:47:27.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.175+0000 7f610062b640 1 -- 172.21.15.103:0/865580805 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f81a7280 con 0x7f60f8102570 2024-09-19T06:47:27.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.176+0000 7f610062b640 1 -- 172.21.15.103:0/865580805 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f8101200 con 0x7f60f8102570 2024-09-19T06:47:27.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.177+0000 7f60ef7fe640 1 -- 172.21.15.103:0/865580805 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f60e8042070 con 0x7f60f8102570 2024-09-19T06:47:27.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.178+0000 7f60ef7fe640 1 --2- 172.21.15.103:0/865580805 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f60e40a3240 0x7f60e40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:27.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.178+0000 7f60fe3c7640 1 --2- 172.21.15.103:0/865580805 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f60e40a3240 0x7f60e40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:27.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.179+0000 7f60ef7fe640 1 -- 172.21.15.103:0/865580805 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f60e80f0850 con 0x7f60f8102570 2024-09-19T06:47:27.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.179+0000 7f60fe3c7640 1 --2- 172.21.15.103:0/865580805 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f60e40a3240 0x7f60e40a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f60f40058c0 tx=0x7f60f4005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:27.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.182+0000 7f60ef7fe640 1 -- 172.21.15.103:0/865580805 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f60e80bd6b0 con 0x7f60f8102570 2024-09-19T06:47:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:26 smithi174 ceph-mon[35637]: pgmap v272: 97 pgs: 7 creating+peering, 62 unknown, 28 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:47:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:26 smithi174 ceph-mon[35637]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T06:47:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:26 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/112710573' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T06:47:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:26 smithi174 ceph-mon[35637]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T06:47:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:26 smithi174 ceph-mon[35637]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T06:47:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:26 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] up:active 2024-09-19T06:47:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:26 smithi174 ceph-mon[35637]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:47:27.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.348+0000 7f610062b640 1 -- 172.21.15.103:0/865580805 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60f81a7450 con 0x7f60f8102570 2024-09-19T06:47:27.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.350+0000 7f60ef7fe640 1 -- 172.21.15.103:0/865580805 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22351 (secure 0 0 0) 0x7f60e80bcfb0 con 0x7f60f8102570 2024-09-19T06:47:27.351 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:47:27.351 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":80,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","created":"2024-09-19T06:39:46.840574+0000","modified":"2024-09-19T06:47:25.445972+0000","last_up_change":"2024-09-19T06:46:35.652960+0000","last_in_change":"2024-09-19T06:46:11.630585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:42:52.050881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:47:21.192611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:47:23.643291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d45dab3-e434-4783-b2e3-f658c3ba583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6803","nonce":3157697740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6805","nonce":3157697740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6809","nonce":3157697740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6807","nonce":3157697740}]},"public_addr":"172.21.15.103:6803/3157697740","cluster_addr":"172.21.15.103:6805/3157697740","heartbeat_back_addr":"172.21.15.103:6809/3157697740","heartbeat_front_addr":"172.21.15.103:6807/3157697740","state":["exists","up"]},{"osd":1,"uuid":"ee0e92bd-15ab-4867-9a18-20abb9f5fc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6801","nonce":1418301195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6803","nonce":1418301195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6807","nonce":1418301195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6805","nonce":1418301195}]},"public_addr":"172.21.15.134:6801/1418301195","cluster_addr":"172.21.15.134:6803/1418301195","heartbeat_back_addr":"172.21.15.134:6807/1418301195","heartbeat_front_addr":"172.21.15.134:6805/1418301195","state":["exists","up"]},{"osd":2,"uuid":"0230caed-025b-46e9-a011-237601d16ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6801","nonce":2292242290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6803","nonce":2292242290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6807","nonce":2292242290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6805","nonce":2292242290}]},"public_addr":"172.21.15.174:6801/2292242290","cluster_addr":"172.21.15.174:6803/2292242290","heartbeat_back_addr":"172.21.15.174:6807/2292242290","heartbeat_front_addr":"172.21.15.174:6805/2292242290","state":["exists","up"]},{"osd":3,"uuid":"c54a8399-bf4b-4d70-b207-a13d85f99590","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6811","nonce":2698986333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6813","nonce":2698986333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6817","nonce":2698986333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6815","nonce":2698986333}]},"public_addr":"172.21.15.103:6811/2698986333","cluster_addr":"172.21.15.103:6813/2698986333","heartbeat_back_addr":"172.21.15.103:6817/2698986333","heartbeat_front_addr":"172.21.15.103:6815/2698986333","state":["exists","up"]},{"osd":4,"uuid":"7c046be6-464a-4ab5-812b-1781121fc317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6809","nonce":3169891968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6811","nonce":3169891968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6815","nonce":3169891968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6813","nonce":3169891968}]},"public_addr":"172.21.15.134:6809/3169891968","cluster_addr":"172.21.15.134:6811/3169891968","heartbeat_back_addr":"172.21.15.134:6815/3169891968","heartbeat_front_addr":"172.21.15.134:6813/3169891968","state":["exists","up"]},{"osd":5,"uuid":"8622596b-6a06-459c-882d-50fba9e296ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6809","nonce":763749644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6811","nonce":763749644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6815","nonce":763749644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6813","nonce":763749644}]},"public_addr":"172.21.15.174:6809/763749644","cluster_addr":"172.21.15.174:6811/763749644","heartbeat_back_addr":"172.21.15.174:6815/763749644","heartbeat_front_addr":"172.21.15.174:6813/763749644","state":["exists","up"]},{"osd":6,"uuid":"64d858d6-a8b3-4f65-a187-af9ab1ccee0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6819","nonce":1396972394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6821","nonce":1396972394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6825","nonce":1396972394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6823","nonce":1396972394}]},"public_addr":"172.21.15.103:6819/1396972394","cluster_addr":"172.21.15.103:6821/1396972394","heartbeat_back_addr":"172.21.15.103:6825/1396972394","heartbeat_front_addr":"172.21.15.103:6823/1396972394","state":["exists","up"]},{"osd":7,"uuid":"d405d1c6-839c-4c92-8261-4adb3163c90f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6817","nonce":2196366765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6819","nonce":2196366765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6823","nonce":2196366765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6821","nonce":2196366765}]},"public_addr":"172.21.15.134:6817/2196366765","cluster_addr":"172.21.15.134:6819/2196366765","heartbeat_back_addr":"172.21.15.134:6823/2196366765","heartbeat_front_addr":"172.21.15.134:6821/2196366765","state":["exists","up"]},{"osd":8,"uuid":"4c452cd9-816c-4f77-9368-2cd4bbeee4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6817","nonce":189977465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6819","nonce":189977465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6823","nonce":189977465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6821","nonce":189977465}]},"public_addr":"172.21.15.174:6817/189977465","cluster_addr":"172.21.15.174:6819/189977465","heartbeat_back_addr":"172.21.15.174:6823/189977465","heartbeat_front_addr":"172.21.15.174:6821/189977465","state":["exists","up"]},{"osd":9,"uuid":"fb3662e0-2d8e-45c4-9119-6d08b35443da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6827","nonce":3161966413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6829","nonce":3161966413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6833","nonce":3161966413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6831","nonce":3161966413}]},"public_addr":"172.21.15.103:6827/3161966413","cluster_addr":"172.21.15.103:6829/3161966413","heartbeat_back_addr":"172.21.15.103:6833/3161966413","heartbeat_front_addr":"172.21.15.103:6831/3161966413","state":["exists","up"]},{"osd":10,"uuid":"e52c3cbe-a6af-441e-b423-b94f3a2da544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6825","nonce":2532758632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6827","nonce":2532758632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6831","nonce":2532758632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6829","nonce":2532758632}]},"public_addr":"172.21.15.134:6825/2532758632","cluster_addr":"172.21.15.134:6827/2532758632","heartbeat_back_addr":"172.21.15.134:6831/2532758632","heartbeat_front_addr":"172.21.15.134:6829/2532758632","state":["exists","up"]},{"osd":11,"uuid":"9040f3f8-3873-4dea-8716-dcda5811c8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6825","nonce":161916442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6827","nonce":161916442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6831","nonce":161916442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6829","nonce":161916442}]},"public_addr":"172.21.15.174:6825/161916442","cluster_addr":"172.21.15.174:6827/161916442","heartbeat_back_addr":"172.21.15.174:6831/161916442","heartbeat_front_addr":"172.21.15.174:6829/161916442","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:41:59.656854+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:24.797551+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:49.685497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:33.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:39.324983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:03.987310+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:28.747971+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:53.827780+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:18.554450+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:43.705691+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:08.401275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:33.502542+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2867049580":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/3727259725":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/1496153385":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:6800/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1642809745":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/2309624637":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1596229922":"2024-09-20T06:40:16.780890+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:47:27.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.353+0000 7f610062b640 1 -- 172.21.15.103:0/865580805 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f60e40a3240 msgr2=0x7f60e40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:27.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.353+0000 7f610062b640 1 --2- 172.21.15.103:0/865580805 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f60e40a3240 0x7f60e40a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f60f40058c0 tx=0x7f60f4005ec0 comp rx=0 tx=0).stop 2024-09-19T06:47:27.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.353+0000 7f610062b640 1 -- 172.21.15.103:0/865580805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60f8102570 msgr2=0x7f60f819ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:27.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.353+0000 7f610062b640 1 --2- 172.21.15.103:0/865580805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60f8102570 0x7f60f819ac40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f60e8006010 tx=0x7f60e8003c80 comp rx=0 tx=0).stop 2024-09-19T06:47:27.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.353+0000 7f610062b640 1 -- 172.21.15.103:0/865580805 shutdown_connections 2024-09-19T06:47:27.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.353+0000 7f610062b640 1 --2- 172.21.15.103:0/865580805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f8102ef0 0x7f60f819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:27.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.353+0000 7f610062b640 1 --2- 172.21.15.103:0/865580805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60f8102570 0x7f60f819ac40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:27.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.353+0000 7f610062b640 1 --2- 172.21.15.103:0/865580805 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f60e40a3240 0x7f60e40a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:27.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.353+0000 7f610062b640 1 --2- 172.21.15.103:0/865580805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f8100e20 0x7f60f819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:27.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.353+0000 7f610062b640 1 -- 172.21.15.103:0/865580805 >> 172.21.15.103:0/865580805 conn(0x7f60f80fc7d0 msgr2=0x7f60f80ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:27.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.353+0000 7f610062b640 1 -- 172.21.15.103:0/865580805 shutdown_connections 2024-09-19T06:47:27.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:27.353+0000 7f610062b640 1 -- 172.21.15.103:0/865580805 wait complete. 2024-09-19T06:47:27.846 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:47:28.092 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:28.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:27 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/865580805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:47:28.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:27 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/865580805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:47:28.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:27 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/865580805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:47:28.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.531+0000 7f9b4f952640 1 -- 172.21.15.103:0/2364529315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b48106f10 msgr2=0x7f9b4810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:28.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.531+0000 7f9b4f952640 1 --2- 172.21.15.103:0/2364529315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b48106f10 0x7f9b4810ea90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009ca0 tx=0x7f9b4002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.531+0000 7f9b4f952640 1 -- 172.21.15.103:0/2364529315 shutdown_connections 2024-09-19T06:47:28.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.531+0000 7f9b4f952640 1 --2- 172.21.15.103:0/2364529315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b48106f10 0x7f9b4810ea90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.531+0000 7f9b4f952640 1 --2- 172.21.15.103:0/2364529315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b48102790 0x7f9b481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.531+0000 7f9b4f952640 1 --2- 172.21.15.103:0/2364529315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b48101de0 0x7f9b481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.531+0000 7f9b4f952640 1 -- 172.21.15.103:0/2364529315 >> 172.21.15.103:0/2364529315 conn(0x7f9b480fd8c0 msgr2=0x7f9b480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:28.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.531+0000 7f9b4f952640 1 -- 172.21.15.103:0/2364529315 shutdown_connections 2024-09-19T06:47:28.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.531+0000 7f9b4f952640 1 -- 172.21.15.103:0/2364529315 wait complete. 2024-09-19T06:47:28.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4f952640 1 Processor -- start 2024-09-19T06:47:28.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4f952640 1 -- start start 2024-09-19T06:47:28.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4f952640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b48101de0 0x7f9b481983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4f952640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b48102790 0x7f9b481988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4f952640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b48106f10 0x7f9b4819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4f952640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48111f90 con 0x7f9b48101de0 2024-09-19T06:47:28.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4f952640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48111e10 con 0x7f9b48106f10 2024-09-19T06:47:28.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4f952640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48112110 con 0x7f9b48102790 2024-09-19T06:47:28.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4d6ee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b48101de0 0x7f9b481983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4d6ee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b48101de0 0x7f9b481983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46678/0 (socket says 172.21.15.103:46678) 2024-09-19T06:47:28.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4d6ee640 1 -- 172.21.15.103:0/1263832081 learned_addr learned my addr 172.21.15.103:0/1263832081 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:28.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4deef640 1 --2- 172.21.15.103:0/1263832081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b48106f10 0x7f9b4819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4d6ee640 1 -- 172.21.15.103:0/1263832081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b48102790 msgr2=0x7f9b481988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:28.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4ceed640 1 --2- 172.21.15.103:0/1263832081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b48102790 0x7f9b481988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4d6ee640 1 --2- 172.21.15.103:0/1263832081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b48102790 0x7f9b481988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4d6ee640 1 -- 172.21.15.103:0/1263832081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b48106f10 msgr2=0x7f9b4819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:28.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4d6ee640 1 --2- 172.21.15.103:0/1263832081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b48106f10 0x7f9b4819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.532+0000 7f9b4d6ee640 1 -- 172.21.15.103:0/1263832081 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4819d360 con 0x7f9b48101de0 2024-09-19T06:47:28.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.533+0000 7f9b4deef640 1 --2- 172.21.15.103:0/1263832081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b48106f10 0x7f9b4819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:28.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.533+0000 7f9b4d6ee640 1 --2- 172.21.15.103:0/1263832081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b48101de0 0x7f9b481983b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400c920 tx=0x7f9b4400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:28.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.533+0000 7f9b3e7fc640 1 -- 172.21.15.103:0/1263832081 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44007c70 con 0x7f9b48101de0 2024-09-19T06:47:28.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.533+0000 7f9b3e7fc640 1 -- 172.21.15.103:0/1263832081 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b44007e10 con 0x7f9b48101de0 2024-09-19T06:47:28.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.533+0000 7f9b4f952640 1 -- 172.21.15.103:0/1263832081 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4819d650 con 0x7f9b48101de0 2024-09-19T06:47:28.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.533+0000 7f9b3e7fc640 1 -- 172.21.15.103:0/1263832081 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44018890 con 0x7f9b48101de0 2024-09-19T06:47:28.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.533+0000 7f9b4f952640 1 -- 172.21.15.103:0/1263832081 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b481a4f30 con 0x7f9b48101de0 2024-09-19T06:47:28.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.534+0000 7f9b4f952640 1 -- 172.21.15.103:0/1263832081 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b481021c0 con 0x7f9b48101de0 2024-09-19T06:47:28.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.535+0000 7f9b3e7fc640 1 -- 172.21.15.103:0/1263832081 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9b44018a50 con 0x7f9b48101de0 2024-09-19T06:47:28.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.536+0000 7f9b3e7fc640 1 --2- 172.21.15.103:0/1263832081 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9b340a3240 0x7f9b340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.536+0000 7f9b4ceed640 1 --2- 172.21.15.103:0/1263832081 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9b340a3240 0x7f9b340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.536+0000 7f9b3e7fc640 1 -- 172.21.15.103:0/1263832081 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f9b440cfbf0 con 0x7f9b48101de0 2024-09-19T06:47:28.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.536+0000 7f9b4ceed640 1 --2- 172.21.15.103:0/1263832081 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9b340a3240 0x7f9b340a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9b48108880 tx=0x7f9b38008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:28.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.539+0000 7f9b3e7fc640 1 -- 172.21.15.103:0/1263832081 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9b44014030 con 0x7f9b48101de0 2024-09-19T06:47:28.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.706+0000 7f9b4f952640 1 -- 172.21.15.103:0/1263832081 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b48199710 con 0x7f9b48101de0 2024-09-19T06:47:28.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.707+0000 7f9b3e7fc640 1 -- 172.21.15.103:0/1263832081 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22351 (secure 0 0 0) 0x7f9b4409caf0 con 0x7f9b48101de0 2024-09-19T06:47:28.709 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:47:28.709 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":80,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","created":"2024-09-19T06:39:46.840574+0000","modified":"2024-09-19T06:47:25.445972+0000","last_up_change":"2024-09-19T06:46:35.652960+0000","last_in_change":"2024-09-19T06:46:11.630585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:42:52.050881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:47:21.192611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:47:23.643291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d45dab3-e434-4783-b2e3-f658c3ba583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6803","nonce":3157697740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6805","nonce":3157697740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6809","nonce":3157697740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6807","nonce":3157697740}]},"public_addr":"172.21.15.103:6803/3157697740","cluster_addr":"172.21.15.103:6805/3157697740","heartbeat_back_addr":"172.21.15.103:6809/3157697740","heartbeat_front_addr":"172.21.15.103:6807/3157697740","state":["exists","up"]},{"osd":1,"uuid":"ee0e92bd-15ab-4867-9a18-20abb9f5fc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6801","nonce":1418301195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6803","nonce":1418301195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6807","nonce":1418301195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6805","nonce":1418301195}]},"public_addr":"172.21.15.134:6801/1418301195","cluster_addr":"172.21.15.134:6803/1418301195","heartbeat_back_addr":"172.21.15.134:6807/1418301195","heartbeat_front_addr":"172.21.15.134:6805/1418301195","state":["exists","up"]},{"osd":2,"uuid":"0230caed-025b-46e9-a011-237601d16ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6801","nonce":2292242290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6803","nonce":2292242290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6807","nonce":2292242290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6805","nonce":2292242290}]},"public_addr":"172.21.15.174:6801/2292242290","cluster_addr":"172.21.15.174:6803/2292242290","heartbeat_back_addr":"172.21.15.174:6807/2292242290","heartbeat_front_addr":"172.21.15.174:6805/2292242290","state":["exists","up"]},{"osd":3,"uuid":"c54a8399-bf4b-4d70-b207-a13d85f99590","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6811","nonce":2698986333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6813","nonce":2698986333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6817","nonce":2698986333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6815","nonce":2698986333}]},"public_addr":"172.21.15.103:6811/2698986333","cluster_addr":"172.21.15.103:6813/2698986333","heartbeat_back_addr":"172.21.15.103:6817/2698986333","heartbeat_front_addr":"172.21.15.103:6815/2698986333","state":["exists","up"]},{"osd":4,"uuid":"7c046be6-464a-4ab5-812b-1781121fc317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6809","nonce":3169891968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6811","nonce":3169891968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6815","nonce":3169891968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6813","nonce":3169891968}]},"public_addr":"172.21.15.134:6809/3169891968","cluster_addr":"172.21.15.134:6811/3169891968","heartbeat_back_addr":"172.21.15.134:6815/3169891968","heartbeat_front_addr":"172.21.15.134:6813/3169891968","state":["exists","up"]},{"osd":5,"uuid":"8622596b-6a06-459c-882d-50fba9e296ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6809","nonce":763749644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6811","nonce":763749644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6815","nonce":763749644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6813","nonce":763749644}]},"public_addr":"172.21.15.174:6809/763749644","cluster_addr":"172.21.15.174:6811/763749644","heartbeat_back_addr":"172.21.15.174:6815/763749644","heartbeat_front_addr":"172.21.15.174:6813/763749644","state":["exists","up"]},{"osd":6,"uuid":"64d858d6-a8b3-4f65-a187-af9ab1ccee0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6819","nonce":1396972394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6821","nonce":1396972394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6825","nonce":1396972394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6823","nonce":1396972394}]},"public_addr":"172.21.15.103:6819/1396972394","cluster_addr":"172.21.15.103:6821/1396972394","heartbeat_back_addr":"172.21.15.103:6825/1396972394","heartbeat_front_addr":"172.21.15.103:6823/1396972394","state":["exists","up"]},{"osd":7,"uuid":"d405d1c6-839c-4c92-8261-4adb3163c90f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6817","nonce":2196366765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6819","nonce":2196366765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6823","nonce":2196366765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6821","nonce":2196366765}]},"public_addr":"172.21.15.134:6817/2196366765","cluster_addr":"172.21.15.134:6819/2196366765","heartbeat_back_addr":"172.21.15.134:6823/2196366765","heartbeat_front_addr":"172.21.15.134:6821/2196366765","state":["exists","up"]},{"osd":8,"uuid":"4c452cd9-816c-4f77-9368-2cd4bbeee4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6817","nonce":189977465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6819","nonce":189977465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6823","nonce":189977465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6821","nonce":189977465}]},"public_addr":"172.21.15.174:6817/189977465","cluster_addr":"172.21.15.174:6819/189977465","heartbeat_back_addr":"172.21.15.174:6823/189977465","heartbeat_front_addr":"172.21.15.174:6821/189977465","state":["exists","up"]},{"osd":9,"uuid":"fb3662e0-2d8e-45c4-9119-6d08b35443da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6827","nonce":3161966413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6829","nonce":3161966413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6833","nonce":3161966413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6831","nonce":3161966413}]},"public_addr":"172.21.15.103:6827/3161966413","cluster_addr":"172.21.15.103:6829/3161966413","heartbeat_back_addr":"172.21.15.103:6833/3161966413","heartbeat_front_addr":"172.21.15.103:6831/3161966413","state":["exists","up"]},{"osd":10,"uuid":"e52c3cbe-a6af-441e-b423-b94f3a2da544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6825","nonce":2532758632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6827","nonce":2532758632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6831","nonce":2532758632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6829","nonce":2532758632}]},"public_addr":"172.21.15.134:6825/2532758632","cluster_addr":"172.21.15.134:6827/2532758632","heartbeat_back_addr":"172.21.15.134:6831/2532758632","heartbeat_front_addr":"172.21.15.134:6829/2532758632","state":["exists","up"]},{"osd":11,"uuid":"9040f3f8-3873-4dea-8716-dcda5811c8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6825","nonce":161916442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6827","nonce":161916442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6831","nonce":161916442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6829","nonce":161916442}]},"public_addr":"172.21.15.174:6825/161916442","cluster_addr":"172.21.15.174:6827/161916442","heartbeat_back_addr":"172.21.15.174:6831/161916442","heartbeat_front_addr":"172.21.15.174:6829/161916442","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:41:59.656854+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:24.797551+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:49.685497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:33.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:39.324983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:03.987310+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:28.747971+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:53.827780+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:18.554450+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:43.705691+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:08.401275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:33.502542+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2867049580":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/3727259725":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/1496153385":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:6800/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1642809745":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/2309624637":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1596229922":"2024-09-20T06:40:16.780890+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:47:28.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.709+0000 7f9b4f952640 1 -- 172.21.15.103:0/1263832081 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9b340a3240 msgr2=0x7f9b340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:28.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.709+0000 7f9b4f952640 1 --2- 172.21.15.103:0/1263832081 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9b340a3240 0x7f9b340a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9b48108880 tx=0x7f9b38008040 comp rx=0 tx=0).stop 2024-09-19T06:47:28.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.709+0000 7f9b4f952640 1 -- 172.21.15.103:0/1263832081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b48101de0 msgr2=0x7f9b481983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:28.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.709+0000 7f9b4f952640 1 --2- 172.21.15.103:0/1263832081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b48101de0 0x7f9b481983b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400c920 tx=0x7f9b4400cde0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.709+0000 7f9b4f952640 1 -- 172.21.15.103:0/1263832081 shutdown_connections 2024-09-19T06:47:28.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.709+0000 7f9b4f952640 1 --2- 172.21.15.103:0/1263832081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b48106f10 0x7f9b4819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.710+0000 7f9b4f952640 1 --2- 172.21.15.103:0/1263832081 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9b340a3240 0x7f9b340a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.710+0000 7f9b4f952640 1 --2- 172.21.15.103:0/1263832081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b48102790 0x7f9b481988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.710+0000 7f9b4f952640 1 --2- 172.21.15.103:0/1263832081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b48101de0 0x7f9b481983b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.710+0000 7f9b4f952640 1 -- 172.21.15.103:0/1263832081 >> 172.21.15.103:0/1263832081 conn(0x7f9b480fd8c0 msgr2=0x7f9b480febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:28.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.710+0000 7f9b4f952640 1 -- 172.21.15.103:0/1263832081 shutdown_connections 2024-09-19T06:47:28.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:28.710+0000 7f9b4f952640 1 -- 172.21.15.103:0/1263832081 wait complete. 2024-09-19T06:47:29.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:28 smithi103 ceph-mon[31158]: pgmap v274: 97 pgs: 97 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-19T06:47:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:28 smithi103 ceph-mon[31158]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T06:47:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:28 smithi103 ceph-mon[31158]: Cluster is now healthy 2024-09-19T06:47:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:28 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1263832081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:47:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:28 smithi134 ceph-mon[35356]: pgmap v274: 97 pgs: 97 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-19T06:47:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:28 smithi134 ceph-mon[35356]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T06:47:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:28 smithi134 ceph-mon[35356]: Cluster is now healthy 2024-09-19T06:47:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:28 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1263832081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:47:29.293 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T06:47:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:28 smithi174 ceph-mon[35637]: pgmap v274: 97 pgs: 97 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-19T06:47:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:28 smithi174 ceph-mon[35637]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T06:47:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:28 smithi174 ceph-mon[35637]: Cluster is now healthy 2024-09-19T06:47:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:28 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1263832081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:47:29.537 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:30.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.167+0000 7f80f6d35640 1 -- 172.21.15.103:0/3005636413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f0100e20 msgr2=0x7f80f0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:30.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.167+0000 7f80f6d35640 1 --2- 172.21.15.103:0/3005636413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f0100e20 0x7f80f0101200 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009900 tx=0x7f80e002a480 comp rx=0 tx=0).stop 2024-09-19T06:47:30.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.167+0000 7f80f6d35640 1 -- 172.21.15.103:0/3005636413 shutdown_connections 2024-09-19T06:47:30.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.167+0000 7f80f6d35640 1 --2- 172.21.15.103:0/3005636413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80f0102ef0 0x7f80f010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:30.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.167+0000 7f80f6d35640 1 --2- 172.21.15.103:0/3005636413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f0102570 0x7f80f01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:30.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.167+0000 7f80f6d35640 1 --2- 172.21.15.103:0/3005636413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f0100e20 0x7f80f0101200 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:30.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.167+0000 7f80f6d35640 1 -- 172.21.15.103:0/3005636413 >> 172.21.15.103:0/3005636413 conn(0x7f80f00fc7d0 msgr2=0x7f80f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:30.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.167+0000 7f80f6d35640 1 -- 172.21.15.103:0/3005636413 shutdown_connections 2024-09-19T06:47:30.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.167+0000 7f80f6d35640 1 -- 172.21.15.103:0/3005636413 wait complete. 2024-09-19T06:47:30.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.167+0000 7f80f6d35640 1 Processor -- start 2024-09-19T06:47:30.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.167+0000 7f80f6d35640 1 -- start start 2024-09-19T06:47:30.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80f6d35640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f0100e20 0x7f80f0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:30.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80f6d35640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f0102570 0x7f80f0198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:30.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80f6d35640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80f0102ef0 0x7f80f019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:30.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80f6d35640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80f0111020 con 0x7f80f0100e20 2024-09-19T06:47:30.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80f6d35640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f80f0110ea0 con 0x7f80f0102570 2024-09-19T06:47:30.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80f6d35640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01111a0 con 0x7f80f0102ef0 2024-09-19T06:47:30.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80f4ad1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f0100e20 0x7f80f0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:30.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80f4ad1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f0100e20 0x7f80f0198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46692/0 (socket says 172.21.15.103:46692) 2024-09-19T06:47:30.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80f4ad1640 1 -- 172.21.15.103:0/3725408061 learned_addr learned my addr 172.21.15.103:0/3725408061 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:30.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80effff640 1 --2- 172.21.15.103:0/3725408061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f0102570 0x7f80f0198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:30.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80f52d2640 1 --2- 172.21.15.103:0/3725408061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80f0102ef0 0x7f80f019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:30.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80f4ad1640 1 -- 172.21.15.103:0/3725408061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80f0102ef0 msgr2=0x7f80f019cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:30.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80f4ad1640 1 --2- 172.21.15.103:0/3725408061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80f0102ef0 0x7f80f019cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:30.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80f4ad1640 1 -- 172.21.15.103:0/3725408061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f0102570 msgr2=0x7f80f0198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:30.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80f4ad1640 1 --2- 172.21.15.103:0/3725408061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f0102570 0x7f80f0198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:30.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80f4ad1640 1 -- 172.21.15.103:0/3725408061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f019d6c0 con 0x7f80f0100e20 2024-09-19T06:47:30.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80effff640 1 --2- 172.21.15.103:0/3725408061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f0102570 0x7f80f0198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:30.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80f4ad1640 1 --2- 172.21.15.103:0/3725408061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f0100e20 0x7f80f0198710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f80e0006010 tx=0x7f80e0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:30.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.168+0000 7f80edffb640 1 -- 172.21.15.103:0/3725408061 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e003f070 con 0x7f80f0100e20 2024-09-19T06:47:30.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.169+0000 7f80edffb640 1 -- 172.21.15.103:0/3725408061 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e0031e30 con 0x7f80f0100e20 2024-09-19T06:47:30.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.169+0000 7f80f6d35640 1 -- 172.21.15.103:0/3725408061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f019d950 con 0x7f80f0100e20 2024-09-19T06:47:30.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.169+0000 7f80edffb640 1 -- 172.21.15.103:0/3725408061 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0035030 con 0x7f80f0100e20 2024-09-19T06:47:30.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.169+0000 7f80f6d35640 1 -- 172.21.15.103:0/3725408061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f01a5290 con 0x7f80f0100e20 2024-09-19T06:47:30.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.170+0000 7f80f6d35640 1 -- 172.21.15.103:0/3725408061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f0101280 con 0x7f80f0100e20 2024-09-19T06:47:30.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.171+0000 7f80edffb640 1 -- 172.21.15.103:0/3725408061 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f80e0042070 con 0x7f80f0100e20 2024-09-19T06:47:30.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.171+0000 7f80edffb640 1 --2- 172.21.15.103:0/3725408061 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f80d80a32c0 0x7f80d80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:30.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.172+0000 7f80edffb640 1 -- 172.21.15.103:0/3725408061 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f80e00f0140 con 0x7f80f0100e20 2024-09-19T06:47:30.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.172+0000 7f80effff640 1 --2- 172.21.15.103:0/3725408061 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f80d80a32c0 0x7f80d80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:30.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.172+0000 7f80effff640 1 --2- 172.21.15.103:0/3725408061 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f80d80a32c0 0x7f80d80a5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f80f00ff0c0 tx=0x7f80e4009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:30.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.176+0000 7f80edffb640 1 -- 172.21.15.103:0/3725408061 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f80e00bcfb0 con 0x7f80f0100e20 2024-09-19T06:47:30.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.375+0000 7f80f6d35640 1 -- 172.21.15.103:0/3725408061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f80f010a0a0 con 0x7f80f0100e20 2024-09-19T06:47:30.437 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 06:47:29 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-b[65626]: 2024-09-19T06:47:29.997+0000 7f2df2ee1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T06:47:30.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.843+0000 7f80edffb640 1 -- 172.21.15.103:0/3725408061 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f80e00bc8b0 con 0x7f80f0100e20 2024-09-19T06:47:30.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.846+0000 7f80f6d35640 1 -- 172.21.15.103:0/3725408061 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f80d80a32c0 msgr2=0x7f80d80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:30.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.846+0000 7f80f6d35640 1 --2- 172.21.15.103:0/3725408061 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f80d80a32c0 0x7f80d80a5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f80f00ff0c0 tx=0x7f80e4009570 comp rx=0 tx=0).stop 2024-09-19T06:47:30.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.846+0000 7f80f6d35640 1 -- 172.21.15.103:0/3725408061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f0100e20 msgr2=0x7f80f0198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:30.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.846+0000 7f80f6d35640 1 --2- 172.21.15.103:0/3725408061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f0100e20 0x7f80f0198710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f80e0006010 tx=0x7f80e0003c80 comp rx=0 tx=0).stop 2024-09-19T06:47:30.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.846+0000 7f80f6d35640 1 -- 172.21.15.103:0/3725408061 shutdown_connections 2024-09-19T06:47:30.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.846+0000 7f80f6d35640 1 --2- 172.21.15.103:0/3725408061 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f80d80a32c0 0x7f80d80a5780 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:30.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.846+0000 7f80f6d35640 1 --2- 172.21.15.103:0/3725408061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80f0102ef0 0x7f80f019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:30.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.846+0000 7f80f6d35640 1 --2- 172.21.15.103:0/3725408061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f0102570 0x7f80f0198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:30.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.846+0000 7f80f6d35640 1 --2- 172.21.15.103:0/3725408061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f0100e20 0x7f80f0198710 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:30.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.846+0000 7f80f6d35640 1 -- 172.21.15.103:0/3725408061 >> 172.21.15.103:0/3725408061 conn(0x7f80f00fc7d0 msgr2=0x7f80f006a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:30.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.846+0000 7f80f6d35640 1 -- 172.21.15.103:0/3725408061 shutdown_connections 2024-09-19T06:47:30.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:30.846+0000 7f80f6d35640 1 -- 172.21.15.103:0/3725408061 wait complete. 2024-09-19T06:47:31.123 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:30 smithi103 ceph-mon[31158]: pgmap v275: 97 pgs: 97 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-19T06:47:31.123 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:30 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3725408061' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T06:47:31.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:30 smithi134 ceph-mon[35356]: pgmap v275: 97 pgs: 97 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-19T06:47:31.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:30 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3725408061' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T06:47:31.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:30 smithi174 ceph-mon[35637]: pgmap v275: 97 pgs: 97 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-19T06:47:31.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:30 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3725408061' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T06:47:31.417 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-19T06:47:31.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-19T06:47:31.662 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:32.044 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:31 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3725408061' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T06:47:32.044 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:31 smithi103 ceph-mon[31158]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:47:32.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.126+0000 7fb30860c640 1 -- 172.21.15.103:0/2012958664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb300101e10 msgr2=0x7fb3001021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:32.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.126+0000 7fb30860c640 1 --2- 172.21.15.103:0/2012958664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb300101e10 0x7fb3001021f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009900 tx=0x7fb2fc02a410 comp rx=0 tx=0).stop 2024-09-19T06:47:32.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.126+0000 7fb30860c640 1 -- 172.21.15.103:0/2012958664 shutdown_connections 2024-09-19T06:47:32.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.126+0000 7fb30860c640 1 --2- 172.21.15.103:0/2012958664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb300106f40 0x7fb30010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:32.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.126+0000 7fb30860c640 1 --2- 172.21.15.103:0/2012958664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3001027c0 0x7fb300106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:32.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.126+0000 7fb30860c640 1 --2- 172.21.15.103:0/2012958664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb300101e10 0x7fb3001021f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:32.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.126+0000 7fb30860c640 1 -- 172.21.15.103:0/2012958664 >> 172.21.15.103:0/2012958664 conn(0x7fb3000fd8d0 msgr2=0x7fb3000ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:32.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.126+0000 7fb30860c640 1 -- 172.21.15.103:0/2012958664 shutdown_connections 2024-09-19T06:47:32.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.126+0000 7fb30860c640 1 -- 172.21.15.103:0/2012958664 wait complete. 2024-09-19T06:47:32.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb30860c640 1 Processor -- start 2024-09-19T06:47:32.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb30860c640 1 -- start start 2024-09-19T06:47:32.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb30860c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb300101e10 0x7fb300198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:32.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb30860c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3001027c0 0x7fb3001989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:32.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb30860c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb300106f40 0x7fb30019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:32.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb30860c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb300111fc0 con 0x7fb300106f40 2024-09-19T06:47:32.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb30860c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb300111e40 con 0x7fb300101e10 2024-09-19T06:47:32.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb30860c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb300112140 con 0x7fb3001027c0 2024-09-19T06:47:32.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb306ba9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb300106f40 0x7fb30019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:32.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb306ba9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb300106f40 0x7fb30019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46708/0 (socket says 172.21.15.103:46708) 2024-09-19T06:47:32.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb306ba9640 1 -- 172.21.15.103:0/2960599334 learned_addr learned my addr 172.21.15.103:0/2960599334 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:32.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb3063a8640 1 --2- 172.21.15.103:0/2960599334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb300101e10 0x7fb300198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:32.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb305ba7640 1 --2- 172.21.15.103:0/2960599334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3001027c0 0x7fb3001989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:32.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb306ba9640 1 -- 172.21.15.103:0/2960599334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3001027c0 msgr2=0x7fb3001989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:32.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb306ba9640 1 --2- 172.21.15.103:0/2960599334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3001027c0 0x7fb3001989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:32.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb306ba9640 1 -- 172.21.15.103:0/2960599334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb300101e10 msgr2=0x7fb300198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:32.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.127+0000 7fb306ba9640 1 --2- 172.21.15.103:0/2960599334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb300101e10 0x7fb300198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:32.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.128+0000 7fb306ba9640 1 -- 172.21.15.103:0/2960599334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30019d430 con 0x7fb300106f40 2024-09-19T06:47:32.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.128+0000 7fb3063a8640 1 --2- 172.21.15.103:0/2960599334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb300101e10 0x7fb300198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:32.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.128+0000 7fb306ba9640 1 --2- 172.21.15.103:0/2960599334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb300106f40 0x7fb30019cd50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb2f800b970 tx=0x7fb2f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:32.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.128+0000 7fb2f77fe640 1 -- 172.21.15.103:0/2960599334 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8019070 con 0x7fb300106f40 2024-09-19T06:47:32.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.128+0000 7fb2f77fe640 1 -- 172.21.15.103:0/2960599334 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2f800cb40 con 0x7fb300106f40 2024-09-19T06:47:32.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.128+0000 7fb2f77fe640 1 -- 172.21.15.103:0/2960599334 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8007650 con 0x7fb300106f40 2024-09-19T06:47:32.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.128+0000 7fb30860c640 1 -- 172.21.15.103:0/2960599334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30019d720 con 0x7fb300106f40 2024-09-19T06:47:32.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.128+0000 7fb30860c640 1 -- 172.21.15.103:0/2960599334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3001a5000 con 0x7fb300106f40 2024-09-19T06:47:32.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.129+0000 7fb30860c640 1 -- 172.21.15.103:0/2960599334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3001021f0 con 0x7fb300106f40 2024-09-19T06:47:32.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.130+0000 7fb2f77fe640 1 -- 172.21.15.103:0/2960599334 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb2f801c070 con 0x7fb300106f40 2024-09-19T06:47:32.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.131+0000 7fb2f77fe640 1 --2- 172.21.15.103:0/2960599334 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb2ec0a32c0 0x7fb2ec0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:32.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.131+0000 7fb3063a8640 1 --2- 172.21.15.103:0/2960599334 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb2ec0a32c0 0x7fb2ec0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:32.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.131+0000 7fb2f77fe640 1 -- 172.21.15.103:0/2960599334 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fb2f80cee90 con 0x7fb300106f40 2024-09-19T06:47:32.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.132+0000 7fb3063a8640 1 --2- 172.21.15.103:0/2960599334 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb2ec0a32c0 0x7fb2ec0a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc000c00 tx=0x7fb2fc032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:32.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.135+0000 7fb2f77fe640 1 -- 172.21.15.103:0/2960599334 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb2f809bc60 con 0x7fb300106f40 2024-09-19T06:47:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:31 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3725408061' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T06:47:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:31 smithi134 ceph-mon[35356]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:47:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:31 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3725408061' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T06:47:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:31 smithi174 ceph-mon[35637]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:47:32.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.361+0000 7fb30860c640 1 -- 172.21.15.103:0/2960599334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fb30002d7c0 con 0x7fb300106f40 2024-09-19T06:47:32.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.860+0000 7fb2f77fe640 1 -- 172.21.15.103:0/2960599334 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7fb2f809b560 con 0x7fb300106f40 2024-09-19T06:47:32.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.862+0000 7fb30860c640 1 -- 172.21.15.103:0/2960599334 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb2ec0a32c0 msgr2=0x7fb2ec0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.862+0000 7fb30860c640 1 --2- 172.21.15.103:0/2960599334 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb2ec0a32c0 0x7fb2ec0a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc000c00 tx=0x7fb2fc032d50 comp rx=0 tx=0).stop 2024-09-19T06:47:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.862+0000 7fb30860c640 1 -- 172.21.15.103:0/2960599334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb300106f40 msgr2=0x7fb30019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.862+0000 7fb30860c640 1 --2- 172.21.15.103:0/2960599334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb300106f40 0x7fb30019cd50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb2f800b970 tx=0x7fb2f800be30 comp rx=0 tx=0).stop 2024-09-19T06:47:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.862+0000 7fb30860c640 1 -- 172.21.15.103:0/2960599334 shutdown_connections 2024-09-19T06:47:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.862+0000 7fb30860c640 1 --2- 172.21.15.103:0/2960599334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb300106f40 0x7fb30019cd50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.862+0000 7fb30860c640 1 --2- 172.21.15.103:0/2960599334 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb2ec0a32c0 0x7fb2ec0a5780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.862+0000 7fb30860c640 1 --2- 172.21.15.103:0/2960599334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3001027c0 0x7fb3001989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.862+0000 7fb30860c640 1 --2- 172.21.15.103:0/2960599334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb300101e10 0x7fb300198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.862+0000 7fb30860c640 1 -- 172.21.15.103:0/2960599334 >> 172.21.15.103:0/2960599334 conn(0x7fb3000fd8d0 msgr2=0x7fb3000fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.863+0000 7fb30860c640 1 -- 172.21.15.103:0/2960599334 shutdown_connections 2024-09-19T06:47:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:32.863+0000 7fb30860c640 1 -- 172.21.15.103:0/2960599334 wait complete. 2024-09-19T06:47:33.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:32 smithi103 ceph-mon[31158]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-19T06:47:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:32 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2960599334' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T06:47:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:32 smithi134 ceph-mon[35356]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-19T06:47:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:32 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2960599334' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T06:47:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:32 smithi174 ceph-mon[35637]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-19T06:47:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:32 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2960599334' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T06:47:33.500 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T06:47:33.748 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:34.127 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:33 smithi103 ceph-mon[31158]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T06:47:34.128 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:33 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2960599334' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T06:47:34.128 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:33 smithi103 ceph-mon[31158]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T06:47:34.128 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:33 smithi103 ceph-mon[31158]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:47:34.128 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:33 smithi103 ceph-mon[31158]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-19T06:47:34.128 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:33 smithi103 ceph-mon[31158]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-19T06:47:34.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:33 smithi134 ceph-mon[35356]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T06:47:34.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:33 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2960599334' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T06:47:34.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:33 smithi134 ceph-mon[35356]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T06:47:34.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:33 smithi134 ceph-mon[35356]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:47:34.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:33 smithi134 ceph-mon[35356]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-19T06:47:34.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:33 smithi134 ceph-mon[35356]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-19T06:47:34.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.210+0000 7fddd4c07640 1 -- 172.21.15.103:0/1960186136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddd0106a80 msgr2=0x7fddd0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:34.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.210+0000 7fddd4c07640 1 --2- 172.21.15.103:0/1960186136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddd0106a80 0x7fddd0106e60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009cd0 tx=0x7fddc002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:47:34.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.210+0000 7fddd4c07640 1 -- 172.21.15.103:0/1960186136 shutdown_connections 2024-09-19T06:47:34.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.210+0000 7fddd4c07640 1 --2- 172.21.15.103:0/1960186136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd0101420 0x7fddd010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:34.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.210+0000 7fddd4c07640 1 --2- 172.21.15.103:0/1960186136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fddd0100a80 0x7fddd0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:34.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.210+0000 7fddd4c07640 1 --2- 172.21.15.103:0/1960186136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddd0106a80 0x7fddd0106e60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:34.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.210+0000 7fddd4c07640 1 -- 172.21.15.103:0/1960186136 >> 172.21.15.103:0/1960186136 conn(0x7fddd00fc7a0 msgr2=0x7fddd00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:34.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.210+0000 7fddd4c07640 1 -- 172.21.15.103:0/1960186136 shutdown_connections 2024-09-19T06:47:34.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.210+0000 7fddd4c07640 1 -- 172.21.15.103:0/1960186136 wait complete. 2024-09-19T06:47:34.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.211+0000 7fddd4c07640 1 Processor -- start 2024-09-19T06:47:34.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.211+0000 7fddd4c07640 1 -- start start 2024-09-19T06:47:34.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.211+0000 7fddd4c07640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd0100a80 0x7fddd0197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:34.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.211+0000 7fddd4c07640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddd0101420 0x7fddd0197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:34.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.211+0000 7fddd4c07640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fddd0106a80 0x7fddd019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:34.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.211+0000 7fddd4c07640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fddd0110e80 con 0x7fddd0101420 2024-09-19T06:47:34.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.211+0000 7fddd4c07640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fddd0110d00 con 0x7fddd0100a80 2024-09-19T06:47:34.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.211+0000 7fddd4c07640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fddd0111000 con 0x7fddd0106a80 2024-09-19T06:47:34.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.211+0000 7fddcdd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddd0101420 0x7fddd0197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:34.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.212+0000 7fddcdd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddd0101420 0x7fddd0197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46718/0 (socket says 172.21.15.103:46718) 2024-09-19T06:47:34.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.212+0000 7fddcdd9b640 1 -- 172.21.15.103:0/1262078838 learned_addr learned my addr 172.21.15.103:0/1262078838 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:34.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.212+0000 7fddced9d640 1 --2- 172.21.15.103:0/1262078838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fddd0106a80 0x7fddd019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:34.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.212+0000 7fddce59c640 1 --2- 172.21.15.103:0/1262078838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd0100a80 0x7fddd0197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:34.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.212+0000 7fddcdd9b640 1 -- 172.21.15.103:0/1262078838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fddd0106a80 msgr2=0x7fddd019bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:34.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.212+0000 7fddcdd9b640 1 --2- 172.21.15.103:0/1262078838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fddd0106a80 0x7fddd019bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:34.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.212+0000 7fddcdd9b640 1 -- 172.21.15.103:0/1262078838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd0100a80 msgr2=0x7fddd0197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:34.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.212+0000 7fddcdd9b640 1 --2- 172.21.15.103:0/1262078838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd0100a80 0x7fddd0197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:34.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.212+0000 7fddcdd9b640 1 -- 172.21.15.103:0/1262078838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd019c2b0 con 0x7fddd0101420 2024-09-19T06:47:34.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.212+0000 7fddce59c640 1 --2- 172.21.15.103:0/1262078838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd0100a80 0x7fddd0197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:34.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.212+0000 7fddcdd9b640 1 --2- 172.21.15.103:0/1262078838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddd0101420 0x7fddd0197840 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fddc400d770 tx=0x7fddc400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:34.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.212+0000 7fddbf7fe640 1 -- 172.21.15.103:0/1262078838 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4017070 con 0x7fddd0101420 2024-09-19T06:47:34.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.212+0000 7fddd4c07640 1 -- 172.21.15.103:0/1262078838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd019c5a0 con 0x7fddd0101420 2024-09-19T06:47:34.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.212+0000 7fddbf7fe640 1 -- 172.21.15.103:0/1262078838 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddc4005500 con 0x7fddd0101420 2024-09-19T06:47:34.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.212+0000 7fddbf7fe640 1 -- 172.21.15.103:0/1262078838 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4004e00 con 0x7fddd0101420 2024-09-19T06:47:34.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.212+0000 7fddd4c07640 1 -- 172.21.15.103:0/1262078838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd01a3e80 con 0x7fddd0101420 2024-09-19T06:47:34.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.213+0000 7fddd4c07640 1 -- 172.21.15.103:0/1262078838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd0100ee0 con 0x7fddd0101420 2024-09-19T06:47:34.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.214+0000 7fddbf7fe640 1 -- 172.21.15.103:0/1262078838 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fddc4004fa0 con 0x7fddd0101420 2024-09-19T06:47:34.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.215+0000 7fddbf7fe640 1 --2- 172.21.15.103:0/1262078838 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fddb80a3240 0x7fddb80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:34.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.215+0000 7fddce59c640 1 --2- 172.21.15.103:0/1262078838 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fddb80a3240 0x7fddb80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:34.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.215+0000 7fddbf7fe640 1 -- 172.21.15.103:0/1262078838 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fddc40cf670 con 0x7fddd0101420 2024-09-19T06:47:34.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.216+0000 7fddce59c640 1 --2- 172.21.15.103:0/1262078838 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fddb80a3240 0x7fddb80a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fddc0000f80 tx=0x7fddc00093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:34.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.218+0000 7fddbf7fe640 1 -- 172.21.15.103:0/1262078838 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fddc409c4e0 con 0x7fddd0101420 2024-09-19T06:47:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:33 smithi174 ceph-mon[35637]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T06:47:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:33 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2960599334' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T06:47:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:33 smithi174 ceph-mon[35637]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T06:47:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:33 smithi174 ceph-mon[35637]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:47:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:33 smithi174 ceph-mon[35637]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-19T06:47:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:33 smithi174 ceph-mon[35637]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-19T06:47:34.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.423+0000 7fddd4c07640 1 -- 172.21.15.103:0/1262078838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fddd0109020 con 0x7fddd0101420 2024-09-19T06:47:34.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.880+0000 7fddbf7fe640 1 -- 172.21.15.103:0/1262078838 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fddc409bde0 con 0x7fddd0101420 2024-09-19T06:47:34.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.883+0000 7fddd4c07640 1 -- 172.21.15.103:0/1262078838 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fddb80a3240 msgr2=0x7fddb80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.883+0000 7fddd4c07640 1 --2- 172.21.15.103:0/1262078838 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fddb80a3240 0x7fddb80a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fddc0000f80 tx=0x7fddc00093f0 comp rx=0 tx=0).stop 2024-09-19T06:47:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.883+0000 7fddd4c07640 1 -- 172.21.15.103:0/1262078838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddd0101420 msgr2=0x7fddd0197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.883+0000 7fddd4c07640 1 --2- 172.21.15.103:0/1262078838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddd0101420 0x7fddd0197840 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fddc400d770 tx=0x7fddc400dc30 comp rx=0 tx=0).stop 2024-09-19T06:47:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.884+0000 7fddd4c07640 1 -- 172.21.15.103:0/1262078838 shutdown_connections 2024-09-19T06:47:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.884+0000 7fddd4c07640 1 --2- 172.21.15.103:0/1262078838 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fddb80a3240 0x7fddb80a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.884+0000 7fddd4c07640 1 --2- 172.21.15.103:0/1262078838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fddd0106a80 0x7fddd019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.884+0000 7fddd4c07640 1 --2- 172.21.15.103:0/1262078838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddd0101420 0x7fddd0197840 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.884+0000 7fddd4c07640 1 --2- 172.21.15.103:0/1262078838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd0100a80 0x7fddd0197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.884+0000 7fddd4c07640 1 -- 172.21.15.103:0/1262078838 >> 172.21.15.103:0/1262078838 conn(0x7fddd00fc7a0 msgr2=0x7fddd00fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.884+0000 7fddd4c07640 1 -- 172.21.15.103:0/1262078838 shutdown_connections 2024-09-19T06:47:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:34.884+0000 7fddd4c07640 1 -- 172.21.15.103:0/1262078838 wait complete. 2024-09-19T06:47:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:34 smithi103 ceph-mon[31158]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-19T06:47:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:34 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] up:active 2024-09-19T06:47:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:34 smithi103 ceph-mon[31158]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T06:47:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:34 smithi103 ceph-mon[31158]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-19T06:47:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:34 smithi103 ceph-mon[31158]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-19T06:47:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:34 smithi103 ceph-mon[31158]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-19T06:47:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:34 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1262078838' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T06:47:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:34 smithi134 ceph-mon[35356]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-19T06:47:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:34 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] up:active 2024-09-19T06:47:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:34 smithi134 ceph-mon[35356]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T06:47:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:34 smithi134 ceph-mon[35356]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-19T06:47:35.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:34 smithi134 ceph-mon[35356]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-19T06:47:35.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:34 smithi134 ceph-mon[35356]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-19T06:47:35.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:34 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1262078838' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T06:47:35.188 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 06:47:34 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-b[65626]: 2024-09-19T06:47:34.998+0000 7f2df2ee1640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-19T06:47:35.188 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 06:47:34 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-b[65626]: 2024-09-19T06:47:34.998+0000 7f2df2ee1640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T06:47:35.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:34 smithi174 ceph-mon[35637]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-19T06:47:35.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:34 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] up:active 2024-09-19T06:47:35.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:34 smithi174 ceph-mon[35637]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T06:47:35.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:34 smithi174 ceph-mon[35637]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-19T06:47:35.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:34 smithi174 ceph-mon[35637]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-19T06:47:35.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:34 smithi174 ceph-mon[35637]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-19T06:47:35.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:34 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1262078838' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T06:47:35.576 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T06:47:35.823 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:36.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:35 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1262078838' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T06:47:36.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:35 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] up:active 2024-09-19T06:47:36.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:35 smithi103 ceph-mon[31158]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T06:47:36.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:35 smithi103 ceph-mon[31158]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-19T06:47:36.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:35 smithi103 ceph-mon[31158]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-19T06:47:36.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:35 smithi103 ceph-mon[31158]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-19T06:47:36.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:35 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1262078838' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T06:47:36.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:35 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] up:active 2024-09-19T06:47:36.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:35 smithi134 ceph-mon[35356]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T06:47:36.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:35 smithi134 ceph-mon[35356]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-19T06:47:36.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:35 smithi134 ceph-mon[35356]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-19T06:47:36.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:35 smithi134 ceph-mon[35356]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-19T06:47:36.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.266+0000 7fb0d30c2640 1 -- 172.21.15.103:0/4151677798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0cc102790 msgr2=0x7fb0cc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:36.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.266+0000 7fb0d30c2640 1 --2- 172.21.15.103:0/4151677798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0cc102790 0x7fb0cc1067e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009900 tx=0x7fb0b802a410 comp rx=0 tx=0).stop 2024-09-19T06:47:36.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.266+0000 7fb0d30c2640 1 -- 172.21.15.103:0/4151677798 shutdown_connections 2024-09-19T06:47:36.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.266+0000 7fb0d30c2640 1 --2- 172.21.15.103:0/4151677798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0cc106f10 0x7fb0cc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:36.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.266+0000 7fb0d30c2640 1 --2- 172.21.15.103:0/4151677798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0cc102790 0x7fb0cc1067e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:36.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.266+0000 7fb0d30c2640 1 --2- 172.21.15.103:0/4151677798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0cc101de0 0x7fb0cc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:36.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.266+0000 7fb0d30c2640 1 -- 172.21.15.103:0/4151677798 >> 172.21.15.103:0/4151677798 conn(0x7fb0cc0fd8c0 msgr2=0x7fb0cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:36.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.266+0000 7fb0d30c2640 1 -- 172.21.15.103:0/4151677798 shutdown_connections 2024-09-19T06:47:36.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.266+0000 7fb0d30c2640 1 -- 172.21.15.103:0/4151677798 wait complete. 2024-09-19T06:47:36.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d30c2640 1 Processor -- start 2024-09-19T06:47:36.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d30c2640 1 -- start start 2024-09-19T06:47:36.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d30c2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0cc101de0 0x7fb0cc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:36.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d30c2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0cc102790 0x7fb0cc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:36.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d30c2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0cc106f10 0x7fb0cc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:36.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d30c2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc111fb0 con 0x7fb0cc106f10 2024-09-19T06:47:36.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d30c2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc111e30 con 0x7fb0cc102790 2024-09-19T06:47:36.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d30c2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc112130 con 0x7fb0cc101de0 2024-09-19T06:47:36.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d165f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0cc106f10 0x7fb0cc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:36.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d165f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0cc106f10 0x7fb0cc19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46730/0 (socket says 172.21.15.103:46730) 2024-09-19T06:47:36.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d165f640 1 -- 172.21.15.103:0/3592076754 learned_addr learned my addr 172.21.15.103:0/3592076754 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:36.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d0e5e640 1 --2- 172.21.15.103:0/3592076754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0cc101de0 0x7fb0cc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:36.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0cbfff640 1 --2- 172.21.15.103:0/3592076754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0cc102790 0x7fb0cc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:36.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d165f640 1 -- 172.21.15.103:0/3592076754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0cc101de0 msgr2=0x7fb0cc198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:36.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d165f640 1 --2- 172.21.15.103:0/3592076754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0cc101de0 0x7fb0cc198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:36.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d165f640 1 -- 172.21.15.103:0/3592076754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0cc102790 msgr2=0x7fb0cc1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:36.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d165f640 1 --2- 172.21.15.103:0/3592076754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0cc102790 0x7fb0cc1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:36.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.267+0000 7fb0d165f640 1 -- 172.21.15.103:0/3592076754 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0cc19d440 con 0x7fb0cc106f10 2024-09-19T06:47:36.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.268+0000 7fb0d165f640 1 --2- 172.21.15.103:0/3592076754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0cc106f10 0x7fb0cc19cd60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400b970 tx=0x7fb0c400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:36.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.268+0000 7fb0c9ffb640 1 -- 172.21.15.103:0/3592076754 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4019070 con 0x7fb0cc106f10 2024-09-19T06:47:36.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.268+0000 7fb0c9ffb640 1 -- 172.21.15.103:0/3592076754 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0c400cb40 con 0x7fb0cc106f10 2024-09-19T06:47:36.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.268+0000 7fb0d30c2640 1 -- 172.21.15.103:0/3592076754 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0cc19d730 con 0x7fb0cc106f10 2024-09-19T06:47:36.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.268+0000 7fb0c9ffb640 1 -- 172.21.15.103:0/3592076754 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4007650 con 0x7fb0cc106f10 2024-09-19T06:47:36.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.268+0000 7fb0d30c2640 1 -- 172.21.15.103:0/3592076754 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0cc1a5010 con 0x7fb0cc106f10 2024-09-19T06:47:36.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.269+0000 7fb0d30c2640 1 -- 172.21.15.103:0/3592076754 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0cc1021c0 con 0x7fb0cc106f10 2024-09-19T06:47:36.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.270+0000 7fb0c9ffb640 1 -- 172.21.15.103:0/3592076754 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb0c401c070 con 0x7fb0cc106f10 2024-09-19T06:47:36.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.270+0000 7fb0c9ffb640 1 --2- 172.21.15.103:0/3592076754 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb0bc0a3240 0x7fb0bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:36.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.270+0000 7fb0d0e5e640 1 --2- 172.21.15.103:0/3592076754 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb0bc0a3240 0x7fb0bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:36.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.271+0000 7fb0c9ffb640 1 -- 172.21.15.103:0/3592076754 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fb0c40cedd0 con 0x7fb0cc106f10 2024-09-19T06:47:36.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.271+0000 7fb0d0e5e640 1 --2- 172.21.15.103:0/3592076754 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb0bc0a3240 0x7fb0bc0a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00058c0 tx=0x7fb0c000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:36.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.275+0000 7fb0c9ffb640 1 -- 172.21.15.103:0/3592076754 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb0c409b5f0 con 0x7fb0cc106f10 2024-09-19T06:47:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:35 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1262078838' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T06:47:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:35 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] up:active 2024-09-19T06:47:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:35 smithi174 ceph-mon[35637]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T06:47:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:35 smithi174 ceph-mon[35637]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-19T06:47:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:35 smithi174 ceph-mon[35637]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-19T06:47:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:35 smithi174 ceph-mon[35637]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-19T06:47:36.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:36.480+0000 7fb0d30c2640 1 -- 172.21.15.103:0/3592076754 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb0cc10a150 con 0x7fb0cc106f10 2024-09-19T06:47:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:36 smithi134 ceph-mon[35356]: pgmap v278: 97 pgs: 97 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 7 op/s 2024-09-19T06:47:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:36 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] up:active 2024-09-19T06:47:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:36 smithi134 ceph-mon[35356]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T06:47:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:36 smithi134 ceph-mon[35356]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T06:47:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:36 smithi134 ceph-mon[35356]: Cluster is now healthy 2024-09-19T06:47:37.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:36 smithi134 ceph-mon[35356]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-19T06:47:37.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:36 smithi134 ceph-mon[35356]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T06:47:37.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:36 smithi134 ceph-mon[35356]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-19T06:47:37.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:36 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] up:active 2024-09-19T06:47:37.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:36 smithi134 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:47:37.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:36 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3592076754' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T06:47:37.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:36 smithi174 ceph-mon[35637]: pgmap v278: 97 pgs: 97 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 7 op/s 2024-09-19T06:47:37.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:36 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] up:active 2024-09-19T06:47:37.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:36 smithi174 ceph-mon[35637]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T06:47:37.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:36 smithi174 ceph-mon[35637]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T06:47:37.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:36 smithi174 ceph-mon[35637]: Cluster is now healthy 2024-09-19T06:47:37.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:36 smithi174 ceph-mon[35637]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-19T06:47:37.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:36 smithi174 ceph-mon[35637]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T06:47:37.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:36 smithi174 ceph-mon[35637]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-19T06:47:37.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:36 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] up:active 2024-09-19T06:47:37.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:36 smithi174 ceph-mon[35637]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:47:37.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:36 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3592076754' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T06:47:37.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:36 smithi103 ceph-mon[31158]: pgmap v278: 97 pgs: 97 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 7 op/s 2024-09-19T06:47:37.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:36 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] up:active 2024-09-19T06:47:37.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:36 smithi103 ceph-mon[31158]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T06:47:37.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:36 smithi103 ceph-mon[31158]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T06:47:37.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:36 smithi103 ceph-mon[31158]: Cluster is now healthy 2024-09-19T06:47:37.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:36 smithi103 ceph-mon[31158]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-19T06:47:37.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:36 smithi103 ceph-mon[31158]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T06:47:37.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:36 smithi103 ceph-mon[31158]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-19T06:47:37.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:36 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] up:active 2024-09-19T06:47:37.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:36 smithi103 ceph-mon[31158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:47:37.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:36 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3592076754' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T06:47:37.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:37.460+0000 7fb0c9ffb640 1 -- 172.21.15.103:0/3592076754 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fb0c409f8f0 con 0x7fb0cc106f10 2024-09-19T06:47:37.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:37.462+0000 7fb0d30c2640 1 -- 172.21.15.103:0/3592076754 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb0bc0a3240 msgr2=0x7fb0bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:37.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:37.462+0000 7fb0d30c2640 1 --2- 172.21.15.103:0/3592076754 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb0bc0a3240 0x7fb0bc0a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00058c0 tx=0x7fb0c000a250 comp rx=0 tx=0).stop 2024-09-19T06:47:37.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:37.462+0000 7fb0d30c2640 1 -- 172.21.15.103:0/3592076754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0cc106f10 msgr2=0x7fb0cc19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:37.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:37.462+0000 7fb0d30c2640 1 --2- 172.21.15.103:0/3592076754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0cc106f10 0x7fb0cc19cd60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400b970 tx=0x7fb0c400be30 comp rx=0 tx=0).stop 2024-09-19T06:47:37.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:37.463+0000 7fb0d30c2640 1 -- 172.21.15.103:0/3592076754 shutdown_connections 2024-09-19T06:47:37.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:37.463+0000 7fb0d30c2640 1 --2- 172.21.15.103:0/3592076754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0cc106f10 0x7fb0cc19cd60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:37.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:37.463+0000 7fb0d30c2640 1 --2- 172.21.15.103:0/3592076754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0cc102790 0x7fb0cc1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:37.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:37.463+0000 7fb0d30c2640 1 --2- 172.21.15.103:0/3592076754 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb0bc0a3240 0x7fb0bc0a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:37.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:37.463+0000 7fb0d30c2640 1 --2- 172.21.15.103:0/3592076754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0cc101de0 0x7fb0cc198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:37.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:37.463+0000 7fb0d30c2640 1 -- 172.21.15.103:0/3592076754 >> 172.21.15.103:0/3592076754 conn(0x7fb0cc0fd8c0 msgr2=0x7fb0cc0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:37.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:37.463+0000 7fb0d30c2640 1 -- 172.21.15.103:0/3592076754 shutdown_connections 2024-09-19T06:47:37.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:37.463+0000 7fb0d30c2640 1 -- 172.21.15.103:0/3592076754 wait complete. 2024-09-19T06:47:38.085 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-09-19T06:47:38.085 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T06:47:38.332 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:38.760 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:38 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3592076754' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T06:47:38.760 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:38 smithi103 ceph-mon[31158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:47:38.760 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:38 smithi103 ceph-mon[31158]: pgmap v279: 97 pgs: 97 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-09-19T06:47:38.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:38 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3592076754' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T06:47:38.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:38 smithi174 ceph-mon[35637]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:47:38.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:38 smithi174 ceph-mon[35637]: pgmap v279: 97 pgs: 97 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-09-19T06:47:38.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.843+0000 7fde37877640 1 -- 172.21.15.103:0/717103392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde30106f50 msgr2=0x7fde3010eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:38.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.843+0000 7fde37877640 1 --2- 172.21.15.103:0/717103392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde30106f50 0x7fde3010eaf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fde28009cd0 tx=0x7fde2802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:47:38.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.843+0000 7fde37877640 1 -- 172.21.15.103:0/717103392 shutdown_connections 2024-09-19T06:47:38.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.843+0000 7fde37877640 1 --2- 172.21.15.103:0/717103392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde30106f50 0x7fde3010eaf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:38.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.843+0000 7fde37877640 1 --2- 172.21.15.103:0/717103392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde30100ca0 0x7fde30106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:38.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.843+0000 7fde37877640 1 --2- 172.21.15.103:0/717103392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde301002f0 0x7fde301006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:38.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.843+0000 7fde37877640 1 -- 172.21.15.103:0/717103392 >> 172.21.15.103:0/717103392 conn(0x7fde30077210 msgr2=0x7fde300fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:38.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.843+0000 7fde37877640 1 -- 172.21.15.103:0/717103392 shutdown_connections 2024-09-19T06:47:38.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.843+0000 7fde37877640 1 -- 172.21.15.103:0/717103392 wait complete. 2024-09-19T06:47:38.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.843+0000 7fde37877640 1 Processor -- start 2024-09-19T06:47:38.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde37877640 1 -- start start 2024-09-19T06:47:38.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde37877640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde301002f0 0x7fde301985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:38.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde37877640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde30100ca0 0x7fde30198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:38.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde37877640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde30106f50 0x7fde3019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:38.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde37877640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde30112080 con 0x7fde301002f0 2024-09-19T06:47:38.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde37877640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde30111f00 con 0x7fde30100ca0 2024-09-19T06:47:38.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde37877640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde30112200 con 0x7fde30106f50 2024-09-19T06:47:38.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde35613640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde301002f0 0x7fde301985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:38.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde35613640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde301002f0 0x7fde301985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53698/0 (socket says 172.21.15.103:53698) 2024-09-19T06:47:38.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde35613640 1 -- 172.21.15.103:0/1581497207 learned_addr learned my addr 172.21.15.103:0/1581497207 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:38.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde35e14640 1 --2- 172.21.15.103:0/1581497207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde30106f50 0x7fde3019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:38.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde34e12640 1 --2- 172.21.15.103:0/1581497207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde30100ca0 0x7fde30198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:38.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde35613640 1 -- 172.21.15.103:0/1581497207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde30106f50 msgr2=0x7fde3019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:38.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde35613640 1 --2- 172.21.15.103:0/1581497207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde30106f50 0x7fde3019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:38.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde35613640 1 -- 172.21.15.103:0/1581497207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde30100ca0 msgr2=0x7fde30198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:38.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde35613640 1 --2- 172.21.15.103:0/1581497207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde30100ca0 0x7fde30198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:38.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde35613640 1 -- 172.21.15.103:0/1581497207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3019d570 con 0x7fde301002f0 2024-09-19T06:47:38.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde34e12640 1 --2- 172.21.15.103:0/1581497207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde30100ca0 0x7fde30198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:38.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde35613640 1 --2- 172.21.15.103:0/1581497207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde301002f0 0x7fde301985c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00b4a0 tx=0x7fde2c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:38.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde267fc640 1 -- 172.21.15.103:0/1581497207 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c014070 con 0x7fde301002f0 2024-09-19T06:47:38.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde37877640 1 -- 172.21.15.103:0/1581497207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde3019d860 con 0x7fde301002f0 2024-09-19T06:47:38.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.844+0000 7fde267fc640 1 -- 172.21.15.103:0/1581497207 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde2c010af0 con 0x7fde301002f0 2024-09-19T06:47:38.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.845+0000 7fde267fc640 1 -- 172.21.15.103:0/1581497207 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c017d10 con 0x7fde301002f0 2024-09-19T06:47:38.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.845+0000 7fde37877640 1 -- 172.21.15.103:0/1581497207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde301a5140 con 0x7fde301002f0 2024-09-19T06:47:38.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.846+0000 7fde37877640 1 -- 172.21.15.103:0/1581497207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde30100750 con 0x7fde301002f0 2024-09-19T06:47:38.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.847+0000 7fde267fc640 1 -- 172.21.15.103:0/1581497207 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fde2c017070 con 0x7fde301002f0 2024-09-19T06:47:38.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.847+0000 7fde267fc640 1 --2- 172.21.15.103:0/1581497207 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fde1c0a3240 0x7fde1c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:38.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.847+0000 7fde34e12640 1 --2- 172.21.15.103:0/1581497207 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fde1c0a3240 0x7fde1c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:38.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.847+0000 7fde267fc640 1 -- 172.21.15.103:0/1581497207 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fde2c0cecb0 con 0x7fde301002f0 2024-09-19T06:47:38.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.848+0000 7fde34e12640 1 --2- 172.21.15.103:0/1581497207 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fde1c0a3240 0x7fde1c0a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fde20006fd0 tx=0x7fde20008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:38.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:38.852+0000 7fde267fc640 1 -- 172.21.15.103:0/1581497207 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fde2c09bb20 con 0x7fde301002f0 2024-09-19T06:47:38.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:38 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3592076754' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T06:47:38.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:38 smithi134 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:47:38.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:38 smithi134 ceph-mon[35356]: pgmap v279: 97 pgs: 97 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-09-19T06:47:39.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:39.045+0000 7fde37877640 1 -- 172.21.15.103:0/1581497207 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fde3010a1b0 con 0x7fde1c0a3240 2024-09-19T06:47:39.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:39.074+0000 7fde267fc640 1 -- 172.21.15.103:0/1581497207 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fde2000ae20 con 0x7fde1c0a3240 2024-09-19T06:47:39.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:39.076+0000 7fde37877640 1 -- 172.21.15.103:0/1581497207 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fde1c0a3240 msgr2=0x7fde1c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:39.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:39.076+0000 7fde37877640 1 --2- 172.21.15.103:0/1581497207 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fde1c0a3240 0x7fde1c0a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fde20006fd0 tx=0x7fde20008040 comp rx=0 tx=0).stop 2024-09-19T06:47:39.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:39.076+0000 7fde37877640 1 -- 172.21.15.103:0/1581497207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde301002f0 msgr2=0x7fde301985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:39.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:39.076+0000 7fde37877640 1 --2- 172.21.15.103:0/1581497207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde301002f0 0x7fde301985c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00b4a0 tx=0x7fde2c00b960 comp rx=0 tx=0).stop 2024-09-19T06:47:39.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:39.077+0000 7fde37877640 1 -- 172.21.15.103:0/1581497207 shutdown_connections 2024-09-19T06:47:39.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:39.077+0000 7fde37877640 1 --2- 172.21.15.103:0/1581497207 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fde1c0a3240 0x7fde1c0a5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:39.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:39.077+0000 7fde37877640 1 --2- 172.21.15.103:0/1581497207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde30106f50 0x7fde3019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:39.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:39.077+0000 7fde37877640 1 --2- 172.21.15.103:0/1581497207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde30100ca0 0x7fde30198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:39.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:39.077+0000 7fde37877640 1 --2- 172.21.15.103:0/1581497207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde301002f0 0x7fde301985c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:39.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:39.077+0000 7fde37877640 1 -- 172.21.15.103:0/1581497207 >> 172.21.15.103:0/1581497207 conn(0x7fde30077210 msgr2=0x7fde301051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:39.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:39.077+0000 7fde37877640 1 -- 172.21.15.103:0/1581497207 shutdown_connections 2024-09-19T06:47:39.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:39.077+0000 7fde37877640 1 -- 172.21.15.103:0/1581497207 wait complete. 2024-09-19T06:47:39.387 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:47:39 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:47:39.050+0000 7f717e5d4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:47:39.388 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:47:39 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:47:39.050+0000 7f717e5d4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:47:39.388 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:47:39 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:47:39.050+0000 7f717e5d4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:47:39.388 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:47:39 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:47:39.050+0000 7f717e5d4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:47:39.388 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:47:39 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:47:39.050+0000 7f717e5d4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:47:39.650 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T06:47:39.902 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:40.023 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-19T06:47:40.023 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-19T06:47:40.023 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-09-19T06:47:40.023 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-09-19T06:47:40.023 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-19T06:47:40.024 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-19T06:47:40.024 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-09-19T06:47:40.024 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:39 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-19T06:47:40.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:39 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-19T06:47:40.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:39 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-19T06:47:40.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:39 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-09-19T06:47:40.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:39 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-09-19T06:47:40.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:39 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-19T06:47:40.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:39 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-19T06:47:40.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:39 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-09-19T06:47:40.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:39 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-19T06:47:40.187 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 06:47:39 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-b[65626]: 2024-09-19T06:47:39.997+0000 7f2df2ee1640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-19T06:47:40.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:39 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-19T06:47:40.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:39 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-19T06:47:40.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:39 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-09-19T06:47:40.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:39 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-09-19T06:47:40.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:39 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-19T06:47:40.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:39 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-19T06:47:40.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:39 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-09-19T06:47:40.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:39 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-19T06:47:40.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.400+0000 7f8c4527e640 1 -- 172.21.15.103:0/4082767322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c40101420 msgr2=0x7f8c4010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:40.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.400+0000 7f8c4527e640 1 --2- 172.21.15.103:0/4082767322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c40101420 0x7f8c4010d950 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8c38009cd0 tx=0x7f8c3802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:47:40.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.400+0000 7f8c4527e640 1 -- 172.21.15.103:0/4082767322 shutdown_connections 2024-09-19T06:47:40.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.400+0000 7f8c4527e640 1 --2- 172.21.15.103:0/4082767322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c40101420 0x7f8c4010d950 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:40.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.400+0000 7f8c4527e640 1 --2- 172.21.15.103:0/4082767322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c40100a80 0x7f8c40100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:40.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.400+0000 7f8c4527e640 1 --2- 172.21.15.103:0/4082767322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c40106a80 0x7f8c40106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:40.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.400+0000 7f8c4527e640 1 -- 172.21.15.103:0/4082767322 >> 172.21.15.103:0/4082767322 conn(0x7f8c400fc7a0 msgr2=0x7f8c400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:40.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.400+0000 7f8c4527e640 1 -- 172.21.15.103:0/4082767322 shutdown_connections 2024-09-19T06:47:40.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.400+0000 7f8c4527e640 1 -- 172.21.15.103:0/4082767322 wait complete. 2024-09-19T06:47:40.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.401+0000 7f8c4527e640 1 Processor -- start 2024-09-19T06:47:40.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.401+0000 7f8c4527e640 1 -- start start 2024-09-19T06:47:40.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.401+0000 7f8c4527e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c40100a80 0x7f8c40197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:40.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.401+0000 7f8c4527e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c40101420 0x7f8c401978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:40.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.401+0000 7f8c4527e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c40106a80 0x7f8c4019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:40.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.401+0000 7f8c4527e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8c40110ea0 con 0x7f8c40101420 2024-09-19T06:47:40.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.401+0000 7f8c4527e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c40110d20 con 0x7f8c40106a80 2024-09-19T06:47:40.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.401+0000 7f8c4527e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c40111020 con 0x7f8c40100a80 2024-09-19T06:47:40.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.401+0000 7f8c3e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c40101420 0x7f8c401978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:40.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.401+0000 7f8c3e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c40101420 0x7f8c401978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53732/0 (socket says 172.21.15.103:53732) 2024-09-19T06:47:40.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.401+0000 7f8c3e7fc640 1 -- 172.21.15.103:0/592603109 learned_addr learned my addr 172.21.15.103:0/592603109 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:40.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.401+0000 7f8c3f7fe640 1 --2- 172.21.15.103:0/592603109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c40106a80 0x7f8c4019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:40.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.402+0000 7f8c3effd640 1 --2- 172.21.15.103:0/592603109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c40100a80 0x7f8c40197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:40.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.402+0000 7f8c3e7fc640 1 -- 172.21.15.103:0/592603109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c40100a80 msgr2=0x7f8c40197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:40.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.402+0000 7f8c3e7fc640 1 --2- 172.21.15.103:0/592603109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c40100a80 0x7f8c40197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:40.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.402+0000 7f8c3e7fc640 1 -- 172.21.15.103:0/592603109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c40106a80 msgr2=0x7f8c4019bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:40.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.402+0000 7f8c3e7fc640 1 --2- 172.21.15.103:0/592603109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c40106a80 0x7f8c4019bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:40.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.402+0000 7f8c3e7fc640 1 -- 172.21.15.103:0/592603109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4019c330 con 0x7f8c40101420 2024-09-19T06:47:40.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.402+0000 7f8c3f7fe640 1 --2- 172.21.15.103:0/592603109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c40106a80 0x7f8c4019bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:40.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.402+0000 7f8c3e7fc640 1 --2- 172.21.15.103:0/592603109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c40101420 0x7f8c401978c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400b4a0 tx=0x7f8c3400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:40.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.402+0000 7f8c27fff640 1 -- 172.21.15.103:0/592603109 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34014070 con 0x7f8c40101420 2024-09-19T06:47:40.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.402+0000 7f8c27fff640 1 -- 172.21.15.103:0/592603109 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c34010af0 con 0x7f8c40101420 2024-09-19T06:47:40.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.402+0000 7f8c4527e640 1 -- 172.21.15.103:0/592603109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c4019c620 con 0x7f8c40101420 2024-09-19T06:47:40.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.402+0000 7f8c27fff640 1 -- 172.21.15.103:0/592603109 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34017d10 con 0x7f8c40101420 2024-09-19T06:47:40.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.402+0000 7f8c4527e640 1 -- 172.21.15.103:0/592603109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c401a3f00 con 0x7f8c40101420 2024-09-19T06:47:40.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.403+0000 7f8c4527e640 1 -- 172.21.15.103:0/592603109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c40100ee0 con 0x7f8c40101420 2024-09-19T06:47:40.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.404+0000 7f8c27fff640 1 -- 172.21.15.103:0/592603109 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f8c34017070 con 0x7f8c40101420 2024-09-19T06:47:40.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.405+0000 7f8c27fff640 1 --2- 172.21.15.103:0/592603109 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8c280a3240 0x7f8c280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:40.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.405+0000 7f8c3effd640 1 --2- 172.21.15.103:0/592603109 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8c280a3240 0x7f8c280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:40.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.405+0000 7f8c27fff640 1 -- 172.21.15.103:0/592603109 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8211+0+0 (secure 0 0 0) 0x7f8c340cecd0 con 0x7f8c40101420 2024-09-19T06:47:40.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.405+0000 7f8c3effd640 1 --2- 172.21.15.103:0/592603109 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8c280a3240 0x7f8c280a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8c30005fd0 tx=0x7f8c30005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:40.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.409+0000 7f8c27fff640 1 -- 172.21.15.103:0/592603109 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8c3409ba00 con 0x7f8c40101420 2024-09-19T06:47:40.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.602+0000 7f8c4527e640 1 -- 172.21.15.103:0/592603109 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8c40062630 con 0x7f8c280a3240 2024-09-19T06:47:40.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.627+0000 7f8c27fff640 1 -- 172.21.15.103:0/592603109 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8c40062630 con 0x7f8c280a3240 2024-09-19T06:47:40.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.629+0000 7f8c4527e640 1 -- 172.21.15.103:0/592603109 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8c280a3240 msgr2=0x7f8c280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:40.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.629+0000 7f8c4527e640 1 --2- 172.21.15.103:0/592603109 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8c280a3240 0x7f8c280a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8c30005fd0 tx=0x7f8c30005850 comp rx=0 tx=0).stop 2024-09-19T06:47:40.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.629+0000 7f8c4527e640 1 -- 172.21.15.103:0/592603109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c40101420 msgr2=0x7f8c401978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:40.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.629+0000 7f8c4527e640 1 --2- 172.21.15.103:0/592603109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c40101420 0x7f8c401978c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400b4a0 tx=0x7f8c3400b960 comp rx=0 tx=0).stop 2024-09-19T06:47:40.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.629+0000 7f8c4527e640 1 -- 172.21.15.103:0/592603109 shutdown_connections 2024-09-19T06:47:40.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.629+0000 7f8c4527e640 1 --2- 172.21.15.103:0/592603109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c40106a80 0x7f8c4019bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:40.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.629+0000 7f8c4527e640 1 --2- 172.21.15.103:0/592603109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c40101420 0x7f8c401978c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:40.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.629+0000 7f8c4527e640 1 --2- 172.21.15.103:0/592603109 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8c280a3240 0x7f8c280a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:40.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.629+0000 7f8c4527e640 1 --2- 172.21.15.103:0/592603109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c40100a80 0x7f8c40197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:40.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.629+0000 7f8c4527e640 1 -- 172.21.15.103:0/592603109 >> 172.21.15.103:0/592603109 conn(0x7f8c400fc7a0 msgr2=0x7f8c400fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:40.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.629+0000 7f8c4527e640 1 -- 172.21.15.103:0/592603109 shutdown_connections 2024-09-19T06:47:40.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:40.629+0000 7f8c4527e640 1 -- 172.21.15.103:0/592603109 wait complete. 2024-09-19T06:47:40.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:40 smithi103 ceph-mon[31158]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:47:40.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-19T06:47:40.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-19T06:47:40.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-09-19T06:47:40.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-09-19T06:47:40.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-19T06:47:40.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-19T06:47:40.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-19T06:47:40.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:40 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-19T06:47:40.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:40 smithi103 ceph-mon[31158]: pgmap v281: 97 pgs: 97 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-09-19T06:47:40.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:40 smithi103 ceph-mon[31158]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T06:47:41.185 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:40 smithi134 ceph-mon[35356]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:47:41.185 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-19T06:47:41.185 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-19T06:47:41.185 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-09-19T06:47:41.185 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-09-19T06:47:41.185 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-19T06:47:41.185 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-19T06:47:41.186 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-19T06:47:41.186 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:40 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-19T06:47:41.186 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:40 smithi134 ceph-mon[35356]: pgmap v281: 97 pgs: 97 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-09-19T06:47:41.186 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:40 smithi134 ceph-mon[35356]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T06:47:41.209 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T06:47:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:40 smithi174 ceph-mon[35637]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:47:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-19T06:47:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-19T06:47:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-09-19T06:47:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-09-19T06:47:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-19T06:47:41.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-19T06:47:41.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-19T06:47:41.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:40 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-19T06:47:41.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:40 smithi174 ceph-mon[35637]: pgmap v281: 97 pgs: 97 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-09-19T06:47:41.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:40 smithi174 ceph-mon[35637]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T06:47:41.456 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:41.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.953+0000 7f4add4e4640 1 -- 172.21.15.103:0/4216938350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad8100a80 msgr2=0x7f4ad8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:41.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.953+0000 7f4add4e4640 1 --2- 172.21.15.103:0/4216938350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad8100a80 0x7f4ad8100ee0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009930 tx=0x7f4acc02a420 comp rx=0 tx=0).stop 2024-09-19T06:47:41.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.953+0000 7f4add4e4640 1 -- 172.21.15.103:0/4216938350 shutdown_connections 2024-09-19T06:47:41.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.953+0000 7f4add4e4640 1 --2- 172.21.15.103:0/4216938350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad8101420 0x7f4ad810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.953+0000 7f4add4e4640 1 --2- 172.21.15.103:0/4216938350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad8100a80 0x7f4ad8100ee0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.953+0000 7f4add4e4640 1 --2- 172.21.15.103:0/4216938350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ad8106a80 0x7f4ad8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.953+0000 7f4add4e4640 1 -- 172.21.15.103:0/4216938350 >> 172.21.15.103:0/4216938350 conn(0x7f4ad80fc7a0 msgr2=0x7f4ad80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:41.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.953+0000 7f4add4e4640 1 -- 172.21.15.103:0/4216938350 shutdown_connections 2024-09-19T06:47:41.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.953+0000 7f4add4e4640 1 -- 172.21.15.103:0/4216938350 wait complete. 2024-09-19T06:47:41.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.954+0000 7f4add4e4640 1 Processor -- start 2024-09-19T06:47:41.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.954+0000 7f4add4e4640 1 -- start start 2024-09-19T06:47:41.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.954+0000 7f4add4e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad8100a80 0x7f4ad8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.954+0000 7f4add4e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ad8101420 0x7f4ad81998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.954+0000 7f4add4e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad8106a80 0x7f4ad81a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.954+0000 7f4add4e4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8110ea0 con 0x7f4ad8100a80 2024-09-19T06:47:41.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.954+0000 7f4add4e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8110d20 con 0x7f4ad8101420 2024-09-19T06:47:41.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.954+0000 7f4add4e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8111020 con 0x7f4ad8106a80 2024-09-19T06:47:41.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.955+0000 7f4ad6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad8100a80 0x7f4ad8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.955+0000 7f4ad6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad8100a80 0x7f4ad8199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53752/0 (socket says 172.21.15.103:53752) 2024-09-19T06:47:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.955+0000 7f4ad6ffd640 1 -- 172.21.15.103:0/3352389336 learned_addr learned my addr 172.21.15.103:0/3352389336 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.955+0000 7f4ad67fc640 1 --2- 172.21.15.103:0/3352389336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ad8101420 0x7f4ad81998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.955+0000 7f4ad77fe640 1 --2- 172.21.15.103:0/3352389336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad8106a80 0x7f4ad81a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.955+0000 7f4ad6ffd640 1 -- 172.21.15.103:0/3352389336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad8106a80 msgr2=0x7f4ad81a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.955+0000 7f4ad6ffd640 1 --2- 172.21.15.103:0/3352389336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad8106a80 0x7f4ad81a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.955+0000 7f4ad6ffd640 1 -- 172.21.15.103:0/3352389336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ad8101420 msgr2=0x7f4ad81998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.955+0000 7f4ad6ffd640 1 --2- 172.21.15.103:0/3352389336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ad8101420 0x7f4ad81998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.955+0000 7f4ad6ffd640 1 -- 172.21.15.103:0/3352389336 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad81a4950 con 0x7f4ad8100a80 2024-09-19T06:47:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.955+0000 7f4ad6ffd640 1 --2- 172.21.15.103:0/3352389336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad8100a80 0x7f4ad8199390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800eab0 tx=0x7f4ac800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.955+0000 7f4abffff640 1 -- 172.21.15.103:0/3352389336 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac800ce40 con 0x7f4ad8100a80 2024-09-19T06:47:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.955+0000 7f4add4e4640 1 -- 172.21.15.103:0/3352389336 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad81a4b20 con 0x7f4ad8100a80 2024-09-19T06:47:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.955+0000 7f4abffff640 1 -- 172.21.15.103:0/3352389336 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac8011950 con 0x7f4ad8100a80 2024-09-19T06:47:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.955+0000 7f4abffff640 1 -- 172.21.15.103:0/3352389336 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8010ae0 con 0x7f4ad8100a80 2024-09-19T06:47:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.955+0000 7f4add4e4640 1 -- 172.21.15.103:0/3352389336 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad81a5030 con 0x7f4ad8100a80 2024-09-19T06:47:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.956+0000 7f4add4e4640 1 -- 172.21.15.103:0/3352389336 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad81021c0 con 0x7f4ad8100a80 2024-09-19T06:47:41.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.958+0000 7f4abffff640 1 -- 172.21.15.103:0/3352389336 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4ac8011b00 con 0x7f4ad8100a80 2024-09-19T06:47:41.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.958+0000 7f4abffff640 1 --2- 172.21.15.103:0/3352389336 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4ac00a32d0 0x7f4ac00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.958+0000 7f4ad67fc640 1 --2- 172.21.15.103:0/3352389336 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4ac00a32d0 0x7f4ac00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.958+0000 7f4abffff640 1 -- 172.21.15.103:0/3352389336 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4ac80d0160 con 0x7f4ad8100a80 2024-09-19T06:47:41.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.959+0000 7f4ad67fc640 1 --2- 172.21.15.103:0/3352389336 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4ac00a32d0 0x7f4ac00a5790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009900 tx=0x7f4acc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:41.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:41.962+0000 7f4abffff640 1 -- 172.21.15.103:0/3352389336 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4ac8016030 con 0x7f4ad8100a80 2024-09-19T06:47:42.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:41 smithi103 ceph-mon[31158]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:47:42.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:41 smithi103 ceph-mon[31158]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T06:47:42.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:41 smithi103 ceph-mon[31158]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T06:47:42.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:42.155+0000 7f4add4e4640 1 -- 172.21.15.103:0/3352389336 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f4ad8109080 con 0x7f4ac00a32d0 2024-09-19T06:47:42.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:42.161+0000 7f4abffff640 1 -- 172.21.15.103:0/3352389336 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4acc031730 con 0x7f4ac00a32d0 2024-09-19T06:47:42.162 INFO:teuthology.orchestra.run.smithi103.stdout:/volumes/qa/sv_0/52de0da2-d862-4475-b9b6-d62af055b481 2024-09-19T06:47:42.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:42.162+0000 7f4add4e4640 1 -- 172.21.15.103:0/3352389336 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4ac00a32d0 msgr2=0x7f4ac00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:42.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:42.162+0000 7f4add4e4640 1 --2- 172.21.15.103:0/3352389336 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4ac00a32d0 0x7f4ac00a5790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009900 tx=0x7f4acc0058d0 comp rx=0 tx=0).stop 2024-09-19T06:47:42.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:42.162+0000 7f4add4e4640 1 -- 172.21.15.103:0/3352389336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad8100a80 msgr2=0x7f4ad8199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:42.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:42.162+0000 7f4add4e4640 1 --2- 172.21.15.103:0/3352389336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad8100a80 0x7f4ad8199390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800eab0 tx=0x7f4ac800ef70 comp rx=0 tx=0).stop 2024-09-19T06:47:42.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:42.162+0000 7f4add4e4640 1 -- 172.21.15.103:0/3352389336 shutdown_connections 2024-09-19T06:47:42.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:42.162+0000 7f4add4e4640 1 --2- 172.21.15.103:0/3352389336 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4ac00a32d0 0x7f4ac00a5790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:42.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:42.162+0000 7f4add4e4640 1 --2- 172.21.15.103:0/3352389336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad8106a80 0x7f4ad81a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:42.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:42.162+0000 7f4add4e4640 1 --2- 172.21.15.103:0/3352389336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ad8101420 0x7f4ad81998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:42.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:42.162+0000 7f4add4e4640 1 --2- 172.21.15.103:0/3352389336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad8100a80 0x7f4ad8199390 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:42.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:42.162+0000 7f4add4e4640 1 -- 172.21.15.103:0/3352389336 >> 172.21.15.103:0/3352389336 conn(0x7f4ad80fc7a0 msgr2=0x7f4ad80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:42.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:42.162+0000 7f4add4e4640 1 -- 172.21.15.103:0/3352389336 shutdown_connections 2024-09-19T06:47:42.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:42.162+0000 7f4add4e4640 1 -- 172.21.15.103:0/3352389336 wait complete. 2024-09-19T06:47:42.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:41 smithi134 ceph-mon[35356]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:47:42.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:41 smithi134 ceph-mon[35356]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T06:47:42.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:41 smithi134 ceph-mon[35356]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T06:47:42.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:41 smithi174 ceph-mon[35637]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:47:42.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:41 smithi174 ceph-mon[35637]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T06:47:42.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:41 smithi174 ceph-mon[35637]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T06:47:42.732 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T06:47:42.982 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:43.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:42 smithi103 ceph-mon[31158]: pgmap v283: 97 pgs: 1 activating+degraded, 96 active+clean; 591 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.2 KiB/s wr, 28 op/s; 6/300 objects degraded (2.000%); 25 B/s, 2 objects/s recovering 2024-09-19T06:47:43.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:42 smithi103 ceph-mon[31158]: Health check failed: Degraded data redundancy: 6/300 objects degraded (2.000%), 1 pg degraded (PG_DEGRADED) 2024-09-19T06:47:43.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:42 smithi134 ceph-mon[35356]: pgmap v283: 97 pgs: 1 activating+degraded, 96 active+clean; 591 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.2 KiB/s wr, 28 op/s; 6/300 objects degraded (2.000%); 25 B/s, 2 objects/s recovering 2024-09-19T06:47:43.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:42 smithi134 ceph-mon[35356]: Health check failed: Degraded data redundancy: 6/300 objects degraded (2.000%), 1 pg degraded (PG_DEGRADED) 2024-09-19T06:47:43.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:42 smithi174 ceph-mon[35637]: pgmap v283: 97 pgs: 1 activating+degraded, 96 active+clean; 591 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.2 KiB/s wr, 28 op/s; 6/300 objects degraded (2.000%); 25 B/s, 2 objects/s recovering 2024-09-19T06:47:43.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:42 smithi174 ceph-mon[35637]: Health check failed: Degraded data redundancy: 6/300 objects degraded (2.000%), 1 pg degraded (PG_DEGRADED) 2024-09-19T06:47:43.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.458+0000 7f8fab74c640 1 -- 172.21.15.103:0/222030481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4108160 msgr2=0x7f8fa410eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:43.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.458+0000 7f8fab74c640 1 --2- 172.21.15.103:0/222030481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4108160 0x7f8fa410eb00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009930 tx=0x7f8f9c02a440 comp rx=0 tx=0).stop 2024-09-19T06:47:43.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.459+0000 7f8fab74c640 1 -- 172.21.15.103:0/222030481 shutdown_connections 2024-09-19T06:47:43.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.459+0000 7f8fab74c640 1 --2- 172.21.15.103:0/222030481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4108160 0x7f8fa410eb00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:43.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.459+0000 7f8fab74c640 1 --2- 172.21.15.103:0/222030481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fa4101290 0x7f8fa4107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:43.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.459+0000 7f8fab74c640 1 --2- 172.21.15.103:0/222030481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa41008e0 0x7f8fa4100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:43.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.459+0000 7f8fab74c640 1 -- 172.21.15.103:0/222030481 >> 172.21.15.103:0/222030481 conn(0x7f8fa4077210 msgr2=0x7f8fa4077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:43.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.459+0000 7f8fab74c640 1 -- 172.21.15.103:0/222030481 shutdown_connections 2024-09-19T06:47:43.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.459+0000 7f8fab74c640 1 -- 172.21.15.103:0/222030481 wait complete. 2024-09-19T06:47:43.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.459+0000 7f8fab74c640 1 Processor -- start 2024-09-19T06:47:43.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.459+0000 7f8fab74c640 1 -- start start 2024-09-19T06:47:43.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.459+0000 7f8fab74c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fa41008e0 0x7f8fa4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:43.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.459+0000 7f8fab74c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4101290 0x7f8fa41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.459+0000 7f8fab74c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa4108160 0x7f8fa419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.459+0000 7f8fab74c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa4112010 con 0x7f8fa4101290 2024-09-19T06:47:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.459+0000 7f8fab74c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa4111e90 con 0x7f8fa4108160 2024-09-19T06:47:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.459+0000 7f8fab74c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa4112190 con 0x7f8fa41008e0 2024-09-19T06:47:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8fa8ce7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4101290 0x7f8fa41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8fa8ce7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4101290 0x7f8fa41989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53768/0 (socket says 172.21.15.103:53768) 2024-09-19T06:47:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8fa8ce7640 1 -- 172.21.15.103:0/666265438 learned_addr learned my addr 172.21.15.103:0/666265438 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8fa9ce9640 1 --2- 172.21.15.103:0/666265438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa4108160 0x7f8fa419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8fa94e8640 1 --2- 172.21.15.103:0/666265438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fa41008e0 0x7f8fa4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8fa8ce7640 1 -- 172.21.15.103:0/666265438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fa41008e0 msgr2=0x7f8fa4198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8fa8ce7640 1 --2- 172.21.15.103:0/666265438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fa41008e0 0x7f8fa4198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8fa8ce7640 1 -- 172.21.15.103:0/666265438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa4108160 msgr2=0x7f8fa419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:43.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8fa8ce7640 1 --2- 172.21.15.103:0/666265438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa4108160 0x7f8fa419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:43.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8fa8ce7640 1 -- 172.21.15.103:0/666265438 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa419d430 con 0x7f8fa4101290 2024-09-19T06:47:43.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8fa9ce9640 1 --2- 172.21.15.103:0/666265438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa4108160 0x7f8fa419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:43.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8fa8ce7640 1 --2- 172.21.15.103:0/666265438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4101290 0x7f8fa41989c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800bdd0 tx=0x7f8f9800bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:43.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8f967fc640 1 -- 172.21.15.103:0/666265438 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98019070 con 0x7f8fa4101290 2024-09-19T06:47:43.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8f967fc640 1 -- 172.21.15.103:0/666265438 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f98014920 con 0x7f8fa4101290 2024-09-19T06:47:43.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8fab74c640 1 -- 172.21.15.103:0/666265438 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa419d720 con 0x7f8fa4101290 2024-09-19T06:47:43.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8f967fc640 1 -- 172.21.15.103:0/666265438 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98012a20 con 0x7f8fa4101290 2024-09-19T06:47:43.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.460+0000 7f8fab74c640 1 -- 172.21.15.103:0/666265438 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa41a5000 con 0x7f8fa4101290 2024-09-19T06:47:43.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.461+0000 7f8fab74c640 1 -- 172.21.15.103:0/666265438 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa4100cc0 con 0x7f8fa4101290 2024-09-19T06:47:43.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.462+0000 7f8f967fc640 1 -- 172.21.15.103:0/666265438 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8f9801c070 con 0x7f8fa4101290 2024-09-19T06:47:43.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.463+0000 7f8f967fc640 1 --2- 172.21.15.103:0/666265438 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8f900a32d0 0x7f8f900a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:43.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.463+0000 7f8fa94e8640 1 --2- 172.21.15.103:0/666265438 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8f900a32d0 0x7f8f900a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:43.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.464+0000 7f8f967fc640 1 -- 172.21.15.103:0/666265438 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8f980cf340 con 0x7f8fa4101290 2024-09-19T06:47:43.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.464+0000 7f8fa94e8640 1 --2- 172.21.15.103:0/666265438 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8f900a32d0 0x7f8f900a5790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00058c0 tx=0x7f8fa000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:43.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.468+0000 7f8f967fc640 1 -- 172.21.15.103:0/666265438 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8f9809c020 con 0x7f8fa4101290 2024-09-19T06:47:43.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.660+0000 7f8fab74c640 1 -- 172.21.15.103:0/666265438 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8fa4062630 con 0x7f8f900a32d0 2024-09-19T06:47:43.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.686+0000 7f8f967fc640 1 -- 172.21.15.103:0/666265438 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8fa4062630 con 0x7f8f900a32d0 2024-09-19T06:47:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.688+0000 7f8fab74c640 1 -- 172.21.15.103:0/666265438 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8f900a32d0 msgr2=0x7f8f900a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.688+0000 7f8fab74c640 1 --2- 172.21.15.103:0/666265438 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8f900a32d0 0x7f8f900a5790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00058c0 tx=0x7f8fa000a4c0 comp rx=0 tx=0).stop 2024-09-19T06:47:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.688+0000 7f8fab74c640 1 -- 172.21.15.103:0/666265438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4101290 msgr2=0x7f8fa41989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.688+0000 7f8fab74c640 1 --2- 172.21.15.103:0/666265438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4101290 0x7f8fa41989c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800bdd0 tx=0x7f8f9800bec0 comp rx=0 tx=0).stop 2024-09-19T06:47:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.688+0000 7f8fab74c640 1 -- 172.21.15.103:0/666265438 shutdown_connections 2024-09-19T06:47:43.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.688+0000 7f8fab74c640 1 --2- 172.21.15.103:0/666265438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa4108160 0x7f8fa419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.688+0000 7f8fab74c640 1 --2- 172.21.15.103:0/666265438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4101290 0x7f8fa41989c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.688+0000 7f8fab74c640 1 --2- 172.21.15.103:0/666265438 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8f900a32d0 0x7f8f900a5790 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.688+0000 7f8fab74c640 1 --2- 172.21.15.103:0/666265438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fa41008e0 0x7f8fa4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.688+0000 7f8fab74c640 1 -- 172.21.15.103:0/666265438 >> 172.21.15.103:0/666265438 conn(0x7f8fa4077210 msgr2=0x7f8fa40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.688+0000 7f8fab74c640 1 -- 172.21.15.103:0/666265438 shutdown_connections 2024-09-19T06:47:43.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:43.688+0000 7f8fab74c640 1 -- 172.21.15.103:0/666265438 wait complete. 2024-09-19T06:47:43.979 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:43 smithi103 ceph-mon[31158]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:47:44.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:43 smithi134 ceph-mon[35356]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:47:44.299 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T06:47:44.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:43 smithi174 ceph-mon[35637]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:47:44.544 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:44.967 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:44 smithi103 ceph-mon[31158]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:47:44.967 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:44 smithi103 ceph-mon[31158]: pgmap v284: 97 pgs: 1 activating+degraded, 96 active+clean; 592 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 26 op/s; 6/300 objects degraded (2.000%); 25 B/s, 2 objects/s recovering 2024-09-19T06:47:44.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.965+0000 7fb223788640 1 -- 172.21.15.103:0/4101716452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb21c10c170 msgr2=0x7fb21c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:44.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.965+0000 7fb223788640 1 --2- 172.21.15.103:0/4101716452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb21c10c170 0x7fb21c10e610 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb214009ca0 tx=0x7fb21402a950 comp rx=0 tx=0).stop 2024-09-19T06:47:44.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.965+0000 7fb223788640 1 -- 172.21.15.103:0/4101716452 shutdown_connections 2024-09-19T06:47:44.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.965+0000 7fb223788640 1 --2- 172.21.15.103:0/4101716452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb21c10c170 0x7fb21c10e610 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:44.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.965+0000 7fb223788640 1 --2- 172.21.15.103:0/4101716452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb21c103660 0x7fb21c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:44.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.965+0000 7fb223788640 1 --2- 172.21.15.103:0/4101716452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb21c102d40 0x7fb21c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:44.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.965+0000 7fb223788640 1 -- 172.21.15.103:0/4101716452 >> 172.21.15.103:0/4101716452 conn(0x7fb21c0fc7a0 msgr2=0x7fb21c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:44.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.965+0000 7fb223788640 1 -- 172.21.15.103:0/4101716452 shutdown_connections 2024-09-19T06:47:44.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.965+0000 7fb223788640 1 -- 172.21.15.103:0/4101716452 wait complete. 2024-09-19T06:47:44.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.966+0000 7fb223788640 1 Processor -- start 2024-09-19T06:47:44.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.966+0000 7fb223788640 1 -- start start 2024-09-19T06:47:44.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.966+0000 7fb223788640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb21c102d40 0x7fb21c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:44.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.966+0000 7fb223788640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb21c103660 0x7fb21c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:44.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.966+0000 7fb223788640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb21c10c170 0x7fb21c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:44.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.966+0000 7fb223788640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c111fb0 con 0x7fb21c103660 2024-09-19T06:47:44.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.966+0000 7fb223788640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c111e30 con 0x7fb21c10c170 2024-09-19T06:47:44.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.966+0000 7fb223788640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c112130 con 0x7fb21c102d40 2024-09-19T06:47:44.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.966+0000 7fb220d23640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb21c103660 0x7fb21c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:44.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.966+0000 7fb220d23640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb21c103660 0x7fb21c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53786/0 (socket says 172.21.15.103:53786) 2024-09-19T06:47:44.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.966+0000 7fb221d25640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb21c10c170 0x7fb21c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:44.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.966+0000 7fb220d23640 1 -- 172.21.15.103:0/4293491066 learned_addr learned my addr 172.21.15.103:0/4293491066 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:44.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.967+0000 7fb221524640 1 --2- 172.21.15.103:0/4293491066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb21c102d40 0x7fb21c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:44.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.967+0000 7fb220d23640 1 -- 172.21.15.103:0/4293491066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb21c102d40 msgr2=0x7fb21c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:44.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.967+0000 7fb220d23640 1 --2- 172.21.15.103:0/4293491066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb21c102d40 0x7fb21c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:44.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.967+0000 7fb220d23640 1 -- 172.21.15.103:0/4293491066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb21c10c170 msgr2=0x7fb21c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:44.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.967+0000 7fb220d23640 1 --2- 172.21.15.103:0/4293491066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb21c10c170 0x7fb21c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:44.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.967+0000 7fb220d23640 1 -- 172.21.15.103:0/4293491066 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21c19d3e0 con 0x7fb21c103660 2024-09-19T06:47:44.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.967+0000 7fb221d25640 1 --2- 172.21.15.103:0/4293491066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb21c10c170 0x7fb21c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:44.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.967+0000 7fb220d23640 1 --2- 172.21.15.103:0/4293491066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb21c103660 0x7fb21c198970 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb21800b3f0 tx=0x7fb21800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:44.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.967+0000 7fb20e7fc640 1 -- 172.21.15.103:0/4293491066 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218015020 con 0x7fb21c103660 2024-09-19T06:47:44.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.967+0000 7fb20e7fc640 1 -- 172.21.15.103:0/4293491066 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb218011d10 con 0x7fb21c103660 2024-09-19T06:47:44.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.967+0000 7fb223788640 1 -- 172.21.15.103:0/4293491066 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb21c19d6d0 con 0x7fb21c103660 2024-09-19T06:47:44.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.967+0000 7fb20e7fc640 1 -- 172.21.15.103:0/4293491066 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21800f2e0 con 0x7fb21c103660 2024-09-19T06:47:44.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.967+0000 7fb223788640 1 -- 172.21.15.103:0/4293491066 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb21c1a4fb0 con 0x7fb21c103660 2024-09-19T06:47:44.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.968+0000 7fb223788640 1 -- 172.21.15.103:0/4293491066 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb21c103120 con 0x7fb21c103660 2024-09-19T06:47:44.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.969+0000 7fb20e7fc640 1 -- 172.21.15.103:0/4293491066 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb218018020 con 0x7fb21c103660 2024-09-19T06:47:44.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.970+0000 7fb20e7fc640 1 --2- 172.21.15.103:0/4293491066 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb2080a32d0 0x7fb2080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:44.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.970+0000 7fb221524640 1 --2- 172.21.15.103:0/4293491066 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb2080a32d0 0x7fb2080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:44.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.970+0000 7fb20e7fc640 1 -- 172.21.15.103:0/4293491066 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb2180cf180 con 0x7fb21c103660 2024-09-19T06:47:44.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.970+0000 7fb221524640 1 --2- 172.21.15.103:0/4293491066 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb2080a32d0 0x7fb2080a5790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb21000bbe0 tx=0x7fb21000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:44.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:44.974+0000 7fb20e7fc640 1 -- 172.21.15.103:0/4293491066 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb21801f020 con 0x7fb21c103660 2024-09-19T06:47:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:45.169+0000 7fb223788640 1 -- 172.21.15.103:0/4293491066 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb21c0fe480 con 0x7fb2080a32d0 2024-09-19T06:47:45.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:45.174+0000 7fb20e7fc640 1 -- 172.21.15.103:0/4293491066 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb21c0fe480 con 0x7fb2080a32d0 2024-09-19T06:47:45.176 INFO:teuthology.orchestra.run.smithi103.stdout:/volumes/qa/sv_1/e871e164-e13b-40d7-a1ed-57f3d47a3fa9 2024-09-19T06:47:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:45.176+0000 7fb223788640 1 -- 172.21.15.103:0/4293491066 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb2080a32d0 msgr2=0x7fb2080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:45.176+0000 7fb223788640 1 --2- 172.21.15.103:0/4293491066 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb2080a32d0 0x7fb2080a5790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb21000bbe0 tx=0x7fb21000b390 comp rx=0 tx=0).stop 2024-09-19T06:47:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:45.176+0000 7fb223788640 1 -- 172.21.15.103:0/4293491066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb21c103660 msgr2=0x7fb21c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:45.176+0000 7fb223788640 1 --2- 172.21.15.103:0/4293491066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb21c103660 0x7fb21c198970 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb21800b3f0 tx=0x7fb21800b8b0 comp rx=0 tx=0).stop 2024-09-19T06:47:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:45.176+0000 7fb223788640 1 -- 172.21.15.103:0/4293491066 shutdown_connections 2024-09-19T06:47:45.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:45.176+0000 7fb223788640 1 --2- 172.21.15.103:0/4293491066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb21c10c170 0x7fb21c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:45.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:45.176+0000 7fb223788640 1 --2- 172.21.15.103:0/4293491066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb21c103660 0x7fb21c198970 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:45.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:45.176+0000 7fb223788640 1 --2- 172.21.15.103:0/4293491066 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb2080a32d0 0x7fb2080a5790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:45.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:45.176+0000 7fb223788640 1 --2- 172.21.15.103:0/4293491066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb21c102d40 0x7fb21c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:45.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:45.176+0000 7fb223788640 1 -- 172.21.15.103:0/4293491066 >> 172.21.15.103:0/4293491066 conn(0x7fb21c0fc7a0 msgr2=0x7fb21c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:45.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:45.176+0000 7fb223788640 1 -- 172.21.15.103:0/4293491066 shutdown_connections 2024-09-19T06:47:45.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:45.176+0000 7fb223788640 1 -- 172.21.15.103:0/4293491066 wait complete. 2024-09-19T06:47:45.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:44 smithi134 ceph-mon[35356]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:47:45.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:44 smithi134 ceph-mon[35356]: pgmap v284: 97 pgs: 1 activating+degraded, 96 active+clean; 592 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 26 op/s; 6/300 objects degraded (2.000%); 25 B/s, 2 objects/s recovering 2024-09-19T06:47:45.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:44 smithi174 ceph-mon[35637]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:47:45.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:44 smithi174 ceph-mon[35637]: pgmap v284: 97 pgs: 1 activating+degraded, 96 active+clean; 592 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 26 op/s; 6/300 objects degraded (2.000%); 25 B/s, 2 objects/s recovering 2024-09-19T06:47:45.752 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:47:46.002 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:46.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.577+0000 7f77ef188640 1 -- 172.21.15.103:0/786972143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e8102790 msgr2=0x7f77e81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:46.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.577+0000 7f77ef188640 1 --2- 172.21.15.103:0/786972143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e8102790 0x7f77e81067e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009900 tx=0x7f77d402a410 comp rx=0 tx=0).stop 2024-09-19T06:47:46.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.577+0000 7f77ef188640 1 -- 172.21.15.103:0/786972143 shutdown_connections 2024-09-19T06:47:46.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.577+0000 7f77ef188640 1 --2- 172.21.15.103:0/786972143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77e8106f10 0x7f77e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.577+0000 7f77ef188640 1 --2- 172.21.15.103:0/786972143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e8102790 0x7f77e81067e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.577+0000 7f77ef188640 1 --2- 172.21.15.103:0/786972143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e8101de0 0x7f77e81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.577+0000 7f77ef188640 1 -- 172.21.15.103:0/786972143 >> 172.21.15.103:0/786972143 conn(0x7f77e80fd8c0 msgr2=0x7f77e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:46.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.577+0000 7f77ef188640 1 -- 172.21.15.103:0/786972143 shutdown_connections 2024-09-19T06:47:46.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.577+0000 7f77ef188640 1 -- 172.21.15.103:0/786972143 wait complete. 2024-09-19T06:47:46.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.577+0000 7f77ef188640 1 Processor -- start 2024-09-19T06:47:46.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.577+0000 7f77ef188640 1 -- start start 2024-09-19T06:47:46.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77ef188640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77e8101de0 0x7f77e80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77ef188640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e8102790 0x7f77e8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77ef188640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e8106f10 0x7f77e8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77ef188640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8112030 con 0x7f77e8102790 2024-09-19T06:47:46.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77ef188640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8111eb0 con 0x7f77e8106f10 2024-09-19T06:47:46.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77ef188640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f77e81121b0 con 0x7f77e8101de0 2024-09-19T06:47:46.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e8102790 0x7f77e8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e8102790 0x7f77e8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53812/0 (socket says 172.21.15.103:53812) 2024-09-19T06:47:46.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77e7fff640 1 -- 172.21.15.103:0/2392248563 learned_addr learned my addr 172.21.15.103:0/2392248563 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:46.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77ed725640 1 --2- 172.21.15.103:0/2392248563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e8106f10 0x7f77e8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77ecf24640 1 --2- 172.21.15.103:0/2392248563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77e8101de0 0x7f77e80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77e7fff640 1 -- 172.21.15.103:0/2392248563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77e8101de0 msgr2=0x7f77e80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:46.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77e7fff640 1 --2- 172.21.15.103:0/2392248563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77e8101de0 0x7f77e80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77e7fff640 1 -- 172.21.15.103:0/2392248563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e8106f10 msgr2=0x7f77e8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:46.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77e7fff640 1 --2- 172.21.15.103:0/2392248563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e8106f10 0x7f77e8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77e7fff640 1 -- 172.21.15.103:0/2392248563 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e81a6b10 con 0x7f77e8102790 2024-09-19T06:47:46.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77ed725640 1 --2- 172.21.15.103:0/2392248563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e8106f10 0x7f77e8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:46.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77ecf24640 1 --2- 172.21.15.103:0/2392248563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77e8101de0 0x7f77e80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:46.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.578+0000 7f77e7fff640 1 --2- 172.21.15.103:0/2392248563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e8102790 0x7f77e8074a30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f77d4031550 tx=0x7f77d4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:46.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.579+0000 7f77e5ffb640 1 -- 172.21.15.103:0/2392248563 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d403f030 con 0x7f77e8102790 2024-09-19T06:47:46.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.579+0000 7f77ef188640 1 -- 172.21.15.103:0/2392248563 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e81a6da0 con 0x7f77e8102790 2024-09-19T06:47:46.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.579+0000 7f77e5ffb640 1 -- 172.21.15.103:0/2392248563 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d4003ce0 con 0x7f77e8102790 2024-09-19T06:47:46.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.579+0000 7f77e5ffb640 1 -- 172.21.15.103:0/2392248563 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d4035070 con 0x7f77e8102790 2024-09-19T06:47:46.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.579+0000 7f77ef188640 1 -- 172.21.15.103:0/2392248563 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e81a7220 con 0x7f77e8102790 2024-09-19T06:47:46.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.580+0000 7f77ef188640 1 -- 172.21.15.103:0/2392248563 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e8102240 con 0x7f77e8102790 2024-09-19T06:47:46.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.581+0000 7f77e5ffb640 1 -- 172.21.15.103:0/2392248563 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f77d4042030 con 0x7f77e8102790 2024-09-19T06:47:46.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.581+0000 7f77e5ffb640 1 --2- 172.21.15.103:0/2392248563 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f77d80a32d0 0x7f77d80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.582+0000 7f77ecf24640 1 --2- 172.21.15.103:0/2392248563 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f77d80a32d0 0x7f77d80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.582+0000 7f77e5ffb640 1 -- 172.21.15.103:0/2392248563 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f77d40f04c0 con 0x7f77e8102790 2024-09-19T06:47:46.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.582+0000 7f77ecf24640 1 --2- 172.21.15.103:0/2392248563 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f77d80a32d0 0x7f77d80a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f77dc006fd0 tx=0x7f77dc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:46.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.586+0000 7f77e5ffb640 1 -- 172.21.15.103:0/2392248563 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f77d40bd090 con 0x7f77e8102790 2024-09-19T06:47:46.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.787+0000 7f77ef188640 1 -- 172.21.15.103:0/2392248563 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77e810a120 con 0x7f77e8102790 2024-09-19T06:47:46.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.788+0000 7f77e5ffb640 1 -- 172.21.15.103:0/2392248563 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11994 (secure 0 0 0) 0x7f77d40bc990 con 0x7f77e8102790 2024-09-19T06:47:46.791 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:47:46.791 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-19T06:47:37:459674+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:47:36.481871+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24455,24454,24436,24431]},"id":1}]} 2024-09-19T06:47:46.791 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-19T06:47:46.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.791+0000 7f77ef188640 1 -- 172.21.15.103:0/2392248563 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f77d80a32d0 msgr2=0x7f77d80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:46.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.791+0000 7f77ef188640 1 --2- 172.21.15.103:0/2392248563 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f77d80a32d0 0x7f77d80a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f77dc006fd0 tx=0x7f77dc008040 comp rx=0 tx=0).stop 2024-09-19T06:47:46.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.791+0000 7f77ef188640 1 -- 172.21.15.103:0/2392248563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e8102790 msgr2=0x7f77e8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:46.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.791+0000 7f77ef188640 1 --2- 172.21.15.103:0/2392248563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e8102790 0x7f77e8074a30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f77d4031550 tx=0x7f77d4004360 comp rx=0 tx=0).stop 2024-09-19T06:47:46.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.791+0000 7f77ef188640 1 -- 172.21.15.103:0/2392248563 shutdown_connections 2024-09-19T06:47:46.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.791+0000 7f77ef188640 1 --2- 172.21.15.103:0/2392248563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e8106f10 0x7f77e8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.791+0000 7f77ef188640 1 --2- 172.21.15.103:0/2392248563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e8102790 0x7f77e8074a30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.791+0000 7f77ef188640 1 --2- 172.21.15.103:0/2392248563 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f77d80a32d0 0x7f77d80a5790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.791+0000 7f77ef188640 1 --2- 172.21.15.103:0/2392248563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77e8101de0 0x7f77e80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.791+0000 7f77ef188640 1 -- 172.21.15.103:0/2392248563 >> 172.21.15.103:0/2392248563 conn(0x7f77e80fd8c0 msgr2=0x7f77e80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:46.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.791+0000 7f77ef188640 1 -- 172.21.15.103:0/2392248563 shutdown_connections 2024-09-19T06:47:46.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:46.791+0000 7f77ef188640 1 -- 172.21.15.103:0/2392248563 wait complete. 2024-09-19T06:47:47.135 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:46 smithi174 ceph-mon[35637]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:47:47.135 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:46 smithi174 ceph-mon[35637]: pgmap v285: 97 pgs: 1 activating+degraded, 96 active+clean; 629 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 16 op/s; 6/303 objects degraded (1.980%); 25 B/s, 2 objects/s recovering 2024-09-19T06:47:47.135 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:46 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2392248563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:47.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:46 smithi103 ceph-mon[31158]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:47:47.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:46 smithi103 ceph-mon[31158]: pgmap v285: 97 pgs: 1 activating+degraded, 96 active+clean; 629 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 16 op/s; 6/303 objects degraded (1.980%); 25 B/s, 2 objects/s recovering 2024-09-19T06:47:47.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:46 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2392248563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:47.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:46 smithi134 ceph-mon[35356]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:47:47.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:46 smithi134 ceph-mon[35356]: pgmap v285: 97 pgs: 1 activating+degraded, 96 active+clean; 629 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 16 op/s; 6/303 objects degraded (1.980%); 25 B/s, 2 objects/s recovering 2024-09-19T06:47:47.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:46 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2392248563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:47.554 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:47:47.804 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:48.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:47 smithi134 ceph-mon[35356]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/303 objects degraded (1.980%), 1 pg degraded) 2024-09-19T06:47:48.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:47 smithi134 ceph-mon[35356]: Cluster is now healthy 2024-09-19T06:47:48.193 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:47 smithi103 ceph-mon[31158]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/303 objects degraded (1.980%), 1 pg degraded) 2024-09-19T06:47:48.193 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:47 smithi103 ceph-mon[31158]: Cluster is now healthy 2024-09-19T06:47:48.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.270+0000 7fc552347640 1 -- 172.21.15.103:0/3058023419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc54c106f10 msgr2=0x7fc54c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:48.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.270+0000 7fc552347640 1 --2- 172.21.15.103:0/3058023419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc54c106f10 0x7fc54c10ea90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc544009ca0 tx=0x7fc54402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:47:48.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.270+0000 7fc552347640 1 -- 172.21.15.103:0/3058023419 shutdown_connections 2024-09-19T06:47:48.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.270+0000 7fc552347640 1 --2- 172.21.15.103:0/3058023419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc54c106f10 0x7fc54c10ea90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:48.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.270+0000 7fc552347640 1 --2- 172.21.15.103:0/3058023419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc54c102790 0x7fc54c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:48.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.270+0000 7fc552347640 1 --2- 172.21.15.103:0/3058023419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc54c101de0 0x7fc54c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:48.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.270+0000 7fc552347640 1 -- 172.21.15.103:0/3058023419 >> 172.21.15.103:0/3058023419 conn(0x7fc54c0fd8c0 msgr2=0x7fc54c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:48.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.270+0000 7fc552347640 1 -- 172.21.15.103:0/3058023419 shutdown_connections 2024-09-19T06:47:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.270+0000 7fc552347640 1 -- 172.21.15.103:0/3058023419 wait complete. 2024-09-19T06:47:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.270+0000 7fc552347640 1 Processor -- start 2024-09-19T06:47:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.270+0000 7fc552347640 1 -- start start 2024-09-19T06:47:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc552347640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc54c101de0 0x7fc54c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc552347640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc54c102790 0x7fc54c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc552347640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc54c106f10 0x7fc54c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc552347640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c111fe0 con 0x7fc54c102790 2024-09-19T06:47:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc552347640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c111e60 con 0x7fc54c101de0 2024-09-19T06:47:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc552347640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c112160 con 0x7fc54c106f10 2024-09-19T06:47:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc543fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc54c102790 0x7fc54c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc543fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc54c102790 0x7fc54c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47268/0 (socket says 172.21.15.103:47268) 2024-09-19T06:47:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc543fff640 1 -- 172.21.15.103:0/3706961003 learned_addr learned my addr 172.21.15.103:0/3706961003 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc5508e4640 1 --2- 172.21.15.103:0/3706961003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc54c106f10 0x7fc54c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc54bfff640 1 --2- 172.21.15.103:0/3706961003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc54c101de0 0x7fc54c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc543fff640 1 -- 172.21.15.103:0/3706961003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc54c106f10 msgr2=0x7fc54c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc543fff640 1 --2- 172.21.15.103:0/3706961003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc54c106f10 0x7fc54c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc543fff640 1 -- 172.21.15.103:0/3706961003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc54c101de0 msgr2=0x7fc54c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc543fff640 1 --2- 172.21.15.103:0/3706961003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc54c101de0 0x7fc54c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc543fff640 1 -- 172.21.15.103:0/3706961003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c1a6870 con 0x7fc54c102790 2024-09-19T06:47:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc54bfff640 1 --2- 172.21.15.103:0/3706961003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc54c101de0 0x7fc54c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc543fff640 1 --2- 172.21.15.103:0/3706961003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc54c102790 0x7fc54c074a30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c920 tx=0x7fc53400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc549ffb640 1 -- 172.21.15.103:0/3706961003 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534007c70 con 0x7fc54c102790 2024-09-19T06:47:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc552347640 1 -- 172.21.15.103:0/3706961003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54c1a6b60 con 0x7fc54c102790 2024-09-19T06:47:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc549ffb640 1 -- 172.21.15.103:0/3706961003 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc534007e10 con 0x7fc54c102790 2024-09-19T06:47:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc549ffb640 1 -- 172.21.15.103:0/3706961003 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534018890 con 0x7fc54c102790 2024-09-19T06:47:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.271+0000 7fc552347640 1 -- 172.21.15.103:0/3706961003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54c1a70f0 con 0x7fc54c102790 2024-09-19T06:47:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.273+0000 7fc552347640 1 -- 172.21.15.103:0/3706961003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54c1021c0 con 0x7fc54c102790 2024-09-19T06:47:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.273+0000 7fc549ffb640 1 -- 172.21.15.103:0/3706961003 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc534018a70 con 0x7fc54c102790 2024-09-19T06:47:48.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.274+0000 7fc549ffb640 1 --2- 172.21.15.103:0/3706961003 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc5380a32d0 0x7fc5380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:48.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.274+0000 7fc54bfff640 1 --2- 172.21.15.103:0/3706961003 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc5380a32d0 0x7fc5380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:48.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.274+0000 7fc549ffb640 1 -- 172.21.15.103:0/3706961003 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc5340cfc90 con 0x7fc54c102790 2024-09-19T06:47:48.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.275+0000 7fc54bfff640 1 --2- 172.21.15.103:0/3706961003 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc5380a32d0 0x7fc5380a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc53c006fd0 tx=0x7fc53c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:48.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.278+0000 7fc549ffb640 1 -- 172.21.15.103:0/3706961003 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc5340d5020 con 0x7fc54c102790 2024-09-19T06:47:48.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:47 smithi174 ceph-mon[35637]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/303 objects degraded (1.980%), 1 pg degraded) 2024-09-19T06:47:48.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:47 smithi174 ceph-mon[35637]: Cluster is now healthy 2024-09-19T06:47:48.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.447+0000 7fc552347640 1 -- 172.21.15.103:0/3706961003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc54c1a72c0 con 0x7fc54c102790 2024-09-19T06:47:48.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.448+0000 7fc549ffb640 1 -- 172.21.15.103:0/3706961003 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22729 (secure 0 0 0) 0x7fc534014030 con 0x7fc54c102790 2024-09-19T06:47:48.451 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:47:48.451 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":82,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","created":"2024-09-19T06:39:46.840574+0000","modified":"2024-09-19T06:47:40.819637+0000","last_up_change":"2024-09-19T06:46:35.652960+0000","last_in_change":"2024-09-19T06:46:11.630585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:42:52.050881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:47:21.192611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:47:23.643291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d45dab3-e434-4783-b2e3-f658c3ba583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6803","nonce":3157697740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6805","nonce":3157697740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6809","nonce":3157697740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6807","nonce":3157697740}]},"public_addr":"172.21.15.103:6803/3157697740","cluster_addr":"172.21.15.103:6805/3157697740","heartbeat_back_addr":"172.21.15.103:6809/3157697740","heartbeat_front_addr":"172.21.15.103:6807/3157697740","state":["exists","up"]},{"osd":1,"uuid":"ee0e92bd-15ab-4867-9a18-20abb9f5fc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6801","nonce":1418301195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6803","nonce":1418301195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6807","nonce":1418301195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6805","nonce":1418301195}]},"public_addr":"172.21.15.134:6801/1418301195","cluster_addr":"172.21.15.134:6803/1418301195","heartbeat_back_addr":"172.21.15.134:6807/1418301195","heartbeat_front_addr":"172.21.15.134:6805/1418301195","state":["exists","up"]},{"osd":2,"uuid":"0230caed-025b-46e9-a011-237601d16ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6801","nonce":2292242290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6803","nonce":2292242290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6807","nonce":2292242290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6805","nonce":2292242290}]},"public_addr":"172.21.15.174:6801/2292242290","cluster_addr":"172.21.15.174:6803/2292242290","heartbeat_back_addr":"172.21.15.174:6807/2292242290","heartbeat_front_addr":"172.21.15.174:6805/2292242290","state":["exists","up"]},{"osd":3,"uuid":"c54a8399-bf4b-4d70-b207-a13d85f99590","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6811","nonce":2698986333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6813","nonce":2698986333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6817","nonce":2698986333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6815","nonce":2698986333}]},"public_addr":"172.21.15.103:6811/2698986333","cluster_addr":"172.21.15.103:6813/2698986333","heartbeat_back_addr":"172.21.15.103:6817/2698986333","heartbeat_front_addr":"172.21.15.103:6815/2698986333","state":["exists","up"]},{"osd":4,"uuid":"7c046be6-464a-4ab5-812b-1781121fc317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6809","nonce":3169891968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6811","nonce":3169891968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6815","nonce":3169891968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6813","nonce":3169891968}]},"public_addr":"172.21.15.134:6809/3169891968","cluster_addr":"172.21.15.134:6811/3169891968","heartbeat_back_addr":"172.21.15.134:6815/3169891968","heartbeat_front_addr":"172.21.15.134:6813/3169891968","state":["exists","up"]},{"osd":5,"uuid":"8622596b-6a06-459c-882d-50fba9e296ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6809","nonce":763749644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6811","nonce":763749644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6815","nonce":763749644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6813","nonce":763749644}]},"public_addr":"172.21.15.174:6809/763749644","cluster_addr":"172.21.15.174:6811/763749644","heartbeat_back_addr":"172.21.15.174:6815/763749644","heartbeat_front_addr":"172.21.15.174:6813/763749644","state":["exists","up"]},{"osd":6,"uuid":"64d858d6-a8b3-4f65-a187-af9ab1ccee0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6819","nonce":1396972394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6821","nonce":1396972394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6825","nonce":1396972394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6823","nonce":1396972394}]},"public_addr":"172.21.15.103:6819/1396972394","cluster_addr":"172.21.15.103:6821/1396972394","heartbeat_back_addr":"172.21.15.103:6825/1396972394","heartbeat_front_addr":"172.21.15.103:6823/1396972394","state":["exists","up"]},{"osd":7,"uuid":"d405d1c6-839c-4c92-8261-4adb3163c90f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6817","nonce":2196366765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6819","nonce":2196366765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6823","nonce":2196366765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6821","nonce":2196366765}]},"public_addr":"172.21.15.134:6817/2196366765","cluster_addr":"172.21.15.134:6819/2196366765","heartbeat_back_addr":"172.21.15.134:6823/2196366765","heartbeat_front_addr":"172.21.15.134:6821/2196366765","state":["exists","up"]},{"osd":8,"uuid":"4c452cd9-816c-4f77-9368-2cd4bbeee4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6817","nonce":189977465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6819","nonce":189977465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6823","nonce":189977465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6821","nonce":189977465}]},"public_addr":"172.21.15.174:6817/189977465","cluster_addr":"172.21.15.174:6819/189977465","heartbeat_back_addr":"172.21.15.174:6823/189977465","heartbeat_front_addr":"172.21.15.174:6821/189977465","state":["exists","up"]},{"osd":9,"uuid":"fb3662e0-2d8e-45c4-9119-6d08b35443da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6827","nonce":3161966413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6829","nonce":3161966413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6833","nonce":3161966413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6831","nonce":3161966413}]},"public_addr":"172.21.15.103:6827/3161966413","cluster_addr":"172.21.15.103:6829/3161966413","heartbeat_back_addr":"172.21.15.103:6833/3161966413","heartbeat_front_addr":"172.21.15.103:6831/3161966413","state":["exists","up"]},{"osd":10,"uuid":"e52c3cbe-a6af-441e-b423-b94f3a2da544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6825","nonce":2532758632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6827","nonce":2532758632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6831","nonce":2532758632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6829","nonce":2532758632}]},"public_addr":"172.21.15.134:6825/2532758632","cluster_addr":"172.21.15.134:6827/2532758632","heartbeat_back_addr":"172.21.15.134:6831/2532758632","heartbeat_front_addr":"172.21.15.134:6829/2532758632","state":["exists","up"]},{"osd":11,"uuid":"9040f3f8-3873-4dea-8716-dcda5811c8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6825","nonce":161916442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6827","nonce":161916442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6831","nonce":161916442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6829","nonce":161916442}]},"public_addr":"172.21.15.174:6825/161916442","cluster_addr":"172.21.15.174:6827/161916442","heartbeat_back_addr":"172.21.15.174:6831/161916442","heartbeat_front_addr":"172.21.15.174:6829/161916442","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:41:59.656854+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:24.797551+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:49.685497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:33.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:39.324983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:03.987310+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:28.747971+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:53.827780+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:18.554450+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:43.705691+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:08.401275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:33.502542+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2867049580":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/3727259725":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/1496153385":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:6800/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1642809745":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/2309624637":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1596229922":"2024-09-20T06:40:16.780890+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:47:48.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.450+0000 7fc552347640 1 -- 172.21.15.103:0/3706961003 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc5380a32d0 msgr2=0x7fc5380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:48.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.450+0000 7fc552347640 1 --2- 172.21.15.103:0/3706961003 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc5380a32d0 0x7fc5380a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc53c006fd0 tx=0x7fc53c008040 comp rx=0 tx=0).stop 2024-09-19T06:47:48.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.450+0000 7fc552347640 1 -- 172.21.15.103:0/3706961003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc54c102790 msgr2=0x7fc54c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:48.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.450+0000 7fc552347640 1 --2- 172.21.15.103:0/3706961003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc54c102790 0x7fc54c074a30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c920 tx=0x7fc53400cde0 comp rx=0 tx=0).stop 2024-09-19T06:47:48.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.450+0000 7fc552347640 1 -- 172.21.15.103:0/3706961003 shutdown_connections 2024-09-19T06:47:48.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.450+0000 7fc552347640 1 --2- 172.21.15.103:0/3706961003 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc5380a32d0 0x7fc5380a5790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:48.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.450+0000 7fc552347640 1 --2- 172.21.15.103:0/3706961003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc54c106f10 0x7fc54c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:48.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.450+0000 7fc552347640 1 --2- 172.21.15.103:0/3706961003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc54c102790 0x7fc54c074a30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:48.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.450+0000 7fc552347640 1 --2- 172.21.15.103:0/3706961003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc54c101de0 0x7fc54c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:48.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.450+0000 7fc552347640 1 -- 172.21.15.103:0/3706961003 >> 172.21.15.103:0/3706961003 conn(0x7fc54c0fd8c0 msgr2=0x7fc54c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:48.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.451+0000 7fc552347640 1 -- 172.21.15.103:0/3706961003 shutdown_connections 2024-09-19T06:47:48.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:48.451+0000 7fc552347640 1 -- 172.21.15.103:0/3706961003 wait complete. 2024-09-19T06:47:49.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:48 smithi103 ceph-mon[31158]: pgmap v286: 97 pgs: 97 active+clean; 630 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 KiB/s wr, 20 op/s; 109 B/s, 2 objects/s recovering 2024-09-19T06:47:49.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:48 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3706961003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:47:49.109 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T06:47:49.109 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:47:49.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:48 smithi134 ceph-mon[35356]: pgmap v286: 97 pgs: 97 active+clean; 630 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 KiB/s wr, 20 op/s; 109 B/s, 2 objects/s recovering 2024-09-19T06:47:49.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:48 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3706961003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:47:49.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:48 smithi174 ceph-mon[35637]: pgmap v286: 97 pgs: 97 active+clean; 630 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 KiB/s wr, 20 op/s; 109 B/s, 2 objects/s recovering 2024-09-19T06:47:49.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:48 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3706961003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:47:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:49.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.786+0000 7f181d1c1640 1 -- 172.21.15.103:0/2458185357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1818106ba0 msgr2=0x7f1818106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:49.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.786+0000 7f181d1c1640 1 --2- 172.21.15.103:0/2458185357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1818106ba0 0x7f1818106f80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1800009900 tx=0x7f180002a410 comp rx=0 tx=0).stop 2024-09-19T06:47:49.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.787+0000 7f181d1c1640 1 -- 172.21.15.103:0/2458185357 shutdown_connections 2024-09-19T06:47:49.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.787+0000 7f181d1c1640 1 --2- 172.21.15.103:0/2458185357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18181005c0 0x7f181810ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.787+0000 7f181d1c1640 1 --2- 172.21.15.103:0/2458185357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18180ffc20 0x7f1818100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.787+0000 7f181d1c1640 1 --2- 172.21.15.103:0/2458185357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1818106ba0 0x7f1818106f80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.787+0000 7f181d1c1640 1 -- 172.21.15.103:0/2458185357 >> 172.21.15.103:0/2458185357 conn(0x7f18180fba90 msgr2=0x7f18180fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:49.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.787+0000 7f181d1c1640 1 -- 172.21.15.103:0/2458185357 shutdown_connections 2024-09-19T06:47:49.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.787+0000 7f181d1c1640 1 -- 172.21.15.103:0/2458185357 wait complete. 2024-09-19T06:47:49.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.787+0000 7f181d1c1640 1 Processor -- start 2024-09-19T06:47:49.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181d1c1640 1 -- start start 2024-09-19T06:47:49.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181d1c1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18180ffc20 0x7f1818198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181d1c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18181005c0 0x7f1818198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181d1c1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1818106ba0 0x7f181819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181d1c1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1818112000 con 0x7f1818106ba0 2024-09-19T06:47:49.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181d1c1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1818111e80 con 0x7f18181005c0 2024-09-19T06:47:49.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181d1c1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1818112180 con 0x7f18180ffc20 2024-09-19T06:47:49.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181759e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1818106ba0 0x7f181819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181759e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1818106ba0 0x7f181819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47282/0 (socket says 172.21.15.103:47282) 2024-09-19T06:47:49.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181759e640 1 -- 172.21.15.103:0/338155330 learned_addr learned my addr 172.21.15.103:0/338155330 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:49.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f1816d9d640 1 --2- 172.21.15.103:0/338155330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18180ffc20 0x7f1818198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181659c640 1 --2- 172.21.15.103:0/338155330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18181005c0 0x7f1818198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181759e640 1 -- 172.21.15.103:0/338155330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18180ffc20 msgr2=0x7f1818198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:49.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181759e640 1 --2- 172.21.15.103:0/338155330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18180ffc20 0x7f1818198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181759e640 1 -- 172.21.15.103:0/338155330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18181005c0 msgr2=0x7f1818198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:49.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181759e640 1 --2- 172.21.15.103:0/338155330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18181005c0 0x7f1818198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181759e640 1 -- 172.21.15.103:0/338155330 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181819d4b0 con 0x7f1818106ba0 2024-09-19T06:47:49.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.788+0000 7f181759e640 1 --2- 172.21.15.103:0/338155330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1818106ba0 0x7f181819cdd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f181000eaa0 tx=0x7f181000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:49.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.789+0000 7f180ffff640 1 -- 172.21.15.103:0/338155330 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181000cdb0 con 0x7f1818106ba0 2024-09-19T06:47:49.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.789+0000 7f180ffff640 1 -- 172.21.15.103:0/338155330 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1810005020 con 0x7f1818106ba0 2024-09-19T06:47:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.789+0000 7f181d1c1640 1 -- 172.21.15.103:0/338155330 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181819d7a0 con 0x7f1818106ba0 2024-09-19T06:47:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.789+0000 7f180ffff640 1 -- 172.21.15.103:0/338155330 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18100116b0 con 0x7f1818106ba0 2024-09-19T06:47:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.789+0000 7f181d1c1640 1 -- 172.21.15.103:0/338155330 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18181a5080 con 0x7f1818106ba0 2024-09-19T06:47:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.790+0000 7f181d1c1640 1 -- 172.21.15.103:0/338155330 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1818100080 con 0x7f1818106ba0 2024-09-19T06:47:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.791+0000 7f180ffff640 1 -- 172.21.15.103:0/338155330 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1810005560 con 0x7f1818106ba0 2024-09-19T06:47:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.792+0000 7f180ffff640 1 --2- 172.21.15.103:0/338155330 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f18040a32d0 0x7f18040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.792+0000 7f1816d9d640 1 --2- 172.21.15.103:0/338155330 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f18040a32d0 0x7f18040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.792+0000 7f180ffff640 1 -- 172.21.15.103:0/338155330 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f18100cfd00 con 0x7f1818106ba0 2024-09-19T06:47:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.792+0000 7f1816d9d640 1 --2- 172.21.15.103:0/338155330 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f18040a32d0 0x7f18040a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1800009230 tx=0x7f18000058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.796+0000 7f180ffff640 1 -- 172.21.15.103:0/338155330 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1810016030 con 0x7f1818106ba0 2024-09-19T06:47:49.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.965+0000 7f181d1c1640 1 -- 172.21.15.103:0/338155330 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1818062630 con 0x7f18040a32d0 2024-09-19T06:47:49.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.972+0000 7f180ffff640 1 -- 172.21.15.103:0/338155330 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258767 (secure 0 0 0) 0x7f1818062630 con 0x7f18040a32d0 2024-09-19T06:47:49.976 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:47:49.978 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T06:47:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.975+0000 7f181d1c1640 1 -- 172.21.15.103:0/338155330 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f18040a32d0 msgr2=0x7f18040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.975+0000 7f181d1c1640 1 --2- 172.21.15.103:0/338155330 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f18040a32d0 0x7f18040a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1800009230 tx=0x7f18000058d0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.975+0000 7f181d1c1640 1 -- 172.21.15.103:0/338155330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1818106ba0 msgr2=0x7f181819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.975+0000 7f181d1c1640 1 --2- 172.21.15.103:0/338155330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1818106ba0 0x7f181819cdd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f181000eaa0 tx=0x7f181000ef60 comp rx=0 tx=0).stop 2024-09-19T06:47:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.975+0000 7f181d1c1640 1 -- 172.21.15.103:0/338155330 shutdown_connections 2024-09-19T06:47:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.975+0000 7f181d1c1640 1 --2- 172.21.15.103:0/338155330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1818106ba0 0x7f181819cdd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.975+0000 7f181d1c1640 1 --2- 172.21.15.103:0/338155330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18181005c0 0x7f1818198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.975+0000 7f181d1c1640 1 --2- 172.21.15.103:0/338155330 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f18040a32d0 0x7f18040a5790 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.975+0000 7f181d1c1640 1 --2- 172.21.15.103:0/338155330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18180ffc20 0x7f1818198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.975+0000 7f181d1c1640 1 -- 172.21.15.103:0/338155330 >> 172.21.15.103:0/338155330 conn(0x7f18180fba90 msgr2=0x7f18180fc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:49.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.975+0000 7f181d1c1640 1 -- 172.21.15.103:0/338155330 shutdown_connections 2024-09-19T06:47:49.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:49.975+0000 7f181d1c1640 1 -- 172.21.15.103:0/338155330 wait complete. 2024-09-19T06:47:50.533 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-19T06:47:49.823777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644612,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":265,"num_read_kb":236,"num_write":412,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":1181610,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":338,"ondisk_log_size":338,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353644,"kb_used_data":12688,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588180,"statfs":{"total":1151940427776,"available":1151578296320,"internally_reserved":0,"allocated":12992512,"data_stored":7177463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46910,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":115,"num_read_kb":105,"num_write":48,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002512"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831591+0000","last_change":"2024-09-19T06:47:24.853118+0000","last_active":"2024-09-19T06:47:40.831591+0000","last_peered":"2024-09-19T06:47:40.831591+0000","last_clean":"2024-09-19T06:47:40.831591+0000","last_became_active":"2024-09-19T06:47:24.852801+0000","last_became_peered":"2024-09-19T06:47:24.852801+0000","last_unstale":"2024-09-19T06:47:40.831591+0000","last_undegraded":"2024-09-19T06:47:40.831591+0000","last_fullsized":"2024-09-19T06:47:40.831591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:16.729103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839300+0000","last_change":"2024-09-19T06:47:24.841095+0000","last_active":"2024-09-19T06:47:40.839300+0000","last_peered":"2024-09-19T06:47:40.839300+0000","last_clean":"2024-09-19T06:47:40.839300+0000","last_became_active":"2024-09-19T06:47:24.840774+0000","last_became_peered":"2024-09-19T06:47:24.840774+0000","last_unstale":"2024-09-19T06:47:40.839300+0000","last_undegraded":"2024-09-19T06:47:40.839300+0000","last_fullsized":"2024-09-19T06:47:40.839300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:27.262766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839649+0000","last_change":"2024-09-19T06:47:24.848444+0000","last_active":"2024-09-19T06:47:40.839649+0000","last_peered":"2024-09-19T06:47:40.839649+0000","last_clean":"2024-09-19T06:47:40.839649+0000","last_became_active":"2024-09-19T06:47:24.847354+0000","last_became_peered":"2024-09-19T06:47:24.847354+0000","last_unstale":"2024-09-19T06:47:40.839649+0000","last_undegraded":"2024-09-19T06:47:40.839649+0000","last_fullsized":"2024-09-19T06:47:40.839649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:44.901265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829447+0000","last_change":"2024-09-19T06:47:24.851915+0000","last_active":"2024-09-19T06:47:40.829447+0000","last_peered":"2024-09-19T06:47:40.829447+0000","last_clean":"2024-09-19T06:47:40.829447+0000","last_became_active":"2024-09-19T06:47:24.851600+0000","last_became_peered":"2024-09-19T06:47:24.851600+0000","last_unstale":"2024-09-19T06:47:40.829447+0000","last_undegraded":"2024-09-19T06:47:40.829447+0000","last_fullsized":"2024-09-19T06:47:40.829447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:52.424288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833135+0000","last_change":"2024-09-19T06:47:24.843184+0000","last_active":"2024-09-19T06:47:40.833135+0000","last_peered":"2024-09-19T06:47:40.833135+0000","last_clean":"2024-09-19T06:47:40.833135+0000","last_became_active":"2024-09-19T06:47:24.842857+0000","last_became_peered":"2024-09-19T06:47:24.842857+0000","last_unstale":"2024-09-19T06:47:40.833135+0000","last_undegraded":"2024-09-19T06:47:40.833135+0000","last_fullsized":"2024-09-19T06:47:40.833135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:06.608699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.171834+0000","last_change":"2024-09-19T06:47:24.846631+0000","last_active":"2024-09-19T06:47:41.171834+0000","last_peered":"2024-09-19T06:47:41.171834+0000","last_clean":"2024-09-19T06:47:41.171834+0000","last_became_active":"2024-09-19T06:47:24.846271+0000","last_became_peered":"2024-09-19T06:47:24.846271+0000","last_unstale":"2024-09-19T06:47:41.171834+0000","last_undegraded":"2024-09-19T06:47:41.171834+0000","last_fullsized":"2024-09-19T06:47:41.171834+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.272273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830458+0000","last_change":"2024-09-19T06:47:24.840392+0000","last_active":"2024-09-19T06:47:40.830458+0000","last_peered":"2024-09-19T06:47:40.830458+0000","last_clean":"2024-09-19T06:47:40.830458+0000","last_became_active":"2024-09-19T06:47:24.840092+0000","last_became_peered":"2024-09-19T06:47:24.840092+0000","last_unstale":"2024-09-19T06:47:40.830458+0000","last_undegraded":"2024-09-19T06:47:40.830458+0000","last_fullsized":"2024-09-19T06:47:40.830458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:25.579096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833345+0000","last_change":"2024-09-19T06:47:24.843336+0000","last_active":"2024-09-19T06:47:40.833345+0000","last_peered":"2024-09-19T06:47:40.833345+0000","last_clean":"2024-09-19T06:47:40.833345+0000","last_became_active":"2024-09-19T06:47:24.842016+0000","last_became_peered":"2024-09-19T06:47:24.842016+0000","last_unstale":"2024-09-19T06:47:40.833345+0000","last_undegraded":"2024-09-19T06:47:40.833345+0000","last_fullsized":"2024-09-19T06:47:40.833345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:49.478359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172619+0000","last_change":"2024-09-19T06:47:24.847841+0000","last_active":"2024-09-19T06:47:41.172619+0000","last_peered":"2024-09-19T06:47:41.172619+0000","last_clean":"2024-09-19T06:47:41.172619+0000","last_became_active":"2024-09-19T06:47:24.847492+0000","last_became_peered":"2024-09-19T06:47:24.847492+0000","last_unstale":"2024-09-19T06:47:41.172619+0000","last_undegraded":"2024-09-19T06:47:41.172619+0000","last_fullsized":"2024-09-19T06:47:41.172619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:59.229022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.171833+0000","last_change":"2024-09-19T06:47:24.843773+0000","last_active":"2024-09-19T06:47:41.171833+0000","last_peered":"2024-09-19T06:47:41.171833+0000","last_clean":"2024-09-19T06:47:41.171833+0000","last_became_active":"2024-09-19T06:47:24.843409+0000","last_became_peered":"2024-09-19T06:47:24.843409+0000","last_unstale":"2024-09-19T06:47:41.171833+0000","last_undegraded":"2024-09-19T06:47:41.171833+0000","last_fullsized":"2024-09-19T06:47:41.171833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:22.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830801+0000","last_change":"2024-09-19T06:47:24.846637+0000","last_active":"2024-09-19T06:47:40.830801+0000","last_peered":"2024-09-19T06:47:40.830801+0000","last_clean":"2024-09-19T06:47:40.830801+0000","last_became_active":"2024-09-19T06:47:24.846337+0000","last_became_peered":"2024-09-19T06:47:24.846337+0000","last_unstale":"2024-09-19T06:47:40.830801+0000","last_undegraded":"2024-09-19T06:47:40.830801+0000","last_fullsized":"2024-09-19T06:47:40.830801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:23:35.869485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829501+0000","last_change":"2024-09-19T06:47:24.851399+0000","last_active":"2024-09-19T06:47:40.829501+0000","last_peered":"2024-09-19T06:47:40.829501+0000","last_clean":"2024-09-19T06:47:40.829501+0000","last_became_active":"2024-09-19T06:47:24.851090+0000","last_became_peered":"2024-09-19T06:47:24.851090+0000","last_unstale":"2024-09-19T06:47:40.829501+0000","last_undegraded":"2024-09-19T06:47:40.829501+0000","last_fullsized":"2024-09-19T06:47:40.829501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:52.839956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830657+0000","last_change":"2024-09-19T06:47:24.833858+0000","last_active":"2024-09-19T06:47:40.830657+0000","last_peered":"2024-09-19T06:47:40.830657+0000","last_clean":"2024-09-19T06:47:40.830657+0000","last_became_active":"2024-09-19T06:47:24.833616+0000","last_became_peered":"2024-09-19T06:47:24.833616+0000","last_unstale":"2024-09-19T06:47:40.830657+0000","last_undegraded":"2024-09-19T06:47:40.830657+0000","last_fullsized":"2024-09-19T06:47:40.830657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:54.918144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839219+0000","last_change":"2024-09-19T06:47:24.841030+0000","last_active":"2024-09-19T06:47:40.839219+0000","last_peered":"2024-09-19T06:47:40.839219+0000","last_clean":"2024-09-19T06:47:40.839219+0000","last_became_active":"2024-09-19T06:47:24.839631+0000","last_became_peered":"2024-09-19T06:47:24.839631+0000","last_unstale":"2024-09-19T06:47:40.839219+0000","last_undegraded":"2024-09-19T06:47:40.839219+0000","last_fullsized":"2024-09-19T06:47:40.839219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:21.764812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839091+0000","last_change":"2024-09-19T06:47:24.849281+0000","last_active":"2024-09-19T06:47:40.839091+0000","last_peered":"2024-09-19T06:47:40.839091+0000","last_clean":"2024-09-19T06:47:40.839091+0000","last_became_active":"2024-09-19T06:47:24.848977+0000","last_became_peered":"2024-09-19T06:47:24.848977+0000","last_unstale":"2024-09-19T06:47:40.839091+0000","last_undegraded":"2024-09-19T06:47:40.839091+0000","last_fullsized":"2024-09-19T06:47:40.839091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:36.693758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172068+0000","last_change":"2024-09-19T06:47:24.843619+0000","last_active":"2024-09-19T06:47:41.172068+0000","last_peered":"2024-09-19T06:47:41.172068+0000","last_clean":"2024-09-19T06:47:41.172068+0000","last_became_active":"2024-09-19T06:47:24.843323+0000","last_became_peered":"2024-09-19T06:47:24.843323+0000","last_unstale":"2024-09-19T06:47:41.172068+0000","last_undegraded":"2024-09-19T06:47:41.172068+0000","last_fullsized":"2024-09-19T06:47:41.172068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:52.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831828+0000","last_change":"2024-09-19T06:47:24.853165+0000","last_active":"2024-09-19T06:47:40.831828+0000","last_peered":"2024-09-19T06:47:40.831828+0000","last_clean":"2024-09-19T06:47:40.831828+0000","last_became_active":"2024-09-19T06:47:24.852864+0000","last_became_peered":"2024-09-19T06:47:24.852864+0000","last_unstale":"2024-09-19T06:47:40.831828+0000","last_undegraded":"2024-09-19T06:47:40.831828+0000","last_fullsized":"2024-09-19T06:47:40.831828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:38.803478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834536+0000","last_change":"2024-09-19T06:47:24.838151+0000","last_active":"2024-09-19T06:47:40.834536+0000","last_peered":"2024-09-19T06:47:40.834536+0000","last_clean":"2024-09-19T06:47:40.834536+0000","last_became_active":"2024-09-19T06:47:24.837855+0000","last_became_peered":"2024-09-19T06:47:24.837855+0000","last_unstale":"2024-09-19T06:47:40.834536+0000","last_undegraded":"2024-09-19T06:47:40.834536+0000","last_fullsized":"2024-09-19T06:47:40.834536+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:16.675557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830451+0000","last_change":"2024-09-19T06:47:24.843941+0000","last_active":"2024-09-19T06:47:40.830451+0000","last_peered":"2024-09-19T06:47:40.830451+0000","last_clean":"2024-09-19T06:47:40.830451+0000","last_became_active":"2024-09-19T06:47:24.843551+0000","last_became_peered":"2024-09-19T06:47:24.843551+0000","last_unstale":"2024-09-19T06:47:40.830451+0000","last_undegraded":"2024-09-19T06:47:40.830451+0000","last_fullsized":"2024-09-19T06:47:40.830451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:00:49.388265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.836228+0000","last_change":"2024-09-19T06:47:40.836228+0000","last_active":"2024-09-19T06:47:40.836228+0000","last_peered":"2024-09-19T06:47:40.836228+0000","last_clean":"2024-09-19T06:47:40.836228+0000","last_became_active":"2024-09-19T06:47:40.835818+0000","last_became_peered":"2024-09-19T06:47:40.835818+0000","last_unstale":"2024-09-19T06:47:40.836228+0000","last_undegraded":"2024-09-19T06:47:40.836228+0000","last_fullsized":"2024-09-19T06:47:40.836228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:45.341160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830676+0000","last_change":"2024-09-19T06:47:24.843746+0000","last_active":"2024-09-19T06:47:40.830676+0000","last_peered":"2024-09-19T06:47:40.830676+0000","last_clean":"2024-09-19T06:47:40.830676+0000","last_became_active":"2024-09-19T06:47:24.841495+0000","last_became_peered":"2024-09-19T06:47:24.841495+0000","last_unstale":"2024-09-19T06:47:40.830676+0000","last_undegraded":"2024-09-19T06:47:40.830676+0000","last_fullsized":"2024-09-19T06:47:40.830676+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:06:29.995595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.173085+0000","last_change":"2024-09-19T06:47:24.844747+0000","last_active":"2024-09-19T06:47:41.173085+0000","last_peered":"2024-09-19T06:47:41.173085+0000","last_clean":"2024-09-19T06:47:41.173085+0000","last_became_active":"2024-09-19T06:47:24.844425+0000","last_became_peered":"2024-09-19T06:47:24.844425+0000","last_unstale":"2024-09-19T06:47:41.173085+0000","last_undegraded":"2024-09-19T06:47:41.173085+0000","last_fullsized":"2024-09-19T06:47:41.173085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:10.309044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834053+0000","last_change":"2024-09-19T06:47:24.848227+0000","last_active":"2024-09-19T06:47:40.834053+0000","last_peered":"2024-09-19T06:47:40.834053+0000","last_clean":"2024-09-19T06:47:40.834053+0000","last_became_active":"2024-09-19T06:47:24.847914+0000","last_became_peered":"2024-09-19T06:47:24.847914+0000","last_unstale":"2024-09-19T06:47:40.834053+0000","last_undegraded":"2024-09-19T06:47:40.834053+0000","last_fullsized":"2024-09-19T06:47:40.834053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:47.281613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834014+0000","last_change":"2024-09-19T06:47:24.844275+0000","last_active":"2024-09-19T06:47:40.834014+0000","last_peered":"2024-09-19T06:47:40.834014+0000","last_clean":"2024-09-19T06:47:40.834014+0000","last_became_active":"2024-09-19T06:47:24.841442+0000","last_became_peered":"2024-09-19T06:47:24.841442+0000","last_unstale":"2024-09-19T06:47:40.834014+0000","last_undegraded":"2024-09-19T06:47:40.834014+0000","last_fullsized":"2024-09-19T06:47:40.834014+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:30.198655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834062+0000","last_change":"2024-09-19T06:47:24.846301+0000","last_active":"2024-09-19T06:47:40.834062+0000","last_peered":"2024-09-19T06:47:40.834062+0000","last_clean":"2024-09-19T06:47:40.834062+0000","last_became_active":"2024-09-19T06:47:24.845970+0000","last_became_peered":"2024-09-19T06:47:24.845970+0000","last_unstale":"2024-09-19T06:47:40.834062+0000","last_undegraded":"2024-09-19T06:47:40.834062+0000","last_fullsized":"2024-09-19T06:47:40.834062+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:13.245536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830554+0000","last_change":"2024-09-19T06:47:24.844444+0000","last_active":"2024-09-19T06:47:40.830554+0000","last_peered":"2024-09-19T06:47:40.830554+0000","last_clean":"2024-09-19T06:47:40.830554+0000","last_became_active":"2024-09-19T06:47:24.844141+0000","last_became_peered":"2024-09-19T06:47:24.844141+0000","last_unstale":"2024-09-19T06:47:40.830554+0000","last_undegraded":"2024-09-19T06:47:40.830554+0000","last_fullsized":"2024-09-19T06:47:40.830554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:44.739242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831557+0000","last_change":"2024-09-19T06:47:24.845633+0000","last_active":"2024-09-19T06:47:40.831557+0000","last_peered":"2024-09-19T06:47:40.831557+0000","last_clean":"2024-09-19T06:47:40.831557+0000","last_became_active":"2024-09-19T06:47:24.845319+0000","last_became_peered":"2024-09-19T06:47:24.845319+0000","last_unstale":"2024-09-19T06:47:40.831557+0000","last_undegraded":"2024-09-19T06:47:40.831557+0000","last_fullsized":"2024-09-19T06:47:40.831557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:00.820006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830650+0000","last_change":"2024-09-19T06:47:24.846457+0000","last_active":"2024-09-19T06:47:40.830650+0000","last_peered":"2024-09-19T06:47:40.830650+0000","last_clean":"2024-09-19T06:47:40.830650+0000","last_became_active":"2024-09-19T06:47:24.846130+0000","last_became_peered":"2024-09-19T06:47:24.846130+0000","last_unstale":"2024-09-19T06:47:40.830650+0000","last_undegraded":"2024-09-19T06:47:40.830650+0000","last_fullsized":"2024-09-19T06:47:40.830650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:42.179145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839379+0000","last_change":"2024-09-19T06:47:24.843141+0000","last_active":"2024-09-19T06:47:40.839379+0000","last_peered":"2024-09-19T06:47:40.839379+0000","last_clean":"2024-09-19T06:47:40.839379+0000","last_became_active":"2024-09-19T06:47:24.842810+0000","last_became_peered":"2024-09-19T06:47:24.842810+0000","last_unstale":"2024-09-19T06:47:40.839379+0000","last_undegraded":"2024-09-19T06:47:40.839379+0000","last_fullsized":"2024-09-19T06:47:40.839379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:43.440453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829981+0000","last_change":"2024-09-19T06:47:24.846171+0000","last_active":"2024-09-19T06:47:40.829981+0000","last_peered":"2024-09-19T06:47:40.829981+0000","last_clean":"2024-09-19T06:47:40.829981+0000","last_became_active":"2024-09-19T06:47:24.845799+0000","last_became_peered":"2024-09-19T06:47:24.845799+0000","last_unstale":"2024-09-19T06:47:40.829981+0000","last_undegraded":"2024-09-19T06:47:40.829981+0000","last_fullsized":"2024-09-19T06:47:40.829981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:23.103061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830061+0000","last_change":"2024-09-19T06:47:24.848723+0000","last_active":"2024-09-19T06:47:40.830061+0000","last_peered":"2024-09-19T06:47:40.830061+0000","last_clean":"2024-09-19T06:47:40.830061+0000","last_became_active":"2024-09-19T06:47:24.848367+0000","last_became_peered":"2024-09-19T06:47:24.848367+0000","last_unstale":"2024-09-19T06:47:40.830061+0000","last_undegraded":"2024-09-19T06:47:40.830061+0000","last_fullsized":"2024-09-19T06:47:40.830061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:15.985293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831354+0000","last_change":"2024-09-19T06:47:24.841379+0000","last_active":"2024-09-19T06:47:40.831354+0000","last_peered":"2024-09-19T06:47:40.831354+0000","last_clean":"2024-09-19T06:47:40.831354+0000","last_became_active":"2024-09-19T06:47:24.841041+0000","last_became_peered":"2024-09-19T06:47:24.841041+0000","last_unstale":"2024-09-19T06:47:40.831354+0000","last_undegraded":"2024-09-19T06:47:40.831354+0000","last_fullsized":"2024-09-19T06:47:40.831354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:11.799022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.832906+0000","last_change":"2024-09-19T06:47:24.849508+0000","last_active":"2024-09-19T06:47:40.832906+0000","last_peered":"2024-09-19T06:47:40.832906+0000","last_clean":"2024-09-19T06:47:40.832906+0000","last_became_active":"2024-09-19T06:47:24.849196+0000","last_became_peered":"2024-09-19T06:47:24.849196+0000","last_unstale":"2024-09-19T06:47:40.832906+0000","last_undegraded":"2024-09-19T06:47:40.832906+0000","last_fullsized":"2024-09-19T06:47:40.832906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:35.079720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:43.477887+0000","last_change":"2024-09-19T06:47:22.829848+0000","last_active":"2024-09-19T06:47:43.477887+0000","last_peered":"2024-09-19T06:47:43.477887+0000","last_clean":"2024-09-19T06:47:43.477887+0000","last_became_active":"2024-09-19T06:47:22.829511+0000","last_became_peered":"2024-09-19T06:47:22.829511+0000","last_unstale":"2024-09-19T06:47:43.477887+0000","last_undegraded":"2024-09-19T06:47:43.477887+0000","last_fullsized":"2024-09-19T06:47:43.477887+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:32.665252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834585+0000","last_change":"2024-09-19T06:47:24.843245+0000","last_active":"2024-09-19T06:47:40.834585+0000","last_peered":"2024-09-19T06:47:40.834585+0000","last_clean":"2024-09-19T06:47:40.834585+0000","last_became_active":"2024-09-19T06:47:24.842885+0000","last_became_peered":"2024-09-19T06:47:24.842885+0000","last_unstale":"2024-09-19T06:47:40.834585+0000","last_undegraded":"2024-09-19T06:47:40.834585+0000","last_fullsized":"2024-09-19T06:47:40.834585+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:26.229003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172292+0000","last_change":"2024-09-19T06:47:22.831456+0000","last_active":"2024-09-19T06:47:41.172292+0000","last_peered":"2024-09-19T06:47:41.172292+0000","last_clean":"2024-09-19T06:47:41.172292+0000","last_became_active":"2024-09-19T06:47:22.831132+0000","last_became_peered":"2024-09-19T06:47:22.831132+0000","last_unstale":"2024-09-19T06:47:41.172292+0000","last_undegraded":"2024-09-19T06:47:41.172292+0000","last_fullsized":"2024-09-19T06:47:41.172292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:49.867713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830418+0000","last_change":"2024-09-19T06:47:22.827742+0000","last_active":"2024-09-19T06:47:40.830418+0000","last_peered":"2024-09-19T06:47:40.830418+0000","last_clean":"2024-09-19T06:47:40.830418+0000","last_became_active":"2024-09-19T06:47:22.827371+0000","last_became_peered":"2024-09-19T06:47:22.827371+0000","last_unstale":"2024-09-19T06:47:40.830418+0000","last_undegraded":"2024-09-19T06:47:40.830418+0000","last_fullsized":"2024-09-19T06:47:40.830418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:35.153129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833435+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:47:40.833435+0000","last_peered":"2024-09-19T06:47:40.833435+0000","last_clean":"2024-09-19T06:47:40.833435+0000","last_became_active":"2024-09-19T06:47:24.841693+0000","last_became_peered":"2024-09-19T06:47:24.841693+0000","last_unstale":"2024-09-19T06:47:40.833435+0000","last_undegraded":"2024-09-19T06:47:40.833435+0000","last_fullsized":"2024-09-19T06:47:40.833435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:27.828296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834254+0000","last_change":"2024-09-19T06:47:22.832531+0000","last_active":"2024-09-19T06:47:40.834254+0000","last_peered":"2024-09-19T06:47:40.834254+0000","last_clean":"2024-09-19T06:47:40.834254+0000","last_became_active":"2024-09-19T06:47:22.832213+0000","last_became_peered":"2024-09-19T06:47:22.832213+0000","last_unstale":"2024-09-19T06:47:40.834254+0000","last_undegraded":"2024-09-19T06:47:40.834254+0000","last_fullsized":"2024-09-19T06:47:40.834254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:46:03.168406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833353+0000","last_change":"2024-09-19T06:47:24.837690+0000","last_active":"2024-09-19T06:47:40.833353+0000","last_peered":"2024-09-19T06:47:40.833353+0000","last_clean":"2024-09-19T06:47:40.833353+0000","last_became_active":"2024-09-19T06:47:24.837389+0000","last_became_peered":"2024-09-19T06:47:24.837389+0000","last_unstale":"2024-09-19T06:47:40.833353+0000","last_undegraded":"2024-09-19T06:47:40.833353+0000","last_fullsized":"2024-09-19T06:47:40.833353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:20.653832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:44.478429+0000","last_change":"2024-09-19T06:47:22.833808+0000","last_active":"2024-09-19T06:47:44.478429+0000","last_peered":"2024-09-19T06:47:44.478429+0000","last_clean":"2024-09-19T06:47:44.478429+0000","last_became_active":"2024-09-19T06:47:22.832468+0000","last_became_peered":"2024-09-19T06:47:22.832468+0000","last_unstale":"2024-09-19T06:47:44.478429+0000","last_undegraded":"2024-09-19T06:47:44.478429+0000","last_fullsized":"2024-09-19T06:47:44.478429+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:39:31.131321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172236+0000","last_change":"2024-09-19T06:47:24.840928+0000","last_active":"2024-09-19T06:47:41.172236+0000","last_peered":"2024-09-19T06:47:41.172236+0000","last_clean":"2024-09-19T06:47:41.172236+0000","last_became_active":"2024-09-19T06:47:24.839928+0000","last_became_peered":"2024-09-19T06:47:24.839928+0000","last_unstale":"2024-09-19T06:47:41.172236+0000","last_undegraded":"2024-09-19T06:47:41.172236+0000","last_fullsized":"2024-09-19T06:47:41.172236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:29.486003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830932+0000","last_change":"2024-09-19T06:47:22.833310+0000","last_active":"2024-09-19T06:47:40.830932+0000","last_peered":"2024-09-19T06:47:40.830932+0000","last_clean":"2024-09-19T06:47:40.830932+0000","last_became_active":"2024-09-19T06:47:22.832976+0000","last_became_peered":"2024-09-19T06:47:22.832976+0000","last_unstale":"2024-09-19T06:47:40.830932+0000","last_undegraded":"2024-09-19T06:47:40.830932+0000","last_fullsized":"2024-09-19T06:47:40.830932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:47.604853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.840138+0000","last_change":"2024-09-19T06:47:24.848687+0000","last_active":"2024-09-19T06:47:40.840138+0000","last_peered":"2024-09-19T06:47:40.840138+0000","last_clean":"2024-09-19T06:47:40.840138+0000","last_became_active":"2024-09-19T06:47:24.848352+0000","last_became_peered":"2024-09-19T06:47:24.848352+0000","last_unstale":"2024-09-19T06:47:40.840138+0000","last_undegraded":"2024-09-19T06:47:40.840138+0000","last_fullsized":"2024-09-19T06:47:40.840138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:54.459206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.835734+0000","last_change":"2024-09-19T06:47:22.851479+0000","last_active":"2024-09-19T06:47:40.835734+0000","last_peered":"2024-09-19T06:47:40.835734+0000","last_clean":"2024-09-19T06:47:40.835734+0000","last_became_active":"2024-09-19T06:47:22.851125+0000","last_became_peered":"2024-09-19T06:47:22.851125+0000","last_unstale":"2024-09-19T06:47:40.835734+0000","last_undegraded":"2024-09-19T06:47:40.835734+0000","last_fullsized":"2024-09-19T06:47:40.835734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:42.954034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172823+0000","last_change":"2024-09-19T06:47:24.851007+0000","last_active":"2024-09-19T06:47:41.172823+0000","last_peered":"2024-09-19T06:47:41.172823+0000","last_clean":"2024-09-19T06:47:41.172823+0000","last_became_active":"2024-09-19T06:47:24.850465+0000","last_became_peered":"2024-09-19T06:47:24.850465+0000","last_unstale":"2024-09-19T06:47:41.172823+0000","last_undegraded":"2024-09-19T06:47:41.172823+0000","last_fullsized":"2024-09-19T06:47:41.172823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:10.824672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'24","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:44.750340+0000","last_change":"2024-09-19T06:47:22.834893+0000","last_active":"2024-09-19T06:47:44.750340+0000","last_peered":"2024-09-19T06:47:44.750340+0000","last_clean":"2024-09-19T06:47:44.750340+0000","last_became_active":"2024-09-19T06:47:22.834521+0000","last_became_peered":"2024-09-19T06:47:22.834521+0000","last_unstale":"2024-09-19T06:47:44.750340+0000","last_undegraded":"2024-09-19T06:47:44.750340+0000","last_fullsized":"2024-09-19T06:47:44.750340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:29.742284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":23,"num_read_kb":51,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833599+0000","last_change":"2024-09-19T06:47:24.848191+0000","last_active":"2024-09-19T06:47:40.833599+0000","last_peered":"2024-09-19T06:47:40.833599+0000","last_clean":"2024-09-19T06:47:40.833599+0000","last_became_active":"2024-09-19T06:47:24.847889+0000","last_became_peered":"2024-09-19T06:47:24.847889+0000","last_unstale":"2024-09-19T06:47:40.833599+0000","last_undegraded":"2024-09-19T06:47:40.833599+0000","last_fullsized":"2024-09-19T06:47:40.833599+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:32.574360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830250+0000","last_change":"2024-09-19T06:47:22.832005+0000","last_active":"2024-09-19T06:47:40.830250+0000","last_peered":"2024-09-19T06:47:40.830250+0000","last_clean":"2024-09-19T06:47:40.830250+0000","last_became_active":"2024-09-19T06:47:22.831342+0000","last_became_peered":"2024-09-19T06:47:22.831342+0000","last_unstale":"2024-09-19T06:47:40.830250+0000","last_undegraded":"2024-09-19T06:47:40.830250+0000","last_fullsized":"2024-09-19T06:47:40.830250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:53.444748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830453+0000","last_change":"2024-09-19T06:47:24.843328+0000","last_active":"2024-09-19T06:47:40.830453+0000","last_peered":"2024-09-19T06:47:40.830453+0000","last_clean":"2024-09-19T06:47:40.830453+0000","last_became_active":"2024-09-19T06:47:24.842344+0000","last_became_peered":"2024-09-19T06:47:24.842344+0000","last_unstale":"2024-09-19T06:47:40.830453+0000","last_undegraded":"2024-09-19T06:47:40.830453+0000","last_fullsized":"2024-09-19T06:47:40.830453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:11.247819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.844801+0000","last_change":"2024-09-19T06:47:40.844086+0000","last_active":"2024-09-19T06:47:40.844801+0000","last_peered":"2024-09-19T06:47:40.844801+0000","last_clean":"2024-09-19T06:47:40.844801+0000","last_became_active":"2024-09-19T06:47:40.835015+0000","last_became_peered":"2024-09-19T06:47:40.835015+0000","last_unstale":"2024-09-19T06:47:40.844801+0000","last_undegraded":"2024-09-19T06:47:40.844801+0000","last_fullsized":"2024-09-19T06:47:40.844801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:09.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.836466+0000","last_change":"2024-09-19T06:47:40.836466+0000","last_active":"2024-09-19T06:47:40.836466+0000","last_peered":"2024-09-19T06:47:40.836466+0000","last_clean":"2024-09-19T06:47:40.836466+0000","last_became_active":"2024-09-19T06:47:40.836237+0000","last_became_peered":"2024-09-19T06:47:40.836237+0000","last_unstale":"2024-09-19T06:47:40.836466+0000","last_undegraded":"2024-09-19T06:47:40.836466+0000","last_fullsized":"2024-09-19T06:47:40.836466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:00.094685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:45.481085+0000","last_change":"2024-09-19T06:47:22.830359+0000","last_active":"2024-09-19T06:47:45.481085+0000","last_peered":"2024-09-19T06:47:45.481085+0000","last_clean":"2024-09-19T06:47:45.481085+0000","last_became_active":"2024-09-19T06:47:22.829960+0000","last_became_peered":"2024-09-19T06:47:22.829960+0000","last_unstale":"2024-09-19T06:47:45.481085+0000","last_undegraded":"2024-09-19T06:47:45.481085+0000","last_fullsized":"2024-09-19T06:47:45.481085+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:55.998243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830424+0000","last_change":"2024-09-19T06:47:24.836668+0000","last_active":"2024-09-19T06:47:40.830424+0000","last_peered":"2024-09-19T06:47:40.830424+0000","last_clean":"2024-09-19T06:47:40.830424+0000","last_became_active":"2024-09-19T06:47:24.836393+0000","last_became_peered":"2024-09-19T06:47:24.836393+0000","last_unstale":"2024-09-19T06:47:40.830424+0000","last_undegraded":"2024-09-19T06:47:40.830424+0000","last_fullsized":"2024-09-19T06:47:40.830424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:26.248948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.858323+0000","last_change":"2024-09-19T06:47:40.858323+0000","last_active":"2024-09-19T06:47:40.858323+0000","last_peered":"2024-09-19T06:47:40.858323+0000","last_clean":"2024-09-19T06:47:40.858323+0000","last_became_active":"2024-09-19T06:47:40.840802+0000","last_became_peered":"2024-09-19T06:47:40.840802+0000","last_unstale":"2024-09-19T06:47:40.858323+0000","last_undegraded":"2024-09-19T06:47:40.858323+0000","last_fullsized":"2024-09-19T06:47:40.858323+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:14.666916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834268+0000","last_change":"2024-09-19T06:47:24.843681+0000","last_active":"2024-09-19T06:47:40.834268+0000","last_peered":"2024-09-19T06:47:40.834268+0000","last_clean":"2024-09-19T06:47:40.834268+0000","last_became_active":"2024-09-19T06:47:24.841228+0000","last_became_peered":"2024-09-19T06:47:24.841228+0000","last_unstale":"2024-09-19T06:47:40.834268+0000","last_undegraded":"2024-09-19T06:47:40.834268+0000","last_fullsized":"2024-09-19T06:47:40.834268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:56.418962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833647+0000","last_change":"2024-09-19T06:47:22.833893+0000","last_active":"2024-09-19T06:47:40.833647+0000","last_peered":"2024-09-19T06:47:40.833647+0000","last_clean":"2024-09-19T06:47:40.833647+0000","last_became_active":"2024-09-19T06:47:22.833217+0000","last_became_peered":"2024-09-19T06:47:22.833217+0000","last_unstale":"2024-09-19T06:47:40.833647+0000","last_undegraded":"2024-09-19T06:47:40.833647+0000","last_fullsized":"2024-09-19T06:47:40.833647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:19.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831588+0000","last_change":"2024-09-19T06:47:24.848399+0000","last_active":"2024-09-19T06:47:40.831588+0000","last_peered":"2024-09-19T06:47:40.831588+0000","last_clean":"2024-09-19T06:47:40.831588+0000","last_became_active":"2024-09-19T06:47:24.848081+0000","last_became_peered":"2024-09-19T06:47:24.848081+0000","last_unstale":"2024-09-19T06:47:40.831588+0000","last_undegraded":"2024-09-19T06:47:40.831588+0000","last_fullsized":"2024-09-19T06:47:40.831588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:44.292939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.860955+0000","last_change":"2024-09-19T06:47:40.860955+0000","last_active":"2024-09-19T06:47:40.860955+0000","last_peered":"2024-09-19T06:47:40.860955+0000","last_clean":"2024-09-19T06:47:40.860955+0000","last_became_active":"2024-09-19T06:47:40.835126+0000","last_became_peered":"2024-09-19T06:47:40.835126+0000","last_unstale":"2024-09-19T06:47:40.860955+0000","last_undegraded":"2024-09-19T06:47:40.860955+0000","last_fullsized":"2024-09-19T06:47:40.860955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:02:26.763229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830803+0000","last_change":"2024-09-19T06:47:24.849680+0000","last_active":"2024-09-19T06:47:40.830803+0000","last_peered":"2024-09-19T06:47:40.830803+0000","last_clean":"2024-09-19T06:47:40.830803+0000","last_became_active":"2024-09-19T06:47:24.849323+0000","last_became_peered":"2024-09-19T06:47:24.849323+0000","last_unstale":"2024-09-19T06:47:40.830803+0000","last_undegraded":"2024-09-19T06:47:40.830803+0000","last_fullsized":"2024-09-19T06:47:40.830803+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:59.763503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.835875+0000","last_change":"2024-09-19T06:47:22.831689+0000","last_active":"2024-09-19T06:47:40.835875+0000","last_peered":"2024-09-19T06:47:40.835875+0000","last_clean":"2024-09-19T06:47:40.835875+0000","last_became_active":"2024-09-19T06:47:22.829283+0000","last_became_peered":"2024-09-19T06:47:22.829283+0000","last_unstale":"2024-09-19T06:47:40.835875+0000","last_undegraded":"2024-09-19T06:47:40.835875+0000","last_fullsized":"2024-09-19T06:47:40.835875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:37.109542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830659+0000","last_change":"2024-09-19T06:47:24.842793+0000","last_active":"2024-09-19T06:47:40.830659+0000","last_peered":"2024-09-19T06:47:40.830659+0000","last_clean":"2024-09-19T06:47:40.830659+0000","last_became_active":"2024-09-19T06:47:24.842474+0000","last_became_peered":"2024-09-19T06:47:24.842474+0000","last_unstale":"2024-09-19T06:47:40.830659+0000","last_undegraded":"2024-09-19T06:47:40.830659+0000","last_fullsized":"2024-09-19T06:47:40.830659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:23.110736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'6","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:45.480803+0000","last_change":"2024-09-19T06:47:23.047356+0000","last_active":"2024-09-19T06:47:45.480803+0000","last_peered":"2024-09-19T06:47:45.480803+0000","last_clean":"2024-09-19T06:47:45.480803+0000","last_became_active":"2024-09-19T06:47:23.046941+0000","last_became_peered":"2024-09-19T06:47:23.046941+0000","last_unstale":"2024-09-19T06:47:45.480803+0000","last_undegraded":"2024-09-19T06:47:45.480803+0000","last_fullsized":"2024-09-19T06:47:45.480803+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:43.490952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830805+0000","last_change":"2024-09-19T06:47:24.851025+0000","last_active":"2024-09-19T06:47:40.830805+0000","last_peered":"2024-09-19T06:47:40.830805+0000","last_clean":"2024-09-19T06:47:40.830805+0000","last_became_active":"2024-09-19T06:47:24.850681+0000","last_became_peered":"2024-09-19T06:47:24.850681+0000","last_unstale":"2024-09-19T06:47:40.830805+0000","last_undegraded":"2024-09-19T06:47:40.830805+0000","last_fullsized":"2024-09-19T06:47:40.830805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:46.884747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":345,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830558+0000","last_change":"2024-09-19T06:46:13.927379+0000","last_active":"2024-09-19T06:47:40.830558+0000","last_peered":"2024-09-19T06:47:40.830558+0000","last_clean":"2024-09-19T06:47:40.830558+0000","last_became_active":"2024-09-19T06:46:13.926926+0000","last_became_peered":"2024-09-19T06:46:13.926926+0000","last_unstale":"2024-09-19T06:47:40.830558+0000","last_undegraded":"2024-09-19T06:47:40.830558+0000","last_fullsized":"2024-09-19T06:47:40.830558+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_clean_scrub_stamp":"2024-09-19T06:42:52.051335+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:29.542128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829278+0000","last_change":"2024-09-19T06:47:22.830387+0000","last_active":"2024-09-19T06:47:40.829278+0000","last_peered":"2024-09-19T06:47:40.829278+0000","last_clean":"2024-09-19T06:47:40.829278+0000","last_became_active":"2024-09-19T06:47:22.829970+0000","last_became_peered":"2024-09-19T06:47:22.829970+0000","last_unstale":"2024-09-19T06:47:40.829278+0000","last_undegraded":"2024-09-19T06:47:40.829278+0000","last_fullsized":"2024-09-19T06:47:40.829278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:51.385456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834053+0000","last_change":"2024-09-19T06:47:24.843752+0000","last_active":"2024-09-19T06:47:40.834053+0000","last_peered":"2024-09-19T06:47:40.834053+0000","last_clean":"2024-09-19T06:47:40.834053+0000","last_became_active":"2024-09-19T06:47:24.842631+0000","last_became_peered":"2024-09-19T06:47:24.842631+0000","last_unstale":"2024-09-19T06:47:40.834053+0000","last_undegraded":"2024-09-19T06:47:40.834053+0000","last_fullsized":"2024-09-19T06:47:40.834053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:19.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830650+0000","last_change":"2024-09-19T06:47:22.832647+0000","last_active":"2024-09-19T06:47:40.830650+0000","last_peered":"2024-09-19T06:47:40.830650+0000","last_clean":"2024-09-19T06:47:40.830650+0000","last_became_active":"2024-09-19T06:47:22.831774+0000","last_became_peered":"2024-09-19T06:47:22.831774+0000","last_unstale":"2024-09-19T06:47:40.830650+0000","last_undegraded":"2024-09-19T06:47:40.830650+0000","last_fullsized":"2024-09-19T06:47:40.830650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:57.824610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830409+0000","last_change":"2024-09-19T06:47:24.844301+0000","last_active":"2024-09-19T06:47:40.830409+0000","last_peered":"2024-09-19T06:47:40.830409+0000","last_clean":"2024-09-19T06:47:40.830409+0000","last_became_active":"2024-09-19T06:47:24.839748+0000","last_became_peered":"2024-09-19T06:47:24.839748+0000","last_unstale":"2024-09-19T06:47:40.830409+0000","last_undegraded":"2024-09-19T06:47:40.830409+0000","last_fullsized":"2024-09-19T06:47:40.830409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:46.407693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:44.478973+0000","last_change":"2024-09-19T06:47:22.851008+0000","last_active":"2024-09-19T06:47:44.478973+0000","last_peered":"2024-09-19T06:47:44.478973+0000","last_clean":"2024-09-19T06:47:44.478973+0000","last_became_active":"2024-09-19T06:47:22.850601+0000","last_became_peered":"2024-09-19T06:47:22.850601+0000","last_unstale":"2024-09-19T06:47:44.478973+0000","last_undegraded":"2024-09-19T06:47:44.478973+0000","last_fullsized":"2024-09-19T06:47:44.478973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:04.610277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834177+0000","last_change":"2024-09-19T06:47:24.839684+0000","last_active":"2024-09-19T06:47:40.834177+0000","last_peered":"2024-09-19T06:47:40.834177+0000","last_clean":"2024-09-19T06:47:40.834177+0000","last_became_active":"2024-09-19T06:47:24.839126+0000","last_became_peered":"2024-09-19T06:47:24.839126+0000","last_unstale":"2024-09-19T06:47:40.834177+0000","last_undegraded":"2024-09-19T06:47:40.834177+0000","last_fullsized":"2024-09-19T06:47:40.834177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:09.458535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:44.179598+0000","last_change":"2024-09-19T06:47:22.831819+0000","last_active":"2024-09-19T06:47:44.179598+0000","last_peered":"2024-09-19T06:47:44.179598+0000","last_clean":"2024-09-19T06:47:44.179598+0000","last_became_active":"2024-09-19T06:47:22.831332+0000","last_became_peered":"2024-09-19T06:47:22.831332+0000","last_unstale":"2024-09-19T06:47:44.179598+0000","last_undegraded":"2024-09-19T06:47:44.179598+0000","last_fullsized":"2024-09-19T06:47:44.179598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:35.032949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.171750+0000","last_change":"2024-09-19T06:47:24.846738+0000","last_active":"2024-09-19T06:47:41.171750+0000","last_peered":"2024-09-19T06:47:41.171750+0000","last_clean":"2024-09-19T06:47:41.171750+0000","last_became_active":"2024-09-19T06:47:24.846389+0000","last_became_peered":"2024-09-19T06:47:24.846389+0000","last_unstale":"2024-09-19T06:47:41.171750+0000","last_undegraded":"2024-09-19T06:47:41.171750+0000","last_fullsized":"2024-09-19T06:47:41.171750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:19.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833696+0000","last_change":"2024-09-19T06:47:22.851065+0000","last_active":"2024-09-19T06:47:40.833696+0000","last_peered":"2024-09-19T06:47:40.833696+0000","last_clean":"2024-09-19T06:47:40.833696+0000","last_became_active":"2024-09-19T06:47:22.850647+0000","last_became_peered":"2024-09-19T06:47:22.850647+0000","last_unstale":"2024-09-19T06:47:40.833696+0000","last_undegraded":"2024-09-19T06:47:40.833696+0000","last_fullsized":"2024-09-19T06:47:40.833696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:41.993501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172204+0000","last_change":"2024-09-19T06:47:24.838905+0000","last_active":"2024-09-19T06:47:41.172204+0000","last_peered":"2024-09-19T06:47:41.172204+0000","last_clean":"2024-09-19T06:47:41.172204+0000","last_became_active":"2024-09-19T06:47:24.838443+0000","last_became_peered":"2024-09-19T06:47:24.838443+0000","last_unstale":"2024-09-19T06:47:41.172204+0000","last_undegraded":"2024-09-19T06:47:41.172204+0000","last_fullsized":"2024-09-19T06:47:41.172204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:12.486137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833766+0000","last_change":"2024-09-19T06:47:22.851431+0000","last_active":"2024-09-19T06:47:40.833766+0000","last_peered":"2024-09-19T06:47:40.833766+0000","last_clean":"2024-09-19T06:47:40.833766+0000","last_became_active":"2024-09-19T06:47:22.851092+0000","last_became_peered":"2024-09-19T06:47:22.851092+0000","last_unstale":"2024-09-19T06:47:40.833766+0000","last_undegraded":"2024-09-19T06:47:40.833766+0000","last_fullsized":"2024-09-19T06:47:40.833766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:26.183803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830789+0000","last_change":"2024-09-19T06:47:24.839046+0000","last_active":"2024-09-19T06:47:40.830789+0000","last_peered":"2024-09-19T06:47:40.830789+0000","last_clean":"2024-09-19T06:47:40.830789+0000","last_became_active":"2024-09-19T06:47:24.838680+0000","last_became_peered":"2024-09-19T06:47:24.838680+0000","last_unstale":"2024-09-19T06:47:40.830789+0000","last_undegraded":"2024-09-19T06:47:40.830789+0000","last_fullsized":"2024-09-19T06:47:40.830789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:45.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:43.845607+0000","last_change":"2024-09-19T06:47:40.865241+0000","last_active":"2024-09-19T06:47:43.845607+0000","last_peered":"2024-09-19T06:47:43.845607+0000","last_clean":"2024-09-19T06:47:43.845607+0000","last_became_active":"2024-09-19T06:47:40.837765+0000","last_became_peered":"2024-09-19T06:47:40.837765+0000","last_unstale":"2024-09-19T06:47:43.845607+0000","last_undegraded":"2024-09-19T06:47:43.845607+0000","last_fullsized":"2024-09-19T06:47:43.845607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834162+0000","last_change":"2024-09-19T06:47:24.848599+0000","last_active":"2024-09-19T06:47:40.834162+0000","last_peered":"2024-09-19T06:47:40.834162+0000","last_clean":"2024-09-19T06:47:40.834162+0000","last_became_active":"2024-09-19T06:47:24.848305+0000","last_became_peered":"2024-09-19T06:47:24.848305+0000","last_unstale":"2024-09-19T06:47:40.834162+0000","last_undegraded":"2024-09-19T06:47:40.834162+0000","last_fullsized":"2024-09-19T06:47:40.834162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:31.496188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829462+0000","last_change":"2024-09-19T06:47:23.047484+0000","last_active":"2024-09-19T06:47:40.829462+0000","last_peered":"2024-09-19T06:47:40.829462+0000","last_clean":"2024-09-19T06:47:40.829462+0000","last_became_active":"2024-09-19T06:47:23.047058+0000","last_became_peered":"2024-09-19T06:47:23.047058+0000","last_unstale":"2024-09-19T06:47:40.829462+0000","last_undegraded":"2024-09-19T06:47:40.829462+0000","last_fullsized":"2024-09-19T06:47:40.829462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:34.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839396+0000","last_change":"2024-09-19T06:47:24.843498+0000","last_active":"2024-09-19T06:47:40.839396+0000","last_peered":"2024-09-19T06:47:40.839396+0000","last_clean":"2024-09-19T06:47:40.839396+0000","last_became_active":"2024-09-19T06:47:24.843204+0000","last_became_peered":"2024-09-19T06:47:24.843204+0000","last_unstale":"2024-09-19T06:47:40.839396+0000","last_undegraded":"2024-09-19T06:47:40.839396+0000","last_fullsized":"2024-09-19T06:47:40.839396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:25.585023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172992+0000","last_change":"2024-09-19T06:47:22.851519+0000","last_active":"2024-09-19T06:47:41.172992+0000","last_peered":"2024-09-19T06:47:41.172992+0000","last_clean":"2024-09-19T06:47:41.172992+0000","last_became_active":"2024-09-19T06:47:22.851163+0000","last_became_peered":"2024-09-19T06:47:22.851163+0000","last_unstale":"2024-09-19T06:47:41.172992+0000","last_undegraded":"2024-09-19T06:47:41.172992+0000","last_fullsized":"2024-09-19T06:47:41.172992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:04.502795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.832719+0000","last_change":"2024-09-19T06:47:24.845239+0000","last_active":"2024-09-19T06:47:40.832719+0000","last_peered":"2024-09-19T06:47:40.832719+0000","last_clean":"2024-09-19T06:47:40.832719+0000","last_became_active":"2024-09-19T06:47:24.844934+0000","last_became_peered":"2024-09-19T06:47:24.844934+0000","last_unstale":"2024-09-19T06:47:40.832719+0000","last_undegraded":"2024-09-19T06:47:40.832719+0000","last_fullsized":"2024-09-19T06:47:40.832719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:43.610604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834147+0000","last_change":"2024-09-19T06:47:22.830232+0000","last_active":"2024-09-19T06:47:40.834147+0000","last_peered":"2024-09-19T06:47:40.834147+0000","last_clean":"2024-09-19T06:47:40.834147+0000","last_became_active":"2024-09-19T06:47:22.829901+0000","last_became_peered":"2024-09-19T06:47:22.829901+0000","last_unstale":"2024-09-19T06:47:40.834147+0000","last_undegraded":"2024-09-19T06:47:40.834147+0000","last_fullsized":"2024-09-19T06:47:40.834147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:12.251427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831384+0000","last_change":"2024-09-19T06:47:24.839304+0000","last_active":"2024-09-19T06:47:40.831384+0000","last_peered":"2024-09-19T06:47:40.831384+0000","last_clean":"2024-09-19T06:47:40.831384+0000","last_became_active":"2024-09-19T06:47:24.838986+0000","last_became_peered":"2024-09-19T06:47:24.838986+0000","last_unstale":"2024-09-19T06:47:40.831384+0000","last_undegraded":"2024-09-19T06:47:40.831384+0000","last_fullsized":"2024-09-19T06:47:40.831384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:39.377886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830021+0000","last_change":"2024-09-19T06:47:22.828553+0000","last_active":"2024-09-19T06:47:40.830021+0000","last_peered":"2024-09-19T06:47:40.830021+0000","last_clean":"2024-09-19T06:47:40.830021+0000","last_became_active":"2024-09-19T06:47:22.828213+0000","last_became_peered":"2024-09-19T06:47:22.828213+0000","last_unstale":"2024-09-19T06:47:40.830021+0000","last_undegraded":"2024-09-19T06:47:40.830021+0000","last_fullsized":"2024-09-19T06:47:40.830021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:31.649761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833576+0000","last_change":"2024-09-19T06:47:24.840747+0000","last_active":"2024-09-19T06:47:40.833576+0000","last_peered":"2024-09-19T06:47:40.833576+0000","last_clean":"2024-09-19T06:47:40.833576+0000","last_became_active":"2024-09-19T06:47:24.840451+0000","last_became_peered":"2024-09-19T06:47:24.840451+0000","last_unstale":"2024-09-19T06:47:40.833576+0000","last_undegraded":"2024-09-19T06:47:40.833576+0000","last_fullsized":"2024-09-19T06:47:40.833576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:41.588751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833228+0000","last_change":"2024-09-19T06:47:22.831697+0000","last_active":"2024-09-19T06:47:40.833228+0000","last_peered":"2024-09-19T06:47:40.833228+0000","last_clean":"2024-09-19T06:47:40.833228+0000","last_became_active":"2024-09-19T06:47:22.831178+0000","last_became_peered":"2024-09-19T06:47:22.831178+0000","last_unstale":"2024-09-19T06:47:40.833228+0000","last_undegraded":"2024-09-19T06:47:40.833228+0000","last_fullsized":"2024-09-19T06:47:40.833228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:30.975782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830575+0000","last_change":"2024-09-19T06:47:24.845097+0000","last_active":"2024-09-19T06:47:40.830575+0000","last_peered":"2024-09-19T06:47:40.830575+0000","last_clean":"2024-09-19T06:47:40.830575+0000","last_became_active":"2024-09-19T06:47:24.844801+0000","last_became_peered":"2024-09-19T06:47:24.844801+0000","last_unstale":"2024-09-19T06:47:40.830575+0000","last_undegraded":"2024-09-19T06:47:40.830575+0000","last_fullsized":"2024-09-19T06:47:40.830575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:15.264118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.835620+0000","last_change":"2024-09-19T06:47:24.851372+0000","last_active":"2024-09-19T06:47:40.835620+0000","last_peered":"2024-09-19T06:47:40.835620+0000","last_clean":"2024-09-19T06:47:40.835620+0000","last_became_active":"2024-09-19T06:47:24.851044+0000","last_became_peered":"2024-09-19T06:47:24.851044+0000","last_unstale":"2024-09-19T06:47:40.835620+0000","last_undegraded":"2024-09-19T06:47:40.835620+0000","last_fullsized":"2024-09-19T06:47:40.835620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:55.658214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833103+0000","last_change":"2024-09-19T06:47:22.833694+0000","last_active":"2024-09-19T06:47:40.833103+0000","last_peered":"2024-09-19T06:47:40.833103+0000","last_clean":"2024-09-19T06:47:40.833103+0000","last_became_active":"2024-09-19T06:47:22.833215+0000","last_became_peered":"2024-09-19T06:47:22.833215+0000","last_unstale":"2024-09-19T06:47:40.833103+0000","last_undegraded":"2024-09-19T06:47:40.833103+0000","last_fullsized":"2024-09-19T06:47:40.833103+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:32.171522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172973+0000","last_change":"2024-09-19T06:47:24.845737+0000","last_active":"2024-09-19T06:47:41.172973+0000","last_peered":"2024-09-19T06:47:41.172973+0000","last_clean":"2024-09-19T06:47:41.172973+0000","last_became_active":"2024-09-19T06:47:24.845242+0000","last_became_peered":"2024-09-19T06:47:24.845242+0000","last_unstale":"2024-09-19T06:47:41.172973+0000","last_undegraded":"2024-09-19T06:47:41.172973+0000","last_fullsized":"2024-09-19T06:47:41.172973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:51.554946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833601+0000","last_change":"2024-09-19T06:47:22.829869+0000","last_active":"2024-09-19T06:47:40.833601+0000","last_peered":"2024-09-19T06:47:40.833601+0000","last_clean":"2024-09-19T06:47:40.833601+0000","last_became_active":"2024-09-19T06:47:22.829522+0000","last_became_peered":"2024-09-19T06:47:22.829522+0000","last_unstale":"2024-09-19T06:47:40.833601+0000","last_undegraded":"2024-09-19T06:47:40.833601+0000","last_fullsized":"2024-09-19T06:47:40.833601+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:26.871434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.840141+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:47:40.840141+0000","last_peered":"2024-09-19T06:47:40.840141+0000","last_clean":"2024-09-19T06:47:40.840141+0000","last_became_active":"2024-09-19T06:47:24.841289+0000","last_became_peered":"2024-09-19T06:47:24.841289+0000","last_unstale":"2024-09-19T06:47:40.840141+0000","last_undegraded":"2024-09-19T06:47:40.840141+0000","last_fullsized":"2024-09-19T06:47:40.840141+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:21:12.611458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834465+0000","last_change":"2024-09-19T06:47:22.852198+0000","last_active":"2024-09-19T06:47:40.834465+0000","last_peered":"2024-09-19T06:47:40.834465+0000","last_clean":"2024-09-19T06:47:40.834465+0000","last_became_active":"2024-09-19T06:47:22.850984+0000","last_became_peered":"2024-09-19T06:47:22.850984+0000","last_unstale":"2024-09-19T06:47:40.834465+0000","last_undegraded":"2024-09-19T06:47:40.834465+0000","last_fullsized":"2024-09-19T06:47:40.834465+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:13.620145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839399+0000","last_change":"2024-09-19T06:47:40.837614+0000","last_active":"2024-09-19T06:47:40.839399+0000","last_peered":"2024-09-19T06:47:40.839399+0000","last_clean":"2024-09-19T06:47:40.839399+0000","last_became_active":"2024-09-19T06:47:40.837259+0000","last_became_peered":"2024-09-19T06:47:40.837259+0000","last_unstale":"2024-09-19T06:47:40.839399+0000","last_undegraded":"2024-09-19T06:47:40.839399+0000","last_fullsized":"2024-09-19T06:47:40.839399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:41.429962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:44.178758+0000","last_change":"2024-09-19T06:47:41.182639+0000","last_active":"2024-09-19T06:47:44.178758+0000","last_peered":"2024-09-19T06:47:44.178758+0000","last_clean":"2024-09-19T06:47:44.178758+0000","last_became_active":"2024-09-19T06:47:41.174305+0000","last_became_peered":"2024-09-19T06:47:41.174305+0000","last_unstale":"2024-09-19T06:47:44.178758+0000","last_undegraded":"2024-09-19T06:47:44.178758+0000","last_fullsized":"2024-09-19T06:47:44.178758+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:10.199502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54010,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":129,"num_read_kb":117,"num_write":229,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":162030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579921,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":439122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1071126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972379,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":438569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234016,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":913408,"data_stored":438275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462948,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712484,"statfs":{"total":95995035648,"available":95961583616,"internally_reserved":0,"allocated":1519616,"data_stored":1031467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626474,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":909312,"data_stored":438286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855405,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1030662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018931,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":438652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215156,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":438111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444094,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":486502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":486673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":440018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":49226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":49055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:47:50.537 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:47:50.784 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:51.172 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:50 smithi103 ceph-mon[31158]: pgmap v287: 97 pgs: 97 active+clean; 630 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 KiB/s wr, 16 op/s; 87 B/s, 2 objects/s recovering 2024-09-19T06:47:51.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.174+0000 7fc47e703640 1 -- 172.21.15.103:0/1635434979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4781082e0 msgr2=0x7fc47810ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:51.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.174+0000 7fc47e703640 1 --2- 172.21.15.103:0/1635434979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4781082e0 0x7fc47810ec50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc470009cd0 tx=0x7fc47002a910 comp rx=0 tx=0).stop 2024-09-19T06:47:51.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.174+0000 7fc47e703640 1 -- 172.21.15.103:0/1635434979 shutdown_connections 2024-09-19T06:47:51.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.174+0000 7fc47e703640 1 --2- 172.21.15.103:0/1635434979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4781082e0 0x7fc47810ec50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.174+0000 7fc47e703640 1 --2- 172.21.15.103:0/1635434979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc478103760 0x7fc478107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.174+0000 7fc47e703640 1 --2- 172.21.15.103:0/1635434979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc478102db0 0x7fc478103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.174+0000 7fc47e703640 1 -- 172.21.15.103:0/1635434979 >> 172.21.15.103:0/1635434979 conn(0x7fc4780fc7d0 msgr2=0x7fc4780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:51.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.174+0000 7fc47e703640 1 -- 172.21.15.103:0/1635434979 shutdown_connections 2024-09-19T06:47:51.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.174+0000 7fc47e703640 1 -- 172.21.15.103:0/1635434979 wait complete. 2024-09-19T06:47:51.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.175+0000 7fc47e703640 1 Processor -- start 2024-09-19T06:47:51.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.175+0000 7fc47e703640 1 -- start start 2024-09-19T06:47:51.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.175+0000 7fc47e703640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc478102db0 0x7fc4781986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.175+0000 7fc47e703640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc478103760 0x7fc478198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.175+0000 7fc47e703640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4781082e0 0x7fc47819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.175+0000 7fc47e703640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc478112170 con 0x7fc4781082e0 2024-09-19T06:47:51.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.175+0000 7fc47e703640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc478111ff0 con 0x7fc478103760 2024-09-19T06:47:51.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.175+0000 7fc47e703640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781122f0 con 0x7fc478102db0 2024-09-19T06:47:51.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.175+0000 7fc47cca0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4781082e0 0x7fc47819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.175+0000 7fc47cca0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4781082e0 0x7fc47819cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47302/0 (socket says 172.21.15.103:47302) 2024-09-19T06:47:51.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.175+0000 7fc47cca0640 1 -- 172.21.15.103:0/2033849874 learned_addr learned my addr 172.21.15.103:0/2033849874 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:51.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.176+0000 7fc47cca0640 1 -- 172.21.15.103:0/2033849874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc478102db0 msgr2=0x7fc4781986a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T06:47:51.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.176+0000 7fc47cca0640 1 --2- 172.21.15.103:0/2033849874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc478102db0 0x7fc4781986a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.176+0000 7fc47cca0640 1 -- 172.21.15.103:0/2033849874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc478103760 msgr2=0x7fc478198be0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T06:47:51.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.176+0000 7fc47cca0640 1 --2- 172.21.15.103:0/2033849874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc478103760 0x7fc478198be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.176+0000 7fc47cca0640 1 -- 172.21.15.103:0/2033849874 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47819d650 con 0x7fc4781082e0 2024-09-19T06:47:51.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.176+0000 7fc47cca0640 1 --2- 172.21.15.103:0/2033849874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4781082e0 0x7fc47819cf70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc470009ca0 tx=0x7fc470003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:51.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.176+0000 7fc4757fa640 1 -- 172.21.15.103:0/2033849874 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4700317e0 con 0x7fc4781082e0 2024-09-19T06:47:51.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.176+0000 7fc47e703640 1 -- 172.21.15.103:0/2033849874 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47819d8e0 con 0x7fc4781082e0 2024-09-19T06:47:51.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.176+0000 7fc4757fa640 1 -- 172.21.15.103:0/2033849874 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc470031980 con 0x7fc4781082e0 2024-09-19T06:47:51.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.176+0000 7fc4757fa640 1 -- 172.21.15.103:0/2033849874 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47002fdd0 con 0x7fc4781082e0 2024-09-19T06:47:51.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.176+0000 7fc47e703640 1 -- 172.21.15.103:0/2033849874 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4781a5220 con 0x7fc4781082e0 2024-09-19T06:47:51.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.177+0000 7fc47e703640 1 -- 172.21.15.103:0/2033849874 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc478103210 con 0x7fc4781082e0 2024-09-19T06:47:51.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.178+0000 7fc4757fa640 1 -- 172.21.15.103:0/2033849874 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc470004040 con 0x7fc4781082e0 2024-09-19T06:47:51.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.179+0000 7fc4757fa640 1 --2- 172.21.15.103:0/2033849874 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc4600a32d0 0x7fc4600a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.179+0000 7fc477fff640 1 --2- 172.21.15.103:0/2033849874 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc4600a32d0 0x7fc4600a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.179+0000 7fc4757fa640 1 -- 172.21.15.103:0/2033849874 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc4700f0fe0 con 0x7fc4781082e0 2024-09-19T06:47:51.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.179+0000 7fc477fff640 1 --2- 172.21.15.103:0/2033849874 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc4600a32d0 0x7fc4600a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc4680096b0 tx=0x7fc468006c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:51.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.183+0000 7fc4757fa640 1 -- 172.21.15.103:0/2033849874 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc4700bdcc0 con 0x7fc4781082e0 2024-09-19T06:47:51.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:50 smithi134 ceph-mon[35356]: pgmap v287: 97 pgs: 97 active+clean; 630 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 KiB/s wr, 16 op/s; 87 B/s, 2 objects/s recovering 2024-09-19T06:47:51.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:50 smithi174 ceph-mon[35637]: pgmap v287: 97 pgs: 97 active+clean; 630 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 KiB/s wr, 16 op/s; 87 B/s, 2 objects/s recovering 2024-09-19T06:47:51.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.353+0000 7fc47e703640 1 -- 172.21.15.103:0/2033849874 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc478062630 con 0x7fc4600a32d0 2024-09-19T06:47:51.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.359+0000 7fc4757fa640 1 -- 172.21.15.103:0/2033849874 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258767 (secure 0 0 0) 0x7fc478062630 con 0x7fc4600a32d0 2024-09-19T06:47:51.363 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:47:51.364 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T06:47:51.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.362+0000 7fc47e703640 1 -- 172.21.15.103:0/2033849874 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc4600a32d0 msgr2=0x7fc4600a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:51.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.362+0000 7fc47e703640 1 --2- 172.21.15.103:0/2033849874 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc4600a32d0 0x7fc4600a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc4680096b0 tx=0x7fc468006c20 comp rx=0 tx=0).stop 2024-09-19T06:47:51.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.362+0000 7fc47e703640 1 -- 172.21.15.103:0/2033849874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4781082e0 msgr2=0x7fc47819cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:51.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.362+0000 7fc47e703640 1 --2- 172.21.15.103:0/2033849874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4781082e0 0x7fc47819cf70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc470009ca0 tx=0x7fc470003f90 comp rx=0 tx=0).stop 2024-09-19T06:47:51.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.362+0000 7fc47e703640 1 -- 172.21.15.103:0/2033849874 shutdown_connections 2024-09-19T06:47:51.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.362+0000 7fc47e703640 1 --2- 172.21.15.103:0/2033849874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4781082e0 0x7fc47819cf70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.362+0000 7fc47e703640 1 --2- 172.21.15.103:0/2033849874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc478103760 0x7fc478198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.362+0000 7fc47e703640 1 --2- 172.21.15.103:0/2033849874 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc4600a32d0 0x7fc4600a5790 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.362+0000 7fc47e703640 1 --2- 172.21.15.103:0/2033849874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc478102db0 0x7fc4781986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.362+0000 7fc47e703640 1 -- 172.21.15.103:0/2033849874 >> 172.21.15.103:0/2033849874 conn(0x7fc4780fc7d0 msgr2=0x7fc478105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:51.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.362+0000 7fc47e703640 1 -- 172.21.15.103:0/2033849874 shutdown_connections 2024-09-19T06:47:51.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:51.362+0000 7fc47e703640 1 -- 172.21.15.103:0/2033849874 wait complete. 2024-09-19T06:47:52.119 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:51 smithi103 ceph-mon[31158]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:47:52.147 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-19T06:47:49.823777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644612,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":265,"num_read_kb":236,"num_write":412,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":1181610,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":338,"ondisk_log_size":338,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353644,"kb_used_data":12688,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588180,"statfs":{"total":1151940427776,"available":1151578296320,"internally_reserved":0,"allocated":12992512,"data_stored":7177463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46910,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":115,"num_read_kb":105,"num_write":48,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002512"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831591+0000","last_change":"2024-09-19T06:47:24.853118+0000","last_active":"2024-09-19T06:47:40.831591+0000","last_peered":"2024-09-19T06:47:40.831591+0000","last_clean":"2024-09-19T06:47:40.831591+0000","last_became_active":"2024-09-19T06:47:24.852801+0000","last_became_peered":"2024-09-19T06:47:24.852801+0000","last_unstale":"2024-09-19T06:47:40.831591+0000","last_undegraded":"2024-09-19T06:47:40.831591+0000","last_fullsized":"2024-09-19T06:47:40.831591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:16.729103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839300+0000","last_change":"2024-09-19T06:47:24.841095+0000","last_active":"2024-09-19T06:47:40.839300+0000","last_peered":"2024-09-19T06:47:40.839300+0000","last_clean":"2024-09-19T06:47:40.839300+0000","last_became_active":"2024-09-19T06:47:24.840774+0000","last_became_peered":"2024-09-19T06:47:24.840774+0000","last_unstale":"2024-09-19T06:47:40.839300+0000","last_undegraded":"2024-09-19T06:47:40.839300+0000","last_fullsized":"2024-09-19T06:47:40.839300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:27.262766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839649+0000","last_change":"2024-09-19T06:47:24.848444+0000","last_active":"2024-09-19T06:47:40.839649+0000","last_peered":"2024-09-19T06:47:40.839649+0000","last_clean":"2024-09-19T06:47:40.839649+0000","last_became_active":"2024-09-19T06:47:24.847354+0000","last_became_peered":"2024-09-19T06:47:24.847354+0000","last_unstale":"2024-09-19T06:47:40.839649+0000","last_undegraded":"2024-09-19T06:47:40.839649+0000","last_fullsized":"2024-09-19T06:47:40.839649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:44.901265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829447+0000","last_change":"2024-09-19T06:47:24.851915+0000","last_active":"2024-09-19T06:47:40.829447+0000","last_peered":"2024-09-19T06:47:40.829447+0000","last_clean":"2024-09-19T06:47:40.829447+0000","last_became_active":"2024-09-19T06:47:24.851600+0000","last_became_peered":"2024-09-19T06:47:24.851600+0000","last_unstale":"2024-09-19T06:47:40.829447+0000","last_undegraded":"2024-09-19T06:47:40.829447+0000","last_fullsized":"2024-09-19T06:47:40.829447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:52.424288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833135+0000","last_change":"2024-09-19T06:47:24.843184+0000","last_active":"2024-09-19T06:47:40.833135+0000","last_peered":"2024-09-19T06:47:40.833135+0000","last_clean":"2024-09-19T06:47:40.833135+0000","last_became_active":"2024-09-19T06:47:24.842857+0000","last_became_peered":"2024-09-19T06:47:24.842857+0000","last_unstale":"2024-09-19T06:47:40.833135+0000","last_undegraded":"2024-09-19T06:47:40.833135+0000","last_fullsized":"2024-09-19T06:47:40.833135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:06.608699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.171834+0000","last_change":"2024-09-19T06:47:24.846631+0000","last_active":"2024-09-19T06:47:41.171834+0000","last_peered":"2024-09-19T06:47:41.171834+0000","last_clean":"2024-09-19T06:47:41.171834+0000","last_became_active":"2024-09-19T06:47:24.846271+0000","last_became_peered":"2024-09-19T06:47:24.846271+0000","last_unstale":"2024-09-19T06:47:41.171834+0000","last_undegraded":"2024-09-19T06:47:41.171834+0000","last_fullsized":"2024-09-19T06:47:41.171834+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.272273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830458+0000","last_change":"2024-09-19T06:47:24.840392+0000","last_active":"2024-09-19T06:47:40.830458+0000","last_peered":"2024-09-19T06:47:40.830458+0000","last_clean":"2024-09-19T06:47:40.830458+0000","last_became_active":"2024-09-19T06:47:24.840092+0000","last_became_peered":"2024-09-19T06:47:24.840092+0000","last_unstale":"2024-09-19T06:47:40.830458+0000","last_undegraded":"2024-09-19T06:47:40.830458+0000","last_fullsized":"2024-09-19T06:47:40.830458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:25.579096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833345+0000","last_change":"2024-09-19T06:47:24.843336+0000","last_active":"2024-09-19T06:47:40.833345+0000","last_peered":"2024-09-19T06:47:40.833345+0000","last_clean":"2024-09-19T06:47:40.833345+0000","last_became_active":"2024-09-19T06:47:24.842016+0000","last_became_peered":"2024-09-19T06:47:24.842016+0000","last_unstale":"2024-09-19T06:47:40.833345+0000","last_undegraded":"2024-09-19T06:47:40.833345+0000","last_fullsized":"2024-09-19T06:47:40.833345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:49.478359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172619+0000","last_change":"2024-09-19T06:47:24.847841+0000","last_active":"2024-09-19T06:47:41.172619+0000","last_peered":"2024-09-19T06:47:41.172619+0000","last_clean":"2024-09-19T06:47:41.172619+0000","last_became_active":"2024-09-19T06:47:24.847492+0000","last_became_peered":"2024-09-19T06:47:24.847492+0000","last_unstale":"2024-09-19T06:47:41.172619+0000","last_undegraded":"2024-09-19T06:47:41.172619+0000","last_fullsized":"2024-09-19T06:47:41.172619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:59.229022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.171833+0000","last_change":"2024-09-19T06:47:24.843773+0000","last_active":"2024-09-19T06:47:41.171833+0000","last_peered":"2024-09-19T06:47:41.171833+0000","last_clean":"2024-09-19T06:47:41.171833+0000","last_became_active":"2024-09-19T06:47:24.843409+0000","last_became_peered":"2024-09-19T06:47:24.843409+0000","last_unstale":"2024-09-19T06:47:41.171833+0000","last_undegraded":"2024-09-19T06:47:41.171833+0000","last_fullsized":"2024-09-19T06:47:41.171833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:22.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830801+0000","last_change":"2024-09-19T06:47:24.846637+0000","last_active":"2024-09-19T06:47:40.830801+0000","last_peered":"2024-09-19T06:47:40.830801+0000","last_clean":"2024-09-19T06:47:40.830801+0000","last_became_active":"2024-09-19T06:47:24.846337+0000","last_became_peered":"2024-09-19T06:47:24.846337+0000","last_unstale":"2024-09-19T06:47:40.830801+0000","last_undegraded":"2024-09-19T06:47:40.830801+0000","last_fullsized":"2024-09-19T06:47:40.830801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:23:35.869485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829501+0000","last_change":"2024-09-19T06:47:24.851399+0000","last_active":"2024-09-19T06:47:40.829501+0000","last_peered":"2024-09-19T06:47:40.829501+0000","last_clean":"2024-09-19T06:47:40.829501+0000","last_became_active":"2024-09-19T06:47:24.851090+0000","last_became_peered":"2024-09-19T06:47:24.851090+0000","last_unstale":"2024-09-19T06:47:40.829501+0000","last_undegraded":"2024-09-19T06:47:40.829501+0000","last_fullsized":"2024-09-19T06:47:40.829501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:52.839956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830657+0000","last_change":"2024-09-19T06:47:24.833858+0000","last_active":"2024-09-19T06:47:40.830657+0000","last_peered":"2024-09-19T06:47:40.830657+0000","last_clean":"2024-09-19T06:47:40.830657+0000","last_became_active":"2024-09-19T06:47:24.833616+0000","last_became_peered":"2024-09-19T06:47:24.833616+0000","last_unstale":"2024-09-19T06:47:40.830657+0000","last_undegraded":"2024-09-19T06:47:40.830657+0000","last_fullsized":"2024-09-19T06:47:40.830657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:54.918144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839219+0000","last_change":"2024-09-19T06:47:24.841030+0000","last_active":"2024-09-19T06:47:40.839219+0000","last_peered":"2024-09-19T06:47:40.839219+0000","last_clean":"2024-09-19T06:47:40.839219+0000","last_became_active":"2024-09-19T06:47:24.839631+0000","last_became_peered":"2024-09-19T06:47:24.839631+0000","last_unstale":"2024-09-19T06:47:40.839219+0000","last_undegraded":"2024-09-19T06:47:40.839219+0000","last_fullsized":"2024-09-19T06:47:40.839219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:21.764812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839091+0000","last_change":"2024-09-19T06:47:24.849281+0000","last_active":"2024-09-19T06:47:40.839091+0000","last_peered":"2024-09-19T06:47:40.839091+0000","last_clean":"2024-09-19T06:47:40.839091+0000","last_became_active":"2024-09-19T06:47:24.848977+0000","last_became_peered":"2024-09-19T06:47:24.848977+0000","last_unstale":"2024-09-19T06:47:40.839091+0000","last_undegraded":"2024-09-19T06:47:40.839091+0000","last_fullsized":"2024-09-19T06:47:40.839091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:36.693758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172068+0000","last_change":"2024-09-19T06:47:24.843619+0000","last_active":"2024-09-19T06:47:41.172068+0000","last_peered":"2024-09-19T06:47:41.172068+0000","last_clean":"2024-09-19T06:47:41.172068+0000","last_became_active":"2024-09-19T06:47:24.843323+0000","last_became_peered":"2024-09-19T06:47:24.843323+0000","last_unstale":"2024-09-19T06:47:41.172068+0000","last_undegraded":"2024-09-19T06:47:41.172068+0000","last_fullsized":"2024-09-19T06:47:41.172068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:52.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831828+0000","last_change":"2024-09-19T06:47:24.853165+0000","last_active":"2024-09-19T06:47:40.831828+0000","last_peered":"2024-09-19T06:47:40.831828+0000","last_clean":"2024-09-19T06:47:40.831828+0000","last_became_active":"2024-09-19T06:47:24.852864+0000","last_became_peered":"2024-09-19T06:47:24.852864+0000","last_unstale":"2024-09-19T06:47:40.831828+0000","last_undegraded":"2024-09-19T06:47:40.831828+0000","last_fullsized":"2024-09-19T06:47:40.831828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:38.803478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834536+0000","last_change":"2024-09-19T06:47:24.838151+0000","last_active":"2024-09-19T06:47:40.834536+0000","last_peered":"2024-09-19T06:47:40.834536+0000","last_clean":"2024-09-19T06:47:40.834536+0000","last_became_active":"2024-09-19T06:47:24.837855+0000","last_became_peered":"2024-09-19T06:47:24.837855+0000","last_unstale":"2024-09-19T06:47:40.834536+0000","last_undegraded":"2024-09-19T06:47:40.834536+0000","last_fullsized":"2024-09-19T06:47:40.834536+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:16.675557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830451+0000","last_change":"2024-09-19T06:47:24.843941+0000","last_active":"2024-09-19T06:47:40.830451+0000","last_peered":"2024-09-19T06:47:40.830451+0000","last_clean":"2024-09-19T06:47:40.830451+0000","last_became_active":"2024-09-19T06:47:24.843551+0000","last_became_peered":"2024-09-19T06:47:24.843551+0000","last_unstale":"2024-09-19T06:47:40.830451+0000","last_undegraded":"2024-09-19T06:47:40.830451+0000","last_fullsized":"2024-09-19T06:47:40.830451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:00:49.388265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.836228+0000","last_change":"2024-09-19T06:47:40.836228+0000","last_active":"2024-09-19T06:47:40.836228+0000","last_peered":"2024-09-19T06:47:40.836228+0000","last_clean":"2024-09-19T06:47:40.836228+0000","last_became_active":"2024-09-19T06:47:40.835818+0000","last_became_peered":"2024-09-19T06:47:40.835818+0000","last_unstale":"2024-09-19T06:47:40.836228+0000","last_undegraded":"2024-09-19T06:47:40.836228+0000","last_fullsized":"2024-09-19T06:47:40.836228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:45.341160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830676+0000","last_change":"2024-09-19T06:47:24.843746+0000","last_active":"2024-09-19T06:47:40.830676+0000","last_peered":"2024-09-19T06:47:40.830676+0000","last_clean":"2024-09-19T06:47:40.830676+0000","last_became_active":"2024-09-19T06:47:24.841495+0000","last_became_peered":"2024-09-19T06:47:24.841495+0000","last_unstale":"2024-09-19T06:47:40.830676+0000","last_undegraded":"2024-09-19T06:47:40.830676+0000","last_fullsized":"2024-09-19T06:47:40.830676+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:06:29.995595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.173085+0000","last_change":"2024-09-19T06:47:24.844747+0000","last_active":"2024-09-19T06:47:41.173085+0000","last_peered":"2024-09-19T06:47:41.173085+0000","last_clean":"2024-09-19T06:47:41.173085+0000","last_became_active":"2024-09-19T06:47:24.844425+0000","last_became_peered":"2024-09-19T06:47:24.844425+0000","last_unstale":"2024-09-19T06:47:41.173085+0000","last_undegraded":"2024-09-19T06:47:41.173085+0000","last_fullsized":"2024-09-19T06:47:41.173085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:10.309044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834053+0000","last_change":"2024-09-19T06:47:24.848227+0000","last_active":"2024-09-19T06:47:40.834053+0000","last_peered":"2024-09-19T06:47:40.834053+0000","last_clean":"2024-09-19T06:47:40.834053+0000","last_became_active":"2024-09-19T06:47:24.847914+0000","last_became_peered":"2024-09-19T06:47:24.847914+0000","last_unstale":"2024-09-19T06:47:40.834053+0000","last_undegraded":"2024-09-19T06:47:40.834053+0000","last_fullsized":"2024-09-19T06:47:40.834053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:47.281613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834014+0000","last_change":"2024-09-19T06:47:24.844275+0000","last_active":"2024-09-19T06:47:40.834014+0000","last_peered":"2024-09-19T06:47:40.834014+0000","last_clean":"2024-09-19T06:47:40.834014+0000","last_became_active":"2024-09-19T06:47:24.841442+0000","last_became_peered":"2024-09-19T06:47:24.841442+0000","last_unstale":"2024-09-19T06:47:40.834014+0000","last_undegraded":"2024-09-19T06:47:40.834014+0000","last_fullsized":"2024-09-19T06:47:40.834014+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:30.198655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834062+0000","last_change":"2024-09-19T06:47:24.846301+0000","last_active":"2024-09-19T06:47:40.834062+0000","last_peered":"2024-09-19T06:47:40.834062+0000","last_clean":"2024-09-19T06:47:40.834062+0000","last_became_active":"2024-09-19T06:47:24.845970+0000","last_became_peered":"2024-09-19T06:47:24.845970+0000","last_unstale":"2024-09-19T06:47:40.834062+0000","last_undegraded":"2024-09-19T06:47:40.834062+0000","last_fullsized":"2024-09-19T06:47:40.834062+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:13.245536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830554+0000","last_change":"2024-09-19T06:47:24.844444+0000","last_active":"2024-09-19T06:47:40.830554+0000","last_peered":"2024-09-19T06:47:40.830554+0000","last_clean":"2024-09-19T06:47:40.830554+0000","last_became_active":"2024-09-19T06:47:24.844141+0000","last_became_peered":"2024-09-19T06:47:24.844141+0000","last_unstale":"2024-09-19T06:47:40.830554+0000","last_undegraded":"2024-09-19T06:47:40.830554+0000","last_fullsized":"2024-09-19T06:47:40.830554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:44.739242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831557+0000","last_change":"2024-09-19T06:47:24.845633+0000","last_active":"2024-09-19T06:47:40.831557+0000","last_peered":"2024-09-19T06:47:40.831557+0000","last_clean":"2024-09-19T06:47:40.831557+0000","last_became_active":"2024-09-19T06:47:24.845319+0000","last_became_peered":"2024-09-19T06:47:24.845319+0000","last_unstale":"2024-09-19T06:47:40.831557+0000","last_undegraded":"2024-09-19T06:47:40.831557+0000","last_fullsized":"2024-09-19T06:47:40.831557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:00.820006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830650+0000","last_change":"2024-09-19T06:47:24.846457+0000","last_active":"2024-09-19T06:47:40.830650+0000","last_peered":"2024-09-19T06:47:40.830650+0000","last_clean":"2024-09-19T06:47:40.830650+0000","last_became_active":"2024-09-19T06:47:24.846130+0000","last_became_peered":"2024-09-19T06:47:24.846130+0000","last_unstale":"2024-09-19T06:47:40.830650+0000","last_undegraded":"2024-09-19T06:47:40.830650+0000","last_fullsized":"2024-09-19T06:47:40.830650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:42.179145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839379+0000","last_change":"2024-09-19T06:47:24.843141+0000","last_active":"2024-09-19T06:47:40.839379+0000","last_peered":"2024-09-19T06:47:40.839379+0000","last_clean":"2024-09-19T06:47:40.839379+0000","last_became_active":"2024-09-19T06:47:24.842810+0000","last_became_peered":"2024-09-19T06:47:24.842810+0000","last_unstale":"2024-09-19T06:47:40.839379+0000","last_undegraded":"2024-09-19T06:47:40.839379+0000","last_fullsized":"2024-09-19T06:47:40.839379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:43.440453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829981+0000","last_change":"2024-09-19T06:47:24.846171+0000","last_active":"2024-09-19T06:47:40.829981+0000","last_peered":"2024-09-19T06:47:40.829981+0000","last_clean":"2024-09-19T06:47:40.829981+0000","last_became_active":"2024-09-19T06:47:24.845799+0000","last_became_peered":"2024-09-19T06:47:24.845799+0000","last_unstale":"2024-09-19T06:47:40.829981+0000","last_undegraded":"2024-09-19T06:47:40.829981+0000","last_fullsized":"2024-09-19T06:47:40.829981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:23.103061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830061+0000","last_change":"2024-09-19T06:47:24.848723+0000","last_active":"2024-09-19T06:47:40.830061+0000","last_peered":"2024-09-19T06:47:40.830061+0000","last_clean":"2024-09-19T06:47:40.830061+0000","last_became_active":"2024-09-19T06:47:24.848367+0000","last_became_peered":"2024-09-19T06:47:24.848367+0000","last_unstale":"2024-09-19T06:47:40.830061+0000","last_undegraded":"2024-09-19T06:47:40.830061+0000","last_fullsized":"2024-09-19T06:47:40.830061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:15.985293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831354+0000","last_change":"2024-09-19T06:47:24.841379+0000","last_active":"2024-09-19T06:47:40.831354+0000","last_peered":"2024-09-19T06:47:40.831354+0000","last_clean":"2024-09-19T06:47:40.831354+0000","last_became_active":"2024-09-19T06:47:24.841041+0000","last_became_peered":"2024-09-19T06:47:24.841041+0000","last_unstale":"2024-09-19T06:47:40.831354+0000","last_undegraded":"2024-09-19T06:47:40.831354+0000","last_fullsized":"2024-09-19T06:47:40.831354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:11.799022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.832906+0000","last_change":"2024-09-19T06:47:24.849508+0000","last_active":"2024-09-19T06:47:40.832906+0000","last_peered":"2024-09-19T06:47:40.832906+0000","last_clean":"2024-09-19T06:47:40.832906+0000","last_became_active":"2024-09-19T06:47:24.849196+0000","last_became_peered":"2024-09-19T06:47:24.849196+0000","last_unstale":"2024-09-19T06:47:40.832906+0000","last_undegraded":"2024-09-19T06:47:40.832906+0000","last_fullsized":"2024-09-19T06:47:40.832906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:35.079720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:43.477887+0000","last_change":"2024-09-19T06:47:22.829848+0000","last_active":"2024-09-19T06:47:43.477887+0000","last_peered":"2024-09-19T06:47:43.477887+0000","last_clean":"2024-09-19T06:47:43.477887+0000","last_became_active":"2024-09-19T06:47:22.829511+0000","last_became_peered":"2024-09-19T06:47:22.829511+0000","last_unstale":"2024-09-19T06:47:43.477887+0000","last_undegraded":"2024-09-19T06:47:43.477887+0000","last_fullsized":"2024-09-19T06:47:43.477887+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:32.665252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834585+0000","last_change":"2024-09-19T06:47:24.843245+0000","last_active":"2024-09-19T06:47:40.834585+0000","last_peered":"2024-09-19T06:47:40.834585+0000","last_clean":"2024-09-19T06:47:40.834585+0000","last_became_active":"2024-09-19T06:47:24.842885+0000","last_became_peered":"2024-09-19T06:47:24.842885+0000","last_unstale":"2024-09-19T06:47:40.834585+0000","last_undegraded":"2024-09-19T06:47:40.834585+0000","last_fullsized":"2024-09-19T06:47:40.834585+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:26.229003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172292+0000","last_change":"2024-09-19T06:47:22.831456+0000","last_active":"2024-09-19T06:47:41.172292+0000","last_peered":"2024-09-19T06:47:41.172292+0000","last_clean":"2024-09-19T06:47:41.172292+0000","last_became_active":"2024-09-19T06:47:22.831132+0000","last_became_peered":"2024-09-19T06:47:22.831132+0000","last_unstale":"2024-09-19T06:47:41.172292+0000","last_undegraded":"2024-09-19T06:47:41.172292+0000","last_fullsized":"2024-09-19T06:47:41.172292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:49.867713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830418+0000","last_change":"2024-09-19T06:47:22.827742+0000","last_active":"2024-09-19T06:47:40.830418+0000","last_peered":"2024-09-19T06:47:40.830418+0000","last_clean":"2024-09-19T06:47:40.830418+0000","last_became_active":"2024-09-19T06:47:22.827371+0000","last_became_peered":"2024-09-19T06:47:22.827371+0000","last_unstale":"2024-09-19T06:47:40.830418+0000","last_undegraded":"2024-09-19T06:47:40.830418+0000","last_fullsized":"2024-09-19T06:47:40.830418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:35.153129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833435+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:47:40.833435+0000","last_peered":"2024-09-19T06:47:40.833435+0000","last_clean":"2024-09-19T06:47:40.833435+0000","last_became_active":"2024-09-19T06:47:24.841693+0000","last_became_peered":"2024-09-19T06:47:24.841693+0000","last_unstale":"2024-09-19T06:47:40.833435+0000","last_undegraded":"2024-09-19T06:47:40.833435+0000","last_fullsized":"2024-09-19T06:47:40.833435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:27.828296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834254+0000","last_change":"2024-09-19T06:47:22.832531+0000","last_active":"2024-09-19T06:47:40.834254+0000","last_peered":"2024-09-19T06:47:40.834254+0000","last_clean":"2024-09-19T06:47:40.834254+0000","last_became_active":"2024-09-19T06:47:22.832213+0000","last_became_peered":"2024-09-19T06:47:22.832213+0000","last_unstale":"2024-09-19T06:47:40.834254+0000","last_undegraded":"2024-09-19T06:47:40.834254+0000","last_fullsized":"2024-09-19T06:47:40.834254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:46:03.168406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833353+0000","last_change":"2024-09-19T06:47:24.837690+0000","last_active":"2024-09-19T06:47:40.833353+0000","last_peered":"2024-09-19T06:47:40.833353+0000","last_clean":"2024-09-19T06:47:40.833353+0000","last_became_active":"2024-09-19T06:47:24.837389+0000","last_became_peered":"2024-09-19T06:47:24.837389+0000","last_unstale":"2024-09-19T06:47:40.833353+0000","last_undegraded":"2024-09-19T06:47:40.833353+0000","last_fullsized":"2024-09-19T06:47:40.833353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:20.653832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:44.478429+0000","last_change":"2024-09-19T06:47:22.833808+0000","last_active":"2024-09-19T06:47:44.478429+0000","last_peered":"2024-09-19T06:47:44.478429+0000","last_clean":"2024-09-19T06:47:44.478429+0000","last_became_active":"2024-09-19T06:47:22.832468+0000","last_became_peered":"2024-09-19T06:47:22.832468+0000","last_unstale":"2024-09-19T06:47:44.478429+0000","last_undegraded":"2024-09-19T06:47:44.478429+0000","last_fullsized":"2024-09-19T06:47:44.478429+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:39:31.131321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172236+0000","last_change":"2024-09-19T06:47:24.840928+0000","last_active":"2024-09-19T06:47:41.172236+0000","last_peered":"2024-09-19T06:47:41.172236+0000","last_clean":"2024-09-19T06:47:41.172236+0000","last_became_active":"2024-09-19T06:47:24.839928+0000","last_became_peered":"2024-09-19T06:47:24.839928+0000","last_unstale":"2024-09-19T06:47:41.172236+0000","last_undegraded":"2024-09-19T06:47:41.172236+0000","last_fullsized":"2024-09-19T06:47:41.172236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:29.486003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830932+0000","last_change":"2024-09-19T06:47:22.833310+0000","last_active":"2024-09-19T06:47:40.830932+0000","last_peered":"2024-09-19T06:47:40.830932+0000","last_clean":"2024-09-19T06:47:40.830932+0000","last_became_active":"2024-09-19T06:47:22.832976+0000","last_became_peered":"2024-09-19T06:47:22.832976+0000","last_unstale":"2024-09-19T06:47:40.830932+0000","last_undegraded":"2024-09-19T06:47:40.830932+0000","last_fullsized":"2024-09-19T06:47:40.830932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:47.604853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.840138+0000","last_change":"2024-09-19T06:47:24.848687+0000","last_active":"2024-09-19T06:47:40.840138+0000","last_peered":"2024-09-19T06:47:40.840138+0000","last_clean":"2024-09-19T06:47:40.840138+0000","last_became_active":"2024-09-19T06:47:24.848352+0000","last_became_peered":"2024-09-19T06:47:24.848352+0000","last_unstale":"2024-09-19T06:47:40.840138+0000","last_undegraded":"2024-09-19T06:47:40.840138+0000","last_fullsized":"2024-09-19T06:47:40.840138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:54.459206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.835734+0000","last_change":"2024-09-19T06:47:22.851479+0000","last_active":"2024-09-19T06:47:40.835734+0000","last_peered":"2024-09-19T06:47:40.835734+0000","last_clean":"2024-09-19T06:47:40.835734+0000","last_became_active":"2024-09-19T06:47:22.851125+0000","last_became_peered":"2024-09-19T06:47:22.851125+0000","last_unstale":"2024-09-19T06:47:40.835734+0000","last_undegraded":"2024-09-19T06:47:40.835734+0000","last_fullsized":"2024-09-19T06:47:40.835734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:42.954034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172823+0000","last_change":"2024-09-19T06:47:24.851007+0000","last_active":"2024-09-19T06:47:41.172823+0000","last_peered":"2024-09-19T06:47:41.172823+0000","last_clean":"2024-09-19T06:47:41.172823+0000","last_became_active":"2024-09-19T06:47:24.850465+0000","last_became_peered":"2024-09-19T06:47:24.850465+0000","last_unstale":"2024-09-19T06:47:41.172823+0000","last_undegraded":"2024-09-19T06:47:41.172823+0000","last_fullsized":"2024-09-19T06:47:41.172823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:10.824672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'24","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:44.750340+0000","last_change":"2024-09-19T06:47:22.834893+0000","last_active":"2024-09-19T06:47:44.750340+0000","last_peered":"2024-09-19T06:47:44.750340+0000","last_clean":"2024-09-19T06:47:44.750340+0000","last_became_active":"2024-09-19T06:47:22.834521+0000","last_became_peered":"2024-09-19T06:47:22.834521+0000","last_unstale":"2024-09-19T06:47:44.750340+0000","last_undegraded":"2024-09-19T06:47:44.750340+0000","last_fullsized":"2024-09-19T06:47:44.750340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:29.742284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":23,"num_read_kb":51,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833599+0000","last_change":"2024-09-19T06:47:24.848191+0000","last_active":"2024-09-19T06:47:40.833599+0000","last_peered":"2024-09-19T06:47:40.833599+0000","last_clean":"2024-09-19T06:47:40.833599+0000","last_became_active":"2024-09-19T06:47:24.847889+0000","last_became_peered":"2024-09-19T06:47:24.847889+0000","last_unstale":"2024-09-19T06:47:40.833599+0000","last_undegraded":"2024-09-19T06:47:40.833599+0000","last_fullsized":"2024-09-19T06:47:40.833599+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:32.574360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830250+0000","last_change":"2024-09-19T06:47:22.832005+0000","last_active":"2024-09-19T06:47:40.830250+0000","last_peered":"2024-09-19T06:47:40.830250+0000","last_clean":"2024-09-19T06:47:40.830250+0000","last_became_active":"2024-09-19T06:47:22.831342+0000","last_became_peered":"2024-09-19T06:47:22.831342+0000","last_unstale":"2024-09-19T06:47:40.830250+0000","last_undegraded":"2024-09-19T06:47:40.830250+0000","last_fullsized":"2024-09-19T06:47:40.830250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:53.444748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830453+0000","last_change":"2024-09-19T06:47:24.843328+0000","last_active":"2024-09-19T06:47:40.830453+0000","last_peered":"2024-09-19T06:47:40.830453+0000","last_clean":"2024-09-19T06:47:40.830453+0000","last_became_active":"2024-09-19T06:47:24.842344+0000","last_became_peered":"2024-09-19T06:47:24.842344+0000","last_unstale":"2024-09-19T06:47:40.830453+0000","last_undegraded":"2024-09-19T06:47:40.830453+0000","last_fullsized":"2024-09-19T06:47:40.830453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:11.247819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.844801+0000","last_change":"2024-09-19T06:47:40.844086+0000","last_active":"2024-09-19T06:47:40.844801+0000","last_peered":"2024-09-19T06:47:40.844801+0000","last_clean":"2024-09-19T06:47:40.844801+0000","last_became_active":"2024-09-19T06:47:40.835015+0000","last_became_peered":"2024-09-19T06:47:40.835015+0000","last_unstale":"2024-09-19T06:47:40.844801+0000","last_undegraded":"2024-09-19T06:47:40.844801+0000","last_fullsized":"2024-09-19T06:47:40.844801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:09.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.836466+0000","last_change":"2024-09-19T06:47:40.836466+0000","last_active":"2024-09-19T06:47:40.836466+0000","last_peered":"2024-09-19T06:47:40.836466+0000","last_clean":"2024-09-19T06:47:40.836466+0000","last_became_active":"2024-09-19T06:47:40.836237+0000","last_became_peered":"2024-09-19T06:47:40.836237+0000","last_unstale":"2024-09-19T06:47:40.836466+0000","last_undegraded":"2024-09-19T06:47:40.836466+0000","last_fullsized":"2024-09-19T06:47:40.836466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:00.094685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:45.481085+0000","last_change":"2024-09-19T06:47:22.830359+0000","last_active":"2024-09-19T06:47:45.481085+0000","last_peered":"2024-09-19T06:47:45.481085+0000","last_clean":"2024-09-19T06:47:45.481085+0000","last_became_active":"2024-09-19T06:47:22.829960+0000","last_became_peered":"2024-09-19T06:47:22.829960+0000","last_unstale":"2024-09-19T06:47:45.481085+0000","last_undegraded":"2024-09-19T06:47:45.481085+0000","last_fullsized":"2024-09-19T06:47:45.481085+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:55.998243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830424+0000","last_change":"2024-09-19T06:47:24.836668+0000","last_active":"2024-09-19T06:47:40.830424+0000","last_peered":"2024-09-19T06:47:40.830424+0000","last_clean":"2024-09-19T06:47:40.830424+0000","last_became_active":"2024-09-19T06:47:24.836393+0000","last_became_peered":"2024-09-19T06:47:24.836393+0000","last_unstale":"2024-09-19T06:47:40.830424+0000","last_undegraded":"2024-09-19T06:47:40.830424+0000","last_fullsized":"2024-09-19T06:47:40.830424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:26.248948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.858323+0000","last_change":"2024-09-19T06:47:40.858323+0000","last_active":"2024-09-19T06:47:40.858323+0000","last_peered":"2024-09-19T06:47:40.858323+0000","last_clean":"2024-09-19T06:47:40.858323+0000","last_became_active":"2024-09-19T06:47:40.840802+0000","last_became_peered":"2024-09-19T06:47:40.840802+0000","last_unstale":"2024-09-19T06:47:40.858323+0000","last_undegraded":"2024-09-19T06:47:40.858323+0000","last_fullsized":"2024-09-19T06:47:40.858323+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:14.666916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834268+0000","last_change":"2024-09-19T06:47:24.843681+0000","last_active":"2024-09-19T06:47:40.834268+0000","last_peered":"2024-09-19T06:47:40.834268+0000","last_clean":"2024-09-19T06:47:40.834268+0000","last_became_active":"2024-09-19T06:47:24.841228+0000","last_became_peered":"2024-09-19T06:47:24.841228+0000","last_unstale":"2024-09-19T06:47:40.834268+0000","last_undegraded":"2024-09-19T06:47:40.834268+0000","last_fullsized":"2024-09-19T06:47:40.834268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:56.418962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833647+0000","last_change":"2024-09-19T06:47:22.833893+0000","last_active":"2024-09-19T06:47:40.833647+0000","last_peered":"2024-09-19T06:47:40.833647+0000","last_clean":"2024-09-19T06:47:40.833647+0000","last_became_active":"2024-09-19T06:47:22.833217+0000","last_became_peered":"2024-09-19T06:47:22.833217+0000","last_unstale":"2024-09-19T06:47:40.833647+0000","last_undegraded":"2024-09-19T06:47:40.833647+0000","last_fullsized":"2024-09-19T06:47:40.833647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:19.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831588+0000","last_change":"2024-09-19T06:47:24.848399+0000","last_active":"2024-09-19T06:47:40.831588+0000","last_peered":"2024-09-19T06:47:40.831588+0000","last_clean":"2024-09-19T06:47:40.831588+0000","last_became_active":"2024-09-19T06:47:24.848081+0000","last_became_peered":"2024-09-19T06:47:24.848081+0000","last_unstale":"2024-09-19T06:47:40.831588+0000","last_undegraded":"2024-09-19T06:47:40.831588+0000","last_fullsized":"2024-09-19T06:47:40.831588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:44.292939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.860955+0000","last_change":"2024-09-19T06:47:40.860955+0000","last_active":"2024-09-19T06:47:40.860955+0000","last_peered":"2024-09-19T06:47:40.860955+0000","last_clean":"2024-09-19T06:47:40.860955+0000","last_became_active":"2024-09-19T06:47:40.835126+0000","last_became_peered":"2024-09-19T06:47:40.835126+0000","last_unstale":"2024-09-19T06:47:40.860955+0000","last_undegraded":"2024-09-19T06:47:40.860955+0000","last_fullsized":"2024-09-19T06:47:40.860955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:02:26.763229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830803+0000","last_change":"2024-09-19T06:47:24.849680+0000","last_active":"2024-09-19T06:47:40.830803+0000","last_peered":"2024-09-19T06:47:40.830803+0000","last_clean":"2024-09-19T06:47:40.830803+0000","last_became_active":"2024-09-19T06:47:24.849323+0000","last_became_peered":"2024-09-19T06:47:24.849323+0000","last_unstale":"2024-09-19T06:47:40.830803+0000","last_undegraded":"2024-09-19T06:47:40.830803+0000","last_fullsized":"2024-09-19T06:47:40.830803+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:59.763503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.835875+0000","last_change":"2024-09-19T06:47:22.831689+0000","last_active":"2024-09-19T06:47:40.835875+0000","last_peered":"2024-09-19T06:47:40.835875+0000","last_clean":"2024-09-19T06:47:40.835875+0000","last_became_active":"2024-09-19T06:47:22.829283+0000","last_became_peered":"2024-09-19T06:47:22.829283+0000","last_unstale":"2024-09-19T06:47:40.835875+0000","last_undegraded":"2024-09-19T06:47:40.835875+0000","last_fullsized":"2024-09-19T06:47:40.835875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:37.109542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830659+0000","last_change":"2024-09-19T06:47:24.842793+0000","last_active":"2024-09-19T06:47:40.830659+0000","last_peered":"2024-09-19T06:47:40.830659+0000","last_clean":"2024-09-19T06:47:40.830659+0000","last_became_active":"2024-09-19T06:47:24.842474+0000","last_became_peered":"2024-09-19T06:47:24.842474+0000","last_unstale":"2024-09-19T06:47:40.830659+0000","last_undegraded":"2024-09-19T06:47:40.830659+0000","last_fullsized":"2024-09-19T06:47:40.830659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:23.110736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'6","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:45.480803+0000","last_change":"2024-09-19T06:47:23.047356+0000","last_active":"2024-09-19T06:47:45.480803+0000","last_peered":"2024-09-19T06:47:45.480803+0000","last_clean":"2024-09-19T06:47:45.480803+0000","last_became_active":"2024-09-19T06:47:23.046941+0000","last_became_peered":"2024-09-19T06:47:23.046941+0000","last_unstale":"2024-09-19T06:47:45.480803+0000","last_undegraded":"2024-09-19T06:47:45.480803+0000","last_fullsized":"2024-09-19T06:47:45.480803+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:43.490952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830805+0000","last_change":"2024-09-19T06:47:24.851025+0000","last_active":"2024-09-19T06:47:40.830805+0000","last_peered":"2024-09-19T06:47:40.830805+0000","last_clean":"2024-09-19T06:47:40.830805+0000","last_became_active":"2024-09-19T06:47:24.850681+0000","last_became_peered":"2024-09-19T06:47:24.850681+0000","last_unstale":"2024-09-19T06:47:40.830805+0000","last_undegraded":"2024-09-19T06:47:40.830805+0000","last_fullsized":"2024-09-19T06:47:40.830805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:46.884747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":345,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830558+0000","last_change":"2024-09-19T06:46:13.927379+0000","last_active":"2024-09-19T06:47:40.830558+0000","last_peered":"2024-09-19T06:47:40.830558+0000","last_clean":"2024-09-19T06:47:40.830558+0000","last_became_active":"2024-09-19T06:46:13.926926+0000","last_became_peered":"2024-09-19T06:46:13.926926+0000","last_unstale":"2024-09-19T06:47:40.830558+0000","last_undegraded":"2024-09-19T06:47:40.830558+0000","last_fullsized":"2024-09-19T06:47:40.830558+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_clean_scrub_stamp":"2024-09-19T06:42:52.051335+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:29.542128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829278+0000","last_change":"2024-09-19T06:47:22.830387+0000","last_active":"2024-09-19T06:47:40.829278+0000","last_peered":"2024-09-19T06:47:40.829278+0000","last_clean":"2024-09-19T06:47:40.829278+0000","last_became_active":"2024-09-19T06:47:22.829970+0000","last_became_peered":"2024-09-19T06:47:22.829970+0000","last_unstale":"2024-09-19T06:47:40.829278+0000","last_undegraded":"2024-09-19T06:47:40.829278+0000","last_fullsized":"2024-09-19T06:47:40.829278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:51.385456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834053+0000","last_change":"2024-09-19T06:47:24.843752+0000","last_active":"2024-09-19T06:47:40.834053+0000","last_peered":"2024-09-19T06:47:40.834053+0000","last_clean":"2024-09-19T06:47:40.834053+0000","last_became_active":"2024-09-19T06:47:24.842631+0000","last_became_peered":"2024-09-19T06:47:24.842631+0000","last_unstale":"2024-09-19T06:47:40.834053+0000","last_undegraded":"2024-09-19T06:47:40.834053+0000","last_fullsized":"2024-09-19T06:47:40.834053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:19.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830650+0000","last_change":"2024-09-19T06:47:22.832647+0000","last_active":"2024-09-19T06:47:40.830650+0000","last_peered":"2024-09-19T06:47:40.830650+0000","last_clean":"2024-09-19T06:47:40.830650+0000","last_became_active":"2024-09-19T06:47:22.831774+0000","last_became_peered":"2024-09-19T06:47:22.831774+0000","last_unstale":"2024-09-19T06:47:40.830650+0000","last_undegraded":"2024-09-19T06:47:40.830650+0000","last_fullsized":"2024-09-19T06:47:40.830650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:57.824610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830409+0000","last_change":"2024-09-19T06:47:24.844301+0000","last_active":"2024-09-19T06:47:40.830409+0000","last_peered":"2024-09-19T06:47:40.830409+0000","last_clean":"2024-09-19T06:47:40.830409+0000","last_became_active":"2024-09-19T06:47:24.839748+0000","last_became_peered":"2024-09-19T06:47:24.839748+0000","last_unstale":"2024-09-19T06:47:40.830409+0000","last_undegraded":"2024-09-19T06:47:40.830409+0000","last_fullsized":"2024-09-19T06:47:40.830409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:46.407693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:44.478973+0000","last_change":"2024-09-19T06:47:22.851008+0000","last_active":"2024-09-19T06:47:44.478973+0000","last_peered":"2024-09-19T06:47:44.478973+0000","last_clean":"2024-09-19T06:47:44.478973+0000","last_became_active":"2024-09-19T06:47:22.850601+0000","last_became_peered":"2024-09-19T06:47:22.850601+0000","last_unstale":"2024-09-19T06:47:44.478973+0000","last_undegraded":"2024-09-19T06:47:44.478973+0000","last_fullsized":"2024-09-19T06:47:44.478973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:04.610277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834177+0000","last_change":"2024-09-19T06:47:24.839684+0000","last_active":"2024-09-19T06:47:40.834177+0000","last_peered":"2024-09-19T06:47:40.834177+0000","last_clean":"2024-09-19T06:47:40.834177+0000","last_became_active":"2024-09-19T06:47:24.839126+0000","last_became_peered":"2024-09-19T06:47:24.839126+0000","last_unstale":"2024-09-19T06:47:40.834177+0000","last_undegraded":"2024-09-19T06:47:40.834177+0000","last_fullsized":"2024-09-19T06:47:40.834177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:09.458535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:44.179598+0000","last_change":"2024-09-19T06:47:22.831819+0000","last_active":"2024-09-19T06:47:44.179598+0000","last_peered":"2024-09-19T06:47:44.179598+0000","last_clean":"2024-09-19T06:47:44.179598+0000","last_became_active":"2024-09-19T06:47:22.831332+0000","last_became_peered":"2024-09-19T06:47:22.831332+0000","last_unstale":"2024-09-19T06:47:44.179598+0000","last_undegraded":"2024-09-19T06:47:44.179598+0000","last_fullsized":"2024-09-19T06:47:44.179598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:35.032949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.171750+0000","last_change":"2024-09-19T06:47:24.846738+0000","last_active":"2024-09-19T06:47:41.171750+0000","last_peered":"2024-09-19T06:47:41.171750+0000","last_clean":"2024-09-19T06:47:41.171750+0000","last_became_active":"2024-09-19T06:47:24.846389+0000","last_became_peered":"2024-09-19T06:47:24.846389+0000","last_unstale":"2024-09-19T06:47:41.171750+0000","last_undegraded":"2024-09-19T06:47:41.171750+0000","last_fullsized":"2024-09-19T06:47:41.171750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:19.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833696+0000","last_change":"2024-09-19T06:47:22.851065+0000","last_active":"2024-09-19T06:47:40.833696+0000","last_peered":"2024-09-19T06:47:40.833696+0000","last_clean":"2024-09-19T06:47:40.833696+0000","last_became_active":"2024-09-19T06:47:22.850647+0000","last_became_peered":"2024-09-19T06:47:22.850647+0000","last_unstale":"2024-09-19T06:47:40.833696+0000","last_undegraded":"2024-09-19T06:47:40.833696+0000","last_fullsized":"2024-09-19T06:47:40.833696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:41.993501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172204+0000","last_change":"2024-09-19T06:47:24.838905+0000","last_active":"2024-09-19T06:47:41.172204+0000","last_peered":"2024-09-19T06:47:41.172204+0000","last_clean":"2024-09-19T06:47:41.172204+0000","last_became_active":"2024-09-19T06:47:24.838443+0000","last_became_peered":"2024-09-19T06:47:24.838443+0000","last_unstale":"2024-09-19T06:47:41.172204+0000","last_undegraded":"2024-09-19T06:47:41.172204+0000","last_fullsized":"2024-09-19T06:47:41.172204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:12.486137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833766+0000","last_change":"2024-09-19T06:47:22.851431+0000","last_active":"2024-09-19T06:47:40.833766+0000","last_peered":"2024-09-19T06:47:40.833766+0000","last_clean":"2024-09-19T06:47:40.833766+0000","last_became_active":"2024-09-19T06:47:22.851092+0000","last_became_peered":"2024-09-19T06:47:22.851092+0000","last_unstale":"2024-09-19T06:47:40.833766+0000","last_undegraded":"2024-09-19T06:47:40.833766+0000","last_fullsized":"2024-09-19T06:47:40.833766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:26.183803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830789+0000","last_change":"2024-09-19T06:47:24.839046+0000","last_active":"2024-09-19T06:47:40.830789+0000","last_peered":"2024-09-19T06:47:40.830789+0000","last_clean":"2024-09-19T06:47:40.830789+0000","last_became_active":"2024-09-19T06:47:24.838680+0000","last_became_peered":"2024-09-19T06:47:24.838680+0000","last_unstale":"2024-09-19T06:47:40.830789+0000","last_undegraded":"2024-09-19T06:47:40.830789+0000","last_fullsized":"2024-09-19T06:47:40.830789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:45.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:43.845607+0000","last_change":"2024-09-19T06:47:40.865241+0000","last_active":"2024-09-19T06:47:43.845607+0000","last_peered":"2024-09-19T06:47:43.845607+0000","last_clean":"2024-09-19T06:47:43.845607+0000","last_became_active":"2024-09-19T06:47:40.837765+0000","last_became_peered":"2024-09-19T06:47:40.837765+0000","last_unstale":"2024-09-19T06:47:43.845607+0000","last_undegraded":"2024-09-19T06:47:43.845607+0000","last_fullsized":"2024-09-19T06:47:43.845607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834162+0000","last_change":"2024-09-19T06:47:24.848599+0000","last_active":"2024-09-19T06:47:40.834162+0000","last_peered":"2024-09-19T06:47:40.834162+0000","last_clean":"2024-09-19T06:47:40.834162+0000","last_became_active":"2024-09-19T06:47:24.848305+0000","last_became_peered":"2024-09-19T06:47:24.848305+0000","last_unstale":"2024-09-19T06:47:40.834162+0000","last_undegraded":"2024-09-19T06:47:40.834162+0000","last_fullsized":"2024-09-19T06:47:40.834162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:31.496188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829462+0000","last_change":"2024-09-19T06:47:23.047484+0000","last_active":"2024-09-19T06:47:40.829462+0000","last_peered":"2024-09-19T06:47:40.829462+0000","last_clean":"2024-09-19T06:47:40.829462+0000","last_became_active":"2024-09-19T06:47:23.047058+0000","last_became_peered":"2024-09-19T06:47:23.047058+0000","last_unstale":"2024-09-19T06:47:40.829462+0000","last_undegraded":"2024-09-19T06:47:40.829462+0000","last_fullsized":"2024-09-19T06:47:40.829462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:34.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839396+0000","last_change":"2024-09-19T06:47:24.843498+0000","last_active":"2024-09-19T06:47:40.839396+0000","last_peered":"2024-09-19T06:47:40.839396+0000","last_clean":"2024-09-19T06:47:40.839396+0000","last_became_active":"2024-09-19T06:47:24.843204+0000","last_became_peered":"2024-09-19T06:47:24.843204+0000","last_unstale":"2024-09-19T06:47:40.839396+0000","last_undegraded":"2024-09-19T06:47:40.839396+0000","last_fullsized":"2024-09-19T06:47:40.839396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:25.585023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172992+0000","last_change":"2024-09-19T06:47:22.851519+0000","last_active":"2024-09-19T06:47:41.172992+0000","last_peered":"2024-09-19T06:47:41.172992+0000","last_clean":"2024-09-19T06:47:41.172992+0000","last_became_active":"2024-09-19T06:47:22.851163+0000","last_became_peered":"2024-09-19T06:47:22.851163+0000","last_unstale":"2024-09-19T06:47:41.172992+0000","last_undegraded":"2024-09-19T06:47:41.172992+0000","last_fullsized":"2024-09-19T06:47:41.172992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:04.502795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.832719+0000","last_change":"2024-09-19T06:47:24.845239+0000","last_active":"2024-09-19T06:47:40.832719+0000","last_peered":"2024-09-19T06:47:40.832719+0000","last_clean":"2024-09-19T06:47:40.832719+0000","last_became_active":"2024-09-19T06:47:24.844934+0000","last_became_peered":"2024-09-19T06:47:24.844934+0000","last_unstale":"2024-09-19T06:47:40.832719+0000","last_undegraded":"2024-09-19T06:47:40.832719+0000","last_fullsized":"2024-09-19T06:47:40.832719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:43.610604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834147+0000","last_change":"2024-09-19T06:47:22.830232+0000","last_active":"2024-09-19T06:47:40.834147+0000","last_peered":"2024-09-19T06:47:40.834147+0000","last_clean":"2024-09-19T06:47:40.834147+0000","last_became_active":"2024-09-19T06:47:22.829901+0000","last_became_peered":"2024-09-19T06:47:22.829901+0000","last_unstale":"2024-09-19T06:47:40.834147+0000","last_undegraded":"2024-09-19T06:47:40.834147+0000","last_fullsized":"2024-09-19T06:47:40.834147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:12.251427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831384+0000","last_change":"2024-09-19T06:47:24.839304+0000","last_active":"2024-09-19T06:47:40.831384+0000","last_peered":"2024-09-19T06:47:40.831384+0000","last_clean":"2024-09-19T06:47:40.831384+0000","last_became_active":"2024-09-19T06:47:24.838986+0000","last_became_peered":"2024-09-19T06:47:24.838986+0000","last_unstale":"2024-09-19T06:47:40.831384+0000","last_undegraded":"2024-09-19T06:47:40.831384+0000","last_fullsized":"2024-09-19T06:47:40.831384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:39.377886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830021+0000","last_change":"2024-09-19T06:47:22.828553+0000","last_active":"2024-09-19T06:47:40.830021+0000","last_peered":"2024-09-19T06:47:40.830021+0000","last_clean":"2024-09-19T06:47:40.830021+0000","last_became_active":"2024-09-19T06:47:22.828213+0000","last_became_peered":"2024-09-19T06:47:22.828213+0000","last_unstale":"2024-09-19T06:47:40.830021+0000","last_undegraded":"2024-09-19T06:47:40.830021+0000","last_fullsized":"2024-09-19T06:47:40.830021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:31.649761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833576+0000","last_change":"2024-09-19T06:47:24.840747+0000","last_active":"2024-09-19T06:47:40.833576+0000","last_peered":"2024-09-19T06:47:40.833576+0000","last_clean":"2024-09-19T06:47:40.833576+0000","last_became_active":"2024-09-19T06:47:24.840451+0000","last_became_peered":"2024-09-19T06:47:24.840451+0000","last_unstale":"2024-09-19T06:47:40.833576+0000","last_undegraded":"2024-09-19T06:47:40.833576+0000","last_fullsized":"2024-09-19T06:47:40.833576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:41.588751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833228+0000","last_change":"2024-09-19T06:47:22.831697+0000","last_active":"2024-09-19T06:47:40.833228+0000","last_peered":"2024-09-19T06:47:40.833228+0000","last_clean":"2024-09-19T06:47:40.833228+0000","last_became_active":"2024-09-19T06:47:22.831178+0000","last_became_peered":"2024-09-19T06:47:22.831178+0000","last_unstale":"2024-09-19T06:47:40.833228+0000","last_undegraded":"2024-09-19T06:47:40.833228+0000","last_fullsized":"2024-09-19T06:47:40.833228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:30.975782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830575+0000","last_change":"2024-09-19T06:47:24.845097+0000","last_active":"2024-09-19T06:47:40.830575+0000","last_peered":"2024-09-19T06:47:40.830575+0000","last_clean":"2024-09-19T06:47:40.830575+0000","last_became_active":"2024-09-19T06:47:24.844801+0000","last_became_peered":"2024-09-19T06:47:24.844801+0000","last_unstale":"2024-09-19T06:47:40.830575+0000","last_undegraded":"2024-09-19T06:47:40.830575+0000","last_fullsized":"2024-09-19T06:47:40.830575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:15.264118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.835620+0000","last_change":"2024-09-19T06:47:24.851372+0000","last_active":"2024-09-19T06:47:40.835620+0000","last_peered":"2024-09-19T06:47:40.835620+0000","last_clean":"2024-09-19T06:47:40.835620+0000","last_became_active":"2024-09-19T06:47:24.851044+0000","last_became_peered":"2024-09-19T06:47:24.851044+0000","last_unstale":"2024-09-19T06:47:40.835620+0000","last_undegraded":"2024-09-19T06:47:40.835620+0000","last_fullsized":"2024-09-19T06:47:40.835620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:55.658214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833103+0000","last_change":"2024-09-19T06:47:22.833694+0000","last_active":"2024-09-19T06:47:40.833103+0000","last_peered":"2024-09-19T06:47:40.833103+0000","last_clean":"2024-09-19T06:47:40.833103+0000","last_became_active":"2024-09-19T06:47:22.833215+0000","last_became_peered":"2024-09-19T06:47:22.833215+0000","last_unstale":"2024-09-19T06:47:40.833103+0000","last_undegraded":"2024-09-19T06:47:40.833103+0000","last_fullsized":"2024-09-19T06:47:40.833103+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:32.171522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172973+0000","last_change":"2024-09-19T06:47:24.845737+0000","last_active":"2024-09-19T06:47:41.172973+0000","last_peered":"2024-09-19T06:47:41.172973+0000","last_clean":"2024-09-19T06:47:41.172973+0000","last_became_active":"2024-09-19T06:47:24.845242+0000","last_became_peered":"2024-09-19T06:47:24.845242+0000","last_unstale":"2024-09-19T06:47:41.172973+0000","last_undegraded":"2024-09-19T06:47:41.172973+0000","last_fullsized":"2024-09-19T06:47:41.172973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:51.554946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833601+0000","last_change":"2024-09-19T06:47:22.829869+0000","last_active":"2024-09-19T06:47:40.833601+0000","last_peered":"2024-09-19T06:47:40.833601+0000","last_clean":"2024-09-19T06:47:40.833601+0000","last_became_active":"2024-09-19T06:47:22.829522+0000","last_became_peered":"2024-09-19T06:47:22.829522+0000","last_unstale":"2024-09-19T06:47:40.833601+0000","last_undegraded":"2024-09-19T06:47:40.833601+0000","last_fullsized":"2024-09-19T06:47:40.833601+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:26.871434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.840141+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:47:40.840141+0000","last_peered":"2024-09-19T06:47:40.840141+0000","last_clean":"2024-09-19T06:47:40.840141+0000","last_became_active":"2024-09-19T06:47:24.841289+0000","last_became_peered":"2024-09-19T06:47:24.841289+0000","last_unstale":"2024-09-19T06:47:40.840141+0000","last_undegraded":"2024-09-19T06:47:40.840141+0000","last_fullsized":"2024-09-19T06:47:40.840141+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:21:12.611458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834465+0000","last_change":"2024-09-19T06:47:22.852198+0000","last_active":"2024-09-19T06:47:40.834465+0000","last_peered":"2024-09-19T06:47:40.834465+0000","last_clean":"2024-09-19T06:47:40.834465+0000","last_became_active":"2024-09-19T06:47:22.850984+0000","last_became_peered":"2024-09-19T06:47:22.850984+0000","last_unstale":"2024-09-19T06:47:40.834465+0000","last_undegraded":"2024-09-19T06:47:40.834465+0000","last_fullsized":"2024-09-19T06:47:40.834465+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:13.620145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839399+0000","last_change":"2024-09-19T06:47:40.837614+0000","last_active":"2024-09-19T06:47:40.839399+0000","last_peered":"2024-09-19T06:47:40.839399+0000","last_clean":"2024-09-19T06:47:40.839399+0000","last_became_active":"2024-09-19T06:47:40.837259+0000","last_became_peered":"2024-09-19T06:47:40.837259+0000","last_unstale":"2024-09-19T06:47:40.839399+0000","last_undegraded":"2024-09-19T06:47:40.839399+0000","last_fullsized":"2024-09-19T06:47:40.839399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:41.429962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:44.178758+0000","last_change":"2024-09-19T06:47:41.182639+0000","last_active":"2024-09-19T06:47:44.178758+0000","last_peered":"2024-09-19T06:47:44.178758+0000","last_clean":"2024-09-19T06:47:44.178758+0000","last_became_active":"2024-09-19T06:47:41.174305+0000","last_became_peered":"2024-09-19T06:47:41.174305+0000","last_unstale":"2024-09-19T06:47:44.178758+0000","last_undegraded":"2024-09-19T06:47:44.178758+0000","last_fullsized":"2024-09-19T06:47:44.178758+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:10.199502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54010,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":129,"num_read_kb":117,"num_write":229,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":162030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579921,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":439122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1071126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972379,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":438569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234016,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":913408,"data_stored":438275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462948,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712484,"statfs":{"total":95995035648,"available":95961583616,"internally_reserved":0,"allocated":1519616,"data_stored":1031467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626474,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":909312,"data_stored":438286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855405,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1030662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018931,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":438652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215156,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":438111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444094,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":486502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":486673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":440018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":49226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":49055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:47:52.151 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T06:47:52.152 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T06:47:52.152 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T06:47:52.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:51 smithi134 ceph-mon[35356]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:47:52.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:51 smithi174 ceph-mon[35637]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:47:52.400 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:52.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.819+0000 7fad4d406640 1 -- 172.21.15.103:0/1327820871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad481026a0 msgr2=0x7fad48104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:52.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.819+0000 7fad4d406640 1 --2- 172.21.15.103:0/1327820871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad481026a0 0x7fad48104a90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fad38009900 tx=0x7fad3802a410 comp rx=0 tx=0).stop 2024-09-19T06:47:52.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.819+0000 7fad4d406640 1 -- 172.21.15.103:0/1327820871 shutdown_connections 2024-09-19T06:47:52.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.819+0000 7fad4d406640 1 --2- 172.21.15.103:0/1327820871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad48107900 0x7fad48109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:52.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.819+0000 7fad4d406640 1 --2- 172.21.15.103:0/1327820871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad48104fd0 0x7fad481073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:52.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.819+0000 7fad4d406640 1 --2- 172.21.15.103:0/1327820871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad481026a0 0x7fad48104a90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:52.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.819+0000 7fad4d406640 1 -- 172.21.15.103:0/1327820871 >> 172.21.15.103:0/1327820871 conn(0x7fad480fc2a0 msgr2=0x7fad480fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:52.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.819+0000 7fad4d406640 1 -- 172.21.15.103:0/1327820871 shutdown_connections 2024-09-19T06:47:52.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.819+0000 7fad4d406640 1 -- 172.21.15.103:0/1327820871 wait complete. 2024-09-19T06:47:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.820+0000 7fad4d406640 1 Processor -- start 2024-09-19T06:47:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.820+0000 7fad4d406640 1 -- start start 2024-09-19T06:47:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.820+0000 7fad4d406640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad481026a0 0x7fad48198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.820+0000 7fad4d406640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad48104fd0 0x7fad481985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad4d406640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad48107900 0x7fad4819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad4d406640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad4810d990 con 0x7fad481026a0 2024-09-19T06:47:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad4d406640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad4810d810 con 0x7fad48107900 2024-09-19T06:47:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad4d406640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fad4810db10 con 0x7fad48104fd0 2024-09-19T06:47:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad46ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad481026a0 0x7fad48198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad46ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad481026a0 0x7fad48198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47312/0 (socket says 172.21.15.103:47312) 2024-09-19T06:47:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad46ffd640 1 -- 172.21.15.103:0/1680252815 learned_addr learned my addr 172.21.15.103:0/1680252815 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad3e7fc640 1 --2- 172.21.15.103:0/1680252815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad48104fd0 0x7fad481985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad477fe640 1 --2- 172.21.15.103:0/1680252815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad48107900 0x7fad4819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad46ffd640 1 -- 172.21.15.103:0/1680252815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad48104fd0 msgr2=0x7fad481985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad46ffd640 1 --2- 172.21.15.103:0/1680252815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad48104fd0 0x7fad481985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad46ffd640 1 -- 172.21.15.103:0/1680252815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad48107900 msgr2=0x7fad4819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad46ffd640 1 --2- 172.21.15.103:0/1680252815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad48107900 0x7fad4819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad46ffd640 1 -- 172.21.15.103:0/1680252815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4819fca0 con 0x7fad481026a0 2024-09-19T06:47:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad477fe640 1 --2- 172.21.15.103:0/1680252815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad48107900 0x7fad4819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad46ffd640 1 --2- 172.21.15.103:0/1680252815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad481026a0 0x7fad48198070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fad38009230 tx=0x7fad38004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad44ff9640 1 -- 172.21.15.103:0/1680252815 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3803f030 con 0x7fad481026a0 2024-09-19T06:47:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad4d406640 1 -- 172.21.15.103:0/1680252815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4819ff30 con 0x7fad481026a0 2024-09-19T06:47:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad44ff9640 1 -- 172.21.15.103:0/1680252815 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad38003ce0 con 0x7fad481026a0 2024-09-19T06:47:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad44ff9640 1 -- 172.21.15.103:0/1680252815 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad38035070 con 0x7fad481026a0 2024-09-19T06:47:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.821+0000 7fad4d406640 1 -- 172.21.15.103:0/1680252815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad481a03b0 con 0x7fad481026a0 2024-09-19T06:47:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.822+0000 7fad4d406640 1 -- 172.21.15.103:0/1680252815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad480fdd90 con 0x7fad481026a0 2024-09-19T06:47:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.823+0000 7fad44ff9640 1 -- 172.21.15.103:0/1680252815 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fad38042030 con 0x7fad481026a0 2024-09-19T06:47:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.824+0000 7fad44ff9640 1 --2- 172.21.15.103:0/1680252815 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fad340a3300 0x7fad340a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.824+0000 7fad3e7fc640 1 --2- 172.21.15.103:0/1680252815 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fad340a3300 0x7fad340a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.824+0000 7fad44ff9640 1 -- 172.21.15.103:0/1680252815 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fad380f0180 con 0x7fad481026a0 2024-09-19T06:47:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.825+0000 7fad3e7fc640 1 --2- 172.21.15.103:0/1680252815 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fad340a3300 0x7fad340a57c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fad48199590 tx=0x7fad30009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:52.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:52.828+0000 7fad44ff9640 1 -- 172.21.15.103:0/1680252815 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fad380bcd50 con 0x7fad481026a0 2024-09-19T06:47:53.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:53.058+0000 7fad4d406640 1 -- 172.21.15.103:0/1680252815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fad4802da10 con 0x7fad481026a0 2024-09-19T06:47:53.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:53.060+0000 7fad44ff9640 1 -- 172.21.15.103:0/1680252815 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fad380bc650 con 0x7fad481026a0 2024-09-19T06:47:53.062 INFO:teuthology.orchestra.run.smithi103.stdout:[client.0] 2024-09-19T06:47:53.062 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQAZyetmVvOWAxAAqzDqgyT54THCeK4qzriPlQ== 2024-09-19T06:47:53.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:53.062+0000 7fad4d406640 1 -- 172.21.15.103:0/1680252815 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fad340a3300 msgr2=0x7fad340a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:53.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:53.062+0000 7fad4d406640 1 --2- 172.21.15.103:0/1680252815 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fad340a3300 0x7fad340a57c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fad48199590 tx=0x7fad30009110 comp rx=0 tx=0).stop 2024-09-19T06:47:53.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:53.062+0000 7fad4d406640 1 -- 172.21.15.103:0/1680252815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad481026a0 msgr2=0x7fad48198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:53.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:53.062+0000 7fad4d406640 1 --2- 172.21.15.103:0/1680252815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad481026a0 0x7fad48198070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fad38009230 tx=0x7fad38004430 comp rx=0 tx=0).stop 2024-09-19T06:47:53.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:53.062+0000 7fad4d406640 1 -- 172.21.15.103:0/1680252815 shutdown_connections 2024-09-19T06:47:53.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:53.062+0000 7fad4d406640 1 --2- 172.21.15.103:0/1680252815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad48107900 0x7fad4819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:53.062+0000 7fad4d406640 1 --2- 172.21.15.103:0/1680252815 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fad340a3300 0x7fad340a57c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:53.062+0000 7fad4d406640 1 --2- 172.21.15.103:0/1680252815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad48104fd0 0x7fad481985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:53.062+0000 7fad4d406640 1 --2- 172.21.15.103:0/1680252815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad481026a0 0x7fad48198070 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:53.062+0000 7fad4d406640 1 -- 172.21.15.103:0/1680252815 >> 172.21.15.103:0/1680252815 conn(0x7fad480fc2a0 msgr2=0x7fad481030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:53.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:53.062+0000 7fad4d406640 1 -- 172.21.15.103:0/1680252815 shutdown_connections 2024-09-19T06:47:53.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:53.062+0000 7fad4d406640 1 -- 172.21.15.103:0/1680252815 wait complete. 2024-09-19T06:47:53.065 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:52 smithi103 ceph-mon[31158]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:47:53.066 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:52 smithi103 ceph-mon[31158]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 16 op/s; 79 B/s, 2 objects/s recovering 2024-09-19T06:47:53.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:52 smithi134 ceph-mon[35356]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:47:53.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:52 smithi134 ceph-mon[35356]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 16 op/s; 79 B/s, 2 objects/s recovering 2024-09-19T06:47:53.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:52 smithi174 ceph-mon[35637]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:47:53.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:52 smithi174 ceph-mon[35637]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 16 op/s; 79 B/s, 2 objects/s recovering 2024-09-19T06:47:53.689 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:47:53.689 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T06:47:53.689 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T06:47:53.725 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T06:47:53.725 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T06:47:53.726 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T06:47:54.013 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:54.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:53 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1680252815' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T06:47:54.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:53 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1680252815' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T06:47:54.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:53 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1680252815' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T06:47:54.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:53 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1680252815' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T06:47:54.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:53 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1680252815' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T06:47:54.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:53 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1680252815' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T06:47:54.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 -- 172.21.15.103:0/1051829242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f391c102d40 msgr2=0x7f391c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:54.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 --2- 172.21.15.103:0/1051829242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f391c102d40 0x7f391c103120 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f390c009900 tx=0x7f390c02a480 comp rx=0 tx=0).stop 2024-09-19T06:47:54.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 -- 172.21.15.103:0/1051829242 shutdown_connections 2024-09-19T06:47:54.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 --2- 172.21.15.103:0/1051829242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f391c10c170 0x7f391c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 --2- 172.21.15.103:0/1051829242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f391c103660 0x7f391c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 --2- 172.21.15.103:0/1051829242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f391c102d40 0x7f391c103120 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 -- 172.21.15.103:0/1051829242 >> 172.21.15.103:0/1051829242 conn(0x7f391c0fc7a0 msgr2=0x7f391c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:54.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 -- 172.21.15.103:0/1051829242 shutdown_connections 2024-09-19T06:47:54.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 -- 172.21.15.103:0/1051829242 wait complete. 2024-09-19T06:47:54.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 Processor -- start 2024-09-19T06:47:54.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 -- start start 2024-09-19T06:47:54.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f391c102d40 0x7f391c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f391c103660 0x7f391c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f391c10c170 0x7f391c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f391c111f70 con 0x7f391c10c170 2024-09-19T06:47:54.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f391c111df0 con 0x7f391c102d40 2024-09-19T06:47:54.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.433+0000 7f39225a9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1120f0 con 0x7f391c103660 2024-09-19T06:47:54.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.434+0000 7f3920b46640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f391c10c170 0x7f391c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.434+0000 7f3920b46640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f391c10c170 0x7f391c19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47334/0 (socket says 172.21.15.103:47334) 2024-09-19T06:47:54.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.434+0000 7f3920b46640 1 -- 172.21.15.103:0/1684393686 learned_addr learned my addr 172.21.15.103:0/1684393686 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:54.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.434+0000 7f391bfff640 1 --2- 172.21.15.103:0/1684393686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f391c102d40 0x7f391c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.434+0000 7f3913fff640 1 --2- 172.21.15.103:0/1684393686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f391c103660 0x7f391c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.434+0000 7f3920b46640 1 -- 172.21.15.103:0/1684393686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f391c103660 msgr2=0x7f391c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:54.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.434+0000 7f3920b46640 1 --2- 172.21.15.103:0/1684393686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f391c103660 0x7f391c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.434+0000 7f3920b46640 1 -- 172.21.15.103:0/1684393686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f391c102d40 msgr2=0x7f391c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:54.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.434+0000 7f3920b46640 1 --2- 172.21.15.103:0/1684393686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f391c102d40 0x7f391c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.434+0000 7f3920b46640 1 -- 172.21.15.103:0/1684393686 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391c19d380 con 0x7f391c10c170 2024-09-19T06:47:54.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.434+0000 7f3920b46640 1 --2- 172.21.15.103:0/1684393686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f391c10c170 0x7f391c19cca0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f391400cce0 tx=0x7f391400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.434+0000 7f3919ffb640 1 -- 172.21.15.103:0/1684393686 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391400f450 con 0x7f391c10c170 2024-09-19T06:47:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.434+0000 7f3919ffb640 1 -- 172.21.15.103:0/1684393686 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f391400fad0 con 0x7f391c10c170 2024-09-19T06:47:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.434+0000 7f39225a9640 1 -- 172.21.15.103:0/1684393686 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391c19d670 con 0x7f391c10c170 2024-09-19T06:47:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.434+0000 7f3919ffb640 1 -- 172.21.15.103:0/1684393686 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914018c10 con 0x7f391c10c170 2024-09-19T06:47:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.434+0000 7f39225a9640 1 -- 172.21.15.103:0/1684393686 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391c1a4f50 con 0x7f391c10c170 2024-09-19T06:47:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.435+0000 7f39225a9640 1 -- 172.21.15.103:0/1684393686 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391c103120 con 0x7f391c10c170 2024-09-19T06:47:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.436+0000 7f3919ffb640 1 -- 172.21.15.103:0/1684393686 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f391400f5f0 con 0x7f391c10c170 2024-09-19T06:47:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.437+0000 7f3919ffb640 1 --2- 172.21.15.103:0/1684393686 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f39080a32d0 0x7f39080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.437+0000 7f3919ffb640 1 -- 172.21.15.103:0/1684393686 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f39140d0110 con 0x7f391c10c170 2024-09-19T06:47:54.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.438+0000 7f391bfff640 1 --2- 172.21.15.103:0/1684393686 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f39080a32d0 0x7f39080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.438+0000 7f391bfff640 1 --2- 172.21.15.103:0/1684393686 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f39080a32d0 0x7f39080a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f390c000c00 tx=0x7f390c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:54.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.442+0000 7f3919ffb640 1 -- 172.21.15.103:0/1684393686 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3914014030 con 0x7f391c10c170 2024-09-19T06:47:54.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.652+0000 7f39225a9640 1 -- 172.21.15.103:0/1684393686 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f391c1a51b0 con 0x7f391c10c170 2024-09-19T06:47:54.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.657+0000 7f3919ffb640 1 -- 172.21.15.103:0/1684393686 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264056 (secure 0 0 0) 0x7f391409cce0 con 0x7f391c10c170 2024-09-19T06:47:54.662 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:47:54.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.660+0000 7f39225a9640 1 -- 172.21.15.103:0/1684393686 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f39080a32d0 msgr2=0x7f39080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:54.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.660+0000 7f39225a9640 1 --2- 172.21.15.103:0/1684393686 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f39080a32d0 0x7f39080a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f390c000c00 tx=0x7f390c0058d0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.660+0000 7f39225a9640 1 -- 172.21.15.103:0/1684393686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f391c10c170 msgr2=0x7f391c19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:54.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.660+0000 7f39225a9640 1 --2- 172.21.15.103:0/1684393686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f391c10c170 0x7f391c19cca0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f391400cce0 tx=0x7f391400a620 comp rx=0 tx=0).stop 2024-09-19T06:47:54.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.660+0000 7f39225a9640 1 -- 172.21.15.103:0/1684393686 shutdown_connections 2024-09-19T06:47:54.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.660+0000 7f39225a9640 1 --2- 172.21.15.103:0/1684393686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f391c10c170 0x7f391c19cca0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.660+0000 7f39225a9640 1 --2- 172.21.15.103:0/1684393686 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f39080a32d0 0x7f39080a5790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.660+0000 7f39225a9640 1 --2- 172.21.15.103:0/1684393686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f391c103660 0x7f391c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.660+0000 7f39225a9640 1 --2- 172.21.15.103:0/1684393686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f391c102d40 0x7f391c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.660+0000 7f39225a9640 1 -- 172.21.15.103:0/1684393686 >> 172.21.15.103:0/1684393686 conn(0x7f391c0fc7a0 msgr2=0x7f391c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:54.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.660+0000 7f39225a9640 1 -- 172.21.15.103:0/1684393686 shutdown_connections 2024-09-19T06:47:54.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:54.660+0000 7f39225a9640 1 -- 172.21.15.103:0/1684393686 wait complete. 2024-09-19T06:47:55.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:54 smithi134 ceph-mon[35356]: pgmap v289: 97 pgs: 97 active+clean; 637 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.7 KiB/s wr, 11 op/s; 55 B/s, 0 objects/s recovering 2024-09-19T06:47:55.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:54 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1684393686' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T06:47:55.285 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:54 smithi103 ceph-mon[31158]: pgmap v289: 97 pgs: 97 active+clean; 637 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.7 KiB/s wr, 11 op/s; 55 B/s, 0 objects/s recovering 2024-09-19T06:47:55.285 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:54 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1684393686' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T06:47:55.312 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2590139910},{"type":"v1","addr":"172.21.15.103:6801","nonce":2590139910}]},"active_addr":"172.21.15.103:6801/2590139910","active_change":"2024-09-19T06:40:39.739787+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.103:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":471230312}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":1074662744}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":70266056}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":2169657832}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":2050139492}]}]} 2024-09-19T06:47:55.317 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T06:47:55.317 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T06:47:55.318 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:47:55.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:54 smithi174 ceph-mon[35637]: pgmap v289: 97 pgs: 97 active+clean; 637 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.7 KiB/s wr, 11 op/s; 55 B/s, 0 objects/s recovering 2024-09-19T06:47:55.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:54 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1684393686' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T06:47:55.567 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:56.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.982+0000 7f9678c2d640 1 -- 172.21.15.103:0/4235022947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9674106a80 msgr2=0x7f9674106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:56.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.982+0000 7f9678c2d640 1 --2- 172.21.15.103:0/4235022947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9674106a80 0x7f9674106e60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9664009930 tx=0x7f966402a420 comp rx=0 tx=0).stop 2024-09-19T06:47:56.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.982+0000 7f9678c2d640 1 -- 172.21.15.103:0/4235022947 shutdown_connections 2024-09-19T06:47:56.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.982+0000 7f9678c2d640 1 --2- 172.21.15.103:0/4235022947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9674101420 0x7f967410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:56.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.982+0000 7f9678c2d640 1 --2- 172.21.15.103:0/4235022947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9674100a80 0x7f9674100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:56.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.982+0000 7f9678c2d640 1 --2- 172.21.15.103:0/4235022947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9674106a80 0x7f9674106e60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:56.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.982+0000 7f9678c2d640 1 -- 172.21.15.103:0/4235022947 >> 172.21.15.103:0/4235022947 conn(0x7f96740fc7a0 msgr2=0x7f96740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:56.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.982+0000 7f9678c2d640 1 -- 172.21.15.103:0/4235022947 shutdown_connections 2024-09-19T06:47:56.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.982+0000 7f9678c2d640 1 -- 172.21.15.103:0/4235022947 wait complete. 2024-09-19T06:47:56.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.983+0000 7f9678c2d640 1 Processor -- start 2024-09-19T06:47:56.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.983+0000 7f9678c2d640 1 -- start start 2024-09-19T06:47:56.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.983+0000 7f9678c2d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9674100a80 0x7f96740fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:56.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.983+0000 7f9678c2d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9674101420 0x7f96740fd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:56.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.983+0000 7f9678c2d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9674106a80 0x7f96740fd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:56.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.983+0000 7f9678c2d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9674110ea0 con 0x7f9674100a80 2024-09-19T06:47:56.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.983+0000 7f9678c2d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9674110d20 con 0x7f9674106a80 2024-09-19T06:47:56.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.983+0000 7f9678c2d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9674111020 con 0x7f9674101420 2024-09-19T06:47:56.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.983+0000 7f967259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9674100a80 0x7f96740fcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:56.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.983+0000 7f967259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9674100a80 0x7f96740fcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47340/0 (socket says 172.21.15.103:47340) 2024-09-19T06:47:56.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.983+0000 7f967259c640 1 -- 172.21.15.103:0/2323699451 learned_addr learned my addr 172.21.15.103:0/2323699451 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:56.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.984+0000 7f966bfff640 1 --2- 172.21.15.103:0/2323699451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9674101420 0x7f96740fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:56.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.984+0000 7f9672d9d640 1 --2- 172.21.15.103:0/2323699451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9674106a80 0x7f96740fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:56.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.984+0000 7f967259c640 1 -- 172.21.15.103:0/2323699451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9674101420 msgr2=0x7f96740fd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:56.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.984+0000 7f967259c640 1 --2- 172.21.15.103:0/2323699451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9674101420 0x7f96740fd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:56.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.984+0000 7f967259c640 1 -- 172.21.15.103:0/2323699451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9674106a80 msgr2=0x7f96740fd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:56.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.984+0000 7f967259c640 1 --2- 172.21.15.103:0/2323699451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9674106a80 0x7f96740fd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:56.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.984+0000 7f967259c640 1 -- 172.21.15.103:0/2323699451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96741a6950 con 0x7f9674100a80 2024-09-19T06:47:56.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.984+0000 7f967259c640 1 --2- 172.21.15.103:0/2323699451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9674100a80 0x7f96740fcd10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9664009a50 tx=0x7f9664004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:56.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.984+0000 7f966b7fe640 1 -- 172.21.15.103:0/2323699451 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966403f030 con 0x7f9674100a80 2024-09-19T06:47:56.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.984+0000 7f9678c2d640 1 -- 172.21.15.103:0/2323699451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96741a6be0 con 0x7f9674100a80 2024-09-19T06:47:56.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.984+0000 7f966b7fe640 1 -- 172.21.15.103:0/2323699451 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9664003ce0 con 0x7f9674100a80 2024-09-19T06:47:56.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.984+0000 7f966b7fe640 1 -- 172.21.15.103:0/2323699451 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9664035070 con 0x7f9674100a80 2024-09-19T06:47:56.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.984+0000 7f9678c2d640 1 -- 172.21.15.103:0/2323699451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96741a7110 con 0x7f9674100a80 2024-09-19T06:47:56.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.985+0000 7f9678c2d640 1 -- 172.21.15.103:0/2323699451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9674100ee0 con 0x7f9674100a80 2024-09-19T06:47:56.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.986+0000 7f966b7fe640 1 -- 172.21.15.103:0/2323699451 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9664042030 con 0x7f9674100a80 2024-09-19T06:47:56.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.987+0000 7f966b7fe640 1 --2- 172.21.15.103:0/2323699451 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f96600a32d0 0x7f96600a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:56.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.987+0000 7f966bfff640 1 --2- 172.21.15.103:0/2323699451 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f96600a32d0 0x7f96600a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:56.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.987+0000 7f966b7fe640 1 -- 172.21.15.103:0/2323699451 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f96640f04f0 con 0x7f9674100a80 2024-09-19T06:47:56.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.987+0000 7f966bfff640 1 --2- 172.21.15.103:0/2323699451 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f96600a32d0 0x7f96600a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f965c005e10 tx=0x7f965c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:56.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:55.990+0000 7f966b7fe640 1 -- 172.21.15.103:0/2323699451 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f96640bd0c0 con 0x7f9674100a80 2024-09-19T06:47:56.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:56.160+0000 7f9678c2d640 1 -- 172.21.15.103:0/2323699451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f967410a000 con 0x7f9674100a80 2024-09-19T06:47:56.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:56.161+0000 7f966b7fe640 1 -- 172.21.15.103:0/2323699451 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22729 (secure 0 0 0) 0x7f96640bc9c0 con 0x7f9674100a80 2024-09-19T06:47:56.163 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:47:56.164 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":82,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","created":"2024-09-19T06:39:46.840574+0000","modified":"2024-09-19T06:47:40.819637+0000","last_up_change":"2024-09-19T06:46:35.652960+0000","last_in_change":"2024-09-19T06:46:11.630585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:42:52.050881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:47:21.192611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:47:23.643291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d45dab3-e434-4783-b2e3-f658c3ba583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6803","nonce":3157697740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6805","nonce":3157697740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6809","nonce":3157697740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6807","nonce":3157697740}]},"public_addr":"172.21.15.103:6803/3157697740","cluster_addr":"172.21.15.103:6805/3157697740","heartbeat_back_addr":"172.21.15.103:6809/3157697740","heartbeat_front_addr":"172.21.15.103:6807/3157697740","state":["exists","up"]},{"osd":1,"uuid":"ee0e92bd-15ab-4867-9a18-20abb9f5fc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6801","nonce":1418301195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6803","nonce":1418301195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6807","nonce":1418301195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6805","nonce":1418301195}]},"public_addr":"172.21.15.134:6801/1418301195","cluster_addr":"172.21.15.134:6803/1418301195","heartbeat_back_addr":"172.21.15.134:6807/1418301195","heartbeat_front_addr":"172.21.15.134:6805/1418301195","state":["exists","up"]},{"osd":2,"uuid":"0230caed-025b-46e9-a011-237601d16ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6801","nonce":2292242290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6803","nonce":2292242290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6807","nonce":2292242290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6805","nonce":2292242290}]},"public_addr":"172.21.15.174:6801/2292242290","cluster_addr":"172.21.15.174:6803/2292242290","heartbeat_back_addr":"172.21.15.174:6807/2292242290","heartbeat_front_addr":"172.21.15.174:6805/2292242290","state":["exists","up"]},{"osd":3,"uuid":"c54a8399-bf4b-4d70-b207-a13d85f99590","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6811","nonce":2698986333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6813","nonce":2698986333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6817","nonce":2698986333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6815","nonce":2698986333}]},"public_addr":"172.21.15.103:6811/2698986333","cluster_addr":"172.21.15.103:6813/2698986333","heartbeat_back_addr":"172.21.15.103:6817/2698986333","heartbeat_front_addr":"172.21.15.103:6815/2698986333","state":["exists","up"]},{"osd":4,"uuid":"7c046be6-464a-4ab5-812b-1781121fc317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6809","nonce":3169891968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6811","nonce":3169891968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6815","nonce":3169891968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6813","nonce":3169891968}]},"public_addr":"172.21.15.134:6809/3169891968","cluster_addr":"172.21.15.134:6811/3169891968","heartbeat_back_addr":"172.21.15.134:6815/3169891968","heartbeat_front_addr":"172.21.15.134:6813/3169891968","state":["exists","up"]},{"osd":5,"uuid":"8622596b-6a06-459c-882d-50fba9e296ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6809","nonce":763749644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6811","nonce":763749644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6815","nonce":763749644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6813","nonce":763749644}]},"public_addr":"172.21.15.174:6809/763749644","cluster_addr":"172.21.15.174:6811/763749644","heartbeat_back_addr":"172.21.15.174:6815/763749644","heartbeat_front_addr":"172.21.15.174:6813/763749644","state":["exists","up"]},{"osd":6,"uuid":"64d858d6-a8b3-4f65-a187-af9ab1ccee0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6819","nonce":1396972394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6821","nonce":1396972394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6825","nonce":1396972394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6823","nonce":1396972394}]},"public_addr":"172.21.15.103:6819/1396972394","cluster_addr":"172.21.15.103:6821/1396972394","heartbeat_back_addr":"172.21.15.103:6825/1396972394","heartbeat_front_addr":"172.21.15.103:6823/1396972394","state":["exists","up"]},{"osd":7,"uuid":"d405d1c6-839c-4c92-8261-4adb3163c90f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6817","nonce":2196366765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6819","nonce":2196366765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6823","nonce":2196366765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6821","nonce":2196366765}]},"public_addr":"172.21.15.134:6817/2196366765","cluster_addr":"172.21.15.134:6819/2196366765","heartbeat_back_addr":"172.21.15.134:6823/2196366765","heartbeat_front_addr":"172.21.15.134:6821/2196366765","state":["exists","up"]},{"osd":8,"uuid":"4c452cd9-816c-4f77-9368-2cd4bbeee4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6817","nonce":189977465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6819","nonce":189977465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6823","nonce":189977465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6821","nonce":189977465}]},"public_addr":"172.21.15.174:6817/189977465","cluster_addr":"172.21.15.174:6819/189977465","heartbeat_back_addr":"172.21.15.174:6823/189977465","heartbeat_front_addr":"172.21.15.174:6821/189977465","state":["exists","up"]},{"osd":9,"uuid":"fb3662e0-2d8e-45c4-9119-6d08b35443da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6827","nonce":3161966413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6829","nonce":3161966413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6833","nonce":3161966413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6831","nonce":3161966413}]},"public_addr":"172.21.15.103:6827/3161966413","cluster_addr":"172.21.15.103:6829/3161966413","heartbeat_back_addr":"172.21.15.103:6833/3161966413","heartbeat_front_addr":"172.21.15.103:6831/3161966413","state":["exists","up"]},{"osd":10,"uuid":"e52c3cbe-a6af-441e-b423-b94f3a2da544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6825","nonce":2532758632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6827","nonce":2532758632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6831","nonce":2532758632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6829","nonce":2532758632}]},"public_addr":"172.21.15.134:6825/2532758632","cluster_addr":"172.21.15.134:6827/2532758632","heartbeat_back_addr":"172.21.15.134:6831/2532758632","heartbeat_front_addr":"172.21.15.134:6829/2532758632","state":["exists","up"]},{"osd":11,"uuid":"9040f3f8-3873-4dea-8716-dcda5811c8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6825","nonce":161916442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6827","nonce":161916442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6831","nonce":161916442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6829","nonce":161916442}]},"public_addr":"172.21.15.174:6825/161916442","cluster_addr":"172.21.15.174:6827/161916442","heartbeat_back_addr":"172.21.15.174:6831/161916442","heartbeat_front_addr":"172.21.15.174:6829/161916442","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:41:59.656854+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:24.797551+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:49.685497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:33.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:39.324983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:03.987310+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:28.747971+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:53.827780+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:18.554450+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:43.705691+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:08.401275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:33.502542+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2867049580":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/3727259725":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/1496153385":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:6800/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1642809745":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/2309624637":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1596229922":"2024-09-20T06:40:16.780890+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:47:56.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:56.164+0000 7f9678c2d640 1 -- 172.21.15.103:0/2323699451 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f96600a32d0 msgr2=0x7f96600a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:56.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:56.164+0000 7f9678c2d640 1 --2- 172.21.15.103:0/2323699451 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f96600a32d0 0x7f96600a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f965c005e10 tx=0x7f965c005da0 comp rx=0 tx=0).stop 2024-09-19T06:47:56.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:56.164+0000 7f9678c2d640 1 -- 172.21.15.103:0/2323699451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9674100a80 msgr2=0x7f96740fcd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:56.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:56.164+0000 7f9678c2d640 1 --2- 172.21.15.103:0/2323699451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9674100a80 0x7f96740fcd10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9664009a50 tx=0x7f9664004270 comp rx=0 tx=0).stop 2024-09-19T06:47:56.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:56.164+0000 7f9678c2d640 1 -- 172.21.15.103:0/2323699451 shutdown_connections 2024-09-19T06:47:56.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:56.164+0000 7f9678c2d640 1 --2- 172.21.15.103:0/2323699451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9674106a80 0x7f96740fd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:56.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:56.164+0000 7f9678c2d640 1 --2- 172.21.15.103:0/2323699451 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f96600a32d0 0x7f96600a5790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:56.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:56.164+0000 7f9678c2d640 1 --2- 172.21.15.103:0/2323699451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9674101420 0x7f96740fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:56.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:56.164+0000 7f9678c2d640 1 --2- 172.21.15.103:0/2323699451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9674100a80 0x7f96740fcd10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:56.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:56.164+0000 7f9678c2d640 1 -- 172.21.15.103:0/2323699451 >> 172.21.15.103:0/2323699451 conn(0x7f96740fc7a0 msgr2=0x7f967406aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:56.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:56.164+0000 7f9678c2d640 1 -- 172.21.15.103:0/2323699451 shutdown_connections 2024-09-19T06:47:56.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:56.164+0000 7f9678c2d640 1 -- 172.21.15.103:0/2323699451 wait complete. 2024-09-19T06:47:56.778 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T06:47:56.779 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:47:57.028 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:57.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:56 smithi103 ceph-mon[31158]: pgmap v290: 97 pgs: 97 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.9 KiB/s wr, 12 op/s; 55 B/s, 0 objects/s recovering 2024-09-19T06:47:57.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:56 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2323699451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:47:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:56 smithi134 ceph-mon[35356]: pgmap v290: 97 pgs: 97 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.9 KiB/s wr, 12 op/s; 55 B/s, 0 objects/s recovering 2024-09-19T06:47:57.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:56 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2323699451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:47:57.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:56 smithi174 ceph-mon[35637]: pgmap v290: 97 pgs: 97 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.9 KiB/s wr, 12 op/s; 55 B/s, 0 objects/s recovering 2024-09-19T06:47:57.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:56 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2323699451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:47:57.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.440+0000 7f12f32f9640 1 -- 172.21.15.103:0/147351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12ec102790 msgr2=0x7f12ec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:57.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.440+0000 7f12f32f9640 1 --2- 172.21.15.103:0/147351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12ec102790 0x7f12ec1067e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009900 tx=0x7f12e802a410 comp rx=0 tx=0).stop 2024-09-19T06:47:57.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.440+0000 7f12f32f9640 1 -- 172.21.15.103:0/147351 shutdown_connections 2024-09-19T06:47:57.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.440+0000 7f12f32f9640 1 --2- 172.21.15.103:0/147351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12ec106f10 0x7f12ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.440+0000 7f12f32f9640 1 --2- 172.21.15.103:0/147351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12ec102790 0x7f12ec1067e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.440+0000 7f12f32f9640 1 --2- 172.21.15.103:0/147351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12ec101de0 0x7f12ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.440+0000 7f12f32f9640 1 -- 172.21.15.103:0/147351 >> 172.21.15.103:0/147351 conn(0x7f12ec0fd8c0 msgr2=0x7f12ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:57.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.440+0000 7f12f32f9640 1 -- 172.21.15.103:0/147351 shutdown_connections 2024-09-19T06:47:57.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.440+0000 7f12f32f9640 1 -- 172.21.15.103:0/147351 wait complete. 2024-09-19T06:47:57.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.440+0000 7f12f32f9640 1 Processor -- start 2024-09-19T06:47:57.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.440+0000 7f12f32f9640 1 -- start start 2024-09-19T06:47:57.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f32f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12ec101de0 0x7f12ec0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f32f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12ec102790 0x7f12ec074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f32f9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12ec106f10 0x7f12ec074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f32f9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec111f30 con 0x7f12ec101de0 2024-09-19T06:47:57.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f32f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec111db0 con 0x7f12ec102790 2024-09-19T06:47:57.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f32f9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec1120b0 con 0x7f12ec106f10 2024-09-19T06:47:57.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f1095640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12ec101de0 0x7f12ec0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f1095640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12ec101de0 0x7f12ec0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36606/0 (socket says 172.21.15.103:36606) 2024-09-19T06:47:57.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f1095640 1 -- 172.21.15.103:0/1382501738 learned_addr learned my addr 172.21.15.103:0/1382501738 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:47:57.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f0894640 1 --2- 172.21.15.103:0/1382501738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12ec102790 0x7f12ec074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f1896640 1 --2- 172.21.15.103:0/1382501738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12ec106f10 0x7f12ec074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f1095640 1 -- 172.21.15.103:0/1382501738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12ec106f10 msgr2=0x7f12ec074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:57.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f1095640 1 --2- 172.21.15.103:0/1382501738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12ec106f10 0x7f12ec074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f1095640 1 -- 172.21.15.103:0/1382501738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12ec102790 msgr2=0x7f12ec074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:57.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f1095640 1 --2- 172.21.15.103:0/1382501738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12ec102790 0x7f12ec074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f1095640 1 -- 172.21.15.103:0/1382501738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ec1a6880 con 0x7f12ec101de0 2024-09-19T06:47:57.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f0894640 1 --2- 172.21.15.103:0/1382501738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12ec102790 0x7f12ec074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:57.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f1896640 1 --2- 172.21.15.103:0/1382501738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12ec106f10 0x7f12ec074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:57.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f1095640 1 --2- 172.21.15.103:0/1382501738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12ec101de0 0x7f12ec0744f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00c920 tx=0x7f12dc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:57.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12e27fc640 1 -- 172.21.15.103:0/1382501738 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc007c70 con 0x7f12ec101de0 2024-09-19T06:47:57.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12e27fc640 1 -- 172.21.15.103:0/1382501738 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12dc007e10 con 0x7f12ec101de0 2024-09-19T06:47:57.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.441+0000 7f12f32f9640 1 -- 172.21.15.103:0/1382501738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ec1a6b70 con 0x7f12ec101de0 2024-09-19T06:47:57.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.442+0000 7f12e27fc640 1 -- 172.21.15.103:0/1382501738 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc018890 con 0x7f12ec101de0 2024-09-19T06:47:57.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.442+0000 7f12f32f9640 1 -- 172.21.15.103:0/1382501738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ec1a70b0 con 0x7f12ec101de0 2024-09-19T06:47:57.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.443+0000 7f12f32f9640 1 -- 172.21.15.103:0/1382501738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12ec1021c0 con 0x7f12ec101de0 2024-09-19T06:47:57.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.444+0000 7f12e27fc640 1 -- 172.21.15.103:0/1382501738 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f12dc018a70 con 0x7f12ec101de0 2024-09-19T06:47:57.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.444+0000 7f12e27fc640 1 --2- 172.21.15.103:0/1382501738 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f12d80a32d0 0x7f12d80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.444+0000 7f12f0894640 1 --2- 172.21.15.103:0/1382501738 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f12d80a32d0 0x7f12d80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.444+0000 7f12e27fc640 1 -- 172.21.15.103:0/1382501738 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f12dc0cfc90 con 0x7f12ec101de0 2024-09-19T06:47:57.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.445+0000 7f12f0894640 1 --2- 172.21.15.103:0/1382501738 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f12d80a32d0 0x7f12d80a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12e8000c00 tx=0x7f12e8032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:57.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.449+0000 7f12e27fc640 1 -- 172.21.15.103:0/1382501738 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f12dc0d5020 con 0x7f12ec101de0 2024-09-19T06:47:57.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.618+0000 7f12f32f9640 1 -- 172.21.15.103:0/1382501738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12ec10a120 con 0x7f12ec101de0 2024-09-19T06:47:57.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.619+0000 7f12e27fc640 1 -- 172.21.15.103:0/1382501738 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22729 (secure 0 0 0) 0x7f12dc014030 con 0x7f12ec101de0 2024-09-19T06:47:57.622 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:47:57.622 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":82,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","created":"2024-09-19T06:39:46.840574+0000","modified":"2024-09-19T06:47:40.819637+0000","last_up_change":"2024-09-19T06:46:35.652960+0000","last_in_change":"2024-09-19T06:46:11.630585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:42:52.050881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:47:21.192611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:47:23.643291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d45dab3-e434-4783-b2e3-f658c3ba583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6803","nonce":3157697740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6805","nonce":3157697740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6809","nonce":3157697740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6807","nonce":3157697740}]},"public_addr":"172.21.15.103:6803/3157697740","cluster_addr":"172.21.15.103:6805/3157697740","heartbeat_back_addr":"172.21.15.103:6809/3157697740","heartbeat_front_addr":"172.21.15.103:6807/3157697740","state":["exists","up"]},{"osd":1,"uuid":"ee0e92bd-15ab-4867-9a18-20abb9f5fc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6801","nonce":1418301195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6803","nonce":1418301195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6807","nonce":1418301195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6805","nonce":1418301195}]},"public_addr":"172.21.15.134:6801/1418301195","cluster_addr":"172.21.15.134:6803/1418301195","heartbeat_back_addr":"172.21.15.134:6807/1418301195","heartbeat_front_addr":"172.21.15.134:6805/1418301195","state":["exists","up"]},{"osd":2,"uuid":"0230caed-025b-46e9-a011-237601d16ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6801","nonce":2292242290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6803","nonce":2292242290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6807","nonce":2292242290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6805","nonce":2292242290}]},"public_addr":"172.21.15.174:6801/2292242290","cluster_addr":"172.21.15.174:6803/2292242290","heartbeat_back_addr":"172.21.15.174:6807/2292242290","heartbeat_front_addr":"172.21.15.174:6805/2292242290","state":["exists","up"]},{"osd":3,"uuid":"c54a8399-bf4b-4d70-b207-a13d85f99590","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6811","nonce":2698986333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6813","nonce":2698986333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6817","nonce":2698986333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6815","nonce":2698986333}]},"public_addr":"172.21.15.103:6811/2698986333","cluster_addr":"172.21.15.103:6813/2698986333","heartbeat_back_addr":"172.21.15.103:6817/2698986333","heartbeat_front_addr":"172.21.15.103:6815/2698986333","state":["exists","up"]},{"osd":4,"uuid":"7c046be6-464a-4ab5-812b-1781121fc317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6809","nonce":3169891968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6811","nonce":3169891968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6815","nonce":3169891968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6813","nonce":3169891968}]},"public_addr":"172.21.15.134:6809/3169891968","cluster_addr":"172.21.15.134:6811/3169891968","heartbeat_back_addr":"172.21.15.134:6815/3169891968","heartbeat_front_addr":"172.21.15.134:6813/3169891968","state":["exists","up"]},{"osd":5,"uuid":"8622596b-6a06-459c-882d-50fba9e296ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6809","nonce":763749644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6811","nonce":763749644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6815","nonce":763749644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6813","nonce":763749644}]},"public_addr":"172.21.15.174:6809/763749644","cluster_addr":"172.21.15.174:6811/763749644","heartbeat_back_addr":"172.21.15.174:6815/763749644","heartbeat_front_addr":"172.21.15.174:6813/763749644","state":["exists","up"]},{"osd":6,"uuid":"64d858d6-a8b3-4f65-a187-af9ab1ccee0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6819","nonce":1396972394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6821","nonce":1396972394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6825","nonce":1396972394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6823","nonce":1396972394}]},"public_addr":"172.21.15.103:6819/1396972394","cluster_addr":"172.21.15.103:6821/1396972394","heartbeat_back_addr":"172.21.15.103:6825/1396972394","heartbeat_front_addr":"172.21.15.103:6823/1396972394","state":["exists","up"]},{"osd":7,"uuid":"d405d1c6-839c-4c92-8261-4adb3163c90f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6817","nonce":2196366765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6819","nonce":2196366765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6823","nonce":2196366765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6821","nonce":2196366765}]},"public_addr":"172.21.15.134:6817/2196366765","cluster_addr":"172.21.15.134:6819/2196366765","heartbeat_back_addr":"172.21.15.134:6823/2196366765","heartbeat_front_addr":"172.21.15.134:6821/2196366765","state":["exists","up"]},{"osd":8,"uuid":"4c452cd9-816c-4f77-9368-2cd4bbeee4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6817","nonce":189977465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6819","nonce":189977465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6823","nonce":189977465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6821","nonce":189977465}]},"public_addr":"172.21.15.174:6817/189977465","cluster_addr":"172.21.15.174:6819/189977465","heartbeat_back_addr":"172.21.15.174:6823/189977465","heartbeat_front_addr":"172.21.15.174:6821/189977465","state":["exists","up"]},{"osd":9,"uuid":"fb3662e0-2d8e-45c4-9119-6d08b35443da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6827","nonce":3161966413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6829","nonce":3161966413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6833","nonce":3161966413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6831","nonce":3161966413}]},"public_addr":"172.21.15.103:6827/3161966413","cluster_addr":"172.21.15.103:6829/3161966413","heartbeat_back_addr":"172.21.15.103:6833/3161966413","heartbeat_front_addr":"172.21.15.103:6831/3161966413","state":["exists","up"]},{"osd":10,"uuid":"e52c3cbe-a6af-441e-b423-b94f3a2da544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6825","nonce":2532758632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6827","nonce":2532758632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6831","nonce":2532758632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6829","nonce":2532758632}]},"public_addr":"172.21.15.134:6825/2532758632","cluster_addr":"172.21.15.134:6827/2532758632","heartbeat_back_addr":"172.21.15.134:6831/2532758632","heartbeat_front_addr":"172.21.15.134:6829/2532758632","state":["exists","up"]},{"osd":11,"uuid":"9040f3f8-3873-4dea-8716-dcda5811c8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6825","nonce":161916442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6827","nonce":161916442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6831","nonce":161916442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6829","nonce":161916442}]},"public_addr":"172.21.15.174:6825/161916442","cluster_addr":"172.21.15.174:6827/161916442","heartbeat_back_addr":"172.21.15.174:6831/161916442","heartbeat_front_addr":"172.21.15.174:6829/161916442","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:41:59.656854+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:24.797551+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:49.685497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:33.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:39.324983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:03.987310+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:28.747971+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:53.827780+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:18.554450+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:43.705691+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:08.401275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:33.502542+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2867049580":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/3727259725":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/1496153385":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:6800/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1642809745":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/2309624637":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1596229922":"2024-09-20T06:40:16.780890+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:47:57.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.622+0000 7f12f32f9640 1 -- 172.21.15.103:0/1382501738 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f12d80a32d0 msgr2=0x7f12d80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:57.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.622+0000 7f12f32f9640 1 --2- 172.21.15.103:0/1382501738 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f12d80a32d0 0x7f12d80a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12e8000c00 tx=0x7f12e8032c70 comp rx=0 tx=0).stop 2024-09-19T06:47:57.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.622+0000 7f12f32f9640 1 -- 172.21.15.103:0/1382501738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12ec101de0 msgr2=0x7f12ec0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:57.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.622+0000 7f12f32f9640 1 --2- 172.21.15.103:0/1382501738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12ec101de0 0x7f12ec0744f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00c920 tx=0x7f12dc00cde0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.622+0000 7f12f32f9640 1 -- 172.21.15.103:0/1382501738 shutdown_connections 2024-09-19T06:47:57.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.622+0000 7f12f32f9640 1 --2- 172.21.15.103:0/1382501738 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f12d80a32d0 0x7f12d80a5790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.622+0000 7f12f32f9640 1 --2- 172.21.15.103:0/1382501738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12ec106f10 0x7f12ec074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.622+0000 7f12f32f9640 1 --2- 172.21.15.103:0/1382501738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12ec102790 0x7f12ec074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.622+0000 7f12f32f9640 1 --2- 172.21.15.103:0/1382501738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12ec101de0 0x7f12ec0744f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.622+0000 7f12f32f9640 1 -- 172.21.15.103:0/1382501738 >> 172.21.15.103:0/1382501738 conn(0x7f12ec0fd8c0 msgr2=0x7f12ec0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:57.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.622+0000 7f12f32f9640 1 -- 172.21.15.103:0/1382501738 shutdown_connections 2024-09-19T06:47:57.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:47:57.622+0000 7f12f32f9640 1 -- 172.21.15.103:0/1382501738 wait complete. 2024-09-19T06:47:58.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:57 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1382501738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:47:58.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:57 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1382501738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:47:58.342 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:57 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1382501738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:47:58.371 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T06:47:58.372 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T06:47:58.372 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T06:47:58.373 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T06:47:58.373 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T06:47:58.374 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T06:47:58.374 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T06:47:58.374 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T06:47:58.375 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T06:47:58.376 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T06:47:58.376 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T06:47:58.377 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T06:47:59.093 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:59.115 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:59.116 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:59.117 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:59.118 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:59.126 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:59.129 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:59.132 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:59.159 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:59.159 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:59.159 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:59.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:47:58 smithi134 ceph-mon[35356]: pgmap v291: 97 pgs: 97 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s; 55 B/s, 0 objects/s recovering 2024-09-19T06:47:59.244 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:47:58 smithi103 ceph-mon[31158]: pgmap v291: 97 pgs: 97 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s; 55 B/s, 0 objects/s recovering 2024-09-19T06:47:59.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:47:58 smithi174 ceph-mon[35637]: pgmap v291: 97 pgs: 97 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s; 55 B/s, 0 objects/s recovering 2024-09-19T06:48:00.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.018+0000 7fb9a62a9640 1 -- 172.21.15.103:0/1862990491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9a01027a0 msgr2=0x7fb9a01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:00.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.018+0000 7fb9a62a9640 1 --2- 172.21.15.103:0/1862990491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9a01027a0 0x7fb9a01067f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb988009900 tx=0x7fb98802a410 comp rx=0 tx=0).stop 2024-09-19T06:48:00.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.018+0000 7fb9a62a9640 1 -- 172.21.15.103:0/1862990491 shutdown_connections 2024-09-19T06:48:00.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.018+0000 7fb9a62a9640 1 --2- 172.21.15.103:0/1862990491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a0106f20 0x7fb9a010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:00.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.018+0000 7fb9a62a9640 1 --2- 172.21.15.103:0/1862990491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9a01027a0 0x7fb9a01067f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:00.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.018+0000 7fb9a62a9640 1 --2- 172.21.15.103:0/1862990491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9a0101df0 0x7fb9a01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:00.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.018+0000 7fb9a62a9640 1 -- 172.21.15.103:0/1862990491 >> 172.21.15.103:0/1862990491 conn(0x7fb9a00fd8b0 msgr2=0x7fb9a00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:00.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.018+0000 7fb9a62a9640 1 -- 172.21.15.103:0/1862990491 shutdown_connections 2024-09-19T06:48:00.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.018+0000 7fb9a62a9640 1 -- 172.21.15.103:0/1862990491 wait complete. 2024-09-19T06:48:00.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.018+0000 7fb9a62a9640 1 Processor -- start 2024-09-19T06:48:00.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.018+0000 7fb9a62a9640 1 -- start start 2024-09-19T06:48:00.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb9a62a9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a0101df0 0x7fb9a00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:00.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb9a62a9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9a01027a0 0x7fb9a0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:00.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb9a62a9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9a0106f20 0x7fb9a0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:00.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb9a62a9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0111fa0 con 0x7fb9a01027a0 2024-09-19T06:48:00.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb9a62a9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0111e20 con 0x7fb9a0101df0 2024-09-19T06:48:00.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb9a62a9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0112120 con 0x7fb9a0106f20 2024-09-19T06:48:00.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb997fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9a01027a0 0x7fb9a0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:00.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb997fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9a01027a0 0x7fb9a0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36630/0 (socket says 172.21.15.103:36630) 2024-09-19T06:48:00.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb997fff640 1 -- 172.21.15.103:0/4099762966 learned_addr learned my addr 172.21.15.103:0/4099762966 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:00.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb99ffff640 1 --2- 172.21.15.103:0/4099762966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a0101df0 0x7fb9a00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:00.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb9a4846640 1 --2- 172.21.15.103:0/4099762966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9a0106f20 0x7fb9a0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:00.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb997fff640 1 -- 172.21.15.103:0/4099762966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9a0106f20 msgr2=0x7fb9a0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:00.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb997fff640 1 --2- 172.21.15.103:0/4099762966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9a0106f20 0x7fb9a0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:00.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb997fff640 1 -- 172.21.15.103:0/4099762966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a0101df0 msgr2=0x7fb9a00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:00.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb997fff640 1 --2- 172.21.15.103:0/4099762966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a0101df0 0x7fb9a00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:00.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb997fff640 1 -- 172.21.15.103:0/4099762966 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a01a69e0 con 0x7fb9a01027a0 2024-09-19T06:48:00.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb9a4846640 1 --2- 172.21.15.103:0/4099762966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9a0106f20 0x7fb9a0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:00.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb99ffff640 1 --2- 172.21.15.103:0/4099762966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a0101df0 0x7fb9a00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:00.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb997fff640 1 --2- 172.21.15.103:0/4099762966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9a01027a0 0x7fb9a0074a30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb988005ab0 tx=0x7fb9880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:00.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb99dffb640 1 -- 172.21.15.103:0/4099762966 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98803f030 con 0x7fb9a01027a0 2024-09-19T06:48:00.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb9a62a9640 1 -- 172.21.15.103:0/4099762966 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a01a6c70 con 0x7fb9a01027a0 2024-09-19T06:48:00.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb99dffb640 1 -- 172.21.15.103:0/4099762966 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb988003ce0 con 0x7fb9a01027a0 2024-09-19T06:48:00.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb99dffb640 1 -- 172.21.15.103:0/4099762966 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988035070 con 0x7fb9a01027a0 2024-09-19T06:48:00.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.019+0000 7fb9a62a9640 1 -- 172.21.15.103:0/4099762966 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a01a7150 con 0x7fb9a01027a0 2024-09-19T06:48:00.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.020+0000 7fb9a62a9640 1 -- 172.21.15.103:0/4099762966 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb9a00628c0 con 0x7fb9a01027a0 2024-09-19T06:48:00.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.021+0000 7fb99dffb640 1 -- 172.21.15.103:0/4099762966 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb988042030 con 0x7fb9a01027a0 2024-09-19T06:48:00.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.022+0000 7fb99dffb640 1 --2- 172.21.15.103:0/4099762966 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb98c0a32d0 0x7fb98c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:00.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.022+0000 7fb99ffff640 1 --2- 172.21.15.103:0/4099762966 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb98c0a32d0 0x7fb98c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:00.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.022+0000 7fb99dffb640 1 -- 172.21.15.103:0/4099762966 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb9880f0450 con 0x7fb9a01027a0 2024-09-19T06:48:00.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.022+0000 7fb99dffb640 1 --2- 172.21.15.103:0/4099762966 >> [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740] conn(0x7fb98c0ace80 0x7fb98c0af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:00.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.022+0000 7fb99dffb640 1 -- 172.21.15.103:0/4099762966 --> [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb98c0af9f0 con 0x7fb98c0ace80 2024-09-19T06:48:00.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.022+0000 7fb99dffb640 1 -- 172.21.15.103:0/4099762966 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fb9880f7050 con 0x7fb9a01027a0 2024-09-19T06:48:00.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.022+0000 7fb9a4846640 1 --2- 172.21.15.103:0/4099762966 >> [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740] conn(0x7fb98c0ace80 0x7fb98c0af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:00.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.022+0000 7fb99ffff640 1 --2- 172.21.15.103:0/4099762966 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb98c0a32d0 0x7fb98c0a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb990009930 tx=0x7fb990008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:00.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.023+0000 7fb9a4846640 1 --2- 172.21.15.103:0/4099762966 >> [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740] conn(0x7fb98c0ace80 0x7fb98c0af320 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:00.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.023+0000 7fb99dffb640 1 -- 172.21.15.103:0/4099762966 <== osd.0 v2:172.21.15.103:6802/3157697740 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb98c0af9f0 con 0x7fb98c0ace80 2024-09-19T06:48:00.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.038+0000 7fb9a62a9640 1 -- 172.21.15.103:0/4099762966 --> [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb9a0062cc0 con 0x7fb98c0ace80 2024-09-19T06:48:00.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.038+0000 7fb99dffb640 1 -- 172.21.15.103:0/4099762966 <== osd.0 v2:172.21.15.103:6802/3157697740 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb9a0062cc0 con 0x7fb98c0ace80 2024-09-19T06:48:00.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.038+0000 7fb9a62a9640 1 -- 172.21.15.103:0/4099762966 >> [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740] conn(0x7fb98c0ace80 msgr2=0x7fb98c0af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:00.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.038+0000 7fb9a62a9640 1 --2- 172.21.15.103:0/4099762966 >> [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740] conn(0x7fb98c0ace80 0x7fb98c0af320 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:00.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.038+0000 7fb9a62a9640 1 -- 172.21.15.103:0/4099762966 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb98c0a32d0 msgr2=0x7fb98c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:00.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.038+0000 7fb9a62a9640 1 --2- 172.21.15.103:0/4099762966 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb98c0a32d0 0x7fb98c0a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb990009930 tx=0x7fb990008040 comp rx=0 tx=0).stop 2024-09-19T06:48:00.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.038+0000 7fb9a62a9640 1 -- 172.21.15.103:0/4099762966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9a01027a0 msgr2=0x7fb9a0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:00.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.038+0000 7fb9a62a9640 1 --2- 172.21.15.103:0/4099762966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9a01027a0 0x7fb9a0074a30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb988005ab0 tx=0x7fb9880042f0 comp rx=0 tx=0).stop 2024-09-19T06:48:00.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.038+0000 7fb9a4846640 1 -- 172.21.15.103:0/4099762966 reap_dead start 2024-09-19T06:48:00.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.038+0000 7fb9a62a9640 1 -- 172.21.15.103:0/4099762966 shutdown_connections 2024-09-19T06:48:00.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.038+0000 7fb9a62a9640 1 -- 172.21.15.103:0/4099762966 >> 172.21.15.103:0/4099762966 conn(0x7fb9a00fd8b0 msgr2=0x7fb9a00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:00.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.039+0000 7fb9a62a9640 1 -- 172.21.15.103:0/4099762966 shutdown_connections 2024-09-19T06:48:00.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.039+0000 7fb9a62a9640 1 -- 172.21.15.103:0/4099762966 wait complete. 2024-09-19T06:48:00.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.449+0000 7f463f855640 1 -- 172.21.15.103:0/818359726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4638103640 msgr2=0x7f463810bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:00.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.449+0000 7f463f855640 1 --2- 172.21.15.103:0/818359726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4638103640 0x7f463810bb30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4628009900 tx=0x7f462802a480 comp rx=0 tx=0).stop 2024-09-19T06:48:00.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.449+0000 7f463f855640 1 -- 172.21.15.103:0/818359726 shutdown_connections 2024-09-19T06:48:00.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.449+0000 7f463f855640 1 --2- 172.21.15.103:0/818359726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f463810c150 0x7f463810e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:00.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.449+0000 7f463f855640 1 --2- 172.21.15.103:0/818359726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4638103640 0x7f463810bb30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:00.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.449+0000 7f463f855640 1 --2- 172.21.15.103:0/818359726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4638102d20 0x7f4638103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:00.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.449+0000 7f463f855640 1 -- 172.21.15.103:0/818359726 >> 172.21.15.103:0/818359726 conn(0x7f46380fc780 msgr2=0x7f46380feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:00.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.449+0000 7f463f855640 1 -- 172.21.15.103:0/818359726 shutdown_connections 2024-09-19T06:48:00.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.449+0000 7f463f855640 1 -- 172.21.15.103:0/818359726 wait complete. 2024-09-19T06:48:00.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463f855640 1 Processor -- start 2024-09-19T06:48:00.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463f855640 1 -- start start 2024-09-19T06:48:00.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463f855640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4638102d20 0x7f46381984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:00.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463f855640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4638103640 0x7f46381989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:00.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463f855640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f463810c150 0x7f463819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:00.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463f855640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4638111f50 con 0x7f4638103640 2024-09-19T06:48:00.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463f855640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4638111dd0 con 0x7f463810c150 2024-09-19T06:48:00.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463f855640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46381120d0 con 0x7f4638102d20 2024-09-19T06:48:00.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463cdf0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4638103640 0x7f46381989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463cdf0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4638103640 0x7f46381989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36644/0 (socket says 172.21.15.103:36644) 2024-09-19T06:48:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463cdf0640 1 -- 172.21.15.103:0/218161319 learned_addr learned my addr 172.21.15.103:0/218161319 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463ddf2640 1 --2- 172.21.15.103:0/218161319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f463810c150 0x7f463819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463d5f1640 1 --2- 172.21.15.103:0/218161319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4638102d20 0x7f46381984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463cdf0640 1 -- 172.21.15.103:0/218161319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4638102d20 msgr2=0x7f46381984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463cdf0640 1 --2- 172.21.15.103:0/218161319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4638102d20 0x7f46381984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463cdf0640 1 -- 172.21.15.103:0/218161319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f463810c150 msgr2=0x7f463819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463cdf0640 1 --2- 172.21.15.103:0/218161319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f463810c150 0x7f463819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.450+0000 7f463cdf0640 1 -- 172.21.15.103:0/218161319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463819d450 con 0x7f4638103640 2024-09-19T06:48:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.451+0000 7f463cdf0640 1 --2- 172.21.15.103:0/218161319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4638103640 0x7f46381989e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4628005ab0 tx=0x7f46280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.451+0000 7f462e7fc640 1 -- 172.21.15.103:0/218161319 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462803f030 con 0x7f4638103640 2024-09-19T06:48:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.451+0000 7f462e7fc640 1 -- 172.21.15.103:0/218161319 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4628003ce0 con 0x7f4638103640 2024-09-19T06:48:00.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.451+0000 7f463f855640 1 -- 172.21.15.103:0/218161319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463819d6e0 con 0x7f4638103640 2024-09-19T06:48:00.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.451+0000 7f462e7fc640 1 -- 172.21.15.103:0/218161319 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628035070 con 0x7f4638103640 2024-09-19T06:48:00.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.451+0000 7f463f855640 1 -- 172.21.15.103:0/218161319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46381a5020 con 0x7f4638103640 2024-09-19T06:48:00.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.452+0000 7f463f855640 1 -- 172.21.15.103:0/218161319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4638062900 con 0x7f4638103640 2024-09-19T06:48:00.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.453+0000 7f462e7fc640 1 -- 172.21.15.103:0/218161319 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4628042030 con 0x7f4638103640 2024-09-19T06:48:00.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.453+0000 7f462e7fc640 1 --2- 172.21.15.103:0/218161319 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f46240a32d0 0x7f46240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:00.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.454+0000 7f463d5f1640 1 --2- 172.21.15.103:0/218161319 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f46240a32d0 0x7f46240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:00.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.454+0000 7f462e7fc640 1 -- 172.21.15.103:0/218161319 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f46280f0600 con 0x7f4638103640 2024-09-19T06:48:00.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.454+0000 7f462e7fc640 1 --2- 172.21.15.103:0/218161319 >> [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394] conn(0x7f46240ace80 0x7f46240af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:00.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.454+0000 7f462e7fc640 1 -- 172.21.15.103:0/218161319 --> [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4628001d30 con 0x7f46240ace80 2024-09-19T06:48:00.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.454+0000 7f462e7fc640 1 -- 172.21.15.103:0/218161319 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f46280f0a50 con 0x7f4638103640 2024-09-19T06:48:00.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.454+0000 7f463ddf2640 1 --2- 172.21.15.103:0/218161319 >> [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394] conn(0x7f46240ace80 0x7f46240af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:00.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.454+0000 7f463d5f1640 1 --2- 172.21.15.103:0/218161319 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f46240a32d0 0x7f46240a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4634005e10 tx=0x7f4634005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:00.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.454+0000 7f463ddf2640 1 --2- 172.21.15.103:0/218161319 >> [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394] conn(0x7f46240ace80 0x7f46240af320 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:00.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.455+0000 7f462e7fc640 1 -- 172.21.15.103:0/218161319 <== osd.6 v2:172.21.15.103:6818/1396972394 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4628001d30 con 0x7f46240ace80 2024-09-19T06:48:00.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.468+0000 7f463f855640 1 -- 172.21.15.103:0/218161319 --> [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4638062d80 con 0x7f46240ace80 2024-09-19T06:48:00.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.468+0000 7f462e7fc640 1 -- 172.21.15.103:0/218161319 <== osd.6 v2:172.21.15.103:6818/1396972394 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4638062d80 con 0x7f46240ace80 2024-09-19T06:48:00.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.469+0000 7f463f855640 1 -- 172.21.15.103:0/218161319 >> [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394] conn(0x7f46240ace80 msgr2=0x7f46240af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:00.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.469+0000 7f463f855640 1 --2- 172.21.15.103:0/218161319 >> [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394] conn(0x7f46240ace80 0x7f46240af320 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:00.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.469+0000 7f463f855640 1 -- 172.21.15.103:0/218161319 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f46240a32d0 msgr2=0x7f46240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:00.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.469+0000 7f463f855640 1 --2- 172.21.15.103:0/218161319 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f46240a32d0 0x7f46240a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4634005e10 tx=0x7f4634005da0 comp rx=0 tx=0).stop 2024-09-19T06:48:00.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.469+0000 7f463f855640 1 -- 172.21.15.103:0/218161319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4638103640 msgr2=0x7f46381989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:00.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.469+0000 7f463f855640 1 --2- 172.21.15.103:0/218161319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4638103640 0x7f46381989e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4628005ab0 tx=0x7f46280042f0 comp rx=0 tx=0).stop 2024-09-19T06:48:00.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.469+0000 7f463ddf2640 1 -- 172.21.15.103:0/218161319 reap_dead start 2024-09-19T06:48:00.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.469+0000 7f463f855640 1 -- 172.21.15.103:0/218161319 shutdown_connections 2024-09-19T06:48:00.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.469+0000 7f463f855640 1 -- 172.21.15.103:0/218161319 >> 172.21.15.103:0/218161319 conn(0x7f46380fc780 msgr2=0x7f46380fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:00.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.469+0000 7f463f855640 1 -- 172.21.15.103:0/218161319 shutdown_connections 2024-09-19T06:48:00.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:00.469+0000 7f463f855640 1 -- 172.21.15.103:0/218161319 wait complete. 2024-09-19T06:48:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:01.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:00 smithi103 ceph-mon[31158]: pgmap v292: 97 pgs: 97 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-19T06:48:01.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:00 smithi134 ceph-mon[35356]: pgmap v292: 97 pgs: 97 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-19T06:48:01.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:00 smithi174 ceph-mon[35637]: pgmap v292: 97 pgs: 97 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-19T06:48:01.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.472+0000 7ffbcec2e640 1 -- 172.21.15.103:0/2559300517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc8101240 msgr2=0x7ffbc8101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:01.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.472+0000 7ffbcec2e640 1 --2- 172.21.15.103:0/2559300517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc8101240 0x7ffbc8101680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009900 tx=0x7ffbb402a410 comp rx=0 tx=0).stop 2024-09-19T06:48:01.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.472+0000 7ffbcec2e640 1 -- 172.21.15.103:0/2559300517 shutdown_connections 2024-09-19T06:48:01.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.472+0000 7ffbcec2e640 1 --2- 172.21.15.103:0/2559300517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc8101bc0 0x7ffbc810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:01.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.472+0000 7ffbcec2e640 1 --2- 172.21.15.103:0/2559300517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc8101240 0x7ffbc8101680 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:01.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.472+0000 7ffbcec2e640 1 --2- 172.21.15.103:0/2559300517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbc8105a90 0x7ffbc8105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:01.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.472+0000 7ffbcec2e640 1 -- 172.21.15.103:0/2559300517 >> 172.21.15.103:0/2559300517 conn(0x7ffbc8077210 msgr2=0x7ffbc80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:01.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.472+0000 7ffbcec2e640 1 -- 172.21.15.103:0/2559300517 shutdown_connections 2024-09-19T06:48:01.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.472+0000 7ffbcec2e640 1 -- 172.21.15.103:0/2559300517 wait complete. 2024-09-19T06:48:01.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcec2e640 1 Processor -- start 2024-09-19T06:48:01.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcec2e640 1 -- start start 2024-09-19T06:48:01.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcec2e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc8101240 0x7ffbc81a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:01.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcec2e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbc8101bc0 0x7ffbc81a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:01.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcec2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc8105a90 0x7ffbc819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:01.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcec2e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc8111030 con 0x7ffbc8105a90 2024-09-19T06:48:01.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcec2e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc8110eb0 con 0x7ffbc8101240 2024-09-19T06:48:01.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcec2e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc81111b0 con 0x7ffbc8101bc0 2024-09-19T06:48:01.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcd1cb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc8105a90 0x7ffbc819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:01.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcc9ca640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc8101240 0x7ffbc81a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:01.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcd1cb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc8105a90 0x7ffbc819a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36654/0 (socket says 172.21.15.103:36654) 2024-09-19T06:48:01.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcd1cb640 1 -- 172.21.15.103:0/3022841543 learned_addr learned my addr 172.21.15.103:0/3022841543 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:01.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbc7fff640 1 --2- 172.21.15.103:0/3022841543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbc8101bc0 0x7ffbc81a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:01.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcd1cb640 1 -- 172.21.15.103:0/3022841543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbc8101bc0 msgr2=0x7ffbc81a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:01.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcd1cb640 1 --2- 172.21.15.103:0/3022841543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbc8101bc0 0x7ffbc81a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:01.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcd1cb640 1 -- 172.21.15.103:0/3022841543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc8101240 msgr2=0x7ffbc81a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:01.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcd1cb640 1 --2- 172.21.15.103:0/3022841543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbc8101240 0x7ffbc81a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:01.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.473+0000 7ffbcd1cb640 1 -- 172.21.15.103:0/3022841543 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc819af50 con 0x7ffbc8105a90 2024-09-19T06:48:01.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.474+0000 7ffbcd1cb640 1 --2- 172.21.15.103:0/3022841543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc8105a90 0x7ffbc819a720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffbc000b970 tx=0x7ffbc000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:01.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.474+0000 7ffbc5ffb640 1 -- 172.21.15.103:0/3022841543 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0019070 con 0x7ffbc8105a90 2024-09-19T06:48:01.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.474+0000 7ffbc5ffb640 1 -- 172.21.15.103:0/3022841543 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc0014440 con 0x7ffbc8105a90 2024-09-19T06:48:01.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.474+0000 7ffbcec2e640 1 -- 172.21.15.103:0/3022841543 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc819b240 con 0x7ffbc8105a90 2024-09-19T06:48:01.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.474+0000 7ffbc5ffb640 1 -- 172.21.15.103:0/3022841543 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00075d0 con 0x7ffbc8105a90 2024-09-19T06:48:01.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.474+0000 7ffbcec2e640 1 -- 172.21.15.103:0/3022841543 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc81a75b0 con 0x7ffbc8105a90 2024-09-19T06:48:01.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.475+0000 7ffbcec2e640 1 -- 172.21.15.103:0/3022841543 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffbc819b580 con 0x7ffbc8105a90 2024-09-19T06:48:01.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.476+0000 7ffbc5ffb640 1 -- 172.21.15.103:0/3022841543 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ffbc001c070 con 0x7ffbc8105a90 2024-09-19T06:48:01.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.477+0000 7ffbc5ffb640 1 --2- 172.21.15.103:0/3022841543 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ffbb80a3350 0x7ffbb80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:01.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.477+0000 7ffbcc9ca640 1 --2- 172.21.15.103:0/3022841543 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ffbb80a3350 0x7ffbb80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:01.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.477+0000 7ffbc5ffb640 1 -- 172.21.15.103:0/3022841543 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ffbc00cf670 con 0x7ffbc8105a90 2024-09-19T06:48:01.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.477+0000 7ffbc5ffb640 1 --2- 172.21.15.103:0/3022841543 >> [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290] conn(0x7ffbb80acf00 0x7ffbb80af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:01.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.477+0000 7ffbc5ffb640 1 -- 172.21.15.103:0/3022841543 --> [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffbb80afa70 con 0x7ffbb80acf00 2024-09-19T06:48:01.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.477+0000 7ffbc5ffb640 1 -- 172.21.15.103:0/3022841543 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7ffbc00cfac0 con 0x7ffbc8105a90 2024-09-19T06:48:01.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.477+0000 7ffbcc9ca640 1 --2- 172.21.15.103:0/3022841543 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ffbb80a3350 0x7ffbb80a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009990 tx=0x7ffbbc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:01.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.477+0000 7ffbc7fff640 1 --2- 172.21.15.103:0/3022841543 >> [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290] conn(0x7ffbb80acf00 0x7ffbb80af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:01.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.478+0000 7ffbc7fff640 1 --2- 172.21.15.103:0/3022841543 >> [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290] conn(0x7ffbb80acf00 0x7ffbb80af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:01.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.478+0000 7ffbc5ffb640 1 -- 172.21.15.103:0/3022841543 <== osd.2 v2:172.21.15.174:6800/2292242290 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffbb80afa70 con 0x7ffbb80acf00 2024-09-19T06:48:01.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.492+0000 7ffbcec2e640 1 -- 172.21.15.103:0/3022841543 --> [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffbc80fead0 con 0x7ffbb80acf00 2024-09-19T06:48:01.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.492+0000 7ffbc5ffb640 1 -- 172.21.15.103:0/3022841543 <== osd.2 v2:172.21.15.174:6800/2292242290 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffbc80fead0 con 0x7ffbb80acf00 2024-09-19T06:48:01.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.492+0000 7ffbcec2e640 1 -- 172.21.15.103:0/3022841543 >> [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290] conn(0x7ffbb80acf00 msgr2=0x7ffbb80af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:01.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.492+0000 7ffbcec2e640 1 --2- 172.21.15.103:0/3022841543 >> [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290] conn(0x7ffbb80acf00 0x7ffbb80af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:01.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.493+0000 7ffbcec2e640 1 -- 172.21.15.103:0/3022841543 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ffbb80a3350 msgr2=0x7ffbb80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:01.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.493+0000 7ffbcec2e640 1 --2- 172.21.15.103:0/3022841543 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ffbb80a3350 0x7ffbb80a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009990 tx=0x7ffbbc009110 comp rx=0 tx=0).stop 2024-09-19T06:48:01.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.493+0000 7ffbcec2e640 1 -- 172.21.15.103:0/3022841543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc8105a90 msgr2=0x7ffbc819a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:01.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.493+0000 7ffbcec2e640 1 --2- 172.21.15.103:0/3022841543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbc8105a90 0x7ffbc819a720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffbc000b970 tx=0x7ffbc000bd70 comp rx=0 tx=0).stop 2024-09-19T06:48:01.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.493+0000 7ffbcd1cb640 1 -- 172.21.15.103:0/3022841543 reap_dead start 2024-09-19T06:48:01.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.493+0000 7ffbcec2e640 1 -- 172.21.15.103:0/3022841543 shutdown_connections 2024-09-19T06:48:01.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.493+0000 7ffbcec2e640 1 -- 172.21.15.103:0/3022841543 >> 172.21.15.103:0/3022841543 conn(0x7ffbc8077210 msgr2=0x7ffbc81045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:01.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.493+0000 7ffbcec2e640 1 -- 172.21.15.103:0/3022841543 shutdown_connections 2024-09-19T06:48:01.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:01.493+0000 7ffbcec2e640 1 -- 172.21.15.103:0/3022841543 wait complete. 2024-09-19T06:48:02.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.052+0000 7f5ba76e7640 1 -- 172.21.15.103:0/2542808806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ba0100a80 msgr2=0x7f5ba0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:02.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.052+0000 7f5ba76e7640 1 --2- 172.21.15.103:0/2542808806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ba0100a80 0x7f5ba0100ee0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009930 tx=0x7f5b9402a420 comp rx=0 tx=0).stop 2024-09-19T06:48:02.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.052+0000 7f5ba76e7640 1 -- 172.21.15.103:0/2542808806 shutdown_connections 2024-09-19T06:48:02.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.052+0000 7f5ba76e7640 1 --2- 172.21.15.103:0/2542808806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ba0101420 0x7f5ba010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:02.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.052+0000 7f5ba76e7640 1 --2- 172.21.15.103:0/2542808806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ba0100a80 0x7f5ba0100ee0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:02.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.052+0000 7f5ba76e7640 1 --2- 172.21.15.103:0/2542808806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ba0106a80 0x7f5ba0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:02.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.052+0000 7f5ba76e7640 1 -- 172.21.15.103:0/2542808806 >> 172.21.15.103:0/2542808806 conn(0x7f5ba00fc7a0 msgr2=0x7f5ba00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:02.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.052+0000 7f5ba76e7640 1 -- 172.21.15.103:0/2542808806 shutdown_connections 2024-09-19T06:48:02.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.052+0000 7f5ba76e7640 1 -- 172.21.15.103:0/2542808806 wait complete. 2024-09-19T06:48:02.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.052+0000 7f5ba76e7640 1 Processor -- start 2024-09-19T06:48:02.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba76e7640 1 -- start start 2024-09-19T06:48:02.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba76e7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ba0100a80 0x7f5ba00fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:02.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba76e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ba0101420 0x7f5ba00fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:02.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba76e7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ba0106a80 0x7f5ba00fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:02.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba76e7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba0110ec0 con 0x7f5ba0101420 2024-09-19T06:48:02.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba76e7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba0110d40 con 0x7f5ba0106a80 2024-09-19T06:48:02.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba76e7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba0111040 con 0x7f5ba0100a80 2024-09-19T06:48:02.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba4c82640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ba0101420 0x7f5ba00fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:02.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba4c82640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ba0101420 0x7f5ba00fd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36682/0 (socket says 172.21.15.103:36682) 2024-09-19T06:48:02.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba4c82640 1 -- 172.21.15.103:0/1774549922 learned_addr learned my addr 172.21.15.103:0/1774549922 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:02.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba5c84640 1 --2- 172.21.15.103:0/1774549922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ba0106a80 0x7f5ba00fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:02.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba5483640 1 --2- 172.21.15.103:0/1774549922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ba0100a80 0x7f5ba00fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:02.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba4c82640 1 -- 172.21.15.103:0/1774549922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ba0100a80 msgr2=0x7f5ba00fcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:02.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba4c82640 1 --2- 172.21.15.103:0/1774549922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ba0100a80 0x7f5ba00fcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:02.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba4c82640 1 -- 172.21.15.103:0/1774549922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ba0106a80 msgr2=0x7f5ba00fd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:02.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba4c82640 1 --2- 172.21.15.103:0/1774549922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ba0106a80 0x7f5ba00fd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:02.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba4c82640 1 -- 172.21.15.103:0/1774549922 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba01a6960 con 0x7f5ba0101420 2024-09-19T06:48:02.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.053+0000 7f5ba4c82640 1 --2- 172.21.15.103:0/1774549922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ba0101420 0x7f5ba00fd280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009230 tx=0x7f5b94004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:02.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.054+0000 7f5b927fc640 1 -- 172.21.15.103:0/1774549922 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9403f030 con 0x7f5ba0101420 2024-09-19T06:48:02.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.054+0000 7f5ba76e7640 1 -- 172.21.15.103:0/1774549922 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba01a6bf0 con 0x7f5ba0101420 2024-09-19T06:48:02.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.054+0000 7f5b927fc640 1 -- 172.21.15.103:0/1774549922 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b94003ad0 con 0x7f5ba0101420 2024-09-19T06:48:02.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.054+0000 7f5b927fc640 1 -- 172.21.15.103:0/1774549922 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b94035070 con 0x7f5ba0101420 2024-09-19T06:48:02.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.054+0000 7f5ba76e7640 1 -- 172.21.15.103:0/1774549922 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba01a7120 con 0x7f5ba0101420 2024-09-19T06:48:02.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.054+0000 7f5ba76e7640 1 -- 172.21.15.103:0/1774549922 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ba00628c0 con 0x7f5ba0101420 2024-09-19T06:48:02.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.056+0000 7f5b927fc640 1 -- 172.21.15.103:0/1774549922 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5b94042030 con 0x7f5ba0101420 2024-09-19T06:48:02.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.056+0000 7f5b927fc640 1 --2- 172.21.15.103:0/1774549922 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5b8c0a32d0 0x7f5b8c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:02.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.056+0000 7f5ba5483640 1 --2- 172.21.15.103:0/1774549922 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5b8c0a32d0 0x7f5b8c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:02.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.056+0000 7f5b927fc640 1 -- 172.21.15.103:0/1774549922 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5b940f0300 con 0x7f5ba0101420 2024-09-19T06:48:02.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.057+0000 7f5b927fc640 1 --2- 172.21.15.103:0/1774549922 >> [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968] conn(0x7f5b8c0ace80 0x7f5b8c0af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:02.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.057+0000 7f5b927fc640 1 -- 172.21.15.103:0/1774549922 --> [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b9404eea0 con 0x7f5b8c0ace80 2024-09-19T06:48:02.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.057+0000 7f5b927fc640 1 -- 172.21.15.103:0/1774549922 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f5b940f0750 con 0x7f5ba0101420 2024-09-19T06:48:02.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.057+0000 7f5ba5483640 1 --2- 172.21.15.103:0/1774549922 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5b8c0a32d0 0x7f5b8c0a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c005fd0 tx=0x7f5b9c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:02.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.057+0000 7f5ba5c84640 1 --2- 172.21.15.103:0/1774549922 >> [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968] conn(0x7f5b8c0ace80 0x7f5b8c0af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:02.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.057+0000 7f5ba5c84640 1 --2- 172.21.15.103:0/1774549922 >> [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968] conn(0x7f5b8c0ace80 0x7f5b8c0af320 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:02.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.058+0000 7f5b927fc640 1 -- 172.21.15.103:0/1774549922 <== osd.4 v2:172.21.15.134:6808/3169891968 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5b9404eea0 con 0x7f5b8c0ace80 2024-09-19T06:48:02.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.071+0000 7f5ba76e7640 1 -- 172.21.15.103:0/1774549922 --> [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ba0062cf0 con 0x7f5b8c0ace80 2024-09-19T06:48:02.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.072+0000 7f5b927fc640 1 -- 172.21.15.103:0/1774549922 <== osd.4 v2:172.21.15.134:6808/3169891968 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5ba0062cf0 con 0x7f5b8c0ace80 2024-09-19T06:48:02.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.072+0000 7f5ba76e7640 1 -- 172.21.15.103:0/1774549922 >> [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968] conn(0x7f5b8c0ace80 msgr2=0x7f5b8c0af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:02.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.072+0000 7f5ba76e7640 1 --2- 172.21.15.103:0/1774549922 >> [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968] conn(0x7f5b8c0ace80 0x7f5b8c0af320 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:02.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.073+0000 7f5ba76e7640 1 -- 172.21.15.103:0/1774549922 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5b8c0a32d0 msgr2=0x7f5b8c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:02.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.073+0000 7f5ba76e7640 1 --2- 172.21.15.103:0/1774549922 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5b8c0a32d0 0x7f5b8c0a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c005fd0 tx=0x7f5b9c00b390 comp rx=0 tx=0).stop 2024-09-19T06:48:02.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.073+0000 7f5ba76e7640 1 -- 172.21.15.103:0/1774549922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ba0101420 msgr2=0x7f5ba00fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:02.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.073+0000 7f5ba76e7640 1 --2- 172.21.15.103:0/1774549922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ba0101420 0x7f5ba00fd280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009230 tx=0x7f5b94004270 comp rx=0 tx=0).stop 2024-09-19T06:48:02.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.073+0000 7f5ba5c84640 1 -- 172.21.15.103:0/1774549922 reap_dead start 2024-09-19T06:48:02.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.073+0000 7f5ba76e7640 1 -- 172.21.15.103:0/1774549922 shutdown_connections 2024-09-19T06:48:02.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.073+0000 7f5ba76e7640 1 -- 172.21.15.103:0/1774549922 >> 172.21.15.103:0/1774549922 conn(0x7f5ba00fc7a0 msgr2=0x7f5ba006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:02.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.073+0000 7f5ba76e7640 1 -- 172.21.15.103:0/1774549922 shutdown_connections 2024-09-19T06:48:02.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:02.073+0000 7f5ba76e7640 1 -- 172.21.15.103:0/1774549922 wait complete. 2024-09-19T06:48:02.835 INFO:teuthology.orchestra.run.smithi103.stdout:73014444097 2024-09-19T06:48:02.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T06:48:03.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:02 smithi103 ceph-mon[31158]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-19T06:48:03.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:02 smithi134 ceph-mon[35356]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-19T06:48:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:02 smithi174 ceph-mon[35637]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-19T06:48:03.409 INFO:teuthology.orchestra.run.smithi103.stdout:34359738442 2024-09-19T06:48:03.410 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T06:48:03.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.591+0000 7f8b351ea640 1 -- 172.21.15.103:0/166066746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b30102ef0 msgr2=0x7f8b3010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:03.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.591+0000 7f8b351ea640 1 --2- 172.21.15.103:0/166066746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b30102ef0 0x7f8b3010da40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009ca0 tx=0x7f8b2802a950 comp rx=0 tx=0).stop 2024-09-19T06:48:03.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.591+0000 7f8b351ea640 1 -- 172.21.15.103:0/166066746 shutdown_connections 2024-09-19T06:48:03.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.591+0000 7f8b351ea640 1 --2- 172.21.15.103:0/166066746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b30102ef0 0x7f8b3010da40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:03.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.591+0000 7f8b351ea640 1 --2- 172.21.15.103:0/166066746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b30102570 0x7f8b301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:03.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.591+0000 7f8b351ea640 1 --2- 172.21.15.103:0/166066746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b30100e20 0x7f8b30101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:03.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.591+0000 7f8b351ea640 1 -- 172.21.15.103:0/166066746 >> 172.21.15.103:0/166066746 conn(0x7f8b300fc7d0 msgr2=0x7f8b300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:03.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.591+0000 7f8b351ea640 1 -- 172.21.15.103:0/166066746 shutdown_connections 2024-09-19T06:48:03.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.591+0000 7f8b351ea640 1 -- 172.21.15.103:0/166066746 wait complete. 2024-09-19T06:48:03.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.592+0000 7f8b351ea640 1 Processor -- start 2024-09-19T06:48:03.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.592+0000 7f8b351ea640 1 -- start start 2024-09-19T06:48:03.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.592+0000 7f8b351ea640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b30100e20 0x7f8b301984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:03.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.592+0000 7f8b351ea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b30102570 0x7f8b301989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:03.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.592+0000 7f8b351ea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b30102ef0 0x7f8b3019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:03.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.592+0000 7f8b351ea640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8b30110fd0 con 0x7f8b30102ef0 2024-09-19T06:48:03.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.592+0000 7f8b351ea640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b30110e50 con 0x7f8b30102570 2024-09-19T06:48:03.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.592+0000 7f8b351ea640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8b30111150 con 0x7f8b30100e20 2024-09-19T06:48:03.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.592+0000 7f8b2f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b30102ef0 0x7f8b3019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:03.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.592+0000 7f8b2f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b30102ef0 0x7f8b3019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36694/0 (socket says 172.21.15.103:36694) 2024-09-19T06:48:03.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.592+0000 7f8b2f59e640 1 -- 172.21.15.103:0/2924025387 learned_addr learned my addr 172.21.15.103:0/2924025387 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:03.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.593+0000 7f8b2ed9d640 1 --2- 172.21.15.103:0/2924025387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b30100e20 0x7f8b301984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:03.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.593+0000 7f8b2e59c640 1 --2- 172.21.15.103:0/2924025387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b30102570 0x7f8b301989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:03.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.593+0000 7f8b2f59e640 1 -- 172.21.15.103:0/2924025387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b30100e20 msgr2=0x7f8b301984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:03.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.593+0000 7f8b2f59e640 1 --2- 172.21.15.103:0/2924025387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b30100e20 0x7f8b301984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:03.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.593+0000 7f8b2f59e640 1 -- 172.21.15.103:0/2924025387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b30102570 msgr2=0x7f8b301989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:03.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.593+0000 7f8b2f59e640 1 --2- 172.21.15.103:0/2924025387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b30102570 0x7f8b301989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:03.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.593+0000 7f8b2f59e640 1 -- 172.21.15.103:0/2924025387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3019d460 con 0x7f8b30102ef0 2024-09-19T06:48:03.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.593+0000 7f8b2e59c640 1 --2- 172.21.15.103:0/2924025387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b30102570 0x7f8b301989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:03.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.593+0000 7f8b2f59e640 1 --2- 172.21.15.103:0/2924025387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b30102ef0 0x7f8b3019cd80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009dc0 tx=0x7f8b28003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:03.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.593+0000 7f8b17fff640 1 -- 172.21.15.103:0/2924025387 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28034040 con 0x7f8b30102ef0 2024-09-19T06:48:03.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.594+0000 7f8b17fff640 1 -- 172.21.15.103:0/2924025387 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b28003e50 con 0x7f8b30102ef0 2024-09-19T06:48:03.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.594+0000 7f8b17fff640 1 -- 172.21.15.103:0/2924025387 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b280046c0 con 0x7f8b30102ef0 2024-09-19T06:48:03.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.594+0000 7f8b351ea640 1 -- 172.21.15.103:0/2924025387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3019d6f0 con 0x7f8b30102ef0 2024-09-19T06:48:03.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.594+0000 7f8b351ea640 1 -- 172.21.15.103:0/2924025387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b301a5030 con 0x7f8b30102ef0 2024-09-19T06:48:03.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.595+0000 7f8b351ea640 1 -- 172.21.15.103:0/2924025387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b30062980 con 0x7f8b30102ef0 2024-09-19T06:48:03.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.596+0000 7f8b17fff640 1 -- 172.21.15.103:0/2924025387 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8b28037020 con 0x7f8b30102ef0 2024-09-19T06:48:03.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.596+0000 7f8b17fff640 1 --2- 172.21.15.103:0/2924025387 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8b180a3350 0x7f8b180a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:03.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.597+0000 7f8b2ed9d640 1 --2- 172.21.15.103:0/2924025387 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8b180a3350 0x7f8b180a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:03.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.597+0000 7f8b17fff640 1 -- 172.21.15.103:0/2924025387 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8b280f08a0 con 0x7f8b30102ef0 2024-09-19T06:48:03.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.597+0000 7f8b17fff640 1 --2- 172.21.15.103:0/2924025387 >> [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765] conn(0x7f8b180acf00 0x7f8b180af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:03.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.597+0000 7f8b17fff640 1 -- 172.21.15.103:0/2924025387 --> [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b28033e50 con 0x7f8b180acf00 2024-09-19T06:48:03.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.597+0000 7f8b17fff640 1 -- 172.21.15.103:0/2924025387 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f8b280f0c50 con 0x7f8b30102ef0 2024-09-19T06:48:03.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.597+0000 7f8b2ed9d640 1 --2- 172.21.15.103:0/2924025387 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8b180a3350 0x7f8b180a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b200058c0 tx=0x7f8b2000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:03.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.597+0000 7f8b2e59c640 1 --2- 172.21.15.103:0/2924025387 >> [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765] conn(0x7f8b180acf00 0x7f8b180af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:03.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.597+0000 7f8b2e59c640 1 --2- 172.21.15.103:0/2924025387 >> [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765] conn(0x7f8b180acf00 0x7f8b180af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:03.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.598+0000 7f8b17fff640 1 -- 172.21.15.103:0/2924025387 <== osd.7 v2:172.21.15.134:6816/2196366765 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b28033e50 con 0x7f8b180acf00 2024-09-19T06:48:03.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.612+0000 7f8b351ea640 1 -- 172.21.15.103:0/2924025387 --> [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8b30062d80 con 0x7f8b180acf00 2024-09-19T06:48:03.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.613+0000 7f8b17fff640 1 -- 172.21.15.103:0/2924025387 <== osd.7 v2:172.21.15.134:6816/2196366765 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8b30062d80 con 0x7f8b180acf00 2024-09-19T06:48:03.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.613+0000 7f8b351ea640 1 -- 172.21.15.103:0/2924025387 >> [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765] conn(0x7f8b180acf00 msgr2=0x7f8b180af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:03.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.613+0000 7f8b351ea640 1 --2- 172.21.15.103:0/2924025387 >> [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765] conn(0x7f8b180acf00 0x7f8b180af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:03.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.613+0000 7f8b351ea640 1 -- 172.21.15.103:0/2924025387 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8b180a3350 msgr2=0x7f8b180a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:03.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.613+0000 7f8b351ea640 1 --2- 172.21.15.103:0/2924025387 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8b180a3350 0x7f8b180a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b200058c0 tx=0x7f8b2000a2d0 comp rx=0 tx=0).stop 2024-09-19T06:48:03.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.613+0000 7f8b351ea640 1 -- 172.21.15.103:0/2924025387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b30102ef0 msgr2=0x7f8b3019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:03.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.613+0000 7f8b351ea640 1 --2- 172.21.15.103:0/2924025387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b30102ef0 0x7f8b3019cd80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009dc0 tx=0x7f8b28003ce0 comp rx=0 tx=0).stop 2024-09-19T06:48:03.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.613+0000 7f8b2f59e640 1 -- 172.21.15.103:0/2924025387 reap_dead start 2024-09-19T06:48:03.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.614+0000 7f8b351ea640 1 -- 172.21.15.103:0/2924025387 shutdown_connections 2024-09-19T06:48:03.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.614+0000 7f8b351ea640 1 -- 172.21.15.103:0/2924025387 >> 172.21.15.103:0/2924025387 conn(0x7f8b300fc7d0 msgr2=0x7f8b3006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:03.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.614+0000 7f8b351ea640 1 -- 172.21.15.103:0/2924025387 shutdown_connections 2024-09-19T06:48:03.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:03.614+0000 7f8b351ea640 1 -- 172.21.15.103:0/2924025387 wait complete. 2024-09-19T06:48:04.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.383+0000 7f9f95113640 1 -- 172.21.15.103:0/2945306438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f90102ef0 msgr2=0x7f9f9010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.383+0000 7f9f95113640 1 --2- 172.21.15.103:0/2945306438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f90102ef0 0x7f9f9010da40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009ca0 tx=0x7f9f8802a950 comp rx=0 tx=0).stop 2024-09-19T06:48:04.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.383+0000 7f9f95113640 1 -- 172.21.15.103:0/2945306438 shutdown_connections 2024-09-19T06:48:04.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.383+0000 7f9f95113640 1 --2- 172.21.15.103:0/2945306438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f90102ef0 0x7f9f9010da40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.383+0000 7f9f95113640 1 --2- 172.21.15.103:0/2945306438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f90102570 0x7f9f901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.383+0000 7f9f95113640 1 --2- 172.21.15.103:0/2945306438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f90100e20 0x7f9f90101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.383+0000 7f9f95113640 1 -- 172.21.15.103:0/2945306438 >> 172.21.15.103:0/2945306438 conn(0x7f9f900fc7d0 msgr2=0x7f9f900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:04.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.383+0000 7f9f95113640 1 -- 172.21.15.103:0/2945306438 shutdown_connections 2024-09-19T06:48:04.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.384+0000 7f9f95113640 1 -- 172.21.15.103:0/2945306438 wait complete. 2024-09-19T06:48:04.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.384+0000 7f9f95113640 1 Processor -- start 2024-09-19T06:48:04.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f95113640 1 -- start start 2024-09-19T06:48:04.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f95113640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f90100e20 0x7f9f901992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:04.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f95113640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f90102570 0x7f9f90199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:04.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f95113640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f90102ef0 0x7f9f901a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:04.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f95113640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f90110f80 con 0x7f9f90102ef0 2024-09-19T06:48:04.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f95113640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9f90110e00 con 0x7f9f90102570 2024-09-19T06:48:04.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f95113640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9f90111100 con 0x7f9f90100e20 2024-09-19T06:48:04.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f8f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f90102ef0 0x7f9f901a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:04.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f8f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f90102ef0 0x7f9f901a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36712/0 (socket says 172.21.15.103:36712) 2024-09-19T06:48:04.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f8f59e640 1 -- 172.21.15.103:0/55135293 learned_addr learned my addr 172.21.15.103:0/55135293 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:04.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f8e59c640 1 --2- 172.21.15.103:0/55135293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f90102570 0x7f9f90199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:04.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f8ed9d640 1 --2- 172.21.15.103:0/55135293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f90100e20 0x7f9f901992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:04.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f8e59c640 1 -- 172.21.15.103:0/55135293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f90100e20 msgr2=0x7f9f901992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f8e59c640 1 --2- 172.21.15.103:0/55135293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f90100e20 0x7f9f901992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f8e59c640 1 -- 172.21.15.103:0/55135293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f90102ef0 msgr2=0x7f9f901a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f8e59c640 1 --2- 172.21.15.103:0/55135293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f90102ef0 0x7f9f901a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f8e59c640 1 -- 172.21.15.103:0/55135293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f901a4890 con 0x7f9f90102570 2024-09-19T06:48:04.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.385+0000 7f9f8f59e640 1 --2- 172.21.15.103:0/55135293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f90102ef0 0x7f9f901a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:04.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.386+0000 7f9f8e59c640 1 --2- 172.21.15.103:0/55135293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f90102570 0x7f9f90199810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800d6d0 tx=0x7f9f7800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:04.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.386+0000 7f9f87fff640 1 -- 172.21.15.103:0/55135293 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78017070 con 0x7f9f90102570 2024-09-19T06:48:04.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.386+0000 7f9f95113640 1 -- 172.21.15.103:0/55135293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f901a4a60 con 0x7f9f90102570 2024-09-19T06:48:04.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.386+0000 7f9f87fff640 1 -- 172.21.15.103:0/55135293 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78009820 con 0x7f9f90102570 2024-09-19T06:48:04.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.386+0000 7f9f87fff640 1 -- 172.21.15.103:0/55135293 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78004cd0 con 0x7f9f90102570 2024-09-19T06:48:04.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.386+0000 7f9f95113640 1 -- 172.21.15.103:0/55135293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f901a4f70 con 0x7f9f90102570 2024-09-19T06:48:04.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.387+0000 7f9f95113640 1 -- 172.21.15.103:0/55135293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f90107280 con 0x7f9f90102570 2024-09-19T06:48:04.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.388+0000 7f9f87fff640 1 -- 172.21.15.103:0/55135293 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9f78005220 con 0x7f9f90102570 2024-09-19T06:48:04.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.389+0000 7f9f87fff640 1 --2- 172.21.15.103:0/55135293 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9f7c0a32d0 0x7f9f7c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:04.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.389+0000 7f9f8ed9d640 1 --2- 172.21.15.103:0/55135293 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9f7c0a32d0 0x7f9f7c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:04.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.389+0000 7f9f87fff640 1 -- 172.21.15.103:0/55135293 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9f780cfaf0 con 0x7f9f90102570 2024-09-19T06:48:04.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.389+0000 7f9f87fff640 1 --2- 172.21.15.103:0/55135293 >> [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632] conn(0x7f9f7c0ace80 0x7f9f7c0af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:04.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.389+0000 7f9f87fff640 1 -- 172.21.15.103:0/55135293 --> [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f7801ee70 con 0x7f9f7c0ace80 2024-09-19T06:48:04.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.389+0000 7f9f87fff640 1 -- 172.21.15.103:0/55135293 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f9f780cfea0 con 0x7f9f90102570 2024-09-19T06:48:04.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.389+0000 7f9f8ed9d640 1 --2- 172.21.15.103:0/55135293 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9f7c0a32d0 0x7f9f7c0a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000bbe0 tx=0x7f9f8000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:04.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.390+0000 7f9f8f59e640 1 --2- 172.21.15.103:0/55135293 >> [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632] conn(0x7f9f7c0ace80 0x7f9f7c0af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:04.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.390+0000 7f9f8f59e640 1 --2- 172.21.15.103:0/55135293 >> [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632] conn(0x7f9f7c0ace80 0x7f9f7c0af320 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:04.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.391+0000 7f9f87fff640 1 -- 172.21.15.103:0/55135293 <== osd.10 v2:172.21.15.134:6824/2532758632 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f7801ee70 con 0x7f9f7c0ace80 2024-09-19T06:48:04.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.404+0000 7f9f95113640 1 -- 172.21.15.103:0/55135293 --> [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9f901021b0 con 0x7f9f7c0ace80 2024-09-19T06:48:04.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.405+0000 7f9f87fff640 1 -- 172.21.15.103:0/55135293 <== osd.10 v2:172.21.15.134:6824/2532758632 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9f901021b0 con 0x7f9f7c0ace80 2024-09-19T06:48:04.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.405+0000 7f9f95113640 1 -- 172.21.15.103:0/55135293 >> [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632] conn(0x7f9f7c0ace80 msgr2=0x7f9f7c0af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.405+0000 7f9f95113640 1 --2- 172.21.15.103:0/55135293 >> [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632] conn(0x7f9f7c0ace80 0x7f9f7c0af320 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.406+0000 7f9f95113640 1 -- 172.21.15.103:0/55135293 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9f7c0a32d0 msgr2=0x7f9f7c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.406+0000 7f9f95113640 1 --2- 172.21.15.103:0/55135293 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f9f7c0a32d0 0x7f9f7c0a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000bbe0 tx=0x7f9f8000b390 comp rx=0 tx=0).stop 2024-09-19T06:48:04.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.406+0000 7f9f95113640 1 -- 172.21.15.103:0/55135293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f90102570 msgr2=0x7f9f90199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.406+0000 7f9f95113640 1 --2- 172.21.15.103:0/55135293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f90102570 0x7f9f90199810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800d6d0 tx=0x7f9f7800db90 comp rx=0 tx=0).stop 2024-09-19T06:48:04.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.406+0000 7f9f8f59e640 1 -- 172.21.15.103:0/55135293 reap_dead start 2024-09-19T06:48:04.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.406+0000 7f9f95113640 1 -- 172.21.15.103:0/55135293 shutdown_connections 2024-09-19T06:48:04.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.406+0000 7f9f95113640 1 -- 172.21.15.103:0/55135293 >> 172.21.15.103:0/55135293 conn(0x7f9f900fc7d0 msgr2=0x7f9f90106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:04.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.406+0000 7f9f95113640 1 -- 172.21.15.103:0/55135293 shutdown_connections 2024-09-19T06:48:04.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.406+0000 7f9f95113640 1 -- 172.21.15.103:0/55135293 wait complete. 2024-09-19T06:48:04.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.607+0000 7f03c8f41640 1 -- 172.21.15.103:0/3086886183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c4106a80 msgr2=0x7f03c4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.607+0000 7f03c8f41640 1 --2- 172.21.15.103:0/3086886183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c4106a80 0x7f03c4106e60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009930 tx=0x7f03b802a420 comp rx=0 tx=0).stop 2024-09-19T06:48:04.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.607+0000 7f03c8f41640 1 -- 172.21.15.103:0/3086886183 shutdown_connections 2024-09-19T06:48:04.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.607+0000 7f03c8f41640 1 --2- 172.21.15.103:0/3086886183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c4101420 0x7f03c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.607+0000 7f03c8f41640 1 --2- 172.21.15.103:0/3086886183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03c4100a80 0x7f03c4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.607+0000 7f03c8f41640 1 --2- 172.21.15.103:0/3086886183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c4106a80 0x7f03c4106e60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.607+0000 7f03c8f41640 1 -- 172.21.15.103:0/3086886183 >> 172.21.15.103:0/3086886183 conn(0x7f03c40fc7a0 msgr2=0x7f03c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:04.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.607+0000 7f03c8f41640 1 -- 172.21.15.103:0/3086886183 shutdown_connections 2024-09-19T06:48:04.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.608+0000 7f03c8f41640 1 -- 172.21.15.103:0/3086886183 wait complete. 2024-09-19T06:48:04.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.608+0000 7f03c8f41640 1 Processor -- start 2024-09-19T06:48:04.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.608+0000 7f03c8f41640 1 -- start start 2024-09-19T06:48:04.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c8f41640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c4100a80 0x7f03c4199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:04.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c8f41640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c4101420 0x7f03c4199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:04.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c8f41640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03c4106a80 0x7f03c41a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:04.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c8f41640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03c4110e40 con 0x7f03c4100a80 2024-09-19T06:48:04.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c8f41640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f03c4110cc0 con 0x7f03c4101420 2024-09-19T06:48:04.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c8f41640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f03c4110fc0 con 0x7f03c4106a80 2024-09-19T06:48:04.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c4100a80 0x7f03c4199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:04.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c4100a80 0x7f03c4199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36722/0 (socket says 172.21.15.103:36722) 2024-09-19T06:48:04.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c259c640 1 -- 172.21.15.103:0/3475262780 learned_addr learned my addr 172.21.15.103:0/3475262780 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:04.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c1d9b640 1 --2- 172.21.15.103:0/3475262780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c4101420 0x7f03c4199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:04.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c2d9d640 1 --2- 172.21.15.103:0/3475262780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03c4106a80 0x7f03c41a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:04.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c259c640 1 -- 172.21.15.103:0/3475262780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03c4106a80 msgr2=0x7f03c41a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c259c640 1 --2- 172.21.15.103:0/3475262780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03c4106a80 0x7f03c41a42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c259c640 1 -- 172.21.15.103:0/3475262780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c4101420 msgr2=0x7f03c4199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c259c640 1 --2- 172.21.15.103:0/3475262780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c4101420 0x7f03c4199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c259c640 1 -- 172.21.15.103:0/3475262780 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c41a4810 con 0x7f03c4100a80 2024-09-19T06:48:04.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c259c640 1 --2- 172.21.15.103:0/3475262780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c4100a80 0x7f03c4199250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009230 tx=0x7f03b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:04.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03b37fe640 1 -- 172.21.15.103:0/3475262780 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b803f030 con 0x7f03c4100a80 2024-09-19T06:48:04.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03c8f41640 1 -- 172.21.15.103:0/3475262780 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c41a49e0 con 0x7f03c4100a80 2024-09-19T06:48:04.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03b37fe640 1 -- 172.21.15.103:0/3475262780 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03b8003ad0 con 0x7f03c4100a80 2024-09-19T06:48:04.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.609+0000 7f03b37fe640 1 -- 172.21.15.103:0/3475262780 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8035070 con 0x7f03c4100a80 2024-09-19T06:48:04.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.610+0000 7f03c8f41640 1 -- 172.21.15.103:0/3475262780 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c41a4e30 con 0x7f03c4100a80 2024-09-19T06:48:04.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.610+0000 7f03c8f41640 1 -- 172.21.15.103:0/3475262780 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f03c4062900 con 0x7f03c4100a80 2024-09-19T06:48:04.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.612+0000 7f03b37fe640 1 -- 172.21.15.103:0/3475262780 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f03b8042030 con 0x7f03c4100a80 2024-09-19T06:48:04.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.612+0000 7f03b37fe640 1 --2- 172.21.15.103:0/3475262780 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f03ac0a32d0 0x7f03ac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:04.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.613+0000 7f03c1d9b640 1 --2- 172.21.15.103:0/3475262780 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f03ac0a32d0 0x7f03ac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:04.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.613+0000 7f03b37fe640 1 -- 172.21.15.103:0/3475262780 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f03b80eff80 con 0x7f03c4100a80 2024-09-19T06:48:04.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.613+0000 7f03b37fe640 1 --2- 172.21.15.103:0/3475262780 >> [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413] conn(0x7f03ac0ace80 0x7f03ac0af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:04.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.613+0000 7f03b37fe640 1 -- 172.21.15.103:0/3475262780 --> [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03b8044e50 con 0x7f03ac0ace80 2024-09-19T06:48:04.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.613+0000 7f03b37fe640 1 -- 172.21.15.103:0/3475262780 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f03b80f03d0 con 0x7f03c4100a80 2024-09-19T06:48:04.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.613+0000 7f03c1d9b640 1 --2- 172.21.15.103:0/3475262780 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f03ac0a32d0 0x7f03ac0a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f03c406b2e0 tx=0x7f03b400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:04.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.613+0000 7f03c2d9d640 1 --2- 172.21.15.103:0/3475262780 >> [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413] conn(0x7f03ac0ace80 0x7f03ac0af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:04.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.613+0000 7f03c2d9d640 1 --2- 172.21.15.103:0/3475262780 >> [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413] conn(0x7f03ac0ace80 0x7f03ac0af320 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:04.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.614+0000 7f03b37fe640 1 -- 172.21.15.103:0/3475262780 <== osd.9 v2:172.21.15.103:6826/3161966413 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03b8044e50 con 0x7f03ac0ace80 2024-09-19T06:48:04.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.641+0000 7f03c8f41640 1 -- 172.21.15.103:0/3475262780 --> [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f03c4062d80 con 0x7f03ac0ace80 2024-09-19T06:48:04.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.641+0000 7f03b37fe640 1 -- 172.21.15.103:0/3475262780 <== osd.9 v2:172.21.15.103:6826/3161966413 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f03c4062d80 con 0x7f03ac0ace80 2024-09-19T06:48:04.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.642+0000 7f03c8f41640 1 -- 172.21.15.103:0/3475262780 >> [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413] conn(0x7f03ac0ace80 msgr2=0x7f03ac0af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.642+0000 7f03c8f41640 1 --2- 172.21.15.103:0/3475262780 >> [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413] conn(0x7f03ac0ace80 0x7f03ac0af320 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.642+0000 7f03c8f41640 1 -- 172.21.15.103:0/3475262780 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f03ac0a32d0 msgr2=0x7f03ac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.642+0000 7f03c8f41640 1 --2- 172.21.15.103:0/3475262780 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f03ac0a32d0 0x7f03ac0a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f03c406b2e0 tx=0x7f03b400a2d0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.642+0000 7f03c8f41640 1 -- 172.21.15.103:0/3475262780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c4100a80 msgr2=0x7f03c4199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.642+0000 7f03c8f41640 1 --2- 172.21.15.103:0/3475262780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c4100a80 0x7f03c4199250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009230 tx=0x7f03b8004270 comp rx=0 tx=0).stop 2024-09-19T06:48:04.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.642+0000 7f03c2d9d640 1 -- 172.21.15.103:0/3475262780 reap_dead start 2024-09-19T06:48:04.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.642+0000 7f03c8f41640 1 -- 172.21.15.103:0/3475262780 shutdown_connections 2024-09-19T06:48:04.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.642+0000 7f03c8f41640 1 -- 172.21.15.103:0/3475262780 >> 172.21.15.103:0/3475262780 conn(0x7f03c40fc7a0 msgr2=0x7f03c40fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:04.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.642+0000 7f03c8f41640 1 -- 172.21.15.103:0/3475262780 shutdown_connections 2024-09-19T06:48:04.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.642+0000 7f03c8f41640 1 -- 172.21.15.103:0/3475262780 wait complete. 2024-09-19T06:48:04.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.811+0000 7f83beb32640 1 -- 172.21.15.103:0/3330281347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b81082e0 msgr2=0x7f83b810ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.811+0000 7f83beb32640 1 --2- 172.21.15.103:0/3330281347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b81082e0 0x7f83b810ec50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009cd0 tx=0x7f83b002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.811+0000 7f83beb32640 1 -- 172.21.15.103:0/3330281347 shutdown_connections 2024-09-19T06:48:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.811+0000 7f83beb32640 1 --2- 172.21.15.103:0/3330281347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b81082e0 0x7f83b810ec50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.811+0000 7f83beb32640 1 --2- 172.21.15.103:0/3330281347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83b8103760 0x7f83b8107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.811+0000 7f83beb32640 1 --2- 172.21.15.103:0/3330281347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83b8102db0 0x7f83b8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.811+0000 7f83beb32640 1 -- 172.21.15.103:0/3330281347 >> 172.21.15.103:0/3330281347 conn(0x7f83b80fc7d0 msgr2=0x7f83b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.811+0000 7f83beb32640 1 -- 172.21.15.103:0/3330281347 shutdown_connections 2024-09-19T06:48:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.811+0000 7f83beb32640 1 -- 172.21.15.103:0/3330281347 wait complete. 2024-09-19T06:48:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.812+0000 7f83beb32640 1 Processor -- start 2024-09-19T06:48:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.812+0000 7f83beb32640 1 -- start start 2024-09-19T06:48:04.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.812+0000 7f83beb32640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83b8102db0 0x7f83b81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.812+0000 7f83beb32640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83b8103760 0x7f83b8198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.812+0000 7f83beb32640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b81082e0 0x7f83b819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.812+0000 7f83beb32640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f83b81120b0 con 0x7f83b81082e0 2024-09-19T06:48:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.812+0000 7f83beb32640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83b8111f30 con 0x7f83b8102db0 2024-09-19T06:48:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.812+0000 7f83beb32640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83b8112230 con 0x7f83b8103760 2024-09-19T06:48:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.812+0000 7f83bc8ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83b8102db0 0x7f83b81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.812+0000 7f83bc8ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83b8102db0 0x7f83b81984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:55120/0 (socket says 172.21.15.103:55120) 2024-09-19T06:48:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.812+0000 7f83bc8ce640 1 -- 172.21.15.103:0/1907793795 learned_addr learned my addr 172.21.15.103:0/1907793795 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.812+0000 7f83bd0cf640 1 --2- 172.21.15.103:0/1907793795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b81082e0 0x7f83b819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.813+0000 7f83b7fff640 1 --2- 172.21.15.103:0/1907793795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83b8103760 0x7f83b8198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.813+0000 7f83bc8ce640 1 -- 172.21.15.103:0/1907793795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83b8103760 msgr2=0x7f83b8198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.813+0000 7f83bc8ce640 1 --2- 172.21.15.103:0/1907793795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83b8103760 0x7f83b8198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.813+0000 7f83bc8ce640 1 -- 172.21.15.103:0/1907793795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b81082e0 msgr2=0x7f83b819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.813+0000 7f83bc8ce640 1 --2- 172.21.15.103:0/1907793795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b81082e0 0x7f83b819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.813+0000 7f83bc8ce640 1 -- 172.21.15.103:0/1907793795 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b819d490 con 0x7f83b8102db0 2024-09-19T06:48:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.813+0000 7f83b7fff640 1 --2- 172.21.15.103:0/1907793795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83b8103760 0x7f83b8198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.813+0000 7f83bd0cf640 1 --2- 172.21.15.103:0/1907793795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b81082e0 0x7f83b819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.813+0000 7f83bc8ce640 1 --2- 172.21.15.103:0/1907793795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83b8102db0 0x7f83b81984e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00c920 tx=0x7f83ac00cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.813+0000 7f83b5ffb640 1 -- 172.21.15.103:0/1907793795 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac007c70 con 0x7f83b8102db0 2024-09-19T06:48:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.813+0000 7f83beb32640 1 -- 172.21.15.103:0/1907793795 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b819d780 con 0x7f83b8102db0 2024-09-19T06:48:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.813+0000 7f83b5ffb640 1 -- 172.21.15.103:0/1907793795 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83ac007e10 con 0x7f83b8102db0 2024-09-19T06:48:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.813+0000 7f83b5ffb640 1 -- 172.21.15.103:0/1907793795 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac018890 con 0x7f83b8102db0 2024-09-19T06:48:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.814+0000 7f83beb32640 1 -- 172.21.15.103:0/1907793795 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b8100940 con 0x7f83b8102db0 2024-09-19T06:48:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.815+0000 7f83beb32640 1 -- 172.21.15.103:0/1907793795 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83b8062980 con 0x7f83b8102db0 2024-09-19T06:48:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.817+0000 7f83b5ffb640 1 -- 172.21.15.103:0/1907793795 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f83ac018a30 con 0x7f83b8102db0 2024-09-19T06:48:04.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.817+0000 7f83b5ffb640 1 --2- 172.21.15.103:0/1907793795 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f83a80a3350 0x7f83a80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:04.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.817+0000 7f83b7fff640 1 --2- 172.21.15.103:0/1907793795 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f83a80a3350 0x7f83a80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:04.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.818+0000 7f83b5ffb640 1 -- 172.21.15.103:0/1907793795 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f83ac0cff90 con 0x7f83b8102db0 2024-09-19T06:48:04.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.818+0000 7f83b5ffb640 1 --2- 172.21.15.103:0/1907793795 >> [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195] conn(0x7f83a80acf00 0x7f83a80af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:04.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.818+0000 7f83b5ffb640 1 -- 172.21.15.103:0/1907793795 --> [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83a80afa70 con 0x7f83a80acf00 2024-09-19T06:48:04.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.818+0000 7f83b5ffb640 1 -- 172.21.15.103:0/1907793795 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f83ac0d03e0 con 0x7f83b8102db0 2024-09-19T06:48:04.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.818+0000 7f83b7fff640 1 --2- 172.21.15.103:0/1907793795 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f83a80a3350 0x7f83a80a5810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f83b80fed60 tx=0x7f83a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:04.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.818+0000 7f83bd0cf640 1 --2- 172.21.15.103:0/1907793795 >> [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195] conn(0x7f83a80acf00 0x7f83a80af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:04.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.819+0000 7f83bd0cf640 1 --2- 172.21.15.103:0/1907793795 >> [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195] conn(0x7f83a80acf00 0x7f83a80af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:04.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.820+0000 7f83b5ffb640 1 -- 172.21.15.103:0/1907793795 <== osd.1 v2:172.21.15.134:6800/1418301195 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83a80afa70 con 0x7f83a80acf00 2024-09-19T06:48:04.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.833+0000 7f83beb32640 1 -- 172.21.15.103:0/1907793795 --> [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83b8062d80 con 0x7f83a80acf00 2024-09-19T06:48:04.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.834+0000 7f83b5ffb640 1 -- 172.21.15.103:0/1907793795 <== osd.1 v2:172.21.15.134:6800/1418301195 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f83b8062d80 con 0x7f83a80acf00 2024-09-19T06:48:04.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.834+0000 7f83beb32640 1 -- 172.21.15.103:0/1907793795 >> [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195] conn(0x7f83a80acf00 msgr2=0x7f83a80af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.834+0000 7f83beb32640 1 --2- 172.21.15.103:0/1907793795 >> [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195] conn(0x7f83a80acf00 0x7f83a80af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.834+0000 7f83beb32640 1 -- 172.21.15.103:0/1907793795 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f83a80a3350 msgr2=0x7f83a80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.834+0000 7f83beb32640 1 --2- 172.21.15.103:0/1907793795 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f83a80a3350 0x7f83a80a5810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f83b80fed60 tx=0x7f83a4005f90 comp rx=0 tx=0).stop 2024-09-19T06:48:04.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.834+0000 7f83beb32640 1 -- 172.21.15.103:0/1907793795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83b8102db0 msgr2=0x7f83b81984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:04.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.834+0000 7f83beb32640 1 --2- 172.21.15.103:0/1907793795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83b8102db0 0x7f83b81984e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00c920 tx=0x7f83ac00cde0 comp rx=0 tx=0).stop 2024-09-19T06:48:04.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.834+0000 7f83bd0cf640 1 -- 172.21.15.103:0/1907793795 reap_dead start 2024-09-19T06:48:04.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.834+0000 7f83beb32640 1 -- 172.21.15.103:0/1907793795 shutdown_connections 2024-09-19T06:48:04.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.834+0000 7f83beb32640 1 -- 172.21.15.103:0/1907793795 >> 172.21.15.103:0/1907793795 conn(0x7f83b80fc7d0 msgr2=0x7f83b8106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:04.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.834+0000 7f83beb32640 1 -- 172.21.15.103:0/1907793795 shutdown_connections 2024-09-19T06:48:04.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:04.834+0000 7f83beb32640 1 -- 172.21.15.103:0/1907793795 wait complete. 2024-09-19T06:48:05.045 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:04 smithi103 ceph-mon[31158]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T06:48:05.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:04 smithi134 ceph-mon[35356]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T06:48:05.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:04 smithi174 ceph-mon[35637]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T06:48:06.044 INFO:teuthology.orchestra.run.smithi103.stdout:128849018934 2024-09-19T06:48:06.044 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T06:48:06.635 INFO:teuthology.orchestra.run.smithi103.stdout:266287972383 2024-09-19T06:48:06.636 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T06:48:06.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.943+0000 7f2c1fe33640 1 -- 172.21.15.103:0/2903240206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102ef0 msgr2=0x7f2c1810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:06.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.943+0000 7f2c1fe33640 1 --2- 172.21.15.103:0/2903240206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102ef0 0x7f2c1810da40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009ca0 tx=0x7f2c1002a950 comp rx=0 tx=0).stop 2024-09-19T06:48:06.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.943+0000 7f2c1fe33640 1 -- 172.21.15.103:0/2903240206 shutdown_connections 2024-09-19T06:48:06.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.943+0000 7f2c1fe33640 1 --2- 172.21.15.103:0/2903240206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102ef0 0x7f2c1810da40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:06.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.943+0000 7f2c1fe33640 1 --2- 172.21.15.103:0/2903240206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c18102570 0x7f2c181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:06.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.943+0000 7f2c1fe33640 1 --2- 172.21.15.103:0/2903240206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c18100e20 0x7f2c18101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:06.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.943+0000 7f2c1fe33640 1 -- 172.21.15.103:0/2903240206 >> 172.21.15.103:0/2903240206 conn(0x7f2c180fc7d0 msgr2=0x7f2c180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:06.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.943+0000 7f2c1fe33640 1 -- 172.21.15.103:0/2903240206 shutdown_connections 2024-09-19T06:48:06.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.943+0000 7f2c1fe33640 1 -- 172.21.15.103:0/2903240206 wait complete. 2024-09-19T06:48:06.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1fe33640 1 Processor -- start 2024-09-19T06:48:06.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1fe33640 1 -- start start 2024-09-19T06:48:06.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1fe33640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c18100e20 0x7f2c180fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:06.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1fe33640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c18102570 0x7f2c180fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:06.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1fe33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102ef0 0x7f2c180fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:06.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1fe33640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18110f80 con 0x7f2c18102ef0 2024-09-19T06:48:06.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1fe33640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18110e00 con 0x7f2c18100e20 2024-09-19T06:48:06.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1fe33640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18111100 con 0x7f2c18102570 2024-09-19T06:48:06.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1e3d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102ef0 0x7f2c180fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:06.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1e3d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102ef0 0x7f2c180fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36752/0 (socket says 172.21.15.103:36752) 2024-09-19T06:48:06.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1e3d0640 1 -- 172.21.15.103:0/2618560098 learned_addr learned my addr 172.21.15.103:0/2618560098 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:06.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1dbcf640 1 --2- 172.21.15.103:0/2618560098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c18100e20 0x7f2c180fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:06.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1d3ce640 1 --2- 172.21.15.103:0/2618560098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c18102570 0x7f2c180fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:06.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1e3d0640 1 -- 172.21.15.103:0/2618560098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c18102570 msgr2=0x7f2c180fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:06.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1e3d0640 1 --2- 172.21.15.103:0/2618560098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c18102570 0x7f2c180fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:06.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1e3d0640 1 -- 172.21.15.103:0/2618560098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c18100e20 msgr2=0x7f2c180fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:06.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1e3d0640 1 --2- 172.21.15.103:0/2618560098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c18100e20 0x7f2c180fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:06.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1e3d0640 1 -- 172.21.15.103:0/2618560098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c181a6930 con 0x7f2c18102ef0 2024-09-19T06:48:06.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1d3ce640 1 --2- 172.21.15.103:0/2618560098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c18102570 0x7f2c180fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:06.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1dbcf640 1 --2- 172.21.15.103:0/2618560098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c18100e20 0x7f2c180fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:06.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.944+0000 7f2c1e3d0640 1 --2- 172.21.15.103:0/2618560098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102ef0 0x7f2c180fd700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2c1002ae70 tx=0x7f2c1002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:06.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.945+0000 7f2c0effd640 1 -- 172.21.15.103:0/2618560098 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10034040 con 0x7f2c18102ef0 2024-09-19T06:48:06.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.945+0000 7f2c0effd640 1 -- 172.21.15.103:0/2618560098 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c10030920 con 0x7f2c18102ef0 2024-09-19T06:48:06.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.945+0000 7f2c1fe33640 1 -- 172.21.15.103:0/2618560098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c181a6bc0 con 0x7f2c18102ef0 2024-09-19T06:48:06.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.945+0000 7f2c0effd640 1 -- 172.21.15.103:0/2618560098 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10045480 con 0x7f2c18102ef0 2024-09-19T06:48:06.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.945+0000 7f2c1fe33640 1 -- 172.21.15.103:0/2618560098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c181a70f0 con 0x7f2c18102ef0 2024-09-19T06:48:06.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.946+0000 7f2c1fe33640 1 -- 172.21.15.103:0/2618560098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2c18062940 con 0x7f2c18102ef0 2024-09-19T06:48:06.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.947+0000 7f2c0effd640 1 -- 172.21.15.103:0/2618560098 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2c10037020 con 0x7f2c18102ef0 2024-09-19T06:48:06.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.947+0000 7f2c0effd640 1 --2- 172.21.15.103:0/2618560098 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2c040a3350 0x7f2c040a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:06.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.948+0000 7f2c1dbcf640 1 --2- 172.21.15.103:0/2618560098 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2c040a3350 0x7f2c040a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:06.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.948+0000 7f2c0effd640 1 -- 172.21.15.103:0/2618560098 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2c100f0c90 con 0x7f2c18102ef0 2024-09-19T06:48:06.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.948+0000 7f2c0effd640 1 --2- 172.21.15.103:0/2618560098 >> [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442] conn(0x7f2c040acf00 0x7f2c040af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:06.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.948+0000 7f2c0effd640 1 -- 172.21.15.103:0/2618560098 --> [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2c10044ec0 con 0x7f2c040acf00 2024-09-19T06:48:06.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.948+0000 7f2c0effd640 1 -- 172.21.15.103:0/2618560098 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f2c100f10e0 con 0x7f2c18102ef0 2024-09-19T06:48:06.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.948+0000 7f2c1dbcf640 1 --2- 172.21.15.103:0/2618560098 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2c040a3350 0x7f2c040a5810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2c140058c0 tx=0x7f2c14005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:06.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.948+0000 7f2c1d3ce640 1 --2- 172.21.15.103:0/2618560098 >> [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442] conn(0x7f2c040acf00 0x7f2c040af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:06.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.948+0000 7f2c1d3ce640 1 --2- 172.21.15.103:0/2618560098 >> [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442] conn(0x7f2c040acf00 0x7f2c040af3a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:06.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.949+0000 7f2c0effd640 1 -- 172.21.15.103:0/2618560098 <== osd.11 v2:172.21.15.174:6824/161916442 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c10044ec0 con 0x7f2c040acf00 2024-09-19T06:48:06.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.962+0000 7f2c1fe33640 1 -- 172.21.15.103:0/2618560098 --> [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2c18062d40 con 0x7f2c040acf00 2024-09-19T06:48:06.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.963+0000 7f2c0effd640 1 -- 172.21.15.103:0/2618560098 <== osd.11 v2:172.21.15.174:6824/161916442 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2c18062d40 con 0x7f2c040acf00 2024-09-19T06:48:06.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.963+0000 7f2c1fe33640 1 -- 172.21.15.103:0/2618560098 >> [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442] conn(0x7f2c040acf00 msgr2=0x7f2c040af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:06.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.963+0000 7f2c1fe33640 1 --2- 172.21.15.103:0/2618560098 >> [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442] conn(0x7f2c040acf00 0x7f2c040af3a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:06.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.963+0000 7f2c1fe33640 1 -- 172.21.15.103:0/2618560098 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2c040a3350 msgr2=0x7f2c040a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:06.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.963+0000 7f2c1fe33640 1 --2- 172.21.15.103:0/2618560098 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2c040a3350 0x7f2c040a5810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2c140058c0 tx=0x7f2c14005850 comp rx=0 tx=0).stop 2024-09-19T06:48:06.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.963+0000 7f2c1fe33640 1 -- 172.21.15.103:0/2618560098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102ef0 msgr2=0x7f2c180fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:06.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.963+0000 7f2c1fe33640 1 --2- 172.21.15.103:0/2618560098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102ef0 0x7f2c180fd700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2c1002ae70 tx=0x7f2c1002f5d0 comp rx=0 tx=0).stop 2024-09-19T06:48:06.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.963+0000 7f2c1e3d0640 1 -- 172.21.15.103:0/2618560098 reap_dead start 2024-09-19T06:48:06.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.963+0000 7f2c1fe33640 1 -- 172.21.15.103:0/2618560098 shutdown_connections 2024-09-19T06:48:06.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.963+0000 7f2c1fe33640 1 -- 172.21.15.103:0/2618560098 >> 172.21.15.103:0/2618560098 conn(0x7f2c180fc7d0 msgr2=0x7f2c1806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:06.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.964+0000 7f2c1fe33640 1 -- 172.21.15.103:0/2618560098 shutdown_connections 2024-09-19T06:48:06.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:06.964+0000 7f2c1fe33640 1 -- 172.21.15.103:0/2618560098 wait complete. 2024-09-19T06:48:07.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:06 smithi103 ceph-mon[31158]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T06:48:07.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:06 smithi174 ceph-mon[35637]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T06:48:07.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:06 smithi134 ceph-mon[35356]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T06:48:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.536+0000 7fe3a2c6b640 1 -- 172.21.15.103:0/4111021031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe39c106a80 msgr2=0x7fe39c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.536+0000 7fe3a2c6b640 1 --2- 172.21.15.103:0/4111021031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe39c106a80 0x7fe39c106e60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe390009930 tx=0x7fe39002a420 comp rx=0 tx=0).stop 2024-09-19T06:48:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.536+0000 7fe3a2c6b640 1 -- 172.21.15.103:0/4111021031 shutdown_connections 2024-09-19T06:48:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.536+0000 7fe3a2c6b640 1 --2- 172.21.15.103:0/4111021031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe39c101420 0x7fe39c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.536+0000 7fe3a2c6b640 1 --2- 172.21.15.103:0/4111021031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe39c100a80 0x7fe39c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.536+0000 7fe3a2c6b640 1 --2- 172.21.15.103:0/4111021031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe39c106a80 0x7fe39c106e60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.536+0000 7fe3a2c6b640 1 -- 172.21.15.103:0/4111021031 >> 172.21.15.103:0/4111021031 conn(0x7fe39c0fc7a0 msgr2=0x7fe39c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.536+0000 7fe3a2c6b640 1 -- 172.21.15.103:0/4111021031 shutdown_connections 2024-09-19T06:48:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.536+0000 7fe3a2c6b640 1 -- 172.21.15.103:0/4111021031 wait complete. 2024-09-19T06:48:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe3a2c6b640 1 Processor -- start 2024-09-19T06:48:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe3a2c6b640 1 -- start start 2024-09-19T06:48:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe3a2c6b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe39c100a80 0x7fe39c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe3a2c6b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe39c101420 0x7fe39c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe3a2c6b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe39c106a80 0x7fe39c1a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe3a2c6b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c110ef0 con 0x7fe39c106a80 2024-09-19T06:48:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe3a2c6b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c110d70 con 0x7fe39c101420 2024-09-19T06:48:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe3a2c6b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c111070 con 0x7fe39c100a80 2024-09-19T06:48:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe3a1208640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe39c106a80 0x7fe39c1a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe3a1208640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe39c106a80 0x7fe39c1a4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58196/0 (socket says 172.21.15.103:58196) 2024-09-19T06:48:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe3a1208640 1 -- 172.21.15.103:0/885829959 learned_addr learned my addr 172.21.15.103:0/885829959 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe39bfff640 1 --2- 172.21.15.103:0/885829959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe39c101420 0x7fe39c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe3a0a07640 1 --2- 172.21.15.103:0/885829959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe39c100a80 0x7fe39c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe3a1208640 1 -- 172.21.15.103:0/885829959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe39c100a80 msgr2=0x7fe39c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe3a1208640 1 --2- 172.21.15.103:0/885829959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe39c100a80 0x7fe39c199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe3a1208640 1 -- 172.21.15.103:0/885829959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe39c101420 msgr2=0x7fe39c1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.537+0000 7fe3a1208640 1 --2- 172.21.15.103:0/885829959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe39c101420 0x7fe39c1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.538+0000 7fe3a1208640 1 -- 172.21.15.103:0/885829959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c1a4840 con 0x7fe39c106a80 2024-09-19T06:48:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.538+0000 7fe3a1208640 1 --2- 172.21.15.103:0/885829959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe39c106a80 0x7fe39c1a4300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe39400b970 tx=0x7fe39400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.538+0000 7fe399ffb640 1 -- 172.21.15.103:0/885829959 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394019070 con 0x7fe39c106a80 2024-09-19T06:48:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.538+0000 7fe399ffb640 1 -- 172.21.15.103:0/885829959 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe39400cb40 con 0x7fe39c106a80 2024-09-19T06:48:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.538+0000 7fe3a2c6b640 1 -- 172.21.15.103:0/885829959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39c1a4a10 con 0x7fe39c106a80 2024-09-19T06:48:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.538+0000 7fe399ffb640 1 -- 172.21.15.103:0/885829959 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394007650 con 0x7fe39c106a80 2024-09-19T06:48:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.538+0000 7fe3a2c6b640 1 -- 172.21.15.103:0/885829959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe39c1a4ed0 con 0x7fe39c106a80 2024-09-19T06:48:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.539+0000 7fe3a2c6b640 1 -- 172.21.15.103:0/885829959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe39c0fd750 con 0x7fe39c106a80 2024-09-19T06:48:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.540+0000 7fe399ffb640 1 -- 172.21.15.103:0/885829959 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe39401c070 con 0x7fe39c106a80 2024-09-19T06:48:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.541+0000 7fe399ffb640 1 --2- 172.21.15.103:0/885829959 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe38c0a32d0 0x7fe38c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.541+0000 7fe3a0a07640 1 --2- 172.21.15.103:0/885829959 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe38c0a32d0 0x7fe38c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.541+0000 7fe399ffb640 1 -- 172.21.15.103:0/885829959 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe3940cf820 con 0x7fe39c106a80 2024-09-19T06:48:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.541+0000 7fe399ffb640 1 --2- 172.21.15.103:0/885829959 >> [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333] conn(0x7fe38c0ace80 0x7fe38c0af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.541+0000 7fe399ffb640 1 -- 172.21.15.103:0/885829959 --> [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe3940047d0 con 0x7fe38c0ace80 2024-09-19T06:48:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.541+0000 7fe399ffb640 1 -- 172.21.15.103:0/885829959 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fe3940cfc70 con 0x7fe39c106a80 2024-09-19T06:48:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.541+0000 7fe39bfff640 1 --2- 172.21.15.103:0/885829959 >> [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333] conn(0x7fe38c0ace80 0x7fe38c0af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.541+0000 7fe3a0a07640 1 --2- 172.21.15.103:0/885829959 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe38c0a32d0 0x7fe38c0a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe3900096c0 tx=0x7fe390005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.541+0000 7fe39bfff640 1 --2- 172.21.15.103:0/885829959 >> [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333] conn(0x7fe38c0ace80 0x7fe38c0af320 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.542+0000 7fe399ffb640 1 -- 172.21.15.103:0/885829959 <== osd.3 v2:172.21.15.103:6810/2698986333 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3940047d0 con 0x7fe38c0ace80 2024-09-19T06:48:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.555+0000 7fe3a2c6b640 1 -- 172.21.15.103:0/885829959 --> [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe39c1021c0 con 0x7fe38c0ace80 2024-09-19T06:48:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.556+0000 7fe399ffb640 1 -- 172.21.15.103:0/885829959 <== osd.3 v2:172.21.15.103:6810/2698986333 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe39c1021c0 con 0x7fe38c0ace80 2024-09-19T06:48:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.556+0000 7fe3a2c6b640 1 -- 172.21.15.103:0/885829959 >> [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333] conn(0x7fe38c0ace80 msgr2=0x7fe38c0af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.556+0000 7fe3a2c6b640 1 --2- 172.21.15.103:0/885829959 >> [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333] conn(0x7fe38c0ace80 0x7fe38c0af320 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.556+0000 7fe3a2c6b640 1 -- 172.21.15.103:0/885829959 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe38c0a32d0 msgr2=0x7fe38c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.556+0000 7fe3a2c6b640 1 --2- 172.21.15.103:0/885829959 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe38c0a32d0 0x7fe38c0a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe3900096c0 tx=0x7fe390005fd0 comp rx=0 tx=0).stop 2024-09-19T06:48:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.556+0000 7fe3a2c6b640 1 -- 172.21.15.103:0/885829959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe39c106a80 msgr2=0x7fe39c1a4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.556+0000 7fe3a2c6b640 1 --2- 172.21.15.103:0/885829959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe39c106a80 0x7fe39c1a4300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe39400b970 tx=0x7fe39400be30 comp rx=0 tx=0).stop 2024-09-19T06:48:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.556+0000 7fe3a1208640 1 -- 172.21.15.103:0/885829959 reap_dead start 2024-09-19T06:48:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.556+0000 7fe3a2c6b640 1 -- 172.21.15.103:0/885829959 shutdown_connections 2024-09-19T06:48:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.556+0000 7fe3a2c6b640 1 -- 172.21.15.103:0/885829959 >> 172.21.15.103:0/885829959 conn(0x7fe39c0fc7a0 msgr2=0x7fe39c0fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.556+0000 7fe3a2c6b640 1 -- 172.21.15.103:0/885829959 shutdown_connections 2024-09-19T06:48:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:07.556+0000 7fe3a2c6b640 1 -- 172.21.15.103:0/885829959 wait complete. 2024-09-19T06:48:08.323 INFO:teuthology.orchestra.run.smithi103.stdout:103079215161 2024-09-19T06:48:08.323 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T06:48:08.885 INFO:teuthology.orchestra.run.smithi103.stdout:55834574919 2024-09-19T06:48:08.886 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T06:48:09.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:08 smithi103 ceph-mon[31158]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T06:48:09.239 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:08 smithi134 ceph-mon[35356]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T06:48:09.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:08 smithi174 ceph-mon[35637]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T06:48:09.388 INFO:teuthology.orchestra.run.smithi103.stdout:201863462952 2024-09-19T06:48:09.389 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T06:48:09.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.611+0000 7f3f032de640 1 -- 172.21.15.103:0/3886643354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3efc1036e0 msgr2=0x7f3efc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:09.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.611+0000 7f3f032de640 1 --2- 172.21.15.103:0/3886643354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3efc1036e0 0x7f3efc107b90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009930 tx=0x7f3eec02a420 comp rx=0 tx=0).stop 2024-09-19T06:48:09.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.611+0000 7f3f032de640 1 -- 172.21.15.103:0/3886643354 shutdown_connections 2024-09-19T06:48:09.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.611+0000 7f3f032de640 1 --2- 172.21.15.103:0/3886643354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3efc1082a0 0x7f3efc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:09.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.611+0000 7f3f032de640 1 --2- 172.21.15.103:0/3886643354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3efc1036e0 0x7f3efc107b90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:09.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.611+0000 7f3f032de640 1 --2- 172.21.15.103:0/3886643354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3efc102d30 0x7f3efc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:09.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.611+0000 7f3f032de640 1 -- 172.21.15.103:0/3886643354 >> 172.21.15.103:0/3886643354 conn(0x7f3efc0fc7d0 msgr2=0x7f3efc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:09.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.611+0000 7f3f032de640 1 -- 172.21.15.103:0/3886643354 shutdown_connections 2024-09-19T06:48:09.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.611+0000 7f3f032de640 1 -- 172.21.15.103:0/3886643354 wait complete. 2024-09-19T06:48:09.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.612+0000 7f3f032de640 1 Processor -- start 2024-09-19T06:48:09.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.612+0000 7f3f032de640 1 -- start start 2024-09-19T06:48:09.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.612+0000 7f3f032de640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3efc102d30 0x7f3efc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:09.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.612+0000 7f3f032de640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3efc1036e0 0x7f3efc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:09.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.612+0000 7f3f032de640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3efc1082a0 0x7f3efc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:09.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.612+0000 7f3f032de640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1120a0 con 0x7f3efc1082a0 2024-09-19T06:48:09.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.612+0000 7f3f032de640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc111f20 con 0x7f3efc102d30 2024-09-19T06:48:09.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.612+0000 7f3f032de640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc112220 con 0x7f3efc1036e0 2024-09-19T06:48:09.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.612+0000 7f3f0187b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3efc1082a0 0x7f3efc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:09.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.612+0000 7f3f0187b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3efc1082a0 0x7f3efc101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58220/0 (socket says 172.21.15.103:58220) 2024-09-19T06:48:09.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.612+0000 7f3f0187b640 1 -- 172.21.15.103:0/1431661703 learned_addr learned my addr 172.21.15.103:0/1431661703 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:09.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.612+0000 7f3f0107a640 1 --2- 172.21.15.103:0/1431661703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3efc102d30 0x7f3efc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:09.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.612+0000 7f3f00879640 1 --2- 172.21.15.103:0/1431661703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3efc1036e0 0x7f3efc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:09.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.613+0000 7f3f0187b640 1 -- 172.21.15.103:0/1431661703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3efc1036e0 msgr2=0x7f3efc100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:09.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.613+0000 7f3f0187b640 1 --2- 172.21.15.103:0/1431661703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3efc1036e0 0x7f3efc100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:09.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.613+0000 7f3f0187b640 1 -- 172.21.15.103:0/1431661703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3efc102d30 msgr2=0x7f3efc100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:09.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.613+0000 7f3f0187b640 1 --2- 172.21.15.103:0/1431661703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3efc102d30 0x7f3efc100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:09.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.613+0000 7f3f0187b640 1 -- 172.21.15.103:0/1431661703 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc1a6ae0 con 0x7f3efc1082a0 2024-09-19T06:48:09.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.613+0000 7f3f0107a640 1 --2- 172.21.15.103:0/1431661703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3efc102d30 0x7f3efc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:09.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.613+0000 7f3f0187b640 1 --2- 172.21.15.103:0/1431661703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3efc1082a0 0x7f3efc101410 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400eb10 tx=0x7f3ef400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:09.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.613+0000 7f3ef27fc640 1 -- 172.21.15.103:0/1431661703 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4004b20 con 0x7f3efc1082a0 2024-09-19T06:48:09.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.613+0000 7f3ef27fc640 1 -- 172.21.15.103:0/1431661703 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ef4004cc0 con 0x7f3efc1082a0 2024-09-19T06:48:09.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.613+0000 7f3ef27fc640 1 -- 172.21.15.103:0/1431661703 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4010760 con 0x7f3efc1082a0 2024-09-19T06:48:09.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.613+0000 7f3f032de640 1 -- 172.21.15.103:0/1431661703 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc1a6dd0 con 0x7f3efc1082a0 2024-09-19T06:48:09.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.613+0000 7f3f032de640 1 -- 172.21.15.103:0/1431661703 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc1a7230 con 0x7f3efc1082a0 2024-09-19T06:48:09.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.614+0000 7f3f032de640 1 -- 172.21.15.103:0/1431661703 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3efc062900 con 0x7f3efc1082a0 2024-09-19T06:48:09.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.617+0000 7f3ef27fc640 1 -- 172.21.15.103:0/1431661703 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3ef4005000 con 0x7f3efc1082a0 2024-09-19T06:48:09.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.618+0000 7f3ef27fc640 1 --2- 172.21.15.103:0/1431661703 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3ee80a32d0 0x7f3ee80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:09.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.618+0000 7f3ef27fc640 1 -- 172.21.15.103:0/1431661703 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3ef409a8c0 con 0x7f3efc1082a0 2024-09-19T06:48:09.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.619+0000 7f3f0107a640 1 --2- 172.21.15.103:0/1431661703 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3ee80a32d0 0x7f3ee80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:09.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.619+0000 7f3ef27fc640 1 --2- 172.21.15.103:0/1431661703 >> [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644] conn(0x7f3ee80ada00 0x7f3ee80afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:09.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.619+0000 7f3ef27fc640 1 -- 172.21.15.103:0/1431661703 --> [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ee80b0570 con 0x7f3ee80ada00 2024-09-19T06:48:09.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.619+0000 7f3ef27fc640 1 -- 172.21.15.103:0/1431661703 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f3ef4099250 con 0x7f3efc1082a0 2024-09-19T06:48:09.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.619+0000 7f3f00879640 1 --2- 172.21.15.103:0/1431661703 >> [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644] conn(0x7f3ee80ada00 0x7f3ee80afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:09.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.619+0000 7f3f0107a640 1 --2- 172.21.15.103:0/1431661703 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3ee80a32d0 0x7f3ee80a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8005fd0 tx=0x7f3ef800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:09.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.620+0000 7f3f00879640 1 --2- 172.21.15.103:0/1431661703 >> [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644] conn(0x7f3ee80ada00 0x7f3ee80afea0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:09.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.621+0000 7f3ef27fc640 1 -- 172.21.15.103:0/1431661703 <== osd.5 v2:172.21.15.174:6808/763749644 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3ee80b0570 con 0x7f3ee80ada00 2024-09-19T06:48:09.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.636+0000 7f3f032de640 1 -- 172.21.15.103:0/1431661703 --> [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3efc062dc0 con 0x7f3ee80ada00 2024-09-19T06:48:09.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.636+0000 7f3ef27fc640 1 -- 172.21.15.103:0/1431661703 <== osd.5 v2:172.21.15.174:6808/763749644 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3efc062dc0 con 0x7f3ee80ada00 2024-09-19T06:48:09.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.636+0000 7f3f032de640 1 -- 172.21.15.103:0/1431661703 >> [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644] conn(0x7f3ee80ada00 msgr2=0x7f3ee80afea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:09.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.636+0000 7f3f032de640 1 --2- 172.21.15.103:0/1431661703 >> [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644] conn(0x7f3ee80ada00 0x7f3ee80afea0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:09.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.636+0000 7f3f032de640 1 -- 172.21.15.103:0/1431661703 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3ee80a32d0 msgr2=0x7f3ee80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:09.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.637+0000 7f3f032de640 1 --2- 172.21.15.103:0/1431661703 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3ee80a32d0 0x7f3ee80a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8005fd0 tx=0x7f3ef800b390 comp rx=0 tx=0).stop 2024-09-19T06:48:09.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.637+0000 7f3f032de640 1 -- 172.21.15.103:0/1431661703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3efc1082a0 msgr2=0x7f3efc101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:09.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.637+0000 7f3f032de640 1 --2- 172.21.15.103:0/1431661703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3efc1082a0 0x7f3efc101410 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400eb10 tx=0x7f3ef400c2d0 comp rx=0 tx=0).stop 2024-09-19T06:48:09.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.637+0000 7f3f0187b640 1 -- 172.21.15.103:0/1431661703 reap_dead start 2024-09-19T06:48:09.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.637+0000 7f3f032de640 1 -- 172.21.15.103:0/1431661703 shutdown_connections 2024-09-19T06:48:09.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.637+0000 7f3f032de640 1 -- 172.21.15.103:0/1431661703 >> 172.21.15.103:0/1431661703 conn(0x7f3efc0fc7d0 msgr2=0x7f3efc105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:09.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.637+0000 7f3f032de640 1 -- 172.21.15.103:0/1431661703 shutdown_connections 2024-09-19T06:48:09.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:09.637+0000 7f3f032de640 1 -- 172.21.15.103:0/1431661703 wait complete. 2024-09-19T06:48:10.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.197+0000 7f003ba53640 1 -- 172.21.15.103:0/2203167216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0034106f20 msgr2=0x7f003410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.197+0000 7f003ba53640 1 --2- 172.21.15.103:0/2203167216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0034106f20 0x7f003410eaa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f002c009ca0 tx=0x7f002c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:48:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.197+0000 7f003ba53640 1 -- 172.21.15.103:0/2203167216 shutdown_connections 2024-09-19T06:48:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.197+0000 7f003ba53640 1 --2- 172.21.15.103:0/2203167216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0034106f20 0x7f003410eaa0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.197+0000 7f003ba53640 1 --2- 172.21.15.103:0/2203167216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00341027a0 0x7f00341067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.197+0000 7f003ba53640 1 --2- 172.21.15.103:0/2203167216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0034101df0 0x7f00341021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.197+0000 7f003ba53640 1 -- 172.21.15.103:0/2203167216 >> 172.21.15.103:0/2203167216 conn(0x7f00340fd8f0 msgr2=0x7f00340ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.197+0000 7f003ba53640 1 -- 172.21.15.103:0/2203167216 shutdown_connections 2024-09-19T06:48:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.197+0000 7f003ba53640 1 -- 172.21.15.103:0/2203167216 wait complete. 2024-09-19T06:48:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.198+0000 7f003ba53640 1 Processor -- start 2024-09-19T06:48:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.198+0000 7f003ba53640 1 -- start start 2024-09-19T06:48:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.198+0000 7f003ba53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0034101df0 0x7f00340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.198+0000 7f003ba53640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00341027a0 0x7f0034074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.198+0000 7f003ba53640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0034106f20 0x7f0034074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.198+0000 7f003ba53640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0034112020 con 0x7f0034101df0 2024-09-19T06:48:10.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.198+0000 7f003ba53640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0034111ea0 con 0x7f00341027a0 2024-09-19T06:48:10.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.198+0000 7f003ba53640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f00341121a0 con 0x7f0034106f20 2024-09-19T06:48:10.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.198+0000 7f00397ef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0034101df0 0x7f00340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:10.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.198+0000 7f00397ef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0034101df0 0x7f00340744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58246/0 (socket says 172.21.15.103:58246) 2024-09-19T06:48:10.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.198+0000 7f00397ef640 1 -- 172.21.15.103:0/3878306315 learned_addr learned my addr 172.21.15.103:0/3878306315 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:10.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.198+0000 7f0039ff0640 1 --2- 172.21.15.103:0/3878306315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0034106f20 0x7f0034074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:10.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.198+0000 7f00397ef640 1 -- 172.21.15.103:0/3878306315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0034106f20 msgr2=0x7f0034074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:10.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.199+0000 7f00397ef640 1 --2- 172.21.15.103:0/3878306315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0034106f20 0x7f0034074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:10.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.199+0000 7f00397ef640 1 -- 172.21.15.103:0/3878306315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00341027a0 msgr2=0x7f0034074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:10.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.199+0000 7f00397ef640 1 --2- 172.21.15.103:0/3878306315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00341027a0 0x7f0034074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:10.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.199+0000 7f00397ef640 1 -- 172.21.15.103:0/3878306315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00341a6a60 con 0x7f0034101df0 2024-09-19T06:48:10.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.199+0000 7f0039ff0640 1 --2- 172.21.15.103:0/3878306315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0034106f20 0x7f0034074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:10.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.199+0000 7f00397ef640 1 --2- 172.21.15.103:0/3878306315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0034101df0 0x7f00340744f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f003000b4a0 tx=0x7f003000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:10.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.199+0000 7f002a7fc640 1 -- 172.21.15.103:0/3878306315 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0030014070 con 0x7f0034101df0 2024-09-19T06:48:10.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.199+0000 7f002a7fc640 1 -- 172.21.15.103:0/3878306315 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0030010af0 con 0x7f0034101df0 2024-09-19T06:48:10.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.199+0000 7f003ba53640 1 -- 172.21.15.103:0/3878306315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00341a6d50 con 0x7f0034101df0 2024-09-19T06:48:10.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.199+0000 7f002a7fc640 1 -- 172.21.15.103:0/3878306315 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0030017d10 con 0x7f0034101df0 2024-09-19T06:48:10.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.199+0000 7f003ba53640 1 -- 172.21.15.103:0/3878306315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00341a7230 con 0x7f0034101df0 2024-09-19T06:48:10.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.200+0000 7f003ba53640 1 -- 172.21.15.103:0/3878306315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0034062880 con 0x7f0034101df0 2024-09-19T06:48:10.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.201+0000 7f002a7fc640 1 -- 172.21.15.103:0/3878306315 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0030017070 con 0x7f0034101df0 2024-09-19T06:48:10.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.202+0000 7f002a7fc640 1 --2- 172.21.15.103:0/3878306315 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f00200a32d0 0x7f00200a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:10.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.202+0000 7f002a7fc640 1 -- 172.21.15.103:0/3878306315 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f00300cf190 con 0x7f0034101df0 2024-09-19T06:48:10.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.202+0000 7f0038fee640 1 --2- 172.21.15.103:0/3878306315 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f00200a32d0 0x7f00200a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:10.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.202+0000 7f002a7fc640 1 --2- 172.21.15.103:0/3878306315 >> [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465] conn(0x7f00200ace80 0x7f00200af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:10.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.202+0000 7f002a7fc640 1 -- 172.21.15.103:0/3878306315 --> [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f003000dec0 con 0x7f00200ace80 2024-09-19T06:48:10.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.202+0000 7f002a7fc640 1 -- 172.21.15.103:0/3878306315 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f00300cf5e0 con 0x7f0034101df0 2024-09-19T06:48:10.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.202+0000 7f0038fee640 1 --2- 172.21.15.103:0/3878306315 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f00200a32d0 0x7f00200a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0034075f80 tx=0x7f0024006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:10.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.202+0000 7f0039ff0640 1 --2- 172.21.15.103:0/3878306315 >> [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465] conn(0x7f00200ace80 0x7f00200af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:10.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.203+0000 7f0039ff0640 1 --2- 172.21.15.103:0/3878306315 >> [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465] conn(0x7f00200ace80 0x7f00200af320 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:10.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.204+0000 7f002a7fc640 1 -- 172.21.15.103:0/3878306315 <== osd.8 v2:172.21.15.174:6816/189977465 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f003000dec0 con 0x7f00200ace80 2024-09-19T06:48:10.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.217+0000 7f003ba53640 1 -- 172.21.15.103:0/3878306315 --> [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0034062d00 con 0x7f00200ace80 2024-09-19T06:48:10.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.218+0000 7f002a7fc640 1 -- 172.21.15.103:0/3878306315 <== osd.8 v2:172.21.15.174:6816/189977465 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0034062d00 con 0x7f00200ace80 2024-09-19T06:48:10.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.218+0000 7f003ba53640 1 -- 172.21.15.103:0/3878306315 >> [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465] conn(0x7f00200ace80 msgr2=0x7f00200af320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:10.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.218+0000 7f003ba53640 1 --2- 172.21.15.103:0/3878306315 >> [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465] conn(0x7f00200ace80 0x7f00200af320 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:10.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.218+0000 7f003ba53640 1 -- 172.21.15.103:0/3878306315 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f00200a32d0 msgr2=0x7f00200a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:10.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.218+0000 7f003ba53640 1 --2- 172.21.15.103:0/3878306315 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f00200a32d0 0x7f00200a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0034075f80 tx=0x7f0024006c90 comp rx=0 tx=0).stop 2024-09-19T06:48:10.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.218+0000 7f003ba53640 1 -- 172.21.15.103:0/3878306315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0034101df0 msgr2=0x7f00340744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:10.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.218+0000 7f003ba53640 1 --2- 172.21.15.103:0/3878306315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0034101df0 0x7f00340744f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f003000b4a0 tx=0x7f003000b960 comp rx=0 tx=0).stop 2024-09-19T06:48:10.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.218+0000 7f0039ff0640 1 -- 172.21.15.103:0/3878306315 reap_dead start 2024-09-19T06:48:10.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.218+0000 7f003ba53640 1 -- 172.21.15.103:0/3878306315 shutdown_connections 2024-09-19T06:48:10.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.218+0000 7f003ba53640 1 -- 172.21.15.103:0/3878306315 >> 172.21.15.103:0/3878306315 conn(0x7f00340fd8f0 msgr2=0x7f00340fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:10.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.218+0000 7f003ba53640 1 -- 172.21.15.103:0/3878306315 shutdown_connections 2024-09-19T06:48:10.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:10.218+0000 7f003ba53640 1 -- 172.21.15.103:0/3878306315 wait complete. 2024-09-19T06:48:10.776 INFO:teuthology.orchestra.run.smithi103.stdout:287762808857 2024-09-19T06:48:10.776 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T06:48:11.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:10 smithi103 ceph-mon[31158]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T06:48:11.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:10 smithi174 ceph-mon[35637]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T06:48:11.367 INFO:teuthology.orchestra.run.smithi103.stdout:180388626477 2024-09-19T06:48:11.367 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T06:48:11.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:10 smithi134 ceph-mon[35356]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T06:48:11.739 INFO:teuthology.orchestra.run.smithi103.stdout:317827579926 2024-09-19T06:48:11.739 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T06:48:12.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:11 smithi103 ceph-mon[31158]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T06:48:12.160 INFO:teuthology.orchestra.run.smithi103.stdout:231928234021 2024-09-19T06:48:12.161 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T06:48:12.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:11 smithi174 ceph-mon[35637]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T06:48:12.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:11 smithi134 ceph-mon[35356]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T06:48:12.744 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:12.789 INFO:teuthology.orchestra.run.smithi103.stdout:150323855410 2024-09-19T06:48:12.790 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T06:48:13.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.266+0000 7fdbf24be640 1 -- 172.21.15.103:0/292945663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbec101420 msgr2=0x7fdbec10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.266+0000 7fdbf24be640 1 --2- 172.21.15.103:0/292945663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbec101420 0x7fdbec10d950 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009cd0 tx=0x7fdbe402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:48:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.266+0000 7fdbf24be640 1 -- 172.21.15.103:0/292945663 shutdown_connections 2024-09-19T06:48:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.266+0000 7fdbf24be640 1 --2- 172.21.15.103:0/292945663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbec101420 0x7fdbec10d950 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.266+0000 7fdbf24be640 1 --2- 172.21.15.103:0/292945663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbec100a80 0x7fdbec100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.266+0000 7fdbf24be640 1 --2- 172.21.15.103:0/292945663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbec106a80 0x7fdbec106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.266+0000 7fdbf24be640 1 -- 172.21.15.103:0/292945663 >> 172.21.15.103:0/292945663 conn(0x7fdbec0fc7a0 msgr2=0x7fdbec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.266+0000 7fdbf24be640 1 -- 172.21.15.103:0/292945663 shutdown_connections 2024-09-19T06:48:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.266+0000 7fdbf24be640 1 -- 172.21.15.103:0/292945663 wait complete. 2024-09-19T06:48:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.267+0000 7fdbf24be640 1 Processor -- start 2024-09-19T06:48:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.267+0000 7fdbf24be640 1 -- start start 2024-09-19T06:48:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbf24be640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbec100a80 0x7fdbec199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbf24be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbec101420 0x7fdbec199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbf24be640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbec106a80 0x7fdbec1a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbf24be640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec110e80 con 0x7fdbec101420 2024-09-19T06:48:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbf24be640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec110d00 con 0x7fdbec100a80 2024-09-19T06:48:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbf24be640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec111000 con 0x7fdbec106a80 2024-09-19T06:48:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbeb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbec101420 0x7fdbec199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbeb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbec101420 0x7fdbec199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58274/0 (socket says 172.21.15.103:58274) 2024-09-19T06:48:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbeb7fe640 1 -- 172.21.15.103:0/545210528 learned_addr learned my addr 172.21.15.103:0/545210528 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbebfff640 1 --2- 172.21.15.103:0/545210528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbec100a80 0x7fdbec199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbeb7fe640 1 -- 172.21.15.103:0/545210528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbec106a80 msgr2=0x7fdbec1a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbeb7fe640 1 --2- 172.21.15.103:0/545210528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbec106a80 0x7fdbec1a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbeb7fe640 1 -- 172.21.15.103:0/545210528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbec100a80 msgr2=0x7fdbec199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbeb7fe640 1 --2- 172.21.15.103:0/545210528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbec100a80 0x7fdbec199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbeb7fe640 1 -- 172.21.15.103:0/545210528 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbec1a4810 con 0x7fdbec101420 2024-09-19T06:48:13.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbebfff640 1 --2- 172.21.15.103:0/545210528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbec100a80 0x7fdbec199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:13.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbeb7fe640 1 --2- 172.21.15.103:0/545210528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbec101420 0x7fdbec199790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000c970 tx=0x7fdbe000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:13.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbe97fa640 1 -- 172.21.15.103:0/545210528 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0007bb0 con 0x7fdbec101420 2024-09-19T06:48:13.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbe97fa640 1 -- 172.21.15.103:0/545210528 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe0007d50 con 0x7fdbec101420 2024-09-19T06:48:13.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbf24be640 1 -- 172.21.15.103:0/545210528 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbec1a49e0 con 0x7fdbec101420 2024-09-19T06:48:13.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbe97fa640 1 -- 172.21.15.103:0/545210528 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000f450 con 0x7fdbec101420 2024-09-19T06:48:13.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.268+0000 7fdbf24be640 1 -- 172.21.15.103:0/545210528 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbec1a4ea0 con 0x7fdbec101420 2024-09-19T06:48:13.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.269+0000 7fdbf24be640 1 -- 172.21.15.103:0/545210528 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbec100ee0 con 0x7fdbec101420 2024-09-19T06:48:13.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.270+0000 7fdbe97fa640 1 -- 172.21.15.103:0/545210528 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fdbe0005020 con 0x7fdbec101420 2024-09-19T06:48:13.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.271+0000 7fdbe97fa640 1 --2- 172.21.15.103:0/545210528 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdbd40a32d0 0x7fdbd40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:13.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.271+0000 7fdbebfff640 1 --2- 172.21.15.103:0/545210528 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdbd40a32d0 0x7fdbd40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:13.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.271+0000 7fdbe97fa640 1 -- 172.21.15.103:0/545210528 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdbe00cf920 con 0x7fdbec101420 2024-09-19T06:48:13.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.271+0000 7fdbebfff640 1 --2- 172.21.15.103:0/545210528 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdbd40a32d0 0x7fdbd40a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0058c0 tx=0x7fdbdc005cb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:13.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.276+0000 7fdbe97fa640 1 -- 172.21.15.103:0/545210528 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdbe0014030 con 0x7fdbec101420 2024-09-19T06:48:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:13.341 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:13.346 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:13.347 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:13.347 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:13.361 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:13.362 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:13.367 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:13.368 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:13.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.499+0000 7fdbf24be640 1 -- 172.21.15.103:0/545210528 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fdbec1a5130 con 0x7fdbec101420 2024-09-19T06:48:13.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.499+0000 7fdbe97fa640 1 -- 172.21.15.103:0/545210528 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdbe009c4f0 con 0x7fdbec101420 2024-09-19T06:48:13.500 INFO:teuthology.orchestra.run.smithi103.stdout:73014444099 2024-09-19T06:48:13.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.501+0000 7fdbf24be640 1 -- 172.21.15.103:0/545210528 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdbd40a32d0 msgr2=0x7fdbd40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:13.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.501+0000 7fdbf24be640 1 --2- 172.21.15.103:0/545210528 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdbd40a32d0 0x7fdbd40a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0058c0 tx=0x7fdbdc005cb0 comp rx=0 tx=0).stop 2024-09-19T06:48:13.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.501+0000 7fdbf24be640 1 -- 172.21.15.103:0/545210528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbec101420 msgr2=0x7fdbec199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:13.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.501+0000 7fdbf24be640 1 --2- 172.21.15.103:0/545210528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbec101420 0x7fdbec199790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000c970 tx=0x7fdbe000ce30 comp rx=0 tx=0).stop 2024-09-19T06:48:13.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.501+0000 7fdbf24be640 1 -- 172.21.15.103:0/545210528 shutdown_connections 2024-09-19T06:48:13.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.501+0000 7fdbf24be640 1 --2- 172.21.15.103:0/545210528 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdbd40a32d0 0x7fdbd40a5790 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:13.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.501+0000 7fdbf24be640 1 --2- 172.21.15.103:0/545210528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbec106a80 0x7fdbec1a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:13.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.501+0000 7fdbf24be640 1 --2- 172.21.15.103:0/545210528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbec101420 0x7fdbec199790 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:13.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.501+0000 7fdbf24be640 1 --2- 172.21.15.103:0/545210528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbec100a80 0x7fdbec199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:13.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.501+0000 7fdbf24be640 1 -- 172.21.15.103:0/545210528 >> 172.21.15.103:0/545210528 conn(0x7fdbec0fc7a0 msgr2=0x7fdbec0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:13.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.501+0000 7fdbf24be640 1 -- 172.21.15.103:0/545210528 shutdown_connections 2024-09-19T06:48:13.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:13.501+0000 7fdbf24be640 1 -- 172.21.15.103:0/545210528 wait complete. 2024-09-19T06:48:13.823 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:13 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/545210528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T06:48:13.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:13 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/545210528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T06:48:13.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:13 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/545210528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T06:48:14.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.032+0000 7f3fd477f640 1 -- 172.21.15.103:0/367532581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fcc100a80 msgr2=0x7f3fcc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:14.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.032+0000 7f3fd477f640 1 --2- 172.21.15.103:0/367532581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fcc100a80 0x7f3fcc100ee0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009930 tx=0x7f3fc802a420 comp rx=0 tx=0).stop 2024-09-19T06:48:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.032+0000 7f3fd477f640 1 -- 172.21.15.103:0/367532581 shutdown_connections 2024-09-19T06:48:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.032+0000 7f3fd477f640 1 --2- 172.21.15.103:0/367532581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fcc101420 0x7f3fcc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.032+0000 7f3fd477f640 1 --2- 172.21.15.103:0/367532581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fcc100a80 0x7f3fcc100ee0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.032+0000 7f3fd477f640 1 --2- 172.21.15.103:0/367532581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fcc106a80 0x7f3fcc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.032+0000 7f3fd477f640 1 -- 172.21.15.103:0/367532581 >> 172.21.15.103:0/367532581 conn(0x7f3fcc0fc7a0 msgr2=0x7f3fcc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.032+0000 7f3fd477f640 1 -- 172.21.15.103:0/367532581 shutdown_connections 2024-09-19T06:48:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.032+0000 7f3fd477f640 1 -- 172.21.15.103:0/367532581 wait complete. 2024-09-19T06:48:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.032+0000 7f3fd477f640 1 Processor -- start 2024-09-19T06:48:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd477f640 1 -- start start 2024-09-19T06:48:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd477f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fcc100a80 0x7f3fcc0fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd477f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fcc101420 0x7f3fcc0fd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd477f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fcc106a80 0x7f3fcc0fd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd477f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc110ea0 con 0x7f3fcc101420 2024-09-19T06:48:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd477f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc110d20 con 0x7f3fcc100a80 2024-09-19T06:48:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd477f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc111020 con 0x7f3fcc106a80 2024-09-19T06:48:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd1d1a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fcc101420 0x7f3fcc0fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd1d1a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fcc101420 0x7f3fcc0fd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58306/0 (socket says 172.21.15.103:58306) 2024-09-19T06:48:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd1d1a640 1 -- 172.21.15.103:0/1562560496 learned_addr learned my addr 172.21.15.103:0/1562560496 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd251b640 1 --2- 172.21.15.103:0/1562560496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fcc100a80 0x7f3fcc0fcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd2d1c640 1 --2- 172.21.15.103:0/1562560496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fcc106a80 0x7f3fcc0fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd1d1a640 1 -- 172.21.15.103:0/1562560496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fcc106a80 msgr2=0x7f3fcc0fd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd1d1a640 1 --2- 172.21.15.103:0/1562560496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fcc106a80 0x7f3fcc0fd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd1d1a640 1 -- 172.21.15.103:0/1562560496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fcc100a80 msgr2=0x7f3fcc0fcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd1d1a640 1 --2- 172.21.15.103:0/1562560496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fcc100a80 0x7f3fcc0fcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd1d1a640 1 -- 172.21.15.103:0/1562560496 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fcc1a6980 con 0x7f3fcc101420 2024-09-19T06:48:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.033+0000 7f3fd1d1a640 1 --2- 172.21.15.103:0/1562560496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fcc101420 0x7f3fcc0fd250 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009a50 tx=0x7f3fc8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.034+0000 7f3fbf7fe640 1 -- 172.21.15.103:0/1562560496 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc803f030 con 0x7f3fcc101420 2024-09-19T06:48:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.034+0000 7f3fbf7fe640 1 -- 172.21.15.103:0/1562560496 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fc8003ad0 con 0x7f3fcc101420 2024-09-19T06:48:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.034+0000 7f3fbf7fe640 1 -- 172.21.15.103:0/1562560496 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8035070 con 0x7f3fcc101420 2024-09-19T06:48:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.034+0000 7f3fd477f640 1 -- 172.21.15.103:0/1562560496 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fcc1a6c10 con 0x7f3fcc101420 2024-09-19T06:48:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.034+0000 7f3fd477f640 1 -- 172.21.15.103:0/1562560496 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fcc1a7140 con 0x7f3fcc101420 2024-09-19T06:48:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.035+0000 7f3fd477f640 1 -- 172.21.15.103:0/1562560496 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fcc100ee0 con 0x7f3fcc101420 2024-09-19T06:48:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.036+0000 7f3fbf7fe640 1 -- 172.21.15.103:0/1562560496 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3fc8042030 con 0x7f3fcc101420 2024-09-19T06:48:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.036+0000 7f3fbf7fe640 1 --2- 172.21.15.103:0/1562560496 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3fb80a3350 0x7f3fb80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.037+0000 7f3fd251b640 1 --2- 172.21.15.103:0/1562560496 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3fb80a3350 0x7f3fb80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.037+0000 7f3fbf7fe640 1 -- 172.21.15.103:0/1562560496 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3fc80f0570 con 0x7f3fcc101420 2024-09-19T06:48:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.037+0000 7f3fd251b640 1 --2- 172.21.15.103:0/1562560496 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3fb80a3350 0x7f3fb80a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000abe0 tx=0x7f3fc000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.039+0000 7f3fbf7fe640 1 -- 172.21.15.103:0/1562560496 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3fc80bd140 con 0x7f3fcc101420 2024-09-19T06:48:14.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.211+0000 7f3fd477f640 1 -- 172.21.15.103:0/1562560496 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f3fcc10a020 con 0x7f3fcc101420 2024-09-19T06:48:14.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.211+0000 7f3fbf7fe640 1 -- 172.21.15.103:0/1562560496 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3fc80bca40 con 0x7f3fcc101420 2024-09-19T06:48:14.213 INFO:teuthology.orchestra.run.smithi103.stdout:266287972385 2024-09-19T06:48:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.213+0000 7f3fd477f640 1 -- 172.21.15.103:0/1562560496 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3fb80a3350 msgr2=0x7f3fb80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.213+0000 7f3fd477f640 1 --2- 172.21.15.103:0/1562560496 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3fb80a3350 0x7f3fb80a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000abe0 tx=0x7f3fc000a2d0 comp rx=0 tx=0).stop 2024-09-19T06:48:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.213+0000 7f3fd477f640 1 -- 172.21.15.103:0/1562560496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fcc101420 msgr2=0x7f3fcc0fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.213+0000 7f3fd477f640 1 --2- 172.21.15.103:0/1562560496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fcc101420 0x7f3fcc0fd250 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009a50 tx=0x7f3fc8004270 comp rx=0 tx=0).stop 2024-09-19T06:48:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.213+0000 7f3fd477f640 1 -- 172.21.15.103:0/1562560496 shutdown_connections 2024-09-19T06:48:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.213+0000 7f3fd477f640 1 --2- 172.21.15.103:0/1562560496 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3fb80a3350 0x7f3fb80a5810 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.213+0000 7f3fd477f640 1 --2- 172.21.15.103:0/1562560496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fcc106a80 0x7f3fcc0fd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.213+0000 7f3fd477f640 1 --2- 172.21.15.103:0/1562560496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fcc101420 0x7f3fcc0fd250 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.213+0000 7f3fd477f640 1 --2- 172.21.15.103:0/1562560496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fcc100a80 0x7f3fcc0fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.213+0000 7f3fd477f640 1 -- 172.21.15.103:0/1562560496 >> 172.21.15.103:0/1562560496 conn(0x7f3fcc0fc7a0 msgr2=0x7f3fcc06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.213+0000 7f3fd477f640 1 -- 172.21.15.103:0/1562560496 shutdown_connections 2024-09-19T06:48:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:14.213+0000 7f3fd477f640 1 -- 172.21.15.103:0/1562560496 wait complete. 2024-09-19T06:48:14.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:14 smithi174 ceph-mon[35637]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T06:48:14.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:14 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1562560496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T06:48:14.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:14 smithi103 ceph-mon[31158]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T06:48:14.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:14 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1562560496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T06:48:14.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:14 smithi134 ceph-mon[35356]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T06:48:14.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:14 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1562560496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T06:48:15.160 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972383 got 266287972385 for osd.9 2024-09-19T06:48:15.161 DEBUG:teuthology.parallel:result is None 2024-09-19T06:48:15.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.802+0000 7f7b2b292640 1 -- 172.21.15.103:0/3750108972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b24108290 msgr2=0x7f7b2410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:15.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.802+0000 7f7b2b292640 1 --2- 172.21.15.103:0/3750108972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b24108290 0x7f7b2410ec30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c009930 tx=0x7f7b1c02a420 comp rx=0 tx=0).stop 2024-09-19T06:48:15.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.802+0000 7f7b2b292640 1 -- 172.21.15.103:0/3750108972 shutdown_connections 2024-09-19T06:48:15.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.802+0000 7f7b2b292640 1 --2- 172.21.15.103:0/3750108972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b24108290 0x7f7b2410ec30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:15.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.802+0000 7f7b2b292640 1 --2- 172.21.15.103:0/3750108972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b24103740 0x7f7b24107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:15.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.802+0000 7f7b2b292640 1 --2- 172.21.15.103:0/3750108972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b24102d90 0x7f7b24103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:15.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.802+0000 7f7b2b292640 1 -- 172.21.15.103:0/3750108972 >> 172.21.15.103:0/3750108972 conn(0x7f7b240fc7d0 msgr2=0x7f7b240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:15.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.803+0000 7f7b2b292640 1 -- 172.21.15.103:0/3750108972 shutdown_connections 2024-09-19T06:48:15.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.803+0000 7f7b2b292640 1 -- 172.21.15.103:0/3750108972 wait complete. 2024-09-19T06:48:15.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.803+0000 7f7b2b292640 1 Processor -- start 2024-09-19T06:48:15.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.803+0000 7f7b2b292640 1 -- start start 2024-09-19T06:48:15.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2b292640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b24102d90 0x7f7b241985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:15.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2b292640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b24103740 0x7f7b24198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2b292640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b24108290 0x7f7b2419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2b292640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b241120d0 con 0x7f7b24108290 2024-09-19T06:48:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2b292640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b24111f50 con 0x7f7b24102d90 2024-09-19T06:48:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2b292640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b24112250 con 0x7f7b24103740 2024-09-19T06:48:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2982f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b24108290 0x7f7b2419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2982f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b24108290 0x7f7b2419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58334/0 (socket says 172.21.15.103:58334) 2024-09-19T06:48:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2982f640 1 -- 172.21.15.103:0/1881111733 learned_addr learned my addr 172.21.15.103:0/1881111733 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2902e640 1 --2- 172.21.15.103:0/1881111733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b24102d90 0x7f7b241985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2882d640 1 --2- 172.21.15.103:0/1881111733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b24103740 0x7f7b24198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2982f640 1 -- 172.21.15.103:0/1881111733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b24103740 msgr2=0x7f7b24198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2982f640 1 --2- 172.21.15.103:0/1881111733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b24103740 0x7f7b24198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2982f640 1 -- 172.21.15.103:0/1881111733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b24102d90 msgr2=0x7f7b241985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2982f640 1 --2- 172.21.15.103:0/1881111733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b24102d90 0x7f7b241985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2982f640 1 -- 172.21.15.103:0/1881111733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2419d560 con 0x7f7b24108290 2024-09-19T06:48:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2982f640 1 --2- 172.21.15.103:0/1881111733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b24108290 0x7f7b2419ce80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c005ab0 tx=0x7f7b1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b227fc640 1 -- 172.21.15.103:0/1881111733 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c03f030 con 0x7f7b24108290 2024-09-19T06:48:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b227fc640 1 -- 172.21.15.103:0/1881111733 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b1c003ce0 con 0x7f7b24108290 2024-09-19T06:48:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b227fc640 1 -- 172.21.15.103:0/1881111733 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c035070 con 0x7f7b24108290 2024-09-19T06:48:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.804+0000 7f7b2b292640 1 -- 172.21.15.103:0/1881111733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2419d7f0 con 0x7f7b24108290 2024-09-19T06:48:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.805+0000 7f7b2b292640 1 -- 172.21.15.103:0/1881111733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b241a5130 con 0x7f7b24108290 2024-09-19T06:48:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.805+0000 7f7b2b292640 1 -- 172.21.15.103:0/1881111733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b2406a7f0 con 0x7f7b24108290 2024-09-19T06:48:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.807+0000 7f7b227fc640 1 -- 172.21.15.103:0/1881111733 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7b1c042030 con 0x7f7b24108290 2024-09-19T06:48:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.807+0000 7f7b227fc640 1 --2- 172.21.15.103:0/1881111733 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f7b0c0a3350 0x7f7b0c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.808+0000 7f7b2902e640 1 --2- 172.21.15.103:0/1881111733 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f7b0c0a3350 0x7f7b0c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.808+0000 7f7b227fc640 1 -- 172.21.15.103:0/1881111733 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7b1c0f0380 con 0x7f7b24108290 2024-09-19T06:48:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.808+0000 7f7b2902e640 1 --2- 172.21.15.103:0/1881111733 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f7b0c0a3350 0x7f7b0c0a5810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400a400 tx=0x7f7b1400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.811+0000 7f7b227fc640 1 -- 172.21.15.103:0/1881111733 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7b1c0bcef0 con 0x7f7b24108290 2024-09-19T06:48:15.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.988+0000 7f7b2b292640 1 -- 172.21.15.103:0/1881111733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7b24103170 con 0x7f7b24108290 2024-09-19T06:48:15.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.988+0000 7f7b227fc640 1 -- 172.21.15.103:0/1881111733 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7b24103170 con 0x7f7b24108290 2024-09-19T06:48:15.990 INFO:teuthology.orchestra.run.smithi103.stdout:180388626480 2024-09-19T06:48:15.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.991+0000 7f7b2b292640 1 -- 172.21.15.103:0/1881111733 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f7b0c0a3350 msgr2=0x7f7b0c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:15.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.991+0000 7f7b2b292640 1 --2- 172.21.15.103:0/1881111733 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f7b0c0a3350 0x7f7b0c0a5810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400a400 tx=0x7f7b1400d040 comp rx=0 tx=0).stop 2024-09-19T06:48:15.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.991+0000 7f7b2b292640 1 -- 172.21.15.103:0/1881111733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b24108290 msgr2=0x7f7b2419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:15.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.991+0000 7f7b2b292640 1 --2- 172.21.15.103:0/1881111733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b24108290 0x7f7b2419ce80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c005ab0 tx=0x7f7b1c0042f0 comp rx=0 tx=0).stop 2024-09-19T06:48:15.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.991+0000 7f7b2b292640 1 -- 172.21.15.103:0/1881111733 shutdown_connections 2024-09-19T06:48:15.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.991+0000 7f7b2b292640 1 --2- 172.21.15.103:0/1881111733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b24108290 0x7f7b2419ce80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:15.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.991+0000 7f7b2b292640 1 --2- 172.21.15.103:0/1881111733 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f7b0c0a3350 0x7f7b0c0a5810 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:15.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.991+0000 7f7b2b292640 1 --2- 172.21.15.103:0/1881111733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b24103740 0x7f7b24198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:15.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.991+0000 7f7b2b292640 1 --2- 172.21.15.103:0/1881111733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b24102d90 0x7f7b241985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:15.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.991+0000 7f7b2b292640 1 -- 172.21.15.103:0/1881111733 >> 172.21.15.103:0/1881111733 conn(0x7f7b240fc7d0 msgr2=0x7f7b24106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:15.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.991+0000 7f7b2b292640 1 -- 172.21.15.103:0/1881111733 shutdown_connections 2024-09-19T06:48:15.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:15.991+0000 7f7b2b292640 1 -- 172.21.15.103:0/1881111733 wait complete. 2024-09-19T06:48:16.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.474+0000 7f06446d2640 1 -- 172.21.15.103:0/523365293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f063c102570 msgr2=0x7f063c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:16.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.474+0000 7f06446d2640 1 --2- 172.21.15.103:0/523365293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f063c102570 0x7f063c1029b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0630009900 tx=0x7f063002a480 comp rx=0 tx=0).stop 2024-09-19T06:48:16.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.474+0000 7f06446d2640 1 -- 172.21.15.103:0/523365293 shutdown_connections 2024-09-19T06:48:16.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.474+0000 7f06446d2640 1 --2- 172.21.15.103:0/523365293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f063c102ef0 0x7f063c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.474+0000 7f06446d2640 1 --2- 172.21.15.103:0/523365293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f063c102570 0x7f063c1029b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.474+0000 7f06446d2640 1 --2- 172.21.15.103:0/523365293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f063c100e20 0x7f063c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.474+0000 7f06446d2640 1 -- 172.21.15.103:0/523365293 >> 172.21.15.103:0/523365293 conn(0x7f063c0fc7d0 msgr2=0x7f063c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:16.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.475+0000 7f06446d2640 1 -- 172.21.15.103:0/523365293 shutdown_connections 2024-09-19T06:48:16.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.475+0000 7f06446d2640 1 -- 172.21.15.103:0/523365293 wait complete. 2024-09-19T06:48:16.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.475+0000 7f06446d2640 1 Processor -- start 2024-09-19T06:48:16.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.475+0000 7f06446d2640 1 -- start start 2024-09-19T06:48:16.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.475+0000 7f06446d2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f063c100e20 0x7f063c199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:16.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.475+0000 7f06446d2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f063c102570 0x7f063c1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:16.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.475+0000 7f06446d2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f063c102ef0 0x7f063c1a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:16.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.475+0000 7f06446d2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f063c110fa0 con 0x7f063c102570 2024-09-19T06:48:16.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.475+0000 7f06446d2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f063c110e20 con 0x7f063c102ef0 2024-09-19T06:48:16.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.475+0000 7f06446d2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f063c111120 con 0x7f063c100e20 2024-09-19T06:48:16.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.475+0000 7f0641c6d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f063c102570 0x7f063c1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:16.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.475+0000 7f0641c6d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f063c102570 0x7f063c1998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58350/0 (socket says 172.21.15.103:58350) 2024-09-19T06:48:16.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.475+0000 7f0641c6d640 1 -- 172.21.15.103:0/1237266269 learned_addr learned my addr 172.21.15.103:0/1237266269 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:16.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.476+0000 7f0642c6f640 1 --2- 172.21.15.103:0/1237266269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f063c102ef0 0x7f063c1a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:16.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.476+0000 7f064246e640 1 --2- 172.21.15.103:0/1237266269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f063c100e20 0x7f063c199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:16.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.476+0000 7f0641c6d640 1 -- 172.21.15.103:0/1237266269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f063c100e20 msgr2=0x7f063c199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:16.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.476+0000 7f0641c6d640 1 --2- 172.21.15.103:0/1237266269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f063c100e20 0x7f063c199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.476+0000 7f0641c6d640 1 -- 172.21.15.103:0/1237266269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f063c102ef0 msgr2=0x7f063c1a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:16.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.476+0000 7f0641c6d640 1 --2- 172.21.15.103:0/1237266269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f063c102ef0 0x7f063c1a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.476+0000 7f0641c6d640 1 -- 172.21.15.103:0/1237266269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c1a4950 con 0x7f063c102570 2024-09-19T06:48:16.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.476+0000 7f0641c6d640 1 --2- 172.21.15.103:0/1237266269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f063c102570 0x7f063c1998d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0630006010 tx=0x7f0630003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:16.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.476+0000 7f063b7fe640 1 -- 172.21.15.103:0/1237266269 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063003f070 con 0x7f063c102570 2024-09-19T06:48:16.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.476+0000 7f063b7fe640 1 -- 172.21.15.103:0/1237266269 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0630031e30 con 0x7f063c102570 2024-09-19T06:48:16.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.476+0000 7f06446d2640 1 -- 172.21.15.103:0/1237266269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063c1a4b20 con 0x7f063c102570 2024-09-19T06:48:16.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.476+0000 7f063b7fe640 1 -- 172.21.15.103:0/1237266269 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630035030 con 0x7f063c102570 2024-09-19T06:48:16.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.476+0000 7f06446d2640 1 -- 172.21.15.103:0/1237266269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f063c1a4f70 con 0x7f063c102570 2024-09-19T06:48:16.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.477+0000 7f06446d2640 1 -- 172.21.15.103:0/1237266269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063c101200 con 0x7f063c102570 2024-09-19T06:48:16.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.478+0000 7f063b7fe640 1 -- 172.21.15.103:0/1237266269 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0630042070 con 0x7f063c102570 2024-09-19T06:48:16.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.479+0000 7f063b7fe640 1 --2- 172.21.15.103:0/1237266269 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f06240a32d0 0x7f06240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:16.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.479+0000 7f063b7fe640 1 -- 172.21.15.103:0/1237266269 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f06300f09d0 con 0x7f063c102570 2024-09-19T06:48:16.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.479+0000 7f064246e640 1 --2- 172.21.15.103:0/1237266269 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f06240a32d0 0x7f06240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:16.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.479+0000 7f064246e640 1 --2- 172.21.15.103:0/1237266269 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f06240a32d0 0x7f06240a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f062c005c40 tx=0x7f062c00a780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:16.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.482+0000 7f063b7fe640 1 -- 172.21.15.103:0/1237266269 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f06300bd630 con 0x7f063c102570 2024-09-19T06:48:16.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.653+0000 7f06446d2640 1 -- 172.21.15.103:0/1237266269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f063c109090 con 0x7f063c102570 2024-09-19T06:48:16.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.654+0000 7f063b7fe640 1 -- 172.21.15.103:0/1237266269 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f06300bcf30 con 0x7f063c102570 2024-09-19T06:48:16.655 INFO:teuthology.orchestra.run.smithi103.stdout:201863462954 2024-09-19T06:48:16.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.656+0000 7f06446d2640 1 -- 172.21.15.103:0/1237266269 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f06240a32d0 msgr2=0x7f06240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:16.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.656+0000 7f06446d2640 1 --2- 172.21.15.103:0/1237266269 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f06240a32d0 0x7f06240a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f062c005c40 tx=0x7f062c00a780 comp rx=0 tx=0).stop 2024-09-19T06:48:16.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.656+0000 7f06446d2640 1 -- 172.21.15.103:0/1237266269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f063c102570 msgr2=0x7f063c1998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:16.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.656+0000 7f06446d2640 1 --2- 172.21.15.103:0/1237266269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f063c102570 0x7f063c1998d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0630006010 tx=0x7f0630003c80 comp rx=0 tx=0).stop 2024-09-19T06:48:16.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.656+0000 7f06446d2640 1 -- 172.21.15.103:0/1237266269 shutdown_connections 2024-09-19T06:48:16.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.656+0000 7f06446d2640 1 --2- 172.21.15.103:0/1237266269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f063c102ef0 0x7f063c1a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.656+0000 7f06446d2640 1 --2- 172.21.15.103:0/1237266269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f063c102570 0x7f063c1998d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.656+0000 7f06446d2640 1 --2- 172.21.15.103:0/1237266269 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f06240a32d0 0x7f06240a5790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.656+0000 7f06446d2640 1 --2- 172.21.15.103:0/1237266269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f063c100e20 0x7f063c199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.656+0000 7f06446d2640 1 -- 172.21.15.103:0/1237266269 >> 172.21.15.103:0/1237266269 conn(0x7f063c0fc7d0 msgr2=0x7f063c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:16.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.656+0000 7f06446d2640 1 -- 172.21.15.103:0/1237266269 shutdown_connections 2024-09-19T06:48:16.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.656+0000 7f06446d2640 1 -- 172.21.15.103:0/1237266269 wait complete. 2024-09-19T06:48:16.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.689+0000 7fab8aefd640 1 -- 172.21.15.103:0/1733799713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab84102570 msgr2=0x7fab841029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:16.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.689+0000 7fab8aefd640 1 --2- 172.21.15.103:0/1733799713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab84102570 0x7fab841029b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fab70009900 tx=0x7fab7002a480 comp rx=0 tx=0).stop 2024-09-19T06:48:16.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.689+0000 7fab8aefd640 1 -- 172.21.15.103:0/1733799713 shutdown_connections 2024-09-19T06:48:16.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.689+0000 7fab8aefd640 1 --2- 172.21.15.103:0/1733799713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab84102ef0 0x7fab8410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.689+0000 7fab8aefd640 1 --2- 172.21.15.103:0/1733799713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab84102570 0x7fab841029b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.689+0000 7fab8aefd640 1 --2- 172.21.15.103:0/1733799713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab84100e20 0x7fab84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.689+0000 7fab8aefd640 1 -- 172.21.15.103:0/1733799713 >> 172.21.15.103:0/1733799713 conn(0x7fab840fc7d0 msgr2=0x7fab840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:16.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.689+0000 7fab8aefd640 1 -- 172.21.15.103:0/1733799713 shutdown_connections 2024-09-19T06:48:16.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.689+0000 7fab8aefd640 1 -- 172.21.15.103:0/1733799713 wait complete. 2024-09-19T06:48:16.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.690+0000 7fab8aefd640 1 Processor -- start 2024-09-19T06:48:16.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.690+0000 7fab8aefd640 1 -- start start 2024-09-19T06:48:16.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.690+0000 7fab8aefd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab84100e20 0x7fab840fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:16.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.690+0000 7fab8aefd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab84102570 0x7fab840fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:16.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.690+0000 7fab8aefd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab84102ef0 0x7fab840fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:16.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.690+0000 7fab8aefd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fab84110f80 con 0x7fab84102570 2024-09-19T06:48:16.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.690+0000 7fab8aefd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab84110e00 con 0x7fab84102ef0 2024-09-19T06:48:16.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.690+0000 7fab8aefd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fab84111100 con 0x7fab84100e20 2024-09-19T06:48:16.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.690+0000 7fab83fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab84102570 0x7fab840fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:16.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.690+0000 7fab83fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab84102570 0x7fab840fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58372/0 (socket says 172.21.15.103:58372) 2024-09-19T06:48:16.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.690+0000 7fab83fff640 1 -- 172.21.15.103:0/396310869 learned_addr learned my addr 172.21.15.103:0/396310869 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:16.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.690+0000 7fab88c99640 1 --2- 172.21.15.103:0/396310869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab84100e20 0x7fab840fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:16.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.691+0000 7fab83fff640 1 -- 172.21.15.103:0/396310869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab84100e20 msgr2=0x7fab840fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:16.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.691+0000 7fab83fff640 1 --2- 172.21.15.103:0/396310869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab84100e20 0x7fab840fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.691+0000 7fab83fff640 1 -- 172.21.15.103:0/396310869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab84102ef0 msgr2=0x7fab840fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:16.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.691+0000 7fab8949a640 1 --2- 172.21.15.103:0/396310869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab84102ef0 0x7fab840fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:16.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.691+0000 7fab83fff640 1 --2- 172.21.15.103:0/396310869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab84102ef0 0x7fab840fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.691+0000 7fab83fff640 1 -- 172.21.15.103:0/396310869 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab841a69a0 con 0x7fab84102570 2024-09-19T06:48:16.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.691+0000 7fab8949a640 1 --2- 172.21.15.103:0/396310869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab84102ef0 0x7fab840fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:16.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.691+0000 7fab83fff640 1 --2- 172.21.15.103:0/396310869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab84102570 0x7fab840fd1c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fab70006010 tx=0x7fab70003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:16.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.691+0000 7fab81ffb640 1 -- 172.21.15.103:0/396310869 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7003f070 con 0x7fab84102570 2024-09-19T06:48:16.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.691+0000 7fab81ffb640 1 -- 172.21.15.103:0/396310869 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab70031e30 con 0x7fab84102570 2024-09-19T06:48:16.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.691+0000 7fab81ffb640 1 -- 172.21.15.103:0/396310869 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70035030 con 0x7fab84102570 2024-09-19T06:48:16.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.691+0000 7fab8aefd640 1 -- 172.21.15.103:0/396310869 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab841a6c30 con 0x7fab84102570 2024-09-19T06:48:16.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.691+0000 7fab8aefd640 1 -- 172.21.15.103:0/396310869 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab841a7160 con 0x7fab84102570 2024-09-19T06:48:16.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.692+0000 7fab8aefd640 1 -- 172.21.15.103:0/396310869 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab84101200 con 0x7fab84102570 2024-09-19T06:48:16.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.693+0000 7fab81ffb640 1 -- 172.21.15.103:0/396310869 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fab70042070 con 0x7fab84102570 2024-09-19T06:48:16.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.694+0000 7fab81ffb640 1 --2- 172.21.15.103:0/396310869 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fab740a3350 0x7fab740a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:16.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.694+0000 7fab88c99640 1 --2- 172.21.15.103:0/396310869 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fab740a3350 0x7fab740a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:16.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.694+0000 7fab81ffb640 1 -- 172.21.15.103:0/396310869 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fab700f09b0 con 0x7fab84102570 2024-09-19T06:48:16.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.694+0000 7fab88c99640 1 --2- 172.21.15.103:0/396310869 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fab740a3350 0x7fab740a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fab78005c40 tx=0x7fab7800a780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:16.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.697+0000 7fab81ffb640 1 -- 172.21.15.103:0/396310869 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fab700bd610 con 0x7fab84102570 2024-09-19T06:48:16.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.885+0000 7fab8aefd640 1 -- 172.21.15.103:0/396310869 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fab8410a0a0 con 0x7fab84102570 2024-09-19T06:48:16.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.888+0000 7fab81ffb640 1 -- 172.21.15.103:0/396310869 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fab700bcf10 con 0x7fab84102570 2024-09-19T06:48:16.889 INFO:teuthology.orchestra.run.smithi103.stdout:317827579927 2024-09-19T06:48:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.891+0000 7fab8aefd640 1 -- 172.21.15.103:0/396310869 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fab740a3350 msgr2=0x7fab740a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.891+0000 7fab8aefd640 1 --2- 172.21.15.103:0/396310869 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fab740a3350 0x7fab740a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fab78005c40 tx=0x7fab7800a780 comp rx=0 tx=0).stop 2024-09-19T06:48:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.891+0000 7fab8aefd640 1 -- 172.21.15.103:0/396310869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab84102570 msgr2=0x7fab840fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.891+0000 7fab8aefd640 1 --2- 172.21.15.103:0/396310869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab84102570 0x7fab840fd1c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fab70006010 tx=0x7fab70003c80 comp rx=0 tx=0).stop 2024-09-19T06:48:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.891+0000 7fab8aefd640 1 -- 172.21.15.103:0/396310869 shutdown_connections 2024-09-19T06:48:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.891+0000 7fab8aefd640 1 --2- 172.21.15.103:0/396310869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab84102ef0 0x7fab840fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.891+0000 7fab8aefd640 1 --2- 172.21.15.103:0/396310869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab84102570 0x7fab840fd1c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.891+0000 7fab8aefd640 1 --2- 172.21.15.103:0/396310869 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fab740a3350 0x7fab740a5810 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.891+0000 7fab8aefd640 1 --2- 172.21.15.103:0/396310869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab84100e20 0x7fab840fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:16.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.891+0000 7fab8aefd640 1 -- 172.21.15.103:0/396310869 >> 172.21.15.103:0/396310869 conn(0x7fab840fc7d0 msgr2=0x7fab8406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:16.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.891+0000 7fab8aefd640 1 -- 172.21.15.103:0/396310869 shutdown_connections 2024-09-19T06:48:16.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:16.891+0000 7fab8aefd640 1 -- 172.21.15.103:0/396310869 wait complete. 2024-09-19T06:48:16.943 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:16 smithi103 ceph-mon[31158]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-19T06:48:16.944 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:16 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1881111733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T06:48:16.944 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:16 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1237266269' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T06:48:17.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.125+0000 7fefa6b6e640 1 -- 172.21.15.103:0/365054981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefa0102790 msgr2=0x7fefa01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:17.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.125+0000 7fefa6b6e640 1 --2- 172.21.15.103:0/365054981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefa0102790 0x7fefa01067e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009900 tx=0x7fef8c02a3f0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.125+0000 7fefa6b6e640 1 -- 172.21.15.103:0/365054981 shutdown_connections 2024-09-19T06:48:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.125+0000 7fefa6b6e640 1 --2- 172.21.15.103:0/365054981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefa0106f10 0x7fefa010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.125+0000 7fefa6b6e640 1 --2- 172.21.15.103:0/365054981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefa0102790 0x7fefa01067e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.125+0000 7fefa6b6e640 1 --2- 172.21.15.103:0/365054981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefa0101de0 0x7fefa01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.125+0000 7fefa6b6e640 1 -- 172.21.15.103:0/365054981 >> 172.21.15.103:0/365054981 conn(0x7fefa00fd8c0 msgr2=0x7fefa00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.125+0000 7fefa6b6e640 1 -- 172.21.15.103:0/365054981 shutdown_connections 2024-09-19T06:48:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.125+0000 7fefa6b6e640 1 -- 172.21.15.103:0/365054981 wait complete. 2024-09-19T06:48:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.126+0000 7fefa6b6e640 1 Processor -- start 2024-09-19T06:48:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.126+0000 7fefa6b6e640 1 -- start start 2024-09-19T06:48:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.126+0000 7fefa6b6e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefa0101de0 0x7fefa0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.126+0000 7fefa6b6e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefa0102790 0x7fefa0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.126+0000 7fefa6b6e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefa0106f10 0x7fefa019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.126+0000 7fefa6b6e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fefa0111f70 con 0x7fefa0106f10 2024-09-19T06:48:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.126+0000 7fefa6b6e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fefa0111df0 con 0x7fefa0102790 2024-09-19T06:48:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.126+0000 7fefa6b6e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fefa01120f0 con 0x7fefa0101de0 2024-09-19T06:48:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.126+0000 7fefa510b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefa0106f10 0x7fefa019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.126+0000 7fefa510b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefa0106f10 0x7fefa019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58382/0 (socket says 172.21.15.103:58382) 2024-09-19T06:48:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.126+0000 7fefa510b640 1 -- 172.21.15.103:0/2155589982 learned_addr learned my addr 172.21.15.103:0/2155589982 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.126+0000 7fefa510b640 1 -- 172.21.15.103:0/2155589982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefa0101de0 msgr2=0x7fefa0198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.126+0000 7fefa510b640 1 --2- 172.21.15.103:0/2155589982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefa0101de0 0x7fefa0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.126+0000 7fefa510b640 1 -- 172.21.15.103:0/2155589982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefa0102790 msgr2=0x7fefa0198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.126+0000 7fefa510b640 1 --2- 172.21.15.103:0/2155589982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefa0102790 0x7fefa0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.126+0000 7fefa510b640 1 -- 172.21.15.103:0/2155589982 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa019d2f0 con 0x7fefa0106f10 2024-09-19T06:48:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.127+0000 7fefa510b640 1 --2- 172.21.15.103:0/2155589982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefa0106f10 0x7fefa019cc10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fef9800cce0 tx=0x7fef9800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.127+0000 7fef9dffb640 1 -- 172.21.15.103:0/2155589982 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9800f450 con 0x7fefa0106f10 2024-09-19T06:48:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.127+0000 7fef9dffb640 1 -- 172.21.15.103:0/2155589982 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef9800fad0 con 0x7fefa0106f10 2024-09-19T06:48:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.127+0000 7fefa6b6e640 1 -- 172.21.15.103:0/2155589982 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa019d5e0 con 0x7fefa0106f10 2024-09-19T06:48:17.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.127+0000 7fef9dffb640 1 -- 172.21.15.103:0/2155589982 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98018c10 con 0x7fefa0106f10 2024-09-19T06:48:17.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.127+0000 7fefa6b6e640 1 -- 172.21.15.103:0/2155589982 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa01a4ec0 con 0x7fefa0106f10 2024-09-19T06:48:17.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.128+0000 7fefa6b6e640 1 -- 172.21.15.103:0/2155589982 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa01021c0 con 0x7fefa0106f10 2024-09-19T06:48:17.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.129+0000 7fef9dffb640 1 -- 172.21.15.103:0/2155589982 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fef9800f5f0 con 0x7fefa0106f10 2024-09-19T06:48:17.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.130+0000 7fef9dffb640 1 --2- 172.21.15.103:0/2155589982 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fef900a3350 0x7fef900a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:17.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.130+0000 7fefa490a640 1 --2- 172.21.15.103:0/2155589982 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fef900a3350 0x7fef900a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:17.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.130+0000 7fef9dffb640 1 -- 172.21.15.103:0/2155589982 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fef980d00c0 con 0x7fefa0106f10 2024-09-19T06:48:17.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.130+0000 7fefa490a640 1 --2- 172.21.15.103:0/2155589982 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fef900a3350 0x7fef900a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fef9400a990 tx=0x7fef94005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:17.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.134+0000 7fef9dffb640 1 -- 172.21.15.103:0/2155589982 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fef98014030 con 0x7fefa0106f10 2024-09-19T06:48:17.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:16 smithi134 ceph-mon[35356]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-19T06:48:17.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:16 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1881111733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T06:48:17.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:16 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1237266269' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T06:48:17.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.307+0000 7fefa6b6e640 1 -- 172.21.15.103:0/2155589982 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fefa010a1b0 con 0x7fefa0106f10 2024-09-19T06:48:17.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.307+0000 7fef9dffb640 1 -- 172.21.15.103:0/2155589982 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fef9809cc90 con 0x7fefa0106f10 2024-09-19T06:48:17.309 INFO:teuthology.orchestra.run.smithi103.stdout:55834574921 2024-09-19T06:48:17.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.309+0000 7fefa6b6e640 1 -- 172.21.15.103:0/2155589982 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fef900a3350 msgr2=0x7fef900a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:17.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.309+0000 7fefa6b6e640 1 --2- 172.21.15.103:0/2155589982 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fef900a3350 0x7fef900a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fef9400a990 tx=0x7fef94005c70 comp rx=0 tx=0).stop 2024-09-19T06:48:17.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.309+0000 7fefa6b6e640 1 -- 172.21.15.103:0/2155589982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefa0106f10 msgr2=0x7fefa019cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:17.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.309+0000 7fefa6b6e640 1 --2- 172.21.15.103:0/2155589982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefa0106f10 0x7fefa019cc10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fef9800cce0 tx=0x7fef9800a620 comp rx=0 tx=0).stop 2024-09-19T06:48:17.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.310+0000 7fefa6b6e640 1 -- 172.21.15.103:0/2155589982 shutdown_connections 2024-09-19T06:48:17.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.310+0000 7fefa6b6e640 1 --2- 172.21.15.103:0/2155589982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefa0106f10 0x7fefa019cc10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.310+0000 7fefa6b6e640 1 --2- 172.21.15.103:0/2155589982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefa0102790 0x7fefa0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.310+0000 7fefa6b6e640 1 --2- 172.21.15.103:0/2155589982 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fef900a3350 0x7fef900a5810 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.310+0000 7fefa6b6e640 1 --2- 172.21.15.103:0/2155589982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefa0101de0 0x7fefa0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.310+0000 7fefa6b6e640 1 -- 172.21.15.103:0/2155589982 >> 172.21.15.103:0/2155589982 conn(0x7fefa00fd8c0 msgr2=0x7fefa00febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:17.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.310+0000 7fefa6b6e640 1 -- 172.21.15.103:0/2155589982 shutdown_connections 2024-09-19T06:48:17.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.310+0000 7fefa6b6e640 1 -- 172.21.15.103:0/2155589982 wait complete. 2024-09-19T06:48:17.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:16 smithi174 ceph-mon[35637]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-19T06:48:17.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:16 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1881111733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T06:48:17.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:16 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1237266269' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T06:48:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.378+0000 7f88352cd640 1 -- 172.21.15.103:0/1725212209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f883010c170 msgr2=0x7f883010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.378+0000 7f88352cd640 1 --2- 172.21.15.103:0/1725212209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f883010c170 0x7f883010e610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8828009ca0 tx=0x7f882802a950 comp rx=0 tx=0).stop 2024-09-19T06:48:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.379+0000 7f88352cd640 1 -- 172.21.15.103:0/1725212209 shutdown_connections 2024-09-19T06:48:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.379+0000 7f88352cd640 1 --2- 172.21.15.103:0/1725212209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f883010c170 0x7f883010e610 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.379+0000 7f88352cd640 1 --2- 172.21.15.103:0/1725212209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8830103660 0x7f883010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.379+0000 7f88352cd640 1 --2- 172.21.15.103:0/1725212209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8830102d40 0x7f8830103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.379+0000 7f88352cd640 1 -- 172.21.15.103:0/1725212209 >> 172.21.15.103:0/1725212209 conn(0x7f88300fc7a0 msgr2=0x7f88300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.379+0000 7f88352cd640 1 -- 172.21.15.103:0/1725212209 shutdown_connections 2024-09-19T06:48:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.379+0000 7f88352cd640 1 -- 172.21.15.103:0/1725212209 wait complete. 2024-09-19T06:48:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.379+0000 7f88352cd640 1 Processor -- start 2024-09-19T06:48:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.379+0000 7f88352cd640 1 -- start start 2024-09-19T06:48:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.379+0000 7f88352cd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8830102d40 0x7f8830198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.379+0000 7f88352cd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8830103660 0x7f88301989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f88352cd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f883010c170 0x7f883019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f88352cd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8830111fd0 con 0x7f883010c170 2024-09-19T06:48:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f88352cd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8830111e50 con 0x7f8830103660 2024-09-19T06:48:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f88352cd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8830112150 con 0x7f8830102d40 2024-09-19T06:48:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f882f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f883010c170 0x7f883019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f882f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f883010c170 0x7f883019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43238/0 (socket says 172.21.15.103:43238) 2024-09-19T06:48:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f882f7fe640 1 -- 172.21.15.103:0/1652347016 learned_addr learned my addr 172.21.15.103:0/1652347016 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f882effd640 1 --2- 172.21.15.103:0/1652347016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8830102d40 0x7f8830198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f882e7fc640 1 --2- 172.21.15.103:0/1652347016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8830103660 0x7f88301989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f882f7fe640 1 -- 172.21.15.103:0/1652347016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8830102d40 msgr2=0x7f8830198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f882f7fe640 1 --2- 172.21.15.103:0/1652347016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8830102d40 0x7f8830198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.970 INFO:teuthology.orchestra.run.smithi103.stdout:34359738445 2024-09-19T06:48:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f882f7fe640 1 -- 172.21.15.103:0/1652347016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8830103660 msgr2=0x7f88301989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f882f7fe640 1 --2- 172.21.15.103:0/1652347016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8830103660 0x7f88301989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f882f7fe640 1 -- 172.21.15.103:0/1652347016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883019d440 con 0x7f883010c170 2024-09-19T06:48:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f882f7fe640 1 --2- 172.21.15.103:0/1652347016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f883010c170 0x7f883019cd60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f882802ae70 tx=0x7f882802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f8817fff640 1 -- 172.21.15.103:0/1652347016 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828034040 con 0x7f883010c170 2024-09-19T06:48:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f8817fff640 1 -- 172.21.15.103:0/1652347016 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8828030920 con 0x7f883010c170 2024-09-19T06:48:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f88352cd640 1 -- 172.21.15.103:0/1652347016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883019d6d0 con 0x7f883010c170 2024-09-19T06:48:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f8817fff640 1 -- 172.21.15.103:0/1652347016 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88280047b0 con 0x7f883010c170 2024-09-19T06:48:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.380+0000 7f88352cd640 1 -- 172.21.15.103:0/1652347016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88301a5010 con 0x7f883010c170 2024-09-19T06:48:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.381+0000 7f88352cd640 1 -- 172.21.15.103:0/1652347016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8830103120 con 0x7f883010c170 2024-09-19T06:48:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.383+0000 7f8817fff640 1 -- 172.21.15.103:0/1652347016 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8828037020 con 0x7f883010c170 2024-09-19T06:48:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.383+0000 7f8817fff640 1 --2- 172.21.15.103:0/1652347016 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f88180a3350 0x7f88180a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.383+0000 7f8817fff640 1 -- 172.21.15.103:0/1652347016 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f88280f0ab0 con 0x7f883010c170 2024-09-19T06:48:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.383+0000 7f882effd640 1 --2- 172.21.15.103:0/1652347016 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f88180a3350 0x7f88180a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.384+0000 7f882effd640 1 --2- 172.21.15.103:0/1652347016 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f88180a3350 0x7f88180a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8820009990 tx=0x7f8820009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.387+0000 7f8817fff640 1 -- 172.21.15.103:0/1652347016 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f88280bd790 con 0x7f883010c170 2024-09-19T06:48:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.558+0000 7f88352cd640 1 -- 172.21.15.103:0/1652347016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f88301997f0 con 0x7f883010c170 2024-09-19T06:48:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.558+0000 7f8817fff640 1 -- 172.21.15.103:0/1652347016 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f88301997f0 con 0x7f883010c170 2024-09-19T06:48:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.562+0000 7f88352cd640 1 -- 172.21.15.103:0/1652347016 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f88180a3350 msgr2=0x7f88180a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.562+0000 7f88352cd640 1 --2- 172.21.15.103:0/1652347016 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f88180a3350 0x7f88180a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8820009990 tx=0x7f8820009110 comp rx=0 tx=0).stop 2024-09-19T06:48:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.562+0000 7f88352cd640 1 -- 172.21.15.103:0/1652347016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f883010c170 msgr2=0x7f883019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.562+0000 7f88352cd640 1 --2- 172.21.15.103:0/1652347016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f883010c170 0x7f883019cd60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f882802ae70 tx=0x7f882802f5b0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.562+0000 7f88352cd640 1 -- 172.21.15.103:0/1652347016 shutdown_connections 2024-09-19T06:48:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.562+0000 7f88352cd640 1 --2- 172.21.15.103:0/1652347016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f883010c170 0x7f883019cd60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.562+0000 7f88352cd640 1 --2- 172.21.15.103:0/1652347016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8830103660 0x7f88301989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.562+0000 7f88352cd640 1 --2- 172.21.15.103:0/1652347016 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f88180a3350 0x7f88180a5810 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.562+0000 7f88352cd640 1 --2- 172.21.15.103:0/1652347016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8830102d40 0x7f8830198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.562+0000 7f88352cd640 1 -- 172.21.15.103:0/1652347016 >> 172.21.15.103:0/1652347016 conn(0x7f88300fc7a0 msgr2=0x7f88300fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.562+0000 7f88352cd640 1 -- 172.21.15.103:0/1652347016 shutdown_connections 2024-09-19T06:48:17.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:17.562+0000 7f88352cd640 1 -- 172.21.15.103:0/1652347016 wait complete. 2024-09-19T06:48:18.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:17 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/396310869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T06:48:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:17 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2155589982' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T06:48:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:17 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1652347016' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T06:48:18.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:17 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/396310869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T06:48:18.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:17 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2155589982' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T06:48:18.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:17 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1652347016' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T06:48:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:17 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/396310869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T06:48:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:17 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2155589982' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T06:48:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:17 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1652347016' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T06:48:18.508 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738442 got 34359738445 for osd.0 2024-09-19T06:48:18.508 DEBUG:teuthology.parallel:result is None 2024-09-19T06:48:18.838 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462952 got 201863462954 for osd.7 2024-09-19T06:48:18.838 DEBUG:teuthology.parallel:result is None 2024-09-19T06:48:19.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:18 smithi103 ceph-mon[31158]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T06:48:19.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:18 smithi134 ceph-mon[35356]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T06:48:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:18 smithi174 ceph-mon[35637]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T06:48:19.411 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579926 got 317827579927 for osd.11 2024-09-19T06:48:19.411 DEBUG:teuthology.parallel:result is None 2024-09-19T06:48:19.639 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:19.640 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:19.672 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444097 got 73014444099 for osd.2 2024-09-19T06:48:19.672 DEBUG:teuthology.parallel:result is None 2024-09-19T06:48:20.244 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574921 for osd.1 2024-09-19T06:48:20.244 DEBUG:teuthology.parallel:result is None 2024-09-19T06:48:20.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.853+0000 7f1991ac8640 1 -- 172.21.15.103:0/2687503913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f198c102570 msgr2=0x7f198c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:20.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.853+0000 7f1991ac8640 1 --2- 172.21.15.103:0/2687503913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f198c102570 0x7f198c1029b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1980009900 tx=0x7f198002a480 comp rx=0 tx=0).stop 2024-09-19T06:48:20.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.853+0000 7f1991ac8640 1 -- 172.21.15.103:0/2687503913 shutdown_connections 2024-09-19T06:48:20.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.853+0000 7f1991ac8640 1 --2- 172.21.15.103:0/2687503913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f198c102ef0 0x7f198c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:20.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.853+0000 7f1991ac8640 1 --2- 172.21.15.103:0/2687503913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f198c102570 0x7f198c1029b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:20.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.853+0000 7f1991ac8640 1 --2- 172.21.15.103:0/2687503913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f198c100e20 0x7f198c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:20.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.853+0000 7f1991ac8640 1 -- 172.21.15.103:0/2687503913 >> 172.21.15.103:0/2687503913 conn(0x7f198c0fc7d0 msgr2=0x7f198c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:20.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.853+0000 7f1991ac8640 1 -- 172.21.15.103:0/2687503913 shutdown_connections 2024-09-19T06:48:20.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.853+0000 7f1991ac8640 1 -- 172.21.15.103:0/2687503913 wait complete. 2024-09-19T06:48:20.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.854+0000 7f1991ac8640 1 Processor -- start 2024-09-19T06:48:20.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.854+0000 7f1991ac8640 1 -- start start 2024-09-19T06:48:20.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.854+0000 7f1991ac8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f198c100e20 0x7f198c197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:20.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.854+0000 7f1991ac8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f198c102570 0x7f198c197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:20.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.854+0000 7f1991ac8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f198c102ef0 0x7f198c197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:20.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.854+0000 7f1991ac8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f198c110f80 con 0x7f198c102570 2024-09-19T06:48:20.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.854+0000 7f1991ac8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f198c110e00 con 0x7f198c100e20 2024-09-19T06:48:20.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.854+0000 7f1991ac8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f198c111100 con 0x7f198c102ef0 2024-09-19T06:48:20.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.854+0000 7f198affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f198c102570 0x7f198c197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.854+0000 7f198affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f198c102570 0x7f198c197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43258/0 (socket says 172.21.15.103:43258) 2024-09-19T06:48:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.854+0000 7f198affd640 1 -- 172.21.15.103:0/2009470030 learned_addr learned my addr 172.21.15.103:0/2009470030 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.854+0000 7f198b7fe640 1 --2- 172.21.15.103:0/2009470030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f198c100e20 0x7f198c197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.854+0000 7f198bfff640 1 --2- 172.21.15.103:0/2009470030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f198c102ef0 0x7f198c197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.855+0000 7f198affd640 1 -- 172.21.15.103:0/2009470030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f198c102ef0 msgr2=0x7f198c197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.855+0000 7f198affd640 1 --2- 172.21.15.103:0/2009470030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f198c102ef0 0x7f198c197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.855+0000 7f198affd640 1 -- 172.21.15.103:0/2009470030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f198c100e20 msgr2=0x7f198c197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.855+0000 7f198affd640 1 --2- 172.21.15.103:0/2009470030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f198c100e20 0x7f198c197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.855+0000 7f198affd640 1 -- 172.21.15.103:0/2009470030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c1a6990 con 0x7f198c102570 2024-09-19T06:48:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.855+0000 7f198b7fe640 1 --2- 172.21.15.103:0/2009470030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f198c100e20 0x7f198c197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.855+0000 7f198affd640 1 --2- 172.21.15.103:0/2009470030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f198c102570 0x7f198c197660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1980006010 tx=0x7f1980003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.855+0000 7f1988ff9640 1 -- 172.21.15.103:0/2009470030 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198003f070 con 0x7f198c102570 2024-09-19T06:48:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.855+0000 7f1988ff9640 1 -- 172.21.15.103:0/2009470030 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1980031c30 con 0x7f198c102570 2024-09-19T06:48:20.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.855+0000 7f1991ac8640 1 -- 172.21.15.103:0/2009470030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198c1a6c20 con 0x7f198c102570 2024-09-19T06:48:20.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.855+0000 7f1988ff9640 1 -- 172.21.15.103:0/2009470030 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980035030 con 0x7f198c102570 2024-09-19T06:48:20.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.855+0000 7f1991ac8640 1 -- 172.21.15.103:0/2009470030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f198c1a7100 con 0x7f198c102570 2024-09-19T06:48:20.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.856+0000 7f1991ac8640 1 -- 172.21.15.103:0/2009470030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f198c101200 con 0x7f198c102570 2024-09-19T06:48:20.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.857+0000 7f1988ff9640 1 -- 172.21.15.103:0/2009470030 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1980042070 con 0x7f198c102570 2024-09-19T06:48:20.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.858+0000 7f1988ff9640 1 --2- 172.21.15.103:0/2009470030 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f19740a32d0 0x7f19740a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:20.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.858+0000 7f198b7fe640 1 --2- 172.21.15.103:0/2009470030 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f19740a32d0 0x7f19740a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:20.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.858+0000 7f1988ff9640 1 -- 172.21.15.103:0/2009470030 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f19800f0120 con 0x7f198c102570 2024-09-19T06:48:20.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.858+0000 7f198b7fe640 1 --2- 172.21.15.103:0/2009470030 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f19740a32d0 0x7f19740a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f197c009fc0 tx=0x7f197c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:20.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:20.862+0000 7f1988ff9640 1 -- 172.21.15.103:0/2009470030 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f19800bc6a0 con 0x7f198c102570 2024-09-19T06:48:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.037+0000 7f1991ac8640 1 -- 172.21.15.103:0/2009470030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f198c108b00 con 0x7f198c102570 2024-09-19T06:48:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.037+0000 7f1988ff9640 1 -- 172.21.15.103:0/2009470030 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f19800c09a0 con 0x7f198c102570 2024-09-19T06:48:21.038 INFO:teuthology.orchestra.run.smithi103.stdout:128849018938 2024-09-19T06:48:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.039+0000 7f1991ac8640 1 -- 172.21.15.103:0/2009470030 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f19740a32d0 msgr2=0x7f19740a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.039+0000 7f1991ac8640 1 --2- 172.21.15.103:0/2009470030 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f19740a32d0 0x7f19740a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f197c009fc0 tx=0x7f197c005f90 comp rx=0 tx=0).stop 2024-09-19T06:48:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.039+0000 7f1991ac8640 1 -- 172.21.15.103:0/2009470030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f198c102570 msgr2=0x7f198c197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.039+0000 7f1991ac8640 1 --2- 172.21.15.103:0/2009470030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f198c102570 0x7f198c197660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1980006010 tx=0x7f1980003c10 comp rx=0 tx=0).stop 2024-09-19T06:48:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.039+0000 7f1991ac8640 1 -- 172.21.15.103:0/2009470030 shutdown_connections 2024-09-19T06:48:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.039+0000 7f1991ac8640 1 --2- 172.21.15.103:0/2009470030 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f19740a32d0 0x7f19740a5790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.039+0000 7f1991ac8640 1 --2- 172.21.15.103:0/2009470030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f198c102ef0 0x7f198c197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.039+0000 7f1991ac8640 1 --2- 172.21.15.103:0/2009470030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f198c102570 0x7f198c197660 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.039+0000 7f1991ac8640 1 --2- 172.21.15.103:0/2009470030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f198c100e20 0x7f198c197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.039+0000 7f1991ac8640 1 -- 172.21.15.103:0/2009470030 >> 172.21.15.103:0/2009470030 conn(0x7f198c0fc7d0 msgr2=0x7f198c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.040+0000 7f1991ac8640 1 -- 172.21.15.103:0/2009470030 shutdown_connections 2024-09-19T06:48:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.040+0000 7f1991ac8640 1 -- 172.21.15.103:0/2009470030 wait complete. 2024-09-19T06:48:21.104 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:20 smithi103 ceph-mon[31158]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:48:21.105 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:20 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:48:21.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.099+0000 7feae739d640 1 -- 172.21.15.103:0/2423670967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feae0102d40 msgr2=0x7feae0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:21.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.099+0000 7feae739d640 1 --2- 172.21.15.103:0/2423670967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feae0102d40 0x7feae0103120 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fead8009900 tx=0x7fead802a480 comp rx=0 tx=0).stop 2024-09-19T06:48:21.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.099+0000 7feae739d640 1 -- 172.21.15.103:0/2423670967 shutdown_connections 2024-09-19T06:48:21.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.099+0000 7feae739d640 1 --2- 172.21.15.103:0/2423670967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feae010c170 0x7feae010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.099+0000 7feae739d640 1 --2- 172.21.15.103:0/2423670967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feae0103660 0x7feae010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.099+0000 7feae739d640 1 --2- 172.21.15.103:0/2423670967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feae0102d40 0x7feae0103120 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.099+0000 7feae739d640 1 -- 172.21.15.103:0/2423670967 >> 172.21.15.103:0/2423670967 conn(0x7feae00fc7a0 msgr2=0x7feae00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:21.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.099+0000 7feae739d640 1 -- 172.21.15.103:0/2423670967 shutdown_connections 2024-09-19T06:48:21.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.099+0000 7feae739d640 1 -- 172.21.15.103:0/2423670967 wait complete. 2024-09-19T06:48:21.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.099+0000 7feae739d640 1 Processor -- start 2024-09-19T06:48:21.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae739d640 1 -- start start 2024-09-19T06:48:21.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae739d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feae0102d40 0x7feae01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae739d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feae0103660 0x7feae0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae739d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feae010c170 0x7feae0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae739d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feae0111fd0 con 0x7feae0103660 2024-09-19T06:48:21.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae739d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feae0111e50 con 0x7feae0102d40 2024-09-19T06:48:21.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae739d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feae0112150 con 0x7feae010c170 2024-09-19T06:48:21.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae4938640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feae0103660 0x7feae0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae4938640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feae0103660 0x7feae0100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43276/0 (socket says 172.21.15.103:43276) 2024-09-19T06:48:21.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae4938640 1 -- 172.21.15.103:0/818515136 learned_addr learned my addr 172.21.15.103:0/818515136 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:21.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae5139640 1 --2- 172.21.15.103:0/818515136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feae0102d40 0x7feae01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae4938640 1 -- 172.21.15.103:0/818515136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feae010c170 msgr2=0x7feae0101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:21.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae4938640 1 --2- 172.21.15.103:0/818515136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feae010c170 0x7feae0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae4938640 1 -- 172.21.15.103:0/818515136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feae0102d40 msgr2=0x7feae01009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:21.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae4938640 1 --2- 172.21.15.103:0/818515136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feae0102d40 0x7feae01009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae4938640 1 -- 172.21.15.103:0/818515136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae01a6970 con 0x7feae0103660 2024-09-19T06:48:21.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae5139640 1 --2- 172.21.15.103:0/818515136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feae0102d40 0x7feae01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:21.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae4938640 1 --2- 172.21.15.103:0/818515136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feae0103660 0x7feae0100ee0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fead400e9d0 tx=0x7fead400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:21.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feade7fc640 1 -- 172.21.15.103:0/818515136 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead400cd80 con 0x7feae0103660 2024-09-19T06:48:21.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feae739d640 1 -- 172.21.15.103:0/818515136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae01a6c60 con 0x7feae0103660 2024-09-19T06:48:21.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feade7fc640 1 -- 172.21.15.103:0/818515136 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fead4005020 con 0x7feae0103660 2024-09-19T06:48:21.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.100+0000 7feade7fc640 1 -- 172.21.15.103:0/818515136 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead40116d0 con 0x7feae0103660 2024-09-19T06:48:21.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.101+0000 7feae739d640 1 -- 172.21.15.103:0/818515136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae01a71a0 con 0x7feae0103660 2024-09-19T06:48:21.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.102+0000 7feae739d640 1 -- 172.21.15.103:0/818515136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae0103120 con 0x7feae0103660 2024-09-19T06:48:21.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.103+0000 7feade7fc640 1 -- 172.21.15.103:0/818515136 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fead4005560 con 0x7feae0103660 2024-09-19T06:48:21.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.103+0000 7feade7fc640 1 --2- 172.21.15.103:0/818515136 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feac80a32d0 0x7feac80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.103+0000 7feade7fc640 1 -- 172.21.15.103:0/818515136 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fead40cfa30 con 0x7feae0103660 2024-09-19T06:48:21.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.103+0000 7feae5139640 1 --2- 172.21.15.103:0/818515136 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feac80a32d0 0x7feac80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.104+0000 7feae5139640 1 --2- 172.21.15.103:0/818515136 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feac80a32d0 0x7feac80a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fead8005ab0 tx=0x7fead8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:21.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.108+0000 7feade7fc640 1 -- 172.21.15.103:0/818515136 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fead4016030 con 0x7feae0103660 2024-09-19T06:48:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:20 smithi134 ceph-mon[35356]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:48:21.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:20 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:48:21.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.276+0000 7feae739d640 1 -- 172.21.15.103:0/818515136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7feae0107210 con 0x7feae0103660 2024-09-19T06:48:21.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.276+0000 7feade7fc640 1 -- 172.21.15.103:0/818515136 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fead409c600 con 0x7feae0103660 2024-09-19T06:48:21.278 INFO:teuthology.orchestra.run.smithi103.stdout:103079215163 2024-09-19T06:48:21.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.278+0000 7feae739d640 1 -- 172.21.15.103:0/818515136 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feac80a32d0 msgr2=0x7feac80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:21.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.278+0000 7feae739d640 1 --2- 172.21.15.103:0/818515136 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feac80a32d0 0x7feac80a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fead8005ab0 tx=0x7fead8005a20 comp rx=0 tx=0).stop 2024-09-19T06:48:21.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.278+0000 7feae739d640 1 -- 172.21.15.103:0/818515136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feae0103660 msgr2=0x7feae0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:21.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.278+0000 7feae739d640 1 --2- 172.21.15.103:0/818515136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feae0103660 0x7feae0100ee0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fead400e9d0 tx=0x7fead400ee90 comp rx=0 tx=0).stop 2024-09-19T06:48:21.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.278+0000 7feae739d640 1 -- 172.21.15.103:0/818515136 shutdown_connections 2024-09-19T06:48:21.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.278+0000 7feae739d640 1 --2- 172.21.15.103:0/818515136 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feac80a32d0 0x7feac80a5790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.279+0000 7feae739d640 1 --2- 172.21.15.103:0/818515136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feae010c170 0x7feae0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.279+0000 7feae739d640 1 --2- 172.21.15.103:0/818515136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feae0103660 0x7feae0100ee0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.279+0000 7feae739d640 1 --2- 172.21.15.103:0/818515136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feae0102d40 0x7feae01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.279+0000 7feae739d640 1 -- 172.21.15.103:0/818515136 >> 172.21.15.103:0/818515136 conn(0x7feae00fc7a0 msgr2=0x7feae00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:21.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.279+0000 7feae739d640 1 -- 172.21.15.103:0/818515136 shutdown_connections 2024-09-19T06:48:21.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:21.279+0000 7feae739d640 1 -- 172.21.15.103:0/818515136 wait complete. 2024-09-19T06:48:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:20 smithi174 ceph-mon[35637]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:48:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:20 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:48:22.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:21 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2009470030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T06:48:22.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:21 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/818515136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T06:48:22.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:21 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:22.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:21 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:22.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:21 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:22.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:21 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:22.111 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626477 got 180388626480 for osd.6 2024-09-19T06:48:22.112 DEBUG:teuthology.parallel:result is None 2024-09-19T06:48:22.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:21 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2009470030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T06:48:22.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:21 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/818515136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T06:48:22.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:21 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:22.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:21 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:22.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:21 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:22.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:21 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:21 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2009470030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T06:48:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:21 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/818515136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T06:48:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:21 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:21 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:21 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:21 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:22.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.717+0000 7f38fd2e4640 1 -- 172.21.15.103:0/392088189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38f8101420 msgr2=0x7f38f810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:22.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.717+0000 7f38fd2e4640 1 --2- 172.21.15.103:0/392088189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38f8101420 0x7f38f810d950 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009930 tx=0x7f38e802a420 comp rx=0 tx=0).stop 2024-09-19T06:48:22.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.717+0000 7f38fd2e4640 1 -- 172.21.15.103:0/392088189 shutdown_connections 2024-09-19T06:48:22.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.717+0000 7f38fd2e4640 1 --2- 172.21.15.103:0/392088189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38f8101420 0x7f38f810d950 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:22.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.717+0000 7f38fd2e4640 1 --2- 172.21.15.103:0/392088189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38f8100a80 0x7f38f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:22.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.717+0000 7f38fd2e4640 1 --2- 172.21.15.103:0/392088189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38f8106a80 0x7f38f8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:22.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.717+0000 7f38fd2e4640 1 -- 172.21.15.103:0/392088189 >> 172.21.15.103:0/392088189 conn(0x7f38f80fc7a0 msgr2=0x7f38f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:22.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.717+0000 7f38fd2e4640 1 -- 172.21.15.103:0/392088189 shutdown_connections 2024-09-19T06:48:22.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.717+0000 7f38fd2e4640 1 -- 172.21.15.103:0/392088189 wait complete. 2024-09-19T06:48:22.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.718+0000 7f38fd2e4640 1 Processor -- start 2024-09-19T06:48:22.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.718+0000 7f38fd2e4640 1 -- start start 2024-09-19T06:48:22.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.718+0000 7f38fd2e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38f8100a80 0x7f38f8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:22.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.718+0000 7f38fd2e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38f8101420 0x7f38f8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:22.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.718+0000 7f38fd2e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38f8106a80 0x7f38f819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:22.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.718+0000 7f38fd2e4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f38f8110ea0 con 0x7f38f8106a80 2024-09-19T06:48:22.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.718+0000 7f38fd2e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38f8110d20 con 0x7f38f8100a80 2024-09-19T06:48:22.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.718+0000 7f38fd2e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f38f8111020 con 0x7f38f8101420 2024-09-19T06:48:22.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.718+0000 7f38f77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38f8106a80 0x7f38f819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:22.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.718+0000 7f38f77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38f8106a80 0x7f38f819ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43288/0 (socket says 172.21.15.103:43288) 2024-09-19T06:48:22.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.718+0000 7f38f77fe640 1 -- 172.21.15.103:0/1717423360 learned_addr learned my addr 172.21.15.103:0/1717423360 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:22.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.718+0000 7f38f6ffd640 1 --2- 172.21.15.103:0/1717423360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38f8100a80 0x7f38f8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:22.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.718+0000 7f38f67fc640 1 --2- 172.21.15.103:0/1717423360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38f8101420 0x7f38f8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:22.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.719+0000 7f38f77fe640 1 -- 172.21.15.103:0/1717423360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38f8101420 msgr2=0x7f38f8198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:22.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.719+0000 7f38f77fe640 1 --2- 172.21.15.103:0/1717423360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38f8101420 0x7f38f8198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:22.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.719+0000 7f38f77fe640 1 -- 172.21.15.103:0/1717423360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38f8100a80 msgr2=0x7f38f8198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:22.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.719+0000 7f38f77fe640 1 --2- 172.21.15.103:0/1717423360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38f8100a80 0x7f38f8198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:22.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.719+0000 7f38f77fe640 1 -- 172.21.15.103:0/1717423360 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f819d4e0 con 0x7f38f8106a80 2024-09-19T06:48:22.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.719+0000 7f38f77fe640 1 --2- 172.21.15.103:0/1717423360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38f8106a80 0x7f38f819ce00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f38e8005ab0 tx=0x7f38e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:22.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.719+0000 7f38effff640 1 -- 172.21.15.103:0/1717423360 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e803f030 con 0x7f38f8106a80 2024-09-19T06:48:22.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.719+0000 7f38effff640 1 -- 172.21.15.103:0/1717423360 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38e8003ce0 con 0x7f38f8106a80 2024-09-19T06:48:22.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.719+0000 7f38fd2e4640 1 -- 172.21.15.103:0/1717423360 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f819d770 con 0x7f38f8106a80 2024-09-19T06:48:22.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.719+0000 7f38effff640 1 -- 172.21.15.103:0/1717423360 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e8035070 con 0x7f38f8106a80 2024-09-19T06:48:22.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.719+0000 7f38fd2e4640 1 -- 172.21.15.103:0/1717423360 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f81a50b0 con 0x7f38f8106a80 2024-09-19T06:48:22.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.720+0000 7f38fd2e4640 1 -- 172.21.15.103:0/1717423360 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f81021c0 con 0x7f38f8106a80 2024-09-19T06:48:22.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.722+0000 7f38effff640 1 -- 172.21.15.103:0/1717423360 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f38e8042030 con 0x7f38f8106a80 2024-09-19T06:48:22.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.723+0000 7f38effff640 1 --2- 172.21.15.103:0/1717423360 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f38e40a32d0 0x7f38e40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:22.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.723+0000 7f38f6ffd640 1 --2- 172.21.15.103:0/1717423360 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f38e40a32d0 0x7f38e40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:22.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.723+0000 7f38effff640 1 -- 172.21.15.103:0/1717423360 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f38e80f0240 con 0x7f38f8106a80 2024-09-19T06:48:22.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.723+0000 7f38f6ffd640 1 --2- 172.21.15.103:0/1717423360 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f38e40a32d0 0x7f38e40a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f38f0005fd0 tx=0x7f38f0009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:22.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.727+0000 7f38effff640 1 -- 172.21.15.103:0/1717423360 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f38e80bce10 con 0x7f38f8106a80 2024-09-19T06:48:22.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.899+0000 7f38fd2e4640 1 -- 172.21.15.103:0/1717423360 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f38f810a020 con 0x7f38f8106a80 2024-09-19T06:48:22.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.901+0000 7f38effff640 1 -- 172.21.15.103:0/1717423360 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f38e80bc710 con 0x7f38f8106a80 2024-09-19T06:48:22.902 INFO:teuthology.orchestra.run.smithi103.stdout:287762808861 2024-09-19T06:48:22.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.903+0000 7f38fd2e4640 1 -- 172.21.15.103:0/1717423360 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f38e40a32d0 msgr2=0x7f38e40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:22.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.903+0000 7f38fd2e4640 1 --2- 172.21.15.103:0/1717423360 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f38e40a32d0 0x7f38e40a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f38f0005fd0 tx=0x7f38f0009680 comp rx=0 tx=0).stop 2024-09-19T06:48:22.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.903+0000 7f38fd2e4640 1 -- 172.21.15.103:0/1717423360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38f8106a80 msgr2=0x7f38f819ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:22.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.903+0000 7f38fd2e4640 1 --2- 172.21.15.103:0/1717423360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38f8106a80 0x7f38f819ce00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f38e8005ab0 tx=0x7f38e80042f0 comp rx=0 tx=0).stop 2024-09-19T06:48:22.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.903+0000 7f38fd2e4640 1 -- 172.21.15.103:0/1717423360 shutdown_connections 2024-09-19T06:48:22.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.903+0000 7f38fd2e4640 1 --2- 172.21.15.103:0/1717423360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38f8106a80 0x7f38f819ce00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:22.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.903+0000 7f38fd2e4640 1 --2- 172.21.15.103:0/1717423360 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f38e40a32d0 0x7f38e40a5790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:22.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.903+0000 7f38fd2e4640 1 --2- 172.21.15.103:0/1717423360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38f8101420 0x7f38f8198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:22.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.903+0000 7f38fd2e4640 1 --2- 172.21.15.103:0/1717423360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38f8100a80 0x7f38f8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:22.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.903+0000 7f38fd2e4640 1 -- 172.21.15.103:0/1717423360 >> 172.21.15.103:0/1717423360 conn(0x7f38f80fc7a0 msgr2=0x7f38f806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:22.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.904+0000 7f38fd2e4640 1 -- 172.21.15.103:0/1717423360 shutdown_connections 2024-09-19T06:48:22.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.904+0000 7f38fd2e4640 1 -- 172.21.15.103:0/1717423360 wait complete. 2024-09-19T06:48:22.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.921+0000 7f4f2289e640 1 -- 172.21.15.103:0/3648634248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f1c1027a0 msgr2=0x7f4f1c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:22.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.921+0000 7f4f2289e640 1 --2- 172.21.15.103:0/3648634248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f1c1027a0 0x7f4f1c1067f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009900 tx=0x7f4f0402a410 comp rx=0 tx=0).stop 2024-09-19T06:48:22.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.921+0000 7f4f2289e640 1 -- 172.21.15.103:0/3648634248 shutdown_connections 2024-09-19T06:48:22.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.921+0000 7f4f2289e640 1 --2- 172.21.15.103:0/3648634248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f1c106f20 0x7f4f1c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:22.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.921+0000 7f4f2289e640 1 --2- 172.21.15.103:0/3648634248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f1c1027a0 0x7f4f1c1067f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:22.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.921+0000 7f4f2289e640 1 --2- 172.21.15.103:0/3648634248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f1c101df0 0x7f4f1c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:22.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.921+0000 7f4f2289e640 1 -- 172.21.15.103:0/3648634248 >> 172.21.15.103:0/3648634248 conn(0x7f4f1c0fd8f0 msgr2=0x7f4f1c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:22.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.921+0000 7f4f2289e640 1 -- 172.21.15.103:0/3648634248 shutdown_connections 2024-09-19T06:48:22.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.921+0000 7f4f2289e640 1 -- 172.21.15.103:0/3648634248 wait complete. 2024-09-19T06:48:22.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f2289e640 1 Processor -- start 2024-09-19T06:48:22.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f2289e640 1 -- start start 2024-09-19T06:48:22.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f2289e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f1c101df0 0x7f4f1c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:22.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f2289e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f1c1027a0 0x7f4f1c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:22.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f2289e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f1c106f20 0x7f4f1c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:22.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f2289e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c111f40 con 0x7f4f1c106f20 2024-09-19T06:48:22.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f2289e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c111dc0 con 0x7f4f1c101df0 2024-09-19T06:48:22.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f2289e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1120c0 con 0x7f4f1c1027a0 2024-09-19T06:48:22.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f20e3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f1c106f20 0x7f4f1c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:22.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f20e3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f1c106f20 0x7f4f1c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43312/0 (socket says 172.21.15.103:43312) 2024-09-19T06:48:22.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f20e3b640 1 -- 172.21.15.103:0/424757097 learned_addr learned my addr 172.21.15.103:0/424757097 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:22.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f1bfff640 1 --2- 172.21.15.103:0/424757097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f1c101df0 0x7f4f1c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:22.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f13fff640 1 --2- 172.21.15.103:0/424757097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f1c1027a0 0x7f4f1c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:22.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f20e3b640 1 -- 172.21.15.103:0/424757097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f1c1027a0 msgr2=0x7f4f1c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:22.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f20e3b640 1 --2- 172.21.15.103:0/424757097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f1c1027a0 0x7f4f1c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:22.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f20e3b640 1 -- 172.21.15.103:0/424757097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f1c101df0 msgr2=0x7f4f1c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:22.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.922+0000 7f4f20e3b640 1 --2- 172.21.15.103:0/424757097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f1c101df0 0x7f4f1c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:22.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.923+0000 7f4f20e3b640 1 -- 172.21.15.103:0/424757097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c19d350 con 0x7f4f1c106f20 2024-09-19T06:48:22.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.923+0000 7f4f20e3b640 1 --2- 172.21.15.103:0/424757097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f1c106f20 0x7f4f1c19cc70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400eaa0 tx=0x7f4f1400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:22.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.923+0000 7f4f19ffb640 1 -- 172.21.15.103:0/424757097 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f14004d20 con 0x7f4f1c106f20 2024-09-19T06:48:22.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.923+0000 7f4f19ffb640 1 -- 172.21.15.103:0/424757097 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f14005020 con 0x7f4f1c106f20 2024-09-19T06:48:22.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.923+0000 7f4f2289e640 1 -- 172.21.15.103:0/424757097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1c19d640 con 0x7f4f1c106f20 2024-09-19T06:48:22.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.924+0000 7f4f19ffb640 1 -- 172.21.15.103:0/424757097 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f140116d0 con 0x7f4f1c106f20 2024-09-19T06:48:22.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.924+0000 7f4f2289e640 1 -- 172.21.15.103:0/424757097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1c1a4f20 con 0x7f4f1c106f20 2024-09-19T06:48:22.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.925+0000 7f4f2289e640 1 -- 172.21.15.103:0/424757097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f1c1032d0 con 0x7f4f1c106f20 2024-09-19T06:48:22.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.925+0000 7f4f19ffb640 1 -- 172.21.15.103:0/424757097 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4f140051d0 con 0x7f4f1c106f20 2024-09-19T06:48:22.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.926+0000 7f4f19ffb640 1 --2- 172.21.15.103:0/424757097 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4f080a32d0 0x7f4f080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:22.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.926+0000 7f4f19ffb640 1 -- 172.21.15.103:0/424757097 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4f140cfc10 con 0x7f4f1c106f20 2024-09-19T06:48:22.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.926+0000 7f4f1bfff640 1 --2- 172.21.15.103:0/424757097 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4f080a32d0 0x7f4f080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:22.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.927+0000 7f4f1bfff640 1 --2- 172.21.15.103:0/424757097 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4f080a32d0 0x7f4f080a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00abe0 tx=0x7f4f0c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:22.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:22.931+0000 7f4f19ffb640 1 -- 172.21.15.103:0/424757097 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4f14016030 con 0x7f4f1c106f20 2024-09-19T06:48:23.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.102+0000 7f4f2289e640 1 -- 172.21.15.103:0/424757097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f4f1c1021d0 con 0x7f4f1c106f20 2024-09-19T06:48:23.104 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:22 smithi103 ceph-mon[31158]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:48:23.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.103+0000 7f4f19ffb640 1 -- 172.21.15.103:0/424757097 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4f1c1021d0 con 0x7f4f1c106f20 2024-09-19T06:48:23.104 INFO:teuthology.orchestra.run.smithi103.stdout:231928234024 2024-09-19T06:48:23.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.105+0000 7f4f2289e640 1 -- 172.21.15.103:0/424757097 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4f080a32d0 msgr2=0x7f4f080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:23.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.105+0000 7f4f2289e640 1 --2- 172.21.15.103:0/424757097 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4f080a32d0 0x7f4f080a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00abe0 tx=0x7f4f0c00a2d0 comp rx=0 tx=0).stop 2024-09-19T06:48:23.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.105+0000 7f4f2289e640 1 -- 172.21.15.103:0/424757097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f1c106f20 msgr2=0x7f4f1c19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:23.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.105+0000 7f4f2289e640 1 --2- 172.21.15.103:0/424757097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f1c106f20 0x7f4f1c19cc70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400eaa0 tx=0x7f4f1400eea0 comp rx=0 tx=0).stop 2024-09-19T06:48:23.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.105+0000 7f4f2289e640 1 -- 172.21.15.103:0/424757097 shutdown_connections 2024-09-19T06:48:23.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.105+0000 7f4f2289e640 1 --2- 172.21.15.103:0/424757097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f1c106f20 0x7f4f1c19cc70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:23.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.105+0000 7f4f2289e640 1 --2- 172.21.15.103:0/424757097 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f4f080a32d0 0x7f4f080a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:23.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.105+0000 7f4f2289e640 1 --2- 172.21.15.103:0/424757097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f1c1027a0 0x7f4f1c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:23.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.105+0000 7f4f2289e640 1 --2- 172.21.15.103:0/424757097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f1c101df0 0x7f4f1c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:23.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.105+0000 7f4f2289e640 1 -- 172.21.15.103:0/424757097 >> 172.21.15.103:0/424757097 conn(0x7f4f1c0fd8f0 msgr2=0x7f4f1c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:23.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.105+0000 7f4f2289e640 1 -- 172.21.15.103:0/424757097 shutdown_connections 2024-09-19T06:48:23.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.105+0000 7f4f2289e640 1 -- 172.21.15.103:0/424757097 wait complete. 2024-09-19T06:48:23.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:22 smithi134 ceph-mon[35356]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:48:23.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:22 smithi174 ceph-mon[35637]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:48:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.530+0000 7fc147fef640 1 -- 172.21.15.103:0/670933509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc140100a80 msgr2=0x7fc140100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.530+0000 7fc147fef640 1 --2- 172.21.15.103:0/670933509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc140100a80 0x7fc140100ee0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009930 tx=0x7fc13c02a420 comp rx=0 tx=0).stop 2024-09-19T06:48:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.530+0000 7fc147fef640 1 -- 172.21.15.103:0/670933509 shutdown_connections 2024-09-19T06:48:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.530+0000 7fc147fef640 1 --2- 172.21.15.103:0/670933509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc140101420 0x7fc14010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.530+0000 7fc147fef640 1 --2- 172.21.15.103:0/670933509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc140100a80 0x7fc140100ee0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.530+0000 7fc147fef640 1 --2- 172.21.15.103:0/670933509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc140106a80 0x7fc140106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.530+0000 7fc147fef640 1 -- 172.21.15.103:0/670933509 >> 172.21.15.103:0/670933509 conn(0x7fc1400fc7a0 msgr2=0x7fc1400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:23.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.530+0000 7fc147fef640 1 -- 172.21.15.103:0/670933509 shutdown_connections 2024-09-19T06:48:23.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.530+0000 7fc147fef640 1 -- 172.21.15.103:0/670933509 wait complete. 2024-09-19T06:48:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.530+0000 7fc147fef640 1 Processor -- start 2024-09-19T06:48:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc147fef640 1 -- start start 2024-09-19T06:48:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc147fef640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc140100a80 0x7fc140198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc147fef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc140101420 0x7fc1401989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc147fef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc140106a80 0x7fc14019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc147fef640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc140110e80 con 0x7fc140106a80 2024-09-19T06:48:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc147fef640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc140110d00 con 0x7fc140101420 2024-09-19T06:48:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc147fef640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc140111000 con 0x7fc140100a80 2024-09-19T06:48:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc14658c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc140106a80 0x7fc14019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc14658c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc140106a80 0x7fc14019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43322/0 (socket says 172.21.15.103:43322) 2024-09-19T06:48:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc14658c640 1 -- 172.21.15.103:0/3663085906 learned_addr learned my addr 172.21.15.103:0/3663085906 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc145d8b640 1 --2- 172.21.15.103:0/3663085906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc140100a80 0x7fc140198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc14558a640 1 --2- 172.21.15.103:0/3663085906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc140101420 0x7fc1401989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc14658c640 1 -- 172.21.15.103:0/3663085906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc140100a80 msgr2=0x7fc140198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc14658c640 1 --2- 172.21.15.103:0/3663085906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc140100a80 0x7fc140198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc14658c640 1 -- 172.21.15.103:0/3663085906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc140101420 msgr2=0x7fc1401989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc14658c640 1 --2- 172.21.15.103:0/3663085906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc140101420 0x7fc1401989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc14658c640 1 -- 172.21.15.103:0/3663085906 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14019d410 con 0x7fc140106a80 2024-09-19T06:48:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.531+0000 7fc14558a640 1 --2- 172.21.15.103:0/3663085906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc140101420 0x7fc1401989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.532+0000 7fc14658c640 1 --2- 172.21.15.103:0/3663085906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc140106a80 0x7fc14019cd30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc13800cad0 tx=0x7fc13800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.532+0000 7fc132ffd640 1 -- 172.21.15.103:0/3663085906 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13800ae20 con 0x7fc140106a80 2024-09-19T06:48:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.532+0000 7fc132ffd640 1 -- 172.21.15.103:0/3663085906 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc138005020 con 0x7fc140106a80 2024-09-19T06:48:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.532+0000 7fc132ffd640 1 -- 172.21.15.103:0/3663085906 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13800f6f0 con 0x7fc140106a80 2024-09-19T06:48:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.532+0000 7fc147fef640 1 -- 172.21.15.103:0/3663085906 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14019d700 con 0x7fc140106a80 2024-09-19T06:48:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.532+0000 7fc147fef640 1 -- 172.21.15.103:0/3663085906 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1401a4fe0 con 0x7fc140106a80 2024-09-19T06:48:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.533+0000 7fc147fef640 1 -- 172.21.15.103:0/3663085906 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc140100ee0 con 0x7fc140106a80 2024-09-19T06:48:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.534+0000 7fc132ffd640 1 -- 172.21.15.103:0/3663085906 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc1380051d0 con 0x7fc140106a80 2024-09-19T06:48:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.535+0000 7fc132ffd640 1 --2- 172.21.15.103:0/3663085906 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc12c0a32d0 0x7fc12c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.535+0000 7fc145d8b640 1 --2- 172.21.15.103:0/3663085906 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc12c0a32d0 0x7fc12c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.535+0000 7fc132ffd640 1 -- 172.21.15.103:0/3663085906 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc1380cfe00 con 0x7fc140106a80 2024-09-19T06:48:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.535+0000 7fc145d8b640 1 --2- 172.21.15.103:0/3663085906 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc12c0a32d0 0x7fc12c0a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc1340058c0 tx=0x7fc13400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:23.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.538+0000 7fc132ffd640 1 -- 172.21.15.103:0/3663085906 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc138014030 con 0x7fc140106a80 2024-09-19T06:48:23.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.709+0000 7fc147fef640 1 -- 172.21.15.103:0/3663085906 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc140109a90 con 0x7fc140106a80 2024-09-19T06:48:23.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.710+0000 7fc132ffd640 1 -- 172.21.15.103:0/3663085906 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc13809c9d0 con 0x7fc140106a80 2024-09-19T06:48:23.711 INFO:teuthology.orchestra.run.smithi103.stdout:150323855413 2024-09-19T06:48:23.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.712+0000 7fc147fef640 1 -- 172.21.15.103:0/3663085906 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc12c0a32d0 msgr2=0x7fc12c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:23.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.712+0000 7fc147fef640 1 --2- 172.21.15.103:0/3663085906 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc12c0a32d0 0x7fc12c0a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc1340058c0 tx=0x7fc13400a2d0 comp rx=0 tx=0).stop 2024-09-19T06:48:23.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.712+0000 7fc147fef640 1 -- 172.21.15.103:0/3663085906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc140106a80 msgr2=0x7fc14019cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:23.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.712+0000 7fc147fef640 1 --2- 172.21.15.103:0/3663085906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc140106a80 0x7fc14019cd30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc13800cad0 tx=0x7fc13800cf90 comp rx=0 tx=0).stop 2024-09-19T06:48:23.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.712+0000 7fc147fef640 1 -- 172.21.15.103:0/3663085906 shutdown_connections 2024-09-19T06:48:23.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.712+0000 7fc147fef640 1 --2- 172.21.15.103:0/3663085906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc140106a80 0x7fc14019cd30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:23.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.712+0000 7fc147fef640 1 --2- 172.21.15.103:0/3663085906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc140101420 0x7fc1401989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:23.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.712+0000 7fc147fef640 1 --2- 172.21.15.103:0/3663085906 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc12c0a32d0 0x7fc12c0a5790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:23.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.712+0000 7fc147fef640 1 --2- 172.21.15.103:0/3663085906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc140100a80 0x7fc140198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:23.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.712+0000 7fc147fef640 1 -- 172.21.15.103:0/3663085906 >> 172.21.15.103:0/3663085906 conn(0x7fc1400fc7a0 msgr2=0x7fc14006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:23.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.712+0000 7fc147fef640 1 -- 172.21.15.103:0/3663085906 shutdown_connections 2024-09-19T06:48:23.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:23.712+0000 7fc147fef640 1 -- 172.21.15.103:0/3663085906 wait complete. 2024-09-19T06:48:23.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:23 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1717423360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T06:48:23.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:23 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/424757097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T06:48:23.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:23 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3663085906' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T06:48:23.971 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234021 got 231928234024 for osd.8 2024-09-19T06:48:23.972 DEBUG:teuthology.parallel:result is None 2024-09-19T06:48:24.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:23 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1717423360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T06:48:24.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:23 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/424757097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T06:48:24.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:23 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3663085906' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T06:48:24.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:23 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1717423360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T06:48:24.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:23 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/424757097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T06:48:24.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:23 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3663085906' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T06:48:24.587 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018934 got 128849018938 for osd.4 2024-09-19T06:48:24.587 DEBUG:teuthology.parallel:result is None 2024-09-19T06:48:24.960 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:24 smithi103 ceph-mon[31158]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:48:24.988 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808857 got 287762808861 for osd.10 2024-09-19T06:48:24.988 DEBUG:teuthology.parallel:result is None 2024-09-19T06:48:25.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:24 smithi134 ceph-mon[35356]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:48:25.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:24 smithi174 ceph-mon[35637]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:48:25.502 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855410 got 150323855413 for osd.5 2024-09-19T06:48:25.570 DEBUG:teuthology.parallel:result is None 2024-09-19T06:48:25.957 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215161 got 103079215163 for osd.3 2024-09-19T06:48:25.957 DEBUG:teuthology.parallel:result is None 2024-09-19T06:48:25.957 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T06:48:25.958 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:48:26.855 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:26 smithi134 ceph-mon[35356]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T06:48:27.247 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:26 smithi103 ceph-mon[31158]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T06:48:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:26 smithi174 ceph-mon[35637]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T06:48:27.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.326+0000 7f3cf39e4640 1 -- 172.21.15.103:0/3923487272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cec102790 msgr2=0x7f3cec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:27.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.326+0000 7f3cf39e4640 1 --2- 172.21.15.103:0/3923487272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cec102790 0x7f3cec1067e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009900 tx=0x7f3ce802a410 comp rx=0 tx=0).stop 2024-09-19T06:48:27.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.326+0000 7f3cf39e4640 1 -- 172.21.15.103:0/3923487272 shutdown_connections 2024-09-19T06:48:27.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.326+0000 7f3cf39e4640 1 --2- 172.21.15.103:0/3923487272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cec106f10 0x7f3cec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:27.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.326+0000 7f3cf39e4640 1 --2- 172.21.15.103:0/3923487272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cec102790 0x7f3cec1067e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:27.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.326+0000 7f3cf39e4640 1 --2- 172.21.15.103:0/3923487272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cec101de0 0x7f3cec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:27.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.326+0000 7f3cf39e4640 1 -- 172.21.15.103:0/3923487272 >> 172.21.15.103:0/3923487272 conn(0x7f3cec0fd8c0 msgr2=0x7f3cec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:27.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.327+0000 7f3cf39e4640 1 -- 172.21.15.103:0/3923487272 shutdown_connections 2024-09-19T06:48:27.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.327+0000 7f3cf39e4640 1 -- 172.21.15.103:0/3923487272 wait complete. 2024-09-19T06:48:27.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.327+0000 7f3cf39e4640 1 Processor -- start 2024-09-19T06:48:27.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.327+0000 7f3cf39e4640 1 -- start start 2024-09-19T06:48:27.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.327+0000 7f3cf39e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cec101de0 0x7f3cec198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:27.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.327+0000 7f3cf39e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cec102790 0x7f3cec198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:27.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.327+0000 7f3cf39e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cec106f10 0x7f3cec19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:27.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.327+0000 7f3cf39e4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec111f90 con 0x7f3cec106f10 2024-09-19T06:48:27.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.327+0000 7f3cf39e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec111e10 con 0x7f3cec102790 2024-09-19T06:48:27.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.327+0000 7f3cf39e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec112110 con 0x7f3cec101de0 2024-09-19T06:48:27.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.327+0000 7f3cf1f81640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cec106f10 0x7f3cec19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:27.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.327+0000 7f3cf1f81640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cec106f10 0x7f3cec19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46430/0 (socket says 172.21.15.103:46430) 2024-09-19T06:48:27.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.327+0000 7f3cf1f81640 1 -- 172.21.15.103:0/2940175158 learned_addr learned my addr 172.21.15.103:0/2940175158 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:27.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.328+0000 7f3cf1780640 1 --2- 172.21.15.103:0/2940175158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cec101de0 0x7f3cec198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:27.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.328+0000 7f3cf1f81640 1 -- 172.21.15.103:0/2940175158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cec101de0 msgr2=0x7f3cec198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:27.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.328+0000 7f3cf1f81640 1 --2- 172.21.15.103:0/2940175158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cec101de0 0x7f3cec198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:27.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.328+0000 7f3cf0f7f640 1 --2- 172.21.15.103:0/2940175158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cec102790 0x7f3cec198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:27.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.328+0000 7f3cf1f81640 1 -- 172.21.15.103:0/2940175158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cec102790 msgr2=0x7f3cec198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:27.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.328+0000 7f3cf1f81640 1 --2- 172.21.15.103:0/2940175158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cec102790 0x7f3cec198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:27.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.328+0000 7f3cf1f81640 1 -- 172.21.15.103:0/2940175158 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec19d520 con 0x7f3cec106f10 2024-09-19T06:48:27.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.328+0000 7f3cf1780640 1 --2- 172.21.15.103:0/2940175158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cec101de0 0x7f3cec198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:27.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.328+0000 7f3cf0f7f640 1 --2- 172.21.15.103:0/2940175158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cec102790 0x7f3cec198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:27.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.328+0000 7f3cf1f81640 1 --2- 172.21.15.103:0/2940175158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cec106f10 0x7f3cec19ce40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400b970 tx=0x7f3ce400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:27.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.328+0000 7f3ce27fc640 1 -- 172.21.15.103:0/2940175158 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4019070 con 0x7f3cec106f10 2024-09-19T06:48:27.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.328+0000 7f3cf39e4640 1 -- 172.21.15.103:0/2940175158 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cec19d810 con 0x7f3cec106f10 2024-09-19T06:48:27.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.328+0000 7f3ce27fc640 1 -- 172.21.15.103:0/2940175158 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce400cb40 con 0x7f3cec106f10 2024-09-19T06:48:27.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.328+0000 7f3ce27fc640 1 -- 172.21.15.103:0/2940175158 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4007610 con 0x7f3cec106f10 2024-09-19T06:48:27.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.328+0000 7f3cf39e4640 1 -- 172.21.15.103:0/2940175158 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cec1a50f0 con 0x7f3cec106f10 2024-09-19T06:48:27.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.329+0000 7f3cf39e4640 1 -- 172.21.15.103:0/2940175158 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cec1021c0 con 0x7f3cec106f10 2024-09-19T06:48:27.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.330+0000 7f3ce27fc640 1 -- 172.21.15.103:0/2940175158 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3ce401c070 con 0x7f3cec106f10 2024-09-19T06:48:27.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.331+0000 7f3ce27fc640 1 --2- 172.21.15.103:0/2940175158 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3cd80a32d0 0x7f3cd80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:27.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.331+0000 7f3cf1780640 1 --2- 172.21.15.103:0/2940175158 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3cd80a32d0 0x7f3cd80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:27.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.331+0000 7f3ce27fc640 1 -- 172.21.15.103:0/2940175158 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3ce40cf470 con 0x7f3cec106f10 2024-09-19T06:48:27.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.331+0000 7f3cf1780640 1 --2- 172.21.15.103:0/2940175158 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3cd80a32d0 0x7f3cd80a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0058c0 tx=0x7f3cdc005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:27.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.334+0000 7f3ce27fc640 1 -- 172.21.15.103:0/2940175158 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3ce409c150 con 0x7f3cec106f10 2024-09-19T06:48:27.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.512+0000 7f3cf39e4640 1 -- 172.21.15.103:0/2940175158 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3cec10a1c0 con 0x7f3cd80a32d0 2024-09-19T06:48:27.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.518+0000 7f3ce27fc640 1 -- 172.21.15.103:0/2940175158 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258761 (secure 0 0 0) 0x7f3cdc00b6b0 con 0x7f3cd80a32d0 2024-09-19T06:48:27.522 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:48:27.523 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T06:48:27.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.521+0000 7f3cf39e4640 1 -- 172.21.15.103:0/2940175158 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3cd80a32d0 msgr2=0x7f3cd80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:27.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.521+0000 7f3cf39e4640 1 --2- 172.21.15.103:0/2940175158 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3cd80a32d0 0x7f3cd80a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0058c0 tx=0x7f3cdc005d30 comp rx=0 tx=0).stop 2024-09-19T06:48:27.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.522+0000 7f3cf39e4640 1 -- 172.21.15.103:0/2940175158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cec106f10 msgr2=0x7f3cec19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:27.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.522+0000 7f3cf39e4640 1 --2- 172.21.15.103:0/2940175158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cec106f10 0x7f3cec19ce40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400b970 tx=0x7f3ce400be30 comp rx=0 tx=0).stop 2024-09-19T06:48:27.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.522+0000 7f3cf39e4640 1 -- 172.21.15.103:0/2940175158 shutdown_connections 2024-09-19T06:48:27.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.522+0000 7f3cf39e4640 1 --2- 172.21.15.103:0/2940175158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cec106f10 0x7f3cec19ce40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:27.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.522+0000 7f3cf39e4640 1 --2- 172.21.15.103:0/2940175158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cec102790 0x7f3cec198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:27.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.522+0000 7f3cf39e4640 1 --2- 172.21.15.103:0/2940175158 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3cd80a32d0 0x7f3cd80a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:27.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.522+0000 7f3cf39e4640 1 --2- 172.21.15.103:0/2940175158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cec101de0 0x7f3cec198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:27.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.522+0000 7f3cf39e4640 1 -- 172.21.15.103:0/2940175158 >> 172.21.15.103:0/2940175158 conn(0x7f3cec0fd8c0 msgr2=0x7f3cec0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:27.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.522+0000 7f3cf39e4640 1 -- 172.21.15.103:0/2940175158 shutdown_connections 2024-09-19T06:48:27.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:27.522+0000 7f3cf39e4640 1 -- 172.21.15.103:0/2940175158 wait complete. 2024-09-19T06:48:28.212 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-19T06:48:25.835352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658500,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":644,"num_read_kb":447,"num_write":426,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":1181610,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353688,"kb_used_data":12732,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588136,"statfs":{"total":1151940427776,"available":1151578251264,"internally_reserved":0,"allocated":13037568,"data_stored":7224864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002778"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831591+0000","last_change":"2024-09-19T06:47:24.853118+0000","last_active":"2024-09-19T06:47:40.831591+0000","last_peered":"2024-09-19T06:47:40.831591+0000","last_clean":"2024-09-19T06:47:40.831591+0000","last_became_active":"2024-09-19T06:47:24.852801+0000","last_became_peered":"2024-09-19T06:47:24.852801+0000","last_unstale":"2024-09-19T06:47:40.831591+0000","last_undegraded":"2024-09-19T06:47:40.831591+0000","last_fullsized":"2024-09-19T06:47:40.831591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:16.729103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839300+0000","last_change":"2024-09-19T06:47:24.841095+0000","last_active":"2024-09-19T06:47:40.839300+0000","last_peered":"2024-09-19T06:47:40.839300+0000","last_clean":"2024-09-19T06:47:40.839300+0000","last_became_active":"2024-09-19T06:47:24.840774+0000","last_became_peered":"2024-09-19T06:47:24.840774+0000","last_unstale":"2024-09-19T06:47:40.839300+0000","last_undegraded":"2024-09-19T06:47:40.839300+0000","last_fullsized":"2024-09-19T06:47:40.839300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:27.262766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839649+0000","last_change":"2024-09-19T06:47:24.848444+0000","last_active":"2024-09-19T06:47:40.839649+0000","last_peered":"2024-09-19T06:47:40.839649+0000","last_clean":"2024-09-19T06:47:40.839649+0000","last_became_active":"2024-09-19T06:47:24.847354+0000","last_became_peered":"2024-09-19T06:47:24.847354+0000","last_unstale":"2024-09-19T06:47:40.839649+0000","last_undegraded":"2024-09-19T06:47:40.839649+0000","last_fullsized":"2024-09-19T06:47:40.839649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:44.901265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829447+0000","last_change":"2024-09-19T06:47:24.851915+0000","last_active":"2024-09-19T06:47:40.829447+0000","last_peered":"2024-09-19T06:47:40.829447+0000","last_clean":"2024-09-19T06:47:40.829447+0000","last_became_active":"2024-09-19T06:47:24.851600+0000","last_became_peered":"2024-09-19T06:47:24.851600+0000","last_unstale":"2024-09-19T06:47:40.829447+0000","last_undegraded":"2024-09-19T06:47:40.829447+0000","last_fullsized":"2024-09-19T06:47:40.829447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:52.424288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833135+0000","last_change":"2024-09-19T06:47:24.843184+0000","last_active":"2024-09-19T06:47:40.833135+0000","last_peered":"2024-09-19T06:47:40.833135+0000","last_clean":"2024-09-19T06:47:40.833135+0000","last_became_active":"2024-09-19T06:47:24.842857+0000","last_became_peered":"2024-09-19T06:47:24.842857+0000","last_unstale":"2024-09-19T06:47:40.833135+0000","last_undegraded":"2024-09-19T06:47:40.833135+0000","last_fullsized":"2024-09-19T06:47:40.833135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:06.608699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.171834+0000","last_change":"2024-09-19T06:47:24.846631+0000","last_active":"2024-09-19T06:47:41.171834+0000","last_peered":"2024-09-19T06:47:41.171834+0000","last_clean":"2024-09-19T06:47:41.171834+0000","last_became_active":"2024-09-19T06:47:24.846271+0000","last_became_peered":"2024-09-19T06:47:24.846271+0000","last_unstale":"2024-09-19T06:47:41.171834+0000","last_undegraded":"2024-09-19T06:47:41.171834+0000","last_fullsized":"2024-09-19T06:47:41.171834+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.272273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830458+0000","last_change":"2024-09-19T06:47:24.840392+0000","last_active":"2024-09-19T06:47:40.830458+0000","last_peered":"2024-09-19T06:47:40.830458+0000","last_clean":"2024-09-19T06:47:40.830458+0000","last_became_active":"2024-09-19T06:47:24.840092+0000","last_became_peered":"2024-09-19T06:47:24.840092+0000","last_unstale":"2024-09-19T06:47:40.830458+0000","last_undegraded":"2024-09-19T06:47:40.830458+0000","last_fullsized":"2024-09-19T06:47:40.830458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:25.579096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833345+0000","last_change":"2024-09-19T06:47:24.843336+0000","last_active":"2024-09-19T06:47:40.833345+0000","last_peered":"2024-09-19T06:47:40.833345+0000","last_clean":"2024-09-19T06:47:40.833345+0000","last_became_active":"2024-09-19T06:47:24.842016+0000","last_became_peered":"2024-09-19T06:47:24.842016+0000","last_unstale":"2024-09-19T06:47:40.833345+0000","last_undegraded":"2024-09-19T06:47:40.833345+0000","last_fullsized":"2024-09-19T06:47:40.833345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:49.478359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172619+0000","last_change":"2024-09-19T06:47:24.847841+0000","last_active":"2024-09-19T06:47:41.172619+0000","last_peered":"2024-09-19T06:47:41.172619+0000","last_clean":"2024-09-19T06:47:41.172619+0000","last_became_active":"2024-09-19T06:47:24.847492+0000","last_became_peered":"2024-09-19T06:47:24.847492+0000","last_unstale":"2024-09-19T06:47:41.172619+0000","last_undegraded":"2024-09-19T06:47:41.172619+0000","last_fullsized":"2024-09-19T06:47:41.172619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:59.229022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.171833+0000","last_change":"2024-09-19T06:47:24.843773+0000","last_active":"2024-09-19T06:47:41.171833+0000","last_peered":"2024-09-19T06:47:41.171833+0000","last_clean":"2024-09-19T06:47:41.171833+0000","last_became_active":"2024-09-19T06:47:24.843409+0000","last_became_peered":"2024-09-19T06:47:24.843409+0000","last_unstale":"2024-09-19T06:47:41.171833+0000","last_undegraded":"2024-09-19T06:47:41.171833+0000","last_fullsized":"2024-09-19T06:47:41.171833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:22.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830801+0000","last_change":"2024-09-19T06:47:24.846637+0000","last_active":"2024-09-19T06:47:40.830801+0000","last_peered":"2024-09-19T06:47:40.830801+0000","last_clean":"2024-09-19T06:47:40.830801+0000","last_became_active":"2024-09-19T06:47:24.846337+0000","last_became_peered":"2024-09-19T06:47:24.846337+0000","last_unstale":"2024-09-19T06:47:40.830801+0000","last_undegraded":"2024-09-19T06:47:40.830801+0000","last_fullsized":"2024-09-19T06:47:40.830801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:23:35.869485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829501+0000","last_change":"2024-09-19T06:47:24.851399+0000","last_active":"2024-09-19T06:47:40.829501+0000","last_peered":"2024-09-19T06:47:40.829501+0000","last_clean":"2024-09-19T06:47:40.829501+0000","last_became_active":"2024-09-19T06:47:24.851090+0000","last_became_peered":"2024-09-19T06:47:24.851090+0000","last_unstale":"2024-09-19T06:47:40.829501+0000","last_undegraded":"2024-09-19T06:47:40.829501+0000","last_fullsized":"2024-09-19T06:47:40.829501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:52.839956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830657+0000","last_change":"2024-09-19T06:47:24.833858+0000","last_active":"2024-09-19T06:47:40.830657+0000","last_peered":"2024-09-19T06:47:40.830657+0000","last_clean":"2024-09-19T06:47:40.830657+0000","last_became_active":"2024-09-19T06:47:24.833616+0000","last_became_peered":"2024-09-19T06:47:24.833616+0000","last_unstale":"2024-09-19T06:47:40.830657+0000","last_undegraded":"2024-09-19T06:47:40.830657+0000","last_fullsized":"2024-09-19T06:47:40.830657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:54.918144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839219+0000","last_change":"2024-09-19T06:47:24.841030+0000","last_active":"2024-09-19T06:47:40.839219+0000","last_peered":"2024-09-19T06:47:40.839219+0000","last_clean":"2024-09-19T06:47:40.839219+0000","last_became_active":"2024-09-19T06:47:24.839631+0000","last_became_peered":"2024-09-19T06:47:24.839631+0000","last_unstale":"2024-09-19T06:47:40.839219+0000","last_undegraded":"2024-09-19T06:47:40.839219+0000","last_fullsized":"2024-09-19T06:47:40.839219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:21.764812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839091+0000","last_change":"2024-09-19T06:47:24.849281+0000","last_active":"2024-09-19T06:47:40.839091+0000","last_peered":"2024-09-19T06:47:40.839091+0000","last_clean":"2024-09-19T06:47:40.839091+0000","last_became_active":"2024-09-19T06:47:24.848977+0000","last_became_peered":"2024-09-19T06:47:24.848977+0000","last_unstale":"2024-09-19T06:47:40.839091+0000","last_undegraded":"2024-09-19T06:47:40.839091+0000","last_fullsized":"2024-09-19T06:47:40.839091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:36.693758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172068+0000","last_change":"2024-09-19T06:47:24.843619+0000","last_active":"2024-09-19T06:47:41.172068+0000","last_peered":"2024-09-19T06:47:41.172068+0000","last_clean":"2024-09-19T06:47:41.172068+0000","last_became_active":"2024-09-19T06:47:24.843323+0000","last_became_peered":"2024-09-19T06:47:24.843323+0000","last_unstale":"2024-09-19T06:47:41.172068+0000","last_undegraded":"2024-09-19T06:47:41.172068+0000","last_fullsized":"2024-09-19T06:47:41.172068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:52.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831828+0000","last_change":"2024-09-19T06:47:24.853165+0000","last_active":"2024-09-19T06:47:40.831828+0000","last_peered":"2024-09-19T06:47:40.831828+0000","last_clean":"2024-09-19T06:47:40.831828+0000","last_became_active":"2024-09-19T06:47:24.852864+0000","last_became_peered":"2024-09-19T06:47:24.852864+0000","last_unstale":"2024-09-19T06:47:40.831828+0000","last_undegraded":"2024-09-19T06:47:40.831828+0000","last_fullsized":"2024-09-19T06:47:40.831828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:38.803478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834536+0000","last_change":"2024-09-19T06:47:24.838151+0000","last_active":"2024-09-19T06:47:40.834536+0000","last_peered":"2024-09-19T06:47:40.834536+0000","last_clean":"2024-09-19T06:47:40.834536+0000","last_became_active":"2024-09-19T06:47:24.837855+0000","last_became_peered":"2024-09-19T06:47:24.837855+0000","last_unstale":"2024-09-19T06:47:40.834536+0000","last_undegraded":"2024-09-19T06:47:40.834536+0000","last_fullsized":"2024-09-19T06:47:40.834536+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:16.675557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830451+0000","last_change":"2024-09-19T06:47:24.843941+0000","last_active":"2024-09-19T06:47:40.830451+0000","last_peered":"2024-09-19T06:47:40.830451+0000","last_clean":"2024-09-19T06:47:40.830451+0000","last_became_active":"2024-09-19T06:47:24.843551+0000","last_became_peered":"2024-09-19T06:47:24.843551+0000","last_unstale":"2024-09-19T06:47:40.830451+0000","last_undegraded":"2024-09-19T06:47:40.830451+0000","last_fullsized":"2024-09-19T06:47:40.830451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:00:49.388265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.836228+0000","last_change":"2024-09-19T06:47:40.836228+0000","last_active":"2024-09-19T06:47:40.836228+0000","last_peered":"2024-09-19T06:47:40.836228+0000","last_clean":"2024-09-19T06:47:40.836228+0000","last_became_active":"2024-09-19T06:47:40.835818+0000","last_became_peered":"2024-09-19T06:47:40.835818+0000","last_unstale":"2024-09-19T06:47:40.836228+0000","last_undegraded":"2024-09-19T06:47:40.836228+0000","last_fullsized":"2024-09-19T06:47:40.836228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:45.341160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830676+0000","last_change":"2024-09-19T06:47:24.843746+0000","last_active":"2024-09-19T06:47:40.830676+0000","last_peered":"2024-09-19T06:47:40.830676+0000","last_clean":"2024-09-19T06:47:40.830676+0000","last_became_active":"2024-09-19T06:47:24.841495+0000","last_became_peered":"2024-09-19T06:47:24.841495+0000","last_unstale":"2024-09-19T06:47:40.830676+0000","last_undegraded":"2024-09-19T06:47:40.830676+0000","last_fullsized":"2024-09-19T06:47:40.830676+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:06:29.995595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.173085+0000","last_change":"2024-09-19T06:47:24.844747+0000","last_active":"2024-09-19T06:47:41.173085+0000","last_peered":"2024-09-19T06:47:41.173085+0000","last_clean":"2024-09-19T06:47:41.173085+0000","last_became_active":"2024-09-19T06:47:24.844425+0000","last_became_peered":"2024-09-19T06:47:24.844425+0000","last_unstale":"2024-09-19T06:47:41.173085+0000","last_undegraded":"2024-09-19T06:47:41.173085+0000","last_fullsized":"2024-09-19T06:47:41.173085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:10.309044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834053+0000","last_change":"2024-09-19T06:47:24.848227+0000","last_active":"2024-09-19T06:47:40.834053+0000","last_peered":"2024-09-19T06:47:40.834053+0000","last_clean":"2024-09-19T06:47:40.834053+0000","last_became_active":"2024-09-19T06:47:24.847914+0000","last_became_peered":"2024-09-19T06:47:24.847914+0000","last_unstale":"2024-09-19T06:47:40.834053+0000","last_undegraded":"2024-09-19T06:47:40.834053+0000","last_fullsized":"2024-09-19T06:47:40.834053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:47.281613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834014+0000","last_change":"2024-09-19T06:47:24.844275+0000","last_active":"2024-09-19T06:47:40.834014+0000","last_peered":"2024-09-19T06:47:40.834014+0000","last_clean":"2024-09-19T06:47:40.834014+0000","last_became_active":"2024-09-19T06:47:24.841442+0000","last_became_peered":"2024-09-19T06:47:24.841442+0000","last_unstale":"2024-09-19T06:47:40.834014+0000","last_undegraded":"2024-09-19T06:47:40.834014+0000","last_fullsized":"2024-09-19T06:47:40.834014+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:30.198655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834062+0000","last_change":"2024-09-19T06:47:24.846301+0000","last_active":"2024-09-19T06:47:40.834062+0000","last_peered":"2024-09-19T06:47:40.834062+0000","last_clean":"2024-09-19T06:47:40.834062+0000","last_became_active":"2024-09-19T06:47:24.845970+0000","last_became_peered":"2024-09-19T06:47:24.845970+0000","last_unstale":"2024-09-19T06:47:40.834062+0000","last_undegraded":"2024-09-19T06:47:40.834062+0000","last_fullsized":"2024-09-19T06:47:40.834062+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:13.245536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830554+0000","last_change":"2024-09-19T06:47:24.844444+0000","last_active":"2024-09-19T06:47:40.830554+0000","last_peered":"2024-09-19T06:47:40.830554+0000","last_clean":"2024-09-19T06:47:40.830554+0000","last_became_active":"2024-09-19T06:47:24.844141+0000","last_became_peered":"2024-09-19T06:47:24.844141+0000","last_unstale":"2024-09-19T06:47:40.830554+0000","last_undegraded":"2024-09-19T06:47:40.830554+0000","last_fullsized":"2024-09-19T06:47:40.830554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:44.739242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831557+0000","last_change":"2024-09-19T06:47:24.845633+0000","last_active":"2024-09-19T06:47:40.831557+0000","last_peered":"2024-09-19T06:47:40.831557+0000","last_clean":"2024-09-19T06:47:40.831557+0000","last_became_active":"2024-09-19T06:47:24.845319+0000","last_became_peered":"2024-09-19T06:47:24.845319+0000","last_unstale":"2024-09-19T06:47:40.831557+0000","last_undegraded":"2024-09-19T06:47:40.831557+0000","last_fullsized":"2024-09-19T06:47:40.831557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:00.820006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830650+0000","last_change":"2024-09-19T06:47:24.846457+0000","last_active":"2024-09-19T06:47:40.830650+0000","last_peered":"2024-09-19T06:47:40.830650+0000","last_clean":"2024-09-19T06:47:40.830650+0000","last_became_active":"2024-09-19T06:47:24.846130+0000","last_became_peered":"2024-09-19T06:47:24.846130+0000","last_unstale":"2024-09-19T06:47:40.830650+0000","last_undegraded":"2024-09-19T06:47:40.830650+0000","last_fullsized":"2024-09-19T06:47:40.830650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:42.179145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839379+0000","last_change":"2024-09-19T06:47:24.843141+0000","last_active":"2024-09-19T06:47:40.839379+0000","last_peered":"2024-09-19T06:47:40.839379+0000","last_clean":"2024-09-19T06:47:40.839379+0000","last_became_active":"2024-09-19T06:47:24.842810+0000","last_became_peered":"2024-09-19T06:47:24.842810+0000","last_unstale":"2024-09-19T06:47:40.839379+0000","last_undegraded":"2024-09-19T06:47:40.839379+0000","last_fullsized":"2024-09-19T06:47:40.839379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:43.440453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829981+0000","last_change":"2024-09-19T06:47:24.846171+0000","last_active":"2024-09-19T06:47:40.829981+0000","last_peered":"2024-09-19T06:47:40.829981+0000","last_clean":"2024-09-19T06:47:40.829981+0000","last_became_active":"2024-09-19T06:47:24.845799+0000","last_became_peered":"2024-09-19T06:47:24.845799+0000","last_unstale":"2024-09-19T06:47:40.829981+0000","last_undegraded":"2024-09-19T06:47:40.829981+0000","last_fullsized":"2024-09-19T06:47:40.829981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:23.103061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830061+0000","last_change":"2024-09-19T06:47:24.848723+0000","last_active":"2024-09-19T06:47:40.830061+0000","last_peered":"2024-09-19T06:47:40.830061+0000","last_clean":"2024-09-19T06:47:40.830061+0000","last_became_active":"2024-09-19T06:47:24.848367+0000","last_became_peered":"2024-09-19T06:47:24.848367+0000","last_unstale":"2024-09-19T06:47:40.830061+0000","last_undegraded":"2024-09-19T06:47:40.830061+0000","last_fullsized":"2024-09-19T06:47:40.830061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:15.985293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831354+0000","last_change":"2024-09-19T06:47:24.841379+0000","last_active":"2024-09-19T06:47:40.831354+0000","last_peered":"2024-09-19T06:47:40.831354+0000","last_clean":"2024-09-19T06:47:40.831354+0000","last_became_active":"2024-09-19T06:47:24.841041+0000","last_became_peered":"2024-09-19T06:47:24.841041+0000","last_unstale":"2024-09-19T06:47:40.831354+0000","last_undegraded":"2024-09-19T06:47:40.831354+0000","last_fullsized":"2024-09-19T06:47:40.831354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:11.799022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.832906+0000","last_change":"2024-09-19T06:47:24.849508+0000","last_active":"2024-09-19T06:47:40.832906+0000","last_peered":"2024-09-19T06:47:40.832906+0000","last_clean":"2024-09-19T06:47:40.832906+0000","last_became_active":"2024-09-19T06:47:24.849196+0000","last_became_peered":"2024-09-19T06:47:24.849196+0000","last_unstale":"2024-09-19T06:47:40.832906+0000","last_undegraded":"2024-09-19T06:47:40.832906+0000","last_fullsized":"2024-09-19T06:47:40.832906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:35.079720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":78,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:48:23.534779+0000","last_change":"2024-09-19T06:47:22.829848+0000","last_active":"2024-09-19T06:48:23.534779+0000","last_peered":"2024-09-19T06:48:23.534779+0000","last_clean":"2024-09-19T06:48:23.534779+0000","last_became_active":"2024-09-19T06:47:22.829511+0000","last_became_peered":"2024-09-19T06:47:22.829511+0000","last_unstale":"2024-09-19T06:48:23.534779+0000","last_undegraded":"2024-09-19T06:48:23.534779+0000","last_fullsized":"2024-09-19T06:48:23.534779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:32.665252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834585+0000","last_change":"2024-09-19T06:47:24.843245+0000","last_active":"2024-09-19T06:47:40.834585+0000","last_peered":"2024-09-19T06:47:40.834585+0000","last_clean":"2024-09-19T06:47:40.834585+0000","last_became_active":"2024-09-19T06:47:24.842885+0000","last_became_peered":"2024-09-19T06:47:24.842885+0000","last_unstale":"2024-09-19T06:47:40.834585+0000","last_undegraded":"2024-09-19T06:47:40.834585+0000","last_fullsized":"2024-09-19T06:47:40.834585+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:26.229003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172292+0000","last_change":"2024-09-19T06:47:22.831456+0000","last_active":"2024-09-19T06:47:41.172292+0000","last_peered":"2024-09-19T06:47:41.172292+0000","last_clean":"2024-09-19T06:47:41.172292+0000","last_became_active":"2024-09-19T06:47:22.831132+0000","last_became_peered":"2024-09-19T06:47:22.831132+0000","last_unstale":"2024-09-19T06:47:41.172292+0000","last_undegraded":"2024-09-19T06:47:41.172292+0000","last_fullsized":"2024-09-19T06:47:41.172292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:49.867713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830418+0000","last_change":"2024-09-19T06:47:22.827742+0000","last_active":"2024-09-19T06:47:40.830418+0000","last_peered":"2024-09-19T06:47:40.830418+0000","last_clean":"2024-09-19T06:47:40.830418+0000","last_became_active":"2024-09-19T06:47:22.827371+0000","last_became_peered":"2024-09-19T06:47:22.827371+0000","last_unstale":"2024-09-19T06:47:40.830418+0000","last_undegraded":"2024-09-19T06:47:40.830418+0000","last_fullsized":"2024-09-19T06:47:40.830418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:35.153129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833435+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:47:40.833435+0000","last_peered":"2024-09-19T06:47:40.833435+0000","last_clean":"2024-09-19T06:47:40.833435+0000","last_became_active":"2024-09-19T06:47:24.841693+0000","last_became_peered":"2024-09-19T06:47:24.841693+0000","last_unstale":"2024-09-19T06:47:40.833435+0000","last_undegraded":"2024-09-19T06:47:40.833435+0000","last_fullsized":"2024-09-19T06:47:40.833435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:27.828296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834254+0000","last_change":"2024-09-19T06:47:22.832531+0000","last_active":"2024-09-19T06:47:40.834254+0000","last_peered":"2024-09-19T06:47:40.834254+0000","last_clean":"2024-09-19T06:47:40.834254+0000","last_became_active":"2024-09-19T06:47:22.832213+0000","last_became_peered":"2024-09-19T06:47:22.832213+0000","last_unstale":"2024-09-19T06:47:40.834254+0000","last_undegraded":"2024-09-19T06:47:40.834254+0000","last_fullsized":"2024-09-19T06:47:40.834254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:46:03.168406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833353+0000","last_change":"2024-09-19T06:47:24.837690+0000","last_active":"2024-09-19T06:47:40.833353+0000","last_peered":"2024-09-19T06:47:40.833353+0000","last_clean":"2024-09-19T06:47:40.833353+0000","last_became_active":"2024-09-19T06:47:24.837389+0000","last_became_peered":"2024-09-19T06:47:24.837389+0000","last_unstale":"2024-09-19T06:47:40.833353+0000","last_undegraded":"2024-09-19T06:47:40.833353+0000","last_fullsized":"2024-09-19T06:47:40.833353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:20.653832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":124,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:48:24.535500+0000","last_change":"2024-09-19T06:47:22.833808+0000","last_active":"2024-09-19T06:48:24.535500+0000","last_peered":"2024-09-19T06:48:24.535500+0000","last_clean":"2024-09-19T06:48:24.535500+0000","last_became_active":"2024-09-19T06:47:22.832468+0000","last_became_peered":"2024-09-19T06:47:22.832468+0000","last_unstale":"2024-09-19T06:48:24.535500+0000","last_undegraded":"2024-09-19T06:48:24.535500+0000","last_fullsized":"2024-09-19T06:48:24.535500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:39:31.131321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172236+0000","last_change":"2024-09-19T06:47:24.840928+0000","last_active":"2024-09-19T06:47:41.172236+0000","last_peered":"2024-09-19T06:47:41.172236+0000","last_clean":"2024-09-19T06:47:41.172236+0000","last_became_active":"2024-09-19T06:47:24.839928+0000","last_became_peered":"2024-09-19T06:47:24.839928+0000","last_unstale":"2024-09-19T06:47:41.172236+0000","last_undegraded":"2024-09-19T06:47:41.172236+0000","last_fullsized":"2024-09-19T06:47:41.172236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:29.486003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830932+0000","last_change":"2024-09-19T06:47:22.833310+0000","last_active":"2024-09-19T06:47:40.830932+0000","last_peered":"2024-09-19T06:47:40.830932+0000","last_clean":"2024-09-19T06:47:40.830932+0000","last_became_active":"2024-09-19T06:47:22.832976+0000","last_became_peered":"2024-09-19T06:47:22.832976+0000","last_unstale":"2024-09-19T06:47:40.830932+0000","last_undegraded":"2024-09-19T06:47:40.830932+0000","last_fullsized":"2024-09-19T06:47:40.830932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:47.604853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.840138+0000","last_change":"2024-09-19T06:47:24.848687+0000","last_active":"2024-09-19T06:47:40.840138+0000","last_peered":"2024-09-19T06:47:40.840138+0000","last_clean":"2024-09-19T06:47:40.840138+0000","last_became_active":"2024-09-19T06:47:24.848352+0000","last_became_peered":"2024-09-19T06:47:24.848352+0000","last_unstale":"2024-09-19T06:47:40.840138+0000","last_undegraded":"2024-09-19T06:47:40.840138+0000","last_fullsized":"2024-09-19T06:47:40.840138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:54.459206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.835734+0000","last_change":"2024-09-19T06:47:22.851479+0000","last_active":"2024-09-19T06:47:40.835734+0000","last_peered":"2024-09-19T06:47:40.835734+0000","last_clean":"2024-09-19T06:47:40.835734+0000","last_became_active":"2024-09-19T06:47:22.851125+0000","last_became_peered":"2024-09-19T06:47:22.851125+0000","last_unstale":"2024-09-19T06:47:40.835734+0000","last_undegraded":"2024-09-19T06:47:40.835734+0000","last_fullsized":"2024-09-19T06:47:40.835734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:42.954034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172823+0000","last_change":"2024-09-19T06:47:24.851007+0000","last_active":"2024-09-19T06:47:41.172823+0000","last_peered":"2024-09-19T06:47:41.172823+0000","last_clean":"2024-09-19T06:47:41.172823+0000","last_became_active":"2024-09-19T06:47:24.850465+0000","last_became_peered":"2024-09-19T06:47:24.850465+0000","last_unstale":"2024-09-19T06:47:41.172823+0000","last_undegraded":"2024-09-19T06:47:41.172823+0000","last_fullsized":"2024-09-19T06:47:41.172823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:10.824672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'30","reported_seq":159,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:48:24.535560+0000","last_change":"2024-09-19T06:47:22.834893+0000","last_active":"2024-09-19T06:48:24.535560+0000","last_peered":"2024-09-19T06:48:24.535560+0000","last_clean":"2024-09-19T06:48:24.535560+0000","last_became_active":"2024-09-19T06:47:22.834521+0000","last_became_peered":"2024-09-19T06:47:22.834521+0000","last_unstale":"2024-09-19T06:48:24.535560+0000","last_undegraded":"2024-09-19T06:48:24.535560+0000","last_fullsized":"2024-09-19T06:48:24.535560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:29.742284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60690,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":112,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833599+0000","last_change":"2024-09-19T06:47:24.848191+0000","last_active":"2024-09-19T06:47:40.833599+0000","last_peered":"2024-09-19T06:47:40.833599+0000","last_clean":"2024-09-19T06:47:40.833599+0000","last_became_active":"2024-09-19T06:47:24.847889+0000","last_became_peered":"2024-09-19T06:47:24.847889+0000","last_unstale":"2024-09-19T06:47:40.833599+0000","last_undegraded":"2024-09-19T06:47:40.833599+0000","last_fullsized":"2024-09-19T06:47:40.833599+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:32.574360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830250+0000","last_change":"2024-09-19T06:47:22.832005+0000","last_active":"2024-09-19T06:47:40.830250+0000","last_peered":"2024-09-19T06:47:40.830250+0000","last_clean":"2024-09-19T06:47:40.830250+0000","last_became_active":"2024-09-19T06:47:22.831342+0000","last_became_peered":"2024-09-19T06:47:22.831342+0000","last_unstale":"2024-09-19T06:47:40.830250+0000","last_undegraded":"2024-09-19T06:47:40.830250+0000","last_fullsized":"2024-09-19T06:47:40.830250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:53.444748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830453+0000","last_change":"2024-09-19T06:47:24.843328+0000","last_active":"2024-09-19T06:47:40.830453+0000","last_peered":"2024-09-19T06:47:40.830453+0000","last_clean":"2024-09-19T06:47:40.830453+0000","last_became_active":"2024-09-19T06:47:24.842344+0000","last_became_peered":"2024-09-19T06:47:24.842344+0000","last_unstale":"2024-09-19T06:47:40.830453+0000","last_undegraded":"2024-09-19T06:47:40.830453+0000","last_fullsized":"2024-09-19T06:47:40.830453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:11.247819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.844801+0000","last_change":"2024-09-19T06:47:40.844086+0000","last_active":"2024-09-19T06:47:40.844801+0000","last_peered":"2024-09-19T06:47:40.844801+0000","last_clean":"2024-09-19T06:47:40.844801+0000","last_became_active":"2024-09-19T06:47:40.835015+0000","last_became_peered":"2024-09-19T06:47:40.835015+0000","last_unstale":"2024-09-19T06:47:40.844801+0000","last_undegraded":"2024-09-19T06:47:40.844801+0000","last_fullsized":"2024-09-19T06:47:40.844801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:09.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.836466+0000","last_change":"2024-09-19T06:47:40.836466+0000","last_active":"2024-09-19T06:47:40.836466+0000","last_peered":"2024-09-19T06:47:40.836466+0000","last_clean":"2024-09-19T06:47:40.836466+0000","last_became_active":"2024-09-19T06:47:40.836237+0000","last_became_peered":"2024-09-19T06:47:40.836237+0000","last_unstale":"2024-09-19T06:47:40.836466+0000","last_undegraded":"2024-09-19T06:47:40.836466+0000","last_fullsized":"2024-09-19T06:47:40.836466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:00.094685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":77,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:48:20.530968+0000","last_change":"2024-09-19T06:47:22.830359+0000","last_active":"2024-09-19T06:48:20.530968+0000","last_peered":"2024-09-19T06:48:20.530968+0000","last_clean":"2024-09-19T06:48:20.530968+0000","last_became_active":"2024-09-19T06:47:22.829960+0000","last_became_peered":"2024-09-19T06:47:22.829960+0000","last_unstale":"2024-09-19T06:48:20.530968+0000","last_undegraded":"2024-09-19T06:48:20.530968+0000","last_fullsized":"2024-09-19T06:48:20.530968+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:55.998243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830424+0000","last_change":"2024-09-19T06:47:24.836668+0000","last_active":"2024-09-19T06:47:40.830424+0000","last_peered":"2024-09-19T06:47:40.830424+0000","last_clean":"2024-09-19T06:47:40.830424+0000","last_became_active":"2024-09-19T06:47:24.836393+0000","last_became_peered":"2024-09-19T06:47:24.836393+0000","last_unstale":"2024-09-19T06:47:40.830424+0000","last_undegraded":"2024-09-19T06:47:40.830424+0000","last_fullsized":"2024-09-19T06:47:40.830424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:26.248948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.858323+0000","last_change":"2024-09-19T06:47:40.858323+0000","last_active":"2024-09-19T06:47:40.858323+0000","last_peered":"2024-09-19T06:47:40.858323+0000","last_clean":"2024-09-19T06:47:40.858323+0000","last_became_active":"2024-09-19T06:47:40.840802+0000","last_became_peered":"2024-09-19T06:47:40.840802+0000","last_unstale":"2024-09-19T06:47:40.858323+0000","last_undegraded":"2024-09-19T06:47:40.858323+0000","last_fullsized":"2024-09-19T06:47:40.858323+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:14.666916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834268+0000","last_change":"2024-09-19T06:47:24.843681+0000","last_active":"2024-09-19T06:47:40.834268+0000","last_peered":"2024-09-19T06:47:40.834268+0000","last_clean":"2024-09-19T06:47:40.834268+0000","last_became_active":"2024-09-19T06:47:24.841228+0000","last_became_peered":"2024-09-19T06:47:24.841228+0000","last_unstale":"2024-09-19T06:47:40.834268+0000","last_undegraded":"2024-09-19T06:47:40.834268+0000","last_fullsized":"2024-09-19T06:47:40.834268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:56.418962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833647+0000","last_change":"2024-09-19T06:47:22.833893+0000","last_active":"2024-09-19T06:47:40.833647+0000","last_peered":"2024-09-19T06:47:40.833647+0000","last_clean":"2024-09-19T06:47:40.833647+0000","last_became_active":"2024-09-19T06:47:22.833217+0000","last_became_peered":"2024-09-19T06:47:22.833217+0000","last_unstale":"2024-09-19T06:47:40.833647+0000","last_undegraded":"2024-09-19T06:47:40.833647+0000","last_fullsized":"2024-09-19T06:47:40.833647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:19.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831588+0000","last_change":"2024-09-19T06:47:24.848399+0000","last_active":"2024-09-19T06:47:40.831588+0000","last_peered":"2024-09-19T06:47:40.831588+0000","last_clean":"2024-09-19T06:47:40.831588+0000","last_became_active":"2024-09-19T06:47:24.848081+0000","last_became_peered":"2024-09-19T06:47:24.848081+0000","last_unstale":"2024-09-19T06:47:40.831588+0000","last_undegraded":"2024-09-19T06:47:40.831588+0000","last_fullsized":"2024-09-19T06:47:40.831588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:44.292939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.860955+0000","last_change":"2024-09-19T06:47:40.860955+0000","last_active":"2024-09-19T06:47:40.860955+0000","last_peered":"2024-09-19T06:47:40.860955+0000","last_clean":"2024-09-19T06:47:40.860955+0000","last_became_active":"2024-09-19T06:47:40.835126+0000","last_became_peered":"2024-09-19T06:47:40.835126+0000","last_unstale":"2024-09-19T06:47:40.860955+0000","last_undegraded":"2024-09-19T06:47:40.860955+0000","last_fullsized":"2024-09-19T06:47:40.860955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:02:26.763229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830803+0000","last_change":"2024-09-19T06:47:24.849680+0000","last_active":"2024-09-19T06:47:40.830803+0000","last_peered":"2024-09-19T06:47:40.830803+0000","last_clean":"2024-09-19T06:47:40.830803+0000","last_became_active":"2024-09-19T06:47:24.849323+0000","last_became_peered":"2024-09-19T06:47:24.849323+0000","last_unstale":"2024-09-19T06:47:40.830803+0000","last_undegraded":"2024-09-19T06:47:40.830803+0000","last_fullsized":"2024-09-19T06:47:40.830803+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:59.763503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.835875+0000","last_change":"2024-09-19T06:47:22.831689+0000","last_active":"2024-09-19T06:47:40.835875+0000","last_peered":"2024-09-19T06:47:40.835875+0000","last_clean":"2024-09-19T06:47:40.835875+0000","last_became_active":"2024-09-19T06:47:22.829283+0000","last_became_peered":"2024-09-19T06:47:22.829283+0000","last_unstale":"2024-09-19T06:47:40.835875+0000","last_undegraded":"2024-09-19T06:47:40.835875+0000","last_fullsized":"2024-09-19T06:47:40.835875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:37.109542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830659+0000","last_change":"2024-09-19T06:47:24.842793+0000","last_active":"2024-09-19T06:47:40.830659+0000","last_peered":"2024-09-19T06:47:40.830659+0000","last_clean":"2024-09-19T06:47:40.830659+0000","last_became_active":"2024-09-19T06:47:24.842474+0000","last_became_peered":"2024-09-19T06:47:24.842474+0000","last_unstale":"2024-09-19T06:47:40.830659+0000","last_undegraded":"2024-09-19T06:47:40.830659+0000","last_fullsized":"2024-09-19T06:47:40.830659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:23.110736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'6","reported_seq":77,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:48:20.530860+0000","last_change":"2024-09-19T06:47:23.047356+0000","last_active":"2024-09-19T06:48:20.530860+0000","last_peered":"2024-09-19T06:48:20.530860+0000","last_clean":"2024-09-19T06:48:20.530860+0000","last_became_active":"2024-09-19T06:47:23.046941+0000","last_became_peered":"2024-09-19T06:47:23.046941+0000","last_unstale":"2024-09-19T06:48:20.530860+0000","last_undegraded":"2024-09-19T06:48:20.530860+0000","last_fullsized":"2024-09-19T06:48:20.530860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:43.490952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":49,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830805+0000","last_change":"2024-09-19T06:47:24.851025+0000","last_active":"2024-09-19T06:47:40.830805+0000","last_peered":"2024-09-19T06:47:40.830805+0000","last_clean":"2024-09-19T06:47:40.830805+0000","last_became_active":"2024-09-19T06:47:24.850681+0000","last_became_peered":"2024-09-19T06:47:24.850681+0000","last_unstale":"2024-09-19T06:47:40.830805+0000","last_undegraded":"2024-09-19T06:47:40.830805+0000","last_fullsized":"2024-09-19T06:47:40.830805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:46.884747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":345,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830558+0000","last_change":"2024-09-19T06:46:13.927379+0000","last_active":"2024-09-19T06:47:40.830558+0000","last_peered":"2024-09-19T06:47:40.830558+0000","last_clean":"2024-09-19T06:47:40.830558+0000","last_became_active":"2024-09-19T06:46:13.926926+0000","last_became_peered":"2024-09-19T06:46:13.926926+0000","last_unstale":"2024-09-19T06:47:40.830558+0000","last_undegraded":"2024-09-19T06:47:40.830558+0000","last_fullsized":"2024-09-19T06:47:40.830558+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_clean_scrub_stamp":"2024-09-19T06:42:52.051335+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:29.542128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829278+0000","last_change":"2024-09-19T06:47:22.830387+0000","last_active":"2024-09-19T06:47:40.829278+0000","last_peered":"2024-09-19T06:47:40.829278+0000","last_clean":"2024-09-19T06:47:40.829278+0000","last_became_active":"2024-09-19T06:47:22.829970+0000","last_became_peered":"2024-09-19T06:47:22.829970+0000","last_unstale":"2024-09-19T06:47:40.829278+0000","last_undegraded":"2024-09-19T06:47:40.829278+0000","last_fullsized":"2024-09-19T06:47:40.829278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:51.385456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834053+0000","last_change":"2024-09-19T06:47:24.843752+0000","last_active":"2024-09-19T06:47:40.834053+0000","last_peered":"2024-09-19T06:47:40.834053+0000","last_clean":"2024-09-19T06:47:40.834053+0000","last_became_active":"2024-09-19T06:47:24.842631+0000","last_became_peered":"2024-09-19T06:47:24.842631+0000","last_unstale":"2024-09-19T06:47:40.834053+0000","last_undegraded":"2024-09-19T06:47:40.834053+0000","last_fullsized":"2024-09-19T06:47:40.834053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:19.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830650+0000","last_change":"2024-09-19T06:47:22.832647+0000","last_active":"2024-09-19T06:47:40.830650+0000","last_peered":"2024-09-19T06:47:40.830650+0000","last_clean":"2024-09-19T06:47:40.830650+0000","last_became_active":"2024-09-19T06:47:22.831774+0000","last_became_peered":"2024-09-19T06:47:22.831774+0000","last_unstale":"2024-09-19T06:47:40.830650+0000","last_undegraded":"2024-09-19T06:47:40.830650+0000","last_fullsized":"2024-09-19T06:47:40.830650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:57.824610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830409+0000","last_change":"2024-09-19T06:47:24.844301+0000","last_active":"2024-09-19T06:47:40.830409+0000","last_peered":"2024-09-19T06:47:40.830409+0000","last_clean":"2024-09-19T06:47:40.830409+0000","last_became_active":"2024-09-19T06:47:24.839748+0000","last_became_peered":"2024-09-19T06:47:24.839748+0000","last_unstale":"2024-09-19T06:47:40.830409+0000","last_undegraded":"2024-09-19T06:47:40.830409+0000","last_fullsized":"2024-09-19T06:47:40.830409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:46.407693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":73,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:48:19.528993+0000","last_change":"2024-09-19T06:47:22.851008+0000","last_active":"2024-09-19T06:48:19.528993+0000","last_peered":"2024-09-19T06:48:19.528993+0000","last_clean":"2024-09-19T06:48:19.528993+0000","last_became_active":"2024-09-19T06:47:22.850601+0000","last_became_peered":"2024-09-19T06:47:22.850601+0000","last_unstale":"2024-09-19T06:48:19.528993+0000","last_undegraded":"2024-09-19T06:48:19.528993+0000","last_fullsized":"2024-09-19T06:48:19.528993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:04.610277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834177+0000","last_change":"2024-09-19T06:47:24.839684+0000","last_active":"2024-09-19T06:47:40.834177+0000","last_peered":"2024-09-19T06:47:40.834177+0000","last_clean":"2024-09-19T06:47:40.834177+0000","last_became_active":"2024-09-19T06:47:24.839126+0000","last_became_peered":"2024-09-19T06:47:24.839126+0000","last_unstale":"2024-09-19T06:47:40.834177+0000","last_undegraded":"2024-09-19T06:47:40.834177+0000","last_fullsized":"2024-09-19T06:47:40.834177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:09.458535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":72,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:48:19.240026+0000","last_change":"2024-09-19T06:47:22.831819+0000","last_active":"2024-09-19T06:48:19.240026+0000","last_peered":"2024-09-19T06:48:19.240026+0000","last_clean":"2024-09-19T06:48:19.240026+0000","last_became_active":"2024-09-19T06:47:22.831332+0000","last_became_peered":"2024-09-19T06:47:22.831332+0000","last_unstale":"2024-09-19T06:48:19.240026+0000","last_undegraded":"2024-09-19T06:48:19.240026+0000","last_fullsized":"2024-09-19T06:48:19.240026+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:35.032949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.171750+0000","last_change":"2024-09-19T06:47:24.846738+0000","last_active":"2024-09-19T06:47:41.171750+0000","last_peered":"2024-09-19T06:47:41.171750+0000","last_clean":"2024-09-19T06:47:41.171750+0000","last_became_active":"2024-09-19T06:47:24.846389+0000","last_became_peered":"2024-09-19T06:47:24.846389+0000","last_unstale":"2024-09-19T06:47:41.171750+0000","last_undegraded":"2024-09-19T06:47:41.171750+0000","last_fullsized":"2024-09-19T06:47:41.171750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:19.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833696+0000","last_change":"2024-09-19T06:47:22.851065+0000","last_active":"2024-09-19T06:47:40.833696+0000","last_peered":"2024-09-19T06:47:40.833696+0000","last_clean":"2024-09-19T06:47:40.833696+0000","last_became_active":"2024-09-19T06:47:22.850647+0000","last_became_peered":"2024-09-19T06:47:22.850647+0000","last_unstale":"2024-09-19T06:47:40.833696+0000","last_undegraded":"2024-09-19T06:47:40.833696+0000","last_fullsized":"2024-09-19T06:47:40.833696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:41.993501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172204+0000","last_change":"2024-09-19T06:47:24.838905+0000","last_active":"2024-09-19T06:47:41.172204+0000","last_peered":"2024-09-19T06:47:41.172204+0000","last_clean":"2024-09-19T06:47:41.172204+0000","last_became_active":"2024-09-19T06:47:24.838443+0000","last_became_peered":"2024-09-19T06:47:24.838443+0000","last_unstale":"2024-09-19T06:47:41.172204+0000","last_undegraded":"2024-09-19T06:47:41.172204+0000","last_fullsized":"2024-09-19T06:47:41.172204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:12.486137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833766+0000","last_change":"2024-09-19T06:47:22.851431+0000","last_active":"2024-09-19T06:47:40.833766+0000","last_peered":"2024-09-19T06:47:40.833766+0000","last_clean":"2024-09-19T06:47:40.833766+0000","last_became_active":"2024-09-19T06:47:22.851092+0000","last_became_peered":"2024-09-19T06:47:22.851092+0000","last_unstale":"2024-09-19T06:47:40.833766+0000","last_undegraded":"2024-09-19T06:47:40.833766+0000","last_fullsized":"2024-09-19T06:47:40.833766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:26.183803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830789+0000","last_change":"2024-09-19T06:47:24.839046+0000","last_active":"2024-09-19T06:47:40.830789+0000","last_peered":"2024-09-19T06:47:40.830789+0000","last_clean":"2024-09-19T06:47:40.830789+0000","last_became_active":"2024-09-19T06:47:24.838680+0000","last_became_peered":"2024-09-19T06:47:24.838680+0000","last_unstale":"2024-09-19T06:47:40.830789+0000","last_undegraded":"2024-09-19T06:47:40.830789+0000","last_fullsized":"2024-09-19T06:47:40.830789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:45.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'15","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:49.846429+0000","last_change":"2024-09-19T06:47:40.865241+0000","last_active":"2024-09-19T06:47:49.846429+0000","last_peered":"2024-09-19T06:47:49.846429+0000","last_clean":"2024-09-19T06:47:49.846429+0000","last_became_active":"2024-09-19T06:47:40.837765+0000","last_became_peered":"2024-09-19T06:47:40.837765+0000","last_unstale":"2024-09-19T06:47:49.846429+0000","last_undegraded":"2024-09-19T06:47:49.846429+0000","last_fullsized":"2024-09-19T06:47:49.846429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834162+0000","last_change":"2024-09-19T06:47:24.848599+0000","last_active":"2024-09-19T06:47:40.834162+0000","last_peered":"2024-09-19T06:47:40.834162+0000","last_clean":"2024-09-19T06:47:40.834162+0000","last_became_active":"2024-09-19T06:47:24.848305+0000","last_became_peered":"2024-09-19T06:47:24.848305+0000","last_unstale":"2024-09-19T06:47:40.834162+0000","last_undegraded":"2024-09-19T06:47:40.834162+0000","last_fullsized":"2024-09-19T06:47:40.834162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:31.496188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829462+0000","last_change":"2024-09-19T06:47:23.047484+0000","last_active":"2024-09-19T06:47:40.829462+0000","last_peered":"2024-09-19T06:47:40.829462+0000","last_clean":"2024-09-19T06:47:40.829462+0000","last_became_active":"2024-09-19T06:47:23.047058+0000","last_became_peered":"2024-09-19T06:47:23.047058+0000","last_unstale":"2024-09-19T06:47:40.829462+0000","last_undegraded":"2024-09-19T06:47:40.829462+0000","last_fullsized":"2024-09-19T06:47:40.829462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:34.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839396+0000","last_change":"2024-09-19T06:47:24.843498+0000","last_active":"2024-09-19T06:47:40.839396+0000","last_peered":"2024-09-19T06:47:40.839396+0000","last_clean":"2024-09-19T06:47:40.839396+0000","last_became_active":"2024-09-19T06:47:24.843204+0000","last_became_peered":"2024-09-19T06:47:24.843204+0000","last_unstale":"2024-09-19T06:47:40.839396+0000","last_undegraded":"2024-09-19T06:47:40.839396+0000","last_fullsized":"2024-09-19T06:47:40.839396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:25.585023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172992+0000","last_change":"2024-09-19T06:47:22.851519+0000","last_active":"2024-09-19T06:47:41.172992+0000","last_peered":"2024-09-19T06:47:41.172992+0000","last_clean":"2024-09-19T06:47:41.172992+0000","last_became_active":"2024-09-19T06:47:22.851163+0000","last_became_peered":"2024-09-19T06:47:22.851163+0000","last_unstale":"2024-09-19T06:47:41.172992+0000","last_undegraded":"2024-09-19T06:47:41.172992+0000","last_fullsized":"2024-09-19T06:47:41.172992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:04.502795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.832719+0000","last_change":"2024-09-19T06:47:24.845239+0000","last_active":"2024-09-19T06:47:40.832719+0000","last_peered":"2024-09-19T06:47:40.832719+0000","last_clean":"2024-09-19T06:47:40.832719+0000","last_became_active":"2024-09-19T06:47:24.844934+0000","last_became_peered":"2024-09-19T06:47:24.844934+0000","last_unstale":"2024-09-19T06:47:40.832719+0000","last_undegraded":"2024-09-19T06:47:40.832719+0000","last_fullsized":"2024-09-19T06:47:40.832719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:43.610604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834147+0000","last_change":"2024-09-19T06:47:22.830232+0000","last_active":"2024-09-19T06:47:40.834147+0000","last_peered":"2024-09-19T06:47:40.834147+0000","last_clean":"2024-09-19T06:47:40.834147+0000","last_became_active":"2024-09-19T06:47:22.829901+0000","last_became_peered":"2024-09-19T06:47:22.829901+0000","last_unstale":"2024-09-19T06:47:40.834147+0000","last_undegraded":"2024-09-19T06:47:40.834147+0000","last_fullsized":"2024-09-19T06:47:40.834147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:12.251427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831384+0000","last_change":"2024-09-19T06:47:24.839304+0000","last_active":"2024-09-19T06:47:40.831384+0000","last_peered":"2024-09-19T06:47:40.831384+0000","last_clean":"2024-09-19T06:47:40.831384+0000","last_became_active":"2024-09-19T06:47:24.838986+0000","last_became_peered":"2024-09-19T06:47:24.838986+0000","last_unstale":"2024-09-19T06:47:40.831384+0000","last_undegraded":"2024-09-19T06:47:40.831384+0000","last_fullsized":"2024-09-19T06:47:40.831384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:39.377886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830021+0000","last_change":"2024-09-19T06:47:22.828553+0000","last_active":"2024-09-19T06:47:40.830021+0000","last_peered":"2024-09-19T06:47:40.830021+0000","last_clean":"2024-09-19T06:47:40.830021+0000","last_became_active":"2024-09-19T06:47:22.828213+0000","last_became_peered":"2024-09-19T06:47:22.828213+0000","last_unstale":"2024-09-19T06:47:40.830021+0000","last_undegraded":"2024-09-19T06:47:40.830021+0000","last_fullsized":"2024-09-19T06:47:40.830021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:31.649761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833576+0000","last_change":"2024-09-19T06:47:24.840747+0000","last_active":"2024-09-19T06:47:40.833576+0000","last_peered":"2024-09-19T06:47:40.833576+0000","last_clean":"2024-09-19T06:47:40.833576+0000","last_became_active":"2024-09-19T06:47:24.840451+0000","last_became_peered":"2024-09-19T06:47:24.840451+0000","last_unstale":"2024-09-19T06:47:40.833576+0000","last_undegraded":"2024-09-19T06:47:40.833576+0000","last_fullsized":"2024-09-19T06:47:40.833576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:41.588751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833228+0000","last_change":"2024-09-19T06:47:22.831697+0000","last_active":"2024-09-19T06:47:40.833228+0000","last_peered":"2024-09-19T06:47:40.833228+0000","last_clean":"2024-09-19T06:47:40.833228+0000","last_became_active":"2024-09-19T06:47:22.831178+0000","last_became_peered":"2024-09-19T06:47:22.831178+0000","last_unstale":"2024-09-19T06:47:40.833228+0000","last_undegraded":"2024-09-19T06:47:40.833228+0000","last_fullsized":"2024-09-19T06:47:40.833228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:30.975782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830575+0000","last_change":"2024-09-19T06:47:24.845097+0000","last_active":"2024-09-19T06:47:40.830575+0000","last_peered":"2024-09-19T06:47:40.830575+0000","last_clean":"2024-09-19T06:47:40.830575+0000","last_became_active":"2024-09-19T06:47:24.844801+0000","last_became_peered":"2024-09-19T06:47:24.844801+0000","last_unstale":"2024-09-19T06:47:40.830575+0000","last_undegraded":"2024-09-19T06:47:40.830575+0000","last_fullsized":"2024-09-19T06:47:40.830575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:15.264118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.835620+0000","last_change":"2024-09-19T06:47:24.851372+0000","last_active":"2024-09-19T06:47:40.835620+0000","last_peered":"2024-09-19T06:47:40.835620+0000","last_clean":"2024-09-19T06:47:40.835620+0000","last_became_active":"2024-09-19T06:47:24.851044+0000","last_became_peered":"2024-09-19T06:47:24.851044+0000","last_unstale":"2024-09-19T06:47:40.835620+0000","last_undegraded":"2024-09-19T06:47:40.835620+0000","last_fullsized":"2024-09-19T06:47:40.835620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:55.658214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833103+0000","last_change":"2024-09-19T06:47:22.833694+0000","last_active":"2024-09-19T06:47:40.833103+0000","last_peered":"2024-09-19T06:47:40.833103+0000","last_clean":"2024-09-19T06:47:40.833103+0000","last_became_active":"2024-09-19T06:47:22.833215+0000","last_became_peered":"2024-09-19T06:47:22.833215+0000","last_unstale":"2024-09-19T06:47:40.833103+0000","last_undegraded":"2024-09-19T06:47:40.833103+0000","last_fullsized":"2024-09-19T06:47:40.833103+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:32.171522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172973+0000","last_change":"2024-09-19T06:47:24.845737+0000","last_active":"2024-09-19T06:47:41.172973+0000","last_peered":"2024-09-19T06:47:41.172973+0000","last_clean":"2024-09-19T06:47:41.172973+0000","last_became_active":"2024-09-19T06:47:24.845242+0000","last_became_peered":"2024-09-19T06:47:24.845242+0000","last_unstale":"2024-09-19T06:47:41.172973+0000","last_undegraded":"2024-09-19T06:47:41.172973+0000","last_fullsized":"2024-09-19T06:47:41.172973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:51.554946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833601+0000","last_change":"2024-09-19T06:47:22.829869+0000","last_active":"2024-09-19T06:47:40.833601+0000","last_peered":"2024-09-19T06:47:40.833601+0000","last_clean":"2024-09-19T06:47:40.833601+0000","last_became_active":"2024-09-19T06:47:22.829522+0000","last_became_peered":"2024-09-19T06:47:22.829522+0000","last_unstale":"2024-09-19T06:47:40.833601+0000","last_undegraded":"2024-09-19T06:47:40.833601+0000","last_fullsized":"2024-09-19T06:47:40.833601+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:26.871434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.840141+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:47:40.840141+0000","last_peered":"2024-09-19T06:47:40.840141+0000","last_clean":"2024-09-19T06:47:40.840141+0000","last_became_active":"2024-09-19T06:47:24.841289+0000","last_became_peered":"2024-09-19T06:47:24.841289+0000","last_unstale":"2024-09-19T06:47:40.840141+0000","last_undegraded":"2024-09-19T06:47:40.840141+0000","last_fullsized":"2024-09-19T06:47:40.840141+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:21:12.611458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834465+0000","last_change":"2024-09-19T06:47:22.852198+0000","last_active":"2024-09-19T06:47:40.834465+0000","last_peered":"2024-09-19T06:47:40.834465+0000","last_clean":"2024-09-19T06:47:40.834465+0000","last_became_active":"2024-09-19T06:47:22.850984+0000","last_became_peered":"2024-09-19T06:47:22.850984+0000","last_unstale":"2024-09-19T06:47:40.834465+0000","last_undegraded":"2024-09-19T06:47:40.834465+0000","last_fullsized":"2024-09-19T06:47:40.834465+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:13.620145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839399+0000","last_change":"2024-09-19T06:47:40.837614+0000","last_active":"2024-09-19T06:47:40.839399+0000","last_peered":"2024-09-19T06:47:40.839399+0000","last_clean":"2024-09-19T06:47:40.839399+0000","last_became_active":"2024-09-19T06:47:40.837259+0000","last_became_peered":"2024-09-19T06:47:40.837259+0000","last_unstale":"2024-09-19T06:47:40.839399+0000","last_undegraded":"2024-09-19T06:47:40.839399+0000","last_fullsized":"2024-09-19T06:47:40.839399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:41.429962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:48:19.239192+0000","last_change":"2024-09-19T06:47:41.182639+0000","last_active":"2024-09-19T06:48:19.239192+0000","last_peered":"2024-09-19T06:48:19.239192+0000","last_clean":"2024-09-19T06:48:19.239192+0000","last_became_active":"2024-09-19T06:47:41.174305+0000","last_became_peered":"2024-09-19T06:47:41.174305+0000","last_unstale":"2024-09-19T06:48:19.239192+0000","last_undegraded":"2024-09-19T06:48:19.239192+0000","last_fullsized":"2024-09-19T06:48:19.239192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:10.199502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67898,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":508,"num_read_kb":328,"num_write":243,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":203694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579929,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":439122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808862,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1090751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972387,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":438569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234024,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":913408,"data_stored":438275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462956,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712484,"statfs":{"total":95995035648,"available":95961583616,"internally_reserved":0,"allocated":1519616,"data_stored":1031467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626482,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":909312,"data_stored":438286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855414,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1030662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018939,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":438652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215165,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":438111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444102,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":500390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":500561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":440018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:48:28.216 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:48:28.473 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:28.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:28 smithi103 ceph-mon[31158]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:48:28.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:28.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:28.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:28 smithi103 ceph-mon[31158]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T06:48:28.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:48:28.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:48:28.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:28 smithi103 ceph-mon[31158]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:28.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.965+0000 7f55fd342640 1 -- 172.21.15.103:0/904423437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f8102ef0 msgr2=0x7f55f810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:28.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.965+0000 7f55fd342640 1 --2- 172.21.15.103:0/904423437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f8102ef0 0x7f55f810da40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009900 tx=0x7f55e802a480 comp rx=0 tx=0).stop 2024-09-19T06:48:28.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.965+0000 7f55fd342640 1 -- 172.21.15.103:0/904423437 shutdown_connections 2024-09-19T06:48:28.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.965+0000 7f55fd342640 1 --2- 172.21.15.103:0/904423437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f8102ef0 0x7f55f810da40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:28.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.965+0000 7f55fd342640 1 --2- 172.21.15.103:0/904423437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55f8102570 0x7f55f81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:28.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.965+0000 7f55fd342640 1 --2- 172.21.15.103:0/904423437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f8100e20 0x7f55f8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:28.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.965+0000 7f55fd342640 1 -- 172.21.15.103:0/904423437 >> 172.21.15.103:0/904423437 conn(0x7f55f80fc7d0 msgr2=0x7f55f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:28.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.965+0000 7f55fd342640 1 -- 172.21.15.103:0/904423437 shutdown_connections 2024-09-19T06:48:28.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.965+0000 7f55fd342640 1 -- 172.21.15.103:0/904423437 wait complete. 2024-09-19T06:48:28.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.966+0000 7f55fd342640 1 Processor -- start 2024-09-19T06:48:28.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.966+0000 7f55fd342640 1 -- start start 2024-09-19T06:48:28.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.966+0000 7f55fd342640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f8100e20 0x7f55f8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:28.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.966+0000 7f55fd342640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55f8102570 0x7f55f8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:28.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.966+0000 7f55fd342640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f8102ef0 0x7f55f819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:28.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.966+0000 7f55fd342640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81120a0 con 0x7f55f8100e20 2024-09-19T06:48:28.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.966+0000 7f55fd342640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8111f20 con 0x7f55f8102ef0 2024-09-19T06:48:28.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.966+0000 7f55fd342640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8112220 con 0x7f55f8102570 2024-09-19T06:48:28.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.966+0000 7f55f6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f8100e20 0x7f55f8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:28.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.967+0000 7f55f6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f8100e20 0x7f55f8198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46440/0 (socket says 172.21.15.103:46440) 2024-09-19T06:48:28.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.967+0000 7f55f6ffd640 1 -- 172.21.15.103:0/541033630 learned_addr learned my addr 172.21.15.103:0/541033630 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:28.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.967+0000 7f55f77fe640 1 --2- 172.21.15.103:0/541033630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f8102ef0 0x7f55f819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:28.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.967+0000 7f55f67fc640 1 --2- 172.21.15.103:0/541033630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55f8102570 0x7f55f8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:28.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.967+0000 7f55f6ffd640 1 -- 172.21.15.103:0/541033630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55f8102570 msgr2=0x7f55f8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:28.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.967+0000 7f55f6ffd640 1 --2- 172.21.15.103:0/541033630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55f8102570 0x7f55f8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:28.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.967+0000 7f55f6ffd640 1 -- 172.21.15.103:0/541033630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f8102ef0 msgr2=0x7f55f819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:28.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.967+0000 7f55f6ffd640 1 --2- 172.21.15.103:0/541033630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f8102ef0 0x7f55f819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:28.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.967+0000 7f55f6ffd640 1 -- 172.21.15.103:0/541033630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f819d3e0 con 0x7f55f8100e20 2024-09-19T06:48:28.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.967+0000 7f55f67fc640 1 --2- 172.21.15.103:0/541033630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55f8102570 0x7f55f8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:28.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.967+0000 7f55f6ffd640 1 --2- 172.21.15.103:0/541033630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f8100e20 0x7f55f8198430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f55f000cce0 tx=0x7f55f000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:28.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.967+0000 7f55dffff640 1 -- 172.21.15.103:0/541033630 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f000f650 con 0x7f55f8100e20 2024-09-19T06:48:28.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.967+0000 7f55fd342640 1 -- 172.21.15.103:0/541033630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f819d6d0 con 0x7f55f8100e20 2024-09-19T06:48:28.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.967+0000 7f55dffff640 1 -- 172.21.15.103:0/541033630 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f000fcd0 con 0x7f55f8100e20 2024-09-19T06:48:28.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.967+0000 7f55dffff640 1 -- 172.21.15.103:0/541033630 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0018c70 con 0x7f55f8100e20 2024-09-19T06:48:28.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.967+0000 7f55fd342640 1 -- 172.21.15.103:0/541033630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f81a4fb0 con 0x7f55f8100e20 2024-09-19T06:48:28.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.968+0000 7f55fd342640 1 -- 172.21.15.103:0/541033630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f8101200 con 0x7f55f8100e20 2024-09-19T06:48:28.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.969+0000 7f55dffff640 1 -- 172.21.15.103:0/541033630 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f55f000f7f0 con 0x7f55f8100e20 2024-09-19T06:48:28.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.970+0000 7f55dffff640 1 --2- 172.21.15.103:0/541033630 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f55e00a32d0 0x7f55e00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:28.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.970+0000 7f55f67fc640 1 --2- 172.21.15.103:0/541033630 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f55e00a32d0 0x7f55e00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:28.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.970+0000 7f55dffff640 1 -- 172.21.15.103:0/541033630 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f55f00d02f0 con 0x7f55f8100e20 2024-09-19T06:48:28.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.970+0000 7f55f67fc640 1 --2- 172.21.15.103:0/541033630 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f55e00a32d0 0x7f55e00a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f55ec006fd0 tx=0x7f55ec008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:28.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:28.973+0000 7f55dffff640 1 -- 172.21.15.103:0/541033630 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f55f0014030 con 0x7f55f8100e20 2024-09-19T06:48:29.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:28 smithi174 ceph-mon[35637]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:48:29.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:29.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:29.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:28 smithi174 ceph-mon[35637]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T06:48:29.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:48:29.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:48:29.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:28 smithi174 ceph-mon[35637]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:29.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:29.140+0000 7f55fd342640 1 -- 172.21.15.103:0/541033630 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55f8106f10 con 0x7f55e00a32d0 2024-09-19T06:48:29.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:29.147+0000 7f55dffff640 1 -- 172.21.15.103:0/541033630 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258761 (secure 0 0 0) 0x7f55f8106f10 con 0x7f55e00a32d0 2024-09-19T06:48:29.148 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:48:29.153 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T06:48:29.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:29.150+0000 7f55fd342640 1 -- 172.21.15.103:0/541033630 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f55e00a32d0 msgr2=0x7f55e00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:29.150+0000 7f55fd342640 1 --2- 172.21.15.103:0/541033630 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f55e00a32d0 0x7f55e00a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f55ec006fd0 tx=0x7f55ec008040 comp rx=0 tx=0).stop 2024-09-19T06:48:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:29.150+0000 7f55fd342640 1 -- 172.21.15.103:0/541033630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f8100e20 msgr2=0x7f55f8198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:29.150+0000 7f55fd342640 1 --2- 172.21.15.103:0/541033630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f8100e20 0x7f55f8198430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f55f000cce0 tx=0x7f55f000a620 comp rx=0 tx=0).stop 2024-09-19T06:48:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:29.150+0000 7f55fd342640 1 -- 172.21.15.103:0/541033630 shutdown_connections 2024-09-19T06:48:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:29.150+0000 7f55fd342640 1 --2- 172.21.15.103:0/541033630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f8102ef0 0x7f55f819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:29.150+0000 7f55fd342640 1 --2- 172.21.15.103:0/541033630 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f55e00a32d0 0x7f55e00a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:29.150+0000 7f55fd342640 1 --2- 172.21.15.103:0/541033630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55f8102570 0x7f55f8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:29.150+0000 7f55fd342640 1 --2- 172.21.15.103:0/541033630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f8100e20 0x7f55f8198430 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:29.150+0000 7f55fd342640 1 -- 172.21.15.103:0/541033630 >> 172.21.15.103:0/541033630 conn(0x7f55f80fc7d0 msgr2=0x7f55f8106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:29.151+0000 7f55fd342640 1 -- 172.21.15.103:0/541033630 shutdown_connections 2024-09-19T06:48:29.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:29.151+0000 7f55fd342640 1 -- 172.21.15.103:0/541033630 wait complete. 2024-09-19T06:48:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:28 smithi134 ceph-mon[35356]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:48:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:28 smithi134 ceph-mon[35356]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T06:48:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:48:29.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:48:29.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:28 smithi134 ceph-mon[35356]: from='mgr.14150 172.21.15.103:0/3185701916' entity='mgr.x' 2024-09-19T06:48:29.804 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-19T06:48:27.835738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658500,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":669,"num_read_kb":457,"num_write":426,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":1181610,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353688,"kb_used_data":12732,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588136,"statfs":{"total":1151940427776,"available":1151578251264,"internally_reserved":0,"allocated":13037568,"data_stored":7224864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002690"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831591+0000","last_change":"2024-09-19T06:47:24.853118+0000","last_active":"2024-09-19T06:47:40.831591+0000","last_peered":"2024-09-19T06:47:40.831591+0000","last_clean":"2024-09-19T06:47:40.831591+0000","last_became_active":"2024-09-19T06:47:24.852801+0000","last_became_peered":"2024-09-19T06:47:24.852801+0000","last_unstale":"2024-09-19T06:47:40.831591+0000","last_undegraded":"2024-09-19T06:47:40.831591+0000","last_fullsized":"2024-09-19T06:47:40.831591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:16.729103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839300+0000","last_change":"2024-09-19T06:47:24.841095+0000","last_active":"2024-09-19T06:47:40.839300+0000","last_peered":"2024-09-19T06:47:40.839300+0000","last_clean":"2024-09-19T06:47:40.839300+0000","last_became_active":"2024-09-19T06:47:24.840774+0000","last_became_peered":"2024-09-19T06:47:24.840774+0000","last_unstale":"2024-09-19T06:47:40.839300+0000","last_undegraded":"2024-09-19T06:47:40.839300+0000","last_fullsized":"2024-09-19T06:47:40.839300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:27.262766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839649+0000","last_change":"2024-09-19T06:47:24.848444+0000","last_active":"2024-09-19T06:47:40.839649+0000","last_peered":"2024-09-19T06:47:40.839649+0000","last_clean":"2024-09-19T06:47:40.839649+0000","last_became_active":"2024-09-19T06:47:24.847354+0000","last_became_peered":"2024-09-19T06:47:24.847354+0000","last_unstale":"2024-09-19T06:47:40.839649+0000","last_undegraded":"2024-09-19T06:47:40.839649+0000","last_fullsized":"2024-09-19T06:47:40.839649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:44.901265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829447+0000","last_change":"2024-09-19T06:47:24.851915+0000","last_active":"2024-09-19T06:47:40.829447+0000","last_peered":"2024-09-19T06:47:40.829447+0000","last_clean":"2024-09-19T06:47:40.829447+0000","last_became_active":"2024-09-19T06:47:24.851600+0000","last_became_peered":"2024-09-19T06:47:24.851600+0000","last_unstale":"2024-09-19T06:47:40.829447+0000","last_undegraded":"2024-09-19T06:47:40.829447+0000","last_fullsized":"2024-09-19T06:47:40.829447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:52.424288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833135+0000","last_change":"2024-09-19T06:47:24.843184+0000","last_active":"2024-09-19T06:47:40.833135+0000","last_peered":"2024-09-19T06:47:40.833135+0000","last_clean":"2024-09-19T06:47:40.833135+0000","last_became_active":"2024-09-19T06:47:24.842857+0000","last_became_peered":"2024-09-19T06:47:24.842857+0000","last_unstale":"2024-09-19T06:47:40.833135+0000","last_undegraded":"2024-09-19T06:47:40.833135+0000","last_fullsized":"2024-09-19T06:47:40.833135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:06.608699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.171834+0000","last_change":"2024-09-19T06:47:24.846631+0000","last_active":"2024-09-19T06:47:41.171834+0000","last_peered":"2024-09-19T06:47:41.171834+0000","last_clean":"2024-09-19T06:47:41.171834+0000","last_became_active":"2024-09-19T06:47:24.846271+0000","last_became_peered":"2024-09-19T06:47:24.846271+0000","last_unstale":"2024-09-19T06:47:41.171834+0000","last_undegraded":"2024-09-19T06:47:41.171834+0000","last_fullsized":"2024-09-19T06:47:41.171834+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.272273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830458+0000","last_change":"2024-09-19T06:47:24.840392+0000","last_active":"2024-09-19T06:47:40.830458+0000","last_peered":"2024-09-19T06:47:40.830458+0000","last_clean":"2024-09-19T06:47:40.830458+0000","last_became_active":"2024-09-19T06:47:24.840092+0000","last_became_peered":"2024-09-19T06:47:24.840092+0000","last_unstale":"2024-09-19T06:47:40.830458+0000","last_undegraded":"2024-09-19T06:47:40.830458+0000","last_fullsized":"2024-09-19T06:47:40.830458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:25.579096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833345+0000","last_change":"2024-09-19T06:47:24.843336+0000","last_active":"2024-09-19T06:47:40.833345+0000","last_peered":"2024-09-19T06:47:40.833345+0000","last_clean":"2024-09-19T06:47:40.833345+0000","last_became_active":"2024-09-19T06:47:24.842016+0000","last_became_peered":"2024-09-19T06:47:24.842016+0000","last_unstale":"2024-09-19T06:47:40.833345+0000","last_undegraded":"2024-09-19T06:47:40.833345+0000","last_fullsized":"2024-09-19T06:47:40.833345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:49.478359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172619+0000","last_change":"2024-09-19T06:47:24.847841+0000","last_active":"2024-09-19T06:47:41.172619+0000","last_peered":"2024-09-19T06:47:41.172619+0000","last_clean":"2024-09-19T06:47:41.172619+0000","last_became_active":"2024-09-19T06:47:24.847492+0000","last_became_peered":"2024-09-19T06:47:24.847492+0000","last_unstale":"2024-09-19T06:47:41.172619+0000","last_undegraded":"2024-09-19T06:47:41.172619+0000","last_fullsized":"2024-09-19T06:47:41.172619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:59.229022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.171833+0000","last_change":"2024-09-19T06:47:24.843773+0000","last_active":"2024-09-19T06:47:41.171833+0000","last_peered":"2024-09-19T06:47:41.171833+0000","last_clean":"2024-09-19T06:47:41.171833+0000","last_became_active":"2024-09-19T06:47:24.843409+0000","last_became_peered":"2024-09-19T06:47:24.843409+0000","last_unstale":"2024-09-19T06:47:41.171833+0000","last_undegraded":"2024-09-19T06:47:41.171833+0000","last_fullsized":"2024-09-19T06:47:41.171833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:22.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830801+0000","last_change":"2024-09-19T06:47:24.846637+0000","last_active":"2024-09-19T06:47:40.830801+0000","last_peered":"2024-09-19T06:47:40.830801+0000","last_clean":"2024-09-19T06:47:40.830801+0000","last_became_active":"2024-09-19T06:47:24.846337+0000","last_became_peered":"2024-09-19T06:47:24.846337+0000","last_unstale":"2024-09-19T06:47:40.830801+0000","last_undegraded":"2024-09-19T06:47:40.830801+0000","last_fullsized":"2024-09-19T06:47:40.830801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:23:35.869485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829501+0000","last_change":"2024-09-19T06:47:24.851399+0000","last_active":"2024-09-19T06:47:40.829501+0000","last_peered":"2024-09-19T06:47:40.829501+0000","last_clean":"2024-09-19T06:47:40.829501+0000","last_became_active":"2024-09-19T06:47:24.851090+0000","last_became_peered":"2024-09-19T06:47:24.851090+0000","last_unstale":"2024-09-19T06:47:40.829501+0000","last_undegraded":"2024-09-19T06:47:40.829501+0000","last_fullsized":"2024-09-19T06:47:40.829501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:52.839956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830657+0000","last_change":"2024-09-19T06:47:24.833858+0000","last_active":"2024-09-19T06:47:40.830657+0000","last_peered":"2024-09-19T06:47:40.830657+0000","last_clean":"2024-09-19T06:47:40.830657+0000","last_became_active":"2024-09-19T06:47:24.833616+0000","last_became_peered":"2024-09-19T06:47:24.833616+0000","last_unstale":"2024-09-19T06:47:40.830657+0000","last_undegraded":"2024-09-19T06:47:40.830657+0000","last_fullsized":"2024-09-19T06:47:40.830657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:54.918144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839219+0000","last_change":"2024-09-19T06:47:24.841030+0000","last_active":"2024-09-19T06:47:40.839219+0000","last_peered":"2024-09-19T06:47:40.839219+0000","last_clean":"2024-09-19T06:47:40.839219+0000","last_became_active":"2024-09-19T06:47:24.839631+0000","last_became_peered":"2024-09-19T06:47:24.839631+0000","last_unstale":"2024-09-19T06:47:40.839219+0000","last_undegraded":"2024-09-19T06:47:40.839219+0000","last_fullsized":"2024-09-19T06:47:40.839219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:21.764812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839091+0000","last_change":"2024-09-19T06:47:24.849281+0000","last_active":"2024-09-19T06:47:40.839091+0000","last_peered":"2024-09-19T06:47:40.839091+0000","last_clean":"2024-09-19T06:47:40.839091+0000","last_became_active":"2024-09-19T06:47:24.848977+0000","last_became_peered":"2024-09-19T06:47:24.848977+0000","last_unstale":"2024-09-19T06:47:40.839091+0000","last_undegraded":"2024-09-19T06:47:40.839091+0000","last_fullsized":"2024-09-19T06:47:40.839091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:36.693758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172068+0000","last_change":"2024-09-19T06:47:24.843619+0000","last_active":"2024-09-19T06:47:41.172068+0000","last_peered":"2024-09-19T06:47:41.172068+0000","last_clean":"2024-09-19T06:47:41.172068+0000","last_became_active":"2024-09-19T06:47:24.843323+0000","last_became_peered":"2024-09-19T06:47:24.843323+0000","last_unstale":"2024-09-19T06:47:41.172068+0000","last_undegraded":"2024-09-19T06:47:41.172068+0000","last_fullsized":"2024-09-19T06:47:41.172068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:52.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831828+0000","last_change":"2024-09-19T06:47:24.853165+0000","last_active":"2024-09-19T06:47:40.831828+0000","last_peered":"2024-09-19T06:47:40.831828+0000","last_clean":"2024-09-19T06:47:40.831828+0000","last_became_active":"2024-09-19T06:47:24.852864+0000","last_became_peered":"2024-09-19T06:47:24.852864+0000","last_unstale":"2024-09-19T06:47:40.831828+0000","last_undegraded":"2024-09-19T06:47:40.831828+0000","last_fullsized":"2024-09-19T06:47:40.831828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:38.803478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834536+0000","last_change":"2024-09-19T06:47:24.838151+0000","last_active":"2024-09-19T06:47:40.834536+0000","last_peered":"2024-09-19T06:47:40.834536+0000","last_clean":"2024-09-19T06:47:40.834536+0000","last_became_active":"2024-09-19T06:47:24.837855+0000","last_became_peered":"2024-09-19T06:47:24.837855+0000","last_unstale":"2024-09-19T06:47:40.834536+0000","last_undegraded":"2024-09-19T06:47:40.834536+0000","last_fullsized":"2024-09-19T06:47:40.834536+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:16.675557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830451+0000","last_change":"2024-09-19T06:47:24.843941+0000","last_active":"2024-09-19T06:47:40.830451+0000","last_peered":"2024-09-19T06:47:40.830451+0000","last_clean":"2024-09-19T06:47:40.830451+0000","last_became_active":"2024-09-19T06:47:24.843551+0000","last_became_peered":"2024-09-19T06:47:24.843551+0000","last_unstale":"2024-09-19T06:47:40.830451+0000","last_undegraded":"2024-09-19T06:47:40.830451+0000","last_fullsized":"2024-09-19T06:47:40.830451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:00:49.388265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.836228+0000","last_change":"2024-09-19T06:47:40.836228+0000","last_active":"2024-09-19T06:47:40.836228+0000","last_peered":"2024-09-19T06:47:40.836228+0000","last_clean":"2024-09-19T06:47:40.836228+0000","last_became_active":"2024-09-19T06:47:40.835818+0000","last_became_peered":"2024-09-19T06:47:40.835818+0000","last_unstale":"2024-09-19T06:47:40.836228+0000","last_undegraded":"2024-09-19T06:47:40.836228+0000","last_fullsized":"2024-09-19T06:47:40.836228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:45.341160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830676+0000","last_change":"2024-09-19T06:47:24.843746+0000","last_active":"2024-09-19T06:47:40.830676+0000","last_peered":"2024-09-19T06:47:40.830676+0000","last_clean":"2024-09-19T06:47:40.830676+0000","last_became_active":"2024-09-19T06:47:24.841495+0000","last_became_peered":"2024-09-19T06:47:24.841495+0000","last_unstale":"2024-09-19T06:47:40.830676+0000","last_undegraded":"2024-09-19T06:47:40.830676+0000","last_fullsized":"2024-09-19T06:47:40.830676+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:06:29.995595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.173085+0000","last_change":"2024-09-19T06:47:24.844747+0000","last_active":"2024-09-19T06:47:41.173085+0000","last_peered":"2024-09-19T06:47:41.173085+0000","last_clean":"2024-09-19T06:47:41.173085+0000","last_became_active":"2024-09-19T06:47:24.844425+0000","last_became_peered":"2024-09-19T06:47:24.844425+0000","last_unstale":"2024-09-19T06:47:41.173085+0000","last_undegraded":"2024-09-19T06:47:41.173085+0000","last_fullsized":"2024-09-19T06:47:41.173085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:10.309044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834053+0000","last_change":"2024-09-19T06:47:24.848227+0000","last_active":"2024-09-19T06:47:40.834053+0000","last_peered":"2024-09-19T06:47:40.834053+0000","last_clean":"2024-09-19T06:47:40.834053+0000","last_became_active":"2024-09-19T06:47:24.847914+0000","last_became_peered":"2024-09-19T06:47:24.847914+0000","last_unstale":"2024-09-19T06:47:40.834053+0000","last_undegraded":"2024-09-19T06:47:40.834053+0000","last_fullsized":"2024-09-19T06:47:40.834053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:47.281613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834014+0000","last_change":"2024-09-19T06:47:24.844275+0000","last_active":"2024-09-19T06:47:40.834014+0000","last_peered":"2024-09-19T06:47:40.834014+0000","last_clean":"2024-09-19T06:47:40.834014+0000","last_became_active":"2024-09-19T06:47:24.841442+0000","last_became_peered":"2024-09-19T06:47:24.841442+0000","last_unstale":"2024-09-19T06:47:40.834014+0000","last_undegraded":"2024-09-19T06:47:40.834014+0000","last_fullsized":"2024-09-19T06:47:40.834014+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:30.198655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834062+0000","last_change":"2024-09-19T06:47:24.846301+0000","last_active":"2024-09-19T06:47:40.834062+0000","last_peered":"2024-09-19T06:47:40.834062+0000","last_clean":"2024-09-19T06:47:40.834062+0000","last_became_active":"2024-09-19T06:47:24.845970+0000","last_became_peered":"2024-09-19T06:47:24.845970+0000","last_unstale":"2024-09-19T06:47:40.834062+0000","last_undegraded":"2024-09-19T06:47:40.834062+0000","last_fullsized":"2024-09-19T06:47:40.834062+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:13.245536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830554+0000","last_change":"2024-09-19T06:47:24.844444+0000","last_active":"2024-09-19T06:47:40.830554+0000","last_peered":"2024-09-19T06:47:40.830554+0000","last_clean":"2024-09-19T06:47:40.830554+0000","last_became_active":"2024-09-19T06:47:24.844141+0000","last_became_peered":"2024-09-19T06:47:24.844141+0000","last_unstale":"2024-09-19T06:47:40.830554+0000","last_undegraded":"2024-09-19T06:47:40.830554+0000","last_fullsized":"2024-09-19T06:47:40.830554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:44.739242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831557+0000","last_change":"2024-09-19T06:47:24.845633+0000","last_active":"2024-09-19T06:47:40.831557+0000","last_peered":"2024-09-19T06:47:40.831557+0000","last_clean":"2024-09-19T06:47:40.831557+0000","last_became_active":"2024-09-19T06:47:24.845319+0000","last_became_peered":"2024-09-19T06:47:24.845319+0000","last_unstale":"2024-09-19T06:47:40.831557+0000","last_undegraded":"2024-09-19T06:47:40.831557+0000","last_fullsized":"2024-09-19T06:47:40.831557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:00.820006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830650+0000","last_change":"2024-09-19T06:47:24.846457+0000","last_active":"2024-09-19T06:47:40.830650+0000","last_peered":"2024-09-19T06:47:40.830650+0000","last_clean":"2024-09-19T06:47:40.830650+0000","last_became_active":"2024-09-19T06:47:24.846130+0000","last_became_peered":"2024-09-19T06:47:24.846130+0000","last_unstale":"2024-09-19T06:47:40.830650+0000","last_undegraded":"2024-09-19T06:47:40.830650+0000","last_fullsized":"2024-09-19T06:47:40.830650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:42.179145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839379+0000","last_change":"2024-09-19T06:47:24.843141+0000","last_active":"2024-09-19T06:47:40.839379+0000","last_peered":"2024-09-19T06:47:40.839379+0000","last_clean":"2024-09-19T06:47:40.839379+0000","last_became_active":"2024-09-19T06:47:24.842810+0000","last_became_peered":"2024-09-19T06:47:24.842810+0000","last_unstale":"2024-09-19T06:47:40.839379+0000","last_undegraded":"2024-09-19T06:47:40.839379+0000","last_fullsized":"2024-09-19T06:47:40.839379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:43.440453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829981+0000","last_change":"2024-09-19T06:47:24.846171+0000","last_active":"2024-09-19T06:47:40.829981+0000","last_peered":"2024-09-19T06:47:40.829981+0000","last_clean":"2024-09-19T06:47:40.829981+0000","last_became_active":"2024-09-19T06:47:24.845799+0000","last_became_peered":"2024-09-19T06:47:24.845799+0000","last_unstale":"2024-09-19T06:47:40.829981+0000","last_undegraded":"2024-09-19T06:47:40.829981+0000","last_fullsized":"2024-09-19T06:47:40.829981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:23.103061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830061+0000","last_change":"2024-09-19T06:47:24.848723+0000","last_active":"2024-09-19T06:47:40.830061+0000","last_peered":"2024-09-19T06:47:40.830061+0000","last_clean":"2024-09-19T06:47:40.830061+0000","last_became_active":"2024-09-19T06:47:24.848367+0000","last_became_peered":"2024-09-19T06:47:24.848367+0000","last_unstale":"2024-09-19T06:47:40.830061+0000","last_undegraded":"2024-09-19T06:47:40.830061+0000","last_fullsized":"2024-09-19T06:47:40.830061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:15.985293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831354+0000","last_change":"2024-09-19T06:47:24.841379+0000","last_active":"2024-09-19T06:47:40.831354+0000","last_peered":"2024-09-19T06:47:40.831354+0000","last_clean":"2024-09-19T06:47:40.831354+0000","last_became_active":"2024-09-19T06:47:24.841041+0000","last_became_peered":"2024-09-19T06:47:24.841041+0000","last_unstale":"2024-09-19T06:47:40.831354+0000","last_undegraded":"2024-09-19T06:47:40.831354+0000","last_fullsized":"2024-09-19T06:47:40.831354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:11.799022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.832906+0000","last_change":"2024-09-19T06:47:24.849508+0000","last_active":"2024-09-19T06:47:40.832906+0000","last_peered":"2024-09-19T06:47:40.832906+0000","last_clean":"2024-09-19T06:47:40.832906+0000","last_became_active":"2024-09-19T06:47:24.849196+0000","last_became_peered":"2024-09-19T06:47:24.849196+0000","last_unstale":"2024-09-19T06:47:40.832906+0000","last_undegraded":"2024-09-19T06:47:40.832906+0000","last_fullsized":"2024-09-19T06:47:40.832906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:35.079720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":78,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:48:23.534779+0000","last_change":"2024-09-19T06:47:22.829848+0000","last_active":"2024-09-19T06:48:23.534779+0000","last_peered":"2024-09-19T06:48:23.534779+0000","last_clean":"2024-09-19T06:48:23.534779+0000","last_became_active":"2024-09-19T06:47:22.829511+0000","last_became_peered":"2024-09-19T06:47:22.829511+0000","last_unstale":"2024-09-19T06:48:23.534779+0000","last_undegraded":"2024-09-19T06:48:23.534779+0000","last_fullsized":"2024-09-19T06:48:23.534779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:32.665252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834585+0000","last_change":"2024-09-19T06:47:24.843245+0000","last_active":"2024-09-19T06:47:40.834585+0000","last_peered":"2024-09-19T06:47:40.834585+0000","last_clean":"2024-09-19T06:47:40.834585+0000","last_became_active":"2024-09-19T06:47:24.842885+0000","last_became_peered":"2024-09-19T06:47:24.842885+0000","last_unstale":"2024-09-19T06:47:40.834585+0000","last_undegraded":"2024-09-19T06:47:40.834585+0000","last_fullsized":"2024-09-19T06:47:40.834585+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:26.229003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172292+0000","last_change":"2024-09-19T06:47:22.831456+0000","last_active":"2024-09-19T06:47:41.172292+0000","last_peered":"2024-09-19T06:47:41.172292+0000","last_clean":"2024-09-19T06:47:41.172292+0000","last_became_active":"2024-09-19T06:47:22.831132+0000","last_became_peered":"2024-09-19T06:47:22.831132+0000","last_unstale":"2024-09-19T06:47:41.172292+0000","last_undegraded":"2024-09-19T06:47:41.172292+0000","last_fullsized":"2024-09-19T06:47:41.172292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:49.867713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830418+0000","last_change":"2024-09-19T06:47:22.827742+0000","last_active":"2024-09-19T06:47:40.830418+0000","last_peered":"2024-09-19T06:47:40.830418+0000","last_clean":"2024-09-19T06:47:40.830418+0000","last_became_active":"2024-09-19T06:47:22.827371+0000","last_became_peered":"2024-09-19T06:47:22.827371+0000","last_unstale":"2024-09-19T06:47:40.830418+0000","last_undegraded":"2024-09-19T06:47:40.830418+0000","last_fullsized":"2024-09-19T06:47:40.830418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:35.153129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833435+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:47:40.833435+0000","last_peered":"2024-09-19T06:47:40.833435+0000","last_clean":"2024-09-19T06:47:40.833435+0000","last_became_active":"2024-09-19T06:47:24.841693+0000","last_became_peered":"2024-09-19T06:47:24.841693+0000","last_unstale":"2024-09-19T06:47:40.833435+0000","last_undegraded":"2024-09-19T06:47:40.833435+0000","last_fullsized":"2024-09-19T06:47:40.833435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:27.828296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834254+0000","last_change":"2024-09-19T06:47:22.832531+0000","last_active":"2024-09-19T06:47:40.834254+0000","last_peered":"2024-09-19T06:47:40.834254+0000","last_clean":"2024-09-19T06:47:40.834254+0000","last_became_active":"2024-09-19T06:47:22.832213+0000","last_became_peered":"2024-09-19T06:47:22.832213+0000","last_unstale":"2024-09-19T06:47:40.834254+0000","last_undegraded":"2024-09-19T06:47:40.834254+0000","last_fullsized":"2024-09-19T06:47:40.834254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:46:03.168406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833353+0000","last_change":"2024-09-19T06:47:24.837690+0000","last_active":"2024-09-19T06:47:40.833353+0000","last_peered":"2024-09-19T06:47:40.833353+0000","last_clean":"2024-09-19T06:47:40.833353+0000","last_became_active":"2024-09-19T06:47:24.837389+0000","last_became_peered":"2024-09-19T06:47:24.837389+0000","last_unstale":"2024-09-19T06:47:40.833353+0000","last_undegraded":"2024-09-19T06:47:40.833353+0000","last_fullsized":"2024-09-19T06:47:40.833353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:20.653832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":124,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:48:24.535500+0000","last_change":"2024-09-19T06:47:22.833808+0000","last_active":"2024-09-19T06:48:24.535500+0000","last_peered":"2024-09-19T06:48:24.535500+0000","last_clean":"2024-09-19T06:48:24.535500+0000","last_became_active":"2024-09-19T06:47:22.832468+0000","last_became_peered":"2024-09-19T06:47:22.832468+0000","last_unstale":"2024-09-19T06:48:24.535500+0000","last_undegraded":"2024-09-19T06:48:24.535500+0000","last_fullsized":"2024-09-19T06:48:24.535500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:39:31.131321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172236+0000","last_change":"2024-09-19T06:47:24.840928+0000","last_active":"2024-09-19T06:47:41.172236+0000","last_peered":"2024-09-19T06:47:41.172236+0000","last_clean":"2024-09-19T06:47:41.172236+0000","last_became_active":"2024-09-19T06:47:24.839928+0000","last_became_peered":"2024-09-19T06:47:24.839928+0000","last_unstale":"2024-09-19T06:47:41.172236+0000","last_undegraded":"2024-09-19T06:47:41.172236+0000","last_fullsized":"2024-09-19T06:47:41.172236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:29.486003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830932+0000","last_change":"2024-09-19T06:47:22.833310+0000","last_active":"2024-09-19T06:47:40.830932+0000","last_peered":"2024-09-19T06:47:40.830932+0000","last_clean":"2024-09-19T06:47:40.830932+0000","last_became_active":"2024-09-19T06:47:22.832976+0000","last_became_peered":"2024-09-19T06:47:22.832976+0000","last_unstale":"2024-09-19T06:47:40.830932+0000","last_undegraded":"2024-09-19T06:47:40.830932+0000","last_fullsized":"2024-09-19T06:47:40.830932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:47.604853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.840138+0000","last_change":"2024-09-19T06:47:24.848687+0000","last_active":"2024-09-19T06:47:40.840138+0000","last_peered":"2024-09-19T06:47:40.840138+0000","last_clean":"2024-09-19T06:47:40.840138+0000","last_became_active":"2024-09-19T06:47:24.848352+0000","last_became_peered":"2024-09-19T06:47:24.848352+0000","last_unstale":"2024-09-19T06:47:40.840138+0000","last_undegraded":"2024-09-19T06:47:40.840138+0000","last_fullsized":"2024-09-19T06:47:40.840138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:54.459206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.835734+0000","last_change":"2024-09-19T06:47:22.851479+0000","last_active":"2024-09-19T06:47:40.835734+0000","last_peered":"2024-09-19T06:47:40.835734+0000","last_clean":"2024-09-19T06:47:40.835734+0000","last_became_active":"2024-09-19T06:47:22.851125+0000","last_became_peered":"2024-09-19T06:47:22.851125+0000","last_unstale":"2024-09-19T06:47:40.835734+0000","last_undegraded":"2024-09-19T06:47:40.835734+0000","last_fullsized":"2024-09-19T06:47:40.835734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:42.954034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172823+0000","last_change":"2024-09-19T06:47:24.851007+0000","last_active":"2024-09-19T06:47:41.172823+0000","last_peered":"2024-09-19T06:47:41.172823+0000","last_clean":"2024-09-19T06:47:41.172823+0000","last_became_active":"2024-09-19T06:47:24.850465+0000","last_became_peered":"2024-09-19T06:47:24.850465+0000","last_unstale":"2024-09-19T06:47:41.172823+0000","last_undegraded":"2024-09-19T06:47:41.172823+0000","last_fullsized":"2024-09-19T06:47:41.172823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:10.824672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'30","reported_seq":159,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:48:24.535560+0000","last_change":"2024-09-19T06:47:22.834893+0000","last_active":"2024-09-19T06:48:24.535560+0000","last_peered":"2024-09-19T06:48:24.535560+0000","last_clean":"2024-09-19T06:48:24.535560+0000","last_became_active":"2024-09-19T06:47:22.834521+0000","last_became_peered":"2024-09-19T06:47:22.834521+0000","last_unstale":"2024-09-19T06:48:24.535560+0000","last_undegraded":"2024-09-19T06:48:24.535560+0000","last_fullsized":"2024-09-19T06:48:24.535560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:29.742284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60690,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":112,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833599+0000","last_change":"2024-09-19T06:47:24.848191+0000","last_active":"2024-09-19T06:47:40.833599+0000","last_peered":"2024-09-19T06:47:40.833599+0000","last_clean":"2024-09-19T06:47:40.833599+0000","last_became_active":"2024-09-19T06:47:24.847889+0000","last_became_peered":"2024-09-19T06:47:24.847889+0000","last_unstale":"2024-09-19T06:47:40.833599+0000","last_undegraded":"2024-09-19T06:47:40.833599+0000","last_fullsized":"2024-09-19T06:47:40.833599+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:32.574360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830250+0000","last_change":"2024-09-19T06:47:22.832005+0000","last_active":"2024-09-19T06:47:40.830250+0000","last_peered":"2024-09-19T06:47:40.830250+0000","last_clean":"2024-09-19T06:47:40.830250+0000","last_became_active":"2024-09-19T06:47:22.831342+0000","last_became_peered":"2024-09-19T06:47:22.831342+0000","last_unstale":"2024-09-19T06:47:40.830250+0000","last_undegraded":"2024-09-19T06:47:40.830250+0000","last_fullsized":"2024-09-19T06:47:40.830250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:53.444748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830453+0000","last_change":"2024-09-19T06:47:24.843328+0000","last_active":"2024-09-19T06:47:40.830453+0000","last_peered":"2024-09-19T06:47:40.830453+0000","last_clean":"2024-09-19T06:47:40.830453+0000","last_became_active":"2024-09-19T06:47:24.842344+0000","last_became_peered":"2024-09-19T06:47:24.842344+0000","last_unstale":"2024-09-19T06:47:40.830453+0000","last_undegraded":"2024-09-19T06:47:40.830453+0000","last_fullsized":"2024-09-19T06:47:40.830453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:11.247819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.844801+0000","last_change":"2024-09-19T06:47:40.844086+0000","last_active":"2024-09-19T06:47:40.844801+0000","last_peered":"2024-09-19T06:47:40.844801+0000","last_clean":"2024-09-19T06:47:40.844801+0000","last_became_active":"2024-09-19T06:47:40.835015+0000","last_became_peered":"2024-09-19T06:47:40.835015+0000","last_unstale":"2024-09-19T06:47:40.844801+0000","last_undegraded":"2024-09-19T06:47:40.844801+0000","last_fullsized":"2024-09-19T06:47:40.844801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:09.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.836466+0000","last_change":"2024-09-19T06:47:40.836466+0000","last_active":"2024-09-19T06:47:40.836466+0000","last_peered":"2024-09-19T06:47:40.836466+0000","last_clean":"2024-09-19T06:47:40.836466+0000","last_became_active":"2024-09-19T06:47:40.836237+0000","last_became_peered":"2024-09-19T06:47:40.836237+0000","last_unstale":"2024-09-19T06:47:40.836466+0000","last_undegraded":"2024-09-19T06:47:40.836466+0000","last_fullsized":"2024-09-19T06:47:40.836466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:00.094685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":82,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:48:25.537416+0000","last_change":"2024-09-19T06:47:22.830359+0000","last_active":"2024-09-19T06:48:25.537416+0000","last_peered":"2024-09-19T06:48:25.537416+0000","last_clean":"2024-09-19T06:48:25.537416+0000","last_became_active":"2024-09-19T06:47:22.829960+0000","last_became_peered":"2024-09-19T06:47:22.829960+0000","last_unstale":"2024-09-19T06:48:25.537416+0000","last_undegraded":"2024-09-19T06:48:25.537416+0000","last_fullsized":"2024-09-19T06:48:25.537416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:55.998243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830424+0000","last_change":"2024-09-19T06:47:24.836668+0000","last_active":"2024-09-19T06:47:40.830424+0000","last_peered":"2024-09-19T06:47:40.830424+0000","last_clean":"2024-09-19T06:47:40.830424+0000","last_became_active":"2024-09-19T06:47:24.836393+0000","last_became_peered":"2024-09-19T06:47:24.836393+0000","last_unstale":"2024-09-19T06:47:40.830424+0000","last_undegraded":"2024-09-19T06:47:40.830424+0000","last_fullsized":"2024-09-19T06:47:40.830424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:26.248948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.858323+0000","last_change":"2024-09-19T06:47:40.858323+0000","last_active":"2024-09-19T06:47:40.858323+0000","last_peered":"2024-09-19T06:47:40.858323+0000","last_clean":"2024-09-19T06:47:40.858323+0000","last_became_active":"2024-09-19T06:47:40.840802+0000","last_became_peered":"2024-09-19T06:47:40.840802+0000","last_unstale":"2024-09-19T06:47:40.858323+0000","last_undegraded":"2024-09-19T06:47:40.858323+0000","last_fullsized":"2024-09-19T06:47:40.858323+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:14.666916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834268+0000","last_change":"2024-09-19T06:47:24.843681+0000","last_active":"2024-09-19T06:47:40.834268+0000","last_peered":"2024-09-19T06:47:40.834268+0000","last_clean":"2024-09-19T06:47:40.834268+0000","last_became_active":"2024-09-19T06:47:24.841228+0000","last_became_peered":"2024-09-19T06:47:24.841228+0000","last_unstale":"2024-09-19T06:47:40.834268+0000","last_undegraded":"2024-09-19T06:47:40.834268+0000","last_fullsized":"2024-09-19T06:47:40.834268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:56.418962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833647+0000","last_change":"2024-09-19T06:47:22.833893+0000","last_active":"2024-09-19T06:47:40.833647+0000","last_peered":"2024-09-19T06:47:40.833647+0000","last_clean":"2024-09-19T06:47:40.833647+0000","last_became_active":"2024-09-19T06:47:22.833217+0000","last_became_peered":"2024-09-19T06:47:22.833217+0000","last_unstale":"2024-09-19T06:47:40.833647+0000","last_undegraded":"2024-09-19T06:47:40.833647+0000","last_fullsized":"2024-09-19T06:47:40.833647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:19.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831588+0000","last_change":"2024-09-19T06:47:24.848399+0000","last_active":"2024-09-19T06:47:40.831588+0000","last_peered":"2024-09-19T06:47:40.831588+0000","last_clean":"2024-09-19T06:47:40.831588+0000","last_became_active":"2024-09-19T06:47:24.848081+0000","last_became_peered":"2024-09-19T06:47:24.848081+0000","last_unstale":"2024-09-19T06:47:40.831588+0000","last_undegraded":"2024-09-19T06:47:40.831588+0000","last_fullsized":"2024-09-19T06:47:40.831588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:44.292939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.860955+0000","last_change":"2024-09-19T06:47:40.860955+0000","last_active":"2024-09-19T06:47:40.860955+0000","last_peered":"2024-09-19T06:47:40.860955+0000","last_clean":"2024-09-19T06:47:40.860955+0000","last_became_active":"2024-09-19T06:47:40.835126+0000","last_became_peered":"2024-09-19T06:47:40.835126+0000","last_unstale":"2024-09-19T06:47:40.860955+0000","last_undegraded":"2024-09-19T06:47:40.860955+0000","last_fullsized":"2024-09-19T06:47:40.860955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:02:26.763229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830803+0000","last_change":"2024-09-19T06:47:24.849680+0000","last_active":"2024-09-19T06:47:40.830803+0000","last_peered":"2024-09-19T06:47:40.830803+0000","last_clean":"2024-09-19T06:47:40.830803+0000","last_became_active":"2024-09-19T06:47:24.849323+0000","last_became_peered":"2024-09-19T06:47:24.849323+0000","last_unstale":"2024-09-19T06:47:40.830803+0000","last_undegraded":"2024-09-19T06:47:40.830803+0000","last_fullsized":"2024-09-19T06:47:40.830803+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:59.763503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.835875+0000","last_change":"2024-09-19T06:47:22.831689+0000","last_active":"2024-09-19T06:47:40.835875+0000","last_peered":"2024-09-19T06:47:40.835875+0000","last_clean":"2024-09-19T06:47:40.835875+0000","last_became_active":"2024-09-19T06:47:22.829283+0000","last_became_peered":"2024-09-19T06:47:22.829283+0000","last_unstale":"2024-09-19T06:47:40.835875+0000","last_undegraded":"2024-09-19T06:47:40.835875+0000","last_fullsized":"2024-09-19T06:47:40.835875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:37.109542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830659+0000","last_change":"2024-09-19T06:47:24.842793+0000","last_active":"2024-09-19T06:47:40.830659+0000","last_peered":"2024-09-19T06:47:40.830659+0000","last_clean":"2024-09-19T06:47:40.830659+0000","last_became_active":"2024-09-19T06:47:24.842474+0000","last_became_peered":"2024-09-19T06:47:24.842474+0000","last_unstale":"2024-09-19T06:47:40.830659+0000","last_undegraded":"2024-09-19T06:47:40.830659+0000","last_fullsized":"2024-09-19T06:47:40.830659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:23.110736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'6","reported_seq":82,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:48:25.537433+0000","last_change":"2024-09-19T06:47:23.047356+0000","last_active":"2024-09-19T06:48:25.537433+0000","last_peered":"2024-09-19T06:48:25.537433+0000","last_clean":"2024-09-19T06:48:25.537433+0000","last_became_active":"2024-09-19T06:47:23.046941+0000","last_became_peered":"2024-09-19T06:47:23.046941+0000","last_unstale":"2024-09-19T06:48:25.537433+0000","last_undegraded":"2024-09-19T06:48:25.537433+0000","last_fullsized":"2024-09-19T06:48:25.537433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:43.490952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830805+0000","last_change":"2024-09-19T06:47:24.851025+0000","last_active":"2024-09-19T06:47:40.830805+0000","last_peered":"2024-09-19T06:47:40.830805+0000","last_clean":"2024-09-19T06:47:40.830805+0000","last_became_active":"2024-09-19T06:47:24.850681+0000","last_became_peered":"2024-09-19T06:47:24.850681+0000","last_unstale":"2024-09-19T06:47:40.830805+0000","last_undegraded":"2024-09-19T06:47:40.830805+0000","last_fullsized":"2024-09-19T06:47:40.830805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:46.884747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":345,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830558+0000","last_change":"2024-09-19T06:46:13.927379+0000","last_active":"2024-09-19T06:47:40.830558+0000","last_peered":"2024-09-19T06:47:40.830558+0000","last_clean":"2024-09-19T06:47:40.830558+0000","last_became_active":"2024-09-19T06:46:13.926926+0000","last_became_peered":"2024-09-19T06:46:13.926926+0000","last_unstale":"2024-09-19T06:47:40.830558+0000","last_undegraded":"2024-09-19T06:47:40.830558+0000","last_fullsized":"2024-09-19T06:47:40.830558+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_clean_scrub_stamp":"2024-09-19T06:42:52.051335+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:29.542128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829278+0000","last_change":"2024-09-19T06:47:22.830387+0000","last_active":"2024-09-19T06:47:40.829278+0000","last_peered":"2024-09-19T06:47:40.829278+0000","last_clean":"2024-09-19T06:47:40.829278+0000","last_became_active":"2024-09-19T06:47:22.829970+0000","last_became_peered":"2024-09-19T06:47:22.829970+0000","last_unstale":"2024-09-19T06:47:40.829278+0000","last_undegraded":"2024-09-19T06:47:40.829278+0000","last_fullsized":"2024-09-19T06:47:40.829278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:51.385456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834053+0000","last_change":"2024-09-19T06:47:24.843752+0000","last_active":"2024-09-19T06:47:40.834053+0000","last_peered":"2024-09-19T06:47:40.834053+0000","last_clean":"2024-09-19T06:47:40.834053+0000","last_became_active":"2024-09-19T06:47:24.842631+0000","last_became_peered":"2024-09-19T06:47:24.842631+0000","last_unstale":"2024-09-19T06:47:40.834053+0000","last_undegraded":"2024-09-19T06:47:40.834053+0000","last_fullsized":"2024-09-19T06:47:40.834053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:19.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830650+0000","last_change":"2024-09-19T06:47:22.832647+0000","last_active":"2024-09-19T06:47:40.830650+0000","last_peered":"2024-09-19T06:47:40.830650+0000","last_clean":"2024-09-19T06:47:40.830650+0000","last_became_active":"2024-09-19T06:47:22.831774+0000","last_became_peered":"2024-09-19T06:47:22.831774+0000","last_unstale":"2024-09-19T06:47:40.830650+0000","last_undegraded":"2024-09-19T06:47:40.830650+0000","last_fullsized":"2024-09-19T06:47:40.830650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:57.824610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830409+0000","last_change":"2024-09-19T06:47:24.844301+0000","last_active":"2024-09-19T06:47:40.830409+0000","last_peered":"2024-09-19T06:47:40.830409+0000","last_clean":"2024-09-19T06:47:40.830409+0000","last_became_active":"2024-09-19T06:47:24.839748+0000","last_became_peered":"2024-09-19T06:47:24.839748+0000","last_unstale":"2024-09-19T06:47:40.830409+0000","last_undegraded":"2024-09-19T06:47:40.830409+0000","last_fullsized":"2024-09-19T06:47:40.830409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:46.407693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":78,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:48:24.535510+0000","last_change":"2024-09-19T06:47:22.851008+0000","last_active":"2024-09-19T06:48:24.535510+0000","last_peered":"2024-09-19T06:48:24.535510+0000","last_clean":"2024-09-19T06:48:24.535510+0000","last_became_active":"2024-09-19T06:47:22.850601+0000","last_became_peered":"2024-09-19T06:47:22.850601+0000","last_unstale":"2024-09-19T06:48:24.535510+0000","last_undegraded":"2024-09-19T06:48:24.535510+0000","last_fullsized":"2024-09-19T06:48:24.535510+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:04.610277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834177+0000","last_change":"2024-09-19T06:47:24.839684+0000","last_active":"2024-09-19T06:47:40.834177+0000","last_peered":"2024-09-19T06:47:40.834177+0000","last_clean":"2024-09-19T06:47:40.834177+0000","last_became_active":"2024-09-19T06:47:24.839126+0000","last_became_peered":"2024-09-19T06:47:24.839126+0000","last_unstale":"2024-09-19T06:47:40.834177+0000","last_undegraded":"2024-09-19T06:47:40.834177+0000","last_fullsized":"2024-09-19T06:47:40.834177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:09.458535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":77,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:48:24.247031+0000","last_change":"2024-09-19T06:47:22.831819+0000","last_active":"2024-09-19T06:48:24.247031+0000","last_peered":"2024-09-19T06:48:24.247031+0000","last_clean":"2024-09-19T06:48:24.247031+0000","last_became_active":"2024-09-19T06:47:22.831332+0000","last_became_peered":"2024-09-19T06:47:22.831332+0000","last_unstale":"2024-09-19T06:48:24.247031+0000","last_undegraded":"2024-09-19T06:48:24.247031+0000","last_fullsized":"2024-09-19T06:48:24.247031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:35.032949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.171750+0000","last_change":"2024-09-19T06:47:24.846738+0000","last_active":"2024-09-19T06:47:41.171750+0000","last_peered":"2024-09-19T06:47:41.171750+0000","last_clean":"2024-09-19T06:47:41.171750+0000","last_became_active":"2024-09-19T06:47:24.846389+0000","last_became_peered":"2024-09-19T06:47:24.846389+0000","last_unstale":"2024-09-19T06:47:41.171750+0000","last_undegraded":"2024-09-19T06:47:41.171750+0000","last_fullsized":"2024-09-19T06:47:41.171750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:19.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833696+0000","last_change":"2024-09-19T06:47:22.851065+0000","last_active":"2024-09-19T06:47:40.833696+0000","last_peered":"2024-09-19T06:47:40.833696+0000","last_clean":"2024-09-19T06:47:40.833696+0000","last_became_active":"2024-09-19T06:47:22.850647+0000","last_became_peered":"2024-09-19T06:47:22.850647+0000","last_unstale":"2024-09-19T06:47:40.833696+0000","last_undegraded":"2024-09-19T06:47:40.833696+0000","last_fullsized":"2024-09-19T06:47:40.833696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:41.993501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172204+0000","last_change":"2024-09-19T06:47:24.838905+0000","last_active":"2024-09-19T06:47:41.172204+0000","last_peered":"2024-09-19T06:47:41.172204+0000","last_clean":"2024-09-19T06:47:41.172204+0000","last_became_active":"2024-09-19T06:47:24.838443+0000","last_became_peered":"2024-09-19T06:47:24.838443+0000","last_unstale":"2024-09-19T06:47:41.172204+0000","last_undegraded":"2024-09-19T06:47:41.172204+0000","last_fullsized":"2024-09-19T06:47:41.172204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:12.486137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833766+0000","last_change":"2024-09-19T06:47:22.851431+0000","last_active":"2024-09-19T06:47:40.833766+0000","last_peered":"2024-09-19T06:47:40.833766+0000","last_clean":"2024-09-19T06:47:40.833766+0000","last_became_active":"2024-09-19T06:47:22.851092+0000","last_became_peered":"2024-09-19T06:47:22.851092+0000","last_unstale":"2024-09-19T06:47:40.833766+0000","last_undegraded":"2024-09-19T06:47:40.833766+0000","last_fullsized":"2024-09-19T06:47:40.833766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:26.183803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830789+0000","last_change":"2024-09-19T06:47:24.839046+0000","last_active":"2024-09-19T06:47:40.830789+0000","last_peered":"2024-09-19T06:47:40.830789+0000","last_clean":"2024-09-19T06:47:40.830789+0000","last_became_active":"2024-09-19T06:47:24.838680+0000","last_became_peered":"2024-09-19T06:47:24.838680+0000","last_unstale":"2024-09-19T06:47:40.830789+0000","last_undegraded":"2024-09-19T06:47:40.830789+0000","last_fullsized":"2024-09-19T06:47:40.830789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:45.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'15","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:49.846429+0000","last_change":"2024-09-19T06:47:40.865241+0000","last_active":"2024-09-19T06:47:49.846429+0000","last_peered":"2024-09-19T06:47:49.846429+0000","last_clean":"2024-09-19T06:47:49.846429+0000","last_became_active":"2024-09-19T06:47:40.837765+0000","last_became_peered":"2024-09-19T06:47:40.837765+0000","last_unstale":"2024-09-19T06:47:49.846429+0000","last_undegraded":"2024-09-19T06:47:49.846429+0000","last_fullsized":"2024-09-19T06:47:49.846429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834162+0000","last_change":"2024-09-19T06:47:24.848599+0000","last_active":"2024-09-19T06:47:40.834162+0000","last_peered":"2024-09-19T06:47:40.834162+0000","last_clean":"2024-09-19T06:47:40.834162+0000","last_became_active":"2024-09-19T06:47:24.848305+0000","last_became_peered":"2024-09-19T06:47:24.848305+0000","last_unstale":"2024-09-19T06:47:40.834162+0000","last_undegraded":"2024-09-19T06:47:40.834162+0000","last_fullsized":"2024-09-19T06:47:40.834162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:31.496188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.829462+0000","last_change":"2024-09-19T06:47:23.047484+0000","last_active":"2024-09-19T06:47:40.829462+0000","last_peered":"2024-09-19T06:47:40.829462+0000","last_clean":"2024-09-19T06:47:40.829462+0000","last_became_active":"2024-09-19T06:47:23.047058+0000","last_became_peered":"2024-09-19T06:47:23.047058+0000","last_unstale":"2024-09-19T06:47:40.829462+0000","last_undegraded":"2024-09-19T06:47:40.829462+0000","last_fullsized":"2024-09-19T06:47:40.829462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:34.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839396+0000","last_change":"2024-09-19T06:47:24.843498+0000","last_active":"2024-09-19T06:47:40.839396+0000","last_peered":"2024-09-19T06:47:40.839396+0000","last_clean":"2024-09-19T06:47:40.839396+0000","last_became_active":"2024-09-19T06:47:24.843204+0000","last_became_peered":"2024-09-19T06:47:24.843204+0000","last_unstale":"2024-09-19T06:47:40.839396+0000","last_undegraded":"2024-09-19T06:47:40.839396+0000","last_fullsized":"2024-09-19T06:47:40.839396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:25.585023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172992+0000","last_change":"2024-09-19T06:47:22.851519+0000","last_active":"2024-09-19T06:47:41.172992+0000","last_peered":"2024-09-19T06:47:41.172992+0000","last_clean":"2024-09-19T06:47:41.172992+0000","last_became_active":"2024-09-19T06:47:22.851163+0000","last_became_peered":"2024-09-19T06:47:22.851163+0000","last_unstale":"2024-09-19T06:47:41.172992+0000","last_undegraded":"2024-09-19T06:47:41.172992+0000","last_fullsized":"2024-09-19T06:47:41.172992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:04.502795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.832719+0000","last_change":"2024-09-19T06:47:24.845239+0000","last_active":"2024-09-19T06:47:40.832719+0000","last_peered":"2024-09-19T06:47:40.832719+0000","last_clean":"2024-09-19T06:47:40.832719+0000","last_became_active":"2024-09-19T06:47:24.844934+0000","last_became_peered":"2024-09-19T06:47:24.844934+0000","last_unstale":"2024-09-19T06:47:40.832719+0000","last_undegraded":"2024-09-19T06:47:40.832719+0000","last_fullsized":"2024-09-19T06:47:40.832719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:43.610604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834147+0000","last_change":"2024-09-19T06:47:22.830232+0000","last_active":"2024-09-19T06:47:40.834147+0000","last_peered":"2024-09-19T06:47:40.834147+0000","last_clean":"2024-09-19T06:47:40.834147+0000","last_became_active":"2024-09-19T06:47:22.829901+0000","last_became_peered":"2024-09-19T06:47:22.829901+0000","last_unstale":"2024-09-19T06:47:40.834147+0000","last_undegraded":"2024-09-19T06:47:40.834147+0000","last_fullsized":"2024-09-19T06:47:40.834147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:12.251427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.831384+0000","last_change":"2024-09-19T06:47:24.839304+0000","last_active":"2024-09-19T06:47:40.831384+0000","last_peered":"2024-09-19T06:47:40.831384+0000","last_clean":"2024-09-19T06:47:40.831384+0000","last_became_active":"2024-09-19T06:47:24.838986+0000","last_became_peered":"2024-09-19T06:47:24.838986+0000","last_unstale":"2024-09-19T06:47:40.831384+0000","last_undegraded":"2024-09-19T06:47:40.831384+0000","last_fullsized":"2024-09-19T06:47:40.831384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:39.377886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830021+0000","last_change":"2024-09-19T06:47:22.828553+0000","last_active":"2024-09-19T06:47:40.830021+0000","last_peered":"2024-09-19T06:47:40.830021+0000","last_clean":"2024-09-19T06:47:40.830021+0000","last_became_active":"2024-09-19T06:47:22.828213+0000","last_became_peered":"2024-09-19T06:47:22.828213+0000","last_unstale":"2024-09-19T06:47:40.830021+0000","last_undegraded":"2024-09-19T06:47:40.830021+0000","last_fullsized":"2024-09-19T06:47:40.830021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:31.649761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833576+0000","last_change":"2024-09-19T06:47:24.840747+0000","last_active":"2024-09-19T06:47:40.833576+0000","last_peered":"2024-09-19T06:47:40.833576+0000","last_clean":"2024-09-19T06:47:40.833576+0000","last_became_active":"2024-09-19T06:47:24.840451+0000","last_became_peered":"2024-09-19T06:47:24.840451+0000","last_unstale":"2024-09-19T06:47:40.833576+0000","last_undegraded":"2024-09-19T06:47:40.833576+0000","last_fullsized":"2024-09-19T06:47:40.833576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:41.588751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833228+0000","last_change":"2024-09-19T06:47:22.831697+0000","last_active":"2024-09-19T06:47:40.833228+0000","last_peered":"2024-09-19T06:47:40.833228+0000","last_clean":"2024-09-19T06:47:40.833228+0000","last_became_active":"2024-09-19T06:47:22.831178+0000","last_became_peered":"2024-09-19T06:47:22.831178+0000","last_unstale":"2024-09-19T06:47:40.833228+0000","last_undegraded":"2024-09-19T06:47:40.833228+0000","last_fullsized":"2024-09-19T06:47:40.833228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:30.975782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.830575+0000","last_change":"2024-09-19T06:47:24.845097+0000","last_active":"2024-09-19T06:47:40.830575+0000","last_peered":"2024-09-19T06:47:40.830575+0000","last_clean":"2024-09-19T06:47:40.830575+0000","last_became_active":"2024-09-19T06:47:24.844801+0000","last_became_peered":"2024-09-19T06:47:24.844801+0000","last_unstale":"2024-09-19T06:47:40.830575+0000","last_undegraded":"2024-09-19T06:47:40.830575+0000","last_fullsized":"2024-09-19T06:47:40.830575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:15.264118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.835620+0000","last_change":"2024-09-19T06:47:24.851372+0000","last_active":"2024-09-19T06:47:40.835620+0000","last_peered":"2024-09-19T06:47:40.835620+0000","last_clean":"2024-09-19T06:47:40.835620+0000","last_became_active":"2024-09-19T06:47:24.851044+0000","last_became_peered":"2024-09-19T06:47:24.851044+0000","last_unstale":"2024-09-19T06:47:40.835620+0000","last_undegraded":"2024-09-19T06:47:40.835620+0000","last_fullsized":"2024-09-19T06:47:40.835620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:55.658214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833103+0000","last_change":"2024-09-19T06:47:22.833694+0000","last_active":"2024-09-19T06:47:40.833103+0000","last_peered":"2024-09-19T06:47:40.833103+0000","last_clean":"2024-09-19T06:47:40.833103+0000","last_became_active":"2024-09-19T06:47:22.833215+0000","last_became_peered":"2024-09-19T06:47:22.833215+0000","last_unstale":"2024-09-19T06:47:40.833103+0000","last_undegraded":"2024-09-19T06:47:40.833103+0000","last_fullsized":"2024-09-19T06:47:40.833103+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:32.171522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:41.172973+0000","last_change":"2024-09-19T06:47:24.845737+0000","last_active":"2024-09-19T06:47:41.172973+0000","last_peered":"2024-09-19T06:47:41.172973+0000","last_clean":"2024-09-19T06:47:41.172973+0000","last_became_active":"2024-09-19T06:47:24.845242+0000","last_became_peered":"2024-09-19T06:47:24.845242+0000","last_unstale":"2024-09-19T06:47:41.172973+0000","last_undegraded":"2024-09-19T06:47:41.172973+0000","last_fullsized":"2024-09-19T06:47:41.172973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:51.554946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.833601+0000","last_change":"2024-09-19T06:47:22.829869+0000","last_active":"2024-09-19T06:47:40.833601+0000","last_peered":"2024-09-19T06:47:40.833601+0000","last_clean":"2024-09-19T06:47:40.833601+0000","last_became_active":"2024-09-19T06:47:22.829522+0000","last_became_peered":"2024-09-19T06:47:22.829522+0000","last_unstale":"2024-09-19T06:47:40.833601+0000","last_undegraded":"2024-09-19T06:47:40.833601+0000","last_fullsized":"2024-09-19T06:47:40.833601+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:26.871434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.840141+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:47:40.840141+0000","last_peered":"2024-09-19T06:47:40.840141+0000","last_clean":"2024-09-19T06:47:40.840141+0000","last_became_active":"2024-09-19T06:47:24.841289+0000","last_became_peered":"2024-09-19T06:47:24.841289+0000","last_unstale":"2024-09-19T06:47:40.840141+0000","last_undegraded":"2024-09-19T06:47:40.840141+0000","last_fullsized":"2024-09-19T06:47:40.840141+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:21:12.611458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.834465+0000","last_change":"2024-09-19T06:47:22.852198+0000","last_active":"2024-09-19T06:47:40.834465+0000","last_peered":"2024-09-19T06:47:40.834465+0000","last_clean":"2024-09-19T06:47:40.834465+0000","last_became_active":"2024-09-19T06:47:22.850984+0000","last_became_peered":"2024-09-19T06:47:22.850984+0000","last_unstale":"2024-09-19T06:47:40.834465+0000","last_undegraded":"2024-09-19T06:47:40.834465+0000","last_fullsized":"2024-09-19T06:47:40.834465+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:13.620145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:47:40.839399+0000","last_change":"2024-09-19T06:47:40.837614+0000","last_active":"2024-09-19T06:47:40.839399+0000","last_peered":"2024-09-19T06:47:40.839399+0000","last_clean":"2024-09-19T06:47:40.839399+0000","last_became_active":"2024-09-19T06:47:40.837259+0000","last_became_peered":"2024-09-19T06:47:40.837259+0000","last_unstale":"2024-09-19T06:47:40.839399+0000","last_undegraded":"2024-09-19T06:47:40.839399+0000","last_fullsized":"2024-09-19T06:47:40.839399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:41.429962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":105,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T06:48:24.246367+0000","last_change":"2024-09-19T06:47:41.182639+0000","last_active":"2024-09-19T06:48:24.246367+0000","last_peered":"2024-09-19T06:48:24.246367+0000","last_clean":"2024-09-19T06:48:24.246367+0000","last_became_active":"2024-09-19T06:47:41.174305+0000","last_became_peered":"2024-09-19T06:47:41.174305+0000","last_unstale":"2024-09-19T06:48:24.246367+0000","last_undegraded":"2024-09-19T06:48:24.246367+0000","last_fullsized":"2024-09-19T06:48:24.246367+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:10.199502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67898,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":533,"num_read_kb":338,"num_write":243,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":203694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579930,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":439122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808862,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1090751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972388,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":438569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234025,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":913408,"data_stored":438275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462957,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712484,"statfs":{"total":95995035648,"available":95961583616,"internally_reserved":0,"allocated":1519616,"data_stored":1031467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626483,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":909312,"data_stored":438286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855414,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1030662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018940,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":438652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215165,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":438111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444103,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":500390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":500561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":440018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:48:29.811 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T06:48:29.811 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T06:48:29.811 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T06:48:29.812 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T06:48:30.061 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:30.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.464+0000 7f1bfc1ba640 1 -- 172.21.15.103:0/4113937255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1bf4102570 msgr2=0x7f1bf41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:30.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.464+0000 7f1bfc1ba640 1 --2- 172.21.15.103:0/4113937255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1bf4102570 0x7f1bf41029b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009900 tx=0x7f1be802a480 comp rx=0 tx=0).stop 2024-09-19T06:48:30.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.464+0000 7f1bfc1ba640 1 -- 172.21.15.103:0/4113937255 shutdown_connections 2024-09-19T06:48:30.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.464+0000 7f1bfc1ba640 1 --2- 172.21.15.103:0/4113937255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bf4102ef0 0x7f1bf410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:30.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.464+0000 7f1bfc1ba640 1 --2- 172.21.15.103:0/4113937255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1bf4102570 0x7f1bf41029b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:30.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.464+0000 7f1bfc1ba640 1 --2- 172.21.15.103:0/4113937255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bf4100e20 0x7f1bf4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:30.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.464+0000 7f1bfc1ba640 1 -- 172.21.15.103:0/4113937255 >> 172.21.15.103:0/4113937255 conn(0x7f1bf40fc7d0 msgr2=0x7f1bf40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.465+0000 7f1bfc1ba640 1 -- 172.21.15.103:0/4113937255 shutdown_connections 2024-09-19T06:48:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.465+0000 7f1bfc1ba640 1 -- 172.21.15.103:0/4113937255 wait complete. 2024-09-19T06:48:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.465+0000 7f1bfc1ba640 1 Processor -- start 2024-09-19T06:48:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.465+0000 7f1bfc1ba640 1 -- start start 2024-09-19T06:48:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bfc1ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bf4100e20 0x7f1bf41984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bfc1ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1bf4102570 0x7f1bf4198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bfc1ba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bf4102ef0 0x7f1bf419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bfc1ba640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf4110f80 con 0x7f1bf4102570 2024-09-19T06:48:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bfc1ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf4110e00 con 0x7f1bf4100e20 2024-09-19T06:48:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bfc1ba640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf4111100 con 0x7f1bf4102ef0 2024-09-19T06:48:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bf9755640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1bf4102570 0x7f1bf4198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bf9755640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1bf4102570 0x7f1bf4198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46466/0 (socket says 172.21.15.103:46466) 2024-09-19T06:48:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bf9755640 1 -- 172.21.15.103:0/3229776936 learned_addr learned my addr 172.21.15.103:0/3229776936 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:30.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bfa757640 1 --2- 172.21.15.103:0/3229776936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bf4102ef0 0x7f1bf419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bf9f56640 1 --2- 172.21.15.103:0/3229776936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bf4100e20 0x7f1bf41984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bf9755640 1 -- 172.21.15.103:0/3229776936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bf4102ef0 msgr2=0x7f1bf419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bf9755640 1 --2- 172.21.15.103:0/3229776936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bf4102ef0 0x7f1bf419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bf9755640 1 -- 172.21.15.103:0/3229776936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bf4100e20 msgr2=0x7f1bf41984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bf9755640 1 --2- 172.21.15.103:0/3229776936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bf4100e20 0x7f1bf41984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bf9755640 1 -- 172.21.15.103:0/3229776936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf419d470 con 0x7f1bf4102570 2024-09-19T06:48:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bfa757640 1 --2- 172.21.15.103:0/3229776936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bf4102ef0 0x7f1bf419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bf9f56640 1 --2- 172.21.15.103:0/3229776936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bf4100e20 0x7f1bf41984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bf9755640 1 --2- 172.21.15.103:0/3229776936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1bf4102570 0x7f1bf4198a00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1be8006010 tx=0x7f1be8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bf2ffd640 1 -- 172.21.15.103:0/3229776936 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be803f070 con 0x7f1bf4102570 2024-09-19T06:48:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bf2ffd640 1 -- 172.21.15.103:0/3229776936 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1be8031c30 con 0x7f1bf4102570 2024-09-19T06:48:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bfc1ba640 1 -- 172.21.15.103:0/3229776936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf419d700 con 0x7f1bf4102570 2024-09-19T06:48:30.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bf2ffd640 1 -- 172.21.15.103:0/3229776936 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8035030 con 0x7f1bf4102570 2024-09-19T06:48:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.466+0000 7f1bfc1ba640 1 -- 172.21.15.103:0/3229776936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf40fcbb0 con 0x7f1bf4102570 2024-09-19T06:48:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.467+0000 7f1bfc1ba640 1 -- 172.21.15.103:0/3229776936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf41021b0 con 0x7f1bf4102570 2024-09-19T06:48:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.468+0000 7f1bf2ffd640 1 -- 172.21.15.103:0/3229776936 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1be8042070 con 0x7f1bf4102570 2024-09-19T06:48:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.469+0000 7f1bf2ffd640 1 --2- 172.21.15.103:0/3229776936 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f1bdc0a32d0 0x7f1bdc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.469+0000 7f1bf9f56640 1 --2- 172.21.15.103:0/3229776936 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f1bdc0a32d0 0x7f1bdc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.469+0000 7f1bf2ffd640 1 -- 172.21.15.103:0/3229776936 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1be80f03a0 con 0x7f1bf4102570 2024-09-19T06:48:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.470+0000 7f1bf9f56640 1 --2- 172.21.15.103:0/3229776936 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f1bdc0a32d0 0x7f1bdc0a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1be4005c40 tx=0x7f1be400a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.473+0000 7f1bf2ffd640 1 -- 172.21.15.103:0/3229776936 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1be8031de0 con 0x7f1bf4102570 2024-09-19T06:48:30.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.681+0000 7f1bfc1ba640 1 -- 172.21.15.103:0/3229776936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1bf40fce10 con 0x7f1bf4102570 2024-09-19T06:48:30.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.681+0000 7f1bf2ffd640 1 -- 172.21.15.103:0/3229776936 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f1be80bcf70 con 0x7f1bf4102570 2024-09-19T06:48:30.683 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:48:30.683 INFO:teuthology.orchestra.run.smithi103.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T06:48:30.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.683+0000 7f1bfc1ba640 1 -- 172.21.15.103:0/3229776936 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f1bdc0a32d0 msgr2=0x7f1bdc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:30.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.683+0000 7f1bfc1ba640 1 --2- 172.21.15.103:0/3229776936 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f1bdc0a32d0 0x7f1bdc0a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1be4005c40 tx=0x7f1be400a410 comp rx=0 tx=0).stop 2024-09-19T06:48:30.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.683+0000 7f1bfc1ba640 1 -- 172.21.15.103:0/3229776936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1bf4102570 msgr2=0x7f1bf4198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:30.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.683+0000 7f1bfc1ba640 1 --2- 172.21.15.103:0/3229776936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1bf4102570 0x7f1bf4198a00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1be8006010 tx=0x7f1be8003c10 comp rx=0 tx=0).stop 2024-09-19T06:48:30.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.683+0000 7f1bfc1ba640 1 -- 172.21.15.103:0/3229776936 shutdown_connections 2024-09-19T06:48:30.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.683+0000 7f1bfc1ba640 1 --2- 172.21.15.103:0/3229776936 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f1bdc0a32d0 0x7f1bdc0a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:30.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.683+0000 7f1bfc1ba640 1 --2- 172.21.15.103:0/3229776936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bf4102ef0 0x7f1bf419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:30.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.683+0000 7f1bfc1ba640 1 --2- 172.21.15.103:0/3229776936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1bf4102570 0x7f1bf4198a00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:30.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.683+0000 7f1bfc1ba640 1 --2- 172.21.15.103:0/3229776936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bf4100e20 0x7f1bf41984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:30.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.683+0000 7f1bfc1ba640 1 -- 172.21.15.103:0/3229776936 >> 172.21.15.103:0/3229776936 conn(0x7f1bf40fc7d0 msgr2=0x7f1bf4069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:30.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.683+0000 7f1bfc1ba640 1 -- 172.21.15.103:0/3229776936 shutdown_connections 2024-09-19T06:48:30.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:30.683+0000 7f1bfc1ba640 1 -- 172.21.15.103:0/3229776936 wait complete. 2024-09-19T06:48:31.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:30 smithi134 ceph-mon[35356]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:48:31.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:30 smithi134 ceph-mon[35356]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:48:31.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:30 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3229776936' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:48:31.188 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:30 smithi103 ceph-mon[31158]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:48:31.188 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:30 smithi103 ceph-mon[31158]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:48:31.188 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:30 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3229776936' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:48:31.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:30 smithi174 ceph-mon[35637]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:48:31.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:30 smithi174 ceph-mon[35637]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:48:31.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:30 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3229776936' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:48:31.369 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T06:48:31.370 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:48:31.618 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:32.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.081+0000 7fc88200c640 1 -- 172.21.15.103:0/978096846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc87c1036e0 msgr2=0x7fc87c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:32.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.081+0000 7fc88200c640 1 --2- 172.21.15.103:0/978096846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc87c1036e0 0x7fc87c107b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc864009930 tx=0x7fc86402a420 comp rx=0 tx=0).stop 2024-09-19T06:48:32.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.081+0000 7fc88200c640 1 -- 172.21.15.103:0/978096846 shutdown_connections 2024-09-19T06:48:32.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.081+0000 7fc88200c640 1 --2- 172.21.15.103:0/978096846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc87c1082a0 0x7fc87c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.081+0000 7fc88200c640 1 --2- 172.21.15.103:0/978096846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc87c1036e0 0x7fc87c107b90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.081+0000 7fc88200c640 1 --2- 172.21.15.103:0/978096846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc87c102d30 0x7fc87c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.081+0000 7fc88200c640 1 -- 172.21.15.103:0/978096846 >> 172.21.15.103:0/978096846 conn(0x7fc87c0fc7d0 msgr2=0x7fc87c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:32.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.081+0000 7fc88200c640 1 -- 172.21.15.103:0/978096846 shutdown_connections 2024-09-19T06:48:32.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.081+0000 7fc88200c640 1 -- 172.21.15.103:0/978096846 wait complete. 2024-09-19T06:48:32.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.081+0000 7fc88200c640 1 Processor -- start 2024-09-19T06:48:32.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc88200c640 1 -- start start 2024-09-19T06:48:32.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc88200c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc87c102d30 0x7fc87c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc88200c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc87c1036e0 0x7fc87c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc88200c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc87c1082a0 0x7fc87c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc88200c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c112140 con 0x7fc87c1082a0 2024-09-19T06:48:32.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc88200c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c111fc0 con 0x7fc87c102d30 2024-09-19T06:48:32.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc88200c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1122c0 con 0x7fc87c1036e0 2024-09-19T06:48:32.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc87bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc87c1082a0 0x7fc87c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc87bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc87c1082a0 0x7fc87c101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46492/0 (socket says 172.21.15.103:46492) 2024-09-19T06:48:32.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc87bfff640 1 -- 172.21.15.103:0/4136362455 learned_addr learned my addr 172.21.15.103:0/4136362455 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:32.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc87affd640 1 --2- 172.21.15.103:0/4136362455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc87c1036e0 0x7fc87c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc87b7fe640 1 --2- 172.21.15.103:0/4136362455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc87c102d30 0x7fc87c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc87bfff640 1 -- 172.21.15.103:0/4136362455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc87c1036e0 msgr2=0x7fc87c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:32.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc87bfff640 1 --2- 172.21.15.103:0/4136362455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc87c1036e0 0x7fc87c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc87bfff640 1 -- 172.21.15.103:0/4136362455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc87c102d30 msgr2=0x7fc87c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:32.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc87bfff640 1 --2- 172.21.15.103:0/4136362455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc87c102d30 0x7fc87c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc87bfff640 1 -- 172.21.15.103:0/4136362455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87c1a6bc0 con 0x7fc87c1082a0 2024-09-19T06:48:32.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc87b7fe640 1 --2- 172.21.15.103:0/4136362455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc87c102d30 0x7fc87c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:32.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.082+0000 7fc87bfff640 1 --2- 172.21.15.103:0/4136362455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc87c1082a0 0x7fc87c101410 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009fd0 tx=0x7fc86c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:32.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.083+0000 7fc878ff9640 1 -- 172.21.15.103:0/4136362455 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c011830 con 0x7fc87c1082a0 2024-09-19T06:48:32.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.083+0000 7fc878ff9640 1 -- 172.21.15.103:0/4136362455 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc86c016070 con 0x7fc87c1082a0 2024-09-19T06:48:32.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.083+0000 7fc88200c640 1 -- 172.21.15.103:0/4136362455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87c1a6eb0 con 0x7fc87c1082a0 2024-09-19T06:48:32.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.083+0000 7fc878ff9640 1 -- 172.21.15.103:0/4136362455 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00f2e0 con 0x7fc87c1082a0 2024-09-19T06:48:32.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.083+0000 7fc88200c640 1 -- 172.21.15.103:0/4136362455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87c1a7370 con 0x7fc87c1082a0 2024-09-19T06:48:32.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.083+0000 7fc88200c640 1 -- 172.21.15.103:0/4136362455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc87c103190 con 0x7fc87c1082a0 2024-09-19T06:48:32.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.085+0000 7fc878ff9640 1 -- 172.21.15.103:0/4136362455 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc86c0119d0 con 0x7fc87c1082a0 2024-09-19T06:48:32.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.085+0000 7fc878ff9640 1 --2- 172.21.15.103:0/4136362455 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc8680a32d0 0x7fc8680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.086+0000 7fc87b7fe640 1 --2- 172.21.15.103:0/4136362455 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc8680a32d0 0x7fc8680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.086+0000 7fc878ff9640 1 -- 172.21.15.103:0/4136362455 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc86c0cf6a0 con 0x7fc87c1082a0 2024-09-19T06:48:32.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.086+0000 7fc87b7fe640 1 --2- 172.21.15.103:0/4136362455 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc8680a32d0 0x7fc8680a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc874009fa0 tx=0x7fc874009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:32.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.089+0000 7fc878ff9640 1 -- 172.21.15.103:0/4136362455 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc86c09c380 con 0x7fc87c1082a0 2024-09-19T06:48:32.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.290+0000 7fc88200c640 1 -- 172.21.15.103:0/4136362455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc87c101ec0 con 0x7fc87c1082a0 2024-09-19T06:48:32.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.291+0000 7fc878ff9640 1 -- 172.21.15.103:0/4136362455 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11994 (secure 0 0 0) 0x7fc87c101ec0 con 0x7fc87c1082a0 2024-09-19T06:48:32.292 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:48:32.293 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-19T06:47:37:459674+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:47:36.481871+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24455,24454,24436,24431]},"id":1}]} 2024-09-19T06:48:32.293 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-19T06:48:32.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.293+0000 7fc88200c640 1 -- 172.21.15.103:0/4136362455 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc8680a32d0 msgr2=0x7fc8680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:32.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.293+0000 7fc88200c640 1 --2- 172.21.15.103:0/4136362455 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc8680a32d0 0x7fc8680a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc874009fa0 tx=0x7fc874009570 comp rx=0 tx=0).stop 2024-09-19T06:48:32.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.293+0000 7fc88200c640 1 -- 172.21.15.103:0/4136362455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc87c1082a0 msgr2=0x7fc87c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:32.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.293+0000 7fc88200c640 1 --2- 172.21.15.103:0/4136362455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc87c1082a0 0x7fc87c101410 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009fd0 tx=0x7fc86c00b8e0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.293+0000 7fc88200c640 1 -- 172.21.15.103:0/4136362455 shutdown_connections 2024-09-19T06:48:32.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.293+0000 7fc88200c640 1 --2- 172.21.15.103:0/4136362455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc87c1082a0 0x7fc87c101410 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.293+0000 7fc88200c640 1 --2- 172.21.15.103:0/4136362455 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc8680a32d0 0x7fc8680a5790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.293+0000 7fc88200c640 1 --2- 172.21.15.103:0/4136362455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc87c1036e0 0x7fc87c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.293+0000 7fc88200c640 1 --2- 172.21.15.103:0/4136362455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc87c102d30 0x7fc87c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.293+0000 7fc88200c640 1 -- 172.21.15.103:0/4136362455 >> 172.21.15.103:0/4136362455 conn(0x7fc87c0fc7d0 msgr2=0x7fc87c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:32.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.293+0000 7fc88200c640 1 -- 172.21.15.103:0/4136362455 shutdown_connections 2024-09-19T06:48:32.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:32.293+0000 7fc88200c640 1 -- 172.21.15.103:0/4136362455 wait complete. 2024-09-19T06:48:32.915 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:32 smithi103 ceph-mon[31158]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:48:32.915 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:32 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/4136362455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:32.944 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:48:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:32 smithi134 ceph-mon[35356]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:48:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:32 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/4136362455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:33.189 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:32 smithi174 ceph-mon[35637]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:48:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:32 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/4136362455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:33.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.720+0000 7ff4a2ecf640 1 -- 172.21.15.103:0/1339596384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff49c106a90 msgr2=0x7ff49c106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:33.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.720+0000 7ff4a2ecf640 1 --2- 172.21.15.103:0/1339596384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff49c106a90 0x7ff49c106e70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff490009930 tx=0x7ff49002a420 comp rx=0 tx=0).stop 2024-09-19T06:48:33.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.720+0000 7ff4a2ecf640 1 -- 172.21.15.103:0/1339596384 shutdown_connections 2024-09-19T06:48:33.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.720+0000 7ff4a2ecf640 1 --2- 172.21.15.103:0/1339596384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff49c101430 0x7ff49c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:33.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.720+0000 7ff4a2ecf640 1 --2- 172.21.15.103:0/1339596384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff49c100a90 0x7ff49c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:33.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.720+0000 7ff4a2ecf640 1 --2- 172.21.15.103:0/1339596384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff49c106a90 0x7ff49c106e70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:33.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.720+0000 7ff4a2ecf640 1 -- 172.21.15.103:0/1339596384 >> 172.21.15.103:0/1339596384 conn(0x7ff49c0fc7d0 msgr2=0x7ff49c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:33.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.720+0000 7ff4a2ecf640 1 -- 172.21.15.103:0/1339596384 shutdown_connections 2024-09-19T06:48:33.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.720+0000 7ff4a2ecf640 1 -- 172.21.15.103:0/1339596384 wait complete. 2024-09-19T06:48:33.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.720+0000 7ff4a2ecf640 1 Processor -- start 2024-09-19T06:48:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.720+0000 7ff4a2ecf640 1 -- start start 2024-09-19T06:48:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff4a2ecf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff49c100a90 0x7ff49c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff4a2ecf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff49c101430 0x7ff49c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff4a2ecf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff49c106a90 0x7ff49c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff4a2ecf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c110f30 con 0x7ff49c101430 2024-09-19T06:48:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff4a2ecf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c110db0 con 0x7ff49c106a90 2024-09-19T06:48:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff4a2ecf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c1110b0 con 0x7ff49c100a90 2024-09-19T06:48:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff49bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff49c101430 0x7ff49c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff49bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff49c101430 0x7ff49c198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46514/0 (socket says 172.21.15.103:46514) 2024-09-19T06:48:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff49bfff640 1 -- 172.21.15.103:0/1643140558 learned_addr learned my addr 172.21.15.103:0/1643140558 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff4a0c6b640 1 --2- 172.21.15.103:0/1643140558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff49c100a90 0x7ff49c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff4a146c640 1 --2- 172.21.15.103:0/1643140558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff49c106a90 0x7ff49c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:33.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff49bfff640 1 -- 172.21.15.103:0/1643140558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff49c100a90 msgr2=0x7ff49c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:33.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff49bfff640 1 --2- 172.21.15.103:0/1643140558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff49c100a90 0x7ff49c1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:33.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff49bfff640 1 -- 172.21.15.103:0/1643140558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff49c106a90 msgr2=0x7ff49c19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:33.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff49bfff640 1 --2- 172.21.15.103:0/1643140558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff49c106a90 0x7ff49c19ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:33.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff49bfff640 1 -- 172.21.15.103:0/1643140558 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c19d550 con 0x7ff49c101430 2024-09-19T06:48:33.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff49bfff640 1 --2- 172.21.15.103:0/1643140558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff49c101430 0x7ff49c198ae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b4a0 tx=0x7ff48800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:33.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff499ffb640 1 -- 172.21.15.103:0/1643140558 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488014070 con 0x7ff49c101430 2024-09-19T06:48:33.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.721+0000 7ff499ffb640 1 -- 172.21.15.103:0/1643140558 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff488010af0 con 0x7ff49c101430 2024-09-19T06:48:33.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.722+0000 7ff4a2ecf640 1 -- 172.21.15.103:0/1643140558 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49c19d840 con 0x7ff49c101430 2024-09-19T06:48:33.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.722+0000 7ff499ffb640 1 -- 172.21.15.103:0/1643140558 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488017d10 con 0x7ff49c101430 2024-09-19T06:48:33.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.722+0000 7ff4a2ecf640 1 -- 172.21.15.103:0/1643140558 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49c1a5120 con 0x7ff49c101430 2024-09-19T06:48:33.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.723+0000 7ff4a2ecf640 1 -- 172.21.15.103:0/1643140558 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49c100f70 con 0x7ff49c101430 2024-09-19T06:48:33.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.724+0000 7ff499ffb640 1 -- 172.21.15.103:0/1643140558 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff488017070 con 0x7ff49c101430 2024-09-19T06:48:33.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.724+0000 7ff499ffb640 1 --2- 172.21.15.103:0/1643140558 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff48c0a32d0 0x7ff48c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:33.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.725+0000 7ff4a0c6b640 1 --2- 172.21.15.103:0/1643140558 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff48c0a32d0 0x7ff48c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:33.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.725+0000 7ff499ffb640 1 -- 172.21.15.103:0/1643140558 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff4880cedc0 con 0x7ff49c101430 2024-09-19T06:48:33.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.725+0000 7ff4a0c6b640 1 --2- 172.21.15.103:0/1643140558 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff48c0a32d0 0x7ff48c0a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff490009900 tx=0x7ff4900058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:33.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.729+0000 7ff499ffb640 1 -- 172.21.15.103:0/1643140558 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff48809baa0 con 0x7ff49c101430 2024-09-19T06:48:33.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.931+0000 7ff4a2ecf640 1 -- 172.21.15.103:0/1643140558 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff49c02d7c0 con 0x7ff49c101430 2024-09-19T06:48:33.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.932+0000 7ff499ffb640 1 -- 172.21.15.103:0/1643140558 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11994 (secure 0 0 0) 0x7ff48809b3a0 con 0x7ff49c101430 2024-09-19T06:48:33.933 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:48:33.933 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-19T06:47:37:459674+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:47:36.481871+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24455,24454,24436,24431]},"id":1}]} 2024-09-19T06:48:33.933 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-19T06:48:33.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.934+0000 7ff4a2ecf640 1 -- 172.21.15.103:0/1643140558 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff48c0a32d0 msgr2=0x7ff48c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:33.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.934+0000 7ff4a2ecf640 1 --2- 172.21.15.103:0/1643140558 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff48c0a32d0 0x7ff48c0a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff490009900 tx=0x7ff4900058d0 comp rx=0 tx=0).stop 2024-09-19T06:48:33.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.934+0000 7ff4a2ecf640 1 -- 172.21.15.103:0/1643140558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff49c101430 msgr2=0x7ff49c198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:33.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.934+0000 7ff4a2ecf640 1 --2- 172.21.15.103:0/1643140558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff49c101430 0x7ff49c198ae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b4a0 tx=0x7ff48800b960 comp rx=0 tx=0).stop 2024-09-19T06:48:33.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.934+0000 7ff4a2ecf640 1 -- 172.21.15.103:0/1643140558 shutdown_connections 2024-09-19T06:48:33.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.934+0000 7ff4a2ecf640 1 --2- 172.21.15.103:0/1643140558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff49c106a90 0x7ff49c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:33.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.934+0000 7ff4a2ecf640 1 --2- 172.21.15.103:0/1643140558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff49c101430 0x7ff49c198ae0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:33.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.934+0000 7ff4a2ecf640 1 --2- 172.21.15.103:0/1643140558 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ff48c0a32d0 0x7ff48c0a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:33.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.934+0000 7ff4a2ecf640 1 --2- 172.21.15.103:0/1643140558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff49c100a90 0x7ff49c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:33.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.934+0000 7ff4a2ecf640 1 -- 172.21.15.103:0/1643140558 >> 172.21.15.103:0/1643140558 conn(0x7ff49c0fc7d0 msgr2=0x7ff49c06ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:33.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.934+0000 7ff4a2ecf640 1 -- 172.21.15.103:0/1643140558 shutdown_connections 2024-09-19T06:48:33.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:33.934+0000 7ff4a2ecf640 1 -- 172.21.15.103:0/1643140558 wait complete. 2024-09-19T06:48:34.575 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:48:34.835 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:35.113 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:34 smithi103 ceph-mon[31158]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:48:35.113 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:34 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1643140558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:34 smithi134 ceph-mon[35356]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:48:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:34 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1643140558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:35.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.298+0000 7f768c7ef640 1 -- 172.21.15.103:0/1044040136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7684102ef0 msgr2=0x7f768410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:35.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.298+0000 7f768c7ef640 1 --2- 172.21.15.103:0/1044040136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7684102ef0 0x7f768410da40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f767c009ca0 tx=0x7f767c02a950 comp rx=0 tx=0).stop 2024-09-19T06:48:35.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.298+0000 7f768c7ef640 1 -- 172.21.15.103:0/1044040136 shutdown_connections 2024-09-19T06:48:35.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.298+0000 7f768c7ef640 1 --2- 172.21.15.103:0/1044040136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7684102ef0 0x7f768410da40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:35.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.298+0000 7f768c7ef640 1 --2- 172.21.15.103:0/1044040136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7684102570 0x7f76841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:35.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.298+0000 7f768c7ef640 1 --2- 172.21.15.103:0/1044040136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7684100e20 0x7f7684101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:35.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.298+0000 7f768c7ef640 1 -- 172.21.15.103:0/1044040136 >> 172.21.15.103:0/1044040136 conn(0x7f76840fc7d0 msgr2=0x7f76840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:35.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.298+0000 7f768c7ef640 1 -- 172.21.15.103:0/1044040136 shutdown_connections 2024-09-19T06:48:35.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.298+0000 7f768c7ef640 1 -- 172.21.15.103:0/1044040136 wait complete. 2024-09-19T06:48:35.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.298+0000 7f768c7ef640 1 Processor -- start 2024-09-19T06:48:35.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.298+0000 7f768c7ef640 1 -- start start 2024-09-19T06:48:35.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768c7ef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7684100e20 0x7f768419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:35.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768c7ef640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7684102570 0x7f768419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:35.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768c7ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7684102ef0 0x7f768419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:35.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768c7ef640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7684110f80 con 0x7f7684100e20 2024-09-19T06:48:35.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768c7ef640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7684110e00 con 0x7f7684102ef0 2024-09-19T06:48:35.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768c7ef640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7684111100 con 0x7f7684102570 2024-09-19T06:48:35.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768a58b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7684100e20 0x7f768419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:35.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768a58b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7684100e20 0x7f768419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46540/0 (socket says 172.21.15.103:46540) 2024-09-19T06:48:35.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768a58b640 1 -- 172.21.15.103:0/2319304563 learned_addr learned my addr 172.21.15.103:0/2319304563 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:35.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768ad8c640 1 --2- 172.21.15.103:0/2319304563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7684102ef0 0x7f768419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:35.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f7689d8a640 1 --2- 172.21.15.103:0/2319304563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7684102570 0x7f768419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:35.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768a58b640 1 -- 172.21.15.103:0/2319304563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7684102570 msgr2=0x7f768419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:35.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768a58b640 1 --2- 172.21.15.103:0/2319304563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7684102570 0x7f768419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:35.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768a58b640 1 -- 172.21.15.103:0/2319304563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7684102ef0 msgr2=0x7f768419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:35.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768a58b640 1 --2- 172.21.15.103:0/2319304563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7684102ef0 0x7f768419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:35.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768a58b640 1 -- 172.21.15.103:0/2319304563 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768419f630 con 0x7f7684100e20 2024-09-19T06:48:35.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f7689d8a640 1 --2- 172.21.15.103:0/2319304563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7684102570 0x7f768419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:35.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768a58b640 1 --2- 172.21.15.103:0/2319304563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7684100e20 0x7f768419a680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f768000d770 tx=0x7f768000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:35.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f76777fe640 1 -- 172.21.15.103:0/2319304563 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680017070 con 0x7f7684100e20 2024-09-19T06:48:35.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768c7ef640 1 -- 172.21.15.103:0/2319304563 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768419f920 con 0x7f7684100e20 2024-09-19T06:48:35.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f76777fe640 1 -- 172.21.15.103:0/2319304563 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7680005500 con 0x7f7684100e20 2024-09-19T06:48:35.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f768c7ef640 1 -- 172.21.15.103:0/2319304563 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76841a7200 con 0x7f7684100e20 2024-09-19T06:48:35.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.299+0000 7f76777fe640 1 -- 172.21.15.103:0/2319304563 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680020060 con 0x7f7684100e20 2024-09-19T06:48:35.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.300+0000 7f768c7ef640 1 -- 172.21.15.103:0/2319304563 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7684101200 con 0x7f7684100e20 2024-09-19T06:48:35.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.302+0000 7f76777fe640 1 -- 172.21.15.103:0/2319304563 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7680004da0 con 0x7f7684100e20 2024-09-19T06:48:35.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.303+0000 7f76777fe640 1 --2- 172.21.15.103:0/2319304563 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f76700a32d0 0x7f76700a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:35.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.303+0000 7f7689d8a640 1 --2- 172.21.15.103:0/2319304563 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f76700a32d0 0x7f76700a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:35.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.303+0000 7f76777fe640 1 -- 172.21.15.103:0/2319304563 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f76800cfac0 con 0x7f7684100e20 2024-09-19T06:48:35.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.303+0000 7f7689d8a640 1 --2- 172.21.15.103:0/2319304563 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f76700a32d0 0x7f76700a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f768410ddf0 tx=0x7f7678005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:35.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.306+0000 7f76777fe640 1 -- 172.21.15.103:0/2319304563 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f768009c630 con 0x7f7684100e20 2024-09-19T06:48:35.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:34 smithi174 ceph-mon[35637]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:48:35.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:34 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1643140558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:35.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.475+0000 7f768c7ef640 1 -- 172.21.15.103:0/2319304563 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76841a7400 con 0x7f7684100e20 2024-09-19T06:48:35.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.476+0000 7f76777fe640 1 -- 172.21.15.103:0/2319304563 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22729 (secure 0 0 0) 0x7f768009bf30 con 0x7f7684100e20 2024-09-19T06:48:35.479 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:48:35.479 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":82,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","created":"2024-09-19T06:39:46.840574+0000","modified":"2024-09-19T06:47:40.819637+0000","last_up_change":"2024-09-19T06:46:35.652960+0000","last_in_change":"2024-09-19T06:46:11.630585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:42:52.050881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:47:21.192611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:47:23.643291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d45dab3-e434-4783-b2e3-f658c3ba583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6803","nonce":3157697740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6805","nonce":3157697740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6809","nonce":3157697740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6807","nonce":3157697740}]},"public_addr":"172.21.15.103:6803/3157697740","cluster_addr":"172.21.15.103:6805/3157697740","heartbeat_back_addr":"172.21.15.103:6809/3157697740","heartbeat_front_addr":"172.21.15.103:6807/3157697740","state":["exists","up"]},{"osd":1,"uuid":"ee0e92bd-15ab-4867-9a18-20abb9f5fc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6801","nonce":1418301195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6803","nonce":1418301195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6807","nonce":1418301195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6805","nonce":1418301195}]},"public_addr":"172.21.15.134:6801/1418301195","cluster_addr":"172.21.15.134:6803/1418301195","heartbeat_back_addr":"172.21.15.134:6807/1418301195","heartbeat_front_addr":"172.21.15.134:6805/1418301195","state":["exists","up"]},{"osd":2,"uuid":"0230caed-025b-46e9-a011-237601d16ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6801","nonce":2292242290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6803","nonce":2292242290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6807","nonce":2292242290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6805","nonce":2292242290}]},"public_addr":"172.21.15.174:6801/2292242290","cluster_addr":"172.21.15.174:6803/2292242290","heartbeat_back_addr":"172.21.15.174:6807/2292242290","heartbeat_front_addr":"172.21.15.174:6805/2292242290","state":["exists","up"]},{"osd":3,"uuid":"c54a8399-bf4b-4d70-b207-a13d85f99590","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6811","nonce":2698986333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6813","nonce":2698986333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6817","nonce":2698986333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6815","nonce":2698986333}]},"public_addr":"172.21.15.103:6811/2698986333","cluster_addr":"172.21.15.103:6813/2698986333","heartbeat_back_addr":"172.21.15.103:6817/2698986333","heartbeat_front_addr":"172.21.15.103:6815/2698986333","state":["exists","up"]},{"osd":4,"uuid":"7c046be6-464a-4ab5-812b-1781121fc317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6809","nonce":3169891968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6811","nonce":3169891968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6815","nonce":3169891968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6813","nonce":3169891968}]},"public_addr":"172.21.15.134:6809/3169891968","cluster_addr":"172.21.15.134:6811/3169891968","heartbeat_back_addr":"172.21.15.134:6815/3169891968","heartbeat_front_addr":"172.21.15.134:6813/3169891968","state":["exists","up"]},{"osd":5,"uuid":"8622596b-6a06-459c-882d-50fba9e296ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6809","nonce":763749644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6811","nonce":763749644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6815","nonce":763749644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6813","nonce":763749644}]},"public_addr":"172.21.15.174:6809/763749644","cluster_addr":"172.21.15.174:6811/763749644","heartbeat_back_addr":"172.21.15.174:6815/763749644","heartbeat_front_addr":"172.21.15.174:6813/763749644","state":["exists","up"]},{"osd":6,"uuid":"64d858d6-a8b3-4f65-a187-af9ab1ccee0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6819","nonce":1396972394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6821","nonce":1396972394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6825","nonce":1396972394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6823","nonce":1396972394}]},"public_addr":"172.21.15.103:6819/1396972394","cluster_addr":"172.21.15.103:6821/1396972394","heartbeat_back_addr":"172.21.15.103:6825/1396972394","heartbeat_front_addr":"172.21.15.103:6823/1396972394","state":["exists","up"]},{"osd":7,"uuid":"d405d1c6-839c-4c92-8261-4adb3163c90f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6817","nonce":2196366765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6819","nonce":2196366765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6823","nonce":2196366765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6821","nonce":2196366765}]},"public_addr":"172.21.15.134:6817/2196366765","cluster_addr":"172.21.15.134:6819/2196366765","heartbeat_back_addr":"172.21.15.134:6823/2196366765","heartbeat_front_addr":"172.21.15.134:6821/2196366765","state":["exists","up"]},{"osd":8,"uuid":"4c452cd9-816c-4f77-9368-2cd4bbeee4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6817","nonce":189977465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6819","nonce":189977465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6823","nonce":189977465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6821","nonce":189977465}]},"public_addr":"172.21.15.174:6817/189977465","cluster_addr":"172.21.15.174:6819/189977465","heartbeat_back_addr":"172.21.15.174:6823/189977465","heartbeat_front_addr":"172.21.15.174:6821/189977465","state":["exists","up"]},{"osd":9,"uuid":"fb3662e0-2d8e-45c4-9119-6d08b35443da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6827","nonce":3161966413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6829","nonce":3161966413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6833","nonce":3161966413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6831","nonce":3161966413}]},"public_addr":"172.21.15.103:6827/3161966413","cluster_addr":"172.21.15.103:6829/3161966413","heartbeat_back_addr":"172.21.15.103:6833/3161966413","heartbeat_front_addr":"172.21.15.103:6831/3161966413","state":["exists","up"]},{"osd":10,"uuid":"e52c3cbe-a6af-441e-b423-b94f3a2da544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6825","nonce":2532758632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6827","nonce":2532758632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6831","nonce":2532758632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6829","nonce":2532758632}]},"public_addr":"172.21.15.134:6825/2532758632","cluster_addr":"172.21.15.134:6827/2532758632","heartbeat_back_addr":"172.21.15.134:6831/2532758632","heartbeat_front_addr":"172.21.15.134:6829/2532758632","state":["exists","up"]},{"osd":11,"uuid":"9040f3f8-3873-4dea-8716-dcda5811c8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6825","nonce":161916442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6827","nonce":161916442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6831","nonce":161916442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6829","nonce":161916442}]},"public_addr":"172.21.15.174:6825/161916442","cluster_addr":"172.21.15.174:6827/161916442","heartbeat_back_addr":"172.21.15.174:6831/161916442","heartbeat_front_addr":"172.21.15.174:6829/161916442","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:41:59.656854+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:24.797551+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:49.685497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:33.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:39.324983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:03.987310+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:28.747971+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:53.827780+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:18.554450+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:43.705691+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:08.401275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:33.502542+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2867049580":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/3727259725":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/1496153385":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:6800/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1642809745":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/2309624637":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1596229922":"2024-09-20T06:40:16.780890+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:48:35.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.478+0000 7f768c7ef640 1 -- 172.21.15.103:0/2319304563 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f76700a32d0 msgr2=0x7f76700a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:35.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.478+0000 7f768c7ef640 1 --2- 172.21.15.103:0/2319304563 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f76700a32d0 0x7f76700a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f768410ddf0 tx=0x7f7678005d30 comp rx=0 tx=0).stop 2024-09-19T06:48:35.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.478+0000 7f768c7ef640 1 -- 172.21.15.103:0/2319304563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7684100e20 msgr2=0x7f768419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:35.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.478+0000 7f768c7ef640 1 --2- 172.21.15.103:0/2319304563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7684100e20 0x7f768419a680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f768000d770 tx=0x7f768000dc30 comp rx=0 tx=0).stop 2024-09-19T06:48:35.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.479+0000 7f768c7ef640 1 -- 172.21.15.103:0/2319304563 shutdown_connections 2024-09-19T06:48:35.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.479+0000 7f768c7ef640 1 --2- 172.21.15.103:0/2319304563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7684102ef0 0x7f768419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:35.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.479+0000 7f768c7ef640 1 --2- 172.21.15.103:0/2319304563 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f76700a32d0 0x7f76700a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:35.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.479+0000 7f768c7ef640 1 --2- 172.21.15.103:0/2319304563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7684102570 0x7f768419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:35.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.479+0000 7f768c7ef640 1 --2- 172.21.15.103:0/2319304563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7684100e20 0x7f768419a680 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:35.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.479+0000 7f768c7ef640 1 -- 172.21.15.103:0/2319304563 >> 172.21.15.103:0/2319304563 conn(0x7f76840fc7d0 msgr2=0x7f7684069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:35.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.479+0000 7f768c7ef640 1 -- 172.21.15.103:0/2319304563 shutdown_connections 2024-09-19T06:48:35.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:35.479+0000 7f768c7ef640 1 -- 172.21.15.103:0/2319304563 wait complete. 2024-09-19T06:48:36.121 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:35 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2319304563' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:48:36.124 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:48:36.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:35 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2319304563' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:48:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:35 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2319304563' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:48:36.371 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:36.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.786+0000 7f0011576640 1 -- 172.21.15.103:0/1312961094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100e20 msgr2=0x7f000c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:36.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.786+0000 7f0011576640 1 --2- 172.21.15.103:0/1312961094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100e20 0x7f000c101200 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0004009ca0 tx=0x7f000402a950 comp rx=0 tx=0).stop 2024-09-19T06:48:36.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.786+0000 7f0011576640 1 -- 172.21.15.103:0/1312961094 shutdown_connections 2024-09-19T06:48:36.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.786+0000 7f0011576640 1 --2- 172.21.15.103:0/1312961094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f000c102ef0 0x7f000c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.786+0000 7f0011576640 1 --2- 172.21.15.103:0/1312961094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f000c102570 0x7f000c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.786+0000 7f0011576640 1 --2- 172.21.15.103:0/1312961094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100e20 0x7f000c101200 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.786+0000 7f0011576640 1 -- 172.21.15.103:0/1312961094 >> 172.21.15.103:0/1312961094 conn(0x7f000c0fc7d0 msgr2=0x7f000c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:36.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.786+0000 7f0011576640 1 -- 172.21.15.103:0/1312961094 shutdown_connections 2024-09-19T06:48:36.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.786+0000 7f0011576640 1 -- 172.21.15.103:0/1312961094 wait complete. 2024-09-19T06:48:36.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.786+0000 7f0011576640 1 Processor -- start 2024-09-19T06:48:36.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f0011576640 1 -- start start 2024-09-19T06:48:36.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f0011576640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100e20 0x7f000c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f0011576640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f000c102570 0x7f000c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f0011576640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f000c102ef0 0x7f000c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f0011576640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f000c110fc0 con 0x7f000c100e20 2024-09-19T06:48:36.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f0011576640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f000c110e40 con 0x7f000c102570 2024-09-19T06:48:36.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f0011576640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f000c111140 con 0x7f000c102ef0 2024-09-19T06:48:36.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f000affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100e20 0x7f000c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f000affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100e20 0x7f000c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46562/0 (socket says 172.21.15.103:46562) 2024-09-19T06:48:36.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f000affd640 1 -- 172.21.15.103:0/2565378265 learned_addr learned my addr 172.21.15.103:0/2565378265 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:36.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f000a7fc640 1 --2- 172.21.15.103:0/2565378265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f000c102570 0x7f000c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f000b7fe640 1 --2- 172.21.15.103:0/2565378265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f000c102ef0 0x7f000c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f000affd640 1 -- 172.21.15.103:0/2565378265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f000c102ef0 msgr2=0x7f000c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:36.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f000affd640 1 --2- 172.21.15.103:0/2565378265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f000c102ef0 0x7f000c19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f000affd640 1 -- 172.21.15.103:0/2565378265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f000c102570 msgr2=0x7f000c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:36.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f000affd640 1 --2- 172.21.15.103:0/2565378265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f000c102570 0x7f000c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f000affd640 1 -- 172.21.15.103:0/2565378265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c19d560 con 0x7f000c100e20 2024-09-19T06:48:36.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.787+0000 7f000affd640 1 --2- 172.21.15.103:0/2565378265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100e20 0x7f000c1985b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0004005cc0 tx=0x7f000402f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:36.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.788+0000 7f0003fff640 1 -- 172.21.15.103:0/2565378265 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004034040 con 0x7f000c100e20 2024-09-19T06:48:36.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.788+0000 7f0011576640 1 -- 172.21.15.103:0/2565378265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000c19d7f0 con 0x7f000c100e20 2024-09-19T06:48:36.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.788+0000 7f0003fff640 1 -- 172.21.15.103:0/2565378265 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0004030920 con 0x7f000c100e20 2024-09-19T06:48:36.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.788+0000 7f0003fff640 1 -- 172.21.15.103:0/2565378265 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004004830 con 0x7f000c100e20 2024-09-19T06:48:36.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.788+0000 7f0011576640 1 -- 172.21.15.103:0/2565378265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000c1a5130 con 0x7f000c100e20 2024-09-19T06:48:36.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.789+0000 7f0011576640 1 -- 172.21.15.103:0/2565378265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f000c101200 con 0x7f000c100e20 2024-09-19T06:48:36.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.790+0000 7f0003fff640 1 -- 172.21.15.103:0/2565378265 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0004037020 con 0x7f000c100e20 2024-09-19T06:48:36.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.790+0000 7f0003fff640 1 --2- 172.21.15.103:0/2565378265 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efff80a32d0 0x7efff80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.791+0000 7f000a7fc640 1 --2- 172.21.15.103:0/2565378265 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efff80a32d0 0x7efff80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.791+0000 7f0003fff640 1 -- 172.21.15.103:0/2565378265 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f00040f0c70 con 0x7f000c100e20 2024-09-19T06:48:36.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.791+0000 7f000a7fc640 1 --2- 172.21.15.103:0/2565378265 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efff80a32d0 0x7efff80a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f000c0fff90 tx=0x7efff4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:36.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:36.794+0000 7f0003fff640 1 -- 172.21.15.103:0/2565378265 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0004081210 con 0x7f000c100e20 2024-09-19T06:48:37.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:37.003+0000 7f0011576640 1 -- 172.21.15.103:0/2565378265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f000c10a0a0 con 0x7f000c100e20 2024-09-19T06:48:37.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:37.004+0000 7f0003fff640 1 -- 172.21.15.103:0/2565378265 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11994 (secure 0 0 0) 0x7f00040bd950 con 0x7f000c100e20 2024-09-19T06:48:37.006 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:48:37.006 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-19T06:47:37:459674+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:47:36.481871+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24455,24454,24436,24431]},"id":1}]} 2024-09-19T06:48:37.006 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-19T06:48:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:37.006+0000 7f0011576640 1 -- 172.21.15.103:0/2565378265 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efff80a32d0 msgr2=0x7efff80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:37.006+0000 7f0011576640 1 --2- 172.21.15.103:0/2565378265 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efff80a32d0 0x7efff80a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f000c0fff90 tx=0x7efff4008040 comp rx=0 tx=0).stop 2024-09-19T06:48:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:37.006+0000 7f0011576640 1 -- 172.21.15.103:0/2565378265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100e20 msgr2=0x7f000c1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:37.006+0000 7f0011576640 1 --2- 172.21.15.103:0/2565378265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100e20 0x7f000c1985b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0004005cc0 tx=0x7f000402f650 comp rx=0 tx=0).stop 2024-09-19T06:48:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:37.006+0000 7f0011576640 1 -- 172.21.15.103:0/2565378265 shutdown_connections 2024-09-19T06:48:37.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:37.006+0000 7f0011576640 1 --2- 172.21.15.103:0/2565378265 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7efff80a32d0 0x7efff80a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:37.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:37.006+0000 7f0011576640 1 --2- 172.21.15.103:0/2565378265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f000c102ef0 0x7f000c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:37.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:37.006+0000 7f0011576640 1 --2- 172.21.15.103:0/2565378265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f000c102570 0x7f000c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:37.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:37.006+0000 7f0011576640 1 --2- 172.21.15.103:0/2565378265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100e20 0x7f000c1985b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:37.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:37.006+0000 7f0011576640 1 -- 172.21.15.103:0/2565378265 >> 172.21.15.103:0/2565378265 conn(0x7f000c0fc7d0 msgr2=0x7f000c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:37.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:37.006+0000 7f0011576640 1 -- 172.21.15.103:0/2565378265 shutdown_connections 2024-09-19T06:48:37.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:37.006+0000 7f0011576640 1 -- 172.21.15.103:0/2565378265 wait complete. 2024-09-19T06:48:37.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:36 smithi103 ceph-mon[31158]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T06:48:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:36 smithi134 ceph-mon[35356]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T06:48:37.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:36 smithi174 ceph-mon[35637]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T06:48:37.598 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T06:47:25.446176+0000', 'modified': '2024-09-19T06:47:36.481871+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24482, 'mds_1': 24455, 'mds_2': 24454, 'mds_3': 24436, 'mds_4': 24431}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14586': {'gid': 14586, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6839/3119326077', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 3119326077}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 3119326077}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14598': {'gid': 14598, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6841/3771437304', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6840', 'nonce': 3771437304}, {'type': 'v1', 'addr': '172.21.15.103:6841', 'nonce': 3771437304}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24397': {'gid': 24397, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/2319563070', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 2319563070}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 2319563070}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/1556743259', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 1556743259}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 1556743259}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/963732203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 963732203}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 963732203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'e', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.134:6835/3092414367', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 3092414367}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 3092414367}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.174:6837/1053388911', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 1053388911}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 1053388911}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.174:6839/4202229983', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 4202229983}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 4202229983}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.134:6837/1590453330', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 1590453330}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 1590453330}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.134:6839/2333076106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 2333076106}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 2333076106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24482, 24455, 24454, 24436, 24431]} 2024-09-19T06:48:37.598 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T06:48:37.598 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.e status 2024-09-19T06:48:37.844 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:38.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:37 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2565378265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:38.273 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:37 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2565378265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:38.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:37 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2565378265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.352+0000 7fe7eb54b640 1 -- 172.21.15.103:0/583916050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e41036e0 msgr2=0x7fe7e4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.352+0000 7fe7eb54b640 1 --2- 172.21.15.103:0/583916050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e41036e0 0x7fe7e4107b90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009930 tx=0x7fe7d802a420 comp rx=0 tx=0).stop 2024-09-19T06:48:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.352+0000 7fe7eb54b640 1 -- 172.21.15.103:0/583916050 shutdown_connections 2024-09-19T06:48:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.352+0000 7fe7eb54b640 1 --2- 172.21.15.103:0/583916050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7e41082a0 0x7fe7e410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.352+0000 7fe7eb54b640 1 --2- 172.21.15.103:0/583916050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e41036e0 0x7fe7e4107b90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.352+0000 7fe7eb54b640 1 --2- 172.21.15.103:0/583916050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7e4102d30 0x7fe7e4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.352+0000 7fe7eb54b640 1 -- 172.21.15.103:0/583916050 >> 172.21.15.103:0/583916050 conn(0x7fe7e40fc7d0 msgr2=0x7fe7e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:38.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.352+0000 7fe7eb54b640 1 -- 172.21.15.103:0/583916050 shutdown_connections 2024-09-19T06:48:38.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.352+0000 7fe7eb54b640 1 -- 172.21.15.103:0/583916050 wait complete. 2024-09-19T06:48:38.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.352+0000 7fe7eb54b640 1 Processor -- start 2024-09-19T06:48:38.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7eb54b640 1 -- start start 2024-09-19T06:48:38.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7eb54b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7e4102d30 0x7fe7e41a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7eb54b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7e41036e0 0x7fe7e41a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7eb54b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e41082a0 0x7fe7e419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7eb54b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e41120c0 con 0x7fe7e41082a0 2024-09-19T06:48:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7eb54b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e4111f40 con 0x7fe7e41036e0 2024-09-19T06:48:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7eb54b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e4112240 con 0x7fe7e4102d30 2024-09-19T06:48:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7e9ae8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e41082a0 0x7fe7e419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7e9ae8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e41082a0 0x7fe7e419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55086/0 (socket says 172.21.15.103:55086) 2024-09-19T06:48:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7e9ae8640 1 -- 172.21.15.103:0/3305080229 learned_addr learned my addr 172.21.15.103:0/3305080229 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7e92e7640 1 --2- 172.21.15.103:0/3305080229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7e4102d30 0x7fe7e41a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7e8ae6640 1 --2- 172.21.15.103:0/3305080229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7e41036e0 0x7fe7e41a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7e9ae8640 1 -- 172.21.15.103:0/3305080229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7e4102d30 msgr2=0x7fe7e41a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7e9ae8640 1 --2- 172.21.15.103:0/3305080229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7e4102d30 0x7fe7e41a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7e9ae8640 1 -- 172.21.15.103:0/3305080229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7e41036e0 msgr2=0x7fe7e41a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7e9ae8640 1 --2- 172.21.15.103:0/3305080229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7e41036e0 0x7fe7e41a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7e9ae8640 1 -- 172.21.15.103:0/3305080229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e419aec0 con 0x7fe7e41082a0 2024-09-19T06:48:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7e92e7640 1 --2- 172.21.15.103:0/3305080229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7e4102d30 0x7fe7e41a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7e8ae6640 1 --2- 172.21.15.103:0/3305080229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7e41036e0 0x7fe7e41a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.353+0000 7fe7e9ae8640 1 --2- 172.21.15.103:0/3305080229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e41082a0 0x7fe7e419a650 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400b970 tx=0x7fe7d400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.354+0000 7fe7e27fc640 1 -- 172.21.15.103:0/3305080229 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4019070 con 0x7fe7e41082a0 2024-09-19T06:48:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.354+0000 7fe7e27fc640 1 -- 172.21.15.103:0/3305080229 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d400cb40 con 0x7fe7e41082a0 2024-09-19T06:48:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.354+0000 7fe7eb54b640 1 -- 172.21.15.103:0/3305080229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e419b1b0 con 0x7fe7e41082a0 2024-09-19T06:48:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.354+0000 7fe7e27fc640 1 -- 172.21.15.103:0/3305080229 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4007650 con 0x7fe7e41082a0 2024-09-19T06:48:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.354+0000 7fe7eb54b640 1 -- 172.21.15.103:0/3305080229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e41a7500 con 0x7fe7e41082a0 2024-09-19T06:48:38.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.356+0000 7fe7e27fc640 1 -- 172.21.15.103:0/3305080229 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe7d401c070 con 0x7fe7e41082a0 2024-09-19T06:48:38.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.356+0000 7fe7ed924740 1 Processor -- start 2024-09-19T06:48:38.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.356+0000 7fe7ed924740 1 -- start start 2024-09-19T06:48:38.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7e27fc640 1 --2- 172.21.15.103:0/3305080229 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe7cc0a3350 0x7fe7cc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7ed924740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fa9d0 0x5598642fadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7ed924740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598642fb2f0 0x559864305670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7ed924740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559864305bb0 0x559864307fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7ed924740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559864261090 con 0x559864305bb0 2024-09-19T06:48:38.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7ed924740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559864208a70 con 0x5598642fa9d0 2024-09-19T06:48:38.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7ed924740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55986425d050 con 0x5598642fb2f0 2024-09-19T06:48:38.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7e9ae8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559864305bb0 0x559864307fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7e8ae6640 1 --2- 172.21.15.103:0/3305080229 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe7cc0a3350 0x7fe7cc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7e9ae8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559864305bb0 0x559864307fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55090/0 (socket says 172.21.15.103:55090) 2024-09-19T06:48:38.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7e9ae8640 1 -- 172.21.15.103:0/757416420 learned_addr learned my addr 172.21.15.103:0/757416420 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:38.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7e27fc640 1 -- 172.21.15.103:0/3305080229 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe7d40cefb0 con 0x7fe7e41082a0 2024-09-19T06:48:38.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7e92e7640 1 --2- 172.21.15.103:0/757416420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fa9d0 0x5598642fadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7e8ae6640 1 --2- 172.21.15.103:0/757416420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598642fb2f0 0x559864305670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7e9ae8640 1 -- 172.21.15.103:0/757416420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598642fb2f0 msgr2=0x559864305670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:38.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7e9ae8640 1 --2- 172.21.15.103:0/757416420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598642fb2f0 0x559864305670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7e9ae8640 1 -- 172.21.15.103:0/757416420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fa9d0 msgr2=0x5598642fadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:38.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7e9ae8640 1 --2- 172.21.15.103:0/757416420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fa9d0 0x5598642fadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7e9ae8640 1 -- 172.21.15.103:0/757416420 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559864215130 con 0x559864305bb0 2024-09-19T06:48:38.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7e8ae6640 1 --2- 172.21.15.103:0/3305080229 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe7cc0a3350 0x7fe7cc0a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009900 tx=0x7fe7d8031790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:38.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7e9ae8640 1 --2- 172.21.15.103:0/757416420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559864305bb0 0x559864307fa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40a3cd0 tx=0x7fe7d4014d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253bfcadf5a4bdca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:38.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.357+0000 7fe7cb7fe640 1 -- 172.21.15.103:0/757416420 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d409bc90 con 0x559864305bb0 2024-09-19T06:48:38.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.358+0000 7fe7cb7fe640 1 -- 172.21.15.103:0/757416420 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d406f7d0 con 0x559864305bb0 2024-09-19T06:48:38.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.358+0000 7fe7cb7fe640 1 -- 172.21.15.103:0/757416420 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d40a4200 con 0x559864305bb0 2024-09-19T06:48:38.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.358+0000 7fe7ed924740 1 -- 172.21.15.103:0/757416420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559864305bb0 msgr2=0x559864307fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.358+0000 7fe7ed924740 1 --2- 172.21.15.103:0/757416420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559864305bb0 0x559864307fa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40a3cd0 tx=0x7fe7d4014d00 comp rx=0 tx=0).stop 2024-09-19T06:48:38.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.358+0000 7fe7ed924740 1 -- 172.21.15.103:0/757416420 shutdown_connections 2024-09-19T06:48:38.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.358+0000 7fe7ed924740 1 --2- 172.21.15.103:0/757416420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559864305bb0 0x559864307fa0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.358+0000 7fe7ed924740 1 --2- 172.21.15.103:0/757416420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598642fb2f0 0x559864305670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.358+0000 7fe7ed924740 1 --2- 172.21.15.103:0/757416420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fa9d0 0x5598642fadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.358+0000 7fe7ed924740 1 -- 172.21.15.103:0/757416420 >> 172.21.15.103:0/757416420 conn(0x5598642f6f60 msgr2=0x5598642f7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:38.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.358+0000 7fe7ed924740 1 -- 172.21.15.103:0/757416420 shutdown_connections 2024-09-19T06:48:38.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.358+0000 7fe7ed924740 1 -- 172.21.15.103:0/757416420 wait complete. 2024-09-19T06:48:38.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.359+0000 7fe7ed924740 1 -- start start 2024-09-19T06:48:38.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.359+0000 7fe7ed924740 1 Processor -- start 2024-09-19T06:48:38.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.359+0000 7fe7ed924740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fa9d0 0x559864305390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.359+0000 7fe7ed924740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fb2f0 0x5598643026e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.359+0000 7fe7ed924740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559864305bb0 0x559864302c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.359+0000 7fe7ed924740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559864308780 con 0x5598642fb2f0 2024-09-19T06:48:38.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.359+0000 7fe7ed924740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55986421bf20 con 0x5598642fa9d0 2024-09-19T06:48:38.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.359+0000 7fe7ed924740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559864308900 con 0x559864305bb0 2024-09-19T06:48:38.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.359+0000 7fe7ed924740 10 client.0 _ll_register_callbacks cb 0x5598641e5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:38.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.359+0000 7fe7e8ae6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fb2f0 0x5598643026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.359+0000 7fe7e8ae6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fb2f0 0x5598643026e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55096/0 (socket says 172.21.15.103:55096) 2024-09-19T06:48:38.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.359+0000 7fe7e8ae6640 1 -- 172.21.15.103:0/2822444818 learned_addr learned my addr 172.21.15.103:0/2822444818 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:38.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7e92e7640 1 --2- 172.21.15.103:0/2822444818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fa9d0 0x559864305390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7e9ae8640 1 --2- 172.21.15.103:0/2822444818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559864305bb0 0x559864302c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7e8ae6640 1 -- 172.21.15.103:0/2822444818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559864305bb0 msgr2=0x559864302c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:38.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7e8ae6640 1 --2- 172.21.15.103:0/2822444818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559864305bb0 0x559864302c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7e8ae6640 1 -- 172.21.15.103:0/2822444818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fa9d0 msgr2=0x559864305390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:38.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7e8ae6640 1 --2- 172.21.15.103:0/2822444818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fa9d0 0x559864305390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7e8ae6640 1 -- 172.21.15.103:0/2822444818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559863fd22b0 con 0x5598642fb2f0 2024-09-19T06:48:38.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7e92e7640 1 --2- 172.21.15.103:0/2822444818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fa9d0 0x559864305390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:38.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7e8ae6640 1 --2- 172.21.15.103:0/2822444818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fb2f0 0x5598643026e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8000c00 tx=0x7fe7d802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4487ca314bf2c6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:38.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:48:38.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7c8ff9640 1 -- 172.21.15.103:0/2822444818 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d804c020 con 0x5598642fb2f0 2024-09-19T06:48:38.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7c8ff9640 1 -- 172.21.15.103:0/2822444818 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe7d803d890 con 0x5598642fb2f0 2024-09-19T06:48:38.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7ed924740 1 -- 172.21.15.103:0/2822444818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7d406f7d0 con 0x5598642fb2f0 2024-09-19T06:48:38.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7c8ff9640 1 -- 172.21.15.103:0/2822444818 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe7d8035050 con 0x5598642fb2f0 2024-09-19T06:48:38.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7ed924740 10 client.14958 fetch_fsmap learned FSMap version 26 2024-09-19T06:48:38.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7ed924740 1 -- 172.21.15.103:0/2822444818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559864313fe0 con 0x5598642fb2f0 2024-09-19T06:48:38.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.360+0000 7fe7c8ff9640 1 -- 172.21.15.103:0/2822444818 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d406f7d0 con 0x5598642fb2f0 2024-09-19T06:48:38.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.361+0000 7fe7c8ff9640 1 -- 172.21.15.103:0/2822444818 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fe7d8047070 con 0x5598642fb2f0 2024-09-19T06:48:38.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.361+0000 7fe7ed924740 10 client.14958 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:48:38.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.361+0000 7fe7ed924740 10 client.14958 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-19T06:48:38.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.361+0000 7fe7ed924740 20 client.14958 populate_metadata read hostname 'smithi103' 2024-09-19T06:48:38.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.361+0000 7fe7ed924740 1 --2- 172.21.15.103:0/2822444818 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x559864316620 0x559864317a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.361+0000 7fe7ed924740 4 client.14958 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:48:38.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.361+0000 7fe7ed924740 1 -- 172.21.15.103:0/2822444818 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55986426e600 con 0x559864316620 2024-09-19T06:48:38.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.361+0000 7fe7e92e7640 1 --2- 172.21.15.103:0/2822444818 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x559864316620 0x559864317a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.362+0000 7fe7e92e7640 1 --2- 172.21.15.103:0/2822444818 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x559864316620 0x559864317a90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a315ff4565fecd6b server_cookie=54347d1f3337fc1 in_seq=0 out_seq=0 2024-09-19T06:48:38.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.362+0000 7fe7c8ff9640 10 client.14958 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:48:38.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.362+0000 7fe7c8ff9640 1 -- 172.21.15.103:0/2822444818 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55986426e600 con 0x559864316620 2024-09-19T06:48:38.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.362+0000 7fe7c8ff9640 10 client.14958 handle_command_reply: tid=0 2024-09-19T06:48:38.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.362+0000 7fe7ed924740 1 client.14958 shutdown 2024-09-19T06:48:38.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.362+0000 7fe7ed924740 1 -- 172.21.15.103:0/2822444818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fb2f0 msgr2=0x5598643026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.362+0000 7fe7ed924740 1 --2- 172.21.15.103:0/2822444818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fb2f0 0x5598643026e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8000c00 tx=0x7fe7d802a420 comp rx=0 tx=0).stop 2024-09-19T06:48:38.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.363+0000 7fe7ed924740 1 -- 172.21.15.103:0/2822444818 shutdown_connections 2024-09-19T06:48:38.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.363+0000 7fe7ed924740 1 --2- 172.21.15.103:0/2822444818 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x559864316620 0x559864317a90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.363+0000 7fe7ed924740 1 --2- 172.21.15.103:0/2822444818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559864305bb0 0x559864302c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.363+0000 7fe7ed924740 1 --2- 172.21.15.103:0/2822444818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fb2f0 0x5598643026e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.363+0000 7fe7ed924740 1 --2- 172.21.15.103:0/2822444818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fa9d0 0x559864305390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.363+0000 7fe7ed924740 1 -- 172.21.15.103:0/2822444818 >> 172.21.15.103:0/2822444818 conn(0x5598642f6f60 msgr2=0x5598642fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:38.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.363+0000 7fe7c8ff9640 0 client.14958 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:48:38.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.363+0000 7fe7ed924740 1 -- 172.21.15.103:0/2822444818 shutdown_connections 2024-09-19T06:48:38.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.363+0000 7fe7ed924740 1 -- 172.21.15.103:0/2822444818 wait complete. 2024-09-19T06:48:38.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.363+0000 7fe7ed924740 20 client.14958 trim_cache size 0 max 16384 2024-09-19T06:48:38.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.380+0000 7fe7ed924740 1 Processor -- start 2024-09-19T06:48:38.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.380+0000 7fe7ed924740 1 -- start start 2024-09-19T06:48:38.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.380+0000 7fe7ed924740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fa9d0 0x559864315f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7ed924740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fb2f0 0x5598643173c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7ed924740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559864305bb0 0x5598642f7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7ed924740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559864315dd0 con 0x5598642fa9d0 2024-09-19T06:48:38.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7ed924740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5598643164a0 con 0x5598642fb2f0 2024-09-19T06:48:38.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7ed924740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559864316b70 con 0x559864305bb0 2024-09-19T06:48:38.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7e92e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fa9d0 0x559864315f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7e92e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fa9d0 0x559864315f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55110/0 (socket says 172.21.15.103:55110) 2024-09-19T06:48:38.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7e92e7640 1 -- 172.21.15.103:0/1751865524 learned_addr learned my addr 172.21.15.103:0/1751865524 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:38.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7e92e7640 1 -- 172.21.15.103:0/1751865524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559864305bb0 msgr2=0x5598642f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:38.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7e92e7640 1 --2- 172.21.15.103:0/1751865524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559864305bb0 0x5598642f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7e92e7640 1 -- 172.21.15.103:0/1751865524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fb2f0 msgr2=0x5598643173c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:38.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7e92e7640 1 --2- 172.21.15.103:0/1751865524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fb2f0 0x5598643173c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7e92e7640 1 -- 172.21.15.103:0/1751865524 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598642f6d90 con 0x5598642fa9d0 2024-09-19T06:48:38.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7e92e7640 1 --2- 172.21.15.103:0/1751865524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fa9d0 0x559864315f50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00d010 tx=0x7fe7dc00c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfff26a290896640 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:38.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7caffd640 1 -- 172.21.15.103:0/1751865524 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc004020 con 0x5598642fa9d0 2024-09-19T06:48:38.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7caffd640 1 -- 172.21.15.103:0/1751865524 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7dc0041c0 con 0x5598642fa9d0 2024-09-19T06:48:38.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.381+0000 7fe7caffd640 1 -- 172.21.15.103:0/1751865524 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc01a590 con 0x5598642fa9d0 2024-09-19T06:48:38.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.382+0000 7fe7ed924740 1 -- 172.21.15.103:0/1751865524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fa9d0 msgr2=0x559864315f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.382+0000 7fe7ed924740 1 --2- 172.21.15.103:0/1751865524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fa9d0 0x559864315f50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00d010 tx=0x7fe7dc00c410 comp rx=0 tx=0).stop 2024-09-19T06:48:38.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.382+0000 7fe7ed924740 1 -- 172.21.15.103:0/1751865524 shutdown_connections 2024-09-19T06:48:38.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.382+0000 7fe7ed924740 1 --2- 172.21.15.103:0/1751865524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559864305bb0 0x5598642f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.382+0000 7fe7ed924740 1 --2- 172.21.15.103:0/1751865524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fb2f0 0x5598643173c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.382+0000 7fe7ed924740 1 --2- 172.21.15.103:0/1751865524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fa9d0 0x559864315f50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.382+0000 7fe7ed924740 1 -- 172.21.15.103:0/1751865524 >> 172.21.15.103:0/1751865524 conn(0x5598642f6f60 msgr2=0x5598642f1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:38.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.382+0000 7fe7ed924740 1 -- 172.21.15.103:0/1751865524 shutdown_connections 2024-09-19T06:48:38.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.382+0000 7fe7ed924740 1 -- 172.21.15.103:0/1751865524 wait complete. 2024-09-19T06:48:38.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.382+0000 7fe7ed924740 1 -- start start 2024-09-19T06:48:38.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.382+0000 7fe7ed924740 1 Processor -- start 2024-09-19T06:48:38.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7ed924740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fa9d0 0x559864318d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7ed924740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fb2f0 0x559864319240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7ed924740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559864305bb0 0x559864319780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7ed924740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559864317910 con 0x5598642fa9d0 2024-09-19T06:48:38.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7ed924740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559864317240 con 0x5598642fb2f0 2024-09-19T06:48:38.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7ed924740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559864317fe0 con 0x559864305bb0 2024-09-19T06:48:38.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7ed924740 10 client.0 _ll_register_callbacks cb 0x7fe7d8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:38.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7e92e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fa9d0 0x559864318d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7e92e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fa9d0 0x559864318d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55120/0 (socket says 172.21.15.103:55120) 2024-09-19T06:48:38.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7e92e7640 1 -- 172.21.15.103:0/454042171 learned_addr learned my addr 172.21.15.103:0/454042171 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:38.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7e9ae8640 1 --2- 172.21.15.103:0/454042171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559864305bb0 0x559864319780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7e8ae6640 1 --2- 172.21.15.103:0/454042171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fb2f0 0x559864319240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7e92e7640 1 -- 172.21.15.103:0/454042171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559864305bb0 msgr2=0x559864319780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:38.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7e92e7640 1 --2- 172.21.15.103:0/454042171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559864305bb0 0x559864319780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7e92e7640 1 -- 172.21.15.103:0/454042171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fb2f0 msgr2=0x559864319240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:38.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7e92e7640 1 --2- 172.21.15.103:0/454042171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fb2f0 0x559864319240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7e92e7640 1 -- 172.21.15.103:0/454042171 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55986437efc0 con 0x5598642fa9d0 2024-09-19T06:48:38.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7e8ae6640 1 --2- 172.21.15.103:0/454042171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fb2f0 0x559864319240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:38.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7e92e7640 1 --2- 172.21.15.103:0/454042171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fa9d0 0x559864318d00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0040a0 tx=0x7fe7dc0040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c34810c853711b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:38.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:48:38.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.383+0000 7fe7c8ff9640 1 -- 172.21.15.103:0/454042171 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc0037f0 con 0x5598642fa9d0 2024-09-19T06:48:38.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.384+0000 7fe7c8ff9640 1 -- 172.21.15.103:0/454042171 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe7dc003990 con 0x5598642fa9d0 2024-09-19T06:48:38.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.384+0000 7fe7ed924740 1 -- 172.21.15.103:0/454042171 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7dc0041c0 con 0x5598642fa9d0 2024-09-19T06:48:38.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.384+0000 7fe7c8ff9640 1 -- 172.21.15.103:0/454042171 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe7dc003b90 con 0x5598642fa9d0 2024-09-19T06:48:38.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.384+0000 7fe7ed924740 10 client.14964 fetch_fsmap learned FSMap version 26 2024-09-19T06:48:38.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.384+0000 7fe7ed924740 1 -- 172.21.15.103:0/454042171 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559864380280 con 0x5598642fa9d0 2024-09-19T06:48:38.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.384+0000 7fe7c8ff9640 1 -- 172.21.15.103:0/454042171 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7dc0041c0 con 0x5598642fa9d0 2024-09-19T06:48:38.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.384+0000 7fe7c8ff9640 1 -- 172.21.15.103:0/454042171 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fe7dc02c020 con 0x5598642fa9d0 2024-09-19T06:48:38.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.384+0000 7fe7ed924740 10 client.14964 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:48:38.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.384+0000 7fe7ed924740 10 client.14964 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-19T06:48:38.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.384+0000 7fe7ed924740 20 client.14964 populate_metadata read hostname 'smithi103' 2024-09-19T06:48:38.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.384+0000 7fe7ed924740 1 --2- 172.21.15.103:0/454042171 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x559864384c40 0x559864383950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.384+0000 7fe7ed924740 4 client.14964 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:48:38.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.384+0000 7fe7ed924740 1 -- 172.21.15.103:0/454042171 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "status"}) -- 0x559864268b10 con 0x559864384c40 2024-09-19T06:48:38.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.384+0000 7fe7e8ae6640 1 --2- 172.21.15.103:0/454042171 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x559864384c40 0x559864383950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7c8ff9640 10 client.14964 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:48:38.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7e8ae6640 1 --2- 172.21.15.103:0/454042171 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x559864384c40 0x559864383950 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee2452336c93e3cc server_cookie=b5bcd2932858df19 in_seq=0 out_seq=0 2024-09-19T06:48:38.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7c8ff9640 1 -- 172.21.15.103:0/454042171 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559864268b10 con 0x559864384c40 2024-09-19T06:48:38.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7c8ff9640 10 client.14964 handle_command_reply: tid=0 2024-09-19T06:48:38.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7ed924740 1 client.14964 shutdown 2024-09-19T06:48:38.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7ed924740 1 -- 172.21.15.103:0/454042171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fa9d0 msgr2=0x559864318d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7ed924740 1 --2- 172.21.15.103:0/454042171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fa9d0 0x559864318d00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0040a0 tx=0x7fe7dc0040d0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7ed924740 1 -- 172.21.15.103:0/454042171 shutdown_connections 2024-09-19T06:48:38.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7ed924740 1 --2- 172.21.15.103:0/454042171 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x559864384c40 0x559864383950 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7ed924740 1 --2- 172.21.15.103:0/454042171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559864305bb0 0x559864319780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7ed924740 1 --2- 172.21.15.103:0/454042171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598642fb2f0 0x559864319240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7ed924740 1 --2- 172.21.15.103:0/454042171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598642fa9d0 0x559864318d00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7ed924740 1 -- 172.21.15.103:0/454042171 >> 172.21.15.103:0/454042171 conn(0x5598642f6f60 msgr2=0x5598642f7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:38.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7c8ff9640 0 client.14964 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:48:38.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7ed924740 1 -- 172.21.15.103:0/454042171 shutdown_connections 2024-09-19T06:48:38.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7ed924740 1 -- 172.21.15.103:0/454042171 wait complete. 2024-09-19T06:48:38.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.385+0000 7fe7ed924740 20 client.14964 trim_cache size 0 max 16384 2024-09-19T06:48:38.393 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:48:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:48:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 4, 2024-09-19T06:48:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24431, 2024-09-19T06:48:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T06:48:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T06:48:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T06:48:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 62.487112523999997, 2024-09-19T06:48:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T06:48:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 82, 2024-09-19T06:48:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T06:48:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 102.17440712299999 2024-09-19T06:48:38.393 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:48:38.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.386+0000 7fe7eb54b640 1 -- 172.21.15.103:0/3305080229 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe7cc0a3350 msgr2=0x7fe7cc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.386+0000 7fe7eb54b640 1 --2- 172.21.15.103:0/3305080229 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe7cc0a3350 0x7fe7cc0a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009900 tx=0x7fe7d8031790 comp rx=0 tx=0).stop 2024-09-19T06:48:38.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.386+0000 7fe7eb54b640 1 -- 172.21.15.103:0/3305080229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e41082a0 msgr2=0x7fe7e419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.386+0000 7fe7eb54b640 1 --2- 172.21.15.103:0/3305080229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e41082a0 0x7fe7e419a650 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400b970 tx=0x7fe7d400be30 comp rx=0 tx=0).stop 2024-09-19T06:48:38.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.386+0000 7fe7eb54b640 1 -- 172.21.15.103:0/3305080229 shutdown_connections 2024-09-19T06:48:38.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.386+0000 7fe7eb54b640 1 --2- 172.21.15.103:0/3305080229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7e41082a0 0x7fe7e419a650 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.386+0000 7fe7eb54b640 1 --2- 172.21.15.103:0/3305080229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7e41036e0 0x7fe7e41a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.386+0000 7fe7eb54b640 1 --2- 172.21.15.103:0/3305080229 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe7cc0a3350 0x7fe7cc0a5810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.386+0000 7fe7eb54b640 1 --2- 172.21.15.103:0/3305080229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7e4102d30 0x7fe7e41a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.386+0000 7fe7eb54b640 1 -- 172.21.15.103:0/3305080229 >> 172.21.15.103:0/3305080229 conn(0x7fe7e40fc7d0 msgr2=0x7fe7e40fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:38.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.386+0000 7fe7eb54b640 1 -- 172.21.15.103:0/3305080229 shutdown_connections 2024-09-19T06:48:38.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:38.386+0000 7fe7eb54b640 1 -- 172.21.15.103:0/3305080229 wait complete. 2024-09-19T06:48:39.019 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:38 smithi103 ceph-mon[31158]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T06:48:39.047 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T06:48:39.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:38 smithi134 ceph-mon[35356]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T06:48:39.291 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:39.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:38 smithi174 ceph-mon[35637]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T06:48:39.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.733+0000 7fb6d3c1c640 1 -- 172.21.15.103:0/3114680492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6cc074f40 msgr2=0x7fb6cc10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:39.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.733+0000 7fb6d3c1c640 1 --2- 172.21.15.103:0/3114680492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6cc074f40 0x7fb6cc10de80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009900 tx=0x7fb6c002a480 comp rx=0 tx=0).stop 2024-09-19T06:48:39.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.733+0000 7fb6d3c1c640 1 -- 172.21.15.103:0/3114680492 shutdown_connections 2024-09-19T06:48:39.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.733+0000 7fb6d3c1c640 1 --2- 172.21.15.103:0/3114680492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6cc10e3c0 0x7fb6cc110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.733+0000 7fb6d3c1c640 1 --2- 172.21.15.103:0/3114680492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6cc074f40 0x7fb6cc10de80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.733+0000 7fb6d3c1c640 1 --2- 172.21.15.103:0/3114680492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6cc074620 0x7fb6cc074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.733+0000 7fb6d3c1c640 1 -- 172.21.15.103:0/3114680492 >> 172.21.15.103:0/3114680492 conn(0x7fb6cc0fd8c0 msgr2=0x7fb6cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:39.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.733+0000 7fb6d3c1c640 1 -- 172.21.15.103:0/3114680492 shutdown_connections 2024-09-19T06:48:39.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.733+0000 7fb6d3c1c640 1 -- 172.21.15.103:0/3114680492 wait complete. 2024-09-19T06:48:39.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d3c1c640 1 Processor -- start 2024-09-19T06:48:39.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d3c1c640 1 -- start start 2024-09-19T06:48:39.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d3c1c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6cc074620 0x7fb6cc19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d3c1c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6cc074f40 0x7fb6cc19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d3c1c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6cc10e3c0 0x7fb6cc19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d3c1c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc114160 con 0x7fb6cc074620 2024-09-19T06:48:39.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d3c1c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc113fe0 con 0x7fb6cc10e3c0 2024-09-19T06:48:39.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d3c1c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc1142e0 con 0x7fb6cc074f40 2024-09-19T06:48:39.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d19b8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6cc074620 0x7fb6cc19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:39.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d19b8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6cc074620 0x7fb6cc19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55136/0 (socket says 172.21.15.103:55136) 2024-09-19T06:48:39.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d19b8640 1 -- 172.21.15.103:0/769153009 learned_addr learned my addr 172.21.15.103:0/769153009 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:39.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d11b7640 1 --2- 172.21.15.103:0/769153009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6cc074f40 0x7fb6cc19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:39.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d19b8640 1 -- 172.21.15.103:0/769153009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6cc074f40 msgr2=0x7fb6cc19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:39.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d19b8640 1 --2- 172.21.15.103:0/769153009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6cc074f40 0x7fb6cc19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d19b8640 1 -- 172.21.15.103:0/769153009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6cc10e3c0 msgr2=0x7fb6cc19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:39.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d21b9640 1 --2- 172.21.15.103:0/769153009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6cc10e3c0 0x7fb6cc19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:39.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d19b8640 1 --2- 172.21.15.103:0/769153009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6cc10e3c0 0x7fb6cc19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.734+0000 7fb6d19b8640 1 -- 172.21.15.103:0/769153009 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6cc19f600 con 0x7fb6cc074620 2024-09-19T06:48:39.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.735+0000 7fb6d19b8640 1 --2- 172.21.15.103:0/769153009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6cc074620 0x7fb6cc19a650 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400cce0 tx=0x7fb6c400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:39.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.735+0000 7fb6caffd640 1 -- 172.21.15.103:0/769153009 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4004b20 con 0x7fb6cc074620 2024-09-19T06:48:39.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.735+0000 7fb6caffd640 1 -- 172.21.15.103:0/769153009 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c4004cc0 con 0x7fb6cc074620 2024-09-19T06:48:39.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.735+0000 7fb6d3c1c640 1 -- 172.21.15.103:0/769153009 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6cc19f8f0 con 0x7fb6cc074620 2024-09-19T06:48:39.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.735+0000 7fb6caffd640 1 -- 172.21.15.103:0/769153009 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c400f7d0 con 0x7fb6cc074620 2024-09-19T06:48:39.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.735+0000 7fb6d3c1c640 1 -- 172.21.15.103:0/769153009 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6cc1a71d0 con 0x7fb6cc074620 2024-09-19T06:48:39.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.737+0000 7fb6d5ff5740 1 Processor -- start 2024-09-19T06:48:39.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.737+0000 7fb6d5ff5740 1 -- start start 2024-09-19T06:48:39.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.737+0000 7fb6caffd640 1 -- 172.21.15.103:0/769153009 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb6c400f970 con 0x7fb6cc074620 2024-09-19T06:48:39.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.737+0000 7fb6d5ff5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0da59d0 0x5615c0da5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d5ff5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da62f0 0x5615c0db0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d5ff5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0db0bb0 0x5615c0db2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d5ff5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5615c0d0c090 con 0x5615c0da62f0 2024-09-19T06:48:39.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d5ff5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5615c0cb3a70 con 0x5615c0da59d0 2024-09-19T06:48:39.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d5ff5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5615c0d08050 con 0x5615c0db0bb0 2024-09-19T06:48:39.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d19b8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da62f0 0x5615c0db0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:39.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d19b8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da62f0 0x5615c0db0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55142/0 (socket says 172.21.15.103:55142) 2024-09-19T06:48:39.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d19b8640 1 -- 172.21.15.103:0/1241833742 learned_addr learned my addr 172.21.15.103:0/1241833742 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:39.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d21b9640 1 --2- 172.21.15.103:0/1241833742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0da59d0 0x5615c0da5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:39.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d11b7640 1 --2- 172.21.15.103:0/1241833742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0db0bb0 0x5615c0db2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:39.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d19b8640 1 -- 172.21.15.103:0/1241833742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0db0bb0 msgr2=0x5615c0db2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:39.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d19b8640 1 --2- 172.21.15.103:0/1241833742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0db0bb0 0x5615c0db2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d19b8640 1 -- 172.21.15.103:0/1241833742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0da59d0 msgr2=0x5615c0da5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:39.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d19b8640 1 --2- 172.21.15.103:0/1241833742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0da59d0 0x5615c0da5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d19b8640 1 -- 172.21.15.103:0/1241833742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615c0cc0130 con 0x5615c0da62f0 2024-09-19T06:48:39.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d11b7640 1 --2- 172.21.15.103:0/1241833742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0db0bb0 0x5615c0db2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:39.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6d19b8640 1 --2- 172.21.15.103:0/1241833742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da62f0 0x5615c0db0670 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40cd490 tx=0x7fb6c40d86f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f08dcb7a1826716 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:39.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.738+0000 7fb6caffd640 1 --2- 172.21.15.103:0/769153009 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb6b40a3350 0x7fb6b40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.739+0000 7fb6d21b9640 1 --2- 172.21.15.103:0/769153009 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb6b40a3350 0x7fb6b40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:39.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.739+0000 7fb6b3fff640 1 -- 172.21.15.103:0/1241833742 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40de070 con 0x5615c0da62f0 2024-09-19T06:48:39.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.739+0000 7fb6caffd640 1 -- 172.21.15.103:0/769153009 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb6c40cfb90 con 0x7fb6cc074620 2024-09-19T06:48:39.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.739+0000 7fb6b3fff640 1 -- 172.21.15.103:0/1241833742 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c409c760 con 0x5615c0da62f0 2024-09-19T06:48:39.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.739+0000 7fb6b3fff640 1 -- 172.21.15.103:0/1241833742 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c409be30 con 0x5615c0da62f0 2024-09-19T06:48:39.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.739+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/1241833742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da62f0 msgr2=0x5615c0db0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:39.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.739+0000 7fb6d21b9640 1 --2- 172.21.15.103:0/769153009 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb6b40a3350 0x7fb6b40a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009b20 tx=0x7fb6bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:39.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.739+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/1241833742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da62f0 0x5615c0db0670 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40cd490 tx=0x7fb6c40d86f0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.739+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/1241833742 shutdown_connections 2024-09-19T06:48:39.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.739+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/1241833742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0db0bb0 0x5615c0db2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.739+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/1241833742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da62f0 0x5615c0db0670 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.739+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/1241833742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0da59d0 0x5615c0da5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.739+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/1241833742 >> 172.21.15.103:0/1241833742 conn(0x5615c0da1f60 msgr2=0x5615c0da2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:39.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.740+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/1241833742 shutdown_connections 2024-09-19T06:48:39.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.740+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/1241833742 wait complete. 2024-09-19T06:48:39.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.740+0000 7fb6d5ff5740 1 -- start start 2024-09-19T06:48:39.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.740+0000 7fb6d5ff5740 1 Processor -- start 2024-09-19T06:48:39.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d5ff5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0db0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d5ff5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0da62f0 0x5615c0dad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d5ff5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0db0bb0 0x5615c0dadd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d5ff5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5615c0db3780 con 0x5615c0da59d0 2024-09-19T06:48:39.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d5ff5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5615c0cc6f20 con 0x5615c0db0bb0 2024-09-19T06:48:39.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d5ff5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5615c0db3900 con 0x5615c0da62f0 2024-09-19T06:48:39.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d5ff5740 10 client.0 _ll_register_callbacks cb 0x5615c0c90030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:39.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d19b8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0db0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:39.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d19b8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0db0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55152/0 (socket says 172.21.15.103:55152) 2024-09-19T06:48:39.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d19b8640 1 -- 172.21.15.103:0/4254097343 learned_addr learned my addr 172.21.15.103:0/4254097343 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:39.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d11b7640 1 --2- 172.21.15.103:0/4254097343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0da62f0 0x5615c0dad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:39.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d19b8640 1 -- 172.21.15.103:0/4254097343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0da62f0 msgr2=0x5615c0dad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:39.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d19b8640 1 --2- 172.21.15.103:0/4254097343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0da62f0 0x5615c0dad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d19b8640 1 -- 172.21.15.103:0/4254097343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0db0bb0 msgr2=0x5615c0dadd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:39.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d19b8640 1 --2- 172.21.15.103:0/4254097343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0db0bb0 0x5615c0dadd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d19b8640 1 -- 172.21.15.103:0/4254097343 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615c0a7d2b0 con 0x5615c0da59d0 2024-09-19T06:48:39.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d11b7640 1 --2- 172.21.15.103:0/4254097343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0da62f0 0x5615c0dad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:39.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d19b8640 1 --2- 172.21.15.103:0/4254097343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0db0480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4098a30 tx=0x7fb6c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8102383f6f06ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:39.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:48:39.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6b17fa640 1 -- 172.21.15.103:0/4254097343 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c401d030 con 0x5615c0da59d0 2024-09-19T06:48:39.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6b17fa640 1 -- 172.21.15.103:0/4254097343 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb6c40bc070 con 0x5615c0da59d0 2024-09-19T06:48:39.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/4254097343 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c409c760 con 0x5615c0da59d0 2024-09-19T06:48:39.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6b17fa640 1 -- 172.21.15.103:0/4254097343 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb6c40de070 con 0x5615c0da59d0 2024-09-19T06:48:39.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d5ff5740 10 client.14976 fetch_fsmap learned FSMap version 26 2024-09-19T06:48:39.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.741+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/4254097343 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5615c0dbf0d0 con 0x5615c0da59d0 2024-09-19T06:48:39.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.742+0000 7fb6b17fa640 1 -- 172.21.15.103:0/4254097343 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c409c760 con 0x5615c0da59d0 2024-09-19T06:48:39.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.742+0000 7fb6b17fa640 1 -- 172.21.15.103:0/4254097343 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fb6c400fc60 con 0x5615c0da59d0 2024-09-19T06:48:39.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.742+0000 7fb6d5ff5740 10 client.14976 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:48:39.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.742+0000 7fb6d5ff5740 10 client.14976 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T06:48:39.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.742+0000 7fb6d5ff5740 20 client.14976 populate_metadata read hostname 'smithi103' 2024-09-19T06:48:39.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.742+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/4254097343 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5615c0dc0420 0x5615c0dc25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.742+0000 7fb6d5ff5740 4 client.14976 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:48:39.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.742+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/4254097343 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615c0d19600 con 0x5615c0dc0420 2024-09-19T06:48:39.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.742+0000 7fb6d11b7640 1 --2- 172.21.15.103:0/4254097343 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5615c0dc0420 0x5615c0dc25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:39.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.743+0000 7fb6b17fa640 10 client.14976 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:48:39.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.743+0000 7fb6d11b7640 1 --2- 172.21.15.103:0/4254097343 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5615c0dc0420 0x5615c0dc25a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=14ac11458693e572 server_cookie=c8e5544aec701d3d in_seq=0 out_seq=0 2024-09-19T06:48:39.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.743+0000 7fb6b17fa640 1 -- 172.21.15.103:0/4254097343 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615c0d19600 con 0x5615c0dc0420 2024-09-19T06:48:39.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.743+0000 7fb6b17fa640 10 client.14976 handle_command_reply: tid=0 2024-09-19T06:48:39.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.744+0000 7fb6d5ff5740 1 client.14976 shutdown 2024-09-19T06:48:39.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.744+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/4254097343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 msgr2=0x5615c0db0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:39.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.744+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/4254097343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0db0480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4098a30 tx=0x7fb6c4004a40 comp rx=0 tx=0).stop 2024-09-19T06:48:39.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.744+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/4254097343 shutdown_connections 2024-09-19T06:48:39.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.744+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/4254097343 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5615c0dc0420 0x5615c0dc25a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.744+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/4254097343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0db0bb0 0x5615c0dadd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.744+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/4254097343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0da62f0 0x5615c0dad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.744+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/4254097343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0db0480 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.744+0000 7fb6b17fa640 0 client.14976 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:48:39.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.744+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/4254097343 >> 172.21.15.103:0/4254097343 conn(0x5615c0da1f60 msgr2=0x5615c0da7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:39.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.744+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/4254097343 shutdown_connections 2024-09-19T06:48:39.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.744+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/4254097343 wait complete. 2024-09-19T06:48:39.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.744+0000 7fb6d5ff5740 20 client.14976 trim_cache size 0 max 16384 2024-09-19T06:48:39.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.761+0000 7fb6d5ff5740 1 Processor -- start 2024-09-19T06:48:39.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.761+0000 7fb6d5ff5740 1 -- start start 2024-09-19T06:48:39.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6d5ff5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0dc0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6d5ff5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0da62f0 0x5615c0dc1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6d5ff5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0db0bb0 0x5615c0da0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6d5ff5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5615c0dc08e0 con 0x5615c0da59d0 2024-09-19T06:48:39.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6d5ff5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5615c0dc0fb0 con 0x5615c0db0bb0 2024-09-19T06:48:39.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6d5ff5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5615c0dc1680 con 0x5615c0da62f0 2024-09-19T06:48:39.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6d19b8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0dc0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:39.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6d19b8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0dc0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55166/0 (socket says 172.21.15.103:55166) 2024-09-19T06:48:39.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6d19b8640 1 -- 172.21.15.103:0/1039658027 learned_addr learned my addr 172.21.15.103:0/1039658027 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:39.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6d19b8640 1 -- 172.21.15.103:0/1039658027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0da62f0 msgr2=0x5615c0dc1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:39.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6d19b8640 1 --2- 172.21.15.103:0/1039658027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0da62f0 0x5615c0dc1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6d19b8640 1 -- 172.21.15.103:0/1039658027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0db0bb0 msgr2=0x5615c0da0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:39.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6d19b8640 1 --2- 172.21.15.103:0/1039658027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0db0bb0 0x5615c0da0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6d19b8640 1 -- 172.21.15.103:0/1039658027 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615c0db3890 con 0x5615c0da59d0 2024-09-19T06:48:39.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6d19b8640 1 --2- 172.21.15.103:0/1039658027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0dc0a60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40984c0 tx=0x7fb6c4098430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478bfd295107096 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:39.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6b37fe640 1 -- 172.21.15.103:0/1039658027 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c401d050 con 0x5615c0da59d0 2024-09-19T06:48:39.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6b37fe640 1 -- 172.21.15.103:0/1039658027 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c40d8bf0 con 0x5615c0da59d0 2024-09-19T06:48:39.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.762+0000 7fb6b37fe640 1 -- 172.21.15.103:0/1039658027 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c409b9c0 con 0x5615c0da59d0 2024-09-19T06:48:39.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.763+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/1039658027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 msgr2=0x5615c0dc0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:39.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.763+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/1039658027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0dc0a60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40984c0 tx=0x7fb6c4098430 comp rx=0 tx=0).stop 2024-09-19T06:48:39.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.763+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/1039658027 shutdown_connections 2024-09-19T06:48:39.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.763+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/1039658027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0db0bb0 0x5615c0da0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.763+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/1039658027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0da62f0 0x5615c0dc1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.763+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/1039658027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0dc0a60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.763+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/1039658027 >> 172.21.15.103:0/1039658027 conn(0x5615c0da1f60 msgr2=0x5615c0da07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:39.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.763+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/1039658027 shutdown_connections 2024-09-19T06:48:39.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.763+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/1039658027 wait complete. 2024-09-19T06:48:39.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.763+0000 7fb6d5ff5740 1 -- start start 2024-09-19T06:48:39.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d5ff5740 1 Processor -- start 2024-09-19T06:48:39.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d5ff5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0dcaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d5ff5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0da62f0 0x5615c0dcaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d5ff5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0db0bb0 0x5615c0dc5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d5ff5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5615c0dc2420 con 0x5615c0da59d0 2024-09-19T06:48:39.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d5ff5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5615c0dc1d50 con 0x5615c0da62f0 2024-09-19T06:48:39.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d5ff5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5615c0dc2af0 con 0x5615c0db0bb0 2024-09-19T06:48:39.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d5ff5740 10 client.0 _ll_register_callbacks cb 0x7fb6bc005eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:39.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d19b8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0dcaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:39.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d19b8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0dcaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55174/0 (socket says 172.21.15.103:55174) 2024-09-19T06:48:39.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d19b8640 1 -- 172.21.15.103:0/4097257781 learned_addr learned my addr 172.21.15.103:0/4097257781 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:39.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d19b8640 1 -- 172.21.15.103:0/4097257781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0db0bb0 msgr2=0x5615c0dc5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:39.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d19b8640 1 --2- 172.21.15.103:0/4097257781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0db0bb0 0x5615c0dc5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d19b8640 1 -- 172.21.15.103:0/4097257781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0da62f0 msgr2=0x5615c0dcaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:39.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d11b7640 1 --2- 172.21.15.103:0/4097257781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0da62f0 0x5615c0dcaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:39.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d19b8640 1 --2- 172.21.15.103:0/4097257781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0da62f0 0x5615c0dcaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.764+0000 7fb6d19b8640 1 -- 172.21.15.103:0/4097257781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615c0dc65e0 con 0x5615c0da59d0 2024-09-19T06:48:39.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.765+0000 7fb6d19b8640 1 --2- 172.21.15.103:0/4097257781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0dcaa00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400ab20 tx=0x7fb6c4005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59c3216cccab511 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:39.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.765+0000 7fb6b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:48:39.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.765+0000 7fb6b17fa640 1 -- 172.21.15.103:0/4097257781 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c401d070 con 0x5615c0da59d0 2024-09-19T06:48:39.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.765+0000 7fb6b17fa640 1 -- 172.21.15.103:0/4097257781 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb6c40a4840 con 0x5615c0da59d0 2024-09-19T06:48:39.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.765+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/4097257781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c40d8bf0 con 0x5615c0da59d0 2024-09-19T06:48:39.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.765+0000 7fb6b17fa640 1 -- 172.21.15.103:0/4097257781 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb6c409bba0 con 0x5615c0da59d0 2024-09-19T06:48:39.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.765+0000 7fb6d5ff5740 10 client.14982 fetch_fsmap learned FSMap version 26 2024-09-19T06:48:39.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.765+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/4097257781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5615c0dc65e0 con 0x5615c0da59d0 2024-09-19T06:48:39.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.765+0000 7fb6b17fa640 1 -- 172.21.15.103:0/4097257781 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c40d8bf0 con 0x5615c0da59d0 2024-09-19T06:48:39.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.765+0000 7fb6b17fa640 1 -- 172.21.15.103:0/4097257781 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fb6c4096bd0 con 0x5615c0da59d0 2024-09-19T06:48:39.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.765+0000 7fb6d5ff5740 10 client.14982 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:48:39.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.765+0000 7fb6d5ff5740 10 client.14982 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T06:48:39.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.765+0000 7fb6d5ff5740 20 client.14982 populate_metadata read hostname 'smithi103' 2024-09-19T06:48:39.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.765+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/4097257781 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5615c0e2f750 0x5615c0e2e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:39.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.765+0000 7fb6d5ff5740 4 client.14982 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:48:39.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.765+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/4097257781 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "status"}) -- 0x5615c0d13b10 con 0x5615c0e2f750 2024-09-19T06:48:39.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.766+0000 7fb6d11b7640 1 --2- 172.21.15.103:0/4097257781 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5615c0e2f750 0x5615c0e2e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:39.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.766+0000 7fb6b17fa640 10 client.14982 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:48:39.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.766+0000 7fb6d11b7640 1 --2- 172.21.15.103:0/4097257781 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5615c0e2f750 0x5615c0e2e460 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a1e6334f98d185a server_cookie=fc067e2cac7554a9 in_seq=0 out_seq=0 2024-09-19T06:48:39.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.766+0000 7fb6b17fa640 1 -- 172.21.15.103:0/4097257781 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5615c0d13b10 con 0x5615c0e2f750 2024-09-19T06:48:39.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.766+0000 7fb6b17fa640 10 client.14982 handle_command_reply: tid=0 2024-09-19T06:48:39.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.766+0000 7fb6d5ff5740 1 client.14982 shutdown 2024-09-19T06:48:39.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.766+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/4097257781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 msgr2=0x5615c0dcaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:39.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.766+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/4097257781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0dcaa00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400ab20 tx=0x7fb6c4005940 comp rx=0 tx=0).stop 2024-09-19T06:48:39.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.766+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/4097257781 shutdown_connections 2024-09-19T06:48:39.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.766+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/4097257781 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5615c0e2f750 0x5615c0e2e460 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.766+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/4097257781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615c0db0bb0 0x5615c0dc5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.766+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/4097257781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615c0da62f0 0x5615c0dcaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.766+0000 7fb6d5ff5740 1 --2- 172.21.15.103:0/4097257781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615c0da59d0 0x5615c0dcaa00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.766+0000 7fb6b17fa640 0 client.14982 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:48:39.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.766+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/4097257781 >> 172.21.15.103:0/4097257781 conn(0x5615c0da1f60 msgr2=0x5615c0da1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:39.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.766+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/4097257781 shutdown_connections 2024-09-19T06:48:39.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.767+0000 7fb6d5ff5740 1 -- 172.21.15.103:0/4097257781 wait complete. 2024-09-19T06:48:39.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.767+0000 7fb6d5ff5740 20 client.14982 trim_cache size 0 max 16384 2024-09-19T06:48:39.779 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:48:39.779 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:48:39.779 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 3, 2024-09-19T06:48:39.779 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24436, 2024-09-19T06:48:39.779 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T06:48:39.779 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T06:48:39.779 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T06:48:39.779 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 64.878501494999995, 2024-09-19T06:48:39.779 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T06:48:39.779 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 82, 2024-09-19T06:48:39.779 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T06:48:39.779 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 98.674517551999998 2024-09-19T06:48:39.780 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:48:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.767+0000 7fb6d3c1c640 1 -- 172.21.15.103:0/769153009 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb6b40a3350 msgr2=0x7fb6b40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.767+0000 7fb6d3c1c640 1 --2- 172.21.15.103:0/769153009 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb6b40a3350 0x7fb6b40a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009b20 tx=0x7fb6bc005f90 comp rx=0 tx=0).stop 2024-09-19T06:48:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.767+0000 7fb6d3c1c640 1 -- 172.21.15.103:0/769153009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6cc074620 msgr2=0x7fb6cc19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:39.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.767+0000 7fb6d3c1c640 1 --2- 172.21.15.103:0/769153009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6cc074620 0x7fb6cc19a650 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400cce0 tx=0x7fb6c400a620 comp rx=0 tx=0).stop 2024-09-19T06:48:39.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.767+0000 7fb6d3c1c640 1 -- 172.21.15.103:0/769153009 shutdown_connections 2024-09-19T06:48:39.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.767+0000 7fb6d3c1c640 1 --2- 172.21.15.103:0/769153009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6cc10e3c0 0x7fb6cc19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.767+0000 7fb6d3c1c640 1 --2- 172.21.15.103:0/769153009 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fb6b40a3350 0x7fb6b40a5810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.767+0000 7fb6d3c1c640 1 --2- 172.21.15.103:0/769153009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6cc074f40 0x7fb6cc19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.767+0000 7fb6d3c1c640 1 --2- 172.21.15.103:0/769153009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6cc074620 0x7fb6cc19a650 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:39.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.767+0000 7fb6d3c1c640 1 -- 172.21.15.103:0/769153009 >> 172.21.15.103:0/769153009 conn(0x7fb6cc0fd8c0 msgr2=0x7fb6cc0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:39.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.767+0000 7fb6d3c1c640 1 -- 172.21.15.103:0/769153009 shutdown_connections 2024-09-19T06:48:39.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:39.767+0000 7fb6d3c1c640 1 -- 172.21.15.103:0/769153009 wait complete. 2024-09-19T06:48:40.439 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T06:48:40.685 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:41.113 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:40 smithi103 ceph-mon[31158]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:48:41.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.116+0000 7fdefa063640 1 -- 172.21.15.103:0/2351715857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdef4101420 msgr2=0x7fdef410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:41.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.116+0000 7fdefa063640 1 --2- 172.21.15.103:0/2351715857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdef4101420 0x7fdef410d950 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009cd0 tx=0x7fdeec02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.117+0000 7fdefa063640 1 -- 172.21.15.103:0/2351715857 shutdown_connections 2024-09-19T06:48:41.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.117+0000 7fdefa063640 1 --2- 172.21.15.103:0/2351715857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdef4101420 0x7fdef410d950 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.117+0000 7fdefa063640 1 --2- 172.21.15.103:0/2351715857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdef4100a80 0x7fdef4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.117+0000 7fdefa063640 1 --2- 172.21.15.103:0/2351715857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdef4106a80 0x7fdef4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.117+0000 7fdefa063640 1 -- 172.21.15.103:0/2351715857 >> 172.21.15.103:0/2351715857 conn(0x7fdef40fc7a0 msgr2=0x7fdef40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:41.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.117+0000 7fdefa063640 1 -- 172.21.15.103:0/2351715857 shutdown_connections 2024-09-19T06:48:41.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.117+0000 7fdefa063640 1 -- 172.21.15.103:0/2351715857 wait complete. 2024-09-19T06:48:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.117+0000 7fdefa063640 1 Processor -- start 2024-09-19T06:48:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdefa063640 1 -- start start 2024-09-19T06:48:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdefa063640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdef4100a80 0x7fdef4197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdefa063640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdef4101420 0x7fdef4197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdefa063640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdef4106a80 0x7fdef419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdefa063640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdef4110e80 con 0x7fdef4100a80 2024-09-19T06:48:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdefa063640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdef4110d00 con 0x7fdef4101420 2024-09-19T06:48:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdefa063640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdef4111000 con 0x7fdef4106a80 2024-09-19T06:48:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdef37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdef4100a80 0x7fdef4197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdef37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdef4100a80 0x7fdef4197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55184/0 (socket says 172.21.15.103:55184) 2024-09-19T06:48:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdef37fe640 1 -- 172.21.15.103:0/1230159650 learned_addr learned my addr 172.21.15.103:0/1230159650 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdeebfff640 1 --2- 172.21.15.103:0/1230159650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdef4101420 0x7fdef4197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdef3fff640 1 --2- 172.21.15.103:0/1230159650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdef4106a80 0x7fdef419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdef37fe640 1 -- 172.21.15.103:0/1230159650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdef4106a80 msgr2=0x7fdef419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdef37fe640 1 --2- 172.21.15.103:0/1230159650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdef4106a80 0x7fdef419bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdef37fe640 1 -- 172.21.15.103:0/1230159650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdef4101420 msgr2=0x7fdef4197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdef37fe640 1 --2- 172.21.15.103:0/1230159650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdef4101420 0x7fdef4197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdef37fe640 1 -- 172.21.15.103:0/1230159650 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef419c2b0 con 0x7fdef4100a80 2024-09-19T06:48:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdeebfff640 1 --2- 172.21.15.103:0/1230159650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdef4101420 0x7fdef4197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdef37fe640 1 --2- 172.21.15.103:0/1230159650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdef4100a80 0x7fdef4197300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdee400e9f0 tx=0x7fdee400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdef17fa640 1 -- 172.21.15.103:0/1230159650 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee400cd80 con 0x7fdef4100a80 2024-09-19T06:48:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.118+0000 7fdefa063640 1 -- 172.21.15.103:0/1230159650 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef419c5a0 con 0x7fdef4100a80 2024-09-19T06:48:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.119+0000 7fdef17fa640 1 -- 172.21.15.103:0/1230159650 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee4010910 con 0x7fdef4100a80 2024-09-19T06:48:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.119+0000 7fdef17fa640 1 -- 172.21.15.103:0/1230159650 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee40188f0 con 0x7fdef4100a80 2024-09-19T06:48:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.119+0000 7fdefa063640 1 -- 172.21.15.103:0/1230159650 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef406a790 con 0x7fdef4100a80 2024-09-19T06:48:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.121+0000 7fdef17fa640 1 -- 172.21.15.103:0/1230159650 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fdee4010ac0 con 0x7fdef4100a80 2024-09-19T06:48:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.121+0000 7fdefc43c740 1 Processor -- start 2024-09-19T06:48:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.121+0000 7fdefc43c740 1 -- start start 2024-09-19T06:48:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdefc43c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c396c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdefc43c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3971d0 0x55d77c3a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdefc43c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3a5fc0 0x55d77c3a83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdefc43c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d77c2fd090 con 0x55d77c3971d0 2024-09-19T06:48:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdefc43c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d77c2a4a70 con 0x55d77c3968b0 2024-09-19T06:48:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdefc43c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d77c2f9050 con 0x55d77c3a5fc0 2024-09-19T06:48:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdef17fa640 1 --2- 172.21.15.103:0/1230159650 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdee00a3350 0x7fdee00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdef37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3971d0 0x55d77c3a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdef3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c396c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdef37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3971d0 0x55d77c3a5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55198/0 (socket says 172.21.15.103:55198) 2024-09-19T06:48:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdef37fe640 1 -- 172.21.15.103:0/3210896011 learned_addr learned my addr 172.21.15.103:0/3210896011 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdef3fff640 1 --2- 172.21.15.103:0/1230159650 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdee00a3350 0x7fdee00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdeebfff640 1 --2- 172.21.15.103:0/3210896011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3a5fc0 0x55d77c3a83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdef17fa640 1 -- 172.21.15.103:0/1230159650 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdee40cfdf0 con 0x7fdef4100a80 2024-09-19T06:48:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdef37fe640 1 -- 172.21.15.103:0/3210896011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3a5fc0 msgr2=0x55d77c3a83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdef37fe640 1 --2- 172.21.15.103:0/3210896011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3a5fc0 0x55d77c3a83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdef37fe640 1 -- 172.21.15.103:0/3210896011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 msgr2=0x55d77c396c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdef37fe640 1 --2- 172.21.15.103:0/3210896011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c396c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdef37fe640 1 -- 172.21.15.103:0/3210896011 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d77c2b1130 con 0x55d77c3971d0 2024-09-19T06:48:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdef3fff640 1 --2- 172.21.15.103:0/3210896011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c396c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdef37fe640 1 --2- 172.21.15.103:0/3210896011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3971d0 0x55d77c3a5a80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdee4098720 tx=0x7fdee409bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215769595e30b81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:41.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdef3fff640 1 --2- 172.21.15.103:0/1230159650 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdee00a3350 0x7fdee00a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdeec004af0 tx=0x7fdeec02a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:41.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.122+0000 7fdee9ffb640 1 -- 172.21.15.103:0/3210896011 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee40a0670 con 0x55d77c3971d0 2024-09-19T06:48:41.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.123+0000 7fdee9ffb640 1 -- 172.21.15.103:0/3210896011 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee40a4f30 con 0x55d77c3971d0 2024-09-19T06:48:41.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.123+0000 7fdee9ffb640 1 -- 172.21.15.103:0/3210896011 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee40ade90 con 0x55d77c3971d0 2024-09-19T06:48:41.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.123+0000 7fdefc43c740 1 -- 172.21.15.103:0/3210896011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3971d0 msgr2=0x55d77c3a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:41.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.123+0000 7fdefc43c740 1 --2- 172.21.15.103:0/3210896011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3971d0 0x55d77c3a5a80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdee4098720 tx=0x7fdee409bc20 comp rx=0 tx=0).stop 2024-09-19T06:48:41.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.123+0000 7fdefc43c740 1 -- 172.21.15.103:0/3210896011 shutdown_connections 2024-09-19T06:48:41.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.123+0000 7fdefc43c740 1 --2- 172.21.15.103:0/3210896011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3a5fc0 0x55d77c3a83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.123+0000 7fdefc43c740 1 --2- 172.21.15.103:0/3210896011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3971d0 0x55d77c3a5a80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.123+0000 7fdefc43c740 1 --2- 172.21.15.103:0/3210896011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c396c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.123+0000 7fdefc43c740 1 -- 172.21.15.103:0/3210896011 >> 172.21.15.103:0/3210896011 conn(0x55d77c392f60 msgr2=0x55d77c393340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.123+0000 7fdefc43c740 1 -- 172.21.15.103:0/3210896011 shutdown_connections 2024-09-19T06:48:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.123+0000 7fdefc43c740 1 -- 172.21.15.103:0/3210896011 wait complete. 2024-09-19T06:48:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.124+0000 7fdefc43c740 1 -- start start 2024-09-19T06:48:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.124+0000 7fdefc43c740 1 Processor -- start 2024-09-19T06:48:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.124+0000 7fdefc43c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c3a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.124+0000 7fdefc43c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3971d0 0x55d77c3a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.124+0000 7fdefc43c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3a5fc0 0x55d77c3a3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.124+0000 7fdefc43c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d77c3a8b90 con 0x55d77c3971d0 2024-09-19T06:48:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.124+0000 7fdefc43c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d77c2b7f20 con 0x55d77c3968b0 2024-09-19T06:48:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.124+0000 7fdefc43c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d77c3a8d10 con 0x55d77c3a5fc0 2024-09-19T06:48:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.124+0000 7fdefc43c740 10 client.0 _ll_register_callbacks cb 0x55d77c281030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.124+0000 7fdeebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3971d0 0x55d77c3a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fdeebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3971d0 0x55d77c3a2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55212/0 (socket says 172.21.15.103:55212) 2024-09-19T06:48:41.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fdeebfff640 1 -- 172.21.15.103:0/1272002074 learned_addr learned my addr 172.21.15.103:0/1272002074 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:41.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fdef37fe640 1 --2- 172.21.15.103:0/1272002074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c3a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fdef3fff640 1 --2- 172.21.15.103:0/1272002074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3a5fc0 0x55d77c3a3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fdeebfff640 1 -- 172.21.15.103:0/1272002074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3a5fc0 msgr2=0x55d77c3a3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:41.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fdeebfff640 1 --2- 172.21.15.103:0/1272002074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3a5fc0 0x55d77c3a3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fdeebfff640 1 -- 172.21.15.103:0/1272002074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 msgr2=0x55d77c3a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:41.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fdeebfff640 1 --2- 172.21.15.103:0/1272002074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c3a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fdeebfff640 1 -- 172.21.15.103:0/1272002074 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d77c06e2b0 con 0x55d77c3971d0 2024-09-19T06:48:41.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fdef37fe640 1 --2- 172.21.15.103:0/1272002074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c3a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:41.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fdef3fff640 1 --2- 172.21.15.103:0/1272002074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3a5fc0 0x55d77c3a3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:41.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fdeebfff640 1 --2- 172.21.15.103:0/1272002074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3971d0 0x55d77c3a2c20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdedc004410 tx=0x7fdedc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f1a5d933b8a7ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:41.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fded2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:48:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fded2ffd640 1 -- 172.21.15.103:0/1272002074 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc011710 con 0x55d77c3971d0 2024-09-19T06:48:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fded2ffd640 1 -- 172.21.15.103:0/1272002074 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdedc0118b0 con 0x55d77c3971d0 2024-09-19T06:48:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fdefc43c740 1 -- 172.21.15.103:0/1272002074 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdee40a4f30 con 0x55d77c3971d0 2024-09-19T06:48:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fded2ffd640 1 -- 172.21.15.103:0/1272002074 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdedc011ab0 con 0x55d77c3971d0 2024-09-19T06:48:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fdefc43c740 10 client.14994 fetch_fsmap learned FSMap version 26 2024-09-19T06:48:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fdefc43c740 1 -- 172.21.15.103:0/1272002074 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d77c3b45b0 con 0x55d77c3971d0 2024-09-19T06:48:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.125+0000 7fded2ffd640 1 -- 172.21.15.103:0/1272002074 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee40a4f30 con 0x55d77c3971d0 2024-09-19T06:48:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.126+0000 7fded2ffd640 1 -- 172.21.15.103:0/1272002074 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fdedc010440 con 0x55d77c3971d0 2024-09-19T06:48:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.126+0000 7fdefc43c740 10 client.14994 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:48:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.126+0000 7fdefc43c740 10 client.14994 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T06:48:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.126+0000 7fdefc43c740 20 client.14994 populate_metadata read hostname 'smithi103' 2024-09-19T06:48:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.126+0000 7fdefc43c740 1 --2- 172.21.15.103:0/1272002074 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55d77c3b6c70 0x55d77c3b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.126+0000 7fdefc43c740 4 client.14994 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:48:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.126+0000 7fdefc43c740 1 -- 172.21.15.103:0/1272002074 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d77c30a600 con 0x55d77c3b6c70 2024-09-19T06:48:41.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.126+0000 7fdef37fe640 1 --2- 172.21.15.103:0/1272002074 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55d77c3b6c70 0x55d77c3b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.126+0000 7fdef37fe640 1 --2- 172.21.15.103:0/1272002074 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55d77c3b6c70 0x55d77c3b80e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7b92fab9eeff7d7 server_cookie=cbc582c87907c8f7 in_seq=0 out_seq=0 2024-09-19T06:48:41.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.126+0000 7fded2ffd640 10 client.14994 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:48:41.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.127+0000 7fded2ffd640 1 -- 172.21.15.103:0/1272002074 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fdee40abe80 con 0x55d77c3b6c70 2024-09-19T06:48:41.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.127+0000 7fded2ffd640 10 client.14994 handle_command_reply: tid=0 2024-09-19T06:48:41.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.127+0000 7fdefc43c740 1 client.14994 shutdown 2024-09-19T06:48:41.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.127+0000 7fdefc43c740 1 -- 172.21.15.103:0/1272002074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3971d0 msgr2=0x55d77c3a2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:41.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.127+0000 7fdefc43c740 1 --2- 172.21.15.103:0/1272002074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3971d0 0x55d77c3a2c20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdedc004410 tx=0x7fdedc0039c0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.127+0000 7fdefc43c740 1 -- 172.21.15.103:0/1272002074 shutdown_connections 2024-09-19T06:48:41.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.127+0000 7fdefc43c740 1 --2- 172.21.15.103:0/1272002074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3a5fc0 0x55d77c3a3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.127+0000 7fdefc43c740 1 --2- 172.21.15.103:0/1272002074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3971d0 0x55d77c3a2c20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.127+0000 7fdefc43c740 1 --2- 172.21.15.103:0/1272002074 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55d77c3b6c70 0x55d77c3b80e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.127+0000 7fdefc43c740 1 --2- 172.21.15.103:0/1272002074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c3a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.127+0000 7fdefc43c740 1 -- 172.21.15.103:0/1272002074 >> 172.21.15.103:0/1272002074 conn(0x55d77c392f60 msgr2=0x55d77c398cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:41.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.127+0000 7fded2ffd640 0 client.14994 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:48:41.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.128+0000 7fdefc43c740 1 -- 172.21.15.103:0/1272002074 shutdown_connections 2024-09-19T06:48:41.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.128+0000 7fdefc43c740 1 -- 172.21.15.103:0/1272002074 wait complete. 2024-09-19T06:48:41.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.128+0000 7fdefc43c740 20 client.14994 trim_cache size 0 max 16384 2024-09-19T06:48:41.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.145+0000 7fdefc43c740 1 Processor -- start 2024-09-19T06:48:41.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.145+0000 7fdefc43c740 1 -- start start 2024-09-19T06:48:41.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.145+0000 7fdefc43c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c3b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.145+0000 7fdefc43c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3971d0 0x55d77c3b7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.145+0000 7fdefc43c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3a5fc0 0x55d77c39fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.145+0000 7fdefc43c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d77c3b6420 con 0x55d77c3a5fc0 2024-09-19T06:48:41.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.145+0000 7fdefc43c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d77c3b6af0 con 0x55d77c3968b0 2024-09-19T06:48:41.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.145+0000 7fdefc43c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d77c3b71c0 con 0x55d77c3971d0 2024-09-19T06:48:41.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.146+0000 7fdef3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3a5fc0 0x55d77c39fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.146+0000 7fdef37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c3b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.146+0000 7fdef3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3a5fc0 0x55d77c39fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55214/0 (socket says 172.21.15.103:55214) 2024-09-19T06:48:41.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.146+0000 7fdef3fff640 1 -- 172.21.15.103:0/3582610987 learned_addr learned my addr 172.21.15.103:0/3582610987 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:41.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.146+0000 7fdeebfff640 1 --2- 172.21.15.103:0/3582610987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3971d0 0x55d77c3b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.146+0000 7fdef3fff640 1 -- 172.21.15.103:0/3582610987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3971d0 msgr2=0x55d77c3b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:41.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.146+0000 7fdef3fff640 1 --2- 172.21.15.103:0/3582610987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3971d0 0x55d77c3b7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.146+0000 7fdef3fff640 1 -- 172.21.15.103:0/3582610987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 msgr2=0x55d77c3b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:41.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.146+0000 7fdef3fff640 1 --2- 172.21.15.103:0/3582610987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c3b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.146+0000 7fdef3fff640 1 -- 172.21.15.103:0/3582610987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d77c392a00 con 0x55d77c3a5fc0 2024-09-19T06:48:41.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.146+0000 7fdef37fe640 1 --2- 172.21.15.103:0/3582610987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c3b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:41.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.146+0000 7fdef3fff640 1 --2- 172.21.15.103:0/3582610987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3a5fc0 0x55d77c39fe80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009cd0 tx=0x7fdeec042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896c123a0694939 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:41.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.146+0000 7fdee97fa640 1 -- 172.21.15.103:0/3582610987 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec003d40 con 0x55d77c3a5fc0 2024-09-19T06:48:41.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.146+0000 7fdee97fa640 1 -- 172.21.15.103:0/3582610987 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeec036040 con 0x55d77c3a5fc0 2024-09-19T06:48:41.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.146+0000 7fdee97fa640 1 -- 172.21.15.103:0/3582610987 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec030ce0 con 0x55d77c3a5fc0 2024-09-19T06:48:41.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.147+0000 7fdefc43c740 1 -- 172.21.15.103:0/3582610987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3a5fc0 msgr2=0x55d77c39fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:41.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.147+0000 7fdefc43c740 1 --2- 172.21.15.103:0/3582610987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3a5fc0 0x55d77c39fe80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009cd0 tx=0x7fdeec042000 comp rx=0 tx=0).stop 2024-09-19T06:48:41.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.147+0000 7fdefc43c740 1 -- 172.21.15.103:0/3582610987 shutdown_connections 2024-09-19T06:48:41.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.147+0000 7fdefc43c740 1 --2- 172.21.15.103:0/3582610987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3a5fc0 0x55d77c39fe80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.147+0000 7fdefc43c740 1 --2- 172.21.15.103:0/3582610987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3971d0 0x55d77c3b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.147+0000 7fdefc43c740 1 --2- 172.21.15.103:0/3582610987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c3b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.147+0000 7fdefc43c740 1 -- 172.21.15.103:0/3582610987 >> 172.21.15.103:0/3582610987 conn(0x55d77c392f60 msgr2=0x55d77c3a1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:41.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.147+0000 7fdefc43c740 1 -- 172.21.15.103:0/3582610987 shutdown_connections 2024-09-19T06:48:41.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.147+0000 7fdefc43c740 1 -- 172.21.15.103:0/3582610987 wait complete. 2024-09-19T06:48:41.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.147+0000 7fdefc43c740 1 -- start start 2024-09-19T06:48:41.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdefc43c740 1 Processor -- start 2024-09-19T06:48:41.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdefc43c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c3befd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdefc43c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3971d0 0x55d77c3bf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdefc43c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3a5fc0 0x55d77c41f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdefc43c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d77c3b7f60 con 0x55d77c3a5fc0 2024-09-19T06:48:41.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdefc43c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d77c3b7890 con 0x55d77c3968b0 2024-09-19T06:48:41.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdefc43c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d77c3b8630 con 0x55d77c3971d0 2024-09-19T06:48:41.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdefc43c740 10 client.0 _ll_register_callbacks cb 0x7fdedc00bc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:41.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdef3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3a5fc0 0x55d77c41f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdef3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3a5fc0 0x55d77c41f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55216/0 (socket says 172.21.15.103:55216) 2024-09-19T06:48:41.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdef3fff640 1 -- 172.21.15.103:0/1284128795 learned_addr learned my addr 172.21.15.103:0/1284128795 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:41.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdef37fe640 1 --2- 172.21.15.103:0/1284128795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c3befd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdeebfff640 1 --2- 172.21.15.103:0/1284128795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3971d0 0x55d77c3bf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdef3fff640 1 -- 172.21.15.103:0/1284128795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3971d0 msgr2=0x55d77c3bf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdef3fff640 1 --2- 172.21.15.103:0/1284128795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3971d0 0x55d77c3bf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdef3fff640 1 -- 172.21.15.103:0/1284128795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 msgr2=0x55d77c3befd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdef3fff640 1 --2- 172.21.15.103:0/1284128795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c3befd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdef3fff640 1 -- 172.21.15.103:0/1284128795 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d77c41f8b0 con 0x55d77c3a5fc0 2024-09-19T06:48:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdeebfff640 1 --2- 172.21.15.103:0/1284128795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3971d0 0x55d77c3bf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdef37fe640 1 --2- 172.21.15.103:0/1284128795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c3befd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fdef3fff640 1 --2- 172.21.15.103:0/1284128795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3a5fc0 0x55d77c41f370 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009a60 tx=0x7fdeec003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef8fc18bcfccd5f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.148+0000 7fded2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:48:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.149+0000 7fded2ffd640 1 -- 172.21.15.103:0/1284128795 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec061060 con 0x55d77c3a5fc0 2024-09-19T06:48:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.149+0000 7fded2ffd640 1 -- 172.21.15.103:0/1284128795 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdeec0307b0 con 0x55d77c3a5fc0 2024-09-19T06:48:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.149+0000 7fdefc43c740 1 -- 172.21.15.103:0/1284128795 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeec036040 con 0x55d77c3a5fc0 2024-09-19T06:48:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.149+0000 7fded2ffd640 1 -- 172.21.15.103:0/1284128795 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdeec004840 con 0x55d77c3a5fc0 2024-09-19T06:48:41.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.149+0000 7fdefc43c740 10 client.15000 fetch_fsmap learned FSMap version 26 2024-09-19T06:48:41.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.149+0000 7fdefc43c740 1 -- 172.21.15.103:0/1284128795 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d77c420a70 con 0x55d77c3a5fc0 2024-09-19T06:48:41.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.149+0000 7fded2ffd640 1 -- 172.21.15.103:0/1284128795 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeec036040 con 0x55d77c3a5fc0 2024-09-19T06:48:41.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.149+0000 7fded2ffd640 1 -- 172.21.15.103:0/1284128795 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fdeec047040 con 0x55d77c3a5fc0 2024-09-19T06:48:41.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.149+0000 7fdefc43c740 10 client.15000 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:48:41.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.149+0000 7fdefc43c740 10 client.15000 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T06:48:41.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.149+0000 7fdefc43c740 20 client.15000 populate_metadata read hostname 'smithi103' 2024-09-19T06:48:41.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.149+0000 7fdefc43c740 1 --2- 172.21.15.103:0/1284128795 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55d77c4252f0 0x55d77c424000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:41.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.149+0000 7fdefc43c740 4 client.15000 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:48:41.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.149+0000 7fdefc43c740 1 -- 172.21.15.103:0/1284128795 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "status"}) -- 0x55d77c304b10 con 0x55d77c4252f0 2024-09-19T06:48:41.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.149+0000 7fdef37fe640 1 --2- 172.21.15.103:0/1284128795 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55d77c4252f0 0x55d77c424000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:41.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fded2ffd640 10 client.15000 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:48:41.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fdef37fe640 1 --2- 172.21.15.103:0/1284128795 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55d77c4252f0 0x55d77c424000 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=574ca7f1f61ac219 server_cookie=992cfbd3cbe0aff2 in_seq=0 out_seq=0 2024-09-19T06:48:41.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fded2ffd640 1 -- 172.21.15.103:0/1284128795 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d77c304b10 con 0x55d77c4252f0 2024-09-19T06:48:41.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fded2ffd640 10 client.15000 handle_command_reply: tid=0 2024-09-19T06:48:41.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fdefc43c740 1 client.15000 shutdown 2024-09-19T06:48:41.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fdefc43c740 1 -- 172.21.15.103:0/1284128795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3a5fc0 msgr2=0x55d77c41f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:41.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fdefc43c740 1 --2- 172.21.15.103:0/1284128795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3a5fc0 0x55d77c41f370 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009a60 tx=0x7fdeec003f50 comp rx=0 tx=0).stop 2024-09-19T06:48:41.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fdefc43c740 1 -- 172.21.15.103:0/1284128795 shutdown_connections 2024-09-19T06:48:41.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fdefc43c740 1 --2- 172.21.15.103:0/1284128795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d77c3a5fc0 0x55d77c41f370 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fdefc43c740 1 --2- 172.21.15.103:0/1284128795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d77c3971d0 0x55d77c3bf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fdefc43c740 1 --2- 172.21.15.103:0/1284128795 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55d77c4252f0 0x55d77c424000 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fdefc43c740 1 --2- 172.21.15.103:0/1284128795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d77c3968b0 0x55d77c3befd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fded2ffd640 0 client.15000 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:48:41.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fdefc43c740 1 -- 172.21.15.103:0/1284128795 >> 172.21.15.103:0/1284128795 conn(0x55d77c392f60 msgr2=0x55d77c3a1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:41.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fdefc43c740 1 -- 172.21.15.103:0/1284128795 shutdown_connections 2024-09-19T06:48:41.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fdefc43c740 1 -- 172.21.15.103:0/1284128795 wait complete. 2024-09-19T06:48:41.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.150+0000 7fdefc43c740 20 client.15000 trim_cache size 0 max 16384 2024-09-19T06:48:41.159 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:48:41.159 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:48:41.159 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 2, 2024-09-19T06:48:41.159 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24454, 2024-09-19T06:48:41.159 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T06:48:41.159 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T06:48:41.159 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T06:48:41.159 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 67.271541338000006, 2024-09-19T06:48:41.159 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T06:48:41.160 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 82, 2024-09-19T06:48:41.160 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T06:48:41.160 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 94.888937999000007 2024-09-19T06:48:41.160 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:48:41.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.151+0000 7fdefa063640 1 -- 172.21.15.103:0/1230159650 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdee00a3350 msgr2=0x7fdee00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:41.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.151+0000 7fdefa063640 1 --2- 172.21.15.103:0/1230159650 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdee00a3350 0x7fdee00a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdeec004af0 tx=0x7fdeec02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.151+0000 7fdefa063640 1 -- 172.21.15.103:0/1230159650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdef4100a80 msgr2=0x7fdef4197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:41.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.151+0000 7fdefa063640 1 --2- 172.21.15.103:0/1230159650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdef4100a80 0x7fdef4197300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdee400e9f0 tx=0x7fdee400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.151+0000 7fdefa063640 1 -- 172.21.15.103:0/1230159650 shutdown_connections 2024-09-19T06:48:41.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.151+0000 7fdefa063640 1 --2- 172.21.15.103:0/1230159650 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdee00a3350 0x7fdee00a5810 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.151+0000 7fdefa063640 1 --2- 172.21.15.103:0/1230159650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdef4106a80 0x7fdef419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.151+0000 7fdefa063640 1 --2- 172.21.15.103:0/1230159650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdef4101420 0x7fdef4197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.151+0000 7fdefa063640 1 --2- 172.21.15.103:0/1230159650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdef4100a80 0x7fdef4197300 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:41.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.151+0000 7fdefa063640 1 -- 172.21.15.103:0/1230159650 >> 172.21.15.103:0/1230159650 conn(0x7fdef40fc7a0 msgr2=0x7fdef40fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:41.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.151+0000 7fdefa063640 1 -- 172.21.15.103:0/1230159650 shutdown_connections 2024-09-19T06:48:41.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:41.151+0000 7fdefa063640 1 -- 172.21.15.103:0/1230159650 wait complete. 2024-09-19T06:48:41.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:40 smithi134 ceph-mon[35356]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:48:41.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:40 smithi174 ceph-mon[35637]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:48:41.778 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T06:48:42.028 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:42.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.465+0000 7f5dd4a6c640 1 -- 172.21.15.103:0/221043565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dd0101420 msgr2=0x7f5dd010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:42.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.465+0000 7f5dd4a6c640 1 --2- 172.21.15.103:0/221043565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dd0101420 0x7f5dd010d950 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009930 tx=0x7f5dc002a420 comp rx=0 tx=0).stop 2024-09-19T06:48:42.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.465+0000 7f5dd4a6c640 1 -- 172.21.15.103:0/221043565 shutdown_connections 2024-09-19T06:48:42.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.465+0000 7f5dd4a6c640 1 --2- 172.21.15.103:0/221043565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dd0101420 0x7f5dd010d950 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.465+0000 7f5dd4a6c640 1 --2- 172.21.15.103:0/221043565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dd0100a80 0x7f5dd0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.465+0000 7f5dd4a6c640 1 --2- 172.21.15.103:0/221043565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dd0106a80 0x7f5dd0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.465+0000 7f5dd4a6c640 1 -- 172.21.15.103:0/221043565 >> 172.21.15.103:0/221043565 conn(0x7f5dd00fc7a0 msgr2=0x7f5dd00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:42.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.465+0000 7f5dd4a6c640 1 -- 172.21.15.103:0/221043565 shutdown_connections 2024-09-19T06:48:42.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.465+0000 7f5dd4a6c640 1 -- 172.21.15.103:0/221043565 wait complete. 2024-09-19T06:48:42.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.466+0000 7f5dd4a6c640 1 Processor -- start 2024-09-19T06:48:42.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.466+0000 7f5dd4a6c640 1 -- start start 2024-09-19T06:48:42.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.466+0000 7f5dd4a6c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dd0100a80 0x7f5dd0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.466+0000 7f5dd4a6c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dd0101420 0x7f5dd0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.466+0000 7f5dd4a6c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dd0106a80 0x7f5dd019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.466+0000 7f5dd4a6c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd0112030 con 0x7f5dd0101420 2024-09-19T06:48:42.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.466+0000 7f5dd4a6c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd0111eb0 con 0x7f5dd0106a80 2024-09-19T06:48:42.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.466+0000 7f5dd4a6c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd01121b0 con 0x7f5dd0100a80 2024-09-19T06:48:42.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dcdd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dd0101420 0x7f5dd0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dcdd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dd0101420 0x7f5dd0198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55248/0 (socket says 172.21.15.103:55248) 2024-09-19T06:48:42.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dcdd9b640 1 -- 172.21.15.103:0/2900210757 learned_addr learned my addr 172.21.15.103:0/2900210757 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:42.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dce59c640 1 --2- 172.21.15.103:0/2900210757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dd0100a80 0x7f5dd0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dced9d640 1 --2- 172.21.15.103:0/2900210757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dd0106a80 0x7f5dd019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dcdd9b640 1 -- 172.21.15.103:0/2900210757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dd0100a80 msgr2=0x7f5dd0198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:42.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dcdd9b640 1 --2- 172.21.15.103:0/2900210757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dd0100a80 0x7f5dd0198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dcdd9b640 1 -- 172.21.15.103:0/2900210757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dd0106a80 msgr2=0x7f5dd019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:42.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dcdd9b640 1 --2- 172.21.15.103:0/2900210757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dd0106a80 0x7f5dd019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dcdd9b640 1 -- 172.21.15.103:0/2900210757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd019d540 con 0x7f5dd0101420 2024-09-19T06:48:42.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dced9d640 1 --2- 172.21.15.103:0/2900210757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dd0106a80 0x7f5dd019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:42.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dcdd9b640 1 --2- 172.21.15.103:0/2900210757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dd0101420 0x7f5dd0198ad0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5db800b4a0 tx=0x7f5db800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:42.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dc77fe640 1 -- 172.21.15.103:0/2900210757 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8014070 con 0x7f5dd0101420 2024-09-19T06:48:42.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dc77fe640 1 -- 172.21.15.103:0/2900210757 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db8010af0 con 0x7f5dd0101420 2024-09-19T06:48:42.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dd4a6c640 1 -- 172.21.15.103:0/2900210757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd019d830 con 0x7f5dd0101420 2024-09-19T06:48:42.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dc77fe640 1 -- 172.21.15.103:0/2900210757 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8017d10 con 0x7f5dd0101420 2024-09-19T06:48:42.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.467+0000 7f5dd4a6c640 1 -- 172.21.15.103:0/2900210757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd01a5110 con 0x7f5dd0101420 2024-09-19T06:48:42.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.470+0000 7f5dc77fe640 1 -- 172.21.15.103:0/2900210757 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5db8017070 con 0x7f5dd0101420 2024-09-19T06:48:42.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.470+0000 7f5dd6e45740 1 Processor -- start 2024-09-19T06:48:42.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.470+0000 7f5dd6e45740 1 -- start start 2024-09-19T06:48:42.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dd6e45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46d38d0 0x5602f46d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dd6e45740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46d41f0 0x5602f46e2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dd6e45740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46e2fc0 0x5602f46e53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dd6e45740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5602f463a090 con 0x5602f46e2fc0 2024-09-19T06:48:42.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dd6e45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5602f45e1a70 con 0x5602f46d38d0 2024-09-19T06:48:42.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dd6e45740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5602f4636050 con 0x5602f46d41f0 2024-09-19T06:48:42.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dcdd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46e2fc0 0x5602f46e53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dced9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46d38d0 0x5602f46d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dce59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46d41f0 0x5602f46e2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dced9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46d38d0 0x5602f46d3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43166/0 (socket says 172.21.15.103:43166) 2024-09-19T06:48:42.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dced9d640 1 -- 172.21.15.103:0/1789596465 learned_addr learned my addr 172.21.15.103:0/1789596465 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:42.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dc77fe640 1 --2- 172.21.15.103:0/2900210757 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5dbc0a32d0 0x7f5dbc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dcdd9b640 1 -- 172.21.15.103:0/1789596465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46d41f0 msgr2=0x5602f46e2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:42.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dced9d640 1 --2- 172.21.15.103:0/2900210757 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5dbc0a32d0 0x7f5dbc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dcdd9b640 1 --2- 172.21.15.103:0/1789596465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46d41f0 0x5602f46e2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dcdd9b640 1 -- 172.21.15.103:0/1789596465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46d38d0 msgr2=0x5602f46d3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:42.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dcdd9b640 1 --2- 172.21.15.103:0/1789596465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46d38d0 0x5602f46d3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dcdd9b640 1 -- 172.21.15.103:0/1789596465 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602f45ee130 con 0x5602f46e2fc0 2024-09-19T06:48:42.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dce59c640 1 --2- 172.21.15.103:0/1789596465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46d41f0 0x5602f46e2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:42.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.471+0000 7f5dc77fe640 1 -- 172.21.15.103:0/2900210757 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5db80cee00 con 0x7f5dd0101420 2024-09-19T06:48:42.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.472+0000 7f5dcdd9b640 1 --2- 172.21.15.103:0/1789596465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46e2fc0 0x5602f46e53b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5db80cc8d0 tx=0x7f5db8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c60f37710b18a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:42.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.472+0000 7f5dced9d640 1 --2- 172.21.15.103:0/2900210757 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5dbc0a32d0 0x7f5dbc0a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0005ec0 tx=0x7f5dc002a420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:42.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.472+0000 7f5dc4ff9640 1 -- 172.21.15.103:0/1789596465 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db809af20 con 0x5602f46e2fc0 2024-09-19T06:48:42.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.472+0000 7f5dc4ff9640 1 -- 172.21.15.103:0/1789596465 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db809bae0 con 0x5602f46e2fc0 2024-09-19T06:48:42.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.472+0000 7f5dc4ff9640 1 -- 172.21.15.103:0/1789596465 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db80d2660 con 0x5602f46e2fc0 2024-09-19T06:48:42.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.472+0000 7f5dd6e45740 1 -- 172.21.15.103:0/1789596465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46e2fc0 msgr2=0x5602f46e53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:42.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.472+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/1789596465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46e2fc0 0x5602f46e53b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5db80cc8d0 tx=0x7f5db8020000 comp rx=0 tx=0).stop 2024-09-19T06:48:42.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dd6e45740 1 -- 172.21.15.103:0/1789596465 shutdown_connections 2024-09-19T06:48:42.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/1789596465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46e2fc0 0x5602f46e53b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/1789596465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46d41f0 0x5602f46e2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/1789596465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46d38d0 0x5602f46d3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dd6e45740 1 -- 172.21.15.103:0/1789596465 >> 172.21.15.103:0/1789596465 conn(0x5602f46cff60 msgr2=0x5602f46d0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:42.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dd6e45740 1 -- 172.21.15.103:0/1789596465 shutdown_connections 2024-09-19T06:48:42.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dd6e45740 1 -- 172.21.15.103:0/1789596465 wait complete. 2024-09-19T06:48:42.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dd6e45740 1 -- start start 2024-09-19T06:48:42.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dd6e45740 1 Processor -- start 2024-09-19T06:48:42.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dd6e45740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d38d0 0x5602f46e27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dd6e45740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46d41f0 0x5602f46dfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dd6e45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46e2fc0 0x5602f46e0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dd6e45740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5602f46e5b90 con 0x5602f46d38d0 2024-09-19T06:48:42.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dd6e45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5602f45f4f20 con 0x5602f46e2fc0 2024-09-19T06:48:42.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dd6e45740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5602f46e5d10 con 0x5602f46d41f0 2024-09-19T06:48:42.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dd6e45740 10 client.0 _ll_register_callbacks cb 0x5602f45be030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:42.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d38d0 0x5602f46e27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d38d0 0x5602f46e27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55270/0 (socket says 172.21.15.103:55270) 2024-09-19T06:48:42.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.473+0000 7f5dce59c640 1 -- 172.21.15.103:0/2199467489 learned_addr learned my addr 172.21.15.103:0/2199467489 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:42.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5dcdd9b640 1 --2- 172.21.15.103:0/2199467489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46d41f0 0x5602f46dfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5dced9d640 1 --2- 172.21.15.103:0/2199467489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46e2fc0 0x5602f46e0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5dce59c640 1 -- 172.21.15.103:0/2199467489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46d41f0 msgr2=0x5602f46dfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:42.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5dce59c640 1 --2- 172.21.15.103:0/2199467489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46d41f0 0x5602f46dfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5dce59c640 1 -- 172.21.15.103:0/2199467489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46e2fc0 msgr2=0x5602f46e0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:42.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5dce59c640 1 --2- 172.21.15.103:0/2199467489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46e2fc0 0x5602f46e0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5dce59c640 1 -- 172.21.15.103:0/2199467489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602f43ab2b0 con 0x5602f46d38d0 2024-09-19T06:48:42.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5dced9d640 1 --2- 172.21.15.103:0/2199467489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46e2fc0 0x5602f46e0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:42.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5dce59c640 1 --2- 172.21.15.103:0/2199467489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d38d0 0x5602f46e27c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800f8c0 tx=0x7f5dc800fd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33f450dd0a3372a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:42.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5db5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:48:42.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5db5ffb640 1 -- 172.21.15.103:0/2199467489 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8017070 con 0x5602f46d38d0 2024-09-19T06:48:42.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5dd6e45740 1 -- 172.21.15.103:0/2199467489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5db809bae0 con 0x5602f46d38d0 2024-09-19T06:48:42.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5db5ffb640 1 -- 172.21.15.103:0/2199467489 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5dc800dbf0 con 0x5602f46d38d0 2024-09-19T06:48:42.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5db5ffb640 1 -- 172.21.15.103:0/2199467489 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5dc8023050 con 0x5602f46d38d0 2024-09-19T06:48:42.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5dd6e45740 10 client.15012 fetch_fsmap learned FSMap version 26 2024-09-19T06:48:42.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5dd6e45740 1 -- 172.21.15.103:0/2199467489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5602f46f14c0 con 0x5602f46d38d0 2024-09-19T06:48:42.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.474+0000 7f5db5ffb640 1 -- 172.21.15.103:0/2199467489 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db809bae0 con 0x5602f46d38d0 2024-09-19T06:48:42.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.475+0000 7f5db5ffb640 1 -- 172.21.15.103:0/2199467489 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f5dc8011040 con 0x5602f46d38d0 2024-09-19T06:48:42.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.475+0000 7f5dd6e45740 10 client.15012 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:48:42.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.475+0000 7f5dd6e45740 10 client.15012 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T06:48:42.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.475+0000 7f5dd6e45740 20 client.15012 populate_metadata read hostname 'smithi103' 2024-09-19T06:48:42.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.475+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/2199467489 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5602f46f3b00 0x5602f46f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.475+0000 7f5dd6e45740 4 client.15012 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:48:42.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.475+0000 7f5dd6e45740 1 -- 172.21.15.103:0/2199467489 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602f4647600 con 0x5602f46f3b00 2024-09-19T06:48:42.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.475+0000 7f5dcdd9b640 1 --2- 172.21.15.103:0/2199467489 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5602f46f3b00 0x5602f46f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.475+0000 7f5db5ffb640 10 client.15012 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:48:42.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.475+0000 7f5dcdd9b640 1 --2- 172.21.15.103:0/2199467489 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5602f46f3b00 0x5602f46f4f70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a6dd5d6782d4253 server_cookie=7729be98ce9b25b4 in_seq=0 out_seq=0 2024-09-19T06:48:42.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.476+0000 7f5db5ffb640 1 -- 172.21.15.103:0/2199467489 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602f4647600 con 0x5602f46f3b00 2024-09-19T06:48:42.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.476+0000 7f5db5ffb640 10 client.15012 handle_command_reply: tid=0 2024-09-19T06:48:42.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.476+0000 7f5dd6e45740 1 client.15012 shutdown 2024-09-19T06:48:42.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.476+0000 7f5dd6e45740 1 -- 172.21.15.103:0/2199467489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d38d0 msgr2=0x5602f46e27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:42.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.476+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/2199467489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d38d0 0x5602f46e27c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800f8c0 tx=0x7f5dc800fd80 comp rx=0 tx=0).stop 2024-09-19T06:48:42.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.476+0000 7f5dd6e45740 1 -- 172.21.15.103:0/2199467489 shutdown_connections 2024-09-19T06:48:42.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.476+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/2199467489 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5602f46f3b00 0x5602f46f4f70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.476+0000 7f5db5ffb640 0 client.15012 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:48:42.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.476+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/2199467489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46e2fc0 0x5602f46e0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.476+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/2199467489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46d41f0 0x5602f46dfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.476+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/2199467489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d38d0 0x5602f46e27c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.477+0000 7f5dd6e45740 1 -- 172.21.15.103:0/2199467489 >> 172.21.15.103:0/2199467489 conn(0x5602f46cff60 msgr2=0x5602f46d5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:42.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.477+0000 7f5dd6e45740 1 -- 172.21.15.103:0/2199467489 shutdown_connections 2024-09-19T06:48:42.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.477+0000 7f5dd6e45740 1 -- 172.21.15.103:0/2199467489 wait complete. 2024-09-19T06:48:42.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.477+0000 7f5dd6e45740 20 client.15012 trim_cache size 0 max 16384 2024-09-19T06:48:42.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dd6e45740 1 Processor -- start 2024-09-19T06:48:42.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dd6e45740 1 -- start start 2024-09-19T06:48:42.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dd6e45740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d38d0 0x5602f46f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dd6e45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46d41f0 0x5602f46db130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dd6e45740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46e2fc0 0x5602f46dcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dd6e45740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5602f46f32b0 con 0x5602f46d38d0 2024-09-19T06:48:42.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dd6e45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5602f46f3980 con 0x5602f46d41f0 2024-09-19T06:48:42.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dd6e45740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5602f46f4050 con 0x5602f46e2fc0 2024-09-19T06:48:42.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d38d0 0x5602f46f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d38d0 0x5602f46f4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55278/0 (socket says 172.21.15.103:55278) 2024-09-19T06:48:42.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dce59c640 1 -- 172.21.15.103:0/846341454 learned_addr learned my addr 172.21.15.103:0/846341454 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:42.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dcdd9b640 1 --2- 172.21.15.103:0/846341454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46d41f0 0x5602f46db130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dced9d640 1 --2- 172.21.15.103:0/846341454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46e2fc0 0x5602f46dcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dce59c640 1 -- 172.21.15.103:0/846341454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46e2fc0 msgr2=0x5602f46dcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:42.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dce59c640 1 --2- 172.21.15.103:0/846341454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46e2fc0 0x5602f46dcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dce59c640 1 -- 172.21.15.103:0/846341454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46d41f0 msgr2=0x5602f46db130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:42.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.494+0000 7f5dce59c640 1 --2- 172.21.15.103:0/846341454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46d41f0 0x5602f46db130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.495+0000 7f5dce59c640 1 -- 172.21.15.103:0/846341454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602f46e5d00 con 0x5602f46d38d0 2024-09-19T06:48:42.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.495+0000 7f5dced9d640 1 --2- 172.21.15.103:0/846341454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46e2fc0 0x5602f46dcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:42.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.495+0000 7f5dce59c640 1 --2- 172.21.15.103:0/846341454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d38d0 0x5602f46f4f70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8007f90 tx=0x7f5dc8004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96f0338e8c0eb78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:42.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.495+0000 7f5db7fff640 1 -- 172.21.15.103:0/846341454 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc802c070 con 0x5602f46d38d0 2024-09-19T06:48:42.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.495+0000 7f5db7fff640 1 -- 172.21.15.103:0/846341454 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dc800c040 con 0x5602f46d38d0 2024-09-19T06:48:42.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.495+0000 7f5db7fff640 1 -- 172.21.15.103:0/846341454 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8017070 con 0x5602f46d38d0 2024-09-19T06:48:42.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.495+0000 7f5dd6e45740 1 -- 172.21.15.103:0/846341454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d38d0 msgr2=0x5602f46f4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:42.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.495+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/846341454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d38d0 0x5602f46f4f70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8007f90 tx=0x7f5dc8004b50 comp rx=0 tx=0).stop 2024-09-19T06:48:42.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.495+0000 7f5dd6e45740 1 -- 172.21.15.103:0/846341454 shutdown_connections 2024-09-19T06:48:42.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.495+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/846341454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46e2fc0 0x5602f46dcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.495+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/846341454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46d41f0 0x5602f46db130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.495+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/846341454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d38d0 0x5602f46f4f70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.495+0000 7f5dd6e45740 1 -- 172.21.15.103:0/846341454 >> 172.21.15.103:0/846341454 conn(0x5602f46cff60 msgr2=0x5602f46daa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:42.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.496+0000 7f5dd6e45740 1 -- 172.21.15.103:0/846341454 shutdown_connections 2024-09-19T06:48:42.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.496+0000 7f5dd6e45740 1 -- 172.21.15.103:0/846341454 wait complete. 2024-09-19T06:48:42.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.496+0000 7f5dd6e45740 1 -- start start 2024-09-19T06:48:42.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.496+0000 7f5dd6e45740 1 Processor -- start 2024-09-19T06:48:42.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.496+0000 7f5dd6e45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46d38d0 0x5602f46fd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.496+0000 7f5dd6e45740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d41f0 0x5602f46e3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.496+0000 7f5dd6e45740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46e2fc0 0x5602f46e4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.496+0000 7f5dd6e45740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5602f46f4df0 con 0x5602f46d41f0 2024-09-19T06:48:42.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.496+0000 7f5dd6e45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5602f46f4720 con 0x5602f46d38d0 2024-09-19T06:48:42.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.496+0000 7f5dd6e45740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5602f46f54c0 con 0x5602f46e2fc0 2024-09-19T06:48:42.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.496+0000 7f5dd6e45740 10 client.0 _ll_register_callbacks cb 0x7f5dc8007940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:42.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.496+0000 7f5dcdd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d41f0 0x5602f46e3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.496+0000 7f5dcdd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d41f0 0x5602f46e3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55290/0 (socket says 172.21.15.103:55290) 2024-09-19T06:48:42.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.496+0000 7f5dcdd9b640 1 -- 172.21.15.103:0/921279963 learned_addr learned my addr 172.21.15.103:0/921279963 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:42.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.496+0000 7f5dce59c640 1 --2- 172.21.15.103:0/921279963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46d38d0 0x5602f46fd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5dced9d640 1 --2- 172.21.15.103:0/921279963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46e2fc0 0x5602f46e4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5dcdd9b640 1 -- 172.21.15.103:0/921279963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46e2fc0 msgr2=0x5602f46e4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:42.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5dcdd9b640 1 --2- 172.21.15.103:0/921279963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46e2fc0 0x5602f46e4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5dcdd9b640 1 -- 172.21.15.103:0/921279963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46d38d0 msgr2=0x5602f46fd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:42.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5dcdd9b640 1 --2- 172.21.15.103:0/921279963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46d38d0 0x5602f46fd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5dcdd9b640 1 -- 172.21.15.103:0/921279963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602f46e4b40 con 0x5602f46d41f0 2024-09-19T06:48:42.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5dced9d640 1 --2- 172.21.15.103:0/921279963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46e2fc0 0x5602f46e4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:42.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5dcdd9b640 1 --2- 172.21.15.103:0/921279963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d41f0 0x5602f46e3fc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5db80cc950 tx=0x7f5db80d2a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3fdfcc39fd69c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:42.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5db5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:48:42.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5db5ffb640 1 -- 172.21.15.103:0/921279963 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db809ad40 con 0x5602f46d41f0 2024-09-19T06:48:42.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5db5ffb640 1 -- 172.21.15.103:0/921279963 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5db80996e0 con 0x5602f46d41f0 2024-09-19T06:48:42.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5dd6e45740 1 -- 172.21.15.103:0/921279963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dc800c040 con 0x5602f46d41f0 2024-09-19T06:48:42.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5db5ffb640 1 -- 172.21.15.103:0/921279963 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5db8009230 con 0x5602f46d41f0 2024-09-19T06:48:42.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5dd6e45740 10 client.15018 fetch_fsmap learned FSMap version 26 2024-09-19T06:48:42.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5dd6e45740 1 -- 172.21.15.103:0/921279963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5602f46fa0a0 con 0x5602f46d41f0 2024-09-19T06:48:42.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5db5ffb640 1 -- 172.21.15.103:0/921279963 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dc800c040 con 0x5602f46d41f0 2024-09-19T06:48:42.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5db5ffb640 1 -- 172.21.15.103:0/921279963 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f5db80cc980 con 0x5602f46d41f0 2024-09-19T06:48:42.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5dd6e45740 10 client.15018 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:48:42.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5dd6e45740 10 client.15018 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T06:48:42.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5dd6e45740 20 client.15018 populate_metadata read hostname 'smithi103' 2024-09-19T06:48:42.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/921279963 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5602f46f3b00 0x5602f47614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:42.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.497+0000 7f5dd6e45740 4 client.15018 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:48:42.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5dd6e45740 1 -- 172.21.15.103:0/921279963 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "status"}) -- 0x5602f4641b10 con 0x5602f46f3b00 2024-09-19T06:48:42.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5dce59c640 1 --2- 172.21.15.103:0/921279963 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5602f46f3b00 0x5602f47614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:42.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5dce59c640 1 --2- 172.21.15.103:0/921279963 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5602f46f3b00 0x5602f47614b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38baf1000f60fea server_cookie=39a5d95c082ba706 in_seq=0 out_seq=0 2024-09-19T06:48:42.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5db5ffb640 10 client.15018 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:48:42.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5db5ffb640 1 -- 172.21.15.103:0/921279963 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5602f4641b10 con 0x5602f46f3b00 2024-09-19T06:48:42.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5db5ffb640 10 client.15018 handle_command_reply: tid=0 2024-09-19T06:48:42.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5dd6e45740 1 client.15018 shutdown 2024-09-19T06:48:42.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5dd6e45740 1 -- 172.21.15.103:0/921279963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d41f0 msgr2=0x5602f46e3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:42.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/921279963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d41f0 0x5602f46e3fc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5db80cc950 tx=0x7f5db80d2a30 comp rx=0 tx=0).stop 2024-09-19T06:48:42.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5dd6e45740 1 -- 172.21.15.103:0/921279963 shutdown_connections 2024-09-19T06:48:42.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/921279963 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5602f46f3b00 0x5602f47614b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/921279963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602f46e2fc0 0x5602f46e4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/921279963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602f46d41f0 0x5602f46e3fc0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5dd6e45740 1 --2- 172.21.15.103:0/921279963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602f46d38d0 0x5602f46fd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5dd6e45740 1 -- 172.21.15.103:0/921279963 >> 172.21.15.103:0/921279963 conn(0x5602f46cff60 msgr2=0x5602f46d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:42.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5db5ffb640 0 client.15018 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:48:42.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5dd6e45740 1 -- 172.21.15.103:0/921279963 shutdown_connections 2024-09-19T06:48:42.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.498+0000 7f5dd6e45740 1 -- 172.21.15.103:0/921279963 wait complete. 2024-09-19T06:48:42.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.499+0000 7f5dd6e45740 20 client.15018 trim_cache size 0 max 16384 2024-09-19T06:48:42.509 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:48:42.509 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:48:42.509 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 1, 2024-09-19T06:48:42.509 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24455, 2024-09-19T06:48:42.509 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T06:48:42.509 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T06:48:42.509 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T06:48:42.509 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 69.632557868000006, 2024-09-19T06:48:42.509 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T06:48:42.509 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 82, 2024-09-19T06:48:42.510 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T06:48:42.510 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 98.709036460999997 2024-09-19T06:48:42.510 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:48:42.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.499+0000 7f5dd4a6c640 1 -- 172.21.15.103:0/2900210757 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5dbc0a32d0 msgr2=0x7f5dbc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:42.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.499+0000 7f5dd4a6c640 1 --2- 172.21.15.103:0/2900210757 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5dbc0a32d0 0x7f5dbc0a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0005ec0 tx=0x7f5dc002a420 comp rx=0 tx=0).stop 2024-09-19T06:48:42.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.499+0000 7f5dd4a6c640 1 -- 172.21.15.103:0/2900210757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dd0101420 msgr2=0x7f5dd0198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:42.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.499+0000 7f5dd4a6c640 1 --2- 172.21.15.103:0/2900210757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dd0101420 0x7f5dd0198ad0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5db800b4a0 tx=0x7f5db800b960 comp rx=0 tx=0).stop 2024-09-19T06:48:42.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.499+0000 7f5dd4a6c640 1 -- 172.21.15.103:0/2900210757 shutdown_connections 2024-09-19T06:48:42.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.499+0000 7f5dd4a6c640 1 --2- 172.21.15.103:0/2900210757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dd0106a80 0x7f5dd019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.499+0000 7f5dd4a6c640 1 --2- 172.21.15.103:0/2900210757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dd0101420 0x7f5dd0198ad0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.499+0000 7f5dd4a6c640 1 --2- 172.21.15.103:0/2900210757 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f5dbc0a32d0 0x7f5dbc0a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.499+0000 7f5dd4a6c640 1 --2- 172.21.15.103:0/2900210757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dd0100a80 0x7f5dd0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:42.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.499+0000 7f5dd4a6c640 1 -- 172.21.15.103:0/2900210757 >> 172.21.15.103:0/2900210757 conn(0x7f5dd00fc7a0 msgr2=0x7f5dd00fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:42.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.499+0000 7f5dd4a6c640 1 -- 172.21.15.103:0/2900210757 shutdown_connections 2024-09-19T06:48:42.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:42.499+0000 7f5dd4a6c640 1 -- 172.21.15.103:0/2900210757 wait complete. 2024-09-19T06:48:43.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:42 smithi103 ceph-mon[31158]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:48:43.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:42 smithi134 ceph-mon[35356]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:48:43.202 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T06:48:43.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:42 smithi174 ceph-mon[35637]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:48:43.448 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.906+0000 7f8d84db8640 1 -- 172.21.15.103:0/2777133564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d80100ca0 msgr2=0x7f8d80106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.906+0000 7f8d84db8640 1 --2- 172.21.15.103:0/2777133564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d80100ca0 0x7f8d80106840 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009930 tx=0x7f8d7402a420 comp rx=0 tx=0).stop 2024-09-19T06:48:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.906+0000 7f8d84db8640 1 -- 172.21.15.103:0/2777133564 shutdown_connections 2024-09-19T06:48:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.906+0000 7f8d84db8640 1 --2- 172.21.15.103:0/2777133564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d80106f50 0x7f8d8010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.906+0000 7f8d84db8640 1 --2- 172.21.15.103:0/2777133564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d80100ca0 0x7f8d80106840 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.906+0000 7f8d84db8640 1 --2- 172.21.15.103:0/2777133564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d801002f0 0x7f8d801006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.906+0000 7f8d84db8640 1 -- 172.21.15.103:0/2777133564 >> 172.21.15.103:0/2777133564 conn(0x7f8d80077210 msgr2=0x7f8d800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.906+0000 7f8d84db8640 1 -- 172.21.15.103:0/2777133564 shutdown_connections 2024-09-19T06:48:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.906+0000 7f8d84db8640 1 -- 172.21.15.103:0/2777133564 wait complete. 2024-09-19T06:48:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d84db8640 1 Processor -- start 2024-09-19T06:48:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d84db8640 1 -- start start 2024-09-19T06:48:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d84db8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d801002f0 0x7f8d801a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d84db8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d80100ca0 0x7f8d801a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d84db8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d80106f50 0x7f8d8019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d84db8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d80112060 con 0x7f8d801002f0 2024-09-19T06:48:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d84db8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d80111ee0 con 0x7f8d80100ca0 2024-09-19T06:48:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d84db8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d801121e0 con 0x7f8d80106f50 2024-09-19T06:48:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d7e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d801002f0 0x7f8d801a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d7e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d801002f0 0x7f8d801a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55306/0 (socket says 172.21.15.103:55306) 2024-09-19T06:48:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d7e59c640 1 -- 172.21.15.103:0/2335433541 learned_addr learned my addr 172.21.15.103:0/2335433541 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d7ed9d640 1 --2- 172.21.15.103:0/2335433541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d80106f50 0x7f8d8019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d7e59c640 1 -- 172.21.15.103:0/2335433541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d80106f50 msgr2=0x7f8d8019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d7e59c640 1 --2- 172.21.15.103:0/2335433541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d80106f50 0x7f8d8019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d7e59c640 1 -- 172.21.15.103:0/2335433541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d80100ca0 msgr2=0x7f8d801a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d7e59c640 1 --2- 172.21.15.103:0/2335433541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d80100ca0 0x7f8d801a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d7e59c640 1 -- 172.21.15.103:0/2335433541 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d8019aec0 con 0x7f8d801002f0 2024-09-19T06:48:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d7ed9d640 1 --2- 172.21.15.103:0/2335433541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d80106f50 0x7f8d8019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.907+0000 7f8d7e59c640 1 --2- 172.21.15.103:0/2335433541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d801002f0 0x7f8d801a0580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000b4a0 tx=0x7f8d7000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.908+0000 7f8d6f7fe640 1 -- 172.21.15.103:0/2335433541 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70014070 con 0x7f8d801002f0 2024-09-19T06:48:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.908+0000 7f8d84db8640 1 -- 172.21.15.103:0/2335433541 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d8019b1b0 con 0x7f8d801002f0 2024-09-19T06:48:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.908+0000 7f8d6f7fe640 1 -- 172.21.15.103:0/2335433541 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d70010af0 con 0x7f8d801002f0 2024-09-19T06:48:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.908+0000 7f8d6f7fe640 1 -- 172.21.15.103:0/2335433541 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70017d10 con 0x7f8d801002f0 2024-09-19T06:48:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.908+0000 7f8d84db8640 1 -- 172.21.15.103:0/2335433541 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d801a7500 con 0x7f8d801002f0 2024-09-19T06:48:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.910+0000 7f8d6f7fe640 1 -- 172.21.15.103:0/2335433541 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8d70017070 con 0x7f8d801002f0 2024-09-19T06:48:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.910+0000 7f8d87191740 1 Processor -- start 2024-09-19T06:48:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d87191740 1 -- start start 2024-09-19T06:48:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d87191740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24c39d0 0x55eac24c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d87191740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c42f0 0x55eac24ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d87191740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24cebb0 0x55eac24d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d87191740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eac242a090 con 0x55eac24c42f0 2024-09-19T06:48:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d87191740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55eac23d1a70 con 0x55eac24c39d0 2024-09-19T06:48:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d87191740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eac2426050 con 0x55eac24cebb0 2024-09-19T06:48:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d6f7fe640 1 --2- 172.21.15.103:0/2335433541 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8d680a32d0 0x7f8d680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d7e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c42f0 0x55eac24ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d7e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c42f0 0x55eac24ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55316/0 (socket says 172.21.15.103:55316) 2024-09-19T06:48:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d7e59c640 1 -- 172.21.15.103:0/3375816134 learned_addr learned my addr 172.21.15.103:0/3375816134 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d7ed9d640 1 --2- 172.21.15.103:0/2335433541 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8d680a32d0 0x7f8d680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d7dd9b640 1 --2- 172.21.15.103:0/3375816134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24cebb0 0x55eac24d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d7ed9d640 1 --2- 172.21.15.103:0/3375816134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24c39d0 0x55eac24c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d6f7fe640 1 -- 172.21.15.103:0/2335433541 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8d700cee60 con 0x7f8d801002f0 2024-09-19T06:48:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d7dd9b640 1 -- 172.21.15.103:0/3375816134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24c39d0 msgr2=0x55eac24c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d7dd9b640 1 --2- 172.21.15.103:0/3375816134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24c39d0 0x55eac24c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.911+0000 7f8d7dd9b640 1 -- 172.21.15.103:0/3375816134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c42f0 msgr2=0x55eac24ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.912+0000 7f8d7dd9b640 1 --2- 172.21.15.103:0/3375816134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c42f0 0x55eac24ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.912+0000 7f8d7dd9b640 1 -- 172.21.15.103:0/3375816134 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eac23de130 con 0x55eac24cebb0 2024-09-19T06:48:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.912+0000 7f8d7e59c640 1 --2- 172.21.15.103:0/3375816134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c42f0 0x55eac24ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.912+0000 7f8d7ed9d640 1 --2- 172.21.15.103:0/2335433541 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8d680a32d0 0x7f8d680a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8d78012b70 tx=0x7f8d78008db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.912+0000 7f8d7dd9b640 1 --2- 172.21.15.103:0/3375816134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24cebb0 0x55eac24d0fa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8d8019bea0 tx=0x7f8d74005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f71b134248b26854 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.915+0000 7f8d6cff9640 1 -- 172.21.15.103:0/3375816134 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74040070 con 0x55eac24cebb0 2024-09-19T06:48:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.915+0000 7f8d6cff9640 1 -- 172.21.15.103:0/3375816134 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d74004040 con 0x55eac24cebb0 2024-09-19T06:48:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.915+0000 7f8d6cff9640 1 -- 172.21.15.103:0/3375816134 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74035040 con 0x55eac24cebb0 2024-09-19T06:48:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.916+0000 7f8d87191740 1 -- 172.21.15.103:0/3375816134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24cebb0 msgr2=0x55eac24d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.916+0000 7f8d87191740 1 --2- 172.21.15.103:0/3375816134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24cebb0 0x55eac24d0fa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8d8019bea0 tx=0x7f8d74005e80 comp rx=0 tx=0).stop 2024-09-19T06:48:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.916+0000 7f8d87191740 1 -- 172.21.15.103:0/3375816134 shutdown_connections 2024-09-19T06:48:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.916+0000 7f8d87191740 1 --2- 172.21.15.103:0/3375816134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24cebb0 0x55eac24d0fa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.916+0000 7f8d87191740 1 --2- 172.21.15.103:0/3375816134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c42f0 0x55eac24ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.916+0000 7f8d87191740 1 --2- 172.21.15.103:0/3375816134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24c39d0 0x55eac24c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.916+0000 7f8d87191740 1 -- 172.21.15.103:0/3375816134 >> 172.21.15.103:0/3375816134 conn(0x55eac24bff60 msgr2=0x55eac24c0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.916+0000 7f8d87191740 1 -- 172.21.15.103:0/3375816134 shutdown_connections 2024-09-19T06:48:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.916+0000 7f8d87191740 1 -- 172.21.15.103:0/3375816134 wait complete. 2024-09-19T06:48:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.916+0000 7f8d87191740 1 -- start start 2024-09-19T06:48:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.916+0000 7f8d87191740 1 Processor -- start 2024-09-19T06:48:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.916+0000 7f8d87191740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24c39d0 0x55eac24ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d87191740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24c42f0 0x55eac24cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d87191740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24cebb0 0x55eac24cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d87191740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eac24d1780 con 0x55eac24cebb0 2024-09-19T06:48:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d87191740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55eac23e4f20 con 0x55eac24c42f0 2024-09-19T06:48:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d87191740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eac24d1900 con 0x55eac24c39d0 2024-09-19T06:48:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d87191740 10 client.0 _ll_register_callbacks cb 0x55eac23ae030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d7ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24cebb0 0x55eac24cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d7ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24cebb0 0x55eac24cbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55318/0 (socket says 172.21.15.103:55318) 2024-09-19T06:48:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d7ed9d640 1 -- 172.21.15.103:0/1851495229 learned_addr learned my addr 172.21.15.103:0/1851495229 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d7e59c640 1 --2- 172.21.15.103:0/1851495229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24c39d0 0x55eac24ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d7dd9b640 1 --2- 172.21.15.103:0/1851495229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24c42f0 0x55eac24cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d7ed9d640 1 -- 172.21.15.103:0/1851495229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24c39d0 msgr2=0x55eac24ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d7ed9d640 1 --2- 172.21.15.103:0/1851495229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24c39d0 0x55eac24ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d7ed9d640 1 -- 172.21.15.103:0/1851495229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24c42f0 msgr2=0x55eac24cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d7ed9d640 1 --2- 172.21.15.103:0/1851495229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24c42f0 0x55eac24cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d7ed9d640 1 -- 172.21.15.103:0/1851495229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eac219b2b0 con 0x55eac24cebb0 2024-09-19T06:48:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d7ed9d640 1 --2- 172.21.15.103:0/1851495229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24cebb0 0x55eac24cbbc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8d78014fd0 tx=0x7f8d78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9152bf1559b59d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:48:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d65ffb640 1 -- 172.21.15.103:0/1851495229 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78023030 con 0x55eac24cebb0 2024-09-19T06:48:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d65ffb640 1 -- 172.21.15.103:0/1851495229 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8d7801a070 con 0x55eac24cebb0 2024-09-19T06:48:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d87191740 1 -- 172.21.15.103:0/1851495229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d74004040 con 0x55eac24cebb0 2024-09-19T06:48:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d65ffb640 1 -- 172.21.15.103:0/1851495229 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d7801f6d0 con 0x55eac24cebb0 2024-09-19T06:48:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d87191740 10 client.15030 fetch_fsmap learned FSMap version 26 2024-09-19T06:48:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.917+0000 7f8d87191740 1 -- 172.21.15.103:0/1851495229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eac24dcfc0 con 0x55eac24cebb0 2024-09-19T06:48:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.918+0000 7f8d65ffb640 1 -- 172.21.15.103:0/1851495229 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d74004040 con 0x55eac24cebb0 2024-09-19T06:48:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.918+0000 7f8d65ffb640 1 -- 172.21.15.103:0/1851495229 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f8d7801e040 con 0x55eac24cebb0 2024-09-19T06:48:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.918+0000 7f8d87191740 10 client.15030 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:48:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.918+0000 7f8d87191740 10 client.15030 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T06:48:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.918+0000 7f8d87191740 20 client.15030 populate_metadata read hostname 'smithi103' 2024-09-19T06:48:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.918+0000 7f8d87191740 1 --2- 172.21.15.103:0/1851495229 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eac24df5f0 0x55eac24e0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.918+0000 7f8d87191740 4 client.15030 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:48:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.918+0000 7f8d87191740 1 -- 172.21.15.103:0/1851495229 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eac2437600 con 0x55eac24df5f0 2024-09-19T06:48:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.918+0000 7f8d7e59c640 1 --2- 172.21.15.103:0/1851495229 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eac24df5f0 0x55eac24e0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.919+0000 7f8d7e59c640 1 --2- 172.21.15.103:0/1851495229 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eac24df5f0 0x55eac24e0a60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74b7f0845d4113c5 server_cookie=d26f9d7e71deaa69 in_seq=0 out_seq=0 2024-09-19T06:48:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.919+0000 7f8d65ffb640 10 client.15030 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:48:43.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.919+0000 7f8d65ffb640 1 -- 172.21.15.103:0/1851495229 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eac2437600 con 0x55eac24df5f0 2024-09-19T06:48:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.919+0000 7f8d65ffb640 10 client.15030 handle_command_reply: tid=0 2024-09-19T06:48:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.919+0000 7f8d87191740 1 client.15030 shutdown 2024-09-19T06:48:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.920+0000 7f8d87191740 1 -- 172.21.15.103:0/1851495229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24cebb0 msgr2=0x55eac24cbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.920+0000 7f8d87191740 1 --2- 172.21.15.103:0/1851495229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24cebb0 0x55eac24cbbc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8d78014fd0 tx=0x7f8d78004a40 comp rx=0 tx=0).stop 2024-09-19T06:48:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.920+0000 7f8d87191740 1 -- 172.21.15.103:0/1851495229 shutdown_connections 2024-09-19T06:48:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.920+0000 7f8d87191740 1 --2- 172.21.15.103:0/1851495229 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eac24df5f0 0x55eac24e0a60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.920+0000 7f8d87191740 1 --2- 172.21.15.103:0/1851495229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24cebb0 0x55eac24cbbc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.920+0000 7f8d87191740 1 --2- 172.21.15.103:0/1851495229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24c42f0 0x55eac24cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.920+0000 7f8d87191740 1 --2- 172.21.15.103:0/1851495229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24c39d0 0x55eac24ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.920+0000 7f8d65ffb640 0 client.15030 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:48:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.920+0000 7f8d87191740 1 -- 172.21.15.103:0/1851495229 >> 172.21.15.103:0/1851495229 conn(0x55eac24bff60 msgr2=0x55eac24c5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.920+0000 7f8d87191740 1 -- 172.21.15.103:0/1851495229 shutdown_connections 2024-09-19T06:48:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.920+0000 7f8d87191740 1 -- 172.21.15.103:0/1851495229 wait complete. 2024-09-19T06:48:43.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.920+0000 7f8d87191740 20 client.15030 trim_cache size 0 max 16384 2024-09-19T06:48:43.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.937+0000 7f8d87191740 1 Processor -- start 2024-09-19T06:48:43.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.937+0000 7f8d87191740 1 -- start start 2024-09-19T06:48:43.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.937+0000 7f8d87191740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24c39d0 0x55eac24cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.937+0000 7f8d87191740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c42f0 0x55eac24cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.937+0000 7f8d87191740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24cebb0 0x55eac24c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.937+0000 7f8d87191740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eac24deda0 con 0x55eac24c42f0 2024-09-19T06:48:43.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.937+0000 7f8d87191740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55eac24df470 con 0x55eac24cebb0 2024-09-19T06:48:43.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.937+0000 7f8d87191740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eac24dfb40 con 0x55eac24c39d0 2024-09-19T06:48:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.938+0000 7f8d7dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c42f0 0x55eac24cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.938+0000 7f8d7dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c42f0 0x55eac24cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55332/0 (socket says 172.21.15.103:55332) 2024-09-19T06:48:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.938+0000 7f8d7dd9b640 1 -- 172.21.15.103:0/3306912621 learned_addr learned my addr 172.21.15.103:0/3306912621 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.938+0000 7f8d7e59c640 1 --2- 172.21.15.103:0/3306912621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24c39d0 0x55eac24cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.938+0000 7f8d7dd9b640 1 -- 172.21.15.103:0/3306912621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24c39d0 msgr2=0x55eac24cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.938+0000 7f8d7dd9b640 1 --2- 172.21.15.103:0/3306912621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24c39d0 0x55eac24cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.938+0000 7f8d7dd9b640 1 -- 172.21.15.103:0/3306912621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24cebb0 msgr2=0x55eac24c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.938+0000 7f8d7dd9b640 1 --2- 172.21.15.103:0/3306912621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24cebb0 0x55eac24c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.938+0000 7f8d7dd9b640 1 -- 172.21.15.103:0/3306912621 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eac24bfd90 con 0x55eac24c42f0 2024-09-19T06:48:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.938+0000 7f8d7dd9b640 1 --2- 172.21.15.103:0/3306912621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c42f0 0x55eac24cbb80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009930 tx=0x7f8d74005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a7b4d2120267b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.938+0000 7f8d67fff640 1 -- 172.21.15.103:0/3306912621 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7404c070 con 0x55eac24c42f0 2024-09-19T06:48:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.938+0000 7f8d67fff640 1 -- 172.21.15.103:0/3306912621 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d740035f0 con 0x55eac24c42f0 2024-09-19T06:48:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.938+0000 7f8d67fff640 1 -- 172.21.15.103:0/3306912621 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74040070 con 0x55eac24c42f0 2024-09-19T06:48:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 -- 172.21.15.103:0/3306912621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c42f0 msgr2=0x55eac24cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 --2- 172.21.15.103:0/3306912621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c42f0 0x55eac24cbb80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009930 tx=0x7f8d74005760 comp rx=0 tx=0).stop 2024-09-19T06:48:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 -- 172.21.15.103:0/3306912621 shutdown_connections 2024-09-19T06:48:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 --2- 172.21.15.103:0/3306912621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24cebb0 0x55eac24c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 --2- 172.21.15.103:0/3306912621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c42f0 0x55eac24cbb80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 --2- 172.21.15.103:0/3306912621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24c39d0 0x55eac24cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 -- 172.21.15.103:0/3306912621 >> 172.21.15.103:0/3306912621 conn(0x55eac24bff60 msgr2=0x55eac24c0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 -- 172.21.15.103:0/3306912621 shutdown_connections 2024-09-19T06:48:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 -- 172.21.15.103:0/3306912621 wait complete. 2024-09-19T06:48:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 -- start start 2024-09-19T06:48:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 Processor -- start 2024-09-19T06:48:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c39d0 0x55eac24c8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24c42f0 0x55eac24c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24cebb0 0x55eac2547cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eac24e08e0 con 0x55eac24c39d0 2024-09-19T06:48:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55eac24e0210 con 0x55eac24cebb0 2024-09-19T06:48:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.939+0000 7f8d87191740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eac24e0fb0 con 0x55eac24c42f0 2024-09-19T06:48:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d87191740 10 client.0 _ll_register_callbacks cb 0x7f8d78008660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d7e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c39d0 0x55eac24c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d7e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c39d0 0x55eac24c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55348/0 (socket says 172.21.15.103:55348) 2024-09-19T06:48:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d7e59c640 1 -- 172.21.15.103:0/846673222 learned_addr learned my addr 172.21.15.103:0/846673222 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d7ed9d640 1 --2- 172.21.15.103:0/846673222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24cebb0 0x55eac2547cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d7dd9b640 1 --2- 172.21.15.103:0/846673222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24c42f0 0x55eac24c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d7e59c640 1 -- 172.21.15.103:0/846673222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24c42f0 msgr2=0x55eac24c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d7e59c640 1 --2- 172.21.15.103:0/846673222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24c42f0 0x55eac24c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d7e59c640 1 -- 172.21.15.103:0/846673222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24cebb0 msgr2=0x55eac2547cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d7e59c640 1 --2- 172.21.15.103:0/846673222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24cebb0 0x55eac2547cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d7e59c640 1 -- 172.21.15.103:0/846673222 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eac24c8c50 con 0x55eac24c39d0 2024-09-19T06:48:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d7e59c640 1 --2- 172.21.15.103:0/846673222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c39d0 0x55eac24c8070 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8d700176e0 tx=0x7f8d700047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933f5c2ffc7f9875 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:48:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d65ffb640 1 -- 172.21.15.103:0/846673222 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d700b3070 con 0x55eac24c39d0 2024-09-19T06:48:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d65ffb640 1 -- 172.21.15.103:0/846673222 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8d70010ca0 con 0x55eac24c39d0 2024-09-19T06:48:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d87191740 1 -- 172.21.15.103:0/846673222 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d740035f0 con 0x55eac24c39d0 2024-09-19T06:48:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d65ffb640 1 -- 172.21.15.103:0/846673222 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d7001e040 con 0x55eac24c39d0 2024-09-19T06:48:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d87191740 10 client.15036 fetch_fsmap learned FSMap version 26 2024-09-19T06:48:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.940+0000 7f8d87191740 1 -- 172.21.15.103:0/846673222 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eac2549340 con 0x55eac24c39d0 2024-09-19T06:48:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.941+0000 7f8d65ffb640 1 -- 172.21.15.103:0/846673222 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d7007e030 con 0x55eac24c39d0 2024-09-19T06:48:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.941+0000 7f8d65ffb640 1 -- 172.21.15.103:0/846673222 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f8d70097a50 con 0x55eac24c39d0 2024-09-19T06:48:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.941+0000 7f8d87191740 10 client.15036 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:48:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.941+0000 7f8d87191740 10 client.15036 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T06:48:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.941+0000 7f8d87191740 20 client.15036 populate_metadata read hostname 'smithi103' 2024-09-19T06:48:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.941+0000 7f8d87191740 1 --2- 172.21.15.103:0/846673222 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eac254df10 0x55eac254cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.941+0000 7f8d87191740 4 client.15036 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:48:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.941+0000 7f8d87191740 1 -- 172.21.15.103:0/846673222 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "status"}) -- 0x55eac2431b10 con 0x55eac254df10 2024-09-19T06:48:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.941+0000 7f8d7dd9b640 1 --2- 172.21.15.103:0/846673222 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eac254df10 0x55eac254cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.941+0000 7f8d65ffb640 10 client.15036 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:48:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.941+0000 7f8d7dd9b640 1 --2- 172.21.15.103:0/846673222 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eac254df10 0x55eac254cc20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a533291eeed7769b server_cookie=a6730c53fc934435 in_seq=0 out_seq=0 2024-09-19T06:48:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d65ffb640 1 -- 172.21.15.103:0/846673222 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55eac2431b10 con 0x55eac254df10 2024-09-19T06:48:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d65ffb640 10 client.15036 handle_command_reply: tid=0 2024-09-19T06:48:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d87191740 1 client.15036 shutdown 2024-09-19T06:48:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d87191740 1 -- 172.21.15.103:0/846673222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c39d0 msgr2=0x55eac24c8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d87191740 1 --2- 172.21.15.103:0/846673222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c39d0 0x55eac24c8070 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8d700176e0 tx=0x7f8d700047d0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d87191740 1 -- 172.21.15.103:0/846673222 shutdown_connections 2024-09-19T06:48:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d87191740 1 --2- 172.21.15.103:0/846673222 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eac254df10 0x55eac254cc20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d65ffb640 0 client.15036 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:48:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d87191740 1 --2- 172.21.15.103:0/846673222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eac24cebb0 0x55eac2547cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d87191740 1 --2- 172.21.15.103:0/846673222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eac24c42f0 0x55eac24c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d87191740 1 --2- 172.21.15.103:0/846673222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eac24c39d0 0x55eac24c8070 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d87191740 1 -- 172.21.15.103:0/846673222 >> 172.21.15.103:0/846673222 conn(0x55eac24bff60 msgr2=0x55eac24cb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d87191740 1 -- 172.21.15.103:0/846673222 shutdown_connections 2024-09-19T06:48:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d87191740 1 -- 172.21.15.103:0/846673222 wait complete. 2024-09-19T06:48:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d87191740 20 client.15036 trim_cache size 0 max 16384 2024-09-19T06:48:43.950 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:48:43.950 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:48:43.950 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-09-19T06:48:43.950 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24482, 2024-09-19T06:48:43.950 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T06:48:43.950 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T06:48:43.950 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T06:48:43.950 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 78.105617722999995, 2024-09-19T06:48:43.951 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T06:48:43.951 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 82, 2024-09-19T06:48:43.951 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T06:48:43.951 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 89.197645374000004 2024-09-19T06:48:43.951 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:48:43.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d84db8640 1 -- 172.21.15.103:0/2335433541 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8d680a32d0 msgr2=0x7f8d680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d84db8640 1 --2- 172.21.15.103:0/2335433541 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8d680a32d0 0x7f8d680a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8d78012b70 tx=0x7f8d78008db0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d84db8640 1 -- 172.21.15.103:0/2335433541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d801002f0 msgr2=0x7f8d801a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.942+0000 7f8d84db8640 1 --2- 172.21.15.103:0/2335433541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d801002f0 0x7f8d801a0580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000b4a0 tx=0x7f8d7000b960 comp rx=0 tx=0).stop 2024-09-19T06:48:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.943+0000 7f8d84db8640 1 -- 172.21.15.103:0/2335433541 shutdown_connections 2024-09-19T06:48:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.943+0000 7f8d84db8640 1 --2- 172.21.15.103:0/2335433541 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8d680a32d0 0x7f8d680a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.943+0000 7f8d84db8640 1 --2- 172.21.15.103:0/2335433541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d80106f50 0x7f8d8019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.943+0000 7f8d84db8640 1 --2- 172.21.15.103:0/2335433541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d80100ca0 0x7f8d801a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.943+0000 7f8d84db8640 1 --2- 172.21.15.103:0/2335433541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d801002f0 0x7f8d801a0580 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.943+0000 7f8d84db8640 1 -- 172.21.15.103:0/2335433541 >> 172.21.15.103:0/2335433541 conn(0x7f8d80077210 msgr2=0x7f8d80105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.943+0000 7f8d84db8640 1 -- 172.21.15.103:0/2335433541 shutdown_connections 2024-09-19T06:48:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:43.943+0000 7f8d84db8640 1 -- 172.21.15.103:0/2335433541 wait complete. 2024-09-19T06:48:44.587 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T06:48:44.587 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T06:48:44.598 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T06:48:44.598 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T06:48:44.848 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:45.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:44 smithi134 ceph-mon[35356]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:48:45.210 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:44 smithi103 ceph-mon[31158]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:48:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.288+0000 7f64826bd640 1 -- 172.21.15.103:0/2317393538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f647c101cd0 msgr2=0x7f647c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.288+0000 7f64826bd640 1 --2- 172.21.15.103:0/2317393538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f647c101cd0 0x7f647c102150 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6470009900 tx=0x7f647002a410 comp rx=0 tx=0).stop 2024-09-19T06:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.288+0000 7f64826bd640 1 -- 172.21.15.103:0/2317393538 shutdown_connections 2024-09-19T06:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.288+0000 7f64826bd640 1 --2- 172.21.15.103:0/2317393538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f647c102690 0x7f647c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.288+0000 7f64826bd640 1 --2- 172.21.15.103:0/2317393538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f647c101cd0 0x7f647c102150 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.288+0000 7f64826bd640 1 --2- 172.21.15.103:0/2317393538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f647c100ad0 0x7f647c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.288+0000 7f64826bd640 1 -- 172.21.15.103:0/2317393538 >> 172.21.15.103:0/2317393538 conn(0x7f647c0fc280 msgr2=0x7f647c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.288+0000 7f64826bd640 1 -- 172.21.15.103:0/2317393538 shutdown_connections 2024-09-19T06:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.288+0000 7f64826bd640 1 -- 172.21.15.103:0/2317393538 wait complete. 2024-09-19T06:48:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.289+0000 7f64826bd640 1 Processor -- start 2024-09-19T06:48:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.289+0000 7f64826bd640 1 -- start start 2024-09-19T06:48:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.289+0000 7f64826bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f647c100ad0 0x7f647c196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.289+0000 7f64826bd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f647c101cd0 0x7f647c1974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.289+0000 7f64826bd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f647c102690 0x7f647c19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.289+0000 7f64826bd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f647c10c850 con 0x7f647c101cd0 2024-09-19T06:48:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.289+0000 7f64826bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f647c10c6d0 con 0x7f647c100ad0 2024-09-19T06:48:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.289+0000 7f64826bd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f647c10c9d0 con 0x7f647c102690 2024-09-19T06:48:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.289+0000 7f647b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f647c101cd0 0x7f647c1974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.290+0000 7f647b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f647c101cd0 0x7f647c1974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55368/0 (socket says 172.21.15.103:55368) 2024-09-19T06:48:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.290+0000 7f647b7fe640 1 -- 172.21.15.103:0/761719894 learned_addr learned my addr 172.21.15.103:0/761719894 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.290+0000 7f647bfff640 1 --2- 172.21.15.103:0/761719894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f647c100ad0 0x7f647c196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.290+0000 7f6480c5a640 1 --2- 172.21.15.103:0/761719894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f647c102690 0x7f647c19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.290+0000 7f647b7fe640 1 -- 172.21.15.103:0/761719894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f647c102690 msgr2=0x7f647c19e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.290+0000 7f647b7fe640 1 --2- 172.21.15.103:0/761719894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f647c102690 0x7f647c19e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.290+0000 7f647b7fe640 1 -- 172.21.15.103:0/761719894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f647c100ad0 msgr2=0x7f647c196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:45.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.290+0000 7f647b7fe640 1 --2- 172.21.15.103:0/761719894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f647c100ad0 0x7f647c196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:45.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.290+0000 7f647b7fe640 1 -- 172.21.15.103:0/761719894 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647c19ebc0 con 0x7f647c101cd0 2024-09-19T06:48:45.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.290+0000 7f647b7fe640 1 --2- 172.21.15.103:0/761719894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f647c101cd0 0x7f647c1974d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6470009230 tx=0x7f6470004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:45.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.290+0000 7f64797fa640 1 -- 172.21.15.103:0/761719894 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647003f030 con 0x7f647c101cd0 2024-09-19T06:48:45.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.290+0000 7f64797fa640 1 -- 172.21.15.103:0/761719894 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6470003ce0 con 0x7f647c101cd0 2024-09-19T06:48:45.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.290+0000 7f64826bd640 1 -- 172.21.15.103:0/761719894 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f647c19ee50 con 0x7f647c101cd0 2024-09-19T06:48:45.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.290+0000 7f64797fa640 1 -- 172.21.15.103:0/761719894 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470035070 con 0x7f647c101cd0 2024-09-19T06:48:45.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.290+0000 7f64826bd640 1 -- 172.21.15.103:0/761719894 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f647c19f320 con 0x7f647c101cd0 2024-09-19T06:48:45.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.291+0000 7f64826bd640 1 -- 172.21.15.103:0/761719894 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f647c107f60 con 0x7f647c101cd0 2024-09-19T06:48:45.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.292+0000 7f64797fa640 1 -- 172.21.15.103:0/761719894 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6470042030 con 0x7f647c101cd0 2024-09-19T06:48:45.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.293+0000 7f64797fa640 1 --2- 172.21.15.103:0/761719894 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f64640a3300 0x7f64640a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:45.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.293+0000 7f647bfff640 1 --2- 172.21.15.103:0/761719894 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f64640a3300 0x7f64640a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:45.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.293+0000 7f64797fa640 1 -- 172.21.15.103:0/761719894 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f64700f00d0 con 0x7f647c101cd0 2024-09-19T06:48:45.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.293+0000 7f647bfff640 1 --2- 172.21.15.103:0/761719894 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f64640a3300 0x7f64640a57c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f647c101b30 tx=0x7f646c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:45.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.297+0000 7f64797fa640 1 -- 172.21.15.103:0/761719894 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f64700bcca0 con 0x7f647c101cd0 2024-09-19T06:48:45.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:44 smithi174 ceph-mon[35637]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:48:45.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.476+0000 7f64826bd640 1 -- 172.21.15.103:0/761719894 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f647c062630 con 0x7f64640a3300 2024-09-19T06:48:45.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.477+0000 7f64797fa640 1 -- 172.21.15.103:0/761719894 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f647c062630 con 0x7f64640a3300 2024-09-19T06:48:45.479 INFO:teuthology.orchestra.run.smithi103.stdout:Backend: cephadm 2024-09-19T06:48:45.479 INFO:teuthology.orchestra.run.smithi103.stdout:Available: Yes 2024-09-19T06:48:45.479 INFO:teuthology.orchestra.run.smithi103.stdout:Paused: No 2024-09-19T06:48:45.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.479+0000 7f64826bd640 1 -- 172.21.15.103:0/761719894 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f64640a3300 msgr2=0x7f64640a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:45.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.479+0000 7f64826bd640 1 --2- 172.21.15.103:0/761719894 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f64640a3300 0x7f64640a57c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f647c101b30 tx=0x7f646c009110 comp rx=0 tx=0).stop 2024-09-19T06:48:45.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.479+0000 7f64826bd640 1 -- 172.21.15.103:0/761719894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f647c101cd0 msgr2=0x7f647c1974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:45.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.479+0000 7f64826bd640 1 --2- 172.21.15.103:0/761719894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f647c101cd0 0x7f647c1974d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6470009230 tx=0x7f6470004430 comp rx=0 tx=0).stop 2024-09-19T06:48:45.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.479+0000 7f64826bd640 1 -- 172.21.15.103:0/761719894 shutdown_connections 2024-09-19T06:48:45.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.479+0000 7f64826bd640 1 --2- 172.21.15.103:0/761719894 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f64640a3300 0x7f64640a57c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:45.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.479+0000 7f64826bd640 1 --2- 172.21.15.103:0/761719894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f647c102690 0x7f647c19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:45.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.479+0000 7f64826bd640 1 --2- 172.21.15.103:0/761719894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f647c101cd0 0x7f647c1974d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:45.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.479+0000 7f64826bd640 1 --2- 172.21.15.103:0/761719894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f647c100ad0 0x7f647c196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:45.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.479+0000 7f64826bd640 1 -- 172.21.15.103:0/761719894 >> 172.21.15.103:0/761719894 conn(0x7f647c0fc280 msgr2=0x7f647c105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:45.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.479+0000 7f64826bd640 1 -- 172.21.15.103:0/761719894 shutdown_connections 2024-09-19T06:48:45.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:45.479+0000 7f64826bd640 1 -- 172.21.15.103:0/761719894 wait complete. 2024-09-19T06:48:46.167 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T06:48:46.414 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:46.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.889+0000 7f895d732640 1 -- 172.21.15.103:0/1529609814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8958102e00 msgr2=0x7f8958103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:46.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.889+0000 7f895d732640 1 --2- 172.21.15.103:0/1529609814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8958102e00 0x7f8958103280 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8940009900 tx=0x7f894002a410 comp rx=0 tx=0).stop 2024-09-19T06:48:46.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.889+0000 7f895d732640 1 -- 172.21.15.103:0/1529609814 shutdown_connections 2024-09-19T06:48:46.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.889+0000 7f895d732640 1 --2- 172.21.15.103:0/1529609814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89581037c0 0x7f895810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:46.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.889+0000 7f895d732640 1 --2- 172.21.15.103:0/1529609814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8958102e00 0x7f8958103280 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:46.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.889+0000 7f895d732640 1 --2- 172.21.15.103:0/1529609814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8958101c00 0x7f8958102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:46.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.889+0000 7f895d732640 1 -- 172.21.15.103:0/1529609814 >> 172.21.15.103:0/1529609814 conn(0x7f89580fd390 msgr2=0x7f89580ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:46.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.889+0000 7f895d732640 1 -- 172.21.15.103:0/1529609814 shutdown_connections 2024-09-19T06:48:46.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.889+0000 7f895d732640 1 -- 172.21.15.103:0/1529609814 wait complete. 2024-09-19T06:48:46.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f895d732640 1 Processor -- start 2024-09-19T06:48:46.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f895d732640 1 -- start start 2024-09-19T06:48:46.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f895d732640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8958101c00 0x7f8958075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:46.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f895d732640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8958102e00 0x7f89580745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:46.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f895d732640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89581037c0 0x7f8958074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:46.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f895d732640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f895810d8e0 con 0x7f89581037c0 2024-09-19T06:48:46.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f895d732640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f895810d760 con 0x7f8958101c00 2024-09-19T06:48:46.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f895d732640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f895810da60 con 0x7f8958102e00 2024-09-19T06:48:46.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f89577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89581037c0 0x7f8958074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:46.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f89577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89581037c0 0x7f8958074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55378/0 (socket says 172.21.15.103:55378) 2024-09-19T06:48:46.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f89577fe640 1 -- 172.21.15.103:0/1683845431 learned_addr learned my addr 172.21.15.103:0/1683845431 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:46.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f8956ffd640 1 --2- 172.21.15.103:0/1683845431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8958101c00 0x7f8958075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:46.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f89567fc640 1 --2- 172.21.15.103:0/1683845431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8958102e00 0x7f89580745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:46.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f89577fe640 1 -- 172.21.15.103:0/1683845431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8958102e00 msgr2=0x7f89580745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:46.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f89577fe640 1 --2- 172.21.15.103:0/1683845431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8958102e00 0x7f89580745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:46.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f89577fe640 1 -- 172.21.15.103:0/1683845431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8958101c00 msgr2=0x7f8958075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:46.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f89577fe640 1 --2- 172.21.15.103:0/1683845431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8958101c00 0x7f8958075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:46.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f89577fe640 1 -- 172.21.15.103:0/1683845431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89580753e0 con 0x7f89581037c0 2024-09-19T06:48:46.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.890+0000 7f8956ffd640 1 --2- 172.21.15.103:0/1683845431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8958101c00 0x7f8958075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:46.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.891+0000 7f89577fe640 1 --2- 172.21.15.103:0/1683845431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89581037c0 0x7f8958074b20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8948009fd0 tx=0x7f894800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:46.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.891+0000 7f894ffff640 1 -- 172.21.15.103:0/1683845431 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948011830 con 0x7f89581037c0 2024-09-19T06:48:46.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.891+0000 7f894ffff640 1 -- 172.21.15.103:0/1683845431 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8948016070 con 0x7f89581037c0 2024-09-19T06:48:46.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.891+0000 7f895d732640 1 -- 172.21.15.103:0/1683845431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89581a20f0 con 0x7f89581037c0 2024-09-19T06:48:46.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.891+0000 7f894ffff640 1 -- 172.21.15.103:0/1683845431 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894800f2e0 con 0x7f89581037c0 2024-09-19T06:48:46.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.891+0000 7f895d732640 1 -- 172.21.15.103:0/1683845431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89581a2680 con 0x7f89581037c0 2024-09-19T06:48:46.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.892+0000 7f895d732640 1 -- 172.21.15.103:0/1683845431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8958108fd0 con 0x7f89581037c0 2024-09-19T06:48:46.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.893+0000 7f894ffff640 1 -- 172.21.15.103:0/1683845431 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f89480119d0 con 0x7f89581037c0 2024-09-19T06:48:46.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.894+0000 7f894ffff640 1 --2- 172.21.15.103:0/1683845431 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f89440a32d0 0x7f89440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:46.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.894+0000 7f8956ffd640 1 --2- 172.21.15.103:0/1683845431 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f89440a32d0 0x7f89440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:46.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.894+0000 7f894ffff640 1 -- 172.21.15.103:0/1683845431 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f89480cefd0 con 0x7f89581037c0 2024-09-19T06:48:46.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.894+0000 7f8956ffd640 1 --2- 172.21.15.103:0/1683845431 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f89440a32d0 0x7f89440a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8958102c60 tx=0x7f895000a800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:46.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:46.898+0000 7f894ffff640 1 -- 172.21.15.103:0/1683845431 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f894809bd50 con 0x7f89581037c0 2024-09-19T06:48:47.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:47.073+0000 7f895d732640 1 -- 172.21.15.103:0/1683845431 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f8958062630 con 0x7f89440a32d0 2024-09-19T06:48:47.076 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:46 smithi103 ceph-mon[31158]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:48:47.076 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:46 smithi103 ceph-mon[31158]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T06:48:47.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:47.086+0000 7f894ffff640 1 -- 172.21.15.103:0/1683845431 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f8958062630 con 0x7f89440a32d0 2024-09-19T06:48:47.088 INFO:teuthology.orchestra.run.smithi103.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T06:48:47.088 INFO:teuthology.orchestra.run.smithi103.stdout:mds.a smithi103 running (95s) 19s ago 94s 17.1M - 19.1.1-361-g2f599a46 d2baa855c067 a036b7678ef6 2024-09-19T06:48:47.088 INFO:teuthology.orchestra.run.smithi103.stdout:mds.b smithi134 running (92s) 25s ago 92s 26.5M - 19.1.1-361-g2f599a46 d2baa855c067 a91eef13aaeb 2024-09-19T06:48:47.088 INFO:teuthology.orchestra.run.smithi103.stdout:mds.c smithi174 running (106s) 25s ago 105s 19.2M - 19.1.1-361-g2f599a46 d2baa855c067 e81cc5d6af58 2024-09-19T06:48:47.088 INFO:teuthology.orchestra.run.smithi103.stdout:mds.d smithi103 running (113s) 19s ago 113s 14.2M - 19.1.1-361-g2f599a46 d2baa855c067 940e4279d328 2024-09-19T06:48:47.088 INFO:teuthology.orchestra.run.smithi103.stdout:mds.e smithi134 running (111s) 25s ago 110s 17.7M - 19.1.1-361-g2f599a46 d2baa855c067 3d782529ff85 2024-09-19T06:48:47.088 INFO:teuthology.orchestra.run.smithi103.stdout:mds.f smithi174 running (108s) 25s ago 108s 20.1M - 19.1.1-361-g2f599a46 d2baa855c067 93a672d45a6f 2024-09-19T06:48:47.088 INFO:teuthology.orchestra.run.smithi103.stdout:mds.g smithi103 running (98s) 19s ago 98s 17.2M - 19.1.1-361-g2f599a46 d2baa855c067 e59a59eb9a94 2024-09-19T06:48:47.088 INFO:teuthology.orchestra.run.smithi103.stdout:mds.h smithi134 running (103s) 25s ago 103s 20.0M - 19.1.1-361-g2f599a46 d2baa855c067 91cf7bec2287 2024-09-19T06:48:47.088 INFO:teuthology.orchestra.run.smithi103.stdout:mds.i smithi174 running (101s) 25s ago 100s 20.3M - 19.1.1-361-g2f599a46 d2baa855c067 8fbbb96b41ec 2024-09-19T06:48:47.088 INFO:teuthology.orchestra.run.smithi103.stdout:mds.j smithi103 running (117s) 19s ago 117s 14.5M - 19.1.1-361-g2f599a46 d2baa855c067 c9305037c62e 2024-09-19T06:48:47.088 INFO:teuthology.orchestra.run.smithi103.stdout:mds.k smithi134 running (2m) 25s ago 2m 15.9M - 19.1.1-361-g2f599a46 d2baa855c067 d2780a11bcfb 2024-09-19T06:48:47.089 INFO:teuthology.orchestra.run.smithi103.stdout:mds.l smithi174 running (2m) 25s ago 2m 16.7M - 19.1.1-361-g2f599a46 d2baa855c067 ad715ddcc610 2024-09-19T06:48:47.089 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.x smithi103 *:9283,8765 running (8m) 19s ago 8m 510M - 19.1.1-361-g2f599a46 d2baa855c067 b4b96839d358 2024-09-19T06:48:47.089 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.y smithi134 *:8443,8765 running (7m) 25s ago 7m 443M - 19.1.1-361-g2f599a46 d2baa855c067 534d03192976 2024-09-19T06:48:47.089 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.z smithi174 *:8443,8765 running (7m) 25s ago 7m 442M - 19.1.1-361-g2f599a46 d2baa855c067 ba463a50c210 2024-09-19T06:48:47.089 INFO:teuthology.orchestra.run.smithi103.stdout:mon.a smithi103 running (8m) 19s ago 9m 52.6M 2048M 19.1.1-361-g2f599a46 d2baa855c067 0b2d620a0814 2024-09-19T06:48:47.089 INFO:teuthology.orchestra.run.smithi103.stdout:mon.b smithi134 running (7m) 25s ago 7m 40.4M 2048M 19.1.1-361-g2f599a46 d2baa855c067 7a24aa27c53b 2024-09-19T06:48:47.089 INFO:teuthology.orchestra.run.smithi103.stdout:mon.c smithi174 running (7m) 25s ago 7m 40.4M 2048M 19.1.1-361-g2f599a46 d2baa855c067 20df60485978 2024-09-19T06:48:47.089 INFO:teuthology.orchestra.run.smithi103.stdout:osd.0 smithi103 running (6m) 19s ago 6m 48.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 19100663f040 2024-09-19T06:48:47.089 INFO:teuthology.orchestra.run.smithi103.stdout:osd.1 smithi134 running (6m) 25s ago 6m 55.5M 4272M 19.1.1-361-g2f599a46 d2baa855c067 719f1614387f 2024-09-19T06:48:47.089 INFO:teuthology.orchestra.run.smithi103.stdout:osd.2 smithi174 running (6m) 25s ago 6m 49.1M 4272M 19.1.1-361-g2f599a46 d2baa855c067 68b1ae96be17 2024-09-19T06:48:47.089 INFO:teuthology.orchestra.run.smithi103.stdout:osd.3 smithi103 running (5m) 19s ago 5m 54.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 cb1e7247b99f 2024-09-19T06:48:47.089 INFO:teuthology.orchestra.run.smithi103.stdout:osd.4 smithi134 running (5m) 25s ago 5m 48.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 8b4812b261e4 2024-09-19T06:48:47.089 INFO:teuthology.orchestra.run.smithi103.stdout:osd.5 smithi174 running (4m) 25s ago 4m 46.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 92483c47a451 2024-09-19T06:48:47.090 INFO:teuthology.orchestra.run.smithi103.stdout:osd.6 smithi103 running (4m) 19s ago 4m 45.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 be69a923ec5d 2024-09-19T06:48:47.090 INFO:teuthology.orchestra.run.smithi103.stdout:osd.7 smithi134 running (4m) 25s ago 4m 54.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 154f7848ea2f 2024-09-19T06:48:47.090 INFO:teuthology.orchestra.run.smithi103.stdout:osd.8 smithi174 running (3m) 25s ago 3m 46.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 06c393ec9db5 2024-09-19T06:48:47.090 INFO:teuthology.orchestra.run.smithi103.stdout:osd.9 smithi103 running (3m) 19s ago 3m 43.9M 4272M 19.1.1-361-g2f599a46 d2baa855c067 7d9e98cfcc59 2024-09-19T06:48:47.090 INFO:teuthology.orchestra.run.smithi103.stdout:osd.10 smithi134 running (2m) 25s ago 2m 46.4M 4272M 19.1.1-361-g2f599a46 d2baa855c067 c739b2754c19 2024-09-19T06:48:47.090 INFO:teuthology.orchestra.run.smithi103.stdout:osd.11 smithi174 running (2m) 25s ago 2m 50.5M 4272M 19.1.1-361-g2f599a46 d2baa855c067 7dfe262d614f 2024-09-19T06:48:47.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:47.088+0000 7f895d732640 1 -- 172.21.15.103:0/1683845431 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f89440a32d0 msgr2=0x7f89440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:47.088+0000 7f895d732640 1 --2- 172.21.15.103:0/1683845431 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f89440a32d0 0x7f89440a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8958102c60 tx=0x7f895000a800 comp rx=0 tx=0).stop 2024-09-19T06:48:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:47.088+0000 7f895d732640 1 -- 172.21.15.103:0/1683845431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89581037c0 msgr2=0x7f8958074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:47.088+0000 7f895d732640 1 --2- 172.21.15.103:0/1683845431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89581037c0 0x7f8958074b20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8948009fd0 tx=0x7f894800b8e0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:47.088+0000 7f895d732640 1 -- 172.21.15.103:0/1683845431 shutdown_connections 2024-09-19T06:48:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:47.088+0000 7f895d732640 1 --2- 172.21.15.103:0/1683845431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89581037c0 0x7f8958074b20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:47.088+0000 7f895d732640 1 --2- 172.21.15.103:0/1683845431 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f89440a32d0 0x7f89440a5790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:47.088+0000 7f895d732640 1 --2- 172.21.15.103:0/1683845431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8958102e00 0x7f89580745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:47.088+0000 7f895d732640 1 --2- 172.21.15.103:0/1683845431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8958101c00 0x7f8958075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:47.088+0000 7f895d732640 1 -- 172.21.15.103:0/1683845431 >> 172.21.15.103:0/1683845431 conn(0x7f89580fd390 msgr2=0x7f89581061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:47.088+0000 7f895d732640 1 -- 172.21.15.103:0/1683845431 shutdown_connections 2024-09-19T06:48:47.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:47.089+0000 7f895d732640 1 -- 172.21.15.103:0/1683845431 wait complete. 2024-09-19T06:48:47.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:46 smithi134 ceph-mon[35356]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:48:47.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:46 smithi134 ceph-mon[35356]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T06:48:47.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:46 smithi174 ceph-mon[35637]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:48:47.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:46 smithi174 ceph-mon[35637]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T06:48:47.709 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T06:48:47.955 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:48.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.455+0000 7ffb3ac7f640 1 -- 172.21.15.103:0/3759863111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb34104ff0 msgr2=0x7ffb341073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:48.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.455+0000 7ffb3ac7f640 1 --2- 172.21.15.103:0/3759863111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb34104ff0 0x7ffb341073e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009ca0 tx=0x7ffb2402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.455+0000 7ffb3ac7f640 1 -- 172.21.15.103:0/3759863111 shutdown_connections 2024-09-19T06:48:48.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.455+0000 7ffb3ac7f640 1 --2- 172.21.15.103:0/3759863111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb34107920 0x7ffb34109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.455+0000 7ffb3ac7f640 1 --2- 172.21.15.103:0/3759863111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb34104ff0 0x7ffb341073e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.455+0000 7ffb3ac7f640 1 --2- 172.21.15.103:0/3759863111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb341026c0 0x7ffb34104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.455+0000 7ffb3ac7f640 1 -- 172.21.15.103:0/3759863111 >> 172.21.15.103:0/3759863111 conn(0x7ffb340fc280 msgr2=0x7ffb340fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:48.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.455+0000 7ffb3ac7f640 1 -- 172.21.15.103:0/3759863111 shutdown_connections 2024-09-19T06:48:48.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.455+0000 7ffb3ac7f640 1 -- 172.21.15.103:0/3759863111 wait complete. 2024-09-19T06:48:48.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.455+0000 7ffb3ac7f640 1 Processor -- start 2024-09-19T06:48:48.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.455+0000 7ffb3ac7f640 1 -- start start 2024-09-19T06:48:48.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb3ac7f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb341026c0 0x7ffb341980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:48.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb3ac7f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb34104ff0 0x7ffb341985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:48.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb3ac7f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb34107920 0x7ffb3419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:48.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb3ac7f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3410d930 con 0x7ffb341026c0 2024-09-19T06:48:48.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb3ac7f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3410d7b0 con 0x7ffb34104ff0 2024-09-19T06:48:48.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb3ac7f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3410dab0 con 0x7ffb34107920 2024-09-19T06:48:48.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb38a1b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb341026c0 0x7ffb341980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:48.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb38a1b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb341026c0 0x7ffb341980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59414/0 (socket says 172.21.15.103:59414) 2024-09-19T06:48:48.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb38a1b640 1 -- 172.21.15.103:0/2806095038 learned_addr learned my addr 172.21.15.103:0/2806095038 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:48.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb3921c640 1 --2- 172.21.15.103:0/2806095038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb34107920 0x7ffb3419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:48.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb33fff640 1 --2- 172.21.15.103:0/2806095038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb34104ff0 0x7ffb341985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:48.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb38a1b640 1 -- 172.21.15.103:0/2806095038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb34107920 msgr2=0x7ffb3419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:48.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb38a1b640 1 --2- 172.21.15.103:0/2806095038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb34107920 0x7ffb3419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb38a1b640 1 -- 172.21.15.103:0/2806095038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb34104ff0 msgr2=0x7ffb341985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:48.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb38a1b640 1 --2- 172.21.15.103:0/2806095038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb34104ff0 0x7ffb341985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb38a1b640 1 -- 172.21.15.103:0/2806095038 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3419fcd0 con 0x7ffb341026c0 2024-09-19T06:48:48.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb33fff640 1 --2- 172.21.15.103:0/2806095038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb34104ff0 0x7ffb341985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:48.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb38a1b640 1 --2- 172.21.15.103:0/2806095038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb341026c0 0x7ffb341980a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00b4a0 tx=0x7ffb1c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:48.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb31ffb640 1 -- 172.21.15.103:0/2806095038 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c014070 con 0x7ffb341026c0 2024-09-19T06:48:48.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb3ac7f640 1 -- 172.21.15.103:0/2806095038 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb3419ffc0 con 0x7ffb341026c0 2024-09-19T06:48:48.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb31ffb640 1 -- 172.21.15.103:0/2806095038 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb1c010af0 con 0x7ffb341026c0 2024-09-19T06:48:48.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.456+0000 7ffb31ffb640 1 -- 172.21.15.103:0/2806095038 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c017d10 con 0x7ffb341026c0 2024-09-19T06:48:48.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.457+0000 7ffb3ac7f640 1 -- 172.21.15.103:0/2806095038 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb341a0490 con 0x7ffb341026c0 2024-09-19T06:48:48.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.458+0000 7ffb3ac7f640 1 -- 172.21.15.103:0/2806095038 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb340fdd70 con 0x7ffb341026c0 2024-09-19T06:48:48.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.459+0000 7ffb31ffb640 1 -- 172.21.15.103:0/2806095038 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ffb1c017070 con 0x7ffb341026c0 2024-09-19T06:48:48.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.459+0000 7ffb31ffb640 1 --2- 172.21.15.103:0/2806095038 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ffb200a3300 0x7ffb200a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:48.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.459+0000 7ffb33fff640 1 --2- 172.21.15.103:0/2806095038 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ffb200a3300 0x7ffb200a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:48.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.459+0000 7ffb31ffb640 1 -- 172.21.15.103:0/2806095038 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ffb1c0cede0 con 0x7ffb341026c0 2024-09-19T06:48:48.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.460+0000 7ffb33fff640 1 --2- 172.21.15.103:0/2806095038 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ffb200a3300 0x7ffb200a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ffb24006010 tx=0x7ffb24000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:48.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.464+0000 7ffb31ffb640 1 -- 172.21.15.103:0/2806095038 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffb1c09bac0 con 0x7ffb341026c0 2024-09-19T06:48:48.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.638+0000 7ffb3ac7f640 1 -- 172.21.15.103:0/2806095038 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7ffb34062630 con 0x7ffb200a3300 2024-09-19T06:48:48.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.644+0000 7ffb31ffb640 1 -- 172.21.15.103:0/2806095038 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7ffb34062630 con 0x7ffb200a3300 2024-09-19T06:48:48.646 INFO:teuthology.orchestra.run.smithi103.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T06:48:48.647 INFO:teuthology.orchestra.run.smithi103.stdout:mds.all 12/12 26s ago 2m smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-19T06:48:48.647 INFO:teuthology.orchestra.run.smithi103.stdout:mgr 3/3 26s ago 7m smithi103=x;smithi134=y;smithi174=z;count:3 2024-09-19T06:48:48.647 INFO:teuthology.orchestra.run.smithi103.stdout:mon 3/3 26s ago 7m smithi103:172.21.15.103=a;smithi134:172.21.15.134=b;smithi174:172.21.15.174=c;count:3 2024-09-19T06:48:48.647 INFO:teuthology.orchestra.run.smithi103.stdout:osd 12 26s ago - 2024-09-19T06:48:48.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.646+0000 7ffb3ac7f640 1 -- 172.21.15.103:0/2806095038 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ffb200a3300 msgr2=0x7ffb200a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:48.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.647+0000 7ffb3ac7f640 1 --2- 172.21.15.103:0/2806095038 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ffb200a3300 0x7ffb200a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ffb24006010 tx=0x7ffb24000f40 comp rx=0 tx=0).stop 2024-09-19T06:48:48.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.647+0000 7ffb3ac7f640 1 -- 172.21.15.103:0/2806095038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb341026c0 msgr2=0x7ffb341980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:48.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.647+0000 7ffb3ac7f640 1 --2- 172.21.15.103:0/2806095038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb341026c0 0x7ffb341980a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00b4a0 tx=0x7ffb1c00b960 comp rx=0 tx=0).stop 2024-09-19T06:48:48.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.647+0000 7ffb3ac7f640 1 -- 172.21.15.103:0/2806095038 shutdown_connections 2024-09-19T06:48:48.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.647+0000 7ffb3ac7f640 1 --2- 172.21.15.103:0/2806095038 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7ffb200a3300 0x7ffb200a57c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.647+0000 7ffb3ac7f640 1 --2- 172.21.15.103:0/2806095038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb34107920 0x7ffb3419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.647+0000 7ffb3ac7f640 1 --2- 172.21.15.103:0/2806095038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb34104ff0 0x7ffb341985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.647+0000 7ffb3ac7f640 1 --2- 172.21.15.103:0/2806095038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb341026c0 0x7ffb341980a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.647+0000 7ffb3ac7f640 1 -- 172.21.15.103:0/2806095038 >> 172.21.15.103:0/2806095038 conn(0x7ffb340fc280 msgr2=0x7ffb341030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:48.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.647+0000 7ffb3ac7f640 1 -- 172.21.15.103:0/2806095038 shutdown_connections 2024-09-19T06:48:48.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:48.647+0000 7ffb3ac7f640 1 -- 172.21.15.103:0/2806095038 wait complete. 2024-09-19T06:48:49.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:48 smithi134 ceph-mon[35356]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:48:49.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:48 smithi134 ceph-mon[35356]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T06:48:49.265 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:48 smithi103 ceph-mon[31158]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:48:49.265 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:48 smithi103 ceph-mon[31158]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T06:48:49.293 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T06:48:49.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:48 smithi174 ceph-mon[35637]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:48:49.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:48 smithi174 ceph-mon[35637]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T06:48:49.538 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:49.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.973+0000 7f9673e7c640 1 -- 172.21.15.103:0/2267206906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c100290 msgr2=0x7f966c100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:49.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.973+0000 7f9673e7c640 1 --2- 172.21.15.103:0/2267206906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c100290 0x7f966c100690 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9668009900 tx=0x7f966802a410 comp rx=0 tx=0).stop 2024-09-19T06:48:49.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.973+0000 7f9673e7c640 1 -- 172.21.15.103:0/2267206906 shutdown_connections 2024-09-19T06:48:49.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.973+0000 7f9673e7c640 1 --2- 172.21.15.103:0/2267206906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f966c101e50 0x7f966c1086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:49.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.973+0000 7f9673e7c640 1 --2- 172.21.15.103:0/2267206906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f966c101490 0x7f966c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:49.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.973+0000 7f9673e7c640 1 --2- 172.21.15.103:0/2267206906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c100290 0x7f966c100690 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:49.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.973+0000 7f9673e7c640 1 -- 172.21.15.103:0/2267206906 >> 172.21.15.103:0/2267206906 conn(0x7f966c0fba40 msgr2=0x7f966c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:49.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.973+0000 7f9673e7c640 1 -- 172.21.15.103:0/2267206906 shutdown_connections 2024-09-19T06:48:49.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.973+0000 7f9673e7c640 1 -- 172.21.15.103:0/2267206906 wait complete. 2024-09-19T06:48:49.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9673e7c640 1 Processor -- start 2024-09-19T06:48:49.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9673e7c640 1 -- start start 2024-09-19T06:48:49.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9673e7c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c100290 0x7f966c104a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:49.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9673e7c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f966c101490 0x7f966c103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:49.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9673e7c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f966c101e50 0x7f966c1035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:49.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9673e7c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f966c10bf70 con 0x7f966c100290 2024-09-19T06:48:49.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9673e7c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f966c10bdf0 con 0x7f966c101490 2024-09-19T06:48:49.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9673e7c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f966c10c0f0 con 0x7f966c101e50 2024-09-19T06:48:49.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9671c18640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c100290 0x7f966c104a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:49.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9671c18640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c100290 0x7f966c104a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59438/0 (socket says 172.21.15.103:59438) 2024-09-19T06:48:49.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9671c18640 1 -- 172.21.15.103:0/2342844569 learned_addr learned my addr 172.21.15.103:0/2342844569 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:49.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9671417640 1 --2- 172.21.15.103:0/2342844569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f966c101490 0x7f966c103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9672419640 1 --2- 172.21.15.103:0/2342844569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f966c101e50 0x7f966c1035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9671c18640 1 -- 172.21.15.103:0/2342844569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f966c101e50 msgr2=0x7f966c1035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9671c18640 1 --2- 172.21.15.103:0/2342844569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f966c101e50 0x7f966c1035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9671c18640 1 -- 172.21.15.103:0/2342844569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f966c101490 msgr2=0x7f966c103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9671c18640 1 --2- 172.21.15.103:0/2342844569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f966c101490 0x7f966c103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9671c18640 1 -- 172.21.15.103:0/2342844569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966c103b10 con 0x7f966c100290 2024-09-19T06:48:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9672419640 1 --2- 172.21.15.103:0/2342844569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f966c101e50 0x7f966c1035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.974+0000 7f9671c18640 1 --2- 172.21.15.103:0/2342844569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c100290 0x7f966c104a40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9668005ab0 tx=0x7f9668004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.975+0000 7f9662ffd640 1 -- 172.21.15.103:0/2342844569 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966803f030 con 0x7f966c100290 2024-09-19T06:48:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.975+0000 7f9673e7c640 1 -- 172.21.15.103:0/2342844569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966c103da0 con 0x7f966c100290 2024-09-19T06:48:49.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.975+0000 7f9662ffd640 1 -- 172.21.15.103:0/2342844569 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9668003ce0 con 0x7f966c100290 2024-09-19T06:48:49.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.975+0000 7f9662ffd640 1 -- 172.21.15.103:0/2342844569 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668035070 con 0x7f966c100290 2024-09-19T06:48:49.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.975+0000 7f9673e7c640 1 -- 172.21.15.103:0/2342844569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f966c1a6a80 con 0x7f966c100290 2024-09-19T06:48:49.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.976+0000 7f9673e7c640 1 -- 172.21.15.103:0/2342844569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f966c100690 con 0x7f966c100290 2024-09-19T06:48:49.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.978+0000 7f9662ffd640 1 -- 172.21.15.103:0/2342844569 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9668042030 con 0x7f966c100290 2024-09-19T06:48:49.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.978+0000 7f9662ffd640 1 --2- 172.21.15.103:0/2342844569 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f96580a32d0 0x7f96580a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:49.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.978+0000 7f9671417640 1 --2- 172.21.15.103:0/2342844569 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f96580a32d0 0x7f96580a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:49.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.978+0000 7f9662ffd640 1 -- 172.21.15.103:0/2342844569 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f96680f0490 con 0x7f966c100290 2024-09-19T06:48:49.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.979+0000 7f9671417640 1 --2- 172.21.15.103:0/2342844569 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f96580a32d0 0x7f96580a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f966c1047c0 tx=0x7f965c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:49.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:49.981+0000 7f9662ffd640 1 -- 172.21.15.103:0/2342844569 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f96680bd060 con 0x7f966c100290 2024-09-19T06:48:50.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:50.156+0000 7f9673e7c640 1 -- 172.21.15.103:0/2342844569 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f966c0fd7d0 con 0x7f96580a32d0 2024-09-19T06:48:50.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:50.157+0000 7f9662ffd640 1 -- 172.21.15.103:0/2342844569 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f966c0fd7d0 con 0x7f96580a32d0 2024-09-19T06:48:50.159 INFO:teuthology.orchestra.run.smithi103.stdout:HOST ADDR LABELS STATUS 2024-09-19T06:48:50.159 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103 172.21.15.103 2024-09-19T06:48:50.160 INFO:teuthology.orchestra.run.smithi103.stdout:smithi134 172.21.15.134 2024-09-19T06:48:50.160 INFO:teuthology.orchestra.run.smithi103.stdout:smithi174 172.21.15.174 2024-09-19T06:48:50.160 INFO:teuthology.orchestra.run.smithi103.stdout:3 hosts in cluster 2024-09-19T06:48:50.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:50.159+0000 7f9673e7c640 1 -- 172.21.15.103:0/2342844569 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f96580a32d0 msgr2=0x7f96580a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:50.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:50.159+0000 7f9673e7c640 1 --2- 172.21.15.103:0/2342844569 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f96580a32d0 0x7f96580a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f966c1047c0 tx=0x7f965c008040 comp rx=0 tx=0).stop 2024-09-19T06:48:50.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:50.159+0000 7f9673e7c640 1 -- 172.21.15.103:0/2342844569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c100290 msgr2=0x7f966c104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:50.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:50.159+0000 7f9673e7c640 1 --2- 172.21.15.103:0/2342844569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c100290 0x7f966c104a40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9668005ab0 tx=0x7f9668004360 comp rx=0 tx=0).stop 2024-09-19T06:48:50.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:50.159+0000 7f9673e7c640 1 -- 172.21.15.103:0/2342844569 shutdown_connections 2024-09-19T06:48:50.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:50.159+0000 7f9673e7c640 1 --2- 172.21.15.103:0/2342844569 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f96580a32d0 0x7f96580a5790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:50.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:50.159+0000 7f9673e7c640 1 --2- 172.21.15.103:0/2342844569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f966c101e50 0x7f966c1035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:50.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:50.160+0000 7f9673e7c640 1 --2- 172.21.15.103:0/2342844569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f966c101490 0x7f966c103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:50.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:50.160+0000 7f9673e7c640 1 --2- 172.21.15.103:0/2342844569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c100290 0x7f966c104a40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:50.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:50.160+0000 7f9673e7c640 1 -- 172.21.15.103:0/2342844569 >> 172.21.15.103:0/2342844569 conn(0x7f966c0fba40 msgr2=0x7f966c0fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:50.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:50.160+0000 7f9673e7c640 1 -- 172.21.15.103:0/2342844569 shutdown_connections 2024-09-19T06:48:50.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:50.160+0000 7f9673e7c640 1 -- 172.21.15.103:0/2342844569 wait complete. 2024-09-19T06:48:50.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:49 smithi134 ceph-mon[35356]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:48:50.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:49 smithi174 ceph-mon[35637]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:48:50.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:49 smithi103 ceph-mon[31158]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:48:50.792 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T06:48:51.040 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:51.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:50 smithi103 ceph-mon[31158]: pgmap v317: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:48:51.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:50 smithi134 ceph-mon[35356]: pgmap v317: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:48:51.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:50 smithi174 ceph-mon[35637]: pgmap v317: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:48:51.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.497+0000 7f39f0094640 1 -- 172.21.15.103:0/2199809599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39e81026d0 msgr2=0x7f39e8108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.497+0000 7f39f0094640 1 --2- 172.21.15.103:0/2199809599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39e81026d0 0x7f39e8108f60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f39e0009ca0 tx=0x7f39e002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:48:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.497+0000 7f39f0094640 1 -- 172.21.15.103:0/2199809599 shutdown_connections 2024-09-19T06:48:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.497+0000 7f39f0094640 1 --2- 172.21.15.103:0/2199809599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39e81026d0 0x7f39e8108f60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.497+0000 7f39f0094640 1 --2- 172.21.15.103:0/2199809599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39e8101d10 0x7f39e8102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.497+0000 7f39f0094640 1 --2- 172.21.15.103:0/2199809599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39e8100b10 0x7f39e8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.497+0000 7f39f0094640 1 -- 172.21.15.103:0/2199809599 >> 172.21.15.103:0/2199809599 conn(0x7f39e80fc2a0 msgr2=0x7f39e80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.497+0000 7f39f0094640 1 -- 172.21.15.103:0/2199809599 shutdown_connections 2024-09-19T06:48:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.497+0000 7f39f0094640 1 -- 172.21.15.103:0/2199809599 wait complete. 2024-09-19T06:48:51.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.498+0000 7f39f0094640 1 Processor -- start 2024-09-19T06:48:51.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.498+0000 7f39f0094640 1 -- start start 2024-09-19T06:48:51.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.498+0000 7f39f0094640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39e8100b10 0x7f39e8196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:51.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.498+0000 7f39f0094640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39e8101d10 0x7f39e8197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:51.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.498+0000 7f39f0094640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39e81026d0 0x7f39e819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:51.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.498+0000 7f39f0094640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f39e810c7f0 con 0x7f39e8100b10 2024-09-19T06:48:51.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.498+0000 7f39f0094640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f39e810c670 con 0x7f39e8101d10 2024-09-19T06:48:51.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.498+0000 7f39f0094640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f39e810c970 con 0x7f39e81026d0 2024-09-19T06:48:51.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39ede30640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39e8100b10 0x7f39e8196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:51.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39ede30640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39e8100b10 0x7f39e8196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59450/0 (socket says 172.21.15.103:59450) 2024-09-19T06:48:51.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39ede30640 1 -- 172.21.15.103:0/2810865397 learned_addr learned my addr 172.21.15.103:0/2810865397 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:51.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39ed62f640 1 --2- 172.21.15.103:0/2810865397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39e8101d10 0x7f39e8197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:51.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39ee631640 1 --2- 172.21.15.103:0/2810865397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39e81026d0 0x7f39e819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:51.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39ede30640 1 -- 172.21.15.103:0/2810865397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39e81026d0 msgr2=0x7f39e819e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:51.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39ede30640 1 --2- 172.21.15.103:0/2810865397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39e81026d0 0x7f39e819e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:51.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39ede30640 1 -- 172.21.15.103:0/2810865397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39e8101d10 msgr2=0x7f39e8197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39ede30640 1 --2- 172.21.15.103:0/2810865397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39e8101d10 0x7f39e8197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39ede30640 1 -- 172.21.15.103:0/2810865397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e819ea40 con 0x7f39e8100b10 2024-09-19T06:48:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39ee631640 1 --2- 172.21.15.103:0/2810865397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39e81026d0 0x7f39e819e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39ed62f640 1 --2- 172.21.15.103:0/2810865397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39e8101d10 0x7f39e8197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39ede30640 1 --2- 172.21.15.103:0/2810865397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39e8100b10 0x7f39e8196e10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f39e400d950 tx=0x7f39e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39daffd640 1 -- 172.21.15.103:0/2810865397 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e4017020 con 0x7f39e8100b10 2024-09-19T06:48:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39daffd640 1 -- 172.21.15.103:0/2810865397 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e4009c30 con 0x7f39e8100b10 2024-09-19T06:48:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39f0094640 1 -- 172.21.15.103:0/2810865397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e819ed30 con 0x7f39e8100b10 2024-09-19T06:48:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39daffd640 1 -- 172.21.15.103:0/2810865397 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e400f720 con 0x7f39e8100b10 2024-09-19T06:48:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.499+0000 7f39f0094640 1 -- 172.21.15.103:0/2810865397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e819f200 con 0x7f39e8100b10 2024-09-19T06:48:51.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.500+0000 7f39f0094640 1 -- 172.21.15.103:0/2810865397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e8107ec0 con 0x7f39e8100b10 2024-09-19T06:48:51.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.502+0000 7f39daffd640 1 -- 172.21.15.103:0/2810865397 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f39e401a020 con 0x7f39e8100b10 2024-09-19T06:48:51.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.502+0000 7f39daffd640 1 --2- 172.21.15.103:0/2810865397 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f39d40a3380 0x7f39d40a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:51.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.502+0000 7f39ed62f640 1 --2- 172.21.15.103:0/2810865397 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f39d40a3380 0x7f39d40a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:51.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.502+0000 7f39daffd640 1 -- 172.21.15.103:0/2810865397 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f39e40cf9f0 con 0x7f39e8100b10 2024-09-19T06:48:51.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.503+0000 7f39ed62f640 1 --2- 172.21.15.103:0/2810865397 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f39d40a3380 0x7f39d40a5840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39e8198330 tx=0x7f39dc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:51.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.506+0000 7f39daffd640 1 -- 172.21.15.103:0/2810865397 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f39e4021020 con 0x7f39e8100b10 2024-09-19T06:48:51.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.680+0000 7f39f0094640 1 -- 172.21.15.103:0/2810865397 --> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f39e8062630 con 0x7f39d40a3380 2024-09-19T06:48:51.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.682+0000 7f39daffd640 1 -- 172.21.15.103:0/2810865397 <== mgr.14150 v2:172.21.15.103:6800/2590139910 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f39e8062630 con 0x7f39d40a3380 2024-09-19T06:48:51.683 INFO:teuthology.orchestra.run.smithi103.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T06:48:51.683 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204012Z400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-19T06:48:51.683 INFO:teuthology.orchestra.run.smithi103.stdout:smithi134 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62420002400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T06:48:51.683 INFO:teuthology.orchestra.run.smithi103.stdout:smithi174 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050074400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T06:48:51.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.684+0000 7f39f0094640 1 -- 172.21.15.103:0/2810865397 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f39d40a3380 msgr2=0x7f39d40a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:51.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.684+0000 7f39f0094640 1 --2- 172.21.15.103:0/2810865397 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f39d40a3380 0x7f39d40a5840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39e8198330 tx=0x7f39dc008040 comp rx=0 tx=0).stop 2024-09-19T06:48:51.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.684+0000 7f39f0094640 1 -- 172.21.15.103:0/2810865397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39e8100b10 msgr2=0x7f39e8196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:51.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.684+0000 7f39f0094640 1 --2- 172.21.15.103:0/2810865397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39e8100b10 0x7f39e8196e10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f39e400d950 tx=0x7f39e400de10 comp rx=0 tx=0).stop 2024-09-19T06:48:51.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.684+0000 7f39f0094640 1 -- 172.21.15.103:0/2810865397 shutdown_connections 2024-09-19T06:48:51.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.684+0000 7f39f0094640 1 --2- 172.21.15.103:0/2810865397 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f39d40a3380 0x7f39d40a5840 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:51.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.684+0000 7f39f0094640 1 --2- 172.21.15.103:0/2810865397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39e81026d0 0x7f39e819e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:51.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.684+0000 7f39f0094640 1 --2- 172.21.15.103:0/2810865397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39e8101d10 0x7f39e8197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:51.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.684+0000 7f39f0094640 1 --2- 172.21.15.103:0/2810865397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39e8100b10 0x7f39e8196e10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:51.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.684+0000 7f39f0094640 1 -- 172.21.15.103:0/2810865397 >> 172.21.15.103:0/2810865397 conn(0x7f39e80fc2a0 msgr2=0x7f39e8105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:51.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.684+0000 7f39f0094640 1 -- 172.21.15.103:0/2810865397 shutdown_connections 2024-09-19T06:48:51.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:51.684+0000 7f39f0094640 1 -- 172.21.15.103:0/2810865397 wait complete. 2024-09-19T06:48:52.184 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:51 smithi103 ceph-mon[31158]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:48:52.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:51 smithi134 ceph-mon[35356]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:48:52.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:51 smithi174 ceph-mon[35637]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:48:52.338 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T06:48:52.346 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T06:48:52.347 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T06:48:52.593 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:52.983 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:52 smithi103 ceph-mon[31158]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:48:52.983 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:52 smithi103 ceph-mon[31158]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:48:53.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.062+0000 7f3c5cd8e640 1 -- 172.21.15.103:0/4202122447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c58101c00 msgr2=0x7f3c58102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:53.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.062+0000 7f3c5cd8e640 1 --2- 172.21.15.103:0/4202122447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c58101c00 0x7f3c58102000 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009900 tx=0x7f3c4c02a410 comp rx=0 tx=0).stop 2024-09-19T06:48:53.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.062+0000 7f3c5cd8e640 1 -- 172.21.15.103:0/4202122447 shutdown_connections 2024-09-19T06:48:53.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.062+0000 7f3c5cd8e640 1 --2- 172.21.15.103:0/4202122447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c581037c0 0x7f3c5810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:53.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.062+0000 7f3c5cd8e640 1 --2- 172.21.15.103:0/4202122447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c58102e00 0x7f3c58103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:53.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.062+0000 7f3c5cd8e640 1 --2- 172.21.15.103:0/4202122447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c58101c00 0x7f3c58102000 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:53.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.062+0000 7f3c5cd8e640 1 -- 172.21.15.103:0/4202122447 >> 172.21.15.103:0/4202122447 conn(0x7f3c580fd390 msgr2=0x7f3c580ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:53.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.062+0000 7f3c5cd8e640 1 -- 172.21.15.103:0/4202122447 shutdown_connections 2024-09-19T06:48:53.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.062+0000 7f3c5cd8e640 1 -- 172.21.15.103:0/4202122447 wait complete. 2024-09-19T06:48:53.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.063+0000 7f3c5cd8e640 1 Processor -- start 2024-09-19T06:48:53.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.063+0000 7f3c5cd8e640 1 -- start start 2024-09-19T06:48:53.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.063+0000 7f3c5cd8e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c58101c00 0x7f3c58197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:53.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.063+0000 7f3c5cd8e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c58102e00 0x7f3c581983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:53.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.063+0000 7f3c5cd8e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c581037c0 0x7f3c5819f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:53.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.063+0000 7f3c5cd8e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5810d860 con 0x7f3c581037c0 2024-09-19T06:48:53.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.063+0000 7f3c5cd8e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5810d6e0 con 0x7f3c58101c00 2024-09-19T06:48:53.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.063+0000 7f3c5cd8e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5810d9e0 con 0x7f3c58102e00 2024-09-19T06:48:53.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.063+0000 7f3c56d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c581037c0 0x7f3c5819f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:53.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.063+0000 7f3c56d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c581037c0 0x7f3c5819f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59468/0 (socket says 172.21.15.103:59468) 2024-09-19T06:48:53.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.063+0000 7f3c56d9d640 1 -- 172.21.15.103:0/384784361 learned_addr learned my addr 172.21.15.103:0/384784361 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:53.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.063+0000 7f3c5659c640 1 --2- 172.21.15.103:0/384784361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c58101c00 0x7f3c58197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:53.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.063+0000 7f3c55d9b640 1 --2- 172.21.15.103:0/384784361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c58102e00 0x7f3c581983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:53.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.063+0000 7f3c56d9d640 1 -- 172.21.15.103:0/384784361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c58102e00 msgr2=0x7f3c581983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:53.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.064+0000 7f3c56d9d640 1 --2- 172.21.15.103:0/384784361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c58102e00 0x7f3c581983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:53.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.064+0000 7f3c56d9d640 1 -- 172.21.15.103:0/384784361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c58101c00 msgr2=0x7f3c58197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:53.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.064+0000 7f3c56d9d640 1 --2- 172.21.15.103:0/384784361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c58101c00 0x7f3c58197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:53.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.064+0000 7f3c56d9d640 1 -- 172.21.15.103:0/384784361 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5819fae0 con 0x7f3c581037c0 2024-09-19T06:48:53.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.064+0000 7f3c55d9b640 1 --2- 172.21.15.103:0/384784361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c58102e00 0x7f3c581983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:53.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.064+0000 7f3c5659c640 1 --2- 172.21.15.103:0/384784361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c58101c00 0x7f3c58197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:53.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.064+0000 7f3c56d9d640 1 --2- 172.21.15.103:0/384784361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c581037c0 0x7f3c5819f470 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000cce0 tx=0x7f3c5000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:53.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.064+0000 7f3c477fe640 1 -- 172.21.15.103:0/384784361 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5000f450 con 0x7f3c581037c0 2024-09-19T06:48:53.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.064+0000 7f3c5cd8e640 1 -- 172.21.15.103:0/384784361 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c5819fdd0 con 0x7f3c581037c0 2024-09-19T06:48:53.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.064+0000 7f3c477fe640 1 -- 172.21.15.103:0/384784361 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c5000fad0 con 0x7f3c581037c0 2024-09-19T06:48:53.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.064+0000 7f3c477fe640 1 -- 172.21.15.103:0/384784361 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50018c10 con 0x7f3c581037c0 2024-09-19T06:48:53.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.064+0000 7f3c5cd8e640 1 -- 172.21.15.103:0/384784361 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c581a0280 con 0x7f3c581037c0 2024-09-19T06:48:53.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.065+0000 7f3c5cd8e640 1 -- 172.21.15.103:0/384784361 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c58108fa0 con 0x7f3c581037c0 2024-09-19T06:48:53.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.066+0000 7f3c477fe640 1 -- 172.21.15.103:0/384784361 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3c5000f5f0 con 0x7f3c581037c0 2024-09-19T06:48:53.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.067+0000 7f3c477fe640 1 --2- 172.21.15.103:0/384784361 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3c400a3300 0x7f3c400a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:53.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.067+0000 7f3c5659c640 1 --2- 172.21.15.103:0/384784361 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3c400a3300 0x7f3c400a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:53.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.067+0000 7f3c477fe640 1 -- 172.21.15.103:0/384784361 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3c500d01d0 con 0x7f3c581037c0 2024-09-19T06:48:53.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.067+0000 7f3c5659c640 1 --2- 172.21.15.103:0/384784361 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3c400a3300 0x7f3c400a57c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c000c00 tx=0x7f3c4c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:53.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.071+0000 7f3c477fe640 1 -- 172.21.15.103:0/384784361 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3c50014030 con 0x7f3c581037c0 2024-09-19T06:48:53.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:52 smithi134 ceph-mon[35356]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:48:53.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:52 smithi134 ceph-mon[35356]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:48:53.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.273+0000 7f3c5cd8e640 1 -- 172.21.15.103:0/384784361 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f3c5802d7c0 con 0x7f3c581037c0 2024-09-19T06:48:53.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.273+0000 7f3c477fe640 1 -- 172.21.15.103:0/384784361 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3142 (secure 0 0 0) 0x7f3c5009cda0 con 0x7f3c581037c0 2024-09-19T06:48:53.274 INFO:teuthology.orchestra.run.smithi103.stdout:e26 2024-09-19T06:48:53.274 INFO:teuthology.orchestra.run.smithi103.stdout:btime 2024-09-19T06:47:37:459674+0000 2024-09-19T06:48:53.274 INFO:teuthology.orchestra.run.smithi103.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T06:48:53.275 INFO:teuthology.orchestra.run.smithi103.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T06:48:53.275 INFO:teuthology.orchestra.run.smithi103.stdout:legacy client fscid: 1 2024-09-19T06:48:53.275 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:48:53.275 INFO:teuthology.orchestra.run.smithi103.stdout:Filesystem 'cephfs' (1) 2024-09-19T06:48:53.275 INFO:teuthology.orchestra.run.smithi103.stdout:fs_name cephfs 2024-09-19T06:48:53.275 INFO:teuthology.orchestra.run.smithi103.stdout:epoch 26 2024-09-19T06:48:53.275 INFO:teuthology.orchestra.run.smithi103.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T06:48:53.275 INFO:teuthology.orchestra.run.smithi103.stdout:created 2024-09-19T06:47:25.446176+0000 2024-09-19T06:48:53.275 INFO:teuthology.orchestra.run.smithi103.stdout:modified 2024-09-19T06:47:36.481871+0000 2024-09-19T06:48:53.275 INFO:teuthology.orchestra.run.smithi103.stdout:tableserver 0 2024-09-19T06:48:53.275 INFO:teuthology.orchestra.run.smithi103.stdout:root 0 2024-09-19T06:48:53.275 INFO:teuthology.orchestra.run.smithi103.stdout:session_timeout 300 2024-09-19T06:48:53.275 INFO:teuthology.orchestra.run.smithi103.stdout:session_autoclose 300 2024-09-19T06:48:53.275 INFO:teuthology.orchestra.run.smithi103.stdout:max_file_size 1099511627776 2024-09-19T06:48:53.275 INFO:teuthology.orchestra.run.smithi103.stdout:max_xattr_size 65536 2024-09-19T06:48:53.276 INFO:teuthology.orchestra.run.smithi103.stdout:required_client_features {} 2024-09-19T06:48:53.276 INFO:teuthology.orchestra.run.smithi103.stdout:last_failure 0 2024-09-19T06:48:53.276 INFO:teuthology.orchestra.run.smithi103.stdout:last_failure_osd_epoch 0 2024-09-19T06:48:53.276 INFO:teuthology.orchestra.run.smithi103.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T06:48:53.276 INFO:teuthology.orchestra.run.smithi103.stdout:max_mds 5 2024-09-19T06:48:53.276 INFO:teuthology.orchestra.run.smithi103.stdout:in 0,1,2,3,4 2024-09-19T06:48:53.276 INFO:teuthology.orchestra.run.smithi103.stdout:up {0=24482,1=24455,2=24454,3=24436,4=24431} 2024-09-19T06:48:53.276 INFO:teuthology.orchestra.run.smithi103.stdout:failed 2024-09-19T06:48:53.276 INFO:teuthology.orchestra.run.smithi103.stdout:damaged 2024-09-19T06:48:53.276 INFO:teuthology.orchestra.run.smithi103.stdout:stopped 2024-09-19T06:48:53.276 INFO:teuthology.orchestra.run.smithi103.stdout:data_pools [3] 2024-09-19T06:48:53.276 INFO:teuthology.orchestra.run.smithi103.stdout:metadata_pool 2 2024-09-19T06:48:53.276 INFO:teuthology.orchestra.run.smithi103.stdout:inline_data disabled 2024-09-19T06:48:53.276 INFO:teuthology.orchestra.run.smithi103.stdout:balancer 2024-09-19T06:48:53.276 INFO:teuthology.orchestra.run.smithi103.stdout:bal_rank_mask -1 2024-09-19T06:48:53.277 INFO:teuthology.orchestra.run.smithi103.stdout:standby_count_wanted 0 2024-09-19T06:48:53.277 INFO:teuthology.orchestra.run.smithi103.stdout:qdb_cluster leader: 24482 members: 24482,24455,24454,24436,24431 2024-09-19T06:48:53.277 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.b{0:24482} state up:active seq 4 addr [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:48:53.277 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.f{0:24430} state up:standby-replay seq 1 addr [v2:172.21.15.174:6834/963732203,v1:172.21.15.174:6835/963732203] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:48:53.277 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.h{1:24455} state up:active seq 9 addr [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:48:53.277 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.k{1:24407} state up:standby-replay seq 1 addr [v2:172.21.15.134:6832/1556743259,v1:172.21.15.134:6833/1556743259] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:48:53.277 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.i{2:24454} state up:active seq 8 addr [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:48:53.277 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.l{2:24397} state up:standby-replay seq 1 addr [v2:172.21.15.174:6832/2319563070,v1:172.21.15.174:6833/2319563070] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:48:53.277 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.c{3:24436} state up:active seq 10 addr [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:48:53.277 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.a{3:14598} state up:standby-replay seq 1 addr [v2:172.21.15.103:6840/3771437304,v1:172.21.15.103:6841/3771437304] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:48:53.277 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.e{4:24431} state up:active seq 11 addr [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:48:53.277 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.g{4:14586} state up:standby-replay seq 1 addr [v2:172.21.15.103:6838/3119326077,v1:172.21.15.103:6839/3119326077] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:48:53.277 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:48:53.277 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:48:53.278 INFO:teuthology.orchestra.run.smithi103.stdout:Standby daemons: 2024-09-19T06:48:53.278 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:48:53.278 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.j{-1:14526} state up:standby seq 1 addr [v2:172.21.15.103:6834/4290108970,v1:172.21.15.103:6835/4290108970] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:48:53.278 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.d{-1:14538} state up:standby seq 1 addr [v2:172.21.15.103:6836/323370484,v1:172.21.15.103:6837/323370484] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:48:53.278 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-19T06:48:53.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.275+0000 7f3c5cd8e640 1 -- 172.21.15.103:0/384784361 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3c400a3300 msgr2=0x7f3c400a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:53.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.275+0000 7f3c5cd8e640 1 --2- 172.21.15.103:0/384784361 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3c400a3300 0x7f3c400a57c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c000c00 tx=0x7f3c4c031400 comp rx=0 tx=0).stop 2024-09-19T06:48:53.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.275+0000 7f3c5cd8e640 1 -- 172.21.15.103:0/384784361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c581037c0 msgr2=0x7f3c5819f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:53.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.275+0000 7f3c5cd8e640 1 --2- 172.21.15.103:0/384784361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c581037c0 0x7f3c5819f470 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000cce0 tx=0x7f3c5000a620 comp rx=0 tx=0).stop 2024-09-19T06:48:53.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.275+0000 7f3c5cd8e640 1 -- 172.21.15.103:0/384784361 shutdown_connections 2024-09-19T06:48:53.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.275+0000 7f3c5cd8e640 1 --2- 172.21.15.103:0/384784361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c581037c0 0x7f3c5819f470 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:53.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.275+0000 7f3c5cd8e640 1 --2- 172.21.15.103:0/384784361 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f3c400a3300 0x7f3c400a57c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:53.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.275+0000 7f3c5cd8e640 1 --2- 172.21.15.103:0/384784361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c58102e00 0x7f3c581983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:53.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.275+0000 7f3c5cd8e640 1 --2- 172.21.15.103:0/384784361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c58101c00 0x7f3c58197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:53.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.275+0000 7f3c5cd8e640 1 -- 172.21.15.103:0/384784361 >> 172.21.15.103:0/384784361 conn(0x7f3c580fd390 msgr2=0x7f3c58106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:53.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.276+0000 7f3c5cd8e640 1 -- 172.21.15.103:0/384784361 shutdown_connections 2024-09-19T06:48:53.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:53.276+0000 7f3c5cd8e640 1 -- 172.21.15.103:0/384784361 wait complete. 2024-09-19T06:48:53.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:52 smithi174 ceph-mon[35637]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:48:53.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:52 smithi174 ceph-mon[35637]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:48:53.925 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d158d934-7651-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T06:48:54.173 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:54.293 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:53 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/384784361' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T06:48:54.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:53 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/384784361' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T06:48:54.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:53 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/384784361' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T06:48:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.643+0000 7f879b102640 1 -- 172.21.15.103:0/575528428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8794101490 msgr2=0x7f8794101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.643+0000 7f879b102640 1 --2- 172.21.15.103:0/575528428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8794101490 0x7f8794101910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8780009900 tx=0x7f878002a410 comp rx=0 tx=0).stop 2024-09-19T06:48:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.643+0000 7f879b102640 1 -- 172.21.15.103:0/575528428 shutdown_connections 2024-09-19T06:48:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.643+0000 7f879b102640 1 --2- 172.21.15.103:0/575528428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8794101e50 0x7f87941086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.643+0000 7f879b102640 1 --2- 172.21.15.103:0/575528428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8794101490 0x7f8794101910 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.643+0000 7f879b102640 1 --2- 172.21.15.103:0/575528428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8794100290 0x7f8794100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.643+0000 7f879b102640 1 -- 172.21.15.103:0/575528428 >> 172.21.15.103:0/575528428 conn(0x7f87940fba40 msgr2=0x7f87940fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.643+0000 7f879b102640 1 -- 172.21.15.103:0/575528428 shutdown_connections 2024-09-19T06:48:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.643+0000 7f879b102640 1 -- 172.21.15.103:0/575528428 wait complete. 2024-09-19T06:48:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.643+0000 7f879b102640 1 Processor -- start 2024-09-19T06:48:54.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f879b102640 1 -- start start 2024-09-19T06:48:54.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f879b102640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8794100290 0x7f8794104220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:54.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f879b102640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8794101490 0x7f8794102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:54.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f879b102640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8794101e50 0x7f8794102db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:54.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f879b102640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f879410b6a0 con 0x7f8794100290 2024-09-19T06:48:54.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f879b102640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f879410b520 con 0x7f8794101490 2024-09-19T06:48:54.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f879b102640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f879410b820 con 0x7f8794101e50 2024-09-19T06:48:54.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f8798e9e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8794100290 0x7f8794104220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:54.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f8798e9e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8794100290 0x7f8794104220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59488/0 (socket says 172.21.15.103:59488) 2024-09-19T06:48:54.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f8798e9e640 1 -- 172.21.15.103:0/3318593143 learned_addr learned my addr 172.21.15.103:0/3318593143 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:54.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f879969f640 1 --2- 172.21.15.103:0/3318593143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8794101e50 0x7f8794102db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:54.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f8793fff640 1 --2- 172.21.15.103:0/3318593143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8794101490 0x7f8794102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:54.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f8798e9e640 1 -- 172.21.15.103:0/3318593143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8794101e50 msgr2=0x7f8794102db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f8798e9e640 1 --2- 172.21.15.103:0/3318593143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8794101e50 0x7f8794102db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f8798e9e640 1 -- 172.21.15.103:0/3318593143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8794101490 msgr2=0x7f8794102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f8798e9e640 1 --2- 172.21.15.103:0/3318593143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8794101490 0x7f8794102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f8798e9e640 1 -- 172.21.15.103:0/3318593143 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8794103670 con 0x7f8794100290 2024-09-19T06:48:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f8798e9e640 1 --2- 172.21.15.103:0/3318593143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8794100290 0x7f8794104220 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f878800c970 tx=0x7f878800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f8791ffb640 1 -- 172.21.15.103:0/3318593143 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788007c10 con 0x7f8794100290 2024-09-19T06:48:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f8791ffb640 1 -- 172.21.15.103:0/3318593143 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8788007db0 con 0x7f8794100290 2024-09-19T06:48:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f879b102640 1 -- 172.21.15.103:0/3318593143 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87941a1ff0 con 0x7f8794100290 2024-09-19T06:48:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.644+0000 7f8791ffb640 1 -- 172.21.15.103:0/3318593143 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878800f940 con 0x7f8794100290 2024-09-19T06:48:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.645+0000 7f879b102640 1 -- 172.21.15.103:0/3318593143 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87941a23e0 con 0x7f8794100290 2024-09-19T06:48:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.646+0000 7f879b102640 1 -- 172.21.15.103:0/3318593143 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87941a2670 con 0x7f8794100290 2024-09-19T06:48:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.647+0000 7f8791ffb640 1 -- 172.21.15.103:0/3318593143 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f878800fae0 con 0x7f8794100290 2024-09-19T06:48:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.647+0000 7f8791ffb640 1 --2- 172.21.15.103:0/3318593143 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f87840a32d0 0x7f87840a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.648+0000 7f8793fff640 1 --2- 172.21.15.103:0/3318593143 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f87840a32d0 0x7f87840a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:54.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.648+0000 7f8791ffb640 1 -- 172.21.15.103:0/3318593143 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f87880cff20 con 0x7f8794100290 2024-09-19T06:48:54.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.648+0000 7f8793fff640 1 --2- 172.21.15.103:0/3318593143 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f87840a32d0 0x7f87840a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8794103fa0 tx=0x7f87800314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:54.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.652+0000 7f8791ffb640 1 -- 172.21.15.103:0/3318593143 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f878809cb80 con 0x7f8794100290 2024-09-19T06:48:54.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.822+0000 7f879b102640 1 -- 172.21.15.103:0/3318593143 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f87941a2840 con 0x7f8794100290 2024-09-19T06:48:54.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.822+0000 7f8791ffb640 1 -- 172.21.15.103:0/3318593143 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) ==== 56+0+5312 (secure 0 0 0) 0x7f878809c480 con 0x7f8794100290 2024-09-19T06:48:54.823 INFO:teuthology.orchestra.run.smithi103.stdout:epoch 82 2024-09-19T06:48:54.824 INFO:teuthology.orchestra.run.smithi103.stdout:fsid d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T06:48:54.824 INFO:teuthology.orchestra.run.smithi103.stdout:created 2024-09-19T06:39:46.840574+0000 2024-09-19T06:48:54.824 INFO:teuthology.orchestra.run.smithi103.stdout:modified 2024-09-19T06:47:40.819637+0000 2024-09-19T06:48:54.824 INFO:teuthology.orchestra.run.smithi103.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T06:48:54.824 INFO:teuthology.orchestra.run.smithi103.stdout:crush_version 26 2024-09-19T06:48:54.824 INFO:teuthology.orchestra.run.smithi103.stdout:full_ratio 0.95 2024-09-19T06:48:54.824 INFO:teuthology.orchestra.run.smithi103.stdout:backfillfull_ratio 0.9 2024-09-19T06:48:54.824 INFO:teuthology.orchestra.run.smithi103.stdout:nearfull_ratio 0.85 2024-09-19T06:48:54.824 INFO:teuthology.orchestra.run.smithi103.stdout:require_min_compat_client luminous 2024-09-19T06:48:54.824 INFO:teuthology.orchestra.run.smithi103.stdout:min_compat_client luminous 2024-09-19T06:48:54.824 INFO:teuthology.orchestra.run.smithi103.stdout:require_osd_release squid 2024-09-19T06:48:54.824 INFO:teuthology.orchestra.run.smithi103.stdout:stretch_mode_enabled false 2024-09-19T06:48:54.824 INFO:teuthology.orchestra.run.smithi103.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T06:48:54.825 INFO:teuthology.orchestra.run.smithi103.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-19T06:48:54.825 INFO:teuthology.orchestra.run.smithi103.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-19T06:48:54.825 INFO:teuthology.orchestra.run.smithi103.stdout:max_osd 12 2024-09-19T06:48:54.825 INFO:teuthology.orchestra.run.smithi103.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6802/3157697740,v1:172.21.15.103:6803/3157697740] [v2:172.21.15.103:6804/3157697740,v1:172.21.15.103:6805/3157697740] exists,up 5d45dab3-e434-4783-b2e3-f658c3ba583b 2024-09-19T06:48:54.825 INFO:teuthology.orchestra.run.smithi103.stdout:osd.1 up in weight 1 up_from 13 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6800/1418301195,v1:172.21.15.134:6801/1418301195] [v2:172.21.15.134:6802/1418301195,v1:172.21.15.134:6803/1418301195] exists,up ee0e92bd-15ab-4867-9a18-20abb9f5fc86 2024-09-19T06:48:54.825 INFO:teuthology.orchestra.run.smithi103.stdout:osd.2 up in weight 1 up_from 17 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6800/2292242290,v1:172.21.15.174:6801/2292242290] [v2:172.21.15.174:6802/2292242290,v1:172.21.15.174:6803/2292242290] exists,up 0230caed-025b-46e9-a011-237601d16ac5 2024-09-19T06:48:54.825 INFO:teuthology.orchestra.run.smithi103.stdout:osd.3 up in weight 1 up_from 24 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6810/2698986333,v1:172.21.15.103:6811/2698986333] [v2:172.21.15.103:6812/2698986333,v1:172.21.15.103:6813/2698986333] exists,up c54a8399-bf4b-4d70-b207-a13d85f99590 2024-09-19T06:48:54.825 INFO:teuthology.orchestra.run.smithi103.stdout:osd.4 up in weight 1 up_from 30 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6808/3169891968,v1:172.21.15.134:6809/3169891968] [v2:172.21.15.134:6810/3169891968,v1:172.21.15.134:6811/3169891968] exists,up 7c046be6-464a-4ab5-812b-1781121fc317 2024-09-19T06:48:54.825 INFO:teuthology.orchestra.run.smithi103.stdout:osd.5 up in weight 1 up_from 35 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6808/763749644,v1:172.21.15.174:6809/763749644] [v2:172.21.15.174:6810/763749644,v1:172.21.15.174:6811/763749644] exists,up 8622596b-6a06-459c-882d-50fba9e296ed 2024-09-19T06:48:54.825 INFO:teuthology.orchestra.run.smithi103.stdout:osd.6 up in weight 1 up_from 42 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6818/1396972394,v1:172.21.15.103:6819/1396972394] [v2:172.21.15.103:6820/1396972394,v1:172.21.15.103:6821/1396972394] exists,up 64d858d6-a8b3-4f65-a187-af9ab1ccee0f 2024-09-19T06:48:54.825 INFO:teuthology.orchestra.run.smithi103.stdout:osd.7 up in weight 1 up_from 47 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6816/2196366765,v1:172.21.15.134:6817/2196366765] [v2:172.21.15.134:6818/2196366765,v1:172.21.15.134:6819/2196366765] exists,up d405d1c6-839c-4c92-8261-4adb3163c90f 2024-09-19T06:48:54.825 INFO:teuthology.orchestra.run.smithi103.stdout:osd.8 up in weight 1 up_from 54 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6816/189977465,v1:172.21.15.174:6817/189977465] [v2:172.21.15.174:6818/189977465,v1:172.21.15.174:6819/189977465] exists,up 4c452cd9-816c-4f77-9368-2cd4bbeee4ff 2024-09-19T06:48:54.825 INFO:teuthology.orchestra.run.smithi103.stdout:osd.9 up in weight 1 up_from 62 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6826/3161966413,v1:172.21.15.103:6827/3161966413] [v2:172.21.15.103:6828/3161966413,v1:172.21.15.103:6829/3161966413] exists,up fb3662e0-2d8e-45c4-9119-6d08b35443da 2024-09-19T06:48:54.826 INFO:teuthology.orchestra.run.smithi103.stdout:osd.10 up in weight 1 up_from 67 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6824/2532758632,v1:172.21.15.134:6825/2532758632] [v2:172.21.15.134:6826/2532758632,v1:172.21.15.134:6827/2532758632] exists,up e52c3cbe-a6af-441e-b423-b94f3a2da544 2024-09-19T06:48:54.826 INFO:teuthology.orchestra.run.smithi103.stdout:osd.11 up in weight 1 up_from 74 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6824/161916442,v1:172.21.15.174:6825/161916442] [v2:172.21.15.174:6826/161916442,v1:172.21.15.174:6827/161916442] exists,up 9040f3f8-3873-4dea-8716-dcda5811c8d1 2024-09-19T06:48:54.826 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.6 [8,11] 2024-09-19T06:48:54.826 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.e [2,4] 2024-09-19T06:48:54.826 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.10 [2,4] 2024-09-19T06:48:54.826 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.12 [5,11] 2024-09-19T06:48:54.826 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.1f [8,1] 2024-09-19T06:48:54.826 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.13 [4,9] 2024-09-19T06:48:54.826 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.1e [3,9] 2024-09-19T06:48:54.826 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.2c [4,9] 2024-09-19T06:48:54.826 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/2867049580 expires 2024-09-20T06:40:39.739638+0000 2024-09-19T06:48:54.826 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/3727259725 expires 2024-09-20T06:40:39.739638+0000 2024-09-19T06:48:54.826 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6800/4122140716 expires 2024-09-20T06:40:39.739638+0000 2024-09-19T06:48:54.826 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6801/4122140716 expires 2024-09-20T06:40:39.739638+0000 2024-09-19T06:48:54.826 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/1496153385 expires 2024-09-20T06:40:39.739638+0000 2024-09-19T06:48:54.827 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6801/2003588522 expires 2024-09-20T06:40:16.780890+0000 2024-09-19T06:48:54.827 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6800/2003588522 expires 2024-09-20T06:40:16.780890+0000 2024-09-19T06:48:54.827 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/1642809745 expires 2024-09-20T06:40:16.780890+0000 2024-09-19T06:48:54.827 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/2309624637 expires 2024-09-20T06:40:16.780890+0000 2024-09-19T06:48:54.827 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/1596229922 expires 2024-09-20T06:40:16.780890+0000 2024-09-19T06:48:54.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.825+0000 7f879b102640 1 -- 172.21.15.103:0/3318593143 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f87840a32d0 msgr2=0x7f87840a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:54.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.825+0000 7f879b102640 1 --2- 172.21.15.103:0/3318593143 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f87840a32d0 0x7f87840a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8794103fa0 tx=0x7f87800314a0 comp rx=0 tx=0).stop 2024-09-19T06:48:54.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.825+0000 7f879b102640 1 -- 172.21.15.103:0/3318593143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8794100290 msgr2=0x7f8794104220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:54.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.825+0000 7f879b102640 1 --2- 172.21.15.103:0/3318593143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8794100290 0x7f8794104220 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f878800c970 tx=0x7f878800ce30 comp rx=0 tx=0).stop 2024-09-19T06:48:54.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.825+0000 7f879b102640 1 -- 172.21.15.103:0/3318593143 shutdown_connections 2024-09-19T06:48:54.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.825+0000 7f879b102640 1 --2- 172.21.15.103:0/3318593143 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f87840a32d0 0x7f87840a5790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:54.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.825+0000 7f879b102640 1 --2- 172.21.15.103:0/3318593143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8794101e50 0x7f8794102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:54.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.825+0000 7f879b102640 1 --2- 172.21.15.103:0/3318593143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8794101490 0x7f8794102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:54.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.825+0000 7f879b102640 1 --2- 172.21.15.103:0/3318593143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8794100290 0x7f8794104220 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:54.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.825+0000 7f879b102640 1 -- 172.21.15.103:0/3318593143 >> 172.21.15.103:0/3318593143 conn(0x7f87940fba40 msgr2=0x7f87941069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:54.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.825+0000 7f879b102640 1 -- 172.21.15.103:0/3318593143 shutdown_connections 2024-09-19T06:48:54.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:54.825+0000 7f879b102640 1 -- 172.21.15.103:0/3318593143 wait complete. 2024-09-19T06:48:55.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:54 smithi103 ceph-mon[31158]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:48:55.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:54 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3318593143' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T06:48:55.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:54 smithi174 ceph-mon[35637]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:48:55.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:54 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3318593143' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T06:48:55.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:54 smithi134 ceph-mon[35356]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:48:55.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:54 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3318593143' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T06:48:55.472 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T06:48:55.484 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:48:55.731 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:56.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.233+0000 7feac2ee9640 1 -- 172.21.15.103:0/2138125017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feabc1055b0 msgr2=0x7feabc10d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:56.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.233+0000 7feac2ee9640 1 --2- 172.21.15.103:0/2138125017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feabc1055b0 0x7feabc10d130 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7feab4009ca0 tx=0x7feab402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:48:56.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.233+0000 7feac2ee9640 1 -- 172.21.15.103:0/2138125017 shutdown_connections 2024-09-19T06:48:56.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.233+0000 7feac2ee9640 1 --2- 172.21.15.103:0/2138125017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feabc1055b0 0x7feabc10d130 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:56.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.233+0000 7feac2ee9640 1 --2- 172.21.15.103:0/2138125017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feabc100e30 0x7feabc104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:56.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.233+0000 7feac2ee9640 1 --2- 172.21.15.103:0/2138125017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feabc100480 0x7feabc100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:56.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.233+0000 7feac2ee9640 1 -- 172.21.15.103:0/2138125017 >> 172.21.15.103:0/2138125017 conn(0x7feabc077210 msgr2=0x7feabc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:56.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.233+0000 7feac2ee9640 1 -- 172.21.15.103:0/2138125017 shutdown_connections 2024-09-19T06:48:56.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.233+0000 7feac2ee9640 1 -- 172.21.15.103:0/2138125017 wait complete. 2024-09-19T06:48:56.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.233+0000 7feac2ee9640 1 Processor -- start 2024-09-19T06:48:56.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feac2ee9640 1 -- start start 2024-09-19T06:48:56.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feac2ee9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feabc100480 0x7feabc1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:56.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feac2ee9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feabc100e30 0x7feabc1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:56.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feac2ee9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feabc1055b0 0x7feabc1a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:56.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feac2ee9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feabc1106b0 con 0x7feabc100e30 2024-09-19T06:48:56.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feac2ee9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feabc110530 con 0x7feabc100480 2024-09-19T06:48:56.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feac2ee9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feabc110830 con 0x7feabc1055b0 2024-09-19T06:48:56.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feabbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feabc100e30 0x7feabc1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:56.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feabbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feabc100e30 0x7feabc1998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59512/0 (socket says 172.21.15.103:59512) 2024-09-19T06:48:56.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feabbfff640 1 -- 172.21.15.103:0/2041474494 learned_addr learned my addr 172.21.15.103:0/2041474494 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:56.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feac0c85640 1 --2- 172.21.15.103:0/2041474494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feabc100480 0x7feabc1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:56.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feac1486640 1 --2- 172.21.15.103:0/2041474494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feabc1055b0 0x7feabc1a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:56.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feabbfff640 1 -- 172.21.15.103:0/2041474494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feabc1055b0 msgr2=0x7feabc1a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:56.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feabbfff640 1 --2- 172.21.15.103:0/2041474494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feabc1055b0 0x7feabc1a4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:56.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feabbfff640 1 -- 172.21.15.103:0/2041474494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feabc100480 msgr2=0x7feabc1993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:56.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feabbfff640 1 --2- 172.21.15.103:0/2041474494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feabc100480 0x7feabc1993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:56.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feabbfff640 1 -- 172.21.15.103:0/2041474494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feabc1a4970 con 0x7feabc100e30 2024-09-19T06:48:56.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feac1486640 1 --2- 172.21.15.103:0/2041474494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feabc1055b0 0x7feabc1a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:56.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feac0c85640 1 --2- 172.21.15.103:0/2041474494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feabc100480 0x7feabc1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:56.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feabbfff640 1 --2- 172.21.15.103:0/2041474494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feabc100e30 0x7feabc1998f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7feaa800b4a0 tx=0x7feaa800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:56.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feab9ffb640 1 -- 172.21.15.103:0/2041474494 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8014070 con 0x7feabc100e30 2024-09-19T06:48:56.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.234+0000 7feac2ee9640 1 -- 172.21.15.103:0/2041474494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feabc1a4b40 con 0x7feabc100e30 2024-09-19T06:48:56.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.235+0000 7feab9ffb640 1 -- 172.21.15.103:0/2041474494 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa8010af0 con 0x7feabc100e30 2024-09-19T06:48:56.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.235+0000 7feab9ffb640 1 -- 172.21.15.103:0/2041474494 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8017d10 con 0x7feabc100e30 2024-09-19T06:48:56.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.235+0000 7feac2ee9640 1 -- 172.21.15.103:0/2041474494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feabc1a5080 con 0x7feabc100e30 2024-09-19T06:48:56.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.236+0000 7feac2ee9640 1 -- 172.21.15.103:0/2041474494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feabc1008e0 con 0x7feabc100e30 2024-09-19T06:48:56.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.237+0000 7feab9ffb640 1 -- 172.21.15.103:0/2041474494 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7feaa8017070 con 0x7feabc100e30 2024-09-19T06:48:56.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.237+0000 7feab9ffb640 1 --2- 172.21.15.103:0/2041474494 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feaac0a32d0 0x7feaac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:56.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.237+0000 7feac0c85640 1 --2- 172.21.15.103:0/2041474494 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feaac0a32d0 0x7feaac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:56.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.238+0000 7feab9ffb640 1 -- 172.21.15.103:0/2041474494 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7feaa80ceda0 con 0x7feabc100e30 2024-09-19T06:48:56.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.238+0000 7feac0c85640 1 --2- 172.21.15.103:0/2041474494 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feaac0a32d0 0x7feaac0a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feab0006fd0 tx=0x7feab0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:56.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.242+0000 7feab9ffb640 1 -- 172.21.15.103:0/2041474494 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feaa809ba80 con 0x7feabc100e30 2024-09-19T06:48:56.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.443+0000 7feac2ee9640 1 -- 172.21.15.103:0/2041474494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feabc1a52b0 con 0x7feabc100e30 2024-09-19T06:48:56.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.443+0000 7feab9ffb640 1 -- 172.21.15.103:0/2041474494 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11994 (secure 0 0 0) 0x7feaa809b380 con 0x7feabc100e30 2024-09-19T06:48:56.444 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:48:56.445 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-19T06:47:37:459674+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:47:36.481871+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24455,24454,24436,24431]},"id":1}]} 2024-09-19T06:48:56.445 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-19T06:48:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.445+0000 7feac2ee9640 1 -- 172.21.15.103:0/2041474494 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feaac0a32d0 msgr2=0x7feaac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.445+0000 7feac2ee9640 1 --2- 172.21.15.103:0/2041474494 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feaac0a32d0 0x7feaac0a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feab0006fd0 tx=0x7feab0008040 comp rx=0 tx=0).stop 2024-09-19T06:48:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.445+0000 7feac2ee9640 1 -- 172.21.15.103:0/2041474494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feabc100e30 msgr2=0x7feabc1998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.445+0000 7feac2ee9640 1 --2- 172.21.15.103:0/2041474494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feabc100e30 0x7feabc1998f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7feaa800b4a0 tx=0x7feaa800b960 comp rx=0 tx=0).stop 2024-09-19T06:48:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.445+0000 7feac2ee9640 1 -- 172.21.15.103:0/2041474494 shutdown_connections 2024-09-19T06:48:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.445+0000 7feac2ee9640 1 --2- 172.21.15.103:0/2041474494 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7feaac0a32d0 0x7feaac0a5790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.446+0000 7feac2ee9640 1 --2- 172.21.15.103:0/2041474494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feabc1055b0 0x7feabc1a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.446+0000 7feac2ee9640 1 --2- 172.21.15.103:0/2041474494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feabc100e30 0x7feabc1998f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.446+0000 7feac2ee9640 1 --2- 172.21.15.103:0/2041474494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feabc100480 0x7feabc1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.446+0000 7feac2ee9640 1 -- 172.21.15.103:0/2041474494 >> 172.21.15.103:0/2041474494 conn(0x7feabc077210 msgr2=0x7feabc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.446+0000 7feac2ee9640 1 -- 172.21.15.103:0/2041474494 shutdown_connections 2024-09-19T06:48:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:56.446+0000 7feac2ee9640 1 -- 172.21.15.103:0/2041474494 wait complete. 2024-09-19T06:48:57.103 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:56 smithi103 ceph-mon[31158]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T06:48:57.103 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:56 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2041474494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:57.132 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:48:57.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:56 smithi174 ceph-mon[35637]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T06:48:57.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:56 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2041474494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:57.379 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:57.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:56 smithi134 ceph-mon[35356]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T06:48:57.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:56 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2041474494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:57.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.776+0000 7fdae683c640 1 -- 172.21.15.103:0/849442304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae0100e20 msgr2=0x7fdae0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:57.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.776+0000 7fdae683c640 1 --2- 172.21.15.103:0/849442304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae0100e20 0x7fdae0101200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009900 tx=0x7fdad002a480 comp rx=0 tx=0).stop 2024-09-19T06:48:57.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.776+0000 7fdae683c640 1 -- 172.21.15.103:0/849442304 shutdown_connections 2024-09-19T06:48:57.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.776+0000 7fdae683c640 1 --2- 172.21.15.103:0/849442304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdae0102ef0 0x7fdae010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:57.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.776+0000 7fdae683c640 1 --2- 172.21.15.103:0/849442304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae0102570 0x7fdae01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:57.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.776+0000 7fdae683c640 1 --2- 172.21.15.103:0/849442304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae0100e20 0x7fdae0101200 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:57.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.776+0000 7fdae683c640 1 -- 172.21.15.103:0/849442304 >> 172.21.15.103:0/849442304 conn(0x7fdae00fc7d0 msgr2=0x7fdae00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:57.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.776+0000 7fdae683c640 1 -- 172.21.15.103:0/849442304 shutdown_connections 2024-09-19T06:48:57.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.776+0000 7fdae683c640 1 -- 172.21.15.103:0/849442304 wait complete. 2024-09-19T06:48:57.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.776+0000 7fdae683c640 1 Processor -- start 2024-09-19T06:48:57.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.776+0000 7fdae683c640 1 -- start start 2024-09-19T06:48:57.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdae683c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdae0100e20 0x7fdae0198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:57.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdae683c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae0102570 0x7fdae0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:57.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdae683c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae0102ef0 0x7fdae019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:57.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdae683c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdae0110fc0 con 0x7fdae0102570 2024-09-19T06:48:57.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdae683c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdae0110e40 con 0x7fdae0100e20 2024-09-19T06:48:57.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdae683c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdae0111140 con 0x7fdae0102ef0 2024-09-19T06:48:57.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae0102570 0x7fdae0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:57.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae0102570 0x7fdae0198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36432/0 (socket says 172.21.15.103:36432) 2024-09-19T06:48:57.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdad7fff640 1 -- 172.21.15.103:0/95361246 learned_addr learned my addr 172.21.15.103:0/95361246 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:57.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdadffff640 1 --2- 172.21.15.103:0/95361246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdae0100e20 0x7fdae0198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:57.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdae4dd9640 1 --2- 172.21.15.103:0/95361246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae0102ef0 0x7fdae019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:57.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdad7fff640 1 -- 172.21.15.103:0/95361246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae0102ef0 msgr2=0x7fdae019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:57.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdad7fff640 1 --2- 172.21.15.103:0/95361246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae0102ef0 0x7fdae019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:57.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdad7fff640 1 -- 172.21.15.103:0/95361246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdae0100e20 msgr2=0x7fdae0198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:57.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdad7fff640 1 --2- 172.21.15.103:0/95361246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdae0100e20 0x7fdae0198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:57.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdad7fff640 1 -- 172.21.15.103:0/95361246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae019d4b0 con 0x7fdae0102570 2024-09-19T06:48:57.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdadffff640 1 --2- 172.21.15.103:0/95361246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdae0100e20 0x7fdae0198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:57.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdad7fff640 1 --2- 172.21.15.103:0/95361246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae0102570 0x7fdae0198a40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdac800d770 tx=0x7fdac800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:57.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdaddffb640 1 -- 172.21.15.103:0/95361246 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8017070 con 0x7fdae0102570 2024-09-19T06:48:57.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdaddffb640 1 -- 172.21.15.103:0/95361246 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac8005500 con 0x7fdae0102570 2024-09-19T06:48:57.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdaddffb640 1 -- 172.21.15.103:0/95361246 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8004e00 con 0x7fdae0102570 2024-09-19T06:48:57.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.777+0000 7fdae683c640 1 -- 172.21.15.103:0/95361246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae019d7a0 con 0x7fdae0102570 2024-09-19T06:48:57.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.778+0000 7fdae683c640 1 -- 172.21.15.103:0/95361246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae0103680 con 0x7fdae0102570 2024-09-19T06:48:57.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.778+0000 7fdae683c640 1 -- 172.21.15.103:0/95361246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae0101200 con 0x7fdae0102570 2024-09-19T06:48:57.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.780+0000 7fdaddffb640 1 -- 172.21.15.103:0/95361246 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fdac8004fa0 con 0x7fdae0102570 2024-09-19T06:48:57.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.780+0000 7fdaddffb640 1 --2- 172.21.15.103:0/95361246 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdacc0a3350 0x7fdacc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:57.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.780+0000 7fdadffff640 1 --2- 172.21.15.103:0/95361246 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdacc0a3350 0x7fdacc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:57.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.780+0000 7fdaddffb640 1 -- 172.21.15.103:0/95361246 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdac80cf510 con 0x7fdae0102570 2024-09-19T06:48:57.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.781+0000 7fdadffff640 1 --2- 172.21.15.103:0/95361246 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdacc0a3350 0x7fdacc0a5810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdad0005ab0 tx=0x7fdad0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:57.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.784+0000 7fdaddffb640 1 -- 172.21.15.103:0/95361246 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdac809c0e0 con 0x7fdae0102570 2024-09-19T06:48:57.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.988+0000 7fdae683c640 1 -- 172.21.15.103:0/95361246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdae01038e0 con 0x7fdae0102570 2024-09-19T06:48:57.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.989+0000 7fdaddffb640 1 -- 172.21.15.103:0/95361246 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11994 (secure 0 0 0) 0x7fdac809b9e0 con 0x7fdae0102570 2024-09-19T06:48:57.990 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:48:57.990 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-19T06:47:37:459674+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:47:36.481871+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24455,24454,24436,24431]},"id":1}]} 2024-09-19T06:48:57.991 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-19T06:48:57.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.991+0000 7fdae683c640 1 -- 172.21.15.103:0/95361246 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdacc0a3350 msgr2=0x7fdacc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:57.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.991+0000 7fdae683c640 1 --2- 172.21.15.103:0/95361246 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdacc0a3350 0x7fdacc0a5810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdad0005ab0 tx=0x7fdad0005a20 comp rx=0 tx=0).stop 2024-09-19T06:48:57.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.991+0000 7fdae683c640 1 -- 172.21.15.103:0/95361246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae0102570 msgr2=0x7fdae0198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:57.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.991+0000 7fdae683c640 1 --2- 172.21.15.103:0/95361246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae0102570 0x7fdae0198a40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdac800d770 tx=0x7fdac800dc30 comp rx=0 tx=0).stop 2024-09-19T06:48:57.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.991+0000 7fdae683c640 1 -- 172.21.15.103:0/95361246 shutdown_connections 2024-09-19T06:48:57.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.991+0000 7fdae683c640 1 --2- 172.21.15.103:0/95361246 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fdacc0a3350 0x7fdacc0a5810 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:57.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.991+0000 7fdae683c640 1 --2- 172.21.15.103:0/95361246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae0102ef0 0x7fdae019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:57.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.991+0000 7fdae683c640 1 --2- 172.21.15.103:0/95361246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae0102570 0x7fdae0198a40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:57.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.991+0000 7fdae683c640 1 --2- 172.21.15.103:0/95361246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdae0100e20 0x7fdae0198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:57.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.991+0000 7fdae683c640 1 -- 172.21.15.103:0/95361246 >> 172.21.15.103:0/95361246 conn(0x7fdae00fc7d0 msgr2=0x7fdae006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:57.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.991+0000 7fdae683c640 1 -- 172.21.15.103:0/95361246 shutdown_connections 2024-09-19T06:48:57.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:57.991+0000 7fdae683c640 1 -- 172.21.15.103:0/95361246 wait complete. 2024-09-19T06:48:58.659 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:48:58.908 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:59.197 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:58 smithi103 ceph-mon[31158]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T06:48:59.199 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:58 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/95361246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:59.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:58 smithi174 ceph-mon[35637]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T06:48:59.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:58 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/95361246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:59.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.385+0000 7f0baa974640 1 -- 172.21.15.103:0/865674394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ba4102790 msgr2=0x7f0ba41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:59.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.385+0000 7f0baa974640 1 --2- 172.21.15.103:0/865674394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ba4102790 0x7f0ba41067e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009900 tx=0x7f0b9802a410 comp rx=0 tx=0).stop 2024-09-19T06:48:59.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.386+0000 7f0baa974640 1 -- 172.21.15.103:0/865674394 shutdown_connections 2024-09-19T06:48:59.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.386+0000 7f0baa974640 1 --2- 172.21.15.103:0/865674394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ba4106f10 0x7f0ba410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:59.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.386+0000 7f0baa974640 1 --2- 172.21.15.103:0/865674394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ba4102790 0x7f0ba41067e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:59.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.386+0000 7f0baa974640 1 --2- 172.21.15.103:0/865674394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ba4101de0 0x7f0ba41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:59.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.386+0000 7f0baa974640 1 -- 172.21.15.103:0/865674394 >> 172.21.15.103:0/865674394 conn(0x7f0ba40fd8c0 msgr2=0x7f0ba40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:59.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.386+0000 7f0baa974640 1 -- 172.21.15.103:0/865674394 shutdown_connections 2024-09-19T06:48:59.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.386+0000 7f0baa974640 1 -- 172.21.15.103:0/865674394 wait complete. 2024-09-19T06:48:59.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.386+0000 7f0baa974640 1 Processor -- start 2024-09-19T06:48:59.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.386+0000 7f0baa974640 1 -- start start 2024-09-19T06:48:59.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0baa974640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ba4101de0 0x7f0ba41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:59.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0baa974640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ba4102790 0x7f0ba4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:59.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0baa974640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ba4106f10 0x7f0ba419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:59.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0baa974640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba4112010 con 0x7f0ba4106f10 2024-09-19T06:48:59.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0baa974640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba4111e90 con 0x7f0ba4101de0 2024-09-19T06:48:59.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0baa974640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba4112190 con 0x7f0ba4102790 2024-09-19T06:48:59.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0ba8f11640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ba4106f10 0x7f0ba419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:59.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0ba8f11640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ba4106f10 0x7f0ba419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36440/0 (socket says 172.21.15.103:36440) 2024-09-19T06:48:59.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0ba8f11640 1 -- 172.21.15.103:0/3995883401 learned_addr learned my addr 172.21.15.103:0/3995883401 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:48:59.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0ba3fff640 1 --2- 172.21.15.103:0/3995883401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ba4101de0 0x7f0ba41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:59.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0ba37fe640 1 --2- 172.21.15.103:0/3995883401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ba4102790 0x7f0ba4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:59.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0ba8f11640 1 -- 172.21.15.103:0/3995883401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ba4102790 msgr2=0x7f0ba4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:59.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0ba8f11640 1 --2- 172.21.15.103:0/3995883401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ba4102790 0x7f0ba4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:59.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0ba8f11640 1 -- 172.21.15.103:0/3995883401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ba4101de0 msgr2=0x7f0ba41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:59.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0ba8f11640 1 --2- 172.21.15.103:0/3995883401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ba4101de0 0x7f0ba41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:59.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0ba8f11640 1 -- 172.21.15.103:0/3995883401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba419d560 con 0x7f0ba4106f10 2024-09-19T06:48:59.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0ba3fff640 1 --2- 172.21.15.103:0/3995883401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ba4101de0 0x7f0ba41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:59.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0ba8f11640 1 --2- 172.21.15.103:0/3995883401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ba4106f10 0x7f0ba419ce80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00b970 tx=0x7f0b9c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:59.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0ba17fa640 1 -- 172.21.15.103:0/3995883401 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c019070 con 0x7f0ba4106f10 2024-09-19T06:48:59.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0baa974640 1 -- 172.21.15.103:0/3995883401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba419d850 con 0x7f0ba4106f10 2024-09-19T06:48:59.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0ba17fa640 1 -- 172.21.15.103:0/3995883401 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b9c00cb40 con 0x7f0ba4106f10 2024-09-19T06:48:59.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0ba17fa640 1 -- 172.21.15.103:0/3995883401 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c007610 con 0x7f0ba4106f10 2024-09-19T06:48:59.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.387+0000 7f0baa974640 1 -- 172.21.15.103:0/3995883401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba41a5130 con 0x7f0ba4106f10 2024-09-19T06:48:59.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.388+0000 7f0baa974640 1 -- 172.21.15.103:0/3995883401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba4102240 con 0x7f0ba4106f10 2024-09-19T06:48:59.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.389+0000 7f0ba17fa640 1 -- 172.21.15.103:0/3995883401 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0b9c01c070 con 0x7f0ba4106f10 2024-09-19T06:48:59.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.390+0000 7f0ba17fa640 1 --2- 172.21.15.103:0/3995883401 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f0b8c0a32d0 0x7f0b8c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:59.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.390+0000 7f0ba3fff640 1 --2- 172.21.15.103:0/3995883401 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f0b8c0a32d0 0x7f0b8c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:59.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.390+0000 7f0ba17fa640 1 -- 172.21.15.103:0/3995883401 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0b9c0cefc0 con 0x7f0ba4106f10 2024-09-19T06:48:59.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.391+0000 7f0ba3fff640 1 --2- 172.21.15.103:0/3995883401 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f0b8c0a32d0 0x7f0b8c0a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009930 tx=0x7f0b94008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:59.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.394+0000 7f0ba17fa640 1 -- 172.21.15.103:0/3995883401 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0b9c09bca0 con 0x7f0ba4106f10 2024-09-19T06:48:59.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:58 smithi134 ceph-mon[35356]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T06:48:59.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:58 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/95361246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.563+0000 7f0baa974640 1 -- 172.21.15.103:0/3995883401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ba4109b60 con 0x7f0ba4106f10 2024-09-19T06:48:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.564+0000 7f0ba17fa640 1 -- 172.21.15.103:0/3995883401 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22729 (secure 0 0 0) 0x7f0b9c09b5a0 con 0x7f0ba4106f10 2024-09-19T06:48:59.566 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:48:59.566 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":82,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","created":"2024-09-19T06:39:46.840574+0000","modified":"2024-09-19T06:47:40.819637+0000","last_up_change":"2024-09-19T06:46:35.652960+0000","last_in_change":"2024-09-19T06:46:11.630585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:42:52.050881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:47:21.192611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:47:23.643291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d45dab3-e434-4783-b2e3-f658c3ba583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6803","nonce":3157697740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6805","nonce":3157697740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6809","nonce":3157697740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6807","nonce":3157697740}]},"public_addr":"172.21.15.103:6803/3157697740","cluster_addr":"172.21.15.103:6805/3157697740","heartbeat_back_addr":"172.21.15.103:6809/3157697740","heartbeat_front_addr":"172.21.15.103:6807/3157697740","state":["exists","up"]},{"osd":1,"uuid":"ee0e92bd-15ab-4867-9a18-20abb9f5fc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6801","nonce":1418301195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6803","nonce":1418301195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6807","nonce":1418301195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6805","nonce":1418301195}]},"public_addr":"172.21.15.134:6801/1418301195","cluster_addr":"172.21.15.134:6803/1418301195","heartbeat_back_addr":"172.21.15.134:6807/1418301195","heartbeat_front_addr":"172.21.15.134:6805/1418301195","state":["exists","up"]},{"osd":2,"uuid":"0230caed-025b-46e9-a011-237601d16ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6801","nonce":2292242290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6803","nonce":2292242290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6807","nonce":2292242290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6805","nonce":2292242290}]},"public_addr":"172.21.15.174:6801/2292242290","cluster_addr":"172.21.15.174:6803/2292242290","heartbeat_back_addr":"172.21.15.174:6807/2292242290","heartbeat_front_addr":"172.21.15.174:6805/2292242290","state":["exists","up"]},{"osd":3,"uuid":"c54a8399-bf4b-4d70-b207-a13d85f99590","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6811","nonce":2698986333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6813","nonce":2698986333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6817","nonce":2698986333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6815","nonce":2698986333}]},"public_addr":"172.21.15.103:6811/2698986333","cluster_addr":"172.21.15.103:6813/2698986333","heartbeat_back_addr":"172.21.15.103:6817/2698986333","heartbeat_front_addr":"172.21.15.103:6815/2698986333","state":["exists","up"]},{"osd":4,"uuid":"7c046be6-464a-4ab5-812b-1781121fc317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6809","nonce":3169891968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6811","nonce":3169891968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6815","nonce":3169891968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6813","nonce":3169891968}]},"public_addr":"172.21.15.134:6809/3169891968","cluster_addr":"172.21.15.134:6811/3169891968","heartbeat_back_addr":"172.21.15.134:6815/3169891968","heartbeat_front_addr":"172.21.15.134:6813/3169891968","state":["exists","up"]},{"osd":5,"uuid":"8622596b-6a06-459c-882d-50fba9e296ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6809","nonce":763749644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6811","nonce":763749644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6815","nonce":763749644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6813","nonce":763749644}]},"public_addr":"172.21.15.174:6809/763749644","cluster_addr":"172.21.15.174:6811/763749644","heartbeat_back_addr":"172.21.15.174:6815/763749644","heartbeat_front_addr":"172.21.15.174:6813/763749644","state":["exists","up"]},{"osd":6,"uuid":"64d858d6-a8b3-4f65-a187-af9ab1ccee0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6819","nonce":1396972394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6821","nonce":1396972394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6825","nonce":1396972394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6823","nonce":1396972394}]},"public_addr":"172.21.15.103:6819/1396972394","cluster_addr":"172.21.15.103:6821/1396972394","heartbeat_back_addr":"172.21.15.103:6825/1396972394","heartbeat_front_addr":"172.21.15.103:6823/1396972394","state":["exists","up"]},{"osd":7,"uuid":"d405d1c6-839c-4c92-8261-4adb3163c90f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6817","nonce":2196366765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6819","nonce":2196366765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6823","nonce":2196366765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6821","nonce":2196366765}]},"public_addr":"172.21.15.134:6817/2196366765","cluster_addr":"172.21.15.134:6819/2196366765","heartbeat_back_addr":"172.21.15.134:6823/2196366765","heartbeat_front_addr":"172.21.15.134:6821/2196366765","state":["exists","up"]},{"osd":8,"uuid":"4c452cd9-816c-4f77-9368-2cd4bbeee4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6817","nonce":189977465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6819","nonce":189977465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6823","nonce":189977465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6821","nonce":189977465}]},"public_addr":"172.21.15.174:6817/189977465","cluster_addr":"172.21.15.174:6819/189977465","heartbeat_back_addr":"172.21.15.174:6823/189977465","heartbeat_front_addr":"172.21.15.174:6821/189977465","state":["exists","up"]},{"osd":9,"uuid":"fb3662e0-2d8e-45c4-9119-6d08b35443da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6827","nonce":3161966413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6829","nonce":3161966413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6833","nonce":3161966413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6831","nonce":3161966413}]},"public_addr":"172.21.15.103:6827/3161966413","cluster_addr":"172.21.15.103:6829/3161966413","heartbeat_back_addr":"172.21.15.103:6833/3161966413","heartbeat_front_addr":"172.21.15.103:6831/3161966413","state":["exists","up"]},{"osd":10,"uuid":"e52c3cbe-a6af-441e-b423-b94f3a2da544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6825","nonce":2532758632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6827","nonce":2532758632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6831","nonce":2532758632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6829","nonce":2532758632}]},"public_addr":"172.21.15.134:6825/2532758632","cluster_addr":"172.21.15.134:6827/2532758632","heartbeat_back_addr":"172.21.15.134:6831/2532758632","heartbeat_front_addr":"172.21.15.134:6829/2532758632","state":["exists","up"]},{"osd":11,"uuid":"9040f3f8-3873-4dea-8716-dcda5811c8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6825","nonce":161916442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6827","nonce":161916442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6831","nonce":161916442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6829","nonce":161916442}]},"public_addr":"172.21.15.174:6825/161916442","cluster_addr":"172.21.15.174:6827/161916442","heartbeat_back_addr":"172.21.15.174:6831/161916442","heartbeat_front_addr":"172.21.15.174:6829/161916442","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:41:59.656854+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:24.797551+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:49.685497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:33.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:39.324983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:03.987310+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:28.747971+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:53.827780+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:18.554450+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:43.705691+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:08.401275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:33.502542+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2867049580":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/3727259725":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/1496153385":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:6800/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1642809745":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/2309624637":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1596229922":"2024-09-20T06:40:16.780890+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:48:59.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.567+0000 7f0baa974640 1 -- 172.21.15.103:0/3995883401 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f0b8c0a32d0 msgr2=0x7f0b8c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.567+0000 7f0baa974640 1 --2- 172.21.15.103:0/3995883401 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f0b8c0a32d0 0x7f0b8c0a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009930 tx=0x7f0b94008040 comp rx=0 tx=0).stop 2024-09-19T06:48:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.567+0000 7f0baa974640 1 -- 172.21.15.103:0/3995883401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ba4106f10 msgr2=0x7f0ba419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.567+0000 7f0baa974640 1 --2- 172.21.15.103:0/3995883401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ba4106f10 0x7f0ba419ce80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00b970 tx=0x7f0b9c00be30 comp rx=0 tx=0).stop 2024-09-19T06:48:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.568+0000 7f0baa974640 1 -- 172.21.15.103:0/3995883401 shutdown_connections 2024-09-19T06:48:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.568+0000 7f0baa974640 1 --2- 172.21.15.103:0/3995883401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ba4106f10 0x7f0ba419ce80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.568+0000 7f0baa974640 1 --2- 172.21.15.103:0/3995883401 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f0b8c0a32d0 0x7f0b8c0a5790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.568+0000 7f0baa974640 1 --2- 172.21.15.103:0/3995883401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ba4102790 0x7f0ba4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.568+0000 7f0baa974640 1 --2- 172.21.15.103:0/3995883401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ba4101de0 0x7f0ba41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.568+0000 7f0baa974640 1 -- 172.21.15.103:0/3995883401 >> 172.21.15.103:0/3995883401 conn(0x7f0ba40fd8c0 msgr2=0x7f0ba40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.568+0000 7f0baa974640 1 -- 172.21.15.103:0/3995883401 shutdown_connections 2024-09-19T06:48:59.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:48:59.568+0000 7f0baa974640 1 -- 172.21.15.103:0/3995883401 wait complete. 2024-09-19T06:49:00.142 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T06:47:25.446176+0000', 'modified': '2024-09-19T06:47:36.481871+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24482, 'mds_1': 24455, 'mds_2': 24454, 'mds_3': 24436, 'mds_4': 24431}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14586': {'gid': 14586, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6839/3119326077', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 3119326077}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 3119326077}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14598': {'gid': 14598, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6841/3771437304', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6840', 'nonce': 3771437304}, {'type': 'v1', 'addr': '172.21.15.103:6841', 'nonce': 3771437304}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24397': {'gid': 24397, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/2319563070', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 2319563070}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 2319563070}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/1556743259', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 1556743259}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 1556743259}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/963732203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 963732203}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 963732203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'e', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.134:6835/3092414367', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 3092414367}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 3092414367}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.174:6837/1053388911', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 1053388911}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 1053388911}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.174:6839/4202229983', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 4202229983}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 4202229983}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.134:6837/1590453330', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 1590453330}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 1590453330}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.134:6839/2333076106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 2333076106}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 2333076106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24482, 24455, 24454, 24436, 24431]} 2024-09-19T06:49:00.143 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T06:49:00.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.e status 2024-09-19T06:49:00.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:48:59 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3995883401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:49:00.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:48:59 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3995883401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:49:00.390 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:48:59 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3995883401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:49:00.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.831+0000 7fe50bead640 1 -- 172.21.15.103:0/2748224530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe504106f20 msgr2=0x7fe50410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:00.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.831+0000 7fe50bead640 1 --2- 172.21.15.103:0/2748224530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe504106f20 0x7fe50410eaa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009900 tx=0x7fe4fc02a410 comp rx=0 tx=0).stop 2024-09-19T06:49:00.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.831+0000 7fe50bead640 1 -- 172.21.15.103:0/2748224530 shutdown_connections 2024-09-19T06:49:00.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.831+0000 7fe50bead640 1 --2- 172.21.15.103:0/2748224530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe504106f20 0x7fe50410eaa0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.831+0000 7fe50bead640 1 --2- 172.21.15.103:0/2748224530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5041027a0 0x7fe5041067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.831+0000 7fe50bead640 1 --2- 172.21.15.103:0/2748224530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe504101df0 0x7fe5041021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.831+0000 7fe50bead640 1 -- 172.21.15.103:0/2748224530 >> 172.21.15.103:0/2748224530 conn(0x7fe5040fd8f0 msgr2=0x7fe5040ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:00.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.831+0000 7fe50bead640 1 -- 172.21.15.103:0/2748224530 shutdown_connections 2024-09-19T06:49:00.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.831+0000 7fe50bead640 1 -- 172.21.15.103:0/2748224530 wait complete. 2024-09-19T06:49:00.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.831+0000 7fe50bead640 1 Processor -- start 2024-09-19T06:49:00.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe50bead640 1 -- start start 2024-09-19T06:49:00.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe50bead640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe504101df0 0x7fe50419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe50bead640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5041027a0 0x7fe50419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe50bead640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe504106f20 0x7fe50419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe50bead640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe504111fa0 con 0x7fe504106f20 2024-09-19T06:49:00.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe50bead640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe504111e20 con 0x7fe5041027a0 2024-09-19T06:49:00.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe50bead640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe504112120 con 0x7fe504101df0 2024-09-19T06:49:00.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe50a44a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe504106f20 0x7fe50419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe50a44a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe504106f20 0x7fe50419ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36452/0 (socket says 172.21.15.103:36452) 2024-09-19T06:49:00.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe50a44a640 1 -- 172.21.15.103:0/4029687040 learned_addr learned my addr 172.21.15.103:0/4029687040 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:00.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe509c49640 1 --2- 172.21.15.103:0/4029687040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe504101df0 0x7fe50419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe509448640 1 --2- 172.21.15.103:0/4029687040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5041027a0 0x7fe50419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe50a44a640 1 -- 172.21.15.103:0/4029687040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe504101df0 msgr2=0x7fe50419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:00.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe50a44a640 1 --2- 172.21.15.103:0/4029687040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe504101df0 0x7fe50419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe50a44a640 1 -- 172.21.15.103:0/4029687040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5041027a0 msgr2=0x7fe50419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:00.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe50a44a640 1 --2- 172.21.15.103:0/4029687040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5041027a0 0x7fe50419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe50a44a640 1 -- 172.21.15.103:0/4029687040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50419f630 con 0x7fe504106f20 2024-09-19T06:49:00.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe509c49640 1 --2- 172.21.15.103:0/4029687040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe504101df0 0x7fe50419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:00.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe509448640 1 --2- 172.21.15.103:0/4029687040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5041027a0 0x7fe50419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:00.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.832+0000 7fe50a44a640 1 --2- 172.21.15.103:0/4029687040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe504106f20 0x7fe50419ef50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009230 tx=0x7fe4fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:00.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.833+0000 7fe502ffd640 1 -- 172.21.15.103:0/4029687040 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc03f030 con 0x7fe504106f20 2024-09-19T06:49:00.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.833+0000 7fe502ffd640 1 -- 172.21.15.103:0/4029687040 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4fc003ce0 con 0x7fe504106f20 2024-09-19T06:49:00.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.833+0000 7fe502ffd640 1 -- 172.21.15.103:0/4029687040 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc035070 con 0x7fe504106f20 2024-09-19T06:49:00.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.833+0000 7fe50bead640 1 -- 172.21.15.103:0/4029687040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50419f8c0 con 0x7fe504106f20 2024-09-19T06:49:00.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.833+0000 7fe50bead640 1 -- 172.21.15.103:0/4029687040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5041a7200 con 0x7fe504106f20 2024-09-19T06:49:00.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.835+0000 7fe50e286740 1 Processor -- start 2024-09-19T06:49:00.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.836+0000 7fe502ffd640 1 -- 172.21.15.103:0/4029687040 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe4fc042030 con 0x7fe504106f20 2024-09-19T06:49:00.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.836+0000 7fe50e286740 1 -- start start 2024-09-19T06:49:00.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.836+0000 7fe50e286740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7b9d0 0x5597aca7bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.836+0000 7fe50e286740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca7c2f0 0x5597aca86670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.836+0000 7fe50e286740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca86bb0 0x5597aca88fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.836+0000 7fe50e286740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5597ac9e2090 con 0x5597aca7b9d0 2024-09-19T06:49:00.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.836+0000 7fe50e286740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5597ac989a70 con 0x5597aca86bb0 2024-09-19T06:49:00.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.836+0000 7fe50e286740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5597ac9de050 con 0x5597aca7c2f0 2024-09-19T06:49:00.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.836+0000 7fe509448640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7b9d0 0x5597aca7bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.836+0000 7fe509448640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7b9d0 0x5597aca7bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36466/0 (socket says 172.21.15.103:36466) 2024-09-19T06:49:00.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.836+0000 7fe509448640 1 -- 172.21.15.103:0/2871596300 learned_addr learned my addr 172.21.15.103:0/2871596300 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:00.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.836+0000 7fe509c49640 1 --2- 172.21.15.103:0/2871596300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca7c2f0 0x5597aca86670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.837+0000 7fe509448640 1 --2- 172.21.15.103:0/2871596300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca86bb0 0x5597aca88fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.837+0000 7fe502ffd640 1 --2- 172.21.15.103:0/4029687040 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe4ec0a32d0 0x7fe4ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.837+0000 7fe509448640 1 -- 172.21.15.103:0/2871596300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca7c2f0 msgr2=0x5597aca86670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:00.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.837+0000 7fe509448640 1 --2- 172.21.15.103:0/2871596300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca7c2f0 0x5597aca86670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.837+0000 7fe509448640 1 -- 172.21.15.103:0/2871596300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca86bb0 msgr2=0x5597aca88fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:00.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.837+0000 7fe509448640 1 --2- 172.21.15.103:0/2871596300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca86bb0 0x5597aca88fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.837+0000 7fe509448640 1 -- 172.21.15.103:0/2871596300 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ac996130 con 0x5597aca7b9d0 2024-09-19T06:49:00.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.837+0000 7fe502ffd640 1 -- 172.21.15.103:0/4029687040 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe4fc0f0470 con 0x7fe504106f20 2024-09-19T06:49:00.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.837+0000 7fe509448640 1 --2- 172.21.15.103:0/2871596300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7b9d0 0x5597aca7bdb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8006fd0 tx=0x7fe4f8007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0b4fdeef270290 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:00.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.837+0000 7fe50a44a640 1 --2- 172.21.15.103:0/4029687040 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe4ec0a32d0 0x7fe4ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.837+0000 7fe4ebfff640 1 -- 172.21.15.103:0/2871596300 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f801c070 con 0x5597aca7b9d0 2024-09-19T06:49:00.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.837+0000 7fe4ebfff640 1 -- 172.21.15.103:0/2871596300 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f8010040 con 0x5597aca7b9d0 2024-09-19T06:49:00.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.837+0000 7fe4ebfff640 1 -- 172.21.15.103:0/2871596300 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8014730 con 0x5597aca7b9d0 2024-09-19T06:49:00.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.838+0000 7fe50e286740 1 -- 172.21.15.103:0/2871596300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7b9d0 msgr2=0x5597aca7bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:00.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.838+0000 7fe50a44a640 1 --2- 172.21.15.103:0/4029687040 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe4ec0a32d0 0x7fe4ec0a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0c5080 tx=0x7fe4fc03b790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:00.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.838+0000 7fe50e286740 1 --2- 172.21.15.103:0/2871596300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7b9d0 0x5597aca7bdb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8006fd0 tx=0x7fe4f8007420 comp rx=0 tx=0).stop 2024-09-19T06:49:00.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.838+0000 7fe50e286740 1 -- 172.21.15.103:0/2871596300 shutdown_connections 2024-09-19T06:49:00.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.838+0000 7fe50e286740 1 --2- 172.21.15.103:0/2871596300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca86bb0 0x5597aca88fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.838+0000 7fe50e286740 1 --2- 172.21.15.103:0/2871596300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca7c2f0 0x5597aca86670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.838+0000 7fe50e286740 1 --2- 172.21.15.103:0/2871596300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7b9d0 0x5597aca7bdb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.838+0000 7fe50e286740 1 -- 172.21.15.103:0/2871596300 >> 172.21.15.103:0/2871596300 conn(0x5597aca77f60 msgr2=0x5597aca78340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:00.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.839+0000 7fe50e286740 1 -- 172.21.15.103:0/2871596300 shutdown_connections 2024-09-19T06:49:00.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.839+0000 7fe50e286740 1 -- 172.21.15.103:0/2871596300 wait complete. 2024-09-19T06:49:00.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.839+0000 7fe50e286740 1 -- start start 2024-09-19T06:49:00.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.839+0000 7fe50e286740 1 Processor -- start 2024-09-19T06:49:00.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.839+0000 7fe50e286740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7b9d0 0x5597aca86490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.839+0000 7fe50e286740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca7c2f0 0x5597aca837e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.839+0000 7fe50e286740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca86bb0 0x5597aca83d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.839+0000 7fe50e286740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5597aca89780 con 0x5597aca7b9d0 2024-09-19T06:49:00.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.839+0000 7fe50e286740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5597ac99cf20 con 0x5597aca86bb0 2024-09-19T06:49:00.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.839+0000 7fe50e286740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5597aca89900 con 0x5597aca7c2f0 2024-09-19T06:49:00.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.840+0000 7fe50e286740 10 client.0 _ll_register_callbacks cb 0x5597ac966030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:00.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.840+0000 7fe509448640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7b9d0 0x5597aca86490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.840+0000 7fe509448640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7b9d0 0x5597aca86490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36468/0 (socket says 172.21.15.103:36468) 2024-09-19T06:49:00.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.840+0000 7fe509448640 1 -- 172.21.15.103:0/448285781 learned_addr learned my addr 172.21.15.103:0/448285781 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:00.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.840+0000 7fe509c49640 1 --2- 172.21.15.103:0/448285781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca7c2f0 0x5597aca837e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.840+0000 7fe509448640 1 --2- 172.21.15.103:0/448285781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca86bb0 0x5597aca83d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.840+0000 7fe509448640 1 -- 172.21.15.103:0/448285781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca7c2f0 msgr2=0x5597aca837e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:00.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.840+0000 7fe509448640 1 --2- 172.21.15.103:0/448285781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca7c2f0 0x5597aca837e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.840+0000 7fe509448640 1 -- 172.21.15.103:0/448285781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca86bb0 msgr2=0x5597aca83d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:00.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.840+0000 7fe509448640 1 --2- 172.21.15.103:0/448285781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca86bb0 0x5597aca83d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.840+0000 7fe509448640 1 -- 172.21.15.103:0/448285781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597ac7532b0 con 0x5597aca7b9d0 2024-09-19T06:49:00.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.840+0000 7fe509448640 1 --2- 172.21.15.103:0/448285781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7b9d0 0x5597aca86490 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8017f90 tx=0x7fe4f80143b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e484585647794765 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:00.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.840+0000 7fe4e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:00.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.840+0000 7fe4e97fa640 1 -- 172.21.15.103:0/448285781 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f801c040 con 0x5597aca7b9d0 2024-09-19T06:49:00.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.841+0000 7fe4e97fa640 1 -- 172.21.15.103:0/448285781 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe4f8014570 con 0x5597aca7b9d0 2024-09-19T06:49:00.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.841+0000 7fe50e286740 1 -- 172.21.15.103:0/448285781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f8010040 con 0x5597aca7b9d0 2024-09-19T06:49:00.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.841+0000 7fe4e97fa640 1 -- 172.21.15.103:0/448285781 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4f80059c0 con 0x5597aca7b9d0 2024-09-19T06:49:00.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.841+0000 7fe50e286740 10 client.15108 fetch_fsmap learned FSMap version 26 2024-09-19T06:49:00.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.841+0000 7fe50e286740 1 -- 172.21.15.103:0/448285781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5597aca950f0 con 0x5597aca7b9d0 2024-09-19T06:49:00.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.841+0000 7fe4e97fa640 1 -- 172.21.15.103:0/448285781 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f8010040 con 0x5597aca7b9d0 2024-09-19T06:49:00.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.841+0000 7fe4e97fa640 1 -- 172.21.15.103:0/448285781 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fe4f8027780 con 0x5597aca7b9d0 2024-09-19T06:49:00.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.841+0000 7fe50e286740 10 client.15108 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:49:00.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.841+0000 7fe50e286740 10 client.15108 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-19T06:49:00.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.841+0000 7fe50e286740 20 client.15108 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:00.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.841+0000 7fe50e286740 1 --2- 172.21.15.103:0/448285781 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5597aca97570 0x5597aca989e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.841+0000 7fe50e286740 4 client.15108 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:00.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.841+0000 7fe50e286740 1 -- 172.21.15.103:0/448285781 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597ac9ef600 con 0x5597aca97570 2024-09-19T06:49:00.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.842+0000 7fe509c49640 1 --2- 172.21.15.103:0/448285781 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5597aca97570 0x5597aca989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.842+0000 7fe509c49640 1 --2- 172.21.15.103:0/448285781 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5597aca97570 0x5597aca989e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7f8a000c7bec342e server_cookie=ddcf5d493ab107c2 in_seq=0 out_seq=0 2024-09-19T06:49:00.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.842+0000 7fe4e97fa640 10 client.15108 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:49:00.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.843+0000 7fe4e97fa640 1 -- 172.21.15.103:0/448285781 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597ac9ef600 con 0x5597aca97570 2024-09-19T06:49:00.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.843+0000 7fe4e97fa640 10 client.15108 handle_command_reply: tid=0 2024-09-19T06:49:00.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.843+0000 7fe50e286740 1 client.15108 shutdown 2024-09-19T06:49:00.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.843+0000 7fe50e286740 1 -- 172.21.15.103:0/448285781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7b9d0 msgr2=0x5597aca86490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:00.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.843+0000 7fe50e286740 1 --2- 172.21.15.103:0/448285781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7b9d0 0x5597aca86490 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8017f90 tx=0x7fe4f80143b0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.843+0000 7fe50e286740 1 -- 172.21.15.103:0/448285781 shutdown_connections 2024-09-19T06:49:00.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.843+0000 7fe50e286740 1 --2- 172.21.15.103:0/448285781 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5597aca97570 0x5597aca989e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.843+0000 7fe50e286740 1 --2- 172.21.15.103:0/448285781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca86bb0 0x5597aca83d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.843+0000 7fe50e286740 1 --2- 172.21.15.103:0/448285781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca7c2f0 0x5597aca837e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.843+0000 7fe50e286740 1 --2- 172.21.15.103:0/448285781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7b9d0 0x5597aca86490 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.843+0000 7fe50e286740 1 -- 172.21.15.103:0/448285781 >> 172.21.15.103:0/448285781 conn(0x5597aca77f60 msgr2=0x5597aca7dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:00.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.844+0000 7fe4e97fa640 0 client.15108 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:49:00.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.844+0000 7fe50e286740 1 -- 172.21.15.103:0/448285781 shutdown_connections 2024-09-19T06:49:00.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.844+0000 7fe50e286740 1 -- 172.21.15.103:0/448285781 wait complete. 2024-09-19T06:49:00.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.844+0000 7fe50e286740 20 client.15108 trim_cache size 0 max 16384 2024-09-19T06:49:00.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.862+0000 7fe50e286740 1 Processor -- start 2024-09-19T06:49:00.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.862+0000 7fe50e286740 1 -- start start 2024-09-19T06:49:00.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.862+0000 7fe50e286740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca7b9d0 0x5597aca97570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.862+0000 7fe50e286740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca7c2f0 0x5597aca72ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.862+0000 7fe50e286740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca86bb0 0x5597aca820b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.862+0000 7fe50e286740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5597aca96d20 con 0x5597aca86bb0 2024-09-19T06:49:00.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.862+0000 7fe50e286740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5597aca973f0 con 0x5597aca7b9d0 2024-09-19T06:49:00.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.862+0000 7fe50e286740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5597aca97ac0 con 0x5597aca7c2f0 2024-09-19T06:49:00.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.862+0000 7fe509448640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca86bb0 0x5597aca820b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.862+0000 7fe509448640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca86bb0 0x5597aca820b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36484/0 (socket says 172.21.15.103:36484) 2024-09-19T06:49:00.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.862+0000 7fe509448640 1 -- 172.21.15.103:0/1404190757 learned_addr learned my addr 172.21.15.103:0/1404190757 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:00.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.863+0000 7fe509c49640 1 --2- 172.21.15.103:0/1404190757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca7c2f0 0x5597aca72ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.863+0000 7fe509448640 1 --2- 172.21.15.103:0/1404190757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca7b9d0 0x5597aca97570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.863+0000 7fe509448640 1 -- 172.21.15.103:0/1404190757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca7c2f0 msgr2=0x5597aca72ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:00.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.863+0000 7fe509448640 1 --2- 172.21.15.103:0/1404190757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca7c2f0 0x5597aca72ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.863+0000 7fe509448640 1 -- 172.21.15.103:0/1404190757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca7b9d0 msgr2=0x5597aca97570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:00.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.863+0000 7fe509448640 1 --2- 172.21.15.103:0/1404190757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca7b9d0 0x5597aca97570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.863+0000 7fe509448640 1 -- 172.21.15.103:0/1404190757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597aca860b0 con 0x5597aca86bb0 2024-09-19T06:49:00.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.863+0000 7fe509448640 1 --2- 172.21.15.103:0/1404190757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca86bb0 0x5597aca820b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8000c00 tx=0x7fe4f8006ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d12298c12952aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:00.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.863+0000 7fe4eb7fe640 1 -- 172.21.15.103:0/1404190757 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8036070 con 0x5597aca86bb0 2024-09-19T06:49:00.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.863+0000 7fe4eb7fe640 1 -- 172.21.15.103:0/1404190757 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f80278f0 con 0x5597aca86bb0 2024-09-19T06:49:00.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.863+0000 7fe4eb7fe640 1 -- 172.21.15.103:0/1404190757 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f801c040 con 0x5597aca86bb0 2024-09-19T06:49:00.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 -- 172.21.15.103:0/1404190757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca86bb0 msgr2=0x5597aca820b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:00.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 --2- 172.21.15.103:0/1404190757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca86bb0 0x5597aca820b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8000c00 tx=0x7fe4f8006ce0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 -- 172.21.15.103:0/1404190757 shutdown_connections 2024-09-19T06:49:00.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 --2- 172.21.15.103:0/1404190757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca86bb0 0x5597aca820b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 --2- 172.21.15.103:0/1404190757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca7c2f0 0x5597aca72ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 --2- 172.21.15.103:0/1404190757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca7b9d0 0x5597aca97570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 -- 172.21.15.103:0/1404190757 >> 172.21.15.103:0/1404190757 conn(0x5597aca77f60 msgr2=0x5597aca96550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:00.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 -- 172.21.15.103:0/1404190757 shutdown_connections 2024-09-19T06:49:00.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 -- 172.21.15.103:0/1404190757 wait complete. 2024-09-19T06:49:00.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 -- start start 2024-09-19T06:49:00.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 Processor -- start 2024-09-19T06:49:00.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca7b9d0 0x5597acaa1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7c2f0 0x5597aca99be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca86bb0 0x5597aca9a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5597aca98860 con 0x5597aca7c2f0 2024-09-19T06:49:00.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5597aca98190 con 0x5597aca7b9d0 2024-09-19T06:49:00.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.864+0000 7fe50e286740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5597aca98f30 con 0x5597aca86bb0 2024-09-19T06:49:00.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe50e286740 10 client.0 _ll_register_callbacks cb 0x7fe4f801a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:00.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe509c49640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7c2f0 0x5597aca99be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe509c49640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7c2f0 0x5597aca99be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36486/0 (socket says 172.21.15.103:36486) 2024-09-19T06:49:00.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe509c49640 1 -- 172.21.15.103:0/1215099317 learned_addr learned my addr 172.21.15.103:0/1215099317 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:00.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe509448640 1 --2- 172.21.15.103:0/1215099317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca86bb0 0x5597aca9a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe509448640 1 --2- 172.21.15.103:0/1215099317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca7b9d0 0x5597acaa1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe509c49640 1 -- 172.21.15.103:0/1215099317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca86bb0 msgr2=0x5597aca9a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:00.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe509c49640 1 --2- 172.21.15.103:0/1215099317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca86bb0 0x5597aca9a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe509c49640 1 -- 172.21.15.103:0/1215099317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca7b9d0 msgr2=0x5597acaa1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:00.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe509c49640 1 --2- 172.21.15.103:0/1215099317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca7b9d0 0x5597acaa1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe509c49640 1 -- 172.21.15.103:0/1215099317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597acaffc70 con 0x5597aca7c2f0 2024-09-19T06:49:00.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe509c49640 1 --2- 172.21.15.103:0/1215099317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7c2f0 0x5597aca99be0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4000f80 tx=0x7fe4f4011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e823d62a70f7680a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:00.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe4e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:00.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe4e97fa640 1 -- 172.21.15.103:0/1215099317 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4004d30 con 0x5597aca7c2f0 2024-09-19T06:49:00.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe4e97fa640 1 -- 172.21.15.103:0/1215099317 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe4f4003970 con 0x5597aca7c2f0 2024-09-19T06:49:00.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe50e286740 1 -- 172.21.15.103:0/1215099317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f80278f0 con 0x5597aca7c2f0 2024-09-19T06:49:00.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe4e97fa640 1 -- 172.21.15.103:0/1215099317 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4f4003b70 con 0x5597aca7c2f0 2024-09-19T06:49:00.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe50e286740 10 client.15114 fetch_fsmap learned FSMap version 26 2024-09-19T06:49:00.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.865+0000 7fe50e286740 1 -- 172.21.15.103:0/1215099317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5597acb00e20 con 0x5597aca7c2f0 2024-09-19T06:49:00.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.866+0000 7fe4e97fa640 1 -- 172.21.15.103:0/1215099317 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f80278f0 con 0x5597aca7c2f0 2024-09-19T06:49:00.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.866+0000 7fe4e97fa640 1 -- 172.21.15.103:0/1215099317 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fe4f4004290 con 0x5597aca7c2f0 2024-09-19T06:49:00.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.866+0000 7fe50e286740 10 client.15114 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:49:00.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.866+0000 7fe50e286740 10 client.15114 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-19T06:49:00.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.866+0000 7fe50e286740 20 client.15114 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:00.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.866+0000 7fe50e286740 1 --2- 172.21.15.103:0/1215099317 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5597acb05a80 0x5597acb04790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:00.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.866+0000 7fe50e286740 4 client.15114 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:49:00.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.866+0000 7fe50e286740 1 -- 172.21.15.103:0/1215099317 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "status"}) -- 0x5597ac9e9b10 con 0x5597acb05a80 2024-09-19T06:49:00.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.866+0000 7fe509448640 1 --2- 172.21.15.103:0/1215099317 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5597acb05a80 0x5597acb04790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:00.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.866+0000 7fe509448640 1 --2- 172.21.15.103:0/1215099317 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5597acb05a80 0x5597acb04790 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=30ecc240cbd81f46 server_cookie=622e25817e89c8e3 in_seq=0 out_seq=0 2024-09-19T06:49:00.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.866+0000 7fe4e97fa640 10 client.15114 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:49:00.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.867+0000 7fe4e97fa640 1 -- 172.21.15.103:0/1215099317 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5597ac9e9b10 con 0x5597acb05a80 2024-09-19T06:49:00.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.867+0000 7fe4e97fa640 10 client.15114 handle_command_reply: tid=0 2024-09-19T06:49:00.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.867+0000 7fe50e286740 1 client.15114 shutdown 2024-09-19T06:49:00.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.867+0000 7fe50e286740 1 -- 172.21.15.103:0/1215099317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7c2f0 msgr2=0x5597aca99be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:00.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.867+0000 7fe50e286740 1 --2- 172.21.15.103:0/1215099317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7c2f0 0x5597aca99be0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4000f80 tx=0x7fe4f4011cb0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.867+0000 7fe50e286740 1 -- 172.21.15.103:0/1215099317 shutdown_connections 2024-09-19T06:49:00.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.867+0000 7fe50e286740 1 --2- 172.21.15.103:0/1215099317 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5597acb05a80 0x5597acb04790 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.867+0000 7fe50e286740 1 --2- 172.21.15.103:0/1215099317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aca86bb0 0x5597aca9a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.867+0000 7fe50e286740 1 --2- 172.21.15.103:0/1215099317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597aca7c2f0 0x5597aca99be0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.867+0000 7fe50e286740 1 --2- 172.21.15.103:0/1215099317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597aca7b9d0 0x5597acaa1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.867+0000 7fe4e97fa640 0 client.15114 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:49:00.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.867+0000 7fe50e286740 1 -- 172.21.15.103:0/1215099317 >> 172.21.15.103:0/1215099317 conn(0x5597aca77f60 msgr2=0x5597aca72ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:00.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.867+0000 7fe50e286740 1 -- 172.21.15.103:0/1215099317 shutdown_connections 2024-09-19T06:49:00.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.867+0000 7fe50e286740 1 -- 172.21.15.103:0/1215099317 wait complete. 2024-09-19T06:49:00.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.867+0000 7fe50e286740 20 client.15114 trim_cache size 0 max 16384 2024-09-19T06:49:00.874 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:49:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:49:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 4, 2024-09-19T06:49:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24431, 2024-09-19T06:49:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T06:49:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T06:49:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T06:49:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 84.967870700000006, 2024-09-19T06:49:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T06:49:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 82, 2024-09-19T06:49:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T06:49:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 124.65516597 2024-09-19T06:49:00.875 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:49:00.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.868+0000 7fe50bead640 1 -- 172.21.15.103:0/4029687040 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe4ec0a32d0 msgr2=0x7fe4ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:00.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.868+0000 7fe50bead640 1 --2- 172.21.15.103:0/4029687040 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe4ec0a32d0 0x7fe4ec0a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0c5080 tx=0x7fe4fc03b790 comp rx=0 tx=0).stop 2024-09-19T06:49:00.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.868+0000 7fe50bead640 1 -- 172.21.15.103:0/4029687040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe504106f20 msgr2=0x7fe50419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:00.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.868+0000 7fe50bead640 1 --2- 172.21.15.103:0/4029687040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe504106f20 0x7fe50419ef50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009230 tx=0x7fe4fc0042f0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.868+0000 7fe50bead640 1 -- 172.21.15.103:0/4029687040 shutdown_connections 2024-09-19T06:49:00.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.868+0000 7fe50bead640 1 --2- 172.21.15.103:0/4029687040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe504106f20 0x7fe50419ef50 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.868+0000 7fe50bead640 1 --2- 172.21.15.103:0/4029687040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5041027a0 0x7fe50419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.868+0000 7fe50bead640 1 --2- 172.21.15.103:0/4029687040 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fe4ec0a32d0 0x7fe4ec0a5790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.868+0000 7fe50bead640 1 --2- 172.21.15.103:0/4029687040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe504101df0 0x7fe50419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:00.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.868+0000 7fe50bead640 1 -- 172.21.15.103:0/4029687040 >> 172.21.15.103:0/4029687040 conn(0x7fe5040fd8f0 msgr2=0x7fe5040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:00.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.868+0000 7fe50bead640 1 -- 172.21.15.103:0/4029687040 shutdown_connections 2024-09-19T06:49:00.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:00.868+0000 7fe50bead640 1 -- 172.21.15.103:0/4029687040 wait complete. 2024-09-19T06:49:01.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:00 smithi103 ceph-mon[31158]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:49:01.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:00 smithi174 ceph-mon[35637]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:49:01.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:00 smithi134 ceph-mon[35356]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:49:01.535 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T06:49:01.780 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:02.152 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:01 smithi103 ceph-mon[31158]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:49:02.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.229+0000 7fd04cca6640 1 -- 172.21.15.103:0/1392688050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd048100a80 msgr2=0x7fd048100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:02.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.229+0000 7fd04cca6640 1 --2- 172.21.15.103:0/1392688050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd048100a80 0x7fd048100ee0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd030009930 tx=0x7fd03002a420 comp rx=0 tx=0).stop 2024-09-19T06:49:02.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.229+0000 7fd04cca6640 1 -- 172.21.15.103:0/1392688050 shutdown_connections 2024-09-19T06:49:02.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.229+0000 7fd04cca6640 1 --2- 172.21.15.103:0/1392688050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048101420 0x7fd04810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.229+0000 7fd04cca6640 1 --2- 172.21.15.103:0/1392688050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd048100a80 0x7fd048100ee0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.229+0000 7fd04cca6640 1 --2- 172.21.15.103:0/1392688050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd048106a80 0x7fd048106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.229+0000 7fd04cca6640 1 -- 172.21.15.103:0/1392688050 >> 172.21.15.103:0/1392688050 conn(0x7fd0480fc7a0 msgr2=0x7fd0480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:02.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.229+0000 7fd04cca6640 1 -- 172.21.15.103:0/1392688050 shutdown_connections 2024-09-19T06:49:02.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.229+0000 7fd04cca6640 1 -- 172.21.15.103:0/1392688050 wait complete. 2024-09-19T06:49:02.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.230+0000 7fd04cca6640 1 Processor -- start 2024-09-19T06:49:02.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.230+0000 7fd04cca6640 1 -- start start 2024-09-19T06:49:02.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.230+0000 7fd04cca6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd048100a80 0x7fd0481974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.230+0000 7fd04cca6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd048101420 0x7fd048197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.230+0000 7fd04cca6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048106a80 0x7fd04819bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.230+0000 7fd04cca6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd048110f20 con 0x7fd048100a80 2024-09-19T06:49:02.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.230+0000 7fd04cca6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd048110da0 con 0x7fd048106a80 2024-09-19T06:49:02.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.230+0000 7fd04cca6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0481110a0 con 0x7fd048101420 2024-09-19T06:49:02.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.230+0000 7fd04659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd048100a80 0x7fd0481974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:02.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.230+0000 7fd04659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd048100a80 0x7fd0481974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36500/0 (socket says 172.21.15.103:36500) 2024-09-19T06:49:02.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.230+0000 7fd04659c640 1 -- 172.21.15.103:0/4101493030 learned_addr learned my addr 172.21.15.103:0/4101493030 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:02.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.230+0000 7fd04659c640 1 -- 172.21.15.103:0/4101493030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd048101420 msgr2=0x7fd048197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:02.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.230+0000 7fd04659c640 1 --2- 172.21.15.103:0/4101493030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd048101420 0x7fd048197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.230+0000 7fd04659c640 1 -- 172.21.15.103:0/4101493030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048106a80 msgr2=0x7fd04819bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:02.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.230+0000 7fd04659c640 1 --2- 172.21.15.103:0/4101493030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048106a80 0x7fd04819bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.230+0000 7fd04659c640 1 -- 172.21.15.103:0/4101493030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04819c4a0 con 0x7fd048100a80 2024-09-19T06:49:02.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.231+0000 7fd04659c640 1 --2- 172.21.15.103:0/4101493030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd048100a80 0x7fd0481974f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd04000ba50 tx=0x7fd04000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:02.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.231+0000 7fd03f7fe640 1 -- 172.21.15.103:0/4101493030 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd040019070 con 0x7fd048100a80 2024-09-19T06:49:02.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.231+0000 7fd03f7fe640 1 -- 172.21.15.103:0/4101493030 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd04000cbe0 con 0x7fd048100a80 2024-09-19T06:49:02.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.231+0000 7fd04cca6640 1 -- 172.21.15.103:0/4101493030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04819c790 con 0x7fd048100a80 2024-09-19T06:49:02.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.231+0000 7fd03f7fe640 1 -- 172.21.15.103:0/4101493030 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0400076e0 con 0x7fd048100a80 2024-09-19T06:49:02.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.231+0000 7fd04cca6640 1 -- 172.21.15.103:0/4101493030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0481a4070 con 0x7fd048100a80 2024-09-19T06:49:02.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.233+0000 7fd03f7fe640 1 -- 172.21.15.103:0/4101493030 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd04001c070 con 0x7fd048100a80 2024-09-19T06:49:02.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.233+0000 7fd04f07f740 1 Processor -- start 2024-09-19T06:49:02.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.233+0000 7fd04f07f740 1 -- start start 2024-09-19T06:49:02.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd04f07f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9ca89d0 0x55c2f9ca8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd04f07f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cb3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd04f07f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9cb5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd04f07f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2f9c0f090 con 0x55c2f9ca89d0 2024-09-19T06:49:02.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd04f07f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c2f9bb6a70 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd04f07f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2f9c0b050 con 0x55c2f9ca92f0 2024-09-19T06:49:02.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd046d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9ca89d0 0x55c2f9ca8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:02.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd046d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9ca89d0 0x55c2f9ca8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36502/0 (socket says 172.21.15.103:36502) 2024-09-19T06:49:02.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd046d9d640 1 -- 172.21.15.103:0/764307821 learned_addr learned my addr 172.21.15.103:0/764307821 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:02.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd03f7fe640 1 --2- 172.21.15.103:0/4101493030 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fd0340a32d0 0x7fd0340a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd04659c640 1 --2- 172.21.15.103:0/764307821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cb3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:02.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd046d9d640 1 --2- 172.21.15.103:0/4101493030 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fd0340a32d0 0x7fd0340a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:02.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd045d9b640 1 --2- 172.21.15.103:0/764307821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9cb5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:02.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd046d9d640 1 -- 172.21.15.103:0/764307821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca92f0 msgr2=0x55c2f9cb3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:02.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd046d9d640 1 --2- 172.21.15.103:0/764307821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cb3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd046d9d640 1 -- 172.21.15.103:0/764307821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9cb3bb0 msgr2=0x55c2f9cb5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:02.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd046d9d640 1 --2- 172.21.15.103:0/764307821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9cb5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd046d9d640 1 -- 172.21.15.103:0/764307821 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f9bc3130 con 0x55c2f9ca89d0 2024-09-19T06:49:02.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd03f7fe640 1 -- 172.21.15.103:0/4101493030 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd0400ceff0 con 0x7fd048100a80 2024-09-19T06:49:02.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd046d9d640 1 --2- 172.21.15.103:0/764307821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9ca89d0 0x55c2f9ca8db0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd038009f70 tx=0x7fd0380126a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238cb1b4459b21cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:02.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.234+0000 7fd03cff9640 1 -- 172.21.15.103:0/764307821 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03800f050 con 0x55c2f9ca89d0 2024-09-19T06:49:02.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.235+0000 7fd03cff9640 1 -- 172.21.15.103:0/764307821 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd038010040 con 0x55c2f9ca89d0 2024-09-19T06:49:02.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.235+0000 7fd03cff9640 1 -- 172.21.15.103:0/764307821 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03801a030 con 0x55c2f9ca89d0 2024-09-19T06:49:02.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.235+0000 7fd046d9d640 1 --2- 172.21.15.103:0/4101493030 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fd0340a32d0 0x7fd0340a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd03801d000 tx=0x7fd038019000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:02.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.235+0000 7fd04f07f740 1 -- 172.21.15.103:0/764307821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9ca89d0 msgr2=0x55c2f9ca8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:02.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.235+0000 7fd04f07f740 1 --2- 172.21.15.103:0/764307821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9ca89d0 0x55c2f9ca8db0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd038009f70 tx=0x7fd0380126a0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.235+0000 7fd04f07f740 1 -- 172.21.15.103:0/764307821 shutdown_connections 2024-09-19T06:49:02.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.235+0000 7fd04f07f740 1 --2- 172.21.15.103:0/764307821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9cb5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.235+0000 7fd04f07f740 1 --2- 172.21.15.103:0/764307821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cb3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.235+0000 7fd04f07f740 1 --2- 172.21.15.103:0/764307821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9ca89d0 0x55c2f9ca8db0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.235+0000 7fd04f07f740 1 -- 172.21.15.103:0/764307821 >> 172.21.15.103:0/764307821 conn(0x55c2f9ca4f60 msgr2=0x55c2f9ca5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:02.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.235+0000 7fd04f07f740 1 -- 172.21.15.103:0/764307821 shutdown_connections 2024-09-19T06:49:02.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.235+0000 7fd04f07f740 1 -- 172.21.15.103:0/764307821 wait complete. 2024-09-19T06:49:02.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd04f07f740 1 -- start start 2024-09-19T06:49:02.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd04f07f740 1 Processor -- start 2024-09-19T06:49:02.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd04f07f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca89d0 0x55c2f9cb3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd04f07f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cb07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd04f07f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9cb0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd04f07f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2f9cb6780 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd04f07f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c2f9bc9f20 con 0x55c2f9ca89d0 2024-09-19T06:49:02.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd04f07f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2f9cb6900 con 0x55c2f9ca92f0 2024-09-19T06:49:02.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd04f07f740 10 client.0 _ll_register_callbacks cb 0x55c2f9b93030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:02.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd046d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9cb0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:02.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd046d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9cb0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36514/0 (socket says 172.21.15.103:36514) 2024-09-19T06:49:02.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd046d9d640 1 -- 172.21.15.103:0/1786015209 learned_addr learned my addr 172.21.15.103:0/1786015209 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:02.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd045d9b640 1 --2- 172.21.15.103:0/1786015209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cb07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:02.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd046d9d640 1 -- 172.21.15.103:0/1786015209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca92f0 msgr2=0x55c2f9cb07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:02.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd04659c640 1 --2- 172.21.15.103:0/1786015209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca89d0 0x55c2f9cb3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:02.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd046d9d640 1 --2- 172.21.15.103:0/1786015209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cb07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd046d9d640 1 -- 172.21.15.103:0/1786015209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca89d0 msgr2=0x55c2f9cb3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:02.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd046d9d640 1 --2- 172.21.15.103:0/1786015209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca89d0 0x55c2f9cb3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.236+0000 7fd046d9d640 1 -- 172.21.15.103:0/1786015209 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f99802b0 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd045d9b640 1 --2- 172.21.15.103:0/1786015209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cb07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:02.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd046d9d640 1 --2- 172.21.15.103:0/1786015209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9cb0ce0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd0380090e0 tx=0x7fd038009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e2d2779b44541e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:02.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd02dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:02.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd02dffb640 1 -- 172.21.15.103:0/1786015209 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03801a040 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd02dffb640 1 -- 172.21.15.103:0/1786015209 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd038027640 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd04f07f740 1 -- 172.21.15.103:0/1786015209 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd038010040 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd02dffb640 1 -- 172.21.15.103:0/1786015209 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd038009880 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd04f07f740 10 client.15126 fetch_fsmap learned FSMap version 26 2024-09-19T06:49:02.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd04f07f740 1 -- 172.21.15.103:0/1786015209 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2f9cc20a0 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd02dffb640 1 -- 172.21.15.103:0/1786015209 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd038010040 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd02dffb640 1 -- 172.21.15.103:0/1786015209 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fd038015070 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd04f07f740 10 client.15126 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:49:02.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd04f07f740 10 client.15126 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T06:49:02.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd04f07f740 20 client.15126 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:02.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd04f07f740 1 --2- 172.21.15.103:0/1786015209 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c2f9cc4400 0x55c2f9cc5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd04f07f740 4 client.15126 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:02.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.237+0000 7fd04f07f740 1 -- 172.21.15.103:0/1786015209 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2f9c1c600 con 0x55c2f9cc4400 2024-09-19T06:49:02.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.238+0000 7fd04659c640 1 --2- 172.21.15.103:0/1786015209 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c2f9cc4400 0x55c2f9cc5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:02.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.238+0000 7fd02dffb640 10 client.15126 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:49:02.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.238+0000 7fd04659c640 1 --2- 172.21.15.103:0/1786015209 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c2f9cc4400 0x55c2f9cc5870 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd7863e67fef3761 server_cookie=22d2eb10ff424cef in_seq=0 out_seq=0 2024-09-19T06:49:02.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.239+0000 7fd02dffb640 1 -- 172.21.15.103:0/1786015209 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2f9c1c600 con 0x55c2f9cc4400 2024-09-19T06:49:02.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.239+0000 7fd02dffb640 10 client.15126 handle_command_reply: tid=0 2024-09-19T06:49:02.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.239+0000 7fd04f07f740 1 client.15126 shutdown 2024-09-19T06:49:02.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.239+0000 7fd04f07f740 1 -- 172.21.15.103:0/1786015209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 msgr2=0x55c2f9cb0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:02.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.239+0000 7fd04f07f740 1 --2- 172.21.15.103:0/1786015209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9cb0ce0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd0380090e0 tx=0x7fd038009bc0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.239+0000 7fd04f07f740 1 -- 172.21.15.103:0/1786015209 shutdown_connections 2024-09-19T06:49:02.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.239+0000 7fd04f07f740 1 --2- 172.21.15.103:0/1786015209 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c2f9cc4400 0x55c2f9cc5870 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.239+0000 7fd04f07f740 1 --2- 172.21.15.103:0/1786015209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9cb0ce0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.239+0000 7fd02dffb640 0 client.15126 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:49:02.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.239+0000 7fd04f07f740 1 --2- 172.21.15.103:0/1786015209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cb07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.239+0000 7fd04f07f740 1 --2- 172.21.15.103:0/1786015209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca89d0 0x55c2f9cb3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.239+0000 7fd04f07f740 1 -- 172.21.15.103:0/1786015209 >> 172.21.15.103:0/1786015209 conn(0x55c2f9ca4f60 msgr2=0x55c2f9caac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:02.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.239+0000 7fd04f07f740 1 -- 172.21.15.103:0/1786015209 shutdown_connections 2024-09-19T06:49:02.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.239+0000 7fd04f07f740 1 -- 172.21.15.103:0/1786015209 wait complete. 2024-09-19T06:49:02.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.239+0000 7fd04f07f740 20 client.15126 trim_cache size 0 max 16384 2024-09-19T06:49:02.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd04f07f740 1 Processor -- start 2024-09-19T06:49:02.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd04f07f740 1 -- start start 2024-09-19T06:49:02.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd04f07f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca89d0 0x55c2f9cc5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd04f07f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd04f07f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9cadf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd04f07f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2f9cc3bb0 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd04f07f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c2f9cc4280 con 0x55c2f9ca92f0 2024-09-19T06:49:02.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd04f07f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2f9cc4950 con 0x55c2f9ca89d0 2024-09-19T06:49:02.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd046d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9cadf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:02.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd046d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9cadf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36522/0 (socket says 172.21.15.103:36522) 2024-09-19T06:49:02.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd046d9d640 1 -- 172.21.15.103:0/2474840376 learned_addr learned my addr 172.21.15.103:0/2474840376 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:02.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd045d9b640 1 --2- 172.21.15.103:0/2474840376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:02.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd04659c640 1 --2- 172.21.15.103:0/2474840376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca89d0 0x55c2f9cc5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:02.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd046d9d640 1 -- 172.21.15.103:0/2474840376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca89d0 msgr2=0x55c2f9cc5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:02.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd046d9d640 1 --2- 172.21.15.103:0/2474840376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca89d0 0x55c2f9cc5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd046d9d640 1 -- 172.21.15.103:0/2474840376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca92f0 msgr2=0x55c2f9cada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:02.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd046d9d640 1 --2- 172.21.15.103:0/2474840376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd046d9d640 1 -- 172.21.15.103:0/2474840376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f9cb3070 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd045d9b640 1 --2- 172.21.15.103:0/2474840376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:02.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.257+0000 7fd04659c640 1 --2- 172.21.15.103:0/2474840376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca89d0 0x55c2f9cc5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:02.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.258+0000 7fd046d9d640 1 --2- 172.21.15.103:0/2474840376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9cadf40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd038001eb0 tx=0x7fd0380100c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f8cfe1ac4f22b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:02.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.258+0000 7fd02ffff640 1 -- 172.21.15.103:0/2474840376 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03801a070 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.258+0000 7fd02ffff640 1 -- 172.21.15.103:0/2474840376 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd038023070 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.258+0000 7fd02ffff640 1 -- 172.21.15.103:0/2474840376 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038026e00 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.258+0000 7fd04f07f740 1 -- 172.21.15.103:0/2474840376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 msgr2=0x55c2f9cadf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:02.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.258+0000 7fd04f07f740 1 --2- 172.21.15.103:0/2474840376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9cadf40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd038001eb0 tx=0x7fd0380100c0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.258+0000 7fd04f07f740 1 -- 172.21.15.103:0/2474840376 shutdown_connections 2024-09-19T06:49:02.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.258+0000 7fd04f07f740 1 --2- 172.21.15.103:0/2474840376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9cadf40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.258+0000 7fd04f07f740 1 --2- 172.21.15.103:0/2474840376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.258+0000 7fd04f07f740 1 --2- 172.21.15.103:0/2474840376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca89d0 0x55c2f9cc5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.259+0000 7fd04f07f740 1 -- 172.21.15.103:0/2474840376 >> 172.21.15.103:0/2474840376 conn(0x55c2f9ca4f60 msgr2=0x55c2f9caac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:02.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.259+0000 7fd04f07f740 1 -- 172.21.15.103:0/2474840376 shutdown_connections 2024-09-19T06:49:02.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.259+0000 7fd04f07f740 1 -- 172.21.15.103:0/2474840376 wait complete. 2024-09-19T06:49:02.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.259+0000 7fd04f07f740 1 -- start start 2024-09-19T06:49:02.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.259+0000 7fd04f07f740 1 Processor -- start 2024-09-19T06:49:02.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.259+0000 7fd04f07f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca89d0 0x55c2f9cc9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.259+0000 7fd04f07f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cc9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.259+0000 7fd04f07f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9d2aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.259+0000 7fd04f07f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2f9cc56f0 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.259+0000 7fd04f07f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c2f9cc5020 con 0x55c2f9ca92f0 2024-09-19T06:49:02.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.259+0000 7fd04f07f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2f9cc5dc0 con 0x55c2f9ca89d0 2024-09-19T06:49:02.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.259+0000 7fd04f07f740 10 client.0 _ll_register_callbacks cb 0x7fd04009b3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:02.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.259+0000 7fd046d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9d2aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:02.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.259+0000 7fd046d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9d2aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36530/0 (socket says 172.21.15.103:36530) 2024-09-19T06:49:02.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.259+0000 7fd046d9d640 1 -- 172.21.15.103:0/3005446156 learned_addr learned my addr 172.21.15.103:0/3005446156 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:02.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd046d9d640 1 -- 172.21.15.103:0/3005446156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca89d0 msgr2=0x55c2f9cc9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:02.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd046d9d640 1 --2- 172.21.15.103:0/3005446156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca89d0 0x55c2f9cc9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd046d9d640 1 -- 172.21.15.103:0/3005446156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca92f0 msgr2=0x55c2f9cc9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:02.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd045d9b640 1 --2- 172.21.15.103:0/3005446156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cc9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:02.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd046d9d640 1 --2- 172.21.15.103:0/3005446156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cc9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd046d9d640 1 -- 172.21.15.103:0/3005446156 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f9d2b520 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd045d9b640 1 --2- 172.21.15.103:0/3005446156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cc9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:02.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd046d9d640 1 --2- 172.21.15.103:0/3005446156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9d2aee0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd0380266f0 tx=0x7fd0380095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a34fc2f30787de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:02.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd02dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:02.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd02dffb640 1 -- 172.21.15.103:0/3005446156 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03801a050 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd02dffb640 1 -- 172.21.15.103:0/3005446156 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd038026b80 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd04f07f740 1 -- 172.21.15.103:0/3005446156 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd038023070 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd02dffb640 1 -- 172.21.15.103:0/3005446156 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd03804a050 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd04f07f740 10 client.15132 fetch_fsmap learned FSMap version 26 2024-09-19T06:49:02.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd04f07f740 1 -- 172.21.15.103:0/3005446156 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2f9d32800 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd02dffb640 1 -- 172.21.15.103:0/3005446156 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd038023070 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.260+0000 7fd02dffb640 1 -- 172.21.15.103:0/3005446156 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fd03803a030 con 0x55c2f9cb3bb0 2024-09-19T06:49:02.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.261+0000 7fd04f07f740 10 client.15132 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:49:02.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.261+0000 7fd04f07f740 10 client.15132 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T06:49:02.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.261+0000 7fd04f07f740 20 client.15132 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:02.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.261+0000 7fd04f07f740 1 --2- 172.21.15.103:0/3005446156 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c2f9d37020 0x55c2f9d35d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:02.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.261+0000 7fd04f07f740 4 client.15132 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:49:02.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.261+0000 7fd04f07f740 1 -- 172.21.15.103:0/3005446156 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "status"}) -- 0x55c2f9c16b10 con 0x55c2f9d37020 2024-09-19T06:49:02.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.261+0000 7fd04659c640 1 --2- 172.21.15.103:0/3005446156 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c2f9d37020 0x55c2f9d35d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:02.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.261+0000 7fd02dffb640 10 client.15132 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:49:02.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.261+0000 7fd04659c640 1 --2- 172.21.15.103:0/3005446156 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c2f9d37020 0x55c2f9d35d30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=66c88f235739faca server_cookie=ce24d0a9ed6eb315 in_seq=0 out_seq=0 2024-09-19T06:49:02.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.261+0000 7fd02dffb640 1 -- 172.21.15.103:0/3005446156 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c2f9c16b10 con 0x55c2f9d37020 2024-09-19T06:49:02.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.261+0000 7fd02dffb640 10 client.15132 handle_command_reply: tid=0 2024-09-19T06:49:02.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.261+0000 7fd04f07f740 1 client.15132 shutdown 2024-09-19T06:49:02.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04f07f740 1 -- 172.21.15.103:0/3005446156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 msgr2=0x55c2f9d2aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:02.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04f07f740 1 --2- 172.21.15.103:0/3005446156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9d2aee0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd0380266f0 tx=0x7fd0380095e0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04f07f740 1 -- 172.21.15.103:0/3005446156 shutdown_connections 2024-09-19T06:49:02.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04f07f740 1 --2- 172.21.15.103:0/3005446156 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c2f9d37020 0x55c2f9d35d30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04f07f740 1 --2- 172.21.15.103:0/3005446156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f9cb3bb0 0x55c2f9d2aee0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04f07f740 1 --2- 172.21.15.103:0/3005446156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f9ca92f0 0x55c2f9cc9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04f07f740 1 --2- 172.21.15.103:0/3005446156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f9ca89d0 0x55c2f9cc9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04f07f740 1 -- 172.21.15.103:0/3005446156 >> 172.21.15.103:0/3005446156 conn(0x55c2f9ca4f60 msgr2=0x55c2f9cab590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:02.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd02dffb640 0 client.15132 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:49:02.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04f07f740 1 -- 172.21.15.103:0/3005446156 shutdown_connections 2024-09-19T06:49:02.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04f07f740 1 -- 172.21.15.103:0/3005446156 wait complete. 2024-09-19T06:49:02.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04f07f740 20 client.15132 trim_cache size 0 max 16384 2024-09-19T06:49:02.268 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:49:02.269 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:49:02.269 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 3, 2024-09-19T06:49:02.269 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24436, 2024-09-19T06:49:02.269 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T06:49:02.269 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T06:49:02.269 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T06:49:02.269 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 87.372715463999995, 2024-09-19T06:49:02.269 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T06:49:02.269 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 82, 2024-09-19T06:49:02.269 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T06:49:02.269 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 121.168732808 2024-09-19T06:49:02.269 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:49:02.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04cca6640 1 -- 172.21.15.103:0/4101493030 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fd0340a32d0 msgr2=0x7fd0340a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:02.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04cca6640 1 --2- 172.21.15.103:0/4101493030 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fd0340a32d0 0x7fd0340a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd03801d000 tx=0x7fd038019000 comp rx=0 tx=0).stop 2024-09-19T06:49:02.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04cca6640 1 -- 172.21.15.103:0/4101493030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd048100a80 msgr2=0x7fd0481974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:02.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04cca6640 1 --2- 172.21.15.103:0/4101493030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd048100a80 0x7fd0481974f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd04000ba50 tx=0x7fd04000bf10 comp rx=0 tx=0).stop 2024-09-19T06:49:02.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04cca6640 1 -- 172.21.15.103:0/4101493030 shutdown_connections 2024-09-19T06:49:02.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04cca6640 1 --2- 172.21.15.103:0/4101493030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048106a80 0x7fd04819bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04cca6640 1 --2- 172.21.15.103:0/4101493030 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fd0340a32d0 0x7fd0340a5790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04cca6640 1 --2- 172.21.15.103:0/4101493030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd048101420 0x7fd048197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04cca6640 1 --2- 172.21.15.103:0/4101493030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd048100a80 0x7fd0481974f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:02.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04cca6640 1 -- 172.21.15.103:0/4101493030 >> 172.21.15.103:0/4101493030 conn(0x7fd0480fc7a0 msgr2=0x7fd0480fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:02.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04cca6640 1 -- 172.21.15.103:0/4101493030 shutdown_connections 2024-09-19T06:49:02.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:02.262+0000 7fd04cca6640 1 -- 172.21.15.103:0/4101493030 wait complete. 2024-09-19T06:49:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:01 smithi174 ceph-mon[35637]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:49:02.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:01 smithi134 ceph-mon[35356]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T06:49:03.062 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T06:49:03.308 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:03.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.711+0000 7f6672d40640 1 -- 172.21.15.103:0/3731882523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c101de0 msgr2=0x7f666c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:03.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.711+0000 7f6672d40640 1 --2- 172.21.15.103:0/3731882523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c101de0 0x7f666c1021c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6664009900 tx=0x7f666402a410 comp rx=0 tx=0).stop 2024-09-19T06:49:03.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.711+0000 7f6672d40640 1 -- 172.21.15.103:0/3731882523 shutdown_connections 2024-09-19T06:49:03.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.711+0000 7f6672d40640 1 --2- 172.21.15.103:0/3731882523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f666c106f10 0x7f666c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.711+0000 7f6672d40640 1 --2- 172.21.15.103:0/3731882523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f666c102790 0x7f666c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.711+0000 7f6672d40640 1 --2- 172.21.15.103:0/3731882523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c101de0 0x7f666c1021c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.711+0000 7f6672d40640 1 -- 172.21.15.103:0/3731882523 >> 172.21.15.103:0/3731882523 conn(0x7f666c0fd8c0 msgr2=0x7f666c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:03.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.711+0000 7f6672d40640 1 -- 172.21.15.103:0/3731882523 shutdown_connections 2024-09-19T06:49:03.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.711+0000 7f6672d40640 1 -- 172.21.15.103:0/3731882523 wait complete. 2024-09-19T06:49:03.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.711+0000 7f6672d40640 1 Processor -- start 2024-09-19T06:49:03.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.711+0000 7f6672d40640 1 -- start start 2024-09-19T06:49:03.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f6672d40640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f666c101de0 0x7f666c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f6672d40640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c102790 0x7f666c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f6672d40640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f666c106f10 0x7f666c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f6672d40640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f666c112010 con 0x7f666c102790 2024-09-19T06:49:03.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f6672d40640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f666c111e90 con 0x7f666c101de0 2024-09-19T06:49:03.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f6672d40640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f666c112190 con 0x7f666c106f10 2024-09-19T06:49:03.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f666bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c102790 0x7f666c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f666bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c102790 0x7f666c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36546/0 (socket says 172.21.15.103:36546) 2024-09-19T06:49:03.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f666bfff640 1 -- 172.21.15.103:0/2057276379 learned_addr learned my addr 172.21.15.103:0/2057276379 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:03.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f6670adc640 1 --2- 172.21.15.103:0/2057276379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f666c101de0 0x7f666c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f66712dd640 1 --2- 172.21.15.103:0/2057276379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f666c106f10 0x7f666c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f666bfff640 1 -- 172.21.15.103:0/2057276379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f666c106f10 msgr2=0x7f666c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:03.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f666bfff640 1 --2- 172.21.15.103:0/2057276379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f666c106f10 0x7f666c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f666bfff640 1 -- 172.21.15.103:0/2057276379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f666c101de0 msgr2=0x7f666c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:03.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f666bfff640 1 --2- 172.21.15.103:0/2057276379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f666c101de0 0x7f666c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f666bfff640 1 -- 172.21.15.103:0/2057276379 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c1a6a70 con 0x7f666c102790 2024-09-19T06:49:03.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f66712dd640 1 --2- 172.21.15.103:0/2057276379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f666c106f10 0x7f666c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:03.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f666bfff640 1 --2- 172.21.15.103:0/2057276379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c102790 0x7f666c074a30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f665c00ba50 tx=0x7f665c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:03.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f6669ffb640 1 -- 172.21.15.103:0/2057276379 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c019070 con 0x7f666c102790 2024-09-19T06:49:03.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f6669ffb640 1 -- 172.21.15.103:0/2057276379 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665c00cbe0 con 0x7f666c102790 2024-09-19T06:49:03.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f6672d40640 1 -- 172.21.15.103:0/2057276379 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666c1a6d60 con 0x7f666c102790 2024-09-19T06:49:03.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.712+0000 7f6669ffb640 1 -- 172.21.15.103:0/2057276379 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0076e0 con 0x7f666c102790 2024-09-19T06:49:03.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.713+0000 7f6672d40640 1 -- 172.21.15.103:0/2057276379 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666c1a71f0 con 0x7f666c102790 2024-09-19T06:49:03.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.715+0000 7f6669ffb640 1 -- 172.21.15.103:0/2057276379 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f665c01c070 con 0x7f666c102790 2024-09-19T06:49:03.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.715+0000 7f6675119740 1 Processor -- start 2024-09-19T06:49:03.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.715+0000 7f6675119740 1 -- start start 2024-09-19T06:49:03.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.715+0000 7f6669ffb640 1 --2- 172.21.15.103:0/2057276379 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f66580a32d0 0x7f66580a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.715+0000 7f6675119740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 0x559a55505db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.715+0000 7f6675119740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a555062f0 0x559a55510670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.715+0000 7f6675119740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a55510bb0 0x559a55512fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.715+0000 7f6675119740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559a5546c090 con 0x559a555062f0 2024-09-19T06:49:03.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.715+0000 7f6675119740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559a55413a70 con 0x559a555059d0 2024-09-19T06:49:03.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.715+0000 7f6675119740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559a55468050 con 0x559a55510bb0 2024-09-19T06:49:03.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.715+0000 7f66712dd640 1 --2- 172.21.15.103:0/2057276379 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f66580a32d0 0x7f66580a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.715+0000 7f666bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a555062f0 0x559a55510670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.716+0000 7f666bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a555062f0 0x559a55510670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36556/0 (socket says 172.21.15.103:36556) 2024-09-19T06:49:03.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.716+0000 7f666bfff640 1 -- 172.21.15.103:0/3227028200 learned_addr learned my addr 172.21.15.103:0/3227028200 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:03.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.716+0000 7f6670adc640 1 --2- 172.21.15.103:0/3227028200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 0x559a55505db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.716+0000 7f6669ffb640 1 -- 172.21.15.103:0/2057276379 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f665c0cefe0 con 0x7f666c102790 2024-09-19T06:49:03.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.716+0000 7f66712dd640 1 --2- 172.21.15.103:0/3227028200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a55510bb0 0x559a55512fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.716+0000 7f666bfff640 1 -- 172.21.15.103:0/3227028200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a55510bb0 msgr2=0x559a55512fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:03.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.716+0000 7f666bfff640 1 --2- 172.21.15.103:0/3227028200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a55510bb0 0x559a55512fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.716+0000 7f666bfff640 1 -- 172.21.15.103:0/3227028200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 msgr2=0x559a55505db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:03.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.716+0000 7f666bfff640 1 --2- 172.21.15.103:0/3227028200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 0x559a55505db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.716+0000 7f666bfff640 1 -- 172.21.15.103:0/3227028200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a55420130 con 0x559a555062f0 2024-09-19T06:49:03.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.716+0000 7f66712dd640 1 --2- 172.21.15.103:0/2057276379 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f66580a32d0 0x7f66580a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6654009d70 tx=0x7f6654014660 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:03.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.716+0000 7f666bfff640 1 --2- 172.21.15.103:0/3227028200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a555062f0 0x559a55510670 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f665c0cca80 tx=0x7f665c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1158d8447a59e4bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:03.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.716+0000 7f6662ffd640 1 -- 172.21.15.103:0/3227028200 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c09f910 con 0x559a555062f0 2024-09-19T06:49:03.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.716+0000 7f6662ffd640 1 -- 172.21.15.103:0/3227028200 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665c0a41d0 con 0x559a555062f0 2024-09-19T06:49:03.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.716+0000 7f6662ffd640 1 -- 172.21.15.103:0/3227028200 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0a3d40 con 0x559a555062f0 2024-09-19T06:49:03.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.717+0000 7f6675119740 1 -- 172.21.15.103:0/3227028200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a555062f0 msgr2=0x559a55510670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:03.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.717+0000 7f6675119740 1 --2- 172.21.15.103:0/3227028200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a555062f0 0x559a55510670 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f665c0cca80 tx=0x7f665c004b50 comp rx=0 tx=0).stop 2024-09-19T06:49:03.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.717+0000 7f6675119740 1 -- 172.21.15.103:0/3227028200 shutdown_connections 2024-09-19T06:49:03.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.717+0000 7f6675119740 1 --2- 172.21.15.103:0/3227028200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a55510bb0 0x559a55512fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.717+0000 7f6675119740 1 --2- 172.21.15.103:0/3227028200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a555062f0 0x559a55510670 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.717+0000 7f6675119740 1 --2- 172.21.15.103:0/3227028200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 0x559a55505db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.717+0000 7f6675119740 1 -- 172.21.15.103:0/3227028200 >> 172.21.15.103:0/3227028200 conn(0x559a55501f60 msgr2=0x559a55502340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:03.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.717+0000 7f6675119740 1 -- 172.21.15.103:0/3227028200 shutdown_connections 2024-09-19T06:49:03.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.717+0000 7f6675119740 1 -- 172.21.15.103:0/3227028200 wait complete. 2024-09-19T06:49:03.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.717+0000 7f6675119740 1 -- start start 2024-09-19T06:49:03.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f6675119740 1 Processor -- start 2024-09-19T06:49:03.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f6675119740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 0x559a55510490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f6675119740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a555062f0 0x559a5550d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f6675119740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5550dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f6675119740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559a55513780 con 0x559a55510bb0 2024-09-19T06:49:03.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f6675119740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559a55426f20 con 0x559a555059d0 2024-09-19T06:49:03.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f6675119740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559a55513900 con 0x559a555062f0 2024-09-19T06:49:03.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f6675119740 10 client.0 _ll_register_callbacks cb 0x559a553f0030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:03.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f66712dd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5550dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f66712dd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5550dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36558/0 (socket says 172.21.15.103:36558) 2024-09-19T06:49:03.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f66712dd640 1 -- 172.21.15.103:0/2361438792 learned_addr learned my addr 172.21.15.103:0/2361438792 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:03.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f6670adc640 1 --2- 172.21.15.103:0/2361438792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 0x559a55510490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f666bfff640 1 --2- 172.21.15.103:0/2361438792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a555062f0 0x559a5550d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f66712dd640 1 -- 172.21.15.103:0/2361438792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a555062f0 msgr2=0x559a5550d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:03.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f66712dd640 1 --2- 172.21.15.103:0/2361438792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a555062f0 0x559a5550d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f66712dd640 1 -- 172.21.15.103:0/2361438792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 msgr2=0x559a55510490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:03.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f66712dd640 1 --2- 172.21.15.103:0/2361438792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 0x559a55510490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f66712dd640 1 -- 172.21.15.103:0/2361438792 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a551dd2b0 con 0x559a55510bb0 2024-09-19T06:49:03.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.718+0000 7f6670adc640 1 --2- 172.21.15.103:0/2361438792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 0x559a55510490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:03.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.719+0000 7f66712dd640 1 --2- 172.21.15.103:0/2361438792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5550dd20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6654007fd0 tx=0x7f66540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6171c80d646b633 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:03.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.719+0000 7f664bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:03.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.719+0000 7f664bfff640 1 -- 172.21.15.103:0/2361438792 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654016e40 con 0x559a55510bb0 2024-09-19T06:49:03.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.719+0000 7f664bfff640 1 -- 172.21.15.103:0/2361438792 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6654017d50 con 0x559a55510bb0 2024-09-19T06:49:03.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.719+0000 7f6675119740 1 -- 172.21.15.103:0/2361438792 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f665c0a41d0 con 0x559a55510bb0 2024-09-19T06:49:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.719+0000 7f664bfff640 1 -- 172.21.15.103:0/2361438792 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f66540059c0 con 0x559a55510bb0 2024-09-19T06:49:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.719+0000 7f6675119740 10 client.15144 fetch_fsmap learned FSMap version 26 2024-09-19T06:49:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.719+0000 7f6675119740 1 -- 172.21.15.103:0/2361438792 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559a5551f0f0 con 0x559a55510bb0 2024-09-19T06:49:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.719+0000 7f664bfff640 1 -- 172.21.15.103:0/2361438792 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665c0a41d0 con 0x559a55510bb0 2024-09-19T06:49:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.719+0000 7f664bfff640 1 -- 172.21.15.103:0/2361438792 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f665402d430 con 0x559a55510bb0 2024-09-19T06:49:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.719+0000 7f6675119740 10 client.15144 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:49:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.719+0000 7f6675119740 10 client.15144 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T06:49:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.719+0000 7f6675119740 20 client.15144 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.719+0000 7f6675119740 1 --2- 172.21.15.103:0/2361438792 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x559a55521770 0x559a55522be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.719+0000 7f6675119740 4 client.15144 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.719+0000 7f6675119740 1 -- 172.21.15.103:0/2361438792 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a55479600 con 0x559a55521770 2024-09-19T06:49:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.720+0000 7f6670adc640 1 --2- 172.21.15.103:0/2361438792 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x559a55521770 0x559a55522be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.720+0000 7f6670adc640 1 --2- 172.21.15.103:0/2361438792 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x559a55521770 0x559a55522be0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ba08d8324d59db1 server_cookie=b4b9d486d57b4a98 in_seq=0 out_seq=0 2024-09-19T06:49:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.720+0000 7f664bfff640 10 client.15144 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:49:03.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.721+0000 7f664bfff640 1 -- 172.21.15.103:0/2361438792 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a55479600 con 0x559a55521770 2024-09-19T06:49:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.721+0000 7f664bfff640 10 client.15144 handle_command_reply: tid=0 2024-09-19T06:49:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.721+0000 7f6675119740 1 client.15144 shutdown 2024-09-19T06:49:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.721+0000 7f6675119740 1 -- 172.21.15.103:0/2361438792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 msgr2=0x559a5550dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.721+0000 7f6675119740 1 --2- 172.21.15.103:0/2361438792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5550dd20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6654007fd0 tx=0x7f66540046c0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.721+0000 7f6675119740 1 -- 172.21.15.103:0/2361438792 shutdown_connections 2024-09-19T06:49:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.721+0000 7f6675119740 1 --2- 172.21.15.103:0/2361438792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5550dd20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.721+0000 7f6675119740 1 --2- 172.21.15.103:0/2361438792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a555062f0 0x559a5550d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.721+0000 7f6675119740 1 --2- 172.21.15.103:0/2361438792 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x559a55521770 0x559a55522be0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.721+0000 7f664bfff640 0 client.15144 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:49:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.721+0000 7f6675119740 1 --2- 172.21.15.103:0/2361438792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 0x559a55510490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.721+0000 7f6675119740 1 -- 172.21.15.103:0/2361438792 >> 172.21.15.103:0/2361438792 conn(0x559a55501f60 msgr2=0x559a55507c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:03.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.721+0000 7f6675119740 1 -- 172.21.15.103:0/2361438792 shutdown_connections 2024-09-19T06:49:03.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.721+0000 7f6675119740 1 -- 172.21.15.103:0/2361438792 wait complete. 2024-09-19T06:49:03.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.721+0000 7f6675119740 20 client.15144 trim_cache size 0 max 16384 2024-09-19T06:49:03.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.738+0000 7f6675119740 1 Processor -- start 2024-09-19T06:49:03.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.738+0000 7f6675119740 1 -- start start 2024-09-19T06:49:03.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f6675119740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 0x559a55522be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f6675119740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a555062f0 0x559a5550d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f6675119740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5550dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f6675119740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559a55520f20 con 0x559a55510bb0 2024-09-19T06:49:03.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f6675119740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559a555215f0 con 0x559a555059d0 2024-09-19T06:49:03.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f6675119740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559a55521cc0 con 0x559a555062f0 2024-09-19T06:49:03.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f66712dd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5550dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f66712dd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5550dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36568/0 (socket says 172.21.15.103:36568) 2024-09-19T06:49:03.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f66712dd640 1 -- 172.21.15.103:0/3589154883 learned_addr learned my addr 172.21.15.103:0/3589154883 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:03.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f6670adc640 1 --2- 172.21.15.103:0/3589154883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 0x559a55522be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f666bfff640 1 --2- 172.21.15.103:0/3589154883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a555062f0 0x559a5550d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f66712dd640 1 -- 172.21.15.103:0/3589154883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a555062f0 msgr2=0x559a5550d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:03.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f66712dd640 1 --2- 172.21.15.103:0/3589154883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a555062f0 0x559a5550d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f66712dd640 1 -- 172.21.15.103:0/3589154883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 msgr2=0x559a55522be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:03.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f66712dd640 1 --2- 172.21.15.103:0/3589154883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 0x559a55522be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f66712dd640 1 -- 172.21.15.103:0/3589154883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a555100b0 con 0x559a55510bb0 2024-09-19T06:49:03.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f6670adc640 1 --2- 172.21.15.103:0/3589154883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 0x559a55522be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:03.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f66712dd640 1 --2- 172.21.15.103:0/3589154883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5550dce0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f66540092b0 tx=0x7f6654014be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd00fb3d63075ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:03.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f66627fc640 1 -- 172.21.15.103:0/3589154883 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654016bd0 con 0x559a55510bb0 2024-09-19T06:49:03.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f66627fc640 1 -- 172.21.15.103:0/3589154883 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6654027070 con 0x559a55510bb0 2024-09-19T06:49:03.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.739+0000 7f66627fc640 1 -- 172.21.15.103:0/3589154883 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66540228d0 con 0x559a55510bb0 2024-09-19T06:49:03.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.740+0000 7f6675119740 1 -- 172.21.15.103:0/3589154883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 msgr2=0x559a5550dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:03.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.740+0000 7f6675119740 1 --2- 172.21.15.103:0/3589154883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5550dce0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f66540092b0 tx=0x7f6654014be0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.740+0000 7f6675119740 1 -- 172.21.15.103:0/3589154883 shutdown_connections 2024-09-19T06:49:03.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.740+0000 7f6675119740 1 --2- 172.21.15.103:0/3589154883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5550dce0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.740+0000 7f6675119740 1 --2- 172.21.15.103:0/3589154883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a555062f0 0x559a5550d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.740+0000 7f6675119740 1 --2- 172.21.15.103:0/3589154883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555059d0 0x559a55522be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.740+0000 7f6675119740 1 -- 172.21.15.103:0/3589154883 >> 172.21.15.103:0/3589154883 conn(0x559a55501f60 msgr2=0x559a55502340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.740+0000 7f6675119740 1 -- 172.21.15.103:0/3589154883 shutdown_connections 2024-09-19T06:49:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.740+0000 7f6675119740 1 -- 172.21.15.103:0/3589154883 wait complete. 2024-09-19T06:49:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.740+0000 7f6675119740 1 -- start start 2024-09-19T06:49:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.740+0000 7f6675119740 1 Processor -- start 2024-09-19T06:49:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f6675119740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a555059d0 0x559a555297b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f6675119740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555062f0 0x559a55529cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f6675119740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5552a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f6675119740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559a55522a60 con 0x559a55510bb0 2024-09-19T06:49:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f6675119740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559a55522390 con 0x559a555062f0 2024-09-19T06:49:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f6675119740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559a55523130 con 0x559a555059d0 2024-09-19T06:49:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f6675119740 10 client.0 _ll_register_callbacks cb 0x7f66540090c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f66712dd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5552a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f66712dd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5552a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36580/0 (socket says 172.21.15.103:36580) 2024-09-19T06:49:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f66712dd640 1 -- 172.21.15.103:0/999510289 learned_addr learned my addr 172.21.15.103:0/999510289 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f6670adc640 1 --2- 172.21.15.103:0/999510289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a555059d0 0x559a555297b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f666bfff640 1 --2- 172.21.15.103:0/999510289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555062f0 0x559a55529cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f66712dd640 1 -- 172.21.15.103:0/999510289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a555059d0 msgr2=0x559a555297b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f66712dd640 1 --2- 172.21.15.103:0/999510289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a555059d0 0x559a555297b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f66712dd640 1 -- 172.21.15.103:0/999510289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555062f0 msgr2=0x559a55529cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f66712dd640 1 --2- 172.21.15.103:0/999510289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555062f0 0x559a55529cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.741+0000 7f66712dd640 1 -- 172.21.15.103:0/999510289 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a55523f40 con 0x559a55510bb0 2024-09-19T06:49:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.742+0000 7f66712dd640 1 --2- 172.21.15.103:0/999510289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5552a230 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6654014c80 tx=0x7f66540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcea55b705c0e6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.742+0000 7f664bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.742+0000 7f664bfff640 1 -- 172.21.15.103:0/999510289 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654043060 con 0x559a55510bb0 2024-09-19T06:49:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.742+0000 7f664bfff640 1 -- 172.21.15.103:0/999510289 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f665402dc40 con 0x559a55510bb0 2024-09-19T06:49:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.742+0000 7f6675119740 1 -- 172.21.15.103:0/999510289 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6654027070 con 0x559a55510bb0 2024-09-19T06:49:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.742+0000 7f664bfff640 1 -- 172.21.15.103:0/999510289 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f665402de40 con 0x559a55510bb0 2024-09-19T06:49:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.742+0000 7f6675119740 10 client.15150 fetch_fsmap learned FSMap version 26 2024-09-19T06:49:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.742+0000 7f6675119740 1 -- 172.21.15.103:0/999510289 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559a55528990 con 0x559a55510bb0 2024-09-19T06:49:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.742+0000 7f664bfff640 1 -- 172.21.15.103:0/999510289 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6654027070 con 0x559a55510bb0 2024-09-19T06:49:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.742+0000 7f664bfff640 1 -- 172.21.15.103:0/999510289 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f66540237d0 con 0x559a55510bb0 2024-09-19T06:49:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.742+0000 7f6675119740 10 client.15150 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:49:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.742+0000 7f6675119740 10 client.15150 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T06:49:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.742+0000 7f6675119740 20 client.15150 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.742+0000 7f6675119740 1 --2- 172.21.15.103:0/999510289 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x559a55521770 0x559a5558e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.742+0000 7f6675119740 4 client.15150 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:49:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.742+0000 7f6675119740 1 -- 172.21.15.103:0/999510289 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "status"}) -- 0x559a55473b10 con 0x559a55521770 2024-09-19T06:49:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.743+0000 7f6670adc640 1 --2- 172.21.15.103:0/999510289 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x559a55521770 0x559a5558e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.743+0000 7f664bfff640 10 client.15150 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:49:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.743+0000 7f6670adc640 1 --2- 172.21.15.103:0/999510289 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x559a55521770 0x559a5558e540 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e735c1d357cce73 server_cookie=e43e42ef56436257 in_seq=0 out_seq=0 2024-09-19T06:49:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.743+0000 7f664bfff640 1 -- 172.21.15.103:0/999510289 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559a55473b10 con 0x559a55521770 2024-09-19T06:49:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.743+0000 7f664bfff640 10 client.15150 handle_command_reply: tid=0 2024-09-19T06:49:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.743+0000 7f6675119740 1 client.15150 shutdown 2024-09-19T06:49:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.743+0000 7f6675119740 1 -- 172.21.15.103:0/999510289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 msgr2=0x559a5552a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.743+0000 7f6675119740 1 --2- 172.21.15.103:0/999510289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5552a230 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6654014c80 tx=0x7f66540046c0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.743+0000 7f6675119740 1 -- 172.21.15.103:0/999510289 shutdown_connections 2024-09-19T06:49:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.743+0000 7f6675119740 1 --2- 172.21.15.103:0/999510289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a55510bb0 0x559a5552a230 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.743+0000 7f6675119740 1 --2- 172.21.15.103:0/999510289 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x559a55521770 0x559a5558e540 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.743+0000 7f6675119740 1 --2- 172.21.15.103:0/999510289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a555062f0 0x559a55529cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.743+0000 7f6675119740 1 --2- 172.21.15.103:0/999510289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a555059d0 0x559a555297b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.743+0000 7f664bfff640 0 client.15150 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:49:03.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.743+0000 7f6675119740 1 -- 172.21.15.103:0/999510289 >> 172.21.15.103:0/999510289 conn(0x559a55501f60 msgr2=0x559a5550d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:03.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.744+0000 7f6675119740 1 -- 172.21.15.103:0/999510289 shutdown_connections 2024-09-19T06:49:03.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.744+0000 7f6675119740 1 -- 172.21.15.103:0/999510289 wait complete. 2024-09-19T06:49:03.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.744+0000 7f6675119740 20 client.15150 trim_cache size 0 max 16384 2024-09-19T06:49:03.754 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:49:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:49:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 2, 2024-09-19T06:49:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24454, 2024-09-19T06:49:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T06:49:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T06:49:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T06:49:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 89.863747286000006, 2024-09-19T06:49:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T06:49:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 82, 2024-09-19T06:49:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T06:49:03.755 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 117.481143879 2024-09-19T06:49:03.755 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:49:03.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.744+0000 7f6672d40640 1 -- 172.21.15.103:0/2057276379 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f66580a32d0 msgr2=0x7f66580a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:03.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.744+0000 7f6672d40640 1 --2- 172.21.15.103:0/2057276379 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f66580a32d0 0x7f66580a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6654009d70 tx=0x7f6654014660 comp rx=0 tx=0).stop 2024-09-19T06:49:03.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.744+0000 7f6672d40640 1 -- 172.21.15.103:0/2057276379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c102790 msgr2=0x7f666c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:03.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.744+0000 7f6672d40640 1 --2- 172.21.15.103:0/2057276379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c102790 0x7f666c074a30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f665c00ba50 tx=0x7f665c00bf10 comp rx=0 tx=0).stop 2024-09-19T06:49:03.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.744+0000 7f6672d40640 1 -- 172.21.15.103:0/2057276379 shutdown_connections 2024-09-19T06:49:03.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.744+0000 7f6672d40640 1 --2- 172.21.15.103:0/2057276379 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f66580a32d0 0x7f66580a5790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.744+0000 7f6672d40640 1 --2- 172.21.15.103:0/2057276379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f666c106f10 0x7f666c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.744+0000 7f6672d40640 1 --2- 172.21.15.103:0/2057276379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c102790 0x7f666c074a30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.744+0000 7f6672d40640 1 --2- 172.21.15.103:0/2057276379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f666c101de0 0x7f666c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.744+0000 7f6672d40640 1 -- 172.21.15.103:0/2057276379 >> 172.21.15.103:0/2057276379 conn(0x7f666c0fd8c0 msgr2=0x7f666c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:03.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.744+0000 7f6672d40640 1 -- 172.21.15.103:0/2057276379 shutdown_connections 2024-09-19T06:49:03.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:03.744+0000 7f6672d40640 1 -- 172.21.15.103:0/2057276379 wait complete. 2024-09-19T06:49:04.665 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T06:49:04.925 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:04.942 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:04 smithi103 ceph-mon[31158]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:49:05.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:04 smithi134 ceph-mon[35356]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:49:05.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:04 smithi174 ceph-mon[35637]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T06:49:05.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.398+0000 7fca059f7640 1 -- 172.21.15.103:0/3849496421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca00102db0 msgr2=0x7fca00103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:05.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.398+0000 7fca059f7640 1 --2- 172.21.15.103:0/3849496421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca00102db0 0x7fca00103190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009930 tx=0x7fc9f002a420 comp rx=0 tx=0).stop 2024-09-19T06:49:05.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.398+0000 7fca059f7640 1 -- 172.21.15.103:0/3849496421 shutdown_connections 2024-09-19T06:49:05.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.398+0000 7fca059f7640 1 --2- 172.21.15.103:0/3849496421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca001082e0 0x7fca0010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.398+0000 7fca059f7640 1 --2- 172.21.15.103:0/3849496421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca00103760 0x7fca00107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.398+0000 7fca059f7640 1 --2- 172.21.15.103:0/3849496421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca00102db0 0x7fca00103190 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.398+0000 7fca059f7640 1 -- 172.21.15.103:0/3849496421 >> 172.21.15.103:0/3849496421 conn(0x7fca000fc7d0 msgr2=0x7fca000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:05.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.398+0000 7fca059f7640 1 -- 172.21.15.103:0/3849496421 shutdown_connections 2024-09-19T06:49:05.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.398+0000 7fca059f7640 1 -- 172.21.15.103:0/3849496421 wait complete. 2024-09-19T06:49:05.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.399+0000 7fca059f7640 1 Processor -- start 2024-09-19T06:49:05.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.399+0000 7fca059f7640 1 -- start start 2024-09-19T06:49:05.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fca059f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca00102db0 0x7fca00100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fca059f7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca00103760 0x7fca00100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fca059f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca001082e0 0x7fca00101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fca059f7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca00112090 con 0x7fca001082e0 2024-09-19T06:49:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fca059f7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca00111f10 con 0x7fca00103760 2024-09-19T06:49:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fca059f7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca00112210 con 0x7fca00102db0 2024-09-19T06:49:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fc9ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca001082e0 0x7fca00101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fc9ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca001082e0 0x7fca00101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36598/0 (socket says 172.21.15.103:36598) 2024-09-19T06:49:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fc9ff7fe640 1 -- 172.21.15.103:0/834613331 learned_addr learned my addr 172.21.15.103:0/834613331 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fc9feffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca00102db0 0x7fca00100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fc9fe7fc640 1 --2- 172.21.15.103:0/834613331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca00103760 0x7fca00100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fc9ff7fe640 1 -- 172.21.15.103:0/834613331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca00102db0 msgr2=0x7fca00100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fc9ff7fe640 1 --2- 172.21.15.103:0/834613331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca00102db0 0x7fca00100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fc9ff7fe640 1 -- 172.21.15.103:0/834613331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca00103760 msgr2=0x7fca00100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fc9ff7fe640 1 --2- 172.21.15.103:0/834613331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca00103760 0x7fca00100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fc9ff7fe640 1 -- 172.21.15.103:0/834613331 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca001a69f0 con 0x7fca001082e0 2024-09-19T06:49:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fc9ff7fe640 1 --2- 172.21.15.103:0/834613331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca001082e0 0x7fca00101490 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800cce0 tx=0x7fc9f800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fca049f5640 1 -- 172.21.15.103:0/834613331 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f800f450 con 0x7fca001082e0 2024-09-19T06:49:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fca059f7640 1 -- 172.21.15.103:0/834613331 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca001a6ce0 con 0x7fca001082e0 2024-09-19T06:49:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.400+0000 7fca049f5640 1 -- 172.21.15.103:0/834613331 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9f800fad0 con 0x7fca001082e0 2024-09-19T06:49:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.401+0000 7fca049f5640 1 -- 172.21.15.103:0/834613331 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8018c10 con 0x7fca001082e0 2024-09-19T06:49:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.401+0000 7fca059f7640 1 -- 172.21.15.103:0/834613331 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca001a71c0 con 0x7fca001082e0 2024-09-19T06:49:05.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.403+0000 7fca049f5640 1 -- 172.21.15.103:0/834613331 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc9f800f5f0 con 0x7fca001082e0 2024-09-19T06:49:05.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.403+0000 7fca07dd0740 1 Processor -- start 2024-09-19T06:49:05.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.403+0000 7fca07dd0740 1 -- start start 2024-09-19T06:49:05.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fca07dd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e48f9d0 0x559b1e48fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fca07dd0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e4902f0 0x559b1e49a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fca07dd0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e49abb0 0x559b1e49cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fca07dd0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559b1e3f6090 con 0x559b1e4902f0 2024-09-19T06:49:05.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fca07dd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559b1e39da70 con 0x559b1e48f9d0 2024-09-19T06:49:05.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fca07dd0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559b1e3f2050 con 0x559b1e49abb0 2024-09-19T06:49:05.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fc9feffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e4902f0 0x559b1e49a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fc9feffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e4902f0 0x559b1e49a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36608/0 (socket says 172.21.15.103:36608) 2024-09-19T06:49:05.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fc9feffd640 1 -- 172.21.15.103:0/644195465 learned_addr learned my addr 172.21.15.103:0/644195465 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:05.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fc9fe7fc640 1 --2- 172.21.15.103:0/644195465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e48f9d0 0x559b1e48fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fc9fe7fc640 1 --2- 172.21.15.103:0/644195465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e49abb0 0x559b1e49cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fca049f5640 1 --2- 172.21.15.103:0/834613331 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc9ec0a32d0 0x7fc9ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fc9feffd640 1 -- 172.21.15.103:0/644195465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e49abb0 msgr2=0x559b1e49cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:05.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fc9feffd640 1 --2- 172.21.15.103:0/644195465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e49abb0 0x559b1e49cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fc9feffd640 1 -- 172.21.15.103:0/644195465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e48f9d0 msgr2=0x559b1e48fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:05.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fc9feffd640 1 --2- 172.21.15.103:0/644195465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e48f9d0 0x559b1e48fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fc9feffd640 1 -- 172.21.15.103:0/644195465 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b1e3aa130 con 0x559b1e4902f0 2024-09-19T06:49:05.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fc9fe7fc640 1 --2- 172.21.15.103:0/644195465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e48f9d0 0x559b1e48fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:05.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.404+0000 7fc9ff7fe640 1 --2- 172.21.15.103:0/834613331 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc9ec0a32d0 0x7fc9ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.405+0000 7fc9feffd640 1 --2- 172.21.15.103:0/644195465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e4902f0 0x559b1e49a670 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009930 tx=0x7fc9f0004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1dff3e9e490b7d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:05.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.405+0000 7fca049f5640 1 -- 172.21.15.103:0/834613331 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc9f80d0060 con 0x7fca001082e0 2024-09-19T06:49:05.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.405+0000 7fc9f5ffb640 1 -- 172.21.15.103:0/644195465 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0035030 con 0x559b1e4902f0 2024-09-19T06:49:05.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.405+0000 7fc9f5ffb640 1 -- 172.21.15.103:0/644195465 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9f0040070 con 0x559b1e4902f0 2024-09-19T06:49:05.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.405+0000 7fc9f5ffb640 1 -- 172.21.15.103:0/644195465 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0031c60 con 0x559b1e4902f0 2024-09-19T06:49:05.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.405+0000 7fca07dd0740 1 -- 172.21.15.103:0/644195465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e4902f0 msgr2=0x559b1e49a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:05.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.405+0000 7fc9ff7fe640 1 --2- 172.21.15.103:0/834613331 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc9ec0a32d0 0x7fc9ec0a5790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80a4c70 tx=0x7fc9f806f3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:05.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.405+0000 7fca07dd0740 1 --2- 172.21.15.103:0/644195465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e4902f0 0x559b1e49a670 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009930 tx=0x7fc9f0004200 comp rx=0 tx=0).stop 2024-09-19T06:49:05.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.406+0000 7fca07dd0740 1 -- 172.21.15.103:0/644195465 shutdown_connections 2024-09-19T06:49:05.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.406+0000 7fca07dd0740 1 --2- 172.21.15.103:0/644195465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e49abb0 0x559b1e49cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.406+0000 7fca07dd0740 1 --2- 172.21.15.103:0/644195465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e4902f0 0x559b1e49a670 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.406+0000 7fca07dd0740 1 --2- 172.21.15.103:0/644195465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e48f9d0 0x559b1e48fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.406+0000 7fca07dd0740 1 -- 172.21.15.103:0/644195465 >> 172.21.15.103:0/644195465 conn(0x559b1e48bf60 msgr2=0x559b1e48c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:05.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.406+0000 7fca07dd0740 1 -- 172.21.15.103:0/644195465 shutdown_connections 2024-09-19T06:49:05.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.406+0000 7fca07dd0740 1 -- 172.21.15.103:0/644195465 wait complete. 2024-09-19T06:49:05.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.406+0000 7fca07dd0740 1 -- start start 2024-09-19T06:49:05.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.406+0000 7fca07dd0740 1 Processor -- start 2024-09-19T06:49:05.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.406+0000 7fca07dd0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e48f9d0 0x559b1e49a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fca07dd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e4902f0 0x559b1e4977c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fca07dd0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e49abb0 0x559b1e497d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fca07dd0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559b1e49d780 con 0x559b1e49abb0 2024-09-19T06:49:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fca07dd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559b1e3b0f20 con 0x559b1e4902f0 2024-09-19T06:49:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fca07dd0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559b1e49d900 con 0x559b1e48f9d0 2024-09-19T06:49:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fca07dd0740 10 client.0 _ll_register_callbacks cb 0x559b1e37a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fc9fe7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e49abb0 0x559b1e497d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fc9feffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e4902f0 0x559b1e4977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fc9fe7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e49abb0 0x559b1e497d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36622/0 (socket says 172.21.15.103:36622) 2024-09-19T06:49:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fc9fe7fc640 1 -- 172.21.15.103:0/818316122 learned_addr learned my addr 172.21.15.103:0/818316122 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fc9fe7fc640 1 --2- 172.21.15.103:0/818316122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e48f9d0 0x559b1e49a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fc9fe7fc640 1 -- 172.21.15.103:0/818316122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e48f9d0 msgr2=0x559b1e49a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fc9fe7fc640 1 --2- 172.21.15.103:0/818316122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e48f9d0 0x559b1e49a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fc9fe7fc640 1 -- 172.21.15.103:0/818316122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e4902f0 msgr2=0x559b1e4977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fc9fe7fc640 1 --2- 172.21.15.103:0/818316122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e4902f0 0x559b1e4977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fc9fe7fc640 1 -- 172.21.15.103:0/818316122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b1e1672b0 con 0x559b1e49abb0 2024-09-19T06:49:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fc9feffd640 1 --2- 172.21.15.103:0/818316122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e4902f0 0x559b1e4977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fc9fe7fc640 1 --2- 172.21.15.103:0/818316122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e49abb0 0x559b1e497d00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8010e90 tx=0x7fc9e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cc4b81ebb2ba3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fc9deffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fc9deffd640 1 -- 172.21.15.103:0/818316122 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e801a040 con 0x559b1e49abb0 2024-09-19T06:49:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.407+0000 7fc9deffd640 1 -- 172.21.15.103:0/818316122 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc9e800c770 con 0x559b1e49abb0 2024-09-19T06:49:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.408+0000 7fca07dd0740 1 -- 172.21.15.103:0/818316122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9f0040070 con 0x559b1e49abb0 2024-09-19T06:49:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.408+0000 7fc9deffd640 1 -- 172.21.15.103:0/818316122 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc9e8004020 con 0x559b1e49abb0 2024-09-19T06:49:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.408+0000 7fca07dd0740 10 client.15162 fetch_fsmap learned FSMap version 26 2024-09-19T06:49:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.408+0000 7fca07dd0740 1 -- 172.21.15.103:0/818316122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b1e4a9160 con 0x559b1e49abb0 2024-09-19T06:49:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.408+0000 7fc9deffd640 1 -- 172.21.15.103:0/818316122 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9f0040070 con 0x559b1e49abb0 2024-09-19T06:49:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.408+0000 7fc9deffd640 1 -- 172.21.15.103:0/818316122 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fc9e802a020 con 0x559b1e49abb0 2024-09-19T06:49:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.408+0000 7fca07dd0740 10 client.15162 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:49:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.408+0000 7fca07dd0740 10 client.15162 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T06:49:05.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.408+0000 7fca07dd0740 20 client.15162 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:05.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.408+0000 7fca07dd0740 1 --2- 172.21.15.103:0/818316122 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x559b1e4ab730 0x559b1e4acba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.408+0000 7fca07dd0740 4 client.15162 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:05.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.408+0000 7fca07dd0740 1 -- 172.21.15.103:0/818316122 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b1e403600 con 0x559b1e4ab730 2024-09-19T06:49:05.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.409+0000 7fc9feffd640 1 --2- 172.21.15.103:0/818316122 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x559b1e4ab730 0x559b1e4acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.410+0000 7fc9feffd640 1 --2- 172.21.15.103:0/818316122 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x559b1e4ab730 0x559b1e4acba0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9bf2e9f0622a946 server_cookie=b8a9d7caca880c4a in_seq=0 out_seq=0 2024-09-19T06:49:05.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.410+0000 7fc9deffd640 10 client.15162 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:49:05.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.410+0000 7fc9deffd640 1 -- 172.21.15.103:0/818316122 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b1e403600 con 0x559b1e4ab730 2024-09-19T06:49:05.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.410+0000 7fc9deffd640 10 client.15162 handle_command_reply: tid=0 2024-09-19T06:49:05.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.410+0000 7fca07dd0740 1 client.15162 shutdown 2024-09-19T06:49:05.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.411+0000 7fca07dd0740 1 -- 172.21.15.103:0/818316122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e49abb0 msgr2=0x559b1e497d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:05.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.411+0000 7fca07dd0740 1 --2- 172.21.15.103:0/818316122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e49abb0 0x559b1e497d00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8010e90 tx=0x7fc9e8004730 comp rx=0 tx=0).stop 2024-09-19T06:49:05.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.411+0000 7fca07dd0740 1 -- 172.21.15.103:0/818316122 shutdown_connections 2024-09-19T06:49:05.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.411+0000 7fca07dd0740 1 --2- 172.21.15.103:0/818316122 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x559b1e4ab730 0x559b1e4acba0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.411+0000 7fca07dd0740 1 --2- 172.21.15.103:0/818316122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e49abb0 0x559b1e497d00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.411+0000 7fca07dd0740 1 --2- 172.21.15.103:0/818316122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e4902f0 0x559b1e4977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.411+0000 7fca07dd0740 1 --2- 172.21.15.103:0/818316122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e48f9d0 0x559b1e49a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.411+0000 7fca07dd0740 1 -- 172.21.15.103:0/818316122 >> 172.21.15.103:0/818316122 conn(0x559b1e48bf60 msgr2=0x559b1e491c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:05.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.411+0000 7fc9deffd640 0 client.15162 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:49:05.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.411+0000 7fca07dd0740 1 -- 172.21.15.103:0/818316122 shutdown_connections 2024-09-19T06:49:05.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.411+0000 7fca07dd0740 1 -- 172.21.15.103:0/818316122 wait complete. 2024-09-19T06:49:05.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.411+0000 7fca07dd0740 20 client.15162 trim_cache size 0 max 16384 2024-09-19T06:49:05.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fca07dd0740 1 Processor -- start 2024-09-19T06:49:05.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fca07dd0740 1 -- start start 2024-09-19T06:49:05.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fca07dd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e48f9d0 0x559b1e4ab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fca07dd0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e4902f0 0x559b1e4acba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fca07dd0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e4a6b90 0x559b1e497780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fca07dd0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559b1e4aaee0 con 0x559b1e4902f0 2024-09-19T06:49:05.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fca07dd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559b1e4ab5b0 con 0x559b1e48f9d0 2024-09-19T06:49:05.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fca07dd0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559b1e4abc80 con 0x559b1e4a6b90 2024-09-19T06:49:05.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fc9feffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e4902f0 0x559b1e4acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fc9feffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e4902f0 0x559b1e4acba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36624/0 (socket says 172.21.15.103:36624) 2024-09-19T06:49:05.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fc9feffd640 1 -- 172.21.15.103:0/3194528353 learned_addr learned my addr 172.21.15.103:0/3194528353 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:05.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fc9fe7fc640 1 --2- 172.21.15.103:0/3194528353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e48f9d0 0x559b1e4ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fc9fe7fc640 1 --2- 172.21.15.103:0/3194528353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e4a6b90 0x559b1e497780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fc9feffd640 1 -- 172.21.15.103:0/3194528353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e4a6b90 msgr2=0x559b1e497780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:05.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fc9feffd640 1 --2- 172.21.15.103:0/3194528353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e4a6b90 0x559b1e497780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fc9feffd640 1 -- 172.21.15.103:0/3194528353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e48f9d0 msgr2=0x559b1e4ab060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:05.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fc9feffd640 1 --2- 172.21.15.103:0/3194528353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e48f9d0 0x559b1e4ab060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fc9feffd640 1 -- 172.21.15.103:0/3194528353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b1e49d310 con 0x559b1e4902f0 2024-09-19T06:49:05.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.429+0000 7fc9fe7fc640 1 --2- 172.21.15.103:0/3194528353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e4a6b90 0x559b1e497780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:05.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.430+0000 7fc9feffd640 1 --2- 172.21.15.103:0/3194528353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e4902f0 0x559b1e4acba0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009930 tx=0x7fc9f002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79eb788cbcef224 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:05.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.430+0000 7fc9f57fa640 1 -- 172.21.15.103:0/3194528353 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0035040 con 0x559b1e4902f0 2024-09-19T06:49:05.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.430+0000 7fc9f57fa640 1 -- 172.21.15.103:0/3194528353 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9f0040040 con 0x559b1e4902f0 2024-09-19T06:49:05.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.430+0000 7fc9f57fa640 1 -- 172.21.15.103:0/3194528353 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0003bc0 con 0x559b1e4902f0 2024-09-19T06:49:05.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.430+0000 7fca07dd0740 1 -- 172.21.15.103:0/3194528353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e4902f0 msgr2=0x559b1e4acba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:05.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.430+0000 7fca07dd0740 1 --2- 172.21.15.103:0/3194528353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e4902f0 0x559b1e4acba0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009930 tx=0x7fc9f002a420 comp rx=0 tx=0).stop 2024-09-19T06:49:05.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.431+0000 7fca07dd0740 1 -- 172.21.15.103:0/3194528353 shutdown_connections 2024-09-19T06:49:05.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.431+0000 7fca07dd0740 1 --2- 172.21.15.103:0/3194528353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e4a6b90 0x559b1e497780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.431+0000 7fca07dd0740 1 --2- 172.21.15.103:0/3194528353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e4902f0 0x559b1e4acba0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.431+0000 7fca07dd0740 1 --2- 172.21.15.103:0/3194528353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e48f9d0 0x559b1e4ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.431+0000 7fca07dd0740 1 -- 172.21.15.103:0/3194528353 >> 172.21.15.103:0/3194528353 conn(0x559b1e48bf60 msgr2=0x559b1e486ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:05.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.431+0000 7fca07dd0740 1 -- 172.21.15.103:0/3194528353 shutdown_connections 2024-09-19T06:49:05.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.431+0000 7fca07dd0740 1 -- 172.21.15.103:0/3194528353 wait complete. 2024-09-19T06:49:05.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.431+0000 7fca07dd0740 1 -- start start 2024-09-19T06:49:05.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.431+0000 7fca07dd0740 1 Processor -- start 2024-09-19T06:49:05.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.431+0000 7fca07dd0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e48f9d0 0x559b1e4b5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.431+0000 7fca07dd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e4902f0 0x559b1e4afc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fca07dd0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e4a6b90 0x559b1e4b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fca07dd0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559b1e4aca20 con 0x559b1e48f9d0 2024-09-19T06:49:05.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fca07dd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559b1e4ac350 con 0x559b1e4902f0 2024-09-19T06:49:05.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fca07dd0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559b1e4ad0f0 con 0x559b1e4a6b90 2024-09-19T06:49:05.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fca07dd0740 10 client.0 _ll_register_callbacks cb 0x7fc9e800ac50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:05.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fc9fe7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e48f9d0 0x559b1e4b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fc9fe7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e48f9d0 0x559b1e4b5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36634/0 (socket says 172.21.15.103:36634) 2024-09-19T06:49:05.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fc9fe7fc640 1 -- 172.21.15.103:0/852813968 learned_addr learned my addr 172.21.15.103:0/852813968 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:05.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fc9feffd640 1 --2- 172.21.15.103:0/852813968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e4902f0 0x559b1e4afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fc9fe7fc640 1 -- 172.21.15.103:0/852813968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e4a6b90 msgr2=0x559b1e4b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:05.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fc9fe7fc640 1 --2- 172.21.15.103:0/852813968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e4a6b90 0x559b1e4b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fc9fe7fc640 1 -- 172.21.15.103:0/852813968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e4902f0 msgr2=0x559b1e4afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:05.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fc9fe7fc640 1 --2- 172.21.15.103:0/852813968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e4902f0 0x559b1e4afc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fc9fe7fc640 1 -- 172.21.15.103:0/852813968 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b1e513e30 con 0x559b1e48f9d0 2024-09-19T06:49:05.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fc9feffd640 1 --2- 172.21.15.103:0/852813968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e4902f0 0x559b1e4afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:05.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fc9fe7fc640 1 --2- 172.21.15.103:0/852813968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e48f9d0 0x559b1e4b5790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800fea0 tx=0x7fc9e800a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583acb471705789f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:05.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fc9deffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:05.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fc9deffd640 1 -- 172.21.15.103:0/852813968 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8003ca0 con 0x559b1e48f9d0 2024-09-19T06:49:05.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fc9deffd640 1 -- 172.21.15.103:0/852813968 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc9e8003840 con 0x559b1e48f9d0 2024-09-19T06:49:05.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.432+0000 7fca07dd0740 1 -- 172.21.15.103:0/852813968 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9f0040040 con 0x559b1e48f9d0 2024-09-19T06:49:05.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.433+0000 7fc9deffd640 1 -- 172.21.15.103:0/852813968 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc9e8003e40 con 0x559b1e48f9d0 2024-09-19T06:49:05.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.433+0000 7fca07dd0740 10 client.15168 fetch_fsmap learned FSMap version 26 2024-09-19T06:49:05.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.433+0000 7fca07dd0740 1 -- 172.21.15.103:0/852813968 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b1e514fe0 con 0x559b1e48f9d0 2024-09-19T06:49:05.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.433+0000 7fc9deffd640 1 -- 172.21.15.103:0/852813968 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9f0040040 con 0x559b1e48f9d0 2024-09-19T06:49:05.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.433+0000 7fc9deffd640 1 -- 172.21.15.103:0/852813968 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fc9e8032030 con 0x559b1e48f9d0 2024-09-19T06:49:05.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.433+0000 7fca07dd0740 10 client.15168 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:49:05.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.433+0000 7fca07dd0740 10 client.15168 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T06:49:05.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.433+0000 7fca07dd0740 20 client.15168 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:05.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.433+0000 7fca07dd0740 1 --2- 172.21.15.103:0/852813968 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x559b1e519c00 0x559b1e518910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:05.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.433+0000 7fca07dd0740 4 client.15168 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:49:05.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.433+0000 7fca07dd0740 1 -- 172.21.15.103:0/852813968 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "status"}) -- 0x559b1e3fdb10 con 0x559b1e519c00 2024-09-19T06:49:05.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.433+0000 7fc9feffd640 1 --2- 172.21.15.103:0/852813968 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x559b1e519c00 0x559b1e518910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:05.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fc9deffd640 10 client.15168 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:49:05.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fc9feffd640 1 --2- 172.21.15.103:0/852813968 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x559b1e519c00 0x559b1e518910 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e965d90795835436 server_cookie=157860824e2f04e6 in_seq=0 out_seq=0 2024-09-19T06:49:05.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fc9deffd640 1 -- 172.21.15.103:0/852813968 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559b1e3fdb10 con 0x559b1e519c00 2024-09-19T06:49:05.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fc9deffd640 10 client.15168 handle_command_reply: tid=0 2024-09-19T06:49:05.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fca07dd0740 1 client.15168 shutdown 2024-09-19T06:49:05.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fca07dd0740 1 -- 172.21.15.103:0/852813968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e48f9d0 msgr2=0x559b1e4b5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:05.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fca07dd0740 1 --2- 172.21.15.103:0/852813968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e48f9d0 0x559b1e4b5790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800fea0 tx=0x7fc9e800a2d0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fca07dd0740 1 -- 172.21.15.103:0/852813968 shutdown_connections 2024-09-19T06:49:05.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fca07dd0740 1 --2- 172.21.15.103:0/852813968 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x559b1e519c00 0x559b1e518910 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fca07dd0740 1 --2- 172.21.15.103:0/852813968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b1e4a6b90 0x559b1e4b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fca07dd0740 1 --2- 172.21.15.103:0/852813968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b1e4902f0 0x559b1e4afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fca07dd0740 1 --2- 172.21.15.103:0/852813968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b1e48f9d0 0x559b1e4b5790 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fc9deffd640 0 client.15168 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:49:05.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fca07dd0740 1 -- 172.21.15.103:0/852813968 >> 172.21.15.103:0/852813968 conn(0x559b1e48bf60 msgr2=0x559b1e496d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:05.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fca07dd0740 1 -- 172.21.15.103:0/852813968 shutdown_connections 2024-09-19T06:49:05.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fca07dd0740 1 -- 172.21.15.103:0/852813968 wait complete. 2024-09-19T06:49:05.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.434+0000 7fca07dd0740 20 client.15168 trim_cache size 0 max 16384 2024-09-19T06:49:05.441 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:49:05.441 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:49:05.441 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 1, 2024-09-19T06:49:05.441 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24455, 2024-09-19T06:49:05.441 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T06:49:05.441 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T06:49:05.442 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T06:49:05.442 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 92.567178929999997, 2024-09-19T06:49:05.442 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T06:49:05.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 82, 2024-09-19T06:49:05.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T06:49:05.442 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 121.643658188 2024-09-19T06:49:05.442 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:49:05.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.435+0000 7fca059f7640 1 -- 172.21.15.103:0/834613331 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc9ec0a32d0 msgr2=0x7fc9ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:05.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.435+0000 7fca059f7640 1 --2- 172.21.15.103:0/834613331 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc9ec0a32d0 0x7fc9ec0a5790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80a4c70 tx=0x7fc9f806f3b0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.435+0000 7fca059f7640 1 -- 172.21.15.103:0/834613331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca001082e0 msgr2=0x7fca00101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:05.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.435+0000 7fca059f7640 1 --2- 172.21.15.103:0/834613331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca001082e0 0x7fca00101490 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800cce0 tx=0x7fc9f800a620 comp rx=0 tx=0).stop 2024-09-19T06:49:05.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.435+0000 7fca059f7640 1 -- 172.21.15.103:0/834613331 shutdown_connections 2024-09-19T06:49:05.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.435+0000 7fca059f7640 1 --2- 172.21.15.103:0/834613331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca001082e0 0x7fca00101490 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.435+0000 7fca059f7640 1 --2- 172.21.15.103:0/834613331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca00103760 0x7fca00100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.435+0000 7fca059f7640 1 --2- 172.21.15.103:0/834613331 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7fc9ec0a32d0 0x7fc9ec0a5790 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.435+0000 7fca059f7640 1 --2- 172.21.15.103:0/834613331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca00102db0 0x7fca00100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:05.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.435+0000 7fca059f7640 1 -- 172.21.15.103:0/834613331 >> 172.21.15.103:0/834613331 conn(0x7fca000fc7d0 msgr2=0x7fca00106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:05.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.435+0000 7fca059f7640 1 -- 172.21.15.103:0/834613331 shutdown_connections 2024-09-19T06:49:05.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:05.435+0000 7fca059f7640 1 -- 172.21.15.103:0/834613331 wait complete. 2024-09-19T06:49:06.181 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T06:49:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:06.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.883+0000 7f678316b640 1 -- 172.21.15.103:0/1224834545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f677c108290 msgr2=0x7f677c10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.883+0000 7f678316b640 1 --2- 172.21.15.103:0/1224834545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f677c108290 0x7f677c10ec30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6764009930 tx=0x7f676402a420 comp rx=0 tx=0).stop 2024-09-19T06:49:06.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.883+0000 7f678316b640 1 -- 172.21.15.103:0/1224834545 shutdown_connections 2024-09-19T06:49:06.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.883+0000 7f678316b640 1 --2- 172.21.15.103:0/1224834545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f677c108290 0x7f677c10ec30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.883+0000 7f678316b640 1 --2- 172.21.15.103:0/1224834545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f677c103740 0x7f677c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.883+0000 7f678316b640 1 --2- 172.21.15.103:0/1224834545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f677c102d90 0x7f677c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.883+0000 7f678316b640 1 -- 172.21.15.103:0/1224834545 >> 172.21.15.103:0/1224834545 conn(0x7f677c0fc7d0 msgr2=0x7f677c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:06.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.883+0000 7f678316b640 1 -- 172.21.15.103:0/1224834545 shutdown_connections 2024-09-19T06:49:06.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.883+0000 7f678316b640 1 -- 172.21.15.103:0/1224834545 wait complete. 2024-09-19T06:49:06.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.883+0000 7f678316b640 1 Processor -- start 2024-09-19T06:49:06.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.883+0000 7f678316b640 1 -- start start 2024-09-19T06:49:06.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f678316b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f677c102d90 0x7f677c1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f678316b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f677c103740 0x7f677c100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f678316b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f677c108290 0x7f677c101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f678316b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f677c112090 con 0x7f677c103740 2024-09-19T06:49:06.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f678316b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f677c111f10 con 0x7f677c102d90 2024-09-19T06:49:06.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f678316b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f677c112210 con 0x7f677c108290 2024-09-19T06:49:06.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f677bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f677c103740 0x7f677c100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f677bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f677c103740 0x7f677c100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36650/0 (socket says 172.21.15.103:36650) 2024-09-19T06:49:06.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f677bfff640 1 -- 172.21.15.103:0/1452240318 learned_addr learned my addr 172.21.15.103:0/1452240318 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:06.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f6781708640 1 --2- 172.21.15.103:0/1452240318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f677c108290 0x7f677c101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f6780f07640 1 --2- 172.21.15.103:0/1452240318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f677c102d90 0x7f677c1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f677bfff640 1 -- 172.21.15.103:0/1452240318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f677c108290 msgr2=0x7f677c101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f677bfff640 1 --2- 172.21.15.103:0/1452240318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f677c108290 0x7f677c101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f677bfff640 1 -- 172.21.15.103:0/1452240318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f677c102d90 msgr2=0x7f677c1009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f677bfff640 1 --2- 172.21.15.103:0/1452240318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f677c102d90 0x7f677c1009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f677bfff640 1 -- 172.21.15.103:0/1452240318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c1a69a0 con 0x7f677c103740 2024-09-19T06:49:06.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f677bfff640 1 --2- 172.21.15.103:0/1452240318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f677c103740 0x7f677c100f30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f676c00ccb0 tx=0x7f676c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f6779ffb640 1 -- 172.21.15.103:0/1452240318 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c00ce60 con 0x7f677c103740 2024-09-19T06:49:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f6779ffb640 1 -- 172.21.15.103:0/1452240318 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f676c00fb30 con 0x7f677c103740 2024-09-19T06:49:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f6779ffb640 1 -- 172.21.15.103:0/1452240318 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c018c30 con 0x7f677c103740 2024-09-19T06:49:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.884+0000 7f678316b640 1 -- 172.21.15.103:0/1452240318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677c1a6c90 con 0x7f677c103740 2024-09-19T06:49:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.885+0000 7f678316b640 1 -- 172.21.15.103:0/1452240318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677c1a7180 con 0x7f677c103740 2024-09-19T06:49:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.887+0000 7f6779ffb640 1 -- 172.21.15.103:0/1452240318 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f676c00f650 con 0x7f677c103740 2024-09-19T06:49:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.887+0000 7f6785544740 1 Processor -- start 2024-09-19T06:49:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.887+0000 7f6785544740 1 -- start start 2024-09-19T06:49:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.887+0000 7f6785544740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9569d0 0x5570da956db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.887+0000 7f6785544740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da9572f0 0x5570da961670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.887+0000 7f6785544740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da961bb0 0x5570da963fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6785544740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5570da8bd090 con 0x5570da9569d0 2024-09-19T06:49:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6779ffb640 1 --2- 172.21.15.103:0/1452240318 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f67680a32d0 0x7f67680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6785544740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5570da864a70 con 0x5570da9572f0 2024-09-19T06:49:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6785544740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5570da8b9050 con 0x5570da961bb0 2024-09-19T06:49:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6781708640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9569d0 0x5570da956db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6780f07640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da9572f0 0x5570da961670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6781708640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9569d0 0x5570da956db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36666/0 (socket says 172.21.15.103:36666) 2024-09-19T06:49:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6781708640 1 -- 172.21.15.103:0/1880088023 learned_addr learned my addr 172.21.15.103:0/1880088023 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6779ffb640 1 -- 172.21.15.103:0/1452240318 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f676c0cff30 con 0x7f677c103740 2024-09-19T06:49:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6781708640 1 --2- 172.21.15.103:0/1452240318 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f67680a32d0 0x7f67680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f677bfff640 1 --2- 172.21.15.103:0/1880088023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da961bb0 0x5570da963fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6780f07640 1 -- 172.21.15.103:0/1880088023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da961bb0 msgr2=0x5570da963fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6780f07640 1 --2- 172.21.15.103:0/1880088023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da961bb0 0x5570da963fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6780f07640 1 -- 172.21.15.103:0/1880088023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9569d0 msgr2=0x5570da956db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6780f07640 1 --2- 172.21.15.103:0/1880088023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9569d0 0x5570da956db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6780f07640 1 -- 172.21.15.103:0/1880088023 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570da871130 con 0x5570da9572f0 2024-09-19T06:49:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6781708640 1 --2- 172.21.15.103:0/1452240318 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f67680a32d0 0x7f67680a5790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6764031550 tx=0x7f6764009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6780f07640 1 --2- 172.21.15.103:0/1880088023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da9572f0 0x5570da961670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f677400b9e0 tx=0x7f677400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95354426e3480cde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.888+0000 7f6772ffd640 1 -- 172.21.15.103:0/1880088023 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677400de20 con 0x5570da9572f0 2024-09-19T06:49:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.889+0000 7f6772ffd640 1 -- 172.21.15.103:0/1880088023 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6774017900 con 0x5570da9572f0 2024-09-19T06:49:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.889+0000 7f6772ffd640 1 -- 172.21.15.103:0/1880088023 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677400fa30 con 0x5570da9572f0 2024-09-19T06:49:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.889+0000 7f6785544740 1 -- 172.21.15.103:0/1880088023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da9572f0 msgr2=0x5570da961670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.889+0000 7f6785544740 1 --2- 172.21.15.103:0/1880088023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da9572f0 0x5570da961670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f677400b9e0 tx=0x7f677400bea0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.889+0000 7f6785544740 1 -- 172.21.15.103:0/1880088023 shutdown_connections 2024-09-19T06:49:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.889+0000 7f6785544740 1 --2- 172.21.15.103:0/1880088023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da961bb0 0x5570da963fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.889+0000 7f6785544740 1 --2- 172.21.15.103:0/1880088023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da9572f0 0x5570da961670 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.889+0000 7f6785544740 1 --2- 172.21.15.103:0/1880088023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9569d0 0x5570da956db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.889+0000 7f6785544740 1 -- 172.21.15.103:0/1880088023 >> 172.21.15.103:0/1880088023 conn(0x5570da952f60 msgr2=0x5570da953340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.889+0000 7f6785544740 1 -- 172.21.15.103:0/1880088023 shutdown_connections 2024-09-19T06:49:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.889+0000 7f6785544740 1 -- 172.21.15.103:0/1880088023 wait complete. 2024-09-19T06:49:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.890+0000 7f6785544740 1 -- start start 2024-09-19T06:49:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.890+0000 7f6785544740 1 Processor -- start 2024-09-19T06:49:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.890+0000 7f6785544740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da9569d0 0x5570da9613b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.890+0000 7f6785544740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9572f0 0x5570da95e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.890+0000 7f6785544740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da961bb0 0x5570da95ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.890+0000 7f6785544740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5570da964780 con 0x5570da9572f0 2024-09-19T06:49:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.890+0000 7f6785544740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5570da877f20 con 0x5570da961bb0 2024-09-19T06:49:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.890+0000 7f6785544740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5570da964900 con 0x5570da9569d0 2024-09-19T06:49:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.890+0000 7f6785544740 10 client.0 _ll_register_callbacks cb 0x5570da841030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.890+0000 7f677bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9572f0 0x5570da95e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.890+0000 7f677bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9572f0 0x5570da95e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36680/0 (socket says 172.21.15.103:36680) 2024-09-19T06:49:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.890+0000 7f677bfff640 1 -- 172.21.15.103:0/1852692986 learned_addr learned my addr 172.21.15.103:0/1852692986 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.890+0000 7f6780f07640 1 --2- 172.21.15.103:0/1852692986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da9569d0 0x5570da9613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.890+0000 7f6781708640 1 --2- 172.21.15.103:0/1852692986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da961bb0 0x5570da95ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.891+0000 7f677bfff640 1 -- 172.21.15.103:0/1852692986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da9569d0 msgr2=0x5570da9613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.891+0000 7f677bfff640 1 --2- 172.21.15.103:0/1852692986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da9569d0 0x5570da9613b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.891+0000 7f677bfff640 1 -- 172.21.15.103:0/1852692986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da961bb0 msgr2=0x5570da95ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.891+0000 7f677bfff640 1 --2- 172.21.15.103:0/1852692986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da961bb0 0x5570da95ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.891+0000 7f677bfff640 1 -- 172.21.15.103:0/1852692986 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570da62e2b0 con 0x5570da9572f0 2024-09-19T06:49:06.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.891+0000 7f677bfff640 1 --2- 172.21.15.103:0/1852692986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9572f0 0x5570da95e700 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f676c098dd0 tx=0x7f676c09bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfc7a35bd860a8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:06.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.891+0000 7f675bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:06.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.891+0000 7f675bfff640 1 -- 172.21.15.103:0/1852692986 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c0af120 con 0x5570da9572f0 2024-09-19T06:49:06.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.891+0000 7f675bfff640 1 -- 172.21.15.103:0/1852692986 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f676c0cdb40 con 0x5570da9572f0 2024-09-19T06:49:06.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.891+0000 7f6785544740 1 -- 172.21.15.103:0/1852692986 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6774017900 con 0x5570da9572f0 2024-09-19T06:49:06.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.891+0000 7f675bfff640 1 -- 172.21.15.103:0/1852692986 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f676c0aef40 con 0x5570da9572f0 2024-09-19T06:49:06.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.891+0000 7f6785544740 10 client.15180 fetch_fsmap learned FSMap version 26 2024-09-19T06:49:06.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.891+0000 7f6785544740 1 -- 172.21.15.103:0/1852692986 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5570da970070 con 0x5570da9572f0 2024-09-19T06:49:06.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.891+0000 7f675bfff640 1 -- 172.21.15.103:0/1852692986 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6774017900 con 0x5570da9572f0 2024-09-19T06:49:06.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.891+0000 7f675bfff640 1 -- 172.21.15.103:0/1852692986 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f676c0a4be0 con 0x5570da9572f0 2024-09-19T06:49:06.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.892+0000 7f6785544740 10 client.15180 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:49:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.892+0000 7f6785544740 10 client.15180 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T06:49:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.892+0000 7f6785544740 20 client.15180 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.892+0000 7f6785544740 1 --2- 172.21.15.103:0/1852692986 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5570da972670 0x5570da973ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.892+0000 7f6785544740 4 client.15180 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.892+0000 7f6785544740 1 -- 172.21.15.103:0/1852692986 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570da8ca600 con 0x5570da972670 2024-09-19T06:49:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.892+0000 7f6780f07640 1 --2- 172.21.15.103:0/1852692986 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5570da972670 0x5570da973ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.892+0000 7f6780f07640 1 --2- 172.21.15.103:0/1852692986 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5570da972670 0x5570da973ae0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=108e6769a81533a1 server_cookie=f47db914021088b4 in_seq=0 out_seq=0 2024-09-19T06:49:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.892+0000 7f675bfff640 10 client.15180 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.893+0000 7f675bfff640 1 -- 172.21.15.103:0/1852692986 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570da8ca600 con 0x5570da972670 2024-09-19T06:49:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.893+0000 7f675bfff640 10 client.15180 handle_command_reply: tid=0 2024-09-19T06:49:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.893+0000 7f6785544740 1 client.15180 shutdown 2024-09-19T06:49:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.893+0000 7f6785544740 1 -- 172.21.15.103:0/1852692986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9572f0 msgr2=0x5570da95e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.893+0000 7f6785544740 1 --2- 172.21.15.103:0/1852692986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9572f0 0x5570da95e700 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f676c098dd0 tx=0x7f676c09bf40 comp rx=0 tx=0).stop 2024-09-19T06:49:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.893+0000 7f6785544740 1 -- 172.21.15.103:0/1852692986 shutdown_connections 2024-09-19T06:49:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.893+0000 7f6785544740 1 --2- 172.21.15.103:0/1852692986 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5570da972670 0x5570da973ae0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.893+0000 7f6785544740 1 --2- 172.21.15.103:0/1852692986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da961bb0 0x5570da95ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.893+0000 7f6785544740 1 --2- 172.21.15.103:0/1852692986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9572f0 0x5570da95e700 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.893+0000 7f6785544740 1 --2- 172.21.15.103:0/1852692986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da9569d0 0x5570da9613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.893+0000 7f675bfff640 0 client.15180 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.893+0000 7f6785544740 1 -- 172.21.15.103:0/1852692986 >> 172.21.15.103:0/1852692986 conn(0x5570da952f60 msgr2=0x5570da958c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.893+0000 7f6785544740 1 -- 172.21.15.103:0/1852692986 shutdown_connections 2024-09-19T06:49:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.893+0000 7f6785544740 1 -- 172.21.15.103:0/1852692986 wait complete. 2024-09-19T06:49:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.893+0000 7f6785544740 20 client.15180 trim_cache size 0 max 16384 2024-09-19T06:49:06.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.911+0000 7f6785544740 1 Processor -- start 2024-09-19T06:49:06.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.911+0000 7f6785544740 1 -- start start 2024-09-19T06:49:06.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.911+0000 7f6785544740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da9569d0 0x5570da973ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.911+0000 7f6785544740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da9572f0 0x5570da95dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.911+0000 7f6785544740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da961bb0 0x5570da95cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.911+0000 7f6785544740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5570da971e20 con 0x5570da961bb0 2024-09-19T06:49:06.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.911+0000 7f6785544740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5570da9724f0 con 0x5570da9572f0 2024-09-19T06:49:06.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.911+0000 7f6785544740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5570da972bc0 con 0x5570da9569d0 2024-09-19T06:49:06.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.911+0000 7f6781708640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da961bb0 0x5570da95cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.911+0000 7f6781708640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da961bb0 0x5570da95cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36684/0 (socket says 172.21.15.103:36684) 2024-09-19T06:49:06.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.911+0000 7f6781708640 1 -- 172.21.15.103:0/4235158895 learned_addr learned my addr 172.21.15.103:0/4235158895 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:06.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.911+0000 7f6780f07640 1 --2- 172.21.15.103:0/4235158895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da9569d0 0x5570da973ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.912+0000 7f677bfff640 1 --2- 172.21.15.103:0/4235158895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da9572f0 0x5570da95dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.912+0000 7f6781708640 1 -- 172.21.15.103:0/4235158895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da9569d0 msgr2=0x5570da973ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:06.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.912+0000 7f6781708640 1 --2- 172.21.15.103:0/4235158895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da9569d0 0x5570da973ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.912+0000 7f6781708640 1 -- 172.21.15.103:0/4235158895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da9572f0 msgr2=0x5570da95dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:06.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.912+0000 7f6781708640 1 --2- 172.21.15.103:0/4235158895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da9572f0 0x5570da95dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.912+0000 7f6781708640 1 -- 172.21.15.103:0/4235158895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570da952a00 con 0x5570da961bb0 2024-09-19T06:49:06.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.912+0000 7f6781708640 1 --2- 172.21.15.103:0/4235158895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da961bb0 0x5570da95cfd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6764009930 tx=0x7f6764040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7511a149dca5095 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:06.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.912+0000 7f67727fc640 1 -- 172.21.15.103:0/4235158895 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764035040 con 0x5570da961bb0 2024-09-19T06:49:06.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.912+0000 7f67727fc640 1 -- 172.21.15.103:0/4235158895 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67640047a0 con 0x5570da961bb0 2024-09-19T06:49:06.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.912+0000 7f67727fc640 1 -- 172.21.15.103:0/4235158895 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764003850 con 0x5570da961bb0 2024-09-19T06:49:06.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.913+0000 7f6785544740 1 -- 172.21.15.103:0/4235158895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da961bb0 msgr2=0x5570da95cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.913+0000 7f6785544740 1 --2- 172.21.15.103:0/4235158895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da961bb0 0x5570da95cfd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6764009930 tx=0x7f6764040000 comp rx=0 tx=0).stop 2024-09-19T06:49:06.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.913+0000 7f6785544740 1 -- 172.21.15.103:0/4235158895 shutdown_connections 2024-09-19T06:49:06.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.913+0000 7f6785544740 1 --2- 172.21.15.103:0/4235158895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da961bb0 0x5570da95cfd0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.913+0000 7f6785544740 1 --2- 172.21.15.103:0/4235158895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da9572f0 0x5570da95dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.913+0000 7f6785544740 1 --2- 172.21.15.103:0/4235158895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da9569d0 0x5570da973ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.913+0000 7f6785544740 1 -- 172.21.15.103:0/4235158895 >> 172.21.15.103:0/4235158895 conn(0x5570da952f60 msgr2=0x5570da95e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:06.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.913+0000 7f6785544740 1 -- 172.21.15.103:0/4235158895 shutdown_connections 2024-09-19T06:49:06.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.913+0000 7f6785544740 1 -- 172.21.15.103:0/4235158895 wait complete. 2024-09-19T06:49:06.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.913+0000 7f6785544740 1 -- start start 2024-09-19T06:49:06.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.913+0000 7f6785544740 1 Processor -- start 2024-09-19T06:49:06.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6785544740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9569d0 0x5570da97bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6785544740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da9572f0 0x5570da97c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6785544740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da961bb0 0x5570da9772a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6785544740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5570da973960 con 0x5570da9569d0 2024-09-19T06:49:06.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6785544740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5570da973290 con 0x5570da961bb0 2024-09-19T06:49:06.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6785544740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5570da974030 con 0x5570da9572f0 2024-09-19T06:49:06.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6785544740 10 client.0 _ll_register_callbacks cb 0x7f676c098790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:06.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6780f07640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9569d0 0x5570da97bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6780f07640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9569d0 0x5570da97bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36686/0 (socket says 172.21.15.103:36686) 2024-09-19T06:49:06.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6780f07640 1 -- 172.21.15.103:0/197519870 learned_addr learned my addr 172.21.15.103:0/197519870 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:06.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f677bfff640 1 --2- 172.21.15.103:0/197519870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da9572f0 0x5570da97c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6780f07640 1 -- 172.21.15.103:0/197519870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da9572f0 msgr2=0x5570da97c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:06.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6781708640 1 --2- 172.21.15.103:0/197519870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da961bb0 0x5570da9772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6780f07640 1 --2- 172.21.15.103:0/197519870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da9572f0 0x5570da97c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6780f07640 1 -- 172.21.15.103:0/197519870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da961bb0 msgr2=0x5570da9772a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:06.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6780f07640 1 --2- 172.21.15.103:0/197519870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da961bb0 0x5570da9772a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6780f07640 1 -- 172.21.15.103:0/197519870 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570da977aa0 con 0x5570da9569d0 2024-09-19T06:49:06.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6781708640 1 --2- 172.21.15.103:0/197519870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da961bb0 0x5570da9772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:06.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6780f07640 1 --2- 172.21.15.103:0/197519870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9569d0 0x5570da97bdb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f677400c010 tx=0x7f677400f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6d186101068313 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:06.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f675bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:06.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f675bfff640 1 -- 172.21.15.103:0/197519870 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774005040 con 0x5570da9569d0 2024-09-19T06:49:06.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f675bfff640 1 -- 172.21.15.103:0/197519870 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f677401d040 con 0x5570da9569d0 2024-09-19T06:49:06.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.914+0000 7f6785544740 1 -- 172.21.15.103:0/197519870 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67640047a0 con 0x5570da9569d0 2024-09-19T06:49:06.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.915+0000 7f675bfff640 1 -- 172.21.15.103:0/197519870 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f67740051e0 con 0x5570da9569d0 2024-09-19T06:49:06.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.915+0000 7f6785544740 10 client.15186 fetch_fsmap learned FSMap version 26 2024-09-19T06:49:06.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.915+0000 7f6785544740 1 -- 172.21.15.103:0/197519870 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5570da9dc7c0 con 0x5570da9569d0 2024-09-19T06:49:06.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.915+0000 7f675bfff640 1 -- 172.21.15.103:0/197519870 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67640047a0 con 0x5570da9569d0 2024-09-19T06:49:06.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.915+0000 7f675bfff640 1 -- 172.21.15.103:0/197519870 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f677400f5a0 con 0x5570da9569d0 2024-09-19T06:49:06.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.915+0000 7f6785544740 10 client.15186 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:49:06.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.915+0000 7f6785544740 10 client.15186 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T06:49:06.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.915+0000 7f6785544740 20 client.15186 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:06.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.915+0000 7f6785544740 1 --2- 172.21.15.103:0/197519870 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5570da9e1180 0x5570da9dfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.915+0000 7f6785544740 4 client.15186 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:49:06.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.915+0000 7f6785544740 1 -- 172.21.15.103:0/197519870 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "status"}) -- 0x5570da8c4b10 con 0x5570da9e1180 2024-09-19T06:49:06.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.915+0000 7f677bfff640 1 --2- 172.21.15.103:0/197519870 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5570da9e1180 0x5570da9dfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f677bfff640 1 --2- 172.21.15.103:0/197519870 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5570da9e1180 0x5570da9dfe90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adb326346258a80a server_cookie=90e6a9536b550cd0 in_seq=0 out_seq=0 2024-09-19T06:49:06.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f675bfff640 10 client.15186 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:06.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f675bfff640 1 -- 172.21.15.103:0/197519870 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5570da8c4b10 con 0x5570da9e1180 2024-09-19T06:49:06.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f675bfff640 10 client.15186 handle_command_reply: tid=0 2024-09-19T06:49:06.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f6785544740 1 client.15186 shutdown 2024-09-19T06:49:06.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f6785544740 1 -- 172.21.15.103:0/197519870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9569d0 msgr2=0x5570da97bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f6785544740 1 --2- 172.21.15.103:0/197519870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9569d0 0x5570da97bdb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f677400c010 tx=0x7f677400f4b0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f6785544740 1 -- 172.21.15.103:0/197519870 shutdown_connections 2024-09-19T06:49:06.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f6785544740 1 --2- 172.21.15.103:0/197519870 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5570da9e1180 0x5570da9dfe90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f6785544740 1 --2- 172.21.15.103:0/197519870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570da961bb0 0x5570da9772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f6785544740 1 --2- 172.21.15.103:0/197519870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570da9572f0 0x5570da97c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f6785544740 1 --2- 172.21.15.103:0/197519870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570da9569d0 0x5570da97bdb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f6785544740 1 -- 172.21.15.103:0/197519870 >> 172.21.15.103:0/197519870 conn(0x5570da952f60 msgr2=0x5570da95cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:06.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f675bfff640 0 client.15186 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:06.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f6785544740 1 -- 172.21.15.103:0/197519870 shutdown_connections 2024-09-19T06:49:06.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f6785544740 1 -- 172.21.15.103:0/197519870 wait complete. 2024-09-19T06:49:06.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.916+0000 7f6785544740 20 client.15186 trim_cache size 0 max 16384 2024-09-19T06:49:06.924 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:49:06.924 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:49:06.924 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-09-19T06:49:06.924 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24482, 2024-09-19T06:49:06.924 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T06:49:06.924 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T06:49:06.924 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T06:49:06.924 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 101.078933221, 2024-09-19T06:49:06.924 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T06:49:06.924 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 82, 2024-09-19T06:49:06.924 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T06:49:06.924 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 112.17095715400001 2024-09-19T06:49:06.925 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:49:06.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.917+0000 7f678316b640 1 -- 172.21.15.103:0/1452240318 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f67680a32d0 msgr2=0x7f67680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.917+0000 7f678316b640 1 --2- 172.21.15.103:0/1452240318 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f67680a32d0 0x7f67680a5790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6764031550 tx=0x7f6764009260 comp rx=0 tx=0).stop 2024-09-19T06:49:06.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.917+0000 7f678316b640 1 -- 172.21.15.103:0/1452240318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f677c103740 msgr2=0x7f677c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.917+0000 7f678316b640 1 --2- 172.21.15.103:0/1452240318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f677c103740 0x7f677c100f30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f676c00ccb0 tx=0x7f676c00a620 comp rx=0 tx=0).stop 2024-09-19T06:49:06.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.917+0000 7f678316b640 1 -- 172.21.15.103:0/1452240318 shutdown_connections 2024-09-19T06:49:06.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.917+0000 7f678316b640 1 --2- 172.21.15.103:0/1452240318 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f67680a32d0 0x7f67680a5790 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.917+0000 7f678316b640 1 --2- 172.21.15.103:0/1452240318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f677c108290 0x7f677c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.917+0000 7f678316b640 1 --2- 172.21.15.103:0/1452240318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f677c103740 0x7f677c100f30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.917+0000 7f678316b640 1 --2- 172.21.15.103:0/1452240318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f677c102d90 0x7f677c1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.917+0000 7f678316b640 1 -- 172.21.15.103:0/1452240318 >> 172.21.15.103:0/1452240318 conn(0x7f677c0fc7d0 msgr2=0x7f677c106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:06.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.917+0000 7f678316b640 1 -- 172.21.15.103:0/1452240318 shutdown_connections 2024-09-19T06:49:06.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:06.917+0000 7f678316b640 1 -- 172.21.15.103:0/1452240318 wait complete. 2024-09-19T06:49:07.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:06 smithi103 ceph-mon[31158]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T06:49:07.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:06 smithi134 ceph-mon[35356]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T06:49:07.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:06 smithi174 ceph-mon[35637]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T06:49:07.616 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T06:49:07.626 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T06:49:07.626 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T06:49:07.627 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T06:49:07.627 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-19T06:49:07.628 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T06:49:07.628 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T06:49:07.628 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-19T06:49:07.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.728+0000 7f2dd38d0640 1 -- 172.21.15.103:0/672446083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2dcc100a40 msgr2=0x7f2dcc100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:07.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.728+0000 7f2dd38d0640 1 --2- 172.21.15.103:0/672446083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2dcc100a40 0x7f2dcc100e40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009900 tx=0x7f2dc802a410 comp rx=0 tx=0).stop 2024-09-19T06:49:07.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.728+0000 7f2dd38d0640 1 -- 172.21.15.103:0/672446083 shutdown_connections 2024-09-19T06:49:07.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.728+0000 7f2dd38d0640 1 --2- 172.21.15.103:0/672446083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dcc1025f0 0x7f2dcc108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:07.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.728+0000 7f2dd38d0640 1 --2- 172.21.15.103:0/672446083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dcc101c30 0x7f2dcc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:07.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.728+0000 7f2dd38d0640 1 --2- 172.21.15.103:0/672446083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2dcc100a40 0x7f2dcc100e40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:07.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.728+0000 7f2dd38d0640 1 -- 172.21.15.103:0/672446083 >> 172.21.15.103:0/672446083 conn(0x7f2dcc0fc1e0 msgr2=0x7f2dcc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.728+0000 7f2dd38d0640 1 -- 172.21.15.103:0/672446083 shutdown_connections 2024-09-19T06:49:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.728+0000 7f2dd38d0640 1 -- 172.21.15.103:0/672446083 wait complete. 2024-09-19T06:49:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.728+0000 7f2dd38d0640 1 Processor -- start 2024-09-19T06:49:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.728+0000 7f2dd38d0640 1 -- start start 2024-09-19T06:49:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd38d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dcc100a40 0x7f2dcc197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd38d0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dcc101c30 0x7f2dcc1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd38d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2dcc1025f0 0x7f2dcc19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd38d0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc10c6f0 con 0x7f2dcc1025f0 2024-09-19T06:49:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd38d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc10c570 con 0x7f2dcc100a40 2024-09-19T06:49:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd38d0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc10c870 con 0x7f2dcc101c30 2024-09-19T06:49:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd1e6d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2dcc1025f0 0x7f2dcc19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd1e6d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2dcc1025f0 0x7f2dcc19f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59184/0 (socket says 172.21.15.103:59184) 2024-09-19T06:49:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd1e6d640 1 -- 172.21.15.103:0/3635244037 learned_addr learned my addr 172.21.15.103:0/3635244037 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd166c640 1 --2- 172.21.15.103:0/3635244037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dcc100a40 0x7f2dcc197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd0e6b640 1 --2- 172.21.15.103:0/3635244037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dcc101c30 0x7f2dcc1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd1e6d640 1 -- 172.21.15.103:0/3635244037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dcc101c30 msgr2=0x7f2dcc1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd1e6d640 1 --2- 172.21.15.103:0/3635244037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dcc101c30 0x7f2dcc1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd1e6d640 1 -- 172.21.15.103:0/3635244037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dcc100a40 msgr2=0x7f2dcc197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd1e6d640 1 --2- 172.21.15.103:0/3635244037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dcc100a40 0x7f2dcc197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd1e6d640 1 -- 172.21.15.103:0/3635244037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dcc19f980 con 0x7f2dcc1025f0 2024-09-19T06:49:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd0e6b640 1 --2- 172.21.15.103:0/3635244037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dcc101c30 0x7f2dcc1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd1e6d640 1 --2- 172.21.15.103:0/3635244037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2dcc1025f0 0x7f2dcc19f310 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400cad0 tx=0x7f2dc400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dc27fc640 1 -- 172.21.15.103:0/3635244037 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc400ae20 con 0x7f2dcc1025f0 2024-09-19T06:49:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dc27fc640 1 -- 172.21.15.103:0/3635244037 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dc4005020 con 0x7f2dcc1025f0 2024-09-19T06:49:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd38d0640 1 -- 172.21.15.103:0/3635244037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dcc19fc70 con 0x7f2dcc1025f0 2024-09-19T06:49:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dc27fc640 1 -- 172.21.15.103:0/3635244037 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc400f6f0 con 0x7f2dcc1025f0 2024-09-19T06:49:07.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.729+0000 7f2dd38d0640 1 -- 172.21.15.103:0/3635244037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dcc1a0170 con 0x7f2dcc1025f0 2024-09-19T06:49:07.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.730+0000 7f2dd38d0640 1 -- 172.21.15.103:0/3635244037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dcc1a03a0 con 0x7f2dcc1025f0 2024-09-19T06:49:07.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.731+0000 7f2dc27fc640 1 -- 172.21.15.103:0/3635244037 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2dc40051d0 con 0x7f2dcc1025f0 2024-09-19T06:49:07.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.732+0000 7f2dc27fc640 1 --2- 172.21.15.103:0/3635244037 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2db80a3390 0x7f2db80a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:07.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.732+0000 7f2dd166c640 1 --2- 172.21.15.103:0/3635244037 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2db80a3390 0x7f2db80a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:07.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.732+0000 7f2dc27fc640 1 -- 172.21.15.103:0/3635244037 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2dc40cfb50 con 0x7f2dcc1025f0 2024-09-19T06:49:07.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.733+0000 7f2dd166c640 1 --2- 172.21.15.103:0/3635244037 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2db80a3390 0x7f2db80a5850 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2dc802aa00 tx=0x7f2dc80315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:07.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.737+0000 7f2dc27fc640 1 -- 172.21.15.103:0/3635244037 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2dc4014030 con 0x7f2dcc1025f0 2024-09-19T06:49:07.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:07.953+0000 7f2dd38d0640 1 -- 172.21.15.103:0/3635244037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f2dcc1a0570 con 0x7f2dcc1025f0 2024-09-19T06:49:08.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:08.904+0000 7f2dc27fc640 1 -- 172.21.15.103:0/3635244037 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) ==== 92+0+0 (secure 0 0 0) 0x7f2dc409c720 con 0x7f2dcc1025f0 2024-09-19T06:49:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:08.907+0000 7f2dd38d0640 1 -- 172.21.15.103:0/3635244037 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2db80a3390 msgr2=0x7f2db80a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:08.907+0000 7f2dd38d0640 1 --2- 172.21.15.103:0/3635244037 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2db80a3390 0x7f2db80a5850 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2dc802aa00 tx=0x7f2dc80315b0 comp rx=0 tx=0).stop 2024-09-19T06:49:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:08.907+0000 7f2dd38d0640 1 -- 172.21.15.103:0/3635244037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2dcc1025f0 msgr2=0x7f2dcc19f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:08.907+0000 7f2dd38d0640 1 --2- 172.21.15.103:0/3635244037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2dcc1025f0 0x7f2dcc19f310 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400cad0 tx=0x7f2dc400cf90 comp rx=0 tx=0).stop 2024-09-19T06:49:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:08.907+0000 7f2dd38d0640 1 -- 172.21.15.103:0/3635244037 shutdown_connections 2024-09-19T06:49:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:08.907+0000 7f2dd38d0640 1 --2- 172.21.15.103:0/3635244037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2dcc1025f0 0x7f2dcc19f310 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:08.907+0000 7f2dd38d0640 1 --2- 172.21.15.103:0/3635244037 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f2db80a3390 0x7f2db80a5850 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:08.907+0000 7f2dd38d0640 1 --2- 172.21.15.103:0/3635244037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dcc101c30 0x7f2dcc1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:08.907+0000 7f2dd38d0640 1 --2- 172.21.15.103:0/3635244037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dcc100a40 0x7f2dcc197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:08.907+0000 7f2dd38d0640 1 -- 172.21.15.103:0/3635244037 >> 172.21.15.103:0/3635244037 conn(0x7f2dcc0fc1e0 msgr2=0x7f2dcc0fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:08.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:08.907+0000 7f2dd38d0640 1 -- 172.21.15.103:0/3635244037 shutdown_connections 2024-09-19T06:49:08.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:08.907+0000 7f2dd38d0640 1 -- 172.21.15.103:0/3635244037 wait complete. 2024-09-19T06:49:08.929 INFO:teuthology.run_tasks:Running task kclient... 2024-09-19T06:49:08.955 INFO:tasks.kclient:Mounting kernel clients... 2024-09-19T06:49:08.955 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-19T06:49:08.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:49:08.955 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-19T06:49:08.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:49:08.973 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2024-09-19T06:49:09.041 INFO:teuthology.orchestra.run.smithi103.stderr:Cannot find device "ceph-brx" 2024-09-19T06:49:09.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:49:09.043 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-19T06:49:09.043 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T06:49:09.043 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T06:49:09.044 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T06:49:09.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:08 smithi134 ceph-mon[35356]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T06:49:09.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:08 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3635244037' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T06:49:09.188 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:08 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: ignoring --setuser ceph since I am not root 2024-09-19T06:49:09.188 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:08 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: ignoring --setgroup ceph since I am not root 2024-09-19T06:49:09.188 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:08 smithi134 ceph-mgr[36723]: -- 172.21.15.134:0/1521336459 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559c538f10e0 con 0x559c538cf000 2024-09-19T06:49:09.188 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:09 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:09.059+0000 7f80e6e2a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:49:09.188 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:09 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:09.115+0000 7f80e6e2a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:49:09.267 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:08 smithi174 ceph-mon[35637]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T06:49:09.267 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:08 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3635244037' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T06:49:09.267 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:08 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: ignoring --setuser ceph since I am not root 2024-09-19T06:49:09.267 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:08 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: ignoring --setgroup ceph since I am not root 2024-09-19T06:49:09.267 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:08 smithi174 ceph-mgr[37442]: -- 172.21.15.174:0/1714563626 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c772c6f0e0 con 0x55c772c4c800 2024-09-19T06:49:09.267 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:09 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:09.062+0000 7f0ce2e34040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:49:09.267 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:09 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:09.119+0000 7f0ce2e34040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:49:09.268 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:09 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:09.265+0000 7f0ce2e34040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:49:09.280 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:08 smithi103 ceph-mon[31158]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T06:49:09.281 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:08 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3635244037' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T06:49:09.281 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:08 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: ignoring --setuser ceph since I am not root 2024-09-19T06:49:09.281 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:08 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: ignoring --setgroup ceph since I am not root 2024-09-19T06:49:09.281 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:09 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:09.065+0000 7fb39120c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:49:09.281 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:09 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:09.126+0000 7fb39120c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:49:09.281 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:09 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:09.279+0000 7fb39120c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:49:09.336 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:09.473 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:09 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:09.261+0000 7f80e6e2a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:49:09.473 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:09 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:09.327+0000 7f80e6e2a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:49:09.583 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:09 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:09.331+0000 7f0ce2e34040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:49:09.583 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:09 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:09.477+0000 7f0ce2e34040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:49:09.609 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:09 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:09.362+0000 7fb39120c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:49:09.609 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:09 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:09.516+0000 7fb39120c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:49:09.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.798+0000 7f8ae36f3640 1 -- 172.21.15.103:0/638977005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8adc100b20 msgr2=0x7f8adc100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:09.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.798+0000 7f8ae36f3640 1 --2- 172.21.15.103:0/638977005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8adc100b20 0x7f8adc100f00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009900 tx=0x7f8ad802a410 comp rx=0 tx=0).stop 2024-09-19T06:49:09.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.798+0000 7f8ae36f3640 1 -- 172.21.15.103:0/638977005 shutdown_connections 2024-09-19T06:49:09.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.798+0000 7f8ae36f3640 1 --2- 172.21.15.103:0/638977005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8adc10e500 0x7f8adc110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:09.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.798+0000 7f8ae36f3640 1 --2- 172.21.15.103:0/638977005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8adc101440 0x7f8adc10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:09.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.798+0000 7f8ae36f3640 1 --2- 172.21.15.103:0/638977005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8adc100b20 0x7f8adc100f00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:09.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.798+0000 7f8ae36f3640 1 -- 172.21.15.103:0/638977005 >> 172.21.15.103:0/638977005 conn(0x7f8adc0fc7d0 msgr2=0x7f8adc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:09.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.798+0000 7f8ae36f3640 1 -- 172.21.15.103:0/638977005 shutdown_connections 2024-09-19T06:49:09.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.798+0000 7f8ae36f3640 1 -- 172.21.15.103:0/638977005 wait complete. 2024-09-19T06:49:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.798+0000 7f8ae36f3640 1 Processor -- start 2024-09-19T06:49:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.798+0000 7f8ae36f3640 1 -- start start 2024-09-19T06:49:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae36f3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8adc100b20 0x7f8adc1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae36f3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8adc101440 0x7f8adc1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae36f3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8adc10e500 0x7f8adc19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae36f3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc114280 con 0x7f8adc10e500 2024-09-19T06:49:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae36f3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc114100 con 0x7f8adc101440 2024-09-19T06:49:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae36f3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc114400 con 0x7f8adc100b20 2024-09-19T06:49:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae1c90640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8adc10e500 0x7f8adc19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae148f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8adc100b20 0x7f8adc1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae1c90640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8adc10e500 0x7f8adc19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59220/0 (socket says 172.21.15.103:59220) 2024-09-19T06:49:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae1c90640 1 -- 172.21.15.103:0/439397549 learned_addr learned my addr 172.21.15.103:0/439397549 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:09.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae0c8e640 1 --2- 172.21.15.103:0/439397549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8adc101440 0x7f8adc1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:09.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae1c90640 1 -- 172.21.15.103:0/439397549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8adc100b20 msgr2=0x7f8adc1a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:09.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae1c90640 1 --2- 172.21.15.103:0/439397549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8adc100b20 0x7f8adc1a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:09.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae1c90640 1 -- 172.21.15.103:0/439397549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8adc101440 msgr2=0x7f8adc1a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:09.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae1c90640 1 --2- 172.21.15.103:0/439397549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8adc101440 0x7f8adc1a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:09.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae1c90640 1 -- 172.21.15.103:0/439397549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8adc19ace0 con 0x7f8adc10e500 2024-09-19T06:49:09.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae1c90640 1 --2- 172.21.15.103:0/439397549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8adc10e500 0x7f8adc19a5b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400e9d0 tx=0x7f8ad400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:09.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ad27fc640 1 -- 172.21.15.103:0/439397549 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad400cd80 con 0x7f8adc10e500 2024-09-19T06:49:09.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ad27fc640 1 -- 172.21.15.103:0/439397549 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ad4005020 con 0x7f8adc10e500 2024-09-19T06:49:09.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.799+0000 7f8ae36f3640 1 -- 172.21.15.103:0/439397549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8adc19afd0 con 0x7f8adc10e500 2024-09-19T06:49:09.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.800+0000 7f8ad27fc640 1 -- 172.21.15.103:0/439397549 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad40116d0 con 0x7f8adc10e500 2024-09-19T06:49:09.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.800+0000 7f8ae36f3640 1 -- 172.21.15.103:0/439397549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8adc1a72c0 con 0x7f8adc10e500 2024-09-19T06:49:09.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.801+0000 7f8ae36f3640 1 -- 172.21.15.103:0/439397549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8adc06a850 con 0x7f8adc10e500 2024-09-19T06:49:09.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.802+0000 7f8ad27fc640 1 -- 172.21.15.103:0/439397549 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 138424+0+0 (secure 0 0 0) 0x7f8ad40051d0 con 0x7f8adc10e500 2024-09-19T06:49:09.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.802+0000 7f8ad27fc640 1 --2- 172.21.15.103:0/439397549 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8ac80a3320 0x7f8ac80a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:09.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.802+0000 7f8ae148f640 1 -- 172.21.15.103:0/439397549 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8ac80a3320 msgr2=0x7f8ac80a57e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2590139910 2024-09-19T06:49:09.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.802+0000 7f8ad27fc640 1 -- 172.21.15.103:0/439397549 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8ad40cff40 con 0x7f8adc10e500 2024-09-19T06:49:09.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.802+0000 7f8ae148f640 1 --2- 172.21.15.103:0/439397549 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8ac80a3320 0x7f8ac80a57e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:49:09.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:09.806+0000 7f8ad27fc640 1 -- 172.21.15.103:0/439397549 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8ad4016030 con 0x7f8adc10e500 2024-09-19T06:49:09.887 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:09 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:09.662+0000 7fb39120c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:49:09.920 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:09 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:09.471+0000 7f80e6e2a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:49:09.921 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:09 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:09.612+0000 7f80e6e2a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:49:09.921 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:09 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:09.619+0000 7f0ce2e34040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:49:10.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10.003+0000 7f8ae148f640 1 -- 172.21.15.103:0/439397549 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8ac80a3320 msgr2=0x7f8ac80a57e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2590139910 2024-09-19T06:49:10.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10.003+0000 7f8ae148f640 1 --2- 172.21.15.103:0/439397549 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8ac80a3320 0x7f8ac80a57e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:49:10.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10.016+0000 7f8ae36f3640 1 -- 172.21.15.103:0/439397549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8adc109600 con 0x7f8adc10e500 2024-09-19T06:49:10.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10.016+0000 7f8ad27fc640 1 -- 172.21.15.103:0/439397549 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f8ad409c4c0 con 0x7f8adc10e500 2024-09-19T06:49:10.017 INFO:teuthology.orchestra.run.smithi103.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T06:49:10.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10.018+0000 7f8ae36f3640 1 -- 172.21.15.103:0/439397549 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8ac80a3320 msgr2=0x7f8ac80a57e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:49:10.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10.018+0000 7f8ae36f3640 1 --2- 172.21.15.103:0/439397549 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8ac80a3320 0x7f8ac80a57e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:10.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10.018+0000 7f8ae36f3640 1 -- 172.21.15.103:0/439397549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8adc10e500 msgr2=0x7f8adc19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:10.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10.018+0000 7f8ae36f3640 1 --2- 172.21.15.103:0/439397549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8adc10e500 0x7f8adc19a5b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400e9d0 tx=0x7f8ad400ee90 comp rx=0 tx=0).stop 2024-09-19T06:49:10.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10.018+0000 7f8ae36f3640 1 -- 172.21.15.103:0/439397549 shutdown_connections 2024-09-19T06:49:10.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10.018+0000 7f8ae36f3640 1 --2- 172.21.15.103:0/439397549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8adc10e500 0x7f8adc19a5b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:10.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10.018+0000 7f8ae36f3640 1 --2- 172.21.15.103:0/439397549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8adc101440 0x7f8adc1a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:10.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10.018+0000 7f8ae36f3640 1 --2- 172.21.15.103:0/439397549 >> [v2:172.21.15.103:6800/2590139910,v1:172.21.15.103:6801/2590139910] conn(0x7f8ac80a3320 0x7f8ac80a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:10.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10.019+0000 7f8ae36f3640 1 --2- 172.21.15.103:0/439397549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8adc100b20 0x7f8adc1a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:10.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10.019+0000 7f8ae36f3640 1 -- 172.21.15.103:0/439397549 >> 172.21.15.103:0/439397549 conn(0x7f8adc0fc7d0 msgr2=0x7f8adc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:10.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10.019+0000 7f8ae36f3640 1 -- 172.21.15.103:0/439397549 shutdown_connections 2024-09-19T06:49:10.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10.019+0000 7f8ae36f3640 1 -- 172.21.15.103:0/439397549 wait complete. 2024-09-19T06:49:10.177 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:09 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3635244037' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T06:49:10.177 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:09 smithi174 ceph-mon[35637]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T06:49:10.177 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:09 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:09.930+0000 7f0ce2e34040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:49:10.178 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:09 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:09.991+0000 7f0ce2e34040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:49:10.178 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:10 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:10.175+0000 7f0ce2e34040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:49:10.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:09 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3635244037' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T06:49:10.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:09 smithi134 ceph-mon[35356]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T06:49:10.187 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:09 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:09.922+0000 7f80e6e2a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:49:10.187 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:09 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:09.983+0000 7f80e6e2a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:49:10.187 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:10 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:10.166+0000 7f80e6e2a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:49:10.245 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:09 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3635244037' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T06:49:10.245 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:09 smithi103 ceph-mon[31158]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T06:49:10.245 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:09 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:09.990+0000 7fb39120c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:49:10.245 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:10 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:10.055+0000 7fb39120c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:49:10.245 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:10 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:10.243+0000 7fb39120c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:49:10.669 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T06:49:10.669 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T06:49:10.669 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi103.front.sepia.ceph.com 2024-09-19T06:49:10.669 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T06:49:10.669 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T06:49:10.669 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T06:49:10.669 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T06:49:10.669 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T06:49:10.669 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T06:49:10.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:49:10.669 DEBUG:teuthology.orchestra.run.smithi103:> ip addr 2024-09-19T06:49:10.685 INFO:teuthology.orchestra.run.smithi103.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T06:49:10.685 INFO:teuthology.orchestra.run.smithi103.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T06:49:10.685 INFO:teuthology.orchestra.run.smithi103.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T06:49:10.685 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-19T06:49:10.686 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 ::1/128 scope host 2024-09-19T06:49:10.686 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-19T06:49:10.686 INFO:teuthology.orchestra.run.smithi103.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T06:49:10.686 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c4 brd ff:ff:ff:ff:ff:ff 2024-09-19T06:49:10.686 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f0 2024-09-19T06:49:10.686 INFO:teuthology.orchestra.run.smithi103.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T06:49:10.686 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c5 brd ff:ff:ff:ff:ff:ff 2024-09-19T06:49:10.686 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f1 2024-09-19T06:49:10.686 INFO:teuthology.orchestra.run.smithi103.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T06:49:10.686 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0a brd ff:ff:ff:ff:ff:ff 2024-09-19T06:49:10.686 INFO:teuthology.orchestra.run.smithi103.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T06:49:10.686 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0b brd ff:ff:ff:ff:ff:ff 2024-09-19T06:49:10.686 INFO:teuthology.orchestra.run.smithi103.stdout: inet 172.21.15.103/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T06:49:10.687 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft 42213sec preferred_lft 42213sec 2024-09-19T06:49:10.687 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::ec4:7aff:fe8f:cc0b/64 scope link noprefixroute 2024-09-19T06:49:10.687 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-19T06:49:10.687 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T06:49:10.687 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:49:10.687 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-19T06:49:10.687 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add name ceph-brx type bridge 2024-09-19T06:49:10.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr flush dev ceph-brx 2024-09-19T06:49:10.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set ceph-brx up 2024-09-19T06:49:10.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T06:49:10.688 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-19T06:49:10.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:49:10.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:49:10.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:49:10.895 DEBUG:teuthology.orchestra.run.smithi103:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T06:49:10.922 INFO:teuthology.orchestra.run.smithi103.stdout:1 2024-09-19T06:49:10.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:49:10.924 DEBUG:teuthology.orchestra.run.smithi103:> route 2024-09-19T06:49:10.998 INFO:teuthology.orchestra.run.smithi103.stdout:Kernel IP routing table 2024-09-19T06:49:10.999 INFO:teuthology.orchestra.run.smithi103.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T06:49:10.999 INFO:teuthology.orchestra.run.smithi103.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T06:49:10.999 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T06:49:10.999 INFO:teuthology.orchestra.run.smithi103.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T06:49:10.999 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:49:10.999 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-19T06:49:10.999 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T06:49:10.999 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T06:49:11.000 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T06:49:11.000 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-19T06:49:11.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:49:11.173 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:10 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/439397549' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T06:49:11.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:10 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/439397549' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T06:49:11.187 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:11 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:11.047+0000 7f80e6e2a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:49:11.187 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:11 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:11.114+0000 7f80e6e2a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:49:11.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:49:11.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:49:11.313 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-19T06:49:11.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:49:11.329 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list-id 2024-09-19T06:49:11.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:10 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/439397549' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T06:49:11.333 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:11 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:11.056+0000 7f0ce2e34040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:49:11.333 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:11 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:11.125+0000 7f0ce2e34040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:49:11.385 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:49:11.385 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-19T06:49:11.385 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:49:11.385 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T06:49:11.385 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-19T06:49:11.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:49:11.481 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:11 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:11.171+0000 7fb39120c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:49:11.482 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:11 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:11.244+0000 7fb39120c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:49:11.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:49:11.547 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T06:49:11.547 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:49:11.547 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-19T06:49:11.547 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T06:49:11.547 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T06:49:11.547 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T06:49:11.547 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T06:49:11.548 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T06:49:11.548 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-19T06:49:11.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:49:11.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:49:11.676 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:49:11.676 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-19T06:49:11.676 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set brx.0 up 2024-09-19T06:49:11.676 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T06:49:11.676 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-19T06:49:11.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:49:11.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:49:11.755 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T06:49:11.755 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T06:49:11.770 INFO:teuthology.orchestra.run.smithi103.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T06:49:11.770 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T06:49:11.770 DEBUG:teuthology.orchestra.run.smithi103:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T06:49:11.826 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-19T06:49:11.826 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-19T06:49:11.826 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/e871e164-e13b-40d7-a1ed-57f3d47a3fa9 2024-09-19T06:49:11.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:49:11.826 DEBUG:teuthology.orchestra.run.smithi103:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/e871e164-e13b-40d7-a1ed-57f3d47a3fa9 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-19T06:49:11.922 INFO:teuthology.orchestra.run.smithi103.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-19T06:49:11.922 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-19T06:49:11.922 INFO:teuthology.orchestra.run.smithi103.stdout:invalid new device string format 2024-09-19T06:49:12.064 INFO:teuthology.orchestra.run.smithi103.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-19T06:49:12.064 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-19T06:49:12.064 INFO:teuthology.orchestra.run.smithi103.stdout:invalid new device string format 2024-09-19T06:49:12.064 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: resolved to: "172.21.15.103:3300,172.21.15.134:3300,172.21.15.174:3300" 2024-09-19T06:49:12.064 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.103:3300,172.21.15.134:3300,172.21.15.174:3300:/volumes/qa/sv_1/e871e164-e13b-40d7-a1ed-57f3d47a3fa9 2024-09-19T06:49:12.064 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=d158d934-7651-11ef-bceb-c7b262605968" will pass to kernel 2024-09-19T06:49:12.065 INFO:teuthology.orchestra.run.smithi103.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-19T06:49:12.065 INFO:teuthology.orchestra.run.smithi103.stdout: You just mounted a file system that supports labels which does not 2024-09-19T06:49:12.065 INFO:teuthology.orchestra.run.smithi103.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-19T06:49:12.065 INFO:teuthology.orchestra.run.smithi103.stdout: applications will generate AVC messages and not be allowed access to 2024-09-19T06:49:12.065 INFO:teuthology.orchestra.run.smithi103.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-19T06:49:12.067 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-19T06:49:12.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:49:12.067 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T06:49:12.074 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:11 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:11.813+0000 7f80e6e2a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:49:12.074 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:11 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:11.928+0000 7f80e6e2a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:49:12.102 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:11 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:11.838+0000 7f0ce2e34040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:49:12.102 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:11 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:11.956+0000 7f0ce2e34040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:49:12.281 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:12.008+0000 7fb39120c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:49:12.281 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:12.137+0000 7fb39120c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:49:12.350 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:12 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:12.072+0000 7f80e6e2a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:49:12.351 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:12 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:12.133+0000 7f80e6e2a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:49:12.351 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:12 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:12.281+0000 7f80e6e2a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:49:12.351 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:12 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:12.348+0000 7f80e6e2a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:49:12.382 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:12.100+0000 7f0ce2e34040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:49:12.382 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:12.162+0000 7f0ce2e34040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:49:12.383 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:12.312+0000 7f0ce2e34040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:49:12.383 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:12.380+0000 7f0ce2e34040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:49:12.560 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:12.280+0000 7fb39120c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:49:12.561 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:12.342+0000 7fb39120c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:49:12.561 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:12.491+0000 7fb39120c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:49:12.561 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:12.559+0000 7fb39120c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:49:12.633 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:12 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:12.409+0000 7f80e6e2a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:49:12.633 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:12 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:12.566+0000 7f80e6e2a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:49:12.633 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:12 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:12.631+0000 7f80e6e2a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:49:12.667 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:12.440+0000 7f0ce2e34040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:49:12.668 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:12.599+0000 7f0ce2e34040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:49:12.845 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:12.620+0000 7fb39120c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:49:12.845 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:12.777+0000 7fb39120c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:49:12.845 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:12.843+0000 7fb39120c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:49:12.899 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:12 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:12.696+0000 7f80e6e2a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:49:12.899 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:12 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:12.778+0000 7f80e6e2a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:49:12.899 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:12 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:12.837+0000 7f80e6e2a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:49:12.900 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:12 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:12.897+0000 7f80e6e2a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:49:12.939 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:12.665+0000 7f0ce2e34040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:49:12.939 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:12.732+0000 7f0ce2e34040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:49:12.939 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:12.815+0000 7f0ce2e34040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:49:12.939 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:12.875+0000 7f0ce2e34040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:49:13.115 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:12.911+0000 7fb39120c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:49:13.115 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:12.995+0000 7fb39120c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:49:13.115 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:13 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:13.054+0000 7fb39120c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:49:13.115 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:13 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:13.114+0000 7fb39120c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:49:13.333 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:12.937+0000 7f0ce2e34040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:49:13.937 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 06:49:13 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y[36719]: 2024-09-19T06:49:13.683+0000 7f80e6e2a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:49:13.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:13 smithi134 ceph-mon[35356]: Standby manager daemon y restarted 2024-09-19T06:49:13.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:13 smithi134 ceph-mon[35356]: Standby manager daemon y started 2024-09-19T06:49:13.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:13 smithi134 ceph-mon[35356]: from='mgr.? 172.21.15.134:0/4253110061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T06:49:13.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:13 smithi134 ceph-mon[35356]: from='mgr.? 172.21.15.134:0/4253110061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:49:13.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:13 smithi134 ceph-mon[35356]: from='mgr.? 172.21.15.134:0/4253110061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T06:49:13.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:13 smithi134 ceph-mon[35356]: from='mgr.? 172.21.15.134:0/4253110061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:49:13.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:13 smithi134 ceph-mon[35356]: Standby manager daemon z restarted 2024-09-19T06:49:13.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:13 smithi134 ceph-mon[35356]: Standby manager daemon z started 2024-09-19T06:49:13.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:13 smithi134 ceph-mon[35356]: from='mgr.? 172.21.15.174:0/3776086272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T06:49:13.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:13 smithi134 ceph-mon[35356]: from='mgr.? 172.21.15.174:0/3776086272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:49:13.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:13 smithi134 ceph-mon[35356]: from='mgr.? 172.21.15.174:0/3776086272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T06:49:13.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:13 smithi134 ceph-mon[35356]: from='mgr.? 172.21.15.174:0/3776086272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:49:13.977 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T06:49:14.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T06:49:14.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:49:14.041 INFO:teuthology.orchestra.run.smithi103.stdout:client24794 2024-09-19T06:49:14.041 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:49:14.041 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T06:49:14.063 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T06:49:14.083 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 06:49:13 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z[37438]: 2024-09-19T06:49:13.727+0000 7f0ce2e34040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:49:14.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:13 smithi174 ceph-mon[35637]: Standby manager daemon y restarted 2024-09-19T06:49:14.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:13 smithi174 ceph-mon[35637]: Standby manager daemon y started 2024-09-19T06:49:14.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:13 smithi174 ceph-mon[35637]: from='mgr.? 172.21.15.134:0/4253110061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T06:49:14.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:13 smithi174 ceph-mon[35637]: from='mgr.? 172.21.15.134:0/4253110061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:49:14.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:13 smithi174 ceph-mon[35637]: from='mgr.? 172.21.15.134:0/4253110061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T06:49:14.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:13 smithi174 ceph-mon[35637]: from='mgr.? 172.21.15.134:0/4253110061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:49:14.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:13 smithi174 ceph-mon[35637]: Standby manager daemon z restarted 2024-09-19T06:49:14.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:13 smithi174 ceph-mon[35637]: Standby manager daemon z started 2024-09-19T06:49:14.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:13 smithi174 ceph-mon[35637]: from='mgr.? 172.21.15.174:0/3776086272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T06:49:14.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:13 smithi174 ceph-mon[35637]: from='mgr.? 172.21.15.174:0/3776086272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:49:14.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:13 smithi174 ceph-mon[35637]: from='mgr.? 172.21.15.174:0/3776086272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T06:49:14.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:13 smithi174 ceph-mon[35637]: from='mgr.? 172.21.15.174:0/3776086272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:49:14.138 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:13 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:13.907+0000 7fb39120c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:49:14.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:13 smithi103 ceph-mon[31158]: Standby manager daemon y restarted 2024-09-19T06:49:14.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:13 smithi103 ceph-mon[31158]: Standby manager daemon y started 2024-09-19T06:49:14.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:13 smithi103 ceph-mon[31158]: from='mgr.? 172.21.15.134:0/4253110061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T06:49:14.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:13 smithi103 ceph-mon[31158]: from='mgr.? 172.21.15.134:0/4253110061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:49:14.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:13 smithi103 ceph-mon[31158]: from='mgr.? 172.21.15.134:0/4253110061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T06:49:14.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:13 smithi103 ceph-mon[31158]: from='mgr.? 172.21.15.134:0/4253110061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:49:14.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:13 smithi103 ceph-mon[31158]: Standby manager daemon z restarted 2024-09-19T06:49:14.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:13 smithi103 ceph-mon[31158]: Standby manager daemon z started 2024-09-19T06:49:14.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:13 smithi103 ceph-mon[31158]: from='mgr.? 172.21.15.174:0/3776086272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T06:49:14.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:13 smithi103 ceph-mon[31158]: from='mgr.? 172.21.15.174:0/3776086272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:49:14.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:13 smithi103 ceph-mon[31158]: from='mgr.? 172.21.15.174:0/3776086272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T06:49:14.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:13 smithi103 ceph-mon[31158]: from='mgr.? 172.21.15.174:0/3776086272' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:49:14.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T06:49:14.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:49:14.186 INFO:teuthology.orchestra.run.smithi103.stdout:client24794 2024-09-19T06:49:14.187 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T06:49:14.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T06:49:14.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:49:14.292 INFO:teuthology.orchestra.run.smithi103.stdout:client24794 2024-09-19T06:49:14.292 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d158d934-7651-11ef-bceb-c7b262605968.client24794/status') 2024-09-19T06:49:14.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:49:14.356 INFO:teuthology.orchestra.run.smithi103.stdout:instance: client.24794 (3)172.21.15.103:0/2032471038 2024-09-19T06:49:14.356 INFO:teuthology.orchestra.run.smithi103.stdout:blocklisted: false 2024-09-19T06:49:14.356 INFO:teuthology.orchestra.run.smithi103.stderr:0+1 records in 2024-09-19T06:49:14.356 INFO:teuthology.orchestra.run.smithi103.stderr:0+1 records out 2024-09-19T06:49:14.356 INFO:teuthology.orchestra.run.smithi103.stderr:72 bytes copied, 2.2852e-05 s, 3.2 MB/s 2024-09-19T06:49:14.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:49:14.364 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T06:49:14.375 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T06:49:14.375 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T06:49:14.375 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-19T06:49:14.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.477+0000 7f37c0ed0640 1 -- 172.21.15.103:0/3810696159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc104ef0 msgr2=0x7f37bc1072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:14.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.477+0000 7f37c0ed0640 1 --2- 172.21.15.103:0/3810696159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc104ef0 0x7f37bc1072e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009900 tx=0x7f37b002a410 comp rx=0 tx=0).stop 2024-09-19T06:49:14.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.477+0000 7f37c0ed0640 1 -- 172.21.15.103:0/3810696159 shutdown_connections 2024-09-19T06:49:14.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.477+0000 7f37c0ed0640 1 --2- 172.21.15.103:0/3810696159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37bc107820 0x7f37bc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.477+0000 7f37c0ed0640 1 --2- 172.21.15.103:0/3810696159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc104ef0 0x7f37bc1072e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.477+0000 7f37c0ed0640 1 --2- 172.21.15.103:0/3810696159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc1025c0 0x7f37bc1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.477+0000 7f37c0ed0640 1 -- 172.21.15.103:0/3810696159 >> 172.21.15.103:0/3810696159 conn(0x7f37bc0fc1e0 msgr2=0x7f37bc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:14.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.477+0000 7f37c0ed0640 1 -- 172.21.15.103:0/3810696159 shutdown_connections 2024-09-19T06:49:14.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.477+0000 7f37c0ed0640 1 -- 172.21.15.103:0/3810696159 wait complete. 2024-09-19T06:49:14.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.478+0000 7f37c0ed0640 1 Processor -- start 2024-09-19T06:49:14.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.478+0000 7f37c0ed0640 1 -- start start 2024-09-19T06:49:14.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.478+0000 7f37c0ed0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc1025c0 0x7f37bc101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.478+0000 7f37c0ed0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37bc104ef0 0x7f37bc100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.478+0000 7f37c0ed0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc107820 0x7f37bc100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.478+0000 7f37c0ed0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc10d800 con 0x7f37bc107820 2024-09-19T06:49:14.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.478+0000 7f37c0ed0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc10d680 con 0x7f37bc1025c0 2024-09-19T06:49:14.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.478+0000 7f37c0ed0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc10d980 con 0x7f37bc104ef0 2024-09-19T06:49:14.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.478+0000 7f37bad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc107820 0x7f37bc100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.478+0000 7f37bad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc107820 0x7f37bc100ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59350/0 (socket says 172.21.15.103:59350) 2024-09-19T06:49:14.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.478+0000 7f37bad9d640 1 -- 172.21.15.103:0/1448457829 learned_addr learned my addr 172.21.15.103:0/1448457829 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:14.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.478+0000 7f37b9d9b640 1 --2- 172.21.15.103:0/1448457829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37bc104ef0 0x7f37bc100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.478+0000 7f37ba59c640 1 --2- 172.21.15.103:0/1448457829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc1025c0 0x7f37bc101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.478+0000 7f37bad9d640 1 -- 172.21.15.103:0/1448457829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37bc104ef0 msgr2=0x7f37bc100570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:14.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.478+0000 7f37bad9d640 1 --2- 172.21.15.103:0/1448457829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37bc104ef0 0x7f37bc100570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.478+0000 7f37bad9d640 1 -- 172.21.15.103:0/1448457829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc1025c0 msgr2=0x7f37bc101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:14.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.479+0000 7f37bad9d640 1 --2- 172.21.15.103:0/1448457829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc1025c0 0x7f37bc101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.479+0000 7f37bad9d640 1 -- 172.21.15.103:0/1448457829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37bc101340 con 0x7f37bc107820 2024-09-19T06:49:14.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.479+0000 7f37b9d9b640 1 --2- 172.21.15.103:0/1448457829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37bc104ef0 0x7f37bc100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:14.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.479+0000 7f37bad9d640 1 --2- 172.21.15.103:0/1448457829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc107820 0x7f37bc100ab0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f37b400e9d0 tx=0x7f37b400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:14.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.479+0000 7f37ab7fe640 1 -- 172.21.15.103:0/1448457829 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b400cd80 con 0x7f37bc107820 2024-09-19T06:49:14.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.479+0000 7f37ab7fe640 1 -- 172.21.15.103:0/1448457829 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b4005020 con 0x7f37bc107820 2024-09-19T06:49:14.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.479+0000 7f37c0ed0640 1 -- 172.21.15.103:0/1448457829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37bc1a1f50 con 0x7f37bc107820 2024-09-19T06:49:14.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.479+0000 7f37ab7fe640 1 -- 172.21.15.103:0/1448457829 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b40116d0 con 0x7f37bc107820 2024-09-19T06:49:14.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.479+0000 7f37c0ed0640 1 -- 172.21.15.103:0/1448457829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37bc1a23b0 con 0x7f37bc107820 2024-09-19T06:49:14.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.480+0000 7f37c0ed0640 1 -- 172.21.15.103:0/1448457829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37bc0fdcd0 con 0x7f37bc107820 2024-09-19T06:49:14.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.482+0000 7f37ab7fe640 1 -- 172.21.15.103:0/1448457829 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138040+0+0 (secure 0 0 0) 0x7f37b40051d0 con 0x7f37bc107820 2024-09-19T06:49:14.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.483+0000 7f37ab7fe640 1 -- 172.21.15.103:0/1448457829 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f37b40cf990 con 0x7f37bc107820 2024-09-19T06:49:14.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.486+0000 7f37ab7fe640 1 -- 172.21.15.103:0/1448457829 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f37b4016030 con 0x7f37bc107820 2024-09-19T06:49:14.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:14.191+0000 7fb2f8ade640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:49:14.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:14.191+0000 7fb2f8ade640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:49:14.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:14.191+0000 7fb2f8ade640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:49:14.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:14.191+0000 7fb2f8ade640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:49:14.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:14.191+0000 7fb2f8ade640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:49:14.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:14.191+0000 7fb2faae2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:49:14.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:14.191+0000 7fb2faae2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:49:14.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:14.191+0000 7fb2faae2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:49:14.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:14.191+0000 7fb2faae2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:49:14.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:14.191+0000 7fb2faae2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:49:14.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.706+0000 7f37c0ed0640 1 -- 172.21.15.103:0/1448457829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f37bc02d7c0 con 0x7f37bc107820 2024-09-19T06:49:14.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.927+0000 7f37ab7fe640 1 -- 172.21.15.103:0/1448457829 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 21) ==== 138167+0+0 (secure 0 0 0) 0x7f37b4097ff0 con 0x7f37bc107820 2024-09-19T06:49:14.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.927+0000 7f37ab7fe640 1 --2- 172.21.15.103:0/1448457829 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f37a40aa060 0x7f37a40ac450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.932+0000 7f37ab7fe640 1 -- 172.21.15.103:0/1448457829 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7f37b409c9a0 con 0x7f37bc107820 2024-09-19T06:49:14.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.932+0000 7f37ba59c640 1 --2- 172.21.15.103:0/1448457829 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f37a40aa060 0x7f37a40ac450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.936+0000 7f37ba59c640 1 --2- 172.21.15.103:0/1448457829 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f37a40aa060 0x7f37a40ac450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f37ac005d00 tx=0x7f37ac005c50 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:14.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.937+0000 7f37c0ed0640 1 -- 172.21.15.103:0/1448457829 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f37a40aa060 msgr2=0x7f37a40ac450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:14.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.937+0000 7f37c0ed0640 1 --2- 172.21.15.103:0/1448457829 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f37a40aa060 0x7f37a40ac450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f37ac005d00 tx=0x7f37ac005c50 comp rx=0 tx=0).stop 2024-09-19T06:49:14.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.937+0000 7f37c0ed0640 1 -- 172.21.15.103:0/1448457829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc107820 msgr2=0x7f37bc100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:14.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.937+0000 7f37c0ed0640 1 --2- 172.21.15.103:0/1448457829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc107820 0x7f37bc100ab0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f37b400e9d0 tx=0x7f37b400ee90 comp rx=0 tx=0).stop 2024-09-19T06:49:14.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.937+0000 7f37c0ed0640 1 -- 172.21.15.103:0/1448457829 shutdown_connections 2024-09-19T06:49:14.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.937+0000 7f37c0ed0640 1 --2- 172.21.15.103:0/1448457829 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f37a40aa060 0x7f37a40ac450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.937+0000 7f37c0ed0640 1 --2- 172.21.15.103:0/1448457829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc107820 0x7f37bc100ab0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.937+0000 7f37c0ed0640 1 --2- 172.21.15.103:0/1448457829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37bc104ef0 0x7f37bc100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.937+0000 7f37c0ed0640 1 --2- 172.21.15.103:0/1448457829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc1025c0 0x7f37bc101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.937+0000 7f37c0ed0640 1 -- 172.21.15.103:0/1448457829 >> 172.21.15.103:0/1448457829 conn(0x7f37bc0fc1e0 msgr2=0x7f37bc102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:14.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.937+0000 7f37c0ed0640 1 -- 172.21.15.103:0/1448457829 shutdown_connections 2024-09-19T06:49:14.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:14.937+0000 7f37c0ed0640 1 -- 172.21.15.103:0/1448457829 wait complete. 2024-09-19T06:49:14.949 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-19T06:49:15.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.053+0000 7f77ea847640 1 -- 172.21.15.103:0/1127564726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e4100a40 msgr2=0x7f77e4100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.053+0000 7f77ea847640 1 --2- 172.21.15.103:0/1127564726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e4100a40 0x7f77e4100e40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009900 tx=0x7f77d402a410 comp rx=0 tx=0).stop 2024-09-19T06:49:15.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.053+0000 7f77ea847640 1 -- 172.21.15.103:0/1127564726 shutdown_connections 2024-09-19T06:49:15.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.053+0000 7f77ea847640 1 --2- 172.21.15.103:0/1127564726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77e41025f0 0x7f77e4108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.053+0000 7f77ea847640 1 --2- 172.21.15.103:0/1127564726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e4101c30 0x7f77e41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.053+0000 7f77ea847640 1 --2- 172.21.15.103:0/1127564726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e4100a40 0x7f77e4100e40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.053+0000 7f77ea847640 1 -- 172.21.15.103:0/1127564726 >> 172.21.15.103:0/1127564726 conn(0x7f77e40fc1e0 msgr2=0x7f77e40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:15.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.053+0000 7f77ea847640 1 -- 172.21.15.103:0/1127564726 shutdown_connections 2024-09-19T06:49:15.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.053+0000 7f77ea847640 1 -- 172.21.15.103:0/1127564726 wait complete. 2024-09-19T06:49:15.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77ea847640 1 Processor -- start 2024-09-19T06:49:15.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77ea847640 1 -- start start 2024-09-19T06:49:15.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77ea847640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77e4100a40 0x7f77e41049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77ea847640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e4101c30 0x7f77e4103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77ea847640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e41025f0 0x7f77e4103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77ea847640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77e410c6a0 con 0x7f77e4101c30 2024-09-19T06:49:15.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77ea847640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f77e410c520 con 0x7f77e41025f0 2024-09-19T06:49:15.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77ea847640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f77e410c820 con 0x7f77e4100a40 2024-09-19T06:49:15.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77e37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e4101c30 0x7f77e4103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77e37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e4101c30 0x7f77e4103010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59368/0 (socket says 172.21.15.103:59368) 2024-09-19T06:49:15.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77e37fe640 1 -- 172.21.15.103:0/3331503982 learned_addr learned my addr 172.21.15.103:0/3331503982 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:15.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77e8de4640 1 --2- 172.21.15.103:0/3331503982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e41025f0 0x7f77e4103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77e3fff640 1 --2- 172.21.15.103:0/3331503982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77e4100a40 0x7f77e41049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77e37fe640 1 -- 172.21.15.103:0/3331503982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77e4100a40 msgr2=0x7f77e41049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77e37fe640 1 --2- 172.21.15.103:0/3331503982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77e4100a40 0x7f77e41049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77e37fe640 1 -- 172.21.15.103:0/3331503982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e41025f0 msgr2=0x7f77e4103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77e37fe640 1 --2- 172.21.15.103:0/3331503982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e41025f0 0x7f77e4103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77e37fe640 1 -- 172.21.15.103:0/3331503982 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e4103de0 con 0x7f77e4101c30 2024-09-19T06:49:15.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77e3fff640 1 --2- 172.21.15.103:0/3331503982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77e4100a40 0x7f77e41049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:15.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.054+0000 7f77e37fe640 1 --2- 172.21.15.103:0/3331503982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e4101c30 0x7f77e4103010 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00e9f0 tx=0x7f77cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:15.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.055+0000 7f77e17fa640 1 -- 172.21.15.103:0/3331503982 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc00ccc0 con 0x7f77e4101c30 2024-09-19T06:49:15.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.055+0000 7f77ea847640 1 -- 172.21.15.103:0/3331503982 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e41a1f20 con 0x7f77e4101c30 2024-09-19T06:49:15.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.055+0000 7f77e17fa640 1 -- 172.21.15.103:0/3331503982 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77cc010910 con 0x7f77e4101c30 2024-09-19T06:49:15.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.055+0000 7f77e17fa640 1 -- 172.21.15.103:0/3331503982 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0188a0 con 0x7f77e4101c30 2024-09-19T06:49:15.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.055+0000 7f77ea847640 1 -- 172.21.15.103:0/3331503982 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e41a2380 con 0x7f77e4101c30 2024-09-19T06:49:15.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.056+0000 7f77ea847640 1 -- 172.21.15.103:0/3331503982 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e41a2670 con 0x7f77e4101c30 2024-09-19T06:49:15.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.058+0000 7f77e17fa640 1 -- 172.21.15.103:0/3331503982 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 21) ==== 138167+0+0 (secure 0 0 0) 0x7f77cc010ac0 con 0x7f77e4101c30 2024-09-19T06:49:15.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.059+0000 7f77e17fa640 1 --2- 172.21.15.103:0/3331503982 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f77d00a3100 0x7f77d00a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.059+0000 7f77e3fff640 1 --2- 172.21.15.103:0/3331503982 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f77d00a3100 0x7f77d00a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.059+0000 7f77e17fa640 1 -- 172.21.15.103:0/3331503982 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f77cc0cff60 con 0x7f77e4101c30 2024-09-19T06:49:15.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.059+0000 7f77e3fff640 1 --2- 172.21.15.103:0/3331503982 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f77d00a3100 0x7f77d00a55c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f77d4006010 tx=0x7f77d4032cf0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:15.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.061+0000 7f77e17fa640 1 -- 172.21.15.103:0/3331503982 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f77cc09ce60 con 0x7f77e4101c30 2024-09-19T06:49:15.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-09-19T06:49:15.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: Active manager daemon x restarted 2024-09-19T06:49:15.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: Activating manager daemon x 2024-09-19T06:49:15.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T06:49:15.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: mgrmap e20: x(active, starting, since 0.0148475s), standbys: y, z 2024-09-19T06:49:15.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:49:15.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:49:15.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:49:15.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:49:15.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:49:15.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:49:15.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:49:15.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:49:15.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:49:15.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:49:15.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:49:15.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:49:15.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:49:15.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:49:15.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:49:15.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T06:49:15.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T06:49:15.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T06:49:15.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:49:15.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:49:15.085 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:49:15.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:49:15.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:49:15.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:49:15.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:49:15.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:49:15.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:49:15.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:49:15.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:49:15.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:49:15.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:49:15.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:49:15.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:49:15.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: Manager daemon x is now available 2024-09-19T06:49:15.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:15.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:49:15.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T06:49:15.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:14 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1448457829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T06:49:15.094 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-09-19T06:49:15.095 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: Active manager daemon x restarted 2024-09-19T06:49:15.095 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: Activating manager daemon x 2024-09-19T06:49:15.095 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T06:49:15.095 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: mgrmap e20: x(active, starting, since 0.0148475s), standbys: y, z 2024-09-19T06:49:15.095 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:49:15.095 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:49:15.095 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:49:15.095 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:49:15.095 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:49:15.095 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:49:15.095 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:49:15.096 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:49:15.096 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:49:15.096 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:49:15.096 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:49:15.096 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:49:15.096 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:49:15.096 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:49:15.096 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:49:15.096 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T06:49:15.096 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T06:49:15.097 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T06:49:15.097 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:49:15.097 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:49:15.097 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:49:15.097 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:49:15.097 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:49:15.097 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:49:15.097 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:49:15.097 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:49:15.098 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:49:15.098 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:49:15.098 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:49:15.098 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:49:15.098 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:49:15.098 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:49:15.098 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:49:15.098 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: Manager daemon x is now available 2024-09-19T06:49:15.098 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:15.098 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:49:15.098 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T06:49:15.098 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:14 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1448457829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T06:49:15.124 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-09-19T06:49:15.124 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: Active manager daemon x restarted 2024-09-19T06:49:15.124 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: Activating manager daemon x 2024-09-19T06:49:15.124 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T06:49:15.124 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: mgrmap e20: x(active, starting, since 0.0148475s), standbys: y, z 2024-09-19T06:49:15.124 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:49:15.124 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:49:15.125 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:49:15.125 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:49:15.125 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:49:15.125 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:49:15.125 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:49:15.125 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:49:15.125 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:49:15.125 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:49:15.125 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:49:15.125 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:49:15.125 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:49:15.126 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:49:15.126 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:49:15.126 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T06:49:15.126 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T06:49:15.126 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T06:49:15.126 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:49:15.126 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:49:15.126 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:49:15.126 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:49:15.126 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:49:15.126 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:49:15.126 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:49:15.126 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:49:15.127 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:49:15.127 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:49:15.127 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:49:15.127 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:49:15.127 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:49:15.127 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:49:15.127 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:49:15.127 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: Manager daemon x is now available 2024-09-19T06:49:15.127 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:15.127 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:49:15.127 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T06:49:15.127 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:14 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1448457829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T06:49:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.264+0000 7f77ea847640 1 -- 172.21.15.103:0/3331503982 --> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f77e40fe4b0 con 0x7f77d00a3100 2024-09-19T06:49:15.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.280+0000 7f77e17fa640 1 -- 172.21.15.103:0/3331503982 <== mgr.15210 v2:172.21.15.103:6800/82882809 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f77e40fe4b0 con 0x7f77d00a3100 2024-09-19T06:49:15.281 INFO:teuthology.orchestra.run.smithi103.stdout:{} 2024-09-19T06:49:15.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.283+0000 7f77ea847640 1 -- 172.21.15.103:0/3331503982 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f77d00a3100 msgr2=0x7f77d00a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.283+0000 7f77ea847640 1 --2- 172.21.15.103:0/3331503982 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f77d00a3100 0x7f77d00a55c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f77d4006010 tx=0x7f77d4032cf0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.283+0000 7f77ea847640 1 -- 172.21.15.103:0/3331503982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e4101c30 msgr2=0x7f77e4103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.283+0000 7f77ea847640 1 --2- 172.21.15.103:0/3331503982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e4101c30 0x7f77e4103010 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00e9f0 tx=0x7f77cc00edf0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.283+0000 7f77ea847640 1 -- 172.21.15.103:0/3331503982 shutdown_connections 2024-09-19T06:49:15.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.283+0000 7f77ea847640 1 --2- 172.21.15.103:0/3331503982 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f77d00a3100 0x7f77d00a55c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.283+0000 7f77ea847640 1 --2- 172.21.15.103:0/3331503982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e41025f0 0x7f77e4103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.283+0000 7f77ea847640 1 --2- 172.21.15.103:0/3331503982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77e4101c30 0x7f77e4103010 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.283+0000 7f77ea847640 1 --2- 172.21.15.103:0/3331503982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77e4100a40 0x7f77e41049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.283+0000 7f77ea847640 1 -- 172.21.15.103:0/3331503982 >> 172.21.15.103:0/3331503982 conn(0x7f77e40fc1e0 msgr2=0x7f77e40fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:15.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.283+0000 7f77ea847640 1 -- 172.21.15.103:0/3331503982 shutdown_connections 2024-09-19T06:49:15.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.283+0000 7f77ea847640 1 -- 172.21.15.103:0/3331503982 wait complete. 2024-09-19T06:49:15.295 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T06:49:15.306 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-19T06:49:15.323 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T06:49:15.388 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:15.251+0000 7fb2e4a37640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:06:49:15] ENGINE Error in HTTPServer.serve 2024-09-19T06:49:15.389 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: Traceback (most recent call last): 2024-09-19T06:49:15.389 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:49:15.389 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: self._connections.run(self.expiration_interval) 2024-09-19T06:49:15.389 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:49:15.389 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: self._run(expiration_interval) 2024-09-19T06:49:15.389 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:49:15.389 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:49:15.389 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:49:15.389 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:49:15.389 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:49:15.389 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: s = self.context.wrap_socket( 2024-09-19T06:49:15.389 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:49:15.389 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: return self.sslsocket_class._create( 2024-09-19T06:49:15.390 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:49:15.390 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: self.do_handshake() 2024-09-19T06:49:15.390 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:49:15.390 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: self._sslobj.do_handshake() 2024-09-19T06:49:15.390 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:49:15.390 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:15.390 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:15.269+0000 7fb2f5a98640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:49:15.390 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:15.269+0000 7fb2f5a98640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:49:15.390 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:15.269+0000 7fb2f5a98640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:49:15.390 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:15.269+0000 7fb2f5a98640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:49:15.390 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:49:15.269+0000 7fb2f5a98640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:49:15.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.424+0000 7f48c23d8640 1 -- 172.21.15.103:0/3384558751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48bc100f50 msgr2=0x7f48bc101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.424+0000 7f48c23d8640 1 --2- 172.21.15.103:0/3384558751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48bc100f50 0x7f48bc101350 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009900 tx=0x7f48ac02a480 comp rx=0 tx=0).stop 2024-09-19T06:49:15.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.424+0000 7f48c23d8640 1 -- 172.21.15.103:0/3384558751 shutdown_connections 2024-09-19T06:49:15.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.424+0000 7f48c23d8640 1 --2- 172.21.15.103:0/3384558751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48bc102620 0x7f48bc108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.424+0000 7f48c23d8640 1 --2- 172.21.15.103:0/3384558751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48bc101c60 0x7f48bc1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.424+0000 7f48c23d8640 1 --2- 172.21.15.103:0/3384558751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48bc100f50 0x7f48bc101350 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.424+0000 7f48c23d8640 1 -- 172.21.15.103:0/3384558751 >> 172.21.15.103:0/3384558751 conn(0x7f48bc0fb950 msgr2=0x7f48bc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:15.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.424+0000 7f48c23d8640 1 -- 172.21.15.103:0/3384558751 shutdown_connections 2024-09-19T06:49:15.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.424+0000 7f48c23d8640 1 -- 172.21.15.103:0/3384558751 wait complete. 2024-09-19T06:49:15.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.425+0000 7f48c23d8640 1 Processor -- start 2024-09-19T06:49:15.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.425+0000 7f48c23d8640 1 -- start start 2024-09-19T06:49:15.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.425+0000 7f48c23d8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48bc100f50 0x7f48bc197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.425+0000 7f48c23d8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48bc101c60 0x7f48bc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.425+0000 7f48c23d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48bc102620 0x7f48bc19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.425+0000 7f48c23d8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc10c6d0 con 0x7f48bc102620 2024-09-19T06:49:15.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.425+0000 7f48c23d8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc10c550 con 0x7f48bc101c60 2024-09-19T06:49:15.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.425+0000 7f48c23d8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc10c850 con 0x7f48bc100f50 2024-09-19T06:49:15.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.425+0000 7f48c0975640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48bc102620 0x7f48bc19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.425+0000 7f48c0975640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48bc102620 0x7f48bc19f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59402/0 (socket says 172.21.15.103:59402) 2024-09-19T06:49:15.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.425+0000 7f48c0975640 1 -- 172.21.15.103:0/2505819648 learned_addr learned my addr 172.21.15.103:0/2505819648 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:15.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.426+0000 7f48bbfff640 1 --2- 172.21.15.103:0/2505819648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48bc100f50 0x7f48bc197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.426+0000 7f48bb7fe640 1 --2- 172.21.15.103:0/2505819648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48bc101c60 0x7f48bc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.426+0000 7f48c0975640 1 -- 172.21.15.103:0/2505819648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48bc100f50 msgr2=0x7f48bc197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.426+0000 7f48c0975640 1 --2- 172.21.15.103:0/2505819648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48bc100f50 0x7f48bc197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.426+0000 7f48c0975640 1 -- 172.21.15.103:0/2505819648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48bc101c60 msgr2=0x7f48bc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.426+0000 7f48c0975640 1 --2- 172.21.15.103:0/2505819648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48bc101c60 0x7f48bc198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.426+0000 7f48c0975640 1 -- 172.21.15.103:0/2505819648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48bc19fa50 con 0x7f48bc102620 2024-09-19T06:49:15.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.426+0000 7f48bb7fe640 1 --2- 172.21.15.103:0/2505819648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48bc101c60 0x7f48bc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:15.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.426+0000 7f48bbfff640 1 --2- 172.21.15.103:0/2505819648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48bc100f50 0x7f48bc197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:15.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.426+0000 7f48c0975640 1 --2- 172.21.15.103:0/2505819648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48bc102620 0x7f48bc19f3e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f48b400cce0 tx=0x7f48b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:15.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.426+0000 7f48b97fa640 1 -- 172.21.15.103:0/2505819648 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b400f450 con 0x7f48bc102620 2024-09-19T06:49:15.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.426+0000 7f48b97fa640 1 -- 172.21.15.103:0/2505819648 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48b400fad0 con 0x7f48bc102620 2024-09-19T06:49:15.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.426+0000 7f48c23d8640 1 -- 172.21.15.103:0/2505819648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48bc19fd40 con 0x7f48bc102620 2024-09-19T06:49:15.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.426+0000 7f48b97fa640 1 -- 172.21.15.103:0/2505819648 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b4018c10 con 0x7f48bc102620 2024-09-19T06:49:15.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.426+0000 7f48c23d8640 1 -- 172.21.15.103:0/2505819648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48bc1a01e0 con 0x7f48bc102620 2024-09-19T06:49:15.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.427+0000 7f48c23d8640 1 -- 172.21.15.103:0/2505819648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48bc1a0470 con 0x7f48bc102620 2024-09-19T06:49:15.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.428+0000 7f48b97fa640 1 -- 172.21.15.103:0/2505819648 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 21) ==== 138167+0+0 (secure 0 0 0) 0x7f48b400f5f0 con 0x7f48bc102620 2024-09-19T06:49:15.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.429+0000 7f48b97fa640 1 --2- 172.21.15.103:0/2505819648 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48a40a3050 0x7f48a40a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.429+0000 7f48b97fa640 1 -- 172.21.15.103:0/2505819648 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f48b40cfd90 con 0x7f48bc102620 2024-09-19T06:49:15.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.429+0000 7f48bbfff640 1 --2- 172.21.15.103:0/2505819648 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48a40a3050 0x7f48a40a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.429+0000 7f48bbfff640 1 --2- 172.21.15.103:0/2505819648 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48a40a3050 0x7f48a40a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f48ac006010 tx=0x7f48ac032dd0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:15.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.433+0000 7f48b97fa640 1 -- 172.21.15.103:0/2505819648 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f48b4014030 con 0x7f48bc102620 2024-09-19T06:49:15.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.619+0000 7f48c23d8640 1 -- 172.21.15.103:0/2505819648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48bc1a07d0 con 0x7f48bc102620 2024-09-19T06:49:15.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.620+0000 7f48b97fa640 1 -- 172.21.15.103:0/2505819648 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+23173 (secure 0 0 0) 0x7f48b409cc90 con 0x7f48bc102620 2024-09-19T06:49:15.624 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:49:15.625 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":83,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","created":"2024-09-19T06:39:46.840574+0000","modified":"2024-09-19T06:49:13.909427+0000","last_up_change":"2024-09-19T06:46:35.652960+0000","last_in_change":"2024-09-19T06:46:11.630585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:42:52.050881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:47:21.192611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:47:23.643291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d45dab3-e434-4783-b2e3-f658c3ba583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6803","nonce":3157697740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6805","nonce":3157697740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6809","nonce":3157697740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6807","nonce":3157697740}]},"public_addr":"172.21.15.103:6803/3157697740","cluster_addr":"172.21.15.103:6805/3157697740","heartbeat_back_addr":"172.21.15.103:6809/3157697740","heartbeat_front_addr":"172.21.15.103:6807/3157697740","state":["exists","up"]},{"osd":1,"uuid":"ee0e92bd-15ab-4867-9a18-20abb9f5fc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6801","nonce":1418301195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6803","nonce":1418301195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6807","nonce":1418301195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6805","nonce":1418301195}]},"public_addr":"172.21.15.134:6801/1418301195","cluster_addr":"172.21.15.134:6803/1418301195","heartbeat_back_addr":"172.21.15.134:6807/1418301195","heartbeat_front_addr":"172.21.15.134:6805/1418301195","state":["exists","up"]},{"osd":2,"uuid":"0230caed-025b-46e9-a011-237601d16ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6801","nonce":2292242290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6803","nonce":2292242290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6807","nonce":2292242290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6805","nonce":2292242290}]},"public_addr":"172.21.15.174:6801/2292242290","cluster_addr":"172.21.15.174:6803/2292242290","heartbeat_back_addr":"172.21.15.174:6807/2292242290","heartbeat_front_addr":"172.21.15.174:6805/2292242290","state":["exists","up"]},{"osd":3,"uuid":"c54a8399-bf4b-4d70-b207-a13d85f99590","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6811","nonce":2698986333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6813","nonce":2698986333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6817","nonce":2698986333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6815","nonce":2698986333}]},"public_addr":"172.21.15.103:6811/2698986333","cluster_addr":"172.21.15.103:6813/2698986333","heartbeat_back_addr":"172.21.15.103:6817/2698986333","heartbeat_front_addr":"172.21.15.103:6815/2698986333","state":["exists","up"]},{"osd":4,"uuid":"7c046be6-464a-4ab5-812b-1781121fc317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6809","nonce":3169891968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6811","nonce":3169891968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6815","nonce":3169891968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6813","nonce":3169891968}]},"public_addr":"172.21.15.134:6809/3169891968","cluster_addr":"172.21.15.134:6811/3169891968","heartbeat_back_addr":"172.21.15.134:6815/3169891968","heartbeat_front_addr":"172.21.15.134:6813/3169891968","state":["exists","up"]},{"osd":5,"uuid":"8622596b-6a06-459c-882d-50fba9e296ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6809","nonce":763749644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6811","nonce":763749644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6815","nonce":763749644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6813","nonce":763749644}]},"public_addr":"172.21.15.174:6809/763749644","cluster_addr":"172.21.15.174:6811/763749644","heartbeat_back_addr":"172.21.15.174:6815/763749644","heartbeat_front_addr":"172.21.15.174:6813/763749644","state":["exists","up"]},{"osd":6,"uuid":"64d858d6-a8b3-4f65-a187-af9ab1ccee0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6819","nonce":1396972394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6821","nonce":1396972394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6825","nonce":1396972394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6823","nonce":1396972394}]},"public_addr":"172.21.15.103:6819/1396972394","cluster_addr":"172.21.15.103:6821/1396972394","heartbeat_back_addr":"172.21.15.103:6825/1396972394","heartbeat_front_addr":"172.21.15.103:6823/1396972394","state":["exists","up"]},{"osd":7,"uuid":"d405d1c6-839c-4c92-8261-4adb3163c90f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6817","nonce":2196366765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6819","nonce":2196366765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6823","nonce":2196366765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6821","nonce":2196366765}]},"public_addr":"172.21.15.134:6817/2196366765","cluster_addr":"172.21.15.134:6819/2196366765","heartbeat_back_addr":"172.21.15.134:6823/2196366765","heartbeat_front_addr":"172.21.15.134:6821/2196366765","state":["exists","up"]},{"osd":8,"uuid":"4c452cd9-816c-4f77-9368-2cd4bbeee4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6817","nonce":189977465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6819","nonce":189977465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6823","nonce":189977465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6821","nonce":189977465}]},"public_addr":"172.21.15.174:6817/189977465","cluster_addr":"172.21.15.174:6819/189977465","heartbeat_back_addr":"172.21.15.174:6823/189977465","heartbeat_front_addr":"172.21.15.174:6821/189977465","state":["exists","up"]},{"osd":9,"uuid":"fb3662e0-2d8e-45c4-9119-6d08b35443da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6827","nonce":3161966413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6829","nonce":3161966413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6833","nonce":3161966413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6831","nonce":3161966413}]},"public_addr":"172.21.15.103:6827/3161966413","cluster_addr":"172.21.15.103:6829/3161966413","heartbeat_back_addr":"172.21.15.103:6833/3161966413","heartbeat_front_addr":"172.21.15.103:6831/3161966413","state":["exists","up"]},{"osd":10,"uuid":"e52c3cbe-a6af-441e-b423-b94f3a2da544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6825","nonce":2532758632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6827","nonce":2532758632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6831","nonce":2532758632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6829","nonce":2532758632}]},"public_addr":"172.21.15.134:6825/2532758632","cluster_addr":"172.21.15.134:6827/2532758632","heartbeat_back_addr":"172.21.15.134:6831/2532758632","heartbeat_front_addr":"172.21.15.134:6829/2532758632","state":["exists","up"]},{"osd":11,"uuid":"9040f3f8-3873-4dea-8716-dcda5811c8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6825","nonce":161916442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6827","nonce":161916442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6831","nonce":161916442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6829","nonce":161916442}]},"public_addr":"172.21.15.174:6825/161916442","cluster_addr":"172.21.15.174:6827/161916442","heartbeat_back_addr":"172.21.15.174:6831/161916442","heartbeat_front_addr":"172.21.15.174:6829/161916442","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:41:59.656854+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:24.797551+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:49.685497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:33.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:39.324983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:03.987310+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:28.747971+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:53.827780+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:18.554450+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:43.705691+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:08.401275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:33.502542+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2169657832":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/1596229922":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1642809745":"2024-09-20T06:40:16.780890+0000","172.21.15.103:6800/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/70266056":"2024-09-20T06:49:13.909393+0000","172.21.15.103:6801/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1496153385":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/471230312":"2024-09-20T06:49:13.909393+0000","172.21.15.103:6801/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/2309624637":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/3727259725":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/2590139910":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/2050139492":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/2867049580":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/2590139910":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/1074662744":"2024-09-20T06:49:13.909393+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:49:15.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.622+0000 7f48c23d8640 1 -- 172.21.15.103:0/2505819648 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48a40a3050 msgr2=0x7f48a40a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.622+0000 7f48c23d8640 1 --2- 172.21.15.103:0/2505819648 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48a40a3050 0x7f48a40a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f48ac006010 tx=0x7f48ac032dd0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.623+0000 7f48c23d8640 1 -- 172.21.15.103:0/2505819648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48bc102620 msgr2=0x7f48bc19f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.623+0000 7f48c23d8640 1 --2- 172.21.15.103:0/2505819648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48bc102620 0x7f48bc19f3e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f48b400cce0 tx=0x7f48b400a620 comp rx=0 tx=0).stop 2024-09-19T06:49:15.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.623+0000 7f48c23d8640 1 -- 172.21.15.103:0/2505819648 shutdown_connections 2024-09-19T06:49:15.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.623+0000 7f48c23d8640 1 --2- 172.21.15.103:0/2505819648 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48a40a3050 0x7f48a40a5510 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.623+0000 7f48c23d8640 1 --2- 172.21.15.103:0/2505819648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48bc102620 0x7f48bc19f3e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.623+0000 7f48c23d8640 1 --2- 172.21.15.103:0/2505819648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48bc101c60 0x7f48bc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.623+0000 7f48c23d8640 1 --2- 172.21.15.103:0/2505819648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48bc100f50 0x7f48bc197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.623+0000 7f48c23d8640 1 -- 172.21.15.103:0/2505819648 >> 172.21.15.103:0/2505819648 conn(0x7f48bc0fb950 msgr2=0x7f48bc0ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:15.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.623+0000 7f48c23d8640 1 -- 172.21.15.103:0/2505819648 shutdown_connections 2024-09-19T06:49:15.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.623+0000 7f48c23d8640 1 -- 172.21.15.103:0/2505819648 wait complete. 2024-09-19T06:49:15.636 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T06:42:52.050881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T06:47:21.192611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T06:47:23.643291+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T06:49:15.636 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T06:49:15.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.750+0000 7fc1b23c0640 1 -- 172.21.15.103:0/753047745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1ac104ee0 msgr2=0x7fc1ac1072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.750+0000 7fc1b23c0640 1 --2- 172.21.15.103:0/753047745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1ac104ee0 0x7fc1ac1072d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc194009900 tx=0x7fc19402a410 comp rx=0 tx=0).stop 2024-09-19T06:49:15.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.751+0000 7fc1b23c0640 1 -- 172.21.15.103:0/753047745 shutdown_connections 2024-09-19T06:49:15.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.751+0000 7fc1b23c0640 1 --2- 172.21.15.103:0/753047745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ac107810 0x7fc1ac109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.751+0000 7fc1b23c0640 1 --2- 172.21.15.103:0/753047745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1ac104ee0 0x7fc1ac1072d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.751+0000 7fc1b23c0640 1 --2- 172.21.15.103:0/753047745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1ac1025b0 0x7fc1ac1049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.751+0000 7fc1b23c0640 1 -- 172.21.15.103:0/753047745 >> 172.21.15.103:0/753047745 conn(0x7fc1ac0fc1b0 msgr2=0x7fc1ac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:15.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.751+0000 7fc1b23c0640 1 -- 172.21.15.103:0/753047745 shutdown_connections 2024-09-19T06:49:15.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.751+0000 7fc1b23c0640 1 -- 172.21.15.103:0/753047745 wait complete. 2024-09-19T06:49:15.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.751+0000 7fc1b23c0640 1 Processor -- start 2024-09-19T06:49:15.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.751+0000 7fc1b23c0640 1 -- start start 2024-09-19T06:49:15.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.751+0000 7fc1b23c0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1ac1025b0 0x7fc1ac197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.751+0000 7fc1b23c0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1ac104ee0 0x7fc1ac198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.751+0000 7fc1b23c0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ac107810 0x7fc1ac19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.751+0000 7fc1b23c0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac10d840 con 0x7fc1ac104ee0 2024-09-19T06:49:15.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.751+0000 7fc1b23c0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac10d6c0 con 0x7fc1ac1025b0 2024-09-19T06:49:15.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.751+0000 7fc1b23c0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac10d9c0 con 0x7fc1ac107810 2024-09-19T06:49:15.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.752+0000 7fc1a3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1ac104ee0 0x7fc1ac198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.752+0000 7fc1abfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1ac1025b0 0x7fc1ac197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.752+0000 7fc1abfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1ac1025b0 0x7fc1ac197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:60532/0 (socket says 172.21.15.103:60532) 2024-09-19T06:49:15.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.752+0000 7fc1abfff640 1 -- 172.21.15.103:0/356694969 learned_addr learned my addr 172.21.15.103:0/356694969 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:15.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.752+0000 7fc1b095d640 1 --2- 172.21.15.103:0/356694969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ac107810 0x7fc1ac19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.752+0000 7fc1abfff640 1 -- 172.21.15.103:0/356694969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ac107810 msgr2=0x7fc1ac19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.752+0000 7fc1abfff640 1 --2- 172.21.15.103:0/356694969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ac107810 0x7fc1ac19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.752+0000 7fc1abfff640 1 -- 172.21.15.103:0/356694969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1ac104ee0 msgr2=0x7fc1ac198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.752+0000 7fc1abfff640 1 --2- 172.21.15.103:0/356694969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1ac104ee0 0x7fc1ac198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.752+0000 7fc1abfff640 1 -- 172.21.15.103:0/356694969 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac19fb10 con 0x7fc1ac1025b0 2024-09-19T06:49:15.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.752+0000 7fc1abfff640 1 --2- 172.21.15.103:0/356694969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1ac1025b0 0x7fc1ac197ef0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00e9f0 tx=0x7fc19c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:15.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.752+0000 7fc1a9ffb640 1 -- 172.21.15.103:0/356694969 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00cd80 con 0x7fc1ac1025b0 2024-09-19T06:49:15.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.752+0000 7fc1a9ffb640 1 -- 172.21.15.103:0/356694969 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc19c010910 con 0x7fc1ac1025b0 2024-09-19T06:49:15.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.752+0000 7fc1b23c0640 1 -- 172.21.15.103:0/356694969 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ac19fe00 con 0x7fc1ac1025b0 2024-09-19T06:49:15.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.752+0000 7fc1a9ffb640 1 -- 172.21.15.103:0/356694969 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c0188f0 con 0x7fc1ac1025b0 2024-09-19T06:49:15.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.752+0000 7fc1b23c0640 1 -- 172.21.15.103:0/356694969 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ac1a0260 con 0x7fc1ac1025b0 2024-09-19T06:49:15.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.753+0000 7fc1b23c0640 1 -- 172.21.15.103:0/356694969 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ac0fdca0 con 0x7fc1ac1025b0 2024-09-19T06:49:15.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.755+0000 7fc1a9ffb640 1 -- 172.21.15.103:0/356694969 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 21) ==== 138167+0+0 (secure 0 0 0) 0x7fc19c010ac0 con 0x7fc1ac1025b0 2024-09-19T06:49:15.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.755+0000 7fc1a9ffb640 1 --2- 172.21.15.103:0/356694969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc1980a3050 0x7fc1980a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.755+0000 7fc1a9ffb640 1 -- 172.21.15.103:0/356694969 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc19c0cfd50 con 0x7fc1ac1025b0 2024-09-19T06:49:15.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.756+0000 7fc1a3fff640 1 --2- 172.21.15.103:0/356694969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc1980a3050 0x7fc1980a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.756+0000 7fc1a3fff640 1 --2- 172.21.15.103:0/356694969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc1980a3050 0x7fc1980a5510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc19402aa00 tx=0x7fc1940315b0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:15.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.758+0000 7fc1a9ffb640 1 -- 172.21.15.103:0/356694969 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc19c09c600 con 0x7fc1ac1025b0 2024-09-19T06:49:15.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.950+0000 7fc1a9ffb640 1 -- 172.21.15.103:0/356694969 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mgrmap(e 22) ==== 138527+0+0 (secure 0 0 0) 0x7fc19c098c70 con 0x7fc1ac1025b0 2024-09-19T06:49:15.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.960+0000 7fc1b23c0640 1 -- 172.21.15.103:0/356694969 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc1ac02da10 con 0x7fc1ac1025b0 2024-09-19T06:49:15.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.960+0000 7fc1a9ffb640 1 -- 172.21.15.103:0/356694969 <== mon.2 v2:172.21.15.134:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fc19c048540 con 0x7fc1ac1025b0 2024-09-19T06:49:15.961 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 1 2024-09-19T06:49:15.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.962+0000 7fc1b23c0640 1 -- 172.21.15.103:0/356694969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc1980a3050 msgr2=0x7fc1980a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.962+0000 7fc1b23c0640 1 --2- 172.21.15.103:0/356694969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc1980a3050 0x7fc1980a5510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc19402aa00 tx=0x7fc1940315b0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.962+0000 7fc1b23c0640 1 -- 172.21.15.103:0/356694969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1ac1025b0 msgr2=0x7fc1ac197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.962+0000 7fc1b23c0640 1 --2- 172.21.15.103:0/356694969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1ac1025b0 0x7fc1ac197ef0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00e9f0 tx=0x7fc19c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.962+0000 7fc1b23c0640 1 -- 172.21.15.103:0/356694969 shutdown_connections 2024-09-19T06:49:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.962+0000 7fc1b23c0640 1 --2- 172.21.15.103:0/356694969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc1980a3050 0x7fc1980a5510 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.962+0000 7fc1b23c0640 1 --2- 172.21.15.103:0/356694969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ac107810 0x7fc1ac19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.962+0000 7fc1b23c0640 1 --2- 172.21.15.103:0/356694969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1ac104ee0 0x7fc1ac198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.962+0000 7fc1b23c0640 1 --2- 172.21.15.103:0/356694969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1ac1025b0 0x7fc1ac197ef0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.962+0000 7fc1b23c0640 1 -- 172.21.15.103:0/356694969 >> 172.21.15.103:0/356694969 conn(0x7fc1ac0fc1b0 msgr2=0x7fc1ac102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.963+0000 7fc1b23c0640 1 -- 172.21.15.103:0/356694969 shutdown_connections 2024-09-19T06:49:15.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:15.963+0000 7fc1b23c0640 1 -- 172.21.15.103:0/356694969 wait complete. 2024-09-19T06:49:15.975 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T06:49:16.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.087+0000 7f352e7bd640 1 -- 172.21.15.103:0/1402531106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3528102580 msgr2=0x7f3528104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:16.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.087+0000 7f352e7bd640 1 --2- 172.21.15.103:0/1402531106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3528102580 0x7f3528104970 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3518009900 tx=0x7f351802a410 comp rx=0 tx=0).stop 2024-09-19T06:49:16.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.087+0000 7f352e7bd640 1 -- 172.21.15.103:0/1402531106 shutdown_connections 2024-09-19T06:49:16.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.087+0000 7f352e7bd640 1 --2- 172.21.15.103:0/1402531106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35281077e0 0x7f3528109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.087+0000 7f352e7bd640 1 --2- 172.21.15.103:0/1402531106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3528104eb0 0x7f35281072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.087+0000 7f352e7bd640 1 --2- 172.21.15.103:0/1402531106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3528102580 0x7f3528104970 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.087+0000 7f352e7bd640 1 -- 172.21.15.103:0/1402531106 >> 172.21.15.103:0/1402531106 conn(0x7f35280fc140 msgr2=0x7f35280fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:16.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.088+0000 7f352e7bd640 1 -- 172.21.15.103:0/1402531106 shutdown_connections 2024-09-19T06:49:16.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.088+0000 7f352e7bd640 1 -- 172.21.15.103:0/1402531106 wait complete. 2024-09-19T06:49:16.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.088+0000 7f352e7bd640 1 Processor -- start 2024-09-19T06:49:16.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.088+0000 7f352e7bd640 1 -- start start 2024-09-19T06:49:16.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.088+0000 7f352e7bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3528102580 0x7f3528197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:16.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.088+0000 7f352e7bd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3528104eb0 0x7f3528198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:16.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.088+0000 7f352e7bd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35281077e0 0x7f352819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:16.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.088+0000 7f352e7bd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f352810d7c0 con 0x7f3528104eb0 2024-09-19T06:49:16.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.088+0000 7f352e7bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f352810d640 con 0x7f3528102580 2024-09-19T06:49:16.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.088+0000 7f352e7bd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f352810d940 con 0x7f35281077e0 2024-09-19T06:49:16.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f35277fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3528104eb0 0x7f3528198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:16.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f3527fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3528102580 0x7f3528197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:16.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f3527fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3528102580 0x7f3528197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:60550/0 (socket says 172.21.15.103:60550) 2024-09-19T06:49:16.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f3527fff640 1 -- 172.21.15.103:0/2084928166 learned_addr learned my addr 172.21.15.103:0/2084928166 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:16.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f352cd5a640 1 --2- 172.21.15.103:0/2084928166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35281077e0 0x7f352819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:16.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f3527fff640 1 -- 172.21.15.103:0/2084928166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35281077e0 msgr2=0x7f352819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:16.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f3527fff640 1 --2- 172.21.15.103:0/2084928166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35281077e0 0x7f352819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f3527fff640 1 -- 172.21.15.103:0/2084928166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3528104eb0 msgr2=0x7f3528198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:16.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f3527fff640 1 --2- 172.21.15.103:0/2084928166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3528104eb0 0x7f3528198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f3527fff640 1 -- 172.21.15.103:0/2084928166 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352819faf0 con 0x7f3528102580 2024-09-19T06:49:16.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f35277fe640 1 --2- 172.21.15.103:0/2084928166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3528104eb0 0x7f3528198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:16.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f3527fff640 1 --2- 172.21.15.103:0/2084928166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3528102580 0x7f3528197ed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3518009230 tx=0x7f3518031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:16.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f35257fa640 1 -- 172.21.15.103:0/2084928166 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351803f030 con 0x7f3528102580 2024-09-19T06:49:16.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f352e7bd640 1 -- 172.21.15.103:0/2084928166 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352819fd80 con 0x7f3528102580 2024-09-19T06:49:16.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f35257fa640 1 -- 172.21.15.103:0/2084928166 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3518003ad0 con 0x7f3528102580 2024-09-19T06:49:16.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f35257fa640 1 -- 172.21.15.103:0/2084928166 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518035070 con 0x7f3528102580 2024-09-19T06:49:16.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.089+0000 7f352e7bd640 1 -- 172.21.15.103:0/2084928166 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35281a0200 con 0x7f3528102580 2024-09-19T06:49:16.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.091+0000 7f352e7bd640 1 -- 172.21.15.103:0/2084928166 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35280fdc30 con 0x7f3528102580 2024-09-19T06:49:16.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.092+0000 7f35257fa640 1 -- 172.21.15.103:0/2084928166 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 22) ==== 138527+0+0 (secure 0 0 0) 0x7f3518042030 con 0x7f3528102580 2024-09-19T06:49:16.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.093+0000 7f35257fa640 1 --2- 172.21.15.103:0/2084928166 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f35140a3450 0x7f35140a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:16.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.093+0000 7f35277fe640 1 --2- 172.21.15.103:0/2084928166 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f35140a3450 0x7f35140a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:16.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.093+0000 7f35257fa640 1 -- 172.21.15.103:0/2084928166 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f35180f03b0 con 0x7f3528102580 2024-09-19T06:49:16.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.093+0000 7f35277fe640 1 --2- 172.21.15.103:0/2084928166 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f35140a3450 0x7f35140a5910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f35281993e0 tx=0x7f3510005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:16.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.097+0000 7f35257fa640 1 -- 172.21.15.103:0/2084928166 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f35180bc5e0 con 0x7f3528102580 2024-09-19T06:49:16.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:15 smithi134 ceph-mon[35356]: mgrmap e21: x(active, since 1.02149s), standbys: y, z 2024-09-19T06:49:16.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:15 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1448457829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T06:49:16.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:15 smithi134 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:49:16.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:15 smithi134 ceph-mon[35356]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:49:16.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:15 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2505819648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:49:16.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.272+0000 7f352e7bd640 1 -- 172.21.15.103:0/2084928166 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3528062df0 con 0x7f3528102580 2024-09-19T06:49:16.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.273+0000 7f35257fa640 1 -- 172.21.15.103:0/2084928166 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7f35180c08e0 con 0x7f3528102580 2024-09-19T06:49:16.274 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 32 2024-09-19T06:49:16.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.275+0000 7f352e7bd640 1 -- 172.21.15.103:0/2084928166 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f35140a3450 msgr2=0x7f35140a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:16.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.275+0000 7f352e7bd640 1 --2- 172.21.15.103:0/2084928166 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f35140a3450 0x7f35140a5910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f35281993e0 tx=0x7f3510005f90 comp rx=0 tx=0).stop 2024-09-19T06:49:16.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.275+0000 7f352e7bd640 1 -- 172.21.15.103:0/2084928166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3528102580 msgr2=0x7f3528197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:16.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.275+0000 7f352e7bd640 1 --2- 172.21.15.103:0/2084928166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3528102580 0x7f3528197ed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3518009230 tx=0x7f3518031b60 comp rx=0 tx=0).stop 2024-09-19T06:49:16.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.275+0000 7f352e7bd640 1 -- 172.21.15.103:0/2084928166 shutdown_connections 2024-09-19T06:49:16.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.275+0000 7f352e7bd640 1 --2- 172.21.15.103:0/2084928166 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f35140a3450 0x7f35140a5910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.275+0000 7f352e7bd640 1 --2- 172.21.15.103:0/2084928166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35281077e0 0x7f352819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.275+0000 7f352e7bd640 1 --2- 172.21.15.103:0/2084928166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3528104eb0 0x7f3528198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.275+0000 7f352e7bd640 1 --2- 172.21.15.103:0/2084928166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3528102580 0x7f3528197ed0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.275+0000 7f352e7bd640 1 -- 172.21.15.103:0/2084928166 >> 172.21.15.103:0/2084928166 conn(0x7f35280fc140 msgr2=0x7f3528102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:16.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.275+0000 7f352e7bd640 1 -- 172.21.15.103:0/2084928166 shutdown_connections 2024-09-19T06:49:16.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.275+0000 7f352e7bd640 1 -- 172.21.15.103:0/2084928166 wait complete. 2024-09-19T06:49:16.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-mon[31158]: mgrmap e21: x(active, since 1.02149s), standbys: y, z 2024-09-19T06:49:16.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1448457829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T06:49:16.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-mon[31158]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:49:16.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-mon[31158]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:49:16.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:15 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2505819648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:49:16.287 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T06:49:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:15 smithi174 ceph-mon[35637]: mgrmap e21: x(active, since 1.02149s), standbys: y, z 2024-09-19T06:49:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:15 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1448457829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T06:49:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:15 smithi174 ceph-mon[35637]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T06:49:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:15 smithi174 ceph-mon[35637]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:49:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:15 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2505819648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:49:16.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.416+0000 7f83796d0640 1 -- 172.21.15.103:0/2474333609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83741025e0 msgr2=0x7f8374108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:16.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.416+0000 7f83796d0640 1 --2- 172.21.15.103:0/2474333609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83741025e0 0x7f8374108e70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f836c009ca0 tx=0x7f836c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.416+0000 7f83796d0640 1 -- 172.21.15.103:0/2474333609 shutdown_connections 2024-09-19T06:49:16.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.416+0000 7f83796d0640 1 --2- 172.21.15.103:0/2474333609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83741025e0 0x7f8374108e70 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.416+0000 7f83796d0640 1 --2- 172.21.15.103:0/2474333609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8374101c20 0x7f83741020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.416+0000 7f83796d0640 1 --2- 172.21.15.103:0/2474333609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8374100a30 0x7f8374100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.416+0000 7f83796d0640 1 -- 172.21.15.103:0/2474333609 >> 172.21.15.103:0/2474333609 conn(0x7f83740fc1b0 msgr2=0x7f83740fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:16.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.416+0000 7f83796d0640 1 -- 172.21.15.103:0/2474333609 shutdown_connections 2024-09-19T06:49:16.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.416+0000 7f83796d0640 1 -- 172.21.15.103:0/2474333609 wait complete. 2024-09-19T06:49:16.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.416+0000 7f83796d0640 1 Processor -- start 2024-09-19T06:49:16.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.416+0000 7f83796d0640 1 -- start start 2024-09-19T06:49:16.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f83796d0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8374100a30 0x7f837406aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:16.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f83796d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8374101c20 0x7f837419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:16.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f83796d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83741025e0 0x7f837419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:16.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f83796d0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f837410c6b0 con 0x7f83741025e0 2024-09-19T06:49:16.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f83796d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f837410c530 con 0x7f8374101c20 2024-09-19T06:49:16.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f83796d0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f837410c830 con 0x7f8374100a30 2024-09-19T06:49:16.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f83737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83741025e0 0x7f837419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:16.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f83737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83741025e0 0x7f837419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59446/0 (socket says 172.21.15.103:59446) 2024-09-19T06:49:16.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f8372ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8374100a30 0x7f837406aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:16.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f83737fe640 1 -- 172.21.15.103:0/1196593171 learned_addr learned my addr 172.21.15.103:0/1196593171 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:16.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f83727fc640 1 --2- 172.21.15.103:0/1196593171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8374101c20 0x7f837419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:16.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f8372ffd640 1 -- 172.21.15.103:0/1196593171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8374101c20 msgr2=0x7f837419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:16.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f8372ffd640 1 --2- 172.21.15.103:0/1196593171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8374101c20 0x7f837419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f8372ffd640 1 -- 172.21.15.103:0/1196593171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83741025e0 msgr2=0x7f837419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:16.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f8372ffd640 1 --2- 172.21.15.103:0/1196593171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83741025e0 0x7f837419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f8372ffd640 1 -- 172.21.15.103:0/1196593171 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837419fab0 con 0x7f8374100a30 2024-09-19T06:49:16.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f83737fe640 1 --2- 172.21.15.103:0/1196593171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83741025e0 0x7f837419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:16.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f8372ffd640 1 --2- 172.21.15.103:0/1196593171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8374100a30 0x7f837406aaf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f836400e9c0 tx=0x7f836400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:16.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f836bfff640 1 -- 172.21.15.103:0/1196593171 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836400ccc0 con 0x7f8374100a30 2024-09-19T06:49:16.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f83796d0640 1 -- 172.21.15.103:0/1196593171 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f837419fd40 con 0x7f8374100a30 2024-09-19T06:49:16.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f83796d0640 1 -- 172.21.15.103:0/1196593171 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83741a0220 con 0x7f8374100a30 2024-09-19T06:49:16.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.417+0000 7f836bfff640 1 -- 172.21.15.103:0/1196593171 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83640035f0 con 0x7f8374100a30 2024-09-19T06:49:16.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.418+0000 7f836bfff640 1 -- 172.21.15.103:0/1196593171 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836400ce60 con 0x7f8374100a30 2024-09-19T06:49:16.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.418+0000 7f83796d0640 1 -- 172.21.15.103:0/1196593171 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8374107dd0 con 0x7f8374100a30 2024-09-19T06:49:16.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.420+0000 7f836bfff640 1 -- 172.21.15.103:0/1196593171 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) ==== 138527+0+0 (secure 0 0 0) 0x7f8364003b30 con 0x7f8374100a30 2024-09-19T06:49:16.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.420+0000 7f836bfff640 1 --2- 172.21.15.103:0/1196593171 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83600a3450 0x7f83600a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:16.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.420+0000 7f83727fc640 1 --2- 172.21.15.103:0/1196593171 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83600a3450 0x7f83600a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:16.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.421+0000 7f836bfff640 1 -- 172.21.15.103:0/1196593171 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f83640d0460 con 0x7f8374100a30 2024-09-19T06:49:16.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.421+0000 7f83727fc640 1 --2- 172.21.15.103:0/1196593171 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83600a3450 0x7f83600a5910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f837406a860 tx=0x7f835c005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:16.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.423+0000 7f836bfff640 1 -- 172.21.15.103:0/1196593171 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f836409cce0 con 0x7f8374100a30 2024-09-19T06:49:16.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.603+0000 7f83796d0640 1 -- 172.21.15.103:0/1196593171 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f837402d920 con 0x7f8374100a30 2024-09-19T06:49:16.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.604+0000 7f836bfff640 1 -- 172.21.15.103:0/1196593171 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f836409c5e0 con 0x7f8374100a30 2024-09-19T06:49:16.605 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 64 2024-09-19T06:49:16.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.606+0000 7f83796d0640 1 -- 172.21.15.103:0/1196593171 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83600a3450 msgr2=0x7f83600a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:16.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.606+0000 7f83796d0640 1 --2- 172.21.15.103:0/1196593171 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83600a3450 0x7f83600a5910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f837406a860 tx=0x7f835c005f90 comp rx=0 tx=0).stop 2024-09-19T06:49:16.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.606+0000 7f83796d0640 1 -- 172.21.15.103:0/1196593171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8374100a30 msgr2=0x7f837406aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:16.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.606+0000 7f83796d0640 1 --2- 172.21.15.103:0/1196593171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8374100a30 0x7f837406aaf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f836400e9c0 tx=0x7f836400ee80 comp rx=0 tx=0).stop 2024-09-19T06:49:16.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.606+0000 7f83796d0640 1 -- 172.21.15.103:0/1196593171 shutdown_connections 2024-09-19T06:49:16.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.606+0000 7f83796d0640 1 --2- 172.21.15.103:0/1196593171 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83600a3450 0x7f83600a5910 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.606+0000 7f83796d0640 1 --2- 172.21.15.103:0/1196593171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83741025e0 0x7f837419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.606+0000 7f83796d0640 1 --2- 172.21.15.103:0/1196593171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8374101c20 0x7f837419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.606+0000 7f83796d0640 1 --2- 172.21.15.103:0/1196593171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8374100a30 0x7f837406aaf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:16.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.606+0000 7f83796d0640 1 -- 172.21.15.103:0/1196593171 >> 172.21.15.103:0/1196593171 conn(0x7f83740fc1b0 msgr2=0x7f8374104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:16.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.606+0000 7f83796d0640 1 -- 172.21.15.103:0/1196593171 shutdown_connections 2024-09-19T06:49:16.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:16.606+0000 7f83796d0640 1 -- 172.21.15.103:0/1196593171 wait complete. 2024-09-19T06:49:16.619 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-19T06:49:16.619 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:49:16.894 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:17.195 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: [19/Sep/2024:06:49:15] ENGINE Bus STARTING 2024-09-19T06:49:17.196 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: [19/Sep/2024:06:49:15] ENGINE Serving on https://172.21.15.103:7150 2024-09-19T06:49:17.196 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: [19/Sep/2024:06:49:15] ENGINE Error in HTTPServer.serve 2024-09-19T06:49:17.196 INFO:journalctl@ceph.mon.a.smithi103.stdout: Traceback (most recent call last): 2024-09-19T06:49:17.196 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:49:17.196 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:49:17.196 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:49:17.196 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._run(expiration_interval) 2024-09-19T06:49:17.196 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:49:17.196 INFO:journalctl@ceph.mon.a.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:49:17.196 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:49:17.196 INFO:journalctl@ceph.mon.a.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:49:17.196 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:49:17.197 INFO:journalctl@ceph.mon.a.smithi103.stdout: s = self.context.wrap_socket( 2024-09-19T06:49:17.197 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:49:17.197 INFO:journalctl@ceph.mon.a.smithi103.stdout: return self.sslsocket_class._create( 2024-09-19T06:49:17.197 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:49:17.197 INFO:journalctl@ceph.mon.a.smithi103.stdout: self.do_handshake() 2024-09-19T06:49:17.197 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:49:17.197 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._sslobj.do_handshake() 2024-09-19T06:49:17.197 INFO:journalctl@ceph.mon.a.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:49:17.197 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:49:17.197 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: [19/Sep/2024:06:49:15] ENGINE Serving on http://172.21.15.103:8765 2024-09-19T06:49:17.197 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: [19/Sep/2024:06:49:15] ENGINE Bus STARTED 2024-09-19T06:49:17.197 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:49:17.197 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-09-19T06:49:17.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/356694969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:49:17.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2084928166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T06:49:17.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1196593171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T06:49:17.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.199 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.199 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.199 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.199 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.199 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.199 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.199 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.199 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.199 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.199 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.199 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.199 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.200 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.200 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.200 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.200 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.200 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:17.200 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:16 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:17.234 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: [19/Sep/2024:06:49:15] ENGINE Bus STARTING 2024-09-19T06:49:17.234 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: [19/Sep/2024:06:49:15] ENGINE Serving on https://172.21.15.103:7150 2024-09-19T06:49:17.234 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: [19/Sep/2024:06:49:15] ENGINE Error in HTTPServer.serve 2024-09-19T06:49:17.234 INFO:journalctl@ceph.mon.c.smithi174.stdout: Traceback (most recent call last): 2024-09-19T06:49:17.235 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:49:17.235 INFO:journalctl@ceph.mon.c.smithi174.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:49:17.235 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:49:17.235 INFO:journalctl@ceph.mon.c.smithi174.stdout: self._run(expiration_interval) 2024-09-19T06:49:17.235 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:49:17.235 INFO:journalctl@ceph.mon.c.smithi174.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:49:17.235 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:49:17.235 INFO:journalctl@ceph.mon.c.smithi174.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:49:17.235 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:49:17.235 INFO:journalctl@ceph.mon.c.smithi174.stdout: s = self.context.wrap_socket( 2024-09-19T06:49:17.235 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:49:17.235 INFO:journalctl@ceph.mon.c.smithi174.stdout: return self.sslsocket_class._create( 2024-09-19T06:49:17.235 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:49:17.236 INFO:journalctl@ceph.mon.c.smithi174.stdout: self.do_handshake() 2024-09-19T06:49:17.236 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:49:17.236 INFO:journalctl@ceph.mon.c.smithi174.stdout: self._sslobj.do_handshake() 2024-09-19T06:49:17.236 INFO:journalctl@ceph.mon.c.smithi174.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:49:17.236 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:49:17.236 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: [19/Sep/2024:06:49:15] ENGINE Serving on http://172.21.15.103:8765 2024-09-19T06:49:17.236 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: [19/Sep/2024:06:49:15] ENGINE Bus STARTED 2024-09-19T06:49:17.236 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:49:17.236 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-09-19T06:49:17.236 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/356694969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:49:17.236 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.236 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.236 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.236 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.237 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.237 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.237 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2084928166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T06:49:17.237 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1196593171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T06:49:17.237 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.237 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.237 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.237 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.237 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.237 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.237 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.237 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.237 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.237 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.238 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.238 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.238 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.238 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.238 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.238 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.238 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.238 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.238 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:17.238 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:16 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:17.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: [19/Sep/2024:06:49:15] ENGINE Bus STARTING 2024-09-19T06:49:17.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: [19/Sep/2024:06:49:15] ENGINE Serving on https://172.21.15.103:7150 2024-09-19T06:49:17.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: [19/Sep/2024:06:49:15] ENGINE Error in HTTPServer.serve 2024-09-19T06:49:17.438 INFO:journalctl@ceph.mon.b.smithi134.stdout: Traceback (most recent call last): 2024-09-19T06:49:17.438 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:49:17.438 INFO:journalctl@ceph.mon.b.smithi134.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:49:17.438 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:49:17.438 INFO:journalctl@ceph.mon.b.smithi134.stdout: self._run(expiration_interval) 2024-09-19T06:49:17.438 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:49:17.438 INFO:journalctl@ceph.mon.b.smithi134.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:49:17.438 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:49:17.438 INFO:journalctl@ceph.mon.b.smithi134.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:49:17.438 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:49:17.438 INFO:journalctl@ceph.mon.b.smithi134.stdout: s = self.context.wrap_socket( 2024-09-19T06:49:17.438 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:49:17.438 INFO:journalctl@ceph.mon.b.smithi134.stdout: return self.sslsocket_class._create( 2024-09-19T06:49:17.439 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:49:17.439 INFO:journalctl@ceph.mon.b.smithi134.stdout: self.do_handshake() 2024-09-19T06:49:17.439 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:49:17.439 INFO:journalctl@ceph.mon.b.smithi134.stdout: self._sslobj.do_handshake() 2024-09-19T06:49:17.439 INFO:journalctl@ceph.mon.b.smithi134.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:49:17.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:49:17.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: [19/Sep/2024:06:49:15] ENGINE Serving on http://172.21.15.103:8765 2024-09-19T06:49:17.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: [19/Sep/2024:06:49:15] ENGINE Bus STARTED 2024-09-19T06:49:17.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:49:17.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-09-19T06:49:17.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/356694969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:49:17.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2084928166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T06:49:17.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1196593171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T06:49:17.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.411+0000 7fc3c4a55640 1 -- 172.21.15.103:0/4089466455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3c01009d0 msgr2=0x7fc3c0100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:17.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.411+0000 7fc3c4a55640 1 --2- 172.21.15.103:0/4089466455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3c01009d0 0x7fc3c0100db0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0009900 tx=0x7fc3b002a410 comp rx=0 tx=0).stop 2024-09-19T06:49:17.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.411+0000 7fc3c4a55640 1 -- 172.21.15.103:0/4089466455 shutdown_connections 2024-09-19T06:49:17.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.411+0000 7fc3c4a55640 1 --2- 172.21.15.103:0/4089466455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3c010e4f0 0x7fc3c0110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.411+0000 7fc3c4a55640 1 --2- 172.21.15.103:0/4089466455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3c0101380 0x7fc3c010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.411+0000 7fc3c4a55640 1 --2- 172.21.15.103:0/4089466455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3c01009d0 0x7fc3c0100db0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.411+0000 7fc3c4a55640 1 -- 172.21.15.103:0/4089466455 >> 172.21.15.103:0/4089466455 conn(0x7fc3c00fc7d0 msgr2=0x7fc3c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:17.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.411+0000 7fc3c4a55640 1 -- 172.21.15.103:0/4089466455 shutdown_connections 2024-09-19T06:49:17.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.411+0000 7fc3c4a55640 1 -- 172.21.15.103:0/4089466455 wait complete. 2024-09-19T06:49:17.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.412+0000 7fc3c4a55640 1 Processor -- start 2024-09-19T06:49:17.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.412+0000 7fc3c4a55640 1 -- start start 2024-09-19T06:49:17.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.412+0000 7fc3c4a55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3c01009d0 0x7fc3c01a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.412+0000 7fc3c4a55640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3c0101380 0x7fc3c01a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.412+0000 7fc3c4a55640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3c010e4f0 0x7fc3c019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.412+0000 7fc3c4a55640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c0114270 con 0x7fc3c01009d0 2024-09-19T06:49:17.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.412+0000 7fc3c4a55640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c01140f0 con 0x7fc3c010e4f0 2024-09-19T06:49:17.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.412+0000 7fc3c4a55640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c01143f0 con 0x7fc3c0101380 2024-09-19T06:49:17.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.412+0000 7fc3be59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3c01009d0 0x7fc3c01a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.412+0000 7fc3be59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3c01009d0 0x7fc3c01a0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42336/0 (socket says 172.21.15.103:42336) 2024-09-19T06:49:17.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.412+0000 7fc3be59c640 1 -- 172.21.15.103:0/4242397 learned_addr learned my addr 172.21.15.103:0/4242397 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:17.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.413+0000 7fc3bdd9b640 1 --2- 172.21.15.103:0/4242397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3c0101380 0x7fc3c01a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.413+0000 7fc3bed9d640 1 --2- 172.21.15.103:0/4242397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3c010e4f0 0x7fc3c019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.413+0000 7fc3be59c640 1 -- 172.21.15.103:0/4242397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3c0101380 msgr2=0x7fc3c01a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.413+0000 7fc3be59c640 1 --2- 172.21.15.103:0/4242397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3c0101380 0x7fc3c01a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.413+0000 7fc3be59c640 1 -- 172.21.15.103:0/4242397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3c010e4f0 msgr2=0x7fc3c019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.413+0000 7fc3be59c640 1 --2- 172.21.15.103:0/4242397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3c010e4f0 0x7fc3c019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.413+0000 7fc3be59c640 1 -- 172.21.15.103:0/4242397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c019aeb0 con 0x7fc3c01009d0 2024-09-19T06:49:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.413+0000 7fc3bed9d640 1 --2- 172.21.15.103:0/4242397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3c010e4f0 0x7fc3c019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.413+0000 7fc3be59c640 1 --2- 172.21.15.103:0/4242397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3c01009d0 0x7fc3c01a0520 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0006010 tx=0x7fc3b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.413+0000 7fc3af7fe640 1 -- 172.21.15.103:0/4242397 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b003f030 con 0x7fc3c01009d0 2024-09-19T06:49:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.413+0000 7fc3c4a55640 1 -- 172.21.15.103:0/4242397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c019b140 con 0x7fc3c01009d0 2024-09-19T06:49:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.413+0000 7fc3af7fe640 1 -- 172.21.15.103:0/4242397 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3b0003ce0 con 0x7fc3c01009d0 2024-09-19T06:49:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.413+0000 7fc3af7fe640 1 -- 172.21.15.103:0/4242397 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0035070 con 0x7fc3c01009d0 2024-09-19T06:49:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.413+0000 7fc3c4a55640 1 -- 172.21.15.103:0/4242397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c01a7340 con 0x7fc3c01009d0 2024-09-19T06:49:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.414+0000 7fc3c4a55640 1 -- 172.21.15.103:0/4242397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c006a7d0 con 0x7fc3c01009d0 2024-09-19T06:49:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.416+0000 7fc3af7fe640 1 -- 172.21.15.103:0/4242397 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 22) ==== 138527+0+0 (secure 0 0 0) 0x7fc3b0042030 con 0x7fc3c01009d0 2024-09-19T06:49:17.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.416+0000 7fc3af7fe640 1 --2- 172.21.15.103:0/4242397 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3a80a3440 0x7fc3a80a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.416+0000 7fc3bdd9b640 1 --2- 172.21.15.103:0/4242397 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3a80a3440 0x7fc3a80a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.416+0000 7fc3af7fe640 1 -- 172.21.15.103:0/4242397 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc3b00f05a0 con 0x7fc3c01009d0 2024-09-19T06:49:17.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.417+0000 7fc3bdd9b640 1 --2- 172.21.15.103:0/4242397 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3a80a3440 0x7fc3a80a5900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0102520 tx=0x7fc3b4005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:17.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.420+0000 7fc3af7fe640 1 -- 172.21.15.103:0/4242397 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc3b00bc890 con 0x7fc3c01009d0 2024-09-19T06:49:17.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:17.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:49:17.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:17.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:16 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:17.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.690+0000 7fc3c4a55640 1 -- 172.21.15.103:0/4242397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3c0100db0 con 0x7fc3c01009d0 2024-09-19T06:49:17.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.691+0000 7fc3af7fe640 1 -- 172.21.15.103:0/4242397 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fc3c0100db0 con 0x7fc3c01009d0 2024-09-19T06:49:17.692 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:49:17.693 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:49:17.693 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:49:17.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.693+0000 7fc3c4a55640 1 -- 172.21.15.103:0/4242397 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3a80a3440 msgr2=0x7fc3a80a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:17.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.693+0000 7fc3c4a55640 1 --2- 172.21.15.103:0/4242397 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3a80a3440 0x7fc3a80a5900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0102520 tx=0x7fc3b4005f90 comp rx=0 tx=0).stop 2024-09-19T06:49:17.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.693+0000 7fc3c4a55640 1 -- 172.21.15.103:0/4242397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3c01009d0 msgr2=0x7fc3c01a0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:17.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.693+0000 7fc3c4a55640 1 --2- 172.21.15.103:0/4242397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3c01009d0 0x7fc3c01a0520 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0006010 tx=0x7fc3b00042f0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.694+0000 7fc3c4a55640 1 -- 172.21.15.103:0/4242397 shutdown_connections 2024-09-19T06:49:17.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.694+0000 7fc3c4a55640 1 --2- 172.21.15.103:0/4242397 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3a80a3440 0x7fc3a80a5900 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.694+0000 7fc3c4a55640 1 --2- 172.21.15.103:0/4242397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3c010e4f0 0x7fc3c019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.694+0000 7fc3c4a55640 1 --2- 172.21.15.103:0/4242397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3c0101380 0x7fc3c01a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.694+0000 7fc3c4a55640 1 --2- 172.21.15.103:0/4242397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3c01009d0 0x7fc3c01a0520 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.694+0000 7fc3c4a55640 1 -- 172.21.15.103:0/4242397 >> 172.21.15.103:0/4242397 conn(0x7fc3c00fc7d0 msgr2=0x7fc3c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:17.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.694+0000 7fc3c4a55640 1 -- 172.21.15.103:0/4242397 shutdown_connections 2024-09-19T06:49:17.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:17.694+0000 7fc3c4a55640 1 -- 172.21.15.103:0/4242397 wait complete. 2024-09-19T06:49:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T06:49:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: Unable to set osd_memory_target on smithi174 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T06:49:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-19T06:49:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: Unable to set osd_memory_target on smithi134 to 184595353: error parsing value: Value '184595353' is below minimum 939524096 2024-09-19T06:49:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T06:49:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T06:49:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-19T06:49:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-19T06:49:18.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T06:49:18.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/4242397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:18.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:18.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:18.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:18.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:17 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T06:49:18.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: Unable to set osd_memory_target on smithi174 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T06:49:18.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-19T06:49:18.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: Unable to set osd_memory_target on smithi134 to 184595353: error parsing value: Value '184595353' is below minimum 939524096 2024-09-19T06:49:18.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T06:49:18.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T06:49:18.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-19T06:49:18.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-19T06:49:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T06:49:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/4242397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:17 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.438 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-19T06:49:18.438 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-19T06:49:18.438 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T06:49:18.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T06:49:18.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: Unable to set osd_memory_target on smithi174 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T06:49:18.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-19T06:49:18.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: Unable to set osd_memory_target on smithi134 to 184595353: error parsing value: Value '184595353' is below minimum 939524096 2024-09-19T06:49:18.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T06:49:18.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T06:49:18.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-19T06:49:18.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-19T06:49:18.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T06:49:18.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/4242397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:18.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:18.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:18.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:18.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:17 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:18.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.541+0000 7fc718ae4640 1 -- 172.21.15.103:0/626783596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc714100040 msgr2=0x7fc7141004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.541+0000 7fc718ae4640 1 --2- 172.21.15.103:0/626783596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc714100040 0x7fc7141004a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc708009900 tx=0x7fc70802a410 comp rx=0 tx=0).stop 2024-09-19T06:49:18.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.541+0000 7fc718ae4640 1 -- 172.21.15.103:0/626783596 shutdown_connections 2024-09-19T06:49:18.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.541+0000 7fc718ae4640 1 --2- 172.21.15.103:0/626783596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7141009e0 0x7fc71410ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.541+0000 7fc718ae4640 1 --2- 172.21.15.103:0/626783596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc714100040 0x7fc7141004a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.541+0000 7fc718ae4640 1 --2- 172.21.15.103:0/626783596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc714107c50 0x7fc714108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.541+0000 7fc718ae4640 1 -- 172.21.15.103:0/626783596 >> 172.21.15.103:0/626783596 conn(0x7fc7140771e0 msgr2=0x7fc7140fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:18.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.541+0000 7fc718ae4640 1 -- 172.21.15.103:0/626783596 shutdown_connections 2024-09-19T06:49:18.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.541+0000 7fc718ae4640 1 -- 172.21.15.103:0/626783596 wait complete. 2024-09-19T06:49:18.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.542+0000 7fc718ae4640 1 Processor -- start 2024-09-19T06:49:18.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.542+0000 7fc718ae4640 1 -- start start 2024-09-19T06:49:18.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.542+0000 7fc718ae4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc714100040 0x7fc714198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.542+0000 7fc718ae4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7141009e0 0x7fc714198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.542+0000 7fc718ae4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc714107c50 0x7fc71419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.542+0000 7fc718ae4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc714111fd0 con 0x7fc7141009e0 2024-09-19T06:49:18.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.542+0000 7fc718ae4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc714111e50 con 0x7fc714100040 2024-09-19T06:49:18.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.542+0000 7fc718ae4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc714112150 con 0x7fc714107c50 2024-09-19T06:49:18.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.542+0000 7fc711d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7141009e0 0x7fc714198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.542+0000 7fc711d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7141009e0 0x7fc714198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42358/0 (socket says 172.21.15.103:42358) 2024-09-19T06:49:18.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.542+0000 7fc711d9b640 1 -- 172.21.15.103:0/2532468099 learned_addr learned my addr 172.21.15.103:0/2532468099 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:18.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.542+0000 7fc71259c640 1 --2- 172.21.15.103:0/2532468099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc714100040 0x7fc714198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.542+0000 7fc711d9b640 1 -- 172.21.15.103:0/2532468099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc714107c50 msgr2=0x7fc71419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.542+0000 7fc712d9d640 1 --2- 172.21.15.103:0/2532468099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc714107c50 0x7fc71419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.542+0000 7fc711d9b640 1 --2- 172.21.15.103:0/2532468099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc714107c50 0x7fc71419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.543+0000 7fc711d9b640 1 -- 172.21.15.103:0/2532468099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc714100040 msgr2=0x7fc714198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.543+0000 7fc711d9b640 1 --2- 172.21.15.103:0/2532468099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc714100040 0x7fc714198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.543+0000 7fc711d9b640 1 -- 172.21.15.103:0/2532468099 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71419d530 con 0x7fc7141009e0 2024-09-19T06:49:18.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.543+0000 7fc712d9d640 1 --2- 172.21.15.103:0/2532468099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc714107c50 0x7fc71419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:18.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.543+0000 7fc711d9b640 1 --2- 172.21.15.103:0/2532468099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7141009e0 0x7fc714198a80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc708006010 tx=0x7fc7080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:18.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.543+0000 7fc7037fe640 1 -- 172.21.15.103:0/2532468099 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70803f030 con 0x7fc7141009e0 2024-09-19T06:49:18.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.543+0000 7fc718ae4640 1 -- 172.21.15.103:0/2532468099 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71419d7c0 con 0x7fc7141009e0 2024-09-19T06:49:18.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.543+0000 7fc7037fe640 1 -- 172.21.15.103:0/2532468099 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc708003ce0 con 0x7fc7141009e0 2024-09-19T06:49:18.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.543+0000 7fc7037fe640 1 -- 172.21.15.103:0/2532468099 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708035070 con 0x7fc7141009e0 2024-09-19T06:49:18.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.543+0000 7fc718ae4640 1 -- 172.21.15.103:0/2532468099 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7141a50b0 con 0x7fc7141009e0 2024-09-19T06:49:18.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.544+0000 7fc718ae4640 1 -- 172.21.15.103:0/2532468099 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7141004a0 con 0x7fc7141009e0 2024-09-19T06:49:18.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.545+0000 7fc7037fe640 1 -- 172.21.15.103:0/2532468099 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 22) ==== 138527+0+0 (secure 0 0 0) 0x7fc708042030 con 0x7fc7141009e0 2024-09-19T06:49:18.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.546+0000 7fc7037fe640 1 --2- 172.21.15.103:0/2532468099 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc6fc0a3450 0x7fc6fc0a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.546+0000 7fc7037fe640 1 -- 172.21.15.103:0/2532468099 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc7080f0310 con 0x7fc7141009e0 2024-09-19T06:49:18.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.546+0000 7fc71259c640 1 --2- 172.21.15.103:0/2532468099 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc6fc0a3450 0x7fc6fc0a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.546+0000 7fc71259c640 1 --2- 172.21.15.103:0/2532468099 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc6fc0a3450 0x7fc6fc0a5910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc704009990 tx=0x7fc704009110 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:18.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.550+0000 7fc7037fe640 1 -- 172.21.15.103:0/2532468099 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc7080bcb90 con 0x7fc7141009e0 2024-09-19T06:49:18.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.714+0000 7fc718ae4640 1 -- 172.21.15.103:0/2532468099 --> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc714062630 con 0x7fc6fc0a3450 2024-09-19T06:49:18.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.721+0000 7fc7037fe640 1 -- 172.21.15.103:0/2532468099 <== mgr.15210 v2:172.21.15.103:6800/82882809 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257913 (secure 0 0 0) 0x7fc714062630 con 0x7fc6fc0a3450 2024-09-19T06:49:18.725 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:49:18.726 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T06:49:18.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.724+0000 7fc718ae4640 1 -- 172.21.15.103:0/2532468099 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc6fc0a3450 msgr2=0x7fc6fc0a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.724+0000 7fc718ae4640 1 --2- 172.21.15.103:0/2532468099 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc6fc0a3450 0x7fc6fc0a5910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc704009990 tx=0x7fc704009110 comp rx=0 tx=0).stop 2024-09-19T06:49:18.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.724+0000 7fc718ae4640 1 -- 172.21.15.103:0/2532468099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7141009e0 msgr2=0x7fc714198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.724+0000 7fc718ae4640 1 --2- 172.21.15.103:0/2532468099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7141009e0 0x7fc714198a80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc708006010 tx=0x7fc7080042f0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.724+0000 7fc718ae4640 1 -- 172.21.15.103:0/2532468099 shutdown_connections 2024-09-19T06:49:18.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.724+0000 7fc718ae4640 1 --2- 172.21.15.103:0/2532468099 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc6fc0a3450 0x7fc6fc0a5910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.724+0000 7fc718ae4640 1 --2- 172.21.15.103:0/2532468099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc714107c50 0x7fc71419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.724+0000 7fc718ae4640 1 --2- 172.21.15.103:0/2532468099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7141009e0 0x7fc714198a80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.724+0000 7fc718ae4640 1 --2- 172.21.15.103:0/2532468099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc714100040 0x7fc714198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.724+0000 7fc718ae4640 1 -- 172.21.15.103:0/2532468099 >> 172.21.15.103:0/2532468099 conn(0x7fc7140771e0 msgr2=0x7fc714104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:18.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.724+0000 7fc718ae4640 1 -- 172.21.15.103:0/2532468099 shutdown_connections 2024-09-19T06:49:18.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.724+0000 7fc718ae4640 1 -- 172.21.15.103:0/2532468099 wait complete. 2024-09-19T06:49:18.736 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T06:49:17.932591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191921,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1260,"num_read_kb":795,"num_write":562,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":1181610,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":943,"ondisk_log_size":943,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355580,"kb_used_data":14496,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124586244,"statfs":{"total":1151940427776,"available":1151576313856,"internally_reserved":0,"allocated":14843904,"data_stored":8931711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.991091"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960112+0000","last_change":"2024-09-19T06:47:24.853118+0000","last_active":"2024-09-19T06:49:13.960112+0000","last_peered":"2024-09-19T06:49:13.960112+0000","last_clean":"2024-09-19T06:49:13.960112+0000","last_became_active":"2024-09-19T06:47:24.852801+0000","last_became_peered":"2024-09-19T06:47:24.852801+0000","last_unstale":"2024-09-19T06:49:13.960112+0000","last_undegraded":"2024-09-19T06:49:13.960112+0000","last_fullsized":"2024-09-19T06:49:13.960112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:16.729103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.957024+0000","last_change":"2024-09-19T06:47:24.841095+0000","last_active":"2024-09-19T06:49:13.957024+0000","last_peered":"2024-09-19T06:49:13.957024+0000","last_clean":"2024-09-19T06:49:13.957024+0000","last_became_active":"2024-09-19T06:47:24.840774+0000","last_became_peered":"2024-09-19T06:47:24.840774+0000","last_unstale":"2024-09-19T06:49:13.957024+0000","last_undegraded":"2024-09-19T06:49:13.957024+0000","last_fullsized":"2024-09-19T06:49:13.957024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:27.262766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973374+0000","last_change":"2024-09-19T06:47:24.848444+0000","last_active":"2024-09-19T06:49:13.973374+0000","last_peered":"2024-09-19T06:49:13.973374+0000","last_clean":"2024-09-19T06:49:13.973374+0000","last_became_active":"2024-09-19T06:47:24.847354+0000","last_became_peered":"2024-09-19T06:47:24.847354+0000","last_unstale":"2024-09-19T06:49:13.973374+0000","last_undegraded":"2024-09-19T06:49:13.973374+0000","last_fullsized":"2024-09-19T06:49:13.973374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:44.901265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.949178+0000","last_change":"2024-09-19T06:47:24.851915+0000","last_active":"2024-09-19T06:49:13.949178+0000","last_peered":"2024-09-19T06:49:13.949178+0000","last_clean":"2024-09-19T06:49:13.949178+0000","last_became_active":"2024-09-19T06:47:24.851600+0000","last_became_peered":"2024-09-19T06:47:24.851600+0000","last_unstale":"2024-09-19T06:49:13.949178+0000","last_undegraded":"2024-09-19T06:49:13.949178+0000","last_fullsized":"2024-09-19T06:49:13.949178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:52.424288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.950906+0000","last_change":"2024-09-19T06:47:24.843184+0000","last_active":"2024-09-19T06:49:13.950906+0000","last_peered":"2024-09-19T06:49:13.950906+0000","last_clean":"2024-09-19T06:49:13.950906+0000","last_became_active":"2024-09-19T06:47:24.842857+0000","last_became_peered":"2024-09-19T06:47:24.842857+0000","last_unstale":"2024-09-19T06:49:13.950906+0000","last_undegraded":"2024-09-19T06:49:13.950906+0000","last_fullsized":"2024-09-19T06:49:13.950906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:06.608699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969815+0000","last_change":"2024-09-19T06:47:24.846631+0000","last_active":"2024-09-19T06:49:13.969815+0000","last_peered":"2024-09-19T06:49:13.969815+0000","last_clean":"2024-09-19T06:49:13.969815+0000","last_became_active":"2024-09-19T06:47:24.846271+0000","last_became_peered":"2024-09-19T06:47:24.846271+0000","last_unstale":"2024-09-19T06:49:13.969815+0000","last_undegraded":"2024-09-19T06:49:13.969815+0000","last_fullsized":"2024-09-19T06:49:13.969815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.272273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.967306+0000","last_change":"2024-09-19T06:47:24.840392+0000","last_active":"2024-09-19T06:49:13.967306+0000","last_peered":"2024-09-19T06:49:13.967306+0000","last_clean":"2024-09-19T06:49:13.967306+0000","last_became_active":"2024-09-19T06:47:24.840092+0000","last_became_peered":"2024-09-19T06:47:24.840092+0000","last_unstale":"2024-09-19T06:49:13.967306+0000","last_undegraded":"2024-09-19T06:49:13.967306+0000","last_fullsized":"2024-09-19T06:49:13.967306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:25.579096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.967567+0000","last_change":"2024-09-19T06:47:24.843336+0000","last_active":"2024-09-19T06:49:13.967567+0000","last_peered":"2024-09-19T06:49:13.967567+0000","last_clean":"2024-09-19T06:49:13.967567+0000","last_became_active":"2024-09-19T06:47:24.842016+0000","last_became_peered":"2024-09-19T06:47:24.842016+0000","last_unstale":"2024-09-19T06:49:13.967567+0000","last_undegraded":"2024-09-19T06:49:13.967567+0000","last_fullsized":"2024-09-19T06:49:13.967567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:49.478359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.968002+0000","last_change":"2024-09-19T06:47:24.847841+0000","last_active":"2024-09-19T06:49:13.968002+0000","last_peered":"2024-09-19T06:49:13.968002+0000","last_clean":"2024-09-19T06:49:13.968002+0000","last_became_active":"2024-09-19T06:47:24.847492+0000","last_became_peered":"2024-09-19T06:47:24.847492+0000","last_unstale":"2024-09-19T06:49:13.968002+0000","last_undegraded":"2024-09-19T06:49:13.968002+0000","last_fullsized":"2024-09-19T06:49:13.968002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:59.229022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955381+0000","last_change":"2024-09-19T06:47:24.843773+0000","last_active":"2024-09-19T06:49:13.955381+0000","last_peered":"2024-09-19T06:49:13.955381+0000","last_clean":"2024-09-19T06:49:13.955381+0000","last_became_active":"2024-09-19T06:47:24.843409+0000","last_became_peered":"2024-09-19T06:47:24.843409+0000","last_unstale":"2024-09-19T06:49:13.955381+0000","last_undegraded":"2024-09-19T06:49:13.955381+0000","last_fullsized":"2024-09-19T06:49:13.955381+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:22.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969613+0000","last_change":"2024-09-19T06:47:24.846637+0000","last_active":"2024-09-19T06:49:13.969613+0000","last_peered":"2024-09-19T06:49:13.969613+0000","last_clean":"2024-09-19T06:49:13.969613+0000","last_became_active":"2024-09-19T06:47:24.846337+0000","last_became_peered":"2024-09-19T06:47:24.846337+0000","last_unstale":"2024-09-19T06:49:13.969613+0000","last_undegraded":"2024-09-19T06:49:13.969613+0000","last_fullsized":"2024-09-19T06:49:13.969613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:23:35.869485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.946579+0000","last_change":"2024-09-19T06:47:24.851399+0000","last_active":"2024-09-19T06:49:13.946579+0000","last_peered":"2024-09-19T06:49:13.946579+0000","last_clean":"2024-09-19T06:49:13.946579+0000","last_became_active":"2024-09-19T06:47:24.851090+0000","last_became_peered":"2024-09-19T06:47:24.851090+0000","last_unstale":"2024-09-19T06:49:13.946579+0000","last_undegraded":"2024-09-19T06:49:13.946579+0000","last_fullsized":"2024-09-19T06:49:13.946579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:52.839956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970074+0000","last_change":"2024-09-19T06:47:24.833858+0000","last_active":"2024-09-19T06:49:13.970074+0000","last_peered":"2024-09-19T06:49:13.970074+0000","last_clean":"2024-09-19T06:49:13.970074+0000","last_became_active":"2024-09-19T06:47:24.833616+0000","last_became_peered":"2024-09-19T06:47:24.833616+0000","last_unstale":"2024-09-19T06:49:13.970074+0000","last_undegraded":"2024-09-19T06:49:13.970074+0000","last_fullsized":"2024-09-19T06:49:13.970074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:54.918144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966731+0000","last_change":"2024-09-19T06:47:24.841030+0000","last_active":"2024-09-19T06:49:13.966731+0000","last_peered":"2024-09-19T06:49:13.966731+0000","last_clean":"2024-09-19T06:49:13.966731+0000","last_became_active":"2024-09-19T06:47:24.839631+0000","last_became_peered":"2024-09-19T06:47:24.839631+0000","last_unstale":"2024-09-19T06:49:13.966731+0000","last_undegraded":"2024-09-19T06:49:13.966731+0000","last_fullsized":"2024-09-19T06:49:13.966731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:21.764812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970278+0000","last_change":"2024-09-19T06:47:24.849281+0000","last_active":"2024-09-19T06:49:13.970278+0000","last_peered":"2024-09-19T06:49:13.970278+0000","last_clean":"2024-09-19T06:49:13.970278+0000","last_became_active":"2024-09-19T06:47:24.848977+0000","last_became_peered":"2024-09-19T06:47:24.848977+0000","last_unstale":"2024-09-19T06:49:13.970278+0000","last_undegraded":"2024-09-19T06:49:13.970278+0000","last_fullsized":"2024-09-19T06:49:13.970278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:36.693758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973886+0000","last_change":"2024-09-19T06:47:24.843619+0000","last_active":"2024-09-19T06:49:13.973886+0000","last_peered":"2024-09-19T06:49:13.973886+0000","last_clean":"2024-09-19T06:49:13.973886+0000","last_became_active":"2024-09-19T06:47:24.843323+0000","last_became_peered":"2024-09-19T06:47:24.843323+0000","last_unstale":"2024-09-19T06:49:13.973886+0000","last_undegraded":"2024-09-19T06:49:13.973886+0000","last_fullsized":"2024-09-19T06:49:13.973886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:52.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960333+0000","last_change":"2024-09-19T06:47:24.853165+0000","last_active":"2024-09-19T06:49:13.960333+0000","last_peered":"2024-09-19T06:49:13.960333+0000","last_clean":"2024-09-19T06:49:13.960333+0000","last_became_active":"2024-09-19T06:47:24.852864+0000","last_became_peered":"2024-09-19T06:47:24.852864+0000","last_unstale":"2024-09-19T06:49:13.960333+0000","last_undegraded":"2024-09-19T06:49:13.960333+0000","last_fullsized":"2024-09-19T06:49:13.960333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:38.803478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.975021+0000","last_change":"2024-09-19T06:47:24.838151+0000","last_active":"2024-09-19T06:49:13.975021+0000","last_peered":"2024-09-19T06:49:13.975021+0000","last_clean":"2024-09-19T06:49:13.975021+0000","last_became_active":"2024-09-19T06:47:24.837855+0000","last_became_peered":"2024-09-19T06:47:24.837855+0000","last_unstale":"2024-09-19T06:49:13.975021+0000","last_undegraded":"2024-09-19T06:49:13.975021+0000","last_fullsized":"2024-09-19T06:49:13.975021+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:16.675557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.964466+0000","last_change":"2024-09-19T06:47:24.843941+0000","last_active":"2024-09-19T06:49:13.964466+0000","last_peered":"2024-09-19T06:49:13.964466+0000","last_clean":"2024-09-19T06:49:13.964466+0000","last_became_active":"2024-09-19T06:47:24.843551+0000","last_became_peered":"2024-09-19T06:47:24.843551+0000","last_unstale":"2024-09-19T06:49:13.964466+0000","last_undegraded":"2024-09-19T06:49:13.964466+0000","last_fullsized":"2024-09-19T06:49:13.964466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:00:49.388265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.954208+0000","last_change":"2024-09-19T06:47:40.836228+0000","last_active":"2024-09-19T06:49:13.954208+0000","last_peered":"2024-09-19T06:49:13.954208+0000","last_clean":"2024-09-19T06:49:13.954208+0000","last_became_active":"2024-09-19T06:47:40.835818+0000","last_became_peered":"2024-09-19T06:47:40.835818+0000","last_unstale":"2024-09-19T06:49:13.954208+0000","last_undegraded":"2024-09-19T06:49:13.954208+0000","last_fullsized":"2024-09-19T06:49:13.954208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:45.341160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972919+0000","last_change":"2024-09-19T06:47:24.843746+0000","last_active":"2024-09-19T06:49:13.972919+0000","last_peered":"2024-09-19T06:49:13.972919+0000","last_clean":"2024-09-19T06:49:13.972919+0000","last_became_active":"2024-09-19T06:47:24.841495+0000","last_became_peered":"2024-09-19T06:47:24.841495+0000","last_unstale":"2024-09-19T06:49:13.972919+0000","last_undegraded":"2024-09-19T06:49:13.972919+0000","last_fullsized":"2024-09-19T06:49:13.972919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:06:29.995595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.974565+0000","last_change":"2024-09-19T06:47:24.844747+0000","last_active":"2024-09-19T06:49:13.974565+0000","last_peered":"2024-09-19T06:49:13.974565+0000","last_clean":"2024-09-19T06:49:13.974565+0000","last_became_active":"2024-09-19T06:47:24.844425+0000","last_became_peered":"2024-09-19T06:47:24.844425+0000","last_unstale":"2024-09-19T06:49:13.974565+0000","last_undegraded":"2024-09-19T06:49:13.974565+0000","last_fullsized":"2024-09-19T06:49:13.974565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:10.309044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963556+0000","last_change":"2024-09-19T06:47:24.848227+0000","last_active":"2024-09-19T06:49:13.963556+0000","last_peered":"2024-09-19T06:49:13.963556+0000","last_clean":"2024-09-19T06:49:13.963556+0000","last_became_active":"2024-09-19T06:47:24.847914+0000","last_became_peered":"2024-09-19T06:47:24.847914+0000","last_unstale":"2024-09-19T06:49:13.963556+0000","last_undegraded":"2024-09-19T06:49:13.963556+0000","last_fullsized":"2024-09-19T06:49:13.963556+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:47.281613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.968012+0000","last_change":"2024-09-19T06:47:24.844275+0000","last_active":"2024-09-19T06:49:13.968012+0000","last_peered":"2024-09-19T06:49:13.968012+0000","last_clean":"2024-09-19T06:49:13.968012+0000","last_became_active":"2024-09-19T06:47:24.841442+0000","last_became_peered":"2024-09-19T06:47:24.841442+0000","last_unstale":"2024-09-19T06:49:13.968012+0000","last_undegraded":"2024-09-19T06:49:13.968012+0000","last_fullsized":"2024-09-19T06:49:13.968012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:30.198655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963003+0000","last_change":"2024-09-19T06:47:24.846301+0000","last_active":"2024-09-19T06:49:13.963003+0000","last_peered":"2024-09-19T06:49:13.963003+0000","last_clean":"2024-09-19T06:49:13.963003+0000","last_became_active":"2024-09-19T06:47:24.845970+0000","last_became_peered":"2024-09-19T06:47:24.845970+0000","last_unstale":"2024-09-19T06:49:13.963003+0000","last_undegraded":"2024-09-19T06:49:13.963003+0000","last_fullsized":"2024-09-19T06:49:13.963003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:13.245536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973777+0000","last_change":"2024-09-19T06:47:24.844444+0000","last_active":"2024-09-19T06:49:13.973777+0000","last_peered":"2024-09-19T06:49:13.973777+0000","last_clean":"2024-09-19T06:49:13.973777+0000","last_became_active":"2024-09-19T06:47:24.844141+0000","last_became_peered":"2024-09-19T06:47:24.844141+0000","last_unstale":"2024-09-19T06:49:13.973777+0000","last_undegraded":"2024-09-19T06:49:13.973777+0000","last_fullsized":"2024-09-19T06:49:13.973777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:44.739242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955255+0000","last_change":"2024-09-19T06:47:24.845633+0000","last_active":"2024-09-19T06:49:13.955255+0000","last_peered":"2024-09-19T06:49:13.955255+0000","last_clean":"2024-09-19T06:49:13.955255+0000","last_became_active":"2024-09-19T06:47:24.845319+0000","last_became_peered":"2024-09-19T06:47:24.845319+0000","last_unstale":"2024-09-19T06:49:13.955255+0000","last_undegraded":"2024-09-19T06:49:13.955255+0000","last_fullsized":"2024-09-19T06:49:13.955255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:00.820006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959353+0000","last_change":"2024-09-19T06:47:24.846457+0000","last_active":"2024-09-19T06:49:13.959353+0000","last_peered":"2024-09-19T06:49:13.959353+0000","last_clean":"2024-09-19T06:49:13.959353+0000","last_became_active":"2024-09-19T06:47:24.846130+0000","last_became_peered":"2024-09-19T06:47:24.846130+0000","last_unstale":"2024-09-19T06:49:13.959353+0000","last_undegraded":"2024-09-19T06:49:13.959353+0000","last_fullsized":"2024-09-19T06:49:13.959353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:42.179145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969751+0000","last_change":"2024-09-19T06:47:24.843141+0000","last_active":"2024-09-19T06:49:13.969751+0000","last_peered":"2024-09-19T06:49:13.969751+0000","last_clean":"2024-09-19T06:49:13.969751+0000","last_became_active":"2024-09-19T06:47:24.842810+0000","last_became_peered":"2024-09-19T06:47:24.842810+0000","last_unstale":"2024-09-19T06:49:13.969751+0000","last_undegraded":"2024-09-19T06:49:13.969751+0000","last_fullsized":"2024-09-19T06:49:13.969751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:43.440453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.965342+0000","last_change":"2024-09-19T06:47:24.846171+0000","last_active":"2024-09-19T06:49:13.965342+0000","last_peered":"2024-09-19T06:49:13.965342+0000","last_clean":"2024-09-19T06:49:13.965342+0000","last_became_active":"2024-09-19T06:47:24.845799+0000","last_became_peered":"2024-09-19T06:47:24.845799+0000","last_unstale":"2024-09-19T06:49:13.965342+0000","last_undegraded":"2024-09-19T06:49:13.965342+0000","last_fullsized":"2024-09-19T06:49:13.965342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:23.103061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972106+0000","last_change":"2024-09-19T06:47:24.848723+0000","last_active":"2024-09-19T06:49:13.972106+0000","last_peered":"2024-09-19T06:49:13.972106+0000","last_clean":"2024-09-19T06:49:13.972106+0000","last_became_active":"2024-09-19T06:47:24.848367+0000","last_became_peered":"2024-09-19T06:47:24.848367+0000","last_unstale":"2024-09-19T06:49:13.972106+0000","last_undegraded":"2024-09-19T06:49:13.972106+0000","last_fullsized":"2024-09-19T06:49:13.972106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:15.985293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960543+0000","last_change":"2024-09-19T06:47:24.841379+0000","last_active":"2024-09-19T06:49:13.960543+0000","last_peered":"2024-09-19T06:49:13.960543+0000","last_clean":"2024-09-19T06:49:13.960543+0000","last_became_active":"2024-09-19T06:47:24.841041+0000","last_became_peered":"2024-09-19T06:47:24.841041+0000","last_unstale":"2024-09-19T06:49:13.960543+0000","last_undegraded":"2024-09-19T06:49:13.960543+0000","last_fullsized":"2024-09-19T06:49:13.960543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:11.799022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955643+0000","last_change":"2024-09-19T06:47:24.849508+0000","last_active":"2024-09-19T06:49:13.955643+0000","last_peered":"2024-09-19T06:49:13.955643+0000","last_clean":"2024-09-19T06:49:13.955643+0000","last_became_active":"2024-09-19T06:47:24.849196+0000","last_became_peered":"2024-09-19T06:47:24.849196+0000","last_unstale":"2024-09-19T06:49:13.955643+0000","last_undegraded":"2024-09-19T06:49:13.955643+0000","last_fullsized":"2024-09-19T06:49:13.955643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:35.079720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":132,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.604832+0000","last_change":"2024-09-19T06:47:22.829848+0000","last_active":"2024-09-19T06:49:14.604832+0000","last_peered":"2024-09-19T06:49:14.604832+0000","last_clean":"2024-09-19T06:49:14.604832+0000","last_became_active":"2024-09-19T06:47:22.829511+0000","last_became_peered":"2024-09-19T06:47:22.829511+0000","last_unstale":"2024-09-19T06:49:14.604832+0000","last_undegraded":"2024-09-19T06:49:14.604832+0000","last_fullsized":"2024-09-19T06:49:14.604832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:32.665252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971999+0000","last_change":"2024-09-19T06:47:24.843245+0000","last_active":"2024-09-19T06:49:13.971999+0000","last_peered":"2024-09-19T06:49:13.971999+0000","last_clean":"2024-09-19T06:49:13.971999+0000","last_became_active":"2024-09-19T06:47:24.842885+0000","last_became_peered":"2024-09-19T06:47:24.842885+0000","last_unstale":"2024-09-19T06:49:13.971999+0000","last_undegraded":"2024-09-19T06:49:13.971999+0000","last_fullsized":"2024-09-19T06:49:13.971999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:26.229003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.926577+0000","last_change":"2024-09-19T06:47:22.831456+0000","last_active":"2024-09-19T06:49:13.926577+0000","last_peered":"2024-09-19T06:49:13.926577+0000","last_clean":"2024-09-19T06:49:13.926577+0000","last_became_active":"2024-09-19T06:47:22.831132+0000","last_became_peered":"2024-09-19T06:47:22.831132+0000","last_unstale":"2024-09-19T06:49:13.926577+0000","last_undegraded":"2024-09-19T06:49:13.926577+0000","last_fullsized":"2024-09-19T06:49:13.926577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:49.867713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932598+0000","last_change":"2024-09-19T06:47:22.827742+0000","last_active":"2024-09-19T06:49:13.932598+0000","last_peered":"2024-09-19T06:49:13.932598+0000","last_clean":"2024-09-19T06:49:13.932598+0000","last_became_active":"2024-09-19T06:47:22.827371+0000","last_became_peered":"2024-09-19T06:47:22.827371+0000","last_unstale":"2024-09-19T06:49:13.932598+0000","last_undegraded":"2024-09-19T06:49:13.932598+0000","last_fullsized":"2024-09-19T06:49:13.932598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:35.153129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963305+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:49:13.963305+0000","last_peered":"2024-09-19T06:49:13.963305+0000","last_clean":"2024-09-19T06:49:13.963305+0000","last_became_active":"2024-09-19T06:47:24.841693+0000","last_became_peered":"2024-09-19T06:47:24.841693+0000","last_unstale":"2024-09-19T06:49:13.963305+0000","last_undegraded":"2024-09-19T06:49:13.963305+0000","last_fullsized":"2024-09-19T06:49:13.963305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:27.828296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.917388+0000","last_change":"2024-09-19T06:47:22.832531+0000","last_active":"2024-09-19T06:49:13.917388+0000","last_peered":"2024-09-19T06:49:13.917388+0000","last_clean":"2024-09-19T06:49:13.917388+0000","last_became_active":"2024-09-19T06:47:22.832213+0000","last_became_peered":"2024-09-19T06:47:22.832213+0000","last_unstale":"2024-09-19T06:49:13.917388+0000","last_undegraded":"2024-09-19T06:49:13.917388+0000","last_fullsized":"2024-09-19T06:49:13.917388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:46:03.168406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970720+0000","last_change":"2024-09-19T06:47:24.837690+0000","last_active":"2024-09-19T06:49:13.970720+0000","last_peered":"2024-09-19T06:49:13.970720+0000","last_clean":"2024-09-19T06:49:13.970720+0000","last_became_active":"2024-09-19T06:47:24.837389+0000","last_became_peered":"2024-09-19T06:47:24.837389+0000","last_unstale":"2024-09-19T06:49:13.970720+0000","last_undegraded":"2024-09-19T06:49:13.970720+0000","last_fullsized":"2024-09-19T06:49:13.970720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:20.653832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":228,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.604318+0000","last_change":"2024-09-19T06:47:22.833808+0000","last_active":"2024-09-19T06:49:14.604318+0000","last_peered":"2024-09-19T06:49:14.604318+0000","last_clean":"2024-09-19T06:49:14.604318+0000","last_became_active":"2024-09-19T06:47:22.832468+0000","last_became_peered":"2024-09-19T06:47:22.832468+0000","last_unstale":"2024-09-19T06:49:14.604318+0000","last_undegraded":"2024-09-19T06:49:14.604318+0000","last_fullsized":"2024-09-19T06:49:14.604318+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:39:31.131321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":198,"num_read_kb":198,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973091+0000","last_change":"2024-09-19T06:47:24.840928+0000","last_active":"2024-09-19T06:49:13.973091+0000","last_peered":"2024-09-19T06:49:13.973091+0000","last_clean":"2024-09-19T06:49:13.973091+0000","last_became_active":"2024-09-19T06:47:24.839928+0000","last_became_peered":"2024-09-19T06:47:24.839928+0000","last_unstale":"2024-09-19T06:49:13.973091+0000","last_undegraded":"2024-09-19T06:49:13.973091+0000","last_fullsized":"2024-09-19T06:49:13.973091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:29.486003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.928845+0000","last_change":"2024-09-19T06:47:22.833310+0000","last_active":"2024-09-19T06:49:13.928845+0000","last_peered":"2024-09-19T06:49:13.928845+0000","last_clean":"2024-09-19T06:49:13.928845+0000","last_became_active":"2024-09-19T06:47:22.832976+0000","last_became_peered":"2024-09-19T06:47:22.832976+0000","last_unstale":"2024-09-19T06:49:13.928845+0000","last_undegraded":"2024-09-19T06:49:13.928845+0000","last_fullsized":"2024-09-19T06:49:13.928845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:47.604853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969853+0000","last_change":"2024-09-19T06:47:24.848687+0000","last_active":"2024-09-19T06:49:13.969853+0000","last_peered":"2024-09-19T06:49:13.969853+0000","last_clean":"2024-09-19T06:49:13.969853+0000","last_became_active":"2024-09-19T06:47:24.848352+0000","last_became_peered":"2024-09-19T06:47:24.848352+0000","last_unstale":"2024-09-19T06:49:13.969853+0000","last_undegraded":"2024-09-19T06:49:13.969853+0000","last_fullsized":"2024-09-19T06:49:13.969853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:54.459206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.917489+0000","last_change":"2024-09-19T06:47:22.851479+0000","last_active":"2024-09-19T06:49:13.917489+0000","last_peered":"2024-09-19T06:49:13.917489+0000","last_clean":"2024-09-19T06:49:13.917489+0000","last_became_active":"2024-09-19T06:47:22.851125+0000","last_became_peered":"2024-09-19T06:47:22.851125+0000","last_unstale":"2024-09-19T06:49:13.917489+0000","last_undegraded":"2024-09-19T06:49:13.917489+0000","last_fullsized":"2024-09-19T06:49:13.917489+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:42.954034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973229+0000","last_change":"2024-09-19T06:47:24.851007+0000","last_active":"2024-09-19T06:49:13.973229+0000","last_peered":"2024-09-19T06:49:13.973229+0000","last_clean":"2024-09-19T06:49:13.973229+0000","last_became_active":"2024-09-19T06:47:24.850465+0000","last_became_peered":"2024-09-19T06:47:24.850465+0000","last_unstale":"2024-09-19T06:49:13.973229+0000","last_undegraded":"2024-09-19T06:49:13.973229+0000","last_fullsized":"2024-09-19T06:49:13.973229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:10.824672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":268,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.604918+0000","last_change":"2024-09-19T06:47:22.834893+0000","last_active":"2024-09-19T06:49:14.604918+0000","last_peered":"2024-09-19T06:49:14.604918+0000","last_clean":"2024-09-19T06:49:14.604918+0000","last_became_active":"2024-09-19T06:47:22.834521+0000","last_became_peered":"2024-09-19T06:47:22.834521+0000","last_unstale":"2024-09-19T06:49:14.604918+0000","last_undegraded":"2024-09-19T06:49:14.604918+0000","last_fullsized":"2024-09-19T06:49:14.604918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:29.742284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66347,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":208,"num_read_kb":168,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.957411+0000","last_change":"2024-09-19T06:47:24.848191+0000","last_active":"2024-09-19T06:49:13.957411+0000","last_peered":"2024-09-19T06:49:13.957411+0000","last_clean":"2024-09-19T06:49:13.957411+0000","last_became_active":"2024-09-19T06:47:24.847889+0000","last_became_peered":"2024-09-19T06:47:24.847889+0000","last_unstale":"2024-09-19T06:49:13.957411+0000","last_undegraded":"2024-09-19T06:49:13.957411+0000","last_fullsized":"2024-09-19T06:49:13.957411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:32.574360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.924532+0000","last_change":"2024-09-19T06:47:22.832005+0000","last_active":"2024-09-19T06:49:13.924532+0000","last_peered":"2024-09-19T06:49:13.924532+0000","last_clean":"2024-09-19T06:49:13.924532+0000","last_became_active":"2024-09-19T06:47:22.831342+0000","last_became_peered":"2024-09-19T06:47:22.831342+0000","last_unstale":"2024-09-19T06:49:13.924532+0000","last_undegraded":"2024-09-19T06:49:13.924532+0000","last_fullsized":"2024-09-19T06:49:13.924532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:53.444748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.964849+0000","last_change":"2024-09-19T06:47:24.843328+0000","last_active":"2024-09-19T06:49:13.964849+0000","last_peered":"2024-09-19T06:49:13.964849+0000","last_clean":"2024-09-19T06:49:13.964849+0000","last_became_active":"2024-09-19T06:47:24.842344+0000","last_became_peered":"2024-09-19T06:47:24.842344+0000","last_unstale":"2024-09-19T06:49:13.964849+0000","last_undegraded":"2024-09-19T06:49:13.964849+0000","last_fullsized":"2024-09-19T06:49:13.964849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:11.247819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.926463+0000","last_change":"2024-09-19T06:47:40.844086+0000","last_active":"2024-09-19T06:49:13.926463+0000","last_peered":"2024-09-19T06:49:13.926463+0000","last_clean":"2024-09-19T06:49:13.926463+0000","last_became_active":"2024-09-19T06:47:40.835015+0000","last_became_peered":"2024-09-19T06:47:40.835015+0000","last_unstale":"2024-09-19T06:49:13.926463+0000","last_undegraded":"2024-09-19T06:49:13.926463+0000","last_fullsized":"2024-09-19T06:49:13.926463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:09.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972972+0000","last_change":"2024-09-19T06:47:40.836466+0000","last_active":"2024-09-19T06:49:13.972972+0000","last_peered":"2024-09-19T06:49:13.972972+0000","last_clean":"2024-09-19T06:49:13.972972+0000","last_became_active":"2024-09-19T06:47:40.836237+0000","last_became_peered":"2024-09-19T06:47:40.836237+0000","last_unstale":"2024-09-19T06:49:13.972972+0000","last_undegraded":"2024-09-19T06:49:13.972972+0000","last_fullsized":"2024-09-19T06:49:13.972972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:00.094685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'10","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.606091+0000","last_change":"2024-09-19T06:47:22.830359+0000","last_active":"2024-09-19T06:49:14.606091+0000","last_peered":"2024-09-19T06:49:14.606091+0000","last_clean":"2024-09-19T06:49:14.606091+0000","last_became_active":"2024-09-19T06:47:22.829960+0000","last_became_peered":"2024-09-19T06:47:22.829960+0000","last_unstale":"2024-09-19T06:49:14.606091+0000","last_undegraded":"2024-09-19T06:49:14.606091+0000","last_fullsized":"2024-09-19T06:49:14.606091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:55.998243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":6,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971872+0000","last_change":"2024-09-19T06:47:24.836668+0000","last_active":"2024-09-19T06:49:13.971872+0000","last_peered":"2024-09-19T06:49:13.971872+0000","last_clean":"2024-09-19T06:49:13.971872+0000","last_became_active":"2024-09-19T06:47:24.836393+0000","last_became_peered":"2024-09-19T06:47:24.836393+0000","last_unstale":"2024-09-19T06:49:13.971872+0000","last_undegraded":"2024-09-19T06:49:13.971872+0000","last_fullsized":"2024-09-19T06:49:13.971872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:26.248948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":66,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.931451+0000","last_change":"2024-09-19T06:47:40.858323+0000","last_active":"2024-09-19T06:49:13.931451+0000","last_peered":"2024-09-19T06:49:13.931451+0000","last_clean":"2024-09-19T06:49:13.931451+0000","last_became_active":"2024-09-19T06:47:40.840802+0000","last_became_peered":"2024-09-19T06:47:40.840802+0000","last_unstale":"2024-09-19T06:49:13.931451+0000","last_undegraded":"2024-09-19T06:49:13.931451+0000","last_fullsized":"2024-09-19T06:49:13.931451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:14.666916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970280+0000","last_change":"2024-09-19T06:47:24.843681+0000","last_active":"2024-09-19T06:49:13.970280+0000","last_peered":"2024-09-19T06:49:13.970280+0000","last_clean":"2024-09-19T06:49:13.970280+0000","last_became_active":"2024-09-19T06:47:24.841228+0000","last_became_peered":"2024-09-19T06:47:24.841228+0000","last_unstale":"2024-09-19T06:49:13.970280+0000","last_undegraded":"2024-09-19T06:49:13.970280+0000","last_fullsized":"2024-09-19T06:49:13.970280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:56.418962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.933661+0000","last_change":"2024-09-19T06:47:22.833893+0000","last_active":"2024-09-19T06:49:13.933661+0000","last_peered":"2024-09-19T06:49:13.933661+0000","last_clean":"2024-09-19T06:49:13.933661+0000","last_became_active":"2024-09-19T06:47:22.833217+0000","last_became_peered":"2024-09-19T06:47:22.833217+0000","last_unstale":"2024-09-19T06:49:13.933661+0000","last_undegraded":"2024-09-19T06:49:13.933661+0000","last_fullsized":"2024-09-19T06:49:13.933661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:19.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955096+0000","last_change":"2024-09-19T06:47:24.848399+0000","last_active":"2024-09-19T06:49:13.955096+0000","last_peered":"2024-09-19T06:49:13.955096+0000","last_clean":"2024-09-19T06:49:13.955096+0000","last_became_active":"2024-09-19T06:47:24.848081+0000","last_became_peered":"2024-09-19T06:47:24.848081+0000","last_unstale":"2024-09-19T06:49:13.955096+0000","last_undegraded":"2024-09-19T06:49:13.955096+0000","last_fullsized":"2024-09-19T06:49:13.955096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:44.292939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.931968+0000","last_change":"2024-09-19T06:47:40.860955+0000","last_active":"2024-09-19T06:49:13.931968+0000","last_peered":"2024-09-19T06:49:13.931968+0000","last_clean":"2024-09-19T06:49:13.931968+0000","last_became_active":"2024-09-19T06:47:40.835126+0000","last_became_peered":"2024-09-19T06:47:40.835126+0000","last_unstale":"2024-09-19T06:49:13.931968+0000","last_undegraded":"2024-09-19T06:49:13.931968+0000","last_fullsized":"2024-09-19T06:49:13.931968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:02:26.763229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971391+0000","last_change":"2024-09-19T06:47:24.849680+0000","last_active":"2024-09-19T06:49:13.971391+0000","last_peered":"2024-09-19T06:49:13.971391+0000","last_clean":"2024-09-19T06:49:13.971391+0000","last_became_active":"2024-09-19T06:47:24.849323+0000","last_became_peered":"2024-09-19T06:47:24.849323+0000","last_unstale":"2024-09-19T06:49:13.971391+0000","last_undegraded":"2024-09-19T06:49:13.971391+0000","last_fullsized":"2024-09-19T06:49:13.971391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:59.763503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'41","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.148665+0000","last_change":"2024-09-19T06:47:22.831689+0000","last_active":"2024-09-19T06:49:14.148665+0000","last_peered":"2024-09-19T06:49:14.148665+0000","last_clean":"2024-09-19T06:49:14.148665+0000","last_became_active":"2024-09-19T06:47:22.829283+0000","last_became_peered":"2024-09-19T06:47:22.829283+0000","last_unstale":"2024-09-19T06:49:14.148665+0000","last_undegraded":"2024-09-19T06:49:14.148665+0000","last_fullsized":"2024-09-19T06:49:14.148665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:37.109542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971792+0000","last_change":"2024-09-19T06:47:24.842793+0000","last_active":"2024-09-19T06:49:13.971792+0000","last_peered":"2024-09-19T06:49:13.971792+0000","last_clean":"2024-09-19T06:49:13.971792+0000","last_became_active":"2024-09-19T06:47:24.842474+0000","last_became_peered":"2024-09-19T06:47:24.842474+0000","last_unstale":"2024-09-19T06:49:13.971792+0000","last_undegraded":"2024-09-19T06:49:13.971792+0000","last_fullsized":"2024-09-19T06:49:13.971792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:23.110736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'9","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.605146+0000","last_change":"2024-09-19T06:47:23.047356+0000","last_active":"2024-09-19T06:49:14.605146+0000","last_peered":"2024-09-19T06:49:14.605146+0000","last_clean":"2024-09-19T06:49:14.605146+0000","last_became_active":"2024-09-19T06:47:23.046941+0000","last_became_peered":"2024-09-19T06:47:23.046941+0000","last_unstale":"2024-09-19T06:49:14.605146+0000","last_undegraded":"2024-09-19T06:49:14.605146+0000","last_fullsized":"2024-09-19T06:49:14.605146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:43.490952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":4,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971243+0000","last_change":"2024-09-19T06:47:24.851025+0000","last_active":"2024-09-19T06:49:13.971243+0000","last_peered":"2024-09-19T06:49:13.971243+0000","last_clean":"2024-09-19T06:49:13.971243+0000","last_became_active":"2024-09-19T06:47:24.850681+0000","last_became_peered":"2024-09-19T06:47:24.850681+0000","last_unstale":"2024-09-19T06:49:13.971243+0000","last_undegraded":"2024-09-19T06:49:13.971243+0000","last_fullsized":"2024-09-19T06:49:13.971243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:46.884747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":348,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.928776+0000","last_change":"2024-09-19T06:46:13.927379+0000","last_active":"2024-09-19T06:49:13.928776+0000","last_peered":"2024-09-19T06:49:13.928776+0000","last_clean":"2024-09-19T06:49:13.928776+0000","last_became_active":"2024-09-19T06:46:13.926926+0000","last_became_peered":"2024-09-19T06:46:13.926926+0000","last_unstale":"2024-09-19T06:49:13.928776+0000","last_undegraded":"2024-09-19T06:49:13.928776+0000","last_fullsized":"2024-09-19T06:49:13.928776+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_clean_scrub_stamp":"2024-09-19T06:42:52.051335+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:29.542128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.927403+0000","last_change":"2024-09-19T06:47:22.830387+0000","last_active":"2024-09-19T06:49:13.927403+0000","last_peered":"2024-09-19T06:49:13.927403+0000","last_clean":"2024-09-19T06:49:13.927403+0000","last_became_active":"2024-09-19T06:47:22.829970+0000","last_became_peered":"2024-09-19T06:47:22.829970+0000","last_unstale":"2024-09-19T06:49:13.927403+0000","last_undegraded":"2024-09-19T06:49:13.927403+0000","last_fullsized":"2024-09-19T06:49:13.927403+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:51.385456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.974671+0000","last_change":"2024-09-19T06:47:24.843752+0000","last_active":"2024-09-19T06:49:13.974671+0000","last_peered":"2024-09-19T06:49:13.974671+0000","last_clean":"2024-09-19T06:49:13.974671+0000","last_became_active":"2024-09-19T06:47:24.842631+0000","last_became_peered":"2024-09-19T06:47:24.842631+0000","last_unstale":"2024-09-19T06:49:13.974671+0000","last_undegraded":"2024-09-19T06:49:13.974671+0000","last_fullsized":"2024-09-19T06:49:13.974671+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:19.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.928732+0000","last_change":"2024-09-19T06:47:22.832647+0000","last_active":"2024-09-19T06:49:13.928732+0000","last_peered":"2024-09-19T06:49:13.928732+0000","last_clean":"2024-09-19T06:49:13.928732+0000","last_became_active":"2024-09-19T06:47:22.831774+0000","last_became_peered":"2024-09-19T06:47:22.831774+0000","last_unstale":"2024-09-19T06:49:13.928732+0000","last_undegraded":"2024-09-19T06:49:13.928732+0000","last_fullsized":"2024-09-19T06:49:13.928732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:57.824610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.948364+0000","last_change":"2024-09-19T06:47:24.844301+0000","last_active":"2024-09-19T06:49:13.948364+0000","last_peered":"2024-09-19T06:49:13.948364+0000","last_clean":"2024-09-19T06:49:13.948364+0000","last_became_active":"2024-09-19T06:47:24.839748+0000","last_became_peered":"2024-09-19T06:47:24.839748+0000","last_unstale":"2024-09-19T06:49:13.948364+0000","last_undegraded":"2024-09-19T06:49:13.948364+0000","last_fullsized":"2024-09-19T06:49:13.948364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:46.407693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":130,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932811+0000","last_change":"2024-09-19T06:47:22.851008+0000","last_active":"2024-09-19T06:49:13.932811+0000","last_peered":"2024-09-19T06:49:13.932811+0000","last_clean":"2024-09-19T06:49:13.932811+0000","last_became_active":"2024-09-19T06:47:22.850601+0000","last_became_peered":"2024-09-19T06:47:22.850601+0000","last_unstale":"2024-09-19T06:49:13.932811+0000","last_undegraded":"2024-09-19T06:49:13.932811+0000","last_fullsized":"2024-09-19T06:49:13.932811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:04.610277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970407+0000","last_change":"2024-09-19T06:47:24.839684+0000","last_active":"2024-09-19T06:49:13.970407+0000","last_peered":"2024-09-19T06:49:13.970407+0000","last_clean":"2024-09-19T06:49:13.970407+0000","last_became_active":"2024-09-19T06:47:24.839126+0000","last_became_peered":"2024-09-19T06:47:24.839126+0000","last_unstale":"2024-09-19T06:49:13.970407+0000","last_undegraded":"2024-09-19T06:49:13.970407+0000","last_fullsized":"2024-09-19T06:49:13.970407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:09.458535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":130,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.314141+0000","last_change":"2024-09-19T06:47:22.831819+0000","last_active":"2024-09-19T06:49:14.314141+0000","last_peered":"2024-09-19T06:49:14.314141+0000","last_clean":"2024-09-19T06:49:14.314141+0000","last_became_active":"2024-09-19T06:47:22.831332+0000","last_became_peered":"2024-09-19T06:47:22.831332+0000","last_unstale":"2024-09-19T06:49:14.314141+0000","last_undegraded":"2024-09-19T06:49:14.314141+0000","last_fullsized":"2024-09-19T06:49:14.314141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:35.032949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.965109+0000","last_change":"2024-09-19T06:47:24.846738+0000","last_active":"2024-09-19T06:49:13.965109+0000","last_peered":"2024-09-19T06:49:13.965109+0000","last_clean":"2024-09-19T06:49:13.965109+0000","last_became_active":"2024-09-19T06:47:24.846389+0000","last_became_peered":"2024-09-19T06:47:24.846389+0000","last_unstale":"2024-09-19T06:49:13.965109+0000","last_undegraded":"2024-09-19T06:49:13.965109+0000","last_fullsized":"2024-09-19T06:49:13.965109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:19.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.918173+0000","last_change":"2024-09-19T06:47:22.851065+0000","last_active":"2024-09-19T06:49:13.918173+0000","last_peered":"2024-09-19T06:49:13.918173+0000","last_clean":"2024-09-19T06:49:13.918173+0000","last_became_active":"2024-09-19T06:47:22.850647+0000","last_became_peered":"2024-09-19T06:47:22.850647+0000","last_unstale":"2024-09-19T06:49:13.918173+0000","last_undegraded":"2024-09-19T06:49:13.918173+0000","last_fullsized":"2024-09-19T06:49:13.918173+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:41.993501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966795+0000","last_change":"2024-09-19T06:47:24.838905+0000","last_active":"2024-09-19T06:49:13.966795+0000","last_peered":"2024-09-19T06:49:13.966795+0000","last_clean":"2024-09-19T06:49:13.966795+0000","last_became_active":"2024-09-19T06:47:24.838443+0000","last_became_peered":"2024-09-19T06:47:24.838443+0000","last_unstale":"2024-09-19T06:49:13.966795+0000","last_undegraded":"2024-09-19T06:49:13.966795+0000","last_fullsized":"2024-09-19T06:49:13.966795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:12.486137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932653+0000","last_change":"2024-09-19T06:47:22.851431+0000","last_active":"2024-09-19T06:49:13.932653+0000","last_peered":"2024-09-19T06:49:13.932653+0000","last_clean":"2024-09-19T06:49:13.932653+0000","last_became_active":"2024-09-19T06:47:22.851092+0000","last_became_peered":"2024-09-19T06:47:22.851092+0000","last_unstale":"2024-09-19T06:49:13.932653+0000","last_undegraded":"2024-09-19T06:49:13.932653+0000","last_fullsized":"2024-09-19T06:49:13.932653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:26.183803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959143+0000","last_change":"2024-09-19T06:47:24.839046+0000","last_active":"2024-09-19T06:49:13.959143+0000","last_peered":"2024-09-19T06:49:13.959143+0000","last_clean":"2024-09-19T06:49:13.959143+0000","last_became_active":"2024-09-19T06:47:24.838680+0000","last_became_peered":"2024-09-19T06:47:24.838680+0000","last_unstale":"2024-09-19T06:49:13.959143+0000","last_undegraded":"2024-09-19T06:49:13.959143+0000","last_fullsized":"2024-09-19T06:49:13.959143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:45.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'50","reported_seq":113,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.156330+0000","last_change":"2024-09-19T06:47:40.865241+0000","last_active":"2024-09-19T06:49:14.156330+0000","last_peered":"2024-09-19T06:49:14.156330+0000","last_clean":"2024-09-19T06:49:14.156330+0000","last_became_active":"2024-09-19T06:47:40.837765+0000","last_became_peered":"2024-09-19T06:47:40.837765+0000","last_unstale":"2024-09-19T06:49:14.156330+0000","last_undegraded":"2024-09-19T06:49:14.156330+0000","last_fullsized":"2024-09-19T06:49:14.156330+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966118+0000","last_change":"2024-09-19T06:47:24.848599+0000","last_active":"2024-09-19T06:49:13.966118+0000","last_peered":"2024-09-19T06:49:13.966118+0000","last_clean":"2024-09-19T06:49:13.966118+0000","last_became_active":"2024-09-19T06:47:24.848305+0000","last_became_peered":"2024-09-19T06:47:24.848305+0000","last_unstale":"2024-09-19T06:49:13.966118+0000","last_undegraded":"2024-09-19T06:49:13.966118+0000","last_fullsized":"2024-09-19T06:49:13.966118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:31.496188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.927755+0000","last_change":"2024-09-19T06:47:23.047484+0000","last_active":"2024-09-19T06:49:13.927755+0000","last_peered":"2024-09-19T06:49:13.927755+0000","last_clean":"2024-09-19T06:49:13.927755+0000","last_became_active":"2024-09-19T06:47:23.047058+0000","last_became_peered":"2024-09-19T06:47:23.047058+0000","last_unstale":"2024-09-19T06:49:13.927755+0000","last_undegraded":"2024-09-19T06:49:13.927755+0000","last_fullsized":"2024-09-19T06:49:13.927755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:34.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970265+0000","last_change":"2024-09-19T06:47:24.843498+0000","last_active":"2024-09-19T06:49:13.970265+0000","last_peered":"2024-09-19T06:49:13.970265+0000","last_clean":"2024-09-19T06:49:13.970265+0000","last_became_active":"2024-09-19T06:47:24.843204+0000","last_became_peered":"2024-09-19T06:47:24.843204+0000","last_unstale":"2024-09-19T06:49:13.970265+0000","last_undegraded":"2024-09-19T06:49:13.970265+0000","last_fullsized":"2024-09-19T06:49:13.970265+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:25.585023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.919373+0000","last_change":"2024-09-19T06:47:22.851519+0000","last_active":"2024-09-19T06:49:13.919373+0000","last_peered":"2024-09-19T06:49:13.919373+0000","last_clean":"2024-09-19T06:49:13.919373+0000","last_became_active":"2024-09-19T06:47:22.851163+0000","last_became_peered":"2024-09-19T06:47:22.851163+0000","last_unstale":"2024-09-19T06:49:13.919373+0000","last_undegraded":"2024-09-19T06:49:13.919373+0000","last_fullsized":"2024-09-19T06:49:13.919373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:04.502795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.945308+0000","last_change":"2024-09-19T06:47:24.845239+0000","last_active":"2024-09-19T06:49:13.945308+0000","last_peered":"2024-09-19T06:49:13.945308+0000","last_clean":"2024-09-19T06:49:13.945308+0000","last_became_active":"2024-09-19T06:47:24.844934+0000","last_became_peered":"2024-09-19T06:47:24.844934+0000","last_unstale":"2024-09-19T06:49:13.945308+0000","last_undegraded":"2024-09-19T06:49:13.945308+0000","last_fullsized":"2024-09-19T06:49:13.945308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:43.610604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.934380+0000","last_change":"2024-09-19T06:47:22.830232+0000","last_active":"2024-09-19T06:49:13.934380+0000","last_peered":"2024-09-19T06:49:13.934380+0000","last_clean":"2024-09-19T06:49:13.934380+0000","last_became_active":"2024-09-19T06:47:22.829901+0000","last_became_peered":"2024-09-19T06:47:22.829901+0000","last_unstale":"2024-09-19T06:49:13.934380+0000","last_undegraded":"2024-09-19T06:49:13.934380+0000","last_fullsized":"2024-09-19T06:49:13.934380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:12.251427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955845+0000","last_change":"2024-09-19T06:47:24.839304+0000","last_active":"2024-09-19T06:49:13.955845+0000","last_peered":"2024-09-19T06:49:13.955845+0000","last_clean":"2024-09-19T06:49:13.955845+0000","last_became_active":"2024-09-19T06:47:24.838986+0000","last_became_peered":"2024-09-19T06:47:24.838986+0000","last_unstale":"2024-09-19T06:49:13.955845+0000","last_undegraded":"2024-09-19T06:49:13.955845+0000","last_fullsized":"2024-09-19T06:49:13.955845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:39.377886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.885319+0000","last_change":"2024-09-19T06:47:22.828553+0000","last_active":"2024-09-19T06:49:14.885319+0000","last_peered":"2024-09-19T06:49:14.885319+0000","last_clean":"2024-09-19T06:49:14.885319+0000","last_became_active":"2024-09-19T06:47:22.828213+0000","last_became_peered":"2024-09-19T06:47:22.828213+0000","last_unstale":"2024-09-19T06:49:14.885319+0000","last_undegraded":"2024-09-19T06:49:14.885319+0000","last_fullsized":"2024-09-19T06:49:14.885319+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:31.649761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960387+0000","last_change":"2024-09-19T06:47:24.840747+0000","last_active":"2024-09-19T06:49:13.960387+0000","last_peered":"2024-09-19T06:49:13.960387+0000","last_clean":"2024-09-19T06:49:13.960387+0000","last_became_active":"2024-09-19T06:47:24.840451+0000","last_became_peered":"2024-09-19T06:47:24.840451+0000","last_unstale":"2024-09-19T06:49:13.960387+0000","last_undegraded":"2024-09-19T06:49:13.960387+0000","last_fullsized":"2024-09-19T06:49:13.960387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:41.588751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.933845+0000","last_change":"2024-09-19T06:47:22.831697+0000","last_active":"2024-09-19T06:49:13.933845+0000","last_peered":"2024-09-19T06:49:13.933845+0000","last_clean":"2024-09-19T06:49:13.933845+0000","last_became_active":"2024-09-19T06:47:22.831178+0000","last_became_peered":"2024-09-19T06:47:22.831178+0000","last_unstale":"2024-09-19T06:49:13.933845+0000","last_undegraded":"2024-09-19T06:49:13.933845+0000","last_fullsized":"2024-09-19T06:49:13.933845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:30.975782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971917+0000","last_change":"2024-09-19T06:47:24.845097+0000","last_active":"2024-09-19T06:49:13.971917+0000","last_peered":"2024-09-19T06:49:13.971917+0000","last_clean":"2024-09-19T06:49:13.971917+0000","last_became_active":"2024-09-19T06:47:24.844801+0000","last_became_peered":"2024-09-19T06:47:24.844801+0000","last_unstale":"2024-09-19T06:49:13.971917+0000","last_undegraded":"2024-09-19T06:49:13.971917+0000","last_fullsized":"2024-09-19T06:49:13.971917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:15.264118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971733+0000","last_change":"2024-09-19T06:47:24.851372+0000","last_active":"2024-09-19T06:49:13.971733+0000","last_peered":"2024-09-19T06:49:13.971733+0000","last_clean":"2024-09-19T06:49:13.971733+0000","last_became_active":"2024-09-19T06:47:24.851044+0000","last_became_peered":"2024-09-19T06:47:24.851044+0000","last_unstale":"2024-09-19T06:49:13.971733+0000","last_undegraded":"2024-09-19T06:49:13.971733+0000","last_fullsized":"2024-09-19T06:49:13.971733+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:55.658214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932580+0000","last_change":"2024-09-19T06:47:22.833694+0000","last_active":"2024-09-19T06:49:13.932580+0000","last_peered":"2024-09-19T06:49:13.932580+0000","last_clean":"2024-09-19T06:49:13.932580+0000","last_became_active":"2024-09-19T06:47:22.833215+0000","last_became_peered":"2024-09-19T06:47:22.833215+0000","last_unstale":"2024-09-19T06:49:13.932580+0000","last_undegraded":"2024-09-19T06:49:13.932580+0000","last_fullsized":"2024-09-19T06:49:13.932580+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:32.171522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973551+0000","last_change":"2024-09-19T06:47:24.845737+0000","last_active":"2024-09-19T06:49:13.973551+0000","last_peered":"2024-09-19T06:49:13.973551+0000","last_clean":"2024-09-19T06:49:13.973551+0000","last_became_active":"2024-09-19T06:47:24.845242+0000","last_became_peered":"2024-09-19T06:47:24.845242+0000","last_unstale":"2024-09-19T06:49:13.973551+0000","last_undegraded":"2024-09-19T06:49:13.973551+0000","last_fullsized":"2024-09-19T06:49:13.973551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:51.554946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932499+0000","last_change":"2024-09-19T06:47:22.829869+0000","last_active":"2024-09-19T06:49:13.932499+0000","last_peered":"2024-09-19T06:49:13.932499+0000","last_clean":"2024-09-19T06:49:13.932499+0000","last_became_active":"2024-09-19T06:47:22.829522+0000","last_became_peered":"2024-09-19T06:47:22.829522+0000","last_unstale":"2024-09-19T06:49:13.932499+0000","last_undegraded":"2024-09-19T06:49:13.932499+0000","last_fullsized":"2024-09-19T06:49:13.932499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:26.871434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973419+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:49:13.973419+0000","last_peered":"2024-09-19T06:49:13.973419+0000","last_clean":"2024-09-19T06:49:13.973419+0000","last_became_active":"2024-09-19T06:47:24.841289+0000","last_became_peered":"2024-09-19T06:47:24.841289+0000","last_unstale":"2024-09-19T06:49:13.973419+0000","last_undegraded":"2024-09-19T06:49:13.973419+0000","last_fullsized":"2024-09-19T06:49:13.973419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:21:12.611458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.937218+0000","last_change":"2024-09-19T06:47:22.852198+0000","last_active":"2024-09-19T06:49:13.937218+0000","last_peered":"2024-09-19T06:49:13.937218+0000","last_clean":"2024-09-19T06:49:13.937218+0000","last_became_active":"2024-09-19T06:47:22.850984+0000","last_became_peered":"2024-09-19T06:47:22.850984+0000","last_unstale":"2024-09-19T06:49:13.937218+0000","last_undegraded":"2024-09-19T06:49:13.937218+0000","last_fullsized":"2024-09-19T06:49:13.937218+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:13.620145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959682+0000","last_change":"2024-09-19T06:47:40.837614+0000","last_active":"2024-09-19T06:49:13.959682+0000","last_peered":"2024-09-19T06:49:13.959682+0000","last_clean":"2024-09-19T06:49:13.959682+0000","last_became_active":"2024-09-19T06:47:40.837259+0000","last_became_peered":"2024-09-19T06:47:40.837259+0000","last_unstale":"2024-09-19T06:49:13.959682+0000","last_undegraded":"2024-09-19T06:49:13.959682+0000","last_fullsized":"2024-09-19T06:49:13.959682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:41.429962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":157,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.197340+0000","last_change":"2024-09-19T06:47:41.182639+0000","last_active":"2024-09-19T06:49:14.197340+0000","last_peered":"2024-09-19T06:49:14.197340+0000","last_clean":"2024-09-19T06:49:14.197340+0000","last_became_active":"2024-09-19T06:47:41.174305+0000","last_became_peered":"2024-09-19T06:47:41.174305+0000","last_unstale":"2024-09-19T06:49:14.197340+0000","last_undegraded":"2024-09-19T06:49:14.197340+0000","last_fullsized":"2024-09-19T06:49:14.197340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:10.199502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601319,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1124,"num_read_kb":676,"num_write":379,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1803957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579939,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32472,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712680,"statfs":{"total":95995035648,"available":95961784320,"internally_reserved":0,"allocated":1314816,"data_stored":842278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28864,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716288,"statfs":{"total":95995035648,"available":95965478912,"internally_reserved":0,"allocated":1761280,"data_stored":1238256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972397,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28332,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716820,"statfs":{"total":95995035648,"available":95966023680,"internally_reserved":0,"allocated":1335296,"data_stored":840798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234034,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":447288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462966,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32688,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712464,"statfs":{"total":95995035648,"available":95961563136,"internally_reserved":0,"allocated":1536000,"data_stored":1042501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626492,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":447692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855423,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1171668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018949,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":448114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215174,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32620,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712532,"statfs":{"total":95995035648,"available":95961632768,"internally_reserved":0,"allocated":1470464,"data_stored":971940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444112,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":516380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1003520,"data_stored":515624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":449172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":134723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":201311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":395818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:49:18.740 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T06:49:18.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.883+0000 7f048d1fd640 1 -- 172.21.15.103:0/3273583145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0488100a20 msgr2=0x7f0488100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.883+0000 7f048d1fd640 1 --2- 172.21.15.103:0/3273583145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0488100a20 0x7f0488100e80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f047c009930 tx=0x7f047c02a420 comp rx=0 tx=0).stop 2024-09-19T06:49:18.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.883+0000 7f048d1fd640 1 -- 172.21.15.103:0/3273583145 shutdown_connections 2024-09-19T06:49:18.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.883+0000 7f048d1fd640 1 --2- 172.21.15.103:0/3273583145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04881013c0 0x7f048810d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.883+0000 7f048d1fd640 1 --2- 172.21.15.103:0/3273583145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0488100a20 0x7f0488100e80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.883+0000 7f048d1fd640 1 --2- 172.21.15.103:0/3273583145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0488106a30 0x7f0488106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.883+0000 7f048d1fd640 1 -- 172.21.15.103:0/3273583145 >> 172.21.15.103:0/3273583145 conn(0x7f04880fc740 msgr2=0x7f04880feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:18.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.883+0000 7f048d1fd640 1 -- 172.21.15.103:0/3273583145 shutdown_connections 2024-09-19T06:49:18.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.883+0000 7f048d1fd640 1 -- 172.21.15.103:0/3273583145 wait complete. 2024-09-19T06:49:18.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.884+0000 7f048d1fd640 1 Processor -- start 2024-09-19T06:49:18.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.884+0000 7f048d1fd640 1 -- start start 2024-09-19T06:49:18.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.884+0000 7f048d1fd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0488100a20 0x7f0488199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.884+0000 7f048d1fd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04881013c0 0x7f0488199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.884+0000 7f048d1fd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0488106a30 0x7f04881a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.884+0000 7f048d1fd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0488110e70 con 0x7f0488106a30 2024-09-19T06:49:18.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.884+0000 7f048d1fd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0488110cf0 con 0x7f04881013c0 2024-09-19T06:49:18.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.884+0000 7f048d1fd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0488110ff0 con 0x7f0488100a20 2024-09-19T06:49:18.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.885+0000 7f048759e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0488106a30 0x7f04881a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.885+0000 7f048759e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0488106a30 0x7f04881a43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42382/0 (socket says 172.21.15.103:42382) 2024-09-19T06:49:18.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.885+0000 7f048759e640 1 -- 172.21.15.103:0/4220047172 learned_addr learned my addr 172.21.15.103:0/4220047172 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:18.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.885+0000 7f0486d9d640 1 --2- 172.21.15.103:0/4220047172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0488100a20 0x7f0488199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.885+0000 7f048659c640 1 --2- 172.21.15.103:0/4220047172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04881013c0 0x7f0488199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.885+0000 7f048759e640 1 -- 172.21.15.103:0/4220047172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0488100a20 msgr2=0x7f0488199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.885+0000 7f048759e640 1 --2- 172.21.15.103:0/4220047172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0488100a20 0x7f0488199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.885+0000 7f048759e640 1 -- 172.21.15.103:0/4220047172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04881013c0 msgr2=0x7f0488199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.885+0000 7f048759e640 1 --2- 172.21.15.103:0/4220047172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04881013c0 0x7f0488199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.885+0000 7f048759e640 1 -- 172.21.15.103:0/4220047172 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04881a4900 con 0x7f0488106a30 2024-09-19T06:49:18.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.885+0000 7f048759e640 1 --2- 172.21.15.103:0/4220047172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0488106a30 0x7f04881a43c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f048000b970 tx=0x7f048000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:18.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.885+0000 7f046ffff640 1 -- 172.21.15.103:0/4220047172 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480019070 con 0x7f0488106a30 2024-09-19T06:49:18.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.885+0000 7f048d1fd640 1 -- 172.21.15.103:0/4220047172 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04881a4ad0 con 0x7f0488106a30 2024-09-19T06:49:18.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.885+0000 7f046ffff640 1 -- 172.21.15.103:0/4220047172 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f048000cb40 con 0x7f0488106a30 2024-09-19T06:49:18.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.885+0000 7f048d1fd640 1 -- 172.21.15.103:0/4220047172 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04881a5040 con 0x7f0488106a30 2024-09-19T06:49:18.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.885+0000 7f046ffff640 1 -- 172.21.15.103:0/4220047172 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04800075f0 con 0x7f0488106a30 2024-09-19T06:49:18.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.886+0000 7f048d1fd640 1 -- 172.21.15.103:0/4220047172 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0488100e80 con 0x7f0488106a30 2024-09-19T06:49:18.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.888+0000 7f046ffff640 1 -- 172.21.15.103:0/4220047172 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f048001c070 con 0x7f0488106a30 2024-09-19T06:49:18.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.888+0000 7f046ffff640 1 --2- 172.21.15.103:0/4220047172 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f04700a34e0 0x7f04700a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.888+0000 7f0486d9d640 1 --2- 172.21.15.103:0/4220047172 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f04700a34e0 0x7f04700a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.888+0000 7f046ffff640 1 -- 172.21.15.103:0/4220047172 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f04800cf870 con 0x7f0488106a30 2024-09-19T06:49:18.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.889+0000 7f0486d9d640 1 --2- 172.21.15.103:0/4220047172 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f04700a34e0 0x7f04700a59a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f047800bbe0 tx=0x7f047800b390 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:18.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:18.891+0000 7f046ffff640 1 -- 172.21.15.103:0/4220047172 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f048009bad0 con 0x7f0488106a30 2024-09-19T06:49:19.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.060+0000 7f048d1fd640 1 -- 172.21.15.103:0/4220047172 --> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0488062630 con 0x7f04700a34e0 2024-09-19T06:49:19.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.067+0000 7f046ffff640 1 -- 172.21.15.103:0/4220047172 <== mgr.15210 v2:172.21.15.103:6800/82882809 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257913 (secure 0 0 0) 0x7f0488062630 con 0x7f04700a34e0 2024-09-19T06:49:19.070 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:49:19.071 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T06:49:19.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.069+0000 7f048d1fd640 1 -- 172.21.15.103:0/4220047172 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f04700a34e0 msgr2=0x7f04700a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:19.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.069+0000 7f048d1fd640 1 --2- 172.21.15.103:0/4220047172 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f04700a34e0 0x7f04700a59a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f047800bbe0 tx=0x7f047800b390 comp rx=0 tx=0).stop 2024-09-19T06:49:19.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.069+0000 7f048d1fd640 1 -- 172.21.15.103:0/4220047172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0488106a30 msgr2=0x7f04881a43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:19.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.069+0000 7f048d1fd640 1 --2- 172.21.15.103:0/4220047172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0488106a30 0x7f04881a43c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f048000b970 tx=0x7f048000be30 comp rx=0 tx=0).stop 2024-09-19T06:49:19.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.070+0000 7f048d1fd640 1 -- 172.21.15.103:0/4220047172 shutdown_connections 2024-09-19T06:49:19.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.070+0000 7f048d1fd640 1 --2- 172.21.15.103:0/4220047172 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f04700a34e0 0x7f04700a59a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:19.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.070+0000 7f048d1fd640 1 --2- 172.21.15.103:0/4220047172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0488106a30 0x7f04881a43c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:19.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.070+0000 7f048d1fd640 1 --2- 172.21.15.103:0/4220047172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04881013c0 0x7f0488199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:19.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.070+0000 7f048d1fd640 1 --2- 172.21.15.103:0/4220047172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0488100a20 0x7f0488199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:19.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.070+0000 7f048d1fd640 1 -- 172.21.15.103:0/4220047172 >> 172.21.15.103:0/4220047172 conn(0x7f04880fc740 msgr2=0x7f04880fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:19.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.070+0000 7f048d1fd640 1 -- 172.21.15.103:0/4220047172 shutdown_connections 2024-09-19T06:49:19.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.070+0000 7f048d1fd640 1 -- 172.21.15.103:0/4220047172 wait complete. 2024-09-19T06:49:19.080 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:18 smithi103 ceph-mon[31158]: Updating smithi134:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:49:19.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:18 smithi103 ceph-mon[31158]: Updating smithi174:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:49:19.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:18 smithi103 ceph-mon[31158]: Updating smithi103:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:49:19.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:18 smithi103 ceph-mon[31158]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:49:19.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:18 smithi103 ceph-mon[31158]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:49:19.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:18 smithi103 ceph-mon[31158]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:49:19.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:18 smithi103 ceph-mon[31158]: Updating smithi134:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:49:19.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:18 smithi103 ceph-mon[31158]: Updating smithi174:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:49:19.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:18 smithi103 ceph-mon[31158]: Updating smithi103:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:49:19.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:18 smithi103 ceph-mon[31158]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:49:19.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:18 smithi103 ceph-mon[31158]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-09-19T06:49:19.082 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T06:49:17.932591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191921,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1260,"num_read_kb":795,"num_write":562,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":1181610,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":943,"ondisk_log_size":943,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355580,"kb_used_data":14496,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124586244,"statfs":{"total":1151940427776,"available":1151576313856,"internally_reserved":0,"allocated":14843904,"data_stored":8931711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.991091"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960112+0000","last_change":"2024-09-19T06:47:24.853118+0000","last_active":"2024-09-19T06:49:13.960112+0000","last_peered":"2024-09-19T06:49:13.960112+0000","last_clean":"2024-09-19T06:49:13.960112+0000","last_became_active":"2024-09-19T06:47:24.852801+0000","last_became_peered":"2024-09-19T06:47:24.852801+0000","last_unstale":"2024-09-19T06:49:13.960112+0000","last_undegraded":"2024-09-19T06:49:13.960112+0000","last_fullsized":"2024-09-19T06:49:13.960112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:16.729103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.957024+0000","last_change":"2024-09-19T06:47:24.841095+0000","last_active":"2024-09-19T06:49:13.957024+0000","last_peered":"2024-09-19T06:49:13.957024+0000","last_clean":"2024-09-19T06:49:13.957024+0000","last_became_active":"2024-09-19T06:47:24.840774+0000","last_became_peered":"2024-09-19T06:47:24.840774+0000","last_unstale":"2024-09-19T06:49:13.957024+0000","last_undegraded":"2024-09-19T06:49:13.957024+0000","last_fullsized":"2024-09-19T06:49:13.957024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:27.262766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973374+0000","last_change":"2024-09-19T06:47:24.848444+0000","last_active":"2024-09-19T06:49:13.973374+0000","last_peered":"2024-09-19T06:49:13.973374+0000","last_clean":"2024-09-19T06:49:13.973374+0000","last_became_active":"2024-09-19T06:47:24.847354+0000","last_became_peered":"2024-09-19T06:47:24.847354+0000","last_unstale":"2024-09-19T06:49:13.973374+0000","last_undegraded":"2024-09-19T06:49:13.973374+0000","last_fullsized":"2024-09-19T06:49:13.973374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:44.901265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.949178+0000","last_change":"2024-09-19T06:47:24.851915+0000","last_active":"2024-09-19T06:49:13.949178+0000","last_peered":"2024-09-19T06:49:13.949178+0000","last_clean":"2024-09-19T06:49:13.949178+0000","last_became_active":"2024-09-19T06:47:24.851600+0000","last_became_peered":"2024-09-19T06:47:24.851600+0000","last_unstale":"2024-09-19T06:49:13.949178+0000","last_undegraded":"2024-09-19T06:49:13.949178+0000","last_fullsized":"2024-09-19T06:49:13.949178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:52.424288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.950906+0000","last_change":"2024-09-19T06:47:24.843184+0000","last_active":"2024-09-19T06:49:13.950906+0000","last_peered":"2024-09-19T06:49:13.950906+0000","last_clean":"2024-09-19T06:49:13.950906+0000","last_became_active":"2024-09-19T06:47:24.842857+0000","last_became_peered":"2024-09-19T06:47:24.842857+0000","last_unstale":"2024-09-19T06:49:13.950906+0000","last_undegraded":"2024-09-19T06:49:13.950906+0000","last_fullsized":"2024-09-19T06:49:13.950906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:06.608699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969815+0000","last_change":"2024-09-19T06:47:24.846631+0000","last_active":"2024-09-19T06:49:13.969815+0000","last_peered":"2024-09-19T06:49:13.969815+0000","last_clean":"2024-09-19T06:49:13.969815+0000","last_became_active":"2024-09-19T06:47:24.846271+0000","last_became_peered":"2024-09-19T06:47:24.846271+0000","last_unstale":"2024-09-19T06:49:13.969815+0000","last_undegraded":"2024-09-19T06:49:13.969815+0000","last_fullsized":"2024-09-19T06:49:13.969815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.272273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.967306+0000","last_change":"2024-09-19T06:47:24.840392+0000","last_active":"2024-09-19T06:49:13.967306+0000","last_peered":"2024-09-19T06:49:13.967306+0000","last_clean":"2024-09-19T06:49:13.967306+0000","last_became_active":"2024-09-19T06:47:24.840092+0000","last_became_peered":"2024-09-19T06:47:24.840092+0000","last_unstale":"2024-09-19T06:49:13.967306+0000","last_undegraded":"2024-09-19T06:49:13.967306+0000","last_fullsized":"2024-09-19T06:49:13.967306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:25.579096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.967567+0000","last_change":"2024-09-19T06:47:24.843336+0000","last_active":"2024-09-19T06:49:13.967567+0000","last_peered":"2024-09-19T06:49:13.967567+0000","last_clean":"2024-09-19T06:49:13.967567+0000","last_became_active":"2024-09-19T06:47:24.842016+0000","last_became_peered":"2024-09-19T06:47:24.842016+0000","last_unstale":"2024-09-19T06:49:13.967567+0000","last_undegraded":"2024-09-19T06:49:13.967567+0000","last_fullsized":"2024-09-19T06:49:13.967567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:49.478359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.968002+0000","last_change":"2024-09-19T06:47:24.847841+0000","last_active":"2024-09-19T06:49:13.968002+0000","last_peered":"2024-09-19T06:49:13.968002+0000","last_clean":"2024-09-19T06:49:13.968002+0000","last_became_active":"2024-09-19T06:47:24.847492+0000","last_became_peered":"2024-09-19T06:47:24.847492+0000","last_unstale":"2024-09-19T06:49:13.968002+0000","last_undegraded":"2024-09-19T06:49:13.968002+0000","last_fullsized":"2024-09-19T06:49:13.968002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:59.229022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955381+0000","last_change":"2024-09-19T06:47:24.843773+0000","last_active":"2024-09-19T06:49:13.955381+0000","last_peered":"2024-09-19T06:49:13.955381+0000","last_clean":"2024-09-19T06:49:13.955381+0000","last_became_active":"2024-09-19T06:47:24.843409+0000","last_became_peered":"2024-09-19T06:47:24.843409+0000","last_unstale":"2024-09-19T06:49:13.955381+0000","last_undegraded":"2024-09-19T06:49:13.955381+0000","last_fullsized":"2024-09-19T06:49:13.955381+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:22.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969613+0000","last_change":"2024-09-19T06:47:24.846637+0000","last_active":"2024-09-19T06:49:13.969613+0000","last_peered":"2024-09-19T06:49:13.969613+0000","last_clean":"2024-09-19T06:49:13.969613+0000","last_became_active":"2024-09-19T06:47:24.846337+0000","last_became_peered":"2024-09-19T06:47:24.846337+0000","last_unstale":"2024-09-19T06:49:13.969613+0000","last_undegraded":"2024-09-19T06:49:13.969613+0000","last_fullsized":"2024-09-19T06:49:13.969613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:23:35.869485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.946579+0000","last_change":"2024-09-19T06:47:24.851399+0000","last_active":"2024-09-19T06:49:13.946579+0000","last_peered":"2024-09-19T06:49:13.946579+0000","last_clean":"2024-09-19T06:49:13.946579+0000","last_became_active":"2024-09-19T06:47:24.851090+0000","last_became_peered":"2024-09-19T06:47:24.851090+0000","last_unstale":"2024-09-19T06:49:13.946579+0000","last_undegraded":"2024-09-19T06:49:13.946579+0000","last_fullsized":"2024-09-19T06:49:13.946579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:52.839956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970074+0000","last_change":"2024-09-19T06:47:24.833858+0000","last_active":"2024-09-19T06:49:13.970074+0000","last_peered":"2024-09-19T06:49:13.970074+0000","last_clean":"2024-09-19T06:49:13.970074+0000","last_became_active":"2024-09-19T06:47:24.833616+0000","last_became_peered":"2024-09-19T06:47:24.833616+0000","last_unstale":"2024-09-19T06:49:13.970074+0000","last_undegraded":"2024-09-19T06:49:13.970074+0000","last_fullsized":"2024-09-19T06:49:13.970074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:54.918144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966731+0000","last_change":"2024-09-19T06:47:24.841030+0000","last_active":"2024-09-19T06:49:13.966731+0000","last_peered":"2024-09-19T06:49:13.966731+0000","last_clean":"2024-09-19T06:49:13.966731+0000","last_became_active":"2024-09-19T06:47:24.839631+0000","last_became_peered":"2024-09-19T06:47:24.839631+0000","last_unstale":"2024-09-19T06:49:13.966731+0000","last_undegraded":"2024-09-19T06:49:13.966731+0000","last_fullsized":"2024-09-19T06:49:13.966731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:21.764812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970278+0000","last_change":"2024-09-19T06:47:24.849281+0000","last_active":"2024-09-19T06:49:13.970278+0000","last_peered":"2024-09-19T06:49:13.970278+0000","last_clean":"2024-09-19T06:49:13.970278+0000","last_became_active":"2024-09-19T06:47:24.848977+0000","last_became_peered":"2024-09-19T06:47:24.848977+0000","last_unstale":"2024-09-19T06:49:13.970278+0000","last_undegraded":"2024-09-19T06:49:13.970278+0000","last_fullsized":"2024-09-19T06:49:13.970278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:36.693758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973886+0000","last_change":"2024-09-19T06:47:24.843619+0000","last_active":"2024-09-19T06:49:13.973886+0000","last_peered":"2024-09-19T06:49:13.973886+0000","last_clean":"2024-09-19T06:49:13.973886+0000","last_became_active":"2024-09-19T06:47:24.843323+0000","last_became_peered":"2024-09-19T06:47:24.843323+0000","last_unstale":"2024-09-19T06:49:13.973886+0000","last_undegraded":"2024-09-19T06:49:13.973886+0000","last_fullsized":"2024-09-19T06:49:13.973886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:52.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960333+0000","last_change":"2024-09-19T06:47:24.853165+0000","last_active":"2024-09-19T06:49:13.960333+0000","last_peered":"2024-09-19T06:49:13.960333+0000","last_clean":"2024-09-19T06:49:13.960333+0000","last_became_active":"2024-09-19T06:47:24.852864+0000","last_became_peered":"2024-09-19T06:47:24.852864+0000","last_unstale":"2024-09-19T06:49:13.960333+0000","last_undegraded":"2024-09-19T06:49:13.960333+0000","last_fullsized":"2024-09-19T06:49:13.960333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:38.803478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.975021+0000","last_change":"2024-09-19T06:47:24.838151+0000","last_active":"2024-09-19T06:49:13.975021+0000","last_peered":"2024-09-19T06:49:13.975021+0000","last_clean":"2024-09-19T06:49:13.975021+0000","last_became_active":"2024-09-19T06:47:24.837855+0000","last_became_peered":"2024-09-19T06:47:24.837855+0000","last_unstale":"2024-09-19T06:49:13.975021+0000","last_undegraded":"2024-09-19T06:49:13.975021+0000","last_fullsized":"2024-09-19T06:49:13.975021+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:16.675557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.964466+0000","last_change":"2024-09-19T06:47:24.843941+0000","last_active":"2024-09-19T06:49:13.964466+0000","last_peered":"2024-09-19T06:49:13.964466+0000","last_clean":"2024-09-19T06:49:13.964466+0000","last_became_active":"2024-09-19T06:47:24.843551+0000","last_became_peered":"2024-09-19T06:47:24.843551+0000","last_unstale":"2024-09-19T06:49:13.964466+0000","last_undegraded":"2024-09-19T06:49:13.964466+0000","last_fullsized":"2024-09-19T06:49:13.964466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:00:49.388265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.954208+0000","last_change":"2024-09-19T06:47:40.836228+0000","last_active":"2024-09-19T06:49:13.954208+0000","last_peered":"2024-09-19T06:49:13.954208+0000","last_clean":"2024-09-19T06:49:13.954208+0000","last_became_active":"2024-09-19T06:47:40.835818+0000","last_became_peered":"2024-09-19T06:47:40.835818+0000","last_unstale":"2024-09-19T06:49:13.954208+0000","last_undegraded":"2024-09-19T06:49:13.954208+0000","last_fullsized":"2024-09-19T06:49:13.954208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:45.341160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972919+0000","last_change":"2024-09-19T06:47:24.843746+0000","last_active":"2024-09-19T06:49:13.972919+0000","last_peered":"2024-09-19T06:49:13.972919+0000","last_clean":"2024-09-19T06:49:13.972919+0000","last_became_active":"2024-09-19T06:47:24.841495+0000","last_became_peered":"2024-09-19T06:47:24.841495+0000","last_unstale":"2024-09-19T06:49:13.972919+0000","last_undegraded":"2024-09-19T06:49:13.972919+0000","last_fullsized":"2024-09-19T06:49:13.972919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:06:29.995595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.974565+0000","last_change":"2024-09-19T06:47:24.844747+0000","last_active":"2024-09-19T06:49:13.974565+0000","last_peered":"2024-09-19T06:49:13.974565+0000","last_clean":"2024-09-19T06:49:13.974565+0000","last_became_active":"2024-09-19T06:47:24.844425+0000","last_became_peered":"2024-09-19T06:47:24.844425+0000","last_unstale":"2024-09-19T06:49:13.974565+0000","last_undegraded":"2024-09-19T06:49:13.974565+0000","last_fullsized":"2024-09-19T06:49:13.974565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:10.309044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963556+0000","last_change":"2024-09-19T06:47:24.848227+0000","last_active":"2024-09-19T06:49:13.963556+0000","last_peered":"2024-09-19T06:49:13.963556+0000","last_clean":"2024-09-19T06:49:13.963556+0000","last_became_active":"2024-09-19T06:47:24.847914+0000","last_became_peered":"2024-09-19T06:47:24.847914+0000","last_unstale":"2024-09-19T06:49:13.963556+0000","last_undegraded":"2024-09-19T06:49:13.963556+0000","last_fullsized":"2024-09-19T06:49:13.963556+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:47.281613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.968012+0000","last_change":"2024-09-19T06:47:24.844275+0000","last_active":"2024-09-19T06:49:13.968012+0000","last_peered":"2024-09-19T06:49:13.968012+0000","last_clean":"2024-09-19T06:49:13.968012+0000","last_became_active":"2024-09-19T06:47:24.841442+0000","last_became_peered":"2024-09-19T06:47:24.841442+0000","last_unstale":"2024-09-19T06:49:13.968012+0000","last_undegraded":"2024-09-19T06:49:13.968012+0000","last_fullsized":"2024-09-19T06:49:13.968012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:30.198655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963003+0000","last_change":"2024-09-19T06:47:24.846301+0000","last_active":"2024-09-19T06:49:13.963003+0000","last_peered":"2024-09-19T06:49:13.963003+0000","last_clean":"2024-09-19T06:49:13.963003+0000","last_became_active":"2024-09-19T06:47:24.845970+0000","last_became_peered":"2024-09-19T06:47:24.845970+0000","last_unstale":"2024-09-19T06:49:13.963003+0000","last_undegraded":"2024-09-19T06:49:13.963003+0000","last_fullsized":"2024-09-19T06:49:13.963003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:13.245536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973777+0000","last_change":"2024-09-19T06:47:24.844444+0000","last_active":"2024-09-19T06:49:13.973777+0000","last_peered":"2024-09-19T06:49:13.973777+0000","last_clean":"2024-09-19T06:49:13.973777+0000","last_became_active":"2024-09-19T06:47:24.844141+0000","last_became_peered":"2024-09-19T06:47:24.844141+0000","last_unstale":"2024-09-19T06:49:13.973777+0000","last_undegraded":"2024-09-19T06:49:13.973777+0000","last_fullsized":"2024-09-19T06:49:13.973777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:44.739242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955255+0000","last_change":"2024-09-19T06:47:24.845633+0000","last_active":"2024-09-19T06:49:13.955255+0000","last_peered":"2024-09-19T06:49:13.955255+0000","last_clean":"2024-09-19T06:49:13.955255+0000","last_became_active":"2024-09-19T06:47:24.845319+0000","last_became_peered":"2024-09-19T06:47:24.845319+0000","last_unstale":"2024-09-19T06:49:13.955255+0000","last_undegraded":"2024-09-19T06:49:13.955255+0000","last_fullsized":"2024-09-19T06:49:13.955255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:00.820006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959353+0000","last_change":"2024-09-19T06:47:24.846457+0000","last_active":"2024-09-19T06:49:13.959353+0000","last_peered":"2024-09-19T06:49:13.959353+0000","last_clean":"2024-09-19T06:49:13.959353+0000","last_became_active":"2024-09-19T06:47:24.846130+0000","last_became_peered":"2024-09-19T06:47:24.846130+0000","last_unstale":"2024-09-19T06:49:13.959353+0000","last_undegraded":"2024-09-19T06:49:13.959353+0000","last_fullsized":"2024-09-19T06:49:13.959353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:42.179145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969751+0000","last_change":"2024-09-19T06:47:24.843141+0000","last_active":"2024-09-19T06:49:13.969751+0000","last_peered":"2024-09-19T06:49:13.969751+0000","last_clean":"2024-09-19T06:49:13.969751+0000","last_became_active":"2024-09-19T06:47:24.842810+0000","last_became_peered":"2024-09-19T06:47:24.842810+0000","last_unstale":"2024-09-19T06:49:13.969751+0000","last_undegraded":"2024-09-19T06:49:13.969751+0000","last_fullsized":"2024-09-19T06:49:13.969751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:43.440453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.965342+0000","last_change":"2024-09-19T06:47:24.846171+0000","last_active":"2024-09-19T06:49:13.965342+0000","last_peered":"2024-09-19T06:49:13.965342+0000","last_clean":"2024-09-19T06:49:13.965342+0000","last_became_active":"2024-09-19T06:47:24.845799+0000","last_became_peered":"2024-09-19T06:47:24.845799+0000","last_unstale":"2024-09-19T06:49:13.965342+0000","last_undegraded":"2024-09-19T06:49:13.965342+0000","last_fullsized":"2024-09-19T06:49:13.965342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:23.103061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972106+0000","last_change":"2024-09-19T06:47:24.848723+0000","last_active":"2024-09-19T06:49:13.972106+0000","last_peered":"2024-09-19T06:49:13.972106+0000","last_clean":"2024-09-19T06:49:13.972106+0000","last_became_active":"2024-09-19T06:47:24.848367+0000","last_became_peered":"2024-09-19T06:47:24.848367+0000","last_unstale":"2024-09-19T06:49:13.972106+0000","last_undegraded":"2024-09-19T06:49:13.972106+0000","last_fullsized":"2024-09-19T06:49:13.972106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:15.985293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960543+0000","last_change":"2024-09-19T06:47:24.841379+0000","last_active":"2024-09-19T06:49:13.960543+0000","last_peered":"2024-09-19T06:49:13.960543+0000","last_clean":"2024-09-19T06:49:13.960543+0000","last_became_active":"2024-09-19T06:47:24.841041+0000","last_became_peered":"2024-09-19T06:47:24.841041+0000","last_unstale":"2024-09-19T06:49:13.960543+0000","last_undegraded":"2024-09-19T06:49:13.960543+0000","last_fullsized":"2024-09-19T06:49:13.960543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:11.799022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955643+0000","last_change":"2024-09-19T06:47:24.849508+0000","last_active":"2024-09-19T06:49:13.955643+0000","last_peered":"2024-09-19T06:49:13.955643+0000","last_clean":"2024-09-19T06:49:13.955643+0000","last_became_active":"2024-09-19T06:47:24.849196+0000","last_became_peered":"2024-09-19T06:47:24.849196+0000","last_unstale":"2024-09-19T06:49:13.955643+0000","last_undegraded":"2024-09-19T06:49:13.955643+0000","last_fullsized":"2024-09-19T06:49:13.955643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:35.079720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":132,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.604832+0000","last_change":"2024-09-19T06:47:22.829848+0000","last_active":"2024-09-19T06:49:14.604832+0000","last_peered":"2024-09-19T06:49:14.604832+0000","last_clean":"2024-09-19T06:49:14.604832+0000","last_became_active":"2024-09-19T06:47:22.829511+0000","last_became_peered":"2024-09-19T06:47:22.829511+0000","last_unstale":"2024-09-19T06:49:14.604832+0000","last_undegraded":"2024-09-19T06:49:14.604832+0000","last_fullsized":"2024-09-19T06:49:14.604832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:32.665252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971999+0000","last_change":"2024-09-19T06:47:24.843245+0000","last_active":"2024-09-19T06:49:13.971999+0000","last_peered":"2024-09-19T06:49:13.971999+0000","last_clean":"2024-09-19T06:49:13.971999+0000","last_became_active":"2024-09-19T06:47:24.842885+0000","last_became_peered":"2024-09-19T06:47:24.842885+0000","last_unstale":"2024-09-19T06:49:13.971999+0000","last_undegraded":"2024-09-19T06:49:13.971999+0000","last_fullsized":"2024-09-19T06:49:13.971999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:26.229003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.926577+0000","last_change":"2024-09-19T06:47:22.831456+0000","last_active":"2024-09-19T06:49:13.926577+0000","last_peered":"2024-09-19T06:49:13.926577+0000","last_clean":"2024-09-19T06:49:13.926577+0000","last_became_active":"2024-09-19T06:47:22.831132+0000","last_became_peered":"2024-09-19T06:47:22.831132+0000","last_unstale":"2024-09-19T06:49:13.926577+0000","last_undegraded":"2024-09-19T06:49:13.926577+0000","last_fullsized":"2024-09-19T06:49:13.926577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:49.867713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932598+0000","last_change":"2024-09-19T06:47:22.827742+0000","last_active":"2024-09-19T06:49:13.932598+0000","last_peered":"2024-09-19T06:49:13.932598+0000","last_clean":"2024-09-19T06:49:13.932598+0000","last_became_active":"2024-09-19T06:47:22.827371+0000","last_became_peered":"2024-09-19T06:47:22.827371+0000","last_unstale":"2024-09-19T06:49:13.932598+0000","last_undegraded":"2024-09-19T06:49:13.932598+0000","last_fullsized":"2024-09-19T06:49:13.932598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:35.153129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963305+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:49:13.963305+0000","last_peered":"2024-09-19T06:49:13.963305+0000","last_clean":"2024-09-19T06:49:13.963305+0000","last_became_active":"2024-09-19T06:47:24.841693+0000","last_became_peered":"2024-09-19T06:47:24.841693+0000","last_unstale":"2024-09-19T06:49:13.963305+0000","last_undegraded":"2024-09-19T06:49:13.963305+0000","last_fullsized":"2024-09-19T06:49:13.963305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:27.828296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.917388+0000","last_change":"2024-09-19T06:47:22.832531+0000","last_active":"2024-09-19T06:49:13.917388+0000","last_peered":"2024-09-19T06:49:13.917388+0000","last_clean":"2024-09-19T06:49:13.917388+0000","last_became_active":"2024-09-19T06:47:22.832213+0000","last_became_peered":"2024-09-19T06:47:22.832213+0000","last_unstale":"2024-09-19T06:49:13.917388+0000","last_undegraded":"2024-09-19T06:49:13.917388+0000","last_fullsized":"2024-09-19T06:49:13.917388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:46:03.168406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970720+0000","last_change":"2024-09-19T06:47:24.837690+0000","last_active":"2024-09-19T06:49:13.970720+0000","last_peered":"2024-09-19T06:49:13.970720+0000","last_clean":"2024-09-19T06:49:13.970720+0000","last_became_active":"2024-09-19T06:47:24.837389+0000","last_became_peered":"2024-09-19T06:47:24.837389+0000","last_unstale":"2024-09-19T06:49:13.970720+0000","last_undegraded":"2024-09-19T06:49:13.970720+0000","last_fullsized":"2024-09-19T06:49:13.970720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:20.653832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":228,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.604318+0000","last_change":"2024-09-19T06:47:22.833808+0000","last_active":"2024-09-19T06:49:14.604318+0000","last_peered":"2024-09-19T06:49:14.604318+0000","last_clean":"2024-09-19T06:49:14.604318+0000","last_became_active":"2024-09-19T06:47:22.832468+0000","last_became_peered":"2024-09-19T06:47:22.832468+0000","last_unstale":"2024-09-19T06:49:14.604318+0000","last_undegraded":"2024-09-19T06:49:14.604318+0000","last_fullsized":"2024-09-19T06:49:14.604318+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:39:31.131321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":198,"num_read_kb":198,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973091+0000","last_change":"2024-09-19T06:47:24.840928+0000","last_active":"2024-09-19T06:49:13.973091+0000","last_peered":"2024-09-19T06:49:13.973091+0000","last_clean":"2024-09-19T06:49:13.973091+0000","last_became_active":"2024-09-19T06:47:24.839928+0000","last_became_peered":"2024-09-19T06:47:24.839928+0000","last_unstale":"2024-09-19T06:49:13.973091+0000","last_undegraded":"2024-09-19T06:49:13.973091+0000","last_fullsized":"2024-09-19T06:49:13.973091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:29.486003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.928845+0000","last_change":"2024-09-19T06:47:22.833310+0000","last_active":"2024-09-19T06:49:13.928845+0000","last_peered":"2024-09-19T06:49:13.928845+0000","last_clean":"2024-09-19T06:49:13.928845+0000","last_became_active":"2024-09-19T06:47:22.832976+0000","last_became_peered":"2024-09-19T06:47:22.832976+0000","last_unstale":"2024-09-19T06:49:13.928845+0000","last_undegraded":"2024-09-19T06:49:13.928845+0000","last_fullsized":"2024-09-19T06:49:13.928845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:47.604853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969853+0000","last_change":"2024-09-19T06:47:24.848687+0000","last_active":"2024-09-19T06:49:13.969853+0000","last_peered":"2024-09-19T06:49:13.969853+0000","last_clean":"2024-09-19T06:49:13.969853+0000","last_became_active":"2024-09-19T06:47:24.848352+0000","last_became_peered":"2024-09-19T06:47:24.848352+0000","last_unstale":"2024-09-19T06:49:13.969853+0000","last_undegraded":"2024-09-19T06:49:13.969853+0000","last_fullsized":"2024-09-19T06:49:13.969853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:54.459206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.917489+0000","last_change":"2024-09-19T06:47:22.851479+0000","last_active":"2024-09-19T06:49:13.917489+0000","last_peered":"2024-09-19T06:49:13.917489+0000","last_clean":"2024-09-19T06:49:13.917489+0000","last_became_active":"2024-09-19T06:47:22.851125+0000","last_became_peered":"2024-09-19T06:47:22.851125+0000","last_unstale":"2024-09-19T06:49:13.917489+0000","last_undegraded":"2024-09-19T06:49:13.917489+0000","last_fullsized":"2024-09-19T06:49:13.917489+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:42.954034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973229+0000","last_change":"2024-09-19T06:47:24.851007+0000","last_active":"2024-09-19T06:49:13.973229+0000","last_peered":"2024-09-19T06:49:13.973229+0000","last_clean":"2024-09-19T06:49:13.973229+0000","last_became_active":"2024-09-19T06:47:24.850465+0000","last_became_peered":"2024-09-19T06:47:24.850465+0000","last_unstale":"2024-09-19T06:49:13.973229+0000","last_undegraded":"2024-09-19T06:49:13.973229+0000","last_fullsized":"2024-09-19T06:49:13.973229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:10.824672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":268,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.604918+0000","last_change":"2024-09-19T06:47:22.834893+0000","last_active":"2024-09-19T06:49:14.604918+0000","last_peered":"2024-09-19T06:49:14.604918+0000","last_clean":"2024-09-19T06:49:14.604918+0000","last_became_active":"2024-09-19T06:47:22.834521+0000","last_became_peered":"2024-09-19T06:47:22.834521+0000","last_unstale":"2024-09-19T06:49:14.604918+0000","last_undegraded":"2024-09-19T06:49:14.604918+0000","last_fullsized":"2024-09-19T06:49:14.604918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:29.742284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66347,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":208,"num_read_kb":168,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.957411+0000","last_change":"2024-09-19T06:47:24.848191+0000","last_active":"2024-09-19T06:49:13.957411+0000","last_peered":"2024-09-19T06:49:13.957411+0000","last_clean":"2024-09-19T06:49:13.957411+0000","last_became_active":"2024-09-19T06:47:24.847889+0000","last_became_peered":"2024-09-19T06:47:24.847889+0000","last_unstale":"2024-09-19T06:49:13.957411+0000","last_undegraded":"2024-09-19T06:49:13.957411+0000","last_fullsized":"2024-09-19T06:49:13.957411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:32.574360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.924532+0000","last_change":"2024-09-19T06:47:22.832005+0000","last_active":"2024-09-19T06:49:13.924532+0000","last_peered":"2024-09-19T06:49:13.924532+0000","last_clean":"2024-09-19T06:49:13.924532+0000","last_became_active":"2024-09-19T06:47:22.831342+0000","last_became_peered":"2024-09-19T06:47:22.831342+0000","last_unstale":"2024-09-19T06:49:13.924532+0000","last_undegraded":"2024-09-19T06:49:13.924532+0000","last_fullsized":"2024-09-19T06:49:13.924532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:53.444748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.964849+0000","last_change":"2024-09-19T06:47:24.843328+0000","last_active":"2024-09-19T06:49:13.964849+0000","last_peered":"2024-09-19T06:49:13.964849+0000","last_clean":"2024-09-19T06:49:13.964849+0000","last_became_active":"2024-09-19T06:47:24.842344+0000","last_became_peered":"2024-09-19T06:47:24.842344+0000","last_unstale":"2024-09-19T06:49:13.964849+0000","last_undegraded":"2024-09-19T06:49:13.964849+0000","last_fullsized":"2024-09-19T06:49:13.964849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:11.247819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.926463+0000","last_change":"2024-09-19T06:47:40.844086+0000","last_active":"2024-09-19T06:49:13.926463+0000","last_peered":"2024-09-19T06:49:13.926463+0000","last_clean":"2024-09-19T06:49:13.926463+0000","last_became_active":"2024-09-19T06:47:40.835015+0000","last_became_peered":"2024-09-19T06:47:40.835015+0000","last_unstale":"2024-09-19T06:49:13.926463+0000","last_undegraded":"2024-09-19T06:49:13.926463+0000","last_fullsized":"2024-09-19T06:49:13.926463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:09.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972972+0000","last_change":"2024-09-19T06:47:40.836466+0000","last_active":"2024-09-19T06:49:13.972972+0000","last_peered":"2024-09-19T06:49:13.972972+0000","last_clean":"2024-09-19T06:49:13.972972+0000","last_became_active":"2024-09-19T06:47:40.836237+0000","last_became_peered":"2024-09-19T06:47:40.836237+0000","last_unstale":"2024-09-19T06:49:13.972972+0000","last_undegraded":"2024-09-19T06:49:13.972972+0000","last_fullsized":"2024-09-19T06:49:13.972972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:00.094685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'10","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.606091+0000","last_change":"2024-09-19T06:47:22.830359+0000","last_active":"2024-09-19T06:49:14.606091+0000","last_peered":"2024-09-19T06:49:14.606091+0000","last_clean":"2024-09-19T06:49:14.606091+0000","last_became_active":"2024-09-19T06:47:22.829960+0000","last_became_peered":"2024-09-19T06:47:22.829960+0000","last_unstale":"2024-09-19T06:49:14.606091+0000","last_undegraded":"2024-09-19T06:49:14.606091+0000","last_fullsized":"2024-09-19T06:49:14.606091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:55.998243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":6,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971872+0000","last_change":"2024-09-19T06:47:24.836668+0000","last_active":"2024-09-19T06:49:13.971872+0000","last_peered":"2024-09-19T06:49:13.971872+0000","last_clean":"2024-09-19T06:49:13.971872+0000","last_became_active":"2024-09-19T06:47:24.836393+0000","last_became_peered":"2024-09-19T06:47:24.836393+0000","last_unstale":"2024-09-19T06:49:13.971872+0000","last_undegraded":"2024-09-19T06:49:13.971872+0000","last_fullsized":"2024-09-19T06:49:13.971872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:26.248948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":66,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.931451+0000","last_change":"2024-09-19T06:47:40.858323+0000","last_active":"2024-09-19T06:49:13.931451+0000","last_peered":"2024-09-19T06:49:13.931451+0000","last_clean":"2024-09-19T06:49:13.931451+0000","last_became_active":"2024-09-19T06:47:40.840802+0000","last_became_peered":"2024-09-19T06:47:40.840802+0000","last_unstale":"2024-09-19T06:49:13.931451+0000","last_undegraded":"2024-09-19T06:49:13.931451+0000","last_fullsized":"2024-09-19T06:49:13.931451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:14.666916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970280+0000","last_change":"2024-09-19T06:47:24.843681+0000","last_active":"2024-09-19T06:49:13.970280+0000","last_peered":"2024-09-19T06:49:13.970280+0000","last_clean":"2024-09-19T06:49:13.970280+0000","last_became_active":"2024-09-19T06:47:24.841228+0000","last_became_peered":"2024-09-19T06:47:24.841228+0000","last_unstale":"2024-09-19T06:49:13.970280+0000","last_undegraded":"2024-09-19T06:49:13.970280+0000","last_fullsized":"2024-09-19T06:49:13.970280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:56.418962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.933661+0000","last_change":"2024-09-19T06:47:22.833893+0000","last_active":"2024-09-19T06:49:13.933661+0000","last_peered":"2024-09-19T06:49:13.933661+0000","last_clean":"2024-09-19T06:49:13.933661+0000","last_became_active":"2024-09-19T06:47:22.833217+0000","last_became_peered":"2024-09-19T06:47:22.833217+0000","last_unstale":"2024-09-19T06:49:13.933661+0000","last_undegraded":"2024-09-19T06:49:13.933661+0000","last_fullsized":"2024-09-19T06:49:13.933661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:19.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955096+0000","last_change":"2024-09-19T06:47:24.848399+0000","last_active":"2024-09-19T06:49:13.955096+0000","last_peered":"2024-09-19T06:49:13.955096+0000","last_clean":"2024-09-19T06:49:13.955096+0000","last_became_active":"2024-09-19T06:47:24.848081+0000","last_became_peered":"2024-09-19T06:47:24.848081+0000","last_unstale":"2024-09-19T06:49:13.955096+0000","last_undegraded":"2024-09-19T06:49:13.955096+0000","last_fullsized":"2024-09-19T06:49:13.955096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:44.292939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.931968+0000","last_change":"2024-09-19T06:47:40.860955+0000","last_active":"2024-09-19T06:49:13.931968+0000","last_peered":"2024-09-19T06:49:13.931968+0000","last_clean":"2024-09-19T06:49:13.931968+0000","last_became_active":"2024-09-19T06:47:40.835126+0000","last_became_peered":"2024-09-19T06:47:40.835126+0000","last_unstale":"2024-09-19T06:49:13.931968+0000","last_undegraded":"2024-09-19T06:49:13.931968+0000","last_fullsized":"2024-09-19T06:49:13.931968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:02:26.763229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971391+0000","last_change":"2024-09-19T06:47:24.849680+0000","last_active":"2024-09-19T06:49:13.971391+0000","last_peered":"2024-09-19T06:49:13.971391+0000","last_clean":"2024-09-19T06:49:13.971391+0000","last_became_active":"2024-09-19T06:47:24.849323+0000","last_became_peered":"2024-09-19T06:47:24.849323+0000","last_unstale":"2024-09-19T06:49:13.971391+0000","last_undegraded":"2024-09-19T06:49:13.971391+0000","last_fullsized":"2024-09-19T06:49:13.971391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:59.763503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'41","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.148665+0000","last_change":"2024-09-19T06:47:22.831689+0000","last_active":"2024-09-19T06:49:14.148665+0000","last_peered":"2024-09-19T06:49:14.148665+0000","last_clean":"2024-09-19T06:49:14.148665+0000","last_became_active":"2024-09-19T06:47:22.829283+0000","last_became_peered":"2024-09-19T06:47:22.829283+0000","last_unstale":"2024-09-19T06:49:14.148665+0000","last_undegraded":"2024-09-19T06:49:14.148665+0000","last_fullsized":"2024-09-19T06:49:14.148665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:37.109542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971792+0000","last_change":"2024-09-19T06:47:24.842793+0000","last_active":"2024-09-19T06:49:13.971792+0000","last_peered":"2024-09-19T06:49:13.971792+0000","last_clean":"2024-09-19T06:49:13.971792+0000","last_became_active":"2024-09-19T06:47:24.842474+0000","last_became_peered":"2024-09-19T06:47:24.842474+0000","last_unstale":"2024-09-19T06:49:13.971792+0000","last_undegraded":"2024-09-19T06:49:13.971792+0000","last_fullsized":"2024-09-19T06:49:13.971792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:23.110736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'9","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.605146+0000","last_change":"2024-09-19T06:47:23.047356+0000","last_active":"2024-09-19T06:49:14.605146+0000","last_peered":"2024-09-19T06:49:14.605146+0000","last_clean":"2024-09-19T06:49:14.605146+0000","last_became_active":"2024-09-19T06:47:23.046941+0000","last_became_peered":"2024-09-19T06:47:23.046941+0000","last_unstale":"2024-09-19T06:49:14.605146+0000","last_undegraded":"2024-09-19T06:49:14.605146+0000","last_fullsized":"2024-09-19T06:49:14.605146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:43.490952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":4,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971243+0000","last_change":"2024-09-19T06:47:24.851025+0000","last_active":"2024-09-19T06:49:13.971243+0000","last_peered":"2024-09-19T06:49:13.971243+0000","last_clean":"2024-09-19T06:49:13.971243+0000","last_became_active":"2024-09-19T06:47:24.850681+0000","last_became_peered":"2024-09-19T06:47:24.850681+0000","last_unstale":"2024-09-19T06:49:13.971243+0000","last_undegraded":"2024-09-19T06:49:13.971243+0000","last_fullsized":"2024-09-19T06:49:13.971243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:46.884747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":348,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.928776+0000","last_change":"2024-09-19T06:46:13.927379+0000","last_active":"2024-09-19T06:49:13.928776+0000","last_peered":"2024-09-19T06:49:13.928776+0000","last_clean":"2024-09-19T06:49:13.928776+0000","last_became_active":"2024-09-19T06:46:13.926926+0000","last_became_peered":"2024-09-19T06:46:13.926926+0000","last_unstale":"2024-09-19T06:49:13.928776+0000","last_undegraded":"2024-09-19T06:49:13.928776+0000","last_fullsized":"2024-09-19T06:49:13.928776+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_clean_scrub_stamp":"2024-09-19T06:42:52.051335+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:29.542128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.927403+0000","last_change":"2024-09-19T06:47:22.830387+0000","last_active":"2024-09-19T06:49:13.927403+0000","last_peered":"2024-09-19T06:49:13.927403+0000","last_clean":"2024-09-19T06:49:13.927403+0000","last_became_active":"2024-09-19T06:47:22.829970+0000","last_became_peered":"2024-09-19T06:47:22.829970+0000","last_unstale":"2024-09-19T06:49:13.927403+0000","last_undegraded":"2024-09-19T06:49:13.927403+0000","last_fullsized":"2024-09-19T06:49:13.927403+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:51.385456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.974671+0000","last_change":"2024-09-19T06:47:24.843752+0000","last_active":"2024-09-19T06:49:13.974671+0000","last_peered":"2024-09-19T06:49:13.974671+0000","last_clean":"2024-09-19T06:49:13.974671+0000","last_became_active":"2024-09-19T06:47:24.842631+0000","last_became_peered":"2024-09-19T06:47:24.842631+0000","last_unstale":"2024-09-19T06:49:13.974671+0000","last_undegraded":"2024-09-19T06:49:13.974671+0000","last_fullsized":"2024-09-19T06:49:13.974671+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:19.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.928732+0000","last_change":"2024-09-19T06:47:22.832647+0000","last_active":"2024-09-19T06:49:13.928732+0000","last_peered":"2024-09-19T06:49:13.928732+0000","last_clean":"2024-09-19T06:49:13.928732+0000","last_became_active":"2024-09-19T06:47:22.831774+0000","last_became_peered":"2024-09-19T06:47:22.831774+0000","last_unstale":"2024-09-19T06:49:13.928732+0000","last_undegraded":"2024-09-19T06:49:13.928732+0000","last_fullsized":"2024-09-19T06:49:13.928732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:57.824610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.948364+0000","last_change":"2024-09-19T06:47:24.844301+0000","last_active":"2024-09-19T06:49:13.948364+0000","last_peered":"2024-09-19T06:49:13.948364+0000","last_clean":"2024-09-19T06:49:13.948364+0000","last_became_active":"2024-09-19T06:47:24.839748+0000","last_became_peered":"2024-09-19T06:47:24.839748+0000","last_unstale":"2024-09-19T06:49:13.948364+0000","last_undegraded":"2024-09-19T06:49:13.948364+0000","last_fullsized":"2024-09-19T06:49:13.948364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:46.407693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":130,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932811+0000","last_change":"2024-09-19T06:47:22.851008+0000","last_active":"2024-09-19T06:49:13.932811+0000","last_peered":"2024-09-19T06:49:13.932811+0000","last_clean":"2024-09-19T06:49:13.932811+0000","last_became_active":"2024-09-19T06:47:22.850601+0000","last_became_peered":"2024-09-19T06:47:22.850601+0000","last_unstale":"2024-09-19T06:49:13.932811+0000","last_undegraded":"2024-09-19T06:49:13.932811+0000","last_fullsized":"2024-09-19T06:49:13.932811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:04.610277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970407+0000","last_change":"2024-09-19T06:47:24.839684+0000","last_active":"2024-09-19T06:49:13.970407+0000","last_peered":"2024-09-19T06:49:13.970407+0000","last_clean":"2024-09-19T06:49:13.970407+0000","last_became_active":"2024-09-19T06:47:24.839126+0000","last_became_peered":"2024-09-19T06:47:24.839126+0000","last_unstale":"2024-09-19T06:49:13.970407+0000","last_undegraded":"2024-09-19T06:49:13.970407+0000","last_fullsized":"2024-09-19T06:49:13.970407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:09.458535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":130,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.314141+0000","last_change":"2024-09-19T06:47:22.831819+0000","last_active":"2024-09-19T06:49:14.314141+0000","last_peered":"2024-09-19T06:49:14.314141+0000","last_clean":"2024-09-19T06:49:14.314141+0000","last_became_active":"2024-09-19T06:47:22.831332+0000","last_became_peered":"2024-09-19T06:47:22.831332+0000","last_unstale":"2024-09-19T06:49:14.314141+0000","last_undegraded":"2024-09-19T06:49:14.314141+0000","last_fullsized":"2024-09-19T06:49:14.314141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:35.032949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.965109+0000","last_change":"2024-09-19T06:47:24.846738+0000","last_active":"2024-09-19T06:49:13.965109+0000","last_peered":"2024-09-19T06:49:13.965109+0000","last_clean":"2024-09-19T06:49:13.965109+0000","last_became_active":"2024-09-19T06:47:24.846389+0000","last_became_peered":"2024-09-19T06:47:24.846389+0000","last_unstale":"2024-09-19T06:49:13.965109+0000","last_undegraded":"2024-09-19T06:49:13.965109+0000","last_fullsized":"2024-09-19T06:49:13.965109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:19.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.918173+0000","last_change":"2024-09-19T06:47:22.851065+0000","last_active":"2024-09-19T06:49:13.918173+0000","last_peered":"2024-09-19T06:49:13.918173+0000","last_clean":"2024-09-19T06:49:13.918173+0000","last_became_active":"2024-09-19T06:47:22.850647+0000","last_became_peered":"2024-09-19T06:47:22.850647+0000","last_unstale":"2024-09-19T06:49:13.918173+0000","last_undegraded":"2024-09-19T06:49:13.918173+0000","last_fullsized":"2024-09-19T06:49:13.918173+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:41.993501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966795+0000","last_change":"2024-09-19T06:47:24.838905+0000","last_active":"2024-09-19T06:49:13.966795+0000","last_peered":"2024-09-19T06:49:13.966795+0000","last_clean":"2024-09-19T06:49:13.966795+0000","last_became_active":"2024-09-19T06:47:24.838443+0000","last_became_peered":"2024-09-19T06:47:24.838443+0000","last_unstale":"2024-09-19T06:49:13.966795+0000","last_undegraded":"2024-09-19T06:49:13.966795+0000","last_fullsized":"2024-09-19T06:49:13.966795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:12.486137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932653+0000","last_change":"2024-09-19T06:47:22.851431+0000","last_active":"2024-09-19T06:49:13.932653+0000","last_peered":"2024-09-19T06:49:13.932653+0000","last_clean":"2024-09-19T06:49:13.932653+0000","last_became_active":"2024-09-19T06:47:22.851092+0000","last_became_peered":"2024-09-19T06:47:22.851092+0000","last_unstale":"2024-09-19T06:49:13.932653+0000","last_undegraded":"2024-09-19T06:49:13.932653+0000","last_fullsized":"2024-09-19T06:49:13.932653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:26.183803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959143+0000","last_change":"2024-09-19T06:47:24.839046+0000","last_active":"2024-09-19T06:49:13.959143+0000","last_peered":"2024-09-19T06:49:13.959143+0000","last_clean":"2024-09-19T06:49:13.959143+0000","last_became_active":"2024-09-19T06:47:24.838680+0000","last_became_peered":"2024-09-19T06:47:24.838680+0000","last_unstale":"2024-09-19T06:49:13.959143+0000","last_undegraded":"2024-09-19T06:49:13.959143+0000","last_fullsized":"2024-09-19T06:49:13.959143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:45.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'50","reported_seq":113,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.156330+0000","last_change":"2024-09-19T06:47:40.865241+0000","last_active":"2024-09-19T06:49:14.156330+0000","last_peered":"2024-09-19T06:49:14.156330+0000","last_clean":"2024-09-19T06:49:14.156330+0000","last_became_active":"2024-09-19T06:47:40.837765+0000","last_became_peered":"2024-09-19T06:47:40.837765+0000","last_unstale":"2024-09-19T06:49:14.156330+0000","last_undegraded":"2024-09-19T06:49:14.156330+0000","last_fullsized":"2024-09-19T06:49:14.156330+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966118+0000","last_change":"2024-09-19T06:47:24.848599+0000","last_active":"2024-09-19T06:49:13.966118+0000","last_peered":"2024-09-19T06:49:13.966118+0000","last_clean":"2024-09-19T06:49:13.966118+0000","last_became_active":"2024-09-19T06:47:24.848305+0000","last_became_peered":"2024-09-19T06:47:24.848305+0000","last_unstale":"2024-09-19T06:49:13.966118+0000","last_undegraded":"2024-09-19T06:49:13.966118+0000","last_fullsized":"2024-09-19T06:49:13.966118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:31.496188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.927755+0000","last_change":"2024-09-19T06:47:23.047484+0000","last_active":"2024-09-19T06:49:13.927755+0000","last_peered":"2024-09-19T06:49:13.927755+0000","last_clean":"2024-09-19T06:49:13.927755+0000","last_became_active":"2024-09-19T06:47:23.047058+0000","last_became_peered":"2024-09-19T06:47:23.047058+0000","last_unstale":"2024-09-19T06:49:13.927755+0000","last_undegraded":"2024-09-19T06:49:13.927755+0000","last_fullsized":"2024-09-19T06:49:13.927755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:34.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970265+0000","last_change":"2024-09-19T06:47:24.843498+0000","last_active":"2024-09-19T06:49:13.970265+0000","last_peered":"2024-09-19T06:49:13.970265+0000","last_clean":"2024-09-19T06:49:13.970265+0000","last_became_active":"2024-09-19T06:47:24.843204+0000","last_became_peered":"2024-09-19T06:47:24.843204+0000","last_unstale":"2024-09-19T06:49:13.970265+0000","last_undegraded":"2024-09-19T06:49:13.970265+0000","last_fullsized":"2024-09-19T06:49:13.970265+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:25.585023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.919373+0000","last_change":"2024-09-19T06:47:22.851519+0000","last_active":"2024-09-19T06:49:13.919373+0000","last_peered":"2024-09-19T06:49:13.919373+0000","last_clean":"2024-09-19T06:49:13.919373+0000","last_became_active":"2024-09-19T06:47:22.851163+0000","last_became_peered":"2024-09-19T06:47:22.851163+0000","last_unstale":"2024-09-19T06:49:13.919373+0000","last_undegraded":"2024-09-19T06:49:13.919373+0000","last_fullsized":"2024-09-19T06:49:13.919373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:04.502795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.945308+0000","last_change":"2024-09-19T06:47:24.845239+0000","last_active":"2024-09-19T06:49:13.945308+0000","last_peered":"2024-09-19T06:49:13.945308+0000","last_clean":"2024-09-19T06:49:13.945308+0000","last_became_active":"2024-09-19T06:47:24.844934+0000","last_became_peered":"2024-09-19T06:47:24.844934+0000","last_unstale":"2024-09-19T06:49:13.945308+0000","last_undegraded":"2024-09-19T06:49:13.945308+0000","last_fullsized":"2024-09-19T06:49:13.945308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:43.610604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.934380+0000","last_change":"2024-09-19T06:47:22.830232+0000","last_active":"2024-09-19T06:49:13.934380+0000","last_peered":"2024-09-19T06:49:13.934380+0000","last_clean":"2024-09-19T06:49:13.934380+0000","last_became_active":"2024-09-19T06:47:22.829901+0000","last_became_peered":"2024-09-19T06:47:22.829901+0000","last_unstale":"2024-09-19T06:49:13.934380+0000","last_undegraded":"2024-09-19T06:49:13.934380+0000","last_fullsized":"2024-09-19T06:49:13.934380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:12.251427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955845+0000","last_change":"2024-09-19T06:47:24.839304+0000","last_active":"2024-09-19T06:49:13.955845+0000","last_peered":"2024-09-19T06:49:13.955845+0000","last_clean":"2024-09-19T06:49:13.955845+0000","last_became_active":"2024-09-19T06:47:24.838986+0000","last_became_peered":"2024-09-19T06:47:24.838986+0000","last_unstale":"2024-09-19T06:49:13.955845+0000","last_undegraded":"2024-09-19T06:49:13.955845+0000","last_fullsized":"2024-09-19T06:49:13.955845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:39.377886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.885319+0000","last_change":"2024-09-19T06:47:22.828553+0000","last_active":"2024-09-19T06:49:14.885319+0000","last_peered":"2024-09-19T06:49:14.885319+0000","last_clean":"2024-09-19T06:49:14.885319+0000","last_became_active":"2024-09-19T06:47:22.828213+0000","last_became_peered":"2024-09-19T06:47:22.828213+0000","last_unstale":"2024-09-19T06:49:14.885319+0000","last_undegraded":"2024-09-19T06:49:14.885319+0000","last_fullsized":"2024-09-19T06:49:14.885319+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:31.649761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960387+0000","last_change":"2024-09-19T06:47:24.840747+0000","last_active":"2024-09-19T06:49:13.960387+0000","last_peered":"2024-09-19T06:49:13.960387+0000","last_clean":"2024-09-19T06:49:13.960387+0000","last_became_active":"2024-09-19T06:47:24.840451+0000","last_became_peered":"2024-09-19T06:47:24.840451+0000","last_unstale":"2024-09-19T06:49:13.960387+0000","last_undegraded":"2024-09-19T06:49:13.960387+0000","last_fullsized":"2024-09-19T06:49:13.960387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:41.588751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.933845+0000","last_change":"2024-09-19T06:47:22.831697+0000","last_active":"2024-09-19T06:49:13.933845+0000","last_peered":"2024-09-19T06:49:13.933845+0000","last_clean":"2024-09-19T06:49:13.933845+0000","last_became_active":"2024-09-19T06:47:22.831178+0000","last_became_peered":"2024-09-19T06:47:22.831178+0000","last_unstale":"2024-09-19T06:49:13.933845+0000","last_undegraded":"2024-09-19T06:49:13.933845+0000","last_fullsized":"2024-09-19T06:49:13.933845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:30.975782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971917+0000","last_change":"2024-09-19T06:47:24.845097+0000","last_active":"2024-09-19T06:49:13.971917+0000","last_peered":"2024-09-19T06:49:13.971917+0000","last_clean":"2024-09-19T06:49:13.971917+0000","last_became_active":"2024-09-19T06:47:24.844801+0000","last_became_peered":"2024-09-19T06:47:24.844801+0000","last_unstale":"2024-09-19T06:49:13.971917+0000","last_undegraded":"2024-09-19T06:49:13.971917+0000","last_fullsized":"2024-09-19T06:49:13.971917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:15.264118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971733+0000","last_change":"2024-09-19T06:47:24.851372+0000","last_active":"2024-09-19T06:49:13.971733+0000","last_peered":"2024-09-19T06:49:13.971733+0000","last_clean":"2024-09-19T06:49:13.971733+0000","last_became_active":"2024-09-19T06:47:24.851044+0000","last_became_peered":"2024-09-19T06:47:24.851044+0000","last_unstale":"2024-09-19T06:49:13.971733+0000","last_undegraded":"2024-09-19T06:49:13.971733+0000","last_fullsized":"2024-09-19T06:49:13.971733+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:55.658214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932580+0000","last_change":"2024-09-19T06:47:22.833694+0000","last_active":"2024-09-19T06:49:13.932580+0000","last_peered":"2024-09-19T06:49:13.932580+0000","last_clean":"2024-09-19T06:49:13.932580+0000","last_became_active":"2024-09-19T06:47:22.833215+0000","last_became_peered":"2024-09-19T06:47:22.833215+0000","last_unstale":"2024-09-19T06:49:13.932580+0000","last_undegraded":"2024-09-19T06:49:13.932580+0000","last_fullsized":"2024-09-19T06:49:13.932580+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:32.171522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973551+0000","last_change":"2024-09-19T06:47:24.845737+0000","last_active":"2024-09-19T06:49:13.973551+0000","last_peered":"2024-09-19T06:49:13.973551+0000","last_clean":"2024-09-19T06:49:13.973551+0000","last_became_active":"2024-09-19T06:47:24.845242+0000","last_became_peered":"2024-09-19T06:47:24.845242+0000","last_unstale":"2024-09-19T06:49:13.973551+0000","last_undegraded":"2024-09-19T06:49:13.973551+0000","last_fullsized":"2024-09-19T06:49:13.973551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:51.554946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932499+0000","last_change":"2024-09-19T06:47:22.829869+0000","last_active":"2024-09-19T06:49:13.932499+0000","last_peered":"2024-09-19T06:49:13.932499+0000","last_clean":"2024-09-19T06:49:13.932499+0000","last_became_active":"2024-09-19T06:47:22.829522+0000","last_became_peered":"2024-09-19T06:47:22.829522+0000","last_unstale":"2024-09-19T06:49:13.932499+0000","last_undegraded":"2024-09-19T06:49:13.932499+0000","last_fullsized":"2024-09-19T06:49:13.932499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:26.871434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973419+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:49:13.973419+0000","last_peered":"2024-09-19T06:49:13.973419+0000","last_clean":"2024-09-19T06:49:13.973419+0000","last_became_active":"2024-09-19T06:47:24.841289+0000","last_became_peered":"2024-09-19T06:47:24.841289+0000","last_unstale":"2024-09-19T06:49:13.973419+0000","last_undegraded":"2024-09-19T06:49:13.973419+0000","last_fullsized":"2024-09-19T06:49:13.973419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:21:12.611458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.937218+0000","last_change":"2024-09-19T06:47:22.852198+0000","last_active":"2024-09-19T06:49:13.937218+0000","last_peered":"2024-09-19T06:49:13.937218+0000","last_clean":"2024-09-19T06:49:13.937218+0000","last_became_active":"2024-09-19T06:47:22.850984+0000","last_became_peered":"2024-09-19T06:47:22.850984+0000","last_unstale":"2024-09-19T06:49:13.937218+0000","last_undegraded":"2024-09-19T06:49:13.937218+0000","last_fullsized":"2024-09-19T06:49:13.937218+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:13.620145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959682+0000","last_change":"2024-09-19T06:47:40.837614+0000","last_active":"2024-09-19T06:49:13.959682+0000","last_peered":"2024-09-19T06:49:13.959682+0000","last_clean":"2024-09-19T06:49:13.959682+0000","last_became_active":"2024-09-19T06:47:40.837259+0000","last_became_peered":"2024-09-19T06:47:40.837259+0000","last_unstale":"2024-09-19T06:49:13.959682+0000","last_undegraded":"2024-09-19T06:49:13.959682+0000","last_fullsized":"2024-09-19T06:49:13.959682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:41.429962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":157,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.197340+0000","last_change":"2024-09-19T06:47:41.182639+0000","last_active":"2024-09-19T06:49:14.197340+0000","last_peered":"2024-09-19T06:49:14.197340+0000","last_clean":"2024-09-19T06:49:14.197340+0000","last_became_active":"2024-09-19T06:47:41.174305+0000","last_became_peered":"2024-09-19T06:47:41.174305+0000","last_unstale":"2024-09-19T06:49:14.197340+0000","last_undegraded":"2024-09-19T06:49:14.197340+0000","last_fullsized":"2024-09-19T06:49:14.197340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:10.199502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601319,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1124,"num_read_kb":676,"num_write":379,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1803957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579939,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32472,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712680,"statfs":{"total":95995035648,"available":95961784320,"internally_reserved":0,"allocated":1314816,"data_stored":842278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28864,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716288,"statfs":{"total":95995035648,"available":95965478912,"internally_reserved":0,"allocated":1761280,"data_stored":1238256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972397,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28332,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716820,"statfs":{"total":95995035648,"available":95966023680,"internally_reserved":0,"allocated":1335296,"data_stored":840798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234034,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":447288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462966,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32688,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712464,"statfs":{"total":95995035648,"available":95961563136,"internally_reserved":0,"allocated":1536000,"data_stored":1042501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626492,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":447692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855423,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1171668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018949,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":448114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215174,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32620,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712532,"statfs":{"total":95995035648,"available":95961632768,"internally_reserved":0,"allocated":1470464,"data_stored":971940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444112,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":516380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1003520,"data_stored":515624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":449172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":134723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":201311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":395818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:49:19.086 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-19T06:49:19.086 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T06:49:19.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.222+0000 7efe8afa6640 1 -- 172.21.15.103:0/3229835429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe84106990 msgr2=0x7efe84106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:19.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.222+0000 7efe8afa6640 1 --2- 172.21.15.103:0/3229835429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe84106990 0x7efe84106d70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efe78009930 tx=0x7efe7802a420 comp rx=0 tx=0).stop 2024-09-19T06:49:19.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.222+0000 7efe8afa6640 1 -- 172.21.15.103:0/3229835429 shutdown_connections 2024-09-19T06:49:19.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.222+0000 7efe8afa6640 1 --2- 172.21.15.103:0/3229835429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe84101320 0x7efe8410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:19.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.222+0000 7efe8afa6640 1 --2- 172.21.15.103:0/3229835429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe84100980 0x7efe84100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:19.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.222+0000 7efe8afa6640 1 --2- 172.21.15.103:0/3229835429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe84106990 0x7efe84106d70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:19.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.222+0000 7efe8afa6640 1 -- 172.21.15.103:0/3229835429 >> 172.21.15.103:0/3229835429 conn(0x7efe84077a10 msgr2=0x7efe840fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:19.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.222+0000 7efe8afa6640 1 -- 172.21.15.103:0/3229835429 shutdown_connections 2024-09-19T06:49:19.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.222+0000 7efe8afa6640 1 -- 172.21.15.103:0/3229835429 wait complete. 2024-09-19T06:49:19.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.222+0000 7efe8afa6640 1 Processor -- start 2024-09-19T06:49:19.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.222+0000 7efe8afa6640 1 -- start start 2024-09-19T06:49:19.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.222+0000 7efe8afa6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe84100980 0x7efe84197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:19.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe8afa6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe84101320 0x7efe84197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:19.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe8afa6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe84106990 0x7efe8419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:19.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe8afa6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe84110dd0 con 0x7efe84101320 2024-09-19T06:49:19.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe8afa6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efe84110c50 con 0x7efe84106990 2024-09-19T06:49:19.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe8afa6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efe84110f50 con 0x7efe84100980 2024-09-19T06:49:19.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe83fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe84101320 0x7efe84197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:19.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe83fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe84101320 0x7efe84197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42400/0 (socket says 172.21.15.103:42400) 2024-09-19T06:49:19.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe83fff640 1 -- 172.21.15.103:0/3505549161 learned_addr learned my addr 172.21.15.103:0/3505549161 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:19.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe88d42640 1 --2- 172.21.15.103:0/3505549161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe84100980 0x7efe84197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:19.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe89543640 1 --2- 172.21.15.103:0/3505549161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe84106990 0x7efe8419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:19.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe83fff640 1 -- 172.21.15.103:0/3505549161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe84100980 msgr2=0x7efe84197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:19.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe83fff640 1 --2- 172.21.15.103:0/3505549161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe84100980 0x7efe84197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:19.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe83fff640 1 -- 172.21.15.103:0/3505549161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe84106990 msgr2=0x7efe8419bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:19.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe83fff640 1 --2- 172.21.15.103:0/3505549161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe84106990 0x7efe8419bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:19.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe83fff640 1 -- 172.21.15.103:0/3505549161 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8419c310 con 0x7efe84101320 2024-09-19T06:49:19.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe83fff640 1 --2- 172.21.15.103:0/3505549161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe84101320 0x7efe84197860 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efe7000c890 tx=0x7efe7000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:19.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe81ffb640 1 -- 172.21.15.103:0/3505549161 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70004220 con 0x7efe84101320 2024-09-19T06:49:19.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe8afa6640 1 -- 172.21.15.103:0/3505549161 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8419c600 con 0x7efe84101320 2024-09-19T06:49:19.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe81ffb640 1 -- 172.21.15.103:0/3505549161 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe700043c0 con 0x7efe84101320 2024-09-19T06:49:19.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.223+0000 7efe81ffb640 1 -- 172.21.15.103:0/3505549161 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70004a00 con 0x7efe84101320 2024-09-19T06:49:19.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.224+0000 7efe8afa6640 1 -- 172.21.15.103:0/3505549161 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe841a3e90 con 0x7efe84101320 2024-09-19T06:49:19.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.224+0000 7efe8afa6640 1 -- 172.21.15.103:0/3505549161 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe84100de0 con 0x7efe84101320 2024-09-19T06:49:19.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.225+0000 7efe81ffb640 1 -- 172.21.15.103:0/3505549161 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7efe700079b0 con 0x7efe84101320 2024-09-19T06:49:19.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.226+0000 7efe81ffb640 1 --2- 172.21.15.103:0/3505549161 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efe740a34e0 0x7efe740a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:19.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.226+0000 7efe88d42640 1 --2- 172.21.15.103:0/3505549161 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efe740a34e0 0x7efe740a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:19.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.226+0000 7efe81ffb640 1 -- 172.21.15.103:0/3505549161 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7efe700cfcc0 con 0x7efe84101320 2024-09-19T06:49:19.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.227+0000 7efe88d42640 1 --2- 172.21.15.103:0/3505549161 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efe740a34e0 0x7efe740a59a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efe78009900 tx=0x7efe780058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:19.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.230+0000 7efe81ffb640 1 -- 172.21.15.103:0/3505549161 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efe70014030 con 0x7efe84101320 2024-09-19T06:49:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:18 smithi174 ceph-mon[35637]: Updating smithi134:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:49:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:18 smithi174 ceph-mon[35637]: Updating smithi174:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:49:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:18 smithi174 ceph-mon[35637]: Updating smithi103:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:49:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:18 smithi174 ceph-mon[35637]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:49:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:18 smithi174 ceph-mon[35637]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:49:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:18 smithi174 ceph-mon[35637]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:49:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:18 smithi174 ceph-mon[35637]: Updating smithi134:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:49:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:18 smithi174 ceph-mon[35637]: Updating smithi174:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:49:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:18 smithi174 ceph-mon[35637]: Updating smithi103:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:49:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:18 smithi174 ceph-mon[35637]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:49:19.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:18 smithi174 ceph-mon[35637]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-09-19T06:49:19.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.393+0000 7efe8afa6640 1 -- 172.21.15.103:0/3505549161 --> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe84108f50 con 0x7efe740a34e0 2024-09-19T06:49:19.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.399+0000 7efe81ffb640 1 -- 172.21.15.103:0/3505549161 <== mgr.15210 v2:172.21.15.103:6800/82882809 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257913 (secure 0 0 0) 0x7efe84108f50 con 0x7efe740a34e0 2024-09-19T06:49:19.403 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:49:19.404 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T06:49:19.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.402+0000 7efe8afa6640 1 -- 172.21.15.103:0/3505549161 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efe740a34e0 msgr2=0x7efe740a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:19.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.402+0000 7efe8afa6640 1 --2- 172.21.15.103:0/3505549161 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efe740a34e0 0x7efe740a59a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efe78009900 tx=0x7efe780058d0 comp rx=0 tx=0).stop 2024-09-19T06:49:19.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.402+0000 7efe8afa6640 1 -- 172.21.15.103:0/3505549161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe84101320 msgr2=0x7efe84197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:19.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.402+0000 7efe8afa6640 1 --2- 172.21.15.103:0/3505549161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe84101320 0x7efe84197860 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efe7000c890 tx=0x7efe7000cd50 comp rx=0 tx=0).stop 2024-09-19T06:49:19.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.402+0000 7efe8afa6640 1 -- 172.21.15.103:0/3505549161 shutdown_connections 2024-09-19T06:49:19.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.402+0000 7efe8afa6640 1 --2- 172.21.15.103:0/3505549161 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efe740a34e0 0x7efe740a59a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:19.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.402+0000 7efe8afa6640 1 --2- 172.21.15.103:0/3505549161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe84106990 0x7efe8419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:19.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.402+0000 7efe8afa6640 1 --2- 172.21.15.103:0/3505549161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe84101320 0x7efe84197860 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:19.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.402+0000 7efe8afa6640 1 --2- 172.21.15.103:0/3505549161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe84100980 0x7efe84197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:19.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.402+0000 7efe8afa6640 1 -- 172.21.15.103:0/3505549161 >> 172.21.15.103:0/3505549161 conn(0x7efe84077a10 msgr2=0x7efe84103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:19.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.402+0000 7efe8afa6640 1 -- 172.21.15.103:0/3505549161 shutdown_connections 2024-09-19T06:49:19.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:19.402+0000 7efe8afa6640 1 -- 172.21.15.103:0/3505549161 wait complete. 2024-09-19T06:49:19.414 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T06:49:17.932591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191921,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1260,"num_read_kb":795,"num_write":562,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":1181610,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":943,"ondisk_log_size":943,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355580,"kb_used_data":14496,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124586244,"statfs":{"total":1151940427776,"available":1151576313856,"internally_reserved":0,"allocated":14843904,"data_stored":8931711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.991091"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960112+0000","last_change":"2024-09-19T06:47:24.853118+0000","last_active":"2024-09-19T06:49:13.960112+0000","last_peered":"2024-09-19T06:49:13.960112+0000","last_clean":"2024-09-19T06:49:13.960112+0000","last_became_active":"2024-09-19T06:47:24.852801+0000","last_became_peered":"2024-09-19T06:47:24.852801+0000","last_unstale":"2024-09-19T06:49:13.960112+0000","last_undegraded":"2024-09-19T06:49:13.960112+0000","last_fullsized":"2024-09-19T06:49:13.960112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:16.729103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.957024+0000","last_change":"2024-09-19T06:47:24.841095+0000","last_active":"2024-09-19T06:49:13.957024+0000","last_peered":"2024-09-19T06:49:13.957024+0000","last_clean":"2024-09-19T06:49:13.957024+0000","last_became_active":"2024-09-19T06:47:24.840774+0000","last_became_peered":"2024-09-19T06:47:24.840774+0000","last_unstale":"2024-09-19T06:49:13.957024+0000","last_undegraded":"2024-09-19T06:49:13.957024+0000","last_fullsized":"2024-09-19T06:49:13.957024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:27.262766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973374+0000","last_change":"2024-09-19T06:47:24.848444+0000","last_active":"2024-09-19T06:49:13.973374+0000","last_peered":"2024-09-19T06:49:13.973374+0000","last_clean":"2024-09-19T06:49:13.973374+0000","last_became_active":"2024-09-19T06:47:24.847354+0000","last_became_peered":"2024-09-19T06:47:24.847354+0000","last_unstale":"2024-09-19T06:49:13.973374+0000","last_undegraded":"2024-09-19T06:49:13.973374+0000","last_fullsized":"2024-09-19T06:49:13.973374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:44.901265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.949178+0000","last_change":"2024-09-19T06:47:24.851915+0000","last_active":"2024-09-19T06:49:13.949178+0000","last_peered":"2024-09-19T06:49:13.949178+0000","last_clean":"2024-09-19T06:49:13.949178+0000","last_became_active":"2024-09-19T06:47:24.851600+0000","last_became_peered":"2024-09-19T06:47:24.851600+0000","last_unstale":"2024-09-19T06:49:13.949178+0000","last_undegraded":"2024-09-19T06:49:13.949178+0000","last_fullsized":"2024-09-19T06:49:13.949178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:52.424288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.950906+0000","last_change":"2024-09-19T06:47:24.843184+0000","last_active":"2024-09-19T06:49:13.950906+0000","last_peered":"2024-09-19T06:49:13.950906+0000","last_clean":"2024-09-19T06:49:13.950906+0000","last_became_active":"2024-09-19T06:47:24.842857+0000","last_became_peered":"2024-09-19T06:47:24.842857+0000","last_unstale":"2024-09-19T06:49:13.950906+0000","last_undegraded":"2024-09-19T06:49:13.950906+0000","last_fullsized":"2024-09-19T06:49:13.950906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:06.608699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969815+0000","last_change":"2024-09-19T06:47:24.846631+0000","last_active":"2024-09-19T06:49:13.969815+0000","last_peered":"2024-09-19T06:49:13.969815+0000","last_clean":"2024-09-19T06:49:13.969815+0000","last_became_active":"2024-09-19T06:47:24.846271+0000","last_became_peered":"2024-09-19T06:47:24.846271+0000","last_unstale":"2024-09-19T06:49:13.969815+0000","last_undegraded":"2024-09-19T06:49:13.969815+0000","last_fullsized":"2024-09-19T06:49:13.969815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.272273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.967306+0000","last_change":"2024-09-19T06:47:24.840392+0000","last_active":"2024-09-19T06:49:13.967306+0000","last_peered":"2024-09-19T06:49:13.967306+0000","last_clean":"2024-09-19T06:49:13.967306+0000","last_became_active":"2024-09-19T06:47:24.840092+0000","last_became_peered":"2024-09-19T06:47:24.840092+0000","last_unstale":"2024-09-19T06:49:13.967306+0000","last_undegraded":"2024-09-19T06:49:13.967306+0000","last_fullsized":"2024-09-19T06:49:13.967306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:25.579096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.967567+0000","last_change":"2024-09-19T06:47:24.843336+0000","last_active":"2024-09-19T06:49:13.967567+0000","last_peered":"2024-09-19T06:49:13.967567+0000","last_clean":"2024-09-19T06:49:13.967567+0000","last_became_active":"2024-09-19T06:47:24.842016+0000","last_became_peered":"2024-09-19T06:47:24.842016+0000","last_unstale":"2024-09-19T06:49:13.967567+0000","last_undegraded":"2024-09-19T06:49:13.967567+0000","last_fullsized":"2024-09-19T06:49:13.967567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:49.478359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.968002+0000","last_change":"2024-09-19T06:47:24.847841+0000","last_active":"2024-09-19T06:49:13.968002+0000","last_peered":"2024-09-19T06:49:13.968002+0000","last_clean":"2024-09-19T06:49:13.968002+0000","last_became_active":"2024-09-19T06:47:24.847492+0000","last_became_peered":"2024-09-19T06:47:24.847492+0000","last_unstale":"2024-09-19T06:49:13.968002+0000","last_undegraded":"2024-09-19T06:49:13.968002+0000","last_fullsized":"2024-09-19T06:49:13.968002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:59.229022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955381+0000","last_change":"2024-09-19T06:47:24.843773+0000","last_active":"2024-09-19T06:49:13.955381+0000","last_peered":"2024-09-19T06:49:13.955381+0000","last_clean":"2024-09-19T06:49:13.955381+0000","last_became_active":"2024-09-19T06:47:24.843409+0000","last_became_peered":"2024-09-19T06:47:24.843409+0000","last_unstale":"2024-09-19T06:49:13.955381+0000","last_undegraded":"2024-09-19T06:49:13.955381+0000","last_fullsized":"2024-09-19T06:49:13.955381+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:22.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969613+0000","last_change":"2024-09-19T06:47:24.846637+0000","last_active":"2024-09-19T06:49:13.969613+0000","last_peered":"2024-09-19T06:49:13.969613+0000","last_clean":"2024-09-19T06:49:13.969613+0000","last_became_active":"2024-09-19T06:47:24.846337+0000","last_became_peered":"2024-09-19T06:47:24.846337+0000","last_unstale":"2024-09-19T06:49:13.969613+0000","last_undegraded":"2024-09-19T06:49:13.969613+0000","last_fullsized":"2024-09-19T06:49:13.969613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:23:35.869485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.946579+0000","last_change":"2024-09-19T06:47:24.851399+0000","last_active":"2024-09-19T06:49:13.946579+0000","last_peered":"2024-09-19T06:49:13.946579+0000","last_clean":"2024-09-19T06:49:13.946579+0000","last_became_active":"2024-09-19T06:47:24.851090+0000","last_became_peered":"2024-09-19T06:47:24.851090+0000","last_unstale":"2024-09-19T06:49:13.946579+0000","last_undegraded":"2024-09-19T06:49:13.946579+0000","last_fullsized":"2024-09-19T06:49:13.946579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:52.839956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970074+0000","last_change":"2024-09-19T06:47:24.833858+0000","last_active":"2024-09-19T06:49:13.970074+0000","last_peered":"2024-09-19T06:49:13.970074+0000","last_clean":"2024-09-19T06:49:13.970074+0000","last_became_active":"2024-09-19T06:47:24.833616+0000","last_became_peered":"2024-09-19T06:47:24.833616+0000","last_unstale":"2024-09-19T06:49:13.970074+0000","last_undegraded":"2024-09-19T06:49:13.970074+0000","last_fullsized":"2024-09-19T06:49:13.970074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:54.918144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966731+0000","last_change":"2024-09-19T06:47:24.841030+0000","last_active":"2024-09-19T06:49:13.966731+0000","last_peered":"2024-09-19T06:49:13.966731+0000","last_clean":"2024-09-19T06:49:13.966731+0000","last_became_active":"2024-09-19T06:47:24.839631+0000","last_became_peered":"2024-09-19T06:47:24.839631+0000","last_unstale":"2024-09-19T06:49:13.966731+0000","last_undegraded":"2024-09-19T06:49:13.966731+0000","last_fullsized":"2024-09-19T06:49:13.966731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:21.764812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970278+0000","last_change":"2024-09-19T06:47:24.849281+0000","last_active":"2024-09-19T06:49:13.970278+0000","last_peered":"2024-09-19T06:49:13.970278+0000","last_clean":"2024-09-19T06:49:13.970278+0000","last_became_active":"2024-09-19T06:47:24.848977+0000","last_became_peered":"2024-09-19T06:47:24.848977+0000","last_unstale":"2024-09-19T06:49:13.970278+0000","last_undegraded":"2024-09-19T06:49:13.970278+0000","last_fullsized":"2024-09-19T06:49:13.970278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:36.693758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973886+0000","last_change":"2024-09-19T06:47:24.843619+0000","last_active":"2024-09-19T06:49:13.973886+0000","last_peered":"2024-09-19T06:49:13.973886+0000","last_clean":"2024-09-19T06:49:13.973886+0000","last_became_active":"2024-09-19T06:47:24.843323+0000","last_became_peered":"2024-09-19T06:47:24.843323+0000","last_unstale":"2024-09-19T06:49:13.973886+0000","last_undegraded":"2024-09-19T06:49:13.973886+0000","last_fullsized":"2024-09-19T06:49:13.973886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:52.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960333+0000","last_change":"2024-09-19T06:47:24.853165+0000","last_active":"2024-09-19T06:49:13.960333+0000","last_peered":"2024-09-19T06:49:13.960333+0000","last_clean":"2024-09-19T06:49:13.960333+0000","last_became_active":"2024-09-19T06:47:24.852864+0000","last_became_peered":"2024-09-19T06:47:24.852864+0000","last_unstale":"2024-09-19T06:49:13.960333+0000","last_undegraded":"2024-09-19T06:49:13.960333+0000","last_fullsized":"2024-09-19T06:49:13.960333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:38.803478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.975021+0000","last_change":"2024-09-19T06:47:24.838151+0000","last_active":"2024-09-19T06:49:13.975021+0000","last_peered":"2024-09-19T06:49:13.975021+0000","last_clean":"2024-09-19T06:49:13.975021+0000","last_became_active":"2024-09-19T06:47:24.837855+0000","last_became_peered":"2024-09-19T06:47:24.837855+0000","last_unstale":"2024-09-19T06:49:13.975021+0000","last_undegraded":"2024-09-19T06:49:13.975021+0000","last_fullsized":"2024-09-19T06:49:13.975021+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:16.675557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.964466+0000","last_change":"2024-09-19T06:47:24.843941+0000","last_active":"2024-09-19T06:49:13.964466+0000","last_peered":"2024-09-19T06:49:13.964466+0000","last_clean":"2024-09-19T06:49:13.964466+0000","last_became_active":"2024-09-19T06:47:24.843551+0000","last_became_peered":"2024-09-19T06:47:24.843551+0000","last_unstale":"2024-09-19T06:49:13.964466+0000","last_undegraded":"2024-09-19T06:49:13.964466+0000","last_fullsized":"2024-09-19T06:49:13.964466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:00:49.388265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.954208+0000","last_change":"2024-09-19T06:47:40.836228+0000","last_active":"2024-09-19T06:49:13.954208+0000","last_peered":"2024-09-19T06:49:13.954208+0000","last_clean":"2024-09-19T06:49:13.954208+0000","last_became_active":"2024-09-19T06:47:40.835818+0000","last_became_peered":"2024-09-19T06:47:40.835818+0000","last_unstale":"2024-09-19T06:49:13.954208+0000","last_undegraded":"2024-09-19T06:49:13.954208+0000","last_fullsized":"2024-09-19T06:49:13.954208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:45.341160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972919+0000","last_change":"2024-09-19T06:47:24.843746+0000","last_active":"2024-09-19T06:49:13.972919+0000","last_peered":"2024-09-19T06:49:13.972919+0000","last_clean":"2024-09-19T06:49:13.972919+0000","last_became_active":"2024-09-19T06:47:24.841495+0000","last_became_peered":"2024-09-19T06:47:24.841495+0000","last_unstale":"2024-09-19T06:49:13.972919+0000","last_undegraded":"2024-09-19T06:49:13.972919+0000","last_fullsized":"2024-09-19T06:49:13.972919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:06:29.995595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.974565+0000","last_change":"2024-09-19T06:47:24.844747+0000","last_active":"2024-09-19T06:49:13.974565+0000","last_peered":"2024-09-19T06:49:13.974565+0000","last_clean":"2024-09-19T06:49:13.974565+0000","last_became_active":"2024-09-19T06:47:24.844425+0000","last_became_peered":"2024-09-19T06:47:24.844425+0000","last_unstale":"2024-09-19T06:49:13.974565+0000","last_undegraded":"2024-09-19T06:49:13.974565+0000","last_fullsized":"2024-09-19T06:49:13.974565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:10.309044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963556+0000","last_change":"2024-09-19T06:47:24.848227+0000","last_active":"2024-09-19T06:49:13.963556+0000","last_peered":"2024-09-19T06:49:13.963556+0000","last_clean":"2024-09-19T06:49:13.963556+0000","last_became_active":"2024-09-19T06:47:24.847914+0000","last_became_peered":"2024-09-19T06:47:24.847914+0000","last_unstale":"2024-09-19T06:49:13.963556+0000","last_undegraded":"2024-09-19T06:49:13.963556+0000","last_fullsized":"2024-09-19T06:49:13.963556+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:47.281613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.968012+0000","last_change":"2024-09-19T06:47:24.844275+0000","last_active":"2024-09-19T06:49:13.968012+0000","last_peered":"2024-09-19T06:49:13.968012+0000","last_clean":"2024-09-19T06:49:13.968012+0000","last_became_active":"2024-09-19T06:47:24.841442+0000","last_became_peered":"2024-09-19T06:47:24.841442+0000","last_unstale":"2024-09-19T06:49:13.968012+0000","last_undegraded":"2024-09-19T06:49:13.968012+0000","last_fullsized":"2024-09-19T06:49:13.968012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:30.198655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963003+0000","last_change":"2024-09-19T06:47:24.846301+0000","last_active":"2024-09-19T06:49:13.963003+0000","last_peered":"2024-09-19T06:49:13.963003+0000","last_clean":"2024-09-19T06:49:13.963003+0000","last_became_active":"2024-09-19T06:47:24.845970+0000","last_became_peered":"2024-09-19T06:47:24.845970+0000","last_unstale":"2024-09-19T06:49:13.963003+0000","last_undegraded":"2024-09-19T06:49:13.963003+0000","last_fullsized":"2024-09-19T06:49:13.963003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:13.245536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973777+0000","last_change":"2024-09-19T06:47:24.844444+0000","last_active":"2024-09-19T06:49:13.973777+0000","last_peered":"2024-09-19T06:49:13.973777+0000","last_clean":"2024-09-19T06:49:13.973777+0000","last_became_active":"2024-09-19T06:47:24.844141+0000","last_became_peered":"2024-09-19T06:47:24.844141+0000","last_unstale":"2024-09-19T06:49:13.973777+0000","last_undegraded":"2024-09-19T06:49:13.973777+0000","last_fullsized":"2024-09-19T06:49:13.973777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:44.739242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955255+0000","last_change":"2024-09-19T06:47:24.845633+0000","last_active":"2024-09-19T06:49:13.955255+0000","last_peered":"2024-09-19T06:49:13.955255+0000","last_clean":"2024-09-19T06:49:13.955255+0000","last_became_active":"2024-09-19T06:47:24.845319+0000","last_became_peered":"2024-09-19T06:47:24.845319+0000","last_unstale":"2024-09-19T06:49:13.955255+0000","last_undegraded":"2024-09-19T06:49:13.955255+0000","last_fullsized":"2024-09-19T06:49:13.955255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:00.820006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959353+0000","last_change":"2024-09-19T06:47:24.846457+0000","last_active":"2024-09-19T06:49:13.959353+0000","last_peered":"2024-09-19T06:49:13.959353+0000","last_clean":"2024-09-19T06:49:13.959353+0000","last_became_active":"2024-09-19T06:47:24.846130+0000","last_became_peered":"2024-09-19T06:47:24.846130+0000","last_unstale":"2024-09-19T06:49:13.959353+0000","last_undegraded":"2024-09-19T06:49:13.959353+0000","last_fullsized":"2024-09-19T06:49:13.959353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:42.179145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969751+0000","last_change":"2024-09-19T06:47:24.843141+0000","last_active":"2024-09-19T06:49:13.969751+0000","last_peered":"2024-09-19T06:49:13.969751+0000","last_clean":"2024-09-19T06:49:13.969751+0000","last_became_active":"2024-09-19T06:47:24.842810+0000","last_became_peered":"2024-09-19T06:47:24.842810+0000","last_unstale":"2024-09-19T06:49:13.969751+0000","last_undegraded":"2024-09-19T06:49:13.969751+0000","last_fullsized":"2024-09-19T06:49:13.969751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:43.440453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.965342+0000","last_change":"2024-09-19T06:47:24.846171+0000","last_active":"2024-09-19T06:49:13.965342+0000","last_peered":"2024-09-19T06:49:13.965342+0000","last_clean":"2024-09-19T06:49:13.965342+0000","last_became_active":"2024-09-19T06:47:24.845799+0000","last_became_peered":"2024-09-19T06:47:24.845799+0000","last_unstale":"2024-09-19T06:49:13.965342+0000","last_undegraded":"2024-09-19T06:49:13.965342+0000","last_fullsized":"2024-09-19T06:49:13.965342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:23.103061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972106+0000","last_change":"2024-09-19T06:47:24.848723+0000","last_active":"2024-09-19T06:49:13.972106+0000","last_peered":"2024-09-19T06:49:13.972106+0000","last_clean":"2024-09-19T06:49:13.972106+0000","last_became_active":"2024-09-19T06:47:24.848367+0000","last_became_peered":"2024-09-19T06:47:24.848367+0000","last_unstale":"2024-09-19T06:49:13.972106+0000","last_undegraded":"2024-09-19T06:49:13.972106+0000","last_fullsized":"2024-09-19T06:49:13.972106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:15.985293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960543+0000","last_change":"2024-09-19T06:47:24.841379+0000","last_active":"2024-09-19T06:49:13.960543+0000","last_peered":"2024-09-19T06:49:13.960543+0000","last_clean":"2024-09-19T06:49:13.960543+0000","last_became_active":"2024-09-19T06:47:24.841041+0000","last_became_peered":"2024-09-19T06:47:24.841041+0000","last_unstale":"2024-09-19T06:49:13.960543+0000","last_undegraded":"2024-09-19T06:49:13.960543+0000","last_fullsized":"2024-09-19T06:49:13.960543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:11.799022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955643+0000","last_change":"2024-09-19T06:47:24.849508+0000","last_active":"2024-09-19T06:49:13.955643+0000","last_peered":"2024-09-19T06:49:13.955643+0000","last_clean":"2024-09-19T06:49:13.955643+0000","last_became_active":"2024-09-19T06:47:24.849196+0000","last_became_peered":"2024-09-19T06:47:24.849196+0000","last_unstale":"2024-09-19T06:49:13.955643+0000","last_undegraded":"2024-09-19T06:49:13.955643+0000","last_fullsized":"2024-09-19T06:49:13.955643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:35.079720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":132,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.604832+0000","last_change":"2024-09-19T06:47:22.829848+0000","last_active":"2024-09-19T06:49:14.604832+0000","last_peered":"2024-09-19T06:49:14.604832+0000","last_clean":"2024-09-19T06:49:14.604832+0000","last_became_active":"2024-09-19T06:47:22.829511+0000","last_became_peered":"2024-09-19T06:47:22.829511+0000","last_unstale":"2024-09-19T06:49:14.604832+0000","last_undegraded":"2024-09-19T06:49:14.604832+0000","last_fullsized":"2024-09-19T06:49:14.604832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:32.665252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971999+0000","last_change":"2024-09-19T06:47:24.843245+0000","last_active":"2024-09-19T06:49:13.971999+0000","last_peered":"2024-09-19T06:49:13.971999+0000","last_clean":"2024-09-19T06:49:13.971999+0000","last_became_active":"2024-09-19T06:47:24.842885+0000","last_became_peered":"2024-09-19T06:47:24.842885+0000","last_unstale":"2024-09-19T06:49:13.971999+0000","last_undegraded":"2024-09-19T06:49:13.971999+0000","last_fullsized":"2024-09-19T06:49:13.971999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:26.229003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.926577+0000","last_change":"2024-09-19T06:47:22.831456+0000","last_active":"2024-09-19T06:49:13.926577+0000","last_peered":"2024-09-19T06:49:13.926577+0000","last_clean":"2024-09-19T06:49:13.926577+0000","last_became_active":"2024-09-19T06:47:22.831132+0000","last_became_peered":"2024-09-19T06:47:22.831132+0000","last_unstale":"2024-09-19T06:49:13.926577+0000","last_undegraded":"2024-09-19T06:49:13.926577+0000","last_fullsized":"2024-09-19T06:49:13.926577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:49.867713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932598+0000","last_change":"2024-09-19T06:47:22.827742+0000","last_active":"2024-09-19T06:49:13.932598+0000","last_peered":"2024-09-19T06:49:13.932598+0000","last_clean":"2024-09-19T06:49:13.932598+0000","last_became_active":"2024-09-19T06:47:22.827371+0000","last_became_peered":"2024-09-19T06:47:22.827371+0000","last_unstale":"2024-09-19T06:49:13.932598+0000","last_undegraded":"2024-09-19T06:49:13.932598+0000","last_fullsized":"2024-09-19T06:49:13.932598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:35.153129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963305+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:49:13.963305+0000","last_peered":"2024-09-19T06:49:13.963305+0000","last_clean":"2024-09-19T06:49:13.963305+0000","last_became_active":"2024-09-19T06:47:24.841693+0000","last_became_peered":"2024-09-19T06:47:24.841693+0000","last_unstale":"2024-09-19T06:49:13.963305+0000","last_undegraded":"2024-09-19T06:49:13.963305+0000","last_fullsized":"2024-09-19T06:49:13.963305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:27.828296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.917388+0000","last_change":"2024-09-19T06:47:22.832531+0000","last_active":"2024-09-19T06:49:13.917388+0000","last_peered":"2024-09-19T06:49:13.917388+0000","last_clean":"2024-09-19T06:49:13.917388+0000","last_became_active":"2024-09-19T06:47:22.832213+0000","last_became_peered":"2024-09-19T06:47:22.832213+0000","last_unstale":"2024-09-19T06:49:13.917388+0000","last_undegraded":"2024-09-19T06:49:13.917388+0000","last_fullsized":"2024-09-19T06:49:13.917388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:46:03.168406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970720+0000","last_change":"2024-09-19T06:47:24.837690+0000","last_active":"2024-09-19T06:49:13.970720+0000","last_peered":"2024-09-19T06:49:13.970720+0000","last_clean":"2024-09-19T06:49:13.970720+0000","last_became_active":"2024-09-19T06:47:24.837389+0000","last_became_peered":"2024-09-19T06:47:24.837389+0000","last_unstale":"2024-09-19T06:49:13.970720+0000","last_undegraded":"2024-09-19T06:49:13.970720+0000","last_fullsized":"2024-09-19T06:49:13.970720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:20.653832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":228,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.604318+0000","last_change":"2024-09-19T06:47:22.833808+0000","last_active":"2024-09-19T06:49:14.604318+0000","last_peered":"2024-09-19T06:49:14.604318+0000","last_clean":"2024-09-19T06:49:14.604318+0000","last_became_active":"2024-09-19T06:47:22.832468+0000","last_became_peered":"2024-09-19T06:47:22.832468+0000","last_unstale":"2024-09-19T06:49:14.604318+0000","last_undegraded":"2024-09-19T06:49:14.604318+0000","last_fullsized":"2024-09-19T06:49:14.604318+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:39:31.131321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":198,"num_read_kb":198,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973091+0000","last_change":"2024-09-19T06:47:24.840928+0000","last_active":"2024-09-19T06:49:13.973091+0000","last_peered":"2024-09-19T06:49:13.973091+0000","last_clean":"2024-09-19T06:49:13.973091+0000","last_became_active":"2024-09-19T06:47:24.839928+0000","last_became_peered":"2024-09-19T06:47:24.839928+0000","last_unstale":"2024-09-19T06:49:13.973091+0000","last_undegraded":"2024-09-19T06:49:13.973091+0000","last_fullsized":"2024-09-19T06:49:13.973091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:29.486003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.928845+0000","last_change":"2024-09-19T06:47:22.833310+0000","last_active":"2024-09-19T06:49:13.928845+0000","last_peered":"2024-09-19T06:49:13.928845+0000","last_clean":"2024-09-19T06:49:13.928845+0000","last_became_active":"2024-09-19T06:47:22.832976+0000","last_became_peered":"2024-09-19T06:47:22.832976+0000","last_unstale":"2024-09-19T06:49:13.928845+0000","last_undegraded":"2024-09-19T06:49:13.928845+0000","last_fullsized":"2024-09-19T06:49:13.928845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:47.604853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969853+0000","last_change":"2024-09-19T06:47:24.848687+0000","last_active":"2024-09-19T06:49:13.969853+0000","last_peered":"2024-09-19T06:49:13.969853+0000","last_clean":"2024-09-19T06:49:13.969853+0000","last_became_active":"2024-09-19T06:47:24.848352+0000","last_became_peered":"2024-09-19T06:47:24.848352+0000","last_unstale":"2024-09-19T06:49:13.969853+0000","last_undegraded":"2024-09-19T06:49:13.969853+0000","last_fullsized":"2024-09-19T06:49:13.969853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:54.459206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.917489+0000","last_change":"2024-09-19T06:47:22.851479+0000","last_active":"2024-09-19T06:49:13.917489+0000","last_peered":"2024-09-19T06:49:13.917489+0000","last_clean":"2024-09-19T06:49:13.917489+0000","last_became_active":"2024-09-19T06:47:22.851125+0000","last_became_peered":"2024-09-19T06:47:22.851125+0000","last_unstale":"2024-09-19T06:49:13.917489+0000","last_undegraded":"2024-09-19T06:49:13.917489+0000","last_fullsized":"2024-09-19T06:49:13.917489+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:42.954034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973229+0000","last_change":"2024-09-19T06:47:24.851007+0000","last_active":"2024-09-19T06:49:13.973229+0000","last_peered":"2024-09-19T06:49:13.973229+0000","last_clean":"2024-09-19T06:49:13.973229+0000","last_became_active":"2024-09-19T06:47:24.850465+0000","last_became_peered":"2024-09-19T06:47:24.850465+0000","last_unstale":"2024-09-19T06:49:13.973229+0000","last_undegraded":"2024-09-19T06:49:13.973229+0000","last_fullsized":"2024-09-19T06:49:13.973229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:10.824672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":268,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.604918+0000","last_change":"2024-09-19T06:47:22.834893+0000","last_active":"2024-09-19T06:49:14.604918+0000","last_peered":"2024-09-19T06:49:14.604918+0000","last_clean":"2024-09-19T06:49:14.604918+0000","last_became_active":"2024-09-19T06:47:22.834521+0000","last_became_peered":"2024-09-19T06:47:22.834521+0000","last_unstale":"2024-09-19T06:49:14.604918+0000","last_undegraded":"2024-09-19T06:49:14.604918+0000","last_fullsized":"2024-09-19T06:49:14.604918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:29.742284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66347,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":208,"num_read_kb":168,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.957411+0000","last_change":"2024-09-19T06:47:24.848191+0000","last_active":"2024-09-19T06:49:13.957411+0000","last_peered":"2024-09-19T06:49:13.957411+0000","last_clean":"2024-09-19T06:49:13.957411+0000","last_became_active":"2024-09-19T06:47:24.847889+0000","last_became_peered":"2024-09-19T06:47:24.847889+0000","last_unstale":"2024-09-19T06:49:13.957411+0000","last_undegraded":"2024-09-19T06:49:13.957411+0000","last_fullsized":"2024-09-19T06:49:13.957411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:32.574360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.924532+0000","last_change":"2024-09-19T06:47:22.832005+0000","last_active":"2024-09-19T06:49:13.924532+0000","last_peered":"2024-09-19T06:49:13.924532+0000","last_clean":"2024-09-19T06:49:13.924532+0000","last_became_active":"2024-09-19T06:47:22.831342+0000","last_became_peered":"2024-09-19T06:47:22.831342+0000","last_unstale":"2024-09-19T06:49:13.924532+0000","last_undegraded":"2024-09-19T06:49:13.924532+0000","last_fullsized":"2024-09-19T06:49:13.924532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:53.444748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.964849+0000","last_change":"2024-09-19T06:47:24.843328+0000","last_active":"2024-09-19T06:49:13.964849+0000","last_peered":"2024-09-19T06:49:13.964849+0000","last_clean":"2024-09-19T06:49:13.964849+0000","last_became_active":"2024-09-19T06:47:24.842344+0000","last_became_peered":"2024-09-19T06:47:24.842344+0000","last_unstale":"2024-09-19T06:49:13.964849+0000","last_undegraded":"2024-09-19T06:49:13.964849+0000","last_fullsized":"2024-09-19T06:49:13.964849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:11.247819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.926463+0000","last_change":"2024-09-19T06:47:40.844086+0000","last_active":"2024-09-19T06:49:13.926463+0000","last_peered":"2024-09-19T06:49:13.926463+0000","last_clean":"2024-09-19T06:49:13.926463+0000","last_became_active":"2024-09-19T06:47:40.835015+0000","last_became_peered":"2024-09-19T06:47:40.835015+0000","last_unstale":"2024-09-19T06:49:13.926463+0000","last_undegraded":"2024-09-19T06:49:13.926463+0000","last_fullsized":"2024-09-19T06:49:13.926463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:09.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972972+0000","last_change":"2024-09-19T06:47:40.836466+0000","last_active":"2024-09-19T06:49:13.972972+0000","last_peered":"2024-09-19T06:49:13.972972+0000","last_clean":"2024-09-19T06:49:13.972972+0000","last_became_active":"2024-09-19T06:47:40.836237+0000","last_became_peered":"2024-09-19T06:47:40.836237+0000","last_unstale":"2024-09-19T06:49:13.972972+0000","last_undegraded":"2024-09-19T06:49:13.972972+0000","last_fullsized":"2024-09-19T06:49:13.972972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:00.094685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'10","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.606091+0000","last_change":"2024-09-19T06:47:22.830359+0000","last_active":"2024-09-19T06:49:14.606091+0000","last_peered":"2024-09-19T06:49:14.606091+0000","last_clean":"2024-09-19T06:49:14.606091+0000","last_became_active":"2024-09-19T06:47:22.829960+0000","last_became_peered":"2024-09-19T06:47:22.829960+0000","last_unstale":"2024-09-19T06:49:14.606091+0000","last_undegraded":"2024-09-19T06:49:14.606091+0000","last_fullsized":"2024-09-19T06:49:14.606091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:55.998243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":6,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971872+0000","last_change":"2024-09-19T06:47:24.836668+0000","last_active":"2024-09-19T06:49:13.971872+0000","last_peered":"2024-09-19T06:49:13.971872+0000","last_clean":"2024-09-19T06:49:13.971872+0000","last_became_active":"2024-09-19T06:47:24.836393+0000","last_became_peered":"2024-09-19T06:47:24.836393+0000","last_unstale":"2024-09-19T06:49:13.971872+0000","last_undegraded":"2024-09-19T06:49:13.971872+0000","last_fullsized":"2024-09-19T06:49:13.971872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:26.248948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":66,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.931451+0000","last_change":"2024-09-19T06:47:40.858323+0000","last_active":"2024-09-19T06:49:13.931451+0000","last_peered":"2024-09-19T06:49:13.931451+0000","last_clean":"2024-09-19T06:49:13.931451+0000","last_became_active":"2024-09-19T06:47:40.840802+0000","last_became_peered":"2024-09-19T06:47:40.840802+0000","last_unstale":"2024-09-19T06:49:13.931451+0000","last_undegraded":"2024-09-19T06:49:13.931451+0000","last_fullsized":"2024-09-19T06:49:13.931451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:14.666916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970280+0000","last_change":"2024-09-19T06:47:24.843681+0000","last_active":"2024-09-19T06:49:13.970280+0000","last_peered":"2024-09-19T06:49:13.970280+0000","last_clean":"2024-09-19T06:49:13.970280+0000","last_became_active":"2024-09-19T06:47:24.841228+0000","last_became_peered":"2024-09-19T06:47:24.841228+0000","last_unstale":"2024-09-19T06:49:13.970280+0000","last_undegraded":"2024-09-19T06:49:13.970280+0000","last_fullsized":"2024-09-19T06:49:13.970280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:56.418962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.933661+0000","last_change":"2024-09-19T06:47:22.833893+0000","last_active":"2024-09-19T06:49:13.933661+0000","last_peered":"2024-09-19T06:49:13.933661+0000","last_clean":"2024-09-19T06:49:13.933661+0000","last_became_active":"2024-09-19T06:47:22.833217+0000","last_became_peered":"2024-09-19T06:47:22.833217+0000","last_unstale":"2024-09-19T06:49:13.933661+0000","last_undegraded":"2024-09-19T06:49:13.933661+0000","last_fullsized":"2024-09-19T06:49:13.933661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:19.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955096+0000","last_change":"2024-09-19T06:47:24.848399+0000","last_active":"2024-09-19T06:49:13.955096+0000","last_peered":"2024-09-19T06:49:13.955096+0000","last_clean":"2024-09-19T06:49:13.955096+0000","last_became_active":"2024-09-19T06:47:24.848081+0000","last_became_peered":"2024-09-19T06:47:24.848081+0000","last_unstale":"2024-09-19T06:49:13.955096+0000","last_undegraded":"2024-09-19T06:49:13.955096+0000","last_fullsized":"2024-09-19T06:49:13.955096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:44.292939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.931968+0000","last_change":"2024-09-19T06:47:40.860955+0000","last_active":"2024-09-19T06:49:13.931968+0000","last_peered":"2024-09-19T06:49:13.931968+0000","last_clean":"2024-09-19T06:49:13.931968+0000","last_became_active":"2024-09-19T06:47:40.835126+0000","last_became_peered":"2024-09-19T06:47:40.835126+0000","last_unstale":"2024-09-19T06:49:13.931968+0000","last_undegraded":"2024-09-19T06:49:13.931968+0000","last_fullsized":"2024-09-19T06:49:13.931968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:02:26.763229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971391+0000","last_change":"2024-09-19T06:47:24.849680+0000","last_active":"2024-09-19T06:49:13.971391+0000","last_peered":"2024-09-19T06:49:13.971391+0000","last_clean":"2024-09-19T06:49:13.971391+0000","last_became_active":"2024-09-19T06:47:24.849323+0000","last_became_peered":"2024-09-19T06:47:24.849323+0000","last_unstale":"2024-09-19T06:49:13.971391+0000","last_undegraded":"2024-09-19T06:49:13.971391+0000","last_fullsized":"2024-09-19T06:49:13.971391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:59.763503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'41","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.148665+0000","last_change":"2024-09-19T06:47:22.831689+0000","last_active":"2024-09-19T06:49:14.148665+0000","last_peered":"2024-09-19T06:49:14.148665+0000","last_clean":"2024-09-19T06:49:14.148665+0000","last_became_active":"2024-09-19T06:47:22.829283+0000","last_became_peered":"2024-09-19T06:47:22.829283+0000","last_unstale":"2024-09-19T06:49:14.148665+0000","last_undegraded":"2024-09-19T06:49:14.148665+0000","last_fullsized":"2024-09-19T06:49:14.148665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:37.109542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971792+0000","last_change":"2024-09-19T06:47:24.842793+0000","last_active":"2024-09-19T06:49:13.971792+0000","last_peered":"2024-09-19T06:49:13.971792+0000","last_clean":"2024-09-19T06:49:13.971792+0000","last_became_active":"2024-09-19T06:47:24.842474+0000","last_became_peered":"2024-09-19T06:47:24.842474+0000","last_unstale":"2024-09-19T06:49:13.971792+0000","last_undegraded":"2024-09-19T06:49:13.971792+0000","last_fullsized":"2024-09-19T06:49:13.971792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:23.110736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'9","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.605146+0000","last_change":"2024-09-19T06:47:23.047356+0000","last_active":"2024-09-19T06:49:14.605146+0000","last_peered":"2024-09-19T06:49:14.605146+0000","last_clean":"2024-09-19T06:49:14.605146+0000","last_became_active":"2024-09-19T06:47:23.046941+0000","last_became_peered":"2024-09-19T06:47:23.046941+0000","last_unstale":"2024-09-19T06:49:14.605146+0000","last_undegraded":"2024-09-19T06:49:14.605146+0000","last_fullsized":"2024-09-19T06:49:14.605146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:43.490952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":4,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971243+0000","last_change":"2024-09-19T06:47:24.851025+0000","last_active":"2024-09-19T06:49:13.971243+0000","last_peered":"2024-09-19T06:49:13.971243+0000","last_clean":"2024-09-19T06:49:13.971243+0000","last_became_active":"2024-09-19T06:47:24.850681+0000","last_became_peered":"2024-09-19T06:47:24.850681+0000","last_unstale":"2024-09-19T06:49:13.971243+0000","last_undegraded":"2024-09-19T06:49:13.971243+0000","last_fullsized":"2024-09-19T06:49:13.971243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:46.884747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":348,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.928776+0000","last_change":"2024-09-19T06:46:13.927379+0000","last_active":"2024-09-19T06:49:13.928776+0000","last_peered":"2024-09-19T06:49:13.928776+0000","last_clean":"2024-09-19T06:49:13.928776+0000","last_became_active":"2024-09-19T06:46:13.926926+0000","last_became_peered":"2024-09-19T06:46:13.926926+0000","last_unstale":"2024-09-19T06:49:13.928776+0000","last_undegraded":"2024-09-19T06:49:13.928776+0000","last_fullsized":"2024-09-19T06:49:13.928776+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_clean_scrub_stamp":"2024-09-19T06:42:52.051335+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:29.542128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.927403+0000","last_change":"2024-09-19T06:47:22.830387+0000","last_active":"2024-09-19T06:49:13.927403+0000","last_peered":"2024-09-19T06:49:13.927403+0000","last_clean":"2024-09-19T06:49:13.927403+0000","last_became_active":"2024-09-19T06:47:22.829970+0000","last_became_peered":"2024-09-19T06:47:22.829970+0000","last_unstale":"2024-09-19T06:49:13.927403+0000","last_undegraded":"2024-09-19T06:49:13.927403+0000","last_fullsized":"2024-09-19T06:49:13.927403+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:51.385456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.974671+0000","last_change":"2024-09-19T06:47:24.843752+0000","last_active":"2024-09-19T06:49:13.974671+0000","last_peered":"2024-09-19T06:49:13.974671+0000","last_clean":"2024-09-19T06:49:13.974671+0000","last_became_active":"2024-09-19T06:47:24.842631+0000","last_became_peered":"2024-09-19T06:47:24.842631+0000","last_unstale":"2024-09-19T06:49:13.974671+0000","last_undegraded":"2024-09-19T06:49:13.974671+0000","last_fullsized":"2024-09-19T06:49:13.974671+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:19.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.928732+0000","last_change":"2024-09-19T06:47:22.832647+0000","last_active":"2024-09-19T06:49:13.928732+0000","last_peered":"2024-09-19T06:49:13.928732+0000","last_clean":"2024-09-19T06:49:13.928732+0000","last_became_active":"2024-09-19T06:47:22.831774+0000","last_became_peered":"2024-09-19T06:47:22.831774+0000","last_unstale":"2024-09-19T06:49:13.928732+0000","last_undegraded":"2024-09-19T06:49:13.928732+0000","last_fullsized":"2024-09-19T06:49:13.928732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:57.824610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.948364+0000","last_change":"2024-09-19T06:47:24.844301+0000","last_active":"2024-09-19T06:49:13.948364+0000","last_peered":"2024-09-19T06:49:13.948364+0000","last_clean":"2024-09-19T06:49:13.948364+0000","last_became_active":"2024-09-19T06:47:24.839748+0000","last_became_peered":"2024-09-19T06:47:24.839748+0000","last_unstale":"2024-09-19T06:49:13.948364+0000","last_undegraded":"2024-09-19T06:49:13.948364+0000","last_fullsized":"2024-09-19T06:49:13.948364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:46.407693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":130,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932811+0000","last_change":"2024-09-19T06:47:22.851008+0000","last_active":"2024-09-19T06:49:13.932811+0000","last_peered":"2024-09-19T06:49:13.932811+0000","last_clean":"2024-09-19T06:49:13.932811+0000","last_became_active":"2024-09-19T06:47:22.850601+0000","last_became_peered":"2024-09-19T06:47:22.850601+0000","last_unstale":"2024-09-19T06:49:13.932811+0000","last_undegraded":"2024-09-19T06:49:13.932811+0000","last_fullsized":"2024-09-19T06:49:13.932811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:04.610277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970407+0000","last_change":"2024-09-19T06:47:24.839684+0000","last_active":"2024-09-19T06:49:13.970407+0000","last_peered":"2024-09-19T06:49:13.970407+0000","last_clean":"2024-09-19T06:49:13.970407+0000","last_became_active":"2024-09-19T06:47:24.839126+0000","last_became_peered":"2024-09-19T06:47:24.839126+0000","last_unstale":"2024-09-19T06:49:13.970407+0000","last_undegraded":"2024-09-19T06:49:13.970407+0000","last_fullsized":"2024-09-19T06:49:13.970407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:09.458535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":130,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.314141+0000","last_change":"2024-09-19T06:47:22.831819+0000","last_active":"2024-09-19T06:49:14.314141+0000","last_peered":"2024-09-19T06:49:14.314141+0000","last_clean":"2024-09-19T06:49:14.314141+0000","last_became_active":"2024-09-19T06:47:22.831332+0000","last_became_peered":"2024-09-19T06:47:22.831332+0000","last_unstale":"2024-09-19T06:49:14.314141+0000","last_undegraded":"2024-09-19T06:49:14.314141+0000","last_fullsized":"2024-09-19T06:49:14.314141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:35.032949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.965109+0000","last_change":"2024-09-19T06:47:24.846738+0000","last_active":"2024-09-19T06:49:13.965109+0000","last_peered":"2024-09-19T06:49:13.965109+0000","last_clean":"2024-09-19T06:49:13.965109+0000","last_became_active":"2024-09-19T06:47:24.846389+0000","last_became_peered":"2024-09-19T06:47:24.846389+0000","last_unstale":"2024-09-19T06:49:13.965109+0000","last_undegraded":"2024-09-19T06:49:13.965109+0000","last_fullsized":"2024-09-19T06:49:13.965109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:19.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.918173+0000","last_change":"2024-09-19T06:47:22.851065+0000","last_active":"2024-09-19T06:49:13.918173+0000","last_peered":"2024-09-19T06:49:13.918173+0000","last_clean":"2024-09-19T06:49:13.918173+0000","last_became_active":"2024-09-19T06:47:22.850647+0000","last_became_peered":"2024-09-19T06:47:22.850647+0000","last_unstale":"2024-09-19T06:49:13.918173+0000","last_undegraded":"2024-09-19T06:49:13.918173+0000","last_fullsized":"2024-09-19T06:49:13.918173+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:41.993501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966795+0000","last_change":"2024-09-19T06:47:24.838905+0000","last_active":"2024-09-19T06:49:13.966795+0000","last_peered":"2024-09-19T06:49:13.966795+0000","last_clean":"2024-09-19T06:49:13.966795+0000","last_became_active":"2024-09-19T06:47:24.838443+0000","last_became_peered":"2024-09-19T06:47:24.838443+0000","last_unstale":"2024-09-19T06:49:13.966795+0000","last_undegraded":"2024-09-19T06:49:13.966795+0000","last_fullsized":"2024-09-19T06:49:13.966795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:12.486137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932653+0000","last_change":"2024-09-19T06:47:22.851431+0000","last_active":"2024-09-19T06:49:13.932653+0000","last_peered":"2024-09-19T06:49:13.932653+0000","last_clean":"2024-09-19T06:49:13.932653+0000","last_became_active":"2024-09-19T06:47:22.851092+0000","last_became_peered":"2024-09-19T06:47:22.851092+0000","last_unstale":"2024-09-19T06:49:13.932653+0000","last_undegraded":"2024-09-19T06:49:13.932653+0000","last_fullsized":"2024-09-19T06:49:13.932653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:26.183803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959143+0000","last_change":"2024-09-19T06:47:24.839046+0000","last_active":"2024-09-19T06:49:13.959143+0000","last_peered":"2024-09-19T06:49:13.959143+0000","last_clean":"2024-09-19T06:49:13.959143+0000","last_became_active":"2024-09-19T06:47:24.838680+0000","last_became_peered":"2024-09-19T06:47:24.838680+0000","last_unstale":"2024-09-19T06:49:13.959143+0000","last_undegraded":"2024-09-19T06:49:13.959143+0000","last_fullsized":"2024-09-19T06:49:13.959143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:45.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'50","reported_seq":113,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.156330+0000","last_change":"2024-09-19T06:47:40.865241+0000","last_active":"2024-09-19T06:49:14.156330+0000","last_peered":"2024-09-19T06:49:14.156330+0000","last_clean":"2024-09-19T06:49:14.156330+0000","last_became_active":"2024-09-19T06:47:40.837765+0000","last_became_peered":"2024-09-19T06:47:40.837765+0000","last_unstale":"2024-09-19T06:49:14.156330+0000","last_undegraded":"2024-09-19T06:49:14.156330+0000","last_fullsized":"2024-09-19T06:49:14.156330+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966118+0000","last_change":"2024-09-19T06:47:24.848599+0000","last_active":"2024-09-19T06:49:13.966118+0000","last_peered":"2024-09-19T06:49:13.966118+0000","last_clean":"2024-09-19T06:49:13.966118+0000","last_became_active":"2024-09-19T06:47:24.848305+0000","last_became_peered":"2024-09-19T06:47:24.848305+0000","last_unstale":"2024-09-19T06:49:13.966118+0000","last_undegraded":"2024-09-19T06:49:13.966118+0000","last_fullsized":"2024-09-19T06:49:13.966118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:31.496188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.927755+0000","last_change":"2024-09-19T06:47:23.047484+0000","last_active":"2024-09-19T06:49:13.927755+0000","last_peered":"2024-09-19T06:49:13.927755+0000","last_clean":"2024-09-19T06:49:13.927755+0000","last_became_active":"2024-09-19T06:47:23.047058+0000","last_became_peered":"2024-09-19T06:47:23.047058+0000","last_unstale":"2024-09-19T06:49:13.927755+0000","last_undegraded":"2024-09-19T06:49:13.927755+0000","last_fullsized":"2024-09-19T06:49:13.927755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:34.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970265+0000","last_change":"2024-09-19T06:47:24.843498+0000","last_active":"2024-09-19T06:49:13.970265+0000","last_peered":"2024-09-19T06:49:13.970265+0000","last_clean":"2024-09-19T06:49:13.970265+0000","last_became_active":"2024-09-19T06:47:24.843204+0000","last_became_peered":"2024-09-19T06:47:24.843204+0000","last_unstale":"2024-09-19T06:49:13.970265+0000","last_undegraded":"2024-09-19T06:49:13.970265+0000","last_fullsized":"2024-09-19T06:49:13.970265+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:25.585023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.919373+0000","last_change":"2024-09-19T06:47:22.851519+0000","last_active":"2024-09-19T06:49:13.919373+0000","last_peered":"2024-09-19T06:49:13.919373+0000","last_clean":"2024-09-19T06:49:13.919373+0000","last_became_active":"2024-09-19T06:47:22.851163+0000","last_became_peered":"2024-09-19T06:47:22.851163+0000","last_unstale":"2024-09-19T06:49:13.919373+0000","last_undegraded":"2024-09-19T06:49:13.919373+0000","last_fullsized":"2024-09-19T06:49:13.919373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:04.502795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.945308+0000","last_change":"2024-09-19T06:47:24.845239+0000","last_active":"2024-09-19T06:49:13.945308+0000","last_peered":"2024-09-19T06:49:13.945308+0000","last_clean":"2024-09-19T06:49:13.945308+0000","last_became_active":"2024-09-19T06:47:24.844934+0000","last_became_peered":"2024-09-19T06:47:24.844934+0000","last_unstale":"2024-09-19T06:49:13.945308+0000","last_undegraded":"2024-09-19T06:49:13.945308+0000","last_fullsized":"2024-09-19T06:49:13.945308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:43.610604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.934380+0000","last_change":"2024-09-19T06:47:22.830232+0000","last_active":"2024-09-19T06:49:13.934380+0000","last_peered":"2024-09-19T06:49:13.934380+0000","last_clean":"2024-09-19T06:49:13.934380+0000","last_became_active":"2024-09-19T06:47:22.829901+0000","last_became_peered":"2024-09-19T06:47:22.829901+0000","last_unstale":"2024-09-19T06:49:13.934380+0000","last_undegraded":"2024-09-19T06:49:13.934380+0000","last_fullsized":"2024-09-19T06:49:13.934380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:12.251427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955845+0000","last_change":"2024-09-19T06:47:24.839304+0000","last_active":"2024-09-19T06:49:13.955845+0000","last_peered":"2024-09-19T06:49:13.955845+0000","last_clean":"2024-09-19T06:49:13.955845+0000","last_became_active":"2024-09-19T06:47:24.838986+0000","last_became_peered":"2024-09-19T06:47:24.838986+0000","last_unstale":"2024-09-19T06:49:13.955845+0000","last_undegraded":"2024-09-19T06:49:13.955845+0000","last_fullsized":"2024-09-19T06:49:13.955845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:39.377886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.885319+0000","last_change":"2024-09-19T06:47:22.828553+0000","last_active":"2024-09-19T06:49:14.885319+0000","last_peered":"2024-09-19T06:49:14.885319+0000","last_clean":"2024-09-19T06:49:14.885319+0000","last_became_active":"2024-09-19T06:47:22.828213+0000","last_became_peered":"2024-09-19T06:47:22.828213+0000","last_unstale":"2024-09-19T06:49:14.885319+0000","last_undegraded":"2024-09-19T06:49:14.885319+0000","last_fullsized":"2024-09-19T06:49:14.885319+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:31.649761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960387+0000","last_change":"2024-09-19T06:47:24.840747+0000","last_active":"2024-09-19T06:49:13.960387+0000","last_peered":"2024-09-19T06:49:13.960387+0000","last_clean":"2024-09-19T06:49:13.960387+0000","last_became_active":"2024-09-19T06:47:24.840451+0000","last_became_peered":"2024-09-19T06:47:24.840451+0000","last_unstale":"2024-09-19T06:49:13.960387+0000","last_undegraded":"2024-09-19T06:49:13.960387+0000","last_fullsized":"2024-09-19T06:49:13.960387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:41.588751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.933845+0000","last_change":"2024-09-19T06:47:22.831697+0000","last_active":"2024-09-19T06:49:13.933845+0000","last_peered":"2024-09-19T06:49:13.933845+0000","last_clean":"2024-09-19T06:49:13.933845+0000","last_became_active":"2024-09-19T06:47:22.831178+0000","last_became_peered":"2024-09-19T06:47:22.831178+0000","last_unstale":"2024-09-19T06:49:13.933845+0000","last_undegraded":"2024-09-19T06:49:13.933845+0000","last_fullsized":"2024-09-19T06:49:13.933845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:30.975782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971917+0000","last_change":"2024-09-19T06:47:24.845097+0000","last_active":"2024-09-19T06:49:13.971917+0000","last_peered":"2024-09-19T06:49:13.971917+0000","last_clean":"2024-09-19T06:49:13.971917+0000","last_became_active":"2024-09-19T06:47:24.844801+0000","last_became_peered":"2024-09-19T06:47:24.844801+0000","last_unstale":"2024-09-19T06:49:13.971917+0000","last_undegraded":"2024-09-19T06:49:13.971917+0000","last_fullsized":"2024-09-19T06:49:13.971917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:15.264118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971733+0000","last_change":"2024-09-19T06:47:24.851372+0000","last_active":"2024-09-19T06:49:13.971733+0000","last_peered":"2024-09-19T06:49:13.971733+0000","last_clean":"2024-09-19T06:49:13.971733+0000","last_became_active":"2024-09-19T06:47:24.851044+0000","last_became_peered":"2024-09-19T06:47:24.851044+0000","last_unstale":"2024-09-19T06:49:13.971733+0000","last_undegraded":"2024-09-19T06:49:13.971733+0000","last_fullsized":"2024-09-19T06:49:13.971733+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:55.658214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932580+0000","last_change":"2024-09-19T06:47:22.833694+0000","last_active":"2024-09-19T06:49:13.932580+0000","last_peered":"2024-09-19T06:49:13.932580+0000","last_clean":"2024-09-19T06:49:13.932580+0000","last_became_active":"2024-09-19T06:47:22.833215+0000","last_became_peered":"2024-09-19T06:47:22.833215+0000","last_unstale":"2024-09-19T06:49:13.932580+0000","last_undegraded":"2024-09-19T06:49:13.932580+0000","last_fullsized":"2024-09-19T06:49:13.932580+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:32.171522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973551+0000","last_change":"2024-09-19T06:47:24.845737+0000","last_active":"2024-09-19T06:49:13.973551+0000","last_peered":"2024-09-19T06:49:13.973551+0000","last_clean":"2024-09-19T06:49:13.973551+0000","last_became_active":"2024-09-19T06:47:24.845242+0000","last_became_peered":"2024-09-19T06:47:24.845242+0000","last_unstale":"2024-09-19T06:49:13.973551+0000","last_undegraded":"2024-09-19T06:49:13.973551+0000","last_fullsized":"2024-09-19T06:49:13.973551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:51.554946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932499+0000","last_change":"2024-09-19T06:47:22.829869+0000","last_active":"2024-09-19T06:49:13.932499+0000","last_peered":"2024-09-19T06:49:13.932499+0000","last_clean":"2024-09-19T06:49:13.932499+0000","last_became_active":"2024-09-19T06:47:22.829522+0000","last_became_peered":"2024-09-19T06:47:22.829522+0000","last_unstale":"2024-09-19T06:49:13.932499+0000","last_undegraded":"2024-09-19T06:49:13.932499+0000","last_fullsized":"2024-09-19T06:49:13.932499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:26.871434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973419+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:49:13.973419+0000","last_peered":"2024-09-19T06:49:13.973419+0000","last_clean":"2024-09-19T06:49:13.973419+0000","last_became_active":"2024-09-19T06:47:24.841289+0000","last_became_peered":"2024-09-19T06:47:24.841289+0000","last_unstale":"2024-09-19T06:49:13.973419+0000","last_undegraded":"2024-09-19T06:49:13.973419+0000","last_fullsized":"2024-09-19T06:49:13.973419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:21:12.611458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.937218+0000","last_change":"2024-09-19T06:47:22.852198+0000","last_active":"2024-09-19T06:49:13.937218+0000","last_peered":"2024-09-19T06:49:13.937218+0000","last_clean":"2024-09-19T06:49:13.937218+0000","last_became_active":"2024-09-19T06:47:22.850984+0000","last_became_peered":"2024-09-19T06:47:22.850984+0000","last_unstale":"2024-09-19T06:49:13.937218+0000","last_undegraded":"2024-09-19T06:49:13.937218+0000","last_fullsized":"2024-09-19T06:49:13.937218+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:13.620145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959682+0000","last_change":"2024-09-19T06:47:40.837614+0000","last_active":"2024-09-19T06:49:13.959682+0000","last_peered":"2024-09-19T06:49:13.959682+0000","last_clean":"2024-09-19T06:49:13.959682+0000","last_became_active":"2024-09-19T06:47:40.837259+0000","last_became_peered":"2024-09-19T06:47:40.837259+0000","last_unstale":"2024-09-19T06:49:13.959682+0000","last_undegraded":"2024-09-19T06:49:13.959682+0000","last_fullsized":"2024-09-19T06:49:13.959682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:41.429962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":157,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.197340+0000","last_change":"2024-09-19T06:47:41.182639+0000","last_active":"2024-09-19T06:49:14.197340+0000","last_peered":"2024-09-19T06:49:14.197340+0000","last_clean":"2024-09-19T06:49:14.197340+0000","last_became_active":"2024-09-19T06:47:41.174305+0000","last_became_peered":"2024-09-19T06:47:41.174305+0000","last_unstale":"2024-09-19T06:49:14.197340+0000","last_undegraded":"2024-09-19T06:49:14.197340+0000","last_fullsized":"2024-09-19T06:49:14.197340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:10.199502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601319,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1124,"num_read_kb":676,"num_write":379,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1803957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579939,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32472,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712680,"statfs":{"total":95995035648,"available":95961784320,"internally_reserved":0,"allocated":1314816,"data_stored":842278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28864,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716288,"statfs":{"total":95995035648,"available":95965478912,"internally_reserved":0,"allocated":1761280,"data_stored":1238256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972397,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28332,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716820,"statfs":{"total":95995035648,"available":95966023680,"internally_reserved":0,"allocated":1335296,"data_stored":840798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234034,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":447288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462966,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32688,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712464,"statfs":{"total":95995035648,"available":95961563136,"internally_reserved":0,"allocated":1536000,"data_stored":1042501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626492,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":447692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855423,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1171668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018949,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":448114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215174,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32620,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712532,"statfs":{"total":95995035648,"available":95961632768,"internally_reserved":0,"allocated":1470464,"data_stored":971940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444112,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":516380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1003520,"data_stored":515624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":449172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":134723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":201311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":395818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:49:19.418 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:49:19.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:18 smithi134 ceph-mon[35356]: Updating smithi134:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:49:19.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:18 smithi134 ceph-mon[35356]: Updating smithi174:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:49:19.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:18 smithi134 ceph-mon[35356]: Updating smithi103:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:49:19.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:18 smithi134 ceph-mon[35356]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:49:19.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:18 smithi134 ceph-mon[35356]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:49:19.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:18 smithi134 ceph-mon[35356]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:49:19.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:18 smithi134 ceph-mon[35356]: Updating smithi134:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:49:19.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:18 smithi134 ceph-mon[35356]: Updating smithi174:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:49:19.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:18 smithi134 ceph-mon[35356]: Updating smithi103:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:49:19.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:18 smithi134 ceph-mon[35356]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:49:19.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:18 smithi134 ceph-mon[35356]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-09-19T06:49:19.702 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:20.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:19 smithi103 ceph-mon[31158]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:20.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.211+0000 7f0c1e427640 1 -- 172.21.15.103:0/3344777984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c18106f10 msgr2=0x7f0c1810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:20.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.211+0000 7f0c1e427640 1 --2- 172.21.15.103:0/3344777984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c18106f10 0x7f0c1810ea90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009ca0 tx=0x7f0c1002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:49:20.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.211+0000 7f0c1e427640 1 -- 172.21.15.103:0/3344777984 shutdown_connections 2024-09-19T06:49:20.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.211+0000 7f0c1e427640 1 --2- 172.21.15.103:0/3344777984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c18106f10 0x7f0c1810ea90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:20.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.211+0000 7f0c1e427640 1 --2- 172.21.15.103:0/3344777984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c18102790 0x7f0c181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:20.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.211+0000 7f0c1e427640 1 --2- 172.21.15.103:0/3344777984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c18101de0 0x7f0c181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:20.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.211+0000 7f0c1e427640 1 -- 172.21.15.103:0/3344777984 >> 172.21.15.103:0/3344777984 conn(0x7f0c180fd8c0 msgr2=0x7f0c180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:20.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.211+0000 7f0c1e427640 1 -- 172.21.15.103:0/3344777984 shutdown_connections 2024-09-19T06:49:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.211+0000 7f0c1e427640 1 -- 172.21.15.103:0/3344777984 wait complete. 2024-09-19T06:49:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.211+0000 7f0c1e427640 1 Processor -- start 2024-09-19T06:49:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.211+0000 7f0c1e427640 1 -- start start 2024-09-19T06:49:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1e427640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c18101de0 0x7f0c180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1e427640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c18102790 0x7f0c18074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1e427640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c18106f10 0x7f0c18074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1e427640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c18111f70 con 0x7f0c18106f10 2024-09-19T06:49:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1e427640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0c18111df0 con 0x7f0c18102790 2024-09-19T06:49:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1e427640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c181120f0 con 0x7f0c18101de0 2024-09-19T06:49:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1c9c4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c18106f10 0x7f0c18074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1c9c4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c18106f10 0x7f0c18074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42412/0 (socket says 172.21.15.103:42412) 2024-09-19T06:49:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1c9c4640 1 -- 172.21.15.103:0/534191733 learned_addr learned my addr 172.21.15.103:0/534191733 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:20.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c17fff640 1 --2- 172.21.15.103:0/534191733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c18101de0 0x7f0c180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:20.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c0ffff640 1 --2- 172.21.15.103:0/534191733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c18102790 0x7f0c18074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:20.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1c9c4640 1 -- 172.21.15.103:0/534191733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c18101de0 msgr2=0x7f0c180744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:20.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1c9c4640 1 --2- 172.21.15.103:0/534191733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c18101de0 0x7f0c180744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:20.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1c9c4640 1 -- 172.21.15.103:0/534191733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c18102790 msgr2=0x7f0c18074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:20.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1c9c4640 1 --2- 172.21.15.103:0/534191733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c18102790 0x7f0c18074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:20.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1c9c4640 1 -- 172.21.15.103:0/534191733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c181a6890 con 0x7f0c18106f10 2024-09-19T06:49:20.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1c9c4640 1 --2- 172.21.15.103:0/534191733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c18106f10 0x7f0c18074f70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0c10031a20 tx=0x7f0c10031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:20.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c15ffb640 1 -- 172.21.15.103:0/534191733 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10003c30 con 0x7f0c18106f10 2024-09-19T06:49:20.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1e427640 1 -- 172.21.15.103:0/534191733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c181a6b20 con 0x7f0c18106f10 2024-09-19T06:49:20.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c15ffb640 1 -- 172.21.15.103:0/534191733 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c10003dd0 con 0x7f0c18106f10 2024-09-19T06:49:20.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c15ffb640 1 -- 172.21.15.103:0/534191733 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1002fcf0 con 0x7f0c18106f10 2024-09-19T06:49:20.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.212+0000 7f0c1e427640 1 -- 172.21.15.103:0/534191733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c181a7060 con 0x7f0c18106f10 2024-09-19T06:49:20.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.213+0000 7f0c1e427640 1 -- 172.21.15.103:0/534191733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c181021c0 con 0x7f0c18106f10 2024-09-19T06:49:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.215+0000 7f0c15ffb640 1 -- 172.21.15.103:0/534191733 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f0c10004b40 con 0x7f0c18106f10 2024-09-19T06:49:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.215+0000 7f0c15ffb640 1 --2- 172.21.15.103:0/534191733 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0c040a34d0 0x7f0c040a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.215+0000 7f0c17fff640 1 --2- 172.21.15.103:0/534191733 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0c040a34d0 0x7f0c040a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.215+0000 7f0c15ffb640 1 -- 172.21.15.103:0/534191733 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f0c100f1370 con 0x7f0c18106f10 2024-09-19T06:49:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.216+0000 7f0c17fff640 1 --2- 172.21.15.103:0/534191733 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0c040a34d0 0x7f0c040a5990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0c08005fd0 tx=0x7f0c0800b390 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:20.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.218+0000 7f0c15ffb640 1 -- 172.21.15.103:0/534191733 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0c100bdcb0 con 0x7f0c18106f10 2024-09-19T06:49:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:19 smithi174 ceph-mon[35637]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.419+0000 7f0c1e427640 1 -- 172.21.15.103:0/534191733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c18075a20 con 0x7f0c18106f10 2024-09-19T06:49:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.420+0000 7f0c15ffb640 1 -- 172.21.15.103:0/534191733 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f0c18075a20 con 0x7f0c18106f10 2024-09-19T06:49:20.421 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:49:20.421 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:49:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:49:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.422+0000 7f0c1e427640 1 -- 172.21.15.103:0/534191733 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0c040a34d0 msgr2=0x7f0c040a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.422+0000 7f0c1e427640 1 --2- 172.21.15.103:0/534191733 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0c040a34d0 0x7f0c040a5990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0c08005fd0 tx=0x7f0c0800b390 comp rx=0 tx=0).stop 2024-09-19T06:49:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.422+0000 7f0c1e427640 1 -- 172.21.15.103:0/534191733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c18106f10 msgr2=0x7f0c18074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.422+0000 7f0c1e427640 1 --2- 172.21.15.103:0/534191733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c18106f10 0x7f0c18074f70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0c10031a20 tx=0x7f0c10031a50 comp rx=0 tx=0).stop 2024-09-19T06:49:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.422+0000 7f0c1e427640 1 -- 172.21.15.103:0/534191733 shutdown_connections 2024-09-19T06:49:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.422+0000 7f0c1e427640 1 --2- 172.21.15.103:0/534191733 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0c040a34d0 0x7f0c040a5990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.422+0000 7f0c1e427640 1 --2- 172.21.15.103:0/534191733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c18106f10 0x7f0c18074f70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.422+0000 7f0c1e427640 1 --2- 172.21.15.103:0/534191733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c18102790 0x7f0c18074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.422+0000 7f0c1e427640 1 --2- 172.21.15.103:0/534191733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c18101de0 0x7f0c180744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.422+0000 7f0c1e427640 1 -- 172.21.15.103:0/534191733 >> 172.21.15.103:0/534191733 conn(0x7f0c180fd8c0 msgr2=0x7f0c180feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.422+0000 7f0c1e427640 1 -- 172.21.15.103:0/534191733 shutdown_connections 2024-09-19T06:49:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:20.422+0000 7f0c1e427640 1 -- 172.21.15.103:0/534191733 wait complete. 2024-09-19T06:49:20.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:19 smithi134 ceph-mon[35356]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:21.115 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:20 smithi103 ceph-mon[31158]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:21.115 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:20 smithi103 ceph-mon[31158]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:21.115 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:20 smithi103 ceph-mon[31158]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:49:21.115 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:20 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/534191733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:21.116 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:49:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:20 smithi174 ceph-mon[35637]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:20 smithi174 ceph-mon[35637]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:20 smithi174 ceph-mon[35637]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:49:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:20 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/534191733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:21.364 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:21.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:20 smithi134 ceph-mon[35356]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:21.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:20 smithi134 ceph-mon[35356]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:21.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:20 smithi134 ceph-mon[35356]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:49:21.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:20 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/534191733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:21.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.809+0000 7f3133972640 1 -- 172.21.15.103:0/3009838697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f312c100e30 msgr2=0x7f312c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:21.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.809+0000 7f3133972640 1 --2- 172.21.15.103:0/3009838697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f312c100e30 0x7f312c104e80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3128009900 tx=0x7f312802a410 comp rx=0 tx=0).stop 2024-09-19T06:49:21.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.809+0000 7f3133972640 1 -- 172.21.15.103:0/3009838697 shutdown_connections 2024-09-19T06:49:21.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.809+0000 7f3133972640 1 --2- 172.21.15.103:0/3009838697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f312c1055b0 0x7f312c10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.809+0000 7f3133972640 1 --2- 172.21.15.103:0/3009838697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f312c100e30 0x7f312c104e80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.809+0000 7f3133972640 1 --2- 172.21.15.103:0/3009838697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f312c100480 0x7f312c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.809+0000 7f3133972640 1 -- 172.21.15.103:0/3009838697 >> 172.21.15.103:0/3009838697 conn(0x7f312c077210 msgr2=0x7f312c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:21.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.809+0000 7f3133972640 1 -- 172.21.15.103:0/3009838697 shutdown_connections 2024-09-19T06:49:21.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.809+0000 7f3133972640 1 -- 172.21.15.103:0/3009838697 wait complete. 2024-09-19T06:49:21.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3133972640 1 Processor -- start 2024-09-19T06:49:21.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3133972640 1 -- start start 2024-09-19T06:49:21.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3133972640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f312c100480 0x7f312c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3133972640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f312c100e30 0x7f312c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3133972640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f312c1055b0 0x7f312c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3133972640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f312c111fd0 con 0x7f312c1055b0 2024-09-19T06:49:21.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3133972640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f312c111e50 con 0x7f312c100e30 2024-09-19T06:49:21.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3133972640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f312c112150 con 0x7f312c100480 2024-09-19T06:49:21.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3131f0f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f312c1055b0 0x7f312c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3131f0f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f312c1055b0 0x7f312c19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42422/0 (socket says 172.21.15.103:42422) 2024-09-19T06:49:21.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3131f0f640 1 -- 172.21.15.103:0/3276768238 learned_addr learned my addr 172.21.15.103:0/3276768238 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:21.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f313170e640 1 --2- 172.21.15.103:0/3276768238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f312c100480 0x7f312c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3130f0d640 1 --2- 172.21.15.103:0/3276768238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f312c100e30 0x7f312c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3131f0f640 1 -- 172.21.15.103:0/3276768238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f312c100480 msgr2=0x7f312c198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:21.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3131f0f640 1 --2- 172.21.15.103:0/3276768238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f312c100480 0x7f312c198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3131f0f640 1 -- 172.21.15.103:0/3276768238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f312c100e30 msgr2=0x7f312c198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:21.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3131f0f640 1 --2- 172.21.15.103:0/3276768238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f312c100e30 0x7f312c198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.810+0000 7f3131f0f640 1 -- 172.21.15.103:0/3276768238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c19d520 con 0x7f312c1055b0 2024-09-19T06:49:21.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.811+0000 7f3131f0f640 1 --2- 172.21.15.103:0/3276768238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f312c1055b0 0x7f312c19ce40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f312400cad0 tx=0x7f312400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:21.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.811+0000 7f31227fc640 1 -- 172.21.15.103:0/3276768238 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312400ad60 con 0x7f312c1055b0 2024-09-19T06:49:21.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.811+0000 7f31227fc640 1 -- 172.21.15.103:0/3276768238 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3124005020 con 0x7f312c1055b0 2024-09-19T06:49:21.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.811+0000 7f3133972640 1 -- 172.21.15.103:0/3276768238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312c19d810 con 0x7f312c1055b0 2024-09-19T06:49:21.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.811+0000 7f31227fc640 1 -- 172.21.15.103:0/3276768238 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312400f6b0 con 0x7f312c1055b0 2024-09-19T06:49:21.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.811+0000 7f3133972640 1 -- 172.21.15.103:0/3276768238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312c1a50f0 con 0x7f312c1055b0 2024-09-19T06:49:21.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.812+0000 7f3133972640 1 -- 172.21.15.103:0/3276768238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f312c100860 con 0x7f312c1055b0 2024-09-19T06:49:21.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.813+0000 7f31227fc640 1 -- 172.21.15.103:0/3276768238 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f3124005560 con 0x7f312c1055b0 2024-09-19T06:49:21.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.813+0000 7f31227fc640 1 --2- 172.21.15.103:0/3276768238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f31180a34d0 0x7f31180a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.814+0000 7f31227fc640 1 -- 172.21.15.103:0/3276768238 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f31240d0000 con 0x7f312c1055b0 2024-09-19T06:49:21.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.814+0000 7f313170e640 1 --2- 172.21.15.103:0/3276768238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f31180a34d0 0x7f31180a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.814+0000 7f313170e640 1 --2- 172.21.15.103:0/3276768238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f31180a34d0 0x7f31180a5990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f311c009990 tx=0x7f311c009110 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:21.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.818+0000 7f31227fc640 1 -- 172.21.15.103:0/3276768238 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3124014030 con 0x7f312c1055b0 2024-09-19T06:49:21.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.991+0000 7f3133972640 1 -- 172.21.15.103:0/3276768238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f312c1087f0 con 0x7f312c1055b0 2024-09-19T06:49:21.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.992+0000 7f31227fc640 1 -- 172.21.15.103:0/3276768238 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+23173 (secure 0 0 0) 0x7f312409c7a0 con 0x7f312c1055b0 2024-09-19T06:49:21.993 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:49:21.993 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":83,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","created":"2024-09-19T06:39:46.840574+0000","modified":"2024-09-19T06:49:13.909427+0000","last_up_change":"2024-09-19T06:46:35.652960+0000","last_in_change":"2024-09-19T06:46:11.630585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:42:52.050881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:47:21.192611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:47:23.643291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d45dab3-e434-4783-b2e3-f658c3ba583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6803","nonce":3157697740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6805","nonce":3157697740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6809","nonce":3157697740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6807","nonce":3157697740}]},"public_addr":"172.21.15.103:6803/3157697740","cluster_addr":"172.21.15.103:6805/3157697740","heartbeat_back_addr":"172.21.15.103:6809/3157697740","heartbeat_front_addr":"172.21.15.103:6807/3157697740","state":["exists","up"]},{"osd":1,"uuid":"ee0e92bd-15ab-4867-9a18-20abb9f5fc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6801","nonce":1418301195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6803","nonce":1418301195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6807","nonce":1418301195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6805","nonce":1418301195}]},"public_addr":"172.21.15.134:6801/1418301195","cluster_addr":"172.21.15.134:6803/1418301195","heartbeat_back_addr":"172.21.15.134:6807/1418301195","heartbeat_front_addr":"172.21.15.134:6805/1418301195","state":["exists","up"]},{"osd":2,"uuid":"0230caed-025b-46e9-a011-237601d16ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6801","nonce":2292242290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6803","nonce":2292242290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6807","nonce":2292242290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6805","nonce":2292242290}]},"public_addr":"172.21.15.174:6801/2292242290","cluster_addr":"172.21.15.174:6803/2292242290","heartbeat_back_addr":"172.21.15.174:6807/2292242290","heartbeat_front_addr":"172.21.15.174:6805/2292242290","state":["exists","up"]},{"osd":3,"uuid":"c54a8399-bf4b-4d70-b207-a13d85f99590","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6811","nonce":2698986333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6813","nonce":2698986333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6817","nonce":2698986333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6815","nonce":2698986333}]},"public_addr":"172.21.15.103:6811/2698986333","cluster_addr":"172.21.15.103:6813/2698986333","heartbeat_back_addr":"172.21.15.103:6817/2698986333","heartbeat_front_addr":"172.21.15.103:6815/2698986333","state":["exists","up"]},{"osd":4,"uuid":"7c046be6-464a-4ab5-812b-1781121fc317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6809","nonce":3169891968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6811","nonce":3169891968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6815","nonce":3169891968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6813","nonce":3169891968}]},"public_addr":"172.21.15.134:6809/3169891968","cluster_addr":"172.21.15.134:6811/3169891968","heartbeat_back_addr":"172.21.15.134:6815/3169891968","heartbeat_front_addr":"172.21.15.134:6813/3169891968","state":["exists","up"]},{"osd":5,"uuid":"8622596b-6a06-459c-882d-50fba9e296ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6809","nonce":763749644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6811","nonce":763749644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6815","nonce":763749644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6813","nonce":763749644}]},"public_addr":"172.21.15.174:6809/763749644","cluster_addr":"172.21.15.174:6811/763749644","heartbeat_back_addr":"172.21.15.174:6815/763749644","heartbeat_front_addr":"172.21.15.174:6813/763749644","state":["exists","up"]},{"osd":6,"uuid":"64d858d6-a8b3-4f65-a187-af9ab1ccee0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6819","nonce":1396972394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6821","nonce":1396972394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6825","nonce":1396972394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6823","nonce":1396972394}]},"public_addr":"172.21.15.103:6819/1396972394","cluster_addr":"172.21.15.103:6821/1396972394","heartbeat_back_addr":"172.21.15.103:6825/1396972394","heartbeat_front_addr":"172.21.15.103:6823/1396972394","state":["exists","up"]},{"osd":7,"uuid":"d405d1c6-839c-4c92-8261-4adb3163c90f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6817","nonce":2196366765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6819","nonce":2196366765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6823","nonce":2196366765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6821","nonce":2196366765}]},"public_addr":"172.21.15.134:6817/2196366765","cluster_addr":"172.21.15.134:6819/2196366765","heartbeat_back_addr":"172.21.15.134:6823/2196366765","heartbeat_front_addr":"172.21.15.134:6821/2196366765","state":["exists","up"]},{"osd":8,"uuid":"4c452cd9-816c-4f77-9368-2cd4bbeee4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6817","nonce":189977465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6819","nonce":189977465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6823","nonce":189977465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6821","nonce":189977465}]},"public_addr":"172.21.15.174:6817/189977465","cluster_addr":"172.21.15.174:6819/189977465","heartbeat_back_addr":"172.21.15.174:6823/189977465","heartbeat_front_addr":"172.21.15.174:6821/189977465","state":["exists","up"]},{"osd":9,"uuid":"fb3662e0-2d8e-45c4-9119-6d08b35443da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6827","nonce":3161966413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6829","nonce":3161966413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6833","nonce":3161966413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6831","nonce":3161966413}]},"public_addr":"172.21.15.103:6827/3161966413","cluster_addr":"172.21.15.103:6829/3161966413","heartbeat_back_addr":"172.21.15.103:6833/3161966413","heartbeat_front_addr":"172.21.15.103:6831/3161966413","state":["exists","up"]},{"osd":10,"uuid":"e52c3cbe-a6af-441e-b423-b94f3a2da544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6825","nonce":2532758632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6827","nonce":2532758632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6831","nonce":2532758632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6829","nonce":2532758632}]},"public_addr":"172.21.15.134:6825/2532758632","cluster_addr":"172.21.15.134:6827/2532758632","heartbeat_back_addr":"172.21.15.134:6831/2532758632","heartbeat_front_addr":"172.21.15.134:6829/2532758632","state":["exists","up"]},{"osd":11,"uuid":"9040f3f8-3873-4dea-8716-dcda5811c8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6825","nonce":161916442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6827","nonce":161916442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6831","nonce":161916442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6829","nonce":161916442}]},"public_addr":"172.21.15.174:6825/161916442","cluster_addr":"172.21.15.174:6827/161916442","heartbeat_back_addr":"172.21.15.174:6831/161916442","heartbeat_front_addr":"172.21.15.174:6829/161916442","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:41:59.656854+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:24.797551+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:49.685497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:33.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:39.324983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:03.987310+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:28.747971+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:53.827780+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:18.554450+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:43.705691+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:08.401275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:33.502542+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2169657832":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/1596229922":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1642809745":"2024-09-20T06:40:16.780890+0000","172.21.15.103:6800/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/70266056":"2024-09-20T06:49:13.909393+0000","172.21.15.103:6801/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1496153385":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/471230312":"2024-09-20T06:49:13.909393+0000","172.21.15.103:6801/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/2309624637":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/3727259725":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/2590139910":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/2050139492":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/2867049580":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/2590139910":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/1074662744":"2024-09-20T06:49:13.909393+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:49:21.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.994+0000 7f3133972640 1 -- 172.21.15.103:0/3276768238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f31180a34d0 msgr2=0x7f31180a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:21.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.994+0000 7f3133972640 1 --2- 172.21.15.103:0/3276768238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f31180a34d0 0x7f31180a5990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f311c009990 tx=0x7f311c009110 comp rx=0 tx=0).stop 2024-09-19T06:49:21.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.994+0000 7f3133972640 1 -- 172.21.15.103:0/3276768238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f312c1055b0 msgr2=0x7f312c19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:21.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.994+0000 7f3133972640 1 --2- 172.21.15.103:0/3276768238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f312c1055b0 0x7f312c19ce40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f312400cad0 tx=0x7f312400ced0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.994+0000 7f3133972640 1 -- 172.21.15.103:0/3276768238 shutdown_connections 2024-09-19T06:49:21.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.994+0000 7f3133972640 1 --2- 172.21.15.103:0/3276768238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f31180a34d0 0x7f31180a5990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.994+0000 7f3133972640 1 --2- 172.21.15.103:0/3276768238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f312c1055b0 0x7f312c19ce40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.994+0000 7f3133972640 1 --2- 172.21.15.103:0/3276768238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f312c100e30 0x7f312c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.994+0000 7f3133972640 1 --2- 172.21.15.103:0/3276768238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f312c100480 0x7f312c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.994+0000 7f3133972640 1 -- 172.21.15.103:0/3276768238 >> 172.21.15.103:0/3276768238 conn(0x7f312c077210 msgr2=0x7f312c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:21.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.994+0000 7f3133972640 1 -- 172.21.15.103:0/3276768238 shutdown_connections 2024-09-19T06:49:21.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:21.994+0000 7f3133972640 1 -- 172.21.15.103:0/3276768238 wait complete. 2024-09-19T06:49:22.699 DEBUG:tasks.fwd_scrub:Yielding 2024-09-19T06:49:22.699 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T06:49:22.708 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T06:49:22.709 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T06:49:22.709 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-19T06:49:22.709 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-19T06:49:22.709 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:49:22.709 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:49:22.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.814+0000 7f172685e640 1 -- 172.21.15.103:0/932874508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720074f40 msgr2=0x7f172010ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:22.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.814+0000 7f172685e640 1 --2- 172.21.15.103:0/932874508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720074f40 0x7f172010ddd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1710009900 tx=0x7f171002a480 comp rx=0 tx=0).stop 2024-09-19T06:49:22.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.814+0000 7f172685e640 1 -- 172.21.15.103:0/932874508 shutdown_connections 2024-09-19T06:49:22.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.814+0000 7f172685e640 1 --2- 172.21.15.103:0/932874508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f172010e310 0x7f1720110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.814+0000 7f172685e640 1 --2- 172.21.15.103:0/932874508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720074f40 0x7f172010ddd0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.814+0000 7f172685e640 1 --2- 172.21.15.103:0/932874508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1720074620 0x7f1720074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.814+0000 7f172685e640 1 -- 172.21.15.103:0/932874508 >> 172.21.15.103:0/932874508 conn(0x7f17200fd7d0 msgr2=0x7f17200ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:22.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.814+0000 7f172685e640 1 -- 172.21.15.103:0/932874508 shutdown_connections 2024-09-19T06:49:22.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.814+0000 7f172685e640 1 -- 172.21.15.103:0/932874508 wait complete. 2024-09-19T06:49:22.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f172685e640 1 Processor -- start 2024-09-19T06:49:22.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f172685e640 1 -- start start 2024-09-19T06:49:22.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f172685e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1720074620 0x7f172019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f172685e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720074f40 0x7f172019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f172685e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f172010e310 0x7f172019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f172685e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17201140a0 con 0x7f1720074f40 2024-09-19T06:49:22.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f172685e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1720113f20 con 0x7f172010e310 2024-09-19T06:49:22.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f172685e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1720114220 con 0x7f1720074620 2024-09-19T06:49:22.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f171f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720074f40 0x7f172019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f171f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720074f40 0x7f172019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42444/0 (socket says 172.21.15.103:42444) 2024-09-19T06:49:22.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f171f7fe640 1 -- 172.21.15.103:0/2368243596 learned_addr learned my addr 172.21.15.103:0/2368243596 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:22.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f171ffff640 1 --2- 172.21.15.103:0/2368243596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1720074620 0x7f172019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f1724dfb640 1 --2- 172.21.15.103:0/2368243596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f172010e310 0x7f172019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f171f7fe640 1 -- 172.21.15.103:0/2368243596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1720074620 msgr2=0x7f172019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:22.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f171f7fe640 1 --2- 172.21.15.103:0/2368243596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1720074620 0x7f172019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f171f7fe640 1 -- 172.21.15.103:0/2368243596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f172010e310 msgr2=0x7f172019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:22.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f171f7fe640 1 --2- 172.21.15.103:0/2368243596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f172010e310 0x7f172019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f171f7fe640 1 -- 172.21.15.103:0/2368243596 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172019f570 con 0x7f1720074f40 2024-09-19T06:49:22.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.815+0000 7f171ffff640 1 --2- 172.21.15.103:0/2368243596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1720074620 0x7f172019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:22.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.816+0000 7f171f7fe640 1 --2- 172.21.15.103:0/2368243596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720074f40 0x7f172019aac0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1710006010 tx=0x7f17100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:22.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.816+0000 7f171d7fa640 1 -- 172.21.15.103:0/2368243596 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171003f030 con 0x7f1720074f40 2024-09-19T06:49:22.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.816+0000 7f172685e640 1 -- 172.21.15.103:0/2368243596 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172019f800 con 0x7f1720074f40 2024-09-19T06:49:22.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.816+0000 7f172685e640 1 -- 172.21.15.103:0/2368243596 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17201a70f0 con 0x7f1720074f40 2024-09-19T06:49:22.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.816+0000 7f171d7fa640 1 -- 172.21.15.103:0/2368243596 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1710003ce0 con 0x7f1720074f40 2024-09-19T06:49:22.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.816+0000 7f171d7fa640 1 -- 172.21.15.103:0/2368243596 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1710049060 con 0x7f1720074f40 2024-09-19T06:49:22.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.817+0000 7f172685e640 1 -- 172.21.15.103:0/2368243596 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1700005290 con 0x7f1720074f40 2024-09-19T06:49:22.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.818+0000 7f171d7fa640 1 -- 172.21.15.103:0/2368243596 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f1710042030 con 0x7f1720074f40 2024-09-19T06:49:22.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.819+0000 7f171d7fa640 1 --2- 172.21.15.103:0/2368243596 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f170c0a3580 0x7f170c0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.819+0000 7f171ffff640 1 --2- 172.21.15.103:0/2368243596 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f170c0a3580 0x7f170c0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.819+0000 7f171d7fa640 1 -- 172.21.15.103:0/2368243596 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f17100f0480 con 0x7f1720074f40 2024-09-19T06:49:22.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.819+0000 7f171ffff640 1 --2- 172.21.15.103:0/2368243596 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f170c0a3580 0x7f170c0a5a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1708005fd0 tx=0x7f1708005ea0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:22.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:22.826+0000 7f171d7fa640 1 -- 172.21.15.103:0/2368243596 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f17100bccf0 con 0x7f1720074f40 2024-09-19T06:49:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:23.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.008+0000 7f172685e640 1 -- 172.21.15.103:0/2368243596 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f1700005580 con 0x7f1720074f40 2024-09-19T06:49:23.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.012+0000 7f171d7fa640 1 -- 172.21.15.103:0/2368243596 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v27)=0 v27) ==== 129+0+0 (secure 0 0 0) 0x7f17100bc5f0 con 0x7f1720074f40 2024-09-19T06:49:23.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.014+0000 7f172685e640 1 -- 172.21.15.103:0/2368243596 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f170c0a3580 msgr2=0x7f170c0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.014+0000 7f172685e640 1 --2- 172.21.15.103:0/2368243596 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f170c0a3580 0x7f170c0a5a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1708005fd0 tx=0x7f1708005ea0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.014+0000 7f172685e640 1 -- 172.21.15.103:0/2368243596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720074f40 msgr2=0x7f172019aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.014+0000 7f172685e640 1 --2- 172.21.15.103:0/2368243596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720074f40 0x7f172019aac0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1710006010 tx=0x7f17100042f0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.014+0000 7f172685e640 1 -- 172.21.15.103:0/2368243596 shutdown_connections 2024-09-19T06:49:23.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.014+0000 7f172685e640 1 --2- 172.21.15.103:0/2368243596 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f170c0a3580 0x7f170c0a5a40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.014+0000 7f172685e640 1 --2- 172.21.15.103:0/2368243596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f172010e310 0x7f172019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.014+0000 7f172685e640 1 --2- 172.21.15.103:0/2368243596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720074f40 0x7f172019aac0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.014+0000 7f172685e640 1 --2- 172.21.15.103:0/2368243596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1720074620 0x7f172019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.014+0000 7f172685e640 1 -- 172.21.15.103:0/2368243596 >> 172.21.15.103:0/2368243596 conn(0x7f17200fd7d0 msgr2=0x7f17200fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:23.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.015+0000 7f172685e640 1 -- 172.21.15.103:0/2368243596 shutdown_connections 2024-09-19T06:49:23.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.015+0000 7f172685e640 1 -- 172.21.15.103:0/2368243596 wait complete. 2024-09-19T06:49:23.028 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-19T06:49:23.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.131+0000 7fae9fe82640 1 -- 172.21.15.103:0/198277334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae981035e0 msgr2=0x7fae9810bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.131+0000 7fae9fe82640 1 --2- 172.21.15.103:0/198277334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae981035e0 0x7fae9810bac0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fae88009900 tx=0x7fae8802a480 comp rx=0 tx=0).stop 2024-09-19T06:49:23.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.131+0000 7fae9fe82640 1 -- 172.21.15.103:0/198277334 shutdown_connections 2024-09-19T06:49:23.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.131+0000 7fae9fe82640 1 --2- 172.21.15.103:0/198277334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae9810c110 0x7fae9810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.131+0000 7fae9fe82640 1 --2- 172.21.15.103:0/198277334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae981035e0 0x7fae9810bac0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.131+0000 7fae9fe82640 1 --2- 172.21.15.103:0/198277334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae98102cc0 0x7fae981030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.131+0000 7fae9fe82640 1 -- 172.21.15.103:0/198277334 >> 172.21.15.103:0/198277334 conn(0x7fae980fc6b0 msgr2=0x7fae980fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:23.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.131+0000 7fae9fe82640 1 -- 172.21.15.103:0/198277334 shutdown_connections 2024-09-19T06:49:23.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.131+0000 7fae9fe82640 1 -- 172.21.15.103:0/198277334 wait complete. 2024-09-19T06:49:23.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.131+0000 7fae9fe82640 1 Processor -- start 2024-09-19T06:49:23.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.131+0000 7fae9fe82640 1 -- start start 2024-09-19T06:49:23.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9fe82640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae98102cc0 0x7fae98198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9fe82640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae981035e0 0x7fae981988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9fe82640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae9810c110 0x7fae9819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9fe82640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fae98111ef0 con 0x7fae9810c110 2024-09-19T06:49:23.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9fe82640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae98111d70 con 0x7fae98102cc0 2024-09-19T06:49:23.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9fe82640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fae98112070 con 0x7fae981035e0 2024-09-19T06:49:23.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9e41f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae9810c110 0x7fae9819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9e41f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae9810c110 0x7fae9819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42462/0 (socket says 172.21.15.103:42462) 2024-09-19T06:49:23.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9e41f640 1 -- 172.21.15.103:0/2759492856 learned_addr learned my addr 172.21.15.103:0/2759492856 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:23.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9d41d640 1 --2- 172.21.15.103:0/2759492856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae981035e0 0x7fae981988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9dc1e640 1 --2- 172.21.15.103:0/2759492856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae98102cc0 0x7fae98198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9e41f640 1 -- 172.21.15.103:0/2759492856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae981035e0 msgr2=0x7fae981988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9e41f640 1 --2- 172.21.15.103:0/2759492856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae981035e0 0x7fae981988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9e41f640 1 -- 172.21.15.103:0/2759492856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae98102cc0 msgr2=0x7fae98198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9e41f640 1 --2- 172.21.15.103:0/2759492856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae98102cc0 0x7fae98198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9e41f640 1 -- 172.21.15.103:0/2759492856 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9819d370 con 0x7fae9810c110 2024-09-19T06:49:23.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9d41d640 1 --2- 172.21.15.103:0/2759492856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae981035e0 0x7fae981988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:23.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9dc1e640 1 --2- 172.21.15.103:0/2759492856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae98102cc0 0x7fae98198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:23.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9e41f640 1 --2- 172.21.15.103:0/2759492856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae9810c110 0x7fae9819cc40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fae9000eaa0 tx=0x7fae9000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:23.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae8effd640 1 -- 172.21.15.103:0/2759492856 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9000cdb0 con 0x7fae9810c110 2024-09-19T06:49:23.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae8effd640 1 -- 172.21.15.103:0/2759492856 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae90005020 con 0x7fae9810c110 2024-09-19T06:49:23.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae9fe82640 1 -- 172.21.15.103:0/2759492856 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9819d660 con 0x7fae9810c110 2024-09-19T06:49:23.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.132+0000 7fae8effd640 1 -- 172.21.15.103:0/2759492856 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae900116b0 con 0x7fae9810c110 2024-09-19T06:49:23.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.133+0000 7fae9fe82640 1 -- 172.21.15.103:0/2759492856 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae981a4ef0 con 0x7fae9810c110 2024-09-19T06:49:23.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.133+0000 7fae9fe82640 1 -- 172.21.15.103:0/2759492856 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae981030a0 con 0x7fae9810c110 2024-09-19T06:49:23.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.135+0000 7fae8effd640 1 -- 172.21.15.103:0/2759492856 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7fae90005560 con 0x7fae9810c110 2024-09-19T06:49:23.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.135+0000 7fae8effd640 1 --2- 172.21.15.103:0/2759492856 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fae840a3600 0x7fae840a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.135+0000 7fae8effd640 1 -- 172.21.15.103:0/2759492856 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fae900d05a0 con 0x7fae9810c110 2024-09-19T06:49:23.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.135+0000 7fae9dc1e640 1 --2- 172.21.15.103:0/2759492856 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fae840a3600 0x7fae840a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.136+0000 7fae9dc1e640 1 --2- 172.21.15.103:0/2759492856 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fae840a3600 0x7fae840a5ac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fae9400abe0 tx=0x7fae9400a5c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:23.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.139+0000 7fae8effd640 1 -- 172.21.15.103:0/2759492856 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fae90016030 con 0x7fae9810c110 2024-09-19T06:49:23.243 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:22 smithi103 ceph-mon[31158]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.0 KiB/s wr, 25 op/s 2024-09-19T06:49:23.243 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:22 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3276768238' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:49:23.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.313+0000 7fae9fe82640 1 -- 172.21.15.103:0/2759492856 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fae981071c0 con 0x7fae9810c110 2024-09-19T06:49:23.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.318+0000 7fae8effd640 1 -- 172.21.15.103:0/2759492856 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7fae9009cd80 con 0x7fae9810c110 2024-09-19T06:49:23.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.320+0000 7fae9fe82640 1 -- 172.21.15.103:0/2759492856 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fae840a3600 msgr2=0x7fae840a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.320+0000 7fae9fe82640 1 --2- 172.21.15.103:0/2759492856 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fae840a3600 0x7fae840a5ac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fae9400abe0 tx=0x7fae9400a5c0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.320+0000 7fae9fe82640 1 -- 172.21.15.103:0/2759492856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae9810c110 msgr2=0x7fae9819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.320+0000 7fae9fe82640 1 --2- 172.21.15.103:0/2759492856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae9810c110 0x7fae9819cc40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fae9000eaa0 tx=0x7fae9000ef60 comp rx=0 tx=0).stop 2024-09-19T06:49:23.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.320+0000 7fae9fe82640 1 -- 172.21.15.103:0/2759492856 shutdown_connections 2024-09-19T06:49:23.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.320+0000 7fae9fe82640 1 --2- 172.21.15.103:0/2759492856 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fae840a3600 0x7fae840a5ac0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.320+0000 7fae9fe82640 1 --2- 172.21.15.103:0/2759492856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae9810c110 0x7fae9819cc40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.320+0000 7fae9fe82640 1 --2- 172.21.15.103:0/2759492856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae981035e0 0x7fae981988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.320+0000 7fae9fe82640 1 --2- 172.21.15.103:0/2759492856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae98102cc0 0x7fae98198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.320+0000 7fae9fe82640 1 -- 172.21.15.103:0/2759492856 >> 172.21.15.103:0/2759492856 conn(0x7fae980fc6b0 msgr2=0x7fae980fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:23.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.320+0000 7fae9fe82640 1 -- 172.21.15.103:0/2759492856 shutdown_connections 2024-09-19T06:49:23.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.320+0000 7fae9fe82640 1 -- 172.21.15.103:0/2759492856 wait complete. 2024-09-19T06:49:23.333 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-19T06:49:23.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:22 smithi174 ceph-mon[35637]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.0 KiB/s wr, 25 op/s 2024-09-19T06:49:23.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:22 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3276768238' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:49:23.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:22 smithi134 ceph-mon[35356]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.0 KiB/s wr, 25 op/s 2024-09-19T06:49:23.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:22 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3276768238' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:49:23.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.438+0000 7fc505265640 1 -- 172.21.15.103:0/1565428755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc500102cf0 msgr2=0x7fc5001030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.438+0000 7fc505265640 1 --2- 172.21.15.103:0/1565428755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc500102cf0 0x7fc5001030d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009900 tx=0x7fc4f002a480 comp rx=0 tx=0).stop 2024-09-19T06:49:23.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.438+0000 7fc505265640 1 -- 172.21.15.103:0/1565428755 shutdown_connections 2024-09-19T06:49:23.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.438+0000 7fc505265640 1 --2- 172.21.15.103:0/1565428755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc50010c0c0 0x7fc50010e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.438+0000 7fc505265640 1 --2- 172.21.15.103:0/1565428755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc500103610 0x7fc50010baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.438+0000 7fc505265640 1 --2- 172.21.15.103:0/1565428755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc500102cf0 0x7fc5001030d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.438+0000 7fc505265640 1 -- 172.21.15.103:0/1565428755 >> 172.21.15.103:0/1565428755 conn(0x7fc5000fc700 msgr2=0x7fc5000feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:23.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.438+0000 7fc505265640 1 -- 172.21.15.103:0/1565428755 shutdown_connections 2024-09-19T06:49:23.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.438+0000 7fc505265640 1 -- 172.21.15.103:0/1565428755 wait complete. 2024-09-19T06:49:23.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.439+0000 7fc505265640 1 Processor -- start 2024-09-19T06:49:23.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.439+0000 7fc505265640 1 -- start start 2024-09-19T06:49:23.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.439+0000 7fc505265640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc500102cf0 0x7fc500198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.439+0000 7fc505265640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc500103610 0x7fc5001988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.439+0000 7fc505265640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50010c0c0 0x7fc50019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.439+0000 7fc505265640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc500111ec0 con 0x7fc50010c0c0 2024-09-19T06:49:23.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.439+0000 7fc505265640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc500111d40 con 0x7fc500102cf0 2024-09-19T06:49:23.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.439+0000 7fc505265640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc500112040 con 0x7fc500103610 2024-09-19T06:49:23.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.439+0000 7fc4ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50010c0c0 0x7fc50019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.439+0000 7fc4ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50010c0c0 0x7fc50019cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42472/0 (socket says 172.21.15.103:42472) 2024-09-19T06:49:23.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.439+0000 7fc4ff7fe640 1 -- 172.21.15.103:0/465295771 learned_addr learned my addr 172.21.15.103:0/465295771 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:23.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.439+0000 7fc4f7fff640 1 --2- 172.21.15.103:0/465295771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc500103610 0x7fc5001988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.439+0000 7fc4feffd640 1 --2- 172.21.15.103:0/465295771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc500102cf0 0x7fc500198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.440+0000 7fc4ff7fe640 1 -- 172.21.15.103:0/465295771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc500103610 msgr2=0x7fc5001988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.440+0000 7fc4ff7fe640 1 --2- 172.21.15.103:0/465295771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc500103610 0x7fc5001988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.440+0000 7fc4ff7fe640 1 -- 172.21.15.103:0/465295771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc500102cf0 msgr2=0x7fc500198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.440+0000 7fc4ff7fe640 1 --2- 172.21.15.103:0/465295771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc500102cf0 0x7fc500198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.440+0000 7fc4ff7fe640 1 -- 172.21.15.103:0/465295771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50019d380 con 0x7fc50010c0c0 2024-09-19T06:49:23.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.440+0000 7fc4ff7fe640 1 --2- 172.21.15.103:0/465295771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50010c0c0 0x7fc50019cc50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800eb10 tx=0x7fc4f800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:23.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.440+0000 7fc4fcff9640 1 -- 172.21.15.103:0/465295771 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8004b20 con 0x7fc50010c0c0 2024-09-19T06:49:23.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.440+0000 7fc4fcff9640 1 -- 172.21.15.103:0/465295771 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f8004cc0 con 0x7fc50010c0c0 2024-09-19T06:49:23.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.440+0000 7fc505265640 1 -- 172.21.15.103:0/465295771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50019d670 con 0x7fc50010c0c0 2024-09-19T06:49:23.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.440+0000 7fc4fcff9640 1 -- 172.21.15.103:0/465295771 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8010760 con 0x7fc50010c0c0 2024-09-19T06:49:23.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.440+0000 7fc505265640 1 -- 172.21.15.103:0/465295771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5001a4f00 con 0x7fc50010c0c0 2024-09-19T06:49:23.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.441+0000 7fc505265640 1 -- 172.21.15.103:0/465295771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5001030d0 con 0x7fc50010c0c0 2024-09-19T06:49:23.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.442+0000 7fc4fcff9640 1 -- 172.21.15.103:0/465295771 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7fc4f8005020 con 0x7fc50010c0c0 2024-09-19T06:49:23.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.443+0000 7fc4fcff9640 1 --2- 172.21.15.103:0/465295771 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc4ec0a3600 0x7fc4ec0a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.443+0000 7fc4fcff9640 1 -- 172.21.15.103:0/465295771 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc4f80d0450 con 0x7fc50010c0c0 2024-09-19T06:49:23.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.443+0000 7fc4feffd640 1 --2- 172.21.15.103:0/465295771 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc4ec0a3600 0x7fc4ec0a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.444+0000 7fc4feffd640 1 --2- 172.21.15.103:0/465295771 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc4ec0a3600 0x7fc4ec0a5ac0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009230 tx=0x7fc4f00058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:23.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.447+0000 7fc4fcff9640 1 -- 172.21.15.103:0/465295771 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc4f809cbf0 con 0x7fc50010c0c0 2024-09-19T06:49:23.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.499+0000 7fecf6c83640 1 -- 172.21.15.103:0/4108755835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecf0100100 msgr2=0x7fecf0100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.499+0000 7fecf6c83640 1 --2- 172.21.15.103:0/4108755835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecf0100100 0x7fecf0100560 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009900 tx=0x7fecdc02a410 comp rx=0 tx=0).stop 2024-09-19T06:49:23.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.499+0000 7fecf6c83640 1 -- 172.21.15.103:0/4108755835 shutdown_connections 2024-09-19T06:49:23.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.499+0000 7fecf6c83640 1 --2- 172.21.15.103:0/4108755835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecf0100aa0 0x7fecf010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.499+0000 7fecf6c83640 1 --2- 172.21.15.103:0/4108755835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecf0100100 0x7fecf0100560 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.499+0000 7fecf6c83640 1 --2- 172.21.15.103:0/4108755835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecf0106b80 0x7fecf0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.499+0000 7fecf6c83640 1 -- 172.21.15.103:0/4108755835 >> 172.21.15.103:0/4108755835 conn(0x7fecf0077230 msgr2=0x7fecf00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:23.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.499+0000 7fecf6c83640 1 -- 172.21.15.103:0/4108755835 shutdown_connections 2024-09-19T06:49:23.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.499+0000 7fecf6c83640 1 -- 172.21.15.103:0/4108755835 wait complete. 2024-09-19T06:49:23.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7fecf6c83640 1 Processor -- start 2024-09-19T06:49:23.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7fecf6c83640 1 -- start start 2024-09-19T06:49:23.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7fecf6c83640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecf0100100 0x7fecf01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7fecf6c83640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecf0100aa0 0x7fecf0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7fecf6c83640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecf0106b80 0x7fecf019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7fecf6c83640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fecf01120a0 con 0x7fecf0100aa0 2024-09-19T06:49:23.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7fecf6c83640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fecf0111f20 con 0x7fecf0100100 2024-09-19T06:49:23.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7fecf6c83640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fecf0112220 con 0x7fecf0106b80 2024-09-19T06:49:23.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7feceffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecf0100aa0 0x7fecf0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7feceffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecf0100aa0 0x7fecf0198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42490/0 (socket says 172.21.15.103:42490) 2024-09-19T06:49:23.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7feceffff640 1 -- 172.21.15.103:0/460075257 learned_addr learned my addr 172.21.15.103:0/460075257 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:23.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7fecf5220640 1 --2- 172.21.15.103:0/460075257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecf0106b80 0x7fecf019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7fecf4a1f640 1 --2- 172.21.15.103:0/460075257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecf0100100 0x7fecf01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7feceffff640 1 -- 172.21.15.103:0/460075257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecf0106b80 msgr2=0x7fecf019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7feceffff640 1 --2- 172.21.15.103:0/460075257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecf0106b80 0x7fecf019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7feceffff640 1 -- 172.21.15.103:0/460075257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecf0100100 msgr2=0x7fecf01985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7feceffff640 1 --2- 172.21.15.103:0/460075257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecf0100100 0x7fecf01985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.500+0000 7feceffff640 1 -- 172.21.15.103:0/460075257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf019d550 con 0x7fecf0100aa0 2024-09-19T06:49:23.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.501+0000 7feceffff640 1 --2- 172.21.15.103:0/460075257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecf0100aa0 0x7fecf0198ae0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fecdc004770 tx=0x7fecdc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:23.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.501+0000 7fecedffb640 1 -- 172.21.15.103:0/460075257 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc03f070 con 0x7fecf0100aa0 2024-09-19T06:49:23.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.501+0000 7fecf6c83640 1 -- 172.21.15.103:0/460075257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf019d7e0 con 0x7fecf0100aa0 2024-09-19T06:49:23.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.501+0000 7fecedffb640 1 -- 172.21.15.103:0/460075257 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecdc031ca0 con 0x7fecf0100aa0 2024-09-19T06:49:23.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.501+0000 7fecedffb640 1 -- 172.21.15.103:0/460075257 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc035030 con 0x7fecf0100aa0 2024-09-19T06:49:23.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.501+0000 7fecf6c83640 1 -- 172.21.15.103:0/460075257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf01a5120 con 0x7fecf0100aa0 2024-09-19T06:49:23.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.503+0000 7fecedffb640 1 -- 172.21.15.103:0/460075257 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7fecdc042070 con 0x7fecf0100aa0 2024-09-19T06:49:23.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf905c740 1 Processor -- start 2024-09-19T06:49:23.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf905c740 1 -- start start 2024-09-19T06:49:23.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf905c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4c8a60 0x564d5f4c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf905c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4d7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf905c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4d8150 0x564d5f4da540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf905c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564d5f42e280 con 0x564d5f4c9380 2024-09-19T06:49:23.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf905c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564d5f3d5b10 con 0x564d5f4c8a60 2024-09-19T06:49:23.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf905c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564d5f42a240 con 0x564d5f4d8150 2024-09-19T06:49:23.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf4a1f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4d7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf4a1f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4d7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42504/0 (socket says 172.21.15.103:42504) 2024-09-19T06:49:23.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf4a1f640 1 -- 172.21.15.103:0/1159716013 learned_addr learned my addr 172.21.15.103:0/1159716013 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:23.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf5220640 1 --2- 172.21.15.103:0/1159716013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4c8a60 0x564d5f4c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7feceffff640 1 --2- 172.21.15.103:0/1159716013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4d8150 0x564d5f4da540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf4a1f640 1 -- 172.21.15.103:0/1159716013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4d8150 msgr2=0x564d5f4da540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:23.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf4a1f640 1 --2- 172.21.15.103:0/1159716013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4d8150 0x564d5f4da540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf4a1f640 1 -- 172.21.15.103:0/1159716013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4c8a60 msgr2=0x564d5f4c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:23.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf4a1f640 1 --2- 172.21.15.103:0/1159716013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4c8a60 0x564d5f4c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecf4a1f640 1 -- 172.21.15.103:0/1159716013 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d5f3e21f0 con 0x564d5f4c9380 2024-09-19T06:49:23.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7fecedffb640 1 --2- 172.21.15.103:0/460075257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fece00a3550 0x7fece00a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.504+0000 7feceffff640 1 --2- 172.21.15.103:0/1159716013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4d8150 0x564d5f4da540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:23.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.505+0000 7feceffff640 1 --2- 172.21.15.103:0/460075257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fece00a3550 0x7fece00a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.505+0000 7fecf4a1f640 1 --2- 172.21.15.103:0/1159716013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4d7c10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fece400bc20 tx=0x7fece400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c55f234cbcad9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:23.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.505+0000 7fecedffb640 1 -- 172.21.15.103:0/460075257 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fecdc0f0640 con 0x7fecf0100aa0 2024-09-19T06:49:23.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.505+0000 7fecdaffd640 1 -- 172.21.15.103:0/1159716013 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4015070 con 0x564d5f4c9380 2024-09-19T06:49:23.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.505+0000 7fecdaffd640 1 -- 172.21.15.103:0/1159716013 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece400be50 con 0x564d5f4c9380 2024-09-19T06:49:23.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.505+0000 7fecdaffd640 1 -- 172.21.15.103:0/1159716013 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece40116a0 con 0x564d5f4c9380 2024-09-19T06:49:23.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.505+0000 7feceffff640 1 --2- 172.21.15.103:0/460075257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fece00a3550 0x7fece00a5a10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0edc80 tx=0x7fecdc02a410 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:23.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.506+0000 7fecf905c740 1 -- 172.21.15.103:0/1159716013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 msgr2=0x564d5f4d7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.506+0000 7fecf905c740 1 --2- 172.21.15.103:0/1159716013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4d7c10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fece400bc20 tx=0x7fece400f730 comp rx=0 tx=0).stop 2024-09-19T06:49:23.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.506+0000 7fecf905c740 1 -- 172.21.15.103:0/1159716013 shutdown_connections 2024-09-19T06:49:23.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.506+0000 7fecf905c740 1 --2- 172.21.15.103:0/1159716013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4d8150 0x564d5f4da540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.506+0000 7fecf905c740 1 --2- 172.21.15.103:0/1159716013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4d7c10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.506+0000 7fecf905c740 1 --2- 172.21.15.103:0/1159716013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4c8a60 0x564d5f4c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.506+0000 7fecf905c740 1 -- 172.21.15.103:0/1159716013 >> 172.21.15.103:0/1159716013 conn(0x564d5f4c50f0 msgr2=0x564d5f4c54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:23.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.506+0000 7fecf905c740 1 -- 172.21.15.103:0/1159716013 shutdown_connections 2024-09-19T06:49:23.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.506+0000 7fecf905c740 1 -- 172.21.15.103:0/1159716013 wait complete. 2024-09-19T06:49:23.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.506+0000 7fecf905c740 1 -- start start 2024-09-19T06:49:23.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.506+0000 7fecf905c740 1 Processor -- start 2024-09-19T06:49:23.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf905c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4c8a60 0x564d5f4d50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf905c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4c9380 0x564d5f4d55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf905c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4d8150 0x564d5f4d5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf905c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564d5f4dad80 con 0x564d5f4d8150 2024-09-19T06:49:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf905c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564d5f3e8f10 con 0x564d5f4c8a60 2024-09-19T06:49:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf905c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564d5f4daf00 con 0x564d5f4c9380 2024-09-19T06:49:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf905c740 10 client.0 _ll_register_callbacks cb 0x564d5f3b1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf5220640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4d8150 0x564d5f4d5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf5220640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4d8150 0x564d5f4d5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42518/0 (socket says 172.21.15.103:42518) 2024-09-19T06:49:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf5220640 1 -- 172.21.15.103:0/2365563028 learned_addr learned my addr 172.21.15.103:0/2365563028 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf5220640 1 --2- 172.21.15.103:0/2365563028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4c8a60 0x564d5f4d50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf4a1f640 1 --2- 172.21.15.103:0/2365563028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4c9380 0x564d5f4d55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf5220640 1 -- 172.21.15.103:0/2365563028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4c9380 msgr2=0x564d5f4d55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf5220640 1 --2- 172.21.15.103:0/2365563028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4c9380 0x564d5f4d55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf5220640 1 -- 172.21.15.103:0/2365563028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4c8a60 msgr2=0x564d5f4d50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf5220640 1 --2- 172.21.15.103:0/2365563028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4c8a60 0x564d5f4d50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf5220640 1 -- 172.21.15.103:0/2365563028 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d5f19f6a0 con 0x564d5f4d8150 2024-09-19T06:49:23.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.507+0000 7fecf4a1f640 1 --2- 172.21.15.103:0/2365563028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4c9380 0x564d5f4d55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.508+0000 7fecf5220640 1 --2- 172.21.15.103:0/2365563028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4d8150 0x564d5f4d5b20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fece8010e60 tx=0x7fece800a4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2501f5eeb3a2717 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.508+0000 7feccffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.508+0000 7feccffff640 1 -- 172.21.15.103:0/2365563028 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8014020 con 0x564d5f4d8150 2024-09-19T06:49:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.508+0000 7feccffff640 1 -- 172.21.15.103:0/2365563028 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fece800ebc0 con 0x564d5f4d8150 2024-09-19T06:49:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.508+0000 7fecf905c740 1 -- 172.21.15.103:0/2365563028 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fece400be50 con 0x564d5f4d8150 2024-09-19T06:49:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.508+0000 7feccffff640 1 -- 172.21.15.103:0/2365563028 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fece8027050 con 0x564d5f4d8150 2024-09-19T06:49:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.508+0000 7fecf905c740 10 client.15366 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.508+0000 7fecf905c740 1 -- 172.21.15.103:0/2365563028 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564d5f4e6a10 con 0x564d5f4d8150 2024-09-19T06:49:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.508+0000 7feccffff640 1 -- 172.21.15.103:0/2365563028 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece400be50 con 0x564d5f4d8150 2024-09-19T06:49:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.508+0000 7feccffff640 1 -- 172.21.15.103:0/2365563028 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fece8004260 con 0x564d5f4d8150 2024-09-19T06:49:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.508+0000 7fecf905c740 10 client.15366 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.508+0000 7fecf905c740 10 client.15366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.508+0000 7fecf905c740 20 client.15366 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.508+0000 7fecf905c740 1 --2- 172.21.15.103:0/2365563028 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564d5f4e7e20 0x564d5f4ea2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.508+0000 7fecf905c740 4 client.15366 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.509+0000 7fecf905c740 1 -- 172.21.15.103:0/2365563028 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d5f43b7f0 con 0x564d5f4e7e20 2024-09-19T06:49:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.509+0000 7fecf4a1f640 1 --2- 172.21.15.103:0/2365563028 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564d5f4e7e20 0x564d5f4ea2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.509+0000 7feccffff640 10 client.15366 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.509+0000 7fecf4a1f640 1 --2- 172.21.15.103:0/2365563028 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564d5f4e7e20 0x564d5f4ea2a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49b2005b63a43f24 server_cookie=17c9ccfca43b1230 in_seq=0 out_seq=0 2024-09-19T06:49:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.510+0000 7feccffff640 1 -- 172.21.15.103:0/2365563028 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d5f43b7f0 con 0x564d5f4e7e20 2024-09-19T06:49:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.510+0000 7feccffff640 10 client.15366 handle_command_reply: tid=0 2024-09-19T06:49:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.510+0000 7fecf905c740 1 client.15366 shutdown 2024-09-19T06:49:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.510+0000 7fecf905c740 1 -- 172.21.15.103:0/2365563028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4d8150 msgr2=0x564d5f4d5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.510+0000 7fecf905c740 1 --2- 172.21.15.103:0/2365563028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4d8150 0x564d5f4d5b20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fece8010e60 tx=0x7fece800a4f0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.510+0000 7fecf905c740 1 -- 172.21.15.103:0/2365563028 shutdown_connections 2024-09-19T06:49:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.510+0000 7fecf905c740 1 --2- 172.21.15.103:0/2365563028 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564d5f4e7e20 0x564d5f4ea2a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.510+0000 7fecf905c740 1 --2- 172.21.15.103:0/2365563028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4d8150 0x564d5f4d5b20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.510+0000 7fecf905c740 1 --2- 172.21.15.103:0/2365563028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4c9380 0x564d5f4d55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.510+0000 7fecf905c740 1 --2- 172.21.15.103:0/2365563028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4c8a60 0x564d5f4d50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.510+0000 7fecf905c740 1 -- 172.21.15.103:0/2365563028 >> 172.21.15.103:0/2365563028 conn(0x564d5f4c50f0 msgr2=0x564d5f4cae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:23.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.510+0000 7feccffff640 0 client.15366 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:23.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.510+0000 7fecf905c740 1 -- 172.21.15.103:0/2365563028 shutdown_connections 2024-09-19T06:49:23.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.510+0000 7fecf905c740 1 -- 172.21.15.103:0/2365563028 wait complete. 2024-09-19T06:49:23.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.510+0000 7fecf905c740 20 client.15366 trim_cache size 0 max 16384 2024-09-19T06:49:23.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.527+0000 7fecf905c740 1 Processor -- start 2024-09-19T06:49:23.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.527+0000 7fecf905c740 1 -- start start 2024-09-19T06:49:23.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.527+0000 7fecf905c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4c8a60 0x564d5f4d6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.527+0000 7fecf905c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4efb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.527+0000 7fecf905c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4d8150 0x564d5f4f00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.527+0000 7fecf905c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fecf01ab700 con 0x564d5f4c9380 2024-09-19T06:49:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.527+0000 7fecf905c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fecf01ad240 con 0x564d5f4d8150 2024-09-19T06:49:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.527+0000 7fecf905c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fecf01abdd0 con 0x564d5f4c8a60 2024-09-19T06:49:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.528+0000 7fecf4a1f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4efb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.528+0000 7fecf4a1f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4efb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42526/0 (socket says 172.21.15.103:42526) 2024-09-19T06:49:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.528+0000 7fecf4a1f640 1 -- 172.21.15.103:0/834394433 learned_addr learned my addr 172.21.15.103:0/834394433 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.528+0000 7fecf5220640 1 --2- 172.21.15.103:0/834394433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4d8150 0x564d5f4f00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.528+0000 7fecf5220640 1 --2- 172.21.15.103:0/834394433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4c8a60 0x564d5f4d6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.528+0000 7fecf4a1f640 1 -- 172.21.15.103:0/834394433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4c8a60 msgr2=0x564d5f4d6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.528+0000 7fecf4a1f640 1 --2- 172.21.15.103:0/834394433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4c8a60 0x564d5f4d6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.528+0000 7fecf4a1f640 1 -- 172.21.15.103:0/834394433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4d8150 msgr2=0x564d5f4f00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.528+0000 7fecf4a1f640 1 --2- 172.21.15.103:0/834394433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4d8150 0x564d5f4f00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.528+0000 7fecf4a1f640 1 -- 172.21.15.103:0/834394433 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d5f4d4050 con 0x564d5f4c9380 2024-09-19T06:49:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.528+0000 7fecf5220640 1 --2- 172.21.15.103:0/834394433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4c8a60 0x564d5f4d6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.528+0000 7fecf4a1f640 1 --2- 172.21.15.103:0/834394433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4efb70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x564d5f4cb750 tx=0x7fece400ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4a9cb519c449d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.528+0000 7fecda7fc640 1 -- 172.21.15.103:0/834394433 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4015040 con 0x564d5f4c9380 2024-09-19T06:49:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.528+0000 7fecda7fc640 1 -- 172.21.15.103:0/834394433 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece400bc50 con 0x564d5f4c9380 2024-09-19T06:49:23.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.528+0000 7fecda7fc640 1 -- 172.21.15.103:0/834394433 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece40054b0 con 0x564d5f4c9380 2024-09-19T06:49:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.529+0000 7fecf905c740 1 -- 172.21.15.103:0/834394433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 msgr2=0x564d5f4efb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.529+0000 7fecf905c740 1 --2- 172.21.15.103:0/834394433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4efb70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x564d5f4cb750 tx=0x7fece400ff60 comp rx=0 tx=0).stop 2024-09-19T06:49:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.529+0000 7fecf905c740 1 -- 172.21.15.103:0/834394433 shutdown_connections 2024-09-19T06:49:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.529+0000 7fecf905c740 1 --2- 172.21.15.103:0/834394433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4d8150 0x564d5f4f00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.529+0000 7fecf905c740 1 --2- 172.21.15.103:0/834394433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4efb70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.529+0000 7fecf905c740 1 --2- 172.21.15.103:0/834394433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4c8a60 0x564d5f4d6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.529+0000 7fecf905c740 1 -- 172.21.15.103:0/834394433 >> 172.21.15.103:0/834394433 conn(0x564d5f4c50f0 msgr2=0x564d5f4d67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.529+0000 7fecf905c740 1 -- 172.21.15.103:0/834394433 shutdown_connections 2024-09-19T06:49:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.529+0000 7fecf905c740 1 -- 172.21.15.103:0/834394433 wait complete. 2024-09-19T06:49:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.529+0000 7fecf905c740 1 -- start start 2024-09-19T06:49:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf905c740 1 Processor -- start 2024-09-19T06:49:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf905c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4c8a60 0x564d5f4dc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf905c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4dcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf905c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4d8150 0x564d5f551b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf905c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fecf01acb70 con 0x564d5f4c9380 2024-09-19T06:49:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf905c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fecf01ac4a0 con 0x564d5f4c8a60 2024-09-19T06:49:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf905c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fecf01a96e0 con 0x564d5f4d8150 2024-09-19T06:49:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf905c740 10 client.0 _ll_register_callbacks cb 0x7fece800aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:23.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf4a1f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4dcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf4a1f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4dcac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42532/0 (socket says 172.21.15.103:42532) 2024-09-19T06:49:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf4a1f640 1 -- 172.21.15.103:0/3498469367 learned_addr learned my addr 172.21.15.103:0/3498469367 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf5220640 1 --2- 172.21.15.103:0/3498469367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4c8a60 0x564d5f4dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf5220640 1 --2- 172.21.15.103:0/3498469367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4d8150 0x564d5f551b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf4a1f640 1 -- 172.21.15.103:0/3498469367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4d8150 msgr2=0x564d5f551b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf4a1f640 1 --2- 172.21.15.103:0/3498469367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4d8150 0x564d5f551b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf4a1f640 1 -- 172.21.15.103:0/3498469367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4c8a60 msgr2=0x564d5f4dc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.530+0000 7fecf4a1f640 1 --2- 172.21.15.103:0/3498469367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4c8a60 0x564d5f4dc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.531+0000 7fecf4a1f640 1 -- 172.21.15.103:0/3498469367 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d5f552080 con 0x564d5f4c9380 2024-09-19T06:49:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.531+0000 7fecf4a1f640 1 --2- 172.21.15.103:0/3498469367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4dcac0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fece4028040 tx=0x7fece4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5728bcf90ab6662 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.531+0000 7feccffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.531+0000 7feccffff640 1 -- 172.21.15.103:0/3498469367 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4015030 con 0x564d5f4c9380 2024-09-19T06:49:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.531+0000 7feccffff640 1 -- 172.21.15.103:0/3498469367 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fece4005250 con 0x564d5f4c9380 2024-09-19T06:49:23.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.531+0000 7fecf905c740 1 -- 172.21.15.103:0/3498469367 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fece400bc50 con 0x564d5f4c9380 2024-09-19T06:49:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.531+0000 7feccffff640 1 -- 172.21.15.103:0/3498469367 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fece403d050 con 0x564d5f4c9380 2024-09-19T06:49:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.531+0000 7fecf905c740 10 client.15372 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.531+0000 7fecf905c740 1 -- 172.21.15.103:0/3498469367 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564d5f553290 con 0x564d5f4c9380 2024-09-19T06:49:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.531+0000 7feccffff640 1 -- 172.21.15.103:0/3498469367 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece400bc50 con 0x564d5f4c9380 2024-09-19T06:49:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.532+0000 7feccffff640 1 -- 172.21.15.103:0/3498469367 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fece4038030 con 0x564d5f4c9380 2024-09-19T06:49:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.532+0000 7fecf905c740 10 client.15372 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.532+0000 7fecf905c740 10 client.15372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.532+0000 7fecf905c740 20 client.15372 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.532+0000 7fecf905c740 1 --2- 172.21.15.103:0/3498469367 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564d5f553790 0x564d5f555bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.532+0000 7fecf905c740 4 client.15372 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:49:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.532+0000 7fecf905c740 1 -- 172.21.15.103:0/3498469367 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564d5f435d00 con 0x564d5f553790 2024-09-19T06:49:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.532+0000 7fecf5220640 1 --2- 172.21.15.103:0/3498469367 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564d5f553790 0x564d5f555bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.532+0000 7feccffff640 10 client.15372 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.532+0000 7fecf5220640 1 --2- 172.21.15.103:0/3498469367 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564d5f553790 0x564d5f555bb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2b0734e5204f29d server_cookie=6af9113edfd3ef2f in_seq=0 out_seq=0 2024-09-19T06:49:23.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.533+0000 7feccffff640 1 -- 172.21.15.103:0/3498469367 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564d5f435d00 con 0x564d5f553790 2024-09-19T06:49:23.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.533+0000 7feccffff640 10 client.15372 handle_command_reply: tid=0 2024-09-19T06:49:23.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.533+0000 7fecf905c740 1 client.15372 shutdown 2024-09-19T06:49:23.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.533+0000 7fecf905c740 1 -- 172.21.15.103:0/3498469367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 msgr2=0x564d5f4dcac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.533+0000 7fecf905c740 1 --2- 172.21.15.103:0/3498469367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4dcac0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fece4028040 tx=0x7fece4005850 comp rx=0 tx=0).stop 2024-09-19T06:49:23.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.533+0000 7fecf905c740 1 -- 172.21.15.103:0/3498469367 shutdown_connections 2024-09-19T06:49:23.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.533+0000 7fecf905c740 1 --2- 172.21.15.103:0/3498469367 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564d5f553790 0x564d5f555bb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.534+0000 7feccffff640 0 client.15372 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:23.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.534+0000 7fecf905c740 1 --2- 172.21.15.103:0/3498469367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d5f4d8150 0x564d5f551b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.534+0000 7fecf905c740 1 --2- 172.21.15.103:0/3498469367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d5f4c9380 0x564d5f4dcac0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.534+0000 7fecf905c740 1 --2- 172.21.15.103:0/3498469367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d5f4c8a60 0x564d5f4dc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.534+0000 7fecf905c740 1 -- 172.21.15.103:0/3498469367 >> 172.21.15.103:0/3498469367 conn(0x564d5f4c50f0 msgr2=0x564d5f4d5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:23.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.534+0000 7fecf905c740 1 -- 172.21.15.103:0/3498469367 shutdown_connections 2024-09-19T06:49:23.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.534+0000 7fecf905c740 1 -- 172.21.15.103:0/3498469367 wait complete. 2024-09-19T06:49:23.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.534+0000 7fecf905c740 20 client.15372 trim_cache size 0 max 16384 2024-09-19T06:49:23.540 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:49:23.540 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T06:49:23.540 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "3c9bffcb-1f20-4240-95a8-7d1c977ae872", 2024-09-19T06:49:23.540 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T06:49:23.540 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:49:23.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.535+0000 7fecf6c83640 1 -- 172.21.15.103:0/460075257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fece00a3550 msgr2=0x7fece00a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.535+0000 7fecf6c83640 1 --2- 172.21.15.103:0/460075257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fece00a3550 0x7fece00a5a10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0edc80 tx=0x7fecdc02a410 comp rx=0 tx=0).stop 2024-09-19T06:49:23.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.535+0000 7fecf6c83640 1 -- 172.21.15.103:0/460075257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecf0100aa0 msgr2=0x7fecf0198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.535+0000 7fecf6c83640 1 --2- 172.21.15.103:0/460075257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecf0100aa0 0x7fecf0198ae0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fecdc004770 tx=0x7fecdc003c10 comp rx=0 tx=0).stop 2024-09-19T06:49:23.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.535+0000 7fecf6c83640 1 -- 172.21.15.103:0/460075257 shutdown_connections 2024-09-19T06:49:23.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.535+0000 7fecf6c83640 1 --2- 172.21.15.103:0/460075257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fece00a3550 0x7fece00a5a10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.535+0000 7fecf6c83640 1 --2- 172.21.15.103:0/460075257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecf0106b80 0x7fecf019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.535+0000 7fecf6c83640 1 --2- 172.21.15.103:0/460075257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecf0100aa0 0x7fecf0198ae0 secure :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fecdc004770 tx=0x7fecdc003c10 comp rx=0 tx=0).stop 2024-09-19T06:49:23.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.535+0000 7fecf6c83640 1 --2- 172.21.15.103:0/460075257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecf0100100 0x7fecf01985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.535+0000 7fecf6c83640 1 -- 172.21.15.103:0/460075257 >> 172.21.15.103:0/460075257 conn(0x7fecf0077230 msgr2=0x7fecf0104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:23.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.535+0000 7fecf6c83640 1 -- 172.21.15.103:0/460075257 shutdown_connections 2024-09-19T06:49:23.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.535+0000 7fecf6c83640 1 -- 172.21.15.103:0/460075257 wait complete. 2024-09-19T06:49:23.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.634+0000 7fc505265640 1 -- 172.21.15.103:0/465295771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fc5001071d0 con 0x7fc50010c0c0 2024-09-19T06:49:23.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.638+0000 7fc4fcff9640 1 -- 172.21.15.103:0/465295771 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7fc4f809c4f0 con 0x7fc50010c0c0 2024-09-19T06:49:23.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.641+0000 7fc505265640 1 -- 172.21.15.103:0/465295771 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc4ec0a3600 msgr2=0x7fc4ec0a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.641+0000 7fc505265640 1 --2- 172.21.15.103:0/465295771 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc4ec0a3600 0x7fc4ec0a5ac0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009230 tx=0x7fc4f00058d0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.641+0000 7fc505265640 1 -- 172.21.15.103:0/465295771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50010c0c0 msgr2=0x7fc50019cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.641+0000 7fc505265640 1 --2- 172.21.15.103:0/465295771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50010c0c0 0x7fc50019cc50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800eb10 tx=0x7fc4f800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.641+0000 7fc505265640 1 -- 172.21.15.103:0/465295771 shutdown_connections 2024-09-19T06:49:23.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.641+0000 7fc505265640 1 --2- 172.21.15.103:0/465295771 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc4ec0a3600 0x7fc4ec0a5ac0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.641+0000 7fc505265640 1 --2- 172.21.15.103:0/465295771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50010c0c0 0x7fc50019cc50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.641+0000 7fc505265640 1 --2- 172.21.15.103:0/465295771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc500103610 0x7fc5001988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.641+0000 7fc505265640 1 --2- 172.21.15.103:0/465295771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc500102cf0 0x7fc500198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.641+0000 7fc505265640 1 -- 172.21.15.103:0/465295771 >> 172.21.15.103:0/465295771 conn(0x7fc5000fc700 msgr2=0x7fc5000fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:23.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.641+0000 7fc505265640 1 -- 172.21.15.103:0/465295771 shutdown_connections 2024-09-19T06:49:23.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.641+0000 7fc505265640 1 -- 172.21.15.103:0/465295771 wait complete. 2024-09-19T06:49:23.654 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-19T06:49:23.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.763+0000 7f443181e640 1 -- 172.21.15.103:0/4163979522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f442c107b70 msgr2=0x7f442c107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.763+0000 7f443181e640 1 --2- 172.21.15.103:0/4163979522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f442c107b70 0x7f442c107f50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f441c009cd0 tx=0x7f441c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.763+0000 7f443181e640 1 -- 172.21.15.103:0/4163979522 shutdown_connections 2024-09-19T06:49:23.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.763+0000 7f443181e640 1 --2- 172.21.15.103:0/4163979522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f442c102470 0x7f442c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.763+0000 7f443181e640 1 --2- 172.21.15.103:0/4163979522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f442c101ad0 0x7f442c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.763+0000 7f443181e640 1 --2- 172.21.15.103:0/4163979522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f442c107b70 0x7f442c107f50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.763+0000 7f443181e640 1 -- 172.21.15.103:0/4163979522 >> 172.21.15.103:0/4163979522 conn(0x7f442c0fd7b0 msgr2=0x7f442c0ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:23.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.763+0000 7f443181e640 1 -- 172.21.15.103:0/4163979522 shutdown_connections 2024-09-19T06:49:23.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.763+0000 7f443181e640 1 -- 172.21.15.103:0/4163979522 wait complete. 2024-09-19T06:49:23.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.763+0000 7f443181e640 1 Processor -- start 2024-09-19T06:49:23.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.763+0000 7f443181e640 1 -- start start 2024-09-19T06:49:23.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f443181e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f442c101ad0 0x7f442c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f443181e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f442c102470 0x7f442c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f443181e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f442c107b70 0x7f442c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f443181e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f442c111f20 con 0x7f442c107b70 2024-09-19T06:49:23.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f443181e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f442c111da0 con 0x7f442c101ad0 2024-09-19T06:49:23.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f443181e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f442c1120a0 con 0x7f442c102470 2024-09-19T06:49:23.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f442b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f442c107b70 0x7f442c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f442b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f442c107b70 0x7f442c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42548/0 (socket says 172.21.15.103:42548) 2024-09-19T06:49:23.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f442b7fe640 1 -- 172.21.15.103:0/573521122 learned_addr learned my addr 172.21.15.103:0/573521122 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:23.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f442a7fc640 1 --2- 172.21.15.103:0/573521122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f442c102470 0x7f442c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f442affd640 1 --2- 172.21.15.103:0/573521122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f442c101ad0 0x7f442c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f442b7fe640 1 -- 172.21.15.103:0/573521122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f442c102470 msgr2=0x7f442c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f442b7fe640 1 --2- 172.21.15.103:0/573521122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f442c102470 0x7f442c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f442b7fe640 1 -- 172.21.15.103:0/573521122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f442c101ad0 msgr2=0x7f442c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:23.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f442b7fe640 1 --2- 172.21.15.103:0/573521122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f442c101ad0 0x7f442c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:23.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f442b7fe640 1 -- 172.21.15.103:0/573521122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442c1a68b0 con 0x7f442c107b70 2024-09-19T06:49:23.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f442affd640 1 --2- 172.21.15.103:0/573521122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f442c101ad0 0x7f442c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:23.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f442a7fc640 1 --2- 172.21.15.103:0/573521122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f442c102470 0x7f442c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:23.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f442b7fe640 1 --2- 172.21.15.103:0/573521122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f442c107b70 0x7f442c074f70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f442400d750 tx=0x7f442400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:23.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f443081c640 1 -- 172.21.15.103:0/573521122 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424017070 con 0x7f442c107b70 2024-09-19T06:49:23.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f443081c640 1 -- 172.21.15.103:0/573521122 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4424005500 con 0x7f442c107b70 2024-09-19T06:49:23.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f443181e640 1 -- 172.21.15.103:0/573521122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442c1a6ba0 con 0x7f442c107b70 2024-09-19T06:49:23.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.764+0000 7f443081c640 1 -- 172.21.15.103:0/573521122 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424004d90 con 0x7f442c107b70 2024-09-19T06:49:23.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.765+0000 7f443181e640 1 -- 172.21.15.103:0/573521122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f442c1a70f0 con 0x7f442c107b70 2024-09-19T06:49:23.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.765+0000 7f443181e640 1 -- 172.21.15.103:0/573521122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f442c101f30 con 0x7f442c107b70 2024-09-19T06:49:23.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.768+0000 7f443081c640 1 -- 172.21.15.103:0/573521122 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f4424004f30 con 0x7f442c107b70 2024-09-19T06:49:23.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.768+0000 7f443081c640 1 --2- 172.21.15.103:0/573521122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f44140a3600 0x7f44140a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:23.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.768+0000 7f443081c640 1 -- 172.21.15.103:0/573521122 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f44240cfc60 con 0x7f442c107b70 2024-09-19T06:49:23.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.768+0000 7f442affd640 1 --2- 172.21.15.103:0/573521122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f44140a3600 0x7f44140a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:23.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.769+0000 7f442affd640 1 --2- 172.21.15.103:0/573521122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f44140a3600 0x7f44140a5ac0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f441c000f80 tx=0x7f441c0093f0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:23.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.771+0000 7f443081c640 1 -- 172.21.15.103:0/573521122 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f442409c480 con 0x7f442c107b70 2024-09-19T06:49:23.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:23.965+0000 7f443181e640 1 -- 172.21.15.103:0/573521122 --> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f442c0fe6d0 con 0x7f44140a3600 2024-09-19T06:49:24.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.031+0000 7f443081c640 1 -- 172.21.15.103:0/573521122 <== mgr.15210 v2:172.21.15.103:6800/82882809 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f442c0fe6d0 con 0x7f44140a3600 2024-09-19T06:49:24.032 INFO:teuthology.orchestra.run.smithi103.stdout:Schedule set for path / 2024-09-19T06:49:24.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.033+0000 7f443181e640 1 -- 172.21.15.103:0/573521122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f44140a3600 msgr2=0x7f44140a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.033+0000 7f443181e640 1 --2- 172.21.15.103:0/573521122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f44140a3600 0x7f44140a5ac0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f441c000f80 tx=0x7f441c0093f0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.033+0000 7f443181e640 1 -- 172.21.15.103:0/573521122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f442c107b70 msgr2=0x7f442c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.033+0000 7f443181e640 1 --2- 172.21.15.103:0/573521122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f442c107b70 0x7f442c074f70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f442400d750 tx=0x7f442400dc10 comp rx=0 tx=0).stop 2024-09-19T06:49:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.033+0000 7f443181e640 1 -- 172.21.15.103:0/573521122 shutdown_connections 2024-09-19T06:49:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.033+0000 7f443181e640 1 --2- 172.21.15.103:0/573521122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f44140a3600 0x7f44140a5ac0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.033+0000 7f443181e640 1 --2- 172.21.15.103:0/573521122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f442c107b70 0x7f442c074f70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.033+0000 7f443181e640 1 --2- 172.21.15.103:0/573521122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f442c102470 0x7f442c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.033+0000 7f443181e640 1 --2- 172.21.15.103:0/573521122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f442c101ad0 0x7f442c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.033+0000 7f443181e640 1 -- 172.21.15.103:0/573521122 >> 172.21.15.103:0/573521122 conn(0x7f442c0fd7b0 msgr2=0x7f442c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.033+0000 7f443181e640 1 -- 172.21.15.103:0/573521122 shutdown_connections 2024-09-19T06:49:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.033+0000 7f443181e640 1 -- 172.21.15.103:0/573521122 wait complete. 2024-09-19T06:49:24.045 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-19T06:49:24.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.148+0000 7faf6afd6640 1 -- 172.21.15.103:0/3522413997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf64101350 msgr2=0x7faf6410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.148+0000 7faf6afd6640 1 --2- 172.21.15.103:0/3522413997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf64101350 0x7faf6410d880 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009930 tx=0x7faf5c02a420 comp rx=0 tx=0).stop 2024-09-19T06:49:24.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.148+0000 7faf6afd6640 1 -- 172.21.15.103:0/3522413997 shutdown_connections 2024-09-19T06:49:24.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.148+0000 7faf6afd6640 1 --2- 172.21.15.103:0/3522413997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf64101350 0x7faf6410d880 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.148+0000 7faf6afd6640 1 --2- 172.21.15.103:0/3522413997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf641009b0 0x7faf64100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.148+0000 7faf6afd6640 1 --2- 172.21.15.103:0/3522413997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf641069c0 0x7faf64106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.148+0000 7faf6afd6640 1 -- 172.21.15.103:0/3522413997 >> 172.21.15.103:0/3522413997 conn(0x7faf64077990 msgr2=0x7faf640fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:24.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.149+0000 7faf6afd6640 1 -- 172.21.15.103:0/3522413997 shutdown_connections 2024-09-19T06:49:24.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.149+0000 7faf6afd6640 1 -- 172.21.15.103:0/3522413997 wait complete. 2024-09-19T06:49:24.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.149+0000 7faf6afd6640 1 Processor -- start 2024-09-19T06:49:24.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.149+0000 7faf6afd6640 1 -- start start 2024-09-19T06:49:24.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.149+0000 7faf6afd6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf641009b0 0x7faf64198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.149+0000 7faf6afd6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf64101350 0x7faf641988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.149+0000 7faf6afd6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf641069c0 0x7faf6419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.149+0000 7faf6afd6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faf64110e00 con 0x7faf64101350 2024-09-19T06:49:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.149+0000 7faf6afd6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faf64110c80 con 0x7faf641009b0 2024-09-19T06:49:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.149+0000 7faf6afd6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faf64110f80 con 0x7faf641069c0 2024-09-19T06:49:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.149+0000 7faf5bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf64101350 0x7faf641988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.149+0000 7faf5bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf64101350 0x7faf641988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42566/0 (socket says 172.21.15.103:42566) 2024-09-19T06:49:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.149+0000 7faf5bfff640 1 -- 172.21.15.103:0/3428036686 learned_addr learned my addr 172.21.15.103:0/3428036686 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.150+0000 7faf68d72640 1 --2- 172.21.15.103:0/3428036686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf641009b0 0x7faf64198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.150+0000 7faf69573640 1 --2- 172.21.15.103:0/3428036686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf641069c0 0x7faf6419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.150+0000 7faf5bfff640 1 -- 172.21.15.103:0/3428036686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf641069c0 msgr2=0x7faf6419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.150+0000 7faf5bfff640 1 --2- 172.21.15.103:0/3428036686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf641069c0 0x7faf6419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.150+0000 7faf5bfff640 1 -- 172.21.15.103:0/3428036686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf641009b0 msgr2=0x7faf64198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.150+0000 7faf5bfff640 1 --2- 172.21.15.103:0/3428036686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf641009b0 0x7faf64198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.150+0000 7faf5bfff640 1 -- 172.21.15.103:0/3428036686 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6419d370 con 0x7faf64101350 2024-09-19T06:49:24.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.150+0000 7faf69573640 1 --2- 172.21.15.103:0/3428036686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf641069c0 0x7faf6419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:24.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.150+0000 7faf5bfff640 1 --2- 172.21.15.103:0/3428036686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf64101350 0x7faf641988c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7faf5400d770 tx=0x7faf5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:24.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.150+0000 7faf59ffb640 1 -- 172.21.15.103:0/3428036686 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54017070 con 0x7faf64101350 2024-09-19T06:49:24.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.150+0000 7faf59ffb640 1 -- 172.21.15.103:0/3428036686 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf54005500 con 0x7faf64101350 2024-09-19T06:49:24.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.150+0000 7faf6afd6640 1 -- 172.21.15.103:0/3428036686 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6419d660 con 0x7faf64101350 2024-09-19T06:49:24.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.150+0000 7faf59ffb640 1 -- 172.21.15.103:0/3428036686 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54004e00 con 0x7faf64101350 2024-09-19T06:49:24.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.150+0000 7faf6afd6640 1 -- 172.21.15.103:0/3428036686 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf641a4ef0 con 0x7faf64101350 2024-09-19T06:49:24.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.151+0000 7faf6afd6640 1 -- 172.21.15.103:0/3428036686 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf64100e10 con 0x7faf64101350 2024-09-19T06:49:24.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.152+0000 7faf59ffb640 1 -- 172.21.15.103:0/3428036686 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7faf54004fa0 con 0x7faf64101350 2024-09-19T06:49:24.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.153+0000 7faf59ffb640 1 --2- 172.21.15.103:0/3428036686 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faf4c0a3580 0x7faf4c0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.153+0000 7faf68d72640 1 --2- 172.21.15.103:0/3428036686 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faf4c0a3580 0x7faf4c0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.153+0000 7faf59ffb640 1 -- 172.21.15.103:0/3428036686 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7faf540cfa40 con 0x7faf64101350 2024-09-19T06:49:24.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.153+0000 7faf68d72640 1 --2- 172.21.15.103:0/3428036686 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faf4c0a3580 0x7faf4c0a5a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faf60005fd0 tx=0x7faf60009500 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:24.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.157+0000 7faf59ffb640 1 -- 172.21.15.103:0/3428036686 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faf540d1e20 con 0x7faf64101350 2024-09-19T06:49:24.208 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:49:24.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2368243596' entity='client.admin' 2024-09-19T06:49:24.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:24.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:24.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:24.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:24.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2759492856' entity='client.admin' 2024-09-19T06:49:24.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:24.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:24.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:24.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:24.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T06:49:24.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:49:24.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T06:49:24.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:49:24.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/465295771' entity='client.admin' 2024-09-19T06:49:24.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:24.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:24.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:24.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:24.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 797 B/s wr, 20 op/s 2024-09-19T06:49:24.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:24 smithi174 ceph-mon[35637]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:49:24.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.353+0000 7faf6afd6640 1 -- 172.21.15.103:0/3428036686 --> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7faf64103620 con 0x7faf4c0a3580 2024-09-19T06:49:24.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2368243596' entity='client.admin' 2024-09-19T06:49:24.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:24.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:24.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:24.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:24.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2759492856' entity='client.admin' 2024-09-19T06:49:24.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:24.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:24.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:24.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:24.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T06:49:24.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:49:24.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T06:49:24.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:49:24.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/465295771' entity='client.admin' 2024-09-19T06:49:24.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:24.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:24.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:24.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:24.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 797 B/s wr, 20 op/s 2024-09-19T06:49:24.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:24 smithi103 ceph-mon[31158]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:49:24.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.416+0000 7faf59ffb640 1 -- 172.21.15.103:0/3428036686 <== mgr.15210 v2:172.21.15.103:6800/82882809 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7faf64103620 con 0x7faf4c0a3580 2024-09-19T06:49:24.417 INFO:teuthology.orchestra.run.smithi103.stdout:Retention added to path / 2024-09-19T06:49:24.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.419+0000 7faf6afd6640 1 -- 172.21.15.103:0/3428036686 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faf4c0a3580 msgr2=0x7faf4c0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.419+0000 7faf6afd6640 1 --2- 172.21.15.103:0/3428036686 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faf4c0a3580 0x7faf4c0a5a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faf60005fd0 tx=0x7faf60009500 comp rx=0 tx=0).stop 2024-09-19T06:49:24.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.419+0000 7faf6afd6640 1 -- 172.21.15.103:0/3428036686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf64101350 msgr2=0x7faf641988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.419+0000 7faf6afd6640 1 --2- 172.21.15.103:0/3428036686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf64101350 0x7faf641988c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7faf5400d770 tx=0x7faf5400dc30 comp rx=0 tx=0).stop 2024-09-19T06:49:24.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.419+0000 7faf6afd6640 1 -- 172.21.15.103:0/3428036686 shutdown_connections 2024-09-19T06:49:24.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.419+0000 7faf6afd6640 1 --2- 172.21.15.103:0/3428036686 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faf4c0a3580 0x7faf4c0a5a40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.419+0000 7faf6afd6640 1 --2- 172.21.15.103:0/3428036686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf641069c0 0x7faf6419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.419+0000 7faf6afd6640 1 --2- 172.21.15.103:0/3428036686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf64101350 0x7faf641988c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.419+0000 7faf6afd6640 1 --2- 172.21.15.103:0/3428036686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf641009b0 0x7faf64198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.419+0000 7faf6afd6640 1 -- 172.21.15.103:0/3428036686 >> 172.21.15.103:0/3428036686 conn(0x7faf64077990 msgr2=0x7faf640ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:24.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.420+0000 7faf6afd6640 1 -- 172.21.15.103:0/3428036686 shutdown_connections 2024-09-19T06:49:24.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.420+0000 7faf6afd6640 1 -- 172.21.15.103:0/3428036686 wait complete. 2024-09-19T06:49:24.433 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T06:49:24.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2368243596' entity='client.admin' 2024-09-19T06:49:24.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:24.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:24.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:24.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:24.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2759492856' entity='client.admin' 2024-09-19T06:49:24.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:24.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:24.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:24.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:24.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T06:49:24.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:49:24.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T06:49:24.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:49:24.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/465295771' entity='client.admin' 2024-09-19T06:49:24.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:24.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:24.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:24.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:49:24.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 797 B/s wr, 20 op/s 2024-09-19T06:49:24.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:24 smithi134 ceph-mon[35356]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:49:24.470 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:24.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.539+0000 7f3468991640 1 -- 172.21.15.103:0/3343731721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3464101350 msgr2=0x7f346410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.539+0000 7f3468991640 1 --2- 172.21.15.103:0/3343731721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3464101350 0x7f346410d880 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f345c009cd0 tx=0x7f345c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.539+0000 7f3468991640 1 -- 172.21.15.103:0/3343731721 shutdown_connections 2024-09-19T06:49:24.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.539+0000 7f3468991640 1 --2- 172.21.15.103:0/3343731721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3464101350 0x7f346410d880 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.539+0000 7f3468991640 1 --2- 172.21.15.103:0/3343731721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34641009b0 0x7f3464100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.539+0000 7f3468991640 1 --2- 172.21.15.103:0/3343731721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34641069c0 0x7f3464106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.539+0000 7f3468991640 1 -- 172.21.15.103:0/3343731721 >> 172.21.15.103:0/3343731721 conn(0x7f3464077990 msgr2=0x7f34640fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:24.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.539+0000 7f3468991640 1 -- 172.21.15.103:0/3343731721 shutdown_connections 2024-09-19T06:49:24.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.539+0000 7f3468991640 1 -- 172.21.15.103:0/3343731721 wait complete. 2024-09-19T06:49:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.540+0000 7f3468991640 1 Processor -- start 2024-09-19T06:49:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.540+0000 7f3468991640 1 -- start start 2024-09-19T06:49:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.540+0000 7f3468991640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34641009b0 0x7f3464199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.540+0000 7f3468991640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3464101350 0x7f34641996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.540+0000 7f3468991640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34641069c0 0x7f34641a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.540+0000 7f3468991640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3464110e00 con 0x7f3464101350 2024-09-19T06:49:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.540+0000 7f3468991640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3464110c80 con 0x7f34641009b0 2024-09-19T06:49:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.540+0000 7f3468991640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3464110f80 con 0x7f34641069c0 2024-09-19T06:49:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.540+0000 7f3461d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3464101350 0x7f34641996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.540+0000 7f3461d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3464101350 0x7f34641996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42578/0 (socket says 172.21.15.103:42578) 2024-09-19T06:49:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.540+0000 7f3461d9b640 1 -- 172.21.15.103:0/1865620643 learned_addr learned my addr 172.21.15.103:0/1865620643 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.540+0000 7f346259c640 1 --2- 172.21.15.103:0/1865620643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34641009b0 0x7f3464199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.541+0000 7f3461d9b640 1 -- 172.21.15.103:0/1865620643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34641069c0 msgr2=0x7f34641a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.541+0000 7f3461d9b640 1 --2- 172.21.15.103:0/1865620643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34641069c0 0x7f34641a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.541+0000 7f3461d9b640 1 -- 172.21.15.103:0/1865620643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34641009b0 msgr2=0x7f3464199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.541+0000 7f3461d9b640 1 --2- 172.21.15.103:0/1865620643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34641009b0 0x7f3464199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.541+0000 7f3461d9b640 1 -- 172.21.15.103:0/1865620643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34641a4730 con 0x7f3464101350 2024-09-19T06:49:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.541+0000 7f346259c640 1 --2- 172.21.15.103:0/1865620643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34641009b0 0x7f3464199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.541+0000 7f3461d9b640 1 --2- 172.21.15.103:0/1865620643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3464101350 0x7f34641996b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f345800d950 tx=0x7f345800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.541+0000 7f34537fe640 1 -- 172.21.15.103:0/1865620643 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3458017030 con 0x7f3464101350 2024-09-19T06:49:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.541+0000 7f34537fe640 1 -- 172.21.15.103:0/1865620643 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3458004020 con 0x7f3464101350 2024-09-19T06:49:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.541+0000 7f34537fe640 1 -- 172.21.15.103:0/1865620643 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34580047a0 con 0x7f3464101350 2024-09-19T06:49:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.541+0000 7f3468991640 1 -- 172.21.15.103:0/1865620643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34641a4900 con 0x7f3464101350 2024-09-19T06:49:24.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.541+0000 7f3468991640 1 -- 172.21.15.103:0/1865620643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34641a4e20 con 0x7f3464101350 2024-09-19T06:49:24.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.542+0000 7f3468991640 1 -- 172.21.15.103:0/1865620643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3464100e10 con 0x7f3464101350 2024-09-19T06:49:24.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.544+0000 7f34537fe640 1 -- 172.21.15.103:0/1865620643 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f34580041d0 con 0x7f3464101350 2024-09-19T06:49:24.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.544+0000 7f34537fe640 1 --2- 172.21.15.103:0/1865620643 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f344c0a3580 0x7f344c0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.545+0000 7f346259c640 1 --2- 172.21.15.103:0/1865620643 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f344c0a3580 0x7f344c0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.545+0000 7f34537fe640 1 -- 172.21.15.103:0/1865620643 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f34580d00d0 con 0x7f3464101350 2024-09-19T06:49:24.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.545+0000 7f346259c640 1 --2- 172.21.15.103:0/1865620643 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f344c0a3580 0x7f344c0a5a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3454005d70 tx=0x7f3454005d00 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:24.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.548+0000 7f34537fe640 1 -- 172.21.15.103:0/1865620643 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f345809c870 con 0x7f3464101350 2024-09-19T06:49:24.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.742+0000 7f3468991640 1 -- 172.21.15.103:0/1865620643 --> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f3464103d90 con 0x7f344c0a3580 2024-09-19T06:49:24.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.749+0000 7f34537fe640 1 -- 172.21.15.103:0/1865620643 <== mgr.15210 v2:172.21.15.103:6800/82882809 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f3464103d90 con 0x7f344c0a3580 2024-09-19T06:49:24.750 INFO:teuthology.orchestra.run.smithi103.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T06:49:23", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-19T06:49:24.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.752+0000 7f3468991640 1 -- 172.21.15.103:0/1865620643 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f344c0a3580 msgr2=0x7f344c0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.752+0000 7f3468991640 1 --2- 172.21.15.103:0/1865620643 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f344c0a3580 0x7f344c0a5a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3454005d70 tx=0x7f3454005d00 comp rx=0 tx=0).stop 2024-09-19T06:49:24.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.752+0000 7f3468991640 1 -- 172.21.15.103:0/1865620643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3464101350 msgr2=0x7f34641996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.752+0000 7f3468991640 1 --2- 172.21.15.103:0/1865620643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3464101350 0x7f34641996b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f345800d950 tx=0x7f345800de10 comp rx=0 tx=0).stop 2024-09-19T06:49:24.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.752+0000 7f3468991640 1 -- 172.21.15.103:0/1865620643 shutdown_connections 2024-09-19T06:49:24.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.752+0000 7f3468991640 1 --2- 172.21.15.103:0/1865620643 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f344c0a3580 0x7f344c0a5a40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.752+0000 7f3468991640 1 --2- 172.21.15.103:0/1865620643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34641069c0 0x7f34641a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.752+0000 7f3468991640 1 --2- 172.21.15.103:0/1865620643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3464101350 0x7f34641996b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.752+0000 7f3468991640 1 --2- 172.21.15.103:0/1865620643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34641009b0 0x7f3464199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.752+0000 7f3468991640 1 -- 172.21.15.103:0/1865620643 >> 172.21.15.103:0/1865620643 conn(0x7f3464077990 msgr2=0x7f3464103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:24.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.752+0000 7f3468991640 1 -- 172.21.15.103:0/1865620643 shutdown_connections 2024-09-19T06:49:24.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.752+0000 7f3468991640 1 -- 172.21.15.103:0/1865620643 wait complete. 2024-09-19T06:49:24.764 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T06:49:24.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.918+0000 7f038a54b640 1 -- 172.21.15.103:0/2510893238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03841069c0 msgr2=0x7f0384106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.918+0000 7f038a54b640 1 --2- 172.21.15.103:0/2510893238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03841069c0 0x7f0384106da0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0374009930 tx=0x7f037402a420 comp rx=0 tx=0).stop 2024-09-19T06:49:24.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.918+0000 7f038a54b640 1 -- 172.21.15.103:0/2510893238 shutdown_connections 2024-09-19T06:49:24.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.918+0000 7f038a54b640 1 --2- 172.21.15.103:0/2510893238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0384101350 0x7f038410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.918+0000 7f038a54b640 1 --2- 172.21.15.103:0/2510893238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03841009b0 0x7f0384100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.918+0000 7f038a54b640 1 --2- 172.21.15.103:0/2510893238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03841069c0 0x7f0384106da0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.918+0000 7f038a54b640 1 -- 172.21.15.103:0/2510893238 >> 172.21.15.103:0/2510893238 conn(0x7f0384077990 msgr2=0x7f03840fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:24.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.918+0000 7f038a54b640 1 -- 172.21.15.103:0/2510893238 shutdown_connections 2024-09-19T06:49:24.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.918+0000 7f038a54b640 1 -- 172.21.15.103:0/2510893238 wait complete. 2024-09-19T06:49:24.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f038a54b640 1 Processor -- start 2024-09-19T06:49:24.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f038a54b640 1 -- start start 2024-09-19T06:49:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f038a54b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03841009b0 0x7f0384197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f038a54b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0384101350 0x7f03841977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f038a54b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03841069c0 0x7f038419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f038a54b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0384110dc0 con 0x7f03841009b0 2024-09-19T06:49:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f038a54b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0384110c40 con 0x7f03841069c0 2024-09-19T06:49:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f038a54b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0384110f40 con 0x7f0384101350 2024-09-19T06:49:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f0383fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03841009b0 0x7f0384197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f0383fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03841009b0 0x7f0384197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42588/0 (socket says 172.21.15.103:42588) 2024-09-19T06:49:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f0383fff640 1 -- 172.21.15.103:0/3880681181 learned_addr learned my addr 172.21.15.103:0/3880681181 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f0388ae8640 1 --2- 172.21.15.103:0/3880681181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03841069c0 0x7f038419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f0383fff640 1 -- 172.21.15.103:0/3880681181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0384101350 msgr2=0x7f03841977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f0383fff640 1 --2- 172.21.15.103:0/3880681181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0384101350 0x7f03841977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f0383fff640 1 -- 172.21.15.103:0/3880681181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03841069c0 msgr2=0x7f038419bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f0383fff640 1 --2- 172.21.15.103:0/3880681181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03841069c0 0x7f038419bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f0383fff640 1 -- 172.21.15.103:0/3880681181 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038419c250 con 0x7f03841009b0 2024-09-19T06:49:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.919+0000 7f0388ae8640 1 --2- 172.21.15.103:0/3880681181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03841069c0 0x7f038419bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.920+0000 7f0383fff640 1 --2- 172.21.15.103:0/3880681181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03841009b0 0x7f0384197260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0374009230 tx=0x7f0374004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.920+0000 7f03817fa640 1 -- 172.21.15.103:0/3880681181 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037403f030 con 0x7f03841009b0 2024-09-19T06:49:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.920+0000 7f03817fa640 1 -- 172.21.15.103:0/3880681181 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0374003ad0 con 0x7f03841009b0 2024-09-19T06:49:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.920+0000 7f038a54b640 1 -- 172.21.15.103:0/3880681181 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f038419c4e0 con 0x7f03841009b0 2024-09-19T06:49:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.920+0000 7f03817fa640 1 -- 172.21.15.103:0/3880681181 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0374035070 con 0x7f03841009b0 2024-09-19T06:49:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.920+0000 7f038a54b640 1 -- 172.21.15.103:0/3880681181 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03841a3dd0 con 0x7f03841009b0 2024-09-19T06:49:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.921+0000 7f038a54b640 1 -- 172.21.15.103:0/3880681181 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0384100e10 con 0x7f03841009b0 2024-09-19T06:49:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.922+0000 7f03817fa640 1 -- 172.21.15.103:0/3880681181 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f0374042030 con 0x7f03841009b0 2024-09-19T06:49:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.923+0000 7f03817fa640 1 --2- 172.21.15.103:0/3880681181 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03700a3580 0x7f03700a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.923+0000 7f03817fa640 1 -- 172.21.15.103:0/3880681181 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f03740f04a0 con 0x7f03841009b0 2024-09-19T06:49:24.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.923+0000 7f03837fe640 1 --2- 172.21.15.103:0/3880681181 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03700a3580 0x7f03700a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.923+0000 7f03837fe640 1 --2- 172.21.15.103:0/3880681181 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03700a3580 0x7f03700a5a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f036c005fd0 tx=0x7f036c005850 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:24.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.927+0000 7f03817fa640 1 -- 172.21.15.103:0/3880681181 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f03740bcd10 con 0x7f03841009b0 2024-09-19T06:49:24.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.959+0000 7f50feccc640 1 -- 172.21.15.103:0/2076035935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50f8074510 msgr2=0x7f50f80748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.959+0000 7f50feccc640 1 --2- 172.21.15.103:0/2076035935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50f8074510 0x7f50f80748f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009900 tx=0x7f50ec02a480 comp rx=0 tx=0).stop 2024-09-19T06:49:24.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.959+0000 7f50feccc640 1 -- 172.21.15.103:0/2076035935 shutdown_connections 2024-09-19T06:49:24.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.959+0000 7f50feccc640 1 --2- 172.21.15.103:0/2076035935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f810e450 0x7f50f8110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.959+0000 7f50feccc640 1 --2- 172.21.15.103:0/2076035935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50f8074e30 0x7f50f810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.959+0000 7f50feccc640 1 --2- 172.21.15.103:0/2076035935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50f8074510 0x7f50f80748f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.959+0000 7f50feccc640 1 -- 172.21.15.103:0/2076035935 >> 172.21.15.103:0/2076035935 conn(0x7f50f80fd8f0 msgr2=0x7f50f80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:24.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.959+0000 7f50feccc640 1 -- 172.21.15.103:0/2076035935 shutdown_connections 2024-09-19T06:49:24.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.959+0000 7f50feccc640 1 -- 172.21.15.103:0/2076035935 wait complete. 2024-09-19T06:49:24.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50feccc640 1 Processor -- start 2024-09-19T06:49:24.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50feccc640 1 -- start start 2024-09-19T06:49:24.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50feccc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50f8074510 0x7f50f819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50feccc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f8074e30 0x7f50f819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50feccc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50f810e450 0x7f50f819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50feccc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50f8114160 con 0x7f50f8074510 2024-09-19T06:49:24.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50feccc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f50f8113fe0 con 0x7f50f810e450 2024-09-19T06:49:24.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50feccc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81142e0 con 0x7f50f8074e30 2024-09-19T06:49:24.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50fca68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50f8074510 0x7f50f819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50fca68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50f8074510 0x7f50f819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42596/0 (socket says 172.21.15.103:42596) 2024-09-19T06:49:24.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50fca68640 1 -- 172.21.15.103:0/4290863112 learned_addr learned my addr 172.21.15.103:0/4290863112 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:24.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50fd269640 1 --2- 172.21.15.103:0/4290863112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50f810e450 0x7f50f819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50f7fff640 1 --2- 172.21.15.103:0/4290863112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f8074e30 0x7f50f819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50fca68640 1 -- 172.21.15.103:0/4290863112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f8074e30 msgr2=0x7f50f819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50fca68640 1 --2- 172.21.15.103:0/4290863112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f8074e30 0x7f50f819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50fca68640 1 -- 172.21.15.103:0/4290863112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50f810e450 msgr2=0x7f50f819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50fca68640 1 --2- 172.21.15.103:0/4290863112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50f810e450 0x7f50f819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.960+0000 7f50fca68640 1 -- 172.21.15.103:0/4290863112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f819f5d0 con 0x7f50f8074510 2024-09-19T06:49:24.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.961+0000 7f50fca68640 1 --2- 172.21.15.103:0/4290863112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50f8074510 0x7f50f819a620 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f50ec006010 tx=0x7f50ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:24.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.961+0000 7f50f5ffb640 1 -- 172.21.15.103:0/4290863112 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec03f030 con 0x7f50f8074510 2024-09-19T06:49:24.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.961+0000 7f50feccc640 1 -- 172.21.15.103:0/4290863112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f819f860 con 0x7f50f8074510 2024-09-19T06:49:24.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.961+0000 7f50f5ffb640 1 -- 172.21.15.103:0/4290863112 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50ec003ce0 con 0x7f50f8074510 2024-09-19T06:49:24.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.961+0000 7f50f5ffb640 1 -- 172.21.15.103:0/4290863112 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec035070 con 0x7f50f8074510 2024-09-19T06:49:24.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.961+0000 7f50feccc640 1 -- 172.21.15.103:0/4290863112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f81a71a0 con 0x7f50f8074510 2024-09-19T06:49:24.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.963+0000 7f50f5ffb640 1 -- 172.21.15.103:0/4290863112 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f50ec042030 con 0x7f50f8074510 2024-09-19T06:49:24.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.963+0000 7f51010a5740 1 Processor -- start 2024-09-19T06:49:24.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f51010a5740 1 -- start start 2024-09-19T06:49:24.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f51010a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f364a0 0x558439f36880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f51010a5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f36dc0 0x558439f41140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f51010a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f41680 0x558439f43a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f51010a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558439e9c2e0 con 0x558439f364a0 2024-09-19T06:49:24.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f51010a5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558439e43ce0 con 0x558439f36dc0 2024-09-19T06:49:24.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f51010a5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558439e982a0 con 0x558439f41680 2024-09-19T06:49:24.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f50f5ffb640 1 --2- 172.21.15.103:0/4290863112 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f50e80a34d0 0x7f50e80a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f50fd269640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f364a0 0x558439f36880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f50fd269640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f364a0 0x558439f36880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42610/0 (socket says 172.21.15.103:42610) 2024-09-19T06:49:24.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f50fd269640 1 -- 172.21.15.103:0/3105697398 learned_addr learned my addr 172.21.15.103:0/3105697398 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:24.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f50f5ffb640 1 -- 172.21.15.103:0/4290863112 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f50ec0f0490 con 0x7f50f8074510 2024-09-19T06:49:24.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f50fd269640 1 --2- 172.21.15.103:0/4290863112 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f50e80a34d0 0x7f50e80a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f50fca68640 1 --2- 172.21.15.103:0/3105697398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f36dc0 0x558439f41140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f50f7fff640 1 --2- 172.21.15.103:0/3105697398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f41680 0x558439f43a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f50fd269640 1 -- 172.21.15.103:0/3105697398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f41680 msgr2=0x558439f43a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:24.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f50fd269640 1 --2- 172.21.15.103:0/3105697398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f41680 0x558439f43a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f50fd269640 1 -- 172.21.15.103:0/3105697398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f36dc0 msgr2=0x558439f41140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:24.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.964+0000 7f50fd269640 1 --2- 172.21.15.103:0/3105697398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f36dc0 0x558439f41140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.965+0000 7f50fd269640 1 -- 172.21.15.103:0/3105697398 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558439e28440 con 0x558439f364a0 2024-09-19T06:49:24.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.965+0000 7f50fd269640 1 --2- 172.21.15.103:0/3105697398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f364a0 0x558439f36880 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f50f0014e30 tx=0x7f50f00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e862d38dbef9b37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:24.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.965+0000 7f50fd269640 1 --2- 172.21.15.103:0/4290863112 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f50e80a34d0 0x7f50e80a5990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f50f0011cb0 tx=0x7f50f001c000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:24.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.965+0000 7f50e2ffd640 1 -- 172.21.15.103:0/3105697398 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0022070 con 0x558439f364a0 2024-09-19T06:49:24.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.965+0000 7f50e2ffd640 1 -- 172.21.15.103:0/3105697398 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f0007be0 con 0x558439f364a0 2024-09-19T06:49:24.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.965+0000 7f50e2ffd640 1 -- 172.21.15.103:0/3105697398 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0004280 con 0x558439f364a0 2024-09-19T06:49:24.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.966+0000 7f51010a5740 1 -- 172.21.15.103:0/3105697398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f364a0 msgr2=0x558439f36880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.966+0000 7f51010a5740 1 --2- 172.21.15.103:0/3105697398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f364a0 0x558439f36880 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f50f0014e30 tx=0x7f50f00117b0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.966+0000 7f51010a5740 1 -- 172.21.15.103:0/3105697398 shutdown_connections 2024-09-19T06:49:24.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.966+0000 7f51010a5740 1 --2- 172.21.15.103:0/3105697398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f41680 0x558439f43a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.966+0000 7f51010a5740 1 --2- 172.21.15.103:0/3105697398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f36dc0 0x558439f41140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.966+0000 7f51010a5740 1 --2- 172.21.15.103:0/3105697398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f364a0 0x558439f36880 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.966+0000 7f51010a5740 1 -- 172.21.15.103:0/3105697398 >> 172.21.15.103:0/3105697398 conn(0x558439f32a30 msgr2=0x558439f32e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:24.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.966+0000 7f51010a5740 1 -- 172.21.15.103:0/3105697398 shutdown_connections 2024-09-19T06:49:24.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.966+0000 7f51010a5740 1 -- 172.21.15.103:0/3105697398 wait complete. 2024-09-19T06:49:24.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.966+0000 7f51010a5740 1 -- start start 2024-09-19T06:49:24.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.966+0000 7f51010a5740 1 Processor -- start 2024-09-19T06:49:24.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f51010a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f364a0 0x558439f41010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f51010a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f36dc0 0x558439f3e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f51010a5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f41680 0x558439f3e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f51010a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558439f44250 con 0x558439f36dc0 2024-09-19T06:49:24.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f51010a5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558439e57050 con 0x558439f41680 2024-09-19T06:49:24.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f51010a5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558439f443d0 con 0x558439f364a0 2024-09-19T06:49:24.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f51010a5740 10 client.0 _ll_register_callbacks cb 0x558439e97160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:24.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f50f7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f36dc0 0x558439f3e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f50f7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f36dc0 0x558439f3e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42614/0 (socket says 172.21.15.103:42614) 2024-09-19T06:49:24.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f50f7fff640 1 -- 172.21.15.103:0/3746087232 learned_addr learned my addr 172.21.15.103:0/3746087232 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:24.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f50fd269640 1 --2- 172.21.15.103:0/3746087232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f41680 0x558439f3e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f50fca68640 1 --2- 172.21.15.103:0/3746087232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f364a0 0x558439f41010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f50f7fff640 1 -- 172.21.15.103:0/3746087232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f364a0 msgr2=0x558439f41010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:24.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f50f7fff640 1 --2- 172.21.15.103:0/3746087232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f364a0 0x558439f41010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f50f7fff640 1 -- 172.21.15.103:0/3746087232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f41680 msgr2=0x558439f3e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:24.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f50f7fff640 1 --2- 172.21.15.103:0/3746087232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f41680 0x558439f3e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f50f7fff640 1 -- 172.21.15.103:0/3746087232 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558439c0d3c0 con 0x558439f36dc0 2024-09-19T06:49:24.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.967+0000 7f50fca68640 1 --2- 172.21.15.103:0/3746087232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f364a0 0x558439f41010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:24.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.968+0000 7f50f7fff640 1 --2- 172.21.15.103:0/3746087232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f36dc0 0x558439f3e2c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009d40 tx=0x7f50e40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593af6b8f74d5e9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:24.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.968+0000 7f50d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:24.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.968+0000 7f50d7fff640 1 -- 172.21.15.103:0/3746087232 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4012860 con 0x558439f36dc0 2024-09-19T06:49:24.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.968+0000 7f50d7fff640 1 -- 172.21.15.103:0/3746087232 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f50e4012a00 con 0x558439f36dc0 2024-09-19T06:49:24.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.968+0000 7f51010a5740 1 -- 172.21.15.103:0/3746087232 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50f0007be0 con 0x558439f36dc0 2024-09-19T06:49:24.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.968+0000 7f50d7fff640 1 -- 172.21.15.103:0/3746087232 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f50e4009e30 con 0x558439f36dc0 2024-09-19T06:49:24.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.968+0000 7f51010a5740 10 client.15408 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:24.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.968+0000 7f51010a5740 1 -- 172.21.15.103:0/3746087232 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558439f4fca0 con 0x558439f36dc0 2024-09-19T06:49:24.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.968+0000 7f50d7fff640 1 -- 172.21.15.103:0/3746087232 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f0007be0 con 0x558439f36dc0 2024-09-19T06:49:24.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.968+0000 7f50d7fff640 1 -- 172.21.15.103:0/3746087232 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f50e400f040 con 0x558439f36dc0 2024-09-19T06:49:24.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.968+0000 7f51010a5740 10 client.15408 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:24.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.968+0000 7f51010a5740 10 client.15408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:24.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.968+0000 7f51010a5740 20 client.15408 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:24.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.968+0000 7f51010a5740 1 --2- 172.21.15.103:0/3746087232 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558439f51200 0x558439f53620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.968+0000 7f51010a5740 4 client.15408 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:24.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.968+0000 7f51010a5740 1 -- 172.21.15.103:0/3746087232 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558439ea9850 con 0x558439f51200 2024-09-19T06:49:24.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.969+0000 7f50fca68640 1 --2- 172.21.15.103:0/3746087232 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558439f51200 0x558439f53620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.969+0000 7f50fca68640 1 --2- 172.21.15.103:0/3746087232 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558439f51200 0x558439f53620 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c85b8e0bc42a4fd4 server_cookie=4e08b62c03be835c in_seq=0 out_seq=0 2024-09-19T06:49:24.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.969+0000 7f50d7fff640 10 client.15408 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:24.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.970+0000 7f50d7fff640 1 -- 172.21.15.103:0/3746087232 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558439ea9850 con 0x558439f51200 2024-09-19T06:49:24.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.970+0000 7f50d7fff640 10 client.15408 handle_command_reply: tid=0 2024-09-19T06:49:24.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.970+0000 7f51010a5740 1 client.15408 shutdown 2024-09-19T06:49:24.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.970+0000 7f51010a5740 1 -- 172.21.15.103:0/3746087232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f36dc0 msgr2=0x558439f3e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.970+0000 7f51010a5740 1 --2- 172.21.15.103:0/3746087232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f36dc0 0x558439f3e2c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009d40 tx=0x7f50e40117b0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.970+0000 7f51010a5740 1 -- 172.21.15.103:0/3746087232 shutdown_connections 2024-09-19T06:49:24.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.970+0000 7f51010a5740 1 --2- 172.21.15.103:0/3746087232 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558439f51200 0x558439f53620 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.970+0000 7f51010a5740 1 --2- 172.21.15.103:0/3746087232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f41680 0x558439f3e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.970+0000 7f51010a5740 1 --2- 172.21.15.103:0/3746087232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f36dc0 0x558439f3e2c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.970+0000 7f51010a5740 1 --2- 172.21.15.103:0/3746087232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f364a0 0x558439f41010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.970+0000 7f51010a5740 1 -- 172.21.15.103:0/3746087232 >> 172.21.15.103:0/3746087232 conn(0x558439f32a30 msgr2=0x558439f31b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:24.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.970+0000 7f50d7fff640 0 client.15408 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:24.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.970+0000 7f51010a5740 1 -- 172.21.15.103:0/3746087232 shutdown_connections 2024-09-19T06:49:24.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.970+0000 7f51010a5740 1 -- 172.21.15.103:0/3746087232 wait complete. 2024-09-19T06:49:24.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.970+0000 7f51010a5740 20 client.15408 trim_cache size 0 max 16384 2024-09-19T06:49:24.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.987+0000 7f51010a5740 1 Processor -- start 2024-09-19T06:49:24.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.987+0000 7f51010a5740 1 -- start start 2024-09-19T06:49:24.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.987+0000 7f51010a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f364a0 0x558439f4d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.987+0000 7f51010a5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f36dc0 0x558439f58ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.987+0000 7f51010a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f41680 0x558439f59430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.987+0000 7f51010a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81afa50 con 0x558439f41680 2024-09-19T06:49:24.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.987+0000 7f51010a5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81b1590 con 0x558439f36dc0 2024-09-19T06:49:24.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.987+0000 7f51010a5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81b0120 con 0x558439f364a0 2024-09-19T06:49:24.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.987+0000 7f50fd269640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f41680 0x558439f59430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.988+0000 7f50fd269640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f41680 0x558439f59430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42624/0 (socket says 172.21.15.103:42624) 2024-09-19T06:49:24.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.988+0000 7f50fd269640 1 -- 172.21.15.103:0/3948785030 learned_addr learned my addr 172.21.15.103:0/3948785030 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:24.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.988+0000 7f50f7fff640 1 --2- 172.21.15.103:0/3948785030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f36dc0 0x558439f58ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.988+0000 7f50fca68640 1 --2- 172.21.15.103:0/3948785030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f364a0 0x558439f4d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.988+0000 7f50fd269640 1 -- 172.21.15.103:0/3948785030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f364a0 msgr2=0x558439f4d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:24.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.988+0000 7f50fd269640 1 --2- 172.21.15.103:0/3948785030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f364a0 0x558439f4d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.988+0000 7f50fd269640 1 -- 172.21.15.103:0/3948785030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f36dc0 msgr2=0x558439f58ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:24.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.988+0000 7f50fd269640 1 --2- 172.21.15.103:0/3948785030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f36dc0 0x558439f58ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.988+0000 7f50fd269640 1 -- 172.21.15.103:0/3948785030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558439f3c810 con 0x558439f41680 2024-09-19T06:49:24.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.988+0000 7f50f7fff640 1 --2- 172.21.15.103:0/3948785030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f36dc0 0x558439f58ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:24.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.988+0000 7f50fd269640 1 --2- 172.21.15.103:0/3948785030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f41680 0x558439f59430 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f50f0014e30 tx=0x7f50f0007540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e24e323f703657 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:24.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.988+0000 7f50e27fc640 1 -- 172.21.15.103:0/3948785030 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0022040 con 0x558439f41680 2024-09-19T06:49:24.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.988+0000 7f50e27fc640 1 -- 172.21.15.103:0/3948785030 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f0014990 con 0x558439f41680 2024-09-19T06:49:24.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.988+0000 7f50e27fc640 1 -- 172.21.15.103:0/3948785030 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0005880 con 0x558439f41680 2024-09-19T06:49:24.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.989+0000 7f51010a5740 1 -- 172.21.15.103:0/3948785030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f41680 msgr2=0x558439f59430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.989+0000 7f51010a5740 1 --2- 172.21.15.103:0/3948785030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f41680 0x558439f59430 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f50f0014e30 tx=0x7f50f0007540 comp rx=0 tx=0).stop 2024-09-19T06:49:24.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.989+0000 7f51010a5740 1 -- 172.21.15.103:0/3948785030 shutdown_connections 2024-09-19T06:49:24.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.989+0000 7f51010a5740 1 --2- 172.21.15.103:0/3948785030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f41680 0x558439f59430 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.989+0000 7f51010a5740 1 --2- 172.21.15.103:0/3948785030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f36dc0 0x558439f58ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.989+0000 7f51010a5740 1 --2- 172.21.15.103:0/3948785030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f364a0 0x558439f4d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.989+0000 7f51010a5740 1 -- 172.21.15.103:0/3948785030 >> 172.21.15.103:0/3948785030 conn(0x558439f32a30 msgr2=0x558439f3d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:24.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.989+0000 7f51010a5740 1 -- 172.21.15.103:0/3948785030 shutdown_connections 2024-09-19T06:49:24.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.989+0000 7f51010a5740 1 -- 172.21.15.103:0/3948785030 wait complete. 2024-09-19T06:49:24.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.989+0000 7f51010a5740 1 -- start start 2024-09-19T06:49:24.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.989+0000 7f51010a5740 1 Processor -- start 2024-09-19T06:49:24.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f51010a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f364a0 0x558439f49cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f51010a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f36dc0 0x558439f45000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f51010a5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f41680 0x558439f45540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f51010a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81b0ec0 con 0x558439f364a0 2024-09-19T06:49:24.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f51010a5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81b07f0 con 0x558439f41680 2024-09-19T06:49:24.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f51010a5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81ada10 con 0x558439f36dc0 2024-09-19T06:49:24.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f51010a5740 10 client.0 _ll_register_callbacks cb 0x7f50e4009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:24.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f50fca68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f364a0 0x558439f49cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f50fca68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f364a0 0x558439f49cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42636/0 (socket says 172.21.15.103:42636) 2024-09-19T06:49:24.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f50fca68640 1 -- 172.21.15.103:0/3447265267 learned_addr learned my addr 172.21.15.103:0/3447265267 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:24.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f50f7fff640 1 --2- 172.21.15.103:0/3447265267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f36dc0 0x558439f45000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f50fd269640 1 --2- 172.21.15.103:0/3447265267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f41680 0x558439f45540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f50fca68640 1 -- 172.21.15.103:0/3447265267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f36dc0 msgr2=0x558439f45000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:24.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f50fca68640 1 --2- 172.21.15.103:0/3447265267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f36dc0 0x558439f45000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f50fca68640 1 -- 172.21.15.103:0/3447265267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f41680 msgr2=0x558439f45540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:24.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f50fca68640 1 --2- 172.21.15.103:0/3447265267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f41680 0x558439f45540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f50fca68640 1 -- 172.21.15.103:0/3447265267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558439f45d40 con 0x558439f364a0 2024-09-19T06:49:24.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f50fd269640 1 --2- 172.21.15.103:0/3447265267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f41680 0x558439f45540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:24.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f50fca68640 1 --2- 172.21.15.103:0/3447265267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f364a0 0x558439f49cd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0edad0 tx=0x7f50ec005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc2cbb0de8dd6b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:24.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.990+0000 7f50d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:24.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.991+0000 7f50d7fff640 1 -- 172.21.15.103:0/3447265267 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec0a5070 con 0x558439f364a0 2024-09-19T06:49:24.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.991+0000 7f50d7fff640 1 -- 172.21.15.103:0/3447265267 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f50ec0b8c70 con 0x558439f364a0 2024-09-19T06:49:24.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.991+0000 7f51010a5740 1 -- 172.21.15.103:0/3447265267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50f0014990 con 0x558439f364a0 2024-09-19T06:49:24.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.991+0000 7f50d7fff640 1 -- 172.21.15.103:0/3447265267 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f50ec049040 con 0x558439f364a0 2024-09-19T06:49:24.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.991+0000 7f51010a5740 10 client.15414 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:24.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.991+0000 7f51010a5740 1 -- 172.21.15.103:0/3447265267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558439fbbdb0 con 0x558439f364a0 2024-09-19T06:49:24.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.991+0000 7f50d7fff640 1 -- 172.21.15.103:0/3447265267 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50ec03b580 con 0x558439f364a0 2024-09-19T06:49:24.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.991+0000 7f50d7fff640 1 -- 172.21.15.103:0/3447265267 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f50ec0c4de0 con 0x558439f364a0 2024-09-19T06:49:24.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.991+0000 7f51010a5740 10 client.15414 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:24.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.991+0000 7f51010a5740 10 client.15414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:24.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.991+0000 7f51010a5740 20 client.15414 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:24.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.991+0000 7f51010a5740 1 --2- 172.21.15.103:0/3447265267 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558439fbc4d0 0x558439fbe8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:24.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.991+0000 7f51010a5740 4 client.15414 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:49:24.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.991+0000 7f51010a5740 1 -- 172.21.15.103:0/3447265267 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f50f0014990 con 0x558439fbc4d0 2024-09-19T06:49:24.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.992+0000 7f50f7fff640 1 --2- 172.21.15.103:0/3447265267 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558439fbc4d0 0x558439fbe8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:24.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.992+0000 7f50d7fff640 10 client.15414 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:24.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.992+0000 7f50f7fff640 1 --2- 172.21.15.103:0/3447265267 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558439fbc4d0 0x558439fbe8c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43da0c58b15a2747 server_cookie=b5a1d76fd2095de0 in_seq=0 out_seq=0 2024-09-19T06:49:24.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.992+0000 7f50d7fff640 1 -- 172.21.15.103:0/3447265267 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f50f0014990 con 0x558439fbc4d0 2024-09-19T06:49:24.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.992+0000 7f50d7fff640 10 client.15414 handle_command_reply: tid=0 2024-09-19T06:49:24.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.992+0000 7f51010a5740 1 client.15414 shutdown 2024-09-19T06:49:24.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.992+0000 7f51010a5740 1 -- 172.21.15.103:0/3447265267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f364a0 msgr2=0x558439f49cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:24.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.992+0000 7f51010a5740 1 --2- 172.21.15.103:0/3447265267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f364a0 0x558439f49cd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0edad0 tx=0x7f50ec005c00 comp rx=0 tx=0).stop 2024-09-19T06:49:24.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.992+0000 7f51010a5740 1 -- 172.21.15.103:0/3447265267 shutdown_connections 2024-09-19T06:49:24.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.992+0000 7f51010a5740 1 --2- 172.21.15.103:0/3447265267 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558439fbc4d0 0x558439fbe8c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.992+0000 7f51010a5740 1 --2- 172.21.15.103:0/3447265267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558439f41680 0x558439f45540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.992+0000 7f51010a5740 1 --2- 172.21.15.103:0/3447265267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558439f36dc0 0x558439f45000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:24.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.992+0000 7f51010a5740 1 --2- 172.21.15.103:0/3447265267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558439f364a0 0x558439f49cd0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:25.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.992+0000 7f50d7fff640 0 client.15414 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:25.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.992+0000 7f51010a5740 1 -- 172.21.15.103:0/3447265267 >> 172.21.15.103:0/3447265267 conn(0x558439f32a30 msgr2=0x558439f31350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:25.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.993+0000 7f51010a5740 1 -- 172.21.15.103:0/3447265267 shutdown_connections 2024-09-19T06:49:25.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.993+0000 7f51010a5740 1 -- 172.21.15.103:0/3447265267 wait complete. 2024-09-19T06:49:25.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.993+0000 7f51010a5740 20 client.15414 trim_cache size 0 max 16384 2024-09-19T06:49:25.000 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:49:25.000 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:49:25.000 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T06:49:25.000 INFO:teuthology.orchestra.run.smithi103.stdout: "3c9bffcb-1f20-4240-95a8-7d1c977ae872": { 2024-09-19T06:49:25.000 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T06:49:25.001 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "3c9bffcb-1f20-4240-95a8-7d1c977ae872", 2024-09-19T06:49:25.001 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T06:49:25.001 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:49:25.001 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:49:25.001 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:49:25.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.993+0000 7f50feccc640 1 -- 172.21.15.103:0/4290863112 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f50e80a34d0 msgr2=0x7f50e80a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:25.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.993+0000 7f50feccc640 1 --2- 172.21.15.103:0/4290863112 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f50e80a34d0 0x7f50e80a5990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f50f0011cb0 tx=0x7f50f001c000 comp rx=0 tx=0).stop 2024-09-19T06:49:25.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.993+0000 7f50feccc640 1 -- 172.21.15.103:0/4290863112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50f8074510 msgr2=0x7f50f819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:25.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.993+0000 7f50feccc640 1 --2- 172.21.15.103:0/4290863112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50f8074510 0x7f50f819a620 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f50ec006010 tx=0x7f50ec0042f0 comp rx=0 tx=0).stop 2024-09-19T06:49:25.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.993+0000 7f50feccc640 1 -- 172.21.15.103:0/4290863112 shutdown_connections 2024-09-19T06:49:25.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.993+0000 7f50feccc640 1 --2- 172.21.15.103:0/4290863112 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f50e80a34d0 0x7f50e80a5990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:25.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.993+0000 7f50feccc640 1 --2- 172.21.15.103:0/4290863112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50f810e450 0x7f50f819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:25.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.993+0000 7f50feccc640 1 --2- 172.21.15.103:0/4290863112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f8074e30 0x7f50f819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:25.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.993+0000 7f50feccc640 1 --2- 172.21.15.103:0/4290863112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50f8074510 0x7f50f819a620 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:25.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.993+0000 7f50feccc640 1 -- 172.21.15.103:0/4290863112 >> 172.21.15.103:0/4290863112 conn(0x7f50f80fd8f0 msgr2=0x7f50f80fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:25.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.994+0000 7f50feccc640 1 -- 172.21.15.103:0/4290863112 shutdown_connections 2024-09-19T06:49:25.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:24.994+0000 7f50feccc640 1 -- 172.21.15.103:0/4290863112 wait complete. 2024-09-19T06:49:25.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:25.130+0000 7f038a54b640 1 -- 172.21.15.103:0/3880681181 --> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f0384108f90 con 0x7f03700a3580 2024-09-19T06:49:25.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:25.137+0000 7f03817fa640 1 -- 172.21.15.103:0/3880681181 <== mgr.15210 v2:172.21.15.103:6800/82882809 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f036c00d9f0 con 0x7f03700a3580 2024-09-19T06:49:25.138 INFO:teuthology.orchestra.run.smithi103.stdout:/ 1m 6m3h 2024-09-19T06:49:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:25.139+0000 7f038a54b640 1 -- 172.21.15.103:0/3880681181 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03700a3580 msgr2=0x7f03700a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:25.139+0000 7f038a54b640 1 --2- 172.21.15.103:0/3880681181 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03700a3580 0x7f03700a5a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f036c005fd0 tx=0x7f036c005850 comp rx=0 tx=0).stop 2024-09-19T06:49:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:25.139+0000 7f038a54b640 1 -- 172.21.15.103:0/3880681181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03841009b0 msgr2=0x7f0384197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:25.139+0000 7f038a54b640 1 --2- 172.21.15.103:0/3880681181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03841009b0 0x7f0384197260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0374009230 tx=0x7f0374004270 comp rx=0 tx=0).stop 2024-09-19T06:49:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:25.139+0000 7f038a54b640 1 -- 172.21.15.103:0/3880681181 shutdown_connections 2024-09-19T06:49:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:25.139+0000 7f038a54b640 1 --2- 172.21.15.103:0/3880681181 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03700a3580 0x7f03700a5a40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:25.139+0000 7f038a54b640 1 --2- 172.21.15.103:0/3880681181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03841069c0 0x7f038419bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:25.139+0000 7f038a54b640 1 --2- 172.21.15.103:0/3880681181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0384101350 0x7f03841977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:25.139+0000 7f038a54b640 1 --2- 172.21.15.103:0/3880681181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03841009b0 0x7f0384197260 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:25.139+0000 7f038a54b640 1 -- 172.21.15.103:0/3880681181 >> 172.21.15.103:0/3880681181 conn(0x7f0384077990 msgr2=0x7f03841038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:25.139+0000 7f038a54b640 1 -- 172.21.15.103:0/3880681181 shutdown_connections 2024-09-19T06:49:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:25.139+0000 7f038a54b640 1 -- 172.21.15.103:0/3880681181 wait complete. 2024-09-19T06:49:25.151 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-19T06:49:25.177 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-19T06:49:25.188 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-19T06:49:25.199 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T06:49:25.200 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-19T06:49:25.220 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-19T06:49:25.233 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T06:49:25.234 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:49:25.668 INFO:tasks.cephfs.filesystem:scrub status for tag:3c9bffcb-1f20-4240-95a8-7d1c977ae872 - {'path': '/', 'tag': '3c9bffcb-1f20-4240-95a8-7d1c977ae872', 'options': 'recursive,force'} 2024-09-19T06:49:25.714 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:25.918 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:25 smithi103 ceph-mon[31158]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:49:25.918 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:25 smithi103 ceph-mon[31158]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:49:25.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:25 smithi134 ceph-mon[35356]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:49:25.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:25 smithi134 ceph-mon[35356]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:49:26.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:25 smithi174 ceph-mon[35637]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:49:26.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:25 smithi174 ceph-mon[35637]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:49:26.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.178+0000 7f71a8e8a640 1 -- 172.21.15.103:0/3652804861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71a4102570 msgr2=0x7f71a41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:26.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.178+0000 7f71a8e8a640 1 --2- 172.21.15.103:0/3652804861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71a4102570 0x7f71a41029b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f718c009900 tx=0x7f718c02a480 comp rx=0 tx=0).stop 2024-09-19T06:49:26.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.178+0000 7f71a8e8a640 1 -- 172.21.15.103:0/3652804861 shutdown_connections 2024-09-19T06:49:26.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.178+0000 7f71a8e8a640 1 --2- 172.21.15.103:0/3652804861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4102ef0 0x7f71a410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:26.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.178+0000 7f71a8e8a640 1 --2- 172.21.15.103:0/3652804861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71a4102570 0x7f71a41029b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:26.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.178+0000 7f71a8e8a640 1 --2- 172.21.15.103:0/3652804861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71a4100e20 0x7f71a4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:26.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.178+0000 7f71a8e8a640 1 -- 172.21.15.103:0/3652804861 >> 172.21.15.103:0/3652804861 conn(0x7f71a40fc7d0 msgr2=0x7f71a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:26.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.178+0000 7f71a8e8a640 1 -- 172.21.15.103:0/3652804861 shutdown_connections 2024-09-19T06:49:26.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.178+0000 7f71a8e8a640 1 -- 172.21.15.103:0/3652804861 wait complete. 2024-09-19T06:49:26.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.179+0000 7f71a8e8a640 1 Processor -- start 2024-09-19T06:49:26.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.179+0000 7f71a8e8a640 1 -- start start 2024-09-19T06:49:26.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.179+0000 7f71a8e8a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4100e20 0x7f71a4197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:26.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.179+0000 7f71a8e8a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71a4102570 0x7f71a4197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:26.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.179+0000 7f71a8e8a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71a4102ef0 0x7f71a419bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:26.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.179+0000 7f71a8e8a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71a4110f80 con 0x7f71a4102570 2024-09-19T06:49:26.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.179+0000 7f71a8e8a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71a4110e00 con 0x7f71a4100e20 2024-09-19T06:49:26.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.179+0000 7f71a8e8a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f71a4111100 con 0x7f71a4102ef0 2024-09-19T06:49:26.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.179+0000 7f7199d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71a4102570 0x7f71a4197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:26.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.179+0000 7f7199d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71a4102570 0x7f71a4197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42664/0 (socket says 172.21.15.103:42664) 2024-09-19T06:49:26.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.179+0000 7f7199d9b640 1 -- 172.21.15.103:0/1288125170 learned_addr learned my addr 172.21.15.103:0/1288125170 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:26.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.180+0000 7f71a2d9d640 1 --2- 172.21.15.103:0/1288125170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71a4102ef0 0x7f71a419bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:26.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.180+0000 7f71a259c640 1 --2- 172.21.15.103:0/1288125170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4100e20 0x7f71a4197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:26.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.180+0000 7f7199d9b640 1 -- 172.21.15.103:0/1288125170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71a4102ef0 msgr2=0x7f71a419bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:26.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.180+0000 7f7199d9b640 1 --2- 172.21.15.103:0/1288125170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71a4102ef0 0x7f71a419bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:26.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.180+0000 7f7199d9b640 1 -- 172.21.15.103:0/1288125170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4100e20 msgr2=0x7f71a4197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:26.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.180+0000 7f7199d9b640 1 --2- 172.21.15.103:0/1288125170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4100e20 0x7f71a4197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:26.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.180+0000 7f7199d9b640 1 -- 172.21.15.103:0/1288125170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a419c3d0 con 0x7f71a4102570 2024-09-19T06:49:26.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.180+0000 7f7199d9b640 1 --2- 172.21.15.103:0/1288125170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71a4102570 0x7f71a4197960 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f718c006010 tx=0x7f718c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:26.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.180+0000 7f719bfff640 1 -- 172.21.15.103:0/1288125170 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c03f070 con 0x7f71a4102570 2024-09-19T06:49:26.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.180+0000 7f71a8e8a640 1 -- 172.21.15.103:0/1288125170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a419c660 con 0x7f71a4102570 2024-09-19T06:49:26.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.180+0000 7f719bfff640 1 -- 172.21.15.103:0/1288125170 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f718c031e30 con 0x7f71a4102570 2024-09-19T06:49:26.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.180+0000 7f719bfff640 1 -- 172.21.15.103:0/1288125170 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c035030 con 0x7f71a4102570 2024-09-19T06:49:26.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.180+0000 7f71a8e8a640 1 -- 172.21.15.103:0/1288125170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a4069f70 con 0x7f71a4102570 2024-09-19T06:49:26.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.181+0000 7f71a8e8a640 1 -- 172.21.15.103:0/1288125170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a41021b0 con 0x7f71a4102570 2024-09-19T06:49:26.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.182+0000 7f719bfff640 1 -- 172.21.15.103:0/1288125170 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f718c042070 con 0x7f71a4102570 2024-09-19T06:49:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.183+0000 7f719bfff640 1 --2- 172.21.15.103:0/1288125170 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f71900a34d0 0x7f71900a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.183+0000 7f71a259c640 1 --2- 172.21.15.103:0/1288125170 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f71900a34d0 0x7f71900a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.183+0000 7f719bfff640 1 -- 172.21.15.103:0/1288125170 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f718c0f0d80 con 0x7f71a4102570 2024-09-19T06:49:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.183+0000 7f71a259c640 1 --2- 172.21.15.103:0/1288125170 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f71900a34d0 0x7f71900a5990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f71940058c0 tx=0x7f719400a250 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:26.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.186+0000 7f719bfff640 1 -- 172.21.15.103:0/1288125170 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f718c0bd030 con 0x7f71a4102570 2024-09-19T06:49:26.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.350+0000 7f71a8e8a640 1 -- 172.21.15.103:0/1288125170 --> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f71a4198780 con 0x7f71900a34d0 2024-09-19T06:49:26.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.356+0000 7f719bfff640 1 -- 172.21.15.103:0/1288125170 <== mgr.15210 v2:172.21.15.103:6800/82882809 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257822 (secure 0 0 0) 0x7f71a4198780 con 0x7f71900a34d0 2024-09-19T06:49:26.359 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:49:26.360 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T06:49:26.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.359+0000 7f71a8e8a640 1 -- 172.21.15.103:0/1288125170 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f71900a34d0 msgr2=0x7f71900a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:26.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.359+0000 7f71a8e8a640 1 --2- 172.21.15.103:0/1288125170 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f71900a34d0 0x7f71900a5990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f71940058c0 tx=0x7f719400a250 comp rx=0 tx=0).stop 2024-09-19T06:49:26.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.359+0000 7f71a8e8a640 1 -- 172.21.15.103:0/1288125170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71a4102570 msgr2=0x7f71a4197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:26.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.359+0000 7f71a8e8a640 1 --2- 172.21.15.103:0/1288125170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71a4102570 0x7f71a4197960 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f718c006010 tx=0x7f718c003c80 comp rx=0 tx=0).stop 2024-09-19T06:49:26.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.359+0000 7f71a8e8a640 1 -- 172.21.15.103:0/1288125170 shutdown_connections 2024-09-19T06:49:26.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.359+0000 7f71a8e8a640 1 --2- 172.21.15.103:0/1288125170 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f71900a34d0 0x7f71900a5990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:26.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.359+0000 7f71a8e8a640 1 --2- 172.21.15.103:0/1288125170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71a4102ef0 0x7f71a419bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:26.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.359+0000 7f71a8e8a640 1 --2- 172.21.15.103:0/1288125170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71a4102570 0x7f71a4197960 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:26.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.359+0000 7f71a8e8a640 1 --2- 172.21.15.103:0/1288125170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4100e20 0x7f71a4197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:26.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.359+0000 7f71a8e8a640 1 -- 172.21.15.103:0/1288125170 >> 172.21.15.103:0/1288125170 conn(0x7f71a40fc7d0 msgr2=0x7f71a4105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:26.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.359+0000 7f71a8e8a640 1 -- 172.21.15.103:0/1288125170 shutdown_connections 2024-09-19T06:49:26.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:26.359+0000 7f71a8e8a640 1 -- 172.21.15.103:0/1288125170 wait complete. 2024-09-19T06:49:26.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:26 smithi134 ceph-mon[35356]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:49:26.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:26 smithi134 ceph-mon[35356]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 54 op/s 2024-09-19T06:49:27.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:26 smithi103 ceph-mon[31158]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:49:27.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:26 smithi103 ceph-mon[31158]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 54 op/s 2024-09-19T06:49:27.033 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-19T06:49:25.934066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1393390,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1710,"num_read_kb":1286,"num_write":711,"num_write_kb":5330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":1181610,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1037,"ondisk_log_size":1037,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356268,"kb_used_data":15084,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124585556,"statfs":{"total":1151940427776,"available":1151575609344,"internally_reserved":0,"allocated":15446016,"data_stored":9537690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201469,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":450,"num_read_kb":491,"num_write":149,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.992567"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960112+0000","last_change":"2024-09-19T06:47:24.853118+0000","last_active":"2024-09-19T06:49:13.960112+0000","last_peered":"2024-09-19T06:49:13.960112+0000","last_clean":"2024-09-19T06:49:13.960112+0000","last_became_active":"2024-09-19T06:47:24.852801+0000","last_became_peered":"2024-09-19T06:47:24.852801+0000","last_unstale":"2024-09-19T06:49:13.960112+0000","last_undegraded":"2024-09-19T06:49:13.960112+0000","last_fullsized":"2024-09-19T06:49:13.960112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:16.729103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.957024+0000","last_change":"2024-09-19T06:47:24.841095+0000","last_active":"2024-09-19T06:49:13.957024+0000","last_peered":"2024-09-19T06:49:13.957024+0000","last_clean":"2024-09-19T06:49:13.957024+0000","last_became_active":"2024-09-19T06:47:24.840774+0000","last_became_peered":"2024-09-19T06:47:24.840774+0000","last_unstale":"2024-09-19T06:49:13.957024+0000","last_undegraded":"2024-09-19T06:49:13.957024+0000","last_fullsized":"2024-09-19T06:49:13.957024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:27.262766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973374+0000","last_change":"2024-09-19T06:47:24.848444+0000","last_active":"2024-09-19T06:49:13.973374+0000","last_peered":"2024-09-19T06:49:13.973374+0000","last_clean":"2024-09-19T06:49:13.973374+0000","last_became_active":"2024-09-19T06:47:24.847354+0000","last_became_peered":"2024-09-19T06:47:24.847354+0000","last_unstale":"2024-09-19T06:49:13.973374+0000","last_undegraded":"2024-09-19T06:49:13.973374+0000","last_fullsized":"2024-09-19T06:49:13.973374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:44.901265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.949178+0000","last_change":"2024-09-19T06:47:24.851915+0000","last_active":"2024-09-19T06:49:13.949178+0000","last_peered":"2024-09-19T06:49:13.949178+0000","last_clean":"2024-09-19T06:49:13.949178+0000","last_became_active":"2024-09-19T06:47:24.851600+0000","last_became_peered":"2024-09-19T06:47:24.851600+0000","last_unstale":"2024-09-19T06:49:13.949178+0000","last_undegraded":"2024-09-19T06:49:13.949178+0000","last_fullsized":"2024-09-19T06:49:13.949178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:52.424288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.950906+0000","last_change":"2024-09-19T06:47:24.843184+0000","last_active":"2024-09-19T06:49:13.950906+0000","last_peered":"2024-09-19T06:49:13.950906+0000","last_clean":"2024-09-19T06:49:13.950906+0000","last_became_active":"2024-09-19T06:47:24.842857+0000","last_became_peered":"2024-09-19T06:47:24.842857+0000","last_unstale":"2024-09-19T06:49:13.950906+0000","last_undegraded":"2024-09-19T06:49:13.950906+0000","last_fullsized":"2024-09-19T06:49:13.950906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:06.608699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969815+0000","last_change":"2024-09-19T06:47:24.846631+0000","last_active":"2024-09-19T06:49:13.969815+0000","last_peered":"2024-09-19T06:49:13.969815+0000","last_clean":"2024-09-19T06:49:13.969815+0000","last_became_active":"2024-09-19T06:47:24.846271+0000","last_became_peered":"2024-09-19T06:47:24.846271+0000","last_unstale":"2024-09-19T06:49:13.969815+0000","last_undegraded":"2024-09-19T06:49:13.969815+0000","last_fullsized":"2024-09-19T06:49:13.969815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.272273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.967306+0000","last_change":"2024-09-19T06:47:24.840392+0000","last_active":"2024-09-19T06:49:13.967306+0000","last_peered":"2024-09-19T06:49:13.967306+0000","last_clean":"2024-09-19T06:49:13.967306+0000","last_became_active":"2024-09-19T06:47:24.840092+0000","last_became_peered":"2024-09-19T06:47:24.840092+0000","last_unstale":"2024-09-19T06:49:13.967306+0000","last_undegraded":"2024-09-19T06:49:13.967306+0000","last_fullsized":"2024-09-19T06:49:13.967306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:25.579096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.967567+0000","last_change":"2024-09-19T06:47:24.843336+0000","last_active":"2024-09-19T06:49:13.967567+0000","last_peered":"2024-09-19T06:49:13.967567+0000","last_clean":"2024-09-19T06:49:13.967567+0000","last_became_active":"2024-09-19T06:47:24.842016+0000","last_became_peered":"2024-09-19T06:47:24.842016+0000","last_unstale":"2024-09-19T06:49:13.967567+0000","last_undegraded":"2024-09-19T06:49:13.967567+0000","last_fullsized":"2024-09-19T06:49:13.967567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:49.478359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.968002+0000","last_change":"2024-09-19T06:47:24.847841+0000","last_active":"2024-09-19T06:49:13.968002+0000","last_peered":"2024-09-19T06:49:13.968002+0000","last_clean":"2024-09-19T06:49:13.968002+0000","last_became_active":"2024-09-19T06:47:24.847492+0000","last_became_peered":"2024-09-19T06:47:24.847492+0000","last_unstale":"2024-09-19T06:49:13.968002+0000","last_undegraded":"2024-09-19T06:49:13.968002+0000","last_fullsized":"2024-09-19T06:49:13.968002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:59.229022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955381+0000","last_change":"2024-09-19T06:47:24.843773+0000","last_active":"2024-09-19T06:49:13.955381+0000","last_peered":"2024-09-19T06:49:13.955381+0000","last_clean":"2024-09-19T06:49:13.955381+0000","last_became_active":"2024-09-19T06:47:24.843409+0000","last_became_peered":"2024-09-19T06:47:24.843409+0000","last_unstale":"2024-09-19T06:49:13.955381+0000","last_undegraded":"2024-09-19T06:49:13.955381+0000","last_fullsized":"2024-09-19T06:49:13.955381+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:22.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969613+0000","last_change":"2024-09-19T06:47:24.846637+0000","last_active":"2024-09-19T06:49:13.969613+0000","last_peered":"2024-09-19T06:49:13.969613+0000","last_clean":"2024-09-19T06:49:13.969613+0000","last_became_active":"2024-09-19T06:47:24.846337+0000","last_became_peered":"2024-09-19T06:47:24.846337+0000","last_unstale":"2024-09-19T06:49:13.969613+0000","last_undegraded":"2024-09-19T06:49:13.969613+0000","last_fullsized":"2024-09-19T06:49:13.969613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:23:35.869485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.946579+0000","last_change":"2024-09-19T06:47:24.851399+0000","last_active":"2024-09-19T06:49:13.946579+0000","last_peered":"2024-09-19T06:49:13.946579+0000","last_clean":"2024-09-19T06:49:13.946579+0000","last_became_active":"2024-09-19T06:47:24.851090+0000","last_became_peered":"2024-09-19T06:47:24.851090+0000","last_unstale":"2024-09-19T06:49:13.946579+0000","last_undegraded":"2024-09-19T06:49:13.946579+0000","last_fullsized":"2024-09-19T06:49:13.946579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:52.839956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970074+0000","last_change":"2024-09-19T06:47:24.833858+0000","last_active":"2024-09-19T06:49:13.970074+0000","last_peered":"2024-09-19T06:49:13.970074+0000","last_clean":"2024-09-19T06:49:13.970074+0000","last_became_active":"2024-09-19T06:47:24.833616+0000","last_became_peered":"2024-09-19T06:47:24.833616+0000","last_unstale":"2024-09-19T06:49:13.970074+0000","last_undegraded":"2024-09-19T06:49:13.970074+0000","last_fullsized":"2024-09-19T06:49:13.970074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:54.918144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966731+0000","last_change":"2024-09-19T06:47:24.841030+0000","last_active":"2024-09-19T06:49:13.966731+0000","last_peered":"2024-09-19T06:49:13.966731+0000","last_clean":"2024-09-19T06:49:13.966731+0000","last_became_active":"2024-09-19T06:47:24.839631+0000","last_became_peered":"2024-09-19T06:47:24.839631+0000","last_unstale":"2024-09-19T06:49:13.966731+0000","last_undegraded":"2024-09-19T06:49:13.966731+0000","last_fullsized":"2024-09-19T06:49:13.966731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:21.764812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970278+0000","last_change":"2024-09-19T06:47:24.849281+0000","last_active":"2024-09-19T06:49:13.970278+0000","last_peered":"2024-09-19T06:49:13.970278+0000","last_clean":"2024-09-19T06:49:13.970278+0000","last_became_active":"2024-09-19T06:47:24.848977+0000","last_became_peered":"2024-09-19T06:47:24.848977+0000","last_unstale":"2024-09-19T06:49:13.970278+0000","last_undegraded":"2024-09-19T06:49:13.970278+0000","last_fullsized":"2024-09-19T06:49:13.970278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:36.693758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973886+0000","last_change":"2024-09-19T06:47:24.843619+0000","last_active":"2024-09-19T06:49:13.973886+0000","last_peered":"2024-09-19T06:49:13.973886+0000","last_clean":"2024-09-19T06:49:13.973886+0000","last_became_active":"2024-09-19T06:47:24.843323+0000","last_became_peered":"2024-09-19T06:47:24.843323+0000","last_unstale":"2024-09-19T06:49:13.973886+0000","last_undegraded":"2024-09-19T06:49:13.973886+0000","last_fullsized":"2024-09-19T06:49:13.973886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:52.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960333+0000","last_change":"2024-09-19T06:47:24.853165+0000","last_active":"2024-09-19T06:49:13.960333+0000","last_peered":"2024-09-19T06:49:13.960333+0000","last_clean":"2024-09-19T06:49:13.960333+0000","last_became_active":"2024-09-19T06:47:24.852864+0000","last_became_peered":"2024-09-19T06:47:24.852864+0000","last_unstale":"2024-09-19T06:49:13.960333+0000","last_undegraded":"2024-09-19T06:49:13.960333+0000","last_fullsized":"2024-09-19T06:49:13.960333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:38.803478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.975021+0000","last_change":"2024-09-19T06:47:24.838151+0000","last_active":"2024-09-19T06:49:13.975021+0000","last_peered":"2024-09-19T06:49:13.975021+0000","last_clean":"2024-09-19T06:49:13.975021+0000","last_became_active":"2024-09-19T06:47:24.837855+0000","last_became_peered":"2024-09-19T06:47:24.837855+0000","last_unstale":"2024-09-19T06:49:13.975021+0000","last_undegraded":"2024-09-19T06:49:13.975021+0000","last_fullsized":"2024-09-19T06:49:13.975021+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:16.675557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.964466+0000","last_change":"2024-09-19T06:47:24.843941+0000","last_active":"2024-09-19T06:49:13.964466+0000","last_peered":"2024-09-19T06:49:13.964466+0000","last_clean":"2024-09-19T06:49:13.964466+0000","last_became_active":"2024-09-19T06:47:24.843551+0000","last_became_peered":"2024-09-19T06:47:24.843551+0000","last_unstale":"2024-09-19T06:49:13.964466+0000","last_undegraded":"2024-09-19T06:49:13.964466+0000","last_fullsized":"2024-09-19T06:49:13.964466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:00:49.388265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.954208+0000","last_change":"2024-09-19T06:47:40.836228+0000","last_active":"2024-09-19T06:49:13.954208+0000","last_peered":"2024-09-19T06:49:13.954208+0000","last_clean":"2024-09-19T06:49:13.954208+0000","last_became_active":"2024-09-19T06:47:40.835818+0000","last_became_peered":"2024-09-19T06:47:40.835818+0000","last_unstale":"2024-09-19T06:49:13.954208+0000","last_undegraded":"2024-09-19T06:49:13.954208+0000","last_fullsized":"2024-09-19T06:49:13.954208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:45.341160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972919+0000","last_change":"2024-09-19T06:47:24.843746+0000","last_active":"2024-09-19T06:49:13.972919+0000","last_peered":"2024-09-19T06:49:13.972919+0000","last_clean":"2024-09-19T06:49:13.972919+0000","last_became_active":"2024-09-19T06:47:24.841495+0000","last_became_peered":"2024-09-19T06:47:24.841495+0000","last_unstale":"2024-09-19T06:49:13.972919+0000","last_undegraded":"2024-09-19T06:49:13.972919+0000","last_fullsized":"2024-09-19T06:49:13.972919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:06:29.995595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.974565+0000","last_change":"2024-09-19T06:47:24.844747+0000","last_active":"2024-09-19T06:49:13.974565+0000","last_peered":"2024-09-19T06:49:13.974565+0000","last_clean":"2024-09-19T06:49:13.974565+0000","last_became_active":"2024-09-19T06:47:24.844425+0000","last_became_peered":"2024-09-19T06:47:24.844425+0000","last_unstale":"2024-09-19T06:49:13.974565+0000","last_undegraded":"2024-09-19T06:49:13.974565+0000","last_fullsized":"2024-09-19T06:49:13.974565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:10.309044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963556+0000","last_change":"2024-09-19T06:47:24.848227+0000","last_active":"2024-09-19T06:49:13.963556+0000","last_peered":"2024-09-19T06:49:13.963556+0000","last_clean":"2024-09-19T06:49:13.963556+0000","last_became_active":"2024-09-19T06:47:24.847914+0000","last_became_peered":"2024-09-19T06:47:24.847914+0000","last_unstale":"2024-09-19T06:49:13.963556+0000","last_undegraded":"2024-09-19T06:49:13.963556+0000","last_fullsized":"2024-09-19T06:49:13.963556+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:47.281613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.968012+0000","last_change":"2024-09-19T06:47:24.844275+0000","last_active":"2024-09-19T06:49:13.968012+0000","last_peered":"2024-09-19T06:49:13.968012+0000","last_clean":"2024-09-19T06:49:13.968012+0000","last_became_active":"2024-09-19T06:47:24.841442+0000","last_became_peered":"2024-09-19T06:47:24.841442+0000","last_unstale":"2024-09-19T06:49:13.968012+0000","last_undegraded":"2024-09-19T06:49:13.968012+0000","last_fullsized":"2024-09-19T06:49:13.968012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:30.198655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963003+0000","last_change":"2024-09-19T06:47:24.846301+0000","last_active":"2024-09-19T06:49:13.963003+0000","last_peered":"2024-09-19T06:49:13.963003+0000","last_clean":"2024-09-19T06:49:13.963003+0000","last_became_active":"2024-09-19T06:47:24.845970+0000","last_became_peered":"2024-09-19T06:47:24.845970+0000","last_unstale":"2024-09-19T06:49:13.963003+0000","last_undegraded":"2024-09-19T06:49:13.963003+0000","last_fullsized":"2024-09-19T06:49:13.963003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:13.245536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973777+0000","last_change":"2024-09-19T06:47:24.844444+0000","last_active":"2024-09-19T06:49:13.973777+0000","last_peered":"2024-09-19T06:49:13.973777+0000","last_clean":"2024-09-19T06:49:13.973777+0000","last_became_active":"2024-09-19T06:47:24.844141+0000","last_became_peered":"2024-09-19T06:47:24.844141+0000","last_unstale":"2024-09-19T06:49:13.973777+0000","last_undegraded":"2024-09-19T06:49:13.973777+0000","last_fullsized":"2024-09-19T06:49:13.973777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:44.739242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955255+0000","last_change":"2024-09-19T06:47:24.845633+0000","last_active":"2024-09-19T06:49:13.955255+0000","last_peered":"2024-09-19T06:49:13.955255+0000","last_clean":"2024-09-19T06:49:13.955255+0000","last_became_active":"2024-09-19T06:47:24.845319+0000","last_became_peered":"2024-09-19T06:47:24.845319+0000","last_unstale":"2024-09-19T06:49:13.955255+0000","last_undegraded":"2024-09-19T06:49:13.955255+0000","last_fullsized":"2024-09-19T06:49:13.955255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:00.820006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959353+0000","last_change":"2024-09-19T06:47:24.846457+0000","last_active":"2024-09-19T06:49:13.959353+0000","last_peered":"2024-09-19T06:49:13.959353+0000","last_clean":"2024-09-19T06:49:13.959353+0000","last_became_active":"2024-09-19T06:47:24.846130+0000","last_became_peered":"2024-09-19T06:47:24.846130+0000","last_unstale":"2024-09-19T06:49:13.959353+0000","last_undegraded":"2024-09-19T06:49:13.959353+0000","last_fullsized":"2024-09-19T06:49:13.959353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:42.179145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969751+0000","last_change":"2024-09-19T06:47:24.843141+0000","last_active":"2024-09-19T06:49:13.969751+0000","last_peered":"2024-09-19T06:49:13.969751+0000","last_clean":"2024-09-19T06:49:13.969751+0000","last_became_active":"2024-09-19T06:47:24.842810+0000","last_became_peered":"2024-09-19T06:47:24.842810+0000","last_unstale":"2024-09-19T06:49:13.969751+0000","last_undegraded":"2024-09-19T06:49:13.969751+0000","last_fullsized":"2024-09-19T06:49:13.969751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:43.440453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.965342+0000","last_change":"2024-09-19T06:47:24.846171+0000","last_active":"2024-09-19T06:49:13.965342+0000","last_peered":"2024-09-19T06:49:13.965342+0000","last_clean":"2024-09-19T06:49:13.965342+0000","last_became_active":"2024-09-19T06:47:24.845799+0000","last_became_peered":"2024-09-19T06:47:24.845799+0000","last_unstale":"2024-09-19T06:49:13.965342+0000","last_undegraded":"2024-09-19T06:49:13.965342+0000","last_fullsized":"2024-09-19T06:49:13.965342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:23.103061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972106+0000","last_change":"2024-09-19T06:47:24.848723+0000","last_active":"2024-09-19T06:49:13.972106+0000","last_peered":"2024-09-19T06:49:13.972106+0000","last_clean":"2024-09-19T06:49:13.972106+0000","last_became_active":"2024-09-19T06:47:24.848367+0000","last_became_peered":"2024-09-19T06:47:24.848367+0000","last_unstale":"2024-09-19T06:49:13.972106+0000","last_undegraded":"2024-09-19T06:49:13.972106+0000","last_fullsized":"2024-09-19T06:49:13.972106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:15.985293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960543+0000","last_change":"2024-09-19T06:47:24.841379+0000","last_active":"2024-09-19T06:49:13.960543+0000","last_peered":"2024-09-19T06:49:13.960543+0000","last_clean":"2024-09-19T06:49:13.960543+0000","last_became_active":"2024-09-19T06:47:24.841041+0000","last_became_peered":"2024-09-19T06:47:24.841041+0000","last_unstale":"2024-09-19T06:49:13.960543+0000","last_undegraded":"2024-09-19T06:49:13.960543+0000","last_fullsized":"2024-09-19T06:49:13.960543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:11.799022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955643+0000","last_change":"2024-09-19T06:47:24.849508+0000","last_active":"2024-09-19T06:49:13.955643+0000","last_peered":"2024-09-19T06:49:13.955643+0000","last_clean":"2024-09-19T06:49:13.955643+0000","last_became_active":"2024-09-19T06:47:24.849196+0000","last_became_peered":"2024-09-19T06:47:24.849196+0000","last_unstale":"2024-09-19T06:49:13.955643+0000","last_undegraded":"2024-09-19T06:49:13.955643+0000","last_fullsized":"2024-09-19T06:49:13.955643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:35.079720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.617944+0000","last_change":"2024-09-19T06:47:22.829848+0000","last_active":"2024-09-19T06:49:23.617944+0000","last_peered":"2024-09-19T06:49:23.617944+0000","last_clean":"2024-09-19T06:49:23.617944+0000","last_became_active":"2024-09-19T06:47:22.829511+0000","last_became_peered":"2024-09-19T06:47:22.829511+0000","last_unstale":"2024-09-19T06:49:23.617944+0000","last_undegraded":"2024-09-19T06:49:23.617944+0000","last_fullsized":"2024-09-19T06:49:23.617944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:32.665252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971999+0000","last_change":"2024-09-19T06:47:24.843245+0000","last_active":"2024-09-19T06:49:13.971999+0000","last_peered":"2024-09-19T06:49:13.971999+0000","last_clean":"2024-09-19T06:49:13.971999+0000","last_became_active":"2024-09-19T06:47:24.842885+0000","last_became_peered":"2024-09-19T06:47:24.842885+0000","last_unstale":"2024-09-19T06:49:13.971999+0000","last_undegraded":"2024-09-19T06:49:13.971999+0000","last_fullsized":"2024-09-19T06:49:13.971999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:26.229003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.926577+0000","last_change":"2024-09-19T06:47:22.831456+0000","last_active":"2024-09-19T06:49:13.926577+0000","last_peered":"2024-09-19T06:49:13.926577+0000","last_clean":"2024-09-19T06:49:13.926577+0000","last_became_active":"2024-09-19T06:47:22.831132+0000","last_became_peered":"2024-09-19T06:47:22.831132+0000","last_unstale":"2024-09-19T06:49:13.926577+0000","last_undegraded":"2024-09-19T06:49:13.926577+0000","last_fullsized":"2024-09-19T06:49:13.926577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:49.867713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932598+0000","last_change":"2024-09-19T06:47:22.827742+0000","last_active":"2024-09-19T06:49:13.932598+0000","last_peered":"2024-09-19T06:49:13.932598+0000","last_clean":"2024-09-19T06:49:13.932598+0000","last_became_active":"2024-09-19T06:47:22.827371+0000","last_became_peered":"2024-09-19T06:47:22.827371+0000","last_unstale":"2024-09-19T06:49:13.932598+0000","last_undegraded":"2024-09-19T06:49:13.932598+0000","last_fullsized":"2024-09-19T06:49:13.932598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:35.153129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963305+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:49:13.963305+0000","last_peered":"2024-09-19T06:49:13.963305+0000","last_clean":"2024-09-19T06:49:13.963305+0000","last_became_active":"2024-09-19T06:47:24.841693+0000","last_became_peered":"2024-09-19T06:47:24.841693+0000","last_unstale":"2024-09-19T06:49:13.963305+0000","last_undegraded":"2024-09-19T06:49:13.963305+0000","last_fullsized":"2024-09-19T06:49:13.963305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:27.828296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.917388+0000","last_change":"2024-09-19T06:47:22.832531+0000","last_active":"2024-09-19T06:49:13.917388+0000","last_peered":"2024-09-19T06:49:13.917388+0000","last_clean":"2024-09-19T06:49:13.917388+0000","last_became_active":"2024-09-19T06:47:22.832213+0000","last_became_peered":"2024-09-19T06:47:22.832213+0000","last_unstale":"2024-09-19T06:49:13.917388+0000","last_undegraded":"2024-09-19T06:49:13.917388+0000","last_fullsized":"2024-09-19T06:49:13.917388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:46:03.168406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970720+0000","last_change":"2024-09-19T06:47:24.837690+0000","last_active":"2024-09-19T06:49:13.970720+0000","last_peered":"2024-09-19T06:49:13.970720+0000","last_clean":"2024-09-19T06:49:13.970720+0000","last_became_active":"2024-09-19T06:47:24.837389+0000","last_became_peered":"2024-09-19T06:47:24.837389+0000","last_unstale":"2024-09-19T06:49:13.970720+0000","last_undegraded":"2024-09-19T06:49:13.970720+0000","last_fullsized":"2024-09-19T06:49:13.970720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:20.653832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":248,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:24.618359+0000","last_change":"2024-09-19T06:47:22.833808+0000","last_active":"2024-09-19T06:49:24.618359+0000","last_peered":"2024-09-19T06:49:24.618359+0000","last_clean":"2024-09-19T06:49:24.618359+0000","last_became_active":"2024-09-19T06:47:22.832468+0000","last_became_peered":"2024-09-19T06:47:22.832468+0000","last_unstale":"2024-09-19T06:49:24.618359+0000","last_undegraded":"2024-09-19T06:49:24.618359+0000","last_fullsized":"2024-09-19T06:49:24.618359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:39:31.131321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":218,"num_read_kb":218,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973091+0000","last_change":"2024-09-19T06:47:24.840928+0000","last_active":"2024-09-19T06:49:13.973091+0000","last_peered":"2024-09-19T06:49:13.973091+0000","last_clean":"2024-09-19T06:49:13.973091+0000","last_became_active":"2024-09-19T06:47:24.839928+0000","last_became_peered":"2024-09-19T06:47:24.839928+0000","last_unstale":"2024-09-19T06:49:13.973091+0000","last_undegraded":"2024-09-19T06:49:13.973091+0000","last_fullsized":"2024-09-19T06:49:13.973091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:29.486003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.928845+0000","last_change":"2024-09-19T06:47:22.833310+0000","last_active":"2024-09-19T06:49:13.928845+0000","last_peered":"2024-09-19T06:49:13.928845+0000","last_clean":"2024-09-19T06:49:13.928845+0000","last_became_active":"2024-09-19T06:47:22.832976+0000","last_became_peered":"2024-09-19T06:47:22.832976+0000","last_unstale":"2024-09-19T06:49:13.928845+0000","last_undegraded":"2024-09-19T06:49:13.928845+0000","last_fullsized":"2024-09-19T06:49:13.928845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:47.604853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969853+0000","last_change":"2024-09-19T06:47:24.848687+0000","last_active":"2024-09-19T06:49:13.969853+0000","last_peered":"2024-09-19T06:49:13.969853+0000","last_clean":"2024-09-19T06:49:13.969853+0000","last_became_active":"2024-09-19T06:47:24.848352+0000","last_became_peered":"2024-09-19T06:47:24.848352+0000","last_unstale":"2024-09-19T06:49:13.969853+0000","last_undegraded":"2024-09-19T06:49:13.969853+0000","last_fullsized":"2024-09-19T06:49:13.969853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:54.459206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.917489+0000","last_change":"2024-09-19T06:47:22.851479+0000","last_active":"2024-09-19T06:49:13.917489+0000","last_peered":"2024-09-19T06:49:13.917489+0000","last_clean":"2024-09-19T06:49:13.917489+0000","last_became_active":"2024-09-19T06:47:22.851125+0000","last_became_peered":"2024-09-19T06:47:22.851125+0000","last_unstale":"2024-09-19T06:49:13.917489+0000","last_undegraded":"2024-09-19T06:49:13.917489+0000","last_fullsized":"2024-09-19T06:49:13.917489+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:42.954034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973229+0000","last_change":"2024-09-19T06:47:24.851007+0000","last_active":"2024-09-19T06:49:13.973229+0000","last_peered":"2024-09-19T06:49:13.973229+0000","last_clean":"2024-09-19T06:49:13.973229+0000","last_became_active":"2024-09-19T06:47:24.850465+0000","last_became_peered":"2024-09-19T06:47:24.850465+0000","last_unstale":"2024-09-19T06:49:13.973229+0000","last_undegraded":"2024-09-19T06:49:13.973229+0000","last_fullsized":"2024-09-19T06:49:13.973229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:10.824672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'39","reported_seq":295,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:24.618888+0000","last_change":"2024-09-19T06:47:22.834893+0000","last_active":"2024-09-19T06:49:24.618888+0000","last_peered":"2024-09-19T06:49:24.618888+0000","last_clean":"2024-09-19T06:49:24.618888+0000","last_became_active":"2024-09-19T06:47:22.834521+0000","last_became_peered":"2024-09-19T06:47:22.834521+0000","last_unstale":"2024-09-19T06:49:24.618888+0000","last_undegraded":"2024-09-19T06:49:24.618888+0000","last_fullsized":"2024-09-19T06:49:24.618888+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:29.742284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70398,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":232,"num_read_kb":185,"num_write":41,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.957411+0000","last_change":"2024-09-19T06:47:24.848191+0000","last_active":"2024-09-19T06:49:13.957411+0000","last_peered":"2024-09-19T06:49:13.957411+0000","last_clean":"2024-09-19T06:49:13.957411+0000","last_became_active":"2024-09-19T06:47:24.847889+0000","last_became_peered":"2024-09-19T06:47:24.847889+0000","last_unstale":"2024-09-19T06:49:13.957411+0000","last_undegraded":"2024-09-19T06:49:13.957411+0000","last_fullsized":"2024-09-19T06:49:13.957411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:32.574360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.924532+0000","last_change":"2024-09-19T06:47:22.832005+0000","last_active":"2024-09-19T06:49:13.924532+0000","last_peered":"2024-09-19T06:49:13.924532+0000","last_clean":"2024-09-19T06:49:13.924532+0000","last_became_active":"2024-09-19T06:47:22.831342+0000","last_became_peered":"2024-09-19T06:47:22.831342+0000","last_unstale":"2024-09-19T06:49:13.924532+0000","last_undegraded":"2024-09-19T06:49:13.924532+0000","last_fullsized":"2024-09-19T06:49:13.924532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:53.444748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.964849+0000","last_change":"2024-09-19T06:47:24.843328+0000","last_active":"2024-09-19T06:49:13.964849+0000","last_peered":"2024-09-19T06:49:13.964849+0000","last_clean":"2024-09-19T06:49:13.964849+0000","last_became_active":"2024-09-19T06:47:24.842344+0000","last_became_peered":"2024-09-19T06:47:24.842344+0000","last_unstale":"2024-09-19T06:49:13.964849+0000","last_undegraded":"2024-09-19T06:49:13.964849+0000","last_fullsized":"2024-09-19T06:49:13.964849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:11.247819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.926463+0000","last_change":"2024-09-19T06:47:40.844086+0000","last_active":"2024-09-19T06:49:13.926463+0000","last_peered":"2024-09-19T06:49:13.926463+0000","last_clean":"2024-09-19T06:49:13.926463+0000","last_became_active":"2024-09-19T06:47:40.835015+0000","last_became_peered":"2024-09-19T06:47:40.835015+0000","last_unstale":"2024-09-19T06:49:13.926463+0000","last_undegraded":"2024-09-19T06:49:13.926463+0000","last_fullsized":"2024-09-19T06:49:13.926463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:09.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972972+0000","last_change":"2024-09-19T06:47:40.836466+0000","last_active":"2024-09-19T06:49:13.972972+0000","last_peered":"2024-09-19T06:49:13.972972+0000","last_clean":"2024-09-19T06:49:13.972972+0000","last_became_active":"2024-09-19T06:47:40.836237+0000","last_became_peered":"2024-09-19T06:47:40.836237+0000","last_unstale":"2024-09-19T06:49:13.972972+0000","last_undegraded":"2024-09-19T06:49:13.972972+0000","last_fullsized":"2024-09-19T06:49:13.972972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:00.094685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'11","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.617882+0000","last_change":"2024-09-19T06:47:22.830359+0000","last_active":"2024-09-19T06:49:23.617882+0000","last_peered":"2024-09-19T06:49:23.617882+0000","last_clean":"2024-09-19T06:49:23.617882+0000","last_became_active":"2024-09-19T06:47:22.829960+0000","last_became_peered":"2024-09-19T06:47:22.829960+0000","last_unstale":"2024-09-19T06:49:23.617882+0000","last_undegraded":"2024-09-19T06:49:23.617882+0000","last_fullsized":"2024-09-19T06:49:23.617882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:55.998243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2609,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":7,"num_write":11,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971872+0000","last_change":"2024-09-19T06:47:24.836668+0000","last_active":"2024-09-19T06:49:13.971872+0000","last_peered":"2024-09-19T06:49:13.971872+0000","last_clean":"2024-09-19T06:49:13.971872+0000","last_became_active":"2024-09-19T06:47:24.836393+0000","last_became_peered":"2024-09-19T06:47:24.836393+0000","last_unstale":"2024-09-19T06:49:13.971872+0000","last_undegraded":"2024-09-19T06:49:13.971872+0000","last_fullsized":"2024-09-19T06:49:13.971872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:26.248948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":66,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.931451+0000","last_change":"2024-09-19T06:47:40.858323+0000","last_active":"2024-09-19T06:49:13.931451+0000","last_peered":"2024-09-19T06:49:13.931451+0000","last_clean":"2024-09-19T06:49:13.931451+0000","last_became_active":"2024-09-19T06:47:40.840802+0000","last_became_peered":"2024-09-19T06:47:40.840802+0000","last_unstale":"2024-09-19T06:49:13.931451+0000","last_undegraded":"2024-09-19T06:49:13.931451+0000","last_fullsized":"2024-09-19T06:49:13.931451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:14.666916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970280+0000","last_change":"2024-09-19T06:47:24.843681+0000","last_active":"2024-09-19T06:49:13.970280+0000","last_peered":"2024-09-19T06:49:13.970280+0000","last_clean":"2024-09-19T06:49:13.970280+0000","last_became_active":"2024-09-19T06:47:24.841228+0000","last_became_peered":"2024-09-19T06:47:24.841228+0000","last_unstale":"2024-09-19T06:49:13.970280+0000","last_undegraded":"2024-09-19T06:49:13.970280+0000","last_fullsized":"2024-09-19T06:49:13.970280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:56.418962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.933661+0000","last_change":"2024-09-19T06:47:22.833893+0000","last_active":"2024-09-19T06:49:13.933661+0000","last_peered":"2024-09-19T06:49:13.933661+0000","last_clean":"2024-09-19T06:49:13.933661+0000","last_became_active":"2024-09-19T06:47:22.833217+0000","last_became_peered":"2024-09-19T06:47:22.833217+0000","last_unstale":"2024-09-19T06:49:13.933661+0000","last_undegraded":"2024-09-19T06:49:13.933661+0000","last_fullsized":"2024-09-19T06:49:13.933661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:19.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955096+0000","last_change":"2024-09-19T06:47:24.848399+0000","last_active":"2024-09-19T06:49:13.955096+0000","last_peered":"2024-09-19T06:49:13.955096+0000","last_clean":"2024-09-19T06:49:13.955096+0000","last_became_active":"2024-09-19T06:47:24.848081+0000","last_became_peered":"2024-09-19T06:47:24.848081+0000","last_unstale":"2024-09-19T06:49:13.955096+0000","last_undegraded":"2024-09-19T06:49:13.955096+0000","last_fullsized":"2024-09-19T06:49:13.955096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:44.292939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.931968+0000","last_change":"2024-09-19T06:47:40.860955+0000","last_active":"2024-09-19T06:49:13.931968+0000","last_peered":"2024-09-19T06:49:13.931968+0000","last_clean":"2024-09-19T06:49:13.931968+0000","last_became_active":"2024-09-19T06:47:40.835126+0000","last_became_peered":"2024-09-19T06:47:40.835126+0000","last_unstale":"2024-09-19T06:49:13.931968+0000","last_undegraded":"2024-09-19T06:49:13.931968+0000","last_fullsized":"2024-09-19T06:49:13.931968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:02:26.763229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971391+0000","last_change":"2024-09-19T06:47:24.849680+0000","last_active":"2024-09-19T06:49:13.971391+0000","last_peered":"2024-09-19T06:49:13.971391+0000","last_clean":"2024-09-19T06:49:13.971391+0000","last_became_active":"2024-09-19T06:47:24.849323+0000","last_became_peered":"2024-09-19T06:47:24.849323+0000","last_unstale":"2024-09-19T06:49:13.971391+0000","last_undegraded":"2024-09-19T06:49:13.971391+0000","last_fullsized":"2024-09-19T06:49:13.971391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:59.763503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'67","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:24.414577+0000","last_change":"2024-09-19T06:47:22.831689+0000","last_active":"2024-09-19T06:49:24.414577+0000","last_peered":"2024-09-19T06:49:24.414577+0000","last_clean":"2024-09-19T06:49:24.414577+0000","last_became_active":"2024-09-19T06:47:22.829283+0000","last_became_peered":"2024-09-19T06:47:22.829283+0000","last_unstale":"2024-09-19T06:49:24.414577+0000","last_undegraded":"2024-09-19T06:49:24.414577+0000","last_fullsized":"2024-09-19T06:49:24.414577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:37.109542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971792+0000","last_change":"2024-09-19T06:47:24.842793+0000","last_active":"2024-09-19T06:49:13.971792+0000","last_peered":"2024-09-19T06:49:13.971792+0000","last_clean":"2024-09-19T06:49:13.971792+0000","last_became_active":"2024-09-19T06:47:24.842474+0000","last_became_peered":"2024-09-19T06:47:24.842474+0000","last_unstale":"2024-09-19T06:49:13.971792+0000","last_undegraded":"2024-09-19T06:49:13.971792+0000","last_fullsized":"2024-09-19T06:49:13.971792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:23.110736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'10","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.617464+0000","last_change":"2024-09-19T06:47:23.047356+0000","last_active":"2024-09-19T06:49:23.617464+0000","last_peered":"2024-09-19T06:49:23.617464+0000","last_clean":"2024-09-19T06:49:23.617464+0000","last_became_active":"2024-09-19T06:47:23.046941+0000","last_became_peered":"2024-09-19T06:47:23.046941+0000","last_unstale":"2024-09-19T06:49:23.617464+0000","last_undegraded":"2024-09-19T06:49:23.617464+0000","last_fullsized":"2024-09-19T06:49:23.617464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:43.490952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":5,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971243+0000","last_change":"2024-09-19T06:47:24.851025+0000","last_active":"2024-09-19T06:49:13.971243+0000","last_peered":"2024-09-19T06:49:13.971243+0000","last_clean":"2024-09-19T06:49:13.971243+0000","last_became_active":"2024-09-19T06:47:24.850681+0000","last_became_peered":"2024-09-19T06:47:24.850681+0000","last_unstale":"2024-09-19T06:49:13.971243+0000","last_undegraded":"2024-09-19T06:49:13.971243+0000","last_fullsized":"2024-09-19T06:49:13.971243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:46.884747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"83'148","reported_seq":392,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:16.044071+0000","last_change":"2024-09-19T06:46:13.927379+0000","last_active":"2024-09-19T06:49:16.044071+0000","last_peered":"2024-09-19T06:49:16.044071+0000","last_clean":"2024-09-19T06:49:16.044071+0000","last_became_active":"2024-09-19T06:46:13.926926+0000","last_became_peered":"2024-09-19T06:46:13.926926+0000","last_unstale":"2024-09-19T06:49:16.044071+0000","last_undegraded":"2024-09-19T06:49:16.044071+0000","last_fullsized":"2024-09-19T06:49:16.044071+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_clean_scrub_stamp":"2024-09-19T06:42:52.051335+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:29.542128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.927403+0000","last_change":"2024-09-19T06:47:22.830387+0000","last_active":"2024-09-19T06:49:13.927403+0000","last_peered":"2024-09-19T06:49:13.927403+0000","last_clean":"2024-09-19T06:49:13.927403+0000","last_became_active":"2024-09-19T06:47:22.829970+0000","last_became_peered":"2024-09-19T06:47:22.829970+0000","last_unstale":"2024-09-19T06:49:13.927403+0000","last_undegraded":"2024-09-19T06:49:13.927403+0000","last_fullsized":"2024-09-19T06:49:13.927403+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:51.385456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.974671+0000","last_change":"2024-09-19T06:47:24.843752+0000","last_active":"2024-09-19T06:49:13.974671+0000","last_peered":"2024-09-19T06:49:13.974671+0000","last_clean":"2024-09-19T06:49:13.974671+0000","last_became_active":"2024-09-19T06:47:24.842631+0000","last_became_peered":"2024-09-19T06:47:24.842631+0000","last_unstale":"2024-09-19T06:49:13.974671+0000","last_undegraded":"2024-09-19T06:49:13.974671+0000","last_fullsized":"2024-09-19T06:49:13.974671+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:19.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.928732+0000","last_change":"2024-09-19T06:47:22.832647+0000","last_active":"2024-09-19T06:49:13.928732+0000","last_peered":"2024-09-19T06:49:13.928732+0000","last_clean":"2024-09-19T06:49:13.928732+0000","last_became_active":"2024-09-19T06:47:22.831774+0000","last_became_peered":"2024-09-19T06:47:22.831774+0000","last_unstale":"2024-09-19T06:49:13.928732+0000","last_undegraded":"2024-09-19T06:49:13.928732+0000","last_fullsized":"2024-09-19T06:49:13.928732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:57.824610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.948364+0000","last_change":"2024-09-19T06:47:24.844301+0000","last_active":"2024-09-19T06:49:13.948364+0000","last_peered":"2024-09-19T06:49:13.948364+0000","last_clean":"2024-09-19T06:49:13.948364+0000","last_became_active":"2024-09-19T06:47:24.839748+0000","last_became_peered":"2024-09-19T06:47:24.839748+0000","last_unstale":"2024-09-19T06:49:13.948364+0000","last_undegraded":"2024-09-19T06:49:13.948364+0000","last_fullsized":"2024-09-19T06:49:13.948364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:46.407693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":140,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.617287+0000","last_change":"2024-09-19T06:47:22.851008+0000","last_active":"2024-09-19T06:49:23.617287+0000","last_peered":"2024-09-19T06:49:23.617287+0000","last_clean":"2024-09-19T06:49:23.617287+0000","last_became_active":"2024-09-19T06:47:22.850601+0000","last_became_peered":"2024-09-19T06:47:22.850601+0000","last_unstale":"2024-09-19T06:49:23.617287+0000","last_undegraded":"2024-09-19T06:49:23.617287+0000","last_fullsized":"2024-09-19T06:49:23.617287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:04.610277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970407+0000","last_change":"2024-09-19T06:47:24.839684+0000","last_active":"2024-09-19T06:49:13.970407+0000","last_peered":"2024-09-19T06:49:13.970407+0000","last_clean":"2024-09-19T06:49:13.970407+0000","last_became_active":"2024-09-19T06:47:24.839126+0000","last_became_peered":"2024-09-19T06:47:24.839126+0000","last_unstale":"2024-09-19T06:49:13.970407+0000","last_undegraded":"2024-09-19T06:49:13.970407+0000","last_fullsized":"2024-09-19T06:49:13.970407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:09.458535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":140,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.325749+0000","last_change":"2024-09-19T06:47:22.831819+0000","last_active":"2024-09-19T06:49:23.325749+0000","last_peered":"2024-09-19T06:49:23.325749+0000","last_clean":"2024-09-19T06:49:23.325749+0000","last_became_active":"2024-09-19T06:47:22.831332+0000","last_became_peered":"2024-09-19T06:47:22.831332+0000","last_unstale":"2024-09-19T06:49:23.325749+0000","last_undegraded":"2024-09-19T06:49:23.325749+0000","last_fullsized":"2024-09-19T06:49:23.325749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:35.032949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.965109+0000","last_change":"2024-09-19T06:47:24.846738+0000","last_active":"2024-09-19T06:49:13.965109+0000","last_peered":"2024-09-19T06:49:13.965109+0000","last_clean":"2024-09-19T06:49:13.965109+0000","last_became_active":"2024-09-19T06:47:24.846389+0000","last_became_peered":"2024-09-19T06:47:24.846389+0000","last_unstale":"2024-09-19T06:49:13.965109+0000","last_undegraded":"2024-09-19T06:49:13.965109+0000","last_fullsized":"2024-09-19T06:49:13.965109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:19.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.918173+0000","last_change":"2024-09-19T06:47:22.851065+0000","last_active":"2024-09-19T06:49:13.918173+0000","last_peered":"2024-09-19T06:49:13.918173+0000","last_clean":"2024-09-19T06:49:13.918173+0000","last_became_active":"2024-09-19T06:47:22.850647+0000","last_became_peered":"2024-09-19T06:47:22.850647+0000","last_unstale":"2024-09-19T06:49:13.918173+0000","last_undegraded":"2024-09-19T06:49:13.918173+0000","last_fullsized":"2024-09-19T06:49:13.918173+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:41.993501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966795+0000","last_change":"2024-09-19T06:47:24.838905+0000","last_active":"2024-09-19T06:49:13.966795+0000","last_peered":"2024-09-19T06:49:13.966795+0000","last_clean":"2024-09-19T06:49:13.966795+0000","last_became_active":"2024-09-19T06:47:24.838443+0000","last_became_peered":"2024-09-19T06:47:24.838443+0000","last_unstale":"2024-09-19T06:49:13.966795+0000","last_undegraded":"2024-09-19T06:49:13.966795+0000","last_fullsized":"2024-09-19T06:49:13.966795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:12.486137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932653+0000","last_change":"2024-09-19T06:47:22.851431+0000","last_active":"2024-09-19T06:49:13.932653+0000","last_peered":"2024-09-19T06:49:13.932653+0000","last_clean":"2024-09-19T06:49:13.932653+0000","last_became_active":"2024-09-19T06:47:22.851092+0000","last_became_peered":"2024-09-19T06:47:22.851092+0000","last_unstale":"2024-09-19T06:49:13.932653+0000","last_undegraded":"2024-09-19T06:49:13.932653+0000","last_fullsized":"2024-09-19T06:49:13.932653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:26.183803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959143+0000","last_change":"2024-09-19T06:47:24.839046+0000","last_active":"2024-09-19T06:49:13.959143+0000","last_peered":"2024-09-19T06:49:13.959143+0000","last_clean":"2024-09-19T06:49:13.959143+0000","last_became_active":"2024-09-19T06:47:24.838680+0000","last_became_peered":"2024-09-19T06:47:24.838680+0000","last_unstale":"2024-09-19T06:49:13.959143+0000","last_undegraded":"2024-09-19T06:49:13.959143+0000","last_fullsized":"2024-09-19T06:49:13.959143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:45.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'102","reported_seq":231,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:24.747427+0000","last_change":"2024-09-19T06:47:40.865241+0000","last_active":"2024-09-19T06:49:24.747427+0000","last_peered":"2024-09-19T06:49:24.747427+0000","last_clean":"2024-09-19T06:49:24.747427+0000","last_became_active":"2024-09-19T06:47:40.837765+0000","last_became_peered":"2024-09-19T06:47:40.837765+0000","last_unstale":"2024-09-19T06:49:24.747427+0000","last_undegraded":"2024-09-19T06:49:24.747427+0000","last_fullsized":"2024-09-19T06:49:24.747427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":186,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966118+0000","last_change":"2024-09-19T06:47:24.848599+0000","last_active":"2024-09-19T06:49:13.966118+0000","last_peered":"2024-09-19T06:49:13.966118+0000","last_clean":"2024-09-19T06:49:13.966118+0000","last_became_active":"2024-09-19T06:47:24.848305+0000","last_became_peered":"2024-09-19T06:47:24.848305+0000","last_unstale":"2024-09-19T06:49:13.966118+0000","last_undegraded":"2024-09-19T06:49:13.966118+0000","last_fullsized":"2024-09-19T06:49:13.966118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:31.496188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.927755+0000","last_change":"2024-09-19T06:47:23.047484+0000","last_active":"2024-09-19T06:49:13.927755+0000","last_peered":"2024-09-19T06:49:13.927755+0000","last_clean":"2024-09-19T06:49:13.927755+0000","last_became_active":"2024-09-19T06:47:23.047058+0000","last_became_peered":"2024-09-19T06:47:23.047058+0000","last_unstale":"2024-09-19T06:49:13.927755+0000","last_undegraded":"2024-09-19T06:49:13.927755+0000","last_fullsized":"2024-09-19T06:49:13.927755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:34.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970265+0000","last_change":"2024-09-19T06:47:24.843498+0000","last_active":"2024-09-19T06:49:13.970265+0000","last_peered":"2024-09-19T06:49:13.970265+0000","last_clean":"2024-09-19T06:49:13.970265+0000","last_became_active":"2024-09-19T06:47:24.843204+0000","last_became_peered":"2024-09-19T06:47:24.843204+0000","last_unstale":"2024-09-19T06:49:13.970265+0000","last_undegraded":"2024-09-19T06:49:13.970265+0000","last_fullsized":"2024-09-19T06:49:13.970265+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:25.585023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.919373+0000","last_change":"2024-09-19T06:47:22.851519+0000","last_active":"2024-09-19T06:49:13.919373+0000","last_peered":"2024-09-19T06:49:13.919373+0000","last_clean":"2024-09-19T06:49:13.919373+0000","last_became_active":"2024-09-19T06:47:22.851163+0000","last_became_peered":"2024-09-19T06:47:22.851163+0000","last_unstale":"2024-09-19T06:49:13.919373+0000","last_undegraded":"2024-09-19T06:49:13.919373+0000","last_fullsized":"2024-09-19T06:49:13.919373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:04.502795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.945308+0000","last_change":"2024-09-19T06:47:24.845239+0000","last_active":"2024-09-19T06:49:13.945308+0000","last_peered":"2024-09-19T06:49:13.945308+0000","last_clean":"2024-09-19T06:49:13.945308+0000","last_became_active":"2024-09-19T06:47:24.844934+0000","last_became_peered":"2024-09-19T06:47:24.844934+0000","last_unstale":"2024-09-19T06:49:13.945308+0000","last_undegraded":"2024-09-19T06:49:13.945308+0000","last_fullsized":"2024-09-19T06:49:13.945308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:43.610604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.934380+0000","last_change":"2024-09-19T06:47:22.830232+0000","last_active":"2024-09-19T06:49:13.934380+0000","last_peered":"2024-09-19T06:49:13.934380+0000","last_clean":"2024-09-19T06:49:13.934380+0000","last_became_active":"2024-09-19T06:47:22.829901+0000","last_became_peered":"2024-09-19T06:47:22.829901+0000","last_unstale":"2024-09-19T06:49:13.934380+0000","last_undegraded":"2024-09-19T06:49:13.934380+0000","last_fullsized":"2024-09-19T06:49:13.934380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:12.251427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955845+0000","last_change":"2024-09-19T06:47:24.839304+0000","last_active":"2024-09-19T06:49:13.955845+0000","last_peered":"2024-09-19T06:49:13.955845+0000","last_clean":"2024-09-19T06:49:13.955845+0000","last_became_active":"2024-09-19T06:47:24.838986+0000","last_became_peered":"2024-09-19T06:47:24.838986+0000","last_unstale":"2024-09-19T06:49:13.955845+0000","last_undegraded":"2024-09-19T06:49:13.955845+0000","last_fullsized":"2024-09-19T06:49:13.955845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:39.377886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.885319+0000","last_change":"2024-09-19T06:47:22.828553+0000","last_active":"2024-09-19T06:49:14.885319+0000","last_peered":"2024-09-19T06:49:14.885319+0000","last_clean":"2024-09-19T06:49:14.885319+0000","last_became_active":"2024-09-19T06:47:22.828213+0000","last_became_peered":"2024-09-19T06:47:22.828213+0000","last_unstale":"2024-09-19T06:49:14.885319+0000","last_undegraded":"2024-09-19T06:49:14.885319+0000","last_fullsized":"2024-09-19T06:49:14.885319+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:31.649761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960387+0000","last_change":"2024-09-19T06:47:24.840747+0000","last_active":"2024-09-19T06:49:13.960387+0000","last_peered":"2024-09-19T06:49:13.960387+0000","last_clean":"2024-09-19T06:49:13.960387+0000","last_became_active":"2024-09-19T06:47:24.840451+0000","last_became_peered":"2024-09-19T06:47:24.840451+0000","last_unstale":"2024-09-19T06:49:13.960387+0000","last_undegraded":"2024-09-19T06:49:13.960387+0000","last_fullsized":"2024-09-19T06:49:13.960387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:41.588751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.933845+0000","last_change":"2024-09-19T06:47:22.831697+0000","last_active":"2024-09-19T06:49:13.933845+0000","last_peered":"2024-09-19T06:49:13.933845+0000","last_clean":"2024-09-19T06:49:13.933845+0000","last_became_active":"2024-09-19T06:47:22.831178+0000","last_became_peered":"2024-09-19T06:47:22.831178+0000","last_unstale":"2024-09-19T06:49:13.933845+0000","last_undegraded":"2024-09-19T06:49:13.933845+0000","last_fullsized":"2024-09-19T06:49:13.933845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:30.975782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971917+0000","last_change":"2024-09-19T06:47:24.845097+0000","last_active":"2024-09-19T06:49:13.971917+0000","last_peered":"2024-09-19T06:49:13.971917+0000","last_clean":"2024-09-19T06:49:13.971917+0000","last_became_active":"2024-09-19T06:47:24.844801+0000","last_became_peered":"2024-09-19T06:47:24.844801+0000","last_unstale":"2024-09-19T06:49:13.971917+0000","last_undegraded":"2024-09-19T06:49:13.971917+0000","last_fullsized":"2024-09-19T06:49:13.971917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:15.264118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971733+0000","last_change":"2024-09-19T06:47:24.851372+0000","last_active":"2024-09-19T06:49:13.971733+0000","last_peered":"2024-09-19T06:49:13.971733+0000","last_clean":"2024-09-19T06:49:13.971733+0000","last_became_active":"2024-09-19T06:47:24.851044+0000","last_became_peered":"2024-09-19T06:47:24.851044+0000","last_unstale":"2024-09-19T06:49:13.971733+0000","last_undegraded":"2024-09-19T06:49:13.971733+0000","last_fullsized":"2024-09-19T06:49:13.971733+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:55.658214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932580+0000","last_change":"2024-09-19T06:47:22.833694+0000","last_active":"2024-09-19T06:49:13.932580+0000","last_peered":"2024-09-19T06:49:13.932580+0000","last_clean":"2024-09-19T06:49:13.932580+0000","last_became_active":"2024-09-19T06:47:22.833215+0000","last_became_peered":"2024-09-19T06:47:22.833215+0000","last_unstale":"2024-09-19T06:49:13.932580+0000","last_undegraded":"2024-09-19T06:49:13.932580+0000","last_fullsized":"2024-09-19T06:49:13.932580+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:32.171522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973551+0000","last_change":"2024-09-19T06:47:24.845737+0000","last_active":"2024-09-19T06:49:13.973551+0000","last_peered":"2024-09-19T06:49:13.973551+0000","last_clean":"2024-09-19T06:49:13.973551+0000","last_became_active":"2024-09-19T06:47:24.845242+0000","last_became_peered":"2024-09-19T06:47:24.845242+0000","last_unstale":"2024-09-19T06:49:13.973551+0000","last_undegraded":"2024-09-19T06:49:13.973551+0000","last_fullsized":"2024-09-19T06:49:13.973551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:51.554946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932499+0000","last_change":"2024-09-19T06:47:22.829869+0000","last_active":"2024-09-19T06:49:13.932499+0000","last_peered":"2024-09-19T06:49:13.932499+0000","last_clean":"2024-09-19T06:49:13.932499+0000","last_became_active":"2024-09-19T06:47:22.829522+0000","last_became_peered":"2024-09-19T06:47:22.829522+0000","last_unstale":"2024-09-19T06:49:13.932499+0000","last_undegraded":"2024-09-19T06:49:13.932499+0000","last_fullsized":"2024-09-19T06:49:13.932499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:26.871434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973419+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:49:13.973419+0000","last_peered":"2024-09-19T06:49:13.973419+0000","last_clean":"2024-09-19T06:49:13.973419+0000","last_became_active":"2024-09-19T06:47:24.841289+0000","last_became_peered":"2024-09-19T06:47:24.841289+0000","last_unstale":"2024-09-19T06:49:13.973419+0000","last_undegraded":"2024-09-19T06:49:13.973419+0000","last_fullsized":"2024-09-19T06:49:13.973419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:21:12.611458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.937218+0000","last_change":"2024-09-19T06:47:22.852198+0000","last_active":"2024-09-19T06:49:13.937218+0000","last_peered":"2024-09-19T06:49:13.937218+0000","last_clean":"2024-09-19T06:49:13.937218+0000","last_became_active":"2024-09-19T06:47:22.850984+0000","last_became_peered":"2024-09-19T06:47:22.850984+0000","last_unstale":"2024-09-19T06:49:13.937218+0000","last_undegraded":"2024-09-19T06:49:13.937218+0000","last_fullsized":"2024-09-19T06:49:13.937218+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:13.620145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959682+0000","last_change":"2024-09-19T06:47:40.837614+0000","last_active":"2024-09-19T06:49:13.959682+0000","last_peered":"2024-09-19T06:49:13.959682+0000","last_clean":"2024-09-19T06:49:13.959682+0000","last_became_active":"2024-09-19T06:47:40.837259+0000","last_became_peered":"2024-09-19T06:47:40.837259+0000","last_unstale":"2024-09-19T06:49:13.959682+0000","last_undegraded":"2024-09-19T06:49:13.959682+0000","last_fullsized":"2024-09-19T06:49:13.959682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:41.429962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":168,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.534111+0000","last_change":"2024-09-19T06:47:41.182639+0000","last_active":"2024-09-19T06:49:23.534111+0000","last_peered":"2024-09-19T06:49:23.534111+0000","last_clean":"2024-09-19T06:49:23.534111+0000","last_became_active":"2024-09-19T06:47:41.174305+0000","last_became_peered":"2024-09-19T06:47:41.174305+0000","last_unstale":"2024-09-19T06:49:23.534111+0000","last_undegraded":"2024-09-19T06:49:23.534111+0000","last_fullsized":"2024-09-19T06:49:23.534111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:10.199502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":802788,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1473,"num_read_kb":956,"num_write":508,"num_write_kb":2390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2777088,"data_stored":2408364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":387,"ondisk_log_size":387,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712684,"statfs":{"total":95995035648,"available":95961788416,"internally_reserved":0,"allocated":1314816,"data_stored":842671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29060,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716092,"statfs":{"total":95995035648,"available":95965278208,"internally_reserved":0,"allocated":1961984,"data_stored":1439332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972399,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28328,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716824,"statfs":{"total":95995035648,"available":95966027776,"internally_reserved":0,"allocated":1335296,"data_stored":840798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234036,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":447288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32684,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712468,"statfs":{"total":95995035648,"available":95961567232,"internally_reserved":0,"allocated":1536000,"data_stored":1042894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626494,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":447692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855425,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28976,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716176,"statfs":{"total":95995035648,"available":95965364224,"internally_reserved":0,"allocated":1875968,"data_stored":1368300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018951,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":449293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215176,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1667072,"data_stored":1168572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":520824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":519675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":450351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":74364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":331355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":402387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":396211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:49:27.037 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:49:27.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:26 smithi174 ceph-mon[35637]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:49:27.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:26 smithi174 ceph-mon[35637]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 54 op/s 2024-09-19T06:49:27.281 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:27.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.723+0000 7f6955a7f640 1 -- 172.21.15.103:0/1631492973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6950102ef0 msgr2=0x7f695010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.723+0000 7f6955a7f640 1 --2- 172.21.15.103:0/1631492973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6950102ef0 0x7f695010da40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6948009ca0 tx=0x7f694802a950 comp rx=0 tx=0).stop 2024-09-19T06:49:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.723+0000 7f6955a7f640 1 -- 172.21.15.103:0/1631492973 shutdown_connections 2024-09-19T06:49:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.723+0000 7f6955a7f640 1 --2- 172.21.15.103:0/1631492973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6950102ef0 0x7f695010da40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.723+0000 7f6955a7f640 1 --2- 172.21.15.103:0/1631492973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6950102570 0x7f69501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.723+0000 7f6955a7f640 1 --2- 172.21.15.103:0/1631492973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6950100e20 0x7f6950101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.723+0000 7f6955a7f640 1 -- 172.21.15.103:0/1631492973 >> 172.21.15.103:0/1631492973 conn(0x7f69500fc7d0 msgr2=0x7f69500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.723+0000 7f6955a7f640 1 -- 172.21.15.103:0/1631492973 shutdown_connections 2024-09-19T06:49:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.723+0000 7f6955a7f640 1 -- 172.21.15.103:0/1631492973 wait complete. 2024-09-19T06:49:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.723+0000 7f6955a7f640 1 Processor -- start 2024-09-19T06:49:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f6955a7f640 1 -- start start 2024-09-19T06:49:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f6955a7f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6950100e20 0x7f69500fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f6955a7f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6950102570 0x7f69500fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f6955a7f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6950102ef0 0x7f69500fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f6955a7f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6950110f80 con 0x7f6950102ef0 2024-09-19T06:49:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f6955a7f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6950110e00 con 0x7f6950102570 2024-09-19T06:49:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f6955a7f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6950111100 con 0x7f6950100e20 2024-09-19T06:49:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f694ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6950102ef0 0x7f69500fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f694ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6950102ef0 0x7f69500fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40342/0 (socket says 172.21.15.103:40342) 2024-09-19T06:49:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f694ffff640 1 -- 172.21.15.103:0/2552322570 learned_addr learned my addr 172.21.15.103:0/2552322570 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f694f7fe640 1 --2- 172.21.15.103:0/2552322570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6950100e20 0x7f69500fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f6947fff640 1 --2- 172.21.15.103:0/2552322570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6950102570 0x7f69500fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f694ffff640 1 -- 172.21.15.103:0/2552322570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6950100e20 msgr2=0x7f69500fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f694ffff640 1 --2- 172.21.15.103:0/2552322570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6950100e20 0x7f69500fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f694ffff640 1 -- 172.21.15.103:0/2552322570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6950102570 msgr2=0x7f69500fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f694ffff640 1 --2- 172.21.15.103:0/2552322570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6950102570 0x7f69500fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f694ffff640 1 -- 172.21.15.103:0/2552322570 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69501a68c0 con 0x7f6950102ef0 2024-09-19T06:49:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f6947fff640 1 --2- 172.21.15.103:0/2552322570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6950102570 0x7f69500fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.724+0000 7f694ffff640 1 --2- 172.21.15.103:0/2552322570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6950102ef0 0x7f69500fd700 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f694802ae70 tx=0x7f694802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.725+0000 7f694d7fa640 1 -- 172.21.15.103:0/2552322570 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948034040 con 0x7f6950102ef0 2024-09-19T06:49:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.725+0000 7f6955a7f640 1 -- 172.21.15.103:0/2552322570 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69501a6b50 con 0x7f6950102ef0 2024-09-19T06:49:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.725+0000 7f694d7fa640 1 -- 172.21.15.103:0/2552322570 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6948030920 con 0x7f6950102ef0 2024-09-19T06:49:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.725+0000 7f694d7fa640 1 -- 172.21.15.103:0/2552322570 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69480047f0 con 0x7f6950102ef0 2024-09-19T06:49:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.725+0000 7f6955a7f640 1 -- 172.21.15.103:0/2552322570 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69501a7030 con 0x7f6950102ef0 2024-09-19T06:49:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.726+0000 7f6955a7f640 1 -- 172.21.15.103:0/2552322570 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6950101200 con 0x7f6950102ef0 2024-09-19T06:49:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.727+0000 7f694d7fa640 1 -- 172.21.15.103:0/2552322570 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f6948037020 con 0x7f6950102ef0 2024-09-19T06:49:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.727+0000 7f694d7fa640 1 --2- 172.21.15.103:0/2552322570 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f693c0a34d0 0x7f693c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.728+0000 7f694f7fe640 1 --2- 172.21.15.103:0/2552322570 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f693c0a34d0 0x7f693c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.728+0000 7f694d7fa640 1 -- 172.21.15.103:0/2552322570 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f69480f0de0 con 0x7f6950102ef0 2024-09-19T06:49:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.728+0000 7f694f7fe640 1 --2- 172.21.15.103:0/2552322570 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f693c0a34d0 0x7f693c0a5990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f694000abe0 tx=0x7f694000a2d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:27.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.732+0000 7f694d7fa640 1 -- 172.21.15.103:0/2552322570 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f69480bd690 con 0x7f6950102ef0 2024-09-19T06:49:27.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.897+0000 7f6955a7f640 1 -- 172.21.15.103:0/2552322570 --> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f695010a110 con 0x7f693c0a34d0 2024-09-19T06:49:27.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.903+0000 7f694d7fa640 1 -- 172.21.15.103:0/2552322570 <== mgr.15210 v2:172.21.15.103:6800/82882809 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257822 (secure 0 0 0) 0x7f694000fc20 con 0x7f693c0a34d0 2024-09-19T06:49:27.906 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:49:27.907 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T06:49:27.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.906+0000 7f6955a7f640 1 -- 172.21.15.103:0/2552322570 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f693c0a34d0 msgr2=0x7f693c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:27.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.906+0000 7f6955a7f640 1 --2- 172.21.15.103:0/2552322570 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f693c0a34d0 0x7f693c0a5990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f694000abe0 tx=0x7f694000a2d0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.906+0000 7f6955a7f640 1 -- 172.21.15.103:0/2552322570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6950102ef0 msgr2=0x7f69500fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:27.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.906+0000 7f6955a7f640 1 --2- 172.21.15.103:0/2552322570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6950102ef0 0x7f69500fd700 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f694802ae70 tx=0x7f694802f5d0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.906+0000 7f6955a7f640 1 -- 172.21.15.103:0/2552322570 shutdown_connections 2024-09-19T06:49:27.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.906+0000 7f6955a7f640 1 --2- 172.21.15.103:0/2552322570 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f693c0a34d0 0x7f693c0a5990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.906+0000 7f6955a7f640 1 --2- 172.21.15.103:0/2552322570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6950102ef0 0x7f69500fd700 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.906+0000 7f6955a7f640 1 --2- 172.21.15.103:0/2552322570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6950102570 0x7f69500fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.906+0000 7f6955a7f640 1 --2- 172.21.15.103:0/2552322570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6950100e20 0x7f69500fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.907+0000 7f6955a7f640 1 -- 172.21.15.103:0/2552322570 >> 172.21.15.103:0/2552322570 conn(0x7f69500fc7d0 msgr2=0x7f695006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:27.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.907+0000 7f6955a7f640 1 -- 172.21.15.103:0/2552322570 shutdown_connections 2024-09-19T06:49:27.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:27.907+0000 7f6955a7f640 1 -- 172.21.15.103:0/2552322570 wait complete. 2024-09-19T06:49:27.917 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:27 smithi103 ceph-mon[31158]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:27.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:27 smithi134 ceph-mon[35356]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:28.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:27 smithi174 ceph-mon[35637]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:28.693 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:28 smithi103 ceph-mon[31158]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:28.694 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:28 smithi103 ceph-mon[31158]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-19T06:49:28.695 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-19T06:49:25.934066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1393390,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1710,"num_read_kb":1286,"num_write":711,"num_write_kb":5330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":1181610,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1037,"ondisk_log_size":1037,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356268,"kb_used_data":15084,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124585556,"statfs":{"total":1151940427776,"available":1151575609344,"internally_reserved":0,"allocated":15446016,"data_stored":9537690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201469,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":450,"num_read_kb":491,"num_write":149,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.992567"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960112+0000","last_change":"2024-09-19T06:47:24.853118+0000","last_active":"2024-09-19T06:49:13.960112+0000","last_peered":"2024-09-19T06:49:13.960112+0000","last_clean":"2024-09-19T06:49:13.960112+0000","last_became_active":"2024-09-19T06:47:24.852801+0000","last_became_peered":"2024-09-19T06:47:24.852801+0000","last_unstale":"2024-09-19T06:49:13.960112+0000","last_undegraded":"2024-09-19T06:49:13.960112+0000","last_fullsized":"2024-09-19T06:49:13.960112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:16.729103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.957024+0000","last_change":"2024-09-19T06:47:24.841095+0000","last_active":"2024-09-19T06:49:13.957024+0000","last_peered":"2024-09-19T06:49:13.957024+0000","last_clean":"2024-09-19T06:49:13.957024+0000","last_became_active":"2024-09-19T06:47:24.840774+0000","last_became_peered":"2024-09-19T06:47:24.840774+0000","last_unstale":"2024-09-19T06:49:13.957024+0000","last_undegraded":"2024-09-19T06:49:13.957024+0000","last_fullsized":"2024-09-19T06:49:13.957024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:27.262766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973374+0000","last_change":"2024-09-19T06:47:24.848444+0000","last_active":"2024-09-19T06:49:13.973374+0000","last_peered":"2024-09-19T06:49:13.973374+0000","last_clean":"2024-09-19T06:49:13.973374+0000","last_became_active":"2024-09-19T06:47:24.847354+0000","last_became_peered":"2024-09-19T06:47:24.847354+0000","last_unstale":"2024-09-19T06:49:13.973374+0000","last_undegraded":"2024-09-19T06:49:13.973374+0000","last_fullsized":"2024-09-19T06:49:13.973374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:44.901265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.949178+0000","last_change":"2024-09-19T06:47:24.851915+0000","last_active":"2024-09-19T06:49:13.949178+0000","last_peered":"2024-09-19T06:49:13.949178+0000","last_clean":"2024-09-19T06:49:13.949178+0000","last_became_active":"2024-09-19T06:47:24.851600+0000","last_became_peered":"2024-09-19T06:47:24.851600+0000","last_unstale":"2024-09-19T06:49:13.949178+0000","last_undegraded":"2024-09-19T06:49:13.949178+0000","last_fullsized":"2024-09-19T06:49:13.949178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:52.424288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.950906+0000","last_change":"2024-09-19T06:47:24.843184+0000","last_active":"2024-09-19T06:49:13.950906+0000","last_peered":"2024-09-19T06:49:13.950906+0000","last_clean":"2024-09-19T06:49:13.950906+0000","last_became_active":"2024-09-19T06:47:24.842857+0000","last_became_peered":"2024-09-19T06:47:24.842857+0000","last_unstale":"2024-09-19T06:49:13.950906+0000","last_undegraded":"2024-09-19T06:49:13.950906+0000","last_fullsized":"2024-09-19T06:49:13.950906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:06.608699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969815+0000","last_change":"2024-09-19T06:47:24.846631+0000","last_active":"2024-09-19T06:49:13.969815+0000","last_peered":"2024-09-19T06:49:13.969815+0000","last_clean":"2024-09-19T06:49:13.969815+0000","last_became_active":"2024-09-19T06:47:24.846271+0000","last_became_peered":"2024-09-19T06:47:24.846271+0000","last_unstale":"2024-09-19T06:49:13.969815+0000","last_undegraded":"2024-09-19T06:49:13.969815+0000","last_fullsized":"2024-09-19T06:49:13.969815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.272273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.967306+0000","last_change":"2024-09-19T06:47:24.840392+0000","last_active":"2024-09-19T06:49:13.967306+0000","last_peered":"2024-09-19T06:49:13.967306+0000","last_clean":"2024-09-19T06:49:13.967306+0000","last_became_active":"2024-09-19T06:47:24.840092+0000","last_became_peered":"2024-09-19T06:47:24.840092+0000","last_unstale":"2024-09-19T06:49:13.967306+0000","last_undegraded":"2024-09-19T06:49:13.967306+0000","last_fullsized":"2024-09-19T06:49:13.967306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:25.579096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.967567+0000","last_change":"2024-09-19T06:47:24.843336+0000","last_active":"2024-09-19T06:49:13.967567+0000","last_peered":"2024-09-19T06:49:13.967567+0000","last_clean":"2024-09-19T06:49:13.967567+0000","last_became_active":"2024-09-19T06:47:24.842016+0000","last_became_peered":"2024-09-19T06:47:24.842016+0000","last_unstale":"2024-09-19T06:49:13.967567+0000","last_undegraded":"2024-09-19T06:49:13.967567+0000","last_fullsized":"2024-09-19T06:49:13.967567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:49.478359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.968002+0000","last_change":"2024-09-19T06:47:24.847841+0000","last_active":"2024-09-19T06:49:13.968002+0000","last_peered":"2024-09-19T06:49:13.968002+0000","last_clean":"2024-09-19T06:49:13.968002+0000","last_became_active":"2024-09-19T06:47:24.847492+0000","last_became_peered":"2024-09-19T06:47:24.847492+0000","last_unstale":"2024-09-19T06:49:13.968002+0000","last_undegraded":"2024-09-19T06:49:13.968002+0000","last_fullsized":"2024-09-19T06:49:13.968002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:59.229022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955381+0000","last_change":"2024-09-19T06:47:24.843773+0000","last_active":"2024-09-19T06:49:13.955381+0000","last_peered":"2024-09-19T06:49:13.955381+0000","last_clean":"2024-09-19T06:49:13.955381+0000","last_became_active":"2024-09-19T06:47:24.843409+0000","last_became_peered":"2024-09-19T06:47:24.843409+0000","last_unstale":"2024-09-19T06:49:13.955381+0000","last_undegraded":"2024-09-19T06:49:13.955381+0000","last_fullsized":"2024-09-19T06:49:13.955381+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:22.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969613+0000","last_change":"2024-09-19T06:47:24.846637+0000","last_active":"2024-09-19T06:49:13.969613+0000","last_peered":"2024-09-19T06:49:13.969613+0000","last_clean":"2024-09-19T06:49:13.969613+0000","last_became_active":"2024-09-19T06:47:24.846337+0000","last_became_peered":"2024-09-19T06:47:24.846337+0000","last_unstale":"2024-09-19T06:49:13.969613+0000","last_undegraded":"2024-09-19T06:49:13.969613+0000","last_fullsized":"2024-09-19T06:49:13.969613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:23:35.869485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.946579+0000","last_change":"2024-09-19T06:47:24.851399+0000","last_active":"2024-09-19T06:49:13.946579+0000","last_peered":"2024-09-19T06:49:13.946579+0000","last_clean":"2024-09-19T06:49:13.946579+0000","last_became_active":"2024-09-19T06:47:24.851090+0000","last_became_peered":"2024-09-19T06:47:24.851090+0000","last_unstale":"2024-09-19T06:49:13.946579+0000","last_undegraded":"2024-09-19T06:49:13.946579+0000","last_fullsized":"2024-09-19T06:49:13.946579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:52.839956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970074+0000","last_change":"2024-09-19T06:47:24.833858+0000","last_active":"2024-09-19T06:49:13.970074+0000","last_peered":"2024-09-19T06:49:13.970074+0000","last_clean":"2024-09-19T06:49:13.970074+0000","last_became_active":"2024-09-19T06:47:24.833616+0000","last_became_peered":"2024-09-19T06:47:24.833616+0000","last_unstale":"2024-09-19T06:49:13.970074+0000","last_undegraded":"2024-09-19T06:49:13.970074+0000","last_fullsized":"2024-09-19T06:49:13.970074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:54.918144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966731+0000","last_change":"2024-09-19T06:47:24.841030+0000","last_active":"2024-09-19T06:49:13.966731+0000","last_peered":"2024-09-19T06:49:13.966731+0000","last_clean":"2024-09-19T06:49:13.966731+0000","last_became_active":"2024-09-19T06:47:24.839631+0000","last_became_peered":"2024-09-19T06:47:24.839631+0000","last_unstale":"2024-09-19T06:49:13.966731+0000","last_undegraded":"2024-09-19T06:49:13.966731+0000","last_fullsized":"2024-09-19T06:49:13.966731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:21.764812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970278+0000","last_change":"2024-09-19T06:47:24.849281+0000","last_active":"2024-09-19T06:49:13.970278+0000","last_peered":"2024-09-19T06:49:13.970278+0000","last_clean":"2024-09-19T06:49:13.970278+0000","last_became_active":"2024-09-19T06:47:24.848977+0000","last_became_peered":"2024-09-19T06:47:24.848977+0000","last_unstale":"2024-09-19T06:49:13.970278+0000","last_undegraded":"2024-09-19T06:49:13.970278+0000","last_fullsized":"2024-09-19T06:49:13.970278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:36.693758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973886+0000","last_change":"2024-09-19T06:47:24.843619+0000","last_active":"2024-09-19T06:49:13.973886+0000","last_peered":"2024-09-19T06:49:13.973886+0000","last_clean":"2024-09-19T06:49:13.973886+0000","last_became_active":"2024-09-19T06:47:24.843323+0000","last_became_peered":"2024-09-19T06:47:24.843323+0000","last_unstale":"2024-09-19T06:49:13.973886+0000","last_undegraded":"2024-09-19T06:49:13.973886+0000","last_fullsized":"2024-09-19T06:49:13.973886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:52.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960333+0000","last_change":"2024-09-19T06:47:24.853165+0000","last_active":"2024-09-19T06:49:13.960333+0000","last_peered":"2024-09-19T06:49:13.960333+0000","last_clean":"2024-09-19T06:49:13.960333+0000","last_became_active":"2024-09-19T06:47:24.852864+0000","last_became_peered":"2024-09-19T06:47:24.852864+0000","last_unstale":"2024-09-19T06:49:13.960333+0000","last_undegraded":"2024-09-19T06:49:13.960333+0000","last_fullsized":"2024-09-19T06:49:13.960333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:38.803478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.975021+0000","last_change":"2024-09-19T06:47:24.838151+0000","last_active":"2024-09-19T06:49:13.975021+0000","last_peered":"2024-09-19T06:49:13.975021+0000","last_clean":"2024-09-19T06:49:13.975021+0000","last_became_active":"2024-09-19T06:47:24.837855+0000","last_became_peered":"2024-09-19T06:47:24.837855+0000","last_unstale":"2024-09-19T06:49:13.975021+0000","last_undegraded":"2024-09-19T06:49:13.975021+0000","last_fullsized":"2024-09-19T06:49:13.975021+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:16.675557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.964466+0000","last_change":"2024-09-19T06:47:24.843941+0000","last_active":"2024-09-19T06:49:13.964466+0000","last_peered":"2024-09-19T06:49:13.964466+0000","last_clean":"2024-09-19T06:49:13.964466+0000","last_became_active":"2024-09-19T06:47:24.843551+0000","last_became_peered":"2024-09-19T06:47:24.843551+0000","last_unstale":"2024-09-19T06:49:13.964466+0000","last_undegraded":"2024-09-19T06:49:13.964466+0000","last_fullsized":"2024-09-19T06:49:13.964466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:00:49.388265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.954208+0000","last_change":"2024-09-19T06:47:40.836228+0000","last_active":"2024-09-19T06:49:13.954208+0000","last_peered":"2024-09-19T06:49:13.954208+0000","last_clean":"2024-09-19T06:49:13.954208+0000","last_became_active":"2024-09-19T06:47:40.835818+0000","last_became_peered":"2024-09-19T06:47:40.835818+0000","last_unstale":"2024-09-19T06:49:13.954208+0000","last_undegraded":"2024-09-19T06:49:13.954208+0000","last_fullsized":"2024-09-19T06:49:13.954208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:45.341160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972919+0000","last_change":"2024-09-19T06:47:24.843746+0000","last_active":"2024-09-19T06:49:13.972919+0000","last_peered":"2024-09-19T06:49:13.972919+0000","last_clean":"2024-09-19T06:49:13.972919+0000","last_became_active":"2024-09-19T06:47:24.841495+0000","last_became_peered":"2024-09-19T06:47:24.841495+0000","last_unstale":"2024-09-19T06:49:13.972919+0000","last_undegraded":"2024-09-19T06:49:13.972919+0000","last_fullsized":"2024-09-19T06:49:13.972919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:06:29.995595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.974565+0000","last_change":"2024-09-19T06:47:24.844747+0000","last_active":"2024-09-19T06:49:13.974565+0000","last_peered":"2024-09-19T06:49:13.974565+0000","last_clean":"2024-09-19T06:49:13.974565+0000","last_became_active":"2024-09-19T06:47:24.844425+0000","last_became_peered":"2024-09-19T06:47:24.844425+0000","last_unstale":"2024-09-19T06:49:13.974565+0000","last_undegraded":"2024-09-19T06:49:13.974565+0000","last_fullsized":"2024-09-19T06:49:13.974565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:10.309044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963556+0000","last_change":"2024-09-19T06:47:24.848227+0000","last_active":"2024-09-19T06:49:13.963556+0000","last_peered":"2024-09-19T06:49:13.963556+0000","last_clean":"2024-09-19T06:49:13.963556+0000","last_became_active":"2024-09-19T06:47:24.847914+0000","last_became_peered":"2024-09-19T06:47:24.847914+0000","last_unstale":"2024-09-19T06:49:13.963556+0000","last_undegraded":"2024-09-19T06:49:13.963556+0000","last_fullsized":"2024-09-19T06:49:13.963556+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:47.281613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.968012+0000","last_change":"2024-09-19T06:47:24.844275+0000","last_active":"2024-09-19T06:49:13.968012+0000","last_peered":"2024-09-19T06:49:13.968012+0000","last_clean":"2024-09-19T06:49:13.968012+0000","last_became_active":"2024-09-19T06:47:24.841442+0000","last_became_peered":"2024-09-19T06:47:24.841442+0000","last_unstale":"2024-09-19T06:49:13.968012+0000","last_undegraded":"2024-09-19T06:49:13.968012+0000","last_fullsized":"2024-09-19T06:49:13.968012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:30.198655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963003+0000","last_change":"2024-09-19T06:47:24.846301+0000","last_active":"2024-09-19T06:49:13.963003+0000","last_peered":"2024-09-19T06:49:13.963003+0000","last_clean":"2024-09-19T06:49:13.963003+0000","last_became_active":"2024-09-19T06:47:24.845970+0000","last_became_peered":"2024-09-19T06:47:24.845970+0000","last_unstale":"2024-09-19T06:49:13.963003+0000","last_undegraded":"2024-09-19T06:49:13.963003+0000","last_fullsized":"2024-09-19T06:49:13.963003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:13.245536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973777+0000","last_change":"2024-09-19T06:47:24.844444+0000","last_active":"2024-09-19T06:49:13.973777+0000","last_peered":"2024-09-19T06:49:13.973777+0000","last_clean":"2024-09-19T06:49:13.973777+0000","last_became_active":"2024-09-19T06:47:24.844141+0000","last_became_peered":"2024-09-19T06:47:24.844141+0000","last_unstale":"2024-09-19T06:49:13.973777+0000","last_undegraded":"2024-09-19T06:49:13.973777+0000","last_fullsized":"2024-09-19T06:49:13.973777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:44.739242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955255+0000","last_change":"2024-09-19T06:47:24.845633+0000","last_active":"2024-09-19T06:49:13.955255+0000","last_peered":"2024-09-19T06:49:13.955255+0000","last_clean":"2024-09-19T06:49:13.955255+0000","last_became_active":"2024-09-19T06:47:24.845319+0000","last_became_peered":"2024-09-19T06:47:24.845319+0000","last_unstale":"2024-09-19T06:49:13.955255+0000","last_undegraded":"2024-09-19T06:49:13.955255+0000","last_fullsized":"2024-09-19T06:49:13.955255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:00.820006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959353+0000","last_change":"2024-09-19T06:47:24.846457+0000","last_active":"2024-09-19T06:49:13.959353+0000","last_peered":"2024-09-19T06:49:13.959353+0000","last_clean":"2024-09-19T06:49:13.959353+0000","last_became_active":"2024-09-19T06:47:24.846130+0000","last_became_peered":"2024-09-19T06:47:24.846130+0000","last_unstale":"2024-09-19T06:49:13.959353+0000","last_undegraded":"2024-09-19T06:49:13.959353+0000","last_fullsized":"2024-09-19T06:49:13.959353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:42.179145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969751+0000","last_change":"2024-09-19T06:47:24.843141+0000","last_active":"2024-09-19T06:49:13.969751+0000","last_peered":"2024-09-19T06:49:13.969751+0000","last_clean":"2024-09-19T06:49:13.969751+0000","last_became_active":"2024-09-19T06:47:24.842810+0000","last_became_peered":"2024-09-19T06:47:24.842810+0000","last_unstale":"2024-09-19T06:49:13.969751+0000","last_undegraded":"2024-09-19T06:49:13.969751+0000","last_fullsized":"2024-09-19T06:49:13.969751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:43.440453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.965342+0000","last_change":"2024-09-19T06:47:24.846171+0000","last_active":"2024-09-19T06:49:13.965342+0000","last_peered":"2024-09-19T06:49:13.965342+0000","last_clean":"2024-09-19T06:49:13.965342+0000","last_became_active":"2024-09-19T06:47:24.845799+0000","last_became_peered":"2024-09-19T06:47:24.845799+0000","last_unstale":"2024-09-19T06:49:13.965342+0000","last_undegraded":"2024-09-19T06:49:13.965342+0000","last_fullsized":"2024-09-19T06:49:13.965342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:23.103061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972106+0000","last_change":"2024-09-19T06:47:24.848723+0000","last_active":"2024-09-19T06:49:13.972106+0000","last_peered":"2024-09-19T06:49:13.972106+0000","last_clean":"2024-09-19T06:49:13.972106+0000","last_became_active":"2024-09-19T06:47:24.848367+0000","last_became_peered":"2024-09-19T06:47:24.848367+0000","last_unstale":"2024-09-19T06:49:13.972106+0000","last_undegraded":"2024-09-19T06:49:13.972106+0000","last_fullsized":"2024-09-19T06:49:13.972106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:15.985293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960543+0000","last_change":"2024-09-19T06:47:24.841379+0000","last_active":"2024-09-19T06:49:13.960543+0000","last_peered":"2024-09-19T06:49:13.960543+0000","last_clean":"2024-09-19T06:49:13.960543+0000","last_became_active":"2024-09-19T06:47:24.841041+0000","last_became_peered":"2024-09-19T06:47:24.841041+0000","last_unstale":"2024-09-19T06:49:13.960543+0000","last_undegraded":"2024-09-19T06:49:13.960543+0000","last_fullsized":"2024-09-19T06:49:13.960543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:11.799022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955643+0000","last_change":"2024-09-19T06:47:24.849508+0000","last_active":"2024-09-19T06:49:13.955643+0000","last_peered":"2024-09-19T06:49:13.955643+0000","last_clean":"2024-09-19T06:49:13.955643+0000","last_became_active":"2024-09-19T06:47:24.849196+0000","last_became_peered":"2024-09-19T06:47:24.849196+0000","last_unstale":"2024-09-19T06:49:13.955643+0000","last_undegraded":"2024-09-19T06:49:13.955643+0000","last_fullsized":"2024-09-19T06:49:13.955643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:35.079720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.617944+0000","last_change":"2024-09-19T06:47:22.829848+0000","last_active":"2024-09-19T06:49:23.617944+0000","last_peered":"2024-09-19T06:49:23.617944+0000","last_clean":"2024-09-19T06:49:23.617944+0000","last_became_active":"2024-09-19T06:47:22.829511+0000","last_became_peered":"2024-09-19T06:47:22.829511+0000","last_unstale":"2024-09-19T06:49:23.617944+0000","last_undegraded":"2024-09-19T06:49:23.617944+0000","last_fullsized":"2024-09-19T06:49:23.617944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:32.665252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971999+0000","last_change":"2024-09-19T06:47:24.843245+0000","last_active":"2024-09-19T06:49:13.971999+0000","last_peered":"2024-09-19T06:49:13.971999+0000","last_clean":"2024-09-19T06:49:13.971999+0000","last_became_active":"2024-09-19T06:47:24.842885+0000","last_became_peered":"2024-09-19T06:47:24.842885+0000","last_unstale":"2024-09-19T06:49:13.971999+0000","last_undegraded":"2024-09-19T06:49:13.971999+0000","last_fullsized":"2024-09-19T06:49:13.971999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:26.229003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.926577+0000","last_change":"2024-09-19T06:47:22.831456+0000","last_active":"2024-09-19T06:49:13.926577+0000","last_peered":"2024-09-19T06:49:13.926577+0000","last_clean":"2024-09-19T06:49:13.926577+0000","last_became_active":"2024-09-19T06:47:22.831132+0000","last_became_peered":"2024-09-19T06:47:22.831132+0000","last_unstale":"2024-09-19T06:49:13.926577+0000","last_undegraded":"2024-09-19T06:49:13.926577+0000","last_fullsized":"2024-09-19T06:49:13.926577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:49.867713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932598+0000","last_change":"2024-09-19T06:47:22.827742+0000","last_active":"2024-09-19T06:49:13.932598+0000","last_peered":"2024-09-19T06:49:13.932598+0000","last_clean":"2024-09-19T06:49:13.932598+0000","last_became_active":"2024-09-19T06:47:22.827371+0000","last_became_peered":"2024-09-19T06:47:22.827371+0000","last_unstale":"2024-09-19T06:49:13.932598+0000","last_undegraded":"2024-09-19T06:49:13.932598+0000","last_fullsized":"2024-09-19T06:49:13.932598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:35.153129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963305+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:49:13.963305+0000","last_peered":"2024-09-19T06:49:13.963305+0000","last_clean":"2024-09-19T06:49:13.963305+0000","last_became_active":"2024-09-19T06:47:24.841693+0000","last_became_peered":"2024-09-19T06:47:24.841693+0000","last_unstale":"2024-09-19T06:49:13.963305+0000","last_undegraded":"2024-09-19T06:49:13.963305+0000","last_fullsized":"2024-09-19T06:49:13.963305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:27.828296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.917388+0000","last_change":"2024-09-19T06:47:22.832531+0000","last_active":"2024-09-19T06:49:13.917388+0000","last_peered":"2024-09-19T06:49:13.917388+0000","last_clean":"2024-09-19T06:49:13.917388+0000","last_became_active":"2024-09-19T06:47:22.832213+0000","last_became_peered":"2024-09-19T06:47:22.832213+0000","last_unstale":"2024-09-19T06:49:13.917388+0000","last_undegraded":"2024-09-19T06:49:13.917388+0000","last_fullsized":"2024-09-19T06:49:13.917388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:46:03.168406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970720+0000","last_change":"2024-09-19T06:47:24.837690+0000","last_active":"2024-09-19T06:49:13.970720+0000","last_peered":"2024-09-19T06:49:13.970720+0000","last_clean":"2024-09-19T06:49:13.970720+0000","last_became_active":"2024-09-19T06:47:24.837389+0000","last_became_peered":"2024-09-19T06:47:24.837389+0000","last_unstale":"2024-09-19T06:49:13.970720+0000","last_undegraded":"2024-09-19T06:49:13.970720+0000","last_fullsized":"2024-09-19T06:49:13.970720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:20.653832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":248,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:24.618359+0000","last_change":"2024-09-19T06:47:22.833808+0000","last_active":"2024-09-19T06:49:24.618359+0000","last_peered":"2024-09-19T06:49:24.618359+0000","last_clean":"2024-09-19T06:49:24.618359+0000","last_became_active":"2024-09-19T06:47:22.832468+0000","last_became_peered":"2024-09-19T06:47:22.832468+0000","last_unstale":"2024-09-19T06:49:24.618359+0000","last_undegraded":"2024-09-19T06:49:24.618359+0000","last_fullsized":"2024-09-19T06:49:24.618359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:39:31.131321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":218,"num_read_kb":218,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973091+0000","last_change":"2024-09-19T06:47:24.840928+0000","last_active":"2024-09-19T06:49:13.973091+0000","last_peered":"2024-09-19T06:49:13.973091+0000","last_clean":"2024-09-19T06:49:13.973091+0000","last_became_active":"2024-09-19T06:47:24.839928+0000","last_became_peered":"2024-09-19T06:47:24.839928+0000","last_unstale":"2024-09-19T06:49:13.973091+0000","last_undegraded":"2024-09-19T06:49:13.973091+0000","last_fullsized":"2024-09-19T06:49:13.973091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:29.486003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.928845+0000","last_change":"2024-09-19T06:47:22.833310+0000","last_active":"2024-09-19T06:49:13.928845+0000","last_peered":"2024-09-19T06:49:13.928845+0000","last_clean":"2024-09-19T06:49:13.928845+0000","last_became_active":"2024-09-19T06:47:22.832976+0000","last_became_peered":"2024-09-19T06:47:22.832976+0000","last_unstale":"2024-09-19T06:49:13.928845+0000","last_undegraded":"2024-09-19T06:49:13.928845+0000","last_fullsized":"2024-09-19T06:49:13.928845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:47.604853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969853+0000","last_change":"2024-09-19T06:47:24.848687+0000","last_active":"2024-09-19T06:49:13.969853+0000","last_peered":"2024-09-19T06:49:13.969853+0000","last_clean":"2024-09-19T06:49:13.969853+0000","last_became_active":"2024-09-19T06:47:24.848352+0000","last_became_peered":"2024-09-19T06:47:24.848352+0000","last_unstale":"2024-09-19T06:49:13.969853+0000","last_undegraded":"2024-09-19T06:49:13.969853+0000","last_fullsized":"2024-09-19T06:49:13.969853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:54.459206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.917489+0000","last_change":"2024-09-19T06:47:22.851479+0000","last_active":"2024-09-19T06:49:13.917489+0000","last_peered":"2024-09-19T06:49:13.917489+0000","last_clean":"2024-09-19T06:49:13.917489+0000","last_became_active":"2024-09-19T06:47:22.851125+0000","last_became_peered":"2024-09-19T06:47:22.851125+0000","last_unstale":"2024-09-19T06:49:13.917489+0000","last_undegraded":"2024-09-19T06:49:13.917489+0000","last_fullsized":"2024-09-19T06:49:13.917489+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:42.954034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973229+0000","last_change":"2024-09-19T06:47:24.851007+0000","last_active":"2024-09-19T06:49:13.973229+0000","last_peered":"2024-09-19T06:49:13.973229+0000","last_clean":"2024-09-19T06:49:13.973229+0000","last_became_active":"2024-09-19T06:47:24.850465+0000","last_became_peered":"2024-09-19T06:47:24.850465+0000","last_unstale":"2024-09-19T06:49:13.973229+0000","last_undegraded":"2024-09-19T06:49:13.973229+0000","last_fullsized":"2024-09-19T06:49:13.973229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:10.824672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'39","reported_seq":295,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:24.618888+0000","last_change":"2024-09-19T06:47:22.834893+0000","last_active":"2024-09-19T06:49:24.618888+0000","last_peered":"2024-09-19T06:49:24.618888+0000","last_clean":"2024-09-19T06:49:24.618888+0000","last_became_active":"2024-09-19T06:47:22.834521+0000","last_became_peered":"2024-09-19T06:47:22.834521+0000","last_unstale":"2024-09-19T06:49:24.618888+0000","last_undegraded":"2024-09-19T06:49:24.618888+0000","last_fullsized":"2024-09-19T06:49:24.618888+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:29.742284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70398,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":232,"num_read_kb":185,"num_write":41,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.957411+0000","last_change":"2024-09-19T06:47:24.848191+0000","last_active":"2024-09-19T06:49:13.957411+0000","last_peered":"2024-09-19T06:49:13.957411+0000","last_clean":"2024-09-19T06:49:13.957411+0000","last_became_active":"2024-09-19T06:47:24.847889+0000","last_became_peered":"2024-09-19T06:47:24.847889+0000","last_unstale":"2024-09-19T06:49:13.957411+0000","last_undegraded":"2024-09-19T06:49:13.957411+0000","last_fullsized":"2024-09-19T06:49:13.957411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:32.574360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.924532+0000","last_change":"2024-09-19T06:47:22.832005+0000","last_active":"2024-09-19T06:49:13.924532+0000","last_peered":"2024-09-19T06:49:13.924532+0000","last_clean":"2024-09-19T06:49:13.924532+0000","last_became_active":"2024-09-19T06:47:22.831342+0000","last_became_peered":"2024-09-19T06:47:22.831342+0000","last_unstale":"2024-09-19T06:49:13.924532+0000","last_undegraded":"2024-09-19T06:49:13.924532+0000","last_fullsized":"2024-09-19T06:49:13.924532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:53.444748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.964849+0000","last_change":"2024-09-19T06:47:24.843328+0000","last_active":"2024-09-19T06:49:13.964849+0000","last_peered":"2024-09-19T06:49:13.964849+0000","last_clean":"2024-09-19T06:49:13.964849+0000","last_became_active":"2024-09-19T06:47:24.842344+0000","last_became_peered":"2024-09-19T06:47:24.842344+0000","last_unstale":"2024-09-19T06:49:13.964849+0000","last_undegraded":"2024-09-19T06:49:13.964849+0000","last_fullsized":"2024-09-19T06:49:13.964849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:11.247819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.926463+0000","last_change":"2024-09-19T06:47:40.844086+0000","last_active":"2024-09-19T06:49:13.926463+0000","last_peered":"2024-09-19T06:49:13.926463+0000","last_clean":"2024-09-19T06:49:13.926463+0000","last_became_active":"2024-09-19T06:47:40.835015+0000","last_became_peered":"2024-09-19T06:47:40.835015+0000","last_unstale":"2024-09-19T06:49:13.926463+0000","last_undegraded":"2024-09-19T06:49:13.926463+0000","last_fullsized":"2024-09-19T06:49:13.926463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:09.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972972+0000","last_change":"2024-09-19T06:47:40.836466+0000","last_active":"2024-09-19T06:49:13.972972+0000","last_peered":"2024-09-19T06:49:13.972972+0000","last_clean":"2024-09-19T06:49:13.972972+0000","last_became_active":"2024-09-19T06:47:40.836237+0000","last_became_peered":"2024-09-19T06:47:40.836237+0000","last_unstale":"2024-09-19T06:49:13.972972+0000","last_undegraded":"2024-09-19T06:49:13.972972+0000","last_fullsized":"2024-09-19T06:49:13.972972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:00.094685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'11","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.617882+0000","last_change":"2024-09-19T06:47:22.830359+0000","last_active":"2024-09-19T06:49:23.617882+0000","last_peered":"2024-09-19T06:49:23.617882+0000","last_clean":"2024-09-19T06:49:23.617882+0000","last_became_active":"2024-09-19T06:47:22.829960+0000","last_became_peered":"2024-09-19T06:47:22.829960+0000","last_unstale":"2024-09-19T06:49:23.617882+0000","last_undegraded":"2024-09-19T06:49:23.617882+0000","last_fullsized":"2024-09-19T06:49:23.617882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:55.998243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2609,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":7,"num_write":11,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971872+0000","last_change":"2024-09-19T06:47:24.836668+0000","last_active":"2024-09-19T06:49:13.971872+0000","last_peered":"2024-09-19T06:49:13.971872+0000","last_clean":"2024-09-19T06:49:13.971872+0000","last_became_active":"2024-09-19T06:47:24.836393+0000","last_became_peered":"2024-09-19T06:47:24.836393+0000","last_unstale":"2024-09-19T06:49:13.971872+0000","last_undegraded":"2024-09-19T06:49:13.971872+0000","last_fullsized":"2024-09-19T06:49:13.971872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:26.248948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":66,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.931451+0000","last_change":"2024-09-19T06:47:40.858323+0000","last_active":"2024-09-19T06:49:13.931451+0000","last_peered":"2024-09-19T06:49:13.931451+0000","last_clean":"2024-09-19T06:49:13.931451+0000","last_became_active":"2024-09-19T06:47:40.840802+0000","last_became_peered":"2024-09-19T06:47:40.840802+0000","last_unstale":"2024-09-19T06:49:13.931451+0000","last_undegraded":"2024-09-19T06:49:13.931451+0000","last_fullsized":"2024-09-19T06:49:13.931451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:14.666916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970280+0000","last_change":"2024-09-19T06:47:24.843681+0000","last_active":"2024-09-19T06:49:13.970280+0000","last_peered":"2024-09-19T06:49:13.970280+0000","last_clean":"2024-09-19T06:49:13.970280+0000","last_became_active":"2024-09-19T06:47:24.841228+0000","last_became_peered":"2024-09-19T06:47:24.841228+0000","last_unstale":"2024-09-19T06:49:13.970280+0000","last_undegraded":"2024-09-19T06:49:13.970280+0000","last_fullsized":"2024-09-19T06:49:13.970280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:56.418962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.933661+0000","last_change":"2024-09-19T06:47:22.833893+0000","last_active":"2024-09-19T06:49:13.933661+0000","last_peered":"2024-09-19T06:49:13.933661+0000","last_clean":"2024-09-19T06:49:13.933661+0000","last_became_active":"2024-09-19T06:47:22.833217+0000","last_became_peered":"2024-09-19T06:47:22.833217+0000","last_unstale":"2024-09-19T06:49:13.933661+0000","last_undegraded":"2024-09-19T06:49:13.933661+0000","last_fullsized":"2024-09-19T06:49:13.933661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:19.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955096+0000","last_change":"2024-09-19T06:47:24.848399+0000","last_active":"2024-09-19T06:49:13.955096+0000","last_peered":"2024-09-19T06:49:13.955096+0000","last_clean":"2024-09-19T06:49:13.955096+0000","last_became_active":"2024-09-19T06:47:24.848081+0000","last_became_peered":"2024-09-19T06:47:24.848081+0000","last_unstale":"2024-09-19T06:49:13.955096+0000","last_undegraded":"2024-09-19T06:49:13.955096+0000","last_fullsized":"2024-09-19T06:49:13.955096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:44.292939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.931968+0000","last_change":"2024-09-19T06:47:40.860955+0000","last_active":"2024-09-19T06:49:13.931968+0000","last_peered":"2024-09-19T06:49:13.931968+0000","last_clean":"2024-09-19T06:49:13.931968+0000","last_became_active":"2024-09-19T06:47:40.835126+0000","last_became_peered":"2024-09-19T06:47:40.835126+0000","last_unstale":"2024-09-19T06:49:13.931968+0000","last_undegraded":"2024-09-19T06:49:13.931968+0000","last_fullsized":"2024-09-19T06:49:13.931968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:02:26.763229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971391+0000","last_change":"2024-09-19T06:47:24.849680+0000","last_active":"2024-09-19T06:49:13.971391+0000","last_peered":"2024-09-19T06:49:13.971391+0000","last_clean":"2024-09-19T06:49:13.971391+0000","last_became_active":"2024-09-19T06:47:24.849323+0000","last_became_peered":"2024-09-19T06:47:24.849323+0000","last_unstale":"2024-09-19T06:49:13.971391+0000","last_undegraded":"2024-09-19T06:49:13.971391+0000","last_fullsized":"2024-09-19T06:49:13.971391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:59.763503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'67","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:24.414577+0000","last_change":"2024-09-19T06:47:22.831689+0000","last_active":"2024-09-19T06:49:24.414577+0000","last_peered":"2024-09-19T06:49:24.414577+0000","last_clean":"2024-09-19T06:49:24.414577+0000","last_became_active":"2024-09-19T06:47:22.829283+0000","last_became_peered":"2024-09-19T06:47:22.829283+0000","last_unstale":"2024-09-19T06:49:24.414577+0000","last_undegraded":"2024-09-19T06:49:24.414577+0000","last_fullsized":"2024-09-19T06:49:24.414577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:37.109542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971792+0000","last_change":"2024-09-19T06:47:24.842793+0000","last_active":"2024-09-19T06:49:13.971792+0000","last_peered":"2024-09-19T06:49:13.971792+0000","last_clean":"2024-09-19T06:49:13.971792+0000","last_became_active":"2024-09-19T06:47:24.842474+0000","last_became_peered":"2024-09-19T06:47:24.842474+0000","last_unstale":"2024-09-19T06:49:13.971792+0000","last_undegraded":"2024-09-19T06:49:13.971792+0000","last_fullsized":"2024-09-19T06:49:13.971792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:23.110736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'10","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.617464+0000","last_change":"2024-09-19T06:47:23.047356+0000","last_active":"2024-09-19T06:49:23.617464+0000","last_peered":"2024-09-19T06:49:23.617464+0000","last_clean":"2024-09-19T06:49:23.617464+0000","last_became_active":"2024-09-19T06:47:23.046941+0000","last_became_peered":"2024-09-19T06:47:23.046941+0000","last_unstale":"2024-09-19T06:49:23.617464+0000","last_undegraded":"2024-09-19T06:49:23.617464+0000","last_fullsized":"2024-09-19T06:49:23.617464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:43.490952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":5,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971243+0000","last_change":"2024-09-19T06:47:24.851025+0000","last_active":"2024-09-19T06:49:13.971243+0000","last_peered":"2024-09-19T06:49:13.971243+0000","last_clean":"2024-09-19T06:49:13.971243+0000","last_became_active":"2024-09-19T06:47:24.850681+0000","last_became_peered":"2024-09-19T06:47:24.850681+0000","last_unstale":"2024-09-19T06:49:13.971243+0000","last_undegraded":"2024-09-19T06:49:13.971243+0000","last_fullsized":"2024-09-19T06:49:13.971243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:46.884747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"83'148","reported_seq":392,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:16.044071+0000","last_change":"2024-09-19T06:46:13.927379+0000","last_active":"2024-09-19T06:49:16.044071+0000","last_peered":"2024-09-19T06:49:16.044071+0000","last_clean":"2024-09-19T06:49:16.044071+0000","last_became_active":"2024-09-19T06:46:13.926926+0000","last_became_peered":"2024-09-19T06:46:13.926926+0000","last_unstale":"2024-09-19T06:49:16.044071+0000","last_undegraded":"2024-09-19T06:49:16.044071+0000","last_fullsized":"2024-09-19T06:49:16.044071+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_clean_scrub_stamp":"2024-09-19T06:42:52.051335+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:29.542128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.927403+0000","last_change":"2024-09-19T06:47:22.830387+0000","last_active":"2024-09-19T06:49:13.927403+0000","last_peered":"2024-09-19T06:49:13.927403+0000","last_clean":"2024-09-19T06:49:13.927403+0000","last_became_active":"2024-09-19T06:47:22.829970+0000","last_became_peered":"2024-09-19T06:47:22.829970+0000","last_unstale":"2024-09-19T06:49:13.927403+0000","last_undegraded":"2024-09-19T06:49:13.927403+0000","last_fullsized":"2024-09-19T06:49:13.927403+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:51.385456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.974671+0000","last_change":"2024-09-19T06:47:24.843752+0000","last_active":"2024-09-19T06:49:13.974671+0000","last_peered":"2024-09-19T06:49:13.974671+0000","last_clean":"2024-09-19T06:49:13.974671+0000","last_became_active":"2024-09-19T06:47:24.842631+0000","last_became_peered":"2024-09-19T06:47:24.842631+0000","last_unstale":"2024-09-19T06:49:13.974671+0000","last_undegraded":"2024-09-19T06:49:13.974671+0000","last_fullsized":"2024-09-19T06:49:13.974671+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:19.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.928732+0000","last_change":"2024-09-19T06:47:22.832647+0000","last_active":"2024-09-19T06:49:13.928732+0000","last_peered":"2024-09-19T06:49:13.928732+0000","last_clean":"2024-09-19T06:49:13.928732+0000","last_became_active":"2024-09-19T06:47:22.831774+0000","last_became_peered":"2024-09-19T06:47:22.831774+0000","last_unstale":"2024-09-19T06:49:13.928732+0000","last_undegraded":"2024-09-19T06:49:13.928732+0000","last_fullsized":"2024-09-19T06:49:13.928732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:57.824610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.948364+0000","last_change":"2024-09-19T06:47:24.844301+0000","last_active":"2024-09-19T06:49:13.948364+0000","last_peered":"2024-09-19T06:49:13.948364+0000","last_clean":"2024-09-19T06:49:13.948364+0000","last_became_active":"2024-09-19T06:47:24.839748+0000","last_became_peered":"2024-09-19T06:47:24.839748+0000","last_unstale":"2024-09-19T06:49:13.948364+0000","last_undegraded":"2024-09-19T06:49:13.948364+0000","last_fullsized":"2024-09-19T06:49:13.948364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:46.407693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":140,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.617287+0000","last_change":"2024-09-19T06:47:22.851008+0000","last_active":"2024-09-19T06:49:23.617287+0000","last_peered":"2024-09-19T06:49:23.617287+0000","last_clean":"2024-09-19T06:49:23.617287+0000","last_became_active":"2024-09-19T06:47:22.850601+0000","last_became_peered":"2024-09-19T06:47:22.850601+0000","last_unstale":"2024-09-19T06:49:23.617287+0000","last_undegraded":"2024-09-19T06:49:23.617287+0000","last_fullsized":"2024-09-19T06:49:23.617287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:04.610277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970407+0000","last_change":"2024-09-19T06:47:24.839684+0000","last_active":"2024-09-19T06:49:13.970407+0000","last_peered":"2024-09-19T06:49:13.970407+0000","last_clean":"2024-09-19T06:49:13.970407+0000","last_became_active":"2024-09-19T06:47:24.839126+0000","last_became_peered":"2024-09-19T06:47:24.839126+0000","last_unstale":"2024-09-19T06:49:13.970407+0000","last_undegraded":"2024-09-19T06:49:13.970407+0000","last_fullsized":"2024-09-19T06:49:13.970407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:09.458535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":140,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.325749+0000","last_change":"2024-09-19T06:47:22.831819+0000","last_active":"2024-09-19T06:49:23.325749+0000","last_peered":"2024-09-19T06:49:23.325749+0000","last_clean":"2024-09-19T06:49:23.325749+0000","last_became_active":"2024-09-19T06:47:22.831332+0000","last_became_peered":"2024-09-19T06:47:22.831332+0000","last_unstale":"2024-09-19T06:49:23.325749+0000","last_undegraded":"2024-09-19T06:49:23.325749+0000","last_fullsized":"2024-09-19T06:49:23.325749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:35.032949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.965109+0000","last_change":"2024-09-19T06:47:24.846738+0000","last_active":"2024-09-19T06:49:13.965109+0000","last_peered":"2024-09-19T06:49:13.965109+0000","last_clean":"2024-09-19T06:49:13.965109+0000","last_became_active":"2024-09-19T06:47:24.846389+0000","last_became_peered":"2024-09-19T06:47:24.846389+0000","last_unstale":"2024-09-19T06:49:13.965109+0000","last_undegraded":"2024-09-19T06:49:13.965109+0000","last_fullsized":"2024-09-19T06:49:13.965109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:19.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.918173+0000","last_change":"2024-09-19T06:47:22.851065+0000","last_active":"2024-09-19T06:49:13.918173+0000","last_peered":"2024-09-19T06:49:13.918173+0000","last_clean":"2024-09-19T06:49:13.918173+0000","last_became_active":"2024-09-19T06:47:22.850647+0000","last_became_peered":"2024-09-19T06:47:22.850647+0000","last_unstale":"2024-09-19T06:49:13.918173+0000","last_undegraded":"2024-09-19T06:49:13.918173+0000","last_fullsized":"2024-09-19T06:49:13.918173+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:41.993501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966795+0000","last_change":"2024-09-19T06:47:24.838905+0000","last_active":"2024-09-19T06:49:13.966795+0000","last_peered":"2024-09-19T06:49:13.966795+0000","last_clean":"2024-09-19T06:49:13.966795+0000","last_became_active":"2024-09-19T06:47:24.838443+0000","last_became_peered":"2024-09-19T06:47:24.838443+0000","last_unstale":"2024-09-19T06:49:13.966795+0000","last_undegraded":"2024-09-19T06:49:13.966795+0000","last_fullsized":"2024-09-19T06:49:13.966795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:12.486137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932653+0000","last_change":"2024-09-19T06:47:22.851431+0000","last_active":"2024-09-19T06:49:13.932653+0000","last_peered":"2024-09-19T06:49:13.932653+0000","last_clean":"2024-09-19T06:49:13.932653+0000","last_became_active":"2024-09-19T06:47:22.851092+0000","last_became_peered":"2024-09-19T06:47:22.851092+0000","last_unstale":"2024-09-19T06:49:13.932653+0000","last_undegraded":"2024-09-19T06:49:13.932653+0000","last_fullsized":"2024-09-19T06:49:13.932653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:26.183803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959143+0000","last_change":"2024-09-19T06:47:24.839046+0000","last_active":"2024-09-19T06:49:13.959143+0000","last_peered":"2024-09-19T06:49:13.959143+0000","last_clean":"2024-09-19T06:49:13.959143+0000","last_became_active":"2024-09-19T06:47:24.838680+0000","last_became_peered":"2024-09-19T06:47:24.838680+0000","last_unstale":"2024-09-19T06:49:13.959143+0000","last_undegraded":"2024-09-19T06:49:13.959143+0000","last_fullsized":"2024-09-19T06:49:13.959143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:45.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'102","reported_seq":231,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:24.747427+0000","last_change":"2024-09-19T06:47:40.865241+0000","last_active":"2024-09-19T06:49:24.747427+0000","last_peered":"2024-09-19T06:49:24.747427+0000","last_clean":"2024-09-19T06:49:24.747427+0000","last_became_active":"2024-09-19T06:47:40.837765+0000","last_became_peered":"2024-09-19T06:47:40.837765+0000","last_unstale":"2024-09-19T06:49:24.747427+0000","last_undegraded":"2024-09-19T06:49:24.747427+0000","last_fullsized":"2024-09-19T06:49:24.747427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":186,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966118+0000","last_change":"2024-09-19T06:47:24.848599+0000","last_active":"2024-09-19T06:49:13.966118+0000","last_peered":"2024-09-19T06:49:13.966118+0000","last_clean":"2024-09-19T06:49:13.966118+0000","last_became_active":"2024-09-19T06:47:24.848305+0000","last_became_peered":"2024-09-19T06:47:24.848305+0000","last_unstale":"2024-09-19T06:49:13.966118+0000","last_undegraded":"2024-09-19T06:49:13.966118+0000","last_fullsized":"2024-09-19T06:49:13.966118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:31.496188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.927755+0000","last_change":"2024-09-19T06:47:23.047484+0000","last_active":"2024-09-19T06:49:13.927755+0000","last_peered":"2024-09-19T06:49:13.927755+0000","last_clean":"2024-09-19T06:49:13.927755+0000","last_became_active":"2024-09-19T06:47:23.047058+0000","last_became_peered":"2024-09-19T06:47:23.047058+0000","last_unstale":"2024-09-19T06:49:13.927755+0000","last_undegraded":"2024-09-19T06:49:13.927755+0000","last_fullsized":"2024-09-19T06:49:13.927755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:34.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970265+0000","last_change":"2024-09-19T06:47:24.843498+0000","last_active":"2024-09-19T06:49:13.970265+0000","last_peered":"2024-09-19T06:49:13.970265+0000","last_clean":"2024-09-19T06:49:13.970265+0000","last_became_active":"2024-09-19T06:47:24.843204+0000","last_became_peered":"2024-09-19T06:47:24.843204+0000","last_unstale":"2024-09-19T06:49:13.970265+0000","last_undegraded":"2024-09-19T06:49:13.970265+0000","last_fullsized":"2024-09-19T06:49:13.970265+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:25.585023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.919373+0000","last_change":"2024-09-19T06:47:22.851519+0000","last_active":"2024-09-19T06:49:13.919373+0000","last_peered":"2024-09-19T06:49:13.919373+0000","last_clean":"2024-09-19T06:49:13.919373+0000","last_became_active":"2024-09-19T06:47:22.851163+0000","last_became_peered":"2024-09-19T06:47:22.851163+0000","last_unstale":"2024-09-19T06:49:13.919373+0000","last_undegraded":"2024-09-19T06:49:13.919373+0000","last_fullsized":"2024-09-19T06:49:13.919373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:04.502795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.945308+0000","last_change":"2024-09-19T06:47:24.845239+0000","last_active":"2024-09-19T06:49:13.945308+0000","last_peered":"2024-09-19T06:49:13.945308+0000","last_clean":"2024-09-19T06:49:13.945308+0000","last_became_active":"2024-09-19T06:47:24.844934+0000","last_became_peered":"2024-09-19T06:47:24.844934+0000","last_unstale":"2024-09-19T06:49:13.945308+0000","last_undegraded":"2024-09-19T06:49:13.945308+0000","last_fullsized":"2024-09-19T06:49:13.945308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:43.610604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.934380+0000","last_change":"2024-09-19T06:47:22.830232+0000","last_active":"2024-09-19T06:49:13.934380+0000","last_peered":"2024-09-19T06:49:13.934380+0000","last_clean":"2024-09-19T06:49:13.934380+0000","last_became_active":"2024-09-19T06:47:22.829901+0000","last_became_peered":"2024-09-19T06:47:22.829901+0000","last_unstale":"2024-09-19T06:49:13.934380+0000","last_undegraded":"2024-09-19T06:49:13.934380+0000","last_fullsized":"2024-09-19T06:49:13.934380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:12.251427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955845+0000","last_change":"2024-09-19T06:47:24.839304+0000","last_active":"2024-09-19T06:49:13.955845+0000","last_peered":"2024-09-19T06:49:13.955845+0000","last_clean":"2024-09-19T06:49:13.955845+0000","last_became_active":"2024-09-19T06:47:24.838986+0000","last_became_peered":"2024-09-19T06:47:24.838986+0000","last_unstale":"2024-09-19T06:49:13.955845+0000","last_undegraded":"2024-09-19T06:49:13.955845+0000","last_fullsized":"2024-09-19T06:49:13.955845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:39.377886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.885319+0000","last_change":"2024-09-19T06:47:22.828553+0000","last_active":"2024-09-19T06:49:14.885319+0000","last_peered":"2024-09-19T06:49:14.885319+0000","last_clean":"2024-09-19T06:49:14.885319+0000","last_became_active":"2024-09-19T06:47:22.828213+0000","last_became_peered":"2024-09-19T06:47:22.828213+0000","last_unstale":"2024-09-19T06:49:14.885319+0000","last_undegraded":"2024-09-19T06:49:14.885319+0000","last_fullsized":"2024-09-19T06:49:14.885319+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:31.649761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960387+0000","last_change":"2024-09-19T06:47:24.840747+0000","last_active":"2024-09-19T06:49:13.960387+0000","last_peered":"2024-09-19T06:49:13.960387+0000","last_clean":"2024-09-19T06:49:13.960387+0000","last_became_active":"2024-09-19T06:47:24.840451+0000","last_became_peered":"2024-09-19T06:47:24.840451+0000","last_unstale":"2024-09-19T06:49:13.960387+0000","last_undegraded":"2024-09-19T06:49:13.960387+0000","last_fullsized":"2024-09-19T06:49:13.960387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:41.588751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.933845+0000","last_change":"2024-09-19T06:47:22.831697+0000","last_active":"2024-09-19T06:49:13.933845+0000","last_peered":"2024-09-19T06:49:13.933845+0000","last_clean":"2024-09-19T06:49:13.933845+0000","last_became_active":"2024-09-19T06:47:22.831178+0000","last_became_peered":"2024-09-19T06:47:22.831178+0000","last_unstale":"2024-09-19T06:49:13.933845+0000","last_undegraded":"2024-09-19T06:49:13.933845+0000","last_fullsized":"2024-09-19T06:49:13.933845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:30.975782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971917+0000","last_change":"2024-09-19T06:47:24.845097+0000","last_active":"2024-09-19T06:49:13.971917+0000","last_peered":"2024-09-19T06:49:13.971917+0000","last_clean":"2024-09-19T06:49:13.971917+0000","last_became_active":"2024-09-19T06:47:24.844801+0000","last_became_peered":"2024-09-19T06:47:24.844801+0000","last_unstale":"2024-09-19T06:49:13.971917+0000","last_undegraded":"2024-09-19T06:49:13.971917+0000","last_fullsized":"2024-09-19T06:49:13.971917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:15.264118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971733+0000","last_change":"2024-09-19T06:47:24.851372+0000","last_active":"2024-09-19T06:49:13.971733+0000","last_peered":"2024-09-19T06:49:13.971733+0000","last_clean":"2024-09-19T06:49:13.971733+0000","last_became_active":"2024-09-19T06:47:24.851044+0000","last_became_peered":"2024-09-19T06:47:24.851044+0000","last_unstale":"2024-09-19T06:49:13.971733+0000","last_undegraded":"2024-09-19T06:49:13.971733+0000","last_fullsized":"2024-09-19T06:49:13.971733+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:55.658214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932580+0000","last_change":"2024-09-19T06:47:22.833694+0000","last_active":"2024-09-19T06:49:13.932580+0000","last_peered":"2024-09-19T06:49:13.932580+0000","last_clean":"2024-09-19T06:49:13.932580+0000","last_became_active":"2024-09-19T06:47:22.833215+0000","last_became_peered":"2024-09-19T06:47:22.833215+0000","last_unstale":"2024-09-19T06:49:13.932580+0000","last_undegraded":"2024-09-19T06:49:13.932580+0000","last_fullsized":"2024-09-19T06:49:13.932580+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:32.171522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973551+0000","last_change":"2024-09-19T06:47:24.845737+0000","last_active":"2024-09-19T06:49:13.973551+0000","last_peered":"2024-09-19T06:49:13.973551+0000","last_clean":"2024-09-19T06:49:13.973551+0000","last_became_active":"2024-09-19T06:47:24.845242+0000","last_became_peered":"2024-09-19T06:47:24.845242+0000","last_unstale":"2024-09-19T06:49:13.973551+0000","last_undegraded":"2024-09-19T06:49:13.973551+0000","last_fullsized":"2024-09-19T06:49:13.973551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:51.554946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932499+0000","last_change":"2024-09-19T06:47:22.829869+0000","last_active":"2024-09-19T06:49:13.932499+0000","last_peered":"2024-09-19T06:49:13.932499+0000","last_clean":"2024-09-19T06:49:13.932499+0000","last_became_active":"2024-09-19T06:47:22.829522+0000","last_became_peered":"2024-09-19T06:47:22.829522+0000","last_unstale":"2024-09-19T06:49:13.932499+0000","last_undegraded":"2024-09-19T06:49:13.932499+0000","last_fullsized":"2024-09-19T06:49:13.932499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:26.871434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973419+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:49:13.973419+0000","last_peered":"2024-09-19T06:49:13.973419+0000","last_clean":"2024-09-19T06:49:13.973419+0000","last_became_active":"2024-09-19T06:47:24.841289+0000","last_became_peered":"2024-09-19T06:47:24.841289+0000","last_unstale":"2024-09-19T06:49:13.973419+0000","last_undegraded":"2024-09-19T06:49:13.973419+0000","last_fullsized":"2024-09-19T06:49:13.973419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:21:12.611458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.937218+0000","last_change":"2024-09-19T06:47:22.852198+0000","last_active":"2024-09-19T06:49:13.937218+0000","last_peered":"2024-09-19T06:49:13.937218+0000","last_clean":"2024-09-19T06:49:13.937218+0000","last_became_active":"2024-09-19T06:47:22.850984+0000","last_became_peered":"2024-09-19T06:47:22.850984+0000","last_unstale":"2024-09-19T06:49:13.937218+0000","last_undegraded":"2024-09-19T06:49:13.937218+0000","last_fullsized":"2024-09-19T06:49:13.937218+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:13.620145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959682+0000","last_change":"2024-09-19T06:47:40.837614+0000","last_active":"2024-09-19T06:49:13.959682+0000","last_peered":"2024-09-19T06:49:13.959682+0000","last_clean":"2024-09-19T06:49:13.959682+0000","last_became_active":"2024-09-19T06:47:40.837259+0000","last_became_peered":"2024-09-19T06:47:40.837259+0000","last_unstale":"2024-09-19T06:49:13.959682+0000","last_undegraded":"2024-09-19T06:49:13.959682+0000","last_fullsized":"2024-09-19T06:49:13.959682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:41.429962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":168,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.534111+0000","last_change":"2024-09-19T06:47:41.182639+0000","last_active":"2024-09-19T06:49:23.534111+0000","last_peered":"2024-09-19T06:49:23.534111+0000","last_clean":"2024-09-19T06:49:23.534111+0000","last_became_active":"2024-09-19T06:47:41.174305+0000","last_became_peered":"2024-09-19T06:47:41.174305+0000","last_unstale":"2024-09-19T06:49:23.534111+0000","last_undegraded":"2024-09-19T06:49:23.534111+0000","last_fullsized":"2024-09-19T06:49:23.534111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:10.199502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":802788,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1473,"num_read_kb":956,"num_write":508,"num_write_kb":2390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2777088,"data_stored":2408364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":387,"ondisk_log_size":387,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712684,"statfs":{"total":95995035648,"available":95961788416,"internally_reserved":0,"allocated":1314816,"data_stored":842671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29060,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716092,"statfs":{"total":95995035648,"available":95965278208,"internally_reserved":0,"allocated":1961984,"data_stored":1439332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972399,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28328,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716824,"statfs":{"total":95995035648,"available":95966027776,"internally_reserved":0,"allocated":1335296,"data_stored":840798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234036,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":447288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32684,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712468,"statfs":{"total":95995035648,"available":95961567232,"internally_reserved":0,"allocated":1536000,"data_stored":1042894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626494,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":447692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855425,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28976,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716176,"statfs":{"total":95995035648,"available":95965364224,"internally_reserved":0,"allocated":1875968,"data_stored":1368300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018951,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":449293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215176,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1667072,"data_stored":1168572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":520824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":519675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":450351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":74364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":331355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":402387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":396211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:49:28.698 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T06:49:28.699 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:49:28.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:28 smithi174 ceph-mon[35637]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:28.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:28 smithi174 ceph-mon[35637]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-19T06:49:28.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:28 smithi134 ceph-mon[35356]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:28.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:28 smithi134 ceph-mon[35356]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-19T06:49:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.454+0000 7ff439148640 1 -- 172.21.15.103:0/2257595928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434106f10 msgr2=0x7ff43410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.454+0000 7ff439148640 1 --2- 172.21.15.103:0/2257595928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434106f10 0x7ff43410ea90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009ca0 tx=0x7ff42c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.454+0000 7ff439148640 1 -- 172.21.15.103:0/2257595928 shutdown_connections 2024-09-19T06:49:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.454+0000 7ff439148640 1 --2- 172.21.15.103:0/2257595928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434106f10 0x7ff43410ea90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.454+0000 7ff439148640 1 --2- 172.21.15.103:0/2257595928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff434102790 0x7ff4341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.454+0000 7ff439148640 1 --2- 172.21.15.103:0/2257595928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff434101de0 0x7ff4341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.454+0000 7ff439148640 1 -- 172.21.15.103:0/2257595928 >> 172.21.15.103:0/2257595928 conn(0x7ff4340fd8c0 msgr2=0x7ff4340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.454+0000 7ff439148640 1 -- 172.21.15.103:0/2257595928 shutdown_connections 2024-09-19T06:49:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.454+0000 7ff439148640 1 -- 172.21.15.103:0/2257595928 wait complete. 2024-09-19T06:49:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff439148640 1 Processor -- start 2024-09-19T06:49:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff439148640 1 -- start start 2024-09-19T06:49:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff439148640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434101de0 0x7ff434198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff439148640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff434102790 0x7ff434198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff439148640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff434106f10 0x7ff43419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff439148640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff434111f70 con 0x7ff434101de0 2024-09-19T06:49:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff439148640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff434111df0 con 0x7ff434106f10 2024-09-19T06:49:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff439148640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff4341120f0 con 0x7ff434102790 2024-09-19T06:49:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff432d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434101de0 0x7ff434198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff432d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434101de0 0x7ff434198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40350/0 (socket says 172.21.15.103:40350) 2024-09-19T06:49:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff432d9d640 1 -- 172.21.15.103:0/3202724031 learned_addr learned my addr 172.21.15.103:0/3202724031 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff43359e640 1 --2- 172.21.15.103:0/3202724031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff434106f10 0x7ff43419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff43259c640 1 --2- 172.21.15.103:0/3202724031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff434102790 0x7ff434198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff432d9d640 1 -- 172.21.15.103:0/3202724031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff434102790 msgr2=0x7ff434198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff432d9d640 1 --2- 172.21.15.103:0/3202724031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff434102790 0x7ff434198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff432d9d640 1 -- 172.21.15.103:0/3202724031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff434106f10 msgr2=0x7ff43419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff432d9d640 1 --2- 172.21.15.103:0/3202724031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff434106f10 0x7ff43419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.455+0000 7ff432d9d640 1 -- 172.21.15.103:0/3202724031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43419d1d0 con 0x7ff434101de0 2024-09-19T06:49:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.456+0000 7ff432d9d640 1 --2- 172.21.15.103:0/3202724031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434101de0 0x7ff434198220 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff42400d770 tx=0x7ff42400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.456+0000 7ff42bfff640 1 -- 172.21.15.103:0/3202724031 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424017070 con 0x7ff434101de0 2024-09-19T06:49:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.456+0000 7ff42bfff640 1 -- 172.21.15.103:0/3202724031 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff424005500 con 0x7ff434101de0 2024-09-19T06:49:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.456+0000 7ff439148640 1 -- 172.21.15.103:0/3202724031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43419d4c0 con 0x7ff434101de0 2024-09-19T06:49:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.456+0000 7ff42bfff640 1 -- 172.21.15.103:0/3202724031 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424004e00 con 0x7ff434101de0 2024-09-19T06:49:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.456+0000 7ff439148640 1 -- 172.21.15.103:0/3202724031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4341a4da0 con 0x7ff434101de0 2024-09-19T06:49:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.457+0000 7ff439148640 1 -- 172.21.15.103:0/3202724031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff434199580 con 0x7ff434101de0 2024-09-19T06:49:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.458+0000 7ff42bfff640 1 -- 172.21.15.103:0/3202724031 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7ff424004fa0 con 0x7ff434101de0 2024-09-19T06:49:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.459+0000 7ff42bfff640 1 --2- 172.21.15.103:0/3202724031 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4200a34d0 0x7ff4200a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.459+0000 7ff43259c640 1 --2- 172.21.15.103:0/3202724031 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4200a34d0 0x7ff4200a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.459+0000 7ff42bfff640 1 -- 172.21.15.103:0/3202724031 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff4240cf990 con 0x7ff434101de0 2024-09-19T06:49:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.460+0000 7ff43259c640 1 --2- 172.21.15.103:0/3202724031 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4200a34d0 0x7ff4200a5990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff4341082e0 tx=0x7ff41c00b390 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.463+0000 7ff42bfff640 1 -- 172.21.15.103:0/3202724031 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff42409bae0 con 0x7ff434101de0 2024-09-19T06:49:29.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.626+0000 7ff439148640 1 -- 172.21.15.103:0/3202724031 --> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4340fe8e0 con 0x7ff4200a34d0 2024-09-19T06:49:29.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.633+0000 7ff42bfff640 1 -- 172.21.15.103:0/3202724031 <== mgr.15210 v2:172.21.15.103:6800/82882809 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257823 (secure 0 0 0) 0x7ff4340fe8e0 con 0x7ff4200a34d0 2024-09-19T06:49:29.637 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:49:29.638 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T06:49:29.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.635+0000 7ff439148640 1 -- 172.21.15.103:0/3202724031 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4200a34d0 msgr2=0x7ff4200a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:29.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.635+0000 7ff439148640 1 --2- 172.21.15.103:0/3202724031 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4200a34d0 0x7ff4200a5990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff4341082e0 tx=0x7ff41c00b390 comp rx=0 tx=0).stop 2024-09-19T06:49:29.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.635+0000 7ff439148640 1 -- 172.21.15.103:0/3202724031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434101de0 msgr2=0x7ff434198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:29.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.635+0000 7ff439148640 1 --2- 172.21.15.103:0/3202724031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434101de0 0x7ff434198220 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff42400d770 tx=0x7ff42400dc30 comp rx=0 tx=0).stop 2024-09-19T06:49:29.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.635+0000 7ff439148640 1 -- 172.21.15.103:0/3202724031 shutdown_connections 2024-09-19T06:49:29.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.635+0000 7ff439148640 1 --2- 172.21.15.103:0/3202724031 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4200a34d0 0x7ff4200a5990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.635+0000 7ff439148640 1 --2- 172.21.15.103:0/3202724031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff434106f10 0x7ff43419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.635+0000 7ff439148640 1 --2- 172.21.15.103:0/3202724031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff434102790 0x7ff434198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.635+0000 7ff439148640 1 --2- 172.21.15.103:0/3202724031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434101de0 0x7ff434198220 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.635+0000 7ff439148640 1 -- 172.21.15.103:0/3202724031 >> 172.21.15.103:0/3202724031 conn(0x7ff4340fd8c0 msgr2=0x7ff4340fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:29.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.636+0000 7ff439148640 1 -- 172.21.15.103:0/3202724031 shutdown_connections 2024-09-19T06:49:29.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:29.636+0000 7ff439148640 1 -- 172.21.15.103:0/3202724031 wait complete. 2024-09-19T06:49:30.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:30 smithi103 ceph-mon[31158]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:30.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:30 smithi103 ceph-mon[31158]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-09-19T06:49:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:30 smithi174 ceph-mon[35637]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:30 smithi174 ceph-mon[35637]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-09-19T06:49:30.336 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-19T06:49:27.934569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1393390,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1710,"num_read_kb":1286,"num_write":711,"num_write_kb":5330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":1181610,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1037,"ondisk_log_size":1037,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356268,"kb_used_data":15084,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124585556,"statfs":{"total":1151940427776,"available":1151575609344,"internally_reserved":0,"allocated":15446016,"data_stored":9537690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201469,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":450,"num_read_kb":491,"num_write":149,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002334"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960112+0000","last_change":"2024-09-19T06:47:24.853118+0000","last_active":"2024-09-19T06:49:13.960112+0000","last_peered":"2024-09-19T06:49:13.960112+0000","last_clean":"2024-09-19T06:49:13.960112+0000","last_became_active":"2024-09-19T06:47:24.852801+0000","last_became_peered":"2024-09-19T06:47:24.852801+0000","last_unstale":"2024-09-19T06:49:13.960112+0000","last_undegraded":"2024-09-19T06:49:13.960112+0000","last_fullsized":"2024-09-19T06:49:13.960112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:16.729103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.957024+0000","last_change":"2024-09-19T06:47:24.841095+0000","last_active":"2024-09-19T06:49:13.957024+0000","last_peered":"2024-09-19T06:49:13.957024+0000","last_clean":"2024-09-19T06:49:13.957024+0000","last_became_active":"2024-09-19T06:47:24.840774+0000","last_became_peered":"2024-09-19T06:47:24.840774+0000","last_unstale":"2024-09-19T06:49:13.957024+0000","last_undegraded":"2024-09-19T06:49:13.957024+0000","last_fullsized":"2024-09-19T06:49:13.957024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:27.262766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973374+0000","last_change":"2024-09-19T06:47:24.848444+0000","last_active":"2024-09-19T06:49:13.973374+0000","last_peered":"2024-09-19T06:49:13.973374+0000","last_clean":"2024-09-19T06:49:13.973374+0000","last_became_active":"2024-09-19T06:47:24.847354+0000","last_became_peered":"2024-09-19T06:47:24.847354+0000","last_unstale":"2024-09-19T06:49:13.973374+0000","last_undegraded":"2024-09-19T06:49:13.973374+0000","last_fullsized":"2024-09-19T06:49:13.973374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:44.901265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.949178+0000","last_change":"2024-09-19T06:47:24.851915+0000","last_active":"2024-09-19T06:49:13.949178+0000","last_peered":"2024-09-19T06:49:13.949178+0000","last_clean":"2024-09-19T06:49:13.949178+0000","last_became_active":"2024-09-19T06:47:24.851600+0000","last_became_peered":"2024-09-19T06:47:24.851600+0000","last_unstale":"2024-09-19T06:49:13.949178+0000","last_undegraded":"2024-09-19T06:49:13.949178+0000","last_fullsized":"2024-09-19T06:49:13.949178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:52.424288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.950906+0000","last_change":"2024-09-19T06:47:24.843184+0000","last_active":"2024-09-19T06:49:13.950906+0000","last_peered":"2024-09-19T06:49:13.950906+0000","last_clean":"2024-09-19T06:49:13.950906+0000","last_became_active":"2024-09-19T06:47:24.842857+0000","last_became_peered":"2024-09-19T06:47:24.842857+0000","last_unstale":"2024-09-19T06:49:13.950906+0000","last_undegraded":"2024-09-19T06:49:13.950906+0000","last_fullsized":"2024-09-19T06:49:13.950906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:06.608699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969815+0000","last_change":"2024-09-19T06:47:24.846631+0000","last_active":"2024-09-19T06:49:13.969815+0000","last_peered":"2024-09-19T06:49:13.969815+0000","last_clean":"2024-09-19T06:49:13.969815+0000","last_became_active":"2024-09-19T06:47:24.846271+0000","last_became_peered":"2024-09-19T06:47:24.846271+0000","last_unstale":"2024-09-19T06:49:13.969815+0000","last_undegraded":"2024-09-19T06:49:13.969815+0000","last_fullsized":"2024-09-19T06:49:13.969815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.272273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.967306+0000","last_change":"2024-09-19T06:47:24.840392+0000","last_active":"2024-09-19T06:49:13.967306+0000","last_peered":"2024-09-19T06:49:13.967306+0000","last_clean":"2024-09-19T06:49:13.967306+0000","last_became_active":"2024-09-19T06:47:24.840092+0000","last_became_peered":"2024-09-19T06:47:24.840092+0000","last_unstale":"2024-09-19T06:49:13.967306+0000","last_undegraded":"2024-09-19T06:49:13.967306+0000","last_fullsized":"2024-09-19T06:49:13.967306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:25.579096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.967567+0000","last_change":"2024-09-19T06:47:24.843336+0000","last_active":"2024-09-19T06:49:13.967567+0000","last_peered":"2024-09-19T06:49:13.967567+0000","last_clean":"2024-09-19T06:49:13.967567+0000","last_became_active":"2024-09-19T06:47:24.842016+0000","last_became_peered":"2024-09-19T06:47:24.842016+0000","last_unstale":"2024-09-19T06:49:13.967567+0000","last_undegraded":"2024-09-19T06:49:13.967567+0000","last_fullsized":"2024-09-19T06:49:13.967567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:49.478359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.968002+0000","last_change":"2024-09-19T06:47:24.847841+0000","last_active":"2024-09-19T06:49:13.968002+0000","last_peered":"2024-09-19T06:49:13.968002+0000","last_clean":"2024-09-19T06:49:13.968002+0000","last_became_active":"2024-09-19T06:47:24.847492+0000","last_became_peered":"2024-09-19T06:47:24.847492+0000","last_unstale":"2024-09-19T06:49:13.968002+0000","last_undegraded":"2024-09-19T06:49:13.968002+0000","last_fullsized":"2024-09-19T06:49:13.968002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:59.229022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955381+0000","last_change":"2024-09-19T06:47:24.843773+0000","last_active":"2024-09-19T06:49:13.955381+0000","last_peered":"2024-09-19T06:49:13.955381+0000","last_clean":"2024-09-19T06:49:13.955381+0000","last_became_active":"2024-09-19T06:47:24.843409+0000","last_became_peered":"2024-09-19T06:47:24.843409+0000","last_unstale":"2024-09-19T06:49:13.955381+0000","last_undegraded":"2024-09-19T06:49:13.955381+0000","last_fullsized":"2024-09-19T06:49:13.955381+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:22.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969613+0000","last_change":"2024-09-19T06:47:24.846637+0000","last_active":"2024-09-19T06:49:13.969613+0000","last_peered":"2024-09-19T06:49:13.969613+0000","last_clean":"2024-09-19T06:49:13.969613+0000","last_became_active":"2024-09-19T06:47:24.846337+0000","last_became_peered":"2024-09-19T06:47:24.846337+0000","last_unstale":"2024-09-19T06:49:13.969613+0000","last_undegraded":"2024-09-19T06:49:13.969613+0000","last_fullsized":"2024-09-19T06:49:13.969613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:23:35.869485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.946579+0000","last_change":"2024-09-19T06:47:24.851399+0000","last_active":"2024-09-19T06:49:13.946579+0000","last_peered":"2024-09-19T06:49:13.946579+0000","last_clean":"2024-09-19T06:49:13.946579+0000","last_became_active":"2024-09-19T06:47:24.851090+0000","last_became_peered":"2024-09-19T06:47:24.851090+0000","last_unstale":"2024-09-19T06:49:13.946579+0000","last_undegraded":"2024-09-19T06:49:13.946579+0000","last_fullsized":"2024-09-19T06:49:13.946579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:52.839956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970074+0000","last_change":"2024-09-19T06:47:24.833858+0000","last_active":"2024-09-19T06:49:13.970074+0000","last_peered":"2024-09-19T06:49:13.970074+0000","last_clean":"2024-09-19T06:49:13.970074+0000","last_became_active":"2024-09-19T06:47:24.833616+0000","last_became_peered":"2024-09-19T06:47:24.833616+0000","last_unstale":"2024-09-19T06:49:13.970074+0000","last_undegraded":"2024-09-19T06:49:13.970074+0000","last_fullsized":"2024-09-19T06:49:13.970074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:54.918144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966731+0000","last_change":"2024-09-19T06:47:24.841030+0000","last_active":"2024-09-19T06:49:13.966731+0000","last_peered":"2024-09-19T06:49:13.966731+0000","last_clean":"2024-09-19T06:49:13.966731+0000","last_became_active":"2024-09-19T06:47:24.839631+0000","last_became_peered":"2024-09-19T06:47:24.839631+0000","last_unstale":"2024-09-19T06:49:13.966731+0000","last_undegraded":"2024-09-19T06:49:13.966731+0000","last_fullsized":"2024-09-19T06:49:13.966731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:21.764812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970278+0000","last_change":"2024-09-19T06:47:24.849281+0000","last_active":"2024-09-19T06:49:13.970278+0000","last_peered":"2024-09-19T06:49:13.970278+0000","last_clean":"2024-09-19T06:49:13.970278+0000","last_became_active":"2024-09-19T06:47:24.848977+0000","last_became_peered":"2024-09-19T06:47:24.848977+0000","last_unstale":"2024-09-19T06:49:13.970278+0000","last_undegraded":"2024-09-19T06:49:13.970278+0000","last_fullsized":"2024-09-19T06:49:13.970278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:36.693758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973886+0000","last_change":"2024-09-19T06:47:24.843619+0000","last_active":"2024-09-19T06:49:13.973886+0000","last_peered":"2024-09-19T06:49:13.973886+0000","last_clean":"2024-09-19T06:49:13.973886+0000","last_became_active":"2024-09-19T06:47:24.843323+0000","last_became_peered":"2024-09-19T06:47:24.843323+0000","last_unstale":"2024-09-19T06:49:13.973886+0000","last_undegraded":"2024-09-19T06:49:13.973886+0000","last_fullsized":"2024-09-19T06:49:13.973886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:52.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960333+0000","last_change":"2024-09-19T06:47:24.853165+0000","last_active":"2024-09-19T06:49:13.960333+0000","last_peered":"2024-09-19T06:49:13.960333+0000","last_clean":"2024-09-19T06:49:13.960333+0000","last_became_active":"2024-09-19T06:47:24.852864+0000","last_became_peered":"2024-09-19T06:47:24.852864+0000","last_unstale":"2024-09-19T06:49:13.960333+0000","last_undegraded":"2024-09-19T06:49:13.960333+0000","last_fullsized":"2024-09-19T06:49:13.960333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:38.803478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.975021+0000","last_change":"2024-09-19T06:47:24.838151+0000","last_active":"2024-09-19T06:49:13.975021+0000","last_peered":"2024-09-19T06:49:13.975021+0000","last_clean":"2024-09-19T06:49:13.975021+0000","last_became_active":"2024-09-19T06:47:24.837855+0000","last_became_peered":"2024-09-19T06:47:24.837855+0000","last_unstale":"2024-09-19T06:49:13.975021+0000","last_undegraded":"2024-09-19T06:49:13.975021+0000","last_fullsized":"2024-09-19T06:49:13.975021+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:16.675557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.964466+0000","last_change":"2024-09-19T06:47:24.843941+0000","last_active":"2024-09-19T06:49:13.964466+0000","last_peered":"2024-09-19T06:49:13.964466+0000","last_clean":"2024-09-19T06:49:13.964466+0000","last_became_active":"2024-09-19T06:47:24.843551+0000","last_became_peered":"2024-09-19T06:47:24.843551+0000","last_unstale":"2024-09-19T06:49:13.964466+0000","last_undegraded":"2024-09-19T06:49:13.964466+0000","last_fullsized":"2024-09-19T06:49:13.964466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:00:49.388265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.954208+0000","last_change":"2024-09-19T06:47:40.836228+0000","last_active":"2024-09-19T06:49:13.954208+0000","last_peered":"2024-09-19T06:49:13.954208+0000","last_clean":"2024-09-19T06:49:13.954208+0000","last_became_active":"2024-09-19T06:47:40.835818+0000","last_became_peered":"2024-09-19T06:47:40.835818+0000","last_unstale":"2024-09-19T06:49:13.954208+0000","last_undegraded":"2024-09-19T06:49:13.954208+0000","last_fullsized":"2024-09-19T06:49:13.954208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:45.341160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972919+0000","last_change":"2024-09-19T06:47:24.843746+0000","last_active":"2024-09-19T06:49:13.972919+0000","last_peered":"2024-09-19T06:49:13.972919+0000","last_clean":"2024-09-19T06:49:13.972919+0000","last_became_active":"2024-09-19T06:47:24.841495+0000","last_became_peered":"2024-09-19T06:47:24.841495+0000","last_unstale":"2024-09-19T06:49:13.972919+0000","last_undegraded":"2024-09-19T06:49:13.972919+0000","last_fullsized":"2024-09-19T06:49:13.972919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:06:29.995595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.974565+0000","last_change":"2024-09-19T06:47:24.844747+0000","last_active":"2024-09-19T06:49:13.974565+0000","last_peered":"2024-09-19T06:49:13.974565+0000","last_clean":"2024-09-19T06:49:13.974565+0000","last_became_active":"2024-09-19T06:47:24.844425+0000","last_became_peered":"2024-09-19T06:47:24.844425+0000","last_unstale":"2024-09-19T06:49:13.974565+0000","last_undegraded":"2024-09-19T06:49:13.974565+0000","last_fullsized":"2024-09-19T06:49:13.974565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:10.309044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963556+0000","last_change":"2024-09-19T06:47:24.848227+0000","last_active":"2024-09-19T06:49:13.963556+0000","last_peered":"2024-09-19T06:49:13.963556+0000","last_clean":"2024-09-19T06:49:13.963556+0000","last_became_active":"2024-09-19T06:47:24.847914+0000","last_became_peered":"2024-09-19T06:47:24.847914+0000","last_unstale":"2024-09-19T06:49:13.963556+0000","last_undegraded":"2024-09-19T06:49:13.963556+0000","last_fullsized":"2024-09-19T06:49:13.963556+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:47.281613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.968012+0000","last_change":"2024-09-19T06:47:24.844275+0000","last_active":"2024-09-19T06:49:13.968012+0000","last_peered":"2024-09-19T06:49:13.968012+0000","last_clean":"2024-09-19T06:49:13.968012+0000","last_became_active":"2024-09-19T06:47:24.841442+0000","last_became_peered":"2024-09-19T06:47:24.841442+0000","last_unstale":"2024-09-19T06:49:13.968012+0000","last_undegraded":"2024-09-19T06:49:13.968012+0000","last_fullsized":"2024-09-19T06:49:13.968012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:30.198655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963003+0000","last_change":"2024-09-19T06:47:24.846301+0000","last_active":"2024-09-19T06:49:13.963003+0000","last_peered":"2024-09-19T06:49:13.963003+0000","last_clean":"2024-09-19T06:49:13.963003+0000","last_became_active":"2024-09-19T06:47:24.845970+0000","last_became_peered":"2024-09-19T06:47:24.845970+0000","last_unstale":"2024-09-19T06:49:13.963003+0000","last_undegraded":"2024-09-19T06:49:13.963003+0000","last_fullsized":"2024-09-19T06:49:13.963003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:13.245536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973777+0000","last_change":"2024-09-19T06:47:24.844444+0000","last_active":"2024-09-19T06:49:13.973777+0000","last_peered":"2024-09-19T06:49:13.973777+0000","last_clean":"2024-09-19T06:49:13.973777+0000","last_became_active":"2024-09-19T06:47:24.844141+0000","last_became_peered":"2024-09-19T06:47:24.844141+0000","last_unstale":"2024-09-19T06:49:13.973777+0000","last_undegraded":"2024-09-19T06:49:13.973777+0000","last_fullsized":"2024-09-19T06:49:13.973777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:44.739242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955255+0000","last_change":"2024-09-19T06:47:24.845633+0000","last_active":"2024-09-19T06:49:13.955255+0000","last_peered":"2024-09-19T06:49:13.955255+0000","last_clean":"2024-09-19T06:49:13.955255+0000","last_became_active":"2024-09-19T06:47:24.845319+0000","last_became_peered":"2024-09-19T06:47:24.845319+0000","last_unstale":"2024-09-19T06:49:13.955255+0000","last_undegraded":"2024-09-19T06:49:13.955255+0000","last_fullsized":"2024-09-19T06:49:13.955255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:00.820006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959353+0000","last_change":"2024-09-19T06:47:24.846457+0000","last_active":"2024-09-19T06:49:13.959353+0000","last_peered":"2024-09-19T06:49:13.959353+0000","last_clean":"2024-09-19T06:49:13.959353+0000","last_became_active":"2024-09-19T06:47:24.846130+0000","last_became_peered":"2024-09-19T06:47:24.846130+0000","last_unstale":"2024-09-19T06:49:13.959353+0000","last_undegraded":"2024-09-19T06:49:13.959353+0000","last_fullsized":"2024-09-19T06:49:13.959353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:42.179145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969751+0000","last_change":"2024-09-19T06:47:24.843141+0000","last_active":"2024-09-19T06:49:13.969751+0000","last_peered":"2024-09-19T06:49:13.969751+0000","last_clean":"2024-09-19T06:49:13.969751+0000","last_became_active":"2024-09-19T06:47:24.842810+0000","last_became_peered":"2024-09-19T06:47:24.842810+0000","last_unstale":"2024-09-19T06:49:13.969751+0000","last_undegraded":"2024-09-19T06:49:13.969751+0000","last_fullsized":"2024-09-19T06:49:13.969751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:43.440453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.965342+0000","last_change":"2024-09-19T06:47:24.846171+0000","last_active":"2024-09-19T06:49:13.965342+0000","last_peered":"2024-09-19T06:49:13.965342+0000","last_clean":"2024-09-19T06:49:13.965342+0000","last_became_active":"2024-09-19T06:47:24.845799+0000","last_became_peered":"2024-09-19T06:47:24.845799+0000","last_unstale":"2024-09-19T06:49:13.965342+0000","last_undegraded":"2024-09-19T06:49:13.965342+0000","last_fullsized":"2024-09-19T06:49:13.965342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:23.103061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972106+0000","last_change":"2024-09-19T06:47:24.848723+0000","last_active":"2024-09-19T06:49:13.972106+0000","last_peered":"2024-09-19T06:49:13.972106+0000","last_clean":"2024-09-19T06:49:13.972106+0000","last_became_active":"2024-09-19T06:47:24.848367+0000","last_became_peered":"2024-09-19T06:47:24.848367+0000","last_unstale":"2024-09-19T06:49:13.972106+0000","last_undegraded":"2024-09-19T06:49:13.972106+0000","last_fullsized":"2024-09-19T06:49:13.972106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:15.985293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960543+0000","last_change":"2024-09-19T06:47:24.841379+0000","last_active":"2024-09-19T06:49:13.960543+0000","last_peered":"2024-09-19T06:49:13.960543+0000","last_clean":"2024-09-19T06:49:13.960543+0000","last_became_active":"2024-09-19T06:47:24.841041+0000","last_became_peered":"2024-09-19T06:47:24.841041+0000","last_unstale":"2024-09-19T06:49:13.960543+0000","last_undegraded":"2024-09-19T06:49:13.960543+0000","last_fullsized":"2024-09-19T06:49:13.960543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:11.799022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955643+0000","last_change":"2024-09-19T06:47:24.849508+0000","last_active":"2024-09-19T06:49:13.955643+0000","last_peered":"2024-09-19T06:49:13.955643+0000","last_clean":"2024-09-19T06:49:13.955643+0000","last_became_active":"2024-09-19T06:47:24.849196+0000","last_became_peered":"2024-09-19T06:47:24.849196+0000","last_unstale":"2024-09-19T06:49:13.955643+0000","last_undegraded":"2024-09-19T06:49:13.955643+0000","last_fullsized":"2024-09-19T06:49:13.955643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:35.079720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.617944+0000","last_change":"2024-09-19T06:47:22.829848+0000","last_active":"2024-09-19T06:49:23.617944+0000","last_peered":"2024-09-19T06:49:23.617944+0000","last_clean":"2024-09-19T06:49:23.617944+0000","last_became_active":"2024-09-19T06:47:22.829511+0000","last_became_peered":"2024-09-19T06:47:22.829511+0000","last_unstale":"2024-09-19T06:49:23.617944+0000","last_undegraded":"2024-09-19T06:49:23.617944+0000","last_fullsized":"2024-09-19T06:49:23.617944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:32.665252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971999+0000","last_change":"2024-09-19T06:47:24.843245+0000","last_active":"2024-09-19T06:49:13.971999+0000","last_peered":"2024-09-19T06:49:13.971999+0000","last_clean":"2024-09-19T06:49:13.971999+0000","last_became_active":"2024-09-19T06:47:24.842885+0000","last_became_peered":"2024-09-19T06:47:24.842885+0000","last_unstale":"2024-09-19T06:49:13.971999+0000","last_undegraded":"2024-09-19T06:49:13.971999+0000","last_fullsized":"2024-09-19T06:49:13.971999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:26.229003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.926577+0000","last_change":"2024-09-19T06:47:22.831456+0000","last_active":"2024-09-19T06:49:13.926577+0000","last_peered":"2024-09-19T06:49:13.926577+0000","last_clean":"2024-09-19T06:49:13.926577+0000","last_became_active":"2024-09-19T06:47:22.831132+0000","last_became_peered":"2024-09-19T06:47:22.831132+0000","last_unstale":"2024-09-19T06:49:13.926577+0000","last_undegraded":"2024-09-19T06:49:13.926577+0000","last_fullsized":"2024-09-19T06:49:13.926577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:49.867713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932598+0000","last_change":"2024-09-19T06:47:22.827742+0000","last_active":"2024-09-19T06:49:13.932598+0000","last_peered":"2024-09-19T06:49:13.932598+0000","last_clean":"2024-09-19T06:49:13.932598+0000","last_became_active":"2024-09-19T06:47:22.827371+0000","last_became_peered":"2024-09-19T06:47:22.827371+0000","last_unstale":"2024-09-19T06:49:13.932598+0000","last_undegraded":"2024-09-19T06:49:13.932598+0000","last_fullsized":"2024-09-19T06:49:13.932598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:35.153129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.963305+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:49:13.963305+0000","last_peered":"2024-09-19T06:49:13.963305+0000","last_clean":"2024-09-19T06:49:13.963305+0000","last_became_active":"2024-09-19T06:47:24.841693+0000","last_became_peered":"2024-09-19T06:47:24.841693+0000","last_unstale":"2024-09-19T06:49:13.963305+0000","last_undegraded":"2024-09-19T06:49:13.963305+0000","last_fullsized":"2024-09-19T06:49:13.963305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:27.828296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.917388+0000","last_change":"2024-09-19T06:47:22.832531+0000","last_active":"2024-09-19T06:49:13.917388+0000","last_peered":"2024-09-19T06:49:13.917388+0000","last_clean":"2024-09-19T06:49:13.917388+0000","last_became_active":"2024-09-19T06:47:22.832213+0000","last_became_peered":"2024-09-19T06:47:22.832213+0000","last_unstale":"2024-09-19T06:49:13.917388+0000","last_undegraded":"2024-09-19T06:49:13.917388+0000","last_fullsized":"2024-09-19T06:49:13.917388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:46:03.168406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970720+0000","last_change":"2024-09-19T06:47:24.837690+0000","last_active":"2024-09-19T06:49:13.970720+0000","last_peered":"2024-09-19T06:49:13.970720+0000","last_clean":"2024-09-19T06:49:13.970720+0000","last_became_active":"2024-09-19T06:47:24.837389+0000","last_became_peered":"2024-09-19T06:47:24.837389+0000","last_unstale":"2024-09-19T06:49:13.970720+0000","last_undegraded":"2024-09-19T06:49:13.970720+0000","last_fullsized":"2024-09-19T06:49:13.970720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:20.653832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":248,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:24.618359+0000","last_change":"2024-09-19T06:47:22.833808+0000","last_active":"2024-09-19T06:49:24.618359+0000","last_peered":"2024-09-19T06:49:24.618359+0000","last_clean":"2024-09-19T06:49:24.618359+0000","last_became_active":"2024-09-19T06:47:22.832468+0000","last_became_peered":"2024-09-19T06:47:22.832468+0000","last_unstale":"2024-09-19T06:49:24.618359+0000","last_undegraded":"2024-09-19T06:49:24.618359+0000","last_fullsized":"2024-09-19T06:49:24.618359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:39:31.131321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":218,"num_read_kb":218,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973091+0000","last_change":"2024-09-19T06:47:24.840928+0000","last_active":"2024-09-19T06:49:13.973091+0000","last_peered":"2024-09-19T06:49:13.973091+0000","last_clean":"2024-09-19T06:49:13.973091+0000","last_became_active":"2024-09-19T06:47:24.839928+0000","last_became_peered":"2024-09-19T06:47:24.839928+0000","last_unstale":"2024-09-19T06:49:13.973091+0000","last_undegraded":"2024-09-19T06:49:13.973091+0000","last_fullsized":"2024-09-19T06:49:13.973091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:29.486003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.928845+0000","last_change":"2024-09-19T06:47:22.833310+0000","last_active":"2024-09-19T06:49:13.928845+0000","last_peered":"2024-09-19T06:49:13.928845+0000","last_clean":"2024-09-19T06:49:13.928845+0000","last_became_active":"2024-09-19T06:47:22.832976+0000","last_became_peered":"2024-09-19T06:47:22.832976+0000","last_unstale":"2024-09-19T06:49:13.928845+0000","last_undegraded":"2024-09-19T06:49:13.928845+0000","last_fullsized":"2024-09-19T06:49:13.928845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:47.604853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.969853+0000","last_change":"2024-09-19T06:47:24.848687+0000","last_active":"2024-09-19T06:49:13.969853+0000","last_peered":"2024-09-19T06:49:13.969853+0000","last_clean":"2024-09-19T06:49:13.969853+0000","last_became_active":"2024-09-19T06:47:24.848352+0000","last_became_peered":"2024-09-19T06:47:24.848352+0000","last_unstale":"2024-09-19T06:49:13.969853+0000","last_undegraded":"2024-09-19T06:49:13.969853+0000","last_fullsized":"2024-09-19T06:49:13.969853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:54.459206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.917489+0000","last_change":"2024-09-19T06:47:22.851479+0000","last_active":"2024-09-19T06:49:13.917489+0000","last_peered":"2024-09-19T06:49:13.917489+0000","last_clean":"2024-09-19T06:49:13.917489+0000","last_became_active":"2024-09-19T06:47:22.851125+0000","last_became_peered":"2024-09-19T06:47:22.851125+0000","last_unstale":"2024-09-19T06:49:13.917489+0000","last_undegraded":"2024-09-19T06:49:13.917489+0000","last_fullsized":"2024-09-19T06:49:13.917489+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:42.954034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973229+0000","last_change":"2024-09-19T06:47:24.851007+0000","last_active":"2024-09-19T06:49:13.973229+0000","last_peered":"2024-09-19T06:49:13.973229+0000","last_clean":"2024-09-19T06:49:13.973229+0000","last_became_active":"2024-09-19T06:47:24.850465+0000","last_became_peered":"2024-09-19T06:47:24.850465+0000","last_unstale":"2024-09-19T06:49:13.973229+0000","last_undegraded":"2024-09-19T06:49:13.973229+0000","last_fullsized":"2024-09-19T06:49:13.973229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:10.824672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'39","reported_seq":295,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:24.618888+0000","last_change":"2024-09-19T06:47:22.834893+0000","last_active":"2024-09-19T06:49:24.618888+0000","last_peered":"2024-09-19T06:49:24.618888+0000","last_clean":"2024-09-19T06:49:24.618888+0000","last_became_active":"2024-09-19T06:47:22.834521+0000","last_became_peered":"2024-09-19T06:47:22.834521+0000","last_unstale":"2024-09-19T06:49:24.618888+0000","last_undegraded":"2024-09-19T06:49:24.618888+0000","last_fullsized":"2024-09-19T06:49:24.618888+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:29.742284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70398,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":232,"num_read_kb":185,"num_write":41,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.957411+0000","last_change":"2024-09-19T06:47:24.848191+0000","last_active":"2024-09-19T06:49:13.957411+0000","last_peered":"2024-09-19T06:49:13.957411+0000","last_clean":"2024-09-19T06:49:13.957411+0000","last_became_active":"2024-09-19T06:47:24.847889+0000","last_became_peered":"2024-09-19T06:47:24.847889+0000","last_unstale":"2024-09-19T06:49:13.957411+0000","last_undegraded":"2024-09-19T06:49:13.957411+0000","last_fullsized":"2024-09-19T06:49:13.957411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:32.574360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.924532+0000","last_change":"2024-09-19T06:47:22.832005+0000","last_active":"2024-09-19T06:49:13.924532+0000","last_peered":"2024-09-19T06:49:13.924532+0000","last_clean":"2024-09-19T06:49:13.924532+0000","last_became_active":"2024-09-19T06:47:22.831342+0000","last_became_peered":"2024-09-19T06:47:22.831342+0000","last_unstale":"2024-09-19T06:49:13.924532+0000","last_undegraded":"2024-09-19T06:49:13.924532+0000","last_fullsized":"2024-09-19T06:49:13.924532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:53.444748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.964849+0000","last_change":"2024-09-19T06:47:24.843328+0000","last_active":"2024-09-19T06:49:13.964849+0000","last_peered":"2024-09-19T06:49:13.964849+0000","last_clean":"2024-09-19T06:49:13.964849+0000","last_became_active":"2024-09-19T06:47:24.842344+0000","last_became_peered":"2024-09-19T06:47:24.842344+0000","last_unstale":"2024-09-19T06:49:13.964849+0000","last_undegraded":"2024-09-19T06:49:13.964849+0000","last_fullsized":"2024-09-19T06:49:13.964849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:11.247819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.926463+0000","last_change":"2024-09-19T06:47:40.844086+0000","last_active":"2024-09-19T06:49:13.926463+0000","last_peered":"2024-09-19T06:49:13.926463+0000","last_clean":"2024-09-19T06:49:13.926463+0000","last_became_active":"2024-09-19T06:47:40.835015+0000","last_became_peered":"2024-09-19T06:47:40.835015+0000","last_unstale":"2024-09-19T06:49:13.926463+0000","last_undegraded":"2024-09-19T06:49:13.926463+0000","last_fullsized":"2024-09-19T06:49:13.926463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:09.847070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.972972+0000","last_change":"2024-09-19T06:47:40.836466+0000","last_active":"2024-09-19T06:49:13.972972+0000","last_peered":"2024-09-19T06:49:13.972972+0000","last_clean":"2024-09-19T06:49:13.972972+0000","last_became_active":"2024-09-19T06:47:40.836237+0000","last_became_peered":"2024-09-19T06:47:40.836237+0000","last_unstale":"2024-09-19T06:49:13.972972+0000","last_undegraded":"2024-09-19T06:49:13.972972+0000","last_fullsized":"2024-09-19T06:49:13.972972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:00.094685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'11","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.617882+0000","last_change":"2024-09-19T06:47:22.830359+0000","last_active":"2024-09-19T06:49:23.617882+0000","last_peered":"2024-09-19T06:49:23.617882+0000","last_clean":"2024-09-19T06:49:23.617882+0000","last_became_active":"2024-09-19T06:47:22.829960+0000","last_became_peered":"2024-09-19T06:47:22.829960+0000","last_unstale":"2024-09-19T06:49:23.617882+0000","last_undegraded":"2024-09-19T06:49:23.617882+0000","last_fullsized":"2024-09-19T06:49:23.617882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:55.998243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2609,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":7,"num_write":11,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971872+0000","last_change":"2024-09-19T06:47:24.836668+0000","last_active":"2024-09-19T06:49:13.971872+0000","last_peered":"2024-09-19T06:49:13.971872+0000","last_clean":"2024-09-19T06:49:13.971872+0000","last_became_active":"2024-09-19T06:47:24.836393+0000","last_became_peered":"2024-09-19T06:47:24.836393+0000","last_unstale":"2024-09-19T06:49:13.971872+0000","last_undegraded":"2024-09-19T06:49:13.971872+0000","last_fullsized":"2024-09-19T06:49:13.971872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:26.248948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":66,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.931451+0000","last_change":"2024-09-19T06:47:40.858323+0000","last_active":"2024-09-19T06:49:13.931451+0000","last_peered":"2024-09-19T06:49:13.931451+0000","last_clean":"2024-09-19T06:49:13.931451+0000","last_became_active":"2024-09-19T06:47:40.840802+0000","last_became_peered":"2024-09-19T06:47:40.840802+0000","last_unstale":"2024-09-19T06:49:13.931451+0000","last_undegraded":"2024-09-19T06:49:13.931451+0000","last_fullsized":"2024-09-19T06:49:13.931451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:14.666916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970280+0000","last_change":"2024-09-19T06:47:24.843681+0000","last_active":"2024-09-19T06:49:13.970280+0000","last_peered":"2024-09-19T06:49:13.970280+0000","last_clean":"2024-09-19T06:49:13.970280+0000","last_became_active":"2024-09-19T06:47:24.841228+0000","last_became_peered":"2024-09-19T06:47:24.841228+0000","last_unstale":"2024-09-19T06:49:13.970280+0000","last_undegraded":"2024-09-19T06:49:13.970280+0000","last_fullsized":"2024-09-19T06:49:13.970280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:56.418962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.933661+0000","last_change":"2024-09-19T06:47:22.833893+0000","last_active":"2024-09-19T06:49:13.933661+0000","last_peered":"2024-09-19T06:49:13.933661+0000","last_clean":"2024-09-19T06:49:13.933661+0000","last_became_active":"2024-09-19T06:47:22.833217+0000","last_became_peered":"2024-09-19T06:47:22.833217+0000","last_unstale":"2024-09-19T06:49:13.933661+0000","last_undegraded":"2024-09-19T06:49:13.933661+0000","last_fullsized":"2024-09-19T06:49:13.933661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:19.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955096+0000","last_change":"2024-09-19T06:47:24.848399+0000","last_active":"2024-09-19T06:49:13.955096+0000","last_peered":"2024-09-19T06:49:13.955096+0000","last_clean":"2024-09-19T06:49:13.955096+0000","last_became_active":"2024-09-19T06:47:24.848081+0000","last_became_peered":"2024-09-19T06:47:24.848081+0000","last_unstale":"2024-09-19T06:49:13.955096+0000","last_undegraded":"2024-09-19T06:49:13.955096+0000","last_fullsized":"2024-09-19T06:49:13.955096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:44.292939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.931968+0000","last_change":"2024-09-19T06:47:40.860955+0000","last_active":"2024-09-19T06:49:13.931968+0000","last_peered":"2024-09-19T06:49:13.931968+0000","last_clean":"2024-09-19T06:49:13.931968+0000","last_became_active":"2024-09-19T06:47:40.835126+0000","last_became_peered":"2024-09-19T06:47:40.835126+0000","last_unstale":"2024-09-19T06:49:13.931968+0000","last_undegraded":"2024-09-19T06:49:13.931968+0000","last_fullsized":"2024-09-19T06:49:13.931968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:02:26.763229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971391+0000","last_change":"2024-09-19T06:47:24.849680+0000","last_active":"2024-09-19T06:49:13.971391+0000","last_peered":"2024-09-19T06:49:13.971391+0000","last_clean":"2024-09-19T06:49:13.971391+0000","last_became_active":"2024-09-19T06:47:24.849323+0000","last_became_peered":"2024-09-19T06:47:24.849323+0000","last_unstale":"2024-09-19T06:49:13.971391+0000","last_undegraded":"2024-09-19T06:49:13.971391+0000","last_fullsized":"2024-09-19T06:49:13.971391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:59.763503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'67","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:24.414577+0000","last_change":"2024-09-19T06:47:22.831689+0000","last_active":"2024-09-19T06:49:24.414577+0000","last_peered":"2024-09-19T06:49:24.414577+0000","last_clean":"2024-09-19T06:49:24.414577+0000","last_became_active":"2024-09-19T06:47:22.829283+0000","last_became_peered":"2024-09-19T06:47:22.829283+0000","last_unstale":"2024-09-19T06:49:24.414577+0000","last_undegraded":"2024-09-19T06:49:24.414577+0000","last_fullsized":"2024-09-19T06:49:24.414577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:37.109542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971792+0000","last_change":"2024-09-19T06:47:24.842793+0000","last_active":"2024-09-19T06:49:13.971792+0000","last_peered":"2024-09-19T06:49:13.971792+0000","last_clean":"2024-09-19T06:49:13.971792+0000","last_became_active":"2024-09-19T06:47:24.842474+0000","last_became_peered":"2024-09-19T06:47:24.842474+0000","last_unstale":"2024-09-19T06:49:13.971792+0000","last_undegraded":"2024-09-19T06:49:13.971792+0000","last_fullsized":"2024-09-19T06:49:13.971792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:23.110736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'10","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.617464+0000","last_change":"2024-09-19T06:47:23.047356+0000","last_active":"2024-09-19T06:49:23.617464+0000","last_peered":"2024-09-19T06:49:23.617464+0000","last_clean":"2024-09-19T06:49:23.617464+0000","last_became_active":"2024-09-19T06:47:23.046941+0000","last_became_peered":"2024-09-19T06:47:23.046941+0000","last_unstale":"2024-09-19T06:49:23.617464+0000","last_undegraded":"2024-09-19T06:49:23.617464+0000","last_fullsized":"2024-09-19T06:49:23.617464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:43.490952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":5,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971243+0000","last_change":"2024-09-19T06:47:24.851025+0000","last_active":"2024-09-19T06:49:13.971243+0000","last_peered":"2024-09-19T06:49:13.971243+0000","last_clean":"2024-09-19T06:49:13.971243+0000","last_became_active":"2024-09-19T06:47:24.850681+0000","last_became_peered":"2024-09-19T06:47:24.850681+0000","last_unstale":"2024-09-19T06:49:13.971243+0000","last_undegraded":"2024-09-19T06:49:13.971243+0000","last_fullsized":"2024-09-19T06:49:13.971243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:46.884747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"83'148","reported_seq":392,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:16.044071+0000","last_change":"2024-09-19T06:46:13.927379+0000","last_active":"2024-09-19T06:49:16.044071+0000","last_peered":"2024-09-19T06:49:16.044071+0000","last_clean":"2024-09-19T06:49:16.044071+0000","last_became_active":"2024-09-19T06:46:13.926926+0000","last_became_peered":"2024-09-19T06:46:13.926926+0000","last_unstale":"2024-09-19T06:49:16.044071+0000","last_undegraded":"2024-09-19T06:49:16.044071+0000","last_fullsized":"2024-09-19T06:49:16.044071+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:42:52.051335+0000","last_clean_scrub_stamp":"2024-09-19T06:42:52.051335+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:29.542128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.927403+0000","last_change":"2024-09-19T06:47:22.830387+0000","last_active":"2024-09-19T06:49:13.927403+0000","last_peered":"2024-09-19T06:49:13.927403+0000","last_clean":"2024-09-19T06:49:13.927403+0000","last_became_active":"2024-09-19T06:47:22.829970+0000","last_became_peered":"2024-09-19T06:47:22.829970+0000","last_unstale":"2024-09-19T06:49:13.927403+0000","last_undegraded":"2024-09-19T06:49:13.927403+0000","last_fullsized":"2024-09-19T06:49:13.927403+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:51.385456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.974671+0000","last_change":"2024-09-19T06:47:24.843752+0000","last_active":"2024-09-19T06:49:13.974671+0000","last_peered":"2024-09-19T06:49:13.974671+0000","last_clean":"2024-09-19T06:49:13.974671+0000","last_became_active":"2024-09-19T06:47:24.842631+0000","last_became_peered":"2024-09-19T06:47:24.842631+0000","last_unstale":"2024-09-19T06:49:13.974671+0000","last_undegraded":"2024-09-19T06:49:13.974671+0000","last_fullsized":"2024-09-19T06:49:13.974671+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:19.033519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.928732+0000","last_change":"2024-09-19T06:47:22.832647+0000","last_active":"2024-09-19T06:49:13.928732+0000","last_peered":"2024-09-19T06:49:13.928732+0000","last_clean":"2024-09-19T06:49:13.928732+0000","last_became_active":"2024-09-19T06:47:22.831774+0000","last_became_peered":"2024-09-19T06:47:22.831774+0000","last_unstale":"2024-09-19T06:49:13.928732+0000","last_undegraded":"2024-09-19T06:49:13.928732+0000","last_fullsized":"2024-09-19T06:49:13.928732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:57.824610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.948364+0000","last_change":"2024-09-19T06:47:24.844301+0000","last_active":"2024-09-19T06:49:13.948364+0000","last_peered":"2024-09-19T06:49:13.948364+0000","last_clean":"2024-09-19T06:49:13.948364+0000","last_became_active":"2024-09-19T06:47:24.839748+0000","last_became_peered":"2024-09-19T06:47:24.839748+0000","last_unstale":"2024-09-19T06:49:13.948364+0000","last_undegraded":"2024-09-19T06:49:13.948364+0000","last_fullsized":"2024-09-19T06:49:13.948364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:46.407693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":140,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.617287+0000","last_change":"2024-09-19T06:47:22.851008+0000","last_active":"2024-09-19T06:49:23.617287+0000","last_peered":"2024-09-19T06:49:23.617287+0000","last_clean":"2024-09-19T06:49:23.617287+0000","last_became_active":"2024-09-19T06:47:22.850601+0000","last_became_peered":"2024-09-19T06:47:22.850601+0000","last_unstale":"2024-09-19T06:49:23.617287+0000","last_undegraded":"2024-09-19T06:49:23.617287+0000","last_fullsized":"2024-09-19T06:49:23.617287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:04.610277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970407+0000","last_change":"2024-09-19T06:47:24.839684+0000","last_active":"2024-09-19T06:49:13.970407+0000","last_peered":"2024-09-19T06:49:13.970407+0000","last_clean":"2024-09-19T06:49:13.970407+0000","last_became_active":"2024-09-19T06:47:24.839126+0000","last_became_peered":"2024-09-19T06:47:24.839126+0000","last_unstale":"2024-09-19T06:49:13.970407+0000","last_undegraded":"2024-09-19T06:49:13.970407+0000","last_fullsized":"2024-09-19T06:49:13.970407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:09.458535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":140,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.325749+0000","last_change":"2024-09-19T06:47:22.831819+0000","last_active":"2024-09-19T06:49:23.325749+0000","last_peered":"2024-09-19T06:49:23.325749+0000","last_clean":"2024-09-19T06:49:23.325749+0000","last_became_active":"2024-09-19T06:47:22.831332+0000","last_became_peered":"2024-09-19T06:47:22.831332+0000","last_unstale":"2024-09-19T06:49:23.325749+0000","last_undegraded":"2024-09-19T06:49:23.325749+0000","last_fullsized":"2024-09-19T06:49:23.325749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:35.032949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.965109+0000","last_change":"2024-09-19T06:47:24.846738+0000","last_active":"2024-09-19T06:49:13.965109+0000","last_peered":"2024-09-19T06:49:13.965109+0000","last_clean":"2024-09-19T06:49:13.965109+0000","last_became_active":"2024-09-19T06:47:24.846389+0000","last_became_peered":"2024-09-19T06:47:24.846389+0000","last_unstale":"2024-09-19T06:49:13.965109+0000","last_undegraded":"2024-09-19T06:49:13.965109+0000","last_fullsized":"2024-09-19T06:49:13.965109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:19.658109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.918173+0000","last_change":"2024-09-19T06:47:22.851065+0000","last_active":"2024-09-19T06:49:13.918173+0000","last_peered":"2024-09-19T06:49:13.918173+0000","last_clean":"2024-09-19T06:49:13.918173+0000","last_became_active":"2024-09-19T06:47:22.850647+0000","last_became_peered":"2024-09-19T06:47:22.850647+0000","last_unstale":"2024-09-19T06:49:13.918173+0000","last_undegraded":"2024-09-19T06:49:13.918173+0000","last_fullsized":"2024-09-19T06:49:13.918173+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:41.993501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966795+0000","last_change":"2024-09-19T06:47:24.838905+0000","last_active":"2024-09-19T06:49:13.966795+0000","last_peered":"2024-09-19T06:49:13.966795+0000","last_clean":"2024-09-19T06:49:13.966795+0000","last_became_active":"2024-09-19T06:47:24.838443+0000","last_became_peered":"2024-09-19T06:47:24.838443+0000","last_unstale":"2024-09-19T06:49:13.966795+0000","last_undegraded":"2024-09-19T06:49:13.966795+0000","last_fullsized":"2024-09-19T06:49:13.966795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:12.486137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932653+0000","last_change":"2024-09-19T06:47:22.851431+0000","last_active":"2024-09-19T06:49:13.932653+0000","last_peered":"2024-09-19T06:49:13.932653+0000","last_clean":"2024-09-19T06:49:13.932653+0000","last_became_active":"2024-09-19T06:47:22.851092+0000","last_became_peered":"2024-09-19T06:47:22.851092+0000","last_unstale":"2024-09-19T06:49:13.932653+0000","last_undegraded":"2024-09-19T06:49:13.932653+0000","last_fullsized":"2024-09-19T06:49:13.932653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:26.183803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959143+0000","last_change":"2024-09-19T06:47:24.839046+0000","last_active":"2024-09-19T06:49:13.959143+0000","last_peered":"2024-09-19T06:49:13.959143+0000","last_clean":"2024-09-19T06:49:13.959143+0000","last_became_active":"2024-09-19T06:47:24.838680+0000","last_became_peered":"2024-09-19T06:47:24.838680+0000","last_unstale":"2024-09-19T06:49:13.959143+0000","last_undegraded":"2024-09-19T06:49:13.959143+0000","last_fullsized":"2024-09-19T06:49:13.959143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:45.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'102","reported_seq":231,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:24.747427+0000","last_change":"2024-09-19T06:47:40.865241+0000","last_active":"2024-09-19T06:49:24.747427+0000","last_peered":"2024-09-19T06:49:24.747427+0000","last_clean":"2024-09-19T06:49:24.747427+0000","last_became_active":"2024-09-19T06:47:40.837765+0000","last_became_peered":"2024-09-19T06:47:40.837765+0000","last_unstale":"2024-09-19T06:49:24.747427+0000","last_undegraded":"2024-09-19T06:49:24.747427+0000","last_fullsized":"2024-09-19T06:49:24.747427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:13.052976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":186,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.966118+0000","last_change":"2024-09-19T06:47:24.848599+0000","last_active":"2024-09-19T06:49:13.966118+0000","last_peered":"2024-09-19T06:49:13.966118+0000","last_clean":"2024-09-19T06:49:13.966118+0000","last_became_active":"2024-09-19T06:47:24.848305+0000","last_became_peered":"2024-09-19T06:47:24.848305+0000","last_unstale":"2024-09-19T06:49:13.966118+0000","last_undegraded":"2024-09-19T06:49:13.966118+0000","last_fullsized":"2024-09-19T06:49:13.966118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:31.496188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.927755+0000","last_change":"2024-09-19T06:47:23.047484+0000","last_active":"2024-09-19T06:49:13.927755+0000","last_peered":"2024-09-19T06:49:13.927755+0000","last_clean":"2024-09-19T06:49:13.927755+0000","last_became_active":"2024-09-19T06:47:23.047058+0000","last_became_peered":"2024-09-19T06:47:23.047058+0000","last_unstale":"2024-09-19T06:49:13.927755+0000","last_undegraded":"2024-09-19T06:49:13.927755+0000","last_fullsized":"2024-09-19T06:49:13.927755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:34.752772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.970265+0000","last_change":"2024-09-19T06:47:24.843498+0000","last_active":"2024-09-19T06:49:13.970265+0000","last_peered":"2024-09-19T06:49:13.970265+0000","last_clean":"2024-09-19T06:49:13.970265+0000","last_became_active":"2024-09-19T06:47:24.843204+0000","last_became_peered":"2024-09-19T06:47:24.843204+0000","last_unstale":"2024-09-19T06:49:13.970265+0000","last_undegraded":"2024-09-19T06:49:13.970265+0000","last_fullsized":"2024-09-19T06:49:13.970265+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:25.585023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.919373+0000","last_change":"2024-09-19T06:47:22.851519+0000","last_active":"2024-09-19T06:49:13.919373+0000","last_peered":"2024-09-19T06:49:13.919373+0000","last_clean":"2024-09-19T06:49:13.919373+0000","last_became_active":"2024-09-19T06:47:22.851163+0000","last_became_peered":"2024-09-19T06:47:22.851163+0000","last_unstale":"2024-09-19T06:49:13.919373+0000","last_undegraded":"2024-09-19T06:49:13.919373+0000","last_fullsized":"2024-09-19T06:49:13.919373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:04.502795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.945308+0000","last_change":"2024-09-19T06:47:24.845239+0000","last_active":"2024-09-19T06:49:13.945308+0000","last_peered":"2024-09-19T06:49:13.945308+0000","last_clean":"2024-09-19T06:49:13.945308+0000","last_became_active":"2024-09-19T06:47:24.844934+0000","last_became_peered":"2024-09-19T06:47:24.844934+0000","last_unstale":"2024-09-19T06:49:13.945308+0000","last_undegraded":"2024-09-19T06:49:13.945308+0000","last_fullsized":"2024-09-19T06:49:13.945308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:43.610604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.934380+0000","last_change":"2024-09-19T06:47:22.830232+0000","last_active":"2024-09-19T06:49:13.934380+0000","last_peered":"2024-09-19T06:49:13.934380+0000","last_clean":"2024-09-19T06:49:13.934380+0000","last_became_active":"2024-09-19T06:47:22.829901+0000","last_became_peered":"2024-09-19T06:47:22.829901+0000","last_unstale":"2024-09-19T06:49:13.934380+0000","last_undegraded":"2024-09-19T06:49:13.934380+0000","last_fullsized":"2024-09-19T06:49:13.934380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:12.251427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.955845+0000","last_change":"2024-09-19T06:47:24.839304+0000","last_active":"2024-09-19T06:49:13.955845+0000","last_peered":"2024-09-19T06:49:13.955845+0000","last_clean":"2024-09-19T06:49:13.955845+0000","last_became_active":"2024-09-19T06:47:24.838986+0000","last_became_peered":"2024-09-19T06:47:24.838986+0000","last_unstale":"2024-09-19T06:49:13.955845+0000","last_undegraded":"2024-09-19T06:49:13.955845+0000","last_fullsized":"2024-09-19T06:49:13.955845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:39.377886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:14.885319+0000","last_change":"2024-09-19T06:47:22.828553+0000","last_active":"2024-09-19T06:49:14.885319+0000","last_peered":"2024-09-19T06:49:14.885319+0000","last_clean":"2024-09-19T06:49:14.885319+0000","last_became_active":"2024-09-19T06:47:22.828213+0000","last_became_peered":"2024-09-19T06:47:22.828213+0000","last_unstale":"2024-09-19T06:49:14.885319+0000","last_undegraded":"2024-09-19T06:49:14.885319+0000","last_fullsized":"2024-09-19T06:49:14.885319+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:31.649761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.960387+0000","last_change":"2024-09-19T06:47:24.840747+0000","last_active":"2024-09-19T06:49:13.960387+0000","last_peered":"2024-09-19T06:49:13.960387+0000","last_clean":"2024-09-19T06:49:13.960387+0000","last_became_active":"2024-09-19T06:47:24.840451+0000","last_became_peered":"2024-09-19T06:47:24.840451+0000","last_unstale":"2024-09-19T06:49:13.960387+0000","last_undegraded":"2024-09-19T06:49:13.960387+0000","last_fullsized":"2024-09-19T06:49:13.960387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:41.588751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.933845+0000","last_change":"2024-09-19T06:47:22.831697+0000","last_active":"2024-09-19T06:49:13.933845+0000","last_peered":"2024-09-19T06:49:13.933845+0000","last_clean":"2024-09-19T06:49:13.933845+0000","last_became_active":"2024-09-19T06:47:22.831178+0000","last_became_peered":"2024-09-19T06:47:22.831178+0000","last_unstale":"2024-09-19T06:49:13.933845+0000","last_undegraded":"2024-09-19T06:49:13.933845+0000","last_fullsized":"2024-09-19T06:49:13.933845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:30.975782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971917+0000","last_change":"2024-09-19T06:47:24.845097+0000","last_active":"2024-09-19T06:49:13.971917+0000","last_peered":"2024-09-19T06:49:13.971917+0000","last_clean":"2024-09-19T06:49:13.971917+0000","last_became_active":"2024-09-19T06:47:24.844801+0000","last_became_peered":"2024-09-19T06:47:24.844801+0000","last_unstale":"2024-09-19T06:49:13.971917+0000","last_undegraded":"2024-09-19T06:49:13.971917+0000","last_fullsized":"2024-09-19T06:49:13.971917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:15.264118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.971733+0000","last_change":"2024-09-19T06:47:24.851372+0000","last_active":"2024-09-19T06:49:13.971733+0000","last_peered":"2024-09-19T06:49:13.971733+0000","last_clean":"2024-09-19T06:49:13.971733+0000","last_became_active":"2024-09-19T06:47:24.851044+0000","last_became_peered":"2024-09-19T06:47:24.851044+0000","last_unstale":"2024-09-19T06:49:13.971733+0000","last_undegraded":"2024-09-19T06:49:13.971733+0000","last_fullsized":"2024-09-19T06:49:13.971733+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:55.658214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932580+0000","last_change":"2024-09-19T06:47:22.833694+0000","last_active":"2024-09-19T06:49:13.932580+0000","last_peered":"2024-09-19T06:49:13.932580+0000","last_clean":"2024-09-19T06:49:13.932580+0000","last_became_active":"2024-09-19T06:47:22.833215+0000","last_became_peered":"2024-09-19T06:47:22.833215+0000","last_unstale":"2024-09-19T06:49:13.932580+0000","last_undegraded":"2024-09-19T06:49:13.932580+0000","last_fullsized":"2024-09-19T06:49:13.932580+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:32.171522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973551+0000","last_change":"2024-09-19T06:47:24.845737+0000","last_active":"2024-09-19T06:49:13.973551+0000","last_peered":"2024-09-19T06:49:13.973551+0000","last_clean":"2024-09-19T06:49:13.973551+0000","last_became_active":"2024-09-19T06:47:24.845242+0000","last_became_peered":"2024-09-19T06:47:24.845242+0000","last_unstale":"2024-09-19T06:49:13.973551+0000","last_undegraded":"2024-09-19T06:49:13.973551+0000","last_fullsized":"2024-09-19T06:49:13.973551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:51.554946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.932499+0000","last_change":"2024-09-19T06:47:22.829869+0000","last_active":"2024-09-19T06:49:13.932499+0000","last_peered":"2024-09-19T06:49:13.932499+0000","last_clean":"2024-09-19T06:49:13.932499+0000","last_became_active":"2024-09-19T06:47:22.829522+0000","last_became_peered":"2024-09-19T06:47:22.829522+0000","last_unstale":"2024-09-19T06:49:13.932499+0000","last_undegraded":"2024-09-19T06:49:13.932499+0000","last_fullsized":"2024-09-19T06:49:13.932499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:26.871434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.973419+0000","last_change":"2024-09-19T06:47:24.842023+0000","last_active":"2024-09-19T06:49:13.973419+0000","last_peered":"2024-09-19T06:49:13.973419+0000","last_clean":"2024-09-19T06:49:13.973419+0000","last_became_active":"2024-09-19T06:47:24.841289+0000","last_became_peered":"2024-09-19T06:47:24.841289+0000","last_unstale":"2024-09-19T06:49:13.973419+0000","last_undegraded":"2024-09-19T06:49:13.973419+0000","last_fullsized":"2024-09-19T06:49:13.973419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:21:12.611458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.937218+0000","last_change":"2024-09-19T06:47:22.852198+0000","last_active":"2024-09-19T06:49:13.937218+0000","last_peered":"2024-09-19T06:49:13.937218+0000","last_clean":"2024-09-19T06:49:13.937218+0000","last_became_active":"2024-09-19T06:47:22.850984+0000","last_became_peered":"2024-09-19T06:47:22.850984+0000","last_unstale":"2024-09-19T06:49:13.937218+0000","last_undegraded":"2024-09-19T06:49:13.937218+0000","last_fullsized":"2024-09-19T06:49:13.937218+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:13.620145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:13.959682+0000","last_change":"2024-09-19T06:47:40.837614+0000","last_active":"2024-09-19T06:49:13.959682+0000","last_peered":"2024-09-19T06:49:13.959682+0000","last_clean":"2024-09-19T06:49:13.959682+0000","last_became_active":"2024-09-19T06:47:40.837259+0000","last_became_peered":"2024-09-19T06:47:40.837259+0000","last_unstale":"2024-09-19T06:49:13.959682+0000","last_undegraded":"2024-09-19T06:49:13.959682+0000","last_fullsized":"2024-09-19T06:49:13.959682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:23.819447+0000","last_clean_scrub_stamp":"2024-09-19T06:47:23.819447+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:41.429962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":168,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T06:49:23.534111+0000","last_change":"2024-09-19T06:47:41.182639+0000","last_active":"2024-09-19T06:49:23.534111+0000","last_peered":"2024-09-19T06:49:23.534111+0000","last_clean":"2024-09-19T06:49:23.534111+0000","last_became_active":"2024-09-19T06:47:41.174305+0000","last_became_peered":"2024-09-19T06:47:41.174305+0000","last_unstale":"2024-09-19T06:49:23.534111+0000","last_undegraded":"2024-09-19T06:49:23.534111+0000","last_fullsized":"2024-09-19T06:49:23.534111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:47:21.805542+0000","last_clean_scrub_stamp":"2024-09-19T06:47:21.805542+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:10.199502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":802788,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1473,"num_read_kb":956,"num_write":508,"num_write_kb":2390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2777088,"data_stored":2408364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":387,"ondisk_log_size":387,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712684,"statfs":{"total":95995035648,"available":95961788416,"internally_reserved":0,"allocated":1314816,"data_stored":842671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29060,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716092,"statfs":{"total":95995035648,"available":95965278208,"internally_reserved":0,"allocated":1961984,"data_stored":1439332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972399,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28328,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716824,"statfs":{"total":95995035648,"available":95966027776,"internally_reserved":0,"allocated":1335296,"data_stored":840798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234036,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":447288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32684,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712468,"statfs":{"total":95995035648,"available":95961567232,"internally_reserved":0,"allocated":1536000,"data_stored":1042894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626494,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":447692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855425,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28976,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716176,"statfs":{"total":95995035648,"available":95965364224,"internally_reserved":0,"allocated":1875968,"data_stored":1368300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018951,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":449293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215176,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1667072,"data_stored":1168572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":520824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":519675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":450351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":74364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":331355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":402387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":396211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:49:30.340 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:49:30.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:30 smithi134 ceph-mon[35356]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:49:30.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:30 smithi134 ceph-mon[35356]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-09-19T06:49:30.587 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.037+0000 7fb4788c5640 1 -- 172.21.15.103:0/1919559978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb474101de0 msgr2=0x7fb4741021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.037+0000 7fb4788c5640 1 --2- 172.21.15.103:0/1919559978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb474101de0 0x7fb4741021c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb468009900 tx=0x7fb46802a410 comp rx=0 tx=0).stop 2024-09-19T06:49:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.037+0000 7fb4788c5640 1 -- 172.21.15.103:0/1919559978 shutdown_connections 2024-09-19T06:49:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.037+0000 7fb4788c5640 1 --2- 172.21.15.103:0/1919559978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb474106f10 0x7fb47410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.037+0000 7fb4788c5640 1 --2- 172.21.15.103:0/1919559978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb474102790 0x7fb4741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.037+0000 7fb4788c5640 1 --2- 172.21.15.103:0/1919559978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb474101de0 0x7fb4741021c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.037+0000 7fb4788c5640 1 -- 172.21.15.103:0/1919559978 >> 172.21.15.103:0/1919559978 conn(0x7fb4740fd8c0 msgr2=0x7fb4740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.037+0000 7fb4788c5640 1 -- 172.21.15.103:0/1919559978 shutdown_connections 2024-09-19T06:49:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.037+0000 7fb4788c5640 1 -- 172.21.15.103:0/1919559978 wait complete. 2024-09-19T06:49:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb4788c5640 1 Processor -- start 2024-09-19T06:49:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb4788c5640 1 -- start start 2024-09-19T06:49:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb4788c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb474101de0 0x7fb4741984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb4788c5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb474102790 0x7fb474198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb4788c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb474106f10 0x7fb47419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb4788c5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb474111f70 con 0x7fb474106f10 2024-09-19T06:49:31.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb4788c5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb474111df0 con 0x7fb474101de0 2024-09-19T06:49:31.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb4788c5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741120f0 con 0x7fb474102790 2024-09-19T06:49:31.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb472d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb474106f10 0x7fb47419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb472d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb474106f10 0x7fb47419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40376/0 (socket says 172.21.15.103:40376) 2024-09-19T06:49:31.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb472d9d640 1 -- 172.21.15.103:0/2444202003 learned_addr learned my addr 172.21.15.103:0/2444202003 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:31.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb47259c640 1 --2- 172.21.15.103:0/2444202003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb474101de0 0x7fb4741984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb471d9b640 1 --2- 172.21.15.103:0/2444202003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb474102790 0x7fb474198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb472d9d640 1 -- 172.21.15.103:0/2444202003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb474102790 msgr2=0x7fb474198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:31.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb472d9d640 1 --2- 172.21.15.103:0/2444202003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb474102790 0x7fb474198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb472d9d640 1 -- 172.21.15.103:0/2444202003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb474101de0 msgr2=0x7fb4741984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:31.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb472d9d640 1 --2- 172.21.15.103:0/2444202003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb474101de0 0x7fb4741984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.038+0000 7fb472d9d640 1 -- 172.21.15.103:0/2444202003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47419d480 con 0x7fb474106f10 2024-09-19T06:49:31.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.039+0000 7fb47259c640 1 --2- 172.21.15.103:0/2444202003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb474101de0 0x7fb4741984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:31.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.039+0000 7fb472d9d640 1 --2- 172.21.15.103:0/2444202003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb474106f10 0x7fb47419cda0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00d750 tx=0x7fb46c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:31.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.039+0000 7fb4637fe640 1 -- 172.21.15.103:0/2444202003 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c017070 con 0x7fb474106f10 2024-09-19T06:49:31.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.039+0000 7fb4637fe640 1 -- 172.21.15.103:0/2444202003 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb46c005500 con 0x7fb474106f10 2024-09-19T06:49:31.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.039+0000 7fb4788c5640 1 -- 172.21.15.103:0/2444202003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47419d770 con 0x7fb474106f10 2024-09-19T06:49:31.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.039+0000 7fb4637fe640 1 -- 172.21.15.103:0/2444202003 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c004d90 con 0x7fb474106f10 2024-09-19T06:49:31.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.039+0000 7fb4788c5640 1 -- 172.21.15.103:0/2444202003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4741a5050 con 0x7fb474106f10 2024-09-19T06:49:31.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.040+0000 7fb4788c5640 1 -- 172.21.15.103:0/2444202003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4741021c0 con 0x7fb474106f10 2024-09-19T06:49:31.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.041+0000 7fb4637fe640 1 -- 172.21.15.103:0/2444202003 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7fb46c004f30 con 0x7fb474106f10 2024-09-19T06:49:31.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.042+0000 7fb4637fe640 1 --2- 172.21.15.103:0/2444202003 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb45c0a34d0 0x7fb45c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.042+0000 7fb47259c640 1 --2- 172.21.15.103:0/2444202003 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb45c0a34d0 0x7fb45c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.042+0000 7fb4637fe640 1 -- 172.21.15.103:0/2444202003 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fb46c0cf9c0 con 0x7fb474106f10 2024-09-19T06:49:31.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.042+0000 7fb47259c640 1 --2- 172.21.15.103:0/2444202003 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb45c0a34d0 0x7fb45c0a5990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb468005ab0 tx=0x7fb468005a20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:31.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.046+0000 7fb4637fe640 1 -- 172.21.15.103:0/2444202003 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb46c09c160 con 0x7fb474106f10 2024-09-19T06:49:31.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.247+0000 7fb4788c5640 1 -- 172.21.15.103:0/2444202003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb47410a0f0 con 0x7fb474106f10 2024-09-19T06:49:31.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.247+0000 7fb4637fe640 1 -- 172.21.15.103:0/2444202003 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fb46c09ba60 con 0x7fb474106f10 2024-09-19T06:49:31.250 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:49:31.250 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:49:31.250 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:49:31.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.250+0000 7fb4788c5640 1 -- 172.21.15.103:0/2444202003 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb45c0a34d0 msgr2=0x7fb45c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:31.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.250+0000 7fb4788c5640 1 --2- 172.21.15.103:0/2444202003 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb45c0a34d0 0x7fb45c0a5990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb468005ab0 tx=0x7fb468005a20 comp rx=0 tx=0).stop 2024-09-19T06:49:31.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.250+0000 7fb4788c5640 1 -- 172.21.15.103:0/2444202003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb474106f10 msgr2=0x7fb47419cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:31.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.250+0000 7fb4788c5640 1 --2- 172.21.15.103:0/2444202003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb474106f10 0x7fb47419cda0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00d750 tx=0x7fb46c00dc10 comp rx=0 tx=0).stop 2024-09-19T06:49:31.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.250+0000 7fb4788c5640 1 -- 172.21.15.103:0/2444202003 shutdown_connections 2024-09-19T06:49:31.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.250+0000 7fb4788c5640 1 --2- 172.21.15.103:0/2444202003 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb45c0a34d0 0x7fb45c0a5990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.250+0000 7fb4788c5640 1 --2- 172.21.15.103:0/2444202003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb474106f10 0x7fb47419cda0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.250+0000 7fb4788c5640 1 --2- 172.21.15.103:0/2444202003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb474102790 0x7fb474198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.250+0000 7fb4788c5640 1 --2- 172.21.15.103:0/2444202003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb474101de0 0x7fb4741984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.250+0000 7fb4788c5640 1 -- 172.21.15.103:0/2444202003 >> 172.21.15.103:0/2444202003 conn(0x7fb4740fd8c0 msgr2=0x7fb4740fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:31.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.250+0000 7fb4788c5640 1 -- 172.21.15.103:0/2444202003 shutdown_connections 2024-09-19T06:49:31.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:31.250+0000 7fb4788c5640 1 -- 172.21.15.103:0/2444202003 wait complete. 2024-09-19T06:49:31.572 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:31 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2444202003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:31.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:31 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2444202003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:31.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:31 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2444202003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:31.858 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:49:32.107 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:32.492 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:32 smithi103 ceph-mon[31158]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-19T06:49:32.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:32 smithi174 ceph-mon[35637]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-19T06:49:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.569+0000 7ff21c2d6640 1 -- 172.21.15.103:0/2388455983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214102790 msgr2=0x7ff2141067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.569+0000 7ff21c2d6640 1 --2- 172.21.15.103:0/2388455983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214102790 0x7ff2141067e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff208009900 tx=0x7ff20802a410 comp rx=0 tx=0).stop 2024-09-19T06:49:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.569+0000 7ff21c2d6640 1 -- 172.21.15.103:0/2388455983 shutdown_connections 2024-09-19T06:49:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.569+0000 7ff21c2d6640 1 --2- 172.21.15.103:0/2388455983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff214106f10 0x7ff21410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.569+0000 7ff21c2d6640 1 --2- 172.21.15.103:0/2388455983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214102790 0x7ff2141067e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.569+0000 7ff21c2d6640 1 --2- 172.21.15.103:0/2388455983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff214101de0 0x7ff2141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.569+0000 7ff21c2d6640 1 -- 172.21.15.103:0/2388455983 >> 172.21.15.103:0/2388455983 conn(0x7ff2140fd8c0 msgr2=0x7ff2140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.569+0000 7ff21c2d6640 1 -- 172.21.15.103:0/2388455983 shutdown_connections 2024-09-19T06:49:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.569+0000 7ff21c2d6640 1 -- 172.21.15.103:0/2388455983 wait complete. 2024-09-19T06:49:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff21c2d6640 1 Processor -- start 2024-09-19T06:49:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff21c2d6640 1 -- start start 2024-09-19T06:49:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff21c2d6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff214101de0 0x7ff2141985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff21c2d6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214102790 0x7ff214198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff21c2d6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff214106f10 0x7ff21419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff21c2d6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff214112010 con 0x7ff214102790 2024-09-19T06:49:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff21c2d6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff214111e90 con 0x7ff214106f10 2024-09-19T06:49:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff21c2d6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff214112190 con 0x7ff214101de0 2024-09-19T06:49:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff219871640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214102790 0x7ff214198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff219871640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214102790 0x7ff214198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40400/0 (socket says 172.21.15.103:40400) 2024-09-19T06:49:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff219871640 1 -- 172.21.15.103:0/154960626 learned_addr learned my addr 172.21.15.103:0/154960626 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff21a873640 1 --2- 172.21.15.103:0/154960626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff214106f10 0x7ff21419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff21a072640 1 --2- 172.21.15.103:0/154960626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff214101de0 0x7ff2141985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff219871640 1 -- 172.21.15.103:0/154960626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff214101de0 msgr2=0x7ff2141985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff219871640 1 --2- 172.21.15.103:0/154960626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff214101de0 0x7ff2141985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff219871640 1 -- 172.21.15.103:0/154960626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff214106f10 msgr2=0x7ff21419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff219871640 1 --2- 172.21.15.103:0/154960626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff214106f10 0x7ff21419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.570+0000 7ff219871640 1 -- 172.21.15.103:0/154960626 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21419d560 con 0x7ff214102790 2024-09-19T06:49:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.571+0000 7ff219871640 1 --2- 172.21.15.103:0/154960626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214102790 0x7ff214198af0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff208005ab0 tx=0x7ff2080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.571+0000 7ff2137fe640 1 -- 172.21.15.103:0/154960626 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20803f030 con 0x7ff214102790 2024-09-19T06:49:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.571+0000 7ff2137fe640 1 -- 172.21.15.103:0/154960626 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff208003ce0 con 0x7ff214102790 2024-09-19T06:49:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.571+0000 7ff21c2d6640 1 -- 172.21.15.103:0/154960626 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21419d7f0 con 0x7ff214102790 2024-09-19T06:49:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.571+0000 7ff2137fe640 1 -- 172.21.15.103:0/154960626 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208035070 con 0x7ff214102790 2024-09-19T06:49:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.571+0000 7ff21c2d6640 1 -- 172.21.15.103:0/154960626 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2141a5130 con 0x7ff214102790 2024-09-19T06:49:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.572+0000 7ff21c2d6640 1 -- 172.21.15.103:0/154960626 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2141032c0 con 0x7ff214102790 2024-09-19T06:49:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.573+0000 7ff2137fe640 1 -- 172.21.15.103:0/154960626 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7ff208042030 con 0x7ff214102790 2024-09-19T06:49:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.574+0000 7ff2137fe640 1 --2- 172.21.15.103:0/154960626 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1fc0a3550 0x7ff1fc0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.574+0000 7ff21a072640 1 --2- 172.21.15.103:0/154960626 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1fc0a3550 0x7ff1fc0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:32.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.574+0000 7ff2137fe640 1 -- 172.21.15.103:0/154960626 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff2080f0b20 con 0x7ff214102790 2024-09-19T06:49:32.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.574+0000 7ff21a072640 1 --2- 172.21.15.103:0/154960626 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1fc0a3550 0x7ff1fc0a5a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009890 tx=0x7ff20c008040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:32.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.577+0000 7ff2137fe640 1 -- 172.21.15.103:0/154960626 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff2080bd350 con 0x7ff214102790 2024-09-19T06:49:32.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:32 smithi134 ceph-mon[35356]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-19T06:49:32.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.779+0000 7ff21c2d6640 1 -- 172.21.15.103:0/154960626 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff21410a0f0 con 0x7ff214102790 2024-09-19T06:49:32.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.780+0000 7ff2137fe640 1 -- 172.21.15.103:0/154960626 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7ff2080bcc50 con 0x7ff214102790 2024-09-19T06:49:32.782 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:49:32.782 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:49:32.782 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:49:32.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.782+0000 7ff21c2d6640 1 -- 172.21.15.103:0/154960626 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1fc0a3550 msgr2=0x7ff1fc0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:32.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.782+0000 7ff21c2d6640 1 --2- 172.21.15.103:0/154960626 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1fc0a3550 0x7ff1fc0a5a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009890 tx=0x7ff20c008040 comp rx=0 tx=0).stop 2024-09-19T06:49:32.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.782+0000 7ff21c2d6640 1 -- 172.21.15.103:0/154960626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214102790 msgr2=0x7ff214198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:32.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.782+0000 7ff21c2d6640 1 --2- 172.21.15.103:0/154960626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214102790 0x7ff214198af0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff208005ab0 tx=0x7ff2080042f0 comp rx=0 tx=0).stop 2024-09-19T06:49:32.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.782+0000 7ff21c2d6640 1 -- 172.21.15.103:0/154960626 shutdown_connections 2024-09-19T06:49:32.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.782+0000 7ff21c2d6640 1 --2- 172.21.15.103:0/154960626 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1fc0a3550 0x7ff1fc0a5a10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:32.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.782+0000 7ff21c2d6640 1 --2- 172.21.15.103:0/154960626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff214106f10 0x7ff21419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:32.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.782+0000 7ff21c2d6640 1 --2- 172.21.15.103:0/154960626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff214102790 0x7ff214198af0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:32.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.782+0000 7ff21c2d6640 1 --2- 172.21.15.103:0/154960626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff214101de0 0x7ff2141985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:32.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.782+0000 7ff21c2d6640 1 -- 172.21.15.103:0/154960626 >> 172.21.15.103:0/154960626 conn(0x7ff2140fd8c0 msgr2=0x7ff2140fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:32.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.782+0000 7ff21c2d6640 1 -- 172.21.15.103:0/154960626 shutdown_connections 2024-09-19T06:49:32.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:32.782+0000 7ff21c2d6640 1 -- 172.21.15.103:0/154960626 wait complete. 2024-09-19T06:49:33.450 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:49:33.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:33 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/154960626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:33.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:33 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/154960626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:33.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:33 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/154960626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:33.699 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:34.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.086+0000 7f40a2f67640 1 -- 172.21.15.103:0/2271690413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f409c102ef0 msgr2=0x7f409c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:34.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.086+0000 7f40a2f67640 1 --2- 172.21.15.103:0/2271690413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f409c102ef0 0x7f409c10da40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4084009900 tx=0x7f408402a480 comp rx=0 tx=0).stop 2024-09-19T06:49:34.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.087+0000 7f40a2f67640 1 -- 172.21.15.103:0/2271690413 shutdown_connections 2024-09-19T06:49:34.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.087+0000 7f40a2f67640 1 --2- 172.21.15.103:0/2271690413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f409c102ef0 0x7f409c10da40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.087+0000 7f40a2f67640 1 --2- 172.21.15.103:0/2271690413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f409c102570 0x7f409c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.087+0000 7f40a2f67640 1 --2- 172.21.15.103:0/2271690413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f409c100e20 0x7f409c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.087+0000 7f40a2f67640 1 -- 172.21.15.103:0/2271690413 >> 172.21.15.103:0/2271690413 conn(0x7f409c0fc7d0 msgr2=0x7f409c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:34.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.087+0000 7f40a2f67640 1 -- 172.21.15.103:0/2271690413 shutdown_connections 2024-09-19T06:49:34.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.087+0000 7f40a2f67640 1 -- 172.21.15.103:0/2271690413 wait complete. 2024-09-19T06:49:34.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.087+0000 7f40a2f67640 1 Processor -- start 2024-09-19T06:49:34.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.087+0000 7f40a2f67640 1 -- start start 2024-09-19T06:49:34.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a2f67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f409c100e20 0x7f409c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a2f67640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f409c102570 0x7f409c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a2f67640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f409c102ef0 0x7f409c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a2f67640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f409c111020 con 0x7f409c100e20 2024-09-19T06:49:34.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a2f67640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f409c110ea0 con 0x7f409c102ef0 2024-09-19T06:49:34.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a2f67640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1111a0 con 0x7f409c102570 2024-09-19T06:49:34.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a0d03640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f409c100e20 0x7f409c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a0d03640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f409c100e20 0x7f409c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40408/0 (socket says 172.21.15.103:40408) 2024-09-19T06:49:34.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a0d03640 1 -- 172.21.15.103:0/232257371 learned_addr learned my addr 172.21.15.103:0/232257371 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:34.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f409bfff640 1 --2- 172.21.15.103:0/232257371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f409c102570 0x7f409c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a1504640 1 --2- 172.21.15.103:0/232257371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f409c102ef0 0x7f409c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a0d03640 1 -- 172.21.15.103:0/232257371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f409c102570 msgr2=0x7f409c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:34.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a0d03640 1 --2- 172.21.15.103:0/232257371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f409c102570 0x7f409c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a0d03640 1 -- 172.21.15.103:0/232257371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f409c102ef0 msgr2=0x7f409c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:34.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a0d03640 1 --2- 172.21.15.103:0/232257371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f409c102ef0 0x7f409c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a0d03640 1 -- 172.21.15.103:0/232257371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c1a6aa0 con 0x7f409c100e20 2024-09-19T06:49:34.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a1504640 1 --2- 172.21.15.103:0/232257371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f409c102ef0 0x7f409c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:34.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.088+0000 7f40a0d03640 1 --2- 172.21.15.103:0/232257371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f409c100e20 0x7f409c0fcc80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f409400b4a0 tx=0x7f409400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:34.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.089+0000 7f4099ffb640 1 -- 172.21.15.103:0/232257371 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094014070 con 0x7f409c100e20 2024-09-19T06:49:34.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.089+0000 7f40a2f67640 1 -- 172.21.15.103:0/232257371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409c1a6d90 con 0x7f409c100e20 2024-09-19T06:49:34.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.089+0000 7f4099ffb640 1 -- 172.21.15.103:0/232257371 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4094010af0 con 0x7f409c100e20 2024-09-19T06:49:34.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.089+0000 7f4099ffb640 1 -- 172.21.15.103:0/232257371 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094017d10 con 0x7f409c100e20 2024-09-19T06:49:34.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.089+0000 7f40a2f67640 1 -- 172.21.15.103:0/232257371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409c1a7250 con 0x7f409c100e20 2024-09-19T06:49:34.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.090+0000 7f40a2f67640 1 -- 172.21.15.103:0/232257371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f409c101280 con 0x7f409c100e20 2024-09-19T06:49:34.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.091+0000 7f4099ffb640 1 -- 172.21.15.103:0/232257371 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f4094017070 con 0x7f409c100e20 2024-09-19T06:49:34.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.092+0000 7f4099ffb640 1 --2- 172.21.15.103:0/232257371 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f40880a34d0 0x7f40880a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.092+0000 7f409bfff640 1 --2- 172.21.15.103:0/232257371 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f40880a34d0 0x7f40880a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.092+0000 7f4099ffb640 1 -- 172.21.15.103:0/232257371 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f40940cf220 con 0x7f409c100e20 2024-09-19T06:49:34.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.093+0000 7f409bfff640 1 --2- 172.21.15.103:0/232257371 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f40880a34d0 0x7f40880a5990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f409c0fe710 tx=0x7f408c00d040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:34.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.096+0000 7f4099ffb640 1 -- 172.21.15.103:0/232257371 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f409409bad0 con 0x7f409c100e20 2024-09-19T06:49:34.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.273+0000 7f40a2f67640 1 -- 172.21.15.103:0/232257371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f409c10a0a0 con 0x7f409c100e20 2024-09-19T06:49:34.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.274+0000 7f4099ffb640 1 -- 172.21.15.103:0/232257371 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+23173 (secure 0 0 0) 0x7f409409b3d0 con 0x7f409c100e20 2024-09-19T06:49:34.277 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:49:34.277 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":83,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","created":"2024-09-19T06:39:46.840574+0000","modified":"2024-09-19T06:49:13.909427+0000","last_up_change":"2024-09-19T06:46:35.652960+0000","last_in_change":"2024-09-19T06:46:11.630585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:42:52.050881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:47:21.192611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:47:23.643291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d45dab3-e434-4783-b2e3-f658c3ba583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6803","nonce":3157697740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6805","nonce":3157697740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6809","nonce":3157697740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6807","nonce":3157697740}]},"public_addr":"172.21.15.103:6803/3157697740","cluster_addr":"172.21.15.103:6805/3157697740","heartbeat_back_addr":"172.21.15.103:6809/3157697740","heartbeat_front_addr":"172.21.15.103:6807/3157697740","state":["exists","up"]},{"osd":1,"uuid":"ee0e92bd-15ab-4867-9a18-20abb9f5fc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6801","nonce":1418301195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6803","nonce":1418301195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6807","nonce":1418301195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6805","nonce":1418301195}]},"public_addr":"172.21.15.134:6801/1418301195","cluster_addr":"172.21.15.134:6803/1418301195","heartbeat_back_addr":"172.21.15.134:6807/1418301195","heartbeat_front_addr":"172.21.15.134:6805/1418301195","state":["exists","up"]},{"osd":2,"uuid":"0230caed-025b-46e9-a011-237601d16ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6801","nonce":2292242290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6803","nonce":2292242290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6807","nonce":2292242290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6805","nonce":2292242290}]},"public_addr":"172.21.15.174:6801/2292242290","cluster_addr":"172.21.15.174:6803/2292242290","heartbeat_back_addr":"172.21.15.174:6807/2292242290","heartbeat_front_addr":"172.21.15.174:6805/2292242290","state":["exists","up"]},{"osd":3,"uuid":"c54a8399-bf4b-4d70-b207-a13d85f99590","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6811","nonce":2698986333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6813","nonce":2698986333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6817","nonce":2698986333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6815","nonce":2698986333}]},"public_addr":"172.21.15.103:6811/2698986333","cluster_addr":"172.21.15.103:6813/2698986333","heartbeat_back_addr":"172.21.15.103:6817/2698986333","heartbeat_front_addr":"172.21.15.103:6815/2698986333","state":["exists","up"]},{"osd":4,"uuid":"7c046be6-464a-4ab5-812b-1781121fc317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6809","nonce":3169891968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6811","nonce":3169891968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6815","nonce":3169891968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6813","nonce":3169891968}]},"public_addr":"172.21.15.134:6809/3169891968","cluster_addr":"172.21.15.134:6811/3169891968","heartbeat_back_addr":"172.21.15.134:6815/3169891968","heartbeat_front_addr":"172.21.15.134:6813/3169891968","state":["exists","up"]},{"osd":5,"uuid":"8622596b-6a06-459c-882d-50fba9e296ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6809","nonce":763749644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6811","nonce":763749644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6815","nonce":763749644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6813","nonce":763749644}]},"public_addr":"172.21.15.174:6809/763749644","cluster_addr":"172.21.15.174:6811/763749644","heartbeat_back_addr":"172.21.15.174:6815/763749644","heartbeat_front_addr":"172.21.15.174:6813/763749644","state":["exists","up"]},{"osd":6,"uuid":"64d858d6-a8b3-4f65-a187-af9ab1ccee0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6819","nonce":1396972394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6821","nonce":1396972394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6825","nonce":1396972394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6823","nonce":1396972394}]},"public_addr":"172.21.15.103:6819/1396972394","cluster_addr":"172.21.15.103:6821/1396972394","heartbeat_back_addr":"172.21.15.103:6825/1396972394","heartbeat_front_addr":"172.21.15.103:6823/1396972394","state":["exists","up"]},{"osd":7,"uuid":"d405d1c6-839c-4c92-8261-4adb3163c90f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6817","nonce":2196366765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6819","nonce":2196366765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6823","nonce":2196366765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6821","nonce":2196366765}]},"public_addr":"172.21.15.134:6817/2196366765","cluster_addr":"172.21.15.134:6819/2196366765","heartbeat_back_addr":"172.21.15.134:6823/2196366765","heartbeat_front_addr":"172.21.15.134:6821/2196366765","state":["exists","up"]},{"osd":8,"uuid":"4c452cd9-816c-4f77-9368-2cd4bbeee4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6817","nonce":189977465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6819","nonce":189977465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6823","nonce":189977465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6821","nonce":189977465}]},"public_addr":"172.21.15.174:6817/189977465","cluster_addr":"172.21.15.174:6819/189977465","heartbeat_back_addr":"172.21.15.174:6823/189977465","heartbeat_front_addr":"172.21.15.174:6821/189977465","state":["exists","up"]},{"osd":9,"uuid":"fb3662e0-2d8e-45c4-9119-6d08b35443da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6827","nonce":3161966413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6829","nonce":3161966413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6833","nonce":3161966413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6831","nonce":3161966413}]},"public_addr":"172.21.15.103:6827/3161966413","cluster_addr":"172.21.15.103:6829/3161966413","heartbeat_back_addr":"172.21.15.103:6833/3161966413","heartbeat_front_addr":"172.21.15.103:6831/3161966413","state":["exists","up"]},{"osd":10,"uuid":"e52c3cbe-a6af-441e-b423-b94f3a2da544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6825","nonce":2532758632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6827","nonce":2532758632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6831","nonce":2532758632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6829","nonce":2532758632}]},"public_addr":"172.21.15.134:6825/2532758632","cluster_addr":"172.21.15.134:6827/2532758632","heartbeat_back_addr":"172.21.15.134:6831/2532758632","heartbeat_front_addr":"172.21.15.134:6829/2532758632","state":["exists","up"]},{"osd":11,"uuid":"9040f3f8-3873-4dea-8716-dcda5811c8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6825","nonce":161916442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6827","nonce":161916442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6831","nonce":161916442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6829","nonce":161916442}]},"public_addr":"172.21.15.174:6825/161916442","cluster_addr":"172.21.15.174:6827/161916442","heartbeat_back_addr":"172.21.15.174:6831/161916442","heartbeat_front_addr":"172.21.15.174:6829/161916442","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:41:59.656854+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:24.797551+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:49.685497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:33.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:39.324983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:03.987310+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:28.747971+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:53.827780+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:18.554450+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:43.705691+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:08.401275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:33.502542+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2169657832":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/1596229922":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1642809745":"2024-09-20T06:40:16.780890+0000","172.21.15.103:6800/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/70266056":"2024-09-20T06:49:13.909393+0000","172.21.15.103:6801/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1496153385":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/471230312":"2024-09-20T06:49:13.909393+0000","172.21.15.103:6801/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/2309624637":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/3727259725":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/2590139910":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/2050139492":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/2867049580":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/2590139910":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/1074662744":"2024-09-20T06:49:13.909393+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:49:34.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.277+0000 7f40a2f67640 1 -- 172.21.15.103:0/232257371 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f40880a34d0 msgr2=0x7f40880a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:34.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.277+0000 7f40a2f67640 1 --2- 172.21.15.103:0/232257371 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f40880a34d0 0x7f40880a5990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f409c0fe710 tx=0x7f408c00d040 comp rx=0 tx=0).stop 2024-09-19T06:49:34.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.277+0000 7f40a2f67640 1 -- 172.21.15.103:0/232257371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f409c100e20 msgr2=0x7f409c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:34.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.277+0000 7f40a2f67640 1 --2- 172.21.15.103:0/232257371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f409c100e20 0x7f409c0fcc80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f409400b4a0 tx=0x7f409400b960 comp rx=0 tx=0).stop 2024-09-19T06:49:34.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.277+0000 7f40a2f67640 1 -- 172.21.15.103:0/232257371 shutdown_connections 2024-09-19T06:49:34.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.277+0000 7f40a2f67640 1 --2- 172.21.15.103:0/232257371 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f40880a34d0 0x7f40880a5990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.277+0000 7f40a2f67640 1 --2- 172.21.15.103:0/232257371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f409c102ef0 0x7f409c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.277+0000 7f40a2f67640 1 --2- 172.21.15.103:0/232257371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f409c102570 0x7f409c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.277+0000 7f40a2f67640 1 --2- 172.21.15.103:0/232257371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f409c100e20 0x7f409c0fcc80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.277+0000 7f40a2f67640 1 -- 172.21.15.103:0/232257371 >> 172.21.15.103:0/232257371 conn(0x7f409c0fc7d0 msgr2=0x7f409c06a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:34.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.277+0000 7f40a2f67640 1 -- 172.21.15.103:0/232257371 shutdown_connections 2024-09-19T06:49:34.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:34.277+0000 7f40a2f67640 1 -- 172.21.15.103:0/232257371 wait complete. 2024-09-19T06:49:34.365 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:34 smithi103 ceph-mon[31158]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-19T06:49:34.365 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:34 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/232257371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:49:34.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:34 smithi174 ceph-mon[35637]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-19T06:49:34.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:34 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/232257371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:49:34.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:34 smithi134 ceph-mon[35356]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-19T06:49:34.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:34 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/232257371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:49:35.059 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-19T06:49:35.314 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:35.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:35 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T06:49:35.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:35 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T06:49:35.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:35 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T06:49:36.428 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:36 smithi103 ceph-mon[31158]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-19T06:49:36.457 INFO:teuthology.orchestra.run.smithi103.stdout:{"run_dir":"/var/run/ceph"} 2024-09-19T06:49:36.458 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-19T06:49:36.458 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1747 2024-09-19T06:49:36.458 DEBUG:tasks.quiescer:Yielding 2024-09-19T06:49:36.458 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T06:49:36.474 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T06:49:36.474 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T06:49:36.474 INFO:tasks.workunit:timeout=3h 2024-09-19T06:49:36.474 INFO:tasks.workunit:cleanup=True 2024-09-19T06:49:36.475 DEBUG:teuthology.orchestra.run.smithi103:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T06:49:36.475 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:49:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-19T06:49:36.493 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-19T06:49:36.493 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-19T06:49:36.493 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:49:36.493 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-19T06:49:36.493 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-19 06:47:43.665814736 +0000 2024-09-19T06:49:36.493 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-19 06:47:43.665814736 +0000 2024-09-19T06:49:36.493 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-19 06:49:13.937278701 +0000 2024-09-19T06:49:36.493 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-19 06:47:43.665814736 +0000 2024-09-19T06:49:36.493 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-19T06:49:36.494 DEBUG:teuthology.orchestra.run.smithi103:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-19T06:49:36.566 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T06:49:36.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:36 smithi174 ceph-mon[35637]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-19T06:49:36.622 INFO:tasks.workunit.client.0.smithi103.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T06:49:36.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:36 smithi134 ceph-mon[35356]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-19T06:49:36.732 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:37.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.214+0000 7f4d2d1e2640 1 -- 172.21.15.103:0/2960065849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28106a60 msgr2=0x7f4d28106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:37.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.214+0000 7f4d2d1e2640 1 --2- 172.21.15.103:0/2960065849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28106a60 0x7f4d28106e40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009930 tx=0x7f4d1802a420 comp rx=0 tx=0).stop 2024-09-19T06:49:37.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.214+0000 7f4d2d1e2640 1 -- 172.21.15.103:0/2960065849 shutdown_connections 2024-09-19T06:49:37.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.214+0000 7f4d2d1e2640 1 --2- 172.21.15.103:0/2960065849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d28101400 0x7f4d2810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:37.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.214+0000 7f4d2d1e2640 1 --2- 172.21.15.103:0/2960065849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d28100a60 0x7f4d28100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:37.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.214+0000 7f4d2d1e2640 1 --2- 172.21.15.103:0/2960065849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28106a60 0x7f4d28106e40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:37.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.214+0000 7f4d2d1e2640 1 -- 172.21.15.103:0/2960065849 >> 172.21.15.103:0/2960065849 conn(0x7f4d280fc780 msgr2=0x7f4d280feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:37.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.214+0000 7f4d2d1e2640 1 -- 172.21.15.103:0/2960065849 shutdown_connections 2024-09-19T06:49:37.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.214+0000 7f4d2d1e2640 1 -- 172.21.15.103:0/2960065849 wait complete. 2024-09-19T06:49:37.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.215+0000 7f4d2d1e2640 1 Processor -- start 2024-09-19T06:49:37.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.215+0000 7f4d2d1e2640 1 -- start start 2024-09-19T06:49:37.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.215+0000 7f4d2d1e2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28100a60 0x7f4d28197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:37.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.215+0000 7f4d2d1e2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d28101400 0x7f4d281978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:37.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.215+0000 7f4d2d1e2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d28106a60 0x7f4d2819bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:37.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.215+0000 7f4d2d1e2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28110e20 con 0x7f4d28100a60 2024-09-19T06:49:37.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.215+0000 7f4d2d1e2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28110ca0 con 0x7f4d28106a60 2024-09-19T06:49:37.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.215+0000 7f4d2d1e2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28110fa0 con 0x7f4d28101400 2024-09-19T06:49:37.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.215+0000 7f4d26d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28100a60 0x7f4d28197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:37.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.215+0000 7f4d26d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28100a60 0x7f4d28197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40426/0 (socket says 172.21.15.103:40426) 2024-09-19T06:49:37.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.215+0000 7f4d26d9d640 1 -- 172.21.15.103:0/612770731 learned_addr learned my addr 172.21.15.103:0/612770731 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:37.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.216+0000 7f4d2759e640 1 --2- 172.21.15.103:0/612770731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d28106a60 0x7f4d2819bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:37.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.216+0000 7f4d2659c640 1 --2- 172.21.15.103:0/612770731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d28101400 0x7f4d281978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:37.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.216+0000 7f4d26d9d640 1 -- 172.21.15.103:0/612770731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d28101400 msgr2=0x7f4d281978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:37.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.216+0000 7f4d26d9d640 1 --2- 172.21.15.103:0/612770731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d28101400 0x7f4d281978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:37.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.216+0000 7f4d26d9d640 1 -- 172.21.15.103:0/612770731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d28106a60 msgr2=0x7f4d2819bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:37.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.216+0000 7f4d26d9d640 1 --2- 172.21.15.103:0/612770731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d28106a60 0x7f4d2819bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:37.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.216+0000 7f4d26d9d640 1 -- 172.21.15.103:0/612770731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2819c320 con 0x7f4d28100a60 2024-09-19T06:49:37.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.216+0000 7f4d2659c640 1 --2- 172.21.15.103:0/612770731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d28101400 0x7f4d281978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:37.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.216+0000 7f4d26d9d640 1 --2- 172.21.15.103:0/612770731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28100a60 0x7f4d28197370 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009230 tx=0x7f4d18004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:37.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.216+0000 7f4d0ffff640 1 -- 172.21.15.103:0/612770731 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18046070 con 0x7f4d28100a60 2024-09-19T06:49:37.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.216+0000 7f4d2d1e2640 1 -- 172.21.15.103:0/612770731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2819c5b0 con 0x7f4d28100a60 2024-09-19T06:49:37.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.216+0000 7f4d0ffff640 1 -- 172.21.15.103:0/612770731 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d18003ad0 con 0x7f4d28100a60 2024-09-19T06:49:37.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.216+0000 7f4d0ffff640 1 -- 172.21.15.103:0/612770731 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18035050 con 0x7f4d28100a60 2024-09-19T06:49:37.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.216+0000 7f4d2d1e2640 1 -- 172.21.15.103:0/612770731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2806a790 con 0x7f4d28100a60 2024-09-19T06:49:37.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.217+0000 7f4d2d1e2640 1 -- 172.21.15.103:0/612770731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d281021a0 con 0x7f4d28100a60 2024-09-19T06:49:37.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.218+0000 7f4d0ffff640 1 -- 172.21.15.103:0/612770731 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f4d18049070 con 0x7f4d28100a60 2024-09-19T06:49:37.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.219+0000 7f4d0ffff640 1 --2- 172.21.15.103:0/612770731 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4d100a34d0 0x7f4d100a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:37.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.219+0000 7f4d0ffff640 1 -- 172.21.15.103:0/612770731 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f4d180f1c50 con 0x7f4d28100a60 2024-09-19T06:49:37.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.219+0000 7f4d2659c640 1 --2- 172.21.15.103:0/612770731 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4d100a34d0 0x7f4d100a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:37.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.220+0000 7f4d2659c640 1 --2- 172.21.15.103:0/612770731 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4d100a34d0 0x7f4d100a5990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0058c0 tx=0x7f4d1c005850 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:37.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.223+0000 7f4d0ffff640 1 -- 172.21.15.103:0/612770731 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4d18050080 con 0x7f4d28100a60 2024-09-19T06:49:37.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.430+0000 7f4d2d1e2640 1 -- 172.21.15.103:0/612770731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d2806b1b0 con 0x7f4d28100a60 2024-09-19T06:49:37.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.431+0000 7f4d0ffff640 1 -- 172.21.15.103:0/612770731 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f4d180be4c0 con 0x7f4d28100a60 2024-09-19T06:49:37.433 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:49:37.433 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:49:37.434 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:49:37.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.433+0000 7f4d2d1e2640 1 -- 172.21.15.103:0/612770731 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4d100a34d0 msgr2=0x7f4d100a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:37.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.433+0000 7f4d2d1e2640 1 --2- 172.21.15.103:0/612770731 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4d100a34d0 0x7f4d100a5990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0058c0 tx=0x7f4d1c005850 comp rx=0 tx=0).stop 2024-09-19T06:49:37.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.433+0000 7f4d2d1e2640 1 -- 172.21.15.103:0/612770731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28100a60 msgr2=0x7f4d28197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:37.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.433+0000 7f4d2d1e2640 1 --2- 172.21.15.103:0/612770731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28100a60 0x7f4d28197370 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009230 tx=0x7f4d18004270 comp rx=0 tx=0).stop 2024-09-19T06:49:37.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.433+0000 7f4d2d1e2640 1 -- 172.21.15.103:0/612770731 shutdown_connections 2024-09-19T06:49:37.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.433+0000 7f4d2d1e2640 1 --2- 172.21.15.103:0/612770731 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4d100a34d0 0x7f4d100a5990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:37.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.433+0000 7f4d2d1e2640 1 --2- 172.21.15.103:0/612770731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d28106a60 0x7f4d2819bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:37.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.433+0000 7f4d2d1e2640 1 --2- 172.21.15.103:0/612770731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d28101400 0x7f4d281978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:37.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.433+0000 7f4d2d1e2640 1 --2- 172.21.15.103:0/612770731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d28100a60 0x7f4d28197370 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:37.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.433+0000 7f4d2d1e2640 1 -- 172.21.15.103:0/612770731 >> 172.21.15.103:0/612770731 conn(0x7f4d280fc780 msgr2=0x7f4d280fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:37.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.433+0000 7f4d2d1e2640 1 -- 172.21.15.103:0/612770731 shutdown_connections 2024-09-19T06:49:37.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:37.433+0000 7f4d2d1e2640 1 -- 172.21.15.103:0/612770731 wait complete. 2024-09-19T06:49:37.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:37 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/612770731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:37.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:37 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/612770731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:37.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:37 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/612770731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:38.133 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T06:47:25.446176+0000', 'modified': '2024-09-19T06:49:14.707403+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24482, 'mds_1': 24455, 'mds_2': 24454, 'mds_3': 24436, 'mds_4': 24431}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14586': {'gid': 14586, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6839/3119326077', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 3119326077}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 3119326077}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14598': {'gid': 14598, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6841/3771437304', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6840', 'nonce': 3771437304}, {'type': 'v1', 'addr': '172.21.15.103:6841', 'nonce': 3771437304}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24397': {'gid': 24397, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/2319563070', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 2319563070}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 2319563070}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/1556743259', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 1556743259}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 1556743259}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/963732203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 963732203}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 963732203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'e', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.134:6835/3092414367', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 3092414367}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 3092414367}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.174:6837/1053388911', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 1053388911}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 1053388911}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.174:6839/4202229983', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 4202229983}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 4202229983}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.134:6837/1590453330', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 1590453330}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 1590453330}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.134:6839/2333076106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 2333076106}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 2333076106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24431, 24436, 24454, 24455, 24482]} 2024-09-19T06:49:38.133 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T06:49:38.133 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.e status 2024-09-19T06:49:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.348+0000 7fce3d2c0640 1 -- 172.21.15.103:0/3136800755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce381035c0 msgr2=0x7fce3810baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.348+0000 7fce3d2c0640 1 --2- 172.21.15.103:0/3136800755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce381035c0 0x7fce3810baa0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009900 tx=0x7fce2c02a410 comp rx=0 tx=0).stop 2024-09-19T06:49:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.348+0000 7fce3d2c0640 1 -- 172.21.15.103:0/3136800755 shutdown_connections 2024-09-19T06:49:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.348+0000 7fce3d2c0640 1 --2- 172.21.15.103:0/3136800755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce3810c0d0 0x7fce3810e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.348+0000 7fce3d2c0640 1 --2- 172.21.15.103:0/3136800755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce381035c0 0x7fce3810baa0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.348+0000 7fce3d2c0640 1 --2- 172.21.15.103:0/3136800755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce38102ca0 0x7fce38103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.348+0000 7fce3d2c0640 1 -- 172.21.15.103:0/3136800755 >> 172.21.15.103:0/3136800755 conn(0x7fce380fc6b0 msgr2=0x7fce380fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.348+0000 7fce3d2c0640 1 -- 172.21.15.103:0/3136800755 shutdown_connections 2024-09-19T06:49:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.348+0000 7fce3d2c0640 1 -- 172.21.15.103:0/3136800755 wait complete. 2024-09-19T06:49:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.349+0000 7fce3d2c0640 1 Processor -- start 2024-09-19T06:49:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.349+0000 7fce3d2c0640 1 -- start start 2024-09-19T06:49:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.349+0000 7fce3d2c0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce38102ca0 0x7fce38198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.349+0000 7fce3d2c0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce381035c0 0x7fce381988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.349+0000 7fce3d2c0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce3810c0d0 0x7fce3819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.349+0000 7fce3d2c0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fce38111ed0 con 0x7fce38102ca0 2024-09-19T06:49:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.349+0000 7fce3d2c0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce38111d50 con 0x7fce381035c0 2024-09-19T06:49:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.349+0000 7fce3d2c0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fce38112050 con 0x7fce3810c0d0 2024-09-19T06:49:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.349+0000 7fce36ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce38102ca0 0x7fce38198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.349+0000 7fce36ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce38102ca0 0x7fce38198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37966/0 (socket says 172.21.15.103:37966) 2024-09-19T06:49:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.349+0000 7fce36ffd640 1 -- 172.21.15.103:0/4199307521 learned_addr learned my addr 172.21.15.103:0/4199307521 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.349+0000 7fce377fe640 1 --2- 172.21.15.103:0/4199307521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce3810c0d0 0x7fce3819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.349+0000 7fce367fc640 1 --2- 172.21.15.103:0/4199307521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce381035c0 0x7fce381988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.350+0000 7fce36ffd640 1 -- 172.21.15.103:0/4199307521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce3810c0d0 msgr2=0x7fce3819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.350+0000 7fce36ffd640 1 --2- 172.21.15.103:0/4199307521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce3810c0d0 0x7fce3819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.350+0000 7fce36ffd640 1 -- 172.21.15.103:0/4199307521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce381035c0 msgr2=0x7fce381988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.350+0000 7fce36ffd640 1 --2- 172.21.15.103:0/4199307521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce381035c0 0x7fce381988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.350+0000 7fce36ffd640 1 -- 172.21.15.103:0/4199307521 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3819d350 con 0x7fce38102ca0 2024-09-19T06:49:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.350+0000 7fce367fc640 1 --2- 172.21.15.103:0/4199307521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce381035c0 0x7fce381988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.350+0000 7fce36ffd640 1 --2- 172.21.15.103:0/4199307521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce38102ca0 0x7fce38198360 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fce2800e9f0 tx=0x7fce2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.350+0000 7fce1ffff640 1 -- 172.21.15.103:0/4199307521 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2800cd80 con 0x7fce38102ca0 2024-09-19T06:49:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.350+0000 7fce3d2c0640 1 -- 172.21.15.103:0/4199307521 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3819d640 con 0x7fce38102ca0 2024-09-19T06:49:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.350+0000 7fce1ffff640 1 -- 172.21.15.103:0/4199307521 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce28005500 con 0x7fce38102ca0 2024-09-19T06:49:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.350+0000 7fce1ffff640 1 -- 172.21.15.103:0/4199307521 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28010690 con 0x7fce38102ca0 2024-09-19T06:49:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.350+0000 7fce3d2c0640 1 -- 172.21.15.103:0/4199307521 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce381a4ed0 con 0x7fce38102ca0 2024-09-19T06:49:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.352+0000 7fce1ffff640 1 -- 172.21.15.103:0/4199307521 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7fce280046c0 con 0x7fce38102ca0 2024-09-19T06:49:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce40054740 1 Processor -- start 2024-09-19T06:49:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce40054740 1 -- start start 2024-09-19T06:49:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce40054740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a869d0 0x561e94a86db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce40054740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a872f0 0x561e94a95b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce40054740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a960c0 0x561e94a984b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce40054740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561e949eb2f0 con 0x561e94a869d0 2024-09-19T06:49:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce40054740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561e94993270 con 0x561e94a872f0 2024-09-19T06:49:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce40054740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561e949e72b0 con 0x561e94a960c0 2024-09-19T06:49:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce1ffff640 1 --2- 172.21.15.103:0/4199307521 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fce200a3580 0x7fce200a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a869d0 0x561e94a86db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a869d0 0x561e94a86db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37968/0 (socket says 172.21.15.103:37968) 2024-09-19T06:49:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce377fe640 1 -- 172.21.15.103:0/592638592 learned_addr learned my addr 172.21.15.103:0/592638592 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce377fe640 1 --2- 172.21.15.103:0/4199307521 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fce200a3580 0x7fce200a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce36ffd640 1 --2- 172.21.15.103:0/592638592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a872f0 0x561e94a95b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce367fc640 1 --2- 172.21.15.103:0/592638592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a960c0 0x561e94a984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce1ffff640 1 -- 172.21.15.103:0/4199307521 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fce280d0440 con 0x7fce38102ca0 2024-09-19T06:49:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce377fe640 1 -- 172.21.15.103:0/592638592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a960c0 msgr2=0x561e94a984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce377fe640 1 --2- 172.21.15.103:0/592638592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a960c0 0x561e94a984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce377fe640 1 -- 172.21.15.103:0/592638592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a872f0 msgr2=0x561e94a95b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce377fe640 1 --2- 172.21.15.103:0/592638592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a872f0 0x561e94a95b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.353+0000 7fce377fe640 1 -- 172.21.15.103:0/592638592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e94992340 con 0x561e94a869d0 2024-09-19T06:49:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.354+0000 7fce377fe640 1 --2- 172.21.15.103:0/592638592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a869d0 0x561e94a86db0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fce30014eb0 tx=0x7fce300117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af1f8159be762d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.354+0000 7fce377fe640 1 --2- 172.21.15.103:0/4199307521 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fce200a3580 0x7fce200a5a40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce30011cb0 tx=0x7fce30016000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.354+0000 7fce1d7fa640 1 -- 172.21.15.103:0/592638592 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30024070 con 0x561e94a869d0 2024-09-19T06:49:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.354+0000 7fce1d7fa640 1 -- 172.21.15.103:0/592638592 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce3000ce30 con 0x561e94a869d0 2024-09-19T06:49:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.354+0000 7fce1d7fa640 1 -- 172.21.15.103:0/592638592 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30020680 con 0x561e94a869d0 2024-09-19T06:49:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 -- 172.21.15.103:0/592638592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a869d0 msgr2=0x561e94a86db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 --2- 172.21.15.103:0/592638592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a869d0 0x561e94a86db0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fce30014eb0 tx=0x7fce300117b0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 -- 172.21.15.103:0/592638592 shutdown_connections 2024-09-19T06:49:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 --2- 172.21.15.103:0/592638592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a960c0 0x561e94a984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 --2- 172.21.15.103:0/592638592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a872f0 0x561e94a95b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 --2- 172.21.15.103:0/592638592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a869d0 0x561e94a86db0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 -- 172.21.15.103:0/592638592 >> 172.21.15.103:0/592638592 conn(0x561e94a83060 msgr2=0x561e94a83440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 -- 172.21.15.103:0/592638592 shutdown_connections 2024-09-19T06:49:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 -- 172.21.15.103:0/592638592 wait complete. 2024-09-19T06:49:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 -- start start 2024-09-19T06:49:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 Processor -- start 2024-09-19T06:49:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a869d0 0x561e94a93070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a872f0 0x561e94a935b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a960c0 0x561e94a93af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561e94a98c90 con 0x561e94a872f0 2024-09-19T06:49:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561e94974290 con 0x561e94a869d0 2024-09-19T06:49:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.355+0000 7fce40054740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561e94a98e10 con 0x561e94a960c0 2024-09-19T06:49:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce40054740 10 client.0 _ll_register_callbacks cb 0x561e949a5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce367fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a872f0 0x561e94a935b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce367fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a872f0 0x561e94a935b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37980/0 (socket says 172.21.15.103:37980) 2024-09-19T06:49:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce367fc640 1 -- 172.21.15.103:0/2396603169 learned_addr learned my addr 172.21.15.103:0/2396603169 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce36ffd640 1 --2- 172.21.15.103:0/2396603169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a869d0 0x561e94a93070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce377fe640 1 --2- 172.21.15.103:0/2396603169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a960c0 0x561e94a93af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce367fc640 1 -- 172.21.15.103:0/2396603169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a960c0 msgr2=0x561e94a93af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce367fc640 1 --2- 172.21.15.103:0/2396603169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a960c0 0x561e94a93af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce367fc640 1 -- 172.21.15.103:0/2396603169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a869d0 msgr2=0x561e94a93070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce367fc640 1 --2- 172.21.15.103:0/2396603169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a869d0 0x561e94a93070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce367fc640 1 -- 172.21.15.103:0/2396603169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e946fe210 con 0x561e94a872f0 2024-09-19T06:49:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce377fe640 1 --2- 172.21.15.103:0/2396603169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a960c0 0x561e94a93af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce367fc640 1 --2- 172.21.15.103:0/2396603169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a872f0 0x561e94a935b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fce2c031550 tx=0x7fce2c0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5a48e6436767f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce1aff5640 1 -- 172.21.15.103:0/2396603169 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c040070 con 0x561e94a872f0 2024-09-19T06:49:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce40054740 1 -- 172.21.15.103:0/2396603169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce3000ce30 con 0x561e94a872f0 2024-09-19T06:49:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce1aff5640 1 -- 172.21.15.103:0/2396603169 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fce2c004240 con 0x561e94a872f0 2024-09-19T06:49:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce1aff5640 1 -- 172.21.15.103:0/2396603169 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fce2c035040 con 0x561e94a872f0 2024-09-19T06:49:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce40054740 10 client.15468 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.356+0000 7fce40054740 1 -- 172.21.15.103:0/2396603169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e947ca620 con 0x561e94a872f0 2024-09-19T06:49:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.357+0000 7fce1aff5640 1 -- 172.21.15.103:0/2396603169 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce3000ce30 con 0x561e94a872f0 2024-09-19T06:49:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.357+0000 7fce1aff5640 1 -- 172.21.15.103:0/2396603169 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fce2c004900 con 0x561e94a872f0 2024-09-19T06:49:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.357+0000 7fce40054740 10 client.15468 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.357+0000 7fce40054740 10 client.15468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.357+0000 7fce40054740 20 client.15468 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.357+0000 7fce40054740 1 --2- 172.21.15.103:0/2396603169 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561e94aa60c0 0x561e94aa84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.357+0000 7fce40054740 4 client.15468 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.357+0000 7fce40054740 1 -- 172.21.15.103:0/2396603169 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e949f8a20 con 0x561e94aa60c0 2024-09-19T06:49:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.357+0000 7fce36ffd640 1 --2- 172.21.15.103:0/2396603169 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561e94aa60c0 0x561e94aa84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.358+0000 7fce1aff5640 10 client.15468 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.358+0000 7fce36ffd640 1 --2- 172.21.15.103:0/2396603169 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561e94aa60c0 0x561e94aa84b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9352a059ddf2caad server_cookie=801b958f2583b749 in_seq=0 out_seq=0 2024-09-19T06:49:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.358+0000 7fce1aff5640 1 -- 172.21.15.103:0/2396603169 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e949f8a20 con 0x561e94aa60c0 2024-09-19T06:49:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.358+0000 7fce1aff5640 10 client.15468 handle_command_reply: tid=0 2024-09-19T06:49:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.358+0000 7fce40054740 1 client.15468 shutdown 2024-09-19T06:49:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.359+0000 7fce40054740 1 -- 172.21.15.103:0/2396603169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a872f0 msgr2=0x561e94a935b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.359+0000 7fce40054740 1 --2- 172.21.15.103:0/2396603169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a872f0 0x561e94a935b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fce2c031550 tx=0x7fce2c0315b0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.359+0000 7fce40054740 1 -- 172.21.15.103:0/2396603169 shutdown_connections 2024-09-19T06:49:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.359+0000 7fce40054740 1 --2- 172.21.15.103:0/2396603169 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561e94aa60c0 0x561e94aa84b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.359+0000 7fce40054740 1 --2- 172.21.15.103:0/2396603169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a960c0 0x561e94a93af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.359+0000 7fce40054740 1 --2- 172.21.15.103:0/2396603169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a872f0 0x561e94a935b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.359+0000 7fce40054740 1 --2- 172.21.15.103:0/2396603169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a869d0 0x561e94a93070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.359+0000 7fce1aff5640 0 client.15468 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.359+0000 7fce40054740 1 -- 172.21.15.103:0/2396603169 >> 172.21.15.103:0/2396603169 conn(0x561e94a83060 msgr2=0x561e94a88d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.359+0000 7fce40054740 1 -- 172.21.15.103:0/2396603169 shutdown_connections 2024-09-19T06:49:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.359+0000 7fce40054740 1 -- 172.21.15.103:0/2396603169 wait complete. 2024-09-19T06:49:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.359+0000 7fce40054740 20 client.15468 trim_cache size 0 max 16384 2024-09-19T06:49:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.374+0000 7fce40054740 1 Processor -- start 2024-09-19T06:49:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.374+0000 7fce40054740 1 -- start start 2024-09-19T06:49:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce40054740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a869d0 0x561e94a93920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce40054740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a872f0 0x561e94a94fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce40054740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a960c0 0x561e94aadda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce40054740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fce381ab660 con 0x561e94a872f0 2024-09-19T06:49:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce40054740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce381ad1a0 con 0x561e94a960c0 2024-09-19T06:49:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce40054740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fce381abd30 con 0x561e94a869d0 2024-09-19T06:49:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce367fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a872f0 0x561e94a94fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce367fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a872f0 0x561e94a94fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37990/0 (socket says 172.21.15.103:37990) 2024-09-19T06:49:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce367fc640 1 -- 172.21.15.103:0/2522588973 learned_addr learned my addr 172.21.15.103:0/2522588973 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce377fe640 1 --2- 172.21.15.103:0/2522588973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a960c0 0x561e94aadda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce36ffd640 1 --2- 172.21.15.103:0/2522588973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a869d0 0x561e94a93920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce367fc640 1 -- 172.21.15.103:0/2522588973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a869d0 msgr2=0x561e94a93920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce367fc640 1 --2- 172.21.15.103:0/2522588973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a869d0 0x561e94a93920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce367fc640 1 -- 172.21.15.103:0/2522588973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a960c0 msgr2=0x561e94aadda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce367fc640 1 --2- 172.21.15.103:0/2522588973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a960c0 0x561e94aadda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce367fc640 1 -- 172.21.15.103:0/2522588973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e94a93e60 con 0x561e94a872f0 2024-09-19T06:49:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce367fc640 1 --2- 172.21.15.103:0/2522588973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a872f0 0x561e94a94fe0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fce2c004cd0 tx=0x7fce2c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e51839bda018c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce1cff9640 1 -- 172.21.15.103:0/2522588973 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c051040 con 0x561e94a872f0 2024-09-19T06:49:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce1cff9640 1 -- 172.21.15.103:0/2522588973 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce2c009520 con 0x561e94a872f0 2024-09-19T06:49:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.375+0000 7fce1cff9640 1 -- 172.21.15.103:0/2522588973 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c040040 con 0x561e94a872f0 2024-09-19T06:49:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.376+0000 7fce40054740 1 -- 172.21.15.103:0/2522588973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a872f0 msgr2=0x561e94a94fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.376+0000 7fce40054740 1 --2- 172.21.15.103:0/2522588973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a872f0 0x561e94a94fe0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fce2c004cd0 tx=0x7fce2c001e70 comp rx=0 tx=0).stop 2024-09-19T06:49:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.376+0000 7fce40054740 1 -- 172.21.15.103:0/2522588973 shutdown_connections 2024-09-19T06:49:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.376+0000 7fce40054740 1 --2- 172.21.15.103:0/2522588973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a960c0 0x561e94aadda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.376+0000 7fce40054740 1 --2- 172.21.15.103:0/2522588973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a872f0 0x561e94a94fe0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.376+0000 7fce40054740 1 --2- 172.21.15.103:0/2522588973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a869d0 0x561e94a93920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.376+0000 7fce40054740 1 -- 172.21.15.103:0/2522588973 >> 172.21.15.103:0/2522588973 conn(0x561e94a83060 msgr2=0x561e94a948d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.376+0000 7fce40054740 1 -- 172.21.15.103:0/2522588973 shutdown_connections 2024-09-19T06:49:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.376+0000 7fce40054740 1 -- 172.21.15.103:0/2522588973 wait complete. 2024-09-19T06:49:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce40054740 1 -- start start 2024-09-19T06:49:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce40054740 1 Processor -- start 2024-09-19T06:49:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce40054740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a869d0 0x561e94a9e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce40054740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a872f0 0x561e94a99900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce40054740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a960c0 0x561e94a99e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce40054740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fce381acad0 con 0x561e94a960c0 2024-09-19T06:49:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce40054740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce381ac400 con 0x561e94a872f0 2024-09-19T06:49:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce40054740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fce381a9620 con 0x561e94a869d0 2024-09-19T06:49:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce40054740 10 client.0 _ll_register_callbacks cb 0x7fce2c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a960c0 0x561e94a99e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a960c0 0x561e94a99e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37996/0 (socket says 172.21.15.103:37996) 2024-09-19T06:49:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce377fe640 1 -- 172.21.15.103:0/882984118 learned_addr learned my addr 172.21.15.103:0/882984118 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce36ffd640 1 --2- 172.21.15.103:0/882984118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a869d0 0x561e94a9e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce367fc640 1 --2- 172.21.15.103:0/882984118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a872f0 0x561e94a99900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce377fe640 1 -- 172.21.15.103:0/882984118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a869d0 msgr2=0x561e94a9e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce377fe640 1 --2- 172.21.15.103:0/882984118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a869d0 0x561e94a9e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce377fe640 1 -- 172.21.15.103:0/882984118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a872f0 msgr2=0x561e94a99900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce377fe640 1 --2- 172.21.15.103:0/882984118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a872f0 0x561e94a99900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce377fe640 1 -- 172.21.15.103:0/882984118 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e94a9a480 con 0x561e94a960c0 2024-09-19T06:49:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.377+0000 7fce367fc640 1 --2- 172.21.15.103:0/882984118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a872f0 0x561e94a99900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.378+0000 7fce377fe640 1 --2- 172.21.15.103:0/882984118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a960c0 0x561e94a99e40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fce30014700 tx=0x7fce3000aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6cc62764d7a96fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.378+0000 7fce1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.378+0000 7fce1aff5640 1 -- 172.21.15.103:0/882984118 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30024040 con 0x561e94a960c0 2024-09-19T06:49:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.378+0000 7fce1aff5640 1 -- 172.21.15.103:0/882984118 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fce30004a00 con 0x561e94a960c0 2024-09-19T06:49:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.378+0000 7fce40054740 1 -- 172.21.15.103:0/882984118 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce2c009520 con 0x561e94a960c0 2024-09-19T06:49:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.378+0000 7fce1aff5640 1 -- 172.21.15.103:0/882984118 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fce3000ce30 con 0x561e94a960c0 2024-09-19T06:49:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.378+0000 7fce40054740 10 client.15474 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.378+0000 7fce40054740 1 -- 172.21.15.103:0/882984118 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e94b10be0 con 0x561e94a960c0 2024-09-19T06:49:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.378+0000 7fce1aff5640 1 -- 172.21.15.103:0/882984118 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce2c009520 con 0x561e94a960c0 2024-09-19T06:49:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.378+0000 7fce1aff5640 1 -- 172.21.15.103:0/882984118 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fce30005040 con 0x561e94a960c0 2024-09-19T06:49:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.378+0000 7fce40054740 10 client.15474 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.378+0000 7fce40054740 10 client.15474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.378+0000 7fce40054740 20 client.15474 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.378+0000 7fce40054740 1 --2- 172.21.15.103:0/882984118 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561e94b11120 0x561e94b13540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.378+0000 7fce40054740 4 client.15474 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:49:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.378+0000 7fce40054740 1 -- 172.21.15.103:0/882984118 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e949f2d70 con 0x561e94b11120 2024-09-19T06:49:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.379+0000 7fce36ffd640 1 --2- 172.21.15.103:0/882984118 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561e94b11120 0x561e94b13540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.379+0000 7fce1aff5640 10 client.15474 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.379+0000 7fce36ffd640 1 --2- 172.21.15.103:0/882984118 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561e94b11120 0x561e94b13540 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcd4c131345f9fd1 server_cookie=9c663bc6862646d9 in_seq=0 out_seq=0 2024-09-19T06:49:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.390+0000 7fce1aff5640 1 -- 172.21.15.103:0/882984118 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e949f2d70 con 0x561e94b11120 2024-09-19T06:49:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.390+0000 7fce1aff5640 10 client.15474 handle_command_reply: tid=0 2024-09-19T06:49:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.390+0000 7fce40054740 1 client.15474 shutdown 2024-09-19T06:49:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.390+0000 7fce40054740 1 -- 172.21.15.103:0/882984118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a960c0 msgr2=0x561e94a99e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.390+0000 7fce40054740 1 --2- 172.21.15.103:0/882984118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a960c0 0x561e94a99e40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fce30014700 tx=0x7fce3000aef0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.390+0000 7fce40054740 1 -- 172.21.15.103:0/882984118 shutdown_connections 2024-09-19T06:49:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.390+0000 7fce40054740 1 --2- 172.21.15.103:0/882984118 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561e94b11120 0x561e94b13540 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.390+0000 7fce40054740 1 --2- 172.21.15.103:0/882984118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e94a960c0 0x561e94a99e40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.390+0000 7fce40054740 1 --2- 172.21.15.103:0/882984118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e94a872f0 0x561e94a99900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.390+0000 7fce1aff5640 0 client.15474 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.390+0000 7fce40054740 1 --2- 172.21.15.103:0/882984118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e94a869d0 0x561e94a9e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.390+0000 7fce40054740 1 -- 172.21.15.103:0/882984118 >> 172.21.15.103:0/882984118 conn(0x561e94a83060 msgr2=0x561e94a88d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.391+0000 7fce40054740 1 -- 172.21.15.103:0/882984118 shutdown_connections 2024-09-19T06:49:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.391+0000 7fce40054740 1 -- 172.21.15.103:0/882984118 wait complete. 2024-09-19T06:49:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.391+0000 7fce40054740 20 client.15474 trim_cache size 0 max 16384 2024-09-19T06:49:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:49:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:49:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:49:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:49:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.391+0000 7fce3d2c0640 1 -- 172.21.15.103:0/4199307521 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fce200a3580 msgr2=0x7fce200a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.391+0000 7fce3d2c0640 1 --2- 172.21.15.103:0/4199307521 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fce200a3580 0x7fce200a5a40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce30011cb0 tx=0x7fce30016000 comp rx=0 tx=0).stop 2024-09-19T06:49:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.391+0000 7fce3d2c0640 1 -- 172.21.15.103:0/4199307521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce38102ca0 msgr2=0x7fce38198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.391+0000 7fce3d2c0640 1 --2- 172.21.15.103:0/4199307521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce38102ca0 0x7fce38198360 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fce2800e9f0 tx=0x7fce2800edf0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.391+0000 7fce3d2c0640 1 -- 172.21.15.103:0/4199307521 shutdown_connections 2024-09-19T06:49:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.391+0000 7fce3d2c0640 1 --2- 172.21.15.103:0/4199307521 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fce200a3580 0x7fce200a5a40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.391+0000 7fce3d2c0640 1 --2- 172.21.15.103:0/4199307521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce3810c0d0 0x7fce3819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.391+0000 7fce3d2c0640 1 --2- 172.21.15.103:0/4199307521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce381035c0 0x7fce381988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.391+0000 7fce3d2c0640 1 --2- 172.21.15.103:0/4199307521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce38102ca0 0x7fce38198360 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.391+0000 7fce3d2c0640 1 -- 172.21.15.103:0/4199307521 >> 172.21.15.103:0/4199307521 conn(0x7fce380fc6b0 msgr2=0x7fce380fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.391+0000 7fce3d2c0640 1 -- 172.21.15.103:0/4199307521 shutdown_connections 2024-09-19T06:49:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:38.391+0000 7fce3d2c0640 1 -- 172.21.15.103:0/4199307521 wait complete. 2024-09-19T06:49:38.432 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:38.783 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:38 smithi103 ceph-mon[31158]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T06:49:38.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:38 smithi174 ceph-mon[35637]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T06:49:38.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.862+0000 7f92229d5640 1 -- 172.21.15.103:0/1085504102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f921c102ef0 msgr2=0x7f921c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.862+0000 7f92229d5640 1 --2- 172.21.15.103:0/1085504102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f921c102ef0 0x7f921c10da40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9214009ca0 tx=0x7f921402a950 comp rx=0 tx=0).stop 2024-09-19T06:49:38.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.862+0000 7f92229d5640 1 -- 172.21.15.103:0/1085504102 shutdown_connections 2024-09-19T06:49:38.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.862+0000 7f92229d5640 1 --2- 172.21.15.103:0/1085504102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f921c102ef0 0x7f921c10da40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.862+0000 7f92229d5640 1 --2- 172.21.15.103:0/1085504102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f921c102570 0x7f921c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.862+0000 7f92229d5640 1 --2- 172.21.15.103:0/1085504102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f921c100e20 0x7f921c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.862+0000 7f92229d5640 1 -- 172.21.15.103:0/1085504102 >> 172.21.15.103:0/1085504102 conn(0x7f921c0fc7d0 msgr2=0x7f921c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:38.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.862+0000 7f92229d5640 1 -- 172.21.15.103:0/1085504102 shutdown_connections 2024-09-19T06:49:38.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.862+0000 7f92229d5640 1 -- 172.21.15.103:0/1085504102 wait complete. 2024-09-19T06:49:38.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.862+0000 7f92229d5640 1 Processor -- start 2024-09-19T06:49:38.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f92229d5640 1 -- start start 2024-09-19T06:49:38.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f92229d5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f921c100e20 0x7f921c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f92229d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f921c102570 0x7f921c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f92229d5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f921c102ef0 0x7f921c19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f92229d5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f921c111020 con 0x7f921c102570 2024-09-19T06:49:38.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f92229d5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f921c110ea0 con 0x7f921c102ef0 2024-09-19T06:49:38.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f92229d5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f921c1111a0 con 0x7f921c100e20 2024-09-19T06:49:38.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f921b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f921c102570 0x7f921c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f921b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f921c102570 0x7f921c198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38010/0 (socket says 172.21.15.103:38010) 2024-09-19T06:49:38.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f921b7fe640 1 -- 172.21.15.103:0/2581507877 learned_addr learned my addr 172.21.15.103:0/2581507877 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:38.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f9220f72640 1 --2- 172.21.15.103:0/2581507877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f921c102ef0 0x7f921c19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f921bfff640 1 --2- 172.21.15.103:0/2581507877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f921c100e20 0x7f921c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f921b7fe640 1 -- 172.21.15.103:0/2581507877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f921c100e20 msgr2=0x7f921c198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f921b7fe640 1 --2- 172.21.15.103:0/2581507877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f921c100e20 0x7f921c198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f921b7fe640 1 -- 172.21.15.103:0/2581507877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f921c102ef0 msgr2=0x7f921c19cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f921b7fe640 1 --2- 172.21.15.103:0/2581507877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f921c102ef0 0x7f921c19cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f921b7fe640 1 -- 172.21.15.103:0/2581507877 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921c19d620 con 0x7f921c102570 2024-09-19T06:49:38.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.863+0000 7f921bfff640 1 --2- 172.21.15.103:0/2581507877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f921c100e20 0x7f921c198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:38.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.864+0000 7f921b7fe640 1 --2- 172.21.15.103:0/2581507877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f921c102570 0x7f921c198bb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f920400b4a0 tx=0x7f920400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:38.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.864+0000 7f92197fa640 1 -- 172.21.15.103:0/2581507877 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204014070 con 0x7f921c102570 2024-09-19T06:49:38.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.864+0000 7f92197fa640 1 -- 172.21.15.103:0/2581507877 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9204010af0 con 0x7f921c102570 2024-09-19T06:49:38.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.864+0000 7f92229d5640 1 -- 172.21.15.103:0/2581507877 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f921c19d910 con 0x7f921c102570 2024-09-19T06:49:38.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.864+0000 7f92197fa640 1 -- 172.21.15.103:0/2581507877 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204017d10 con 0x7f921c102570 2024-09-19T06:49:38.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.864+0000 7f92229d5640 1 -- 172.21.15.103:0/2581507877 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f921c1a51f0 con 0x7f921c102570 2024-09-19T06:49:38.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.866+0000 7f92197fa640 1 -- 172.21.15.103:0/2581507877 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f9204017070 con 0x7f921c102570 2024-09-19T06:49:38.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.866+0000 7f9224dae740 1 Processor -- start 2024-09-19T06:49:38.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.866+0000 7f9224dae740 1 -- start start 2024-09-19T06:49:38.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f9224dae740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee66f8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f9224dae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee66f91d0 0x560ee6707a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f9224dae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee6707fc0 0x560ee670a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f9224dae740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560ee665f090 con 0x560ee66f88b0 2024-09-19T06:49:38.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f9224dae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560ee6606a70 con 0x560ee66f91d0 2024-09-19T06:49:38.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f9224dae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560ee665b050 con 0x560ee6707fc0 2024-09-19T06:49:38.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f9220f72640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee66f8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f9220f72640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee66f8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38022/0 (socket says 172.21.15.103:38022) 2024-09-19T06:49:38.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f9220f72640 1 -- 172.21.15.103:0/2621321605 learned_addr learned my addr 172.21.15.103:0/2621321605 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:38.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f921b7fe640 1 --2- 172.21.15.103:0/2621321605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee6707fc0 0x560ee670a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f921bfff640 1 --2- 172.21.15.103:0/2621321605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee66f91d0 0x560ee6707a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f92197fa640 1 --2- 172.21.15.103:0/2581507877 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f92080a3550 0x7f92080a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f9220f72640 1 --2- 172.21.15.103:0/2581507877 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f92080a3550 0x7f92080a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f9220f72640 1 -- 172.21.15.103:0/2621321605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee6707fc0 msgr2=0x560ee670a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:38.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f9220f72640 1 --2- 172.21.15.103:0/2621321605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee6707fc0 0x560ee670a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f92197fa640 1 -- 172.21.15.103:0/2581507877 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f92040cf210 con 0x7f921c102570 2024-09-19T06:49:38.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f9220f72640 1 -- 172.21.15.103:0/2621321605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee66f91d0 msgr2=0x560ee6707a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:38.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f9220f72640 1 --2- 172.21.15.103:0/2621321605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee66f91d0 0x560ee6707a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.867+0000 7f9220f72640 1 -- 172.21.15.103:0/2621321605 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ee6613130 con 0x560ee66f88b0 2024-09-19T06:49:38.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.868+0000 7f9220f72640 1 --2- 172.21.15.103:0/2621321605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee66f8c90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9214035000 tx=0x7f921402f550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad81aa2781902515 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:38.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.868+0000 7f92127fc640 1 -- 172.21.15.103:0/2621321605 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214043050 con 0x560ee66f88b0 2024-09-19T06:49:38.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.868+0000 7f92127fc640 1 -- 172.21.15.103:0/2621321605 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9214044040 con 0x560ee66f88b0 2024-09-19T06:49:38.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.868+0000 7f92127fc640 1 -- 172.21.15.103:0/2621321605 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921404d020 con 0x560ee66f88b0 2024-09-19T06:49:38.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.868+0000 7f9224dae740 1 -- 172.21.15.103:0/2621321605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 msgr2=0x560ee66f8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.868+0000 7f9220f72640 1 --2- 172.21.15.103:0/2581507877 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f92080a3550 0x7f92080a5a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f921404e000 tx=0x7f921404c000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:38.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.868+0000 7f9224dae740 1 --2- 172.21.15.103:0/2621321605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee66f8c90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9214035000 tx=0x7f921402f550 comp rx=0 tx=0).stop 2024-09-19T06:49:38.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.869+0000 7f9224dae740 1 -- 172.21.15.103:0/2621321605 shutdown_connections 2024-09-19T06:49:38.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.869+0000 7f9224dae740 1 --2- 172.21.15.103:0/2621321605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee6707fc0 0x560ee670a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.869+0000 7f9224dae740 1 --2- 172.21.15.103:0/2621321605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee66f91d0 0x560ee6707a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.869+0000 7f9224dae740 1 --2- 172.21.15.103:0/2621321605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee66f8c90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.869+0000 7f9224dae740 1 -- 172.21.15.103:0/2621321605 >> 172.21.15.103:0/2621321605 conn(0x560ee66f4f60 msgr2=0x560ee66f5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:38.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.869+0000 7f9224dae740 1 -- 172.21.15.103:0/2621321605 shutdown_connections 2024-09-19T06:49:38.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.869+0000 7f9224dae740 1 -- 172.21.15.103:0/2621321605 wait complete. 2024-09-19T06:49:38.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.869+0000 7f9224dae740 1 -- start start 2024-09-19T06:49:38.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.869+0000 7f9224dae740 1 Processor -- start 2024-09-19T06:49:38.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.869+0000 7f9224dae740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee6707840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.869+0000 7f9224dae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee66f91d0 0x560ee6704b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.869+0000 7f9224dae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee6707fc0 0x560ee67050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.869+0000 7f9224dae740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560ee670ab90 con 0x560ee66f88b0 2024-09-19T06:49:38.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.869+0000 7f9224dae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560ee6619f20 con 0x560ee66f91d0 2024-09-19T06:49:38.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.869+0000 7f9224dae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560ee670ad10 con 0x560ee6707fc0 2024-09-19T06:49:38.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.869+0000 7f9224dae740 10 client.0 _ll_register_callbacks cb 0x560ee65e3030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:38.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f921bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee6707840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f921bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee6707840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38034/0 (socket says 172.21.15.103:38034) 2024-09-19T06:49:38.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f921bfff640 1 -- 172.21.15.103:0/3984790690 learned_addr learned my addr 172.21.15.103:0/3984790690 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:38.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f921b7fe640 1 --2- 172.21.15.103:0/3984790690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee66f91d0 0x560ee6704b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f9220f72640 1 --2- 172.21.15.103:0/3984790690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee6707fc0 0x560ee67050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f921bfff640 1 -- 172.21.15.103:0/3984790690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee6707fc0 msgr2=0x560ee67050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:38.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f921bfff640 1 --2- 172.21.15.103:0/3984790690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee6707fc0 0x560ee67050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f921bfff640 1 -- 172.21.15.103:0/3984790690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee66f91d0 msgr2=0x560ee6704b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:38.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f921bfff640 1 --2- 172.21.15.103:0/3984790690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee66f91d0 0x560ee6704b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f921bfff640 1 -- 172.21.15.103:0/3984790690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ee63d02b0 con 0x560ee66f88b0 2024-09-19T06:49:38.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f921bfff640 1 --2- 172.21.15.103:0/3984790690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee6707840 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f920c00f4f0 tx=0x7f920c00f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f87eb292aa2a5ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:38.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f91fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:38.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f91fb7fe640 1 -- 172.21.15.103:0/3984790690 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c013850 con 0x560ee66f88b0 2024-09-19T06:49:38.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f91fb7fe640 1 -- 172.21.15.103:0/3984790690 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f920c0139f0 con 0x560ee66f88b0 2024-09-19T06:49:38.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f9224dae740 1 -- 172.21.15.103:0/3984790690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9214044040 con 0x560ee66f88b0 2024-09-19T06:49:38.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f91fb7fe640 1 -- 172.21.15.103:0/3984790690 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f920c013bf0 con 0x560ee66f88b0 2024-09-19T06:49:38.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f9224dae740 10 client.15486 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:38.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.870+0000 7f9224dae740 1 -- 172.21.15.103:0/3984790690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ee6716540 con 0x560ee66f88b0 2024-09-19T06:49:38.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.871+0000 7f91fb7fe640 1 -- 172.21.15.103:0/3984790690 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9214044040 con 0x560ee66f88b0 2024-09-19T06:49:38.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.871+0000 7f91fb7fe640 1 -- 172.21.15.103:0/3984790690 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f920c0113e0 con 0x560ee66f88b0 2024-09-19T06:49:38.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.871+0000 7f9224dae740 10 client.15486 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:38.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.871+0000 7f9224dae740 10 client.15486 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-19T06:49:38.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.871+0000 7f9224dae740 20 client.15486 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:38.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.871+0000 7f9224dae740 1 --2- 172.21.15.103:0/3984790690 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x560ee6718b80 0x560ee6719ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.871+0000 7f9224dae740 4 client.15486 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:38.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.871+0000 7f9224dae740 1 -- 172.21.15.103:0/3984790690 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ee666c600 con 0x560ee6718b80 2024-09-19T06:49:38.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.871+0000 7f921b7fe640 1 --2- 172.21.15.103:0/3984790690 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x560ee6718b80 0x560ee6719ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.871+0000 7f91fb7fe640 10 client.15486 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:49:38.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.871+0000 7f921b7fe640 1 --2- 172.21.15.103:0/3984790690 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x560ee6718b80 0x560ee6719ff0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=262f8358d9f2eb8c server_cookie=4af0a070cba9457c in_seq=0 out_seq=0 2024-09-19T06:49:38.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.872+0000 7f91fb7fe640 1 -- 172.21.15.103:0/3984790690 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ee666c600 con 0x560ee6718b80 2024-09-19T06:49:38.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.872+0000 7f91fb7fe640 10 client.15486 handle_command_reply: tid=0 2024-09-19T06:49:38.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.872+0000 7f9224dae740 1 client.15486 shutdown 2024-09-19T06:49:38.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.872+0000 7f9224dae740 1 -- 172.21.15.103:0/3984790690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 msgr2=0x560ee6707840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.872+0000 7f9224dae740 1 --2- 172.21.15.103:0/3984790690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee6707840 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f920c00f4f0 tx=0x7f920c00f9b0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.872+0000 7f9224dae740 1 -- 172.21.15.103:0/3984790690 shutdown_connections 2024-09-19T06:49:38.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.872+0000 7f9224dae740 1 --2- 172.21.15.103:0/3984790690 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x560ee6718b80 0x560ee6719ff0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.872+0000 7f9224dae740 1 --2- 172.21.15.103:0/3984790690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee6707fc0 0x560ee67050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.872+0000 7f9224dae740 1 --2- 172.21.15.103:0/3984790690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee66f91d0 0x560ee6704b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.872+0000 7f9224dae740 1 --2- 172.21.15.103:0/3984790690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee6707840 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.872+0000 7f9224dae740 1 -- 172.21.15.103:0/3984790690 >> 172.21.15.103:0/3984790690 conn(0x560ee66f4f60 msgr2=0x560ee66facc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:38.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.872+0000 7f91fb7fe640 0 client.15486 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:49:38.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.873+0000 7f9224dae740 1 -- 172.21.15.103:0/3984790690 shutdown_connections 2024-09-19T06:49:38.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.873+0000 7f9224dae740 1 -- 172.21.15.103:0/3984790690 wait complete. 2024-09-19T06:49:38.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.873+0000 7f9224dae740 20 client.15486 trim_cache size 0 max 16384 2024-09-19T06:49:38.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.890+0000 7f9224dae740 1 Processor -- start 2024-09-19T06:49:38.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.890+0000 7f9224dae740 1 -- start start 2024-09-19T06:49:38.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f9224dae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee66f88b0 0x560ee6718b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f9224dae740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f91d0 0x560ee6700130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f9224dae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee6707fc0 0x560ee6701df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f9224dae740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560ee6718330 con 0x560ee66f91d0 2024-09-19T06:49:38.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f9224dae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560ee6718a00 con 0x560ee66f88b0 2024-09-19T06:49:38.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f9224dae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560ee67190d0 con 0x560ee6707fc0 2024-09-19T06:49:38.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f921b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f91d0 0x560ee6700130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f921b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f91d0 0x560ee6700130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38048/0 (socket says 172.21.15.103:38048) 2024-09-19T06:49:38.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f921b7fe640 1 -- 172.21.15.103:0/230926688 learned_addr learned my addr 172.21.15.103:0/230926688 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:38.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f921b7fe640 1 -- 172.21.15.103:0/230926688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee6707fc0 msgr2=0x560ee6701df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:38.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f921b7fe640 1 --2- 172.21.15.103:0/230926688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee6707fc0 0x560ee6701df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f921b7fe640 1 -- 172.21.15.103:0/230926688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee66f88b0 msgr2=0x560ee6718b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:38.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f921b7fe640 1 --2- 172.21.15.103:0/230926688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee66f88b0 0x560ee6718b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f921b7fe640 1 -- 172.21.15.103:0/230926688 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ee6705950 con 0x560ee66f91d0 2024-09-19T06:49:38.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f921b7fe640 1 --2- 172.21.15.103:0/230926688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f91d0 0x560ee6700130 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9204017fd0 tx=0x7f92040bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716e45233742a1b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:38.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f9211ffb640 1 -- 172.21.15.103:0/230926688 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920401e040 con 0x560ee66f91d0 2024-09-19T06:49:38.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f9211ffb640 1 -- 172.21.15.103:0/230926688 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f920409f770 con 0x560ee66f91d0 2024-09-19T06:49:38.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.891+0000 7f9211ffb640 1 -- 172.21.15.103:0/230926688 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204010ca0 con 0x560ee66f91d0 2024-09-19T06:49:38.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.892+0000 7f9224dae740 1 -- 172.21.15.103:0/230926688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f91d0 msgr2=0x560ee6700130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.892+0000 7f9224dae740 1 --2- 172.21.15.103:0/230926688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f91d0 0x560ee6700130 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9204017fd0 tx=0x7f92040bb040 comp rx=0 tx=0).stop 2024-09-19T06:49:38.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.892+0000 7f9224dae740 1 -- 172.21.15.103:0/230926688 shutdown_connections 2024-09-19T06:49:38.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.892+0000 7f9224dae740 1 --2- 172.21.15.103:0/230926688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee6707fc0 0x560ee6701df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.892+0000 7f9224dae740 1 --2- 172.21.15.103:0/230926688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f91d0 0x560ee6700130 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.892+0000 7f9224dae740 1 --2- 172.21.15.103:0/230926688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee66f88b0 0x560ee6718b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.892+0000 7f9224dae740 1 -- 172.21.15.103:0/230926688 >> 172.21.15.103:0/230926688 conn(0x560ee66f4f60 msgr2=0x560ee66ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:38.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.892+0000 7f9224dae740 1 -- 172.21.15.103:0/230926688 shutdown_connections 2024-09-19T06:49:38.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.892+0000 7f9224dae740 1 -- 172.21.15.103:0/230926688 wait complete. 2024-09-19T06:49:38.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.892+0000 7f9224dae740 1 -- start start 2024-09-19T06:49:38.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f9224dae740 1 Processor -- start 2024-09-19T06:49:38.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f9224dae740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee6722930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f9224dae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee6707fc0 0x560ee671bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f9224dae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee671c100 0x560ee671c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f9224dae740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560ee6719e70 con 0x560ee66f88b0 2024-09-19T06:49:38.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f9224dae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560ee67197a0 con 0x560ee6707fc0 2024-09-19T06:49:38.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f9224dae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560ee671a540 con 0x560ee671c100 2024-09-19T06:49:38.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f9224dae740 10 client.0 _ll_register_callbacks cb 0x7f920c00a530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:38.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f921bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee6722930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f921bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee6722930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38062/0 (socket says 172.21.15.103:38062) 2024-09-19T06:49:38.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f921bfff640 1 -- 172.21.15.103:0/1431947961 learned_addr learned my addr 172.21.15.103:0/1431947961 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:38.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f921b7fe640 1 --2- 172.21.15.103:0/1431947961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee6707fc0 0x560ee671bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f921bfff640 1 -- 172.21.15.103:0/1431947961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee671c100 msgr2=0x560ee671c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:38.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f921bfff640 1 --2- 172.21.15.103:0/1431947961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee671c100 0x560ee671c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f921bfff640 1 -- 172.21.15.103:0/1431947961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee6707fc0 msgr2=0x560ee671bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:38.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f921bfff640 1 --2- 172.21.15.103:0/1431947961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee6707fc0 0x560ee671bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f921bfff640 1 -- 172.21.15.103:0/1431947961 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ee6720500 con 0x560ee66f88b0 2024-09-19T06:49:38.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f921b7fe640 1 --2- 172.21.15.103:0/1431947961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee6707fc0 0x560ee671bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:38.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f921bfff640 1 --2- 172.21.15.103:0/1431947961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee6722930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f920c011850 tx=0x7f920c00ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9adb3408ef3a463 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:38.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.893+0000 7f91fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:38.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.894+0000 7f91fb7fe640 1 -- 172.21.15.103:0/1431947961 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c032020 con 0x560ee66f88b0 2024-09-19T06:49:38.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.894+0000 7f91fb7fe640 1 -- 172.21.15.103:0/1431947961 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f920c013850 con 0x560ee66f88b0 2024-09-19T06:49:38.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.894+0000 7f9224dae740 1 -- 172.21.15.103:0/1431947961 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f920409f770 con 0x560ee66f88b0 2024-09-19T06:49:38.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.894+0000 7f91fb7fe640 1 -- 172.21.15.103:0/1431947961 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f920c028070 con 0x560ee66f88b0 2024-09-19T06:49:38.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.894+0000 7f9224dae740 10 client.15492 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:38.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.894+0000 7f9224dae740 1 -- 172.21.15.103:0/1431947961 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ee6783670 con 0x560ee66f88b0 2024-09-19T06:49:38.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.894+0000 7f91fb7fe640 1 -- 172.21.15.103:0/1431947961 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f920409f770 con 0x560ee66f88b0 2024-09-19T06:49:38.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.894+0000 7f91fb7fe640 1 -- 172.21.15.103:0/1431947961 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f920c006160 con 0x560ee66f88b0 2024-09-19T06:49:38.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.894+0000 7f9224dae740 10 client.15492 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:38.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.894+0000 7f9224dae740 10 client.15492 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-19T06:49:38.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.894+0000 7f9224dae740 20 client.15492 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:38.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.894+0000 7f9224dae740 1 --2- 172.21.15.103:0/1431947961 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x560ee67882e0 0x560ee6786ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:38.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.894+0000 7f9224dae740 4 client.15492 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:49:38.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.894+0000 7f9224dae740 1 -- 172.21.15.103:0/1431947961 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "status"}) -- 0x560ee6666b10 con 0x560ee67882e0 2024-09-19T06:49:38.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f921b7fe640 1 --2- 172.21.15.103:0/1431947961 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x560ee67882e0 0x560ee6786ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:38.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f921b7fe640 1 --2- 172.21.15.103:0/1431947961 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x560ee67882e0 0x560ee6786ff0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9c090f17bd5a1089 server_cookie=8c3e7097fdacaf56 in_seq=0 out_seq=0 2024-09-19T06:49:38.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f91fb7fe640 10 client.15492 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:49:38.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f91fb7fe640 1 -- 172.21.15.103:0/1431947961 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560ee6666b10 con 0x560ee67882e0 2024-09-19T06:49:38.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f91fb7fe640 10 client.15492 handle_command_reply: tid=0 2024-09-19T06:49:38.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f9224dae740 1 client.15492 shutdown 2024-09-19T06:49:38.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f9224dae740 1 -- 172.21.15.103:0/1431947961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 msgr2=0x560ee6722930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f9224dae740 1 --2- 172.21.15.103:0/1431947961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee6722930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f920c011850 tx=0x7f920c00ff30 comp rx=0 tx=0).stop 2024-09-19T06:49:38.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f9224dae740 1 -- 172.21.15.103:0/1431947961 shutdown_connections 2024-09-19T06:49:38.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f9224dae740 1 --2- 172.21.15.103:0/1431947961 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x560ee67882e0 0x560ee6786ff0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f9224dae740 1 --2- 172.21.15.103:0/1431947961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ee671c100 0x560ee671c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f9224dae740 1 --2- 172.21.15.103:0/1431947961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ee6707fc0 0x560ee671bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f9224dae740 1 --2- 172.21.15.103:0/1431947961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ee66f88b0 0x560ee6722930 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f9224dae740 1 -- 172.21.15.103:0/1431947961 >> 172.21.15.103:0/1431947961 conn(0x560ee66f4f60 msgr2=0x560ee6700240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:38.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f91fb7fe640 0 client.15492 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:49:38.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f9224dae740 1 -- 172.21.15.103:0/1431947961 shutdown_connections 2024-09-19T06:49:38.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.895+0000 7f9224dae740 1 -- 172.21.15.103:0/1431947961 wait complete. 2024-09-19T06:49:38.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.896+0000 7f9224dae740 20 client.15492 trim_cache size 0 max 16384 2024-09-19T06:49:38.903 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:49:38.904 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:49:38.904 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 4, 2024-09-19T06:49:38.904 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24431, 2024-09-19T06:49:38.904 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T06:49:38.904 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T06:49:38.904 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T06:49:38.904 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 122.997344564, 2024-09-19T06:49:38.904 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 27, 2024-09-19T06:49:38.904 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 83, 2024-09-19T06:49:38.904 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:49:38.904 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 162.68463956299999 2024-09-19T06:49:38.905 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:49:38.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.896+0000 7f92229d5640 1 -- 172.21.15.103:0/2581507877 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f92080a3550 msgr2=0x7f92080a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.896+0000 7f92229d5640 1 --2- 172.21.15.103:0/2581507877 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f92080a3550 0x7f92080a5a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f921404e000 tx=0x7f921404c000 comp rx=0 tx=0).stop 2024-09-19T06:49:38.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.896+0000 7f92229d5640 1 -- 172.21.15.103:0/2581507877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f921c102570 msgr2=0x7f921c198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:38.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.896+0000 7f92229d5640 1 --2- 172.21.15.103:0/2581507877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f921c102570 0x7f921c198bb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f920400b4a0 tx=0x7f920400b960 comp rx=0 tx=0).stop 2024-09-19T06:49:38.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.896+0000 7f92229d5640 1 -- 172.21.15.103:0/2581507877 shutdown_connections 2024-09-19T06:49:38.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.896+0000 7f92229d5640 1 --2- 172.21.15.103:0/2581507877 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f92080a3550 0x7f92080a5a10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.896+0000 7f92229d5640 1 --2- 172.21.15.103:0/2581507877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f921c102ef0 0x7f921c19cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.896+0000 7f92229d5640 1 --2- 172.21.15.103:0/2581507877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f921c102570 0x7f921c198bb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.896+0000 7f92229d5640 1 --2- 172.21.15.103:0/2581507877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f921c100e20 0x7f921c198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:38.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.896+0000 7f92229d5640 1 -- 172.21.15.103:0/2581507877 >> 172.21.15.103:0/2581507877 conn(0x7f921c0fc7d0 msgr2=0x7f921c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:38.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.896+0000 7f92229d5640 1 -- 172.21.15.103:0/2581507877 shutdown_connections 2024-09-19T06:49:38.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:38.896+0000 7f92229d5640 1 -- 172.21.15.103:0/2581507877 wait complete. 2024-09-19T06:49:38.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:38 smithi134 ceph-mon[35356]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T06:49:39.649 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T06:49:39.893 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:40.282 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:40 smithi103 ceph-mon[31158]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-19T06:49:40.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:40 smithi174 ceph-mon[35637]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-19T06:49:40.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.364+0000 7f17a0015640 1 -- 172.21.15.103:0/1716535260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1798101410 msgr2=0x7f179810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.364+0000 7f17a0015640 1 --2- 172.21.15.103:0/1716535260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1798101410 0x7f179810d940 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1794009930 tx=0x7f179402a420 comp rx=0 tx=0).stop 2024-09-19T06:49:40.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.364+0000 7f17a0015640 1 -- 172.21.15.103:0/1716535260 shutdown_connections 2024-09-19T06:49:40.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.364+0000 7f17a0015640 1 --2- 172.21.15.103:0/1716535260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1798101410 0x7f179810d940 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.364+0000 7f17a0015640 1 --2- 172.21.15.103:0/1716535260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1798100a70 0x7f1798100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.364+0000 7f17a0015640 1 --2- 172.21.15.103:0/1716535260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1798106a70 0x7f1798106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.364+0000 7f17a0015640 1 -- 172.21.15.103:0/1716535260 >> 172.21.15.103:0/1716535260 conn(0x7f17980fc790 msgr2=0x7f17980febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:40.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.364+0000 7f17a0015640 1 -- 172.21.15.103:0/1716535260 shutdown_connections 2024-09-19T06:49:40.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.364+0000 7f17a0015640 1 -- 172.21.15.103:0/1716535260 wait complete. 2024-09-19T06:49:40.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.364+0000 7f17a0015640 1 Processor -- start 2024-09-19T06:49:40.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.364+0000 7f17a0015640 1 -- start start 2024-09-19T06:49:40.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f17a0015640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1798100a70 0x7f1798199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f17a0015640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1798101410 0x7f1798199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f17a0015640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1798106a70 0x7f17981a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f17a0015640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1798110e90 con 0x7f1798106a70 2024-09-19T06:49:40.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f17a0015640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1798110d10 con 0x7f1798101410 2024-09-19T06:49:40.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f17a0015640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1798111010 con 0x7f1798100a70 2024-09-19T06:49:40.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f179e5b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1798106a70 0x7f17981a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f179e5b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1798106a70 0x7f17981a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38084/0 (socket says 172.21.15.103:38084) 2024-09-19T06:49:40.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f179e5b2640 1 -- 172.21.15.103:0/4133088191 learned_addr learned my addr 172.21.15.103:0/4133088191 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:40.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f179ddb1640 1 --2- 172.21.15.103:0/4133088191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1798100a70 0x7f1798199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f179d5b0640 1 --2- 172.21.15.103:0/4133088191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1798101410 0x7f1798199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f179e5b2640 1 -- 172.21.15.103:0/4133088191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1798100a70 msgr2=0x7f1798199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f179e5b2640 1 --2- 172.21.15.103:0/4133088191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1798100a70 0x7f1798199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f179e5b2640 1 -- 172.21.15.103:0/4133088191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1798101410 msgr2=0x7f1798199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f179e5b2640 1 --2- 172.21.15.103:0/4133088191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1798101410 0x7f1798199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f179e5b2640 1 -- 172.21.15.103:0/4133088191 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17981a48d0 con 0x7f1798106a70 2024-09-19T06:49:40.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f179e5b2640 1 --2- 172.21.15.103:0/4133088191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1798106a70 0x7f17981a4390 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1794005ab0 tx=0x7f17940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:40.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f178effd640 1 -- 172.21.15.103:0/4133088191 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179403f030 con 0x7f1798106a70 2024-09-19T06:49:40.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f178effd640 1 -- 172.21.15.103:0/4133088191 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1794003ce0 con 0x7f1798106a70 2024-09-19T06:49:40.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f17a0015640 1 -- 172.21.15.103:0/4133088191 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17981a4aa0 con 0x7f1798106a70 2024-09-19T06:49:40.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.365+0000 7f178effd640 1 -- 172.21.15.103:0/4133088191 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794035070 con 0x7f1798106a70 2024-09-19T06:49:40.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.366+0000 7f17a0015640 1 -- 172.21.15.103:0/4133088191 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17981a4f50 con 0x7f1798106a70 2024-09-19T06:49:40.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.368+0000 7f178effd640 1 -- 172.21.15.103:0/4133088191 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f1794042030 con 0x7f1798106a70 2024-09-19T06:49:40.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.368+0000 7f178effd640 1 --2- 172.21.15.103:0/4133088191 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17840a3550 0x7f17840a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.368+0000 7f17a23ee740 1 Processor -- start 2024-09-19T06:49:40.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.368+0000 7f179ddb1640 1 --2- 172.21.15.103:0/4133088191 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17840a3550 0x7f17840a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.368+0000 7f178effd640 1 -- 172.21.15.103:0/4133088191 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f17940f03c0 con 0x7f1798106a70 2024-09-19T06:49:40.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f17a23ee740 1 -- start start 2024-09-19T06:49:40.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f17a23ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66581f9d0 0x55c66581fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f17a23ee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6658202f0 0x55c66582a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f17a23ee740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66582abb0 0x55c66582cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f17a23ee740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c665786090 con 0x55c66582abb0 2024-09-19T06:49:40.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f17a23ee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c66572da70 con 0x55c6658202f0 2024-09-19T06:49:40.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f17a23ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c665782050 con 0x55c66581f9d0 2024-09-19T06:49:40.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f179ddb1640 1 --2- 172.21.15.103:0/4133088191 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17840a3550 0x7f17840a5a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1790009f20 tx=0x7f17900094f0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:40.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f179e5b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66582abb0 0x55c66582cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f179e5b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66582abb0 0x55c66582cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38090/0 (socket says 172.21.15.103:38090) 2024-09-19T06:49:40.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f179e5b2640 1 -- 172.21.15.103:0/1981476534 learned_addr learned my addr 172.21.15.103:0/1981476534 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:40.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f179d5b0640 1 --2- 172.21.15.103:0/1981476534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6658202f0 0x55c66582a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f179e5b2640 1 -- 172.21.15.103:0/1981476534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66581f9d0 msgr2=0x55c66581fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:40.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f179ddb1640 1 --2- 172.21.15.103:0/1981476534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66581f9d0 0x55c66581fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f179e5b2640 1 --2- 172.21.15.103:0/1981476534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66581f9d0 0x55c66581fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f179e5b2640 1 -- 172.21.15.103:0/1981476534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6658202f0 msgr2=0x55c66582a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:40.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f179e5b2640 1 --2- 172.21.15.103:0/1981476534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6658202f0 0x55c66582a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f179e5b2640 1 -- 172.21.15.103:0/1981476534 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c66573a130 con 0x55c66582abb0 2024-09-19T06:49:40.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f179ddb1640 1 --2- 172.21.15.103:0/1981476534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66581f9d0 0x55c66581fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:40.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.369+0000 7f179e5b2640 1 --2- 172.21.15.103:0/1981476534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66582abb0 0x55c66582cfa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f17940b8990 tx=0x7f17940bc3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba017c3486ff90a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:40.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.370+0000 7f1783fff640 1 -- 172.21.15.103:0/1981476534 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17940dc050 con 0x55c66582abb0 2024-09-19T06:49:40.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.370+0000 7f1783fff640 1 -- 172.21.15.103:0/1981476534 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179403b460 con 0x55c66582abb0 2024-09-19T06:49:40.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.370+0000 7f1783fff640 1 -- 172.21.15.103:0/1981476534 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794049040 con 0x55c66582abb0 2024-09-19T06:49:40.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.370+0000 7f17a23ee740 1 -- 172.21.15.103:0/1981476534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66582abb0 msgr2=0x55c66582cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.370+0000 7f17a23ee740 1 --2- 172.21.15.103:0/1981476534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66582abb0 0x55c66582cfa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f17940b8990 tx=0x7f17940bc3c0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.370+0000 7f17a23ee740 1 -- 172.21.15.103:0/1981476534 shutdown_connections 2024-09-19T06:49:40.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.370+0000 7f17a23ee740 1 --2- 172.21.15.103:0/1981476534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66582abb0 0x55c66582cfa0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.370+0000 7f17a23ee740 1 --2- 172.21.15.103:0/1981476534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6658202f0 0x55c66582a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.370+0000 7f17a23ee740 1 --2- 172.21.15.103:0/1981476534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66581f9d0 0x55c66581fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.370+0000 7f17a23ee740 1 -- 172.21.15.103:0/1981476534 >> 172.21.15.103:0/1981476534 conn(0x55c66581bf60 msgr2=0x55c66581c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:40.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.371+0000 7f17a23ee740 1 -- 172.21.15.103:0/1981476534 shutdown_connections 2024-09-19T06:49:40.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.371+0000 7f17a23ee740 1 -- 172.21.15.103:0/1981476534 wait complete. 2024-09-19T06:49:40.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.371+0000 7f17a23ee740 1 -- start start 2024-09-19T06:49:40.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.371+0000 7f17a23ee740 1 Processor -- start 2024-09-19T06:49:40.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.371+0000 7f17a23ee740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66581f9d0 0x55c66582a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.371+0000 7f17a23ee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6658202f0 0x55c6658277a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.371+0000 7f17a23ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66582abb0 0x55c665827ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.371+0000 7f17a23ee740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c66582d540 con 0x55c66581f9d0 2024-09-19T06:49:40.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.371+0000 7f17a23ee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c665740f20 con 0x55c6658202f0 2024-09-19T06:49:40.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.371+0000 7f17a23ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c66582d6c0 con 0x55c66582abb0 2024-09-19T06:49:40.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.371+0000 7f17a23ee740 10 client.0 _ll_register_callbacks cb 0x55c66570a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:40.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.371+0000 7f179ddb1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66581f9d0 0x55c66582a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.371+0000 7f179ddb1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66581f9d0 0x55c66582a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38098/0 (socket says 172.21.15.103:38098) 2024-09-19T06:49:40.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.371+0000 7f179ddb1640 1 -- 172.21.15.103:0/2827147630 learned_addr learned my addr 172.21.15.103:0/2827147630 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:40.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.372+0000 7f179d5b0640 1 --2- 172.21.15.103:0/2827147630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6658202f0 0x55c6658277a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.372+0000 7f179e5b2640 1 --2- 172.21.15.103:0/2827147630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66582abb0 0x55c665827ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.372+0000 7f179ddb1640 1 -- 172.21.15.103:0/2827147630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66582abb0 msgr2=0x55c665827ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:40.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.372+0000 7f179ddb1640 1 --2- 172.21.15.103:0/2827147630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66582abb0 0x55c665827ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.372+0000 7f179ddb1640 1 -- 172.21.15.103:0/2827147630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6658202f0 msgr2=0x55c6658277a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:40.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.372+0000 7f179ddb1640 1 --2- 172.21.15.103:0/2827147630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6658202f0 0x55c6658277a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.372+0000 7f179ddb1640 1 -- 172.21.15.103:0/2827147630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6654f72b0 con 0x55c66581f9d0 2024-09-19T06:49:40.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.372+0000 7f179e5b2640 1 --2- 172.21.15.103:0/2827147630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66582abb0 0x55c665827ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:40.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.372+0000 7f179ddb1640 1 --2- 172.21.15.103:0/2827147630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66581f9d0 0x55c66582a450 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1790017ce0 tx=0x7f1790004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fa6c9955f75f0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:40.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.372+0000 7f17817fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:40.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.372+0000 7f17817fa640 1 -- 172.21.15.103:0/2827147630 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790027060 con 0x55c66581f9d0 2024-09-19T06:49:40.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.372+0000 7f17817fa640 1 -- 172.21.15.103:0/2827147630 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1790016b30 con 0x55c66581f9d0 2024-09-19T06:49:40.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.372+0000 7f17a23ee740 1 -- 172.21.15.103:0/2827147630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c665793600 con 0x55c66581f9d0 2024-09-19T06:49:40.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.372+0000 7f17817fa640 1 -- 172.21.15.103:0/2827147630 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1790016d30 con 0x55c66581f9d0 2024-09-19T06:49:40.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.372+0000 7f17a23ee740 10 client.15504 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:40.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.372+0000 7f17a23ee740 1 -- 172.21.15.103:0/2827147630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c665838da0 con 0x55c66581f9d0 2024-09-19T06:49:40.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.373+0000 7f17817fa640 1 -- 172.21.15.103:0/2827147630 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c665793600 con 0x55c66581f9d0 2024-09-19T06:49:40.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.373+0000 7f17817fa640 1 -- 172.21.15.103:0/2827147630 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1790021070 con 0x55c66581f9d0 2024-09-19T06:49:40.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.373+0000 7f17a23ee740 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:40.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.373+0000 7f17a23ee740 10 client.15504 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T06:49:40.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.373+0000 7f17a23ee740 20 client.15504 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:40.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.373+0000 7f17a23ee740 1 --2- 172.21.15.103:0/2827147630 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c66583b440 0x55c66583c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.373+0000 7f17a23ee740 4 client.15504 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:40.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.373+0000 7f17a23ee740 1 -- 172.21.15.103:0/2827147630 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c66578db10 con 0x55c66583b440 2024-09-19T06:49:40.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.373+0000 7f179d5b0640 1 --2- 172.21.15.103:0/2827147630 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c66583b440 0x55c66583c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.374+0000 7f179d5b0640 1 --2- 172.21.15.103:0/2827147630 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c66583b440 0x55c66583c8b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e0bf56fb60d54e39 server_cookie=938572966e3885de in_seq=0 out_seq=0 2024-09-19T06:49:40.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.374+0000 7f17817fa640 10 client.15504 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:49:40.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.374+0000 7f17817fa640 1 -- 172.21.15.103:0/2827147630 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c66578db10 con 0x55c66583b440 2024-09-19T06:49:40.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.374+0000 7f17817fa640 10 client.15504 handle_command_reply: tid=0 2024-09-19T06:49:40.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.374+0000 7f17a23ee740 1 client.15504 shutdown 2024-09-19T06:49:40.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.374+0000 7f17a23ee740 1 -- 172.21.15.103:0/2827147630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66581f9d0 msgr2=0x55c66582a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.374+0000 7f17a23ee740 1 --2- 172.21.15.103:0/2827147630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66581f9d0 0x55c66582a450 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1790017ce0 tx=0x7f1790004a40 comp rx=0 tx=0).stop 2024-09-19T06:49:40.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.374+0000 7f17a23ee740 1 -- 172.21.15.103:0/2827147630 shutdown_connections 2024-09-19T06:49:40.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.374+0000 7f17a23ee740 1 --2- 172.21.15.103:0/2827147630 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c66583b440 0x55c66583c8b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.374+0000 7f17a23ee740 1 --2- 172.21.15.103:0/2827147630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66582abb0 0x55c665827ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.374+0000 7f17a23ee740 1 --2- 172.21.15.103:0/2827147630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6658202f0 0x55c6658277a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.374+0000 7f17817fa640 0 client.15504 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:49:40.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.374+0000 7f17a23ee740 1 --2- 172.21.15.103:0/2827147630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66581f9d0 0x55c66582a450 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.374+0000 7f17a23ee740 1 -- 172.21.15.103:0/2827147630 >> 172.21.15.103:0/2827147630 conn(0x55c66581bf60 msgr2=0x55c665821c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:40.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.375+0000 7f17a23ee740 1 -- 172.21.15.103:0/2827147630 shutdown_connections 2024-09-19T06:49:40.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.375+0000 7f17a23ee740 1 -- 172.21.15.103:0/2827147630 wait complete. 2024-09-19T06:49:40.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.375+0000 7f17a23ee740 20 client.15504 trim_cache size 0 max 16384 2024-09-19T06:49:40.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.392+0000 7f17a23ee740 1 Processor -- start 2024-09-19T06:49:40.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.392+0000 7f17a23ee740 1 -- start start 2024-09-19T06:49:40.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.392+0000 7f17a23ee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c66581f9d0 0x55c665836dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.392+0000 7f17a23ee740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6658202f0 0x55c665836590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.392+0000 7f17a23ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66582abb0 0x55c665822580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.392+0000 7f17a23ee740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c66583abf0 con 0x55c6658202f0 2024-09-19T06:49:40.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.392+0000 7f17a23ee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c66583b2c0 con 0x55c66581f9d0 2024-09-19T06:49:40.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.392+0000 7f17a23ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c66583b990 con 0x55c66582abb0 2024-09-19T06:49:40.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.392+0000 7f179d5b0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6658202f0 0x55c665836590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.392+0000 7f179d5b0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6658202f0 0x55c665836590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38108/0 (socket says 172.21.15.103:38108) 2024-09-19T06:49:40.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.392+0000 7f179d5b0640 1 -- 172.21.15.103:0/3323050651 learned_addr learned my addr 172.21.15.103:0/3323050651 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:40.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.393+0000 7f179ddb1640 1 --2- 172.21.15.103:0/3323050651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c66581f9d0 0x55c665836dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.393+0000 7f179e5b2640 1 --2- 172.21.15.103:0/3323050651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66582abb0 0x55c665822580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.393+0000 7f179d5b0640 1 -- 172.21.15.103:0/3323050651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66582abb0 msgr2=0x55c665822580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:40.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.393+0000 7f179d5b0640 1 --2- 172.21.15.103:0/3323050651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66582abb0 0x55c665822580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.393+0000 7f179d5b0640 1 -- 172.21.15.103:0/3323050651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c66581f9d0 msgr2=0x55c665836dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:40.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.393+0000 7f179d5b0640 1 --2- 172.21.15.103:0/3323050651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c66581f9d0 0x55c665836dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.393+0000 7f179d5b0640 1 -- 172.21.15.103:0/3323050651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c66581d0f0 con 0x55c6658202f0 2024-09-19T06:49:40.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.393+0000 7f179ddb1640 1 --2- 172.21.15.103:0/3323050651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c66581f9d0 0x55c665836dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:40.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.393+0000 7f179d5b0640 1 --2- 172.21.15.103:0/3323050651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6658202f0 0x55c665836590 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1788000c00 tx=0x7f1788004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cecb4982cd64ad7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:40.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.393+0000 7f17837fe640 1 -- 172.21.15.103:0/3323050651 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178801d420 con 0x55c6658202f0 2024-09-19T06:49:40.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.393+0000 7f17837fe640 1 -- 172.21.15.103:0/3323050651 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178801daa0 con 0x55c6658202f0 2024-09-19T06:49:40.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.393+0000 7f17837fe640 1 -- 172.21.15.103:0/3323050651 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788011c50 con 0x55c6658202f0 2024-09-19T06:49:40.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.393+0000 7f17a23ee740 1 -- 172.21.15.103:0/3323050651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6658202f0 msgr2=0x55c665836590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.393+0000 7f17a23ee740 1 --2- 172.21.15.103:0/3323050651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6658202f0 0x55c665836590 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1788000c00 tx=0x7f1788004580 comp rx=0 tx=0).stop 2024-09-19T06:49:40.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f17a23ee740 1 -- 172.21.15.103:0/3323050651 shutdown_connections 2024-09-19T06:49:40.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f17a23ee740 1 --2- 172.21.15.103:0/3323050651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66582abb0 0x55c665822580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f17a23ee740 1 --2- 172.21.15.103:0/3323050651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6658202f0 0x55c665836590 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f17a23ee740 1 --2- 172.21.15.103:0/3323050651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c66581f9d0 0x55c665836dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f17a23ee740 1 -- 172.21.15.103:0/3323050651 >> 172.21.15.103:0/3323050651 conn(0x55c66581bf60 msgr2=0x55c665826070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:40.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f17a23ee740 1 -- 172.21.15.103:0/3323050651 shutdown_connections 2024-09-19T06:49:40.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f17a23ee740 1 -- 172.21.15.103:0/3323050651 wait complete. 2024-09-19T06:49:40.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f17a23ee740 1 -- start start 2024-09-19T06:49:40.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f17a23ee740 1 Processor -- start 2024-09-19T06:49:40.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f17a23ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66581f9d0 0x55c665823dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f17a23ee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6658202f0 0x55c665824310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f17a23ee740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66582abb0 0x55c665843b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f17a23ee740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c66583c730 con 0x55c66582abb0 2024-09-19T06:49:40.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f17a23ee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c66583c060 con 0x55c6658202f0 2024-09-19T06:49:40.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f17a23ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c66583ce00 con 0x55c66581f9d0 2024-09-19T06:49:40.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f17a23ee740 10 client.0 _ll_register_callbacks cb 0x7f17900096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:40.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f179e5b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66582abb0 0x55c665843b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f179e5b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66582abb0 0x55c665843b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38114/0 (socket says 172.21.15.103:38114) 2024-09-19T06:49:40.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.394+0000 7f179e5b2640 1 -- 172.21.15.103:0/2760886555 learned_addr learned my addr 172.21.15.103:0/2760886555 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:40.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.395+0000 7f179d5b0640 1 --2- 172.21.15.103:0/2760886555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6658202f0 0x55c665824310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.395+0000 7f179e5b2640 1 -- 172.21.15.103:0/2760886555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66581f9d0 msgr2=0x55c665823dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:40.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.395+0000 7f179ddb1640 1 --2- 172.21.15.103:0/2760886555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66581f9d0 0x55c665823dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.395+0000 7f179e5b2640 1 --2- 172.21.15.103:0/2760886555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66581f9d0 0x55c665823dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.395+0000 7f179e5b2640 1 -- 172.21.15.103:0/2760886555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6658202f0 msgr2=0x55c665824310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:40.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.395+0000 7f179e5b2640 1 --2- 172.21.15.103:0/2760886555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6658202f0 0x55c665824310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.395+0000 7f179e5b2640 1 -- 172.21.15.103:0/2760886555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c665844040 con 0x55c66582abb0 2024-09-19T06:49:40.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.395+0000 7f179e5b2640 1 --2- 172.21.15.103:0/2760886555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66582abb0 0x55c665843b00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1794006010 tx=0x7f17940b8180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fe7248c60bcf12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:40.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.395+0000 7f17817fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:40.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.395+0000 7f17817fa640 1 -- 172.21.15.103:0/2760886555 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794049040 con 0x55c66582abb0 2024-09-19T06:49:40.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.395+0000 7f17817fa640 1 -- 172.21.15.103:0/2760886555 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f17940b8ba0 con 0x55c66582abb0 2024-09-19T06:49:40.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.395+0000 7f17a23ee740 1 -- 172.21.15.103:0/2760886555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f178801daa0 con 0x55c66582abb0 2024-09-19T06:49:40.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.395+0000 7f17817fa640 1 -- 172.21.15.103:0/2760886555 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f17940dc040 con 0x55c66582abb0 2024-09-19T06:49:40.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.395+0000 7f17a23ee740 10 client.15510 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:40.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.395+0000 7f17a23ee740 1 -- 172.21.15.103:0/2760886555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c665845270 con 0x55c66582abb0 2024-09-19T06:49:40.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.395+0000 7f17817fa640 1 -- 172.21.15.103:0/2760886555 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178801daa0 con 0x55c66582abb0 2024-09-19T06:49:40.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.396+0000 7f17817fa640 1 -- 172.21.15.103:0/2760886555 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1794031bd0 con 0x55c66582abb0 2024-09-19T06:49:40.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.396+0000 7f17a23ee740 10 client.15510 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:40.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.396+0000 7f17a23ee740 10 client.15510 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T06:49:40.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.396+0000 7f17a23ee740 20 client.15510 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:40.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.396+0000 7f17a23ee740 1 --2- 172.21.15.103:0/2760886555 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c6658a9bb0 0x55c6658a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.396+0000 7f17a23ee740 4 client.15510 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:49:40.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.396+0000 7f17a23ee740 1 -- 172.21.15.103:0/2760886555 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "status"}) -- 0x55c66570a0e0 con 0x55c6658a9bb0 2024-09-19T06:49:40.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.396+0000 7f179ddb1640 1 --2- 172.21.15.103:0/2760886555 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c6658a9bb0 0x55c6658a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.396+0000 7f179ddb1640 1 --2- 172.21.15.103:0/2760886555 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c6658a9bb0 0x55c6658a88c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=98b3c1a2a00eca4c server_cookie=90ab627904f0c1ea in_seq=0 out_seq=0 2024-09-19T06:49:40.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.396+0000 7f17817fa640 10 client.15510 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:49:40.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.396+0000 7f17817fa640 1 -- 172.21.15.103:0/2760886555 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c66570a0e0 con 0x55c6658a9bb0 2024-09-19T06:49:40.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.396+0000 7f17817fa640 10 client.15510 handle_command_reply: tid=0 2024-09-19T06:49:40.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.396+0000 7f17a23ee740 1 client.15510 shutdown 2024-09-19T06:49:40.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a23ee740 1 -- 172.21.15.103:0/2760886555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66582abb0 msgr2=0x55c665843b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a23ee740 1 --2- 172.21.15.103:0/2760886555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66582abb0 0x55c665843b00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1794006010 tx=0x7f17940b8180 comp rx=0 tx=0).stop 2024-09-19T06:49:40.425 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:49:40.425 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:49:40.425 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 3, 2024-09-19T06:49:40.425 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24436, 2024-09-19T06:49:40.425 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T06:49:40.426 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T06:49:40.426 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T06:49:40.426 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 125.508737791, 2024-09-19T06:49:40.426 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 27, 2024-09-19T06:49:40.426 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 83, 2024-09-19T06:49:40.426 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:49:40.426 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 159.30475143300001 2024-09-19T06:49:40.426 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:49:40.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a23ee740 1 -- 172.21.15.103:0/2760886555 shutdown_connections 2024-09-19T06:49:40.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a23ee740 1 --2- 172.21.15.103:0/2760886555 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c6658a9bb0 0x55c6658a88c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a23ee740 1 --2- 172.21.15.103:0/2760886555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c66582abb0 0x55c665843b00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a23ee740 1 --2- 172.21.15.103:0/2760886555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6658202f0 0x55c665824310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17817fa640 0 client.15510 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:49:40.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a23ee740 1 --2- 172.21.15.103:0/2760886555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c66581f9d0 0x55c665823dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a23ee740 1 -- 172.21.15.103:0/2760886555 >> 172.21.15.103:0/2760886555 conn(0x55c66581bf60 msgr2=0x55c665821420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:40.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a23ee740 1 -- 172.21.15.103:0/2760886555 shutdown_connections 2024-09-19T06:49:40.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a23ee740 1 -- 172.21.15.103:0/2760886555 wait complete. 2024-09-19T06:49:40.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a23ee740 20 client.15510 trim_cache size 0 max 16384 2024-09-19T06:49:40.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a0015640 1 -- 172.21.15.103:0/4133088191 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17840a3550 msgr2=0x7f17840a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a0015640 1 --2- 172.21.15.103:0/4133088191 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17840a3550 0x7f17840a5a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1790009f20 tx=0x7f17900094f0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a0015640 1 -- 172.21.15.103:0/4133088191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1798106a70 msgr2=0x7f17981a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a0015640 1 --2- 172.21.15.103:0/4133088191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1798106a70 0x7f17981a4390 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1794005ab0 tx=0x7f17940042f0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a0015640 1 -- 172.21.15.103:0/4133088191 shutdown_connections 2024-09-19T06:49:40.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a0015640 1 --2- 172.21.15.103:0/4133088191 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17840a3550 0x7f17840a5a10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a0015640 1 --2- 172.21.15.103:0/4133088191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1798106a70 0x7f17981a4390 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a0015640 1 --2- 172.21.15.103:0/4133088191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1798101410 0x7f1798199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a0015640 1 --2- 172.21.15.103:0/4133088191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1798100a70 0x7f1798199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.397+0000 7f17a0015640 1 -- 172.21.15.103:0/4133088191 >> 172.21.15.103:0/4133088191 conn(0x7f17980fc790 msgr2=0x7f17980fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:40.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.398+0000 7f17a0015640 1 -- 172.21.15.103:0/4133088191 shutdown_connections 2024-09-19T06:49:40.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:40.398+0000 7f17a0015640 1 -- 172.21.15.103:0/4133088191 wait complete. 2024-09-19T06:49:40.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:40 smithi134 ceph-mon[35356]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-19T06:49:41.092 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T06:49:41.338 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:41.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.729+0000 7f70c50e0640 1 -- 172.21.15.103:0/3624370480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0106a80 msgr2=0x7f70c0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:41.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.729+0000 7f70c50e0640 1 --2- 172.21.15.103:0/3624370480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0106a80 0x7f70c0106e60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009930 tx=0x7f70b802a420 comp rx=0 tx=0).stop 2024-09-19T06:49:41.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.729+0000 7f70c50e0640 1 -- 172.21.15.103:0/3624370480 shutdown_connections 2024-09-19T06:49:41.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.729+0000 7f70c50e0640 1 --2- 172.21.15.103:0/3624370480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c0101420 0x7f70c010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.729+0000 7f70c50e0640 1 --2- 172.21.15.103:0/3624370480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0100a80 0x7f70c0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.729+0000 7f70c50e0640 1 --2- 172.21.15.103:0/3624370480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0106a80 0x7f70c0106e60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.729+0000 7f70c50e0640 1 -- 172.21.15.103:0/3624370480 >> 172.21.15.103:0/3624370480 conn(0x7f70c00fc7a0 msgr2=0x7f70c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:41.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.729+0000 7f70c50e0640 1 -- 172.21.15.103:0/3624370480 shutdown_connections 2024-09-19T06:49:41.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.729+0000 7f70c50e0640 1 -- 172.21.15.103:0/3624370480 wait complete. 2024-09-19T06:49:41.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.730+0000 7f70c50e0640 1 Processor -- start 2024-09-19T06:49:41.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.730+0000 7f70c50e0640 1 -- start start 2024-09-19T06:49:41.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.730+0000 7f70c50e0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c0100a80 0x7f70c0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.730+0000 7f70c50e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0101420 0x7f70c01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.730+0000 7f70c50e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0106a80 0x7f70c019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.730+0000 7f70c50e0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70c0110ea0 con 0x7f70c0101420 2024-09-19T06:49:41.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.730+0000 7f70c50e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70c0110d20 con 0x7f70c0106a80 2024-09-19T06:49:41.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.730+0000 7f70c50e0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70c0111020 con 0x7f70c0100a80 2024-09-19T06:49:41.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.730+0000 7f70be59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0101420 0x7f70c01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.730+0000 7f70be59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0101420 0x7f70c01989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38122/0 (socket says 172.21.15.103:38122) 2024-09-19T06:49:41.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.730+0000 7f70be59c640 1 -- 172.21.15.103:0/2635613920 learned_addr learned my addr 172.21.15.103:0/2635613920 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:41.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.731+0000 7f70bf59e640 1 --2- 172.21.15.103:0/2635613920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0106a80 0x7f70c019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.731+0000 7f70bed9d640 1 --2- 172.21.15.103:0/2635613920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c0100a80 0x7f70c0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.731+0000 7f70be59c640 1 -- 172.21.15.103:0/2635613920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c0100a80 msgr2=0x7f70c0198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:41.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.731+0000 7f70be59c640 1 --2- 172.21.15.103:0/2635613920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c0100a80 0x7f70c0198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.731+0000 7f70be59c640 1 -- 172.21.15.103:0/2635613920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0106a80 msgr2=0x7f70c019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:41.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.731+0000 7f70be59c640 1 --2- 172.21.15.103:0/2635613920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0106a80 0x7f70c019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.731+0000 7f70be59c640 1 -- 172.21.15.103:0/2635613920 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c019d430 con 0x7f70c0101420 2024-09-19T06:49:41.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.731+0000 7f70be59c640 1 --2- 172.21.15.103:0/2635613920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0101420 0x7f70c01989c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f70a800d950 tx=0x7f70a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:41.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.731+0000 7f70b7fff640 1 -- 172.21.15.103:0/2635613920 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8017070 con 0x7f70c0101420 2024-09-19T06:49:41.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.731+0000 7f70c50e0640 1 -- 172.21.15.103:0/2635613920 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c019d720 con 0x7f70c0101420 2024-09-19T06:49:41.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.731+0000 7f70b7fff640 1 -- 172.21.15.103:0/2635613920 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a8009c30 con 0x7f70c0101420 2024-09-19T06:49:41.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.731+0000 7f70b7fff640 1 -- 172.21.15.103:0/2635613920 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a800f6b0 con 0x7f70c0101420 2024-09-19T06:49:41.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.731+0000 7f70c50e0640 1 -- 172.21.15.103:0/2635613920 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c01a5000 con 0x7f70c0101420 2024-09-19T06:49:41.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.733+0000 7f70b7fff640 1 -- 172.21.15.103:0/2635613920 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f70a80109d0 con 0x7f70c0101420 2024-09-19T06:49:41.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.734+0000 7f70c74b9740 1 Processor -- start 2024-09-19T06:49:41.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.734+0000 7f70c74b9740 1 -- start start 2024-09-19T06:49:41.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.734+0000 7f70c74b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c556db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.734+0000 7f70c74b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c5572f0 0x563e4c561670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.734+0000 7f70c74b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c561bb0 0x563e4c563fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.734+0000 7f70c74b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563e4c4bd090 con 0x563e4c5569d0 2024-09-19T06:49:41.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.734+0000 7f70c74b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563e4c464a70 con 0x563e4c561bb0 2024-09-19T06:49:41.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.734+0000 7f70c74b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563e4c4b9050 con 0x563e4c5572f0 2024-09-19T06:49:41.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.734+0000 7f70bf59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c556db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.734+0000 7f70bf59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c556db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38132/0 (socket says 172.21.15.103:38132) 2024-09-19T06:49:41.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.734+0000 7f70bf59e640 1 -- 172.21.15.103:0/3734542020 learned_addr learned my addr 172.21.15.103:0/3734542020 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:41.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.734+0000 7f70bed9d640 1 --2- 172.21.15.103:0/3734542020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c5572f0 0x563e4c561670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.734+0000 7f70b7fff640 1 --2- 172.21.15.103:0/2635613920 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70ac0a34d0 0x7f70ac0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.735+0000 7f70be59c640 1 --2- 172.21.15.103:0/3734542020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c561bb0 0x563e4c563fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.734+0000 7f70bf59e640 1 -- 172.21.15.103:0/3734542020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c5572f0 msgr2=0x563e4c561670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:41.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.735+0000 7f70bf59e640 1 --2- 172.21.15.103:0/3734542020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c5572f0 0x563e4c561670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.735+0000 7f70bf59e640 1 -- 172.21.15.103:0/3734542020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c561bb0 msgr2=0x563e4c563fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:41.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.735+0000 7f70bf59e640 1 --2- 172.21.15.103:0/3734542020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c561bb0 0x563e4c563fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.735+0000 7f70bf59e640 1 -- 172.21.15.103:0/3734542020 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e4c471130 con 0x563e4c5569d0 2024-09-19T06:49:41.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.735+0000 7f70bed9d640 1 --2- 172.21.15.103:0/3734542020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c5572f0 0x563e4c561670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:41.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.735+0000 7f70be59c640 1 --2- 172.21.15.103:0/3734542020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c561bb0 0x563e4c563fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:41.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.735+0000 7f70b7fff640 1 -- 172.21.15.103:0/2635613920 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f70a80cfc50 con 0x7f70c0101420 2024-09-19T06:49:41.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.735+0000 7f70bf59e640 1 --2- 172.21.15.103:0/3734542020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c556db0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f70b000a430 tx=0x7f70b0009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af790c072a27d034 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:41.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.735+0000 7f70bf59e640 1 --2- 172.21.15.103:0/2635613920 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70ac0a34d0 0x7f70ac0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.735+0000 7f70b57fa640 1 -- 172.21.15.103:0/3734542020 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0010c10 con 0x563e4c5569d0 2024-09-19T06:49:41.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.735+0000 7f70b57fa640 1 -- 172.21.15.103:0/3734542020 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b0014dd0 con 0x563e4c5569d0 2024-09-19T06:49:41.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.735+0000 7f70b57fa640 1 -- 172.21.15.103:0/3734542020 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b001f5a0 con 0x563e4c5569d0 2024-09-19T06:49:41.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.736+0000 7f70c74b9740 1 -- 172.21.15.103:0/3734542020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 msgr2=0x563e4c556db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:41.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.736+0000 7f70bf59e640 1 --2- 172.21.15.103:0/2635613920 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70ac0a34d0 0x7f70ac0a5990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f70b001e000 tx=0x7f70b001f000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:41.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.736+0000 7f70c74b9740 1 --2- 172.21.15.103:0/3734542020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c556db0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f70b000a430 tx=0x7f70b0009650 comp rx=0 tx=0).stop 2024-09-19T06:49:41.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.736+0000 7f70c74b9740 1 -- 172.21.15.103:0/3734542020 shutdown_connections 2024-09-19T06:49:41.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.736+0000 7f70c74b9740 1 --2- 172.21.15.103:0/3734542020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c561bb0 0x563e4c563fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.736+0000 7f70c74b9740 1 --2- 172.21.15.103:0/3734542020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c5572f0 0x563e4c561670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.736+0000 7f70c74b9740 1 --2- 172.21.15.103:0/3734542020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c556db0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.736+0000 7f70c74b9740 1 -- 172.21.15.103:0/3734542020 >> 172.21.15.103:0/3734542020 conn(0x563e4c552f60 msgr2=0x563e4c553340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:41.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.736+0000 7f70c74b9740 1 -- 172.21.15.103:0/3734542020 shutdown_connections 2024-09-19T06:49:41.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.736+0000 7f70c74b9740 1 -- 172.21.15.103:0/3734542020 wait complete. 2024-09-19T06:49:41.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.736+0000 7f70c74b9740 1 -- start start 2024-09-19T06:49:41.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.736+0000 7f70c74b9740 1 Processor -- start 2024-09-19T06:49:41.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70c74b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c561350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70c74b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c5572f0 0x563e4c55e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70c74b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c561bb0 0x563e4c55ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70c74b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563e4c564780 con 0x563e4c5569d0 2024-09-19T06:49:41.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70c74b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563e4c477f20 con 0x563e4c5572f0 2024-09-19T06:49:41.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70c74b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563e4c564900 con 0x563e4c561bb0 2024-09-19T06:49:41.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70c74b9740 10 client.0 _ll_register_callbacks cb 0x563e4c441030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:41.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70bed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c561350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70be59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c5572f0 0x563e4c55e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70bed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c561350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38136/0 (socket says 172.21.15.103:38136) 2024-09-19T06:49:41.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70bed9d640 1 -- 172.21.15.103:0/13259126 learned_addr learned my addr 172.21.15.103:0/13259126 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:41.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70bf59e640 1 --2- 172.21.15.103:0/13259126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c561bb0 0x563e4c55ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70be59c640 1 -- 172.21.15.103:0/13259126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c561bb0 msgr2=0x563e4c55ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:41.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70be59c640 1 --2- 172.21.15.103:0/13259126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c561bb0 0x563e4c55ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70be59c640 1 -- 172.21.15.103:0/13259126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 msgr2=0x563e4c561350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:41.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70be59c640 1 --2- 172.21.15.103:0/13259126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c561350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70be59c640 1 -- 172.21.15.103:0/13259126 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e4c22e2b0 con 0x563e4c5572f0 2024-09-19T06:49:41.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70bed9d640 1 --2- 172.21.15.103:0/13259126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c561350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:41.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70be59c640 1 --2- 172.21.15.103:0/13259126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c5572f0 0x563e4c55e6a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f70a80cd230 tx=0x7f70a809b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e10f59bf69606733 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:41.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.737+0000 7f70a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:49:41.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.738+0000 7f70a67fc640 1 -- 172.21.15.103:0/13259126 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8020040 con 0x563e4c5572f0 2024-09-19T06:49:41.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.738+0000 7f70a67fc640 1 -- 172.21.15.103:0/13259126 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f70a80cd710 con 0x563e4c5572f0 2024-09-19T06:49:41.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.738+0000 7f70c74b9740 1 -- 172.21.15.103:0/13259126 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b0014dd0 con 0x563e4c5572f0 2024-09-19T06:49:41.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.738+0000 7f70a67fc640 1 -- 172.21.15.103:0/13259126 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f70a80a00a0 con 0x563e4c5572f0 2024-09-19T06:49:41.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.738+0000 7f70c74b9740 10 client.25028 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:41.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.738+0000 7f70c74b9740 1 -- 172.21.15.103:0/13259126 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e4c56ff00 con 0x563e4c5572f0 2024-09-19T06:49:41.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.738+0000 7f70a67fc640 1 -- 172.21.15.103:0/13259126 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b0014dd0 con 0x563e4c5572f0 2024-09-19T06:49:41.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.738+0000 7f70a67fc640 1 -- 172.21.15.103:0/13259126 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f70a8097a40 con 0x563e4c5572f0 2024-09-19T06:49:41.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.738+0000 7f70c74b9740 10 client.25028 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:41.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.738+0000 7f70c74b9740 10 client.25028 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T06:49:41.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.738+0000 7f70c74b9740 20 client.25028 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:41.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.738+0000 7f70c74b9740 1 --2- 172.21.15.103:0/13259126 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x563e4c572490 0x563e4c573900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.738+0000 7f70c74b9740 4 client.25028 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:41.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.738+0000 7f70c74b9740 1 -- 172.21.15.103:0/13259126 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e4c4ca600 con 0x563e4c572490 2024-09-19T06:49:41.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.739+0000 7f70bed9d640 1 --2- 172.21.15.103:0/13259126 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x563e4c572490 0x563e4c573900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.739+0000 7f70bed9d640 1 --2- 172.21.15.103:0/13259126 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x563e4c572490 0x563e4c573900 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d322faf56e8b9487 server_cookie=975d5baf06e97464 in_seq=0 out_seq=0 2024-09-19T06:49:41.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.739+0000 7f70a67fc640 10 client.25028 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:49:41.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.740+0000 7f70a67fc640 1 -- 172.21.15.103:0/13259126 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e4c4ca600 con 0x563e4c572490 2024-09-19T06:49:41.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.740+0000 7f70a67fc640 10 client.25028 handle_command_reply: tid=0 2024-09-19T06:49:41.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.740+0000 7f70c74b9740 1 client.25028 shutdown 2024-09-19T06:49:41.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.740+0000 7f70c74b9740 1 -- 172.21.15.103:0/13259126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c5572f0 msgr2=0x563e4c55e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:41.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.740+0000 7f70c74b9740 1 --2- 172.21.15.103:0/13259126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c5572f0 0x563e4c55e6a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f70a80cd230 tx=0x7f70a809b920 comp rx=0 tx=0).stop 2024-09-19T06:49:41.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.740+0000 7f70c74b9740 1 -- 172.21.15.103:0/13259126 shutdown_connections 2024-09-19T06:49:41.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.740+0000 7f70c74b9740 1 --2- 172.21.15.103:0/13259126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c561bb0 0x563e4c55ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.740+0000 7f70c74b9740 1 --2- 172.21.15.103:0/13259126 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x563e4c572490 0x563e4c573900 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.740+0000 7f70c74b9740 1 --2- 172.21.15.103:0/13259126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c5572f0 0x563e4c55e6a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.740+0000 7f70c74b9740 1 --2- 172.21.15.103:0/13259126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c561350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.740+0000 7f70a67fc640 0 client.25028 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:49:41.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.740+0000 7f70c74b9740 1 -- 172.21.15.103:0/13259126 >> 172.21.15.103:0/13259126 conn(0x563e4c552f60 msgr2=0x563e4c558c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:41.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.740+0000 7f70c74b9740 1 -- 172.21.15.103:0/13259126 shutdown_connections 2024-09-19T06:49:41.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.740+0000 7f70c74b9740 1 -- 172.21.15.103:0/13259126 wait complete. 2024-09-19T06:49:41.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.740+0000 7f70c74b9740 20 client.25028 trim_cache size 0 max 16384 2024-09-19T06:49:41.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.757+0000 7f70c74b9740 1 Processor -- start 2024-09-19T06:49:41.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.757+0000 7f70c74b9740 1 -- start start 2024-09-19T06:49:41.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.757+0000 7f70c74b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c571dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.757+0000 7f70c74b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c5572f0 0x563e4c573900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.757+0000 7f70c74b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c561bb0 0x563e4c57bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.757+0000 7f70c74b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563e4c571c40 con 0x563e4c5569d0 2024-09-19T06:49:41.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.757+0000 7f70c74b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563e4c572310 con 0x563e4c5572f0 2024-09-19T06:49:41.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.757+0000 7f70c74b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563e4c5729e0 con 0x563e4c561bb0 2024-09-19T06:49:41.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.758+0000 7f70bed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c571dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.758+0000 7f70bed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c571dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38152/0 (socket says 172.21.15.103:38152) 2024-09-19T06:49:41.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.758+0000 7f70bed9d640 1 -- 172.21.15.103:0/2155098152 learned_addr learned my addr 172.21.15.103:0/2155098152 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:41.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.758+0000 7f70bf59e640 1 --2- 172.21.15.103:0/2155098152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c561bb0 0x563e4c57bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.758+0000 7f70be59c640 1 --2- 172.21.15.103:0/2155098152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c5572f0 0x563e4c573900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.758+0000 7f70bed9d640 1 -- 172.21.15.103:0/2155098152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c561bb0 msgr2=0x563e4c57bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:41.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.758+0000 7f70bed9d640 1 --2- 172.21.15.103:0/2155098152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c561bb0 0x563e4c57bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.758+0000 7f70bed9d640 1 -- 172.21.15.103:0/2155098152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c5572f0 msgr2=0x563e4c573900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:41.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.758+0000 7f70bed9d640 1 --2- 172.21.15.103:0/2155098152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c5572f0 0x563e4c573900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.758+0000 7f70bed9d640 1 -- 172.21.15.103:0/2155098152 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e4c557120 con 0x563e4c5569d0 2024-09-19T06:49:41.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.758+0000 7f70be59c640 1 --2- 172.21.15.103:0/2155098152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c5572f0 0x563e4c573900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:41.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.758+0000 7f70bed9d640 1 --2- 172.21.15.103:0/2155098152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c571dc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f70b8031550 tx=0x7f70b80058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff1a0b781191d35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:41.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.758+0000 7f70b4ff9640 1 -- 172.21.15.103:0/2155098152 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8035040 con 0x563e4c5569d0 2024-09-19T06:49:41.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.758+0000 7f70b4ff9640 1 -- 172.21.15.103:0/2155098152 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b8005520 con 0x563e4c5569d0 2024-09-19T06:49:41.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.758+0000 7f70b4ff9640 1 -- 172.21.15.103:0/2155098152 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b80432d0 con 0x563e4c5569d0 2024-09-19T06:49:41.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.759+0000 7f70c74b9740 1 -- 172.21.15.103:0/2155098152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 msgr2=0x563e4c571dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:41.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.759+0000 7f70c74b9740 1 --2- 172.21.15.103:0/2155098152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c571dc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f70b8031550 tx=0x7f70b80058b0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.759+0000 7f70c74b9740 1 -- 172.21.15.103:0/2155098152 shutdown_connections 2024-09-19T06:49:41.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.759+0000 7f70c74b9740 1 --2- 172.21.15.103:0/2155098152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c561bb0 0x563e4c57bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.759+0000 7f70c74b9740 1 --2- 172.21.15.103:0/2155098152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c5572f0 0x563e4c573900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.759+0000 7f70c74b9740 1 --2- 172.21.15.103:0/2155098152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c571dc0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.759+0000 7f70c74b9740 1 -- 172.21.15.103:0/2155098152 >> 172.21.15.103:0/2155098152 conn(0x563e4c552f60 msgr2=0x563e4c55cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:41.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.759+0000 7f70c74b9740 1 -- 172.21.15.103:0/2155098152 shutdown_connections 2024-09-19T06:49:41.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.759+0000 7f70c74b9740 1 -- 172.21.15.103:0/2155098152 wait complete. 2024-09-19T06:49:41.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.759+0000 7f70c74b9740 1 -- start start 2024-09-19T06:49:41.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.759+0000 7f70c74b9740 1 Processor -- start 2024-09-19T06:49:41.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70c74b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c5771f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70c74b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c5572f0 0x563e4c577730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70c74b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c561bb0 0x563e4c5d4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70c74b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563e4c573780 con 0x563e4c5569d0 2024-09-19T06:49:41.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70c74b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563e4c5730b0 con 0x563e4c561bb0 2024-09-19T06:49:41.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70c74b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563e4c573e50 con 0x563e4c5572f0 2024-09-19T06:49:41.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70c74b9740 10 client.0 _ll_register_callbacks cb 0x7f70a800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:41.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70bed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c5771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70bed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c5771f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38164/0 (socket says 172.21.15.103:38164) 2024-09-19T06:49:41.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70bed9d640 1 -- 172.21.15.103:0/797358652 learned_addr learned my addr 172.21.15.103:0/797358652 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:41.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70be59c640 1 --2- 172.21.15.103:0/797358652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c5572f0 0x563e4c577730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70bf59e640 1 --2- 172.21.15.103:0/797358652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c561bb0 0x563e4c5d4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70bed9d640 1 -- 172.21.15.103:0/797358652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c5572f0 msgr2=0x563e4c577730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:41.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70bed9d640 1 --2- 172.21.15.103:0/797358652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c5572f0 0x563e4c577730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70bed9d640 1 -- 172.21.15.103:0/797358652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c561bb0 msgr2=0x563e4c5d4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:41.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70bed9d640 1 --2- 172.21.15.103:0/797358652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c561bb0 0x563e4c5d4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70bed9d640 1 -- 172.21.15.103:0/797358652 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e4c5d5200 con 0x563e4c5569d0 2024-09-19T06:49:41.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70bf59e640 1 --2- 172.21.15.103:0/797358652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c561bb0 0x563e4c5d4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:41.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70bed9d640 1 --2- 172.21.15.103:0/797358652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c5771f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009900 tx=0x7f70b8005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249084b205ae1a42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:41.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.760+0000 7f70a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:41.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.761+0000 7f70a67fc640 1 -- 172.21.15.103:0/797358652 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8051020 con 0x563e4c5569d0 2024-09-19T06:49:41.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.761+0000 7f70c74b9740 1 -- 172.21.15.103:0/797358652 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b8005520 con 0x563e4c5569d0 2024-09-19T06:49:41.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.761+0000 7f70a67fc640 1 -- 172.21.15.103:0/797358652 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f70b8004dd0 con 0x563e4c5569d0 2024-09-19T06:49:41.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.761+0000 7f70a67fc640 1 -- 172.21.15.103:0/797358652 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f70b805c050 con 0x563e4c5569d0 2024-09-19T06:49:41.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.761+0000 7f70c74b9740 10 client.15528 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:41.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.761+0000 7f70c74b9740 1 -- 172.21.15.103:0/797358652 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e4c5dc500 con 0x563e4c5569d0 2024-09-19T06:49:41.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.761+0000 7f70a67fc640 1 -- 172.21.15.103:0/797358652 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b8005520 con 0x563e4c5569d0 2024-09-19T06:49:41.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.761+0000 7f70a67fc640 1 -- 172.21.15.103:0/797358652 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f70b8032050 con 0x563e4c5569d0 2024-09-19T06:49:41.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.761+0000 7f70c74b9740 10 client.15528 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:41.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.761+0000 7f70c74b9740 10 client.15528 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T06:49:41.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.761+0000 7f70c74b9740 20 client.15528 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:41.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.761+0000 7f70c74b9740 1 --2- 172.21.15.103:0/797358652 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x563e4c5e0c70 0x563e4c5df980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:41.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.761+0000 7f70c74b9740 4 client.15528 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:49:41.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.761+0000 7f70c74b9740 1 -- 172.21.15.103:0/797358652 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "status"}) -- 0x563e4c4c4b10 con 0x563e4c5e0c70 2024-09-19T06:49:41.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.761+0000 7f70be59c640 1 --2- 172.21.15.103:0/797358652 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x563e4c5e0c70 0x563e4c5df980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:41.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70be59c640 1 --2- 172.21.15.103:0/797358652 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x563e4c5e0c70 0x563e4c5df980 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b051df87dd0ff75d server_cookie=36b760f2a79f1aab in_seq=0 out_seq=0 2024-09-19T06:49:41.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70a67fc640 10 client.15528 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:49:41.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70a67fc640 1 -- 172.21.15.103:0/797358652 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x563e4c4c4b10 con 0x563e4c5e0c70 2024-09-19T06:49:41.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70a67fc640 10 client.15528 handle_command_reply: tid=0 2024-09-19T06:49:41.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70c74b9740 1 client.15528 shutdown 2024-09-19T06:49:41.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70c74b9740 1 -- 172.21.15.103:0/797358652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 msgr2=0x563e4c5771f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:41.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70c74b9740 1 --2- 172.21.15.103:0/797358652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c5771f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009900 tx=0x7f70b8005760 comp rx=0 tx=0).stop 2024-09-19T06:49:41.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70c74b9740 1 -- 172.21.15.103:0/797358652 shutdown_connections 2024-09-19T06:49:41.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70c74b9740 1 --2- 172.21.15.103:0/797358652 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x563e4c5e0c70 0x563e4c5df980 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70c74b9740 1 --2- 172.21.15.103:0/797358652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e4c561bb0 0x563e4c5d4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70c74b9740 1 --2- 172.21.15.103:0/797358652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e4c5572f0 0x563e4c577730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70a67fc640 0 client.15528 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:49:41.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70c74b9740 1 --2- 172.21.15.103:0/797358652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e4c5569d0 0x563e4c5771f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70c74b9740 1 -- 172.21.15.103:0/797358652 >> 172.21.15.103:0/797358652 conn(0x563e4c552f60 msgr2=0x563e4c551800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:41.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70c74b9740 1 -- 172.21.15.103:0/797358652 shutdown_connections 2024-09-19T06:49:41.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70c74b9740 1 -- 172.21.15.103:0/797358652 wait complete. 2024-09-19T06:49:41.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.762+0000 7f70c74b9740 20 client.15528 trim_cache size 0 max 16384 2024-09-19T06:49:41.770 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:49:41.770 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:49:41.771 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 2, 2024-09-19T06:49:41.771 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24454, 2024-09-19T06:49:41.771 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T06:49:41.771 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T06:49:41.771 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T06:49:41.771 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 127.883601246, 2024-09-19T06:49:41.771 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 27, 2024-09-19T06:49:41.771 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 83, 2024-09-19T06:49:41.771 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:49:41.771 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 155.50099774 2024-09-19T06:49:41.771 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:49:41.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.763+0000 7f70c50e0640 1 -- 172.21.15.103:0/2635613920 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70ac0a34d0 msgr2=0x7f70ac0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:41.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.763+0000 7f70c50e0640 1 --2- 172.21.15.103:0/2635613920 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70ac0a34d0 0x7f70ac0a5990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f70b001e000 tx=0x7f70b001f000 comp rx=0 tx=0).stop 2024-09-19T06:49:41.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.763+0000 7f70c50e0640 1 -- 172.21.15.103:0/2635613920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0101420 msgr2=0x7f70c01989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:41.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.763+0000 7f70c50e0640 1 --2- 172.21.15.103:0/2635613920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0101420 0x7f70c01989c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f70a800d950 tx=0x7f70a800de10 comp rx=0 tx=0).stop 2024-09-19T06:49:41.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.763+0000 7f70c50e0640 1 -- 172.21.15.103:0/2635613920 shutdown_connections 2024-09-19T06:49:41.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.763+0000 7f70c50e0640 1 --2- 172.21.15.103:0/2635613920 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70ac0a34d0 0x7f70ac0a5990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.763+0000 7f70c50e0640 1 --2- 172.21.15.103:0/2635613920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0106a80 0x7f70c019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.763+0000 7f70c50e0640 1 --2- 172.21.15.103:0/2635613920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0101420 0x7f70c01989c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.763+0000 7f70c50e0640 1 --2- 172.21.15.103:0/2635613920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c0100a80 0x7f70c0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:41.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.763+0000 7f70c50e0640 1 -- 172.21.15.103:0/2635613920 >> 172.21.15.103:0/2635613920 conn(0x7f70c00fc7a0 msgr2=0x7f70c006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:41.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.763+0000 7f70c50e0640 1 -- 172.21.15.103:0/2635613920 shutdown_connections 2024-09-19T06:49:41.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:41.763+0000 7f70c50e0640 1 -- 172.21.15.103:0/2635613920 wait complete. 2024-09-19T06:49:42.466 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T06:49:42.715 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:42.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:42 smithi174 ceph-mon[35637]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.4 KiB/s wr, 18 op/s 2024-09-19T06:49:42.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:42 smithi103 ceph-mon[31158]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.4 KiB/s wr, 18 op/s 2024-09-19T06:49:42.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:42 smithi134 ceph-mon[35356]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.4 KiB/s wr, 18 op/s 2024-09-19T06:49:43.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.178+0000 7f22bd53b640 1 -- 172.21.15.103:0/879199472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22b8101df0 msgr2=0x7f22b81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:43.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.178+0000 7f22bd53b640 1 --2- 172.21.15.103:0/879199472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22b8101df0 0x7f22b81021d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009900 tx=0x7f22b002a410 comp rx=0 tx=0).stop 2024-09-19T06:49:43.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.178+0000 7f22bd53b640 1 -- 172.21.15.103:0/879199472 shutdown_connections 2024-09-19T06:49:43.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.178+0000 7f22bd53b640 1 --2- 172.21.15.103:0/879199472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22b8106f20 0x7f22b810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.178+0000 7f22bd53b640 1 --2- 172.21.15.103:0/879199472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22b81027a0 0x7f22b81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.178+0000 7f22bd53b640 1 --2- 172.21.15.103:0/879199472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22b8101df0 0x7f22b81021d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.178+0000 7f22bd53b640 1 -- 172.21.15.103:0/879199472 >> 172.21.15.103:0/879199472 conn(0x7f22b80fd8b0 msgr2=0x7f22b80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:43.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.178+0000 7f22bd53b640 1 -- 172.21.15.103:0/879199472 shutdown_connections 2024-09-19T06:49:43.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.178+0000 7f22bd53b640 1 -- 172.21.15.103:0/879199472 wait complete. 2024-09-19T06:49:43.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.179+0000 7f22bd53b640 1 Processor -- start 2024-09-19T06:49:43.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.179+0000 7f22bd53b640 1 -- start start 2024-09-19T06:49:43.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.179+0000 7f22bd53b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22b8101df0 0x7f22b8198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.179+0000 7f22bd53b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22b81027a0 0x7f22b8198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.179+0000 7f22bd53b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22b8106f20 0x7f22b819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.179+0000 7f22bd53b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f22b8111f40 con 0x7f22b81027a0 2024-09-19T06:49:43.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.179+0000 7f22bd53b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f22b8111dc0 con 0x7f22b8106f20 2024-09-19T06:49:43.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.179+0000 7f22bd53b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f22b81120c0 con 0x7f22b8101df0 2024-09-19T06:49:43.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.179+0000 7f22b67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22b81027a0 0x7f22b8198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.179+0000 7f22b67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22b81027a0 0x7f22b8198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38180/0 (socket says 172.21.15.103:38180) 2024-09-19T06:49:43.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.179+0000 7f22b67fc640 1 -- 172.21.15.103:0/101891936 learned_addr learned my addr 172.21.15.103:0/101891936 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:43.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.180+0000 7f22b77fe640 1 --2- 172.21.15.103:0/101891936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22b8106f20 0x7f22b819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.180+0000 7f22b6ffd640 1 --2- 172.21.15.103:0/101891936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22b8101df0 0x7f22b8198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.180+0000 7f22b67fc640 1 -- 172.21.15.103:0/101891936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22b8101df0 msgr2=0x7f22b8198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:43.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.180+0000 7f22b67fc640 1 --2- 172.21.15.103:0/101891936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22b8101df0 0x7f22b8198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.180+0000 7f22b67fc640 1 -- 172.21.15.103:0/101891936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22b8106f20 msgr2=0x7f22b819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:43.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.180+0000 7f22b67fc640 1 --2- 172.21.15.103:0/101891936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22b8106f20 0x7f22b819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.180+0000 7f22b67fc640 1 -- 172.21.15.103:0/101891936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b819d4f0 con 0x7f22b81027a0 2024-09-19T06:49:43.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.180+0000 7f22b77fe640 1 --2- 172.21.15.103:0/101891936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22b8106f20 0x7f22b819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:43.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.180+0000 7f22b67fc640 1 --2- 172.21.15.103:0/101891936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22b81027a0 0x7f22b8198a80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f22a000c890 tx=0x7f22a000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:43.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.180+0000 7f22affff640 1 -- 172.21.15.103:0/101891936 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0004220 con 0x7f22b81027a0 2024-09-19T06:49:43.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.180+0000 7f22affff640 1 -- 172.21.15.103:0/101891936 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a00043c0 con 0x7f22b81027a0 2024-09-19T06:49:43.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.180+0000 7f22affff640 1 -- 172.21.15.103:0/101891936 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a00049e0 con 0x7f22b81027a0 2024-09-19T06:49:43.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.180+0000 7f22bd53b640 1 -- 172.21.15.103:0/101891936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b819d7e0 con 0x7f22b81027a0 2024-09-19T06:49:43.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.180+0000 7f22bd53b640 1 -- 172.21.15.103:0/101891936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b81a50c0 con 0x7f22b81027a0 2024-09-19T06:49:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.182+0000 7f22affff640 1 -- 172.21.15.103:0/101891936 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f22a00079b0 con 0x7f22b81027a0 2024-09-19T06:49:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.183+0000 7f22bf914740 1 Processor -- start 2024-09-19T06:49:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.183+0000 7f22bf914740 1 -- start start 2024-09-19T06:49:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.183+0000 7f22affff640 1 --2- 172.21.15.103:0/101891936 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f22a40a3550 0x7f22a40a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.183+0000 7f22bf914740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70519d0 0x55c2e7051db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.183+0000 7f22bf914740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e70522f0 0x55c2e705c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.183+0000 7f22bf914740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e705cbb0 0x55c2e705efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.183+0000 7f22bf914740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2e6fb8090 con 0x55c2e70519d0 2024-09-19T06:49:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.183+0000 7f22bf914740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c2e6f5fa70 con 0x55c2e70522f0 2024-09-19T06:49:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.183+0000 7f22bf914740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2e6fb4050 con 0x55c2e705cbb0 2024-09-19T06:49:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.183+0000 7f22b77fe640 1 --2- 172.21.15.103:0/101891936 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f22a40a3550 0x7f22a40a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.183+0000 7f22b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70519d0 0x55c2e7051db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.183+0000 7f22b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70519d0 0x55c2e7051db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38190/0 (socket says 172.21.15.103:38190) 2024-09-19T06:49:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.183+0000 7f22b6ffd640 1 -- 172.21.15.103:0/1231594248 learned_addr learned my addr 172.21.15.103:0/1231594248 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.183+0000 7f22affff640 1 -- 172.21.15.103:0/101891936 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f22a00d0090 con 0x7f22b81027a0 2024-09-19T06:49:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.184+0000 7f22b77fe640 1 --2- 172.21.15.103:0/1231594248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e705cbb0 0x55c2e705efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.184+0000 7f22b67fc640 1 --2- 172.21.15.103:0/1231594248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e70522f0 0x55c2e705c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.184+0000 7f22b6ffd640 1 -- 172.21.15.103:0/1231594248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e705cbb0 msgr2=0x55c2e705efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.184+0000 7f22b6ffd640 1 --2- 172.21.15.103:0/1231594248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e705cbb0 0x55c2e705efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.184+0000 7f22b6ffd640 1 -- 172.21.15.103:0/1231594248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e70522f0 msgr2=0x55c2e705c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.184+0000 7f22b6ffd640 1 --2- 172.21.15.103:0/1231594248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e70522f0 0x55c2e705c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.184+0000 7f22b6ffd640 1 -- 172.21.15.103:0/1231594248 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2e6f6c130 con 0x55c2e70519d0 2024-09-19T06:49:43.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.184+0000 7f22b77fe640 1 --2- 172.21.15.103:0/101891936 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f22a40a3550 0x7f22a40a5a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f22a8005cc0 tx=0x7f22a8012a30 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:43.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.184+0000 7f22b6ffd640 1 --2- 172.21.15.103:0/1231594248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70519d0 0x55c2e7051db0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f22b0005ab0 tx=0x7f22b0031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31fa120be9861208 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:43.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.184+0000 7f22ad7fa640 1 -- 172.21.15.103:0/1231594248 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b003f070 con 0x55c2e70519d0 2024-09-19T06:49:43.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.184+0000 7f22ad7fa640 1 -- 172.21.15.103:0/1231594248 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22b00319a0 con 0x55c2e70519d0 2024-09-19T06:49:43.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.184+0000 7f22ad7fa640 1 -- 172.21.15.103:0/1231594248 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b0035070 con 0x55c2e70519d0 2024-09-19T06:49:43.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.185+0000 7f22bf914740 1 -- 172.21.15.103:0/1231594248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70519d0 msgr2=0x55c2e7051db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:43.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.185+0000 7f22bf914740 1 --2- 172.21.15.103:0/1231594248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70519d0 0x55c2e7051db0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f22b0005ab0 tx=0x7f22b0031630 comp rx=0 tx=0).stop 2024-09-19T06:49:43.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.185+0000 7f22bf914740 1 -- 172.21.15.103:0/1231594248 shutdown_connections 2024-09-19T06:49:43.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.185+0000 7f22bf914740 1 --2- 172.21.15.103:0/1231594248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e705cbb0 0x55c2e705efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.185+0000 7f22bf914740 1 --2- 172.21.15.103:0/1231594248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e70522f0 0x55c2e705c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.185+0000 7f22bf914740 1 --2- 172.21.15.103:0/1231594248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70519d0 0x55c2e7051db0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.185+0000 7f22bf914740 1 -- 172.21.15.103:0/1231594248 >> 172.21.15.103:0/1231594248 conn(0x55c2e704df60 msgr2=0x55c2e704e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.185+0000 7f22bf914740 1 -- 172.21.15.103:0/1231594248 shutdown_connections 2024-09-19T06:49:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.185+0000 7f22bf914740 1 -- 172.21.15.103:0/1231594248 wait complete. 2024-09-19T06:49:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.185+0000 7f22bf914740 1 -- start start 2024-09-19T06:49:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.185+0000 7f22bf914740 1 Processor -- start 2024-09-19T06:49:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22bf914740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70519d0 0x55c2e705c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22bf914740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e70522f0 0x55c2e7059720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22bf914740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e705cbb0 0x55c2e7059c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22bf914740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2e705f780 con 0x55c2e70519d0 2024-09-19T06:49:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22bf914740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c2e6f72f20 con 0x55c2e70522f0 2024-09-19T06:49:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22bf914740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2e705f900 con 0x55c2e705cbb0 2024-09-19T06:49:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70519d0 0x55c2e705c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22bf914740 10 client.0 _ll_register_callbacks cb 0x55c2e6f3c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70519d0 0x55c2e705c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38200/0 (socket says 172.21.15.103:38200) 2024-09-19T06:49:43.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22b6ffd640 1 -- 172.21.15.103:0/4284477166 learned_addr learned my addr 172.21.15.103:0/4284477166 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:43.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22b77fe640 1 --2- 172.21.15.103:0/4284477166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e705cbb0 0x55c2e7059c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22b67fc640 1 --2- 172.21.15.103:0/4284477166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e70522f0 0x55c2e7059720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22b6ffd640 1 -- 172.21.15.103:0/4284477166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e705cbb0 msgr2=0x55c2e7059c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:43.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22b6ffd640 1 --2- 172.21.15.103:0/4284477166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e705cbb0 0x55c2e7059c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22b6ffd640 1 -- 172.21.15.103:0/4284477166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e70522f0 msgr2=0x55c2e7059720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:43.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22b6ffd640 1 --2- 172.21.15.103:0/4284477166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e70522f0 0x55c2e7059720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22b6ffd640 1 -- 172.21.15.103:0/4284477166 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2e6d292b0 con 0x55c2e70519d0 2024-09-19T06:49:43.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22b77fe640 1 --2- 172.21.15.103:0/4284477166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e705cbb0 0x55c2e7059c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:43.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22b67fc640 1 --2- 172.21.15.103:0/4284477166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e70522f0 0x55c2e7059720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:43.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f22b6ffd640 1 --2- 172.21.15.103:0/4284477166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70519d0 0x55c2e705c3d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f22b002a450 tx=0x7f22b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fc8d388f4875c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:43.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.186+0000 7f229e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:43.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.187+0000 7f229e7fc640 1 -- 172.21.15.103:0/4284477166 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b004a020 con 0x55c2e70519d0 2024-09-19T06:49:43.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.187+0000 7f229e7fc640 1 -- 172.21.15.103:0/4284477166 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f22b00049b0 con 0x55c2e70519d0 2024-09-19T06:49:43.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.187+0000 7f22bf914740 1 -- 172.21.15.103:0/4284477166 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22b00319a0 con 0x55c2e70519d0 2024-09-19T06:49:43.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.187+0000 7f229e7fc640 1 -- 172.21.15.103:0/4284477166 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f22b0055050 con 0x55c2e70519d0 2024-09-19T06:49:43.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.187+0000 7f22bf914740 10 client.15540 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:43.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.187+0000 7f22bf914740 1 -- 172.21.15.103:0/4284477166 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2e706b030 con 0x55c2e70519d0 2024-09-19T06:49:43.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.187+0000 7f229e7fc640 1 -- 172.21.15.103:0/4284477166 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22b00319a0 con 0x55c2e70519d0 2024-09-19T06:49:43.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.187+0000 7f229e7fc640 1 -- 172.21.15.103:0/4284477166 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f22b003b430 con 0x55c2e70519d0 2024-09-19T06:49:43.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.187+0000 7f22bf914740 10 client.15540 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:43.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.187+0000 7f22bf914740 10 client.15540 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T06:49:43.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.187+0000 7f22bf914740 20 client.15540 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:43.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.187+0000 7f22bf914740 1 --2- 172.21.15.103:0/4284477166 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c2e706d180 0x55c2e706e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.187+0000 7f22bf914740 4 client.15540 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:43.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.187+0000 7f22bf914740 1 -- 172.21.15.103:0/4284477166 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2e6fc5600 con 0x55c2e706d180 2024-09-19T06:49:43.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.188+0000 7f22b67fc640 1 --2- 172.21.15.103:0/4284477166 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c2e706d180 0x55c2e706e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.188+0000 7f22b67fc640 1 --2- 172.21.15.103:0/4284477166 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c2e706d180 0x55c2e706e5f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4645af4df912495 server_cookie=be754c5f9400044e in_seq=0 out_seq=0 2024-09-19T06:49:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.188+0000 7f229e7fc640 10 client.15540 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:49:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.189+0000 7f229e7fc640 1 -- 172.21.15.103:0/4284477166 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2e6fc5600 con 0x55c2e706d180 2024-09-19T06:49:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.189+0000 7f229e7fc640 10 client.15540 handle_command_reply: tid=0 2024-09-19T06:49:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.189+0000 7f22bf914740 1 client.15540 shutdown 2024-09-19T06:49:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.189+0000 7f22bf914740 1 -- 172.21.15.103:0/4284477166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70519d0 msgr2=0x55c2e705c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.189+0000 7f22bf914740 1 --2- 172.21.15.103:0/4284477166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70519d0 0x55c2e705c3d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f22b002a450 tx=0x7f22b00042f0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.189+0000 7f22bf914740 1 -- 172.21.15.103:0/4284477166 shutdown_connections 2024-09-19T06:49:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.189+0000 7f22bf914740 1 --2- 172.21.15.103:0/4284477166 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c2e706d180 0x55c2e706e5f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.189+0000 7f22bf914740 1 --2- 172.21.15.103:0/4284477166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e705cbb0 0x55c2e7059c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.189+0000 7f22bf914740 1 --2- 172.21.15.103:0/4284477166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e70522f0 0x55c2e7059720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.189+0000 7f22bf914740 1 --2- 172.21.15.103:0/4284477166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70519d0 0x55c2e705c3d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.189+0000 7f22bf914740 1 -- 172.21.15.103:0/4284477166 >> 172.21.15.103:0/4284477166 conn(0x55c2e704df60 msgr2=0x55c2e7053c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.189+0000 7f229e7fc640 0 client.15540 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:49:43.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.189+0000 7f22bf914740 1 -- 172.21.15.103:0/4284477166 shutdown_connections 2024-09-19T06:49:43.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.189+0000 7f22bf914740 1 -- 172.21.15.103:0/4284477166 wait complete. 2024-09-19T06:49:43.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.189+0000 7f22bf914740 20 client.15540 trim_cache size 0 max 16384 2024-09-19T06:49:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.207+0000 7f22bf914740 1 Processor -- start 2024-09-19T06:49:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.207+0000 7f22bf914740 1 -- start start 2024-09-19T06:49:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.207+0000 7f22bf914740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e70519d0 0x55c2e7068260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.207+0000 7f22bf914740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70522f0 0x55c2e70687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.207+0000 7f22bf914740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e705cbb0 0x55c2e7056980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.207+0000 7f22bf914740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2e706c930 con 0x55c2e70522f0 2024-09-19T06:49:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.207+0000 7f22bf914740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c2e706d000 con 0x55c2e70519d0 2024-09-19T06:49:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.207+0000 7f22bf914740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2e706d6d0 con 0x55c2e705cbb0 2024-09-19T06:49:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.207+0000 7f22b67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70522f0 0x55c2e70687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.207+0000 7f22b67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70522f0 0x55c2e70687a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38204/0 (socket says 172.21.15.103:38204) 2024-09-19T06:49:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.207+0000 7f22b67fc640 1 -- 172.21.15.103:0/779553117 learned_addr learned my addr 172.21.15.103:0/779553117 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.207+0000 7f22b6ffd640 1 --2- 172.21.15.103:0/779553117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e70519d0 0x55c2e7068260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.208+0000 7f22b67fc640 1 -- 172.21.15.103:0/779553117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e705cbb0 msgr2=0x55c2e7056980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.208+0000 7f22b77fe640 1 --2- 172.21.15.103:0/779553117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e705cbb0 0x55c2e7056980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.208+0000 7f22b67fc640 1 --2- 172.21.15.103:0/779553117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e705cbb0 0x55c2e7056980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.208+0000 7f22b67fc640 1 -- 172.21.15.103:0/779553117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e70519d0 msgr2=0x55c2e7068260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.208+0000 7f22b67fc640 1 --2- 172.21.15.103:0/779553117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e70519d0 0x55c2e7068260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.208+0000 7f22b67fc640 1 -- 172.21.15.103:0/779553117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2e704dd90 con 0x55c2e70522f0 2024-09-19T06:49:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.208+0000 7f22b67fc640 1 --2- 172.21.15.103:0/779553117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70522f0 0x55c2e70687a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f22a0098b00 tx=0x7f22a001dfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe264a5a8f82d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.208+0000 7f22acff9640 1 -- 172.21.15.103:0/779553117 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a001d040 con 0x55c2e70522f0 2024-09-19T06:49:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.208+0000 7f22acff9640 1 -- 172.21.15.103:0/779553117 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a009c130 con 0x55c2e70522f0 2024-09-19T06:49:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.208+0000 7f22acff9640 1 -- 172.21.15.103:0/779553117 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a00987a0 con 0x55c2e70522f0 2024-09-19T06:49:43.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.209+0000 7f22bf914740 1 -- 172.21.15.103:0/779553117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70522f0 msgr2=0x55c2e70687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:43.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.209+0000 7f22bf914740 1 --2- 172.21.15.103:0/779553117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70522f0 0x55c2e70687a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f22a0098b00 tx=0x7f22a001dfb0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.209+0000 7f22bf914740 1 -- 172.21.15.103:0/779553117 shutdown_connections 2024-09-19T06:49:43.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.209+0000 7f22bf914740 1 --2- 172.21.15.103:0/779553117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e705cbb0 0x55c2e7056980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.209+0000 7f22bf914740 1 --2- 172.21.15.103:0/779553117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70522f0 0x55c2e70687a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.209+0000 7f22bf914740 1 --2- 172.21.15.103:0/779553117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e70519d0 0x55c2e7068260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.209+0000 7f22bf914740 1 -- 172.21.15.103:0/779553117 >> 172.21.15.103:0/779553117 conn(0x55c2e704df60 msgr2=0x55c2e7067b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:43.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.209+0000 7f22bf914740 1 -- 172.21.15.103:0/779553117 shutdown_connections 2024-09-19T06:49:43.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.209+0000 7f22bf914740 1 -- 172.21.15.103:0/779553117 wait complete. 2024-09-19T06:49:43.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.209+0000 7f22bf914740 1 -- start start 2024-09-19T06:49:43.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.209+0000 7f22bf914740 1 Processor -- start 2024-09-19T06:49:43.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22bf914740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e70519d0 0x55c2e7076c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22bf914740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70522f0 0x55c2e7077140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22bf914740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e705cbb0 0x55c2e7072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22bf914740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2e706e470 con 0x55c2e70522f0 2024-09-19T06:49:43.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22bf914740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c2e706dda0 con 0x55c2e705cbb0 2024-09-19T06:49:43.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22bf914740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2e706eb40 con 0x55c2e70519d0 2024-09-19T06:49:43.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22bf914740 10 client.0 _ll_register_callbacks cb 0x7f22b0009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:43.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22b67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70522f0 0x55c2e7077140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22b67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70522f0 0x55c2e7077140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38216/0 (socket says 172.21.15.103:38216) 2024-09-19T06:49:43.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22b67fc640 1 -- 172.21.15.103:0/3447345847 learned_addr learned my addr 172.21.15.103:0/3447345847 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:43.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22b6ffd640 1 --2- 172.21.15.103:0/3447345847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e70519d0 0x55c2e7076c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22b77fe640 1 --2- 172.21.15.103:0/3447345847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e705cbb0 0x55c2e7072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22b67fc640 1 -- 172.21.15.103:0/3447345847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e70519d0 msgr2=0x55c2e7076c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:43.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22b67fc640 1 --2- 172.21.15.103:0/3447345847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e70519d0 0x55c2e7076c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22b67fc640 1 -- 172.21.15.103:0/3447345847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e705cbb0 msgr2=0x55c2e7072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:43.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22b67fc640 1 --2- 172.21.15.103:0/3447345847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e705cbb0 0x55c2e7072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22b67fc640 1 -- 172.21.15.103:0/3447345847 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2e7072830 con 0x55c2e70522f0 2024-09-19T06:49:43.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22b67fc640 1 --2- 172.21.15.103:0/3447345847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70522f0 0x55c2e7077140 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f22a0005fd0 tx=0x7f22a00a01b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc98e8f0c36b014 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:43.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f229e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:43.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f229e7fc640 1 -- 172.21.15.103:0/3447345847 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a001d040 con 0x55c2e70522f0 2024-09-19T06:49:43.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f229e7fc640 1 -- 172.21.15.103:0/3447345847 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f22a00987a0 con 0x55c2e70522f0 2024-09-19T06:49:43.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.210+0000 7f22bf914740 1 -- 172.21.15.103:0/3447345847 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a009c130 con 0x55c2e70522f0 2024-09-19T06:49:43.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.211+0000 7f229e7fc640 1 -- 172.21.15.103:0/3447345847 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f22a00a4da0 con 0x55c2e70522f0 2024-09-19T06:49:43.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.211+0000 7f22bf914740 10 client.15546 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.211+0000 7f22bf914740 1 -- 172.21.15.103:0/3447345847 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2e70d7490 con 0x55c2e70522f0 2024-09-19T06:49:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.211+0000 7f229e7fc640 1 -- 172.21.15.103:0/3447345847 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a009c130 con 0x55c2e70522f0 2024-09-19T06:49:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.211+0000 7f229e7fc640 1 -- 172.21.15.103:0/3447345847 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f22a0046d20 con 0x55c2e70522f0 2024-09-19T06:49:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.211+0000 7f22bf914740 10 client.15546 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.211+0000 7f22bf914740 10 client.15546 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T06:49:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.211+0000 7f22bf914740 20 client.15546 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.211+0000 7f22bf914740 1 --2- 172.21.15.103:0/3447345847 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c2e70dbbb0 0x55c2e70da8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.211+0000 7f22bf914740 4 client.15546 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:49:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.211+0000 7f22bf914740 1 -- 172.21.15.103:0/3447345847 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "status"}) -- 0x55c2e6fbfb10 con 0x55c2e70dbbb0 2024-09-19T06:49:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.211+0000 7f22b6ffd640 1 --2- 172.21.15.103:0/3447345847 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c2e70dbbb0 0x55c2e70da8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f229e7fc640 10 client.15546 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:49:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f22b6ffd640 1 --2- 172.21.15.103:0/3447345847 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c2e70dbbb0 0x55c2e70da8c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=826461018eaa203a server_cookie=6c2636f0512aaa33 in_seq=0 out_seq=0 2024-09-19T06:49:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f229e7fc640 1 -- 172.21.15.103:0/3447345847 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c2e6fbfb10 con 0x55c2e70dbbb0 2024-09-19T06:49:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f229e7fc640 10 client.15546 handle_command_reply: tid=0 2024-09-19T06:49:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f22bf914740 1 client.15546 shutdown 2024-09-19T06:49:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f22bf914740 1 -- 172.21.15.103:0/3447345847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70522f0 msgr2=0x55c2e7077140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f22bf914740 1 --2- 172.21.15.103:0/3447345847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70522f0 0x55c2e7077140 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f22a0005fd0 tx=0x7f22a00a01b0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f22bf914740 1 -- 172.21.15.103:0/3447345847 shutdown_connections 2024-09-19T06:49:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f22bf914740 1 --2- 172.21.15.103:0/3447345847 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c2e70dbbb0 0x55c2e70da8c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f22bf914740 1 --2- 172.21.15.103:0/3447345847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2e705cbb0 0x55c2e7072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f22bf914740 1 --2- 172.21.15.103:0/3447345847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2e70522f0 0x55c2e7077140 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f22bf914740 1 --2- 172.21.15.103:0/3447345847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2e70519d0 0x55c2e7076c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f229e7fc640 0 client.15546 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:49:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f22bf914740 1 -- 172.21.15.103:0/3447345847 >> 172.21.15.103:0/3447345847 conn(0x55c2e704df60 msgr2=0x55c2e704cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f22bf914740 1 -- 172.21.15.103:0/3447345847 shutdown_connections 2024-09-19T06:49:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f22bf914740 1 -- 172.21.15.103:0/3447345847 wait complete. 2024-09-19T06:49:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.212+0000 7f22bf914740 20 client.15546 trim_cache size 0 max 16384 2024-09-19T06:49:43.220 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:49:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:49:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 1, 2024-09-19T06:49:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24455, 2024-09-19T06:49:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T06:49:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T06:49:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T06:49:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 130.346221308, 2024-09-19T06:49:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 27, 2024-09-19T06:49:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 83, 2024-09-19T06:49:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:49:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 159.42269857700001 2024-09-19T06:49:43.221 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:49:43.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.213+0000 7f22bd53b640 1 -- 172.21.15.103:0/101891936 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f22a40a3550 msgr2=0x7f22a40a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:43.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.213+0000 7f22bd53b640 1 --2- 172.21.15.103:0/101891936 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f22a40a3550 0x7f22a40a5a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f22a8005cc0 tx=0x7f22a8012a30 comp rx=0 tx=0).stop 2024-09-19T06:49:43.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.213+0000 7f22bd53b640 1 -- 172.21.15.103:0/101891936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22b81027a0 msgr2=0x7f22b8198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:43.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.213+0000 7f22bd53b640 1 --2- 172.21.15.103:0/101891936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22b81027a0 0x7f22b8198a80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f22a000c890 tx=0x7f22a000cd50 comp rx=0 tx=0).stop 2024-09-19T06:49:43.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.213+0000 7f22bd53b640 1 -- 172.21.15.103:0/101891936 shutdown_connections 2024-09-19T06:49:43.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.213+0000 7f22bd53b640 1 --2- 172.21.15.103:0/101891936 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f22a40a3550 0x7f22a40a5a10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.213+0000 7f22bd53b640 1 --2- 172.21.15.103:0/101891936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22b8106f20 0x7f22b819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.213+0000 7f22bd53b640 1 --2- 172.21.15.103:0/101891936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22b81027a0 0x7f22b8198a80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.213+0000 7f22bd53b640 1 --2- 172.21.15.103:0/101891936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22b8101df0 0x7f22b8198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:43.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.213+0000 7f22bd53b640 1 -- 172.21.15.103:0/101891936 >> 172.21.15.103:0/101891936 conn(0x7f22b80fd8b0 msgr2=0x7f22b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:43.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.213+0000 7f22bd53b640 1 -- 172.21.15.103:0/101891936 shutdown_connections 2024-09-19T06:49:43.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:43.213+0000 7f22bd53b640 1 -- 172.21.15.103:0/101891936 wait complete. 2024-09-19T06:49:43.882 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T06:49:44.131 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:44.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:44 smithi103 ceph-mon[31158]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-09-19T06:49:44.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:44 smithi174 ceph-mon[35637]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-09-19T06:49:44.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:44 smithi134 ceph-mon[35356]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-09-19T06:49:44.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.586+0000 7fd2a6f99640 1 -- 172.21.15.103:0/535197754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2a0106a80 msgr2=0x7fd2a0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:44.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.586+0000 7fd2a6f99640 1 --2- 172.21.15.103:0/535197754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2a0106a80 0x7fd2a0106e60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009930 tx=0x7fd29c02a420 comp rx=0 tx=0).stop 2024-09-19T06:49:44.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.586+0000 7fd2a6f99640 1 -- 172.21.15.103:0/535197754 shutdown_connections 2024-09-19T06:49:44.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.586+0000 7fd2a6f99640 1 --2- 172.21.15.103:0/535197754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a0101420 0x7fd2a010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.586+0000 7fd2a6f99640 1 --2- 172.21.15.103:0/535197754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2a0100a80 0x7fd2a0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.586+0000 7fd2a6f99640 1 --2- 172.21.15.103:0/535197754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2a0106a80 0x7fd2a0106e60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.586+0000 7fd2a6f99640 1 -- 172.21.15.103:0/535197754 >> 172.21.15.103:0/535197754 conn(0x7fd2a00fc7a0 msgr2=0x7fd2a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:44.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.586+0000 7fd2a6f99640 1 -- 172.21.15.103:0/535197754 shutdown_connections 2024-09-19T06:49:44.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.586+0000 7fd2a6f99640 1 -- 172.21.15.103:0/535197754 wait complete. 2024-09-19T06:49:44.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.587+0000 7fd2a6f99640 1 Processor -- start 2024-09-19T06:49:44.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.587+0000 7fd2a6f99640 1 -- start start 2024-09-19T06:49:44.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.587+0000 7fd2a6f99640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2a0100a80 0x7fd2a0197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.587+0000 7fd2a6f99640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2a0101420 0x7fd2a0197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.587+0000 7fd2a6f99640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a0106a80 0x7fd2a019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.587+0000 7fd2a6f99640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a0110ea0 con 0x7fd2a0100a80 2024-09-19T06:49:44.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.587+0000 7fd2a6f99640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a0110d20 con 0x7fd2a0106a80 2024-09-19T06:49:44.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.587+0000 7fd2a6f99640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a0111020 con 0x7fd2a0101420 2024-09-19T06:49:44.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.588+0000 7fd2a4d35640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2a0100a80 0x7fd2a0197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.588+0000 7fd2a4d35640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2a0100a80 0x7fd2a0197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38244/0 (socket says 172.21.15.103:38244) 2024-09-19T06:49:44.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.588+0000 7fd2a4d35640 1 -- 172.21.15.103:0/156492727 learned_addr learned my addr 172.21.15.103:0/156492727 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:44.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.588+0000 7fd2a5536640 1 --2- 172.21.15.103:0/156492727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a0106a80 0x7fd2a019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.588+0000 7fd297fff640 1 --2- 172.21.15.103:0/156492727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2a0101420 0x7fd2a0197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.588+0000 7fd2a4d35640 1 -- 172.21.15.103:0/156492727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2a0101420 msgr2=0x7fd2a0197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:44.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.588+0000 7fd2a4d35640 1 --2- 172.21.15.103:0/156492727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2a0101420 0x7fd2a0197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.588+0000 7fd2a4d35640 1 -- 172.21.15.103:0/156492727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a0106a80 msgr2=0x7fd2a019bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:44.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.588+0000 7fd2a4d35640 1 --2- 172.21.15.103:0/156492727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a0106a80 0x7fd2a019bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.588+0000 7fd2a4d35640 1 -- 172.21.15.103:0/156492727 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a019c3b0 con 0x7fd2a0100a80 2024-09-19T06:49:44.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.588+0000 7fd297fff640 1 --2- 172.21.15.103:0/156492727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2a0101420 0x7fd2a0197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:44.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.588+0000 7fd2a4d35640 1 --2- 172.21.15.103:0/156492727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2a0100a80 0x7fd2a0197400 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009a50 tx=0x7fd29c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:44.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.588+0000 7fd295ffb640 1 -- 172.21.15.103:0/156492727 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c03f030 con 0x7fd2a0100a80 2024-09-19T06:49:44.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.588+0000 7fd2a6f99640 1 -- 172.21.15.103:0/156492727 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a019c640 con 0x7fd2a0100a80 2024-09-19T06:49:44.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.588+0000 7fd295ffb640 1 -- 172.21.15.103:0/156492727 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29c003ce0 con 0x7fd2a0100a80 2024-09-19T06:49:44.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.588+0000 7fd295ffb640 1 -- 172.21.15.103:0/156492727 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c035070 con 0x7fd2a0100a80 2024-09-19T06:49:44.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.589+0000 7fd2a6f99640 1 -- 172.21.15.103:0/156492727 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a01a3f80 con 0x7fd2a0100a80 2024-09-19T06:49:44.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.591+0000 7fd295ffb640 1 -- 172.21.15.103:0/156492727 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7fd29c042030 con 0x7fd2a0100a80 2024-09-19T06:49:44.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.591+0000 7fd2a9372740 1 Processor -- start 2024-09-19T06:49:44.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.591+0000 7fd2a9372740 1 -- start start 2024-09-19T06:49:44.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.591+0000 7fd2a9372740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884617870 0x55d884617c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.591+0000 7fd2a9372740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884618190 0x55d884624880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.591+0000 7fd2a9372740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884624dc0 0x55d8846271b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd2a9372740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d88457e090 con 0x55d884618190 2024-09-19T06:49:44.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd2a9372740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d884525a70 con 0x55d884617870 2024-09-19T06:49:44.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd2a9372740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d88457a050 con 0x55d884624dc0 2024-09-19T06:49:44.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd295ffb640 1 --2- 172.21.15.103:0/156492727 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2880a34d0 0x7fd2880a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd2a4d35640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884618190 0x55d884624880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd2a4d35640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884618190 0x55d884624880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38248/0 (socket says 172.21.15.103:38248) 2024-09-19T06:49:44.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd2a4d35640 1 -- 172.21.15.103:0/167392350 learned_addr learned my addr 172.21.15.103:0/167392350 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:44.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd2a5536640 1 --2- 172.21.15.103:0/167392350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884617870 0x55d884617c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd295ffb640 1 -- 172.21.15.103:0/156492727 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd29c0f0580 con 0x7fd2a0100a80 2024-09-19T06:49:44.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd2a5536640 1 --2- 172.21.15.103:0/156492727 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2880a34d0 0x7fd2880a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd297fff640 1 --2- 172.21.15.103:0/167392350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884624dc0 0x55d8846271b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd2a4d35640 1 -- 172.21.15.103:0/167392350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884624dc0 msgr2=0x55d8846271b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:44.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd2a4d35640 1 --2- 172.21.15.103:0/167392350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884624dc0 0x55d8846271b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd2a4d35640 1 -- 172.21.15.103:0/167392350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884617870 msgr2=0x55d884617c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:44.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd2a4d35640 1 --2- 172.21.15.103:0/167392350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884617870 0x55d884617c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd2a4d35640 1 -- 172.21.15.103:0/167392350 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d884532130 con 0x55d884618190 2024-09-19T06:49:44.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd2a4d35640 1 --2- 172.21.15.103:0/167392350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884618190 0x55d884624880 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009900 tx=0x7fd29c0314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe05fc247dc566c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:44.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.592+0000 7fd2a5536640 1 --2- 172.21.15.103:0/156492727 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2880a34d0 0x7fd2880a5990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd29800bfd0 tx=0x7fd298007220 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:44.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.593+0000 7fd286ffd640 1 -- 172.21.15.103:0/167392350 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c049030 con 0x55d884618190 2024-09-19T06:49:44.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.593+0000 7fd286ffd640 1 -- 172.21.15.103:0/167392350 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29c0bc6b0 con 0x55d884618190 2024-09-19T06:49:44.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.593+0000 7fd286ffd640 1 -- 172.21.15.103:0/167392350 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c0b8eb0 con 0x55d884618190 2024-09-19T06:49:44.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.593+0000 7fd2a9372740 1 -- 172.21.15.103:0/167392350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884618190 msgr2=0x55d884624880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:44.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.593+0000 7fd2a9372740 1 --2- 172.21.15.103:0/167392350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884618190 0x55d884624880 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009900 tx=0x7fd29c0314b0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.593+0000 7fd2a9372740 1 -- 172.21.15.103:0/167392350 shutdown_connections 2024-09-19T06:49:44.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.593+0000 7fd2a9372740 1 --2- 172.21.15.103:0/167392350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884624dc0 0x55d8846271b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.593+0000 7fd2a9372740 1 --2- 172.21.15.103:0/167392350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884618190 0x55d884624880 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.593+0000 7fd2a9372740 1 --2- 172.21.15.103:0/167392350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884617870 0x55d884617c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.593+0000 7fd2a9372740 1 -- 172.21.15.103:0/167392350 >> 172.21.15.103:0/167392350 conn(0x55d884613f60 msgr2=0x55d884614340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:44.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.593+0000 7fd2a9372740 1 -- 172.21.15.103:0/167392350 shutdown_connections 2024-09-19T06:49:44.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.593+0000 7fd2a9372740 1 -- 172.21.15.103:0/167392350 wait complete. 2024-09-19T06:49:44.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.594+0000 7fd2a9372740 1 -- start start 2024-09-19T06:49:44.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.594+0000 7fd2a9372740 1 Processor -- start 2024-09-19T06:49:44.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.594+0000 7fd2a9372740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884617870 0x55d884624620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.594+0000 7fd2a9372740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884618190 0x55d884621970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.594+0000 7fd2a9372740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884624dc0 0x55d884621eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.594+0000 7fd2a9372740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d884627750 con 0x55d884624dc0 2024-09-19T06:49:44.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.594+0000 7fd2a9372740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d884538f20 con 0x55d884617870 2024-09-19T06:49:44.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.594+0000 7fd2a9372740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d8846278d0 con 0x55d884618190 2024-09-19T06:49:44.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.594+0000 7fd2a9372740 10 client.0 _ll_register_callbacks cb 0x55d884502030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:44.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.594+0000 7fd2a5536640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884624dc0 0x55d884621eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.594+0000 7fd2a5536640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884624dc0 0x55d884621eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38252/0 (socket says 172.21.15.103:38252) 2024-09-19T06:49:44.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.594+0000 7fd2a5536640 1 -- 172.21.15.103:0/1247093670 learned_addr learned my addr 172.21.15.103:0/1247093670 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:44.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd297fff640 1 --2- 172.21.15.103:0/1247093670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884618190 0x55d884621970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd2a4d35640 1 --2- 172.21.15.103:0/1247093670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884617870 0x55d884624620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd2a5536640 1 -- 172.21.15.103:0/1247093670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884618190 msgr2=0x55d884621970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:44.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd2a5536640 1 --2- 172.21.15.103:0/1247093670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884618190 0x55d884621970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd2a5536640 1 -- 172.21.15.103:0/1247093670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884617870 msgr2=0x55d884624620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:44.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd2a5536640 1 --2- 172.21.15.103:0/1247093670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884617870 0x55d884624620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd2a5536640 1 -- 172.21.15.103:0/1247093670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8842ef2b0 con 0x55d884624dc0 2024-09-19T06:49:44.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd2a4d35640 1 --2- 172.21.15.103:0/1247093670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884617870 0x55d884624620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:44.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd2a5536640 1 --2- 172.21.15.103:0/1247093670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884624dc0 0x55d884621eb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd298010bb0 tx=0x7fd298004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6ceca1fe961c52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:44.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd27ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:44.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd27ffff640 1 -- 172.21.15.103:0/1247093670 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298025070 con 0x55d884624dc0 2024-09-19T06:49:44.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd27ffff640 1 -- 172.21.15.103:0/1247093670 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd298003970 con 0x55d884624dc0 2024-09-19T06:49:44.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd2a9372740 1 -- 172.21.15.103:0/1247093670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd29c0bc6b0 con 0x55d884624dc0 2024-09-19T06:49:44.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd27ffff640 1 -- 172.21.15.103:0/1247093670 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd298030050 con 0x55d884624dc0 2024-09-19T06:49:44.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd2a9372740 10 client.15558 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:44.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd2a9372740 1 -- 172.21.15.103:0/1247093670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d884632fc0 con 0x55d884624dc0 2024-09-19T06:49:44.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.595+0000 7fd27ffff640 1 -- 172.21.15.103:0/1247093670 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29c0bc6b0 con 0x55d884624dc0 2024-09-19T06:49:44.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.596+0000 7fd27ffff640 1 -- 172.21.15.103:0/1247093670 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd29801c050 con 0x55d884624dc0 2024-09-19T06:49:44.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.596+0000 7fd2a9372740 10 client.15558 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:44.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.596+0000 7fd2a9372740 10 client.15558 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T06:49:44.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.596+0000 7fd2a9372740 20 client.15558 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:44.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.596+0000 7fd2a9372740 1 --2- 172.21.15.103:0/1247093670 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d884635660 0x55d884636ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.596+0000 7fd2a9372740 4 client.15558 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:44.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.596+0000 7fd2a9372740 1 -- 172.21.15.103:0/1247093670 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d88458b600 con 0x55d884635660 2024-09-19T06:49:44.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.596+0000 7fd2a4d35640 1 --2- 172.21.15.103:0/1247093670 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d884635660 0x55d884636ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.596+0000 7fd2a4d35640 1 --2- 172.21.15.103:0/1247093670 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d884635660 0x55d884636ad0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a75f84f842ad246 server_cookie=7667806ab0a3a1a1 in_seq=0 out_seq=0 2024-09-19T06:49:44.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.596+0000 7fd27ffff640 10 client.15558 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:44.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.597+0000 7fd27ffff640 1 -- 172.21.15.103:0/1247093670 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d88458b600 con 0x55d884635660 2024-09-19T06:49:44.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.597+0000 7fd27ffff640 10 client.15558 handle_command_reply: tid=0 2024-09-19T06:49:44.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.597+0000 7fd2a9372740 1 client.15558 shutdown 2024-09-19T06:49:44.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.597+0000 7fd2a9372740 1 -- 172.21.15.103:0/1247093670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884624dc0 msgr2=0x55d884621eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:44.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.597+0000 7fd2a9372740 1 --2- 172.21.15.103:0/1247093670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884624dc0 0x55d884621eb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd298010bb0 tx=0x7fd298004a40 comp rx=0 tx=0).stop 2024-09-19T06:49:44.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.597+0000 7fd2a9372740 1 -- 172.21.15.103:0/1247093670 shutdown_connections 2024-09-19T06:49:44.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.597+0000 7fd2a9372740 1 --2- 172.21.15.103:0/1247093670 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d884635660 0x55d884636ad0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.597+0000 7fd2a9372740 1 --2- 172.21.15.103:0/1247093670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884624dc0 0x55d884621eb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.597+0000 7fd2a9372740 1 --2- 172.21.15.103:0/1247093670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884618190 0x55d884621970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.597+0000 7fd27ffff640 0 client.15558 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:44.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.597+0000 7fd2a9372740 1 --2- 172.21.15.103:0/1247093670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884617870 0x55d884624620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.597+0000 7fd2a9372740 1 -- 172.21.15.103:0/1247093670 >> 172.21.15.103:0/1247093670 conn(0x55d884613f60 msgr2=0x55d884619cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:44.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.597+0000 7fd2a9372740 1 -- 172.21.15.103:0/1247093670 shutdown_connections 2024-09-19T06:49:44.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.597+0000 7fd2a9372740 1 -- 172.21.15.103:0/1247093670 wait complete. 2024-09-19T06:49:44.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.598+0000 7fd2a9372740 20 client.15558 trim_cache size 0 max 16384 2024-09-19T06:49:44.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a9372740 1 Processor -- start 2024-09-19T06:49:44.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a9372740 1 -- start start 2024-09-19T06:49:44.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a9372740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884617870 0x55d884621230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a9372740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884618190 0x55d88463f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a9372740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884624dc0 0x55d88463f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a9372740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d884634e10 con 0x55d884617870 2024-09-19T06:49:44.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a9372740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d8846354e0 con 0x55d884618190 2024-09-19T06:49:44.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a9372740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d884635bb0 con 0x55d884624dc0 2024-09-19T06:49:44.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a4d35640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884617870 0x55d884621230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a4d35640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884617870 0x55d884621230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38268/0 (socket says 172.21.15.103:38268) 2024-09-19T06:49:44.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a4d35640 1 -- 172.21.15.103:0/2217128782 learned_addr learned my addr 172.21.15.103:0/2217128782 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:44.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a4d35640 1 -- 172.21.15.103:0/2217128782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884624dc0 msgr2=0x55d88463f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:44.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a4d35640 1 --2- 172.21.15.103:0/2217128782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884624dc0 0x55d88463f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd297fff640 1 --2- 172.21.15.103:0/2217128782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884618190 0x55d88463f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a4d35640 1 -- 172.21.15.103:0/2217128782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884618190 msgr2=0x55d88463f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:44.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a4d35640 1 --2- 172.21.15.103:0/2217128782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884618190 0x55d88463f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a4d35640 1 -- 172.21.15.103:0/2217128782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d884613810 con 0x55d884617870 2024-09-19T06:49:44.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2a4d35640 1 --2- 172.21.15.103:0/2217128782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884617870 0x55d884621230 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0c5290 tx=0x7fd29c0b8950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3dcb553ce99e080 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:44.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2867fc640 1 -- 172.21.15.103:0/2217128782 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c049040 con 0x55d884617870 2024-09-19T06:49:44.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2867fc640 1 -- 172.21.15.103:0/2217128782 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29c0c0a60 con 0x55d884617870 2024-09-19T06:49:44.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.616+0000 7fd2867fc640 1 -- 172.21.15.103:0/2217128782 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c031e30 con 0x55d884617870 2024-09-19T06:49:44.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 -- 172.21.15.103:0/2217128782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884617870 msgr2=0x55d884621230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:44.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 --2- 172.21.15.103:0/2217128782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884617870 0x55d884621230 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0c5290 tx=0x7fd29c0b8950 comp rx=0 tx=0).stop 2024-09-19T06:49:44.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 -- 172.21.15.103:0/2217128782 shutdown_connections 2024-09-19T06:49:44.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 --2- 172.21.15.103:0/2217128782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884624dc0 0x55d88463f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 --2- 172.21.15.103:0/2217128782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884618190 0x55d88463f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 --2- 172.21.15.103:0/2217128782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884617870 0x55d884621230 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 -- 172.21.15.103:0/2217128782 >> 172.21.15.103:0/2217128782 conn(0x55d884613f60 msgr2=0x55d884614340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:44.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 -- 172.21.15.103:0/2217128782 shutdown_connections 2024-09-19T06:49:44.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 -- 172.21.15.103:0/2217128782 wait complete. 2024-09-19T06:49:44.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 -- start start 2024-09-19T06:49:44.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 Processor -- start 2024-09-19T06:49:44.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884617870 0x55d8846388c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884618190 0x55d884638e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884624dc0 0x55d884699db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d884636950 con 0x55d884624dc0 2024-09-19T06:49:44.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d884636280 con 0x55d884618190 2024-09-19T06:49:44.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d884637020 con 0x55d884617870 2024-09-19T06:49:44.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 10 client.0 _ll_register_callbacks cb 0x7fd2980137b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:44.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a5536640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884624dc0 0x55d884699db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a5536640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884624dc0 0x55d884699db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38284/0 (socket says 172.21.15.103:38284) 2024-09-19T06:49:44.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a5536640 1 -- 172.21.15.103:0/3015444183 learned_addr learned my addr 172.21.15.103:0/3015444183 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:44.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd297fff640 1 --2- 172.21.15.103:0/3015444183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884618190 0x55d884638e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a4d35640 1 --2- 172.21.15.103:0/3015444183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884617870 0x55d8846388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a5536640 1 -- 172.21.15.103:0/3015444183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884617870 msgr2=0x55d8846388c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:44.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a5536640 1 --2- 172.21.15.103:0/3015444183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884617870 0x55d8846388c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a5536640 1 -- 172.21.15.103:0/3015444183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884618190 msgr2=0x55d884638e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:44.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a5536640 1 --2- 172.21.15.103:0/3015444183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884618190 0x55d884638e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a5536640 1 -- 172.21.15.103:0/3015444183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88469a3f0 con 0x55d884624dc0 2024-09-19T06:49:44.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a5536640 1 --2- 172.21.15.103:0/3015444183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884624dc0 0x55d884699db0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd29800e500 tx=0x7fd298005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfc365226b5504d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:44.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd27ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:44.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd27ffff640 1 -- 172.21.15.103:0/3015444183 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298038020 con 0x55d884624dc0 2024-09-19T06:49:44.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd27ffff640 1 -- 172.21.15.103:0/3015444183 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd29800e6f0 con 0x55d884624dc0 2024-09-19T06:49:44.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 1 -- 172.21.15.103:0/3015444183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd29c0c0a60 con 0x55d884624dc0 2024-09-19T06:49:44.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd27ffff640 1 -- 172.21.15.103:0/3015444183 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd298025070 con 0x55d884624dc0 2024-09-19T06:49:44.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.618+0000 7fd2a9372740 10 client.15564 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:44.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.619+0000 7fd2a9372740 1 -- 172.21.15.103:0/3015444183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8846a1740 con 0x55d884624dc0 2024-09-19T06:49:44.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.619+0000 7fd27ffff640 1 -- 172.21.15.103:0/3015444183 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29c0c0a60 con 0x55d884624dc0 2024-09-19T06:49:44.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.619+0000 7fd27ffff640 1 -- 172.21.15.103:0/3015444183 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd29801c040 con 0x55d884624dc0 2024-09-19T06:49:44.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.619+0000 7fd2a9372740 10 client.15564 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:44.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.619+0000 7fd2a9372740 10 client.15564 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T06:49:44.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.619+0000 7fd2a9372740 20 client.15564 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:44.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.619+0000 7fd2a9372740 1 --2- 172.21.15.103:0/3015444183 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d8846a6330 0x55d8846a5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:44.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.619+0000 7fd2a9372740 4 client.15564 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:49:44.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.619+0000 7fd2a9372740 1 -- 172.21.15.103:0/3015444183 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "status"}) -- 0x55d884585b10 con 0x55d8846a6330 2024-09-19T06:49:44.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.619+0000 7fd2a4d35640 1 --2- 172.21.15.103:0/3015444183 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d8846a6330 0x55d8846a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:44.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.620+0000 7fd27ffff640 10 client.15564 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:44.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.620+0000 7fd2a4d35640 1 --2- 172.21.15.103:0/3015444183 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d8846a6330 0x55d8846a5040 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64d5f301213370f7 server_cookie=dfc376fab4664b92 in_seq=0 out_seq=0 2024-09-19T06:49:44.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.620+0000 7fd27ffff640 1 -- 172.21.15.103:0/3015444183 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d884585b10 con 0x55d8846a6330 2024-09-19T06:49:44.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.620+0000 7fd27ffff640 10 client.15564 handle_command_reply: tid=0 2024-09-19T06:49:44.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.620+0000 7fd2a9372740 1 client.15564 shutdown 2024-09-19T06:49:44.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.620+0000 7fd2a9372740 1 -- 172.21.15.103:0/3015444183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884624dc0 msgr2=0x55d884699db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:44.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.620+0000 7fd2a9372740 1 --2- 172.21.15.103:0/3015444183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884624dc0 0x55d884699db0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd29800e500 tx=0x7fd298005ae0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.620+0000 7fd2a9372740 1 -- 172.21.15.103:0/3015444183 shutdown_connections 2024-09-19T06:49:44.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.620+0000 7fd2a9372740 1 --2- 172.21.15.103:0/3015444183 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d8846a6330 0x55d8846a5040 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.620+0000 7fd2a9372740 1 --2- 172.21.15.103:0/3015444183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d884624dc0 0x55d884699db0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.620+0000 7fd2a9372740 1 --2- 172.21.15.103:0/3015444183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d884618190 0x55d884638e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.620+0000 7fd27ffff640 0 client.15564 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:44.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.620+0000 7fd2a9372740 1 --2- 172.21.15.103:0/3015444183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d884617870 0x55d8846388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.620+0000 7fd2a9372740 1 -- 172.21.15.103:0/3015444183 >> 172.21.15.103:0/3015444183 conn(0x55d884613f60 msgr2=0x55d88461fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:44.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.620+0000 7fd2a9372740 1 -- 172.21.15.103:0/3015444183 shutdown_connections 2024-09-19T06:49:44.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.621+0000 7fd2a9372740 1 -- 172.21.15.103:0/3015444183 wait complete. 2024-09-19T06:49:44.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.621+0000 7fd2a9372740 20 client.15564 trim_cache size 0 max 16384 2024-09-19T06:49:44.627 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:49:44.628 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "d158d934-7651-11ef-bceb-c7b262605968", 2024-09-19T06:49:44.628 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-09-19T06:49:44.628 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24482, 2024-09-19T06:49:44.628 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T06:49:44.628 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T06:49:44.628 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T06:49:44.628 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 138.78378062100001, 2024-09-19T06:49:44.628 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 27, 2024-09-19T06:49:44.628 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 83, 2024-09-19T06:49:44.628 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:49:44.628 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 149.875804805 2024-09-19T06:49:44.628 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:49:44.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.621+0000 7fd2a6f99640 1 -- 172.21.15.103:0/156492727 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2880a34d0 msgr2=0x7fd2880a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:44.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.621+0000 7fd2a6f99640 1 --2- 172.21.15.103:0/156492727 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2880a34d0 0x7fd2880a5990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd29800bfd0 tx=0x7fd298007220 comp rx=0 tx=0).stop 2024-09-19T06:49:44.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.621+0000 7fd2a6f99640 1 -- 172.21.15.103:0/156492727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2a0100a80 msgr2=0x7fd2a0197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:44.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.621+0000 7fd2a6f99640 1 --2- 172.21.15.103:0/156492727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2a0100a80 0x7fd2a0197400 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009a50 tx=0x7fd29c004270 comp rx=0 tx=0).stop 2024-09-19T06:49:44.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.621+0000 7fd2a6f99640 1 -- 172.21.15.103:0/156492727 shutdown_connections 2024-09-19T06:49:44.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.621+0000 7fd2a6f99640 1 --2- 172.21.15.103:0/156492727 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2880a34d0 0x7fd2880a5990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.621+0000 7fd2a6f99640 1 --2- 172.21.15.103:0/156492727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a0106a80 0x7fd2a019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.621+0000 7fd2a6f99640 1 --2- 172.21.15.103:0/156492727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2a0101420 0x7fd2a0197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.621+0000 7fd2a6f99640 1 --2- 172.21.15.103:0/156492727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2a0100a80 0x7fd2a0197400 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:44.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.621+0000 7fd2a6f99640 1 -- 172.21.15.103:0/156492727 >> 172.21.15.103:0/156492727 conn(0x7fd2a00fc7a0 msgr2=0x7fd2a00fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:44.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.621+0000 7fd2a6f99640 1 -- 172.21.15.103:0/156492727 shutdown_connections 2024-09-19T06:49:44.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:44.621+0000 7fd2a6f99640 1 -- 172.21.15.103:0/156492727 wait complete. 2024-09-19T06:49:45.346 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-19T06:49:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:46 smithi174 ceph-mon[35637]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.4 KiB/s wr, 17 op/s 2024-09-19T06:49:46.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:46 smithi103 ceph-mon[31158]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.4 KiB/s wr, 17 op/s 2024-09-19T06:49:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:46 smithi134 ceph-mon[35356]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.4 KiB/s wr, 17 op/s 2024-09-19T06:49:48.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:48 smithi174 ceph-mon[35637]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-09-19T06:49:48.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:48 smithi103 ceph-mon[31158]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-09-19T06:49:48.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:48 smithi134 ceph-mon[35356]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-09-19T06:49:50.331 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:50 smithi103 ceph-mon[31158]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-09-19T06:49:50.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:50 smithi174 ceph-mon[35637]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-09-19T06:49:50.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:50 smithi134 ceph-mon[35356]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-09-19T06:49:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.489+0000 7f6c07caf640 1 -- 172.21.15.103:0/2431309761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c001008d0 msgr2=0x7f6c00100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.489+0000 7f6c07caf640 1 --2- 172.21.15.103:0/2431309761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c001008d0 0x7f6c00100cb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009ca0 tx=0x7f6bf002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.489+0000 7f6c07caf640 1 -- 172.21.15.103:0/2431309761 shutdown_connections 2024-09-19T06:49:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.489+0000 7f6c07caf640 1 --2- 172.21.15.103:0/2431309761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c0010e480 0x7f6c00110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.489+0000 7f6c07caf640 1 --2- 172.21.15.103:0/2431309761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c00101280 0x7f6c0010deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.489+0000 7f6c07caf640 1 --2- 172.21.15.103:0/2431309761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c001008d0 0x7f6c00100cb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.489+0000 7f6c07caf640 1 -- 172.21.15.103:0/2431309761 >> 172.21.15.103:0/2431309761 conn(0x7f6c000fc6b0 msgr2=0x7f6c000fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.489+0000 7f6c07caf640 1 -- 172.21.15.103:0/2431309761 shutdown_connections 2024-09-19T06:49:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.489+0000 7f6c07caf640 1 -- 172.21.15.103:0/2431309761 wait complete. 2024-09-19T06:49:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.489+0000 7f6c07caf640 1 Processor -- start 2024-09-19T06:49:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c07caf640 1 -- start start 2024-09-19T06:49:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c07caf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c001008d0 0x7f6c0019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c07caf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c00101280 0x7f6c0019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c07caf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c0010e480 0x7f6c0019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c07caf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c00114100 con 0x7f6c00101280 2024-09-19T06:49:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c07caf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c00113f80 con 0x7f6c001008d0 2024-09-19T06:49:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c07caf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6c00114280 con 0x7f6c0010e480 2024-09-19T06:49:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c0524a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c00101280 0x7f6c0019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c0524a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c00101280 0x7f6c0019ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43718/0 (socket says 172.21.15.103:43718) 2024-09-19T06:49:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c0524a640 1 -- 172.21.15.103:0/2157524619 learned_addr learned my addr 172.21.15.103:0/2157524619 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c05a4b640 1 --2- 172.21.15.103:0/2157524619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c001008d0 0x7f6c0019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c0624c640 1 --2- 172.21.15.103:0/2157524619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c0010e480 0x7f6c0019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c0524a640 1 -- 172.21.15.103:0/2157524619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c0010e480 msgr2=0x7f6c0019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c0524a640 1 --2- 172.21.15.103:0/2157524619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c0010e480 0x7f6c0019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c0524a640 1 -- 172.21.15.103:0/2157524619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c001008d0 msgr2=0x7f6c0019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c0524a640 1 --2- 172.21.15.103:0/2157524619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c001008d0 0x7f6c0019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c0524a640 1 -- 172.21.15.103:0/2157524619 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0019f5c0 con 0x7f6c00101280 2024-09-19T06:49:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c0624c640 1 --2- 172.21.15.103:0/2157524619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c0010e480 0x7f6c0019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.490+0000 7f6c0524a640 1 --2- 172.21.15.103:0/2157524619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c00101280 0x7f6c0019ab10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400d770 tx=0x7f6bf400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.491+0000 7f6bfeffd640 1 -- 172.21.15.103:0/2157524619 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4017070 con 0x7f6c00101280 2024-09-19T06:49:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.491+0000 7f6bfeffd640 1 -- 172.21.15.103:0/2157524619 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bf4005500 con 0x7f6c00101280 2024-09-19T06:49:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.491+0000 7f6c07caf640 1 -- 172.21.15.103:0/2157524619 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c0019f8b0 con 0x7f6c00101280 2024-09-19T06:49:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.491+0000 7f6bfeffd640 1 -- 172.21.15.103:0/2157524619 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4004e00 con 0x7f6c00101280 2024-09-19T06:49:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.491+0000 7f6c07caf640 1 -- 172.21.15.103:0/2157524619 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c001a7140 con 0x7f6c00101280 2024-09-19T06:49:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.493+0000 7f6bfeffd640 1 -- 172.21.15.103:0/2157524619 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f6bf4004fa0 con 0x7f6c00101280 2024-09-19T06:49:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.494+0000 7f6c0aa42740 1 Processor -- start 2024-09-19T06:49:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.494+0000 7f6c0aa42740 1 -- start start 2024-09-19T06:49:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.494+0000 7f6c0aa42740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323db1af0 0x55a323db1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.494+0000 7f6c0aa42740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323db2410 0x55a323dbc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.494+0000 7f6c0aa42740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323dbccb0 0x55a323dbf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.494+0000 7f6c0aa42740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a323d162f0 con 0x55a323dbccb0 2024-09-19T06:49:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.494+0000 7f6c0aa42740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a323cbe270 con 0x55a323db2410 2024-09-19T06:49:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.494+0000 7f6c0aa42740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a323d122b0 con 0x55a323db1af0 2024-09-19T06:49:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.494+0000 7f6bfeffd640 1 --2- 172.21.15.103:0/2157524619 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6be80a3600 0x7f6be80a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.494+0000 7f6c0524a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323dbccb0 0x55a323dbf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.494+0000 7f6c0524a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323dbccb0 0x55a323dbf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43732/0 (socket says 172.21.15.103:43732) 2024-09-19T06:49:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.494+0000 7f6c0524a640 1 -- 172.21.15.103:0/3562604513 learned_addr learned my addr 172.21.15.103:0/3562604513 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.495+0000 7f6c0624c640 1 --2- 172.21.15.103:0/2157524619 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6be80a3600 0x7f6be80a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.495+0000 7f6c05a4b640 1 --2- 172.21.15.103:0/3562604513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323db2410 0x55a323dbc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.495+0000 7f6c0624c640 1 --2- 172.21.15.103:0/3562604513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323db1af0 0x55a323db1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.495+0000 7f6bfeffd640 1 -- 172.21.15.103:0/2157524619 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6bf40cfa10 con 0x7f6c00101280 2024-09-19T06:49:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.495+0000 7f6c0524a640 1 -- 172.21.15.103:0/3562604513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323db1af0 msgr2=0x55a323db1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.495+0000 7f6c0524a640 1 --2- 172.21.15.103:0/3562604513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323db1af0 0x55a323db1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.495+0000 7f6c0524a640 1 -- 172.21.15.103:0/3562604513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323db2410 msgr2=0x55a323dbc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.495+0000 7f6c0524a640 1 --2- 172.21.15.103:0/3562604513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323db2410 0x55a323dbc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.495+0000 7f6c0524a640 1 -- 172.21.15.103:0/3562604513 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a323cbd340 con 0x55a323dbccb0 2024-09-19T06:49:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.495+0000 7f6c0624c640 1 --2- 172.21.15.103:0/3562604513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323db1af0 0x55a323db1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.495+0000 7f6c0524a640 1 --2- 172.21.15.103:0/3562604513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323dbccb0 0x55a323dbf0a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40cd070 tx=0x7f6bf40048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01e4f662b678b73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.495+0000 7f6c0624c640 1 --2- 172.21.15.103:0/2157524619 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6be80a3600 0x7f6be80a5ac0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80058c0 tx=0x7f6bf8005f30 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.495+0000 7f6be7fff640 1 -- 172.21.15.103:0/3562604513 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4020070 con 0x55a323dbccb0 2024-09-19T06:49:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.495+0000 7f6be7fff640 1 -- 172.21.15.103:0/3562604513 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bf409c170 con 0x55a323dbccb0 2024-09-19T06:49:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.495+0000 7f6be7fff640 1 -- 172.21.15.103:0/3562604513 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40980e0 con 0x55a323dbccb0 2024-09-19T06:49:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.496+0000 7f6c0aa42740 1 -- 172.21.15.103:0/3562604513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323dbccb0 msgr2=0x55a323dbf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.496+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/3562604513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323dbccb0 0x55a323dbf0a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40cd070 tx=0x7f6bf40048a0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.496+0000 7f6c0aa42740 1 -- 172.21.15.103:0/3562604513 shutdown_connections 2024-09-19T06:49:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.496+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/3562604513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323dbccb0 0x55a323dbf0a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.496+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/3562604513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323db2410 0x55a323dbc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.496+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/3562604513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323db1af0 0x55a323db1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.496+0000 7f6c0aa42740 1 -- 172.21.15.103:0/3562604513 >> 172.21.15.103:0/3562604513 conn(0x55a323dae060 msgr2=0x55a323dae440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.496+0000 7f6c0aa42740 1 -- 172.21.15.103:0/3562604513 shutdown_connections 2024-09-19T06:49:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.496+0000 7f6c0aa42740 1 -- 172.21.15.103:0/3562604513 wait complete. 2024-09-19T06:49:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.496+0000 7f6c0aa42740 1 -- start start 2024-09-19T06:49:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c0aa42740 1 Processor -- start 2024-09-19T06:49:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c0aa42740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db1af0 0x55a323db9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c0aa42740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323db2410 0x55a323dba100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c0aa42740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323dbccb0 0x55a323dba640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c0aa42740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a323dbf880 con 0x55a323db1af0 2024-09-19T06:49:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c0aa42740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a323c9f290 con 0x55a323db2410 2024-09-19T06:49:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c0aa42740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a323dbfa00 con 0x55a323dbccb0 2024-09-19T06:49:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c0aa42740 10 client.0 _ll_register_callbacks cb 0x55a323cd0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c05a4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db1af0 0x55a323db9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c05a4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db1af0 0x55a323db9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43748/0 (socket says 172.21.15.103:43748) 2024-09-19T06:49:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c05a4b640 1 -- 172.21.15.103:0/1212822764 learned_addr learned my addr 172.21.15.103:0/1212822764 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c0524a640 1 --2- 172.21.15.103:0/1212822764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323db2410 0x55a323dba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c0624c640 1 --2- 172.21.15.103:0/1212822764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323dbccb0 0x55a323dba640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c05a4b640 1 -- 172.21.15.103:0/1212822764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323dbccb0 msgr2=0x55a323dba640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c05a4b640 1 --2- 172.21.15.103:0/1212822764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323dbccb0 0x55a323dba640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c05a4b640 1 -- 172.21.15.103:0/1212822764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323db2410 msgr2=0x55a323dba100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.497+0000 7f6c05a4b640 1 --2- 172.21.15.103:0/1212822764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323db2410 0x55a323dba100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6c05a4b640 1 -- 172.21.15.103:0/1212822764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a323a29210 con 0x55a323db1af0 2024-09-19T06:49:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6c0524a640 1 --2- 172.21.15.103:0/1212822764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323db2410 0x55a323dba100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6c05a4b640 1 --2- 172.21.15.103:0/1212822764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db1af0 0x55a323db9bc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009bf0 tx=0x7f6bf0031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab49bf2c405ee7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6be57fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6be57fa640 1 -- 172.21.15.103:0/1212822764 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf002f810 con 0x55a323db1af0 2024-09-19T06:49:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6be57fa640 1 -- 172.21.15.103:0/1212822764 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6bf002f9b0 con 0x55a323db1af0 2024-09-19T06:49:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6c0aa42740 1 -- 172.21.15.103:0/1212822764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bf409c170 con 0x55a323db1af0 2024-09-19T06:49:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6be57fa640 1 -- 172.21.15.103:0/1212822764 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6bf0053050 con 0x55a323db1af0 2024-09-19T06:49:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6c0aa42740 10 client.15576 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6c0aa42740 1 -- 172.21.15.103:0/1212822764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a323af5620 con 0x55a323db1af0 2024-09-19T06:49:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6be57fa640 1 -- 172.21.15.103:0/1212822764 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bf409c170 con 0x55a323db1af0 2024-09-19T06:49:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6be57fa640 1 -- 172.21.15.103:0/1212822764 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6bf004e030 con 0x55a323db1af0 2024-09-19T06:49:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6c0aa42740 10 client.15576 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6c0aa42740 10 client.15576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6c0aa42740 20 client.15576 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/1212822764 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a323dccc70 0x55a323dcf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6c0aa42740 4 client.15576 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.498+0000 7f6c0aa42740 1 -- 172.21.15.103:0/1212822764 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a323d23a20 con 0x55a323dccc70 2024-09-19T06:49:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.499+0000 7f6c0524a640 1 --2- 172.21.15.103:0/1212822764 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a323dccc70 0x55a323dcf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.499+0000 7f6be57fa640 10 client.15576 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.499+0000 7f6c0524a640 1 --2- 172.21.15.103:0/1212822764 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a323dccc70 0x55a323dcf080 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6885c17da8aeda3c server_cookie=3ab78fdde7aeefb3 in_seq=0 out_seq=0 2024-09-19T06:49:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.500+0000 7f6be57fa640 1 -- 172.21.15.103:0/1212822764 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a323d23a20 con 0x55a323dccc70 2024-09-19T06:49:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.500+0000 7f6be57fa640 10 client.15576 handle_command_reply: tid=0 2024-09-19T06:49:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.500+0000 7f6c0aa42740 1 client.15576 shutdown 2024-09-19T06:49:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.500+0000 7f6c0aa42740 1 -- 172.21.15.103:0/1212822764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db1af0 msgr2=0x55a323db9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.500+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/1212822764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db1af0 0x55a323db9bc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009bf0 tx=0x7f6bf0031680 comp rx=0 tx=0).stop 2024-09-19T06:49:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.500+0000 7f6c0aa42740 1 -- 172.21.15.103:0/1212822764 shutdown_connections 2024-09-19T06:49:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.500+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/1212822764 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a323dccc70 0x55a323dcf080 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.500+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/1212822764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323dbccb0 0x55a323dba640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.500+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/1212822764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323db2410 0x55a323dba100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.500+0000 7f6be57fa640 0 client.15576 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.500+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/1212822764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db1af0 0x55a323db9bc0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.500+0000 7f6c0aa42740 1 -- 172.21.15.103:0/1212822764 >> 172.21.15.103:0/1212822764 conn(0x55a323dae060 msgr2=0x55a323db3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.500+0000 7f6c0aa42740 1 -- 172.21.15.103:0/1212822764 shutdown_connections 2024-09-19T06:49:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.500+0000 7f6c0aa42740 1 -- 172.21.15.103:0/1212822764 wait complete. 2024-09-19T06:49:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.500+0000 7f6c0aa42740 20 client.15576 trim_cache size 0 max 16384 2024-09-19T06:49:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0aa42740 1 Processor -- start 2024-09-19T06:49:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0aa42740 1 -- start start 2024-09-19T06:49:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0aa42740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323db1af0 0x55a323dba470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0aa42740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db2410 0x55a323dbbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0aa42740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323dbccb0 0x55a323dd49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0aa42740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001af9d0 con 0x55a323db2410 2024-09-19T06:49:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0aa42740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001b1510 con 0x55a323db1af0 2024-09-19T06:49:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0aa42740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001b00a0 con 0x55a323dbccb0 2024-09-19T06:49:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0524a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db2410 0x55a323dbbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0524a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db2410 0x55a323dbbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43752/0 (socket says 172.21.15.103:43752) 2024-09-19T06:49:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0524a640 1 -- 172.21.15.103:0/2342635760 learned_addr learned my addr 172.21.15.103:0/2342635760 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0524a640 1 -- 172.21.15.103:0/2342635760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323dbccb0 msgr2=0x55a323dd49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0524a640 1 --2- 172.21.15.103:0/2342635760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323dbccb0 0x55a323dd49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0524a640 1 -- 172.21.15.103:0/2342635760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323db1af0 msgr2=0x55a323dba470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0524a640 1 --2- 172.21.15.103:0/2342635760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323db1af0 0x55a323dba470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0524a640 1 -- 172.21.15.103:0/2342635760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a323daf1f0 con 0x55a323db2410 2024-09-19T06:49:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.515+0000 7f6c0524a640 1 --2- 172.21.15.103:0/2342635760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db2410 0x55a323dbbb30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6bf409b770 tx=0x7f6bf40aff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef290739209208ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.516+0000 7f6be77fe640 1 -- 172.21.15.103:0/2342635760 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4020040 con 0x55a323db2410 2024-09-19T06:49:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.516+0000 7f6be77fe640 1 -- 172.21.15.103:0/2342635760 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bf409c170 con 0x55a323db2410 2024-09-19T06:49:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.516+0000 7f6be77fe640 1 -- 172.21.15.103:0/2342635760 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40860b0 con 0x55a323db2410 2024-09-19T06:49:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.516+0000 7f6c0aa42740 1 -- 172.21.15.103:0/2342635760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db2410 msgr2=0x55a323dbbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.516+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/2342635760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db2410 0x55a323dbbb30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6bf409b770 tx=0x7f6bf40aff90 comp rx=0 tx=0).stop 2024-09-19T06:49:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.516+0000 7f6c0aa42740 1 -- 172.21.15.103:0/2342635760 shutdown_connections 2024-09-19T06:49:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.516+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/2342635760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323dbccb0 0x55a323dd49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.516+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/2342635760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db2410 0x55a323dbbb30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.516+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/2342635760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323db1af0 0x55a323dba470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.516+0000 7f6c0aa42740 1 -- 172.21.15.103:0/2342635760 >> 172.21.15.103:0/2342635760 conn(0x55a323dae060 msgr2=0x55a323dbb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.516+0000 7f6c0aa42740 1 -- 172.21.15.103:0/2342635760 shutdown_connections 2024-09-19T06:49:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.516+0000 7f6c0aa42740 1 -- 172.21.15.103:0/2342635760 wait complete. 2024-09-19T06:49:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c0aa42740 1 -- start start 2024-09-19T06:49:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c0aa42740 1 Processor -- start 2024-09-19T06:49:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c0aa42740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323db1af0 0x55a323dc4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c0aa42740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db2410 0x55a323dc5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c0aa42740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323dbccb0 0x55a323dc0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c0aa42740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001b0e40 con 0x55a323db2410 2024-09-19T06:49:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c0aa42740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001b0770 con 0x55a323dbccb0 2024-09-19T06:49:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c0aa42740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001ad990 con 0x55a323db1af0 2024-09-19T06:49:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c0aa42740 10 client.0 _ll_register_callbacks cb 0x7f6bf002f3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c0524a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db2410 0x55a323dc5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c0524a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db2410 0x55a323dc5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43758/0 (socket says 172.21.15.103:43758) 2024-09-19T06:49:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c0524a640 1 -- 172.21.15.103:0/4099005103 learned_addr learned my addr 172.21.15.103:0/4099005103 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c05a4b640 1 --2- 172.21.15.103:0/4099005103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323db1af0 0x55a323dc4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c0524a640 1 -- 172.21.15.103:0/4099005103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323db1af0 msgr2=0x55a323dc4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c0624c640 1 --2- 172.21.15.103:0/4099005103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323dbccb0 0x55a323dc0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c0524a640 1 --2- 172.21.15.103:0/4099005103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323db1af0 0x55a323dc4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.517+0000 7f6c0524a640 1 -- 172.21.15.103:0/4099005103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323dbccb0 msgr2=0x55a323dc0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6c0524a640 1 --2- 172.21.15.103:0/4099005103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323dbccb0 0x55a323dc0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6c0524a640 1 -- 172.21.15.103:0/4099005103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a323dc0c70 con 0x55a323db2410 2024-09-19T06:49:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6c0624c640 1 --2- 172.21.15.103:0/4099005103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323dbccb0 0x55a323dc0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6c0524a640 1 --2- 172.21.15.103:0/4099005103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db2410 0x55a323dc5520 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40a48a0 tx=0x7f6bf409fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0daa1c6e0084b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6be57fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6be57fa640 1 -- 172.21.15.103:0/4099005103 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4020070 con 0x55a323db2410 2024-09-19T06:49:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6be57fa640 1 -- 172.21.15.103:0/4099005103 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6bf40860b0 con 0x55a323db2410 2024-09-19T06:49:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6c0aa42740 1 -- 172.21.15.103:0/4099005103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bf409c170 con 0x55a323db2410 2024-09-19T06:49:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6be57fa640 1 -- 172.21.15.103:0/4099005103 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6bf409ba70 con 0x55a323db2410 2024-09-19T06:49:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6c0aa42740 10 client.15582 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6c0aa42740 1 -- 172.21.15.103:0/4099005103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a323e375e0 con 0x55a323db2410 2024-09-19T06:49:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6be57fa640 1 -- 172.21.15.103:0/4099005103 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bf409c170 con 0x55a323db2410 2024-09-19T06:49:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6be57fa640 1 -- 172.21.15.103:0/4099005103 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6bf40cd290 con 0x55a323db2410 2024-09-19T06:49:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6c0aa42740 10 client.15582 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6c0aa42740 10 client.15582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6c0aa42740 20 client.15582 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/4099005103 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a323e37810 0x55a323e39c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6c0aa42740 4 client.15582 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:49:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.518+0000 7f6c0aa42740 1 -- 172.21.15.103:0/4099005103 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a323d1dd70 con 0x55a323e37810 2024-09-19T06:49:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.519+0000 7f6c05a4b640 1 --2- 172.21.15.103:0/4099005103 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a323e37810 0x55a323e39c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.519+0000 7f6c05a4b640 1 --2- 172.21.15.103:0/4099005103 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a323e37810 0x55a323e39c00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb83155547d1b8bf server_cookie=7f09b5a4b0c3bb40 in_seq=0 out_seq=0 2024-09-19T06:49:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.519+0000 7f6be57fa640 10 client.15582 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.523+0000 7f6be57fa640 1 -- 172.21.15.103:0/4099005103 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a323d1dd70 con 0x55a323e37810 2024-09-19T06:49:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.523+0000 7f6be57fa640 10 client.15582 handle_command_reply: tid=0 2024-09-19T06:49:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.523+0000 7f6c0aa42740 1 client.15582 shutdown 2024-09-19T06:49:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.523+0000 7f6c0aa42740 1 -- 172.21.15.103:0/4099005103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db2410 msgr2=0x55a323dc5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.523+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/4099005103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db2410 0x55a323dc5520 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40a48a0 tx=0x7f6bf409fe20 comp rx=0 tx=0).stop 2024-09-19T06:49:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.523+0000 7f6c0aa42740 1 -- 172.21.15.103:0/4099005103 shutdown_connections 2024-09-19T06:49:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.523+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/4099005103 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a323e37810 0x55a323e39c00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.523+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/4099005103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a323dbccb0 0x55a323dc0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.523+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/4099005103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a323db2410 0x55a323dc5520 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.523+0000 7f6be57fa640 0 client.15582 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.523+0000 7f6c0aa42740 1 --2- 172.21.15.103:0/4099005103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a323db1af0 0x55a323dc4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.523+0000 7f6c0aa42740 1 -- 172.21.15.103:0/4099005103 >> 172.21.15.103:0/4099005103 conn(0x55a323dae060 msgr2=0x55a323db3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.523+0000 7f6c0aa42740 1 -- 172.21.15.103:0/4099005103 shutdown_connections 2024-09-19T06:49:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.523+0000 7f6c0aa42740 1 -- 172.21.15.103:0/4099005103 wait complete. 2024-09-19T06:49:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.523+0000 7f6c0aa42740 20 client.15582 trim_cache size 0 max 16384 2024-09-19T06:49:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:49:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:49:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:49:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:49:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.524+0000 7f6c07caf640 1 -- 172.21.15.103:0/2157524619 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6be80a3600 msgr2=0x7f6be80a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.524+0000 7f6c07caf640 1 --2- 172.21.15.103:0/2157524619 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6be80a3600 0x7f6be80a5ac0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80058c0 tx=0x7f6bf8005f30 comp rx=0 tx=0).stop 2024-09-19T06:49:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.524+0000 7f6c07caf640 1 -- 172.21.15.103:0/2157524619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c00101280 msgr2=0x7f6c0019ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.524+0000 7f6c07caf640 1 --2- 172.21.15.103:0/2157524619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c00101280 0x7f6c0019ab10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400d770 tx=0x7f6bf400dc30 comp rx=0 tx=0).stop 2024-09-19T06:49:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.524+0000 7f6c07caf640 1 -- 172.21.15.103:0/2157524619 shutdown_connections 2024-09-19T06:49:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.524+0000 7f6c07caf640 1 --2- 172.21.15.103:0/2157524619 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6be80a3600 0x7f6be80a5ac0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.524+0000 7f6c07caf640 1 --2- 172.21.15.103:0/2157524619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c0010e480 0x7f6c0019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.524+0000 7f6c07caf640 1 --2- 172.21.15.103:0/2157524619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c00101280 0x7f6c0019ab10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.524+0000 7f6c07caf640 1 --2- 172.21.15.103:0/2157524619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c001008d0 0x7f6c0019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.524+0000 7f6c07caf640 1 -- 172.21.15.103:0/2157524619 >> 172.21.15.103:0/2157524619 conn(0x7f6c000fc6b0 msgr2=0x7f6c000feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.524+0000 7f6c07caf640 1 -- 172.21.15.103:0/2157524619 shutdown_connections 2024-09-19T06:49:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:49:51.524+0000 7f6c07caf640 1 -- 172.21.15.103:0/2157524619 wait complete. 2024-09-19T06:49:52.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:52 smithi174 ceph-mon[35637]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-09-19T06:49:52.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:52 smithi103 ceph-mon[31158]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-09-19T06:49:52.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:52 smithi134 ceph-mon[35356]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-09-19T06:49:54.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:54 smithi174 ceph-mon[35637]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:49:54.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:54 smithi103 ceph-mon[31158]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:49:54.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:54 smithi134 ceph-mon[35356]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T06:49:55.670 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:49:55.932 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:56 smithi174 ceph-mon[35637]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T06:49:56.340 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:56 smithi103 ceph-mon[31158]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T06:49:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:56 smithi134 ceph-mon[35356]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T06:49:56.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.423+0000 7f2f97649640 1 -- 172.21.15.103:0/406086863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f90106a70 msgr2=0x7f2f90106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:56.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.423+0000 7f2f97649640 1 --2- 172.21.15.103:0/406086863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f90106a70 0x7f2f90106e50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009930 tx=0x7f2f8c02a420 comp rx=0 tx=0).stop 2024-09-19T06:49:56.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.423+0000 7f2f97649640 1 -- 172.21.15.103:0/406086863 shutdown_connections 2024-09-19T06:49:56.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.423+0000 7f2f97649640 1 --2- 172.21.15.103:0/406086863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f90101410 0x7f2f9010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.423+0000 7f2f97649640 1 --2- 172.21.15.103:0/406086863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f90100a70 0x7f2f90100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.423+0000 7f2f97649640 1 --2- 172.21.15.103:0/406086863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f90106a70 0x7f2f90106e50 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.423+0000 7f2f97649640 1 -- 172.21.15.103:0/406086863 >> 172.21.15.103:0/406086863 conn(0x7f2f900fc790 msgr2=0x7f2f900febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:56.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.423+0000 7f2f97649640 1 -- 172.21.15.103:0/406086863 shutdown_connections 2024-09-19T06:49:56.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.423+0000 7f2f97649640 1 -- 172.21.15.103:0/406086863 wait complete. 2024-09-19T06:49:56.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.424+0000 7f2f97649640 1 Processor -- start 2024-09-19T06:49:56.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.424+0000 7f2f97649640 1 -- start start 2024-09-19T06:49:56.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f97649640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f90100a70 0x7f2f90197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f97649640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f90101410 0x7f2f90197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f97649640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f90106a70 0x7f2f9019bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f97649640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90110e70 con 0x7f2f90106a70 2024-09-19T06:49:56.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f97649640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90110cf0 con 0x7f2f90101410 2024-09-19T06:49:56.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f97649640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90110ff0 con 0x7f2f90100a70 2024-09-19T06:49:56.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f95be6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f90106a70 0x7f2f9019bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f95be6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f90106a70 0x7f2f9019bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43776/0 (socket says 172.21.15.103:43776) 2024-09-19T06:49:56.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f95be6640 1 -- 172.21.15.103:0/286508964 learned_addr learned my addr 172.21.15.103:0/286508964 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:56.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f953e5640 1 --2- 172.21.15.103:0/286508964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f90100a70 0x7f2f90197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f94be4640 1 --2- 172.21.15.103:0/286508964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f90101410 0x7f2f90197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f95be6640 1 -- 172.21.15.103:0/286508964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f90100a70 msgr2=0x7f2f90197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:56.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f95be6640 1 --2- 172.21.15.103:0/286508964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f90100a70 0x7f2f90197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f95be6640 1 -- 172.21.15.103:0/286508964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f90101410 msgr2=0x7f2f90197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:56.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f95be6640 1 --2- 172.21.15.103:0/286508964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f90101410 0x7f2f90197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f95be6640 1 -- 172.21.15.103:0/286508964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9019c2c0 con 0x7f2f90106a70 2024-09-19T06:49:56.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f94be4640 1 --2- 172.21.15.103:0/286508964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f90101410 0x7f2f90197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:56.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f95be6640 1 --2- 172.21.15.103:0/286508964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f90106a70 0x7f2f9019bbe0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800eaa0 tx=0x7f2f8800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:56.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f867fc640 1 -- 172.21.15.103:0/286508964 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8800cdb0 con 0x7f2f90106a70 2024-09-19T06:49:56.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f867fc640 1 -- 172.21.15.103:0/286508964 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f88005020 con 0x7f2f90106a70 2024-09-19T06:49:56.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f97649640 1 -- 172.21.15.103:0/286508964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9019c5b0 con 0x7f2f90106a70 2024-09-19T06:49:56.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.425+0000 7f2f867fc640 1 -- 172.21.15.103:0/286508964 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f880116b0 con 0x7f2f90106a70 2024-09-19T06:49:56.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.426+0000 7f2f97649640 1 -- 172.21.15.103:0/286508964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f901a3e90 con 0x7f2f90106a70 2024-09-19T06:49:56.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.428+0000 7f2f867fc640 1 -- 172.21.15.103:0/286508964 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f2f88005560 con 0x7f2f90106a70 2024-09-19T06:49:56.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.428+0000 7f2f99a22740 1 Processor -- start 2024-09-19T06:49:56.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.428+0000 7f2f867fc640 1 --2- 172.21.15.103:0/286508964 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2f7c0a34d0 0x7f2f7c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f99a22740 1 -- start start 2024-09-19T06:49:56.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f94be4640 1 --2- 172.21.15.103:0/286508964 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2f7c0a34d0 0x7f2f7c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f99a22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa780174a0 0x55fa78017880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f867fc640 1 -- 172.21.15.103:0/286508964 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f2f880d0290 con 0x7f2f90106a70 2024-09-19T06:49:56.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f99a22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa78017dc0 0x55fa78022140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f99a22740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78024890 0x55fa78026c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f99a22740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fa77f7d2e0 con 0x55fa78024890 2024-09-19T06:49:56.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f99a22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fa77f24ce0 con 0x55fa78017dc0 2024-09-19T06:49:56.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f99a22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fa77f792a0 con 0x55fa780174a0 2024-09-19T06:49:56.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f95be6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78024890 0x55fa78026c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f95be6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78024890 0x55fa78026c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43784/0 (socket says 172.21.15.103:43784) 2024-09-19T06:49:56.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f95be6640 1 -- 172.21.15.103:0/2189612857 learned_addr learned my addr 172.21.15.103:0/2189612857 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:56.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f953e5640 1 --2- 172.21.15.103:0/2189612857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa780174a0 0x55fa78017880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f95be6640 1 -- 172.21.15.103:0/2189612857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa780174a0 msgr2=0x55fa78017880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:56.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f95be6640 1 --2- 172.21.15.103:0/2189612857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa780174a0 0x55fa78017880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f95be6640 1 -- 172.21.15.103:0/2189612857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa78017dc0 msgr2=0x55fa78022140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:56.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f95be6640 1 --2- 172.21.15.103:0/2189612857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa78017dc0 0x55fa78022140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f95be6640 1 -- 172.21.15.103:0/2189612857 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa77f09440 con 0x55fa78024890 2024-09-19T06:49:56.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f94be4640 1 --2- 172.21.15.103:0/286508964 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2f7c0a34d0 0x7f2f7c0a5990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2f900fed90 tx=0x7f2f8000a610 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:56.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f953e5640 1 --2- 172.21.15.103:0/2189612857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa780174a0 0x55fa78017880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:56.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.429+0000 7f2f95be6640 1 --2- 172.21.15.103:0/2189612857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78024890 0x55fa78026c80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2f88098cc0 tx=0x7f2f8809c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25a6d40140ebcc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:56.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.430+0000 7f2f7b7fe640 1 -- 172.21.15.103:0/2189612857 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8801f030 con 0x55fa78024890 2024-09-19T06:49:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.430+0000 7f2f7b7fe640 1 -- 172.21.15.103:0/2189612857 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f880a4f60 con 0x55fa78024890 2024-09-19T06:49:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.430+0000 7f2f7b7fe640 1 -- 172.21.15.103:0/2189612857 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f880d0070 con 0x55fa78024890 2024-09-19T06:49:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.430+0000 7f2f99a22740 1 -- 172.21.15.103:0/2189612857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78024890 msgr2=0x55fa78026c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.430+0000 7f2f99a22740 1 --2- 172.21.15.103:0/2189612857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78024890 0x55fa78026c80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2f88098cc0 tx=0x7f2f8809c3b0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.430+0000 7f2f99a22740 1 -- 172.21.15.103:0/2189612857 shutdown_connections 2024-09-19T06:49:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.430+0000 7f2f99a22740 1 --2- 172.21.15.103:0/2189612857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78024890 0x55fa78026c80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.430+0000 7f2f99a22740 1 --2- 172.21.15.103:0/2189612857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa78017dc0 0x55fa78022140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.430+0000 7f2f99a22740 1 --2- 172.21.15.103:0/2189612857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa780174a0 0x55fa78017880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.430+0000 7f2f99a22740 1 -- 172.21.15.103:0/2189612857 >> 172.21.15.103:0/2189612857 conn(0x55fa78013a30 msgr2=0x55fa78013e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.430+0000 7f2f99a22740 1 -- 172.21.15.103:0/2189612857 shutdown_connections 2024-09-19T06:49:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.431+0000 7f2f99a22740 1 -- 172.21.15.103:0/2189612857 wait complete. 2024-09-19T06:49:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.431+0000 7f2f99a22740 1 -- start start 2024-09-19T06:49:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.431+0000 7f2f99a22740 1 Processor -- start 2024-09-19T06:49:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.431+0000 7f2f99a22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa780174a0 0x55fa780220d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.431+0000 7f2f99a22740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78017dc0 0x55fa7801f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.431+0000 7f2f99a22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa78024890 0x55fa7801fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.431+0000 7f2f99a22740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fa78027400 con 0x55fa78017dc0 2024-09-19T06:49:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.431+0000 7f2f99a22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fa77f38050 con 0x55fa780174a0 2024-09-19T06:49:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.431+0000 7f2f99a22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fa78027580 con 0x55fa78024890 2024-09-19T06:49:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.431+0000 7f2f99a22740 10 client.0 _ll_register_callbacks cb 0x55fa77f78160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.431+0000 7f2f94be4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78017dc0 0x55fa7801f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f94be4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78017dc0 0x55fa7801f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43794/0 (socket says 172.21.15.103:43794) 2024-09-19T06:49:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f94be4640 1 -- 172.21.15.103:0/2113582776 learned_addr learned my addr 172.21.15.103:0/2113582776 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f953e5640 1 --2- 172.21.15.103:0/2113582776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa780174a0 0x55fa780220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f95be6640 1 --2- 172.21.15.103:0/2113582776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa78024890 0x55fa7801fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f94be4640 1 -- 172.21.15.103:0/2113582776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa78024890 msgr2=0x55fa7801fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:56.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f94be4640 1 --2- 172.21.15.103:0/2113582776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa78024890 0x55fa7801fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f94be4640 1 -- 172.21.15.103:0/2113582776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa780174a0 msgr2=0x55fa780220d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:56.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f94be4640 1 --2- 172.21.15.103:0/2113582776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa780174a0 0x55fa780220d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f94be4640 1 -- 172.21.15.103:0/2113582776 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa77cee3c0 con 0x55fa78017dc0 2024-09-19T06:49:56.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f953e5640 1 --2- 172.21.15.103:0/2113582776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa780174a0 0x55fa780220d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:56.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f95be6640 1 --2- 172.21.15.103:0/2113582776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa78024890 0x55fa7801fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:56.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f94be4640 1 --2- 172.21.15.103:0/2113582776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78017dc0 0x55fa7801f380 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2f80010990 tx=0x7f2f80010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc131beb44d2b4bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:56.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:56.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f78ff9640 1 -- 172.21.15.103:0/2113582776 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80021070 con 0x55fa78017dc0 2024-09-19T06:49:56.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f78ff9640 1 -- 172.21.15.103:0/2113582776 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f2f8000fd50 con 0x55fa78017dc0 2024-09-19T06:49:56.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f99a22740 1 -- 172.21.15.103:0/2113582776 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f880a4f60 con 0x55fa78017dc0 2024-09-19T06:49:56.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f78ff9640 1 -- 172.21.15.103:0/2113582776 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2f8002d050 con 0x55fa78017dc0 2024-09-19T06:49:56.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f99a22740 10 client.15594 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:56.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.432+0000 7f2f99a22740 1 -- 172.21.15.103:0/2113582776 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa78032f70 con 0x55fa78017dc0 2024-09-19T06:49:56.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.433+0000 7f2f78ff9640 1 -- 172.21.15.103:0/2113582776 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f880a4f60 con 0x55fa78017dc0 2024-09-19T06:49:56.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.433+0000 7f2f78ff9640 1 -- 172.21.15.103:0/2113582776 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2f80025570 con 0x55fa78017dc0 2024-09-19T06:49:56.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.433+0000 7f2f99a22740 10 client.15594 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:56.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.433+0000 7f2f99a22740 10 client.15594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:56.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.433+0000 7f2f99a22740 20 client.15594 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:56.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.433+0000 7f2f99a22740 1 --2- 172.21.15.103:0/2113582776 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fa78034380 0x55fa78036800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.433+0000 7f2f99a22740 4 client.15594 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:56.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.433+0000 7f2f99a22740 1 -- 172.21.15.103:0/2113582776 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa77f8a850 con 0x55fa78034380 2024-09-19T06:49:56.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.433+0000 7f2f953e5640 1 --2- 172.21.15.103:0/2113582776 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fa78034380 0x55fa78036800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.434+0000 7f2f78ff9640 10 client.15594 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:56.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.434+0000 7f2f953e5640 1 --2- 172.21.15.103:0/2113582776 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fa78034380 0x55fa78036800 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b93466ad59d870e server_cookie=c00198a1d194189f in_seq=0 out_seq=0 2024-09-19T06:49:56.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.434+0000 7f2f78ff9640 1 -- 172.21.15.103:0/2113582776 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa77f8a850 con 0x55fa78034380 2024-09-19T06:49:56.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.434+0000 7f2f78ff9640 10 client.15594 handle_command_reply: tid=0 2024-09-19T06:49:56.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.434+0000 7f2f99a22740 1 client.15594 shutdown 2024-09-19T06:49:56.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.435+0000 7f2f99a22740 1 -- 172.21.15.103:0/2113582776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78017dc0 msgr2=0x55fa7801f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:56.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.435+0000 7f2f99a22740 1 --2- 172.21.15.103:0/2113582776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78017dc0 0x55fa7801f380 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2f80010990 tx=0x7f2f80010d90 comp rx=0 tx=0).stop 2024-09-19T06:49:56.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.435+0000 7f2f99a22740 1 -- 172.21.15.103:0/2113582776 shutdown_connections 2024-09-19T06:49:56.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.435+0000 7f2f99a22740 1 --2- 172.21.15.103:0/2113582776 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fa78034380 0x55fa78036800 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.435+0000 7f2f99a22740 1 --2- 172.21.15.103:0/2113582776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa78024890 0x55fa7801fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.435+0000 7f2f99a22740 1 --2- 172.21.15.103:0/2113582776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78017dc0 0x55fa7801f380 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.435+0000 7f2f99a22740 1 --2- 172.21.15.103:0/2113582776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa780174a0 0x55fa780220d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.435+0000 7f2f99a22740 1 -- 172.21.15.103:0/2113582776 >> 172.21.15.103:0/2113582776 conn(0x55fa78013a30 msgr2=0x55fa78019790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:56.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.435+0000 7f2f78ff9640 0 client.15594 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:56.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.435+0000 7f2f99a22740 1 -- 172.21.15.103:0/2113582776 shutdown_connections 2024-09-19T06:49:56.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.435+0000 7f2f99a22740 1 -- 172.21.15.103:0/2113582776 wait complete. 2024-09-19T06:49:56.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.435+0000 7f2f99a22740 20 client.15594 trim_cache size 0 max 16384 2024-09-19T06:49:56.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.452+0000 7f2f99a22740 1 Processor -- start 2024-09-19T06:49:56.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.452+0000 7f2f99a22740 1 -- start start 2024-09-19T06:49:56.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f99a22740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa780174a0 0x55fa780212f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f99a22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa78017dc0 0x55fa7803c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f99a22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa78024890 0x55fa7803c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f99a22740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901ab6c0 con 0x55fa780174a0 2024-09-19T06:49:56.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f99a22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901ad200 con 0x55fa78024890 2024-09-19T06:49:56.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f99a22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901abd90 con 0x55fa78017dc0 2024-09-19T06:49:56.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f953e5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa780174a0 0x55fa780212f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f953e5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa780174a0 0x55fa780212f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43808/0 (socket says 172.21.15.103:43808) 2024-09-19T06:49:56.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f953e5640 1 -- 172.21.15.103:0/2434589757 learned_addr learned my addr 172.21.15.103:0/2434589757 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:56.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f94be4640 1 --2- 172.21.15.103:0/2434589757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa78017dc0 0x55fa7803c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f953e5640 1 -- 172.21.15.103:0/2434589757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa78017dc0 msgr2=0x55fa7803c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:56.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f953e5640 1 --2- 172.21.15.103:0/2434589757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa78017dc0 0x55fa7803c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f95be6640 1 --2- 172.21.15.103:0/2434589757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa78024890 0x55fa7803c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f953e5640 1 -- 172.21.15.103:0/2434589757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa78024890 msgr2=0x55fa7803c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:56.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f953e5640 1 --2- 172.21.15.103:0/2434589757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa78024890 0x55fa7803c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f953e5640 1 -- 172.21.15.103:0/2434589757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa7801d8f0 con 0x55fa780174a0 2024-09-19T06:49:56.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.453+0000 7f2f95be6640 1 --2- 172.21.15.103:0/2434589757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa78024890 0x55fa7803c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:56.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.454+0000 7f2f953e5640 1 --2- 172.21.15.103:0/2434589757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa780174a0 0x55fa780212f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c02a940 tx=0x7f2f8c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b0ab8718db13a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:56.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.454+0000 7f2f7affd640 1 -- 172.21.15.103:0/2434589757 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c035070 con 0x55fa780174a0 2024-09-19T06:49:56.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.454+0000 7f2f7affd640 1 -- 172.21.15.103:0/2434589757 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f8c048070 con 0x55fa780174a0 2024-09-19T06:49:56.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.454+0000 7f2f7affd640 1 -- 172.21.15.103:0/2434589757 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c0046c0 con 0x55fa780174a0 2024-09-19T06:49:56.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.455+0000 7f2f99a22740 1 -- 172.21.15.103:0/2434589757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa780174a0 msgr2=0x55fa780212f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.455+0000 7f2f99a22740 1 --2- 172.21.15.103:0/2434589757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa780174a0 0x55fa780212f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c02a940 tx=0x7f2f8c005d50 comp rx=0 tx=0).stop 2024-09-19T06:49:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.455+0000 7f2f99a22740 1 -- 172.21.15.103:0/2434589757 shutdown_connections 2024-09-19T06:49:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.455+0000 7f2f99a22740 1 --2- 172.21.15.103:0/2434589757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa78024890 0x55fa7803c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.455+0000 7f2f99a22740 1 --2- 172.21.15.103:0/2434589757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa78017dc0 0x55fa7803c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.455+0000 7f2f99a22740 1 --2- 172.21.15.103:0/2434589757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa780174a0 0x55fa780212f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.455+0000 7f2f99a22740 1 -- 172.21.15.103:0/2434589757 >> 172.21.15.103:0/2434589757 conn(0x55fa78013a30 msgr2=0x55fa78020be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.455+0000 7f2f99a22740 1 -- 172.21.15.103:0/2434589757 shutdown_connections 2024-09-19T06:49:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.455+0000 7f2f99a22740 1 -- 172.21.15.103:0/2434589757 wait complete. 2024-09-19T06:49:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.455+0000 7f2f99a22740 1 -- start start 2024-09-19T06:49:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.455+0000 7f2f99a22740 1 Processor -- start 2024-09-19T06:49:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f99a22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa780174a0 0x55fa7802d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f99a22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa78017dc0 0x55fa7801c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f99a22740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78024890 0x55fa7801c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f99a22740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901acb30 con 0x55fa78024890 2024-09-19T06:49:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f99a22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901ac460 con 0x55fa78017dc0 2024-09-19T06:49:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f99a22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901a9680 con 0x55fa780174a0 2024-09-19T06:49:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f99a22740 10 client.0 _ll_register_callbacks cb 0x7f2f80014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f95be6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78024890 0x55fa7801c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f95be6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78024890 0x55fa7801c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43822/0 (socket says 172.21.15.103:43822) 2024-09-19T06:49:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f95be6640 1 -- 172.21.15.103:0/3138494527 learned_addr learned my addr 172.21.15.103:0/3138494527 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f94be4640 1 --2- 172.21.15.103:0/3138494527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa78017dc0 0x55fa7801c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f953e5640 1 --2- 172.21.15.103:0/3138494527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa780174a0 0x55fa7802d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f95be6640 1 -- 172.21.15.103:0/3138494527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa780174a0 msgr2=0x55fa7802d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f95be6640 1 --2- 172.21.15.103:0/3138494527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa780174a0 0x55fa7802d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f95be6640 1 -- 172.21.15.103:0/3138494527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa78017dc0 msgr2=0x55fa7801c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f95be6640 1 --2- 172.21.15.103:0/3138494527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa78017dc0 0x55fa7801c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.456+0000 7f2f95be6640 1 -- 172.21.15.103:0/3138494527 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa7809e0a0 con 0x55fa78024890 2024-09-19T06:49:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.457+0000 7f2f95be6640 1 --2- 172.21.15.103:0/3138494527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78024890 0x55fa7801c9d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2f8809bfa0 tx=0x7f2f8809cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15f2b9258d100eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.457+0000 7f2f78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.457+0000 7f2f78ff9640 1 -- 172.21.15.103:0/3138494527 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8801f040 con 0x55fa78024890 2024-09-19T06:49:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.457+0000 7f2f78ff9640 1 -- 172.21.15.103:0/3138494527 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f2f880b5070 con 0x55fa78024890 2024-09-19T06:49:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.457+0000 7f2f99a22740 1 -- 172.21.15.103:0/3138494527 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f8c048070 con 0x55fa78024890 2024-09-19T06:49:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.457+0000 7f2f78ff9640 1 -- 172.21.15.103:0/3138494527 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2f880d0040 con 0x55fa78024890 2024-09-19T06:49:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.457+0000 7f2f99a22740 10 client.15600 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.457+0000 7f2f99a22740 1 -- 172.21.15.103:0/3138494527 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa7809f1b0 con 0x55fa78024890 2024-09-19T06:49:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.457+0000 7f2f78ff9640 1 -- 172.21.15.103:0/3138494527 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f8c048070 con 0x55fa78024890 2024-09-19T06:49:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.457+0000 7f2f78ff9640 1 -- 172.21.15.103:0/3138494527 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2f8800f040 con 0x55fa78024890 2024-09-19T06:49:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.457+0000 7f2f99a22740 10 client.15600 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.457+0000 7f2f99a22740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.457+0000 7f2f99a22740 20 client.15600 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.457+0000 7f2f99a22740 1 --2- 172.21.15.103:0/3138494527 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fa7809f4e0 0x55fa780a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.457+0000 7f2f99a22740 4 client.15600 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:49:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.457+0000 7f2f99a22740 1 -- 172.21.15.103:0/3138494527 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fa77f84d60 con 0x55fa7809f4e0 2024-09-19T06:49:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.458+0000 7f2f953e5640 1 --2- 172.21.15.103:0/3138494527 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fa7809f4e0 0x55fa780a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.458+0000 7f2f78ff9640 10 client.15600 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.458+0000 7f2f953e5640 1 --2- 172.21.15.103:0/3138494527 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fa7809f4e0 0x55fa780a18d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=650bbec4b8056231 server_cookie=7d1877042dfb7d5d in_seq=0 out_seq=0 2024-09-19T06:49:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.458+0000 7f2f78ff9640 1 -- 172.21.15.103:0/3138494527 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fa77f84d60 con 0x55fa7809f4e0 2024-09-19T06:49:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.458+0000 7f2f78ff9640 10 client.15600 handle_command_reply: tid=0 2024-09-19T06:49:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.458+0000 7f2f99a22740 1 client.15600 shutdown 2024-09-19T06:49:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.458+0000 7f2f99a22740 1 -- 172.21.15.103:0/3138494527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78024890 msgr2=0x55fa7801c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.458+0000 7f2f99a22740 1 --2- 172.21.15.103:0/3138494527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78024890 0x55fa7801c9d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2f8809bfa0 tx=0x7f2f8809cb30 comp rx=0 tx=0).stop 2024-09-19T06:49:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.458+0000 7f2f99a22740 1 -- 172.21.15.103:0/3138494527 shutdown_connections 2024-09-19T06:49:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.458+0000 7f2f99a22740 1 --2- 172.21.15.103:0/3138494527 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fa7809f4e0 0x55fa780a18d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.459+0000 7f2f99a22740 1 --2- 172.21.15.103:0/3138494527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa78024890 0x55fa7801c9d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.459+0000 7f2f99a22740 1 --2- 172.21.15.103:0/3138494527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa78017dc0 0x55fa7801c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.459+0000 7f2f99a22740 1 --2- 172.21.15.103:0/3138494527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa780174a0 0x55fa7802d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.459+0000 7f2f99a22740 1 -- 172.21.15.103:0/3138494527 >> 172.21.15.103:0/3138494527 conn(0x55fa78013a30 msgr2=0x55fa78021400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:56.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.459+0000 7f2f78ff9640 0 client.15600 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:49:56.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.459+0000 7f2f99a22740 1 -- 172.21.15.103:0/3138494527 shutdown_connections 2024-09-19T06:49:56.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.459+0000 7f2f99a22740 1 -- 172.21.15.103:0/3138494527 wait complete. 2024-09-19T06:49:56.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.459+0000 7f2f99a22740 20 client.15600 trim_cache size 0 max 16384 2024-09-19T06:49:56.465 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:49:56.465 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T06:49:56.465 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T06:49:56.465 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:49:56.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.460+0000 7f2f97649640 1 -- 172.21.15.103:0/286508964 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2f7c0a34d0 msgr2=0x7f2f7c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:56.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.460+0000 7f2f97649640 1 --2- 172.21.15.103:0/286508964 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2f7c0a34d0 0x7f2f7c0a5990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2f900fed90 tx=0x7f2f8000a610 comp rx=0 tx=0).stop 2024-09-19T06:49:56.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.460+0000 7f2f97649640 1 -- 172.21.15.103:0/286508964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f90106a70 msgr2=0x7f2f9019bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:56.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.460+0000 7f2f97649640 1 --2- 172.21.15.103:0/286508964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f90106a70 0x7f2f9019bbe0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800eaa0 tx=0x7f2f8800ef60 comp rx=0 tx=0).stop 2024-09-19T06:49:56.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.460+0000 7f2f97649640 1 -- 172.21.15.103:0/286508964 shutdown_connections 2024-09-19T06:49:56.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.460+0000 7f2f97649640 1 --2- 172.21.15.103:0/286508964 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2f7c0a34d0 0x7f2f7c0a5990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.460+0000 7f2f97649640 1 --2- 172.21.15.103:0/286508964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f90106a70 0x7f2f9019bbe0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.460+0000 7f2f97649640 1 --2- 172.21.15.103:0/286508964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f90101410 0x7f2f90197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.460+0000 7f2f97649640 1 --2- 172.21.15.103:0/286508964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f90100a70 0x7f2f90197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.460+0000 7f2f97649640 1 -- 172.21.15.103:0/286508964 >> 172.21.15.103:0/286508964 conn(0x7f2f900fc790 msgr2=0x7f2f900fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:56.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.460+0000 7f2f97649640 1 -- 172.21.15.103:0/286508964 shutdown_connections 2024-09-19T06:49:56.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:56.460+0000 7f2f97649640 1 -- 172.21.15.103:0/286508964 wait complete. 2024-09-19T06:49:57.155 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:49:57.156 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:49:57.410 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:57.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.886+0000 7f7d16e3c640 1 -- 172.21.15.103:0/3787866059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d10102570 msgr2=0x7f7d101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:57.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.886+0000 7f7d16e3c640 1 --2- 172.21.15.103:0/3787866059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d10102570 0x7f7d101029b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009900 tx=0x7f7cfc02a480 comp rx=0 tx=0).stop 2024-09-19T06:49:57.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.886+0000 7f7d16e3c640 1 -- 172.21.15.103:0/3787866059 shutdown_connections 2024-09-19T06:49:57.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.886+0000 7f7d16e3c640 1 --2- 172.21.15.103:0/3787866059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d10102ef0 0x7f7d1010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.886+0000 7f7d16e3c640 1 --2- 172.21.15.103:0/3787866059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d10102570 0x7f7d101029b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.886+0000 7f7d16e3c640 1 --2- 172.21.15.103:0/3787866059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d10100e20 0x7f7d10101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.886+0000 7f7d16e3c640 1 -- 172.21.15.103:0/3787866059 >> 172.21.15.103:0/3787866059 conn(0x7f7d100fc7d0 msgr2=0x7f7d100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:57.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.886+0000 7f7d16e3c640 1 -- 172.21.15.103:0/3787866059 shutdown_connections 2024-09-19T06:49:57.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.886+0000 7f7d16e3c640 1 -- 172.21.15.103:0/3787866059 wait complete. 2024-09-19T06:49:57.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.886+0000 7f7d16e3c640 1 Processor -- start 2024-09-19T06:49:57.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.886+0000 7f7d16e3c640 1 -- start start 2024-09-19T06:49:57.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.886+0000 7f7d16e3c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d10100e20 0x7f7d1019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d16e3c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d10102570 0x7f7d1019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d16e3c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d10102ef0 0x7f7d1019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d16e3c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d10110f80 con 0x7f7d10102570 2024-09-19T06:49:57.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d16e3c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d10110e00 con 0x7f7d10102ef0 2024-09-19T06:49:57.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d16e3c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d10111100 con 0x7f7d10100e20 2024-09-19T06:49:57.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d0ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d10102570 0x7f7d1019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d0ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d10102570 0x7f7d1019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47720/0 (socket says 172.21.15.103:47720) 2024-09-19T06:49:57.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d0ffff640 1 -- 172.21.15.103:0/1402496855 learned_addr learned my addr 172.21.15.103:0/1402496855 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:57.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d14bd8640 1 --2- 172.21.15.103:0/1402496855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d10100e20 0x7f7d1019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d153d9640 1 --2- 172.21.15.103:0/1402496855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d10102ef0 0x7f7d1019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d0ffff640 1 -- 172.21.15.103:0/1402496855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d10100e20 msgr2=0x7f7d1019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:57.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d0ffff640 1 --2- 172.21.15.103:0/1402496855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d10100e20 0x7f7d1019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d0ffff640 1 -- 172.21.15.103:0/1402496855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d10102ef0 msgr2=0x7f7d1019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:57.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d0ffff640 1 --2- 172.21.15.103:0/1402496855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d10102ef0 0x7f7d1019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d0ffff640 1 -- 172.21.15.103:0/1402496855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1019f5b0 con 0x7f7d10102570 2024-09-19T06:49:57.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d0ffff640 1 --2- 172.21.15.103:0/1402496855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d10102570 0x7f7d1019ab40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc006010 tx=0x7f7cfc003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:57.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d0dffb640 1 -- 172.21.15.103:0/1402496855 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc03f070 con 0x7f7d10102570 2024-09-19T06:49:57.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d16e3c640 1 -- 172.21.15.103:0/1402496855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1019f840 con 0x7f7d10102570 2024-09-19T06:49:57.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d0dffb640 1 -- 172.21.15.103:0/1402496855 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cfc031e30 con 0x7f7d10102570 2024-09-19T06:49:57.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.887+0000 7f7d0dffb640 1 -- 172.21.15.103:0/1402496855 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc035030 con 0x7f7d10102570 2024-09-19T06:49:57.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.888+0000 7f7d16e3c640 1 -- 172.21.15.103:0/1402496855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d101a7180 con 0x7f7d10102570 2024-09-19T06:49:57.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.889+0000 7f7d16e3c640 1 -- 172.21.15.103:0/1402496855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d101021b0 con 0x7f7d10102570 2024-09-19T06:49:57.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.890+0000 7f7d0dffb640 1 -- 172.21.15.103:0/1402496855 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f7cfc042070 con 0x7f7d10102570 2024-09-19T06:49:57.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.891+0000 7f7d0dffb640 1 --2- 172.21.15.103:0/1402496855 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d000a34d0 0x7f7d000a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.891+0000 7f7d14bd8640 1 --2- 172.21.15.103:0/1402496855 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d000a34d0 0x7f7d000a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.891+0000 7f7d0dffb640 1 -- 172.21.15.103:0/1402496855 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f7cfc0f07e0 con 0x7f7d10102570 2024-09-19T06:49:57.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.891+0000 7f7d14bd8640 1 --2- 172.21.15.103:0/1402496855 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d000a34d0 0x7f7d000a5990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400abe0 tx=0x7f7d0400a2d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:57.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:57.895+0000 7f7d0dffb640 1 -- 172.21.15.103:0/1402496855 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7cfc0bd050 con 0x7f7d10102570 2024-09-19T06:49:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:58.131+0000 7f7d16e3c640 1 -- 172.21.15.103:0/1402496855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d101a7350 con 0x7f7d10102570 2024-09-19T06:49:58.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:58 smithi103 ceph-mon[31158]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:49:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:58.131+0000 7f7d0dffb640 1 -- 172.21.15.103:0/1402496855 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f7cfc0bc950 con 0x7f7d10102570 2024-09-19T06:49:58.133 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:49:58.133 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:49:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:49:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:58.134+0000 7f7d16e3c640 1 -- 172.21.15.103:0/1402496855 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d000a34d0 msgr2=0x7f7d000a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:58.134+0000 7f7d16e3c640 1 --2- 172.21.15.103:0/1402496855 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d000a34d0 0x7f7d000a5990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400abe0 tx=0x7f7d0400a2d0 comp rx=0 tx=0).stop 2024-09-19T06:49:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:58.134+0000 7f7d16e3c640 1 -- 172.21.15.103:0/1402496855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d10102570 msgr2=0x7f7d1019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:58.134+0000 7f7d16e3c640 1 --2- 172.21.15.103:0/1402496855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d10102570 0x7f7d1019ab40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc006010 tx=0x7f7cfc003c80 comp rx=0 tx=0).stop 2024-09-19T06:49:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:58.134+0000 7f7d16e3c640 1 -- 172.21.15.103:0/1402496855 shutdown_connections 2024-09-19T06:49:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:58.134+0000 7f7d16e3c640 1 --2- 172.21.15.103:0/1402496855 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d000a34d0 0x7f7d000a5990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:58.134+0000 7f7d16e3c640 1 --2- 172.21.15.103:0/1402496855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d10102ef0 0x7f7d1019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:58.134+0000 7f7d16e3c640 1 --2- 172.21.15.103:0/1402496855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d10102570 0x7f7d1019ab40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:58.134+0000 7f7d16e3c640 1 --2- 172.21.15.103:0/1402496855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d10100e20 0x7f7d1019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:58.134+0000 7f7d16e3c640 1 -- 172.21.15.103:0/1402496855 >> 172.21.15.103:0/1402496855 conn(0x7f7d100fc7d0 msgr2=0x7f7d1006a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:58.134+0000 7f7d16e3c640 1 -- 172.21.15.103:0/1402496855 shutdown_connections 2024-09-19T06:49:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:58.134+0000 7f7d16e3c640 1 -- 172.21.15.103:0/1402496855 wait complete. 2024-09-19T06:49:58.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:58 smithi174 ceph-mon[35637]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:49:58.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:58 smithi134 ceph-mon[35356]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:49:58.791 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:49:59.047 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:59.299 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:49:59 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1402496855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:59.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:49:59 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1402496855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:59.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:49:59 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1402496855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:59.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.542+0000 7f5904a5a640 1 -- 172.21.15.103:0/2354163330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5900103650 msgr2=0x7f590010bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.542+0000 7f5904a5a640 1 --2- 172.21.15.103:0/2354163330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5900103650 0x7f590010bb40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009900 tx=0x7f58f402a480 comp rx=0 tx=0).stop 2024-09-19T06:49:59.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.542+0000 7f5904a5a640 1 -- 172.21.15.103:0/2354163330 shutdown_connections 2024-09-19T06:49:59.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.542+0000 7f5904a5a640 1 --2- 172.21.15.103:0/2354163330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f590010c160 0x7f590010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.542+0000 7f5904a5a640 1 --2- 172.21.15.103:0/2354163330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5900103650 0x7f590010bb40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.542+0000 7f5904a5a640 1 --2- 172.21.15.103:0/2354163330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5900102d30 0x7f5900103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.542+0000 7f5904a5a640 1 -- 172.21.15.103:0/2354163330 >> 172.21.15.103:0/2354163330 conn(0x7f59000fc790 msgr2=0x7f59000febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:59.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.542+0000 7f5904a5a640 1 -- 172.21.15.103:0/2354163330 shutdown_connections 2024-09-19T06:49:59.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.542+0000 7f5904a5a640 1 -- 172.21.15.103:0/2354163330 wait complete. 2024-09-19T06:49:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.543+0000 7f5904a5a640 1 Processor -- start 2024-09-19T06:49:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.543+0000 7f5904a5a640 1 -- start start 2024-09-19T06:49:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.543+0000 7f5904a5a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5900102d30 0x7f59001984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.543+0000 7f5904a5a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5900103650 0x7f59001989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.543+0000 7f5904a5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f590010c160 0x7f590019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.543+0000 7f5904a5a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5900111fa0 con 0x7f590010c160 2024-09-19T06:49:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.543+0000 7f5904a5a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5900111e20 con 0x7f5900103650 2024-09-19T06:49:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.543+0000 7f5904a5a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5900112120 con 0x7f5900102d30 2024-09-19T06:49:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.544+0000 7f58fed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f590010c160 0x7f590019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.544+0000 7f58fed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f590010c160 0x7f590019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47736/0 (socket says 172.21.15.103:47736) 2024-09-19T06:49:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.544+0000 7f58fed9d640 1 -- 172.21.15.103:0/4194404261 learned_addr learned my addr 172.21.15.103:0/4194404261 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.544+0000 7f58fdd9b640 1 --2- 172.21.15.103:0/4194404261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5900103650 0x7f59001989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.544+0000 7f58fe59c640 1 --2- 172.21.15.103:0/4194404261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5900102d30 0x7f59001984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.544+0000 7f58fed9d640 1 -- 172.21.15.103:0/4194404261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5900102d30 msgr2=0x7f59001984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.544+0000 7f58fed9d640 1 --2- 172.21.15.103:0/4194404261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5900102d30 0x7f59001984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.544+0000 7f58fed9d640 1 -- 172.21.15.103:0/4194404261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5900103650 msgr2=0x7f59001989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.544+0000 7f58fed9d640 1 --2- 172.21.15.103:0/4194404261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5900103650 0x7f59001989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.544+0000 7f58fed9d640 1 -- 172.21.15.103:0/4194404261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590019d450 con 0x7f590010c160 2024-09-19T06:49:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.544+0000 7f58fed9d640 1 --2- 172.21.15.103:0/4194404261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f590010c160 0x7f590019cd70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f58f800b970 tx=0x7f58f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.544+0000 7f58ef7fe640 1 -- 172.21.15.103:0/4194404261 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8019070 con 0x7f590010c160 2024-09-19T06:49:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.544+0000 7f58ef7fe640 1 -- 172.21.15.103:0/4194404261 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58f800cb40 con 0x7f590010c160 2024-09-19T06:49:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.544+0000 7f58ef7fe640 1 -- 172.21.15.103:0/4194404261 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8007650 con 0x7f590010c160 2024-09-19T06:49:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.544+0000 7f5904a5a640 1 -- 172.21.15.103:0/4194404261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590019d740 con 0x7f590010c160 2024-09-19T06:49:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.544+0000 7f5904a5a640 1 -- 172.21.15.103:0/4194404261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59001a5020 con 0x7f590010c160 2024-09-19T06:49:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.547+0000 7f5906e33740 1 Processor -- start 2024-09-19T06:49:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.547+0000 7f58ef7fe640 1 -- 172.21.15.103:0/4194404261 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138583+0+0 (secure 0 0 0) 0x7f58f801c070 con 0x7f590010c160 2024-09-19T06:49:59.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.547+0000 7f5906e33740 1 -- start start 2024-09-19T06:49:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f5906e33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e616904b40 0x55e616903090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f5906e33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6169035d0 0x55e616903a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f5906e33740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e616912780 0x55e616914b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f5906e33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e6168692f0 con 0x55e616904b40 2024-09-19T06:49:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f5906e33740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e616810cf0 con 0x55e616912780 2024-09-19T06:49:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f5906e33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e6168652b0 con 0x55e6169035d0 2024-09-19T06:49:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f58fdd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e616904b40 0x55e616903090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f58fe59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6169035d0 0x55e616903a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f58fdd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e616904b40 0x55e616903090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47752/0 (socket says 172.21.15.103:47752) 2024-09-19T06:49:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f58fdd9b640 1 -- 172.21.15.103:0/677384657 learned_addr learned my addr 172.21.15.103:0/677384657 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f58fdd9b640 1 --2- 172.21.15.103:0/677384657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e616912780 0x55e616914b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f58fdd9b640 1 -- 172.21.15.103:0/677384657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6169035d0 msgr2=0x55e616903a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:59.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f58fdd9b640 1 --2- 172.21.15.103:0/677384657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6169035d0 0x55e616903a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f58fdd9b640 1 -- 172.21.15.103:0/677384657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e616912780 msgr2=0x55e616914b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f58fdd9b640 1 --2- 172.21.15.103:0/677384657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e616912780 0x55e616914b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f58fdd9b640 1 -- 172.21.15.103:0/677384657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6167f53a0 con 0x55e616904b40 2024-09-19T06:49:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.548+0000 7f58ef7fe640 1 --2- 172.21.15.103:0/4194404261 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f58e80a3550 0x7f58e80a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.549+0000 7f58ef7fe640 1 -- 172.21.15.103:0/4194404261 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f58f80cf3e0 con 0x7f590010c160 2024-09-19T06:49:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.549+0000 7f58fdd9b640 1 --2- 172.21.15.103:0/677384657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e616904b40 0x55e616903090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f58f402a9a0 tx=0x7f58f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cad3bf04ff4875 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.549+0000 7f58fe59c640 1 --2- 172.21.15.103:0/4194404261 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f58e80a3550 0x7f58e80a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.549+0000 7f58ecff9640 1 -- 172.21.15.103:0/677384657 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4044030 con 0x55e616904b40 2024-09-19T06:49:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.549+0000 7f58ecff9640 1 -- 172.21.15.103:0/677384657 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58f4031a00 con 0x55e616904b40 2024-09-19T06:49:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.549+0000 7f58ecff9640 1 -- 172.21.15.103:0/677384657 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4035040 con 0x55e616904b40 2024-09-19T06:49:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.550+0000 7f5906e33740 1 -- 172.21.15.103:0/677384657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e616904b40 msgr2=0x55e616903090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.550+0000 7f58fe59c640 1 --2- 172.21.15.103:0/4194404261 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f58e80a3550 0x7f58e80a5a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55e616900ac0 tx=0x7f58f000b390 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.550+0000 7f5906e33740 1 --2- 172.21.15.103:0/677384657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e616904b40 0x55e616903090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f58f402a9a0 tx=0x7f58f4000bc0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.550+0000 7f5906e33740 1 -- 172.21.15.103:0/677384657 shutdown_connections 2024-09-19T06:49:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.550+0000 7f5906e33740 1 --2- 172.21.15.103:0/677384657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e616912780 0x55e616914b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.550+0000 7f5906e33740 1 --2- 172.21.15.103:0/677384657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6169035d0 0x55e616903a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.550+0000 7f5906e33740 1 --2- 172.21.15.103:0/677384657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e616904b40 0x55e616903090 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.550+0000 7f5906e33740 1 -- 172.21.15.103:0/677384657 >> 172.21.15.103:0/677384657 conn(0x55e6168cc310 msgr2=0x55e6168ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.550+0000 7f5906e33740 1 -- 172.21.15.103:0/677384657 shutdown_connections 2024-09-19T06:49:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.550+0000 7f5906e33740 1 -- 172.21.15.103:0/677384657 wait complete. 2024-09-19T06:49:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.550+0000 7f5906e33740 1 -- start start 2024-09-19T06:49:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.550+0000 7f5906e33740 1 Processor -- start 2024-09-19T06:49:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f5906e33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6169035d0 0x55e616912200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f5906e33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e616904b40 0x55e61690f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f5906e33740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e616912780 0x55e61690fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f5906e33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e616915420 con 0x55e6169035d0 2024-09-19T06:49:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f5906e33740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e616824060 con 0x55e616912780 2024-09-19T06:49:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f5906e33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e6169155a0 con 0x55e616904b40 2024-09-19T06:49:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f5906e33740 10 client.0 _ll_register_callbacks cb 0x55e6165778c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f58fe59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6169035d0 0x55e616912200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f58fe59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6169035d0 0x55e616912200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47756/0 (socket says 172.21.15.103:47756) 2024-09-19T06:49:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f58fe59c640 1 -- 172.21.15.103:0/3679111539 learned_addr learned my addr 172.21.15.103:0/3679111539 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f58fdd9b640 1 --2- 172.21.15.103:0/3679111539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e616904b40 0x55e61690f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f58fed9d640 1 --2- 172.21.15.103:0/3679111539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e616912780 0x55e61690fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f58fe59c640 1 -- 172.21.15.103:0/3679111539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e616904b40 msgr2=0x55e61690f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f58fe59c640 1 --2- 172.21.15.103:0/3679111539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e616904b40 0x55e61690f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f58fe59c640 1 -- 172.21.15.103:0/3679111539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e616912780 msgr2=0x55e61690fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f58fe59c640 1 --2- 172.21.15.103:0/3679111539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e616912780 0x55e61690fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f58fe59c640 1 -- 172.21.15.103:0/3679111539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6165da3c0 con 0x55e6169035d0 2024-09-19T06:49:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f58fe59c640 1 --2- 172.21.15.103:0/3679111539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6169035d0 0x55e616912200 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f58f0004130 tx=0x7f58f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6840a8e5391fb460 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:59.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f58e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f58e5ffb640 1 -- 172.21.15.103:0/3679111539 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f002c060 con 0x55e6169035d0 2024-09-19T06:49:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f58e5ffb640 1 -- 172.21.15.103:0/3679111539 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f58f0017030 con 0x55e6169035d0 2024-09-19T06:49:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.551+0000 7f5906e33740 1 -- 172.21.15.103:0/3679111539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58f4031a00 con 0x55e6169035d0 2024-09-19T06:49:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.552+0000 7f58e5ffb640 1 -- 172.21.15.103:0/3679111539 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f58f0027410 con 0x55e6169035d0 2024-09-19T06:49:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.552+0000 7f5906e33740 10 client.15618 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.552+0000 7f5906e33740 1 -- 172.21.15.103:0/3679111539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e616920fb0 con 0x55e6169035d0 2024-09-19T06:49:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.552+0000 7f58e5ffb640 1 -- 172.21.15.103:0/3679111539 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58f4031a00 con 0x55e6169035d0 2024-09-19T06:49:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.552+0000 7f58e5ffb640 1 -- 172.21.15.103:0/3679111539 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f58f0011710 con 0x55e6169035d0 2024-09-19T06:49:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.552+0000 7f5906e33740 10 client.15618 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.552+0000 7f5906e33740 10 client.15618 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:49:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.552+0000 7f5906e33740 20 client.15618 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.552+0000 7f5906e33740 1 --2- 172.21.15.103:0/3679111539 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55e6169224c0 0x55e616924920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.552+0000 7f5906e33740 4 client.15618 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.552+0000 7f5906e33740 1 -- 172.21.15.103:0/3679111539 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e616876860 con 0x55e6169224c0 2024-09-19T06:49:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.552+0000 7f58fdd9b640 1 --2- 172.21.15.103:0/3679111539 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55e6169224c0 0x55e616924920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.553+0000 7f58e5ffb640 10 client.15618 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:49:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.553+0000 7f58fdd9b640 1 --2- 172.21.15.103:0/3679111539 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55e6169224c0 0x55e616924920 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a7990a76c29a62db server_cookie=4db680e8dab9d4aa in_seq=0 out_seq=0 2024-09-19T06:49:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.554+0000 7f58e5ffb640 1 -- 172.21.15.103:0/3679111539 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e616876860 con 0x55e6169224c0 2024-09-19T06:49:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.554+0000 7f58e5ffb640 10 client.15618 handle_command_reply: tid=0 2024-09-19T06:49:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.554+0000 7f5906e33740 1 client.15618 shutdown 2024-09-19T06:49:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.554+0000 7f5906e33740 1 -- 172.21.15.103:0/3679111539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6169035d0 msgr2=0x55e616912200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.554+0000 7f5906e33740 1 --2- 172.21.15.103:0/3679111539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6169035d0 0x55e616912200 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f58f0004130 tx=0x7f58f00046c0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.554+0000 7f5906e33740 1 -- 172.21.15.103:0/3679111539 shutdown_connections 2024-09-19T06:49:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.554+0000 7f5906e33740 1 --2- 172.21.15.103:0/3679111539 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55e6169224c0 0x55e616924920 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.554+0000 7f58e5ffb640 0 client.15618 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:49:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.554+0000 7f5906e33740 1 --2- 172.21.15.103:0/3679111539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e616912780 0x55e61690fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.554+0000 7f5906e33740 1 --2- 172.21.15.103:0/3679111539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e616904b40 0x55e61690f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.554+0000 7f5906e33740 1 --2- 172.21.15.103:0/3679111539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6169035d0 0x55e616912200 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.554+0000 7f5906e33740 1 -- 172.21.15.103:0/3679111539 >> 172.21.15.103:0/3679111539 conn(0x55e6168cc310 msgr2=0x55e616905960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:59.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.554+0000 7f5906e33740 1 -- 172.21.15.103:0/3679111539 shutdown_connections 2024-09-19T06:49:59.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.554+0000 7f5906e33740 1 -- 172.21.15.103:0/3679111539 wait complete. 2024-09-19T06:49:59.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.554+0000 7f5906e33740 20 client.15618 trim_cache size 0 max 16384 2024-09-19T06:49:59.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.573+0000 7f5906e33740 1 Processor -- start 2024-09-19T06:49:59.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.573+0000 7f5906e33740 1 -- start start 2024-09-19T06:49:59.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.573+0000 7f5906e33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6169035d0 0x55e61691e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.573+0000 7f5906e33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e616904b40 0x55e61690e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.573+0000 7f5906e33740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e616912780 0x55e61692a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.573+0000 7f5906e33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59001ab650 con 0x55e6169035d0 2024-09-19T06:49:59.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.573+0000 7f5906e33740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59001ad190 con 0x55e616912780 2024-09-19T06:49:59.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.573+0000 7f5906e33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59001abd20 con 0x55e616904b40 2024-09-19T06:49:59.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.574+0000 7f58fe59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6169035d0 0x55e61691e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.574+0000 7f58fe59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6169035d0 0x55e61691e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47760/0 (socket says 172.21.15.103:47760) 2024-09-19T06:49:59.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.574+0000 7f58fe59c640 1 -- 172.21.15.103:0/3855030081 learned_addr learned my addr 172.21.15.103:0/3855030081 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:59.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.574+0000 7f58fdd9b640 1 --2- 172.21.15.103:0/3855030081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e616904b40 0x55e61690e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.574+0000 7f58fed9d640 1 --2- 172.21.15.103:0/3855030081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e616912780 0x55e61692a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.574+0000 7f58fe59c640 1 -- 172.21.15.103:0/3855030081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e616904b40 msgr2=0x55e61690e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:59.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.574+0000 7f58fe59c640 1 --2- 172.21.15.103:0/3855030081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e616904b40 0x55e61690e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.574+0000 7f58fe59c640 1 -- 172.21.15.103:0/3855030081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e616912780 msgr2=0x55e61692a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:59.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.574+0000 7f58fe59c640 1 --2- 172.21.15.103:0/3855030081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e616912780 0x55e61692a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.574+0000 7f58fe59c640 1 -- 172.21.15.103:0/3855030081 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61691ec00 con 0x55e6169035d0 2024-09-19T06:49:59.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.574+0000 7f58fed9d640 1 --2- 172.21.15.103:0/3855030081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e616912780 0x55e61692a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:59.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.574+0000 7f58fe59c640 1 --2- 172.21.15.103:0/3855030081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6169035d0 0x55e61691e6c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f58f00273e0 tx=0x7f58f00043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f3367bab3095af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:59.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.574+0000 7f58e7fff640 1 -- 172.21.15.103:0/3855030081 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0011440 con 0x55e6169035d0 2024-09-19T06:49:59.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.574+0000 7f58e7fff640 1 -- 172.21.15.103:0/3855030081 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58f00115e0 con 0x55e6169035d0 2024-09-19T06:49:59.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.574+0000 7f58e7fff640 1 -- 172.21.15.103:0/3855030081 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f001c490 con 0x55e6169035d0 2024-09-19T06:49:59.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.575+0000 7f5906e33740 1 -- 172.21.15.103:0/3855030081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6169035d0 msgr2=0x55e61691e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.575+0000 7f5906e33740 1 --2- 172.21.15.103:0/3855030081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6169035d0 0x55e61691e6c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f58f00273e0 tx=0x7f58f00043f0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.575+0000 7f5906e33740 1 -- 172.21.15.103:0/3855030081 shutdown_connections 2024-09-19T06:49:59.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.575+0000 7f5906e33740 1 --2- 172.21.15.103:0/3855030081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e616912780 0x55e61692a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.575+0000 7f5906e33740 1 --2- 172.21.15.103:0/3855030081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e616904b40 0x55e61690e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.575+0000 7f5906e33740 1 --2- 172.21.15.103:0/3855030081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6169035d0 0x55e61691e6c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.575+0000 7f5906e33740 1 -- 172.21.15.103:0/3855030081 >> 172.21.15.103:0/3855030081 conn(0x55e6168cc310 msgr2=0x55e61690dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:59.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.575+0000 7f5906e33740 1 -- 172.21.15.103:0/3855030081 shutdown_connections 2024-09-19T06:49:59.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.575+0000 7f5906e33740 1 -- 172.21.15.103:0/3855030081 wait complete. 2024-09-19T06:49:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.575+0000 7f5906e33740 1 -- start start 2024-09-19T06:49:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f5906e33740 1 Processor -- start 2024-09-19T06:49:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f5906e33740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6169035d0 0x55e61691aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f5906e33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e616904b40 0x55e61691af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f5906e33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e616912780 0x55e616917800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f5906e33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59001acac0 con 0x55e616912780 2024-09-19T06:49:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f5906e33740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59001ac3f0 con 0x55e6169035d0 2024-09-19T06:49:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f5906e33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59001a9610 con 0x55e616904b40 2024-09-19T06:49:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f5906e33740 10 client.0 _ll_register_callbacks cb 0x7f58f0004340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f58fed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e616912780 0x55e616917800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f58fed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e616912780 0x55e616917800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47766/0 (socket says 172.21.15.103:47766) 2024-09-19T06:49:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f58fed9d640 1 -- 172.21.15.103:0/2444810467 learned_addr learned my addr 172.21.15.103:0/2444810467 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:49:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f58fe59c640 1 --2- 172.21.15.103:0/2444810467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6169035d0 0x55e61691aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f58fdd9b640 1 --2- 172.21.15.103:0/2444810467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e616904b40 0x55e61691af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f58fed9d640 1 -- 172.21.15.103:0/2444810467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e616904b40 msgr2=0x55e61691af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f58fed9d640 1 --2- 172.21.15.103:0/2444810467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e616904b40 0x55e61691af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f58fed9d640 1 -- 172.21.15.103:0/2444810467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6169035d0 msgr2=0x55e61691aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f58fed9d640 1 --2- 172.21.15.103:0/2444810467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6169035d0 0x55e61691aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f58fed9d640 1 -- 172.21.15.103:0/2444810467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e616917e40 con 0x55e616912780 2024-09-19T06:49:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f58fe59c640 1 --2- 172.21.15.103:0/2444810467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6169035d0 0x55e61691aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.576+0000 7f58fdd9b640 1 --2- 172.21.15.103:0/2444810467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e616904b40 0x55e61691af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.577+0000 7f58fed9d640 1 --2- 172.21.15.103:0/2444810467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e616912780 0x55e616917800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f58f8000f80 tx=0x7f58f809b060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a665fb424927b6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.577+0000 7f58e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:49:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.577+0000 7f58e5ffb640 1 -- 172.21.15.103:0/2444810467 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f80b3070 con 0x55e616912780 2024-09-19T06:49:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.577+0000 7f58e5ffb640 1 -- 172.21.15.103:0/2444810467 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f58f80a3d70 con 0x55e616912780 2024-09-19T06:49:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.577+0000 7f5906e33740 1 -- 172.21.15.103:0/2444810467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58f00115e0 con 0x55e616912780 2024-09-19T06:49:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.577+0000 7f58e5ffb640 1 -- 172.21.15.103:0/2444810467 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f58f80a4200 con 0x55e616912780 2024-09-19T06:49:59.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.577+0000 7f5906e33740 10 client.15624 fetch_fsmap learned FSMap version 27 2024-09-19T06:49:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.577+0000 7f5906e33740 1 -- 172.21.15.103:0/2444810467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e61698db70 con 0x55e616912780 2024-09-19T06:49:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.577+0000 7f58e5ffb640 1 -- 172.21.15.103:0/2444810467 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58f00115e0 con 0x55e616912780 2024-09-19T06:49:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.577+0000 7f58e5ffb640 1 -- 172.21.15.103:0/2444810467 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f58f80b42d0 con 0x55e616912780 2024-09-19T06:49:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.577+0000 7f5906e33740 10 client.15624 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:49:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.577+0000 7f5906e33740 10 client.15624 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:49:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.577+0000 7f5906e33740 20 client.15624 populate_metadata read hostname 'smithi103' 2024-09-19T06:49:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.577+0000 7f5906e33740 1 --2- 172.21.15.103:0/2444810467 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55e6169224c0 0x55e616990320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.577+0000 7f5906e33740 4 client.15624 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:49:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.577+0000 7f5906e33740 1 -- 172.21.15.103:0/2444810467 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e616870d70 con 0x55e6169224c0 2024-09-19T06:49:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.578+0000 7f58fe59c640 1 --2- 172.21.15.103:0/2444810467 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55e6169224c0 0x55e616990320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.578+0000 7f58fe59c640 1 --2- 172.21.15.103:0/2444810467 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55e6169224c0 0x55e616990320 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=460de5805c762574 server_cookie=1068fc0a77551d9 in_seq=0 out_seq=0 2024-09-19T06:49:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.578+0000 7f58e5ffb640 10 client.15624 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:49:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.578+0000 7f58e5ffb640 1 -- 172.21.15.103:0/2444810467 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e616870d70 con 0x55e6169224c0 2024-09-19T06:49:59.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.578+0000 7f58e5ffb640 10 client.15624 handle_command_reply: tid=0 2024-09-19T06:49:59.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.578+0000 7f5906e33740 1 client.15624 shutdown 2024-09-19T06:49:59.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.578+0000 7f5906e33740 1 -- 172.21.15.103:0/2444810467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e616912780 msgr2=0x55e616917800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.578+0000 7f5906e33740 1 --2- 172.21.15.103:0/2444810467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e616912780 0x55e616917800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f58f8000f80 tx=0x7f58f809b060 comp rx=0 tx=0).stop 2024-09-19T06:49:59.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.578+0000 7f5906e33740 1 -- 172.21.15.103:0/2444810467 shutdown_connections 2024-09-19T06:49:59.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.578+0000 7f5906e33740 1 --2- 172.21.15.103:0/2444810467 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55e6169224c0 0x55e616990320 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.578+0000 7f5906e33740 1 --2- 172.21.15.103:0/2444810467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e616912780 0x55e616917800 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.578+0000 7f5906e33740 1 --2- 172.21.15.103:0/2444810467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e616904b40 0x55e61691af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.578+0000 7f58e5ffb640 0 client.15624 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:49:59.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.578+0000 7f5906e33740 1 --2- 172.21.15.103:0/2444810467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6169035d0 0x55e61691aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.578+0000 7f5906e33740 1 -- 172.21.15.103:0/2444810467 >> 172.21.15.103:0/2444810467 conn(0x55e6168cc310 msgr2=0x55e616905930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:59.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.578+0000 7f5906e33740 1 -- 172.21.15.103:0/2444810467 shutdown_connections 2024-09-19T06:49:59.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.579+0000 7f5906e33740 1 -- 172.21.15.103:0/2444810467 wait complete. 2024-09-19T06:49:59.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.579+0000 7f5906e33740 20 client.15624 trim_cache size 0 max 16384 2024-09-19T06:49:59.585 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:49:59.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.579+0000 7f5904a5a640 1 -- 172.21.15.103:0/4194404261 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f58e80a3550 msgr2=0x7f58e80a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.579+0000 7f5904a5a640 1 --2- 172.21.15.103:0/4194404261 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f58e80a3550 0x7f58e80a5a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55e616900ac0 tx=0x7f58f000b390 comp rx=0 tx=0).stop 2024-09-19T06:49:59.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.579+0000 7f5904a5a640 1 -- 172.21.15.103:0/4194404261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f590010c160 msgr2=0x7f590019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.579+0000 7f5904a5a640 1 --2- 172.21.15.103:0/4194404261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f590010c160 0x7f590019cd70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f58f800b970 tx=0x7f58f800be30 comp rx=0 tx=0).stop 2024-09-19T06:49:59.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.579+0000 7f5904a5a640 1 -- 172.21.15.103:0/4194404261 shutdown_connections 2024-09-19T06:49:59.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.579+0000 7f5904a5a640 1 --2- 172.21.15.103:0/4194404261 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f58e80a3550 0x7f58e80a5a10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.579+0000 7f5904a5a640 1 --2- 172.21.15.103:0/4194404261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f590010c160 0x7f590019cd70 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.579+0000 7f5904a5a640 1 --2- 172.21.15.103:0/4194404261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5900103650 0x7f59001989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.579+0000 7f5904a5a640 1 --2- 172.21.15.103:0/4194404261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5900102d30 0x7f59001984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.579+0000 7f5904a5a640 1 -- 172.21.15.103:0/4194404261 >> 172.21.15.103:0/4194404261 conn(0x7f59000fc790 msgr2=0x7f59000fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:59.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.579+0000 7f5904a5a640 1 -- 172.21.15.103:0/4194404261 shutdown_connections 2024-09-19T06:49:59.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:49:59.579+0000 7f5904a5a640 1 -- 172.21.15.103:0/4194404261 wait complete. 2024-09-19T06:50:00.288 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:00 smithi103 ceph-mon[31158]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T06:50:00.288 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:00 smithi103 ceph-mon[31158]: overall HEALTH_OK 2024-09-19T06:50:00.318 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:50:00.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:00 smithi174 ceph-mon[35637]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T06:50:00.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:00 smithi174 ceph-mon[35637]: overall HEALTH_OK 2024-09-19T06:50:00.347 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.2 2024-09-19T06:50:00.348 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:50:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:00 smithi134 ceph-mon[35356]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T06:50:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:00 smithi134 ceph-mon[35356]: overall HEALTH_OK 2024-09-19T06:50:00.610 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:00.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:50:00 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:50:00.428+0000 7fb2d08cf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:50:00.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:50:00 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:50:00.428+0000 7fb2d08cf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:50:00.638 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:50:00 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:50:00.428+0000 7fb2d08cf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:50:00.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:50:00 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:50:00.428+0000 7fb2d08cf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:50:00.639 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 06:50:00 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T06:50:00.428+0000 7fb2d08cf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:50:00.650 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:03.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:02 smithi103 ceph-mon[31158]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T06:50:03.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:02 smithi103 ceph-mon[31158]: mgrmap e24: x(active, since 48s), standbys: y, z 2024-09-19T06:50:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:02 smithi174 ceph-mon[35637]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T06:50:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:02 smithi174 ceph-mon[35637]: mgrmap e24: x(active, since 48s), standbys: y, z 2024-09-19T06:50:03.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:02 smithi134 ceph-mon[35356]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T06:50:03.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:02 smithi134 ceph-mon[35356]: mgrmap e24: x(active, since 48s), standbys: y, z 2024-09-19T06:50:04.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:03 smithi103 ceph-mon[31158]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:50:04.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:03 smithi174 ceph-mon[35637]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:50:04.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:03 smithi134 ceph-mon[35356]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:50:04.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.954+0000 7fd02b59e640 1 -- 172.21.15.103:0/1186728067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd02c10bfc0 msgr2=0x7fd02c10e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.954+0000 7fd02b59e640 1 --2- 172.21.15.103:0/1186728067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd02c10bfc0 0x7fd02c10e460 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd024009ca0 tx=0x7fd02402a950 comp rx=0 tx=0).stop 2024-09-19T06:50:04.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.954+0000 7fd02b59e640 1 -- 172.21.15.103:0/1186728067 shutdown_connections 2024-09-19T06:50:04.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.954+0000 7fd02b59e640 1 --2- 172.21.15.103:0/1186728067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd02c10bfc0 0x7fd02c10e460 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.954+0000 7fd02b59e640 1 --2- 172.21.15.103:0/1186728067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd02c1034b0 0x7fd02c10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.954+0000 7fd02b59e640 1 --2- 172.21.15.103:0/1186728067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd02c102b90 0x7fd02c102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.954+0000 7fd02b59e640 1 -- 172.21.15.103:0/1186728067 >> 172.21.15.103:0/1186728067 conn(0x7fd02c0fc610 msgr2=0x7fd02c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.954+0000 7fd02b59e640 1 -- 172.21.15.103:0/1186728067 shutdown_connections 2024-09-19T06:50:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.954+0000 7fd02b59e640 1 -- 172.21.15.103:0/1186728067 wait complete. 2024-09-19T06:50:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.955+0000 7fd02b59e640 1 Processor -- start 2024-09-19T06:50:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.955+0000 7fd02b59e640 1 -- start start 2024-09-19T06:50:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.955+0000 7fd02b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd02c102b90 0x7fd02c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.955+0000 7fd02b59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd02c1034b0 0x7fd02c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.955+0000 7fd02b59e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd02c10bfc0 0x7fd02c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.955+0000 7fd02b59e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c111e00 con 0x7fd02c102b90 2024-09-19T06:50:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.955+0000 7fd02b59e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c111c80 con 0x7fd02c1034b0 2024-09-19T06:50:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.955+0000 7fd02b59e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c111f80 con 0x7fd02c10bfc0 2024-09-19T06:50:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.955+0000 7fd02a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd02c102b90 0x7fd02c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.955+0000 7fd02a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd02c102b90 0x7fd02c198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47792/0 (socket says 172.21.15.103:47792) 2024-09-19T06:50:04.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.955+0000 7fd02a59c640 1 -- 172.21.15.103:0/2387498093 learned_addr learned my addr 172.21.15.103:0/2387498093 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:04.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.956+0000 7fd02ad9d640 1 --2- 172.21.15.103:0/2387498093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd02c10bfc0 0x7fd02c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.956+0000 7fd029d9b640 1 --2- 172.21.15.103:0/2387498093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd02c1034b0 0x7fd02c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.956+0000 7fd02a59c640 1 -- 172.21.15.103:0/2387498093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd02c10bfc0 msgr2=0x7fd02c19cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.956+0000 7fd02a59c640 1 --2- 172.21.15.103:0/2387498093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd02c10bfc0 0x7fd02c19cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.956+0000 7fd02a59c640 1 -- 172.21.15.103:0/2387498093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd02c1034b0 msgr2=0x7fd02c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.956+0000 7fd02a59c640 1 --2- 172.21.15.103:0/2387498093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd02c1034b0 0x7fd02c1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.956+0000 7fd02a59c640 1 -- 172.21.15.103:0/2387498093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02c19d230 con 0x7fd02c102b90 2024-09-19T06:50:04.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.956+0000 7fd02a59c640 1 --2- 172.21.15.103:0/2387498093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd02c102b90 0x7fd02c198280 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00b4a0 tx=0x7fd01c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.956+0000 7fd01b7fe640 1 -- 172.21.15.103:0/2387498093 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c014070 con 0x7fd02c102b90 2024-09-19T06:50:04.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.956+0000 7fd01b7fe640 1 -- 172.21.15.103:0/2387498093 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01c010af0 con 0x7fd02c102b90 2024-09-19T06:50:04.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.956+0000 7fd02b59e640 1 -- 172.21.15.103:0/2387498093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd02c19d520 con 0x7fd02c102b90 2024-09-19T06:50:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.956+0000 7fd01b7fe640 1 -- 172.21.15.103:0/2387498093 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c017d10 con 0x7fd02c102b90 2024-09-19T06:50:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.956+0000 7fd02b59e640 1 -- 172.21.15.103:0/2387498093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd02c1a4e00 con 0x7fd02c102b90 2024-09-19T06:50:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.958+0000 7fd01b7fe640 1 -- 172.21.15.103:0/2387498093 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7fd01c017070 con 0x7fd02c102b90 2024-09-19T06:50:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.958+0000 7fd032dda740 1 Processor -- start 2024-09-19T06:50:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd032dda740 1 -- start start 2024-09-19T06:50:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd032dda740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e2085d20 0x5592e2084270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd032dda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e20847b0 0x5592e2084bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd032dda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e208f550 0x5592e2091940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd032dda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5592e1fea2f0 con 0x5592e20847b0 2024-09-19T06:50:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd032dda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5592e1f91cf0 con 0x5592e208f550 2024-09-19T06:50:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd032dda740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592e1fe62b0 con 0x5592e2085d20 2024-09-19T06:50:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd01b7fe640 1 --2- 172.21.15.103:0/2387498093 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd0140a35e0 0x7fd0140a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd02a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e20847b0 0x5592e2084bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd02a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e20847b0 0x5592e2084bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47796/0 (socket says 172.21.15.103:47796) 2024-09-19T06:50:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd02a59c640 1 -- 172.21.15.103:0/1964320827 learned_addr learned my addr 172.21.15.103:0/1964320827 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd02ad9d640 1 --2- 172.21.15.103:0/2387498093 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd0140a35e0 0x7fd0140a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd029d9b640 1 --2- 172.21.15.103:0/1964320827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e208f550 0x5592e2091940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd02ad9d640 1 --2- 172.21.15.103:0/1964320827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e2085d20 0x5592e2084270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd01b7fe640 1 -- 172.21.15.103:0/2387498093 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd01c0cf220 con 0x7fd02c102b90 2024-09-19T06:50:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd02a59c640 1 -- 172.21.15.103:0/1964320827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e2085d20 msgr2=0x5592e2084270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd02a59c640 1 --2- 172.21.15.103:0/1964320827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e2085d20 0x5592e2084270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd02a59c640 1 -- 172.21.15.103:0/1964320827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e208f550 msgr2=0x5592e2091940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd02a59c640 1 --2- 172.21.15.103:0/1964320827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e208f550 0x5592e2091940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.959+0000 7fd02a59c640 1 -- 172.21.15.103:0/1964320827 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592e1f763a0 con 0x5592e20847b0 2024-09-19T06:50:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.960+0000 7fd02a59c640 1 --2- 172.21.15.103:0/1964320827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e20847b0 0x5592e2084bf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd01c097880 tx=0x7fd01c017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f1502e51ca6b2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.960+0000 7fd02ad9d640 1 --2- 172.21.15.103:0/2387498093 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd0140a35e0 0x7fd0140a5aa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd024005e30 tx=0x7fd024009800 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.960+0000 7fd018ff9640 1 -- 172.21.15.103:0/1964320827 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0cf030 con 0x5592e20847b0 2024-09-19T06:50:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.960+0000 7fd018ff9640 1 -- 172.21.15.103:0/1964320827 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01c09f7d0 con 0x5592e20847b0 2024-09-19T06:50:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.960+0000 7fd018ff9640 1 -- 172.21.15.103:0/1964320827 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c01e070 con 0x5592e20847b0 2024-09-19T06:50:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.960+0000 7fd032dda740 1 -- 172.21.15.103:0/1964320827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e20847b0 msgr2=0x5592e2084bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.960+0000 7fd032dda740 1 --2- 172.21.15.103:0/1964320827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e20847b0 0x5592e2084bf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd01c097880 tx=0x7fd01c017730 comp rx=0 tx=0).stop 2024-09-19T06:50:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd032dda740 1 -- 172.21.15.103:0/1964320827 shutdown_connections 2024-09-19T06:50:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd032dda740 1 --2- 172.21.15.103:0/1964320827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e208f550 0x5592e2091940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd032dda740 1 --2- 172.21.15.103:0/1964320827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e20847b0 0x5592e2084bf0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd032dda740 1 --2- 172.21.15.103:0/1964320827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e2085d20 0x5592e2084270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd032dda740 1 -- 172.21.15.103:0/1964320827 >> 172.21.15.103:0/1964320827 conn(0x5592e204d310 msgr2=0x5592e2080e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd032dda740 1 -- 172.21.15.103:0/1964320827 shutdown_connections 2024-09-19T06:50:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd032dda740 1 -- 172.21.15.103:0/1964320827 wait complete. 2024-09-19T06:50:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd032dda740 1 -- start start 2024-09-19T06:50:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd032dda740 1 Processor -- start 2024-09-19T06:50:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd032dda740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e20847b0 0x5592e208f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd032dda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e2085d20 0x5592e208c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd032dda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e208f550 0x5592e208ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd032dda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5592e20921f0 con 0x5592e208f550 2024-09-19T06:50:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd032dda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5592e1fa5060 con 0x5592e2085d20 2024-09-19T06:50:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd032dda740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592e2092370 con 0x5592e20847b0 2024-09-19T06:50:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd032dda740 10 client.0 _ll_register_callbacks cb 0x5592e1cf88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd02ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e208f550 0x5592e208ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd02ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e208f550 0x5592e208ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47802/0 (socket says 172.21.15.103:47802) 2024-09-19T06:50:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.961+0000 7fd02ad9d640 1 -- 172.21.15.103:0/1141469109 learned_addr learned my addr 172.21.15.103:0/1141469109 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd029d9b640 1 --2- 172.21.15.103:0/1141469109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e2085d20 0x5592e208c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd02ad9d640 1 -- 172.21.15.103:0/1141469109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e20847b0 msgr2=0x5592e208f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd02a59c640 1 --2- 172.21.15.103:0/1141469109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e20847b0 0x5592e208f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd02ad9d640 1 --2- 172.21.15.103:0/1141469109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e20847b0 0x5592e208f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd02ad9d640 1 -- 172.21.15.103:0/1141469109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e2085d20 msgr2=0x5592e208c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd02ad9d640 1 --2- 172.21.15.103:0/1141469109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e2085d20 0x5592e208c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd02ad9d640 1 -- 172.21.15.103:0/1141469109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592e1d5b3c0 con 0x5592e208f550 2024-09-19T06:50:04.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd029d9b640 1 --2- 172.21.15.103:0/1141469109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e2085d20 0x5592e208c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:04.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd02ad9d640 1 --2- 172.21.15.103:0/1141469109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e208f550 0x5592e208ca40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd02402fd90 tx=0x7fd024047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6433c3097400642 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:04.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd011ffb640 1 -- 172.21.15.103:0/1141469109 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02404e070 con 0x5592e208f550 2024-09-19T06:50:04.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd011ffb640 1 -- 172.21.15.103:0/1141469109 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd0240044c0 con 0x5592e208f550 2024-09-19T06:50:04.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd032dda740 1 -- 172.21.15.103:0/1141469109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd01c09f7d0 con 0x5592e208f550 2024-09-19T06:50:04.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd011ffb640 1 -- 172.21.15.103:0/1141469109 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd024034040 con 0x5592e208f550 2024-09-19T06:50:04.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd032dda740 10 client.15642 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:04.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd032dda740 1 -- 172.21.15.103:0/1141469109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5592e209dd60 con 0x5592e208f550 2024-09-19T06:50:04.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd011ffb640 1 -- 172.21.15.103:0/1141469109 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01c09f7d0 con 0x5592e208f550 2024-09-19T06:50:04.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.962+0000 7fd011ffb640 1 -- 172.21.15.103:0/1141469109 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd024030440 con 0x5592e208f550 2024-09-19T06:50:04.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.963+0000 7fd032dda740 10 client.15642 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:04.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.963+0000 7fd032dda740 10 client.15642 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:50:04.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.963+0000 7fd032dda740 20 client.15642 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:04.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.963+0000 7fd032dda740 1 --2- 172.21.15.103:0/1141469109 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5592e209f1c0 0x5592e20a1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.963+0000 7fd032dda740 4 client.15642 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:04.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.963+0000 7fd032dda740 1 -- 172.21.15.103:0/1141469109 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592e1ff7860 con 0x5592e209f1c0 2024-09-19T06:50:04.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.963+0000 7fd02a59c640 1 --2- 172.21.15.103:0/1141469109 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5592e209f1c0 0x5592e20a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.963+0000 7fd011ffb640 10 client.15642 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:50:04.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.963+0000 7fd02a59c640 1 --2- 172.21.15.103:0/1141469109 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5592e209f1c0 0x5592e20a1640 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6864a1f52bf8e325 server_cookie=b356c009bb5b660a in_seq=0 out_seq=0 2024-09-19T06:50:04.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.964+0000 7fd011ffb640 1 -- 172.21.15.103:0/1141469109 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592e1ff7860 con 0x5592e209f1c0 2024-09-19T06:50:04.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.964+0000 7fd011ffb640 10 client.15642 handle_command_reply: tid=0 2024-09-19T06:50:04.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.964+0000 7fd032dda740 1 client.15642 shutdown 2024-09-19T06:50:04.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.964+0000 7fd032dda740 1 -- 172.21.15.103:0/1141469109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e208f550 msgr2=0x5592e208ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.964+0000 7fd032dda740 1 --2- 172.21.15.103:0/1141469109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e208f550 0x5592e208ca40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd02402fd90 tx=0x7fd024047000 comp rx=0 tx=0).stop 2024-09-19T06:50:04.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.964+0000 7fd032dda740 1 -- 172.21.15.103:0/1141469109 shutdown_connections 2024-09-19T06:50:04.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.964+0000 7fd032dda740 1 --2- 172.21.15.103:0/1141469109 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5592e209f1c0 0x5592e20a1640 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.964+0000 7fd032dda740 1 --2- 172.21.15.103:0/1141469109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e208f550 0x5592e208ca40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.964+0000 7fd032dda740 1 --2- 172.21.15.103:0/1141469109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e2085d20 0x5592e208c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.964+0000 7fd011ffb640 0 client.15642 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:50:04.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.964+0000 7fd032dda740 1 --2- 172.21.15.103:0/1141469109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e20847b0 0x5592e208f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.964+0000 7fd032dda740 1 -- 172.21.15.103:0/1141469109 >> 172.21.15.103:0/1141469109 conn(0x5592e204d310 msgr2=0x5592e2086b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.964+0000 7fd032dda740 1 -- 172.21.15.103:0/1141469109 shutdown_connections 2024-09-19T06:50:04.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.964+0000 7fd032dda740 1 -- 172.21.15.103:0/1141469109 wait complete. 2024-09-19T06:50:04.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.964+0000 7fd032dda740 20 client.15642 trim_cache size 0 max 16384 2024-09-19T06:50:04.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.977+0000 7fd032dda740 1 Processor -- start 2024-09-19T06:50:04.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.977+0000 7fd032dda740 1 -- start start 2024-09-19T06:50:04.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.977+0000 7fd032dda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e20847b0 0x5592e208c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.977+0000 7fd032dda740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e2085d20 0x5592e20808e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.977+0000 7fd032dda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e208f550 0x5592e2080e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.977+0000 7fd032dda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1ad780 con 0x5592e208f550 2024-09-19T06:50:04.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.977+0000 7fd032dda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1af2c0 con 0x5592e20847b0 2024-09-19T06:50:04.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.977+0000 7fd032dda740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1ade50 con 0x5592e2085d20 2024-09-19T06:50:04.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.977+0000 7fd02ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e208f550 0x5592e2080e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.977+0000 7fd02ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e208f550 0x5592e2080e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47818/0 (socket says 172.21.15.103:47818) 2024-09-19T06:50:04.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.977+0000 7fd02ad9d640 1 -- 172.21.15.103:0/1272137641 learned_addr learned my addr 172.21.15.103:0/1272137641 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:04.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.978+0000 7fd02a59c640 1 --2- 172.21.15.103:0/1272137641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e20847b0 0x5592e208c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.978+0000 7fd02ad9d640 1 -- 172.21.15.103:0/1272137641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e2085d20 msgr2=0x5592e20808e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.978+0000 7fd029d9b640 1 --2- 172.21.15.103:0/1272137641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e2085d20 0x5592e20808e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.978+0000 7fd02ad9d640 1 --2- 172.21.15.103:0/1272137641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e2085d20 0x5592e20808e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.978+0000 7fd02ad9d640 1 -- 172.21.15.103:0/1272137641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e20847b0 msgr2=0x5592e208c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.978+0000 7fd02ad9d640 1 --2- 172.21.15.103:0/1272137641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e20847b0 0x5592e208c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.978+0000 7fd02ad9d640 1 -- 172.21.15.103:0/1272137641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592e2081360 con 0x5592e208f550 2024-09-19T06:50:04.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.978+0000 7fd02ad9d640 1 --2- 172.21.15.103:0/1272137641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e208f550 0x5592e2080e20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd02402f8b0 tx=0x7fd024005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4f778a0f259026 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.978+0000 7fd013fff640 1 -- 172.21.15.103:0/1272137641 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024061030 con 0x5592e208f550 2024-09-19T06:50:04.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.978+0000 7fd013fff640 1 -- 172.21.15.103:0/1272137641 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd02403f070 con 0x5592e208f550 2024-09-19T06:50:04.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.978+0000 7fd013fff640 1 -- 172.21.15.103:0/1272137641 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02404e040 con 0x5592e208f550 2024-09-19T06:50:04.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.978+0000 7fd032dda740 1 -- 172.21.15.103:0/1272137641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e208f550 msgr2=0x5592e2080e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 1 --2- 172.21.15.103:0/1272137641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e208f550 0x5592e2080e20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd02402f8b0 tx=0x7fd024005c90 comp rx=0 tx=0).stop 2024-09-19T06:50:04.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 1 -- 172.21.15.103:0/1272137641 shutdown_connections 2024-09-19T06:50:04.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 1 --2- 172.21.15.103:0/1272137641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e208f550 0x5592e2080e20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 1 --2- 172.21.15.103:0/1272137641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e2085d20 0x5592e20808e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 1 --2- 172.21.15.103:0/1272137641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e20847b0 0x5592e208c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 1 -- 172.21.15.103:0/1272137641 >> 172.21.15.103:0/1272137641 conn(0x5592e204d310 msgr2=0x5592e207bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 1 -- 172.21.15.103:0/1272137641 shutdown_connections 2024-09-19T06:50:04.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 1 -- 172.21.15.103:0/1272137641 wait complete. 2024-09-19T06:50:04.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 1 -- start start 2024-09-19T06:50:04.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 1 Processor -- start 2024-09-19T06:50:04.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e20847b0 0x5592e20933a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e2085d20 0x5592e20938e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e208f550 0x5592e2093e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1aebf0 con 0x5592e2085d20 2024-09-19T06:50:04.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1ae520 con 0x5592e208f550 2024-09-19T06:50:04.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1ab740 con 0x5592e20847b0 2024-09-19T06:50:04.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd029d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e2085d20 0x5592e20938e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd029d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e2085d20 0x5592e20938e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47820/0 (socket says 172.21.15.103:47820) 2024-09-19T06:50:04.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd032dda740 10 client.0 _ll_register_callbacks cb 0x7fd024005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:04.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd029d9b640 1 -- 172.21.15.103:0/1702043935 learned_addr learned my addr 172.21.15.103:0/1702043935 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:04.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd02a59c640 1 --2- 172.21.15.103:0/1702043935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e20847b0 0x5592e20933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.979+0000 7fd02ad9d640 1 --2- 172.21.15.103:0/1702043935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e208f550 0x5592e2093e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.980+0000 7fd029d9b640 1 -- 172.21.15.103:0/1702043935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e20847b0 msgr2=0x5592e20933a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.980+0000 7fd029d9b640 1 --2- 172.21.15.103:0/1702043935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e20847b0 0x5592e20933a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.980+0000 7fd029d9b640 1 -- 172.21.15.103:0/1702043935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e208f550 msgr2=0x5592e2093e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.980+0000 7fd029d9b640 1 --2- 172.21.15.103:0/1702043935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e208f550 0x5592e2093e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.980+0000 7fd029d9b640 1 -- 172.21.15.103:0/1702043935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592e2108ee0 con 0x5592e2085d20 2024-09-19T06:50:04.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.980+0000 7fd029d9b640 1 --2- 172.21.15.103:0/1702043935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e2085d20 0x5592e20938e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd02c10d2e0 tx=0x7fd020013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e769c15e02abe09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.980+0000 7fd011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:04.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.982+0000 7fd011ffb640 1 -- 172.21.15.103:0/1702043935 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020020060 con 0x5592e2085d20 2024-09-19T06:50:04.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.982+0000 7fd011ffb640 1 -- 172.21.15.103:0/1702043935 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd02001b410 con 0x5592e2085d20 2024-09-19T06:50:04.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.982+0000 7fd032dda740 1 -- 172.21.15.103:0/1702043935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd02403f070 con 0x5592e2085d20 2024-09-19T06:50:04.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.982+0000 7fd011ffb640 1 -- 172.21.15.103:0/1702043935 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd02001b610 con 0x5592e2085d20 2024-09-19T06:50:04.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.982+0000 7fd032dda740 10 client.15648 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:04.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.982+0000 7fd032dda740 1 -- 172.21.15.103:0/1702043935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5592e210a250 con 0x5592e2085d20 2024-09-19T06:50:04.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.983+0000 7fd011ffb640 1 -- 172.21.15.103:0/1702043935 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd02403f070 con 0x5592e2085d20 2024-09-19T06:50:04.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.983+0000 7fd011ffb640 1 -- 172.21.15.103:0/1702043935 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd02000fa50 con 0x5592e2085d20 2024-09-19T06:50:04.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.983+0000 7fd032dda740 10 client.15648 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:04.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.983+0000 7fd032dda740 10 client.15648 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:50:04.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.983+0000 7fd032dda740 20 client.15648 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:04.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.983+0000 7fd032dda740 1 --2- 172.21.15.103:0/1702043935 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5592e210aa30 0x5592e210ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.983+0000 7fd032dda740 4 client.15648 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:04.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.983+0000 7fd032dda740 1 -- 172.21.15.103:0/1702043935 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592e1ff1d70 con 0x5592e210aa30 2024-09-19T06:50:04.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.983+0000 7fd02a59c640 1 --2- 172.21.15.103:0/1702043935 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5592e210aa30 0x5592e210ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.983+0000 7fd02a59c640 1 --2- 172.21.15.103:0/1702043935 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5592e210aa30 0x5592e210ce50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=abce500ebc9d4d1c server_cookie=2f914774978ff944 in_seq=0 out_seq=0 2024-09-19T06:50:04.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.983+0000 7fd011ffb640 10 client.15648 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:50:04.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.984+0000 7fd011ffb640 1 -- 172.21.15.103:0/1702043935 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592e1ff1d70 con 0x5592e210aa30 2024-09-19T06:50:04.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.984+0000 7fd011ffb640 10 client.15648 handle_command_reply: tid=0 2024-09-19T06:50:04.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.984+0000 7fd032dda740 1 client.15648 shutdown 2024-09-19T06:50:04.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.984+0000 7fd032dda740 1 -- 172.21.15.103:0/1702043935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e2085d20 msgr2=0x5592e20938e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.984+0000 7fd032dda740 1 --2- 172.21.15.103:0/1702043935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e2085d20 0x5592e20938e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd02c10d2e0 tx=0x7fd020013830 comp rx=0 tx=0).stop 2024-09-19T06:50:04.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.984+0000 7fd032dda740 1 -- 172.21.15.103:0/1702043935 shutdown_connections 2024-09-19T06:50:04.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.984+0000 7fd032dda740 1 --2- 172.21.15.103:0/1702043935 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5592e210aa30 0x5592e210ce50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.984+0000 7fd032dda740 1 --2- 172.21.15.103:0/1702043935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592e208f550 0x5592e2093e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.984+0000 7fd032dda740 1 --2- 172.21.15.103:0/1702043935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592e2085d20 0x5592e20938e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.984+0000 7fd032dda740 1 --2- 172.21.15.103:0/1702043935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592e20847b0 0x5592e20933a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.984+0000 7fd011ffb640 0 client.15648 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:50:04.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.984+0000 7fd032dda740 1 -- 172.21.15.103:0/1702043935 >> 172.21.15.103:0/1702043935 conn(0x5592e204d310 msgr2=0x5592e20808c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.984+0000 7fd032dda740 1 -- 172.21.15.103:0/1702043935 shutdown_connections 2024-09-19T06:50:04.480 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:50:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.984+0000 7fd032dda740 1 -- 172.21.15.103:0/1702043935 wait complete. 2024-09-19T06:50:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.984+0000 7fd032dda740 20 client.15648 trim_cache size 0 max 16384 2024-09-19T06:50:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.985+0000 7fd02b59e640 1 -- 172.21.15.103:0/2387498093 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd0140a35e0 msgr2=0x7fd0140a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.985+0000 7fd02b59e640 1 --2- 172.21.15.103:0/2387498093 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd0140a35e0 0x7fd0140a5aa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd024005e30 tx=0x7fd024009800 comp rx=0 tx=0).stop 2024-09-19T06:50:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.985+0000 7fd02b59e640 1 -- 172.21.15.103:0/2387498093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd02c102b90 msgr2=0x7fd02c198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.985+0000 7fd02b59e640 1 --2- 172.21.15.103:0/2387498093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd02c102b90 0x7fd02c198280 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00b4a0 tx=0x7fd01c00b960 comp rx=0 tx=0).stop 2024-09-19T06:50:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.985+0000 7fd02b59e640 1 -- 172.21.15.103:0/2387498093 shutdown_connections 2024-09-19T06:50:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.985+0000 7fd02b59e640 1 --2- 172.21.15.103:0/2387498093 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd0140a35e0 0x7fd0140a5aa0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.985+0000 7fd02b59e640 1 --2- 172.21.15.103:0/2387498093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd02c10bfc0 0x7fd02c19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.985+0000 7fd02b59e640 1 --2- 172.21.15.103:0/2387498093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd02c1034b0 0x7fd02c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.985+0000 7fd02b59e640 1 --2- 172.21.15.103:0/2387498093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd02c102b90 0x7fd02c198280 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.985+0000 7fd02b59e640 1 -- 172.21.15.103:0/2387498093 >> 172.21.15.103:0/2387498093 conn(0x7fd02c0fc610 msgr2=0x7fd02c0fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.985+0000 7fd02b59e640 1 -- 172.21.15.103:0/2387498093 shutdown_connections 2024-09-19T06:50:04.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:03.985+0000 7fd02b59e640 1 -- 172.21.15.103:0/2387498093 wait complete. 2024-09-19T06:50:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.613+0000 7f3c3e8c9640 1 -- 172.21.15.103:0/1900917524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c381069c0 msgr2=0x7f3c38106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.613+0000 7f3c3e8c9640 1 --2- 172.21.15.103:0/1900917524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c381069c0 0x7f3c38106da0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009930 tx=0x7f3c2802a420 comp rx=0 tx=0).stop 2024-09-19T06:50:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.613+0000 7f3c3e8c9640 1 -- 172.21.15.103:0/1900917524 shutdown_connections 2024-09-19T06:50:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.613+0000 7f3c3e8c9640 1 --2- 172.21.15.103:0/1900917524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c38101350 0x7f3c3810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.613+0000 7f3c3e8c9640 1 --2- 172.21.15.103:0/1900917524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c381009b0 0x7f3c38100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.613+0000 7f3c3e8c9640 1 --2- 172.21.15.103:0/1900917524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c381069c0 0x7f3c38106da0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.613+0000 7f3c3e8c9640 1 -- 172.21.15.103:0/1900917524 >> 172.21.15.103:0/1900917524 conn(0x7f3c38077990 msgr2=0x7f3c380fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.613+0000 7f3c3e8c9640 1 -- 172.21.15.103:0/1900917524 shutdown_connections 2024-09-19T06:50:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.613+0000 7f3c3e8c9640 1 -- 172.21.15.103:0/1900917524 wait complete. 2024-09-19T06:50:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.614+0000 7f3c3e8c9640 1 Processor -- start 2024-09-19T06:50:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.614+0000 7f3c3e8c9640 1 -- start start 2024-09-19T06:50:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.614+0000 7f3c3e8c9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c381009b0 0x7f3c381991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c3e8c9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c38101350 0x7f3c38199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c3e8c9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c381069c0 0x7f3c381a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c3e8c9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c38110e00 con 0x7f3c381069c0 2024-09-19T06:50:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c3e8c9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c38110c80 con 0x7f3c381009b0 2024-09-19T06:50:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c3e8c9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c38110f80 con 0x7f3c38101350 2024-09-19T06:50:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c3ce66640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c381069c0 0x7f3c381a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c3ce66640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c381069c0 0x7f3c381a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47840/0 (socket says 172.21.15.103:47840) 2024-09-19T06:50:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c3ce66640 1 -- 172.21.15.103:0/3601317549 learned_addr learned my addr 172.21.15.103:0/3601317549 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c37fff640 1 --2- 172.21.15.103:0/3601317549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c381009b0 0x7f3c381991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c377fe640 1 --2- 172.21.15.103:0/3601317549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c38101350 0x7f3c38199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c3ce66640 1 -- 172.21.15.103:0/3601317549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c38101350 msgr2=0x7f3c38199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c3ce66640 1 --2- 172.21.15.103:0/3601317549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c38101350 0x7f3c38199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c3ce66640 1 -- 172.21.15.103:0/3601317549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c381009b0 msgr2=0x7f3c381991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c3ce66640 1 --2- 172.21.15.103:0/3601317549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c381009b0 0x7f3c381991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c3ce66640 1 -- 172.21.15.103:0/3601317549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c381a47b0 con 0x7f3c381069c0 2024-09-19T06:50:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c377fe640 1 --2- 172.21.15.103:0/3601317549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c38101350 0x7f3c38199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c3ce66640 1 --2- 172.21.15.103:0/3601317549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c381069c0 0x7f3c381a4270 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000eaa0 tx=0x7f3c3000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c357fa640 1 -- 172.21.15.103:0/3601317549 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3000cdb0 con 0x7f3c381069c0 2024-09-19T06:50:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c357fa640 1 -- 172.21.15.103:0/3601317549 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c30005020 con 0x7f3c381069c0 2024-09-19T06:50:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c357fa640 1 -- 172.21.15.103:0/3601317549 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c300116b0 con 0x7f3c381069c0 2024-09-19T06:50:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c3e8c9640 1 -- 172.21.15.103:0/3601317549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c381a4980 con 0x7f3c381069c0 2024-09-19T06:50:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.615+0000 7f3c3e8c9640 1 -- 172.21.15.103:0/3601317549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c381a4ea0 con 0x7f3c381069c0 2024-09-19T06:50:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.618+0000 7f3c357fa640 1 -- 172.21.15.103:0/3601317549 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7f3c30005560 con 0x7f3c381069c0 2024-09-19T06:50:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c4165f740 1 Processor -- start 2024-09-19T06:50:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c4165f740 1 -- start start 2024-09-19T06:50:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c4165f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322bfaf0 0x5565322bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c4165f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c4165f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322cacb0 0x5565322cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c4165f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5565322242f0 con 0x5565322c0410 2024-09-19T06:50:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c4165f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565321cc270 con 0x5565322bfaf0 2024-09-19T06:50:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c4165f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5565322202b0 con 0x5565322cacb0 2024-09-19T06:50:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c37fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c37fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47848/0 (socket says 172.21.15.103:47848) 2024-09-19T06:50:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c37fff640 1 -- 172.21.15.103:0/186888529 learned_addr learned my addr 172.21.15.103:0/186888529 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c357fa640 1 --2- 172.21.15.103:0/3601317549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c200a3690 0x7f3c200a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c377fe640 1 --2- 172.21.15.103:0/186888529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322bfaf0 0x5565322bfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c3ce66640 1 --2- 172.21.15.103:0/3601317549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c200a3690 0x7f3c200a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c377fe640 1 --2- 172.21.15.103:0/186888529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322cacb0 0x5565322cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c37fff640 1 -- 172.21.15.103:0/186888529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322cacb0 msgr2=0x5565322cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c37fff640 1 --2- 172.21.15.103:0/186888529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322cacb0 0x5565322cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c37fff640 1 -- 172.21.15.103:0/186888529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322bfaf0 msgr2=0x5565322bfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c37fff640 1 --2- 172.21.15.103:0/186888529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322bfaf0 0x5565322bfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c37fff640 1 -- 172.21.15.103:0/186888529 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565321cb340 con 0x5565322c0410 2024-09-19T06:50:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.619+0000 7f3c357fa640 1 -- 172.21.15.103:0/3601317549 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f3c300d00c0 con 0x7f3c381069c0 2024-09-19T06:50:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.620+0000 7f3c37fff640 1 --2- 172.21.15.103:0/186888529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322ca770 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009930 tx=0x7f3c28031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebba660a861ca278 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.620+0000 7f3c377fe640 1 --2- 172.21.15.103:0/186888529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322bfaf0 0x5565322bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.620+0000 7f3c1e7fc640 1 -- 172.21.15.103:0/186888529 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28040070 con 0x5565322c0410 2024-09-19T06:50:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.620+0000 7f3c3ce66640 1 --2- 172.21.15.103:0/3601317549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c200a3690 0x7f3c200a5b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3c300cd6e0 tx=0x7f3c3006df90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.620+0000 7f3c1e7fc640 1 -- 172.21.15.103:0/186888529 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c28005020 con 0x5565322c0410 2024-09-19T06:50:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.620+0000 7f3c1e7fc640 1 -- 172.21.15.103:0/186888529 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28035040 con 0x5565322c0410 2024-09-19T06:50:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.621+0000 7f3c4165f740 1 -- 172.21.15.103:0/186888529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 msgr2=0x5565322ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.621+0000 7f3c4165f740 1 --2- 172.21.15.103:0/186888529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322ca770 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009930 tx=0x7f3c28031630 comp rx=0 tx=0).stop 2024-09-19T06:50:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.621+0000 7f3c4165f740 1 -- 172.21.15.103:0/186888529 shutdown_connections 2024-09-19T06:50:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.621+0000 7f3c4165f740 1 --2- 172.21.15.103:0/186888529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322cacb0 0x5565322cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.621+0000 7f3c4165f740 1 --2- 172.21.15.103:0/186888529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322ca770 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.621+0000 7f3c4165f740 1 --2- 172.21.15.103:0/186888529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322bfaf0 0x5565322bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.621+0000 7f3c4165f740 1 -- 172.21.15.103:0/186888529 >> 172.21.15.103:0/186888529 conn(0x5565322bc060 msgr2=0x5565322bc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.621+0000 7f3c4165f740 1 -- 172.21.15.103:0/186888529 shutdown_connections 2024-09-19T06:50:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.621+0000 7f3c4165f740 1 -- 172.21.15.103:0/186888529 wait complete. 2024-09-19T06:50:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.621+0000 7f3c4165f740 1 -- start start 2024-09-19T06:50:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.621+0000 7f3c4165f740 1 Processor -- start 2024-09-19T06:50:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c4165f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322bfaf0 0x5565322c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c4165f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322c0410 0x5565322c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c4165f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322cacb0 0x5565322c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c4165f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5565322cd880 con 0x5565322bfaf0 2024-09-19T06:50:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c4165f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565321ad290 con 0x5565322c0410 2024-09-19T06:50:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c4165f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5565322cda00 con 0x5565322cacb0 2024-09-19T06:50:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c4165f740 10 client.0 _ll_register_callbacks cb 0x5565321de140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322bfaf0 0x5565322c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322bfaf0 0x5565322c7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47856/0 (socket says 172.21.15.103:47856) 2024-09-19T06:50:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c377fe640 1 -- 172.21.15.103:0/1317583428 learned_addr learned my addr 172.21.15.103:0/1317583428 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c37fff640 1 --2- 172.21.15.103:0/1317583428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322c0410 0x5565322c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c377fe640 1 -- 172.21.15.103:0/1317583428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322cacb0 msgr2=0x5565322c8510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:50:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c377fe640 1 --2- 172.21.15.103:0/1317583428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322cacb0 0x5565322c8510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c377fe640 1 -- 172.21.15.103:0/1317583428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322c0410 msgr2=0x5565322c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c377fe640 1 --2- 172.21.15.103:0/1317583428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322c0410 0x5565322c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c377fe640 1 -- 172.21.15.103:0/1317583428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556531f37210 con 0x5565322bfaf0 2024-09-19T06:50:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c377fe640 1 --2- 172.21.15.103:0/1317583428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322bfaf0 0x5565322c7a90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00cfd0 tx=0x7f3c2c010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da19246e68ec83a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c177fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c177fe640 1 -- 172.21.15.103:0/1317583428 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00fc40 con 0x5565322bfaf0 2024-09-19T06:50:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c177fe640 1 -- 172.21.15.103:0/1317583428 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f3c2c00fde0 con 0x5565322bfaf0 2024-09-19T06:50:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.622+0000 7f3c4165f740 1 -- 172.21.15.103:0/1317583428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c28005020 con 0x5565322bfaf0 2024-09-19T06:50:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.623+0000 7f3c177fe640 1 -- 172.21.15.103:0/1317583428 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c2c01c480 con 0x5565322bfaf0 2024-09-19T06:50:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.623+0000 7f3c4165f740 10 client.15660 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.623+0000 7f3c4165f740 1 -- 172.21.15.103:0/1317583428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556532003620 con 0x5565322bfaf0 2024-09-19T06:50:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.623+0000 7f3c177fe640 1 -- 172.21.15.103:0/1317583428 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c28005020 con 0x5565322bfaf0 2024-09-19T06:50:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.623+0000 7f3c177fe640 1 -- 172.21.15.103:0/1317583428 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3c2c0258b0 con 0x5565322bfaf0 2024-09-19T06:50:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.623+0000 7f3c4165f740 10 client.15660 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.623+0000 7f3c4165f740 10 client.15660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.623+0000 7f3c4165f740 20 client.15660 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.623+0000 7f3c4165f740 1 --2- 172.21.15.103:0/1317583428 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565322daad0 0x5565322dcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.623+0000 7f3c4165f740 4 client.15660 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.623+0000 7f3c4165f740 1 -- 172.21.15.103:0/1317583428 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556532231a20 con 0x5565322daad0 2024-09-19T06:50:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.623+0000 7f3c37fff640 1 --2- 172.21.15.103:0/1317583428 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565322daad0 0x5565322dcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.624+0000 7f3c177fe640 10 client.15660 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.624+0000 7f3c37fff640 1 --2- 172.21.15.103:0/1317583428 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565322daad0 0x5565322dcec0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f3e4c712920f17b server_cookie=f0d035b727aff50c in_seq=0 out_seq=0 2024-09-19T06:50:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.624+0000 7f3c177fe640 1 -- 172.21.15.103:0/1317583428 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556532231a20 con 0x5565322daad0 2024-09-19T06:50:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.624+0000 7f3c177fe640 10 client.15660 handle_command_reply: tid=0 2024-09-19T06:50:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.624+0000 7f3c4165f740 1 client.15660 shutdown 2024-09-19T06:50:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.625+0000 7f3c4165f740 1 -- 172.21.15.103:0/1317583428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322bfaf0 msgr2=0x5565322c7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.625+0000 7f3c4165f740 1 --2- 172.21.15.103:0/1317583428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322bfaf0 0x5565322c7a90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00cfd0 tx=0x7f3c2c010e50 comp rx=0 tx=0).stop 2024-09-19T06:50:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.625+0000 7f3c4165f740 1 -- 172.21.15.103:0/1317583428 shutdown_connections 2024-09-19T06:50:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.625+0000 7f3c4165f740 1 --2- 172.21.15.103:0/1317583428 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565322daad0 0x5565322dcec0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.625+0000 7f3c4165f740 1 --2- 172.21.15.103:0/1317583428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322cacb0 0x5565322c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.625+0000 7f3c4165f740 1 --2- 172.21.15.103:0/1317583428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322c0410 0x5565322c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.625+0000 7f3c4165f740 1 --2- 172.21.15.103:0/1317583428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322bfaf0 0x5565322c7a90 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.625+0000 7f3c4165f740 1 -- 172.21.15.103:0/1317583428 >> 172.21.15.103:0/1317583428 conn(0x5565322bc060 msgr2=0x5565322c1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.625+0000 7f3c177fe640 0 client.15660 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.625+0000 7f3c4165f740 1 -- 172.21.15.103:0/1317583428 shutdown_connections 2024-09-19T06:50:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.625+0000 7f3c4165f740 1 -- 172.21.15.103:0/1317583428 wait complete. 2024-09-19T06:50:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.625+0000 7f3c4165f740 20 client.15660 trim_cache size 0 max 16384 2024-09-19T06:50:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c4165f740 1 Processor -- start 2024-09-19T06:50:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c4165f740 1 -- start start 2024-09-19T06:50:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c4165f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322bc060 0x5565322c6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c4165f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322bfaf0 0x5565322c8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c4165f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322e2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c4165f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c381ad810 con 0x5565322c0410 2024-09-19T06:50:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c4165f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c381af350 con 0x5565322bfaf0 2024-09-19T06:50:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c4165f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c381adee0 con 0x5565322bc060 2024-09-19T06:50:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322e2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322e2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47866/0 (socket says 172.21.15.103:47866) 2024-09-19T06:50:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c377fe640 1 -- 172.21.15.103:0/3300412753 learned_addr learned my addr 172.21.15.103:0/3300412753 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c37fff640 1 --2- 172.21.15.103:0/3300412753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322bfaf0 0x5565322c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c377fe640 1 -- 172.21.15.103:0/3300412753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322bc060 msgr2=0x5565322c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c377fe640 1 --2- 172.21.15.103:0/3300412753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322bc060 0x5565322c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c377fe640 1 -- 172.21.15.103:0/3300412753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322bfaf0 msgr2=0x5565322c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c377fe640 1 --2- 172.21.15.103:0/3300412753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322bfaf0 0x5565322c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.639+0000 7f3c377fe640 1 -- 172.21.15.103:0/3300412753 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565322bf870 con 0x5565322c0410 2024-09-19T06:50:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.640+0000 7f3c377fe640 1 --2- 172.21.15.103:0/3300412753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322e2740 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00c840 tx=0x7f3c2c0115d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792503277551e46d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.640+0000 7f3c1dffb640 1 -- 172.21.15.103:0/3300412753 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00fc40 con 0x5565322c0410 2024-09-19T06:50:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.640+0000 7f3c1dffb640 1 -- 172.21.15.103:0/3300412753 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2c00fde0 con 0x5565322c0410 2024-09-19T06:50:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.640+0000 7f3c1dffb640 1 -- 172.21.15.103:0/3300412753 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c011d50 con 0x5565322c0410 2024-09-19T06:50:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.640+0000 7f3c4165f740 1 -- 172.21.15.103:0/3300412753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 msgr2=0x5565322e2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.640+0000 7f3c4165f740 1 --2- 172.21.15.103:0/3300412753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322e2740 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00c840 tx=0x7f3c2c0115d0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.641+0000 7f3c4165f740 1 -- 172.21.15.103:0/3300412753 shutdown_connections 2024-09-19T06:50:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.641+0000 7f3c4165f740 1 --2- 172.21.15.103:0/3300412753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322e2740 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.641+0000 7f3c4165f740 1 --2- 172.21.15.103:0/3300412753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322bfaf0 0x5565322c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.641+0000 7f3c4165f740 1 --2- 172.21.15.103:0/3300412753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322bc060 0x5565322c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.641+0000 7f3c4165f740 1 -- 172.21.15.103:0/3300412753 >> 172.21.15.103:0/3300412753 conn(0x5565322bcae0 msgr2=0x5565322c9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.641+0000 7f3c4165f740 1 -- 172.21.15.103:0/3300412753 shutdown_connections 2024-09-19T06:50:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.641+0000 7f3c4165f740 1 -- 172.21.15.103:0/3300412753 wait complete. 2024-09-19T06:50:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.641+0000 7f3c4165f740 1 -- start start 2024-09-19T06:50:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.641+0000 7f3c4165f740 1 Processor -- start 2024-09-19T06:50:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.641+0000 7f3c4165f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322bc060 0x5565322d32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.641+0000 7f3c4165f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322bfaf0 0x5565322cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.641+0000 7f3c4165f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.641+0000 7f3c4165f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c381aec80 con 0x5565322c0410 2024-09-19T06:50:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.641+0000 7f3c4165f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c381ae5b0 con 0x5565322bc060 2024-09-19T06:50:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.641+0000 7f3c4165f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c381ab7d0 con 0x5565322bfaf0 2024-09-19T06:50:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.641+0000 7f3c4165f740 10 client.0 _ll_register_callbacks cb 0x7f3c2c00cf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322cbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47878/0 (socket says 172.21.15.103:47878) 2024-09-19T06:50:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c377fe640 1 -- 172.21.15.103:0/634588021 learned_addr learned my addr 172.21.15.103:0/634588021 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c377fe640 1 --2- 172.21.15.103:0/634588021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322bc060 0x5565322d32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c377fe640 1 -- 172.21.15.103:0/634588021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322bfaf0 msgr2=0x5565322cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c377fe640 1 --2- 172.21.15.103:0/634588021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322bfaf0 0x5565322cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c377fe640 1 -- 172.21.15.103:0/634588021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322bc060 msgr2=0x5565322d32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c377fe640 1 --2- 172.21.15.103:0/634588021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322bc060 0x5565322d32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c377fe640 1 -- 172.21.15.103:0/634588021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565322cc510 con 0x5565322c0410 2024-09-19T06:50:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c377fe640 1 --2- 172.21.15.103:0/634588021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322cbed0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c005bd0 tx=0x7f3c2c011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a5417754461c5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c177fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c177fe640 1 -- 172.21.15.103:0/634588021 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c011de0 con 0x5565322c0410 2024-09-19T06:50:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c177fe640 1 -- 172.21.15.103:0/634588021 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f3c2c025470 con 0x5565322c0410 2024-09-19T06:50:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c4165f740 1 -- 172.21.15.103:0/634588021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c2c00fde0 con 0x5565322c0410 2024-09-19T06:50:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c177fe640 1 -- 172.21.15.103:0/634588021 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c2c025670 con 0x5565322c0410 2024-09-19T06:50:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c4165f740 10 client.15666 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.642+0000 7f3c4165f740 1 -- 172.21.15.103:0/634588021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556532345710 con 0x5565322c0410 2024-09-19T06:50:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.643+0000 7f3c177fe640 1 -- 172.21.15.103:0/634588021 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2c00fde0 con 0x5565322c0410 2024-09-19T06:50:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.643+0000 7f3c177fe640 1 -- 172.21.15.103:0/634588021 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3c2c031430 con 0x5565322c0410 2024-09-19T06:50:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.643+0000 7f3c4165f740 10 client.15666 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.643+0000 7f3c4165f740 10 client.15666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.643+0000 7f3c4165f740 20 client.15666 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.643+0000 7f3c4165f740 1 --2- 172.21.15.103:0/634588021 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565322cacb0 0x556532347b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.643+0000 7f3c4165f740 4 client.15666 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:50:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.643+0000 7f3c4165f740 1 -- 172.21.15.103:0/634588021 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55653222bd70 con 0x5565322cacb0 2024-09-19T06:50:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.643+0000 7f3c37fff640 1 --2- 172.21.15.103:0/634588021 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565322cacb0 0x556532347b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.643+0000 7f3c37fff640 1 --2- 172.21.15.103:0/634588021 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565322cacb0 0x556532347b90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e4d4bfd9072957 server_cookie=fb85aeb11d0d27f2 in_seq=0 out_seq=0 2024-09-19T06:50:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.643+0000 7f3c177fe640 10 client.15666 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.757+0000 7f3c177fe640 1 -- 172.21.15.103:0/634588021 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55653222bd70 con 0x5565322cacb0 2024-09-19T06:50:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.757+0000 7f3c177fe640 10 client.15666 handle_command_reply: tid=0 2024-09-19T06:50:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.757+0000 7f3c4165f740 1 client.15666 shutdown 2024-09-19T06:50:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.758+0000 7f3c4165f740 1 -- 172.21.15.103:0/634588021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 msgr2=0x5565322cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.758+0000 7f3c4165f740 1 --2- 172.21.15.103:0/634588021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322cbed0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c005bd0 tx=0x7f3c2c011930 comp rx=0 tx=0).stop 2024-09-19T06:50:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.758+0000 7f3c4165f740 1 -- 172.21.15.103:0/634588021 shutdown_connections 2024-09-19T06:50:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.758+0000 7f3c4165f740 1 --2- 172.21.15.103:0/634588021 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565322cacb0 0x556532347b90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.758+0000 7f3c4165f740 1 --2- 172.21.15.103:0/634588021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565322c0410 0x5565322cbed0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.758+0000 7f3c4165f740 1 --2- 172.21.15.103:0/634588021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565322bfaf0 0x5565322cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.758+0000 7f3c4165f740 1 --2- 172.21.15.103:0/634588021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565322bc060 0x5565322d32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.758+0000 7f3c177fe640 0 client.15666 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.758+0000 7f3c4165f740 1 -- 172.21.15.103:0/634588021 >> 172.21.15.103:0/634588021 conn(0x5565322bcae0 msgr2=0x5565322ba9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.758+0000 7f3c4165f740 1 -- 172.21.15.103:0/634588021 shutdown_connections 2024-09-19T06:50:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.758+0000 7f3c4165f740 1 -- 172.21.15.103:0/634588021 wait complete. 2024-09-19T06:50:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.758+0000 7f3c4165f740 20 client.15666 trim_cache size 0 max 16384 2024-09-19T06:50:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:50:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:50:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:50:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:50:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.758+0000 7f3c3e8c9640 1 -- 172.21.15.103:0/3601317549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c200a3690 msgr2=0x7f3c200a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.758+0000 7f3c3e8c9640 1 --2- 172.21.15.103:0/3601317549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c200a3690 0x7f3c200a5b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3c300cd6e0 tx=0x7f3c3006df90 comp rx=0 tx=0).stop 2024-09-19T06:50:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.758+0000 7f3c3e8c9640 1 -- 172.21.15.103:0/3601317549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c381069c0 msgr2=0x7f3c381a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.758+0000 7f3c3e8c9640 1 --2- 172.21.15.103:0/3601317549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c381069c0 0x7f3c381a4270 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000eaa0 tx=0x7f3c3000ef60 comp rx=0 tx=0).stop 2024-09-19T06:50:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.759+0000 7f3c3e8c9640 1 -- 172.21.15.103:0/3601317549 shutdown_connections 2024-09-19T06:50:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.759+0000 7f3c3e8c9640 1 --2- 172.21.15.103:0/3601317549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c200a3690 0x7f3c200a5b50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.759+0000 7f3c3e8c9640 1 --2- 172.21.15.103:0/3601317549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c381069c0 0x7f3c381a4270 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.759+0000 7f3c3e8c9640 1 --2- 172.21.15.103:0/3601317549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c38101350 0x7f3c38199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.759+0000 7f3c3e8c9640 1 --2- 172.21.15.103:0/3601317549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c381009b0 0x7f3c381991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.759+0000 7f3c3e8c9640 1 -- 172.21.15.103:0/3601317549 >> 172.21.15.103:0/3601317549 conn(0x7f3c38077990 msgr2=0x7f3c38103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.759+0000 7f3c3e8c9640 1 -- 172.21.15.103:0/3601317549 shutdown_connections 2024-09-19T06:50:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:04.759+0000 7f3c3e8c9640 1 -- 172.21.15.103:0/3601317549 wait complete. 2024-09-19T06:50:05.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.118+0000 7f16c94f6640 1 -- 172.21.15.103:0/1858595752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16c4102570 msgr2=0x7f16c41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:05.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.118+0000 7f16c94f6640 1 --2- 172.21.15.103:0/1858595752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16c4102570 0x7f16c41029b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009900 tx=0x7f16b802a480 comp rx=0 tx=0).stop 2024-09-19T06:50:05.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.118+0000 7f16c94f6640 1 -- 172.21.15.103:0/1858595752 shutdown_connections 2024-09-19T06:50:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.118+0000 7f16c94f6640 1 --2- 172.21.15.103:0/1858595752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16c4102ef0 0x7f16c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.118+0000 7f16c94f6640 1 --2- 172.21.15.103:0/1858595752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16c4102570 0x7f16c41029b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.118+0000 7f16c94f6640 1 --2- 172.21.15.103:0/1858595752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16c4100e20 0x7f16c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.118+0000 7f16c94f6640 1 -- 172.21.15.103:0/1858595752 >> 172.21.15.103:0/1858595752 conn(0x7f16c40fc7d0 msgr2=0x7f16c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.118+0000 7f16c94f6640 1 -- 172.21.15.103:0/1858595752 shutdown_connections 2024-09-19T06:50:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.118+0000 7f16c94f6640 1 -- 172.21.15.103:0/1858595752 wait complete. 2024-09-19T06:50:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.119+0000 7f16c94f6640 1 Processor -- start 2024-09-19T06:50:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.119+0000 7f16c94f6640 1 -- start start 2024-09-19T06:50:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.119+0000 7f16c94f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16c4100e20 0x7f16c40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.119+0000 7f16c94f6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16c4102570 0x7f16c40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.119+0000 7f16c94f6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16c4102ef0 0x7f16c40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.119+0000 7f16c94f6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4110fa0 con 0x7f16c4100e20 2024-09-19T06:50:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.119+0000 7f16c94f6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4110e20 con 0x7f16c4102ef0 2024-09-19T06:50:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.119+0000 7f16c94f6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4111120 con 0x7f16c4102570 2024-09-19T06:50:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.120+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16c4100e20 0x7f16c40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.120+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16c4100e20 0x7f16c40fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47896/0 (socket says 172.21.15.103:47896) 2024-09-19T06:50:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.120+0000 7f16c2ffd640 1 -- 172.21.15.103:0/1093218272 learned_addr learned my addr 172.21.15.103:0/1093218272 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.120+0000 7f16c37fe640 1 --2- 172.21.15.103:0/1093218272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16c4102ef0 0x7f16c40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.120+0000 7f16c27fc640 1 --2- 172.21.15.103:0/1093218272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16c4102570 0x7f16c40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.120+0000 7f16c2ffd640 1 -- 172.21.15.103:0/1093218272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16c4102570 msgr2=0x7f16c40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.120+0000 7f16c2ffd640 1 --2- 172.21.15.103:0/1093218272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16c4102570 0x7f16c40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.120+0000 7f16c2ffd640 1 -- 172.21.15.103:0/1093218272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16c4102ef0 msgr2=0x7f16c40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.120+0000 7f16c2ffd640 1 --2- 172.21.15.103:0/1093218272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16c4102ef0 0x7f16c40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.120+0000 7f16c2ffd640 1 -- 172.21.15.103:0/1093218272 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c41a6a20 con 0x7f16c4100e20 2024-09-19T06:50:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.120+0000 7f16c2ffd640 1 --2- 172.21.15.103:0/1093218272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16c4100e20 0x7f16c40fcc80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f16b400d950 tx=0x7f16b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.120+0000 7f16abfff640 1 -- 172.21.15.103:0/1093218272 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4017070 con 0x7f16c4100e20 2024-09-19T06:50:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.120+0000 7f16c94f6640 1 -- 172.21.15.103:0/1093218272 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c41a6d10 con 0x7f16c4100e20 2024-09-19T06:50:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.120+0000 7f16abfff640 1 -- 172.21.15.103:0/1093218272 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16b4009c30 con 0x7f16c4100e20 2024-09-19T06:50:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.120+0000 7f16abfff640 1 -- 172.21.15.103:0/1093218272 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b400f6b0 con 0x7f16c4100e20 2024-09-19T06:50:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.121+0000 7f16c94f6640 1 -- 172.21.15.103:0/1093218272 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c41a7240 con 0x7f16c4100e20 2024-09-19T06:50:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.122+0000 7f16c94f6640 1 -- 172.21.15.103:0/1093218272 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16c4101200 con 0x7f16c4100e20 2024-09-19T06:50:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.123+0000 7f16abfff640 1 -- 172.21.15.103:0/1093218272 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7f16b40109d0 con 0x7f16c4100e20 2024-09-19T06:50:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.123+0000 7f16abfff640 1 --2- 172.21.15.103:0/1093218272 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f16ac0a35e0 0x7f16ac0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.124+0000 7f16c27fc640 1 --2- 172.21.15.103:0/1093218272 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f16ac0a35e0 0x7f16ac0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.124+0000 7f16abfff640 1 -- 172.21.15.103:0/1093218272 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f16b40cfb40 con 0x7f16c4100e20 2024-09-19T06:50:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.124+0000 7f16c27fc640 1 --2- 172.21.15.103:0/1093218272 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f16ac0a35e0 0x7f16ac0a5aa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16c40fe710 tx=0x7f16b8005a20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.131+0000 7f16abfff640 1 -- 172.21.15.103:0/1093218272 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f16b4009de0 con 0x7f16c4100e20 2024-09-19T06:50:05.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.373+0000 7f16c94f6640 1 -- 172.21.15.103:0/1093218272 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16c410a0d0 con 0x7f16c4100e20 2024-09-19T06:50:05.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.374+0000 7f16abfff640 1 -- 172.21.15.103:0/1093218272 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f16b409c370 con 0x7f16c4100e20 2024-09-19T06:50:05.375 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:50:05.375 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:50:05.376 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:50:05.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.377+0000 7f16c94f6640 1 -- 172.21.15.103:0/1093218272 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f16ac0a35e0 msgr2=0x7f16ac0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:05.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.377+0000 7f16c94f6640 1 --2- 172.21.15.103:0/1093218272 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f16ac0a35e0 0x7f16ac0a5aa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16c40fe710 tx=0x7f16b8005a20 comp rx=0 tx=0).stop 2024-09-19T06:50:05.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.377+0000 7f16c94f6640 1 -- 172.21.15.103:0/1093218272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16c4100e20 msgr2=0x7f16c40fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:05.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.377+0000 7f16c94f6640 1 --2- 172.21.15.103:0/1093218272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16c4100e20 0x7f16c40fcc80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f16b400d950 tx=0x7f16b400de10 comp rx=0 tx=0).stop 2024-09-19T06:50:05.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.377+0000 7f16c94f6640 1 -- 172.21.15.103:0/1093218272 shutdown_connections 2024-09-19T06:50:05.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.377+0000 7f16c94f6640 1 --2- 172.21.15.103:0/1093218272 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f16ac0a35e0 0x7f16ac0a5aa0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:05.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.377+0000 7f16c94f6640 1 --2- 172.21.15.103:0/1093218272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16c4102ef0 0x7f16c40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:05.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.377+0000 7f16c94f6640 1 --2- 172.21.15.103:0/1093218272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16c4102570 0x7f16c40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:05.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.377+0000 7f16c94f6640 1 --2- 172.21.15.103:0/1093218272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16c4100e20 0x7f16c40fcc80 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:05.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.377+0000 7f16c94f6640 1 -- 172.21.15.103:0/1093218272 >> 172.21.15.103:0/1093218272 conn(0x7f16c40fc7d0 msgr2=0x7f16c406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:05.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.377+0000 7f16c94f6640 1 -- 172.21.15.103:0/1093218272 shutdown_connections 2024-09-19T06:50:05.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:05.377+0000 7f16c94f6640 1 -- 172.21.15.103:0/1093218272 wait complete. 2024-09-19T06:50:05.601 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:50:05.626 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:05 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1093218272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:05.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:05 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1093218272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:05.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:05 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1093218272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:06.262 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0' 2024-09-19T06:50:06.262 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0 2024-09-19T06:50:06.308 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:06.561 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:06 smithi103 ceph-mon[31158]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-09-19T06:50:06.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:06 smithi134 ceph-mon[35356]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-09-19T06:50:06.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:06 smithi174 ceph-mon[35637]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-09-19T06:50:06.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.849+0000 7fc32cead640 1 -- 172.21.15.103:0/2234402158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc32810d520 msgr2=0x7fc3281150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.849+0000 7fc32cead640 1 --2- 172.21.15.103:0/2234402158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc32810d520 0x7fc3281150a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc320009ca0 tx=0x7fc32002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.849+0000 7fc32cead640 1 -- 172.21.15.103:0/2234402158 shutdown_connections 2024-09-19T06:50:06.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.849+0000 7fc32cead640 1 --2- 172.21.15.103:0/2234402158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc32810d520 0x7fc3281150a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.849+0000 7fc32cead640 1 --2- 172.21.15.103:0/2234402158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc328108da0 0x7fc32810cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.849+0000 7fc32cead640 1 --2- 172.21.15.103:0/2234402158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3281083f0 0x7fc3281087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.849+0000 7fc32cead640 1 -- 172.21.15.103:0/2234402158 >> 172.21.15.103:0/2234402158 conn(0x7fc328077af0 msgr2=0x7fc328079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:06.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.849+0000 7fc32cead640 1 -- 172.21.15.103:0/2234402158 shutdown_connections 2024-09-19T06:50:06.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.849+0000 7fc32cead640 1 -- 172.21.15.103:0/2234402158 wait complete. 2024-09-19T06:50:06.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.850+0000 7fc32cead640 1 Processor -- start 2024-09-19T06:50:06.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.850+0000 7fc32cead640 1 -- start start 2024-09-19T06:50:06.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.850+0000 7fc32cead640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3281083f0 0x7fc328082610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.850+0000 7fc32cead640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328108da0 0x7fc328082b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.850+0000 7fc32cead640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc32810d520 0x7fc32807b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.851+0000 7fc32cead640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3281185a0 con 0x7fc328108da0 2024-09-19T06:50:06.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.851+0000 7fc32cead640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc328118420 con 0x7fc3281083f0 2024-09-19T06:50:06.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.851+0000 7fc32659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3281083f0 0x7fc328082610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.851+0000 7fc32659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3281083f0 0x7fc328082610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:55420/0 (socket says 172.21.15.103:55420) 2024-09-19T06:50:06.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.851+0000 7fc32659c640 1 -- 172.21.15.103:0/2667285457 learned_addr learned my addr 172.21.15.103:0/2667285457 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:06.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.851+0000 7fc32cead640 1 -- 172.21.15.103:0/2667285457 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc328118720 con 0x7fc32810d520 2024-09-19T06:50:06.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.851+0000 7fc325d9b640 1 --2- 172.21.15.103:0/2667285457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328108da0 0x7fc328082b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.851+0000 7fc326d9d640 1 --2- 172.21.15.103:0/2667285457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc32810d520 0x7fc32807b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.851+0000 7fc32659c640 1 -- 172.21.15.103:0/2667285457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc32810d520 msgr2=0x7fc32807b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.851+0000 7fc32659c640 1 --2- 172.21.15.103:0/2667285457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc32810d520 0x7fc32807b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.851+0000 7fc32659c640 1 -- 172.21.15.103:0/2667285457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328108da0 msgr2=0x7fc328082b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.851+0000 7fc32659c640 1 --2- 172.21.15.103:0/2667285457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328108da0 0x7fc328082b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.851+0000 7fc32659c640 1 -- 172.21.15.103:0/2667285457 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32807bf80 con 0x7fc3281083f0 2024-09-19T06:50:06.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.851+0000 7fc326d9d640 1 --2- 172.21.15.103:0/2667285457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc32810d520 0x7fc32807b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:06.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.852+0000 7fc32659c640 1 --2- 172.21.15.103:0/2667285457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3281083f0 0x7fc328082610 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc31800e9f0 tx=0x7fc31800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:06.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.852+0000 7fc3177fe640 1 -- 172.21.15.103:0/2667285457 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31800cd80 con 0x7fc3281083f0 2024-09-19T06:50:06.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.852+0000 7fc32cead640 1 -- 172.21.15.103:0/2667285457 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32807c270 con 0x7fc3281083f0 2024-09-19T06:50:06.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.852+0000 7fc3177fe640 1 -- 172.21.15.103:0/2667285457 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc318010910 con 0x7fc3281083f0 2024-09-19T06:50:06.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.852+0000 7fc3177fe640 1 -- 172.21.15.103:0/2667285457 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3180188f0 con 0x7fc3281083f0 2024-09-19T06:50:06.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.852+0000 7fc32cead640 1 -- 172.21.15.103:0/2667285457 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc328080310 con 0x7fc3281083f0 2024-09-19T06:50:06.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.855+0000 7fc3177fe640 1 -- 172.21.15.103:0/2667285457 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7fc318010ac0 con 0x7fc3281083f0 2024-09-19T06:50:06.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.855+0000 7fc3177fe640 1 --2- 172.21.15.103:0/2667285457 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3100a35e0 0x7fc3100a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.855+0000 7fc325d9b640 1 --2- 172.21.15.103:0/2667285457 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3100a35e0 0x7fc3100a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.856+0000 7fc3177fe640 1 -- 172.21.15.103:0/2667285457 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc3180d03a0 con 0x7fc3281083f0 2024-09-19T06:50:06.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.856+0000 7fc32f286740 1 Processor -- start 2024-09-19T06:50:06.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.856+0000 7fc32f286740 1 -- start start 2024-09-19T06:50:06.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.856+0000 7fc325d9b640 1 --2- 172.21.15.103:0/2667285457 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3100a35e0 0x7fc3100a5aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0095f0 tx=0x7fc31c005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:06.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.856+0000 7fc32f286740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 0x55a05f13c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.856+0000 7fc32f286740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f13c5d0 0x55a05f13ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.856+0000 7fc32f286740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f147370 0x55a05f149760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.856+0000 7fc32f286740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a05f0a22f0 con 0x55a05f13db40 2024-09-19T06:50:06.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.856+0000 7fc32f286740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a05f049cf0 con 0x55a05f147370 2024-09-19T06:50:06.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.856+0000 7fc32f286740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a05f09e2b0 con 0x55a05f13c5d0 2024-09-19T06:50:06.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.857+0000 7fc326d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f147370 0x55a05f149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.857+0000 7fc325d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f13c5d0 0x55a05f13ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.857+0000 7fc32659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 0x55a05f13c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.857+0000 7fc326d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f147370 0x55a05f149760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:55432/0 (socket says 172.21.15.103:55432) 2024-09-19T06:50:06.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.857+0000 7fc325d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f13c5d0 0x55a05f13ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:48280/0 (socket says 172.21.15.103:48280) 2024-09-19T06:50:06.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.857+0000 7fc326d9d640 1 -- 172.21.15.103:0/665947720 learned_addr learned my addr 172.21.15.103:0/665947720 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:06.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.857+0000 7fc32659c640 1 -- 172.21.15.103:0/665947720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f13c5d0 msgr2=0x55a05f13ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:06.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.857+0000 7fc32659c640 1 --2- 172.21.15.103:0/665947720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f13c5d0 0x55a05f13ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.857+0000 7fc32659c640 1 -- 172.21.15.103:0/665947720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f147370 msgr2=0x55a05f149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:06.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.857+0000 7fc32659c640 1 --2- 172.21.15.103:0/665947720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f147370 0x55a05f149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.857+0000 7fc32659c640 1 -- 172.21.15.103:0/665947720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05f02e3a0 con 0x55a05f13db40 2024-09-19T06:50:06.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.858+0000 7fc32659c640 1 --2- 172.21.15.103:0/665947720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 0x55a05f13c090 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc3180cd8c0 tx=0x7fc31809cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80604061dcc6cfe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:06.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.858+0000 7fc314ff9640 1 -- 172.21.15.103:0/665947720 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3180a50c0 con 0x55a05f13db40 2024-09-19T06:50:06.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.858+0000 7fc314ff9640 1 -- 172.21.15.103:0/665947720 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc318098ac0 con 0x55a05f13db40 2024-09-19T06:50:06.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.859+0000 7fc32f286740 1 -- 172.21.15.103:0/665947720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 msgr2=0x55a05f13c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.859+0000 7fc32f286740 1 --2- 172.21.15.103:0/665947720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 0x55a05f13c090 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc3180cd8c0 tx=0x7fc31809cc10 comp rx=0 tx=0).stop 2024-09-19T06:50:06.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.859+0000 7fc32f286740 1 -- 172.21.15.103:0/665947720 shutdown_connections 2024-09-19T06:50:06.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.859+0000 7fc32f286740 1 --2- 172.21.15.103:0/665947720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f147370 0x55a05f149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.859+0000 7fc32f286740 1 --2- 172.21.15.103:0/665947720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f13c5d0 0x55a05f13ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.859+0000 7fc32f286740 1 --2- 172.21.15.103:0/665947720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 0x55a05f13c090 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.859+0000 7fc32f286740 1 -- 172.21.15.103:0/665947720 >> 172.21.15.103:0/665947720 conn(0x55a05f105310 msgr2=0x55a05f138c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:06.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.860+0000 7fc32f286740 1 -- 172.21.15.103:0/665947720 shutdown_connections 2024-09-19T06:50:06.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.860+0000 7fc32f286740 1 -- 172.21.15.103:0/665947720 wait complete. 2024-09-19T06:50:06.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.860+0000 7fc32f286740 1 -- start start 2024-09-19T06:50:06.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.860+0000 7fc32f286740 1 Processor -- start 2024-09-19T06:50:06.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.860+0000 7fc32f286740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f13c5d0 0x55a05f146d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.861+0000 7fc32f286740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 0x55a05f143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.861+0000 7fc32f286740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f147370 0x55a05f144570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.861+0000 7fc32f286740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a05f14a0b0 con 0x55a05f13db40 2024-09-19T06:50:06.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.861+0000 7fc32f286740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a05f05d060 con 0x55a05f13c5d0 2024-09-19T06:50:06.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.861+0000 7fc32f286740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a05f14a230 con 0x55a05f147370 2024-09-19T06:50:06.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.861+0000 7fc325d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 0x55a05f143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.861+0000 7fc32f286740 10 client.0 _ll_register_callbacks cb 0x55a05edb08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:06.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.861+0000 7fc325d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 0x55a05f143fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47956/0 (socket says 172.21.15.103:47956) 2024-09-19T06:50:06.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.861+0000 7fc325d9b640 1 -- 172.21.15.103:0/2151812633 learned_addr learned my addr 172.21.15.103:0/2151812633 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:06.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.861+0000 7fc326d9d640 1 --2- 172.21.15.103:0/2151812633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f147370 0x55a05f144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.861+0000 7fc32659c640 1 --2- 172.21.15.103:0/2151812633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f13c5d0 0x55a05f146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.862+0000 7fc325d9b640 1 -- 172.21.15.103:0/2151812633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f147370 msgr2=0x55a05f144570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:06.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.862+0000 7fc325d9b640 1 --2- 172.21.15.103:0/2151812633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f147370 0x55a05f144570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.862+0000 7fc325d9b640 1 -- 172.21.15.103:0/2151812633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f13c5d0 msgr2=0x55a05f146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:06.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.862+0000 7fc325d9b640 1 --2- 172.21.15.103:0/2151812633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f13c5d0 0x55a05f146d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.862+0000 7fc325d9b640 1 -- 172.21.15.103:0/2151812633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a05ee133c0 con 0x55a05f13db40 2024-09-19T06:50:06.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.862+0000 7fc32659c640 1 --2- 172.21.15.103:0/2151812633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f13c5d0 0x55a05f146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:06.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.862+0000 7fc325d9b640 1 --2- 172.21.15.103:0/2151812633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 0x55a05f143fd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0045f0 tx=0x7fc31c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9899177573445089 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:06.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.862+0000 7fc30dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:06.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.862+0000 7fc30dffb640 1 -- 172.21.15.103:0/2151812633 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c01f900 con 0x55a05f13db40 2024-09-19T06:50:06.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.862+0000 7fc30dffb640 1 -- 172.21.15.103:0/2151812633 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc31c01faa0 con 0x55a05f13db40 2024-09-19T06:50:06.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.863+0000 7fc32f286740 1 -- 172.21.15.103:0/2151812633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a05f0af860 con 0x55a05f13db40 2024-09-19T06:50:06.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.863+0000 7fc30dffb640 1 -- 172.21.15.103:0/2151812633 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc31c01fca0 con 0x55a05f13db40 2024-09-19T06:50:06.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.863+0000 7fc32f286740 10 client.15684 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:06.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.863+0000 7fc32f286740 1 -- 172.21.15.103:0/2151812633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a05f155c90 con 0x55a05f13db40 2024-09-19T06:50:06.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.864+0000 7fc30dffb640 1 -- 172.21.15.103:0/2151812633 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc31c01fe40 con 0x55a05f13db40 2024-09-19T06:50:06.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.864+0000 7fc30dffb640 1 -- 172.21.15.103:0/2151812633 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc31c026070 con 0x55a05f13db40 2024-09-19T06:50:06.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.864+0000 7fc32f286740 10 client.15684 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:06.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.864+0000 7fc32f286740 10 client.15684 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:50:06.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.864+0000 7fc32f286740 20 client.15684 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:06.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.864+0000 7fc32f286740 1 --2- 172.21.15.103:0/2151812633 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55a05f1570f0 0x55a05f159510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.864+0000 7fc32f286740 4 client.15684 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:06.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.864+0000 7fc32f286740 1 -- 172.21.15.103:0/2151812633 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a05f0af860 con 0x55a05f1570f0 2024-09-19T06:50:06.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.864+0000 7fc32659c640 1 --2- 172.21.15.103:0/2151812633 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55a05f1570f0 0x55a05f159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.865+0000 7fc32659c640 1 --2- 172.21.15.103:0/2151812633 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55a05f1570f0 0x55a05f159510 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec823ee683d9e6dd server_cookie=8e592b964716fa90 in_seq=0 out_seq=0 2024-09-19T06:50:06.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.865+0000 7fc30dffb640 10 client.15684 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:50:06.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.866+0000 7fc30dffb640 1 -- 172.21.15.103:0/2151812633 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a05f0af860 con 0x55a05f1570f0 2024-09-19T06:50:06.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.866+0000 7fc30dffb640 10 client.15684 handle_command_reply: tid=0 2024-09-19T06:50:06.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.866+0000 7fc32f286740 1 client.15684 shutdown 2024-09-19T06:50:06.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.866+0000 7fc32f286740 1 -- 172.21.15.103:0/2151812633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 msgr2=0x55a05f143fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.866+0000 7fc32f286740 1 --2- 172.21.15.103:0/2151812633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 0x55a05f143fd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0045f0 tx=0x7fc31c004730 comp rx=0 tx=0).stop 2024-09-19T06:50:06.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.867+0000 7fc32f286740 1 -- 172.21.15.103:0/2151812633 shutdown_connections 2024-09-19T06:50:06.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.867+0000 7fc32f286740 1 --2- 172.21.15.103:0/2151812633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f147370 0x55a05f144570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.867+0000 7fc32f286740 1 --2- 172.21.15.103:0/2151812633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 0x55a05f143fd0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.867+0000 7fc32f286740 1 --2- 172.21.15.103:0/2151812633 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55a05f1570f0 0x55a05f159510 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.867+0000 7fc32f286740 1 --2- 172.21.15.103:0/2151812633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f13c5d0 0x55a05f146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.867+0000 7fc32f286740 1 -- 172.21.15.103:0/2151812633 >> 172.21.15.103:0/2151812633 conn(0x55a05f105310 msgr2=0x55a05f13e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:06.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.867+0000 7fc30dffb640 0 client.15684 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:50:06.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.867+0000 7fc32f286740 1 -- 172.21.15.103:0/2151812633 shutdown_connections 2024-09-19T06:50:06.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.867+0000 7fc32f286740 1 -- 172.21.15.103:0/2151812633 wait complete. 2024-09-19T06:50:06.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.867+0000 7fc32f286740 20 client.15684 trim_cache size 0 max 16384 2024-09-19T06:50:06.869 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.892+0000 7fc32f286740 1 Processor -- start 2024-09-19T06:50:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.892+0000 7fc32f286740 1 -- start start 2024-09-19T06:50:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.892+0000 7fc32f286740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f13c5d0 0x55a05f1533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.892+0000 7fc32f286740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 0x55a05f143890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.892+0000 7fc32f286740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f147370 0x55a05f13f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.892+0000 7fc32f286740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3281ab810 con 0x55a05f13db40 2024-09-19T06:50:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.892+0000 7fc32f286740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3281ad350 con 0x55a05f147370 2024-09-19T06:50:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.892+0000 7fc32f286740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3281abee0 con 0x55a05f13c5d0 2024-09-19T06:50:06.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.892+0000 7fc325d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 0x55a05f143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.892+0000 7fc325d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 0x55a05f143890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47968/0 (socket says 172.21.15.103:47968) 2024-09-19T06:50:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.892+0000 7fc325d9b640 1 -- 172.21.15.103:0/1276661471 learned_addr learned my addr 172.21.15.103:0/1276661471 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.892+0000 7fc326d9d640 1 --2- 172.21.15.103:0/1276661471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f147370 0x55a05f13f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.892+0000 7fc32659c640 1 --2- 172.21.15.103:0/1276661471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f13c5d0 0x55a05f1533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.893+0000 7fc326d9d640 1 -- 172.21.15.103:0/1276661471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f13c5d0 msgr2=0x55a05f1533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.893+0000 7fc326d9d640 1 --2- 172.21.15.103:0/1276661471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f13c5d0 0x55a05f1533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.893+0000 7fc326d9d640 1 -- 172.21.15.103:0/1276661471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 msgr2=0x55a05f143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.893+0000 7fc326d9d640 1 --2- 172.21.15.103:0/1276661471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 0x55a05f143890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.893+0000 7fc326d9d640 1 -- 172.21.15.103:0/1276661471 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05f147090 con 0x55a05f147370 2024-09-19T06:50:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.893+0000 7fc32659c640 1 --2- 172.21.15.103:0/1276661471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f13c5d0 0x55a05f1533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.893+0000 7fc326d9d640 1 --2- 172.21.15.103:0/1276661471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f147370 0x55a05f13f120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc320009ca0 tx=0x7fc32002f450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5443dbcb9518085b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.894+0000 7fc30ffff640 1 -- 172.21.15.103:0/1276661471 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320003970 con 0x55a05f147370 2024-09-19T06:50:06.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.894+0000 7fc30ffff640 1 -- 172.21.15.103:0/1276661471 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc32004b070 con 0x55a05f147370 2024-09-19T06:50:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.894+0000 7fc30ffff640 1 -- 172.21.15.103:0/1276661471 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320004330 con 0x55a05f147370 2024-09-19T06:50:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.895+0000 7fc32f286740 1 -- 172.21.15.103:0/1276661471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f147370 msgr2=0x55a05f13f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.895+0000 7fc32f286740 1 --2- 172.21.15.103:0/1276661471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f147370 0x55a05f13f120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc320009ca0 tx=0x7fc32002f450 comp rx=0 tx=0).stop 2024-09-19T06:50:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.895+0000 7fc32f286740 1 -- 172.21.15.103:0/1276661471 shutdown_connections 2024-09-19T06:50:06.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.895+0000 7fc32f286740 1 --2- 172.21.15.103:0/1276661471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f147370 0x55a05f13f120 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.895+0000 7fc32f286740 1 --2- 172.21.15.103:0/1276661471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13db40 0x55a05f143890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.895+0000 7fc32f286740 1 --2- 172.21.15.103:0/1276661471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f13c5d0 0x55a05f1533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.895+0000 7fc32f286740 1 -- 172.21.15.103:0/1276661471 >> 172.21.15.103:0/1276661471 conn(0x55a05f105310 msgr2=0x55a05f1428a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.895+0000 7fc32f286740 1 -- 172.21.15.103:0/1276661471 shutdown_connections 2024-09-19T06:50:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.895+0000 7fc32f286740 1 -- 172.21.15.103:0/1276661471 wait complete. 2024-09-19T06:50:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.896+0000 7fc32f286740 1 -- start start 2024-09-19T06:50:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.896+0000 7fc32f286740 1 Processor -- start 2024-09-19T06:50:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.897+0000 7fc32f286740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13c5d0 0x55a05f14fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.897+0000 7fc32f286740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f13db40 0x55a05f14a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.897+0000 7fc32f286740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f147370 0x55a05f14ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.897+0000 7fc32f286740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3281acc80 con 0x55a05f13c5d0 2024-09-19T06:50:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.897+0000 7fc32f286740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3281ac5b0 con 0x55a05f13db40 2024-09-19T06:50:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.897+0000 7fc32659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13c5d0 0x55a05f14fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.897+0000 7fc325d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f13db40 0x55a05f14a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.897+0000 7fc325d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f13db40 0x55a05f14a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:55464/0 (socket says 172.21.15.103:55464) 2024-09-19T06:50:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.897+0000 7fc325d9b640 1 -- 172.21.15.103:0/3905462653 learned_addr learned my addr 172.21.15.103:0/3905462653 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.897+0000 7fc32f286740 1 -- 172.21.15.103:0/3905462653 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3281a97d0 con 0x55a05f147370 2024-09-19T06:50:06.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.897+0000 7fc326d9d640 1 --2- 172.21.15.103:0/3905462653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f147370 0x55a05f14ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.898+0000 7fc32659c640 1 -- 172.21.15.103:0/3905462653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f147370 msgr2=0x55a05f14ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.898+0000 7fc32659c640 1 --2- 172.21.15.103:0/3905462653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f147370 0x55a05f14ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.898+0000 7fc32659c640 1 -- 172.21.15.103:0/3905462653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f13db40 msgr2=0x55a05f14a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.898+0000 7fc32659c640 1 --2- 172.21.15.103:0/3905462653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f13db40 0x55a05f14a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.898+0000 7fc32659c640 1 -- 172.21.15.103:0/3905462653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a05f1bfda0 con 0x55a05f13c5d0 2024-09-19T06:50:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.898+0000 7fc32f286740 10 client.0 _ll_register_callbacks cb 0x7fc31c0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.898+0000 7fc32f286740 1 -- 172.21.15.103:0/3905462653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc32004b070 con 0x55a05f13c5d0 2024-09-19T06:50:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.898+0000 7fc32659c640 1 --2- 172.21.15.103:0/3905462653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13c5d0 0x55a05f14fe00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc3180a09c0 tx=0x7fc31800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f250a49b4f8ef224 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.899+0000 7fc30dffb640 10 client.15690 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.899+0000 7fc30dffb640 1 -- 172.21.15.103:0/3905462653 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3180a50c0 con 0x55a05f13c5d0 2024-09-19T06:50:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.899+0000 7fc30dffb640 1 -- 172.21.15.103:0/3905462653 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc3180b5240 con 0x55a05f13c5d0 2024-09-19T06:50:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.899+0000 7fc30dffb640 1 -- 172.21.15.103:0/3905462653 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc3180b5440 con 0x55a05f13c5d0 2024-09-19T06:50:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.899+0000 7fc32f286740 10 client.15690 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:06.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.899+0000 7fc32f286740 1 -- 172.21.15.103:0/3905462653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55a05f1c0d80 con 0x55a05f13c5d0 2024-09-19T06:50:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.899+0000 7fc30dffb640 1 -- 172.21.15.103:0/3905462653 <== mon.0 v2:172.21.15.103:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc3180b03a0 con 0x55a05f13c5d0 2024-09-19T06:50:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.900+0000 7fc32f286740 10 client.15690 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.900+0000 7fc32f286740 10 client.15690 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:50:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.900+0000 7fc32f286740 20 client.15690 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.900+0000 7fc32f286740 1 --2- 172.21.15.103:0/3905462653 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55a05f1c1050 0x55a05f1c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.900+0000 7fc32f286740 4 client.15690 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.900+0000 7fc32f286740 1 -- 172.21.15.103:0/3905462653 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a05f0a9d70 con 0x55a05f1c1050 2024-09-19T06:50:06.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.901+0000 7fc325d9b640 1 --2- 172.21.15.103:0/3905462653 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55a05f1c1050 0x55a05f1c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.904+0000 7fc30dffb640 10 client.15690 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:50:06.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.904+0000 7fc325d9b640 1 --2- 172.21.15.103:0/3905462653 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55a05f1c1050 0x55a05f1c34e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60e42268bf8f1756 server_cookie=a98691ee5d95f586 in_seq=0 out_seq=0 2024-09-19T06:50:06.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.905+0000 7fc30dffb640 1 -- 172.21.15.103:0/3905462653 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a05f0a9d70 con 0x55a05f1c1050 2024-09-19T06:50:06.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.905+0000 7fc30dffb640 10 client.15690 handle_command_reply: tid=0 2024-09-19T06:50:06.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.905+0000 7fc32f286740 1 client.15690 shutdown 2024-09-19T06:50:06.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.905+0000 7fc32f286740 1 -- 172.21.15.103:0/3905462653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13c5d0 msgr2=0x55a05f14fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.905+0000 7fc32f286740 1 --2- 172.21.15.103:0/3905462653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13c5d0 0x55a05f14fe00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc3180a09c0 tx=0x7fc31800e940 comp rx=0 tx=0).stop 2024-09-19T06:50:06.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.905+0000 7fc32f286740 1 -- 172.21.15.103:0/3905462653 shutdown_connections 2024-09-19T06:50:06.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.905+0000 7fc32f286740 1 --2- 172.21.15.103:0/3905462653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a05f147370 0x55a05f14ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.905+0000 7fc32f286740 1 --2- 172.21.15.103:0/3905462653 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55a05f1c1050 0x55a05f1c34e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.905+0000 7fc30dffb640 0 client.15690 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:50:06.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.905+0000 7fc32f286740 1 --2- 172.21.15.103:0/3905462653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a05f13db40 0x55a05f14a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.905+0000 7fc32f286740 1 --2- 172.21.15.103:0/3905462653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a05f13c5d0 0x55a05f14fe00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.905+0000 7fc32f286740 1 -- 172.21.15.103:0/3905462653 >> 172.21.15.103:0/3905462653 conn(0x55a05f105310 msgr2=0x55a05f142860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:06.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.905+0000 7fc32f286740 1 -- 172.21.15.103:0/3905462653 shutdown_connections 2024-09-19T06:50:06.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.905+0000 7fc32f286740 1 -- 172.21.15.103:0/3905462653 wait complete. 2024-09-19T06:50:06.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.906+0000 7fc32f286740 20 client.15690 trim_cache size 0 max 16384 2024-09-19T06:50:06.909 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:50:06.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.906+0000 7fc32cead640 1 -- 172.21.15.103:0/2667285457 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3100a35e0 msgr2=0x7fc3100a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.906+0000 7fc32cead640 1 --2- 172.21.15.103:0/2667285457 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3100a35e0 0x7fc3100a5aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0095f0 tx=0x7fc31c005f90 comp rx=0 tx=0).stop 2024-09-19T06:50:06.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.906+0000 7fc32cead640 1 -- 172.21.15.103:0/2667285457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3281083f0 msgr2=0x7fc328082610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.906+0000 7fc32cead640 1 --2- 172.21.15.103:0/2667285457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3281083f0 0x7fc328082610 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc31800e9f0 tx=0x7fc31800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.907+0000 7fc32cead640 1 -- 172.21.15.103:0/2667285457 shutdown_connections 2024-09-19T06:50:06.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.907+0000 7fc32cead640 1 --2- 172.21.15.103:0/2667285457 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3100a35e0 0x7fc3100a5aa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.907+0000 7fc32cead640 1 --2- 172.21.15.103:0/2667285457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc32810d520 0x7fc32807b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.907+0000 7fc32cead640 1 --2- 172.21.15.103:0/2667285457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328108da0 0x7fc328082b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.907+0000 7fc32cead640 1 --2- 172.21.15.103:0/2667285457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3281083f0 0x7fc328082610 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.907+0000 7fc32cead640 1 -- 172.21.15.103:0/2667285457 >> 172.21.15.103:0/2667285457 conn(0x7fc328077af0 msgr2=0x7fc328078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:06.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.907+0000 7fc32cead640 1 -- 172.21.15.103:0/2667285457 shutdown_connections 2024-09-19T06:50:06.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:06.907+0000 7fc32cead640 1 -- 172.21.15.103:0/2667285457 wait complete. 2024-09-19T06:50:07.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.600+0000 7f7d1efb6640 1 -- 172.21.15.103:0/452691119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d18102570 msgr2=0x7f7d181029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.600+0000 7f7d1efb6640 1 --2- 172.21.15.103:0/452691119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d18102570 0x7f7d181029b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009900 tx=0x7f7d0402a480 comp rx=0 tx=0).stop 2024-09-19T06:50:07.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.600+0000 7f7d1efb6640 1 -- 172.21.15.103:0/452691119 shutdown_connections 2024-09-19T06:50:07.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.600+0000 7f7d1efb6640 1 --2- 172.21.15.103:0/452691119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d18102ef0 0x7f7d1810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.600+0000 7f7d1efb6640 1 --2- 172.21.15.103:0/452691119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d18102570 0x7f7d181029b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.600+0000 7f7d1efb6640 1 --2- 172.21.15.103:0/452691119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d18100e20 0x7f7d18101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.600+0000 7f7d1efb6640 1 -- 172.21.15.103:0/452691119 >> 172.21.15.103:0/452691119 conn(0x7f7d180fc7d0 msgr2=0x7f7d180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:07.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.600+0000 7f7d1efb6640 1 -- 172.21.15.103:0/452691119 shutdown_connections 2024-09-19T06:50:07.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.600+0000 7f7d1efb6640 1 -- 172.21.15.103:0/452691119 wait complete. 2024-09-19T06:50:07.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.602+0000 7f7d1efb6640 1 Processor -- start 2024-09-19T06:50:07.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.602+0000 7f7d1efb6640 1 -- start start 2024-09-19T06:50:07.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.602+0000 7f7d1efb6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d18100e20 0x7f7d18196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.602+0000 7f7d1efb6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d18102570 0x7f7d181967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.602+0000 7f7d1efb6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d18102ef0 0x7f7d1819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.602+0000 7f7d1efb6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1810fe80 con 0x7f7d18102570 2024-09-19T06:50:07.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.602+0000 7f7d1efb6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1810fd00 con 0x7f7d18102ef0 2024-09-19T06:50:07.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.602+0000 7f7d1efb6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d18110000 con 0x7f7d18100e20 2024-09-19T06:50:07.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.603+0000 7f7d1d553640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d18102ef0 0x7f7d1819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.603+0000 7f7d1d553640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d18102ef0 0x7f7d1819ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:48000/0 (socket says 172.21.15.103:48000) 2024-09-19T06:50:07.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.603+0000 7f7d1d553640 1 -- 172.21.15.103:0/2204128323 learned_addr learned my addr 172.21.15.103:0/2204128323 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:07.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.603+0000 7f7d1cd52640 1 --2- 172.21.15.103:0/2204128323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d18100e20 0x7f7d18196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.603+0000 7f7d17fff640 1 --2- 172.21.15.103:0/2204128323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d18102570 0x7f7d181967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.603+0000 7f7d1d553640 1 -- 172.21.15.103:0/2204128323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d18100e20 msgr2=0x7f7d18196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.603+0000 7f7d1d553640 1 --2- 172.21.15.103:0/2204128323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d18100e20 0x7f7d18196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.603+0000 7f7d1d553640 1 -- 172.21.15.103:0/2204128323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d18102570 msgr2=0x7f7d181967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.603+0000 7f7d1d553640 1 --2- 172.21.15.103:0/2204128323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d18102570 0x7f7d181967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.603+0000 7f7d1d553640 1 -- 172.21.15.103:0/2204128323 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1819b230 con 0x7f7d18102ef0 2024-09-19T06:50:07.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.603+0000 7f7d1cd52640 1 --2- 172.21.15.103:0/2204128323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d18100e20 0x7f7d18196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:07.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.603+0000 7f7d17fff640 1 --2- 172.21.15.103:0/2204128323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d18102570 0x7f7d181967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:07.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.604+0000 7f7d1d553640 1 --2- 172.21.15.103:0/2204128323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d18102ef0 0x7f7d1819ab50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000eaa0 tx=0x7f7d1000ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:07.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.604+0000 7f7d15ffb640 1 -- 172.21.15.103:0/2204128323 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1000cdb0 con 0x7f7d18102ef0 2024-09-19T06:50:07.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.604+0000 7f7d15ffb640 1 -- 172.21.15.103:0/2204128323 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d10005020 con 0x7f7d18102ef0 2024-09-19T06:50:07.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.604+0000 7f7d15ffb640 1 -- 172.21.15.103:0/2204128323 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d100116b0 con 0x7f7d18102ef0 2024-09-19T06:50:07.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.604+0000 7f7d1efb6640 1 -- 172.21.15.103:0/2204128323 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1819b520 con 0x7f7d18102ef0 2024-09-19T06:50:07.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.604+0000 7f7d1efb6640 1 -- 172.21.15.103:0/2204128323 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d181a7210 con 0x7f7d18102ef0 2024-09-19T06:50:07.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.608+0000 7f7d2138f740 1 Processor -- start 2024-09-19T06:50:07.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.608+0000 7f7d15ffb640 1 -- 172.21.15.103:0/2204128323 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7f7d100051d0 con 0x7f7d18102ef0 2024-09-19T06:50:07.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.608+0000 7f7d2138f740 1 -- start start 2024-09-19T06:50:07.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.608+0000 7f7d2138f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e793f70 0x55ae0e794350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.608+0000 7f7d2138f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e794890 0x55ae0e79ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.609+0000 7f7d2138f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e79f150 0x55ae0e7a1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.609+0000 7f7d2138f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ae0e6f12a0 con 0x55ae0e794890 2024-09-19T06:50:07.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.609+0000 7f7d2138f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ae0e698b10 con 0x55ae0e793f70 2024-09-19T06:50:07.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.609+0000 7f7d2138f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ae0e6ed260 con 0x55ae0e79f150 2024-09-19T06:50:07.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.609+0000 7f7d1cd52640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e794890 0x55ae0e79ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.609+0000 7f7d1cd52640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e794890 0x55ae0e79ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46032/0 (socket says 172.21.15.103:46032) 2024-09-19T06:50:07.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.609+0000 7f7d1cd52640 1 -- 172.21.15.103:0/3179700100 learned_addr learned my addr 172.21.15.103:0/3179700100 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:07.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.609+0000 7f7d1cd52640 1 -- 172.21.15.103:0/3179700100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e79f150 msgr2=0x55ae0e7a1540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:50:07.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.609+0000 7f7d1cd52640 1 --2- 172.21.15.103:0/3179700100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e79f150 0x55ae0e7a1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.609+0000 7f7d1cd52640 1 -- 172.21.15.103:0/3179700100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e793f70 msgr2=0x55ae0e794350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:07.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.609+0000 7f7d17fff640 1 --2- 172.21.15.103:0/3179700100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e793f70 0x55ae0e794350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.609+0000 7f7d1cd52640 1 --2- 172.21.15.103:0/3179700100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e793f70 0x55ae0e794350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.609+0000 7f7d1cd52640 1 -- 172.21.15.103:0/3179700100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae0e462b90 con 0x55ae0e794890 2024-09-19T06:50:07.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.609+0000 7f7d17fff640 1 --2- 172.21.15.103:0/3179700100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e793f70 0x55ae0e794350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:07.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.610+0000 7f7d1cd52640 1 --2- 172.21.15.103:0/3179700100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e794890 0x55ae0e79ec10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00bd90 tx=0x7f7d0c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4143f0cea21ad42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:07.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.610+0000 7f7d02ffd640 1 -- 172.21.15.103:0/3179700100 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c015400 con 0x55ae0e794890 2024-09-19T06:50:07.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.610+0000 7f7d02ffd640 1 -- 172.21.15.103:0/3179700100 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c01a070 con 0x55ae0e794890 2024-09-19T06:50:07.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.610+0000 7f7d02ffd640 1 -- 172.21.15.103:0/3179700100 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0148c0 con 0x55ae0e794890 2024-09-19T06:50:07.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.610+0000 7f7d15ffb640 1 --2- 172.21.15.103:0/2204128323 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d080a35e0 0x7f7d080a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.610+0000 7f7d15ffb640 1 -- 172.21.15.103:0/2204128323 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f7d100d0200 con 0x7f7d18102ef0 2024-09-19T06:50:07.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.611+0000 7f7d17fff640 1 --2- 172.21.15.103:0/2204128323 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d080a35e0 0x7f7d080a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.611+0000 7f7d2138f740 1 -- 172.21.15.103:0/3179700100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e794890 msgr2=0x55ae0e79ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.611+0000 7f7d2138f740 1 --2- 172.21.15.103:0/3179700100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e794890 0x55ae0e79ec10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00bd90 tx=0x7f7d0c005f90 comp rx=0 tx=0).stop 2024-09-19T06:50:07.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.612+0000 7f7d17fff640 1 --2- 172.21.15.103:0/2204128323 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d080a35e0 0x7f7d080a5aa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d04005ab0 tx=0x7f7d04000bc0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:07.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.613+0000 7f7d2138f740 1 -- 172.21.15.103:0/3179700100 shutdown_connections 2024-09-19T06:50:07.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.613+0000 7f7d2138f740 1 --2- 172.21.15.103:0/3179700100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e79f150 0x55ae0e7a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.613+0000 7f7d2138f740 1 --2- 172.21.15.103:0/3179700100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e794890 0x55ae0e79ec10 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.613+0000 7f7d2138f740 1 --2- 172.21.15.103:0/3179700100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e793f70 0x55ae0e794350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.613+0000 7f7d2138f740 1 -- 172.21.15.103:0/3179700100 >> 172.21.15.103:0/3179700100 conn(0x55ae0e790500 msgr2=0x55ae0e7908e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:07.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.614+0000 7f7d2138f740 1 -- 172.21.15.103:0/3179700100 shutdown_connections 2024-09-19T06:50:07.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.614+0000 7f7d2138f740 1 -- 172.21.15.103:0/3179700100 wait complete. 2024-09-19T06:50:07.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.615+0000 7f7d2138f740 1 -- start start 2024-09-19T06:50:07.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.615+0000 7f7d2138f740 1 Processor -- start 2024-09-19T06:50:07.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.615+0000 7f7d2138f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e793f70 0x55ae0e79bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.615+0000 7f7d2138f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e794890 0x55ae0e79c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.615+0000 7f7d2138f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e79f150 0x55ae0e79ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.616+0000 7f7d2138f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ae0e7a1d20 con 0x55ae0e794890 2024-09-19T06:50:07.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.616+0000 7f7d2138f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ae0e6796e0 con 0x55ae0e79f150 2024-09-19T06:50:07.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.616+0000 7f7d2138f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ae0e7a1ea0 con 0x55ae0e793f70 2024-09-19T06:50:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.616+0000 7f7d2138f740 10 client.0 _ll_register_callbacks cb 0x55ae0e69da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.616+0000 7f7d17fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e794890 0x55ae0e79c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.616+0000 7f7d17fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e794890 0x55ae0e79c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46048/0 (socket says 172.21.15.103:46048) 2024-09-19T06:50:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.616+0000 7f7d17fff640 1 -- 172.21.15.103:0/4177133293 learned_addr learned my addr 172.21.15.103:0/4177133293 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.616+0000 7f7d1cd52640 1 --2- 172.21.15.103:0/4177133293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e793f70 0x55ae0e79bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.616+0000 7f7d1d553640 1 --2- 172.21.15.103:0/4177133293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e79f150 0x55ae0e79ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.616+0000 7f7d17fff640 1 -- 172.21.15.103:0/4177133293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e793f70 msgr2=0x55ae0e79bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.616+0000 7f7d17fff640 1 --2- 172.21.15.103:0/4177133293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e793f70 0x55ae0e79bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.616+0000 7f7d17fff640 1 -- 172.21.15.103:0/4177133293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e79f150 msgr2=0x55ae0e79ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.616+0000 7f7d17fff640 1 --2- 172.21.15.103:0/4177133293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e79f150 0x55ae0e79ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.616+0000 7f7d17fff640 1 -- 172.21.15.103:0/4177133293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae0e7ab210 con 0x55ae0e794890 2024-09-19T06:50:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.616+0000 7f7d17fff640 1 --2- 172.21.15.103:0/4177133293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e794890 0x55ae0e79c4c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7d04033000 tx=0x7f7d0402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54a028646b25587 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.616+0000 7f7cf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.617+0000 7f7cf7fff640 1 -- 172.21.15.103:0/4177133293 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04035050 con 0x55ae0e794890 2024-09-19T06:50:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.617+0000 7f7d2138f740 1 -- 172.21.15.103:0/4177133293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d0c01a070 con 0x55ae0e794890 2024-09-19T06:50:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.617+0000 7f7cf7fff640 1 -- 172.21.15.103:0/4177133293 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f7d04004dc0 con 0x55ae0e794890 2024-09-19T06:50:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.617+0000 7f7cf7fff640 1 -- 172.21.15.103:0/4177133293 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d040055d0 con 0x55ae0e794890 2024-09-19T06:50:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.617+0000 7f7d2138f740 10 client.15699 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.617+0000 7f7d2138f740 1 -- 172.21.15.103:0/4177133293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ae0e787710 con 0x55ae0e794890 2024-09-19T06:50:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.617+0000 7f7cf7fff640 1 -- 172.21.15.103:0/4177133293 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c01a070 con 0x55ae0e794890 2024-09-19T06:50:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.617+0000 7f7cf7fff640 1 -- 172.21.15.103:0/4177133293 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7d040327c0 con 0x55ae0e794890 2024-09-19T06:50:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.617+0000 7f7d2138f740 10 client.15699 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.617+0000 7f7d2138f740 10 client.15699 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:50:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.617+0000 7f7d2138f740 20 client.15699 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.617+0000 7f7d2138f740 1 --2- 172.21.15.103:0/4177133293 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ae0e788fe0 0x55ae0e78b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.617+0000 7f7d2138f740 4 client.15699 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.617+0000 7f7d2138f740 1 -- 172.21.15.103:0/4177133293 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae0e6fe9d0 con 0x55ae0e788fe0 2024-09-19T06:50:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.618+0000 7f7d1cd52640 1 --2- 172.21.15.103:0/4177133293 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ae0e788fe0 0x55ae0e78b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.618+0000 7f7cf7fff640 10 client.15699 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.618+0000 7f7d1cd52640 1 --2- 172.21.15.103:0/4177133293 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ae0e788fe0 0x55ae0e78b420 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e367875c3634effd server_cookie=c8923ce94f743493 in_seq=0 out_seq=0 2024-09-19T06:50:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.619+0000 7f7cf7fff640 1 -- 172.21.15.103:0/4177133293 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae0e6fe9d0 con 0x55ae0e788fe0 2024-09-19T06:50:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.619+0000 7f7cf7fff640 10 client.15699 handle_command_reply: tid=0 2024-09-19T06:50:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.619+0000 7f7d2138f740 1 client.15699 shutdown 2024-09-19T06:50:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.619+0000 7f7d2138f740 1 -- 172.21.15.103:0/4177133293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e794890 msgr2=0x55ae0e79c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.619+0000 7f7d2138f740 1 --2- 172.21.15.103:0/4177133293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e794890 0x55ae0e79c4c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7d04033000 tx=0x7f7d0402a480 comp rx=0 tx=0).stop 2024-09-19T06:50:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.619+0000 7f7d2138f740 1 -- 172.21.15.103:0/4177133293 shutdown_connections 2024-09-19T06:50:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.619+0000 7f7d2138f740 1 --2- 172.21.15.103:0/4177133293 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ae0e788fe0 0x55ae0e78b420 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.619+0000 7f7d2138f740 1 --2- 172.21.15.103:0/4177133293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e79f150 0x55ae0e79ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.619+0000 7f7d2138f740 1 --2- 172.21.15.103:0/4177133293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e794890 0x55ae0e79c4c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.619+0000 7f7d2138f740 1 --2- 172.21.15.103:0/4177133293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e793f70 0x55ae0e79bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.619+0000 7f7d2138f740 1 -- 172.21.15.103:0/4177133293 >> 172.21.15.103:0/4177133293 conn(0x55ae0e790500 msgr2=0x55ae0e796230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.619+0000 7f7cf7fff640 0 client.15699 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.620+0000 7f7d2138f740 1 -- 172.21.15.103:0/4177133293 shutdown_connections 2024-09-19T06:50:07.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.620+0000 7f7d2138f740 1 -- 172.21.15.103:0/4177133293 wait complete. 2024-09-19T06:50:07.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.620+0000 7f7d2138f740 20 client.15699 trim_cache size 0 max 16384 2024-09-19T06:50:07.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.652+0000 7f7d2138f740 1 Processor -- start 2024-09-19T06:50:07.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.652+0000 7f7d2138f740 1 -- start start 2024-09-19T06:50:07.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.652+0000 7f7d2138f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e793f70 0x55ae0e7991e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.652+0000 7f7d2138f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e79f150 0x55ae0e799720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.652+0000 7f7d2138f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e799c60 0x55ae0e7b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.652+0000 7f7d2138f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d181ab660 con 0x55ae0e793f70 2024-09-19T06:50:07.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.652+0000 7f7d2138f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d181ad1a0 con 0x55ae0e799c60 2024-09-19T06:50:07.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.652+0000 7f7d2138f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d181abd30 con 0x55ae0e79f150 2024-09-19T06:50:07.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.652+0000 7f7d1cd52640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e793f70 0x55ae0e7991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.652+0000 7f7d1cd52640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e793f70 0x55ae0e7991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46060/0 (socket says 172.21.15.103:46060) 2024-09-19T06:50:07.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.652+0000 7f7d1cd52640 1 -- 172.21.15.103:0/3000478311 learned_addr learned my addr 172.21.15.103:0/3000478311 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:07.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.652+0000 7f7d17fff640 1 --2- 172.21.15.103:0/3000478311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e79f150 0x55ae0e799720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.653+0000 7f7d1d553640 1 --2- 172.21.15.103:0/3000478311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e799c60 0x55ae0e7b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.653+0000 7f7d1cd52640 1 -- 172.21.15.103:0/3000478311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e79f150 msgr2=0x55ae0e799720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:07.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.653+0000 7f7d1cd52640 1 --2- 172.21.15.103:0/3000478311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e79f150 0x55ae0e799720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.653+0000 7f7d1cd52640 1 -- 172.21.15.103:0/3000478311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e799c60 msgr2=0x55ae0e7b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:07.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.653+0000 7f7d1cd52640 1 --2- 172.21.15.103:0/3000478311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e799c60 0x55ae0e7b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.653+0000 7f7d1cd52640 1 -- 172.21.15.103:0/3000478311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae0e790330 con 0x55ae0e793f70 2024-09-19T06:50:07.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.653+0000 7f7d1cd52640 1 --2- 172.21.15.103:0/3000478311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e793f70 0x55ae0e7991e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c014470 tx=0x7f7d0c014400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3e9e52a11e9991 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:07.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.653+0000 7f7d027fc640 1 -- 172.21.15.103:0/3000478311 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c005040 con 0x55ae0e793f70 2024-09-19T06:50:07.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.653+0000 7f7d027fc640 1 -- 172.21.15.103:0/3000478311 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c01a070 con 0x55ae0e793f70 2024-09-19T06:50:07.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.653+0000 7f7d027fc640 1 -- 172.21.15.103:0/3000478311 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0035f0 con 0x55ae0e793f70 2024-09-19T06:50:07.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.654+0000 7f7d2138f740 1 -- 172.21.15.103:0/3000478311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e793f70 msgr2=0x55ae0e7991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.654+0000 7f7d2138f740 1 --2- 172.21.15.103:0/3000478311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e793f70 0x55ae0e7991e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c014470 tx=0x7f7d0c014400 comp rx=0 tx=0).stop 2024-09-19T06:50:07.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.655+0000 7f7d2138f740 1 -- 172.21.15.103:0/3000478311 shutdown_connections 2024-09-19T06:50:07.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.655+0000 7f7d2138f740 1 --2- 172.21.15.103:0/3000478311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e799c60 0x55ae0e7b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.655+0000 7f7d2138f740 1 --2- 172.21.15.103:0/3000478311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e79f150 0x55ae0e799720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.655+0000 7f7d2138f740 1 --2- 172.21.15.103:0/3000478311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e793f70 0x55ae0e7991e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.655+0000 7f7d2138f740 1 -- 172.21.15.103:0/3000478311 >> 172.21.15.103:0/3000478311 conn(0x55ae0e790500 msgr2=0x55ae0e7aa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:07.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.655+0000 7f7d2138f740 1 -- 172.21.15.103:0/3000478311 shutdown_connections 2024-09-19T06:50:07.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.655+0000 7f7d2138f740 1 -- 172.21.15.103:0/3000478311 wait complete. 2024-09-19T06:50:07.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.655+0000 7f7d2138f740 1 -- start start 2024-09-19T06:50:07.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.655+0000 7f7d2138f740 1 Processor -- start 2024-09-19T06:50:07.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d2138f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e793f70 0x55ae0e7b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d2138f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e799c60 0x55ae0e7b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d2138f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e79f150 0x55ae0e7b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d2138f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d181acad0 con 0x55ae0e793f70 2024-09-19T06:50:07.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d2138f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d181ac400 con 0x55ae0e79f150 2024-09-19T06:50:07.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d2138f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d181a9640 con 0x55ae0e799c60 2024-09-19T06:50:07.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d2138f740 10 client.0 _ll_register_callbacks cb 0x7f7d04005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:07.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d17fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e799c60 0x55ae0e7b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d17fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e799c60 0x55ae0e7b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:35938/0 (socket says 172.21.15.103:35938) 2024-09-19T06:50:07.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d17fff640 1 -- 172.21.15.103:0/1311480230 learned_addr learned my addr 172.21.15.103:0/1311480230 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:07.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d1cd52640 1 --2- 172.21.15.103:0/1311480230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e793f70 0x55ae0e7b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d17fff640 1 -- 172.21.15.103:0/1311480230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e79f150 msgr2=0x55ae0e7b1420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:50:07.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d17fff640 1 --2- 172.21.15.103:0/1311480230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e79f150 0x55ae0e7b1420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d17fff640 1 -- 172.21.15.103:0/1311480230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e793f70 msgr2=0x55ae0e7b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:07.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d17fff640 1 --2- 172.21.15.103:0/1311480230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e793f70 0x55ae0e7b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d17fff640 1 -- 172.21.15.103:0/1311480230 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae0e783260 con 0x55ae0e799c60 2024-09-19T06:50:07.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d1cd52640 1 --2- 172.21.15.103:0/1311480230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e793f70 0x55ae0e7b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:07.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7d17fff640 1 --2- 172.21.15.103:0/1311480230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e799c60 0x55ae0e7b33b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7d04005770 tx=0x7f7d04032760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3473bee4e425164c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:07.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.656+0000 7f7cf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T06:50:07.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.657+0000 7f7cf7fff640 1 -- 172.21.15.103:0/1311480230 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04035040 con 0x55ae0e799c60 2024-09-19T06:50:07.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.657+0000 7f7cf7fff640 1 -- 172.21.15.103:0/1311480230 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f7d040037f0 con 0x55ae0e799c60 2024-09-19T06:50:07.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.657+0000 7f7d2138f740 1 -- 172.21.15.103:0/1311480230 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d0c01a070 con 0x55ae0e799c60 2024-09-19T06:50:07.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.657+0000 7f7cf7fff640 1 -- 172.21.15.103:0/1311480230 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d04078050 con 0x55ae0e799c60 2024-09-19T06:50:07.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.657+0000 7f7d2138f740 10 client.25114 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:07.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.657+0000 7f7d2138f740 1 -- 172.21.15.103:0/1311480230 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ae0e812570 con 0x55ae0e799c60 2024-09-19T06:50:07.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.657+0000 7f7cf7fff640 1 -- 172.21.15.103:0/1311480230 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c01a070 con 0x55ae0e799c60 2024-09-19T06:50:07.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.658+0000 7f7cf7fff640 1 -- 172.21.15.103:0/1311480230 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7d04058650 con 0x55ae0e799c60 2024-09-19T06:50:07.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.658+0000 7f7d2138f740 10 client.25114 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:07.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.658+0000 7f7d2138f740 10 client.25114 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:50:07.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.658+0000 7f7d2138f740 20 client.25114 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:07.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.658+0000 7f7d2138f740 1 --2- 172.21.15.103:0/1311480230 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ae0e812db0 0x55ae0e8151d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.658+0000 7f7d2138f740 4 client.25114 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}] 2024-09-19T06:50:07.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.658+0000 7f7d2138f740 1 -- 172.21.15.103:0/1311480230 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}) -- 0x55ae0e6f8d20 con 0x55ae0e812db0 2024-09-19T06:50:07.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.658+0000 7f7d1cd52640 1 --2- 172.21.15.103:0/1311480230 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ae0e812db0 0x55ae0e8151d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.658+0000 7f7cf7fff640 10 client.25114 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:07.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.658+0000 7f7d1cd52640 1 --2- 172.21.15.103:0/1311480230 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ae0e812db0 0x55ae0e8151d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0f10c975f219aec server_cookie=aa950b5813e60b9a in_seq=0 out_seq=0 2024-09-19T06:50:07.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.670+0000 7f7cf7fff640 1 -- 172.21.15.103:0/1311480230 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55ae0e6f8d20 con 0x55ae0e812db0 2024-09-19T06:50:07.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.670+0000 7f7cf7fff640 10 client.25114 handle_command_reply: tid=0 2024-09-19T06:50:07.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.670+0000 7f7d2138f740 1 client.25114 shutdown 2024-09-19T06:50:07.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.670+0000 7f7d2138f740 1 -- 172.21.15.103:0/1311480230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e799c60 msgr2=0x55ae0e7b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.670+0000 7f7d2138f740 1 --2- 172.21.15.103:0/1311480230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e799c60 0x55ae0e7b33b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7d04005770 tx=0x7f7d04032760 comp rx=0 tx=0).stop 2024-09-19T06:50:07.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.671+0000 7f7d2138f740 1 -- 172.21.15.103:0/1311480230 shutdown_connections 2024-09-19T06:50:07.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.671+0000 7f7d2138f740 1 --2- 172.21.15.103:0/1311480230 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ae0e812db0 0x55ae0e8151d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.671+0000 7f7d2138f740 1 --2- 172.21.15.103:0/1311480230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae0e79f150 0x55ae0e7b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.671+0000 7f7d2138f740 1 --2- 172.21.15.103:0/1311480230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae0e799c60 0x55ae0e7b33b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.671+0000 7f7d2138f740 1 --2- 172.21.15.103:0/1311480230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae0e793f70 0x55ae0e7b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.671+0000 7f7d2138f740 1 -- 172.21.15.103:0/1311480230 >> 172.21.15.103:0/1311480230 conn(0x55ae0e790500 msgr2=0x55ae0e79bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:07.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.671+0000 7f7cf7fff640 0 client.25114 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:07.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.671+0000 7f7d2138f740 1 -- 172.21.15.103:0/1311480230 shutdown_connections 2024-09-19T06:50:07.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.671+0000 7f7d2138f740 1 -- 172.21.15.103:0/1311480230 wait complete. 2024-09-19T06:50:07.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.671+0000 7f7d2138f740 20 client.25114 trim_cache size 0 max 16384 2024-09-19T06:50:07.674 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:50:07.674 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:50:07.674 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:50:07.674 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 2, 2024-09-19T06:50:07.674 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:50:07.674 INFO:teuthology.orchestra.run.smithi103.stdout: "29305e6a": { 2024-09-19T06:50:07.674 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 2, 2024-09-19T06:50:07.674 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:50:07.674 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:50:07.675 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:50:07.675 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:50:07.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:50:07.675 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:50:07.675 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 131.2, 2024-09-19T06:50:07.675 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:50:07.675 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:50:07.675 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:50:07.675 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:50:07.675 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:50:07.675 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:50:07.675 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:07.675 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:07.675 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:07.675 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:07.676 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:07.676 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:50:07.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.672+0000 7f7d1efb6640 1 -- 172.21.15.103:0/2204128323 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d080a35e0 msgr2=0x7f7d080a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.672+0000 7f7d1efb6640 1 --2- 172.21.15.103:0/2204128323 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d080a35e0 0x7f7d080a5aa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d04005ab0 tx=0x7f7d04000bc0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.672+0000 7f7d1efb6640 1 -- 172.21.15.103:0/2204128323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d18102ef0 msgr2=0x7f7d1819ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.672+0000 7f7d1efb6640 1 --2- 172.21.15.103:0/2204128323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d18102ef0 0x7f7d1819ab50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000eaa0 tx=0x7f7d1000ef60 comp rx=0 tx=0).stop 2024-09-19T06:50:07.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.672+0000 7f7d1efb6640 1 -- 172.21.15.103:0/2204128323 shutdown_connections 2024-09-19T06:50:07.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.672+0000 7f7d1efb6640 1 --2- 172.21.15.103:0/2204128323 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d080a35e0 0x7f7d080a5aa0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.672+0000 7f7d1efb6640 1 --2- 172.21.15.103:0/2204128323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d18102ef0 0x7f7d1819ab50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.672+0000 7f7d1efb6640 1 --2- 172.21.15.103:0/2204128323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d18102570 0x7f7d181967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.672+0000 7f7d1efb6640 1 --2- 172.21.15.103:0/2204128323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d18100e20 0x7f7d18196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.672+0000 7f7d1efb6640 1 -- 172.21.15.103:0/2204128323 >> 172.21.15.103:0/2204128323 conn(0x7f7d180fc7d0 msgr2=0x7f7d180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:07.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.672+0000 7f7d1efb6640 1 -- 172.21.15.103:0/2204128323 shutdown_connections 2024-09-19T06:50:07.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:07.672+0000 7f7d1efb6640 1 -- 172.21.15.103:0/2204128323 wait complete. 2024-09-19T06:50:08.020 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:50:08.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:08 smithi174 ceph-mon[35637]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-19T06:50:08.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:08 smithi103 ceph-mon[31158]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-19T06:50:08.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:08 smithi134 ceph-mon[35356]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-19T06:50:08.662 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '29305e6a', quiesce took 8.3 seconds. Will release after: 2.8999999999999986 2024-09-19T06:50:08.709 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:09.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.216+0000 7fb852158640 1 -- 172.21.15.103:0/3165522895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb84c100e20 msgr2=0x7fb84c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:09.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.216+0000 7fb852158640 1 --2- 172.21.15.103:0/3165522895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb84c100e20 0x7fb84c101200 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009900 tx=0x7fb83c02a480 comp rx=0 tx=0).stop 2024-09-19T06:50:09.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.216+0000 7fb852158640 1 -- 172.21.15.103:0/3165522895 shutdown_connections 2024-09-19T06:50:09.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.216+0000 7fb852158640 1 --2- 172.21.15.103:0/3165522895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c102ef0 0x7fb84c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.216+0000 7fb852158640 1 --2- 172.21.15.103:0/3165522895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb84c102570 0x7fb84c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.216+0000 7fb852158640 1 --2- 172.21.15.103:0/3165522895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb84c100e20 0x7fb84c101200 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.216+0000 7fb852158640 1 -- 172.21.15.103:0/3165522895 >> 172.21.15.103:0/3165522895 conn(0x7fb84c0fc7d0 msgr2=0x7fb84c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:09.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.216+0000 7fb852158640 1 -- 172.21.15.103:0/3165522895 shutdown_connections 2024-09-19T06:50:09.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.216+0000 7fb852158640 1 -- 172.21.15.103:0/3165522895 wait complete. 2024-09-19T06:50:09.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.217+0000 7fb852158640 1 Processor -- start 2024-09-19T06:50:09.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.217+0000 7fb852158640 1 -- start start 2024-09-19T06:50:09.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.217+0000 7fb852158640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb84c100e20 0x7fb84c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.217+0000 7fb852158640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c102570 0x7fb84c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.217+0000 7fb852158640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb84c102ef0 0x7fb84c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.217+0000 7fb852158640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c112080 con 0x7fb84c102ef0 2024-09-19T06:50:09.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.217+0000 7fb852158640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c111f00 con 0x7fb84c102570 2024-09-19T06:50:09.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.217+0000 7fb852158640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c112200 con 0x7fb84c100e20 2024-09-19T06:50:09.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.218+0000 7fb84b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb84c100e20 0x7fb84c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.218+0000 7fb84b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb84c100e20 0x7fb84c1984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:35962/0 (socket says 172.21.15.103:35962) 2024-09-19T06:50:09.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.218+0000 7fb84b7fe640 1 -- 172.21.15.103:0/2568290131 learned_addr learned my addr 172.21.15.103:0/2568290131 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:09.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.218+0000 7fb84affd640 1 --2- 172.21.15.103:0/2568290131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c102570 0x7fb84c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.218+0000 7fb84bfff640 1 --2- 172.21.15.103:0/2568290131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb84c102ef0 0x7fb84c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.218+0000 7fb84b7fe640 1 -- 172.21.15.103:0/2568290131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c102570 msgr2=0x7fb84c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:09.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.218+0000 7fb84b7fe640 1 --2- 172.21.15.103:0/2568290131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c102570 0x7fb84c198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.218+0000 7fb84b7fe640 1 -- 172.21.15.103:0/2568290131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb84c102ef0 msgr2=0x7fb84c19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:09.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.218+0000 7fb84b7fe640 1 --2- 172.21.15.103:0/2568290131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb84c102ef0 0x7fb84c19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.218+0000 7fb84b7fe640 1 -- 172.21.15.103:0/2568290131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84c19d490 con 0x7fb84c100e20 2024-09-19T06:50:09.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.219+0000 7fb84b7fe640 1 --2- 172.21.15.103:0/2568290131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb84c100e20 0x7fb84c1984c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb83c006010 tx=0x7fb83c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:09.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.219+0000 7fb848ff9640 1 -- 172.21.15.103:0/2568290131 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c03f070 con 0x7fb84c100e20 2024-09-19T06:50:09.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.219+0000 7fb852158640 1 -- 172.21.15.103:0/2568290131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84c19d720 con 0x7fb84c100e20 2024-09-19T06:50:09.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.219+0000 7fb852158640 1 -- 172.21.15.103:0/2568290131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb84c06bc10 con 0x7fb84c100e20 2024-09-19T06:50:09.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.219+0000 7fb848ff9640 1 -- 172.21.15.103:0/2568290131 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83c003e30 con 0x7fb84c100e20 2024-09-19T06:50:09.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.219+0000 7fb848ff9640 1 -- 172.21.15.103:0/2568290131 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c035030 con 0x7fb84c100e20 2024-09-19T06:50:09.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.222+0000 7fb854531740 1 Processor -- start 2024-09-19T06:50:09.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.222+0000 7fb854531740 1 -- start start 2024-09-19T06:50:09.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.222+0000 7fb854531740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee14440 0x55906ee14820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.222+0000 7fb854531740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee14df0 0x55906ee19470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.222+0000 7fb854531740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee19a40 0x55906ee19ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.222+0000 7fb854531740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55906ed702f0 con 0x55906ee19a40 2024-09-19T06:50:09.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.222+0000 7fb854531740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55906ed17cf0 con 0x55906ee14440 2024-09-19T06:50:09.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.222+0000 7fb854531740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55906ed6c2b0 con 0x55906ee14df0 2024-09-19T06:50:09.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.222+0000 7fb848ff9640 1 -- 172.21.15.103:0/2568290131 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7fb83c042070 con 0x7fb84c100e20 2024-09-19T06:50:09.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.223+0000 7fb84affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee19a40 0x55906ee19ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.223+0000 7fb84affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee19a40 0x55906ee19ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46104/0 (socket says 172.21.15.103:46104) 2024-09-19T06:50:09.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.223+0000 7fb84affd640 1 -- 172.21.15.103:0/139951763 learned_addr learned my addr 172.21.15.103:0/139951763 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:09.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.223+0000 7fb84bfff640 1 --2- 172.21.15.103:0/139951763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee14440 0x55906ee14820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.223+0000 7fb84b7fe640 1 --2- 172.21.15.103:0/139951763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee14df0 0x55906ee19470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.223+0000 7fb84affd640 1 -- 172.21.15.103:0/139951763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee14df0 msgr2=0x55906ee19470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.223+0000 7fb84affd640 1 --2- 172.21.15.103:0/139951763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee14df0 0x55906ee19470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.223+0000 7fb84affd640 1 -- 172.21.15.103:0/139951763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee14440 msgr2=0x55906ee14820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.223+0000 7fb84affd640 1 --2- 172.21.15.103:0/139951763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee14440 0x55906ee14820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.223+0000 7fb84affd640 1 -- 172.21.15.103:0/139951763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55906ecfc3a0 con 0x55906ee19a40 2024-09-19T06:50:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.223+0000 7fb84bfff640 1 --2- 172.21.15.103:0/139951763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee14440 0x55906ee14820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.223+0000 7fb84b7fe640 1 --2- 172.21.15.103:0/139951763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee14df0 0x55906ee19470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.224+0000 7fb848ff9640 1 --2- 172.21.15.103:0/2568290131 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb8340a3760 0x7fb8340a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.224+0000 7fb84bfff640 1 --2- 172.21.15.103:0/2568290131 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb8340a3760 0x7fb8340a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.224+0000 7fb848ff9640 1 -- 172.21.15.103:0/2568290131 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fb83c03b900 con 0x7fb84c100e20 2024-09-19T06:50:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.224+0000 7fb84affd640 1 --2- 172.21.15.103:0/139951763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee19a40 0x55906ee19ef0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb84000acd0 tx=0x7fb84000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbea88f9d5f8060 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.225+0000 7fb831ffb640 1 -- 172.21.15.103:0/139951763 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84001d070 con 0x55906ee19a40 2024-09-19T06:50:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.225+0000 7fb84bfff640 1 --2- 172.21.15.103:0/2568290131 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb8340a3760 0x7fb8340a5c20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb844010540 tx=0x7fb84400a4d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.225+0000 7fb831ffb640 1 -- 172.21.15.103:0/139951763 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb840018b20 con 0x55906ee19a40 2024-09-19T06:50:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.225+0000 7fb831ffb640 1 -- 172.21.15.103:0/139951763 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840010c90 con 0x55906ee19a40 2024-09-19T06:50:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.226+0000 7fb854531740 1 -- 172.21.15.103:0/139951763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee19a40 msgr2=0x55906ee19ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.226+0000 7fb854531740 1 --2- 172.21.15.103:0/139951763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee19a40 0x55906ee19ef0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb84000acd0 tx=0x7fb84000f790 comp rx=0 tx=0).stop 2024-09-19T06:50:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.226+0000 7fb854531740 1 -- 172.21.15.103:0/139951763 shutdown_connections 2024-09-19T06:50:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.226+0000 7fb854531740 1 --2- 172.21.15.103:0/139951763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee19a40 0x55906ee19ef0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.226+0000 7fb854531740 1 --2- 172.21.15.103:0/139951763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee14df0 0x55906ee19470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.226+0000 7fb854531740 1 --2- 172.21.15.103:0/139951763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee14440 0x55906ee14820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.226+0000 7fb854531740 1 -- 172.21.15.103:0/139951763 >> 172.21.15.103:0/139951763 conn(0x55906edd3310 msgr2=0x55906ee066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.226+0000 7fb854531740 1 -- 172.21.15.103:0/139951763 shutdown_connections 2024-09-19T06:50:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.226+0000 7fb854531740 1 -- 172.21.15.103:0/139951763 wait complete. 2024-09-19T06:50:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.226+0000 7fb854531740 1 -- start start 2024-09-19T06:50:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.227+0000 7fb854531740 1 Processor -- start 2024-09-19T06:50:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.227+0000 7fb854531740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee14440 0x55906ee1fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.227+0000 7fb854531740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee14df0 0x55906ee1d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.227+0000 7fb854531740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee19a40 0x55906ee1d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.227+0000 7fb854531740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55906ee20970 con 0x55906ee14440 2024-09-19T06:50:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.227+0000 7fb854531740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55906ed2b060 con 0x55906ee19a40 2024-09-19T06:50:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.227+0000 7fb854531740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55906ee20af0 con 0x55906ee14df0 2024-09-19T06:50:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.227+0000 7fb854531740 10 client.0 _ll_register_callbacks cb 0x55906ea7e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.227+0000 7fb84bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee19a40 0x55906ee1d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.227+0000 7fb84b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee14440 0x55906ee1fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.227+0000 7fb84bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee19a40 0x55906ee1d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:48080/0 (socket says 172.21.15.103:48080) 2024-09-19T06:50:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.227+0000 7fb84bfff640 1 -- 172.21.15.103:0/2995532293 learned_addr learned my addr 172.21.15.103:0/2995532293 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.228+0000 7fb84affd640 1 --2- 172.21.15.103:0/2995532293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee14df0 0x55906ee1d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.228+0000 7fb84bfff640 1 -- 172.21.15.103:0/2995532293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee14df0 msgr2=0x55906ee1d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.228+0000 7fb84bfff640 1 --2- 172.21.15.103:0/2995532293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee14df0 0x55906ee1d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.228+0000 7fb84bfff640 1 -- 172.21.15.103:0/2995532293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee14440 msgr2=0x55906ee1fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.228+0000 7fb84bfff640 1 --2- 172.21.15.103:0/2995532293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee14440 0x55906ee1fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.228+0000 7fb84bfff640 1 -- 172.21.15.103:0/2995532293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55906eae13c0 con 0x55906ee19a40 2024-09-19T06:50:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.228+0000 7fb84bfff640 1 --2- 172.21.15.103:0/2995532293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee19a40 0x55906ee1d5e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb844005e50 tx=0x7fb844004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9d12b6c5fb2725e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.228+0000 7fb82affd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:50:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.228+0000 7fb82affd640 1 -- 172.21.15.103:0/2995532293 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84400fd80 con 0x55906ee19a40 2024-09-19T06:50:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.228+0000 7fb82affd640 1 -- 172.21.15.103:0/2995532293 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fb844016030 con 0x55906ee19a40 2024-09-19T06:50:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.228+0000 7fb854531740 1 -- 172.21.15.103:0/2995532293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb840018b20 con 0x55906ee19a40 2024-09-19T06:50:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.229+0000 7fb82affd640 1 -- 172.21.15.103:0/2995532293 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb844003970 con 0x55906ee19a40 2024-09-19T06:50:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.229+0000 7fb854531740 10 client.25169 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.229+0000 7fb854531740 1 -- 172.21.15.103:0/2995532293 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55906ee2c3c0 con 0x55906ee19a40 2024-09-19T06:50:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.229+0000 7fb82affd640 1 -- 172.21.15.103:0/2995532293 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb840018b20 con 0x55906ee19a40 2024-09-19T06:50:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.229+0000 7fb82affd640 1 -- 172.21.15.103:0/2995532293 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb8440040b0 con 0x55906ee19a40 2024-09-19T06:50:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.229+0000 7fb854531740 10 client.25169 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.229+0000 7fb854531740 10 client.25169 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:50:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.229+0000 7fb854531740 20 client.25169 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.229+0000 7fb854531740 1 --2- 172.21.15.103:0/2995532293 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55906ee2d920 0x55906ee2fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.229+0000 7fb854531740 4 client.25169 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.229+0000 7fb854531740 1 -- 172.21.15.103:0/2995532293 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55906ed7d860 con 0x55906ee2d920 2024-09-19T06:50:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.230+0000 7fb84b7fe640 1 --2- 172.21.15.103:0/2995532293 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55906ee2d920 0x55906ee2fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.230+0000 7fb84b7fe640 1 --2- 172.21.15.103:0/2995532293 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55906ee2d920 0x55906ee2fdc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5c8bc9de48e7964 server_cookie=81bfea81367c1cec in_seq=0 out_seq=0 2024-09-19T06:50:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.230+0000 7fb82affd640 10 client.25169 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:50:09.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.231+0000 7fb82affd640 1 -- 172.21.15.103:0/2995532293 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55906ed7d860 con 0x55906ee2d920 2024-09-19T06:50:09.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.231+0000 7fb82affd640 10 client.25169 handle_command_reply: tid=0 2024-09-19T06:50:09.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.231+0000 7fb854531740 1 client.25169 shutdown 2024-09-19T06:50:09.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.231+0000 7fb854531740 1 -- 172.21.15.103:0/2995532293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee19a40 msgr2=0x55906ee1d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.231+0000 7fb854531740 1 --2- 172.21.15.103:0/2995532293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee19a40 0x55906ee1d5e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb844005e50 tx=0x7fb844004ab0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.231+0000 7fb854531740 1 -- 172.21.15.103:0/2995532293 shutdown_connections 2024-09-19T06:50:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.231+0000 7fb854531740 1 --2- 172.21.15.103:0/2995532293 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55906ee2d920 0x55906ee2fdc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.231+0000 7fb82affd640 0 client.25169 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:50:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.231+0000 7fb854531740 1 --2- 172.21.15.103:0/2995532293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee19a40 0x55906ee1d5e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.231+0000 7fb854531740 1 --2- 172.21.15.103:0/2995532293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee14df0 0x55906ee1d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.231+0000 7fb854531740 1 --2- 172.21.15.103:0/2995532293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee14440 0x55906ee1fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.231+0000 7fb854531740 1 -- 172.21.15.103:0/2995532293 >> 172.21.15.103:0/2995532293 conn(0x55906edd3310 msgr2=0x55906ee13a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.231+0000 7fb854531740 1 -- 172.21.15.103:0/2995532293 shutdown_connections 2024-09-19T06:50:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.231+0000 7fb854531740 1 -- 172.21.15.103:0/2995532293 wait complete. 2024-09-19T06:50:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.232+0000 7fb854531740 20 client.25169 trim_cache size 0 max 16384 2024-09-19T06:50:09.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.255+0000 7fb854531740 1 Processor -- start 2024-09-19T06:50:09.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.255+0000 7fb854531740 1 -- start start 2024-09-19T06:50:09.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.255+0000 7fb854531740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee14440 0x55906ee29c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.255+0000 7fb854531740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee14df0 0x55906ee13960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.255+0000 7fb854531740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee19a40 0x55906ee161e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.255+0000 7fb854531740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb83800e740 con 0x55906ee19a40 2024-09-19T06:50:09.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.255+0000 7fb854531740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb838010280 con 0x55906ee14df0 2024-09-19T06:50:09.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.255+0000 7fb854531740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb83800ee10 con 0x55906ee14440 2024-09-19T06:50:09.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.256+0000 7fb84bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee19a40 0x55906ee161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.256+0000 7fb84bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee19a40 0x55906ee161e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46110/0 (socket says 172.21.15.103:46110) 2024-09-19T06:50:09.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.256+0000 7fb84bfff640 1 -- 172.21.15.103:0/988080949 learned_addr learned my addr 172.21.15.103:0/988080949 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:09.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.256+0000 7fb84affd640 1 --2- 172.21.15.103:0/988080949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee14df0 0x55906ee13960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.256+0000 7fb84b7fe640 1 --2- 172.21.15.103:0/988080949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee14440 0x55906ee29c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.256+0000 7fb84bfff640 1 -- 172.21.15.103:0/988080949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee14440 msgr2=0x55906ee29c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:09.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.256+0000 7fb84bfff640 1 --2- 172.21.15.103:0/988080949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee14440 0x55906ee29c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.256+0000 7fb84bfff640 1 -- 172.21.15.103:0/988080949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee14df0 msgr2=0x55906ee13960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:09.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.256+0000 7fb84bfff640 1 --2- 172.21.15.103:0/988080949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee14df0 0x55906ee13960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.256+0000 7fb84bfff640 1 -- 172.21.15.103:0/988080949 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55906ee1b5f0 con 0x55906ee19a40 2024-09-19T06:50:09.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.256+0000 7fb84bfff640 1 --2- 172.21.15.103:0/988080949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee19a40 0x55906ee161e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb844017040 tx=0x7fb8440304f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54227e99f8ebcb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:09.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.256+0000 7fb8317fa640 1 -- 172.21.15.103:0/988080949 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844004740 con 0x55906ee19a40 2024-09-19T06:50:09.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.256+0000 7fb8317fa640 1 -- 172.21.15.103:0/988080949 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84403d070 con 0x55906ee19a40 2024-09-19T06:50:09.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.256+0000 7fb8317fa640 1 -- 172.21.15.103:0/988080949 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844003fb0 con 0x55906ee19a40 2024-09-19T06:50:09.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.257+0000 7fb854531740 1 -- 172.21.15.103:0/988080949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee19a40 msgr2=0x55906ee161e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:09.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.257+0000 7fb854531740 1 --2- 172.21.15.103:0/988080949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee19a40 0x55906ee161e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb844017040 tx=0x7fb8440304f0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.258+0000 7fb854531740 1 -- 172.21.15.103:0/988080949 shutdown_connections 2024-09-19T06:50:09.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.258+0000 7fb854531740 1 --2- 172.21.15.103:0/988080949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee19a40 0x55906ee161e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.258+0000 7fb854531740 1 --2- 172.21.15.103:0/988080949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee14df0 0x55906ee13960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.258+0000 7fb854531740 1 --2- 172.21.15.103:0/988080949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee14440 0x55906ee29c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.258+0000 7fb854531740 1 -- 172.21.15.103:0/988080949 >> 172.21.15.103:0/988080949 conn(0x55906edd3310 msgr2=0x55906ee13250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:09.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.258+0000 7fb854531740 1 -- 172.21.15.103:0/988080949 shutdown_connections 2024-09-19T06:50:09.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.258+0000 7fb854531740 1 -- 172.21.15.103:0/988080949 wait complete. 2024-09-19T06:50:09.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.258+0000 7fb854531740 1 -- start start 2024-09-19T06:50:09.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.258+0000 7fb854531740 1 Processor -- start 2024-09-19T06:50:09.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb854531740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee14440 0x55906ee25df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb854531740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee14df0 0x55906ee26330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb84b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee14440 0x55906ee25df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb84b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee14440 0x55906ee25df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46112/0 (socket says 172.21.15.103:46112) 2024-09-19T06:50:09.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb84b7fe640 1 -- 172.21.15.103:0/3199405676 learned_addr learned my addr 172.21.15.103:0/3199405676 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:09.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb854531740 1 --2- 172.21.15.103:0/3199405676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee19a40 0x55906ee20da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb854531740 1 -- 172.21.15.103:0/3199405676 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb83800fbb0 con 0x55906ee14440 2024-09-19T06:50:09.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb854531740 1 -- 172.21.15.103:0/3199405676 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb83800f4e0 con 0x55906ee14df0 2024-09-19T06:50:09.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb854531740 1 -- 172.21.15.103:0/3199405676 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb83800c700 con 0x55906ee19a40 2024-09-19T06:50:09.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb854531740 10 client.0 _ll_register_callbacks cb 0x7fb84400ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:09.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb84affd640 1 --2- 172.21.15.103:0/3199405676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee14df0 0x55906ee26330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb84b7fe640 1 -- 172.21.15.103:0/3199405676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee19a40 msgr2=0x55906ee20da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:09.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb84b7fe640 1 --2- 172.21.15.103:0/3199405676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee19a40 0x55906ee20da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb84b7fe640 1 -- 172.21.15.103:0/3199405676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee14df0 msgr2=0x55906ee26330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:09.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb84b7fe640 1 --2- 172.21.15.103:0/3199405676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee14df0 0x55906ee26330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb84b7fe640 1 -- 172.21.15.103:0/3199405676 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55906ee212e0 con 0x55906ee14440 2024-09-19T06:50:09.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.259+0000 7fb84b7fe640 1 --2- 172.21.15.103:0/3199405676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee14440 0x55906ee25df0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0f3b80 tx=0x7fb83c0b88e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929340aab8aa9453 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:09.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.260+0000 7fb82affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:09.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.260+0000 7fb82affd640 1 -- 172.21.15.103:0/3199405676 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0bbee0 con 0x55906ee14440 2024-09-19T06:50:09.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.260+0000 7fb82affd640 1 -- 172.21.15.103:0/3199405676 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fb83c03b460 con 0x55906ee14440 2024-09-19T06:50:09.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.260+0000 7fb854531740 1 -- 172.21.15.103:0/3199405676 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb84403d070 con 0x55906ee14440 2024-09-19T06:50:09.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.260+0000 7fb82affd640 1 -- 172.21.15.103:0/3199405676 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb83c0bcc80 con 0x55906ee14440 2024-09-19T06:50:09.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.260+0000 7fb854531740 10 client.15723 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:09.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.260+0000 7fb854531740 1 -- 172.21.15.103:0/3199405676 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55906ee98ea0 con 0x55906ee14440 2024-09-19T06:50:09.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.261+0000 7fb82affd640 1 -- 172.21.15.103:0/3199405676 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84403d070 con 0x55906ee14440 2024-09-19T06:50:09.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.261+0000 7fb82affd640 1 -- 172.21.15.103:0/3199405676 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb83c0bc870 con 0x55906ee14440 2024-09-19T06:50:09.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.261+0000 7fb854531740 10 client.15723 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:09.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.261+0000 7fb854531740 10 client.15723 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:50:09.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.261+0000 7fb854531740 20 client.15723 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:09.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.261+0000 7fb854531740 1 --2- 172.21.15.103:0/3199405676 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55906ee99520 0x55906ee9b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:09.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.261+0000 7fb854531740 4 client.15723 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:09.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.261+0000 7fb854531740 1 -- 172.21.15.103:0/3199405676 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55906ed77d70 con 0x55906ee99520 2024-09-19T06:50:09.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.261+0000 7fb84affd640 1 --2- 172.21.15.103:0/3199405676 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55906ee99520 0x55906ee9b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:09.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.261+0000 7fb84affd640 1 --2- 172.21.15.103:0/3199405676 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55906ee99520 0x55906ee9b940 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81e08abdf1e9640a server_cookie=24006fcaa1559843 in_seq=0 out_seq=0 2024-09-19T06:50:09.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.261+0000 7fb82affd640 10 client.15723 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:50:09.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.262+0000 7fb82affd640 1 -- 172.21.15.103:0/3199405676 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55906ed77d70 con 0x55906ee99520 2024-09-19T06:50:09.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.262+0000 7fb82affd640 10 client.15723 handle_command_reply: tid=0 2024-09-19T06:50:09.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.262+0000 7fb854531740 1 client.15723 shutdown 2024-09-19T06:50:09.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.262+0000 7fb854531740 1 -- 172.21.15.103:0/3199405676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee14440 msgr2=0x55906ee25df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:09.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.262+0000 7fb854531740 1 --2- 172.21.15.103:0/3199405676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee14440 0x55906ee25df0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0f3b80 tx=0x7fb83c0b88e0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.262+0000 7fb854531740 1 -- 172.21.15.103:0/3199405676 shutdown_connections 2024-09-19T06:50:09.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.262+0000 7fb854531740 1 --2- 172.21.15.103:0/3199405676 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55906ee99520 0x55906ee9b940 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.262+0000 7fb854531740 1 --2- 172.21.15.103:0/3199405676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55906ee19a40 0x55906ee20da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.262+0000 7fb854531740 1 --2- 172.21.15.103:0/3199405676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55906ee14df0 0x55906ee26330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.262+0000 7fb854531740 1 --2- 172.21.15.103:0/3199405676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55906ee14440 0x55906ee25df0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.262+0000 7fb854531740 1 -- 172.21.15.103:0/3199405676 >> 172.21.15.103:0/3199405676 conn(0x55906edd3310 msgr2=0x55906ee13a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:09.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.262+0000 7fb82affd640 0 client.15723 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:50:09.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.262+0000 7fb854531740 1 -- 172.21.15.103:0/3199405676 shutdown_connections 2024-09-19T06:50:09.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.262+0000 7fb854531740 1 -- 172.21.15.103:0/3199405676 wait complete. 2024-09-19T06:50:09.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.262+0000 7fb854531740 20 client.15723 trim_cache size 0 max 16384 2024-09-19T06:50:09.270 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:50:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.263+0000 7fb852158640 1 -- 172.21.15.103:0/2568290131 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb8340a3760 msgr2=0x7fb8340a5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.263+0000 7fb852158640 1 --2- 172.21.15.103:0/2568290131 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb8340a3760 0x7fb8340a5c20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb844010540 tx=0x7fb84400a4d0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.263+0000 7fb852158640 1 -- 172.21.15.103:0/2568290131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb84c100e20 msgr2=0x7fb84c1984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.263+0000 7fb852158640 1 --2- 172.21.15.103:0/2568290131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb84c100e20 0x7fb84c1984c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb83c006010 tx=0x7fb83c003c10 comp rx=0 tx=0).stop 2024-09-19T06:50:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.263+0000 7fb852158640 1 -- 172.21.15.103:0/2568290131 shutdown_connections 2024-09-19T06:50:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.263+0000 7fb852158640 1 --2- 172.21.15.103:0/2568290131 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb8340a3760 0x7fb8340a5c20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.263+0000 7fb852158640 1 --2- 172.21.15.103:0/2568290131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb84c102ef0 0x7fb84c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.263+0000 7fb852158640 1 --2- 172.21.15.103:0/2568290131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c102570 0x7fb84c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.263+0000 7fb852158640 1 --2- 172.21.15.103:0/2568290131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb84c100e20 0x7fb84c1984c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:09.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.263+0000 7fb852158640 1 -- 172.21.15.103:0/2568290131 >> 172.21.15.103:0/2568290131 conn(0x7fb84c0fc7d0 msgr2=0x7fb84c106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:09.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.263+0000 7fb852158640 1 -- 172.21.15.103:0/2568290131 shutdown_connections 2024-09-19T06:50:09.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:09.263+0000 7fb852158640 1 -- 172.21.15.103:0/2568290131 wait complete. 2024-09-19T06:50:10.012 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:50:10.275 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:10.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:10 smithi174 ceph-mon[35637]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-09-19T06:50:10.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:10 smithi103 ceph-mon[31158]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-09-19T06:50:10.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:10 smithi134 ceph-mon[35356]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-09-19T06:50:10.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.724+0000 7fce0c5be640 1 -- 172.21.15.103:0/434794533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04102ef0 msgr2=0x7fce0410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:10.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.724+0000 7fce0c5be640 1 --2- 172.21.15.103:0/434794533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04102ef0 0x7fce0410da40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009ca0 tx=0x7fcdfc02a950 comp rx=0 tx=0).stop 2024-09-19T06:50:10.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.724+0000 7fce0c5be640 1 -- 172.21.15.103:0/434794533 shutdown_connections 2024-09-19T06:50:10.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.724+0000 7fce0c5be640 1 --2- 172.21.15.103:0/434794533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04102ef0 0x7fce0410da40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.724+0000 7fce0c5be640 1 --2- 172.21.15.103:0/434794533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce04102570 0x7fce041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.724+0000 7fce0c5be640 1 --2- 172.21.15.103:0/434794533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce04100e20 0x7fce04101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.724+0000 7fce0c5be640 1 -- 172.21.15.103:0/434794533 >> 172.21.15.103:0/434794533 conn(0x7fce040fc7d0 msgr2=0x7fce040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:10.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.724+0000 7fce0c5be640 1 -- 172.21.15.103:0/434794533 shutdown_connections 2024-09-19T06:50:10.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.724+0000 7fce0c5be640 1 -- 172.21.15.103:0/434794533 wait complete. 2024-09-19T06:50:10.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.725+0000 7fce0c5be640 1 Processor -- start 2024-09-19T06:50:10.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.725+0000 7fce0c5be640 1 -- start start 2024-09-19T06:50:10.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.725+0000 7fce0c5be640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce04100e20 0x7fce041970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.725+0000 7fce0c5be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04102570 0x7fce04197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.725+0000 7fce0c5be640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce04102ef0 0x7fce04197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.725+0000 7fce0c5be640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fce04110f80 con 0x7fce04102570 2024-09-19T06:50:10.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.725+0000 7fce0c5be640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce04110e00 con 0x7fce04100e20 2024-09-19T06:50:10.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.725+0000 7fce0c5be640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fce04111100 con 0x7fce04102ef0 2024-09-19T06:50:10.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.726+0000 7fce09b59640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04102570 0x7fce04197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:10.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.726+0000 7fce0a35a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce04100e20 0x7fce041970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:10.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.726+0000 7fce0ab5b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce04102ef0 0x7fce04197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:10.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.726+0000 7fce0ab5b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce04102ef0 0x7fce04197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:36010/0 (socket says 172.21.15.103:36010) 2024-09-19T06:50:10.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.726+0000 7fce0a35a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce04100e20 0x7fce041970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:48126/0 (socket says 172.21.15.103:48126) 2024-09-19T06:50:10.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.726+0000 7fce09b59640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04102570 0x7fce04197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46126/0 (socket says 172.21.15.103:46126) 2024-09-19T06:50:10.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.726+0000 7fce0ab5b640 1 -- 172.21.15.103:0/3093447379 learned_addr learned my addr 172.21.15.103:0/3093447379 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:10.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.726+0000 7fce0ab5b640 1 -- 172.21.15.103:0/3093447379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce04100e20 msgr2=0x7fce041970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:10.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.726+0000 7fce0ab5b640 1 --2- 172.21.15.103:0/3093447379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce04100e20 0x7fce041970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.726+0000 7fce0ab5b640 1 -- 172.21.15.103:0/3093447379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04102570 msgr2=0x7fce04197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:10.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.726+0000 7fce0ab5b640 1 --2- 172.21.15.103:0/3093447379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04102570 0x7fce04197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.726+0000 7fce0ab5b640 1 -- 172.21.15.103:0/3093447379 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce041a6950 con 0x7fce04102ef0 2024-09-19T06:50:10.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.726+0000 7fce09b59640 1 --2- 172.21.15.103:0/3093447379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04102570 0x7fce04197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:10.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.726+0000 7fce0ab5b640 1 --2- 172.21.15.103:0/3093447379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce04102ef0 0x7fce04197b60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc02ae70 tx=0x7fcdfc02f5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:10.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.727+0000 7fcdfb7fe640 1 -- 172.21.15.103:0/3093447379 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc034040 con 0x7fce04102ef0 2024-09-19T06:50:10.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.727+0000 7fcdfb7fe640 1 -- 172.21.15.103:0/3093447379 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdfc030920 con 0x7fce04102ef0 2024-09-19T06:50:10.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.727+0000 7fce0c5be640 1 -- 172.21.15.103:0/3093447379 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce041a6be0 con 0x7fce04102ef0 2024-09-19T06:50:10.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.727+0000 7fcdfb7fe640 1 -- 172.21.15.103:0/3093447379 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc045480 con 0x7fce04102ef0 2024-09-19T06:50:10.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.727+0000 7fce0c5be640 1 -- 172.21.15.103:0/3093447379 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce041a70c0 con 0x7fce04102ef0 2024-09-19T06:50:10.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.729+0000 7fce0e997740 1 Processor -- start 2024-09-19T06:50:10.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.730+0000 7fcdfb7fe640 1 -- 172.21.15.103:0/3093447379 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7fcdfc037020 con 0x7fce04102ef0 2024-09-19T06:50:10.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.730+0000 7fce0e997740 1 -- start start 2024-09-19T06:50:10.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.730+0000 7fce0e997740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d23039b40 0x557d23038090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.730+0000 7fce0e997740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d230385d0 0x557d23038a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fce0e997740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23043370 0x557d23045760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fce0e997740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557d22f9e2f0 con 0x557d23039b40 2024-09-19T06:50:10.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fce0e997740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d22f45cf0 con 0x557d230385d0 2024-09-19T06:50:10.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fce0e997740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d22f9a2b0 con 0x557d23043370 2024-09-19T06:50:10.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fce0a35a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d230385d0 0x557d23038a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:10.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fce09b59640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d23039b40 0x557d23038090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:10.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fce0a35a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d230385d0 0x557d23038a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:48138/0 (socket says 172.21.15.103:48138) 2024-09-19T06:50:10.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fce0a35a640 1 -- 172.21.15.103:0/2077578044 learned_addr learned my addr 172.21.15.103:0/2077578044 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:10.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fce09b59640 1 --2- 172.21.15.103:0/2077578044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23043370 0x557d23045760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:10.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fce0a35a640 1 -- 172.21.15.103:0/2077578044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23043370 msgr2=0x557d23045760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:10.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fce0a35a640 1 --2- 172.21.15.103:0/2077578044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23043370 0x557d23045760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fce0a35a640 1 -- 172.21.15.103:0/2077578044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d23039b40 msgr2=0x557d23038090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:10.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fce0a35a640 1 --2- 172.21.15.103:0/2077578044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d23039b40 0x557d23038090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fce0a35a640 1 -- 172.21.15.103:0/2077578044 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d22f2a3a0 con 0x557d230385d0 2024-09-19T06:50:10.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fce09b59640 1 --2- 172.21.15.103:0/2077578044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d23039b40 0x557d23038090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:10.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fcdfb7fe640 1 --2- 172.21.15.103:0/3093447379 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcdf00a3560 0x7fcdf00a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.731+0000 7fce0a35a640 1 --2- 172.21.15.103:0/2077578044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d230385d0 0x557d23038a10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fce0000ac20 tx=0x7fce0000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fac726ef6681a31a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:10.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.732+0000 7fcdfb7fe640 1 -- 172.21.15.103:0/3093447379 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fcdfc0f0d00 con 0x7fce04102ef0 2024-09-19T06:50:10.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.732+0000 7fcdf8ff9640 1 -- 172.21.15.103:0/2077578044 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0001d070 con 0x557d230385d0 2024-09-19T06:50:10.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.732+0000 7fcdf8ff9640 1 -- 172.21.15.103:0/2077578044 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce00018b20 con 0x557d230385d0 2024-09-19T06:50:10.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.732+0000 7fcdf8ff9640 1 -- 172.21.15.103:0/2077578044 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00010ba0 con 0x557d230385d0 2024-09-19T06:50:10.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.732+0000 7fce09b59640 1 --2- 172.21.15.103:0/3093447379 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcdf00a3560 0x7fcdf00a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:10.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.733+0000 7fce0e997740 1 -- 172.21.15.103:0/2077578044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d230385d0 msgr2=0x557d23038a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:10.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.733+0000 7fce0e997740 1 --2- 172.21.15.103:0/2077578044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d230385d0 0x557d23038a10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fce0000ac20 tx=0x7fce0000f790 comp rx=0 tx=0).stop 2024-09-19T06:50:10.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.733+0000 7fce09b59640 1 --2- 172.21.15.103:0/3093447379 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcdf00a3560 0x7fcdf00a5a20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x557d23039070 tx=0x7fcdf400bf90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:10.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.733+0000 7fce0e997740 1 -- 172.21.15.103:0/2077578044 shutdown_connections 2024-09-19T06:50:10.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.733+0000 7fce0e997740 1 --2- 172.21.15.103:0/2077578044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23043370 0x557d23045760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.733+0000 7fce0e997740 1 --2- 172.21.15.103:0/2077578044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d230385d0 0x557d23038a10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.733+0000 7fce0e997740 1 --2- 172.21.15.103:0/2077578044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d23039b40 0x557d23038090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.733+0000 7fce0e997740 1 -- 172.21.15.103:0/2077578044 >> 172.21.15.103:0/2077578044 conn(0x557d23001310 msgr2=0x557d23034c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:10.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.733+0000 7fce0e997740 1 -- 172.21.15.103:0/2077578044 shutdown_connections 2024-09-19T06:50:10.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.733+0000 7fce0e997740 1 -- 172.21.15.103:0/2077578044 wait complete. 2024-09-19T06:50:10.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.733+0000 7fce0e997740 1 -- start start 2024-09-19T06:50:10.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.733+0000 7fce0e997740 1 Processor -- start 2024-09-19T06:50:10.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0e997740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d230385d0 0x557d23042d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0e997740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23039b40 0x557d2303fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0e997740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d23043370 0x557d230405e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0e997740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557d23046070 con 0x557d230385d0 2024-09-19T06:50:10.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0e997740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d22f59060 con 0x557d23043370 2024-09-19T06:50:10.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0e997740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d230461f0 con 0x557d23039b40 2024-09-19T06:50:10.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0e997740 10 client.0 _ll_register_callbacks cb 0x557d22cac8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:10.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0a35a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d230385d0 0x557d23042d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:10.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0a35a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d230385d0 0x557d23042d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46136/0 (socket says 172.21.15.103:46136) 2024-09-19T06:50:10.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0a35a640 1 -- 172.21.15.103:0/2028349156 learned_addr learned my addr 172.21.15.103:0/2028349156 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:10.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0ab5b640 1 --2- 172.21.15.103:0/2028349156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d23043370 0x557d230405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:10.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0a35a640 1 -- 172.21.15.103:0/2028349156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23039b40 msgr2=0x557d2303fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:10.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0a35a640 1 --2- 172.21.15.103:0/2028349156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23039b40 0x557d2303fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0a35a640 1 -- 172.21.15.103:0/2028349156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d23043370 msgr2=0x557d230405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:10.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0a35a640 1 --2- 172.21.15.103:0/2028349156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d23043370 0x557d230405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0a35a640 1 -- 172.21.15.103:0/2028349156 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d22d0f3c0 con 0x557d230385d0 2024-09-19T06:50:10.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0ab5b640 1 --2- 172.21.15.103:0/2028349156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d23043370 0x557d230405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:10.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fce0a35a640 1 --2- 172.21.15.103:0/2028349156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d230385d0 0x557d23042d40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fce000186f0 tx=0x7fce00018720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdd097cd3c2112b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:10.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.734+0000 7fcdedffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:10.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.735+0000 7fcdedffb640 1 -- 172.21.15.103:0/2028349156 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0001d050 con 0x557d230385d0 2024-09-19T06:50:10.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.735+0000 7fcdedffb640 1 -- 172.21.15.103:0/2028349156 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fce00010d10 con 0x557d230385d0 2024-09-19T06:50:10.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.735+0000 7fce0e997740 1 -- 172.21.15.103:0/2028349156 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce00018b20 con 0x557d230385d0 2024-09-19T06:50:10.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.735+0000 7fcdedffb640 1 -- 172.21.15.103:0/2028349156 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fce0000a830 con 0x557d230385d0 2024-09-19T06:50:10.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.735+0000 7fce0e997740 10 client.15735 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:10.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.735+0000 7fce0e997740 1 -- 172.21.15.103:0/2028349156 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557d23051b10 con 0x557d230385d0 2024-09-19T06:50:10.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.735+0000 7fcdedffb640 1 -- 172.21.15.103:0/2028349156 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce00018b20 con 0x557d230385d0 2024-09-19T06:50:10.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.735+0000 7fcdedffb640 1 -- 172.21.15.103:0/2028349156 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fce000046c0 con 0x557d230385d0 2024-09-19T06:50:10.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.736+0000 7fce0e997740 10 client.15735 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:10.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.736+0000 7fce0e997740 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:10.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.736+0000 7fce0e997740 20 client.15735 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:10.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.736+0000 7fce0e997740 1 --2- 172.21.15.103:0/2028349156 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557d23052f70 0x557d23055360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.736+0000 7fce0e997740 4 client.15735 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:10.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.736+0000 7fce0e997740 1 -- 172.21.15.103:0/2028349156 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d22fab860 con 0x557d23052f70 2024-09-19T06:50:10.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.736+0000 7fce09b59640 1 --2- 172.21.15.103:0/2028349156 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557d23052f70 0x557d23055360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:10.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.736+0000 7fce09b59640 1 --2- 172.21.15.103:0/2028349156 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557d23052f70 0x557d23055360 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7edea3869a367b0 server_cookie=dba100eae005f6b4 in_seq=0 out_seq=0 2024-09-19T06:50:10.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.736+0000 7fcdedffb640 10 client.15735 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:10.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.737+0000 7fcdedffb640 1 -- 172.21.15.103:0/2028349156 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d22fab860 con 0x557d23052f70 2024-09-19T06:50:10.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.737+0000 7fcdedffb640 10 client.15735 handle_command_reply: tid=0 2024-09-19T06:50:10.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.737+0000 7fce0e997740 1 client.15735 shutdown 2024-09-19T06:50:10.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.737+0000 7fce0e997740 1 -- 172.21.15.103:0/2028349156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d230385d0 msgr2=0x557d23042d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:10.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.737+0000 7fce0e997740 1 --2- 172.21.15.103:0/2028349156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d230385d0 0x557d23042d40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fce000186f0 tx=0x7fce00018720 comp rx=0 tx=0).stop 2024-09-19T06:50:10.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.737+0000 7fce0e997740 1 -- 172.21.15.103:0/2028349156 shutdown_connections 2024-09-19T06:50:10.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.737+0000 7fce0e997740 1 --2- 172.21.15.103:0/2028349156 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557d23052f70 0x557d23055360 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.737+0000 7fcdedffb640 0 client.15735 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:10.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.738+0000 7fce0e997740 1 --2- 172.21.15.103:0/2028349156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d23043370 0x557d230405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.738+0000 7fce0e997740 1 --2- 172.21.15.103:0/2028349156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23039b40 0x557d2303fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.738+0000 7fce0e997740 1 --2- 172.21.15.103:0/2028349156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d230385d0 0x557d23042d40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.738+0000 7fce0e997740 1 -- 172.21.15.103:0/2028349156 >> 172.21.15.103:0/2028349156 conn(0x557d23001310 msgr2=0x557d2303a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:10.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.738+0000 7fce0e997740 1 -- 172.21.15.103:0/2028349156 shutdown_connections 2024-09-19T06:50:10.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.738+0000 7fce0e997740 1 -- 172.21.15.103:0/2028349156 wait complete. 2024-09-19T06:50:10.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.738+0000 7fce0e997740 20 client.15735 trim_cache size 0 max 16384 2024-09-19T06:50:10.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0e997740 1 Processor -- start 2024-09-19T06:50:10.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0e997740 1 -- start start 2024-09-19T06:50:10.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0e997740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d230385d0 0x557d23033c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0e997740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23039b40 0x557d2303b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0e997740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d23043370 0x557d2303b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0e997740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fce041af8f0 con 0x557d23043370 2024-09-19T06:50:10.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0e997740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce041b1430 con 0x557d230385d0 2024-09-19T06:50:10.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0e997740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fce041affc0 con 0x557d23039b40 2024-09-19T06:50:10.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0ab5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d23043370 0x557d2303b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:10.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0ab5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d23043370 0x557d2303b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46142/0 (socket says 172.21.15.103:46142) 2024-09-19T06:50:10.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0ab5b640 1 -- 172.21.15.103:0/3479040340 learned_addr learned my addr 172.21.15.103:0/3479040340 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:10.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0a35a640 1 --2- 172.21.15.103:0/3479040340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d230385d0 0x557d23033c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:10.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce09b59640 1 --2- 172.21.15.103:0/3479040340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23039b40 0x557d2303b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:10.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0a35a640 1 -- 172.21.15.103:0/3479040340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23039b40 msgr2=0x557d2303b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:10.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0a35a640 1 --2- 172.21.15.103:0/3479040340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23039b40 0x557d2303b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0a35a640 1 -- 172.21.15.103:0/3479040340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d23043370 msgr2=0x557d2303b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:10.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0a35a640 1 --2- 172.21.15.103:0/3479040340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d23043370 0x557d2303b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.751+0000 7fce0a35a640 1 -- 172.21.15.103:0/3479040340 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d2303e540 con 0x557d230385d0 2024-09-19T06:50:10.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.752+0000 7fce0ab5b640 1 --2- 172.21.15.103:0/3479040340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d23043370 0x557d2303b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:10.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.752+0000 7fce0a35a640 1 --2- 172.21.15.103:0/3479040340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d230385d0 0x557d23033c10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fce0000a800 tx=0x7fce0000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d33802cd37a3610d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:10.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.752+0000 7fcdeffff640 1 -- 172.21.15.103:0/3479040340 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0001d040 con 0x557d230385d0 2024-09-19T06:50:10.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.752+0000 7fcdeffff640 1 -- 172.21.15.103:0/3479040340 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce00029070 con 0x557d230385d0 2024-09-19T06:50:10.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.752+0000 7fcdeffff640 1 -- 172.21.15.103:0/3479040340 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00018ae0 con 0x557d230385d0 2024-09-19T06:50:10.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.753+0000 7fce0e997740 1 -- 172.21.15.103:0/3479040340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d230385d0 msgr2=0x557d23033c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:10.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.753+0000 7fce0e997740 1 --2- 172.21.15.103:0/3479040340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d230385d0 0x557d23033c10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fce0000a800 tx=0x7fce0000f790 comp rx=0 tx=0).stop 2024-09-19T06:50:10.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.753+0000 7fce0e997740 1 -- 172.21.15.103:0/3479040340 shutdown_connections 2024-09-19T06:50:10.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.753+0000 7fce0e997740 1 --2- 172.21.15.103:0/3479040340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d23043370 0x557d2303b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.753+0000 7fce0e997740 1 --2- 172.21.15.103:0/3479040340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23039b40 0x557d2303b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.753+0000 7fce0e997740 1 --2- 172.21.15.103:0/3479040340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d230385d0 0x557d23033c10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.753+0000 7fce0e997740 1 -- 172.21.15.103:0/3479040340 >> 172.21.15.103:0/3479040340 conn(0x557d23001310 msgr2=0x557d23033500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:10.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.753+0000 7fce0e997740 1 -- 172.21.15.103:0/3479040340 shutdown_connections 2024-09-19T06:50:10.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.753+0000 7fce0e997740 1 -- 172.21.15.103:0/3479040340 wait complete. 2024-09-19T06:50:10.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.754+0000 7fce0e997740 1 -- start start 2024-09-19T06:50:10.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.754+0000 7fce0e997740 1 Processor -- start 2024-09-19T06:50:10.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.754+0000 7fce0e997740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d230385d0 0x557d2304bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.754+0000 7fce0e997740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23039b40 0x557d2303cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.754+0000 7fce0e997740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d23043370 0x557d2303d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.754+0000 7fce0e997740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fce041b0d60 con 0x557d230385d0 2024-09-19T06:50:10.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.754+0000 7fce0e997740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce041b0690 con 0x557d23043370 2024-09-19T06:50:10.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.754+0000 7fce0e997740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fce041ad8b0 con 0x557d23039b40 2024-09-19T06:50:10.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.754+0000 7fce0e997740 10 client.0 _ll_register_callbacks cb 0x7fce0001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:10.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.754+0000 7fce09b59640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23039b40 0x557d2303cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:10.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.755+0000 7fce09b59640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23039b40 0x557d2303cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:36056/0 (socket says 172.21.15.103:36056) 2024-09-19T06:50:10.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.755+0000 7fce09b59640 1 -- 172.21.15.103:0/3749464985 learned_addr learned my addr 172.21.15.103:0/3749464985 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:10.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.755+0000 7fce0a35a640 1 --2- 172.21.15.103:0/3749464985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d230385d0 0x557d2304bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:10.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.755+0000 7fce09b59640 1 -- 172.21.15.103:0/3749464985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d23043370 msgr2=0x557d2303d120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:50:10.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.755+0000 7fce09b59640 1 --2- 172.21.15.103:0/3749464985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d23043370 0x557d2303d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.755+0000 7fce09b59640 1 -- 172.21.15.103:0/3749464985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d230385d0 msgr2=0x557d2304bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:10.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.755+0000 7fce09b59640 1 --2- 172.21.15.103:0/3749464985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d230385d0 0x557d2304bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.755+0000 7fce09b59640 1 -- 172.21.15.103:0/3749464985 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d2303d7c0 con 0x557d23039b40 2024-09-19T06:50:10.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.755+0000 7fce09b59640 1 --2- 172.21.15.103:0/3749464985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23039b40 0x557d2303cbe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x557d2303f880 tx=0x7fcdf4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d84d32b78747fde7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:10.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.755+0000 7fcdedffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T06:50:10.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.755+0000 7fcdedffb640 1 -- 172.21.15.103:0/3749464985 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4010900 con 0x557d23039b40 2024-09-19T06:50:10.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.755+0000 7fcdedffb640 1 -- 172.21.15.103:0/3749464985 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fcdf4010aa0 con 0x557d23039b40 2024-09-19T06:50:10.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.755+0000 7fce0e997740 1 -- 172.21.15.103:0/3749464985 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce00029070 con 0x557d23039b40 2024-09-19T06:50:10.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.756+0000 7fcdedffb640 1 -- 172.21.15.103:0/3749464985 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcdf4010ca0 con 0x557d23039b40 2024-09-19T06:50:10.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.756+0000 7fce0e997740 10 client.25141 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:10.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.756+0000 7fce0e997740 1 -- 172.21.15.103:0/3749464985 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557d230bd820 con 0x557d23039b40 2024-09-19T06:50:10.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.756+0000 7fcdedffb640 1 -- 172.21.15.103:0/3749464985 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce00029070 con 0x557d23039b40 2024-09-19T06:50:10.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.756+0000 7fcdedffb640 1 -- 172.21.15.103:0/3749464985 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fce00019ce0 con 0x557d23039b40 2024-09-19T06:50:10.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.756+0000 7fce0e997740 10 client.25141 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:10.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.756+0000 7fce0e997740 10 client.25141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:10.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.756+0000 7fce0e997740 20 client.25141 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:10.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.756+0000 7fce0e997740 1 --2- 172.21.15.103:0/3749464985 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557d23052f70 0x557d230bff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:10.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.756+0000 7fce0e997740 4 client.25141 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:10.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.756+0000 7fce0e997740 1 -- 172.21.15.103:0/3749464985 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d22fa5d70 con 0x557d23052f70 2024-09-19T06:50:10.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.757+0000 7fce0a35a640 1 --2- 172.21.15.103:0/3749464985 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557d23052f70 0x557d230bff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:10.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.757+0000 7fcdedffb640 10 client.25141 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:10.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.757+0000 7fce0a35a640 1 --2- 172.21.15.103:0/3749464985 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557d23052f70 0x557d230bff80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb5805b0325eca79 server_cookie=a99ef4e3df463116 in_seq=0 out_seq=0 2024-09-19T06:50:10.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.758+0000 7fcdedffb640 1 -- 172.21.15.103:0/3749464985 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d22fa5d70 con 0x557d23052f70 2024-09-19T06:50:10.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.758+0000 7fcdedffb640 10 client.25141 handle_command_reply: tid=0 2024-09-19T06:50:10.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.758+0000 7fce0e997740 1 client.25141 shutdown 2024-09-19T06:50:10.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.758+0000 7fce0e997740 1 -- 172.21.15.103:0/3749464985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23039b40 msgr2=0x557d2303cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:10.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.758+0000 7fce0e997740 1 --2- 172.21.15.103:0/3749464985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23039b40 0x557d2303cbe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x557d2303f880 tx=0x7fcdf4004730 comp rx=0 tx=0).stop 2024-09-19T06:50:10.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.758+0000 7fce0e997740 1 -- 172.21.15.103:0/3749464985 shutdown_connections 2024-09-19T06:50:10.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.758+0000 7fce0e997740 1 --2- 172.21.15.103:0/3749464985 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557d23052f70 0x557d230bff80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.758+0000 7fce0e997740 1 --2- 172.21.15.103:0/3749464985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d23043370 0x557d2303d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.758+0000 7fce0e997740 1 --2- 172.21.15.103:0/3749464985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d23039b40 0x557d2303cbe0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.758+0000 7fce0e997740 1 --2- 172.21.15.103:0/3749464985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d230385d0 0x557d2304bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.758+0000 7fce0e997740 1 -- 172.21.15.103:0/3749464985 >> 172.21.15.103:0/3749464985 conn(0x557d23001310 msgr2=0x557d23033d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:10.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.758+0000 7fcdedffb640 0 client.25141 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:10.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.758+0000 7fce0e997740 1 -- 172.21.15.103:0/3749464985 shutdown_connections 2024-09-19T06:50:10.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.758+0000 7fce0e997740 1 -- 172.21.15.103:0/3749464985 wait complete. 2024-09-19T06:50:10.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.758+0000 7fce0e997740 20 client.25141 trim_cache size 0 max 16384 2024-09-19T06:50:10.763 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:50:10.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.759+0000 7fce0c5be640 1 -- 172.21.15.103:0/3093447379 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcdf00a3560 msgr2=0x7fcdf00a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:10.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.759+0000 7fce0c5be640 1 --2- 172.21.15.103:0/3093447379 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcdf00a3560 0x7fcdf00a5a20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x557d23039070 tx=0x7fcdf400bf90 comp rx=0 tx=0).stop 2024-09-19T06:50:10.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.759+0000 7fce0c5be640 1 -- 172.21.15.103:0/3093447379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce04102ef0 msgr2=0x7fce04197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:10.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.759+0000 7fce0c5be640 1 --2- 172.21.15.103:0/3093447379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce04102ef0 0x7fce04197b60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc02ae70 tx=0x7fcdfc02f5d0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.759+0000 7fce0c5be640 1 -- 172.21.15.103:0/3093447379 shutdown_connections 2024-09-19T06:50:10.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.759+0000 7fce0c5be640 1 --2- 172.21.15.103:0/3093447379 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcdf00a3560 0x7fcdf00a5a20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.759+0000 7fce0c5be640 1 --2- 172.21.15.103:0/3093447379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce04102ef0 0x7fce04197b60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.759+0000 7fce0c5be640 1 --2- 172.21.15.103:0/3093447379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04102570 0x7fce04197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.759+0000 7fce0c5be640 1 --2- 172.21.15.103:0/3093447379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce04100e20 0x7fce041970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:10.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.759+0000 7fce0c5be640 1 -- 172.21.15.103:0/3093447379 >> 172.21.15.103:0/3093447379 conn(0x7fce040fc7d0 msgr2=0x7fce04106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:10.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.759+0000 7fce0c5be640 1 -- 172.21.15.103:0/3093447379 shutdown_connections 2024-09-19T06:50:10.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:10.759+0000 7fce0c5be640 1 -- 172.21.15.103:0/3093447379 wait complete. 2024-09-19T06:50:11.563 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:50:11.851 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:12.252 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:12 smithi103 ceph-mon[31158]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-19T06:50:12.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:12 smithi174 ceph-mon[35637]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-19T06:50:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.347+0000 7f1404544640 1 -- 172.21.15.103:0/319019963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13fc1009b0 msgr2=0x7f13fc100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.347+0000 7f1404544640 1 --2- 172.21.15.103:0/319019963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13fc1009b0 0x7f13fc100d90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009900 tx=0x7f13f802a410 comp rx=0 tx=0).stop 2024-09-19T06:50:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.347+0000 7f1404544640 1 -- 172.21.15.103:0/319019963 shutdown_connections 2024-09-19T06:50:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.347+0000 7f1404544640 1 --2- 172.21.15.103:0/319019963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13fc10e4d0 0x7f13fc1108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.347+0000 7f1404544640 1 --2- 172.21.15.103:0/319019963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13fc101360 0x7f13fc10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:12.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.347+0000 7f1404544640 1 --2- 172.21.15.103:0/319019963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13fc1009b0 0x7f13fc100d90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:12.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.347+0000 7f1404544640 1 -- 172.21.15.103:0/319019963 >> 172.21.15.103:0/319019963 conn(0x7f13fc0fc7d0 msgr2=0x7f13fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:12.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.348+0000 7f1404544640 1 -- 172.21.15.103:0/319019963 shutdown_connections 2024-09-19T06:50:12.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.348+0000 7f1404544640 1 -- 172.21.15.103:0/319019963 wait complete. 2024-09-19T06:50:12.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.348+0000 7f1404544640 1 Processor -- start 2024-09-19T06:50:12.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.348+0000 7f1404544640 1 -- start start 2024-09-19T06:50:12.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.349+0000 7f1404544640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13fc1009b0 0x7f13fc19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:12.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.349+0000 7f1404544640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13fc101360 0x7f13fc19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:12.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.349+0000 7f1404544640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13fc10e4d0 0x7f13fc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:12.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.349+0000 7f1404544640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc114230 con 0x7f13fc1009b0 2024-09-19T06:50:12.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.349+0000 7f1404544640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc1140b0 con 0x7f13fc10e4d0 2024-09-19T06:50:12.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.349+0000 7f1404544640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc1143b0 con 0x7f13fc101360 2024-09-19T06:50:12.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.349+0000 7f14022e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13fc1009b0 0x7f13fc19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:12.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.349+0000 7f14022e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13fc1009b0 0x7f13fc19a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46156/0 (socket says 172.21.15.103:46156) 2024-09-19T06:50:12.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.349+0000 7f14022e0640 1 -- 172.21.15.103:0/324776778 learned_addr learned my addr 172.21.15.103:0/324776778 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:12.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.349+0000 7f1401adf640 1 --2- 172.21.15.103:0/324776778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13fc101360 0x7f13fc19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:12.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.349+0000 7f14022e0640 1 -- 172.21.15.103:0/324776778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13fc101360 msgr2=0x7f13fc19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:12.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.349+0000 7f14022e0640 1 --2- 172.21.15.103:0/324776778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13fc101360 0x7f13fc19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:12.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.349+0000 7f14022e0640 1 -- 172.21.15.103:0/324776778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13fc10e4d0 msgr2=0x7f13fc19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:12.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.349+0000 7f14022e0640 1 --2- 172.21.15.103:0/324776778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13fc10e4d0 0x7f13fc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:12.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.349+0000 7f14022e0640 1 -- 172.21.15.103:0/324776778 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13fc19f6a0 con 0x7f13fc1009b0 2024-09-19T06:50:12.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.349+0000 7f1401adf640 1 --2- 172.21.15.103:0/324776778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13fc101360 0x7f13fc19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:12.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.350+0000 7f14022e0640 1 --2- 172.21.15.103:0/324776778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13fc1009b0 0x7f13fc19a6f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f13f8006010 tx=0x7f13f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:12.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.350+0000 7f13f37fe640 1 -- 172.21.15.103:0/324776778 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f803f030 con 0x7f13fc1009b0 2024-09-19T06:50:12.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.350+0000 7f13f37fe640 1 -- 172.21.15.103:0/324776778 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13f8003ce0 con 0x7f13fc1009b0 2024-09-19T06:50:12.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.350+0000 7f13f37fe640 1 -- 172.21.15.103:0/324776778 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8035070 con 0x7f13fc1009b0 2024-09-19T06:50:12.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.350+0000 7f1404544640 1 -- 172.21.15.103:0/324776778 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13fc19f930 con 0x7f13fc1009b0 2024-09-19T06:50:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.350+0000 7f1404544640 1 -- 172.21.15.103:0/324776778 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13fc1a7270 con 0x7f13fc1009b0 2024-09-19T06:50:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.351+0000 7f1404544640 1 -- 172.21.15.103:0/324776778 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13fc06a7f0 con 0x7f13fc1009b0 2024-09-19T06:50:12.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.353+0000 7f13f37fe640 1 -- 172.21.15.103:0/324776778 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7f13f8042030 con 0x7f13fc1009b0 2024-09-19T06:50:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.353+0000 7f13f37fe640 1 --2- 172.21.15.103:0/324776778 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f13e80a3660 0x7f13e80a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.353+0000 7f13f37fe640 1 -- 172.21.15.103:0/324776778 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f13f80f0410 con 0x7f13fc1009b0 2024-09-19T06:50:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.354+0000 7f1401adf640 1 --2- 172.21.15.103:0/324776778 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f13e80a3660 0x7f13e80a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:12.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.354+0000 7f1401adf640 1 --2- 172.21.15.103:0/324776778 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f13e80a3660 0x7f13e80a5b20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f13fc102860 tx=0x7f13ec005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:12.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.361+0000 7f13f37fe640 1 -- 172.21.15.103:0/324776778 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f13f80bcbc0 con 0x7f13fc1009b0 2024-09-19T06:50:12.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:12 smithi134 ceph-mon[35356]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-19T06:50:12.467 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:50:12.467 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:50:12.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.662+0000 7f1404544640 1 -- 172.21.15.103:0/324776778 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13fc109660 con 0x7f13fc1009b0 2024-09-19T06:50:12.664 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:50:12.665 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:50:12.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.663+0000 7f13f37fe640 1 -- 172.21.15.103:0/324776778 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f13f80bc4c0 con 0x7f13fc1009b0 2024-09-19T06:50:12.665 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:50:12.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.666+0000 7f1404544640 1 -- 172.21.15.103:0/324776778 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f13e80a3660 msgr2=0x7f13e80a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:12.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.666+0000 7f1404544640 1 --2- 172.21.15.103:0/324776778 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f13e80a3660 0x7f13e80a5b20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f13fc102860 tx=0x7f13ec005f90 comp rx=0 tx=0).stop 2024-09-19T06:50:12.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.666+0000 7f1404544640 1 -- 172.21.15.103:0/324776778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13fc1009b0 msgr2=0x7f13fc19a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:12.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.666+0000 7f1404544640 1 --2- 172.21.15.103:0/324776778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13fc1009b0 0x7f13fc19a6f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f13f8006010 tx=0x7f13f80042f0 comp rx=0 tx=0).stop 2024-09-19T06:50:12.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.667+0000 7f1404544640 1 -- 172.21.15.103:0/324776778 shutdown_connections 2024-09-19T06:50:12.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.667+0000 7f1404544640 1 --2- 172.21.15.103:0/324776778 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f13e80a3660 0x7f13e80a5b20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:12.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.667+0000 7f1404544640 1 --2- 172.21.15.103:0/324776778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13fc10e4d0 0x7f13fc19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:12.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.667+0000 7f1404544640 1 --2- 172.21.15.103:0/324776778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13fc101360 0x7f13fc19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:12.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.667+0000 7f1404544640 1 --2- 172.21.15.103:0/324776778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13fc1009b0 0x7f13fc19a6f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:12.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.667+0000 7f1404544640 1 -- 172.21.15.103:0/324776778 >> 172.21.15.103:0/324776778 conn(0x7f13fc0fc7d0 msgr2=0x7f13fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:12.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.667+0000 7f1404544640 1 -- 172.21.15.103:0/324776778 shutdown_connections 2024-09-19T06:50:12.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:12.667+0000 7f1404544640 1 -- 172.21.15.103:0/324776778 wait complete. 2024-09-19T06:50:13.309 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:13 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/324776778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:13.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:13 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/324776778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:13.352 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 29305e6a --release --await' 2024-09-19T06:50:13.352 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 29305e6a --release --await 2024-09-19T06:50:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:13.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:13 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/324776778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:13.942 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:13.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.948+0000 7fe5c0c5a640 1 -- 172.21.15.103:0/3673253581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5bc106b80 msgr2=0x7fe5bc106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:13.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.948+0000 7fe5c0c5a640 1 --2- 172.21.15.103:0/3673253581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5bc106b80 0x7fe5bc106f60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009900 tx=0x7fe5ac02a410 comp rx=0 tx=0).stop 2024-09-19T06:50:13.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.948+0000 7fe5c0c5a640 1 -- 172.21.15.103:0/3673253581 shutdown_connections 2024-09-19T06:50:13.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.948+0000 7fe5c0c5a640 1 --2- 172.21.15.103:0/3673253581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5bc100aa0 0x7fe5bc10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.948+0000 7fe5c0c5a640 1 --2- 172.21.15.103:0/3673253581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5bc100100 0x7fe5bc100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.948+0000 7fe5c0c5a640 1 --2- 172.21.15.103:0/3673253581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5bc106b80 0x7fe5bc106f60 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.948+0000 7fe5c0c5a640 1 -- 172.21.15.103:0/3673253581 >> 172.21.15.103:0/3673253581 conn(0x7fe5bc077230 msgr2=0x7fe5bc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:13.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.948+0000 7fe5c0c5a640 1 -- 172.21.15.103:0/3673253581 shutdown_connections 2024-09-19T06:50:13.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.948+0000 7fe5c0c5a640 1 -- 172.21.15.103:0/3673253581 wait complete. 2024-09-19T06:50:13.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.949+0000 7fe5c0c5a640 1 Processor -- start 2024-09-19T06:50:13.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.949+0000 7fe5c0c5a640 1 -- start start 2024-09-19T06:50:13.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.950+0000 7fe5c0c5a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5bc100100 0x7fe5bc1a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.950+0000 7fe5c0c5a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5bc100aa0 0x7fe5bc1a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.950+0000 7fe5c0c5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5bc106b80 0x7fe5bc10db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.950+0000 7fe5c0c5a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1120a0 con 0x7fe5bc106b80 2024-09-19T06:50:13.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.950+0000 7fe5c0c5a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc111f20 con 0x7fe5bc100aa0 2024-09-19T06:50:13.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.950+0000 7fe5c0c5a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc112220 con 0x7fe5bc100100 2024-09-19T06:50:13.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.950+0000 7fe5bad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5bc106b80 0x7fe5bc10db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:13.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.950+0000 7fe5bad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5bc106b80 0x7fe5bc10db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46166/0 (socket says 172.21.15.103:46166) 2024-09-19T06:50:13.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.950+0000 7fe5bad9d640 1 -- 172.21.15.103:0/3287200483 learned_addr learned my addr 172.21.15.103:0/3287200483 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:13.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.950+0000 7fe5ba59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5bc100100 0x7fe5bc1a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:13.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.950+0000 7fe5bad9d640 1 -- 172.21.15.103:0/3287200483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5bc100100 msgr2=0x7fe5bc1a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:13.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.950+0000 7fe5b9d9b640 1 --2- 172.21.15.103:0/3287200483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5bc100aa0 0x7fe5bc1a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:13.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.951+0000 7fe5bad9d640 1 --2- 172.21.15.103:0/3287200483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5bc100100 0x7fe5bc1a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.951+0000 7fe5bad9d640 1 -- 172.21.15.103:0/3287200483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5bc100aa0 msgr2=0x7fe5bc1a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:13.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.951+0000 7fe5bad9d640 1 --2- 172.21.15.103:0/3287200483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5bc100aa0 0x7fe5bc1a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.951+0000 7fe5bad9d640 1 -- 172.21.15.103:0/3287200483 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5bc10e380 con 0x7fe5bc106b80 2024-09-19T06:50:13.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.951+0000 7fe5ba59c640 1 --2- 172.21.15.103:0/3287200483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5bc100100 0x7fe5bc1a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:13.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.951+0000 7fe5b9d9b640 1 --2- 172.21.15.103:0/3287200483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5bc100aa0 0x7fe5bc1a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:13.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.951+0000 7fe5bad9d640 1 --2- 172.21.15.103:0/3287200483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5bc106b80 0x7fe5bc10db50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400e9d0 tx=0x7fe5b400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:13.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.951+0000 7fe5ab7fe640 1 -- 172.21.15.103:0/3287200483 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b400cd80 con 0x7fe5bc106b80 2024-09-19T06:50:13.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.952+0000 7fe5ab7fe640 1 -- 172.21.15.103:0/3287200483 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b4005020 con 0x7fe5bc106b80 2024-09-19T06:50:13.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.952+0000 7fe5ab7fe640 1 -- 172.21.15.103:0/3287200483 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b40116d0 con 0x7fe5bc106b80 2024-09-19T06:50:13.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.952+0000 7fe5c0c5a640 1 -- 172.21.15.103:0/3287200483 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5bc10e670 con 0x7fe5bc106b80 2024-09-19T06:50:13.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.953+0000 7fe5c0c5a640 1 -- 172.21.15.103:0/3287200483 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5bc1ab5c0 con 0x7fe5bc106b80 2024-09-19T06:50:13.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.956+0000 7fe5c3033740 1 Processor -- start 2024-09-19T06:50:13.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.956+0000 7fe5ab7fe640 1 -- 172.21.15.103:0/3287200483 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7fe5b40051d0 con 0x7fe5bc106b80 2024-09-19T06:50:13.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.957+0000 7fe5c3033740 1 -- start start 2024-09-19T06:50:13.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.957+0000 7fe5c3033740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a68f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.957+0000 7fe5c3033740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a69480 0x55dc60a73800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.957+0000 7fe5b9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a68f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:13.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.957+0000 7fe5c3033740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a73d40 0x55dc60a76130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.957+0000 7fe5c3033740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dc609ce280 con 0x55dc60a68b60 2024-09-19T06:50:13.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.957+0000 7fe5c3033740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dc60975b10 con 0x55dc60a73d40 2024-09-19T06:50:13.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.957+0000 7fe5c3033740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc609ca240 con 0x55dc60a69480 2024-09-19T06:50:13.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.957+0000 7fe5b9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a68f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46168/0 (socket says 172.21.15.103:46168) 2024-09-19T06:50:13.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.957+0000 7fe5b9d9b640 1 -- 172.21.15.103:0/4002113704 learned_addr learned my addr 172.21.15.103:0/4002113704 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:13.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.957+0000 7fe5ba59c640 1 --2- 172.21.15.103:0/4002113704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a69480 0x55dc60a73800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:13.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.958+0000 7fe5b9d9b640 1 -- 172.21.15.103:0/4002113704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a69480 msgr2=0x55dc60a73800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:13.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.958+0000 7fe5b9d9b640 1 --2- 172.21.15.103:0/4002113704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a69480 0x55dc60a73800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.958+0000 7fe5b9d9b640 1 -- 172.21.15.103:0/4002113704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a73d40 msgr2=0x55dc60a76130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:50:13.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.958+0000 7fe5b9d9b640 1 --2- 172.21.15.103:0/4002113704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a73d40 0x55dc60a76130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.958+0000 7fe5b9d9b640 1 -- 172.21.15.103:0/4002113704 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc609821f0 con 0x55dc60a68b60 2024-09-19T06:50:13.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.958+0000 7fe5ab7fe640 1 --2- 172.21.15.103:0/3287200483 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe5a40a3660 0x7fe5a40a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.958+0000 7fe5b9d9b640 1 --2- 172.21.15.103:0/4002113704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a68f40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009b90 tx=0x7fe5b000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd043c586e56ca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:13.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.958+0000 7fe5ab7fe640 1 -- 172.21.15.103:0/3287200483 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fe5b40d0450 con 0x7fe5bc106b80 2024-09-19T06:50:13.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.959+0000 7fe5a8ff9640 1 -- 172.21.15.103:0/4002113704 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0016070 con 0x55dc60a68b60 2024-09-19T06:50:13.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.959+0000 7fe5a8ff9640 1 -- 172.21.15.103:0/4002113704 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b0011950 con 0x55dc60a68b60 2024-09-19T06:50:13.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.959+0000 7fe5a8ff9640 1 -- 172.21.15.103:0/4002113704 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0010890 con 0x55dc60a68b60 2024-09-19T06:50:13.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.959+0000 7fe5ba59c640 1 --2- 172.21.15.103:0/3287200483 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe5a40a3660 0x7fe5a40a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:13.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.960+0000 7fe5c3033740 1 -- 172.21.15.103:0/4002113704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 msgr2=0x55dc60a68f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:13.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.960+0000 7fe5ba59c640 1 --2- 172.21.15.103:0/3287200483 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe5a40a3660 0x7fe5a40a5b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac000c00 tx=0x7fe5ac031510 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:13.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.960+0000 7fe5c3033740 1 --2- 172.21.15.103:0/4002113704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a68f40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009b90 tx=0x7fe5b000e6c0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.960+0000 7fe5c3033740 1 -- 172.21.15.103:0/4002113704 shutdown_connections 2024-09-19T06:50:13.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.960+0000 7fe5c3033740 1 --2- 172.21.15.103:0/4002113704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a73d40 0x55dc60a76130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.960+0000 7fe5c3033740 1 --2- 172.21.15.103:0/4002113704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a69480 0x55dc60a73800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.960+0000 7fe5c3033740 1 --2- 172.21.15.103:0/4002113704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a68f40 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.960+0000 7fe5c3033740 1 -- 172.21.15.103:0/4002113704 >> 172.21.15.103:0/4002113704 conn(0x55dc60a650f0 msgr2=0x55dc60a654d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:13.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.960+0000 7fe5c3033740 1 -- 172.21.15.103:0/4002113704 shutdown_connections 2024-09-19T06:50:13.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.960+0000 7fe5c3033740 1 -- 172.21.15.103:0/4002113704 wait complete. 2024-09-19T06:50:13.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.961+0000 7fe5c3033740 1 -- start start 2024-09-19T06:50:13.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.961+0000 7fe5c3033740 1 Processor -- start 2024-09-19T06:50:13.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.962+0000 7fe5c3033740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a68b60 0x55dc60a70bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.962+0000 7fe5c3033740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a69480 0x55dc60a71130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.962+0000 7fe5c3033740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a73d40 0x55dc60a71670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.962+0000 7fe5c3033740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dc60a76910 con 0x55dc60a73d40 2024-09-19T06:50:13.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.962+0000 7fe5c3033740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dc60988f10 con 0x55dc60a69480 2024-09-19T06:50:13.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.962+0000 7fe5c3033740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc60a76a90 con 0x55dc60a68b60 2024-09-19T06:50:13.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.962+0000 7fe5c3033740 10 client.0 _ll_register_callbacks cb 0x55dc60951a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:13.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.962+0000 7fe5bad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a73d40 0x55dc60a71670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:13.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.962+0000 7fe5bad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a73d40 0x55dc60a71670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46182/0 (socket says 172.21.15.103:46182) 2024-09-19T06:50:13.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.962+0000 7fe5bad9d640 1 -- 172.21.15.103:0/1561725835 learned_addr learned my addr 172.21.15.103:0/1561725835 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:13.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.963+0000 7fe5b9d9b640 1 --2- 172.21.15.103:0/1561725835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a69480 0x55dc60a71130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:13.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.963+0000 7fe5ba59c640 1 --2- 172.21.15.103:0/1561725835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a68b60 0x55dc60a70bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:13.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.963+0000 7fe5bad9d640 1 -- 172.21.15.103:0/1561725835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a68b60 msgr2=0x55dc60a70bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:13.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.963+0000 7fe5bad9d640 1 --2- 172.21.15.103:0/1561725835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a68b60 0x55dc60a70bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.963+0000 7fe5bad9d640 1 -- 172.21.15.103:0/1561725835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a69480 msgr2=0x55dc60a71130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:13.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.963+0000 7fe5bad9d640 1 --2- 172.21.15.103:0/1561725835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a69480 0x55dc60a71130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.963+0000 7fe5bad9d640 1 -- 172.21.15.103:0/1561725835 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc6073f6a0 con 0x55dc60a73d40 2024-09-19T06:50:13.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.963+0000 7fe5ba59c640 1 --2- 172.21.15.103:0/1561725835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a68b60 0x55dc60a70bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:13.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.963+0000 7fe5bad9d640 1 --2- 172.21.15.103:0/1561725835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a73d40 0x55dc60a71670 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4098ed0 tx=0x7fe5b409bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6087eaa0745b5d82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:13.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.963+0000 7fe5a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:13.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.963+0000 7fe5a1ffb640 1 -- 172.21.15.103:0/1561725835 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b40b4070 con 0x55dc60a73d40 2024-09-19T06:50:13.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.964+0000 7fe5c3033740 1 -- 172.21.15.103:0/1561725835 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b0011950 con 0x55dc60a73d40 2024-09-19T06:50:13.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.964+0000 7fe5a1ffb640 1 -- 172.21.15.103:0/1561725835 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fe5b40b5460 con 0x55dc60a73d40 2024-09-19T06:50:13.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.964+0000 7fe5a1ffb640 1 -- 172.21.15.103:0/1561725835 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5b40a5170 con 0x55dc60a73d40 2024-09-19T06:50:13.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.964+0000 7fe5c3033740 10 client.15759 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:13.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.964+0000 7fe5c3033740 1 -- 172.21.15.103:0/1561725835 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dc60a824e0 con 0x55dc60a73d40 2024-09-19T06:50:13.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.965+0000 7fe5a1ffb640 1 -- 172.21.15.103:0/1561725835 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b0011950 con 0x55dc60a73d40 2024-09-19T06:50:13.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.965+0000 7fe5a1ffb640 1 -- 172.21.15.103:0/1561725835 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe5b400f040 con 0x55dc60a73d40 2024-09-19T06:50:13.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.965+0000 7fe5c3033740 10 client.15759 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:13.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.965+0000 7fe5c3033740 10 client.15759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:13.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.965+0000 7fe5c3033740 20 client.15759 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:13.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.965+0000 7fe5c3033740 1 --2- 172.21.15.103:0/1561725835 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dc60a83eb0 0x55dc60a86310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.965+0000 7fe5c3033740 4 client.15759 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:13.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.965+0000 7fe5c3033740 1 -- 172.21.15.103:0/1561725835 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc609db7f0 con 0x55dc60a83eb0 2024-09-19T06:50:13.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.965+0000 7fe5ba59c640 1 --2- 172.21.15.103:0/1561725835 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dc60a83eb0 0x55dc60a86310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:13.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.966+0000 7fe5ba59c640 1 --2- 172.21.15.103:0/1561725835 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dc60a83eb0 0x55dc60a86310 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d05c399bec26e14f server_cookie=264c16a50195a547 in_seq=0 out_seq=0 2024-09-19T06:50:13.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.966+0000 7fe5a1ffb640 10 client.15759 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:13.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.966+0000 7fe5a1ffb640 1 -- 172.21.15.103:0/1561725835 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc609db7f0 con 0x55dc60a83eb0 2024-09-19T06:50:13.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.967+0000 7fe5a1ffb640 10 client.15759 handle_command_reply: tid=0 2024-09-19T06:50:13.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.967+0000 7fe5c3033740 1 client.15759 shutdown 2024-09-19T06:50:13.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.967+0000 7fe5c3033740 1 -- 172.21.15.103:0/1561725835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a73d40 msgr2=0x55dc60a71670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:13.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.967+0000 7fe5c3033740 1 --2- 172.21.15.103:0/1561725835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a73d40 0x55dc60a71670 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4098ed0 tx=0x7fe5b409bf50 comp rx=0 tx=0).stop 2024-09-19T06:50:13.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.967+0000 7fe5c3033740 1 -- 172.21.15.103:0/1561725835 shutdown_connections 2024-09-19T06:50:13.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.967+0000 7fe5c3033740 1 --2- 172.21.15.103:0/1561725835 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dc60a83eb0 0x55dc60a86310 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.967+0000 7fe5c3033740 1 --2- 172.21.15.103:0/1561725835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a73d40 0x55dc60a71670 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.967+0000 7fe5c3033740 1 --2- 172.21.15.103:0/1561725835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a69480 0x55dc60a71130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.967+0000 7fe5c3033740 1 --2- 172.21.15.103:0/1561725835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a68b60 0x55dc60a70bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.967+0000 7fe5c3033740 1 -- 172.21.15.103:0/1561725835 >> 172.21.15.103:0/1561725835 conn(0x55dc60a650f0 msgr2=0x55dc60a6ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:13.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.967+0000 7fe5a1ffb640 0 client.15759 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:13.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.967+0000 7fe5c3033740 1 -- 172.21.15.103:0/1561725835 shutdown_connections 2024-09-19T06:50:13.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.967+0000 7fe5c3033740 1 -- 172.21.15.103:0/1561725835 wait complete. 2024-09-19T06:50:13.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.968+0000 7fe5c3033740 20 client.15759 trim_cache size 0 max 16384 2024-09-19T06:50:13.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.986+0000 7fe5c3033740 1 Processor -- start 2024-09-19T06:50:13.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.986+0000 7fe5c3033740 1 -- start start 2024-09-19T06:50:13.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.986+0000 7fe5c3033740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a6b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.986+0000 7fe5c3033740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a69480 0x55dc60a72a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.986+0000 7fe5c3033740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a73d40 0x55dc60a8bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.986+0000 7fe5c3033740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1b3e80 con 0x55dc60a68b60 2024-09-19T06:50:13.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.986+0000 7fe5c3033740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1b59c0 con 0x55dc60a69480 2024-09-19T06:50:13.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.986+0000 7fe5c3033740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1b4550 con 0x55dc60a73d40 2024-09-19T06:50:13.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.986+0000 7fe5b9d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a69480 0x55dc60a72a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:13.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.986+0000 7fe5b9d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a69480 0x55dc60a72a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:48218/0 (socket says 172.21.15.103:48218) 2024-09-19T06:50:13.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.986+0000 7fe5b9d9b640 1 -- 172.21.15.103:0/1006453289 learned_addr learned my addr 172.21.15.103:0/1006453289 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:13.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.987+0000 7fe5ba59c640 1 --2- 172.21.15.103:0/1006453289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a6b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:13.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.987+0000 7fe5bad9d640 1 --2- 172.21.15.103:0/1006453289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a73d40 0x55dc60a8bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:13.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.987+0000 7fe5ba59c640 1 -- 172.21.15.103:0/1006453289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a73d40 msgr2=0x55dc60a8bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:13.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.987+0000 7fe5ba59c640 1 --2- 172.21.15.103:0/1006453289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a73d40 0x55dc60a8bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.987+0000 7fe5ba59c640 1 -- 172.21.15.103:0/1006453289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a69480 msgr2=0x55dc60a72a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:13.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.987+0000 7fe5ba59c640 1 --2- 172.21.15.103:0/1006453289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a69480 0x55dc60a72a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.987+0000 7fe5ba59c640 1 -- 172.21.15.103:0/1006453289 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc60a60830 con 0x55dc60a68b60 2024-09-19T06:50:13.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.987+0000 7fe5ba59c640 1 --2- 172.21.15.103:0/1006453289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a6b5c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac02aa00 tx=0x7fe5ac03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ffa2b4f42a62e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:13.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.988+0000 7fe5a3fff640 1 -- 172.21.15.103:0/1006453289 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac057070 con 0x55dc60a68b60 2024-09-19T06:50:13.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.988+0000 7fe5a3fff640 1 -- 172.21.15.103:0/1006453289 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5ac004e40 con 0x55dc60a68b60 2024-09-19T06:50:13.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.988+0000 7fe5a3fff640 1 -- 172.21.15.103:0/1006453289 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac035040 con 0x55dc60a68b60 2024-09-19T06:50:13.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.989+0000 7fe5c3033740 1 -- 172.21.15.103:0/1006453289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 msgr2=0x55dc60a6b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:13.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.989+0000 7fe5c3033740 1 --2- 172.21.15.103:0/1006453289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a6b5c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac02aa00 tx=0x7fe5ac03f040 comp rx=0 tx=0).stop 2024-09-19T06:50:13.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.990+0000 7fe5c3033740 1 -- 172.21.15.103:0/1006453289 shutdown_connections 2024-09-19T06:50:13.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.990+0000 7fe5c3033740 1 --2- 172.21.15.103:0/1006453289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a73d40 0x55dc60a8bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.990+0000 7fe5c3033740 1 --2- 172.21.15.103:0/1006453289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a69480 0x55dc60a72a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.990+0000 7fe5c3033740 1 --2- 172.21.15.103:0/1006453289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a6b5c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.990+0000 7fe5c3033740 1 -- 172.21.15.103:0/1006453289 >> 172.21.15.103:0/1006453289 conn(0x55dc60a650f0 msgr2=0x55dc60a72340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:13.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.991+0000 7fe5c3033740 1 -- 172.21.15.103:0/1006453289 shutdown_connections 2024-09-19T06:50:13.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.991+0000 7fe5c3033740 1 -- 172.21.15.103:0/1006453289 wait complete. 2024-09-19T06:50:13.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.991+0000 7fe5c3033740 1 -- start start 2024-09-19T06:50:13.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.991+0000 7fe5c3033740 1 Processor -- start 2024-09-19T06:50:13.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.991+0000 7fe5c3033740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a7c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.991+0000 7fe5c3033740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a69480 0x55dc60a776b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.991+0000 7fe5c3033740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a73d40 0x55dc60a77bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.992+0000 7fe5c3033740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1b52f0 con 0x55dc60a68b60 2024-09-19T06:50:13.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.992+0000 7fe5c3033740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1b4c20 con 0x55dc60a69480 2024-09-19T06:50:13.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.992+0000 7fe5c3033740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1b1e40 con 0x55dc60a73d40 2024-09-19T06:50:13.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.992+0000 7fe5ba59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a7c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:13.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.992+0000 7fe5c3033740 10 client.0 _ll_register_callbacks cb 0x7fe5b40980e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:13.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.992+0000 7fe5ba59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a7c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46204/0 (socket says 172.21.15.103:46204) 2024-09-19T06:50:13.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.992+0000 7fe5ba59c640 1 -- 172.21.15.103:0/3097272562 learned_addr learned my addr 172.21.15.103:0/3097272562 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:13.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.992+0000 7fe5bad9d640 1 --2- 172.21.15.103:0/3097272562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a73d40 0x55dc60a77bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:13.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.992+0000 7fe5ba59c640 1 -- 172.21.15.103:0/3097272562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a73d40 msgr2=0x55dc60a77bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:13.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.992+0000 7fe5ba59c640 1 --2- 172.21.15.103:0/3097272562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a73d40 0x55dc60a77bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.992+0000 7fe5ba59c640 1 -- 172.21.15.103:0/3097272562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a69480 msgr2=0x55dc60a776b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:50:13.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.992+0000 7fe5ba59c640 1 --2- 172.21.15.103:0/3097272562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a69480 0x55dc60a776b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:13.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.992+0000 7fe5ba59c640 1 -- 172.21.15.103:0/3097272562 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc60a78420 con 0x55dc60a68b60 2024-09-19T06:50:13.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.992+0000 7fe5ba59c640 1 --2- 172.21.15.103:0/3097272562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a7c380 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0050d0 tx=0x7fe5ac005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1140e68132b6f7a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:13.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.992+0000 7fe5a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:13.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.993+0000 7fe5a1ffb640 1 -- 172.21.15.103:0/3097272562 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac057070 con 0x55dc60a68b60 2024-09-19T06:50:13.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.993+0000 7fe5a1ffb640 1 -- 172.21.15.103:0/3097272562 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fe5ac004960 con 0x55dc60a68b60 2024-09-19T06:50:13.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.993+0000 7fe5c3033740 1 -- 172.21.15.103:0/3097272562 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5ac004e40 con 0x55dc60a68b60 2024-09-19T06:50:13.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.993+0000 7fe5a1ffb640 1 -- 172.21.15.103:0/3097272562 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5ac004b60 con 0x55dc60a68b60 2024-09-19T06:50:13.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.993+0000 7fe5c3033740 10 client.15765 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:13.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.993+0000 7fe5c3033740 1 -- 172.21.15.103:0/3097272562 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dc60aeeba0 con 0x55dc60a68b60 2024-09-19T06:50:13.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.994+0000 7fe5a1ffb640 1 -- 172.21.15.103:0/3097272562 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5ac0317c0 con 0x55dc60a68b60 2024-09-19T06:50:13.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.994+0000 7fe5a1ffb640 1 -- 172.21.15.103:0/3097272562 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe5ac044040 con 0x55dc60a68b60 2024-09-19T06:50:13.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.994+0000 7fe5c3033740 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:13.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.994+0000 7fe5c3033740 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:13.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.994+0000 7fe5c3033740 20 client.15765 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:13.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.994+0000 7fe5c3033740 1 --2- 172.21.15.103:0/3097272562 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dc60aef1c0 0x55dc60af15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:13.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.994+0000 7fe5c3033740 4 client.15765 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:50:13.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.994+0000 7fe5c3033740 1 -- 172.21.15.103:0/3097272562 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fe5ac004e40 con 0x55dc60aef1c0 2024-09-19T06:50:13.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.994+0000 7fe5b9d9b640 1 --2- 172.21.15.103:0/3097272562 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dc60aef1c0 0x55dc60af15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:13.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.995+0000 7fe5a1ffb640 10 client.15765 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:13.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.995+0000 7fe5b9d9b640 1 --2- 172.21.15.103:0/3097272562 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dc60aef1c0 0x55dc60af15e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2382db7eba25d5ca server_cookie=a350c02d07b148b8 in_seq=0 out_seq=0 2024-09-19T06:50:13.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.998+0000 7fe5a1ffb640 1 -- 172.21.15.103:0/3097272562 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fe5ac004e40 con 0x55dc60aef1c0 2024-09-19T06:50:13.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.998+0000 7fe5a1ffb640 10 client.15765 handle_command_reply: tid=0 2024-09-19T06:50:13.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.998+0000 7fe5c3033740 1 client.15765 shutdown 2024-09-19T06:50:14.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.998+0000 7fe5c3033740 1 -- 172.21.15.103:0/3097272562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 msgr2=0x55dc60a7c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:14.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.998+0000 7fe5c3033740 1 --2- 172.21.15.103:0/3097272562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a7c380 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0050d0 tx=0x7fe5ac005c00 comp rx=0 tx=0).stop 2024-09-19T06:50:14.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.998+0000 7fe5c3033740 1 -- 172.21.15.103:0/3097272562 shutdown_connections 2024-09-19T06:50:14.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.998+0000 7fe5c3033740 1 --2- 172.21.15.103:0/3097272562 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dc60aef1c0 0x55dc60af15e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.999+0000 7fe5a1ffb640 0 client.15765 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:14.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.999+0000 7fe5c3033740 1 --2- 172.21.15.103:0/3097272562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc60a73d40 0x55dc60a77bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.999+0000 7fe5c3033740 1 --2- 172.21.15.103:0/3097272562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc60a69480 0x55dc60a776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.999+0000 7fe5c3033740 1 --2- 172.21.15.103:0/3097272562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc60a68b60 0x55dc60a7c380 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.999+0000 7fe5c3033740 1 -- 172.21.15.103:0/3097272562 >> 172.21.15.103:0/3097272562 conn(0x55dc60a650f0 msgr2=0x55dc60a64270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:14.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.999+0000 7fe5c3033740 1 -- 172.21.15.103:0/3097272562 shutdown_connections 2024-09-19T06:50:14.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.999+0000 7fe5c3033740 1 -- 172.21.15.103:0/3097272562 wait complete. 2024-09-19T06:50:14.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:13.999+0000 7fe5c3033740 20 client.15765 trim_cache size 0 max 16384 2024-09-19T06:50:14.001 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:50:14.001 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T06:50:14.001 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "a53f0bfd-0260-489f-a94e-b6b432c3b071", 2024-09-19T06:50:14.001 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T06:50:14.001 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:50:14.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.001+0000 7fe5c0c5a640 1 -- 172.21.15.103:0/3287200483 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe5a40a3660 msgr2=0x7fe5a40a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:14.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.001+0000 7fe5c0c5a640 1 --2- 172.21.15.103:0/3287200483 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe5a40a3660 0x7fe5a40a5b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac000c00 tx=0x7fe5ac031510 comp rx=0 tx=0).stop 2024-09-19T06:50:14.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.001+0000 7fe5c0c5a640 1 -- 172.21.15.103:0/3287200483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5bc106b80 msgr2=0x7fe5bc10db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:14.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.001+0000 7fe5c0c5a640 1 --2- 172.21.15.103:0/3287200483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5bc106b80 0x7fe5bc10db50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400e9d0 tx=0x7fe5b400ee90 comp rx=0 tx=0).stop 2024-09-19T06:50:14.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.001+0000 7fe5c0c5a640 1 -- 172.21.15.103:0/3287200483 shutdown_connections 2024-09-19T06:50:14.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.001+0000 7fe5c0c5a640 1 --2- 172.21.15.103:0/3287200483 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe5a40a3660 0x7fe5a40a5b20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.001+0000 7fe5c0c5a640 1 --2- 172.21.15.103:0/3287200483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5bc106b80 0x7fe5bc10db50 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.001+0000 7fe5c0c5a640 1 --2- 172.21.15.103:0/3287200483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5bc100aa0 0x7fe5bc1a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.001+0000 7fe5c0c5a640 1 --2- 172.21.15.103:0/3287200483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5bc100100 0x7fe5bc1a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.001+0000 7fe5c0c5a640 1 -- 172.21.15.103:0/3287200483 >> 172.21.15.103:0/3287200483 conn(0x7fe5bc077230 msgr2=0x7fe5bc104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:14.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.001+0000 7fe5c0c5a640 1 -- 172.21.15.103:0/3287200483 shutdown_connections 2024-09-19T06:50:14.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.001+0000 7fe5c0c5a640 1 -- 172.21.15.103:0/3287200483 wait complete. 2024-09-19T06:50:14.215 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:14 smithi103 ceph-mon[31158]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-19T06:50:14.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:14 smithi174 ceph-mon[35637]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-19T06:50:14.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:14 smithi134 ceph-mon[35356]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-19T06:50:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.684+0000 7fd1219b2640 1 -- 172.21.15.103:0/773237411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd11c100aa0 msgr2=0x7fd11c10ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.684+0000 7fd1219b2640 1 --2- 172.21.15.103:0/773237411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd11c100aa0 0x7fd11c10ea20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd114009ca0 tx=0x7fd11402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.685+0000 7fd1219b2640 1 -- 172.21.15.103:0/773237411 shutdown_connections 2024-09-19T06:50:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.685+0000 7fd1219b2640 1 --2- 172.21.15.103:0/773237411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd11c100aa0 0x7fd11c10ea20 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.685+0000 7fd1219b2640 1 --2- 172.21.15.103:0/773237411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd11c100100 0x7fd11c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.685+0000 7fd1219b2640 1 --2- 172.21.15.103:0/773237411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c106b80 0x7fd11c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.685+0000 7fd1219b2640 1 -- 172.21.15.103:0/773237411 >> 172.21.15.103:0/773237411 conn(0x7fd11c077230 msgr2=0x7fd11c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.685+0000 7fd1219b2640 1 -- 172.21.15.103:0/773237411 shutdown_connections 2024-09-19T06:50:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.685+0000 7fd1219b2640 1 -- 172.21.15.103:0/773237411 wait complete. 2024-09-19T06:50:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.686+0000 7fd1219b2640 1 Processor -- start 2024-09-19T06:50:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.686+0000 7fd1219b2640 1 -- start start 2024-09-19T06:50:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.687+0000 7fd1219b2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c100100 0x7fd11c1a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.687+0000 7fd1219b2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd11c100aa0 0x7fd11c1a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.687+0000 7fd1219b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd11c106b80 0x7fd11c10d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.687+0000 7fd11affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c100100 0x7fd11c1a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.687+0000 7fd11affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c100100 0x7fd11c1a4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:48228/0 (socket says 172.21.15.103:48228) 2024-09-19T06:50:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.687+0000 7fd11affd640 1 -- 172.21.15.103:0/582956844 learned_addr learned my addr 172.21.15.103:0/582956844 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.687+0000 7fd1219b2640 1 -- 172.21.15.103:0/582956844 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c1120a0 con 0x7fd11c106b80 2024-09-19T06:50:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.687+0000 7fd1219b2640 1 -- 172.21.15.103:0/582956844 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c111f20 con 0x7fd11c100100 2024-09-19T06:50:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.687+0000 7fd1219b2640 1 -- 172.21.15.103:0/582956844 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c112220 con 0x7fd11c100aa0 2024-09-19T06:50:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.687+0000 7fd11a7fc640 1 --2- 172.21.15.103:0/582956844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd11c100aa0 0x7fd11c1a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.687+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/582956844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd11c106b80 0x7fd11c10d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.687+0000 7fd11affd640 1 -- 172.21.15.103:0/582956844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd11c100aa0 msgr2=0x7fd11c1a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.687+0000 7fd11affd640 1 --2- 172.21.15.103:0/582956844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd11c100aa0 0x7fd11c1a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.687+0000 7fd11affd640 1 -- 172.21.15.103:0/582956844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd11c106b80 msgr2=0x7fd11c10d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.687+0000 7fd11affd640 1 --2- 172.21.15.103:0/582956844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd11c106b80 0x7fd11c10d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.687+0000 7fd11affd640 1 -- 172.21.15.103:0/582956844 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11c10e100 con 0x7fd11c100100 2024-09-19T06:50:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.688+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/582956844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd11c106b80 0x7fd11c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.688+0000 7fd11affd640 1 --2- 172.21.15.103:0/582956844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c100100 0x7fd11c1a4530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00e9f0 tx=0x7fd10c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.688+0000 7fd1209b0640 1 -- 172.21.15.103:0/582956844 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c00cd80 con 0x7fd11c100100 2024-09-19T06:50:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.688+0000 7fd1219b2640 1 -- 172.21.15.103:0/582956844 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11c10e3f0 con 0x7fd11c100100 2024-09-19T06:50:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.688+0000 7fd1209b0640 1 -- 172.21.15.103:0/582956844 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd10c010910 con 0x7fd11c100100 2024-09-19T06:50:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.688+0000 7fd1209b0640 1 -- 172.21.15.103:0/582956844 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c0188f0 con 0x7fd11c100100 2024-09-19T06:50:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.689+0000 7fd1219b2640 1 -- 172.21.15.103:0/582956844 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd11c1ab350 con 0x7fd11c100100 2024-09-19T06:50:14.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.692+0000 7fd1209b0640 1 -- 172.21.15.103:0/582956844 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7fd10c010ac0 con 0x7fd11c100100 2024-09-19T06:50:14.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.692+0000 7fd123d8b740 1 Processor -- start 2024-09-19T06:50:14.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.692+0000 7fd123d8b740 1 -- start start 2024-09-19T06:50:14.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.692+0000 7fd123d8b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197060 0x555e5f197440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.692+0000 7fd123d8b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f197980 0x555e5f1a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.692+0000 7fd123d8b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f1a2240 0x555e5f1a4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.692+0000 7fd123d8b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555e5f0fc620 con 0x555e5f197060 2024-09-19T06:50:14.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.692+0000 7fd123d8b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555e5f0a3e50 con 0x555e5f1a2240 2024-09-19T06:50:14.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.692+0000 7fd123d8b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555e5f0f85e0 con 0x555e5f197980 2024-09-19T06:50:14.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.693+0000 7fd11affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f197980 0x555e5f1a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.693+0000 7fd11b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197060 0x555e5f197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.693+0000 7fd11b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197060 0x555e5f197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46230/0 (socket says 172.21.15.103:46230) 2024-09-19T06:50:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.693+0000 7fd11b7fe640 1 -- 172.21.15.103:0/2317522227 learned_addr learned my addr 172.21.15.103:0/2317522227 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.693+0000 7fd11a7fc640 1 --2- 172.21.15.103:0/2317522227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f1a2240 0x555e5f1a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.694+0000 7fd11b7fe640 1 -- 172.21.15.103:0/2317522227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f197980 msgr2=0x555e5f1a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.694+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/2317522227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f197980 0x555e5f1a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.694+0000 7fd11b7fe640 1 -- 172.21.15.103:0/2317522227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f1a2240 msgr2=0x555e5f1a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.694+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/2317522227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f1a2240 0x555e5f1a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.694+0000 7fd11b7fe640 1 -- 172.21.15.103:0/2317522227 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e5f094d40 con 0x555e5f197060 2024-09-19T06:50:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.694+0000 7fd11a7fc640 1 --2- 172.21.15.103:0/2317522227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f1a2240 0x555e5f1a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.694+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/2317522227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197060 0x555e5f197440 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd114005fa0 tx=0x7fd11402f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e219ab9c7e6451a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:14.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.694+0000 7fd1209b0640 1 --2- 172.21.15.103:0/582956844 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1040a35e0 0x7fd1040a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.694+0000 7fd11affd640 1 --2- 172.21.15.103:0/582956844 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1040a35e0 0x7fd1040a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.694+0000 7fd101ffb640 1 -- 172.21.15.103:0/2317522227 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114031bb0 con 0x555e5f197060 2024-09-19T06:50:14.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.695+0000 7fd1209b0640 1 -- 172.21.15.103:0/582956844 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd10c0d03a0 con 0x7fd11c100100 2024-09-19T06:50:14.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.695+0000 7fd101ffb640 1 -- 172.21.15.103:0/2317522227 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd114036040 con 0x555e5f197060 2024-09-19T06:50:14.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.695+0000 7fd101ffb640 1 -- 172.21.15.103:0/2317522227 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114004040 con 0x555e5f197060 2024-09-19T06:50:14.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.696+0000 7fd123d8b740 1 -- 172.21.15.103:0/2317522227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197060 msgr2=0x555e5f197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:14.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.696+0000 7fd11affd640 1 --2- 172.21.15.103:0/582956844 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1040a35e0 0x7fd1040a5aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd10c0cd920 tx=0x7fd10c0d3e30 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:14.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.696+0000 7fd123d8b740 1 --2- 172.21.15.103:0/2317522227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197060 0x555e5f197440 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd114005fa0 tx=0x7fd11402f310 comp rx=0 tx=0).stop 2024-09-19T06:50:14.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.696+0000 7fd123d8b740 1 -- 172.21.15.103:0/2317522227 shutdown_connections 2024-09-19T06:50:14.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.696+0000 7fd123d8b740 1 --2- 172.21.15.103:0/2317522227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f1a2240 0x555e5f1a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.696+0000 7fd123d8b740 1 --2- 172.21.15.103:0/2317522227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f197980 0x555e5f1a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.696+0000 7fd123d8b740 1 --2- 172.21.15.103:0/2317522227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197060 0x555e5f197440 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.696+0000 7fd123d8b740 1 -- 172.21.15.103:0/2317522227 >> 172.21.15.103:0/2317522227 conn(0x555e5f1935f0 msgr2=0x555e5f1939d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:14.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.696+0000 7fd123d8b740 1 -- 172.21.15.103:0/2317522227 shutdown_connections 2024-09-19T06:50:14.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.696+0000 7fd123d8b740 1 -- 172.21.15.103:0/2317522227 wait complete. 2024-09-19T06:50:14.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.696+0000 7fd123d8b740 1 -- start start 2024-09-19T06:50:14.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.697+0000 7fd123d8b740 1 Processor -- start 2024-09-19T06:50:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.697+0000 7fd123d8b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f197060 0x555e5f19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.697+0000 7fd123d8b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197980 0x555e5f19f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.697+0000 7fd123d8b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f1a2240 0x555e5f19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.697+0000 7fd123d8b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555e5f1a4e10 con 0x555e5f197980 2024-09-19T06:50:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.698+0000 7fd11a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197980 0x555e5f19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.697+0000 7fd123d8b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555e5f084f10 con 0x555e5f197060 2024-09-19T06:50:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.698+0000 7fd123d8b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555e5f1a4f90 con 0x555e5f1a2240 2024-09-19T06:50:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.698+0000 7fd11a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197980 0x555e5f19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46238/0 (socket says 172.21.15.103:46238) 2024-09-19T06:50:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.698+0000 7fd11a7fc640 1 -- 172.21.15.103:0/1010246568 learned_addr learned my addr 172.21.15.103:0/1010246568 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.698+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/1010246568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f197060 0x555e5f19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.698+0000 7fd123d8b740 10 client.0 _ll_register_callbacks cb 0x555e5f0a8db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.698+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/1010246568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f1a2240 0x555e5f19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.698+0000 7fd11a7fc640 1 -- 172.21.15.103:0/1010246568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f1a2240 msgr2=0x555e5f19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.698+0000 7fd11a7fc640 1 --2- 172.21.15.103:0/1010246568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f1a2240 0x555e5f19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.698+0000 7fd11a7fc640 1 -- 172.21.15.103:0/1010246568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f197060 msgr2=0x555e5f19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.698+0000 7fd11a7fc640 1 --2- 172.21.15.103:0/1010246568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f197060 0x555e5f19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.698+0000 7fd11a7fc640 1 -- 172.21.15.103:0/1010246568 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e5ee0b580 con 0x555e5f197980 2024-09-19T06:50:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.698+0000 7fd11a7fc640 1 --2- 172.21.15.103:0/1010246568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197980 0x555e5f19f700 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd110012700 tx=0x7fd110012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d9959ae8744a57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.698+0000 7fd0faffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.699+0000 7fd0faffd640 1 -- 172.21.15.103:0/1010246568 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110004050 con 0x555e5f197980 2024-09-19T06:50:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.699+0000 7fd0faffd640 1 -- 172.21.15.103:0/1010246568 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd1100041f0 con 0x555e5f197980 2024-09-19T06:50:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.699+0000 7fd123d8b740 1 -- 172.21.15.103:0/1010246568 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd114036040 con 0x555e5f197980 2024-09-19T06:50:14.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.699+0000 7fd0faffd640 1 -- 172.21.15.103:0/1010246568 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd1100043f0 con 0x555e5f197980 2024-09-19T06:50:14.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.699+0000 7fd123d8b740 10 client.15777 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:14.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.699+0000 7fd123d8b740 1 -- 172.21.15.103:0/1010246568 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555e5eed5960 con 0x555e5f197980 2024-09-19T06:50:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.700+0000 7fd0faffd640 1 -- 172.21.15.103:0/1010246568 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd11000fe00 con 0x555e5f197980 2024-09-19T06:50:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.700+0000 7fd0faffd640 1 -- 172.21.15.103:0/1010246568 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd110025470 con 0x555e5f197980 2024-09-19T06:50:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.700+0000 7fd123d8b740 10 client.15777 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.700+0000 7fd123d8b740 10 client.15777 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:50:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.700+0000 7fd123d8b740 20 client.15777 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.700+0000 7fd123d8b740 1 --2- 172.21.15.103:0/1010246568 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555e5f1b1f20 0x555e5f1b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.700+0000 7fd123d8b740 4 client.15777 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.700+0000 7fd123d8b740 1 -- 172.21.15.103:0/1010246568 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd114036040 con 0x555e5f1b1f20 2024-09-19T06:50:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.701+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/1010246568 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555e5f1b1f20 0x555e5f1b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.701+0000 7fd0faffd640 10 client.15777 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.701+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/1010246568 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555e5f1b1f20 0x555e5f1b43a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9a1665060062415 server_cookie=6f01bfa77b95d707 in_seq=0 out_seq=0 2024-09-19T06:50:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.702+0000 7fd0faffd640 1 -- 172.21.15.103:0/1010246568 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd114036040 con 0x555e5f1b1f20 2024-09-19T06:50:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.702+0000 7fd0faffd640 10 client.15777 handle_command_reply: tid=0 2024-09-19T06:50:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.702+0000 7fd123d8b740 1 client.15777 shutdown 2024-09-19T06:50:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.702+0000 7fd123d8b740 1 -- 172.21.15.103:0/1010246568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197980 msgr2=0x555e5f19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.702+0000 7fd123d8b740 1 --2- 172.21.15.103:0/1010246568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197980 0x555e5f19f700 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd110012700 tx=0x7fd110012bc0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.703+0000 7fd123d8b740 1 -- 172.21.15.103:0/1010246568 shutdown_connections 2024-09-19T06:50:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.703+0000 7fd123d8b740 1 --2- 172.21.15.103:0/1010246568 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555e5f1b1f20 0x555e5f1b43a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.703+0000 7fd123d8b740 1 --2- 172.21.15.103:0/1010246568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f1a2240 0x555e5f19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.703+0000 7fd123d8b740 1 --2- 172.21.15.103:0/1010246568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197980 0x555e5f19f700 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.703+0000 7fd123d8b740 1 --2- 172.21.15.103:0/1010246568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f197060 0x555e5f19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.703+0000 7fd123d8b740 1 -- 172.21.15.103:0/1010246568 >> 172.21.15.103:0/1010246568 conn(0x555e5f1935f0 msgr2=0x555e5f199320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.703+0000 7fd0faffd640 0 client.15777 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.703+0000 7fd123d8b740 1 -- 172.21.15.103:0/1010246568 shutdown_connections 2024-09-19T06:50:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.703+0000 7fd123d8b740 1 -- 172.21.15.103:0/1010246568 wait complete. 2024-09-19T06:50:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.703+0000 7fd123d8b740 20 client.15777 trim_cache size 0 max 16384 2024-09-19T06:50:14.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.720+0000 7fd123d8b740 1 Processor -- start 2024-09-19T06:50:14.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.720+0000 7fd123d8b740 1 -- start start 2024-09-19T06:50:14.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.720+0000 7fd123d8b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f197060 0x555e5f1a1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.720+0000 7fd123d8b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197980 0x555e5f1b9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.720+0000 7fd123d8b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f1a2240 0x555e5f1ba1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.720+0000 7fd123d8b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c1b3e80 con 0x555e5f197980 2024-09-19T06:50:14.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.720+0000 7fd123d8b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c1b59c0 con 0x555e5f1a2240 2024-09-19T06:50:14.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.720+0000 7fd123d8b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c1b4550 con 0x555e5f197060 2024-09-19T06:50:14.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.720+0000 7fd11a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197980 0x555e5f1b9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.720+0000 7fd11a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197980 0x555e5f1b9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46242/0 (socket says 172.21.15.103:46242) 2024-09-19T06:50:14.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.720+0000 7fd11a7fc640 1 -- 172.21.15.103:0/2000578665 learned_addr learned my addr 172.21.15.103:0/2000578665 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:14.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.721+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/2000578665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f197060 0x555e5f1a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.721+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/2000578665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f1a2240 0x555e5f1ba1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.721+0000 7fd11b7fe640 1 -- 172.21.15.103:0/2000578665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f1a2240 msgr2=0x555e5f1ba1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:14.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.721+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/2000578665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f1a2240 0x555e5f1ba1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.721+0000 7fd11b7fe640 1 -- 172.21.15.103:0/2000578665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197980 msgr2=0x555e5f1b9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:14.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.721+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/2000578665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197980 0x555e5f1b9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.721+0000 7fd11b7fe640 1 -- 172.21.15.103:0/2000578665 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e5f19e1b0 con 0x555e5f197060 2024-09-19T06:50:14.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.721+0000 7fd11a7fc640 1 --2- 172.21.15.103:0/2000578665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197980 0x555e5f1b9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:14.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.722+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/2000578665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f197060 0x555e5f1a1130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd1140316a0 tx=0x7fd1140317a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1911befcd3dd0780 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:14.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.722+0000 7fd1017fa640 1 -- 172.21.15.103:0/2000578665 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114009800 con 0x555e5f197060 2024-09-19T06:50:14.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.722+0000 7fd1017fa640 1 -- 172.21.15.103:0/2000578665 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd114036040 con 0x555e5f197060 2024-09-19T06:50:14.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.722+0000 7fd1017fa640 1 -- 172.21.15.103:0/2000578665 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1140043a0 con 0x555e5f197060 2024-09-19T06:50:14.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.723+0000 7fd123d8b740 1 -- 172.21.15.103:0/2000578665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f197060 msgr2=0x555e5f1a1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:14.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.723+0000 7fd123d8b740 1 --2- 172.21.15.103:0/2000578665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f197060 0x555e5f1a1130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd1140316a0 tx=0x7fd1140317a0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.723+0000 7fd123d8b740 1 -- 172.21.15.103:0/2000578665 shutdown_connections 2024-09-19T06:50:14.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.723+0000 7fd123d8b740 1 --2- 172.21.15.103:0/2000578665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f1a2240 0x555e5f1ba1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.723+0000 7fd123d8b740 1 --2- 172.21.15.103:0/2000578665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197980 0x555e5f1b9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.723+0000 7fd123d8b740 1 --2- 172.21.15.103:0/2000578665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f197060 0x555e5f1a1130 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.723+0000 7fd123d8b740 1 -- 172.21.15.103:0/2000578665 >> 172.21.15.103:0/2000578665 conn(0x555e5f1935f0 msgr2=0x555e5f1a0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:14.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.723+0000 7fd123d8b740 1 -- 172.21.15.103:0/2000578665 shutdown_connections 2024-09-19T06:50:14.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.723+0000 7fd123d8b740 1 -- 172.21.15.103:0/2000578665 wait complete. 2024-09-19T06:50:14.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.723+0000 7fd123d8b740 1 -- start start 2024-09-19T06:50:14.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.724+0000 7fd123d8b740 1 Processor -- start 2024-09-19T06:50:14.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.724+0000 7fd123d8b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197060 0x555e5f1aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.724+0000 7fd123d8b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f197980 0x555e5f1aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.724+0000 7fd11b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197060 0x555e5f1aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.724+0000 7fd11b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197060 0x555e5f1aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46254/0 (socket says 172.21.15.103:46254) 2024-09-19T06:50:14.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.724+0000 7fd123d8b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f1a2240 0x555e5f1a72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.724+0000 7fd11b7fe640 1 -- 172.21.15.103:0/1821935635 learned_addr learned my addr 172.21.15.103:0/1821935635 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:14.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.724+0000 7fd123d8b740 1 -- 172.21.15.103:0/1821935635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c1b52f0 con 0x555e5f197060 2024-09-19T06:50:14.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.724+0000 7fd123d8b740 1 -- 172.21.15.103:0/1821935635 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c1b4c20 con 0x555e5f1a2240 2024-09-19T06:50:14.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.724+0000 7fd123d8b740 1 -- 172.21.15.103:0/1821935635 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c1b1e40 con 0x555e5f197980 2024-09-19T06:50:14.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.724+0000 7fd123d8b740 10 client.0 _ll_register_callbacks cb 0x7fd110009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:14.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.725+0000 7fd11b7fe640 1 -- 172.21.15.103:0/1821935635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f197980 msgr2=0x555e5f1aaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:14.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.725+0000 7fd11a7fc640 1 --2- 172.21.15.103:0/1821935635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f197980 0x555e5f1aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.725+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/1821935635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f197980 0x555e5f1aaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.725+0000 7fd11b7fe640 1 -- 172.21.15.103:0/1821935635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f1a2240 msgr2=0x555e5f1a72a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:50:14.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.725+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/1821935635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f1a2240 0x555e5f1a72a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.725+0000 7fd11b7fe640 1 -- 172.21.15.103:0/1821935635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e5f1a7a70 con 0x555e5f197060 2024-09-19T06:50:14.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.725+0000 7fd11b7fe640 1 --2- 172.21.15.103:0/1821935635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197060 0x555e5f1aa4c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd114009ca0 tx=0x7fd114031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abef4e7ca859230a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:14.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.725+0000 7fd0faffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:14.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.725+0000 7fd0faffd640 1 -- 172.21.15.103:0/1821935635 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114003970 con 0x555e5f197060 2024-09-19T06:50:14.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.725+0000 7fd0faffd640 1 -- 172.21.15.103:0/1821935635 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd114003b10 con 0x555e5f197060 2024-09-19T06:50:14.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.725+0000 7fd123d8b740 1 -- 172.21.15.103:0/1821935635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd114036040 con 0x555e5f197060 2024-09-19T06:50:14.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.725+0000 7fd0faffd640 1 -- 172.21.15.103:0/1821935635 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd114003d10 con 0x555e5f197060 2024-09-19T06:50:14.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.725+0000 7fd123d8b740 10 client.15783 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:14.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.725+0000 7fd123d8b740 1 -- 172.21.15.103:0/1821935635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555e5f21d870 con 0x555e5f197060 2024-09-19T06:50:14.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.726+0000 7fd0faffd640 1 -- 172.21.15.103:0/1821935635 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd114036040 con 0x555e5f197060 2024-09-19T06:50:14.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.726+0000 7fd0faffd640 1 -- 172.21.15.103:0/1821935635 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd11402f990 con 0x555e5f197060 2024-09-19T06:50:14.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.726+0000 7fd123d8b740 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:14.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.726+0000 7fd123d8b740 10 client.15783 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:50:14.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.726+0000 7fd123d8b740 20 client.15783 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:14.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.726+0000 7fd123d8b740 1 --2- 172.21.15.103:0/1821935635 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555e5f21daa0 0x555e5f21fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:14.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.726+0000 7fd123d8b740 4 client.15783 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "29305e6a", "release": true, "await": true}] 2024-09-19T06:50:14.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.726+0000 7fd123d8b740 1 -- 172.21.15.103:0/1821935635 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "set_id": "29305e6a", "release": true, "await": true}) -- 0x555e5f109b90 con 0x555e5f21daa0 2024-09-19T06:50:14.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.726+0000 7fd11a7fc640 1 --2- 172.21.15.103:0/1821935635 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555e5f21daa0 0x555e5f21fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:14.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.727+0000 7fd0faffd640 10 client.15783 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:14.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.727+0000 7fd11a7fc640 1 --2- 172.21.15.103:0/1821935635 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555e5f21daa0 0x555e5f21fe90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78d634d48af5c71e server_cookie=1ba8493f9c214a86 in_seq=0 out_seq=0 2024-09-19T06:50:14.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.729+0000 7fd0faffd640 1 -- 172.21.15.103:0/1821935635 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x555e5f109b90 con 0x555e5f21daa0 2024-09-19T06:50:14.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.729+0000 7fd0faffd640 10 client.15783 handle_command_reply: tid=0 2024-09-19T06:50:14.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.729+0000 7fd123d8b740 1 client.15783 shutdown 2024-09-19T06:50:14.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.729+0000 7fd123d8b740 1 -- 172.21.15.103:0/1821935635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197060 msgr2=0x555e5f1aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:14.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.729+0000 7fd123d8b740 1 --2- 172.21.15.103:0/1821935635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197060 0x555e5f1aa4c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd114009ca0 tx=0x7fd114031920 comp rx=0 tx=0).stop 2024-09-19T06:50:14.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.729+0000 7fd123d8b740 1 -- 172.21.15.103:0/1821935635 shutdown_connections 2024-09-19T06:50:14.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.729+0000 7fd123d8b740 1 --2- 172.21.15.103:0/1821935635 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555e5f21daa0 0x555e5f21fe90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.729+0000 7fd0faffd640 0 client.15783 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:14.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.729+0000 7fd123d8b740 1 --2- 172.21.15.103:0/1821935635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e5f1a2240 0x555e5f1a72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.729+0000 7fd123d8b740 1 --2- 172.21.15.103:0/1821935635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5f197980 0x555e5f1aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.729+0000 7fd123d8b740 1 --2- 172.21.15.103:0/1821935635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e5f197060 0x555e5f1aa4c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.729+0000 7fd123d8b740 1 -- 172.21.15.103:0/1821935635 >> 172.21.15.103:0/1821935635 conn(0x555e5f1935f0 msgr2=0x555e5f192750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:14.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.730+0000 7fd123d8b740 1 -- 172.21.15.103:0/1821935635 shutdown_connections 2024-09-19T06:50:14.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.730+0000 7fd123d8b740 1 -- 172.21.15.103:0/1821935635 wait complete. 2024-09-19T06:50:14.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.730+0000 7fd123d8b740 20 client.15783 trim_cache size 0 max 16384 2024-09-19T06:50:14.732 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:50:14.733 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:50:14.733 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:50:14.733 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 4, 2024-09-19T06:50:14.733 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:50:14.733 INFO:teuthology.orchestra.run.smithi103.stdout: "29305e6a": { 2024-09-19T06:50:14.733 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 4, 2024-09-19T06:50:14.733 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:50:14.733 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:50:14.733 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:50:14.733 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:50:14.733 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:50:14.733 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:50:14.733 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 131.2, 2024-09-19T06:50:14.734 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:50:14.734 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:50:14.734 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:50:14.734 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:50:14.734 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:50:14.734 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:50:14.734 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:14.734 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:14.734 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:14.734 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:14.734 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:14.734 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:50:14.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.730+0000 7fd1219b2640 1 -- 172.21.15.103:0/582956844 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1040a35e0 msgr2=0x7fd1040a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:14.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.730+0000 7fd1219b2640 1 --2- 172.21.15.103:0/582956844 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1040a35e0 0x7fd1040a5aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd10c0cd920 tx=0x7fd10c0d3e30 comp rx=0 tx=0).stop 2024-09-19T06:50:14.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.730+0000 7fd1219b2640 1 -- 172.21.15.103:0/582956844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c100100 msgr2=0x7fd11c1a4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:14.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.730+0000 7fd1219b2640 1 --2- 172.21.15.103:0/582956844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c100100 0x7fd11c1a4530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00e9f0 tx=0x7fd10c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.731+0000 7fd1219b2640 1 -- 172.21.15.103:0/582956844 shutdown_connections 2024-09-19T06:50:14.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.731+0000 7fd1219b2640 1 --2- 172.21.15.103:0/582956844 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1040a35e0 0x7fd1040a5aa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.731+0000 7fd1219b2640 1 --2- 172.21.15.103:0/582956844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd11c106b80 0x7fd11c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.731+0000 7fd1219b2640 1 --2- 172.21.15.103:0/582956844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd11c100aa0 0x7fd11c1a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.731+0000 7fd1219b2640 1 --2- 172.21.15.103:0/582956844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c100100 0x7fd11c1a4530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:14.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.731+0000 7fd1219b2640 1 -- 172.21.15.103:0/582956844 >> 172.21.15.103:0/582956844 conn(0x7fd11c077230 msgr2=0x7fd11c104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:14.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.731+0000 7fd1219b2640 1 -- 172.21.15.103:0/582956844 shutdown_connections 2024-09-19T06:50:14.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:14.731+0000 7fd1219b2640 1 -- 172.21.15.103:0/582956844 wait complete. 2024-09-19T06:50:15.171 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:50:15.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:15 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T06:50:15.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:15 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:15.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:15 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T06:50:15.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:15 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:15.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:15 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T06:50:15.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:15 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T06:50:15.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:15 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:15.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:15 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T06:50:15.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:15 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:15.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:15 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T06:50:15.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:15 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T06:50:15.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:15 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:15.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:15 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T06:50:15.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:15 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:15.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:15 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T06:50:15.849 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '29305e6a', total seconds elapsed: 15.5 2024-09-19T06:50:15.886 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:16.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:16 smithi103 ceph-mon[31158]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 31 op/s 2024-09-19T06:50:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:16 smithi174 ceph-mon[35637]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 31 op/s 2024-09-19T06:50:16.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.430+0000 7f3c9d39e640 1 -- 172.21.15.103:0/3147366219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c98101430 msgr2=0x7f3c9810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:16.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.430+0000 7f3c9d39e640 1 --2- 172.21.15.103:0/3147366219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c98101430 0x7f3c9810d960 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009f00 tx=0x7f3c9002ab20 comp rx=0 tx=0).stop 2024-09-19T06:50:16.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.431+0000 7f3c9d39e640 1 -- 172.21.15.103:0/3147366219 shutdown_connections 2024-09-19T06:50:16.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.431+0000 7f3c9d39e640 1 --2- 172.21.15.103:0/3147366219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c98101430 0x7f3c9810d960 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.431+0000 7f3c9d39e640 1 --2- 172.21.15.103:0/3147366219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c98100a90 0x7f3c98100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.431+0000 7f3c9d39e640 1 --2- 172.21.15.103:0/3147366219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c98106a90 0x7f3c98106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.431+0000 7f3c9d39e640 1 -- 172.21.15.103:0/3147366219 >> 172.21.15.103:0/3147366219 conn(0x7f3c980fc7d0 msgr2=0x7f3c980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:16.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.431+0000 7f3c9d39e640 1 -- 172.21.15.103:0/3147366219 shutdown_connections 2024-09-19T06:50:16.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.431+0000 7f3c9d39e640 1 -- 172.21.15.103:0/3147366219 wait complete. 2024-09-19T06:50:16.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.432+0000 7f3c9d39e640 1 Processor -- start 2024-09-19T06:50:16.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.432+0000 7f3c9d39e640 1 -- start start 2024-09-19T06:50:16.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.432+0000 7f3c9d39e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c98100a90 0x7f3c9810e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.432+0000 7f3c9d39e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c98101430 0x7f3c9810e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.432+0000 7f3c9d39e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c98106a90 0x7f3c9810eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.432+0000 7f3c9d39e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c98111f70 con 0x7f3c98101430 2024-09-19T06:50:16.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.432+0000 7f3c9d39e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c98111df0 con 0x7f3c98100a90 2024-09-19T06:50:16.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.432+0000 7f3c9d39e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c981120f0 con 0x7f3c98106a90 2024-09-19T06:50:16.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.432+0000 7f3c96ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c98100a90 0x7f3c9810e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.432+0000 7f3c96ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c98100a90 0x7f3c9810e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:48292/0 (socket says 172.21.15.103:48292) 2024-09-19T06:50:16.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.432+0000 7f3c96ffd640 1 -- 172.21.15.103:0/1420775115 learned_addr learned my addr 172.21.15.103:0/1420775115 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:16.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.433+0000 7f3c977fe640 1 --2- 172.21.15.103:0/1420775115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c98106a90 0x7f3c9810eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.433+0000 7f3c8e7fc640 1 --2- 172.21.15.103:0/1420775115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c98101430 0x7f3c9810e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.433+0000 7f3c96ffd640 1 -- 172.21.15.103:0/1420775115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c98106a90 msgr2=0x7f3c9810eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:16.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.433+0000 7f3c96ffd640 1 --2- 172.21.15.103:0/1420775115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c98106a90 0x7f3c9810eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.433+0000 7f3c96ffd640 1 -- 172.21.15.103:0/1420775115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c98101430 msgr2=0x7f3c9810e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:16.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.433+0000 7f3c96ffd640 1 --2- 172.21.15.103:0/1420775115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c98101430 0x7f3c9810e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.433+0000 7f3c96ffd640 1 -- 172.21.15.103:0/1420775115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c981aada0 con 0x7f3c98100a90 2024-09-19T06:50:16.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.433+0000 7f3c8e7fc640 1 --2- 172.21.15.103:0/1420775115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c98101430 0x7f3c9810e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:16.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.433+0000 7f3c96ffd640 1 --2- 172.21.15.103:0/1420775115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c98100a90 0x7f3c9810e0f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800ea60 tx=0x7f3c8800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:16.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.434+0000 7f3c94ff9640 1 -- 172.21.15.103:0/1420775115 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8800cde0 con 0x7f3c98100a90 2024-09-19T06:50:16.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.434+0000 7f3c94ff9640 1 -- 172.21.15.103:0/1420775115 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c880035f0 con 0x7f3c98100a90 2024-09-19T06:50:16.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.434+0000 7f3c94ff9640 1 -- 172.21.15.103:0/1420775115 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88010690 con 0x7f3c98100a90 2024-09-19T06:50:16.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.434+0000 7f3c9d39e640 1 -- 172.21.15.103:0/1420775115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c981aafd0 con 0x7f3c98100a90 2024-09-19T06:50:16.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.434+0000 7f3c9d39e640 1 -- 172.21.15.103:0/1420775115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c981ab420 con 0x7f3c98100a90 2024-09-19T06:50:16.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:16 smithi134 ceph-mon[35356]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 31 op/s 2024-09-19T06:50:16.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.437+0000 7f3c94ff9640 1 -- 172.21.15.103:0/1420775115 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7f3c880037a0 con 0x7f3c98100a90 2024-09-19T06:50:16.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.437+0000 7f3c9f777740 1 Processor -- start 2024-09-19T06:50:16.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.437+0000 7f3c9f777740 1 -- start start 2024-09-19T06:50:16.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.438+0000 7f3c9f777740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a014a0 0x561ad5a01880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.438+0000 7f3c9f777740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 0x561ad5a0c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.438+0000 7f3c9f777740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a0c680 0x561ad5a0ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.438+0000 7f3c9f777740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561ad59672e0 con 0x561ad5a01dc0 2024-09-19T06:50:16.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.438+0000 7f3c9f777740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561ad590ece0 con 0x561ad5a014a0 2024-09-19T06:50:16.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.438+0000 7f3c9f777740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561ad59632a0 con 0x561ad5a0c680 2024-09-19T06:50:16.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.438+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a0c680 0x561ad5a0ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.438+0000 7f3c977fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a014a0 0x561ad5a01880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.438+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a0c680 0x561ad5a0ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:36212/0 (socket says 172.21.15.103:36212) 2024-09-19T06:50:16.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.438+0000 7f3c8e7fc640 1 -- 172.21.15.103:0/4090782582 learned_addr learned my addr 172.21.15.103:0/4090782582 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:16.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.438+0000 7f3c94ff9640 1 --2- 172.21.15.103:0/1420775115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c840a3560 0x7f3c840a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.438+0000 7f3c96ffd640 1 --2- 172.21.15.103:0/4090782582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 0x561ad5a0c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.439+0000 7f3c977fe640 1 --2- 172.21.15.103:0/1420775115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c840a3560 0x7f3c840a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.439+0000 7f3c8e7fc640 1 -- 172.21.15.103:0/4090782582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a014a0 msgr2=0x561ad5a01880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:16.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.439+0000 7f3c8e7fc640 1 --2- 172.21.15.103:0/4090782582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a014a0 0x561ad5a01880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.439+0000 7f3c94ff9640 1 -- 172.21.15.103:0/1420775115 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f3c880d0290 con 0x7f3c98100a90 2024-09-19T06:50:16.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.439+0000 7f3c8e7fc640 1 -- 172.21.15.103:0/4090782582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 msgr2=0x561ad5a0c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:16.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.439+0000 7f3c8e7fc640 1 --2- 172.21.15.103:0/4090782582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 0x561ad5a0c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.439+0000 7f3c8e7fc640 1 -- 172.21.15.103:0/4090782582 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ad58f3440 con 0x561ad5a0c680 2024-09-19T06:50:16.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.439+0000 7f3c96ffd640 1 --2- 172.21.15.103:0/4090782582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 0x561ad5a0c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:16.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.439+0000 7f3c8e7fc640 1 --2- 172.21.15.103:0/4090782582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a0c680 0x561ad5a0ea70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009c90 tx=0x7f3c80010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d295e03c1114edd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:16.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.439+0000 7f3c8d7fa640 1 -- 172.21.15.103:0/4090782582 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c80004020 con 0x561ad5a0c680 2024-09-19T06:50:16.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.439+0000 7f3c8d7fa640 1 -- 172.21.15.103:0/4090782582 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c800041c0 con 0x561ad5a0c680 2024-09-19T06:50:16.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.439+0000 7f3c8d7fa640 1 -- 172.21.15.103:0/4090782582 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c80012740 con 0x561ad5a0c680 2024-09-19T06:50:16.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.440+0000 7f3c9f777740 1 -- 172.21.15.103:0/4090782582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a0c680 msgr2=0x561ad5a0ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:16.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.440+0000 7f3c9f777740 1 --2- 172.21.15.103:0/4090782582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a0c680 0x561ad5a0ea70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009c90 tx=0x7f3c80010740 comp rx=0 tx=0).stop 2024-09-19T06:50:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.440+0000 7f3c977fe640 1 --2- 172.21.15.103:0/1420775115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c840a3560 0x7f3c840a5a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3c90004af0 tx=0x7f3c900098b0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.441+0000 7f3c9f777740 1 -- 172.21.15.103:0/4090782582 shutdown_connections 2024-09-19T06:50:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.441+0000 7f3c9f777740 1 --2- 172.21.15.103:0/4090782582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a0c680 0x561ad5a0ea70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.441+0000 7f3c9f777740 1 --2- 172.21.15.103:0/4090782582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 0x561ad5a0c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.441+0000 7f3c9f777740 1 --2- 172.21.15.103:0/4090782582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a014a0 0x561ad5a01880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.441+0000 7f3c9f777740 1 -- 172.21.15.103:0/4090782582 >> 172.21.15.103:0/4090782582 conn(0x561ad59fda30 msgr2=0x561ad59fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.441+0000 7f3c9f777740 1 -- 172.21.15.103:0/4090782582 shutdown_connections 2024-09-19T06:50:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.441+0000 7f3c9f777740 1 -- 172.21.15.103:0/4090782582 wait complete. 2024-09-19T06:50:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.441+0000 7f3c9f777740 1 -- start start 2024-09-19T06:50:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.441+0000 7f3c9f777740 1 Processor -- start 2024-09-19T06:50:16.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.442+0000 7f3c9f777740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a014a0 0x561ad5a0bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.442+0000 7f3c9f777740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 0x561ad5a09240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.442+0000 7f3c96ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a014a0 0x561ad5a0bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.442+0000 7f3c96ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a014a0 0x561ad5a0bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:36222/0 (socket says 172.21.15.103:36222) 2024-09-19T06:50:16.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.442+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 0x561ad5a09240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.442+0000 7f3c96ffd640 1 -- 172.21.15.103:0/1792453217 learned_addr learned my addr 172.21.15.103:0/1792453217 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:16.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.442+0000 7f3c9f777740 1 --2- 172.21.15.103:0/1792453217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a0c680 0x561ad5a09780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.442+0000 7f3c9f777740 1 -- 172.21.15.103:0/1792453217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561ad5a0f250 con 0x561ad5a01dc0 2024-09-19T06:50:16.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.442+0000 7f3c9f777740 1 -- 172.21.15.103:0/1792453217 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561ad5922050 con 0x561ad5a0c680 2024-09-19T06:50:16.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.442+0000 7f3c9f777740 1 -- 172.21.15.103:0/1792453217 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561ad5a0f3d0 con 0x561ad5a014a0 2024-09-19T06:50:16.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.443+0000 7f3c977fe640 1 --2- 172.21.15.103:0/1792453217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a0c680 0x561ad5a09780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.443+0000 7f3c8e7fc640 1 -- 172.21.15.103:0/1792453217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a014a0 msgr2=0x561ad5a0bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:16.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.443+0000 7f3c8e7fc640 1 --2- 172.21.15.103:0/1792453217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a014a0 0x561ad5a0bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.443+0000 7f3c8e7fc640 1 -- 172.21.15.103:0/1792453217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a0c680 msgr2=0x561ad5a09780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:16.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.443+0000 7f3c8e7fc640 1 --2- 172.21.15.103:0/1792453217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a0c680 0x561ad5a09780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.443+0000 7f3c8e7fc640 1 -- 172.21.15.103:0/1792453217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ad56d83c0 con 0x561ad5a01dc0 2024-09-19T06:50:16.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.443+0000 7f3c977fe640 1 --2- 172.21.15.103:0/1792453217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a0c680 0x561ad5a09780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:16.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.443+0000 7f3c9f777740 10 client.0 _ll_register_callbacks cb 0x561ad5962160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:16.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.443+0000 7f3c9f777740 1 -- 172.21.15.103:0/1792453217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c800041c0 con 0x561ad5a01dc0 2024-09-19T06:50:16.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.443+0000 7f3c8e7fc640 1 --2- 172.21.15.103:0/1792453217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 0x561ad5a09240 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3c80003ee0 tx=0x7f3c80009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b14cbcf5859efb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:16.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.443+0000 7f3c7e7fc640 10 client.15789 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:16.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.443+0000 7f3c7e7fc640 1 -- 172.21.15.103:0/1792453217 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c800035f0 con 0x561ad5a01dc0 2024-09-19T06:50:16.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.444+0000 7f3c7e7fc640 1 -- 172.21.15.103:0/1792453217 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f3c80003790 con 0x561ad5a01dc0 2024-09-19T06:50:16.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.444+0000 7f3c7e7fc640 1 -- 172.21.15.103:0/1792453217 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c80005040 con 0x561ad5a01dc0 2024-09-19T06:50:16.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.444+0000 7f3c9f777740 10 client.15789 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:16.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.444+0000 7f3c9f777740 1 -- 172.21.15.103:0/1792453217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=27}) -- 0x561ad5a1a920 con 0x561ad5a01dc0 2024-09-19T06:50:16.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.444+0000 7f3c7e7fc640 1 -- 172.21.15.103:0/1792453217 <== mon.0 v2:172.21.15.103:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3c8000f040 con 0x561ad5a01dc0 2024-09-19T06:50:16.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.444+0000 7f3c9f777740 10 client.15789 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:16.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.444+0000 7f3c9f777740 10 client.15789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:16.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.444+0000 7f3c9f777740 20 client.15789 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:16.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.444+0000 7f3c9f777740 1 --2- 172.21.15.103:0/1792453217 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561ad5a1bd20 0x561ad5a1e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.444+0000 7f3c9f777740 4 client.15789 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:16.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.444+0000 7f3c9f777740 1 -- 172.21.15.103:0/1792453217 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ad5974850 con 0x561ad5a1bd20 2024-09-19T06:50:16.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.445+0000 7f3c96ffd640 1 --2- 172.21.15.103:0/1792453217 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561ad5a1bd20 0x561ad5a1e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.445+0000 7f3c7e7fc640 10 client.15789 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:16.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.445+0000 7f3c96ffd640 1 --2- 172.21.15.103:0/1792453217 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561ad5a1bd20 0x561ad5a1e110 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cbabee0dda9b5c1 server_cookie=e4ded487af2e8291 in_seq=0 out_seq=0 2024-09-19T06:50:16.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.446+0000 7f3c7e7fc640 1 -- 172.21.15.103:0/1792453217 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ad5974850 con 0x561ad5a1bd20 2024-09-19T06:50:16.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.446+0000 7f3c7e7fc640 10 client.15789 handle_command_reply: tid=0 2024-09-19T06:50:16.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.446+0000 7f3c9f777740 1 client.15789 shutdown 2024-09-19T06:50:16.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.446+0000 7f3c9f777740 1 -- 172.21.15.103:0/1792453217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 msgr2=0x561ad5a09240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:16.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.446+0000 7f3c9f777740 1 --2- 172.21.15.103:0/1792453217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 0x561ad5a09240 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3c80003ee0 tx=0x7f3c80009a00 comp rx=0 tx=0).stop 2024-09-19T06:50:16.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.446+0000 7f3c9f777740 1 -- 172.21.15.103:0/1792453217 shutdown_connections 2024-09-19T06:50:16.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.446+0000 7f3c9f777740 1 --2- 172.21.15.103:0/1792453217 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561ad5a1bd20 0x561ad5a1e110 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.446+0000 7f3c7e7fc640 0 client.15789 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:16.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.446+0000 7f3c9f777740 1 --2- 172.21.15.103:0/1792453217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a0c680 0x561ad5a09780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.446+0000 7f3c9f777740 1 --2- 172.21.15.103:0/1792453217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 0x561ad5a09240 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.446+0000 7f3c9f777740 1 --2- 172.21.15.103:0/1792453217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a014a0 0x561ad5a0bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.446+0000 7f3c9f777740 1 -- 172.21.15.103:0/1792453217 >> 172.21.15.103:0/1792453217 conn(0x561ad59fda30 msgr2=0x561ad5a03760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:16.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.446+0000 7f3c9f777740 1 -- 172.21.15.103:0/1792453217 shutdown_connections 2024-09-19T06:50:16.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.446+0000 7f3c9f777740 1 -- 172.21.15.103:0/1792453217 wait complete. 2024-09-19T06:50:16.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.446+0000 7f3c9f777740 20 client.15789 trim_cache size 0 max 16384 2024-09-19T06:50:16.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.480+0000 7f3c9f777740 1 Processor -- start 2024-09-19T06:50:16.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.480+0000 7f3c9f777740 1 -- start start 2024-09-19T06:50:16.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.480+0000 7f3c9f777740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a014a0 0x561ad5a095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.480+0000 7f3c9f777740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 0x561ad5a23880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.481+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 0x561ad5a23880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.481+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 0x561ad5a23880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46302/0 (socket says 172.21.15.103:46302) 2024-09-19T06:50:16.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.481+0000 7f3c8e7fc640 1 -- 172.21.15.103:0/328359172 learned_addr learned my addr 172.21.15.103:0/328359172 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:16.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.481+0000 7f3c9f777740 1 --2- 172.21.15.103:0/328359172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a0c680 0x561ad5a23dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.481+0000 7f3c9f777740 1 -- 172.21.15.103:0/328359172 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c981b3ea0 con 0x561ad5a01dc0 2024-09-19T06:50:16.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.481+0000 7f3c9f777740 1 -- 172.21.15.103:0/328359172 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c981b59e0 con 0x561ad5a014a0 2024-09-19T06:50:16.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.481+0000 7f3c9f777740 1 -- 172.21.15.103:0/328359172 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c981b4570 con 0x561ad5a0c680 2024-09-19T06:50:16.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.481+0000 7f3c96ffd640 1 --2- 172.21.15.103:0/328359172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a014a0 0x561ad5a095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.481+0000 7f3c8e7fc640 1 -- 172.21.15.103:0/328359172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a0c680 msgr2=0x561ad5a23dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:16.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.481+0000 7f3c8e7fc640 1 --2- 172.21.15.103:0/328359172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a0c680 0x561ad5a23dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.481+0000 7f3c8e7fc640 1 -- 172.21.15.103:0/328359172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a014a0 msgr2=0x561ad5a095d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:16.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.481+0000 7f3c8e7fc640 1 --2- 172.21.15.103:0/328359172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a014a0 0x561ad5a095d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.481+0000 7f3c8e7fc640 1 -- 172.21.15.103:0/328359172 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ad5a0edd0 con 0x561ad5a01dc0 2024-09-19T06:50:16.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.481+0000 7f3c96ffd640 1 --2- 172.21.15.103:0/328359172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a014a0 0x561ad5a095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:16.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.482+0000 7f3c8e7fc640 1 --2- 172.21.15.103:0/328359172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 0x561ad5a23880 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009ed0 tx=0x7f3c80020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db5b98f1480ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:16.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.482+0000 7f3c8cff9640 1 -- 172.21.15.103:0/328359172 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c800054b0 con 0x561ad5a01dc0 2024-09-19T06:50:16.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.482+0000 7f3c8cff9640 1 -- 172.21.15.103:0/328359172 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c800041c0 con 0x561ad5a01dc0 2024-09-19T06:50:16.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.482+0000 7f3c8cff9640 1 -- 172.21.15.103:0/328359172 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c80025b60 con 0x561ad5a01dc0 2024-09-19T06:50:16.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.483+0000 7f3c9f777740 1 -- 172.21.15.103:0/328359172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 msgr2=0x561ad5a23880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:16.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.483+0000 7f3c9f777740 1 --2- 172.21.15.103:0/328359172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 0x561ad5a23880 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009ed0 tx=0x7f3c80020000 comp rx=0 tx=0).stop 2024-09-19T06:50:16.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.483+0000 7f3c9f777740 1 -- 172.21.15.103:0/328359172 shutdown_connections 2024-09-19T06:50:16.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.483+0000 7f3c9f777740 1 --2- 172.21.15.103:0/328359172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a0c680 0x561ad5a23dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.483+0000 7f3c9f777740 1 --2- 172.21.15.103:0/328359172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a01dc0 0x561ad5a23880 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.483+0000 7f3c9f777740 1 --2- 172.21.15.103:0/328359172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a014a0 0x561ad5a095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.483+0000 7f3c9f777740 1 -- 172.21.15.103:0/328359172 >> 172.21.15.103:0/328359172 conn(0x561ad59fda30 msgr2=0x561ad5a03680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:16.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.483+0000 7f3c9f777740 1 -- 172.21.15.103:0/328359172 shutdown_connections 2024-09-19T06:50:16.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.483+0000 7f3c9f777740 1 -- 172.21.15.103:0/328359172 wait complete. 2024-09-19T06:50:16.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.484+0000 7f3c9f777740 1 -- start start 2024-09-19T06:50:16.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.484+0000 7f3c9f777740 1 Processor -- start 2024-09-19T06:50:16.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.484+0000 7f3c9f777740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a014a0 0x561ad5a10b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.484+0000 7f3c9f777740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a01dc0 0x561ad5a11090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.484+0000 7f3c9f777740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a0c680 0x561ad5a85850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.484+0000 7f3c9f777740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c981b5310 con 0x561ad5a0c680 2024-09-19T06:50:16.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.484+0000 7f3c9f777740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c981b4c40 con 0x561ad5a014a0 2024-09-19T06:50:16.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.485+0000 7f3c9f777740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c981b1e60 con 0x561ad5a01dc0 2024-09-19T06:50:16.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.485+0000 7f3c9f777740 10 client.0 _ll_register_callbacks cb 0x7f3c80005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:16.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.485+0000 7f3c977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a0c680 0x561ad5a85850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.485+0000 7f3c977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a0c680 0x561ad5a85850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46316/0 (socket says 172.21.15.103:46316) 2024-09-19T06:50:16.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.485+0000 7f3c977fe640 1 -- 172.21.15.103:0/2478854203 learned_addr learned my addr 172.21.15.103:0/2478854203 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:16.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.485+0000 7f3c8e7fc640 1 --2- 172.21.15.103:0/2478854203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a01dc0 0x561ad5a11090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.485+0000 7f3c96ffd640 1 --2- 172.21.15.103:0/2478854203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a014a0 0x561ad5a10b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.485+0000 7f3c977fe640 1 -- 172.21.15.103:0/2478854203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a01dc0 msgr2=0x561ad5a11090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:16.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.485+0000 7f3c977fe640 1 --2- 172.21.15.103:0/2478854203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a01dc0 0x561ad5a11090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.485+0000 7f3c977fe640 1 -- 172.21.15.103:0/2478854203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a014a0 msgr2=0x561ad5a10b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:16.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.485+0000 7f3c977fe640 1 --2- 172.21.15.103:0/2478854203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a014a0 0x561ad5a10b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.485+0000 7f3c977fe640 1 -- 172.21.15.103:0/2478854203 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ad5a85e40 con 0x561ad5a0c680 2024-09-19T06:50:16.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.485+0000 7f3c977fe640 1 --2- 172.21.15.103:0/2478854203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a0c680 0x561ad5a85850 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009f00 tx=0x7f3c90005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8744d60b571e41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:16.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.485+0000 7f3c7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:16.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.486+0000 7f3c7e7fc640 1 -- 172.21.15.103:0/2478854203 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9004c050 con 0x561ad5a0c680 2024-09-19T06:50:16.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.486+0000 7f3c7e7fc640 1 -- 172.21.15.103:0/2478854203 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f3c90004490 con 0x561ad5a0c680 2024-09-19T06:50:16.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.486+0000 7f3c9f777740 1 -- 172.21.15.103:0/2478854203 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c800041c0 con 0x561ad5a0c680 2024-09-19T06:50:16.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.486+0000 7f3c7e7fc640 1 -- 172.21.15.103:0/2478854203 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c9005e050 con 0x561ad5a0c680 2024-09-19T06:50:16.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.486+0000 7f3c9f777740 10 client.15795 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:16.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.486+0000 7f3c9f777740 1 -- 172.21.15.103:0/2478854203 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ad5a870a0 con 0x561ad5a0c680 2024-09-19T06:50:16.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.487+0000 7f3c7e7fc640 1 -- 172.21.15.103:0/2478854203 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c800041c0 con 0x561ad5a0c680 2024-09-19T06:50:16.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.487+0000 7f3c7e7fc640 1 -- 172.21.15.103:0/2478854203 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3c90059030 con 0x561ad5a0c680 2024-09-19T06:50:16.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.487+0000 7f3c9f777740 10 client.15795 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:16.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.487+0000 7f3c9f777740 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:16.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.487+0000 7f3c9f777740 20 client.15795 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:16.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.487+0000 7f3c9f777740 1 --2- 172.21.15.103:0/2478854203 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561ad5a1bd20 0x561ad5a898b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:16.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.487+0000 7f3c9f777740 4 client.15795 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:50:16.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.487+0000 7f3c9f777740 1 -- 172.21.15.103:0/2478854203 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561ad596ed60 con 0x561ad5a1bd20 2024-09-19T06:50:16.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.487+0000 7f3c96ffd640 1 --2- 172.21.15.103:0/2478854203 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561ad5a1bd20 0x561ad5a898b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:16.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.487+0000 7f3c96ffd640 1 --2- 172.21.15.103:0/2478854203 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561ad5a1bd20 0x561ad5a898b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a124054e0348127 server_cookie=756b728fecb08e50 in_seq=0 out_seq=0 2024-09-19T06:50:16.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.487+0000 7f3c7e7fc640 10 client.15795 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:16.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.488+0000 7f3c7e7fc640 1 -- 172.21.15.103:0/2478854203 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561ad596ed60 con 0x561ad5a1bd20 2024-09-19T06:50:16.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.488+0000 7f3c7e7fc640 10 client.15795 handle_command_reply: tid=0 2024-09-19T06:50:16.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.488+0000 7f3c9f777740 1 client.15795 shutdown 2024-09-19T06:50:16.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.488+0000 7f3c9f777740 1 -- 172.21.15.103:0/2478854203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a0c680 msgr2=0x561ad5a85850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:16.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.488+0000 7f3c9f777740 1 --2- 172.21.15.103:0/2478854203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a0c680 0x561ad5a85850 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009f00 tx=0x7f3c90005dc0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.488+0000 7f3c9f777740 1 -- 172.21.15.103:0/2478854203 shutdown_connections 2024-09-19T06:50:16.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.488+0000 7f3c9f777740 1 --2- 172.21.15.103:0/2478854203 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561ad5a1bd20 0x561ad5a898b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.488+0000 7f3c9f777740 1 --2- 172.21.15.103:0/2478854203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ad5a0c680 0x561ad5a85850 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.488+0000 7f3c9f777740 1 --2- 172.21.15.103:0/2478854203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ad5a01dc0 0x561ad5a11090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.488+0000 7f3c9f777740 1 --2- 172.21.15.103:0/2478854203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ad5a014a0 0x561ad5a10b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.488+0000 7f3c9f777740 1 -- 172.21.15.103:0/2478854203 >> 172.21.15.103:0/2478854203 conn(0x561ad59fda30 msgr2=0x561ad59fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:16.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.488+0000 7f3c7e7fc640 0 client.15795 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:16.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.488+0000 7f3c9f777740 1 -- 172.21.15.103:0/2478854203 shutdown_connections 2024-09-19T06:50:16.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.488+0000 7f3c9f777740 1 -- 172.21.15.103:0/2478854203 wait complete. 2024-09-19T06:50:16.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.489+0000 7f3c9f777740 20 client.15795 trim_cache size 0 max 16384 2024-09-19T06:50:16.498 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:50:16.498 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T06:50:16.498 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T06:50:16.498 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:50:16.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.489+0000 7f3c9d39e640 1 -- 172.21.15.103:0/1420775115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c840a3560 msgr2=0x7f3c840a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:16.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.489+0000 7f3c9d39e640 1 --2- 172.21.15.103:0/1420775115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c840a3560 0x7f3c840a5a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3c90004af0 tx=0x7f3c900098b0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.489+0000 7f3c9d39e640 1 -- 172.21.15.103:0/1420775115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c98100a90 msgr2=0x7f3c9810e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:16.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.489+0000 7f3c9d39e640 1 --2- 172.21.15.103:0/1420775115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c98100a90 0x7f3c9810e0f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800ea60 tx=0x7f3c8800ef20 comp rx=0 tx=0).stop 2024-09-19T06:50:16.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.490+0000 7f3c9d39e640 1 -- 172.21.15.103:0/1420775115 shutdown_connections 2024-09-19T06:50:16.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.490+0000 7f3c9d39e640 1 --2- 172.21.15.103:0/1420775115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c840a3560 0x7f3c840a5a20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.490+0000 7f3c9d39e640 1 --2- 172.21.15.103:0/1420775115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c98106a90 0x7f3c9810eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.490+0000 7f3c9d39e640 1 --2- 172.21.15.103:0/1420775115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c98101430 0x7f3c9810e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.490+0000 7f3c9d39e640 1 --2- 172.21.15.103:0/1420775115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c98100a90 0x7f3c9810e0f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:16.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.490+0000 7f3c9d39e640 1 -- 172.21.15.103:0/1420775115 >> 172.21.15.103:0/1420775115 conn(0x7f3c980fc7d0 msgr2=0x7f3c980fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:16.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.490+0000 7f3c9d39e640 1 -- 172.21.15.103:0/1420775115 shutdown_connections 2024-09-19T06:50:16.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:16.490+0000 7f3c9d39e640 1 -- 172.21.15.103:0/1420775115 wait complete. 2024-09-19T06:50:17.292 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:50:17.293 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:50:17.698 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.854+0000 7ff0fb933640 1 -- 172.21.15.103:0/2168119870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0f4101350 msgr2=0x7ff0f410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.854+0000 7ff0fb933640 1 --2- 172.21.15.103:0/2168119870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0f4101350 0x7ff0f410d880 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009cd0 tx=0x7ff0ec02a930 comp rx=0 tx=0).stop 2024-09-19T06:50:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.854+0000 7ff0fb933640 1 -- 172.21.15.103:0/2168119870 shutdown_connections 2024-09-19T06:50:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.854+0000 7ff0fb933640 1 --2- 172.21.15.103:0/2168119870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0f4101350 0x7ff0f410d880 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.854+0000 7ff0fb933640 1 --2- 172.21.15.103:0/2168119870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0f41009b0 0x7ff0f4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.854+0000 7ff0fb933640 1 --2- 172.21.15.103:0/2168119870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0f41069c0 0x7ff0f4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.854+0000 7ff0fb933640 1 -- 172.21.15.103:0/2168119870 >> 172.21.15.103:0/2168119870 conn(0x7ff0f4077990 msgr2=0x7ff0f40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.854+0000 7ff0fb933640 1 -- 172.21.15.103:0/2168119870 shutdown_connections 2024-09-19T06:50:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.855+0000 7ff0fb933640 1 -- 172.21.15.103:0/2168119870 wait complete. 2024-09-19T06:50:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.856+0000 7ff0fb933640 1 Processor -- start 2024-09-19T06:50:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.856+0000 7ff0fb933640 1 -- start start 2024-09-19T06:50:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.856+0000 7ff0fb933640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0f41009b0 0x7ff0f41971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.856+0000 7ff0fb933640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0f4101350 0x7ff0f4197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.856+0000 7ff0fb933640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0f41069c0 0x7ff0f419ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.856+0000 7ff0fb933640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f4110e50 con 0x7ff0f41069c0 2024-09-19T06:50:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.856+0000 7ff0fb933640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f4110cd0 con 0x7ff0f41009b0 2024-09-19T06:50:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.856+0000 7ff0f9ed0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0f41069c0 0x7ff0f419ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.856+0000 7ff0fb933640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f4110fd0 con 0x7ff0f4101350 2024-09-19T06:50:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.857+0000 7ff0f9ed0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0f41069c0 0x7ff0f419ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60950/0 (socket says 172.21.15.103:60950) 2024-09-19T06:50:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.857+0000 7ff0f9ed0640 1 -- 172.21.15.103:0/1277159478 learned_addr learned my addr 172.21.15.103:0/1277159478 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.857+0000 7ff0f96cf640 1 --2- 172.21.15.103:0/1277159478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0f41009b0 0x7ff0f41971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.857+0000 7ff0f8ece640 1 --2- 172.21.15.103:0/1277159478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0f4101350 0x7ff0f4197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.857+0000 7ff0f9ed0640 1 -- 172.21.15.103:0/1277159478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0f4101350 msgr2=0x7ff0f4197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.857+0000 7ff0f9ed0640 1 --2- 172.21.15.103:0/1277159478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0f4101350 0x7ff0f4197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.857+0000 7ff0f9ed0640 1 -- 172.21.15.103:0/1277159478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0f41009b0 msgr2=0x7ff0f41971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.857+0000 7ff0f9ed0640 1 --2- 172.21.15.103:0/1277159478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0f41009b0 0x7ff0f41971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.857+0000 7ff0f9ed0640 1 -- 172.21.15.103:0/1277159478 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f419c1b0 con 0x7ff0f41069c0 2024-09-19T06:50:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.857+0000 7ff0f96cf640 1 --2- 172.21.15.103:0/1277159478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0f41009b0 0x7ff0f41971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.857+0000 7ff0f9ed0640 1 --2- 172.21.15.103:0/1277159478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0f41069c0 0x7ff0f419ba80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec005e30 tx=0x7ff0ec0319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.857+0000 7ff0ea7fc640 1 -- 172.21.15.103:0/1277159478 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec030db0 con 0x7ff0f41069c0 2024-09-19T06:50:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.858+0000 7ff0ea7fc640 1 -- 172.21.15.103:0/1277159478 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0ec02d070 con 0x7ff0f41069c0 2024-09-19T06:50:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.858+0000 7ff0ea7fc640 1 -- 172.21.15.103:0/1277159478 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec0489e0 con 0x7ff0f41069c0 2024-09-19T06:50:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.858+0000 7ff0fb933640 1 -- 172.21.15.103:0/1277159478 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f419c440 con 0x7ff0f41069c0 2024-09-19T06:50:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.858+0000 7ff0fb933640 1 -- 172.21.15.103:0/1277159478 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f40698a0 con 0x7ff0f41069c0 2024-09-19T06:50:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.860+0000 7ff0ea7fc640 1 -- 172.21.15.103:0/1277159478 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7ff0ec03d610 con 0x7ff0f41069c0 2024-09-19T06:50:17.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.861+0000 7ff0fe6c9740 1 Processor -- start 2024-09-19T06:50:17.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.861+0000 7ff0fe6c9740 1 -- start start 2024-09-19T06:50:17.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.861+0000 7ff0ea7fc640 1 --2- 172.21.15.103:0/1277159478 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff0e00a3610 0x7ff0e00a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.861+0000 7ff0fe6c9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5adaf0 0x557eab5aded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.861+0000 7ff0fe6c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5ae410 0x557eab5b8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.861+0000 7ff0fe6c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5b8cb0 0x557eab5bb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.861+0000 7ff0fe6c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557eab5122f0 con 0x557eab5b8cb0 2024-09-19T06:50:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.861+0000 7ff0fe6c9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557eab4ba270 con 0x557eab5adaf0 2024-09-19T06:50:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.861+0000 7ff0ea7fc640 1 -- 172.21.15.103:0/1277159478 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff0ec0f33c0 con 0x7ff0f41069c0 2024-09-19T06:50:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.862+0000 7ff0fe6c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557eab50e2b0 con 0x557eab5ae410 2024-09-19T06:50:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.862+0000 7ff0f8ece640 1 --2- 172.21.15.103:0/1277159478 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff0e00a3610 0x7ff0e00a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.862+0000 7ff0f9ed0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5b8cb0 0x557eab5bb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.862+0000 7ff0f9ed0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5b8cb0 0x557eab5bb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60956/0 (socket says 172.21.15.103:60956) 2024-09-19T06:50:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.862+0000 7ff0f9ed0640 1 -- 172.21.15.103:0/1961470269 learned_addr learned my addr 172.21.15.103:0/1961470269 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.862+0000 7ff0f96cf640 1 --2- 172.21.15.103:0/1961470269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5adaf0 0x557eab5aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.862+0000 7ff0f9ed0640 1 -- 172.21.15.103:0/1961470269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5ae410 msgr2=0x557eab5b8770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:50:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.862+0000 7ff0f9ed0640 1 --2- 172.21.15.103:0/1961470269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5ae410 0x557eab5b8770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.862+0000 7ff0f9ed0640 1 -- 172.21.15.103:0/1961470269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5adaf0 msgr2=0x557eab5aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.862+0000 7ff0f9ed0640 1 --2- 172.21.15.103:0/1961470269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5adaf0 0x557eab5aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.862+0000 7ff0f9ed0640 1 -- 172.21.15.103:0/1961470269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eab4b9340 con 0x557eab5b8cb0 2024-09-19T06:50:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.862+0000 7ff0f8ece640 1 --2- 172.21.15.103:0/1277159478 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff0e00a3610 0x7ff0e00a5ad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4105e80 tx=0x7ff0e400b410 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.862+0000 7ff0f9ed0640 1 --2- 172.21.15.103:0/1961470269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5b8cb0 0x557eab5bb0a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec0bf510 tx=0x7ff0ec0bf180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77af6265580e24da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.862+0000 7ff0df7fe640 1 -- 172.21.15.103:0/1961470269 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec0c3930 con 0x557eab5b8cb0 2024-09-19T06:50:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.863+0000 7ff0df7fe640 1 -- 172.21.15.103:0/1961470269 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0ec036070 con 0x557eab5b8cb0 2024-09-19T06:50:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.863+0000 7ff0df7fe640 1 -- 172.21.15.103:0/1961470269 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec0c7d60 con 0x557eab5b8cb0 2024-09-19T06:50:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.863+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/1961470269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5b8cb0 msgr2=0x557eab5bb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.863+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/1961470269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5b8cb0 0x557eab5bb0a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec0bf510 tx=0x7ff0ec0bf180 comp rx=0 tx=0).stop 2024-09-19T06:50:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.863+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/1961470269 shutdown_connections 2024-09-19T06:50:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.863+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/1961470269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5b8cb0 0x557eab5bb0a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.863+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/1961470269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5ae410 0x557eab5b8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.863+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/1961470269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5adaf0 0x557eab5aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.863+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/1961470269 >> 172.21.15.103:0/1961470269 conn(0x557eab5aa060 msgr2=0x557eab5aa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.863+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/1961470269 shutdown_connections 2024-09-19T06:50:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.863+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/1961470269 wait complete. 2024-09-19T06:50:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.864+0000 7ff0fe6c9740 1 -- start start 2024-09-19T06:50:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.864+0000 7ff0fe6c9740 1 Processor -- start 2024-09-19T06:50:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.864+0000 7ff0fe6c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5adaf0 0x557eab5b8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.864+0000 7ff0fe6c9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5ae410 0x557eab5b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.864+0000 7ff0fe6c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5b8cb0 0x557eab5b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.864+0000 7ff0fe6c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557eab5bfa50 con 0x557eab5adaf0 2024-09-19T06:50:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.864+0000 7ff0fe6c9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557eab49b290 con 0x557eab5ae410 2024-09-19T06:50:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.864+0000 7ff0fe6c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557eab5bfbd0 con 0x557eab5b8cb0 2024-09-19T06:50:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.864+0000 7ff0f8ece640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5ae410 0x557eab5b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.865+0000 7ff0f8ece640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5ae410 0x557eab5b5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:34776/0 (socket says 172.21.15.103:34776) 2024-09-19T06:50:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.865+0000 7ff0f8ece640 1 -- 172.21.15.103:0/107570810 learned_addr learned my addr 172.21.15.103:0/107570810 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.865+0000 7ff0fe6c9740 10 client.0 _ll_register_callbacks cb 0x557eab4cc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.865+0000 7ff0f9ed0640 1 --2- 172.21.15.103:0/107570810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5b8cb0 0x557eab5b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.865+0000 7ff0f8ece640 1 -- 172.21.15.103:0/107570810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5b8cb0 msgr2=0x557eab5b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.865+0000 7ff0f8ece640 1 --2- 172.21.15.103:0/107570810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5b8cb0 0x557eab5b61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.865+0000 7ff0f8ece640 1 -- 172.21.15.103:0/107570810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5adaf0 msgr2=0x557eab5b8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.865+0000 7ff0f8ece640 1 --2- 172.21.15.103:0/107570810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5adaf0 0x557eab5b8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.865+0000 7ff0f8ece640 1 -- 172.21.15.103:0/107570810 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eab225210 con 0x557eab5ae410 2024-09-19T06:50:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.865+0000 7ff0f8ece640 1 --2- 172.21.15.103:0/107570810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5ae410 0x557eab5b5c70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4011010 tx=0x7ff0e4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d692c90f235437ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.865+0000 7ff0dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:50:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.865+0000 7ff0dcff9640 1 -- 172.21.15.103:0/107570810 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e401e8f0 con 0x557eab5ae410 2024-09-19T06:50:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.865+0000 7ff0dcff9640 1 -- 172.21.15.103:0/107570810 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff0e401ea90 con 0x557eab5ae410 2024-09-19T06:50:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.865+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/107570810 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0ec036070 con 0x557eab5ae410 2024-09-19T06:50:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.866+0000 7ff0dcff9640 1 -- 172.21.15.103:0/107570810 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff0e401ec90 con 0x557eab5ae410 2024-09-19T06:50:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.866+0000 7ff0fe6c9740 10 client.25235 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.866+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/107570810 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557eab2f1620 con 0x557eab5ae410 2024-09-19T06:50:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.866+0000 7ff0dcff9640 1 -- 172.21.15.103:0/107570810 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0ec036070 con 0x557eab5ae410 2024-09-19T06:50:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.866+0000 7ff0dcff9640 1 -- 172.21.15.103:0/107570810 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff0e401d440 con 0x557eab5ae410 2024-09-19T06:50:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.866+0000 7ff0fe6c9740 10 client.25235 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.866+0000 7ff0fe6c9740 10 client.25235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.866+0000 7ff0fe6c9740 20 client.25235 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.866+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/107570810 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557eab5d12b0 0x557eab5d3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.866+0000 7ff0fe6c9740 4 client.25235 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.866+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/107570810 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eab51fa20 con 0x557eab5d12b0 2024-09-19T06:50:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.867+0000 7ff0f96cf640 1 --2- 172.21.15.103:0/107570810 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557eab5d12b0 0x557eab5d3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.867+0000 7ff0f96cf640 1 --2- 172.21.15.103:0/107570810 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557eab5d12b0 0x557eab5d3700 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=907b86e679bf51e0 server_cookie=a881204548d95cfa in_seq=0 out_seq=0 2024-09-19T06:50:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.867+0000 7ff0dcff9640 10 client.25235 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.868+0000 7ff0dcff9640 1 -- 172.21.15.103:0/107570810 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557eab51fa20 con 0x557eab5d12b0 2024-09-19T06:50:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.868+0000 7ff0dcff9640 10 client.25235 handle_command_reply: tid=0 2024-09-19T06:50:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.868+0000 7ff0fe6c9740 1 client.25235 shutdown 2024-09-19T06:50:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.868+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/107570810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5ae410 msgr2=0x557eab5b5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.868+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/107570810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5ae410 0x557eab5b5c70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4011010 tx=0x7ff0e4004730 comp rx=0 tx=0).stop 2024-09-19T06:50:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.868+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/107570810 shutdown_connections 2024-09-19T06:50:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.868+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/107570810 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557eab5d12b0 0x557eab5d3700 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.868+0000 7ff0dcff9640 0 client.25235 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.868+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/107570810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5b8cb0 0x557eab5b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.868+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/107570810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5ae410 0x557eab5b5c70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.868+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/107570810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5adaf0 0x557eab5b8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.868+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/107570810 >> 172.21.15.103:0/107570810 conn(0x557eab5aa060 msgr2=0x557eab5afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.868+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/107570810 shutdown_connections 2024-09-19T06:50:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.868+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/107570810 wait complete. 2024-09-19T06:50:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.868+0000 7ff0fe6c9740 20 client.25235 trim_cache size 0 max 16384 2024-09-19T06:50:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0fe6c9740 1 Processor -- start 2024-09-19T06:50:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0fe6c9740 1 -- start start 2024-09-19T06:50:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0fe6c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5adaf0 0x557eab5be0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0fe6c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5ae410 0x557eab5a90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0fe6c9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5b8cb0 0x557eab5d9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0f8ece640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5ae410 0x557eab5a90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0f8ece640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5ae410 0x557eab5a90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60976/0 (socket says 172.21.15.103:60976) 2024-09-19T06:50:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0fe6c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41ab4f0 con 0x557eab5ae410 2024-09-19T06:50:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0fe6c9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41ad030 con 0x557eab5b8cb0 2024-09-19T06:50:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0fe6c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41abbc0 con 0x557eab5adaf0 2024-09-19T06:50:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0f8ece640 1 -- 172.21.15.103:0/3850922195 learned_addr learned my addr 172.21.15.103:0/3850922195 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0f96cf640 1 --2- 172.21.15.103:0/3850922195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5adaf0 0x557eab5be0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0f8ece640 1 -- 172.21.15.103:0/3850922195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5adaf0 msgr2=0x557eab5be0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0f8ece640 1 --2- 172.21.15.103:0/3850922195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5adaf0 0x557eab5be0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0f8ece640 1 -- 172.21.15.103:0/3850922195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5b8cb0 msgr2=0x557eab5d9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0f8ece640 1 --2- 172.21.15.103:0/3850922195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5b8cb0 0x557eab5d9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.883+0000 7ff0f8ece640 1 -- 172.21.15.103:0/3850922195 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eab5d3d80 con 0x557eab5ae410 2024-09-19T06:50:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.884+0000 7ff0f8ece640 1 --2- 172.21.15.103:0/3850922195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5ae410 0x557eab5a90b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4010c50 tx=0x7ff0e400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f743c846a918094b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.884+0000 7ff0deffd640 1 -- 172.21.15.103:0/3850922195 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4033070 con 0x557eab5ae410 2024-09-19T06:50:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.884+0000 7ff0deffd640 1 -- 172.21.15.103:0/3850922195 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e4034040 con 0x557eab5ae410 2024-09-19T06:50:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.884+0000 7ff0deffd640 1 -- 172.21.15.103:0/3850922195 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e402f9b0 con 0x557eab5ae410 2024-09-19T06:50:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.884+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/3850922195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5ae410 msgr2=0x557eab5a90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.884+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/3850922195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5ae410 0x557eab5a90b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4010c50 tx=0x7ff0e400beb0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.885+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/3850922195 shutdown_connections 2024-09-19T06:50:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.885+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/3850922195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5b8cb0 0x557eab5d9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.885+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/3850922195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5ae410 0x557eab5a90b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.885+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/3850922195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5adaf0 0x557eab5be0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.885+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/3850922195 >> 172.21.15.103:0/3850922195 conn(0x557eab5aa060 msgr2=0x557eab5a89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.885+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/3850922195 shutdown_connections 2024-09-19T06:50:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.885+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/3850922195 wait complete. 2024-09-19T06:50:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.885+0000 7ff0fe6c9740 1 -- start start 2024-09-19T06:50:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.885+0000 7ff0fe6c9740 1 Processor -- start 2024-09-19T06:50:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.885+0000 7ff0fe6c9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5adaf0 0x557eab5cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.885+0000 7ff0fe6c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5ae410 0x557eab5cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0fe6c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5b8cb0 0x557eab5ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0fe6c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41ac960 con 0x557eab5ae410 2024-09-19T06:50:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0fe6c9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41ac290 con 0x557eab5adaf0 2024-09-19T06:50:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0fe6c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41a94b0 con 0x557eab5b8cb0 2024-09-19T06:50:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0f96cf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5adaf0 0x557eab5cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0f96cf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5adaf0 0x557eab5cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:34796/0 (socket says 172.21.15.103:34796) 2024-09-19T06:50:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0f96cf640 1 -- 172.21.15.103:0/1707423493 learned_addr learned my addr 172.21.15.103:0/1707423493 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0f8ece640 1 --2- 172.21.15.103:0/1707423493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5ae410 0x557eab5cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0fe6c9740 10 client.0 _ll_register_callbacks cb 0x7ff0e400b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0f9ed0640 1 --2- 172.21.15.103:0/1707423493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5b8cb0 0x557eab5ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0f96cf640 1 -- 172.21.15.103:0/1707423493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5b8cb0 msgr2=0x557eab5ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0f96cf640 1 --2- 172.21.15.103:0/1707423493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5b8cb0 0x557eab5ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0f96cf640 1 -- 172.21.15.103:0/1707423493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5ae410 msgr2=0x557eab5cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0f96cf640 1 --2- 172.21.15.103:0/1707423493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5ae410 0x557eab5cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0f96cf640 1 -- 172.21.15.103:0/1707423493 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eab5caef0 con 0x557eab5adaf0 2024-09-19T06:50:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0f96cf640 1 --2- 172.21.15.103:0/1707423493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5adaf0 0x557eab5cda30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x557eab5d3fb0 tx=0x7ff0f00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb59d1599e31322 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:50:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0dcff9640 1 -- 172.21.15.103:0/1707423493 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0004220 con 0x557eab5adaf0 2024-09-19T06:50:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0dcff9640 1 -- 172.21.15.103:0/1707423493 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff0f00043c0 con 0x557eab5adaf0 2024-09-19T06:50:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.886+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/1707423493 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0e4034040 con 0x557eab5adaf0 2024-09-19T06:50:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.887+0000 7ff0dcff9640 1 -- 172.21.15.103:0/1707423493 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff0f0011820 con 0x557eab5adaf0 2024-09-19T06:50:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.887+0000 7ff0fe6c9740 10 client.25238 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.887+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/1707423493 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557eab63cba0 con 0x557eab5adaf0 2024-09-19T06:50:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.887+0000 7ff0dcff9640 1 -- 172.21.15.103:0/1707423493 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e4034040 con 0x557eab5adaf0 2024-09-19T06:50:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.887+0000 7ff0dcff9640 1 -- 172.21.15.103:0/1707423493 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff0f0003bc0 con 0x557eab5adaf0 2024-09-19T06:50:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.887+0000 7ff0fe6c9740 10 client.25238 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.887+0000 7ff0fe6c9740 10 client.25238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.887+0000 7ff0fe6c9740 20 client.25238 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.887+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/1707423493 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557eab63d2a0 0x557eab63f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.887+0000 7ff0fe6c9740 4 client.25238 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:50:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.887+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/1707423493 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557eab519d70 con 0x557eab63d2a0 2024-09-19T06:50:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.887+0000 7ff0f8ece640 1 --2- 172.21.15.103:0/1707423493 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557eab63d2a0 0x557eab63f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.888+0000 7ff0dcff9640 10 client.25238 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.888+0000 7ff0f8ece640 1 --2- 172.21.15.103:0/1707423493 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557eab63d2a0 0x557eab63f6c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6244c5d3994b38d server_cookie=aa70e3f9094e96f in_seq=0 out_seq=0 2024-09-19T06:50:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.895+0000 7ff0dcff9640 1 -- 172.21.15.103:0/1707423493 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557eab519d70 con 0x557eab63d2a0 2024-09-19T06:50:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.895+0000 7ff0dcff9640 10 client.25238 handle_command_reply: tid=0 2024-09-19T06:50:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.895+0000 7ff0fe6c9740 1 client.25238 shutdown 2024-09-19T06:50:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.895+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/1707423493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5adaf0 msgr2=0x557eab5cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.895+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/1707423493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5adaf0 0x557eab5cda30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x557eab5d3fb0 tx=0x7ff0f00046c0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.895+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/1707423493 shutdown_connections 2024-09-19T06:50:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.895+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/1707423493 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557eab63d2a0 0x557eab63f6c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.895+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/1707423493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eab5b8cb0 0x557eab5ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.895+0000 7ff0dcff9640 0 client.25238 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.895+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/1707423493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eab5ae410 0x557eab5cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.895+0000 7ff0fe6c9740 1 --2- 172.21.15.103:0/1707423493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557eab5adaf0 0x557eab5cda30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.895+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/1707423493 >> 172.21.15.103:0/1707423493 conn(0x557eab5aa060 msgr2=0x557eab5af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.896+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/1707423493 shutdown_connections 2024-09-19T06:50:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.896+0000 7ff0fe6c9740 1 -- 172.21.15.103:0/1707423493 wait complete. 2024-09-19T06:50:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.896+0000 7ff0fe6c9740 20 client.25238 trim_cache size 0 max 16384 2024-09-19T06:50:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:50:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:50:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:50:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:50:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.896+0000 7ff0fb933640 1 -- 172.21.15.103:0/1277159478 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff0e00a3610 msgr2=0x7ff0e00a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.896+0000 7ff0fb933640 1 --2- 172.21.15.103:0/1277159478 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff0e00a3610 0x7ff0e00a5ad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4105e80 tx=0x7ff0e400b410 comp rx=0 tx=0).stop 2024-09-19T06:50:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.896+0000 7ff0fb933640 1 -- 172.21.15.103:0/1277159478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0f41069c0 msgr2=0x7ff0f419ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.896+0000 7ff0fb933640 1 --2- 172.21.15.103:0/1277159478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0f41069c0 0x7ff0f419ba80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec005e30 tx=0x7ff0ec0319a0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.896+0000 7ff0fb933640 1 -- 172.21.15.103:0/1277159478 shutdown_connections 2024-09-19T06:50:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.896+0000 7ff0fb933640 1 --2- 172.21.15.103:0/1277159478 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff0e00a3610 0x7ff0e00a5ad0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.896+0000 7ff0fb933640 1 --2- 172.21.15.103:0/1277159478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0f41069c0 0x7ff0f419ba80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.896+0000 7ff0fb933640 1 --2- 172.21.15.103:0/1277159478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0f4101350 0x7ff0f4197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.896+0000 7ff0fb933640 1 --2- 172.21.15.103:0/1277159478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0f41009b0 0x7ff0f41971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.896+0000 7ff0fb933640 1 -- 172.21.15.103:0/1277159478 >> 172.21.15.103:0/1277159478 conn(0x7ff0f4077990 msgr2=0x7ff0f40feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.897+0000 7ff0fb933640 1 -- 172.21.15.103:0/1277159478 shutdown_connections 2024-09-19T06:50:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:17.897+0000 7ff0fb933640 1 -- 172.21.15.103:0/1277159478 wait complete. 2024-09-19T06:50:18.122 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:18 smithi103 ceph-mon[31158]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T06:50:18.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.247+0000 7eff28ae6640 1 -- 172.21.15.103:0/863274355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff24100e20 msgr2=0x7eff24101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.247+0000 7eff28ae6640 1 --2- 172.21.15.103:0/863274355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff24100e20 0x7eff24101200 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7eff14009900 tx=0x7eff1402a480 comp rx=0 tx=0).stop 2024-09-19T06:50:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.247+0000 7eff28ae6640 1 -- 172.21.15.103:0/863274355 shutdown_connections 2024-09-19T06:50:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.247+0000 7eff28ae6640 1 --2- 172.21.15.103:0/863274355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff24102ef0 0x7eff2410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.247+0000 7eff28ae6640 1 --2- 172.21.15.103:0/863274355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff24102570 0x7eff241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.247+0000 7eff28ae6640 1 --2- 172.21.15.103:0/863274355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff24100e20 0x7eff24101200 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.247+0000 7eff28ae6640 1 -- 172.21.15.103:0/863274355 >> 172.21.15.103:0/863274355 conn(0x7eff240fc7d0 msgr2=0x7eff240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.247+0000 7eff28ae6640 1 -- 172.21.15.103:0/863274355 shutdown_connections 2024-09-19T06:50:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.247+0000 7eff28ae6640 1 -- 172.21.15.103:0/863274355 wait complete. 2024-09-19T06:50:18.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.249+0000 7eff28ae6640 1 Processor -- start 2024-09-19T06:50:18.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.249+0000 7eff28ae6640 1 -- start start 2024-09-19T06:50:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.249+0000 7eff28ae6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff24100e20 0x7eff241973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.249+0000 7eff28ae6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff24102570 0x7eff24197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.249+0000 7eff28ae6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff24102ef0 0x7eff2419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.250+0000 7eff28ae6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7eff24110fa0 con 0x7eff24102570 2024-09-19T06:50:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.250+0000 7eff28ae6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff24110e20 con 0x7eff24102ef0 2024-09-19T06:50:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.250+0000 7eff21d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff24102570 0x7eff24197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.250+0000 7eff28ae6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7eff24111120 con 0x7eff24100e20 2024-09-19T06:50:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.250+0000 7eff21d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff24102570 0x7eff24197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60992/0 (socket says 172.21.15.103:60992) 2024-09-19T06:50:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.250+0000 7eff21d9b640 1 -- 172.21.15.103:0/1757510635 learned_addr learned my addr 172.21.15.103:0/1757510635 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.250+0000 7eff2259c640 1 --2- 172.21.15.103:0/1757510635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff24100e20 0x7eff241973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.250+0000 7eff22d9d640 1 --2- 172.21.15.103:0/1757510635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff24102ef0 0x7eff2419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.250+0000 7eff21d9b640 1 -- 172.21.15.103:0/1757510635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff24100e20 msgr2=0x7eff241973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.250+0000 7eff21d9b640 1 --2- 172.21.15.103:0/1757510635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff24100e20 0x7eff241973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.250+0000 7eff21d9b640 1 -- 172.21.15.103:0/1757510635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff24102ef0 msgr2=0x7eff2419bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.250+0000 7eff21d9b640 1 --2- 172.21.15.103:0/1757510635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff24102ef0 0x7eff2419bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.251+0000 7eff21d9b640 1 -- 172.21.15.103:0/1757510635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2419c390 con 0x7eff24102570 2024-09-19T06:50:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.251+0000 7eff21d9b640 1 --2- 172.21.15.103:0/1757510635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff24102570 0x7eff24197920 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7eff1800d950 tx=0x7eff1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.251+0000 7eff137fe640 1 -- 172.21.15.103:0/1757510635 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18017030 con 0x7eff24102570 2024-09-19T06:50:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.251+0000 7eff137fe640 1 -- 172.21.15.103:0/1757510635 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff18004020 con 0x7eff24102570 2024-09-19T06:50:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.251+0000 7eff28ae6640 1 -- 172.21.15.103:0/1757510635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2419c680 con 0x7eff24102570 2024-09-19T06:50:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.251+0000 7eff137fe640 1 -- 172.21.15.103:0/1757510635 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff180047a0 con 0x7eff24102570 2024-09-19T06:50:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.252+0000 7eff28ae6640 1 -- 172.21.15.103:0/1757510635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff24069f70 con 0x7eff24102570 2024-09-19T06:50:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.253+0000 7eff28ae6640 1 -- 172.21.15.103:0/1757510635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff241021b0 con 0x7eff24102570 2024-09-19T06:50:18.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.255+0000 7eff137fe640 1 -- 172.21.15.103:0/1757510635 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7eff180041d0 con 0x7eff24102570 2024-09-19T06:50:18.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.256+0000 7eff137fe640 1 --2- 172.21.15.103:0/1757510635 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff0c0a35e0 0x7eff0c0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:18.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.256+0000 7eff137fe640 1 -- 172.21.15.103:0/1757510635 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7eff180cf900 con 0x7eff24102570 2024-09-19T06:50:18.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.256+0000 7eff2259c640 1 --2- 172.21.15.103:0/1757510635 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff0c0a35e0 0x7eff0c0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:18.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.257+0000 7eff2259c640 1 --2- 172.21.15.103:0/1757510635 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff0c0a35e0 0x7eff0c0a5aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7eff14000c00 tx=0x7eff14005b50 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:18.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.264+0000 7eff137fe640 1 -- 172.21.15.103:0/1757510635 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7eff1809c0b0 con 0x7eff24102570 2024-09-19T06:50:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:18 smithi174 ceph-mon[35637]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T06:50:18.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:18 smithi134 ceph-mon[35356]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T06:50:18.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.681+0000 7eff28ae6640 1 -- 172.21.15.103:0/1757510635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff2406a230 con 0x7eff24102570 2024-09-19T06:50:18.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.682+0000 7eff137fe640 1 -- 172.21.15.103:0/1757510635 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7eff1809b9b0 con 0x7eff24102570 2024-09-19T06:50:18.683 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:50:18.683 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:50:18.684 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:50:18.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.685+0000 7eff28ae6640 1 -- 172.21.15.103:0/1757510635 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff0c0a35e0 msgr2=0x7eff0c0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:18.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.685+0000 7eff28ae6640 1 --2- 172.21.15.103:0/1757510635 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff0c0a35e0 0x7eff0c0a5aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7eff14000c00 tx=0x7eff14005b50 comp rx=0 tx=0).stop 2024-09-19T06:50:18.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.685+0000 7eff28ae6640 1 -- 172.21.15.103:0/1757510635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff24102570 msgr2=0x7eff24197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:18.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.685+0000 7eff28ae6640 1 --2- 172.21.15.103:0/1757510635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff24102570 0x7eff24197920 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7eff1800d950 tx=0x7eff1800de10 comp rx=0 tx=0).stop 2024-09-19T06:50:18.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.685+0000 7eff28ae6640 1 -- 172.21.15.103:0/1757510635 shutdown_connections 2024-09-19T06:50:18.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.685+0000 7eff28ae6640 1 --2- 172.21.15.103:0/1757510635 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff0c0a35e0 0x7eff0c0a5aa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:18.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.685+0000 7eff28ae6640 1 --2- 172.21.15.103:0/1757510635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff24102ef0 0x7eff2419bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:18.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.685+0000 7eff28ae6640 1 --2- 172.21.15.103:0/1757510635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff24102570 0x7eff24197920 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:18.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.685+0000 7eff28ae6640 1 --2- 172.21.15.103:0/1757510635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff24100e20 0x7eff241973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:18.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.685+0000 7eff28ae6640 1 -- 172.21.15.103:0/1757510635 >> 172.21.15.103:0/1757510635 conn(0x7eff240fc7d0 msgr2=0x7eff240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:18.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.685+0000 7eff28ae6640 1 -- 172.21.15.103:0/1757510635 shutdown_connections 2024-09-19T06:50:18.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:18.685+0000 7eff28ae6640 1 -- 172.21.15.103:0/1757510635 wait complete. 2024-09-19T06:50:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:19 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1757510635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:19.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:19 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1757510635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:19.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:19 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1757510635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:19.443 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:50:19.816 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:20.189 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:20 smithi103 ceph-mon[31158]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2024-09-19T06:50:20.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.318+0000 7fa63cc56640 1 -- 172.21.15.103:0/774289150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa638100a90 msgr2=0x7fa638100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.318+0000 7fa63cc56640 1 --2- 172.21.15.103:0/774289150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa638100a90 0x7fa638100ef0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009950 tx=0x7fa62c02a460 comp rx=0 tx=0).stop 2024-09-19T06:50:20.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.319+0000 7fa63cc56640 1 -- 172.21.15.103:0/774289150 shutdown_connections 2024-09-19T06:50:20.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.319+0000 7fa63cc56640 1 --2- 172.21.15.103:0/774289150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa638101430 0x7fa63810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.319+0000 7fa63cc56640 1 --2- 172.21.15.103:0/774289150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa638100a90 0x7fa638100ef0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.319+0000 7fa63cc56640 1 --2- 172.21.15.103:0/774289150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa638106a90 0x7fa638106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.319+0000 7fa63cc56640 1 -- 172.21.15.103:0/774289150 >> 172.21.15.103:0/774289150 conn(0x7fa6380fc7d0 msgr2=0x7fa6380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:20.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.319+0000 7fa63cc56640 1 -- 172.21.15.103:0/774289150 shutdown_connections 2024-09-19T06:50:20.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.319+0000 7fa63cc56640 1 -- 172.21.15.103:0/774289150 wait complete. 2024-09-19T06:50:20.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.320+0000 7fa63cc56640 1 Processor -- start 2024-09-19T06:50:20.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.320+0000 7fa63cc56640 1 -- start start 2024-09-19T06:50:20.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.320+0000 7fa63cc56640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa638100a90 0x7fa63810e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.320+0000 7fa63cc56640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa638101430 0x7fa63810e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.320+0000 7fa63cc56640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa638106a90 0x7fa63810eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.320+0000 7fa63cc56640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa638111fb0 con 0x7fa638101430 2024-09-19T06:50:20.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.320+0000 7fa63cc56640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa638111e30 con 0x7fa638100a90 2024-09-19T06:50:20.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.320+0000 7fa63cc56640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa638112130 con 0x7fa638106a90 2024-09-19T06:50:20.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.321+0000 7fa63659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa638100a90 0x7fa63810e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.321+0000 7fa63659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa638100a90 0x7fa63810e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:34832/0 (socket says 172.21.15.103:34832) 2024-09-19T06:50:20.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.321+0000 7fa636d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa638106a90 0x7fa63810eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.321+0000 7fa63659c640 1 -- 172.21.15.103:0/314945163 learned_addr learned my addr 172.21.15.103:0/314945163 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:20.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.321+0000 7fa635d9b640 1 --2- 172.21.15.103:0/314945163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa638101430 0x7fa63810e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.321+0000 7fa63659c640 1 -- 172.21.15.103:0/314945163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa638106a90 msgr2=0x7fa63810eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.321+0000 7fa63659c640 1 --2- 172.21.15.103:0/314945163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa638106a90 0x7fa63810eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.321+0000 7fa63659c640 1 -- 172.21.15.103:0/314945163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa638101430 msgr2=0x7fa63810e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.321+0000 7fa63659c640 1 --2- 172.21.15.103:0/314945163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa638101430 0x7fa63810e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.321+0000 7fa63659c640 1 -- 172.21.15.103:0/314945163 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6381a6930 con 0x7fa638100a90 2024-09-19T06:50:20.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.321+0000 7fa635d9b640 1 --2- 172.21.15.103:0/314945163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa638101430 0x7fa63810e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:20.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.322+0000 7fa63659c640 1 --2- 172.21.15.103:0/314945163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa638100a90 0x7fa63810e0f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa62800e9f0 tx=0x7fa62800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:20.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.322+0000 7fa6277fe640 1 -- 172.21.15.103:0/314945163 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62800cd80 con 0x7fa638100a90 2024-09-19T06:50:20.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.322+0000 7fa6277fe640 1 -- 172.21.15.103:0/314945163 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa628010910 con 0x7fa638100a90 2024-09-19T06:50:20.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.322+0000 7fa6277fe640 1 -- 172.21.15.103:0/314945163 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628018980 con 0x7fa638100a90 2024-09-19T06:50:20.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.322+0000 7fa63cc56640 1 -- 172.21.15.103:0/314945163 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6381a6c20 con 0x7fa638100a90 2024-09-19T06:50:20.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.322+0000 7fa63cc56640 1 -- 172.21.15.103:0/314945163 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6381a7050 con 0x7fa638100a90 2024-09-19T06:50:20.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.326+0000 7fa6277fe640 1 -- 172.21.15.103:0/314945163 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7fa628010ac0 con 0x7fa638100a90 2024-09-19T06:50:20.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.326+0000 7fa63f02f740 1 Processor -- start 2024-09-19T06:50:20.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.326+0000 7fa63f02f740 1 -- start start 2024-09-19T06:50:20.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.326+0000 7fa63f02f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd36b40 0x5592abd35090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.326+0000 7fa63f02f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd355d0 0x5592abd35a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.326+0000 7fa63f02f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd40370 0x5592abd42760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.326+0000 7fa63f02f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5592abc9b2f0 con 0x5592abd355d0 2024-09-19T06:50:20.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:20 smithi174 ceph-mon[35637]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2024-09-19T06:50:20.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.326+0000 7fa63f02f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5592abc42cf0 con 0x5592abd36b40 2024-09-19T06:50:20.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.326+0000 7fa63f02f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592abc972b0 con 0x5592abd40370 2024-09-19T06:50:20.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.327+0000 7fa63659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd355d0 0x5592abd35a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.327+0000 7fa63659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd355d0 0x5592abd35a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32804/0 (socket says 172.21.15.103:32804) 2024-09-19T06:50:20.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.327+0000 7fa63659c640 1 -- 172.21.15.103:0/949349572 learned_addr learned my addr 172.21.15.103:0/949349572 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:20.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.327+0000 7fa635d9b640 1 --2- 172.21.15.103:0/949349572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd40370 0x5592abd42760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.327+0000 7fa636d9d640 1 --2- 172.21.15.103:0/949349572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd36b40 0x5592abd35090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.327+0000 7fa6277fe640 1 --2- 172.21.15.103:0/314945163 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa6200a35e0 0x7fa6200a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.327+0000 7fa6277fe640 1 -- 172.21.15.103:0/314945163 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa6280d0340 con 0x7fa638100a90 2024-09-19T06:50:20.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.327+0000 7fa636d9d640 1 --2- 172.21.15.103:0/314945163 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa6200a35e0 0x7fa6200a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.327+0000 7fa63659c640 1 -- 172.21.15.103:0/949349572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd40370 msgr2=0x5592abd42760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:20.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.327+0000 7fa63659c640 1 --2- 172.21.15.103:0/949349572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd40370 0x5592abd42760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.327+0000 7fa63659c640 1 -- 172.21.15.103:0/949349572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd36b40 msgr2=0x5592abd35090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:20.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.327+0000 7fa63659c640 1 --2- 172.21.15.103:0/949349572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd36b40 0x5592abd35090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.327+0000 7fa63659c640 1 -- 172.21.15.103:0/949349572 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592abc273a0 con 0x5592abd355d0 2024-09-19T06:50:20.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.328+0000 7fa63659c640 1 --2- 172.21.15.103:0/949349572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd355d0 0x5592abd35a10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa628098850 tx=0x7fa62800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372dcfb1e486db5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:20.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.328+0000 7fa636d9d640 1 --2- 172.21.15.103:0/314945163 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa6200a35e0 0x7fa6200a5aa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa630005fd0 tx=0x7fa630009d50 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:20.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.328+0000 7fa624ff9640 1 -- 172.21.15.103:0/949349572 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62809caf0 con 0x5592abd355d0 2024-09-19T06:50:20.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.329+0000 7fa624ff9640 1 -- 172.21.15.103:0/949349572 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6280a5060 con 0x5592abd355d0 2024-09-19T06:50:20.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.329+0000 7fa624ff9640 1 -- 172.21.15.103:0/949349572 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6280a4e30 con 0x5592abd355d0 2024-09-19T06:50:20.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.329+0000 7fa63f02f740 1 -- 172.21.15.103:0/949349572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd355d0 msgr2=0x5592abd35a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.329+0000 7fa63f02f740 1 --2- 172.21.15.103:0/949349572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd355d0 0x5592abd35a10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa628098850 tx=0x7fa62800cf20 comp rx=0 tx=0).stop 2024-09-19T06:50:20.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.330+0000 7fa63f02f740 1 -- 172.21.15.103:0/949349572 shutdown_connections 2024-09-19T06:50:20.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.330+0000 7fa63f02f740 1 --2- 172.21.15.103:0/949349572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd40370 0x5592abd42760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.330+0000 7fa63f02f740 1 --2- 172.21.15.103:0/949349572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd355d0 0x5592abd35a10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.330+0000 7fa63f02f740 1 --2- 172.21.15.103:0/949349572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd36b40 0x5592abd35090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.330+0000 7fa63f02f740 1 -- 172.21.15.103:0/949349572 >> 172.21.15.103:0/949349572 conn(0x5592abcfe310 msgr2=0x5592abd31c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:20.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.330+0000 7fa63f02f740 1 -- 172.21.15.103:0/949349572 shutdown_connections 2024-09-19T06:50:20.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.330+0000 7fa63f02f740 1 -- 172.21.15.103:0/949349572 wait complete. 2024-09-19T06:50:20.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.331+0000 7fa63f02f740 1 -- start start 2024-09-19T06:50:20.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.331+0000 7fa63f02f740 1 Processor -- start 2024-09-19T06:50:20.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.331+0000 7fa63f02f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd36b40 0x5592abd3d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.331+0000 7fa63f02f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd40370 0x5592abd3d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa63f02f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd3dd00 0x5592abd4c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa63f02f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5592abd43010 con 0x5592abd40370 2024-09-19T06:50:20.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa635d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd40370 0x5592abd3d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa63f02f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5592abc56060 con 0x5592abd3dd00 2024-09-19T06:50:20.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa63f02f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592abd43190 con 0x5592abd36b40 2024-09-19T06:50:20.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa636d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd3dd00 0x5592abd4c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa636d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd3dd00 0x5592abd4c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:34844/0 (socket says 172.21.15.103:34844) 2024-09-19T06:50:20.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa636d9d640 1 -- 172.21.15.103:0/3539702418 learned_addr learned my addr 172.21.15.103:0/3539702418 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:20.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa63f02f740 10 client.0 _ll_register_callbacks cb 0x5592ab9a98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:20.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa63659c640 1 --2- 172.21.15.103:0/3539702418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd36b40 0x5592abd3d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa636d9d640 1 -- 172.21.15.103:0/3539702418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd36b40 msgr2=0x5592abd3d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa636d9d640 1 --2- 172.21.15.103:0/3539702418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd36b40 0x5592abd3d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa636d9d640 1 -- 172.21.15.103:0/3539702418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd40370 msgr2=0x5592abd3d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa636d9d640 1 --2- 172.21.15.103:0/3539702418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd40370 0x5592abd3d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa636d9d640 1 -- 172.21.15.103:0/3539702418 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592aba0c3c0 con 0x5592abd3dd00 2024-09-19T06:50:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa636d9d640 1 --2- 172.21.15.103:0/3539702418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd3dd00 0x5592abd4c4f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa630004d10 tx=0x7fa630009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e8fa4fdb6f5c78d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.332+0000 7fa61dffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:50:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.333+0000 7fa61dffb640 1 -- 172.21.15.103:0/3539702418 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630005750 con 0x5592abd3dd00 2024-09-19T06:50:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.333+0000 7fa63f02f740 1 -- 172.21.15.103:0/3539702418 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6280a5060 con 0x5592abd3dd00 2024-09-19T06:50:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.333+0000 7fa61dffb640 1 -- 172.21.15.103:0/3539702418 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa6300058f0 con 0x5592abd3dd00 2024-09-19T06:50:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.333+0000 7fa61dffb640 1 -- 172.21.15.103:0/3539702418 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa630028440 con 0x5592abd3dd00 2024-09-19T06:50:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.333+0000 7fa63f02f740 10 client.25256 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.333+0000 7fa63f02f740 1 -- 172.21.15.103:0/3539702418 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5592abd4f150 con 0x5592abd3dd00 2024-09-19T06:50:20.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.333+0000 7fa61dffb640 1 -- 172.21.15.103:0/3539702418 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6280a5060 con 0x5592abd3dd00 2024-09-19T06:50:20.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.334+0000 7fa61dffb640 1 -- 172.21.15.103:0/3539702418 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa630026070 con 0x5592abd3dd00 2024-09-19T06:50:20.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.334+0000 7fa63f02f740 10 client.25256 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:20.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.334+0000 7fa63f02f740 10 client.25256 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:50:20.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.334+0000 7fa63f02f740 20 client.25256 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:20.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.334+0000 7fa63f02f740 1 --2- 172.21.15.103:0/3539702418 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5592abd50a20 0x5592abd52ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.334+0000 7fa63f02f740 4 client.25256 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:20.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.334+0000 7fa63f02f740 1 -- 172.21.15.103:0/3539702418 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592abca8860 con 0x5592abd50a20 2024-09-19T06:50:20.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.334+0000 7fa63659c640 1 --2- 172.21.15.103:0/3539702418 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5592abd50a20 0x5592abd52ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.335+0000 7fa63659c640 1 --2- 172.21.15.103:0/3539702418 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5592abd50a20 0x5592abd52ea0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d007dc7d281d1902 server_cookie=eae2eb2683f06fb3 in_seq=0 out_seq=0 2024-09-19T06:50:20.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.335+0000 7fa61dffb640 10 client.25256 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:50:20.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.336+0000 7fa61dffb640 1 -- 172.21.15.103:0/3539702418 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592abca8860 con 0x5592abd50a20 2024-09-19T06:50:20.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.336+0000 7fa61dffb640 10 client.25256 handle_command_reply: tid=0 2024-09-19T06:50:20.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.336+0000 7fa63f02f740 1 client.25256 shutdown 2024-09-19T06:50:20.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.336+0000 7fa63f02f740 1 -- 172.21.15.103:0/3539702418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd3dd00 msgr2=0x5592abd4c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.336+0000 7fa63f02f740 1 --2- 172.21.15.103:0/3539702418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd3dd00 0x5592abd4c4f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa630004d10 tx=0x7fa630009840 comp rx=0 tx=0).stop 2024-09-19T06:50:20.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.336+0000 7fa63f02f740 1 -- 172.21.15.103:0/3539702418 shutdown_connections 2024-09-19T06:50:20.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.336+0000 7fa63f02f740 1 --2- 172.21.15.103:0/3539702418 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5592abd50a20 0x5592abd52ea0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.336+0000 7fa63f02f740 1 --2- 172.21.15.103:0/3539702418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd3dd00 0x5592abd4c4f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.336+0000 7fa63f02f740 1 --2- 172.21.15.103:0/3539702418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd40370 0x5592abd3d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.336+0000 7fa63f02f740 1 --2- 172.21.15.103:0/3539702418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd36b40 0x5592abd3d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.336+0000 7fa63f02f740 1 -- 172.21.15.103:0/3539702418 >> 172.21.15.103:0/3539702418 conn(0x5592abcfe310 msgr2=0x5592abd37930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:20.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.336+0000 7fa61dffb640 0 client.25256 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:50:20.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.336+0000 7fa63f02f740 1 -- 172.21.15.103:0/3539702418 shutdown_connections 2024-09-19T06:50:20.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.336+0000 7fa63f02f740 1 -- 172.21.15.103:0/3539702418 wait complete. 2024-09-19T06:50:20.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.337+0000 7fa63f02f740 20 client.25256 trim_cache size 0 max 16384 2024-09-19T06:50:20.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.362+0000 7fa63f02f740 1 Processor -- start 2024-09-19T06:50:20.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.362+0000 7fa63f02f740 1 -- start start 2024-09-19T06:50:20.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.362+0000 7fa63f02f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd36b40 0x5592abd3f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.362+0000 7fa63f02f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd3dd00 0x5592abd587b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.363+0000 7fa63f02f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd40370 0x5592abd58cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.363+0000 7fa63f02f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa6381afb20 con 0x5592abd40370 2024-09-19T06:50:20.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.363+0000 7fa63f02f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa6381b1660 con 0x5592abd36b40 2024-09-19T06:50:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.363+0000 7fa63f02f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6381b01f0 con 0x5592abd3dd00 2024-09-19T06:50:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.363+0000 7fa636d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd40370 0x5592abd58cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.363+0000 7fa63659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd36b40 0x5592abd3f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.363+0000 7fa636d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd40370 0x5592abd58cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32816/0 (socket says 172.21.15.103:32816) 2024-09-19T06:50:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.363+0000 7fa636d9d640 1 -- 172.21.15.103:0/1633402304 learned_addr learned my addr 172.21.15.103:0/1633402304 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.363+0000 7fa635d9b640 1 --2- 172.21.15.103:0/1633402304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd3dd00 0x5592abd587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.364+0000 7fa636d9d640 1 -- 172.21.15.103:0/1633402304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd3dd00 msgr2=0x5592abd587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.364+0000 7fa636d9d640 1 --2- 172.21.15.103:0/1633402304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd3dd00 0x5592abd587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.364+0000 7fa636d9d640 1 -- 172.21.15.103:0/1633402304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd36b40 msgr2=0x5592abd3f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.364+0000 7fa636d9d640 1 --2- 172.21.15.103:0/1633402304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd36b40 0x5592abd3f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.364+0000 7fa636d9d640 1 -- 172.21.15.103:0/1633402304 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592abd3c230 con 0x5592abd40370 2024-09-19T06:50:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.364+0000 7fa636d9d640 1 --2- 172.21.15.103:0/1633402304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd40370 0x5592abd58cf0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa630002230 tx=0x7fa630009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10acdd89c755faa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.365+0000 7fa61ffff640 1 -- 172.21.15.103:0/1633402304 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630014790 con 0x5592abd40370 2024-09-19T06:50:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.365+0000 7fa61ffff640 1 -- 172.21.15.103:0/1633402304 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa63001f440 con 0x5592abd40370 2024-09-19T06:50:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.365+0000 7fa61ffff640 1 -- 172.21.15.103:0/1633402304 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630028ac0 con 0x5592abd40370 2024-09-19T06:50:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.366+0000 7fa63f02f740 1 -- 172.21.15.103:0/1633402304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd40370 msgr2=0x5592abd58cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.366+0000 7fa63f02f740 1 --2- 172.21.15.103:0/1633402304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd40370 0x5592abd58cf0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa630002230 tx=0x7fa630009e00 comp rx=0 tx=0).stop 2024-09-19T06:50:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.366+0000 7fa63f02f740 1 -- 172.21.15.103:0/1633402304 shutdown_connections 2024-09-19T06:50:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.366+0000 7fa63f02f740 1 --2- 172.21.15.103:0/1633402304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd40370 0x5592abd58cf0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.366+0000 7fa63f02f740 1 --2- 172.21.15.103:0/1633402304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd3dd00 0x5592abd587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.366+0000 7fa63f02f740 1 --2- 172.21.15.103:0/1633402304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd36b40 0x5592abd3f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.366+0000 7fa63f02f740 1 -- 172.21.15.103:0/1633402304 >> 172.21.15.103:0/1633402304 conn(0x5592abcfe310 msgr2=0x5592abd3eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.366+0000 7fa63f02f740 1 -- 172.21.15.103:0/1633402304 shutdown_connections 2024-09-19T06:50:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.366+0000 7fa63f02f740 1 -- 172.21.15.103:0/1633402304 wait complete. 2024-09-19T06:50:20.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.367+0000 7fa63f02f740 1 -- start start 2024-09-19T06:50:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.367+0000 7fa63f02f740 1 Processor -- start 2024-09-19T06:50:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.367+0000 7fa63f02f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd36b40 0x5592abd48be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.367+0000 7fa63659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd36b40 0x5592abd48be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.367+0000 7fa63659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd36b40 0x5592abd48be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32818/0 (socket says 172.21.15.103:32818) 2024-09-19T06:50:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.367+0000 7fa63659c640 1 -- 172.21.15.103:0/3504642610 learned_addr learned my addr 172.21.15.103:0/3504642610 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.367+0000 7fa63f02f740 1 --2- 172.21.15.103:0/3504642610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd3dd00 0x5592abd41100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.367+0000 7fa63f02f740 1 --2- 172.21.15.103:0/3504642610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd40370 0x5592abd41640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.367+0000 7fa63f02f740 1 -- 172.21.15.103:0/3504642610 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa6381b0f90 con 0x5592abd36b40 2024-09-19T06:50:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.367+0000 7fa63f02f740 1 -- 172.21.15.103:0/3504642610 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa6381b08c0 con 0x5592abd40370 2024-09-19T06:50:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.368+0000 7fa63f02f740 1 -- 172.21.15.103:0/3504642610 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6381adae0 con 0x5592abd3dd00 2024-09-19T06:50:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.368+0000 7fa63f02f740 10 client.0 _ll_register_callbacks cb 0x7fa630012f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.368+0000 7fa635d9b640 1 --2- 172.21.15.103:0/3504642610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd3dd00 0x5592abd41100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.368+0000 7fa636d9d640 1 --2- 172.21.15.103:0/3504642610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd40370 0x5592abd41640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.368+0000 7fa63659c640 1 -- 172.21.15.103:0/3504642610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd3dd00 msgr2=0x5592abd41100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.368+0000 7fa63659c640 1 --2- 172.21.15.103:0/3504642610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd3dd00 0x5592abd41100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.368+0000 7fa63659c640 1 -- 172.21.15.103:0/3504642610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd40370 msgr2=0x5592abd41640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.368+0000 7fa63659c640 1 --2- 172.21.15.103:0/3504642610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd40370 0x5592abd41640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.368+0000 7fa63659c640 1 -- 172.21.15.103:0/3504642610 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592abd41e40 con 0x5592abd36b40 2024-09-19T06:50:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.368+0000 7fa63659c640 1 --2- 172.21.15.103:0/3504642610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd36b40 0x5592abd48be0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa62809c0f0 tx=0x7fa628098d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4117e39a0e16bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.368+0000 7fa61dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.369+0000 7fa61dffb640 1 -- 172.21.15.103:0/3504642610 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6280a5060 con 0x5592abd36b40 2024-09-19T06:50:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.369+0000 7fa61dffb640 1 -- 172.21.15.103:0/3504642610 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa6280d3b90 con 0x5592abd36b40 2024-09-19T06:50:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.369+0000 7fa63f02f740 1 -- 172.21.15.103:0/3504642610 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa63001f440 con 0x5592abd36b40 2024-09-19T06:50:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.369+0000 7fa61dffb640 1 -- 172.21.15.103:0/3504642610 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa6280a0160 con 0x5592abd36b40 2024-09-19T06:50:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.369+0000 7fa63f02f740 10 client.15831 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:20.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.369+0000 7fa63f02f740 1 -- 172.21.15.103:0/3504642610 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5592abdbb7e0 con 0x5592abd36b40 2024-09-19T06:50:20.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.369+0000 7fa61dffb640 1 -- 172.21.15.103:0/3504642610 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa63001f440 con 0x5592abd36b40 2024-09-19T06:50:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.370+0000 7fa61dffb640 1 -- 172.21.15.103:0/3504642610 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa62809b840 con 0x5592abd36b40 2024-09-19T06:50:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.370+0000 7fa63f02f740 10 client.15831 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.370+0000 7fa63f02f740 10 client.15831 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:50:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.370+0000 7fa63f02f740 20 client.15831 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.370+0000 7fa63f02f740 1 --2- 172.21.15.103:0/3504642610 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5592abdbba70 0x5592abdbde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.370+0000 7fa63f02f740 4 client.15831 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.370+0000 7fa63f02f740 1 -- 172.21.15.103:0/3504642610 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592abca2d70 con 0x5592abdbba70 2024-09-19T06:50:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.370+0000 7fa635d9b640 1 --2- 172.21.15.103:0/3504642610 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5592abdbba70 0x5592abdbde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.371+0000 7fa635d9b640 1 --2- 172.21.15.103:0/3504642610 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5592abdbba70 0x5592abdbde60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ab2d64cb35b5c074 server_cookie=6de7a8e1d064304f in_seq=0 out_seq=0 2024-09-19T06:50:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.371+0000 7fa61dffb640 10 client.15831 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:50:20.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.371+0000 7fa61dffb640 1 -- 172.21.15.103:0/3504642610 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592abca2d70 con 0x5592abdbba70 2024-09-19T06:50:20.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.371+0000 7fa61dffb640 10 client.15831 handle_command_reply: tid=0 2024-09-19T06:50:20.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.371+0000 7fa63f02f740 1 client.15831 shutdown 2024-09-19T06:50:20.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.371+0000 7fa63f02f740 1 -- 172.21.15.103:0/3504642610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd36b40 msgr2=0x5592abd48be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.371+0000 7fa63f02f740 1 --2- 172.21.15.103:0/3504642610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd36b40 0x5592abd48be0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa62809c0f0 tx=0x7fa628098d30 comp rx=0 tx=0).stop 2024-09-19T06:50:20.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.371+0000 7fa63f02f740 1 -- 172.21.15.103:0/3504642610 shutdown_connections 2024-09-19T06:50:20.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.371+0000 7fa63f02f740 1 --2- 172.21.15.103:0/3504642610 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5592abdbba70 0x5592abdbde60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.371+0000 7fa63f02f740 1 --2- 172.21.15.103:0/3504642610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592abd40370 0x5592abd41640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.371+0000 7fa63f02f740 1 --2- 172.21.15.103:0/3504642610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592abd3dd00 0x5592abd41100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.371+0000 7fa63f02f740 1 --2- 172.21.15.103:0/3504642610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592abd36b40 0x5592abd48be0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.371+0000 7fa63f02f740 1 -- 172.21.15.103:0/3504642610 >> 172.21.15.103:0/3504642610 conn(0x5592abcfe310 msgr2=0x5592abd3f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:20.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.372+0000 7fa61dffb640 0 client.15831 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:50:20.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.372+0000 7fa63f02f740 1 -- 172.21.15.103:0/3504642610 shutdown_connections 2024-09-19T06:50:20.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.372+0000 7fa63f02f740 1 -- 172.21.15.103:0/3504642610 wait complete. 2024-09-19T06:50:20.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.372+0000 7fa63f02f740 20 client.15831 trim_cache size 0 max 16384 2024-09-19T06:50:20.376 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:50:20.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.372+0000 7fa63cc56640 1 -- 172.21.15.103:0/314945163 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa6200a35e0 msgr2=0x7fa6200a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.372+0000 7fa63cc56640 1 --2- 172.21.15.103:0/314945163 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa6200a35e0 0x7fa6200a5aa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa630005fd0 tx=0x7fa630009d50 comp rx=0 tx=0).stop 2024-09-19T06:50:20.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.372+0000 7fa63cc56640 1 -- 172.21.15.103:0/314945163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa638100a90 msgr2=0x7fa63810e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.372+0000 7fa63cc56640 1 --2- 172.21.15.103:0/314945163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa638100a90 0x7fa63810e0f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa62800e9f0 tx=0x7fa62800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.373+0000 7fa63cc56640 1 -- 172.21.15.103:0/314945163 shutdown_connections 2024-09-19T06:50:20.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.373+0000 7fa63cc56640 1 --2- 172.21.15.103:0/314945163 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa6200a35e0 0x7fa6200a5aa0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.373+0000 7fa63cc56640 1 --2- 172.21.15.103:0/314945163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa638106a90 0x7fa63810eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.373+0000 7fa63cc56640 1 --2- 172.21.15.103:0/314945163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa638101430 0x7fa63810e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.373+0000 7fa63cc56640 1 --2- 172.21.15.103:0/314945163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa638100a90 0x7fa63810e0f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.373+0000 7fa63cc56640 1 -- 172.21.15.103:0/314945163 >> 172.21.15.103:0/314945163 conn(0x7fa6380fc7d0 msgr2=0x7fa6380fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:20.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.373+0000 7fa63cc56640 1 -- 172.21.15.103:0/314945163 shutdown_connections 2024-09-19T06:50:20.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:20.373+0000 7fa63cc56640 1 -- 172.21.15.103:0/314945163 wait complete. 2024-09-19T06:50:20.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:20 smithi134 ceph-mon[35356]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2024-09-19T06:50:21.082 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:50:21.356 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:21.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.888+0000 7f7c6a84a640 1 -- 172.21.15.103:0/186455827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c6410c160 msgr2=0x7f7c6410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:21.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.888+0000 7f7c6a84a640 1 --2- 172.21.15.103:0/186455827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c6410c160 0x7f7c6410e600 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c009ca0 tx=0x7f7c5c02a950 comp rx=0 tx=0).stop 2024-09-19T06:50:21.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.888+0000 7f7c6a84a640 1 -- 172.21.15.103:0/186455827 shutdown_connections 2024-09-19T06:50:21.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.888+0000 7f7c6a84a640 1 --2- 172.21.15.103:0/186455827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c6410c160 0x7f7c6410e600 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.888+0000 7f7c6a84a640 1 --2- 172.21.15.103:0/186455827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c64103650 0x7f7c6410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.888+0000 7f7c6a84a640 1 --2- 172.21.15.103:0/186455827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c64102d30 0x7f7c64103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.888+0000 7f7c6a84a640 1 -- 172.21.15.103:0/186455827 >> 172.21.15.103:0/186455827 conn(0x7f7c640fc790 msgr2=0x7f7c640febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:21.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.888+0000 7f7c6a84a640 1 -- 172.21.15.103:0/186455827 shutdown_connections 2024-09-19T06:50:21.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.888+0000 7f7c6a84a640 1 -- 172.21.15.103:0/186455827 wait complete. 2024-09-19T06:50:21.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.888+0000 7f7c6a84a640 1 Processor -- start 2024-09-19T06:50:21.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.889+0000 7f7c6a84a640 1 -- start start 2024-09-19T06:50:21.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.889+0000 7f7c6a84a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c64102d30 0x7f7c64100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.889+0000 7f7c6a84a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c64103650 0x7f7c64100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.889+0000 7f7c6a84a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c6410c160 0x7f7c64101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.889+0000 7f7c6a84a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c64111f20 con 0x7f7c64103650 2024-09-19T06:50:21.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.889+0000 7f7c6a84a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c64111da0 con 0x7f7c64102d30 2024-09-19T06:50:21.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.889+0000 7f7c6a84a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c641120a0 con 0x7f7c6410c160 2024-09-19T06:50:21.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.889+0000 7f7c637fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c64103650 0x7f7c64100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:21.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.889+0000 7f7c637fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c64103650 0x7f7c64100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32838/0 (socket says 172.21.15.103:32838) 2024-09-19T06:50:21.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.889+0000 7f7c637fe640 1 -- 172.21.15.103:0/886067197 learned_addr learned my addr 172.21.15.103:0/886067197 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:21.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.889+0000 7f7c68de7640 1 --2- 172.21.15.103:0/886067197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c6410c160 0x7f7c64101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:21.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.890+0000 7f7c63fff640 1 --2- 172.21.15.103:0/886067197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c64102d30 0x7f7c64100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:21.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.890+0000 7f7c637fe640 1 -- 172.21.15.103:0/886067197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c6410c160 msgr2=0x7f7c64101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:21.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.890+0000 7f7c637fe640 1 --2- 172.21.15.103:0/886067197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c6410c160 0x7f7c64101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.890+0000 7f7c637fe640 1 -- 172.21.15.103:0/886067197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c64102d30 msgr2=0x7f7c64100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:21.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.890+0000 7f7c637fe640 1 --2- 172.21.15.103:0/886067197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c64102d30 0x7f7c64100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.890+0000 7f7c637fe640 1 -- 172.21.15.103:0/886067197 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c641a6820 con 0x7f7c64103650 2024-09-19T06:50:21.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.890+0000 7f7c68de7640 1 --2- 172.21.15.103:0/886067197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c6410c160 0x7f7c64101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:21.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.890+0000 7f7c637fe640 1 --2- 172.21.15.103:0/886067197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c64103650 0x7f7c64100ed0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800c940 tx=0x7f7c5800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:21.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.890+0000 7f7c617fa640 1 -- 172.21.15.103:0/886067197 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58004220 con 0x7f7c64103650 2024-09-19T06:50:21.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.890+0000 7f7c6a84a640 1 -- 172.21.15.103:0/886067197 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c641a6b10 con 0x7f7c64103650 2024-09-19T06:50:21.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.890+0000 7f7c6a84a640 1 -- 172.21.15.103:0/886067197 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c641a70a0 con 0x7f7c64103650 2024-09-19T06:50:21.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.891+0000 7f7c617fa640 1 -- 172.21.15.103:0/886067197 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c580043c0 con 0x7f7c64103650 2024-09-19T06:50:21.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.891+0000 7f7c617fa640 1 -- 172.21.15.103:0/886067197 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58004a00 con 0x7f7c64103650 2024-09-19T06:50:21.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.893+0000 7f7c617fa640 1 -- 172.21.15.103:0/886067197 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7f7c58004ba0 con 0x7f7c64103650 2024-09-19T06:50:21.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.893+0000 7f7c6cc23740 1 Processor -- start 2024-09-19T06:50:21.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.893+0000 7f7c6cc23740 1 -- start start 2024-09-19T06:50:21.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.894+0000 7f7c6cc23740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e05350 0x5596a4e038a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.894+0000 7f7c6cc23740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e03de0 0x5596a4e04220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.894+0000 7f7c6cc23740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e0eb80 0x5596a4e10f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.894+0000 7f7c6cc23740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5596a4d612f0 con 0x5596a4e03de0 2024-09-19T06:50:21.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.894+0000 7f7c6cc23740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5596a4d08cf0 con 0x5596a4e0eb80 2024-09-19T06:50:21.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.894+0000 7f7c6cc23740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5596a4d5d2b0 con 0x5596a4e05350 2024-09-19T06:50:21.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.894+0000 7f7c617fa640 1 --2- 172.21.15.103:0/886067197 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c4c0a3660 0x7f7c4c0a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.894+0000 7f7c617fa640 1 -- 172.21.15.103:0/886067197 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f7c580cfde0 con 0x7f7c64103650 2024-09-19T06:50:21.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.894+0000 7f7c63fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e03de0 0x5596a4e04220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:21.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.894+0000 7f7c63fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e03de0 0x5596a4e04220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32850/0 (socket says 172.21.15.103:32850) 2024-09-19T06:50:21.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.894+0000 7f7c63fff640 1 -- 172.21.15.103:0/1950010635 learned_addr learned my addr 172.21.15.103:0/1950010635 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:21.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.894+0000 7f7c68de7640 1 --2- 172.21.15.103:0/886067197 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c4c0a3660 0x7f7c4c0a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:21.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.894+0000 7f7c637fe640 1 --2- 172.21.15.103:0/1950010635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e0eb80 0x5596a4e10f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:21.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.895+0000 7f7c63fff640 1 -- 172.21.15.103:0/1950010635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e05350 msgr2=0x5596a4e038a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:50:21.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.895+0000 7f7c63fff640 1 --2- 172.21.15.103:0/1950010635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e05350 0x5596a4e038a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.895+0000 7f7c63fff640 1 -- 172.21.15.103:0/1950010635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e0eb80 msgr2=0x5596a4e10f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:21.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.895+0000 7f7c63fff640 1 --2- 172.21.15.103:0/1950010635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e0eb80 0x5596a4e10f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.895+0000 7f7c63fff640 1 -- 172.21.15.103:0/1950010635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596a4ced3a0 con 0x5596a4e03de0 2024-09-19T06:50:21.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.895+0000 7f7c63fff640 1 --2- 172.21.15.103:0/1950010635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e03de0 0x5596a4e04220 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400cec0 tx=0x7f7c5400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d1109007e9caea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:21.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.895+0000 7f7c68de7640 1 --2- 172.21.15.103:0/886067197 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c4c0a3660 0x7f7c4c0a5b20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c030040 tx=0x7f7c5c005c30 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:21.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.895+0000 7f7c4a7fc640 1 -- 172.21.15.103:0/1950010635 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54014070 con 0x5596a4e03de0 2024-09-19T06:50:21.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.895+0000 7f7c4a7fc640 1 -- 172.21.15.103:0/1950010635 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c540047a0 con 0x5596a4e03de0 2024-09-19T06:50:21.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.895+0000 7f7c4a7fc640 1 -- 172.21.15.103:0/1950010635 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c540107b0 con 0x5596a4e03de0 2024-09-19T06:50:21.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.896+0000 7f7c6cc23740 1 -- 172.21.15.103:0/1950010635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e03de0 msgr2=0x5596a4e04220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:21.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.896+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/1950010635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e03de0 0x5596a4e04220 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400cec0 tx=0x7f7c5400e720 comp rx=0 tx=0).stop 2024-09-19T06:50:21.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.896+0000 7f7c6cc23740 1 -- 172.21.15.103:0/1950010635 shutdown_connections 2024-09-19T06:50:21.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.896+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/1950010635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e0eb80 0x5596a4e10f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.896+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/1950010635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e03de0 0x5596a4e04220 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.896+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/1950010635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e05350 0x5596a4e038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.896+0000 7f7c6cc23740 1 -- 172.21.15.103:0/1950010635 >> 172.21.15.103:0/1950010635 conn(0x5596a4dc4310 msgr2=0x5596a4e00470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:21.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.896+0000 7f7c6cc23740 1 -- 172.21.15.103:0/1950010635 shutdown_connections 2024-09-19T06:50:21.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.896+0000 7f7c6cc23740 1 -- 172.21.15.103:0/1950010635 wait complete. 2024-09-19T06:50:21.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.897+0000 7f7c6cc23740 1 -- start start 2024-09-19T06:50:21.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.897+0000 7f7c6cc23740 1 Processor -- start 2024-09-19T06:50:21.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.897+0000 7f7c6cc23740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e03de0 0x5596a4e0e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.897+0000 7f7c6cc23740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e05350 0x5596a4e0b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.897+0000 7f7c6cc23740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e0eb80 0x5596a4e0bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.897+0000 7f7c637fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e05350 0x5596a4e0b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:21.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.897+0000 7f7c637fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e05350 0x5596a4e0b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32852/0 (socket says 172.21.15.103:32852) 2024-09-19T06:50:21.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.897+0000 7f7c63fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e03de0 0x5596a4e0e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:21.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.897+0000 7f7c637fe640 1 -- 172.21.15.103:0/3729527203 learned_addr learned my addr 172.21.15.103:0/3729527203 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:21.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.897+0000 7f7c6cc23740 1 -- 172.21.15.103:0/3729527203 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5596a4e11880 con 0x5596a4e05350 2024-09-19T06:50:21.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.897+0000 7f7c6cc23740 1 -- 172.21.15.103:0/3729527203 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5596a4d1c060 con 0x5596a4e03de0 2024-09-19T06:50:21.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.897+0000 7f7c6cc23740 1 -- 172.21.15.103:0/3729527203 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5596a4e11a00 con 0x5596a4e0eb80 2024-09-19T06:50:21.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.898+0000 7f7c6cc23740 10 client.0 _ll_register_callbacks cb 0x5596a4a6f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:21.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.898+0000 7f7c637fe640 1 -- 172.21.15.103:0/3729527203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e0eb80 msgr2=0x5596a4e0bd80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:50:21.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.898+0000 7f7c637fe640 1 --2- 172.21.15.103:0/3729527203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e0eb80 0x5596a4e0bd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.898+0000 7f7c637fe640 1 -- 172.21.15.103:0/3729527203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e03de0 msgr2=0x5596a4e0e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:21.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.898+0000 7f7c637fe640 1 --2- 172.21.15.103:0/3729527203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e03de0 0x5596a4e0e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.898+0000 7f7c637fe640 1 -- 172.21.15.103:0/3729527203 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596a4ad23c0 con 0x5596a4e05350 2024-09-19T06:50:21.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.898+0000 7f7c637fe640 1 --2- 172.21.15.103:0/3729527203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e05350 0x5596a4e0b7e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7c58007590 tx=0x7f7c5809b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9894ef1751688b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:21.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.898+0000 7f7c437fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:21.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.898+0000 7f7c437fe640 1 -- 172.21.15.103:0/3729527203 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5801d040 con 0x5596a4e05350 2024-09-19T06:50:21.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.898+0000 7f7c437fe640 1 -- 172.21.15.103:0/3729527203 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f7c580a4670 con 0x5596a4e05350 2024-09-19T06:50:21.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.898+0000 7f7c6cc23740 1 -- 172.21.15.103:0/3729527203 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c540047a0 con 0x5596a4e05350 2024-09-19T06:50:21.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.899+0000 7f7c437fe640 1 -- 172.21.15.103:0/3729527203 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7c58018e60 con 0x5596a4e05350 2024-09-19T06:50:21.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.899+0000 7f7c6cc23740 10 client.15843 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:21.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.899+0000 7f7c6cc23740 1 -- 172.21.15.103:0/3729527203 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5596a4e1d370 con 0x5596a4e05350 2024-09-19T06:50:21.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.899+0000 7f7c437fe640 1 -- 172.21.15.103:0/3729527203 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c540047a0 con 0x5596a4e05350 2024-09-19T06:50:21.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.899+0000 7f7c437fe640 1 -- 172.21.15.103:0/3729527203 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7c5807bbe0 con 0x5596a4e05350 2024-09-19T06:50:21.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.899+0000 7f7c6cc23740 10 client.15843 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:21.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.899+0000 7f7c6cc23740 10 client.15843 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:50:21.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.899+0000 7f7c6cc23740 20 client.15843 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:21.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.899+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/3729527203 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5596a4e1e870 0x5596a4e20d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.899+0000 7f7c6cc23740 4 client.15843 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:21.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.899+0000 7f7c6cc23740 1 -- 172.21.15.103:0/3729527203 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596a4d6e860 con 0x5596a4e1e870 2024-09-19T06:50:21.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.899+0000 7f7c63fff640 1 --2- 172.21.15.103:0/3729527203 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5596a4e1e870 0x5596a4e20d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:21.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.900+0000 7f7c437fe640 10 client.15843 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:50:21.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.900+0000 7f7c63fff640 1 --2- 172.21.15.103:0/3729527203 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5596a4e1e870 0x5596a4e20d10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ffbc6d625874de6e server_cookie=37183c2053c6beb5 in_seq=0 out_seq=0 2024-09-19T06:50:21.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.901+0000 7f7c437fe640 1 -- 172.21.15.103:0/3729527203 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596a4d6e860 con 0x5596a4e1e870 2024-09-19T06:50:21.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.901+0000 7f7c437fe640 10 client.15843 handle_command_reply: tid=0 2024-09-19T06:50:21.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.901+0000 7f7c6cc23740 1 client.15843 shutdown 2024-09-19T06:50:21.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.901+0000 7f7c6cc23740 1 -- 172.21.15.103:0/3729527203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e05350 msgr2=0x5596a4e0b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:21.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.901+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/3729527203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e05350 0x5596a4e0b7e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7c58007590 tx=0x7f7c5809b8b0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.901+0000 7f7c6cc23740 1 -- 172.21.15.103:0/3729527203 shutdown_connections 2024-09-19T06:50:21.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.901+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/3729527203 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5596a4e1e870 0x5596a4e20d10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.901+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/3729527203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e0eb80 0x5596a4e0bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.901+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/3729527203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e05350 0x5596a4e0b7e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.901+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/3729527203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e03de0 0x5596a4e0e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.901+0000 7f7c6cc23740 1 -- 172.21.15.103:0/3729527203 >> 172.21.15.103:0/3729527203 conn(0x5596a4dc4310 msgr2=0x5596a4e06140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:21.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.901+0000 7f7c437fe640 0 client.15843 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:50:21.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.901+0000 7f7c6cc23740 1 -- 172.21.15.103:0/3729527203 shutdown_connections 2024-09-19T06:50:21.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.901+0000 7f7c6cc23740 1 -- 172.21.15.103:0/3729527203 wait complete. 2024-09-19T06:50:21.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.901+0000 7f7c6cc23740 20 client.15843 trim_cache size 0 max 16384 2024-09-19T06:50:21.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.915+0000 7f7c6cc23740 1 Processor -- start 2024-09-19T06:50:21.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.915+0000 7f7c6cc23740 1 -- start start 2024-09-19T06:50:21.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.915+0000 7f7c6cc23740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e03de0 0x5596a4e1ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.915+0000 7f7c6cc23740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e05350 0x5596a4e0b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.915+0000 7f7c6cc23740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e0eb80 0x5596a4e0a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.915+0000 7f7c6cc23740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5000e6f0 con 0x5596a4e0eb80 2024-09-19T06:50:21.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.915+0000 7f7c6cc23740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c50010230 con 0x5596a4e05350 2024-09-19T06:50:21.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.915+0000 7f7c6cc23740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5000edc0 con 0x5596a4e03de0 2024-09-19T06:50:21.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.915+0000 7f7c68de7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e0eb80 0x5596a4e0a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:21.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.915+0000 7f7c68de7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e0eb80 0x5596a4e0a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32864/0 (socket says 172.21.15.103:32864) 2024-09-19T06:50:21.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.915+0000 7f7c68de7640 1 -- 172.21.15.103:0/1173165057 learned_addr learned my addr 172.21.15.103:0/1173165057 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:21.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.915+0000 7f7c63fff640 1 --2- 172.21.15.103:0/1173165057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e03de0 0x5596a4e1ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:21.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.916+0000 7f7c637fe640 1 --2- 172.21.15.103:0/1173165057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e05350 0x5596a4e0b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:21.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.916+0000 7f7c68de7640 1 -- 172.21.15.103:0/1173165057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e03de0 msgr2=0x5596a4e1ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:21.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.916+0000 7f7c68de7640 1 --2- 172.21.15.103:0/1173165057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e03de0 0x5596a4e1ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.916+0000 7f7c68de7640 1 -- 172.21.15.103:0/1173165057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e05350 msgr2=0x5596a4e0b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:21.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.916+0000 7f7c68de7640 1 --2- 172.21.15.103:0/1173165057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e05350 0x5596a4e0b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.916+0000 7f7c68de7640 1 -- 172.21.15.103:0/1173165057 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596a4e00f40 con 0x5596a4e0eb80 2024-09-19T06:50:21.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.916+0000 7f7c637fe640 1 --2- 172.21.15.103:0/1173165057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e05350 0x5596a4e0b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:21.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.916+0000 7f7c68de7640 1 --2- 172.21.15.103:0/1173165057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e0eb80 0x5596a4e0a0b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c009960 tx=0x7f7c5c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac14d819e89e8338 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:21.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.916+0000 7f7c49ffb640 1 -- 172.21.15.103:0/1173165057 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c034040 con 0x5596a4e0eb80 2024-09-19T06:50:21.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.916+0000 7f7c49ffb640 1 -- 172.21.15.103:0/1173165057 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c5c003b70 con 0x5596a4e0eb80 2024-09-19T06:50:21.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.916+0000 7f7c49ffb640 1 -- 172.21.15.103:0/1173165057 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c03b6a0 con 0x5596a4e0eb80 2024-09-19T06:50:21.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.917+0000 7f7c6cc23740 1 -- 172.21.15.103:0/1173165057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e0eb80 msgr2=0x5596a4e0a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.917+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/1173165057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e0eb80 0x5596a4e0a0b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c009960 tx=0x7f7c5c03e000 comp rx=0 tx=0).stop 2024-09-19T06:50:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.917+0000 7f7c6cc23740 1 -- 172.21.15.103:0/1173165057 shutdown_connections 2024-09-19T06:50:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.917+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/1173165057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e0eb80 0x5596a4e0a0b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.917+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/1173165057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e05350 0x5596a4e0b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.917+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/1173165057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e03de0 0x5596a4e1ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.917+0000 7f7c6cc23740 1 -- 172.21.15.103:0/1173165057 >> 172.21.15.103:0/1173165057 conn(0x5596a4dc4310 msgr2=0x5596a4e06a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.917+0000 7f7c6cc23740 1 -- 172.21.15.103:0/1173165057 shutdown_connections 2024-09-19T06:50:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.917+0000 7f7c6cc23740 1 -- 172.21.15.103:0/1173165057 wait complete. 2024-09-19T06:50:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.917+0000 7f7c6cc23740 1 -- start start 2024-09-19T06:50:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.918+0000 7f7c6cc23740 1 Processor -- start 2024-09-19T06:50:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.918+0000 7f7c6cc23740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e03de0 0x5596a4e17280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.918+0000 7f7c6cc23740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e05350 0x5596a4e25cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.918+0000 7f7c6cc23740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e0eb80 0x5596a4e26210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.918+0000 7f7c6cc23740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5000fb60 con 0x5596a4e0eb80 2024-09-19T06:50:21.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.918+0000 7f7c6cc23740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5000f490 con 0x5596a4e03de0 2024-09-19T06:50:21.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.918+0000 7f7c6cc23740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5000c6b0 con 0x5596a4e05350 2024-09-19T06:50:21.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.918+0000 7f7c6cc23740 10 client.0 _ll_register_callbacks cb 0x7f7c580058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:21.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.918+0000 7f7c68de7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e0eb80 0x5596a4e26210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:21.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.918+0000 7f7c63fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e03de0 0x5596a4e17280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:21.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.918+0000 7f7c68de7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e0eb80 0x5596a4e26210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32872/0 (socket says 172.21.15.103:32872) 2024-09-19T06:50:21.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.918+0000 7f7c68de7640 1 -- 172.21.15.103:0/2683874846 learned_addr learned my addr 172.21.15.103:0/2683874846 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:21.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.918+0000 7f7c637fe640 1 --2- 172.21.15.103:0/2683874846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e05350 0x5596a4e25cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:21.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.919+0000 7f7c68de7640 1 -- 172.21.15.103:0/2683874846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e05350 msgr2=0x5596a4e25cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:21.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.919+0000 7f7c68de7640 1 --2- 172.21.15.103:0/2683874846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e05350 0x5596a4e25cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.919+0000 7f7c68de7640 1 -- 172.21.15.103:0/2683874846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e03de0 msgr2=0x5596a4e17280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:21.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.919+0000 7f7c68de7640 1 --2- 172.21.15.103:0/2683874846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e03de0 0x5596a4e17280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.919+0000 7f7c68de7640 1 -- 172.21.15.103:0/2683874846 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596a4e26800 con 0x5596a4e0eb80 2024-09-19T06:50:21.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.919+0000 7f7c68de7640 1 --2- 172.21.15.103:0/2683874846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e0eb80 0x5596a4e26210 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c005cc0 tx=0x7f7c5c004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb55a6f4fb390ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:21.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.919+0000 7f7c437fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:21.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.919+0000 7f7c437fe640 1 -- 172.21.15.103:0/2683874846 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c003970 con 0x5596a4e0eb80 2024-09-19T06:50:21.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.919+0000 7f7c437fe640 1 -- 172.21.15.103:0/2683874846 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f7c5c030650 con 0x5596a4e0eb80 2024-09-19T06:50:21.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.919+0000 7f7c6cc23740 1 -- 172.21.15.103:0/2683874846 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c5c003b70 con 0x5596a4e0eb80 2024-09-19T06:50:21.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.919+0000 7f7c437fe640 1 -- 172.21.15.103:0/2683874846 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7c5c030850 con 0x5596a4e0eb80 2024-09-19T06:50:21.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.919+0000 7f7c6cc23740 10 client.15849 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:21.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.919+0000 7f7c6cc23740 1 -- 172.21.15.103:0/2683874846 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5596a4e15ac0 con 0x5596a4e0eb80 2024-09-19T06:50:21.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.920+0000 7f7c437fe640 1 -- 172.21.15.103:0/2683874846 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c5c003b70 con 0x5596a4e0eb80 2024-09-19T06:50:21.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.920+0000 7f7c437fe640 1 -- 172.21.15.103:0/2683874846 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7c5c060050 con 0x5596a4e0eb80 2024-09-19T06:50:21.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.920+0000 7f7c6cc23740 10 client.15849 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:21.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.920+0000 7f7c6cc23740 10 client.15849 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:50:21.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.920+0000 7f7c6cc23740 20 client.15849 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:21.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.920+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/2683874846 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5596a4e89920 0x5596a4e8bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:21.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.920+0000 7f7c6cc23740 4 client.15849 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:21.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.920+0000 7f7c6cc23740 1 -- 172.21.15.103:0/2683874846 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596a4d68d70 con 0x5596a4e89920 2024-09-19T06:50:21.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.920+0000 7f7c63fff640 1 --2- 172.21.15.103:0/2683874846 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5596a4e89920 0x5596a4e8bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:21.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.920+0000 7f7c63fff640 1 --2- 172.21.15.103:0/2683874846 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5596a4e89920 0x5596a4e8bd40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7dd731dc7eba0890 server_cookie=930e41b3cdc07c6b in_seq=0 out_seq=0 2024-09-19T06:50:21.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.920+0000 7f7c437fe640 10 client.15849 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:50:21.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.921+0000 7f7c437fe640 1 -- 172.21.15.103:0/2683874846 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596a4d68d70 con 0x5596a4e89920 2024-09-19T06:50:21.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.921+0000 7f7c437fe640 10 client.15849 handle_command_reply: tid=0 2024-09-19T06:50:21.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.921+0000 7f7c6cc23740 1 client.15849 shutdown 2024-09-19T06:50:21.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.921+0000 7f7c6cc23740 1 -- 172.21.15.103:0/2683874846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e0eb80 msgr2=0x5596a4e26210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:21.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.921+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/2683874846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e0eb80 0x5596a4e26210 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c005cc0 tx=0x7f7c5c004350 comp rx=0 tx=0).stop 2024-09-19T06:50:21.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.921+0000 7f7c6cc23740 1 -- 172.21.15.103:0/2683874846 shutdown_connections 2024-09-19T06:50:21.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.921+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/2683874846 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5596a4e89920 0x5596a4e8bd40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.921+0000 7f7c437fe640 0 client.15849 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:50:21.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.921+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/2683874846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5596a4e0eb80 0x5596a4e26210 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.921+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/2683874846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596a4e05350 0x5596a4e25cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.921+0000 7f7c6cc23740 1 --2- 172.21.15.103:0/2683874846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a4e03de0 0x5596a4e17280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.921+0000 7f7c6cc23740 1 -- 172.21.15.103:0/2683874846 >> 172.21.15.103:0/2683874846 conn(0x5596a4dc4310 msgr2=0x5596a4e09ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:21.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.921+0000 7f7c6cc23740 1 -- 172.21.15.103:0/2683874846 shutdown_connections 2024-09-19T06:50:21.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.921+0000 7f7c6cc23740 1 -- 172.21.15.103:0/2683874846 wait complete. 2024-09-19T06:50:21.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.922+0000 7f7c6cc23740 20 client.15849 trim_cache size 0 max 16384 2024-09-19T06:50:21.927 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:50:21.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.922+0000 7f7c6a84a640 1 -- 172.21.15.103:0/886067197 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c4c0a3660 msgr2=0x7f7c4c0a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:21.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.922+0000 7f7c6a84a640 1 --2- 172.21.15.103:0/886067197 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c4c0a3660 0x7f7c4c0a5b20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c030040 tx=0x7f7c5c005c30 comp rx=0 tx=0).stop 2024-09-19T06:50:21.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.922+0000 7f7c6a84a640 1 -- 172.21.15.103:0/886067197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c64103650 msgr2=0x7f7c64100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:21.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.922+0000 7f7c6a84a640 1 --2- 172.21.15.103:0/886067197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c64103650 0x7f7c64100ed0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800c940 tx=0x7f7c5800ce00 comp rx=0 tx=0).stop 2024-09-19T06:50:21.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.922+0000 7f7c6a84a640 1 -- 172.21.15.103:0/886067197 shutdown_connections 2024-09-19T06:50:21.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.922+0000 7f7c6a84a640 1 --2- 172.21.15.103:0/886067197 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c4c0a3660 0x7f7c4c0a5b20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.922+0000 7f7c6a84a640 1 --2- 172.21.15.103:0/886067197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c6410c160 0x7f7c64101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.922+0000 7f7c6a84a640 1 --2- 172.21.15.103:0/886067197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c64103650 0x7f7c64100ed0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.922+0000 7f7c6a84a640 1 --2- 172.21.15.103:0/886067197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c64102d30 0x7f7c64100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:21.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.922+0000 7f7c6a84a640 1 -- 172.21.15.103:0/886067197 >> 172.21.15.103:0/886067197 conn(0x7f7c640fc790 msgr2=0x7f7c640fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:21.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.923+0000 7f7c6a84a640 1 -- 172.21.15.103:0/886067197 shutdown_connections 2024-09-19T06:50:21.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:21.923+0000 7f7c6a84a640 1 -- 172.21.15.103:0/886067197 wait complete. 2024-09-19T06:50:22.311 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:22 smithi103 ceph-mon[31158]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T06:50:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:22 smithi174 ceph-mon[35637]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T06:50:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:22 smithi134 ceph-mon[35356]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T06:50:22.606 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:50:22.869 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:23.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.369+0000 7eff897a2640 1 -- 172.21.15.103:0/4091671973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff84100e20 msgr2=0x7eff84101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:23.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.369+0000 7eff897a2640 1 --2- 172.21.15.103:0/4091671973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff84100e20 0x7eff84101200 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7eff74009900 tx=0x7eff7402a480 comp rx=0 tx=0).stop 2024-09-19T06:50:23.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.369+0000 7eff897a2640 1 -- 172.21.15.103:0/4091671973 shutdown_connections 2024-09-19T06:50:23.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.369+0000 7eff897a2640 1 --2- 172.21.15.103:0/4091671973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff84102ef0 0x7eff8410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.369+0000 7eff897a2640 1 --2- 172.21.15.103:0/4091671973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff84102570 0x7eff841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.369+0000 7eff897a2640 1 --2- 172.21.15.103:0/4091671973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff84100e20 0x7eff84101200 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.369+0000 7eff897a2640 1 -- 172.21.15.103:0/4091671973 >> 172.21.15.103:0/4091671973 conn(0x7eff840fc7d0 msgr2=0x7eff840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:23.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.369+0000 7eff897a2640 1 -- 172.21.15.103:0/4091671973 shutdown_connections 2024-09-19T06:50:23.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.369+0000 7eff897a2640 1 -- 172.21.15.103:0/4091671973 wait complete. 2024-09-19T06:50:23.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.370+0000 7eff897a2640 1 Processor -- start 2024-09-19T06:50:23.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.370+0000 7eff897a2640 1 -- start start 2024-09-19T06:50:23.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.370+0000 7eff897a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff84100e20 0x7eff84107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.370+0000 7eff897a2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff84102570 0x7eff84108020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.370+0000 7eff897a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff84102ef0 0x7eff84108560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.371+0000 7eff897a2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7eff84111020 con 0x7eff84102ef0 2024-09-19T06:50:23.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.371+0000 7eff897a2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff84110ea0 con 0x7eff84102570 2024-09-19T06:50:23.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.371+0000 7eff897a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7eff841111a0 con 0x7eff84100e20 2024-09-19T06:50:23.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.371+0000 7eff827fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff84102570 0x7eff84108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.371+0000 7eff837fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff84102ef0 0x7eff84108560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.371+0000 7eff837fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff84102ef0 0x7eff84108560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32898/0 (socket says 172.21.15.103:32898) 2024-09-19T06:50:23.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.371+0000 7eff837fe640 1 -- 172.21.15.103:0/2450219712 learned_addr learned my addr 172.21.15.103:0/2450219712 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:23.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.371+0000 7eff82ffd640 1 --2- 172.21.15.103:0/2450219712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff84100e20 0x7eff84107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.371+0000 7eff837fe640 1 -- 172.21.15.103:0/2450219712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff84100e20 msgr2=0x7eff84107ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:23.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.371+0000 7eff837fe640 1 --2- 172.21.15.103:0/2450219712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff84100e20 0x7eff84107ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.371+0000 7eff837fe640 1 -- 172.21.15.103:0/2450219712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff84102570 msgr2=0x7eff84108020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:23.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.371+0000 7eff837fe640 1 --2- 172.21.15.103:0/2450219712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff84102570 0x7eff84108020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.371+0000 7eff837fe640 1 -- 172.21.15.103:0/2450219712 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff841ab260 con 0x7eff84102ef0 2024-09-19T06:50:23.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.371+0000 7eff82ffd640 1 --2- 172.21.15.103:0/2450219712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff84100e20 0x7eff84107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:23.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.371+0000 7eff837fe640 1 --2- 172.21.15.103:0/2450219712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff84102ef0 0x7eff84108560 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00b970 tx=0x7eff7c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:23.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.372+0000 7eff6bfff640 1 -- 172.21.15.103:0/2450219712 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c019070 con 0x7eff84102ef0 2024-09-19T06:50:23.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.372+0000 7eff6bfff640 1 -- 172.21.15.103:0/2450219712 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff7c014440 con 0x7eff84102ef0 2024-09-19T06:50:23.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.372+0000 7eff6bfff640 1 -- 172.21.15.103:0/2450219712 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c0075d0 con 0x7eff84102ef0 2024-09-19T06:50:23.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.372+0000 7eff897a2640 1 -- 172.21.15.103:0/2450219712 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff841ab490 con 0x7eff84102ef0 2024-09-19T06:50:23.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.372+0000 7eff897a2640 1 -- 172.21.15.103:0/2450219712 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff841ab980 con 0x7eff84102ef0 2024-09-19T06:50:23.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.375+0000 7eff6bfff640 1 -- 172.21.15.103:0/2450219712 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7eff7c01c070 con 0x7eff84102ef0 2024-09-19T06:50:23.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.376+0000 7eff6bfff640 1 --2- 172.21.15.103:0/2450219712 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff6c0a3560 0x7eff6c0a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.376+0000 7eff8bb7b740 1 Processor -- start 2024-09-19T06:50:23.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.376+0000 7eff8bb7b740 1 -- start start 2024-09-19T06:50:23.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.376+0000 7eff6bfff640 1 -- 172.21.15.103:0/2450219712 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7eff7c0cf550 con 0x7eff84102ef0 2024-09-19T06:50:23.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.376+0000 7eff8bb7b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 0x55a432ad1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.376+0000 7eff82ffd640 1 --2- 172.21.15.103:0/2450219712 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff6c0a3560 0x7eff6c0a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.376+0000 7eff8bb7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 0x55a432ad1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.376+0000 7eff8bb7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432adc370 0x55a432ade760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.376+0000 7eff8bb7b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a432a372f0 con 0x55a432ad2b40 2024-09-19T06:50:23.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.376+0000 7eff8bb7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a4329decf0 con 0x55a432adc370 2024-09-19T06:50:23.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.376+0000 7eff8bb7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a432a332b0 con 0x55a432ad15d0 2024-09-19T06:50:23.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.377+0000 7eff827fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 0x55a432ad1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.377+0000 7eff837fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432adc370 0x55a432ade760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.377+0000 7eff837fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432adc370 0x55a432ade760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:34938/0 (socket says 172.21.15.103:34938) 2024-09-19T06:50:23.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.377+0000 7eff837fe640 1 -- 172.21.15.103:0/632784911 learned_addr learned my addr 172.21.15.103:0/632784911 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:23.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.377+0000 7eff827fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 0x55a432ad1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:43702/0 (socket says 172.21.15.103:43702) 2024-09-19T06:50:23.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.377+0000 7eff837fe640 1 -- 172.21.15.103:0/632784911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 msgr2=0x55a432ad1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:23.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.377+0000 7eff837fe640 1 --2- 172.21.15.103:0/632784911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 0x55a432ad1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.377+0000 7eff837fe640 1 -- 172.21.15.103:0/632784911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 msgr2=0x55a432ad1090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:50:23.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.377+0000 7eff837fe640 1 --2- 172.21.15.103:0/632784911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 0x55a432ad1090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.377+0000 7eff837fe640 1 -- 172.21.15.103:0/632784911 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4329c33a0 con 0x55a432adc370 2024-09-19T06:50:23.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.377+0000 7eff837fe640 1 --2- 172.21.15.103:0/632784911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432adc370 0x55a432ade760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00bfd0 tx=0x7eff7c09be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b5139e1361ae66c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:23.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.377+0000 7eff82ffd640 1 --2- 172.21.15.103:0/2450219712 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff6c0a3560 0x7eff6c0a5a20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff74005ab0 tx=0x7eff74005a20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:23.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.377+0000 7eff697fa640 1 -- 172.21.15.103:0/632784911 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c0a4380 con 0x55a432adc370 2024-09-19T06:50:23.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.377+0000 7eff697fa640 1 -- 172.21.15.103:0/632784911 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff7c0cc040 con 0x55a432adc370 2024-09-19T06:50:23.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.377+0000 7eff697fa640 1 -- 172.21.15.103:0/632784911 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c0caef0 con 0x55a432adc370 2024-09-19T06:50:23.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.378+0000 7eff8bb7b740 1 -- 172.21.15.103:0/632784911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432adc370 msgr2=0x55a432ade760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:23.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.378+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/632784911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432adc370 0x55a432ade760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00bfd0 tx=0x7eff7c09be10 comp rx=0 tx=0).stop 2024-09-19T06:50:23.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.378+0000 7eff8bb7b740 1 -- 172.21.15.103:0/632784911 shutdown_connections 2024-09-19T06:50:23.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.378+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/632784911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432adc370 0x55a432ade760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.378+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/632784911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 0x55a432ad1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.378+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/632784911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 0x55a432ad1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.378+0000 7eff8bb7b740 1 -- 172.21.15.103:0/632784911 >> 172.21.15.103:0/632784911 conn(0x55a432a9a310 msgr2=0x55a432acdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:23.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.378+0000 7eff8bb7b740 1 -- 172.21.15.103:0/632784911 shutdown_connections 2024-09-19T06:50:23.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.378+0000 7eff8bb7b740 1 -- 172.21.15.103:0/632784911 wait complete. 2024-09-19T06:50:23.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.379+0000 7eff8bb7b740 1 -- start start 2024-09-19T06:50:23.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.379+0000 7eff8bb7b740 1 Processor -- start 2024-09-19T06:50:23.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.379+0000 7eff8bb7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 0x55a432adbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.379+0000 7eff8bb7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432ad2b40 0x55a432ad8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.379+0000 7eff8bb7b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432adc370 0x55a432ad9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.379+0000 7eff8bb7b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a432adf070 con 0x55a432adc370 2024-09-19T06:50:23.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.379+0000 7eff8bb7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a4329f2060 con 0x55a432ad2b40 2024-09-19T06:50:23.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.379+0000 7eff8bb7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a432adf1f0 con 0x55a432ad15d0 2024-09-19T06:50:23.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.379+0000 7eff82ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 0x55a432adbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.380+0000 7eff82ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 0x55a432adbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:43714/0 (socket says 172.21.15.103:43714) 2024-09-19T06:50:23.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.380+0000 7eff82ffd640 1 -- 172.21.15.103:0/525385923 learned_addr learned my addr 172.21.15.103:0/525385923 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:23.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.380+0000 7eff8bb7b740 10 client.0 _ll_register_callbacks cb 0x55a4327458c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:23.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.380+0000 7eff837fe640 1 --2- 172.21.15.103:0/525385923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432adc370 0x55a432ad9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.380+0000 7eff837fe640 1 -- 172.21.15.103:0/525385923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 msgr2=0x55a432adbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:23.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.380+0000 7eff827fc640 1 --2- 172.21.15.103:0/525385923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432ad2b40 0x55a432ad8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.380+0000 7eff837fe640 1 --2- 172.21.15.103:0/525385923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 0x55a432adbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.380+0000 7eff837fe640 1 -- 172.21.15.103:0/525385923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432ad2b40 msgr2=0x55a432ad8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:23.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.380+0000 7eff837fe640 1 --2- 172.21.15.103:0/525385923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432ad2b40 0x55a432ad8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.380+0000 7eff837fe640 1 -- 172.21.15.103:0/525385923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4327a83c0 con 0x55a432adc370 2024-09-19T06:50:23.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.380+0000 7eff8bb7b740 1 -- 172.21.15.103:0/525385923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff7c0cc040 con 0x55a432adc370 2024-09-19T06:50:23.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.380+0000 7eff827fc640 1 --2- 172.21.15.103:0/525385923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432ad2b40 0x55a432ad8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:23.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.380+0000 7eff837fe640 1 --2- 172.21.15.103:0/525385923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432adc370 0x55a432ad9570 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00bea0 tx=0x7eff7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4bb00a74f8fe68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:23.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.380+0000 7eff66ff5640 10 client.15858 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:23.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.381+0000 7eff66ff5640 1 -- 172.21.15.103:0/525385923 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c099a10 con 0x55a432adc370 2024-09-19T06:50:23.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.381+0000 7eff66ff5640 1 -- 172.21.15.103:0/525385923 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7eff7c097d80 con 0x55a432adc370 2024-09-19T06:50:23.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.381+0000 7eff66ff5640 1 -- 172.21.15.103:0/525385923 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff7c0a3ef0 con 0x55a432adc370 2024-09-19T06:50:23.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.381+0000 7eff8bb7b740 10 client.15858 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:23.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.381+0000 7eff8bb7b740 1 -- 172.21.15.103:0/525385923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55a432aeaa00 con 0x55a432adc370 2024-09-19T06:50:23.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.381+0000 7eff66ff5640 1 -- 172.21.15.103:0/525385923 <== mon.0 v2:172.21.15.103:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7eff7c07b070 con 0x55a432adc370 2024-09-19T06:50:23.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.381+0000 7eff8bb7b740 10 client.15858 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:23.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.381+0000 7eff8bb7b740 10 client.15858 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:50:23.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.381+0000 7eff8bb7b740 20 client.15858 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:23.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.381+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/525385923 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55a432aebd50 0x55a432aee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.381+0000 7eff8bb7b740 4 client.15858 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:23.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.381+0000 7eff8bb7b740 1 -- 172.21.15.103:0/525385923 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a432a44860 con 0x55a432aebd50 2024-09-19T06:50:23.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.382+0000 7eff82ffd640 1 --2- 172.21.15.103:0/525385923 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55a432aebd50 0x55a432aee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.382+0000 7eff66ff5640 10 client.15858 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:50:23.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.382+0000 7eff82ffd640 1 --2- 172.21.15.103:0/525385923 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55a432aebd50 0x55a432aee140 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0fc8f55865d85a9 server_cookie=6192555315f7cf52 in_seq=0 out_seq=0 2024-09-19T06:50:23.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.383+0000 7eff66ff5640 1 -- 172.21.15.103:0/525385923 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a432a44860 con 0x55a432aebd50 2024-09-19T06:50:23.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.383+0000 7eff66ff5640 10 client.15858 handle_command_reply: tid=0 2024-09-19T06:50:23.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.383+0000 7eff8bb7b740 1 client.15858 shutdown 2024-09-19T06:50:23.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.383+0000 7eff8bb7b740 1 -- 172.21.15.103:0/525385923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432adc370 msgr2=0x55a432ad9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:23.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.383+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/525385923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432adc370 0x55a432ad9570 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00bea0 tx=0x7eff7c004a40 comp rx=0 tx=0).stop 2024-09-19T06:50:23.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.383+0000 7eff8bb7b740 1 -- 172.21.15.103:0/525385923 shutdown_connections 2024-09-19T06:50:23.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.383+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/525385923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432adc370 0x55a432ad9570 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.383+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/525385923 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55a432aebd50 0x55a432aee140 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.383+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/525385923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432ad2b40 0x55a432ad8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.383+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/525385923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 0x55a432adbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.383+0000 7eff8bb7b740 1 -- 172.21.15.103:0/525385923 >> 172.21.15.103:0/525385923 conn(0x55a432a9a310 msgr2=0x55a432ad39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:23.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.383+0000 7eff66ff5640 0 client.15858 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:50:23.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.383+0000 7eff8bb7b740 1 -- 172.21.15.103:0/525385923 shutdown_connections 2024-09-19T06:50:23.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.384+0000 7eff8bb7b740 1 -- 172.21.15.103:0/525385923 wait complete. 2024-09-19T06:50:23.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.384+0000 7eff8bb7b740 20 client.15858 trim_cache size 0 max 16384 2024-09-19T06:50:23.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.396+0000 7eff8bb7b740 1 Processor -- start 2024-09-19T06:50:23.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.396+0000 7eff8bb7b740 1 -- start start 2024-09-19T06:50:23.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.397+0000 7eff8bb7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 0x55a432ad4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.397+0000 7eff8bb7b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 0x55a432ad4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.397+0000 7eff8bb7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432adc370 0x55a432af0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.397+0000 7eff8bb7b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7eff841b4060 con 0x55a432ad2b40 2024-09-19T06:50:23.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.397+0000 7eff8bb7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff841b5ba0 con 0x55a432adc370 2024-09-19T06:50:23.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.397+0000 7eff8bb7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7eff841b4730 con 0x55a432ad15d0 2024-09-19T06:50:23.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.397+0000 7eff827fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 0x55a432ad4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.397+0000 7eff827fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 0x55a432ad4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32920/0 (socket says 172.21.15.103:32920) 2024-09-19T06:50:23.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.397+0000 7eff827fc640 1 -- 172.21.15.103:0/1771034105 learned_addr learned my addr 172.21.15.103:0/1771034105 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:23.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.397+0000 7eff82ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 0x55a432ad4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.397+0000 7eff837fe640 1 --2- 172.21.15.103:0/1771034105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432adc370 0x55a432af0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.397+0000 7eff82ffd640 1 -- 172.21.15.103:0/1771034105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432adc370 msgr2=0x55a432af0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:23.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.397+0000 7eff82ffd640 1 --2- 172.21.15.103:0/1771034105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432adc370 0x55a432af0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.397+0000 7eff82ffd640 1 -- 172.21.15.103:0/1771034105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 msgr2=0x55a432ad4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:23.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.397+0000 7eff82ffd640 1 --2- 172.21.15.103:0/1771034105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 0x55a432ad4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.397+0000 7eff82ffd640 1 -- 172.21.15.103:0/1771034105 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a432acde00 con 0x55a432ad15d0 2024-09-19T06:50:23.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.398+0000 7eff837fe640 1 --2- 172.21.15.103:0/1771034105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432adc370 0x55a432af0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:23.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.398+0000 7eff82ffd640 1 --2- 172.21.15.103:0/1771034105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 0x55a432ad4120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7eff74009900 tx=0x7eff74040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a109f8f0a76b5e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:23.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.398+0000 7eff68ff9640 1 -- 172.21.15.103:0/1771034105 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74031d30 con 0x55a432ad15d0 2024-09-19T06:50:23.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.398+0000 7eff68ff9640 1 -- 172.21.15.103:0/1771034105 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff74057070 con 0x55a432ad15d0 2024-09-19T06:50:23.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.398+0000 7eff68ff9640 1 -- 172.21.15.103:0/1771034105 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff740038e0 con 0x55a432ad15d0 2024-09-19T06:50:23.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.399+0000 7eff8bb7b740 1 -- 172.21.15.103:0/1771034105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 msgr2=0x55a432ad4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:23.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.399+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/1771034105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 0x55a432ad4120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7eff74009900 tx=0x7eff74040000 comp rx=0 tx=0).stop 2024-09-19T06:50:23.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.399+0000 7eff8bb7b740 1 -- 172.21.15.103:0/1771034105 shutdown_connections 2024-09-19T06:50:23.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.399+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/1771034105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432adc370 0x55a432af0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.399+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/1771034105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 0x55a432ad4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.399+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/1771034105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432ad15d0 0x55a432ad4120 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.399+0000 7eff8bb7b740 1 -- 172.21.15.103:0/1771034105 >> 172.21.15.103:0/1771034105 conn(0x55a432a9a310 msgr2=0x55a432adb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:23.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.399+0000 7eff8bb7b740 1 -- 172.21.15.103:0/1771034105 shutdown_connections 2024-09-19T06:50:23.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.399+0000 7eff8bb7b740 1 -- 172.21.15.103:0/1771034105 wait complete. 2024-09-19T06:50:23.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.399+0000 7eff8bb7b740 1 -- start start 2024-09-19T06:50:23.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.400+0000 7eff8bb7b740 1 Processor -- start 2024-09-19T06:50:23.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.400+0000 7eff8bb7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432ad15d0 0x55a432aeeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.400+0000 7eff8bb7b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 0x55a432aeeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.400+0000 7eff8bb7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432adc370 0x55a432aef530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.400+0000 7eff827fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 0x55a432aeeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.400+0000 7eff827fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 0x55a432aeeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32930/0 (socket says 172.21.15.103:32930) 2024-09-19T06:50:23.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.400+0000 7eff8bb7b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7eff841b54d0 con 0x55a432ad2b40 2024-09-19T06:50:23.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.400+0000 7eff8bb7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff841b4e00 con 0x55a432ad15d0 2024-09-19T06:50:23.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.400+0000 7eff8bb7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7eff841b2020 con 0x55a432adc370 2024-09-19T06:50:23.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.400+0000 7eff8bb7b740 10 client.0 _ll_register_callbacks cb 0x7eff780093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:23.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.400+0000 7eff827fc640 1 -- 172.21.15.103:0/3767143670 learned_addr learned my addr 172.21.15.103:0/3767143670 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:23.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.400+0000 7eff827fc640 1 -- 172.21.15.103:0/3767143670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432adc370 msgr2=0x55a432aef530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:23.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.400+0000 7eff837fe640 1 --2- 172.21.15.103:0/3767143670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432adc370 0x55a432aef530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.401+0000 7eff827fc640 1 --2- 172.21.15.103:0/3767143670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432adc370 0x55a432aef530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.401+0000 7eff82ffd640 1 --2- 172.21.15.103:0/3767143670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432ad15d0 0x55a432aeeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.401+0000 7eff827fc640 1 -- 172.21.15.103:0/3767143670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432ad15d0 msgr2=0x55a432aeeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:23.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.401+0000 7eff827fc640 1 --2- 172.21.15.103:0/3767143670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432ad15d0 0x55a432aeeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.401+0000 7eff827fc640 1 -- 172.21.15.103:0/3767143670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a432b55690 con 0x55a432ad2b40 2024-09-19T06:50:23.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.401+0000 7eff837fe640 1 --2- 172.21.15.103:0/3767143670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432adc370 0x55a432aef530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:23.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.401+0000 7eff82ffd640 1 --2- 172.21.15.103:0/3767143670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432ad15d0 0x55a432aeeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:23.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.401+0000 7eff827fc640 1 --2- 172.21.15.103:0/3767143670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 0x55a432aeeff0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7eff84109330 tx=0x7eff78009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a4d0b29a7344f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:23.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.401+0000 7eff66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:23.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.401+0000 7eff66ff5640 1 -- 172.21.15.103:0/3767143670 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78017650 con 0x55a432ad2b40 2024-09-19T06:50:23.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.401+0000 7eff66ff5640 1 -- 172.21.15.103:0/3767143670 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7eff780177f0 con 0x55a432ad2b40 2024-09-19T06:50:23.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.401+0000 7eff8bb7b740 1 -- 172.21.15.103:0/3767143670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff74057070 con 0x55a432ad2b40 2024-09-19T06:50:23.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.401+0000 7eff66ff5640 1 -- 172.21.15.103:0/3767143670 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff780179f0 con 0x55a432ad2b40 2024-09-19T06:50:23.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.401+0000 7eff8bb7b740 10 client.15864 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:23.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.402+0000 7eff8bb7b740 1 -- 172.21.15.103:0/3767143670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a432b568e0 con 0x55a432ad2b40 2024-09-19T06:50:23.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.402+0000 7eff66ff5640 1 -- 172.21.15.103:0/3767143670 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff74057070 con 0x55a432ad2b40 2024-09-19T06:50:23.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.402+0000 7eff66ff5640 1 -- 172.21.15.103:0/3767143670 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7eff78006320 con 0x55a432ad2b40 2024-09-19T06:50:23.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.402+0000 7eff8bb7b740 10 client.15864 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:23.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.402+0000 7eff8bb7b740 10 client.15864 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:50:23.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.402+0000 7eff8bb7b740 20 client.15864 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:23.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.402+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/3767143670 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55a432b570b0 0x55a432b594d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:23.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.402+0000 7eff8bb7b740 4 client.15864 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:23.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.402+0000 7eff8bb7b740 1 -- 172.21.15.103:0/3767143670 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a432a3ed70 con 0x55a432b570b0 2024-09-19T06:50:23.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.403+0000 7eff82ffd640 1 --2- 172.21.15.103:0/3767143670 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55a432b570b0 0x55a432b594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:23.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.403+0000 7eff66ff5640 10 client.15864 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:50:23.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.403+0000 7eff82ffd640 1 --2- 172.21.15.103:0/3767143670 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55a432b570b0 0x55a432b594d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f970bca54b230a57 server_cookie=2f1005bccd921335 in_seq=0 out_seq=0 2024-09-19T06:50:23.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.403+0000 7eff66ff5640 1 -- 172.21.15.103:0/3767143670 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a432a3ed70 con 0x55a432b570b0 2024-09-19T06:50:23.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.403+0000 7eff66ff5640 10 client.15864 handle_command_reply: tid=0 2024-09-19T06:50:23.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.403+0000 7eff8bb7b740 1 client.15864 shutdown 2024-09-19T06:50:23.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.403+0000 7eff8bb7b740 1 -- 172.21.15.103:0/3767143670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 msgr2=0x55a432aeeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:23.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.403+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/3767143670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 0x55a432aeeff0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7eff84109330 tx=0x7eff78009f90 comp rx=0 tx=0).stop 2024-09-19T06:50:23.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.404+0000 7eff8bb7b740 1 -- 172.21.15.103:0/3767143670 shutdown_connections 2024-09-19T06:50:23.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.404+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/3767143670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a432adc370 0x55a432aef530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.404+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/3767143670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a432ad2b40 0x55a432aeeff0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.404+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/3767143670 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55a432b570b0 0x55a432b594d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.404+0000 7eff8bb7b740 1 --2- 172.21.15.103:0/3767143670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a432ad15d0 0x55a432aeeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.404+0000 7eff8bb7b740 1 -- 172.21.15.103:0/3767143670 >> 172.21.15.103:0/3767143670 conn(0x55a432a9a310 msgr2=0x55a432ad7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:23.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.404+0000 7eff66ff5640 0 client.15864 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:50:23.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.404+0000 7eff8bb7b740 1 -- 172.21.15.103:0/3767143670 shutdown_connections 2024-09-19T06:50:23.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.404+0000 7eff8bb7b740 1 -- 172.21.15.103:0/3767143670 wait complete. 2024-09-19T06:50:23.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.404+0000 7eff8bb7b740 20 client.15864 trim_cache size 0 max 16384 2024-09-19T06:50:23.409 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:50:23.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.405+0000 7eff897a2640 1 -- 172.21.15.103:0/2450219712 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff6c0a3560 msgr2=0x7eff6c0a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:23.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.405+0000 7eff897a2640 1 --2- 172.21.15.103:0/2450219712 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff6c0a3560 0x7eff6c0a5a20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff74005ab0 tx=0x7eff74005a20 comp rx=0 tx=0).stop 2024-09-19T06:50:23.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.405+0000 7eff897a2640 1 -- 172.21.15.103:0/2450219712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff84102ef0 msgr2=0x7eff84108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:23.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.405+0000 7eff897a2640 1 --2- 172.21.15.103:0/2450219712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff84102ef0 0x7eff84108560 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00b970 tx=0x7eff7c00bd70 comp rx=0 tx=0).stop 2024-09-19T06:50:23.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.405+0000 7eff897a2640 1 -- 172.21.15.103:0/2450219712 shutdown_connections 2024-09-19T06:50:23.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.405+0000 7eff897a2640 1 --2- 172.21.15.103:0/2450219712 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff6c0a3560 0x7eff6c0a5a20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.405+0000 7eff897a2640 1 --2- 172.21.15.103:0/2450219712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff84102ef0 0x7eff84108560 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.405+0000 7eff897a2640 1 --2- 172.21.15.103:0/2450219712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff84102570 0x7eff84108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.405+0000 7eff897a2640 1 --2- 172.21.15.103:0/2450219712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff84100e20 0x7eff84107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:23.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.405+0000 7eff897a2640 1 -- 172.21.15.103:0/2450219712 >> 172.21.15.103:0/2450219712 conn(0x7eff840fc7d0 msgr2=0x7eff84103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:23.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.405+0000 7eff897a2640 1 -- 172.21.15.103:0/2450219712 shutdown_connections 2024-09-19T06:50:23.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:23.405+0000 7eff897a2640 1 -- 172.21.15.103:0/2450219712 wait complete. 2024-09-19T06:50:24.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:23 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:50:24.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:23 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:50:24.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:23 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:50:24.208 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:50:24.553 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:24.940 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:24 smithi103 ceph-mon[31158]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T06:50:24.940 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:24 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:50:24.940 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:24 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:50:24.941 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:24 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:50:25.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.026+0000 7fd546256640 1 -- 172.21.15.103:0/1564217165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd540102d30 msgr2=0x7fd540103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:25.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.026+0000 7fd546256640 1 --2- 172.21.15.103:0/1564217165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd540102d30 0x7fd540103110 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd530009930 tx=0x7fd53002a420 comp rx=0 tx=0).stop 2024-09-19T06:50:25.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.026+0000 7fd546256640 1 -- 172.21.15.103:0/1564217165 shutdown_connections 2024-09-19T06:50:25.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.026+0000 7fd546256640 1 --2- 172.21.15.103:0/1564217165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5401082a0 0x7fd54010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.026+0000 7fd546256640 1 --2- 172.21.15.103:0/1564217165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5401036e0 0x7fd540107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.026+0000 7fd546256640 1 --2- 172.21.15.103:0/1564217165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd540102d30 0x7fd540103110 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.026+0000 7fd546256640 1 -- 172.21.15.103:0/1564217165 >> 172.21.15.103:0/1564217165 conn(0x7fd5400fc7d0 msgr2=0x7fd5400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:25.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.026+0000 7fd546256640 1 -- 172.21.15.103:0/1564217165 shutdown_connections 2024-09-19T06:50:25.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.026+0000 7fd546256640 1 -- 172.21.15.103:0/1564217165 wait complete. 2024-09-19T06:50:25.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.026+0000 7fd546256640 1 Processor -- start 2024-09-19T06:50:25.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.027+0000 7fd546256640 1 -- start start 2024-09-19T06:50:25.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.027+0000 7fd546256640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd540102d30 0x7fd5401985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.027+0000 7fd546256640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5401036e0 0x7fd540198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.027+0000 7fd546256640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5401082a0 0x7fd54019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.027+0000 7fd546256640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd5401120e0 con 0x7fd540102d30 2024-09-19T06:50:25.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.027+0000 7fd546256640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd540111f60 con 0x7fd5401036e0 2024-09-19T06:50:25.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.027+0000 7fd546256640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd540112260 con 0x7fd5401082a0 2024-09-19T06:50:25.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.027+0000 7fd53f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd540102d30 0x7fd5401985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:25.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.027+0000 7fd53f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd540102d30 0x7fd5401985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32950/0 (socket says 172.21.15.103:32950) 2024-09-19T06:50:25.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.027+0000 7fd53f7fe640 1 -- 172.21.15.103:0/3551055768 learned_addr learned my addr 172.21.15.103:0/3551055768 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:25.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.027+0000 7fd53effd640 1 --2- 172.21.15.103:0/3551055768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5401036e0 0x7fd540198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:25.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.027+0000 7fd53ffff640 1 --2- 172.21.15.103:0/3551055768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5401082a0 0x7fd54019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:25.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.028+0000 7fd53f7fe640 1 -- 172.21.15.103:0/3551055768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5401082a0 msgr2=0x7fd54019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:25.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.028+0000 7fd53f7fe640 1 --2- 172.21.15.103:0/3551055768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5401082a0 0x7fd54019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.028+0000 7fd53f7fe640 1 -- 172.21.15.103:0/3551055768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5401036e0 msgr2=0x7fd540198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:25.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.028+0000 7fd53f7fe640 1 --2- 172.21.15.103:0/3551055768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5401036e0 0x7fd540198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.028+0000 7fd53f7fe640 1 -- 172.21.15.103:0/3551055768 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54019d550 con 0x7fd540102d30 2024-09-19T06:50:25.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.028+0000 7fd53effd640 1 --2- 172.21.15.103:0/3551055768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5401036e0 0x7fd540198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:25.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.028+0000 7fd53f7fe640 1 --2- 172.21.15.103:0/3551055768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd540102d30 0x7fd5401985a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd53002a940 tx=0x7fd530003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:25.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.028+0000 7fd53cff9640 1 -- 172.21.15.103:0/3551055768 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53003f070 con 0x7fd540102d30 2024-09-19T06:50:25.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.028+0000 7fd53cff9640 1 -- 172.21.15.103:0/3551055768 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd530031c90 con 0x7fd540102d30 2024-09-19T06:50:25.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.029+0000 7fd546256640 1 -- 172.21.15.103:0/3551055768 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54019d7e0 con 0x7fd540102d30 2024-09-19T06:50:25.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.029+0000 7fd546256640 1 -- 172.21.15.103:0/3551055768 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5401008c0 con 0x7fd540102d30 2024-09-19T06:50:25.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.029+0000 7fd53cff9640 1 -- 172.21.15.103:0/3551055768 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530035030 con 0x7fd540102d30 2024-09-19T06:50:25.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.032+0000 7fd54862f740 1 Processor -- start 2024-09-19T06:50:25.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.032+0000 7fd54862f740 1 -- start start 2024-09-19T06:50:25.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.032+0000 7fd54862f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400243090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.032+0000 7fd54862f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564002435d0 0x556400243a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.032+0000 7fd54862f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55640024e370 0x556400250760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.032+0000 7fd54862f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5564001a92f0 con 0x556400244b40 2024-09-19T06:50:25.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.032+0000 7fd54862f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556400150cf0 con 0x55640024e370 2024-09-19T06:50:25.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.032+0000 7fd54862f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5564001a52b0 con 0x5564002435d0 2024-09-19T06:50:25.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.032+0000 7fd53ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400243090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:25.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.032+0000 7fd53cff9640 1 -- 172.21.15.103:0/3551055768 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7fd530042070 con 0x7fd540102d30 2024-09-19T06:50:25.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.032+0000 7fd53ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400243090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32956/0 (socket says 172.21.15.103:32956) 2024-09-19T06:50:25.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.032+0000 7fd53effd640 1 --2- 172.21.15.103:0/2990143295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55640024e370 0x556400250760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:25.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.032+0000 7fd53ffff640 1 -- 172.21.15.103:0/2990143295 learned_addr learned my addr 172.21.15.103:0/2990143295 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:25.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.033+0000 7fd53effd640 1 -- 172.21.15.103:0/2990143295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564002435d0 msgr2=0x556400243a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:50:25.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.033+0000 7fd53effd640 1 --2- 172.21.15.103:0/2990143295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564002435d0 0x556400243a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.033+0000 7fd53effd640 1 -- 172.21.15.103:0/2990143295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 msgr2=0x556400243090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:25.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.033+0000 7fd53effd640 1 --2- 172.21.15.103:0/2990143295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400243090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.033+0000 7fd53effd640 1 -- 172.21.15.103:0/2990143295 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564001353a0 con 0x55640024e370 2024-09-19T06:50:25.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.033+0000 7fd53ffff640 1 --2- 172.21.15.103:0/2990143295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400243090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:25.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.033+0000 7fd53effd640 1 --2- 172.21.15.103:0/2990143295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55640024e370 0x556400250760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd5400fed90 tx=0x7fd53400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec5c13e2f0cead33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:25.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.034+0000 7fd525ffb640 1 -- 172.21.15.103:0/2990143295 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534015070 con 0x55640024e370 2024-09-19T06:50:25.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.034+0000 7fd525ffb640 1 -- 172.21.15.103:0/2990143295 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd53401a940 con 0x55640024e370 2024-09-19T06:50:25.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.034+0000 7fd525ffb640 1 -- 172.21.15.103:0/2990143295 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534011a30 con 0x55640024e370 2024-09-19T06:50:25.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.034+0000 7fd53cff9640 1 --2- 172.21.15.103:0/3551055768 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd5280a35e0 0x7fd5280a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.034+0000 7fd53cff9640 1 -- 172.21.15.103:0/3551055768 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd5300f0b10 con 0x7fd540102d30 2024-09-19T06:50:25.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.034+0000 7fd53ffff640 1 --2- 172.21.15.103:0/3551055768 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd5280a35e0 0x7fd5280a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:25.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.034+0000 7fd54862f740 1 -- 172.21.15.103:0/2990143295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55640024e370 msgr2=0x556400250760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:25.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.034+0000 7fd54862f740 1 --2- 172.21.15.103:0/2990143295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55640024e370 0x556400250760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd5400fed90 tx=0x7fd53400f730 comp rx=0 tx=0).stop 2024-09-19T06:50:25.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.035+0000 7fd53ffff640 1 --2- 172.21.15.103:0/3551055768 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd5280a35e0 0x7fd5280a5aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd538009fd0 tx=0x7fd538009760 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:25.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.035+0000 7fd54862f740 1 -- 172.21.15.103:0/2990143295 shutdown_connections 2024-09-19T06:50:25.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.035+0000 7fd54862f740 1 --2- 172.21.15.103:0/2990143295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55640024e370 0x556400250760 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.035+0000 7fd54862f740 1 --2- 172.21.15.103:0/2990143295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564002435d0 0x556400243a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.035+0000 7fd54862f740 1 --2- 172.21.15.103:0/2990143295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400243090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.035+0000 7fd54862f740 1 -- 172.21.15.103:0/2990143295 >> 172.21.15.103:0/2990143295 conn(0x55640020c310 msgr2=0x55640023fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:25.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.035+0000 7fd54862f740 1 -- 172.21.15.103:0/2990143295 shutdown_connections 2024-09-19T06:50:25.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.035+0000 7fd54862f740 1 -- 172.21.15.103:0/2990143295 wait complete. 2024-09-19T06:50:25.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.035+0000 7fd54862f740 1 -- start start 2024-09-19T06:50:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.036+0000 7fd54862f740 1 Processor -- start 2024-09-19T06:50:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.036+0000 7fd54862f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564002435d0 0x55640024b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.036+0000 7fd54862f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556400244b40 0x55640024b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.036+0000 7fd54862f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55640024e370 0x55640024bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.036+0000 7fd54862f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556400251070 con 0x5564002435d0 2024-09-19T06:50:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.036+0000 7fd54862f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556400164060 con 0x556400244b40 2024-09-19T06:50:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.036+0000 7fd54862f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5564002511f0 con 0x55640024e370 2024-09-19T06:50:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.036+0000 7fd53f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564002435d0 0x55640024b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.037+0000 7fd53f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564002435d0 0x55640024b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32962/0 (socket says 172.21.15.103:32962) 2024-09-19T06:50:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.037+0000 7fd53f7fe640 1 -- 172.21.15.103:0/3255874568 learned_addr learned my addr 172.21.15.103:0/3255874568 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.037+0000 7fd54862f740 10 client.0 _ll_register_callbacks cb 0x5563ffeb78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.037+0000 7fd53effd640 1 --2- 172.21.15.103:0/3255874568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556400244b40 0x55640024b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.037+0000 7fd53ffff640 1 --2- 172.21.15.103:0/3255874568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55640024e370 0x55640024bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.037+0000 7fd53f7fe640 1 -- 172.21.15.103:0/3255874568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55640024e370 msgr2=0x55640024bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.037+0000 7fd53f7fe640 1 --2- 172.21.15.103:0/3255874568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55640024e370 0x55640024bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.037+0000 7fd53f7fe640 1 -- 172.21.15.103:0/3255874568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556400244b40 msgr2=0x55640024b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.037+0000 7fd53f7fe640 1 --2- 172.21.15.103:0/3255874568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556400244b40 0x55640024b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.037+0000 7fd53f7fe640 1 -- 172.21.15.103:0/3255874568 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563fff1a3c0 con 0x5564002435d0 2024-09-19T06:50:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.037+0000 7fd53f7fe640 1 --2- 172.21.15.103:0/3255874568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564002435d0 0x55640024b0f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd5300b9020 tx=0x7fd530009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b97d6cf2d0d6b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.037+0000 7fd51effd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.038+0000 7fd51effd640 1 -- 172.21.15.103:0/3255874568 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530049070 con 0x5564002435d0 2024-09-19T06:50:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.038+0000 7fd51effd640 1 -- 172.21.15.103:0/3255874568 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd5300bce20 con 0x5564002435d0 2024-09-19T06:50:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.038+0000 7fd54862f740 1 -- 172.21.15.103:0/3255874568 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd53401a940 con 0x5564002435d0 2024-09-19T06:50:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.038+0000 7fd51effd640 1 -- 172.21.15.103:0/3255874568 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd5300da050 con 0x5564002435d0 2024-09-19T06:50:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.038+0000 7fd54862f740 10 client.15876 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.038+0000 7fd54862f740 1 -- 172.21.15.103:0/3255874568 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55640025cc00 con 0x5564002435d0 2024-09-19T06:50:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.038+0000 7fd51effd640 1 -- 172.21.15.103:0/3255874568 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd53401a940 con 0x5564002435d0 2024-09-19T06:50:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.038+0000 7fd51effd640 1 -- 172.21.15.103:0/3255874568 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd5300c53a0 con 0x5564002435d0 2024-09-19T06:50:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.039+0000 7fd54862f740 10 client.15876 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.039+0000 7fd54862f740 10 client.15876 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:50:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.039+0000 7fd54862f740 20 client.15876 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.039+0000 7fd54862f740 1 --2- 172.21.15.103:0/3255874568 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55640025e240 0x556400260690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.039+0000 7fd54862f740 4 client.15876 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.039+0000 7fd54862f740 1 -- 172.21.15.103:0/3255874568 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564001b6860 con 0x55640025e240 2024-09-19T06:50:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.039+0000 7fd53effd640 1 --2- 172.21.15.103:0/3255874568 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55640025e240 0x556400260690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.039+0000 7fd51effd640 10 client.15876 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:50:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.039+0000 7fd53effd640 1 --2- 172.21.15.103:0/3255874568 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55640025e240 0x556400260690 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d538cfce34c62865 server_cookie=e527fa8eb2c915e2 in_seq=0 out_seq=0 2024-09-19T06:50:25.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.040+0000 7fd51effd640 1 -- 172.21.15.103:0/3255874568 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564001b6860 con 0x55640025e240 2024-09-19T06:50:25.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.040+0000 7fd51effd640 10 client.15876 handle_command_reply: tid=0 2024-09-19T06:50:25.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.040+0000 7fd54862f740 1 client.15876 shutdown 2024-09-19T06:50:25.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.040+0000 7fd54862f740 1 -- 172.21.15.103:0/3255874568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564002435d0 msgr2=0x55640024b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:25.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.040+0000 7fd54862f740 1 --2- 172.21.15.103:0/3255874568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564002435d0 0x55640024b0f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd5300b9020 tx=0x7fd530009620 comp rx=0 tx=0).stop 2024-09-19T06:50:25.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.040+0000 7fd54862f740 1 -- 172.21.15.103:0/3255874568 shutdown_connections 2024-09-19T06:50:25.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.040+0000 7fd54862f740 1 --2- 172.21.15.103:0/3255874568 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55640025e240 0x556400260690 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.041+0000 7fd54862f740 1 --2- 172.21.15.103:0/3255874568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55640024e370 0x55640024bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.041+0000 7fd51effd640 0 client.15876 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:50:25.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.041+0000 7fd54862f740 1 --2- 172.21.15.103:0/3255874568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556400244b40 0x55640024b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.041+0000 7fd54862f740 1 --2- 172.21.15.103:0/3255874568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564002435d0 0x55640024b0f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.041+0000 7fd54862f740 1 -- 172.21.15.103:0/3255874568 >> 172.21.15.103:0/3255874568 conn(0x55640020c310 msgr2=0x556400245960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:25.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.041+0000 7fd54862f740 1 -- 172.21.15.103:0/3255874568 shutdown_connections 2024-09-19T06:50:25.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.041+0000 7fd54862f740 1 -- 172.21.15.103:0/3255874568 wait complete. 2024-09-19T06:50:25.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.041+0000 7fd54862f740 20 client.15876 trim_cache size 0 max 16384 2024-09-19T06:50:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.054+0000 7fd54862f740 1 Processor -- start 2024-09-19T06:50:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.054+0000 7fd54862f740 1 -- start start 2024-09-19T06:50:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.054+0000 7fd54862f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564002435d0 0x556400265f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.054+0000 7fd54862f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400266440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.054+0000 7fd54862f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55640024e370 0x556400245110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.054+0000 7fd54862f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd5401af920 con 0x556400244b40 2024-09-19T06:50:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.054+0000 7fd54862f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd5401b1460 con 0x5564002435d0 2024-09-19T06:50:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.054+0000 7fd54862f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd5401afff0 con 0x55640024e370 2024-09-19T06:50:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.054+0000 7fd53effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400266440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.054+0000 7fd53effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400266440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32964/0 (socket says 172.21.15.103:32964) 2024-09-19T06:50:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.054+0000 7fd53effd640 1 -- 172.21.15.103:0/3154747922 learned_addr learned my addr 172.21.15.103:0/3154747922 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.055+0000 7fd53effd640 1 -- 172.21.15.103:0/3154747922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55640024e370 msgr2=0x556400245110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.055+0000 7fd53effd640 1 --2- 172.21.15.103:0/3154747922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55640024e370 0x556400245110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.055+0000 7fd53effd640 1 -- 172.21.15.103:0/3154747922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564002435d0 msgr2=0x556400265f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:25.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.055+0000 7fd53effd640 1 --2- 172.21.15.103:0/3154747922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564002435d0 0x556400265f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.055+0000 7fd53effd640 1 -- 172.21.15.103:0/3154747922 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556400249640 con 0x556400244b40 2024-09-19T06:50:25.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.055+0000 7fd53effd640 1 --2- 172.21.15.103:0/3154747922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400266440 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd5400fed90 tx=0x7fd53400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250150cd5ebaeb83 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:25.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.055+0000 7fd5257fa640 1 -- 172.21.15.103:0/3154747922 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534015040 con 0x556400244b40 2024-09-19T06:50:25.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.055+0000 7fd5257fa640 1 -- 172.21.15.103:0/3154747922 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd534005520 con 0x556400244b40 2024-09-19T06:50:25.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.055+0000 7fd5257fa640 1 -- 172.21.15.103:0/3154747922 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534004b40 con 0x556400244b40 2024-09-19T06:50:25.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.056+0000 7fd54862f740 1 -- 172.21.15.103:0/3154747922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 msgr2=0x556400266440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:25.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.056+0000 7fd54862f740 1 --2- 172.21.15.103:0/3154747922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400266440 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd5400fed90 tx=0x7fd53400b7c0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.056+0000 7fd54862f740 1 -- 172.21.15.103:0/3154747922 shutdown_connections 2024-09-19T06:50:25.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.056+0000 7fd54862f740 1 --2- 172.21.15.103:0/3154747922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55640024e370 0x556400245110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.056+0000 7fd54862f740 1 --2- 172.21.15.103:0/3154747922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400266440 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.056+0000 7fd54862f740 1 --2- 172.21.15.103:0/3154747922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5564002435d0 0x556400265f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.056+0000 7fd54862f740 1 -- 172.21.15.103:0/3154747922 >> 172.21.15.103:0/3154747922 conn(0x55640020c310 msgr2=0x55640024a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:25.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.056+0000 7fd54862f740 1 -- 172.21.15.103:0/3154747922 shutdown_connections 2024-09-19T06:50:25.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.056+0000 7fd54862f740 1 -- 172.21.15.103:0/3154747922 wait complete. 2024-09-19T06:50:25.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.056+0000 7fd54862f740 1 -- start start 2024-09-19T06:50:25.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.057+0000 7fd54862f740 1 Processor -- start 2024-09-19T06:50:25.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.057+0000 7fd54862f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564002435d0 0x5564002569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.057+0000 7fd54862f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400251db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.057+0000 7fd53effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400251db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:25.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.057+0000 7fd53effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400251db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32966/0 (socket says 172.21.15.103:32966) 2024-09-19T06:50:25.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.057+0000 7fd53effd640 1 -- 172.21.15.103:0/3774598299 learned_addr learned my addr 172.21.15.103:0/3774598299 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:25.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.057+0000 7fd54862f740 1 --2- 172.21.15.103:0/3774598299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55640024e370 0x5564002522f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.057+0000 7fd53f7fe640 1 --2- 172.21.15.103:0/3774598299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564002435d0 0x5564002569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:25.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.057+0000 7fd54862f740 1 -- 172.21.15.103:0/3774598299 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd5401b0d90 con 0x556400244b40 2024-09-19T06:50:25.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.058+0000 7fd54862f740 1 -- 172.21.15.103:0/3774598299 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd5401b06c0 con 0x55640024e370 2024-09-19T06:50:25.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.058+0000 7fd53ffff640 1 --2- 172.21.15.103:0/3774598299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55640024e370 0x5564002522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:25.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.058+0000 7fd54862f740 1 -- 172.21.15.103:0/3774598299 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd5401ad8a0 con 0x5564002435d0 2024-09-19T06:50:25.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.058+0000 7fd54862f740 10 client.0 _ll_register_callbacks cb 0x7fd530009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:25.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.058+0000 7fd53ffff640 1 -- 172.21.15.103:0/3774598299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564002435d0 msgr2=0x5564002569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:25.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.058+0000 7fd53ffff640 1 --2- 172.21.15.103:0/3774598299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564002435d0 0x5564002569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.058+0000 7fd53ffff640 1 -- 172.21.15.103:0/3774598299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 msgr2=0x556400251db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:25.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.058+0000 7fd53ffff640 1 --2- 172.21.15.103:0/3774598299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400251db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.058+0000 7fd53ffff640 1 -- 172.21.15.103:0/3774598299 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556400252930 con 0x55640024e370 2024-09-19T06:50:25.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.058+0000 7fd53effd640 1 --2- 172.21.15.103:0/3774598299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400251db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:25.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.058+0000 7fd53ffff640 1 --2- 172.21.15.103:0/3774598299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55640024e370 0x5564002522f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd538005fd0 tx=0x7fd538004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b66d3ef422cdd6d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:25.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.058+0000 7fd51effd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:50:25.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.059+0000 7fd51effd640 1 -- 172.21.15.103:0/3774598299 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538020d10 con 0x55640024e370 2024-09-19T06:50:25.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.059+0000 7fd51effd640 1 -- 172.21.15.103:0/3774598299 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd538029440 con 0x55640024e370 2024-09-19T06:50:25.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.059+0000 7fd54862f740 1 -- 172.21.15.103:0/3774598299 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd534005520 con 0x55640024e370 2024-09-19T06:50:25.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.059+0000 7fd51effd640 1 -- 172.21.15.103:0/3774598299 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd538029640 con 0x55640024e370 2024-09-19T06:50:25.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.059+0000 7fd54862f740 10 client.25295 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:25.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.059+0000 7fd54862f740 1 -- 172.21.15.103:0/3774598299 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5564002d1530 con 0x55640024e370 2024-09-19T06:50:25.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.059+0000 7fd51effd640 1 -- 172.21.15.103:0/3774598299 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd534005520 con 0x55640024e370 2024-09-19T06:50:25.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.059+0000 7fd51effd640 1 -- 172.21.15.103:0/3774598299 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd538027020 con 0x55640024e370 2024-09-19T06:50:25.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.059+0000 7fd54862f740 10 client.25295 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:25.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.059+0000 7fd54862f740 10 client.25295 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:50:25.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.060+0000 7fd54862f740 20 client.25295 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:25.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.060+0000 7fd54862f740 1 --2- 172.21.15.103:0/3774598299 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55640025e240 0x5564002d3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:25.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.060+0000 7fd54862f740 4 client.25295 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:25.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.060+0000 7fd54862f740 1 -- 172.21.15.103:0/3774598299 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564001b0d70 con 0x55640025e240 2024-09-19T06:50:25.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.060+0000 7fd53f7fe640 1 --2- 172.21.15.103:0/3774598299 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55640025e240 0x5564002d3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:25.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd53f7fe640 1 --2- 172.21.15.103:0/3774598299 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55640025e240 0x5564002d3d20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a3a3e9462dd7793 server_cookie=a831a7f98751f456 in_seq=0 out_seq=0 2024-09-19T06:50:25.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd51effd640 10 client.25295 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:50:25.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd51effd640 1 -- 172.21.15.103:0/3774598299 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564001b0d70 con 0x55640025e240 2024-09-19T06:50:25.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd51effd640 10 client.25295 handle_command_reply: tid=0 2024-09-19T06:50:25.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd54862f740 1 client.25295 shutdown 2024-09-19T06:50:25.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd54862f740 1 -- 172.21.15.103:0/3774598299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55640024e370 msgr2=0x5564002522f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:25.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd54862f740 1 --2- 172.21.15.103:0/3774598299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55640024e370 0x5564002522f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd538005fd0 tx=0x7fd538004ab0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd54862f740 1 -- 172.21.15.103:0/3774598299 shutdown_connections 2024-09-19T06:50:25.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd54862f740 1 --2- 172.21.15.103:0/3774598299 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55640025e240 0x5564002d3d20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd54862f740 1 --2- 172.21.15.103:0/3774598299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55640024e370 0x5564002522f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd51effd640 0 client.25295 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:50:25.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd54862f740 1 --2- 172.21.15.103:0/3774598299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556400244b40 0x556400251db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd54862f740 1 --2- 172.21.15.103:0/3774598299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564002435d0 0x5564002569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd54862f740 1 -- 172.21.15.103:0/3774598299 >> 172.21.15.103:0/3774598299 conn(0x55640020c310 msgr2=0x556400245110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:25.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd54862f740 1 -- 172.21.15.103:0/3774598299 shutdown_connections 2024-09-19T06:50:25.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd54862f740 1 -- 172.21.15.103:0/3774598299 wait complete. 2024-09-19T06:50:25.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.061+0000 7fd54862f740 20 client.25295 trim_cache size 0 max 16384 2024-09-19T06:50:25.068 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:50:25.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.062+0000 7fd546256640 1 -- 172.21.15.103:0/3551055768 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd5280a35e0 msgr2=0x7fd5280a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:25.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.062+0000 7fd546256640 1 --2- 172.21.15.103:0/3551055768 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd5280a35e0 0x7fd5280a5aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd538009fd0 tx=0x7fd538009760 comp rx=0 tx=0).stop 2024-09-19T06:50:25.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.062+0000 7fd546256640 1 -- 172.21.15.103:0/3551055768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd540102d30 msgr2=0x7fd5401985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:25.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.062+0000 7fd546256640 1 --2- 172.21.15.103:0/3551055768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd540102d30 0x7fd5401985a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd53002a940 tx=0x7fd530003b90 comp rx=0 tx=0).stop 2024-09-19T06:50:25.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.062+0000 7fd546256640 1 -- 172.21.15.103:0/3551055768 shutdown_connections 2024-09-19T06:50:25.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.062+0000 7fd546256640 1 --2- 172.21.15.103:0/3551055768 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd5280a35e0 0x7fd5280a5aa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.062+0000 7fd546256640 1 --2- 172.21.15.103:0/3551055768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5401082a0 0x7fd54019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.062+0000 7fd546256640 1 --2- 172.21.15.103:0/3551055768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5401036e0 0x7fd540198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.062+0000 7fd546256640 1 --2- 172.21.15.103:0/3551055768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd540102d30 0x7fd5401985a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:25.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.062+0000 7fd546256640 1 -- 172.21.15.103:0/3551055768 >> 172.21.15.103:0/3551055768 conn(0x7fd5400fc7d0 msgr2=0x7fd540106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:25.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.063+0000 7fd546256640 1 -- 172.21.15.103:0/3551055768 shutdown_connections 2024-09-19T06:50:25.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:25.063+0000 7fd546256640 1 -- 172.21.15.103:0/3551055768 wait complete. 2024-09-19T06:50:25.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:24 smithi174 ceph-mon[35637]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T06:50:25.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:24 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:50:25.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:24 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:50:25.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:24 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:50:25.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:24 smithi134 ceph-mon[35356]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-19T06:50:25.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:24 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:50:25.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:24 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:50:25.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:24 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:50:25.733 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:50:26.048 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:26.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:26 smithi174 ceph-mon[35637]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-19T06:50:26.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:26 smithi103 ceph-mon[31158]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-19T06:50:26.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:26 smithi134 ceph-mon[35356]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-19T06:50:26.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.548+0000 7f0c4c8a5640 1 -- 172.21.15.103:0/1529411245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c48102d90 msgr2=0x7f0c48103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:26.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.548+0000 7f0c4c8a5640 1 --2- 172.21.15.103:0/1529411245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c48102d90 0x7f0c48103170 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009930 tx=0x7f0c3802a420 comp rx=0 tx=0).stop 2024-09-19T06:50:26.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.548+0000 7f0c4c8a5640 1 -- 172.21.15.103:0/1529411245 shutdown_connections 2024-09-19T06:50:26.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.548+0000 7f0c4c8a5640 1 --2- 172.21.15.103:0/1529411245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c48108290 0x7f0c4810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.548+0000 7f0c4c8a5640 1 --2- 172.21.15.103:0/1529411245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c48103740 0x7f0c48107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.548+0000 7f0c4c8a5640 1 --2- 172.21.15.103:0/1529411245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c48102d90 0x7f0c48103170 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.548+0000 7f0c4c8a5640 1 -- 172.21.15.103:0/1529411245 >> 172.21.15.103:0/1529411245 conn(0x7f0c480fc7d0 msgr2=0x7f0c480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:26.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.548+0000 7f0c4c8a5640 1 -- 172.21.15.103:0/1529411245 shutdown_connections 2024-09-19T06:50:26.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.548+0000 7f0c4c8a5640 1 -- 172.21.15.103:0/1529411245 wait complete. 2024-09-19T06:50:26.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.549+0000 7f0c4c8a5640 1 Processor -- start 2024-09-19T06:50:26.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.549+0000 7f0c4c8a5640 1 -- start start 2024-09-19T06:50:26.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.550+0000 7f0c4c8a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c48102d90 0x7f0c481009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.550+0000 7f0c4c8a5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c48103740 0x7f0c48100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.550+0000 7f0c4c8a5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c48108290 0x7f0c48101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.550+0000 7f0c4c8a5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c48112090 con 0x7f0c48103740 2024-09-19T06:50:26.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.550+0000 7f0c4c8a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0c48111f10 con 0x7f0c48102d90 2024-09-19T06:50:26.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.550+0000 7f0c4c8a5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c48112210 con 0x7f0c48108290 2024-09-19T06:50:26.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.550+0000 7f0c45d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c48103740 0x7f0c48100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.550+0000 7f0c45d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c48103740 0x7f0c48100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32996/0 (socket says 172.21.15.103:32996) 2024-09-19T06:50:26.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.550+0000 7f0c45d9b640 1 -- 172.21.15.103:0/2127895761 learned_addr learned my addr 172.21.15.103:0/2127895761 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:26.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.550+0000 7f0c4659c640 1 --2- 172.21.15.103:0/2127895761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c48102d90 0x7f0c481009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.550+0000 7f0c46d9d640 1 --2- 172.21.15.103:0/2127895761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c48108290 0x7f0c48101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.550+0000 7f0c45d9b640 1 -- 172.21.15.103:0/2127895761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c48108290 msgr2=0x7f0c48101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:26.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.550+0000 7f0c45d9b640 1 --2- 172.21.15.103:0/2127895761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c48108290 0x7f0c48101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.550+0000 7f0c45d9b640 1 -- 172.21.15.103:0/2127895761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c48102d90 msgr2=0x7f0c481009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:26.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.550+0000 7f0c45d9b640 1 --2- 172.21.15.103:0/2127895761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c48102d90 0x7f0c481009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.550+0000 7f0c45d9b640 1 -- 172.21.15.103:0/2127895761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c481a6a10 con 0x7f0c48103740 2024-09-19T06:50:26.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.551+0000 7f0c45d9b640 1 --2- 172.21.15.103:0/2127895761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c48103740 0x7f0c48100f30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00c970 tx=0x7f0c3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:26.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.551+0000 7f0c377fe640 1 -- 172.21.15.103:0/2127895761 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c014070 con 0x7f0c48103740 2024-09-19T06:50:26.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.551+0000 7f0c377fe640 1 -- 172.21.15.103:0/2127895761 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c3c004440 con 0x7f0c48103740 2024-09-19T06:50:26.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.551+0000 7f0c4c8a5640 1 -- 172.21.15.103:0/2127895761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c481a6d00 con 0x7f0c48103740 2024-09-19T06:50:26.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.551+0000 7f0c377fe640 1 -- 172.21.15.103:0/2127895761 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c00f450 con 0x7f0c48103740 2024-09-19T06:50:26.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.551+0000 7f0c4c8a5640 1 -- 172.21.15.103:0/2127895761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c481a7230 con 0x7f0c48103740 2024-09-19T06:50:26.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.554+0000 7f0c4ec7e740 1 Processor -- start 2024-09-19T06:50:26.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.554+0000 7f0c4ec7e740 1 -- start start 2024-09-19T06:50:26.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.554+0000 7f0c4ec7e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2576b40 0x5650d2575090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.554+0000 7f0c4ec7e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d2575a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.554+0000 7f0c4ec7e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d2588b80 0x5650d258af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.554+0000 7f0c4ec7e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5650d24db2f0 con 0x5650d2588b80 2024-09-19T06:50:26.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.554+0000 7f0c4ec7e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5650d2482cf0 con 0x5650d25755d0 2024-09-19T06:50:26.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.554+0000 7f0c4ec7e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5650d24d72b0 con 0x5650d2576b40 2024-09-19T06:50:26.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.554+0000 7f0c46d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2576b40 0x5650d2575090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.555+0000 7f0c4659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d2575a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.555+0000 7f0c46d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2576b40 0x5650d2575090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:43810/0 (socket says 172.21.15.103:43810) 2024-09-19T06:50:26.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.555+0000 7f0c46d9d640 1 -- 172.21.15.103:0/2142363329 learned_addr learned my addr 172.21.15.103:0/2142363329 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:26.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.555+0000 7f0c4659c640 1 -- 172.21.15.103:0/2142363329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2576b40 msgr2=0x5650d2575090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:26.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.555+0000 7f0c4659c640 1 --2- 172.21.15.103:0/2142363329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2576b40 0x5650d2575090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.555+0000 7f0c4659c640 1 -- 172.21.15.103:0/2142363329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d2588b80 msgr2=0x5650d258af70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:50:26.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.555+0000 7f0c4659c640 1 --2- 172.21.15.103:0/2142363329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d2588b80 0x5650d258af70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.555+0000 7f0c4659c640 1 -- 172.21.15.103:0/2142363329 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650d24673a0 con 0x5650d25755d0 2024-09-19T06:50:26.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.555+0000 7f0c46d9d640 1 --2- 172.21.15.103:0/2142363329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2576b40 0x5650d2575090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:26.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.555+0000 7f0c4659c640 1 --2- 172.21.15.103:0/2142363329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d2575a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0c38005940 tx=0x7f0c380316e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79c2da5fe81404e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:26.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.555+0000 7f0c34ff9640 1 -- 172.21.15.103:0/2142363329 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38040070 con 0x5650d25755d0 2024-09-19T06:50:26.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.555+0000 7f0c34ff9640 1 -- 172.21.15.103:0/2142363329 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c38005020 con 0x5650d25755d0 2024-09-19T06:50:26.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.555+0000 7f0c34ff9640 1 -- 172.21.15.103:0/2142363329 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38035040 con 0x5650d25755d0 2024-09-19T06:50:26.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.556+0000 7f0c377fe640 1 -- 172.21.15.103:0/2127895761 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7f0c3c017070 con 0x7f0c48103740 2024-09-19T06:50:26.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.556+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2142363329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 msgr2=0x5650d2575a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:26.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.556+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/2142363329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d2575a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0c38005940 tx=0x7f0c380316e0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.556+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2142363329 shutdown_connections 2024-09-19T06:50:26.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.556+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/2142363329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d2588b80 0x5650d258af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.556+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/2142363329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d2575a10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.556+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/2142363329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2576b40 0x5650d2575090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.556+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2142363329 >> 172.21.15.103:0/2142363329 conn(0x5650d253e310 msgr2=0x5650d2571c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:26.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.556+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2142363329 shutdown_connections 2024-09-19T06:50:26.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.556+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2142363329 wait complete. 2024-09-19T06:50:26.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.557+0000 7f0c4ec7e740 1 -- start start 2024-09-19T06:50:26.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.557+0000 7f0c4ec7e740 1 Processor -- start 2024-09-19T06:50:26.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.557+0000 7f0c4ec7e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d257dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.557+0000 7f0c4ec7e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2576b40 0x5650d257e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.557+0000 7f0c4ec7e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d2588b80 0x5650d2578fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.557+0000 7f0c4ec7e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5650d258b820 con 0x5650d2588b80 2024-09-19T06:50:26.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.557+0000 7f0c4ec7e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5650d2496060 con 0x5650d25755d0 2024-09-19T06:50:26.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.557+0000 7f0c4ec7e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5650d258b9a0 con 0x5650d2576b40 2024-09-19T06:50:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.557+0000 7f0c46d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d257dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.557+0000 7f0c46d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d257dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:35032/0 (socket says 172.21.15.103:35032) 2024-09-19T06:50:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.557+0000 7f0c46d9d640 1 -- 172.21.15.103:0/4057443715 learned_addr learned my addr 172.21.15.103:0/4057443715 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.557+0000 7f0c4ec7e740 10 client.0 _ll_register_callbacks cb 0x5650d21e98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.558+0000 7f0c45d9b640 1 --2- 172.21.15.103:0/4057443715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d2588b80 0x5650d2578fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.558+0000 7f0c46d9d640 1 -- 172.21.15.103:0/4057443715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2576b40 msgr2=0x5650d257e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.558+0000 7f0c4659c640 1 --2- 172.21.15.103:0/4057443715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2576b40 0x5650d257e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.558+0000 7f0c46d9d640 1 --2- 172.21.15.103:0/4057443715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2576b40 0x5650d257e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.558+0000 7f0c46d9d640 1 -- 172.21.15.103:0/4057443715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d2588b80 msgr2=0x5650d2578fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.558+0000 7f0c46d9d640 1 --2- 172.21.15.103:0/4057443715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d2588b80 0x5650d2578fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.558+0000 7f0c46d9d640 1 -- 172.21.15.103:0/4057443715 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650d224c3c0 con 0x5650d25755d0 2024-09-19T06:50:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.558+0000 7f0c45d9b640 1 --2- 172.21.15.103:0/4057443715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d2588b80 0x5650d2578fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.558+0000 7f0c46d9d640 1 --2- 172.21.15.103:0/4057443715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d257dc70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0c40010b80 tx=0x7f0c4000f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af3b4c3aff28fafd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.558+0000 7f0c4659c640 1 --2- 172.21.15.103:0/4057443715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2576b40 0x5650d257e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.558+0000 7f0c2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:50:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.558+0000 7f0c2dffb640 1 -- 172.21.15.103:0/4057443715 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c40003b80 con 0x5650d25755d0 2024-09-19T06:50:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.558+0000 7f0c2dffb640 1 -- 172.21.15.103:0/4057443715 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f0c40003d20 con 0x5650d25755d0 2024-09-19T06:50:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.558+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/4057443715 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c38005020 con 0x5650d25755d0 2024-09-19T06:50:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.559+0000 7f0c2dffb640 1 -- 172.21.15.103:0/4057443715 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0c40004030 con 0x5650d25755d0 2024-09-19T06:50:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.559+0000 7f0c4ec7e740 10 client.25304 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.559+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/4057443715 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5650d2584880 con 0x5650d25755d0 2024-09-19T06:50:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.559+0000 7f0c2dffb640 1 -- 172.21.15.103:0/4057443715 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c38005020 con 0x5650d25755d0 2024-09-19T06:50:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.559+0000 7f0c2dffb640 1 -- 172.21.15.103:0/4057443715 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0c40023a10 con 0x5650d25755d0 2024-09-19T06:50:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.559+0000 7f0c4ec7e740 10 client.25304 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.559+0000 7f0c377fe640 1 --2- 172.21.15.103:0/2127895761 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0c300a3570 0x7f0c300a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.559+0000 7f0c4ec7e740 10 client.25304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.559+0000 7f0c4ec7e740 20 client.25304 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.559+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/4057443715 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5650d25a13e0 0x5650d25a3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.559+0000 7f0c4ec7e740 4 client.25304 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.559+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/4057443715 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650d24e8860 con 0x5650d25a13e0 2024-09-19T06:50:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.560+0000 7f0c45d9b640 1 --2- 172.21.15.103:0/4057443715 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5650d25a13e0 0x5650d25a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.560+0000 7f0c4659c640 1 --2- 172.21.15.103:0/2127895761 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0c300a3570 0x7f0c300a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.560+0000 7f0c377fe640 1 -- 172.21.15.103:0/2127895761 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f0c3c0cf9e0 con 0x7f0c48103740 2024-09-19T06:50:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.560+0000 7f0c2dffb640 10 client.25304 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.560+0000 7f0c45d9b640 1 --2- 172.21.15.103:0/4057443715 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5650d25a13e0 0x5650d25a3840 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b44442a009d5e859 server_cookie=3cfaa827648193c4 in_seq=0 out_seq=0 2024-09-19T06:50:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.561+0000 7f0c4659c640 1 --2- 172.21.15.103:0/2127895761 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0c300a3570 0x7f0c300a5a30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c38006010 tx=0x7f0c3802a420 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.561+0000 7f0c2dffb640 1 -- 172.21.15.103:0/4057443715 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650d24e8860 con 0x5650d25a13e0 2024-09-19T06:50:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.561+0000 7f0c2dffb640 10 client.25304 handle_command_reply: tid=0 2024-09-19T06:50:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.561+0000 7f0c4ec7e740 1 client.25304 shutdown 2024-09-19T06:50:26.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.562+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/4057443715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 msgr2=0x5650d257dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:26.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.562+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/4057443715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d257dc70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0c40010b80 tx=0x7f0c4000f7f0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.564+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/4057443715 shutdown_connections 2024-09-19T06:50:26.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.564+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/4057443715 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5650d25a13e0 0x5650d25a3840 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.564+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/4057443715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d2588b80 0x5650d2578fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.564+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/4057443715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2576b40 0x5650d257e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.564+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/4057443715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d257dc70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.564+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/4057443715 >> 172.21.15.103:0/4057443715 conn(0x5650d253e310 msgr2=0x5650d2577930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:26.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.564+0000 7f0c2dffb640 0 client.25304 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:26.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.564+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/4057443715 shutdown_connections 2024-09-19T06:50:26.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.564+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/4057443715 wait complete. 2024-09-19T06:50:26.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.564+0000 7f0c4ec7e740 20 client.25304 trim_cache size 0 max 16384 2024-09-19T06:50:26.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.577+0000 7f0c4ec7e740 1 Processor -- start 2024-09-19T06:50:26.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.577+0000 7f0c4ec7e740 1 -- start start 2024-09-19T06:50:26.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.577+0000 7f0c4ec7e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d2571e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.577+0000 7f0c4ec7e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d2576b40 0x5650d25a9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.577+0000 7f0c4ec7e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2588b80 0x5650d25a9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.578+0000 7f0c4ec7e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481ab770 con 0x5650d2576b40 2024-09-19T06:50:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.578+0000 7f0c4ec7e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481ad2b0 con 0x5650d25755d0 2024-09-19T06:50:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.578+0000 7f0c4ec7e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481abe40 con 0x5650d2588b80 2024-09-19T06:50:26.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.578+0000 7f0c45d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d2576b40 0x5650d25a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.578+0000 7f0c4659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d2571e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.578+0000 7f0c45d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d2576b40 0x5650d25a9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33026/0 (socket says 172.21.15.103:33026) 2024-09-19T06:50:26.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.578+0000 7f0c4659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d2571e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:35042/0 (socket says 172.21.15.103:35042) 2024-09-19T06:50:26.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.578+0000 7f0c45d9b640 1 -- 172.21.15.103:0/2730633518 learned_addr learned my addr 172.21.15.103:0/2730633518 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:26.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.578+0000 7f0c46d9d640 1 --2- 172.21.15.103:0/2730633518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2588b80 0x5650d25a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.578+0000 7f0c4659c640 1 -- 172.21.15.103:0/2730633518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2588b80 msgr2=0x5650d25a9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:26.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.578+0000 7f0c4659c640 1 --2- 172.21.15.103:0/2730633518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2588b80 0x5650d25a9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.578+0000 7f0c4659c640 1 -- 172.21.15.103:0/2730633518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d2576b40 msgr2=0x5650d25a9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:26.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.578+0000 7f0c4659c640 1 --2- 172.21.15.103:0/2730633518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d2576b40 0x5650d25a9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.578+0000 7f0c4659c640 1 -- 172.21.15.103:0/2730633518 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650d2579590 con 0x5650d25755d0 2024-09-19T06:50:26.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.579+0000 7f0c4659c640 1 --2- 172.21.15.103:0/2730633518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d2571e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5650d257a5d0 tx=0x7f0c380316e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d2aa2f79fb917cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:26.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.579+0000 7f0c2ffff640 1 -- 172.21.15.103:0/2730633518 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38040040 con 0x5650d25755d0 2024-09-19T06:50:26.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.579+0000 7f0c2ffff640 1 -- 172.21.15.103:0/2730633518 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c38004480 con 0x5650d25755d0 2024-09-19T06:50:26.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.579+0000 7f0c2ffff640 1 -- 172.21.15.103:0/2730633518 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c380038e0 con 0x5650d25755d0 2024-09-19T06:50:26.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.580+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2730633518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 msgr2=0x5650d2571e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:26.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.580+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/2730633518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d2571e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5650d257a5d0 tx=0x7f0c380316e0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.580+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2730633518 shutdown_connections 2024-09-19T06:50:26.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.580+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/2730633518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2588b80 0x5650d25a9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.580+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/2730633518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d2576b40 0x5650d25a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.580+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/2730633518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d25755d0 0x5650d2571e10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.580+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2730633518 >> 172.21.15.103:0/2730633518 conn(0x5650d253e310 msgr2=0x5650d2571700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:26.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.580+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2730633518 shutdown_connections 2024-09-19T06:50:26.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.580+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2730633518 wait complete. 2024-09-19T06:50:26.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.580+0000 7f0c4ec7e740 1 -- start start 2024-09-19T06:50:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.580+0000 7f0c4ec7e740 1 Processor -- start 2024-09-19T06:50:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c4ec7e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d25755d0 0x5650d25b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c4ec7e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d2576b40 0x5650d25b1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c4ec7e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2588b80 0x5650d25b1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c4ec7e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481acbe0 con 0x5650d25755d0 2024-09-19T06:50:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c4ec7e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481ac510 con 0x5650d2576b40 2024-09-19T06:50:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c4ec7e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481a9750 con 0x5650d2588b80 2024-09-19T06:50:26.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c4ec7e740 10 client.0 _ll_register_callbacks cb 0x7f0c40007ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:26.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c4659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d25755d0 0x5650d25b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c4659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d25755d0 0x5650d25b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33038/0 (socket says 172.21.15.103:33038) 2024-09-19T06:50:26.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c4659c640 1 -- 172.21.15.103:0/2234590299 learned_addr learned my addr 172.21.15.103:0/2234590299 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:26.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c46d9d640 1 --2- 172.21.15.103:0/2234590299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2588b80 0x5650d25b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c45d9b640 1 --2- 172.21.15.103:0/2234590299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d2576b40 0x5650d25b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c46d9d640 1 -- 172.21.15.103:0/2234590299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d2576b40 msgr2=0x5650d25b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:26.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c46d9d640 1 --2- 172.21.15.103:0/2234590299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d2576b40 0x5650d25b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c46d9d640 1 -- 172.21.15.103:0/2234590299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d25755d0 msgr2=0x5650d25b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:26.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c46d9d640 1 --2- 172.21.15.103:0/2234590299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d25755d0 0x5650d25b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c46d9d640 1 -- 172.21.15.103:0/2234590299 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650d260aff0 con 0x5650d2588b80 2024-09-19T06:50:26.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.581+0000 7f0c45d9b640 1 --2- 172.21.15.103:0/2234590299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d2576b40 0x5650d25b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:26.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.582+0000 7f0c46d9d640 1 --2- 172.21.15.103:0/2234590299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2588b80 0x5650d25b1690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x5650d2587fe0 tx=0x7f0c4000fcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a47fe4cf00b6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:26.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.582+0000 7f0c2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T06:50:26.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.582+0000 7f0c2dffb640 1 -- 172.21.15.103:0/2234590299 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c40029030 con 0x5650d2588b80 2024-09-19T06:50:26.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.582+0000 7f0c2dffb640 1 -- 172.21.15.103:0/2234590299 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f0c40005040 con 0x5650d2588b80 2024-09-19T06:50:26.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.582+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2234590299 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c38004480 con 0x5650d2588b80 2024-09-19T06:50:26.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.582+0000 7f0c2dffb640 1 -- 172.21.15.103:0/2234590299 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0c4003b050 con 0x5650d2588b80 2024-09-19T06:50:26.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.582+0000 7f0c4ec7e740 10 client.25273 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:26.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.582+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2234590299 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5650d260c2b0 con 0x5650d2588b80 2024-09-19T06:50:26.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.582+0000 7f0c2dffb640 1 -- 172.21.15.103:0/2234590299 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c38004480 con 0x5650d2588b80 2024-09-19T06:50:26.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.583+0000 7f0c2dffb640 1 -- 172.21.15.103:0/2234590299 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0c40036030 con 0x5650d2588b80 2024-09-19T06:50:26.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.583+0000 7f0c4ec7e740 10 client.25273 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:26.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.583+0000 7f0c4ec7e740 10 client.25273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:26.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.583+0000 7f0c4ec7e740 20 client.25273 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:26.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.583+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/2234590299 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5650d260c9c0 0x5650d260edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:26.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.583+0000 7f0c4ec7e740 4 client.25273 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:26.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.583+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2234590299 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650d24e2d70 con 0x5650d260c9c0 2024-09-19T06:50:26.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.583+0000 7f0c4659c640 1 --2- 172.21.15.103:0/2234590299 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5650d260c9c0 0x5650d260edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:26.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.584+0000 7f0c2dffb640 10 client.25273 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:26.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.584+0000 7f0c4659c640 1 --2- 172.21.15.103:0/2234590299 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5650d260c9c0 0x5650d260edb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d63f7708688403 server_cookie=6c7abf24ab4ca591 in_seq=0 out_seq=0 2024-09-19T06:50:26.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.584+0000 7f0c2dffb640 1 -- 172.21.15.103:0/2234590299 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650d24e2d70 con 0x5650d260c9c0 2024-09-19T06:50:26.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.584+0000 7f0c2dffb640 10 client.25273 handle_command_reply: tid=0 2024-09-19T06:50:26.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.584+0000 7f0c4ec7e740 1 client.25273 shutdown 2024-09-19T06:50:26.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.584+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2234590299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2588b80 msgr2=0x5650d25b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:26.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.584+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/2234590299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2588b80 0x5650d25b1690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x5650d2587fe0 tx=0x7f0c4000fcf0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.584+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2234590299 shutdown_connections 2024-09-19T06:50:26.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.584+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/2234590299 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5650d260c9c0 0x5650d260edb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.584+0000 7f0c2dffb640 0 client.25273 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:26.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.584+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/2234590299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d2588b80 0x5650d25b1690 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.584+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/2234590299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650d2576b40 0x5650d25b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.591 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:50:26.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.584+0000 7f0c4ec7e740 1 --2- 172.21.15.103:0/2234590299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d25755d0 0x5650d25b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.584+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2234590299 >> 172.21.15.103:0/2234590299 conn(0x5650d253e310 msgr2=0x5650d25828e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:26.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.585+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2234590299 shutdown_connections 2024-09-19T06:50:26.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.585+0000 7f0c4ec7e740 1 -- 172.21.15.103:0/2234590299 wait complete. 2024-09-19T06:50:26.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.585+0000 7f0c4ec7e740 20 client.25273 trim_cache size 0 max 16384 2024-09-19T06:50:26.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.586+0000 7f0c4c8a5640 1 -- 172.21.15.103:0/2127895761 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0c300a3570 msgr2=0x7f0c300a5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:26.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.586+0000 7f0c4c8a5640 1 --2- 172.21.15.103:0/2127895761 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0c300a3570 0x7f0c300a5a30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c38006010 tx=0x7f0c3802a420 comp rx=0 tx=0).stop 2024-09-19T06:50:26.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.586+0000 7f0c4c8a5640 1 -- 172.21.15.103:0/2127895761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c48103740 msgr2=0x7f0c48100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.586+0000 7f0c4c8a5640 1 --2- 172.21.15.103:0/2127895761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c48103740 0x7f0c48100f30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00c970 tx=0x7f0c3c00ce30 comp rx=0 tx=0).stop 2024-09-19T06:50:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.586+0000 7f0c4c8a5640 1 -- 172.21.15.103:0/2127895761 shutdown_connections 2024-09-19T06:50:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.586+0000 7f0c4c8a5640 1 --2- 172.21.15.103:0/2127895761 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0c300a3570 0x7f0c300a5a30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.586+0000 7f0c4c8a5640 1 --2- 172.21.15.103:0/2127895761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c48108290 0x7f0c48101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.586+0000 7f0c4c8a5640 1 --2- 172.21.15.103:0/2127895761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c48103740 0x7f0c48100f30 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.586+0000 7f0c4c8a5640 1 --2- 172.21.15.103:0/2127895761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c48102d90 0x7f0c481009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.586+0000 7f0c4c8a5640 1 -- 172.21.15.103:0/2127895761 >> 172.21.15.103:0/2127895761 conn(0x7f0c480fc7d0 msgr2=0x7f0c480fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.586+0000 7f0c4c8a5640 1 -- 172.21.15.103:0/2127895761 shutdown_connections 2024-09-19T06:50:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:26.586+0000 7f0c4c8a5640 1 -- 172.21.15.103:0/2127895761 wait complete. 2024-09-19T06:50:28.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:28 smithi174 ceph-mon[35637]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T06:50:28.367 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:50:28.368 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:50:28.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:28 smithi103 ceph-mon[31158]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T06:50:28.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:28 smithi134 ceph-mon[35356]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T06:50:28.654 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:29.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.157+0000 7f581c188640 1 -- 172.21.15.103:0/27180747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5814106b80 msgr2=0x7f5814106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:29.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.157+0000 7f581c188640 1 --2- 172.21.15.103:0/27180747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5814106b80 0x7f5814106f60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5810009900 tx=0x7f581002a410 comp rx=0 tx=0).stop 2024-09-19T06:50:29.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.157+0000 7f581c188640 1 -- 172.21.15.103:0/27180747 shutdown_connections 2024-09-19T06:50:29.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.157+0000 7f581c188640 1 --2- 172.21.15.103:0/27180747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5814100aa0 0x7f581410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.157+0000 7f581c188640 1 --2- 172.21.15.103:0/27180747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5814100100 0x7f5814100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.157+0000 7f581c188640 1 --2- 172.21.15.103:0/27180747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5814106b80 0x7f5814106f60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.157+0000 7f581c188640 1 -- 172.21.15.103:0/27180747 >> 172.21.15.103:0/27180747 conn(0x7f5814077230 msgr2=0x7f58140fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:29.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.157+0000 7f581c188640 1 -- 172.21.15.103:0/27180747 shutdown_connections 2024-09-19T06:50:29.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.157+0000 7f581c188640 1 -- 172.21.15.103:0/27180747 wait complete. 2024-09-19T06:50:29.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.158+0000 7f581c188640 1 Processor -- start 2024-09-19T06:50:29.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.158+0000 7f581c188640 1 -- start start 2024-09-19T06:50:29.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.158+0000 7f581c188640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5814100100 0x7f58141984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.158+0000 7f581c188640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5814100aa0 0x7f5814198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.158+0000 7f581c188640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5814106b80 0x7f581419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.158+0000 7f581c188640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5814112050 con 0x7f5814106b80 2024-09-19T06:50:29.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.158+0000 7f581c188640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5814111ed0 con 0x7f5814100aa0 2024-09-19T06:50:29.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.158+0000 7f581c188640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f58141121d0 con 0x7f5814100100 2024-09-19T06:50:29.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.159+0000 7f5819f24640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5814100100 0x7f58141984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.159+0000 7f5819f24640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5814100100 0x7f58141984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:54294/0 (socket says 172.21.15.103:54294) 2024-09-19T06:50:29.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.159+0000 7f581a725640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5814106b80 0x7f581419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.159+0000 7f5819f24640 1 -- 172.21.15.103:0/583186766 learned_addr learned my addr 172.21.15.103:0/583186766 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:29.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.159+0000 7f5819723640 1 --2- 172.21.15.103:0/583186766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5814100aa0 0x7f5814198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.159+0000 7f5819723640 1 -- 172.21.15.103:0/583186766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5814100100 msgr2=0x7f58141984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:29.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.159+0000 7f5819723640 1 --2- 172.21.15.103:0/583186766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5814100100 0x7f58141984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.159+0000 7f5819723640 1 -- 172.21.15.103:0/583186766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5814106b80 msgr2=0x7f581419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:29.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.159+0000 7f5819723640 1 --2- 172.21.15.103:0/583186766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5814106b80 0x7f581419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.159+0000 7f5819723640 1 -- 172.21.15.103:0/583186766 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581419d490 con 0x7f5814100aa0 2024-09-19T06:50:29.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.159+0000 7f581a725640 1 --2- 172.21.15.103:0/583186766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5814106b80 0x7f581419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:29.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.160+0000 7f5819723640 1 --2- 172.21.15.103:0/583186766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5814100aa0 0x7f5814198a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f580400c970 tx=0x7f580400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:29.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.160+0000 7f580affd640 1 -- 172.21.15.103:0/583186766 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804007bb0 con 0x7f5814100aa0 2024-09-19T06:50:29.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.160+0000 7f581c188640 1 -- 172.21.15.103:0/583186766 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581419d780 con 0x7f5814100aa0 2024-09-19T06:50:29.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.160+0000 7f580affd640 1 -- 172.21.15.103:0/583186766 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5804007d50 con 0x7f5814100aa0 2024-09-19T06:50:29.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.160+0000 7f580affd640 1 -- 172.21.15.103:0/583186766 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580400f450 con 0x7f5814100aa0 2024-09-19T06:50:29.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.160+0000 7f581c188640 1 -- 172.21.15.103:0/583186766 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58141a5060 con 0x7f5814100aa0 2024-09-19T06:50:29.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.162+0000 7f580affd640 1 -- 172.21.15.103:0/583186766 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7f5804004f30 con 0x7f5814100aa0 2024-09-19T06:50:29.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.162+0000 7f581e561740 1 Processor -- start 2024-09-19T06:50:29.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.163+0000 7f581e561740 1 -- start start 2024-09-19T06:50:29.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.163+0000 7f581e561740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b73cb60 0x55786b73cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.163+0000 7f581e561740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73d480 0x55786b747800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.163+0000 7f581e561740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b747d40 0x55786b74a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.163+0000 7f581e561740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55786b6a2280 con 0x55786b73cb60 2024-09-19T06:50:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.163+0000 7f581e561740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55786b649b10 con 0x55786b73d480 2024-09-19T06:50:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.163+0000 7f581e561740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55786b69e240 con 0x55786b747d40 2024-09-19T06:50:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.163+0000 7f581a725640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b73cb60 0x55786b73cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.163+0000 7f581a725640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b73cb60 0x55786b73cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48016/0 (socket says 172.21.15.103:48016) 2024-09-19T06:50:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.163+0000 7f581a725640 1 -- 172.21.15.103:0/652498865 learned_addr learned my addr 172.21.15.103:0/652498865 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.163+0000 7f5819f24640 1 --2- 172.21.15.103:0/652498865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73d480 0x55786b747800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.164+0000 7f5819723640 1 --2- 172.21.15.103:0/652498865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b747d40 0x55786b74a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.164+0000 7f5819f24640 1 -- 172.21.15.103:0/652498865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b747d40 msgr2=0x55786b74a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.164+0000 7f5819f24640 1 --2- 172.21.15.103:0/652498865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b747d40 0x55786b74a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.164+0000 7f5819f24640 1 -- 172.21.15.103:0/652498865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b73cb60 msgr2=0x55786b73cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:29.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.164+0000 7f5819f24640 1 --2- 172.21.15.103:0/652498865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b73cb60 0x55786b73cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.164+0000 7f5819f24640 1 -- 172.21.15.103:0/652498865 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55786b6561f0 con 0x55786b73d480 2024-09-19T06:50:29.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.164+0000 7f581a725640 1 --2- 172.21.15.103:0/652498865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b73cb60 0x55786b73cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:29.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.164+0000 7f580affd640 1 --2- 172.21.15.103:0/583186766 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f58000a3560 0x7f58000a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.165+0000 7f580affd640 1 -- 172.21.15.103:0/583186766 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f58040cfbd0 con 0x7f5814100aa0 2024-09-19T06:50:29.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.165+0000 7f581a725640 1 --2- 172.21.15.103:0/583186766 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f58000a3560 0x7f58000a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.165+0000 7f5819f24640 1 --2- 172.21.15.103:0/652498865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73d480 0x55786b747800 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5810031550 tx=0x7f5810031770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b9dbc4232efd74c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:29.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.165+0000 7f57fffff640 1 -- 172.21.15.103:0/652498865 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810040070 con 0x55786b73d480 2024-09-19T06:50:29.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.165+0000 7f57fffff640 1 -- 172.21.15.103:0/652498865 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5810031a20 con 0x55786b73d480 2024-09-19T06:50:29.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.165+0000 7f57fffff640 1 -- 172.21.15.103:0/652498865 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810035040 con 0x55786b73d480 2024-09-19T06:50:29.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.166+0000 7f581e561740 1 -- 172.21.15.103:0/652498865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73d480 msgr2=0x55786b747800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:29.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.166+0000 7f581a725640 1 --2- 172.21.15.103:0/583186766 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f58000a3560 0x7f58000a5a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f580c00f940 tx=0x7f580c007540 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:29.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.166+0000 7f581e561740 1 --2- 172.21.15.103:0/652498865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73d480 0x55786b747800 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5810031550 tx=0x7f5810031770 comp rx=0 tx=0).stop 2024-09-19T06:50:29.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.166+0000 7f581e561740 1 -- 172.21.15.103:0/652498865 shutdown_connections 2024-09-19T06:50:29.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.166+0000 7f581e561740 1 --2- 172.21.15.103:0/652498865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b747d40 0x55786b74a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.166+0000 7f581e561740 1 --2- 172.21.15.103:0/652498865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73d480 0x55786b747800 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.166+0000 7f581e561740 1 --2- 172.21.15.103:0/652498865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b73cb60 0x55786b73cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.166+0000 7f581e561740 1 -- 172.21.15.103:0/652498865 >> 172.21.15.103:0/652498865 conn(0x55786b7390f0 msgr2=0x55786b7394d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.166+0000 7f581e561740 1 -- 172.21.15.103:0/652498865 shutdown_connections 2024-09-19T06:50:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.166+0000 7f581e561740 1 -- 172.21.15.103:0/652498865 wait complete. 2024-09-19T06:50:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.167+0000 7f581e561740 1 -- start start 2024-09-19T06:50:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.167+0000 7f581e561740 1 Processor -- start 2024-09-19T06:50:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.167+0000 7f581e561740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73cb60 0x55786b744ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.167+0000 7f581e561740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73d480 0x55786b7450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.167+0000 7f581e561740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b745620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.167+0000 7f581e561740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55786b74ed20 con 0x55786b747d40 2024-09-19T06:50:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.167+0000 7f581e561740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55786b65cf10 con 0x55786b73cb60 2024-09-19T06:50:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.167+0000 7f581e561740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55786b74eea0 con 0x55786b73d480 2024-09-19T06:50:29.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.167+0000 7f581e561740 10 client.0 _ll_register_callbacks cb 0x55786b625a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.168+0000 7f581a725640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b745620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.168+0000 7f5819f24640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73cb60 0x55786b744ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.168+0000 7f581a725640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b745620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48018/0 (socket says 172.21.15.103:48018) 2024-09-19T06:50:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.168+0000 7f581a725640 1 -- 172.21.15.103:0/1163386337 learned_addr learned my addr 172.21.15.103:0/1163386337 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.168+0000 7f5819723640 1 --2- 172.21.15.103:0/1163386337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73d480 0x55786b7450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.168+0000 7f5819f24640 1 -- 172.21.15.103:0/1163386337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73d480 msgr2=0x55786b7450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.168+0000 7f5819f24640 1 --2- 172.21.15.103:0/1163386337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73d480 0x55786b7450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.168+0000 7f5819f24640 1 -- 172.21.15.103:0/1163386337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 msgr2=0x55786b745620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.168+0000 7f5819f24640 1 --2- 172.21.15.103:0/1163386337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b745620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.168+0000 7f5819f24640 1 -- 172.21.15.103:0/1163386337 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55786b4136a0 con 0x55786b73cb60 2024-09-19T06:50:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.168+0000 7f5819f24640 1 --2- 172.21.15.103:0/1163386337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73cb60 0x55786b744ba0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5810005ab0 tx=0x7f5810004260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c979e4c59d64d30 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.168+0000 7f57fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:50:29.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.168+0000 7f57fd7fa640 1 -- 172.21.15.103:0/1163386337 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810040030 con 0x55786b73cb60 2024-09-19T06:50:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.168+0000 7f57fd7fa640 1 -- 172.21.15.103:0/1163386337 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f58100035f0 con 0x55786b73cb60 2024-09-19T06:50:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.168+0000 7f581e561740 1 -- 172.21.15.103:0/1163386337 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5810031a20 con 0x55786b73cb60 2024-09-19T06:50:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.169+0000 7f57fd7fa640 1 -- 172.21.15.103:0/1163386337 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5810035070 con 0x55786b73cb60 2024-09-19T06:50:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.169+0000 7f581e561740 10 client.25319 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.169+0000 7f581e561740 1 -- 172.21.15.103:0/1163386337 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55786b75a8b0 con 0x55786b73cb60 2024-09-19T06:50:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.169+0000 7f57fd7fa640 1 -- 172.21.15.103:0/1163386337 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5810031a20 con 0x55786b73cb60 2024-09-19T06:50:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.169+0000 7f57fd7fa640 1 -- 172.21.15.103:0/1163386337 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f581003c7e0 con 0x55786b73cb60 2024-09-19T06:50:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.169+0000 7f581e561740 10 client.25319 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.169+0000 7f581e561740 10 client.25319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.169+0000 7f581e561740 20 client.25319 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.169+0000 7f581e561740 1 --2- 172.21.15.103:0/1163386337 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55786b75bc00 0x55786b75dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.169+0000 7f581e561740 4 client.25319 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:29.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.169+0000 7f581e561740 1 -- 172.21.15.103:0/1163386337 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55786b6af7f0 con 0x55786b75bc00 2024-09-19T06:50:29.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.170+0000 7f5819723640 1 --2- 172.21.15.103:0/1163386337 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55786b75bc00 0x55786b75dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.170+0000 7f57fd7fa640 10 client.25319 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:29.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.170+0000 7f5819723640 1 --2- 172.21.15.103:0/1163386337 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55786b75bc00 0x55786b75dff0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca859f1c6b0ef3f7 server_cookie=887661b40639b18d in_seq=0 out_seq=0 2024-09-19T06:50:29.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.171+0000 7f57fd7fa640 1 -- 172.21.15.103:0/1163386337 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55786b6af7f0 con 0x55786b75bc00 2024-09-19T06:50:29.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.171+0000 7f57fd7fa640 10 client.25319 handle_command_reply: tid=0 2024-09-19T06:50:29.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.171+0000 7f581e561740 1 client.25319 shutdown 2024-09-19T06:50:29.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.171+0000 7f581e561740 1 -- 172.21.15.103:0/1163386337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73cb60 msgr2=0x55786b744ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:29.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.171+0000 7f581e561740 1 --2- 172.21.15.103:0/1163386337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73cb60 0x55786b744ba0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5810005ab0 tx=0x7f5810004260 comp rx=0 tx=0).stop 2024-09-19T06:50:29.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.171+0000 7f581e561740 1 -- 172.21.15.103:0/1163386337 shutdown_connections 2024-09-19T06:50:29.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.171+0000 7f581e561740 1 --2- 172.21.15.103:0/1163386337 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55786b75bc00 0x55786b75dff0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.171+0000 7f57fd7fa640 0 client.25319 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:29.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.171+0000 7f581e561740 1 --2- 172.21.15.103:0/1163386337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b745620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.171+0000 7f581e561740 1 --2- 172.21.15.103:0/1163386337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73d480 0x55786b7450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.171+0000 7f581e561740 1 --2- 172.21.15.103:0/1163386337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73cb60 0x55786b744ba0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.171+0000 7f581e561740 1 -- 172.21.15.103:0/1163386337 >> 172.21.15.103:0/1163386337 conn(0x55786b7390f0 msgr2=0x55786b73ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:29.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.172+0000 7f581e561740 1 -- 172.21.15.103:0/1163386337 shutdown_connections 2024-09-19T06:50:29.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.172+0000 7f581e561740 1 -- 172.21.15.103:0/1163386337 wait complete. 2024-09-19T06:50:29.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.172+0000 7f581e561740 20 client.25319 trim_cache size 0 max 16384 2024-09-19T06:50:29.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.201+0000 7f581e561740 1 Processor -- start 2024-09-19T06:50:29.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.201+0000 7f581e561740 1 -- start start 2024-09-19T06:50:29.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.201+0000 7f581e561740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73cb60 0x55786b743a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.201+0000 7f581e561740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73d480 0x55786b761010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.201+0000 7f581e561740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b761550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.201+0000 7f581e561740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f58141ab580 con 0x55786b747d40 2024-09-19T06:50:29.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.201+0000 7f581e561740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f58141ad0c0 con 0x55786b73cb60 2024-09-19T06:50:29.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.201+0000 7f581e561740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f58141abc50 con 0x55786b73d480 2024-09-19T06:50:29.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.202+0000 7f581a725640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b761550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.202+0000 7f581a725640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b761550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48020/0 (socket says 172.21.15.103:48020) 2024-09-19T06:50:29.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.202+0000 7f581a725640 1 -- 172.21.15.103:0/3210431935 learned_addr learned my addr 172.21.15.103:0/3210431935 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:29.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.202+0000 7f5819723640 1 --2- 172.21.15.103:0/3210431935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73d480 0x55786b761010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.202+0000 7f5819f24640 1 --2- 172.21.15.103:0/3210431935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73cb60 0x55786b743a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.202+0000 7f5819723640 1 -- 172.21.15.103:0/3210431935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73cb60 msgr2=0x55786b743a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:29.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.202+0000 7f5819723640 1 --2- 172.21.15.103:0/3210431935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73cb60 0x55786b743a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.202+0000 7f5819723640 1 -- 172.21.15.103:0/3210431935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 msgr2=0x55786b761550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:29.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.202+0000 7f5819723640 1 --2- 172.21.15.103:0/3210431935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b761550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.202+0000 7f5819723640 1 -- 172.21.15.103:0/3210431935 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55786b73d2b0 con 0x55786b73d480 2024-09-19T06:50:29.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.202+0000 7f581a725640 1 --2- 172.21.15.103:0/3210431935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b761550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:29.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.202+0000 7f5819723640 1 --2- 172.21.15.103:0/3210431935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73d480 0x55786b761010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f58040cd0f0 tx=0x7f5804004910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9307ec628e431e55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:29.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.202+0000 7f57ff7fe640 1 -- 172.21.15.103:0/3210431935 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580401d070 con 0x55786b73d480 2024-09-19T06:50:29.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.203+0000 7f57ff7fe640 1 -- 172.21.15.103:0/3210431935 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f580409bd30 con 0x55786b73d480 2024-09-19T06:50:29.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.203+0000 7f57ff7fe640 1 -- 172.21.15.103:0/3210431935 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58040983a0 con 0x55786b73d480 2024-09-19T06:50:29.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.203+0000 7f581e561740 1 -- 172.21.15.103:0/3210431935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73d480 msgr2=0x55786b761010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:29.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.203+0000 7f581e561740 1 --2- 172.21.15.103:0/3210431935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73d480 0x55786b761010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f58040cd0f0 tx=0x7f5804004910 comp rx=0 tx=0).stop 2024-09-19T06:50:29.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.203+0000 7f581e561740 1 -- 172.21.15.103:0/3210431935 shutdown_connections 2024-09-19T06:50:29.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.203+0000 7f581e561740 1 --2- 172.21.15.103:0/3210431935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b761550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.203+0000 7f581e561740 1 --2- 172.21.15.103:0/3210431935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73d480 0x55786b761010 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.203+0000 7f581e561740 1 --2- 172.21.15.103:0/3210431935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73cb60 0x55786b743a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.203+0000 7f581e561740 1 -- 172.21.15.103:0/3210431935 >> 172.21.15.103:0/3210431935 conn(0x55786b7390f0 msgr2=0x55786b743340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:29.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.204+0000 7f581e561740 1 -- 172.21.15.103:0/3210431935 shutdown_connections 2024-09-19T06:50:29.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.204+0000 7f581e561740 1 -- 172.21.15.103:0/3210431935 wait complete. 2024-09-19T06:50:29.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.204+0000 7f581e561740 1 -- start start 2024-09-19T06:50:29.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.204+0000 7f581e561740 1 Processor -- start 2024-09-19T06:50:29.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.205+0000 7f581e561740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73cb60 0x55786b75edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.205+0000 7f581e561740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73d480 0x55786b75f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.205+0000 7f581e561740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b75f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.205+0000 7f581a725640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b75f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.205+0000 7f5819f24640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73cb60 0x55786b75edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.205+0000 7f5819f24640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73cb60 0x55786b75edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:54344/0 (socket says 172.21.15.103:54344) 2024-09-19T06:50:29.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.205+0000 7f581a725640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b75f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48032/0 (socket says 172.21.15.103:48032) 2024-09-19T06:50:29.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.205+0000 7f5819723640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73d480 0x55786b75f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.205+0000 7f5819f24640 1 -- 172.21.15.103:0/2784897895 learned_addr learned my addr 172.21.15.103:0/2784897895 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:29.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.205+0000 7f581e561740 1 -- 172.21.15.103:0/2784897895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f58141ac9f0 con 0x55786b747d40 2024-09-19T06:50:29.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.205+0000 7f581e561740 1 -- 172.21.15.103:0/2784897895 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f58141ac320 con 0x55786b73d480 2024-09-19T06:50:29.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.205+0000 7f581e561740 1 -- 172.21.15.103:0/2784897895 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f58141a9560 con 0x55786b73cb60 2024-09-19T06:50:29.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.206+0000 7f581a725640 1 -- 172.21.15.103:0/2784897895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73cb60 msgr2=0x55786b75edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:29.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.206+0000 7f581a725640 1 --2- 172.21.15.103:0/2784897895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73cb60 0x55786b75edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.206+0000 7f581a725640 1 -- 172.21.15.103:0/2784897895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73d480 msgr2=0x55786b75f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:29.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.206+0000 7f581a725640 1 --2- 172.21.15.103:0/2784897895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73d480 0x55786b75f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.206+0000 7f581a725640 1 -- 172.21.15.103:0/2784897895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55786b7c5540 con 0x55786b747d40 2024-09-19T06:50:29.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.206+0000 7f581e561740 10 client.0 _ll_register_callbacks cb 0x7f580c007c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:29.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.206+0000 7f5819723640 1 --2- 172.21.15.103:0/2784897895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73d480 0x55786b75f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:29.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.206+0000 7f581e561740 1 -- 172.21.15.103:0/2784897895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f580409bd30 con 0x55786b747d40 2024-09-19T06:50:29.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.206+0000 7f581a725640 1 --2- 172.21.15.103:0/2784897895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b75f840 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55786b734960 tx=0x7f580c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33450245221caed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:29.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.206+0000 7f57fd7fa640 10 client.15915 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:29.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.206+0000 7f57fd7fa640 1 -- 172.21.15.103:0/2784897895 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c003970 con 0x55786b747d40 2024-09-19T06:50:29.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.207+0000 7f57fd7fa640 1 -- 172.21.15.103:0/2784897895 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f580c003b10 con 0x55786b747d40 2024-09-19T06:50:29.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.207+0000 7f57fd7fa640 1 -- 172.21.15.103:0/2784897895 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f580c003d30 con 0x55786b747d40 2024-09-19T06:50:29.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.207+0000 7f581e561740 10 client.15915 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:29.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.207+0000 7f581e561740 1 -- 172.21.15.103:0/2784897895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55786b7c66d0 con 0x55786b747d40 2024-09-19T06:50:29.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.207+0000 7f57fd7fa640 1 -- 172.21.15.103:0/2784897895 <== mon.0 v2:172.21.15.103:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f580c004030 con 0x55786b747d40 2024-09-19T06:50:29.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.207+0000 7f581e561740 10 client.15915 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:29.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.207+0000 7f581e561740 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:29.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.207+0000 7f581e561740 20 client.15915 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:29.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.207+0000 7f581e561740 1 --2- 172.21.15.103:0/2784897895 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55786b75bc00 0x55786b7c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:29.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.207+0000 7f581e561740 4 client.15915 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:50:29.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.207+0000 7f581e561740 1 -- 172.21.15.103:0/2784897895 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55786b6a9d00 con 0x55786b75bc00 2024-09-19T06:50:29.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.208+0000 7f5819f24640 1 --2- 172.21.15.103:0/2784897895 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55786b75bc00 0x55786b7c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:29.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.208+0000 7f5819f24640 1 --2- 172.21.15.103:0/2784897895 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55786b75bc00 0x55786b7c8e00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c817a967f2adf15 server_cookie=749395b9384b013 in_seq=0 out_seq=0 2024-09-19T06:50:29.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.208+0000 7f57fd7fa640 10 client.15915 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:29.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.209+0000 7f57fd7fa640 1 -- 172.21.15.103:0/2784897895 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55786b6a9d00 con 0x55786b75bc00 2024-09-19T06:50:29.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.209+0000 7f57fd7fa640 10 client.15915 handle_command_reply: tid=0 2024-09-19T06:50:29.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.209+0000 7f581e561740 1 client.15915 shutdown 2024-09-19T06:50:29.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.209+0000 7f581e561740 1 -- 172.21.15.103:0/2784897895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 msgr2=0x55786b75f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:29.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.209+0000 7f581e561740 1 --2- 172.21.15.103:0/2784897895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b75f840 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55786b734960 tx=0x7f580c004a40 comp rx=0 tx=0).stop 2024-09-19T06:50:29.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.210+0000 7f581e561740 1 -- 172.21.15.103:0/2784897895 shutdown_connections 2024-09-19T06:50:29.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.210+0000 7f581e561740 1 --2- 172.21.15.103:0/2784897895 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55786b75bc00 0x55786b7c8e00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.210+0000 7f57fd7fa640 0 client.15915 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:29.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.210+0000 7f581e561740 1 --2- 172.21.15.103:0/2784897895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786b747d40 0x55786b75f840 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.210+0000 7f581e561740 1 --2- 172.21.15.103:0/2784897895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55786b73d480 0x55786b75f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.210+0000 7f581e561740 1 --2- 172.21.15.103:0/2784897895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55786b73cb60 0x55786b75edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.210+0000 7f581e561740 1 -- 172.21.15.103:0/2784897895 >> 172.21.15.103:0/2784897895 conn(0x55786b7390f0 msgr2=0x55786b745380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:29.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.210+0000 7f581e561740 1 -- 172.21.15.103:0/2784897895 shutdown_connections 2024-09-19T06:50:29.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.210+0000 7f581e561740 1 -- 172.21.15.103:0/2784897895 wait complete. 2024-09-19T06:50:29.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.210+0000 7f581e561740 20 client.15915 trim_cache size 0 max 16384 2024-09-19T06:50:29.217 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:50:29.217 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T06:50:29.217 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "1ada1e2f-7d57-48e2-a031-d5a2cbcbdf8e", 2024-09-19T06:50:29.217 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T06:50:29.217 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:50:29.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.211+0000 7f581c188640 1 -- 172.21.15.103:0/583186766 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f58000a3560 msgr2=0x7f58000a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:29.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.211+0000 7f581c188640 1 --2- 172.21.15.103:0/583186766 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f58000a3560 0x7f58000a5a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f580c00f940 tx=0x7f580c007540 comp rx=0 tx=0).stop 2024-09-19T06:50:29.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.211+0000 7f581c188640 1 -- 172.21.15.103:0/583186766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5814100aa0 msgr2=0x7f5814198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:29.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.211+0000 7f581c188640 1 --2- 172.21.15.103:0/583186766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5814100aa0 0x7f5814198a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f580400c970 tx=0x7f580400ce30 comp rx=0 tx=0).stop 2024-09-19T06:50:29.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.211+0000 7f581c188640 1 -- 172.21.15.103:0/583186766 shutdown_connections 2024-09-19T06:50:29.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.211+0000 7f581c188640 1 --2- 172.21.15.103:0/583186766 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f58000a3560 0x7f58000a5a20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.211+0000 7f581c188640 1 --2- 172.21.15.103:0/583186766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5814106b80 0x7f581419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.211+0000 7f581c188640 1 --2- 172.21.15.103:0/583186766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5814100aa0 0x7f5814198a20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.211+0000 7f581c188640 1 --2- 172.21.15.103:0/583186766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5814100100 0x7f58141984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:29.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.211+0000 7f581c188640 1 -- 172.21.15.103:0/583186766 >> 172.21.15.103:0/583186766 conn(0x7f5814077230 msgr2=0x7f5814104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:29.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.211+0000 7f581c188640 1 -- 172.21.15.103:0/583186766 shutdown_connections 2024-09-19T06:50:29.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:29.212+0000 7f581c188640 1 -- 172.21.15.103:0/583186766 wait complete. 2024-09-19T06:50:29.974 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:50:30.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:29 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T06:50:30.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:29 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:30.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:29 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T06:50:30.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:29 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:30.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:29 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T06:50:30.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:29 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T06:50:30.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:29 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:30.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:29 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T06:50:30.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:29 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:30.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:29 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T06:50:30.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:29 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T06:50:30.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:29 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:30.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:29 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T06:50:30.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:29 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:30.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:29 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T06:50:30.288 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:30.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.740+0000 7f34b65da640 1 -- 172.21.15.103:0/2862749408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34b0106f20 msgr2=0x7f34b010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:30.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.740+0000 7f34b65da640 1 --2- 172.21.15.103:0/2862749408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34b0106f20 0x7f34b010eaa0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009ca0 tx=0x7f34a802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.740+0000 7f34b65da640 1 -- 172.21.15.103:0/2862749408 shutdown_connections 2024-09-19T06:50:30.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.740+0000 7f34b65da640 1 --2- 172.21.15.103:0/2862749408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34b0106f20 0x7f34b010eaa0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.740+0000 7f34b65da640 1 --2- 172.21.15.103:0/2862749408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34b01027a0 0x7f34b01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.740+0000 7f34b65da640 1 --2- 172.21.15.103:0/2862749408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34b0101df0 0x7f34b01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.740+0000 7f34b65da640 1 -- 172.21.15.103:0/2862749408 >> 172.21.15.103:0/2862749408 conn(0x7f34b00fd8f0 msgr2=0x7f34b00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:30.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.741+0000 7f34b65da640 1 -- 172.21.15.103:0/2862749408 shutdown_connections 2024-09-19T06:50:30.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.741+0000 7f34b65da640 1 -- 172.21.15.103:0/2862749408 wait complete. 2024-09-19T06:50:30.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.741+0000 7f34b65da640 1 Processor -- start 2024-09-19T06:50:30.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.741+0000 7f34b65da640 1 -- start start 2024-09-19T06:50:30.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.742+0000 7f34b65da640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34b0101df0 0x7f34b0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.742+0000 7f34b65da640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34b01027a0 0x7f34b01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.742+0000 7f34b65da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34b0106f20 0x7f34b019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.742+0000 7f34b65da640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0111f80 con 0x7f34b0101df0 2024-09-19T06:50:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.742+0000 7f34b65da640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0111e00 con 0x7f34b0106f20 2024-09-19T06:50:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.742+0000 7f34b65da640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0112100 con 0x7f34b01027a0 2024-09-19T06:50:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.742+0000 7f34affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34b0101df0 0x7f34b0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.742+0000 7f34affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34b0101df0 0x7f34b0198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48060/0 (socket says 172.21.15.103:48060) 2024-09-19T06:50:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.742+0000 7f34affff640 1 -- 172.21.15.103:0/3809877005 learned_addr learned my addr 172.21.15.103:0/3809877005 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.742+0000 7f34af7fe640 1 --2- 172.21.15.103:0/3809877005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34b01027a0 0x7f34b01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.743+0000 7f34affff640 1 -- 172.21.15.103:0/3809877005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34b01027a0 msgr2=0x7f34b01988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.743+0000 7f34affff640 1 --2- 172.21.15.103:0/3809877005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34b01027a0 0x7f34b01988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.743+0000 7f34affff640 1 -- 172.21.15.103:0/3809877005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34b0106f20 msgr2=0x7f34b019cc40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T06:50:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.743+0000 7f34affff640 1 --2- 172.21.15.103:0/3809877005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34b0106f20 0x7f34b019cc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.743+0000 7f34affff640 1 -- 172.21.15.103:0/3809877005 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b019d320 con 0x7f34b0101df0 2024-09-19T06:50:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.743+0000 7f34affff640 1 --2- 172.21.15.103:0/3809877005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34b0101df0 0x7f34b0198370 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f34a000af00 tx=0x7f34a000cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:30.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.743+0000 7f34ad7fa640 1 -- 172.21.15.103:0/3809877005 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a001e070 con 0x7f34b0101df0 2024-09-19T06:50:30.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.743+0000 7f34ad7fa640 1 -- 172.21.15.103:0/3809877005 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a0011c30 con 0x7f34b0101df0 2024-09-19T06:50:30.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.743+0000 7f34ad7fa640 1 -- 172.21.15.103:0/3809877005 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0019900 con 0x7f34b0101df0 2024-09-19T06:50:30.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.743+0000 7f34b65da640 1 -- 172.21.15.103:0/3809877005 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b019d610 con 0x7f34b0101df0 2024-09-19T06:50:30.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.744+0000 7f34b65da640 1 -- 172.21.15.103:0/3809877005 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b0074420 con 0x7f34b0101df0 2024-09-19T06:50:30.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.747+0000 7f34ad7fa640 1 -- 172.21.15.103:0/3809877005 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7f34a0021070 con 0x7f34b0101df0 2024-09-19T06:50:30.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.747+0000 7f34b89b3740 1 Processor -- start 2024-09-19T06:50:30.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.747+0000 7f34b89b3740 1 -- start start 2024-09-19T06:50:30.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.747+0000 7f34b89b3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640ecb0 0x5561a640f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.747+0000 7f34ad7fa640 1 --2- 172.21.15.103:0/3809877005 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34980a35e0 0x7f34980a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.747+0000 7f34b89b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640f5d0 0x5561a6419950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.747+0000 7f34b89b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a6419e90 0x5561a641c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.747+0000 7f34b89b3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5561a636c2e0 con 0x5561a640ecb0 2024-09-19T06:50:30.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.747+0000 7f34b89b3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5561a6313ce0 con 0x5561a640f5d0 2024-09-19T06:50:30.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.747+0000 7f34b89b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5561a63682a0 con 0x5561a6419e90 2024-09-19T06:50:30.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.748+0000 7f34b4b77640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640ecb0 0x5561a640f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:30.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.748+0000 7f34affff640 1 --2- 172.21.15.103:0/3809877005 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34980a35e0 0x7f34980a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:30.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.748+0000 7f34b4b77640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640ecb0 0x5561a640f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48072/0 (socket says 172.21.15.103:48072) 2024-09-19T06:50:30.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.748+0000 7f34b4b77640 1 -- 172.21.15.103:0/754604433 learned_addr learned my addr 172.21.15.103:0/754604433 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:30.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.748+0000 7f34af7fe640 1 --2- 172.21.15.103:0/754604433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640f5d0 0x5561a6419950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:30.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.748+0000 7f34ad7fa640 1 -- 172.21.15.103:0/3809877005 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f34a00cf690 con 0x7f34b0101df0 2024-09-19T06:50:30.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.748+0000 7f34af7fe640 1 -- 172.21.15.103:0/754604433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a6419e90 msgr2=0x5561a641c280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:50:30.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.748+0000 7f34af7fe640 1 --2- 172.21.15.103:0/754604433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a6419e90 0x5561a641c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.748+0000 7f34af7fe640 1 -- 172.21.15.103:0/754604433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640ecb0 msgr2=0x5561a640f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:30.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.748+0000 7f34af7fe640 1 --2- 172.21.15.103:0/754604433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640ecb0 0x5561a640f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.748+0000 7f34affff640 1 --2- 172.21.15.103:0/3809877005 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34980a35e0 0x7f34980a5aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f34a0011370 tx=0x7f34a009bfe0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:30.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.748+0000 7f34af7fe640 1 -- 172.21.15.103:0/754604433 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561a62f8440 con 0x5561a640f5d0 2024-09-19T06:50:30.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.748+0000 7f34af7fe640 1 --2- 172.21.15.103:0/754604433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640f5d0 0x5561a6419950 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f34a400ab40 tx=0x7f34a400f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a3fd9b382dd8e07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:30.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.748+0000 7f34967fc640 1 -- 172.21.15.103:0/754604433 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4005020 con 0x5561a640f5d0 2024-09-19T06:50:30.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.749+0000 7f34967fc640 1 -- 172.21.15.103:0/754604433 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a40047a0 con 0x5561a640f5d0 2024-09-19T06:50:30.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.749+0000 7f34967fc640 1 -- 172.21.15.103:0/754604433 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4018860 con 0x5561a640f5d0 2024-09-19T06:50:30.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.749+0000 7f34b89b3740 1 -- 172.21.15.103:0/754604433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640f5d0 msgr2=0x5561a6419950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:30.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.749+0000 7f34b89b3740 1 --2- 172.21.15.103:0/754604433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640f5d0 0x5561a6419950 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f34a400ab40 tx=0x7f34a400f3c0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.750+0000 7f34b89b3740 1 -- 172.21.15.103:0/754604433 shutdown_connections 2024-09-19T06:50:30.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.750+0000 7f34b89b3740 1 --2- 172.21.15.103:0/754604433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a6419e90 0x5561a641c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.750+0000 7f34b89b3740 1 --2- 172.21.15.103:0/754604433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640f5d0 0x5561a6419950 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.750+0000 7f34b89b3740 1 --2- 172.21.15.103:0/754604433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640ecb0 0x5561a640f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.750+0000 7f34b89b3740 1 -- 172.21.15.103:0/754604433 >> 172.21.15.103:0/754604433 conn(0x5561a640b240 msgr2=0x5561a640b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:30.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.750+0000 7f34b89b3740 1 -- 172.21.15.103:0/754604433 shutdown_connections 2024-09-19T06:50:30.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.750+0000 7f34b89b3740 1 -- 172.21.15.103:0/754604433 wait complete. 2024-09-19T06:50:30.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.750+0000 7f34b89b3740 1 -- start start 2024-09-19T06:50:30.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.750+0000 7f34b89b3740 1 Processor -- start 2024-09-19T06:50:30.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.751+0000 7f34b89b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a640ecb0 0x5561a6419730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.751+0000 7f34b89b3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640f5d0 0x5561a64169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.751+0000 7f34b89b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a6419e90 0x5561a6416f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.751+0000 7f34af7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640f5d0 0x5561a64169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:30.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.751+0000 7f34af7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640f5d0 0x5561a64169e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48082/0 (socket says 172.21.15.103:48082) 2024-09-19T06:50:30.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.751+0000 7f34b4b77640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a6419e90 0x5561a6416f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:30.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.751+0000 7f34b4b77640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a640ecb0 0x5561a6419730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:30.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.751+0000 7f34b4b77640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a640ecb0 0x5561a6419730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:54394/0 (socket says 172.21.15.103:54394) 2024-09-19T06:50:30.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.751+0000 7f34af7fe640 1 -- 172.21.15.103:0/366601536 learned_addr learned my addr 172.21.15.103:0/366601536 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:30.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.751+0000 7f34b89b3740 1 -- 172.21.15.103:0/366601536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5561a641ca60 con 0x5561a640f5d0 2024-09-19T06:50:30.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.751+0000 7f34b89b3740 1 -- 172.21.15.103:0/366601536 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5561a6327050 con 0x5561a6419e90 2024-09-19T06:50:30.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.751+0000 7f34b89b3740 1 -- 172.21.15.103:0/366601536 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5561a641cbe0 con 0x5561a640ecb0 2024-09-19T06:50:30.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.751+0000 7f34af7fe640 1 -- 172.21.15.103:0/366601536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a640ecb0 msgr2=0x5561a6419730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:30.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.751+0000 7f34af7fe640 1 --2- 172.21.15.103:0/366601536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a640ecb0 0x5561a6419730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.752+0000 7f34af7fe640 1 -- 172.21.15.103:0/366601536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a6419e90 msgr2=0x5561a6416f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:30.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.752+0000 7f34af7fe640 1 --2- 172.21.15.103:0/366601536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a6419e90 0x5561a6416f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.752+0000 7f34af7fe640 1 -- 172.21.15.103:0/366601536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561a60dd3c0 con 0x5561a640f5d0 2024-09-19T06:50:30.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.752+0000 7f34b4b77640 1 --2- 172.21.15.103:0/366601536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a640ecb0 0x5561a6419730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:30.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.752+0000 7f34b89b3740 10 client.0 _ll_register_callbacks cb 0x5561a6367160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:30.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.752+0000 7f34af7fe640 1 --2- 172.21.15.103:0/366601536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640f5d0 0x5561a64169e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f34a401a040 tx=0x7f34a4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80cd0e9d29c2e00b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:30.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.752+0000 7f348f7fe640 10 client.15927 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:30.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.752+0000 7f34b89b3740 1 -- 172.21.15.103:0/366601536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a40047a0 con 0x5561a640f5d0 2024-09-19T06:50:30.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.752+0000 7f348f7fe640 1 -- 172.21.15.103:0/366601536 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4031050 con 0x5561a640f5d0 2024-09-19T06:50:30.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.752+0000 7f348f7fe640 1 -- 172.21.15.103:0/366601536 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f34a4005210 con 0x5561a640f5d0 2024-09-19T06:50:30.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.752+0000 7f348f7fe640 1 -- 172.21.15.103:0/366601536 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f34a4027070 con 0x5561a640f5d0 2024-09-19T06:50:30.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.752+0000 7f34b89b3740 10 client.15927 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:30.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.752+0000 7f34b89b3740 1 -- 172.21.15.103:0/366601536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5561a64281b0 con 0x5561a640f5d0 2024-09-19T06:50:30.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.753+0000 7f348f7fe640 1 -- 172.21.15.103:0/366601536 <== mon.0 v2:172.21.15.103:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f34a40185e0 con 0x5561a640f5d0 2024-09-19T06:50:30.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.753+0000 7f34b89b3740 10 client.15927 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:30.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.753+0000 7f34b89b3740 10 client.15927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:30.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.753+0000 7f34b89b3740 20 client.15927 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:30.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.753+0000 7f34b89b3740 1 --2- 172.21.15.103:0/366601536 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5561a6429a00 0x5561a642be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.753+0000 7f34b89b3740 4 client.15927 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:30.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.753+0000 7f34b89b3740 1 -- 172.21.15.103:0/366601536 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561a6379850 con 0x5561a6429a00 2024-09-19T06:50:30.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.753+0000 7f34b4b77640 1 --2- 172.21.15.103:0/366601536 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5561a6429a00 0x5561a642be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:30.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.754+0000 7f34b4b77640 1 --2- 172.21.15.103:0/366601536 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5561a6429a00 0x5561a642be90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd91846982d67bc server_cookie=e5e0ce807cab3094 in_seq=0 out_seq=0 2024-09-19T06:50:30.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.754+0000 7f348f7fe640 10 client.15927 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:30.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.754+0000 7f348f7fe640 1 -- 172.21.15.103:0/366601536 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5561a6379850 con 0x5561a6429a00 2024-09-19T06:50:30.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.754+0000 7f348f7fe640 10 client.15927 handle_command_reply: tid=0 2024-09-19T06:50:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.754+0000 7f34b89b3740 1 client.15927 shutdown 2024-09-19T06:50:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.755+0000 7f34b89b3740 1 -- 172.21.15.103:0/366601536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640f5d0 msgr2=0x5561a64169e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.755+0000 7f34b89b3740 1 --2- 172.21.15.103:0/366601536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640f5d0 0x5561a64169e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f34a401a040 tx=0x7f34a4005850 comp rx=0 tx=0).stop 2024-09-19T06:50:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.755+0000 7f34b89b3740 1 -- 172.21.15.103:0/366601536 shutdown_connections 2024-09-19T06:50:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.755+0000 7f34b89b3740 1 --2- 172.21.15.103:0/366601536 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5561a6429a00 0x5561a642be90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.755+0000 7f34b89b3740 1 --2- 172.21.15.103:0/366601536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a6419e90 0x5561a6416f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.755+0000 7f34b89b3740 1 --2- 172.21.15.103:0/366601536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640f5d0 0x5561a64169e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.755+0000 7f34b89b3740 1 --2- 172.21.15.103:0/366601536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a640ecb0 0x5561a6419730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.755+0000 7f34b89b3740 1 -- 172.21.15.103:0/366601536 >> 172.21.15.103:0/366601536 conn(0x5561a640b240 msgr2=0x5561a6410f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.755+0000 7f348f7fe640 0 client.15927 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.755+0000 7f34b89b3740 1 -- 172.21.15.103:0/366601536 shutdown_connections 2024-09-19T06:50:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.755+0000 7f34b89b3740 1 -- 172.21.15.103:0/366601536 wait complete. 2024-09-19T06:50:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.755+0000 7f34b89b3740 20 client.15927 trim_cache size 0 max 16384 2024-09-19T06:50:30.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.788+0000 7f34b89b3740 1 Processor -- start 2024-09-19T06:50:30.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.788+0000 7f34b89b3740 1 -- start start 2024-09-19T06:50:30.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34b89b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640ecb0 0x5561a64317a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34b89b3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640f5d0 0x5561a6431ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34b89b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a6419e90 0x5561a6410700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34b89b3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f34b01ab6a0 con 0x5561a640f5d0 2024-09-19T06:50:30.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34b89b3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34b01ad1e0 con 0x5561a640ecb0 2024-09-19T06:50:30.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34b89b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34b01abd70 con 0x5561a6419e90 2024-09-19T06:50:30.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34af7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640f5d0 0x5561a6431ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:30.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34af7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640f5d0 0x5561a6431ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48098/0 (socket says 172.21.15.103:48098) 2024-09-19T06:50:30.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34af7fe640 1 -- 172.21.15.103:0/3134591915 learned_addr learned my addr 172.21.15.103:0/3134591915 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:30.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34b4b77640 1 --2- 172.21.15.103:0/3134591915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640ecb0 0x5561a64317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:30.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34b4b77640 1 --2- 172.21.15.103:0/3134591915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a6419e90 0x5561a6410700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:30.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34af7fe640 1 -- 172.21.15.103:0/3134591915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a6419e90 msgr2=0x5561a6410700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:30.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34af7fe640 1 --2- 172.21.15.103:0/3134591915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a6419e90 0x5561a6410700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34af7fe640 1 -- 172.21.15.103:0/3134591915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640ecb0 msgr2=0x5561a64317a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:30.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34af7fe640 1 --2- 172.21.15.103:0/3134591915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640ecb0 0x5561a64317a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34af7fe640 1 -- 172.21.15.103:0/3134591915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561a640b070 con 0x5561a640f5d0 2024-09-19T06:50:30.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.789+0000 7f34b4b77640 1 --2- 172.21.15.103:0/3134591915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640ecb0 0x5561a64317a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:30.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.790+0000 7f34af7fe640 1 --2- 172.21.15.103:0/3134591915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640f5d0 0x5561a6431ce0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f34a4005940 tx=0x7f34a401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09edb54782ab859 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:30.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.790+0000 7f3495ffb640 1 -- 172.21.15.103:0/3134591915 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4031070 con 0x5561a640f5d0 2024-09-19T06:50:30.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.790+0000 7f3495ffb640 1 -- 172.21.15.103:0/3134591915 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a40047a0 con 0x5561a640f5d0 2024-09-19T06:50:30.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.790+0000 7f3495ffb640 1 -- 172.21.15.103:0/3134591915 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a402c6a0 con 0x5561a640f5d0 2024-09-19T06:50:30.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.791+0000 7f34b89b3740 1 -- 172.21.15.103:0/3134591915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640f5d0 msgr2=0x5561a6431ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:30.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.791+0000 7f34b89b3740 1 --2- 172.21.15.103:0/3134591915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640f5d0 0x5561a6431ce0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f34a4005940 tx=0x7f34a401d000 comp rx=0 tx=0).stop 2024-09-19T06:50:30.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.791+0000 7f34b89b3740 1 -- 172.21.15.103:0/3134591915 shutdown_connections 2024-09-19T06:50:30.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.791+0000 7f34b89b3740 1 --2- 172.21.15.103:0/3134591915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a6419e90 0x5561a6410700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.791+0000 7f34b89b3740 1 --2- 172.21.15.103:0/3134591915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a640f5d0 0x5561a6431ce0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.791+0000 7f34b89b3740 1 --2- 172.21.15.103:0/3134591915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640ecb0 0x5561a64317a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.791+0000 7f34b89b3740 1 -- 172.21.15.103:0/3134591915 >> 172.21.15.103:0/3134591915 conn(0x5561a640b240 msgr2=0x5561a6401b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:30.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.791+0000 7f34b89b3740 1 -- 172.21.15.103:0/3134591915 shutdown_connections 2024-09-19T06:50:30.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.791+0000 7f34b89b3740 1 -- 172.21.15.103:0/3134591915 wait complete. 2024-09-19T06:50:30.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.792+0000 7f34b89b3740 1 -- start start 2024-09-19T06:50:30.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.792+0000 7f34b89b3740 1 Processor -- start 2024-09-19T06:50:30.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.792+0000 7f34b89b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a640ecb0 0x5561a641cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.792+0000 7f34b89b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640f5d0 0x5561a641d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.792+0000 7f34b89b3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a6419e90 0x5561a641d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.792+0000 7f34b89b3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f34b01acb10 con 0x5561a6419e90 2024-09-19T06:50:30.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.792+0000 7f34b89b3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34b01ac440 con 0x5561a640f5d0 2024-09-19T06:50:30.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.792+0000 7f34b89b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34b01a9660 con 0x5561a640ecb0 2024-09-19T06:50:30.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.792+0000 7f34b89b3740 10 client.0 _ll_register_callbacks cb 0x7f34a400a9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:30.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.793+0000 7f34af7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640f5d0 0x5561a641d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:30.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.793+0000 7f34af7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640f5d0 0x5561a641d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:32908/0 (socket says 172.21.15.103:32908) 2024-09-19T06:50:30.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.793+0000 7f34af7fe640 1 -- 172.21.15.103:0/799530848 learned_addr learned my addr 172.21.15.103:0/799530848 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:30.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.793+0000 7f34b4b77640 1 --2- 172.21.15.103:0/799530848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a6419e90 0x5561a641d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:30.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.793+0000 7f34af7fe640 1 -- 172.21.15.103:0/799530848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a640ecb0 msgr2=0x5561a641cdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:50:30.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.793+0000 7f34af7fe640 1 --2- 172.21.15.103:0/799530848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a640ecb0 0x5561a641cdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.793+0000 7f34af7fe640 1 -- 172.21.15.103:0/799530848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a6419e90 msgr2=0x5561a641d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:30.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.793+0000 7f34af7fe640 1 --2- 172.21.15.103:0/799530848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a6419e90 0x5561a641d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.793+0000 7f34af7fe640 1 -- 172.21.15.103:0/799530848 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561a6492810 con 0x5561a640f5d0 2024-09-19T06:50:30.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.793+0000 7f34b4b77640 1 --2- 172.21.15.103:0/799530848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a6419e90 0x5561a641d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:30.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.793+0000 7f34af7fe640 1 --2- 172.21.15.103:0/799530848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640f5d0 0x5561a641d330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f34a4018b30 tx=0x7f34a4004f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d28382ce261a058 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:30.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.793+0000 7f348f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:50:30.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.794+0000 7f348f7fe640 1 -- 172.21.15.103:0/799530848 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4042020 con 0x5561a640f5d0 2024-09-19T06:50:30.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.794+0000 7f348f7fe640 1 -- 172.21.15.103:0/799530848 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f34a402c640 con 0x5561a640f5d0 2024-09-19T06:50:30.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.794+0000 7f34b89b3740 1 -- 172.21.15.103:0/799530848 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a40047a0 con 0x5561a640f5d0 2024-09-19T06:50:30.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.794+0000 7f348f7fe640 1 -- 172.21.15.103:0/799530848 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f34a4031040 con 0x5561a640f5d0 2024-09-19T06:50:30.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.794+0000 7f34b89b3740 10 client.25334 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:30.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.794+0000 7f34b89b3740 1 -- 172.21.15.103:0/799530848 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5561a6493b00 con 0x5561a640f5d0 2024-09-19T06:50:30.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.794+0000 7f348f7fe640 1 -- 172.21.15.103:0/799530848 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a40047a0 con 0x5561a640f5d0 2024-09-19T06:50:30.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.794+0000 7f348f7fe640 1 -- 172.21.15.103:0/799530848 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f34a4023800 con 0x5561a640f5d0 2024-09-19T06:50:30.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.794+0000 7f34b89b3740 10 client.25334 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:30.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.794+0000 7f34b89b3740 10 client.25334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:30.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.794+0000 7f34b89b3740 20 client.25334 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:30.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.794+0000 7f34b89b3740 1 --2- 172.21.15.103:0/799530848 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5561a64940c0 0x5561a64964e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:30.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.794+0000 7f34b89b3740 4 client.25334 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:50:30.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.794+0000 7f34b89b3740 1 -- 172.21.15.103:0/799530848 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5561a6373d60 con 0x5561a64940c0 2024-09-19T06:50:30.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.795+0000 7f34b4b77640 1 --2- 172.21.15.103:0/799530848 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5561a64940c0 0x5561a64964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:30.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.795+0000 7f348f7fe640 10 client.25334 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:30.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.795+0000 7f34b4b77640 1 --2- 172.21.15.103:0/799530848 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5561a64940c0 0x5561a64964e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eada4f8f99dde91 server_cookie=9e96163ca4c5d14 in_seq=0 out_seq=0 2024-09-19T06:50:30.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.796+0000 7f348f7fe640 1 -- 172.21.15.103:0/799530848 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5561a6373d60 con 0x5561a64940c0 2024-09-19T06:50:30.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.796+0000 7f348f7fe640 10 client.25334 handle_command_reply: tid=0 2024-09-19T06:50:30.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.796+0000 7f34b89b3740 1 client.25334 shutdown 2024-09-19T06:50:30.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.796+0000 7f34b89b3740 1 -- 172.21.15.103:0/799530848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640f5d0 msgr2=0x5561a641d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:30.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.796+0000 7f34b89b3740 1 --2- 172.21.15.103:0/799530848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640f5d0 0x5561a641d330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f34a4018b30 tx=0x7f34a4004f70 comp rx=0 tx=0).stop 2024-09-19T06:50:30.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.796+0000 7f34b89b3740 1 -- 172.21.15.103:0/799530848 shutdown_connections 2024-09-19T06:50:30.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.796+0000 7f34b89b3740 1 --2- 172.21.15.103:0/799530848 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5561a64940c0 0x5561a64964e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.796+0000 7f34b89b3740 1 --2- 172.21.15.103:0/799530848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561a6419e90 0x5561a641d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.796+0000 7f34b89b3740 1 --2- 172.21.15.103:0/799530848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561a640f5d0 0x5561a641d330 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.796+0000 7f34b89b3740 1 --2- 172.21.15.103:0/799530848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561a640ecb0 0x5561a641cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.796+0000 7f34b89b3740 1 -- 172.21.15.103:0/799530848 >> 172.21.15.103:0/799530848 conn(0x5561a640b240 msgr2=0x5561a6418240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:30.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.796+0000 7f348f7fe640 0 client.25334 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:30.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.796+0000 7f34b89b3740 1 -- 172.21.15.103:0/799530848 shutdown_connections 2024-09-19T06:50:30.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.796+0000 7f34b89b3740 1 -- 172.21.15.103:0/799530848 wait complete. 2024-09-19T06:50:30.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.796+0000 7f34b89b3740 20 client.25334 trim_cache size 0 max 16384 2024-09-19T06:50:30.802 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:50:30.802 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T06:50:30.802 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T06:50:30.802 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:50:30.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.797+0000 7f34b65da640 1 -- 172.21.15.103:0/3809877005 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34980a35e0 msgr2=0x7f34980a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:30.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.797+0000 7f34b65da640 1 --2- 172.21.15.103:0/3809877005 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34980a35e0 0x7f34980a5aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f34a0011370 tx=0x7f34a009bfe0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.797+0000 7f34b65da640 1 -- 172.21.15.103:0/3809877005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34b0101df0 msgr2=0x7f34b0198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:30.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.797+0000 7f34b65da640 1 --2- 172.21.15.103:0/3809877005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34b0101df0 0x7f34b0198370 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f34a000af00 tx=0x7f34a000cd40 comp rx=0 tx=0).stop 2024-09-19T06:50:30.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.797+0000 7f34b65da640 1 -- 172.21.15.103:0/3809877005 shutdown_connections 2024-09-19T06:50:30.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.797+0000 7f34b65da640 1 --2- 172.21.15.103:0/3809877005 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34980a35e0 0x7f34980a5aa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.797+0000 7f34b65da640 1 --2- 172.21.15.103:0/3809877005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34b0106f20 0x7f34b019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.797+0000 7f34b65da640 1 --2- 172.21.15.103:0/3809877005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34b01027a0 0x7f34b01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.797+0000 7f34b65da640 1 --2- 172.21.15.103:0/3809877005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34b0101df0 0x7f34b0198370 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:30.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.797+0000 7f34b65da640 1 -- 172.21.15.103:0/3809877005 >> 172.21.15.103:0/3809877005 conn(0x7f34b00fd8f0 msgr2=0x7f34b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:30.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.798+0000 7f34b65da640 1 -- 172.21.15.103:0/3809877005 shutdown_connections 2024-09-19T06:50:30.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:30.798+0000 7f34b65da640 1 -- 172.21.15.103:0/3809877005 wait complete. 2024-09-19T06:50:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.029+0000 7f43d776f640 1 -- 172.21.15.103:0/4077938720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43d0105520 msgr2=0x7f43d010d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.029+0000 7f43d776f640 1 --2- 172.21.15.103:0/4077938720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43d0105520 0x7f43d010d070 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f43c8005fd0 tx=0x7f43c802a910 comp rx=0 tx=0).stop 2024-09-19T06:50:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.029+0000 7f43d776f640 1 -- 172.21.15.103:0/4077938720 shutdown_connections 2024-09-19T06:50:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.029+0000 7f43d776f640 1 --2- 172.21.15.103:0/4077938720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43d0105520 0x7f43d010d070 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.029+0000 7f43d776f640 1 --2- 172.21.15.103:0/4077938720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43d01011e0 0x7f43d0104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.029+0000 7f43d776f640 1 --2- 172.21.15.103:0/4077938720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43d0100830 0x7f43d0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.029+0000 7f43d776f640 1 -- 172.21.15.103:0/4077938720 >> 172.21.15.103:0/4077938720 conn(0x7f43d0075f20 msgr2=0x7f43d0076330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.029+0000 7f43d776f640 1 -- 172.21.15.103:0/4077938720 shutdown_connections 2024-09-19T06:50:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.029+0000 7f43d776f640 1 -- 172.21.15.103:0/4077938720 wait complete. 2024-09-19T06:50:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.030+0000 7f43d776f640 1 Processor -- start 2024-09-19T06:50:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.030+0000 7f43d776f640 1 -- start start 2024-09-19T06:50:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.031+0000 7f43d776f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43d0100830 0x7f43d0195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.031+0000 7f43d776f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43d01011e0 0x7f43d0196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.031+0000 7f43d4d0a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43d01011e0 0x7f43d0196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.031+0000 7f43d550b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43d0100830 0x7f43d0195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.031+0000 7f43d550b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43d0100830 0x7f43d0195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:32930/0 (socket says 172.21.15.103:32930) 2024-09-19T06:50:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.031+0000 7f43d550b640 1 -- 172.21.15.103:0/2374135194 learned_addr learned my addr 172.21.15.103:0/2374135194 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.031+0000 7f43d4d0a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43d01011e0 0x7f43d0196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48130/0 (socket says 172.21.15.103:48130) 2024-09-19T06:50:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.031+0000 7f43d776f640 1 --2- 172.21.15.103:0/2374135194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43d0105520 0x7f43d019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.031+0000 7f43d776f640 1 -- 172.21.15.103:0/2374135194 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f43d010fd00 con 0x7f43d01011e0 2024-09-19T06:50:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.031+0000 7f43d776f640 1 -- 172.21.15.103:0/2374135194 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43d010fb80 con 0x7f43d0100830 2024-09-19T06:50:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.031+0000 7f43d776f640 1 -- 172.21.15.103:0/2374135194 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f43d010fe80 con 0x7f43d0105520 2024-09-19T06:50:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.032+0000 7f43d5d0c640 1 --2- 172.21.15.103:0/2374135194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43d0105520 0x7f43d019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.032+0000 7f43d550b640 1 -- 172.21.15.103:0/2374135194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43d0105520 msgr2=0x7f43d019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.032+0000 7f43d550b640 1 --2- 172.21.15.103:0/2374135194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43d0105520 0x7f43d019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.032+0000 7f43d550b640 1 -- 172.21.15.103:0/2374135194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43d01011e0 msgr2=0x7f43d0196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.032+0000 7f43d550b640 1 --2- 172.21.15.103:0/2374135194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43d01011e0 0x7f43d0196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.032+0000 7f43d550b640 1 -- 172.21.15.103:0/2374135194 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d019af00 con 0x7f43d0100830 2024-09-19T06:50:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.032+0000 7f43d4d0a640 1 --2- 172.21.15.103:0/2374135194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43d01011e0 0x7f43d0196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.032+0000 7f43d550b640 1 --2- 172.21.15.103:0/2374135194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43d0100830 0x7f43d0195f10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f43cc010ad0 tx=0x7f43cc010f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.032+0000 7f43c67fc640 1 -- 172.21.15.103:0/2374135194 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc01a020 con 0x7f43d0100830 2024-09-19T06:50:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.032+0000 7f43d776f640 1 -- 172.21.15.103:0/2374135194 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d019b1f0 con 0x7f43d0100830 2024-09-19T06:50:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.033+0000 7f43d776f640 1 -- 172.21.15.103:0/2374135194 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d01a2a80 con 0x7f43d0100830 2024-09-19T06:50:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.033+0000 7f43c67fc640 1 -- 172.21.15.103:0/2374135194 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43cc00ec30 con 0x7f43d0100830 2024-09-19T06:50:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.034+0000 7f43c67fc640 1 -- 172.21.15.103:0/2374135194 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc01f660 con 0x7f43d0100830 2024-09-19T06:50:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.036+0000 7f43da5e1740 1 Processor -- start 2024-09-19T06:50:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.036+0000 7f43da5e1740 1 -- start start 2024-09-19T06:50:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.036+0000 7f43da5e1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6aaf0 0x56123ae6aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.036+0000 7f43da5e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6b410 0x56123ae75770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.036+0000 7f43da5e1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae780a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.036+0000 7f43da5e1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56123adcf2f0 con 0x56123ae6aaf0 2024-09-19T06:50:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.036+0000 7f43da5e1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56123ad77270 con 0x56123ae6b410 2024-09-19T06:50:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.036+0000 7f43da5e1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56123adcb2b0 con 0x56123ae75cb0 2024-09-19T06:50:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.036+0000 7f43c67fc640 1 -- 172.21.15.103:0/2374135194 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7f43cc01d020 con 0x7f43d0100830 2024-09-19T06:50:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.036+0000 7f43d5d0c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6aaf0 0x56123ae6aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.037+0000 7f43d5d0c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6aaf0 0x56123ae6aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48132/0 (socket says 172.21.15.103:48132) 2024-09-19T06:50:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.037+0000 7f43d5d0c640 1 -- 172.21.15.103:0/1225179596 learned_addr learned my addr 172.21.15.103:0/1225179596 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.037+0000 7f43d4d0a640 1 --2- 172.21.15.103:0/1225179596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.037+0000 7f43d550b640 1 --2- 172.21.15.103:0/1225179596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6b410 0x56123ae75770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.037+0000 7f43d4d0a640 1 -- 172.21.15.103:0/1225179596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6b410 msgr2=0x56123ae75770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.037+0000 7f43d4d0a640 1 --2- 172.21.15.103:0/1225179596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6b410 0x56123ae75770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.037+0000 7f43d4d0a640 1 -- 172.21.15.103:0/1225179596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6aaf0 msgr2=0x56123ae6aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.037+0000 7f43d4d0a640 1 --2- 172.21.15.103:0/1225179596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6aaf0 0x56123ae6aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.037+0000 7f43d4d0a640 1 -- 172.21.15.103:0/1225179596 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56123ad76340 con 0x56123ae75cb0 2024-09-19T06:50:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.037+0000 7f43d5d0c640 1 --2- 172.21.15.103:0/1225179596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6aaf0 0x56123ae6aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.037+0000 7f43d4d0a640 1 --2- 172.21.15.103:0/1225179596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae780a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f43c00058c0 tx=0x7f43c000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93ccc0b1ffcd2c9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.037+0000 7f43c67fc640 1 --2- 172.21.15.103:0/2374135194 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f43bc0a3660 0x7f43bc0a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.037+0000 7f43c67fc640 1 -- 172.21.15.103:0/2374135194 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f43cc008f30 con 0x7f43d0100830 2024-09-19T06:50:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.038+0000 7f43d5d0c640 1 --2- 172.21.15.103:0/2374135194 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f43bc0a3660 0x7f43bc0a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.038+0000 7f43d5d0c640 1 --2- 172.21.15.103:0/2374135194 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f43bc0a3660 0x7f43bc0a5b20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f43c802ae80 tx=0x7f43c800ac00 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.038+0000 7f43bb7fe640 1 -- 172.21.15.103:0/1225179596 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c001d070 con 0x56123ae75cb0 2024-09-19T06:50:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.038+0000 7f43bb7fe640 1 -- 172.21.15.103:0/1225179596 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43c0010d20 con 0x56123ae75cb0 2024-09-19T06:50:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.038+0000 7f43bb7fe640 1 -- 172.21.15.103:0/1225179596 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0018e30 con 0x56123ae75cb0 2024-09-19T06:50:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.039+0000 7f43da5e1740 1 -- 172.21.15.103:0/1225179596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 msgr2=0x56123ae780a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.039+0000 7f43da5e1740 1 --2- 172.21.15.103:0/1225179596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae780a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f43c00058c0 tx=0x7f43c000f790 comp rx=0 tx=0).stop 2024-09-19T06:50:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.039+0000 7f43da5e1740 1 -- 172.21.15.103:0/1225179596 shutdown_connections 2024-09-19T06:50:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.039+0000 7f43da5e1740 1 --2- 172.21.15.103:0/1225179596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae780a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.039+0000 7f43da5e1740 1 --2- 172.21.15.103:0/1225179596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6b410 0x56123ae75770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.039+0000 7f43da5e1740 1 --2- 172.21.15.103:0/1225179596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6aaf0 0x56123ae6aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.039+0000 7f43da5e1740 1 -- 172.21.15.103:0/1225179596 >> 172.21.15.103:0/1225179596 conn(0x56123ae67060 msgr2=0x56123ae67440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.039+0000 7f43da5e1740 1 -- 172.21.15.103:0/1225179596 shutdown_connections 2024-09-19T06:50:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.039+0000 7f43da5e1740 1 -- 172.21.15.103:0/1225179596 wait complete. 2024-09-19T06:50:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.040+0000 7f43da5e1740 1 -- start start 2024-09-19T06:50:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.040+0000 7f43da5e1740 1 Processor -- start 2024-09-19T06:50:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.040+0000 7f43da5e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6aaf0 0x56123ae72bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.040+0000 7f43da5e1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6b410 0x56123ae730f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.040+0000 7f43da5e1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae73630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.040+0000 7f43da5e1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56123ae7cc90 con 0x56123ae6b410 2024-09-19T06:50:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.040+0000 7f43da5e1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56123ad58290 con 0x56123ae6aaf0 2024-09-19T06:50:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.040+0000 7f43da5e1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56123ae7ce10 con 0x56123ae75cb0 2024-09-19T06:50:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.040+0000 7f43da5e1740 10 client.0 _ll_register_callbacks cb 0x56123ad89140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.040+0000 7f43d4d0a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6b410 0x56123ae730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.040+0000 7f43d550b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6aaf0 0x56123ae72bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.040+0000 7f43d4d0a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6b410 0x56123ae730f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48134/0 (socket says 172.21.15.103:48134) 2024-09-19T06:50:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.040+0000 7f43d4d0a640 1 -- 172.21.15.103:0/4018264024 learned_addr learned my addr 172.21.15.103:0/4018264024 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.041+0000 7f43d5d0c640 1 --2- 172.21.15.103:0/4018264024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae73630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.041+0000 7f43d550b640 1 -- 172.21.15.103:0/4018264024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 msgr2=0x56123ae73630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.041+0000 7f43d550b640 1 --2- 172.21.15.103:0/4018264024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae73630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.041+0000 7f43d550b640 1 -- 172.21.15.103:0/4018264024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6b410 msgr2=0x56123ae730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.041+0000 7f43d550b640 1 --2- 172.21.15.103:0/4018264024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6b410 0x56123ae730f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.041+0000 7f43d550b640 1 -- 172.21.15.103:0/4018264024 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56123aae2210 con 0x56123ae6aaf0 2024-09-19T06:50:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.041+0000 7f43d5d0c640 1 --2- 172.21.15.103:0/4018264024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae73630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.041+0000 7f43d4d0a640 1 --2- 172.21.15.103:0/4018264024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6b410 0x56123ae730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.041+0000 7f43d550b640 1 --2- 172.21.15.103:0/4018264024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6aaf0 0x56123ae72bb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f43cc008570 tx=0x7f43cc09e1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bbc6b9e2f8afe7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.041+0000 7f43b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:50:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.041+0000 7f43b8ff9640 1 -- 172.21.15.103:0/4018264024 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0b7060 con 0x56123ae6aaf0 2024-09-19T06:50:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.041+0000 7f43da5e1740 1 -- 172.21.15.103:0/4018264024 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43c0010d20 con 0x56123ae6aaf0 2024-09-19T06:50:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.041+0000 7f43b8ff9640 1 -- 172.21.15.103:0/4018264024 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f43cc0b2bc0 con 0x56123ae6aaf0 2024-09-19T06:50:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.041+0000 7f43b8ff9640 1 -- 172.21.15.103:0/4018264024 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43cc00ede0 con 0x56123ae6aaf0 2024-09-19T06:50:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.041+0000 7f43da5e1740 10 client.25343 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.041+0000 7f43da5e1740 1 -- 172.21.15.103:0/4018264024 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56123abae620 con 0x56123ae6aaf0 2024-09-19T06:50:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.042+0000 7f43b8ff9640 1 -- 172.21.15.103:0/4018264024 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43c0010d20 con 0x56123ae6aaf0 2024-09-19T06:50:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.042+0000 7f43b8ff9640 1 -- 172.21.15.103:0/4018264024 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f43cc0692b0 con 0x56123ae6aaf0 2024-09-19T06:50:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.042+0000 7f43da5e1740 10 client.25343 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.042+0000 7f43da5e1740 10 client.25343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.042+0000 7f43da5e1740 20 client.25343 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.042+0000 7f43da5e1740 1 --2- 172.21.15.103:0/4018264024 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56123ae8a030 0x56123ae8c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.042+0000 7f43da5e1740 4 client.25343 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.042+0000 7f43da5e1740 1 -- 172.21.15.103:0/4018264024 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56123addca20 con 0x56123ae8a030 2024-09-19T06:50:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.042+0000 7f43d4d0a640 1 --2- 172.21.15.103:0/4018264024 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56123ae8a030 0x56123ae8c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.043+0000 7f43b8ff9640 10 client.25343 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.043+0000 7f43d4d0a640 1 --2- 172.21.15.103:0/4018264024 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56123ae8a030 0x56123ae8c420 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52aed129db924f8d server_cookie=9a3bffd5c944df9c in_seq=0 out_seq=0 2024-09-19T06:50:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.043+0000 7f43b8ff9640 1 -- 172.21.15.103:0/4018264024 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56123addca20 con 0x56123ae8a030 2024-09-19T06:50:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.043+0000 7f43b8ff9640 10 client.25343 handle_command_reply: tid=0 2024-09-19T06:50:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.043+0000 7f43da5e1740 1 client.25343 shutdown 2024-09-19T06:50:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.043+0000 7f43da5e1740 1 -- 172.21.15.103:0/4018264024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6aaf0 msgr2=0x56123ae72bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.043+0000 7f43da5e1740 1 --2- 172.21.15.103:0/4018264024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6aaf0 0x56123ae72bb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f43cc008570 tx=0x7f43cc09e1e0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.043+0000 7f43da5e1740 1 -- 172.21.15.103:0/4018264024 shutdown_connections 2024-09-19T06:50:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.043+0000 7f43da5e1740 1 --2- 172.21.15.103:0/4018264024 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56123ae8a030 0x56123ae8c420 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.043+0000 7f43da5e1740 1 --2- 172.21.15.103:0/4018264024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae73630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.043+0000 7f43da5e1740 1 --2- 172.21.15.103:0/4018264024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6b410 0x56123ae730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.043+0000 7f43b8ff9640 0 client.25343 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.043+0000 7f43da5e1740 1 --2- 172.21.15.103:0/4018264024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6aaf0 0x56123ae72bb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.044+0000 7f43da5e1740 1 -- 172.21.15.103:0/4018264024 >> 172.21.15.103:0/4018264024 conn(0x56123ae67060 msgr2=0x56123ae6cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.044+0000 7f43da5e1740 1 -- 172.21.15.103:0/4018264024 shutdown_connections 2024-09-19T06:50:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.044+0000 7f43da5e1740 1 -- 172.21.15.103:0/4018264024 wait complete. 2024-09-19T06:50:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.044+0000 7f43da5e1740 20 client.25343 trim_cache size 0 max 16384 2024-09-19T06:50:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.058+0000 7f43da5e1740 1 Processor -- start 2024-09-19T06:50:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.058+0000 7f43da5e1740 1 -- start start 2024-09-19T06:50:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.058+0000 7f43da5e1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6aaf0 0x56123ae6d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.058+0000 7f43da5e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6b410 0x56123ae74b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.058+0000 7f43da5e1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae91c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.058+0000 7f43da5e1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01ab5e0 con 0x56123ae6aaf0 2024-09-19T06:50:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.058+0000 7f43da5e1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01ad120 con 0x56123ae6b410 2024-09-19T06:50:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.058+0000 7f43da5e1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01abcb0 con 0x56123ae75cb0 2024-09-19T06:50:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.058+0000 7f43d550b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6aaf0 0x56123ae6d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.059+0000 7f43d550b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6aaf0 0x56123ae6d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48146/0 (socket says 172.21.15.103:48146) 2024-09-19T06:50:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.059+0000 7f43d550b640 1 -- 172.21.15.103:0/2164706652 learned_addr learned my addr 172.21.15.103:0/2164706652 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.059+0000 7f43d4d0a640 1 --2- 172.21.15.103:0/2164706652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6b410 0x56123ae74b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.059+0000 7f43d5d0c640 1 --2- 172.21.15.103:0/2164706652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae91c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.059+0000 7f43d550b640 1 -- 172.21.15.103:0/2164706652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 msgr2=0x56123ae91c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.059+0000 7f43d550b640 1 --2- 172.21.15.103:0/2164706652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae91c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.059+0000 7f43d550b640 1 -- 172.21.15.103:0/2164706652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6b410 msgr2=0x56123ae74b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.059+0000 7f43d550b640 1 --2- 172.21.15.103:0/2164706652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6b410 0x56123ae74b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.059+0000 7f43d550b640 1 -- 172.21.15.103:0/2164706652 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56123ae6b240 con 0x56123ae6aaf0 2024-09-19T06:50:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.059+0000 7f43d4d0a640 1 --2- 172.21.15.103:0/2164706652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6b410 0x56123ae74b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.059+0000 7f43d550b640 1 --2- 172.21.15.103:0/2164706652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6aaf0 0x56123ae6d5b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f43cc09dbb0 tx=0x7f43cc016c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb907a1b266861b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.059+0000 7f43baffd640 1 -- 172.21.15.103:0/2164706652 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0b7070 con 0x56123ae6aaf0 2024-09-19T06:50:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.059+0000 7f43baffd640 1 -- 172.21.15.103:0/2164706652 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43cc0a7310 con 0x56123ae6aaf0 2024-09-19T06:50:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.059+0000 7f43baffd640 1 -- 172.21.15.103:0/2164706652 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0a2a50 con 0x56123ae6aaf0 2024-09-19T06:50:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.060+0000 7f43da5e1740 1 -- 172.21.15.103:0/2164706652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6aaf0 msgr2=0x56123ae6d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.060+0000 7f43da5e1740 1 --2- 172.21.15.103:0/2164706652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6aaf0 0x56123ae6d5b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f43cc09dbb0 tx=0x7f43cc016c20 comp rx=0 tx=0).stop 2024-09-19T06:50:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.060+0000 7f43da5e1740 1 -- 172.21.15.103:0/2164706652 shutdown_connections 2024-09-19T06:50:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.060+0000 7f43da5e1740 1 --2- 172.21.15.103:0/2164706652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae91c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.060+0000 7f43da5e1740 1 --2- 172.21.15.103:0/2164706652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6b410 0x56123ae74b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.060+0000 7f43da5e1740 1 --2- 172.21.15.103:0/2164706652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6aaf0 0x56123ae6d5b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.060+0000 7f43da5e1740 1 -- 172.21.15.103:0/2164706652 >> 172.21.15.103:0/2164706652 conn(0x56123ae67060 msgr2=0x56123ae74410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.060+0000 7f43da5e1740 1 -- 172.21.15.103:0/2164706652 shutdown_connections 2024-09-19T06:50:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.060+0000 7f43da5e1740 1 -- 172.21.15.103:0/2164706652 wait complete. 2024-09-19T06:50:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.061+0000 7f43da5e1740 1 -- start start 2024-09-19T06:50:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.061+0000 7f43da5e1740 1 Processor -- start 2024-09-19T06:50:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.061+0000 7f43da5e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6aaf0 0x56123ae82390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.061+0000 7f43da5e1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6b410 0x56123ae828d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.062+0000 7f43d550b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6aaf0 0x56123ae82390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.062+0000 7f43d4d0a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6b410 0x56123ae828d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.062+0000 7f43d550b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6aaf0 0x56123ae82390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:32968/0 (socket says 172.21.15.103:32968) 2024-09-19T06:50:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.062+0000 7f43d4d0a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6b410 0x56123ae828d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48162/0 (socket says 172.21.15.103:48162) 2024-09-19T06:50:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.062+0000 7f43d550b640 1 -- 172.21.15.103:0/3669104099 learned_addr learned my addr 172.21.15.103:0/3669104099 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.062+0000 7f43da5e1740 1 --2- 172.21.15.103:0/3669104099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae6f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.062+0000 7f43da5e1740 1 -- 172.21.15.103:0/3669104099 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01aca50 con 0x56123ae6b410 2024-09-19T06:50:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.062+0000 7f43da5e1740 1 -- 172.21.15.103:0/3669104099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01ac380 con 0x56123ae6aaf0 2024-09-19T06:50:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.062+0000 7f43da5e1740 1 -- 172.21.15.103:0/3669104099 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f43d01a95a0 con 0x56123ae75cb0 2024-09-19T06:50:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.062+0000 7f43d5d0c640 1 --2- 172.21.15.103:0/3669104099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae6f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.062+0000 7f43d4d0a640 1 -- 172.21.15.103:0/3669104099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 msgr2=0x56123ae6f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.062+0000 7f43d4d0a640 1 --2- 172.21.15.103:0/3669104099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae6f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.062+0000 7f43d4d0a640 1 -- 172.21.15.103:0/3669104099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6aaf0 msgr2=0x56123ae82390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.062+0000 7f43d4d0a640 1 --2- 172.21.15.103:0/3669104099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6aaf0 0x56123ae82390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.062+0000 7f43d4d0a640 1 -- 172.21.15.103:0/3669104099 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56123ae6fe70 con 0x56123ae6b410 2024-09-19T06:50:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.063+0000 7f43da5e1740 10 client.0 _ll_register_callbacks cb 0x7f43cc00b910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.062+0000 7f43d4d0a640 1 --2- 172.21.15.103:0/3669104099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6b410 0x56123ae828d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f43c0018b10 tx=0x7f43c0018d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd58da0bb372581 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.063+0000 7f43b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.063+0000 7f43da5e1740 1 -- 172.21.15.103:0/3669104099 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43cc0a7310 con 0x56123ae6b410 2024-09-19T06:50:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.063+0000 7f43b8ff9640 1 -- 172.21.15.103:0/3669104099 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c001d040 con 0x56123ae6b410 2024-09-19T06:50:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.063+0000 7f43b8ff9640 1 -- 172.21.15.103:0/3669104099 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f43c00186c0 con 0x56123ae6b410 2024-09-19T06:50:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.063+0000 7f43b8ff9640 1 -- 172.21.15.103:0/3669104099 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43c0018e60 con 0x56123ae6b410 2024-09-19T06:50:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.063+0000 7f43da5e1740 10 client.15948 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.063+0000 7f43da5e1740 1 -- 172.21.15.103:0/3669104099 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=27}) -- 0x56123aef45d0 con 0x56123ae6b410 2024-09-19T06:50:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.063+0000 7f43b8ff9640 1 -- 172.21.15.103:0/3669104099 <== mon.0 v2:172.21.15.103:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f43c0004020 con 0x56123ae6b410 2024-09-19T06:50:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.063+0000 7f43da5e1740 10 client.15948 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.063+0000 7f43da5e1740 10 client.15948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.063+0000 7f43da5e1740 20 client.15948 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.063+0000 7f43da5e1740 1 --2- 172.21.15.103:0/3669104099 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56123aef4b90 0x56123aef7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.063+0000 7f43da5e1740 4 client.15948 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:50:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.063+0000 7f43da5e1740 1 -- 172.21.15.103:0/3669104099 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56123add6d70 con 0x56123aef4b90 2024-09-19T06:50:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.064+0000 7f43d550b640 1 --2- 172.21.15.103:0/3669104099 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56123aef4b90 0x56123aef7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.064+0000 7f43b8ff9640 10 client.15948 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.064+0000 7f43d550b640 1 --2- 172.21.15.103:0/3669104099 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56123aef4b90 0x56123aef7050 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4df367d5967f9ac5 server_cookie=75c6d34bc99ab906 in_seq=0 out_seq=0 2024-09-19T06:50:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.067+0000 7f43b8ff9640 1 -- 172.21.15.103:0/3669104099 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56123add6d70 con 0x56123aef4b90 2024-09-19T06:50:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.067+0000 7f43b8ff9640 10 client.15948 handle_command_reply: tid=0 2024-09-19T06:50:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.067+0000 7f43da5e1740 1 client.15948 shutdown 2024-09-19T06:50:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.068+0000 7f43da5e1740 1 -- 172.21.15.103:0/3669104099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6b410 msgr2=0x56123ae828d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.068+0000 7f43da5e1740 1 --2- 172.21.15.103:0/3669104099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6b410 0x56123ae828d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f43c0018b10 tx=0x7f43c0018d70 comp rx=0 tx=0).stop 2024-09-19T06:50:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.068+0000 7f43da5e1740 1 -- 172.21.15.103:0/3669104099 shutdown_connections 2024-09-19T06:50:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.068+0000 7f43da5e1740 1 --2- 172.21.15.103:0/3669104099 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56123aef4b90 0x56123aef7050 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.068+0000 7f43b8ff9640 0 client.15948 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.068+0000 7f43da5e1740 1 --2- 172.21.15.103:0/3669104099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56123ae75cb0 0x56123ae6f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.068+0000 7f43da5e1740 1 --2- 172.21.15.103:0/3669104099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ae6b410 0x56123ae828d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.068+0000 7f43da5e1740 1 --2- 172.21.15.103:0/3669104099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56123ae6aaf0 0x56123ae82390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.068+0000 7f43da5e1740 1 -- 172.21.15.103:0/3669104099 >> 172.21.15.103:0/3669104099 conn(0x56123ae67060 msgr2=0x56123ae661c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.068+0000 7f43da5e1740 1 -- 172.21.15.103:0/3669104099 shutdown_connections 2024-09-19T06:50:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.068+0000 7f43da5e1740 1 -- 172.21.15.103:0/3669104099 wait complete. 2024-09-19T06:50:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.068+0000 7f43da5e1740 20 client.15948 trim_cache size 0 max 16384 2024-09-19T06:50:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:50:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:50:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:50:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:50:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.069+0000 7f43d776f640 1 -- 172.21.15.103:0/2374135194 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f43bc0a3660 msgr2=0x7f43bc0a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.069+0000 7f43d776f640 1 --2- 172.21.15.103:0/2374135194 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f43bc0a3660 0x7f43bc0a5b20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f43c802ae80 tx=0x7f43c800ac00 comp rx=0 tx=0).stop 2024-09-19T06:50:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.069+0000 7f43d776f640 1 -- 172.21.15.103:0/2374135194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43d0100830 msgr2=0x7f43d0195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.069+0000 7f43d776f640 1 --2- 172.21.15.103:0/2374135194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43d0100830 0x7f43d0195f10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f43cc010ad0 tx=0x7f43cc010f90 comp rx=0 tx=0).stop 2024-09-19T06:50:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.069+0000 7f43d776f640 1 -- 172.21.15.103:0/2374135194 shutdown_connections 2024-09-19T06:50:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.069+0000 7f43d776f640 1 --2- 172.21.15.103:0/2374135194 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f43bc0a3660 0x7f43bc0a5b20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.069+0000 7f43d776f640 1 --2- 172.21.15.103:0/2374135194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43d0105520 0x7f43d019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.069+0000 7f43d776f640 1 --2- 172.21.15.103:0/2374135194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43d01011e0 0x7f43d0196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.069+0000 7f43d776f640 1 --2- 172.21.15.103:0/2374135194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43d0100830 0x7f43d0195f10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.069+0000 7f43d776f640 1 -- 172.21.15.103:0/2374135194 >> 172.21.15.103:0/2374135194 conn(0x7f43d0075f20 msgr2=0x7f43d00fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.069+0000 7f43d776f640 1 -- 172.21.15.103:0/2374135194 shutdown_connections 2024-09-19T06:50:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:31.069+0000 7f43d776f640 1 -- 172.21.15.103:0/2374135194 wait complete. 2024-09-19T06:50:31.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:30 smithi174 ceph-mon[35637]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-19T06:50:31.120 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:30 smithi103 ceph-mon[31158]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-19T06:50:31.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:30 smithi134 ceph-mon[35356]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-19T06:50:31.627 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:50:31.627 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:50:31.944 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:32.317 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:32 smithi103 ceph-mon[31158]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-09-19T06:50:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:32 smithi174 ceph-mon[35637]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-09-19T06:50:32.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.397+0000 7f704c891640 1 -- 172.21.15.103:0/1832792139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f704810e450 msgr2=0x7f7048110840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:32.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.397+0000 7f704c891640 1 --2- 172.21.15.103:0/1832792139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f704810e450 0x7f7048110840 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7040009ca0 tx=0x7f704002a950 comp rx=0 tx=0).stop 2024-09-19T06:50:32.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.397+0000 7f704c891640 1 -- 172.21.15.103:0/1832792139 shutdown_connections 2024-09-19T06:50:32.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.397+0000 7f704c891640 1 --2- 172.21.15.103:0/1832792139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f704810e450 0x7f7048110840 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:32.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.397+0000 7f704c891640 1 --2- 172.21.15.103:0/1832792139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7048074e30 0x7f704810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:32.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.397+0000 7f704c891640 1 --2- 172.21.15.103:0/1832792139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7048074510 0x7f70480748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:32.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.397+0000 7f704c891640 1 -- 172.21.15.103:0/1832792139 >> 172.21.15.103:0/1832792139 conn(0x7f70480fd8f0 msgr2=0x7f70480ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:32.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.398+0000 7f704c891640 1 -- 172.21.15.103:0/1832792139 shutdown_connections 2024-09-19T06:50:32.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.398+0000 7f704c891640 1 -- 172.21.15.103:0/1832792139 wait complete. 2024-09-19T06:50:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.399+0000 7f704c891640 1 Processor -- start 2024-09-19T06:50:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.399+0000 7f704c891640 1 -- start start 2024-09-19T06:50:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.399+0000 7f704c891640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7048074510 0x7f704819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.399+0000 7f704c891640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7048074e30 0x7f704819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.399+0000 7f704c891640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f704810e450 0x7f704819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.399+0000 7f704c891640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70481141c0 con 0x7f7048074510 2024-09-19T06:50:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.399+0000 7f704c891640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7048114040 con 0x7f7048074e30 2024-09-19T06:50:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.399+0000 7f704c891640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7048114340 con 0x7f704810e450 2024-09-19T06:50:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.400+0000 7f7045d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7048074e30 0x7f704819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.400+0000 7f704659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7048074510 0x7f704819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.400+0000 7f7045d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7048074e30 0x7f704819abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:32990/0 (socket says 172.21.15.103:32990) 2024-09-19T06:50:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.400+0000 7f7045d9b640 1 -- 172.21.15.103:0/3089806257 learned_addr learned my addr 172.21.15.103:0/3089806257 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.400+0000 7f7046d9d640 1 --2- 172.21.15.103:0/3089806257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f704810e450 0x7f704819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.400+0000 7f7045d9b640 1 -- 172.21.15.103:0/3089806257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f704810e450 msgr2=0x7f704819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.400+0000 7f7045d9b640 1 --2- 172.21.15.103:0/3089806257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f704810e450 0x7f704819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.400+0000 7f7045d9b640 1 -- 172.21.15.103:0/3089806257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7048074510 msgr2=0x7f704819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.400+0000 7f7045d9b640 1 --2- 172.21.15.103:0/3089806257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7048074510 0x7f704819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.400+0000 7f7045d9b640 1 -- 172.21.15.103:0/3089806257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704819f630 con 0x7f7048074e30 2024-09-19T06:50:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.400+0000 7f704659c640 1 --2- 172.21.15.103:0/3089806257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7048074510 0x7f704819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.400+0000 7f7045d9b640 1 --2- 172.21.15.103:0/3089806257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7048074e30 0x7f704819abc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f703c00b4a0 tx=0x7f703c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.401+0000 7f70377fe640 1 -- 172.21.15.103:0/3089806257 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c014070 con 0x7f7048074e30 2024-09-19T06:50:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.401+0000 7f70377fe640 1 -- 172.21.15.103:0/3089806257 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f703c010af0 con 0x7f7048074e30 2024-09-19T06:50:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.401+0000 7f70377fe640 1 -- 172.21.15.103:0/3089806257 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c017d10 con 0x7f7048074e30 2024-09-19T06:50:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.401+0000 7f704c891640 1 -- 172.21.15.103:0/3089806257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704819f920 con 0x7f7048074e30 2024-09-19T06:50:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.401+0000 7f704c891640 1 -- 172.21.15.103:0/3089806257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70481a7200 con 0x7f7048074e30 2024-09-19T06:50:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.402+0000 7f704c891640 1 -- 172.21.15.103:0/3089806257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70480748f0 con 0x7f7048074e30 2024-09-19T06:50:32.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.404+0000 7f70377fe640 1 -- 172.21.15.103:0/3089806257 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7f703c017070 con 0x7f7048074e30 2024-09-19T06:50:32.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.405+0000 7f70377fe640 1 --2- 172.21.15.103:0/3089806257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70300a3560 0x7f70300a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:32.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.405+0000 7f704659c640 1 --2- 172.21.15.103:0/3089806257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70300a3560 0x7f70300a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:32.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.405+0000 7f70377fe640 1 -- 172.21.15.103:0/3089806257 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f703c0cf1a0 con 0x7f7048074e30 2024-09-19T06:50:32.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.406+0000 7f704659c640 1 --2- 172.21.15.103:0/3089806257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70300a3560 0x7f70300a5a20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f70380058c0 tx=0x7f703800a2d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:32.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.409+0000 7f70377fe640 1 -- 172.21.15.103:0/3089806257 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f703c09b450 con 0x7f7048074e30 2024-09-19T06:50:32.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:32 smithi134 ceph-mon[35356]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-09-19T06:50:32.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.826+0000 7f704c891640 1 -- 172.21.15.103:0/3089806257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7048109040 con 0x7f7048074e30 2024-09-19T06:50:32.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.827+0000 7f70377fe640 1 -- 172.21.15.103:0/3089806257 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f703c09f750 con 0x7f7048074e30 2024-09-19T06:50:32.830 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:50:32.830 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:50:32.830 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:50:32.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.829+0000 7f704c891640 1 -- 172.21.15.103:0/3089806257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70300a3560 msgr2=0x7f70300a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:32.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.829+0000 7f704c891640 1 --2- 172.21.15.103:0/3089806257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70300a3560 0x7f70300a5a20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f70380058c0 tx=0x7f703800a2d0 comp rx=0 tx=0).stop 2024-09-19T06:50:32.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.829+0000 7f704c891640 1 -- 172.21.15.103:0/3089806257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7048074e30 msgr2=0x7f704819abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:32.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.829+0000 7f704c891640 1 --2- 172.21.15.103:0/3089806257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7048074e30 0x7f704819abc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f703c00b4a0 tx=0x7f703c00b960 comp rx=0 tx=0).stop 2024-09-19T06:50:32.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.830+0000 7f704c891640 1 -- 172.21.15.103:0/3089806257 shutdown_connections 2024-09-19T06:50:32.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.830+0000 7f704c891640 1 --2- 172.21.15.103:0/3089806257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70300a3560 0x7f70300a5a20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.830+0000 7f704c891640 1 --2- 172.21.15.103:0/3089806257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f704810e450 0x7f704819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.830+0000 7f704c891640 1 --2- 172.21.15.103:0/3089806257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7048074e30 0x7f704819abc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.830+0000 7f704c891640 1 --2- 172.21.15.103:0/3089806257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7048074510 0x7f704819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.830+0000 7f704c891640 1 -- 172.21.15.103:0/3089806257 >> 172.21.15.103:0/3089806257 conn(0x7f70480fd8f0 msgr2=0x7f70480fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.830+0000 7f704c891640 1 -- 172.21.15.103:0/3089806257 shutdown_connections 2024-09-19T06:50:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:32.830+0000 7f704c891640 1 -- 172.21.15.103:0/3089806257 wait complete. 2024-09-19T06:50:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:33 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3089806257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:33.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:33 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3089806257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:33.441 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:33 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3089806257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:33.610 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:50:33.885 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:34.270 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:34 smithi103 ceph-mon[31158]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-09-19T06:50:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:34 smithi174 ceph-mon[35637]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-09-19T06:50:34.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.354+0000 7fedda8d1640 1 -- 172.21.15.103:0/2832472313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedd4103760 msgr2=0x7fedd4107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:34.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.354+0000 7fedda8d1640 1 --2- 172.21.15.103:0/2832472313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedd4103760 0x7fedd4107ba0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009930 tx=0x7fedbc02a420 comp rx=0 tx=0).stop 2024-09-19T06:50:34.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.354+0000 7fedda8d1640 1 -- 172.21.15.103:0/2832472313 shutdown_connections 2024-09-19T06:50:34.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.354+0000 7fedda8d1640 1 --2- 172.21.15.103:0/2832472313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedd41082e0 0x7fedd410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.354+0000 7fedda8d1640 1 --2- 172.21.15.103:0/2832472313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedd4103760 0x7fedd4107ba0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.354+0000 7fedda8d1640 1 --2- 172.21.15.103:0/2832472313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd4102db0 0x7fedd4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.354+0000 7fedda8d1640 1 -- 172.21.15.103:0/2832472313 >> 172.21.15.103:0/2832472313 conn(0x7fedd40fc7d0 msgr2=0x7fedd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:34.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.354+0000 7fedda8d1640 1 -- 172.21.15.103:0/2832472313 shutdown_connections 2024-09-19T06:50:34.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.354+0000 7fedda8d1640 1 -- 172.21.15.103:0/2832472313 wait complete. 2024-09-19T06:50:34.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.354+0000 7fedda8d1640 1 Processor -- start 2024-09-19T06:50:34.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.354+0000 7fedda8d1640 1 -- start start 2024-09-19T06:50:34.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedda8d1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd4102db0 0x7fedd4100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedda8d1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedd4103760 0x7fedd4100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedda8d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedd41082e0 0x7fedd4101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedda8d1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41120d0 con 0x7fedd4103760 2024-09-19T06:50:34.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedda8d1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fedd4111f50 con 0x7fedd4102db0 2024-09-19T06:50:34.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedda8d1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fedd4112250 con 0x7fedd41082e0 2024-09-19T06:50:34.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedcbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedd4103760 0x7fedd4100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedcbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedd4103760 0x7fedd4100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48186/0 (socket says 172.21.15.103:48186) 2024-09-19T06:50:34.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedcbfff640 1 -- 172.21.15.103:0/1396840870 learned_addr learned my addr 172.21.15.103:0/1396840870 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:34.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedd3fff640 1 --2- 172.21.15.103:0/1396840870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd4102db0 0x7fedd4100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedd8e6e640 1 --2- 172.21.15.103:0/1396840870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedd41082e0 0x7fedd4101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedcbfff640 1 -- 172.21.15.103:0/1396840870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedd41082e0 msgr2=0x7fedd4101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:34.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedcbfff640 1 --2- 172.21.15.103:0/1396840870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedd41082e0 0x7fedd4101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedcbfff640 1 -- 172.21.15.103:0/1396840870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd4102db0 msgr2=0x7fedd4100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:34.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedcbfff640 1 --2- 172.21.15.103:0/1396840870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd4102db0 0x7fedd4100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedcbfff640 1 -- 172.21.15.103:0/1396840870 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd41a6a70 con 0x7fedd4103760 2024-09-19T06:50:34.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedd8e6e640 1 --2- 172.21.15.103:0/1396840870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedd41082e0 0x7fedd4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:34.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedcbfff640 1 --2- 172.21.15.103:0/1396840870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedd4103760 0x7fedd4100f50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009230 tx=0x7fedbc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:34.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.355+0000 7fedd1ffb640 1 -- 172.21.15.103:0/1396840870 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc03f070 con 0x7fedd4103760 2024-09-19T06:50:34.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.356+0000 7fedd1ffb640 1 -- 172.21.15.103:0/1396840870 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedbc031c30 con 0x7fedd4103760 2024-09-19T06:50:34.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.356+0000 7fedda8d1640 1 -- 172.21.15.103:0/1396840870 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd41a6d00 con 0x7fedd4103760 2024-09-19T06:50:34.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.356+0000 7fedd1ffb640 1 -- 172.21.15.103:0/1396840870 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc035030 con 0x7fedd4103760 2024-09-19T06:50:34.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.356+0000 7fedda8d1640 1 -- 172.21.15.103:0/1396840870 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd41a71d0 con 0x7fedd4103760 2024-09-19T06:50:34.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.358+0000 7fedd1ffb640 1 -- 172.21.15.103:0/1396840870 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7fedbc042070 con 0x7fedd4103760 2024-09-19T06:50:34.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.358+0000 7feddccaa740 1 Processor -- start 2024-09-19T06:50:34.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.358+0000 7feddccaa740 1 -- start start 2024-09-19T06:50:34.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7feddccaa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb9b40 0x55c98dcb8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7feddccaa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcb85d0 0x55c98dcb8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7feddccaa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcc3370 0x55c98dcc5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7feddccaa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c98dc1e2f0 con 0x55c98dcb9b40 2024-09-19T06:50:34.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7feddccaa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c98dbc5cf0 con 0x55c98dcb85d0 2024-09-19T06:50:34.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7feddccaa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c98dc1a2b0 con 0x55c98dcc3370 2024-09-19T06:50:34.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7fedd8e6e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb9b40 0x55c98dcb8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7fedd8e6e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb9b40 0x55c98dcb8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48190/0 (socket says 172.21.15.103:48190) 2024-09-19T06:50:34.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7fedd8e6e640 1 -- 172.21.15.103:0/1481703692 learned_addr learned my addr 172.21.15.103:0/1481703692 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:34.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7fedd1ffb640 1 --2- 172.21.15.103:0/1396840870 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fedc00a35e0 0x7fedc00a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7fedd3fff640 1 --2- 172.21.15.103:0/1481703692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcb85d0 0x55c98dcb8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7fedd8e6e640 1 --2- 172.21.15.103:0/1396840870 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fedc00a35e0 0x7fedc00a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7fedcbfff640 1 --2- 172.21.15.103:0/1481703692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcc3370 0x55c98dcc5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7fedd8e6e640 1 -- 172.21.15.103:0/1481703692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcc3370 msgr2=0x55c98dcc5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:34.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7fedd1ffb640 1 -- 172.21.15.103:0/1396840870 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fedbc0f0880 con 0x7fedd4103760 2024-09-19T06:50:34.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7fedd8e6e640 1 --2- 172.21.15.103:0/1481703692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcc3370 0x55c98dcc5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7fedd8e6e640 1 -- 172.21.15.103:0/1481703692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcb85d0 msgr2=0x55c98dcb8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:34.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7fedd8e6e640 1 --2- 172.21.15.103:0/1481703692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcb85d0 0x55c98dcb8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7fedd8e6e640 1 -- 172.21.15.103:0/1481703692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c98dbaa3a0 con 0x55c98dcb9b40 2024-09-19T06:50:34.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7fedd3fff640 1 --2- 172.21.15.103:0/1481703692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcb85d0 0x55c98dcb8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:34.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.359+0000 7fedcbfff640 1 --2- 172.21.15.103:0/1481703692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcc3370 0x55c98dcc5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:34.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.360+0000 7fedd8e6e640 1 --2- 172.21.15.103:0/1481703692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb9b40 0x55c98dcb8090 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00cbf0 tx=0x7fedcc007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74afd49f34a2d48b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:34.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.360+0000 7fedca7fc640 1 -- 172.21.15.103:0/1481703692 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc015070 con 0x55c98dcb9b40 2024-09-19T06:50:34.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.360+0000 7fedca7fc640 1 -- 172.21.15.103:0/1481703692 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedcc016040 con 0x55c98dcb9b40 2024-09-19T06:50:34.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.360+0000 7fedca7fc640 1 -- 172.21.15.103:0/1481703692 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc01e020 con 0x55c98dcb9b40 2024-09-19T06:50:34.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.360+0000 7fedd8e6e640 1 --2- 172.21.15.103:0/1396840870 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fedc00a35e0 0x7fedc00a5aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fedcc01f000 tx=0x7fedcc01b000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:34.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.361+0000 7feddccaa740 1 -- 172.21.15.103:0/1481703692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb9b40 msgr2=0x55c98dcb8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:34.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.361+0000 7feddccaa740 1 --2- 172.21.15.103:0/1481703692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb9b40 0x55c98dcb8090 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00cbf0 tx=0x7fedcc007680 comp rx=0 tx=0).stop 2024-09-19T06:50:34.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.361+0000 7feddccaa740 1 -- 172.21.15.103:0/1481703692 shutdown_connections 2024-09-19T06:50:34.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.361+0000 7feddccaa740 1 --2- 172.21.15.103:0/1481703692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcc3370 0x55c98dcc5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.361+0000 7feddccaa740 1 --2- 172.21.15.103:0/1481703692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcb85d0 0x55c98dcb8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.361+0000 7feddccaa740 1 --2- 172.21.15.103:0/1481703692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb9b40 0x55c98dcb8090 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.361+0000 7feddccaa740 1 -- 172.21.15.103:0/1481703692 >> 172.21.15.103:0/1481703692 conn(0x55c98dc81310 msgr2=0x55c98dcb4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:34.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.361+0000 7feddccaa740 1 -- 172.21.15.103:0/1481703692 shutdown_connections 2024-09-19T06:50:34.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.361+0000 7feddccaa740 1 -- 172.21.15.103:0/1481703692 wait complete. 2024-09-19T06:50:34.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.361+0000 7feddccaa740 1 -- start start 2024-09-19T06:50:34.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.361+0000 7feddccaa740 1 Processor -- start 2024-09-19T06:50:34.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7feddccaa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcb85d0 0x55c98dcc2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7feddccaa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb9b40 0x55c98dcbffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7feddccaa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcc3370 0x55c98dcc0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7feddccaa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c98dcc6070 con 0x55c98dcb9b40 2024-09-19T06:50:34.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7feddccaa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c98dbd9060 con 0x55c98dcc3370 2024-09-19T06:50:34.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7feddccaa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c98dcc61f0 con 0x55c98dcb85d0 2024-09-19T06:50:34.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7feddccaa740 10 client.0 _ll_register_callbacks cb 0x55c98d92c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:34.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7fedcbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb9b40 0x55c98dcbffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7fedcbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb9b40 0x55c98dcbffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48204/0 (socket says 172.21.15.103:48204) 2024-09-19T06:50:34.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7fedcbfff640 1 -- 172.21.15.103:0/552226383 learned_addr learned my addr 172.21.15.103:0/552226383 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:34.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7fedd3fff640 1 --2- 172.21.15.103:0/552226383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcb85d0 0x55c98dcc2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7fedd8e6e640 1 --2- 172.21.15.103:0/552226383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcc3370 0x55c98dcc0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7fedcbfff640 1 -- 172.21.15.103:0/552226383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcb85d0 msgr2=0x55c98dcc2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:34.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7fedcbfff640 1 --2- 172.21.15.103:0/552226383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcb85d0 0x55c98dcc2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7fedcbfff640 1 -- 172.21.15.103:0/552226383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcc3370 msgr2=0x55c98dcc0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:34.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7fedcbfff640 1 --2- 172.21.15.103:0/552226383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcc3370 0x55c98dcc0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7fedcbfff640 1 -- 172.21.15.103:0/552226383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c98d98f3c0 con 0x55c98dcb9b40 2024-09-19T06:50:34.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7fedd8e6e640 1 --2- 172.21.15.103:0/552226383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcc3370 0x55c98dcc0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:34.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7fedcbfff640 1 --2- 172.21.15.103:0/552226383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb9b40 0x55c98dcbffd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0bc630 tx=0x7fedbc0bc7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343e64ad878e9b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:34.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7fedb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:34.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7fedb37fe640 1 -- 172.21.15.103:0/552226383 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc049030 con 0x55c98dcb9b40 2024-09-19T06:50:34.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7fedb37fe640 1 -- 172.21.15.103:0/552226383 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fedbc0bcd90 con 0x55c98dcb9b40 2024-09-19T06:50:34.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.362+0000 7feddccaa740 1 -- 172.21.15.103:0/552226383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedcc016040 con 0x55c98dcb9b40 2024-09-19T06:50:34.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.363+0000 7fedb37fe640 1 -- 172.21.15.103:0/552226383 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fedbc0a70f0 con 0x55c98dcb9b40 2024-09-19T06:50:34.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.363+0000 7feddccaa740 10 client.15966 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:34.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.363+0000 7feddccaa740 1 -- 172.21.15.103:0/552226383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c98dcd1a30 con 0x55c98dcb9b40 2024-09-19T06:50:34.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.363+0000 7fedb37fe640 1 -- 172.21.15.103:0/552226383 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedbc0d0e50 con 0x55c98dcb9b40 2024-09-19T06:50:34.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.363+0000 7fedb37fe640 1 -- 172.21.15.103:0/552226383 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fedbc0c5110 con 0x55c98dcb9b40 2024-09-19T06:50:34.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.363+0000 7feddccaa740 10 client.15966 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:34.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.363+0000 7feddccaa740 10 client.15966 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:50:34.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.363+0000 7feddccaa740 20 client.15966 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:34.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.363+0000 7feddccaa740 1 --2- 172.21.15.103:0/552226383 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55c98dcd2d80 0x55c98dcd5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.363+0000 7feddccaa740 4 client.15966 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:34.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.363+0000 7feddccaa740 1 -- 172.21.15.103:0/552226383 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c98dc2b860 con 0x55c98dcd2d80 2024-09-19T06:50:34.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.363+0000 7fedd3fff640 1 --2- 172.21.15.103:0/552226383 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55c98dcd2d80 0x55c98dcd5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.364+0000 7fedb37fe640 10 client.15966 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:50:34.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.364+0000 7fedd3fff640 1 --2- 172.21.15.103:0/552226383 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55c98dcd2d80 0x55c98dcd5190 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=56b712c7bf89255c server_cookie=29a95d8135ceb852 in_seq=0 out_seq=0 2024-09-19T06:50:34.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.365+0000 7fedb37fe640 1 -- 172.21.15.103:0/552226383 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c98dc2b860 con 0x55c98dcd2d80 2024-09-19T06:50:34.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.365+0000 7fedb37fe640 10 client.15966 handle_command_reply: tid=0 2024-09-19T06:50:34.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.365+0000 7feddccaa740 1 client.15966 shutdown 2024-09-19T06:50:34.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.365+0000 7feddccaa740 1 -- 172.21.15.103:0/552226383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb9b40 msgr2=0x55c98dcbffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:34.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.365+0000 7feddccaa740 1 --2- 172.21.15.103:0/552226383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb9b40 0x55c98dcbffd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0bc630 tx=0x7fedbc0bc7f0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.365+0000 7feddccaa740 1 -- 172.21.15.103:0/552226383 shutdown_connections 2024-09-19T06:50:34.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.365+0000 7feddccaa740 1 --2- 172.21.15.103:0/552226383 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55c98dcd2d80 0x55c98dcd5190 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.365+0000 7fedb37fe640 0 client.15966 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:50:34.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.365+0000 7feddccaa740 1 --2- 172.21.15.103:0/552226383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcc3370 0x55c98dcc0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.365+0000 7feddccaa740 1 --2- 172.21.15.103:0/552226383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb9b40 0x55c98dcbffd0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.365+0000 7feddccaa740 1 --2- 172.21.15.103:0/552226383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcb85d0 0x55c98dcc2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.365+0000 7feddccaa740 1 -- 172.21.15.103:0/552226383 >> 172.21.15.103:0/552226383 conn(0x55c98dc81310 msgr2=0x55c98dcba930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:34.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.365+0000 7feddccaa740 1 -- 172.21.15.103:0/552226383 shutdown_connections 2024-09-19T06:50:34.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.365+0000 7feddccaa740 1 -- 172.21.15.103:0/552226383 wait complete. 2024-09-19T06:50:34.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.365+0000 7feddccaa740 20 client.15966 trim_cache size 0 max 16384 2024-09-19T06:50:34.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.378+0000 7feddccaa740 1 Processor -- start 2024-09-19T06:50:34.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.378+0000 7feddccaa740 1 -- start start 2024-09-19T06:50:34.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.378+0000 7feddccaa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb85d0 0x55c98dccf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.378+0000 7feddccaa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcc3370 0x55c98dcbb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.378+0000 7feddccaa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcd2d80 0x55c98dcbb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.378+0000 7feddccaa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41afa10 con 0x55c98dcb85d0 2024-09-19T06:50:34.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.378+0000 7feddccaa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b1550 con 0x55c98dcc3370 2024-09-19T06:50:34.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.378+0000 7feddccaa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b00e0 con 0x55c98dcd2d80 2024-09-19T06:50:34.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.379+0000 7fedd3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb85d0 0x55c98dccf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.379+0000 7fedd3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb85d0 0x55c98dccf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48210/0 (socket says 172.21.15.103:48210) 2024-09-19T06:50:34.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.379+0000 7fedd3fff640 1 -- 172.21.15.103:0/1063561892 learned_addr learned my addr 172.21.15.103:0/1063561892 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:34.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.379+0000 7fedcbfff640 1 --2- 172.21.15.103:0/1063561892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcc3370 0x55c98dcbb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.379+0000 7fedd8e6e640 1 --2- 172.21.15.103:0/1063561892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcd2d80 0x55c98dcbb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.379+0000 7fedd3fff640 1 -- 172.21.15.103:0/1063561892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcd2d80 msgr2=0x55c98dcbb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:34.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.379+0000 7fedd3fff640 1 --2- 172.21.15.103:0/1063561892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcd2d80 0x55c98dcbb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.379+0000 7fedd3fff640 1 -- 172.21.15.103:0/1063561892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcc3370 msgr2=0x55c98dcbb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:34.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.379+0000 7fedd3fff640 1 --2- 172.21.15.103:0/1063561892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcc3370 0x55c98dcbb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.379+0000 7fedd3fff640 1 -- 172.21.15.103:0/1063561892 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c98dcb4ed0 con 0x55c98dcb85d0 2024-09-19T06:50:34.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.379+0000 7fedd3fff640 1 --2- 172.21.15.103:0/1063561892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb85d0 0x55c98dccf310 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x55c98dcd5a00 tx=0x7fedc400a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edde955fa8ddcc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:34.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.379+0000 7fedc9ffb640 1 -- 172.21.15.103:0/1063561892 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc401e600 con 0x55c98dcb85d0 2024-09-19T06:50:34.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.379+0000 7fedc9ffb640 1 -- 172.21.15.103:0/1063561892 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc401ec80 con 0x55c98dcb85d0 2024-09-19T06:50:34.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.379+0000 7fedc9ffb640 1 -- 172.21.15.103:0/1063561892 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4026df0 con 0x55c98dcb85d0 2024-09-19T06:50:34.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.380+0000 7feddccaa740 1 -- 172.21.15.103:0/1063561892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb85d0 msgr2=0x55c98dccf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:34.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.380+0000 7feddccaa740 1 --2- 172.21.15.103:0/1063561892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb85d0 0x55c98dccf310 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x55c98dcd5a00 tx=0x7fedc400a2d0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.380+0000 7feddccaa740 1 -- 172.21.15.103:0/1063561892 shutdown_connections 2024-09-19T06:50:34.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.380+0000 7feddccaa740 1 --2- 172.21.15.103:0/1063561892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcd2d80 0x55c98dcbb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.380+0000 7feddccaa740 1 --2- 172.21.15.103:0/1063561892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcc3370 0x55c98dcbb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.380+0000 7feddccaa740 1 --2- 172.21.15.103:0/1063561892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcb85d0 0x55c98dccf310 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.380+0000 7feddccaa740 1 -- 172.21.15.103:0/1063561892 >> 172.21.15.103:0/1063561892 conn(0x55c98dc81310 msgr2=0x55c98dcbf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:34.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.380+0000 7feddccaa740 1 -- 172.21.15.103:0/1063561892 shutdown_connections 2024-09-19T06:50:34.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.380+0000 7feddccaa740 1 -- 172.21.15.103:0/1063561892 wait complete. 2024-09-19T06:50:34.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.381+0000 7feddccaa740 1 -- start start 2024-09-19T06:50:34.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.381+0000 7feddccaa740 1 Processor -- start 2024-09-19T06:50:34.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.381+0000 7feddccaa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcb85d0 0x55c98dcc7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.381+0000 7feddccaa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcc3370 0x55c98dcc7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.381+0000 7feddccaa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcd2d80 0x55c98dcc7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.381+0000 7feddccaa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b0e80 con 0x55c98dcc3370 2024-09-19T06:50:34.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.381+0000 7feddccaa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b07b0 con 0x55c98dcb85d0 2024-09-19T06:50:34.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.381+0000 7feddccaa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41ad9d0 con 0x55c98dcd2d80 2024-09-19T06:50:34.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.381+0000 7feddccaa740 10 client.0 _ll_register_callbacks cb 0x7fedbc009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:34.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.381+0000 7fedcbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcc3370 0x55c98dcc7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.381+0000 7fedcbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcc3370 0x55c98dcc7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48224/0 (socket says 172.21.15.103:48224) 2024-09-19T06:50:34.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.381+0000 7fedcbfff640 1 -- 172.21.15.103:0/2793860171 learned_addr learned my addr 172.21.15.103:0/2793860171 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:34.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.381+0000 7fedd3fff640 1 --2- 172.21.15.103:0/2793860171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcb85d0 0x55c98dcc7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.381+0000 7fedd8e6e640 1 --2- 172.21.15.103:0/2793860171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcd2d80 0x55c98dcc7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.382+0000 7fedcbfff640 1 -- 172.21.15.103:0/2793860171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcd2d80 msgr2=0x55c98dcc7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:34.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.382+0000 7fedcbfff640 1 --2- 172.21.15.103:0/2793860171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcd2d80 0x55c98dcc7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.382+0000 7fedcbfff640 1 -- 172.21.15.103:0/2793860171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcb85d0 msgr2=0x55c98dcc7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:34.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.382+0000 7fedcbfff640 1 --2- 172.21.15.103:0/2793860171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcb85d0 0x55c98dcc7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.382+0000 7fedcbfff640 1 -- 172.21.15.103:0/2793860171 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c98dd3ca30 con 0x55c98dcc3370 2024-09-19T06:50:34.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.382+0000 7fedd8e6e640 1 --2- 172.21.15.103:0/2793860171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcd2d80 0x55c98dcc7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:34.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.382+0000 7fedcbfff640 1 --2- 172.21.15.103:0/2793860171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcc3370 0x55c98dcc7640 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0babe0 tx=0x7fedbc034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b002b38d42ac1123 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:34.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.382+0000 7fedb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:34.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.382+0000 7fedb37fe640 1 -- 172.21.15.103:0/2793860171 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc049070 con 0x55c98dcc3370 2024-09-19T06:50:34.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.382+0000 7fedb37fe640 1 -- 172.21.15.103:0/2793860171 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fedbc0bbd80 con 0x55c98dcc3370 2024-09-19T06:50:34.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.382+0000 7feddccaa740 1 -- 172.21.15.103:0/2793860171 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc401ec80 con 0x55c98dcc3370 2024-09-19T06:50:34.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.382+0000 7fedb37fe640 1 -- 172.21.15.103:0/2793860171 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fedbc0bd030 con 0x55c98dcc3370 2024-09-19T06:50:34.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.382+0000 7feddccaa740 10 client.15972 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:34.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.382+0000 7feddccaa740 1 -- 172.21.15.103:0/2793860171 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c98dd3dc30 con 0x55c98dcc3370 2024-09-19T06:50:34.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.382+0000 7fedb37fe640 1 -- 172.21.15.103:0/2793860171 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc401ec80 con 0x55c98dcc3370 2024-09-19T06:50:34.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.383+0000 7fedb37fe640 1 -- 172.21.15.103:0/2793860171 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fedbc0b8680 con 0x55c98dcc3370 2024-09-19T06:50:34.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.383+0000 7feddccaa740 10 client.15972 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:34.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.383+0000 7feddccaa740 10 client.15972 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:50:34.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.383+0000 7feddccaa740 20 client.15972 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:34.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.383+0000 7feddccaa740 1 --2- 172.21.15.103:0/2793860171 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55c98dd3e220 0x55c98dd40640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:34.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.383+0000 7feddccaa740 4 client.15972 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:34.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.383+0000 7feddccaa740 1 -- 172.21.15.103:0/2793860171 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c98dc25d70 con 0x55c98dd3e220 2024-09-19T06:50:34.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.383+0000 7fedd3fff640 1 --2- 172.21.15.103:0/2793860171 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55c98dd3e220 0x55c98dd40640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:34.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.383+0000 7fedb37fe640 10 client.15972 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:50:34.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.383+0000 7fedd3fff640 1 --2- 172.21.15.103:0/2793860171 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55c98dd3e220 0x55c98dd40640 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2e8dc580010fe13 server_cookie=d2cc69b309308f75 in_seq=0 out_seq=0 2024-09-19T06:50:34.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.384+0000 7fedb37fe640 1 -- 172.21.15.103:0/2793860171 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c98dc25d70 con 0x55c98dd3e220 2024-09-19T06:50:34.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.384+0000 7fedb37fe640 10 client.15972 handle_command_reply: tid=0 2024-09-19T06:50:34.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.384+0000 7feddccaa740 1 client.15972 shutdown 2024-09-19T06:50:34.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.384+0000 7feddccaa740 1 -- 172.21.15.103:0/2793860171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcc3370 msgr2=0x55c98dcc7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:34.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.384+0000 7feddccaa740 1 --2- 172.21.15.103:0/2793860171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcc3370 0x55c98dcc7640 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0babe0 tx=0x7fedbc034000 comp rx=0 tx=0).stop 2024-09-19T06:50:34.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.384+0000 7feddccaa740 1 -- 172.21.15.103:0/2793860171 shutdown_connections 2024-09-19T06:50:34.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.384+0000 7feddccaa740 1 --2- 172.21.15.103:0/2793860171 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55c98dd3e220 0x55c98dd40640 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.384+0000 7feddccaa740 1 --2- 172.21.15.103:0/2793860171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c98dcd2d80 0x55c98dcc7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.384+0000 7feddccaa740 1 --2- 172.21.15.103:0/2793860171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98dcc3370 0x55c98dcc7640 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.384+0000 7feddccaa740 1 --2- 172.21.15.103:0/2793860171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c98dcb85d0 0x55c98dcc7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.384+0000 7feddccaa740 1 -- 172.21.15.103:0/2793860171 >> 172.21.15.103:0/2793860171 conn(0x55c98dc81310 msgr2=0x55c98dcc1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:34.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.384+0000 7fedb37fe640 0 client.15972 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:50:34.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.384+0000 7feddccaa740 1 -- 172.21.15.103:0/2793860171 shutdown_connections 2024-09-19T06:50:34.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.384+0000 7feddccaa740 1 -- 172.21.15.103:0/2793860171 wait complete. 2024-09-19T06:50:34.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.384+0000 7feddccaa740 20 client.15972 trim_cache size 0 max 16384 2024-09-19T06:50:34.391 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:50:34.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.385+0000 7fedda8d1640 1 -- 172.21.15.103:0/1396840870 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fedc00a35e0 msgr2=0x7fedc00a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:34.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.385+0000 7fedda8d1640 1 --2- 172.21.15.103:0/1396840870 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fedc00a35e0 0x7fedc00a5aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fedcc01f000 tx=0x7fedcc01b000 comp rx=0 tx=0).stop 2024-09-19T06:50:34.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.385+0000 7fedda8d1640 1 -- 172.21.15.103:0/1396840870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedd4103760 msgr2=0x7fedd4100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:34.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.385+0000 7fedda8d1640 1 --2- 172.21.15.103:0/1396840870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedd4103760 0x7fedd4100f50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009230 tx=0x7fedbc003c10 comp rx=0 tx=0).stop 2024-09-19T06:50:34.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.385+0000 7fedda8d1640 1 -- 172.21.15.103:0/1396840870 shutdown_connections 2024-09-19T06:50:34.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.385+0000 7fedda8d1640 1 --2- 172.21.15.103:0/1396840870 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fedc00a35e0 0x7fedc00a5aa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.385+0000 7fedda8d1640 1 --2- 172.21.15.103:0/1396840870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedd41082e0 0x7fedd4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.385+0000 7fedda8d1640 1 --2- 172.21.15.103:0/1396840870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedd4103760 0x7fedd4100f50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.385+0000 7fedda8d1640 1 --2- 172.21.15.103:0/1396840870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd4102db0 0x7fedd4100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:34.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.385+0000 7fedda8d1640 1 -- 172.21.15.103:0/1396840870 >> 172.21.15.103:0/1396840870 conn(0x7fedd40fc7d0 msgr2=0x7fedd4105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:34.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.385+0000 7fedda8d1640 1 -- 172.21.15.103:0/1396840870 shutdown_connections 2024-09-19T06:50:34.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:34.385+0000 7fedda8d1640 1 -- 172.21.15.103:0/1396840870 wait complete. 2024-09-19T06:50:34.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:34 smithi134 ceph-mon[35356]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-09-19T06:50:35.196 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:50:35.445 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:36.293 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:36 smithi103 ceph-mon[31158]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-19T06:50:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:36 smithi174 ceph-mon[35637]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-19T06:50:36.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:36 smithi134 ceph-mon[35356]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-19T06:50:36.650 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T06:50:36.650 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T06:50:36.650 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:50:36.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.688+0000 7f1701e93640 1 -- 172.21.15.103:0/2863992797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16fc1036e0 msgr2=0x7f16fc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:36.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.688+0000 7f1701e93640 1 --2- 172.21.15.103:0/2863992797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16fc1036e0 0x7f16fc107b90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009930 tx=0x7f16f002a420 comp rx=0 tx=0).stop 2024-09-19T06:50:36.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.688+0000 7f1701e93640 1 -- 172.21.15.103:0/2863992797 shutdown_connections 2024-09-19T06:50:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.688+0000 7f1701e93640 1 --2- 172.21.15.103:0/2863992797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16fc1082a0 0x7f16fc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.688+0000 7f1701e93640 1 --2- 172.21.15.103:0/2863992797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16fc1036e0 0x7f16fc107b90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.688+0000 7f1701e93640 1 --2- 172.21.15.103:0/2863992797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16fc102d30 0x7f16fc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.688+0000 7f1701e93640 1 -- 172.21.15.103:0/2863992797 >> 172.21.15.103:0/2863992797 conn(0x7f16fc0fc7d0 msgr2=0x7f16fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.689+0000 7f1701e93640 1 -- 172.21.15.103:0/2863992797 shutdown_connections 2024-09-19T06:50:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.689+0000 7f1701e93640 1 -- 172.21.15.103:0/2863992797 wait complete. 2024-09-19T06:50:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.689+0000 7f1701e93640 1 Processor -- start 2024-09-19T06:50:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.689+0000 7f1701e93640 1 -- start start 2024-09-19T06:50:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f1701e93640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16fc102d30 0x7f16fc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f1701e93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16fc1036e0 0x7f16fc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f1701e93640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16fc1082a0 0x7f16fc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f1701e93640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1120c0 con 0x7f16fc1036e0 2024-09-19T06:50:36.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f1701e93640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc111f40 con 0x7f16fc1082a0 2024-09-19T06:50:36.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f1701e93640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc112240 con 0x7f16fc102d30 2024-09-19T06:50:36.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f16faffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16fc1036e0 0x7f16fc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f16faffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16fc1036e0 0x7f16fc100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48230/0 (socket says 172.21.15.103:48230) 2024-09-19T06:50:36.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f16faffd640 1 -- 172.21.15.103:0/1208957890 learned_addr learned my addr 172.21.15.103:0/1208957890 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:36.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f16fbfff640 1 --2- 172.21.15.103:0/1208957890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16fc1082a0 0x7f16fc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f16fb7fe640 1 --2- 172.21.15.103:0/1208957890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16fc102d30 0x7f16fc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f16faffd640 1 -- 172.21.15.103:0/1208957890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16fc102d30 msgr2=0x7f16fc100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:36.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f16faffd640 1 --2- 172.21.15.103:0/1208957890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16fc102d30 0x7f16fc100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f16faffd640 1 -- 172.21.15.103:0/1208957890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16fc1082a0 msgr2=0x7f16fc101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:36.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f16faffd640 1 --2- 172.21.15.103:0/1208957890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16fc1082a0 0x7f16fc101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f16faffd640 1 -- 172.21.15.103:0/1208957890 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16fc1a6b00 con 0x7f16fc1036e0 2024-09-19T06:50:36.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.690+0000 7f16faffd640 1 --2- 172.21.15.103:0/1208957890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16fc1036e0 0x7f16fc100ed0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f16f0005940 tx=0x7f16f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:36.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.691+0000 7f16f8ff9640 1 -- 172.21.15.103:0/1208957890 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f003f030 con 0x7f16fc1036e0 2024-09-19T06:50:36.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.691+0000 7f1701e93640 1 -- 172.21.15.103:0/1208957890 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16fc1a6d90 con 0x7f16fc1036e0 2024-09-19T06:50:36.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.691+0000 7f16f8ff9640 1 -- 172.21.15.103:0/1208957890 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f0003ce0 con 0x7f16fc1036e0 2024-09-19T06:50:36.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.691+0000 7f16f8ff9640 1 -- 172.21.15.103:0/1208957890 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0035070 con 0x7f16fc1036e0 2024-09-19T06:50:36.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.691+0000 7f1701e93640 1 -- 172.21.15.103:0/1208957890 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16fc1a7260 con 0x7f16fc1036e0 2024-09-19T06:50:36.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.693+0000 7f16f8ff9640 1 -- 172.21.15.103:0/1208957890 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7f16f0042030 con 0x7f16fc1036e0 2024-09-19T06:50:36.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f170426c740 1 Processor -- start 2024-09-19T06:50:36.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f170426c740 1 -- start start 2024-09-19T06:50:36.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f170426c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea41256b40 0x55ea41255090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f170426c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea412555d0 0x55ea41255a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f170426c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea41260370 0x55ea41262760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f170426c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ea411bb2f0 con 0x55ea412555d0 2024-09-19T06:50:36.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f170426c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ea41162cf0 con 0x55ea41256b40 2024-09-19T06:50:36.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f170426c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ea411b72b0 con 0x55ea41260370 2024-09-19T06:50:36.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f16f8ff9640 1 --2- 172.21.15.103:0/1208957890 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f16e40a35e0 0x7f16e40a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f16fb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea412555d0 0x55ea41255a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f16fb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea412555d0 0x55ea41255a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48238/0 (socket says 172.21.15.103:48238) 2024-09-19T06:50:36.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f16fb7fe640 1 -- 172.21.15.103:0/3936619423 learned_addr learned my addr 172.21.15.103:0/3936619423 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:36.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f16f8ff9640 1 -- 172.21.15.103:0/1208957890 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f16f00f0690 con 0x7f16fc1036e0 2024-09-19T06:50:36.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f16faffd640 1 --2- 172.21.15.103:0/3936619423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea41260370 0x55ea41262760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f16fb7fe640 1 -- 172.21.15.103:0/3936619423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea41260370 msgr2=0x55ea41262760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:36.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f16fb7fe640 1 --2- 172.21.15.103:0/3936619423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea41260370 0x55ea41262760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f16fb7fe640 1 -- 172.21.15.103:0/3936619423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea41256b40 msgr2=0x55ea41255090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:50:36.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f16fb7fe640 1 --2- 172.21.15.103:0/3936619423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea41256b40 0x55ea41255090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.694+0000 7f16fb7fe640 1 -- 172.21.15.103:0/3936619423 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea411473a0 con 0x55ea412555d0 2024-09-19T06:50:36.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.695+0000 7f16fbfff640 1 --2- 172.21.15.103:0/1208957890 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f16e40a35e0 0x7f16e40a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.695+0000 7f16fb7fe640 1 --2- 172.21.15.103:0/3936619423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea412555d0 0x55ea41255a10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009a70 tx=0x7f16ec009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432acf94529a0846 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:36.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.695+0000 7f16e1ffb640 1 -- 172.21.15.103:0/3936619423 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec017070 con 0x55ea412555d0 2024-09-19T06:50:36.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.695+0000 7f16e1ffb640 1 -- 172.21.15.103:0/3936619423 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16ec013470 con 0x55ea412555d0 2024-09-19T06:50:36.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.695+0000 7f16e1ffb640 1 -- 172.21.15.103:0/3936619423 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec012630 con 0x55ea412555d0 2024-09-19T06:50:36.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.696+0000 7f170426c740 1 -- 172.21.15.103:0/3936619423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea412555d0 msgr2=0x55ea41255a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:36.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.696+0000 7f170426c740 1 --2- 172.21.15.103:0/3936619423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea412555d0 0x55ea41255a10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009a70 tx=0x7f16ec009f30 comp rx=0 tx=0).stop 2024-09-19T06:50:36.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.696+0000 7f16fbfff640 1 --2- 172.21.15.103:0/1208957890 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f16e40a35e0 0x7f16e40a5aa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f16f4005fd0 tx=0x7f16f4009600 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:36.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.696+0000 7f170426c740 1 -- 172.21.15.103:0/3936619423 shutdown_connections 2024-09-19T06:50:36.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.696+0000 7f170426c740 1 --2- 172.21.15.103:0/3936619423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea41260370 0x55ea41262760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.696+0000 7f170426c740 1 --2- 172.21.15.103:0/3936619423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea412555d0 0x55ea41255a10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.696+0000 7f170426c740 1 --2- 172.21.15.103:0/3936619423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea41256b40 0x55ea41255090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.696+0000 7f170426c740 1 -- 172.21.15.103:0/3936619423 >> 172.21.15.103:0/3936619423 conn(0x55ea4121e310 msgr2=0x55ea41251c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:36.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.696+0000 7f170426c740 1 -- 172.21.15.103:0/3936619423 shutdown_connections 2024-09-19T06:50:36.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.696+0000 7f170426c740 1 -- 172.21.15.103:0/3936619423 wait complete. 2024-09-19T06:50:36.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.696+0000 7f170426c740 1 -- start start 2024-09-19T06:50:36.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.696+0000 7f170426c740 1 Processor -- start 2024-09-19T06:50:36.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f170426c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea412555d0 0x55ea4125fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f170426c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea41256b40 0x55ea4125d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f170426c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea4125d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f170426c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ea41263070 con 0x55ea41260370 2024-09-19T06:50:36.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f170426c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ea41176060 con 0x55ea41256b40 2024-09-19T06:50:36.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f170426c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ea412631f0 con 0x55ea412555d0 2024-09-19T06:50:36.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f170426c740 10 client.0 _ll_register_callbacks cb 0x55ea40ec98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:36.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f16fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea4125d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f16fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea4125d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48254/0 (socket says 172.21.15.103:48254) 2024-09-19T06:50:36.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f16fbfff640 1 -- 172.21.15.103:0/2498559125 learned_addr learned my addr 172.21.15.103:0/2498559125 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:36.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f16faffd640 1 --2- 172.21.15.103:0/2498559125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea41256b40 0x55ea4125d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f16fb7fe640 1 --2- 172.21.15.103:0/2498559125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea412555d0 0x55ea4125fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f16fbfff640 1 -- 172.21.15.103:0/2498559125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea412555d0 msgr2=0x55ea4125fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f16fbfff640 1 --2- 172.21.15.103:0/2498559125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea412555d0 0x55ea4125fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f16fbfff640 1 -- 172.21.15.103:0/2498559125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea41256b40 msgr2=0x55ea4125d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f16fbfff640 1 --2- 172.21.15.103:0/2498559125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea41256b40 0x55ea4125d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.697+0000 7f16fbfff640 1 -- 172.21.15.103:0/2498559125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea40f2c3c0 con 0x55ea41260370 2024-09-19T06:50:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.698+0000 7f16fbfff640 1 --2- 172.21.15.103:0/2498559125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea4125d5d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f16f4012c20 tx=0x7f16f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203f8f71d8c72faf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.698+0000 7f16daffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.698+0000 7f16daffd640 1 -- 172.21.15.103:0/2498559125 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4012e50 con 0x55ea41260370 2024-09-19T06:50:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.698+0000 7f16daffd640 1 -- 172.21.15.103:0/2498559125 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f16f4003970 con 0x55ea41260370 2024-09-19T06:50:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.698+0000 7f170426c740 1 -- 172.21.15.103:0/2498559125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16ec013470 con 0x55ea41260370 2024-09-19T06:50:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.698+0000 7f16daffd640 1 -- 172.21.15.103:0/2498559125 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16f402e050 con 0x55ea41260370 2024-09-19T06:50:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.698+0000 7f170426c740 10 client.15984 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.698+0000 7f170426c740 1 -- 172.21.15.103:0/2498559125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea4126eb50 con 0x55ea41260370 2024-09-19T06:50:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.699+0000 7f16daffd640 1 -- 172.21.15.103:0/2498559125 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16ec013470 con 0x55ea41260370 2024-09-19T06:50:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.699+0000 7f16daffd640 1 -- 172.21.15.103:0/2498559125 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f16f400f040 con 0x55ea41260370 2024-09-19T06:50:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.699+0000 7f170426c740 10 client.15984 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.699+0000 7f170426c740 10 client.15984 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:50:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.699+0000 7f170426c740 20 client.15984 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.699+0000 7f170426c740 1 --2- 172.21.15.103:0/2498559125 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55ea41270440 0x55ea412728a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.699+0000 7f170426c740 4 client.15984 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.699+0000 7f170426c740 1 -- 172.21.15.103:0/2498559125 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea411c8860 con 0x55ea41270440 2024-09-19T06:50:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.699+0000 7f16fb7fe640 1 --2- 172.21.15.103:0/2498559125 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55ea41270440 0x55ea412728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.700+0000 7f16daffd640 10 client.15984 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:50:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.700+0000 7f16fb7fe640 1 --2- 172.21.15.103:0/2498559125 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55ea41270440 0x55ea412728a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c270df61f6f31044 server_cookie=b48734c6df19dce4 in_seq=0 out_seq=0 2024-09-19T06:50:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.700+0000 7f16daffd640 1 -- 172.21.15.103:0/2498559125 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea411c8860 con 0x55ea41270440 2024-09-19T06:50:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.700+0000 7f16daffd640 10 client.15984 handle_command_reply: tid=0 2024-09-19T06:50:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.701+0000 7f170426c740 1 client.15984 shutdown 2024-09-19T06:50:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.701+0000 7f170426c740 1 -- 172.21.15.103:0/2498559125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 msgr2=0x55ea4125d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.701+0000 7f170426c740 1 --2- 172.21.15.103:0/2498559125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea4125d5d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f16f4012c20 tx=0x7f16f4004a40 comp rx=0 tx=0).stop 2024-09-19T06:50:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.701+0000 7f170426c740 1 -- 172.21.15.103:0/2498559125 shutdown_connections 2024-09-19T06:50:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.701+0000 7f170426c740 1 --2- 172.21.15.103:0/2498559125 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55ea41270440 0x55ea412728a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.701+0000 7f170426c740 1 --2- 172.21.15.103:0/2498559125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea4125d5d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.701+0000 7f170426c740 1 --2- 172.21.15.103:0/2498559125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea41256b40 0x55ea4125d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.701+0000 7f16daffd640 0 client.15984 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:50:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.701+0000 7f170426c740 1 --2- 172.21.15.103:0/2498559125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea412555d0 0x55ea4125fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.701+0000 7f170426c740 1 -- 172.21.15.103:0/2498559125 >> 172.21.15.103:0/2498559125 conn(0x55ea4121e310 msgr2=0x55ea41257930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.701+0000 7f170426c740 1 -- 172.21.15.103:0/2498559125 shutdown_connections 2024-09-19T06:50:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.701+0000 7f170426c740 1 -- 172.21.15.103:0/2498559125 wait complete. 2024-09-19T06:50:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.701+0000 7f170426c740 20 client.15984 trim_cache size 0 max 16384 2024-09-19T06:50:36.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f170426c740 1 Processor -- start 2024-09-19T06:50:36.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f170426c740 1 -- start start 2024-09-19T06:50:36.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f170426c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea412555d0 0x55ea41251e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f170426c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea41256b40 0x55ea4126c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f170426c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea41258120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f170426c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1afb30 con 0x55ea41260370 2024-09-19T06:50:36.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f170426c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1b1670 con 0x55ea412555d0 2024-09-19T06:50:36.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f170426c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1b0200 con 0x55ea41256b40 2024-09-19T06:50:36.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f16fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea41258120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f16fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea41258120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48256/0 (socket says 172.21.15.103:48256) 2024-09-19T06:50:36.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f16fbfff640 1 -- 172.21.15.103:0/2866644721 learned_addr learned my addr 172.21.15.103:0/2866644721 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:36.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f16faffd640 1 --2- 172.21.15.103:0/2866644721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea41256b40 0x55ea4126c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f16fb7fe640 1 --2- 172.21.15.103:0/2866644721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea412555d0 0x55ea41251e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f16fbfff640 1 -- 172.21.15.103:0/2866644721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea41256b40 msgr2=0x55ea4126c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:36.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f16fbfff640 1 --2- 172.21.15.103:0/2866644721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea41256b40 0x55ea4126c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f16fbfff640 1 -- 172.21.15.103:0/2866644721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea412555d0 msgr2=0x55ea41251e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:36.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f16fbfff640 1 --2- 172.21.15.103:0/2866644721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea412555d0 0x55ea41251e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f16fbfff640 1 -- 172.21.15.103:0/2866644721 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea4126ce10 con 0x55ea41260370 2024-09-19T06:50:36.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f16faffd640 1 --2- 172.21.15.103:0/2866644721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea41256b40 0x55ea4126c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:36.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f16fb7fe640 1 --2- 172.21.15.103:0/2866644721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea412555d0 0x55ea41251e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:36.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f16fbfff640 1 --2- 172.21.15.103:0/2866644721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea41258120 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f16f4012bf0 tx=0x7f16f4009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5a594f21d2f746 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:36.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f16e17fa640 1 -- 172.21.15.103:0/2866644721 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4012e50 con 0x55ea41260370 2024-09-19T06:50:36.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f16e17fa640 1 -- 172.21.15.103:0/2866644721 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f4036070 con 0x55ea41260370 2024-09-19T06:50:36.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.729+0000 7f16e17fa640 1 -- 172.21.15.103:0/2866644721 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f401fae0 con 0x55ea41260370 2024-09-19T06:50:36.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.730+0000 7f170426c740 1 -- 172.21.15.103:0/2866644721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 msgr2=0x55ea41258120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:36.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.730+0000 7f170426c740 1 --2- 172.21.15.103:0/2866644721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea41258120 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f16f4012bf0 tx=0x7f16f4009fd0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.730+0000 7f170426c740 1 -- 172.21.15.103:0/2866644721 shutdown_connections 2024-09-19T06:50:36.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.730+0000 7f170426c740 1 --2- 172.21.15.103:0/2866644721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea41258120 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.730+0000 7f170426c740 1 --2- 172.21.15.103:0/2866644721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea41256b40 0x55ea4126c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.730+0000 7f170426c740 1 --2- 172.21.15.103:0/2866644721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea412555d0 0x55ea41251e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.730+0000 7f170426c740 1 -- 172.21.15.103:0/2866644721 >> 172.21.15.103:0/2866644721 conn(0x55ea4121e310 msgr2=0x55ea41251700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:36.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.730+0000 7f170426c740 1 -- 172.21.15.103:0/2866644721 shutdown_connections 2024-09-19T06:50:36.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.730+0000 7f170426c740 1 -- 172.21.15.103:0/2866644721 wait complete. 2024-09-19T06:50:36.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.731+0000 7f170426c740 1 -- start start 2024-09-19T06:50:36.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.731+0000 7f170426c740 1 Processor -- start 2024-09-19T06:50:36.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.731+0000 7f170426c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea412555d0 0x55ea41268b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.731+0000 7f170426c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea41256b40 0x55ea41263e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.731+0000 7f170426c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea412643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.731+0000 7f170426c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1b0fa0 con 0x55ea41260370 2024-09-19T06:50:36.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.731+0000 7f170426c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1b08d0 con 0x55ea412555d0 2024-09-19T06:50:36.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.731+0000 7f170426c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc1adaf0 con 0x55ea41256b40 2024-09-19T06:50:36.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.731+0000 7f170426c740 10 client.0 _ll_register_callbacks cb 0x7f16f4016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:36.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.731+0000 7f16fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea412643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.731+0000 7f16fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea412643a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48268/0 (socket says 172.21.15.103:48268) 2024-09-19T06:50:36.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.731+0000 7f16fbfff640 1 -- 172.21.15.103:0/3841652797 learned_addr learned my addr 172.21.15.103:0/3841652797 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:36.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.731+0000 7f16faffd640 1 --2- 172.21.15.103:0/3841652797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea41256b40 0x55ea41263e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.731+0000 7f16fb7fe640 1 --2- 172.21.15.103:0/3841652797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea412555d0 0x55ea41268b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.732+0000 7f16fbfff640 1 -- 172.21.15.103:0/3841652797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea41256b40 msgr2=0x55ea41263e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:36.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.732+0000 7f16fbfff640 1 --2- 172.21.15.103:0/3841652797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea41256b40 0x55ea41263e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.732+0000 7f16fbfff640 1 -- 172.21.15.103:0/3841652797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea412555d0 msgr2=0x55ea41268b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:36.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.732+0000 7f16fbfff640 1 --2- 172.21.15.103:0/3841652797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea412555d0 0x55ea41268b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.732+0000 7f16fbfff640 1 -- 172.21.15.103:0/3841652797 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea41264ba0 con 0x55ea41260370 2024-09-19T06:50:36.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.732+0000 7f16fbfff640 1 --2- 172.21.15.103:0/3841652797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea412643a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f16f401f4a0 tx=0x7f16f4012f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b7db0e4a3ec507 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:36.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.732+0000 7f16daffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:36.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.732+0000 7f16daffd640 1 -- 172.21.15.103:0/3841652797 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4004a00 con 0x55ea41260370 2024-09-19T06:50:36.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.732+0000 7f16daffd640 1 -- 172.21.15.103:0/3841652797 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f16f4004ba0 con 0x55ea41260370 2024-09-19T06:50:36.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.732+0000 7f170426c740 1 -- 172.21.15.103:0/3841652797 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f4036070 con 0x55ea41260370 2024-09-19T06:50:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.732+0000 7f16daffd640 1 -- 172.21.15.103:0/3841652797 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16f4004da0 con 0x55ea41260370 2024-09-19T06:50:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.732+0000 7f170426c740 10 client.15990 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.732+0000 7f170426c740 1 -- 172.21.15.103:0/3841652797 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea412db0f0 con 0x55ea41260370 2024-09-19T06:50:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.732+0000 7f16daffd640 1 -- 172.21.15.103:0/3841652797 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f4036070 con 0x55ea41260370 2024-09-19T06:50:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.733+0000 7f16daffd640 1 -- 172.21.15.103:0/3841652797 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f16f4041030 con 0x55ea41260370 2024-09-19T06:50:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.733+0000 7f170426c740 10 client.15990 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.733+0000 7f170426c740 10 client.15990 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:50:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.733+0000 7f170426c740 20 client.15990 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.733+0000 7f170426c740 1 --2- 172.21.15.103:0/3841652797 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55ea41270440 0x55ea412dd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.733+0000 7f170426c740 4 client.15990 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.733+0000 7f170426c740 1 -- 172.21.15.103:0/3841652797 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea411c2d70 con 0x55ea41270440 2024-09-19T06:50:36.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.733+0000 7f16fb7fe640 1 --2- 172.21.15.103:0/3841652797 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55ea41270440 0x55ea412dd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:36.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.733+0000 7f16fb7fe640 1 --2- 172.21.15.103:0/3841652797 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55ea41270440 0x55ea412dd710 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=342ae4d3877f1872 server_cookie=7a6edeb1a0033b49 in_seq=0 out_seq=0 2024-09-19T06:50:36.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.733+0000 7f16daffd640 10 client.15990 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:50:36.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.734+0000 7f16daffd640 1 -- 172.21.15.103:0/3841652797 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea411c2d70 con 0x55ea41270440 2024-09-19T06:50:36.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.734+0000 7f16daffd640 10 client.15990 handle_command_reply: tid=0 2024-09-19T06:50:36.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.734+0000 7f170426c740 1 client.15990 shutdown 2024-09-19T06:50:36.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.734+0000 7f170426c740 1 -- 172.21.15.103:0/3841652797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 msgr2=0x55ea412643a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:36.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.734+0000 7f170426c740 1 --2- 172.21.15.103:0/3841652797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea412643a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f16f401f4a0 tx=0x7f16f4012f50 comp rx=0 tx=0).stop 2024-09-19T06:50:36.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.734+0000 7f170426c740 1 -- 172.21.15.103:0/3841652797 shutdown_connections 2024-09-19T06:50:36.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.734+0000 7f170426c740 1 --2- 172.21.15.103:0/3841652797 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55ea41270440 0x55ea412dd710 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.734+0000 7f170426c740 1 --2- 172.21.15.103:0/3841652797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea41260370 0x55ea412643a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.734+0000 7f170426c740 1 --2- 172.21.15.103:0/3841652797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea41256b40 0x55ea41263e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.734+0000 7f170426c740 1 --2- 172.21.15.103:0/3841652797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea412555d0 0x55ea41268b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.734+0000 7f170426c740 1 -- 172.21.15.103:0/3841652797 >> 172.21.15.103:0/3841652797 conn(0x55ea4121e310 msgr2=0x55ea4125e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:36.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.734+0000 7f16daffd640 0 client.15990 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:50:36.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.734+0000 7f170426c740 1 -- 172.21.15.103:0/3841652797 shutdown_connections 2024-09-19T06:50:36.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.734+0000 7f170426c740 1 -- 172.21.15.103:0/3841652797 wait complete. 2024-09-19T06:50:36.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.734+0000 7f170426c740 20 client.15990 trim_cache size 0 max 16384 2024-09-19T06:50:36.741 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:50:36.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.735+0000 7f1701e93640 1 -- 172.21.15.103:0/1208957890 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f16e40a35e0 msgr2=0x7f16e40a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:36.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.735+0000 7f1701e93640 1 --2- 172.21.15.103:0/1208957890 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f16e40a35e0 0x7f16e40a5aa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f16f4005fd0 tx=0x7f16f4009600 comp rx=0 tx=0).stop 2024-09-19T06:50:36.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.735+0000 7f1701e93640 1 -- 172.21.15.103:0/1208957890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16fc1036e0 msgr2=0x7f16fc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:36.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.735+0000 7f1701e93640 1 --2- 172.21.15.103:0/1208957890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16fc1036e0 0x7f16fc100ed0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f16f0005940 tx=0x7f16f00042f0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.735+0000 7f1701e93640 1 -- 172.21.15.103:0/1208957890 shutdown_connections 2024-09-19T06:50:36.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.735+0000 7f1701e93640 1 --2- 172.21.15.103:0/1208957890 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f16e40a35e0 0x7f16e40a5aa0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.735+0000 7f1701e93640 1 --2- 172.21.15.103:0/1208957890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16fc1082a0 0x7f16fc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.735+0000 7f1701e93640 1 --2- 172.21.15.103:0/1208957890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16fc1036e0 0x7f16fc100ed0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.735+0000 7f1701e93640 1 --2- 172.21.15.103:0/1208957890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16fc102d30 0x7f16fc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:36.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.735+0000 7f1701e93640 1 -- 172.21.15.103:0/1208957890 >> 172.21.15.103:0/1208957890 conn(0x7f16fc0fc7d0 msgr2=0x7f16fc105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:36.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.735+0000 7f1701e93640 1 -- 172.21.15.103:0/1208957890 shutdown_connections 2024-09-19T06:50:36.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:36.735+0000 7f1701e93640 1 -- 172.21.15.103:0/1208957890 wait complete. 2024-09-19T06:50:37.398 INFO:tasks.workunit.client.0.smithi103.stderr:Updating files: 92% (11933/12896) Updating files: 93% (11994/12896) Updating files: 94% (12123/12896) Updating files: 95% (12252/12896) Updating files: 96% (12381/12896) Updating files: 97% (12510/12896) Updating files: 98% (12639/12896) Updating files: 99% (12768/12896) Updating files: 100% (12896/12896) Updating files: 100% (12896/12896), done. 2024-09-19T06:50:37.674 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:50:37.785 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:38.123 INFO:tasks.workunit.client.0.smithi103.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T06:50:38.123 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-19T06:50:38.123 INFO:tasks.workunit.client.0.smithi103.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T06:50:38.123 INFO:tasks.workunit.client.0.smithi103.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T06:50:38.123 INFO:tasks.workunit.client.0.smithi103.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T06:50:38.123 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-19T06:50:38.123 INFO:tasks.workunit.client.0.smithi103.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T06:50:38.123 INFO:tasks.workunit.client.0.smithi103.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T06:50:38.123 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-19T06:50:38.123 INFO:tasks.workunit.client.0.smithi103.stderr: git switch -c 2024-09-19T06:50:38.123 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-19T06:50:38.124 INFO:tasks.workunit.client.0.smithi103.stderr:Or undo this operation with: 2024-09-19T06:50:38.124 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-19T06:50:38.124 INFO:tasks.workunit.client.0.smithi103.stderr: git switch - 2024-09-19T06:50:38.124 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-19T06:50:38.124 INFO:tasks.workunit.client.0.smithi103.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T06:50:38.124 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-19T06:50:38.124 INFO:tasks.workunit.client.0.smithi103.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T06:50:38.128 DEBUG:teuthology.orchestra.run.smithi103:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T06:50:38.198 INFO:tasks.workunit.client.0.smithi103.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T06:50:38.200 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T06:50:38.200 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T06:50:38.329 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:38 smithi103 ceph-mon[31158]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T06:50:38.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:38 smithi174 ceph-mon[35637]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T06:50:38.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:38 smithi134 ceph-mon[35356]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T06:50:38.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.410+0000 7f43cdb09640 1 -- 172.21.15.103:0/444741738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c8102d40 msgr2=0x7f43c8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:38.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.410+0000 7f43cdb09640 1 --2- 172.21.15.103:0/444741738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c8102d40 0x7f43c8103120 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009900 tx=0x7f43b802a480 comp rx=0 tx=0).stop 2024-09-19T06:50:38.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.410+0000 7f43cdb09640 1 -- 172.21.15.103:0/444741738 shutdown_connections 2024-09-19T06:50:38.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.410+0000 7f43cdb09640 1 --2- 172.21.15.103:0/444741738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43c810c170 0x7f43c810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.410+0000 7f43cdb09640 1 --2- 172.21.15.103:0/444741738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c8103660 0x7f43c810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.410+0000 7f43cdb09640 1 --2- 172.21.15.103:0/444741738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c8102d40 0x7f43c8103120 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.410+0000 7f43cdb09640 1 -- 172.21.15.103:0/444741738 >> 172.21.15.103:0/444741738 conn(0x7f43c80fc7a0 msgr2=0x7f43c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:38.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.410+0000 7f43cdb09640 1 -- 172.21.15.103:0/444741738 shutdown_connections 2024-09-19T06:50:38.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.410+0000 7f43cdb09640 1 -- 172.21.15.103:0/444741738 wait complete. 2024-09-19T06:50:38.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.410+0000 7f43cdb09640 1 Processor -- start 2024-09-19T06:50:38.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43cdb09640 1 -- start start 2024-09-19T06:50:38.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43cdb09640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c8102d40 0x7f43c81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43cdb09640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43c8103660 0x7f43c8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43cdb09640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c810c170 0x7f43c819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43cdb09640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f43c8111f70 con 0x7f43c810c170 2024-09-19T06:50:38.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43cdb09640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43c8111df0 con 0x7f43c8102d40 2024-09-19T06:50:38.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43cdb09640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f43c81120f0 con 0x7f43c8103660 2024-09-19T06:50:38.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43c7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c810c170 0x7f43c819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43c7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c810c170 0x7f43c819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59102/0 (socket says 172.21.15.103:59102) 2024-09-19T06:50:38.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43c7fff640 1 -- 172.21.15.103:0/4122187934 learned_addr learned my addr 172.21.15.103:0/4122187934 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:38.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43c6ffd640 1 --2- 172.21.15.103:0/4122187934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43c8103660 0x7f43c8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43c77fe640 1 --2- 172.21.15.103:0/4122187934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c8102d40 0x7f43c81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43c7fff640 1 -- 172.21.15.103:0/4122187934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43c8103660 msgr2=0x7f43c8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:38.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43c7fff640 1 --2- 172.21.15.103:0/4122187934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43c8103660 0x7f43c8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43c7fff640 1 -- 172.21.15.103:0/4122187934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c8102d40 msgr2=0x7f43c81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:38.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43c7fff640 1 --2- 172.21.15.103:0/4122187934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c8102d40 0x7f43c81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.411+0000 7f43c7fff640 1 -- 172.21.15.103:0/4122187934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c819d390 con 0x7f43c810c170 2024-09-19T06:50:38.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.412+0000 7f43c6ffd640 1 --2- 172.21.15.103:0/4122187934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43c8103660 0x7f43c8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:38.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.412+0000 7f43c7fff640 1 --2- 172.21.15.103:0/4122187934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c810c170 0x7f43c819ccb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f43c000cce0 tx=0x7f43c000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:38.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.412+0000 7f43c4ff9640 1 -- 172.21.15.103:0/4122187934 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c000f450 con 0x7f43c810c170 2024-09-19T06:50:38.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.412+0000 7f43cdb09640 1 -- 172.21.15.103:0/4122187934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c819d680 con 0x7f43c810c170 2024-09-19T06:50:38.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.412+0000 7f43c4ff9640 1 -- 172.21.15.103:0/4122187934 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43c000fad0 con 0x7f43c810c170 2024-09-19T06:50:38.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.412+0000 7f43c4ff9640 1 -- 172.21.15.103:0/4122187934 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0018c10 con 0x7f43c810c170 2024-09-19T06:50:38.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.412+0000 7f43cdb09640 1 -- 172.21.15.103:0/4122187934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c81a4f60 con 0x7f43c810c170 2024-09-19T06:50:38.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.413+0000 7f43cdb09640 1 -- 172.21.15.103:0/4122187934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c8103120 con 0x7f43c810c170 2024-09-19T06:50:38.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.414+0000 7f43c4ff9640 1 -- 172.21.15.103:0/4122187934 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7f43c000f5f0 con 0x7f43c810c170 2024-09-19T06:50:38.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.415+0000 7f43c4ff9640 1 --2- 172.21.15.103:0/4122187934 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f43b00a3560 0x7f43b00a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.415+0000 7f43c77fe640 1 --2- 172.21.15.103:0/4122187934 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f43b00a3560 0x7f43b00a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.415+0000 7f43c4ff9640 1 -- 172.21.15.103:0/4122187934 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f43c00d06b0 con 0x7f43c810c170 2024-09-19T06:50:38.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.416+0000 7f43c77fe640 1 --2- 172.21.15.103:0/4122187934 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f43b00a3560 0x7f43b00a5a20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f43b8005ab0 tx=0x7f43b8005a20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:38.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.419+0000 7f43c4ff9640 1 -- 172.21.15.103:0/4122187934 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f43c0014030 con 0x7f43c810c170 2024-09-19T06:50:38.596 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:38.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.633+0000 7f43cdb09640 1 -- 172.21.15.103:0/4122187934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43c802d7c0 con 0x7f43c810c170 2024-09-19T06:50:38.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.634+0000 7f43c4ff9640 1 -- 172.21.15.103:0/4122187934 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f43c009ce60 con 0x7f43c810c170 2024-09-19T06:50:38.636 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:50:38.636 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:50:38.636 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:50:38.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.636+0000 7f43cdb09640 1 -- 172.21.15.103:0/4122187934 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f43b00a3560 msgr2=0x7f43b00a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:38.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.636+0000 7f43cdb09640 1 --2- 172.21.15.103:0/4122187934 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f43b00a3560 0x7f43b00a5a20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f43b8005ab0 tx=0x7f43b8005a20 comp rx=0 tx=0).stop 2024-09-19T06:50:38.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.636+0000 7f43cdb09640 1 -- 172.21.15.103:0/4122187934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c810c170 msgr2=0x7f43c819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:38.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.636+0000 7f43cdb09640 1 --2- 172.21.15.103:0/4122187934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c810c170 0x7f43c819ccb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f43c000cce0 tx=0x7f43c000a620 comp rx=0 tx=0).stop 2024-09-19T06:50:38.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.636+0000 7f43cdb09640 1 -- 172.21.15.103:0/4122187934 shutdown_connections 2024-09-19T06:50:38.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.636+0000 7f43cdb09640 1 --2- 172.21.15.103:0/4122187934 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f43b00a3560 0x7f43b00a5a20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.636+0000 7f43cdb09640 1 --2- 172.21.15.103:0/4122187934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c810c170 0x7f43c819ccb0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.636+0000 7f43cdb09640 1 --2- 172.21.15.103:0/4122187934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43c8103660 0x7f43c8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.636+0000 7f43cdb09640 1 --2- 172.21.15.103:0/4122187934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c8102d40 0x7f43c81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.636+0000 7f43cdb09640 1 -- 172.21.15.103:0/4122187934 >> 172.21.15.103:0/4122187934 conn(0x7f43c80fc7a0 msgr2=0x7f43c80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:38.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.636+0000 7f43cdb09640 1 -- 172.21.15.103:0/4122187934 shutdown_connections 2024-09-19T06:50:38.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:38.637+0000 7f43cdb09640 1 -- 172.21.15.103:0/4122187934 wait complete. 2024-09-19T06:50:39.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:39 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/4122187934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:39.351 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:39 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/4122187934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:39.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:39 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/4122187934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:39.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.437+0000 7fc105d70640 1 -- 172.21.15.103:0/4154727788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc100100e20 msgr2=0x7fc100101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:39.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.437+0000 7fc105d70640 1 --2- 172.21.15.103:0/4154727788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc100100e20 0x7fc100101200 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0009900 tx=0x7fc0f002a480 comp rx=0 tx=0).stop 2024-09-19T06:50:39.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.437+0000 7fc105d70640 1 -- 172.21.15.103:0/4154727788 shutdown_connections 2024-09-19T06:50:39.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.437+0000 7fc105d70640 1 --2- 172.21.15.103:0/4154727788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100102ef0 0x7fc10010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.437+0000 7fc105d70640 1 --2- 172.21.15.103:0/4154727788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc100102570 0x7fc1001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.437+0000 7fc105d70640 1 --2- 172.21.15.103:0/4154727788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc100100e20 0x7fc100101200 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.437+0000 7fc105d70640 1 -- 172.21.15.103:0/4154727788 >> 172.21.15.103:0/4154727788 conn(0x7fc1000fc7d0 msgr2=0x7fc1000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:39.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.437+0000 7fc105d70640 1 -- 172.21.15.103:0/4154727788 shutdown_connections 2024-09-19T06:50:39.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.437+0000 7fc105d70640 1 -- 172.21.15.103:0/4154727788 wait complete. 2024-09-19T06:50:39.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.438+0000 7fc105d70640 1 Processor -- start 2024-09-19T06:50:39.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.438+0000 7fc105d70640 1 -- start start 2024-09-19T06:50:39.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.438+0000 7fc105d70640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc100100e20 0x7fc100198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.438+0000 7fc105d70640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc100102570 0x7fc100198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.438+0000 7fc105d70640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100102ef0 0x7fc10019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.438+0000 7fc105d70640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc100111020 con 0x7fc100100e20 2024-09-19T06:50:39.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.438+0000 7fc105d70640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc100110ea0 con 0x7fc100102ef0 2024-09-19T06:50:39.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.438+0000 7fc105d70640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001111a0 con 0x7fc100102570 2024-09-19T06:50:39.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.438+0000 7fc0ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc100100e20 0x7fc100198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.438+0000 7fc0ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc100100e20 0x7fc100198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59118/0 (socket says 172.21.15.103:59118) 2024-09-19T06:50:39.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.438+0000 7fc0ff7fe640 1 -- 172.21.15.103:0/3329282284 learned_addr learned my addr 172.21.15.103:0/3329282284 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:39.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.439+0000 7fc0f7fff640 1 --2- 172.21.15.103:0/3329282284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc100102570 0x7fc100198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.439+0000 7fc0fffff640 1 --2- 172.21.15.103:0/3329282284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100102ef0 0x7fc10019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.439+0000 7fc0ff7fe640 1 -- 172.21.15.103:0/3329282284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc100102570 msgr2=0x7fc100198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:39.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.439+0000 7fc0ff7fe640 1 --2- 172.21.15.103:0/3329282284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc100102570 0x7fc100198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.439+0000 7fc0ff7fe640 1 -- 172.21.15.103:0/3329282284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100102ef0 msgr2=0x7fc10019cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:39.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.439+0000 7fc0ff7fe640 1 --2- 172.21.15.103:0/3329282284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100102ef0 0x7fc10019cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.439+0000 7fc0ff7fe640 1 -- 172.21.15.103:0/3329282284 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10019d620 con 0x7fc100100e20 2024-09-19T06:50:39.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.439+0000 7fc0f7fff640 1 --2- 172.21.15.103:0/3329282284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc100102570 0x7fc100198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:39.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.439+0000 7fc0fffff640 1 --2- 172.21.15.103:0/3329282284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100102ef0 0x7fc10019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:39.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.439+0000 7fc0ff7fe640 1 --2- 172.21.15.103:0/3329282284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc100100e20 0x7fc100198670 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0006010 tx=0x7fc0f0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:39.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.439+0000 7fc0fd7fa640 1 -- 172.21.15.103:0/3329282284 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f003f070 con 0x7fc100100e20 2024-09-19T06:50:39.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.439+0000 7fc0fd7fa640 1 -- 172.21.15.103:0/3329282284 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0f0031e30 con 0x7fc100100e20 2024-09-19T06:50:39.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.439+0000 7fc105d70640 1 -- 172.21.15.103:0/3329282284 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10019d8b0 con 0x7fc100100e20 2024-09-19T06:50:39.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.439+0000 7fc0fd7fa640 1 -- 172.21.15.103:0/3329282284 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0035030 con 0x7fc100100e20 2024-09-19T06:50:39.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.439+0000 7fc105d70640 1 -- 172.21.15.103:0/3329282284 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1001a51f0 con 0x7fc100100e20 2024-09-19T06:50:39.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.442+0000 7fc0fd7fa640 1 -- 172.21.15.103:0/3329282284 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7fc0f0042070 con 0x7fc100100e20 2024-09-19T06:50:39.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.442+0000 7fc108149740 1 Processor -- start 2024-09-19T06:50:39.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.442+0000 7fc108149740 1 -- start start 2024-09-19T06:50:39.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.442+0000 7fc108149740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b08090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.442+0000 7fc108149740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 0x558a88b08a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.442+0000 7fc108149740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b17780 0x558a88b19b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.442+0000 7fc108149740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558a88a6e2f0 con 0x558a88b09b40 2024-09-19T06:50:39.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.442+0000 7fc108149740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558a88a15cf0 con 0x558a88b085d0 2024-09-19T06:50:39.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.442+0000 7fc108149740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558a88a6a2b0 con 0x558a88b17780 2024-09-19T06:50:39.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.442+0000 7fc0fd7fa640 1 --2- 172.21.15.103:0/3329282284 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0ec0a3560 0x7fc0ec0a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.442+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b08090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.442+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b08090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59132/0 (socket says 172.21.15.103:59132) 2024-09-19T06:50:39.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.442+0000 7fc0fffff640 1 -- 172.21.15.103:0/4022414836 learned_addr learned my addr 172.21.15.103:0/4022414836 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:39.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.442+0000 7fc0fffff640 1 --2- 172.21.15.103:0/3329282284 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0ec0a3560 0x7fc0ec0a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.443+0000 7fc0fd7fa640 1 -- 172.21.15.103:0/3329282284 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc0f00f0ac0 con 0x7fc100100e20 2024-09-19T06:50:39.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.443+0000 7fc0ff7fe640 1 --2- 172.21.15.103:0/4022414836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 0x558a88b08a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.443+0000 7fc0f7fff640 1 --2- 172.21.15.103:0/4022414836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b17780 0x558a88b19b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.443+0000 7fc0fffff640 1 -- 172.21.15.103:0/4022414836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b17780 msgr2=0x558a88b19b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:39.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.443+0000 7fc0fffff640 1 --2- 172.21.15.103:0/4022414836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b17780 0x558a88b19b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.443+0000 7fc0fffff640 1 -- 172.21.15.103:0/4022414836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 msgr2=0x558a88b08a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:39.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.443+0000 7fc0fffff640 1 --2- 172.21.15.103:0/4022414836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 0x558a88b08a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.443+0000 7fc0fffff640 1 -- 172.21.15.103:0/4022414836 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a889fa3a0 con 0x558a88b09b40 2024-09-19T06:50:39.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.443+0000 7fc0fffff640 1 --2- 172.21.15.103:0/4022414836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b08090 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800ed70 tx=0x7fc0f800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8353e367639fc306 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:39.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.443+0000 7fc0fffff640 1 --2- 172.21.15.103:0/3329282284 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0ec0a3560 0x7fc0ec0a5a20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800cc40 tx=0x7fc0f8019000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:39.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.443+0000 7fc0f5ffb640 1 -- 172.21.15.103:0/4022414836 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8018b70 con 0x558a88b09b40 2024-09-19T06:50:39.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.443+0000 7fc0f5ffb640 1 -- 172.21.15.103:0/4022414836 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0f8018d10 con 0x558a88b09b40 2024-09-19T06:50:39.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.443+0000 7fc0f5ffb640 1 -- 172.21.15.103:0/4022414836 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8023600 con 0x558a88b09b40 2024-09-19T06:50:39.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.444+0000 7fc108149740 1 -- 172.21.15.103:0/4022414836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 msgr2=0x558a88b08090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:39.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.444+0000 7fc108149740 1 --2- 172.21.15.103:0/4022414836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b08090 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800ed70 tx=0x7fc0f800c740 comp rx=0 tx=0).stop 2024-09-19T06:50:39.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.444+0000 7fc108149740 1 -- 172.21.15.103:0/4022414836 shutdown_connections 2024-09-19T06:50:39.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.444+0000 7fc108149740 1 --2- 172.21.15.103:0/4022414836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b17780 0x558a88b19b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.444+0000 7fc108149740 1 --2- 172.21.15.103:0/4022414836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 0x558a88b08a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.444+0000 7fc108149740 1 --2- 172.21.15.103:0/4022414836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b08090 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.444+0000 7fc108149740 1 -- 172.21.15.103:0/4022414836 >> 172.21.15.103:0/4022414836 conn(0x558a88ad1310 msgr2=0x558a88b04c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:39.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.444+0000 7fc108149740 1 -- 172.21.15.103:0/4022414836 shutdown_connections 2024-09-19T06:50:39.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.444+0000 7fc108149740 1 -- 172.21.15.103:0/4022414836 wait complete. 2024-09-19T06:50:39.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.444+0000 7fc108149740 1 -- start start 2024-09-19T06:50:39.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.444+0000 7fc108149740 1 Processor -- start 2024-09-19T06:50:39.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc108149740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 0x558a88b17210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc108149740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b09b40 0x558a88b144c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc108149740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b17780 0x558a88b14c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc108149740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558a88b1a420 con 0x558a88b17780 2024-09-19T06:50:39.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc108149740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558a88a29060 con 0x558a88b085d0 2024-09-19T06:50:39.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc108149740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558a88b1a5a0 con 0x558a88b09b40 2024-09-19T06:50:39.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc108149740 10 client.0 _ll_register_callbacks cb 0x558a8877c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:39.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b17780 0x558a88b14c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b17780 0x558a88b14c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59144/0 (socket says 172.21.15.103:59144) 2024-09-19T06:50:39.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc0fffff640 1 -- 172.21.15.103:0/4012840524 learned_addr learned my addr 172.21.15.103:0/4012840524 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:39.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc0ff7fe640 1 --2- 172.21.15.103:0/4012840524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 0x558a88b17210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc0f7fff640 1 --2- 172.21.15.103:0/4012840524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b09b40 0x558a88b144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc0fffff640 1 -- 172.21.15.103:0/4012840524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b09b40 msgr2=0x558a88b144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:39.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc0fffff640 1 --2- 172.21.15.103:0/4012840524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b09b40 0x558a88b144c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc0fffff640 1 -- 172.21.15.103:0/4012840524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 msgr2=0x558a88b17210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:39.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc0fffff640 1 --2- 172.21.15.103:0/4012840524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 0x558a88b17210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc0fffff640 1 -- 172.21.15.103:0/4012840524 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a887df3c0 con 0x558a88b17780 2024-09-19T06:50:39.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc0ff7fe640 1 --2- 172.21.15.103:0/4012840524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 0x558a88b17210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:39.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc0fffff640 1 --2- 172.21.15.103:0/4012840524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b17780 0x558a88b14c50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8004b90 tx=0x7fc0f8009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea63d49cd6becaa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:39.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc0deffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:39.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.445+0000 7fc0deffd640 1 -- 172.21.15.103:0/4012840524 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f802a070 con 0x558a88b17780 2024-09-19T06:50:39.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.446+0000 7fc0deffd640 1 -- 172.21.15.103:0/4012840524 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc0f80243f0 con 0x558a88b17780 2024-09-19T06:50:39.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.446+0000 7fc108149740 1 -- 172.21.15.103:0/4012840524 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0f8018d10 con 0x558a88b17780 2024-09-19T06:50:39.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.446+0000 7fc0deffd640 1 -- 172.21.15.103:0/4012840524 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc0f8017040 con 0x558a88b17780 2024-09-19T06:50:39.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.446+0000 7fc108149740 10 client.16008 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:39.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.446+0000 7fc108149740 1 -- 172.21.15.103:0/4012840524 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558a88b25f70 con 0x558a88b17780 2024-09-19T06:50:39.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.446+0000 7fc0deffd640 1 -- 172.21.15.103:0/4012840524 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0f8018d10 con 0x558a88b17780 2024-09-19T06:50:39.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.446+0000 7fc0deffd640 1 -- 172.21.15.103:0/4012840524 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc0f80233a0 con 0x558a88b17780 2024-09-19T06:50:39.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.446+0000 7fc108149740 10 client.16008 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:39.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.446+0000 7fc108149740 10 client.16008 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:50:39.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.446+0000 7fc108149740 20 client.16008 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:39.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.446+0000 7fc108149740 1 --2- 172.21.15.103:0/4012840524 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558a88b27380 0x558a88b29770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.446+0000 7fc108149740 4 client.16008 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:39.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.446+0000 7fc108149740 1 -- 172.21.15.103:0/4012840524 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a88a7b860 con 0x558a88b27380 2024-09-19T06:50:39.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.446+0000 7fc0ff7fe640 1 --2- 172.21.15.103:0/4012840524 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558a88b27380 0x558a88b29770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.447+0000 7fc0deffd640 10 client.16008 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:50:39.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.447+0000 7fc0ff7fe640 1 --2- 172.21.15.103:0/4012840524 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558a88b27380 0x558a88b29770 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a890eb74e26846a server_cookie=a6d212cd385b0e90 in_seq=0 out_seq=0 2024-09-19T06:50:39.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.448+0000 7fc0deffd640 1 -- 172.21.15.103:0/4012840524 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a88a7b860 con 0x558a88b27380 2024-09-19T06:50:39.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.448+0000 7fc0deffd640 10 client.16008 handle_command_reply: tid=0 2024-09-19T06:50:39.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.448+0000 7fc108149740 1 client.16008 shutdown 2024-09-19T06:50:39.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.448+0000 7fc108149740 1 -- 172.21.15.103:0/4012840524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b17780 msgr2=0x558a88b14c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:39.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.448+0000 7fc108149740 1 --2- 172.21.15.103:0/4012840524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b17780 0x558a88b14c50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8004b90 tx=0x7fc0f8009fd0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.448+0000 7fc108149740 1 -- 172.21.15.103:0/4012840524 shutdown_connections 2024-09-19T06:50:39.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.448+0000 7fc108149740 1 --2- 172.21.15.103:0/4012840524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b17780 0x558a88b14c50 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.448+0000 7fc108149740 1 --2- 172.21.15.103:0/4012840524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b09b40 0x558a88b144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.448+0000 7fc108149740 1 --2- 172.21.15.103:0/4012840524 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558a88b27380 0x558a88b29770 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.448+0000 7fc108149740 1 --2- 172.21.15.103:0/4012840524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 0x558a88b17210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.448+0000 7fc0deffd640 0 client.16008 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:50:39.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.448+0000 7fc108149740 1 -- 172.21.15.103:0/4012840524 >> 172.21.15.103:0/4012840524 conn(0x558a88ad1310 msgr2=0x558a88b0a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:39.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.448+0000 7fc108149740 1 -- 172.21.15.103:0/4012840524 shutdown_connections 2024-09-19T06:50:39.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.448+0000 7fc108149740 1 -- 172.21.15.103:0/4012840524 wait complete. 2024-09-19T06:50:39.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.448+0000 7fc108149740 20 client.16008 trim_cache size 0 max 16384 2024-09-19T06:50:39.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.461+0000 7fc108149740 1 Processor -- start 2024-09-19T06:50:39.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.461+0000 7fc108149740 1 -- start start 2024-09-19T06:50:39.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc108149740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b085d0 0x558a88b13d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc108149740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b2ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc108149740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b17780 0x558a88b2f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc108149740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001ab7d0 con 0x558a88b09b40 2024-09-19T06:50:39.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc108149740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001ad310 con 0x558a88b17780 2024-09-19T06:50:39.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc108149740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001abea0 con 0x558a88b085d0 2024-09-19T06:50:39.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc0f7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b2ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc0f7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b2ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59150/0 (socket says 172.21.15.103:59150) 2024-09-19T06:50:39.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc0f7fff640 1 -- 172.21.15.103:0/68471831 learned_addr learned my addr 172.21.15.103:0/68471831 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:39.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc0ff7fe640 1 --2- 172.21.15.103:0/68471831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b085d0 0x558a88b13d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc0fffff640 1 --2- 172.21.15.103:0/68471831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b17780 0x558a88b2f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc0f7fff640 1 -- 172.21.15.103:0/68471831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b085d0 msgr2=0x558a88b13d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:39.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc0f7fff640 1 --2- 172.21.15.103:0/68471831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b085d0 0x558a88b13d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc0f7fff640 1 -- 172.21.15.103:0/68471831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b17780 msgr2=0x558a88b2f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:39.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc0f7fff640 1 --2- 172.21.15.103:0/68471831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b17780 0x558a88b2f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc0f7fff640 1 -- 172.21.15.103:0/68471831 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a88b04de0 con 0x558a88b09b40 2024-09-19T06:50:39.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc0fffff640 1 --2- 172.21.15.103:0/68471831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b17780 0x558a88b2f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:39.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc0f7fff640 1 --2- 172.21.15.103:0/68471831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b2ee60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800a6c0 tx=0x7fc0e800ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52731d28bd36549 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:39.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.462+0000 7fc0f57fa640 1 -- 172.21.15.103:0/68471831 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8018020 con 0x558a88b09b40 2024-09-19T06:50:39.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.463+0000 7fc0f57fa640 1 -- 172.21.15.103:0/68471831 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e800fca0 con 0x558a88b09b40 2024-09-19T06:50:39.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.463+0000 7fc0f57fa640 1 -- 172.21.15.103:0/68471831 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e80133e0 con 0x558a88b09b40 2024-09-19T06:50:39.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.463+0000 7fc108149740 1 -- 172.21.15.103:0/68471831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 msgr2=0x558a88b2ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:39.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.463+0000 7fc108149740 1 --2- 172.21.15.103:0/68471831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b2ee60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800a6c0 tx=0x7fc0e800ab80 comp rx=0 tx=0).stop 2024-09-19T06:50:39.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.463+0000 7fc108149740 1 -- 172.21.15.103:0/68471831 shutdown_connections 2024-09-19T06:50:39.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.463+0000 7fc108149740 1 --2- 172.21.15.103:0/68471831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b17780 0x558a88b2f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.463+0000 7fc108149740 1 --2- 172.21.15.103:0/68471831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b2ee60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.463+0000 7fc108149740 1 --2- 172.21.15.103:0/68471831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b085d0 0x558a88b13d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.463+0000 7fc108149740 1 -- 172.21.15.103:0/68471831 >> 172.21.15.103:0/68471831 conn(0x558a88ad1310 msgr2=0x558a88b16540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:39.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.463+0000 7fc108149740 1 -- 172.21.15.103:0/68471831 shutdown_connections 2024-09-19T06:50:39.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.463+0000 7fc108149740 1 -- 172.21.15.103:0/68471831 wait complete. 2024-09-19T06:50:39.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc108149740 1 -- start start 2024-09-19T06:50:39.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc108149740 1 Processor -- start 2024-09-19T06:50:39.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc108149740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 0x558a88b1f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc108149740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b1fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc108149740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b17780 0x558a88b1c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc108149740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001acc40 con 0x558a88b09b40 2024-09-19T06:50:39.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc108149740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001ac570 con 0x558a88b085d0 2024-09-19T06:50:39.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc108149740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001a97b0 con 0x558a88b17780 2024-09-19T06:50:39.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc108149740 10 client.0 _ll_register_callbacks cb 0x7fc0f800eb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:39.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc0f7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b1fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc0f7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b1fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59154/0 (socket says 172.21.15.103:59154) 2024-09-19T06:50:39.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc0f7fff640 1 -- 172.21.15.103:0/265409581 learned_addr learned my addr 172.21.15.103:0/265409581 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:39.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc0fffff640 1 --2- 172.21.15.103:0/265409581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b17780 0x558a88b1c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc0ff7fe640 1 --2- 172.21.15.103:0/265409581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 0x558a88b1f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc0f7fff640 1 -- 172.21.15.103:0/265409581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b17780 msgr2=0x558a88b1c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:39.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc0f7fff640 1 --2- 172.21.15.103:0/265409581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b17780 0x558a88b1c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc0f7fff640 1 -- 172.21.15.103:0/265409581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 msgr2=0x558a88b1f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:39.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc0f7fff640 1 --2- 172.21.15.103:0/265409581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 0x558a88b1f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc0f7fff640 1 -- 172.21.15.103:0/265409581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a88b204b0 con 0x558a88b09b40 2024-09-19T06:50:39.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.464+0000 7fc0ff7fe640 1 --2- 172.21.15.103:0/265409581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 0x558a88b1f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:39.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.465+0000 7fc0f7fff640 1 --2- 172.21.15.103:0/265409581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b1fd70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8014c20 tx=0x7fc0e80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e26858a4d82ca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:39.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.465+0000 7fc0deffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:39.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.465+0000 7fc0deffd640 1 -- 172.21.15.103:0/265409581 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8018030 con 0x558a88b09b40 2024-09-19T06:50:39.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.465+0000 7fc0deffd640 1 -- 172.21.15.103:0/265409581 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc0e801d630 con 0x558a88b09b40 2024-09-19T06:50:39.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.465+0000 7fc108149740 1 -- 172.21.15.103:0/265409581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0e800fca0 con 0x558a88b09b40 2024-09-19T06:50:39.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.465+0000 7fc0deffd640 1 -- 172.21.15.103:0/265409581 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc0e800fe50 con 0x558a88b09b40 2024-09-19T06:50:39.492 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:50:39.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.465+0000 7fc108149740 10 client.16014 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:39.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.465+0000 7fc108149740 1 -- 172.21.15.103:0/265409581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558a88b923f0 con 0x558a88b09b40 2024-09-19T06:50:39.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.465+0000 7fc0deffd640 1 -- 172.21.15.103:0/265409581 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e800fca0 con 0x558a88b09b40 2024-09-19T06:50:39.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.465+0000 7fc0deffd640 1 -- 172.21.15.103:0/265409581 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc0e8027920 con 0x558a88b09b40 2024-09-19T06:50:39.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.465+0000 7fc108149740 10 client.16014 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:39.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.465+0000 7fc108149740 10 client.16014 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:50:39.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.465+0000 7fc108149740 20 client.16014 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:39.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.465+0000 7fc108149740 1 --2- 172.21.15.103:0/265409581 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558a88b92a60 0x558a88b94e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.465+0000 7fc108149740 4 client.16014 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:39.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.465+0000 7fc108149740 1 -- 172.21.15.103:0/265409581 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a88a75d70 con 0x558a88b92a60 2024-09-19T06:50:39.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.466+0000 7fc0ff7fe640 1 --2- 172.21.15.103:0/265409581 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558a88b92a60 0x558a88b94e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.466+0000 7fc0ff7fe640 1 --2- 172.21.15.103:0/265409581 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558a88b92a60 0x558a88b94e80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=efbb96b84a5dc2e0 server_cookie=1e2a4ee1de98d3ec in_seq=0 out_seq=0 2024-09-19T06:50:39.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.466+0000 7fc0deffd640 10 client.16014 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:50:39.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.466+0000 7fc0deffd640 1 -- 172.21.15.103:0/265409581 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a88a75d70 con 0x558a88b92a60 2024-09-19T06:50:39.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.466+0000 7fc0deffd640 10 client.16014 handle_command_reply: tid=0 2024-09-19T06:50:39.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.466+0000 7fc108149740 1 client.16014 shutdown 2024-09-19T06:50:39.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.466+0000 7fc108149740 1 -- 172.21.15.103:0/265409581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 msgr2=0x558a88b1fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:39.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.466+0000 7fc108149740 1 --2- 172.21.15.103:0/265409581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b1fd70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8014c20 tx=0x7fc0e80047d0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.466+0000 7fc108149740 1 -- 172.21.15.103:0/265409581 shutdown_connections 2024-09-19T06:50:39.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.466+0000 7fc108149740 1 --2- 172.21.15.103:0/265409581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a88b17780 0x558a88b1c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.466+0000 7fc108149740 1 --2- 172.21.15.103:0/265409581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a88b09b40 0x558a88b1fd70 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.466+0000 7fc108149740 1 --2- 172.21.15.103:0/265409581 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558a88b92a60 0x558a88b94e80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.466+0000 7fc108149740 1 --2- 172.21.15.103:0/265409581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a88b085d0 0x558a88b1f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.466+0000 7fc108149740 1 -- 172.21.15.103:0/265409581 >> 172.21.15.103:0/265409581 conn(0x558a88ad1310 msgr2=0x558a88b0a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:39.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.466+0000 7fc0deffd640 0 client.16014 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:50:39.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.467+0000 7fc108149740 1 -- 172.21.15.103:0/265409581 shutdown_connections 2024-09-19T06:50:39.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.467+0000 7fc108149740 1 -- 172.21.15.103:0/265409581 wait complete. 2024-09-19T06:50:39.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.467+0000 7fc108149740 20 client.16014 trim_cache size 0 max 16384 2024-09-19T06:50:39.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.467+0000 7fc105d70640 1 -- 172.21.15.103:0/3329282284 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0ec0a3560 msgr2=0x7fc0ec0a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:39.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.467+0000 7fc105d70640 1 --2- 172.21.15.103:0/3329282284 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0ec0a3560 0x7fc0ec0a5a20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800cc40 tx=0x7fc0f8019000 comp rx=0 tx=0).stop 2024-09-19T06:50:39.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.467+0000 7fc105d70640 1 -- 172.21.15.103:0/3329282284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc100100e20 msgr2=0x7fc100198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:39.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.467+0000 7fc105d70640 1 --2- 172.21.15.103:0/3329282284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc100100e20 0x7fc100198670 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0006010 tx=0x7fc0f0003c80 comp rx=0 tx=0).stop 2024-09-19T06:50:39.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.467+0000 7fc105d70640 1 -- 172.21.15.103:0/3329282284 shutdown_connections 2024-09-19T06:50:39.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.467+0000 7fc105d70640 1 --2- 172.21.15.103:0/3329282284 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0ec0a3560 0x7fc0ec0a5a20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.467+0000 7fc105d70640 1 --2- 172.21.15.103:0/3329282284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100102ef0 0x7fc10019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.467+0000 7fc105d70640 1 --2- 172.21.15.103:0/3329282284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc100102570 0x7fc100198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.467+0000 7fc105d70640 1 --2- 172.21.15.103:0/3329282284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc100100e20 0x7fc100198670 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.467+0000 7fc105d70640 1 -- 172.21.15.103:0/3329282284 >> 172.21.15.103:0/3329282284 conn(0x7fc1000fc7d0 msgr2=0x7fc10006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:39.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.467+0000 7fc105d70640 1 -- 172.21.15.103:0/3329282284 shutdown_connections 2024-09-19T06:50:39.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:39.467+0000 7fc105d70640 1 -- 172.21.15.103:0/3329282284 wait complete. 2024-09-19T06:50:40.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:40 smithi174 ceph-mon[35637]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-19T06:50:40.338 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:40 smithi134 ceph-mon[35356]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-19T06:50:40.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:40 smithi103 ceph-mon[31158]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-19T06:50:40.411 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T06:50:40.438 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T06:50:40.439 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T06:50:40.507 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T06:50:40.553 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T06:50:40.555 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T06:50:40.555 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T06:50:40.596 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T06:50:40.600 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T06:50:40.600 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T06:50:40.616 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-09-19T06:50:40.618 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-09-19T06:50:40.619 DEBUG:teuthology.orchestra.run.smithi103:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-19T06:50:40.679 INFO:tasks.workunit.client.0.smithi103.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-19T06:50:40.679 INFO:tasks.workunit.client.0.smithi103.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-09-19T06:50:40.680 INFO:tasks.workunit.client.0.smithi103.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-09-19T06:50:40.907 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:50:41.028 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:41.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.507+0000 7f78f222b640 1 -- 172.21.15.103:0/3125537167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78ec102790 msgr2=0x7f78ec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.507+0000 7f78f222b640 1 --2- 172.21.15.103:0/3125537167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78ec102790 0x7f78ec1067e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f78d4009900 tx=0x7f78d402a410 comp rx=0 tx=0).stop 2024-09-19T06:50:41.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.507+0000 7f78f222b640 1 -- 172.21.15.103:0/3125537167 shutdown_connections 2024-09-19T06:50:41.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.507+0000 7f78f222b640 1 --2- 172.21.15.103:0/3125537167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78ec106f10 0x7f78ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.507+0000 7f78f222b640 1 --2- 172.21.15.103:0/3125537167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78ec102790 0x7f78ec1067e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.507+0000 7f78f222b640 1 --2- 172.21.15.103:0/3125537167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78ec101de0 0x7f78ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.507+0000 7f78f222b640 1 -- 172.21.15.103:0/3125537167 >> 172.21.15.103:0/3125537167 conn(0x7f78ec0fd8c0 msgr2=0x7f78ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:41.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.507+0000 7f78f222b640 1 -- 172.21.15.103:0/3125537167 shutdown_connections 2024-09-19T06:50:41.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.507+0000 7f78f222b640 1 -- 172.21.15.103:0/3125537167 wait complete. 2024-09-19T06:50:41.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.508+0000 7f78f222b640 1 Processor -- start 2024-09-19T06:50:41.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.508+0000 7f78f222b640 1 -- start start 2024-09-19T06:50:41.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.508+0000 7f78f222b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78ec101de0 0x7f78ec198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.508+0000 7f78f222b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78ec102790 0x7f78ec1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.508+0000 7f78f222b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78ec106f10 0x7f78ec19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.508+0000 7f78f222b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec111f90 con 0x7f78ec106f10 2024-09-19T06:50:41.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.508+0000 7f78f222b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec111e10 con 0x7f78ec101de0 2024-09-19T06:50:41.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.508+0000 7f78f222b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec112110 con 0x7f78ec102790 2024-09-19T06:50:41.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78ebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78ec106f10 0x7f78ec19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78ebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78ec106f10 0x7f78ec19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59168/0 (socket says 172.21.15.103:59168) 2024-09-19T06:50:41.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78ebfff640 1 -- 172.21.15.103:0/3848135720 learned_addr learned my addr 172.21.15.103:0/3848135720 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:41.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78eb7fe640 1 --2- 172.21.15.103:0/3848135720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78ec101de0 0x7f78ec198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78ebfff640 1 -- 172.21.15.103:0/3848135720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78ec102790 msgr2=0x7f78ec1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78eaffd640 1 --2- 172.21.15.103:0/3848135720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78ec102790 0x7f78ec1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78ebfff640 1 --2- 172.21.15.103:0/3848135720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78ec102790 0x7f78ec1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78ebfff640 1 -- 172.21.15.103:0/3848135720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78ec101de0 msgr2=0x7f78ec198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78ebfff640 1 --2- 172.21.15.103:0/3848135720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78ec101de0 0x7f78ec198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78ebfff640 1 -- 172.21.15.103:0/3848135720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ec19d420 con 0x7f78ec106f10 2024-09-19T06:50:41.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78eaffd640 1 --2- 172.21.15.103:0/3848135720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78ec102790 0x7f78ec1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:41.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78ebfff640 1 --2- 172.21.15.103:0/3848135720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78ec106f10 0x7f78ec19cd40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009fd0 tx=0x7f78dc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:41.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78e8ff9640 1 -- 172.21.15.103:0/3848135720 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc011830 con 0x7f78ec106f10 2024-09-19T06:50:41.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78e8ff9640 1 -- 172.21.15.103:0/3848135720 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78dc016070 con 0x7f78ec106f10 2024-09-19T06:50:41.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78e8ff9640 1 -- 172.21.15.103:0/3848135720 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc00f2e0 con 0x7f78ec106f10 2024-09-19T06:50:41.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78f222b640 1 -- 172.21.15.103:0/3848135720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ec19d710 con 0x7f78ec106f10 2024-09-19T06:50:41.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.509+0000 7f78f222b640 1 -- 172.21.15.103:0/3848135720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ec1a4ff0 con 0x7f78ec106f10 2024-09-19T06:50:41.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.511+0000 7f78e8ff9640 1 -- 172.21.15.103:0/3848135720 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7f78dc0119d0 con 0x7f78ec106f10 2024-09-19T06:50:41.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.512+0000 7f78f4604740 1 Processor -- start 2024-09-19T06:50:41.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.512+0000 7f78f4604740 1 -- start start 2024-09-19T06:50:41.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.512+0000 7f78f4604740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a71ffd710 0x556a71ffdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.512+0000 7f78f4604740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffe030 0x556a7200c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.512+0000 7f78e8ff9640 1 --2- 172.21.15.103:0/3848135720 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f78d80a3560 0x7f78d80a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.512+0000 7f78f4604740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a7200ce40 0x556a7200f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.512+0000 7f78f4604740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556a71f62160 con 0x556a71ffe030 2024-09-19T06:50:41.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.512+0000 7f78f4604740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556a71f09720 con 0x556a71ffd710 2024-09-19T06:50:41.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.512+0000 7f78f4604740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556a71f5e120 con 0x556a7200ce40 2024-09-19T06:50:41.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.512+0000 7f78eaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffe030 0x556a7200c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.512+0000 7f78eb7fe640 1 --2- 172.21.15.103:0/3848135720 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f78d80a3560 0x7f78d80a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.512+0000 7f78eaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffe030 0x556a7200c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59172/0 (socket says 172.21.15.103:59172) 2024-09-19T06:50:41.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.512+0000 7f78eaffd640 1 -- 172.21.15.103:0/786871294 learned_addr learned my addr 172.21.15.103:0/786871294 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:41.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.512+0000 7f78e8ff9640 1 -- 172.21.15.103:0/3848135720 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f78dc0cf4e0 con 0x7f78ec106f10 2024-09-19T06:50:41.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.512+0000 7f78eaffd640 1 --2- 172.21.15.103:0/786871294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a71ffd710 0x556a71ffdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.513+0000 7f78ebfff640 1 --2- 172.21.15.103:0/786871294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a7200ce40 0x556a7200f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.513+0000 7f78eaffd640 1 -- 172.21.15.103:0/786871294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a7200ce40 msgr2=0x556a7200f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:41.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.513+0000 7f78eaffd640 1 --2- 172.21.15.103:0/786871294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a7200ce40 0x556a7200f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.513+0000 7f78eaffd640 1 -- 172.21.15.103:0/786871294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a71ffd710 msgr2=0x556a71ffdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:41.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.513+0000 7f78eaffd640 1 --2- 172.21.15.103:0/786871294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a71ffd710 0x556a71ffdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.513+0000 7f78eaffd640 1 -- 172.21.15.103:0/786871294 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a71cd3ba0 con 0x556a71ffe030 2024-09-19T06:50:41.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.513+0000 7f78eb7fe640 1 --2- 172.21.15.103:0/3848135720 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f78d80a3560 0x7f78d80a5a20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f78e4005fd0 tx=0x7f78e4009760 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:41.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.513+0000 7f78eaffd640 1 --2- 172.21.15.103:0/786871294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffe030 0x556a7200c900 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f78d40057c0 tx=0x7f78d4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb96beca3d69c66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:41.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.513+0000 7f78e1ffb640 1 -- 172.21.15.103:0/786871294 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4035040 con 0x556a71ffe030 2024-09-19T06:50:41.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.513+0000 7f78e1ffb640 1 -- 172.21.15.103:0/786871294 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78d40035f0 con 0x556a71ffe030 2024-09-19T06:50:41.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.513+0000 7f78e1ffb640 1 -- 172.21.15.103:0/786871294 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d403c670 con 0x556a71ffe030 2024-09-19T06:50:41.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.514+0000 7f78f4604740 1 -- 172.21.15.103:0/786871294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffe030 msgr2=0x556a7200c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.514+0000 7f78f4604740 1 --2- 172.21.15.103:0/786871294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffe030 0x556a7200c900 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f78d40057c0 tx=0x7f78d4000bc0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.514+0000 7f78f4604740 1 -- 172.21.15.103:0/786871294 shutdown_connections 2024-09-19T06:50:41.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.514+0000 7f78f4604740 1 --2- 172.21.15.103:0/786871294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a7200ce40 0x556a7200f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.514+0000 7f78f4604740 1 --2- 172.21.15.103:0/786871294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffe030 0x556a7200c900 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.514+0000 7f78f4604740 1 --2- 172.21.15.103:0/786871294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a71ffd710 0x556a71ffdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.514+0000 7f78f4604740 1 -- 172.21.15.103:0/786871294 >> 172.21.15.103:0/786871294 conn(0x556a71ff9de0 msgr2=0x556a71ffa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:41.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.514+0000 7f78f4604740 1 -- 172.21.15.103:0/786871294 shutdown_connections 2024-09-19T06:50:41.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.514+0000 7f78f4604740 1 -- 172.21.15.103:0/786871294 wait complete. 2024-09-19T06:50:41.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.514+0000 7f78f4604740 1 -- start start 2024-09-19T06:50:41.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.514+0000 7f78f4604740 1 Processor -- start 2024-09-19T06:50:41.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78f4604740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffd710 0x556a72009de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78f4604740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a71ffe030 0x556a7200a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78f4604740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7200ce40 0x556a7200a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78f4604740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556a7200f9b0 con 0x556a71ffd710 2024-09-19T06:50:41.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78f4604740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556a71eea900 con 0x556a7200ce40 2024-09-19T06:50:41.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78f4604740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556a7200fb30 con 0x556a71ffe030 2024-09-19T06:50:41.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78f4604740 10 client.0 _ll_register_callbacks cb 0x556a71ee8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:41.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffd710 0x556a72009de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffd710 0x556a72009de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59174/0 (socket says 172.21.15.103:59174) 2024-09-19T06:50:41.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78eb7fe640 1 -- 172.21.15.103:0/2400203974 learned_addr learned my addr 172.21.15.103:0/2400203974 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:41.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78eaffd640 1 --2- 172.21.15.103:0/2400203974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a71ffe030 0x556a7200a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78ebfff640 1 --2- 172.21.15.103:0/2400203974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7200ce40 0x556a7200a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78eb7fe640 1 -- 172.21.15.103:0/2400203974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a71ffe030 msgr2=0x556a7200a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:41.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78eb7fe640 1 --2- 172.21.15.103:0/2400203974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a71ffe030 0x556a7200a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78eb7fe640 1 -- 172.21.15.103:0/2400203974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7200ce40 msgr2=0x556a7200a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:41.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78eb7fe640 1 --2- 172.21.15.103:0/2400203974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7200ce40 0x556a7200a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78eb7fe640 1 -- 172.21.15.103:0/2400203974 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a71f2ac50 con 0x556a71ffd710 2024-09-19T06:50:41.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78ebfff640 1 --2- 172.21.15.103:0/2400203974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7200ce40 0x556a7200a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:41.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78eb7fe640 1 --2- 172.21.15.103:0/2400203974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffd710 0x556a72009de0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f78e4009dd0 tx=0x7f78e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ef1424586b8dc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:41.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78caffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:41.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78caffd640 1 -- 172.21.15.103:0/2400203974 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e401e830 con 0x556a71ffd710 2024-09-19T06:50:41.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78caffd640 1 -- 172.21.15.103:0/2400203974 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f78e401e9d0 con 0x556a71ffd710 2024-09-19T06:50:41.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.515+0000 7f78f4604740 1 -- 172.21.15.103:0/2400203974 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d40035f0 con 0x556a71ffd710 2024-09-19T06:50:41.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.516+0000 7f78caffd640 1 -- 172.21.15.103:0/2400203974 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f78e401ebd0 con 0x556a71ffd710 2024-09-19T06:50:41.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.516+0000 7f78f4604740 10 client.16026 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:41.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.516+0000 7f78f4604740 1 -- 172.21.15.103:0/2400203974 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556a7201b7a0 con 0x556a71ffd710 2024-09-19T06:50:41.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.516+0000 7f78caffd640 1 -- 172.21.15.103:0/2400203974 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78d40035f0 con 0x556a71ffd710 2024-09-19T06:50:41.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.516+0000 7f78caffd640 1 -- 172.21.15.103:0/2400203974 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f78e401d3b0 con 0x556a71ffd710 2024-09-19T06:50:41.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.516+0000 7f78f4604740 10 client.16026 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:41.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.516+0000 7f78f4604740 10 client.16026 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:50:41.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.516+0000 7f78f4604740 20 client.16026 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:41.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.516+0000 7f78f4604740 1 --2- 172.21.15.103:0/2400203974 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556a7201cc60 0x556a7201f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.516+0000 7f78f4604740 4 client.16026 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:41.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.516+0000 7f78f4604740 1 -- 172.21.15.103:0/2400203974 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a71f6f890 con 0x556a7201cc60 2024-09-19T06:50:41.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.517+0000 7f78eaffd640 1 --2- 172.21.15.103:0/2400203974 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556a7201cc60 0x556a7201f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.517+0000 7f78eaffd640 1 --2- 172.21.15.103:0/2400203974 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556a7201cc60 0x556a7201f050 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b49d5092b5f246 server_cookie=19d7b528da120a8 in_seq=0 out_seq=0 2024-09-19T06:50:41.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.517+0000 7f78caffd640 10 client.16026 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:41.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.518+0000 7f78caffd640 1 -- 172.21.15.103:0/2400203974 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a71f6f890 con 0x556a7201cc60 2024-09-19T06:50:41.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.518+0000 7f78caffd640 10 client.16026 handle_command_reply: tid=0 2024-09-19T06:50:41.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.518+0000 7f78f4604740 1 client.16026 shutdown 2024-09-19T06:50:41.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.518+0000 7f78f4604740 1 -- 172.21.15.103:0/2400203974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffd710 msgr2=0x556a72009de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.518+0000 7f78f4604740 1 --2- 172.21.15.103:0/2400203974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffd710 0x556a72009de0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f78e4009dd0 tx=0x7f78e4004ab0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.518+0000 7f78f4604740 1 -- 172.21.15.103:0/2400203974 shutdown_connections 2024-09-19T06:50:41.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.518+0000 7f78f4604740 1 --2- 172.21.15.103:0/2400203974 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556a7201cc60 0x556a7201f050 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.518+0000 7f78f4604740 1 --2- 172.21.15.103:0/2400203974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7200ce40 0x556a7200a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.518+0000 7f78f4604740 1 --2- 172.21.15.103:0/2400203974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a71ffe030 0x556a7200a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.518+0000 7f78f4604740 1 --2- 172.21.15.103:0/2400203974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffd710 0x556a72009de0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.518+0000 7f78f4604740 1 -- 172.21.15.103:0/2400203974 >> 172.21.15.103:0/2400203974 conn(0x556a71ff9de0 msgr2=0x556a71fffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:41.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.518+0000 7f78caffd640 0 client.16026 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:41.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.518+0000 7f78f4604740 1 -- 172.21.15.103:0/2400203974 shutdown_connections 2024-09-19T06:50:41.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.518+0000 7f78f4604740 1 -- 172.21.15.103:0/2400203974 wait complete. 2024-09-19T06:50:41.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.518+0000 7f78f4604740 20 client.16026 trim_cache size 0 max 16384 2024-09-19T06:50:41.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78f4604740 1 Processor -- start 2024-09-19T06:50:41.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78f4604740 1 -- start start 2024-09-19T06:50:41.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78f4604740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffe030 0x556a72008f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78f4604740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7200ce40 0x556a72024920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78f4604740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a72024e60 0x556a72007040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78f4604740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1ab770 con 0x556a71ffe030 2024-09-19T06:50:41.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78f4604740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1ad2b0 con 0x556a7200ce40 2024-09-19T06:50:41.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78f4604740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1abe40 con 0x556a72024e60 2024-09-19T06:50:41.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffe030 0x556a72008f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffe030 0x556a72008f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59180/0 (socket says 172.21.15.103:59180) 2024-09-19T06:50:41.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78eb7fe640 1 -- 172.21.15.103:0/3954061120 learned_addr learned my addr 172.21.15.103:0/3954061120 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:41.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78eaffd640 1 --2- 172.21.15.103:0/3954061120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7200ce40 0x556a72024920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78ebfff640 1 --2- 172.21.15.103:0/3954061120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a72024e60 0x556a72007040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78eb7fe640 1 -- 172.21.15.103:0/3954061120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a72024e60 msgr2=0x556a72007040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:41.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78eb7fe640 1 --2- 172.21.15.103:0/3954061120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a72024e60 0x556a72007040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78eb7fe640 1 -- 172.21.15.103:0/3954061120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7200ce40 msgr2=0x556a72024920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:41.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78eb7fe640 1 --2- 172.21.15.103:0/3954061120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7200ce40 0x556a72024920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78eb7fe640 1 -- 172.21.15.103:0/3954061120 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7201f3c0 con 0x556a71ffe030 2024-09-19T06:50:41.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78ebfff640 1 --2- 172.21.15.103:0/3954061120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a72024e60 0x556a72007040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:41.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.536+0000 7f78eaffd640 1 --2- 172.21.15.103:0/3954061120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7200ce40 0x556a72024920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:41.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.537+0000 7f78eb7fe640 1 --2- 172.21.15.103:0/3954061120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffe030 0x556a72008f50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f78e401d9c0 tx=0x7f78e4011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325c3289bc83f0d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:41.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.537+0000 7f78e17fa640 1 -- 172.21.15.103:0/3954061120 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4033070 con 0x556a71ffe030 2024-09-19T06:50:41.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.537+0000 7f78e17fa640 1 -- 172.21.15.103:0/3954061120 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e4034040 con 0x556a71ffe030 2024-09-19T06:50:41.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.537+0000 7f78e17fa640 1 -- 172.21.15.103:0/3954061120 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4038440 con 0x556a71ffe030 2024-09-19T06:50:41.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.538+0000 7f78f4604740 1 -- 172.21.15.103:0/3954061120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffe030 msgr2=0x556a72008f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.538+0000 7f78f4604740 1 --2- 172.21.15.103:0/3954061120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffe030 0x556a72008f50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f78e401d9c0 tx=0x7f78e4011f80 comp rx=0 tx=0).stop 2024-09-19T06:50:41.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.538+0000 7f78f4604740 1 -- 172.21.15.103:0/3954061120 shutdown_connections 2024-09-19T06:50:41.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.538+0000 7f78f4604740 1 --2- 172.21.15.103:0/3954061120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a72024e60 0x556a72007040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.538+0000 7f78f4604740 1 --2- 172.21.15.103:0/3954061120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a7200ce40 0x556a72024920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.538+0000 7f78f4604740 1 --2- 172.21.15.103:0/3954061120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a71ffe030 0x556a72008f50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.538+0000 7f78f4604740 1 -- 172.21.15.103:0/3954061120 >> 172.21.15.103:0/3954061120 conn(0x556a71ff9de0 msgr2=0x556a7200a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:41.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.538+0000 7f78f4604740 1 -- 172.21.15.103:0/3954061120 shutdown_connections 2024-09-19T06:50:41.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.538+0000 7f78f4604740 1 -- 172.21.15.103:0/3954061120 wait complete. 2024-09-19T06:50:41.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.538+0000 7f78f4604740 1 -- start start 2024-09-19T06:50:41.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.538+0000 7f78f4604740 1 Processor -- start 2024-09-19T06:50:41.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78f4604740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a71ffe030 0x556a72010d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78f4604740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a7200ce40 0x556a720112c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78f4604740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a72024e60 0x556a72011800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78f4604740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1acbe0 con 0x556a7200ce40 2024-09-19T06:50:41.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78f4604740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1ac510 con 0x556a72024e60 2024-09-19T06:50:41.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78f4604740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1a9730 con 0x556a71ffe030 2024-09-19T06:50:41.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78eaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a7200ce40 0x556a720112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78ebfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a72024e60 0x556a72011800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78eaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a7200ce40 0x556a720112c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59194/0 (socket says 172.21.15.103:59194) 2024-09-19T06:50:41.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78eaffd640 1 -- 172.21.15.103:0/3246160690 learned_addr learned my addr 172.21.15.103:0/3246160690 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:41.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78eb7fe640 1 --2- 172.21.15.103:0/3246160690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a71ffe030 0x556a72010d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78f4604740 10 client.0 _ll_register_callbacks cb 0x7f78e4005e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:41.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78ebfff640 1 -- 172.21.15.103:0/3246160690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a71ffe030 msgr2=0x556a72010d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:41.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78ebfff640 1 --2- 172.21.15.103:0/3246160690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a71ffe030 0x556a72010d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78ebfff640 1 -- 172.21.15.103:0/3246160690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a7200ce40 msgr2=0x556a720112c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:41.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78ebfff640 1 --2- 172.21.15.103:0/3246160690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a7200ce40 0x556a720112c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78ebfff640 1 -- 172.21.15.103:0/3246160690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a72086b40 con 0x556a72024e60 2024-09-19T06:50:41.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.539+0000 7f78eaffd640 1 --2- 172.21.15.103:0/3246160690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a7200ce40 0x556a720112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:41.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.540+0000 7f78f4604740 1 -- 172.21.15.103:0/3246160690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78e4034040 con 0x556a72024e60 2024-09-19T06:50:41.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.540+0000 7f78ebfff640 1 --2- 172.21.15.103:0/3246160690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a72024e60 0x556a72011800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f78dc0ccb10 tx=0x7f78dc09b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d510c5fa2c2bd4f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:41.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.540+0000 7f78caffd640 10 client.25406 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:50:41.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.540+0000 7f78caffd640 1 -- 172.21.15.103:0/3246160690 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc09b6a0 con 0x556a72024e60 2024-09-19T06:50:41.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.540+0000 7f78caffd640 1 -- 172.21.15.103:0/3246160690 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f78dc097d10 con 0x556a72024e60 2024-09-19T06:50:41.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.540+0000 7f78caffd640 1 -- 172.21.15.103:0/3246160690 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f78dc09fa50 con 0x556a72024e60 2024-09-19T06:50:41.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.540+0000 7f78f4604740 10 client.25406 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:41.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.540+0000 7f78f4604740 1 -- 172.21.15.103:0/3246160690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=27}) -- 0x556a72087c70 con 0x556a72024e60 2024-09-19T06:50:41.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.540+0000 7f78caffd640 1 -- 172.21.15.103:0/3246160690 <== mon.2 v2:172.21.15.134:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f78dc0afbc0 con 0x556a72024e60 2024-09-19T06:50:41.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.540+0000 7f78f4604740 10 client.25406 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:41.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.540+0000 7f78f4604740 10 client.25406 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:50:41.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.540+0000 7f78f4604740 20 client.25406 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:41.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.540+0000 7f78f4604740 1 --2- 172.21.15.103:0/3246160690 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556a72088300 0x556a7208a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.541+0000 7f78f4604740 4 client.25406 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T06:50:41.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.541+0000 7f78f4604740 1 -- 172.21.15.103:0/3246160690 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x556a71f69be0 con 0x556a72088300 2024-09-19T06:50:41.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.541+0000 7f78eb7fe640 1 --2- 172.21.15.103:0/3246160690 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556a72088300 0x556a7208a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.541+0000 7f78caffd640 10 client.25406 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:41.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.541+0000 7f78eb7fe640 1 --2- 172.21.15.103:0/3246160690 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556a72088300 0x556a7208a720 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c95ab8062067a2c3 server_cookie=8ce03f3f793222e7 in_seq=0 out_seq=0 2024-09-19T06:50:41.555 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.693+0000 7f78caffd640 1 -- 172.21.15.103:0/3246160690 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x556a71f69be0 con 0x556a72088300 2024-09-19T06:50:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.693+0000 7f78caffd640 10 client.25406 handle_command_reply: tid=0 2024-09-19T06:50:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.693+0000 7f78f4604740 1 client.25406 shutdown 2024-09-19T06:50:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.693+0000 7f78f4604740 1 -- 172.21.15.103:0/3246160690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a72024e60 msgr2=0x556a72011800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.693+0000 7f78f4604740 1 --2- 172.21.15.103:0/3246160690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a72024e60 0x556a72011800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f78dc0ccb10 tx=0x7f78dc09b560 comp rx=0 tx=0).stop 2024-09-19T06:50:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.693+0000 7f78f4604740 1 -- 172.21.15.103:0/3246160690 shutdown_connections 2024-09-19T06:50:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.693+0000 7f78f4604740 1 --2- 172.21.15.103:0/3246160690 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556a72088300 0x556a7208a720 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.693+0000 7f78f4604740 1 --2- 172.21.15.103:0/3246160690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a72024e60 0x556a72011800 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.693+0000 7f78caffd640 0 client.25406 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.693+0000 7f78f4604740 1 --2- 172.21.15.103:0/3246160690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a7200ce40 0x556a720112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.693+0000 7f78f4604740 1 --2- 172.21.15.103:0/3246160690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a71ffe030 0x556a72010d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.693+0000 7f78f4604740 1 -- 172.21.15.103:0/3246160690 >> 172.21.15.103:0/3246160690 conn(0x556a71ff9de0 msgr2=0x556a71ff8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:41.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.693+0000 7f78f4604740 1 -- 172.21.15.103:0/3246160690 shutdown_connections 2024-09-19T06:50:41.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.693+0000 7f78f4604740 1 -- 172.21.15.103:0/3246160690 wait complete. 2024-09-19T06:50:41.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.693+0000 7f78f4604740 20 client.25406 trim_cache size 0 max 16384 2024-09-19T06:50:41.696 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:50:41.696 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:50:41.696 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:50:41.697 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 6, 2024-09-19T06:50:41.697 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:50:41.697 INFO:teuthology.orchestra.run.smithi103.stdout: "51253b89": { 2024-09-19T06:50:41.697 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 6, 2024-09-19T06:50:41.697 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:50:41.697 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:50:41.697 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:50:41.697 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:50:41.697 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:50:41.697 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:50:41.698 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 140.3, 2024-09-19T06:50:41.698 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:50:41.698 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:50:41.698 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:50:41.698 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:50:41.698 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:50:41.698 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:50:41.698 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:41.698 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:41.698 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:41.698 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:41.698 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:41.698 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:50:41.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.694+0000 7f78f222b640 1 -- 172.21.15.103:0/3848135720 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f78d80a3560 msgr2=0x7f78d80a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.694+0000 7f78f222b640 1 --2- 172.21.15.103:0/3848135720 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f78d80a3560 0x7f78d80a5a20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f78e4005fd0 tx=0x7f78e4009760 comp rx=0 tx=0).stop 2024-09-19T06:50:41.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.694+0000 7f78f222b640 1 -- 172.21.15.103:0/3848135720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78ec106f10 msgr2=0x7f78ec19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.694+0000 7f78f222b640 1 --2- 172.21.15.103:0/3848135720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78ec106f10 0x7f78ec19cd40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009fd0 tx=0x7f78dc00b8e0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.694+0000 7f78f222b640 1 -- 172.21.15.103:0/3848135720 shutdown_connections 2024-09-19T06:50:41.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.694+0000 7f78f222b640 1 --2- 172.21.15.103:0/3848135720 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f78d80a3560 0x7f78d80a5a20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.694+0000 7f78f222b640 1 --2- 172.21.15.103:0/3848135720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78ec106f10 0x7f78ec19cd40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.694+0000 7f78f222b640 1 --2- 172.21.15.103:0/3848135720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78ec102790 0x7f78ec1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.694+0000 7f78f222b640 1 --2- 172.21.15.103:0/3848135720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78ec101de0 0x7f78ec198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.694+0000 7f78f222b640 1 -- 172.21.15.103:0/3848135720 >> 172.21.15.103:0/3848135720 conn(0x7f78ec0fd8c0 msgr2=0x7f78ec0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:41.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.694+0000 7f78f222b640 1 -- 172.21.15.103:0/3848135720 shutdown_connections 2024-09-19T06:50:41.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:41.694+0000 7f78f222b640 1 -- 172.21.15.103:0/3848135720 wait complete. 2024-09-19T06:50:42.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:42 smithi174 ceph-mon[35637]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:50:42.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:42 smithi103 ceph-mon[31158]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:50:42.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:42 smithi134 ceph-mon[35356]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:50:42.587 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '51253b89', quiesce took 5.9 seconds. Will release after: 14.450000000000001 2024-09-19T06:50:42.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.775+0000 7f9df8a38640 1 -- 172.21.15.103:0/2001536983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9df4101240 msgr2=0x7f9df4101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:42.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.775+0000 7f9df8a38640 1 --2- 172.21.15.103:0/2001536983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9df4101240 0x7f9df4101680 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009900 tx=0x7f9de802a410 comp rx=0 tx=0).stop 2024-09-19T06:50:42.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.775+0000 7f9df8a38640 1 -- 172.21.15.103:0/2001536983 shutdown_connections 2024-09-19T06:50:42.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.775+0000 7f9df8a38640 1 --2- 172.21.15.103:0/2001536983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9df4101bc0 0x7f9df410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.775+0000 7f9df8a38640 1 --2- 172.21.15.103:0/2001536983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9df4101240 0x7f9df4101680 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.775+0000 7f9df8a38640 1 --2- 172.21.15.103:0/2001536983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9df4105a90 0x7f9df4105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.775+0000 7f9df8a38640 1 -- 172.21.15.103:0/2001536983 >> 172.21.15.103:0/2001536983 conn(0x7f9df4077210 msgr2=0x7f9df40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:42.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.775+0000 7f9df8a38640 1 -- 172.21.15.103:0/2001536983 shutdown_connections 2024-09-19T06:50:42.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.775+0000 7f9df8a38640 1 -- 172.21.15.103:0/2001536983 wait complete. 2024-09-19T06:50:42.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.776+0000 7f9df8a38640 1 Processor -- start 2024-09-19T06:50:42.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.776+0000 7f9df8a38640 1 -- start start 2024-09-19T06:50:42.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.776+0000 7f9df8a38640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9df4101240 0x7f9df4197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.776+0000 7f9df8a38640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9df4101bc0 0x7f9df4197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.776+0000 7f9df8a38640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9df4105a90 0x7f9df4197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.776+0000 7f9df8a38640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9df4111030 con 0x7f9df4101240 2024-09-19T06:50:42.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.776+0000 7f9df8a38640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9df4110eb0 con 0x7f9df4101bc0 2024-09-19T06:50:42.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.776+0000 7f9df8a38640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41111b0 con 0x7f9df4105a90 2024-09-19T06:50:42.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.776+0000 7f9df259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9df4101240 0x7f9df4197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:42.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.776+0000 7f9df259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9df4101240 0x7f9df4197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59214/0 (socket says 172.21.15.103:59214) 2024-09-19T06:50:42.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.777+0000 7f9df259c640 1 -- 172.21.15.103:0/2752703067 learned_addr learned my addr 172.21.15.103:0/2752703067 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:42.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.777+0000 7f9df1d9b640 1 --2- 172.21.15.103:0/2752703067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9df4101bc0 0x7f9df4197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:42.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.777+0000 7f9df259c640 1 -- 172.21.15.103:0/2752703067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9df4105a90 msgr2=0x7f9df4197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:42.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.777+0000 7f9df2d9d640 1 --2- 172.21.15.103:0/2752703067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9df4105a90 0x7f9df4197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:42.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.777+0000 7f9df259c640 1 --2- 172.21.15.103:0/2752703067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9df4105a90 0x7f9df4197cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.777+0000 7f9df259c640 1 -- 172.21.15.103:0/2752703067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9df4101bc0 msgr2=0x7f9df4197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:42.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.777+0000 7f9df259c640 1 --2- 172.21.15.103:0/2752703067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9df4101bc0 0x7f9df4197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.777+0000 7f9df259c640 1 -- 172.21.15.103:0/2752703067 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df41a6ac0 con 0x7f9df4101240 2024-09-19T06:50:42.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.777+0000 7f9df1d9b640 1 --2- 172.21.15.103:0/2752703067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9df4101bc0 0x7f9df4197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:42.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.777+0000 7f9df259c640 1 --2- 172.21.15.103:0/2752703067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9df4101240 0x7f9df4197250 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9de400b4a0 tx=0x7f9de400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:42.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.777+0000 7f9de37fe640 1 -- 172.21.15.103:0/2752703067 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4014070 con 0x7f9df4101240 2024-09-19T06:50:42.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.777+0000 7f9df8a38640 1 -- 172.21.15.103:0/2752703067 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df41a6db0 con 0x7f9df4101240 2024-09-19T06:50:42.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.777+0000 7f9de37fe640 1 -- 172.21.15.103:0/2752703067 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9de4010af0 con 0x7f9df4101240 2024-09-19T06:50:42.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.777+0000 7f9de37fe640 1 -- 172.21.15.103:0/2752703067 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4017d10 con 0x7f9df4101240 2024-09-19T06:50:42.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.777+0000 7f9df8a38640 1 -- 172.21.15.103:0/2752703067 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df41a7270 con 0x7f9df4101240 2024-09-19T06:50:42.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.779+0000 7f9de37fe640 1 -- 172.21.15.103:0/2752703067 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7f9de4017070 con 0x7f9df4101240 2024-09-19T06:50:42.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.780+0000 7f9dfae11740 1 Processor -- start 2024-09-19T06:50:42.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.780+0000 7f9dfae11740 1 -- start start 2024-09-19T06:50:42.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.780+0000 7f9dfae11740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98c6b40 0x5604d98c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.780+0000 7f9dfae11740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c55d0 0x5604d98c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.780+0000 7f9dfae11740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98d0370 0x5604d98d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.780+0000 7f9dfae11740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5604d982b2f0 con 0x5604d98d0370 2024-09-19T06:50:42.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.780+0000 7f9dfae11740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5604d97d2cf0 con 0x5604d98c6b40 2024-09-19T06:50:42.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.780+0000 7f9dfae11740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5604d98272b0 con 0x5604d98c55d0 2024-09-19T06:50:42.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.780+0000 7f9df1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98d0370 0x5604d98d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:42.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.780+0000 7f9df1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98d0370 0x5604d98d2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59226/0 (socket says 172.21.15.103:59226) 2024-09-19T06:50:42.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.780+0000 7f9df1d9b640 1 -- 172.21.15.103:0/3453868101 learned_addr learned my addr 172.21.15.103:0/3453868101 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:42.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.780+0000 7f9de37fe640 1 --2- 172.21.15.103:0/2752703067 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9ddc0a3560 0x7f9ddc0a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.781+0000 7f9df2d9d640 1 --2- 172.21.15.103:0/3453868101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98c6b40 0x5604d98c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:42.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.781+0000 7f9df259c640 1 --2- 172.21.15.103:0/3453868101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c55d0 0x5604d98c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:42.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.781+0000 7f9df2d9d640 1 --2- 172.21.15.103:0/2752703067 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9ddc0a3560 0x7f9ddc0a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:42.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.781+0000 7f9de37fe640 1 -- 172.21.15.103:0/2752703067 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f9de40cf3c0 con 0x7f9df4101240 2024-09-19T06:50:42.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.781+0000 7f9df1d9b640 1 -- 172.21.15.103:0/3453868101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c55d0 msgr2=0x5604d98c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:42.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.781+0000 7f9df1d9b640 1 --2- 172.21.15.103:0/3453868101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c55d0 0x5604d98c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.781+0000 7f9df1d9b640 1 -- 172.21.15.103:0/3453868101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98c6b40 msgr2=0x5604d98c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:42.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.781+0000 7f9df1d9b640 1 --2- 172.21.15.103:0/3453868101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98c6b40 0x5604d98c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.781+0000 7f9df1d9b640 1 -- 172.21.15.103:0/3453868101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604d97b73a0 con 0x5604d98d0370 2024-09-19T06:50:42.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.781+0000 7f9df1d9b640 1 --2- 172.21.15.103:0/3453868101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98d0370 0x5604d98d2760 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9de8005ab0 tx=0x7f9de8004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2665be05a9a668c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:42.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.781+0000 7f9df2d9d640 1 --2- 172.21.15.103:0/2752703067 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9ddc0a3560 0x7f9ddc0a5a20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9dec014960 tx=0x7f9dec005df0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:42.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.781+0000 7f9de0ff9640 1 -- 172.21.15.103:0/3453868101 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de803f070 con 0x5604d98d0370 2024-09-19T06:50:42.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.781+0000 7f9de0ff9640 1 -- 172.21.15.103:0/3453868101 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9de8005520 con 0x5604d98d0370 2024-09-19T06:50:42.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.781+0000 7f9de0ff9640 1 -- 172.21.15.103:0/3453868101 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de8035030 con 0x5604d98d0370 2024-09-19T06:50:42.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.782+0000 7f9dfae11740 1 -- 172.21.15.103:0/3453868101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98d0370 msgr2=0x5604d98d2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:42.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.782+0000 7f9dfae11740 1 --2- 172.21.15.103:0/3453868101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98d0370 0x5604d98d2760 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9de8005ab0 tx=0x7f9de8004200 comp rx=0 tx=0).stop 2024-09-19T06:50:42.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.782+0000 7f9dfae11740 1 -- 172.21.15.103:0/3453868101 shutdown_connections 2024-09-19T06:50:42.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.782+0000 7f9dfae11740 1 --2- 172.21.15.103:0/3453868101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98d0370 0x5604d98d2760 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.782+0000 7f9dfae11740 1 --2- 172.21.15.103:0/3453868101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c55d0 0x5604d98c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.782+0000 7f9dfae11740 1 --2- 172.21.15.103:0/3453868101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98c6b40 0x5604d98c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.782+0000 7f9dfae11740 1 -- 172.21.15.103:0/3453868101 >> 172.21.15.103:0/3453868101 conn(0x5604d988e310 msgr2=0x5604d98c1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:42.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.782+0000 7f9dfae11740 1 -- 172.21.15.103:0/3453868101 shutdown_connections 2024-09-19T06:50:42.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.782+0000 7f9dfae11740 1 -- 172.21.15.103:0/3453868101 wait complete. 2024-09-19T06:50:42.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.782+0000 7f9dfae11740 1 -- start start 2024-09-19T06:50:42.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.782+0000 7f9dfae11740 1 Processor -- start 2024-09-19T06:50:42.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9dfae11740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c55d0 0x5604d98cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9dfae11740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98c6b40 0x5604d98ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9dfae11740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98d0370 0x5604d98cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9dfae11740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5604d98d3010 con 0x5604d98d0370 2024-09-19T06:50:42.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9dfae11740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5604d97e6060 con 0x5604d98c6b40 2024-09-19T06:50:42.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9dfae11740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5604d98d3190 con 0x5604d98c55d0 2024-09-19T06:50:42.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9dfae11740 10 client.0 _ll_register_callbacks cb 0x5604d95398c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:42.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9df2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98d0370 0x5604d98cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:42.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9df2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98d0370 0x5604d98cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59230/0 (socket says 172.21.15.103:59230) 2024-09-19T06:50:42.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9df2d9d640 1 -- 172.21.15.103:0/1774711407 learned_addr learned my addr 172.21.15.103:0/1774711407 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:42.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9df259c640 1 --2- 172.21.15.103:0/1774711407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c55d0 0x5604d98cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:42.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9df1d9b640 1 --2- 172.21.15.103:0/1774711407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98c6b40 0x5604d98ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:42.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9df2d9d640 1 -- 172.21.15.103:0/1774711407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c55d0 msgr2=0x5604d98cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:42.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9df2d9d640 1 --2- 172.21.15.103:0/1774711407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c55d0 0x5604d98cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9df2d9d640 1 -- 172.21.15.103:0/1774711407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98c6b40 msgr2=0x5604d98ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:42.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9df2d9d640 1 --2- 172.21.15.103:0/1774711407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98c6b40 0x5604d98ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9df2d9d640 1 -- 172.21.15.103:0/1774711407 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604d959c3c0 con 0x5604d98d0370 2024-09-19T06:50:42.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9df2d9d640 1 --2- 172.21.15.103:0/1774711407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98d0370 0x5604d98cd4d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9dec01a700 tx=0x7f9dec004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e777aef12874ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:42.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9dd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:42.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9dd9ffb640 1 -- 172.21.15.103:0/1774711407 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec028060 con 0x5604d98d0370 2024-09-19T06:50:42.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9dd9ffb640 1 -- 172.21.15.103:0/1774711407 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f9dec014e20 con 0x5604d98d0370 2024-09-19T06:50:42.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.783+0000 7f9dfae11740 1 -- 172.21.15.103:0/1774711407 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9de8005520 con 0x5604d98d0370 2024-09-19T06:50:42.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.784+0000 7f9dd9ffb640 1 -- 172.21.15.103:0/1774711407 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9dec018430 con 0x5604d98d0370 2024-09-19T06:50:42.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.784+0000 7f9dfae11740 10 client.16044 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:42.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.784+0000 7f9dfae11740 1 -- 172.21.15.103:0/1774711407 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5604d98dea20 con 0x5604d98d0370 2024-09-19T06:50:42.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.784+0000 7f9dd9ffb640 1 -- 172.21.15.103:0/1774711407 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9de8005520 con 0x5604d98d0370 2024-09-19T06:50:42.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.784+0000 7f9dd9ffb640 1 -- 172.21.15.103:0/1774711407 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9dec023410 con 0x5604d98d0370 2024-09-19T06:50:42.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.784+0000 7f9dfae11740 10 client.16044 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:42.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.784+0000 7f9dfae11740 10 client.16044 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:50:42.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.784+0000 7f9dfae11740 20 client.16044 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:42.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.784+0000 7f9dfae11740 1 --2- 172.21.15.103:0/1774711407 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5604d98dfee0 0x5604d98e2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.784+0000 7f9dfae11740 4 client.16044 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:42.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.784+0000 7f9dfae11740 1 -- 172.21.15.103:0/1774711407 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604d9838860 con 0x5604d98dfee0 2024-09-19T06:50:42.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.784+0000 7f9df259c640 1 --2- 172.21.15.103:0/1774711407 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5604d98dfee0 0x5604d98e2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:42.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.785+0000 7f9dd9ffb640 10 client.16044 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:50:42.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.785+0000 7f9df259c640 1 --2- 172.21.15.103:0/1774711407 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5604d98dfee0 0x5604d98e2310 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79a7daab0f6c7b76 server_cookie=d442cff80bbf6fe7 in_seq=0 out_seq=0 2024-09-19T06:50:42.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.786+0000 7f9dd9ffb640 1 -- 172.21.15.103:0/1774711407 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604d9838860 con 0x5604d98dfee0 2024-09-19T06:50:42.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.786+0000 7f9dd9ffb640 10 client.16044 handle_command_reply: tid=0 2024-09-19T06:50:42.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.786+0000 7f9dfae11740 1 client.16044 shutdown 2024-09-19T06:50:42.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.786+0000 7f9dfae11740 1 -- 172.21.15.103:0/1774711407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98d0370 msgr2=0x5604d98cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:42.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.786+0000 7f9dfae11740 1 --2- 172.21.15.103:0/1774711407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98d0370 0x5604d98cd4d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9dec01a700 tx=0x7f9dec004b50 comp rx=0 tx=0).stop 2024-09-19T06:50:42.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.786+0000 7f9dfae11740 1 -- 172.21.15.103:0/1774711407 shutdown_connections 2024-09-19T06:50:42.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.786+0000 7f9dfae11740 1 --2- 172.21.15.103:0/1774711407 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5604d98dfee0 0x5604d98e2310 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.786+0000 7f9dfae11740 1 --2- 172.21.15.103:0/1774711407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98d0370 0x5604d98cd4d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.786+0000 7f9dd9ffb640 0 client.16044 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:50:42.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.786+0000 7f9dfae11740 1 --2- 172.21.15.103:0/1774711407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98c6b40 0x5604d98ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.786+0000 7f9dfae11740 1 --2- 172.21.15.103:0/1774711407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c55d0 0x5604d98cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.786+0000 7f9dfae11740 1 -- 172.21.15.103:0/1774711407 >> 172.21.15.103:0/1774711407 conn(0x5604d988e310 msgr2=0x5604d98c7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:42.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.786+0000 7f9dfae11740 1 -- 172.21.15.103:0/1774711407 shutdown_connections 2024-09-19T06:50:42.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.786+0000 7f9dfae11740 1 -- 172.21.15.103:0/1774711407 wait complete. 2024-09-19T06:50:42.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.786+0000 7f9dfae11740 20 client.16044 trim_cache size 0 max 16384 2024-09-19T06:50:42.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9dfae11740 1 Processor -- start 2024-09-19T06:50:42.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9dfae11740 1 -- start start 2024-09-19T06:50:42.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9dfae11740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98c55d0 0x5604d98dc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9dfae11740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c6b40 0x5604d98c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9dfae11740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98d0370 0x5604d98e7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9dfae11740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41afc80 con 0x5604d98c55d0 2024-09-19T06:50:42.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9dfae11740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41b17c0 con 0x5604d98d0370 2024-09-19T06:50:42.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9dfae11740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41b0350 con 0x5604d98c6b40 2024-09-19T06:50:42.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9df259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98c55d0 0x5604d98dc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:42.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9df259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98c55d0 0x5604d98dc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59244/0 (socket says 172.21.15.103:59244) 2024-09-19T06:50:42.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9df259c640 1 -- 172.21.15.103:0/1575888844 learned_addr learned my addr 172.21.15.103:0/1575888844 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:42.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9df259c640 1 -- 172.21.15.103:0/1575888844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c6b40 msgr2=0x5604d98c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:42.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9df259c640 1 --2- 172.21.15.103:0/1575888844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c6b40 0x5604d98c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9df259c640 1 -- 172.21.15.103:0/1575888844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98d0370 msgr2=0x5604d98e7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:42.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9df259c640 1 --2- 172.21.15.103:0/1575888844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98d0370 0x5604d98e7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9df259c640 1 -- 172.21.15.103:0/1575888844 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604d98cb4c0 con 0x5604d98c55d0 2024-09-19T06:50:42.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.799+0000 7f9df259c640 1 --2- 172.21.15.103:0/1575888844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98c55d0 0x5604d98dc2b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9de4017fd0 tx=0x7f9de4066280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94050859dcbcdb64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:42.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.800+0000 7f9ddbfff640 1 -- 172.21.15.103:0/1575888844 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de409b4a0 con 0x5604d98c55d0 2024-09-19T06:50:42.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.800+0000 7f9ddbfff640 1 -- 172.21.15.103:0/1575888844 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9de409f850 con 0x5604d98c55d0 2024-09-19T06:50:42.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.800+0000 7f9ddbfff640 1 -- 172.21.15.103:0/1575888844 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4097b10 con 0x5604d98c55d0 2024-09-19T06:50:42.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.800+0000 7f9dfae11740 1 -- 172.21.15.103:0/1575888844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98c55d0 msgr2=0x5604d98dc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:42.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.800+0000 7f9dfae11740 1 --2- 172.21.15.103:0/1575888844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98c55d0 0x5604d98dc2b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9de4017fd0 tx=0x7f9de4066280 comp rx=0 tx=0).stop 2024-09-19T06:50:42.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.800+0000 7f9dfae11740 1 -- 172.21.15.103:0/1575888844 shutdown_connections 2024-09-19T06:50:42.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.800+0000 7f9dfae11740 1 --2- 172.21.15.103:0/1575888844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98d0370 0x5604d98e7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.800+0000 7f9dfae11740 1 --2- 172.21.15.103:0/1575888844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c6b40 0x5604d98c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.800+0000 7f9dfae11740 1 --2- 172.21.15.103:0/1575888844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98c55d0 0x5604d98dc2b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.800+0000 7f9dfae11740 1 -- 172.21.15.103:0/1575888844 >> 172.21.15.103:0/1575888844 conn(0x5604d988e310 msgr2=0x5604d98cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:42.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.800+0000 7f9dfae11740 1 -- 172.21.15.103:0/1575888844 shutdown_connections 2024-09-19T06:50:42.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.800+0000 7f9dfae11740 1 -- 172.21.15.103:0/1575888844 wait complete. 2024-09-19T06:50:42.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.801+0000 7f9dfae11740 1 -- start start 2024-09-19T06:50:42.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.801+0000 7f9dfae11740 1 Processor -- start 2024-09-19T06:50:42.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.801+0000 7f9dfae11740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98c55d0 0x5604d98d4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.801+0000 7f9dfae11740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c6b40 0x5604d98d4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.801+0000 7f9dfae11740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98d0370 0x5604d9949bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.801+0000 7f9dfae11740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41b10f0 con 0x5604d98c55d0 2024-09-19T06:50:42.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.801+0000 7f9dfae11740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41b0a20 con 0x5604d98d0370 2024-09-19T06:50:42.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.801+0000 7f9dfae11740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9df41adc40 con 0x5604d98c6b40 2024-09-19T06:50:42.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9dfae11740 10 client.0 _ll_register_callbacks cb 0x7f9dec008070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:42.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9df259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98c55d0 0x5604d98d4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:42.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9df259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98c55d0 0x5604d98d4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59256/0 (socket says 172.21.15.103:59256) 2024-09-19T06:50:42.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9df2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98d0370 0x5604d9949bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:42.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9df259c640 1 -- 172.21.15.103:0/912554291 learned_addr learned my addr 172.21.15.103:0/912554291 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:42.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9df1d9b640 1 --2- 172.21.15.103:0/912554291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c6b40 0x5604d98d4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:42.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9df259c640 1 -- 172.21.15.103:0/912554291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c6b40 msgr2=0x5604d98d4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:42.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9df259c640 1 --2- 172.21.15.103:0/912554291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c6b40 0x5604d98d4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9df259c640 1 -- 172.21.15.103:0/912554291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98d0370 msgr2=0x5604d9949bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:42.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9df259c640 1 --2- 172.21.15.103:0/912554291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98d0370 0x5604d9949bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9df259c640 1 -- 172.21.15.103:0/912554291 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604d98d5350 con 0x5604d98c55d0 2024-09-19T06:50:42.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9df259c640 1 --2- 172.21.15.103:0/912554291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98c55d0 0x5604d98d4510 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9de4097900 tx=0x7f9de40cf350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a82f1b3ab96ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:42.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9dd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:42.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9dd9ffb640 1 -- 172.21.15.103:0/912554291 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de40b4060 con 0x5604d98c55d0 2024-09-19T06:50:42.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9dd9ffb640 1 -- 172.21.15.103:0/912554291 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f9de4010ca0 con 0x5604d98c55d0 2024-09-19T06:50:42.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9dfae11740 1 -- 172.21.15.103:0/912554291 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9de409f850 con 0x5604d98c55d0 2024-09-19T06:50:42.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9dd9ffb640 1 -- 172.21.15.103:0/912554291 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9de409b4a0 con 0x5604d98c55d0 2024-09-19T06:50:42.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9dfae11740 10 client.16050 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:42.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.802+0000 7f9dfae11740 1 -- 172.21.15.103:0/912554291 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5604d994b220 con 0x5604d98c55d0 2024-09-19T06:50:42.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.803+0000 7f9dd9ffb640 1 -- 172.21.15.103:0/912554291 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9de409f850 con 0x5604d98c55d0 2024-09-19T06:50:42.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.803+0000 7f9dd9ffb640 1 -- 172.21.15.103:0/912554291 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9de40b4200 con 0x5604d98c55d0 2024-09-19T06:50:42.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.803+0000 7f9dfae11740 10 client.16050 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:42.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.803+0000 7f9dfae11740 10 client.16050 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:50:42.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.803+0000 7f9dfae11740 20 client.16050 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:42.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.803+0000 7f9dfae11740 1 --2- 172.21.15.103:0/912554291 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5604d98dfee0 0x5604d994d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:42.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.803+0000 7f9dfae11740 4 client.16050 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:42.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.803+0000 7f9dfae11740 1 -- 172.21.15.103:0/912554291 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604d9832d70 con 0x5604d98dfee0 2024-09-19T06:50:42.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.803+0000 7f9df1d9b640 1 --2- 172.21.15.103:0/912554291 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5604d98dfee0 0x5604d994d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:42.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9df1d9b640 1 --2- 172.21.15.103:0/912554291 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5604d98dfee0 0x5604d994d910 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=742ee03412492a73 server_cookie=4d030b29d1990dbc in_seq=0 out_seq=0 2024-09-19T06:50:42.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9dd9ffb640 10 client.16050 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:50:42.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9dd9ffb640 1 -- 172.21.15.103:0/912554291 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604d9832d70 con 0x5604d98dfee0 2024-09-19T06:50:42.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9dd9ffb640 10 client.16050 handle_command_reply: tid=0 2024-09-19T06:50:42.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9dfae11740 1 client.16050 shutdown 2024-09-19T06:50:42.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9dfae11740 1 -- 172.21.15.103:0/912554291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98c55d0 msgr2=0x5604d98d4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:42.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9dfae11740 1 --2- 172.21.15.103:0/912554291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98c55d0 0x5604d98d4510 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9de4097900 tx=0x7f9de40cf350 comp rx=0 tx=0).stop 2024-09-19T06:50:42.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9dfae11740 1 -- 172.21.15.103:0/912554291 shutdown_connections 2024-09-19T06:50:42.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9dfae11740 1 --2- 172.21.15.103:0/912554291 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5604d98dfee0 0x5604d994d910 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9dfae11740 1 --2- 172.21.15.103:0/912554291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604d98d0370 0x5604d9949bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9dfae11740 1 --2- 172.21.15.103:0/912554291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604d98c6b40 0x5604d98d4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9dfae11740 1 --2- 172.21.15.103:0/912554291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604d98c55d0 0x5604d98d4510 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9dfae11740 1 -- 172.21.15.103:0/912554291 >> 172.21.15.103:0/912554291 conn(0x5604d988e310 msgr2=0x5604d98c7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:42.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9dd9ffb640 0 client.16050 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:50:42.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9dfae11740 1 -- 172.21.15.103:0/912554291 shutdown_connections 2024-09-19T06:50:42.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9dfae11740 1 -- 172.21.15.103:0/912554291 wait complete. 2024-09-19T06:50:42.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.804+0000 7f9dfae11740 20 client.16050 trim_cache size 0 max 16384 2024-09-19T06:50:42.821 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:50:42.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.805+0000 7f9df8a38640 1 -- 172.21.15.103:0/2752703067 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9ddc0a3560 msgr2=0x7f9ddc0a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:42.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.805+0000 7f9df8a38640 1 --2- 172.21.15.103:0/2752703067 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9ddc0a3560 0x7f9ddc0a5a20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9dec014960 tx=0x7f9dec005df0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.805+0000 7f9df8a38640 1 -- 172.21.15.103:0/2752703067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9df4101240 msgr2=0x7f9df4197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:42.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.805+0000 7f9df8a38640 1 --2- 172.21.15.103:0/2752703067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9df4101240 0x7f9df4197250 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9de400b4a0 tx=0x7f9de400b960 comp rx=0 tx=0).stop 2024-09-19T06:50:42.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.805+0000 7f9df8a38640 1 -- 172.21.15.103:0/2752703067 shutdown_connections 2024-09-19T06:50:42.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.805+0000 7f9df8a38640 1 --2- 172.21.15.103:0/2752703067 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9ddc0a3560 0x7f9ddc0a5a20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.805+0000 7f9df8a38640 1 --2- 172.21.15.103:0/2752703067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9df4105a90 0x7f9df4197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.805+0000 7f9df8a38640 1 --2- 172.21.15.103:0/2752703067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9df4101bc0 0x7f9df4197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.805+0000 7f9df8a38640 1 --2- 172.21.15.103:0/2752703067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9df4101240 0x7f9df4197250 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:42.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.805+0000 7f9df8a38640 1 -- 172.21.15.103:0/2752703067 >> 172.21.15.103:0/2752703067 conn(0x7f9df4077210 msgr2=0x7f9df4103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:42.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.805+0000 7f9df8a38640 1 -- 172.21.15.103:0/2752703067 shutdown_connections 2024-09-19T06:50:42.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:42.805+0000 7f9df8a38640 1 -- 172.21.15.103:0/2752703067 wait complete. 2024-09-19T06:50:43.552 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:50:43.801 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.159+0000 7fed19a8b640 1 -- 172.21.15.103:0/736455753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed14101ac0 msgr2=0x7fed14101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.159+0000 7fed19a8b640 1 --2- 172.21.15.103:0/736455753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed14101ac0 0x7fed14101f20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fed08009930 tx=0x7fed0802a420 comp rx=0 tx=0).stop 2024-09-19T06:50:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.159+0000 7fed19a8b640 1 -- 172.21.15.103:0/736455753 shutdown_connections 2024-09-19T06:50:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.159+0000 7fed19a8b640 1 --2- 172.21.15.103:0/736455753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed14102460 0x7fed1410e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.159+0000 7fed19a8b640 1 --2- 172.21.15.103:0/736455753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed14101ac0 0x7fed14101f20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.159+0000 7fed19a8b640 1 --2- 172.21.15.103:0/736455753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed14107ad0 0x7fed14107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.159+0000 7fed19a8b640 1 -- 172.21.15.103:0/736455753 >> 172.21.15.103:0/736455753 conn(0x7fed140fd820 msgr2=0x7fed140ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.159+0000 7fed19a8b640 1 -- 172.21.15.103:0/736455753 shutdown_connections 2024-09-19T06:50:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.159+0000 7fed19a8b640 1 -- 172.21.15.103:0/736455753 wait complete. 2024-09-19T06:50:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.159+0000 7fed19a8b640 1 Processor -- start 2024-09-19T06:50:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.159+0000 7fed19a8b640 1 -- start start 2024-09-19T06:50:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed19a8b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed14101ac0 0x7fed140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed19a8b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed14102460 0x7fed14074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed19a8b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed14107ad0 0x7fed14074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed19a8b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed14111f60 con 0x7fed14101ac0 2024-09-19T06:50:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed19a8b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed14111de0 con 0x7fed14107ad0 2024-09-19T06:50:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed19a8b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed141120e0 con 0x7fed14102460 2024-09-19T06:50:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed137fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed14101ac0 0x7fed140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed137fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed14101ac0 0x7fed140744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59282/0 (socket says 172.21.15.103:59282) 2024-09-19T06:50:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed137fe640 1 -- 172.21.15.103:0/2135283534 learned_addr learned my addr 172.21.15.103:0/2135283534 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed12ffd640 1 --2- 172.21.15.103:0/2135283534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed14102460 0x7fed14074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed13fff640 1 --2- 172.21.15.103:0/2135283534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed14107ad0 0x7fed14074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed137fe640 1 -- 172.21.15.103:0/2135283534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed14102460 msgr2=0x7fed14074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed137fe640 1 --2- 172.21.15.103:0/2135283534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed14102460 0x7fed14074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed137fe640 1 -- 172.21.15.103:0/2135283534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed14107ad0 msgr2=0x7fed14074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed137fe640 1 --2- 172.21.15.103:0/2135283534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed14107ad0 0x7fed14074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed137fe640 1 -- 172.21.15.103:0/2135283534 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed141a6890 con 0x7fed14101ac0 2024-09-19T06:50:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed137fe640 1 --2- 172.21.15.103:0/2135283534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed14101ac0 0x7fed140744f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fed0400d770 tx=0x7fed0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed10ff9640 1 -- 172.21.15.103:0/2135283534 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04017070 con 0x7fed14101ac0 2024-09-19T06:50:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed10ff9640 1 -- 172.21.15.103:0/2135283534 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed04005500 con 0x7fed14101ac0 2024-09-19T06:50:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed19a8b640 1 -- 172.21.15.103:0/2135283534 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed141a6b80 con 0x7fed14101ac0 2024-09-19T06:50:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.160+0000 7fed10ff9640 1 -- 172.21.15.103:0/2135283534 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04004e00 con 0x7fed14101ac0 2024-09-19T06:50:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.161+0000 7fed19a8b640 1 -- 172.21.15.103:0/2135283534 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed141a7050 con 0x7fed14101ac0 2024-09-19T06:50:44.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:44 smithi174 ceph-mon[35637]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:50:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.163+0000 7fed10ff9640 1 -- 172.21.15.103:0/2135283534 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7fed04004fa0 con 0x7fed14101ac0 2024-09-19T06:50:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.163+0000 7fed1c824740 1 Processor -- start 2024-09-19T06:50:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.163+0000 7fed1c824740 1 -- start start 2024-09-19T06:50:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.163+0000 7fed1c824740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a3af0 0x556adb3a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.163+0000 7fed1c824740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3a4410 0x556adb3ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.163+0000 7fed1c824740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3aecb0 0x556adb3b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.163+0000 7fed1c824740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556adb3082f0 con 0x556adb3a3af0 2024-09-19T06:50:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.163+0000 7fed1c824740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556adb2b0270 con 0x556adb3aecb0 2024-09-19T06:50:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.163+0000 7fed1c824740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556adb3042b0 con 0x556adb3a4410 2024-09-19T06:50:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.163+0000 7fed13fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a3af0 0x556adb3a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.163+0000 7fed13fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a3af0 0x556adb3a3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59286/0 (socket says 172.21.15.103:59286) 2024-09-19T06:50:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.163+0000 7fed13fff640 1 -- 172.21.15.103:0/3926792636 learned_addr learned my addr 172.21.15.103:0/3926792636 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.164+0000 7fed12ffd640 1 --2- 172.21.15.103:0/3926792636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3aecb0 0x556adb3b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.164+0000 7fed137fe640 1 --2- 172.21.15.103:0/3926792636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3a4410 0x556adb3ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.164+0000 7fed13fff640 1 -- 172.21.15.103:0/3926792636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3a4410 msgr2=0x556adb3ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.164+0000 7fed13fff640 1 --2- 172.21.15.103:0/3926792636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3a4410 0x556adb3ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.164+0000 7fed13fff640 1 -- 172.21.15.103:0/3926792636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3aecb0 msgr2=0x556adb3b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.164+0000 7fed13fff640 1 --2- 172.21.15.103:0/3926792636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3aecb0 0x556adb3b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.164+0000 7fed13fff640 1 -- 172.21.15.103:0/3926792636 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556adb2af340 con 0x556adb3a3af0 2024-09-19T06:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.164+0000 7fed10ff9640 1 --2- 172.21.15.103:0/2135283534 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fecfc0a3610 0x7fecfc0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.164+0000 7fed137fe640 1 --2- 172.21.15.103:0/2135283534 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fecfc0a3610 0x7fecfc0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.164+0000 7fed13fff640 1 --2- 172.21.15.103:0/3926792636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a3af0 0x556adb3a3ed0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fed0c008c50 tx=0x7fed0c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395debfba4ea72f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.164+0000 7fed10ff9640 1 -- 172.21.15.103:0/2135283534 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fed040cfa70 con 0x7fed14101ac0 2024-09-19T06:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.164+0000 7fecf9ffb640 1 -- 172.21.15.103:0/3926792636 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c018680 con 0x556adb3a3af0 2024-09-19T06:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.164+0000 7fecf9ffb640 1 -- 172.21.15.103:0/3926792636 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed0c018d00 con 0x556adb3a3af0 2024-09-19T06:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.164+0000 7fecf9ffb640 1 -- 172.21.15.103:0/3926792636 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c00fdf0 con 0x556adb3a3af0 2024-09-19T06:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.165+0000 7fed1c824740 1 -- 172.21.15.103:0/3926792636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a3af0 msgr2=0x556adb3a3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.165+0000 7fed137fe640 1 --2- 172.21.15.103:0/2135283534 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fecfc0a3610 0x7fecfc0a5ad0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fed040ccff0 tx=0x7fed04086040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.165+0000 7fed1c824740 1 --2- 172.21.15.103:0/3926792636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a3af0 0x556adb3a3ed0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fed0c008c50 tx=0x7fed0c00d710 comp rx=0 tx=0).stop 2024-09-19T06:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.165+0000 7fed1c824740 1 -- 172.21.15.103:0/3926792636 shutdown_connections 2024-09-19T06:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.165+0000 7fed1c824740 1 --2- 172.21.15.103:0/3926792636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3aecb0 0x556adb3b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.165+0000 7fed1c824740 1 --2- 172.21.15.103:0/3926792636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3a4410 0x556adb3ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.165+0000 7fed1c824740 1 --2- 172.21.15.103:0/3926792636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a3af0 0x556adb3a3ed0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.165+0000 7fed1c824740 1 -- 172.21.15.103:0/3926792636 >> 172.21.15.103:0/3926792636 conn(0x556adb3a0060 msgr2=0x556adb3a0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.165+0000 7fed1c824740 1 -- 172.21.15.103:0/3926792636 shutdown_connections 2024-09-19T06:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.165+0000 7fed1c824740 1 -- 172.21.15.103:0/3926792636 wait complete. 2024-09-19T06:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.165+0000 7fed1c824740 1 -- start start 2024-09-19T06:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.166+0000 7fed1c824740 1 Processor -- start 2024-09-19T06:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.166+0000 7fed1c824740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a3af0 0x556adb3abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.166+0000 7fed1c824740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3a4410 0x556adb3ac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.166+0000 7fed1c824740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3aecb0 0x556adb3ac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.166+0000 7fed1c824740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556adb3b1880 con 0x556adb3a3af0 2024-09-19T06:50:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.166+0000 7fed1c824740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556adb291290 con 0x556adb3a4410 2024-09-19T06:50:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.166+0000 7fed1c824740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556adb3b1a00 con 0x556adb3aecb0 2024-09-19T06:50:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.166+0000 7fed1c824740 10 client.0 _ll_register_callbacks cb 0x556adb2c2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.166+0000 7fed13fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a3af0 0x556adb3abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.166+0000 7fed13fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a3af0 0x556adb3abbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59296/0 (socket says 172.21.15.103:59296) 2024-09-19T06:50:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.166+0000 7fed13fff640 1 -- 172.21.15.103:0/1778500179 learned_addr learned my addr 172.21.15.103:0/1778500179 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.166+0000 7fed13fff640 1 --2- 172.21.15.103:0/1778500179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3aecb0 0x556adb3ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fed12ffd640 1 --2- 172.21.15.103:0/1778500179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3a4410 0x556adb3ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fed13fff640 1 -- 172.21.15.103:0/1778500179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3aecb0 msgr2=0x556adb3ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fed13fff640 1 --2- 172.21.15.103:0/1778500179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3aecb0 0x556adb3ac650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fed13fff640 1 -- 172.21.15.103:0/1778500179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3a4410 msgr2=0x556adb3ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fed13fff640 1 --2- 172.21.15.103:0/1778500179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3a4410 0x556adb3ac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fed13fff640 1 -- 172.21.15.103:0/1778500179 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556adb01b210 con 0x556adb3a3af0 2024-09-19T06:50:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fed12ffd640 1 --2- 172.21.15.103:0/1778500179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3a4410 0x556adb3ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fed13fff640 1 --2- 172.21.15.103:0/1778500179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a3af0 0x556adb3abbd0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0089d0 tx=0x7fed0c008f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660e7fddeba1411a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fecf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fecf2ffd640 1 -- 172.21.15.103:0/1778500179 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c004e50 con 0x556adb3a3af0 2024-09-19T06:50:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fecf2ffd640 1 -- 172.21.15.103:0/1778500179 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fed0c018480 con 0x556adb3a3af0 2024-09-19T06:50:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fed1c824740 1 -- 172.21.15.103:0/1778500179 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed0c018d00 con 0x556adb3a3af0 2024-09-19T06:50:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fecf2ffd640 1 -- 172.21.15.103:0/1778500179 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed0c018680 con 0x556adb3a3af0 2024-09-19T06:50:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fed1c824740 10 client.16062 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fed1c824740 1 -- 172.21.15.103:0/1778500179 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556adb0e7620 con 0x556adb3a3af0 2024-09-19T06:50:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fecf2ffd640 1 -- 172.21.15.103:0/1778500179 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed0c018d00 con 0x556adb3a3af0 2024-09-19T06:50:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.167+0000 7fecf2ffd640 1 -- 172.21.15.103:0/1778500179 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fed0c026070 con 0x556adb3a3af0 2024-09-19T06:50:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.168+0000 7fed1c824740 10 client.16062 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.168+0000 7fed1c824740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.168+0000 7fed1c824740 20 client.16062 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.168+0000 7fed1c824740 1 --2- 172.21.15.103:0/1778500179 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556adb3bebf0 0x556adb3c0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.168+0000 7fed1c824740 4 client.16062 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.168+0000 7fed1c824740 1 -- 172.21.15.103:0/1778500179 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556adb315a20 con 0x556adb3bebf0 2024-09-19T06:50:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.168+0000 7fed12ffd640 1 --2- 172.21.15.103:0/1778500179 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556adb3bebf0 0x556adb3c0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.169+0000 7fed12ffd640 1 --2- 172.21.15.103:0/1778500179 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556adb3bebf0 0x556adb3c0fe0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c8e307f03eb60a7 server_cookie=57edfa11b1c2dfff in_seq=0 out_seq=0 2024-09-19T06:50:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.169+0000 7fecf2ffd640 10 client.16062 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.170+0000 7fecf2ffd640 1 -- 172.21.15.103:0/1778500179 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556adb315a20 con 0x556adb3bebf0 2024-09-19T06:50:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.170+0000 7fecf2ffd640 10 client.16062 handle_command_reply: tid=0 2024-09-19T06:50:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.170+0000 7fed1c824740 1 client.16062 shutdown 2024-09-19T06:50:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.170+0000 7fed1c824740 1 -- 172.21.15.103:0/1778500179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a3af0 msgr2=0x556adb3abbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.170+0000 7fed1c824740 1 --2- 172.21.15.103:0/1778500179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a3af0 0x556adb3abbd0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0089d0 tx=0x7fed0c008f50 comp rx=0 tx=0).stop 2024-09-19T06:50:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.170+0000 7fed1c824740 1 -- 172.21.15.103:0/1778500179 shutdown_connections 2024-09-19T06:50:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.170+0000 7fed1c824740 1 --2- 172.21.15.103:0/1778500179 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556adb3bebf0 0x556adb3c0fe0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.170+0000 7fed1c824740 1 --2- 172.21.15.103:0/1778500179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3aecb0 0x556adb3ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.170+0000 7fecf2ffd640 0 client.16062 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.170+0000 7fed1c824740 1 --2- 172.21.15.103:0/1778500179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3a4410 0x556adb3ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.170+0000 7fed1c824740 1 --2- 172.21.15.103:0/1778500179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a3af0 0x556adb3abbd0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.170+0000 7fed1c824740 1 -- 172.21.15.103:0/1778500179 >> 172.21.15.103:0/1778500179 conn(0x556adb3a0060 msgr2=0x556adb3a5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.170+0000 7fed1c824740 1 -- 172.21.15.103:0/1778500179 shutdown_connections 2024-09-19T06:50:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.170+0000 7fed1c824740 1 -- 172.21.15.103:0/1778500179 wait complete. 2024-09-19T06:50:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.170+0000 7fed1c824740 20 client.16062 trim_cache size 0 max 16384 2024-09-19T06:50:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.186+0000 7fed1c824740 1 Processor -- start 2024-09-19T06:50:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.186+0000 7fed1c824740 1 -- start start 2024-09-19T06:50:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.186+0000 7fed1c824740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3a3af0 0x556adb3a6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.186+0000 7fed1c824740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a4410 0x556adb3ac100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.186+0000 7fed1c824740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3aecb0 0x556adb3ac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.186+0000 7fed1c824740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed141afa30 con 0x556adb3a4410 2024-09-19T06:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.186+0000 7fed1c824740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed141b1570 con 0x556adb3aecb0 2024-09-19T06:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.186+0000 7fed1c824740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed141b0100 con 0x556adb3a3af0 2024-09-19T06:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.186+0000 7fed12ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a4410 0x556adb3ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.186+0000 7fed12ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a4410 0x556adb3ac100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59310/0 (socket says 172.21.15.103:59310) 2024-09-19T06:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.186+0000 7fed13fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3a3af0 0x556adb3a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.186+0000 7fed12ffd640 1 -- 172.21.15.103:0/379899250 learned_addr learned my addr 172.21.15.103:0/379899250 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.187+0000 7fed13fff640 1 --2- 172.21.15.103:0/379899250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3aecb0 0x556adb3ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.187+0000 7fed12ffd640 1 -- 172.21.15.103:0/379899250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3a3af0 msgr2=0x556adb3a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.187+0000 7fed12ffd640 1 --2- 172.21.15.103:0/379899250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3a3af0 0x556adb3a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.187+0000 7fed12ffd640 1 -- 172.21.15.103:0/379899250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3aecb0 msgr2=0x556adb3ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.187+0000 7fed12ffd640 1 --2- 172.21.15.103:0/379899250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3aecb0 0x556adb3ac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.187+0000 7fed12ffd640 1 -- 172.21.15.103:0/379899250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556adb3a0cf0 con 0x556adb3a4410 2024-09-19T06:50:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.187+0000 7fed12ffd640 1 --2- 172.21.15.103:0/379899250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a4410 0x556adb3ac100 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fed0802aa10 tx=0x7fed08031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa018ab50e9b5d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.187+0000 7fecf97fa640 1 -- 172.21.15.103:0/379899250 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed08047020 con 0x556adb3a4410 2024-09-19T06:50:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.187+0000 7fecf97fa640 1 -- 172.21.15.103:0/379899250 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed08005520 con 0x556adb3a4410 2024-09-19T06:50:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.187+0000 7fecf97fa640 1 -- 172.21.15.103:0/379899250 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed08035040 con 0x556adb3a4410 2024-09-19T06:50:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.188+0000 7fed1c824740 1 -- 172.21.15.103:0/379899250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a4410 msgr2=0x556adb3ac100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.188+0000 7fed1c824740 1 --2- 172.21.15.103:0/379899250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a4410 0x556adb3ac100 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fed0802aa10 tx=0x7fed08031400 comp rx=0 tx=0).stop 2024-09-19T06:50:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.188+0000 7fed1c824740 1 -- 172.21.15.103:0/379899250 shutdown_connections 2024-09-19T06:50:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.188+0000 7fed1c824740 1 --2- 172.21.15.103:0/379899250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3aecb0 0x556adb3ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.188+0000 7fed1c824740 1 --2- 172.21.15.103:0/379899250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a4410 0x556adb3ac100 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.188+0000 7fed1c824740 1 --2- 172.21.15.103:0/379899250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3a3af0 0x556adb3a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.188+0000 7fed1c824740 1 -- 172.21.15.103:0/379899250 >> 172.21.15.103:0/379899250 conn(0x556adb3a0060 msgr2=0x556adb3adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.188+0000 7fed1c824740 1 -- 172.21.15.103:0/379899250 shutdown_connections 2024-09-19T06:50:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.188+0000 7fed1c824740 1 -- 172.21.15.103:0/379899250 wait complete. 2024-09-19T06:50:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.188+0000 7fed1c824740 1 -- start start 2024-09-19T06:50:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed1c824740 1 Processor -- start 2024-09-19T06:50:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed1c824740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3a3af0 0x556adb3b2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed1c824740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a4410 0x556adb3b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed1c824740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3aecb0 0x556adb4276c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed1c824740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed141b0ea0 con 0x556adb3a4410 2024-09-19T06:50:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed1c824740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed141b07d0 con 0x556adb3a3af0 2024-09-19T06:50:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed1c824740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed141ad9f0 con 0x556adb3aecb0 2024-09-19T06:50:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed1c824740 10 client.0 _ll_register_callbacks cb 0x7fed08009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed12ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a4410 0x556adb3b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed12ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a4410 0x556adb3b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59314/0 (socket says 172.21.15.103:59314) 2024-09-19T06:50:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed12ffd640 1 -- 172.21.15.103:0/3414032262 learned_addr learned my addr 172.21.15.103:0/3414032262 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed13fff640 1 --2- 172.21.15.103:0/3414032262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3a3af0 0x556adb3b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed13fff640 1 --2- 172.21.15.103:0/3414032262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3aecb0 0x556adb4276c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed12ffd640 1 -- 172.21.15.103:0/3414032262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3aecb0 msgr2=0x556adb4276c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed12ffd640 1 --2- 172.21.15.103:0/3414032262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3aecb0 0x556adb4276c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed12ffd640 1 -- 172.21.15.103:0/3414032262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3a3af0 msgr2=0x556adb3b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed12ffd640 1 --2- 172.21.15.103:0/3414032262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3a3af0 0x556adb3b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.189+0000 7fed12ffd640 1 -- 172.21.15.103:0/3414032262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556adb39b2d0 con 0x556adb3a4410 2024-09-19T06:50:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.190+0000 7fed12ffd640 1 --2- 172.21.15.103:0/3414032262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a4410 0x556adb3b27a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fed08000c00 tx=0x7fed08009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dca9d952b37290e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.190+0000 7fecf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.190+0000 7fecf2ffd640 1 -- 172.21.15.103:0/3414032262 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed08050070 con 0x556adb3a4410 2024-09-19T06:50:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.190+0000 7fecf2ffd640 1 -- 172.21.15.103:0/3414032262 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fed080048d0 con 0x556adb3a4410 2024-09-19T06:50:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.190+0000 7fed1c824740 1 -- 172.21.15.103:0/3414032262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed08005520 con 0x556adb3a4410 2024-09-19T06:50:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.190+0000 7fecf2ffd640 1 -- 172.21.15.103:0/3414032262 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed0805c050 con 0x556adb3a4410 2024-09-19T06:50:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.190+0000 7fed1c824740 10 client.16068 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.190+0000 7fed1c824740 1 -- 172.21.15.103:0/3414032262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556adb428cf0 con 0x556adb3a4410 2024-09-19T06:50:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.190+0000 7fecf2ffd640 1 -- 172.21.15.103:0/3414032262 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed08005520 con 0x556adb3a4410 2024-09-19T06:50:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.190+0000 7fecf2ffd640 1 -- 172.21.15.103:0/3414032262 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fed080318f0 con 0x556adb3a4410 2024-09-19T06:50:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.190+0000 7fed1c824740 10 client.16068 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.190+0000 7fed1c824740 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.190+0000 7fed1c824740 20 client.16068 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.190+0000 7fed1c824740 1 --2- 172.21.15.103:0/3414032262 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556adb429390 0x556adb42b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.190+0000 7fed1c824740 4 client.16068 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:50:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.190+0000 7fed1c824740 1 -- 172.21.15.103:0/3414032262 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556adb30fd70 con 0x556adb429390 2024-09-19T06:50:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.191+0000 7fed13fff640 1 --2- 172.21.15.103:0/3414032262 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556adb429390 0x556adb42b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.191+0000 7fecf2ffd640 10 client.16068 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.191+0000 7fed13fff640 1 --2- 172.21.15.103:0/3414032262 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556adb429390 0x556adb42b7b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18394262cd86341e server_cookie=a98c14d9b12c0e79 in_seq=0 out_seq=0 2024-09-19T06:50:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.200+0000 7fecf2ffd640 1 -- 172.21.15.103:0/3414032262 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556adb30fd70 con 0x556adb429390 2024-09-19T06:50:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.200+0000 7fecf2ffd640 10 client.16068 handle_command_reply: tid=0 2024-09-19T06:50:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.200+0000 7fed1c824740 1 client.16068 shutdown 2024-09-19T06:50:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.200+0000 7fed1c824740 1 -- 172.21.15.103:0/3414032262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a4410 msgr2=0x556adb3b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.200+0000 7fed1c824740 1 --2- 172.21.15.103:0/3414032262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a4410 0x556adb3b27a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fed08000c00 tx=0x7fed08009520 comp rx=0 tx=0).stop 2024-09-19T06:50:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.200+0000 7fed1c824740 1 -- 172.21.15.103:0/3414032262 shutdown_connections 2024-09-19T06:50:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.200+0000 7fed1c824740 1 --2- 172.21.15.103:0/3414032262 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556adb429390 0x556adb42b7b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.200+0000 7fed1c824740 1 --2- 172.21.15.103:0/3414032262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556adb3aecb0 0x556adb4276c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.200+0000 7fecf2ffd640 0 client.16068 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.200+0000 7fed1c824740 1 --2- 172.21.15.103:0/3414032262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556adb3a4410 0x556adb3b27a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.200+0000 7fed1c824740 1 --2- 172.21.15.103:0/3414032262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556adb3a3af0 0x556adb3b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.200+0000 7fed1c824740 1 -- 172.21.15.103:0/3414032262 >> 172.21.15.103:0/3414032262 conn(0x556adb3a0060 msgr2=0x556adb3ad430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.200+0000 7fed1c824740 1 -- 172.21.15.103:0/3414032262 shutdown_connections 2024-09-19T06:50:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.200+0000 7fed1c824740 1 -- 172.21.15.103:0/3414032262 wait complete. 2024-09-19T06:50:44.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.200+0000 7fed1c824740 20 client.16068 trim_cache size 0 max 16384 2024-09-19T06:50:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:50:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:50:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:50:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:50:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.201+0000 7fed19a8b640 1 -- 172.21.15.103:0/2135283534 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fecfc0a3610 msgr2=0x7fecfc0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.201+0000 7fed19a8b640 1 --2- 172.21.15.103:0/2135283534 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fecfc0a3610 0x7fecfc0a5ad0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fed040ccff0 tx=0x7fed04086040 comp rx=0 tx=0).stop 2024-09-19T06:50:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.201+0000 7fed19a8b640 1 -- 172.21.15.103:0/2135283534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed14101ac0 msgr2=0x7fed140744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.201+0000 7fed19a8b640 1 --2- 172.21.15.103:0/2135283534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed14101ac0 0x7fed140744f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fed0400d770 tx=0x7fed0400dc30 comp rx=0 tx=0).stop 2024-09-19T06:50:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.201+0000 7fed19a8b640 1 -- 172.21.15.103:0/2135283534 shutdown_connections 2024-09-19T06:50:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.201+0000 7fed19a8b640 1 --2- 172.21.15.103:0/2135283534 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fecfc0a3610 0x7fecfc0a5ad0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.201+0000 7fed19a8b640 1 --2- 172.21.15.103:0/2135283534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed14107ad0 0x7fed14074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.201+0000 7fed19a8b640 1 --2- 172.21.15.103:0/2135283534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed14102460 0x7fed14074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.201+0000 7fed19a8b640 1 --2- 172.21.15.103:0/2135283534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed14101ac0 0x7fed140744f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.201+0000 7fed19a8b640 1 -- 172.21.15.103:0/2135283534 >> 172.21.15.103:0/2135283534 conn(0x7fed140fd820 msgr2=0x7fed140fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.201+0000 7fed19a8b640 1 -- 172.21.15.103:0/2135283534 shutdown_connections 2024-09-19T06:50:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:44.201+0000 7fed19a8b640 1 -- 172.21.15.103:0/2135283534 wait complete. 2024-09-19T06:50:44.427 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:44 smithi103 ceph-mon[31158]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:50:44.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.246+0000 7f5743fae640 1 -- 172.21.15.103:0/440644367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f573c100a80 msgr2=0x7f573c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.246+0000 7f5743fae640 1 --2- 172.21.15.103:0/440644367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f573c100a80 0x7f573c100ee0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f572c009930 tx=0x7f572c02a420 comp rx=0 tx=0).stop 2024-09-19T06:50:44.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.246+0000 7f5743fae640 1 -- 172.21.15.103:0/440644367 shutdown_connections 2024-09-19T06:50:44.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.246+0000 7f5743fae640 1 --2- 172.21.15.103:0/440644367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f573c101420 0x7f573c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.246+0000 7f5743fae640 1 --2- 172.21.15.103:0/440644367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f573c100a80 0x7f573c100ee0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.246+0000 7f5743fae640 1 --2- 172.21.15.103:0/440644367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f573c106a80 0x7f573c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.246+0000 7f5743fae640 1 -- 172.21.15.103:0/440644367 >> 172.21.15.103:0/440644367 conn(0x7f573c0fc7a0 msgr2=0x7f573c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:44.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.246+0000 7f5743fae640 1 -- 172.21.15.103:0/440644367 shutdown_connections 2024-09-19T06:50:44.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.246+0000 7f5743fae640 1 -- 172.21.15.103:0/440644367 wait complete. 2024-09-19T06:50:44.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.246+0000 7f5743fae640 1 Processor -- start 2024-09-19T06:50:44.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.246+0000 7f5743fae640 1 -- start start 2024-09-19T06:50:44.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5743fae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f573c100a80 0x7f573c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5743fae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f573c101420 0x7f573c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5743fae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f573c106a80 0x7f573c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5743fae640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f573c110e80 con 0x7f573c101420 2024-09-19T06:50:44.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5743fae640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f573c110d00 con 0x7f573c106a80 2024-09-19T06:50:44.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5743fae640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f573c111000 con 0x7f573c100a80 2024-09-19T06:50:44.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5741549640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f573c101420 0x7f573c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5741549640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f573c101420 0x7f573c19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59324/0 (socket says 172.21.15.103:59324) 2024-09-19T06:50:44.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5741549640 1 -- 172.21.15.103:0/2887094728 learned_addr learned my addr 172.21.15.103:0/2887094728 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:44.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5741d4a640 1 --2- 172.21.15.103:0/2887094728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f573c100a80 0x7f573c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f574254b640 1 --2- 172.21.15.103:0/2887094728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f573c106a80 0x7f573c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5741549640 1 -- 172.21.15.103:0/2887094728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f573c100a80 msgr2=0x7f573c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5741549640 1 --2- 172.21.15.103:0/2887094728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f573c100a80 0x7f573c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5741549640 1 -- 172.21.15.103:0/2887094728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f573c106a80 msgr2=0x7f573c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5741549640 1 --2- 172.21.15.103:0/2887094728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f573c106a80 0x7f573c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5741549640 1 -- 172.21.15.103:0/2887094728 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573c19f650 con 0x7f573c101420 2024-09-19T06:50:44.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5741549640 1 --2- 172.21.15.103:0/2887094728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f573c101420 0x7f573c19abe0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f572c009230 tx=0x7f572c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:44.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.247+0000 7f5732ffd640 1 -- 172.21.15.103:0/2887094728 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c03f030 con 0x7f573c101420 2024-09-19T06:50:44.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.248+0000 7f5732ffd640 1 -- 172.21.15.103:0/2887094728 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c003ad0 con 0x7f573c101420 2024-09-19T06:50:44.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.248+0000 7f5743fae640 1 -- 172.21.15.103:0/2887094728 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573c19f8e0 con 0x7f573c101420 2024-09-19T06:50:44.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.248+0000 7f5743fae640 1 -- 172.21.15.103:0/2887094728 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f573c1a7220 con 0x7f573c101420 2024-09-19T06:50:44.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.248+0000 7f5732ffd640 1 -- 172.21.15.103:0/2887094728 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c035070 con 0x7f573c101420 2024-09-19T06:50:44.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.250+0000 7f5732ffd640 1 -- 172.21.15.103:0/2887094728 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138640+0+0 (secure 0 0 0) 0x7f572c042030 con 0x7f573c101420 2024-09-19T06:50:44.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.250+0000 7f5746387740 1 Processor -- start 2024-09-19T06:50:44.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.250+0000 7f5746387740 1 -- start start 2024-09-19T06:50:44.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.250+0000 7f5746387740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1b9bb40 0x5595d1b9a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.250+0000 7f5746387740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9a5d0 0x5595d1b9aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.250+0000 7f5746387740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1ba5370 0x5595d1ba7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.250+0000 7f5746387740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5595d1b002f0 con 0x5595d1b9a5d0 2024-09-19T06:50:44.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.250+0000 7f5746387740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5595d1aa7cf0 con 0x5595d1b9bb40 2024-09-19T06:50:44.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.250+0000 7f5746387740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5595d1afc2b0 con 0x5595d1ba5370 2024-09-19T06:50:44.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f5732ffd640 1 --2- 172.21.15.103:0/2887094728 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f57280a35e0 0x7f57280a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f5741d4a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9a5d0 0x5595d1b9aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f5741d4a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9a5d0 0x5595d1b9aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59332/0 (socket says 172.21.15.103:59332) 2024-09-19T06:50:44.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f5741d4a640 1 -- 172.21.15.103:0/2108602382 learned_addr learned my addr 172.21.15.103:0/2108602382 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:44.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f574254b640 1 --2- 172.21.15.103:0/2108602382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1b9bb40 0x5595d1b9a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f574254b640 1 --2- 172.21.15.103:0/2887094728 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f57280a35e0 0x7f57280a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f5741549640 1 --2- 172.21.15.103:0/2108602382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1ba5370 0x5595d1ba7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f5732ffd640 1 -- 172.21.15.103:0/2887094728 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f572c0f0360 con 0x7f573c101420 2024-09-19T06:50:44.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f5741d4a640 1 -- 172.21.15.103:0/2108602382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1ba5370 msgr2=0x5595d1ba7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:44.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f5741d4a640 1 --2- 172.21.15.103:0/2108602382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1ba5370 0x5595d1ba7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f5741d4a640 1 -- 172.21.15.103:0/2108602382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1b9bb40 msgr2=0x5595d1b9a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:44.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f5741d4a640 1 --2- 172.21.15.103:0/2108602382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1b9bb40 0x5595d1b9a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f5741d4a640 1 -- 172.21.15.103:0/2108602382 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595d1a8c3a0 con 0x5595d1b9a5d0 2024-09-19T06:50:44.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f574254b640 1 --2- 172.21.15.103:0/2108602382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1b9bb40 0x5595d1b9a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:44.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f5741d4a640 1 --2- 172.21.15.103:0/2108602382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9a5d0 0x5595d1b9aa10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f573800bc90 tx=0x7f57380107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec581fa569d1fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:44.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f574254b640 1 --2- 172.21.15.103:0/2887094728 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f57280a35e0 0x7f57280a5aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5734009fd0 tx=0x7f5734009600 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:44.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f5727fff640 1 -- 172.21.15.103:0/2108602382 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573801c480 con 0x5595d1b9a5d0 2024-09-19T06:50:44.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f5727fff640 1 -- 172.21.15.103:0/2108602382 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f573801cb00 con 0x5595d1b9a5d0 2024-09-19T06:50:44.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.251+0000 7f5727fff640 1 -- 172.21.15.103:0/2108602382 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738011bb0 con 0x5595d1b9a5d0 2024-09-19T06:50:44.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.252+0000 7f5746387740 1 -- 172.21.15.103:0/2108602382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9a5d0 msgr2=0x5595d1b9aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.252+0000 7f5746387740 1 --2- 172.21.15.103:0/2108602382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9a5d0 0x5595d1b9aa10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f573800bc90 tx=0x7f57380107a0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.252+0000 7f5746387740 1 -- 172.21.15.103:0/2108602382 shutdown_connections 2024-09-19T06:50:44.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.252+0000 7f5746387740 1 --2- 172.21.15.103:0/2108602382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1ba5370 0x5595d1ba7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.252+0000 7f5746387740 1 --2- 172.21.15.103:0/2108602382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9a5d0 0x5595d1b9aa10 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.252+0000 7f5746387740 1 --2- 172.21.15.103:0/2108602382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1b9bb40 0x5595d1b9a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.252+0000 7f5746387740 1 -- 172.21.15.103:0/2108602382 >> 172.21.15.103:0/2108602382 conn(0x5595d1b63310 msgr2=0x5595d1b96c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:44.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.252+0000 7f5746387740 1 -- 172.21.15.103:0/2108602382 shutdown_connections 2024-09-19T06:50:44.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.252+0000 7f5746387740 1 -- 172.21.15.103:0/2108602382 wait complete. 2024-09-19T06:50:44.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.253+0000 7f5746387740 1 -- start start 2024-09-19T06:50:44.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.253+0000 7f5746387740 1 Processor -- start 2024-09-19T06:50:44.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.253+0000 7f5746387740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1b9a5d0 0x5595d1ba4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.253+0000 7f5746387740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1ba1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.253+0000 7f5746387740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1ba5370 0x5595d1ba25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.253+0000 7f5746387740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5595d1ba8070 con 0x5595d1b9bb40 2024-09-19T06:50:44.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.253+0000 7f5746387740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5595d1abb060 con 0x5595d1ba5370 2024-09-19T06:50:44.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.253+0000 7f5746387740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5595d1ba81f0 con 0x5595d1b9a5d0 2024-09-19T06:50:44.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.253+0000 7f5746387740 10 client.0 _ll_register_callbacks cb 0x5595d180e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:44.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.253+0000 7f5741549640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1ba1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.253+0000 7f5741549640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1ba1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59334/0 (socket says 172.21.15.103:59334) 2024-09-19T06:50:44.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.253+0000 7f5741d4a640 1 --2- 172.21.15.103:0/4076449459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1b9a5d0 0x5595d1ba4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.253+0000 7f5741549640 1 -- 172.21.15.103:0/4076449459 learned_addr learned my addr 172.21.15.103:0/4076449459 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:44.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f574254b640 1 --2- 172.21.15.103:0/4076449459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1ba5370 0x5595d1ba25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f5741549640 1 -- 172.21.15.103:0/4076449459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1b9a5d0 msgr2=0x5595d1ba4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:44.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f5741549640 1 --2- 172.21.15.103:0/4076449459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1b9a5d0 0x5595d1ba4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f5741549640 1 -- 172.21.15.103:0/4076449459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1ba5370 msgr2=0x5595d1ba25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:44.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f5741549640 1 --2- 172.21.15.103:0/4076449459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1ba5370 0x5595d1ba25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f5741549640 1 -- 172.21.15.103:0/4076449459 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595d18713c0 con 0x5595d1b9bb40 2024-09-19T06:50:44.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f5741d4a640 1 --2- 172.21.15.103:0/4076449459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1b9a5d0 0x5595d1ba4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:44.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f574254b640 1 --2- 172.21.15.103:0/4076449459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1ba5370 0x5595d1ba25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:44.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f5741549640 1 --2- 172.21.15.103:0/4076449459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1ba1fe0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f572c0bc450 tx=0x7f572c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a788d6377347eb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:44.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f57257fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:44.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f57257fa640 1 -- 172.21.15.103:0/4076449459 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c0d5060 con 0x5595d1b9bb40 2024-09-19T06:50:44.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f57257fa640 1 -- 172.21.15.103:0/4076449459 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f572c003c80 con 0x5595d1b9bb40 2024-09-19T06:50:44.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f5746387740 1 -- 172.21.15.103:0/4076449459 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f573801cb00 con 0x5595d1b9bb40 2024-09-19T06:50:44.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f57257fa640 1 -- 172.21.15.103:0/4076449459 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f572c049040 con 0x5595d1b9bb40 2024-09-19T06:50:44.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f5746387740 10 client.16080 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:44.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f5746387740 1 -- 172.21.15.103:0/4076449459 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5595d1bb3ad0 con 0x5595d1b9bb40 2024-09-19T06:50:44.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.254+0000 7f57257fa640 1 -- 172.21.15.103:0/4076449459 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f573801cb00 con 0x5595d1b9bb40 2024-09-19T06:50:44.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.255+0000 7f57257fa640 1 -- 172.21.15.103:0/4076449459 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f572c0b8b30 con 0x5595d1b9bb40 2024-09-19T06:50:44.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.255+0000 7f5746387740 10 client.16080 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:44.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.255+0000 7f5746387740 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:44.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.255+0000 7f5746387740 20 client.16080 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:44.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.255+0000 7f5746387740 1 --2- 172.21.15.103:0/4076449459 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5595d1bb4f80 0x5595d1bb7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.255+0000 7f5746387740 4 client.16080 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:44.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.255+0000 7f5746387740 1 -- 172.21.15.103:0/4076449459 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595d1b0d860 con 0x5595d1bb4f80 2024-09-19T06:50:44.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.255+0000 7f5741d4a640 1 --2- 172.21.15.103:0/4076449459 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5595d1bb4f80 0x5595d1bb7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.255+0000 7f5741d4a640 1 --2- 172.21.15.103:0/4076449459 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5595d1bb4f80 0x5595d1bb7400 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e36b425da013e48 server_cookie=aac6b4be8bf076ee in_seq=0 out_seq=0 2024-09-19T06:50:44.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.255+0000 7f57257fa640 10 client.16080 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:44.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.256+0000 7f57257fa640 1 -- 172.21.15.103:0/4076449459 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595d1b0d860 con 0x5595d1bb4f80 2024-09-19T06:50:44.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.256+0000 7f57257fa640 10 client.16080 handle_command_reply: tid=0 2024-09-19T06:50:44.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.256+0000 7f5746387740 1 client.16080 shutdown 2024-09-19T06:50:44.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.256+0000 7f5746387740 1 -- 172.21.15.103:0/4076449459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 msgr2=0x5595d1ba1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.256+0000 7f5746387740 1 --2- 172.21.15.103:0/4076449459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1ba1fe0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f572c0bc450 tx=0x7f572c005e50 comp rx=0 tx=0).stop 2024-09-19T06:50:44.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.256+0000 7f5746387740 1 -- 172.21.15.103:0/4076449459 shutdown_connections 2024-09-19T06:50:44.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.256+0000 7f5746387740 1 --2- 172.21.15.103:0/4076449459 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5595d1bb4f80 0x5595d1bb7400 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.256+0000 7f5746387740 1 --2- 172.21.15.103:0/4076449459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1ba5370 0x5595d1ba25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.256+0000 7f5746387740 1 --2- 172.21.15.103:0/4076449459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1ba1fe0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.256+0000 7f57257fa640 0 client.16080 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:44.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.256+0000 7f5746387740 1 --2- 172.21.15.103:0/4076449459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1b9a5d0 0x5595d1ba4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.256+0000 7f5746387740 1 -- 172.21.15.103:0/4076449459 >> 172.21.15.103:0/4076449459 conn(0x5595d1b63310 msgr2=0x5595d1b9c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:44.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.256+0000 7f5746387740 1 -- 172.21.15.103:0/4076449459 shutdown_connections 2024-09-19T06:50:44.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.256+0000 7f5746387740 1 -- 172.21.15.103:0/4076449459 wait complete. 2024-09-19T06:50:44.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.256+0000 7f5746387740 20 client.16080 trim_cache size 0 max 16384 2024-09-19T06:50:44.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.269+0000 7f5746387740 1 Processor -- start 2024-09-19T06:50:44.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.269+0000 7f5746387740 1 -- start start 2024-09-19T06:50:44.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.269+0000 7f5746387740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1b9a5d0 0x5595d1b96e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.269+0000 7f5746387740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1bb1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.269+0000 7f5746387740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1ba5370 0x5595d1b9b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.269+0000 7f5746387740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f573c1afa30 con 0x5595d1b9bb40 2024-09-19T06:50:44.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.269+0000 7f5746387740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f573c1b1570 con 0x5595d1ba5370 2024-09-19T06:50:44.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.269+0000 7f5746387740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f573c1b0100 con 0x5595d1b9a5d0 2024-09-19T06:50:44.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.270+0000 7f5741549640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1bb1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.270+0000 7f5741549640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1bb1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59340/0 (socket says 172.21.15.103:59340) 2024-09-19T06:50:44.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.270+0000 7f5741549640 1 -- 172.21.15.103:0/3366836360 learned_addr learned my addr 172.21.15.103:0/3366836360 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:44.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.270+0000 7f5741d4a640 1 --2- 172.21.15.103:0/3366836360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1b9a5d0 0x5595d1b96e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.270+0000 7f574254b640 1 --2- 172.21.15.103:0/3366836360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1ba5370 0x5595d1b9b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.270+0000 7f5741549640 1 -- 172.21.15.103:0/3366836360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1b9a5d0 msgr2=0x5595d1b96e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:44.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.270+0000 7f5741549640 1 --2- 172.21.15.103:0/3366836360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1b9a5d0 0x5595d1b96e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.270+0000 7f5741549640 1 -- 172.21.15.103:0/3366836360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1ba5370 msgr2=0x5595d1b9b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:44.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.270+0000 7f5741549640 1 --2- 172.21.15.103:0/3366836360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1ba5370 0x5595d1b9b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.270+0000 7f5741549640 1 -- 172.21.15.103:0/3366836360 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595d1b964f0 con 0x5595d1b9bb40 2024-09-19T06:50:44.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.270+0000 7f5741549640 1 --2- 172.21.15.103:0/3366836360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1bb1310 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f572c0b8e90 tx=0x7f572c0bc640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64b663ba71eb777 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:44.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.270+0000 7f57277fe640 1 -- 172.21.15.103:0/3366836360 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c09e070 con 0x5595d1b9bb40 2024-09-19T06:50:44.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.270+0000 7f57277fe640 1 -- 172.21.15.103:0/3366836360 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c0c5080 con 0x5595d1b9bb40 2024-09-19T06:50:44.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.270+0000 7f57277fe640 1 -- 172.21.15.103:0/3366836360 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c0c4bf0 con 0x5595d1b9bb40 2024-09-19T06:50:44.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.271+0000 7f5746387740 1 -- 172.21.15.103:0/3366836360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 msgr2=0x5595d1bb1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.271+0000 7f5746387740 1 --2- 172.21.15.103:0/3366836360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1bb1310 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f572c0b8e90 tx=0x7f572c0bc640 comp rx=0 tx=0).stop 2024-09-19T06:50:44.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.271+0000 7f5746387740 1 -- 172.21.15.103:0/3366836360 shutdown_connections 2024-09-19T06:50:44.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.271+0000 7f5746387740 1 --2- 172.21.15.103:0/3366836360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1ba5370 0x5595d1b9b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.271+0000 7f5746387740 1 --2- 172.21.15.103:0/3366836360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1bb1310 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.271+0000 7f5746387740 1 --2- 172.21.15.103:0/3366836360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1b9a5d0 0x5595d1b96e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.271+0000 7f5746387740 1 -- 172.21.15.103:0/3366836360 >> 172.21.15.103:0/3366836360 conn(0x5595d1b63310 msgr2=0x5595d1b96700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:44.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.271+0000 7f5746387740 1 -- 172.21.15.103:0/3366836360 shutdown_connections 2024-09-19T06:50:44.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.271+0000 7f5746387740 1 -- 172.21.15.103:0/3366836360 wait complete. 2024-09-19T06:50:44.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.271+0000 7f5746387740 1 -- start start 2024-09-19T06:50:44.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.271+0000 7f5746387740 1 Processor -- start 2024-09-19T06:50:44.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5746387740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1b9a5d0 0x5595d1bad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5746387740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1badd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5746387740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1ba5370 0x5595d1ba8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5746387740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f573c1b0ea0 con 0x5595d1b9bb40 2024-09-19T06:50:44.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5746387740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f573c1b07d0 con 0x5595d1ba5370 2024-09-19T06:50:44.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5746387740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f573c1ad9f0 con 0x5595d1b9a5d0 2024-09-19T06:50:44.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5746387740 10 client.0 _ll_register_callbacks cb 0x7f572c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:44.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5741549640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1badd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5741549640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1badd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59342/0 (socket says 172.21.15.103:59342) 2024-09-19T06:50:44.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5741549640 1 -- 172.21.15.103:0/3666845950 learned_addr learned my addr 172.21.15.103:0/3666845950 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:44.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5741d4a640 1 --2- 172.21.15.103:0/3666845950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1b9a5d0 0x5595d1bad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f574254b640 1 --2- 172.21.15.103:0/3666845950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1ba5370 0x5595d1ba8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5741549640 1 -- 172.21.15.103:0/3666845950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1b9a5d0 msgr2=0x5595d1bad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:44.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5741549640 1 --2- 172.21.15.103:0/3666845950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1b9a5d0 0x5595d1bad7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5741549640 1 -- 172.21.15.103:0/3666845950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1ba5370 msgr2=0x5595d1ba8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:44.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5741549640 1 --2- 172.21.15.103:0/3666845950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1ba5370 0x5595d1ba8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5741549640 1 -- 172.21.15.103:0/3666845950 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595d1ba95e0 con 0x5595d1b9bb40 2024-09-19T06:50:44.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f5741549640 1 --2- 172.21.15.103:0/3666845950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1badd20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f572c0b8e40 tx=0x7f572c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be05aba5d4a14d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:44.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f57257fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:44.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.272+0000 7f57257fa640 1 -- 172.21.15.103:0/3666845950 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c0d9030 con 0x5595d1b9bb40 2024-09-19T06:50:44.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.273+0000 7f57257fa640 1 -- 172.21.15.103:0/3666845950 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f572c0c4bf0 con 0x5595d1b9bb40 2024-09-19T06:50:44.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.273+0000 7f5746387740 1 -- 172.21.15.103:0/3666845950 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f572c0c5080 con 0x5595d1b9bb40 2024-09-19T06:50:44.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.273+0000 7f57257fa640 1 -- 172.21.15.103:0/3666845950 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f572c09e070 con 0x5595d1b9bb40 2024-09-19T06:50:44.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.273+0000 7f5746387740 10 client.16086 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:44.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.273+0000 7f5746387740 1 -- 172.21.15.103:0/3666845950 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5595d1ba95e0 con 0x5595d1b9bb40 2024-09-19T06:50:44.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.273+0000 7f57257fa640 1 -- 172.21.15.103:0/3666845950 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c0c5080 con 0x5595d1b9bb40 2024-09-19T06:50:44.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.273+0000 7f57257fa640 1 -- 172.21.15.103:0/3666845950 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f572c0bb910 con 0x5595d1b9bb40 2024-09-19T06:50:44.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.273+0000 7f5746387740 10 client.16086 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:44.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.273+0000 7f5746387740 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:44.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.273+0000 7f5746387740 20 client.16086 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:44.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.273+0000 7f5746387740 1 --2- 172.21.15.103:0/3666845950 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5595d1c1fb10 0x5595d1c21f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:44.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.273+0000 7f5746387740 4 client.16086 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:44.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.273+0000 7f5746387740 1 -- 172.21.15.103:0/3666845950 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595d1b07d70 con 0x5595d1c1fb10 2024-09-19T06:50:44.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.273+0000 7f5741d4a640 1 --2- 172.21.15.103:0/3666845950 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5595d1c1fb10 0x5595d1c21f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:44.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f57257fa640 10 client.16086 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:44.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f5741d4a640 1 --2- 172.21.15.103:0/3666845950 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5595d1c1fb10 0x5595d1c21f00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f3c1afd69303987 server_cookie=abf57d1ed3ffc58a in_seq=0 out_seq=0 2024-09-19T06:50:44.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f57257fa640 1 -- 172.21.15.103:0/3666845950 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595d1b07d70 con 0x5595d1c1fb10 2024-09-19T06:50:44.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f57257fa640 10 client.16086 handle_command_reply: tid=0 2024-09-19T06:50:44.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f5746387740 1 client.16086 shutdown 2024-09-19T06:50:44.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f5746387740 1 -- 172.21.15.103:0/3666845950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 msgr2=0x5595d1badd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f5746387740 1 --2- 172.21.15.103:0/3666845950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1badd20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f572c0b8e40 tx=0x7f572c009260 comp rx=0 tx=0).stop 2024-09-19T06:50:44.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f5746387740 1 -- 172.21.15.103:0/3666845950 shutdown_connections 2024-09-19T06:50:44.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f5746387740 1 --2- 172.21.15.103:0/3666845950 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5595d1c1fb10 0x5595d1c21f00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f5746387740 1 --2- 172.21.15.103:0/3666845950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595d1ba5370 0x5595d1ba8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f5746387740 1 --2- 172.21.15.103:0/3666845950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595d1b9bb40 0x5595d1badd20 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f5746387740 1 --2- 172.21.15.103:0/3666845950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d1b9a5d0 0x5595d1bad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f57257fa640 0 client.16086 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:44.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f5746387740 1 -- 172.21.15.103:0/3666845950 >> 172.21.15.103:0/3666845950 conn(0x5595d1b63310 msgr2=0x5595d1b966c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:44.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f5746387740 1 -- 172.21.15.103:0/3666845950 shutdown_connections 2024-09-19T06:50:44.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f5746387740 1 -- 172.21.15.103:0/3666845950 wait complete. 2024-09-19T06:50:44.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.274+0000 7f5746387740 20 client.16086 trim_cache size 0 max 16384 2024-09-19T06:50:44.460 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:44 smithi134 ceph-mon[35356]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:50:44.460 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:50:44.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.275+0000 7f5743fae640 1 -- 172.21.15.103:0/2887094728 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f57280a35e0 msgr2=0x7f57280a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.275+0000 7f5743fae640 1 --2- 172.21.15.103:0/2887094728 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f57280a35e0 0x7f57280a5aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5734009fd0 tx=0x7f5734009600 comp rx=0 tx=0).stop 2024-09-19T06:50:44.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.275+0000 7f5743fae640 1 -- 172.21.15.103:0/2887094728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f573c101420 msgr2=0x7f573c19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:44.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.275+0000 7f5743fae640 1 --2- 172.21.15.103:0/2887094728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f573c101420 0x7f573c19abe0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f572c009230 tx=0x7f572c004270 comp rx=0 tx=0).stop 2024-09-19T06:50:44.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.275+0000 7f5743fae640 1 -- 172.21.15.103:0/2887094728 shutdown_connections 2024-09-19T06:50:44.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.275+0000 7f5743fae640 1 --2- 172.21.15.103:0/2887094728 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f57280a35e0 0x7f57280a5aa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.275+0000 7f5743fae640 1 --2- 172.21.15.103:0/2887094728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f573c106a80 0x7f573c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.275+0000 7f5743fae640 1 --2- 172.21.15.103:0/2887094728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f573c101420 0x7f573c19abe0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.275+0000 7f5743fae640 1 --2- 172.21.15.103:0/2887094728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f573c100a80 0x7f573c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:44.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.275+0000 7f5743fae640 1 -- 172.21.15.103:0/2887094728 >> 172.21.15.103:0/2887094728 conn(0x7f573c0fc7a0 msgr2=0x7f573c10c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:44.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.275+0000 7f5743fae640 1 -- 172.21.15.103:0/2887094728 shutdown_connections 2024-09-19T06:50:44.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:44.275+0000 7f5743fae640 1 -- 172.21.15.103:0/2887094728 wait complete. 2024-09-19T06:50:46.047 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:50:46.048 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:50:46.297 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:46 smithi174 ceph-mon[35637]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 18 op/s 2024-09-19T06:50:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:46 smithi174 ceph-mon[35637]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-09-19T06:50:46.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:46 smithi103 ceph-mon[31158]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 18 op/s 2024-09-19T06:50:46.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:46 smithi103 ceph-mon[31158]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-09-19T06:50:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:46 smithi134 ceph-mon[35356]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 18 op/s 2024-09-19T06:50:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:46 smithi134 ceph-mon[35356]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-09-19T06:50:46.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.719+0000 7f9519cf5640 1 -- 172.21.15.103:0/1008441236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9514100a80 msgr2=0x7f9514100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:46.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.719+0000 7f9519cf5640 1 --2- 172.21.15.103:0/1008441236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9514100a80 0x7f9514100ee0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9508009930 tx=0x7f950802a420 comp rx=0 tx=0).stop 2024-09-19T06:50:46.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.719+0000 7f9519cf5640 1 -- 172.21.15.103:0/1008441236 shutdown_connections 2024-09-19T06:50:46.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.719+0000 7f9519cf5640 1 --2- 172.21.15.103:0/1008441236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9514101420 0x7f951410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.719+0000 7f9519cf5640 1 --2- 172.21.15.103:0/1008441236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9514100a80 0x7f9514100ee0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.719+0000 7f9519cf5640 1 --2- 172.21.15.103:0/1008441236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9514106a80 0x7f9514106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.719+0000 7f9519cf5640 1 -- 172.21.15.103:0/1008441236 >> 172.21.15.103:0/1008441236 conn(0x7f95140fc7a0 msgr2=0x7f95140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:46.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.719+0000 7f9519cf5640 1 -- 172.21.15.103:0/1008441236 shutdown_connections 2024-09-19T06:50:46.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.719+0000 7f9519cf5640 1 -- 172.21.15.103:0/1008441236 wait complete. 2024-09-19T06:50:46.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.720+0000 7f9519cf5640 1 Processor -- start 2024-09-19T06:50:46.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.720+0000 7f9519cf5640 1 -- start start 2024-09-19T06:50:46.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.720+0000 7f9519cf5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9514100a80 0x7f9514199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.720+0000 7f9519cf5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9514101420 0x7f9514199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.720+0000 7f9519cf5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9514106a80 0x7f95141a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.720+0000 7f9519cf5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9514110f20 con 0x7f9514101420 2024-09-19T06:50:46.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.720+0000 7f9519cf5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9514110da0 con 0x7f9514106a80 2024-09-19T06:50:46.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.720+0000 7f9519cf5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f95141110a0 con 0x7f9514100a80 2024-09-19T06:50:46.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.720+0000 7f9512ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9514101420 0x7f9514199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.720+0000 7f9512ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9514101420 0x7f9514199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59360/0 (socket says 172.21.15.103:59360) 2024-09-19T06:50:46.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.720+0000 7f9512ffd640 1 -- 172.21.15.103:0/3144051421 learned_addr learned my addr 172.21.15.103:0/3144051421 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:46.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.721+0000 7f9513fff640 1 --2- 172.21.15.103:0/3144051421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9514106a80 0x7f95141a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.721+0000 7f95137fe640 1 --2- 172.21.15.103:0/3144051421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9514100a80 0x7f9514199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.721+0000 7f9512ffd640 1 -- 172.21.15.103:0/3144051421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9514100a80 msgr2=0x7f9514199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:46.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.721+0000 7f9512ffd640 1 --2- 172.21.15.103:0/3144051421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9514100a80 0x7f9514199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.721+0000 7f9512ffd640 1 -- 172.21.15.103:0/3144051421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9514106a80 msgr2=0x7f95141a44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:46.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.721+0000 7f9512ffd640 1 --2- 172.21.15.103:0/3144051421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9514106a80 0x7f95141a44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.721+0000 7f9512ffd640 1 -- 172.21.15.103:0/3144051421 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95141a4a10 con 0x7f9514101420 2024-09-19T06:50:46.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.721+0000 7f95137fe640 1 --2- 172.21.15.103:0/3144051421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9514100a80 0x7f9514199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:46.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.721+0000 7f9513fff640 1 --2- 172.21.15.103:0/3144051421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9514106a80 0x7f95141a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:46.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.721+0000 7f9512ffd640 1 --2- 172.21.15.103:0/3144051421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9514101420 0x7f9514199990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9508009a50 tx=0x7f9508004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:46.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.721+0000 7f9510ff9640 1 -- 172.21.15.103:0/3144051421 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950803f030 con 0x7f9514101420 2024-09-19T06:50:46.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.721+0000 7f9510ff9640 1 -- 172.21.15.103:0/3144051421 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9508003ce0 con 0x7f9514101420 2024-09-19T06:50:46.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.721+0000 7f9519cf5640 1 -- 172.21.15.103:0/3144051421 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95141a4be0 con 0x7f9514101420 2024-09-19T06:50:46.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.721+0000 7f9510ff9640 1 -- 172.21.15.103:0/3144051421 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508035070 con 0x7f9514101420 2024-09-19T06:50:46.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.721+0000 7f9519cf5640 1 -- 172.21.15.103:0/3144051421 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95141a5030 con 0x7f9514101420 2024-09-19T06:50:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.723+0000 7f9510ff9640 1 -- 172.21.15.103:0/3144051421 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9508042030 con 0x7f9514101420 2024-09-19T06:50:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.724+0000 7f951c0ce740 1 Processor -- start 2024-09-19T06:50:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.724+0000 7f951c0ce740 1 -- start start 2024-09-19T06:50:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.724+0000 7f951c0ce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81f3b60 0x55dcd81f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.724+0000 7f951c0ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f4480 0x55dcd81fe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.724+0000 7f951c0ce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81fed40 0x55dcd8201130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.724+0000 7f951c0ce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dcd8159280 con 0x55dcd81fed40 2024-09-19T06:50:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.724+0000 7f951c0ce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dcd8100b10 con 0x55dcd81f3b60 2024-09-19T06:50:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.724+0000 7f951c0ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dcd8155240 con 0x55dcd81f4480 2024-09-19T06:50:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.724+0000 7f9510ff9640 1 --2- 172.21.15.103:0/3144051421 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f94fc0a33b0 0x7f94fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.724+0000 7f9512ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81fed40 0x55dcd8201130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.724+0000 7f9512ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81fed40 0x55dcd8201130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59374/0 (socket says 172.21.15.103:59374) 2024-09-19T06:50:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.724+0000 7f9513fff640 1 --2- 172.21.15.103:0/3144051421 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f94fc0a33b0 0x7f94fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.724+0000 7f9512ffd640 1 -- 172.21.15.103:0/3311863291 learned_addr learned my addr 172.21.15.103:0/3311863291 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.724+0000 7f95137fe640 1 --2- 172.21.15.103:0/3311863291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f4480 0x55dcd81fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.724+0000 7f9513fff640 1 --2- 172.21.15.103:0/3311863291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81f3b60 0x55dcd81f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.724+0000 7f9510ff9640 1 -- 172.21.15.103:0/3144051421 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f95080f0350 con 0x7f9514101420 2024-09-19T06:50:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.725+0000 7f9512ffd640 1 -- 172.21.15.103:0/3311863291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f4480 msgr2=0x55dcd81fe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.725+0000 7f9512ffd640 1 --2- 172.21.15.103:0/3311863291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f4480 0x55dcd81fe800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.725+0000 7f9512ffd640 1 -- 172.21.15.103:0/3311863291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81f3b60 msgr2=0x55dcd81f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.725+0000 7f9512ffd640 1 --2- 172.21.15.103:0/3311863291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81f3b60 0x55dcd81f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.725+0000 7f9512ffd640 1 -- 172.21.15.103:0/3311863291 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcd810d1f0 con 0x55dcd81fed40 2024-09-19T06:50:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.725+0000 7f9513fff640 1 --2- 172.21.15.103:0/3144051421 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f94fc0a33b0 0x7f94fc0a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f950c009440 tx=0x7f950c009e20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.725+0000 7f9512ffd640 1 --2- 172.21.15.103:0/3311863291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81fed40 0x55dcd8201130 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f95080ed9f0 tx=0x7f95080314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a31716ebf6cf10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.725+0000 7f94f9ffb640 1 -- 172.21.15.103:0/3311863291 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508049070 con 0x55dcd81fed40 2024-09-19T06:50:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.725+0000 7f94f9ffb640 1 -- 172.21.15.103:0/3311863291 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95080bcd40 con 0x55dcd81fed40 2024-09-19T06:50:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.725+0000 7f94f9ffb640 1 -- 172.21.15.103:0/3311863291 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950803b460 con 0x55dcd81fed40 2024-09-19T06:50:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.726+0000 7f951c0ce740 1 -- 172.21.15.103:0/3311863291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81fed40 msgr2=0x55dcd8201130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.726+0000 7f951c0ce740 1 --2- 172.21.15.103:0/3311863291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81fed40 0x55dcd8201130 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f95080ed9f0 tx=0x7f95080314b0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.726+0000 7f951c0ce740 1 -- 172.21.15.103:0/3311863291 shutdown_connections 2024-09-19T06:50:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.726+0000 7f951c0ce740 1 --2- 172.21.15.103:0/3311863291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81fed40 0x55dcd8201130 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.726+0000 7f951c0ce740 1 --2- 172.21.15.103:0/3311863291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f4480 0x55dcd81fe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.726+0000 7f951c0ce740 1 --2- 172.21.15.103:0/3311863291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81f3b60 0x55dcd81f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.726+0000 7f951c0ce740 1 -- 172.21.15.103:0/3311863291 >> 172.21.15.103:0/3311863291 conn(0x55dcd81f00f0 msgr2=0x55dcd81f04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:46.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.726+0000 7f951c0ce740 1 -- 172.21.15.103:0/3311863291 shutdown_connections 2024-09-19T06:50:46.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.726+0000 7f951c0ce740 1 -- 172.21.15.103:0/3311863291 wait complete. 2024-09-19T06:50:46.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.726+0000 7f951c0ce740 1 -- start start 2024-09-19T06:50:46.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.726+0000 7f951c0ce740 1 Processor -- start 2024-09-19T06:50:46.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f951c0ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f3b60 0x55dcd81fbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f951c0ce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f4480 0x55dcd81fc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f951c0ce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81fed40 0x55dcd81fc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f951c0ce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dcd82016d0 con 0x55dcd81f4480 2024-09-19T06:50:46.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f951c0ce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dcd8113f10 con 0x55dcd81fed40 2024-09-19T06:50:46.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f951c0ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dcd8201850 con 0x55dcd81f3b60 2024-09-19T06:50:46.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f951c0ce740 10 client.0 _ll_register_callbacks cb 0x55dcd80dca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:46.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f9512ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f4480 0x55dcd81fc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f9512ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f4480 0x55dcd81fc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59390/0 (socket says 172.21.15.103:59390) 2024-09-19T06:50:46.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f9512ffd640 1 -- 172.21.15.103:0/2366046229 learned_addr learned my addr 172.21.15.103:0/2366046229 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:46.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f95137fe640 1 --2- 172.21.15.103:0/2366046229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f3b60 0x55dcd81fbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f9513fff640 1 --2- 172.21.15.103:0/2366046229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81fed40 0x55dcd81fc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f9512ffd640 1 -- 172.21.15.103:0/2366046229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f3b60 msgr2=0x55dcd81fbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:46.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f9512ffd640 1 --2- 172.21.15.103:0/2366046229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f3b60 0x55dcd81fbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f9512ffd640 1 -- 172.21.15.103:0/2366046229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81fed40 msgr2=0x55dcd81fc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:46.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f9512ffd640 1 --2- 172.21.15.103:0/2366046229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81fed40 0x55dcd81fc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f9512ffd640 1 -- 172.21.15.103:0/2366046229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcd7eca6a0 con 0x55dcd81f4480 2024-09-19T06:50:46.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f9512ffd640 1 --2- 172.21.15.103:0/2366046229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f4480 0x55dcd81fc190 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f95080b9010 tx=0x7f95080b8ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a338ae1f2f92e73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:46.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f94f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:46.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f94f2ffd640 1 -- 172.21.15.103:0/2366046229 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508049070 con 0x55dcd81f4480 2024-09-19T06:50:46.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f94f2ffd640 1 -- 172.21.15.103:0/2366046229 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f95080edd30 con 0x55dcd81f4480 2024-09-19T06:50:46.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.727+0000 7f951c0ce740 1 -- 172.21.15.103:0/2366046229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95080bcd40 con 0x55dcd81f4480 2024-09-19T06:50:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.728+0000 7f94f2ffd640 1 -- 172.21.15.103:0/2366046229 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f95080edf30 con 0x55dcd81f4480 2024-09-19T06:50:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.728+0000 7f951c0ce740 10 client.16098 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.728+0000 7f951c0ce740 1 -- 172.21.15.103:0/2366046229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dcd82016d0 con 0x55dcd81f4480 2024-09-19T06:50:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.728+0000 7f94f2ffd640 1 -- 172.21.15.103:0/2366046229 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95080bcd40 con 0x55dcd81f4480 2024-09-19T06:50:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.728+0000 7f94f2ffd640 1 -- 172.21.15.103:0/2366046229 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9508031bd0 con 0x55dcd81f4480 2024-09-19T06:50:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.728+0000 7f951c0ce740 10 client.16098 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.728+0000 7f951c0ce740 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.728+0000 7f951c0ce740 20 client.16098 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.728+0000 7f951c0ce740 1 --2- 172.21.15.103:0/2366046229 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dcd820e500 0x55dcd8210920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.728+0000 7f951c0ce740 4 client.16098 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.728+0000 7f951c0ce740 1 -- 172.21.15.103:0/2366046229 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcd81667f0 con 0x55dcd820e500 2024-09-19T06:50:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.728+0000 7f95137fe640 1 --2- 172.21.15.103:0/2366046229 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dcd820e500 0x55dcd8210920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.729+0000 7f94f2ffd640 10 client.16098 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.729+0000 7f95137fe640 1 --2- 172.21.15.103:0/2366046229 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dcd820e500 0x55dcd8210920 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1398f786f11896f server_cookie=891e75b039096d11 in_seq=0 out_seq=0 2024-09-19T06:50:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.729+0000 7f94f2ffd640 1 -- 172.21.15.103:0/2366046229 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dcd81667f0 con 0x55dcd820e500 2024-09-19T06:50:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.729+0000 7f94f2ffd640 10 client.16098 handle_command_reply: tid=0 2024-09-19T06:50:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.729+0000 7f951c0ce740 1 client.16098 shutdown 2024-09-19T06:50:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.729+0000 7f951c0ce740 1 -- 172.21.15.103:0/2366046229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f4480 msgr2=0x55dcd81fc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.729+0000 7f951c0ce740 1 --2- 172.21.15.103:0/2366046229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f4480 0x55dcd81fc190 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f95080b9010 tx=0x7f95080b8ee0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.730+0000 7f951c0ce740 1 -- 172.21.15.103:0/2366046229 shutdown_connections 2024-09-19T06:50:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.730+0000 7f951c0ce740 1 --2- 172.21.15.103:0/2366046229 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dcd820e500 0x55dcd8210920 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.730+0000 7f94f2ffd640 0 client.16098 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.730+0000 7f951c0ce740 1 --2- 172.21.15.103:0/2366046229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81fed40 0x55dcd81fc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.730+0000 7f951c0ce740 1 --2- 172.21.15.103:0/2366046229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f4480 0x55dcd81fc190 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.730+0000 7f951c0ce740 1 --2- 172.21.15.103:0/2366046229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f3b60 0x55dcd81fbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.730+0000 7f951c0ce740 1 -- 172.21.15.103:0/2366046229 >> 172.21.15.103:0/2366046229 conn(0x55dcd81f00f0 msgr2=0x55dcd81f5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:46.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.730+0000 7f951c0ce740 1 -- 172.21.15.103:0/2366046229 shutdown_connections 2024-09-19T06:50:46.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.730+0000 7f951c0ce740 1 -- 172.21.15.103:0/2366046229 wait complete. 2024-09-19T06:50:46.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.730+0000 7f951c0ce740 20 client.16098 trim_cache size 0 max 16384 2024-09-19T06:50:46.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.747+0000 7f951c0ce740 1 Processor -- start 2024-09-19T06:50:46.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.747+0000 7f951c0ce740 1 -- start start 2024-09-19T06:50:46.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.747+0000 7f951c0ce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f3b60 0x55dcd81fc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.747+0000 7f951c0ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f4480 0x55dcd81f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.747+0000 7f951c0ce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81fed40 0x55dcd82161f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.747+0000 7f951c0ce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f95141ab700 con 0x55dcd81f3b60 2024-09-19T06:50:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.747+0000 7f951c0ce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95141ad240 con 0x55dcd81fed40 2024-09-19T06:50:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.747+0000 7f951c0ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f95141abdd0 con 0x55dcd81f4480 2024-09-19T06:50:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.747+0000 7f95137fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f3b60 0x55dcd81fc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.747+0000 7f95137fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f3b60 0x55dcd81fc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59406/0 (socket says 172.21.15.103:59406) 2024-09-19T06:50:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.747+0000 7f95137fe640 1 -- 172.21.15.103:0/1365445589 learned_addr learned my addr 172.21.15.103:0/1365445589 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.747+0000 7f9512ffd640 1 --2- 172.21.15.103:0/1365445589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f4480 0x55dcd81f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.747+0000 7f95137fe640 1 -- 172.21.15.103:0/1365445589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f4480 msgr2=0x55dcd81f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.747+0000 7f95137fe640 1 --2- 172.21.15.103:0/1365445589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f4480 0x55dcd81f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.748+0000 7f95137fe640 1 -- 172.21.15.103:0/1365445589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81fed40 msgr2=0x55dcd82161f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.748+0000 7f9513fff640 1 --2- 172.21.15.103:0/1365445589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81fed40 0x55dcd82161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.748+0000 7f95137fe640 1 --2- 172.21.15.103:0/1365445589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81fed40 0x55dcd82161f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.748+0000 7f95137fe640 1 -- 172.21.15.103:0/1365445589 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcd81fa1a0 con 0x55dcd81f3b60 2024-09-19T06:50:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.748+0000 7f9512ffd640 1 --2- 172.21.15.103:0/1365445589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f4480 0x55dcd81f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.748+0000 7f9513fff640 1 --2- 172.21.15.103:0/1365445589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81fed40 0x55dcd82161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:46.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.748+0000 7f95137fe640 1 --2- 172.21.15.103:0/1365445589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f3b60 0x55dcd81fc520 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9504006a60 tx=0x7f9504008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde9c4ba0540dbc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:46.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.748+0000 7f94f97fa640 1 -- 172.21.15.103:0/1365445589 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9504014860 con 0x55dcd81f3b60 2024-09-19T06:50:46.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.748+0000 7f94f97fa640 1 -- 172.21.15.103:0/1365445589 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f950401b2d0 con 0x55dcd81f3b60 2024-09-19T06:50:46.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.748+0000 7f94f97fa640 1 -- 172.21.15.103:0/1365445589 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9504013600 con 0x55dcd81f3b60 2024-09-19T06:50:46.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 -- 172.21.15.103:0/1365445589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f3b60 msgr2=0x55dcd81fc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:46.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 --2- 172.21.15.103:0/1365445589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f3b60 0x55dcd81fc520 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9504006a60 tx=0x7f9504008040 comp rx=0 tx=0).stop 2024-09-19T06:50:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 -- 172.21.15.103:0/1365445589 shutdown_connections 2024-09-19T06:50:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 --2- 172.21.15.103:0/1365445589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81fed40 0x55dcd82161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 --2- 172.21.15.103:0/1365445589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f4480 0x55dcd81f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 --2- 172.21.15.103:0/1365445589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f3b60 0x55dcd81fc520 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 -- 172.21.15.103:0/1365445589 >> 172.21.15.103:0/1365445589 conn(0x55dcd81f00f0 msgr2=0x55dcd81f55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 -- 172.21.15.103:0/1365445589 shutdown_connections 2024-09-19T06:50:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 -- 172.21.15.103:0/1365445589 wait complete. 2024-09-19T06:50:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 -- start start 2024-09-19T06:50:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 Processor -- start 2024-09-19T06:50:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f3b60 0x55dcd82027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f4480 0x55dcd8202d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81fed40 0x55dcd8203240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f95141acb70 con 0x55dcd81f4480 2024-09-19T06:50:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95141ac4a0 con 0x55dcd81fed40 2024-09-19T06:50:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.749+0000 7f951c0ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f95141a96c0 con 0x55dcd81f3b60 2024-09-19T06:50:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f951c0ce740 10 client.0 _ll_register_callbacks cb 0x7f950c00a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f9512ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f4480 0x55dcd8202d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f9512ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f4480 0x55dcd8202d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59412/0 (socket says 172.21.15.103:59412) 2024-09-19T06:50:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f9512ffd640 1 -- 172.21.15.103:0/1990491133 learned_addr learned my addr 172.21.15.103:0/1990491133 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f9513fff640 1 --2- 172.21.15.103:0/1990491133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81fed40 0x55dcd8203240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f95137fe640 1 --2- 172.21.15.103:0/1990491133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f3b60 0x55dcd82027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f9512ffd640 1 -- 172.21.15.103:0/1990491133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f3b60 msgr2=0x55dcd82027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f9512ffd640 1 --2- 172.21.15.103:0/1990491133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f3b60 0x55dcd82027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f9512ffd640 1 -- 172.21.15.103:0/1990491133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81fed40 msgr2=0x55dcd8203240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f9512ffd640 1 --2- 172.21.15.103:0/1990491133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81fed40 0x55dcd8203240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f9512ffd640 1 -- 172.21.15.103:0/1990491133 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcd8278210 con 0x55dcd81f4480 2024-09-19T06:50:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f9512ffd640 1 --2- 172.21.15.103:0/1990491133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f4480 0x55dcd8202d00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f950804bff0 tx=0x7f9508005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f4adc023a5cd6c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f94f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f94f2ffd640 1 -- 172.21.15.103:0/1990491133 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95080c5680 con 0x55dcd81f4480 2024-09-19T06:50:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f94f2ffd640 1 -- 172.21.15.103:0/1990491133 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f9508031bd0 con 0x55dcd81f4480 2024-09-19T06:50:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f951c0ce740 1 -- 172.21.15.103:0/1990491133 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f950401b2d0 con 0x55dcd81f4480 2024-09-19T06:50:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f94f2ffd640 1 -- 172.21.15.103:0/1990491133 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f95080bcd40 con 0x55dcd81f4480 2024-09-19T06:50:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f951c0ce740 10 client.16104 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.750+0000 7f951c0ce740 1 -- 172.21.15.103:0/1990491133 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dcd8278210 con 0x55dcd81f4480 2024-09-19T06:50:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.751+0000 7f94f2ffd640 1 -- 172.21.15.103:0/1990491133 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f950401b2d0 con 0x55dcd81f4480 2024-09-19T06:50:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.751+0000 7f94f2ffd640 1 -- 172.21.15.103:0/1990491133 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f95080872f0 con 0x55dcd81f4480 2024-09-19T06:50:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.751+0000 7f951c0ce740 10 client.16104 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.751+0000 7f951c0ce740 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.751+0000 7f951c0ce740 20 client.16104 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.751+0000 7f951c0ce740 1 --2- 172.21.15.103:0/1990491133 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dcd820e500 0x55dcd827b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.751+0000 7f951c0ce740 4 client.16104 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:50:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.751+0000 7f951c0ce740 1 -- 172.21.15.103:0/1990491133 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dcd8160d00 con 0x55dcd820e500 2024-09-19T06:50:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.751+0000 7f95137fe640 1 --2- 172.21.15.103:0/1990491133 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dcd820e500 0x55dcd827b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.752+0000 7f94f2ffd640 10 client.16104 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.752+0000 7f95137fe640 1 --2- 172.21.15.103:0/1990491133 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dcd820e500 0x55dcd827b8f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e34cf068135f514 server_cookie=ccd27a1de9580942 in_seq=0 out_seq=0 2024-09-19T06:50:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.753+0000 7f94f2ffd640 1 -- 172.21.15.103:0/1990491133 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dcd8160d00 con 0x55dcd820e500 2024-09-19T06:50:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.753+0000 7f94f2ffd640 10 client.16104 handle_command_reply: tid=0 2024-09-19T06:50:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.753+0000 7f951c0ce740 1 client.16104 shutdown 2024-09-19T06:50:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.753+0000 7f951c0ce740 1 -- 172.21.15.103:0/1990491133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f4480 msgr2=0x55dcd8202d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.753+0000 7f951c0ce740 1 --2- 172.21.15.103:0/1990491133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f4480 0x55dcd8202d00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f950804bff0 tx=0x7f9508005c90 comp rx=0 tx=0).stop 2024-09-19T06:50:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.753+0000 7f951c0ce740 1 -- 172.21.15.103:0/1990491133 shutdown_connections 2024-09-19T06:50:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.753+0000 7f951c0ce740 1 --2- 172.21.15.103:0/1990491133 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dcd820e500 0x55dcd827b8f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.753+0000 7f951c0ce740 1 --2- 172.21.15.103:0/1990491133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd81fed40 0x55dcd8203240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.753+0000 7f951c0ce740 1 --2- 172.21.15.103:0/1990491133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd81f4480 0x55dcd8202d00 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.753+0000 7f951c0ce740 1 --2- 172.21.15.103:0/1990491133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd81f3b60 0x55dcd82027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.753+0000 7f951c0ce740 1 -- 172.21.15.103:0/1990491133 >> 172.21.15.103:0/1990491133 conn(0x55dcd81f00f0 msgr2=0x55dcd81ef270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.753+0000 7f94f2ffd640 0 client.16104 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.753+0000 7f951c0ce740 1 -- 172.21.15.103:0/1990491133 shutdown_connections 2024-09-19T06:50:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.753+0000 7f951c0ce740 1 -- 172.21.15.103:0/1990491133 wait complete. 2024-09-19T06:50:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.753+0000 7f951c0ce740 20 client.16104 trim_cache size 0 max 16384 2024-09-19T06:50:46.759 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:50:46.759 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T06:50:46.759 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "ad1c75bb-b5ef-4b64-bb0e-3dae641004f0", 2024-09-19T06:50:46.760 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T06:50:46.760 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:50:46.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.754+0000 7f9519cf5640 1 -- 172.21.15.103:0/3144051421 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f94fc0a33b0 msgr2=0x7f94fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:46.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.754+0000 7f9519cf5640 1 --2- 172.21.15.103:0/3144051421 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f94fc0a33b0 0x7f94fc0a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f950c009440 tx=0x7f950c009e20 comp rx=0 tx=0).stop 2024-09-19T06:50:46.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.754+0000 7f9519cf5640 1 -- 172.21.15.103:0/3144051421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9514101420 msgr2=0x7f9514199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:46.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.754+0000 7f9519cf5640 1 --2- 172.21.15.103:0/3144051421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9514101420 0x7f9514199990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9508009a50 tx=0x7f9508004270 comp rx=0 tx=0).stop 2024-09-19T06:50:46.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.754+0000 7f9519cf5640 1 -- 172.21.15.103:0/3144051421 shutdown_connections 2024-09-19T06:50:46.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.754+0000 7f9519cf5640 1 --2- 172.21.15.103:0/3144051421 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f94fc0a33b0 0x7f94fc0a5870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.754+0000 7f9519cf5640 1 --2- 172.21.15.103:0/3144051421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9514106a80 0x7f95141a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.754+0000 7f9519cf5640 1 --2- 172.21.15.103:0/3144051421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9514101420 0x7f9514199990 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.754+0000 7f9519cf5640 1 --2- 172.21.15.103:0/3144051421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9514100a80 0x7f9514199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.754+0000 7f9519cf5640 1 -- 172.21.15.103:0/3144051421 >> 172.21.15.103:0/3144051421 conn(0x7f95140fc7a0 msgr2=0x7f95140fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.754+0000 7f9519cf5640 1 -- 172.21.15.103:0/3144051421 shutdown_connections 2024-09-19T06:50:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:46.754+0000 7f9519cf5640 1 -- 172.21.15.103:0/3144051421 wait complete. 2024-09-19T06:50:47.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:47 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T06:50:47.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:47 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:47.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:47 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T06:50:47.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:47 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:47.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:47 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T06:50:47.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:47 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:47.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:47 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T06:50:47.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:47 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:47.438 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:50:47.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:47 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T06:50:47.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:47 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:47.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:47 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T06:50:47.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:47 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:47.685 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:48.127 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:48 smithi103 ceph-mon[31158]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-09-19T06:50:48.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.126+0000 7f5f2c457640 1 -- 172.21.15.103:0/2858561996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f241082a0 msgr2=0x7f5f2410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.126+0000 7f5f2c457640 1 --2- 172.21.15.103:0/2858561996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f241082a0 0x7f5f2410ec40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009cd0 tx=0x7f5f1c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.126+0000 7f5f2c457640 1 -- 172.21.15.103:0/2858561996 shutdown_connections 2024-09-19T06:50:48.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.126+0000 7f5f2c457640 1 --2- 172.21.15.103:0/2858561996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f241082a0 0x7f5f2410ec40 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.126+0000 7f5f2c457640 1 --2- 172.21.15.103:0/2858561996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f241036e0 0x7f5f24107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.126+0000 7f5f2c457640 1 --2- 172.21.15.103:0/2858561996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f24102d30 0x7f5f24103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.126+0000 7f5f2c457640 1 -- 172.21.15.103:0/2858561996 >> 172.21.15.103:0/2858561996 conn(0x7f5f240fc7d0 msgr2=0x7f5f240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:48.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.126+0000 7f5f2c457640 1 -- 172.21.15.103:0/2858561996 shutdown_connections 2024-09-19T06:50:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.126+0000 7f5f2c457640 1 -- 172.21.15.103:0/2858561996 wait complete. 2024-09-19T06:50:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f2c457640 1 Processor -- start 2024-09-19T06:50:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f2c457640 1 -- start start 2024-09-19T06:50:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f2c457640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f24102d30 0x7f5f241a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f2c457640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f241036e0 0x7f5f241a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f2c457640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f241082a0 0x7f5f2410dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f2c457640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241120a0 con 0x7f5f241036e0 2024-09-19T06:50:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f2c457640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5f24111f20 con 0x7f5f24102d30 2024-09-19T06:50:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f2c457640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f24112220 con 0x7f5f241082a0 2024-09-19T06:50:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f299f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f241036e0 0x7f5f241a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f299f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f241036e0 0x7f5f241a4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43760/0 (socket says 172.21.15.103:43760) 2024-09-19T06:50:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f299f2640 1 -- 172.21.15.103:0/1434470665 learned_addr learned my addr 172.21.15.103:0/1434470665 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f2a1f3640 1 --2- 172.21.15.103:0/1434470665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f24102d30 0x7f5f241a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f299f2640 1 -- 172.21.15.103:0/1434470665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f241082a0 msgr2=0x7f5f2410dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f299f2640 1 --2- 172.21.15.103:0/1434470665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f241082a0 0x7f5f2410dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f299f2640 1 -- 172.21.15.103:0/1434470665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f24102d30 msgr2=0x7f5f241a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f299f2640 1 --2- 172.21.15.103:0/1434470665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f24102d30 0x7f5f241a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f299f2640 1 -- 172.21.15.103:0/1434470665 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2410e5a0 con 0x7f5f241036e0 2024-09-19T06:50:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f2a1f3640 1 --2- 172.21.15.103:0/1434470665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f24102d30 0x7f5f241a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.127+0000 7f5f299f2640 1 --2- 172.21.15.103:0/1434470665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f241036e0 0x7f5f241a4d30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400c8c0 tx=0x7f5f1400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.128+0000 7f5f1b7fe640 1 -- 172.21.15.103:0/1434470665 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14007bb0 con 0x7f5f241036e0 2024-09-19T06:50:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.128+0000 7f5f2c457640 1 -- 172.21.15.103:0/1434470665 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2410e890 con 0x7f5f241036e0 2024-09-19T06:50:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.128+0000 7f5f1b7fe640 1 -- 172.21.15.103:0/1434470665 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f14007d50 con 0x7f5f241036e0 2024-09-19T06:50:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.128+0000 7f5f1b7fe640 1 -- 172.21.15.103:0/1434470665 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14018830 con 0x7f5f241036e0 2024-09-19T06:50:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.128+0000 7f5f2c457640 1 -- 172.21.15.103:0/1434470665 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f241ab7b0 con 0x7f5f241036e0 2024-09-19T06:50:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.130+0000 7f5f1b7fe640 1 -- 172.21.15.103:0/1434470665 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f5f14018a10 con 0x7f5f241036e0 2024-09-19T06:50:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.130+0000 7f5f2e830740 1 Processor -- start 2024-09-19T06:50:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.130+0000 7f5f2e830740 1 -- start start 2024-09-19T06:50:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2e830740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5e4a0 0x5631d6f5e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2e830740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f5edc0 0x5631d6f69140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2e830740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f69680 0x5631d6f6ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f1b7fe640 1 --2- 172.21.15.103:0/1434470665 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5f100a33b0 0x7f5f100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2e830740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5631d6ec42e0 con 0x5631d6f5e4a0 2024-09-19T06:50:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2e830740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5631d6e6bce0 con 0x5631d6f69680 2024-09-19T06:50:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2e830740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5631d6ec02a0 con 0x5631d6f5edc0 2024-09-19T06:50:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2a9f4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5e4a0 0x5631d6f5e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2a9f4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5e4a0 0x5631d6f5e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43768/0 (socket says 172.21.15.103:43768) 2024-09-19T06:50:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2a9f4640 1 -- 172.21.15.103:0/1081567728 learned_addr learned my addr 172.21.15.103:0/1081567728 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2a9f4640 1 --2- 172.21.15.103:0/1434470665 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5f100a33b0 0x7f5f100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f1b7fe640 1 -- 172.21.15.103:0/1434470665 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5f140cfea0 con 0x7f5f241036e0 2024-09-19T06:50:48.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2a1f3640 1 --2- 172.21.15.103:0/1081567728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f5edc0 0x5631d6f69140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f299f2640 1 --2- 172.21.15.103:0/1081567728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f69680 0x5631d6f6ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2a9f4640 1 -- 172.21.15.103:0/1081567728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f5edc0 msgr2=0x5631d6f69140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:48.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2a9f4640 1 --2- 172.21.15.103:0/1081567728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f5edc0 0x5631d6f69140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2a9f4640 1 -- 172.21.15.103:0/1081567728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f69680 msgr2=0x5631d6f6ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:48.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2a9f4640 1 --2- 172.21.15.103:0/1081567728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f69680 0x5631d6f6ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2a9f4640 1 -- 172.21.15.103:0/1081567728 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631d6e50440 con 0x5631d6f5e4a0 2024-09-19T06:50:48.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f299f2640 1 --2- 172.21.15.103:0/1081567728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f69680 0x5631d6f6ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:48.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2a9f4640 1 --2- 172.21.15.103:0/1081567728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5e4a0 0x5631d6f5e880 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0097b0 tx=0x7f5f1c031e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5aa6f0dba1c122d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:48.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.131+0000 7f5f2a9f4640 1 --2- 172.21.15.103:0/1434470665 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5f100a33b0 0x7f5f100a5870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c02ae00 tx=0x7f5f1c03f000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:48.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.132+0000 7f5f18dea640 1 -- 172.21.15.103:0/1081567728 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c04b070 con 0x5631d6f5e4a0 2024-09-19T06:50:48.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.132+0000 7f5f18dea640 1 -- 172.21.15.103:0/1081567728 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f1c036050 con 0x5631d6f5e4a0 2024-09-19T06:50:48.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.132+0000 7f5f18dea640 1 -- 172.21.15.103:0/1081567728 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c03a880 con 0x5631d6f5e4a0 2024-09-19T06:50:48.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.132+0000 7f5f2e830740 1 -- 172.21.15.103:0/1081567728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5e4a0 msgr2=0x5631d6f5e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.132+0000 7f5f2e830740 1 --2- 172.21.15.103:0/1081567728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5e4a0 0x5631d6f5e880 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0097b0 tx=0x7f5f1c031e70 comp rx=0 tx=0).stop 2024-09-19T06:50:48.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.132+0000 7f5f2e830740 1 -- 172.21.15.103:0/1081567728 shutdown_connections 2024-09-19T06:50:48.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.132+0000 7f5f2e830740 1 --2- 172.21.15.103:0/1081567728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f69680 0x5631d6f6ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.132+0000 7f5f2e830740 1 --2- 172.21.15.103:0/1081567728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f5edc0 0x5631d6f69140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.132+0000 7f5f2e830740 1 --2- 172.21.15.103:0/1081567728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5e4a0 0x5631d6f5e880 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.132+0000 7f5f2e830740 1 -- 172.21.15.103:0/1081567728 >> 172.21.15.103:0/1081567728 conn(0x5631d6f5aa30 msgr2=0x5631d6f5ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:48.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.133+0000 7f5f2e830740 1 -- 172.21.15.103:0/1081567728 shutdown_connections 2024-09-19T06:50:48.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.133+0000 7f5f2e830740 1 -- 172.21.15.103:0/1081567728 wait complete. 2024-09-19T06:50:48.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.133+0000 7f5f2e830740 1 -- start start 2024-09-19T06:50:48.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.133+0000 7f5f2e830740 1 Processor -- start 2024-09-19T06:50:48.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.133+0000 7f5f2e830740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5e4a0 0x5631d6f68fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.133+0000 7f5f2e830740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5edc0 0x5631d6f66280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.133+0000 7f5f2e830740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f69680 0x5631d6f667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.133+0000 7f5f2e830740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5631d6f6c250 con 0x5631d6f5edc0 2024-09-19T06:50:48.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.133+0000 7f5f2e830740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5631d6e7f050 con 0x5631d6f5e4a0 2024-09-19T06:50:48.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.133+0000 7f5f2e830740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5631d6f6c3d0 con 0x5631d6f69680 2024-09-19T06:50:48.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.133+0000 7f5f2e830740 10 client.0 _ll_register_callbacks cb 0x5631d6ebf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:48.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.133+0000 7f5f299f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5edc0 0x5631d6f66280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.133+0000 7f5f299f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5edc0 0x5631d6f66280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43786/0 (socket says 172.21.15.103:43786) 2024-09-19T06:50:48.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.133+0000 7f5f299f2640 1 -- 172.21.15.103:0/3941888214 learned_addr learned my addr 172.21.15.103:0/3941888214 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:48.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.134+0000 7f5f2a1f3640 1 --2- 172.21.15.103:0/3941888214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5e4a0 0x5631d6f68fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.134+0000 7f5f299f2640 1 -- 172.21.15.103:0/3941888214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f69680 msgr2=0x5631d6f667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:48.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.134+0000 7f5f299f2640 1 --2- 172.21.15.103:0/3941888214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f69680 0x5631d6f667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.134+0000 7f5f299f2640 1 -- 172.21.15.103:0/3941888214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5e4a0 msgr2=0x5631d6f68fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:48.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.134+0000 7f5f299f2640 1 --2- 172.21.15.103:0/3941888214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5e4a0 0x5631d6f68fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.134+0000 7f5f299f2640 1 -- 172.21.15.103:0/3941888214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631d6c353c0 con 0x5631d6f5edc0 2024-09-19T06:50:48.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.134+0000 7f5f2a1f3640 1 --2- 172.21.15.103:0/3941888214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5e4a0 0x5631d6f68fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:48.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.134+0000 7f5f299f2640 1 --2- 172.21.15.103:0/3941888214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5edc0 0x5631d6f66280 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5f1409be50 tx=0x7f5f1401f300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0058d4f98f5f0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:48.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.134+0000 7f5f0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:48.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.134+0000 7f5f0dffb640 1 -- 172.21.15.103:0/3941888214 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f140a0500 con 0x5631d6f5edc0 2024-09-19T06:50:48.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.134+0000 7f5f0dffb640 1 -- 172.21.15.103:0/3941888214 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5f140a0190 con 0x5631d6f5edc0 2024-09-19T06:50:48.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.134+0000 7f5f2e830740 1 -- 172.21.15.103:0/3941888214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f1c036050 con 0x5631d6f5edc0 2024-09-19T06:50:48.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.134+0000 7f5f0dffb640 1 -- 172.21.15.103:0/3941888214 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f140a4dc0 con 0x5631d6f5edc0 2024-09-19T06:50:48.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.134+0000 7f5f2e830740 10 client.16116 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:48.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.134+0000 7f5f2e830740 1 -- 172.21.15.103:0/3941888214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5631d6f77c20 con 0x5631d6f5edc0 2024-09-19T06:50:48.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.134+0000 7f5f0dffb640 1 -- 172.21.15.103:0/3941888214 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f1c036050 con 0x5631d6f5edc0 2024-09-19T06:50:48.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.135+0000 7f5f0dffb640 1 -- 172.21.15.103:0/3941888214 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5f140cd840 con 0x5631d6f5edc0 2024-09-19T06:50:48.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.135+0000 7f5f2e830740 10 client.16116 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:48.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.135+0000 7f5f2e830740 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:48.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.135+0000 7f5f2e830740 20 client.16116 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:48.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.135+0000 7f5f2e830740 1 --2- 172.21.15.103:0/3941888214 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5631d6f78fc0 0x5631d6f7b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.135+0000 7f5f2e830740 4 client.16116 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:48.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.135+0000 7f5f2e830740 1 -- 172.21.15.103:0/3941888214 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631d6ed1850 con 0x5631d6f78fc0 2024-09-19T06:50:48.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.135+0000 7f5f2a1f3640 1 --2- 172.21.15.103:0/3941888214 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5631d6f78fc0 0x5631d6f7b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.135+0000 7f5f0dffb640 10 client.16116 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.135+0000 7f5f2a1f3640 1 --2- 172.21.15.103:0/3941888214 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5631d6f78fc0 0x5631d6f7b410 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6196e342ebea6511 server_cookie=9241d0c274dc414a in_seq=0 out_seq=0 2024-09-19T06:50:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.136+0000 7f5f0dffb640 1 -- 172.21.15.103:0/3941888214 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631d6ed1850 con 0x5631d6f78fc0 2024-09-19T06:50:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.136+0000 7f5f0dffb640 10 client.16116 handle_command_reply: tid=0 2024-09-19T06:50:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.136+0000 7f5f2e830740 1 client.16116 shutdown 2024-09-19T06:50:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.136+0000 7f5f2e830740 1 -- 172.21.15.103:0/3941888214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5edc0 msgr2=0x5631d6f66280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.136+0000 7f5f2e830740 1 --2- 172.21.15.103:0/3941888214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5edc0 0x5631d6f66280 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5f1409be50 tx=0x7f5f1401f300 comp rx=0 tx=0).stop 2024-09-19T06:50:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.136+0000 7f5f2e830740 1 -- 172.21.15.103:0/3941888214 shutdown_connections 2024-09-19T06:50:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.136+0000 7f5f2e830740 1 --2- 172.21.15.103:0/3941888214 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5631d6f78fc0 0x5631d6f7b410 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.136+0000 7f5f2e830740 1 --2- 172.21.15.103:0/3941888214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f69680 0x5631d6f667e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.136+0000 7f5f2e830740 1 --2- 172.21.15.103:0/3941888214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5edc0 0x5631d6f66280 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.136+0000 7f5f0dffb640 0 client.16116 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.136+0000 7f5f2e830740 1 --2- 172.21.15.103:0/3941888214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5e4a0 0x5631d6f68fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.136+0000 7f5f2e830740 1 -- 172.21.15.103:0/3941888214 >> 172.21.15.103:0/3941888214 conn(0x5631d6f5aa30 msgr2=0x5631d6f60710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.136+0000 7f5f2e830740 1 -- 172.21.15.103:0/3941888214 shutdown_connections 2024-09-19T06:50:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.136+0000 7f5f2e830740 1 -- 172.21.15.103:0/3941888214 wait complete. 2024-09-19T06:50:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.136+0000 7f5f2e830740 20 client.16116 trim_cache size 0 max 16384 2024-09-19T06:50:48.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.153+0000 7f5f2e830740 1 Processor -- start 2024-09-19T06:50:48.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.153+0000 7f5f2e830740 1 -- start start 2024-09-19T06:50:48.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f2e830740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f5e4a0 0x5631d6f64b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f2e830740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5edc0 0x5631d6f80ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f2e830740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f69680 0x5631d6f81220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f2e830740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241b3e00 con 0x5631d6f69680 2024-09-19T06:50:48.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f2e830740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241b5940 con 0x5631d6f5edc0 2024-09-19T06:50:48.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f2e830740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241b44d0 con 0x5631d6f5e4a0 2024-09-19T06:50:48.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f2a9f4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f69680 0x5631d6f81220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f2a9f4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f69680 0x5631d6f81220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43792/0 (socket says 172.21.15.103:43792) 2024-09-19T06:50:48.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f2a9f4640 1 -- 172.21.15.103:0/3141000120 learned_addr learned my addr 172.21.15.103:0/3141000120 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:48.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f299f2640 1 --2- 172.21.15.103:0/3141000120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5edc0 0x5631d6f80ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f2a1f3640 1 --2- 172.21.15.103:0/3141000120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f5e4a0 0x5631d6f64b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f2a9f4640 1 -- 172.21.15.103:0/3141000120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f5e4a0 msgr2=0x5631d6f64b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:48.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f2a9f4640 1 --2- 172.21.15.103:0/3141000120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f5e4a0 0x5631d6f64b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f2a9f4640 1 -- 172.21.15.103:0/3141000120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5edc0 msgr2=0x5631d6f80ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:48.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f2a9f4640 1 --2- 172.21.15.103:0/3141000120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5edc0 0x5631d6f80ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f2a9f4640 1 -- 172.21.15.103:0/3141000120 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631d6f65230 con 0x5631d6f69680 2024-09-19T06:50:48.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f299f2640 1 --2- 172.21.15.103:0/3141000120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5edc0 0x5631d6f80ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:48.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.154+0000 7f5f2a9f4640 1 --2- 172.21.15.103:0/3141000120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f69680 0x5631d6f81220 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c031f00 tx=0x7f5f1c005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fcd26bf2c21943 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:48.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.155+0000 7f5f0ffff640 1 -- 172.21.15.103:0/3141000120 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c055070 con 0x5631d6f69680 2024-09-19T06:50:48.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.155+0000 7f5f0ffff640 1 -- 172.21.15.103:0/3141000120 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f1c036040 con 0x5631d6f69680 2024-09-19T06:50:48.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.155+0000 7f5f0ffff640 1 -- 172.21.15.103:0/3141000120 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c04b040 con 0x5631d6f69680 2024-09-19T06:50:48.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.155+0000 7f5f2e830740 1 -- 172.21.15.103:0/3141000120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f69680 msgr2=0x5631d6f81220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.155+0000 7f5f2e830740 1 --2- 172.21.15.103:0/3141000120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f69680 0x5631d6f81220 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c031f00 tx=0x7f5f1c005c30 comp rx=0 tx=0).stop 2024-09-19T06:50:48.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.155+0000 7f5f2e830740 1 -- 172.21.15.103:0/3141000120 shutdown_connections 2024-09-19T06:50:48.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.155+0000 7f5f2e830740 1 --2- 172.21.15.103:0/3141000120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f69680 0x5631d6f81220 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.156+0000 7f5f2e830740 1 --2- 172.21.15.103:0/3141000120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5edc0 0x5631d6f80ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.156+0000 7f5f2e830740 1 --2- 172.21.15.103:0/3141000120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f5e4a0 0x5631d6f64b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.156+0000 7f5f2e830740 1 -- 172.21.15.103:0/3141000120 >> 172.21.15.103:0/3141000120 conn(0x5631d6f5aa30 msgr2=0x5631d6f68300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:48.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.156+0000 7f5f2e830740 1 -- 172.21.15.103:0/3141000120 shutdown_connections 2024-09-19T06:50:48.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.156+0000 7f5f2e830740 1 -- 172.21.15.103:0/3141000120 wait complete. 2024-09-19T06:50:48.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.156+0000 7f5f2e830740 1 -- start start 2024-09-19T06:50:48.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.156+0000 7f5f2e830740 1 Processor -- start 2024-09-19T06:50:48.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.156+0000 7f5f2e830740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5e4a0 0x5631d6f6f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.156+0000 7f5f2e830740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5edc0 0x5631d6f6fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.156+0000 7f5f2e830740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f69680 0x5631d6f70080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f2e830740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241b5270 con 0x5631d6f5edc0 2024-09-19T06:50:48.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f2e830740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241b4ba0 con 0x5631d6f5e4a0 2024-09-19T06:50:48.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f2e830740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241b1dc0 con 0x5631d6f69680 2024-09-19T06:50:48.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f299f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5edc0 0x5631d6f6fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f299f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5edc0 0x5631d6f6fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43802/0 (socket says 172.21.15.103:43802) 2024-09-19T06:50:48.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f299f2640 1 -- 172.21.15.103:0/1311080597 learned_addr learned my addr 172.21.15.103:0/1311080597 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:48.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f2e830740 10 client.0 _ll_register_callbacks cb 0x7f5f140983f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:48.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f2a1f3640 1 --2- 172.21.15.103:0/1311080597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5e4a0 0x5631d6f6f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f2a9f4640 1 --2- 172.21.15.103:0/1311080597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f69680 0x5631d6f70080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f299f2640 1 -- 172.21.15.103:0/1311080597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f69680 msgr2=0x5631d6f70080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:48.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f299f2640 1 --2- 172.21.15.103:0/1311080597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f69680 0x5631d6f70080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f299f2640 1 -- 172.21.15.103:0/1311080597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5e4a0 msgr2=0x5631d6f6f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:48.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f299f2640 1 --2- 172.21.15.103:0/1311080597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5e4a0 0x5631d6f6f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f299f2640 1 -- 172.21.15.103:0/1311080597 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631d6f6bde0 con 0x5631d6f5edc0 2024-09-19T06:50:48.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f2a1f3640 1 --2- 172.21.15.103:0/1311080597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5e4a0 0x5631d6f6f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:48.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f299f2640 1 --2- 172.21.15.103:0/1311080597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5edc0 0x5631d6f6fb40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5f1409c090 tx=0x7f5f1409c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43022d40fbcaa31f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:48.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:48.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f0dffb640 1 -- 172.21.15.103:0/1311080597 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f140a0500 con 0x5631d6f5edc0 2024-09-19T06:50:48.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f0dffb640 1 -- 172.21.15.103:0/1311080597 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5f140bc0b0 con 0x5631d6f5edc0 2024-09-19T06:50:48.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f2e830740 1 -- 172.21.15.103:0/1311080597 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f1c036040 con 0x5631d6f5edc0 2024-09-19T06:50:48.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.157+0000 7f5f0dffb640 1 -- 172.21.15.103:0/1311080597 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f140a4930 con 0x5631d6f5edc0 2024-09-19T06:50:48.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.158+0000 7f5f2e830740 10 client.16122 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:48.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.158+0000 7f5f2e830740 1 -- 172.21.15.103:0/1311080597 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5631d6fe3fc0 con 0x5631d6f5edc0 2024-09-19T06:50:48.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.158+0000 7f5f0dffb640 1 -- 172.21.15.103:0/1311080597 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f1c036040 con 0x5631d6f5edc0 2024-09-19T06:50:48.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.158+0000 7f5f0dffb640 1 -- 172.21.15.103:0/1311080597 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5f14097ea0 con 0x5631d6f5edc0 2024-09-19T06:50:48.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.158+0000 7f5f2e830740 10 client.16122 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:48.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.158+0000 7f5f2e830740 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:48.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.158+0000 7f5f2e830740 20 client.16122 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:48.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.158+0000 7f5f2e830740 1 --2- 172.21.15.103:0/1311080597 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5631d6fe4520 0x5631d6fe6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.158+0000 7f5f2e830740 4 client.16122 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:50:48.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.158+0000 7f5f2e830740 1 -- 172.21.15.103:0/1311080597 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5631d6ecbd60 con 0x5631d6fe4520 2024-09-19T06:50:48.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.158+0000 7f5f2a1f3640 1 --2- 172.21.15.103:0/1311080597 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5631d6fe4520 0x5631d6fe6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f2a1f3640 1 --2- 172.21.15.103:0/1311080597 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5631d6fe4520 0x5631d6fe6940 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9a9356ac1430428 server_cookie=5e1c3c0126df8411 in_seq=0 out_seq=0 2024-09-19T06:50:48.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f0dffb640 10 client.16122 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:48.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f0dffb640 1 -- 172.21.15.103:0/1311080597 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5631d6ecbd60 con 0x5631d6fe4520 2024-09-19T06:50:48.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f0dffb640 10 client.16122 handle_command_reply: tid=0 2024-09-19T06:50:48.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f2e830740 1 client.16122 shutdown 2024-09-19T06:50:48.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f2e830740 1 -- 172.21.15.103:0/1311080597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5edc0 msgr2=0x5631d6f6fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f2e830740 1 --2- 172.21.15.103:0/1311080597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5edc0 0x5631d6f6fb40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5f1409c090 tx=0x7f5f1409c8d0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f2e830740 1 -- 172.21.15.103:0/1311080597 shutdown_connections 2024-09-19T06:50:48.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f2e830740 1 --2- 172.21.15.103:0/1311080597 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5631d6fe4520 0x5631d6fe6940 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f2e830740 1 --2- 172.21.15.103:0/1311080597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631d6f69680 0x5631d6f70080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f2e830740 1 --2- 172.21.15.103:0/1311080597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631d6f5edc0 0x5631d6f6fb40 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f2e830740 1 --2- 172.21.15.103:0/1311080597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631d6f5e4a0 0x5631d6f6f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f2e830740 1 -- 172.21.15.103:0/1311080597 >> 172.21.15.103:0/1311080597 conn(0x5631d6f5aa30 msgr2=0x5631d6f68300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:48.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f0dffb640 0 client.16122 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:48.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f2e830740 1 -- 172.21.15.103:0/1311080597 shutdown_connections 2024-09-19T06:50:48.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f2e830740 1 -- 172.21.15.103:0/1311080597 wait complete. 2024-09-19T06:50:48.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.159+0000 7f5f2e830740 20 client.16122 trim_cache size 0 max 16384 2024-09-19T06:50:48.176 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:50:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:50:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T06:50:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "ad1c75bb-b5ef-4b64-bb0e-3dae641004f0": { 2024-09-19T06:50:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T06:50:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "ad1c75bb-b5ef-4b64-bb0e-3dae641004f0", 2024-09-19T06:50:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T06:50:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:48.177 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:50:48.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.160+0000 7f5f2c457640 1 -- 172.21.15.103:0/1434470665 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5f100a33b0 msgr2=0x7f5f100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.160+0000 7f5f2c457640 1 --2- 172.21.15.103:0/1434470665 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5f100a33b0 0x7f5f100a5870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c02ae00 tx=0x7f5f1c03f000 comp rx=0 tx=0).stop 2024-09-19T06:50:48.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.160+0000 7f5f2c457640 1 -- 172.21.15.103:0/1434470665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f241036e0 msgr2=0x7f5f241a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.160+0000 7f5f2c457640 1 --2- 172.21.15.103:0/1434470665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f241036e0 0x7f5f241a4d30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400c8c0 tx=0x7f5f1400cd80 comp rx=0 tx=0).stop 2024-09-19T06:50:48.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.160+0000 7f5f2c457640 1 -- 172.21.15.103:0/1434470665 shutdown_connections 2024-09-19T06:50:48.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.160+0000 7f5f2c457640 1 --2- 172.21.15.103:0/1434470665 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5f100a33b0 0x7f5f100a5870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.160+0000 7f5f2c457640 1 --2- 172.21.15.103:0/1434470665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f241082a0 0x7f5f2410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.160+0000 7f5f2c457640 1 --2- 172.21.15.103:0/1434470665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f241036e0 0x7f5f241a4d30 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.160+0000 7f5f2c457640 1 --2- 172.21.15.103:0/1434470665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f24102d30 0x7f5f241a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.160+0000 7f5f2c457640 1 -- 172.21.15.103:0/1434470665 >> 172.21.15.103:0/1434470665 conn(0x7f5f240fc7d0 msgr2=0x7f5f240fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:48.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.160+0000 7f5f2c457640 1 -- 172.21.15.103:0/1434470665 shutdown_connections 2024-09-19T06:50:48.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:48.160+0000 7f5f2c457640 1 -- 172.21.15.103:0/1434470665 wait complete. 2024-09-19T06:50:48.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:48 smithi174 ceph-mon[35637]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-09-19T06:50:48.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:48 smithi134 ceph-mon[35356]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-09-19T06:50:48.720 INFO:tasks.cephfs.filesystem:scrub status for tag:ad1c75bb-b5ef-4b64-bb0e-3dae641004f0 - {'path': '/', 'tag': 'ad1c75bb-b5ef-4b64-bb0e-3dae641004f0', 'options': 'recursive,force'} 2024-09-19T06:50:50.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:49 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T06:50:50.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:49 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T06:50:50.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:49 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T06:50:51.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:50 smithi103 ceph-mon[31158]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 19 op/s 2024-09-19T06:50:51.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:50 smithi134 ceph-mon[35356]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 19 op/s 2024-09-19T06:50:51.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:50 smithi174 ceph-mon[35637]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 19 op/s 2024-09-19T06:50:52.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:52 smithi174 ceph-mon[35637]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-09-19T06:50:52.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:52 smithi103 ceph-mon[31158]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-09-19T06:50:52.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:52 smithi134 ceph-mon[35356]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-09-19T06:50:54.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:54 smithi174 ceph-mon[35637]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-09-19T06:50:54.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:54 smithi103 ceph-mon[31158]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-09-19T06:50:54.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:54 smithi134 ceph-mon[35356]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-09-19T06:50:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:56 smithi174 ceph-mon[35637]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 19 op/s 2024-09-19T06:50:56.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:56 smithi103 ceph-mon[31158]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 19 op/s 2024-09-19T06:50:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:56 smithi134 ceph-mon[35356]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 19 op/s 2024-09-19T06:50:57.038 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:50:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.298+0000 7f1ccac5e640 1 -- 172.21.15.103:0/219615252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cc4101ad0 msgr2=0x7f1cc4101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.298+0000 7f1ccac5e640 1 --2- 172.21.15.103:0/219615252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cc4101ad0 0x7f1cc4101f30 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009930 tx=0x7f1cb002a420 comp rx=0 tx=0).stop 2024-09-19T06:50:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.298+0000 7f1ccac5e640 1 -- 172.21.15.103:0/219615252 shutdown_connections 2024-09-19T06:50:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.298+0000 7f1ccac5e640 1 --2- 172.21.15.103:0/219615252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cc4102470 0x7f1cc410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.298+0000 7f1ccac5e640 1 --2- 172.21.15.103:0/219615252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cc4101ad0 0x7f1cc4101f30 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.298+0000 7f1ccac5e640 1 --2- 172.21.15.103:0/219615252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc4107b70 0x7f1cc4107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.298+0000 7f1ccac5e640 1 -- 172.21.15.103:0/219615252 >> 172.21.15.103:0/219615252 conn(0x7f1cc40fd7d0 msgr2=0x7f1cc40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.298+0000 7f1ccac5e640 1 -- 172.21.15.103:0/219615252 shutdown_connections 2024-09-19T06:50:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.298+0000 7f1ccac5e640 1 -- 172.21.15.103:0/219615252 wait complete. 2024-09-19T06:50:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.299+0000 7f1ccac5e640 1 Processor -- start 2024-09-19T06:50:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.299+0000 7f1ccac5e640 1 -- start start 2024-09-19T06:50:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.299+0000 7f1ccac5e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cc4101ad0 0x7f1cc4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.299+0000 7f1ccac5e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cc4102470 0x7f1cc41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.299+0000 7f1ccac5e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc4107b70 0x7f1cc419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.299+0000 7f1ccac5e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc4111ee0 con 0x7f1cc4102470 2024-09-19T06:50:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.299+0000 7f1ccac5e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc4111d60 con 0x7f1cc4107b70 2024-09-19T06:50:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.299+0000 7f1ccac5e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc4112060 con 0x7f1cc4101ad0 2024-09-19T06:50:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.299+0000 7f1cc3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cc4102470 0x7f1cc41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.299+0000 7f1cc3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cc4102470 0x7f1cc41988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50324/0 (socket says 172.21.15.103:50324) 2024-09-19T06:50:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.299+0000 7f1cc3fff640 1 -- 172.21.15.103:0/3726298108 learned_addr learned my addr 172.21.15.103:0/3726298108 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.299+0000 7f1cc89fa640 1 --2- 172.21.15.103:0/3726298108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cc4101ad0 0x7f1cc4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.300+0000 7f1cc3fff640 1 -- 172.21.15.103:0/3726298108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cc4101ad0 msgr2=0x7f1cc4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.300+0000 7f1cc91fb640 1 --2- 172.21.15.103:0/3726298108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc4107b70 0x7f1cc419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.300+0000 7f1cc3fff640 1 --2- 172.21.15.103:0/3726298108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cc4101ad0 0x7f1cc4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.300+0000 7f1cc3fff640 1 -- 172.21.15.103:0/3726298108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc4107b70 msgr2=0x7f1cc419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.300+0000 7f1cc3fff640 1 --2- 172.21.15.103:0/3726298108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc4107b70 0x7f1cc419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.300+0000 7f1cc3fff640 1 -- 172.21.15.103:0/3726298108 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc419d360 con 0x7f1cc4102470 2024-09-19T06:50:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.300+0000 7f1cc89fa640 1 --2- 172.21.15.103:0/3726298108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cc4101ad0 0x7f1cc4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.300+0000 7f1cc3fff640 1 --2- 172.21.15.103:0/3726298108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cc4102470 0x7f1cc41988b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009230 tx=0x7f1cb0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.300+0000 7f1cc1ffb640 1 -- 172.21.15.103:0/3726298108 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb003f030 con 0x7f1cc4102470 2024-09-19T06:50:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.300+0000 7f1cc1ffb640 1 -- 172.21.15.103:0/3726298108 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb0003ad0 con 0x7f1cc4102470 2024-09-19T06:50:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.300+0000 7f1ccac5e640 1 -- 172.21.15.103:0/3726298108 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc419d5f0 con 0x7f1cc4102470 2024-09-19T06:50:57.304 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.300+0000 7f1cc1ffb640 1 -- 172.21.15.103:0/3726298108 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0035070 con 0x7f1cc4102470 2024-09-19T06:50:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.300+0000 7f1ccac5e640 1 -- 172.21.15.103:0/3726298108 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc41a4ee0 con 0x7f1cc4102470 2024-09-19T06:50:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.302+0000 7f1cc1ffb640 1 -- 172.21.15.103:0/3726298108 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1cb0042030 con 0x7f1cc4102470 2024-09-19T06:50:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1ccd9fb740 1 Processor -- start 2024-09-19T06:50:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1ccd9fb740 1 -- start start 2024-09-19T06:50:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1ccd9fb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1bdaf0 0x56502b1bded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1ccd9fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1be410 0x56502b1c8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1ccd9fb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1ccd9fb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56502b1222f0 con 0x56502b1c8cb0 2024-09-19T06:50:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1ccd9fb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56502b0ca270 con 0x56502b1bdaf0 2024-09-19T06:50:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1ccd9fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56502b11e2b0 con 0x56502b1be410 2024-09-19T06:50:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1cc1ffb640 1 --2- 172.21.15.103:0/3726298108 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1cb40a3460 0x7f1cb40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1cc3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1cc91fb640 1 --2- 172.21.15.103:0/3726298108 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1cb40a3460 0x7f1cb40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1cc3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50338/0 (socket says 172.21.15.103:50338) 2024-09-19T06:50:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1cc3fff640 1 -- 172.21.15.103:0/3419865998 learned_addr learned my addr 172.21.15.103:0/3419865998 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1cc91fb640 1 --2- 172.21.15.103:0/3419865998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1bdaf0 0x56502b1bded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1cc89fa640 1 --2- 172.21.15.103:0/3419865998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1be410 0x56502b1c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1cc1ffb640 1 -- 172.21.15.103:0/3726298108 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1cb00f0470 con 0x7f1cc4102470 2024-09-19T06:50:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1cc3fff640 1 -- 172.21.15.103:0/3419865998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1be410 msgr2=0x56502b1c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1cc3fff640 1 --2- 172.21.15.103:0/3419865998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1be410 0x56502b1c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1cc3fff640 1 -- 172.21.15.103:0/3419865998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1bdaf0 msgr2=0x56502b1bded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1cc3fff640 1 --2- 172.21.15.103:0/3419865998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1bdaf0 0x56502b1bded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1cc3fff640 1 -- 172.21.15.103:0/3419865998 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56502b0c9340 con 0x56502b1c8cb0 2024-09-19T06:50:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.303+0000 7f1cc91fb640 1 --2- 172.21.15.103:0/3419865998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1bdaf0 0x56502b1bded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.304+0000 7f1cc91fb640 1 --2- 172.21.15.103:0/3726298108 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1cb40a3460 0x7f1cb40a5920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009fd0 tx=0x7f1cbc009dc0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.304+0000 7f1cc3fff640 1 --2- 172.21.15.103:0/3419865998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1cb0a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1cb003f000 tx=0x7f1cb00904f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2664b9a4a445c49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.304+0000 7f1caeffd640 1 -- 172.21.15.103:0/3419865998 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0049070 con 0x56502b1c8cb0 2024-09-19T06:50:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.304+0000 7f1caeffd640 1 -- 172.21.15.103:0/3419865998 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb00c5390 con 0x56502b1c8cb0 2024-09-19T06:50:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.304+0000 7f1caeffd640 1 -- 172.21.15.103:0/3419865998 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00b8d90 con 0x56502b1c8cb0 2024-09-19T06:50:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.305+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/3419865998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 msgr2=0x56502b1cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.305+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/3419865998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1cb0a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1cb003f000 tx=0x7f1cb00904f0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.305+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/3419865998 shutdown_connections 2024-09-19T06:50:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.305+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/3419865998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1cb0a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.305+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/3419865998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1be410 0x56502b1c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.305+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/3419865998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1bdaf0 0x56502b1bded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.305+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/3419865998 >> 172.21.15.103:0/3419865998 conn(0x56502b1ba060 msgr2=0x56502b1ba440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.305+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/3419865998 shutdown_connections 2024-09-19T06:50:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.305+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/3419865998 wait complete. 2024-09-19T06:50:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.305+0000 7f1ccd9fb740 1 -- start start 2024-09-19T06:50:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.305+0000 7f1ccd9fb740 1 Processor -- start 2024-09-19T06:50:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1ccd9fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1bdaf0 0x56502b1c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1ccd9fb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1be410 0x56502b1c6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1ccd9fb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1c6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1ccd9fb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56502b1cb640 con 0x56502b1c8cb0 2024-09-19T06:50:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1ccd9fb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56502b0ab290 con 0x56502b1be410 2024-09-19T06:50:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1ccd9fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56502b1cb7c0 con 0x56502b1bdaf0 2024-09-19T06:50:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1ccd9fb740 10 client.0 _ll_register_callbacks cb 0x56502b0dc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1cc91fb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1cc3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1be410 0x56502b1c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1cc91fb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50346/0 (socket says 172.21.15.103:50346) 2024-09-19T06:50:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1cc91fb640 1 -- 172.21.15.103:0/4184477171 learned_addr learned my addr 172.21.15.103:0/4184477171 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1cc89fa640 1 --2- 172.21.15.103:0/4184477171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1bdaf0 0x56502b1c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1cc91fb640 1 -- 172.21.15.103:0/4184477171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1bdaf0 msgr2=0x56502b1c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1cc91fb640 1 --2- 172.21.15.103:0/4184477171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1bdaf0 0x56502b1c5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1cc91fb640 1 -- 172.21.15.103:0/4184477171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1be410 msgr2=0x56502b1c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1cc91fb640 1 --2- 172.21.15.103:0/4184477171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1be410 0x56502b1c6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1cc91fb640 1 -- 172.21.15.103:0/4184477171 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56502ae35210 con 0x56502b1c8cb0 2024-09-19T06:50:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1cc3fff640 1 --2- 172.21.15.103:0/4184477171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1be410 0x56502b1c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1cc89fa640 1 --2- 172.21.15.103:0/4184477171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1bdaf0 0x56502b1c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1cc91fb640 1 --2- 172.21.15.103:0/4184477171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1c6680 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc012010 tx=0x7f1cbc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95379ff899271c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1ca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1ca3fff640 1 -- 172.21.15.103:0/4184477171 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0047d0 con 0x56502b1c8cb0 2024-09-19T06:50:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1ca3fff640 1 -- 172.21.15.103:0/4184477171 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1cbc021480 con 0x56502b1c8cb0 2024-09-19T06:50:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.306+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/4184477171 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb00c5390 con 0x56502b1c8cb0 2024-09-19T06:50:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.307+0000 7f1ca3fff640 1 -- 172.21.15.103:0/4184477171 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1cbc021680 con 0x56502b1c8cb0 2024-09-19T06:50:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.307+0000 7f1ccd9fb740 10 client.16134 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.307+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/4184477171 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56502af01620 con 0x56502b1c8cb0 2024-09-19T06:50:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.307+0000 7f1ca3fff640 1 -- 172.21.15.103:0/4184477171 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb00c5390 con 0x56502b1c8cb0 2024-09-19T06:50:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.307+0000 7f1ca3fff640 1 -- 172.21.15.103:0/4184477171 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1cbc0127e0 con 0x56502b1c8cb0 2024-09-19T06:50:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.307+0000 7f1ccd9fb740 10 client.16134 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.307+0000 7f1ccd9fb740 10 client.16134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.307+0000 7f1ccd9fb740 20 client.16134 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.307+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/4184477171 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56502b1d8970 0x56502b1dad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.307+0000 7f1ccd9fb740 4 client.16134 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.307+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/4184477171 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56502b12fa20 con 0x56502b1d8970 2024-09-19T06:50:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.307+0000 7f1cc89fa640 1 --2- 172.21.15.103:0/4184477171 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56502b1d8970 0x56502b1dad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.308+0000 7f1ca3fff640 10 client.16134 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.308+0000 7f1cc89fa640 1 --2- 172.21.15.103:0/4184477171 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56502b1d8970 0x56502b1dad60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=230a026740200e03 server_cookie=fbddf16660b30f9e in_seq=0 out_seq=0 2024-09-19T06:50:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.308+0000 7f1ca3fff640 1 -- 172.21.15.103:0/4184477171 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56502b12fa20 con 0x56502b1d8970 2024-09-19T06:50:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.308+0000 7f1ca3fff640 10 client.16134 handle_command_reply: tid=0 2024-09-19T06:50:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.308+0000 7f1ccd9fb740 1 client.16134 shutdown 2024-09-19T06:50:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.309+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/4184477171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 msgr2=0x56502b1c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.309+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/4184477171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1c6680 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc012010 tx=0x7f1cbc009500 comp rx=0 tx=0).stop 2024-09-19T06:50:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.309+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/4184477171 shutdown_connections 2024-09-19T06:50:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.309+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/4184477171 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56502b1d8970 0x56502b1dad60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.309+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/4184477171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1c6680 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.309+0000 7f1ca3fff640 0 client.16134 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.309+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/4184477171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1be410 0x56502b1c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.309+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/4184477171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1bdaf0 0x56502b1c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.309+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/4184477171 >> 172.21.15.103:0/4184477171 conn(0x56502b1ba060 msgr2=0x56502b1bfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.309+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/4184477171 shutdown_connections 2024-09-19T06:50:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.309+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/4184477171 wait complete. 2024-09-19T06:50:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.309+0000 7f1ccd9fb740 20 client.16134 trim_cache size 0 max 16384 2024-09-19T06:50:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.324+0000 7f1ccd9fb740 1 Processor -- start 2024-09-19T06:50:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.324+0000 7f1ccd9fb740 1 -- start start 2024-09-19T06:50:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.324+0000 7f1ccd9fb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1bdaf0 0x56502b1c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.324+0000 7f1ccd9fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1be410 0x56502b1e0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.324+0000 7f1ccd9fb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.324+0000 7f1ccd9fb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41ad7b0 con 0x56502b1c8cb0 2024-09-19T06:50:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.324+0000 7f1ccd9fb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41af2f0 con 0x56502b1bdaf0 2024-09-19T06:50:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.324+0000 7f1ccd9fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41ade80 con 0x56502b1be410 2024-09-19T06:50:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.324+0000 7f1cc91fb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.324+0000 7f1cc91fb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1e0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50362/0 (socket says 172.21.15.103:50362) 2024-09-19T06:50:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.324+0000 7f1cc91fb640 1 -- 172.21.15.103:0/4270904447 learned_addr learned my addr 172.21.15.103:0/4270904447 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.324+0000 7f1cc3fff640 1 --2- 172.21.15.103:0/4270904447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1be410 0x56502b1e0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.325+0000 7f1cc89fa640 1 --2- 172.21.15.103:0/4270904447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1bdaf0 0x56502b1c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.325+0000 7f1cc91fb640 1 -- 172.21.15.103:0/4270904447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1be410 msgr2=0x56502b1e0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.325+0000 7f1cc91fb640 1 --2- 172.21.15.103:0/4270904447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1be410 0x56502b1e0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.325+0000 7f1cc91fb640 1 -- 172.21.15.103:0/4270904447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1bdaf0 msgr2=0x56502b1c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.325+0000 7f1cc91fb640 1 --2- 172.21.15.103:0/4270904447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1bdaf0 0x56502b1c64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.325+0000 7f1cc91fb640 1 -- 172.21.15.103:0/4270904447 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56502b1bacf0 con 0x56502b1c8cb0 2024-09-19T06:50:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.325+0000 7f1cc91fb640 1 --2- 172.21.15.103:0/4270904447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1e0b70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0117d0 tx=0x7f1cbc011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d00e93c39a5e4c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.325+0000 7f1cae7fc640 1 -- 172.21.15.103:0/4270904447 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc003e00 con 0x56502b1c8cb0 2024-09-19T06:50:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.325+0000 7f1cae7fc640 1 -- 172.21.15.103:0/4270904447 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cbc014400 con 0x56502b1c8cb0 2024-09-19T06:50:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.325+0000 7f1cae7fc640 1 -- 172.21.15.103:0/4270904447 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0128b0 con 0x56502b1c8cb0 2024-09-19T06:50:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.326+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/4270904447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 msgr2=0x56502b1e0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.326+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/4270904447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1e0b70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0117d0 tx=0x7f1cbc011b90 comp rx=0 tx=0).stop 2024-09-19T06:50:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.326+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/4270904447 shutdown_connections 2024-09-19T06:50:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.326+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/4270904447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1c8cb0 0x56502b1e0b70 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.326+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/4270904447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1be410 0x56502b1e0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.326+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/4270904447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1bdaf0 0x56502b1c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.326+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/4270904447 >> 172.21.15.103:0/4270904447 conn(0x56502b1ba060 msgr2=0x56502b1c7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.326+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/4270904447 shutdown_connections 2024-09-19T06:50:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.326+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/4270904447 wait complete. 2024-09-19T06:50:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.326+0000 7f1ccd9fb740 1 -- start start 2024-09-19T06:50:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.326+0000 7f1ccd9fb740 1 Processor -- start 2024-09-19T06:50:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1ccd9fb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1bdaf0 0x56502b1d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1ccd9fb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1be410 0x56502b1cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1ccd9fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1c8cb0 0x56502b1cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1ccd9fb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41aec20 con 0x56502b1bdaf0 2024-09-19T06:50:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1ccd9fb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41ae550 con 0x56502b1be410 2024-09-19T06:50:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1ccd9fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41ab770 con 0x56502b1c8cb0 2024-09-19T06:50:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1ccd9fb740 10 client.0 _ll_register_callbacks cb 0x7f1cbc009220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1cc89fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1bdaf0 0x56502b1d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1cc89fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1bdaf0 0x56502b1d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50368/0 (socket says 172.21.15.103:50368) 2024-09-19T06:50:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1cc89fa640 1 -- 172.21.15.103:0/3639539470 learned_addr learned my addr 172.21.15.103:0/3639539470 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1cc91fb640 1 --2- 172.21.15.103:0/3639539470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1c8cb0 0x56502b1cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1cc89fa640 1 -- 172.21.15.103:0/3639539470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1c8cb0 msgr2=0x56502b1cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1cc89fa640 1 --2- 172.21.15.103:0/3639539470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1c8cb0 0x56502b1cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1cc89fa640 1 -- 172.21.15.103:0/3639539470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1be410 msgr2=0x56502b1cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1cc89fa640 1 --2- 172.21.15.103:0/3639539470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1be410 0x56502b1cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1cc89fa640 1 -- 172.21.15.103:0/3639539470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56502b1cce90 con 0x56502b1bdaf0 2024-09-19T06:50:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1cc91fb640 1 --2- 172.21.15.103:0/3639539470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1c8cb0 0x56502b1cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1cc89fa640 1 --2- 172.21.15.103:0/3639539470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1bdaf0 0x56502b1d0fc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009880 tx=0x7f1cb8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfa821dc7016236 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1ca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1ca3fff640 1 -- 172.21.15.103:0/3639539470 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8005020 con 0x56502b1bdaf0 2024-09-19T06:50:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1ca3fff640 1 -- 172.21.15.103:0/3639539470 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1cb80051c0 con 0x56502b1bdaf0 2024-09-19T06:50:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.327+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/3639539470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cbc014400 con 0x56502b1bdaf0 2024-09-19T06:50:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.328+0000 7f1ca3fff640 1 -- 172.21.15.103:0/3639539470 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1cb80053c0 con 0x56502b1bdaf0 2024-09-19T06:50:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.328+0000 7f1ccd9fb740 10 client.16140 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.328+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/3639539470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56502b2434e0 con 0x56502b1bdaf0 2024-09-19T06:50:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.328+0000 7f1ca3fff640 1 -- 172.21.15.103:0/3639539470 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cbc014400 con 0x56502b1bdaf0 2024-09-19T06:50:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.328+0000 7f1ca3fff640 1 -- 172.21.15.103:0/3639539470 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1cb800f040 con 0x56502b1bdaf0 2024-09-19T06:50:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.328+0000 7f1ccd9fb740 10 client.16140 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.328+0000 7f1ccd9fb740 10 client.16140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.328+0000 7f1ccd9fb740 20 client.16140 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.328+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/3639539470 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56502b243c30 0x56502b246050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.328+0000 7f1ccd9fb740 4 client.16140 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:50:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.328+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/3639539470 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56502b129d70 con 0x56502b243c30 2024-09-19T06:50:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.328+0000 7f1cc3fff640 1 --2- 172.21.15.103:0/3639539470 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56502b243c30 0x56502b246050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.329+0000 7f1cc3fff640 1 --2- 172.21.15.103:0/3639539470 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56502b243c30 0x56502b246050 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cf6695697236e84 server_cookie=6fda15076dcea5a5 in_seq=0 out_seq=0 2024-09-19T06:50:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.329+0000 7f1ca3fff640 10 client.16140 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.336+0000 7f1ca3fff640 1 -- 172.21.15.103:0/3639539470 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56502b129d70 con 0x56502b243c30 2024-09-19T06:50:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.336+0000 7f1ca3fff640 10 client.16140 handle_command_reply: tid=0 2024-09-19T06:50:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.336+0000 7f1ccd9fb740 1 client.16140 shutdown 2024-09-19T06:50:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.336+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/3639539470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1bdaf0 msgr2=0x56502b1d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.336+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/3639539470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1bdaf0 0x56502b1d0fc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009880 tx=0x7f1cb8012820 comp rx=0 tx=0).stop 2024-09-19T06:50:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.336+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/3639539470 shutdown_connections 2024-09-19T06:50:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.336+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/3639539470 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56502b243c30 0x56502b246050 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.336+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/3639539470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56502b1c8cb0 0x56502b1cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.336+0000 7f1ca3fff640 0 client.16140 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.336+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/3639539470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56502b1be410 0x56502b1cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.336+0000 7f1ccd9fb740 1 --2- 172.21.15.103:0/3639539470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502b1bdaf0 0x56502b1d0fc0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.336+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/3639539470 >> 172.21.15.103:0/3639539470 conn(0x56502b1ba060 msgr2=0x56502b1bf550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.336+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/3639539470 shutdown_connections 2024-09-19T06:50:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.336+0000 7f1ccd9fb740 1 -- 172.21.15.103:0/3639539470 wait complete. 2024-09-19T06:50:57.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.336+0000 7f1ccd9fb740 20 client.16140 trim_cache size 0 max 16384 2024-09-19T06:50:57.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:50:57.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:50:57.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:50:57.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:50:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.337+0000 7f1ccac5e640 1 -- 172.21.15.103:0/3726298108 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1cb40a3460 msgr2=0x7f1cb40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.337+0000 7f1ccac5e640 1 --2- 172.21.15.103:0/3726298108 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1cb40a3460 0x7f1cb40a5920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009fd0 tx=0x7f1cbc009dc0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.337+0000 7f1ccac5e640 1 -- 172.21.15.103:0/3726298108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cc4102470 msgr2=0x7f1cc41988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.337+0000 7f1ccac5e640 1 --2- 172.21.15.103:0/3726298108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cc4102470 0x7f1cc41988b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009230 tx=0x7f1cb0004270 comp rx=0 tx=0).stop 2024-09-19T06:50:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.337+0000 7f1ccac5e640 1 -- 172.21.15.103:0/3726298108 shutdown_connections 2024-09-19T06:50:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.337+0000 7f1ccac5e640 1 --2- 172.21.15.103:0/3726298108 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1cb40a3460 0x7f1cb40a5920 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.337+0000 7f1ccac5e640 1 --2- 172.21.15.103:0/3726298108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cc4107b70 0x7f1cc419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.337+0000 7f1ccac5e640 1 --2- 172.21.15.103:0/3726298108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cc4102470 0x7f1cc41988b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.337+0000 7f1ccac5e640 1 --2- 172.21.15.103:0/3726298108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cc4101ad0 0x7f1cc4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.337+0000 7f1ccac5e640 1 -- 172.21.15.103:0/3726298108 >> 172.21.15.103:0/3726298108 conn(0x7f1cc40fd7d0 msgr2=0x7f1cc40fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.337+0000 7f1ccac5e640 1 -- 172.21.15.103:0/3726298108 shutdown_connections 2024-09-19T06:50:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:50:57.337+0000 7f1ccac5e640 1 -- 172.21.15.103:0/3726298108 wait complete. 2024-09-19T06:50:57.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.757+0000 7f4fdd7ac640 1 -- 172.21.15.103:0/3433312229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fd8101de0 msgr2=0x7f4fd81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:57.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.757+0000 7f4fdd7ac640 1 --2- 172.21.15.103:0/3433312229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fd8101de0 0x7f4fd81021c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009900 tx=0x7f4fc802a410 comp rx=0 tx=0).stop 2024-09-19T06:50:57.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.757+0000 7f4fdd7ac640 1 -- 172.21.15.103:0/3433312229 shutdown_connections 2024-09-19T06:50:57.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.757+0000 7f4fdd7ac640 1 --2- 172.21.15.103:0/3433312229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fd8106f10 0x7f4fd810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.757+0000 7f4fdd7ac640 1 --2- 172.21.15.103:0/3433312229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd8102790 0x7f4fd81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.757+0000 7f4fdd7ac640 1 --2- 172.21.15.103:0/3433312229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fd8101de0 0x7f4fd81021c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.757+0000 7f4fdd7ac640 1 -- 172.21.15.103:0/3433312229 >> 172.21.15.103:0/3433312229 conn(0x7f4fd80fd8c0 msgr2=0x7f4fd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:57.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.757+0000 7f4fdd7ac640 1 -- 172.21.15.103:0/3433312229 shutdown_connections 2024-09-19T06:50:57.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.757+0000 7f4fdd7ac640 1 -- 172.21.15.103:0/3433312229 wait complete. 2024-09-19T06:50:57.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.758+0000 7f4fdd7ac640 1 Processor -- start 2024-09-19T06:50:57.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.758+0000 7f4fdd7ac640 1 -- start start 2024-09-19T06:50:57.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.758+0000 7f4fdd7ac640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fd8101de0 0x7f4fd8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.758+0000 7f4fdd7ac640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd8102790 0x7f4fd8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.758+0000 7f4fdd7ac640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fd8106f10 0x7f4fd819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.758+0000 7f4fdd7ac640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8111f70 con 0x7f4fd8101de0 2024-09-19T06:50:57.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.758+0000 7f4fdd7ac640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8111df0 con 0x7f4fd8102790 2024-09-19T06:50:57.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.758+0000 7f4fdd7ac640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81120f0 con 0x7f4fd8106f10 2024-09-19T06:50:57.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.758+0000 7f4fd6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fd8101de0 0x7f4fd8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.758+0000 7f4fd6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fd8101de0 0x7f4fd8198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50378/0 (socket says 172.21.15.103:50378) 2024-09-19T06:50:57.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.758+0000 7f4fd6ffd640 1 -- 172.21.15.103:0/1552365977 learned_addr learned my addr 172.21.15.103:0/1552365977 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:57.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.758+0000 7f4fd77fe640 1 --2- 172.21.15.103:0/1552365977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fd8106f10 0x7f4fd819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.758+0000 7f4fce7fc640 1 --2- 172.21.15.103:0/1552365977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd8102790 0x7f4fd8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.759+0000 7f4fd6ffd640 1 -- 172.21.15.103:0/1552365977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fd8106f10 msgr2=0x7f4fd819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:57.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.759+0000 7f4fd6ffd640 1 --2- 172.21.15.103:0/1552365977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fd8106f10 0x7f4fd819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.759+0000 7f4fd6ffd640 1 -- 172.21.15.103:0/1552365977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd8102790 msgr2=0x7f4fd8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:57.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.759+0000 7f4fd6ffd640 1 --2- 172.21.15.103:0/1552365977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd8102790 0x7f4fd8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.759+0000 7f4fd6ffd640 1 -- 172.21.15.103:0/1552365977 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd819d3e0 con 0x7f4fd8101de0 2024-09-19T06:50:57.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.759+0000 7f4fd6ffd640 1 --2- 172.21.15.103:0/1552365977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fd8101de0 0x7f4fd8198430 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006010 tx=0x7f4fc8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:57.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.759+0000 7f4fd4ff9640 1 -- 172.21.15.103:0/1552365977 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc803f030 con 0x7f4fd8101de0 2024-09-19T06:50:57.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.759+0000 7f4fdd7ac640 1 -- 172.21.15.103:0/1552365977 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd819d670 con 0x7f4fd8101de0 2024-09-19T06:50:57.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.759+0000 7f4fd4ff9640 1 -- 172.21.15.103:0/1552365977 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fc8004440 con 0x7f4fd8101de0 2024-09-19T06:50:57.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.759+0000 7f4fd4ff9640 1 -- 172.21.15.103:0/1552365977 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8035070 con 0x7f4fd8101de0 2024-09-19T06:50:57.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.759+0000 7f4fdd7ac640 1 -- 172.21.15.103:0/1552365977 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd81a4fb0 con 0x7f4fd8101de0 2024-09-19T06:50:57.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.761+0000 7f4fdd7ac640 1 -- 172.21.15.103:0/1552365977 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd81032c0 con 0x7f4fd8101de0 2024-09-19T06:50:57.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.762+0000 7f4fd4ff9640 1 -- 172.21.15.103:0/1552365977 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4fc8042030 con 0x7f4fd8101de0 2024-09-19T06:50:57.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.762+0000 7f4fd4ff9640 1 --2- 172.21.15.103:0/1552365977 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fc40a33b0 0x7f4fc40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:57.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.763+0000 7f4fce7fc640 1 --2- 172.21.15.103:0/1552365977 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fc40a33b0 0x7f4fc40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:57.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.763+0000 7f4fd4ff9640 1 -- 172.21.15.103:0/1552365977 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f4fc80f00a0 con 0x7f4fd8101de0 2024-09-19T06:50:57.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.763+0000 7f4fce7fc640 1 --2- 172.21.15.103:0/1552365977 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fc40a33b0 0x7f4fc40a5870 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80fdce0 tx=0x7f4fc0005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:57.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.767+0000 7f4fd4ff9640 1 -- 172.21.15.103:0/1552365977 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4fc80bc400 con 0x7f4fd8101de0 2024-09-19T06:50:57.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.967+0000 7f4fdd7ac640 1 -- 172.21.15.103:0/1552365977 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fd810a0f0 con 0x7f4fd8101de0 2024-09-19T06:50:57.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.968+0000 7f4fd4ff9640 1 -- 172.21.15.103:0/1552365977 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f4fc80c0700 con 0x7f4fd8101de0 2024-09-19T06:50:57.969 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:50:57.969 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:50:57.970 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:50:57.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.970+0000 7f4fdd7ac640 1 -- 172.21.15.103:0/1552365977 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fc40a33b0 msgr2=0x7f4fc40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:57.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.970+0000 7f4fdd7ac640 1 --2- 172.21.15.103:0/1552365977 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fc40a33b0 0x7f4fc40a5870 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80fdce0 tx=0x7f4fc0005f90 comp rx=0 tx=0).stop 2024-09-19T06:50:57.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.970+0000 7f4fdd7ac640 1 -- 172.21.15.103:0/1552365977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fd8101de0 msgr2=0x7f4fd8198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:57.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.970+0000 7f4fdd7ac640 1 --2- 172.21.15.103:0/1552365977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fd8101de0 0x7f4fd8198430 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006010 tx=0x7f4fc8004040 comp rx=0 tx=0).stop 2024-09-19T06:50:57.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.970+0000 7f4fdd7ac640 1 -- 172.21.15.103:0/1552365977 shutdown_connections 2024-09-19T06:50:57.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.970+0000 7f4fdd7ac640 1 --2- 172.21.15.103:0/1552365977 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fc40a33b0 0x7f4fc40a5870 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.970+0000 7f4fdd7ac640 1 --2- 172.21.15.103:0/1552365977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fd8106f10 0x7f4fd819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.970+0000 7f4fdd7ac640 1 --2- 172.21.15.103:0/1552365977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd8102790 0x7f4fd8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.970+0000 7f4fdd7ac640 1 --2- 172.21.15.103:0/1552365977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fd8101de0 0x7f4fd8198430 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:57.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.970+0000 7f4fdd7ac640 1 -- 172.21.15.103:0/1552365977 >> 172.21.15.103:0/1552365977 conn(0x7f4fd80fd8c0 msgr2=0x7f4fd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:57.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.970+0000 7f4fdd7ac640 1 -- 172.21.15.103:0/1552365977 shutdown_connections 2024-09-19T06:50:57.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:57.970+0000 7f4fdd7ac640 1 -- 172.21.15.103:0/1552365977 wait complete. 2024-09-19T06:50:58.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:58 smithi174 ceph-mon[35637]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-09-19T06:50:58.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:50:58 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1552365977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:58.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:58 smithi103 ceph-mon[31158]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-09-19T06:50:58.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:50:58 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1552365977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:58.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:58 smithi134 ceph-mon[35356]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-09-19T06:50:58.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:50:58 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1552365977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:58.609 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 51253b89 --release --await' 2024-09-19T06:50:58.609 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 51253b89 --release --await 2024-09-19T06:50:58.854 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:59.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.313+0000 7f906e07c640 1 -- 172.21.15.103:0/1295526657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9068106a80 msgr2=0x7f9068106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:59.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.313+0000 7f906e07c640 1 --2- 172.21.15.103:0/1295526657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9068106a80 0x7f9068106e60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9058009930 tx=0x7f905802a420 comp rx=0 tx=0).stop 2024-09-19T06:50:59.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.313+0000 7f906e07c640 1 -- 172.21.15.103:0/1295526657 shutdown_connections 2024-09-19T06:50:59.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.313+0000 7f906e07c640 1 --2- 172.21.15.103:0/1295526657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9068101420 0x7f906810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.313+0000 7f906e07c640 1 --2- 172.21.15.103:0/1295526657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9068100a80 0x7f9068100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.313+0000 7f906e07c640 1 --2- 172.21.15.103:0/1295526657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9068106a80 0x7f9068106e60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.313+0000 7f906e07c640 1 -- 172.21.15.103:0/1295526657 >> 172.21.15.103:0/1295526657 conn(0x7f90680fc7a0 msgr2=0x7f90680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:59.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.314+0000 7f906e07c640 1 -- 172.21.15.103:0/1295526657 shutdown_connections 2024-09-19T06:50:59.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.314+0000 7f906e07c640 1 -- 172.21.15.103:0/1295526657 wait complete. 2024-09-19T06:50:59.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.314+0000 7f906e07c640 1 Processor -- start 2024-09-19T06:50:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.314+0000 7f906e07c640 1 -- start start 2024-09-19T06:50:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.314+0000 7f906e07c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9068100a80 0x7f90681992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.314+0000 7f906e07c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9068101420 0x7f90681997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.314+0000 7f906e07c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9068106a80 0x7f90681a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.314+0000 7f906e07c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9068110e40 con 0x7f9068101420 2024-09-19T06:50:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.314+0000 7f906e07c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9068110cc0 con 0x7f9068106a80 2024-09-19T06:50:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.314+0000 7f906e07c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9068110fc0 con 0x7f9068100a80 2024-09-19T06:50:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.314+0000 7f905ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9068101420 0x7f90681997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.314+0000 7f905ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9068101420 0x7f90681997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50400/0 (socket says 172.21.15.103:50400) 2024-09-19T06:50:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.314+0000 7f905ffff640 1 -- 172.21.15.103:0/2736519953 learned_addr learned my addr 172.21.15.103:0/2736519953 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.315+0000 7f9067fff640 1 --2- 172.21.15.103:0/2736519953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9068106a80 0x7f90681a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.315+0000 7f90677fe640 1 --2- 172.21.15.103:0/2736519953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9068100a80 0x7f90681992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.315+0000 7f905ffff640 1 -- 172.21.15.103:0/2736519953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9068100a80 msgr2=0x7f90681992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.315+0000 7f905ffff640 1 --2- 172.21.15.103:0/2736519953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9068100a80 0x7f90681992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.315+0000 7f905ffff640 1 -- 172.21.15.103:0/2736519953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9068106a80 msgr2=0x7f90681a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.315+0000 7f905ffff640 1 --2- 172.21.15.103:0/2736519953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9068106a80 0x7f90681a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.315+0000 7f905ffff640 1 -- 172.21.15.103:0/2736519953 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90681a4860 con 0x7f9068101420 2024-09-19T06:50:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.315+0000 7f905ffff640 1 --2- 172.21.15.103:0/2736519953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9068101420 0x7f90681997e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f905000c920 tx=0x7f905000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.315+0000 7f90657fa640 1 -- 172.21.15.103:0/2736519953 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050007c70 con 0x7f9068101420 2024-09-19T06:50:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.315+0000 7f90657fa640 1 -- 172.21.15.103:0/2736519953 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9050007e10 con 0x7f9068101420 2024-09-19T06:50:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.315+0000 7f906e07c640 1 -- 172.21.15.103:0/2736519953 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90681a4a30 con 0x7f9068101420 2024-09-19T06:50:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.315+0000 7f90657fa640 1 -- 172.21.15.103:0/2736519953 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050018890 con 0x7f9068101420 2024-09-19T06:50:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.315+0000 7f906e07c640 1 -- 172.21.15.103:0/2736519953 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90681a4f40 con 0x7f9068101420 2024-09-19T06:50:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.317+0000 7f90657fa640 1 -- 172.21.15.103:0/2736519953 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9050018a70 con 0x7f9068101420 2024-09-19T06:50:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9070455740 1 Processor -- start 2024-09-19T06:50:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9070455740 1 -- start start 2024-09-19T06:50:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9070455740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977060 0x556e22977440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9070455740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22977980 0x556e22981d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9070455740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22982240 0x556e22984630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9070455740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556e228dc620 con 0x556e22977060 2024-09-19T06:50:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9070455740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556e22883e50 con 0x556e22977980 2024-09-19T06:50:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9070455740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556e228d85e0 con 0x556e22982240 2024-09-19T06:50:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9067fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977060 0x556e22977440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f90657fa640 1 --2- 172.21.15.103:0/2736519953 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f90540a33b0 0x7f90540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9067fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977060 0x556e22977440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50412/0 (socket says 172.21.15.103:50412) 2024-09-19T06:50:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9067fff640 1 -- 172.21.15.103:0/359845171 learned_addr learned my addr 172.21.15.103:0/359845171 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9067fff640 1 --2- 172.21.15.103:0/2736519953 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f90540a33b0 0x7f90540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f90677fe640 1 --2- 172.21.15.103:0/359845171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22977980 0x556e22981d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f90657fa640 1 -- 172.21.15.103:0/2736519953 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f90500cfe60 con 0x7f9068101420 2024-09-19T06:50:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f905ffff640 1 --2- 172.21.15.103:0/359845171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22982240 0x556e22984630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9067fff640 1 -- 172.21.15.103:0/359845171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22982240 msgr2=0x556e22984630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:59.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9067fff640 1 --2- 172.21.15.103:0/359845171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22982240 0x556e22984630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9067fff640 1 -- 172.21.15.103:0/359845171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22977980 msgr2=0x556e22981d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:59.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9067fff640 1 --2- 172.21.15.103:0/359845171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22977980 0x556e22981d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.318+0000 7f9067fff640 1 -- 172.21.15.103:0/359845171 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e22874d40 con 0x556e22977060 2024-09-19T06:50:59.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.319+0000 7f90677fe640 1 --2- 172.21.15.103:0/359845171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22977980 0x556e22981d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:59.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.319+0000 7f905ffff640 1 --2- 172.21.15.103:0/359845171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22982240 0x556e22984630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:59.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.319+0000 7f9067fff640 1 --2- 172.21.15.103:0/359845171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977060 0x556e22977440 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9060014a60 tx=0x7f9060014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7efa08c2a95c08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:59.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.319+0000 7f905dffb640 1 -- 172.21.15.103:0/359845171 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906001ec50 con 0x556e22977060 2024-09-19T06:50:59.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.319+0000 7f905dffb640 1 -- 172.21.15.103:0/359845171 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9060023070 con 0x556e22977060 2024-09-19T06:50:59.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.319+0000 7f905dffb640 1 -- 172.21.15.103:0/359845171 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906000e360 con 0x556e22977060 2024-09-19T06:50:59.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.319+0000 7f9067fff640 1 --2- 172.21.15.103:0/2736519953 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f90540a33b0 0x7f90540a5870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9060022000 tx=0x7f906001e000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:59.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.320+0000 7f9070455740 1 -- 172.21.15.103:0/359845171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977060 msgr2=0x556e22977440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:59.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.320+0000 7f9070455740 1 --2- 172.21.15.103:0/359845171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977060 0x556e22977440 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9060014a60 tx=0x7f9060014f20 comp rx=0 tx=0).stop 2024-09-19T06:50:59.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.320+0000 7f9070455740 1 -- 172.21.15.103:0/359845171 shutdown_connections 2024-09-19T06:50:59.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.320+0000 7f9070455740 1 --2- 172.21.15.103:0/359845171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22982240 0x556e22984630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.320+0000 7f9070455740 1 --2- 172.21.15.103:0/359845171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22977980 0x556e22981d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.320+0000 7f9070455740 1 --2- 172.21.15.103:0/359845171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977060 0x556e22977440 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.320+0000 7f9070455740 1 -- 172.21.15.103:0/359845171 >> 172.21.15.103:0/359845171 conn(0x556e229735f0 msgr2=0x556e229739d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:59.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.320+0000 7f9070455740 1 -- 172.21.15.103:0/359845171 shutdown_connections 2024-09-19T06:50:59.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.320+0000 7f9070455740 1 -- 172.21.15.103:0/359845171 wait complete. 2024-09-19T06:50:59.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.320+0000 7f9070455740 1 -- start start 2024-09-19T06:50:59.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.320+0000 7f9070455740 1 Processor -- start 2024-09-19T06:50:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9070455740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22977060 0x556e2297f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9070455740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22977980 0x556e2297f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9070455740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22982240 0x556e2297faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9070455740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556e22984e10 con 0x556e22982240 2024-09-19T06:50:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9070455740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556e22864f10 con 0x556e22977980 2024-09-19T06:50:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9070455740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556e22984f90 con 0x556e22977060 2024-09-19T06:50:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9070455740 10 client.0 _ll_register_callbacks cb 0x556e22888db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9067fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22982240 0x556e2297faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9067fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22982240 0x556e2297faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50420/0 (socket says 172.21.15.103:50420) 2024-09-19T06:50:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9067fff640 1 -- 172.21.15.103:0/1279374598 learned_addr learned my addr 172.21.15.103:0/1279374598 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:59.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f90677fe640 1 --2- 172.21.15.103:0/1279374598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22977060 0x556e2297f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9067fff640 1 -- 172.21.15.103:0/1279374598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22977060 msgr2=0x556e2297f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:59.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9067fff640 1 --2- 172.21.15.103:0/1279374598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22977060 0x556e2297f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f905ffff640 1 --2- 172.21.15.103:0/1279374598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22977980 0x556e2297f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9067fff640 1 -- 172.21.15.103:0/1279374598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22977980 msgr2=0x556e2297f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:59.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9067fff640 1 --2- 172.21.15.103:0/1279374598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22977980 0x556e2297f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9067fff640 1 -- 172.21.15.103:0/1279374598 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e225eb580 con 0x556e22982240 2024-09-19T06:50:59.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f905ffff640 1 --2- 172.21.15.103:0/1279374598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22977980 0x556e2297f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:59.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9067fff640 1 --2- 172.21.15.103:0/1279374598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22982240 0x556e2297faa0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f906001dba0 tx=0x7f906001d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c1e6d02f2bedb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:59.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9046ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:59.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9046ffd640 1 -- 172.21.15.103:0/1279374598 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060036060 con 0x556e22982240 2024-09-19T06:50:59.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9046ffd640 1 -- 172.21.15.103:0/1279374598 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f906001dd00 con 0x556e22982240 2024-09-19T06:50:59.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9070455740 1 -- 172.21.15.103:0/1279374598 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9060023070 con 0x556e22982240 2024-09-19T06:50:59.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.321+0000 7f9046ffd640 1 -- 172.21.15.103:0/1279374598 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f906001ec50 con 0x556e22982240 2024-09-19T06:50:59.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.322+0000 7f9070455740 10 client.16158 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:59.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.322+0000 7f9070455740 1 -- 172.21.15.103:0/1279374598 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e226b5960 con 0x556e22982240 2024-09-19T06:50:59.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.322+0000 7f9046ffd640 1 -- 172.21.15.103:0/1279374598 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9060023070 con 0x556e22982240 2024-09-19T06:50:59.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.322+0000 7f9046ffd640 1 -- 172.21.15.103:0/1279374598 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f906000f040 con 0x556e22982240 2024-09-19T06:50:59.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.322+0000 7f9070455740 10 client.16158 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:59.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.322+0000 7f9070455740 10 client.16158 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:50:59.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.322+0000 7f9070455740 20 client.16158 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:59.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.322+0000 7f9070455740 1 --2- 172.21.15.103:0/1279374598 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556e22991bf0 0x556e22993fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.322+0000 7f9070455740 4 client.16158 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:59.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.322+0000 7f9070455740 1 -- 172.21.15.103:0/1279374598 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e228e9b90 con 0x556e22991bf0 2024-09-19T06:50:59.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.322+0000 7f90677fe640 1 --2- 172.21.15.103:0/1279374598 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556e22991bf0 0x556e22993fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.323+0000 7f90677fe640 1 --2- 172.21.15.103:0/1279374598 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556e22991bf0 0x556e22993fe0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa2319822435a149 server_cookie=dc4a22121ce0c34d in_seq=0 out_seq=0 2024-09-19T06:50:59.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.323+0000 7f9046ffd640 10 client.16158 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:59.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.323+0000 7f9046ffd640 1 -- 172.21.15.103:0/1279374598 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e228e9b90 con 0x556e22991bf0 2024-09-19T06:50:59.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.323+0000 7f9046ffd640 10 client.16158 handle_command_reply: tid=0 2024-09-19T06:50:59.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.324+0000 7f9070455740 1 client.16158 shutdown 2024-09-19T06:50:59.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.324+0000 7f9070455740 1 -- 172.21.15.103:0/1279374598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22982240 msgr2=0x556e2297faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:59.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.324+0000 7f9070455740 1 --2- 172.21.15.103:0/1279374598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22982240 0x556e2297faa0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f906001dba0 tx=0x7f906001d9d0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.324+0000 7f9070455740 1 -- 172.21.15.103:0/1279374598 shutdown_connections 2024-09-19T06:50:59.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.324+0000 7f9070455740 1 --2- 172.21.15.103:0/1279374598 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556e22991bf0 0x556e22993fe0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.324+0000 7f9070455740 1 --2- 172.21.15.103:0/1279374598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22982240 0x556e2297faa0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.324+0000 7f9046ffd640 0 client.16158 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:59.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.324+0000 7f9070455740 1 --2- 172.21.15.103:0/1279374598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22977980 0x556e2297f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.324+0000 7f9070455740 1 --2- 172.21.15.103:0/1279374598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22977060 0x556e2297f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.324+0000 7f9070455740 1 -- 172.21.15.103:0/1279374598 >> 172.21.15.103:0/1279374598 conn(0x556e229735f0 msgr2=0x556e22979320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:59.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.324+0000 7f9070455740 1 -- 172.21.15.103:0/1279374598 shutdown_connections 2024-09-19T06:50:59.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.324+0000 7f9070455740 1 -- 172.21.15.103:0/1279374598 wait complete. 2024-09-19T06:50:59.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.324+0000 7f9070455740 20 client.16158 trim_cache size 0 max 16384 2024-09-19T06:50:59.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.341+0000 7f9070455740 1 Processor -- start 2024-09-19T06:50:59.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.341+0000 7f9070455740 1 -- start start 2024-09-19T06:50:59.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f9070455740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22977060 0x556e229791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f9070455740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977980 0x556e22999610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f9070455740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22982240 0x556e22999b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f9070455740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90681ad880 con 0x556e22977980 2024-09-19T06:50:59.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f9070455740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90681af3c0 con 0x556e22982240 2024-09-19T06:50:59.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f9070455740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f90681adf50 con 0x556e22977060 2024-09-19T06:50:59.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f905ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977980 0x556e22999610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f905ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977980 0x556e22999610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50428/0 (socket says 172.21.15.103:50428) 2024-09-19T06:50:59.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f905ffff640 1 -- 172.21.15.103:0/3761224421 learned_addr learned my addr 172.21.15.103:0/3761224421 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:59.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f9067fff640 1 --2- 172.21.15.103:0/3761224421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22982240 0x556e22999b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f90677fe640 1 --2- 172.21.15.103:0/3761224421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22977060 0x556e229791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f905ffff640 1 -- 172.21.15.103:0/3761224421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22977060 msgr2=0x556e229791c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:59.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f905ffff640 1 --2- 172.21.15.103:0/3761224421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22977060 0x556e229791c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f905ffff640 1 -- 172.21.15.103:0/3761224421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22982240 msgr2=0x556e22999b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:59.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f905ffff640 1 --2- 172.21.15.103:0/3761224421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22982240 0x556e22999b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f905ffff640 1 -- 172.21.15.103:0/3761224421 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e229777b0 con 0x556e22977980 2024-09-19T06:50:59.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f90677fe640 1 --2- 172.21.15.103:0/3761224421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22977060 0x556e229791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:59.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f905ffff640 1 --2- 172.21.15.103:0/3761224421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977980 0x556e22999610 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f905009c0e0 tx=0x7f905009bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930f6c85e7005320 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:59.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f905d7fa640 1 -- 172.21.15.103:0/3761224421 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905009c150 con 0x556e22977980 2024-09-19T06:50:59.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f905d7fa640 1 -- 172.21.15.103:0/3761224421 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90500a0500 con 0x556e22977980 2024-09-19T06:50:59.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.342+0000 7f905d7fa640 1 -- 172.21.15.103:0/3761224421 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90500987c0 con 0x556e22977980 2024-09-19T06:50:59.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.343+0000 7f9070455740 1 -- 172.21.15.103:0/3761224421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977980 msgr2=0x556e22999610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:59.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.343+0000 7f9070455740 1 --2- 172.21.15.103:0/3761224421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977980 0x556e22999610 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f905009c0e0 tx=0x7f905009bab0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.343+0000 7f9070455740 1 -- 172.21.15.103:0/3761224421 shutdown_connections 2024-09-19T06:50:59.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.343+0000 7f9070455740 1 --2- 172.21.15.103:0/3761224421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22982240 0x556e22999b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.343+0000 7f9070455740 1 --2- 172.21.15.103:0/3761224421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977980 0x556e22999610 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.343+0000 7f9070455740 1 --2- 172.21.15.103:0/3761224421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22977060 0x556e229791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.343+0000 7f9070455740 1 -- 172.21.15.103:0/3761224421 >> 172.21.15.103:0/3761224421 conn(0x556e229735f0 msgr2=0x556e22978ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:59.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.343+0000 7f9070455740 1 -- 172.21.15.103:0/3761224421 shutdown_connections 2024-09-19T06:50:59.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.343+0000 7f9070455740 1 -- 172.21.15.103:0/3761224421 wait complete. 2024-09-19T06:50:59.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.343+0000 7f9070455740 1 -- start start 2024-09-19T06:50:59.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f9070455740 1 Processor -- start 2024-09-19T06:50:59.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f9070455740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977060 0x556e22986ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f9070455740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22977980 0x556e22987400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f9070455740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22982240 0x556e22987940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f9070455740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90681aecf0 con 0x556e22977060 2024-09-19T06:50:59.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f9070455740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90681ae620 con 0x556e22982240 2024-09-19T06:50:59.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f9070455740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f90681ab840 con 0x556e22977980 2024-09-19T06:50:59.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f9070455740 10 client.0 _ll_register_callbacks cb 0x7f90600128d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:59.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f90677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977060 0x556e22986ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f90677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977060 0x556e22986ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50432/0 (socket says 172.21.15.103:50432) 2024-09-19T06:50:59.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f90677fe640 1 -- 172.21.15.103:0/4126344903 learned_addr learned my addr 172.21.15.103:0/4126344903 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:50:59.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f9067fff640 1 --2- 172.21.15.103:0/4126344903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22982240 0x556e22987940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f905ffff640 1 --2- 172.21.15.103:0/4126344903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22977980 0x556e22987400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f90677fe640 1 -- 172.21.15.103:0/4126344903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22977980 msgr2=0x556e22987400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:59.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f90677fe640 1 --2- 172.21.15.103:0/4126344903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22977980 0x556e22987400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f90677fe640 1 -- 172.21.15.103:0/4126344903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22982240 msgr2=0x556e22987940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:59.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f90677fe640 1 --2- 172.21.15.103:0/4126344903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22982240 0x556e22987940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.344+0000 7f90677fe640 1 -- 172.21.15.103:0/4126344903 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e229fc5f0 con 0x556e22977060 2024-09-19T06:50:59.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.345+0000 7f90677fe640 1 --2- 172.21.15.103:0/4126344903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977060 0x556e22986ec0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9058005ab0 tx=0x7f90580041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758cd6b50c967024 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:59.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.345+0000 7f9046ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:50:59.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.345+0000 7f9046ffd640 1 -- 172.21.15.103:0/4126344903 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058035040 con 0x556e22977060 2024-09-19T06:50:59.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.345+0000 7f9046ffd640 1 -- 172.21.15.103:0/4126344903 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f9058005040 con 0x556e22977060 2024-09-19T06:50:59.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.345+0000 7f9070455740 1 -- 172.21.15.103:0/4126344903 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90500a0500 con 0x556e22977060 2024-09-19T06:50:59.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.345+0000 7f9046ffd640 1 -- 172.21.15.103:0/4126344903 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9058050050 con 0x556e22977060 2024-09-19T06:50:59.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.345+0000 7f9070455740 10 client.16164 fetch_fsmap learned FSMap version 27 2024-09-19T06:50:59.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.345+0000 7f9070455740 1 -- 172.21.15.103:0/4126344903 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e229fd870 con 0x556e22977060 2024-09-19T06:50:59.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.345+0000 7f9046ffd640 1 -- 172.21.15.103:0/4126344903 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90500a0500 con 0x556e22977060 2024-09-19T06:50:59.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.345+0000 7f9046ffd640 1 -- 172.21.15.103:0/4126344903 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f905804b030 con 0x556e22977060 2024-09-19T06:50:59.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.345+0000 7f9070455740 10 client.16164 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:50:59.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.345+0000 7f9070455740 10 client.16164 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:50:59.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.345+0000 7f9070455740 20 client.16164 populate_metadata read hostname 'smithi103' 2024-09-19T06:50:59.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.345+0000 7f9070455740 1 --2- 172.21.15.103:0/4126344903 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556e22991bf0 0x556e22a00060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:59.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.345+0000 7f9070455740 4 client.16164 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "51253b89", "release": true, "await": true}] 2024-09-19T06:50:59.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.345+0000 7f9070455740 1 -- 172.21.15.103:0/4126344903 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "set_id": "51253b89", "release": true, "await": true}) -- 0x556e228e40a0 con 0x556e22991bf0 2024-09-19T06:50:59.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.346+0000 7f905ffff640 1 --2- 172.21.15.103:0/4126344903 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556e22991bf0 0x556e22a00060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:59.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.346+0000 7f9046ffd640 10 client.16164 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:59.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.346+0000 7f905ffff640 1 --2- 172.21.15.103:0/4126344903 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556e22991bf0 0x556e22a00060 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a687a9598ef94b2 server_cookie=1efee4f364d72a55 in_seq=0 out_seq=0 2024-09-19T06:50:59.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.349+0000 7f9046ffd640 1 -- 172.21.15.103:0/4126344903 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x556e228e40a0 con 0x556e22991bf0 2024-09-19T06:50:59.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.349+0000 7f9046ffd640 10 client.16164 handle_command_reply: tid=0 2024-09-19T06:50:59.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.349+0000 7f9070455740 1 client.16164 shutdown 2024-09-19T06:50:59.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.349+0000 7f9070455740 1 -- 172.21.15.103:0/4126344903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977060 msgr2=0x556e22986ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:59.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.349+0000 7f9070455740 1 --2- 172.21.15.103:0/4126344903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977060 0x556e22986ec0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9058005ab0 tx=0x7f90580041d0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.349+0000 7f9070455740 1 -- 172.21.15.103:0/4126344903 shutdown_connections 2024-09-19T06:50:59.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.349+0000 7f9070455740 1 --2- 172.21.15.103:0/4126344903 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556e22991bf0 0x556e22a00060 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.349+0000 7f9070455740 1 --2- 172.21.15.103:0/4126344903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e22982240 0x556e22987940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.349+0000 7f9070455740 1 --2- 172.21.15.103:0/4126344903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e22977980 0x556e22987400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.349+0000 7f9070455740 1 --2- 172.21.15.103:0/4126344903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e22977060 0x556e22986ec0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.349+0000 7f9070455740 1 -- 172.21.15.103:0/4126344903 >> 172.21.15.103:0/4126344903 conn(0x556e229735f0 msgr2=0x556e22972750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:59.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.349+0000 7f9046ffd640 0 client.16164 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:50:59.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.349+0000 7f9070455740 1 -- 172.21.15.103:0/4126344903 shutdown_connections 2024-09-19T06:50:59.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.349+0000 7f9070455740 1 -- 172.21.15.103:0/4126344903 wait complete. 2024-09-19T06:50:59.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.349+0000 7f9070455740 20 client.16164 trim_cache size 0 max 16384 2024-09-19T06:50:59.361 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:50:59.361 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:50:59.361 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:50:59.361 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 8, 2024-09-19T06:50:59.361 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:50:59.361 INFO:teuthology.orchestra.run.smithi103.stdout: "51253b89": { 2024-09-19T06:50:59.362 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 8, 2024-09-19T06:50:59.362 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:50:59.362 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:50:59.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:50:59.362 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:50:59.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:50:59.362 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:50:59.362 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 140.3, 2024-09-19T06:50:59.362 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:50:59.362 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:50:59.362 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:50:59.363 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:50:59.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:50:59.363 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:50:59.363 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:59.363 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:59.363 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:59.363 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:59.363 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:50:59.363 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:50:59.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.350+0000 7f906e07c640 1 -- 172.21.15.103:0/2736519953 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f90540a33b0 msgr2=0x7f90540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:59.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.350+0000 7f906e07c640 1 --2- 172.21.15.103:0/2736519953 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f90540a33b0 0x7f90540a5870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9060022000 tx=0x7f906001e000 comp rx=0 tx=0).stop 2024-09-19T06:50:59.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.350+0000 7f906e07c640 1 -- 172.21.15.103:0/2736519953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9068101420 msgr2=0x7f90681997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:59.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.350+0000 7f906e07c640 1 --2- 172.21.15.103:0/2736519953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9068101420 0x7f90681997e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f905000c920 tx=0x7f905000cde0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.350+0000 7f906e07c640 1 -- 172.21.15.103:0/2736519953 shutdown_connections 2024-09-19T06:50:59.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.350+0000 7f906e07c640 1 --2- 172.21.15.103:0/2736519953 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f90540a33b0 0x7f90540a5870 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.350+0000 7f906e07c640 1 --2- 172.21.15.103:0/2736519953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9068106a80 0x7f90681a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.350+0000 7f906e07c640 1 --2- 172.21.15.103:0/2736519953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9068101420 0x7f90681997e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.350+0000 7f906e07c640 1 --2- 172.21.15.103:0/2736519953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9068100a80 0x7f90681992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:59.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.350+0000 7f906e07c640 1 -- 172.21.15.103:0/2736519953 >> 172.21.15.103:0/2736519953 conn(0x7f90680fc7a0 msgr2=0x7f90680fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:59.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.350+0000 7f906e07c640 1 -- 172.21.15.103:0/2736519953 shutdown_connections 2024-09-19T06:50:59.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:50:59.350+0000 7f906e07c640 1 -- 172.21.15.103:0/2736519953 wait complete. 2024-09-19T06:51:00.132 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '51253b89', total seconds elapsed: 23.5 2024-09-19T06:51:00.132 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T06:51:00.132 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:51:00.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:00 smithi174 ceph-mon[35637]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 15 op/s 2024-09-19T06:51:00.382 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:00.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:00 smithi103 ceph-mon[31158]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 15 op/s 2024-09-19T06:51:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:00 smithi134 ceph-mon[35356]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 15 op/s 2024-09-19T06:51:00.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.823+0000 7f279705a640 1 -- 172.21.15.103:0/1614420892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27901027a0 msgr2=0x7f27901067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:00.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.823+0000 7f279705a640 1 --2- 172.21.15.103:0/1614420892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27901027a0 0x7f27901067f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2784009ca0 tx=0x7f278402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.823+0000 7f279705a640 1 -- 172.21.15.103:0/1614420892 shutdown_connections 2024-09-19T06:51:00.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.823+0000 7f279705a640 1 --2- 172.21.15.103:0/1614420892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2790106f20 0x7f279010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.823+0000 7f279705a640 1 --2- 172.21.15.103:0/1614420892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27901027a0 0x7f27901067f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.823+0000 7f279705a640 1 --2- 172.21.15.103:0/1614420892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2790101df0 0x7f27901021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.823+0000 7f279705a640 1 -- 172.21.15.103:0/1614420892 >> 172.21.15.103:0/1614420892 conn(0x7f27900fd8b0 msgr2=0x7f27900ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:00.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.823+0000 7f279705a640 1 -- 172.21.15.103:0/1614420892 shutdown_connections 2024-09-19T06:51:00.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.823+0000 7f279705a640 1 -- 172.21.15.103:0/1614420892 wait complete. 2024-09-19T06:51:00.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.824+0000 7f279705a640 1 Processor -- start 2024-09-19T06:51:00.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.824+0000 7f279705a640 1 -- start start 2024-09-19T06:51:00.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.824+0000 7f279705a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2790101df0 0x7f27900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.824+0000 7f279705a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27901027a0 0x7f2790074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.824+0000 7f279705a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2790106f20 0x7f2790074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.824+0000 7f279705a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2790111fa0 con 0x7f2790106f20 2024-09-19T06:51:00.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.824+0000 7f279705a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2790111e20 con 0x7f27901027a0 2024-09-19T06:51:00.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.824+0000 7f279705a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2790112120 con 0x7f2790101df0 2024-09-19T06:51:00.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.824+0000 7f27955f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2790106f20 0x7f2790074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.824+0000 7f27955f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2790106f20 0x7f2790074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50446/0 (socket says 172.21.15.103:50446) 2024-09-19T06:51:00.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.824+0000 7f27955f7640 1 -- 172.21.15.103:0/2128665194 learned_addr learned my addr 172.21.15.103:0/2128665194 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:00.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.824+0000 7f278ffff640 1 --2- 172.21.15.103:0/2128665194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27901027a0 0x7f2790074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.824+0000 7f2794df6640 1 --2- 172.21.15.103:0/2128665194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2790101df0 0x7f27900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.825+0000 7f27955f7640 1 -- 172.21.15.103:0/2128665194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2790101df0 msgr2=0x7f27900744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:00.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.825+0000 7f27955f7640 1 --2- 172.21.15.103:0/2128665194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2790101df0 0x7f27900744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.825+0000 7f27955f7640 1 -- 172.21.15.103:0/2128665194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27901027a0 msgr2=0x7f2790074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:00.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.825+0000 7f27955f7640 1 --2- 172.21.15.103:0/2128665194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27901027a0 0x7f2790074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.825+0000 7f27955f7640 1 -- 172.21.15.103:0/2128665194 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27901a6910 con 0x7f2790106f20 2024-09-19T06:51:00.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.825+0000 7f2794df6640 1 --2- 172.21.15.103:0/2128665194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2790101df0 0x7f27900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:00.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.825+0000 7f27955f7640 1 --2- 172.21.15.103:0/2128665194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2790106f20 0x7f2790074f70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2788009fd0 tx=0x7f278800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:00.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.825+0000 7f278dffb640 1 -- 172.21.15.103:0/2128665194 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788011830 con 0x7f2790106f20 2024-09-19T06:51:00.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.825+0000 7f279705a640 1 -- 172.21.15.103:0/2128665194 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27901a6c00 con 0x7f2790106f20 2024-09-19T06:51:00.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.825+0000 7f278dffb640 1 -- 172.21.15.103:0/2128665194 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2788016070 con 0x7f2790106f20 2024-09-19T06:51:00.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.825+0000 7f278dffb640 1 -- 172.21.15.103:0/2128665194 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278800f2e0 con 0x7f2790106f20 2024-09-19T06:51:00.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.825+0000 7f279705a640 1 -- 172.21.15.103:0/2128665194 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27901a7110 con 0x7f2790106f20 2024-09-19T06:51:00.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.826+0000 7f279705a640 1 -- 172.21.15.103:0/2128665194 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27901021d0 con 0x7f2790106f20 2024-09-19T06:51:00.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.827+0000 7f278dffb640 1 -- 172.21.15.103:0/2128665194 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f27880119d0 con 0x7f2790106f20 2024-09-19T06:51:00.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.828+0000 7f278dffb640 1 --2- 172.21.15.103:0/2128665194 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f27780a33b0 0x7f27780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.828+0000 7f278dffb640 1 -- 172.21.15.103:0/2128665194 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f27880cf270 con 0x7f2790106f20 2024-09-19T06:51:00.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.828+0000 7f2794df6640 1 --2- 172.21.15.103:0/2128665194 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f27780a33b0 0x7f27780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.829+0000 7f2794df6640 1 --2- 172.21.15.103:0/2128665194 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f27780a33b0 0x7f27780a5870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f27800058c0 tx=0x7f2780005ec0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:00.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:00.832+0000 7f278dffb640 1 -- 172.21.15.103:0/2128665194 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f278809bd70 con 0x7f2790106f20 2024-09-19T06:51:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:01.037+0000 7f279705a640 1 -- 172.21.15.103:0/2128665194 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f279010a100 con 0x7f2790106f20 2024-09-19T06:51:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:01.037+0000 7f278dffb640 1 -- 172.21.15.103:0/2128665194 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f278809b670 con 0x7f2790106f20 2024-09-19T06:51:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:51:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:51:01.039 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:51:01.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:01.040+0000 7f279705a640 1 -- 172.21.15.103:0/2128665194 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f27780a33b0 msgr2=0x7f27780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:01.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:01.040+0000 7f279705a640 1 --2- 172.21.15.103:0/2128665194 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f27780a33b0 0x7f27780a5870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f27800058c0 tx=0x7f2780005ec0 comp rx=0 tx=0).stop 2024-09-19T06:51:01.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:01.040+0000 7f279705a640 1 -- 172.21.15.103:0/2128665194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2790106f20 msgr2=0x7f2790074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:01.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:01.040+0000 7f279705a640 1 --2- 172.21.15.103:0/2128665194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2790106f20 0x7f2790074f70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2788009fd0 tx=0x7f278800b8e0 comp rx=0 tx=0).stop 2024-09-19T06:51:01.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:01.040+0000 7f279705a640 1 -- 172.21.15.103:0/2128665194 shutdown_connections 2024-09-19T06:51:01.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:01.040+0000 7f279705a640 1 --2- 172.21.15.103:0/2128665194 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f27780a33b0 0x7f27780a5870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:01.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:01.040+0000 7f279705a640 1 --2- 172.21.15.103:0/2128665194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2790106f20 0x7f2790074f70 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:01.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:01.040+0000 7f279705a640 1 --2- 172.21.15.103:0/2128665194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27901027a0 0x7f2790074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:01.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:01.040+0000 7f279705a640 1 --2- 172.21.15.103:0/2128665194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2790101df0 0x7f27900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:01.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:01.040+0000 7f279705a640 1 -- 172.21.15.103:0/2128665194 >> 172.21.15.103:0/2128665194 conn(0x7f27900fd8b0 msgr2=0x7f27900fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:01.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:01.040+0000 7f279705a640 1 -- 172.21.15.103:0/2128665194 shutdown_connections 2024-09-19T06:51:01.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:01.040+0000 7f279705a640 1 -- 172.21.15.103:0/2128665194 wait complete. 2024-09-19T06:51:01.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:01 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2128665194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:51:01.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:01 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2128665194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:51:01.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:01 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2128665194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:51:01.638 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T06:51:01.638 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T06:51:01.890 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:02.298 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:02 smithi103 ceph-mon[31158]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-19T06:51:02.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:02 smithi134 ceph-mon[35356]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-19T06:51:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.502+0000 7f71e628b640 1 -- 172.21.15.103:0/2455006508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e0100e20 msgr2=0x7f71e0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.502+0000 7f71e628b640 1 --2- 172.21.15.103:0/2455006508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e0100e20 0x7f71e0101200 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009900 tx=0x7f71d002a480 comp rx=0 tx=0).stop 2024-09-19T06:51:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.502+0000 7f71e628b640 1 -- 172.21.15.103:0/2455006508 shutdown_connections 2024-09-19T06:51:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.502+0000 7f71e628b640 1 --2- 172.21.15.103:0/2455006508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71e0102ef0 0x7f71e010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.502+0000 7f71e628b640 1 --2- 172.21.15.103:0/2455006508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71e0102570 0x7f71e01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.502+0000 7f71e628b640 1 --2- 172.21.15.103:0/2455006508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e0100e20 0x7f71e0101200 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.502+0000 7f71e628b640 1 -- 172.21.15.103:0/2455006508 >> 172.21.15.103:0/2455006508 conn(0x7f71e00fc7d0 msgr2=0x7f71e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.502+0000 7f71e628b640 1 -- 172.21.15.103:0/2455006508 shutdown_connections 2024-09-19T06:51:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.502+0000 7f71e628b640 1 -- 172.21.15.103:0/2455006508 wait complete. 2024-09-19T06:51:02.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.502+0000 7f71e628b640 1 Processor -- start 2024-09-19T06:51:02.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.503+0000 7f71e628b640 1 -- start start 2024-09-19T06:51:02.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.503+0000 7f71e628b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e0100e20 0x7f71e01993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.503+0000 7f71e628b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71e0102570 0x7f71e0199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.503+0000 7f71e628b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71e0102ef0 0x7f71e01a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.503+0000 7f71e628b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71e0110f80 con 0x7f71e0100e20 2024-09-19T06:51:02.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.503+0000 7f71e628b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71e0110e00 con 0x7f71e0102570 2024-09-19T06:51:02.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.503+0000 7f71e628b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f71e0111100 con 0x7f71e0102ef0 2024-09-19T06:51:02.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.503+0000 7f71dffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e0100e20 0x7f71e01993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.503+0000 7f71dffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e0100e20 0x7f71e01993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50452/0 (socket says 172.21.15.103:50452) 2024-09-19T06:51:02.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.503+0000 7f71dffff640 1 -- 172.21.15.103:0/2483747119 learned_addr learned my addr 172.21.15.103:0/2483747119 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:02.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.503+0000 7f71e4828640 1 --2- 172.21.15.103:0/2483747119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71e0102ef0 0x7f71e01a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.503+0000 7f71dffff640 1 -- 172.21.15.103:0/2483747119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71e0102ef0 msgr2=0x7f71e01a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:02.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.503+0000 7f71df7fe640 1 --2- 172.21.15.103:0/2483747119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71e0102570 0x7f71e0199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.503+0000 7f71dffff640 1 --2- 172.21.15.103:0/2483747119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71e0102ef0 0x7f71e01a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.503+0000 7f71dffff640 1 -- 172.21.15.103:0/2483747119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71e0102570 msgr2=0x7f71e0199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:02.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.503+0000 7f71dffff640 1 --2- 172.21.15.103:0/2483747119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71e0102570 0x7f71e0199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.504+0000 7f71dffff640 1 -- 172.21.15.103:0/2483747119 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e01a4990 con 0x7f71e0100e20 2024-09-19T06:51:02.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.504+0000 7f71e4828640 1 --2- 172.21.15.103:0/2483747119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71e0102ef0 0x7f71e01a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:02.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.504+0000 7f71df7fe640 1 --2- 172.21.15.103:0/2483747119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71e0102570 0x7f71e0199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:02.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.504+0000 7f71dffff640 1 --2- 172.21.15.103:0/2483747119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e0100e20 0x7f71e01993d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f71d0006010 tx=0x7f71d0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:02.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.504+0000 7f71dd7fa640 1 -- 172.21.15.103:0/2483747119 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d003f070 con 0x7f71e0100e20 2024-09-19T06:51:02.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.504+0000 7f71dd7fa640 1 -- 172.21.15.103:0/2483747119 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d0031c30 con 0x7f71e0100e20 2024-09-19T06:51:02.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.504+0000 7f71e628b640 1 -- 172.21.15.103:0/2483747119 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e01a4b60 con 0x7f71e0100e20 2024-09-19T06:51:02.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.504+0000 7f71dd7fa640 1 -- 172.21.15.103:0/2483747119 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0035030 con 0x7f71e0100e20 2024-09-19T06:51:02.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.504+0000 7f71e628b640 1 -- 172.21.15.103:0/2483747119 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e01a5010 con 0x7f71e0100e20 2024-09-19T06:51:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.506+0000 7f71dd7fa640 1 -- 172.21.15.103:0/2483747119 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f71d0042070 con 0x7f71e0100e20 2024-09-19T06:51:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71e8664740 1 Processor -- start 2024-09-19T06:51:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71e8664740 1 -- start start 2024-09-19T06:51:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71e8664740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b30730 0x5585c7b30b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71e8664740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 0x5585c7b3f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71e8664740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7b42230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71e8664740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5585c7a95160 con 0x5585c7b3fe40 2024-09-19T06:51:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71e8664740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5585c7a3c720 con 0x5585c7b30730 2024-09-19T06:51:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71e8664740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5585c7a91120 con 0x5585c7b31050 2024-09-19T06:51:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71dd7fa640 1 --2- 172.21.15.103:0/2483747119 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f71c80a3430 0x7f71c80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71df7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7b42230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71df7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7b42230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50464/0 (socket says 172.21.15.103:50464) 2024-09-19T06:51:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71df7fe640 1 -- 172.21.15.103:0/2894139401 learned_addr learned my addr 172.21.15.103:0/2894139401 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71e4828640 1 --2- 172.21.15.103:0/2894139401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b30730 0x5585c7b30b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71e4828640 1 --2- 172.21.15.103:0/2483747119 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f71c80a3430 0x7f71c80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71dffff640 1 --2- 172.21.15.103:0/2894139401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 0x5585c7b3f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71dd7fa640 1 -- 172.21.15.103:0/2483747119 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f71d00f05b0 con 0x7f71e0100e20 2024-09-19T06:51:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71df7fe640 1 -- 172.21.15.103:0/2894139401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 msgr2=0x5585c7b3f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71df7fe640 1 --2- 172.21.15.103:0/2894139401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 0x5585c7b3f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71df7fe640 1 -- 172.21.15.103:0/2894139401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b30730 msgr2=0x5585c7b30b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71df7fe640 1 --2- 172.21.15.103:0/2894139401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b30730 0x5585c7b30b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.507+0000 7f71df7fe640 1 -- 172.21.15.103:0/2894139401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585c7806ba0 con 0x5585c7b3fe40 2024-09-19T06:51:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.508+0000 7f71dffff640 1 --2- 172.21.15.103:0/2894139401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 0x5585c7b3f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.508+0000 7f71e4828640 1 --2- 172.21.15.103:0/2483747119 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f71c80a3430 0x7f71c80a58f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005c40 tx=0x7f71d8009280 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.508+0000 7f71df7fe640 1 --2- 172.21.15.103:0/2894139401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7b42230 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009c20 tx=0x7f71d400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71927a2fc8df9e9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.508+0000 7f71c67fc640 1 -- 172.21.15.103:0/2894139401 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4016070 con 0x5585c7b3fe40 2024-09-19T06:51:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.508+0000 7f71c67fc640 1 -- 172.21.15.103:0/2894139401 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d4011950 con 0x5585c7b3fe40 2024-09-19T06:51:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.508+0000 7f71c67fc640 1 -- 172.21.15.103:0/2894139401 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4010870 con 0x5585c7b3fe40 2024-09-19T06:51:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.509+0000 7f71e8664740 1 -- 172.21.15.103:0/2894139401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 msgr2=0x5585c7b42230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.509+0000 7f71e8664740 1 --2- 172.21.15.103:0/2894139401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7b42230 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009c20 tx=0x7f71d400e6c0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.509+0000 7f71e8664740 1 -- 172.21.15.103:0/2894139401 shutdown_connections 2024-09-19T06:51:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.509+0000 7f71e8664740 1 --2- 172.21.15.103:0/2894139401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7b42230 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.509+0000 7f71e8664740 1 --2- 172.21.15.103:0/2894139401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 0x5585c7b3f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.509+0000 7f71e8664740 1 --2- 172.21.15.103:0/2894139401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b30730 0x5585c7b30b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.509+0000 7f71e8664740 1 -- 172.21.15.103:0/2894139401 >> 172.21.15.103:0/2894139401 conn(0x5585c7b2cde0 msgr2=0x5585c7b2d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.509+0000 7f71e8664740 1 -- 172.21.15.103:0/2894139401 shutdown_connections 2024-09-19T06:51:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.509+0000 7f71e8664740 1 -- 172.21.15.103:0/2894139401 wait complete. 2024-09-19T06:51:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.509+0000 7f71e8664740 1 -- start start 2024-09-19T06:51:02.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.509+0000 7f71e8664740 1 Processor -- start 2024-09-19T06:51:02.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71e8664740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b30730 0x5585c7b3cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71e8664740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b31050 0x5585c7b3d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71e8664740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b3fe40 0x5585c7b3d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71e8664740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5585c7b42a10 con 0x5585c7b31050 2024-09-19T06:51:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71e8664740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5585c7a1d900 con 0x5585c7b3fe40 2024-09-19T06:51:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71e8664740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5585c7b42b90 con 0x5585c7b30730 2024-09-19T06:51:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71e8664740 10 client.0 _ll_register_callbacks cb 0x5585c7a1b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71df7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b31050 0x5585c7b3d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71df7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b31050 0x5585c7b3d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50468/0 (socket says 172.21.15.103:50468) 2024-09-19T06:51:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71df7fe640 1 -- 172.21.15.103:0/2798449307 learned_addr learned my addr 172.21.15.103:0/2798449307 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71dffff640 1 --2- 172.21.15.103:0/2798449307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b30730 0x5585c7b3cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71e4828640 1 --2- 172.21.15.103:0/2798449307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b3fe40 0x5585c7b3d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71df7fe640 1 -- 172.21.15.103:0/2798449307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b30730 msgr2=0x5585c7b3cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71df7fe640 1 --2- 172.21.15.103:0/2798449307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b30730 0x5585c7b3cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71df7fe640 1 -- 172.21.15.103:0/2798449307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b3fe40 msgr2=0x5585c7b3d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71df7fe640 1 --2- 172.21.15.103:0/2798449307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b3fe40 0x5585c7b3d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71df7fe640 1 -- 172.21.15.103:0/2798449307 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585c7a5dc50 con 0x5585c7b31050 2024-09-19T06:51:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71e4828640 1 --2- 172.21.15.103:0/2798449307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b3fe40 0x5585c7b3d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71df7fe640 1 --2- 172.21.15.103:0/2798449307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b31050 0x5585c7b3d220 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f71d40104b0 tx=0x7f71d40118e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e7279a9c194332 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71bf7fe640 1 -- 172.21.15.103:0/2798449307 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4027020 con 0x5585c7b31050 2024-09-19T06:51:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71e8664740 1 -- 172.21.15.103:0/2798449307 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d4011950 con 0x5585c7b31050 2024-09-19T06:51:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.510+0000 7f71bf7fe640 1 -- 172.21.15.103:0/2798449307 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f71d4004020 con 0x5585c7b31050 2024-09-19T06:51:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.511+0000 7f71bf7fe640 1 -- 172.21.15.103:0/2798449307 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f71d4016050 con 0x5585c7b31050 2024-09-19T06:51:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.511+0000 7f71e8664740 10 client.16182 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.511+0000 7f71e8664740 1 -- 172.21.15.103:0/2798449307 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5585c7b4e740 con 0x5585c7b31050 2024-09-19T06:51:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.511+0000 7f71bf7fe640 1 -- 172.21.15.103:0/2798449307 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d4011950 con 0x5585c7b31050 2024-09-19T06:51:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.511+0000 7f71bf7fe640 1 -- 172.21.15.103:0/2798449307 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f71d400f040 con 0x5585c7b31050 2024-09-19T06:51:02.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.511+0000 7f71e8664740 10 client.16182 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:02.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.511+0000 7f71e8664740 10 client.16182 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:51:02.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.511+0000 7f71e8664740 20 client.16182 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:02.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.511+0000 7f71e8664740 1 --2- 172.21.15.103:0/2798449307 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5585c7b4fb50 0x5585c7b51f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.511+0000 7f71e8664740 4 client.16182 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:02.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.511+0000 7f71e8664740 1 -- 172.21.15.103:0/2798449307 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585c7aa2890 con 0x5585c7b4fb50 2024-09-19T06:51:02.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.511+0000 7f71dffff640 1 --2- 172.21.15.103:0/2798449307 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5585c7b4fb50 0x5585c7b51f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.512+0000 7f71bf7fe640 10 client.16182 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:02.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.512+0000 7f71dffff640 1 --2- 172.21.15.103:0/2798449307 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5585c7b4fb50 0x5585c7b51f40 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e260edca89b71661 server_cookie=49bba8cf57a20b10 in_seq=0 out_seq=0 2024-09-19T06:51:02.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.512+0000 7f71bf7fe640 1 -- 172.21.15.103:0/2798449307 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585c7aa2890 con 0x5585c7b4fb50 2024-09-19T06:51:02.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.512+0000 7f71bf7fe640 10 client.16182 handle_command_reply: tid=0 2024-09-19T06:51:02.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.512+0000 7f71e8664740 1 client.16182 shutdown 2024-09-19T06:51:02.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.513+0000 7f71e8664740 1 -- 172.21.15.103:0/2798449307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b31050 msgr2=0x5585c7b3d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:02.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.513+0000 7f71e8664740 1 --2- 172.21.15.103:0/2798449307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b31050 0x5585c7b3d220 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f71d40104b0 tx=0x7f71d40118e0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.513+0000 7f71e8664740 1 -- 172.21.15.103:0/2798449307 shutdown_connections 2024-09-19T06:51:02.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.513+0000 7f71e8664740 1 --2- 172.21.15.103:0/2798449307 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5585c7b4fb50 0x5585c7b51f40 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.513+0000 7f71e8664740 1 --2- 172.21.15.103:0/2798449307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b3fe40 0x5585c7b3d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.513+0000 7f71e8664740 1 --2- 172.21.15.103:0/2798449307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b31050 0x5585c7b3d220 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.513+0000 7f71e8664740 1 --2- 172.21.15.103:0/2798449307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b30730 0x5585c7b3cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.513+0000 7f71e8664740 1 -- 172.21.15.103:0/2798449307 >> 172.21.15.103:0/2798449307 conn(0x5585c7b2cde0 msgr2=0x5585c7b32b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:02.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.513+0000 7f71bf7fe640 0 client.16182 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:02.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.513+0000 7f71e8664740 1 -- 172.21.15.103:0/2798449307 shutdown_connections 2024-09-19T06:51:02.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.513+0000 7f71e8664740 1 -- 172.21.15.103:0/2798449307 wait complete. 2024-09-19T06:51:02.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.513+0000 7f71e8664740 20 client.16182 trim_cache size 0 max 16384 2024-09-19T06:51:02.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.529+0000 7f71e8664740 1 Processor -- start 2024-09-19T06:51:02.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.529+0000 7f71e8664740 1 -- start start 2024-09-19T06:51:02.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71e8664740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b30730 0x5585c7b57570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71e8664740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 0x5585c7b57ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71e8664740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7b39f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71e8664740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01ab650 con 0x5585c7b3fe40 2024-09-19T06:51:02.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71e8664740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01ad190 con 0x5585c7b30730 2024-09-19T06:51:02.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71e8664740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01abd20 con 0x5585c7b31050 2024-09-19T06:51:02.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71e4828640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7b39f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71e4828640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7b39f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50476/0 (socket says 172.21.15.103:50476) 2024-09-19T06:51:02.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71e4828640 1 -- 172.21.15.103:0/3663927589 learned_addr learned my addr 172.21.15.103:0/3663927589 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:02.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71dffff640 1 --2- 172.21.15.103:0/3663927589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b30730 0x5585c7b57570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71df7fe640 1 --2- 172.21.15.103:0/3663927589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 0x5585c7b57ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71e4828640 1 -- 172.21.15.103:0/3663927589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 msgr2=0x5585c7b57ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71e4828640 1 --2- 172.21.15.103:0/3663927589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 0x5585c7b57ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71e4828640 1 -- 172.21.15.103:0/3663927589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b30730 msgr2=0x5585c7b57570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71e4828640 1 --2- 172.21.15.103:0/3663927589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b30730 0x5585c7b57570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71e4828640 1 -- 172.21.15.103:0/3663927589 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585c7b3bc90 con 0x5585c7b3fe40 2024-09-19T06:51:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71df7fe640 1 --2- 172.21.15.103:0/3663927589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 0x5585c7b57ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71e4828640 1 --2- 172.21.15.103:0/3663927589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7b39f40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f71d8012520 tx=0x7f71d8009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485fbe5f6a1936f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71c5ffb640 1 -- 172.21.15.103:0/3663927589 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8012d00 con 0x5585c7b3fe40 2024-09-19T06:51:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71c5ffb640 1 -- 172.21.15.103:0/3663927589 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d8021450 con 0x5585c7b3fe40 2024-09-19T06:51:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.530+0000 7f71c5ffb640 1 -- 172.21.15.103:0/3663927589 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8012ea0 con 0x5585c7b3fe40 2024-09-19T06:51:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.531+0000 7f71e8664740 1 -- 172.21.15.103:0/3663927589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 msgr2=0x5585c7b39f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.531+0000 7f71e8664740 1 --2- 172.21.15.103:0/3663927589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7b39f40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f71d8012520 tx=0x7f71d8009700 comp rx=0 tx=0).stop 2024-09-19T06:51:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.531+0000 7f71e8664740 1 -- 172.21.15.103:0/3663927589 shutdown_connections 2024-09-19T06:51:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.531+0000 7f71e8664740 1 --2- 172.21.15.103:0/3663927589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7b39f40 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.531+0000 7f71e8664740 1 --2- 172.21.15.103:0/3663927589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 0x5585c7b57ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.531+0000 7f71e8664740 1 --2- 172.21.15.103:0/3663927589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b30730 0x5585c7b57570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.531+0000 7f71e8664740 1 -- 172.21.15.103:0/3663927589 >> 172.21.15.103:0/3663927589 conn(0x5585c7b2cde0 msgr2=0x5585c7b2be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.531+0000 7f71e8664740 1 -- 172.21.15.103:0/3663927589 shutdown_connections 2024-09-19T06:51:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.531+0000 7f71e8664740 1 -- 172.21.15.103:0/3663927589 wait complete. 2024-09-19T06:51:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.532+0000 7f71e8664740 1 -- start start 2024-09-19T06:51:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.532+0000 7f71e8664740 1 Processor -- start 2024-09-19T06:51:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.532+0000 7f71e8664740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b30730 0x5585c7b43ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.532+0000 7f71e8664740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 0x5585c7b44430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.532+0000 7f71e8664740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7bb9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.532+0000 7f71e8664740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01acac0 con 0x5585c7b3fe40 2024-09-19T06:51:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.532+0000 7f71e8664740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01ac3f0 con 0x5585c7b30730 2024-09-19T06:51:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.532+0000 7f71e8664740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01a9630 con 0x5585c7b31050 2024-09-19T06:51:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.532+0000 7f71e8664740 10 client.0 _ll_register_callbacks cb 0x7f71d8009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.532+0000 7f71e4828640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7bb9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.532+0000 7f71e4828640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7bb9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50490/0 (socket says 172.21.15.103:50490) 2024-09-19T06:51:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.532+0000 7f71e4828640 1 -- 172.21.15.103:0/2258157338 learned_addr learned my addr 172.21.15.103:0/2258157338 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.532+0000 7f71dffff640 1 --2- 172.21.15.103:0/2258157338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b30730 0x5585c7b43ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.533+0000 7f71df7fe640 1 --2- 172.21.15.103:0/2258157338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 0x5585c7b44430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.533+0000 7f71e4828640 1 -- 172.21.15.103:0/2258157338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 msgr2=0x5585c7b44430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.533+0000 7f71e4828640 1 --2- 172.21.15.103:0/2258157338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 0x5585c7b44430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.533+0000 7f71e4828640 1 -- 172.21.15.103:0/2258157338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b30730 msgr2=0x5585c7b43ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.533+0000 7f71e4828640 1 --2- 172.21.15.103:0/2258157338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b30730 0x5585c7b43ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.533+0000 7f71e4828640 1 -- 172.21.15.103:0/2258157338 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585c7b44d10 con 0x5585c7b3fe40 2024-09-19T06:51:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.533+0000 7f71df7fe640 1 --2- 172.21.15.103:0/2258157338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 0x5585c7b44430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.533+0000 7f71e4828640 1 --2- 172.21.15.103:0/2258157338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7bb9540 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f71d8021e70 tx=0x7f71d8021d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30d94fc552d5d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.533+0000 7f71bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.533+0000 7f71bf7fe640 1 -- 172.21.15.103:0/2258157338 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d801b040 con 0x5585c7b3fe40 2024-09-19T06:51:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.533+0000 7f71bf7fe640 1 -- 172.21.15.103:0/2258157338 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f71d8004a40 con 0x5585c7b3fe40 2024-09-19T06:51:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.533+0000 7f71e8664740 1 -- 172.21.15.103:0/2258157338 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d8021450 con 0x5585c7b3fe40 2024-09-19T06:51:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.533+0000 7f71bf7fe640 1 -- 172.21.15.103:0/2258157338 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f71d803f050 con 0x5585c7b3fe40 2024-09-19T06:51:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.533+0000 7f71e8664740 10 client.16188 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.533+0000 7f71e8664740 1 -- 172.21.15.103:0/2258157338 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5585c7bbaae0 con 0x5585c7b3fe40 2024-09-19T06:51:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.533+0000 7f71bf7fe640 1 -- 172.21.15.103:0/2258157338 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d8021450 con 0x5585c7b3fe40 2024-09-19T06:51:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.534+0000 7f71bf7fe640 1 -- 172.21.15.103:0/2258157338 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f71d803a030 con 0x5585c7b3fe40 2024-09-19T06:51:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.534+0000 7f71e8664740 10 client.16188 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.534+0000 7f71e8664740 10 client.16188 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:51:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.534+0000 7f71e8664740 20 client.16188 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.534+0000 7f71e8664740 1 --2- 172.21.15.103:0/2258157338 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5585c7bbb210 0x5585c7bbd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.534+0000 7f71e8664740 4 client.16188 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T06:51:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.534+0000 7f71e8664740 1 -- 172.21.15.103:0/2258157338 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x5585c7a9cbe0 con 0x5585c7bbb210 2024-09-19T06:51:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.534+0000 7f71dffff640 1 --2- 172.21.15.103:0/2258157338 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5585c7bbb210 0x5585c7bbd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.534+0000 7f71bf7fe640 10 client.16188 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.534+0000 7f71dffff640 1 --2- 172.21.15.103:0/2258157338 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5585c7bbb210 0x5585c7bbd620 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7946773a9d636cb3 server_cookie=9a9c119e0e235bce in_seq=0 out_seq=0 2024-09-19T06:51:02.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.549+0000 7f71bf7fe640 1 -- 172.21.15.103:0/2258157338 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5585c7a9cbe0 con 0x5585c7bbb210 2024-09-19T06:51:02.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.549+0000 7f71bf7fe640 10 client.16188 handle_command_reply: tid=0 2024-09-19T06:51:02.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.549+0000 7f71e8664740 1 client.16188 shutdown 2024-09-19T06:51:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.549+0000 7f71e8664740 1 -- 172.21.15.103:0/2258157338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 msgr2=0x5585c7bb9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.549+0000 7f71e8664740 1 --2- 172.21.15.103:0/2258157338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7bb9540 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f71d8021e70 tx=0x7f71d8021d10 comp rx=0 tx=0).stop 2024-09-19T06:51:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.549+0000 7f71e8664740 1 -- 172.21.15.103:0/2258157338 shutdown_connections 2024-09-19T06:51:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.549+0000 7f71e8664740 1 --2- 172.21.15.103:0/2258157338 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5585c7bbb210 0x5585c7bbd620 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.549+0000 7f71e8664740 1 --2- 172.21.15.103:0/2258157338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7b3fe40 0x5585c7bb9540 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.549+0000 7f71e8664740 1 --2- 172.21.15.103:0/2258157338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585c7b31050 0x5585c7b44430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.549+0000 7f71e8664740 1 --2- 172.21.15.103:0/2258157338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585c7b30730 0x5585c7b43ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.549+0000 7f71e8664740 1 -- 172.21.15.103:0/2258157338 >> 172.21.15.103:0/2258157338 conn(0x5585c7b2cde0 msgr2=0x5585c7b32240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:02.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.549+0000 7f71bf7fe640 0 client.16188 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:02.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.550+0000 7f71e8664740 1 -- 172.21.15.103:0/2258157338 shutdown_connections 2024-09-19T06:51:02.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.550+0000 7f71e8664740 1 -- 172.21.15.103:0/2258157338 wait complete. 2024-09-19T06:51:02.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.550+0000 7f71e8664740 20 client.16188 trim_cache size 0 max 16384 2024-09-19T06:51:02.552 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:51:02.552 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:51:02.552 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:51:02.552 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 10, 2024-09-19T06:51:02.552 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:51:02.552 INFO:teuthology.orchestra.run.smithi103.stdout: "ee303a01": { 2024-09-19T06:51:02.553 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 10, 2024-09-19T06:51:02.553 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:51:02.553 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:51:02.553 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:51:02.553 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:51:02.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:51:02.553 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:51:02.553 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 140.3, 2024-09-19T06:51:02.553 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:51:02.553 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:51:02.553 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:51:02.553 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:51:02.553 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:51:02.553 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:51:02.554 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:51:02.554 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:51:02.554 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:51:02.554 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:51:02.554 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:51:02.554 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:51:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.550+0000 7f71e628b640 1 -- 172.21.15.103:0/2483747119 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f71c80a3430 msgr2=0x7f71c80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.550+0000 7f71e628b640 1 --2- 172.21.15.103:0/2483747119 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f71c80a3430 0x7f71c80a58f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005c40 tx=0x7f71d8009280 comp rx=0 tx=0).stop 2024-09-19T06:51:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.550+0000 7f71e628b640 1 -- 172.21.15.103:0/2483747119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e0100e20 msgr2=0x7f71e01993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.550+0000 7f71e628b640 1 --2- 172.21.15.103:0/2483747119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e0100e20 0x7f71e01993d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f71d0006010 tx=0x7f71d0003c10 comp rx=0 tx=0).stop 2024-09-19T06:51:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.550+0000 7f71e628b640 1 -- 172.21.15.103:0/2483747119 shutdown_connections 2024-09-19T06:51:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.550+0000 7f71e628b640 1 --2- 172.21.15.103:0/2483747119 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f71c80a3430 0x7f71c80a58f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.550+0000 7f71e628b640 1 --2- 172.21.15.103:0/2483747119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71e0102ef0 0x7f71e01a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.550+0000 7f71e628b640 1 --2- 172.21.15.103:0/2483747119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71e0102570 0x7f71e0199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.550+0000 7f71e628b640 1 --2- 172.21.15.103:0/2483747119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e0100e20 0x7f71e01993d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.550+0000 7f71e628b640 1 -- 172.21.15.103:0/2483747119 >> 172.21.15.103:0/2483747119 conn(0x7f71e00fc7d0 msgr2=0x7f71e00fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.550+0000 7f71e628b640 1 -- 172.21.15.103:0/2483747119 shutdown_connections 2024-09-19T06:51:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:02.550+0000 7f71e628b640 1 -- 172.21.15.103:0/2483747119 wait complete. 2024-09-19T06:51:02.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:02 smithi174 ceph-mon[35637]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-19T06:51:03.219 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ee303a01', quiesce took 3.1 seconds. Will release after: 17.25 2024-09-19T06:51:04.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:04 smithi174 ceph-mon[35637]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-19T06:51:04.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:04 smithi103 ceph-mon[31158]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-19T06:51:04.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:04 smithi134 ceph-mon[35356]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-19T06:51:06.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:06 smithi174 ceph-mon[35637]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 KiB/s wr, 30 op/s 2024-09-19T06:51:06.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:06 smithi103 ceph-mon[31158]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 KiB/s wr, 30 op/s 2024-09-19T06:51:06.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:06 smithi134 ceph-mon[35356]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 KiB/s wr, 30 op/s 2024-09-19T06:51:08.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:08 smithi174 ceph-mon[35637]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 26 op/s 2024-09-19T06:51:08.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:08 smithi103 ceph-mon[31158]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 26 op/s 2024-09-19T06:51:08.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:08 smithi134 ceph-mon[35356]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 26 op/s 2024-09-19T06:51:10.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:10 smithi174 ceph-mon[35637]: pgmap v61: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 30 KiB/s wr, 30 op/s 2024-09-19T06:51:10.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:10 smithi103 ceph-mon[31158]: pgmap v61: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 30 KiB/s wr, 30 op/s 2024-09-19T06:51:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.434+0000 7f5f4d78f640 1 -- 172.21.15.103:0/2988451739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f48102d20 msgr2=0x7f5f48103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.434+0000 7f5f4d78f640 1 --2- 172.21.15.103:0/2988451739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f48102d20 0x7f5f48103100 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009900 tx=0x7f5f3802a480 comp rx=0 tx=0).stop 2024-09-19T06:51:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.434+0000 7f5f4d78f640 1 -- 172.21.15.103:0/2988451739 shutdown_connections 2024-09-19T06:51:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.434+0000 7f5f4d78f640 1 --2- 172.21.15.103:0/2988451739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f4810c0f0 0x7f5f4810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.434+0000 7f5f4d78f640 1 --2- 172.21.15.103:0/2988451739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f48103640 0x7f5f4810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.434+0000 7f5f4d78f640 1 --2- 172.21.15.103:0/2988451739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f48102d20 0x7f5f48103100 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.434+0000 7f5f4d78f640 1 -- 172.21.15.103:0/2988451739 >> 172.21.15.103:0/2988451739 conn(0x7f5f480fc6b0 msgr2=0x7f5f480fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.434+0000 7f5f4d78f640 1 -- 172.21.15.103:0/2988451739 shutdown_connections 2024-09-19T06:51:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.434+0000 7f5f4d78f640 1 -- 172.21.15.103:0/2988451739 wait complete. 2024-09-19T06:51:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.435+0000 7f5f4d78f640 1 Processor -- start 2024-09-19T06:51:10.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:10 smithi134 ceph-mon[35356]: pgmap v61: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 30 KiB/s wr, 30 op/s 2024-09-19T06:51:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.435+0000 7f5f4d78f640 1 -- start start 2024-09-19T06:51:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.435+0000 7f5f4d78f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f48102d20 0x7f5f48100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.435+0000 7f5f4d78f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f48103640 0x7f5f48100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.435+0000 7f5f4d78f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4810c0f0 0x7f5f481013f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f4d78f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48111ef0 con 0x7f5f48102d20 2024-09-19T06:51:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f4d78f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48111d70 con 0x7f5f4810c0f0 2024-09-19T06:51:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f4d78f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48112070 con 0x7f5f48103640 2024-09-19T06:51:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f46ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f48102d20 0x7f5f48100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f46ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f48102d20 0x7f5f48100970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40524/0 (socket says 172.21.15.103:40524) 2024-09-19T06:51:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f46ffd640 1 -- 172.21.15.103:0/3477372775 learned_addr learned my addr 172.21.15.103:0/3477372775 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f467fc640 1 --2- 172.21.15.103:0/3477372775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f48103640 0x7f5f48100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f477fe640 1 --2- 172.21.15.103:0/3477372775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4810c0f0 0x7f5f481013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f46ffd640 1 -- 172.21.15.103:0/3477372775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f48103640 msgr2=0x7f5f48100eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f46ffd640 1 --2- 172.21.15.103:0/3477372775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f48103640 0x7f5f48100eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f46ffd640 1 -- 172.21.15.103:0/3477372775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4810c0f0 msgr2=0x7f5f481013f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f46ffd640 1 --2- 172.21.15.103:0/3477372775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4810c0f0 0x7f5f481013f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f46ffd640 1 -- 172.21.15.103:0/3477372775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f481a68c0 con 0x7f5f48102d20 2024-09-19T06:51:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f46ffd640 1 --2- 172.21.15.103:0/3477372775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f48102d20 0x7f5f48100970 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5f38006010 tx=0x7f5f380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f3ffff640 1 -- 172.21.15.103:0/3477372775 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3803f030 con 0x7f5f48102d20 2024-09-19T06:51:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f3ffff640 1 -- 172.21.15.103:0/3477372775 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f38031990 con 0x7f5f48102d20 2024-09-19T06:51:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f4d78f640 1 -- 172.21.15.103:0/3477372775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f481a6b50 con 0x7f5f48102d20 2024-09-19T06:51:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f3ffff640 1 -- 172.21.15.103:0/3477372775 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38035070 con 0x7f5f48102d20 2024-09-19T06:51:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.436+0000 7f5f4d78f640 1 -- 172.21.15.103:0/3477372775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f481a7090 con 0x7f5f48102d20 2024-09-19T06:51:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.439+0000 7f5f3ffff640 1 -- 172.21.15.103:0/3477372775 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f5f38042030 con 0x7f5f48102d20 2024-09-19T06:51:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.439+0000 7f5f505ff740 1 Processor -- start 2024-09-19T06:51:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.440+0000 7f5f505ff740 1 -- start start 2024-09-19T06:51:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.440+0000 7f5f505ff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 0x55d72c0d5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.440+0000 7f5f505ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0d6410 0x55d72c0e0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.440+0000 7f5f505ff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0e0cb0 0x55d72c0e30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.440+0000 7f5f505ff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d72c03a2f0 con 0x55d72c0e0cb0 2024-09-19T06:51:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.440+0000 7f5f505ff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d72bfe2270 con 0x55d72c0d5af0 2024-09-19T06:51:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.440+0000 7f5f505ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d72c0362b0 con 0x55d72c0d6410 2024-09-19T06:51:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.440+0000 7f5f467fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0e0cb0 0x55d72c0e30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.440+0000 7f5f467fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0e0cb0 0x55d72c0e30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40540/0 (socket says 172.21.15.103:40540) 2024-09-19T06:51:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.440+0000 7f5f467fc640 1 -- 172.21.15.103:0/3797004881 learned_addr learned my addr 172.21.15.103:0/3797004881 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.440+0000 7f5f3ffff640 1 --2- 172.21.15.103:0/3477372775 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5f340a34e0 0x7f5f340a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.440+0000 7f5f477fe640 1 --2- 172.21.15.103:0/3797004881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 0x55d72c0d5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.440+0000 7f5f46ffd640 1 --2- 172.21.15.103:0/3797004881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0d6410 0x55d72c0e0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.440+0000 7f5f477fe640 1 --2- 172.21.15.103:0/3477372775 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5f340a34e0 0x7f5f340a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.440+0000 7f5f467fc640 1 -- 172.21.15.103:0/3797004881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0d6410 msgr2=0x55d72c0e0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.440+0000 7f5f467fc640 1 --2- 172.21.15.103:0/3797004881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0d6410 0x55d72c0e0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.441+0000 7f5f467fc640 1 -- 172.21.15.103:0/3797004881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 msgr2=0x55d72c0d5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.441+0000 7f5f467fc640 1 --2- 172.21.15.103:0/3797004881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 0x55d72c0d5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.441+0000 7f5f467fc640 1 -- 172.21.15.103:0/3797004881 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d72bfe1340 con 0x55d72c0e0cb0 2024-09-19T06:51:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.441+0000 7f5f3ffff640 1 -- 172.21.15.103:0/3477372775 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5f380f01d0 con 0x7f5f48102d20 2024-09-19T06:51:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.441+0000 7f5f467fc640 1 --2- 172.21.15.103:0/3797004881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0e0cb0 0x55d72c0e30a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009c20 tx=0x7f5f300117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e00200ddc98e01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.441+0000 7f5f477fe640 1 --2- 172.21.15.103:0/3477372775 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5f340a34e0 0x7f5f340a59a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f40005c40 tx=0x7f5f40004ab0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.441+0000 7f5f3d7fa640 1 -- 172.21.15.103:0/3797004881 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30012450 con 0x55d72c0e0cb0 2024-09-19T06:51:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.441+0000 7f5f3d7fa640 1 -- 172.21.15.103:0/3797004881 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f30012ad0 con 0x55d72c0e0cb0 2024-09-19T06:51:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.441+0000 7f5f3d7fa640 1 -- 172.21.15.103:0/3797004881 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3000fbc0 con 0x55d72c0e0cb0 2024-09-19T06:51:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.442+0000 7f5f505ff740 1 -- 172.21.15.103:0/3797004881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0e0cb0 msgr2=0x55d72c0e30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.442+0000 7f5f505ff740 1 --2- 172.21.15.103:0/3797004881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0e0cb0 0x55d72c0e30a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009c20 tx=0x7f5f300117b0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.442+0000 7f5f505ff740 1 -- 172.21.15.103:0/3797004881 shutdown_connections 2024-09-19T06:51:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.442+0000 7f5f505ff740 1 --2- 172.21.15.103:0/3797004881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0e0cb0 0x55d72c0e30a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.442+0000 7f5f505ff740 1 --2- 172.21.15.103:0/3797004881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0d6410 0x55d72c0e0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.442+0000 7f5f505ff740 1 --2- 172.21.15.103:0/3797004881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 0x55d72c0d5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.442+0000 7f5f505ff740 1 -- 172.21.15.103:0/3797004881 >> 172.21.15.103:0/3797004881 conn(0x55d72c0d2060 msgr2=0x55d72c0d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.442+0000 7f5f505ff740 1 -- 172.21.15.103:0/3797004881 shutdown_connections 2024-09-19T06:51:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.442+0000 7f5f505ff740 1 -- 172.21.15.103:0/3797004881 wait complete. 2024-09-19T06:51:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.442+0000 7f5f505ff740 1 -- start start 2024-09-19T06:51:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.442+0000 7f5f505ff740 1 Processor -- start 2024-09-19T06:51:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f505ff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 0x55d72c0ddae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f505ff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0de020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f505ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0e0cb0 0x55d72c0de560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f505ff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d72c0e3880 con 0x55d72c0d6410 2024-09-19T06:51:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f505ff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d72bfc3290 con 0x55d72c0d5af0 2024-09-19T06:51:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f505ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d72c0e3a00 con 0x55d72c0e0cb0 2024-09-19T06:51:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f505ff740 10 client.0 _ll_register_callbacks cb 0x55d72bff4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f467fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0de020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f467fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0de020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40556/0 (socket says 172.21.15.103:40556) 2024-09-19T06:51:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f467fc640 1 -- 172.21.15.103:0/3042672805 learned_addr learned my addr 172.21.15.103:0/3042672805 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f46ffd640 1 --2- 172.21.15.103:0/3042672805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 0x55d72c0ddae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f477fe640 1 --2- 172.21.15.103:0/3042672805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0e0cb0 0x55d72c0de560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f467fc640 1 -- 172.21.15.103:0/3042672805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0e0cb0 msgr2=0x55d72c0de560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f467fc640 1 --2- 172.21.15.103:0/3042672805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0e0cb0 0x55d72c0de560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f467fc640 1 -- 172.21.15.103:0/3042672805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 msgr2=0x55d72c0ddae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f467fc640 1 --2- 172.21.15.103:0/3042672805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 0x55d72c0ddae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f467fc640 1 -- 172.21.15.103:0/3042672805 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d72bd4d210 con 0x55d72c0d6410 2024-09-19T06:51:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f46ffd640 1 --2- 172.21.15.103:0/3042672805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 0x55d72c0ddae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f467fc640 1 --2- 172.21.15.103:0/3042672805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0de020 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5f30000c00 tx=0x7f5f3000f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5e58ad38a78104 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f2e7fc640 1 -- 172.21.15.103:0/3042672805 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30028070 con 0x55d72c0d6410 2024-09-19T06:51:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f2e7fc640 1 -- 172.21.15.103:0/3042672805 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5f30012450 con 0x55d72c0d6410 2024-09-19T06:51:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.443+0000 7f5f505ff740 1 -- 172.21.15.103:0/3042672805 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f30012ad0 con 0x55d72c0d6410 2024-09-19T06:51:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.444+0000 7f5f2e7fc640 1 -- 172.21.15.103:0/3042672805 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f30034050 con 0x55d72c0d6410 2024-09-19T06:51:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.444+0000 7f5f505ff740 10 client.16200 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.444+0000 7f5f505ff740 1 -- 172.21.15.103:0/3042672805 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d72be19620 con 0x55d72c0d6410 2024-09-19T06:51:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.444+0000 7f5f2e7fc640 1 -- 172.21.15.103:0/3042672805 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f30012ad0 con 0x55d72c0d6410 2024-09-19T06:51:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.444+0000 7f5f2e7fc640 1 -- 172.21.15.103:0/3042672805 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5f30003710 con 0x55d72c0d6410 2024-09-19T06:51:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.444+0000 7f5f505ff740 10 client.16200 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.444+0000 7f5f505ff740 10 client.16200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.444+0000 7f5f505ff740 20 client.16200 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.444+0000 7f5f505ff740 1 --2- 172.21.15.103:0/3042672805 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d72c0f0ac0 0x55d72c0f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.444+0000 7f5f505ff740 4 client.16200 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.444+0000 7f5f505ff740 1 -- 172.21.15.103:0/3042672805 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d72c047a20 con 0x55d72c0f0ac0 2024-09-19T06:51:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.444+0000 7f5f46ffd640 1 --2- 172.21.15.103:0/3042672805 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d72c0f0ac0 0x55d72c0f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.445+0000 7f5f2e7fc640 10 client.16200 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.445+0000 7f5f46ffd640 1 --2- 172.21.15.103:0/3042672805 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d72c0f0ac0 0x55d72c0f2eb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d3a07a8e4c632f server_cookie=4968dd05e16b47a2 in_seq=0 out_seq=0 2024-09-19T06:51:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.446+0000 7f5f2e7fc640 1 -- 172.21.15.103:0/3042672805 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d72c047a20 con 0x55d72c0f0ac0 2024-09-19T06:51:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.446+0000 7f5f2e7fc640 10 client.16200 handle_command_reply: tid=0 2024-09-19T06:51:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.446+0000 7f5f505ff740 1 client.16200 shutdown 2024-09-19T06:51:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.446+0000 7f5f505ff740 1 -- 172.21.15.103:0/3042672805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 msgr2=0x55d72c0de020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.446+0000 7f5f505ff740 1 --2- 172.21.15.103:0/3042672805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0de020 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5f30000c00 tx=0x7f5f3000f5b0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.446+0000 7f5f505ff740 1 -- 172.21.15.103:0/3042672805 shutdown_connections 2024-09-19T06:51:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.446+0000 7f5f505ff740 1 --2- 172.21.15.103:0/3042672805 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d72c0f0ac0 0x55d72c0f2eb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.446+0000 7f5f505ff740 1 --2- 172.21.15.103:0/3042672805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0e0cb0 0x55d72c0de560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.446+0000 7f5f505ff740 1 --2- 172.21.15.103:0/3042672805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0de020 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.446+0000 7f5f2e7fc640 0 client.16200 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.446+0000 7f5f505ff740 1 --2- 172.21.15.103:0/3042672805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 0x55d72c0ddae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.446+0000 7f5f505ff740 1 -- 172.21.15.103:0/3042672805 >> 172.21.15.103:0/3042672805 conn(0x55d72c0d2060 msgr2=0x55d72c0d7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.446+0000 7f5f505ff740 1 -- 172.21.15.103:0/3042672805 shutdown_connections 2024-09-19T06:51:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.446+0000 7f5f505ff740 1 -- 172.21.15.103:0/3042672805 wait complete. 2024-09-19T06:51:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.446+0000 7f5f505ff740 20 client.16200 trim_cache size 0 max 16384 2024-09-19T06:51:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.460+0000 7f5f505ff740 1 Processor -- start 2024-09-19T06:51:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.460+0000 7f5f505ff740 1 -- start start 2024-09-19T06:51:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f505ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0d5af0 0x55d72c0dc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f505ff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0de390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f505ff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0e0cb0 0x55d72c0f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f505ff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481af990 con 0x55d72c0d6410 2024-09-19T06:51:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f505ff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481b14d0 con 0x55d72c0e0cb0 2024-09-19T06:51:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f505ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481b0060 con 0x55d72c0d5af0 2024-09-19T06:51:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f467fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0de390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f467fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0de390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40558/0 (socket says 172.21.15.103:40558) 2024-09-19T06:51:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f467fc640 1 -- 172.21.15.103:0/3436053914 learned_addr learned my addr 172.21.15.103:0/3436053914 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f46ffd640 1 --2- 172.21.15.103:0/3436053914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0d5af0 0x55d72c0dc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f477fe640 1 --2- 172.21.15.103:0/3436053914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0e0cb0 0x55d72c0f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f467fc640 1 -- 172.21.15.103:0/3436053914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0d5af0 msgr2=0x55d72c0dc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f467fc640 1 --2- 172.21.15.103:0/3436053914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0d5af0 0x55d72c0dc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f467fc640 1 -- 172.21.15.103:0/3436053914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0e0cb0 msgr2=0x55d72c0f84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f467fc640 1 --2- 172.21.15.103:0/3436053914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0e0cb0 0x55d72c0f84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f467fc640 1 -- 172.21.15.103:0/3436053914 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d72c0d1e90 con 0x55d72c0d6410 2024-09-19T06:51:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f477fe640 1 --2- 172.21.15.103:0/3436053914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0e0cb0 0x55d72c0f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f467fc640 1 --2- 172.21.15.103:0/3436053914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0de390 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5f30003cd0 tx=0x7f5f30009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5268d2f7766d4d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f3cff9640 1 -- 172.21.15.103:0/3436053914 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30028040 con 0x55d72c0d6410 2024-09-19T06:51:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f3cff9640 1 -- 172.21.15.103:0/3436053914 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f30005400 con 0x55d72c0d6410 2024-09-19T06:51:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.461+0000 7f5f3cff9640 1 -- 172.21.15.103:0/3436053914 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30003850 con 0x55d72c0d6410 2024-09-19T06:51:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.462+0000 7f5f505ff740 1 -- 172.21.15.103:0/3436053914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 msgr2=0x55d72c0de390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.462+0000 7f5f505ff740 1 --2- 172.21.15.103:0/3436053914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0de390 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5f30003cd0 tx=0x7f5f30009fa0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.462+0000 7f5f505ff740 1 -- 172.21.15.103:0/3436053914 shutdown_connections 2024-09-19T06:51:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.462+0000 7f5f505ff740 1 --2- 172.21.15.103:0/3436053914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0e0cb0 0x55d72c0f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.462+0000 7f5f505ff740 1 --2- 172.21.15.103:0/3436053914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0de390 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.462+0000 7f5f505ff740 1 --2- 172.21.15.103:0/3436053914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0d5af0 0x55d72c0dc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.462+0000 7f5f505ff740 1 -- 172.21.15.103:0/3436053914 >> 172.21.15.103:0/3436053914 conn(0x55d72c0d2060 msgr2=0x55d72c0dfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.462+0000 7f5f505ff740 1 -- 172.21.15.103:0/3436053914 shutdown_connections 2024-09-19T06:51:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.462+0000 7f5f505ff740 1 -- 172.21.15.103:0/3436053914 wait complete. 2024-09-19T06:51:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.462+0000 7f5f505ff740 1 -- start start 2024-09-19T06:51:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f505ff740 1 Processor -- start 2024-09-19T06:51:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f505ff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 0x55d72c0e1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f505ff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0e2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f505ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0e0cb0 0x55d72c0e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f505ff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481b0e00 con 0x55d72c0d6410 2024-09-19T06:51:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f505ff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481b0730 con 0x55d72c0d5af0 2024-09-19T06:51:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f505ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481ad950 con 0x55d72c0e0cb0 2024-09-19T06:51:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f505ff740 10 client.0 _ll_register_callbacks cb 0x7f5f30009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f467fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0e2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f467fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0e2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40564/0 (socket says 172.21.15.103:40564) 2024-09-19T06:51:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f467fc640 1 -- 172.21.15.103:0/978858374 learned_addr learned my addr 172.21.15.103:0/978858374 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f477fe640 1 --2- 172.21.15.103:0/978858374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0e0cb0 0x55d72c0e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f467fc640 1 -- 172.21.15.103:0/978858374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0e0cb0 msgr2=0x55d72c0e28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f467fc640 1 --2- 172.21.15.103:0/978858374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0e0cb0 0x55d72c0e28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f467fc640 1 -- 172.21.15.103:0/978858374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 msgr2=0x55d72c0e1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f46ffd640 1 --2- 172.21.15.103:0/978858374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 0x55d72c0e1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f467fc640 1 --2- 172.21.15.103:0/978858374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 0x55d72c0e1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f467fc640 1 -- 172.21.15.103:0/978858374 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d72c0e7200 con 0x55d72c0d6410 2024-09-19T06:51:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f46ffd640 1 --2- 172.21.15.103:0/978858374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 0x55d72c0e1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f467fc640 1 --2- 172.21.15.103:0/978858374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0e2360 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009b70 tx=0x7f5f300096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a8bcb3c557b576 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.463+0000 7f5f2e7fc640 1 -- 172.21.15.103:0/978858374 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30028040 con 0x55d72c0d6410 2024-09-19T06:51:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.464+0000 7f5f2e7fc640 1 -- 172.21.15.103:0/978858374 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5f300035f0 con 0x55d72c0d6410 2024-09-19T06:51:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.464+0000 7f5f505ff740 1 -- 172.21.15.103:0/978858374 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f30005400 con 0x55d72c0d6410 2024-09-19T06:51:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.464+0000 7f5f2e7fc640 1 -- 172.21.15.103:0/978858374 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f300037f0 con 0x55d72c0d6410 2024-09-19T06:51:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.464+0000 7f5f505ff740 10 client.16206 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.464+0000 7f5f505ff740 1 -- 172.21.15.103:0/978858374 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d72c15b710 con 0x55d72c0d6410 2024-09-19T06:51:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.464+0000 7f5f2e7fc640 1 -- 172.21.15.103:0/978858374 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f30005400 con 0x55d72c0d6410 2024-09-19T06:51:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.464+0000 7f5f2e7fc640 1 -- 172.21.15.103:0/978858374 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5f30012890 con 0x55d72c0d6410 2024-09-19T06:51:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.464+0000 7f5f505ff740 10 client.16206 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.464+0000 7f5f505ff740 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.464+0000 7f5f505ff740 20 client.16206 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.464+0000 7f5f505ff740 1 --2- 172.21.15.103:0/978858374 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d72c15bdb0 0x55d72c15e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.464+0000 7f5f505ff740 4 client.16206 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:51:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.464+0000 7f5f505ff740 1 -- 172.21.15.103:0/978858374 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d72c041d70 con 0x55d72c15bdb0 2024-09-19T06:51:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.464+0000 7f5f46ffd640 1 --2- 172.21.15.103:0/978858374 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d72c15bdb0 0x55d72c15e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.465+0000 7f5f2e7fc640 10 client.16206 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.465+0000 7f5f46ffd640 1 --2- 172.21.15.103:0/978858374 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d72c15bdb0 0x55d72c15e1d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4a48a090a35dbcc server_cookie=e694b894c1751714 in_seq=0 out_seq=0 2024-09-19T06:51:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.473+0000 7f5f2e7fc640 1 -- 172.21.15.103:0/978858374 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d72c041d70 con 0x55d72c15bdb0 2024-09-19T06:51:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.473+0000 7f5f2e7fc640 10 client.16206 handle_command_reply: tid=0 2024-09-19T06:51:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.473+0000 7f5f505ff740 1 client.16206 shutdown 2024-09-19T06:51:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.473+0000 7f5f505ff740 1 -- 172.21.15.103:0/978858374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 msgr2=0x55d72c0e2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.473+0000 7f5f505ff740 1 --2- 172.21.15.103:0/978858374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0e2360 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009b70 tx=0x7f5f300096d0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.473+0000 7f5f505ff740 1 -- 172.21.15.103:0/978858374 shutdown_connections 2024-09-19T06:51:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.473+0000 7f5f505ff740 1 --2- 172.21.15.103:0/978858374 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d72c15bdb0 0x55d72c15e1d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.473+0000 7f5f505ff740 1 --2- 172.21.15.103:0/978858374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d72c0e0cb0 0x55d72c0e28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.473+0000 7f5f2e7fc640 0 client.16206 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.473+0000 7f5f505ff740 1 --2- 172.21.15.103:0/978858374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d72c0d6410 0x55d72c0e2360 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.473+0000 7f5f505ff740 1 --2- 172.21.15.103:0/978858374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d72c0d5af0 0x55d72c0e1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.473+0000 7f5f505ff740 1 -- 172.21.15.103:0/978858374 >> 172.21.15.103:0/978858374 conn(0x55d72c0d2060 msgr2=0x55d72c0d09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.473+0000 7f5f505ff740 1 -- 172.21.15.103:0/978858374 shutdown_connections 2024-09-19T06:51:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.473+0000 7f5f505ff740 1 -- 172.21.15.103:0/978858374 wait complete. 2024-09-19T06:51:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.473+0000 7f5f505ff740 20 client.16206 trim_cache size 0 max 16384 2024-09-19T06:51:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:51:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:51:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:51:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:51:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.474+0000 7f5f4d78f640 1 -- 172.21.15.103:0/3477372775 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5f340a34e0 msgr2=0x7f5f340a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.474+0000 7f5f4d78f640 1 --2- 172.21.15.103:0/3477372775 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5f340a34e0 0x7f5f340a59a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f40005c40 tx=0x7f5f40004ab0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.474+0000 7f5f4d78f640 1 -- 172.21.15.103:0/3477372775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f48102d20 msgr2=0x7f5f48100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.474+0000 7f5f4d78f640 1 --2- 172.21.15.103:0/3477372775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f48102d20 0x7f5f48100970 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5f38006010 tx=0x7f5f380042f0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.474+0000 7f5f4d78f640 1 -- 172.21.15.103:0/3477372775 shutdown_connections 2024-09-19T06:51:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.474+0000 7f5f4d78f640 1 --2- 172.21.15.103:0/3477372775 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5f340a34e0 0x7f5f340a59a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.474+0000 7f5f4d78f640 1 --2- 172.21.15.103:0/3477372775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4810c0f0 0x7f5f481013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.474+0000 7f5f4d78f640 1 --2- 172.21.15.103:0/3477372775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f48103640 0x7f5f48100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.474+0000 7f5f4d78f640 1 --2- 172.21.15.103:0/3477372775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f48102d20 0x7f5f48100970 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.474+0000 7f5f4d78f640 1 -- 172.21.15.103:0/3477372775 >> 172.21.15.103:0/3477372775 conn(0x7f5f480fc6b0 msgr2=0x7f5f480fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.474+0000 7f5f4d78f640 1 -- 172.21.15.103:0/3477372775 shutdown_connections 2024-09-19T06:51:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:10.474+0000 7f5f4d78f640 1 -- 172.21.15.103:0/3477372775 wait complete. 2024-09-19T06:51:12.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:12 smithi174 ceph-mon[35637]: pgmap v62: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-19T06:51:12.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:12 smithi103 ceph-mon[31158]: pgmap v62: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-19T06:51:12.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:12 smithi134 ceph-mon[35356]: pgmap v62: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-19T06:51:14.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:14 smithi174 ceph-mon[35637]: pgmap v63: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-19T06:51:14.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:14 smithi103 ceph-mon[31158]: pgmap v63: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-19T06:51:14.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:14 smithi134 ceph-mon[35356]: pgmap v63: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-19T06:51:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:16 smithi174 ceph-mon[35637]: pgmap v64: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 30 KiB/s wr, 31 op/s 2024-09-19T06:51:16.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:16 smithi103 ceph-mon[31158]: pgmap v64: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 30 KiB/s wr, 31 op/s 2024-09-19T06:51:16.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:16 smithi134 ceph-mon[35356]: pgmap v64: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 30 KiB/s wr, 31 op/s 2024-09-19T06:51:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:18 smithi174 ceph-mon[35637]: pgmap v65: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-19T06:51:18.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:18 smithi103 ceph-mon[31158]: pgmap v65: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-19T06:51:18.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:18 smithi134 ceph-mon[35356]: pgmap v65: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-19T06:51:18.720 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:51:18.982 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:19.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.429+0000 7f9858e46640 1 -- 172.21.15.103:0/1813743766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f985410c170 msgr2=0x7f985410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:19.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.429+0000 7f9858e46640 1 --2- 172.21.15.103:0/1813743766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f985410c170 0x7f985410e610 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f984c009ca0 tx=0x7f984c02a950 comp rx=0 tx=0).stop 2024-09-19T06:51:19.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.429+0000 7f9858e46640 1 -- 172.21.15.103:0/1813743766 shutdown_connections 2024-09-19T06:51:19.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.429+0000 7f9858e46640 1 --2- 172.21.15.103:0/1813743766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f985410c170 0x7f985410e610 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.429+0000 7f9858e46640 1 --2- 172.21.15.103:0/1813743766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9854103660 0x7f985410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.429+0000 7f9858e46640 1 --2- 172.21.15.103:0/1813743766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9854102d40 0x7f9854103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.429+0000 7f9858e46640 1 -- 172.21.15.103:0/1813743766 >> 172.21.15.103:0/1813743766 conn(0x7f98540fc7a0 msgr2=0x7f98540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:19.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.429+0000 7f9858e46640 1 -- 172.21.15.103:0/1813743766 shutdown_connections 2024-09-19T06:51:19.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.429+0000 7f9858e46640 1 -- 172.21.15.103:0/1813743766 wait complete. 2024-09-19T06:51:19.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.430+0000 7f9858e46640 1 Processor -- start 2024-09-19T06:51:19.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.430+0000 7f9858e46640 1 -- start start 2024-09-19T06:51:19.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.430+0000 7f9858e46640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9854102d40 0x7f9854198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.430+0000 7f9858e46640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9854103660 0x7f98541988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.430+0000 7f9858e46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f985410c170 0x7f985419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.430+0000 7f9858e46640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9854111f70 con 0x7f9854102d40 2024-09-19T06:51:19.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.430+0000 7f9858e46640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9854111df0 con 0x7f9854103660 2024-09-19T06:51:19.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.430+0000 7f9858e46640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98541120f0 con 0x7f985410c170 2024-09-19T06:51:19.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.430+0000 7f985259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9854102d40 0x7f9854198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.430+0000 7f985259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9854102d40 0x7f9854198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46230/0 (socket says 172.21.15.103:46230) 2024-09-19T06:51:19.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.430+0000 7f985259c640 1 -- 172.21.15.103:0/330779103 learned_addr learned my addr 172.21.15.103:0/330779103 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:19.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.430+0000 7f9852d9d640 1 --2- 172.21.15.103:0/330779103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f985410c170 0x7f985419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.431+0000 7f9849d9b640 1 --2- 172.21.15.103:0/330779103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9854103660 0x7f98541988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.431+0000 7f985259c640 1 -- 172.21.15.103:0/330779103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f985410c170 msgr2=0x7f985419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:19.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.431+0000 7f985259c640 1 --2- 172.21.15.103:0/330779103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f985410c170 0x7f985419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.431+0000 7f985259c640 1 -- 172.21.15.103:0/330779103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9854103660 msgr2=0x7f98541988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:19.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.431+0000 7f985259c640 1 --2- 172.21.15.103:0/330779103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9854103660 0x7f98541988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.431+0000 7f985259c640 1 -- 172.21.15.103:0/330779103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985419d340 con 0x7f9854102d40 2024-09-19T06:51:19.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.431+0000 7f9849d9b640 1 --2- 172.21.15.103:0/330779103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9854103660 0x7f98541988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:19.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.431+0000 7f985259c640 1 --2- 172.21.15.103:0/330779103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9854102d40 0x7f9854198390 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f984400d770 tx=0x7f984400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:19.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.431+0000 7f984bfff640 1 -- 172.21.15.103:0/330779103 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844017070 con 0x7f9854102d40 2024-09-19T06:51:19.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.431+0000 7f9858e46640 1 -- 172.21.15.103:0/330779103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f985419d630 con 0x7f9854102d40 2024-09-19T06:51:19.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.431+0000 7f984bfff640 1 -- 172.21.15.103:0/330779103 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9844005500 con 0x7f9854102d40 2024-09-19T06:51:19.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.431+0000 7f984bfff640 1 -- 172.21.15.103:0/330779103 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844004e00 con 0x7f9854102d40 2024-09-19T06:51:19.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.431+0000 7f9858e46640 1 -- 172.21.15.103:0/330779103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98541a4f10 con 0x7f9854102d40 2024-09-19T06:51:19.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.433+0000 7f984bfff640 1 -- 172.21.15.103:0/330779103 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9844004fa0 con 0x7f9854102d40 2024-09-19T06:51:19.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.433+0000 7f985b21f740 1 Processor -- start 2024-09-19T06:51:19.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f985b21f740 1 -- start start 2024-09-19T06:51:19.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f985b21f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce274a0 0x55ca4ce27880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f985b21f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce32140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f985b21f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce32680 0x55ca4ce34a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f985b21f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca4cd8d2e0 con 0x55ca4ce32680 2024-09-19T06:51:19.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f985b21f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ca4cd34ce0 con 0x55ca4ce27dc0 2024-09-19T06:51:19.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f985b21f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ca4cd892a0 con 0x55ca4ce274a0 2024-09-19T06:51:19.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f984bfff640 1 --2- 172.21.15.103:0/330779103 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f98400a33b0 0x7f98400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f9849d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce32680 0x55ca4ce34a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f9849d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce32680 0x55ca4ce34a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46242/0 (socket says 172.21.15.103:46242) 2024-09-19T06:51:19.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f9849d9b640 1 -- 172.21.15.103:0/325930445 learned_addr learned my addr 172.21.15.103:0/325930445 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:19.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f9852d9d640 1 --2- 172.21.15.103:0/330779103 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f98400a33b0 0x7f98400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f984bfff640 1 -- 172.21.15.103:0/330779103 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f98440cf790 con 0x7f9854102d40 2024-09-19T06:51:19.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f985259c640 1 --2- 172.21.15.103:0/325930445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce32140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f9852d9d640 1 --2- 172.21.15.103:0/325930445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce274a0 0x55ca4ce27880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f9849d9b640 1 -- 172.21.15.103:0/325930445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce274a0 msgr2=0x55ca4ce27880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:19.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.434+0000 7f9849d9b640 1 --2- 172.21.15.103:0/325930445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce274a0 0x55ca4ce27880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.435+0000 7f9849d9b640 1 -- 172.21.15.103:0/325930445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce27dc0 msgr2=0x55ca4ce32140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:19.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.435+0000 7f9849d9b640 1 --2- 172.21.15.103:0/325930445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce32140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.435+0000 7f9849d9b640 1 -- 172.21.15.103:0/325930445 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4cd19440 con 0x55ca4ce32680 2024-09-19T06:51:19.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.435+0000 7f985259c640 1 --2- 172.21.15.103:0/325930445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce32140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:19.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.435+0000 7f9852d9d640 1 --2- 172.21.15.103:0/325930445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce274a0 0x55ca4ce27880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:19.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.435+0000 7f9849d9b640 1 --2- 172.21.15.103:0/325930445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce32680 0x55ca4ce34a70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f983c008bb0 tx=0x7f983c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67888914cb9ecfde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:19.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.435+0000 7f9852d9d640 1 --2- 172.21.15.103:0/330779103 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f98400a33b0 0x7f98400a5870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f984c009a60 tx=0x7f984c02fe10 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:19.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.435+0000 7f9848d99640 1 -- 172.21.15.103:0/325930445 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c014070 con 0x55ca4ce32680 2024-09-19T06:51:19.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.435+0000 7f9848d99640 1 -- 172.21.15.103:0/325930445 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983c0047a0 con 0x55ca4ce32680 2024-09-19T06:51:19.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.435+0000 7f9848d99640 1 -- 172.21.15.103:0/325930445 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c010670 con 0x55ca4ce32680 2024-09-19T06:51:19.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.436+0000 7f985b21f740 1 -- 172.21.15.103:0/325930445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce32680 msgr2=0x55ca4ce34a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:19.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.436+0000 7f985b21f740 1 --2- 172.21.15.103:0/325930445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce32680 0x55ca4ce34a70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f983c008bb0 tx=0x7f983c00e720 comp rx=0 tx=0).stop 2024-09-19T06:51:19.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.436+0000 7f985b21f740 1 -- 172.21.15.103:0/325930445 shutdown_connections 2024-09-19T06:51:19.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.436+0000 7f985b21f740 1 --2- 172.21.15.103:0/325930445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce32680 0x55ca4ce34a70 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.436+0000 7f985b21f740 1 --2- 172.21.15.103:0/325930445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce32140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.436+0000 7f985b21f740 1 --2- 172.21.15.103:0/325930445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce274a0 0x55ca4ce27880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.436+0000 7f985b21f740 1 -- 172.21.15.103:0/325930445 >> 172.21.15.103:0/325930445 conn(0x55ca4ce23a30 msgr2=0x55ca4ce23e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:19.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.436+0000 7f985b21f740 1 -- 172.21.15.103:0/325930445 shutdown_connections 2024-09-19T06:51:19.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.436+0000 7f985b21f740 1 -- 172.21.15.103:0/325930445 wait complete. 2024-09-19T06:51:19.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.436+0000 7f985b21f740 1 -- start start 2024-09-19T06:51:19.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.436+0000 7f985b21f740 1 Processor -- start 2024-09-19T06:51:19.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.436+0000 7f985b21f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce274a0 0x55ca4ce31fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.436+0000 7f985b21f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce2f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.436+0000 7f985b21f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce32680 0x55ca4ce2f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.436+0000 7f985b21f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca4ce35250 con 0x55ca4ce27dc0 2024-09-19T06:51:19.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f985b21f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ca4cd48050 con 0x55ca4ce274a0 2024-09-19T06:51:19.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f985b21f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ca4ce353d0 con 0x55ca4ce32680 2024-09-19T06:51:19.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f985b21f740 10 client.0 _ll_register_callbacks cb 0x55ca4cd88160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:19.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f9849d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce2f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f9849d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce2f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46250/0 (socket says 172.21.15.103:46250) 2024-09-19T06:51:19.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f9849d9b640 1 -- 172.21.15.103:0/850800635 learned_addr learned my addr 172.21.15.103:0/850800635 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:19.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f985259c640 1 --2- 172.21.15.103:0/850800635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce274a0 0x55ca4ce31fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f9852d9d640 1 --2- 172.21.15.103:0/850800635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce32680 0x55ca4ce2f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f9849d9b640 1 -- 172.21.15.103:0/850800635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce32680 msgr2=0x55ca4ce2f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:19.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f9849d9b640 1 --2- 172.21.15.103:0/850800635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce32680 0x55ca4ce2f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f9849d9b640 1 -- 172.21.15.103:0/850800635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce274a0 msgr2=0x55ca4ce31fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:19.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f9849d9b640 1 --2- 172.21.15.103:0/850800635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce274a0 0x55ca4ce31fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f9849d9b640 1 -- 172.21.15.103:0/850800635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca4cafe3c0 con 0x55ca4ce27dc0 2024-09-19T06:51:19.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f9849d9b640 1 --2- 172.21.15.103:0/850800635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce2f250 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f983c00ec20 tx=0x7f983c0051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab61cc5f95dcaf17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:19.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f9839ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:19.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f9839ffb640 1 -- 172.21.15.103:0/850800635 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c014070 con 0x55ca4ce27dc0 2024-09-19T06:51:19.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f9839ffb640 1 -- 172.21.15.103:0/850800635 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f983c0054d0 con 0x55ca4ce27dc0 2024-09-19T06:51:19.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f985b21f740 1 -- 172.21.15.103:0/850800635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f983c0047a0 con 0x55ca4ce27dc0 2024-09-19T06:51:19.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f9839ffb640 1 -- 172.21.15.103:0/850800635 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f983c004030 con 0x55ca4ce27dc0 2024-09-19T06:51:19.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f985b21f740 10 client.16218 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:19.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.437+0000 7f985b21f740 1 -- 172.21.15.103:0/850800635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ca4ce40bd0 con 0x55ca4ce27dc0 2024-09-19T06:51:19.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.438+0000 7f9839ffb640 1 -- 172.21.15.103:0/850800635 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983c0047a0 con 0x55ca4ce27dc0 2024-09-19T06:51:19.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.438+0000 7f9839ffb640 1 -- 172.21.15.103:0/850800635 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f983c010430 con 0x55ca4ce27dc0 2024-09-19T06:51:19.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.438+0000 7f985b21f740 10 client.16218 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:19.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.438+0000 7f985b21f740 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:19.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.438+0000 7f985b21f740 20 client.16218 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:19.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.438+0000 7f985b21f740 1 --2- 172.21.15.103:0/850800635 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ca4ce42090 0x55ca4ce44480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.438+0000 7f985b21f740 4 client.16218 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:19.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.438+0000 7f985b21f740 1 -- 172.21.15.103:0/850800635 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca4cd9a850 con 0x55ca4ce42090 2024-09-19T06:51:19.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.438+0000 7f985259c640 1 --2- 172.21.15.103:0/850800635 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ca4ce42090 0x55ca4ce44480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.439+0000 7f985259c640 1 --2- 172.21.15.103:0/850800635 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ca4ce42090 0x55ca4ce44480 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f23438cabc4265b server_cookie=44685cfb2dc09e52 in_seq=0 out_seq=0 2024-09-19T06:51:19.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.439+0000 7f9839ffb640 10 client.16218 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:19.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.439+0000 7f9839ffb640 1 -- 172.21.15.103:0/850800635 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca4cd9a850 con 0x55ca4ce42090 2024-09-19T06:51:19.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.439+0000 7f9839ffb640 10 client.16218 handle_command_reply: tid=0 2024-09-19T06:51:19.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.439+0000 7f985b21f740 1 client.16218 shutdown 2024-09-19T06:51:19.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.439+0000 7f985b21f740 1 -- 172.21.15.103:0/850800635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce27dc0 msgr2=0x55ca4ce2f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:19.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.439+0000 7f985b21f740 1 --2- 172.21.15.103:0/850800635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce2f250 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f983c00ec20 tx=0x7f983c0051a0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.439+0000 7f985b21f740 1 -- 172.21.15.103:0/850800635 shutdown_connections 2024-09-19T06:51:19.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.439+0000 7f985b21f740 1 --2- 172.21.15.103:0/850800635 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ca4ce42090 0x55ca4ce44480 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.439+0000 7f985b21f740 1 --2- 172.21.15.103:0/850800635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce32680 0x55ca4ce2f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.439+0000 7f985b21f740 1 --2- 172.21.15.103:0/850800635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce2f250 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.439+0000 7f9839ffb640 0 client.16218 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:19.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.439+0000 7f985b21f740 1 --2- 172.21.15.103:0/850800635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce274a0 0x55ca4ce31fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.439+0000 7f985b21f740 1 -- 172.21.15.103:0/850800635 >> 172.21.15.103:0/850800635 conn(0x55ca4ce23a30 msgr2=0x55ca4ce29710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:19.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.440+0000 7f985b21f740 1 -- 172.21.15.103:0/850800635 shutdown_connections 2024-09-19T06:51:19.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.440+0000 7f985b21f740 1 -- 172.21.15.103:0/850800635 wait complete. 2024-09-19T06:51:19.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.440+0000 7f985b21f740 20 client.16218 trim_cache size 0 max 16384 2024-09-19T06:51:19.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.456+0000 7f985b21f740 1 Processor -- start 2024-09-19T06:51:19.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.456+0000 7f985b21f740 1 -- start start 2024-09-19T06:51:19.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f985b21f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce274a0 0x55ca4ce3e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f985b21f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce49ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f985b21f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce32680 0x55ca4ce49ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f985b21f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98541ad810 con 0x55ca4ce274a0 2024-09-19T06:51:19.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f985b21f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98541af350 con 0x55ca4ce27dc0 2024-09-19T06:51:19.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f985b21f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98541adee0 con 0x55ca4ce32680 2024-09-19T06:51:19.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f985259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce274a0 0x55ca4ce3e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f985259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce274a0 0x55ca4ce3e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46254/0 (socket says 172.21.15.103:46254) 2024-09-19T06:51:19.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f985259c640 1 -- 172.21.15.103:0/1285986298 learned_addr learned my addr 172.21.15.103:0/1285986298 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:19.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f9849d9b640 1 --2- 172.21.15.103:0/1285986298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce49ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f9852d9d640 1 --2- 172.21.15.103:0/1285986298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce32680 0x55ca4ce49ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f985259c640 1 -- 172.21.15.103:0/1285986298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce32680 msgr2=0x55ca4ce49ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:19.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f985259c640 1 --2- 172.21.15.103:0/1285986298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce32680 0x55ca4ce49ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f985259c640 1 -- 172.21.15.103:0/1285986298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce27dc0 msgr2=0x55ca4ce49ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:19.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f985259c640 1 --2- 172.21.15.103:0/1285986298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce49ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f985259c640 1 -- 172.21.15.103:0/1285986298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4ce34db0 con 0x55ca4ce274a0 2024-09-19T06:51:19.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f985259c640 1 --2- 172.21.15.103:0/1285986298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce274a0 0x55ca4ce3e4b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f984400d740 tx=0x7f98440cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc4381ad96eba06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:19.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f983bfff640 1 -- 172.21.15.103:0/1285986298 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844020040 con 0x55ca4ce274a0 2024-09-19T06:51:19.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f983bfff640 1 -- 172.21.15.103:0/1285986298 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98440a46f0 con 0x55ca4ce274a0 2024-09-19T06:51:19.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.457+0000 7f983bfff640 1 -- 172.21.15.103:0/1285986298 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98440980f0 con 0x55ca4ce274a0 2024-09-19T06:51:19.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.458+0000 7f985b21f740 1 -- 172.21.15.103:0/1285986298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce274a0 msgr2=0x55ca4ce3e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:19.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.458+0000 7f985b21f740 1 --2- 172.21.15.103:0/1285986298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce274a0 0x55ca4ce3e4b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f984400d740 tx=0x7f98440cf040 comp rx=0 tx=0).stop 2024-09-19T06:51:19.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.458+0000 7f985b21f740 1 -- 172.21.15.103:0/1285986298 shutdown_connections 2024-09-19T06:51:19.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.458+0000 7f985b21f740 1 --2- 172.21.15.103:0/1285986298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce32680 0x55ca4ce49ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.458+0000 7f985b21f740 1 --2- 172.21.15.103:0/1285986298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce49ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.458+0000 7f985b21f740 1 --2- 172.21.15.103:0/1285986298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce274a0 0x55ca4ce3e4b0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.458+0000 7f985b21f740 1 -- 172.21.15.103:0/1285986298 >> 172.21.15.103:0/1285986298 conn(0x55ca4ce23a30 msgr2=0x55ca4ce2a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:19.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.458+0000 7f985b21f740 1 -- 172.21.15.103:0/1285986298 shutdown_connections 2024-09-19T06:51:19.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.458+0000 7f985b21f740 1 -- 172.21.15.103:0/1285986298 wait complete. 2024-09-19T06:51:19.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.458+0000 7f985b21f740 1 -- start start 2024-09-19T06:51:19.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f985b21f740 1 Processor -- start 2024-09-19T06:51:19.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f985b21f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce274a0 0x55ca4ce35d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f985b21f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce362a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f985b21f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce32680 0x55ca4ceaaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f985b21f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98541aec80 con 0x55ca4ce27dc0 2024-09-19T06:51:19.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f985b21f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98541ae5b0 con 0x55ca4ce32680 2024-09-19T06:51:19.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f985b21f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98541ab7d0 con 0x55ca4ce274a0 2024-09-19T06:51:19.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f985b21f740 10 client.0 _ll_register_callbacks cb 0x7f983c008a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:19.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f9849d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce362a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f9849d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce362a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46264/0 (socket says 172.21.15.103:46264) 2024-09-19T06:51:19.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f9849d9b640 1 -- 172.21.15.103:0/807564722 learned_addr learned my addr 172.21.15.103:0/807564722 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:19.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f9852d9d640 1 --2- 172.21.15.103:0/807564722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce32680 0x55ca4ceaaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f985259c640 1 --2- 172.21.15.103:0/807564722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce274a0 0x55ca4ce35d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f9849d9b640 1 -- 172.21.15.103:0/807564722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce274a0 msgr2=0x55ca4ce35d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:19.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f9849d9b640 1 --2- 172.21.15.103:0/807564722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce274a0 0x55ca4ce35d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f9849d9b640 1 -- 172.21.15.103:0/807564722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce32680 msgr2=0x55ca4ceaaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:19.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f9849d9b640 1 --2- 172.21.15.103:0/807564722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce32680 0x55ca4ceaaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f9849d9b640 1 -- 172.21.15.103:0/807564722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca4ceab0b0 con 0x55ca4ce27dc0 2024-09-19T06:51:19.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f9849d9b640 1 --2- 172.21.15.103:0/807564722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce362a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x55ca4ce30890 tx=0x7f983c004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4ffaf2708a6b68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:19.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.459+0000 7f9839ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:19.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.460+0000 7f9839ffb640 1 -- 172.21.15.103:0/807564722 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c030070 con 0x55ca4ce27dc0 2024-09-19T06:51:19.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.460+0000 7f985b21f740 1 -- 172.21.15.103:0/807564722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98440a46f0 con 0x55ca4ce27dc0 2024-09-19T06:51:19.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.460+0000 7f9839ffb640 1 -- 172.21.15.103:0/807564722 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f983c010a20 con 0x55ca4ce27dc0 2024-09-19T06:51:19.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.460+0000 7f9839ffb640 1 -- 172.21.15.103:0/807564722 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f983c014030 con 0x55ca4ce27dc0 2024-09-19T06:51:19.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.460+0000 7f985b21f740 10 client.16224 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:19.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.460+0000 7f985b21f740 1 -- 172.21.15.103:0/807564722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ca4ceac3a0 con 0x55ca4ce27dc0 2024-09-19T06:51:19.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.460+0000 7f9839ffb640 1 -- 172.21.15.103:0/807564722 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98440a46f0 con 0x55ca4ce27dc0 2024-09-19T06:51:19.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.460+0000 7f9839ffb640 1 -- 172.21.15.103:0/807564722 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f983c0035f0 con 0x55ca4ce27dc0 2024-09-19T06:51:19.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.460+0000 7f985b21f740 10 client.16224 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:19.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.460+0000 7f985b21f740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:19.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.460+0000 7f985b21f740 20 client.16224 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:19.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.460+0000 7f985b21f740 1 --2- 172.21.15.103:0/807564722 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ca4ceaca60 0x55ca4ceaee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:19.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.460+0000 7f985b21f740 4 client.16224 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:51:19.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.460+0000 7f985b21f740 1 -- 172.21.15.103:0/807564722 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ca4cd94d60 con 0x55ca4ceaca60 2024-09-19T06:51:19.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.460+0000 7f985259c640 1 --2- 172.21.15.103:0/807564722 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ca4ceaca60 0x55ca4ceaee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:19.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f9839ffb640 10 client.16224 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:19.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f985259c640 1 --2- 172.21.15.103:0/807564722 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ca4ceaca60 0x55ca4ceaee80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b88ffdb50880851 server_cookie=a77c3dc2fd875c5b in_seq=0 out_seq=0 2024-09-19T06:51:19.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f9839ffb640 1 -- 172.21.15.103:0/807564722 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ca4cd94d60 con 0x55ca4ceaca60 2024-09-19T06:51:19.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f9839ffb640 10 client.16224 handle_command_reply: tid=0 2024-09-19T06:51:19.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f985b21f740 1 client.16224 shutdown 2024-09-19T06:51:19.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f985b21f740 1 -- 172.21.15.103:0/807564722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce27dc0 msgr2=0x55ca4ce362a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:19.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f985b21f740 1 --2- 172.21.15.103:0/807564722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce362a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x55ca4ce30890 tx=0x7f983c004f00 comp rx=0 tx=0).stop 2024-09-19T06:51:19.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f985b21f740 1 -- 172.21.15.103:0/807564722 shutdown_connections 2024-09-19T06:51:19.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f985b21f740 1 --2- 172.21.15.103:0/807564722 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ca4ceaca60 0x55ca4ceaee80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f985b21f740 1 --2- 172.21.15.103:0/807564722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca4ce32680 0x55ca4ceaaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f985b21f740 1 --2- 172.21.15.103:0/807564722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca4ce27dc0 0x55ca4ce362a0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f9839ffb640 0 client.16224 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:19.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f985b21f740 1 --2- 172.21.15.103:0/807564722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca4ce274a0 0x55ca4ce35d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f985b21f740 1 -- 172.21.15.103:0/807564722 >> 172.21.15.103:0/807564722 conn(0x55ca4ce23a30 msgr2=0x55ca4ce22350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:19.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f985b21f740 1 -- 172.21.15.103:0/807564722 shutdown_connections 2024-09-19T06:51:19.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f985b21f740 1 -- 172.21.15.103:0/807564722 wait complete. 2024-09-19T06:51:19.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.461+0000 7f985b21f740 20 client.16224 trim_cache size 0 max 16384 2024-09-19T06:51:19.475 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:51:19.475 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T06:51:19.475 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T06:51:19.475 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:51:19.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.462+0000 7f9858e46640 1 -- 172.21.15.103:0/330779103 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f98400a33b0 msgr2=0x7f98400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:19.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.462+0000 7f9858e46640 1 --2- 172.21.15.103:0/330779103 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f98400a33b0 0x7f98400a5870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f984c009a60 tx=0x7f984c02fe10 comp rx=0 tx=0).stop 2024-09-19T06:51:19.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.462+0000 7f9858e46640 1 -- 172.21.15.103:0/330779103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9854102d40 msgr2=0x7f9854198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:19.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.462+0000 7f9858e46640 1 --2- 172.21.15.103:0/330779103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9854102d40 0x7f9854198390 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f984400d770 tx=0x7f984400dc30 comp rx=0 tx=0).stop 2024-09-19T06:51:19.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.462+0000 7f9858e46640 1 -- 172.21.15.103:0/330779103 shutdown_connections 2024-09-19T06:51:19.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.462+0000 7f9858e46640 1 --2- 172.21.15.103:0/330779103 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f98400a33b0 0x7f98400a5870 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.462+0000 7f9858e46640 1 --2- 172.21.15.103:0/330779103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f985410c170 0x7f985419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.462+0000 7f9858e46640 1 --2- 172.21.15.103:0/330779103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9854103660 0x7f98541988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.462+0000 7f9858e46640 1 --2- 172.21.15.103:0/330779103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9854102d40 0x7f9854198390 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:19.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.462+0000 7f9858e46640 1 -- 172.21.15.103:0/330779103 >> 172.21.15.103:0/330779103 conn(0x7f98540fc7a0 msgr2=0x7f98540fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:19.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.462+0000 7f9858e46640 1 -- 172.21.15.103:0/330779103 shutdown_connections 2024-09-19T06:51:19.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:19.462+0000 7f9858e46640 1 -- 172.21.15.103:0/330779103 wait complete. 2024-09-19T06:51:20.136 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:51:20.136 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:51:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:20 smithi174 ceph-mon[35637]: pgmap v66: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-09-19T06:51:20.382 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:20.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:20 smithi103 ceph-mon[31158]: pgmap v66: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-09-19T06:51:20.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:20 smithi134 ceph-mon[35356]: pgmap v66: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-09-19T06:51:20.470 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:51:20.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.821+0000 7f6f2893f640 1 -- 172.21.15.103:0/3502226406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f2410c1a0 msgr2=0x7f6f2410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:20.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.821+0000 7f6f2893f640 1 --2- 172.21.15.103:0/3502226406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f2410c1a0 0x7f6f2410e610 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c009ca0 tx=0x7f6f1c02a950 comp rx=0 tx=0).stop 2024-09-19T06:51:20.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.821+0000 7f6f2893f640 1 -- 172.21.15.103:0/3502226406 shutdown_connections 2024-09-19T06:51:20.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.821+0000 7f6f2893f640 1 --2- 172.21.15.103:0/3502226406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f2410c1a0 0x7f6f2410e610 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:20.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.821+0000 7f6f2893f640 1 --2- 172.21.15.103:0/3502226406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f24103720 0x7f6f2410bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:20.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.821+0000 7f6f2893f640 1 --2- 172.21.15.103:0/3502226406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f24102e00 0x7f6f241031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:20.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.821+0000 7f6f2893f640 1 -- 172.21.15.103:0/3502226406 >> 172.21.15.103:0/3502226406 conn(0x7f6f240fc7a0 msgr2=0x7f6f240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:20.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.821+0000 7f6f2893f640 1 -- 172.21.15.103:0/3502226406 shutdown_connections 2024-09-19T06:51:20.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.821+0000 7f6f2893f640 1 -- 172.21.15.103:0/3502226406 wait complete. 2024-09-19T06:51:20.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f2893f640 1 Processor -- start 2024-09-19T06:51:20.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f2893f640 1 -- start start 2024-09-19T06:51:20.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f2893f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f24102e00 0x7f6f241983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:20.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f2893f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f24103720 0x7f6f24198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:20.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f2893f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f2410c1a0 0x7f6f2419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:20.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f2893f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6f24111f50 con 0x7f6f24103720 2024-09-19T06:51:20.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f2893f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f24111dd0 con 0x7f6f24102e00 2024-09-19T06:51:20.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f2893f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6f241120d0 con 0x7f6f2410c1a0 2024-09-19T06:51:20.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f21d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f24103720 0x7f6f24198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:20.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f21d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f24103720 0x7f6f24198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46282/0 (socket says 172.21.15.103:46282) 2024-09-19T06:51:20.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f21d9b640 1 -- 172.21.15.103:0/1834879289 learned_addr learned my addr 172.21.15.103:0/1834879289 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:20.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f2259c640 1 --2- 172.21.15.103:0/1834879289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f24102e00 0x7f6f241983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:20.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f22d9d640 1 --2- 172.21.15.103:0/1834879289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f2410c1a0 0x7f6f2419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:20.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f21d9b640 1 -- 172.21.15.103:0/1834879289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f2410c1a0 msgr2=0x7f6f2419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:20.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f21d9b640 1 --2- 172.21.15.103:0/1834879289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f2410c1a0 0x7f6f2419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:20.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f21d9b640 1 -- 172.21.15.103:0/1834879289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f24102e00 msgr2=0x7f6f241983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:20.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f21d9b640 1 --2- 172.21.15.103:0/1834879289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f24102e00 0x7f6f241983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:20.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f21d9b640 1 -- 172.21.15.103:0/1834879289 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f2419d3a0 con 0x7f6f24103720 2024-09-19T06:51:20.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f2259c640 1 --2- 172.21.15.103:0/1834879289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f24102e00 0x7f6f241983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:20.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.822+0000 7f6f22d9d640 1 --2- 172.21.15.103:0/1834879289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f2410c1a0 0x7f6f2419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:20.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.823+0000 7f6f21d9b640 1 --2- 172.21.15.103:0/1834879289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f24103720 0x7f6f24198930 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00d770 tx=0x7f6f0c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:20.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.823+0000 7f6f1b7fe640 1 -- 172.21.15.103:0/1834879289 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c017070 con 0x7f6f24103720 2024-09-19T06:51:20.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.823+0000 7f6f2893f640 1 -- 172.21.15.103:0/1834879289 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f2419d690 con 0x7f6f24103720 2024-09-19T06:51:20.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.823+0000 7f6f1b7fe640 1 -- 172.21.15.103:0/1834879289 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f0c005500 con 0x7f6f24103720 2024-09-19T06:51:20.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.823+0000 7f6f1b7fe640 1 -- 172.21.15.103:0/1834879289 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c004e00 con 0x7f6f24103720 2024-09-19T06:51:20.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.823+0000 7f6f2893f640 1 -- 172.21.15.103:0/1834879289 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f241a4f70 con 0x7f6f24103720 2024-09-19T06:51:20.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.824+0000 7f6f2893f640 1 -- 172.21.15.103:0/1834879289 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f241031e0 con 0x7f6f24103720 2024-09-19T06:51:20.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.825+0000 7f6f1b7fe640 1 -- 172.21.15.103:0/1834879289 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f6f0c004fa0 con 0x7f6f24103720 2024-09-19T06:51:20.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.826+0000 7f6f1b7fe640 1 --2- 172.21.15.103:0/1834879289 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6f100a33b0 0x7f6f100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:20.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.826+0000 7f6f2259c640 1 --2- 172.21.15.103:0/1834879289 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6f100a33b0 0x7f6f100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:20.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.826+0000 7f6f1b7fe640 1 -- 172.21.15.103:0/1834879289 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6f0c0cf780 con 0x7f6f24103720 2024-09-19T06:51:20.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.826+0000 7f6f2259c640 1 --2- 172.21.15.103:0/1834879289 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6f100a33b0 0x7f6f100a5870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009b40 tx=0x7f6f14005e50 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:20.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:20.830+0000 7f6f1b7fe640 1 -- 172.21.15.103:0/1834879289 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6f0c09c170 con 0x7f6f24103720 2024-09-19T06:51:20.905 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:21.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:21.044+0000 7f6f2893f640 1 -- 172.21.15.103:0/1834879289 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f241072b0 con 0x7f6f24103720 2024-09-19T06:51:21.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:21.045+0000 7f6f1b7fe640 1 -- 172.21.15.103:0/1834879289 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f6f0c09ba70 con 0x7f6f24103720 2024-09-19T06:51:21.047 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:51:21.048 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:51:21.048 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:51:21.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:21.047+0000 7f6f2893f640 1 -- 172.21.15.103:0/1834879289 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6f100a33b0 msgr2=0x7f6f100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:21.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:21.047+0000 7f6f2893f640 1 --2- 172.21.15.103:0/1834879289 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6f100a33b0 0x7f6f100a5870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009b40 tx=0x7f6f14005e50 comp rx=0 tx=0).stop 2024-09-19T06:51:21.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:21.047+0000 7f6f2893f640 1 -- 172.21.15.103:0/1834879289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f24103720 msgr2=0x7f6f24198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:21.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:21.047+0000 7f6f2893f640 1 --2- 172.21.15.103:0/1834879289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f24103720 0x7f6f24198930 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00d770 tx=0x7f6f0c00dc30 comp rx=0 tx=0).stop 2024-09-19T06:51:21.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:21.047+0000 7f6f2893f640 1 -- 172.21.15.103:0/1834879289 shutdown_connections 2024-09-19T06:51:21.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:21.047+0000 7f6f2893f640 1 --2- 172.21.15.103:0/1834879289 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6f100a33b0 0x7f6f100a5870 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:21.047+0000 7f6f2893f640 1 --2- 172.21.15.103:0/1834879289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f2410c1a0 0x7f6f2419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:21.047+0000 7f6f2893f640 1 --2- 172.21.15.103:0/1834879289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f24103720 0x7f6f24198930 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:21.047+0000 7f6f2893f640 1 --2- 172.21.15.103:0/1834879289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f24102e00 0x7f6f241983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:21.047+0000 7f6f2893f640 1 -- 172.21.15.103:0/1834879289 >> 172.21.15.103:0/1834879289 conn(0x7f6f240fc7a0 msgr2=0x7f6f240fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:21.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:21.047+0000 7f6f2893f640 1 -- 172.21.15.103:0/1834879289 shutdown_connections 2024-09-19T06:51:21.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:21.047+0000 7f6f2893f640 1 -- 172.21.15.103:0/1834879289 wait complete. 2024-09-19T06:51:21.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:21 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1834879289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:51:21.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:21 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1834879289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:51:21.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:21 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1834879289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:51:21.846 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:51:22.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.076+0000 7f8038e8d640 1 -- 172.21.15.103:0/3553193330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8034102790 msgr2=0x7f80341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:22.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.076+0000 7f8038e8d640 1 --2- 172.21.15.103:0/3553193330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8034102790 0x7f80341067e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f801c009900 tx=0x7f801c02a410 comp rx=0 tx=0).stop 2024-09-19T06:51:22.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.076+0000 7f8038e8d640 1 -- 172.21.15.103:0/3553193330 shutdown_connections 2024-09-19T06:51:22.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.076+0000 7f8038e8d640 1 --2- 172.21.15.103:0/3553193330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8034106f10 0x7f803410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:22.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.076+0000 7f8038e8d640 1 --2- 172.21.15.103:0/3553193330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8034102790 0x7f80341067e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:22.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.076+0000 7f8038e8d640 1 --2- 172.21.15.103:0/3553193330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8034101de0 0x7f80341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:22.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.076+0000 7f8038e8d640 1 -- 172.21.15.103:0/3553193330 >> 172.21.15.103:0/3553193330 conn(0x7f80340fd8c0 msgr2=0x7f80340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:22.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.076+0000 7f8038e8d640 1 -- 172.21.15.103:0/3553193330 shutdown_connections 2024-09-19T06:51:22.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.076+0000 7f8038e8d640 1 -- 172.21.15.103:0/3553193330 wait complete. 2024-09-19T06:51:22.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.077+0000 7f8038e8d640 1 Processor -- start 2024-09-19T06:51:22.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.077+0000 7f8038e8d640 1 -- start start 2024-09-19T06:51:22.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.077+0000 7f8038e8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8034101de0 0x7f80340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:22.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.077+0000 7f8038e8d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8034102790 0x7f8034074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:22.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.077+0000 7f8038e8d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8034106f10 0x7f8034074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:22.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.077+0000 7f8038e8d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8034111fb0 con 0x7f8034101de0 2024-09-19T06:51:22.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.077+0000 7f8038e8d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8034111e30 con 0x7f8034106f10 2024-09-19T06:51:22.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.077+0000 7f8038e8d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8034112130 con 0x7f8034102790 2024-09-19T06:51:22.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.077+0000 7f803259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8034101de0 0x7f80340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:22.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.077+0000 7f803259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8034101de0 0x7f80340744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46300/0 (socket says 172.21.15.103:46300) 2024-09-19T06:51:22.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.077+0000 7f803259c640 1 -- 172.21.15.103:0/3902287948 learned_addr learned my addr 172.21.15.103:0/3902287948 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:22.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.078+0000 7f8032d9d640 1 --2- 172.21.15.103:0/3902287948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8034106f10 0x7f8034074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:22.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.078+0000 7f8029d9b640 1 --2- 172.21.15.103:0/3902287948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8034102790 0x7f8034074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:22.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.078+0000 7f803259c640 1 -- 172.21.15.103:0/3902287948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8034102790 msgr2=0x7f8034074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:22.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.078+0000 7f803259c640 1 --2- 172.21.15.103:0/3902287948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8034102790 0x7f8034074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:22.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.078+0000 7f803259c640 1 -- 172.21.15.103:0/3902287948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8034106f10 msgr2=0x7f8034074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:22.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.078+0000 7f803259c640 1 --2- 172.21.15.103:0/3902287948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8034106f10 0x7f8034074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:22.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.078+0000 7f803259c640 1 -- 172.21.15.103:0/3902287948 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80341a6970 con 0x7f8034101de0 2024-09-19T06:51:22.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.078+0000 7f8029d9b640 1 --2- 172.21.15.103:0/3902287948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8034102790 0x7f8034074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:22.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.078+0000 7f803259c640 1 --2- 172.21.15.103:0/3902287948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8034101de0 0x7f80340744f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f802400d770 tx=0x7f802400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:22.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.078+0000 7f802bfff640 1 -- 172.21.15.103:0/3902287948 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024017070 con 0x7f8034101de0 2024-09-19T06:51:22.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.078+0000 7f802bfff640 1 -- 172.21.15.103:0/3902287948 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8024005500 con 0x7f8034101de0 2024-09-19T06:51:22.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.078+0000 7f802bfff640 1 -- 172.21.15.103:0/3902287948 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024004e00 con 0x7f8034101de0 2024-09-19T06:51:22.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.078+0000 7f8038e8d640 1 -- 172.21.15.103:0/3902287948 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80341a6c60 con 0x7f8034101de0 2024-09-19T06:51:22.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.078+0000 7f8038e8d640 1 -- 172.21.15.103:0/3902287948 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80341a71a0 con 0x7f8034101de0 2024-09-19T06:51:22.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.079+0000 7f8038e8d640 1 -- 172.21.15.103:0/3902287948 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80341021c0 con 0x7f8034101de0 2024-09-19T06:51:22.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.080+0000 7f802bfff640 1 -- 172.21.15.103:0/3902287948 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8024004fa0 con 0x7f8034101de0 2024-09-19T06:51:22.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.081+0000 7f802bfff640 1 --2- 172.21.15.103:0/3902287948 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80200a3430 0x7f80200a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:22.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.081+0000 7f8029d9b640 1 --2- 172.21.15.103:0/3902287948 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80200a3430 0x7f80200a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:22.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.082+0000 7f802bfff640 1 -- 172.21.15.103:0/3902287948 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f80240cfb50 con 0x7f8034101de0 2024-09-19T06:51:22.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.082+0000 7f8029d9b640 1 --2- 172.21.15.103:0/3902287948 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80200a3430 0x7f80200a58f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8034075f80 tx=0x7f801c005a20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:22.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.085+0000 7f802bfff640 1 -- 172.21.15.103:0/3902287948 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f802409c580 con 0x7f8034101de0 2024-09-19T06:51:22.174 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:22.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.298+0000 7f8038e8d640 1 -- 172.21.15.103:0/3902287948 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f803410a150 con 0x7f8034101de0 2024-09-19T06:51:22.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.299+0000 7f802bfff640 1 -- 172.21.15.103:0/3902287948 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f802409be80 con 0x7f8034101de0 2024-09-19T06:51:22.301 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:51:22.301 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:51:22.301 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:51:22.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.301+0000 7f8038e8d640 1 -- 172.21.15.103:0/3902287948 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80200a3430 msgr2=0x7f80200a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:22.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.301+0000 7f8038e8d640 1 --2- 172.21.15.103:0/3902287948 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80200a3430 0x7f80200a58f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8034075f80 tx=0x7f801c005a20 comp rx=0 tx=0).stop 2024-09-19T06:51:22.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.301+0000 7f8038e8d640 1 -- 172.21.15.103:0/3902287948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8034101de0 msgr2=0x7f80340744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:22.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.301+0000 7f8038e8d640 1 --2- 172.21.15.103:0/3902287948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8034101de0 0x7f80340744f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f802400d770 tx=0x7f802400dc30 comp rx=0 tx=0).stop 2024-09-19T06:51:22.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.301+0000 7f8038e8d640 1 -- 172.21.15.103:0/3902287948 shutdown_connections 2024-09-19T06:51:22.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.301+0000 7f8038e8d640 1 --2- 172.21.15.103:0/3902287948 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80200a3430 0x7f80200a58f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:22.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.301+0000 7f8038e8d640 1 --2- 172.21.15.103:0/3902287948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8034106f10 0x7f8034074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:22.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.301+0000 7f8038e8d640 1 --2- 172.21.15.103:0/3902287948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8034102790 0x7f8034074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:22.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.301+0000 7f8038e8d640 1 --2- 172.21.15.103:0/3902287948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8034101de0 0x7f80340744f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:22.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.301+0000 7f8038e8d640 1 -- 172.21.15.103:0/3902287948 >> 172.21.15.103:0/3902287948 conn(0x7f80340fd8c0 msgr2=0x7f80340feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:22.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.301+0000 7f8038e8d640 1 -- 172.21.15.103:0/3902287948 shutdown_connections 2024-09-19T06:51:22.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:22.301+0000 7f8038e8d640 1 -- 172.21.15.103:0/3902287948 wait complete. 2024-09-19T06:51:22.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:22 smithi103 ceph-mon[31158]: pgmap v67: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-19T06:51:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:22 smithi134 ceph-mon[35356]: pgmap v67: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-19T06:51:22.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:22 smithi174 ceph-mon[35637]: pgmap v67: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-19T06:51:23.080 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id ee303a01 --release --await' 2024-09-19T06:51:23.080 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id ee303a01 --release --await 2024-09-19T06:51:23.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.277+0000 7fbc00cbd640 1 -- 172.21.15.103:0/4202350508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbfc102d30 msgr2=0x7fbbfc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.277+0000 7fbc00cbd640 1 --2- 172.21.15.103:0/4202350508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbfc102d30 0x7fbbfc103110 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009930 tx=0x7fbbec02a440 comp rx=0 tx=0).stop 2024-09-19T06:51:23.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.277+0000 7fbc00cbd640 1 -- 172.21.15.103:0/4202350508 shutdown_connections 2024-09-19T06:51:23.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.277+0000 7fbc00cbd640 1 --2- 172.21.15.103:0/4202350508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbfc1082a0 0x7fbbfc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.277+0000 7fbc00cbd640 1 --2- 172.21.15.103:0/4202350508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbfc1036e0 0x7fbbfc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.277+0000 7fbc00cbd640 1 --2- 172.21.15.103:0/4202350508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbfc102d30 0x7fbbfc103110 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.277+0000 7fbc00cbd640 1 -- 172.21.15.103:0/4202350508 >> 172.21.15.103:0/4202350508 conn(0x7fbbfc0fc7d0 msgr2=0x7fbbfc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:23.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.277+0000 7fbc00cbd640 1 -- 172.21.15.103:0/4202350508 shutdown_connections 2024-09-19T06:51:23.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.277+0000 7fbc00cbd640 1 -- 172.21.15.103:0/4202350508 wait complete. 2024-09-19T06:51:23.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbc00cbd640 1 Processor -- start 2024-09-19T06:51:23.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbc00cbd640 1 -- start start 2024-09-19T06:51:23.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbc00cbd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbfc102d30 0x7fbbfc198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbc00cbd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbfc1036e0 0x7fbbfc198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbc00cbd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbfc1082a0 0x7fbbfc19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbc00cbd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc112160 con 0x7fbbfc1036e0 2024-09-19T06:51:23.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbc00cbd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc111fe0 con 0x7fbbfc1082a0 2024-09-19T06:51:23.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbc00cbd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc1122e0 con 0x7fbbfc102d30 2024-09-19T06:51:23.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbbf3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbfc1036e0 0x7fbbfc198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbbf3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbfc1036e0 0x7fbbfc198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46314/0 (socket says 172.21.15.103:46314) 2024-09-19T06:51:23.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbbf3fff640 1 -- 172.21.15.103:0/4132502261 learned_addr learned my addr 172.21.15.103:0/4132502261 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:23.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbbfa59c640 1 --2- 172.21.15.103:0/4132502261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbfc102d30 0x7fbbfc198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbbfad9d640 1 --2- 172.21.15.103:0/4132502261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbfc1082a0 0x7fbbfc19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbbf3fff640 1 -- 172.21.15.103:0/4132502261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbfc102d30 msgr2=0x7fbbfc198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbbf3fff640 1 --2- 172.21.15.103:0/4132502261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbfc102d30 0x7fbbfc198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbbf3fff640 1 -- 172.21.15.103:0/4132502261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbfc1082a0 msgr2=0x7fbbfc19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbbf3fff640 1 --2- 172.21.15.103:0/4132502261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbfc1082a0 0x7fbbfc19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbbf3fff640 1 -- 172.21.15.103:0/4132502261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbfc19d6d0 con 0x7fbbfc1036e0 2024-09-19T06:51:23.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.278+0000 7fbbfad9d640 1 --2- 172.21.15.103:0/4132502261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbfc1082a0 0x7fbbfc19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:23.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.279+0000 7fbbf3fff640 1 --2- 172.21.15.103:0/4132502261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbfc1036e0 0x7fbbfc198c60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4006fd0 tx=0x7fbbe400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:23.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.279+0000 7fbbf37fe640 1 -- 172.21.15.103:0/4132502261 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4011830 con 0x7fbbfc1036e0 2024-09-19T06:51:23.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.279+0000 7fbbf37fe640 1 -- 172.21.15.103:0/4132502261 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbe4016070 con 0x7fbbfc1036e0 2024-09-19T06:51:23.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.279+0000 7fbbf37fe640 1 -- 172.21.15.103:0/4132502261 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe400f2e0 con 0x7fbbfc1036e0 2024-09-19T06:51:23.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.279+0000 7fbc00cbd640 1 -- 172.21.15.103:0/4132502261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbfc19d9c0 con 0x7fbbfc1036e0 2024-09-19T06:51:23.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.279+0000 7fbc00cbd640 1 -- 172.21.15.103:0/4132502261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbfc1a52a0 con 0x7fbbfc1036e0 2024-09-19T06:51:23.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.281+0000 7fbbf37fe640 1 -- 172.21.15.103:0/4132502261 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fbbe401b460 con 0x7fbbfc1036e0 2024-09-19T06:51:23.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.281+0000 7fbc03096740 1 Processor -- start 2024-09-19T06:51:23.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.282+0000 7fbc03096740 1 -- start start 2024-09-19T06:51:23.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.282+0000 7fbc03096740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981d0b40 0x55eb981cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.282+0000 7fbc03096740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981cf5d0 0x55eb981cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.282+0000 7fbc03096740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981da370 0x55eb981dc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.282+0000 7fbc03096740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eb981352f0 con 0x55eb981d0b40 2024-09-19T06:51:23.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.282+0000 7fbc03096740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55eb980dccf0 con 0x55eb981cf5d0 2024-09-19T06:51:23.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.282+0000 7fbc03096740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eb981312b0 con 0x55eb981da370 2024-09-19T06:51:23.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.282+0000 7fbbfad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981d0b40 0x55eb981cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.282+0000 7fbbfad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981d0b40 0x55eb981cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46330/0 (socket says 172.21.15.103:46330) 2024-09-19T06:51:23.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.282+0000 7fbbfad9d640 1 -- 172.21.15.103:0/910592556 learned_addr learned my addr 172.21.15.103:0/910592556 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:23.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.282+0000 7fbbf37fe640 1 --2- 172.21.15.103:0/4132502261 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbbe80a33b0 0x7fbbe80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.282+0000 7fbbfa59c640 1 --2- 172.21.15.103:0/910592556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981cf5d0 0x55eb981cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.282+0000 7fbbf3fff640 1 --2- 172.21.15.103:0/910592556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981da370 0x55eb981dc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.282+0000 7fbbfad9d640 1 --2- 172.21.15.103:0/4132502261 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbbe80a33b0 0x7fbbe80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.282+0000 7fbbf37fe640 1 -- 172.21.15.103:0/4132502261 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fbbe40cf290 con 0x7fbbfc1036e0 2024-09-19T06:51:23.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.283+0000 7fbbfa59c640 1 -- 172.21.15.103:0/910592556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981da370 msgr2=0x55eb981dc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:23.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.283+0000 7fbbfa59c640 1 --2- 172.21.15.103:0/910592556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981da370 0x55eb981dc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.283+0000 7fbbfa59c640 1 -- 172.21.15.103:0/910592556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981d0b40 msgr2=0x55eb981cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.283+0000 7fbbfa59c640 1 --2- 172.21.15.103:0/910592556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981d0b40 0x55eb981cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.283+0000 7fbbfa59c640 1 -- 172.21.15.103:0/910592556 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb980c13a0 con 0x55eb981cf5d0 2024-09-19T06:51:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.283+0000 7fbbfad9d640 1 --2- 172.21.15.103:0/910592556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981d0b40 0x55eb981cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.283+0000 7fbbf3fff640 1 --2- 172.21.15.103:0/910592556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981da370 0x55eb981dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.283+0000 7fbbfad9d640 1 --2- 172.21.15.103:0/4132502261 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbbe80a33b0 0x7fbbe80a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4010e60 tx=0x7fbbf4004a40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.283+0000 7fbbfa59c640 1 --2- 172.21.15.103:0/910592556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981cf5d0 0x55eb981cfa10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbbec005940 tx=0x7fbbec0316e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92496473d7a0c721 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.283+0000 7fbbf0ff9640 1 -- 172.21.15.103:0/910592556 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec040070 con 0x55eb981cf5d0 2024-09-19T06:51:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.283+0000 7fbbf0ff9640 1 -- 172.21.15.103:0/910592556 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbec005500 con 0x55eb981cf5d0 2024-09-19T06:51:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.283+0000 7fbbf0ff9640 1 -- 172.21.15.103:0/910592556 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec035040 con 0x55eb981cf5d0 2024-09-19T06:51:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.284+0000 7fbc03096740 1 -- 172.21.15.103:0/910592556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981cf5d0 msgr2=0x55eb981cfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.284+0000 7fbc03096740 1 --2- 172.21.15.103:0/910592556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981cf5d0 0x55eb981cfa10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbbec005940 tx=0x7fbbec0316e0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.284+0000 7fbc03096740 1 -- 172.21.15.103:0/910592556 shutdown_connections 2024-09-19T06:51:23.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.284+0000 7fbc03096740 1 --2- 172.21.15.103:0/910592556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981da370 0x55eb981dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.284+0000 7fbc03096740 1 --2- 172.21.15.103:0/910592556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981cf5d0 0x55eb981cfa10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.284+0000 7fbc03096740 1 --2- 172.21.15.103:0/910592556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981d0b40 0x55eb981cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.284+0000 7fbc03096740 1 -- 172.21.15.103:0/910592556 >> 172.21.15.103:0/910592556 conn(0x55eb98198310 msgr2=0x55eb981cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:23.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.284+0000 7fbc03096740 1 -- 172.21.15.103:0/910592556 shutdown_connections 2024-09-19T06:51:23.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.284+0000 7fbc03096740 1 -- 172.21.15.103:0/910592556 wait complete. 2024-09-19T06:51:23.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.284+0000 7fbc03096740 1 -- start start 2024-09-19T06:51:23.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.284+0000 7fbc03096740 1 Processor -- start 2024-09-19T06:51:23.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbc03096740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981cf5d0 0x55eb981d9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbc03096740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981d0b40 0x55eb981d6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbc03096740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981da370 0x55eb981d7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbc03096740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eb981dd010 con 0x55eb981d0b40 2024-09-19T06:51:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbc03096740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55eb980f0060 con 0x55eb981da370 2024-09-19T06:51:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbc03096740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eb981dd190 con 0x55eb981cf5d0 2024-09-19T06:51:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbc03096740 10 client.0 _ll_register_callbacks cb 0x55eb97e438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbbf3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981d0b40 0x55eb981d6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbbf3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981d0b40 0x55eb981d6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46340/0 (socket says 172.21.15.103:46340) 2024-09-19T06:51:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbbf3fff640 1 -- 172.21.15.103:0/864730508 learned_addr learned my addr 172.21.15.103:0/864730508 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbbfa59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981cf5d0 0x55eb981d9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbbfad9d640 1 --2- 172.21.15.103:0/864730508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981da370 0x55eb981d7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbbf3fff640 1 -- 172.21.15.103:0/864730508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981cf5d0 msgr2=0x55eb981d9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:23.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbbf3fff640 1 --2- 172.21.15.103:0/864730508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981cf5d0 0x55eb981d9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbbf3fff640 1 -- 172.21.15.103:0/864730508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981da370 msgr2=0x55eb981d7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:23.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbbf3fff640 1 --2- 172.21.15.103:0/864730508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981da370 0x55eb981d7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbbf3fff640 1 -- 172.21.15.103:0/864730508 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb97ea63c0 con 0x55eb981d0b40 2024-09-19T06:51:23.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbbfad9d640 1 --2- 172.21.15.103:0/864730508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981da370 0x55eb981d7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:23.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbbf3fff640 1 --2- 172.21.15.103:0/864730508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981d0b40 0x55eb981d6fc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4098060 tx=0x7fbbe406a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d35c7497688a48c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:23.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.285+0000 7fbbe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:23.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.286+0000 7fbbe1ffb640 1 -- 172.21.15.103:0/864730508 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4010ec0 con 0x55eb981d0b40 2024-09-19T06:51:23.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.286+0000 7fbbe1ffb640 1 -- 172.21.15.103:0/864730508 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fbbe40ccc70 con 0x55eb981d0b40 2024-09-19T06:51:23.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.286+0000 7fbc03096740 1 -- 172.21.15.103:0/864730508 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbec005500 con 0x55eb981d0b40 2024-09-19T06:51:23.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.286+0000 7fbbe1ffb640 1 -- 172.21.15.103:0/864730508 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbe40cce70 con 0x55eb981d0b40 2024-09-19T06:51:23.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.286+0000 7fbc03096740 10 client.16248 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:23.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.286+0000 7fbc03096740 1 -- 172.21.15.103:0/864730508 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eb981e8a60 con 0x55eb981d0b40 2024-09-19T06:51:23.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.286+0000 7fbbe1ffb640 1 -- 172.21.15.103:0/864730508 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbec005500 con 0x55eb981d0b40 2024-09-19T06:51:23.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.286+0000 7fbbe1ffb640 1 -- 172.21.15.103:0/864730508 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbbe40a3e70 con 0x55eb981d0b40 2024-09-19T06:51:23.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.286+0000 7fbc03096740 10 client.16248 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:23.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.286+0000 7fbc03096740 10 client.16248 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:51:23.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.286+0000 7fbc03096740 20 client.16248 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:23.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.286+0000 7fbc03096740 1 --2- 172.21.15.103:0/864730508 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55eb981e9e70 0x55eb981ec260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.286+0000 7fbc03096740 4 client.16248 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:23.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.286+0000 7fbc03096740 1 -- 172.21.15.103:0/864730508 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb98142860 con 0x55eb981e9e70 2024-09-19T06:51:23.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.287+0000 7fbbfa59c640 1 --2- 172.21.15.103:0/864730508 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55eb981e9e70 0x55eb981ec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.287+0000 7fbbfa59c640 1 --2- 172.21.15.103:0/864730508 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55eb981e9e70 0x55eb981ec260 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=15b923d004d8220f server_cookie=2697114d758653c9 in_seq=0 out_seq=0 2024-09-19T06:51:23.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.287+0000 7fbbe1ffb640 10 client.16248 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:51:23.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.288+0000 7fbbe1ffb640 1 -- 172.21.15.103:0/864730508 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb98142860 con 0x55eb981e9e70 2024-09-19T06:51:23.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.288+0000 7fbbe1ffb640 10 client.16248 handle_command_reply: tid=0 2024-09-19T06:51:23.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.288+0000 7fbc03096740 1 client.16248 shutdown 2024-09-19T06:51:23.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.288+0000 7fbc03096740 1 -- 172.21.15.103:0/864730508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981d0b40 msgr2=0x55eb981d6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.288+0000 7fbc03096740 1 --2- 172.21.15.103:0/864730508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981d0b40 0x55eb981d6fc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4098060 tx=0x7fbbe406a380 comp rx=0 tx=0).stop 2024-09-19T06:51:23.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.288+0000 7fbc03096740 1 -- 172.21.15.103:0/864730508 shutdown_connections 2024-09-19T06:51:23.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.288+0000 7fbc03096740 1 --2- 172.21.15.103:0/864730508 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55eb981e9e70 0x55eb981ec260 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.288+0000 7fbbe1ffb640 0 client.16248 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:51:23.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.288+0000 7fbc03096740 1 --2- 172.21.15.103:0/864730508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981da370 0x55eb981d7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.288+0000 7fbc03096740 1 --2- 172.21.15.103:0/864730508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981d0b40 0x55eb981d6fc0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.288+0000 7fbc03096740 1 --2- 172.21.15.103:0/864730508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981cf5d0 0x55eb981d9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.288+0000 7fbc03096740 1 -- 172.21.15.103:0/864730508 >> 172.21.15.103:0/864730508 conn(0x55eb98198310 msgr2=0x55eb981d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:23.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.288+0000 7fbc03096740 1 -- 172.21.15.103:0/864730508 shutdown_connections 2024-09-19T06:51:23.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.288+0000 7fbc03096740 1 -- 172.21.15.103:0/864730508 wait complete. 2024-09-19T06:51:23.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.288+0000 7fbc03096740 20 client.16248 trim_cache size 0 max 16384 2024-09-19T06:51:23.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.301+0000 7fbc03096740 1 Processor -- start 2024-09-19T06:51:23.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbc03096740 1 -- start start 2024-09-19T06:51:23.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:23 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3902287948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:51:23.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbc03096740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981cf5d0 0x55eb981cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbc03096740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981d0b40 0x55eb981e62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbc03096740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981da370 0x55eb981f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbc03096740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc1add90 con 0x55eb981da370 2024-09-19T06:51:23.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbc03096740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc1af8d0 con 0x55eb981d0b40 2024-09-19T06:51:23.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbc03096740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc1ae460 con 0x55eb981cf5d0 2024-09-19T06:51:23.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbbfad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981da370 0x55eb981f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbbfad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981da370 0x55eb981f1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46346/0 (socket says 172.21.15.103:46346) 2024-09-19T06:51:23.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbbfad9d640 1 -- 172.21.15.103:0/688084820 learned_addr learned my addr 172.21.15.103:0/688084820 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:23.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbbf3fff640 1 --2- 172.21.15.103:0/688084820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981d0b40 0x55eb981e62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbbfa59c640 1 --2- 172.21.15.103:0/688084820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981cf5d0 0x55eb981cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbbfad9d640 1 -- 172.21.15.103:0/688084820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981cf5d0 msgr2=0x55eb981cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:23.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbbfad9d640 1 --2- 172.21.15.103:0/688084820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981cf5d0 0x55eb981cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbbfad9d640 1 -- 172.21.15.103:0/688084820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981d0b40 msgr2=0x55eb981e62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:23.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbbfad9d640 1 --2- 172.21.15.103:0/688084820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981d0b40 0x55eb981e62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.302+0000 7fbbfad9d640 1 -- 172.21.15.103:0/688084820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb981d2bc0 con 0x55eb981da370 2024-09-19T06:51:23.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.303+0000 7fbbfa59c640 1 --2- 172.21.15.103:0/688084820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981cf5d0 0x55eb981cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:23.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.303+0000 7fbbfad9d640 1 --2- 172.21.15.103:0/688084820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981da370 0x55eb981f1b30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4010bd0 tx=0x7fbbf4007960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694ee4ce4fe541cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:23.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.303+0000 7fbbe3fff640 1 -- 172.21.15.103:0/688084820 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf401f460 con 0x55eb981da370 2024-09-19T06:51:23.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.303+0000 7fbbe3fff640 1 -- 172.21.15.103:0/688084820 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf4019040 con 0x55eb981da370 2024-09-19T06:51:23.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.303+0000 7fbbe3fff640 1 -- 172.21.15.103:0/688084820 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf401e920 con 0x55eb981da370 2024-09-19T06:51:23.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.304+0000 7fbc03096740 1 -- 172.21.15.103:0/688084820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981da370 msgr2=0x55eb981f1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.304+0000 7fbc03096740 1 --2- 172.21.15.103:0/688084820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981da370 0x55eb981f1b30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4010bd0 tx=0x7fbbf4007960 comp rx=0 tx=0).stop 2024-09-19T06:51:23.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.304+0000 7fbc03096740 1 -- 172.21.15.103:0/688084820 shutdown_connections 2024-09-19T06:51:23.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.304+0000 7fbc03096740 1 --2- 172.21.15.103:0/688084820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981da370 0x55eb981f1b30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.304+0000 7fbc03096740 1 --2- 172.21.15.103:0/688084820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981d0b40 0x55eb981e62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.304+0000 7fbc03096740 1 --2- 172.21.15.103:0/688084820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981cf5d0 0x55eb981cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.304+0000 7fbc03096740 1 -- 172.21.15.103:0/688084820 >> 172.21.15.103:0/688084820 conn(0x55eb98198310 msgr2=0x55eb981cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:23.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.304+0000 7fbc03096740 1 -- 172.21.15.103:0/688084820 shutdown_connections 2024-09-19T06:51:23.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.305+0000 7fbc03096740 1 -- 172.21.15.103:0/688084820 wait complete. 2024-09-19T06:51:23.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.305+0000 7fbc03096740 1 -- start start 2024-09-19T06:51:23.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.305+0000 7fbc03096740 1 Processor -- start 2024-09-19T06:51:23.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.305+0000 7fbc03096740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981cf5d0 0x55eb981d3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.305+0000 7fbc03096740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981d0b40 0x55eb981d4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.305+0000 7fbc03096740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981da370 0x55eb981d4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.305+0000 7fbc03096740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc1af200 con 0x55eb981cf5d0 2024-09-19T06:51:23.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.305+0000 7fbc03096740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc1aeb30 con 0x55eb981da370 2024-09-19T06:51:23.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.305+0000 7fbc03096740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc1abd50 con 0x55eb981d0b40 2024-09-19T06:51:23.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.305+0000 7fbbfa59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981cf5d0 0x55eb981d3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.305+0000 7fbc03096740 10 client.0 _ll_register_callbacks cb 0x7fbbe4013f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:23.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.305+0000 7fbbfa59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981cf5d0 0x55eb981d3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46360/0 (socket says 172.21.15.103:46360) 2024-09-19T06:51:23.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.305+0000 7fbbfa59c640 1 -- 172.21.15.103:0/1722180750 learned_addr learned my addr 172.21.15.103:0/1722180750 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:23.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.305+0000 7fbbf3fff640 1 --2- 172.21.15.103:0/1722180750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981d0b40 0x55eb981d4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.306+0000 7fbbfad9d640 1 --2- 172.21.15.103:0/1722180750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981da370 0x55eb981d4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.306+0000 7fbbfa59c640 1 -- 172.21.15.103:0/1722180750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981d0b40 msgr2=0x55eb981d4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:23.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.306+0000 7fbbfa59c640 1 --2- 172.21.15.103:0/1722180750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981d0b40 0x55eb981d4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.306+0000 7fbbfa59c640 1 -- 172.21.15.103:0/1722180750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981da370 msgr2=0x55eb981d4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:23.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.306+0000 7fbbfa59c640 1 --2- 172.21.15.103:0/1722180750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981da370 0x55eb981d4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.306+0000 7fbbfa59c640 1 -- 172.21.15.103:0/1722180750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb98253b00 con 0x55eb981cf5d0 2024-09-19T06:51:23.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.306+0000 7fbbfa59c640 1 --2- 172.21.15.103:0/1722180750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981cf5d0 0x55eb981d3fd0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x55eb981ccac0 tx=0x7fbbec004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac831820f373612 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:23.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.306+0000 7fbbe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:23.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.306+0000 7fbbe1ffb640 1 -- 172.21.15.103:0/1722180750 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec05e020 con 0x55eb981cf5d0 2024-09-19T06:51:23.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.307+0000 7fbbe1ffb640 1 -- 172.21.15.103:0/1722180750 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fbbec004240 con 0x55eb981cf5d0 2024-09-19T06:51:23.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.307+0000 7fbc03096740 1 -- 172.21.15.103:0/1722180750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbf4019040 con 0x55eb981cf5d0 2024-09-19T06:51:23.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.307+0000 7fbbe1ffb640 1 -- 172.21.15.103:0/1722180750 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbec069050 con 0x55eb981cf5d0 2024-09-19T06:51:23.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.307+0000 7fbc03096740 10 client.16254 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:23.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.307+0000 7fbc03096740 1 -- 172.21.15.103:0/1722180750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eb98254d30 con 0x55eb981cf5d0 2024-09-19T06:51:23.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.307+0000 7fbbe1ffb640 1 -- 172.21.15.103:0/1722180750 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbec05ae20 con 0x55eb981cf5d0 2024-09-19T06:51:23.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.307+0000 7fbbe1ffb640 1 -- 172.21.15.103:0/1722180750 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbbec04d070 con 0x55eb981cf5d0 2024-09-19T06:51:23.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.307+0000 7fbc03096740 10 client.16254 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:23.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.307+0000 7fbc03096740 10 client.16254 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:51:23.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.307+0000 7fbc03096740 20 client.16254 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:23.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.307+0000 7fbc03096740 1 --2- 172.21.15.103:0/1722180750 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55eb98255030 0x55eb98257420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.307+0000 7fbc03096740 4 client.16254 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:51:23.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.307+0000 7fbc03096740 1 -- 172.21.15.103:0/1722180750 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fbbf4019040 con 0x55eb98255030 2024-09-19T06:51:23.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbbf3fff640 1 --2- 172.21.15.103:0/1722180750 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55eb98255030 0x55eb98257420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbbf3fff640 1 --2- 172.21.15.103:0/1722180750 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55eb98255030 0x55eb98257420 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=828589838ffa5bf0 server_cookie=e1f4969579c5c1a0 in_seq=0 out_seq=0 2024-09-19T06:51:23.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbbe1ffb640 10 client.16254 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:51:23.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbbe1ffb640 1 -- 172.21.15.103:0/1722180750 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fbbf4019040 con 0x55eb98255030 2024-09-19T06:51:23.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbbe1ffb640 10 client.16254 handle_command_reply: tid=0 2024-09-19T06:51:23.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbc03096740 1 client.16254 shutdown 2024-09-19T06:51:23.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbc03096740 1 -- 172.21.15.103:0/1722180750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981cf5d0 msgr2=0x55eb981d3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbc03096740 1 --2- 172.21.15.103:0/1722180750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981cf5d0 0x55eb981d3fd0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x55eb981ccac0 tx=0x7fbbec004da0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbc03096740 1 -- 172.21.15.103:0/1722180750 shutdown_connections 2024-09-19T06:51:23.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbc03096740 1 --2- 172.21.15.103:0/1722180750 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55eb98255030 0x55eb98257420 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbc03096740 1 --2- 172.21.15.103:0/1722180750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb981da370 0x55eb981d4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbc03096740 1 --2- 172.21.15.103:0/1722180750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb981d0b40 0x55eb981d4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbc03096740 1 --2- 172.21.15.103:0/1722180750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb981cf5d0 0x55eb981d3fd0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbc03096740 1 -- 172.21.15.103:0/1722180750 >> 172.21.15.103:0/1722180750 conn(0x55eb98198310 msgr2=0x55eb981d8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:23.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbbe1ffb640 0 client.16254 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:51:23.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbc03096740 1 -- 172.21.15.103:0/1722180750 shutdown_connections 2024-09-19T06:51:23.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.308+0000 7fbc03096740 1 -- 172.21.15.103:0/1722180750 wait complete. 2024-09-19T06:51:23.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.309+0000 7fbc03096740 20 client.16254 trim_cache size 0 max 16384 2024-09-19T06:51:23.315 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:51:23.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.309+0000 7fbc00cbd640 1 -- 172.21.15.103:0/4132502261 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbbe80a33b0 msgr2=0x7fbbe80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.309+0000 7fbc00cbd640 1 --2- 172.21.15.103:0/4132502261 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbbe80a33b0 0x7fbbe80a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4010e60 tx=0x7fbbf4004a40 comp rx=0 tx=0).stop 2024-09-19T06:51:23.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.309+0000 7fbc00cbd640 1 -- 172.21.15.103:0/4132502261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbfc1036e0 msgr2=0x7fbbfc198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.309+0000 7fbc00cbd640 1 --2- 172.21.15.103:0/4132502261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbfc1036e0 0x7fbbfc198c60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4006fd0 tx=0x7fbbe400b8e0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.309+0000 7fbc00cbd640 1 -- 172.21.15.103:0/4132502261 shutdown_connections 2024-09-19T06:51:23.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.309+0000 7fbc00cbd640 1 --2- 172.21.15.103:0/4132502261 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbbe80a33b0 0x7fbbe80a5870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.309+0000 7fbc00cbd640 1 --2- 172.21.15.103:0/4132502261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbfc1082a0 0x7fbbfc19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.309+0000 7fbc00cbd640 1 --2- 172.21.15.103:0/4132502261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbfc1036e0 0x7fbbfc198c60 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.309+0000 7fbc00cbd640 1 --2- 172.21.15.103:0/4132502261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbfc102d30 0x7fbbfc198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.309+0000 7fbc00cbd640 1 -- 172.21.15.103:0/4132502261 >> 172.21.15.103:0/4132502261 conn(0x7fbbfc0fc7d0 msgr2=0x7fbbfc105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:23.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.310+0000 7fbc00cbd640 1 -- 172.21.15.103:0/4132502261 shutdown_connections 2024-09-19T06:51:23.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:23.310+0000 7fbc00cbd640 1 -- 172.21.15.103:0/4132502261 wait complete. 2024-09-19T06:51:23.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:23 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3902287948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:51:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.559+0000 7fa3de6f1640 1 -- 172.21.15.103:0/4293954419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3d810c0c0 msgr2=0x7fa3d810e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.559+0000 7fa3de6f1640 1 --2- 172.21.15.103:0/4293954419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3d810c0c0 0x7fa3d810e580 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009ca0 tx=0x7fa3d002a950 comp rx=0 tx=0).stop 2024-09-19T06:51:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.560+0000 7fa3de6f1640 1 -- 172.21.15.103:0/4293954419 shutdown_connections 2024-09-19T06:51:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.560+0000 7fa3de6f1640 1 --2- 172.21.15.103:0/4293954419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3d810c0c0 0x7fa3d810e580 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.560+0000 7fa3de6f1640 1 --2- 172.21.15.103:0/4293954419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3d81035c0 0x7fa3d810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.560+0000 7fa3de6f1640 1 --2- 172.21.15.103:0/4293954419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3d8102ca0 0x7fa3d8103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.560+0000 7fa3de6f1640 1 -- 172.21.15.103:0/4293954419 >> 172.21.15.103:0/4293954419 conn(0x7fa3d80fc6f0 msgr2=0x7fa3d80feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.560+0000 7fa3de6f1640 1 -- 172.21.15.103:0/4293954419 shutdown_connections 2024-09-19T06:51:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.560+0000 7fa3de6f1640 1 -- 172.21.15.103:0/4293954419 wait complete. 2024-09-19T06:51:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.560+0000 7fa3de6f1640 1 Processor -- start 2024-09-19T06:51:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.560+0000 7fa3de6f1640 1 -- start start 2024-09-19T06:51:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.560+0000 7fa3de6f1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3d8102ca0 0x7fa3d81982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.560+0000 7fa3de6f1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3d81035c0 0x7fa3d81987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3de6f1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3d810c0c0 0x7fa3d819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3de6f1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d8111ee0 con 0x7fa3d810c0c0 2024-09-19T06:51:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3de6f1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d8111d60 con 0x7fa3d8102ca0 2024-09-19T06:51:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3de6f1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d8112060 con 0x7fa3d81035c0 2024-09-19T06:51:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3dcc8e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3d810c0c0 0x7fa3d819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3dcc8e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3d810c0c0 0x7fa3d819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46376/0 (socket says 172.21.15.103:46376) 2024-09-19T06:51:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3dcc8e640 1 -- 172.21.15.103:0/412355179 learned_addr learned my addr 172.21.15.103:0/412355179 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3d7fff640 1 --2- 172.21.15.103:0/412355179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3d8102ca0 0x7fa3d81982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3d77fe640 1 --2- 172.21.15.103:0/412355179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3d81035c0 0x7fa3d81987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3dcc8e640 1 -- 172.21.15.103:0/412355179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3d81035c0 msgr2=0x7fa3d81987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3dcc8e640 1 --2- 172.21.15.103:0/412355179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3d81035c0 0x7fa3d81987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3dcc8e640 1 -- 172.21.15.103:0/412355179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3d8102ca0 msgr2=0x7fa3d81982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3dcc8e640 1 --2- 172.21.15.103:0/412355179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3d8102ca0 0x7fa3d81982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3dcc8e640 1 -- 172.21.15.103:0/412355179 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d819d2a0 con 0x7fa3d810c0c0 2024-09-19T06:51:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3dcc8e640 1 --2- 172.21.15.103:0/412355179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3d810c0c0 0x7fa3d819cb70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0004af0 tx=0x7fa3d002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3d57fa640 1 -- 172.21.15.103:0/412355179 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0034040 con 0x7fa3d810c0c0 2024-09-19T06:51:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3de6f1640 1 -- 172.21.15.103:0/412355179 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d819d530 con 0x7fa3d810c0c0 2024-09-19T06:51:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3d57fa640 1 -- 172.21.15.103:0/412355179 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d0030920 con 0x7fa3d810c0c0 2024-09-19T06:51:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3d57fa640 1 -- 172.21.15.103:0/412355179 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0004670 con 0x7fa3d810c0c0 2024-09-19T06:51:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.561+0000 7fa3de6f1640 1 -- 172.21.15.103:0/412355179 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d81a4e20 con 0x7fa3d810c0c0 2024-09-19T06:51:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.564+0000 7fa3d57fa640 1 -- 172.21.15.103:0/412355179 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa3d0037020 con 0x7fa3d810c0c0 2024-09-19T06:51:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.564+0000 7fa3e1492740 1 Processor -- start 2024-09-19T06:51:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.564+0000 7fa3e1492740 1 -- start start 2024-09-19T06:51:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.564+0000 7fa3e1492740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157caaaf0 0x559157caaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.564+0000 7fa3e1492740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157cab410 0x559157cb5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.564+0000 7fa3e1492740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157cb80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.564+0000 7fa3e1492740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559157c0f2f0 con 0x559157cab410 2024-09-19T06:51:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.564+0000 7fa3e1492740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559157bb7270 con 0x559157cb5cb0 2024-09-19T06:51:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.564+0000 7fa3e1492740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559157c0b2b0 con 0x559157caaaf0 2024-09-19T06:51:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3d7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157cab410 0x559157cb5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3d7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157cab410 0x559157cb5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46382/0 (socket says 172.21.15.103:46382) 2024-09-19T06:51:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3d7fff640 1 -- 172.21.15.103:0/3351071116 learned_addr learned my addr 172.21.15.103:0/3351071116 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3d77fe640 1 --2- 172.21.15.103:0/3351071116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157cb80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3d77fe640 1 --2- 172.21.15.103:0/3351071116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157caaaf0 0x559157caaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3d57fa640 1 --2- 172.21.15.103:0/412355179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa3c00a3460 0x7fa3c00a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3d7fff640 1 -- 172.21.15.103:0/3351071116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157caaaf0 msgr2=0x559157caaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3d7fff640 1 --2- 172.21.15.103:0/3351071116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157caaaf0 0x559157caaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3d7fff640 1 -- 172.21.15.103:0/3351071116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 msgr2=0x559157cb80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3d7fff640 1 --2- 172.21.15.103:0/3351071116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157cb80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3d7fff640 1 -- 172.21.15.103:0/3351071116 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559157bb6340 con 0x559157cab410 2024-09-19T06:51:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3d77fe640 1 --2- 172.21.15.103:0/3351071116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157cb80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3dcc8e640 1 --2- 172.21.15.103:0/412355179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa3c00a3460 0x7fa3c00a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3d57fa640 1 -- 172.21.15.103:0/412355179 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa3d00f08a0 con 0x7fa3d810c0c0 2024-09-19T06:51:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3d7fff640 1 --2- 172.21.15.103:0/3351071116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157cab410 0x559157cb5770 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009cf0 tx=0x7fa3c8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d7ebde8c1f61ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3be7fc640 1 -- 172.21.15.103:0/3351071116 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8004230 con 0x559157cab410 2024-09-19T06:51:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3be7fc640 1 -- 172.21.15.103:0/3351071116 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c80043d0 con 0x559157cab410 2024-09-19T06:51:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.565+0000 7fa3be7fc640 1 -- 172.21.15.103:0/3351071116 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c80198c0 con 0x559157cab410 2024-09-19T06:51:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.566+0000 7fa3e1492740 1 -- 172.21.15.103:0/3351071116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157cab410 msgr2=0x559157cb5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.566+0000 7fa3e1492740 1 --2- 172.21.15.103:0/3351071116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157cab410 0x559157cb5770 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009cf0 tx=0x7fa3c8011810 comp rx=0 tx=0).stop 2024-09-19T06:51:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.566+0000 7fa3dcc8e640 1 --2- 172.21.15.103:0/412355179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa3c00a3460 0x7fa3c00a5920 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00829f0 tx=0x7fa3d00b8900 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.566+0000 7fa3e1492740 1 -- 172.21.15.103:0/3351071116 shutdown_connections 2024-09-19T06:51:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.566+0000 7fa3e1492740 1 --2- 172.21.15.103:0/3351071116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157cb80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.566+0000 7fa3e1492740 1 --2- 172.21.15.103:0/3351071116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157cab410 0x559157cb5770 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.566+0000 7fa3e1492740 1 --2- 172.21.15.103:0/3351071116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157caaaf0 0x559157caaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.566+0000 7fa3e1492740 1 -- 172.21.15.103:0/3351071116 >> 172.21.15.103:0/3351071116 conn(0x559157ca7060 msgr2=0x559157ca7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.566+0000 7fa3e1492740 1 -- 172.21.15.103:0/3351071116 shutdown_connections 2024-09-19T06:51:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.566+0000 7fa3e1492740 1 -- 172.21.15.103:0/3351071116 wait complete. 2024-09-19T06:51:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.567+0000 7fa3e1492740 1 -- start start 2024-09-19T06:51:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.567+0000 7fa3e1492740 1 Processor -- start 2024-09-19T06:51:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.567+0000 7fa3e1492740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157caaaf0 0x559157cb2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.567+0000 7fa3e1492740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157cab410 0x559157cb30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.567+0000 7fa3e1492740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157cb3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.567+0000 7fa3e1492740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559157cb8880 con 0x559157caaaf0 2024-09-19T06:51:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.567+0000 7fa3e1492740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559157b98290 con 0x559157cb5cb0 2024-09-19T06:51:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.567+0000 7fa3e1492740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559157cb8a00 con 0x559157cab410 2024-09-19T06:51:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.567+0000 7fa3e1492740 10 client.0 _ll_register_callbacks cb 0x559157bc9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.567+0000 7fa3d77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157caaaf0 0x559157cb2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.567+0000 7fa3d77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157caaaf0 0x559157cb2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46390/0 (socket says 172.21.15.103:46390) 2024-09-19T06:51:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.567+0000 7fa3d77fe640 1 -- 172.21.15.103:0/3801083934 learned_addr learned my addr 172.21.15.103:0/3801083934 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.567+0000 7fa3d7fff640 1 --2- 172.21.15.103:0/3801083934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157cab410 0x559157cb30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3d77fe640 1 -- 172.21.15.103:0/3801083934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157cab410 msgr2=0x559157cb30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3d77fe640 1 --2- 172.21.15.103:0/3801083934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157cab410 0x559157cb30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3d77fe640 1 -- 172.21.15.103:0/3801083934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 msgr2=0x559157cb3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3d77fe640 1 --2- 172.21.15.103:0/3801083934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157cb3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3d77fe640 1 -- 172.21.15.103:0/3801083934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559157922210 con 0x559157caaaf0 2024-09-19T06:51:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3d77fe640 1 --2- 172.21.15.103:0/3801083934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157caaaf0 0x559157cb2bb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc005850 tx=0x7fa3cc010bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c66a94c7e4fd6de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3b77fe640 1 -- 172.21.15.103:0/3801083934 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00ed40 con 0x559157caaaf0 2024-09-19T06:51:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3e1492740 1 -- 172.21.15.103:0/3801083934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3c80043d0 con 0x559157caaaf0 2024-09-19T06:51:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3b77fe640 1 -- 172.21.15.103:0/3801083934 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa3cc004030 con 0x559157caaaf0 2024-09-19T06:51:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3b77fe640 1 -- 172.21.15.103:0/3801083934 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa3cc004230 con 0x559157caaaf0 2024-09-19T06:51:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3e1492740 10 client.16266 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3e1492740 1 -- 172.21.15.103:0/3801083934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5591579ee620 con 0x559157caaaf0 2024-09-19T06:51:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3b77fe640 1 -- 172.21.15.103:0/3801083934 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c80043d0 con 0x559157caaaf0 2024-09-19T06:51:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3b77fe640 1 -- 172.21.15.103:0/3801083934 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa3cc01b070 con 0x559157caaaf0 2024-09-19T06:51:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3e1492740 10 client.16266 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3e1492740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.568+0000 7fa3e1492740 20 client.16266 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.569+0000 7fa3e1492740 1 --2- 172.21.15.103:0/3801083934 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559157cc5cb0 0x559157cc80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.569+0000 7fa3e1492740 4 client.16266 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.569+0000 7fa3e1492740 1 -- 172.21.15.103:0/3801083934 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559157c1ca20 con 0x559157cc5cb0 2024-09-19T06:51:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.569+0000 7fa3d7fff640 1 --2- 172.21.15.103:0/3801083934 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559157cc5cb0 0x559157cc80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.569+0000 7fa3d7fff640 1 --2- 172.21.15.103:0/3801083934 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559157cc5cb0 0x559157cc80a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64d14c2b3e7bb516 server_cookie=a592ae886bb903fc in_seq=0 out_seq=0 2024-09-19T06:51:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.569+0000 7fa3b77fe640 10 client.16266 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:23.577 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.570+0000 7fa3b77fe640 1 -- 172.21.15.103:0/3801083934 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559157c1ca20 con 0x559157cc5cb0 2024-09-19T06:51:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.570+0000 7fa3b77fe640 10 client.16266 handle_command_reply: tid=0 2024-09-19T06:51:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.570+0000 7fa3e1492740 1 client.16266 shutdown 2024-09-19T06:51:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.570+0000 7fa3e1492740 1 -- 172.21.15.103:0/3801083934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157caaaf0 msgr2=0x559157cb2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.570+0000 7fa3e1492740 1 --2- 172.21.15.103:0/3801083934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157caaaf0 0x559157cb2bb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc005850 tx=0x7fa3cc010bb0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.570+0000 7fa3e1492740 1 -- 172.21.15.103:0/3801083934 shutdown_connections 2024-09-19T06:51:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.570+0000 7fa3e1492740 1 --2- 172.21.15.103:0/3801083934 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559157cc5cb0 0x559157cc80a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.570+0000 7fa3e1492740 1 --2- 172.21.15.103:0/3801083934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157cb3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.570+0000 7fa3e1492740 1 --2- 172.21.15.103:0/3801083934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157cab410 0x559157cb30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.570+0000 7fa3b77fe640 0 client.16266 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.570+0000 7fa3e1492740 1 --2- 172.21.15.103:0/3801083934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157caaaf0 0x559157cb2bb0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.570+0000 7fa3e1492740 1 -- 172.21.15.103:0/3801083934 >> 172.21.15.103:0/3801083934 conn(0x559157ca7060 msgr2=0x559157cacd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.570+0000 7fa3e1492740 1 -- 172.21.15.103:0/3801083934 shutdown_connections 2024-09-19T06:51:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.570+0000 7fa3e1492740 1 -- 172.21.15.103:0/3801083934 wait complete. 2024-09-19T06:51:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.570+0000 7fa3e1492740 20 client.16266 trim_cache size 0 max 16384 2024-09-19T06:51:23.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:23 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3902287948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:51:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.587+0000 7fa3e1492740 1 Processor -- start 2024-09-19T06:51:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.587+0000 7fa3e1492740 1 -- start start 2024-09-19T06:51:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.587+0000 7fa3e1492740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157caaaf0 0x559157cb4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.587+0000 7fa3e1492740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157cab410 0x559157cb3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.587+0000 7fa3e1492740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157ccd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.587+0000 7fa3e1492740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d81ad6f0 con 0x559157caaaf0 2024-09-19T06:51:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.587+0000 7fa3e1492740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d81af230 con 0x559157cb5cb0 2024-09-19T06:51:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.587+0000 7fa3e1492740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d81addc0 con 0x559157cab410 2024-09-19T06:51:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.588+0000 7fa3d77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157caaaf0 0x559157cb4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.588+0000 7fa3d77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157caaaf0 0x559157cb4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46398/0 (socket says 172.21.15.103:46398) 2024-09-19T06:51:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.588+0000 7fa3d77fe640 1 -- 172.21.15.103:0/3378358029 learned_addr learned my addr 172.21.15.103:0/3378358029 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.588+0000 7fa3d7fff640 1 --2- 172.21.15.103:0/3378358029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157cab410 0x559157cb3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.588+0000 7fa3d77fe640 1 --2- 172.21.15.103:0/3378358029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157ccd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.588+0000 7fa3d77fe640 1 -- 172.21.15.103:0/3378358029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157cab410 msgr2=0x559157cb3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.588+0000 7fa3d77fe640 1 --2- 172.21.15.103:0/3378358029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157cab410 0x559157cb3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.588+0000 7fa3d77fe640 1 -- 172.21.15.103:0/3378358029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 msgr2=0x559157ccd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.588+0000 7fa3d77fe640 1 --2- 172.21.15.103:0/3378358029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157ccd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.588+0000 7fa3d77fe640 1 -- 172.21.15.103:0/3378358029 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559157caa920 con 0x559157caaaf0 2024-09-19T06:51:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.588+0000 7fa3d7fff640 1 --2- 172.21.15.103:0/3378358029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157cab410 0x559157cb3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.588+0000 7fa3d77fe640 1 --2- 172.21.15.103:0/3378358029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157caaaf0 0x559157cb4ee0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0097a0 tx=0x7fa3cc005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6af49b2819b902 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.588+0000 7fa3bdffb640 1 -- 172.21.15.103:0/3378358029 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00ed40 con 0x559157caaaf0 2024-09-19T06:51:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.588+0000 7fa3bdffb640 1 -- 172.21.15.103:0/3378358029 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3cc01dbe0 con 0x559157caaaf0 2024-09-19T06:51:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.588+0000 7fa3bdffb640 1 -- 172.21.15.103:0/3378358029 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0258b0 con 0x559157caaaf0 2024-09-19T06:51:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.589+0000 7fa3e1492740 1 -- 172.21.15.103:0/3378358029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157caaaf0 msgr2=0x559157cb4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.589+0000 7fa3e1492740 1 --2- 172.21.15.103:0/3378358029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157caaaf0 0x559157cb4ee0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0097a0 tx=0x7fa3cc005d30 comp rx=0 tx=0).stop 2024-09-19T06:51:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.589+0000 7fa3e1492740 1 -- 172.21.15.103:0/3378358029 shutdown_connections 2024-09-19T06:51:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.589+0000 7fa3e1492740 1 --2- 172.21.15.103:0/3378358029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157ccd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.589+0000 7fa3e1492740 1 --2- 172.21.15.103:0/3378358029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157cab410 0x559157cb3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.589+0000 7fa3e1492740 1 --2- 172.21.15.103:0/3378358029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157caaaf0 0x559157cb4ee0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.589+0000 7fa3e1492740 1 -- 172.21.15.103:0/3378358029 >> 172.21.15.103:0/3378358029 conn(0x559157ca7060 msgr2=0x559157ca61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.589+0000 7fa3e1492740 1 -- 172.21.15.103:0/3378358029 shutdown_connections 2024-09-19T06:51:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.589+0000 7fa3e1492740 1 -- 172.21.15.103:0/3378358029 wait complete. 2024-09-19T06:51:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.590+0000 7fa3e1492740 1 -- start start 2024-09-19T06:51:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.590+0000 7fa3e1492740 1 Processor -- start 2024-09-19T06:51:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.590+0000 7fa3e1492740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157caaaf0 0x559157cafd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.590+0000 7fa3e1492740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157cab410 0x559157cb0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.590+0000 7fa3e1492740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157cb07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.590+0000 7fa3e1492740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d81aeb60 con 0x559157cab410 2024-09-19T06:51:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.590+0000 7fa3e1492740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d81ae490 con 0x559157cb5cb0 2024-09-19T06:51:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.590+0000 7fa3e1492740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d81ab6b0 con 0x559157caaaf0 2024-09-19T06:51:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.590+0000 7fa3e1492740 10 client.0 _ll_register_callbacks cb 0x7fa3cc0098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.590+0000 7fa3d7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157cab410 0x559157cb0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.590+0000 7fa3d7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157cab410 0x559157cb0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46402/0 (socket says 172.21.15.103:46402) 2024-09-19T06:51:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.590+0000 7fa3d7fff640 1 -- 172.21.15.103:0/1558872212 learned_addr learned my addr 172.21.15.103:0/1558872212 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.590+0000 7fa3d77fe640 1 --2- 172.21.15.103:0/1558872212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157caaaf0 0x559157cafd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.591+0000 7fa3d7fff640 1 -- 172.21.15.103:0/1558872212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157caaaf0 msgr2=0x559157cafd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.591+0000 7fa3d77fe640 1 --2- 172.21.15.103:0/1558872212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157cb07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.591+0000 7fa3d7fff640 1 --2- 172.21.15.103:0/1558872212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157caaaf0 0x559157cafd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.591+0000 7fa3d7fff640 1 -- 172.21.15.103:0/1558872212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 msgr2=0x559157cb07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.591+0000 7fa3d7fff640 1 --2- 172.21.15.103:0/1558872212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157cb07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.591+0000 7fa3d7fff640 1 -- 172.21.15.103:0/1558872212 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559157d2fa70 con 0x559157cab410 2024-09-19T06:51:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.591+0000 7fa3d77fe640 1 --2- 172.21.15.103:0/1558872212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157cb07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.591+0000 7fa3d7fff640 1 --2- 172.21.15.103:0/1558872212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157cab410 0x559157cb0260 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80099b0 tx=0x7fa3c80194c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d786715118dca1fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.591+0000 7fa3b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.591+0000 7fa3b77fe640 1 -- 172.21.15.103:0/1558872212 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c802f020 con 0x559157cab410 2024-09-19T06:51:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.591+0000 7fa3b77fe640 1 -- 172.21.15.103:0/1558872212 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa3c8005450 con 0x559157cab410 2024-09-19T06:51:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.591+0000 7fa3e1492740 1 -- 172.21.15.103:0/1558872212 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3cc01dbe0 con 0x559157cab410 2024-09-19T06:51:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.591+0000 7fa3b77fe640 1 -- 172.21.15.103:0/1558872212 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa3c803a050 con 0x559157cab410 2024-09-19T06:51:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.591+0000 7fa3e1492740 10 client.16272 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.591+0000 7fa3e1492740 1 -- 172.21.15.103:0/1558872212 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559157d30d60 con 0x559157cab410 2024-09-19T06:51:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.591+0000 7fa3b77fe640 1 -- 172.21.15.103:0/1558872212 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3cc01dbe0 con 0x559157cab410 2024-09-19T06:51:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.592+0000 7fa3b77fe640 1 -- 172.21.15.103:0/1558872212 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa3c80197d0 con 0x559157cab410 2024-09-19T06:51:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.592+0000 7fa3e1492740 10 client.16272 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.592+0000 7fa3e1492740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.592+0000 7fa3e1492740 20 client.16272 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.592+0000 7fa3e1492740 1 --2- 172.21.15.103:0/1558872212 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559157d31370 0x559157d33790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.592+0000 7fa3e1492740 4 client.16272 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:51:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.592+0000 7fa3e1492740 1 -- 172.21.15.103:0/1558872212 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559157c16d70 con 0x559157d31370 2024-09-19T06:51:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.592+0000 7fa3d77fe640 1 --2- 172.21.15.103:0/1558872212 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559157d31370 0x559157d33790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.592+0000 7fa3b77fe640 10 client.16272 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.592+0000 7fa3d77fe640 1 --2- 172.21.15.103:0/1558872212 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559157d31370 0x559157d33790 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39adeff22e113210 server_cookie=9a277549406eed6a in_seq=0 out_seq=0 2024-09-19T06:51:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.596+0000 7fa3b77fe640 1 -- 172.21.15.103:0/1558872212 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559157c16d70 con 0x559157d31370 2024-09-19T06:51:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.596+0000 7fa3b77fe640 10 client.16272 handle_command_reply: tid=0 2024-09-19T06:51:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.596+0000 7fa3e1492740 1 client.16272 shutdown 2024-09-19T06:51:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3e1492740 1 -- 172.21.15.103:0/1558872212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157cab410 msgr2=0x559157cb0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3e1492740 1 --2- 172.21.15.103:0/1558872212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157cab410 0x559157cb0260 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80099b0 tx=0x7fa3c80194c0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3e1492740 1 -- 172.21.15.103:0/1558872212 shutdown_connections 2024-09-19T06:51:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3e1492740 1 --2- 172.21.15.103:0/1558872212 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559157d31370 0x559157d33790 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3e1492740 1 --2- 172.21.15.103:0/1558872212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559157cb5cb0 0x559157cb07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3e1492740 1 --2- 172.21.15.103:0/1558872212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559157cab410 0x559157cb0260 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3b77fe640 0 client.16272 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3e1492740 1 --2- 172.21.15.103:0/1558872212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559157caaaf0 0x559157cafd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3e1492740 1 -- 172.21.15.103:0/1558872212 >> 172.21.15.103:0/1558872212 conn(0x559157ca7060 msgr2=0x559157cc0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3e1492740 1 -- 172.21.15.103:0/1558872212 shutdown_connections 2024-09-19T06:51:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3e1492740 1 -- 172.21.15.103:0/1558872212 wait complete. 2024-09-19T06:51:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3e1492740 20 client.16272 trim_cache size 0 max 16384 2024-09-19T06:51:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:51:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:51:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:51:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:51:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3de6f1640 1 -- 172.21.15.103:0/412355179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa3c00a3460 msgr2=0x7fa3c00a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3de6f1640 1 --2- 172.21.15.103:0/412355179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa3c00a3460 0x7fa3c00a5920 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00829f0 tx=0x7fa3d00b8900 comp rx=0 tx=0).stop 2024-09-19T06:51:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3de6f1640 1 -- 172.21.15.103:0/412355179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3d810c0c0 msgr2=0x7fa3d819cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3de6f1640 1 --2- 172.21.15.103:0/412355179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3d810c0c0 0x7fa3d819cb70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0004af0 tx=0x7fa3d002f3a0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3de6f1640 1 -- 172.21.15.103:0/412355179 shutdown_connections 2024-09-19T06:51:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3de6f1640 1 --2- 172.21.15.103:0/412355179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa3c00a3460 0x7fa3c00a5920 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3de6f1640 1 --2- 172.21.15.103:0/412355179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3d810c0c0 0x7fa3d819cb70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3de6f1640 1 --2- 172.21.15.103:0/412355179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3d81035c0 0x7fa3d81987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3de6f1640 1 --2- 172.21.15.103:0/412355179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3d8102ca0 0x7fa3d81982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.597+0000 7fa3de6f1640 1 -- 172.21.15.103:0/412355179 >> 172.21.15.103:0/412355179 conn(0x7fa3d80fc6f0 msgr2=0x7fa3d80fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.598+0000 7fa3de6f1640 1 -- 172.21.15.103:0/412355179 shutdown_connections 2024-09-19T06:51:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:23.598+0000 7fa3de6f1640 1 -- 172.21.15.103:0/412355179 wait complete. 2024-09-19T06:51:23.994 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:51:24.266 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:24 smithi103 ceph-mon[31158]: pgmap v68: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-19T06:51:24.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:24 smithi134 ceph-mon[35356]: pgmap v68: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-19T06:51:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.500+0000 7f85661a0640 1 -- 172.21.15.103:0/615059261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8560107af0 msgr2=0x7f8560107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.500+0000 7f85661a0640 1 --2- 172.21.15.103:0/615059261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8560107af0 0x7f8560107ed0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f855c009930 tx=0x7f855c02a420 comp rx=0 tx=0).stop 2024-09-19T06:51:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.501+0000 7f85661a0640 1 -- 172.21.15.103:0/615059261 shutdown_connections 2024-09-19T06:51:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.501+0000 7f85661a0640 1 --2- 172.21.15.103:0/615059261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8560102380 0x7f856010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.501+0000 7f85661a0640 1 --2- 172.21.15.103:0/615059261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85601019e0 0x7f8560101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.501+0000 7f85661a0640 1 --2- 172.21.15.103:0/615059261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8560107af0 0x7f8560107ed0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.501+0000 7f85661a0640 1 -- 172.21.15.103:0/615059261 >> 172.21.15.103:0/615059261 conn(0x7f85600fd700 msgr2=0x7f85600ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:24.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.501+0000 7f85661a0640 1 -- 172.21.15.103:0/615059261 shutdown_connections 2024-09-19T06:51:24.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.501+0000 7f85661a0640 1 -- 172.21.15.103:0/615059261 wait complete. 2024-09-19T06:51:24.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.501+0000 7f85661a0640 1 Processor -- start 2024-09-19T06:51:24.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.501+0000 7f85661a0640 1 -- start start 2024-09-19T06:51:24.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f85661a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85601019e0 0x7f85600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f85661a0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8560102380 0x7f8560074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f85661a0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8560107af0 0x7f8560074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f85661a0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8560111f40 con 0x7f85601019e0 2024-09-19T06:51:24.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f85661a0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8560111dc0 con 0x7f8560102380 2024-09-19T06:51:24.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f85661a0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85601120c0 con 0x7f8560107af0 2024-09-19T06:51:24.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f856519e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85601019e0 0x7f85600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f856519e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85601019e0 0x7f85600744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46416/0 (socket says 172.21.15.103:46416) 2024-09-19T06:51:24.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f856519e640 1 -- 172.21.15.103:0/2693105231 learned_addr learned my addr 172.21.15.103:0/2693105231 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:24.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f856599f640 1 --2- 172.21.15.103:0/2693105231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8560107af0 0x7f8560074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f856499d640 1 --2- 172.21.15.103:0/2693105231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8560102380 0x7f8560074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f856519e640 1 -- 172.21.15.103:0/2693105231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8560107af0 msgr2=0x7f8560074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:24.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f856519e640 1 --2- 172.21.15.103:0/2693105231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8560107af0 0x7f8560074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f856519e640 1 -- 172.21.15.103:0/2693105231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8560102380 msgr2=0x7f8560074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:24.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f856519e640 1 --2- 172.21.15.103:0/2693105231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8560102380 0x7f8560074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f856519e640 1 -- 172.21.15.103:0/2693105231 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85601a6880 con 0x7f85601019e0 2024-09-19T06:51:24.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.502+0000 7f856499d640 1 --2- 172.21.15.103:0/2693105231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8560102380 0x7f8560074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:24.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.503+0000 7f856519e640 1 --2- 172.21.15.103:0/2693105231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85601019e0 0x7f85600744f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f855c02a940 tx=0x7f855c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:24.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.503+0000 7f85567fc640 1 -- 172.21.15.103:0/2693105231 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c03f070 con 0x7f85601019e0 2024-09-19T06:51:24.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.503+0000 7f85661a0640 1 -- 172.21.15.103:0/2693105231 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85601a6b10 con 0x7f85601019e0 2024-09-19T06:51:24.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.503+0000 7f85661a0640 1 -- 172.21.15.103:0/2693105231 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85601a7040 con 0x7f85601019e0 2024-09-19T06:51:24.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.503+0000 7f85567fc640 1 -- 172.21.15.103:0/2693105231 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f855c031c90 con 0x7f85601019e0 2024-09-19T06:51:24.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.503+0000 7f85567fc640 1 -- 172.21.15.103:0/2693105231 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c035030 con 0x7f85601019e0 2024-09-19T06:51:24.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.506+0000 7f85697db740 1 Processor -- start 2024-09-19T06:51:24.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.506+0000 7f85697db740 1 -- start start 2024-09-19T06:51:24.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.507+0000 7f85697db740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38013130 0x559e38013510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.507+0000 7f85697db740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38013a50 0x559e380266c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.507+0000 7f85697db740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38026c00 0x559e38028ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.507+0000 7f85697db740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559e37f78620 con 0x559e38026c00 2024-09-19T06:51:24.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.507+0000 7f85697db740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559e37f1fe50 con 0x559e38013a50 2024-09-19T06:51:24.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.507+0000 7f85697db740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559e37f745e0 con 0x559e38013130 2024-09-19T06:51:24.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.507+0000 7f856499d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38026c00 0x559e38028ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.507+0000 7f856499d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38026c00 0x559e38028ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46420/0 (socket says 172.21.15.103:46420) 2024-09-19T06:51:24.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.507+0000 7f856499d640 1 -- 172.21.15.103:0/180201875 learned_addr learned my addr 172.21.15.103:0/180201875 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:24.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.507+0000 7f856599f640 1 --2- 172.21.15.103:0/180201875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38013130 0x559e38013510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.507+0000 7f856499d640 1 -- 172.21.15.103:0/180201875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38013130 msgr2=0x559e38013510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:24.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.507+0000 7f856499d640 1 --2- 172.21.15.103:0/180201875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38013130 0x559e38013510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.507+0000 7f856499d640 1 -- 172.21.15.103:0/180201875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38013a50 msgr2=0x559e380266c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:51:24.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.507+0000 7f856499d640 1 --2- 172.21.15.103:0/180201875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38013a50 0x559e380266c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.507+0000 7f85567fc640 1 -- 172.21.15.103:0/2693105231 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f855c042070 con 0x7f85601019e0 2024-09-19T06:51:24.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.507+0000 7f856499d640 1 -- 172.21.15.103:0/180201875 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e37f10d40 con 0x559e38026c00 2024-09-19T06:51:24.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.508+0000 7f856499d640 1 --2- 172.21.15.103:0/180201875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38026c00 0x559e38028ff0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f855000bcb0 tx=0x7f855000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7348b9f58c9274 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:24.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.508+0000 7f854b7fe640 1 -- 172.21.15.103:0/180201875 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550011440 con 0x559e38026c00 2024-09-19T06:51:24.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.508+0000 7f854b7fe640 1 -- 172.21.15.103:0/180201875 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8550011ac0 con 0x559e38026c00 2024-09-19T06:51:24.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.508+0000 7f85567fc640 1 --2- 172.21.15.103:0/2693105231 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f854c0a33b0 0x7f854c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.508+0000 7f856599f640 1 --2- 172.21.15.103:0/2693105231 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f854c0a33b0 0x7f854c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.508+0000 7f85567fc640 1 -- 172.21.15.103:0/2693105231 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f855c0f0410 con 0x7f85601019e0 2024-09-19T06:51:24.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.508+0000 7f854b7fe640 1 -- 172.21.15.103:0/180201875 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855001bb50 con 0x559e38026c00 2024-09-19T06:51:24.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.509+0000 7f85697db740 1 -- 172.21.15.103:0/180201875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38026c00 msgr2=0x559e38028ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:24.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.509+0000 7f85697db740 1 --2- 172.21.15.103:0/180201875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38026c00 0x559e38028ff0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f855000bcb0 tx=0x7f855000f730 comp rx=0 tx=0).stop 2024-09-19T06:51:24.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.509+0000 7f85697db740 1 -- 172.21.15.103:0/180201875 shutdown_connections 2024-09-19T06:51:24.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.509+0000 7f856599f640 1 --2- 172.21.15.103:0/2693105231 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f854c0a33b0 0x7f854c0a5870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8558005fd0 tx=0x7f8558009600 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:24.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.509+0000 7f85697db740 1 --2- 172.21.15.103:0/180201875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38026c00 0x559e38028ff0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.509+0000 7f85697db740 1 --2- 172.21.15.103:0/180201875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38013a50 0x559e380266c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.509+0000 7f85697db740 1 --2- 172.21.15.103:0/180201875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38013130 0x559e38013510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.509+0000 7f85697db740 1 -- 172.21.15.103:0/180201875 >> 172.21.15.103:0/180201875 conn(0x559e3800f7a0 msgr2=0x559e3800fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:24.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.509+0000 7f85697db740 1 -- 172.21.15.103:0/180201875 shutdown_connections 2024-09-19T06:51:24.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.509+0000 7f85697db740 1 -- 172.21.15.103:0/180201875 wait complete. 2024-09-19T06:51:24.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.509+0000 7f85697db740 1 -- start start 2024-09-19T06:51:24.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.510+0000 7f85697db740 1 Processor -- start 2024-09-19T06:51:24.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.510+0000 7f85697db740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38013130 0x559e38023aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.510+0000 7f85697db740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013a50 0x559e38023fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.510+0000 7f85697db740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38026c00 0x559e38024520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.510+0000 7f85697db740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559e380297c0 con 0x559e38013a50 2024-09-19T06:51:24.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.510+0000 7f85697db740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559e37f00f10 con 0x559e38026c00 2024-09-19T06:51:24.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.510+0000 7f85697db740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559e38029940 con 0x559e38013130 2024-09-19T06:51:24.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.510+0000 7f85697db740 10 client.0 _ll_register_callbacks cb 0x559e37f24db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:24.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.510+0000 7f856499d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013a50 0x559e38023fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.510+0000 7f856519e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38013130 0x559e38023aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.510+0000 7f856499d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013a50 0x559e38023fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46426/0 (socket says 172.21.15.103:46426) 2024-09-19T06:51:24.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.510+0000 7f856499d640 1 -- 172.21.15.103:0/2564673057 learned_addr learned my addr 172.21.15.103:0/2564673057 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:24.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.510+0000 7f856599f640 1 --2- 172.21.15.103:0/2564673057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38026c00 0x559e38024520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.511+0000 7f856499d640 1 -- 172.21.15.103:0/2564673057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38013130 msgr2=0x559e38023aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:24.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.511+0000 7f856499d640 1 --2- 172.21.15.103:0/2564673057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38013130 0x559e38023aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.511+0000 7f856499d640 1 -- 172.21.15.103:0/2564673057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38026c00 msgr2=0x559e38024520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:24.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.511+0000 7f856499d640 1 --2- 172.21.15.103:0/2564673057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38026c00 0x559e38024520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.511+0000 7f856499d640 1 -- 172.21.15.103:0/2564673057 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e37c87580 con 0x559e38013a50 2024-09-19T06:51:24.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.511+0000 7f856599f640 1 --2- 172.21.15.103:0/2564673057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38026c00 0x559e38024520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:24.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.511+0000 7f856499d640 1 --2- 172.21.15.103:0/2564673057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013a50 0x559e38023fe0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8550000c00 tx=0x7f855000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdc198b0eb2d797 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:24.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.511+0000 7f8548ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:24.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.511+0000 7f8548ff9640 1 -- 172.21.15.103:0/2564673057 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855002a060 con 0x559e38013a50 2024-09-19T06:51:24.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.511+0000 7f8548ff9640 1 -- 172.21.15.103:0/2564673057 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f8550004250 con 0x559e38013a50 2024-09-19T06:51:24.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.512+0000 7f85697db740 1 -- 172.21.15.103:0/2564673057 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8550011ac0 con 0x559e38013a50 2024-09-19T06:51:24.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.512+0000 7f8548ff9640 1 -- 172.21.15.103:0/2564673057 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8550004450 con 0x559e38013a50 2024-09-19T06:51:24.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.512+0000 7f85697db740 10 client.16284 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:24.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.512+0000 7f85697db740 1 -- 172.21.15.103:0/2564673057 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559e37d51960 con 0x559e38013a50 2024-09-19T06:51:24.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.512+0000 7f8548ff9640 1 -- 172.21.15.103:0/2564673057 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8550011ac0 con 0x559e38013a50 2024-09-19T06:51:24.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.512+0000 7f8548ff9640 1 -- 172.21.15.103:0/2564673057 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f85500035f0 con 0x559e38013a50 2024-09-19T06:51:24.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.512+0000 7f85697db740 10 client.16284 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:24.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.512+0000 7f85697db740 10 client.16284 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:51:24.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.512+0000 7f85697db740 20 client.16284 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:24.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.512+0000 7f85697db740 1 --2- 172.21.15.103:0/2564673057 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559e38036770 0x559e38038b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.512+0000 7f85697db740 4 client.16284 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:24.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.512+0000 7f85697db740 1 -- 172.21.15.103:0/2564673057 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e37f85b90 con 0x559e38036770 2024-09-19T06:51:24.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.513+0000 7f856519e640 1 --2- 172.21.15.103:0/2564673057 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559e38036770 0x559e38038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.513+0000 7f8548ff9640 10 client.16284 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:24.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.513+0000 7f856519e640 1 --2- 172.21.15.103:0/2564673057 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559e38036770 0x559e38038b60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70785b8f19a65ff2 server_cookie=1cc7f6d2a51bb12b in_seq=0 out_seq=0 2024-09-19T06:51:24.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.514+0000 7f8548ff9640 1 -- 172.21.15.103:0/2564673057 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e37f85b90 con 0x559e38036770 2024-09-19T06:51:24.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.514+0000 7f8548ff9640 10 client.16284 handle_command_reply: tid=0 2024-09-19T06:51:24.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.514+0000 7f85697db740 1 client.16284 shutdown 2024-09-19T06:51:24.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.514+0000 7f85697db740 1 -- 172.21.15.103:0/2564673057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013a50 msgr2=0x559e38023fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:24.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.514+0000 7f85697db740 1 --2- 172.21.15.103:0/2564673057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013a50 0x559e38023fe0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8550000c00 tx=0x7f855000bfd0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.514+0000 7f85697db740 1 -- 172.21.15.103:0/2564673057 shutdown_connections 2024-09-19T06:51:24.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.514+0000 7f85697db740 1 --2- 172.21.15.103:0/2564673057 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559e38036770 0x559e38038b60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.514+0000 7f85697db740 1 --2- 172.21.15.103:0/2564673057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38026c00 0x559e38024520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.514+0000 7f85697db740 1 --2- 172.21.15.103:0/2564673057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013a50 0x559e38023fe0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.514+0000 7f85697db740 1 --2- 172.21.15.103:0/2564673057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38013130 0x559e38023aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.514+0000 7f8548ff9640 0 client.16284 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:24.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.514+0000 7f85697db740 1 -- 172.21.15.103:0/2564673057 >> 172.21.15.103:0/2564673057 conn(0x559e3800f7a0 msgr2=0x559e38015430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:24.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.515+0000 7f85697db740 1 -- 172.21.15.103:0/2564673057 shutdown_connections 2024-09-19T06:51:24.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.515+0000 7f85697db740 1 -- 172.21.15.103:0/2564673057 wait complete. 2024-09-19T06:51:24.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.515+0000 7f85697db740 20 client.16284 trim_cache size 0 max 16384 2024-09-19T06:51:24.530 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:24.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.534+0000 7f85697db740 1 Processor -- start 2024-09-19T06:51:24.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.534+0000 7f85697db740 1 -- start start 2024-09-19T06:51:24.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.534+0000 7f85697db740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38013130 0x559e3800e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.534+0000 7f85697db740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013a50 0x559e3803e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.534+0000 7f85697db740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38026c00 0x559e3803e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.534+0000 7f85697db740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f854400e740 con 0x559e38013a50 2024-09-19T06:51:24.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.534+0000 7f85697db740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8544010280 con 0x559e38013130 2024-09-19T06:51:24.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.534+0000 7f85697db740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f854400ee10 con 0x559e38026c00 2024-09-19T06:51:24.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.534+0000 7f856499d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013a50 0x559e3803e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.535+0000 7f856499d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013a50 0x559e3803e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46428/0 (socket says 172.21.15.103:46428) 2024-09-19T06:51:24.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.535+0000 7f856499d640 1 -- 172.21.15.103:0/55349007 learned_addr learned my addr 172.21.15.103:0/55349007 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:24.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.535+0000 7f856519e640 1 --2- 172.21.15.103:0/55349007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38013130 0x559e3800e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.535+0000 7f856599f640 1 --2- 172.21.15.103:0/55349007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38026c00 0x559e3803e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.535+0000 7f856499d640 1 -- 172.21.15.103:0/55349007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38026c00 msgr2=0x559e3803e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:24.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.535+0000 7f856499d640 1 --2- 172.21.15.103:0/55349007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38026c00 0x559e3803e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.535+0000 7f856499d640 1 -- 172.21.15.103:0/55349007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38013130 msgr2=0x559e3800e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:24.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.535+0000 7f856499d640 1 --2- 172.21.15.103:0/55349007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38013130 0x559e3800e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.535+0000 7f856499d640 1 -- 172.21.15.103:0/55349007 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3800dd60 con 0x559e38013a50 2024-09-19T06:51:24.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.535+0000 7f856599f640 1 --2- 172.21.15.103:0/55349007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38026c00 0x559e3803e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:24.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.535+0000 7f856499d640 1 --2- 172.21.15.103:0/55349007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013a50 0x559e3803e190 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8550003a40 tx=0x7f855000fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede7edbb73b570de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:24.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.535+0000 7f854affd640 1 -- 172.21.15.103:0/55349007 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855002a070 con 0x559e38013a50 2024-09-19T06:51:24.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.535+0000 7f854affd640 1 -- 172.21.15.103:0/55349007 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f855001b410 con 0x559e38013a50 2024-09-19T06:51:24.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.535+0000 7f854affd640 1 -- 172.21.15.103:0/55349007 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85500052a0 con 0x559e38013a50 2024-09-19T06:51:24.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.536+0000 7f85697db740 1 -- 172.21.15.103:0/55349007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013a50 msgr2=0x559e3803e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:24.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.536+0000 7f85697db740 1 --2- 172.21.15.103:0/55349007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013a50 0x559e3803e190 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8550003a40 tx=0x7f855000fce0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.536+0000 7f85697db740 1 -- 172.21.15.103:0/55349007 shutdown_connections 2024-09-19T06:51:24.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.536+0000 7f85697db740 1 --2- 172.21.15.103:0/55349007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38026c00 0x559e3803e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.536+0000 7f85697db740 1 --2- 172.21.15.103:0/55349007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013a50 0x559e3803e190 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.536+0000 7f85697db740 1 --2- 172.21.15.103:0/55349007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38013130 0x559e3800e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.536+0000 7f85697db740 1 -- 172.21.15.103:0/55349007 >> 172.21.15.103:0/55349007 conn(0x559e3800f7a0 msgr2=0x559e380153a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:24.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.536+0000 7f85697db740 1 -- 172.21.15.103:0/55349007 shutdown_connections 2024-09-19T06:51:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.536+0000 7f85697db740 1 -- 172.21.15.103:0/55349007 wait complete. 2024-09-19T06:51:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.537+0000 7f85697db740 1 -- start start 2024-09-19T06:51:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.537+0000 7f85697db740 1 Processor -- start 2024-09-19T06:51:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.537+0000 7f85697db740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013130 0x559e3802ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.537+0000 7f85697db740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38013a50 0x559e3802b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.537+0000 7f85697db740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38026c00 0x559e380a0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.537+0000 7f85697db740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f854400fbb0 con 0x559e38013130 2024-09-19T06:51:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.537+0000 7f85697db740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f854400f4e0 con 0x559e38013a50 2024-09-19T06:51:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.537+0000 7f85697db740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f854400c700 con 0x559e38026c00 2024-09-19T06:51:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.537+0000 7f85697db740 10 client.0 _ll_register_callbacks cb 0x7f855000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.537+0000 7f856519e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013130 0x559e3802ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.537+0000 7f856519e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013130 0x559e3802ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46430/0 (socket says 172.21.15.103:46430) 2024-09-19T06:51:24.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.537+0000 7f856519e640 1 -- 172.21.15.103:0/954644450 learned_addr learned my addr 172.21.15.103:0/954644450 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.537+0000 7f856499d640 1 --2- 172.21.15.103:0/954644450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38013a50 0x559e3802b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.538+0000 7f856599f640 1 --2- 172.21.15.103:0/954644450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38026c00 0x559e380a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.538+0000 7f856519e640 1 -- 172.21.15.103:0/954644450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38026c00 msgr2=0x559e380a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.538+0000 7f856519e640 1 --2- 172.21.15.103:0/954644450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38026c00 0x559e380a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.538+0000 7f856519e640 1 -- 172.21.15.103:0/954644450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38013a50 msgr2=0x559e3802b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.538+0000 7f856519e640 1 --2- 172.21.15.103:0/954644450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38013a50 0x559e3802b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.538+0000 7f856519e640 1 -- 172.21.15.103:0/954644450 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e3802bb70 con 0x559e38013130 2024-09-19T06:51:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.538+0000 7f856499d640 1 --2- 172.21.15.103:0/954644450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38013a50 0x559e3802b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.538+0000 7f856519e640 1 --2- 172.21.15.103:0/954644450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013130 0x559e3802ad50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f855c0b90d0 tx=0x7f855c005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d2b918d1a4c354 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.538+0000 7f8548ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.538+0000 7f8548ff9640 1 -- 172.21.15.103:0/954644450 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c0da060 con 0x559e38013130 2024-09-19T06:51:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.538+0000 7f8548ff9640 1 -- 172.21.15.103:0/954644450 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f855c0eddf0 con 0x559e38013130 2024-09-19T06:51:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.538+0000 7f85697db740 1 -- 172.21.15.103:0/954644450 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f855001b410 con 0x559e38013130 2024-09-19T06:51:24.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.539+0000 7f8548ff9640 1 -- 172.21.15.103:0/954644450 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f855c0d4030 con 0x559e38013130 2024-09-19T06:51:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.539+0000 7f85697db740 10 client.16290 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.539+0000 7f85697db740 1 -- 172.21.15.103:0/954644450 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559e380a1820 con 0x559e38013130 2024-09-19T06:51:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.539+0000 7f8548ff9640 1 -- 172.21.15.103:0/954644450 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f855001b410 con 0x559e38013130 2024-09-19T06:51:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.539+0000 7f8548ff9640 1 -- 172.21.15.103:0/954644450 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f855c03b460 con 0x559e38013130 2024-09-19T06:51:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.539+0000 7f85697db740 10 client.16290 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.539+0000 7f85697db740 10 client.16290 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:51:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.539+0000 7f85697db740 20 client.16290 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.539+0000 7f85697db740 1 --2- 172.21.15.103:0/954644450 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559e380a1ef0 0x559e380a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.539+0000 7f85697db740 4 client.16290 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ee303a01", "release": true, "await": true}] 2024-09-19T06:51:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.539+0000 7f85697db740 1 -- 172.21.15.103:0/954644450 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ee303a01", "release": true, "await": true}) -- 0x559e37f800a0 con 0x559e380a1ef0 2024-09-19T06:51:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.539+0000 7f856499d640 1 --2- 172.21.15.103:0/954644450 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559e380a1ef0 0x559e380a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.540+0000 7f8548ff9640 10 client.16290 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.540+0000 7f856499d640 1 --2- 172.21.15.103:0/954644450 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559e380a1ef0 0x559e380a42e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=804592482df96fee server_cookie=4acd70d4c020ae40 in_seq=0 out_seq=0 2024-09-19T06:51:24.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.544+0000 7f8548ff9640 1 -- 172.21.15.103:0/954644450 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559e37f800a0 con 0x559e380a1ef0 2024-09-19T06:51:24.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.544+0000 7f8548ff9640 10 client.16290 handle_command_reply: tid=0 2024-09-19T06:51:24.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.544+0000 7f85697db740 1 client.16290 shutdown 2024-09-19T06:51:24.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.544+0000 7f85697db740 1 -- 172.21.15.103:0/954644450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013130 msgr2=0x559e3802ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:24.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.544+0000 7f85697db740 1 --2- 172.21.15.103:0/954644450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013130 0x559e3802ad50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f855c0b90d0 tx=0x7f855c005cb0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.544+0000 7f85697db740 1 -- 172.21.15.103:0/954644450 shutdown_connections 2024-09-19T06:51:24.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.544+0000 7f85697db740 1 --2- 172.21.15.103:0/954644450 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559e380a1ef0 0x559e380a42e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.544+0000 7f85697db740 1 --2- 172.21.15.103:0/954644450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e38026c00 0x559e380a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.544+0000 7f85697db740 1 --2- 172.21.15.103:0/954644450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e38013a50 0x559e3802b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.544+0000 7f85697db740 1 --2- 172.21.15.103:0/954644450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e38013130 0x559e3802ad50 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.544+0000 7f85697db740 1 -- 172.21.15.103:0/954644450 >> 172.21.15.103:0/954644450 conn(0x559e3800f7a0 msgr2=0x559e38014b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:24.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.544+0000 7f8548ff9640 0 client.16290 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:24.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.544+0000 7f85697db740 1 -- 172.21.15.103:0/954644450 shutdown_connections 2024-09-19T06:51:24.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.545+0000 7f85697db740 1 -- 172.21.15.103:0/954644450 wait complete. 2024-09-19T06:51:24.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.545+0000 7f85697db740 20 client.16290 trim_cache size 0 max 16384 2024-09-19T06:51:24.548 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:51:24.548 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:51:24.548 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:51:24.548 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 12, 2024-09-19T06:51:24.548 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:51:24.549 INFO:teuthology.orchestra.run.smithi103.stdout: "ee303a01": { 2024-09-19T06:51:24.549 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 12, 2024-09-19T06:51:24.549 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:51:24.549 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:51:24.549 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:51:24.549 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:51:24.549 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:51:24.549 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:51:24.549 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 140.3, 2024-09-19T06:51:24.549 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:51:24.549 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:51:24.549 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:51:24.549 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:51:24.549 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:51:24.550 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:51:24.550 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:51:24.550 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:51:24.550 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:51:24.550 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:51:24.550 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:51:24.550 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:51:24.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.545+0000 7f85661a0640 1 -- 172.21.15.103:0/2693105231 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f854c0a33b0 msgr2=0x7f854c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:24.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.545+0000 7f85661a0640 1 --2- 172.21.15.103:0/2693105231 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f854c0a33b0 0x7f854c0a5870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8558005fd0 tx=0x7f8558009600 comp rx=0 tx=0).stop 2024-09-19T06:51:24.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.545+0000 7f85661a0640 1 -- 172.21.15.103:0/2693105231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85601019e0 msgr2=0x7f85600744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:24.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.545+0000 7f85661a0640 1 --2- 172.21.15.103:0/2693105231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85601019e0 0x7f85600744f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f855c02a940 tx=0x7f855c003b90 comp rx=0 tx=0).stop 2024-09-19T06:51:24.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.545+0000 7f85661a0640 1 -- 172.21.15.103:0/2693105231 shutdown_connections 2024-09-19T06:51:24.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.545+0000 7f85661a0640 1 --2- 172.21.15.103:0/2693105231 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f854c0a33b0 0x7f854c0a5870 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.545+0000 7f85661a0640 1 --2- 172.21.15.103:0/2693105231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8560107af0 0x7f8560074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.545+0000 7f85661a0640 1 --2- 172.21.15.103:0/2693105231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8560102380 0x7f8560074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.545+0000 7f85661a0640 1 --2- 172.21.15.103:0/2693105231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85601019e0 0x7f85600744f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:24.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.546+0000 7f85661a0640 1 -- 172.21.15.103:0/2693105231 >> 172.21.15.103:0/2693105231 conn(0x7f85600fd700 msgr2=0x7f85600fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:24.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.546+0000 7f85661a0640 1 -- 172.21.15.103:0/2693105231 shutdown_connections 2024-09-19T06:51:24.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:24.546+0000 7f85661a0640 1 -- 172.21.15.103:0/2693105231 wait complete. 2024-09-19T06:51:24.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:24 smithi174 ceph-mon[35637]: pgmap v68: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-19T06:51:25.172 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ee303a01', total seconds elapsed: 25.0 2024-09-19T06:51:25.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:25 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:51:25.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:25 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:25 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:25 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:25 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:25 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:25 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:25 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:51:25.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:25 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:25 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:25 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:25 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:25 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:25 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:25 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:51:25.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:25 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:25 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:25 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:25 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:25 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:25 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:25.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.643+0000 7fa97a9f4640 1 -- 172.21.15.103:0/547738433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa974101de0 msgr2=0x7fa9741021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.643+0000 7fa97a9f4640 1 --2- 172.21.15.103:0/547738433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa974101de0 0x7fa9741021c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa964009900 tx=0x7fa96402a410 comp rx=0 tx=0).stop 2024-09-19T06:51:25.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.643+0000 7fa97a9f4640 1 -- 172.21.15.103:0/547738433 shutdown_connections 2024-09-19T06:51:25.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.643+0000 7fa97a9f4640 1 --2- 172.21.15.103:0/547738433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa974106f10 0x7fa97410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.643+0000 7fa97a9f4640 1 --2- 172.21.15.103:0/547738433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa974102790 0x7fa9741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.643+0000 7fa97a9f4640 1 --2- 172.21.15.103:0/547738433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa974101de0 0x7fa9741021c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.643+0000 7fa97a9f4640 1 -- 172.21.15.103:0/547738433 >> 172.21.15.103:0/547738433 conn(0x7fa9740fd8c0 msgr2=0x7fa9740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:25.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.643+0000 7fa97a9f4640 1 -- 172.21.15.103:0/547738433 shutdown_connections 2024-09-19T06:51:25.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.643+0000 7fa97a9f4640 1 -- 172.21.15.103:0/547738433 wait complete. 2024-09-19T06:51:25.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.644+0000 7fa97a9f4640 1 Processor -- start 2024-09-19T06:51:25.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.644+0000 7fa97a9f4640 1 -- start start 2024-09-19T06:51:25.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.644+0000 7fa97a9f4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa974101de0 0x7fa974198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.644+0000 7fa97a9f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa974102790 0x7fa9741988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.644+0000 7fa97a9f4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa974106f10 0x7fa97419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.644+0000 7fa97a9f4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa974111f30 con 0x7fa974101de0 2024-09-19T06:51:25.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.644+0000 7fa97a9f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa974111db0 con 0x7fa974102790 2024-09-19T06:51:25.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.644+0000 7fa97a9f4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa9741120b0 con 0x7fa974106f10 2024-09-19T06:51:25.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.644+0000 7fa973fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa974101de0 0x7fa974198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.644+0000 7fa973fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa974101de0 0x7fa974198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46442/0 (socket says 172.21.15.103:46442) 2024-09-19T06:51:25.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.644+0000 7fa973fff640 1 -- 172.21.15.103:0/1887104812 learned_addr learned my addr 172.21.15.103:0/1887104812 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:25.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.645+0000 7fa978f91640 1 --2- 172.21.15.103:0/1887104812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa974106f10 0x7fa97419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.645+0000 7fa9737fe640 1 --2- 172.21.15.103:0/1887104812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa974102790 0x7fa9741988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.645+0000 7fa973fff640 1 -- 172.21.15.103:0/1887104812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa974106f10 msgr2=0x7fa97419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.645+0000 7fa973fff640 1 --2- 172.21.15.103:0/1887104812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa974106f10 0x7fa97419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.645+0000 7fa973fff640 1 -- 172.21.15.103:0/1887104812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa974102790 msgr2=0x7fa9741988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.645+0000 7fa973fff640 1 --2- 172.21.15.103:0/1887104812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa974102790 0x7fa9741988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.645+0000 7fa973fff640 1 -- 172.21.15.103:0/1887104812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97419d340 con 0x7fa974101de0 2024-09-19T06:51:25.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.645+0000 7fa9737fe640 1 --2- 172.21.15.103:0/1887104812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa974102790 0x7fa9741988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:25.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.645+0000 7fa973fff640 1 --2- 172.21.15.103:0/1887104812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa974101de0 0x7fa974198390 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa964006010 tx=0x7fa9640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:25.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.645+0000 7fa9717fa640 1 -- 172.21.15.103:0/1887104812 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96403f030 con 0x7fa974101de0 2024-09-19T06:51:25.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.645+0000 7fa9717fa640 1 -- 172.21.15.103:0/1887104812 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa964003ce0 con 0x7fa974101de0 2024-09-19T06:51:25.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.645+0000 7fa97a9f4640 1 -- 172.21.15.103:0/1887104812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97419d5d0 con 0x7fa974101de0 2024-09-19T06:51:25.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.645+0000 7fa9717fa640 1 -- 172.21.15.103:0/1887104812 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964035070 con 0x7fa974101de0 2024-09-19T06:51:25.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.645+0000 7fa97a9f4640 1 -- 172.21.15.103:0/1887104812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9741a4f10 con 0x7fa974101de0 2024-09-19T06:51:25.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.647+0000 7fa9717fa640 1 -- 172.21.15.103:0/1887104812 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa964042030 con 0x7fa974101de0 2024-09-19T06:51:25.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.648+0000 7fa97cdcd740 1 Processor -- start 2024-09-19T06:51:25.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.648+0000 7fa97cdcd740 1 -- start start 2024-09-19T06:51:25.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.648+0000 7fa9717fa640 1 --2- 172.21.15.103:0/1887104812 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa95c0a33b0 0x7fa95c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.648+0000 7fa97cdcd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab62b40 0x557adab61090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.648+0000 7fa978f91640 1 --2- 172.21.15.103:0/1887104812 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa95c0a33b0 0x7fa95c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.648+0000 7fa97cdcd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab615d0 0x557adab61a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.648+0000 7fa97cdcd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab6c370 0x557adab6e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.648+0000 7fa9717fa640 1 -- 172.21.15.103:0/1887104812 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa9640f03d0 con 0x7fa974101de0 2024-09-19T06:51:25.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.648+0000 7fa97cdcd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557adaac72f0 con 0x557adab6c370 2024-09-19T06:51:25.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.648+0000 7fa97cdcd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557adaa6ecf0 con 0x557adab62b40 2024-09-19T06:51:25.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.648+0000 7fa97cdcd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557adaac32b0 con 0x557adab615d0 2024-09-19T06:51:25.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.648+0000 7fa978f91640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab6c370 0x557adab6e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.648+0000 7fa978f91640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab6c370 0x557adab6e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46454/0 (socket says 172.21.15.103:46454) 2024-09-19T06:51:25.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.648+0000 7fa978f91640 1 -- 172.21.15.103:0/867693989 learned_addr learned my addr 172.21.15.103:0/867693989 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:25.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.648+0000 7fa9737fe640 1 --2- 172.21.15.103:0/867693989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab615d0 0x557adab61a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.648+0000 7fa973fff640 1 --2- 172.21.15.103:0/867693989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab62b40 0x557adab61090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.649+0000 7fa978f91640 1 --2- 172.21.15.103:0/1887104812 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa95c0a33b0 0x7fa95c0a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa96c014820 tx=0x7fa96c014c20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:25.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.649+0000 7fa978f91640 1 -- 172.21.15.103:0/867693989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab615d0 msgr2=0x557adab61a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:25.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.649+0000 7fa978f91640 1 --2- 172.21.15.103:0/867693989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab615d0 0x557adab61a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.649+0000 7fa978f91640 1 -- 172.21.15.103:0/867693989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab62b40 msgr2=0x557adab61090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:25.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.649+0000 7fa978f91640 1 --2- 172.21.15.103:0/867693989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab62b40 0x557adab61090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.649+0000 7fa978f91640 1 -- 172.21.15.103:0/867693989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557adaa533a0 con 0x557adab6c370 2024-09-19T06:51:25.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.649+0000 7fa978f91640 1 --2- 172.21.15.103:0/867693989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab6c370 0x557adab6e760 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa96c012ac0 tx=0x7fa96c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c602a5dc97c730f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:25.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.649+0000 7fa95a7fc640 1 -- 172.21.15.103:0/867693989 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c003b70 con 0x557adab6c370 2024-09-19T06:51:25.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.649+0000 7fa95a7fc640 1 -- 172.21.15.103:0/867693989 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa96c01ed90 con 0x557adab6c370 2024-09-19T06:51:25.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.649+0000 7fa95a7fc640 1 -- 172.21.15.103:0/867693989 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c020440 con 0x557adab6c370 2024-09-19T06:51:25.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.650+0000 7fa97cdcd740 1 -- 172.21.15.103:0/867693989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab6c370 msgr2=0x557adab6e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.650+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/867693989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab6c370 0x557adab6e760 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa96c012ac0 tx=0x7fa96c01e000 comp rx=0 tx=0).stop 2024-09-19T06:51:25.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.650+0000 7fa97cdcd740 1 -- 172.21.15.103:0/867693989 shutdown_connections 2024-09-19T06:51:25.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.650+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/867693989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab6c370 0x557adab6e760 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.650+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/867693989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab615d0 0x557adab61a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.650+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/867693989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab62b40 0x557adab61090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.650+0000 7fa97cdcd740 1 -- 172.21.15.103:0/867693989 >> 172.21.15.103:0/867693989 conn(0x557adab2a310 msgr2=0x557adab5dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:25.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.650+0000 7fa97cdcd740 1 -- 172.21.15.103:0/867693989 shutdown_connections 2024-09-19T06:51:25.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.650+0000 7fa97cdcd740 1 -- 172.21.15.103:0/867693989 wait complete. 2024-09-19T06:51:25.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.650+0000 7fa97cdcd740 1 -- start start 2024-09-19T06:51:25.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.650+0000 7fa97cdcd740 1 Processor -- start 2024-09-19T06:51:25.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa97cdcd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab615d0 0x557adab6bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa97cdcd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab62b40 0x557adab69020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa97cdcd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab6c370 0x557adab69610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa97cdcd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557adab6f010 con 0x557adab62b40 2024-09-19T06:51:25.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa97cdcd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557adaa82060 con 0x557adab615d0 2024-09-19T06:51:25.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa97cdcd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557adab6f190 con 0x557adab6c370 2024-09-19T06:51:25.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa97cdcd740 10 client.0 _ll_register_callbacks cb 0x557ada7d58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:25.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa9737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab62b40 0x557adab69020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa9737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab62b40 0x557adab69020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46470/0 (socket says 172.21.15.103:46470) 2024-09-19T06:51:25.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa9737fe640 1 -- 172.21.15.103:0/2892686271 learned_addr learned my addr 172.21.15.103:0/2892686271 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:25.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa978f91640 1 --2- 172.21.15.103:0/2892686271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab6c370 0x557adab69610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa973fff640 1 --2- 172.21.15.103:0/2892686271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab615d0 0x557adab6bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa9737fe640 1 -- 172.21.15.103:0/2892686271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab6c370 msgr2=0x557adab69610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:25.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa9737fe640 1 --2- 172.21.15.103:0/2892686271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab6c370 0x557adab69610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa9737fe640 1 -- 172.21.15.103:0/2892686271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab615d0 msgr2=0x557adab6bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:25.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa9737fe640 1 --2- 172.21.15.103:0/2892686271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab615d0 0x557adab6bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa9737fe640 1 -- 172.21.15.103:0/2892686271 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ada8383c0 con 0x557adab62b40 2024-09-19T06:51:25.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa9737fe640 1 --2- 172.21.15.103:0/2892686271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab62b40 0x557adab69020 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa968010a50 tx=0x7fa968010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a08bc049efb228 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:25.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa9537fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:25.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.651+0000 7fa9537fe640 1 -- 172.21.15.103:0/2892686271 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96801d060 con 0x557adab62b40 2024-09-19T06:51:25.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.652+0000 7fa9537fe640 1 -- 172.21.15.103:0/2892686271 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa96800eb20 con 0x557adab62b40 2024-09-19T06:51:25.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.652+0000 7fa97cdcd740 1 -- 172.21.15.103:0/2892686271 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa96c01ed90 con 0x557adab62b40 2024-09-19T06:51:25.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.652+0000 7fa9537fe640 1 -- 172.21.15.103:0/2892686271 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa96800ed20 con 0x557adab62b40 2024-09-19T06:51:25.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.652+0000 7fa97cdcd740 10 client.16302 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:25.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.652+0000 7fa97cdcd740 1 -- 172.21.15.103:0/2892686271 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557adab7ab10 con 0x557adab62b40 2024-09-19T06:51:25.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.652+0000 7fa9537fe640 1 -- 172.21.15.103:0/2892686271 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa96c01ed90 con 0x557adab62b40 2024-09-19T06:51:25.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.652+0000 7fa9537fe640 1 -- 172.21.15.103:0/2892686271 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa9680174d0 con 0x557adab62b40 2024-09-19T06:51:25.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.652+0000 7fa97cdcd740 10 client.16302 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:25.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.652+0000 7fa97cdcd740 10 client.16302 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:51:25.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.652+0000 7fa97cdcd740 20 client.16302 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:25.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.652+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/2892686271 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x557adab7c020 0x557adab7e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.652+0000 7fa97cdcd740 4 client.16302 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:25.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.652+0000 7fa97cdcd740 1 -- 172.21.15.103:0/2892686271 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557adaad4860 con 0x557adab7c020 2024-09-19T06:51:25.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.653+0000 7fa973fff640 1 --2- 172.21.15.103:0/2892686271 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x557adab7c020 0x557adab7e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.653+0000 7fa973fff640 1 --2- 172.21.15.103:0/2892686271 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x557adab7c020 0x557adab7e430 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ebd118c02a2073d server_cookie=852663db2234426d in_seq=0 out_seq=0 2024-09-19T06:51:25.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.653+0000 7fa9537fe640 10 client.16302 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:51:25.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.654+0000 7fa9537fe640 1 -- 172.21.15.103:0/2892686271 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557adaad4860 con 0x557adab7c020 2024-09-19T06:51:25.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.654+0000 7fa9537fe640 10 client.16302 handle_command_reply: tid=0 2024-09-19T06:51:25.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.654+0000 7fa97cdcd740 1 client.16302 shutdown 2024-09-19T06:51:25.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.654+0000 7fa97cdcd740 1 -- 172.21.15.103:0/2892686271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab62b40 msgr2=0x557adab69020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.654+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/2892686271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab62b40 0x557adab69020 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa968010a50 tx=0x7fa968010f10 comp rx=0 tx=0).stop 2024-09-19T06:51:25.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.654+0000 7fa97cdcd740 1 -- 172.21.15.103:0/2892686271 shutdown_connections 2024-09-19T06:51:25.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.654+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/2892686271 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x557adab7c020 0x557adab7e430 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.654+0000 7fa9537fe640 0 client.16302 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:51:25.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.654+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/2892686271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab6c370 0x557adab69610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.654+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/2892686271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab62b40 0x557adab69020 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.654+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/2892686271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab615d0 0x557adab6bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.654+0000 7fa97cdcd740 1 -- 172.21.15.103:0/2892686271 >> 172.21.15.103:0/2892686271 conn(0x557adab2a310 msgr2=0x557adab63960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:25.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.654+0000 7fa97cdcd740 1 -- 172.21.15.103:0/2892686271 shutdown_connections 2024-09-19T06:51:25.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.654+0000 7fa97cdcd740 1 -- 172.21.15.103:0/2892686271 wait complete. 2024-09-19T06:51:25.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.654+0000 7fa97cdcd740 20 client.16302 trim_cache size 0 max 16384 2024-09-19T06:51:25.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa97cdcd740 1 Processor -- start 2024-09-19T06:51:25.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa97cdcd740 1 -- start start 2024-09-19T06:51:25.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa97cdcd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab615d0 0x557adab6af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa97cdcd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab62b40 0x557adab64120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa97cdcd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab6c370 0x557adab64660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa97cdcd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa9741ad7f0 con 0x557adab62b40 2024-09-19T06:51:25.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa97cdcd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9741af330 con 0x557adab6c370 2024-09-19T06:51:25.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa97cdcd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa9741adec0 con 0x557adab615d0 2024-09-19T06:51:25.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa9737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab62b40 0x557adab64120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa9737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab62b40 0x557adab64120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46476/0 (socket says 172.21.15.103:46476) 2024-09-19T06:51:25.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa9737fe640 1 -- 172.21.15.103:0/1542793163 learned_addr learned my addr 172.21.15.103:0/1542793163 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:25.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa9737fe640 1 -- 172.21.15.103:0/1542793163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab615d0 msgr2=0x557adab6af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:25.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa9737fe640 1 --2- 172.21.15.103:0/1542793163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab615d0 0x557adab6af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa9737fe640 1 -- 172.21.15.103:0/1542793163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab6c370 msgr2=0x557adab64660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:25.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa9737fe640 1 --2- 172.21.15.103:0/1542793163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab6c370 0x557adab64660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa9737fe640 1 -- 172.21.15.103:0/1542793163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557adab5e960 con 0x557adab62b40 2024-09-19T06:51:25.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.667+0000 7fa9737fe640 1 --2- 172.21.15.103:0/1542793163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab62b40 0x557adab64120 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa9680178c0 tx=0x7fa96800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6837eb7bbdabf758 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:25.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.668+0000 7fa959ffb640 1 -- 172.21.15.103:0/1542793163 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96802e070 con 0x557adab62b40 2024-09-19T06:51:25.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.668+0000 7fa959ffb640 1 -- 172.21.15.103:0/1542793163 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa968005520 con 0x557adab62b40 2024-09-19T06:51:25.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.668+0000 7fa959ffb640 1 -- 172.21.15.103:0/1542793163 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968004660 con 0x557adab62b40 2024-09-19T06:51:25.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.668+0000 7fa97cdcd740 1 -- 172.21.15.103:0/1542793163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab62b40 msgr2=0x557adab64120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.668+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/1542793163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab62b40 0x557adab64120 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa9680178c0 tx=0x7fa96800b8e0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.668+0000 7fa97cdcd740 1 -- 172.21.15.103:0/1542793163 shutdown_connections 2024-09-19T06:51:25.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.668+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/1542793163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab6c370 0x557adab64660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.668+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/1542793163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab62b40 0x557adab64120 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.668+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/1542793163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab615d0 0x557adab6af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.668+0000 7fa97cdcd740 1 -- 172.21.15.103:0/1542793163 >> 172.21.15.103:0/1542793163 conn(0x557adab2a310 msgr2=0x557adab6a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:25.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.668+0000 7fa97cdcd740 1 -- 172.21.15.103:0/1542793163 shutdown_connections 2024-09-19T06:51:25.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.668+0000 7fa97cdcd740 1 -- 172.21.15.103:0/1542793163 wait complete. 2024-09-19T06:51:25.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.669+0000 7fa97cdcd740 1 -- start start 2024-09-19T06:51:25.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.669+0000 7fa97cdcd740 1 Processor -- start 2024-09-19T06:51:25.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.669+0000 7fa97cdcd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab615d0 0x557adabe2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.669+0000 7fa97cdcd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab62b40 0x557adab74420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.669+0000 7fa97cdcd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab6c370 0x557adab74a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.669+0000 7fa97cdcd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa9741aec60 con 0x557adab615d0 2024-09-19T06:51:25.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.669+0000 7fa97cdcd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9741ae590 con 0x557adab6c370 2024-09-19T06:51:25.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.669+0000 7fa97cdcd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa9741ab7b0 con 0x557adab62b40 2024-09-19T06:51:25.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.669+0000 7fa97cdcd740 10 client.0 _ll_register_callbacks cb 0x7fa96800b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:25.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.669+0000 7fa973fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab615d0 0x557adabe2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.669+0000 7fa973fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab615d0 0x557adabe2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46488/0 (socket says 172.21.15.103:46488) 2024-09-19T06:51:25.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.669+0000 7fa973fff640 1 -- 172.21.15.103:0/3644818502 learned_addr learned my addr 172.21.15.103:0/3644818502 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:25.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.669+0000 7fa978f91640 1 --2- 172.21.15.103:0/3644818502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab6c370 0x557adab74a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa9737fe640 1 --2- 172.21.15.103:0/3644818502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab62b40 0x557adab74420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa973fff640 1 -- 172.21.15.103:0/3644818502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab62b40 msgr2=0x557adab74420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:25.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa973fff640 1 --2- 172.21.15.103:0/3644818502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab62b40 0x557adab74420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa973fff640 1 -- 172.21.15.103:0/3644818502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab6c370 msgr2=0x557adab74a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:25.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa973fff640 1 --2- 172.21.15.103:0/3644818502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab6c370 0x557adab74a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa973fff640 1 -- 172.21.15.103:0/3644818502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557adab5d4f0 con 0x557adab615d0 2024-09-19T06:51:25.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa9737fe640 1 --2- 172.21.15.103:0/3644818502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab62b40 0x557adab74420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:25.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa978f91640 1 --2- 172.21.15.103:0/3644818502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab6c370 0x557adab74a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:25.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa973fff640 1 --2- 172.21.15.103:0/3644818502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab615d0 0x557adabe2ad0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa964032c00 tx=0x7fa9640bc0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2c80a628b379cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:25.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa9537fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:25.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa9537fe640 1 -- 172.21.15.103:0/3644818502 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9640d9070 con 0x557adab615d0 2024-09-19T06:51:25.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa9537fe640 1 -- 172.21.15.103:0/3644818502 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa9640b8df0 con 0x557adab615d0 2024-09-19T06:51:25.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa97cdcd740 1 -- 172.21.15.103:0/3644818502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa968005520 con 0x557adab615d0 2024-09-19T06:51:25.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa9537fe640 1 -- 172.21.15.103:0/3644818502 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa964049040 con 0x557adab615d0 2024-09-19T06:51:25.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa97cdcd740 10 client.16308 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:25.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa97cdcd740 1 -- 172.21.15.103:0/3644818502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557adabefa60 con 0x557adab615d0 2024-09-19T06:51:25.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa9537fe640 1 -- 172.21.15.103:0/3644818502 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa968005520 con 0x557adab615d0 2024-09-19T06:51:25.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa9537fe640 1 -- 172.21.15.103:0/3644818502 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa9640b8040 con 0x557adab615d0 2024-09-19T06:51:25.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa97cdcd740 10 client.16308 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:25.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.670+0000 7fa97cdcd740 10 client.16308 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:51:25.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.671+0000 7fa97cdcd740 20 client.16308 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:25.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.671+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/3644818502 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x557adab7c020 0x557adabf2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.671+0000 7fa97cdcd740 4 client.16308 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:51:25.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.671+0000 7fa97cdcd740 1 -- 172.21.15.103:0/3644818502 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557adaaced70 con 0x557adab7c020 2024-09-19T06:51:25.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.671+0000 7fa9737fe640 1 --2- 172.21.15.103:0/3644818502 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x557adab7c020 0x557adabf2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.671+0000 7fa9537fe640 10 client.16308 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:51:25.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.671+0000 7fa9737fe640 1 --2- 172.21.15.103:0/3644818502 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x557adab7c020 0x557adabf2170 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4f70c3192a6f884f server_cookie=214d81314984109f in_seq=0 out_seq=0 2024-09-19T06:51:25.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.671+0000 7fa9537fe640 1 -- 172.21.15.103:0/3644818502 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557adaaced70 con 0x557adab7c020 2024-09-19T06:51:25.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.671+0000 7fa9537fe640 10 client.16308 handle_command_reply: tid=0 2024-09-19T06:51:25.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.671+0000 7fa97cdcd740 1 client.16308 shutdown 2024-09-19T06:51:25.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.672+0000 7fa97cdcd740 1 -- 172.21.15.103:0/3644818502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab615d0 msgr2=0x557adabe2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.672+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/3644818502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab615d0 0x557adabe2ad0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa964032c00 tx=0x7fa9640bc0e0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.672+0000 7fa97cdcd740 1 -- 172.21.15.103:0/3644818502 shutdown_connections 2024-09-19T06:51:25.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.672+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/3644818502 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x557adab7c020 0x557adabf2170 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.672+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/3644818502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adab6c370 0x557adab74a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.672+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/3644818502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557adab62b40 0x557adab74420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.672+0000 7fa97cdcd740 1 --2- 172.21.15.103:0/3644818502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557adab615d0 0x557adabe2ad0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.672+0000 7fa97cdcd740 1 -- 172.21.15.103:0/3644818502 >> 172.21.15.103:0/3644818502 conn(0x557adab2a310 msgr2=0x557adab6b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:25.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.672+0000 7fa9537fe640 0 client.16308 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:51:25.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.672+0000 7fa97cdcd740 1 -- 172.21.15.103:0/3644818502 shutdown_connections 2024-09-19T06:51:25.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.672+0000 7fa97cdcd740 1 -- 172.21.15.103:0/3644818502 wait complete. 2024-09-19T06:51:25.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.672+0000 7fa97cdcd740 20 client.16308 trim_cache size 0 max 16384 2024-09-19T06:51:25.679 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:51:25.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.672+0000 7fa97a9f4640 1 -- 172.21.15.103:0/1887104812 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa95c0a33b0 msgr2=0x7fa95c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.672+0000 7fa97a9f4640 1 --2- 172.21.15.103:0/1887104812 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa95c0a33b0 0x7fa95c0a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa96c014820 tx=0x7fa96c014c20 comp rx=0 tx=0).stop 2024-09-19T06:51:25.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.672+0000 7fa97a9f4640 1 -- 172.21.15.103:0/1887104812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa974101de0 msgr2=0x7fa974198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.672+0000 7fa97a9f4640 1 --2- 172.21.15.103:0/1887104812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa974101de0 0x7fa974198390 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa964006010 tx=0x7fa9640042f0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.673+0000 7fa97a9f4640 1 -- 172.21.15.103:0/1887104812 shutdown_connections 2024-09-19T06:51:25.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.673+0000 7fa97a9f4640 1 --2- 172.21.15.103:0/1887104812 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa95c0a33b0 0x7fa95c0a5870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.673+0000 7fa97a9f4640 1 --2- 172.21.15.103:0/1887104812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa974106f10 0x7fa97419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.673+0000 7fa97a9f4640 1 --2- 172.21.15.103:0/1887104812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa974102790 0x7fa9741988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.673+0000 7fa97a9f4640 1 --2- 172.21.15.103:0/1887104812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa974101de0 0x7fa974198390 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.673+0000 7fa97a9f4640 1 -- 172.21.15.103:0/1887104812 >> 172.21.15.103:0/1887104812 conn(0x7fa9740fd8c0 msgr2=0x7fa9740fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:25.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.673+0000 7fa97a9f4640 1 -- 172.21.15.103:0/1887104812 shutdown_connections 2024-09-19T06:51:25.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:25.673+0000 7fa97a9f4640 1 -- 172.21.15.103:0/1887104812 wait complete. 2024-09-19T06:51:26.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:51:26.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:26 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:51:26.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:26 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:51:26.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:26 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:26.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:26 smithi103 ceph-mon[31158]: pgmap v69: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.7 KiB/s wr, 15 op/s 2024-09-19T06:51:26.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:26 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:51:26.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:26 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:51:26.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:26 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:26.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:26 smithi134 ceph-mon[35356]: pgmap v69: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.7 KiB/s wr, 15 op/s 2024-09-19T06:51:26.564 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:26.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:26 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:51:26.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:26 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:51:26.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:26 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:51:26.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:26 smithi174 ceph-mon[35637]: pgmap v69: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.7 KiB/s wr, 15 op/s 2024-09-19T06:51:27.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.995+0000 7f253137d640 1 -- 172.21.15.103:0/70596085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f252c101420 msgr2=0x7f252c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:27.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.995+0000 7f253137d640 1 --2- 172.21.15.103:0/70596085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f252c101420 0x7f252c10d950 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2524009cd0 tx=0x7f252402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.995+0000 7f253137d640 1 -- 172.21.15.103:0/70596085 shutdown_connections 2024-09-19T06:51:27.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.995+0000 7f253137d640 1 --2- 172.21.15.103:0/70596085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f252c101420 0x7f252c10d950 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.995+0000 7f253137d640 1 --2- 172.21.15.103:0/70596085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f252c100a80 0x7f252c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.995+0000 7f253137d640 1 --2- 172.21.15.103:0/70596085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f252c106a80 0x7f252c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.995+0000 7f253137d640 1 -- 172.21.15.103:0/70596085 >> 172.21.15.103:0/70596085 conn(0x7f252c0fc7a0 msgr2=0x7f252c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:27.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.995+0000 7f253137d640 1 -- 172.21.15.103:0/70596085 shutdown_connections 2024-09-19T06:51:27.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.995+0000 7f253137d640 1 -- 172.21.15.103:0/70596085 wait complete. 2024-09-19T06:51:27.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.995+0000 7f253137d640 1 Processor -- start 2024-09-19T06:51:27.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f253137d640 1 -- start start 2024-09-19T06:51:27.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f253137d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f252c100a80 0x7f252c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f253137d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f252c101420 0x7f252c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f253137d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f252c106a80 0x7f252c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f253137d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f252c110ea0 con 0x7f252c100a80 2024-09-19T06:51:27.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f253137d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f252c110d20 con 0x7f252c106a80 2024-09-19T06:51:27.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f253137d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f252c111020 con 0x7f252c101420 2024-09-19T06:51:27.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f252affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f252c100a80 0x7f252c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:27.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f252affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f252c100a80 0x7f252c1984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46496/0 (socket says 172.21.15.103:46496) 2024-09-19T06:51:27.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f252affd640 1 -- 172.21.15.103:0/2795312243 learned_addr learned my addr 172.21.15.103:0/2795312243 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:27.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f2523fff640 1 --2- 172.21.15.103:0/2795312243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f252c101420 0x7f252c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:27.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f252affd640 1 -- 172.21.15.103:0/2795312243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f252c101420 msgr2=0x7f252c198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:27.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f252affd640 1 --2- 172.21.15.103:0/2795312243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f252c101420 0x7f252c198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f252affd640 1 -- 172.21.15.103:0/2795312243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f252c106a80 msgr2=0x7f252c19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:27.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f252affd640 1 --2- 172.21.15.103:0/2795312243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f252c106a80 0x7f252c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f252affd640 1 -- 172.21.15.103:0/2795312243 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252c19d490 con 0x7f252c100a80 2024-09-19T06:51:27.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f252affd640 1 --2- 172.21.15.103:0/2795312243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f252c100a80 0x7f252c1984e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f251c00b4a0 tx=0x7f251c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:27.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f2528ff9640 1 -- 172.21.15.103:0/2795312243 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c014070 con 0x7f252c100a80 2024-09-19T06:51:27.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.996+0000 7f2528ff9640 1 -- 172.21.15.103:0/2795312243 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f251c010af0 con 0x7f252c100a80 2024-09-19T06:51:27.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.997+0000 7f253137d640 1 -- 172.21.15.103:0/2795312243 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f252c19d780 con 0x7f252c100a80 2024-09-19T06:51:27.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.997+0000 7f2528ff9640 1 -- 172.21.15.103:0/2795312243 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c017d10 con 0x7f252c100a80 2024-09-19T06:51:27.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.997+0000 7f253137d640 1 -- 172.21.15.103:0/2795312243 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f252c1a5060 con 0x7f252c100a80 2024-09-19T06:51:27.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.999+0000 7f2528ff9640 1 -- 172.21.15.103:0/2795312243 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f251c017070 con 0x7f252c100a80 2024-09-19T06:51:27.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.999+0000 7f2533756740 1 Processor -- start 2024-09-19T06:51:27.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.999+0000 7f2533756740 1 -- start start 2024-09-19T06:51:27.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.999+0000 7f2533756740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c38ab40 0x56378c389090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.999+0000 7f2533756740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c3895d0 0x56378c389a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.999+0000 7f2533756740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c394370 0x56378c396760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.999+0000 7f2533756740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56378c2ef2f0 con 0x56378c38ab40 2024-09-19T06:51:27.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.999+0000 7f2533756740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56378c296cf0 con 0x56378c3895d0 2024-09-19T06:51:27.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.999+0000 7f2533756740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56378c2eb2b0 con 0x56378c394370 2024-09-19T06:51:27.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:26.999+0000 7f2528ff9640 1 --2- 172.21.15.103:0/2795312243 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f25180a33b0 0x7f25180a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f252b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c38ab40 0x56378c389090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:27.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f252b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c38ab40 0x56378c389090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46508/0 (socket says 172.21.15.103:46508) 2024-09-19T06:51:27.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f252b7fe640 1 -- 172.21.15.103:0/3374791687 learned_addr learned my addr 172.21.15.103:0/3374791687 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:27.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f252b7fe640 1 --2- 172.21.15.103:0/2795312243 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f25180a33b0 0x7f25180a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:27.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f2523fff640 1 --2- 172.21.15.103:0/3374791687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c394370 0x56378c396760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:27.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f2528ff9640 1 -- 172.21.15.103:0/2795312243 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f251c0cf080 con 0x7f252c100a80 2024-09-19T06:51:27.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f252affd640 1 --2- 172.21.15.103:0/3374791687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c3895d0 0x56378c389a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:27.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f252b7fe640 1 -- 172.21.15.103:0/3374791687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c394370 msgr2=0x56378c396760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:27.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f252b7fe640 1 --2- 172.21.15.103:0/3374791687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c394370 0x56378c396760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f252b7fe640 1 -- 172.21.15.103:0/3374791687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c3895d0 msgr2=0x56378c389a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:27.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f252b7fe640 1 --2- 172.21.15.103:0/3374791687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c3895d0 0x56378c389a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f252b7fe640 1 -- 172.21.15.103:0/3374791687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56378c27b3a0 con 0x56378c38ab40 2024-09-19T06:51:27.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f252b7fe640 1 --2- 172.21.15.103:0/3374791687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c38ab40 0x56378c389090 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2524005e30 tx=0x7f2524031d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7f0cfa2c5f5356 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:27.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f252b7fe640 1 --2- 172.21.15.103:0/2795312243 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f25180a33b0 0x7f25180a5870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f252402ae00 tx=0x7f252403f000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:27.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f25217fa640 1 -- 172.21.15.103:0/3374791687 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2524009860 con 0x56378c38ab40 2024-09-19T06:51:27.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f25217fa640 1 -- 172.21.15.103:0/3374791687 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2524036050 con 0x56378c38ab40 2024-09-19T06:51:27.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.000+0000 7f25217fa640 1 -- 172.21.15.103:0/3374791687 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252403c410 con 0x56378c38ab40 2024-09-19T06:51:27.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.001+0000 7f2533756740 1 -- 172.21.15.103:0/3374791687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c38ab40 msgr2=0x56378c389090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:27.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.001+0000 7f2533756740 1 --2- 172.21.15.103:0/3374791687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c38ab40 0x56378c389090 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2524005e30 tx=0x7f2524031d50 comp rx=0 tx=0).stop 2024-09-19T06:51:27.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.001+0000 7f2533756740 1 -- 172.21.15.103:0/3374791687 shutdown_connections 2024-09-19T06:51:27.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.001+0000 7f2533756740 1 --2- 172.21.15.103:0/3374791687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c394370 0x56378c396760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.001+0000 7f2533756740 1 --2- 172.21.15.103:0/3374791687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c3895d0 0x56378c389a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.001+0000 7f2533756740 1 --2- 172.21.15.103:0/3374791687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c38ab40 0x56378c389090 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.001+0000 7f2533756740 1 -- 172.21.15.103:0/3374791687 >> 172.21.15.103:0/3374791687 conn(0x56378c352310 msgr2=0x56378c385c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:27.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.001+0000 7f2533756740 1 -- 172.21.15.103:0/3374791687 shutdown_connections 2024-09-19T06:51:27.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.001+0000 7f2533756740 1 -- 172.21.15.103:0/3374791687 wait complete. 2024-09-19T06:51:27.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f2533756740 1 -- start start 2024-09-19T06:51:27.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f2533756740 1 Processor -- start 2024-09-19T06:51:27.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f2533756740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c3895d0 0x56378c393da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f2533756740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c38ab40 0x56378c391050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f2533756740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c394370 0x56378c3917e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f2533756740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56378c39b420 con 0x56378c394370 2024-09-19T06:51:27.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f2533756740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56378c2aa060 con 0x56378c38ab40 2024-09-19T06:51:27.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f2533756740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56378c39b5a0 con 0x56378c3895d0 2024-09-19T06:51:27.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f2533756740 10 client.0 _ll_register_callbacks cb 0x56378bffd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:27.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f252b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c394370 0x56378c3917e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:27.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f252b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c394370 0x56378c3917e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46510/0 (socket says 172.21.15.103:46510) 2024-09-19T06:51:27.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f252b7fe640 1 -- 172.21.15.103:0/3570567159 learned_addr learned my addr 172.21.15.103:0/3570567159 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:27.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f252affd640 1 --2- 172.21.15.103:0/3570567159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c3895d0 0x56378c393da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:27.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f2523fff640 1 --2- 172.21.15.103:0/3570567159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c38ab40 0x56378c391050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:27.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f252b7fe640 1 -- 172.21.15.103:0/3570567159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c3895d0 msgr2=0x56378c393da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:27.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f252b7fe640 1 --2- 172.21.15.103:0/3570567159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c3895d0 0x56378c393da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.002+0000 7f252b7fe640 1 -- 172.21.15.103:0/3570567159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c38ab40 msgr2=0x56378c391050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:27.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f252b7fe640 1 --2- 172.21.15.103:0/3570567159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c38ab40 0x56378c391050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f252b7fe640 1 -- 172.21.15.103:0/3570567159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56378c0603c0 con 0x56378c394370 2024-09-19T06:51:27.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f2523fff640 1 --2- 172.21.15.103:0/3570567159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c38ab40 0x56378c391050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:27.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f252affd640 1 --2- 172.21.15.103:0/3570567159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c3895d0 0x56378c393da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:27.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f252b7fe640 1 --2- 172.21.15.103:0/3570567159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c394370 0x56378c3917e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2524005e00 tx=0x7f2524003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240ccdbc78fb9644 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:27.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f25127fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:27.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f25127fc640 1 -- 172.21.15.103:0/3570567159 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2524004140 con 0x56378c394370 2024-09-19T06:51:27.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f25127fc640 1 -- 172.21.15.103:0/3570567159 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f25240042e0 con 0x56378c394370 2024-09-19T06:51:27.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f2533756740 1 -- 172.21.15.103:0/3570567159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2524036050 con 0x56378c394370 2024-09-19T06:51:27.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f25127fc640 1 -- 172.21.15.103:0/3570567159 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2524060050 con 0x56378c394370 2024-09-19T06:51:27.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f2533756740 10 client.16320 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:27.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f2533756740 1 -- 172.21.15.103:0/3570567159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56378c3a6f00 con 0x56378c394370 2024-09-19T06:51:27.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f25127fc640 1 -- 172.21.15.103:0/3570567159 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2524036050 con 0x56378c394370 2024-09-19T06:51:27.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f25127fc640 1 -- 172.21.15.103:0/3570567159 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2524030070 con 0x56378c394370 2024-09-19T06:51:27.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f2533756740 10 client.16320 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:27.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f2533756740 10 client.16320 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:51:27.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.003+0000 7f2533756740 20 client.16320 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:27.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.004+0000 7f2533756740 1 --2- 172.21.15.103:0/3570567159 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56378c3a8250 0x56378c3aa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.004+0000 7f2533756740 4 client.16320 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:27.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.004+0000 7f2533756740 1 -- 172.21.15.103:0/3570567159 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56378c2fc860 con 0x56378c3a8250 2024-09-19T06:51:27.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.004+0000 7f252affd640 1 --2- 172.21.15.103:0/3570567159 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56378c3a8250 0x56378c3aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:27.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.004+0000 7f25127fc640 10 client.16320 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:51:27.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.004+0000 7f252affd640 1 --2- 172.21.15.103:0/3570567159 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56378c3a8250 0x56378c3aa640 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8eb35ec8095c1bbb server_cookie=2e7d04576970fe in_seq=0 out_seq=0 2024-09-19T06:51:27.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.005+0000 7f25127fc640 1 -- 172.21.15.103:0/3570567159 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56378c2fc860 con 0x56378c3a8250 2024-09-19T06:51:27.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.005+0000 7f25127fc640 10 client.16320 handle_command_reply: tid=0 2024-09-19T06:51:27.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.005+0000 7f2533756740 1 client.16320 shutdown 2024-09-19T06:51:27.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.005+0000 7f2533756740 1 -- 172.21.15.103:0/3570567159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c394370 msgr2=0x56378c3917e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:27.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.005+0000 7f2533756740 1 --2- 172.21.15.103:0/3570567159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c394370 0x56378c3917e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2524005e00 tx=0x7f2524003ca0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.005+0000 7f2533756740 1 -- 172.21.15.103:0/3570567159 shutdown_connections 2024-09-19T06:51:27.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.005+0000 7f2533756740 1 --2- 172.21.15.103:0/3570567159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c394370 0x56378c3917e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.005+0000 7f2533756740 1 --2- 172.21.15.103:0/3570567159 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56378c3a8250 0x56378c3aa640 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.005+0000 7f2533756740 1 --2- 172.21.15.103:0/3570567159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c38ab40 0x56378c391050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.005+0000 7f25127fc640 0 client.16320 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:51:27.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.005+0000 7f2533756740 1 --2- 172.21.15.103:0/3570567159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c3895d0 0x56378c393da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.005+0000 7f2533756740 1 -- 172.21.15.103:0/3570567159 >> 172.21.15.103:0/3570567159 conn(0x56378c352310 msgr2=0x56378c38b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:27.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.005+0000 7f2533756740 1 -- 172.21.15.103:0/3570567159 shutdown_connections 2024-09-19T06:51:27.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.005+0000 7f2533756740 1 -- 172.21.15.103:0/3570567159 wait complete. 2024-09-19T06:51:27.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.005+0000 7f2533756740 20 client.16320 trim_cache size 0 max 16384 2024-09-19T06:51:27.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.018+0000 7f2533756740 1 Processor -- start 2024-09-19T06:51:27.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.018+0000 7f2533756740 1 -- start start 2024-09-19T06:51:27.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.018+0000 7f2533756740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c3895d0 0x56378c390910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.018+0000 7f2533756740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c38ab40 0x56378c38c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.018+0000 7f2533756740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c394370 0x56378c38c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.018+0000 7f2533756740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f252c1ab710 con 0x56378c394370 2024-09-19T06:51:27.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.018+0000 7f2533756740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f252c1ad250 con 0x56378c3895d0 2024-09-19T06:51:27.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.018+0000 7f2533756740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f252c1abde0 con 0x56378c38ab40 2024-09-19T06:51:27.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.019+0000 7f252b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c394370 0x56378c38c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:27.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.019+0000 7f252b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c394370 0x56378c38c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46516/0 (socket says 172.21.15.103:46516) 2024-09-19T06:51:27.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.019+0000 7f252b7fe640 1 -- 172.21.15.103:0/3281216647 learned_addr learned my addr 172.21.15.103:0/3281216647 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:27.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.019+0000 7f252affd640 1 --2- 172.21.15.103:0/3281216647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c3895d0 0x56378c390910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:27.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.019+0000 7f252b7fe640 1 -- 172.21.15.103:0/3281216647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c38ab40 msgr2=0x56378c38c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:27.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.019+0000 7f252b7fe640 1 --2- 172.21.15.103:0/3281216647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c38ab40 0x56378c38c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.019+0000 7f252b7fe640 1 -- 172.21.15.103:0/3281216647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c3895d0 msgr2=0x56378c390910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:27.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.019+0000 7f252b7fe640 1 --2- 172.21.15.103:0/3281216647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c3895d0 0x56378c390910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.019+0000 7f252b7fe640 1 -- 172.21.15.103:0/3281216647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56378c38f5c0 con 0x56378c394370 2024-09-19T06:51:27.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.019+0000 7f252affd640 1 --2- 172.21.15.103:0/3281216647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c3895d0 0x56378c390910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:27.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.019+0000 7f252b7fe640 1 --2- 172.21.15.103:0/3281216647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c394370 0x56378c38c6c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2524006010 tx=0x7f2524004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2589d644627d6575 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:27.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.019+0000 7f2520ff9640 1 -- 172.21.15.103:0/3281216647 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2524031cd0 con 0x56378c394370 2024-09-19T06:51:27.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.019+0000 7f2520ff9640 1 -- 172.21.15.103:0/3281216647 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2524036040 con 0x56378c394370 2024-09-19T06:51:27.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.019+0000 7f2520ff9640 1 -- 172.21.15.103:0/3281216647 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252404f810 con 0x56378c394370 2024-09-19T06:51:27.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.020+0000 7f2533756740 1 -- 172.21.15.103:0/3281216647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c394370 msgr2=0x56378c38c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:27.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.020+0000 7f2533756740 1 --2- 172.21.15.103:0/3281216647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c394370 0x56378c38c6c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2524006010 tx=0x7f2524004900 comp rx=0 tx=0).stop 2024-09-19T06:51:27.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.020+0000 7f2533756740 1 -- 172.21.15.103:0/3281216647 shutdown_connections 2024-09-19T06:51:27.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.020+0000 7f2533756740 1 --2- 172.21.15.103:0/3281216647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c394370 0x56378c38c6c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.020+0000 7f2533756740 1 --2- 172.21.15.103:0/3281216647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c38ab40 0x56378c38c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.020+0000 7f2533756740 1 --2- 172.21.15.103:0/3281216647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c3895d0 0x56378c390910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.020+0000 7f2533756740 1 -- 172.21.15.103:0/3281216647 >> 172.21.15.103:0/3281216647 conn(0x56378c352310 msgr2=0x56378c390160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:27.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.020+0000 7f2533756740 1 -- 172.21.15.103:0/3281216647 shutdown_connections 2024-09-19T06:51:27.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.020+0000 7f2533756740 1 -- 172.21.15.103:0/3281216647 wait complete. 2024-09-19T06:51:27.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.020+0000 7f2533756740 1 -- start start 2024-09-19T06:51:27.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.020+0000 7f2533756740 1 Processor -- start 2024-09-19T06:51:27.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f2533756740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c3895d0 0x56378c3a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f2533756740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c38ab40 0x56378c38dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f2533756740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c394370 0x56378c38e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f2533756740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f252c1acb80 con 0x56378c3895d0 2024-09-19T06:51:27.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f2533756740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f252c1ac4b0 con 0x56378c38ab40 2024-09-19T06:51:27.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f2533756740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f252c1a96d0 con 0x56378c394370 2024-09-19T06:51:27.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f2533756740 10 client.0 _ll_register_callbacks cb 0x7f251400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:27.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f252affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c3895d0 0x56378c3a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:27.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f252affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c3895d0 0x56378c3a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46528/0 (socket says 172.21.15.103:46528) 2024-09-19T06:51:27.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f252affd640 1 -- 172.21.15.103:0/2656289026 learned_addr learned my addr 172.21.15.103:0/2656289026 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:27.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f252b7fe640 1 --2- 172.21.15.103:0/2656289026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c394370 0x56378c38e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:27.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f2523fff640 1 --2- 172.21.15.103:0/2656289026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c38ab40 0x56378c38dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:27.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f252affd640 1 -- 172.21.15.103:0/2656289026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c394370 msgr2=0x56378c38e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:27.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f252affd640 1 --2- 172.21.15.103:0/2656289026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c394370 0x56378c38e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f252affd640 1 -- 172.21.15.103:0/2656289026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c38ab40 msgr2=0x56378c38dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:27.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f252affd640 1 --2- 172.21.15.103:0/2656289026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c38ab40 0x56378c38dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f252affd640 1 -- 172.21.15.103:0/2656289026 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56378c38e5d0 con 0x56378c3895d0 2024-09-19T06:51:27.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f252b7fe640 1 --2- 172.21.15.103:0/2656289026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c394370 0x56378c38e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:27.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f252affd640 1 --2- 172.21.15.103:0/2656289026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c3895d0 0x56378c3a0cb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f251c0978e0 tx=0x7f251c06b0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4df3b02c3bd02b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:27.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f25127fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:27.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f25127fc640 1 -- 172.21.15.103:0/2656289026 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c01e040 con 0x56378c3895d0 2024-09-19T06:51:27.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f25127fc640 1 -- 172.21.15.103:0/2656289026 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f251c010ca0 con 0x56378c3895d0 2024-09-19T06:51:27.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f2533756740 1 -- 172.21.15.103:0/2656289026 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2524036040 con 0x56378c3895d0 2024-09-19T06:51:27.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f25127fc640 1 -- 172.21.15.103:0/2656289026 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f251c09f830 con 0x56378c3895d0 2024-09-19T06:51:27.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f2533756740 10 client.16326 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:27.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.021+0000 7f2533756740 1 -- 172.21.15.103:0/2656289026 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56378c412a80 con 0x56378c3895d0 2024-09-19T06:51:27.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.022+0000 7f25127fc640 1 -- 172.21.15.103:0/2656289026 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2524036040 con 0x56378c3895d0 2024-09-19T06:51:27.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.022+0000 7f25127fc640 1 -- 172.21.15.103:0/2656289026 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f251c0cca60 con 0x56378c3895d0 2024-09-19T06:51:27.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.022+0000 7f2533756740 10 client.16326 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:27.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.022+0000 7f2533756740 10 client.16326 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:51:27.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.022+0000 7f2533756740 20 client.16326 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:27.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.022+0000 7f2533756740 1 --2- 172.21.15.103:0/2656289026 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56378c413150 0x56378c415570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:27.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.022+0000 7f2533756740 4 client.16326 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:51:27.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.022+0000 7f2533756740 1 -- 172.21.15.103:0/2656289026 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56378c2f6d70 con 0x56378c413150 2024-09-19T06:51:27.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.022+0000 7f2523fff640 1 --2- 172.21.15.103:0/2656289026 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56378c413150 0x56378c415570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:27.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f25127fc640 10 client.16326 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:51:27.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f2523fff640 1 --2- 172.21.15.103:0/2656289026 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56378c413150 0x56378c415570 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=12dadca2722b4377 server_cookie=b2eef83c821d7bb8 in_seq=0 out_seq=0 2024-09-19T06:51:27.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f25127fc640 1 -- 172.21.15.103:0/2656289026 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56378c2f6d70 con 0x56378c413150 2024-09-19T06:51:27.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f25127fc640 10 client.16326 handle_command_reply: tid=0 2024-09-19T06:51:27.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f2533756740 1 client.16326 shutdown 2024-09-19T06:51:27.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f2533756740 1 -- 172.21.15.103:0/2656289026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c3895d0 msgr2=0x56378c3a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:27.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f2533756740 1 --2- 172.21.15.103:0/2656289026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c3895d0 0x56378c3a0cb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f251c0978e0 tx=0x7f251c06b0a0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f2533756740 1 -- 172.21.15.103:0/2656289026 shutdown_connections 2024-09-19T06:51:27.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f2533756740 1 --2- 172.21.15.103:0/2656289026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56378c394370 0x56378c38e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f2533756740 1 --2- 172.21.15.103:0/2656289026 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56378c413150 0x56378c415570 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f2533756740 1 --2- 172.21.15.103:0/2656289026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56378c38ab40 0x56378c38dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f2533756740 1 --2- 172.21.15.103:0/2656289026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56378c3895d0 0x56378c3a0cb0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f25127fc640 0 client.16326 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:51:27.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f2533756740 1 -- 172.21.15.103:0/2656289026 >> 172.21.15.103:0/2656289026 conn(0x56378c352310 msgr2=0x56378c3928b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:27.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f2533756740 1 -- 172.21.15.103:0/2656289026 shutdown_connections 2024-09-19T06:51:27.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f2533756740 1 -- 172.21.15.103:0/2656289026 wait complete. 2024-09-19T06:51:27.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.023+0000 7f2533756740 20 client.16326 trim_cache size 0 max 16384 2024-09-19T06:51:27.041 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:51:27.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.024+0000 7f253137d640 1 -- 172.21.15.103:0/2795312243 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f25180a33b0 msgr2=0x7f25180a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:27.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.024+0000 7f253137d640 1 --2- 172.21.15.103:0/2795312243 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f25180a33b0 0x7f25180a5870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f252402ae00 tx=0x7f252403f000 comp rx=0 tx=0).stop 2024-09-19T06:51:27.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.024+0000 7f253137d640 1 -- 172.21.15.103:0/2795312243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f252c100a80 msgr2=0x7f252c1984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:27.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.024+0000 7f253137d640 1 --2- 172.21.15.103:0/2795312243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f252c100a80 0x7f252c1984e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f251c00b4a0 tx=0x7f251c00b960 comp rx=0 tx=0).stop 2024-09-19T06:51:27.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.024+0000 7f253137d640 1 -- 172.21.15.103:0/2795312243 shutdown_connections 2024-09-19T06:51:27.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.024+0000 7f253137d640 1 --2- 172.21.15.103:0/2795312243 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f25180a33b0 0x7f25180a5870 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.024+0000 7f253137d640 1 --2- 172.21.15.103:0/2795312243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f252c106a80 0x7f252c19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.024+0000 7f253137d640 1 --2- 172.21.15.103:0/2795312243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f252c101420 0x7f252c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.024+0000 7f253137d640 1 --2- 172.21.15.103:0/2795312243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f252c100a80 0x7f252c1984e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.024+0000 7f253137d640 1 -- 172.21.15.103:0/2795312243 >> 172.21.15.103:0/2795312243 conn(0x7f252c0fc7a0 msgr2=0x7f252c06a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:27.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.024+0000 7f253137d640 1 -- 172.21.15.103:0/2795312243 shutdown_connections 2024-09-19T06:51:27.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:27.024+0000 7f253137d640 1 -- 172.21.15.103:0/2795312243 wait complete. 2024-09-19T06:51:27.657 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:51:27.906 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:28.279 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:28 smithi103 ceph-mon[31158]: pgmap v70: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-19T06:51:28.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:28 smithi174 ceph-mon[35637]: pgmap v70: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-19T06:51:28.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.355+0000 7fc7e40aa640 1 -- 172.21.15.103:0/2738036867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7dc100e30 msgr2=0x7fc7dc104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.355+0000 7fc7e40aa640 1 --2- 172.21.15.103:0/2738036867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7dc100e30 0x7fc7dc104e80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0009900 tx=0x7fc7d002a410 comp rx=0 tx=0).stop 2024-09-19T06:51:28.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.355+0000 7fc7e40aa640 1 -- 172.21.15.103:0/2738036867 shutdown_connections 2024-09-19T06:51:28.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.355+0000 7fc7e40aa640 1 --2- 172.21.15.103:0/2738036867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7dc1055b0 0x7fc7dc10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.355+0000 7fc7e40aa640 1 --2- 172.21.15.103:0/2738036867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7dc100e30 0x7fc7dc104e80 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.355+0000 7fc7e40aa640 1 --2- 172.21.15.103:0/2738036867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7dc100480 0x7fc7dc100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.355+0000 7fc7e40aa640 1 -- 172.21.15.103:0/2738036867 >> 172.21.15.103:0/2738036867 conn(0x7fc7dc077210 msgr2=0x7fc7dc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:28.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.355+0000 7fc7e40aa640 1 -- 172.21.15.103:0/2738036867 shutdown_connections 2024-09-19T06:51:28.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.355+0000 7fc7e40aa640 1 -- 172.21.15.103:0/2738036867 wait complete. 2024-09-19T06:51:28.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.355+0000 7fc7e40aa640 1 Processor -- start 2024-09-19T06:51:28.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e40aa640 1 -- start start 2024-09-19T06:51:28.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e40aa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7dc100480 0x7fc7dc1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e40aa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7dc100e30 0x7fc7dc199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e40aa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7dc1055b0 0x7fc7dc1a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e40aa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc1105d0 con 0x7fc7dc100e30 2024-09-19T06:51:28.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e40aa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc110450 con 0x7fc7dc1055b0 2024-09-19T06:51:28.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e40aa640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc110750 con 0x7fc7dc100480 2024-09-19T06:51:28.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e1645640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7dc100e30 0x7fc7dc199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e1645640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7dc100e30 0x7fc7dc199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54026/0 (socket says 172.21.15.103:54026) 2024-09-19T06:51:28.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e1645640 1 -- 172.21.15.103:0/3254213228 learned_addr learned my addr 172.21.15.103:0/3254213228 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:28.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e1e46640 1 --2- 172.21.15.103:0/3254213228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7dc100480 0x7fc7dc1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e2647640 1 --2- 172.21.15.103:0/3254213228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7dc1055b0 0x7fc7dc1a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e1645640 1 -- 172.21.15.103:0/3254213228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7dc100480 msgr2=0x7fc7dc1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e1645640 1 --2- 172.21.15.103:0/3254213228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7dc100480 0x7fc7dc1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e1645640 1 -- 172.21.15.103:0/3254213228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7dc1055b0 msgr2=0x7fc7dc1a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e1645640 1 --2- 172.21.15.103:0/3254213228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7dc1055b0 0x7fc7dc1a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.356+0000 7fc7e1645640 1 -- 172.21.15.103:0/3254213228 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7dc1a48b0 con 0x7fc7dc100e30 2024-09-19T06:51:28.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.357+0000 7fc7e1645640 1 --2- 172.21.15.103:0/3254213228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7dc100e30 0x7fc7dc199830 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0006010 tx=0x7fc7d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:28.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.357+0000 7fc7daffd640 1 -- 172.21.15.103:0/3254213228 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d003f030 con 0x7fc7dc100e30 2024-09-19T06:51:28.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.357+0000 7fc7daffd640 1 -- 172.21.15.103:0/3254213228 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d0003ce0 con 0x7fc7dc100e30 2024-09-19T06:51:28.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.357+0000 7fc7e40aa640 1 -- 172.21.15.103:0/3254213228 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7dc1a4a80 con 0x7fc7dc100e30 2024-09-19T06:51:28.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.357+0000 7fc7daffd640 1 -- 172.21.15.103:0/3254213228 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0035070 con 0x7fc7dc100e30 2024-09-19T06:51:28.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.357+0000 7fc7e40aa640 1 -- 172.21.15.103:0/3254213228 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7dc1a4ed0 con 0x7fc7dc100e30 2024-09-19T06:51:28.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.359+0000 7fc7daffd640 1 -- 172.21.15.103:0/3254213228 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc7d0042030 con 0x7fc7dc100e30 2024-09-19T06:51:28.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.359+0000 7fc7e6483740 1 Processor -- start 2024-09-19T06:51:28.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.360+0000 7fc7e6483740 1 -- start start 2024-09-19T06:51:28.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.360+0000 7fc7e6483740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da0020b40 0x556da001f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.360+0000 7fc7e6483740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da001f5d0 0x556da001fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.360+0000 7fc7e6483740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da002e780 0x556da0030b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.360+0000 7fc7e6483740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556d9ff852f0 con 0x556da002e780 2024-09-19T06:51:28.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.360+0000 7fc7e6483740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556d9ff2ccf0 con 0x556da0020b40 2024-09-19T06:51:28.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.360+0000 7fc7e6483740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556d9ff812b0 con 0x556da001f5d0 2024-09-19T06:51:28.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.360+0000 7fc7daffd640 1 --2- 172.21.15.103:0/3254213228 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7c40a33b0 0x7fc7c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.360+0000 7fc7e1645640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da002e780 0x556da0030b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.360+0000 7fc7e2647640 1 --2- 172.21.15.103:0/3254213228 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7c40a33b0 0x7fc7c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.360+0000 7fc7e2647640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da0020b40 0x556da001f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.360+0000 7fc7e1645640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da002e780 0x556da0030b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54036/0 (socket says 172.21.15.103:54036) 2024-09-19T06:51:28.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.360+0000 7fc7e1645640 1 -- 172.21.15.103:0/1176896349 learned_addr learned my addr 172.21.15.103:0/1176896349 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:28.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.360+0000 7fc7e1e46640 1 --2- 172.21.15.103:0/1176896349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da001f5d0 0x556da001fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.360+0000 7fc7daffd640 1 -- 172.21.15.103:0/3254213228 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc7d00efec0 con 0x7fc7dc100e30 2024-09-19T06:51:28.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.361+0000 7fc7e1645640 1 -- 172.21.15.103:0/1176896349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da001f5d0 msgr2=0x556da001fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:28.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.361+0000 7fc7e1645640 1 --2- 172.21.15.103:0/1176896349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da001f5d0 0x556da001fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.361+0000 7fc7e1645640 1 -- 172.21.15.103:0/1176896349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da0020b40 msgr2=0x556da001f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:28.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.361+0000 7fc7e1645640 1 --2- 172.21.15.103:0/1176896349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da0020b40 0x556da001f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.361+0000 7fc7e1645640 1 -- 172.21.15.103:0/1176896349 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d9ff113a0 con 0x556da002e780 2024-09-19T06:51:28.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.361+0000 7fc7e2647640 1 --2- 172.21.15.103:0/1176896349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da0020b40 0x556da001f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:28.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.361+0000 7fc7e2647640 1 --2- 172.21.15.103:0/3254213228 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7c40a33b0 0x7fc7c40a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4005940 tx=0x7fc7d40058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:28.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.361+0000 7fc7e1645640 1 --2- 172.21.15.103:0/1176896349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da002e780 0x556da0030b70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00c48b0 tx=0x7fc7d008f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafdac7c3c9a0c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:28.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.361+0000 7fc7c3fff640 1 -- 172.21.15.103:0/1176896349 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0049070 con 0x556da002e780 2024-09-19T06:51:28.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.361+0000 7fc7c3fff640 1 -- 172.21.15.103:0/1176896349 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d0090070 con 0x556da002e780 2024-09-19T06:51:28.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.361+0000 7fc7c3fff640 1 -- 172.21.15.103:0/1176896349 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d00dc0d0 con 0x556da002e780 2024-09-19T06:51:28.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.362+0000 7fc7e6483740 1 -- 172.21.15.103:0/1176896349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da002e780 msgr2=0x556da0030b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.362+0000 7fc7e6483740 1 --2- 172.21.15.103:0/1176896349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da002e780 0x556da0030b70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00c48b0 tx=0x7fc7d008f110 comp rx=0 tx=0).stop 2024-09-19T06:51:28.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.362+0000 7fc7e6483740 1 -- 172.21.15.103:0/1176896349 shutdown_connections 2024-09-19T06:51:28.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.362+0000 7fc7e6483740 1 --2- 172.21.15.103:0/1176896349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da002e780 0x556da0030b70 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.362+0000 7fc7e6483740 1 --2- 172.21.15.103:0/1176896349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da001f5d0 0x556da001fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.362+0000 7fc7e6483740 1 --2- 172.21.15.103:0/1176896349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da0020b40 0x556da001f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.362+0000 7fc7e6483740 1 -- 172.21.15.103:0/1176896349 >> 172.21.15.103:0/1176896349 conn(0x556d9ffe8310 msgr2=0x556da001bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:28.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.362+0000 7fc7e6483740 1 -- 172.21.15.103:0/1176896349 shutdown_connections 2024-09-19T06:51:28.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.362+0000 7fc7e6483740 1 -- 172.21.15.103:0/1176896349 wait complete. 2024-09-19T06:51:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.362+0000 7fc7e6483740 1 -- start start 2024-09-19T06:51:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.362+0000 7fc7e6483740 1 Processor -- start 2024-09-19T06:51:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e6483740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da001f5d0 0x556da002e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e6483740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da0020b40 0x556da002b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e6483740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da002e780 0x556da002bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e6483740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556da0031240 con 0x556da0020b40 2024-09-19T06:51:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e6483740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556d9ff40060 con 0x556da002e780 2024-09-19T06:51:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e6483740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556da00313c0 con 0x556da001f5d0 2024-09-19T06:51:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e6483740 10 client.0 _ll_register_callbacks cb 0x556d9fc938c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e1645640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da0020b40 0x556da002b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e1645640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da0020b40 0x556da002b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54048/0 (socket says 172.21.15.103:54048) 2024-09-19T06:51:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e1645640 1 -- 172.21.15.103:0/3419843575 learned_addr learned my addr 172.21.15.103:0/3419843575 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e1e46640 1 --2- 172.21.15.103:0/3419843575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da001f5d0 0x556da002e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e2647640 1 --2- 172.21.15.103:0/3419843575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da002e780 0x556da002bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e1645640 1 -- 172.21.15.103:0/3419843575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da001f5d0 msgr2=0x556da002e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e1645640 1 --2- 172.21.15.103:0/3419843575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da001f5d0 0x556da002e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e1645640 1 -- 172.21.15.103:0/3419843575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da002e780 msgr2=0x556da002bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e1645640 1 --2- 172.21.15.103:0/3419843575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da002e780 0x556da002bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e1645640 1 -- 172.21.15.103:0/3419843575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d9fcf63c0 con 0x556da0020b40 2024-09-19T06:51:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e2647640 1 --2- 172.21.15.103:0/3419843575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da002e780 0x556da002bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7e1645640 1 --2- 172.21.15.103:0/3419843575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da0020b40 0x556da002b460 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00b8bf0 tx=0x7fc7d00b8ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1d3a3d7f42068 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.363+0000 7fc7c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.364+0000 7fc7c17fa640 1 -- 172.21.15.103:0/3419843575 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0049040 con 0x556da0020b40 2024-09-19T06:51:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.364+0000 7fc7c17fa640 1 -- 172.21.15.103:0/3419843575 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc7d00c0170 con 0x556da0020b40 2024-09-19T06:51:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.364+0000 7fc7e6483740 1 -- 172.21.15.103:0/3419843575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7d0090070 con 0x556da0020b40 2024-09-19T06:51:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.364+0000 7fc7c17fa640 1 -- 172.21.15.103:0/3419843575 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc7d003bcb0 con 0x556da0020b40 2024-09-19T06:51:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.364+0000 7fc7e6483740 10 client.16338 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.364+0000 7fc7e6483740 1 -- 172.21.15.103:0/3419843575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556da0031240 con 0x556da0020b40 2024-09-19T06:51:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.364+0000 7fc7c17fa640 1 -- 172.21.15.103:0/3419843575 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d0090070 con 0x556da0020b40 2024-09-19T06:51:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.364+0000 7fc7c17fa640 1 -- 172.21.15.103:0/3419843575 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc7d004c060 con 0x556da0020b40 2024-09-19T06:51:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.364+0000 7fc7e6483740 10 client.16338 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.364+0000 7fc7e6483740 10 client.16338 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:51:28.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.364+0000 7fc7e6483740 20 client.16338 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:28.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.364+0000 7fc7e6483740 1 --2- 172.21.15.103:0/3419843575 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x556da003dea0 0x556da0040290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.364+0000 7fc7e6483740 4 client.16338 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:28.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.364+0000 7fc7e6483740 1 -- 172.21.15.103:0/3419843575 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d9ff92860 con 0x556da003dea0 2024-09-19T06:51:28.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.365+0000 7fc7e1e46640 1 --2- 172.21.15.103:0/3419843575 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x556da003dea0 0x556da0040290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.365+0000 7fc7e1e46640 1 --2- 172.21.15.103:0/3419843575 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x556da003dea0 0x556da0040290 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6fe883315459fbd8 server_cookie=542c9c4328de5b4 in_seq=0 out_seq=0 2024-09-19T06:51:28.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.365+0000 7fc7c17fa640 10 client.16338 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:51:28.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.366+0000 7fc7c17fa640 1 -- 172.21.15.103:0/3419843575 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d9ff92860 con 0x556da003dea0 2024-09-19T06:51:28.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.366+0000 7fc7c17fa640 10 client.16338 handle_command_reply: tid=0 2024-09-19T06:51:28.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.366+0000 7fc7e6483740 1 client.16338 shutdown 2024-09-19T06:51:28.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.366+0000 7fc7e6483740 1 -- 172.21.15.103:0/3419843575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da0020b40 msgr2=0x556da002b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.366+0000 7fc7e6483740 1 --2- 172.21.15.103:0/3419843575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da0020b40 0x556da002b460 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00b8bf0 tx=0x7fc7d00b8ac0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.366+0000 7fc7e6483740 1 -- 172.21.15.103:0/3419843575 shutdown_connections 2024-09-19T06:51:28.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.366+0000 7fc7e6483740 1 --2- 172.21.15.103:0/3419843575 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x556da003dea0 0x556da0040290 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.366+0000 7fc7e6483740 1 --2- 172.21.15.103:0/3419843575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da002e780 0x556da002bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.366+0000 7fc7e6483740 1 --2- 172.21.15.103:0/3419843575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da0020b40 0x556da002b460 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.366+0000 7fc7e6483740 1 --2- 172.21.15.103:0/3419843575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da001f5d0 0x556da002e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.366+0000 7fc7c17fa640 0 client.16338 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:51:28.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.366+0000 7fc7e6483740 1 -- 172.21.15.103:0/3419843575 >> 172.21.15.103:0/3419843575 conn(0x556d9ffe8310 msgr2=0x556da0021990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:28.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.366+0000 7fc7e6483740 1 -- 172.21.15.103:0/3419843575 shutdown_connections 2024-09-19T06:51:28.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.366+0000 7fc7e6483740 1 -- 172.21.15.103:0/3419843575 wait complete. 2024-09-19T06:51:28.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.366+0000 7fc7e6483740 20 client.16338 trim_cache size 0 max 16384 2024-09-19T06:51:28.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e6483740 1 Processor -- start 2024-09-19T06:51:28.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e6483740 1 -- start start 2024-09-19T06:51:28.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e6483740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da001f5d0 0x556da002a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e6483740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da0020b40 0x556da0045b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e6483740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da002e780 0x556da0046080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e6483740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc1ab750 con 0x556da0020b40 2024-09-19T06:51:28.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e6483740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc1ad290 con 0x556da002e780 2024-09-19T06:51:28.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e6483740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc1abe20 con 0x556da001f5d0 2024-09-19T06:51:28.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e1645640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da0020b40 0x556da0045b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e1645640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da0020b40 0x556da0045b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54062/0 (socket says 172.21.15.103:54062) 2024-09-19T06:51:28.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e1645640 1 -- 172.21.15.103:0/2839119404 learned_addr learned my addr 172.21.15.103:0/2839119404 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:28.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e1645640 1 -- 172.21.15.103:0/2839119404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da001f5d0 msgr2=0x556da002a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:28.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e1645640 1 --2- 172.21.15.103:0/2839119404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da001f5d0 0x556da002a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e1645640 1 -- 172.21.15.103:0/2839119404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da002e780 msgr2=0x556da0046080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:28.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e1645640 1 --2- 172.21.15.103:0/2839119404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da002e780 0x556da0046080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e1645640 1 -- 172.21.15.103:0/2839119404 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556da0022120 con 0x556da0020b40 2024-09-19T06:51:28.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.379+0000 7fc7e1645640 1 --2- 172.21.15.103:0/2839119404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da0020b40 0x556da0045b40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00bc480 tx=0x7fc7d00edb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297d43e897f64890 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:28.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.380+0000 7fc7c37fe640 1 -- 172.21.15.103:0/2839119404 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0049070 con 0x556da0020b40 2024-09-19T06:51:28.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.380+0000 7fc7c37fe640 1 -- 172.21.15.103:0/2839119404 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d00ed860 con 0x556da0020b40 2024-09-19T06:51:28.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.380+0000 7fc7c37fe640 1 -- 172.21.15.103:0/2839119404 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d00c4950 con 0x556da0020b40 2024-09-19T06:51:28.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.380+0000 7fc7e6483740 1 -- 172.21.15.103:0/2839119404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da0020b40 msgr2=0x556da0045b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.380+0000 7fc7e6483740 1 --2- 172.21.15.103:0/2839119404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da0020b40 0x556da0045b40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00bc480 tx=0x7fc7d00edb20 comp rx=0 tx=0).stop 2024-09-19T06:51:28.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.380+0000 7fc7e6483740 1 -- 172.21.15.103:0/2839119404 shutdown_connections 2024-09-19T06:51:28.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.380+0000 7fc7e6483740 1 --2- 172.21.15.103:0/2839119404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da002e780 0x556da0046080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.380+0000 7fc7e6483740 1 --2- 172.21.15.103:0/2839119404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da0020b40 0x556da0045b40 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.380+0000 7fc7e6483740 1 --2- 172.21.15.103:0/2839119404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da001f5d0 0x556da002a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.380+0000 7fc7e6483740 1 -- 172.21.15.103:0/2839119404 >> 172.21.15.103:0/2839119404 conn(0x556d9ffe8310 msgr2=0x556da0029d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:28.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.380+0000 7fc7e6483740 1 -- 172.21.15.103:0/2839119404 shutdown_connections 2024-09-19T06:51:28.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.380+0000 7fc7e6483740 1 -- 172.21.15.103:0/2839119404 wait complete. 2024-09-19T06:51:28.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.381+0000 7fc7e6483740 1 -- start start 2024-09-19T06:51:28.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.381+0000 7fc7e6483740 1 Processor -- start 2024-09-19T06:51:28.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.381+0000 7fc7e6483740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da001f5d0 0x556da0036c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.381+0000 7fc7e6483740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da0020b40 0x556da00281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.381+0000 7fc7e6483740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da002e780 0x556da00286f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.381+0000 7fc7e6483740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc1acbc0 con 0x556da002e780 2024-09-19T06:51:28.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.381+0000 7fc7e6483740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc1ac4f0 con 0x556da001f5d0 2024-09-19T06:51:28.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.381+0000 7fc7e6483740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc1a9710 con 0x556da0020b40 2024-09-19T06:51:28.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.381+0000 7fc7e6483740 10 client.0 _ll_register_callbacks cb 0x7fc7d400aec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:28.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.381+0000 7fc7e2647640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da002e780 0x556da00286f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.381+0000 7fc7e2647640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da002e780 0x556da00286f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54074/0 (socket says 172.21.15.103:54074) 2024-09-19T06:51:28.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.381+0000 7fc7e2647640 1 -- 172.21.15.103:0/3351912357 learned_addr learned my addr 172.21.15.103:0/3351912357 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:28.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.381+0000 7fc7e1e46640 1 --2- 172.21.15.103:0/3351912357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da001f5d0 0x556da0036c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.381+0000 7fc7e1645640 1 --2- 172.21.15.103:0/3351912357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da0020b40 0x556da00281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.381+0000 7fc7e2647640 1 -- 172.21.15.103:0/3351912357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da0020b40 msgr2=0x556da00281b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:28.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.382+0000 7fc7e2647640 1 --2- 172.21.15.103:0/3351912357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da0020b40 0x556da00281b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.382+0000 7fc7e2647640 1 -- 172.21.15.103:0/3351912357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da001f5d0 msgr2=0x556da0036c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:28.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.382+0000 7fc7e2647640 1 --2- 172.21.15.103:0/3351912357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da001f5d0 0x556da0036c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.382+0000 7fc7e2647640 1 -- 172.21.15.103:0/3351912357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556da0028ed0 con 0x556da002e780 2024-09-19T06:51:28.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.382+0000 7fc7e1645640 1 --2- 172.21.15.103:0/3351912357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da0020b40 0x556da00281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:28.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.382+0000 7fc7e2647640 1 --2- 172.21.15.103:0/3351912357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da002e780 0x556da00286f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400a8a0 tx=0x7fc7d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23decc416117044 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:28.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.382+0000 7fc7c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:28.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.382+0000 7fc7c17fa640 1 -- 172.21.15.103:0/3351912357 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4024070 con 0x556da002e780 2024-09-19T06:51:28.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.382+0000 7fc7c17fa640 1 -- 172.21.15.103:0/3351912357 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc7d4005220 con 0x556da002e780 2024-09-19T06:51:28.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.382+0000 7fc7e6483740 1 -- 172.21.15.103:0/3351912357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7d00ed860 con 0x556da002e780 2024-09-19T06:51:28.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.382+0000 7fc7c17fa640 1 -- 172.21.15.103:0/3351912357 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc7d4005420 con 0x556da002e780 2024-09-19T06:51:28.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.382+0000 7fc7e6483740 10 client.16344 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:28.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.382+0000 7fc7e6483740 1 -- 172.21.15.103:0/3351912357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556da00a8b20 con 0x556da002e780 2024-09-19T06:51:28.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.382+0000 7fc7c17fa640 1 -- 172.21.15.103:0/3351912357 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d00ed860 con 0x556da002e780 2024-09-19T06:51:28.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.383+0000 7fc7c17fa640 1 -- 172.21.15.103:0/3351912357 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc7d40048e0 con 0x556da002e780 2024-09-19T06:51:28.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.383+0000 7fc7e6483740 10 client.16344 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:28.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.383+0000 7fc7e6483740 10 client.16344 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:51:28.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.383+0000 7fc7e6483740 20 client.16344 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:28.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.383+0000 7fc7e6483740 1 --2- 172.21.15.103:0/3351912357 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x556da00a9300 0x556da00ab720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.383+0000 7fc7e6483740 4 client.16344 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:51:28.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.383+0000 7fc7e6483740 1 -- 172.21.15.103:0/3351912357 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d9ff8cd70 con 0x556da00a9300 2024-09-19T06:51:28.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.383+0000 7fc7e1e46640 1 --2- 172.21.15.103:0/3351912357 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x556da00a9300 0x556da00ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.383+0000 7fc7c17fa640 10 client.16344 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:51:28.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.383+0000 7fc7e1e46640 1 --2- 172.21.15.103:0/3351912357 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x556da00a9300 0x556da00ab720 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c919a37a40140e08 server_cookie=49e317ac90c434c in_seq=0 out_seq=0 2024-09-19T06:51:28.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7c17fa640 1 -- 172.21.15.103:0/3351912357 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d9ff8cd70 con 0x556da00a9300 2024-09-19T06:51:28.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7c17fa640 10 client.16344 handle_command_reply: tid=0 2024-09-19T06:51:28.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7e6483740 1 client.16344 shutdown 2024-09-19T06:51:28.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7e6483740 1 -- 172.21.15.103:0/3351912357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da002e780 msgr2=0x556da00286f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7e6483740 1 --2- 172.21.15.103:0/3351912357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da002e780 0x556da00286f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400a8a0 tx=0x7fc7d4004730 comp rx=0 tx=0).stop 2024-09-19T06:51:28.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7e6483740 1 -- 172.21.15.103:0/3351912357 shutdown_connections 2024-09-19T06:51:28.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7e6483740 1 --2- 172.21.15.103:0/3351912357 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x556da00a9300 0x556da00ab720 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7e6483740 1 --2- 172.21.15.103:0/3351912357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da002e780 0x556da00286f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7e6483740 1 --2- 172.21.15.103:0/3351912357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da0020b40 0x556da00281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7c17fa640 0 client.16344 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:51:28.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7e6483740 1 --2- 172.21.15.103:0/3351912357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da001f5d0 0x556da0036c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7e6483740 1 -- 172.21.15.103:0/3351912357 >> 172.21.15.103:0/3351912357 conn(0x556d9ffe8310 msgr2=0x556da0021110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:28.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7e6483740 1 -- 172.21.15.103:0/3351912357 shutdown_connections 2024-09-19T06:51:28.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7e6483740 1 -- 172.21.15.103:0/3351912357 wait complete. 2024-09-19T06:51:28.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7e6483740 20 client.16344 trim_cache size 0 max 16384 2024-09-19T06:51:28.393 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:51:28.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7e40aa640 1 -- 172.21.15.103:0/3254213228 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7c40a33b0 msgr2=0x7fc7c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7e40aa640 1 --2- 172.21.15.103:0/3254213228 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7c40a33b0 0x7fc7c40a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4005940 tx=0x7fc7d40058d0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7e40aa640 1 -- 172.21.15.103:0/3254213228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7dc100e30 msgr2=0x7fc7dc199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.384+0000 7fc7e40aa640 1 --2- 172.21.15.103:0/3254213228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7dc100e30 0x7fc7dc199830 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0006010 tx=0x7fc7d00042f0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.385+0000 7fc7e40aa640 1 -- 172.21.15.103:0/3254213228 shutdown_connections 2024-09-19T06:51:28.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.385+0000 7fc7e40aa640 1 --2- 172.21.15.103:0/3254213228 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7c40a33b0 0x7fc7c40a5870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.385+0000 7fc7e40aa640 1 --2- 172.21.15.103:0/3254213228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7dc1055b0 0x7fc7dc1a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.385+0000 7fc7e40aa640 1 --2- 172.21.15.103:0/3254213228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7dc100e30 0x7fc7dc199830 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.385+0000 7fc7e40aa640 1 --2- 172.21.15.103:0/3254213228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7dc100480 0x7fc7dc1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.385+0000 7fc7e40aa640 1 -- 172.21.15.103:0/3254213228 >> 172.21.15.103:0/3254213228 conn(0x7fc7dc077210 msgr2=0x7fc7dc103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:28.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.385+0000 7fc7e40aa640 1 -- 172.21.15.103:0/3254213228 shutdown_connections 2024-09-19T06:51:28.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:28.385+0000 7fc7e40aa640 1 -- 172.21.15.103:0/3254213228 wait complete. 2024-09-19T06:51:28.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:28 smithi134 ceph-mon[35356]: pgmap v70: 97 pgs: 97 active+clean; 1.5 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-19T06:51:29.110 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:51:29.357 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:29.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.855+0000 7f79f4dcc640 1 -- 172.21.15.103:0/4168897549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79f0102d40 msgr2=0x7f79f0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:29.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.855+0000 7f79f4dcc640 1 --2- 172.21.15.103:0/4168897549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79f0102d40 0x7f79f0103120 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009900 tx=0x7f79e002a480 comp rx=0 tx=0).stop 2024-09-19T06:51:29.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.855+0000 7f79f4dcc640 1 -- 172.21.15.103:0/4168897549 shutdown_connections 2024-09-19T06:51:29.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.855+0000 7f79f4dcc640 1 --2- 172.21.15.103:0/4168897549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79f010c170 0x7f79f010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.855+0000 7f79f4dcc640 1 --2- 172.21.15.103:0/4168897549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79f0103660 0x7f79f010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.855+0000 7f79f4dcc640 1 --2- 172.21.15.103:0/4168897549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79f0102d40 0x7f79f0103120 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.855+0000 7f79f4dcc640 1 -- 172.21.15.103:0/4168897549 >> 172.21.15.103:0/4168897549 conn(0x7f79f00fc7a0 msgr2=0x7f79f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:29.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.855+0000 7f79f4dcc640 1 -- 172.21.15.103:0/4168897549 shutdown_connections 2024-09-19T06:51:29.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.855+0000 7f79f4dcc640 1 -- 172.21.15.103:0/4168897549 wait complete. 2024-09-19T06:51:29.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.856+0000 7f79f4dcc640 1 Processor -- start 2024-09-19T06:51:29.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.856+0000 7f79f4dcc640 1 -- start start 2024-09-19T06:51:29.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.856+0000 7f79f4dcc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79f0102d40 0x7f79f0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.856+0000 7f79f4dcc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79f0103660 0x7f79f0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.856+0000 7f79f4dcc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79f010c170 0x7f79f019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.856+0000 7f79f4dcc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79f0111f70 con 0x7f79f0102d40 2024-09-19T06:51:29.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.856+0000 7f79f4dcc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f79f0111df0 con 0x7f79f0103660 2024-09-19T06:51:29.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.856+0000 7f79f4dcc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01120f0 con 0x7f79f010c170 2024-09-19T06:51:29.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79ee59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79f0102d40 0x7f79f0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:29.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79ee59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79f0102d40 0x7f79f0198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54090/0 (socket says 172.21.15.103:54090) 2024-09-19T06:51:29.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79ee59c640 1 -- 172.21.15.103:0/1380757801 learned_addr learned my addr 172.21.15.103:0/1380757801 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:29.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79eed9d640 1 --2- 172.21.15.103:0/1380757801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79f010c170 0x7f79f019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:29.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79edd9b640 1 --2- 172.21.15.103:0/1380757801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79f0103660 0x7f79f0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:29.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79ee59c640 1 -- 172.21.15.103:0/1380757801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79f010c170 msgr2=0x7f79f019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:29.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79ee59c640 1 --2- 172.21.15.103:0/1380757801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79f010c170 0x7f79f019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79ee59c640 1 -- 172.21.15.103:0/1380757801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79f0103660 msgr2=0x7f79f0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:29.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79ee59c640 1 --2- 172.21.15.103:0/1380757801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79f0103660 0x7f79f0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79ee59c640 1 -- 172.21.15.103:0/1380757801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f019d3e0 con 0x7f79f0102d40 2024-09-19T06:51:29.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79edd9b640 1 --2- 172.21.15.103:0/1380757801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79f0103660 0x7f79f0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:29.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79ee59c640 1 --2- 172.21.15.103:0/1380757801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79f0102d40 0x7f79f0198430 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f79e0006010 tx=0x7f79e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:29.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79df7fe640 1 -- 172.21.15.103:0/1380757801 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e003f030 con 0x7f79f0102d40 2024-09-19T06:51:29.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79df7fe640 1 -- 172.21.15.103:0/1380757801 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e0003ce0 con 0x7f79f0102d40 2024-09-19T06:51:29.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79f4dcc640 1 -- 172.21.15.103:0/1380757801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f019d670 con 0x7f79f0102d40 2024-09-19T06:51:29.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79df7fe640 1 -- 172.21.15.103:0/1380757801 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0035070 con 0x7f79f0102d40 2024-09-19T06:51:29.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.857+0000 7f79f4dcc640 1 -- 172.21.15.103:0/1380757801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f01a4fb0 con 0x7f79f0102d40 2024-09-19T06:51:29.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.860+0000 7f79df7fe640 1 -- 172.21.15.103:0/1380757801 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f79e0042030 con 0x7f79f0102d40 2024-09-19T06:51:29.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.860+0000 7f79f71a5740 1 Processor -- start 2024-09-19T06:51:29.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.860+0000 7f79f71a5740 1 -- start start 2024-09-19T06:51:29.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.860+0000 7f79df7fe640 1 --2- 172.21.15.103:0/1380757801 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f79d80a33b0 0x7f79d80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79eed9d640 1 --2- 172.21.15.103:0/1380757801 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f79d80a33b0 0x7f79d80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:29.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79f71a5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208cb40 0x56549208b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79df7fe640 1 -- 172.21.15.103:0/1380757801 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f79e00f0410 con 0x7f79f0102d40 2024-09-19T06:51:29.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79f71a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56549208b5d0 0x56549208ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79f71a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565492096370 0x565492098760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79f71a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565491ff12f0 con 0x565492096370 2024-09-19T06:51:29.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79f71a5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565491f98cf0 con 0x56549208cb40 2024-09-19T06:51:29.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79f71a5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565491fed2b0 con 0x56549208b5d0 2024-09-19T06:51:29.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79ee59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208cb40 0x56549208b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:29.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79eed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565492096370 0x565492098760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:29.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79ee59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208cb40 0x56549208b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:52742/0 (socket says 172.21.15.103:52742) 2024-09-19T06:51:29.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79ee59c640 1 -- 172.21.15.103:0/2849365755 learned_addr learned my addr 172.21.15.103:0/2849365755 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:29.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79edd9b640 1 --2- 172.21.15.103:0/2849365755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56549208b5d0 0x56549208ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:29.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79eed9d640 1 --2- 172.21.15.103:0/1380757801 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f79d80a33b0 0x7f79d80a5870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f79e8005fd0 tx=0x7f79e8012ad0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:29.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79eed9d640 1 -- 172.21.15.103:0/2849365755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56549208b5d0 msgr2=0x56549208ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:29.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79eed9d640 1 --2- 172.21.15.103:0/2849365755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56549208b5d0 0x56549208ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79eed9d640 1 -- 172.21.15.103:0/2849365755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208cb40 msgr2=0x56549208b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:29.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79eed9d640 1 --2- 172.21.15.103:0/2849365755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208cb40 0x56549208b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79eed9d640 1 -- 172.21.15.103:0/2849365755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565491f7d3a0 con 0x565492096370 2024-09-19T06:51:29.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79ee59c640 1 --2- 172.21.15.103:0/2849365755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208cb40 0x56549208b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:29.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79eed9d640 1 --2- 172.21.15.103:0/2849365755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565492096370 0x565492098760 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009d90 tx=0x7f79e801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7cda48a9cb3679 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:29.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.861+0000 7f79dcff9640 1 -- 172.21.15.103:0/2849365755 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8003b70 con 0x565492096370 2024-09-19T06:51:29.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.862+0000 7f79dcff9640 1 -- 172.21.15.103:0/2849365755 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e8003d10 con 0x565492096370 2024-09-19T06:51:29.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.862+0000 7f79dcff9640 1 -- 172.21.15.103:0/2849365755 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8004620 con 0x565492096370 2024-09-19T06:51:29.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.862+0000 7f79f71a5740 1 -- 172.21.15.103:0/2849365755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565492096370 msgr2=0x565492098760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:29.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.862+0000 7f79f71a5740 1 --2- 172.21.15.103:0/2849365755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565492096370 0x565492098760 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009d90 tx=0x7f79e801e000 comp rx=0 tx=0).stop 2024-09-19T06:51:29.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.862+0000 7f79f71a5740 1 -- 172.21.15.103:0/2849365755 shutdown_connections 2024-09-19T06:51:29.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.862+0000 7f79f71a5740 1 --2- 172.21.15.103:0/2849365755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565492096370 0x565492098760 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.862+0000 7f79f71a5740 1 --2- 172.21.15.103:0/2849365755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56549208b5d0 0x56549208ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.862+0000 7f79f71a5740 1 --2- 172.21.15.103:0/2849365755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208cb40 0x56549208b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.862+0000 7f79f71a5740 1 -- 172.21.15.103:0/2849365755 >> 172.21.15.103:0/2849365755 conn(0x565492054310 msgr2=0x565492087c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:29.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.862+0000 7f79f71a5740 1 -- 172.21.15.103:0/2849365755 shutdown_connections 2024-09-19T06:51:29.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.862+0000 7f79f71a5740 1 -- 172.21.15.103:0/2849365755 wait complete. 2024-09-19T06:51:29.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79f71a5740 1 -- start start 2024-09-19T06:51:29.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79f71a5740 1 Processor -- start 2024-09-19T06:51:29.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79f71a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208b5d0 0x565492095d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79f71a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56549208cb40 0x565492092fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79f71a5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565492096370 0x5654920935d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79f71a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565492099070 con 0x56549208b5d0 2024-09-19T06:51:29.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79f71a5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565491fac060 con 0x565492096370 2024-09-19T06:51:29.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79f71a5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5654920991f0 con 0x56549208cb40 2024-09-19T06:51:29.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79f71a5740 10 client.0 _ll_register_callbacks cb 0x565491cff8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:29.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79ee59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208b5d0 0x565492095d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:29.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79ee59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208b5d0 0x565492095d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54114/0 (socket says 172.21.15.103:54114) 2024-09-19T06:51:29.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79ee59c640 1 -- 172.21.15.103:0/51946075 learned_addr learned my addr 172.21.15.103:0/51946075 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:29.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79edd9b640 1 --2- 172.21.15.103:0/51946075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56549208cb40 0x565492092fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:29.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79eed9d640 1 --2- 172.21.15.103:0/51946075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565492096370 0x5654920935d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:29.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79ee59c640 1 -- 172.21.15.103:0/51946075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56549208cb40 msgr2=0x565492092fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:29.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79ee59c640 1 --2- 172.21.15.103:0/51946075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56549208cb40 0x565492092fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79ee59c640 1 -- 172.21.15.103:0/51946075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565492096370 msgr2=0x5654920935d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:29.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79ee59c640 1 --2- 172.21.15.103:0/51946075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565492096370 0x5654920935d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79ee59c640 1 -- 172.21.15.103:0/51946075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565491d623c0 con 0x56549208b5d0 2024-09-19T06:51:29.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79eed9d640 1 --2- 172.21.15.103:0/51946075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565492096370 0x5654920935d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:29.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79ee59c640 1 --2- 172.21.15.103:0/51946075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208b5d0 0x565492095d30 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f79e00b8be0 tx=0x7f79e0003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308b748be4f40990 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:29.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.863+0000 7f79d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:29.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.864+0000 7f79d5ffb640 1 -- 172.21.15.103:0/51946075 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0049050 con 0x56549208b5d0 2024-09-19T06:51:29.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.864+0000 7f79d5ffb640 1 -- 172.21.15.103:0/51946075 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f79e00c4f60 con 0x56549208b5d0 2024-09-19T06:51:29.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.864+0000 7f79f71a5740 1 -- 172.21.15.103:0/51946075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79e8003d10 con 0x56549208b5d0 2024-09-19T06:51:29.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.864+0000 7f79d5ffb640 1 -- 172.21.15.103:0/51946075 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f79e00bc0e0 con 0x56549208b5d0 2024-09-19T06:51:29.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.864+0000 7f79f71a5740 10 client.16356 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:29.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.864+0000 7f79f71a5740 1 -- 172.21.15.103:0/51946075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5654920a4ad0 con 0x56549208b5d0 2024-09-19T06:51:29.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.864+0000 7f79d5ffb640 1 -- 172.21.15.103:0/51946075 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e8003d10 con 0x56549208b5d0 2024-09-19T06:51:29.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.864+0000 7f79d5ffb640 1 -- 172.21.15.103:0/51946075 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f79e00f0070 con 0x56549208b5d0 2024-09-19T06:51:29.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.864+0000 7f79f71a5740 10 client.16356 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:29.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.864+0000 7f79f71a5740 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:29.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.864+0000 7f79f71a5740 20 client.16356 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:29.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.864+0000 7f79f71a5740 1 --2- 172.21.15.103:0/51946075 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5654920a5f30 0x5654920a8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.864+0000 7f79f71a5740 4 client.16356 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:29.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.864+0000 7f79f71a5740 1 -- 172.21.15.103:0/51946075 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565491ffe860 con 0x5654920a5f30 2024-09-19T06:51:29.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.864+0000 7f79edd9b640 1 --2- 172.21.15.103:0/51946075 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5654920a5f30 0x5654920a8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:29.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.865+0000 7f79edd9b640 1 --2- 172.21.15.103:0/51946075 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5654920a5f30 0x5654920a8360 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ee85ac915a1252b server_cookie=d2e5543b50683484 in_seq=0 out_seq=0 2024-09-19T06:51:29.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.865+0000 7f79d5ffb640 10 client.16356 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:29.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.866+0000 7f79d5ffb640 1 -- 172.21.15.103:0/51946075 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565491ffe860 con 0x5654920a5f30 2024-09-19T06:51:29.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.866+0000 7f79d5ffb640 10 client.16356 handle_command_reply: tid=0 2024-09-19T06:51:29.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.866+0000 7f79f71a5740 1 client.16356 shutdown 2024-09-19T06:51:29.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.866+0000 7f79f71a5740 1 -- 172.21.15.103:0/51946075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208b5d0 msgr2=0x565492095d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:29.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.866+0000 7f79f71a5740 1 --2- 172.21.15.103:0/51946075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208b5d0 0x565492095d30 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f79e00b8be0 tx=0x7f79e0003f80 comp rx=0 tx=0).stop 2024-09-19T06:51:29.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.866+0000 7f79f71a5740 1 -- 172.21.15.103:0/51946075 shutdown_connections 2024-09-19T06:51:29.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.866+0000 7f79f71a5740 1 --2- 172.21.15.103:0/51946075 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5654920a5f30 0x5654920a8360 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.866+0000 7f79d5ffb640 0 client.16356 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:29.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.866+0000 7f79f71a5740 1 --2- 172.21.15.103:0/51946075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565492096370 0x5654920935d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.866+0000 7f79f71a5740 1 --2- 172.21.15.103:0/51946075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56549208cb40 0x565492092fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.866+0000 7f79f71a5740 1 --2- 172.21.15.103:0/51946075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208b5d0 0x565492095d30 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.866+0000 7f79f71a5740 1 -- 172.21.15.103:0/51946075 >> 172.21.15.103:0/51946075 conn(0x565492054310 msgr2=0x56549208d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:29.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.866+0000 7f79f71a5740 1 -- 172.21.15.103:0/51946075 shutdown_connections 2024-09-19T06:51:29.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.866+0000 7f79f71a5740 1 -- 172.21.15.103:0/51946075 wait complete. 2024-09-19T06:51:29.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.866+0000 7f79f71a5740 20 client.16356 trim_cache size 0 max 16384 2024-09-19T06:51:29.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79f71a5740 1 Processor -- start 2024-09-19T06:51:29.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79f71a5740 1 -- start start 2024-09-19T06:51:29.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79f71a5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208b5d0 0x5654920a2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79f71a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208cb40 0x5654920928a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79f71a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565492096370 0x5654920adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79f71a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01ad860 con 0x56549208cb40 2024-09-19T06:51:29.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79f71a5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01af3a0 con 0x56549208b5d0 2024-09-19T06:51:29.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79f71a5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01adf30 con 0x565492096370 2024-09-19T06:51:29.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79edd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208cb40 0x5654920928a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:29.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79edd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208cb40 0x5654920928a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54116/0 (socket says 172.21.15.103:54116) 2024-09-19T06:51:29.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79edd9b640 1 -- 172.21.15.103:0/2107834741 learned_addr learned my addr 172.21.15.103:0/2107834741 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:29.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79ee59c640 1 --2- 172.21.15.103:0/2107834741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208b5d0 0x5654920a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:29.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79edd9b640 1 -- 172.21.15.103:0/2107834741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565492096370 msgr2=0x5654920adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:29.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79edd9b640 1 --2- 172.21.15.103:0/2107834741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565492096370 0x5654920adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79edd9b640 1 -- 172.21.15.103:0/2107834741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208b5d0 msgr2=0x5654920a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:29.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79edd9b640 1 --2- 172.21.15.103:0/2107834741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208b5d0 0x5654920a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79edd9b640 1 -- 172.21.15.103:0/2107834741 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654920a89e0 con 0x56549208cb40 2024-09-19T06:51:29.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79ee59c640 1 --2- 172.21.15.103:0/2107834741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208b5d0 0x5654920a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:29.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.879+0000 7f79edd9b640 1 --2- 172.21.15.103:0/2107834741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208cb40 0x5654920928a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x565492088ac0 tx=0x7f79e4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6a470657133a08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:29.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.880+0000 7f79d7fff640 1 -- 172.21.15.103:0/2107834741 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4011440 con 0x56549208cb40 2024-09-19T06:51:29.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.880+0000 7f79d7fff640 1 -- 172.21.15.103:0/2107834741 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e4011ac0 con 0x56549208cb40 2024-09-19T06:51:29.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.880+0000 7f79d7fff640 1 -- 172.21.15.103:0/2107834741 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e401cb30 con 0x56549208cb40 2024-09-19T06:51:29.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.880+0000 7f79f71a5740 1 -- 172.21.15.103:0/2107834741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208cb40 msgr2=0x5654920928a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:29.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.880+0000 7f79f71a5740 1 --2- 172.21.15.103:0/2107834741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208cb40 0x5654920928a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x565492088ac0 tx=0x7f79e4005850 comp rx=0 tx=0).stop 2024-09-19T06:51:29.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.880+0000 7f79f71a5740 1 -- 172.21.15.103:0/2107834741 shutdown_connections 2024-09-19T06:51:29.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.880+0000 7f79f71a5740 1 --2- 172.21.15.103:0/2107834741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565492096370 0x5654920adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.880+0000 7f79f71a5740 1 --2- 172.21.15.103:0/2107834741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208cb40 0x5654920928a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.880+0000 7f79f71a5740 1 --2- 172.21.15.103:0/2107834741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208b5d0 0x5654920a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.880+0000 7f79f71a5740 1 -- 172.21.15.103:0/2107834741 >> 172.21.15.103:0/2107834741 conn(0x565492054310 msgr2=0x5654920918f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:29.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.880+0000 7f79f71a5740 1 -- 172.21.15.103:0/2107834741 shutdown_connections 2024-09-19T06:51:29.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.880+0000 7f79f71a5740 1 -- 172.21.15.103:0/2107834741 wait complete. 2024-09-19T06:51:29.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79f71a5740 1 -- start start 2024-09-19T06:51:29.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79f71a5740 1 Processor -- start 2024-09-19T06:51:29.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79f71a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208b5d0 0x56549208ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79f71a5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208cb40 0x565492090460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79f71a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565492096370 0x5654920909a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79f71a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01aecd0 con 0x56549208b5d0 2024-09-19T06:51:29.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79f71a5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01ae600 con 0x56549208cb40 2024-09-19T06:51:29.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79f71a5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79f01ab820 con 0x565492096370 2024-09-19T06:51:29.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79f71a5740 10 client.0 _ll_register_callbacks cb 0x7f79e00b8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:29.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79ee59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208b5d0 0x56549208ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:29.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79ee59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208b5d0 0x56549208ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54124/0 (socket says 172.21.15.103:54124) 2024-09-19T06:51:29.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79ee59c640 1 -- 172.21.15.103:0/2353596500 learned_addr learned my addr 172.21.15.103:0/2353596500 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:29.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79edd9b640 1 --2- 172.21.15.103:0/2353596500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208cb40 0x565492090460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:29.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79ee59c640 1 -- 172.21.15.103:0/2353596500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565492096370 msgr2=0x5654920909a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:29.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79ee59c640 1 --2- 172.21.15.103:0/2353596500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565492096370 0x5654920909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79ee59c640 1 -- 172.21.15.103:0/2353596500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208cb40 msgr2=0x565492090460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:29.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79ee59c640 1 --2- 172.21.15.103:0/2353596500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208cb40 0x565492090460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79ee59c640 1 -- 172.21.15.103:0/2353596500 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56549210fc40 con 0x56549208b5d0 2024-09-19T06:51:29.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.881+0000 7f79edd9b640 1 --2- 172.21.15.103:0/2353596500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208cb40 0x565492090460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:29.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.882+0000 7f79ee59c640 1 --2- 172.21.15.103:0/2353596500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208b5d0 0x56549208ff20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f79e0005ab0 tx=0x7f79e00c5290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28c2877153bffdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:29.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.882+0000 7f79d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:29.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.882+0000 7f79d5ffb640 1 -- 172.21.15.103:0/2353596500 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0049040 con 0x56549208b5d0 2024-09-19T06:51:29.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.882+0000 7f79d5ffb640 1 -- 172.21.15.103:0/2353596500 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f79e0003800 con 0x56549208b5d0 2024-09-19T06:51:29.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.882+0000 7f79f71a5740 1 -- 172.21.15.103:0/2353596500 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79e4011ac0 con 0x56549208b5d0 2024-09-19T06:51:29.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.882+0000 7f79d5ffb640 1 -- 172.21.15.103:0/2353596500 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f79e00044e0 con 0x56549208b5d0 2024-09-19T06:51:29.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.882+0000 7f79f71a5740 10 client.16362 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:29.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.882+0000 7f79f71a5740 1 -- 172.21.15.103:0/2353596500 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565492110e50 con 0x56549208b5d0 2024-09-19T06:51:29.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.882+0000 7f79d5ffb640 1 -- 172.21.15.103:0/2353596500 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e4011ac0 con 0x56549208b5d0 2024-09-19T06:51:29.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.882+0000 7f79d5ffb640 1 -- 172.21.15.103:0/2353596500 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f79e00f0040 con 0x56549208b5d0 2024-09-19T06:51:29.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.882+0000 7f79f71a5740 10 client.16362 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:29.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.882+0000 7f79f71a5740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:29.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.882+0000 7f79f71a5740 20 client.16362 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:29.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.882+0000 7f79f71a5740 1 --2- 172.21.15.103:0/2353596500 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x565492111460 0x5654921138e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:29.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.882+0000 7f79f71a5740 4 client.16362 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:51:29.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.882+0000 7f79f71a5740 1 -- 172.21.15.103:0/2353596500 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565491ff8d70 con 0x565492111460 2024-09-19T06:51:29.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.883+0000 7f79edd9b640 1 --2- 172.21.15.103:0/2353596500 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x565492111460 0x5654921138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:29.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.883+0000 7f79d5ffb640 10 client.16362 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:29.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.883+0000 7f79edd9b640 1 --2- 172.21.15.103:0/2353596500 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x565492111460 0x5654921138e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed5470564fe88215 server_cookie=6275e08347f79c1b in_seq=0 out_seq=0 2024-09-19T06:51:29.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.883+0000 7f79d5ffb640 1 -- 172.21.15.103:0/2353596500 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565491ff8d70 con 0x565492111460 2024-09-19T06:51:29.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.883+0000 7f79d5ffb640 10 client.16362 handle_command_reply: tid=0 2024-09-19T06:51:29.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.883+0000 7f79f71a5740 1 client.16362 shutdown 2024-09-19T06:51:29.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.883+0000 7f79f71a5740 1 -- 172.21.15.103:0/2353596500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208b5d0 msgr2=0x56549208ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:29.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.883+0000 7f79f71a5740 1 --2- 172.21.15.103:0/2353596500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208b5d0 0x56549208ff20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f79e0005ab0 tx=0x7f79e00c5290 comp rx=0 tx=0).stop 2024-09-19T06:51:29.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f71a5740 1 -- 172.21.15.103:0/2353596500 shutdown_connections 2024-09-19T06:51:29.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f71a5740 1 --2- 172.21.15.103:0/2353596500 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x565492111460 0x5654921138e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f71a5740 1 --2- 172.21.15.103:0/2353596500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565492096370 0x5654920909a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79d5ffb640 0 client.16362 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:29.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f71a5740 1 --2- 172.21.15.103:0/2353596500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56549208cb40 0x565492090460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f71a5740 1 --2- 172.21.15.103:0/2353596500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56549208b5d0 0x56549208ff20 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f71a5740 1 -- 172.21.15.103:0/2353596500 >> 172.21.15.103:0/2353596500 conn(0x565492054310 msgr2=0x56549208d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:29.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f71a5740 1 -- 172.21.15.103:0/2353596500 shutdown_connections 2024-09-19T06:51:29.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f71a5740 1 -- 172.21.15.103:0/2353596500 wait complete. 2024-09-19T06:51:29.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f71a5740 20 client.16362 trim_cache size 0 max 16384 2024-09-19T06:51:29.892 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:51:29.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f4dcc640 1 -- 172.21.15.103:0/1380757801 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f79d80a33b0 msgr2=0x7f79d80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:29.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f4dcc640 1 --2- 172.21.15.103:0/1380757801 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f79d80a33b0 0x7f79d80a5870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f79e8005fd0 tx=0x7f79e8012ad0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f4dcc640 1 -- 172.21.15.103:0/1380757801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79f0102d40 msgr2=0x7f79f0198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:29.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f4dcc640 1 --2- 172.21.15.103:0/1380757801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79f0102d40 0x7f79f0198430 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f79e0006010 tx=0x7f79e00042f0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f4dcc640 1 -- 172.21.15.103:0/1380757801 shutdown_connections 2024-09-19T06:51:29.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f4dcc640 1 --2- 172.21.15.103:0/1380757801 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f79d80a33b0 0x7f79d80a5870 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f4dcc640 1 --2- 172.21.15.103:0/1380757801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79f010c170 0x7f79f019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f4dcc640 1 --2- 172.21.15.103:0/1380757801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79f0103660 0x7f79f0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f4dcc640 1 --2- 172.21.15.103:0/1380757801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79f0102d40 0x7f79f0198430 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:29.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.884+0000 7f79f4dcc640 1 -- 172.21.15.103:0/1380757801 >> 172.21.15.103:0/1380757801 conn(0x7f79f00fc7a0 msgr2=0x7f79f00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:29.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.885+0000 7f79f4dcc640 1 -- 172.21.15.103:0/1380757801 shutdown_connections 2024-09-19T06:51:29.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:29.885+0000 7f79f4dcc640 1 -- 172.21.15.103:0/1380757801 wait complete. 2024-09-19T06:51:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:30 smithi174 ceph-mon[35637]: pgmap v71: 97 pgs: 97 active+clean; 1.6 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-19T06:51:30.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:30 smithi103 ceph-mon[31158]: pgmap v71: 97 pgs: 97 active+clean; 1.6 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-19T06:51:30.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:30 smithi134 ceph-mon[35356]: pgmap v71: 97 pgs: 97 active+clean; 1.6 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-19T06:51:31.536 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:51:31.536 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:51:31.794 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:32.193 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:32 smithi103 ceph-mon[31158]: pgmap v72: 97 pgs: 97 active+clean; 1.6 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-19T06:51:32.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.275+0000 7ff454026640 1 -- 172.21.15.103:0/4225880061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff44c102570 msgr2=0x7ff44c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.275+0000 7ff454026640 1 --2- 172.21.15.103:0/4225880061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff44c102570 0x7ff44c1029b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff440009900 tx=0x7ff44002a480 comp rx=0 tx=0).stop 2024-09-19T06:51:32.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.275+0000 7ff454026640 1 -- 172.21.15.103:0/4225880061 shutdown_connections 2024-09-19T06:51:32.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.275+0000 7ff454026640 1 --2- 172.21.15.103:0/4225880061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff44c102ef0 0x7ff44c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.275+0000 7ff454026640 1 --2- 172.21.15.103:0/4225880061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff44c102570 0x7ff44c1029b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.275+0000 7ff454026640 1 --2- 172.21.15.103:0/4225880061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff44c100e20 0x7ff44c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.275+0000 7ff454026640 1 -- 172.21.15.103:0/4225880061 >> 172.21.15.103:0/4225880061 conn(0x7ff44c0fc7d0 msgr2=0x7ff44c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:32.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.275+0000 7ff454026640 1 -- 172.21.15.103:0/4225880061 shutdown_connections 2024-09-19T06:51:32.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.275+0000 7ff454026640 1 -- 172.21.15.103:0/4225880061 wait complete. 2024-09-19T06:51:32.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff454026640 1 Processor -- start 2024-09-19T06:51:32.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff454026640 1 -- start start 2024-09-19T06:51:32.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff454026640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff44c100e20 0x7ff44c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff454026640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff44c102570 0x7ff44c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff454026640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff44c102ef0 0x7ff44c1a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff454026640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c110fa0 con 0x7ff44c100e20 2024-09-19T06:51:32.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff454026640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c110e20 con 0x7ff44c102570 2024-09-19T06:51:32.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff454026640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c111120 con 0x7ff44c102ef0 2024-09-19T06:51:32.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff451dc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff44c100e20 0x7ff44c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff451dc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff44c100e20 0x7ff44c199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54146/0 (socket says 172.21.15.103:54146) 2024-09-19T06:51:32.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff451dc2640 1 -- 172.21.15.103:0/3683775929 learned_addr learned my addr 172.21.15.103:0/3683775929 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:32.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff451dc2640 1 -- 172.21.15.103:0/3683775929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff44c102ef0 msgr2=0x7ff44c1a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff4515c1640 1 --2- 172.21.15.103:0/3683775929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff44c102570 0x7ff44c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff4525c3640 1 --2- 172.21.15.103:0/3683775929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff44c102ef0 0x7ff44c1a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff451dc2640 1 --2- 172.21.15.103:0/3683775929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff44c102ef0 0x7ff44c1a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff451dc2640 1 -- 172.21.15.103:0/3683775929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff44c102570 msgr2=0x7ff44c199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff451dc2640 1 --2- 172.21.15.103:0/3683775929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff44c102570 0x7ff44c199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff451dc2640 1 -- 172.21.15.103:0/3683775929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44c1a48f0 con 0x7ff44c100e20 2024-09-19T06:51:32.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff4525c3640 1 --2- 172.21.15.103:0/3683775929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff44c102ef0 0x7ff44c1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:32.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.276+0000 7ff451dc2640 1 --2- 172.21.15.103:0/3683775929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff44c100e20 0x7ff44c199330 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff44800c920 tx=0x7ff44800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:32.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.277+0000 7ff43effd640 1 -- 172.21.15.103:0/3683775929 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448007c70 con 0x7ff44c100e20 2024-09-19T06:51:32.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.277+0000 7ff43effd640 1 -- 172.21.15.103:0/3683775929 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff448007e10 con 0x7ff44c100e20 2024-09-19T06:51:32.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.277+0000 7ff454026640 1 -- 172.21.15.103:0/3683775929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff44c1a4ac0 con 0x7ff44c100e20 2024-09-19T06:51:32.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.277+0000 7ff43effd640 1 -- 172.21.15.103:0/3683775929 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448018890 con 0x7ff44c100e20 2024-09-19T06:51:32.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.277+0000 7ff454026640 1 -- 172.21.15.103:0/3683775929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff44c1a5000 con 0x7ff44c100e20 2024-09-19T06:51:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.279+0000 7ff43effd640 1 -- 172.21.15.103:0/3683775929 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff448018a70 con 0x7ff44c100e20 2024-09-19T06:51:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.279+0000 7ff4563ff740 1 Processor -- start 2024-09-19T06:51:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff4563ff740 1 -- start start 2024-09-19T06:51:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff43effd640 1 --2- 172.21.15.103:0/3683775929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4380a3430 0x7ff4380a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff4563ff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef7367f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff4563ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7368480 0x563ef7372800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff4563ff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7372d40 0x563ef7375130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff4563ff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563ef72cd280 con 0x563ef7367b60 2024-09-19T06:51:32.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff4563ff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563ef7274b10 con 0x563ef7372d40 2024-09-19T06:51:32.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff4563ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563ef72c9240 con 0x563ef7368480 2024-09-19T06:51:32.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff43effd640 1 -- 172.21.15.103:0/3683775929 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff4480cfeb0 con 0x7ff44c100e20 2024-09-19T06:51:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff451dc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef7367f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff451dc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef7367f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54148/0 (socket says 172.21.15.103:54148) 2024-09-19T06:51:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff451dc2640 1 -- 172.21.15.103:0/3832672646 learned_addr learned my addr 172.21.15.103:0/3832672646 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff4525c3640 1 --2- 172.21.15.103:0/3683775929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4380a3430 0x7ff4380a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff4515c1640 1 --2- 172.21.15.103:0/3832672646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7368480 0x563ef7372800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff451dc2640 1 -- 172.21.15.103:0/3832672646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7368480 msgr2=0x563ef7372800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:32.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff451dc2640 1 --2- 172.21.15.103:0/3832672646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7368480 0x563ef7372800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff451dc2640 1 -- 172.21.15.103:0/3832672646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7372d40 msgr2=0x563ef7375130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:32.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff451dc2640 1 --2- 172.21.15.103:0/3832672646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7372d40 0x563ef7375130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff451dc2640 1 -- 172.21.15.103:0/3832672646 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ef72811f0 con 0x563ef7367b60 2024-09-19T06:51:32.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff451dc2640 1 --2- 172.21.15.103:0/3832672646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef7367f40 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff448098b30 tx=0x7ff448051670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f5dfbb1e90ad77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:32.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.280+0000 7ff4525c3640 1 --2- 172.21.15.103:0/3683775929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4380a3430 0x7ff4380a58f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x563ef7365470 tx=0x7ff444007900 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:32.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.281+0000 7ff437fff640 1 -- 172.21.15.103:0/3832672646 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4480a0510 con 0x563ef7367b60 2024-09-19T06:51:32.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.281+0000 7ff437fff640 1 -- 172.21.15.103:0/3832672646 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4480a4dd0 con 0x563ef7367b60 2024-09-19T06:51:32.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.281+0000 7ff437fff640 1 -- 172.21.15.103:0/3832672646 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4480987d0 con 0x563ef7367b60 2024-09-19T06:51:32.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.281+0000 7ff4563ff740 1 -- 172.21.15.103:0/3832672646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 msgr2=0x563ef7367f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.281+0000 7ff4563ff740 1 --2- 172.21.15.103:0/3832672646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef7367f40 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff448098b30 tx=0x7ff448051670 comp rx=0 tx=0).stop 2024-09-19T06:51:32.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.281+0000 7ff4563ff740 1 -- 172.21.15.103:0/3832672646 shutdown_connections 2024-09-19T06:51:32.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.281+0000 7ff4563ff740 1 --2- 172.21.15.103:0/3832672646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7372d40 0x563ef7375130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.281+0000 7ff4563ff740 1 --2- 172.21.15.103:0/3832672646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7368480 0x563ef7372800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.281+0000 7ff4563ff740 1 --2- 172.21.15.103:0/3832672646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef7367f40 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.281+0000 7ff4563ff740 1 -- 172.21.15.103:0/3832672646 >> 172.21.15.103:0/3832672646 conn(0x563ef73640f0 msgr2=0x563ef73644d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:32.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.281+0000 7ff4563ff740 1 -- 172.21.15.103:0/3832672646 shutdown_connections 2024-09-19T06:51:32.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.281+0000 7ff4563ff740 1 -- 172.21.15.103:0/3832672646 wait complete. 2024-09-19T06:51:32.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.282+0000 7ff4563ff740 1 -- start start 2024-09-19T06:51:32.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.282+0000 7ff4563ff740 1 Processor -- start 2024-09-19T06:51:32.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.282+0000 7ff4563ff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef736fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.282+0000 7ff4563ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7368480 0x563ef73701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.282+0000 7ff4563ff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7372d40 0x563ef73706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.282+0000 7ff4563ff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563ef7375910 con 0x563ef7367b60 2024-09-19T06:51:32.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.282+0000 7ff4563ff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563ef7287f10 con 0x563ef7372d40 2024-09-19T06:51:32.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.282+0000 7ff4563ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563ef7375a90 con 0x563ef7368480 2024-09-19T06:51:32.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.282+0000 7ff4563ff740 10 client.0 _ll_register_callbacks cb 0x563ef7250a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:32.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.282+0000 7ff451dc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef736fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.282+0000 7ff451dc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef736fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54154/0 (socket says 172.21.15.103:54154) 2024-09-19T06:51:32.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.282+0000 7ff451dc2640 1 -- 172.21.15.103:0/1034349803 learned_addr learned my addr 172.21.15.103:0/1034349803 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:32.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.282+0000 7ff4515c1640 1 --2- 172.21.15.103:0/1034349803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7368480 0x563ef73701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff451dc2640 1 -- 172.21.15.103:0/1034349803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7368480 msgr2=0x563ef73701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:32.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff451dc2640 1 --2- 172.21.15.103:0/1034349803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7368480 0x563ef73701b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff451dc2640 1 -- 172.21.15.103:0/1034349803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7372d40 msgr2=0x563ef73706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:32.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff4525c3640 1 --2- 172.21.15.103:0/1034349803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7372d40 0x563ef73706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff451dc2640 1 --2- 172.21.15.103:0/1034349803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7372d40 0x563ef73706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff451dc2640 1 -- 172.21.15.103:0/1034349803 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ef703e6a0 con 0x563ef7367b60 2024-09-19T06:51:32.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff4525c3640 1 --2- 172.21.15.103:0/1034349803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7372d40 0x563ef73706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:32.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff451dc2640 1 --2- 172.21.15.103:0/1034349803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef736fc70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff448000c00 tx=0x7ff4480bc390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbc98529a79f4ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:32.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff4357fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:32.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff4357fa640 1 -- 172.21.15.103:0/1034349803 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4480b5070 con 0x563ef7367b60 2024-09-19T06:51:32.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff4357fa640 1 -- 172.21.15.103:0/1034349803 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff44809a460 con 0x563ef7367b60 2024-09-19T06:51:32.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff4563ff740 1 -- 172.21.15.103:0/1034349803 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4480a4dd0 con 0x563ef7367b60 2024-09-19T06:51:32.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff4357fa640 1 -- 172.21.15.103:0/1034349803 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4480a0510 con 0x563ef7367b60 2024-09-19T06:51:32.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff4563ff740 10 client.16374 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:32.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff4563ff740 1 -- 172.21.15.103:0/1034349803 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563ef7381550 con 0x563ef7367b60 2024-09-19T06:51:32.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff4357fa640 1 -- 172.21.15.103:0/1034349803 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4480a4dd0 con 0x563ef7367b60 2024-09-19T06:51:32.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff4357fa640 1 -- 172.21.15.103:0/1034349803 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff4480518e0 con 0x563ef7367b60 2024-09-19T06:51:32.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff4563ff740 10 client.16374 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:32.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff4563ff740 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:32.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.283+0000 7ff4563ff740 20 client.16374 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:32.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.284+0000 7ff4563ff740 1 --2- 172.21.15.103:0/1034349803 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563ef73828a0 0x563ef7384c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.284+0000 7ff4563ff740 4 client.16374 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:32.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.284+0000 7ff4563ff740 1 -- 172.21.15.103:0/1034349803 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ef72da7f0 con 0x563ef73828a0 2024-09-19T06:51:32.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.284+0000 7ff4515c1640 1 --2- 172.21.15.103:0/1034349803 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563ef73828a0 0x563ef7384c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.284+0000 7ff4357fa640 10 client.16374 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:32.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.284+0000 7ff4515c1640 1 --2- 172.21.15.103:0/1034349803 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563ef73828a0 0x563ef7384c90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3f7bcc8eb3a84f5 server_cookie=675dcfd7e27b0d10 in_seq=0 out_seq=0 2024-09-19T06:51:32.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.285+0000 7ff4357fa640 1 -- 172.21.15.103:0/1034349803 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ef72da7f0 con 0x563ef73828a0 2024-09-19T06:51:32.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.285+0000 7ff4357fa640 10 client.16374 handle_command_reply: tid=0 2024-09-19T06:51:32.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.285+0000 7ff4563ff740 1 client.16374 shutdown 2024-09-19T06:51:32.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.285+0000 7ff4563ff740 1 -- 172.21.15.103:0/1034349803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 msgr2=0x563ef736fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.285+0000 7ff4563ff740 1 --2- 172.21.15.103:0/1034349803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef736fc70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff448000c00 tx=0x7ff4480bc390 comp rx=0 tx=0).stop 2024-09-19T06:51:32.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.285+0000 7ff4563ff740 1 -- 172.21.15.103:0/1034349803 shutdown_connections 2024-09-19T06:51:32.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.285+0000 7ff4563ff740 1 --2- 172.21.15.103:0/1034349803 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563ef73828a0 0x563ef7384c90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.285+0000 7ff4563ff740 1 --2- 172.21.15.103:0/1034349803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7372d40 0x563ef73706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.285+0000 7ff4563ff740 1 --2- 172.21.15.103:0/1034349803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7368480 0x563ef73701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.285+0000 7ff4357fa640 0 client.16374 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:32.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.285+0000 7ff4563ff740 1 --2- 172.21.15.103:0/1034349803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef736fc70 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.285+0000 7ff4563ff740 1 -- 172.21.15.103:0/1034349803 >> 172.21.15.103:0/1034349803 conn(0x563ef73640f0 msgr2=0x563ef7369e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:32.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.285+0000 7ff4563ff740 1 -- 172.21.15.103:0/1034349803 shutdown_connections 2024-09-19T06:51:32.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.285+0000 7ff4563ff740 1 -- 172.21.15.103:0/1034349803 wait complete. 2024-09-19T06:51:32.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.285+0000 7ff4563ff740 20 client.16374 trim_cache size 0 max 16384 2024-09-19T06:51:32.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.302+0000 7ff4563ff740 1 Processor -- start 2024-09-19T06:51:32.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.302+0000 7ff4563ff740 1 -- start start 2024-09-19T06:51:32.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.302+0000 7ff4563ff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef736a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.302+0000 7ff4563ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7368480 0x563ef7371ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.302+0000 7ff4563ff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7372d40 0x563ef7370540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.302+0000 7ff4563ff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c1ad870 con 0x563ef7367b60 2024-09-19T06:51:32.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.302+0000 7ff4563ff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c1af3b0 con 0x563ef7372d40 2024-09-19T06:51:32.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.302+0000 7ff4563ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c1adf40 con 0x563ef7368480 2024-09-19T06:51:32.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.303+0000 7ff451dc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef736a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.303+0000 7ff451dc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef736a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54156/0 (socket says 172.21.15.103:54156) 2024-09-19T06:51:32.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.303+0000 7ff451dc2640 1 -- 172.21.15.103:0/3012015098 learned_addr learned my addr 172.21.15.103:0/3012015098 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:32.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.303+0000 7ff4515c1640 1 --2- 172.21.15.103:0/3012015098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7368480 0x563ef7371ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.303+0000 7ff4525c3640 1 --2- 172.21.15.103:0/3012015098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7372d40 0x563ef7370540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.303+0000 7ff451dc2640 1 -- 172.21.15.103:0/3012015098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7368480 msgr2=0x563ef7371ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:32.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.303+0000 7ff451dc2640 1 --2- 172.21.15.103:0/3012015098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7368480 0x563ef7371ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.303+0000 7ff451dc2640 1 -- 172.21.15.103:0/3012015098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7372d40 msgr2=0x563ef7370540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:32.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.303+0000 7ff451dc2640 1 --2- 172.21.15.103:0/3012015098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7372d40 0x563ef7370540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.303+0000 7ff451dc2640 1 -- 172.21.15.103:0/3012015098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ef73651a0 con 0x563ef7367b60 2024-09-19T06:51:32.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.303+0000 7ff4525c3640 1 --2- 172.21.15.103:0/3012015098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7372d40 0x563ef7370540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:32.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.303+0000 7ff451dc2640 1 --2- 172.21.15.103:0/3012015098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef736a5c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff44809c0f0 tx=0x7ff4480bc110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7af15e2b0d34cd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:32.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.303+0000 7ff4377fe640 1 -- 172.21.15.103:0/3012015098 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4480b5040 con 0x563ef7367b60 2024-09-19T06:51:32.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.303+0000 7ff4377fe640 1 -- 172.21.15.103:0/3012015098 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4480a4dd0 con 0x563ef7367b60 2024-09-19T06:51:32.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.303+0000 7ff4377fe640 1 -- 172.21.15.103:0/3012015098 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4480cd850 con 0x563ef7367b60 2024-09-19T06:51:32.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.304+0000 7ff4563ff740 1 -- 172.21.15.103:0/3012015098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 msgr2=0x563ef736a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.304+0000 7ff4563ff740 1 --2- 172.21.15.103:0/3012015098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef736a5c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff44809c0f0 tx=0x7ff4480bc110 comp rx=0 tx=0).stop 2024-09-19T06:51:32.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.304+0000 7ff4563ff740 1 -- 172.21.15.103:0/3012015098 shutdown_connections 2024-09-19T06:51:32.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.304+0000 7ff4563ff740 1 --2- 172.21.15.103:0/3012015098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7372d40 0x563ef7370540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.304+0000 7ff4563ff740 1 --2- 172.21.15.103:0/3012015098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7368480 0x563ef7371ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.304+0000 7ff4563ff740 1 --2- 172.21.15.103:0/3012015098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef736a5c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.304+0000 7ff4563ff740 1 -- 172.21.15.103:0/3012015098 >> 172.21.15.103:0/3012015098 conn(0x563ef73640f0 msgr2=0x563ef73713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:32.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.304+0000 7ff4563ff740 1 -- 172.21.15.103:0/3012015098 shutdown_connections 2024-09-19T06:51:32.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.304+0000 7ff4563ff740 1 -- 172.21.15.103:0/3012015098 wait complete. 2024-09-19T06:51:32.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.304+0000 7ff4563ff740 1 -- start start 2024-09-19T06:51:32.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.304+0000 7ff4563ff740 1 Processor -- start 2024-09-19T06:51:32.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff4563ff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef73881b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff4563ff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7368480 0x563ef7385500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff4563ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7372d40 0x563ef7385a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff4563ff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c1aece0 con 0x563ef7367b60 2024-09-19T06:51:32.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff4563ff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c1ae610 con 0x563ef7368480 2024-09-19T06:51:32.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff4563ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c1ab830 con 0x563ef7372d40 2024-09-19T06:51:32.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff4563ff740 10 client.0 _ll_register_callbacks cb 0x7ff440009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:32.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff451dc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef73881b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff451dc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef73881b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54170/0 (socket says 172.21.15.103:54170) 2024-09-19T06:51:32.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff451dc2640 1 -- 172.21.15.103:0/3172975324 learned_addr learned my addr 172.21.15.103:0/3172975324 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:32.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff4525c3640 1 --2- 172.21.15.103:0/3172975324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7372d40 0x563ef7385a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff4515c1640 1 --2- 172.21.15.103:0/3172975324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7368480 0x563ef7385500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff451dc2640 1 -- 172.21.15.103:0/3172975324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7372d40 msgr2=0x563ef7385a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:32.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff451dc2640 1 --2- 172.21.15.103:0/3172975324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7372d40 0x563ef7385a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff451dc2640 1 -- 172.21.15.103:0/3172975324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7368480 msgr2=0x563ef7385500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:32.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff451dc2640 1 --2- 172.21.15.103:0/3172975324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7368480 0x563ef7385500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff451dc2640 1 -- 172.21.15.103:0/3172975324 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ef7386000 con 0x563ef7367b60 2024-09-19T06:51:32.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff4515c1640 1 --2- 172.21.15.103:0/3172975324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7368480 0x563ef7385500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:32.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff4525c3640 1 --2- 172.21.15.103:0/3172975324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7372d40 0x563ef7385a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:32.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff451dc2640 1 --2- 172.21.15.103:0/3172975324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef73881b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff4480a06d0 tx=0x7ff448018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11faaf707a726fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:32.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff4357fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:32.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.305+0000 7ff4357fa640 1 -- 172.21.15.103:0/3172975324 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4480b5040 con 0x563ef7367b60 2024-09-19T06:51:32.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.306+0000 7ff4357fa640 1 -- 172.21.15.103:0/3172975324 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff44809a460 con 0x563ef7367b60 2024-09-19T06:51:32.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.306+0000 7ff4563ff740 1 -- 172.21.15.103:0/3172975324 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4480a4dd0 con 0x563ef7367b60 2024-09-19T06:51:32.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.306+0000 7ff4357fa640 1 -- 172.21.15.103:0/3172975324 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4480cda60 con 0x563ef7367b60 2024-09-19T06:51:32.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.306+0000 7ff4563ff740 10 client.16380 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:32.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.306+0000 7ff4563ff740 1 -- 172.21.15.103:0/3172975324 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563ef7386000 con 0x563ef7367b60 2024-09-19T06:51:32.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.306+0000 7ff4357fa640 1 -- 172.21.15.103:0/3172975324 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4480a4dd0 con 0x563ef7367b60 2024-09-19T06:51:32.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.306+0000 7ff4357fa640 1 -- 172.21.15.103:0/3172975324 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff44809b6b0 con 0x563ef7367b60 2024-09-19T06:51:32.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.306+0000 7ff4563ff740 10 client.16380 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:32.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.306+0000 7ff4563ff740 10 client.16380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:32.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.306+0000 7ff4563ff740 20 client.16380 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:32.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.306+0000 7ff4563ff740 1 --2- 172.21.15.103:0/3172975324 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563ef73828a0 0x563ef73f3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.306+0000 7ff4563ff740 4 client.16380 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:51:32.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.306+0000 7ff4563ff740 1 -- 172.21.15.103:0/3172975324 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563ef72d4d00 con 0x563ef73828a0 2024-09-19T06:51:32.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.306+0000 7ff4515c1640 1 --2- 172.21.15.103:0/3172975324 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563ef73828a0 0x563ef73f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.307+0000 7ff4357fa640 10 client.16380 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:32.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.307+0000 7ff4515c1640 1 --2- 172.21.15.103:0/3172975324 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563ef73828a0 0x563ef73f3590 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aad29e98a80c9dfe server_cookie=b5045f358ffeacd5 in_seq=0 out_seq=0 2024-09-19T06:51:32.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.308+0000 7ff4357fa640 1 -- 172.21.15.103:0/3172975324 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563ef72d4d00 con 0x563ef73828a0 2024-09-19T06:51:32.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.308+0000 7ff4357fa640 10 client.16380 handle_command_reply: tid=0 2024-09-19T06:51:32.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.308+0000 7ff4563ff740 1 client.16380 shutdown 2024-09-19T06:51:32.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.308+0000 7ff4563ff740 1 -- 172.21.15.103:0/3172975324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 msgr2=0x563ef73881b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.308+0000 7ff4563ff740 1 --2- 172.21.15.103:0/3172975324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef73881b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff4480a06d0 tx=0x7ff448018f90 comp rx=0 tx=0).stop 2024-09-19T06:51:32.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.308+0000 7ff4563ff740 1 -- 172.21.15.103:0/3172975324 shutdown_connections 2024-09-19T06:51:32.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.308+0000 7ff4563ff740 1 --2- 172.21.15.103:0/3172975324 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563ef73828a0 0x563ef73f3590 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.308+0000 7ff4563ff740 1 --2- 172.21.15.103:0/3172975324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef7372d40 0x563ef7385a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.308+0000 7ff4563ff740 1 --2- 172.21.15.103:0/3172975324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ef7368480 0x563ef7385500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.308+0000 7ff4563ff740 1 --2- 172.21.15.103:0/3172975324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ef7367b60 0x563ef73881b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.308+0000 7ff4357fa640 0 client.16380 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:32.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.308+0000 7ff4563ff740 1 -- 172.21.15.103:0/3172975324 >> 172.21.15.103:0/3172975324 conn(0x563ef73640f0 msgr2=0x563ef7363270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:32.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.308+0000 7ff4563ff740 1 -- 172.21.15.103:0/3172975324 shutdown_connections 2024-09-19T06:51:32.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.308+0000 7ff4563ff740 1 -- 172.21.15.103:0/3172975324 wait complete. 2024-09-19T06:51:32.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.308+0000 7ff4563ff740 20 client.16380 trim_cache size 0 max 16384 2024-09-19T06:51:32.331 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:51:32.332 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T06:51:32.332 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "d9cfd489-e5ff-43ac-b9a5-6309dd8600b0", 2024-09-19T06:51:32.332 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T06:51:32.332 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:51:32.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.309+0000 7ff454026640 1 -- 172.21.15.103:0/3683775929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4380a3430 msgr2=0x7ff4380a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.309+0000 7ff454026640 1 --2- 172.21.15.103:0/3683775929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4380a3430 0x7ff4380a58f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x563ef7365470 tx=0x7ff444007900 comp rx=0 tx=0).stop 2024-09-19T06:51:32.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.309+0000 7ff454026640 1 -- 172.21.15.103:0/3683775929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff44c100e20 msgr2=0x7ff44c199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.309+0000 7ff454026640 1 --2- 172.21.15.103:0/3683775929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff44c100e20 0x7ff44c199330 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff44800c920 tx=0x7ff44800cde0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.309+0000 7ff454026640 1 -- 172.21.15.103:0/3683775929 shutdown_connections 2024-09-19T06:51:32.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.309+0000 7ff454026640 1 --2- 172.21.15.103:0/3683775929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4380a3430 0x7ff4380a58f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.309+0000 7ff454026640 1 --2- 172.21.15.103:0/3683775929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff44c102ef0 0x7ff44c1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.309+0000 7ff454026640 1 --2- 172.21.15.103:0/3683775929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff44c102570 0x7ff44c199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.309+0000 7ff454026640 1 --2- 172.21.15.103:0/3683775929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff44c100e20 0x7ff44c199330 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.309+0000 7ff454026640 1 -- 172.21.15.103:0/3683775929 >> 172.21.15.103:0/3683775929 conn(0x7ff44c0fc7d0 msgr2=0x7ff44c1068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:32.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.309+0000 7ff454026640 1 -- 172.21.15.103:0/3683775929 shutdown_connections 2024-09-19T06:51:32.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:32.309+0000 7ff454026640 1 -- 172.21.15.103:0/3683775929 wait complete. 2024-09-19T06:51:32.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:32 smithi174 ceph-mon[35637]: pgmap v72: 97 pgs: 97 active+clean; 1.6 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-19T06:51:32.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:32 smithi134 ceph-mon[35356]: pgmap v72: 97 pgs: 97 active+clean; 1.6 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-19T06:51:33.141 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:51:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:33 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T06:51:33.385 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:33 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:51:33.385 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:33 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T06:51:33.385 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:33 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:51:33.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:33 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T06:51:33.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:33 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:51:33.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:33 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T06:51:33.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:33 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:51:33.391 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:33.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:33 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T06:51:33.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:33 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:51:33.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:33 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T06:51:33.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:33 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:51:33.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.809+0000 7f4d74380640 1 -- 172.21.15.103:0/2395642548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d6c102ef0 msgr2=0x7f4d6c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:33.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.809+0000 7f4d74380640 1 --2- 172.21.15.103:0/2395642548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d6c102ef0 0x7f4d6c10da40 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4d64009ca0 tx=0x7f4d6402a950 comp rx=0 tx=0).stop 2024-09-19T06:51:33.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.809+0000 7f4d74380640 1 -- 172.21.15.103:0/2395642548 shutdown_connections 2024-09-19T06:51:33.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.809+0000 7f4d74380640 1 --2- 172.21.15.103:0/2395642548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d6c102ef0 0x7f4d6c10da40 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.809+0000 7f4d74380640 1 --2- 172.21.15.103:0/2395642548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d6c102570 0x7f4d6c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.809+0000 7f4d74380640 1 --2- 172.21.15.103:0/2395642548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d6c100e20 0x7f4d6c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.809+0000 7f4d74380640 1 -- 172.21.15.103:0/2395642548 >> 172.21.15.103:0/2395642548 conn(0x7f4d6c0fc7d0 msgr2=0x7f4d6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:33.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.809+0000 7f4d74380640 1 -- 172.21.15.103:0/2395642548 shutdown_connections 2024-09-19T06:51:33.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.809+0000 7f4d74380640 1 -- 172.21.15.103:0/2395642548 wait complete. 2024-09-19T06:51:33.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.810+0000 7f4d74380640 1 Processor -- start 2024-09-19T06:51:33.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.810+0000 7f4d74380640 1 -- start start 2024-09-19T06:51:33.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.810+0000 7f4d74380640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d6c100e20 0x7f4d6c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.810+0000 7f4d74380640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d6c102570 0x7f4d6c1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.810+0000 7f4d74380640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d6c102ef0 0x7f4d6c1a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.810+0000 7f4d74380640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c110f80 con 0x7f4d6c102570 2024-09-19T06:51:33.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.810+0000 7f4d74380640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c110e00 con 0x7f4d6c102ef0 2024-09-19T06:51:33.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.810+0000 7f4d74380640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c111100 con 0x7f4d6c100e20 2024-09-19T06:51:33.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.810+0000 7f4d7191b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d6c102570 0x7f4d6c1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.810+0000 7f4d7191b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d6c102570 0x7f4d6c1998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54176/0 (socket says 172.21.15.103:54176) 2024-09-19T06:51:33.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.810+0000 7f4d7191b640 1 -- 172.21.15.103:0/2793506383 learned_addr learned my addr 172.21.15.103:0/2793506383 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:33.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.811+0000 7f4d7211c640 1 --2- 172.21.15.103:0/2793506383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d6c100e20 0x7f4d6c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.811+0000 7f4d7291d640 1 --2- 172.21.15.103:0/2793506383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d6c102ef0 0x7f4d6c1a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.811+0000 7f4d7191b640 1 -- 172.21.15.103:0/2793506383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d6c100e20 msgr2=0x7f4d6c199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:33.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.811+0000 7f4d7191b640 1 --2- 172.21.15.103:0/2793506383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d6c100e20 0x7f4d6c199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.811+0000 7f4d7191b640 1 -- 172.21.15.103:0/2793506383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d6c102ef0 msgr2=0x7f4d6c1a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:33.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.811+0000 7f4d7191b640 1 --2- 172.21.15.103:0/2793506383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d6c102ef0 0x7f4d6c1a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.811+0000 7f4d7191b640 1 -- 172.21.15.103:0/2793506383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c1a4930 con 0x7f4d6c102570 2024-09-19T06:51:33.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.811+0000 7f4d7191b640 1 --2- 172.21.15.103:0/2793506383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d6c102570 0x7f4d6c1998b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00d950 tx=0x7f4d5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:33.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.811+0000 7f4d637fe640 1 -- 172.21.15.103:0/2793506383 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c017070 con 0x7f4d6c102570 2024-09-19T06:51:33.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.811+0000 7f4d74380640 1 -- 172.21.15.103:0/2793506383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6c1a4b00 con 0x7f4d6c102570 2024-09-19T06:51:33.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.811+0000 7f4d637fe640 1 -- 172.21.15.103:0/2793506383 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5c009c30 con 0x7f4d6c102570 2024-09-19T06:51:33.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.811+0000 7f4d637fe640 1 -- 172.21.15.103:0/2793506383 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c00f6b0 con 0x7f4d6c102570 2024-09-19T06:51:33.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.811+0000 7f4d74380640 1 -- 172.21.15.103:0/2793506383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6c1a4fe0 con 0x7f4d6c102570 2024-09-19T06:51:33.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.813+0000 7f4d637fe640 1 -- 172.21.15.103:0/2793506383 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4d5c0109d0 con 0x7f4d6c102570 2024-09-19T06:51:33.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.814+0000 7f4d76759740 1 Processor -- start 2024-09-19T06:51:33.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.814+0000 7f4d76759740 1 -- start start 2024-09-19T06:51:33.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.814+0000 7f4d76759740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163ed8b0 0x55dd163edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.814+0000 7f4d76759740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd163f8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.814+0000 7f4d76759740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163f8a90 0x55dd163fae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.814+0000 7f4d76759740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dd1634f2e0 con 0x55dd163ed8b0 2024-09-19T06:51:33.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.814+0000 7f4d76759740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dd162f6ce0 con 0x55dd163f8a90 2024-09-19T06:51:33.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.814+0000 7f4d76759740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dd1634b2a0 con 0x55dd163ee1d0 2024-09-19T06:51:33.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.814+0000 7f4d637fe640 1 --2- 172.21.15.103:0/2793506383 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4d580a33b0 0x7f4d580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.814+0000 7f4d7291d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163ed8b0 0x55dd163edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.814+0000 7f4d7191b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163f8a90 0x55dd163fae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.814+0000 7f4d7291d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163ed8b0 0x55dd163edc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54178/0 (socket says 172.21.15.103:54178) 2024-09-19T06:51:33.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.814+0000 7f4d7211c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd163f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.814+0000 7f4d7291d640 1 -- 172.21.15.103:0/2468718542 learned_addr learned my addr 172.21.15.103:0/2468718542 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:33.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.815+0000 7f4d637fe640 1 -- 172.21.15.103:0/2793506383 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f4d5c0cf820 con 0x7f4d6c102570 2024-09-19T06:51:33.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.815+0000 7f4d7291d640 1 --2- 172.21.15.103:0/2793506383 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4d580a33b0 0x7f4d580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.815+0000 7f4d7211c640 1 -- 172.21.15.103:0/2468718542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163f8a90 msgr2=0x55dd163fae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:33.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.815+0000 7f4d7211c640 1 --2- 172.21.15.103:0/2468718542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163f8a90 0x55dd163fae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.815+0000 7f4d7211c640 1 -- 172.21.15.103:0/2468718542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163ed8b0 msgr2=0x55dd163edc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:33.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.815+0000 7f4d7211c640 1 --2- 172.21.15.103:0/2468718542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163ed8b0 0x55dd163edc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.815+0000 7f4d7211c640 1 -- 172.21.15.103:0/2468718542 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd162db440 con 0x55dd163ee1d0 2024-09-19T06:51:33.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.815+0000 7f4d7291d640 1 --2- 172.21.15.103:0/2468718542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163ed8b0 0x55dd163edc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:33.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.815+0000 7f4d7211c640 1 --2- 172.21.15.103:0/2468718542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd163f8550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800ab40 tx=0x7f4d6800f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aad27de7176177c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:33.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.815+0000 7f4d7291d640 1 --2- 172.21.15.103:0/2793506383 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4d580a33b0 0x7f4d580a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4d64030040 tx=0x7f4d6402a950 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:33.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.815+0000 7f4d60dea640 1 -- 172.21.15.103:0/2468718542 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68004020 con 0x55dd163ee1d0 2024-09-19T06:51:33.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.815+0000 7f4d60dea640 1 -- 172.21.15.103:0/2468718542 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d680041c0 con 0x55dd163ee1d0 2024-09-19T06:51:33.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.815+0000 7f4d60dea640 1 -- 172.21.15.103:0/2468718542 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68018780 con 0x55dd163ee1d0 2024-09-19T06:51:33.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.816+0000 7f4d76759740 1 -- 172.21.15.103:0/2468718542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 msgr2=0x55dd163f8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:33.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.816+0000 7f4d76759740 1 --2- 172.21.15.103:0/2468718542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd163f8550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800ab40 tx=0x7f4d6800f3c0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.816+0000 7f4d76759740 1 -- 172.21.15.103:0/2468718542 shutdown_connections 2024-09-19T06:51:33.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.816+0000 7f4d76759740 1 --2- 172.21.15.103:0/2468718542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163f8a90 0x55dd163fae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.816+0000 7f4d76759740 1 --2- 172.21.15.103:0/2468718542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd163f8550 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.816+0000 7f4d76759740 1 --2- 172.21.15.103:0/2468718542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163ed8b0 0x55dd163edc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.816+0000 7f4d76759740 1 -- 172.21.15.103:0/2468718542 >> 172.21.15.103:0/2468718542 conn(0x55dd163e9e40 msgr2=0x55dd163ea220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:33.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.816+0000 7f4d76759740 1 -- 172.21.15.103:0/2468718542 shutdown_connections 2024-09-19T06:51:33.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.816+0000 7f4d76759740 1 -- 172.21.15.103:0/2468718542 wait complete. 2024-09-19T06:51:33.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.816+0000 7f4d76759740 1 -- start start 2024-09-19T06:51:33.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d76759740 1 Processor -- start 2024-09-19T06:51:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d76759740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163ed8b0 0x55dd163f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d76759740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd163f5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d76759740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163f8a90 0x55dd163f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d76759740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dd163fb660 con 0x55dd163f8a90 2024-09-19T06:51:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d76759740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dd1630a050 con 0x55dd163ed8b0 2024-09-19T06:51:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d76759740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dd163fb7e0 con 0x55dd163ee1d0 2024-09-19T06:51:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d76759740 10 client.0 _ll_register_callbacks cb 0x55dd1634a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d7291d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163f8a90 0x55dd163f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d7211c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163ed8b0 0x55dd163f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d7291d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163f8a90 0x55dd163f5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54192/0 (socket says 172.21.15.103:54192) 2024-09-19T06:51:33.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d7291d640 1 -- 172.21.15.103:0/150209412 learned_addr learned my addr 172.21.15.103:0/150209412 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:33.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d7191b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd163f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d7291d640 1 -- 172.21.15.103:0/150209412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 msgr2=0x55dd163f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:33.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d7291d640 1 --2- 172.21.15.103:0/150209412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd163f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d7291d640 1 -- 172.21.15.103:0/150209412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163ed8b0 msgr2=0x55dd163f82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:33.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d7291d640 1 --2- 172.21.15.103:0/150209412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163ed8b0 0x55dd163f82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d7291d640 1 -- 172.21.15.103:0/150209412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd160c03c0 con 0x55dd163f8a90 2024-09-19T06:51:33.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d7191b640 1 --2- 172.21.15.103:0/150209412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd163f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:33.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.817+0000 7f4d7211c640 1 --2- 172.21.15.103:0/150209412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163ed8b0 0x55dd163f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:33.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.818+0000 7f4d7291d640 1 --2- 172.21.15.103:0/150209412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163f8a90 0x55dd163f5ac0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4d64005cc0 tx=0x7f4d640095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2954aea447d91519 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:33.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.818+0000 7f4d55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:33.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.818+0000 7f4d55ffb640 1 -- 172.21.15.103:0/150209412 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d64034040 con 0x55dd163f8a90 2024-09-19T06:51:33.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.818+0000 7f4d55ffb640 1 -- 172.21.15.103:0/150209412 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f4d64004300 con 0x55dd163f8a90 2024-09-19T06:51:33.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.818+0000 7f4d76759740 1 -- 172.21.15.103:0/150209412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d680041c0 con 0x55dd163f8a90 2024-09-19T06:51:33.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.818+0000 7f4d55ffb640 1 -- 172.21.15.103:0/150209412 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d64004500 con 0x55dd163f8a90 2024-09-19T06:51:33.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.818+0000 7f4d76759740 10 client.16392 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:33.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.818+0000 7f4d76759740 1 -- 172.21.15.103:0/150209412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dd16406e90 con 0x55dd163f8a90 2024-09-19T06:51:33.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.818+0000 7f4d55ffb640 1 -- 172.21.15.103:0/150209412 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d680041c0 con 0x55dd163f8a90 2024-09-19T06:51:33.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.818+0000 7f4d55ffb640 1 -- 172.21.15.103:0/150209412 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4d64048070 con 0x55dd163f8a90 2024-09-19T06:51:33.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.818+0000 7f4d76759740 10 client.16392 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:33.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.818+0000 7f4d76759740 10 client.16392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:33.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.818+0000 7f4d76759740 20 client.16392 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:33.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.819+0000 7f4d76759740 1 --2- 172.21.15.103:0/150209412 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dd16408350 0x55dd1640a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.819+0000 7f4d76759740 4 client.16392 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:33.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.819+0000 7f4d76759740 1 -- 172.21.15.103:0/150209412 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd1635c850 con 0x55dd16408350 2024-09-19T06:51:33.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.819+0000 7f4d7211c640 1 --2- 172.21.15.103:0/150209412 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dd16408350 0x55dd1640a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.819+0000 7f4d7211c640 1 --2- 172.21.15.103:0/150209412 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dd16408350 0x55dd1640a760 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dde58f189cb0340 server_cookie=cb90e1d3fb742494 in_seq=0 out_seq=0 2024-09-19T06:51:33.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.819+0000 7f4d55ffb640 10 client.16392 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:33.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.820+0000 7f4d55ffb640 1 -- 172.21.15.103:0/150209412 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd1635c850 con 0x55dd16408350 2024-09-19T06:51:33.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.820+0000 7f4d55ffb640 10 client.16392 handle_command_reply: tid=0 2024-09-19T06:51:33.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.820+0000 7f4d76759740 1 client.16392 shutdown 2024-09-19T06:51:33.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.820+0000 7f4d76759740 1 -- 172.21.15.103:0/150209412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163f8a90 msgr2=0x55dd163f5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:33.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.820+0000 7f4d76759740 1 --2- 172.21.15.103:0/150209412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163f8a90 0x55dd163f5ac0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4d64005cc0 tx=0x7f4d640095c0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.820+0000 7f4d76759740 1 -- 172.21.15.103:0/150209412 shutdown_connections 2024-09-19T06:51:33.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.820+0000 7f4d76759740 1 --2- 172.21.15.103:0/150209412 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dd16408350 0x55dd1640a760 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.820+0000 7f4d76759740 1 --2- 172.21.15.103:0/150209412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163f8a90 0x55dd163f5ac0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.820+0000 7f4d76759740 1 --2- 172.21.15.103:0/150209412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd163f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.820+0000 7f4d76759740 1 --2- 172.21.15.103:0/150209412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163ed8b0 0x55dd163f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.820+0000 7f4d55ffb640 0 client.16392 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:33.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.820+0000 7f4d76759740 1 -- 172.21.15.103:0/150209412 >> 172.21.15.103:0/150209412 conn(0x55dd163e9e40 msgr2=0x55dd163efb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:33.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.820+0000 7f4d76759740 1 -- 172.21.15.103:0/150209412 shutdown_connections 2024-09-19T06:51:33.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.820+0000 7f4d76759740 1 -- 172.21.15.103:0/150209412 wait complete. 2024-09-19T06:51:33.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.821+0000 7f4d76759740 20 client.16392 trim_cache size 0 max 16384 2024-09-19T06:51:33.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.837+0000 7f4d76759740 1 Processor -- start 2024-09-19T06:51:33.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.837+0000 7f4d76759740 1 -- start start 2024-09-19T06:51:33.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.837+0000 7f4d76759740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163e9e40 0x55dd163e4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.837+0000 7f4d76759740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163ed8b0 0x55dd16410030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.837+0000 7f4d76759740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd16410570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.837+0000 7f4d76759740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1ab700 con 0x55dd163e9e40 2024-09-19T06:51:33.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.837+0000 7f4d76759740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1ad240 con 0x55dd163ed8b0 2024-09-19T06:51:33.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.837+0000 7f4d76759740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1abdd0 con 0x55dd163ee1d0 2024-09-19T06:51:33.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.837+0000 7f4d7211c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163e9e40 0x55dd163e4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.837+0000 7f4d7211c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163e9e40 0x55dd163e4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54194/0 (socket says 172.21.15.103:54194) 2024-09-19T06:51:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.838+0000 7f4d7211c640 1 -- 172.21.15.103:0/12466187 learned_addr learned my addr 172.21.15.103:0/12466187 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.838+0000 7f4d7291d640 1 --2- 172.21.15.103:0/12466187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd16410570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.838+0000 7f4d7191b640 1 --2- 172.21.15.103:0/12466187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163ed8b0 0x55dd16410030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.838+0000 7f4d7211c640 1 -- 172.21.15.103:0/12466187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 msgr2=0x55dd16410570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.838+0000 7f4d7211c640 1 --2- 172.21.15.103:0/12466187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd16410570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.838+0000 7f4d7211c640 1 -- 172.21.15.103:0/12466187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163ed8b0 msgr2=0x55dd16410030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.838+0000 7f4d7211c640 1 --2- 172.21.15.103:0/12466187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163ed8b0 0x55dd16410030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.838+0000 7f4d7211c640 1 -- 172.21.15.103:0/12466187 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd163e9c70 con 0x55dd163e9e40 2024-09-19T06:51:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.838+0000 7f4d7191b640 1 --2- 172.21.15.103:0/12466187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163ed8b0 0x55dd16410030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.838+0000 7f4d7211c640 1 --2- 172.21.15.103:0/12466187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163e9e40 0x55dd163e4a60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4d6801a040 tx=0x7f4d6800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2896d239e92d1fb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.838+0000 7f4d57fff640 1 -- 172.21.15.103:0/12466187 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d680040c0 con 0x55dd163e9e40 2024-09-19T06:51:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.838+0000 7f4d57fff640 1 -- 172.21.15.103:0/12466187 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d68003de0 con 0x55dd163e9e40 2024-09-19T06:51:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.838+0000 7f4d57fff640 1 -- 172.21.15.103:0/12466187 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68004660 con 0x55dd163e9e40 2024-09-19T06:51:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 -- 172.21.15.103:0/12466187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163e9e40 msgr2=0x55dd163e4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:33.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 --2- 172.21.15.103:0/12466187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163e9e40 0x55dd163e4a60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4d6801a040 tx=0x7f4d6800f3c0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 -- 172.21.15.103:0/12466187 shutdown_connections 2024-09-19T06:51:33.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 --2- 172.21.15.103:0/12466187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd16410570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 --2- 172.21.15.103:0/12466187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163ed8b0 0x55dd16410030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 --2- 172.21.15.103:0/12466187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163e9e40 0x55dd163e4a60 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 -- 172.21.15.103:0/12466187 >> 172.21.15.103:0/12466187 conn(0x55dd163f5c20 msgr2=0x55dd163e4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:33.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 -- 172.21.15.103:0/12466187 shutdown_connections 2024-09-19T06:51:33.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 -- 172.21.15.103:0/12466187 wait complete. 2024-09-19T06:51:33.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 -- start start 2024-09-19T06:51:33.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 Processor -- start 2024-09-19T06:51:33.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163e9e40 0x55dd16401210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163ed8b0 0x55dd163fc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd163fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1acb70 con 0x55dd163ed8b0 2024-09-19T06:51:33.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1ac4a0 con 0x55dd163e9e40 2024-09-19T06:51:33.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1a96c0 con 0x55dd163ee1d0 2024-09-19T06:51:33.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d76759740 10 client.0 _ll_register_callbacks cb 0x7f4d6402f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:33.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.839+0000 7f4d7191b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163ed8b0 0x55dd163fc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d7191b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163ed8b0 0x55dd163fc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54198/0 (socket says 172.21.15.103:54198) 2024-09-19T06:51:33.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d7191b640 1 -- 172.21.15.103:0/2607363765 learned_addr learned my addr 172.21.15.103:0/2607363765 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:33.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d7211c640 1 --2- 172.21.15.103:0/2607363765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163e9e40 0x55dd16401210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d7291d640 1 --2- 172.21.15.103:0/2607363765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd163fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d7191b640 1 -- 172.21.15.103:0/2607363765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 msgr2=0x55dd163fcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:33.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d7191b640 1 --2- 172.21.15.103:0/2607363765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd163fcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d7191b640 1 -- 172.21.15.103:0/2607363765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163e9e40 msgr2=0x55dd16401210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:33.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d7191b640 1 --2- 172.21.15.103:0/2607363765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163e9e40 0x55dd16401210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d7191b640 1 -- 172.21.15.103:0/2607363765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd163eaad0 con 0x55dd163ed8b0 2024-09-19T06:51:33.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d7291d640 1 --2- 172.21.15.103:0/2607363765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd163fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d7191b640 1 --2- 172.21.15.103:0/2607363765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163ed8b0 0x55dd163fc5e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0cfac0 tx=0x7f4d5c06a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db06964c2ed408c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d55ffb640 1 -- 172.21.15.103:0/2607363765 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c020040 con 0x55dd163ed8b0 2024-09-19T06:51:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d55ffb640 1 -- 172.21.15.103:0/2607363765 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f4d5c0a42b0 con 0x55dd163ed8b0 2024-09-19T06:51:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d76759740 1 -- 172.21.15.103:0/2607363765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d68003de0 con 0x55dd163ed8b0 2024-09-19T06:51:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d55ffb640 1 -- 172.21.15.103:0/2607363765 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d5c0cd3d0 con 0x55dd163ed8b0 2024-09-19T06:51:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d76759740 10 client.16398 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d76759740 1 -- 172.21.15.103:0/2607363765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dd16472e30 con 0x55dd163ed8b0 2024-09-19T06:51:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.840+0000 7f4d55ffb640 1 -- 172.21.15.103:0/2607363765 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d68003de0 con 0x55dd163ed8b0 2024-09-19T06:51:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.841+0000 7f4d55ffb640 1 -- 172.21.15.103:0/2607363765 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4d5c010460 con 0x55dd163ed8b0 2024-09-19T06:51:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.841+0000 7f4d76759740 10 client.16398 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.841+0000 7f4d76759740 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.841+0000 7f4d76759740 20 client.16398 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.841+0000 7f4d76759740 1 --2- 172.21.15.103:0/2607363765 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dd163f8a90 0x55dd164755f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.841+0000 7f4d76759740 4 client.16398 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:51:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.841+0000 7f4d76759740 1 -- 172.21.15.103:0/2607363765 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dd16356d60 con 0x55dd163f8a90 2024-09-19T06:51:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.841+0000 7f4d7211c640 1 --2- 172.21.15.103:0/2607363765 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dd163f8a90 0x55dd164755f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.841+0000 7f4d7211c640 1 --2- 172.21.15.103:0/2607363765 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dd163f8a90 0x55dd164755f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf1fa680915d4797 server_cookie=d17d0255065792f1 in_seq=0 out_seq=0 2024-09-19T06:51:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.841+0000 7f4d55ffb640 10 client.16398 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d55ffb640 1 -- 172.21.15.103:0/2607363765 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dd16356d60 con 0x55dd163f8a90 2024-09-19T06:51:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d55ffb640 10 client.16398 handle_command_reply: tid=0 2024-09-19T06:51:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d76759740 1 client.16398 shutdown 2024-09-19T06:51:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d76759740 1 -- 172.21.15.103:0/2607363765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163ed8b0 msgr2=0x55dd163fc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d76759740 1 --2- 172.21.15.103:0/2607363765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163ed8b0 0x55dd163fc5e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0cfac0 tx=0x7f4d5c06a870 comp rx=0 tx=0).stop 2024-09-19T06:51:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d76759740 1 -- 172.21.15.103:0/2607363765 shutdown_connections 2024-09-19T06:51:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d76759740 1 --2- 172.21.15.103:0/2607363765 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dd163f8a90 0x55dd164755f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d55ffb640 0 client.16398 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d76759740 1 --2- 172.21.15.103:0/2607363765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd163ee1d0 0x55dd163fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d76759740 1 --2- 172.21.15.103:0/2607363765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd163ed8b0 0x55dd163fc5e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d76759740 1 --2- 172.21.15.103:0/2607363765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dd163e9e40 0x55dd16401210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d76759740 1 -- 172.21.15.103:0/2607363765 >> 172.21.15.103:0/2607363765 conn(0x55dd163f5c20 msgr2=0x55dd163f7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:33.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d76759740 1 -- 172.21.15.103:0/2607363765 shutdown_connections 2024-09-19T06:51:33.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d76759740 1 -- 172.21.15.103:0/2607363765 wait complete. 2024-09-19T06:51:33.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d76759740 20 client.16398 trim_cache size 0 max 16384 2024-09-19T06:51:33.850 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:51:33.850 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:51:33.850 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T06:51:33.850 INFO:teuthology.orchestra.run.smithi103.stdout: "d9cfd489-e5ff-43ac-b9a5-6309dd8600b0": { 2024-09-19T06:51:33.851 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T06:51:33.851 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "d9cfd489-e5ff-43ac-b9a5-6309dd8600b0", 2024-09-19T06:51:33.851 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T06:51:33.851 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:51:33.851 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:51:33.851 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:51:33.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d74380640 1 -- 172.21.15.103:0/2793506383 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4d580a33b0 msgr2=0x7f4d580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:33.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d74380640 1 --2- 172.21.15.103:0/2793506383 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4d580a33b0 0x7f4d580a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4d64030040 tx=0x7f4d6402a950 comp rx=0 tx=0).stop 2024-09-19T06:51:33.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d74380640 1 -- 172.21.15.103:0/2793506383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d6c102570 msgr2=0x7f4d6c1998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:33.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d74380640 1 --2- 172.21.15.103:0/2793506383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d6c102570 0x7f4d6c1998b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00d950 tx=0x7f4d5c00de10 comp rx=0 tx=0).stop 2024-09-19T06:51:33.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d74380640 1 -- 172.21.15.103:0/2793506383 shutdown_connections 2024-09-19T06:51:33.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d74380640 1 --2- 172.21.15.103:0/2793506383 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4d580a33b0 0x7f4d580a5870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d74380640 1 --2- 172.21.15.103:0/2793506383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d6c102ef0 0x7f4d6c1a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d74380640 1 --2- 172.21.15.103:0/2793506383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d6c102570 0x7f4d6c1998b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d74380640 1 --2- 172.21.15.103:0/2793506383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d6c100e20 0x7f4d6c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:33.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.842+0000 7f4d74380640 1 -- 172.21.15.103:0/2793506383 >> 172.21.15.103:0/2793506383 conn(0x7f4d6c0fc7d0 msgr2=0x7f4d6c1068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:33.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.843+0000 7f4d74380640 1 -- 172.21.15.103:0/2793506383 shutdown_connections 2024-09-19T06:51:33.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:51:33.843+0000 7f4d74380640 1 -- 172.21.15.103:0/2793506383 wait complete. 2024-09-19T06:51:34.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:34 smithi103 ceph-mon[31158]: pgmap v73: 97 pgs: 97 active+clean; 1.6 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-19T06:51:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:34 smithi174 ceph-mon[35637]: pgmap v73: 97 pgs: 97 active+clean; 1.6 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-19T06:51:34.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:34 smithi134 ceph-mon[35356]: pgmap v73: 97 pgs: 97 active+clean; 1.6 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-19T06:51:34.804 INFO:tasks.cephfs.filesystem:scrub status for tag:d9cfd489-e5ff-43ac-b9a5-6309dd8600b0 - {'path': '/', 'tag': 'd9cfd489-e5ff-43ac-b9a5-6309dd8600b0', 'options': 'recursive,force'} 2024-09-19T06:51:35.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:35 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T06:51:35.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:35 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T06:51:35.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:35 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T06:51:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:36 smithi174 ceph-mon[35637]: pgmap v74: 97 pgs: 97 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-19T06:51:36.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:36 smithi103 ceph-mon[31158]: pgmap v74: 97 pgs: 97 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-19T06:51:36.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:36 smithi134 ceph-mon[35356]: pgmap v74: 97 pgs: 97 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-19T06:51:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.696+0000 7ff90beaf640 1 -- 172.21.15.103:0/2958300418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9041035c0 msgr2=0x7ff90410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.696+0000 7ff90beaf640 1 --2- 172.21.15.103:0/2958300418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9041035c0 0x7ff90410baa0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009900 tx=0x7ff8f402a480 comp rx=0 tx=0).stop 2024-09-19T06:51:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.696+0000 7ff90beaf640 1 -- 172.21.15.103:0/2958300418 shutdown_connections 2024-09-19T06:51:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.696+0000 7ff90beaf640 1 --2- 172.21.15.103:0/2958300418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff90410c0c0 0x7ff90410e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.696+0000 7ff90beaf640 1 --2- 172.21.15.103:0/2958300418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9041035c0 0x7ff90410baa0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.696+0000 7ff90beaf640 1 --2- 172.21.15.103:0/2958300418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff904102ca0 0x7ff904103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.696+0000 7ff90beaf640 1 -- 172.21.15.103:0/2958300418 >> 172.21.15.103:0/2958300418 conn(0x7ff9040fc6f0 msgr2=0x7ff9040feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.696+0000 7ff90beaf640 1 -- 172.21.15.103:0/2958300418 shutdown_connections 2024-09-19T06:51:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.696+0000 7ff90beaf640 1 -- 172.21.15.103:0/2958300418 wait complete. 2024-09-19T06:51:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff90beaf640 1 Processor -- start 2024-09-19T06:51:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff90beaf640 1 -- start start 2024-09-19T06:51:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff90beaf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff904102ca0 0x7ff904198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff90beaf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9041035c0 0x7ff904198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff90beaf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff90410c0c0 0x7ff90419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff90beaf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff904111ee0 con 0x7ff904102ca0 2024-09-19T06:51:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff90beaf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff904111d60 con 0x7ff90410c0c0 2024-09-19T06:51:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff90beaf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff904112060 con 0x7ff9041035c0 2024-09-19T06:51:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff909c4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff904102ca0 0x7ff904198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff909c4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff904102ca0 0x7ff904198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54212/0 (socket says 172.21.15.103:54212) 2024-09-19T06:51:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff909c4b640 1 -- 172.21.15.103:0/4110534891 learned_addr learned my addr 172.21.15.103:0/4110534891 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff90944a640 1 --2- 172.21.15.103:0/4110534891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9041035c0 0x7ff904198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff90a44c640 1 --2- 172.21.15.103:0/4110534891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff90410c0c0 0x7ff90419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff909c4b640 1 -- 172.21.15.103:0/4110534891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9041035c0 msgr2=0x7ff904198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff909c4b640 1 --2- 172.21.15.103:0/4110534891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9041035c0 0x7ff904198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff909c4b640 1 -- 172.21.15.103:0/4110534891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff90410c0c0 msgr2=0x7ff90419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff909c4b640 1 --2- 172.21.15.103:0/4110534891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff90410c0c0 0x7ff90419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff909c4b640 1 -- 172.21.15.103:0/4110534891 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90419d300 con 0x7ff904102ca0 2024-09-19T06:51:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff90944a640 1 --2- 172.21.15.103:0/4110534891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9041035c0 0x7ff904198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.697+0000 7ff90a44c640 1 --2- 172.21.15.103:0/4110534891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff90410c0c0 0x7ff90419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.698+0000 7ff909c4b640 1 --2- 172.21.15.103:0/4110534891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff904102ca0 0x7ff904198310 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff90000d950 tx=0x7ff90000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.698+0000 7ff8faffd640 1 -- 172.21.15.103:0/4110534891 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff900017070 con 0x7ff904102ca0 2024-09-19T06:51:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.698+0000 7ff8faffd640 1 -- 172.21.15.103:0/4110534891 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff900009c30 con 0x7ff904102ca0 2024-09-19T06:51:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.698+0000 7ff90beaf640 1 -- 172.21.15.103:0/4110534891 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff90419d5f0 con 0x7ff904102ca0 2024-09-19T06:51:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.698+0000 7ff8faffd640 1 -- 172.21.15.103:0/4110534891 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90000f6b0 con 0x7ff904102ca0 2024-09-19T06:51:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.698+0000 7ff90beaf640 1 -- 172.21.15.103:0/4110534891 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9041a4e80 con 0x7ff904102ca0 2024-09-19T06:51:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.701+0000 7ff8faffd640 1 -- 172.21.15.103:0/4110534891 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff9000109d0 con 0x7ff904102ca0 2024-09-19T06:51:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.701+0000 7ff90ec40740 1 Processor -- start 2024-09-19T06:51:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.701+0000 7ff90ec40740 1 -- start start 2024-09-19T06:51:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.701+0000 7ff90ec40740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c42af0 0x5571b1c42ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.701+0000 7ff90ec40740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c43410 0x5571b1c4d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.701+0000 7ff90ec40740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c4dcb0 0x5571b1c500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.701+0000 7ff90ec40740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5571b1ba72f0 con 0x5571b1c42af0 2024-09-19T06:51:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.701+0000 7ff90ec40740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5571b1b4f270 con 0x5571b1c4dcb0 2024-09-19T06:51:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.701+0000 7ff90ec40740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5571b1ba32b0 con 0x5571b1c43410 2024-09-19T06:51:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff90a44c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c42af0 0x5571b1c42ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff90a44c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c42af0 0x5571b1c42ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54218/0 (socket says 172.21.15.103:54218) 2024-09-19T06:51:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff90a44c640 1 -- 172.21.15.103:0/2536579625 learned_addr learned my addr 172.21.15.103:0/2536579625 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff90944a640 1 --2- 172.21.15.103:0/2536579625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c4dcb0 0x5571b1c500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff909c4b640 1 --2- 172.21.15.103:0/2536579625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c43410 0x5571b1c4d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff8faffd640 1 --2- 172.21.15.103:0/4110534891 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff8f00a3460 0x7ff8f00a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff90a44c640 1 -- 172.21.15.103:0/2536579625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c43410 msgr2=0x5571b1c4d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff90a44c640 1 --2- 172.21.15.103:0/2536579625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c43410 0x5571b1c4d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff90a44c640 1 -- 172.21.15.103:0/2536579625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c4dcb0 msgr2=0x5571b1c500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff90a44c640 1 --2- 172.21.15.103:0/2536579625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c4dcb0 0x5571b1c500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff90a44c640 1 -- 172.21.15.103:0/2536579625 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571b1b4e340 con 0x5571b1c42af0 2024-09-19T06:51:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff90a44c640 1 --2- 172.21.15.103:0/4110534891 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff8f00a3460 0x7ff8f00a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff8faffd640 1 -- 172.21.15.103:0/4110534891 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff9000cfa10 con 0x7ff904102ca0 2024-09-19T06:51:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff90a44c640 1 --2- 172.21.15.103:0/2536579625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c42af0 0x5571b1c42ed0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00f3e0 tx=0x7ff8fc00f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89d9942e2fe5084 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff8effff640 1 -- 172.21.15.103:0/2536579625 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc01c400 con 0x5571b1c42af0 2024-09-19T06:51:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff8effff640 1 -- 172.21.15.103:0/2536579625 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8fc01ca80 con 0x5571b1c42af0 2024-09-19T06:51:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.702+0000 7ff8effff640 1 -- 172.21.15.103:0/2536579625 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc01b800 con 0x5571b1c42af0 2024-09-19T06:51:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.703+0000 7ff90ec40740 1 -- 172.21.15.103:0/2536579625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c42af0 msgr2=0x5571b1c42ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.703+0000 7ff90a44c640 1 --2- 172.21.15.103:0/4110534891 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff8f00a3460 0x7ff8f00a5920 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc020000 tx=0x7ff8fc01d000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.703+0000 7ff90ec40740 1 --2- 172.21.15.103:0/2536579625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c42af0 0x5571b1c42ed0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00f3e0 tx=0x7ff8fc00f7e0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.703+0000 7ff90ec40740 1 -- 172.21.15.103:0/2536579625 shutdown_connections 2024-09-19T06:51:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.703+0000 7ff90ec40740 1 --2- 172.21.15.103:0/2536579625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c4dcb0 0x5571b1c500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.703+0000 7ff90ec40740 1 --2- 172.21.15.103:0/2536579625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c43410 0x5571b1c4d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.703+0000 7ff90ec40740 1 --2- 172.21.15.103:0/2536579625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c42af0 0x5571b1c42ed0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.703+0000 7ff90ec40740 1 -- 172.21.15.103:0/2536579625 >> 172.21.15.103:0/2536579625 conn(0x5571b1c3f060 msgr2=0x5571b1c3f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.703+0000 7ff90ec40740 1 -- 172.21.15.103:0/2536579625 shutdown_connections 2024-09-19T06:51:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.703+0000 7ff90ec40740 1 -- 172.21.15.103:0/2536579625 wait complete. 2024-09-19T06:51:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.704+0000 7ff90ec40740 1 -- start start 2024-09-19T06:51:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.704+0000 7ff90ec40740 1 Processor -- start 2024-09-19T06:51:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.704+0000 7ff90ec40740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c42af0 0x5571b1c4ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.704+0000 7ff90ec40740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c43410 0x5571b1c4b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.704+0000 7ff90ec40740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c4dcb0 0x5571b1c4b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.704+0000 7ff90ec40740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5571b1c50880 con 0x5571b1c4dcb0 2024-09-19T06:51:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.704+0000 7ff90ec40740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5571b1b30290 con 0x5571b1c43410 2024-09-19T06:51:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.704+0000 7ff90ec40740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5571b1c50a00 con 0x5571b1c42af0 2024-09-19T06:51:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.704+0000 7ff90ec40740 10 client.0 _ll_register_callbacks cb 0x5571b1b61140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.704+0000 7ff90a44c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c4dcb0 0x5571b1c4b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.704+0000 7ff90a44c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c4dcb0 0x5571b1c4b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54226/0 (socket says 172.21.15.103:54226) 2024-09-19T06:51:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.704+0000 7ff90a44c640 1 -- 172.21.15.103:0/3345512197 learned_addr learned my addr 172.21.15.103:0/3345512197 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.705+0000 7ff90944a640 1 --2- 172.21.15.103:0/3345512197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c43410 0x5571b1c4b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.705+0000 7ff909c4b640 1 --2- 172.21.15.103:0/3345512197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c42af0 0x5571b1c4ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.705+0000 7ff90a44c640 1 -- 172.21.15.103:0/3345512197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c42af0 msgr2=0x5571b1c4ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.705+0000 7ff90a44c640 1 --2- 172.21.15.103:0/3345512197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c42af0 0x5571b1c4ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.705+0000 7ff90a44c640 1 -- 172.21.15.103:0/3345512197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c43410 msgr2=0x5571b1c4b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.705+0000 7ff90a44c640 1 --2- 172.21.15.103:0/3345512197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c43410 0x5571b1c4b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.705+0000 7ff90a44c640 1 -- 172.21.15.103:0/3345512197 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571b18ba210 con 0x5571b1c4dcb0 2024-09-19T06:51:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.705+0000 7ff90a44c640 1 --2- 172.21.15.103:0/3345512197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c4dcb0 0x5571b1c4b610 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00fce0 tx=0x7ff8fc01b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703bb6937fb1fc81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.705+0000 7ff8ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.705+0000 7ff8ed7fa640 1 -- 172.21.15.103:0/3345512197 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc01b7d0 con 0x5571b1c4dcb0 2024-09-19T06:51:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.705+0000 7ff90ec40740 1 -- 172.21.15.103:0/3345512197 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8fc01ca80 con 0x5571b1c4dcb0 2024-09-19T06:51:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.705+0000 7ff8ed7fa640 1 -- 172.21.15.103:0/3345512197 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff8fc01bb20 con 0x5571b1c4dcb0 2024-09-19T06:51:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.705+0000 7ff8ed7fa640 1 -- 172.21.15.103:0/3345512197 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff8fc03c050 con 0x5571b1c4dcb0 2024-09-19T06:51:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.705+0000 7ff90ec40740 10 client.16410 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.705+0000 7ff90ec40740 1 -- 172.21.15.103:0/3345512197 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5571b1986620 con 0x5571b1c4dcb0 2024-09-19T06:51:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.705+0000 7ff8ed7fa640 1 -- 172.21.15.103:0/3345512197 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8fc01ca80 con 0x5571b1c4dcb0 2024-09-19T06:51:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.706+0000 7ff8ed7fa640 1 -- 172.21.15.103:0/3345512197 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff8fc037030 con 0x5571b1c4dcb0 2024-09-19T06:51:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.706+0000 7ff90ec40740 10 client.16410 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.706+0000 7ff90ec40740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.706+0000 7ff90ec40740 20 client.16410 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.706+0000 7ff90ec40740 1 --2- 172.21.15.103:0/3345512197 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5571b1c5dba0 0x5571b1c5ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.706+0000 7ff90ec40740 4 client.16410 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.706+0000 7ff90ec40740 1 -- 172.21.15.103:0/3345512197 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571b1bb4a20 con 0x5571b1c5dba0 2024-09-19T06:51:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.706+0000 7ff909c4b640 1 --2- 172.21.15.103:0/3345512197 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5571b1c5dba0 0x5571b1c5ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.706+0000 7ff8ed7fa640 10 client.16410 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.706+0000 7ff909c4b640 1 --2- 172.21.15.103:0/3345512197 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5571b1c5dba0 0x5571b1c5ff90 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=273edd42036566a2 server_cookie=c8ff4f1ebf4cd8fa in_seq=0 out_seq=0 2024-09-19T06:51:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.707+0000 7ff8ed7fa640 1 -- 172.21.15.103:0/3345512197 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571b1bb4a20 con 0x5571b1c5dba0 2024-09-19T06:51:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.707+0000 7ff8ed7fa640 10 client.16410 handle_command_reply: tid=0 2024-09-19T06:51:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.707+0000 7ff90ec40740 1 client.16410 shutdown 2024-09-19T06:51:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.707+0000 7ff90ec40740 1 -- 172.21.15.103:0/3345512197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c4dcb0 msgr2=0x5571b1c4b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.707+0000 7ff90ec40740 1 --2- 172.21.15.103:0/3345512197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c4dcb0 0x5571b1c4b610 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00fce0 tx=0x7ff8fc01b4e0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.707+0000 7ff90ec40740 1 -- 172.21.15.103:0/3345512197 shutdown_connections 2024-09-19T06:51:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.707+0000 7ff90ec40740 1 --2- 172.21.15.103:0/3345512197 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5571b1c5dba0 0x5571b1c5ff90 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.707+0000 7ff90ec40740 1 --2- 172.21.15.103:0/3345512197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c4dcb0 0x5571b1c4b610 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.707+0000 7ff90ec40740 1 --2- 172.21.15.103:0/3345512197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c43410 0x5571b1c4b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.707+0000 7ff90ec40740 1 --2- 172.21.15.103:0/3345512197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c42af0 0x5571b1c4ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.707+0000 7ff8ed7fa640 0 client.16410 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.707+0000 7ff90ec40740 1 -- 172.21.15.103:0/3345512197 >> 172.21.15.103:0/3345512197 conn(0x5571b1c3f060 msgr2=0x5571b1c3e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.707+0000 7ff90ec40740 1 -- 172.21.15.103:0/3345512197 shutdown_connections 2024-09-19T06:51:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.708+0000 7ff90ec40740 1 -- 172.21.15.103:0/3345512197 wait complete. 2024-09-19T06:51:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.708+0000 7ff90ec40740 20 client.16410 trim_cache size 0 max 16384 2024-09-19T06:51:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.722+0000 7ff90ec40740 1 Processor -- start 2024-09-19T06:51:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.722+0000 7ff90ec40740 1 -- start start 2024-09-19T06:51:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.722+0000 7ff90ec40740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c42af0 0x5571b1c45560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.722+0000 7ff90ec40740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c43410 0x5571b1c4b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.722+0000 7ff90ec40740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c4dcb0 0x5571b1c4b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.722+0000 7ff90ec40740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9041ad880 con 0x5571b1c4dcb0 2024-09-19T06:51:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.722+0000 7ff90ec40740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9041af3c0 con 0x5571b1c42af0 2024-09-19T06:51:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.722+0000 7ff90ec40740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9041adf50 con 0x5571b1c43410 2024-09-19T06:51:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.722+0000 7ff90a44c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c4dcb0 0x5571b1c4b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.722+0000 7ff90a44c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c4dcb0 0x5571b1c4b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54232/0 (socket says 172.21.15.103:54232) 2024-09-19T06:51:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.722+0000 7ff90a44c640 1 -- 172.21.15.103:0/402693218 learned_addr learned my addr 172.21.15.103:0/402693218 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.722+0000 7ff909c4b640 1 --2- 172.21.15.103:0/402693218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c42af0 0x5571b1c45560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.722+0000 7ff90944a640 1 --2- 172.21.15.103:0/402693218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c43410 0x5571b1c4b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff90a44c640 1 -- 172.21.15.103:0/402693218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c43410 msgr2=0x5571b1c4b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff90a44c640 1 --2- 172.21.15.103:0/402693218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c43410 0x5571b1c4b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff90a44c640 1 -- 172.21.15.103:0/402693218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c42af0 msgr2=0x5571b1c45560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff90a44c640 1 --2- 172.21.15.103:0/402693218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c42af0 0x5571b1c45560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff90a44c640 1 -- 172.21.15.103:0/402693218 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571b1c46740 con 0x5571b1c4dcb0 2024-09-19T06:51:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff90944a640 1 --2- 172.21.15.103:0/402693218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c43410 0x5571b1c4b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff90a44c640 1 --2- 172.21.15.103:0/402693218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c4dcb0 0x5571b1c4b600 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc010010 tx=0x7ff8fc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcf97232459ed7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff8ef7fe640 1 -- 172.21.15.103:0/402693218 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc044070 con 0x5571b1c4dcb0 2024-09-19T06:51:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff8ef7fe640 1 -- 172.21.15.103:0/402693218 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8fc02a070 con 0x5571b1c4dcb0 2024-09-19T06:51:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff8ef7fe640 1 -- 172.21.15.103:0/402693218 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc01c400 con 0x5571b1c4dcb0 2024-09-19T06:51:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff90ec40740 1 -- 172.21.15.103:0/402693218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c4dcb0 msgr2=0x5571b1c4b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff90ec40740 1 --2- 172.21.15.103:0/402693218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c4dcb0 0x5571b1c4b600 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc010010 tx=0x7ff8fc009920 comp rx=0 tx=0).stop 2024-09-19T06:51:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff90ec40740 1 -- 172.21.15.103:0/402693218 shutdown_connections 2024-09-19T06:51:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff90ec40740 1 --2- 172.21.15.103:0/402693218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c4dcb0 0x5571b1c4b600 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff90ec40740 1 --2- 172.21.15.103:0/402693218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c43410 0x5571b1c4b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff90ec40740 1 --2- 172.21.15.103:0/402693218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c42af0 0x5571b1c45560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.723+0000 7ff90ec40740 1 -- 172.21.15.103:0/402693218 >> 172.21.15.103:0/402693218 conn(0x5571b1c3f060 msgr2=0x5571b1c4cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.724+0000 7ff90ec40740 1 -- 172.21.15.103:0/402693218 shutdown_connections 2024-09-19T06:51:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.724+0000 7ff90ec40740 1 -- 172.21.15.103:0/402693218 wait complete. 2024-09-19T06:51:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.724+0000 7ff90ec40740 1 -- start start 2024-09-19T06:51:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.724+0000 7ff90ec40740 1 Processor -- start 2024-09-19T06:51:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.724+0000 7ff90ec40740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c42af0 0x5571b1c560e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.724+0000 7ff90ec40740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c43410 0x5571b1c51400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.724+0000 7ff90ec40740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c4dcb0 0x5571b1c51940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.724+0000 7ff90ec40740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9041aecf0 con 0x5571b1c43410 2024-09-19T06:51:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.724+0000 7ff90ec40740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9041ae620 con 0x5571b1c42af0 2024-09-19T06:51:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.724+0000 7ff90ec40740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9041ab840 con 0x5571b1c4dcb0 2024-09-19T06:51:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.724+0000 7ff90ec40740 10 client.0 _ll_register_callbacks cb 0x7ff8fc01f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.724+0000 7ff90944a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c43410 0x5571b1c51400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.724+0000 7ff90944a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c43410 0x5571b1c51400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54238/0 (socket says 172.21.15.103:54238) 2024-09-19T06:51:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.724+0000 7ff90944a640 1 -- 172.21.15.103:0/3863102249 learned_addr learned my addr 172.21.15.103:0/3863102249 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff90a44c640 1 --2- 172.21.15.103:0/3863102249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c4dcb0 0x5571b1c51940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff909c4b640 1 --2- 172.21.15.103:0/3863102249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c42af0 0x5571b1c560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff90944a640 1 -- 172.21.15.103:0/3863102249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c4dcb0 msgr2=0x5571b1c51940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff90944a640 1 --2- 172.21.15.103:0/3863102249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c4dcb0 0x5571b1c51940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff90944a640 1 -- 172.21.15.103:0/3863102249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c42af0 msgr2=0x5571b1c560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff90944a640 1 --2- 172.21.15.103:0/3863102249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c42af0 0x5571b1c560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff90944a640 1 -- 172.21.15.103:0/3863102249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571b1c51e80 con 0x5571b1c43410 2024-09-19T06:51:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff909c4b640 1 --2- 172.21.15.103:0/3863102249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c42af0 0x5571b1c560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff90a44c640 1 --2- 172.21.15.103:0/3863102249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c4dcb0 0x5571b1c51940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff90944a640 1 --2- 172.21.15.103:0/3863102249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c43410 0x5571b1c51400 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009900 tx=0x7ff8f4031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5eb45595a83dea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff8ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff8ed7fa640 1 -- 172.21.15.103:0/3863102249 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4035040 con 0x5571b1c43410 2024-09-19T06:51:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff8ed7fa640 1 -- 172.21.15.103:0/3863102249 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff8f4004450 con 0x5571b1c43410 2024-09-19T06:51:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff90ec40740 1 -- 172.21.15.103:0/3863102249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8fc02a070 con 0x5571b1c43410 2024-09-19T06:51:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff8ed7fa640 1 -- 172.21.15.103:0/3863102249 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff8f4005030 con 0x5571b1c43410 2024-09-19T06:51:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff90ec40740 10 client.16416 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff90ec40740 1 -- 172.21.15.103:0/3863102249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5571b1cc8370 con 0x5571b1c43410 2024-09-19T06:51:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.725+0000 7ff8ed7fa640 1 -- 172.21.15.103:0/3863102249 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8fc02a070 con 0x5571b1c43410 2024-09-19T06:51:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.726+0000 7ff8ed7fa640 1 -- 172.21.15.103:0/3863102249 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff8f40035f0 con 0x5571b1c43410 2024-09-19T06:51:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.726+0000 7ff90ec40740 10 client.16416 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.726+0000 7ff90ec40740 10 client.16416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.726+0000 7ff90ec40740 20 client.16416 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.726+0000 7ff90ec40740 1 --2- 172.21.15.103:0/3863102249 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5571b1cc8b20 0x5571b1ccaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.726+0000 7ff90ec40740 4 client.16416 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:51:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.726+0000 7ff90ec40740 1 -- 172.21.15.103:0/3863102249 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5571b1baed70 con 0x5571b1cc8b20 2024-09-19T06:51:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.726+0000 7ff909c4b640 1 --2- 172.21.15.103:0/3863102249 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5571b1cc8b20 0x5571b1ccaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.726+0000 7ff8ed7fa640 10 client.16416 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.726+0000 7ff909c4b640 1 --2- 172.21.15.103:0/3863102249 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5571b1cc8b20 0x5571b1ccaf40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a97f08978020bd58 server_cookie=23d656e10a6ae819 in_seq=0 out_seq=0 2024-09-19T06:51:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.736+0000 7ff8ed7fa640 1 -- 172.21.15.103:0/3863102249 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5571b1baed70 con 0x5571b1cc8b20 2024-09-19T06:51:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.736+0000 7ff8ed7fa640 10 client.16416 handle_command_reply: tid=0 2024-09-19T06:51:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.736+0000 7ff90ec40740 1 client.16416 shutdown 2024-09-19T06:51:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.737+0000 7ff90ec40740 1 -- 172.21.15.103:0/3863102249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c43410 msgr2=0x5571b1c51400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.737+0000 7ff90ec40740 1 --2- 172.21.15.103:0/3863102249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c43410 0x5571b1c51400 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009900 tx=0x7ff8f4031920 comp rx=0 tx=0).stop 2024-09-19T06:51:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.737+0000 7ff90ec40740 1 -- 172.21.15.103:0/3863102249 shutdown_connections 2024-09-19T06:51:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.737+0000 7ff90ec40740 1 --2- 172.21.15.103:0/3863102249 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5571b1cc8b20 0x5571b1ccaf40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.737+0000 7ff90ec40740 1 --2- 172.21.15.103:0/3863102249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b1c4dcb0 0x5571b1c51940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.737+0000 7ff8ed7fa640 0 client.16416 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.737+0000 7ff90ec40740 1 --2- 172.21.15.103:0/3863102249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b1c43410 0x5571b1c51400 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.737+0000 7ff90ec40740 1 --2- 172.21.15.103:0/3863102249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571b1c42af0 0x5571b1c560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.737+0000 7ff90ec40740 1 -- 172.21.15.103:0/3863102249 >> 172.21.15.103:0/3863102249 conn(0x5571b1c3f060 msgr2=0x5571b1c3d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.737+0000 7ff90ec40740 1 -- 172.21.15.103:0/3863102249 shutdown_connections 2024-09-19T06:51:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.737+0000 7ff90ec40740 1 -- 172.21.15.103:0/3863102249 wait complete. 2024-09-19T06:51:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.737+0000 7ff90ec40740 20 client.16416 trim_cache size 0 max 16384 2024-09-19T06:51:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:51:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:51:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:51:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:51:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.737+0000 7ff90beaf640 1 -- 172.21.15.103:0/4110534891 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff8f00a3460 msgr2=0x7ff8f00a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.737+0000 7ff90beaf640 1 --2- 172.21.15.103:0/4110534891 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff8f00a3460 0x7ff8f00a5920 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc020000 tx=0x7ff8fc01d000 comp rx=0 tx=0).stop 2024-09-19T06:51:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.737+0000 7ff90beaf640 1 -- 172.21.15.103:0/4110534891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff904102ca0 msgr2=0x7ff904198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.737+0000 7ff90beaf640 1 --2- 172.21.15.103:0/4110534891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff904102ca0 0x7ff904198310 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff90000d950 tx=0x7ff90000de10 comp rx=0 tx=0).stop 2024-09-19T06:51:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.738+0000 7ff90beaf640 1 -- 172.21.15.103:0/4110534891 shutdown_connections 2024-09-19T06:51:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.738+0000 7ff90beaf640 1 --2- 172.21.15.103:0/4110534891 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff8f00a3460 0x7ff8f00a5920 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.738+0000 7ff90beaf640 1 --2- 172.21.15.103:0/4110534891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff90410c0c0 0x7ff90419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.738+0000 7ff90beaf640 1 --2- 172.21.15.103:0/4110534891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9041035c0 0x7ff904198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.738+0000 7ff90beaf640 1 --2- 172.21.15.103:0/4110534891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff904102ca0 0x7ff904198310 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.738+0000 7ff90beaf640 1 -- 172.21.15.103:0/4110534891 >> 172.21.15.103:0/4110534891 conn(0x7ff9040fc6f0 msgr2=0x7ff9040fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.738+0000 7ff90beaf640 1 -- 172.21.15.103:0/4110534891 shutdown_connections 2024-09-19T06:51:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:36.738+0000 7ff90beaf640 1 -- 172.21.15.103:0/4110534891 wait complete. 2024-09-19T06:51:38.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:38 smithi174 ceph-mon[35637]: pgmap v75: 97 pgs: 97 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-19T06:51:38.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:38 smithi103 ceph-mon[31158]: pgmap v75: 97 pgs: 97 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-19T06:51:38.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:38 smithi134 ceph-mon[35356]: pgmap v75: 97 pgs: 97 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-19T06:51:40.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:40 smithi174 ceph-mon[35637]: pgmap v76: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-09-19T06:51:40.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:40 smithi103 ceph-mon[31158]: pgmap v76: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-09-19T06:51:40.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:40 smithi134 ceph-mon[35356]: pgmap v76: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-09-19T06:51:42.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:42 smithi174 ceph-mon[35637]: pgmap v77: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-19T06:51:42.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:42 smithi103 ceph-mon[31158]: pgmap v77: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-19T06:51:42.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:42 smithi134 ceph-mon[35356]: pgmap v77: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-19T06:51:44.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:44 smithi174 ceph-mon[35637]: pgmap v78: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-19T06:51:44.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:44 smithi103 ceph-mon[31158]: pgmap v78: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-19T06:51:44.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:44 smithi134 ceph-mon[35356]: pgmap v78: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-19T06:51:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:46 smithi174 ceph-mon[35637]: pgmap v79: 97 pgs: 97 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-09-19T06:51:46.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:46 smithi103 ceph-mon[31158]: pgmap v79: 97 pgs: 97 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-09-19T06:51:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:46 smithi134 ceph-mon[35356]: pgmap v79: 97 pgs: 97 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-09-19T06:51:48.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:48 smithi174 ceph-mon[35637]: pgmap v80: 97 pgs: 97 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-19T06:51:48.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:48 smithi103 ceph-mon[31158]: pgmap v80: 97 pgs: 97 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-19T06:51:48.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:48 smithi134 ceph-mon[35356]: pgmap v80: 97 pgs: 97 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-19T06:51:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.835+0000 7f6a5b398640 1 -- 172.21.15.103:0/3521723027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a54109090 msgr2=0x7f6a54110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.835+0000 7f6a5b398640 1 --2- 172.21.15.103:0/3521723027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a54109090 0x7f6a54110be0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009ca0 tx=0x7f6a4c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.835+0000 7f6a5b398640 1 -- 172.21.15.103:0/3521723027 shutdown_connections 2024-09-19T06:51:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.835+0000 7f6a5b398640 1 --2- 172.21.15.103:0/3521723027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a54109090 0x7f6a54110be0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.835+0000 7f6a5b398640 1 --2- 172.21.15.103:0/3521723027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a541048f0 0x7f6a54108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.835+0000 7f6a5b398640 1 --2- 172.21.15.103:0/3521723027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a54103f40 0x7f6a54104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.835+0000 7f6a5b398640 1 -- 172.21.15.103:0/3521723027 >> 172.21.15.103:0/3521723027 conn(0x7f6a540ff9b0 msgr2=0x7f6a54101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.835+0000 7f6a5b398640 1 -- 172.21.15.103:0/3521723027 shutdown_connections 2024-09-19T06:51:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.835+0000 7f6a5b398640 1 -- 172.21.15.103:0/3521723027 wait complete. 2024-09-19T06:51:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.836+0000 7f6a5b398640 1 Processor -- start 2024-09-19T06:51:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.836+0000 7f6a5b398640 1 -- start start 2024-09-19T06:51:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.836+0000 7f6a5b398640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a54103f40 0x7f6a5419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.836+0000 7f6a5b398640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a54109090 0x7f6a5419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.836+0000 7f6a5b398640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a5419ede0 0x7f6a5419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.836+0000 7f6a5b398640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541140d0 con 0x7f6a54109090 2024-09-19T06:51:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.836+0000 7f6a5b398640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a54113f50 con 0x7f6a5419ede0 2024-09-19T06:51:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.836+0000 7f6a5b398640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6a54114250 con 0x7f6a54103f40 2024-09-19T06:51:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.836+0000 7f6a58933640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a54109090 0x7f6a5419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.836+0000 7f6a58933640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a54109090 0x7f6a5419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36798/0 (socket says 172.21.15.103:36798) 2024-09-19T06:51:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.836+0000 7f6a58933640 1 -- 172.21.15.103:0/352021693 learned_addr learned my addr 172.21.15.103:0/352021693 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.837+0000 7f6a59935640 1 --2- 172.21.15.103:0/352021693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a5419ede0 0x7f6a5419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.837+0000 7f6a59134640 1 --2- 172.21.15.103:0/352021693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a54103f40 0x7f6a5419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.837+0000 7f6a58933640 1 -- 172.21.15.103:0/352021693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a54103f40 msgr2=0x7f6a5419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.837+0000 7f6a58933640 1 --2- 172.21.15.103:0/352021693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a54103f40 0x7f6a5419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.837+0000 7f6a58933640 1 -- 172.21.15.103:0/352021693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a5419ede0 msgr2=0x7f6a5419f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.837+0000 7f6a58933640 1 --2- 172.21.15.103:0/352021693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a5419ede0 0x7f6a5419f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.837+0000 7f6a58933640 1 -- 172.21.15.103:0/352021693 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5419f8d0 con 0x7f6a54109090 2024-09-19T06:51:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.837+0000 7f6a59134640 1 --2- 172.21.15.103:0/352021693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a54103f40 0x7f6a5419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.837+0000 7f6a58933640 1 --2- 172.21.15.103:0/352021693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a54109090 0x7f6a5419aa60 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400e9c0 tx=0x7f6a4400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.837+0000 7f6a4a7fc640 1 -- 172.21.15.103:0/352021693 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4400cd80 con 0x7f6a54109090 2024-09-19T06:51:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.837+0000 7f6a4a7fc640 1 -- 172.21.15.103:0/352021693 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a44005500 con 0x7f6a54109090 2024-09-19T06:51:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.837+0000 7f6a5b398640 1 -- 172.21.15.103:0/352021693 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5419fbc0 con 0x7f6a54109090 2024-09-19T06:51:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.837+0000 7f6a4a7fc640 1 -- 172.21.15.103:0/352021693 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44010690 con 0x7f6a54109090 2024-09-19T06:51:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.837+0000 7f6a5b398640 1 -- 172.21.15.103:0/352021693 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a541a73b0 con 0x7f6a54109090 2024-09-19T06:51:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.840+0000 7f6a4a7fc640 1 -- 172.21.15.103:0/352021693 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f6a440046c0 con 0x7f6a54109090 2024-09-19T06:51:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.840+0000 7f6a5e210740 1 Processor -- start 2024-09-19T06:51:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.840+0000 7f6a5e210740 1 -- start start 2024-09-19T06:51:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.840+0000 7f6a5e210740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef59af0 0x55adaef59ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.840+0000 7f6a5e210740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef5a410 0x55adaef64770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.840+0000 7f6a5e210740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef670a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.840+0000 7f6a5e210740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55adaeebe2f0 con 0x55adaef64cb0 2024-09-19T06:51:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.840+0000 7f6a5e210740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55adaee66270 con 0x55adaef5a410 2024-09-19T06:51:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.840+0000 7f6a5e210740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55adaeeba2b0 con 0x55adaef59af0 2024-09-19T06:51:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.840+0000 7f6a58933640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a58933640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef670a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36814/0 (socket says 172.21.15.103:36814) 2024-09-19T06:51:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a58933640 1 -- 172.21.15.103:0/2568335042 learned_addr learned my addr 172.21.15.103:0/2568335042 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a59935640 1 --2- 172.21.15.103:0/2568335042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef59af0 0x55adaef59ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a59134640 1 --2- 172.21.15.103:0/2568335042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef5a410 0x55adaef64770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a4a7fc640 1 --2- 172.21.15.103:0/352021693 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6a400a3460 0x7f6a400a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a59935640 1 --2- 172.21.15.103:0/352021693 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6a400a3460 0x7f6a400a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a58933640 1 -- 172.21.15.103:0/2568335042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef59af0 msgr2=0x55adaef59ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a58933640 1 --2- 172.21.15.103:0/2568335042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef59af0 0x55adaef59ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a58933640 1 -- 172.21.15.103:0/2568335042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef5a410 msgr2=0x55adaef64770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a58933640 1 --2- 172.21.15.103:0/2568335042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef5a410 0x55adaef64770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a58933640 1 -- 172.21.15.103:0/2568335042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adaee65340 con 0x55adaef64cb0 2024-09-19T06:51:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a4a7fc640 1 -- 172.21.15.103:0/352021693 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6a440d00f0 con 0x7f6a54109090 2024-09-19T06:51:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a58933640 1 --2- 172.21.15.103:0/2568335042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef670a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6a440cd790 tx=0x7f6a44021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f156c03f2b67a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a59935640 1 --2- 172.21.15.103:0/352021693 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6a400a3460 0x7f6a400a5920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009430 tx=0x7f6a4c02a8c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a3f7fe640 1 -- 172.21.15.103:0/2568335042 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4409c1b0 con 0x55adaef64cb0 2024-09-19T06:51:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a3f7fe640 1 -- 172.21.15.103:0/2568335042 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a4409cae0 con 0x55adaef64cb0 2024-09-19T06:51:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.841+0000 7f6a3f7fe640 1 -- 172.21.15.103:0/2568335042 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a440d3cd0 con 0x55adaef64cb0 2024-09-19T06:51:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.842+0000 7f6a5e210740 1 -- 172.21.15.103:0/2568335042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 msgr2=0x55adaef670a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.842+0000 7f6a5e210740 1 --2- 172.21.15.103:0/2568335042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef670a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6a440cd790 tx=0x7f6a44021000 comp rx=0 tx=0).stop 2024-09-19T06:51:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.842+0000 7f6a5e210740 1 -- 172.21.15.103:0/2568335042 shutdown_connections 2024-09-19T06:51:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.842+0000 7f6a5e210740 1 --2- 172.21.15.103:0/2568335042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef670a0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.842+0000 7f6a5e210740 1 --2- 172.21.15.103:0/2568335042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef5a410 0x55adaef64770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.842+0000 7f6a5e210740 1 --2- 172.21.15.103:0/2568335042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef59af0 0x55adaef59ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.842+0000 7f6a5e210740 1 -- 172.21.15.103:0/2568335042 >> 172.21.15.103:0/2568335042 conn(0x55adaef56060 msgr2=0x55adaef56440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.842+0000 7f6a5e210740 1 -- 172.21.15.103:0/2568335042 shutdown_connections 2024-09-19T06:51:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.842+0000 7f6a5e210740 1 -- 172.21.15.103:0/2568335042 wait complete. 2024-09-19T06:51:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.843+0000 7f6a5e210740 1 -- start start 2024-09-19T06:51:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.843+0000 7f6a5e210740 1 Processor -- start 2024-09-19T06:51:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.843+0000 7f6a5e210740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef59af0 0x55adaef61bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.843+0000 7f6a5e210740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef5a410 0x55adaef620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.843+0000 7f6a5e210740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef64cb0 0x55adaef62630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.843+0000 7f6a5e210740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55adaef67880 con 0x55adaef5a410 2024-09-19T06:51:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.843+0000 7f6a5e210740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55adaee47290 con 0x55adaef64cb0 2024-09-19T06:51:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.843+0000 7f6a5e210740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55adaef67a00 con 0x55adaef59af0 2024-09-19T06:51:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.843+0000 7f6a5e210740 10 client.0 _ll_register_callbacks cb 0x55adaee78140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.843+0000 7f6a58933640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef5a410 0x55adaef620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.843+0000 7f6a58933640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef5a410 0x55adaef620f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36822/0 (socket says 172.21.15.103:36822) 2024-09-19T06:51:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.843+0000 7f6a58933640 1 -- 172.21.15.103:0/3539886769 learned_addr learned my addr 172.21.15.103:0/3539886769 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.843+0000 7f6a59134640 1 --2- 172.21.15.103:0/3539886769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef59af0 0x55adaef61bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.843+0000 7f6a59935640 1 --2- 172.21.15.103:0/3539886769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef64cb0 0x55adaef62630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a58933640 1 -- 172.21.15.103:0/3539886769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef59af0 msgr2=0x55adaef61bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a58933640 1 --2- 172.21.15.103:0/3539886769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef59af0 0x55adaef61bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a58933640 1 -- 172.21.15.103:0/3539886769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef64cb0 msgr2=0x55adaef62630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a58933640 1 --2- 172.21.15.103:0/3539886769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef64cb0 0x55adaef62630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a58933640 1 -- 172.21.15.103:0/3539886769 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adaebd1210 con 0x55adaef5a410 2024-09-19T06:51:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a59134640 1 --2- 172.21.15.103:0/3539886769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef59af0 0x55adaef61bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a59935640 1 --2- 172.21.15.103:0/3539886769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef64cb0 0x55adaef62630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a58933640 1 --2- 172.21.15.103:0/3539886769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef5a410 0x55adaef620f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6a44098840 tx=0x7f6a44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efce942a078c1a42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a3cff9640 1 -- 172.21.15.103:0/3539886769 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4409bd40 con 0x55adaef5a410 2024-09-19T06:51:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a3cff9640 1 -- 172.21.15.103:0/3539886769 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6a440b05c0 con 0x55adaef5a410 2024-09-19T06:51:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a5e210740 1 -- 172.21.15.103:0/3539886769 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a4409cae0 con 0x55adaef5a410 2024-09-19T06:51:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a3cff9640 1 -- 172.21.15.103:0/3539886769 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a4409bf20 con 0x55adaef5a410 2024-09-19T06:51:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a5e210740 10 client.16428 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a5e210740 1 -- 172.21.15.103:0/3539886769 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55adaec9d620 con 0x55adaef5a410 2024-09-19T06:51:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a3cff9640 1 -- 172.21.15.103:0/3539886769 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a4409cae0 con 0x55adaef5a410 2024-09-19T06:51:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.844+0000 7f6a3cff9640 1 -- 172.21.15.103:0/3539886769 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6a44098a50 con 0x55adaef5a410 2024-09-19T06:51:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.845+0000 7f6a5e210740 10 client.16428 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.845+0000 7f6a5e210740 10 client.16428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.845+0000 7f6a5e210740 20 client.16428 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.845+0000 7f6a5e210740 1 --2- 172.21.15.103:0/3539886769 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55adaef74ab0 0x55adaef76ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.845+0000 7f6a5e210740 4 client.16428 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.845+0000 7f6a5e210740 1 -- 172.21.15.103:0/3539886769 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adaeecba20 con 0x55adaef74ab0 2024-09-19T06:51:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.845+0000 7f6a59134640 1 --2- 172.21.15.103:0/3539886769 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55adaef74ab0 0x55adaef76ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.845+0000 7f6a3cff9640 10 client.16428 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.845+0000 7f6a59134640 1 --2- 172.21.15.103:0/3539886769 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55adaef74ab0 0x55adaef76ea0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe35231cac57d446 server_cookie=59b1ab8dd826a3a6 in_seq=0 out_seq=0 2024-09-19T06:51:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.846+0000 7f6a3cff9640 1 -- 172.21.15.103:0/3539886769 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55adaeecba20 con 0x55adaef74ab0 2024-09-19T06:51:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.846+0000 7f6a3cff9640 10 client.16428 handle_command_reply: tid=0 2024-09-19T06:51:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.846+0000 7f6a5e210740 1 client.16428 shutdown 2024-09-19T06:51:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.846+0000 7f6a5e210740 1 -- 172.21.15.103:0/3539886769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef5a410 msgr2=0x55adaef620f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.846+0000 7f6a5e210740 1 --2- 172.21.15.103:0/3539886769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef5a410 0x55adaef620f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6a44098840 tx=0x7f6a44005880 comp rx=0 tx=0).stop 2024-09-19T06:51:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.846+0000 7f6a5e210740 1 -- 172.21.15.103:0/3539886769 shutdown_connections 2024-09-19T06:51:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.846+0000 7f6a5e210740 1 --2- 172.21.15.103:0/3539886769 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55adaef74ab0 0x55adaef76ea0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.846+0000 7f6a5e210740 1 --2- 172.21.15.103:0/3539886769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef64cb0 0x55adaef62630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.846+0000 7f6a5e210740 1 --2- 172.21.15.103:0/3539886769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef5a410 0x55adaef620f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.846+0000 7f6a5e210740 1 --2- 172.21.15.103:0/3539886769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef59af0 0x55adaef61bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.846+0000 7f6a5e210740 1 -- 172.21.15.103:0/3539886769 >> 172.21.15.103:0/3539886769 conn(0x55adaef56060 msgr2=0x55adaef5bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.846+0000 7f6a3cff9640 0 client.16428 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.846+0000 7f6a5e210740 1 -- 172.21.15.103:0/3539886769 shutdown_connections 2024-09-19T06:51:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.846+0000 7f6a5e210740 1 -- 172.21.15.103:0/3539886769 wait complete. 2024-09-19T06:51:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.846+0000 7f6a5e210740 20 client.16428 trim_cache size 0 max 16384 2024-09-19T06:51:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.861+0000 7f6a5e210740 1 Processor -- start 2024-09-19T06:51:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.861+0000 7f6a5e210740 1 -- start start 2024-09-19T06:51:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.861+0000 7f6a5e210740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef59af0 0x55adaef5c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.861+0000 7f6a5e210740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef5a410 0x55adaef62460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.861+0000 7f6a5e210740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef63b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.861+0000 7f6a5e210740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541afa00 con 0x55adaef64cb0 2024-09-19T06:51:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.861+0000 7f6a5e210740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541b1540 con 0x55adaef5a410 2024-09-19T06:51:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.861+0000 7f6a5e210740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541b00d0 con 0x55adaef59af0 2024-09-19T06:51:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.861+0000 7f6a59935640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef63b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.861+0000 7f6a59935640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef63b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36838/0 (socket says 172.21.15.103:36838) 2024-09-19T06:51:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.861+0000 7f6a59935640 1 -- 172.21.15.103:0/2914917408 learned_addr learned my addr 172.21.15.103:0/2914917408 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.861+0000 7f6a59935640 1 -- 172.21.15.103:0/2914917408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef59af0 msgr2=0x55adaef5c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.861+0000 7f6a59935640 1 --2- 172.21.15.103:0/2914917408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef59af0 0x55adaef5c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.861+0000 7f6a59935640 1 -- 172.21.15.103:0/2914917408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef5a410 msgr2=0x55adaef62460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.861+0000 7f6a59935640 1 --2- 172.21.15.103:0/2914917408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef5a410 0x55adaef62460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.861+0000 7f6a59935640 1 -- 172.21.15.103:0/2914917408 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adaef571f0 con 0x55adaef64cb0 2024-09-19T06:51:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.862+0000 7f6a59935640 1 --2- 172.21.15.103:0/2914917408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef63b20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009ca0 tx=0x7f6a4c040070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6167dab64d2822a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.862+0000 7f6a3effd640 1 -- 172.21.15.103:0/2914917408 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c031920 con 0x55adaef64cb0 2024-09-19T06:51:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.862+0000 7f6a3effd640 1 -- 172.21.15.103:0/2914917408 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a4c036040 con 0x55adaef64cb0 2024-09-19T06:51:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.862+0000 7f6a3effd640 1 -- 172.21.15.103:0/2914917408 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c02fcb0 con 0x55adaef64cb0 2024-09-19T06:51:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.862+0000 7f6a5e210740 1 -- 172.21.15.103:0/2914917408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 msgr2=0x55adaef63b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.862+0000 7f6a5e210740 1 --2- 172.21.15.103:0/2914917408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef63b20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009ca0 tx=0x7f6a4c040070 comp rx=0 tx=0).stop 2024-09-19T06:51:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.862+0000 7f6a5e210740 1 -- 172.21.15.103:0/2914917408 shutdown_connections 2024-09-19T06:51:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.862+0000 7f6a5e210740 1 --2- 172.21.15.103:0/2914917408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef63b20 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.862+0000 7f6a5e210740 1 --2- 172.21.15.103:0/2914917408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef5a410 0x55adaef62460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.862+0000 7f6a5e210740 1 --2- 172.21.15.103:0/2914917408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef59af0 0x55adaef5c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.862+0000 7f6a5e210740 1 -- 172.21.15.103:0/2914917408 >> 172.21.15.103:0/2914917408 conn(0x55adaef56060 msgr2=0x55adaef63410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.862+0000 7f6a5e210740 1 -- 172.21.15.103:0/2914917408 shutdown_connections 2024-09-19T06:51:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.863+0000 7f6a5e210740 1 -- 172.21.15.103:0/2914917408 wait complete. 2024-09-19T06:51:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.863+0000 7f6a5e210740 1 -- start start 2024-09-19T06:51:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.863+0000 7f6a5e210740 1 Processor -- start 2024-09-19T06:51:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.863+0000 7f6a5e210740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef59af0 0x55adaef79f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.863+0000 7f6a5e210740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef5a410 0x55adaef77240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.863+0000 7f6a5e210740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef77780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.863+0000 7f6a5e210740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541b0e70 con 0x55adaef64cb0 2024-09-19T06:51:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.863+0000 7f6a5e210740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541b07a0 con 0x55adaef59af0 2024-09-19T06:51:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.863+0000 7f6a5e210740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541ad9c0 con 0x55adaef5a410 2024-09-19T06:51:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.863+0000 7f6a5e210740 10 client.0 _ll_register_callbacks cb 0x7f6a44096f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.863+0000 7f6a59935640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef77780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.863+0000 7f6a59935640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef77780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36846/0 (socket says 172.21.15.103:36846) 2024-09-19T06:51:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.863+0000 7f6a59935640 1 -- 172.21.15.103:0/4245435940 learned_addr learned my addr 172.21.15.103:0/4245435940 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:51:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a58933640 1 --2- 172.21.15.103:0/4245435940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef5a410 0x55adaef77240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a59134640 1 --2- 172.21.15.103:0/4245435940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef59af0 0x55adaef79f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a59935640 1 -- 172.21.15.103:0/4245435940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef5a410 msgr2=0x55adaef77240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a59935640 1 --2- 172.21.15.103:0/4245435940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef5a410 0x55adaef77240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a59935640 1 -- 172.21.15.103:0/4245435940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef59af0 msgr2=0x55adaef79f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a59935640 1 --2- 172.21.15.103:0/4245435940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef59af0 0x55adaef79f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a59935640 1 -- 172.21.15.103:0/4245435940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adaef77cc0 con 0x55adaef64cb0 2024-09-19T06:51:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a59134640 1 --2- 172.21.15.103:0/4245435940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef59af0 0x55adaef79f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a59935640 1 --2- 172.21.15.103:0/4245435940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef77780 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0049d0 tx=0x7f6a4c009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79de1571df615ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:51:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a3cff9640 1 -- 172.21.15.103:0/4245435940 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c004b40 con 0x55adaef64cb0 2024-09-19T06:51:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a3cff9640 1 -- 172.21.15.103:0/4245435940 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6a4c004ce0 con 0x55adaef64cb0 2024-09-19T06:51:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a5e210740 1 -- 172.21.15.103:0/4245435940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a4c036040 con 0x55adaef64cb0 2024-09-19T06:51:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a3cff9640 1 -- 172.21.15.103:0/4245435940 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a4c004000 con 0x55adaef64cb0 2024-09-19T06:51:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a5e210740 10 client.16434 fetch_fsmap learned FSMap version 27 2024-09-19T06:51:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a5e210740 1 -- 172.21.15.103:0/4245435940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55adaefdf0d0 con 0x55adaef64cb0 2024-09-19T06:51:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.864+0000 7f6a3cff9640 1 -- 172.21.15.103:0/4245435940 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a4c036040 con 0x55adaef64cb0 2024-09-19T06:51:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.865+0000 7f6a3cff9640 1 -- 172.21.15.103:0/4245435940 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6a4c047040 con 0x55adaef64cb0 2024-09-19T06:51:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.865+0000 7f6a5e210740 10 client.16434 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:51:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.865+0000 7f6a5e210740 10 client.16434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.865+0000 7f6a5e210740 20 client.16434 populate_metadata read hostname 'smithi103' 2024-09-19T06:51:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.865+0000 7f6a5e210740 1 --2- 172.21.15.103:0/4245435940 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55adaefdf780 0x55adaefe1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.865+0000 7f6a5e210740 4 client.16434 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:51:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.865+0000 7f6a5e210740 1 -- 172.21.15.103:0/4245435940 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55adaeec5d70 con 0x55adaefdf780 2024-09-19T06:51:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.865+0000 7f6a59134640 1 --2- 172.21.15.103:0/4245435940 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55adaefdf780 0x55adaefe1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.865+0000 7f6a59134640 1 --2- 172.21.15.103:0/4245435940 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55adaefdf780 0x55adaefe1ba0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c6417d6ad22aa65 server_cookie=ce9522d807f3fed4 in_seq=0 out_seq=0 2024-09-19T06:51:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.865+0000 7f6a3cff9640 10 client.16434 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.877+0000 7f6a3cff9640 1 -- 172.21.15.103:0/4245435940 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55adaeec5d70 con 0x55adaefdf780 2024-09-19T06:51:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.877+0000 7f6a3cff9640 10 client.16434 handle_command_reply: tid=0 2024-09-19T06:51:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.877+0000 7f6a5e210740 1 client.16434 shutdown 2024-09-19T06:51:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.877+0000 7f6a5e210740 1 -- 172.21.15.103:0/4245435940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 msgr2=0x55adaef77780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.877+0000 7f6a5e210740 1 --2- 172.21.15.103:0/4245435940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef77780 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0049d0 tx=0x7f6a4c009380 comp rx=0 tx=0).stop 2024-09-19T06:51:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.877+0000 7f6a5e210740 1 -- 172.21.15.103:0/4245435940 shutdown_connections 2024-09-19T06:51:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.877+0000 7f6a5e210740 1 --2- 172.21.15.103:0/4245435940 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55adaefdf780 0x55adaefe1ba0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.877+0000 7f6a5e210740 1 --2- 172.21.15.103:0/4245435940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55adaef64cb0 0x55adaef77780 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.877+0000 7f6a5e210740 1 --2- 172.21.15.103:0/4245435940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55adaef5a410 0x55adaef77240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.877+0000 7f6a3cff9640 0 client.16434 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:51:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.877+0000 7f6a5e210740 1 --2- 172.21.15.103:0/4245435940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adaef59af0 0x55adaef79f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.877+0000 7f6a5e210740 1 -- 172.21.15.103:0/4245435940 >> 172.21.15.103:0/4245435940 conn(0x55adaef56060 msgr2=0x55adaef704b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.878+0000 7f6a5e210740 1 -- 172.21.15.103:0/4245435940 shutdown_connections 2024-09-19T06:51:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.878+0000 7f6a5e210740 1 -- 172.21.15.103:0/4245435940 wait complete. 2024-09-19T06:51:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.878+0000 7f6a5e210740 20 client.16434 trim_cache size 0 max 16384 2024-09-19T06:51:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:51:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:51:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:51:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:51:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.878+0000 7f6a5b398640 1 -- 172.21.15.103:0/352021693 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6a400a3460 msgr2=0x7f6a400a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.878+0000 7f6a5b398640 1 --2- 172.21.15.103:0/352021693 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6a400a3460 0x7f6a400a5920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009430 tx=0x7f6a4c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.878+0000 7f6a5b398640 1 -- 172.21.15.103:0/352021693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a54109090 msgr2=0x7f6a5419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.878+0000 7f6a5b398640 1 --2- 172.21.15.103:0/352021693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a54109090 0x7f6a5419aa60 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400e9c0 tx=0x7f6a4400ee80 comp rx=0 tx=0).stop 2024-09-19T06:51:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.878+0000 7f6a5b398640 1 -- 172.21.15.103:0/352021693 shutdown_connections 2024-09-19T06:51:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.878+0000 7f6a5b398640 1 --2- 172.21.15.103:0/352021693 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6a400a3460 0x7f6a400a5920 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.878+0000 7f6a5b398640 1 --2- 172.21.15.103:0/352021693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a5419ede0 0x7f6a5419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.878+0000 7f6a5b398640 1 --2- 172.21.15.103:0/352021693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a54109090 0x7f6a5419aa60 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.878+0000 7f6a5b398640 1 --2- 172.21.15.103:0/352021693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a54103f40 0x7f6a5419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.878+0000 7f6a5b398640 1 -- 172.21.15.103:0/352021693 >> 172.21.15.103:0/352021693 conn(0x7f6a540ff9b0 msgr2=0x7f6a54100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.878+0000 7f6a5b398640 1 -- 172.21.15.103:0/352021693 shutdown_connections 2024-09-19T06:51:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:51:49.878+0000 7f6a5b398640 1 -- 172.21.15.103:0/352021693 wait complete. 2024-09-19T06:51:50.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:50 smithi174 ceph-mon[35637]: pgmap v81: 97 pgs: 97 active+clean; 2.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-09-19T06:51:50.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:50 smithi103 ceph-mon[31158]: pgmap v81: 97 pgs: 97 active+clean; 2.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-09-19T06:51:50.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:50 smithi134 ceph-mon[35356]: pgmap v81: 97 pgs: 97 active+clean; 2.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-09-19T06:51:52.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:52 smithi174 ceph-mon[35637]: pgmap v82: 97 pgs: 97 active+clean; 2.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-09-19T06:51:52.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:52 smithi103 ceph-mon[31158]: pgmap v82: 97 pgs: 97 active+clean; 2.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-09-19T06:51:52.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:52 smithi134 ceph-mon[35356]: pgmap v82: 97 pgs: 97 active+clean; 2.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-09-19T06:51:54.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:54 smithi174 ceph-mon[35637]: pgmap v83: 97 pgs: 97 active+clean; 2.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-09-19T06:51:54.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:54 smithi103 ceph-mon[31158]: pgmap v83: 97 pgs: 97 active+clean; 2.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-09-19T06:51:54.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:54 smithi134 ceph-mon[35356]: pgmap v83: 97 pgs: 97 active+clean; 2.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-09-19T06:51:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:56 smithi174 ceph-mon[35637]: pgmap v84: 97 pgs: 97 active+clean; 2.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-09-19T06:51:56.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:56 smithi103 ceph-mon[31158]: pgmap v84: 97 pgs: 97 active+clean; 2.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-09-19T06:51:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:56 smithi134 ceph-mon[35356]: pgmap v84: 97 pgs: 97 active+clean; 2.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-09-19T06:51:58.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:51:58 smithi174 ceph-mon[35637]: pgmap v85: 97 pgs: 97 active+clean; 2.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-19T06:51:58.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:51:58 smithi103 ceph-mon[31158]: pgmap v85: 97 pgs: 97 active+clean; 2.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-19T06:51:58.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:51:58 smithi134 ceph-mon[35356]: pgmap v85: 97 pgs: 97 active+clean; 2.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-19T06:52:00.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:00 smithi174 ceph-mon[35637]: pgmap v86: 97 pgs: 97 active+clean; 2.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 21 op/s 2024-09-19T06:52:00.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:00 smithi103 ceph-mon[31158]: pgmap v86: 97 pgs: 97 active+clean; 2.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 21 op/s 2024-09-19T06:52:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:00 smithi134 ceph-mon[35356]: pgmap v86: 97 pgs: 97 active+clean; 2.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 21 op/s 2024-09-19T06:52:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:02 smithi174 ceph-mon[35637]: pgmap v87: 97 pgs: 97 active+clean; 2.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-19T06:52:02.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:02 smithi103 ceph-mon[31158]: pgmap v87: 97 pgs: 97 active+clean; 2.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-19T06:52:02.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:02 smithi134 ceph-mon[35356]: pgmap v87: 97 pgs: 97 active+clean; 2.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-19T06:52:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.976+0000 7fdc278a5640 1 -- 172.21.15.103:0/1487209007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc20100ab0 msgr2=0x7fdc20100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.976+0000 7fdc278a5640 1 --2- 172.21.15.103:0/1487209007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc20100ab0 0x7fdc20100e90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009ca0 tx=0x7fdc1002a950 comp rx=0 tx=0).stop 2024-09-19T06:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.976+0000 7fdc278a5640 1 -- 172.21.15.103:0/1487209007 shutdown_connections 2024-09-19T06:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.976+0000 7fdc278a5640 1 --2- 172.21.15.103:0/1487209007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc20105ee0 0x7fdc20106350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.976+0000 7fdc278a5640 1 --2- 172.21.15.103:0/1487209007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc201013d0 0x7fdc20105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.976+0000 7fdc278a5640 1 --2- 172.21.15.103:0/1487209007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc20100ab0 0x7fdc20100e90 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.976+0000 7fdc278a5640 1 -- 172.21.15.103:0/1487209007 >> 172.21.15.103:0/1487209007 conn(0x7fdc200fc6b0 msgr2=0x7fdc200fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.976+0000 7fdc278a5640 1 -- 172.21.15.103:0/1487209007 shutdown_connections 2024-09-19T06:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.976+0000 7fdc278a5640 1 -- 172.21.15.103:0/1487209007 wait complete. 2024-09-19T06:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.976+0000 7fdc278a5640 1 Processor -- start 2024-09-19T06:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.976+0000 7fdc278a5640 1 -- start start 2024-09-19T06:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc278a5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc20100ab0 0x7fdc201a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc278a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc201013d0 0x7fdc201a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc278a5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc20105ee0 0x7fdc2019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc278a5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201184f0 con 0x7fdc20100ab0 2024-09-19T06:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc278a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc20118370 con 0x7fdc201013d0 2024-09-19T06:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc278a5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc20118670 con 0x7fdc20105ee0 2024-09-19T06:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc25641640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc20100ab0 0x7fdc201a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc25641640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc20100ab0 0x7fdc201a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42410/0 (socket says 172.21.15.103:42410) 2024-09-19T06:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc25641640 1 -- 172.21.15.103:0/322090559 learned_addr learned my addr 172.21.15.103:0/322090559 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc25e42640 1 --2- 172.21.15.103:0/322090559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc20105ee0 0x7fdc2019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc24e40640 1 --2- 172.21.15.103:0/322090559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc201013d0 0x7fdc201a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc25641640 1 -- 172.21.15.103:0/322090559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc20105ee0 msgr2=0x7fdc2019e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc25641640 1 --2- 172.21.15.103:0/322090559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc20105ee0 0x7fdc2019e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc25641640 1 -- 172.21.15.103:0/322090559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc201013d0 msgr2=0x7fdc201a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc25641640 1 --2- 172.21.15.103:0/322090559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc201013d0 0x7fdc201a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc25641640 1 -- 172.21.15.103:0/322090559 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2019efd0 con 0x7fdc20100ab0 2024-09-19T06:52:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc25641640 1 --2- 172.21.15.103:0/322090559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc20100ab0 0x7fdc201a4690 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fdc10005cc0 tx=0x7fdc1002f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc1e7fc640 1 -- 172.21.15.103:0/322090559 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10034040 con 0x7fdc20100ab0 2024-09-19T06:52:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc1e7fc640 1 -- 172.21.15.103:0/322090559 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc10030920 con 0x7fdc20100ab0 2024-09-19T06:52:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc278a5640 1 -- 172.21.15.103:0/322090559 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2019f260 con 0x7fdc20100ab0 2024-09-19T06:52:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.977+0000 7fdc1e7fc640 1 -- 172.21.15.103:0/322090559 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10004780 con 0x7fdc20100ab0 2024-09-19T06:52:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.978+0000 7fdc278a5640 1 -- 172.21.15.103:0/322090559 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc201ab4e0 con 0x7fdc20100ab0 2024-09-19T06:52:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.980+0000 7fdc1e7fc640 1 -- 172.21.15.103:0/322090559 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fdc10037020 con 0x7fdc20100ab0 2024-09-19T06:52:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.980+0000 7fdc2a647740 1 Processor -- start 2024-09-19T06:52:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.980+0000 7fdc2a647740 1 -- start start 2024-09-19T06:52:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.980+0000 7fdc2a647740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0112af0 0x55dcd0112ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.980+0000 7fdc2a647740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 0x55dcd011d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.980+0000 7fdc2a647740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd011dcb0 0x55dcd01200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.980+0000 7fdc2a647740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dcd00772f0 con 0x55dcd0112af0 2024-09-19T06:52:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.980+0000 7fdc2a647740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dcd001f270 con 0x55dcd011dcb0 2024-09-19T06:52:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.980+0000 7fdc2a647740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dcd00732b0 con 0x55dcd0113410 2024-09-19T06:52:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.981+0000 7fdc25e42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0112af0 0x55dcd0112ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.981+0000 7fdc25e42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0112af0 0x55dcd0112ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42414/0 (socket says 172.21.15.103:42414) 2024-09-19T06:52:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.981+0000 7fdc25e42640 1 -- 172.21.15.103:0/995652087 learned_addr learned my addr 172.21.15.103:0/995652087 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.981+0000 7fdc1e7fc640 1 --2- 172.21.15.103:0/322090559 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdc080a3460 0x7fdc080a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.981+0000 7fdc24e40640 1 --2- 172.21.15.103:0/995652087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd011dcb0 0x55dcd01200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.981+0000 7fdc25641640 1 --2- 172.21.15.103:0/995652087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 0x55dcd011d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.981+0000 7fdc25e42640 1 --2- 172.21.15.103:0/322090559 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdc080a3460 0x7fdc080a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.981+0000 7fdc25e42640 1 -- 172.21.15.103:0/995652087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 msgr2=0x55dcd011d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.981+0000 7fdc25e42640 1 --2- 172.21.15.103:0/995652087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 0x55dcd011d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.981+0000 7fdc25e42640 1 -- 172.21.15.103:0/995652087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd011dcb0 msgr2=0x55dcd01200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.981+0000 7fdc25e42640 1 --2- 172.21.15.103:0/995652087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd011dcb0 0x55dcd01200a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.981+0000 7fdc25e42640 1 -- 172.21.15.103:0/995652087 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcd001e340 con 0x55dcd0112af0 2024-09-19T06:52:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.981+0000 7fdc24e40640 1 --2- 172.21.15.103:0/995652087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd011dcb0 0x55dcd01200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.981+0000 7fdc1e7fc640 1 -- 172.21.15.103:0/322090559 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fdc100f0b10 con 0x7fdc20100ab0 2024-09-19T06:52:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.981+0000 7fdc25e42640 1 --2- 172.21.15.103:0/995652087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0112af0 0x55dcd0112ed0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800bd20 tx=0x7fdc180039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b859bc88b56d64e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.981+0000 7fdc077fe640 1 -- 172.21.15.103:0/995652087 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1800b700 con 0x55dcd0112af0 2024-09-19T06:52:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.982+0000 7fdc077fe640 1 -- 172.21.15.103:0/995652087 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc180054b0 con 0x55dcd0112af0 2024-09-19T06:52:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.982+0000 7fdc077fe640 1 -- 172.21.15.103:0/995652087 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1801ed70 con 0x55dcd0112af0 2024-09-19T06:52:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.982+0000 7fdc2a647740 1 -- 172.21.15.103:0/995652087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0112af0 msgr2=0x55dcd0112ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.982+0000 7fdc25e42640 1 --2- 172.21.15.103:0/322090559 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdc080a3460 0x7fdc080a5920 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc18021000 tx=0x7fdc18020000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.982+0000 7fdc2a647740 1 --2- 172.21.15.103:0/995652087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0112af0 0x55dcd0112ed0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800bd20 tx=0x7fdc180039c0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.982+0000 7fdc2a647740 1 -- 172.21.15.103:0/995652087 shutdown_connections 2024-09-19T06:52:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.982+0000 7fdc2a647740 1 --2- 172.21.15.103:0/995652087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd011dcb0 0x55dcd01200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.982+0000 7fdc2a647740 1 --2- 172.21.15.103:0/995652087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 0x55dcd011d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.982+0000 7fdc2a647740 1 --2- 172.21.15.103:0/995652087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0112af0 0x55dcd0112ed0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.982+0000 7fdc2a647740 1 -- 172.21.15.103:0/995652087 >> 172.21.15.103:0/995652087 conn(0x55dcd010f060 msgr2=0x55dcd010f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.982+0000 7fdc2a647740 1 -- 172.21.15.103:0/995652087 shutdown_connections 2024-09-19T06:52:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.982+0000 7fdc2a647740 1 -- 172.21.15.103:0/995652087 wait complete. 2024-09-19T06:52:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.983+0000 7fdc2a647740 1 -- start start 2024-09-19T06:52:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.983+0000 7fdc2a647740 1 Processor -- start 2024-09-19T06:52:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.983+0000 7fdc2a647740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd0112af0 0x55dcd011abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.983+0000 7fdc2a647740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 0x55dcd011b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.983+0000 7fdc2a647740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd011dcb0 0x55dcd011b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.983+0000 7fdc2a647740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dcd0120880 con 0x55dcd011dcb0 2024-09-19T06:52:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.983+0000 7fdc2a647740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dcd0000290 con 0x55dcd0112af0 2024-09-19T06:52:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.983+0000 7fdc2a647740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dcd0120a00 con 0x55dcd0113410 2024-09-19T06:52:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.983+0000 7fdc2a647740 10 client.0 _ll_register_callbacks cb 0x55dcd0031140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.983+0000 7fdc25e42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd011dcb0 0x55dcd011b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.983+0000 7fdc25e42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd011dcb0 0x55dcd011b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42416/0 (socket says 172.21.15.103:42416) 2024-09-19T06:52:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.983+0000 7fdc25e42640 1 -- 172.21.15.103:0/1460576021 learned_addr learned my addr 172.21.15.103:0/1460576021 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.983+0000 7fdc24e40640 1 --2- 172.21.15.103:0/1460576021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 0x55dcd011b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc25641640 1 --2- 172.21.15.103:0/1460576021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd0112af0 0x55dcd011abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc25e42640 1 -- 172.21.15.103:0/1460576021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 msgr2=0x55dcd011b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc25e42640 1 --2- 172.21.15.103:0/1460576021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 0x55dcd011b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc25e42640 1 -- 172.21.15.103:0/1460576021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd0112af0 msgr2=0x55dcd011abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc25e42640 1 --2- 172.21.15.103:0/1460576021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd0112af0 0x55dcd011abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc25e42640 1 -- 172.21.15.103:0/1460576021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dccfd8a210 con 0x55dcd011dcb0 2024-09-19T06:52:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc24e40640 1 --2- 172.21.15.103:0/1460576021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 0x55dcd011b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc25641640 1 --2- 172.21.15.103:0/1460576021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd0112af0 0x55dcd011abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc25e42640 1 --2- 172.21.15.103:0/1460576021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd011dcb0 0x55dcd011b630 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800b320 tx=0x7fdc1801ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f056e0ad6e198351 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc04ff9640 1 -- 172.21.15.103:0/1460576021 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18035060 con 0x55dcd011dcb0 2024-09-19T06:52:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc2a647740 1 -- 172.21.15.103:0/1460576021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc180054b0 con 0x55dcd011dcb0 2024-09-19T06:52:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc04ff9640 1 -- 172.21.15.103:0/1460576021 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fdc1801eaf0 con 0x55dcd011dcb0 2024-09-19T06:52:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc04ff9640 1 -- 172.21.15.103:0/1460576021 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc18004b70 con 0x55dcd011dcb0 2024-09-19T06:52:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc2a647740 10 client.16446 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc2a647740 1 -- 172.21.15.103:0/1460576021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dccfe56620 con 0x55dcd011dcb0 2024-09-19T06:52:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.984+0000 7fdc04ff9640 1 -- 172.21.15.103:0/1460576021 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc180054b0 con 0x55dcd011dcb0 2024-09-19T06:52:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.985+0000 7fdc04ff9640 1 -- 172.21.15.103:0/1460576021 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdc1802f030 con 0x55dcd011dcb0 2024-09-19T06:52:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.985+0000 7fdc2a647740 10 client.16446 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.985+0000 7fdc2a647740 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.985+0000 7fdc2a647740 20 client.16446 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.985+0000 7fdc2a647740 1 --2- 172.21.15.103:0/1460576021 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dcd012dab0 0x55dcd012fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.985+0000 7fdc2a647740 4 client.16446 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.985+0000 7fdc2a647740 1 -- 172.21.15.103:0/1460576021 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcd0084a20 con 0x55dcd012dab0 2024-09-19T06:52:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.985+0000 7fdc25641640 1 --2- 172.21.15.103:0/1460576021 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dcd012dab0 0x55dcd012fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.985+0000 7fdc25641640 1 --2- 172.21.15.103:0/1460576021 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dcd012dab0 0x55dcd012fea0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5363e49f38df10be server_cookie=77ea53a7128c6685 in_seq=0 out_seq=0 2024-09-19T06:52:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.985+0000 7fdc04ff9640 10 client.16446 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.986+0000 7fdc04ff9640 1 -- 172.21.15.103:0/1460576021 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dcd0084a20 con 0x55dcd012dab0 2024-09-19T06:52:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.986+0000 7fdc04ff9640 10 client.16446 handle_command_reply: tid=0 2024-09-19T06:52:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.986+0000 7fdc2a647740 1 client.16446 shutdown 2024-09-19T06:52:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.986+0000 7fdc2a647740 1 -- 172.21.15.103:0/1460576021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd011dcb0 msgr2=0x55dcd011b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.986+0000 7fdc2a647740 1 --2- 172.21.15.103:0/1460576021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd011dcb0 0x55dcd011b630 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800b320 tx=0x7fdc1801ff80 comp rx=0 tx=0).stop 2024-09-19T06:52:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.986+0000 7fdc2a647740 1 -- 172.21.15.103:0/1460576021 shutdown_connections 2024-09-19T06:52:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.986+0000 7fdc2a647740 1 --2- 172.21.15.103:0/1460576021 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dcd012dab0 0x55dcd012fea0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.986+0000 7fdc2a647740 1 --2- 172.21.15.103:0/1460576021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd011dcb0 0x55dcd011b630 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.986+0000 7fdc2a647740 1 --2- 172.21.15.103:0/1460576021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 0x55dcd011b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.986+0000 7fdc04ff9640 0 client.16446 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.986+0000 7fdc2a647740 1 --2- 172.21.15.103:0/1460576021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd0112af0 0x55dcd011abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.986+0000 7fdc2a647740 1 -- 172.21.15.103:0/1460576021 >> 172.21.15.103:0/1460576021 conn(0x55dcd010f060 msgr2=0x55dcd0114d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.987+0000 7fdc2a647740 1 -- 172.21.15.103:0/1460576021 shutdown_connections 2024-09-19T06:52:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.987+0000 7fdc2a647740 1 -- 172.21.15.103:0/1460576021 wait complete. 2024-09-19T06:52:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:02.987+0000 7fdc2a647740 20 client.16446 trim_cache size 0 max 16384 2024-09-19T06:52:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc2a647740 1 Processor -- start 2024-09-19T06:52:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc2a647740 1 -- start start 2024-09-19T06:52:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc2a647740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0112af0 0x55dcd011b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc2a647740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 0x55dcd011cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc2a647740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd011dcb0 0x55dcd01357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc2a647740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201b3db0 con 0x55dcd0112af0 2024-09-19T06:52:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc2a647740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201b58f0 con 0x55dcd011dcb0 2024-09-19T06:52:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc2a647740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201b4480 con 0x55dcd0113410 2024-09-19T06:52:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc25641640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0112af0 0x55dcd011b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc25641640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0112af0 0x55dcd011b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42430/0 (socket says 172.21.15.103:42430) 2024-09-19T06:52:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc25641640 1 -- 172.21.15.103:0/3064102625 learned_addr learned my addr 172.21.15.103:0/3064102625 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc24e40640 1 --2- 172.21.15.103:0/3064102625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 0x55dcd011cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc25641640 1 -- 172.21.15.103:0/3064102625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 msgr2=0x55dcd011cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc25641640 1 --2- 172.21.15.103:0/3064102625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 0x55dcd011cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc25641640 1 -- 172.21.15.103:0/3064102625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd011dcb0 msgr2=0x55dcd01357b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc25641640 1 --2- 172.21.15.103:0/3064102625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd011dcb0 0x55dcd01357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc25641640 1 -- 172.21.15.103:0/3064102625 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcd0113240 con 0x55dcd0112af0 2024-09-19T06:52:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.001+0000 7fdc24e40640 1 --2- 172.21.15.103:0/3064102625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 0x55dcd011cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.002+0000 7fdc25641640 1 --2- 172.21.15.103:0/3064102625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0112af0 0x55dcd011b460 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdc100b9370 tx=0x7fdc1004e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec77663d73ebc50 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.002+0000 7fdc06ffd640 1 -- 172.21.15.103:0/3064102625 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc100bd610 con 0x55dcd0112af0 2024-09-19T06:52:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.002+0000 7fdc06ffd640 1 -- 172.21.15.103:0/3064102625 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc100bcf10 con 0x55dcd0112af0 2024-09-19T06:52:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.002+0000 7fdc06ffd640 1 -- 172.21.15.103:0/3064102625 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc100c56f0 con 0x55dcd0112af0 2024-09-19T06:52:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.002+0000 7fdc2a647740 1 -- 172.21.15.103:0/3064102625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0112af0 msgr2=0x55dcd011b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.002+0000 7fdc2a647740 1 --2- 172.21.15.103:0/3064102625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0112af0 0x55dcd011b460 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdc100b9370 tx=0x7fdc1004e000 comp rx=0 tx=0).stop 2024-09-19T06:52:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.002+0000 7fdc2a647740 1 -- 172.21.15.103:0/3064102625 shutdown_connections 2024-09-19T06:52:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.002+0000 7fdc2a647740 1 --2- 172.21.15.103:0/3064102625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd011dcb0 0x55dcd01357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.002+0000 7fdc2a647740 1 --2- 172.21.15.103:0/3064102625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0113410 0x55dcd011cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.002+0000 7fdc2a647740 1 --2- 172.21.15.103:0/3064102625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0112af0 0x55dcd011b460 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.002+0000 7fdc2a647740 1 -- 172.21.15.103:0/3064102625 >> 172.21.15.103:0/3064102625 conn(0x55dcd010f060 msgr2=0x55dcd011c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.002+0000 7fdc2a647740 1 -- 172.21.15.103:0/3064102625 shutdown_connections 2024-09-19T06:52:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.002+0000 7fdc2a647740 1 -- 172.21.15.103:0/3064102625 wait complete. 2024-09-19T06:52:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.003+0000 7fdc2a647740 1 -- start start 2024-09-19T06:52:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.003+0000 7fdc2a647740 1 Processor -- start 2024-09-19T06:52:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.003+0000 7fdc2a647740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0112af0 0x55dcd0121450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.003+0000 7fdc2a647740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0113410 0x55dcd0121990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.003+0000 7fdc2a647740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd011dcb0 0x55dcd0190740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.003+0000 7fdc2a647740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201b5220 con 0x55dcd0113410 2024-09-19T06:52:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.003+0000 7fdc2a647740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201b4b50 con 0x55dcd011dcb0 2024-09-19T06:52:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.003+0000 7fdc2a647740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201b1d70 con 0x55dcd0112af0 2024-09-19T06:52:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.003+0000 7fdc2a647740 10 client.0 _ll_register_callbacks cb 0x7fdc2006b360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.003+0000 7fdc24e40640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0113410 0x55dcd0121990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.003+0000 7fdc24e40640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0113410 0x55dcd0121990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42442/0 (socket says 172.21.15.103:42442) 2024-09-19T06:52:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.003+0000 7fdc24e40640 1 -- 172.21.15.103:0/3630453030 learned_addr learned my addr 172.21.15.103:0/3630453030 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.003+0000 7fdc25641640 1 --2- 172.21.15.103:0/3630453030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0112af0 0x55dcd0121450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.004+0000 7fdc25e42640 1 --2- 172.21.15.103:0/3630453030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd011dcb0 0x55dcd0190740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.004+0000 7fdc24e40640 1 -- 172.21.15.103:0/3630453030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0112af0 msgr2=0x55dcd0121450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.004+0000 7fdc24e40640 1 --2- 172.21.15.103:0/3630453030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0112af0 0x55dcd0121450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.004+0000 7fdc24e40640 1 -- 172.21.15.103:0/3630453030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd011dcb0 msgr2=0x55dcd0190740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.004+0000 7fdc24e40640 1 --2- 172.21.15.103:0/3630453030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd011dcb0 0x55dcd0190740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.004+0000 7fdc24e40640 1 -- 172.21.15.103:0/3630453030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcd0190d80 con 0x55dcd0113410 2024-09-19T06:52:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.004+0000 7fdc25641640 1 --2- 172.21.15.103:0/3630453030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0112af0 0x55dcd0121450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.004+0000 7fdc24e40640 1 --2- 172.21.15.103:0/3630453030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0113410 0x55dcd0121990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fdc200ffcc0 tx=0x7fdc14012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7f97116531081d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.004+0000 7fdc04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.004+0000 7fdc04ff9640 1 -- 172.21.15.103:0/3630453030 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14022060 con 0x55dcd0113410 2024-09-19T06:52:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.004+0000 7fdc04ff9640 1 -- 172.21.15.103:0/3630453030 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fdc14004250 con 0x55dcd0113410 2024-09-19T06:52:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.004+0000 7fdc2a647740 1 -- 172.21.15.103:0/3630453030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc100bcf10 con 0x55dcd0113410 2024-09-19T06:52:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.004+0000 7fdc04ff9640 1 -- 172.21.15.103:0/3630453030 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc14004450 con 0x55dcd0113410 2024-09-19T06:52:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.004+0000 7fdc2a647740 10 client.16452 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.004+0000 7fdc2a647740 1 -- 172.21.15.103:0/3630453030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dcd0198140 con 0x55dcd0113410 2024-09-19T06:52:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.004+0000 7fdc04ff9640 1 -- 172.21.15.103:0/3630453030 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc100bcf10 con 0x55dcd0113410 2024-09-19T06:52:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.005+0000 7fdc04ff9640 1 -- 172.21.15.103:0/3630453030 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdc1401d400 con 0x55dcd0113410 2024-09-19T06:52:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.005+0000 7fdc2a647740 10 client.16452 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.005+0000 7fdc2a647740 10 client.16452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.005+0000 7fdc2a647740 20 client.16452 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.005+0000 7fdc2a647740 1 --2- 172.21.15.103:0/3630453030 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dcd0198920 0x55dcd019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.005+0000 7fdc2a647740 4 client.16452 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:52:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.005+0000 7fdc2a647740 1 -- 172.21.15.103:0/3630453030 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dcd007ed70 con 0x55dcd0198920 2024-09-19T06:52:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.005+0000 7fdc25641640 1 --2- 172.21.15.103:0/3630453030 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dcd0198920 0x55dcd019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.005+0000 7fdc04ff9640 10 client.16452 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.005+0000 7fdc25641640 1 --2- 172.21.15.103:0/3630453030 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dcd0198920 0x55dcd019ad40 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93ab3f4cd0e71646 server_cookie=4e9d1e5c9ff57379 in_seq=0 out_seq=0 2024-09-19T06:52:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.018+0000 7fdc04ff9640 1 -- 172.21.15.103:0/3630453030 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dcd007ed70 con 0x55dcd0198920 2024-09-19T06:52:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.018+0000 7fdc04ff9640 10 client.16452 handle_command_reply: tid=0 2024-09-19T06:52:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.018+0000 7fdc2a647740 1 client.16452 shutdown 2024-09-19T06:52:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.018+0000 7fdc2a647740 1 -- 172.21.15.103:0/3630453030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0113410 msgr2=0x55dcd0121990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.018+0000 7fdc2a647740 1 --2- 172.21.15.103:0/3630453030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0113410 0x55dcd0121990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fdc200ffcc0 tx=0x7fdc14012b60 comp rx=0 tx=0).stop 2024-09-19T06:52:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.018+0000 7fdc2a647740 1 -- 172.21.15.103:0/3630453030 shutdown_connections 2024-09-19T06:52:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.018+0000 7fdc2a647740 1 --2- 172.21.15.103:0/3630453030 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dcd0198920 0x55dcd019ad40 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc2a647740 1 --2- 172.21.15.103:0/3630453030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcd011dcb0 0x55dcd0190740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc2a647740 1 --2- 172.21.15.103:0/3630453030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcd0113410 0x55dcd0121990 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc2a647740 1 --2- 172.21.15.103:0/3630453030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcd0112af0 0x55dcd0121450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc04ff9640 0 client.16452 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc2a647740 1 -- 172.21.15.103:0/3630453030 >> 172.21.15.103:0/3630453030 conn(0x55dcd010f060 msgr2=0x55dcd0119030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc2a647740 1 -- 172.21.15.103:0/3630453030 shutdown_connections 2024-09-19T06:52:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc2a647740 1 -- 172.21.15.103:0/3630453030 wait complete. 2024-09-19T06:52:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc2a647740 20 client.16452 trim_cache size 0 max 16384 2024-09-19T06:52:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:52:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:52:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:52:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:52:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc278a5640 1 -- 172.21.15.103:0/322090559 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdc080a3460 msgr2=0x7fdc080a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc278a5640 1 --2- 172.21.15.103:0/322090559 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdc080a3460 0x7fdc080a5920 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc18021000 tx=0x7fdc18020000 comp rx=0 tx=0).stop 2024-09-19T06:52:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc278a5640 1 -- 172.21.15.103:0/322090559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc20100ab0 msgr2=0x7fdc201a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc278a5640 1 --2- 172.21.15.103:0/322090559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc20100ab0 0x7fdc201a4690 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fdc10005cc0 tx=0x7fdc1002f630 comp rx=0 tx=0).stop 2024-09-19T06:52:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc278a5640 1 -- 172.21.15.103:0/322090559 shutdown_connections 2024-09-19T06:52:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc278a5640 1 --2- 172.21.15.103:0/322090559 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdc080a3460 0x7fdc080a5920 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc278a5640 1 --2- 172.21.15.103:0/322090559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc20105ee0 0x7fdc2019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc278a5640 1 --2- 172.21.15.103:0/322090559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc201013d0 0x7fdc201a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc278a5640 1 --2- 172.21.15.103:0/322090559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc20100ab0 0x7fdc201a4690 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc278a5640 1 -- 172.21.15.103:0/322090559 >> 172.21.15.103:0/322090559 conn(0x7fdc200fc6b0 msgr2=0x7fdc20103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc278a5640 1 -- 172.21.15.103:0/322090559 shutdown_connections 2024-09-19T06:52:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:03.019+0000 7fdc278a5640 1 -- 172.21.15.103:0/322090559 wait complete. 2024-09-19T06:52:04.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:04 smithi174 ceph-mon[35637]: pgmap v88: 97 pgs: 97 active+clean; 2.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-19T06:52:04.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:04 smithi103 ceph-mon[31158]: pgmap v88: 97 pgs: 97 active+clean; 2.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-19T06:52:04.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:04 smithi134 ceph-mon[35356]: pgmap v88: 97 pgs: 97 active+clean; 2.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-19T06:52:04.806 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:52:05.069 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:05.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.644+0000 7fc924eef640 1 -- 172.21.15.103:0/253155144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc920102570 msgr2=0x7fc9201029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:05.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.644+0000 7fc924eef640 1 --2- 172.21.15.103:0/253155144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc920102570 0x7fc9201029b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc914009900 tx=0x7fc91402a480 comp rx=0 tx=0).stop 2024-09-19T06:52:05.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.644+0000 7fc924eef640 1 -- 172.21.15.103:0/253155144 shutdown_connections 2024-09-19T06:52:05.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.644+0000 7fc924eef640 1 --2- 172.21.15.103:0/253155144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc920102ef0 0x7fc92010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.644+0000 7fc924eef640 1 --2- 172.21.15.103:0/253155144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc920102570 0x7fc9201029b0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.644+0000 7fc924eef640 1 --2- 172.21.15.103:0/253155144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc920100e20 0x7fc920101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.644+0000 7fc924eef640 1 -- 172.21.15.103:0/253155144 >> 172.21.15.103:0/253155144 conn(0x7fc9200fc7d0 msgr2=0x7fc9200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:05.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.644+0000 7fc924eef640 1 -- 172.21.15.103:0/253155144 shutdown_connections 2024-09-19T06:52:05.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.644+0000 7fc924eef640 1 -- 172.21.15.103:0/253155144 wait complete. 2024-09-19T06:52:05.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.645+0000 7fc924eef640 1 Processor -- start 2024-09-19T06:52:05.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.645+0000 7fc924eef640 1 -- start start 2024-09-19T06:52:05.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.645+0000 7fc924eef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc920100e20 0x7fc920199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.645+0000 7fc924eef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc920102570 0x7fc920199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.645+0000 7fc924eef640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc920102ef0 0x7fc9201a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.645+0000 7fc924eef640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc920110f80 con 0x7fc920102570 2024-09-19T06:52:05.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.645+0000 7fc91dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc920102570 0x7fc920199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.645+0000 7fc91e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc920100e20 0x7fc920199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.645+0000 7fc924eef640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc920110e00 con 0x7fc920100e20 2024-09-19T06:52:05.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.645+0000 7fc924eef640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc920111100 con 0x7fc920102ef0 2024-09-19T06:52:05.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.645+0000 7fc91e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc920100e20 0x7fc920199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:48612/0 (socket says 172.21.15.103:48612) 2024-09-19T06:52:05.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.645+0000 7fc91dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc920102570 0x7fc920199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42470/0 (socket says 172.21.15.103:42470) 2024-09-19T06:52:05.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.645+0000 7fc91e59c640 1 -- 172.21.15.103:0/3990481564 learned_addr learned my addr 172.21.15.103:0/3990481564 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:05.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.645+0000 7fc91ed9d640 1 --2- 172.21.15.103:0/3990481564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc920102ef0 0x7fc9201a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.646+0000 7fc91e59c640 1 -- 172.21.15.103:0/3990481564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc920102ef0 msgr2=0x7fc9201a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:05.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.646+0000 7fc91e59c640 1 --2- 172.21.15.103:0/3990481564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc920102ef0 0x7fc9201a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.646+0000 7fc91e59c640 1 -- 172.21.15.103:0/3990481564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc920102570 msgr2=0x7fc920199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:05.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.646+0000 7fc91e59c640 1 --2- 172.21.15.103:0/3990481564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc920102570 0x7fc920199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.646+0000 7fc91e59c640 1 -- 172.21.15.103:0/3990481564 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9201a48f0 con 0x7fc920100e20 2024-09-19T06:52:05.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.646+0000 7fc91e59c640 1 --2- 172.21.15.103:0/3990481564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc920100e20 0x7fc920199330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc91000d950 tx=0x7fc91000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:05.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.646+0000 7fc90f7fe640 1 -- 172.21.15.103:0/3990481564 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910017070 con 0x7fc920100e20 2024-09-19T06:52:05.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.646+0000 7fc924eef640 1 -- 172.21.15.103:0/3990481564 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9201a4ac0 con 0x7fc920100e20 2024-09-19T06:52:05.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.646+0000 7fc90f7fe640 1 -- 172.21.15.103:0/3990481564 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc910009c30 con 0x7fc920100e20 2024-09-19T06:52:05.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.646+0000 7fc924eef640 1 -- 172.21.15.103:0/3990481564 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9201a4fd0 con 0x7fc920100e20 2024-09-19T06:52:05.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.646+0000 7fc90f7fe640 1 -- 172.21.15.103:0/3990481564 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91000f6b0 con 0x7fc920100e20 2024-09-19T06:52:05.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.649+0000 7fc9272c8740 1 Processor -- start 2024-09-19T06:52:05.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.649+0000 7fc9272c8740 1 -- start start 2024-09-19T06:52:05.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.649+0000 7fc9272c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1e4a0 0x562809f1e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.649+0000 7fc9272c8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f1edc0 0x562809f29140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.649+0000 7fc9272c8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f29680 0x562809f2ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.649+0000 7fc9272c8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562809e842e0 con 0x562809f1edc0 2024-09-19T06:52:05.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.649+0000 7fc9272c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562809e2bce0 con 0x562809f1e4a0 2024-09-19T06:52:05.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.649+0000 7fc9272c8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562809e802a0 con 0x562809f29680 2024-09-19T06:52:05.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.649+0000 7fc90f7fe640 1 -- 172.21.15.103:0/3990481564 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc9100109d0 con 0x7fc920100e20 2024-09-19T06:52:05.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.649+0000 7fc91e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f1edc0 0x562809f29140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.649+0000 7fc91e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f1edc0 0x562809f29140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42480/0 (socket says 172.21.15.103:42480) 2024-09-19T06:52:05.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.649+0000 7fc91e59c640 1 -- 172.21.15.103:0/3961322954 learned_addr learned my addr 172.21.15.103:0/3961322954 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:05.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.649+0000 7fc91dd9b640 1 --2- 172.21.15.103:0/3961322954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f29680 0x562809f2ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.649+0000 7fc91ed9d640 1 --2- 172.21.15.103:0/3961322954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1e4a0 0x562809f1e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.650+0000 7fc91e59c640 1 -- 172.21.15.103:0/3961322954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f29680 msgr2=0x562809f2ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:05.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.650+0000 7fc91e59c640 1 --2- 172.21.15.103:0/3961322954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f29680 0x562809f2ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.650+0000 7fc91e59c640 1 -- 172.21.15.103:0/3961322954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1e4a0 msgr2=0x562809f1e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:05.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.650+0000 7fc91e59c640 1 --2- 172.21.15.103:0/3961322954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1e4a0 0x562809f1e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.650+0000 7fc91e59c640 1 -- 172.21.15.103:0/3961322954 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562809e10440 con 0x562809f1edc0 2024-09-19T06:52:05.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.650+0000 7fc91e59c640 1 --2- 172.21.15.103:0/3961322954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f1edc0 0x562809f29140 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc9100cd2b0 tx=0x7fc9100d3e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2cca70468d8754 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:05.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.650+0000 7fc90cff9640 1 -- 172.21.15.103:0/3961322954 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9100d8e50 con 0x562809f1edc0 2024-09-19T06:52:05.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.650+0000 7fc90cff9640 1 -- 172.21.15.103:0/3961322954 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9100da900 con 0x562809f1edc0 2024-09-19T06:52:05.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.650+0000 7fc90cff9640 1 -- 172.21.15.103:0/3961322954 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9100e2a30 con 0x562809f1edc0 2024-09-19T06:52:05.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.650+0000 7fc90f7fe640 1 --2- 172.21.15.103:0/3990481564 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc9080a33b0 0x7fc9080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.651+0000 7fc91ed9d640 1 --2- 172.21.15.103:0/3990481564 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc9080a33b0 0x7fc9080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.651+0000 7fc90f7fe640 1 -- 172.21.15.103:0/3990481564 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc9100cfc10 con 0x7fc920100e20 2024-09-19T06:52:05.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.651+0000 7fc9272c8740 1 -- 172.21.15.103:0/3961322954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f1edc0 msgr2=0x562809f29140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:05.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.651+0000 7fc9272c8740 1 --2- 172.21.15.103:0/3961322954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f1edc0 0x562809f29140 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc9100cd2b0 tx=0x7fc9100d3e80 comp rx=0 tx=0).stop 2024-09-19T06:52:05.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.651+0000 7fc9272c8740 1 -- 172.21.15.103:0/3961322954 shutdown_connections 2024-09-19T06:52:05.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.651+0000 7fc9272c8740 1 --2- 172.21.15.103:0/3961322954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f29680 0x562809f2ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.651+0000 7fc9272c8740 1 --2- 172.21.15.103:0/3961322954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f1edc0 0x562809f29140 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.651+0000 7fc9272c8740 1 --2- 172.21.15.103:0/3961322954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1e4a0 0x562809f1e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.651+0000 7fc9272c8740 1 -- 172.21.15.103:0/3961322954 >> 172.21.15.103:0/3961322954 conn(0x562809f1aa30 msgr2=0x562809f1ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:05.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.651+0000 7fc91ed9d640 1 --2- 172.21.15.103:0/3990481564 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc9080a33b0 0x7fc9080a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc918005fd0 tx=0x7fc918009600 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:05.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.651+0000 7fc9272c8740 1 -- 172.21.15.103:0/3961322954 shutdown_connections 2024-09-19T06:52:05.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.651+0000 7fc9272c8740 1 -- 172.21.15.103:0/3961322954 wait complete. 2024-09-19T06:52:05.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.651+0000 7fc9272c8740 1 -- start start 2024-09-19T06:52:05.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc9272c8740 1 Processor -- start 2024-09-19T06:52:05.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc9272c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1e4a0 0x562809f29010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc9272c8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1edc0 0x562809f262c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc9272c8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f26840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc9272c8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562809f2c250 con 0x562809f29680 2024-09-19T06:52:05.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc9272c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562809e3f050 con 0x562809f1e4a0 2024-09-19T06:52:05.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc9272c8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562809f2c3d0 con 0x562809f1edc0 2024-09-19T06:52:05.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc9272c8740 10 client.0 _ll_register_callbacks cb 0x562809e7f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:05.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc91ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f26840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc91ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f26840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42496/0 (socket says 172.21.15.103:42496) 2024-09-19T06:52:05.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc91ed9d640 1 -- 172.21.15.103:0/1522425937 learned_addr learned my addr 172.21.15.103:0/1522425937 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:05.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc91dd9b640 1 --2- 172.21.15.103:0/1522425937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1edc0 0x562809f262c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc91e59c640 1 --2- 172.21.15.103:0/1522425937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1e4a0 0x562809f29010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc91ed9d640 1 -- 172.21.15.103:0/1522425937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1edc0 msgr2=0x562809f262c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:05.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc91ed9d640 1 --2- 172.21.15.103:0/1522425937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1edc0 0x562809f262c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc91ed9d640 1 -- 172.21.15.103:0/1522425937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1e4a0 msgr2=0x562809f29010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:05.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc91ed9d640 1 --2- 172.21.15.103:0/1522425937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1e4a0 0x562809f29010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc91ed9d640 1 -- 172.21.15.103:0/1522425937 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562809bf53c0 con 0x562809f29680 2024-09-19T06:52:05.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc91e59c640 1 --2- 172.21.15.103:0/1522425937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1e4a0 0x562809f29010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:05.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc91ed9d640 1 --2- 172.21.15.103:0/1522425937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f26840 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc918011d50 tx=0x7fc918004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4df67673a80f334 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:05.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.652+0000 7fc905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:05.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.653+0000 7fc905ffb640 1 -- 172.21.15.103:0/1522425937 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918020620 con 0x562809f29680 2024-09-19T06:52:05.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.653+0000 7fc905ffb640 1 -- 172.21.15.103:0/1522425937 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc9180207c0 con 0x562809f29680 2024-09-19T06:52:05.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.653+0000 7fc9272c8740 1 -- 172.21.15.103:0/1522425937 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9100da900 con 0x562809f29680 2024-09-19T06:52:05.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.653+0000 7fc905ffb640 1 -- 172.21.15.103:0/1522425937 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc918020a00 con 0x562809f29680 2024-09-19T06:52:05.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.653+0000 7fc9272c8740 10 client.16464 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:05.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.653+0000 7fc9272c8740 1 -- 172.21.15.103:0/1522425937 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562809f37ca0 con 0x562809f29680 2024-09-19T06:52:05.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.653+0000 7fc905ffb640 1 -- 172.21.15.103:0/1522425937 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9100da900 con 0x562809f29680 2024-09-19T06:52:05.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.653+0000 7fc905ffb640 1 -- 172.21.15.103:0/1522425937 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc91800f040 con 0x562809f29680 2024-09-19T06:52:05.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.653+0000 7fc9272c8740 10 client.16464 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:05.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.653+0000 7fc9272c8740 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:05.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.653+0000 7fc9272c8740 20 client.16464 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:05.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.653+0000 7fc9272c8740 1 --2- 172.21.15.103:0/1522425937 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562809f39160 0x562809f3b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.653+0000 7fc9272c8740 4 client.16464 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:05.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.653+0000 7fc9272c8740 1 -- 172.21.15.103:0/1522425937 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562809e91850 con 0x562809f39160 2024-09-19T06:52:05.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.654+0000 7fc91e59c640 1 --2- 172.21.15.103:0/1522425937 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562809f39160 0x562809f3b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.654+0000 7fc905ffb640 10 client.16464 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:05.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.654+0000 7fc91e59c640 1 --2- 172.21.15.103:0/1522425937 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562809f39160 0x562809f3b5b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d099296dac7c689c server_cookie=d48a61a4bffb1333 in_seq=0 out_seq=0 2024-09-19T06:52:05.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.655+0000 7fc905ffb640 1 -- 172.21.15.103:0/1522425937 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562809e91850 con 0x562809f39160 2024-09-19T06:52:05.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.655+0000 7fc905ffb640 10 client.16464 handle_command_reply: tid=0 2024-09-19T06:52:05.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.655+0000 7fc9272c8740 1 client.16464 shutdown 2024-09-19T06:52:05.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.655+0000 7fc9272c8740 1 -- 172.21.15.103:0/1522425937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 msgr2=0x562809f26840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:05.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.655+0000 7fc9272c8740 1 --2- 172.21.15.103:0/1522425937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f26840 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc918011d50 tx=0x7fc918004a40 comp rx=0 tx=0).stop 2024-09-19T06:52:05.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.655+0000 7fc9272c8740 1 -- 172.21.15.103:0/1522425937 shutdown_connections 2024-09-19T06:52:05.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.655+0000 7fc9272c8740 1 --2- 172.21.15.103:0/1522425937 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562809f39160 0x562809f3b5b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.655+0000 7fc9272c8740 1 --2- 172.21.15.103:0/1522425937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f26840 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.655+0000 7fc9272c8740 1 --2- 172.21.15.103:0/1522425937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1edc0 0x562809f262c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.655+0000 7fc9272c8740 1 --2- 172.21.15.103:0/1522425937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1e4a0 0x562809f29010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.655+0000 7fc905ffb640 0 client.16464 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:05.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.655+0000 7fc9272c8740 1 -- 172.21.15.103:0/1522425937 >> 172.21.15.103:0/1522425937 conn(0x562809f1aa30 msgr2=0x562809f20760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:05.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.655+0000 7fc9272c8740 1 -- 172.21.15.103:0/1522425937 shutdown_connections 2024-09-19T06:52:05.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.655+0000 7fc9272c8740 1 -- 172.21.15.103:0/1522425937 wait complete. 2024-09-19T06:52:05.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.655+0000 7fc9272c8740 20 client.16464 trim_cache size 0 max 16384 2024-09-19T06:52:05.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.672+0000 7fc9272c8740 1 Processor -- start 2024-09-19T06:52:05.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.672+0000 7fc9272c8740 1 -- start start 2024-09-19T06:52:05.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.672+0000 7fc9272c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1e4a0 0x562809f28230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.672+0000 7fc9272c8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1edc0 0x562809f40ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.672+0000 7fc9272c8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f41410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.672+0000 7fc9272c8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201ad9a0 con 0x562809f29680 2024-09-19T06:52:05.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.672+0000 7fc9272c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201af4e0 con 0x562809f1e4a0 2024-09-19T06:52:05.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.672+0000 7fc9272c8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201ae070 con 0x562809f1edc0 2024-09-19T06:52:05.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.672+0000 7fc91ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f41410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.672+0000 7fc91ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f41410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42506/0 (socket says 172.21.15.103:42506) 2024-09-19T06:52:05.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.672+0000 7fc91ed9d640 1 -- 172.21.15.103:0/287225323 learned_addr learned my addr 172.21.15.103:0/287225323 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.673+0000 7fc91ed9d640 1 -- 172.21.15.103:0/287225323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1edc0 msgr2=0x562809f40ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.673+0000 7fc91dd9b640 1 --2- 172.21.15.103:0/287225323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1edc0 0x562809f40ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.673+0000 7fc91ed9d640 1 --2- 172.21.15.103:0/287225323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1edc0 0x562809f40ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.673+0000 7fc91ed9d640 1 -- 172.21.15.103:0/287225323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1e4a0 msgr2=0x562809f28230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.673+0000 7fc91e59c640 1 --2- 172.21.15.103:0/287225323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1e4a0 0x562809f28230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.673+0000 7fc91ed9d640 1 --2- 172.21.15.103:0/287225323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1e4a0 0x562809f28230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.673+0000 7fc91ed9d640 1 -- 172.21.15.103:0/287225323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562809f1e2b0 con 0x562809f29680 2024-09-19T06:52:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.673+0000 7fc91dd9b640 1 --2- 172.21.15.103:0/287225323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1edc0 0x562809f40ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.673+0000 7fc91ed9d640 1 --2- 172.21.15.103:0/287225323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f41410 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc918014fd0 tx=0x7fc918011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743e2df56b7fa8f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.673+0000 7fc907fff640 1 -- 172.21.15.103:0/287225323 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91801fe80 con 0x562809f29680 2024-09-19T06:52:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.673+0000 7fc907fff640 1 -- 172.21.15.103:0/287225323 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc918020ba0 con 0x562809f29680 2024-09-19T06:52:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.673+0000 7fc907fff640 1 -- 172.21.15.103:0/287225323 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91801d640 con 0x562809f29680 2024-09-19T06:52:05.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.673+0000 7fc9272c8740 1 -- 172.21.15.103:0/287225323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 msgr2=0x562809f41410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.673+0000 7fc9272c8740 1 --2- 172.21.15.103:0/287225323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f41410 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc918014fd0 tx=0x7fc918011980 comp rx=0 tx=0).stop 2024-09-19T06:52:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.674+0000 7fc9272c8740 1 -- 172.21.15.103:0/287225323 shutdown_connections 2024-09-19T06:52:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.674+0000 7fc9272c8740 1 --2- 172.21.15.103:0/287225323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f41410 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.674+0000 7fc9272c8740 1 --2- 172.21.15.103:0/287225323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1edc0 0x562809f40ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.674+0000 7fc9272c8740 1 --2- 172.21.15.103:0/287225323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1e4a0 0x562809f28230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.674+0000 7fc9272c8740 1 -- 172.21.15.103:0/287225323 >> 172.21.15.103:0/287225323 conn(0x562809f1aa30 msgr2=0x562809f27b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.674+0000 7fc9272c8740 1 -- 172.21.15.103:0/287225323 shutdown_connections 2024-09-19T06:52:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.674+0000 7fc9272c8740 1 -- 172.21.15.103:0/287225323 wait complete. 2024-09-19T06:52:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.674+0000 7fc9272c8740 1 -- start start 2024-09-19T06:52:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.674+0000 7fc9272c8740 1 Processor -- start 2024-09-19T06:52:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.674+0000 7fc9272c8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1e4a0 0x562809f31e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.674+0000 7fc9272c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1edc0 0x562809f2f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.674+0000 7fc9272c8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f2f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc9272c8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201aee10 con 0x562809f29680 2024-09-19T06:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc9272c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201ae740 con 0x562809f1edc0 2024-09-19T06:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc9272c8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201ab960 con 0x562809f1e4a0 2024-09-19T06:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc9272c8740 10 client.0 _ll_register_callbacks cb 0x7fc918009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc91e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1e4a0 0x562809f31e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc91e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1e4a0 0x562809f31e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:43608/0 (socket says 172.21.15.103:43608) 2024-09-19T06:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc91e59c640 1 -- 172.21.15.103:0/2234611791 learned_addr learned my addr 172.21.15.103:0/2234611791 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc91ed9d640 1 --2- 172.21.15.103:0/2234611791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f2f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc91dd9b640 1 --2- 172.21.15.103:0/2234611791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1edc0 0x562809f2f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc91e59c640 1 -- 172.21.15.103:0/2234611791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1edc0 msgr2=0x562809f2f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc91e59c640 1 --2- 172.21.15.103:0/2234611791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1edc0 0x562809f2f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc91e59c640 1 -- 172.21.15.103:0/2234611791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 msgr2=0x562809f2f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc91e59c640 1 --2- 172.21.15.103:0/2234611791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f2f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc91e59c640 1 -- 172.21.15.103:0/2234611791 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562809f2ffa0 con 0x562809f1e4a0 2024-09-19T06:52:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc91ed9d640 1 --2- 172.21.15.103:0/2234611791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f2f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc91e59c640 1 --2- 172.21.15.103:0/2234611791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1e4a0 0x562809f31e60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc910098980 tx=0x7fc9100e2530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c8af136799a54d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T06:52:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc905ffb640 1 -- 172.21.15.103:0/2234611791 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910020040 con 0x562809f1e4a0 2024-09-19T06:52:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc905ffb640 1 -- 172.21.15.103:0/2234611791 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc9100d34d0 con 0x562809f1e4a0 2024-09-19T06:52:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.675+0000 7fc9272c8740 1 -- 172.21.15.103:0/2234611791 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc918020ba0 con 0x562809f1e4a0 2024-09-19T06:52:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.676+0000 7fc905ffb640 1 -- 172.21.15.103:0/2234611791 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc910098250 con 0x562809f1e4a0 2024-09-19T06:52:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.676+0000 7fc9272c8740 10 client.25684 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.676+0000 7fc9272c8740 1 -- 172.21.15.103:0/2234611791 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562809fa8320 con 0x562809f1e4a0 2024-09-19T06:52:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.676+0000 7fc905ffb640 1 -- 172.21.15.103:0/2234611791 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc918020ba0 con 0x562809f1e4a0 2024-09-19T06:52:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.676+0000 7fc905ffb640 1 -- 172.21.15.103:0/2234611791 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc9100b9050 con 0x562809f1e4a0 2024-09-19T06:52:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.676+0000 7fc9272c8740 10 client.25684 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.676+0000 7fc9272c8740 10 client.25684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.676+0000 7fc9272c8740 20 client.25684 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.676+0000 7fc9272c8740 1 --2- 172.21.15.103:0/2234611791 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562809fa85a0 0x562809faa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.676+0000 7fc9272c8740 4 client.25684 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:52:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.676+0000 7fc9272c8740 1 -- 172.21.15.103:0/2234611791 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562809e8bd60 con 0x562809fa85a0 2024-09-19T06:52:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.676+0000 7fc91dd9b640 1 --2- 172.21.15.103:0/2234611791 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562809fa85a0 0x562809faa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc91dd9b640 1 --2- 172.21.15.103:0/2234611791 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562809fa85a0 0x562809faa990 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0d1997ede2624de server_cookie=5409c731cbc38741 in_seq=0 out_seq=0 2024-09-19T06:52:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc905ffb640 10 client.25684 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc905ffb640 1 -- 172.21.15.103:0/2234611791 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562809e8bd60 con 0x562809fa85a0 2024-09-19T06:52:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc905ffb640 10 client.25684 handle_command_reply: tid=0 2024-09-19T06:52:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc9272c8740 1 client.25684 shutdown 2024-09-19T06:52:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc9272c8740 1 -- 172.21.15.103:0/2234611791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1e4a0 msgr2=0x562809f31e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc9272c8740 1 --2- 172.21.15.103:0/2234611791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1e4a0 0x562809f31e60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc910098980 tx=0x7fc9100e2530 comp rx=0 tx=0).stop 2024-09-19T06:52:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc9272c8740 1 -- 172.21.15.103:0/2234611791 shutdown_connections 2024-09-19T06:52:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc9272c8740 1 --2- 172.21.15.103:0/2234611791 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562809fa85a0 0x562809faa990 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc9272c8740 1 --2- 172.21.15.103:0/2234611791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562809f29680 0x562809f2f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc9272c8740 1 --2- 172.21.15.103:0/2234611791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562809f1edc0 0x562809f2f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc905ffb640 0 client.25684 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc9272c8740 1 --2- 172.21.15.103:0/2234611791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562809f1e4a0 0x562809f31e60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc9272c8740 1 -- 172.21.15.103:0/2234611791 >> 172.21.15.103:0/2234611791 conn(0x562809f1aa30 msgr2=0x562809f28340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc9272c8740 1 -- 172.21.15.103:0/2234611791 shutdown_connections 2024-09-19T06:52:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc9272c8740 1 -- 172.21.15.103:0/2234611791 wait complete. 2024-09-19T06:52:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.677+0000 7fc9272c8740 20 client.25684 trim_cache size 0 max 16384 2024-09-19T06:52:05.683 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:52:05.683 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T06:52:05.684 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T06:52:05.684 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:52:05.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.678+0000 7fc924eef640 1 -- 172.21.15.103:0/3990481564 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc9080a33b0 msgr2=0x7fc9080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:05.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.678+0000 7fc924eef640 1 --2- 172.21.15.103:0/3990481564 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc9080a33b0 0x7fc9080a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc918005fd0 tx=0x7fc918009600 comp rx=0 tx=0).stop 2024-09-19T06:52:05.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.678+0000 7fc924eef640 1 -- 172.21.15.103:0/3990481564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc920100e20 msgr2=0x7fc920199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:05.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.678+0000 7fc924eef640 1 --2- 172.21.15.103:0/3990481564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc920100e20 0x7fc920199330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc91000d950 tx=0x7fc91000de10 comp rx=0 tx=0).stop 2024-09-19T06:52:05.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.678+0000 7fc924eef640 1 -- 172.21.15.103:0/3990481564 shutdown_connections 2024-09-19T06:52:05.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.678+0000 7fc924eef640 1 --2- 172.21.15.103:0/3990481564 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc9080a33b0 0x7fc9080a5870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.678+0000 7fc924eef640 1 --2- 172.21.15.103:0/3990481564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc920102ef0 0x7fc9201a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.678+0000 7fc924eef640 1 --2- 172.21.15.103:0/3990481564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc920102570 0x7fc920199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.678+0000 7fc924eef640 1 --2- 172.21.15.103:0/3990481564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc920100e20 0x7fc920199330 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:05.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.678+0000 7fc924eef640 1 -- 172.21.15.103:0/3990481564 >> 172.21.15.103:0/3990481564 conn(0x7fc9200fc7d0 msgr2=0x7fc9200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:05.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.678+0000 7fc924eef640 1 -- 172.21.15.103:0/3990481564 shutdown_connections 2024-09-19T06:52:05.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:05.678+0000 7fc924eef640 1 -- 172.21.15.103:0/3990481564 wait complete. 2024-09-19T06:52:06.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:06 smithi174 ceph-mon[35637]: pgmap v89: 97 pgs: 97 active+clean; 2.3 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 55 KiB/s wr, 36 op/s 2024-09-19T06:52:06.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:06 smithi103 ceph-mon[31158]: pgmap v89: 97 pgs: 97 active+clean; 2.3 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 55 KiB/s wr, 36 op/s 2024-09-19T06:52:06.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:06 smithi134 ceph-mon[35356]: pgmap v89: 97 pgs: 97 active+clean; 2.3 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 55 KiB/s wr, 36 op/s 2024-09-19T06:52:06.474 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:52:06.474 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:52:06.722 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:07.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.199+0000 7fc1c0689640 1 -- 172.21.15.103:0/2186579645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1b8102790 msgr2=0x7fc1b81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:07.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.199+0000 7fc1c0689640 1 --2- 172.21.15.103:0/2186579645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1b8102790 0x7fc1b81067e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009900 tx=0x7fc1b402a410 comp rx=0 tx=0).stop 2024-09-19T06:52:07.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.199+0000 7fc1c0689640 1 -- 172.21.15.103:0/2186579645 shutdown_connections 2024-09-19T06:52:07.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.199+0000 7fc1c0689640 1 --2- 172.21.15.103:0/2186579645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b8106f10 0x7fc1b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:07.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.199+0000 7fc1c0689640 1 --2- 172.21.15.103:0/2186579645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1b8102790 0x7fc1b81067e0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:07.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.199+0000 7fc1c0689640 1 --2- 172.21.15.103:0/2186579645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1b8101de0 0x7fc1b81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:07.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.199+0000 7fc1c0689640 1 -- 172.21.15.103:0/2186579645 >> 172.21.15.103:0/2186579645 conn(0x7fc1b80fd8c0 msgr2=0x7fc1b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:07.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.199+0000 7fc1c0689640 1 -- 172.21.15.103:0/2186579645 shutdown_connections 2024-09-19T06:52:07.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.199+0000 7fc1c0689640 1 -- 172.21.15.103:0/2186579645 wait complete. 2024-09-19T06:52:07.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1c0689640 1 Processor -- start 2024-09-19T06:52:07.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1c0689640 1 -- start start 2024-09-19T06:52:07.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1c0689640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b8101de0 0x7fc1b8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:07.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1c0689640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1b8102790 0x7fc1b8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:07.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1c0689640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1b8106f10 0x7fc1b819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:07.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1c0689640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b8111f90 con 0x7fc1b8106f10 2024-09-19T06:52:07.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1c0689640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b8111e10 con 0x7fc1b8101de0 2024-09-19T06:52:07.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1c0689640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b8112110 con 0x7fc1b8102790 2024-09-19T06:52:07.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1bec26640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1b8106f10 0x7fc1b819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:07.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1bec26640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1b8106f10 0x7fc1b819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42536/0 (socket says 172.21.15.103:42536) 2024-09-19T06:52:07.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1bec26640 1 -- 172.21.15.103:0/4077020070 learned_addr learned my addr 172.21.15.103:0/4077020070 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:07.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1bdc24640 1 --2- 172.21.15.103:0/4077020070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1b8102790 0x7fc1b8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:07.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1be425640 1 --2- 172.21.15.103:0/4077020070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b8101de0 0x7fc1b8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:07.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1bec26640 1 -- 172.21.15.103:0/4077020070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1b8102790 msgr2=0x7fc1b8198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:07.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1bec26640 1 --2- 172.21.15.103:0/4077020070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1b8102790 0x7fc1b8198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:07.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1bec26640 1 -- 172.21.15.103:0/4077020070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b8101de0 msgr2=0x7fc1b8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:07.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1bec26640 1 --2- 172.21.15.103:0/4077020070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b8101de0 0x7fc1b8198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:07.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1bec26640 1 -- 172.21.15.103:0/4077020070 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b819d520 con 0x7fc1b8106f10 2024-09-19T06:52:07.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.200+0000 7fc1be425640 1 --2- 172.21.15.103:0/4077020070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b8101de0 0x7fc1b8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:07.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.201+0000 7fc1bec26640 1 --2- 172.21.15.103:0/4077020070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1b8106f10 0x7fc1b819ce40 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009fd0 tx=0x7fc1b000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:07.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.201+0000 7fc1ab7fe640 1 -- 172.21.15.103:0/4077020070 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0011830 con 0x7fc1b8106f10 2024-09-19T06:52:07.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.201+0000 7fc1ab7fe640 1 -- 172.21.15.103:0/4077020070 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1b0016070 con 0x7fc1b8106f10 2024-09-19T06:52:07.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.201+0000 7fc1ab7fe640 1 -- 172.21.15.103:0/4077020070 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b000f2e0 con 0x7fc1b8106f10 2024-09-19T06:52:07.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.201+0000 7fc1c0689640 1 -- 172.21.15.103:0/4077020070 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b819d810 con 0x7fc1b8106f10 2024-09-19T06:52:07.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.201+0000 7fc1c0689640 1 -- 172.21.15.103:0/4077020070 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b81a50f0 con 0x7fc1b8106f10 2024-09-19T06:52:07.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.202+0000 7fc1c0689640 1 -- 172.21.15.103:0/4077020070 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b81021c0 con 0x7fc1b8106f10 2024-09-19T06:52:07.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.203+0000 7fc1ab7fe640 1 -- 172.21.15.103:0/4077020070 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc1b00119d0 con 0x7fc1b8106f10 2024-09-19T06:52:07.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.203+0000 7fc1ab7fe640 1 --2- 172.21.15.103:0/4077020070 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc1a40a33b0 0x7fc1a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:07.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.204+0000 7fc1be425640 1 --2- 172.21.15.103:0/4077020070 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc1a40a33b0 0x7fc1a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:07.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.204+0000 7fc1ab7fe640 1 -- 172.21.15.103:0/4077020070 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc1b00cfa20 con 0x7fc1b8106f10 2024-09-19T06:52:07.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.204+0000 7fc1be425640 1 --2- 172.21.15.103:0/4077020070 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc1a40a33b0 0x7fc1a40a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac005fd0 tx=0x7fc1ac0093e0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:07.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.207+0000 7fc1ab7fe640 1 -- 172.21.15.103:0/4077020070 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc1b009c520 con 0x7fc1b8106f10 2024-09-19T06:52:07.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.408+0000 7fc1c0689640 1 -- 172.21.15.103:0/4077020070 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1b810a150 con 0x7fc1b8106f10 2024-09-19T06:52:07.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.409+0000 7fc1ab7fe640 1 -- 172.21.15.103:0/4077020070 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fc1b009be20 con 0x7fc1b8106f10 2024-09-19T06:52:07.410 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:52:07.410 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:52:07.410 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:52:07.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.411+0000 7fc1c0689640 1 -- 172.21.15.103:0/4077020070 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc1a40a33b0 msgr2=0x7fc1a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:07.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.411+0000 7fc1c0689640 1 --2- 172.21.15.103:0/4077020070 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc1a40a33b0 0x7fc1a40a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac005fd0 tx=0x7fc1ac0093e0 comp rx=0 tx=0).stop 2024-09-19T06:52:07.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.411+0000 7fc1c0689640 1 -- 172.21.15.103:0/4077020070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1b8106f10 msgr2=0x7fc1b819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:07.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.411+0000 7fc1c0689640 1 --2- 172.21.15.103:0/4077020070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1b8106f10 0x7fc1b819ce40 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009fd0 tx=0x7fc1b000b8e0 comp rx=0 tx=0).stop 2024-09-19T06:52:07.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.411+0000 7fc1c0689640 1 -- 172.21.15.103:0/4077020070 shutdown_connections 2024-09-19T06:52:07.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.411+0000 7fc1c0689640 1 --2- 172.21.15.103:0/4077020070 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc1a40a33b0 0x7fc1a40a5870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:07.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.411+0000 7fc1c0689640 1 --2- 172.21.15.103:0/4077020070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1b8106f10 0x7fc1b819ce40 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:07.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.411+0000 7fc1c0689640 1 --2- 172.21.15.103:0/4077020070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1b8102790 0x7fc1b8198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:07.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.411+0000 7fc1c0689640 1 --2- 172.21.15.103:0/4077020070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1b8101de0 0x7fc1b8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:07.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.411+0000 7fc1c0689640 1 -- 172.21.15.103:0/4077020070 >> 172.21.15.103:0/4077020070 conn(0x7fc1b80fd8c0 msgr2=0x7fc1b80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:07.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.411+0000 7fc1c0689640 1 -- 172.21.15.103:0/4077020070 shutdown_connections 2024-09-19T06:52:07.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:07.411+0000 7fc1c0689640 1 -- 172.21.15.103:0/4077020070 wait complete. 2024-09-19T06:52:07.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:07 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/4077020070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:07.873 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:07 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/4077020070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:07.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:07 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/4077020070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:08.139 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:52:08.386 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:08.765 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:08 smithi103 ceph-mon[31158]: pgmap v90: 97 pgs: 97 active+clean; 2.3 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-09-19T06:52:08.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.787+0000 7fa92289f640 1 -- 172.21.15.103:0/1222555535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa91c102790 msgr2=0x7fa91c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.787+0000 7fa92289f640 1 --2- 172.21.15.103:0/1222555535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa91c102790 0x7fa91c1067e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa904009900 tx=0x7fa90402a410 comp rx=0 tx=0).stop 2024-09-19T06:52:08.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.787+0000 7fa92289f640 1 -- 172.21.15.103:0/1222555535 shutdown_connections 2024-09-19T06:52:08.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.787+0000 7fa92289f640 1 --2- 172.21.15.103:0/1222555535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa91c106f10 0x7fa91c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.787+0000 7fa92289f640 1 --2- 172.21.15.103:0/1222555535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa91c102790 0x7fa91c1067e0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.787+0000 7fa92289f640 1 --2- 172.21.15.103:0/1222555535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa91c101de0 0x7fa91c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.787+0000 7fa92289f640 1 -- 172.21.15.103:0/1222555535 >> 172.21.15.103:0/1222555535 conn(0x7fa91c0fd8c0 msgr2=0x7fa91c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:08.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.787+0000 7fa92289f640 1 -- 172.21.15.103:0/1222555535 shutdown_connections 2024-09-19T06:52:08.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.787+0000 7fa92289f640 1 -- 172.21.15.103:0/1222555535 wait complete. 2024-09-19T06:52:08.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa92289f640 1 Processor -- start 2024-09-19T06:52:08.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa92289f640 1 -- start start 2024-09-19T06:52:08.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa92289f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa91c101de0 0x7fa91c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa92289f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa91c102790 0x7fa91c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa92289f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa91c106f10 0x7fa91c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa92289f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c111f70 con 0x7fa91c101de0 2024-09-19T06:52:08.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa92289f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c111df0 con 0x7fa91c106f10 2024-09-19T06:52:08.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa92289f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1120f0 con 0x7fa91c102790 2024-09-19T06:52:08.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa91bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa91c101de0 0x7fa91c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa91bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa91c101de0 0x7fa91c19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53352/0 (socket says 172.21.15.103:53352) 2024-09-19T06:52:08.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa91bfff640 1 -- 172.21.15.103:0/888597770 learned_addr learned my addr 172.21.15.103:0/888597770 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:08.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa913fff640 1 --2- 172.21.15.103:0/888597770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa91c102790 0x7fa91c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa920e3c640 1 --2- 172.21.15.103:0/888597770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa91c106f10 0x7fa91c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa91bfff640 1 -- 172.21.15.103:0/888597770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa91c102790 msgr2=0x7fa91c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa91bfff640 1 --2- 172.21.15.103:0/888597770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa91c102790 0x7fa91c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa91bfff640 1 -- 172.21.15.103:0/888597770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa91c106f10 msgr2=0x7fa91c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa91bfff640 1 --2- 172.21.15.103:0/888597770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa91c106f10 0x7fa91c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa91bfff640 1 -- 172.21.15.103:0/888597770 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c19f560 con 0x7fa91c101de0 2024-09-19T06:52:08.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.788+0000 7fa920e3c640 1 --2- 172.21.15.103:0/888597770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa91c106f10 0x7fa91c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:08.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.789+0000 7fa91bfff640 1 --2- 172.21.15.103:0/888597770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa91c101de0 0x7fa91c19a5b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00d950 tx=0x7fa90c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:08.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.789+0000 7fa919ffb640 1 -- 172.21.15.103:0/888597770 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c017070 con 0x7fa91c101de0 2024-09-19T06:52:08.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.789+0000 7fa919ffb640 1 -- 172.21.15.103:0/888597770 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c009c30 con 0x7fa91c101de0 2024-09-19T06:52:08.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.789+0000 7fa92289f640 1 -- 172.21.15.103:0/888597770 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91c19f850 con 0x7fa91c101de0 2024-09-19T06:52:08.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.789+0000 7fa919ffb640 1 -- 172.21.15.103:0/888597770 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00f6b0 con 0x7fa91c101de0 2024-09-19T06:52:08.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.789+0000 7fa92289f640 1 -- 172.21.15.103:0/888597770 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91c1a7130 con 0x7fa91c101de0 2024-09-19T06:52:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.791+0000 7fa919ffb640 1 -- 172.21.15.103:0/888597770 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa90c0109d0 con 0x7fa91c101de0 2024-09-19T06:52:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.792+0000 7fa924c78740 1 Processor -- start 2024-09-19T06:52:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.792+0000 7fa924c78740 1 -- start start 2024-09-19T06:52:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.792+0000 7fa924c78740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b744b40 0x55696b743090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.792+0000 7fa924c78740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b7435d0 0x55696b743a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.792+0000 7fa924c78740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b74e370 0x55696b750760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.792+0000 7fa924c78740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55696b6a92f0 con 0x55696b7435d0 2024-09-19T06:52:08.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.792+0000 7fa924c78740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55696b650cf0 con 0x55696b74e370 2024-09-19T06:52:08.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.792+0000 7fa924c78740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55696b6a52b0 con 0x55696b744b40 2024-09-19T06:52:08.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.792+0000 7fa919ffb640 1 --2- 172.21.15.103:0/888597770 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa9080a3430 0x7fa9080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.792+0000 7fa91bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b7435d0 0x55696b743a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.792+0000 7fa920e3c640 1 --2- 172.21.15.103:0/888597770 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa9080a3430 0x7fa9080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.792+0000 7fa91bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b7435d0 0x55696b743a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53358/0 (socket says 172.21.15.103:53358) 2024-09-19T06:52:08.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.792+0000 7fa91bfff640 1 -- 172.21.15.103:0/3139160512 learned_addr learned my addr 172.21.15.103:0/3139160512 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:08.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.792+0000 7fa920e3c640 1 --2- 172.21.15.103:0/3139160512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b744b40 0x55696b743090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.792+0000 7fa919ffb640 1 -- 172.21.15.103:0/888597770 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa90c0cf940 con 0x7fa91c101de0 2024-09-19T06:52:08.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.792+0000 7fa913fff640 1 --2- 172.21.15.103:0/3139160512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b74e370 0x55696b750760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.793+0000 7fa91bfff640 1 -- 172.21.15.103:0/3139160512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b744b40 msgr2=0x55696b743090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:08.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.793+0000 7fa91bfff640 1 --2- 172.21.15.103:0/3139160512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b744b40 0x55696b743090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.793+0000 7fa91bfff640 1 -- 172.21.15.103:0/3139160512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b74e370 msgr2=0x55696b750760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:08.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.793+0000 7fa91bfff640 1 --2- 172.21.15.103:0/3139160512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b74e370 0x55696b750760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.793+0000 7fa91bfff640 1 -- 172.21.15.103:0/3139160512 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55696b6353a0 con 0x55696b7435d0 2024-09-19T06:52:08.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.793+0000 7fa920e3c640 1 --2- 172.21.15.103:0/3139160512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b744b40 0x55696b743090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:08.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.793+0000 7fa920e3c640 1 --2- 172.21.15.103:0/888597770 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa9080a3430 0x7fa9080a58f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa914007fd0 tx=0x7fa914007d00 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:08.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.793+0000 7fa91bfff640 1 --2- 172.21.15.103:0/3139160512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b7435d0 0x55696b743a10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0b2150 tx=0x7fa90c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbe0d8165d9dc1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:08.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.793+0000 7fa9127fc640 1 -- 172.21.15.103:0/3139160512 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c020070 con 0x55696b7435d0 2024-09-19T06:52:08.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.793+0000 7fa9127fc640 1 -- 172.21.15.103:0/3139160512 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c0b3070 con 0x55696b7435d0 2024-09-19T06:52:08.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.793+0000 7fa9127fc640 1 -- 172.21.15.103:0/3139160512 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0cd2e0 con 0x55696b7435d0 2024-09-19T06:52:08.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.794+0000 7fa924c78740 1 -- 172.21.15.103:0/3139160512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b7435d0 msgr2=0x55696b743a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.794+0000 7fa924c78740 1 --2- 172.21.15.103:0/3139160512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b7435d0 0x55696b743a10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0b2150 tx=0x7fa90c00bf60 comp rx=0 tx=0).stop 2024-09-19T06:52:08.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.794+0000 7fa924c78740 1 -- 172.21.15.103:0/3139160512 shutdown_connections 2024-09-19T06:52:08.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.794+0000 7fa924c78740 1 --2- 172.21.15.103:0/3139160512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b74e370 0x55696b750760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.794+0000 7fa924c78740 1 --2- 172.21.15.103:0/3139160512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b7435d0 0x55696b743a10 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.794+0000 7fa924c78740 1 --2- 172.21.15.103:0/3139160512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b744b40 0x55696b743090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.794+0000 7fa924c78740 1 -- 172.21.15.103:0/3139160512 >> 172.21.15.103:0/3139160512 conn(0x55696b70c310 msgr2=0x55696b73fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:08.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.794+0000 7fa924c78740 1 -- 172.21.15.103:0/3139160512 shutdown_connections 2024-09-19T06:52:08.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.794+0000 7fa924c78740 1 -- 172.21.15.103:0/3139160512 wait complete. 2024-09-19T06:52:08.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.794+0000 7fa924c78740 1 -- start start 2024-09-19T06:52:08.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.794+0000 7fa924c78740 1 Processor -- start 2024-09-19T06:52:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa924c78740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b7435d0 0x55696b74ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa924c78740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b744b40 0x55696b74b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa924c78740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b74e370 0x55696b74b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa924c78740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55696b751010 con 0x55696b7435d0 2024-09-19T06:52:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa924c78740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55696b664060 con 0x55696b744b40 2024-09-19T06:52:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa924c78740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55696b751190 con 0x55696b74e370 2024-09-19T06:52:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa924c78740 10 client.0 _ll_register_callbacks cb 0x55696b3b78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa91bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b7435d0 0x55696b74ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa91bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b7435d0 0x55696b74ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53372/0 (socket says 172.21.15.103:53372) 2024-09-19T06:52:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa91bfff640 1 -- 172.21.15.103:0/3297745149 learned_addr learned my addr 172.21.15.103:0/3297745149 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:08.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa920e3c640 1 --2- 172.21.15.103:0/3297745149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b74e370 0x55696b74b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa913fff640 1 --2- 172.21.15.103:0/3297745149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b744b40 0x55696b74b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa91bfff640 1 -- 172.21.15.103:0/3297745149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b74e370 msgr2=0x55696b74b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:08.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa91bfff640 1 --2- 172.21.15.103:0/3297745149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b74e370 0x55696b74b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa91bfff640 1 -- 172.21.15.103:0/3297745149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b744b40 msgr2=0x55696b74b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:08.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa91bfff640 1 --2- 172.21.15.103:0/3297745149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b744b40 0x55696b74b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa91bfff640 1 -- 172.21.15.103:0/3297745149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55696b41a3c0 con 0x55696b7435d0 2024-09-19T06:52:08.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa91bfff640 1 --2- 172.21.15.103:0/3297745149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b7435d0 0x55696b74ddd0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa90c098050 tx=0x7fa90c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7974763fb7c379 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:08.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa8fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:08.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa8fb7fe640 1 -- 172.21.15.103:0/3297745149 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c020070 con 0x55696b7435d0 2024-09-19T06:52:08.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa8fb7fe640 1 -- 172.21.15.103:0/3297745149 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa90c0cd2e0 con 0x55696b7435d0 2024-09-19T06:52:08.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa924c78740 1 -- 172.21.15.103:0/3297745149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa90c0b3070 con 0x55696b7435d0 2024-09-19T06:52:08.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.795+0000 7fa8fb7fe640 1 -- 172.21.15.103:0/3297745149 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa90c09ffa0 con 0x55696b7435d0 2024-09-19T06:52:08.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.796+0000 7fa924c78740 10 client.16488 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:08.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.796+0000 7fa924c78740 1 -- 172.21.15.103:0/3297745149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55696b75cb30 con 0x55696b7435d0 2024-09-19T06:52:08.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.796+0000 7fa8fb7fe640 1 -- 172.21.15.103:0/3297745149 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c0b3070 con 0x55696b7435d0 2024-09-19T06:52:08.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.796+0000 7fa8fb7fe640 1 -- 172.21.15.103:0/3297745149 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa90c0861d0 con 0x55696b7435d0 2024-09-19T06:52:08.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.796+0000 7fa924c78740 10 client.16488 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:08.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.796+0000 7fa924c78740 10 client.16488 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:52:08.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.796+0000 7fa924c78740 20 client.16488 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:08.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.796+0000 7fa924c78740 1 --2- 172.21.15.103:0/3297745149 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55696b75de80 0x55696b760270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.796+0000 7fa924c78740 4 client.16488 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.796+0000 7fa924c78740 1 -- 172.21.15.103:0/3297745149 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55696b6b6860 con 0x55696b75de80 2024-09-19T06:52:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.796+0000 7fa913fff640 1 --2- 172.21.15.103:0/3297745149 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55696b75de80 0x55696b760270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.797+0000 7fa8fb7fe640 10 client.16488 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:52:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.797+0000 7fa913fff640 1 --2- 172.21.15.103:0/3297745149 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55696b75de80 0x55696b760270 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16946748f7c08eea server_cookie=510c7b24d0a0e16e in_seq=0 out_seq=0 2024-09-19T06:52:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.797+0000 7fa8fb7fe640 1 -- 172.21.15.103:0/3297745149 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55696b6b6860 con 0x55696b75de80 2024-09-19T06:52:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.797+0000 7fa8fb7fe640 10 client.16488 handle_command_reply: tid=0 2024-09-19T06:52:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.797+0000 7fa924c78740 1 client.16488 shutdown 2024-09-19T06:52:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.797+0000 7fa924c78740 1 -- 172.21.15.103:0/3297745149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b7435d0 msgr2=0x55696b74ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.797+0000 7fa924c78740 1 --2- 172.21.15.103:0/3297745149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b7435d0 0x55696b74ddd0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa90c098050 tx=0x7fa90c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.798+0000 7fa924c78740 1 -- 172.21.15.103:0/3297745149 shutdown_connections 2024-09-19T06:52:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.798+0000 7fa924c78740 1 --2- 172.21.15.103:0/3297745149 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55696b75de80 0x55696b760270 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.798+0000 7fa924c78740 1 --2- 172.21.15.103:0/3297745149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b74e370 0x55696b74b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.798+0000 7fa8fb7fe640 0 client.16488 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:52:08.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.798+0000 7fa924c78740 1 --2- 172.21.15.103:0/3297745149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b744b40 0x55696b74b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.798+0000 7fa924c78740 1 --2- 172.21.15.103:0/3297745149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b7435d0 0x55696b74ddd0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.798+0000 7fa924c78740 1 -- 172.21.15.103:0/3297745149 >> 172.21.15.103:0/3297745149 conn(0x55696b70c310 msgr2=0x55696b745990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:08.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.798+0000 7fa924c78740 1 -- 172.21.15.103:0/3297745149 shutdown_connections 2024-09-19T06:52:08.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.798+0000 7fa924c78740 1 -- 172.21.15.103:0/3297745149 wait complete. 2024-09-19T06:52:08.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.798+0000 7fa924c78740 20 client.16488 trim_cache size 0 max 16384 2024-09-19T06:52:08.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.810+0000 7fa924c78740 1 Processor -- start 2024-09-19T06:52:08.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.810+0000 7fa924c78740 1 -- start start 2024-09-19T06:52:08.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa924c78740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b7435d0 0x55696b744410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa924c78740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b744b40 0x55696b746120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa924c78740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b74e370 0x55696b746660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa924c78740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1af9a0 con 0x55696b74e370 2024-09-19T06:52:08.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa924c78740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1b14e0 con 0x55696b744b40 2024-09-19T06:52:08.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa924c78740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1b0070 con 0x55696b7435d0 2024-09-19T06:52:08.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa920e3c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b74e370 0x55696b746660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa920e3c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b74e370 0x55696b746660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53384/0 (socket says 172.21.15.103:53384) 2024-09-19T06:52:08.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa920e3c640 1 -- 172.21.15.103:0/582396786 learned_addr learned my addr 172.21.15.103:0/582396786 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:08.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa91bfff640 1 --2- 172.21.15.103:0/582396786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b7435d0 0x55696b744410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa913fff640 1 --2- 172.21.15.103:0/582396786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b744b40 0x55696b746120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa920e3c640 1 -- 172.21.15.103:0/582396786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b7435d0 msgr2=0x55696b744410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:08.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa920e3c640 1 --2- 172.21.15.103:0/582396786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b7435d0 0x55696b744410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa920e3c640 1 -- 172.21.15.103:0/582396786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b744b40 msgr2=0x55696b746120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:08.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa920e3c640 1 --2- 172.21.15.103:0/582396786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b744b40 0x55696b746120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa920e3c640 1 -- 172.21.15.103:0/582396786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55696b740960 con 0x55696b74e370 2024-09-19T06:52:08.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa920e3c640 1 --2- 172.21.15.103:0/582396786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b74e370 0x55696b746660 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa914013730 tx=0x7fa914004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907d2011cb98234 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:08.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.811+0000 7fa911ffb640 1 -- 172.21.15.103:0/582396786 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914024070 con 0x55696b74e370 2024-09-19T06:52:08.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.812+0000 7fa911ffb640 1 -- 172.21.15.103:0/582396786 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9140132b0 con 0x55696b74e370 2024-09-19T06:52:08.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.812+0000 7fa911ffb640 1 -- 172.21.15.103:0/582396786 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914020d90 con 0x55696b74e370 2024-09-19T06:52:08.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.812+0000 7fa924c78740 1 -- 172.21.15.103:0/582396786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b74e370 msgr2=0x55696b746660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.812+0000 7fa924c78740 1 --2- 172.21.15.103:0/582396786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b74e370 0x55696b746660 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa914013730 tx=0x7fa914004ab0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.812+0000 7fa924c78740 1 -- 172.21.15.103:0/582396786 shutdown_connections 2024-09-19T06:52:08.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.812+0000 7fa924c78740 1 --2- 172.21.15.103:0/582396786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b74e370 0x55696b746660 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.812+0000 7fa924c78740 1 --2- 172.21.15.103:0/582396786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b744b40 0x55696b746120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.812+0000 7fa924c78740 1 --2- 172.21.15.103:0/582396786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b7435d0 0x55696b744410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.812+0000 7fa924c78740 1 -- 172.21.15.103:0/582396786 >> 172.21.15.103:0/582396786 conn(0x55696b70c310 msgr2=0x55696b74a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:08.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.812+0000 7fa924c78740 1 -- 172.21.15.103:0/582396786 shutdown_connections 2024-09-19T06:52:08.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.812+0000 7fa924c78740 1 -- 172.21.15.103:0/582396786 wait complete. 2024-09-19T06:52:08.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.813+0000 7fa924c78740 1 -- start start 2024-09-19T06:52:08.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.813+0000 7fa924c78740 1 Processor -- start 2024-09-19T06:52:08.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.813+0000 7fa924c78740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b7435d0 0x55696b7639f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.813+0000 7fa924c78740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b744b40 0x55696b760d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.813+0000 7fa924c78740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b74e370 0x55696b761280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.813+0000 7fa924c78740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1b0e10 con 0x55696b74e370 2024-09-19T06:52:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.813+0000 7fa924c78740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1b0740 con 0x55696b744b40 2024-09-19T06:52:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.813+0000 7fa924c78740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1ad960 con 0x55696b7435d0 2024-09-19T06:52:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.813+0000 7fa924c78740 10 client.0 _ll_register_callbacks cb 0x7fa904009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.813+0000 7fa920e3c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b74e370 0x55696b761280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.813+0000 7fa920e3c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b74e370 0x55696b761280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53400/0 (socket says 172.21.15.103:53400) 2024-09-19T06:52:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.813+0000 7fa920e3c640 1 -- 172.21.15.103:0/757750791 learned_addr learned my addr 172.21.15.103:0/757750791 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.813+0000 7fa91bfff640 1 --2- 172.21.15.103:0/757750791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b7435d0 0x55696b7639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa913fff640 1 --2- 172.21.15.103:0/757750791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b744b40 0x55696b760d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa920e3c640 1 -- 172.21.15.103:0/757750791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b7435d0 msgr2=0x55696b7639f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa920e3c640 1 --2- 172.21.15.103:0/757750791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b7435d0 0x55696b7639f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa920e3c640 1 -- 172.21.15.103:0/757750791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b744b40 msgr2=0x55696b760d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa920e3c640 1 --2- 172.21.15.103:0/757750791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b744b40 0x55696b760d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa920e3c640 1 -- 172.21.15.103:0/757750791 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55696b761a60 con 0x55696b74e370 2024-09-19T06:52:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa913fff640 1 --2- 172.21.15.103:0/757750791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b744b40 0x55696b760d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa920e3c640 1 --2- 172.21.15.103:0/757750791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b74e370 0x55696b761280 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa914020a40 tx=0x7fa914004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef489e8f5426958 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa8fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa8fb7fe640 1 -- 172.21.15.103:0/757750791 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914024040 con 0x55696b74e370 2024-09-19T06:52:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa8fb7fe640 1 -- 172.21.15.103:0/757750791 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa914020d20 con 0x55696b74e370 2024-09-19T06:52:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa924c78740 1 -- 172.21.15.103:0/757750791 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9140132b0 con 0x55696b74e370 2024-09-19T06:52:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa8fb7fe640 1 -- 172.21.15.103:0/757750791 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa9140119c0 con 0x55696b74e370 2024-09-19T06:52:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa924c78740 10 client.16494 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa924c78740 1 -- 172.21.15.103:0/757750791 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55696b7c87d0 con 0x55696b74e370 2024-09-19T06:52:08.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa8fb7fe640 1 -- 172.21.15.103:0/757750791 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9140132b0 con 0x55696b74e370 2024-09-19T06:52:08.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.814+0000 7fa8fb7fe640 1 -- 172.21.15.103:0/757750791 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa914034ad0 con 0x55696b74e370 2024-09-19T06:52:08.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.815+0000 7fa924c78740 10 client.16494 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:08.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.815+0000 7fa924c78740 10 client.16494 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:52:08.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.815+0000 7fa924c78740 20 client.16494 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:08.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.815+0000 7fa924c78740 1 --2- 172.21.15.103:0/757750791 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55696b75de80 0x55696b7cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.815+0000 7fa924c78740 4 client.16494 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:52:08.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.815+0000 7fa924c78740 1 -- 172.21.15.103:0/757750791 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55696b6b0d70 con 0x55696b75de80 2024-09-19T06:52:08.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.815+0000 7fa91bfff640 1 --2- 172.21.15.103:0/757750791 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55696b75de80 0x55696b7cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.815+0000 7fa91bfff640 1 --2- 172.21.15.103:0/757750791 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55696b75de80 0x55696b7cae00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b1e7b49b7e688b20 server_cookie=7c7141b36d5da047 in_seq=0 out_seq=0 2024-09-19T06:52:08.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.815+0000 7fa8fb7fe640 10 client.16494 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:52:08.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.815+0000 7fa8fb7fe640 1 -- 172.21.15.103:0/757750791 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55696b6b0d70 con 0x55696b75de80 2024-09-19T06:52:08.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.815+0000 7fa8fb7fe640 10 client.16494 handle_command_reply: tid=0 2024-09-19T06:52:08.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.815+0000 7fa924c78740 1 client.16494 shutdown 2024-09-19T06:52:08.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa924c78740 1 -- 172.21.15.103:0/757750791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b74e370 msgr2=0x55696b761280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa924c78740 1 --2- 172.21.15.103:0/757750791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b74e370 0x55696b761280 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa914020a40 tx=0x7fa914004a40 comp rx=0 tx=0).stop 2024-09-19T06:52:08.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa924c78740 1 -- 172.21.15.103:0/757750791 shutdown_connections 2024-09-19T06:52:08.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa924c78740 1 --2- 172.21.15.103:0/757750791 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55696b75de80 0x55696b7cae00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa924c78740 1 --2- 172.21.15.103:0/757750791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55696b74e370 0x55696b761280 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa924c78740 1 --2- 172.21.15.103:0/757750791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55696b744b40 0x55696b760d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa924c78740 1 --2- 172.21.15.103:0/757750791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55696b7435d0 0x55696b7639f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa924c78740 1 -- 172.21.15.103:0/757750791 >> 172.21.15.103:0/757750791 conn(0x55696b70c310 msgr2=0x55696b759420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:08.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa8fb7fe640 0 client.16494 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:52:08.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa924c78740 1 -- 172.21.15.103:0/757750791 shutdown_connections 2024-09-19T06:52:08.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa924c78740 1 -- 172.21.15.103:0/757750791 wait complete. 2024-09-19T06:52:08.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa924c78740 20 client.16494 trim_cache size 0 max 16384 2024-09-19T06:52:08.824 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:52:08.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa92289f640 1 -- 172.21.15.103:0/888597770 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa9080a3430 msgr2=0x7fa9080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa92289f640 1 --2- 172.21.15.103:0/888597770 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa9080a3430 0x7fa9080a58f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa914007fd0 tx=0x7fa914007d00 comp rx=0 tx=0).stop 2024-09-19T06:52:08.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa92289f640 1 -- 172.21.15.103:0/888597770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa91c101de0 msgr2=0x7fa91c19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa92289f640 1 --2- 172.21.15.103:0/888597770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa91c101de0 0x7fa91c19a5b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00d950 tx=0x7fa90c00de10 comp rx=0 tx=0).stop 2024-09-19T06:52:08.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa92289f640 1 -- 172.21.15.103:0/888597770 shutdown_connections 2024-09-19T06:52:08.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa92289f640 1 --2- 172.21.15.103:0/888597770 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa9080a3430 0x7fa9080a58f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa92289f640 1 --2- 172.21.15.103:0/888597770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa91c106f10 0x7fa91c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa92289f640 1 --2- 172.21.15.103:0/888597770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa91c102790 0x7fa91c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa92289f640 1 --2- 172.21.15.103:0/888597770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa91c101de0 0x7fa91c19a5b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa92289f640 1 -- 172.21.15.103:0/888597770 >> 172.21.15.103:0/888597770 conn(0x7fa91c0fd8c0 msgr2=0x7fa91c0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:08.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.816+0000 7fa92289f640 1 -- 172.21.15.103:0/888597770 shutdown_connections 2024-09-19T06:52:08.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:08.817+0000 7fa92289f640 1 -- 172.21.15.103:0/888597770 wait complete. 2024-09-19T06:52:08.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:08 smithi174 ceph-mon[35637]: pgmap v90: 97 pgs: 97 active+clean; 2.3 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-09-19T06:52:08.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:08 smithi134 ceph-mon[35356]: pgmap v90: 97 pgs: 97 active+clean; 2.3 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-09-19T06:52:09.593 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:52:09.840 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:10.226 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:10 smithi103 ceph-mon[31158]: pgmap v91: 97 pgs: 97 active+clean; 2.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 KiB/s wr, 37 op/s 2024-09-19T06:52:10.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:10 smithi174 ceph-mon[35637]: pgmap v91: 97 pgs: 97 active+clean; 2.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 KiB/s wr, 37 op/s 2024-09-19T06:52:10.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.306+0000 7f1df7d5e640 1 -- 172.21.15.103:0/446557352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df010c180 msgr2=0x7f1df010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.306+0000 7f1df7d5e640 1 --2- 172.21.15.103:0/446557352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df010c180 0x7f1df010e600 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009ca0 tx=0x7f1de802a950 comp rx=0 tx=0).stop 2024-09-19T06:52:10.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.306+0000 7f1df7d5e640 1 -- 172.21.15.103:0/446557352 shutdown_connections 2024-09-19T06:52:10.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.306+0000 7f1df7d5e640 1 --2- 172.21.15.103:0/446557352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df010c180 0x7f1df010e600 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.306+0000 7f1df7d5e640 1 --2- 172.21.15.103:0/446557352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df01036c0 0x7f1df010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.306+0000 7f1df7d5e640 1 --2- 172.21.15.103:0/446557352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1df0102da0 0x7f1df0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.306+0000 7f1df7d5e640 1 -- 172.21.15.103:0/446557352 >> 172.21.15.103:0/446557352 conn(0x7f1df00fc7a0 msgr2=0x7f1df00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:10.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.306+0000 7f1df7d5e640 1 -- 172.21.15.103:0/446557352 shutdown_connections 2024-09-19T06:52:10.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.306+0000 7f1df7d5e640 1 -- 172.21.15.103:0/446557352 wait complete. 2024-09-19T06:52:10.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df7d5e640 1 Processor -- start 2024-09-19T06:52:10.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df7d5e640 1 -- start start 2024-09-19T06:52:10.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df7d5e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1df0102da0 0x7f1df0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df7d5e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df01036c0 0x7f1df0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df7d5e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df010c180 0x7f1df019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df7d5e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1df0111fa0 con 0x7f1df010c180 2024-09-19T06:52:10.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df7d5e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1df0111e20 con 0x7f1df01036c0 2024-09-19T06:52:10.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df7d5e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1df0112120 con 0x7f1df0102da0 2024-09-19T06:52:10.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df62fb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df010c180 0x7f1df019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df62fb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df010c180 0x7f1df019cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53418/0 (socket says 172.21.15.103:53418) 2024-09-19T06:52:10.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df62fb640 1 -- 172.21.15.103:0/2797932682 learned_addr learned my addr 172.21.15.103:0/2797932682 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:10.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df52f9640 1 --2- 172.21.15.103:0/2797932682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df01036c0 0x7f1df0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df62fb640 1 -- 172.21.15.103:0/2797932682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1df0102da0 msgr2=0x7f1df0198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df5afa640 1 --2- 172.21.15.103:0/2797932682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1df0102da0 0x7f1df0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df62fb640 1 --2- 172.21.15.103:0/2797932682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1df0102da0 0x7f1df0198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df62fb640 1 -- 172.21.15.103:0/2797932682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df01036c0 msgr2=0x7f1df0198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df62fb640 1 --2- 172.21.15.103:0/2797932682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df01036c0 0x7f1df0198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df62fb640 1 -- 172.21.15.103:0/2797932682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df019d3f0 con 0x7f1df010c180 2024-09-19T06:52:10.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df52f9640 1 --2- 172.21.15.103:0/2797932682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df01036c0 0x7f1df0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:10.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.307+0000 7f1df5afa640 1 --2- 172.21.15.103:0/2797932682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1df0102da0 0x7f1df0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:10.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.308+0000 7f1df62fb640 1 --2- 172.21.15.103:0/2797932682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df010c180 0x7f1df019cd10 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1de802ae70 tx=0x7f1de802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:10.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.308+0000 7f1de6ffd640 1 -- 172.21.15.103:0/2797932682 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8034040 con 0x7f1df010c180 2024-09-19T06:52:10.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.308+0000 7f1de6ffd640 1 -- 172.21.15.103:0/2797932682 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1de8030920 con 0x7f1df010c180 2024-09-19T06:52:10.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.308+0000 7f1df7d5e640 1 -- 172.21.15.103:0/2797932682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df019d680 con 0x7f1df010c180 2024-09-19T06:52:10.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.308+0000 7f1de6ffd640 1 -- 172.21.15.103:0/2797932682 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8045480 con 0x7f1df010c180 2024-09-19T06:52:10.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.308+0000 7f1df7d5e640 1 -- 172.21.15.103:0/2797932682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df01a4fc0 con 0x7f1df010c180 2024-09-19T06:52:10.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.310+0000 7f1de6ffd640 1 -- 172.21.15.103:0/2797932682 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1de8037020 con 0x7f1df010c180 2024-09-19T06:52:10.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.310+0000 7f1dfa137740 1 Processor -- start 2024-09-19T06:52:10.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1dfa137740 1 -- start start 2024-09-19T06:52:10.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1dfa137740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55090b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1dfa137740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a55090b5d0 0x55a55090ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1dfa137740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a550916370 0x55a550918760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1dfa137740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a5508712f0 con 0x55a55090cb40 2024-09-19T06:52:10.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1dfa137740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a550818cf0 con 0x55a55090b5d0 2024-09-19T06:52:10.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1dfa137740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a55086d2b0 con 0x55a550916370 2024-09-19T06:52:10.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1df52f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55090b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1de6ffd640 1 --2- 172.21.15.103:0/2797932682 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1ddc0a3430 0x7f1ddc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1df52f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55090b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53420/0 (socket says 172.21.15.103:53420) 2024-09-19T06:52:10.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1df52f9640 1 -- 172.21.15.103:0/33801671 learned_addr learned my addr 172.21.15.103:0/33801671 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:10.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1df62fb640 1 --2- 172.21.15.103:0/2797932682 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1ddc0a3430 0x7f1ddc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1df5afa640 1 --2- 172.21.15.103:0/33801671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a55090b5d0 0x55a55090ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1df52f9640 1 -- 172.21.15.103:0/33801671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a550916370 msgr2=0x55a550918760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:10.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1df52f9640 1 --2- 172.21.15.103:0/33801671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a550916370 0x55a550918760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1df52f9640 1 -- 172.21.15.103:0/33801671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a55090b5d0 msgr2=0x55a55090ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:10.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1df52f9640 1 --2- 172.21.15.103:0/33801671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a55090b5d0 0x55a55090ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1df52f9640 1 -- 172.21.15.103:0/33801671 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5507fd3a0 con 0x55a55090cb40 2024-09-19T06:52:10.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.311+0000 7f1de6ffd640 1 -- 172.21.15.103:0/2797932682 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1de80f0c30 con 0x7f1df010c180 2024-09-19T06:52:10.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.312+0000 7f1df52f9640 1 --2- 172.21.15.103:0/33801671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55090b090 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1de00058c0 tx=0x7f1de0011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844e45114bf9fd2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:10.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.312+0000 7f1ddbfff640 1 -- 172.21.15.103:0/33801671 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0011e60 con 0x55a55090cb40 2024-09-19T06:52:10.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.312+0000 7f1df62fb640 1 --2- 172.21.15.103:0/2797932682 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1ddc0a3430 0x7f1ddc0a58f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1de80bccf0 tx=0x7f1de808bce0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:10.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.312+0000 7f1ddbfff640 1 -- 172.21.15.103:0/33801671 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1de001aae0 con 0x55a55090cb40 2024-09-19T06:52:10.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.312+0000 7f1ddbfff640 1 -- 172.21.15.103:0/33801671 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0012b60 con 0x55a55090cb40 2024-09-19T06:52:10.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.313+0000 7f1dfa137740 1 -- 172.21.15.103:0/33801671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 msgr2=0x55a55090b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.313+0000 7f1dfa137740 1 --2- 172.21.15.103:0/33801671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55090b090 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1de00058c0 tx=0x7f1de0011750 comp rx=0 tx=0).stop 2024-09-19T06:52:10.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.313+0000 7f1dfa137740 1 -- 172.21.15.103:0/33801671 shutdown_connections 2024-09-19T06:52:10.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.313+0000 7f1dfa137740 1 --2- 172.21.15.103:0/33801671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a550916370 0x55a550918760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.313+0000 7f1dfa137740 1 --2- 172.21.15.103:0/33801671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a55090b5d0 0x55a55090ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.313+0000 7f1dfa137740 1 --2- 172.21.15.103:0/33801671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55090b090 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.313+0000 7f1dfa137740 1 -- 172.21.15.103:0/33801671 >> 172.21.15.103:0/33801671 conn(0x55a5508d4310 msgr2=0x55a550907c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:10.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.313+0000 7f1dfa137740 1 -- 172.21.15.103:0/33801671 shutdown_connections 2024-09-19T06:52:10.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.313+0000 7f1dfa137740 1 -- 172.21.15.103:0/33801671 wait complete. 2024-09-19T06:52:10.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.313+0000 7f1dfa137740 1 -- start start 2024-09-19T06:52:10.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.313+0000 7f1dfa137740 1 Processor -- start 2024-09-19T06:52:10.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.313+0000 7f1dfa137740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090b5d0 0x55a550915de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.313+0000 7f1dfa137740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090cb40 0x55a550913090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.313+0000 7f1dfa137740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a550916370 0x55a5509137d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1dfa137740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a550919070 con 0x55a55090b5d0 2024-09-19T06:52:10.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1dfa137740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a55082c060 con 0x55a550916370 2024-09-19T06:52:10.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1dfa137740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a5509191f0 con 0x55a55090cb40 2024-09-19T06:52:10.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1dfa137740 10 client.0 _ll_register_callbacks cb 0x55a55057f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:10.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1df52f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090b5d0 0x55a550915de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1df52f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090b5d0 0x55a550915de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53430/0 (socket says 172.21.15.103:53430) 2024-09-19T06:52:10.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1df52f9640 1 -- 172.21.15.103:0/4140801599 learned_addr learned my addr 172.21.15.103:0/4140801599 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:10.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1df52f9640 1 --2- 172.21.15.103:0/4140801599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a550916370 0x55a5509137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1df5afa640 1 --2- 172.21.15.103:0/4140801599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090cb40 0x55a550913090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1df52f9640 1 -- 172.21.15.103:0/4140801599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090cb40 msgr2=0x55a550913090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:10.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1df52f9640 1 --2- 172.21.15.103:0/4140801599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090cb40 0x55a550913090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1df52f9640 1 -- 172.21.15.103:0/4140801599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a550916370 msgr2=0x55a5509137d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:10.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1df52f9640 1 --2- 172.21.15.103:0/4140801599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a550916370 0x55a5509137d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1df52f9640 1 -- 172.21.15.103:0/4140801599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5505e23c0 con 0x55a55090b5d0 2024-09-19T06:52:10.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1df5afa640 1 --2- 172.21.15.103:0/4140801599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090cb40 0x55a550913090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:10.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1df52f9640 1 --2- 172.21.15.103:0/4140801599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090b5d0 0x55a550915de0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1de000b700 tx=0x7f1de000bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6ca1ec58449cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:10.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1dd97fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:10.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1dd97fa640 1 -- 172.21.15.103:0/4140801599 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de001a400 con 0x55a55090b5d0 2024-09-19T06:52:10.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1dd97fa640 1 -- 172.21.15.103:0/4140801599 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1de001a5a0 con 0x55a55090b5d0 2024-09-19T06:52:10.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1dfa137740 1 -- 172.21.15.103:0/4140801599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1de001aae0 con 0x55a55090b5d0 2024-09-19T06:52:10.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1dd97fa640 1 -- 172.21.15.103:0/4140801599 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1de001a7a0 con 0x55a55090b5d0 2024-09-19T06:52:10.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1dfa137740 10 client.16506 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:10.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.314+0000 7f1dfa137740 1 -- 172.21.15.103:0/4140801599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a550924ba0 con 0x55a55090b5d0 2024-09-19T06:52:10.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.315+0000 7f1dd97fa640 1 -- 172.21.15.103:0/4140801599 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1de001aae0 con 0x55a55090b5d0 2024-09-19T06:52:10.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.315+0000 7f1dd97fa640 1 -- 172.21.15.103:0/4140801599 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1de0028070 con 0x55a55090b5d0 2024-09-19T06:52:10.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.315+0000 7f1dfa137740 10 client.16506 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:10.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.315+0000 7f1dfa137740 10 client.16506 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:52:10.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.315+0000 7f1dfa137740 20 client.16506 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:10.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.315+0000 7f1dfa137740 1 --2- 172.21.15.103:0/4140801599 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55a550925ef0 0x55a5509282e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.315+0000 7f1dfa137740 4 client.16506 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:10.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.315+0000 7f1dfa137740 1 -- 172.21.15.103:0/4140801599 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a55087e860 con 0x55a550925ef0 2024-09-19T06:52:10.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.315+0000 7f1df5afa640 1 --2- 172.21.15.103:0/4140801599 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55a550925ef0 0x55a5509282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.316+0000 7f1df5afa640 1 --2- 172.21.15.103:0/4140801599 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55a550925ef0 0x55a5509282e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5e4a2262b10800ef server_cookie=4168060ca32b6ecd in_seq=0 out_seq=0 2024-09-19T06:52:10.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.316+0000 7f1dd97fa640 10 client.16506 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:52:10.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.316+0000 7f1dd97fa640 1 -- 172.21.15.103:0/4140801599 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a55087e860 con 0x55a550925ef0 2024-09-19T06:52:10.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.316+0000 7f1dd97fa640 10 client.16506 handle_command_reply: tid=0 2024-09-19T06:52:10.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.316+0000 7f1dfa137740 1 client.16506 shutdown 2024-09-19T06:52:10.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.316+0000 7f1dfa137740 1 -- 172.21.15.103:0/4140801599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090b5d0 msgr2=0x55a550915de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.316+0000 7f1dfa137740 1 --2- 172.21.15.103:0/4140801599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090b5d0 0x55a550915de0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1de000b700 tx=0x7f1de000bfb0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.317+0000 7f1dfa137740 1 -- 172.21.15.103:0/4140801599 shutdown_connections 2024-09-19T06:52:10.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.317+0000 7f1dfa137740 1 --2- 172.21.15.103:0/4140801599 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55a550925ef0 0x55a5509282e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.317+0000 7f1dfa137740 1 --2- 172.21.15.103:0/4140801599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a550916370 0x55a5509137d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.317+0000 7f1dfa137740 1 --2- 172.21.15.103:0/4140801599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090cb40 0x55a550913090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.317+0000 7f1dd97fa640 0 client.16506 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:52:10.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.317+0000 7f1dfa137740 1 --2- 172.21.15.103:0/4140801599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090b5d0 0x55a550915de0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.317+0000 7f1dfa137740 1 -- 172.21.15.103:0/4140801599 >> 172.21.15.103:0/4140801599 conn(0x55a5508d4310 msgr2=0x55a55090d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:10.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.317+0000 7f1dfa137740 1 -- 172.21.15.103:0/4140801599 shutdown_connections 2024-09-19T06:52:10.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.317+0000 7f1dfa137740 1 -- 172.21.15.103:0/4140801599 wait complete. 2024-09-19T06:52:10.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.317+0000 7f1dfa137740 20 client.16506 trim_cache size 0 max 16384 2024-09-19T06:52:10.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.329+0000 7f1dfa137740 1 Processor -- start 2024-09-19T06:52:10.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.329+0000 7f1dfa137740 1 -- start start 2024-09-19T06:52:10.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1dfa137740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090b5d0 0x55a550922310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1dfa137740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55090e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1dfa137740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a550916370 0x55a55090e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1dfa137740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1df01ad820 con 0x55a55090cb40 2024-09-19T06:52:10.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1dfa137740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1df01af360 con 0x55a550916370 2024-09-19T06:52:10.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1dfa137740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1df01adef0 con 0x55a55090b5d0 2024-09-19T06:52:10.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1df5afa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55090e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1df5afa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55090e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53438/0 (socket says 172.21.15.103:53438) 2024-09-19T06:52:10.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1df5afa640 1 -- 172.21.15.103:0/2820206956 learned_addr learned my addr 172.21.15.103:0/2820206956 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:10.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1df52f9640 1 --2- 172.21.15.103:0/2820206956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090b5d0 0x55a550922310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1df5afa640 1 -- 172.21.15.103:0/2820206956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090b5d0 msgr2=0x55a550922310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:10.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1df5afa640 1 --2- 172.21.15.103:0/2820206956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090b5d0 0x55a550922310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1df5afa640 1 -- 172.21.15.103:0/2820206956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a550916370 msgr2=0x55a55090e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:10.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1df52f9640 1 --2- 172.21.15.103:0/2820206956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a550916370 0x55a55090e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1df5afa640 1 --2- 172.21.15.103:0/2820206956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a550916370 0x55a55090e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1df5afa640 1 -- 172.21.15.103:0/2820206956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a550928650 con 0x55a55090cb40 2024-09-19T06:52:10.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1df52f9640 1 --2- 172.21.15.103:0/2820206956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a550916370 0x55a55090e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:10.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1df5afa640 1 --2- 172.21.15.103:0/2820206956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55090e120 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00ab10 tx=0x7f1dec00a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98abaa7df9e02a91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:10.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1ddb7fe640 1 -- 172.21.15.103:0/2820206956 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec00adc0 con 0x55a55090cb40 2024-09-19T06:52:10.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1ddb7fe640 1 -- 172.21.15.103:0/2820206956 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dec022070 con 0x55a55090cb40 2024-09-19T06:52:10.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.330+0000 7f1ddb7fe640 1 -- 172.21.15.103:0/2820206956 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec01dd70 con 0x55a55090cb40 2024-09-19T06:52:10.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.331+0000 7f1dfa137740 1 -- 172.21.15.103:0/2820206956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 msgr2=0x55a55090e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.331+0000 7f1dfa137740 1 --2- 172.21.15.103:0/2820206956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55090e120 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00ab10 tx=0x7f1dec00a840 comp rx=0 tx=0).stop 2024-09-19T06:52:10.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.331+0000 7f1dfa137740 1 -- 172.21.15.103:0/2820206956 shutdown_connections 2024-09-19T06:52:10.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.331+0000 7f1dfa137740 1 --2- 172.21.15.103:0/2820206956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a550916370 0x55a55090e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.331+0000 7f1dfa137740 1 --2- 172.21.15.103:0/2820206956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55090e120 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.331+0000 7f1dfa137740 1 --2- 172.21.15.103:0/2820206956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090b5d0 0x55a550922310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.331+0000 7f1dfa137740 1 -- 172.21.15.103:0/2820206956 >> 172.21.15.103:0/2820206956 conn(0x55a5508d4310 msgr2=0x55a550912050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:10.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.331+0000 7f1dfa137740 1 -- 172.21.15.103:0/2820206956 shutdown_connections 2024-09-19T06:52:10.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.331+0000 7f1dfa137740 1 -- 172.21.15.103:0/2820206956 wait complete. 2024-09-19T06:52:10.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.331+0000 7f1dfa137740 1 -- start start 2024-09-19T06:52:10.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.331+0000 7f1dfa137740 1 Processor -- start 2024-09-19T06:52:10.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1dfa137740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090b5d0 0x55a55091ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1dfa137740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55091a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1dfa137740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a550916370 0x55a55091a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1dfa137740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1df01aec90 con 0x55a55090cb40 2024-09-19T06:52:10.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1dfa137740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1df01ae5c0 con 0x55a550916370 2024-09-19T06:52:10.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1dfa137740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1df01ab7e0 con 0x55a55090b5d0 2024-09-19T06:52:10.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1dfa137740 10 client.0 _ll_register_callbacks cb 0x7f1de000bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:10.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1df5afa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55091a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1df5afa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55091a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53452/0 (socket says 172.21.15.103:53452) 2024-09-19T06:52:10.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1df5afa640 1 -- 172.21.15.103:0/239276845 learned_addr learned my addr 172.21.15.103:0/239276845 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:10.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1df52f9640 1 --2- 172.21.15.103:0/239276845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a550916370 0x55a55091a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1df52f9640 1 --2- 172.21.15.103:0/239276845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090b5d0 0x55a55091ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1df5afa640 1 -- 172.21.15.103:0/239276845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090b5d0 msgr2=0x55a55091ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:10.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1df5afa640 1 --2- 172.21.15.103:0/239276845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090b5d0 0x55a55091ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1df5afa640 1 -- 172.21.15.103:0/239276845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a550916370 msgr2=0x55a55091a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:10.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1df5afa640 1 --2- 172.21.15.103:0/239276845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a550916370 0x55a55091a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1df5afa640 1 -- 172.21.15.103:0/239276845 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a55098fa70 con 0x55a55090cb40 2024-09-19T06:52:10.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1df52f9640 1 --2- 172.21.15.103:0/239276845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090b5d0 0x55a55091ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:10.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.332+0000 7f1df5afa640 1 --2- 172.21.15.103:0/239276845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55091a080 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1dec008010 tx=0x7f1dec01d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc63673dd2ae4d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:10.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.333+0000 7f1dd97fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:10.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.333+0000 7f1dd97fa640 1 -- 172.21.15.103:0/239276845 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec01ddc0 con 0x55a55090cb40 2024-09-19T06:52:10.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.333+0000 7f1dd97fa640 1 -- 172.21.15.103:0/239276845 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1dec016960 con 0x55a55090cb40 2024-09-19T06:52:10.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.333+0000 7f1dfa137740 1 -- 172.21.15.103:0/239276845 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dec022070 con 0x55a55090cb40 2024-09-19T06:52:10.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.333+0000 7f1dd97fa640 1 -- 172.21.15.103:0/239276845 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1dec016b60 con 0x55a55090cb40 2024-09-19T06:52:10.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.333+0000 7f1dfa137740 10 client.16512 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:10.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.333+0000 7f1dfa137740 1 -- 172.21.15.103:0/239276845 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a550990b70 con 0x55a55090cb40 2024-09-19T06:52:10.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.334+0000 7f1dd97fa640 1 -- 172.21.15.103:0/239276845 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dec022070 con 0x55a55090cb40 2024-09-19T06:52:10.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.334+0000 7f1dd97fa640 1 -- 172.21.15.103:0/239276845 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1dec0303f0 con 0x55a55090cb40 2024-09-19T06:52:10.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.334+0000 7f1dfa137740 10 client.16512 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:10.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.334+0000 7f1dfa137740 10 client.16512 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:52:10.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.334+0000 7f1dfa137740 20 client.16512 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:10.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.334+0000 7f1dfa137740 1 --2- 172.21.15.103:0/239276845 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55a550991240 0x55a550993660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.334+0000 7f1dfa137740 4 client.16512 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:52:10.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.334+0000 7f1dfa137740 1 -- 172.21.15.103:0/239276845 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a550878d70 con 0x55a550991240 2024-09-19T06:52:10.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.334+0000 7f1df52f9640 1 --2- 172.21.15.103:0/239276845 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55a550991240 0x55a550993660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1df52f9640 1 --2- 172.21.15.103:0/239276845 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55a550991240 0x55a550993660 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4e804bb657d59d60 server_cookie=8b083fdc6912a565 in_seq=0 out_seq=0 2024-09-19T06:52:10.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1dd97fa640 10 client.16512 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:52:10.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1dd97fa640 1 -- 172.21.15.103:0/239276845 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a550878d70 con 0x55a550991240 2024-09-19T06:52:10.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1dd97fa640 10 client.16512 handle_command_reply: tid=0 2024-09-19T06:52:10.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1dfa137740 1 client.16512 shutdown 2024-09-19T06:52:10.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1dfa137740 1 -- 172.21.15.103:0/239276845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 msgr2=0x55a55091a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1dfa137740 1 --2- 172.21.15.103:0/239276845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55091a080 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1dec008010 tx=0x7f1dec01d560 comp rx=0 tx=0).stop 2024-09-19T06:52:10.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1dfa137740 1 -- 172.21.15.103:0/239276845 shutdown_connections 2024-09-19T06:52:10.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1dfa137740 1 --2- 172.21.15.103:0/239276845 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55a550991240 0x55a550993660 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1dfa137740 1 --2- 172.21.15.103:0/239276845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a550916370 0x55a55091a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1dfa137740 1 --2- 172.21.15.103:0/239276845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a55090cb40 0x55a55091a080 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1dd97fa640 0 client.16512 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:52:10.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1dfa137740 1 --2- 172.21.15.103:0/239276845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a55090b5d0 0x55a55091ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1dfa137740 1 -- 172.21.15.103:0/239276845 >> 172.21.15.103:0/239276845 conn(0x55a5508d4310 msgr2=0x55a550921300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:10.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1dfa137740 1 -- 172.21.15.103:0/239276845 shutdown_connections 2024-09-19T06:52:10.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1dfa137740 1 -- 172.21.15.103:0/239276845 wait complete. 2024-09-19T06:52:10.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.335+0000 7f1dfa137740 20 client.16512 trim_cache size 0 max 16384 2024-09-19T06:52:10.354 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:52:10.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.336+0000 7f1df7d5e640 1 -- 172.21.15.103:0/2797932682 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1ddc0a3430 msgr2=0x7f1ddc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.336+0000 7f1df7d5e640 1 --2- 172.21.15.103:0/2797932682 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1ddc0a3430 0x7f1ddc0a58f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1de80bccf0 tx=0x7f1de808bce0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.336+0000 7f1df7d5e640 1 -- 172.21.15.103:0/2797932682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df010c180 msgr2=0x7f1df019cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.336+0000 7f1df7d5e640 1 --2- 172.21.15.103:0/2797932682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df010c180 0x7f1df019cd10 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1de802ae70 tx=0x7f1de802f5b0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.336+0000 7f1df7d5e640 1 -- 172.21.15.103:0/2797932682 shutdown_connections 2024-09-19T06:52:10.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.336+0000 7f1df7d5e640 1 --2- 172.21.15.103:0/2797932682 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1ddc0a3430 0x7f1ddc0a58f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.336+0000 7f1df7d5e640 1 --2- 172.21.15.103:0/2797932682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df010c180 0x7f1df019cd10 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.336+0000 7f1df7d5e640 1 --2- 172.21.15.103:0/2797932682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df01036c0 0x7f1df0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.336+0000 7f1df7d5e640 1 --2- 172.21.15.103:0/2797932682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1df0102da0 0x7f1df0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.336+0000 7f1df7d5e640 1 -- 172.21.15.103:0/2797932682 >> 172.21.15.103:0/2797932682 conn(0x7f1df00fc7a0 msgr2=0x7f1df00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:10.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.336+0000 7f1df7d5e640 1 -- 172.21.15.103:0/2797932682 shutdown_connections 2024-09-19T06:52:10.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:10.336+0000 7f1df7d5e640 1 -- 172.21.15.103:0/2797932682 wait complete. 2024-09-19T06:52:10.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:10 smithi134 ceph-mon[35356]: pgmap v91: 97 pgs: 97 active+clean; 2.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 KiB/s wr, 37 op/s 2024-09-19T06:52:11.110 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:52:11.359 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:11.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.783+0000 7f72c1970640 1 -- 172.21.15.103:0/1081849351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72bc1036e0 msgr2=0x7f72bc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:11.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.783+0000 7f72c1970640 1 --2- 172.21.15.103:0/1081849351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72bc1036e0 0x7f72bc107b90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009930 tx=0x7f72a402a420 comp rx=0 tx=0).stop 2024-09-19T06:52:11.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.783+0000 7f72c1970640 1 -- 172.21.15.103:0/1081849351 shutdown_connections 2024-09-19T06:52:11.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.783+0000 7f72c1970640 1 --2- 172.21.15.103:0/1081849351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72bc1082a0 0x7f72bc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.783+0000 7f72c1970640 1 --2- 172.21.15.103:0/1081849351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72bc1036e0 0x7f72bc107b90 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.783+0000 7f72c1970640 1 --2- 172.21.15.103:0/1081849351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72bc102d30 0x7f72bc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.783+0000 7f72c1970640 1 -- 172.21.15.103:0/1081849351 >> 172.21.15.103:0/1081849351 conn(0x7f72bc0fc7d0 msgr2=0x7f72bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:11.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.783+0000 7f72c1970640 1 -- 172.21.15.103:0/1081849351 shutdown_connections 2024-09-19T06:52:11.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.783+0000 7f72c1970640 1 -- 172.21.15.103:0/1081849351 wait complete. 2024-09-19T06:52:11.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.783+0000 7f72c1970640 1 Processor -- start 2024-09-19T06:52:11.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.783+0000 7f72c1970640 1 -- start start 2024-09-19T06:52:11.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72c1970640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72bc102d30 0x7f72bc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72c1970640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72bc1036e0 0x7f72bc198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72c1970640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72bc1082a0 0x7f72bc19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72c1970640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc112140 con 0x7f72bc102d30 2024-09-19T06:52:11.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72c1970640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc111fc0 con 0x7f72bc1082a0 2024-09-19T06:52:11.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72c1970640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1122c0 con 0x7f72bc1036e0 2024-09-19T06:52:11.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72baffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72bc102d30 0x7f72bc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72baffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72bc102d30 0x7f72bc198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53476/0 (socket says 172.21.15.103:53476) 2024-09-19T06:52:11.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72baffd640 1 -- 172.21.15.103:0/1874300915 learned_addr learned my addr 172.21.15.103:0/1874300915 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:11.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72b27fc640 1 --2- 172.21.15.103:0/1874300915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72bc1036e0 0x7f72bc198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72bb7fe640 1 --2- 172.21.15.103:0/1874300915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72bc1082a0 0x7f72bc19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72baffd640 1 -- 172.21.15.103:0/1874300915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72bc1036e0 msgr2=0x7f72bc198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:11.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72baffd640 1 --2- 172.21.15.103:0/1874300915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72bc1036e0 0x7f72bc198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72baffd640 1 -- 172.21.15.103:0/1874300915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72bc1082a0 msgr2=0x7f72bc19cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:11.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72baffd640 1 --2- 172.21.15.103:0/1874300915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72bc1082a0 0x7f72bc19cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72baffd640 1 -- 172.21.15.103:0/1874300915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72bc19d6b0 con 0x7f72bc102d30 2024-09-19T06:52:11.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72b27fc640 1 --2- 172.21.15.103:0/1874300915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72bc1036e0 0x7f72bc198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:11.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72baffd640 1 --2- 172.21.15.103:0/1874300915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72bc102d30 0x7f72bc198700 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00b4a0 tx=0x7f72ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:11.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72c096e640 1 -- 172.21.15.103:0/1874300915 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac014070 con 0x7f72bc102d30 2024-09-19T06:52:11.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72c096e640 1 -- 172.21.15.103:0/1874300915 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ac010af0 con 0x7f72bc102d30 2024-09-19T06:52:11.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72c1970640 1 -- 172.21.15.103:0/1874300915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72bc19d9a0 con 0x7f72bc102d30 2024-09-19T06:52:11.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.784+0000 7f72c096e640 1 -- 172.21.15.103:0/1874300915 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac017d10 con 0x7f72bc102d30 2024-09-19T06:52:11.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.785+0000 7f72c1970640 1 -- 172.21.15.103:0/1874300915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72bc1a5280 con 0x7f72bc102d30 2024-09-19T06:52:11.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.787+0000 7f72c096e640 1 -- 172.21.15.103:0/1874300915 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f72ac017070 con 0x7f72bc102d30 2024-09-19T06:52:11.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.787+0000 7f72c3d49740 1 Processor -- start 2024-09-19T06:52:11.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.787+0000 7f72c3d49740 1 -- start start 2024-09-19T06:52:11.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.787+0000 7f72c3d49740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f81b40 0x561c00f80090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.787+0000 7f72c3d49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f805d0 0x561c00f80a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.787+0000 7f72c3d49740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f8b370 0x561c00f8d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.787+0000 7f72c3d49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561c00ee62f0 con 0x561c00f805d0 2024-09-19T06:52:11.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.787+0000 7f72c3d49740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561c00e8dcf0 con 0x561c00f8b370 2024-09-19T06:52:11.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.787+0000 7f72c3d49740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c00ee22b0 con 0x561c00f81b40 2024-09-19T06:52:11.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.787+0000 7f72baffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f805d0 0x561c00f80a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.788+0000 7f72baffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f805d0 0x561c00f80a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53492/0 (socket says 172.21.15.103:53492) 2024-09-19T06:52:11.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.788+0000 7f72baffd640 1 -- 172.21.15.103:0/135671266 learned_addr learned my addr 172.21.15.103:0/135671266 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:11.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.788+0000 7f72bb7fe640 1 --2- 172.21.15.103:0/135671266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f81b40 0x561c00f80090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.788+0000 7f72b27fc640 1 --2- 172.21.15.103:0/135671266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f8b370 0x561c00f8d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.788+0000 7f72baffd640 1 -- 172.21.15.103:0/135671266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f81b40 msgr2=0x561c00f80090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:11.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.788+0000 7f72baffd640 1 --2- 172.21.15.103:0/135671266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f81b40 0x561c00f80090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.788+0000 7f72baffd640 1 -- 172.21.15.103:0/135671266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f8b370 msgr2=0x561c00f8d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:11.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.788+0000 7f72baffd640 1 --2- 172.21.15.103:0/135671266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f8b370 0x561c00f8d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.788+0000 7f72baffd640 1 -- 172.21.15.103:0/135671266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c00e723a0 con 0x561c00f805d0 2024-09-19T06:52:11.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.788+0000 7f72c096e640 1 --2- 172.21.15.103:0/1874300915 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f72a80a3430 0x7f72a80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.788+0000 7f72baffd640 1 --2- 172.21.15.103:0/135671266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f805d0 0x561c00f80a10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0cc750 tx=0x7f72ac0d2f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262fe2868ac2f8d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:11.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.788+0000 7f72bb7fe640 1 --2- 172.21.15.103:0/1874300915 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f72a80a3430 0x7f72a80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.788+0000 7f72c096e640 1 -- 172.21.15.103:0/1874300915 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f72ac0cefe0 con 0x7f72bc102d30 2024-09-19T06:52:11.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.788+0000 7f72b1ffb640 1 -- 172.21.15.103:0/135671266 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac097640 con 0x561c00f805d0 2024-09-19T06:52:11.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.788+0000 7f72b1ffb640 1 -- 172.21.15.103:0/135671266 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ac09baa0 con 0x561c00f805d0 2024-09-19T06:52:11.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.788+0000 7f72b1ffb640 1 -- 172.21.15.103:0/135671266 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac09ad00 con 0x561c00f805d0 2024-09-19T06:52:11.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.789+0000 7f72c3d49740 1 -- 172.21.15.103:0/135671266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f805d0 msgr2=0x561c00f80a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:11.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.789+0000 7f72bb7fe640 1 --2- 172.21.15.103:0/1874300915 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f72a80a3430 0x7f72a80a58f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f72b4007ed0 tx=0x7f72b40075c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.789+0000 7f72c3d49740 1 --2- 172.21.15.103:0/135671266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f805d0 0x561c00f80a10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0cc750 tx=0x7f72ac0d2f90 comp rx=0 tx=0).stop 2024-09-19T06:52:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.789+0000 7f72c3d49740 1 -- 172.21.15.103:0/135671266 shutdown_connections 2024-09-19T06:52:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.789+0000 7f72c3d49740 1 --2- 172.21.15.103:0/135671266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f8b370 0x561c00f8d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.789+0000 7f72c3d49740 1 --2- 172.21.15.103:0/135671266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f805d0 0x561c00f80a10 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.789+0000 7f72c3d49740 1 --2- 172.21.15.103:0/135671266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f81b40 0x561c00f80090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.789+0000 7f72c3d49740 1 -- 172.21.15.103:0/135671266 >> 172.21.15.103:0/135671266 conn(0x561c00f49310 msgr2=0x561c00f7cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.789+0000 7f72c3d49740 1 -- 172.21.15.103:0/135671266 shutdown_connections 2024-09-19T06:52:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.789+0000 7f72c3d49740 1 -- 172.21.15.103:0/135671266 wait complete. 2024-09-19T06:52:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.790+0000 7f72c3d49740 1 -- start start 2024-09-19T06:52:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.790+0000 7f72c3d49740 1 Processor -- start 2024-09-19T06:52:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.790+0000 7f72c3d49740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f805d0 0x561c00f8adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.790+0000 7f72c3d49740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f81b40 0x561c00f88060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.790+0000 7f72c3d49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f8b370 0x561c00f887f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.790+0000 7f72c3d49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561c00f8e060 con 0x561c00f8b370 2024-09-19T06:52:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.790+0000 7f72c3d49740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561c00ea1060 con 0x561c00f805d0 2024-09-19T06:52:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.790+0000 7f72c3d49740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c00f8e1e0 con 0x561c00f81b40 2024-09-19T06:52:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.790+0000 7f72bb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f8b370 0x561c00f887f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.790+0000 7f72bb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f8b370 0x561c00f887f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53506/0 (socket says 172.21.15.103:53506) 2024-09-19T06:52:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.790+0000 7f72c3d49740 10 client.0 _ll_register_callbacks cb 0x561c00bf48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.790+0000 7f72bb7fe640 1 -- 172.21.15.103:0/2463866858 learned_addr learned my addr 172.21.15.103:0/2463866858 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.790+0000 7f72baffd640 1 --2- 172.21.15.103:0/2463866858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f805d0 0x561c00f8adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.790+0000 7f72b27fc640 1 --2- 172.21.15.103:0/2463866858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f81b40 0x561c00f88060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.791+0000 7f72bb7fe640 1 -- 172.21.15.103:0/2463866858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f81b40 msgr2=0x561c00f88060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.791+0000 7f72bb7fe640 1 --2- 172.21.15.103:0/2463866858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f81b40 0x561c00f88060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.791+0000 7f72bb7fe640 1 -- 172.21.15.103:0/2463866858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f805d0 msgr2=0x561c00f8adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.791+0000 7f72bb7fe640 1 --2- 172.21.15.103:0/2463866858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f805d0 0x561c00f8adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.791+0000 7f72bb7fe640 1 -- 172.21.15.103:0/2463866858 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c00c573c0 con 0x561c00f8b370 2024-09-19T06:52:11.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.791+0000 7f72b27fc640 1 --2- 172.21.15.103:0/2463866858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f81b40 0x561c00f88060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:11.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.791+0000 7f72bb7fe640 1 --2- 172.21.15.103:0/2463866858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f8b370 0x561c00f887f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f72b4012e00 tx=0x7f72b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d364439369b91ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:11.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.791+0000 7f729affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:11.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.791+0000 7f729affd640 1 -- 172.21.15.103:0/2463866858 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4023020 con 0x561c00f8b370 2024-09-19T06:52:11.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.791+0000 7f729affd640 1 -- 172.21.15.103:0/2463866858 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f72b401f440 con 0x561c00f8b370 2024-09-19T06:52:11.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.791+0000 7f72c3d49740 1 -- 172.21.15.103:0/2463866858 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72ac09baa0 con 0x561c00f8b370 2024-09-19T06:52:11.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.791+0000 7f729affd640 1 -- 172.21.15.103:0/2463866858 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f72b402e050 con 0x561c00f8b370 2024-09-19T06:52:11.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.791+0000 7f72c3d49740 10 client.16524 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:11.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.791+0000 7f72c3d49740 1 -- 172.21.15.103:0/2463866858 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c00f99bb0 con 0x561c00f8b370 2024-09-19T06:52:11.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.791+0000 7f729affd640 1 -- 172.21.15.103:0/2463866858 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ac09baa0 con 0x561c00f8b370 2024-09-19T06:52:11.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.791+0000 7f729affd640 1 -- 172.21.15.103:0/2463866858 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f72b4011040 con 0x561c00f8b370 2024-09-19T06:52:11.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.792+0000 7f72c3d49740 10 client.16524 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.792+0000 7f72c3d49740 10 client.16524 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:52:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.792+0000 7f72c3d49740 20 client.16524 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.792+0000 7f72c3d49740 1 --2- 172.21.15.103:0/2463866858 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561c00f9b070 0x561c00f9d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.792+0000 7f72c3d49740 4 client.16524 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.792+0000 7f72c3d49740 1 -- 172.21.15.103:0/2463866858 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c00ef3860 con 0x561c00f9b070 2024-09-19T06:52:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.792+0000 7f72baffd640 1 --2- 172.21.15.103:0/2463866858 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561c00f9b070 0x561c00f9d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.792+0000 7f72baffd640 1 --2- 172.21.15.103:0/2463866858 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561c00f9b070 0x561c00f9d4f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45d9e1eb2f97dd2d server_cookie=367d2024c79ef4e3 in_seq=0 out_seq=0 2024-09-19T06:52:11.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.792+0000 7f729affd640 10 client.16524 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:52:11.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.793+0000 7f729affd640 1 -- 172.21.15.103:0/2463866858 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c00ef3860 con 0x561c00f9b070 2024-09-19T06:52:11.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.793+0000 7f729affd640 10 client.16524 handle_command_reply: tid=0 2024-09-19T06:52:11.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.793+0000 7f72c3d49740 1 client.16524 shutdown 2024-09-19T06:52:11.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.793+0000 7f72c3d49740 1 -- 172.21.15.103:0/2463866858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f8b370 msgr2=0x561c00f887f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:11.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.793+0000 7f72c3d49740 1 --2- 172.21.15.103:0/2463866858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f8b370 0x561c00f887f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f72b4012e00 tx=0x7f72b4004a40 comp rx=0 tx=0).stop 2024-09-19T06:52:11.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.793+0000 7f72c3d49740 1 -- 172.21.15.103:0/2463866858 shutdown_connections 2024-09-19T06:52:11.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.793+0000 7f72c3d49740 1 --2- 172.21.15.103:0/2463866858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f8b370 0x561c00f887f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.793+0000 7f72c3d49740 1 --2- 172.21.15.103:0/2463866858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f81b40 0x561c00f88060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.793+0000 7f72c3d49740 1 --2- 172.21.15.103:0/2463866858 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561c00f9b070 0x561c00f9d4f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.793+0000 7f729affd640 0 client.16524 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:52:11.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.793+0000 7f72c3d49740 1 --2- 172.21.15.103:0/2463866858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f805d0 0x561c00f8adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.793+0000 7f72c3d49740 1 -- 172.21.15.103:0/2463866858 >> 172.21.15.103:0/2463866858 conn(0x561c00f49310 msgr2=0x561c00f82960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:11.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.793+0000 7f72c3d49740 1 -- 172.21.15.103:0/2463866858 shutdown_connections 2024-09-19T06:52:11.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.793+0000 7f72c3d49740 1 -- 172.21.15.103:0/2463866858 wait complete. 2024-09-19T06:52:11.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.793+0000 7f72c3d49740 20 client.16524 trim_cache size 0 max 16384 2024-09-19T06:52:11.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72c3d49740 1 Processor -- start 2024-09-19T06:52:11.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72c3d49740 1 -- start start 2024-09-19T06:52:11.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72c3d49740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f805d0 0x561c00f8adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72c3d49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f81b40 0x561c00f83120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72c3d49740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f8b370 0x561c00f83660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72c3d49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1adaf0 con 0x561c00f81b40 2024-09-19T06:52:11.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72c3d49740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1af630 con 0x561c00f8b370 2024-09-19T06:52:11.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72c3d49740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1ae1c0 con 0x561c00f805d0 2024-09-19T06:52:11.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72b27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f81b40 0x561c00f83120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72b27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f81b40 0x561c00f83120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53512/0 (socket says 172.21.15.103:53512) 2024-09-19T06:52:11.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72b27fc640 1 -- 172.21.15.103:0/1231675380 learned_addr learned my addr 172.21.15.103:0/1231675380 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72baffd640 1 --2- 172.21.15.103:0/1231675380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f805d0 0x561c00f8adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72b27fc640 1 -- 172.21.15.103:0/1231675380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f805d0 msgr2=0x561c00f8adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72b27fc640 1 --2- 172.21.15.103:0/1231675380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f805d0 0x561c00f8adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72bb7fe640 1 --2- 172.21.15.103:0/1231675380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f8b370 0x561c00f83660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72b27fc640 1 -- 172.21.15.103:0/1231675380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f8b370 msgr2=0x561c00f83660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.806+0000 7f72b27fc640 1 --2- 172.21.15.103:0/1231675380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f8b370 0x561c00f83660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.807+0000 7f72b27fc640 1 -- 172.21.15.103:0/1231675380 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c00f8dad0 con 0x561c00f81b40 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.807+0000 7f72bb7fe640 1 --2- 172.21.15.103:0/1231675380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f8b370 0x561c00f83660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.807+0000 7f72b27fc640 1 --2- 172.21.15.103:0/1231675380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f81b40 0x561c00f83120 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f72a4005ab0 tx=0x7f72a4009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40838c91f6f65f54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.807+0000 7f72b17fa640 1 -- 172.21.15.103:0/1231675380 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4035040 con 0x561c00f81b40 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.807+0000 7f72b17fa640 1 -- 172.21.15.103:0/1231675380 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a4031970 con 0x561c00f81b40 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.807+0000 7f72b17fa640 1 -- 172.21.15.103:0/1231675380 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4004920 con 0x561c00f81b40 2024-09-19T06:52:11.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.807+0000 7f72c3d49740 1 -- 172.21.15.103:0/1231675380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f81b40 msgr2=0x561c00f83120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:11.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.807+0000 7f72c3d49740 1 --2- 172.21.15.103:0/1231675380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f81b40 0x561c00f83120 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f72a4005ab0 tx=0x7f72a4009880 comp rx=0 tx=0).stop 2024-09-19T06:52:11.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72c3d49740 1 -- 172.21.15.103:0/1231675380 shutdown_connections 2024-09-19T06:52:11.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72c3d49740 1 --2- 172.21.15.103:0/1231675380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f8b370 0x561c00f83660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72c3d49740 1 --2- 172.21.15.103:0/1231675380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f81b40 0x561c00f83120 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72c3d49740 1 --2- 172.21.15.103:0/1231675380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f805d0 0x561c00f8adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72c3d49740 1 -- 172.21.15.103:0/1231675380 >> 172.21.15.103:0/1231675380 conn(0x561c00f49310 msgr2=0x561c00f77a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:11.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72c3d49740 1 -- 172.21.15.103:0/1231675380 shutdown_connections 2024-09-19T06:52:11.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72c3d49740 1 -- 172.21.15.103:0/1231675380 wait complete. 2024-09-19T06:52:11.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72c3d49740 1 -- start start 2024-09-19T06:52:11.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72c3d49740 1 Processor -- start 2024-09-19T06:52:11.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72c3d49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f805d0 0x561c00f8ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72c3d49740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f81b40 0x561c00f8f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72c3d49740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f8b370 0x561c00ffdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72c3d49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1aef60 con 0x561c00f805d0 2024-09-19T06:52:11.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72c3d49740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1ae890 con 0x561c00f81b40 2024-09-19T06:52:11.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72c3d49740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1abab0 con 0x561c00f8b370 2024-09-19T06:52:11.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72c3d49740 10 client.0 _ll_register_callbacks cb 0x7f72b4007b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:11.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.808+0000 7f72baffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f805d0 0x561c00f8ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f72baffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f805d0 0x561c00f8ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53518/0 (socket says 172.21.15.103:53518) 2024-09-19T06:52:11.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f72baffd640 1 -- 172.21.15.103:0/2170054040 learned_addr learned my addr 172.21.15.103:0/2170054040 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:11.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f72bb7fe640 1 --2- 172.21.15.103:0/2170054040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f8b370 0x561c00ffdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f72b27fc640 1 --2- 172.21.15.103:0/2170054040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f81b40 0x561c00f8f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f72baffd640 1 -- 172.21.15.103:0/2170054040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f8b370 msgr2=0x561c00ffdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:11.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f72baffd640 1 --2- 172.21.15.103:0/2170054040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f8b370 0x561c00ffdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f72baffd640 1 -- 172.21.15.103:0/2170054040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f81b40 msgr2=0x561c00f8f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:11.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f72baffd640 1 --2- 172.21.15.103:0/2170054040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f81b40 0x561c00f8f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f72baffd640 1 -- 172.21.15.103:0/2170054040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c00ffe440 con 0x561c00f805d0 2024-09-19T06:52:11.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f72baffd640 1 --2- 172.21.15.103:0/2170054040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f805d0 0x561c00f8ebe0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f72ac097d70 tx=0x7f72ac0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6375ba2b7e4dc157 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:11.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f729affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:11.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f729affd640 1 -- 172.21.15.103:0/2170054040 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0b4060 con 0x561c00f805d0 2024-09-19T06:52:11.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f729affd640 1 -- 172.21.15.103:0/2170054040 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f72ac017360 con 0x561c00f805d0 2024-09-19T06:52:11.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f72c3d49740 1 -- 172.21.15.103:0/2170054040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72a4031970 con 0x561c00f805d0 2024-09-19T06:52:11.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f729affd640 1 -- 172.21.15.103:0/2170054040 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f72ac01e040 con 0x561c00f805d0 2024-09-19T06:52:11.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f72c3d49740 10 client.16530 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:11.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f72c3d49740 1 -- 172.21.15.103:0/2170054040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c010056e0 con 0x561c00f805d0 2024-09-19T06:52:11.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.809+0000 7f729affd640 1 -- 172.21.15.103:0/2170054040 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a4031970 con 0x561c00f805d0 2024-09-19T06:52:11.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.810+0000 7f729affd640 1 -- 172.21.15.103:0/2170054040 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f72ac0b4200 con 0x561c00f805d0 2024-09-19T06:52:11.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.810+0000 7f72c3d49740 10 client.16530 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:11.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.810+0000 7f72c3d49740 10 client.16530 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:52:11.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.810+0000 7f72c3d49740 20 client.16530 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:11.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.810+0000 7f72c3d49740 1 --2- 172.21.15.103:0/2170054040 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561c00f9b070 0x561c01007980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.810+0000 7f72c3d49740 4 client.16530 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:52:11.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.810+0000 7f72c3d49740 1 -- 172.21.15.103:0/2170054040 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c00eedd70 con 0x561c00f9b070 2024-09-19T06:52:11.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.810+0000 7f72b27fc640 1 --2- 172.21.15.103:0/2170054040 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561c00f9b070 0x561c01007980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.810+0000 7f729affd640 10 client.16530 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:52:11.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.810+0000 7f72b27fc640 1 --2- 172.21.15.103:0/2170054040 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561c00f9b070 0x561c01007980 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2810a96a9b8324ae server_cookie=e5ffd0c463ca62b6 in_seq=0 out_seq=0 2024-09-19T06:52:11.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.811+0000 7f729affd640 1 -- 172.21.15.103:0/2170054040 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c00eedd70 con 0x561c00f9b070 2024-09-19T06:52:11.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.811+0000 7f729affd640 10 client.16530 handle_command_reply: tid=0 2024-09-19T06:52:11.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.811+0000 7f72c3d49740 1 client.16530 shutdown 2024-09-19T06:52:11.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.811+0000 7f72c3d49740 1 -- 172.21.15.103:0/2170054040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f805d0 msgr2=0x561c00f8ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:11.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.811+0000 7f72c3d49740 1 --2- 172.21.15.103:0/2170054040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f805d0 0x561c00f8ebe0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f72ac097d70 tx=0x7f72ac0047d0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.811+0000 7f72c3d49740 1 -- 172.21.15.103:0/2170054040 shutdown_connections 2024-09-19T06:52:11.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.811+0000 7f72c3d49740 1 --2- 172.21.15.103:0/2170054040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c00f8b370 0x561c00ffdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.811+0000 7f72c3d49740 1 --2- 172.21.15.103:0/2170054040 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561c00f9b070 0x561c01007980 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.811+0000 7f72c3d49740 1 --2- 172.21.15.103:0/2170054040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c00f81b40 0x561c00f8f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.811+0000 7f72c3d49740 1 --2- 172.21.15.103:0/2170054040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c00f805d0 0x561c00f8ebe0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.811+0000 7f729affd640 0 client.16530 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:52:11.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.811+0000 7f72c3d49740 1 -- 172.21.15.103:0/2170054040 >> 172.21.15.103:0/2170054040 conn(0x561c00f49310 msgr2=0x561c00f898c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:11.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.811+0000 7f72c3d49740 1 -- 172.21.15.103:0/2170054040 shutdown_connections 2024-09-19T06:52:11.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.811+0000 7f72c3d49740 1 -- 172.21.15.103:0/2170054040 wait complete. 2024-09-19T06:52:11.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.811+0000 7f72c3d49740 20 client.16530 trim_cache size 0 max 16384 2024-09-19T06:52:11.818 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:52:11.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.812+0000 7f72c1970640 1 -- 172.21.15.103:0/1874300915 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f72a80a3430 msgr2=0x7f72a80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:11.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.812+0000 7f72c1970640 1 --2- 172.21.15.103:0/1874300915 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f72a80a3430 0x7f72a80a58f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f72b4007ed0 tx=0x7f72b40075c0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.812+0000 7f72c1970640 1 -- 172.21.15.103:0/1874300915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72bc102d30 msgr2=0x7f72bc198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:11.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.812+0000 7f72c1970640 1 --2- 172.21.15.103:0/1874300915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72bc102d30 0x7f72bc198700 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00b4a0 tx=0x7f72ac00b960 comp rx=0 tx=0).stop 2024-09-19T06:52:11.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.812+0000 7f72c1970640 1 -- 172.21.15.103:0/1874300915 shutdown_connections 2024-09-19T06:52:11.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.812+0000 7f72c1970640 1 --2- 172.21.15.103:0/1874300915 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f72a80a3430 0x7f72a80a58f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.812+0000 7f72c1970640 1 --2- 172.21.15.103:0/1874300915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72bc1082a0 0x7f72bc19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.812+0000 7f72c1970640 1 --2- 172.21.15.103:0/1874300915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72bc1036e0 0x7f72bc198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.812+0000 7f72c1970640 1 --2- 172.21.15.103:0/1874300915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72bc102d30 0x7f72bc198700 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.812+0000 7f72c1970640 1 -- 172.21.15.103:0/1874300915 >> 172.21.15.103:0/1874300915 conn(0x7f72bc0fc7d0 msgr2=0x7f72bc105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:11.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.812+0000 7f72c1970640 1 -- 172.21.15.103:0/1874300915 shutdown_connections 2024-09-19T06:52:11.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:11.812+0000 7f72c1970640 1 -- 172.21.15.103:0/1874300915 wait complete. 2024-09-19T06:52:12.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:12 smithi174 ceph-mon[35637]: pgmap v92: 97 pgs: 97 active+clean; 2.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-09-19T06:52:12.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:12 smithi103 ceph-mon[31158]: pgmap v92: 97 pgs: 97 active+clean; 2.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-09-19T06:52:12.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:12 smithi134 ceph-mon[35356]: pgmap v92: 97 pgs: 97 active+clean; 2.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-09-19T06:52:12.472 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:52:12.721 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.214+0000 7fb0cfa39640 1 -- 172.21.15.103:0/238607897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0c8106a80 msgr2=0x7fb0c8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.214+0000 7fb0cfa39640 1 --2- 172.21.15.103:0/238607897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0c8106a80 0x7fb0c8106e60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009930 tx=0x7fb0bc02a420 comp rx=0 tx=0).stop 2024-09-19T06:52:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.214+0000 7fb0cfa39640 1 -- 172.21.15.103:0/238607897 shutdown_connections 2024-09-19T06:52:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.214+0000 7fb0cfa39640 1 --2- 172.21.15.103:0/238607897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c8101420 0x7fb0c810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.214+0000 7fb0cfa39640 1 --2- 172.21.15.103:0/238607897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0c8100a80 0x7fb0c8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.214+0000 7fb0cfa39640 1 --2- 172.21.15.103:0/238607897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0c8106a80 0x7fb0c8106e60 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.214+0000 7fb0cfa39640 1 -- 172.21.15.103:0/238607897 >> 172.21.15.103:0/238607897 conn(0x7fb0c80fc7a0 msgr2=0x7fb0c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.214+0000 7fb0cfa39640 1 -- 172.21.15.103:0/238607897 shutdown_connections 2024-09-19T06:52:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.214+0000 7fb0cfa39640 1 -- 172.21.15.103:0/238607897 wait complete. 2024-09-19T06:52:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cfa39640 1 Processor -- start 2024-09-19T06:52:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cfa39640 1 -- start start 2024-09-19T06:52:13.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cfa39640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0c8100a80 0x7fb0c80fce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cfa39640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c8101420 0x7fb0c80fd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cfa39640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0c8106a80 0x7fb0c80fd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cfa39640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c8110ec0 con 0x7fb0c8100a80 2024-09-19T06:52:13.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cfa39640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c8110d40 con 0x7fb0c8106a80 2024-09-19T06:52:13.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cfa39640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c8111040 con 0x7fb0c8101420 2024-09-19T06:52:13.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cd7d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0c8100a80 0x7fb0c80fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cd7d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0c8100a80 0x7fb0c80fce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53538/0 (socket says 172.21.15.103:53538) 2024-09-19T06:52:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cd7d5640 1 -- 172.21.15.103:0/2390015507 learned_addr learned my addr 172.21.15.103:0/2390015507 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0ccfd4640 1 --2- 172.21.15.103:0/2390015507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c8101420 0x7fb0c80fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cd7d5640 1 -- 172.21.15.103:0/2390015507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c8101420 msgr2=0x7fb0c80fd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cd7d5640 1 --2- 172.21.15.103:0/2390015507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c8101420 0x7fb0c80fd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cdfd6640 1 --2- 172.21.15.103:0/2390015507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0c8106a80 0x7fb0c80fd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cd7d5640 1 -- 172.21.15.103:0/2390015507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0c8106a80 msgr2=0x7fb0c80fd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cd7d5640 1 --2- 172.21.15.103:0/2390015507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0c8106a80 0x7fb0c80fd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0cd7d5640 1 -- 172.21.15.103:0/2390015507 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c81a6900 con 0x7fb0c8100a80 2024-09-19T06:52:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.215+0000 7fb0ccfd4640 1 --2- 172.21.15.103:0/2390015507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c8101420 0x7fb0c80fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.216+0000 7fb0cd7d5640 1 --2- 172.21.15.103:0/2390015507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0c8100a80 0x7fb0c80fce30 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009230 tx=0x7fb0bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.216+0000 7fb0ba7fc640 1 -- 172.21.15.103:0/2390015507 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc03f030 con 0x7fb0c8100a80 2024-09-19T06:52:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.216+0000 7fb0cfa39640 1 -- 172.21.15.103:0/2390015507 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c81a6b90 con 0x7fb0c8100a80 2024-09-19T06:52:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.216+0000 7fb0ba7fc640 1 -- 172.21.15.103:0/2390015507 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0bc003ad0 con 0x7fb0c8100a80 2024-09-19T06:52:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.216+0000 7fb0ba7fc640 1 -- 172.21.15.103:0/2390015507 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc035070 con 0x7fb0c8100a80 2024-09-19T06:52:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.216+0000 7fb0cfa39640 1 -- 172.21.15.103:0/2390015507 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c81a70c0 con 0x7fb0c8100a80 2024-09-19T06:52:13.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.218+0000 7fb0ba7fc640 1 -- 172.21.15.103:0/2390015507 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fb0bc042030 con 0x7fb0c8100a80 2024-09-19T06:52:13.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.218+0000 7fb0d1e12740 1 Processor -- start 2024-09-19T06:52:13.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.218+0000 7fb0d1e12740 1 -- start start 2024-09-19T06:52:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0d1e12740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c4ab40 0x5642b1c49090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0d1e12740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c495d0 0x5642b1c49a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0d1e12740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c54370 0x5642b1c56760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0d1e12740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5642b1baf2f0 con 0x5642b1c495d0 2024-09-19T06:52:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0d1e12740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5642b1b56cf0 con 0x5642b1c4ab40 2024-09-19T06:52:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0d1e12740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5642b1bab2b0 con 0x5642b1c54370 2024-09-19T06:52:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0ba7fc640 1 --2- 172.21.15.103:0/2390015507 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb0b40a3430 0x7fb0b40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0cd7d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c495d0 0x5642b1c49a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0cdfd6640 1 --2- 172.21.15.103:0/2390015507 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb0b40a3430 0x7fb0b40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0cd7d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c495d0 0x5642b1c49a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53548/0 (socket says 172.21.15.103:53548) 2024-09-19T06:52:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0cd7d5640 1 -- 172.21.15.103:0/3565079848 learned_addr learned my addr 172.21.15.103:0/3565079848 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0ba7fc640 1 -- 172.21.15.103:0/2390015507 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fb0bc0efea0 con 0x7fb0c8100a80 2024-09-19T06:52:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0ccfd4640 1 --2- 172.21.15.103:0/3565079848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c54370 0x5642b1c56760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0cdfd6640 1 --2- 172.21.15.103:0/3565079848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c4ab40 0x5642b1c49090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0cd7d5640 1 -- 172.21.15.103:0/3565079848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c54370 msgr2=0x5642b1c56760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0cd7d5640 1 --2- 172.21.15.103:0/3565079848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c54370 0x5642b1c56760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0cd7d5640 1 -- 172.21.15.103:0/3565079848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c4ab40 msgr2=0x5642b1c49090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0cd7d5640 1 --2- 172.21.15.103:0/3565079848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c4ab40 0x5642b1c49090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0cdfd6640 1 --2- 172.21.15.103:0/2390015507 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb0b40a3430 0x7fb0b40a58f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0005fd0 tx=0x7fb0c00139c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.219+0000 7fb0cd7d5640 1 -- 172.21.15.103:0/3565079848 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642b1b3b3a0 con 0x5642b1c495d0 2024-09-19T06:52:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.220+0000 7fb0cd7d5640 1 --2- 172.21.15.103:0/3565079848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c495d0 0x5642b1c49a10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0b85f0 tx=0x7fb0bc0b1620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39aa7210e3a46a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.220+0000 7fb0b37fe640 1 -- 172.21.15.103:0/3565079848 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc049070 con 0x5642b1c495d0 2024-09-19T06:52:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.220+0000 7fb0b37fe640 1 -- 172.21.15.103:0/3565079848 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0bc0bc890 con 0x5642b1c495d0 2024-09-19T06:52:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.220+0000 7fb0b37fe640 1 -- 172.21.15.103:0/3565079848 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc0b8800 con 0x5642b1c495d0 2024-09-19T06:52:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.220+0000 7fb0d1e12740 1 -- 172.21.15.103:0/3565079848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c495d0 msgr2=0x5642b1c49a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.220+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/3565079848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c495d0 0x5642b1c49a10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0b85f0 tx=0x7fb0bc0b1620 comp rx=0 tx=0).stop 2024-09-19T06:52:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.221+0000 7fb0d1e12740 1 -- 172.21.15.103:0/3565079848 shutdown_connections 2024-09-19T06:52:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.221+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/3565079848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c54370 0x5642b1c56760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.221+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/3565079848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c495d0 0x5642b1c49a10 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.221+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/3565079848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c4ab40 0x5642b1c49090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.221+0000 7fb0d1e12740 1 -- 172.21.15.103:0/3565079848 >> 172.21.15.103:0/3565079848 conn(0x5642b1c12310 msgr2=0x5642b1c45c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.221+0000 7fb0d1e12740 1 -- 172.21.15.103:0/3565079848 shutdown_connections 2024-09-19T06:52:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.221+0000 7fb0d1e12740 1 -- 172.21.15.103:0/3565079848 wait complete. 2024-09-19T06:52:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.221+0000 7fb0d1e12740 1 -- start start 2024-09-19T06:52:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.221+0000 7fb0d1e12740 1 Processor -- start 2024-09-19T06:52:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.221+0000 7fb0d1e12740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c495d0 0x5642b1c53d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.221+0000 7fb0d1e12740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c4ab40 0x5642b1c51030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.221+0000 7fb0d1e12740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 0x5642b1c51640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.221+0000 7fb0d1e12740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5642b1c56e30 con 0x5642b1c4ab40 2024-09-19T06:52:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.221+0000 7fb0d1e12740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5642b1b6a060 con 0x5642b1c54370 2024-09-19T06:52:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.221+0000 7fb0d1e12740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5642b1c56fb0 con 0x5642b1c495d0 2024-09-19T06:52:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.221+0000 7fb0d1e12740 10 client.0 _ll_register_callbacks cb 0x5642b18bd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0ccfd4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c4ab40 0x5642b1c51030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0ccfd4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c4ab40 0x5642b1c51030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53560/0 (socket says 172.21.15.103:53560) 2024-09-19T06:52:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0ccfd4640 1 -- 172.21.15.103:0/1857530958 learned_addr learned my addr 172.21.15.103:0/1857530958 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0cdfd6640 1 --2- 172.21.15.103:0/1857530958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 0x5642b1c51640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0cd7d5640 1 --2- 172.21.15.103:0/1857530958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c495d0 0x5642b1c53d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0ccfd4640 1 -- 172.21.15.103:0/1857530958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c495d0 msgr2=0x5642b1c53d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0ccfd4640 1 --2- 172.21.15.103:0/1857530958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c495d0 0x5642b1c53d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0ccfd4640 1 -- 172.21.15.103:0/1857530958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 msgr2=0x5642b1c51640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0ccfd4640 1 --2- 172.21.15.103:0/1857530958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 0x5642b1c51640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0ccfd4640 1 -- 172.21.15.103:0/1857530958 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642b19203c0 con 0x5642b1c4ab40 2024-09-19T06:52:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0ccfd4640 1 --2- 172.21.15.103:0/1857530958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c4ab40 0x5642b1c51030 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400f4a0 tx=0x7fb0c400f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa45542eb229d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0b0ff9640 1 -- 172.21.15.103:0/1857530958 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4013850 con 0x5642b1c4ab40 2024-09-19T06:52:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0b0ff9640 1 -- 172.21.15.103:0/1857530958 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fb0c40139f0 con 0x5642b1c4ab40 2024-09-19T06:52:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0d1e12740 1 -- 172.21.15.103:0/1857530958 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0bc0bc890 con 0x5642b1c4ab40 2024-09-19T06:52:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0b0ff9640 1 -- 172.21.15.103:0/1857530958 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0c4013bf0 con 0x5642b1c4ab40 2024-09-19T06:52:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0d1e12740 10 client.16542 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.222+0000 7fb0d1e12740 1 -- 172.21.15.103:0/1857530958 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5642b1c62930 con 0x5642b1c4ab40 2024-09-19T06:52:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.223+0000 7fb0b0ff9640 1 -- 172.21.15.103:0/1857530958 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0bc0bc890 con 0x5642b1c4ab40 2024-09-19T06:52:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.223+0000 7fb0b0ff9640 1 -- 172.21.15.103:0/1857530958 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb0c4011400 con 0x5642b1c4ab40 2024-09-19T06:52:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.223+0000 7fb0d1e12740 10 client.16542 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.223+0000 7fb0d1e12740 10 client.16542 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:52:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.223+0000 7fb0d1e12740 20 client.16542 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.223+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/1857530958 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5642b1c63e90 0x5642b1c662b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.223+0000 7fb0d1e12740 4 client.16542 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.223+0000 7fb0d1e12740 1 -- 172.21.15.103:0/1857530958 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642b1bbc860 con 0x5642b1c63e90 2024-09-19T06:52:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.223+0000 7fb0cd7d5640 1 --2- 172.21.15.103:0/1857530958 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5642b1c63e90 0x5642b1c662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.224+0000 7fb0cd7d5640 1 --2- 172.21.15.103:0/1857530958 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5642b1c63e90 0x5642b1c662b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a358d18caaea2cdb server_cookie=14b4e2ec8c68f4c1 in_seq=0 out_seq=0 2024-09-19T06:52:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.224+0000 7fb0b0ff9640 10 client.16542 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:52:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.224+0000 7fb0b0ff9640 1 -- 172.21.15.103:0/1857530958 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642b1bbc860 con 0x5642b1c63e90 2024-09-19T06:52:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.224+0000 7fb0b0ff9640 10 client.16542 handle_command_reply: tid=0 2024-09-19T06:52:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.224+0000 7fb0d1e12740 1 client.16542 shutdown 2024-09-19T06:52:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.224+0000 7fb0d1e12740 1 -- 172.21.15.103:0/1857530958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c4ab40 msgr2=0x5642b1c51030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.224+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/1857530958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c4ab40 0x5642b1c51030 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400f4a0 tx=0x7fb0c400f960 comp rx=0 tx=0).stop 2024-09-19T06:52:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.225+0000 7fb0d1e12740 1 -- 172.21.15.103:0/1857530958 shutdown_connections 2024-09-19T06:52:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.225+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/1857530958 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5642b1c63e90 0x5642b1c662b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.225+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/1857530958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 0x5642b1c51640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.225+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/1857530958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c4ab40 0x5642b1c51030 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.225+0000 7fb0b0ff9640 0 client.16542 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:52:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.225+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/1857530958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c495d0 0x5642b1c53d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.225+0000 7fb0d1e12740 1 -- 172.21.15.103:0/1857530958 >> 172.21.15.103:0/1857530958 conn(0x5642b1c12310 msgr2=0x5642b1c4b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.225+0000 7fb0d1e12740 1 -- 172.21.15.103:0/1857530958 shutdown_connections 2024-09-19T06:52:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.225+0000 7fb0d1e12740 1 -- 172.21.15.103:0/1857530958 wait complete. 2024-09-19T06:52:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.225+0000 7fb0d1e12740 20 client.16542 trim_cache size 0 max 16384 2024-09-19T06:52:13.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0d1e12740 1 Processor -- start 2024-09-19T06:52:13.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0d1e12740 1 -- start start 2024-09-19T06:52:13.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0d1e12740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c495d0 0x5642b1c51000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0d1e12740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c4ab40 0x5642b1c60090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0d1e12740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 0x5642b1c6bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0d1e12740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81af930 con 0x5642b1c495d0 2024-09-19T06:52:13.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0d1e12740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81b1470 con 0x5642b1c54370 2024-09-19T06:52:13.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0d1e12740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81b0000 con 0x5642b1c4ab40 2024-09-19T06:52:13.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0cd7d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c495d0 0x5642b1c51000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0cd7d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c495d0 0x5642b1c51000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53568/0 (socket says 172.21.15.103:53568) 2024-09-19T06:52:13.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0cd7d5640 1 -- 172.21.15.103:0/919905173 learned_addr learned my addr 172.21.15.103:0/919905173 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0cd7d5640 1 -- 172.21.15.103:0/919905173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c4ab40 msgr2=0x5642b1c60090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0cd7d5640 1 --2- 172.21.15.103:0/919905173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c4ab40 0x5642b1c60090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0cd7d5640 1 -- 172.21.15.103:0/919905173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 msgr2=0x5642b1c6bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0cdfd6640 1 --2- 172.21.15.103:0/919905173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 0x5642b1c6bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0cd7d5640 1 --2- 172.21.15.103:0/919905173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 0x5642b1c6bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0cd7d5640 1 -- 172.21.15.103:0/919905173 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642b1c45510 con 0x5642b1c495d0 2024-09-19T06:52:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0cdfd6640 1 --2- 172.21.15.103:0/919905173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 0x5642b1c6bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.238+0000 7fb0cd7d5640 1 --2- 172.21.15.103:0/919905173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c495d0 0x5642b1c51000 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0bc120 tx=0x7fb0bc0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afff3174761a8a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.239+0000 7fb0b2ffd640 1 -- 172.21.15.103:0/919905173 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc049040 con 0x5642b1c495d0 2024-09-19T06:52:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.239+0000 7fb0b2ffd640 1 -- 172.21.15.103:0/919905173 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0bc0c4e00 con 0x5642b1c495d0 2024-09-19T06:52:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.239+0000 7fb0b2ffd640 1 -- 172.21.15.103:0/919905173 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc0dc0b0 con 0x5642b1c495d0 2024-09-19T06:52:13.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.239+0000 7fb0d1e12740 1 -- 172.21.15.103:0/919905173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c495d0 msgr2=0x5642b1c51000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:13.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.239+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/919905173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c495d0 0x5642b1c51000 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0bc120 tx=0x7fb0bc0dc040 comp rx=0 tx=0).stop 2024-09-19T06:52:13.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.239+0000 7fb0d1e12740 1 -- 172.21.15.103:0/919905173 shutdown_connections 2024-09-19T06:52:13.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.239+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/919905173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 0x5642b1c6bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.239+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/919905173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c4ab40 0x5642b1c60090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.239+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/919905173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c495d0 0x5642b1c51000 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.239+0000 7fb0d1e12740 1 -- 172.21.15.103:0/919905173 >> 172.21.15.103:0/919905173 conn(0x5642b1c12310 msgr2=0x5642b1c508f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:13.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.239+0000 7fb0d1e12740 1 -- 172.21.15.103:0/919905173 shutdown_connections 2024-09-19T06:52:13.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.239+0000 7fb0d1e12740 1 -- 172.21.15.103:0/919905173 wait complete. 2024-09-19T06:52:13.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0d1e12740 1 -- start start 2024-09-19T06:52:13.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0d1e12740 1 Processor -- start 2024-09-19T06:52:13.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0d1e12740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c495d0 0x5642b1c58260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0d1e12740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c4ab40 0x5642b1c587a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0d1e12740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 0x5642b1c58d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0d1e12740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81b0da0 con 0x5642b1c4ab40 2024-09-19T06:52:13.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0d1e12740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81b06d0 con 0x5642b1c54370 2024-09-19T06:52:13.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0d1e12740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81ad8f0 con 0x5642b1c495d0 2024-09-19T06:52:13.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0d1e12740 10 client.0 _ll_register_callbacks cb 0x7fb0c400a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:13.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0ccfd4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c4ab40 0x5642b1c587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0ccfd4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c4ab40 0x5642b1c587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53572/0 (socket says 172.21.15.103:53572) 2024-09-19T06:52:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0ccfd4640 1 -- 172.21.15.103:0/2313891217 learned_addr learned my addr 172.21.15.103:0/2313891217 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0cdfd6640 1 --2- 172.21.15.103:0/2313891217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 0x5642b1c58d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0cd7d5640 1 --2- 172.21.15.103:0/2313891217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c495d0 0x5642b1c58260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0ccfd4640 1 -- 172.21.15.103:0/2313891217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c495d0 msgr2=0x5642b1c58260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0ccfd4640 1 --2- 172.21.15.103:0/2313891217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c495d0 0x5642b1c58260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0ccfd4640 1 -- 172.21.15.103:0/2313891217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 msgr2=0x5642b1c58d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0ccfd4640 1 --2- 172.21.15.103:0/2313891217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 0x5642b1c58d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0ccfd4640 1 -- 172.21.15.103:0/2313891217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642b1ccde20 con 0x5642b1c4ab40 2024-09-19T06:52:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.240+0000 7fb0cdfd6640 1 --2- 172.21.15.103:0/2313891217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 0x5642b1c58d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.241+0000 7fb0ccfd4640 1 --2- 172.21.15.103:0/2313891217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c4ab40 0x5642b1c587a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4000c00 tx=0x7fb0c400a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ecc059e4df6d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:13.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.241+0000 7fb0b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:13.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.241+0000 7fb0b0ff9640 1 -- 172.21.15.103:0/2313891217 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4028030 con 0x5642b1c4ab40 2024-09-19T06:52:13.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.241+0000 7fb0b0ff9640 1 -- 172.21.15.103:0/2313891217 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fb0c4013650 con 0x5642b1c4ab40 2024-09-19T06:52:13.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.241+0000 7fb0d1e12740 1 -- 172.21.15.103:0/2313891217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0bc0c4e00 con 0x5642b1c4ab40 2024-09-19T06:52:13.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.241+0000 7fb0b0ff9640 1 -- 172.21.15.103:0/2313891217 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0c400ae50 con 0x5642b1c4ab40 2024-09-19T06:52:13.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.241+0000 7fb0d1e12740 10 client.16548 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:13.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.241+0000 7fb0d1e12740 1 -- 172.21.15.103:0/2313891217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5642b1ccf090 con 0x5642b1c4ab40 2024-09-19T06:52:13.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.241+0000 7fb0b0ff9640 1 -- 172.21.15.103:0/2313891217 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0bc0c4e00 con 0x5642b1c4ab40 2024-09-19T06:52:13.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.241+0000 7fb0b0ff9640 1 -- 172.21.15.103:0/2313891217 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb0c40060d0 con 0x5642b1c4ab40 2024-09-19T06:52:13.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.241+0000 7fb0d1e12740 10 client.16548 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:13.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.241+0000 7fb0d1e12740 10 client.16548 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:52:13.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.241+0000 7fb0d1e12740 20 client.16548 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:13.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.241+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/2313891217 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5642b1ccf780 0x5642b1cd1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.241+0000 7fb0d1e12740 4 client.16548 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:52:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.241+0000 7fb0d1e12740 1 -- 172.21.15.103:0/2313891217 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642b1bb6d70 con 0x5642b1ccf780 2024-09-19T06:52:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.242+0000 7fb0cd7d5640 1 --2- 172.21.15.103:0/2313891217 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5642b1ccf780 0x5642b1cd1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.242+0000 7fb0b0ff9640 10 client.16548 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:52:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.242+0000 7fb0cd7d5640 1 --2- 172.21.15.103:0/2313891217 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5642b1ccf780 0x5642b1cd1ba0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4d7e407c01ac901 server_cookie=4077cad46f6c6601 in_seq=0 out_seq=0 2024-09-19T06:52:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.242+0000 7fb0b0ff9640 1 -- 172.21.15.103:0/2313891217 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642b1bb6d70 con 0x5642b1ccf780 2024-09-19T06:52:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.242+0000 7fb0b0ff9640 10 client.16548 handle_command_reply: tid=0 2024-09-19T06:52:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.242+0000 7fb0d1e12740 1 client.16548 shutdown 2024-09-19T06:52:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.242+0000 7fb0d1e12740 1 -- 172.21.15.103:0/2313891217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c4ab40 msgr2=0x5642b1c587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.242+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/2313891217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c4ab40 0x5642b1c587a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4000c00 tx=0x7fb0c400a6a0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.242+0000 7fb0d1e12740 1 -- 172.21.15.103:0/2313891217 shutdown_connections 2024-09-19T06:52:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.242+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/2313891217 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5642b1ccf780 0x5642b1cd1ba0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.242+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/2313891217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5642b1c54370 0x5642b1c58d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.242+0000 7fb0b0ff9640 0 client.16548 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:52:13.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.242+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/2313891217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642b1c4ab40 0x5642b1c587a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.242+0000 7fb0d1e12740 1 --2- 172.21.15.103:0/2313891217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642b1c495d0 0x5642b1c58260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.243+0000 7fb0d1e12740 1 -- 172.21.15.103:0/2313891217 >> 172.21.15.103:0/2313891217 conn(0x5642b1c12310 msgr2=0x5642b1c444e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:13.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.243+0000 7fb0d1e12740 1 -- 172.21.15.103:0/2313891217 shutdown_connections 2024-09-19T06:52:13.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.243+0000 7fb0d1e12740 1 -- 172.21.15.103:0/2313891217 wait complete. 2024-09-19T06:52:13.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.243+0000 7fb0d1e12740 20 client.16548 trim_cache size 0 max 16384 2024-09-19T06:52:13.252 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:52:13.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.243+0000 7fb0cfa39640 1 -- 172.21.15.103:0/2390015507 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb0b40a3430 msgr2=0x7fb0b40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:13.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.243+0000 7fb0cfa39640 1 --2- 172.21.15.103:0/2390015507 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb0b40a3430 0x7fb0b40a58f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0005fd0 tx=0x7fb0c00139c0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.243+0000 7fb0cfa39640 1 -- 172.21.15.103:0/2390015507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0c8100a80 msgr2=0x7fb0c80fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:13.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.243+0000 7fb0cfa39640 1 --2- 172.21.15.103:0/2390015507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0c8100a80 0x7fb0c80fce30 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009230 tx=0x7fb0bc004270 comp rx=0 tx=0).stop 2024-09-19T06:52:13.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.243+0000 7fb0cfa39640 1 -- 172.21.15.103:0/2390015507 shutdown_connections 2024-09-19T06:52:13.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.243+0000 7fb0cfa39640 1 --2- 172.21.15.103:0/2390015507 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb0b40a3430 0x7fb0b40a58f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.243+0000 7fb0cfa39640 1 --2- 172.21.15.103:0/2390015507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0c8106a80 0x7fb0c80fd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.243+0000 7fb0cfa39640 1 --2- 172.21.15.103:0/2390015507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c8101420 0x7fb0c80fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.243+0000 7fb0cfa39640 1 --2- 172.21.15.103:0/2390015507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0c8100a80 0x7fb0c80fce30 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:13.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.243+0000 7fb0cfa39640 1 -- 172.21.15.103:0/2390015507 >> 172.21.15.103:0/2390015507 conn(0x7fb0c80fc7a0 msgr2=0x7fb0c806aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:13.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.243+0000 7fb0cfa39640 1 -- 172.21.15.103:0/2390015507 shutdown_connections 2024-09-19T06:52:13.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:13.243+0000 7fb0cfa39640 1 -- 172.21.15.103:0/2390015507 wait complete. 2024-09-19T06:52:13.972 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:52:14.222 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:14.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:14 smithi174 ceph-mon[35637]: pgmap v93: 97 pgs: 97 active+clean; 2.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-09-19T06:52:14.335 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:14 smithi103 ceph-mon[31158]: pgmap v93: 97 pgs: 97 active+clean; 2.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-09-19T06:52:14.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:14 smithi134 ceph-mon[35356]: pgmap v93: 97 pgs: 97 active+clean; 2.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-09-19T06:52:14.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.667+0000 7fdfabdb3640 1 -- 172.21.15.103:0/3357420907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa4102ef0 msgr2=0x7fdfa410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.667+0000 7fdfabdb3640 1 --2- 172.21.15.103:0/3357420907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa4102ef0 0x7fdfa410da40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009ca0 tx=0x7fdf9c02a950 comp rx=0 tx=0).stop 2024-09-19T06:52:14.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.667+0000 7fdfabdb3640 1 -- 172.21.15.103:0/3357420907 shutdown_connections 2024-09-19T06:52:14.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.667+0000 7fdfabdb3640 1 --2- 172.21.15.103:0/3357420907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa4102ef0 0x7fdfa410da40 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.667+0000 7fdfabdb3640 1 --2- 172.21.15.103:0/3357420907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdfa4102570 0x7fdfa41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.667+0000 7fdfabdb3640 1 --2- 172.21.15.103:0/3357420907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfa4100e20 0x7fdfa4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.667+0000 7fdfabdb3640 1 -- 172.21.15.103:0/3357420907 >> 172.21.15.103:0/3357420907 conn(0x7fdfa40fc7d0 msgr2=0x7fdfa40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:14.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.667+0000 7fdfabdb3640 1 -- 172.21.15.103:0/3357420907 shutdown_connections 2024-09-19T06:52:14.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.667+0000 7fdfabdb3640 1 -- 172.21.15.103:0/3357420907 wait complete. 2024-09-19T06:52:14.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.668+0000 7fdfabdb3640 1 Processor -- start 2024-09-19T06:52:14.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.668+0000 7fdfabdb3640 1 -- start start 2024-09-19T06:52:14.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.668+0000 7fdfabdb3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdfa4100e20 0x7fdfa4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.668+0000 7fdfabdb3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa4102570 0x7fdfa4198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.668+0000 7fdfabdb3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfa4102ef0 0x7fdfa419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.668+0000 7fdfabdb3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa4111020 con 0x7fdfa4102570 2024-09-19T06:52:14.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.668+0000 7fdfabdb3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa4110ea0 con 0x7fdfa4102ef0 2024-09-19T06:52:14.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.668+0000 7fdfabdb3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41111a0 con 0x7fdfa4100e20 2024-09-19T06:52:14.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.668+0000 7fdfa934e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa4102570 0x7fdfa4198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.668+0000 7fdfa934e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa4102570 0x7fdfa4198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53594/0 (socket says 172.21.15.103:53594) 2024-09-19T06:52:14.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.668+0000 7fdfa934e640 1 -- 172.21.15.103:0/1860616660 learned_addr learned my addr 172.21.15.103:0/1860616660 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:14.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.668+0000 7fdfaa350640 1 --2- 172.21.15.103:0/1860616660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfa4102ef0 0x7fdfa419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.668+0000 7fdfa9b4f640 1 --2- 172.21.15.103:0/1860616660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdfa4100e20 0x7fdfa4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.669+0000 7fdfa934e640 1 -- 172.21.15.103:0/1860616660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdfa4100e20 msgr2=0x7fdfa4198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.669+0000 7fdfa934e640 1 --2- 172.21.15.103:0/1860616660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdfa4100e20 0x7fdfa4198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.669+0000 7fdfa934e640 1 -- 172.21.15.103:0/1860616660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfa4102ef0 msgr2=0x7fdfa419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.669+0000 7fdfa934e640 1 --2- 172.21.15.103:0/1860616660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfa4102ef0 0x7fdfa419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.669+0000 7fdfa934e640 1 -- 172.21.15.103:0/1860616660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa419d5b0 con 0x7fdfa4102570 2024-09-19T06:52:14.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.669+0000 7fdfaa350640 1 --2- 172.21.15.103:0/1860616660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfa4102ef0 0x7fdfa419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:14.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.669+0000 7fdfa934e640 1 --2- 172.21.15.103:0/1860616660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa4102570 0x7fdfa4198b40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400b4a0 tx=0x7fdf9400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:14.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.669+0000 7fdf9affd640 1 -- 172.21.15.103:0/1860616660 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94014070 con 0x7fdfa4102570 2024-09-19T06:52:14.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.669+0000 7fdf9affd640 1 -- 172.21.15.103:0/1860616660 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf94010af0 con 0x7fdfa4102570 2024-09-19T06:52:14.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.669+0000 7fdfabdb3640 1 -- 172.21.15.103:0/1860616660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa419d8a0 con 0x7fdfa4102570 2024-09-19T06:52:14.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.669+0000 7fdf9affd640 1 -- 172.21.15.103:0/1860616660 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94017d10 con 0x7fdfa4102570 2024-09-19T06:52:14.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.669+0000 7fdfabdb3640 1 -- 172.21.15.103:0/1860616660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa41a5180 con 0x7fdfa4102570 2024-09-19T06:52:14.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.672+0000 7fdf9affd640 1 -- 172.21.15.103:0/1860616660 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fdf94017070 con 0x7fdfa4102570 2024-09-19T06:52:14.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.672+0000 7fdfae18c740 1 Processor -- start 2024-09-19T06:52:14.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.672+0000 7fdfae18c740 1 -- start start 2024-09-19T06:52:14.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.672+0000 7fdfae18c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd31bb40 0x5563cd31a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.672+0000 7fdfae18c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31a5d0 0x5563cd31aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.672+0000 7fdfae18c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd329780 0x5563cd32bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdfae18c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5563cd2802f0 con 0x5563cd31a5d0 2024-09-19T06:52:14.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdfae18c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5563cd227cf0 con 0x5563cd31bb40 2024-09-19T06:52:14.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdfae18c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5563cd27c2b0 con 0x5563cd329780 2024-09-19T06:52:14.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdfa9b4f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31a5d0 0x5563cd31aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdfa9b4f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31a5d0 0x5563cd31aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53606/0 (socket says 172.21.15.103:53606) 2024-09-19T06:52:14.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdfa9b4f640 1 -- 172.21.15.103:0/4151283815 learned_addr learned my addr 172.21.15.103:0/4151283815 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:14.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdf9affd640 1 --2- 172.21.15.103:0/1860616660 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdf900a33b0 0x7fdf900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdfaa350640 1 --2- 172.21.15.103:0/4151283815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd31bb40 0x5563cd31a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdfa934e640 1 --2- 172.21.15.103:0/4151283815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd329780 0x5563cd32bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdfa9b4f640 1 -- 172.21.15.103:0/4151283815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd329780 msgr2=0x5563cd32bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:14.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdfa9b4f640 1 --2- 172.21.15.103:0/4151283815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd329780 0x5563cd32bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdfa9b4f640 1 -- 172.21.15.103:0/4151283815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd31bb40 msgr2=0x5563cd31a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:14.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdfa9b4f640 1 --2- 172.21.15.103:0/4151283815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd31bb40 0x5563cd31a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdfa9b4f640 1 -- 172.21.15.103:0/4151283815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563cd20c3a0 con 0x5563cd31a5d0 2024-09-19T06:52:14.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdfaa350640 1 --2- 172.21.15.103:0/4151283815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd31bb40 0x5563cd31a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdf9affd640 1 -- 172.21.15.103:0/1860616660 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fdf940cf080 con 0x7fdfa4102570 2024-09-19T06:52:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdfaa350640 1 --2- 172.21.15.103:0/1860616660 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdf900a33b0 0x7fdf900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdfa9b4f640 1 --2- 172.21.15.103:0/4151283815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31a5d0 0x5563cd31aa10 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000b820 tx=0x7fdfa000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c16ecadd9bfdff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdf8ffff640 1 -- 172.21.15.103:0/4151283815 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa000d950 con 0x5563cd31a5d0 2024-09-19T06:52:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdf8ffff640 1 -- 172.21.15.103:0/4151283815 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfa0019070 con 0x5563cd31a5d0 2024-09-19T06:52:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.673+0000 7fdf8ffff640 1 -- 172.21.15.103:0/4151283815 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa00123e0 con 0x5563cd31a5d0 2024-09-19T06:52:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.674+0000 7fdfae18c740 1 -- 172.21.15.103:0/4151283815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31a5d0 msgr2=0x5563cd31aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.674+0000 7fdfaa350640 1 --2- 172.21.15.103:0/1860616660 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdf900a33b0 0x7fdf900a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c02ae70 tx=0x7fdf9c0095c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.674+0000 7fdfae18c740 1 --2- 172.21.15.103:0/4151283815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31a5d0 0x5563cd31aa10 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000b820 tx=0x7fdfa000bce0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.674+0000 7fdfae18c740 1 -- 172.21.15.103:0/4151283815 shutdown_connections 2024-09-19T06:52:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.674+0000 7fdfae18c740 1 --2- 172.21.15.103:0/4151283815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd329780 0x5563cd32bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.674+0000 7fdfae18c740 1 --2- 172.21.15.103:0/4151283815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31a5d0 0x5563cd31aa10 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.674+0000 7fdfae18c740 1 --2- 172.21.15.103:0/4151283815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd31bb40 0x5563cd31a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.674+0000 7fdfae18c740 1 -- 172.21.15.103:0/4151283815 >> 172.21.15.103:0/4151283815 conn(0x5563cd2e3310 msgr2=0x5563cd316c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.674+0000 7fdfae18c740 1 -- 172.21.15.103:0/4151283815 shutdown_connections 2024-09-19T06:52:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.674+0000 7fdfae18c740 1 -- 172.21.15.103:0/4151283815 wait complete. 2024-09-19T06:52:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfae18c740 1 -- start start 2024-09-19T06:52:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfae18c740 1 Processor -- start 2024-09-19T06:52:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfae18c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd31a5d0 0x5563cd329150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfae18c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31bb40 0x5563cd326400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfae18c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd329780 0x5563cd3269f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfae18c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5563cd32c480 con 0x5563cd31bb40 2024-09-19T06:52:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfae18c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5563cd23b060 con 0x5563cd329780 2024-09-19T06:52:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfae18c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5563cd32c600 con 0x5563cd31a5d0 2024-09-19T06:52:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfae18c740 10 client.0 _ll_register_callbacks cb 0x5563ccf8e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfa934e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31bb40 0x5563cd326400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfa934e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31bb40 0x5563cd326400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53608/0 (socket says 172.21.15.103:53608) 2024-09-19T06:52:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfa934e640 1 -- 172.21.15.103:0/3721479600 learned_addr learned my addr 172.21.15.103:0/3721479600 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfa934e640 1 -- 172.21.15.103:0/3721479600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd31a5d0 msgr2=0x5563cd329150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfa934e640 1 --2- 172.21.15.103:0/3721479600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd31a5d0 0x5563cd329150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfa934e640 1 -- 172.21.15.103:0/3721479600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd329780 msgr2=0x5563cd3269f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfa934e640 1 --2- 172.21.15.103:0/3721479600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd329780 0x5563cd3269f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.675+0000 7fdfa934e640 1 -- 172.21.15.103:0/3721479600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ccff13c0 con 0x5563cd31bb40 2024-09-19T06:52:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.676+0000 7fdfa934e640 1 --2- 172.21.15.103:0/3721479600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31bb40 0x5563cd326400 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fdf94097e50 tx=0x7fdf94017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13eae0a6f2dabb5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.676+0000 7fdf8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.676+0000 7fdf8d7fa640 1 -- 172.21.15.103:0/3721479600 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9401e050 con 0x5563cd31bb40 2024-09-19T06:52:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.676+0000 7fdf8d7fa640 1 -- 172.21.15.103:0/3721479600 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fdf94010ca0 con 0x5563cd31bb40 2024-09-19T06:52:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.676+0000 7fdfae18c740 1 -- 172.21.15.103:0/3721479600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfa0019070 con 0x5563cd31bb40 2024-09-19T06:52:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.676+0000 7fdf8d7fa640 1 -- 172.21.15.103:0/3721479600 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf9409f830 con 0x5563cd31bb40 2024-09-19T06:52:14.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.676+0000 7fdfae18c740 10 client.16560 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.676+0000 7fdfae18c740 1 -- 172.21.15.103:0/3721479600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5563cd337e60 con 0x5563cd31bb40 2024-09-19T06:52:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.676+0000 7fdf8d7fa640 1 -- 172.21.15.103:0/3721479600 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfa0019070 con 0x5563cd31bb40 2024-09-19T06:52:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.676+0000 7fdf8d7fa640 1 -- 172.21.15.103:0/3721479600 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdf940ccaf0 con 0x5563cd31bb40 2024-09-19T06:52:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.676+0000 7fdfae18c740 10 client.16560 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.676+0000 7fdfae18c740 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.676+0000 7fdfae18c740 20 client.16560 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.676+0000 7fdfae18c740 1 --2- 172.21.15.103:0/3721479600 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5563cd339320 0x5563cd33b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.676+0000 7fdfae18c740 4 client.16560 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.676+0000 7fdfae18c740 1 -- 172.21.15.103:0/3721479600 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563cd28d860 con 0x5563cd339320 2024-09-19T06:52:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.677+0000 7fdfa9b4f640 1 --2- 172.21.15.103:0/3721479600 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5563cd339320 0x5563cd33b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.677+0000 7fdf8d7fa640 10 client.16560 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.677+0000 7fdfa9b4f640 1 --2- 172.21.15.103:0/3721479600 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5563cd339320 0x5563cd33b7b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef499181d0bd5d2a server_cookie=b3ccb7ec3b4df57c in_seq=0 out_seq=0 2024-09-19T06:52:14.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.678+0000 7fdf8d7fa640 1 -- 172.21.15.103:0/3721479600 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563cd28d860 con 0x5563cd339320 2024-09-19T06:52:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.678+0000 7fdf8d7fa640 10 client.16560 handle_command_reply: tid=0 2024-09-19T06:52:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.678+0000 7fdfae18c740 1 client.16560 shutdown 2024-09-19T06:52:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.678+0000 7fdfae18c740 1 -- 172.21.15.103:0/3721479600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31bb40 msgr2=0x5563cd326400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.678+0000 7fdfae18c740 1 --2- 172.21.15.103:0/3721479600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31bb40 0x5563cd326400 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fdf94097e50 tx=0x7fdf94017730 comp rx=0 tx=0).stop 2024-09-19T06:52:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.678+0000 7fdfae18c740 1 -- 172.21.15.103:0/3721479600 shutdown_connections 2024-09-19T06:52:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.678+0000 7fdfae18c740 1 --2- 172.21.15.103:0/3721479600 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5563cd339320 0x5563cd33b7b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.678+0000 7fdfae18c740 1 --2- 172.21.15.103:0/3721479600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd329780 0x5563cd3269f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.678+0000 7fdf8d7fa640 0 client.16560 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.678+0000 7fdfae18c740 1 --2- 172.21.15.103:0/3721479600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31bb40 0x5563cd326400 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.678+0000 7fdfae18c740 1 --2- 172.21.15.103:0/3721479600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd31a5d0 0x5563cd329150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.678+0000 7fdfae18c740 1 -- 172.21.15.103:0/3721479600 >> 172.21.15.103:0/3721479600 conn(0x5563cd2e3310 msgr2=0x5563cd31c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.678+0000 7fdfae18c740 1 -- 172.21.15.103:0/3721479600 shutdown_connections 2024-09-19T06:52:14.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.678+0000 7fdfae18c740 1 -- 172.21.15.103:0/3721479600 wait complete. 2024-09-19T06:52:14.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.678+0000 7fdfae18c740 20 client.16560 trim_cache size 0 max 16384 2024-09-19T06:52:14.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.691+0000 7fdfae18c740 1 Processor -- start 2024-09-19T06:52:14.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.691+0000 7fdfae18c740 1 -- start start 2024-09-19T06:52:14.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.691+0000 7fdfae18c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd31a5d0 0x5563cd316e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.691+0000 7fdfae18c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31bb40 0x5563cd335720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.691+0000 7fdfae18c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd329780 0x5563cd3410d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.691+0000 7fdfae18c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41ab710 con 0x5563cd31bb40 2024-09-19T06:52:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.691+0000 7fdfae18c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41ad250 con 0x5563cd329780 2024-09-19T06:52:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.691+0000 7fdfae18c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41abde0 con 0x5563cd31a5d0 2024-09-19T06:52:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.692+0000 7fdfa934e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31bb40 0x5563cd335720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.692+0000 7fdfa934e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31bb40 0x5563cd335720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53624/0 (socket says 172.21.15.103:53624) 2024-09-19T06:52:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.692+0000 7fdfa934e640 1 -- 172.21.15.103:0/2262306923 learned_addr learned my addr 172.21.15.103:0/2262306923 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.692+0000 7fdfa9b4f640 1 --2- 172.21.15.103:0/2262306923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd31a5d0 0x5563cd316e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.692+0000 7fdfaa350640 1 --2- 172.21.15.103:0/2262306923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd329780 0x5563cd3410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.692+0000 7fdfa934e640 1 -- 172.21.15.103:0/2262306923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd31a5d0 msgr2=0x5563cd316e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.692+0000 7fdfa934e640 1 --2- 172.21.15.103:0/2262306923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd31a5d0 0x5563cd316e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.692+0000 7fdfa934e640 1 -- 172.21.15.103:0/2262306923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd329780 msgr2=0x5563cd3410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.692+0000 7fdfa934e640 1 --2- 172.21.15.103:0/2262306923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd329780 0x5563cd3410d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.692+0000 7fdfa934e640 1 -- 172.21.15.103:0/2262306923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563cd31d120 con 0x5563cd31bb40 2024-09-19T06:52:14.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.692+0000 7fdfa934e640 1 --2- 172.21.15.103:0/2262306923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31bb40 0x5563cd335720 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fdf9409b410 tx=0x7fdf94099990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4bbc5f194c7388 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:14.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.692+0000 7fdf8f7fe640 1 -- 172.21.15.103:0/2262306923 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9401e070 con 0x5563cd31bb40 2024-09-19T06:52:14.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.692+0000 7fdf8f7fe640 1 -- 172.21.15.103:0/2262306923 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf94010ca0 con 0x5563cd31bb40 2024-09-19T06:52:14.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.692+0000 7fdf8f7fe640 1 -- 172.21.15.103:0/2262306923 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf940b4260 con 0x5563cd31bb40 2024-09-19T06:52:14.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.693+0000 7fdfae18c740 1 -- 172.21.15.103:0/2262306923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31bb40 msgr2=0x5563cd335720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.693+0000 7fdfae18c740 1 --2- 172.21.15.103:0/2262306923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31bb40 0x5563cd335720 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fdf9409b410 tx=0x7fdf94099990 comp rx=0 tx=0).stop 2024-09-19T06:52:14.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.693+0000 7fdfae18c740 1 -- 172.21.15.103:0/2262306923 shutdown_connections 2024-09-19T06:52:14.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.693+0000 7fdfae18c740 1 --2- 172.21.15.103:0/2262306923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd329780 0x5563cd3410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.693+0000 7fdfae18c740 1 --2- 172.21.15.103:0/2262306923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd31bb40 0x5563cd335720 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.693+0000 7fdfae18c740 1 --2- 172.21.15.103:0/2262306923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd31a5d0 0x5563cd316e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.693+0000 7fdfae18c740 1 -- 172.21.15.103:0/2262306923 >> 172.21.15.103:0/2262306923 conn(0x5563cd2e3310 msgr2=0x5563cd316700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.693+0000 7fdfae18c740 1 -- 172.21.15.103:0/2262306923 shutdown_connections 2024-09-19T06:52:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.693+0000 7fdfae18c740 1 -- 172.21.15.103:0/2262306923 wait complete. 2024-09-19T06:52:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.693+0000 7fdfae18c740 1 -- start start 2024-09-19T06:52:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.693+0000 7fdfae18c740 1 Processor -- start 2024-09-19T06:52:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfae18c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd31a5d0 0x5563cd331860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfae18c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd31bb40 0x5563cd331da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfae18c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd329780 0x5563cd32e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfae18c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41acb80 con 0x5563cd329780 2024-09-19T06:52:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfae18c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41ac4b0 con 0x5563cd31a5d0 2024-09-19T06:52:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfae18c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa41a96d0 con 0x5563cd31bb40 2024-09-19T06:52:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfae18c740 10 client.0 _ll_register_callbacks cb 0x7fdf94017630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:14.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfaa350640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd329780 0x5563cd32e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfa934e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd31bb40 0x5563cd331da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfaa350640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd329780 0x5563cd32e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53630/0 (socket says 172.21.15.103:53630) 2024-09-19T06:52:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfaa350640 1 -- 172.21.15.103:0/2412460358 learned_addr learned my addr 172.21.15.103:0/2412460358 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfa9b4f640 1 --2- 172.21.15.103:0/2412460358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd31a5d0 0x5563cd331860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfaa350640 1 -- 172.21.15.103:0/2412460358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd31bb40 msgr2=0x5563cd331da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfaa350640 1 --2- 172.21.15.103:0/2412460358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd31bb40 0x5563cd331da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfaa350640 1 -- 172.21.15.103:0/2412460358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd31a5d0 msgr2=0x5563cd331860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfaa350640 1 --2- 172.21.15.103:0/2412460358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd31a5d0 0x5563cd331860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfaa350640 1 -- 172.21.15.103:0/2412460358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563cd32ed70 con 0x5563cd329780 2024-09-19T06:52:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdfaa350640 1 --2- 172.21.15.103:0/2412460358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd329780 0x5563cd32e830 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c02f6e0 tx=0x7fdf9c02f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e083503c1806d8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdf8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.694+0000 7fdf8d7fa640 1 -- 172.21.15.103:0/2412460358 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c034040 con 0x5563cd329780 2024-09-19T06:52:14.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.695+0000 7fdf8d7fa640 1 -- 172.21.15.103:0/2412460358 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fdf9c004250 con 0x5563cd329780 2024-09-19T06:52:14.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.695+0000 7fdfae18c740 1 -- 172.21.15.103:0/2412460358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf94010ca0 con 0x5563cd329780 2024-09-19T06:52:14.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.695+0000 7fdf8d7fa640 1 -- 172.21.15.103:0/2412460358 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf9c004450 con 0x5563cd329780 2024-09-19T06:52:14.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.695+0000 7fdfae18c740 10 client.16566 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:14.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.695+0000 7fdfae18c740 1 -- 172.21.15.103:0/2412460358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5563cd3a4790 con 0x5563cd329780 2024-09-19T06:52:14.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.695+0000 7fdf8d7fa640 1 -- 172.21.15.103:0/2412460358 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf94010ca0 con 0x5563cd329780 2024-09-19T06:52:14.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.695+0000 7fdf8d7fa640 1 -- 172.21.15.103:0/2412460358 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdf9c030440 con 0x5563cd329780 2024-09-19T06:52:14.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.695+0000 7fdfae18c740 10 client.16566 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:14.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.695+0000 7fdfae18c740 10 client.16566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:14.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.695+0000 7fdfae18c740 20 client.16566 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:14.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.695+0000 7fdfae18c740 1 --2- 172.21.15.103:0/2412460358 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5563cd3a4f40 0x5563cd3a7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.695+0000 7fdfae18c740 4 client.16566 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:52:14.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.695+0000 7fdfae18c740 1 -- 172.21.15.103:0/2412460358 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563cd287d70 con 0x5563cd3a4f40 2024-09-19T06:52:14.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.695+0000 7fdfa9b4f640 1 --2- 172.21.15.103:0/2412460358 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5563cd3a4f40 0x5563cd3a7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdf8d7fa640 10 client.16566 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdfa9b4f640 1 --2- 172.21.15.103:0/2412460358 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5563cd3a4f40 0x5563cd3a7360 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d2382c0382dc3b1 server_cookie=28e48c4bc5dbdd91 in_seq=0 out_seq=0 2024-09-19T06:52:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdf8d7fa640 1 -- 172.21.15.103:0/2412460358 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563cd287d70 con 0x5563cd3a4f40 2024-09-19T06:52:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdf8d7fa640 10 client.16566 handle_command_reply: tid=0 2024-09-19T06:52:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdfae18c740 1 client.16566 shutdown 2024-09-19T06:52:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdfae18c740 1 -- 172.21.15.103:0/2412460358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd329780 msgr2=0x5563cd32e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdfae18c740 1 --2- 172.21.15.103:0/2412460358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd329780 0x5563cd32e830 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c02f6e0 tx=0x7fdf9c02f710 comp rx=0 tx=0).stop 2024-09-19T06:52:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdfae18c740 1 -- 172.21.15.103:0/2412460358 shutdown_connections 2024-09-19T06:52:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdfae18c740 1 --2- 172.21.15.103:0/2412460358 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5563cd3a4f40 0x5563cd3a7360 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdfae18c740 1 --2- 172.21.15.103:0/2412460358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd329780 0x5563cd32e830 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdfae18c740 1 --2- 172.21.15.103:0/2412460358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563cd31bb40 0x5563cd331da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdfae18c740 1 --2- 172.21.15.103:0/2412460358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563cd31a5d0 0x5563cd331860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdf8d7fa640 0 client.16566 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:14.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdfae18c740 1 -- 172.21.15.103:0/2412460358 >> 172.21.15.103:0/2412460358 conn(0x5563cd2e3310 msgr2=0x5563cd3154e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:14.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdfae18c740 1 -- 172.21.15.103:0/2412460358 shutdown_connections 2024-09-19T06:52:14.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdfae18c740 1 -- 172.21.15.103:0/2412460358 wait complete. 2024-09-19T06:52:14.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.696+0000 7fdfae18c740 20 client.16566 trim_cache size 0 max 16384 2024-09-19T06:52:14.704 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:52:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.697+0000 7fdfabdb3640 1 -- 172.21.15.103:0/1860616660 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdf900a33b0 msgr2=0x7fdf900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.697+0000 7fdfabdb3640 1 --2- 172.21.15.103:0/1860616660 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdf900a33b0 0x7fdf900a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c02ae70 tx=0x7fdf9c0095c0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.697+0000 7fdfabdb3640 1 -- 172.21.15.103:0/1860616660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa4102570 msgr2=0x7fdfa4198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.697+0000 7fdfabdb3640 1 --2- 172.21.15.103:0/1860616660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa4102570 0x7fdfa4198b40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400b4a0 tx=0x7fdf9400b960 comp rx=0 tx=0).stop 2024-09-19T06:52:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.697+0000 7fdfabdb3640 1 -- 172.21.15.103:0/1860616660 shutdown_connections 2024-09-19T06:52:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.697+0000 7fdfabdb3640 1 --2- 172.21.15.103:0/1860616660 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdf900a33b0 0x7fdf900a5870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.697+0000 7fdfabdb3640 1 --2- 172.21.15.103:0/1860616660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfa4102ef0 0x7fdfa419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.697+0000 7fdfabdb3640 1 --2- 172.21.15.103:0/1860616660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa4102570 0x7fdfa4198b40 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.697+0000 7fdfabdb3640 1 --2- 172.21.15.103:0/1860616660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdfa4100e20 0x7fdfa4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.697+0000 7fdfabdb3640 1 -- 172.21.15.103:0/1860616660 >> 172.21.15.103:0/1860616660 conn(0x7fdfa40fc7d0 msgr2=0x7fdfa4069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.697+0000 7fdfabdb3640 1 -- 172.21.15.103:0/1860616660 shutdown_connections 2024-09-19T06:52:14.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:14.697+0000 7fdfabdb3640 1 -- 172.21.15.103:0/1860616660 wait complete. 2024-09-19T06:52:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.117+0000 7f1e473c4640 1 -- 172.21.15.103:0/1412933182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e401009f0 msgr2=0x7f1e40100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.117+0000 7f1e473c4640 1 --2- 172.21.15.103:0/1412933182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e401009f0 0x7f1e40100e50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009930 tx=0x7f1e3002a420 comp rx=0 tx=0).stop 2024-09-19T06:52:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.117+0000 7f1e473c4640 1 -- 172.21.15.103:0/1412933182 shutdown_connections 2024-09-19T06:52:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.117+0000 7f1e473c4640 1 --2- 172.21.15.103:0/1412933182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e40101390 0x7f1e4010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.117+0000 7f1e473c4640 1 --2- 172.21.15.103:0/1412933182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e401009f0 0x7f1e40100e50 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.117+0000 7f1e473c4640 1 --2- 172.21.15.103:0/1412933182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e40106a00 0x7f1e40106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.117+0000 7f1e473c4640 1 -- 172.21.15.103:0/1412933182 >> 172.21.15.103:0/1412933182 conn(0x7f1e400fc6b0 msgr2=0x7f1e400fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.118+0000 7f1e473c4640 1 -- 172.21.15.103:0/1412933182 shutdown_connections 2024-09-19T06:52:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.118+0000 7f1e473c4640 1 -- 172.21.15.103:0/1412933182 wait complete. 2024-09-19T06:52:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.118+0000 7f1e473c4640 1 Processor -- start 2024-09-19T06:52:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.118+0000 7f1e473c4640 1 -- start start 2024-09-19T06:52:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.118+0000 7f1e473c4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e401009f0 0x7f1e401992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.118+0000 7f1e473c4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e40101390 0x7f1e401997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.118+0000 7f1e473c4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e40106a00 0x7f1e401a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.118+0000 7f1e473c4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1e40110e40 con 0x7f1e40106a00 2024-09-19T06:52:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.118+0000 7f1e473c4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1e40110cc0 con 0x7f1e40101390 2024-09-19T06:52:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.118+0000 7f1e473c4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e40110fc0 con 0x7f1e401009f0 2024-09-19T06:52:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.118+0000 7f1e45961640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e40106a00 0x7f1e401a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.119+0000 7f1e45961640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e40106a00 0x7f1e401a4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53644/0 (socket says 172.21.15.103:53644) 2024-09-19T06:52:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.119+0000 7f1e45961640 1 -- 172.21.15.103:0/3107702864 learned_addr learned my addr 172.21.15.103:0/3107702864 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.119+0000 7f1e4495f640 1 --2- 172.21.15.103:0/3107702864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e40101390 0x7f1e401997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.119+0000 7f1e45160640 1 --2- 172.21.15.103:0/3107702864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e401009f0 0x7f1e401992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.119+0000 7f1e45961640 1 -- 172.21.15.103:0/3107702864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e401009f0 msgr2=0x7f1e401992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.119+0000 7f1e45961640 1 --2- 172.21.15.103:0/3107702864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e401009f0 0x7f1e401992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.119+0000 7f1e45961640 1 -- 172.21.15.103:0/3107702864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e40101390 msgr2=0x7f1e401997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.119+0000 7f1e45961640 1 --2- 172.21.15.103:0/3107702864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e40101390 0x7f1e401997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.119+0000 7f1e45961640 1 -- 172.21.15.103:0/3107702864 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e401a4870 con 0x7f1e40106a00 2024-09-19T06:52:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.119+0000 7f1e45160640 1 --2- 172.21.15.103:0/3107702864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e401009f0 0x7f1e401992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.119+0000 7f1e45961640 1 --2- 172.21.15.103:0/3107702864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e40106a00 0x7f1e401a4330 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009fd0 tx=0x7f1e3800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.119+0000 7f1e367fc640 1 -- 172.21.15.103:0/3107702864 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38017070 con 0x7f1e40106a00 2024-09-19T06:52:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.119+0000 7f1e367fc640 1 -- 172.21.15.103:0/3107702864 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e38003e50 con 0x7f1e40106a00 2024-09-19T06:52:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.119+0000 7f1e473c4640 1 -- 172.21.15.103:0/3107702864 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e401a4a40 con 0x7f1e40106a00 2024-09-19T06:52:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.119+0000 7f1e367fc640 1 -- 172.21.15.103:0/3107702864 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e380106c0 con 0x7f1e40106a00 2024-09-19T06:52:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.119+0000 7f1e473c4640 1 -- 172.21.15.103:0/3107702864 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e401a4f10 con 0x7f1e40106a00 2024-09-19T06:52:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.122+0000 7f1e367fc640 1 -- 172.21.15.103:0/3107702864 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1e38003970 con 0x7f1e40106a00 2024-09-19T06:52:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.122+0000 7f1e4a238740 1 Processor -- start 2024-09-19T06:52:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.122+0000 7f1e4a238740 1 -- start start 2024-09-19T06:52:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.122+0000 7f1e4a238740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef556caf0 0x560ef556ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.122+0000 7f1e4a238740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556d410 0x560ef5577770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.122+0000 7f1e4a238740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef5577cb0 0x560ef557a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.122+0000 7f1e4a238740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560ef54d12f0 con 0x560ef556d410 2024-09-19T06:52:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.122+0000 7f1e4a238740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560ef5479270 con 0x560ef556caf0 2024-09-19T06:52:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.122+0000 7f1e4a238740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560ef54cd2b0 con 0x560ef5577cb0 2024-09-19T06:52:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e45160640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556d410 0x560ef5577770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e45160640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556d410 0x560ef5577770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53660/0 (socket says 172.21.15.103:53660) 2024-09-19T06:52:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e45160640 1 -- 172.21.15.103:0/1804555654 learned_addr learned my addr 172.21.15.103:0/1804555654 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e4495f640 1 --2- 172.21.15.103:0/1804555654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef556caf0 0x560ef556ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e4495f640 1 --2- 172.21.15.103:0/1804555654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef5577cb0 0x560ef557a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e367fc640 1 --2- 172.21.15.103:0/3107702864 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1e2c0a3460 0x7f1e2c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e45961640 1 --2- 172.21.15.103:0/3107702864 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1e2c0a3460 0x7f1e2c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e45160640 1 -- 172.21.15.103:0/1804555654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef5577cb0 msgr2=0x560ef557a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e45160640 1 --2- 172.21.15.103:0/1804555654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef5577cb0 0x560ef557a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e45160640 1 -- 172.21.15.103:0/1804555654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef556caf0 msgr2=0x560ef556ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e45160640 1 --2- 172.21.15.103:0/1804555654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef556caf0 0x560ef556ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e45160640 1 -- 172.21.15.103:0/1804555654 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ef5478340 con 0x560ef556d410 2024-09-19T06:52:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e367fc640 1 -- 172.21.15.103:0/3107702864 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1e380cfb60 con 0x7f1e40106a00 2024-09-19T06:52:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e45160640 1 --2- 172.21.15.103:0/1804555654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556d410 0x560ef5577770 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00aa10 tx=0x7f1e3c00aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1079305046c17b52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e2b7fe640 1 -- 172.21.15.103:0/1804555654 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c015070 con 0x560ef556d410 2024-09-19T06:52:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e2b7fe640 1 -- 172.21.15.103:0/1804555654 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e3c00cbe0 con 0x560ef556d410 2024-09-19T06:52:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.123+0000 7f1e2b7fe640 1 -- 172.21.15.103:0/1804555654 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c018580 con 0x560ef556d410 2024-09-19T06:52:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.124+0000 7f1e4a238740 1 -- 172.21.15.103:0/1804555654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556d410 msgr2=0x560ef5577770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.124+0000 7f1e45961640 1 --2- 172.21.15.103:0/3107702864 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1e2c0a3460 0x7f1e2c0a5920 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1e380982b0 tx=0x7f1e38004020 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.124+0000 7f1e4a238740 1 --2- 172.21.15.103:0/1804555654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556d410 0x560ef5577770 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00aa10 tx=0x7f1e3c00aed0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.124+0000 7f1e4a238740 1 -- 172.21.15.103:0/1804555654 shutdown_connections 2024-09-19T06:52:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.124+0000 7f1e4a238740 1 --2- 172.21.15.103:0/1804555654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef5577cb0 0x560ef557a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.124+0000 7f1e4a238740 1 --2- 172.21.15.103:0/1804555654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556d410 0x560ef5577770 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.124+0000 7f1e4a238740 1 --2- 172.21.15.103:0/1804555654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef556caf0 0x560ef556ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.124+0000 7f1e4a238740 1 -- 172.21.15.103:0/1804555654 >> 172.21.15.103:0/1804555654 conn(0x560ef5569060 msgr2=0x560ef5569440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.124+0000 7f1e4a238740 1 -- 172.21.15.103:0/1804555654 shutdown_connections 2024-09-19T06:52:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.124+0000 7f1e4a238740 1 -- 172.21.15.103:0/1804555654 wait complete. 2024-09-19T06:52:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4a238740 1 -- start start 2024-09-19T06:52:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4a238740 1 Processor -- start 2024-09-19T06:52:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4a238740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556caf0 0x560ef5574b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4a238740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef556d410 0x560ef5575050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4a238740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef5577cb0 0x560ef5575590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4a238740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560ef557a880 con 0x560ef556caf0 2024-09-19T06:52:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4a238740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560ef545a290 con 0x560ef5577cb0 2024-09-19T06:52:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4a238740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560ef557aa00 con 0x560ef556d410 2024-09-19T06:52:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4a238740 10 client.0 _ll_register_callbacks cb 0x560ef548b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4495f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556caf0 0x560ef5574b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4495f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556caf0 0x560ef5574b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53670/0 (socket says 172.21.15.103:53670) 2024-09-19T06:52:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4495f640 1 -- 172.21.15.103:0/2298455284 learned_addr learned my addr 172.21.15.103:0/2298455284 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4495f640 1 --2- 172.21.15.103:0/2298455284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef5577cb0 0x560ef5575590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4495f640 1 -- 172.21.15.103:0/2298455284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef556d410 msgr2=0x560ef5575050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4495f640 1 --2- 172.21.15.103:0/2298455284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef556d410 0x560ef5575050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4495f640 1 -- 172.21.15.103:0/2298455284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef5577cb0 msgr2=0x560ef5575590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4495f640 1 --2- 172.21.15.103:0/2298455284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef5577cb0 0x560ef5575590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.125+0000 7f1e4495f640 1 -- 172.21.15.103:0/2298455284 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ef51e4210 con 0x560ef556caf0 2024-09-19T06:52:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.126+0000 7f1e4495f640 1 --2- 172.21.15.103:0/2298455284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556caf0 0x560ef5574b10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1e30033000 tx=0x7f1e30005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3f22a418fba6af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.126+0000 7f1e28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.126+0000 7f1e28ff9640 1 -- 172.21.15.103:0/2298455284 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e30035040 con 0x560ef556caf0 2024-09-19T06:52:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.126+0000 7f1e28ff9640 1 -- 172.21.15.103:0/2298455284 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1e30004450 con 0x560ef556caf0 2024-09-19T06:52:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.126+0000 7f1e4a238740 1 -- 172.21.15.103:0/2298455284 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e3c00cbe0 con 0x560ef556caf0 2024-09-19T06:52:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.126+0000 7f1e28ff9640 1 -- 172.21.15.103:0/2298455284 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e30051050 con 0x560ef556caf0 2024-09-19T06:52:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.126+0000 7f1e4a238740 10 client.16578 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.126+0000 7f1e4a238740 1 -- 172.21.15.103:0/2298455284 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ef52b0620 con 0x560ef556caf0 2024-09-19T06:52:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.126+0000 7f1e28ff9640 1 -- 172.21.15.103:0/2298455284 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e3c00cbe0 con 0x560ef556caf0 2024-09-19T06:52:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.126+0000 7f1e28ff9640 1 -- 172.21.15.103:0/2298455284 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1e30048070 con 0x560ef556caf0 2024-09-19T06:52:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.126+0000 7f1e4a238740 10 client.16578 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.126+0000 7f1e4a238740 10 client.16578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.126+0000 7f1e4a238740 20 client.16578 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.126+0000 7f1e4a238740 1 --2- 172.21.15.103:0/2298455284 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560ef5587bc0 0x560ef558a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.126+0000 7f1e4a238740 4 client.16578 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.126+0000 7f1e4a238740 1 -- 172.21.15.103:0/2298455284 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ef54dea20 con 0x560ef5587bc0 2024-09-19T06:52:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.127+0000 7f1e45160640 1 --2- 172.21.15.103:0/2298455284 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560ef5587bc0 0x560ef558a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.127+0000 7f1e45160640 1 --2- 172.21.15.103:0/2298455284 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560ef5587bc0 0x560ef558a030 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d989e860f3ca3eab server_cookie=9193ea0645855a40 in_seq=0 out_seq=0 2024-09-19T06:52:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.127+0000 7f1e28ff9640 10 client.16578 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.128+0000 7f1e28ff9640 1 -- 172.21.15.103:0/2298455284 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ef54dea20 con 0x560ef5587bc0 2024-09-19T06:52:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.128+0000 7f1e28ff9640 10 client.16578 handle_command_reply: tid=0 2024-09-19T06:52:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.128+0000 7f1e4a238740 1 client.16578 shutdown 2024-09-19T06:52:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.128+0000 7f1e4a238740 1 -- 172.21.15.103:0/2298455284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556caf0 msgr2=0x560ef5574b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.128+0000 7f1e4a238740 1 --2- 172.21.15.103:0/2298455284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556caf0 0x560ef5574b10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1e30033000 tx=0x7f1e30005d50 comp rx=0 tx=0).stop 2024-09-19T06:52:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.128+0000 7f1e4a238740 1 -- 172.21.15.103:0/2298455284 shutdown_connections 2024-09-19T06:52:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.128+0000 7f1e4a238740 1 --2- 172.21.15.103:0/2298455284 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560ef5587bc0 0x560ef558a030 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.128+0000 7f1e4a238740 1 --2- 172.21.15.103:0/2298455284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef5577cb0 0x560ef5575590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.128+0000 7f1e28ff9640 0 client.16578 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.128+0000 7f1e4a238740 1 --2- 172.21.15.103:0/2298455284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef556d410 0x560ef5575050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.128+0000 7f1e4a238740 1 --2- 172.21.15.103:0/2298455284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556caf0 0x560ef5574b10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.128+0000 7f1e4a238740 1 -- 172.21.15.103:0/2298455284 >> 172.21.15.103:0/2298455284 conn(0x560ef5569060 msgr2=0x560ef556edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.128+0000 7f1e4a238740 1 -- 172.21.15.103:0/2298455284 shutdown_connections 2024-09-19T06:52:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.128+0000 7f1e4a238740 1 -- 172.21.15.103:0/2298455284 wait complete. 2024-09-19T06:52:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.128+0000 7f1e4a238740 20 client.16578 trim_cache size 0 max 16384 2024-09-19T06:52:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e4a238740 1 Processor -- start 2024-09-19T06:52:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e4a238740 1 -- start start 2024-09-19T06:52:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e4a238740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef556caf0 0x560ef55753c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e4a238740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556d410 0x560ef55680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e4a238740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef5577cb0 0x560ef558f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e4a238740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401ad710 con 0x560ef556d410 2024-09-19T06:52:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e4a238740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401af250 con 0x560ef5577cb0 2024-09-19T06:52:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e4a238740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401adde0 con 0x560ef556caf0 2024-09-19T06:52:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e45160640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556d410 0x560ef55680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e45160640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556d410 0x560ef55680b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53678/0 (socket says 172.21.15.103:53678) 2024-09-19T06:52:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e45160640 1 -- 172.21.15.103:0/2471497353 learned_addr learned my addr 172.21.15.103:0/2471497353 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e4495f640 1 --2- 172.21.15.103:0/2471497353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef5577cb0 0x560ef558f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e45160640 1 -- 172.21.15.103:0/2471497353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef556caf0 msgr2=0x560ef55753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e45160640 1 --2- 172.21.15.103:0/2471497353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef556caf0 0x560ef55753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e45160640 1 -- 172.21.15.103:0/2471497353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef5577cb0 msgr2=0x560ef558f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e45160640 1 --2- 172.21.15.103:0/2471497353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef5577cb0 0x560ef558f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e45160640 1 -- 172.21.15.103:0/2471497353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ef5569ce0 con 0x560ef556d410 2024-09-19T06:52:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e4495f640 1 --2- 172.21.15.103:0/2471497353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef5577cb0 0x560ef558f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.143+0000 7f1e45160640 1 --2- 172.21.15.103:0/2471497353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556d410 0x560ef55680b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c011d60 tx=0x7f1e3c011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6daa3cccd05432fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.144+0000 7f1e2affd640 1 -- 172.21.15.103:0/2471497353 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c02a070 con 0x560ef556d410 2024-09-19T06:52:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.144+0000 7f1e2affd640 1 -- 172.21.15.103:0/2471497353 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e3c00cd30 con 0x560ef556d410 2024-09-19T06:52:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.144+0000 7f1e2affd640 1 -- 172.21.15.103:0/2471497353 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c015070 con 0x560ef556d410 2024-09-19T06:52:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.144+0000 7f1e4a238740 1 -- 172.21.15.103:0/2471497353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556d410 msgr2=0x560ef55680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.144+0000 7f1e4a238740 1 --2- 172.21.15.103:0/2471497353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556d410 0x560ef55680b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c011d60 tx=0x7f1e3c011ed0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.144+0000 7f1e4a238740 1 -- 172.21.15.103:0/2471497353 shutdown_connections 2024-09-19T06:52:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.144+0000 7f1e4a238740 1 --2- 172.21.15.103:0/2471497353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef5577cb0 0x560ef558f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.144+0000 7f1e4a238740 1 --2- 172.21.15.103:0/2471497353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556d410 0x560ef55680b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.144+0000 7f1e4a238740 1 --2- 172.21.15.103:0/2471497353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef556caf0 0x560ef55753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.144+0000 7f1e4a238740 1 -- 172.21.15.103:0/2471497353 >> 172.21.15.103:0/2471497353 conn(0x560ef5569060 msgr2=0x560ef55679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.144+0000 7f1e4a238740 1 -- 172.21.15.103:0/2471497353 shutdown_connections 2024-09-19T06:52:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.144+0000 7f1e4a238740 1 -- 172.21.15.103:0/2471497353 wait complete. 2024-09-19T06:52:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.145+0000 7f1e4a238740 1 -- start start 2024-09-19T06:52:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.145+0000 7f1e4a238740 1 Processor -- start 2024-09-19T06:52:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.145+0000 7f1e4a238740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556caf0 0x560ef557b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.145+0000 7f1e4a238740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef556d410 0x560ef557be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.145+0000 7f1e4a238740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef5577cb0 0x560ef557c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.145+0000 7f1e4a238740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401aeb80 con 0x560ef556caf0 2024-09-19T06:52:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.145+0000 7f1e4a238740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401ae4b0 con 0x560ef5577cb0 2024-09-19T06:52:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.145+0000 7f1e4a238740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401ab6d0 con 0x560ef556d410 2024-09-19T06:52:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.145+0000 7f1e4a238740 10 client.0 _ll_register_callbacks cb 0x7f1e30031710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.145+0000 7f1e4495f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556caf0 0x560ef557b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.145+0000 7f1e4495f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556caf0 0x560ef557b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53690/0 (socket says 172.21.15.103:53690) 2024-09-19T06:52:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.145+0000 7f1e4495f640 1 -- 172.21.15.103:0/299063435 learned_addr learned my addr 172.21.15.103:0/299063435 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.145+0000 7f1e45160640 1 --2- 172.21.15.103:0/299063435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef556d410 0x560ef557be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e4495f640 1 -- 172.21.15.103:0/299063435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef556d410 msgr2=0x560ef557be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e4495f640 1 --2- 172.21.15.103:0/299063435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef556d410 0x560ef557be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e4495f640 1 -- 172.21.15.103:0/299063435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef5577cb0 msgr2=0x560ef557c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e4495f640 1 --2- 172.21.15.103:0/299063435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef5577cb0 0x560ef557c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e4495f640 1 -- 172.21.15.103:0/299063435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ef55f1910 con 0x560ef556caf0 2024-09-19T06:52:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e45160640 1 --2- 172.21.15.103:0/299063435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef556d410 0x560ef557be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e4495f640 1 --2- 172.21.15.103:0/299063435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556caf0 0x560ef557b930 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1e3002a940 tx=0x7f1e3003d6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26f769b9deda643 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e28ff9640 1 -- 172.21.15.103:0/299063435 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3005a060 con 0x560ef556caf0 2024-09-19T06:52:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e28ff9640 1 -- 172.21.15.103:0/299063435 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1e30004460 con 0x560ef556caf0 2024-09-19T06:52:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e4a238740 1 -- 172.21.15.103:0/299063435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e3c00cd30 con 0x560ef556caf0 2024-09-19T06:52:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e28ff9640 1 -- 172.21.15.103:0/299063435 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e30035040 con 0x560ef556caf0 2024-09-19T06:52:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e4a238740 10 client.16584 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e4a238740 1 -- 172.21.15.103:0/299063435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ef55f2b30 con 0x560ef556caf0 2024-09-19T06:52:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e28ff9640 1 -- 172.21.15.103:0/299063435 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e3c00cd30 con 0x560ef556caf0 2024-09-19T06:52:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e28ff9640 1 -- 172.21.15.103:0/299063435 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1e30048050 con 0x560ef556caf0 2024-09-19T06:52:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e4a238740 10 client.16584 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e4a238740 10 client.16584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e4a238740 20 client.16584 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e4a238740 1 --2- 172.21.15.103:0/299063435 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560ef55f3280 0x560ef55f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e4a238740 4 client.16584 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:52:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.146+0000 7f1e4a238740 1 -- 172.21.15.103:0/299063435 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ef54d8d70 con 0x560ef55f3280 2024-09-19T06:52:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.147+0000 7f1e45160640 1 --2- 172.21.15.103:0/299063435 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560ef55f3280 0x560ef55f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.147+0000 7f1e45160640 1 --2- 172.21.15.103:0/299063435 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560ef55f3280 0x560ef55f56a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f15a80f21428e5a8 server_cookie=c2c5f00a170fd92b in_seq=0 out_seq=0 2024-09-19T06:52:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.147+0000 7f1e28ff9640 10 client.16584 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.161+0000 7f1e28ff9640 1 -- 172.21.15.103:0/299063435 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560ef54d8d70 con 0x560ef55f3280 2024-09-19T06:52:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.161+0000 7f1e28ff9640 10 client.16584 handle_command_reply: tid=0 2024-09-19T06:52:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.161+0000 7f1e4a238740 1 client.16584 shutdown 2024-09-19T06:52:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.161+0000 7f1e4a238740 1 -- 172.21.15.103:0/299063435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556caf0 msgr2=0x560ef557b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.161+0000 7f1e4a238740 1 --2- 172.21.15.103:0/299063435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556caf0 0x560ef557b930 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1e3002a940 tx=0x7f1e3003d6f0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.161+0000 7f1e4a238740 1 -- 172.21.15.103:0/299063435 shutdown_connections 2024-09-19T06:52:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.161+0000 7f1e4a238740 1 --2- 172.21.15.103:0/299063435 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560ef55f3280 0x560ef55f56a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.161+0000 7f1e4a238740 1 --2- 172.21.15.103:0/299063435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ef5577cb0 0x560ef557c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.161+0000 7f1e4a238740 1 --2- 172.21.15.103:0/299063435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef556d410 0x560ef557be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.161+0000 7f1e28ff9640 0 client.16584 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.161+0000 7f1e4a238740 1 --2- 172.21.15.103:0/299063435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ef556caf0 0x560ef557b930 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.161+0000 7f1e4a238740 1 -- 172.21.15.103:0/299063435 >> 172.21.15.103:0/299063435 conn(0x560ef5569060 msgr2=0x560ef55681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.161+0000 7f1e4a238740 1 -- 172.21.15.103:0/299063435 shutdown_connections 2024-09-19T06:52:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.161+0000 7f1e4a238740 1 -- 172.21.15.103:0/299063435 wait complete. 2024-09-19T06:52:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.161+0000 7f1e4a238740 20 client.16584 trim_cache size 0 max 16384 2024-09-19T06:52:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:52:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:52:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:52:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:52:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.162+0000 7f1e473c4640 1 -- 172.21.15.103:0/3107702864 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1e2c0a3460 msgr2=0x7f1e2c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.162+0000 7f1e473c4640 1 --2- 172.21.15.103:0/3107702864 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1e2c0a3460 0x7f1e2c0a5920 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1e380982b0 tx=0x7f1e38004020 comp rx=0 tx=0).stop 2024-09-19T06:52:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.162+0000 7f1e473c4640 1 -- 172.21.15.103:0/3107702864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e40106a00 msgr2=0x7f1e401a4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.162+0000 7f1e473c4640 1 --2- 172.21.15.103:0/3107702864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e40106a00 0x7f1e401a4330 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009fd0 tx=0x7f1e3800b260 comp rx=0 tx=0).stop 2024-09-19T06:52:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.162+0000 7f1e473c4640 1 -- 172.21.15.103:0/3107702864 shutdown_connections 2024-09-19T06:52:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.162+0000 7f1e473c4640 1 --2- 172.21.15.103:0/3107702864 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1e2c0a3460 0x7f1e2c0a5920 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.162+0000 7f1e473c4640 1 --2- 172.21.15.103:0/3107702864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e40106a00 0x7f1e401a4330 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.162+0000 7f1e473c4640 1 --2- 172.21.15.103:0/3107702864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e40101390 0x7f1e401997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.162+0000 7f1e473c4640 1 --2- 172.21.15.103:0/3107702864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e401009f0 0x7f1e401992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.162+0000 7f1e473c4640 1 -- 172.21.15.103:0/3107702864 >> 172.21.15.103:0/3107702864 conn(0x7f1e400fc6b0 msgr2=0x7f1e400fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.162+0000 7f1e473c4640 1 -- 172.21.15.103:0/3107702864 shutdown_connections 2024-09-19T06:52:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:16.162+0000 7f1e473c4640 1 -- 172.21.15.103:0/3107702864 wait complete. 2024-09-19T06:52:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:16 smithi174 ceph-mon[35637]: pgmap v94: 97 pgs: 97 active+clean; 2.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 60 KiB/s wr, 37 op/s 2024-09-19T06:52:16.373 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:52:16.373 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:52:16.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:16 smithi103 ceph-mon[31158]: pgmap v94: 97 pgs: 97 active+clean; 2.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 60 KiB/s wr, 37 op/s 2024-09-19T06:52:16.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:16 smithi134 ceph-mon[35356]: pgmap v94: 97 pgs: 97 active+clean; 2.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 60 KiB/s wr, 37 op/s 2024-09-19T06:52:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:17.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.063+0000 7fed47578640 1 -- 172.21.15.103:0/1805873447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40106a80 msgr2=0x7fed40106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:17.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.063+0000 7fed47578640 1 --2- 172.21.15.103:0/1805873447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40106a80 0x7fed40106e60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fed34009930 tx=0x7fed3402a420 comp rx=0 tx=0).stop 2024-09-19T06:52:17.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.063+0000 7fed47578640 1 -- 172.21.15.103:0/1805873447 shutdown_connections 2024-09-19T06:52:17.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.063+0000 7fed47578640 1 --2- 172.21.15.103:0/1805873447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed40101420 0x7fed4010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.063+0000 7fed47578640 1 --2- 172.21.15.103:0/1805873447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed40100a80 0x7fed40100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.063+0000 7fed47578640 1 --2- 172.21.15.103:0/1805873447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40106a80 0x7fed40106e60 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.063+0000 7fed47578640 1 -- 172.21.15.103:0/1805873447 >> 172.21.15.103:0/1805873447 conn(0x7fed400fc7a0 msgr2=0x7fed400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:17.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.063+0000 7fed47578640 1 -- 172.21.15.103:0/1805873447 shutdown_connections 2024-09-19T06:52:17.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.063+0000 7fed47578640 1 -- 172.21.15.103:0/1805873447 wait complete. 2024-09-19T06:52:17.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.064+0000 7fed47578640 1 Processor -- start 2024-09-19T06:52:17.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.064+0000 7fed47578640 1 -- start start 2024-09-19T06:52:17.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.064+0000 7fed47578640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed40100a80 0x7fed401992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.064+0000 7fed47578640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed40101420 0x7fed40199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.064+0000 7fed47578640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40106a80 0x7fed401a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.064+0000 7fed47578640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed40110e40 con 0x7fed40106a80 2024-09-19T06:52:17.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.064+0000 7fed47578640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed40110cc0 con 0x7fed40100a80 2024-09-19T06:52:17.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.064+0000 7fed47578640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed40110fc0 con 0x7fed40101420 2024-09-19T06:52:17.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.064+0000 7fed45b15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40106a80 0x7fed401a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.064+0000 7fed45b15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40106a80 0x7fed401a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53708/0 (socket says 172.21.15.103:53708) 2024-09-19T06:52:17.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.064+0000 7fed45b15640 1 -- 172.21.15.103:0/398177296 learned_addr learned my addr 172.21.15.103:0/398177296 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:17.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.064+0000 7fed45314640 1 --2- 172.21.15.103:0/398177296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed40100a80 0x7fed401992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.065+0000 7fed44b13640 1 --2- 172.21.15.103:0/398177296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed40101420 0x7fed40199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.065+0000 7fed45b15640 1 -- 172.21.15.103:0/398177296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed40101420 msgr2=0x7fed40199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:17.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.065+0000 7fed45b15640 1 --2- 172.21.15.103:0/398177296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed40101420 0x7fed40199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.065+0000 7fed45b15640 1 -- 172.21.15.103:0/398177296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed40100a80 msgr2=0x7fed401992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:17.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.065+0000 7fed45b15640 1 --2- 172.21.15.103:0/398177296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed40100a80 0x7fed401992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.065+0000 7fed45b15640 1 -- 172.21.15.103:0/398177296 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed401a48b0 con 0x7fed40106a80 2024-09-19T06:52:17.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.065+0000 7fed45b15640 1 --2- 172.21.15.103:0/398177296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40106a80 0x7fed401a4370 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fed3800eaa0 tx=0x7fed3800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:17.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.065+0000 7fed327fc640 1 -- 172.21.15.103:0/398177296 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3800cdb0 con 0x7fed40106a80 2024-09-19T06:52:17.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.065+0000 7fed327fc640 1 -- 172.21.15.103:0/398177296 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed38005020 con 0x7fed40106a80 2024-09-19T06:52:17.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.065+0000 7fed47578640 1 -- 172.21.15.103:0/398177296 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed401a4a80 con 0x7fed40106a80 2024-09-19T06:52:17.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.065+0000 7fed327fc640 1 -- 172.21.15.103:0/398177296 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed380116b0 con 0x7fed40106a80 2024-09-19T06:52:17.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.065+0000 7fed47578640 1 -- 172.21.15.103:0/398177296 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed401a4f40 con 0x7fed40106a80 2024-09-19T06:52:17.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.067+0000 7fed327fc640 1 -- 172.21.15.103:0/398177296 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fed38005560 con 0x7fed40106a80 2024-09-19T06:52:17.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed49951740 1 Processor -- start 2024-09-19T06:52:17.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed49951740 1 -- start start 2024-09-19T06:52:17.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed49951740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bba40 0x563e6b4bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed49951740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 0x563e6b4cac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed49951740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4cb150 0x563e6b4cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed49951740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563e6b421280 con 0x563e6b4bba40 2024-09-19T06:52:17.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed49951740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563e6b3c8b10 con 0x563e6b4bc360 2024-09-19T06:52:17.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed49951740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563e6b41d240 con 0x563e6b4cb150 2024-09-19T06:52:17.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed44b13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bba40 0x563e6b4bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed327fc640 1 --2- 172.21.15.103:0/398177296 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed2c0a33b0 0x7fed2c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed44b13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bba40 0x563e6b4bbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53712/0 (socket says 172.21.15.103:53712) 2024-09-19T06:52:17.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed44b13640 1 -- 172.21.15.103:0/358085790 learned_addr learned my addr 172.21.15.103:0/358085790 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:17.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed45314640 1 --2- 172.21.15.103:0/358085790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 0x563e6b4cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed45b15640 1 --2- 172.21.15.103:0/398177296 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed2c0a33b0 0x7fed2c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed327fc640 1 -- 172.21.15.103:0/398177296 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fed380cff20 con 0x7fed40106a80 2024-09-19T06:52:17.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed44b13640 1 -- 172.21.15.103:0/358085790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4cb150 msgr2=0x563e6b4cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:17.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed44b13640 1 --2- 172.21.15.103:0/358085790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4cb150 0x563e6b4cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed44b13640 1 -- 172.21.15.103:0/358085790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 msgr2=0x563e6b4cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:17.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed44b13640 1 --2- 172.21.15.103:0/358085790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 0x563e6b4cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.068+0000 7fed44b13640 1 -- 172.21.15.103:0/358085790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6b3d51f0 con 0x563e6b4bba40 2024-09-19T06:52:17.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.069+0000 7fed45314640 1 --2- 172.21.15.103:0/358085790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 0x563e6b4cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:17.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.069+0000 7fed44b13640 1 --2- 172.21.15.103:0/358085790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bba40 0x563e6b4bbe20 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fed3c005e00 tx=0x7fed3c0106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e923131e56f91ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:17.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.069+0000 7fed45b15640 1 --2- 172.21.15.103:0/398177296 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed2c0a33b0 0x7fed2c0a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed3800efd0 tx=0x7fed380bc040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:17.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.069+0000 7fed2b7fe640 1 -- 172.21.15.103:0/358085790 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c016070 con 0x563e6b4bba40 2024-09-19T06:52:17.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.069+0000 7fed2b7fe640 1 -- 172.21.15.103:0/358085790 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed3c012900 con 0x563e6b4bba40 2024-09-19T06:52:17.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.069+0000 7fed2b7fe640 1 -- 172.21.15.103:0/358085790 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c016070 con 0x563e6b4bba40 2024-09-19T06:52:17.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.070+0000 7fed49951740 1 -- 172.21.15.103:0/358085790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bba40 msgr2=0x563e6b4bbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:17.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.070+0000 7fed49951740 1 --2- 172.21.15.103:0/358085790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bba40 0x563e6b4bbe20 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fed3c005e00 tx=0x7fed3c0106e0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.070+0000 7fed49951740 1 -- 172.21.15.103:0/358085790 shutdown_connections 2024-09-19T06:52:17.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.070+0000 7fed49951740 1 --2- 172.21.15.103:0/358085790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4cb150 0x563e6b4cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.070+0000 7fed49951740 1 --2- 172.21.15.103:0/358085790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 0x563e6b4cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.070+0000 7fed49951740 1 --2- 172.21.15.103:0/358085790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bba40 0x563e6b4bbe20 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.070+0000 7fed49951740 1 -- 172.21.15.103:0/358085790 >> 172.21.15.103:0/358085790 conn(0x563e6b4b80f0 msgr2=0x563e6b4b84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:17.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.070+0000 7fed49951740 1 -- 172.21.15.103:0/358085790 shutdown_connections 2024-09-19T06:52:17.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.070+0000 7fed49951740 1 -- 172.21.15.103:0/358085790 wait complete. 2024-09-19T06:52:17.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.070+0000 7fed49951740 1 -- start start 2024-09-19T06:52:17.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.070+0000 7fed49951740 1 Processor -- start 2024-09-19T06:52:17.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.070+0000 7fed49951740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bba40 0x563e6b4c8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed49951740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bc360 0x563e6b4c8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed49951740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4cb150 0x563e6b4c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed49951740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563e6b4cdd20 con 0x563e6b4bc360 2024-09-19T06:52:17.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed49951740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563e6b3dbf10 con 0x563e6b4bba40 2024-09-19T06:52:17.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed49951740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563e6b4cdea0 con 0x563e6b4cb150 2024-09-19T06:52:17.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed49951740 10 client.0 _ll_register_callbacks cb 0x563e6b3a4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:17.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed45314640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bc360 0x563e6b4c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed45314640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bc360 0x563e6b4c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53714/0 (socket says 172.21.15.103:53714) 2024-09-19T06:52:17.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed45314640 1 -- 172.21.15.103:0/4087829230 learned_addr learned my addr 172.21.15.103:0/4087829230 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:17.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed44b13640 1 --2- 172.21.15.103:0/4087829230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4cb150 0x563e6b4c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed44b13640 1 --2- 172.21.15.103:0/4087829230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bba40 0x563e6b4c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed45314640 1 -- 172.21.15.103:0/4087829230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4cb150 msgr2=0x563e6b4c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:17.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed45314640 1 --2- 172.21.15.103:0/4087829230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4cb150 0x563e6b4c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed45314640 1 -- 172.21.15.103:0/4087829230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bba40 msgr2=0x563e6b4c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:17.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed45314640 1 --2- 172.21.15.103:0/4087829230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bba40 0x563e6b4c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed45314640 1 -- 172.21.15.103:0/4087829230 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6b1926a0 con 0x563e6b4bc360 2024-09-19T06:52:17.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed45314640 1 --2- 172.21.15.103:0/4087829230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bc360 0x563e6b4c8570 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fed34009930 tx=0x7fed340318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c8f52ee38e51eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:17.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:17.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed28ff9640 1 -- 172.21.15.103:0/4087829230 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34035040 con 0x563e6b4bc360 2024-09-19T06:52:17.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed28ff9640 1 -- 172.21.15.103:0/4087829230 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fed34004240 con 0x563e6b4bc360 2024-09-19T06:52:17.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed49951740 1 -- 172.21.15.103:0/4087829230 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed3c012900 con 0x563e6b4bc360 2024-09-19T06:52:17.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed28ff9640 1 -- 172.21.15.103:0/4087829230 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed34004440 con 0x563e6b4bc360 2024-09-19T06:52:17.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed49951740 10 client.16596 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:17.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.071+0000 7fed49951740 1 -- 172.21.15.103:0/4087829230 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e6b4d9960 con 0x563e6b4bc360 2024-09-19T06:52:17.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.072+0000 7fed28ff9640 1 -- 172.21.15.103:0/4087829230 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed3c012900 con 0x563e6b4bc360 2024-09-19T06:52:17.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.072+0000 7fed28ff9640 1 -- 172.21.15.103:0/4087829230 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fed340035f0 con 0x563e6b4bc360 2024-09-19T06:52:17.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.072+0000 7fed49951740 10 client.16596 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:17.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.072+0000 7fed49951740 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:17.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.072+0000 7fed49951740 20 client.16596 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:17.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.072+0000 7fed49951740 1 --2- 172.21.15.103:0/4087829230 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563e6b4dae20 0x563e6b4dd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.072+0000 7fed49951740 4 client.16596 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:17.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.072+0000 7fed49951740 1 -- 172.21.15.103:0/4087829230 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e6b42e7f0 con 0x563e6b4dae20 2024-09-19T06:52:17.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.072+0000 7fed44b13640 1 --2- 172.21.15.103:0/4087829230 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563e6b4dae20 0x563e6b4dd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.073+0000 7fed44b13640 1 --2- 172.21.15.103:0/4087829230 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563e6b4dae20 0x563e6b4dd230 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdb6669cad3ed02c server_cookie=e4f782bdd7768046 in_seq=0 out_seq=0 2024-09-19T06:52:17.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.073+0000 7fed28ff9640 10 client.16596 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:17.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.073+0000 7fed28ff9640 1 -- 172.21.15.103:0/4087829230 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e6b42e7f0 con 0x563e6b4dae20 2024-09-19T06:52:17.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.073+0000 7fed28ff9640 10 client.16596 handle_command_reply: tid=0 2024-09-19T06:52:17.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.073+0000 7fed49951740 1 client.16596 shutdown 2024-09-19T06:52:17.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.073+0000 7fed49951740 1 -- 172.21.15.103:0/4087829230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bc360 msgr2=0x563e6b4c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:17.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.073+0000 7fed49951740 1 --2- 172.21.15.103:0/4087829230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bc360 0x563e6b4c8570 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fed34009930 tx=0x7fed340318d0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.074+0000 7fed49951740 1 -- 172.21.15.103:0/4087829230 shutdown_connections 2024-09-19T06:52:17.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.074+0000 7fed49951740 1 --2- 172.21.15.103:0/4087829230 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563e6b4dae20 0x563e6b4dd230 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.074+0000 7fed49951740 1 --2- 172.21.15.103:0/4087829230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4cb150 0x563e6b4c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.074+0000 7fed49951740 1 --2- 172.21.15.103:0/4087829230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bc360 0x563e6b4c8570 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.074+0000 7fed49951740 1 --2- 172.21.15.103:0/4087829230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bba40 0x563e6b4c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.074+0000 7fed49951740 1 -- 172.21.15.103:0/4087829230 >> 172.21.15.103:0/4087829230 conn(0x563e6b4b80f0 msgr2=0x563e6b4bde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:17.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.074+0000 7fed28ff9640 0 client.16596 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:17.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.074+0000 7fed49951740 1 -- 172.21.15.103:0/4087829230 shutdown_connections 2024-09-19T06:52:17.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.074+0000 7fed49951740 1 -- 172.21.15.103:0/4087829230 wait complete. 2024-09-19T06:52:17.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.074+0000 7fed49951740 20 client.16596 trim_cache size 0 max 16384 2024-09-19T06:52:17.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.091+0000 7fed49951740 1 Processor -- start 2024-09-19T06:52:17.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.091+0000 7fed49951740 1 -- start start 2024-09-19T06:52:17.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.091+0000 7fed49951740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4bba40 0x563e6b4c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.091+0000 7fed49951740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 0x563e6b4e2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.091+0000 7fed49951740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4cb150 0x563e6b4e3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.091+0000 7fed49951740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed401ab760 con 0x563e6b4cb150 2024-09-19T06:52:17.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.091+0000 7fed49951740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed401ad2a0 con 0x563e6b4bc360 2024-09-19T06:52:17.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.091+0000 7fed49951740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed401abe30 con 0x563e6b4bba40 2024-09-19T06:52:17.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.091+0000 7fed44b13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4cb150 0x563e6b4e3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.091+0000 7fed44b13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4cb150 0x563e6b4e3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53730/0 (socket says 172.21.15.103:53730) 2024-09-19T06:52:17.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.091+0000 7fed44b13640 1 -- 172.21.15.103:0/1326451262 learned_addr learned my addr 172.21.15.103:0/1326451262 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:17.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.091+0000 7fed45314640 1 --2- 172.21.15.103:0/1326451262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 0x563e6b4e2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.091+0000 7fed44b13640 1 --2- 172.21.15.103:0/1326451262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4bba40 0x563e6b4c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.092+0000 7fed44b13640 1 -- 172.21.15.103:0/1326451262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4bba40 msgr2=0x563e6b4c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:17.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.092+0000 7fed44b13640 1 --2- 172.21.15.103:0/1326451262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4bba40 0x563e6b4c9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.092+0000 7fed44b13640 1 -- 172.21.15.103:0/1326451262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 msgr2=0x563e6b4e2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:17.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.092+0000 7fed44b13640 1 --2- 172.21.15.103:0/1326451262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 0x563e6b4e2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.092+0000 7fed44b13640 1 -- 172.21.15.103:0/1326451262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6b4b7f20 con 0x563e6b4cb150 2024-09-19T06:52:17.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.092+0000 7fed45314640 1 --2- 172.21.15.103:0/1326451262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 0x563e6b4e2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:17.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.092+0000 7fed44b13640 1 --2- 172.21.15.103:0/1326451262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4cb150 0x563e6b4e3040 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009490 tx=0x7fed3c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37355709c0f602d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:17.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.092+0000 7fed2affd640 1 -- 172.21.15.103:0/1326451262 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c016070 con 0x563e6b4cb150 2024-09-19T06:52:17.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.092+0000 7fed2affd640 1 -- 172.21.15.103:0/1326451262 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed3c029890 con 0x563e6b4cb150 2024-09-19T06:52:17.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.092+0000 7fed2affd640 1 -- 172.21.15.103:0/1326451262 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c02e040 con 0x563e6b4cb150 2024-09-19T06:52:17.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.093+0000 7fed49951740 1 -- 172.21.15.103:0/1326451262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4cb150 msgr2=0x563e6b4e3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:17.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.093+0000 7fed49951740 1 --2- 172.21.15.103:0/1326451262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4cb150 0x563e6b4e3040 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009490 tx=0x7fed3c014040 comp rx=0 tx=0).stop 2024-09-19T06:52:17.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.093+0000 7fed49951740 1 -- 172.21.15.103:0/1326451262 shutdown_connections 2024-09-19T06:52:17.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.093+0000 7fed49951740 1 --2- 172.21.15.103:0/1326451262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4cb150 0x563e6b4e3040 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.093+0000 7fed49951740 1 --2- 172.21.15.103:0/1326451262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 0x563e6b4e2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.093+0000 7fed49951740 1 --2- 172.21.15.103:0/1326451262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4bba40 0x563e6b4c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.093+0000 7fed49951740 1 -- 172.21.15.103:0/1326451262 >> 172.21.15.103:0/1326451262 conn(0x563e6b4b80f0 msgr2=0x563e6b4c9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:17.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.093+0000 7fed49951740 1 -- 172.21.15.103:0/1326451262 shutdown_connections 2024-09-19T06:52:17.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.093+0000 7fed49951740 1 -- 172.21.15.103:0/1326451262 wait complete. 2024-09-19T06:52:17.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.093+0000 7fed49951740 1 -- start start 2024-09-19T06:52:17.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.093+0000 7fed49951740 1 Processor -- start 2024-09-19T06:52:17.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.093+0000 7fed49951740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bba40 0x563e6b4cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.093+0000 7fed49951740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 0x563e6b4cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.093+0000 7fed49951740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4cb150 0x563e6b544ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.093+0000 7fed49951740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed401acbd0 con 0x563e6b4bba40 2024-09-19T06:52:17.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed49951740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed401ac500 con 0x563e6b4bc360 2024-09-19T06:52:17.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed49951740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed401a9720 con 0x563e6b4cb150 2024-09-19T06:52:17.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed49951740 10 client.0 _ll_register_callbacks cb 0x7fed34005dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:17.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed44b13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bba40 0x563e6b4cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed44b13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bba40 0x563e6b4cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53746/0 (socket says 172.21.15.103:53746) 2024-09-19T06:52:17.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed44b13640 1 -- 172.21.15.103:0/1337281749 learned_addr learned my addr 172.21.15.103:0/1337281749 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:17.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed45314640 1 --2- 172.21.15.103:0/1337281749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 0x563e6b4cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed44b13640 1 --2- 172.21.15.103:0/1337281749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4cb150 0x563e6b544ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed44b13640 1 -- 172.21.15.103:0/1337281749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4cb150 msgr2=0x563e6b544ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:17.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed44b13640 1 --2- 172.21.15.103:0/1337281749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4cb150 0x563e6b544ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed44b13640 1 -- 172.21.15.103:0/1337281749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 msgr2=0x563e6b4cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:17.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed44b13640 1 --2- 172.21.15.103:0/1337281749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 0x563e6b4cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed44b13640 1 -- 172.21.15.103:0/1337281749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6b4bc880 con 0x563e6b4bba40 2024-09-19T06:52:17.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed44b13640 1 --2- 172.21.15.103:0/1337281749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bba40 0x563e6b4cf520 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009eb0 tx=0x7fed3c005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9823fe7ae5a2e828 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:17.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:17.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed28ff9640 1 -- 172.21.15.103:0/1337281749 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c016070 con 0x563e6b4bba40 2024-09-19T06:52:17.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed28ff9640 1 -- 172.21.15.103:0/1337281749 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fed3c029c40 con 0x563e6b4bba40 2024-09-19T06:52:17.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed49951740 1 -- 172.21.15.103:0/1337281749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed3c029890 con 0x563e6b4bba40 2024-09-19T06:52:17.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed28ff9640 1 -- 172.21.15.103:0/1337281749 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed3c0293b0 con 0x563e6b4bba40 2024-09-19T06:52:17.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed49951740 10 client.16602 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:17.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.094+0000 7fed49951740 1 -- 172.21.15.103:0/1337281749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e6b5460e0 con 0x563e6b4bba40 2024-09-19T06:52:17.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.095+0000 7fed28ff9640 1 -- 172.21.15.103:0/1337281749 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed3c029890 con 0x563e6b4bba40 2024-09-19T06:52:17.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.095+0000 7fed28ff9640 1 -- 172.21.15.103:0/1337281749 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fed3c033470 con 0x563e6b4bba40 2024-09-19T06:52:17.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.095+0000 7fed49951740 10 client.16602 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:17.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.095+0000 7fed49951740 10 client.16602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:17.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.095+0000 7fed49951740 20 client.16602 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:17.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.095+0000 7fed49951740 1 --2- 172.21.15.103:0/1337281749 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563e6b546550 0x563e6b548940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.095+0000 7fed49951740 4 client.16602 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:52:17.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.095+0000 7fed49951740 1 -- 172.21.15.103:0/1337281749 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563e6b428d00 con 0x563e6b546550 2024-09-19T06:52:17.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.095+0000 7fed45314640 1 --2- 172.21.15.103:0/1337281749 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563e6b546550 0x563e6b548940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.096+0000 7fed28ff9640 10 client.16602 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:17.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.096+0000 7fed45314640 1 --2- 172.21.15.103:0/1337281749 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563e6b546550 0x563e6b548940 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b7e3e058c424a2 server_cookie=9ba7adec4c7ab712 in_seq=0 out_seq=0 2024-09-19T06:52:17.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.097+0000 7fed28ff9640 1 -- 172.21.15.103:0/1337281749 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563e6b428d00 con 0x563e6b546550 2024-09-19T06:52:17.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.097+0000 7fed28ff9640 10 client.16602 handle_command_reply: tid=0 2024-09-19T06:52:17.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.097+0000 7fed49951740 1 client.16602 shutdown 2024-09-19T06:52:17.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.097+0000 7fed49951740 1 -- 172.21.15.103:0/1337281749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bba40 msgr2=0x563e6b4cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:17.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.097+0000 7fed49951740 1 --2- 172.21.15.103:0/1337281749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bba40 0x563e6b4cf520 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009eb0 tx=0x7fed3c005fb0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.097+0000 7fed49951740 1 -- 172.21.15.103:0/1337281749 shutdown_connections 2024-09-19T06:52:17.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.097+0000 7fed49951740 1 --2- 172.21.15.103:0/1337281749 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563e6b546550 0x563e6b548940 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.097+0000 7fed49951740 1 --2- 172.21.15.103:0/1337281749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e6b4cb150 0x563e6b544ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.097+0000 7fed49951740 1 --2- 172.21.15.103:0/1337281749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e6b4bc360 0x563e6b4cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.097+0000 7fed49951740 1 --2- 172.21.15.103:0/1337281749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e6b4bba40 0x563e6b4cf520 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.097+0000 7fed49951740 1 -- 172.21.15.103:0/1337281749 >> 172.21.15.103:0/1337281749 conn(0x563e6b4b80f0 msgr2=0x563e6b4c87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:17.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.097+0000 7fed28ff9640 0 client.16602 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:17.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.097+0000 7fed49951740 1 -- 172.21.15.103:0/1337281749 shutdown_connections 2024-09-19T06:52:17.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.097+0000 7fed49951740 1 -- 172.21.15.103:0/1337281749 wait complete. 2024-09-19T06:52:17.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.097+0000 7fed49951740 20 client.16602 trim_cache size 0 max 16384 2024-09-19T06:52:17.114 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:52:17.114 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T06:52:17.114 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "7bab123a-a59f-46c8-ba61-f13b801fd73e", 2024-09-19T06:52:17.115 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T06:52:17.115 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:52:17.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.098+0000 7fed47578640 1 -- 172.21.15.103:0/398177296 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed2c0a33b0 msgr2=0x7fed2c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:17.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.098+0000 7fed47578640 1 --2- 172.21.15.103:0/398177296 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed2c0a33b0 0x7fed2c0a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed3800efd0 tx=0x7fed380bc040 comp rx=0 tx=0).stop 2024-09-19T06:52:17.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.098+0000 7fed47578640 1 -- 172.21.15.103:0/398177296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40106a80 msgr2=0x7fed401a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:17.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.098+0000 7fed47578640 1 --2- 172.21.15.103:0/398177296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40106a80 0x7fed401a4370 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fed3800eaa0 tx=0x7fed3800ef60 comp rx=0 tx=0).stop 2024-09-19T06:52:17.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.098+0000 7fed47578640 1 -- 172.21.15.103:0/398177296 shutdown_connections 2024-09-19T06:52:17.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.098+0000 7fed47578640 1 --2- 172.21.15.103:0/398177296 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed2c0a33b0 0x7fed2c0a5870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.098+0000 7fed47578640 1 --2- 172.21.15.103:0/398177296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40106a80 0x7fed401a4370 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.098+0000 7fed47578640 1 --2- 172.21.15.103:0/398177296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed40101420 0x7fed40199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.098+0000 7fed47578640 1 --2- 172.21.15.103:0/398177296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed40100a80 0x7fed401992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.098+0000 7fed47578640 1 -- 172.21.15.103:0/398177296 >> 172.21.15.103:0/398177296 conn(0x7fed400fc7a0 msgr2=0x7fed400fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:17.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.098+0000 7fed47578640 1 -- 172.21.15.103:0/398177296 shutdown_connections 2024-09-19T06:52:17.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:17.098+0000 7fed47578640 1 -- 172.21.15.103:0/398177296 wait complete. 2024-09-19T06:52:17.782 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:52:18.073 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:18.135 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:17 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T06:52:18.135 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:17 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:18.135 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:17 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T06:52:18.136 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:17 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:17 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T06:52:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:17 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:17 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T06:52:18.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:17 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:18.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:17 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T06:52:18.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:17 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:18.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:17 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T06:52:18.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:17 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:18.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.469+0000 7f6c24470640 1 -- 172.21.15.103:0/1836931187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c1c106a80 msgr2=0x7f6c1c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.469+0000 7f6c24470640 1 --2- 172.21.15.103:0/1836931187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c1c106a80 0x7f6c1c106e60 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009930 tx=0x7f6c1402a420 comp rx=0 tx=0).stop 2024-09-19T06:52:18.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.469+0000 7f6c24470640 1 -- 172.21.15.103:0/1836931187 shutdown_connections 2024-09-19T06:52:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.469+0000 7f6c24470640 1 --2- 172.21.15.103:0/1836931187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c1c101420 0x7f6c1c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.469+0000 7f6c24470640 1 --2- 172.21.15.103:0/1836931187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c1c100a80 0x7f6c1c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.469+0000 7f6c24470640 1 --2- 172.21.15.103:0/1836931187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c1c106a80 0x7f6c1c106e60 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.469+0000 7f6c24470640 1 -- 172.21.15.103:0/1836931187 >> 172.21.15.103:0/1836931187 conn(0x7f6c1c0fc7a0 msgr2=0x7f6c1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.469+0000 7f6c24470640 1 -- 172.21.15.103:0/1836931187 shutdown_connections 2024-09-19T06:52:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.469+0000 7f6c24470640 1 -- 172.21.15.103:0/1836931187 wait complete. 2024-09-19T06:52:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.470+0000 7f6c24470640 1 Processor -- start 2024-09-19T06:52:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.470+0000 7f6c24470640 1 -- start start 2024-09-19T06:52:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.470+0000 7f6c24470640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c1c100a80 0x7f6c1c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.470+0000 7f6c24470640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c1c101420 0x7f6c1c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.470+0000 7f6c24470640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c1c106a80 0x7f6c1c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.470+0000 7f6c24470640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c110f20 con 0x7f6c1c101420 2024-09-19T06:52:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.470+0000 7f6c24470640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c110da0 con 0x7f6c1c106a80 2024-09-19T06:52:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.470+0000 7f6c24470640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1110a0 con 0x7f6c1c100a80 2024-09-19T06:52:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.470+0000 7f6c21a0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c1c101420 0x7f6c1c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.470+0000 7f6c21a0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c1c101420 0x7f6c1c198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43200/0 (socket says 172.21.15.103:43200) 2024-09-19T06:52:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.470+0000 7f6c21a0b640 1 -- 172.21.15.103:0/1771170050 learned_addr learned my addr 172.21.15.103:0/1771170050 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.470+0000 7f6c2220c640 1 --2- 172.21.15.103:0/1771170050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c1c100a80 0x7f6c1c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.471+0000 7f6c22a0d640 1 --2- 172.21.15.103:0/1771170050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c1c106a80 0x7f6c1c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.471+0000 7f6c21a0b640 1 -- 172.21.15.103:0/1771170050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c1c100a80 msgr2=0x7f6c1c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.471+0000 7f6c21a0b640 1 --2- 172.21.15.103:0/1771170050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c1c100a80 0x7f6c1c1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.471+0000 7f6c21a0b640 1 -- 172.21.15.103:0/1771170050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c1c106a80 msgr2=0x7f6c1c19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.471+0000 7f6c21a0b640 1 --2- 172.21.15.103:0/1771170050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c1c106a80 0x7f6c1c19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.471+0000 7f6c21a0b640 1 -- 172.21.15.103:0/1771170050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1c19d5a0 con 0x7f6c1c101420 2024-09-19T06:52:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.471+0000 7f6c22a0d640 1 --2- 172.21.15.103:0/1771170050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c1c106a80 0x7f6c1c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.471+0000 7f6c2220c640 1 --2- 172.21.15.103:0/1771170050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c1c100a80 0x7f6c1c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.471+0000 7f6c21a0b640 1 --2- 172.21.15.103:0/1771170050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c1c101420 0x7f6c1c198b30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000b4a0 tx=0x7f6c1000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.471+0000 7f6c1b7fe640 1 -- 172.21.15.103:0/1771170050 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10014070 con 0x7f6c1c101420 2024-09-19T06:52:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.471+0000 7f6c24470640 1 -- 172.21.15.103:0/1771170050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1c19d890 con 0x7f6c1c101420 2024-09-19T06:52:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.471+0000 7f6c1b7fe640 1 -- 172.21.15.103:0/1771170050 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c10010af0 con 0x7f6c1c101420 2024-09-19T06:52:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.471+0000 7f6c1b7fe640 1 -- 172.21.15.103:0/1771170050 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10017d10 con 0x7f6c1c101420 2024-09-19T06:52:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.471+0000 7f6c24470640 1 -- 172.21.15.103:0/1771170050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1c1a5170 con 0x7f6c1c101420 2024-09-19T06:52:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.473+0000 7f6c1b7fe640 1 -- 172.21.15.103:0/1771170050 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f6c10017070 con 0x7f6c1c101420 2024-09-19T06:52:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.473+0000 7f6c26849740 1 Processor -- start 2024-09-19T06:52:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c26849740 1 -- start start 2024-09-19T06:52:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c26849740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae84a0 0x561a6fae8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c26849740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6fae8dc0 0x561a6faf3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c26849740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6faf3680 0x561a6faf5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c26849740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561a6fa4e2e0 con 0x561a6faf3680 2024-09-19T06:52:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c26849740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561a6f9f5ce0 con 0x561a6fae84a0 2024-09-19T06:52:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c26849740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561a6fa4a2a0 con 0x561a6fae8dc0 2024-09-19T06:52:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c1b7fe640 1 --2- 172.21.15.103:0/1771170050 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6c040a33b0 0x7f6c040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c21a0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6faf3680 0x561a6faf5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c22a0d640 1 --2- 172.21.15.103:0/1771170050 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6c040a33b0 0x7f6c040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c21a0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6faf3680 0x561a6faf5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43204/0 (socket says 172.21.15.103:43204) 2024-09-19T06:52:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c21a0b640 1 -- 172.21.15.103:0/1987043095 learned_addr learned my addr 172.21.15.103:0/1987043095 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c22a0d640 1 --2- 172.21.15.103:0/1987043095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae84a0 0x561a6fae8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c2220c640 1 --2- 172.21.15.103:0/1987043095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6fae8dc0 0x561a6faf3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c1b7fe640 1 -- 172.21.15.103:0/1771170050 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6c100cf0f0 con 0x7f6c1c101420 2024-09-19T06:52:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c21a0b640 1 -- 172.21.15.103:0/1987043095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6fae8dc0 msgr2=0x561a6faf3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c21a0b640 1 --2- 172.21.15.103:0/1987043095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6fae8dc0 0x561a6faf3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c21a0b640 1 -- 172.21.15.103:0/1987043095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae84a0 msgr2=0x561a6fae8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c21a0b640 1 --2- 172.21.15.103:0/1987043095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae84a0 0x561a6fae8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.474+0000 7f6c21a0b640 1 -- 172.21.15.103:0/1987043095 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a6f9da440 con 0x561a6faf3680 2024-09-19T06:52:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.475+0000 7f6c22a0d640 1 --2- 172.21.15.103:0/1771170050 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6c040a33b0 0x7f6c040a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0116c0 tx=0x7f6c0c00a990 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.475+0000 7f6c21a0b640 1 --2- 172.21.15.103:0/1987043095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6faf3680 0x561a6faf5a70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000b470 tx=0x7f6c10017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ac0c37d9fea39e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.475+0000 7f6c18dea640 1 -- 172.21.15.103:0/1987043095 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1001e040 con 0x561a6faf3680 2024-09-19T06:52:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.475+0000 7f6c18dea640 1 -- 172.21.15.103:0/1987043095 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c1009bbf0 con 0x561a6faf3680 2024-09-19T06:52:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.475+0000 7f6c18dea640 1 -- 172.21.15.103:0/1987043095 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10010ca0 con 0x561a6faf3680 2024-09-19T06:52:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 -- 172.21.15.103:0/1987043095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6faf3680 msgr2=0x561a6faf5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 --2- 172.21.15.103:0/1987043095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6faf3680 0x561a6faf5a70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000b470 tx=0x7f6c10017730 comp rx=0 tx=0).stop 2024-09-19T06:52:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 -- 172.21.15.103:0/1987043095 shutdown_connections 2024-09-19T06:52:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 --2- 172.21.15.103:0/1987043095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6faf3680 0x561a6faf5a70 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 --2- 172.21.15.103:0/1987043095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6fae8dc0 0x561a6faf3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 --2- 172.21.15.103:0/1987043095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae84a0 0x561a6fae8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 -- 172.21.15.103:0/1987043095 >> 172.21.15.103:0/1987043095 conn(0x561a6fae4a30 msgr2=0x561a6fae4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 -- 172.21.15.103:0/1987043095 shutdown_connections 2024-09-19T06:52:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 -- 172.21.15.103:0/1987043095 wait complete. 2024-09-19T06:52:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 -- start start 2024-09-19T06:52:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 Processor -- start 2024-09-19T06:52:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6fae84a0 0x561a6faf3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae8dc0 0x561a6faf02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6faf3680 0x561a6faf0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561a6faf6250 con 0x561a6fae84a0 2024-09-19T06:52:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561a6fa09050 con 0x561a6fae8dc0 2024-09-19T06:52:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561a6faf63d0 con 0x561a6faf3680 2024-09-19T06:52:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.476+0000 7f6c26849740 10 client.0 _ll_register_callbacks cb 0x561a6fa49160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c2220c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6fae84a0 0x561a6faf3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c2220c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6fae84a0 0x561a6faf3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43214/0 (socket says 172.21.15.103:43214) 2024-09-19T06:52:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c2220c640 1 -- 172.21.15.103:0/1457735921 learned_addr learned my addr 172.21.15.103:0/1457735921 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c21a0b640 1 --2- 172.21.15.103:0/1457735921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae8dc0 0x561a6faf02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c22a0d640 1 --2- 172.21.15.103:0/1457735921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6faf3680 0x561a6faf0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c2220c640 1 -- 172.21.15.103:0/1457735921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6faf3680 msgr2=0x561a6faf0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c2220c640 1 --2- 172.21.15.103:0/1457735921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6faf3680 0x561a6faf0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c2220c640 1 -- 172.21.15.103:0/1457735921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae8dc0 msgr2=0x561a6faf02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c2220c640 1 --2- 172.21.15.103:0/1457735921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae8dc0 0x561a6faf02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c2220c640 1 -- 172.21.15.103:0/1457735921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a6f7bf3c0 con 0x561a6fae84a0 2024-09-19T06:52:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c2220c640 1 --2- 172.21.15.103:0/1457735921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6fae84a0 0x561a6faf3000 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6c14031550 tx=0x7f6c14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f46e8dc73a9dfa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c01ffb640 1 -- 172.21.15.103:0/1457735921 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14035040 con 0x561a6fae84a0 2024-09-19T06:52:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c01ffb640 1 -- 172.21.15.103:0/1457735921 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6c14005020 con 0x561a6fae84a0 2024-09-19T06:52:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c26849740 1 -- 172.21.15.103:0/1457735921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c1009bbf0 con 0x561a6fae84a0 2024-09-19T06:52:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c01ffb640 1 -- 172.21.15.103:0/1457735921 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c14005220 con 0x561a6fae84a0 2024-09-19T06:52:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c26849740 10 client.16614 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.477+0000 7f6c26849740 1 -- 172.21.15.103:0/1457735921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a6fb01c40 con 0x561a6fae84a0 2024-09-19T06:52:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.478+0000 7f6c01ffb640 1 -- 172.21.15.103:0/1457735921 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c1009bbf0 con 0x561a6fae84a0 2024-09-19T06:52:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.478+0000 7f6c01ffb640 1 -- 172.21.15.103:0/1457735921 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6c14003710 con 0x561a6fae84a0 2024-09-19T06:52:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.478+0000 7f6c26849740 10 client.16614 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.478+0000 7f6c26849740 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.478+0000 7f6c26849740 20 client.16614 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.478+0000 7f6c26849740 1 --2- 172.21.15.103:0/1457735921 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561a6fb03100 0x561a6fb05510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.478+0000 7f6c26849740 4 client.16614 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.478+0000 7f6c26849740 1 -- 172.21.15.103:0/1457735921 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a6fa5b850 con 0x561a6fb03100 2024-09-19T06:52:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.478+0000 7f6c21a0b640 1 --2- 172.21.15.103:0/1457735921 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561a6fb03100 0x561a6fb05510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.479+0000 7f6c21a0b640 1 --2- 172.21.15.103:0/1457735921 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561a6fb03100 0x561a6fb05510 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=500f9ff823617e68 server_cookie=262ccecf76c8a0f0 in_seq=0 out_seq=0 2024-09-19T06:52:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.479+0000 7f6c01ffb640 10 client.16614 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.479+0000 7f6c01ffb640 1 -- 172.21.15.103:0/1457735921 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a6fa5b850 con 0x561a6fb03100 2024-09-19T06:52:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.479+0000 7f6c01ffb640 10 client.16614 handle_command_reply: tid=0 2024-09-19T06:52:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.479+0000 7f6c26849740 1 client.16614 shutdown 2024-09-19T06:52:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.479+0000 7f6c26849740 1 -- 172.21.15.103:0/1457735921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6fae84a0 msgr2=0x561a6faf3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.479+0000 7f6c26849740 1 --2- 172.21.15.103:0/1457735921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6fae84a0 0x561a6faf3000 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6c14031550 tx=0x7f6c14004270 comp rx=0 tx=0).stop 2024-09-19T06:52:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.480+0000 7f6c26849740 1 -- 172.21.15.103:0/1457735921 shutdown_connections 2024-09-19T06:52:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.480+0000 7f6c26849740 1 --2- 172.21.15.103:0/1457735921 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561a6fb03100 0x561a6fb05510 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.480+0000 7f6c26849740 1 --2- 172.21.15.103:0/1457735921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6faf3680 0x561a6faf0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.480+0000 7f6c26849740 1 --2- 172.21.15.103:0/1457735921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae8dc0 0x561a6faf02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.480+0000 7f6c26849740 1 --2- 172.21.15.103:0/1457735921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6fae84a0 0x561a6faf3000 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.480+0000 7f6c26849740 1 -- 172.21.15.103:0/1457735921 >> 172.21.15.103:0/1457735921 conn(0x561a6fae4a30 msgr2=0x561a6faea760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.480+0000 7f6c01ffb640 0 client.16614 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.480+0000 7f6c26849740 1 -- 172.21.15.103:0/1457735921 shutdown_connections 2024-09-19T06:52:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.480+0000 7f6c26849740 1 -- 172.21.15.103:0/1457735921 wait complete. 2024-09-19T06:52:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.480+0000 7f6c26849740 20 client.16614 trim_cache size 0 max 16384 2024-09-19T06:52:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.496+0000 7f6c26849740 1 Processor -- start 2024-09-19T06:52:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.496+0000 7f6c26849740 1 -- start start 2024-09-19T06:52:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c26849740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae84a0 0x561a6faff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c26849740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6fae8dc0 0x561a6faeb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c26849740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6faf3680 0x561a6fb0ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c26849740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1ad9f0 con 0x561a6faf3680 2024-09-19T06:52:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c26849740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1af530 con 0x561a6fae84a0 2024-09-19T06:52:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c26849740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1ae0c0 con 0x561a6fae8dc0 2024-09-19T06:52:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c22a0d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6faf3680 0x561a6fb0ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c22a0d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6faf3680 0x561a6fb0ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43228/0 (socket says 172.21.15.103:43228) 2024-09-19T06:52:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c22a0d640 1 -- 172.21.15.103:0/4285652670 learned_addr learned my addr 172.21.15.103:0/4285652670 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c2220c640 1 --2- 172.21.15.103:0/4285652670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae84a0 0x561a6faff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c21a0b640 1 --2- 172.21.15.103:0/4285652670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6fae8dc0 0x561a6faeb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c22a0d640 1 -- 172.21.15.103:0/4285652670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6fae8dc0 msgr2=0x561a6faeb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c22a0d640 1 --2- 172.21.15.103:0/4285652670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6fae8dc0 0x561a6faeb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c22a0d640 1 -- 172.21.15.103:0/4285652670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae84a0 msgr2=0x561a6faff4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c22a0d640 1 --2- 172.21.15.103:0/4285652670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae84a0 0x561a6faff4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c22a0d640 1 -- 172.21.15.103:0/4285652670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a6faef260 con 0x561a6faf3680 2024-09-19T06:52:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c22a0d640 1 --2- 172.21.15.103:0/4285652670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6faf3680 0x561a6fb0ade0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0157c0 tx=0x7f6c0c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e965b3ba844233 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c03fff640 1 -- 172.21.15.103:0/4285652670 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c025070 con 0x561a6faf3680 2024-09-19T06:52:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c03fff640 1 -- 172.21.15.103:0/4285652670 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c012dc0 con 0x561a6faf3680 2024-09-19T06:52:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.497+0000 7f6c03fff640 1 -- 172.21.15.103:0/4285652670 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c021d00 con 0x561a6faf3680 2024-09-19T06:52:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.498+0000 7f6c26849740 1 -- 172.21.15.103:0/4285652670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6faf3680 msgr2=0x561a6fb0ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.498+0000 7f6c26849740 1 --2- 172.21.15.103:0/4285652670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6faf3680 0x561a6fb0ade0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0157c0 tx=0x7f6c0c017000 comp rx=0 tx=0).stop 2024-09-19T06:52:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.498+0000 7f6c26849740 1 -- 172.21.15.103:0/4285652670 shutdown_connections 2024-09-19T06:52:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.498+0000 7f6c26849740 1 --2- 172.21.15.103:0/4285652670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6faf3680 0x561a6fb0ade0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.498+0000 7f6c26849740 1 --2- 172.21.15.103:0/4285652670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6fae8dc0 0x561a6faeb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.498+0000 7f6c26849740 1 --2- 172.21.15.103:0/4285652670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae84a0 0x561a6faff4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.498+0000 7f6c26849740 1 -- 172.21.15.103:0/4285652670 >> 172.21.15.103:0/4285652670 conn(0x561a6fae4a30 msgr2=0x561a6faeb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.499+0000 7f6c26849740 1 -- 172.21.15.103:0/4285652670 shutdown_connections 2024-09-19T06:52:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.499+0000 7f6c26849740 1 -- 172.21.15.103:0/4285652670 wait complete. 2024-09-19T06:52:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.499+0000 7f6c26849740 1 -- start start 2024-09-19T06:52:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.499+0000 7f6c26849740 1 Processor -- start 2024-09-19T06:52:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c26849740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6fae84a0 0x561a6fafb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c26849740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae8dc0 0x561a6fafbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c26849740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6faf3680 0x561a6faf6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c26849740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1aee60 con 0x561a6fae84a0 2024-09-19T06:52:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c26849740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1ae790 con 0x561a6fae8dc0 2024-09-19T06:52:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c26849740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1ab9b0 con 0x561a6faf3680 2024-09-19T06:52:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c26849740 10 client.0 _ll_register_callbacks cb 0x7f6c14031a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c2220c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6fae84a0 0x561a6fafb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c2220c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6fae84a0 0x561a6fafb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43232/0 (socket says 172.21.15.103:43232) 2024-09-19T06:52:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c2220c640 1 -- 172.21.15.103:0/1044776212 learned_addr learned my addr 172.21.15.103:0/1044776212 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c22a0d640 1 --2- 172.21.15.103:0/1044776212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6faf3680 0x561a6faf6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c21a0b640 1 --2- 172.21.15.103:0/1044776212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae8dc0 0x561a6fafbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c2220c640 1 -- 172.21.15.103:0/1044776212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6faf3680 msgr2=0x561a6faf6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c2220c640 1 --2- 172.21.15.103:0/1044776212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6faf3680 0x561a6faf6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c2220c640 1 -- 172.21.15.103:0/1044776212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae8dc0 msgr2=0x561a6fafbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c2220c640 1 --2- 172.21.15.103:0/1044776212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae8dc0 0x561a6fafbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c2220c640 1 -- 172.21.15.103:0/1044776212 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a6faf7380 con 0x561a6fae84a0 2024-09-19T06:52:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c21a0b640 1 --2- 172.21.15.103:0/1044776212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae8dc0 0x561a6fafbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c2220c640 1 --2- 172.21.15.103:0/1044776212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6fae84a0 0x561a6fafb6e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6c14000c00 tx=0x7f6c14031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a1aa6f3cf3be80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:18.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.500+0000 7f6c01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:18.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.501+0000 7f6c01ffb640 1 -- 172.21.15.103:0/1044776212 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1405b060 con 0x561a6fae84a0 2024-09-19T06:52:18.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.501+0000 7f6c01ffb640 1 -- 172.21.15.103:0/1044776212 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6c140053c0 con 0x561a6fae84a0 2024-09-19T06:52:18.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.501+0000 7f6c26849740 1 -- 172.21.15.103:0/1044776212 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c0c012dc0 con 0x561a6fae84a0 2024-09-19T06:52:18.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.501+0000 7f6c01ffb640 1 -- 172.21.15.103:0/1044776212 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c14035070 con 0x561a6fae84a0 2024-09-19T06:52:18.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.501+0000 7f6c26849740 10 client.16620 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:18.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.501+0000 7f6c26849740 1 -- 172.21.15.103:0/1044776212 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a6fb6d8c0 con 0x561a6fae84a0 2024-09-19T06:52:18.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.501+0000 7f6c01ffb640 1 -- 172.21.15.103:0/1044776212 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c012dc0 con 0x561a6fae84a0 2024-09-19T06:52:18.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.501+0000 7f6c01ffb640 1 -- 172.21.15.103:0/1044776212 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6c14045af0 con 0x561a6fae84a0 2024-09-19T06:52:18.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.501+0000 7f6c26849740 10 client.16620 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:18.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.501+0000 7f6c26849740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:18.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.501+0000 7f6c26849740 20 client.16620 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:18.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.501+0000 7f6c26849740 1 --2- 172.21.15.103:0/1044776212 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561a6fb6dfc0 0x561a6fb703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.501+0000 7f6c26849740 4 client.16620 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:52:18.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.501+0000 7f6c26849740 1 -- 172.21.15.103:0/1044776212 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561a6fa55d60 con 0x561a6fb6dfc0 2024-09-19T06:52:18.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.502+0000 7f6c21a0b640 1 --2- 172.21.15.103:0/1044776212 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561a6fb6dfc0 0x561a6fb703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.502+0000 7f6c21a0b640 1 --2- 172.21.15.103:0/1044776212 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561a6fb6dfc0 0x561a6fb703e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74ef2d55b157dd0d server_cookie=1ad34774dea3e3fd in_seq=0 out_seq=0 2024-09-19T06:52:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.502+0000 7f6c01ffb640 10 client.16620 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.502+0000 7f6c01ffb640 1 -- 172.21.15.103:0/1044776212 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561a6fa55d60 con 0x561a6fb6dfc0 2024-09-19T06:52:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.502+0000 7f6c01ffb640 10 client.16620 handle_command_reply: tid=0 2024-09-19T06:52:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.502+0000 7f6c26849740 1 client.16620 shutdown 2024-09-19T06:52:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.502+0000 7f6c26849740 1 -- 172.21.15.103:0/1044776212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6fae84a0 msgr2=0x561a6fafb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.502+0000 7f6c26849740 1 --2- 172.21.15.103:0/1044776212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6fae84a0 0x561a6fafb6e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6c14000c00 tx=0x7f6c14031b60 comp rx=0 tx=0).stop 2024-09-19T06:52:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.502+0000 7f6c26849740 1 -- 172.21.15.103:0/1044776212 shutdown_connections 2024-09-19T06:52:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.502+0000 7f6c26849740 1 --2- 172.21.15.103:0/1044776212 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561a6fb6dfc0 0x561a6fb703e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.502+0000 7f6c26849740 1 --2- 172.21.15.103:0/1044776212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a6faf3680 0x561a6faf6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.502+0000 7f6c26849740 1 --2- 172.21.15.103:0/1044776212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6fae8dc0 0x561a6fafbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.502+0000 7f6c01ffb640 0 client.16620 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:18.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.502+0000 7f6c26849740 1 --2- 172.21.15.103:0/1044776212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a6fae84a0 0x561a6fafb6e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.502+0000 7f6c26849740 1 -- 172.21.15.103:0/1044776212 >> 172.21.15.103:0/1044776212 conn(0x561a6fae4a30 msgr2=0x561a6fae3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:18.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.503+0000 7f6c26849740 1 -- 172.21.15.103:0/1044776212 shutdown_connections 2024-09-19T06:52:18.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.503+0000 7f6c26849740 1 -- 172.21.15.103:0/1044776212 wait complete. 2024-09-19T06:52:18.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.503+0000 7f6c26849740 20 client.16620 trim_cache size 0 max 16384 2024-09-19T06:52:18.520 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:52:18.520 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:52:18.521 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T06:52:18.521 INFO:teuthology.orchestra.run.smithi103.stdout: "7bab123a-a59f-46c8-ba61-f13b801fd73e": { 2024-09-19T06:52:18.521 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T06:52:18.521 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "7bab123a-a59f-46c8-ba61-f13b801fd73e", 2024-09-19T06:52:18.521 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T06:52:18.521 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:52:18.521 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:52:18.521 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:52:18.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.503+0000 7f6c24470640 1 -- 172.21.15.103:0/1771170050 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6c040a33b0 msgr2=0x7f6c040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.503+0000 7f6c24470640 1 --2- 172.21.15.103:0/1771170050 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6c040a33b0 0x7f6c040a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0116c0 tx=0x7f6c0c00a990 comp rx=0 tx=0).stop 2024-09-19T06:52:18.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.503+0000 7f6c24470640 1 -- 172.21.15.103:0/1771170050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c1c101420 msgr2=0x7f6c1c198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.503+0000 7f6c24470640 1 --2- 172.21.15.103:0/1771170050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c1c101420 0x7f6c1c198b30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000b4a0 tx=0x7f6c1000b960 comp rx=0 tx=0).stop 2024-09-19T06:52:18.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.503+0000 7f6c24470640 1 -- 172.21.15.103:0/1771170050 shutdown_connections 2024-09-19T06:52:18.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.503+0000 7f6c24470640 1 --2- 172.21.15.103:0/1771170050 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6c040a33b0 0x7f6c040a5870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.503+0000 7f6c24470640 1 --2- 172.21.15.103:0/1771170050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c1c106a80 0x7f6c1c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.503+0000 7f6c24470640 1 --2- 172.21.15.103:0/1771170050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c1c101420 0x7f6c1c198b30 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.503+0000 7f6c24470640 1 --2- 172.21.15.103:0/1771170050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c1c100a80 0x7f6c1c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.503+0000 7f6c24470640 1 -- 172.21.15.103:0/1771170050 >> 172.21.15.103:0/1771170050 conn(0x7f6c1c0fc7a0 msgr2=0x7f6c1c06ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:18.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.504+0000 7f6c24470640 1 -- 172.21.15.103:0/1771170050 shutdown_connections 2024-09-19T06:52:18.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:18.504+0000 7f6c24470640 1 -- 172.21.15.103:0/1771170050 wait complete. 2024-09-19T06:52:19.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:18 smithi103 ceph-mon[31158]: pgmap v95: 97 pgs: 97 active+clean; 2.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-19T06:52:19.173 INFO:tasks.cephfs.filesystem:scrub status for tag:7bab123a-a59f-46c8-ba61-f13b801fd73e - {'path': '/', 'tag': '7bab123a-a59f-46c8-ba61-f13b801fd73e', 'options': 'recursive,force'} 2024-09-19T06:52:19.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:18 smithi134 ceph-mon[35356]: pgmap v95: 97 pgs: 97 active+clean; 2.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-19T06:52:19.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:18 smithi174 ceph-mon[35637]: pgmap v95: 97 pgs: 97 active+clean; 2.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-19T06:52:20.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:19 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T06:52:20.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:19 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T06:52:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:19 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T06:52:21.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:20 smithi103 ceph-mon[31158]: pgmap v96: 97 pgs: 97 active+clean; 2.7 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 23 op/s 2024-09-19T06:52:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:20 smithi134 ceph-mon[35356]: pgmap v96: 97 pgs: 97 active+clean; 2.7 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 23 op/s 2024-09-19T06:52:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:20 smithi174 ceph-mon[35637]: pgmap v96: 97 pgs: 97 active+clean; 2.7 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 23 op/s 2024-09-19T06:52:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:22 smithi174 ceph-mon[35637]: pgmap v97: 97 pgs: 97 active+clean; 2.7 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-09-19T06:52:22.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:22 smithi103 ceph-mon[31158]: pgmap v97: 97 pgs: 97 active+clean; 2.7 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-09-19T06:52:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:22 smithi134 ceph-mon[35356]: pgmap v97: 97 pgs: 97 active+clean; 2.7 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-09-19T06:52:22.972 INFO:tasks.workunit.client.0.smithi103.stdout:1/600: open fd = 3 2024-09-19T06:52:22.972 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.972 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.972 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.972 INFO:tasks.workunit.client.0.smithi103.stdout:2/600: open fd = 3 2024-09-19T06:52:22.973 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.973 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.973 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.973 INFO:tasks.workunit.client.0.smithi103.stdout:3/600: open fd = 3 2024-09-19T06:52:22.973 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.973 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.973 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.973 INFO:tasks.workunit.client.0.smithi103.stdout:4/600: open fd = 3 2024-09-19T06:52:22.973 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.973 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.973 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.973 INFO:tasks.workunit.client.0.smithi103.stdout:5/600: open fd = 3 2024-09-19T06:52:22.973 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.973 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.973 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.974 INFO:tasks.workunit.client.0.smithi103.stdout:6/600: open fd = 3 2024-09-19T06:52:22.974 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.974 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.974 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.974 INFO:tasks.workunit.client.0.smithi103.stdout:7/600: open fd = 3 2024-09-19T06:52:22.974 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.974 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.974 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.974 INFO:tasks.workunit.client.0.smithi103.stdout:8/600: open fd = 3 2024-09-19T06:52:22.974 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.974 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.974 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.974 INFO:tasks.workunit.client.0.smithi103.stdout:9/600: open fd = 3 2024-09-19T06:52:22.974 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.975 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.975 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.975 INFO:tasks.workunit.client.0.smithi103.stdout:10/600: open fd = 3 2024-09-19T06:52:22.975 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.975 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.975 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.975 INFO:tasks.workunit.client.0.smithi103.stdout:11/600: open fd = 3 2024-09-19T06:52:22.975 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.975 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.975 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.975 INFO:tasks.workunit.client.0.smithi103.stdout:12/600: open fd = 3 2024-09-19T06:52:22.975 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.975 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.975 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.976 INFO:tasks.workunit.client.0.smithi103.stdout:13/600: open fd = 3 2024-09-19T06:52:22.976 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.976 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.976 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.976 INFO:tasks.workunit.client.0.smithi103.stdout:14/600: open fd = 3 2024-09-19T06:52:22.976 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.976 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.976 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.976 INFO:tasks.workunit.client.0.smithi103.stdout:15/600: open fd = 3 2024-09-19T06:52:22.976 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.976 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.976 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.976 INFO:tasks.workunit.client.0.smithi103.stdout:16/600: open fd = 3 2024-09-19T06:52:22.976 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.976 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.977 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.977 INFO:tasks.workunit.client.0.smithi103.stdout:17/600: open fd = 3 2024-09-19T06:52:22.977 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.977 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.977 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.977 INFO:tasks.workunit.client.0.smithi103.stdout:18/600: open fd = 3 2024-09-19T06:52:22.977 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.977 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.977 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.977 INFO:tasks.workunit.client.0.smithi103.stdout:19/600: open fd = 3 2024-09-19T06:52:22.977 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.977 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.977 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.977 INFO:tasks.workunit.client.0.smithi103.stdout:20/600: open fd = 3 2024-09-19T06:52:22.978 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.978 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.978 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.978 INFO:tasks.workunit.client.0.smithi103.stdout:21/600: open fd = 3 2024-09-19T06:52:22.978 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.978 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.978 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.978 INFO:tasks.workunit.client.0.smithi103.stdout:22/600: open fd = 3 2024-09-19T06:52:22.978 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.978 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.978 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.978 INFO:tasks.workunit.client.0.smithi103.stdout:23/600: open fd = 3 2024-09-19T06:52:22.978 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.978 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.979 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.979 INFO:tasks.workunit.client.0.smithi103.stdout:24/600: open fd = 3 2024-09-19T06:52:22.979 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.979 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.979 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.979 INFO:tasks.workunit.client.0.smithi103.stdout:25/600: open fd = 3 2024-09-19T06:52:22.979 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.979 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.979 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.979 INFO:tasks.workunit.client.0.smithi103.stdout:26/600: open fd = 3 2024-09-19T06:52:22.979 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.979 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.979 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.979 INFO:tasks.workunit.client.0.smithi103.stdout:27/600: open fd = 3 2024-09-19T06:52:22.980 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.980 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.980 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.980 INFO:tasks.workunit.client.0.smithi103.stdout:28/600: open fd = 3 2024-09-19T06:52:22.980 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.980 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.980 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.980 INFO:tasks.workunit.client.0.smithi103.stdout:29/600: open fd = 3 2024-09-19T06:52:22.980 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.980 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.980 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.980 INFO:tasks.workunit.client.0.smithi103.stdout:30/600: open fd = 3 2024-09-19T06:52:22.980 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.980 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.981 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.981 INFO:tasks.workunit.client.0.smithi103.stdout:31/600: open fd = 3 2024-09-19T06:52:22.981 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.981 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.981 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.981 INFO:tasks.workunit.client.0.smithi103.stdout:32/600: open fd = 3 2024-09-19T06:52:22.981 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.981 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.981 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.981 INFO:tasks.workunit.client.0.smithi103.stdout:33/600: open fd = 3 2024-09-19T06:52:22.981 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.981 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.981 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.981 INFO:tasks.workunit.client.0.smithi103.stdout:34/600: open fd = 3 2024-09-19T06:52:22.981 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.982 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.982 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.982 INFO:tasks.workunit.client.0.smithi103.stdout:35/600: open fd = 3 2024-09-19T06:52:22.982 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.982 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.982 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.982 INFO:tasks.workunit.client.0.smithi103.stdout:36/600: open fd = 3 2024-09-19T06:52:22.982 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.982 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.982 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.982 INFO:tasks.workunit.client.0.smithi103.stdout:37/600: open fd = 3 2024-09-19T06:52:22.982 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.982 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.982 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.983 INFO:tasks.workunit.client.0.smithi103.stdout:38/600: open fd = 3 2024-09-19T06:52:22.983 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.983 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.983 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.983 INFO:tasks.workunit.client.0.smithi103.stdout:39/600: open fd = 3 2024-09-19T06:52:22.983 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.983 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.983 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.983 INFO:tasks.workunit.client.0.smithi103.stdout:40/600: open fd = 3 2024-09-19T06:52:22.983 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.984 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.984 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.984 INFO:tasks.workunit.client.0.smithi103.stdout:41/600: open fd = 3 2024-09-19T06:52:22.984 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.984 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.984 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.984 INFO:tasks.workunit.client.0.smithi103.stdout:42/600: open fd = 3 2024-09-19T06:52:22.984 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.984 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.984 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.984 INFO:tasks.workunit.client.0.smithi103.stdout:43/600: open fd = 3 2024-09-19T06:52:22.984 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.984 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.984 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.985 INFO:tasks.workunit.client.0.smithi103.stdout:44/600: open fd = 3 2024-09-19T06:52:22.985 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.985 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.985 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.985 INFO:tasks.workunit.client.0.smithi103.stdout:45/600: open fd = 3 2024-09-19T06:52:22.985 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.985 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.985 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.985 INFO:tasks.workunit.client.0.smithi103.stdout:46/600: open fd = 3 2024-09-19T06:52:22.985 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.985 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.985 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.985 INFO:tasks.workunit.client.0.smithi103.stdout:47/600: open fd = 3 2024-09-19T06:52:22.985 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.985 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.986 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.986 INFO:tasks.workunit.client.0.smithi103.stdout:48/600: open fd = 3 2024-09-19T06:52:22.986 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.986 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.986 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.986 INFO:tasks.workunit.client.0.smithi103.stdout:49/600: open fd = 3 2024-09-19T06:52:22.986 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.986 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.986 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.986 INFO:tasks.workunit.client.0.smithi103.stdout:50/600: open fd = 3 2024-09-19T06:52:22.986 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.986 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.986 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.987 INFO:tasks.workunit.client.0.smithi103.stdout:51/600: open fd = 3 2024-09-19T06:52:22.987 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.987 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.987 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.987 INFO:tasks.workunit.client.0.smithi103.stdout:52/600: open fd = 3 2024-09-19T06:52:22.987 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.987 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.987 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.987 INFO:tasks.workunit.client.0.smithi103.stdout:53/600: open fd = 3 2024-09-19T06:52:22.987 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.987 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.987 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.987 INFO:tasks.workunit.client.0.smithi103.stdout:54/600: open fd = 3 2024-09-19T06:52:22.987 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.987 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.988 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.988 INFO:tasks.workunit.client.0.smithi103.stdout:55/600: open fd = 3 2024-09-19T06:52:22.988 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.988 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.988 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.988 INFO:tasks.workunit.client.0.smithi103.stdout:56/600: open fd = 3 2024-09-19T06:52:22.988 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.988 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.988 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.988 INFO:tasks.workunit.client.0.smithi103.stdout:57/600: open fd = 3 2024-09-19T06:52:22.988 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.988 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.988 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.988 INFO:tasks.workunit.client.0.smithi103.stdout:58/600: open fd = 3 2024-09-19T06:52:22.989 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.989 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.989 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.989 INFO:tasks.workunit.client.0.smithi103.stdout:59/600: open fd = 3 2024-09-19T06:52:22.989 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.989 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.989 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.989 INFO:tasks.workunit.client.0.smithi103.stdout:60/600: open fd = 3 2024-09-19T06:52:22.989 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.989 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.989 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.989 INFO:tasks.workunit.client.0.smithi103.stdout:61/600: open fd = 3 2024-09-19T06:52:22.989 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.989 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.989 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.990 INFO:tasks.workunit.client.0.smithi103.stdout:62/600: open fd = 3 2024-09-19T06:52:22.990 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.990 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.990 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:22.990 INFO:tasks.workunit.client.0.smithi103.stdout:63/600: open fd = 3 2024-09-19T06:52:22.990 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.990 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:52:22.990 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:52:24.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:24 smithi174 ceph-mon[35637]: pgmap v98: 97 pgs: 97 active+clean; 2.7 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-09-19T06:52:24.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:24 smithi103 ceph-mon[31158]: pgmap v98: 97 pgs: 97 active+clean; 2.7 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-09-19T06:52:24.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:24 smithi134 ceph-mon[35356]: pgmap v98: 97 pgs: 97 active+clean; 2.7 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-09-19T06:52:25.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:25 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:52:25.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:25 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:52:25.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:25 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:52:26.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:26 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:52:26.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:26 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:52:26.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:26 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:52:26.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:26 smithi174 ceph-mon[35637]: pgmap v99: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 23 op/s 2024-09-19T06:52:26.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:26 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:52:26.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:26 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:52:26.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:26 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:52:26.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:26 smithi103 ceph-mon[31158]: pgmap v99: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 23 op/s 2024-09-19T06:52:26.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:26 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:52:26.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:26 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:52:26.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:26 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:52:26.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:26 smithi134 ceph-mon[35356]: pgmap v99: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 23 op/s 2024-09-19T06:52:28.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:28 smithi174 ceph-mon[35637]: pgmap v100: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-09-19T06:52:28.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:28 smithi103 ceph-mon[31158]: pgmap v100: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-09-19T06:52:28.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:28 smithi134 ceph-mon[35356]: pgmap v100: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-09-19T06:52:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.259+0000 7f838af86640 1 -- 172.21.15.103:0/2730403706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f838410c0b0 msgr2=0x7f838410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.259+0000 7f838af86640 1 --2- 172.21.15.103:0/2730403706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f838410c0b0 0x7f838410e550 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f837c009ca0 tx=0x7f837c02a950 comp rx=0 tx=0).stop 2024-09-19T06:52:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.259+0000 7f838af86640 1 -- 172.21.15.103:0/2730403706 shutdown_connections 2024-09-19T06:52:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.259+0000 7f838af86640 1 --2- 172.21.15.103:0/2730403706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f838410c0b0 0x7f838410e550 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.259+0000 7f838af86640 1 --2- 172.21.15.103:0/2730403706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8384103600 0x7f838410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.259+0000 7f838af86640 1 --2- 172.21.15.103:0/2730403706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8384102ce0 0x7f83841030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.259+0000 7f838af86640 1 -- 172.21.15.103:0/2730403706 >> 172.21.15.103:0/2730403706 conn(0x7f8384077990 msgr2=0x7f83840fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.259+0000 7f838af86640 1 -- 172.21.15.103:0/2730403706 shutdown_connections 2024-09-19T06:52:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.259+0000 7f838af86640 1 -- 172.21.15.103:0/2730403706 wait complete. 2024-09-19T06:52:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f838af86640 1 Processor -- start 2024-09-19T06:52:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f838af86640 1 -- start start 2024-09-19T06:52:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f838af86640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8384102ce0 0x7f8384100930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f838af86640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8384103600 0x7f8384100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f838af86640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f838410c0b0 0x7f83841013b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f838af86640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8384111eb0 con 0x7f838410c0b0 2024-09-19T06:52:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f838af86640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8384111d30 con 0x7f8384102ce0 2024-09-19T06:52:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f838af86640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8384112030 con 0x7f8384103600 2024-09-19T06:52:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f8389523640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f838410c0b0 0x7f83841013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f8389523640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f838410c0b0 0x7f83841013b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53488/0 (socket says 172.21.15.103:53488) 2024-09-19T06:52:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f8389523640 1 -- 172.21.15.103:0/1562996177 learned_addr learned my addr 172.21.15.103:0/1562996177 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f8383fff640 1 --2- 172.21.15.103:0/1562996177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8384103600 0x7f8384100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f8388d22640 1 --2- 172.21.15.103:0/1562996177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8384102ce0 0x7f8384100930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f8389523640 1 -- 172.21.15.103:0/1562996177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8384103600 msgr2=0x7f8384100e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f8389523640 1 --2- 172.21.15.103:0/1562996177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8384103600 0x7f8384100e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f8389523640 1 -- 172.21.15.103:0/1562996177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8384102ce0 msgr2=0x7f8384100930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f8389523640 1 --2- 172.21.15.103:0/1562996177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8384102ce0 0x7f8384100930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.260+0000 7f8389523640 1 -- 172.21.15.103:0/1562996177 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83841a6760 con 0x7f838410c0b0 2024-09-19T06:52:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.261+0000 7f8389523640 1 --2- 172.21.15.103:0/1562996177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f838410c0b0 0x7f83841013b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f837c004af0 tx=0x7f837c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.261+0000 7f8381ffb640 1 -- 172.21.15.103:0/1562996177 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837c034040 con 0x7f838410c0b0 2024-09-19T06:52:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.261+0000 7f838af86640 1 -- 172.21.15.103:0/1562996177 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83841a69f0 con 0x7f838410c0b0 2024-09-19T06:52:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.261+0000 7f8381ffb640 1 -- 172.21.15.103:0/1562996177 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f837c030920 con 0x7f838410c0b0 2024-09-19T06:52:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.261+0000 7f8381ffb640 1 -- 172.21.15.103:0/1562996177 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837c004710 con 0x7f838410c0b0 2024-09-19T06:52:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.261+0000 7f838af86640 1 -- 172.21.15.103:0/1562996177 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83841a6ee0 con 0x7f838410c0b0 2024-09-19T06:52:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.263+0000 7f8381ffb640 1 -- 172.21.15.103:0/1562996177 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f837c037020 con 0x7f838410c0b0 2024-09-19T06:52:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.264+0000 7f838ddf6740 1 Processor -- start 2024-09-19T06:52:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.264+0000 7f838ddf6740 1 -- start start 2024-09-19T06:52:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.264+0000 7f838ddf6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461ae6af0 0x558461ae6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.264+0000 7f838ddf6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461ae7410 0x558461af1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.264+0000 7f838ddf6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461af1cb0 0x558461af40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.264+0000 7f838ddf6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558461a4b2f0 con 0x558461ae7410 2024-09-19T06:52:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.264+0000 7f838ddf6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5584619f3270 con 0x558461af1cb0 2024-09-19T06:52:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.264+0000 7f838ddf6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558461a472b0 con 0x558461ae6af0 2024-09-19T06:52:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.264+0000 7f8381ffb640 1 --2- 172.21.15.103:0/1562996177 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83740a3460 0x7f83740a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.264+0000 7f8388d22640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461ae7410 0x558461af1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.264+0000 7f8388d22640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461ae7410 0x558461af1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53494/0 (socket says 172.21.15.103:53494) 2024-09-19T06:52:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.264+0000 7f8388d22640 1 -- 172.21.15.103:0/3222193405 learned_addr learned my addr 172.21.15.103:0/3222193405 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.264+0000 7f8389523640 1 --2- 172.21.15.103:0/1562996177 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83740a3460 0x7f83740a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.264+0000 7f8381ffb640 1 -- 172.21.15.103:0/1562996177 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f837c0f0a60 con 0x7f838410c0b0 2024-09-19T06:52:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.264+0000 7f8383fff640 1 --2- 172.21.15.103:0/3222193405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461af1cb0 0x558461af40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.265+0000 7f8383fff640 1 --2- 172.21.15.103:0/3222193405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461ae6af0 0x558461ae6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.265+0000 7f8388d22640 1 -- 172.21.15.103:0/3222193405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461ae6af0 msgr2=0x558461ae6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.265+0000 7f8388d22640 1 --2- 172.21.15.103:0/3222193405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461ae6af0 0x558461ae6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.265+0000 7f8388d22640 1 -- 172.21.15.103:0/3222193405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461af1cb0 msgr2=0x558461af40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.265+0000 7f8388d22640 1 --2- 172.21.15.103:0/3222193405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461af1cb0 0x558461af40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.265+0000 7f8388d22640 1 -- 172.21.15.103:0/3222193405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584619f2340 con 0x558461ae7410 2024-09-19T06:52:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.265+0000 7f8383fff640 1 --2- 172.21.15.103:0/3222193405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461af1cb0 0x558461af40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.265+0000 7f8388d22640 1 --2- 172.21.15.103:0/3222193405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461ae7410 0x558461af1770 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8378009c80 tx=0x7f83780117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df64b26a1a6b87ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.265+0000 7f8389523640 1 --2- 172.21.15.103:0/1562996177 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83740a3460 0x7f83740a5920 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f837c0b92c0 tx=0x7f837c0b22f0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.265+0000 7f836effd640 1 -- 172.21.15.103:0/3222193405 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378012450 con 0x558461ae7410 2024-09-19T06:52:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.265+0000 7f836effd640 1 -- 172.21.15.103:0/3222193405 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8378012ad0 con 0x558461ae7410 2024-09-19T06:52:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.265+0000 7f836effd640 1 -- 172.21.15.103:0/3222193405 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837800fc00 con 0x558461ae7410 2024-09-19T06:52:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.266+0000 7f838ddf6740 1 -- 172.21.15.103:0/3222193405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461ae7410 msgr2=0x558461af1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.266+0000 7f838ddf6740 1 --2- 172.21.15.103:0/3222193405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461ae7410 0x558461af1770 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8378009c80 tx=0x7f83780117b0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.266+0000 7f838ddf6740 1 -- 172.21.15.103:0/3222193405 shutdown_connections 2024-09-19T06:52:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.266+0000 7f838ddf6740 1 --2- 172.21.15.103:0/3222193405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461af1cb0 0x558461af40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.266+0000 7f838ddf6740 1 --2- 172.21.15.103:0/3222193405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461ae7410 0x558461af1770 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.266+0000 7f838ddf6740 1 --2- 172.21.15.103:0/3222193405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461ae6af0 0x558461ae6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.266+0000 7f838ddf6740 1 -- 172.21.15.103:0/3222193405 >> 172.21.15.103:0/3222193405 conn(0x558461ae3060 msgr2=0x558461ae3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.266+0000 7f838ddf6740 1 -- 172.21.15.103:0/3222193405 shutdown_connections 2024-09-19T06:52:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.266+0000 7f838ddf6740 1 -- 172.21.15.103:0/3222193405 wait complete. 2024-09-19T06:52:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.266+0000 7f838ddf6740 1 -- start start 2024-09-19T06:52:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.266+0000 7f838ddf6740 1 Processor -- start 2024-09-19T06:52:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f838ddf6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461ae6af0 0x558461aeeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f838ddf6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461ae7410 0x558461aef0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f838ddf6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461af1cb0 0x558461aef610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f838ddf6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558461af4880 con 0x558461af1cb0 2024-09-19T06:52:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f838ddf6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5584619d4290 con 0x558461ae6af0 2024-09-19T06:52:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f838ddf6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558461af4a00 con 0x558461ae7410 2024-09-19T06:52:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f838ddf6740 10 client.0 _ll_register_callbacks cb 0x558461a05140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f8383fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461af1cb0 0x558461aef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f8383fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461af1cb0 0x558461aef610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53502/0 (socket says 172.21.15.103:53502) 2024-09-19T06:52:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f8383fff640 1 -- 172.21.15.103:0/4112202731 learned_addr learned my addr 172.21.15.103:0/4112202731 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f8388d22640 1 --2- 172.21.15.103:0/4112202731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461ae7410 0x558461aef0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f8383fff640 1 --2- 172.21.15.103:0/4112202731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461ae6af0 0x558461aeeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f8383fff640 1 -- 172.21.15.103:0/4112202731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461ae7410 msgr2=0x558461aef0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f8383fff640 1 --2- 172.21.15.103:0/4112202731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461ae7410 0x558461aef0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f8383fff640 1 -- 172.21.15.103:0/4112202731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461ae6af0 msgr2=0x558461aeeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f8383fff640 1 --2- 172.21.15.103:0/4112202731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461ae6af0 0x558461aeeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f8383fff640 1 -- 172.21.15.103:0/4112202731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846175e210 con 0x558461af1cb0 2024-09-19T06:52:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f8383fff640 1 --2- 172.21.15.103:0/4112202731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461af1cb0 0x558461aef610 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f8370011e10 tx=0x7f8370013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c423e0770e3f2114 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f8363fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.267+0000 7f8363fff640 1 -- 172.21.15.103:0/4112202731 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370023060 con 0x558461af1cb0 2024-09-19T06:52:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.268+0000 7f8363fff640 1 -- 172.21.15.103:0/4112202731 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f837001e430 con 0x558461af1cb0 2024-09-19T06:52:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.268+0000 7f838ddf6740 1 -- 172.21.15.103:0/4112202731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8378012ad0 con 0x558461af1cb0 2024-09-19T06:52:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.268+0000 7f8363fff640 1 -- 172.21.15.103:0/4112202731 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f837001e630 con 0x558461af1cb0 2024-09-19T06:52:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.268+0000 7f838ddf6740 10 client.16632 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.268+0000 7f838ddf6740 1 -- 172.21.15.103:0/4112202731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55846182a620 con 0x558461af1cb0 2024-09-19T06:52:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.268+0000 7f8363fff640 1 -- 172.21.15.103:0/4112202731 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8378012ad0 con 0x558461af1cb0 2024-09-19T06:52:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.268+0000 7f8363fff640 1 -- 172.21.15.103:0/4112202731 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f837001d070 con 0x558461af1cb0 2024-09-19T06:52:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.268+0000 7f838ddf6740 10 client.16632 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.268+0000 7f838ddf6740 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.268+0000 7f838ddf6740 20 client.16632 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.268+0000 7f838ddf6740 1 --2- 172.21.15.103:0/4112202731 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558461b01c80 0x558461b04070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.268+0000 7f838ddf6740 4 client.16632 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.268+0000 7f838ddf6740 1 -- 172.21.15.103:0/4112202731 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558461a58a20 con 0x558461b01c80 2024-09-19T06:52:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.268+0000 7f8388d22640 1 --2- 172.21.15.103:0/4112202731 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558461b01c80 0x558461b04070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.269+0000 7f8363fff640 10 client.16632 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.269+0000 7f8388d22640 1 --2- 172.21.15.103:0/4112202731 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558461b01c80 0x558461b04070 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d82557ab16ab46a1 server_cookie=fb7609a61712d1b4 in_seq=0 out_seq=0 2024-09-19T06:52:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.270+0000 7f8363fff640 1 -- 172.21.15.103:0/4112202731 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558461a58a20 con 0x558461b01c80 2024-09-19T06:52:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.270+0000 7f8363fff640 10 client.16632 handle_command_reply: tid=0 2024-09-19T06:52:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.270+0000 7f838ddf6740 1 client.16632 shutdown 2024-09-19T06:52:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.270+0000 7f838ddf6740 1 -- 172.21.15.103:0/4112202731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461af1cb0 msgr2=0x558461aef610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.270+0000 7f838ddf6740 1 --2- 172.21.15.103:0/4112202731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461af1cb0 0x558461aef610 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f8370011e10 tx=0x7f8370013040 comp rx=0 tx=0).stop 2024-09-19T06:52:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.270+0000 7f838ddf6740 1 -- 172.21.15.103:0/4112202731 shutdown_connections 2024-09-19T06:52:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.270+0000 7f838ddf6740 1 --2- 172.21.15.103:0/4112202731 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558461b01c80 0x558461b04070 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.270+0000 7f838ddf6740 1 --2- 172.21.15.103:0/4112202731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461af1cb0 0x558461aef610 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.270+0000 7f8363fff640 0 client.16632 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.270+0000 7f838ddf6740 1 --2- 172.21.15.103:0/4112202731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461ae7410 0x558461aef0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.270+0000 7f838ddf6740 1 --2- 172.21.15.103:0/4112202731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461ae6af0 0x558461aeeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.270+0000 7f838ddf6740 1 -- 172.21.15.103:0/4112202731 >> 172.21.15.103:0/4112202731 conn(0x558461ae3060 msgr2=0x558461ae8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.270+0000 7f838ddf6740 1 -- 172.21.15.103:0/4112202731 shutdown_connections 2024-09-19T06:52:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.270+0000 7f838ddf6740 1 -- 172.21.15.103:0/4112202731 wait complete. 2024-09-19T06:52:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.270+0000 7f838ddf6740 20 client.16632 trim_cache size 0 max 16384 2024-09-19T06:52:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.284+0000 7f838ddf6740 1 Processor -- start 2024-09-19T06:52:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.284+0000 7f838ddf6740 1 -- start start 2024-09-19T06:52:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f838ddf6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461ae6af0 0x558461aef440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f838ddf6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461ae7410 0x558461b098f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f838ddf6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461af1cb0 0x558461b09e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f838ddf6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f83841af8c0 con 0x558461ae6af0 2024-09-19T06:52:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f838ddf6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83841b1400 con 0x558461ae7410 2024-09-19T06:52:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f838ddf6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83841aff90 con 0x558461af1cb0 2024-09-19T06:52:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f8383fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461ae6af0 0x558461aef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f8383fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461ae6af0 0x558461aef440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53508/0 (socket says 172.21.15.103:53508) 2024-09-19T06:52:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f8383fff640 1 -- 172.21.15.103:0/3004040567 learned_addr learned my addr 172.21.15.103:0/3004040567 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f8383fff640 1 -- 172.21.15.103:0/3004040567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461af1cb0 msgr2=0x558461b09e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f8383fff640 1 --2- 172.21.15.103:0/3004040567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461af1cb0 0x558461b09e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f8388d22640 1 --2- 172.21.15.103:0/3004040567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461ae7410 0x558461b098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f8383fff640 1 -- 172.21.15.103:0/3004040567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461ae7410 msgr2=0x558461b098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f8383fff640 1 --2- 172.21.15.103:0/3004040567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461ae7410 0x558461b098f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f8383fff640 1 -- 172.21.15.103:0/3004040567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558461af1720 con 0x558461ae6af0 2024-09-19T06:52:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f8383fff640 1 --2- 172.21.15.103:0/3004040567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461ae6af0 0x558461aef440 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8370011de0 tx=0x7f837001e5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470b52e754b37b3e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f836e7fc640 1 -- 172.21.15.103:0/3004040567 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370000e60 con 0x558461ae6af0 2024-09-19T06:52:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f836e7fc640 1 -- 172.21.15.103:0/3004040567 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f837000a350 con 0x558461ae6af0 2024-09-19T06:52:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.285+0000 7f836e7fc640 1 -- 172.21.15.103:0/3004040567 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370026560 con 0x558461ae6af0 2024-09-19T06:52:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.286+0000 7f838ddf6740 1 -- 172.21.15.103:0/3004040567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461ae6af0 msgr2=0x558461aef440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.286+0000 7f838ddf6740 1 --2- 172.21.15.103:0/3004040567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461ae6af0 0x558461aef440 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8370011de0 tx=0x7f837001e5c0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.286+0000 7f838ddf6740 1 -- 172.21.15.103:0/3004040567 shutdown_connections 2024-09-19T06:52:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.286+0000 7f838ddf6740 1 --2- 172.21.15.103:0/3004040567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461af1cb0 0x558461b09e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.286+0000 7f838ddf6740 1 --2- 172.21.15.103:0/3004040567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461ae7410 0x558461b098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.286+0000 7f838ddf6740 1 --2- 172.21.15.103:0/3004040567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461ae6af0 0x558461aef440 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.286+0000 7f838ddf6740 1 -- 172.21.15.103:0/3004040567 >> 172.21.15.103:0/3004040567 conn(0x558461ae3060 msgr2=0x558461af0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.286+0000 7f838ddf6740 1 -- 172.21.15.103:0/3004040567 shutdown_connections 2024-09-19T06:52:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.286+0000 7f838ddf6740 1 -- 172.21.15.103:0/3004040567 wait complete. 2024-09-19T06:52:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.286+0000 7f838ddf6740 1 -- start start 2024-09-19T06:52:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.286+0000 7f838ddf6740 1 Processor -- start 2024-09-19T06:52:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.287+0000 7f838ddf6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461ae6af0 0x558461afa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.287+0000 7f838ddf6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461ae7410 0x558461af5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.287+0000 7f838ddf6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461af1cb0 0x558461af5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.287+0000 7f838ddf6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f83841b0d30 con 0x558461af1cb0 2024-09-19T06:52:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.287+0000 7f838ddf6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83841b0660 con 0x558461ae7410 2024-09-19T06:52:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.287+0000 7f838ddf6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83841ad880 con 0x558461ae6af0 2024-09-19T06:52:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.287+0000 7f838ddf6740 10 client.0 _ll_register_callbacks cb 0x7f8370005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.287+0000 7f8383fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461af1cb0 0x558461af5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.287+0000 7f8383fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461af1cb0 0x558461af5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53514/0 (socket says 172.21.15.103:53514) 2024-09-19T06:52:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.287+0000 7f8383fff640 1 -- 172.21.15.103:0/3205256822 learned_addr learned my addr 172.21.15.103:0/3205256822 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.287+0000 7f8383fff640 1 -- 172.21.15.103:0/3205256822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461ae6af0 msgr2=0x558461afa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.287+0000 7f8383fff640 1 --2- 172.21.15.103:0/3205256822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461ae6af0 0x558461afa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.287+0000 7f8383fff640 1 -- 172.21.15.103:0/3205256822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461ae7410 msgr2=0x558461af5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.287+0000 7f8383fff640 1 --2- 172.21.15.103:0/3205256822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461ae7410 0x558461af5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.287+0000 7f8383fff640 1 -- 172.21.15.103:0/3205256822 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558461ae9940 con 0x558461af1cb0 2024-09-19T06:52:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.288+0000 7f8383fff640 1 --2- 172.21.15.103:0/3205256822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461af1cb0 0x558461af5c60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8370026750 tx=0x7f837001e520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6277781e1901bfeb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.288+0000 7f8363fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.288+0000 7f8363fff640 1 -- 172.21.15.103:0/3205256822 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370035070 con 0x558461af1cb0 2024-09-19T06:52:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.288+0000 7f8363fff640 1 -- 172.21.15.103:0/3205256822 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f837000fc20 con 0x558461af1cb0 2024-09-19T06:52:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.288+0000 7f838ddf6740 1 -- 172.21.15.103:0/3205256822 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558461a52d70 con 0x558461af1cb0 2024-09-19T06:52:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.288+0000 7f8363fff640 1 -- 172.21.15.103:0/3205256822 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8370023030 con 0x558461af1cb0 2024-09-19T06:52:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.288+0000 7f838ddf6740 10 client.16638 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.288+0000 7f838ddf6740 1 -- 172.21.15.103:0/3205256822 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558461b6c700 con 0x558461af1cb0 2024-09-19T06:52:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.288+0000 7f8363fff640 1 -- 172.21.15.103:0/3205256822 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558461a52d70 con 0x558461af1cb0 2024-09-19T06:52:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.288+0000 7f8363fff640 1 -- 172.21.15.103:0/3205256822 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8370030030 con 0x558461af1cb0 2024-09-19T06:52:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.288+0000 7f838ddf6740 10 client.16638 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.288+0000 7f838ddf6740 10 client.16638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.288+0000 7f838ddf6740 20 client.16638 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.288+0000 7f838ddf6740 1 --2- 172.21.15.103:0/3205256822 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558461b6cdb0 0x558461b6f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.288+0000 7f838ddf6740 4 client.16638 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:52:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.288+0000 7f838ddf6740 1 -- 172.21.15.103:0/3205256822 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5584619cf900 con 0x558461b6cdb0 2024-09-19T06:52:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.289+0000 7f8388d22640 1 --2- 172.21.15.103:0/3205256822 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558461b6cdb0 0x558461b6f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.289+0000 7f8363fff640 10 client.16638 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.289+0000 7f8388d22640 1 --2- 172.21.15.103:0/3205256822 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558461b6cdb0 0x558461b6f1a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0b3bf346ecf930f server_cookie=dd83205a21791d5e in_seq=0 out_seq=0 2024-09-19T06:52:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.303+0000 7f8363fff640 1 -- 172.21.15.103:0/3205256822 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5584619cf900 con 0x558461b6cdb0 2024-09-19T06:52:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.303+0000 7f8363fff640 10 client.16638 handle_command_reply: tid=0 2024-09-19T06:52:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.303+0000 7f838ddf6740 1 client.16638 shutdown 2024-09-19T06:52:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.303+0000 7f838ddf6740 1 -- 172.21.15.103:0/3205256822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461af1cb0 msgr2=0x558461af5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.303+0000 7f838ddf6740 1 --2- 172.21.15.103:0/3205256822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461af1cb0 0x558461af5c60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8370026750 tx=0x7f837001e520 comp rx=0 tx=0).stop 2024-09-19T06:52:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.303+0000 7f838ddf6740 1 -- 172.21.15.103:0/3205256822 shutdown_connections 2024-09-19T06:52:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.303+0000 7f838ddf6740 1 --2- 172.21.15.103:0/3205256822 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558461b6cdb0 0x558461b6f1a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.303+0000 7f838ddf6740 1 --2- 172.21.15.103:0/3205256822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558461af1cb0 0x558461af5c60 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.303+0000 7f838ddf6740 1 --2- 172.21.15.103:0/3205256822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558461ae7410 0x558461af5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.303+0000 7f8363fff640 0 client.16638 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.303+0000 7f838ddf6740 1 --2- 172.21.15.103:0/3205256822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558461ae6af0 0x558461afa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.303+0000 7f838ddf6740 1 -- 172.21.15.103:0/3205256822 >> 172.21.15.103:0/3205256822 conn(0x558461ae3060 msgr2=0x558461ae19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.303+0000 7f838ddf6740 1 -- 172.21.15.103:0/3205256822 shutdown_connections 2024-09-19T06:52:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.303+0000 7f838ddf6740 1 -- 172.21.15.103:0/3205256822 wait complete. 2024-09-19T06:52:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.303+0000 7f838ddf6740 20 client.16638 trim_cache size 0 max 16384 2024-09-19T06:52:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:52:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:52:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:52:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:52:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.304+0000 7f838af86640 1 -- 172.21.15.103:0/1562996177 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83740a3460 msgr2=0x7f83740a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.304+0000 7f838af86640 1 --2- 172.21.15.103:0/1562996177 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83740a3460 0x7f83740a5920 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f837c0b92c0 tx=0x7f837c0b22f0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.304+0000 7f838af86640 1 -- 172.21.15.103:0/1562996177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f838410c0b0 msgr2=0x7f83841013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.304+0000 7f838af86640 1 --2- 172.21.15.103:0/1562996177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f838410c0b0 0x7f83841013b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f837c004af0 tx=0x7f837c02f5d0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.304+0000 7f838af86640 1 -- 172.21.15.103:0/1562996177 shutdown_connections 2024-09-19T06:52:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.304+0000 7f838af86640 1 --2- 172.21.15.103:0/1562996177 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83740a3460 0x7f83740a5920 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.304+0000 7f838af86640 1 --2- 172.21.15.103:0/1562996177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f838410c0b0 0x7f83841013b0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.304+0000 7f838af86640 1 --2- 172.21.15.103:0/1562996177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8384103600 0x7f8384100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.304+0000 7f838af86640 1 --2- 172.21.15.103:0/1562996177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8384102ce0 0x7f8384100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.304+0000 7f838af86640 1 -- 172.21.15.103:0/1562996177 >> 172.21.15.103:0/1562996177 conn(0x7f8384077990 msgr2=0x7f83840fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.304+0000 7f838af86640 1 -- 172.21.15.103:0/1562996177 shutdown_connections 2024-09-19T06:52:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:29.304+0000 7f838af86640 1 -- 172.21.15.103:0/1562996177 wait complete. 2024-09-19T06:52:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:30 smithi174 ceph-mon[35637]: pgmap v101: 97 pgs: 97 active+clean; 2.9 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 25 op/s 2024-09-19T06:52:30.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:30 smithi103 ceph-mon[31158]: pgmap v101: 97 pgs: 97 active+clean; 2.9 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 25 op/s 2024-09-19T06:52:30.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:30 smithi134 ceph-mon[35356]: pgmap v101: 97 pgs: 97 active+clean; 2.9 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 25 op/s 2024-09-19T06:52:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:32 smithi174 ceph-mon[35637]: pgmap v102: 97 pgs: 97 active+clean; 2.9 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-09-19T06:52:32.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:32 smithi103 ceph-mon[31158]: pgmap v102: 97 pgs: 97 active+clean; 2.9 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-09-19T06:52:32.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:32 smithi134 ceph-mon[35356]: pgmap v102: 97 pgs: 97 active+clean; 2.9 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-09-19T06:52:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:34 smithi174 ceph-mon[35637]: pgmap v103: 97 pgs: 97 active+clean; 2.9 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-09-19T06:52:34.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:34 smithi103 ceph-mon[31158]: pgmap v103: 97 pgs: 97 active+clean; 2.9 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-09-19T06:52:34.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:34 smithi134 ceph-mon[35356]: pgmap v103: 97 pgs: 97 active+clean; 2.9 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-09-19T06:52:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:36 smithi174 ceph-mon[35637]: pgmap v104: 97 pgs: 97 active+clean; 3.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 21 op/s 2024-09-19T06:52:36.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:36 smithi103 ceph-mon[31158]: pgmap v104: 97 pgs: 97 active+clean; 3.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 21 op/s 2024-09-19T06:52:36.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:36 smithi134 ceph-mon[35356]: pgmap v104: 97 pgs: 97 active+clean; 3.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 21 op/s 2024-09-19T06:52:38.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:38 smithi174 ceph-mon[35637]: pgmap v105: 97 pgs: 97 active+clean; 3.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 KiB/s wr, 15 op/s 2024-09-19T06:52:38.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:38 smithi103 ceph-mon[31158]: pgmap v105: 97 pgs: 97 active+clean; 3.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 KiB/s wr, 15 op/s 2024-09-19T06:52:38.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:38 smithi134 ceph-mon[35356]: pgmap v105: 97 pgs: 97 active+clean; 3.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 KiB/s wr, 15 op/s 2024-09-19T06:52:40.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:40 smithi174 ceph-mon[35637]: pgmap v106: 97 pgs: 97 active+clean; 3.2 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 21 op/s 2024-09-19T06:52:40.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:40 smithi103 ceph-mon[31158]: pgmap v106: 97 pgs: 97 active+clean; 3.2 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 21 op/s 2024-09-19T06:52:40.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:40 smithi134 ceph-mon[35356]: pgmap v106: 97 pgs: 97 active+clean; 3.2 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 21 op/s 2024-09-19T06:52:40.759 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-19T06:52:40.760 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:52:41.021 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:41.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.470+0000 7fafa3aa3640 1 -- 172.21.15.103:0/3868128532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf9c10e480 msgr2=0x7faf9c110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:41.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.470+0000 7fafa3aa3640 1 --2- 172.21.15.103:0/3868128532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf9c10e480 0x7faf9c110870 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7faf94009ca0 tx=0x7faf9402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:52:41.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.470+0000 7fafa3aa3640 1 -- 172.21.15.103:0/3868128532 shutdown_connections 2024-09-19T06:52:41.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.470+0000 7fafa3aa3640 1 --2- 172.21.15.103:0/3868128532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf9c10e480 0x7faf9c110870 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:41.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.470+0000 7fafa3aa3640 1 --2- 172.21.15.103:0/3868128532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf9c101280 0x7faf9c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:41.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.470+0000 7fafa3aa3640 1 --2- 172.21.15.103:0/3868128532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf9c1008d0 0x7faf9c100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.470+0000 7fafa3aa3640 1 -- 172.21.15.103:0/3868128532 >> 172.21.15.103:0/3868128532 conn(0x7faf9c0fc7a0 msgr2=0x7faf9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.471+0000 7fafa3aa3640 1 -- 172.21.15.103:0/3868128532 shutdown_connections 2024-09-19T06:52:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.471+0000 7fafa3aa3640 1 -- 172.21.15.103:0/3868128532 wait complete. 2024-09-19T06:52:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.471+0000 7fafa3aa3640 1 Processor -- start 2024-09-19T06:52:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.471+0000 7fafa3aa3640 1 -- start start 2024-09-19T06:52:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.471+0000 7fafa3aa3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf9c1008d0 0x7faf9c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.471+0000 7fafa3aa3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf9c101280 0x7faf9c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.471+0000 7fafa3aa3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf9c10e480 0x7faf9c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.471+0000 7fafa3aa3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c114160 con 0x7faf9c10e480 2024-09-19T06:52:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.471+0000 7fafa3aa3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c113fe0 con 0x7faf9c1008d0 2024-09-19T06:52:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.471+0000 7fafa3aa3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1142e0 con 0x7faf9c101280 2024-09-19T06:52:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7fafa2040640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf9c10e480 0x7faf9c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7fafa2040640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf9c10e480 0x7faf9c19eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43858/0 (socket says 172.21.15.103:43858) 2024-09-19T06:52:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7fafa2040640 1 -- 172.21.15.103:0/1474659214 learned_addr learned my addr 172.21.15.103:0/1474659214 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7fafa183f640 1 --2- 172.21.15.103:0/1474659214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf9c1008d0 0x7faf9c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7fafa103e640 1 --2- 172.21.15.103:0/1474659214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf9c101280 0x7faf9c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7fafa2040640 1 -- 172.21.15.103:0/1474659214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf9c101280 msgr2=0x7faf9c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7fafa2040640 1 --2- 172.21.15.103:0/1474659214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf9c101280 0x7faf9c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7fafa2040640 1 -- 172.21.15.103:0/1474659214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf9c1008d0 msgr2=0x7faf9c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7fafa2040640 1 --2- 172.21.15.103:0/1474659214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf9c1008d0 0x7faf9c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7fafa2040640 1 -- 172.21.15.103:0/1474659214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9c19f590 con 0x7faf9c10e480 2024-09-19T06:52:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7fafa103e640 1 --2- 172.21.15.103:0/1474659214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf9c101280 0x7faf9c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7fafa2040640 1 --2- 172.21.15.103:0/1474659214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf9c10e480 0x7faf9c19eeb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7faf9402ade0 tx=0x7faf940319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7faf92ffd640 1 -- 172.21.15.103:0/1474659214 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94003c30 con 0x7faf9c10e480 2024-09-19T06:52:41.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7faf92ffd640 1 -- 172.21.15.103:0/1474659214 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf94003dd0 con 0x7faf9c10e480 2024-09-19T06:52:41.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7faf92ffd640 1 -- 172.21.15.103:0/1474659214 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9402fd60 con 0x7faf9c10e480 2024-09-19T06:52:41.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7fafa3aa3640 1 -- 172.21.15.103:0/1474659214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9c19f820 con 0x7faf9c10e480 2024-09-19T06:52:41.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.472+0000 7fafa3aa3640 1 -- 172.21.15.103:0/1474659214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf9c1a7160 con 0x7faf9c10e480 2024-09-19T06:52:41.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.473+0000 7fafa3aa3640 1 -- 172.21.15.103:0/1474659214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf9c100cb0 con 0x7faf9c10e480 2024-09-19T06:52:41.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.474+0000 7faf92ffd640 1 -- 172.21.15.103:0/1474659214 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7faf94004050 con 0x7faf9c10e480 2024-09-19T06:52:41.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.475+0000 7faf92ffd640 1 --2- 172.21.15.103:0/1474659214 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faf880a3430 0x7faf880a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:41.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.475+0000 7fafa183f640 1 --2- 172.21.15.103:0/1474659214 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faf880a3430 0x7faf880a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:41.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.475+0000 7faf92ffd640 1 -- 172.21.15.103:0/1474659214 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7faf940f10a0 con 0x7faf9c10e480 2024-09-19T06:52:41.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.475+0000 7fafa183f640 1 --2- 172.21.15.103:0/1474659214 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faf880a3430 0x7faf880a58f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faf9800bbe0 tx=0x7faf9800b390 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:41.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.479+0000 7faf92ffd640 1 -- 172.21.15.103:0/1474659214 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faf940bdd50 con 0x7faf9c10e480 2024-09-19T06:52:41.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.682+0000 7fafa3aa3640 1 -- 172.21.15.103:0/1474659214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf9c1a73c0 con 0x7faf9c10e480 2024-09-19T06:52:41.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.683+0000 7faf92ffd640 1 -- 172.21.15.103:0/1474659214 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7faf940bd650 con 0x7faf9c10e480 2024-09-19T06:52:41.711 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:52:41.711 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:52:41.711 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:52:41.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.686+0000 7fafa3aa3640 1 -- 172.21.15.103:0/1474659214 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faf880a3430 msgr2=0x7faf880a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:41.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.686+0000 7fafa3aa3640 1 --2- 172.21.15.103:0/1474659214 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faf880a3430 0x7faf880a58f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faf9800bbe0 tx=0x7faf9800b390 comp rx=0 tx=0).stop 2024-09-19T06:52:41.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.686+0000 7fafa3aa3640 1 -- 172.21.15.103:0/1474659214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf9c10e480 msgr2=0x7faf9c19eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:41.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.686+0000 7fafa3aa3640 1 --2- 172.21.15.103:0/1474659214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf9c10e480 0x7faf9c19eeb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7faf9402ade0 tx=0x7faf940319f0 comp rx=0 tx=0).stop 2024-09-19T06:52:41.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.686+0000 7fafa3aa3640 1 -- 172.21.15.103:0/1474659214 shutdown_connections 2024-09-19T06:52:41.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.686+0000 7fafa3aa3640 1 --2- 172.21.15.103:0/1474659214 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faf880a3430 0x7faf880a58f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:41.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.686+0000 7fafa3aa3640 1 --2- 172.21.15.103:0/1474659214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf9c10e480 0x7faf9c19eeb0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:41.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.686+0000 7fafa3aa3640 1 --2- 172.21.15.103:0/1474659214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf9c101280 0x7faf9c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:41.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.686+0000 7fafa3aa3640 1 --2- 172.21.15.103:0/1474659214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf9c1008d0 0x7faf9c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:41.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.686+0000 7fafa3aa3640 1 -- 172.21.15.103:0/1474659214 >> 172.21.15.103:0/1474659214 conn(0x7faf9c0fc7a0 msgr2=0x7faf9c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:41.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.686+0000 7fafa3aa3640 1 -- 172.21.15.103:0/1474659214 shutdown_connections 2024-09-19T06:52:41.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:41.686+0000 7fafa3aa3640 1 -- 172.21.15.103:0/1474659214 wait complete. 2024-09-19T06:52:42.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:41 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1474659214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:42.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:41 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1474659214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:42.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:41 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1474659214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.393+0000 7f1efbbd6640 1 -- 172.21.15.103:0/4193416937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ef41035d0 msgr2=0x7f1ef410bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.393+0000 7f1efbbd6640 1 --2- 172.21.15.103:0/4193416937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ef41035d0 0x7f1ef410bab0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009900 tx=0x7f1ef002a480 comp rx=0 tx=0).stop 2024-09-19T06:52:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.393+0000 7f1efbbd6640 1 -- 172.21.15.103:0/4193416937 shutdown_connections 2024-09-19T06:52:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.393+0000 7f1efbbd6640 1 --2- 172.21.15.103:0/4193416937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ef410c0d0 0x7f1ef410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.393+0000 7f1efbbd6640 1 --2- 172.21.15.103:0/4193416937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ef41035d0 0x7f1ef410bab0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.393+0000 7f1efbbd6640 1 --2- 172.21.15.103:0/4193416937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ef4102cb0 0x7f1ef4103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.393+0000 7f1efbbd6640 1 -- 172.21.15.103:0/4193416937 >> 172.21.15.103:0/4193416937 conn(0x7f1ef40fc6c0 msgr2=0x7f1ef40feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.393+0000 7f1efbbd6640 1 -- 172.21.15.103:0/4193416937 shutdown_connections 2024-09-19T06:52:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.393+0000 7f1efbbd6640 1 -- 172.21.15.103:0/4193416937 wait complete. 2024-09-19T06:52:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1efbbd6640 1 Processor -- start 2024-09-19T06:52:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1efbbd6640 1 -- start start 2024-09-19T06:52:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1efbbd6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ef4102cb0 0x7f1ef4100900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1efbbd6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ef41035d0 0x7f1ef4100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1efbbd6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ef410c0d0 0x7f1ef4101380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1efbbd6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef4111ef0 con 0x7f1ef41035d0 2024-09-19T06:52:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1efbbd6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef4111d70 con 0x7f1ef4102cb0 2024-09-19T06:52:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1efbbd6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef4112070 con 0x7f1ef410c0d0 2024-09-19T06:52:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1ef9171640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ef41035d0 0x7f1ef4100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1ef9171640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ef41035d0 0x7f1ef4100e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43866/0 (socket says 172.21.15.103:43866) 2024-09-19T06:52:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1ef9171640 1 -- 172.21.15.103:0/3105077789 learned_addr learned my addr 172.21.15.103:0/3105077789 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1ef9972640 1 --2- 172.21.15.103:0/3105077789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ef4102cb0 0x7f1ef4100900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1efa173640 1 --2- 172.21.15.103:0/3105077789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ef410c0d0 0x7f1ef4101380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1ef9171640 1 -- 172.21.15.103:0/3105077789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ef410c0d0 msgr2=0x7f1ef4101380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1ef9171640 1 --2- 172.21.15.103:0/3105077789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ef410c0d0 0x7f1ef4101380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1ef9171640 1 -- 172.21.15.103:0/3105077789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ef4102cb0 msgr2=0x7f1ef4100900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1ef9171640 1 --2- 172.21.15.103:0/3105077789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ef4102cb0 0x7f1ef4100900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.394+0000 7f1ef9171640 1 -- 172.21.15.103:0/3105077789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef41a6880 con 0x7f1ef41035d0 2024-09-19T06:52:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.395+0000 7f1ef9171640 1 --2- 172.21.15.103:0/3105077789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ef41035d0 0x7f1ef4100e40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0006010 tx=0x7f1ef00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.395+0000 7f1eeaffd640 1 -- 172.21.15.103:0/3105077789 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef003f030 con 0x7f1ef41035d0 2024-09-19T06:52:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.395+0000 7f1efbbd6640 1 -- 172.21.15.103:0/3105077789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef41a6b10 con 0x7f1ef41035d0 2024-09-19T06:52:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.395+0000 7f1eeaffd640 1 -- 172.21.15.103:0/3105077789 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef0003ce0 con 0x7f1ef41035d0 2024-09-19T06:52:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.395+0000 7f1eeaffd640 1 -- 172.21.15.103:0/3105077789 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0035070 con 0x7f1ef41035d0 2024-09-19T06:52:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.395+0000 7f1efbbd6640 1 -- 172.21.15.103:0/3105077789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef41a7050 con 0x7f1ef41035d0 2024-09-19T06:52:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.397+0000 7f1eeaffd640 1 -- 172.21.15.103:0/3105077789 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1ef0042030 con 0x7f1ef41035d0 2024-09-19T06:52:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.397+0000 7f1efea52740 1 Processor -- start 2024-09-19T06:52:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1efea52740 1 -- start start 2024-09-19T06:52:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1eeaffd640 1 --2- 172.21.15.103:0/3105077789 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1ee00a3460 0x7f1ee00a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1efea52740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe826daf0 0x55dbe826ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1efea52740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826e410 0x55dbe8278770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1efea52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe8278cb0 0x55dbe827b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1efea52740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dbe81d22f0 con 0x55dbe826e410 2024-09-19T06:52:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1efea52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dbe817a270 con 0x55dbe8278cb0 2024-09-19T06:52:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1efea52740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dbe81ce2b0 con 0x55dbe826daf0 2024-09-19T06:52:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1ef9171640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826e410 0x55dbe8278770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1efa173640 1 --2- 172.21.15.103:0/3105077789 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1ee00a3460 0x7f1ee00a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1ef9171640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826e410 0x55dbe8278770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43870/0 (socket says 172.21.15.103:43870) 2024-09-19T06:52:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1ef9171640 1 -- 172.21.15.103:0/2957624702 learned_addr learned my addr 172.21.15.103:0/2957624702 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1eeaffd640 1 -- 172.21.15.103:0/3105077789 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1ef00f02e0 con 0x7f1ef41035d0 2024-09-19T06:52:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1ef9972640 1 --2- 172.21.15.103:0/2957624702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe826daf0 0x55dbe826ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1efa173640 1 --2- 172.21.15.103:0/2957624702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe8278cb0 0x55dbe827b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1ef9171640 1 -- 172.21.15.103:0/2957624702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe826daf0 msgr2=0x55dbe826ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1ef9171640 1 --2- 172.21.15.103:0/2957624702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe826daf0 0x55dbe826ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1ef9171640 1 -- 172.21.15.103:0/2957624702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe8278cb0 msgr2=0x55dbe827b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1ef9171640 1 --2- 172.21.15.103:0/2957624702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe8278cb0 0x55dbe827b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.398+0000 7f1ef9171640 1 -- 172.21.15.103:0/2957624702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe8179340 con 0x55dbe826e410 2024-09-19T06:52:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.399+0000 7f1efa173640 1 --2- 172.21.15.103:0/3105077789 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1ee00a3460 0x7f1ee00a5920 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0092f0 tx=0x7f1eec009f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.399+0000 7f1ef9171640 1 --2- 172.21.15.103:0/2957624702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826e410 0x55dbe8278770 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00ed970 tx=0x7f1ef003b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beee7750b7ffc174 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.399+0000 7f1edffff640 1 -- 172.21.15.103:0/2957624702 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0049020 con 0x55dbe826e410 2024-09-19T06:52:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.399+0000 7f1edffff640 1 -- 172.21.15.103:0/2957624702 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef00d5070 con 0x55dbe826e410 2024-09-19T06:52:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.399+0000 7f1edffff640 1 -- 172.21.15.103:0/2957624702 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00d3bc0 con 0x55dbe826e410 2024-09-19T06:52:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.399+0000 7f1efea52740 1 -- 172.21.15.103:0/2957624702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826e410 msgr2=0x55dbe8278770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.399+0000 7f1efea52740 1 --2- 172.21.15.103:0/2957624702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826e410 0x55dbe8278770 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00ed970 tx=0x7f1ef003b720 comp rx=0 tx=0).stop 2024-09-19T06:52:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.400+0000 7f1efea52740 1 -- 172.21.15.103:0/2957624702 shutdown_connections 2024-09-19T06:52:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.400+0000 7f1efea52740 1 --2- 172.21.15.103:0/2957624702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe8278cb0 0x55dbe827b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.400+0000 7f1efea52740 1 --2- 172.21.15.103:0/2957624702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826e410 0x55dbe8278770 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.400+0000 7f1efea52740 1 --2- 172.21.15.103:0/2957624702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe826daf0 0x55dbe826ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.400+0000 7f1efea52740 1 -- 172.21.15.103:0/2957624702 >> 172.21.15.103:0/2957624702 conn(0x55dbe826a060 msgr2=0x55dbe826a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.400+0000 7f1efea52740 1 -- 172.21.15.103:0/2957624702 shutdown_connections 2024-09-19T06:52:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.400+0000 7f1efea52740 1 -- 172.21.15.103:0/2957624702 wait complete. 2024-09-19T06:52:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.400+0000 7f1efea52740 1 -- start start 2024-09-19T06:52:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.400+0000 7f1efea52740 1 Processor -- start 2024-09-19T06:52:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.400+0000 7f1efea52740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826daf0 0x55dbe8275b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.400+0000 7f1efea52740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe826e410 0x55dbe82760d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.400+0000 7f1efea52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe8278cb0 0x55dbe8276610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.400+0000 7f1efea52740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dbe827b640 con 0x55dbe826daf0 2024-09-19T06:52:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.400+0000 7f1efea52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dbe815b290 con 0x55dbe8278cb0 2024-09-19T06:52:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.400+0000 7f1efea52740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dbe827b7c0 con 0x55dbe826e410 2024-09-19T06:52:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1ef9972640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826daf0 0x55dbe8275b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1efea52740 10 client.0 _ll_register_callbacks cb 0x55dbe818c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1ef9972640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826daf0 0x55dbe8275b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43882/0 (socket says 172.21.15.103:43882) 2024-09-19T06:52:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1ef9972640 1 -- 172.21.15.103:0/2857540339 learned_addr learned my addr 172.21.15.103:0/2857540339 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1efa173640 1 --2- 172.21.15.103:0/2857540339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe8278cb0 0x55dbe8276610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1ef9171640 1 --2- 172.21.15.103:0/2857540339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe826e410 0x55dbe82760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1ef9972640 1 -- 172.21.15.103:0/2857540339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe826e410 msgr2=0x55dbe82760d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1ef9972640 1 --2- 172.21.15.103:0/2857540339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe826e410 0x55dbe82760d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1ef9972640 1 -- 172.21.15.103:0/2857540339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe8278cb0 msgr2=0x55dbe8276610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1ef9972640 1 --2- 172.21.15.103:0/2857540339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe8278cb0 0x55dbe8276610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1ef9972640 1 -- 172.21.15.103:0/2857540339 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe7ee5210 con 0x55dbe826daf0 2024-09-19T06:52:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1efa173640 1 --2- 172.21.15.103:0/2857540339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe8278cb0 0x55dbe8276610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1ef9972640 1 --2- 172.21.15.103:0/2857540339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826daf0 0x55dbe8275b90 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400bab0 tx=0x7f1ee400bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc05ac39c0ceae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1edd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1edd7fa640 1 -- 172.21.15.103:0/2857540339 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4011440 con 0x55dbe826daf0 2024-09-19T06:52:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1edd7fa640 1 -- 172.21.15.103:0/2857540339 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1ee40115e0 con 0x55dbe826daf0 2024-09-19T06:52:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1efea52740 1 -- 172.21.15.103:0/2857540339 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef00d5070 con 0x55dbe826daf0 2024-09-19T06:52:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1edd7fa640 1 -- 172.21.15.103:0/2857540339 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1ee40117e0 con 0x55dbe826daf0 2024-09-19T06:52:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1efea52740 10 client.16656 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.401+0000 7f1efea52740 1 -- 172.21.15.103:0/2857540339 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dbe7fb1620 con 0x55dbe826daf0 2024-09-19T06:52:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.402+0000 7f1edd7fa640 1 -- 172.21.15.103:0/2857540339 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef00d5070 con 0x55dbe826daf0 2024-09-19T06:52:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.402+0000 7f1edd7fa640 1 -- 172.21.15.103:0/2857540339 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1ee401ba80 con 0x55dbe826daf0 2024-09-19T06:52:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.402+0000 7f1efea52740 10 client.16656 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.402+0000 7f1efea52740 10 client.16656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.402+0000 7f1efea52740 20 client.16656 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.402+0000 7f1efea52740 1 --2- 172.21.15.103:0/2857540339 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dbe8288950 0x55dbe828adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.402+0000 7f1efea52740 4 client.16656 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.402+0000 7f1efea52740 1 -- 172.21.15.103:0/2857540339 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbe81dfa20 con 0x55dbe8288950 2024-09-19T06:52:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.402+0000 7f1ef9171640 1 --2- 172.21.15.103:0/2857540339 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dbe8288950 0x55dbe828adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.403+0000 7f1edd7fa640 10 client.16656 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.403+0000 7f1ef9171640 1 --2- 172.21.15.103:0/2857540339 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dbe8288950 0x55dbe828adc0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce58174a3964fd89 server_cookie=b98b4fceec1eca59 in_seq=0 out_seq=0 2024-09-19T06:52:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.403+0000 7f1edd7fa640 1 -- 172.21.15.103:0/2857540339 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dbe81dfa20 con 0x55dbe8288950 2024-09-19T06:52:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.403+0000 7f1edd7fa640 10 client.16656 handle_command_reply: tid=0 2024-09-19T06:52:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.403+0000 7f1efea52740 1 client.16656 shutdown 2024-09-19T06:52:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.404+0000 7f1efea52740 1 -- 172.21.15.103:0/2857540339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826daf0 msgr2=0x55dbe8275b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.404+0000 7f1efea52740 1 --2- 172.21.15.103:0/2857540339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826daf0 0x55dbe8275b90 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400bab0 tx=0x7f1ee400bf70 comp rx=0 tx=0).stop 2024-09-19T06:52:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.404+0000 7f1efea52740 1 -- 172.21.15.103:0/2857540339 shutdown_connections 2024-09-19T06:52:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.404+0000 7f1efea52740 1 --2- 172.21.15.103:0/2857540339 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dbe8288950 0x55dbe828adc0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.404+0000 7f1efea52740 1 --2- 172.21.15.103:0/2857540339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe8278cb0 0x55dbe8276610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.404+0000 7f1efea52740 1 --2- 172.21.15.103:0/2857540339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe826e410 0x55dbe82760d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.404+0000 7f1efea52740 1 --2- 172.21.15.103:0/2857540339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826daf0 0x55dbe8275b90 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.404+0000 7f1edd7fa640 0 client.16656 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.404+0000 7f1efea52740 1 -- 172.21.15.103:0/2857540339 >> 172.21.15.103:0/2857540339 conn(0x55dbe826a060 msgr2=0x55dbe826fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.404+0000 7f1efea52740 1 -- 172.21.15.103:0/2857540339 shutdown_connections 2024-09-19T06:52:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.404+0000 7f1efea52740 1 -- 172.21.15.103:0/2857540339 wait complete. 2024-09-19T06:52:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.404+0000 7f1efea52740 20 client.16656 trim_cache size 0 max 16384 2024-09-19T06:52:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1efea52740 1 Processor -- start 2024-09-19T06:52:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1efea52740 1 -- start start 2024-09-19T06:52:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1efea52740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826daf0 0x55dbe82760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1efea52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe826e410 0x55dbe8276600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1efea52740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe8278cb0 0x55dbe82906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1efea52740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41afa40 con 0x55dbe826daf0 2024-09-19T06:52:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1efea52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41b1580 con 0x55dbe826e410 2024-09-19T06:52:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1efea52740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41b0110 con 0x55dbe8278cb0 2024-09-19T06:52:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1ef9972640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826daf0 0x55dbe82760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1ef9972640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826daf0 0x55dbe82760c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43898/0 (socket says 172.21.15.103:43898) 2024-09-19T06:52:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1ef9972640 1 -- 172.21.15.103:0/2199428774 learned_addr learned my addr 172.21.15.103:0/2199428774 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1ef9171640 1 --2- 172.21.15.103:0/2199428774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe826e410 0x55dbe8276600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1ef9972640 1 -- 172.21.15.103:0/2199428774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe8278cb0 msgr2=0x55dbe82906d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1ef9972640 1 --2- 172.21.15.103:0/2199428774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe8278cb0 0x55dbe82906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1ef9972640 1 -- 172.21.15.103:0/2199428774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe826e410 msgr2=0x55dbe8276600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1ef9972640 1 --2- 172.21.15.103:0/2199428774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe826e410 0x55dbe8276600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1ef9972640 1 -- 172.21.15.103:0/2199428774 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe826acf0 con 0x55dbe826daf0 2024-09-19T06:52:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.418+0000 7f1ef9171640 1 --2- 172.21.15.103:0/2199428774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe826e410 0x55dbe8276600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.419+0000 7f1ef9972640 1 --2- 172.21.15.103:0/2199428774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826daf0 0x55dbe82760c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4006010 tx=0x7f1ee4011700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12105a83970dd5f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.419+0000 7f1edf7fe640 1 -- 172.21.15.103:0/2199428774 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4011980 con 0x55dbe826daf0 2024-09-19T06:52:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.419+0000 7f1edf7fe640 1 -- 172.21.15.103:0/2199428774 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ee4005040 con 0x55dbe826daf0 2024-09-19T06:52:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.419+0000 7f1edf7fe640 1 -- 172.21.15.103:0/2199428774 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee401b7a0 con 0x55dbe826daf0 2024-09-19T06:52:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.419+0000 7f1efea52740 1 -- 172.21.15.103:0/2199428774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826daf0 msgr2=0x55dbe82760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.419+0000 7f1efea52740 1 --2- 172.21.15.103:0/2199428774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826daf0 0x55dbe82760c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4006010 tx=0x7f1ee4011700 comp rx=0 tx=0).stop 2024-09-19T06:52:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.419+0000 7f1efea52740 1 -- 172.21.15.103:0/2199428774 shutdown_connections 2024-09-19T06:52:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.419+0000 7f1efea52740 1 --2- 172.21.15.103:0/2199428774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe8278cb0 0x55dbe82906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.419+0000 7f1efea52740 1 --2- 172.21.15.103:0/2199428774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe826e410 0x55dbe8276600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.419+0000 7f1efea52740 1 --2- 172.21.15.103:0/2199428774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe826daf0 0x55dbe82760c0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.419+0000 7f1efea52740 1 -- 172.21.15.103:0/2199428774 >> 172.21.15.103:0/2199428774 conn(0x55dbe826a060 msgr2=0x55dbe8277c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.419+0000 7f1efea52740 1 -- 172.21.15.103:0/2199428774 shutdown_connections 2024-09-19T06:52:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.419+0000 7f1efea52740 1 -- 172.21.15.103:0/2199428774 wait complete. 2024-09-19T06:52:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efea52740 1 -- start start 2024-09-19T06:52:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efea52740 1 Processor -- start 2024-09-19T06:52:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efea52740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe826daf0 0x55dbe82810d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efea52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe826e410 0x55dbe827c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efea52740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe8278cb0 0x55dbe827c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efea52740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41b0eb0 con 0x55dbe8278cb0 2024-09-19T06:52:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efea52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41b07e0 con 0x55dbe826e410 2024-09-19T06:52:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efea52740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41ada00 con 0x55dbe826daf0 2024-09-19T06:52:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efea52740 10 client.0 _ll_register_callbacks cb 0x7f1ee4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efa173640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe8278cb0 0x55dbe827c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efa173640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe8278cb0 0x55dbe827c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43904/0 (socket says 172.21.15.103:43904) 2024-09-19T06:52:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efa173640 1 -- 172.21.15.103:0/2339075892 learned_addr learned my addr 172.21.15.103:0/2339075892 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1ef9171640 1 --2- 172.21.15.103:0/2339075892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe826e410 0x55dbe827c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1ef9972640 1 --2- 172.21.15.103:0/2339075892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe826daf0 0x55dbe82810d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efa173640 1 -- 172.21.15.103:0/2339075892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe826daf0 msgr2=0x55dbe82810d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efa173640 1 --2- 172.21.15.103:0/2339075892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe826daf0 0x55dbe82810d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efa173640 1 -- 172.21.15.103:0/2339075892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe826e410 msgr2=0x55dbe827c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efa173640 1 --2- 172.21.15.103:0/2339075892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe826e410 0x55dbe827c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1efa173640 1 -- 172.21.15.103:0/2339075892 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe827d140 con 0x55dbe8278cb0 2024-09-19T06:52:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1ef9972640 1 --2- 172.21.15.103:0/2339075892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe826daf0 0x55dbe82810d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.420+0000 7f1ef9171640 1 --2- 172.21.15.103:0/2339075892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe826e410 0x55dbe827c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.421+0000 7f1efa173640 1 --2- 172.21.15.103:0/2339075892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe8278cb0 0x55dbe827c930 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0116a0 tx=0x7f1eec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe6afa624941dba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.421+0000 7f1edd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.421+0000 7f1edd7fa640 1 -- 172.21.15.103:0/2339075892 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec024070 con 0x55dbe8278cb0 2024-09-19T06:52:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.421+0000 7f1edd7fa640 1 -- 172.21.15.103:0/2339075892 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1eec0208c0 con 0x55dbe8278cb0 2024-09-19T06:52:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.421+0000 7f1efea52740 1 -- 172.21.15.103:0/2339075892 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ee4005040 con 0x55dbe8278cb0 2024-09-19T06:52:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.421+0000 7f1edd7fa640 1 -- 172.21.15.103:0/2339075892 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1eec020ac0 con 0x55dbe8278cb0 2024-09-19T06:52:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.421+0000 7f1efea52740 10 client.16662 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.421+0000 7f1efea52740 1 -- 172.21.15.103:0/2339075892 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dbe82f3590 con 0x55dbe8278cb0 2024-09-19T06:52:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.421+0000 7f1edd7fa640 1 -- 172.21.15.103:0/2339075892 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ee4005040 con 0x55dbe8278cb0 2024-09-19T06:52:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.421+0000 7f1edd7fa640 1 -- 172.21.15.103:0/2339075892 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1eec032420 con 0x55dbe8278cb0 2024-09-19T06:52:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.421+0000 7f1efea52740 10 client.16662 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.421+0000 7f1efea52740 10 client.16662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.421+0000 7f1efea52740 20 client.16662 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.421+0000 7f1efea52740 1 --2- 172.21.15.103:0/2339075892 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dbe82f3cc0 0x55dbe82f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.421+0000 7f1efea52740 4 client.16662 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:52:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.421+0000 7f1efea52740 1 -- 172.21.15.103:0/2339075892 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dbe81d9d70 con 0x55dbe82f3cc0 2024-09-19T06:52:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.422+0000 7f1ef9972640 1 --2- 172.21.15.103:0/2339075892 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dbe82f3cc0 0x55dbe82f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.422+0000 7f1edd7fa640 10 client.16662 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.422+0000 7f1ef9972640 1 --2- 172.21.15.103:0/2339075892 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dbe82f3cc0 0x55dbe82f60e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6118e4cc4e4832cf server_cookie=50b71e4d0972060 in_seq=0 out_seq=0 2024-09-19T06:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.434+0000 7f1edd7fa640 1 -- 172.21.15.103:0/2339075892 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dbe81d9d70 con 0x55dbe82f3cc0 2024-09-19T06:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.434+0000 7f1edd7fa640 10 client.16662 handle_command_reply: tid=0 2024-09-19T06:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.434+0000 7f1efea52740 1 client.16662 shutdown 2024-09-19T06:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.435+0000 7f1efea52740 1 -- 172.21.15.103:0/2339075892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe8278cb0 msgr2=0x55dbe827c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.435+0000 7f1efea52740 1 --2- 172.21.15.103:0/2339075892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe8278cb0 0x55dbe827c930 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0116a0 tx=0x7f1eec004ab0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.435+0000 7f1efea52740 1 -- 172.21.15.103:0/2339075892 shutdown_connections 2024-09-19T06:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.435+0000 7f1efea52740 1 --2- 172.21.15.103:0/2339075892 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dbe82f3cc0 0x55dbe82f60e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.435+0000 7f1efea52740 1 --2- 172.21.15.103:0/2339075892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dbe8278cb0 0x55dbe827c930 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.435+0000 7f1efea52740 1 --2- 172.21.15.103:0/2339075892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbe826e410 0x55dbe827c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.435+0000 7f1efea52740 1 --2- 172.21.15.103:0/2339075892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dbe826daf0 0x55dbe82810d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.435+0000 7f1efea52740 1 -- 172.21.15.103:0/2339075892 >> 172.21.15.103:0/2339075892 conn(0x55dbe826a060 msgr2=0x55dbe82689a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.435+0000 7f1edd7fa640 0 client.16662 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.435+0000 7f1efea52740 1 -- 172.21.15.103:0/2339075892 shutdown_connections 2024-09-19T06:52:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.435+0000 7f1efea52740 1 -- 172.21.15.103:0/2339075892 wait complete. 2024-09-19T06:52:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.435+0000 7f1efea52740 20 client.16662 trim_cache size 0 max 16384 2024-09-19T06:52:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:52:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:52:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:52:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:52:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.435+0000 7f1efbbd6640 1 -- 172.21.15.103:0/3105077789 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1ee00a3460 msgr2=0x7f1ee00a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.435+0000 7f1efbbd6640 1 --2- 172.21.15.103:0/3105077789 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1ee00a3460 0x7f1ee00a5920 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0092f0 tx=0x7f1eec009f90 comp rx=0 tx=0).stop 2024-09-19T06:52:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.435+0000 7f1efbbd6640 1 -- 172.21.15.103:0/3105077789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ef41035d0 msgr2=0x7f1ef4100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.435+0000 7f1efbbd6640 1 --2- 172.21.15.103:0/3105077789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ef41035d0 0x7f1ef4100e40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0006010 tx=0x7f1ef00042f0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.436+0000 7f1efbbd6640 1 -- 172.21.15.103:0/3105077789 shutdown_connections 2024-09-19T06:52:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.436+0000 7f1efbbd6640 1 --2- 172.21.15.103:0/3105077789 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1ee00a3460 0x7f1ee00a5920 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.436+0000 7f1efbbd6640 1 --2- 172.21.15.103:0/3105077789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ef410c0d0 0x7f1ef4101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.436+0000 7f1efbbd6640 1 --2- 172.21.15.103:0/3105077789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ef41035d0 0x7f1ef4100e40 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.436+0000 7f1efbbd6640 1 --2- 172.21.15.103:0/3105077789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ef4102cb0 0x7f1ef4100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.436+0000 7f1efbbd6640 1 -- 172.21.15.103:0/3105077789 >> 172.21.15.103:0/3105077789 conn(0x7f1ef40fc6c0 msgr2=0x7f1ef40fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.436+0000 7f1efbbd6640 1 -- 172.21.15.103:0/3105077789 shutdown_connections 2024-09-19T06:52:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:42.436+0000 7f1efbbd6640 1 -- 172.21.15.103:0/3105077789 wait complete. 2024-09-19T06:52:42.461 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-19T06:52:42.461 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-19T06:52:42.750 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:42.989 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:42 smithi103 ceph-mon[31158]: pgmap v107: 97 pgs: 97 active+clean; 3.2 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-09-19T06:52:43.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:42 smithi174 ceph-mon[35637]: pgmap v107: 97 pgs: 97 active+clean; 3.2 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-09-19T06:52:43.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.174+0000 7ff98395c640 1 -- 172.21.15.103:0/2533068626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c100a80 msgr2=0x7ff97c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:43.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.174+0000 7ff98395c640 1 --2- 172.21.15.103:0/2533068626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c100a80 0x7ff97c100ee0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009930 tx=0x7ff96c02a420 comp rx=0 tx=0).stop 2024-09-19T06:52:43.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.174+0000 7ff98395c640 1 -- 172.21.15.103:0/2533068626 shutdown_connections 2024-09-19T06:52:43.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.174+0000 7ff98395c640 1 --2- 172.21.15.103:0/2533068626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff97c101420 0x7ff97c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.174+0000 7ff98395c640 1 --2- 172.21.15.103:0/2533068626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c100a80 0x7ff97c100ee0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.174+0000 7ff98395c640 1 --2- 172.21.15.103:0/2533068626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff97c106a80 0x7ff97c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.174+0000 7ff98395c640 1 -- 172.21.15.103:0/2533068626 >> 172.21.15.103:0/2533068626 conn(0x7ff97c0fc7a0 msgr2=0x7ff97c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:43.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.174+0000 7ff98395c640 1 -- 172.21.15.103:0/2533068626 shutdown_connections 2024-09-19T06:52:43.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.174+0000 7ff98395c640 1 -- 172.21.15.103:0/2533068626 wait complete. 2024-09-19T06:52:43.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.175+0000 7ff98395c640 1 Processor -- start 2024-09-19T06:52:43.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.175+0000 7ff98395c640 1 -- start start 2024-09-19T06:52:43.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.175+0000 7ff98395c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff97c100a80 0x7ff97c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.175+0000 7ff98395c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff97c101420 0x7ff97c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.175+0000 7ff98395c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106a80 0x7ff97c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.175+0000 7ff98395c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c110e40 con 0x7ff97c106a80 2024-09-19T06:52:43.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.175+0000 7ff98395c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c110cc0 con 0x7ff97c101420 2024-09-19T06:52:43.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.175+0000 7ff98395c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c110fc0 con 0x7ff97c100a80 2024-09-19T06:52:43.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.175+0000 7ff981ef9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106a80 0x7ff97c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:43.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.175+0000 7ff981ef9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106a80 0x7ff97c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43920/0 (socket says 172.21.15.103:43920) 2024-09-19T06:52:43.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.175+0000 7ff981ef9640 1 -- 172.21.15.103:0/1626135160 learned_addr learned my addr 172.21.15.103:0/1626135160 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:43.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.175+0000 7ff9816f8640 1 --2- 172.21.15.103:0/1626135160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff97c100a80 0x7ff97c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:43.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.175+0000 7ff980ef7640 1 --2- 172.21.15.103:0/1626135160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff97c101420 0x7ff97c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:43.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.176+0000 7ff981ef9640 1 -- 172.21.15.103:0/1626135160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff97c100a80 msgr2=0x7ff97c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:43.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.176+0000 7ff981ef9640 1 --2- 172.21.15.103:0/1626135160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff97c100a80 0x7ff97c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.176+0000 7ff981ef9640 1 -- 172.21.15.103:0/1626135160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff97c101420 msgr2=0x7ff97c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:43.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.176+0000 7ff981ef9640 1 --2- 172.21.15.103:0/1626135160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff97c101420 0x7ff97c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.176+0000 7ff981ef9640 1 -- 172.21.15.103:0/1626135160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97c19d400 con 0x7ff97c106a80 2024-09-19T06:52:43.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.176+0000 7ff9816f8640 1 --2- 172.21.15.103:0/1626135160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff97c100a80 0x7ff97c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:43.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.176+0000 7ff981ef9640 1 --2- 172.21.15.103:0/1626135160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106a80 0x7ff97c19cd20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff97400eaa0 tx=0x7ff97400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:43.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.176+0000 7ff9727fc640 1 -- 172.21.15.103:0/1626135160 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97400cdb0 con 0x7ff97c106a80 2024-09-19T06:52:43.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.176+0000 7ff98395c640 1 -- 172.21.15.103:0/1626135160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97c19d6f0 con 0x7ff97c106a80 2024-09-19T06:52:43.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.176+0000 7ff9727fc640 1 -- 172.21.15.103:0/1626135160 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff974005020 con 0x7ff97c106a80 2024-09-19T06:52:43.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.176+0000 7ff9727fc640 1 -- 172.21.15.103:0/1626135160 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9740116d0 con 0x7ff97c106a80 2024-09-19T06:52:43.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.176+0000 7ff98395c640 1 -- 172.21.15.103:0/1626135160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97c1a4fd0 con 0x7ff97c106a80 2024-09-19T06:52:43.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.178+0000 7ff9727fc640 1 -- 172.21.15.103:0/1626135160 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff9740054d0 con 0x7ff97c106a80 2024-09-19T06:52:43.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.179+0000 7ff985d35740 1 Processor -- start 2024-09-19T06:52:43.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.179+0000 7ff985d35740 1 -- start start 2024-09-19T06:52:43.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.179+0000 7ff985d35740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d7342760 0x5567d7342b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.179+0000 7ff985d35740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d7343080 0x5567d734d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.179+0000 7ff985d35740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d734d940 0x5567d734fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.179+0000 7ff985d35740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5567d72a82a0 con 0x5567d7343080 2024-09-19T06:52:43.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.179+0000 7ff985d35740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5567d724fb10 con 0x5567d7342760 2024-09-19T06:52:43.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.179+0000 7ff985d35740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5567d72a4260 con 0x5567d734d940 2024-09-19T06:52:43.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.179+0000 7ff9816f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d7343080 0x5567d734d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:43.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.179+0000 7ff9816f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d7343080 0x5567d734d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43926/0 (socket says 172.21.15.103:43926) 2024-09-19T06:52:43.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.179+0000 7ff9816f8640 1 -- 172.21.15.103:0/409905909 learned_addr learned my addr 172.21.15.103:0/409905909 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:43.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.179+0000 7ff9727fc640 1 --2- 172.21.15.103:0/1626135160 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9680a33b0 0x7ff9680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.179+0000 7ff980ef7640 1 --2- 172.21.15.103:0/409905909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d7342760 0x5567d7342b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:43.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.180+0000 7ff980ef7640 1 --2- 172.21.15.103:0/409905909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d734d940 0x5567d734fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:43.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.180+0000 7ff981ef9640 1 --2- 172.21.15.103:0/1626135160 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9680a33b0 0x7ff9680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:43.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.180+0000 7ff9816f8640 1 -- 172.21.15.103:0/409905909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d734d940 msgr2=0x5567d734fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:43.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.180+0000 7ff9816f8640 1 --2- 172.21.15.103:0/409905909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d734d940 0x5567d734fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.180+0000 7ff9816f8640 1 -- 172.21.15.103:0/409905909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d7342760 msgr2=0x5567d7342b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:43.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.180+0000 7ff9727fc640 1 -- 172.21.15.103:0/1626135160 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff9740cffe0 con 0x7ff97c106a80 2024-09-19T06:52:43.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.180+0000 7ff9816f8640 1 --2- 172.21.15.103:0/409905909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d7342760 0x5567d7342b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.180+0000 7ff9816f8640 1 -- 172.21.15.103:0/409905909 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567d7019b90 con 0x5567d7343080 2024-09-19T06:52:43.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.180+0000 7ff980ef7640 1 --2- 172.21.15.103:0/409905909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d734d940 0x5567d734fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:43.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.180+0000 7ff9816f8640 1 --2- 172.21.15.103:0/409905909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d7343080 0x5567d734d400 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff97800cc20 tx=0x7ff97800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437c66b6aa866013 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:43.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.180+0000 7ff981ef9640 1 --2- 172.21.15.103:0/1626135160 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9680a33b0 0x7ff9680a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff974098730 tx=0x7ff974076040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:43.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.180+0000 7ff9677fe640 1 -- 172.21.15.103:0/409905909 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978005220 con 0x5567d7343080 2024-09-19T06:52:43.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.180+0000 7ff9677fe640 1 -- 172.21.15.103:0/409905909 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff978016070 con 0x5567d7343080 2024-09-19T06:52:43.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.180+0000 7ff9677fe640 1 -- 172.21.15.103:0/409905909 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978010690 con 0x5567d7343080 2024-09-19T06:52:43.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.181+0000 7ff985d35740 1 -- 172.21.15.103:0/409905909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d7343080 msgr2=0x5567d734d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:43.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.181+0000 7ff985d35740 1 --2- 172.21.15.103:0/409905909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d7343080 0x5567d734d400 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff97800cc20 tx=0x7ff97800e720 comp rx=0 tx=0).stop 2024-09-19T06:52:43.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.181+0000 7ff985d35740 1 -- 172.21.15.103:0/409905909 shutdown_connections 2024-09-19T06:52:43.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.181+0000 7ff985d35740 1 --2- 172.21.15.103:0/409905909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d734d940 0x5567d734fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.181+0000 7ff985d35740 1 --2- 172.21.15.103:0/409905909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d7343080 0x5567d734d400 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.181+0000 7ff985d35740 1 --2- 172.21.15.103:0/409905909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d7342760 0x5567d7342b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.181+0000 7ff985d35740 1 -- 172.21.15.103:0/409905909 >> 172.21.15.103:0/409905909 conn(0x5567d733ecf0 msgr2=0x5567d733f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:43.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.181+0000 7ff985d35740 1 -- 172.21.15.103:0/409905909 shutdown_connections 2024-09-19T06:52:43.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.181+0000 7ff985d35740 1 -- 172.21.15.103:0/409905909 wait complete. 2024-09-19T06:52:43.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.181+0000 7ff985d35740 1 -- start start 2024-09-19T06:52:43.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff985d35740 1 Processor -- start 2024-09-19T06:52:43.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff985d35740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d7342760 0x5567d734a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff985d35740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d7343080 0x5567d734ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff985d35740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d734d940 0x5567d734b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff985d35740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5567d7350510 con 0x5567d734d940 2024-09-19T06:52:43.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff985d35740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5567d72306e0 con 0x5567d7343080 2024-09-19T06:52:43.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff985d35740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5567d7350690 con 0x5567d7342760 2024-09-19T06:52:43.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff985d35740 10 client.0 _ll_register_callbacks cb 0x5567d7254a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:43.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff980ef7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d734d940 0x5567d734b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:43.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff980ef7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d734d940 0x5567d734b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43936/0 (socket says 172.21.15.103:43936) 2024-09-19T06:52:43.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff980ef7640 1 -- 172.21.15.103:0/3568186600 learned_addr learned my addr 172.21.15.103:0/3568186600 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff9816f8640 1 --2- 172.21.15.103:0/3568186600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d7343080 0x5567d734ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff980ef7640 1 -- 172.21.15.103:0/3568186600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d7342760 msgr2=0x5567d734a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff980ef7640 1 --2- 172.21.15.103:0/3568186600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d7342760 0x5567d734a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff980ef7640 1 -- 172.21.15.103:0/3568186600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d7343080 msgr2=0x5567d734ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff980ef7640 1 --2- 172.21.15.103:0/3568186600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d7343080 0x5567d734ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.182+0000 7ff980ef7640 1 -- 172.21.15.103:0/3568186600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567d73599d0 con 0x5567d734d940 2024-09-19T06:52:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.183+0000 7ff980ef7640 1 --2- 172.21.15.103:0/3568186600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d734d940 0x5567d734b2b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff96c033000 tx=0x7ff96c0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9024a5906b532cab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.183+0000 7ff964ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.183+0000 7ff964ff9640 1 -- 172.21.15.103:0/3568186600 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c045070 con 0x5567d734d940 2024-09-19T06:52:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.183+0000 7ff964ff9640 1 -- 172.21.15.103:0/3568186600 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff96c004240 con 0x5567d734d940 2024-09-19T06:52:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.183+0000 7ff985d35740 1 -- 172.21.15.103:0/3568186600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff978016070 con 0x5567d734d940 2024-09-19T06:52:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.183+0000 7ff964ff9640 1 -- 172.21.15.103:0/3568186600 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff96c051050 con 0x5567d734d940 2024-09-19T06:52:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.183+0000 7ff985d35740 10 client.16674 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.183+0000 7ff985d35740 1 -- 172.21.15.103:0/3568186600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567d735c380 con 0x5567d734d940 2024-09-19T06:52:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.183+0000 7ff964ff9640 1 -- 172.21.15.103:0/3568186600 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff978016070 con 0x5567d734d940 2024-09-19T06:52:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.183+0000 7ff964ff9640 1 -- 172.21.15.103:0/3568186600 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff96c03a040 con 0x5567d734d940 2024-09-19T06:52:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.183+0000 7ff985d35740 10 client.16674 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.183+0000 7ff985d35740 10 client.16674 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:52:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.183+0000 7ff985d35740 20 client.16674 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.183+0000 7ff985d35740 1 --2- 172.21.15.103:0/3568186600 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5567d735d790 0x5567d735fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.183+0000 7ff985d35740 4 client.16674 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.183+0000 7ff985d35740 1 -- 172.21.15.103:0/3568186600 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567d72b59d0 con 0x5567d735d790 2024-09-19T06:52:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.184+0000 7ff9816f8640 1 --2- 172.21.15.103:0/3568186600 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5567d735d790 0x5567d735fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.184+0000 7ff9816f8640 1 --2- 172.21.15.103:0/3568186600 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5567d735d790 0x5567d735fc00 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da58ac64fddcf4a7 server_cookie=85bc39b0f9f094e3 in_seq=0 out_seq=0 2024-09-19T06:52:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.184+0000 7ff964ff9640 10 client.16674 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.185+0000 7ff964ff9640 1 -- 172.21.15.103:0/3568186600 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5567d72b59d0 con 0x5567d735d790 2024-09-19T06:52:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.185+0000 7ff964ff9640 10 client.16674 handle_command_reply: tid=0 2024-09-19T06:52:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.185+0000 7ff985d35740 1 client.16674 shutdown 2024-09-19T06:52:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.185+0000 7ff985d35740 1 -- 172.21.15.103:0/3568186600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d734d940 msgr2=0x5567d734b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.185+0000 7ff985d35740 1 --2- 172.21.15.103:0/3568186600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d734d940 0x5567d734b2b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff96c033000 tx=0x7ff96c0317e0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.185+0000 7ff985d35740 1 -- 172.21.15.103:0/3568186600 shutdown_connections 2024-09-19T06:52:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.185+0000 7ff985d35740 1 --2- 172.21.15.103:0/3568186600 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5567d735d790 0x5567d735fc00 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.185+0000 7ff964ff9640 0 client.16674 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.185+0000 7ff985d35740 1 --2- 172.21.15.103:0/3568186600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d734d940 0x5567d734b2b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.185+0000 7ff985d35740 1 --2- 172.21.15.103:0/3568186600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d7343080 0x5567d734ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.185+0000 7ff985d35740 1 --2- 172.21.15.103:0/3568186600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d7342760 0x5567d734a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.185+0000 7ff985d35740 1 -- 172.21.15.103:0/3568186600 >> 172.21.15.103:0/3568186600 conn(0x5567d733ecf0 msgr2=0x5567d733de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.185+0000 7ff985d35740 1 -- 172.21.15.103:0/3568186600 shutdown_connections 2024-09-19T06:52:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.185+0000 7ff985d35740 1 -- 172.21.15.103:0/3568186600 wait complete. 2024-09-19T06:52:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.185+0000 7ff985d35740 20 client.16674 trim_cache size 0 max 16384 2024-09-19T06:52:43.192 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:42 smithi134 ceph-mon[35356]: pgmap v107: 97 pgs: 97 active+clean; 3.2 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-09-19T06:52:43.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff985d35740 1 Processor -- start 2024-09-19T06:52:43.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff985d35740 1 -- start start 2024-09-19T06:52:43.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff985d35740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d7342760 0x5567d7349b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff985d35740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d7343080 0x5567d734c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff985d35740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d734d940 0x5567d733dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff985d35740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c1ad980 con 0x5567d7343080 2024-09-19T06:52:43.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff985d35740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c1af4c0 con 0x5567d734d940 2024-09-19T06:52:43.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff985d35740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c1ae050 con 0x5567d7342760 2024-09-19T06:52:43.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff9816f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d7343080 0x5567d734c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:43.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff9816f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d7343080 0x5567d734c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43940/0 (socket says 172.21.15.103:43940) 2024-09-19T06:52:43.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff9816f8640 1 -- 172.21.15.103:0/2600431576 learned_addr learned my addr 172.21.15.103:0/2600431576 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:43.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff980ef7640 1 --2- 172.21.15.103:0/2600431576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d7342760 0x5567d7349b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:43.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff9816f8640 1 -- 172.21.15.103:0/2600431576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d7342760 msgr2=0x5567d7349b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:43.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff9816f8640 1 --2- 172.21.15.103:0/2600431576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d7342760 0x5567d7349b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff9816f8640 1 -- 172.21.15.103:0/2600431576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d734d940 msgr2=0x5567d733dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:43.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff9816f8640 1 --2- 172.21.15.103:0/2600431576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d734d940 0x5567d733dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff9816f8640 1 -- 172.21.15.103:0/2600431576 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567d733f980 con 0x5567d7343080 2024-09-19T06:52:43.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.202+0000 7ff980ef7640 1 --2- 172.21.15.103:0/2600431576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d7342760 0x5567d7349b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:43.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.203+0000 7ff9816f8640 1 --2- 172.21.15.103:0/2600431576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d7343080 0x5567d734c700 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x5567d733fb50 tx=0x7ff97800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa640dde740799c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:43.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.203+0000 7ff966ffd640 1 -- 172.21.15.103:0/2600431576 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978004220 con 0x5567d7343080 2024-09-19T06:52:43.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.203+0000 7ff966ffd640 1 -- 172.21.15.103:0/2600431576 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff978016040 con 0x5567d7343080 2024-09-19T06:52:43.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.203+0000 7ff966ffd640 1 -- 172.21.15.103:0/2600431576 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9780106b0 con 0x5567d7343080 2024-09-19T06:52:43.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.203+0000 7ff985d35740 1 -- 172.21.15.103:0/2600431576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d7343080 msgr2=0x5567d734c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:43.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.203+0000 7ff985d35740 1 --2- 172.21.15.103:0/2600431576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d7343080 0x5567d734c700 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x5567d733fb50 tx=0x7ff97800e720 comp rx=0 tx=0).stop 2024-09-19T06:52:43.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.203+0000 7ff985d35740 1 -- 172.21.15.103:0/2600431576 shutdown_connections 2024-09-19T06:52:43.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.203+0000 7ff985d35740 1 --2- 172.21.15.103:0/2600431576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d734d940 0x5567d733dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.203+0000 7ff985d35740 1 --2- 172.21.15.103:0/2600431576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d7343080 0x5567d734c700 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.203+0000 7ff985d35740 1 --2- 172.21.15.103:0/2600431576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d7342760 0x5567d7349b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.203+0000 7ff985d35740 1 -- 172.21.15.103:0/2600431576 >> 172.21.15.103:0/2600431576 conn(0x5567d733ecf0 msgr2=0x5567d733d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:43.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.204+0000 7ff985d35740 1 -- 172.21.15.103:0/2600431576 shutdown_connections 2024-09-19T06:52:43.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.204+0000 7ff985d35740 1 -- 172.21.15.103:0/2600431576 wait complete. 2024-09-19T06:52:43.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.204+0000 7ff985d35740 1 -- start start 2024-09-19T06:52:43.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.204+0000 7ff985d35740 1 Processor -- start 2024-09-19T06:52:43.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.204+0000 7ff985d35740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d7342760 0x5567d7351d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.204+0000 7ff985d35740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d734d940 0x5567d7352260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.204+0000 7ff985d35740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d73c98a0 0x5567d73cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.204+0000 7ff985d35740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c1aedf0 con 0x5567d734d940 2024-09-19T06:52:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.204+0000 7ff985d35740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c1ae720 con 0x5567d7342760 2024-09-19T06:52:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.204+0000 7ff985d35740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c1ab940 con 0x5567d73c98a0 2024-09-19T06:52:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.204+0000 7ff985d35740 10 client.0 _ll_register_callbacks cb 0x7ff96c032d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.204+0000 7ff9816f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d734d940 0x5567d7352260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff9816f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d734d940 0x5567d7352260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43952/0 (socket says 172.21.15.103:43952) 2024-09-19T06:52:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff9816f8640 1 -- 172.21.15.103:0/1808705617 learned_addr learned my addr 172.21.15.103:0/1808705617 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff980ef7640 1 --2- 172.21.15.103:0/1808705617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d7342760 0x5567d7351d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff980ef7640 1 --2- 172.21.15.103:0/1808705617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d73c98a0 0x5567d73cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff9816f8640 1 -- 172.21.15.103:0/1808705617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d73c98a0 msgr2=0x5567d73cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff9816f8640 1 --2- 172.21.15.103:0/1808705617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d73c98a0 0x5567d73cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff9816f8640 1 -- 172.21.15.103:0/1808705617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d7342760 msgr2=0x5567d7351d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff9816f8640 1 --2- 172.21.15.103:0/1808705617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d7342760 0x5567d7351d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff9816f8640 1 -- 172.21.15.103:0/1808705617 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567d73cc1d0 con 0x5567d734d940 2024-09-19T06:52:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff980ef7640 1 --2- 172.21.15.103:0/1808705617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d73c98a0 0x5567d73cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff9816f8640 1 --2- 172.21.15.103:0/1808705617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d734d940 0x5567d7352260 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff97800ec20 tx=0x7ff978003bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0b5ad9b6c7e883 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff964ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff964ff9640 1 -- 172.21.15.103:0/1808705617 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978038020 con 0x5567d734d940 2024-09-19T06:52:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff964ff9640 1 -- 172.21.15.103:0/1808705617 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff978004000 con 0x5567d734d940 2024-09-19T06:52:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff985d35740 1 -- 172.21.15.103:0/1808705617 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff978016040 con 0x5567d734d940 2024-09-19T06:52:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff964ff9640 1 -- 172.21.15.103:0/1808705617 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff978043050 con 0x5567d734d940 2024-09-19T06:52:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff985d35740 10 client.16680 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.205+0000 7ff985d35740 1 -- 172.21.15.103:0/1808705617 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567d73cd4b0 con 0x5567d734d940 2024-09-19T06:52:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.206+0000 7ff964ff9640 1 -- 172.21.15.103:0/1808705617 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff978016040 con 0x5567d734d940 2024-09-19T06:52:43.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.206+0000 7ff964ff9640 1 -- 172.21.15.103:0/1808705617 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff978010990 con 0x5567d734d940 2024-09-19T06:52:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.206+0000 7ff985d35740 10 client.16680 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.206+0000 7ff985d35740 10 client.16680 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:52:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.206+0000 7ff985d35740 20 client.16680 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.206+0000 7ff985d35740 1 --2- 172.21.15.103:0/1808705617 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5567d73cd9e0 0x5567d73cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.206+0000 7ff985d35740 4 client.16680 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-19T06:52:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.206+0000 7ff985d35740 1 -- 172.21.15.103:0/1808705617 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x5567d72afd20 con 0x5567d73cd9e0 2024-09-19T06:52:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.206+0000 7ff980ef7640 1 --2- 172.21.15.103:0/1808705617 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5567d73cd9e0 0x5567d73cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.206+0000 7ff964ff9640 10 client.16680 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.206+0000 7ff980ef7640 1 --2- 172.21.15.103:0/1808705617 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5567d73cd9e0 0x5567d73cfdd0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=998876f136525f57 server_cookie=a866c2f5c37ab564 in_seq=0 out_seq=0 2024-09-19T06:52:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.218+0000 7ff964ff9640 1 -- 172.21.15.103:0/1808705617 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5567d72afd20 con 0x5567d73cd9e0 2024-09-19T06:52:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.218+0000 7ff964ff9640 10 client.16680 handle_command_reply: tid=0 2024-09-19T06:52:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.218+0000 7ff985d35740 1 client.16680 shutdown 2024-09-19T06:52:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.218+0000 7ff985d35740 1 -- 172.21.15.103:0/1808705617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d734d940 msgr2=0x5567d7352260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.218+0000 7ff985d35740 1 --2- 172.21.15.103:0/1808705617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d734d940 0x5567d7352260 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff97800ec20 tx=0x7ff978003bf0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.218+0000 7ff985d35740 1 -- 172.21.15.103:0/1808705617 shutdown_connections 2024-09-19T06:52:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.218+0000 7ff985d35740 1 --2- 172.21.15.103:0/1808705617 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5567d73cd9e0 0x5567d73cfdd0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.218+0000 7ff985d35740 1 --2- 172.21.15.103:0/1808705617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567d73c98a0 0x5567d73cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.218+0000 7ff985d35740 1 --2- 172.21.15.103:0/1808705617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d734d940 0x5567d7352260 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.218+0000 7ff985d35740 1 --2- 172.21.15.103:0/1808705617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567d7342760 0x5567d7351d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.218+0000 7ff964ff9640 0 client.16680 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.218+0000 7ff985d35740 1 -- 172.21.15.103:0/1808705617 >> 172.21.15.103:0/1808705617 conn(0x5567d733ecf0 msgr2=0x5567d73449a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.218+0000 7ff985d35740 1 -- 172.21.15.103:0/1808705617 shutdown_connections 2024-09-19T06:52:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.218+0000 7ff985d35740 1 -- 172.21.15.103:0/1808705617 wait complete. 2024-09-19T06:52:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.218+0000 7ff985d35740 20 client.16680 trim_cache size 0 max 16384 2024-09-19T06:52:43.220 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:52:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:52:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:52:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 14, 2024-09-19T06:52:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:52:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "95fcbfa8": { 2024-09-19T06:52:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 14, 2024-09-19T06:52:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:52:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:52:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:52:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:52:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:52:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:52:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 136.6, 2024-09-19T06:52:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:52:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:52:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:52:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:52:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:52:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:52:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:52:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:52:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:52:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:52:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:52:43.222 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:52:43.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.219+0000 7ff98395c640 1 -- 172.21.15.103:0/1626135160 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9680a33b0 msgr2=0x7ff9680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:43.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.219+0000 7ff98395c640 1 --2- 172.21.15.103:0/1626135160 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9680a33b0 0x7ff9680a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff974098730 tx=0x7ff974076040 comp rx=0 tx=0).stop 2024-09-19T06:52:43.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.219+0000 7ff98395c640 1 -- 172.21.15.103:0/1626135160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106a80 msgr2=0x7ff97c19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:43.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.219+0000 7ff98395c640 1 --2- 172.21.15.103:0/1626135160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106a80 0x7ff97c19cd20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff97400eaa0 tx=0x7ff97400ef60 comp rx=0 tx=0).stop 2024-09-19T06:52:43.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.219+0000 7ff98395c640 1 -- 172.21.15.103:0/1626135160 shutdown_connections 2024-09-19T06:52:43.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.219+0000 7ff98395c640 1 --2- 172.21.15.103:0/1626135160 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9680a33b0 0x7ff9680a5870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.219+0000 7ff98395c640 1 --2- 172.21.15.103:0/1626135160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106a80 0x7ff97c19cd20 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.219+0000 7ff98395c640 1 --2- 172.21.15.103:0/1626135160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff97c101420 0x7ff97c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.219+0000 7ff98395c640 1 --2- 172.21.15.103:0/1626135160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff97c100a80 0x7ff97c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:43.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.219+0000 7ff98395c640 1 -- 172.21.15.103:0/1626135160 >> 172.21.15.103:0/1626135160 conn(0x7ff97c0fc7a0 msgr2=0x7ff97c104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:43.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.219+0000 7ff98395c640 1 -- 172.21.15.103:0/1626135160 shutdown_connections 2024-09-19T06:52:43.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:43.219+0000 7ff98395c640 1 -- 172.21.15.103:0/1626135160 wait complete. 2024-09-19T06:52:43.926 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '95fcbfa8', quiesce took 3.2 seconds. Will release after: 13.400000000000002 2024-09-19T06:52:44.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:44 smithi174 ceph-mon[35637]: pgmap v108: 97 pgs: 97 active+clean; 3.2 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-09-19T06:52:44.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:44 smithi103 ceph-mon[31158]: pgmap v108: 97 pgs: 97 active+clean; 3.2 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-09-19T06:52:44.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:44 smithi134 ceph-mon[35356]: pgmap v108: 97 pgs: 97 active+clean; 3.2 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-09-19T06:52:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:46 smithi174 ceph-mon[35637]: pgmap v109: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-09-19T06:52:46.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:46 smithi103 ceph-mon[31158]: pgmap v109: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-09-19T06:52:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:46 smithi134 ceph-mon[35356]: pgmap v109: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-09-19T06:52:48.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:48 smithi174 ceph-mon[35637]: pgmap v110: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-09-19T06:52:48.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:48 smithi103 ceph-mon[31158]: pgmap v110: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-09-19T06:52:48.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:48 smithi134 ceph-mon[35356]: pgmap v110: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-09-19T06:52:49.174 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:52:49.434 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:50.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.942+0000 7f7f97d7a640 1 -- 172.21.15.103:0/683728244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f90108290 msgr2=0x7f7f9010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.942+0000 7f7f97d7a640 1 --2- 172.21.15.103:0/683728244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f90108290 0x7f7f9010ec30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009930 tx=0x7f7f8802a420 comp rx=0 tx=0).stop 2024-09-19T06:52:50.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.942+0000 7f7f97d7a640 1 -- 172.21.15.103:0/683728244 shutdown_connections 2024-09-19T06:52:50.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.942+0000 7f7f97d7a640 1 --2- 172.21.15.103:0/683728244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f90108290 0x7f7f9010ec30 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.942+0000 7f7f97d7a640 1 --2- 172.21.15.103:0/683728244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f90103740 0x7f7f90107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.942+0000 7f7f97d7a640 1 --2- 172.21.15.103:0/683728244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f90102d90 0x7f7f90103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.942+0000 7f7f97d7a640 1 -- 172.21.15.103:0/683728244 >> 172.21.15.103:0/683728244 conn(0x7f7f900fc7d0 msgr2=0x7f7f900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:50.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.942+0000 7f7f97d7a640 1 -- 172.21.15.103:0/683728244 shutdown_connections 2024-09-19T06:52:50.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.942+0000 7f7f97d7a640 1 -- 172.21.15.103:0/683728244 wait complete. 2024-09-19T06:52:50.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.942+0000 7f7f97d7a640 1 Processor -- start 2024-09-19T06:52:50.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.942+0000 7f7f97d7a640 1 -- start start 2024-09-19T06:52:50.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f97d7a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f90102d90 0x7f7f901985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f97d7a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f90103740 0x7f7f90198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f97d7a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f90108290 0x7f7f9019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f97d7a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901120d0 con 0x7f7f90108290 2024-09-19T06:52:50.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f97d7a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f90111f50 con 0x7f7f90102d90 2024-09-19T06:52:50.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f97d7a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f90112250 con 0x7f7f90103740 2024-09-19T06:52:50.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f96317640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f90108290 0x7f7f9019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f96317640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f90108290 0x7f7f9019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38796/0 (socket says 172.21.15.103:38796) 2024-09-19T06:52:50.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f96317640 1 -- 172.21.15.103:0/1744244294 learned_addr learned my addr 172.21.15.103:0/1744244294 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:50.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f95315640 1 --2- 172.21.15.103:0/1744244294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f90103740 0x7f7f90198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f95b16640 1 --2- 172.21.15.103:0/1744244294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f90102d90 0x7f7f901985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f96317640 1 -- 172.21.15.103:0/1744244294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f90103740 msgr2=0x7f7f90198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f96317640 1 --2- 172.21.15.103:0/1744244294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f90103740 0x7f7f90198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f96317640 1 -- 172.21.15.103:0/1744244294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f90102d90 msgr2=0x7f7f901985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f96317640 1 --2- 172.21.15.103:0/1744244294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f90102d90 0x7f7f901985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f96317640 1 -- 172.21.15.103:0/1744244294 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f9019d560 con 0x7f7f90108290 2024-09-19T06:52:50.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f96317640 1 --2- 172.21.15.103:0/1744244294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f90108290 0x7f7f9019ce80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7f88005ab0 tx=0x7f7f880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:50.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f8effd640 1 -- 172.21.15.103:0/1744244294 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8803f030 con 0x7f7f90108290 2024-09-19T06:52:50.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f8effd640 1 -- 172.21.15.103:0/1744244294 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f88003ce0 con 0x7f7f90108290 2024-09-19T06:52:50.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.943+0000 7f7f97d7a640 1 -- 172.21.15.103:0/1744244294 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f9019d7f0 con 0x7f7f90108290 2024-09-19T06:52:50.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.944+0000 7f7f8effd640 1 -- 172.21.15.103:0/1744244294 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88035070 con 0x7f7f90108290 2024-09-19T06:52:50.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.944+0000 7f7f97d7a640 1 -- 172.21.15.103:0/1744244294 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f901a5130 con 0x7f7f90108290 2024-09-19T06:52:50.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.946+0000 7f7f9a153740 1 Processor -- start 2024-09-19T06:52:50.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.946+0000 7f7f8effd640 1 -- 172.21.15.103:0/1744244294 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7f88042030 con 0x7f7f90108290 2024-09-19T06:52:50.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f9a153740 1 -- start start 2024-09-19T06:52:50.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f9a153740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 0x563975708880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f9a153740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 0x563975713140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f9a153740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x563975715a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f9a153740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56397566e2e0 con 0x563975713680 2024-09-19T06:52:50.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f9a153740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563975615ce0 con 0x5639757084a0 2024-09-19T06:52:50.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f9a153740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56397566a2a0 con 0x563975708dc0 2024-09-19T06:52:50.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f95315640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x563975715a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f95315640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x563975715a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38808/0 (socket says 172.21.15.103:38808) 2024-09-19T06:52:50.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f95315640 1 -- 172.21.15.103:0/3753213607 learned_addr learned my addr 172.21.15.103:0/3753213607 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f95b16640 1 --2- 172.21.15.103:0/3753213607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 0x563975713140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f95315640 1 --2- 172.21.15.103:0/3753213607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 0x563975708880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f95315640 1 -- 172.21.15.103:0/3753213607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 msgr2=0x563975713140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f95315640 1 --2- 172.21.15.103:0/3753213607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 0x563975713140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f95315640 1 -- 172.21.15.103:0/3753213607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 msgr2=0x563975708880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f95315640 1 --2- 172.21.15.103:0/3753213607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 0x563975708880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f95315640 1 -- 172.21.15.103:0/3753213607 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639755fa440 con 0x563975713680 2024-09-19T06:52:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.947+0000 7f7f8effd640 1 --2- 172.21.15.103:0/1744244294 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7f780a33b0 0x7f7f780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.948+0000 7f7f95315640 1 --2- 172.21.15.103:0/3753213607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x563975715a70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009fa0 tx=0x7f7f84009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c27c1c5dbac895 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.948+0000 7f7f95b16640 1 --2- 172.21.15.103:0/1744244294 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7f780a33b0 0x7f7f780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.948+0000 7f7f8effd640 1 -- 172.21.15.103:0/1744244294 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f7f880f04c0 con 0x7f7f90108290 2024-09-19T06:52:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.948+0000 7f7f77fff640 1 -- 172.21.15.103:0/3753213607 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84017610 con 0x563975713680 2024-09-19T06:52:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.948+0000 7f7f77fff640 1 -- 172.21.15.103:0/3753213607 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f8401c070 con 0x563975713680 2024-09-19T06:52:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.948+0000 7f7f77fff640 1 -- 172.21.15.103:0/3753213607 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84015a40 con 0x563975713680 2024-09-19T06:52:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.949+0000 7f7f9a153740 1 -- 172.21.15.103:0/3753213607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 msgr2=0x563975715a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.949+0000 7f7f95b16640 1 --2- 172.21.15.103:0/1744244294 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7f780a33b0 0x7f7f780a5870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x563975705890 tx=0x7f7f80009680 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.949+0000 7f7f9a153740 1 --2- 172.21.15.103:0/3753213607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x563975715a70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009fa0 tx=0x7f7f84009560 comp rx=0 tx=0).stop 2024-09-19T06:52:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.949+0000 7f7f9a153740 1 -- 172.21.15.103:0/3753213607 shutdown_connections 2024-09-19T06:52:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.949+0000 7f7f9a153740 1 --2- 172.21.15.103:0/3753213607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x563975715a70 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.949+0000 7f7f9a153740 1 --2- 172.21.15.103:0/3753213607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 0x563975713140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.949+0000 7f7f9a153740 1 --2- 172.21.15.103:0/3753213607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 0x563975708880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.949+0000 7f7f9a153740 1 -- 172.21.15.103:0/3753213607 >> 172.21.15.103:0/3753213607 conn(0x563975704a30 msgr2=0x563975704e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.949+0000 7f7f9a153740 1 -- 172.21.15.103:0/3753213607 shutdown_connections 2024-09-19T06:52:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.949+0000 7f7f9a153740 1 -- 172.21.15.103:0/3753213607 wait complete. 2024-09-19T06:52:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.949+0000 7f7f9a153740 1 -- start start 2024-09-19T06:52:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.949+0000 7f7f9a153740 1 Processor -- start 2024-09-19T06:52:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f9a153740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639757084a0 0x563975713090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f9a153740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 0x563975710340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f9a153740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563975713680 0x563975710950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f9a153740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563975716250 con 0x5639757084a0 2024-09-19T06:52:50.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f9a153740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563975629050 con 0x563975713680 2024-09-19T06:52:50.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f9a153740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5639757163d0 con 0x563975708dc0 2024-09-19T06:52:50.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f9a153740 10 client.0 _ll_register_callbacks cb 0x563975669160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:50.034 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:52:50.034 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T06:52:50.035 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T06:52:50.035 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:52:50.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f95b16640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639757084a0 0x563975713090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f95b16640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639757084a0 0x563975713090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38818/0 (socket says 172.21.15.103:38818) 2024-09-19T06:52:50.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f95b16640 1 -- 172.21.15.103:0/1062533170 learned_addr learned my addr 172.21.15.103:0/1062533170 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:50.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f95315640 1 --2- 172.21.15.103:0/1062533170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 0x563975710340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f96317640 1 --2- 172.21.15.103:0/1062533170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563975713680 0x563975710950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f95b16640 1 -- 172.21.15.103:0/1062533170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 msgr2=0x563975710340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f95b16640 1 --2- 172.21.15.103:0/1062533170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 0x563975710340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f95b16640 1 -- 172.21.15.103:0/1062533170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563975713680 msgr2=0x563975710950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f95b16640 1 --2- 172.21.15.103:0/1062533170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563975713680 0x563975710950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f95b16640 1 -- 172.21.15.103:0/1062533170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639753df3c0 con 0x5639757084a0 2024-09-19T06:52:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f95b16640 1 --2- 172.21.15.103:0/1062533170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639757084a0 0x563975713090 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7f80013c20 tx=0x7f7f80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bf8c2cdc1bec07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f757fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.950+0000 7f7f757fa640 1 -- 172.21.15.103:0/1062533170 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8002b060 con 0x5639757084a0 2024-09-19T06:52:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.951+0000 7f7f757fa640 1 -- 172.21.15.103:0/1062533170 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f7f80013c50 con 0x5639757084a0 2024-09-19T06:52:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.951+0000 7f7f9a153740 1 -- 172.21.15.103:0/1062533170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f8401c070 con 0x5639757084a0 2024-09-19T06:52:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.951+0000 7f7f757fa640 1 -- 172.21.15.103:0/1062533170 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7f8001c460 con 0x5639757084a0 2024-09-19T06:52:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.951+0000 7f7f9a153740 10 client.16692 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.951+0000 7f7f9a153740 1 -- 172.21.15.103:0/1062533170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563975721d40 con 0x5639757084a0 2024-09-19T06:52:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.951+0000 7f7f757fa640 1 -- 172.21.15.103:0/1062533170 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f8401c070 con 0x5639757084a0 2024-09-19T06:52:50.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.951+0000 7f7f757fa640 1 -- 172.21.15.103:0/1062533170 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7f8001a040 con 0x5639757084a0 2024-09-19T06:52:50.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.951+0000 7f7f9a153740 10 client.16692 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:50.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.951+0000 7f7f9a153740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:50.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.951+0000 7f7f9a153740 20 client.16692 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:50.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.951+0000 7f7f9a153740 1 --2- 172.21.15.103:0/1062533170 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563975723200 0x563975725630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.951+0000 7f7f9a153740 4 client.16692 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:50.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.951+0000 7f7f9a153740 1 -- 172.21.15.103:0/1062533170 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56397567b850 con 0x563975723200 2024-09-19T06:52:50.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.952+0000 7f7f95315640 1 --2- 172.21.15.103:0/1062533170 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563975723200 0x563975725630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.952+0000 7f7f757fa640 10 client.16692 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:50.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.952+0000 7f7f95315640 1 --2- 172.21.15.103:0/1062533170 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563975723200 0x563975725630 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfb199108aae6e8 server_cookie=ab08dd8779ef1c4c in_seq=0 out_seq=0 2024-09-19T06:52:50.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.953+0000 7f7f757fa640 1 -- 172.21.15.103:0/1062533170 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56397567b850 con 0x563975723200 2024-09-19T06:52:50.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.953+0000 7f7f757fa640 10 client.16692 handle_command_reply: tid=0 2024-09-19T06:52:50.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.953+0000 7f7f9a153740 1 client.16692 shutdown 2024-09-19T06:52:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.953+0000 7f7f9a153740 1 -- 172.21.15.103:0/1062533170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639757084a0 msgr2=0x563975713090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.953+0000 7f7f9a153740 1 --2- 172.21.15.103:0/1062533170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639757084a0 0x563975713090 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7f80013c20 tx=0x7f7f80004a40 comp rx=0 tx=0).stop 2024-09-19T06:52:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.953+0000 7f7f9a153740 1 -- 172.21.15.103:0/1062533170 shutdown_connections 2024-09-19T06:52:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.953+0000 7f7f9a153740 1 --2- 172.21.15.103:0/1062533170 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563975723200 0x563975725630 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.953+0000 7f7f9a153740 1 --2- 172.21.15.103:0/1062533170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563975713680 0x563975710950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.953+0000 7f7f9a153740 1 --2- 172.21.15.103:0/1062533170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 0x563975710340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.953+0000 7f7f757fa640 0 client.16692 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.953+0000 7f7f9a153740 1 --2- 172.21.15.103:0/1062533170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639757084a0 0x563975713090 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.953+0000 7f7f9a153740 1 -- 172.21.15.103:0/1062533170 >> 172.21.15.103:0/1062533170 conn(0x563975704a30 msgr2=0x56397570a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.953+0000 7f7f9a153740 1 -- 172.21.15.103:0/1062533170 shutdown_connections 2024-09-19T06:52:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.953+0000 7f7f9a153740 1 -- 172.21.15.103:0/1062533170 wait complete. 2024-09-19T06:52:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.953+0000 7f7f9a153740 20 client.16692 trim_cache size 0 max 16384 2024-09-19T06:52:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.970+0000 7f7f9a153740 1 Processor -- start 2024-09-19T06:52:50.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.970+0000 7f7f9a153740 1 -- start start 2024-09-19T06:52:50.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f9a153740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 0x56397571f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f9a153740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 0x56397570ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f9a153740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x56397572af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f9a153740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901ad8e0 con 0x563975713680 2024-09-19T06:52:50.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f9a153740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901af420 con 0x5639757084a0 2024-09-19T06:52:50.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f9a153740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901adfb0 con 0x563975708dc0 2024-09-19T06:52:50.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f96317640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x56397572af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f96317640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x56397572af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38822/0 (socket says 172.21.15.103:38822) 2024-09-19T06:52:50.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f96317640 1 -- 172.21.15.103:0/2086329249 learned_addr learned my addr 172.21.15.103:0/2086329249 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:50.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f96317640 1 -- 172.21.15.103:0/2086329249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 msgr2=0x56397570ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:50.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f96317640 1 --2- 172.21.15.103:0/2086329249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 0x56397570ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f95b16640 1 --2- 172.21.15.103:0/2086329249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 0x56397571f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f96317640 1 -- 172.21.15.103:0/2086329249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 msgr2=0x56397571f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:50.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f96317640 1 --2- 172.21.15.103:0/2086329249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 0x56397571f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f96317640 1 -- 172.21.15.103:0/2086329249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56397570f2f0 con 0x563975713680 2024-09-19T06:52:50.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f95b16640 1 --2- 172.21.15.103:0/2086329249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 0x56397571f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:50.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f96317640 1 --2- 172.21.15.103:0/2086329249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x56397572af40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7f880b9180 tx=0x7f7f88031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733960f4ab34e29c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:50.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f777fe640 1 -- 172.21.15.103:0/2086329249 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88049050 con 0x563975713680 2024-09-19T06:52:50.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f777fe640 1 -- 172.21.15.103:0/2086329249 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f880bceb0 con 0x563975713680 2024-09-19T06:52:50.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.971+0000 7f7f777fe640 1 -- 172.21.15.103:0/2086329249 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f880edea0 con 0x563975713680 2024-09-19T06:52:50.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.972+0000 7f7f9a153740 1 -- 172.21.15.103:0/2086329249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 msgr2=0x56397572af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.972+0000 7f7f9a153740 1 --2- 172.21.15.103:0/2086329249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x56397572af40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7f880b9180 tx=0x7f7f88031b20 comp rx=0 tx=0).stop 2024-09-19T06:52:50.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.972+0000 7f7f9a153740 1 -- 172.21.15.103:0/2086329249 shutdown_connections 2024-09-19T06:52:50.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.972+0000 7f7f9a153740 1 --2- 172.21.15.103:0/2086329249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x56397572af40 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.972+0000 7f7f9a153740 1 --2- 172.21.15.103:0/2086329249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 0x56397570ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.972+0000 7f7f9a153740 1 --2- 172.21.15.103:0/2086329249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 0x56397571f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.972+0000 7f7f9a153740 1 -- 172.21.15.103:0/2086329249 >> 172.21.15.103:0/2086329249 conn(0x563975704a30 msgr2=0x56397570b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:50.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.972+0000 7f7f9a153740 1 -- 172.21.15.103:0/2086329249 shutdown_connections 2024-09-19T06:52:50.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.972+0000 7f7f9a153740 1 -- 172.21.15.103:0/2086329249 wait complete. 2024-09-19T06:52:50.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.972+0000 7f7f9a153740 1 -- start start 2024-09-19T06:52:50.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f9a153740 1 Processor -- start 2024-09-19T06:52:50.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f9a153740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 0x563975717560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f9a153740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 0x563975717aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f9a153740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x563975717fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f9a153740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901aed50 con 0x563975713680 2024-09-19T06:52:50.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f9a153740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901ae680 con 0x5639757084a0 2024-09-19T06:52:50.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f9a153740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901ab8a0 con 0x563975708dc0 2024-09-19T06:52:50.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f9a153740 10 client.0 _ll_register_callbacks cb 0x7f7f800138d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:50.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f96317640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x563975717fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f96317640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x563975717fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38834/0 (socket says 172.21.15.103:38834) 2024-09-19T06:52:50.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f96317640 1 -- 172.21.15.103:0/1754692130 learned_addr learned my addr 172.21.15.103:0/1754692130 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:50.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f95315640 1 --2- 172.21.15.103:0/1754692130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 0x563975717aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f95b16640 1 --2- 172.21.15.103:0/1754692130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 0x563975717560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f96317640 1 -- 172.21.15.103:0/1754692130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 msgr2=0x563975717aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:50.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f96317640 1 --2- 172.21.15.103:0/1754692130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 0x563975717aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f96317640 1 -- 172.21.15.103:0/1754692130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 msgr2=0x563975717560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:50.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f96317640 1 --2- 172.21.15.103:0/1754692130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 0x563975717560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f96317640 1 -- 172.21.15.103:0/1754692130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56397578d110 con 0x563975713680 2024-09-19T06:52:50.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f95b16640 1 --2- 172.21.15.103:0/1754692130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 0x563975717560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:50.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f96317640 1 --2- 172.21.15.103:0/1754692130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x563975717fe0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7f880bc740 tx=0x7f7f880bc2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46acfc9ef37c85b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:50.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.973+0000 7f7f757fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:50.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.974+0000 7f7f757fa640 1 -- 172.21.15.103:0/1754692130 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f880da020 con 0x563975713680 2024-09-19T06:52:50.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.974+0000 7f7f757fa640 1 -- 172.21.15.103:0/1754692130 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f7f880bca10 con 0x563975713680 2024-09-19T06:52:50.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.974+0000 7f7f9a153740 1 -- 172.21.15.103:0/1754692130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f880bceb0 con 0x563975713680 2024-09-19T06:52:50.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.974+0000 7f7f757fa640 1 -- 172.21.15.103:0/1754692130 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7f88049020 con 0x563975713680 2024-09-19T06:52:50.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.974+0000 7f7f9a153740 10 client.16698 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:50.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.974+0000 7f7f9a153740 1 -- 172.21.15.103:0/1754692130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56397578e350 con 0x563975713680 2024-09-19T06:52:50.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.974+0000 7f7f757fa640 1 -- 172.21.15.103:0/1754692130 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f880bceb0 con 0x563975713680 2024-09-19T06:52:50.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.974+0000 7f7f757fa640 1 -- 172.21.15.103:0/1754692130 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7f88031bd0 con 0x563975713680 2024-09-19T06:52:50.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.974+0000 7f7f9a153740 10 client.16698 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:50.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.974+0000 7f7f9a153740 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:50.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.974+0000 7f7f9a153740 20 client.16698 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:50.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.974+0000 7f7f9a153740 1 --2- 172.21.15.103:0/1754692130 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56397578e9c0 0x563975790de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.974+0000 7f7f9a153740 4 client.16698 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:52:50.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.974+0000 7f7f9a153740 1 -- 172.21.15.103:0/1754692130 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563975675d60 con 0x56397578e9c0 2024-09-19T06:52:50.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.974+0000 7f7f95b16640 1 --2- 172.21.15.103:0/1754692130 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56397578e9c0 0x563975790de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f757fa640 10 client.16698 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:50.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f95b16640 1 --2- 172.21.15.103:0/1754692130 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56397578e9c0 0x563975790de0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72e25770882d961b server_cookie=1f52190a908c3ff6 in_seq=0 out_seq=0 2024-09-19T06:52:50.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f757fa640 1 -- 172.21.15.103:0/1754692130 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563975675d60 con 0x56397578e9c0 2024-09-19T06:52:50.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f757fa640 10 client.16698 handle_command_reply: tid=0 2024-09-19T06:52:50.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f9a153740 1 client.16698 shutdown 2024-09-19T06:52:50.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f9a153740 1 -- 172.21.15.103:0/1754692130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 msgr2=0x563975717fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f9a153740 1 --2- 172.21.15.103:0/1754692130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x563975717fe0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7f880bc740 tx=0x7f7f880bc2f0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f9a153740 1 -- 172.21.15.103:0/1754692130 shutdown_connections 2024-09-19T06:52:50.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f9a153740 1 --2- 172.21.15.103:0/1754692130 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56397578e9c0 0x563975790de0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f9a153740 1 --2- 172.21.15.103:0/1754692130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563975713680 0x563975717fe0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f9a153740 1 --2- 172.21.15.103:0/1754692130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563975708dc0 0x563975717aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f9a153740 1 --2- 172.21.15.103:0/1754692130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639757084a0 0x563975717560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f757fa640 0 client.16698 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:50.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f9a153740 1 -- 172.21.15.103:0/1754692130 >> 172.21.15.103:0/1754692130 conn(0x563975704a30 msgr2=0x56397570b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:50.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f9a153740 1 -- 172.21.15.103:0/1754692130 shutdown_connections 2024-09-19T06:52:50.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f9a153740 1 -- 172.21.15.103:0/1754692130 wait complete. 2024-09-19T06:52:50.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.975+0000 7f7f9a153740 20 client.16698 trim_cache size 0 max 16384 2024-09-19T06:52:50.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.976+0000 7f7f97d7a640 1 -- 172.21.15.103:0/1744244294 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7f780a33b0 msgr2=0x7f7f780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.976+0000 7f7f97d7a640 1 --2- 172.21.15.103:0/1744244294 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7f780a33b0 0x7f7f780a5870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x563975705890 tx=0x7f7f80009680 comp rx=0 tx=0).stop 2024-09-19T06:52:50.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.976+0000 7f7f97d7a640 1 -- 172.21.15.103:0/1744244294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f90108290 msgr2=0x7f7f9019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.976+0000 7f7f97d7a640 1 --2- 172.21.15.103:0/1744244294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f90108290 0x7f7f9019ce80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7f88005ab0 tx=0x7f7f880042f0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.976+0000 7f7f97d7a640 1 -- 172.21.15.103:0/1744244294 shutdown_connections 2024-09-19T06:52:50.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.976+0000 7f7f97d7a640 1 --2- 172.21.15.103:0/1744244294 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7f780a33b0 0x7f7f780a5870 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.976+0000 7f7f97d7a640 1 --2- 172.21.15.103:0/1744244294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f90108290 0x7f7f9019ce80 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.976+0000 7f7f97d7a640 1 --2- 172.21.15.103:0/1744244294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f90103740 0x7f7f90198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.976+0000 7f7f97d7a640 1 --2- 172.21.15.103:0/1744244294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f90102d90 0x7f7f901985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.976+0000 7f7f97d7a640 1 -- 172.21.15.103:0/1744244294 >> 172.21.15.103:0/1744244294 conn(0x7f7f900fc7d0 msgr2=0x7f7f90105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:50.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.976+0000 7f7f97d7a640 1 -- 172.21.15.103:0/1744244294 shutdown_connections 2024-09-19T06:52:50.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:49.976+0000 7f7f97d7a640 1 -- 172.21.15.103:0/1744244294 wait complete. 2024-09-19T06:52:50.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:50 smithi103 ceph-mon[31158]: pgmap v111: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 19 op/s 2024-09-19T06:52:50.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:50 smithi174 ceph-mon[35637]: pgmap v111: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 19 op/s 2024-09-19T06:52:50.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:50 smithi134 ceph-mon[35356]: pgmap v111: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 19 op/s 2024-09-19T06:52:50.863 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:52:50.864 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:52:51.112 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:51.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.541+0000 7fb25da17640 1 -- 172.21.15.103:0/700126195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb25810c180 msgr2=0x7fb25810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:51.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.541+0000 7fb25da17640 1 --2- 172.21.15.103:0/700126195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb25810c180 0x7fb25810e600 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb248009ca0 tx=0x7fb24802a950 comp rx=0 tx=0).stop 2024-09-19T06:52:51.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.541+0000 7fb25da17640 1 -- 172.21.15.103:0/700126195 shutdown_connections 2024-09-19T06:52:51.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.541+0000 7fb25da17640 1 --2- 172.21.15.103:0/700126195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb25810c180 0x7fb25810e600 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:51.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.541+0000 7fb25da17640 1 --2- 172.21.15.103:0/700126195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2581036c0 0x7fb25810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:51.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.541+0000 7fb25da17640 1 --2- 172.21.15.103:0/700126195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb258102da0 0x7fb258103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:51.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.541+0000 7fb25da17640 1 -- 172.21.15.103:0/700126195 >> 172.21.15.103:0/700126195 conn(0x7fb2580fc7a0 msgr2=0x7fb2580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:51.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.541+0000 7fb25da17640 1 -- 172.21.15.103:0/700126195 shutdown_connections 2024-09-19T06:52:51.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.541+0000 7fb25da17640 1 -- 172.21.15.103:0/700126195 wait complete. 2024-09-19T06:52:51.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.541+0000 7fb25da17640 1 Processor -- start 2024-09-19T06:52:51.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.541+0000 7fb25da17640 1 -- start start 2024-09-19T06:52:51.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb25da17640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb258102da0 0x7fb258198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:51.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb25da17640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2581036c0 0x7fb2581989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:51.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb25da17640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb25810c180 0x7fb25819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:51.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb25da17640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb258111fc0 con 0x7fb258102da0 2024-09-19T06:52:51.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb25da17640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb258111e40 con 0x7fb25810c180 2024-09-19T06:52:51.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb25da17640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb258112140 con 0x7fb2581036c0 2024-09-19T06:52:51.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb256ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb258102da0 0x7fb258198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:51.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb256ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb258102da0 0x7fb258198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38852/0 (socket says 172.21.15.103:38852) 2024-09-19T06:52:51.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb256ffd640 1 -- 172.21.15.103:0/2332612838 learned_addr learned my addr 172.21.15.103:0/2332612838 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:51.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb2567fc640 1 --2- 172.21.15.103:0/2332612838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2581036c0 0x7fb2581989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:51.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb2577fe640 1 --2- 172.21.15.103:0/2332612838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb25810c180 0x7fb25819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:51.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb256ffd640 1 -- 172.21.15.103:0/2332612838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2581036c0 msgr2=0x7fb2581989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:51.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb256ffd640 1 --2- 172.21.15.103:0/2332612838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2581036c0 0x7fb2581989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:51.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb256ffd640 1 -- 172.21.15.103:0/2332612838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb25810c180 msgr2=0x7fb25819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:51.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb256ffd640 1 --2- 172.21.15.103:0/2332612838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb25810c180 0x7fb25819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:51.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb256ffd640 1 -- 172.21.15.103:0/2332612838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25819d410 con 0x7fb258102da0 2024-09-19T06:52:51.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb2567fc640 1 --2- 172.21.15.103:0/2332612838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2581036c0 0x7fb2581989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:51.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb256ffd640 1 --2- 172.21.15.103:0/2332612838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb258102da0 0x7fb258198460 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fb25000b4a0 tx=0x7fb25000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:51.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb25ca15640 1 -- 172.21.15.103:0/2332612838 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb250014070 con 0x7fb258102da0 2024-09-19T06:52:51.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb25ca15640 1 -- 172.21.15.103:0/2332612838 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb250010af0 con 0x7fb258102da0 2024-09-19T06:52:51.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb25da17640 1 -- 172.21.15.103:0/2332612838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25819d700 con 0x7fb258102da0 2024-09-19T06:52:51.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.542+0000 7fb25ca15640 1 -- 172.21.15.103:0/2332612838 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb250017d10 con 0x7fb258102da0 2024-09-19T06:52:51.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.543+0000 7fb25da17640 1 -- 172.21.15.103:0/2332612838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2581a4fe0 con 0x7fb258102da0 2024-09-19T06:52:51.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.543+0000 7fb25da17640 1 -- 172.21.15.103:0/2332612838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb258103180 con 0x7fb258102da0 2024-09-19T06:52:51.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.545+0000 7fb25ca15640 1 -- 172.21.15.103:0/2332612838 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fb250017070 con 0x7fb258102da0 2024-09-19T06:52:51.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.545+0000 7fb25ca15640 1 --2- 172.21.15.103:0/2332612838 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb2440a3430 0x7fb2440a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:51.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.545+0000 7fb2567fc640 1 --2- 172.21.15.103:0/2332612838 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb2440a3430 0x7fb2440a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:51.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.546+0000 7fb25ca15640 1 -- 172.21.15.103:0/2332612838 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fb2500cefa0 con 0x7fb258102da0 2024-09-19T06:52:51.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.546+0000 7fb2567fc640 1 --2- 172.21.15.103:0/2332612838 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb2440a3430 0x7fb2440a58f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb2400058c0 tx=0x7fb240005850 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:51.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.550+0000 7fb25ca15640 1 -- 172.21.15.103:0/2332612838 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb25009baa0 con 0x7fb258102da0 2024-09-19T06:52:51.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.752+0000 7fb25da17640 1 -- 172.21.15.103:0/2332612838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb25802d7c0 con 0x7fb258102da0 2024-09-19T06:52:51.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.753+0000 7fb25ca15640 1 -- 172.21.15.103:0/2332612838 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fb25009b3a0 con 0x7fb258102da0 2024-09-19T06:52:51.754 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:52:51.754 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:52:51.755 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:52:51.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.755+0000 7fb25da17640 1 -- 172.21.15.103:0/2332612838 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb2440a3430 msgr2=0x7fb2440a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:51.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.755+0000 7fb25da17640 1 --2- 172.21.15.103:0/2332612838 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb2440a3430 0x7fb2440a58f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb2400058c0 tx=0x7fb240005850 comp rx=0 tx=0).stop 2024-09-19T06:52:51.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.755+0000 7fb25da17640 1 -- 172.21.15.103:0/2332612838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb258102da0 msgr2=0x7fb258198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:51.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.755+0000 7fb25da17640 1 --2- 172.21.15.103:0/2332612838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb258102da0 0x7fb258198460 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fb25000b4a0 tx=0x7fb25000b960 comp rx=0 tx=0).stop 2024-09-19T06:52:51.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.755+0000 7fb25da17640 1 -- 172.21.15.103:0/2332612838 shutdown_connections 2024-09-19T06:52:51.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.755+0000 7fb25da17640 1 --2- 172.21.15.103:0/2332612838 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb2440a3430 0x7fb2440a58f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:51.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.755+0000 7fb25da17640 1 --2- 172.21.15.103:0/2332612838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb25810c180 0x7fb25819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:51.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.755+0000 7fb25da17640 1 --2- 172.21.15.103:0/2332612838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2581036c0 0x7fb2581989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:51.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.755+0000 7fb25da17640 1 --2- 172.21.15.103:0/2332612838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb258102da0 0x7fb258198460 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:51.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.755+0000 7fb25da17640 1 -- 172.21.15.103:0/2332612838 >> 172.21.15.103:0/2332612838 conn(0x7fb2580fc7a0 msgr2=0x7fb2580fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:51.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.755+0000 7fb25da17640 1 -- 172.21.15.103:0/2332612838 shutdown_connections 2024-09-19T06:52:51.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:51.755+0000 7fb25da17640 1 -- 172.21.15.103:0/2332612838 wait complete. 2024-09-19T06:52:52.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:51 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2332612838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:52.116 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:51 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2332612838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:52.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:51 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2332612838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:52.522 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:52:52.770 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:53.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:52 smithi174 ceph-mon[35637]: pgmap v112: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-19T06:52:53.121 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:52 smithi103 ceph-mon[31158]: pgmap v112: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-19T06:52:53.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:52 smithi134 ceph-mon[35356]: pgmap v112: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-19T06:52:53.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.201+0000 7fb3dd547640 1 -- 172.21.15.103:0/3114267635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3d8102780 msgr2=0x7fb3d81067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.201+0000 7fb3dd547640 1 --2- 172.21.15.103:0/3114267635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3d8102780 0x7fb3d81067d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009900 tx=0x7fb3c002a3f0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.201+0000 7fb3dd547640 1 -- 172.21.15.103:0/3114267635 shutdown_connections 2024-09-19T06:52:53.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.201+0000 7fb3dd547640 1 --2- 172.21.15.103:0/3114267635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3d8106f00 0x7fb3d810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.201+0000 7fb3dd547640 1 --2- 172.21.15.103:0/3114267635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3d8102780 0x7fb3d81067d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.201+0000 7fb3dd547640 1 --2- 172.21.15.103:0/3114267635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3d8101dd0 0x7fb3d81021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.201+0000 7fb3dd547640 1 -- 172.21.15.103:0/3114267635 >> 172.21.15.103:0/3114267635 conn(0x7fb3d80fd8b0 msgr2=0x7fb3d80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:53.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.201+0000 7fb3dd547640 1 -- 172.21.15.103:0/3114267635 shutdown_connections 2024-09-19T06:52:53.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.201+0000 7fb3dd547640 1 -- 172.21.15.103:0/3114267635 wait complete. 2024-09-19T06:52:53.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.201+0000 7fb3dd547640 1 Processor -- start 2024-09-19T06:52:53.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3dd547640 1 -- start start 2024-09-19T06:52:53.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3dd547640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3d8101dd0 0x7fb3d81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3dd547640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3d8102780 0x7fb3d81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3dd547640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3d8106f00 0x7fb3d819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3dd547640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8111f60 con 0x7fb3d8101dd0 2024-09-19T06:52:53.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3dd547640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8111de0 con 0x7fb3d8106f00 2024-09-19T06:52:53.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3dd547640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81120e0 con 0x7fb3d8102780 2024-09-19T06:52:53.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3d8101dd0 0x7fb3d81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3d8101dd0 0x7fb3d81983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38878/0 (socket says 172.21.15.103:38878) 2024-09-19T06:52:53.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3d6ffd640 1 -- 172.21.15.103:0/359198060 learned_addr learned my addr 172.21.15.103:0/359198060 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:53.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3d67fc640 1 --2- 172.21.15.103:0/359198060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3d8102780 0x7fb3d81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3d77fe640 1 --2- 172.21.15.103:0/359198060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3d8106f00 0x7fb3d819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3d6ffd640 1 -- 172.21.15.103:0/359198060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3d8102780 msgr2=0x7fb3d81988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3d6ffd640 1 --2- 172.21.15.103:0/359198060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3d8102780 0x7fb3d81988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3d6ffd640 1 -- 172.21.15.103:0/359198060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3d8106f00 msgr2=0x7fb3d819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3d6ffd640 1 --2- 172.21.15.103:0/359198060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3d8106f00 0x7fb3d819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3d6ffd640 1 -- 172.21.15.103:0/359198060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d819d350 con 0x7fb3d8101dd0 2024-09-19T06:52:53.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.202+0000 7fb3d6ffd640 1 --2- 172.21.15.103:0/359198060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3d8101dd0 0x7fb3d81983a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800ea80 tx=0x7fb3c800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:53.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.203+0000 7fb3cffff640 1 -- 172.21.15.103:0/359198060 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c800ce40 con 0x7fb3d8101dd0 2024-09-19T06:52:53.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.203+0000 7fb3cffff640 1 -- 172.21.15.103:0/359198060 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3c8005020 con 0x7fb3d8101dd0 2024-09-19T06:52:53.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.203+0000 7fb3dd547640 1 -- 172.21.15.103:0/359198060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d819d640 con 0x7fb3d8101dd0 2024-09-19T06:52:53.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.203+0000 7fb3cffff640 1 -- 172.21.15.103:0/359198060 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8011760 con 0x7fb3d8101dd0 2024-09-19T06:52:53.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.203+0000 7fb3dd547640 1 -- 172.21.15.103:0/359198060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d81a4f20 con 0x7fb3d8101dd0 2024-09-19T06:52:53.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.205+0000 7fb3cffff640 1 -- 172.21.15.103:0/359198060 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fb3c80051d0 con 0x7fb3d8101dd0 2024-09-19T06:52:53.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.205+0000 7fb3df920740 1 Processor -- start 2024-09-19T06:52:53.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.205+0000 7fb3df920740 1 -- start start 2024-09-19T06:52:53.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.205+0000 7fb3df920740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e757b40 0x558e7e756090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.205+0000 7fb3df920740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e7565d0 0x558e7e756a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.205+0000 7fb3df920740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e765780 0x558e7e767b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.205+0000 7fb3df920740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e7e6bc2f0 con 0x558e7e757b40 2024-09-19T06:52:53.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.205+0000 7fb3df920740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558e7e663cf0 con 0x558e7e765780 2024-09-19T06:52:53.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.205+0000 7fb3df920740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e7e6b82b0 con 0x558e7e7565d0 2024-09-19T06:52:53.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3cffff640 1 --2- 172.21.15.103:0/359198060 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb3c40a33b0 0x7fb3c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3d77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e757b40 0x558e7e756090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3d77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e757b40 0x558e7e756090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38894/0 (socket says 172.21.15.103:38894) 2024-09-19T06:52:53.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3d77fe640 1 -- 172.21.15.103:0/1621815924 learned_addr learned my addr 172.21.15.103:0/1621815924 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:53.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3d77fe640 1 --2- 172.21.15.103:0/359198060 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb3c40a33b0 0x7fb3c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3d67fc640 1 --2- 172.21.15.103:0/1621815924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e765780 0x558e7e767b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3d6ffd640 1 --2- 172.21.15.103:0/1621815924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e7565d0 0x558e7e756a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3cffff640 1 -- 172.21.15.103:0/359198060 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fb3c80d0330 con 0x7fb3d8101dd0 2024-09-19T06:52:53.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3d77fe640 1 -- 172.21.15.103:0/1621815924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e7565d0 msgr2=0x558e7e756a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:53.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3d77fe640 1 --2- 172.21.15.103:0/1621815924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e7565d0 0x558e7e756a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3d77fe640 1 -- 172.21.15.103:0/1621815924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e765780 msgr2=0x558e7e767b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:53.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3d77fe640 1 --2- 172.21.15.103:0/1621815924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e765780 0x558e7e767b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3d77fe640 1 -- 172.21.15.103:0/1621815924 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e7e6483a0 con 0x558e7e757b40 2024-09-19T06:52:53.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3d6ffd640 1 --2- 172.21.15.103:0/1621815924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e7565d0 0x558e7e756a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:53.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3d77fe640 1 --2- 172.21.15.103:0/1621815924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e757b40 0x558e7e756090 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00129d0 tx=0x7fb3d0012e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c8f37920ada3f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:53.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3d77fe640 1 --2- 172.21.15.103:0/359198060 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb3c40a33b0 0x7fb3c40a5870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000d720 tx=0x7fb3d0014000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:53.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3cd7fa640 1 -- 172.21.15.103:0/1621815924 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0020070 con 0x558e7e757b40 2024-09-19T06:52:53.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3cd7fa640 1 -- 172.21.15.103:0/1621815924 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3d000f7c0 con 0x558e7e757b40 2024-09-19T06:52:53.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.206+0000 7fb3cd7fa640 1 -- 172.21.15.103:0/1621815924 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d001ccf0 con 0x558e7e757b40 2024-09-19T06:52:53.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.207+0000 7fb3df920740 1 -- 172.21.15.103:0/1621815924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e757b40 msgr2=0x558e7e756090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.207+0000 7fb3df920740 1 --2- 172.21.15.103:0/1621815924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e757b40 0x558e7e756090 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00129d0 tx=0x7fb3d0012e90 comp rx=0 tx=0).stop 2024-09-19T06:52:53.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.207+0000 7fb3df920740 1 -- 172.21.15.103:0/1621815924 shutdown_connections 2024-09-19T06:52:53.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.207+0000 7fb3df920740 1 --2- 172.21.15.103:0/1621815924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e765780 0x558e7e767b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.207+0000 7fb3df920740 1 --2- 172.21.15.103:0/1621815924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e7565d0 0x558e7e756a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.207+0000 7fb3df920740 1 --2- 172.21.15.103:0/1621815924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e757b40 0x558e7e756090 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.207+0000 7fb3df920740 1 -- 172.21.15.103:0/1621815924 >> 172.21.15.103:0/1621815924 conn(0x558e7e71f310 msgr2=0x558e7e752c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:53.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.207+0000 7fb3df920740 1 -- 172.21.15.103:0/1621815924 shutdown_connections 2024-09-19T06:52:53.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.207+0000 7fb3df920740 1 -- 172.21.15.103:0/1621815924 wait complete. 2024-09-19T06:52:53.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3df920740 1 -- start start 2024-09-19T06:52:53.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3df920740 1 Processor -- start 2024-09-19T06:52:53.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3df920740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e7565d0 0x558e7e762500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3df920740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e757b40 0x558e7e762a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3df920740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e765780 0x558e7e762f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3df920740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e7e768420 con 0x558e7e757b40 2024-09-19T06:52:53.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3df920740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558e7e677060 con 0x558e7e7565d0 2024-09-19T06:52:53.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3df920740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e7e7685a0 con 0x558e7e765780 2024-09-19T06:52:53.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3df920740 10 client.0 _ll_register_callbacks cb 0x558e7e3ca8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:53.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3d67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e757b40 0x558e7e762a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3d67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e757b40 0x558e7e762a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38906/0 (socket says 172.21.15.103:38906) 2024-09-19T06:52:53.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3d67fc640 1 -- 172.21.15.103:0/989937755 learned_addr learned my addr 172.21.15.103:0/989937755 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:53.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3d6ffd640 1 --2- 172.21.15.103:0/989937755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e7565d0 0x558e7e762500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3d67fc640 1 -- 172.21.15.103:0/989937755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e765780 msgr2=0x558e7e762f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:53.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3d77fe640 1 --2- 172.21.15.103:0/989937755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e765780 0x558e7e762f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3d67fc640 1 --2- 172.21.15.103:0/989937755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e765780 0x558e7e762f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3d67fc640 1 -- 172.21.15.103:0/989937755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e7565d0 msgr2=0x558e7e762500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:53.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3d67fc640 1 --2- 172.21.15.103:0/989937755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e7565d0 0x558e7e762500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.208+0000 7fb3d67fc640 1 -- 172.21.15.103:0/989937755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e7e42d3c0 con 0x558e7e757b40 2024-09-19T06:52:53.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.209+0000 7fb3d67fc640 1 --2- 172.21.15.103:0/989937755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e757b40 0x558e7e762a40 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009900 tx=0x7fb3c0031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e641de6543dbb684 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:53.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.209+0000 7fb3be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:53.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.209+0000 7fb3be7fc640 1 -- 172.21.15.103:0/989937755 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0035040 con 0x558e7e757b40 2024-09-19T06:52:53.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.209+0000 7fb3df920740 1 -- 172.21.15.103:0/989937755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3d000f7c0 con 0x558e7e757b40 2024-09-19T06:52:53.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.209+0000 7fb3be7fc640 1 -- 172.21.15.103:0/989937755 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fb3c0004240 con 0x558e7e757b40 2024-09-19T06:52:53.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.209+0000 7fb3be7fc640 1 -- 172.21.15.103:0/989937755 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb3c0004440 con 0x558e7e757b40 2024-09-19T06:52:53.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.209+0000 7fb3df920740 10 client.16716 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:53.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.209+0000 7fb3df920740 1 -- 172.21.15.103:0/989937755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e7e773fb0 con 0x558e7e757b40 2024-09-19T06:52:53.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.209+0000 7fb3be7fc640 1 -- 172.21.15.103:0/989937755 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3d000f7c0 con 0x558e7e757b40 2024-09-19T06:52:53.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.209+0000 7fb3be7fc640 1 -- 172.21.15.103:0/989937755 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb3c00035f0 con 0x558e7e757b40 2024-09-19T06:52:53.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.209+0000 7fb3df920740 10 client.16716 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:53.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.209+0000 7fb3df920740 10 client.16716 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:52:53.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.209+0000 7fb3df920740 20 client.16716 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:53.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.209+0000 7fb3df920740 1 --2- 172.21.15.103:0/989937755 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x558e7e7753c0 0x558e7e7777f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.209+0000 7fb3df920740 4 client.16716 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:53.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.209+0000 7fb3df920740 1 -- 172.21.15.103:0/989937755 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e7e6c9860 con 0x558e7e7753c0 2024-09-19T06:52:53.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.210+0000 7fb3d6ffd640 1 --2- 172.21.15.103:0/989937755 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x558e7e7753c0 0x558e7e7777f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.210+0000 7fb3be7fc640 10 client.16716 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:52:53.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.210+0000 7fb3d6ffd640 1 --2- 172.21.15.103:0/989937755 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x558e7e7753c0 0x558e7e7777f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c0d8f985c66ac764 server_cookie=eb487e7f1bb35130 in_seq=0 out_seq=0 2024-09-19T06:52:53.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.211+0000 7fb3be7fc640 1 -- 172.21.15.103:0/989937755 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e7e6c9860 con 0x558e7e7753c0 2024-09-19T06:52:53.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.211+0000 7fb3be7fc640 10 client.16716 handle_command_reply: tid=0 2024-09-19T06:52:53.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.211+0000 7fb3df920740 1 client.16716 shutdown 2024-09-19T06:52:53.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.211+0000 7fb3df920740 1 -- 172.21.15.103:0/989937755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e757b40 msgr2=0x558e7e762a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.211+0000 7fb3df920740 1 --2- 172.21.15.103:0/989937755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e757b40 0x558e7e762a40 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009900 tx=0x7fb3c0031920 comp rx=0 tx=0).stop 2024-09-19T06:52:53.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.211+0000 7fb3df920740 1 -- 172.21.15.103:0/989937755 shutdown_connections 2024-09-19T06:52:53.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.211+0000 7fb3df920740 1 --2- 172.21.15.103:0/989937755 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x558e7e7753c0 0x558e7e7777f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.211+0000 7fb3df920740 1 --2- 172.21.15.103:0/989937755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e765780 0x558e7e762f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.211+0000 7fb3df920740 1 --2- 172.21.15.103:0/989937755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e757b40 0x558e7e762a40 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.211+0000 7fb3be7fc640 0 client.16716 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:52:53.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.211+0000 7fb3df920740 1 --2- 172.21.15.103:0/989937755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e7565d0 0x558e7e762500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.211+0000 7fb3df920740 1 -- 172.21.15.103:0/989937755 >> 172.21.15.103:0/989937755 conn(0x558e7e71f310 msgr2=0x558e7e758960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:53.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.211+0000 7fb3df920740 1 -- 172.21.15.103:0/989937755 shutdown_connections 2024-09-19T06:52:53.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.211+0000 7fb3df920740 1 -- 172.21.15.103:0/989937755 wait complete. 2024-09-19T06:52:53.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.211+0000 7fb3df920740 20 client.16716 trim_cache size 0 max 16384 2024-09-19T06:52:53.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.224+0000 7fb3df920740 1 Processor -- start 2024-09-19T06:52:53.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.224+0000 7fb3df920740 1 -- start start 2024-09-19T06:52:53.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.224+0000 7fb3df920740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e7565d0 0x558e7e752d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.224+0000 7fb3df920740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e757b40 0x558e7e77d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.224+0000 7fb3df920740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e765780 0x558e7e77d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.224+0000 7fb3df920740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81ab500 con 0x558e7e765780 2024-09-19T06:52:53.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.224+0000 7fb3df920740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81ad040 con 0x558e7e757b40 2024-09-19T06:52:53.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.224+0000 7fb3df920740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81abbd0 con 0x558e7e7565d0 2024-09-19T06:52:53.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.224+0000 7fb3d77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e765780 0x558e7e77d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.224+0000 7fb3d77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e765780 0x558e7e77d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38922/0 (socket says 172.21.15.103:38922) 2024-09-19T06:52:53.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.224+0000 7fb3d77fe640 1 -- 172.21.15.103:0/1736438581 learned_addr learned my addr 172.21.15.103:0/1736438581 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:53.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.225+0000 7fb3d67fc640 1 --2- 172.21.15.103:0/1736438581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e757b40 0x558e7e77d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.225+0000 7fb3d77fe640 1 -- 172.21.15.103:0/1736438581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e7565d0 msgr2=0x558e7e752d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:53.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.225+0000 7fb3d6ffd640 1 --2- 172.21.15.103:0/1736438581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e7565d0 0x558e7e752d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.225+0000 7fb3d77fe640 1 --2- 172.21.15.103:0/1736438581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e7565d0 0x558e7e752d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.225+0000 7fb3d77fe640 1 -- 172.21.15.103:0/1736438581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e757b40 msgr2=0x558e7e77d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:53.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.225+0000 7fb3d77fe640 1 --2- 172.21.15.103:0/1736438581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e757b40 0x558e7e77d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.225+0000 7fb3d77fe640 1 -- 172.21.15.103:0/1736438581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e7e752510 con 0x558e7e765780 2024-09-19T06:52:53.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.225+0000 7fb3d67fc640 1 --2- 172.21.15.103:0/1736438581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e757b40 0x558e7e77d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:53.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.225+0000 7fb3d77fe640 1 --2- 172.21.15.103:0/1736438581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e765780 0x558e7e77d600 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0005920 tx=0x7fb3d000dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f1cdd1483a2adf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:53.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.225+0000 7fb3ccff9640 1 -- 172.21.15.103:0/1736438581 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0020040 con 0x558e7e765780 2024-09-19T06:52:53.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.225+0000 7fb3ccff9640 1 -- 172.21.15.103:0/1736438581 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3d000adb0 con 0x558e7e765780 2024-09-19T06:52:53.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.225+0000 7fb3ccff9640 1 -- 172.21.15.103:0/1736438581 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d00048a0 con 0x558e7e765780 2024-09-19T06:52:53.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.226+0000 7fb3df920740 1 -- 172.21.15.103:0/1736438581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e765780 msgr2=0x558e7e77d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.226+0000 7fb3df920740 1 --2- 172.21.15.103:0/1736438581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e765780 0x558e7e77d600 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0005920 tx=0x7fb3d000dfa0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.226+0000 7fb3df920740 1 -- 172.21.15.103:0/1736438581 shutdown_connections 2024-09-19T06:52:53.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.226+0000 7fb3df920740 1 --2- 172.21.15.103:0/1736438581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e765780 0x558e7e77d600 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.226+0000 7fb3df920740 1 --2- 172.21.15.103:0/1736438581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e757b40 0x558e7e77d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.226+0000 7fb3df920740 1 --2- 172.21.15.103:0/1736438581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e7565d0 0x558e7e752d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.226+0000 7fb3df920740 1 -- 172.21.15.103:0/1736438581 >> 172.21.15.103:0/1736438581 conn(0x558e7e71f310 msgr2=0x558e7e761650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:53.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.226+0000 7fb3df920740 1 -- 172.21.15.103:0/1736438581 shutdown_connections 2024-09-19T06:52:53.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.226+0000 7fb3df920740 1 -- 172.21.15.103:0/1736438581 wait complete. 2024-09-19T06:52:53.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.226+0000 7fb3df920740 1 -- start start 2024-09-19T06:52:53.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.226+0000 7fb3df920740 1 Processor -- start 2024-09-19T06:52:53.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3df920740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e7565d0 0x558e7e76dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3df920740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e757b40 0x558e7e76e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3df920740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e765780 0x558e7e75f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3df920740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81ac970 con 0x558e7e7565d0 2024-09-19T06:52:53.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3df920740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81ac2a0 con 0x558e7e765780 2024-09-19T06:52:53.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3df920740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81a94e0 con 0x558e7e757b40 2024-09-19T06:52:53.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3df920740 10 client.0 _ll_register_callbacks cb 0x7fb3c00315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:53.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e7565d0 0x558e7e76dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e7565d0 0x558e7e76dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38936/0 (socket says 172.21.15.103:38936) 2024-09-19T06:52:53.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3d6ffd640 1 -- 172.21.15.103:0/4106714767 learned_addr learned my addr 172.21.15.103:0/4106714767 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:53.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3d67fc640 1 --2- 172.21.15.103:0/4106714767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e757b40 0x558e7e76e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3d77fe640 1 --2- 172.21.15.103:0/4106714767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e765780 0x558e7e75f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3d6ffd640 1 -- 172.21.15.103:0/4106714767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e757b40 msgr2=0x558e7e76e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:53.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3d6ffd640 1 --2- 172.21.15.103:0/4106714767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e757b40 0x558e7e76e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3d6ffd640 1 -- 172.21.15.103:0/4106714767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e765780 msgr2=0x558e7e75f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:53.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3d6ffd640 1 --2- 172.21.15.103:0/4106714767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e765780 0x558e7e75f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3d6ffd640 1 -- 172.21.15.103:0/4106714767 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e7e759120 con 0x558e7e7565d0 2024-09-19T06:52:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3d67fc640 1 --2- 172.21.15.103:0/4106714767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e757b40 0x558e7e76e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3d6ffd640 1 --2- 172.21.15.103:0/4106714767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e7565d0 0x558e7e76dba0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800c700 tx=0x7fb3c801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac85ebd4667beff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3be7fc640 1 -- 172.21.15.103:0/4106714767 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c809c620 con 0x558e7e7565d0 2024-09-19T06:52:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3be7fc640 1 -- 172.21.15.103:0/4106714767 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fb3c809a920 con 0x558e7e7565d0 2024-09-19T06:52:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3df920740 1 -- 172.21.15.103:0/4106714767 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3d000adb0 con 0x558e7e7565d0 2024-09-19T06:52:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3be7fc640 1 -- 172.21.15.103:0/4106714767 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb3c80a09d0 con 0x558e7e7565d0 2024-09-19T06:52:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3df920740 10 client.16722 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.227+0000 7fb3df920740 1 -- 172.21.15.103:0/4106714767 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e7e7dfac0 con 0x558e7e7565d0 2024-09-19T06:52:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.228+0000 7fb3be7fc640 1 -- 172.21.15.103:0/4106714767 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3d000adb0 con 0x558e7e7565d0 2024-09-19T06:52:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.228+0000 7fb3be7fc640 1 -- 172.21.15.103:0/4106714767 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb3c800f040 con 0x558e7e7565d0 2024-09-19T06:52:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.228+0000 7fb3df920740 10 client.16722 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.228+0000 7fb3df920740 10 client.16722 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:52:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.228+0000 7fb3df920740 20 client.16722 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.228+0000 7fb3df920740 1 --2- 172.21.15.103:0/4106714767 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x558e7e7753c0 0x558e7e7e2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.228+0000 7fb3df920740 4 client.16722 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:52:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.228+0000 7fb3df920740 1 -- 172.21.15.103:0/4106714767 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e7e6c3d70 con 0x558e7e7753c0 2024-09-19T06:52:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.228+0000 7fb3d67fc640 1 --2- 172.21.15.103:0/4106714767 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x558e7e7753c0 0x558e7e7e2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.228+0000 7fb3d67fc640 1 --2- 172.21.15.103:0/4106714767 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x558e7e7753c0 0x558e7e7e2180 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8b62662e24bd6dd0 server_cookie=77f3c28ee9072d9f in_seq=0 out_seq=0 2024-09-19T06:52:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.228+0000 7fb3be7fc640 10 client.16722 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:52:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3be7fc640 1 -- 172.21.15.103:0/4106714767 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e7e6c3d70 con 0x558e7e7753c0 2024-09-19T06:52:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3be7fc640 10 client.16722 handle_command_reply: tid=0 2024-09-19T06:52:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3df920740 1 client.16722 shutdown 2024-09-19T06:52:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3df920740 1 -- 172.21.15.103:0/4106714767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e7565d0 msgr2=0x558e7e76dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3df920740 1 --2- 172.21.15.103:0/4106714767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e7565d0 0x558e7e76dba0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800c700 tx=0x7fb3c801d000 comp rx=0 tx=0).stop 2024-09-19T06:52:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3df920740 1 -- 172.21.15.103:0/4106714767 shutdown_connections 2024-09-19T06:52:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3df920740 1 --2- 172.21.15.103:0/4106714767 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x558e7e7753c0 0x558e7e7e2180 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3df920740 1 --2- 172.21.15.103:0/4106714767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e7e765780 0x558e7e75f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3df920740 1 --2- 172.21.15.103:0/4106714767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e7e757b40 0x558e7e76e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3df920740 1 --2- 172.21.15.103:0/4106714767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e7e7565d0 0x558e7e76dba0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3df920740 1 -- 172.21.15.103:0/4106714767 >> 172.21.15.103:0/4106714767 conn(0x558e7e71f310 msgr2=0x558e7e762470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3be7fc640 0 client.16722 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:52:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3df920740 1 -- 172.21.15.103:0/4106714767 shutdown_connections 2024-09-19T06:52:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3df920740 1 -- 172.21.15.103:0/4106714767 wait complete. 2024-09-19T06:52:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3df920740 20 client.16722 trim_cache size 0 max 16384 2024-09-19T06:52:53.244 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:52:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3dd547640 1 -- 172.21.15.103:0/359198060 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb3c40a33b0 msgr2=0x7fb3c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3dd547640 1 --2- 172.21.15.103:0/359198060 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb3c40a33b0 0x7fb3c40a5870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000d720 tx=0x7fb3d0014000 comp rx=0 tx=0).stop 2024-09-19T06:52:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3dd547640 1 -- 172.21.15.103:0/359198060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3d8101dd0 msgr2=0x7fb3d81983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.229+0000 7fb3dd547640 1 --2- 172.21.15.103:0/359198060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3d8101dd0 0x7fb3d81983a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800ea80 tx=0x7fb3c800ef40 comp rx=0 tx=0).stop 2024-09-19T06:52:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.230+0000 7fb3dd547640 1 -- 172.21.15.103:0/359198060 shutdown_connections 2024-09-19T06:52:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.230+0000 7fb3dd547640 1 --2- 172.21.15.103:0/359198060 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb3c40a33b0 0x7fb3c40a5870 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.230+0000 7fb3dd547640 1 --2- 172.21.15.103:0/359198060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3d8106f00 0x7fb3d819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.230+0000 7fb3dd547640 1 --2- 172.21.15.103:0/359198060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3d8102780 0x7fb3d81988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.230+0000 7fb3dd547640 1 --2- 172.21.15.103:0/359198060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3d8101dd0 0x7fb3d81983a0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.230+0000 7fb3dd547640 1 -- 172.21.15.103:0/359198060 >> 172.21.15.103:0/359198060 conn(0x7fb3d80fd8b0 msgr2=0x7fb3d80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:53.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.230+0000 7fb3dd547640 1 -- 172.21.15.103:0/359198060 shutdown_connections 2024-09-19T06:52:53.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:53.230+0000 7fb3dd547640 1 -- 172.21.15.103:0/359198060 wait complete. 2024-09-19T06:52:53.925 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:52:54.178 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:54.295 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:54 smithi103 ceph-mon[31158]: pgmap v113: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-19T06:52:54.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:54 smithi174 ceph-mon[35637]: pgmap v113: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-19T06:52:54.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:54 smithi134 ceph-mon[35356]: pgmap v113: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-19T06:52:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.595+0000 7f301455e640 1 -- 172.21.15.103:0/439107420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f300c100e20 msgr2=0x7f300c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.595+0000 7f301455e640 1 --2- 172.21.15.103:0/439107420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f300c100e20 0x7f300c101200 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3000009900 tx=0x7f300002a480 comp rx=0 tx=0).stop 2024-09-19T06:52:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.595+0000 7f301455e640 1 -- 172.21.15.103:0/439107420 shutdown_connections 2024-09-19T06:52:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.595+0000 7f301455e640 1 --2- 172.21.15.103:0/439107420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f300c102ef0 0x7f300c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.595+0000 7f301455e640 1 --2- 172.21.15.103:0/439107420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f300c102570 0x7f300c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.595+0000 7f301455e640 1 --2- 172.21.15.103:0/439107420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f300c100e20 0x7f300c101200 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.595+0000 7f301455e640 1 -- 172.21.15.103:0/439107420 >> 172.21.15.103:0/439107420 conn(0x7f300c0fc7d0 msgr2=0x7f300c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.595+0000 7f301455e640 1 -- 172.21.15.103:0/439107420 shutdown_connections 2024-09-19T06:52:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.595+0000 7f301455e640 1 -- 172.21.15.103:0/439107420 wait complete. 2024-09-19T06:52:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f301455e640 1 Processor -- start 2024-09-19T06:52:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f301455e640 1 -- start start 2024-09-19T06:52:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f301455e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f300c100e20 0x7f300c1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f301455e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f300c102570 0x7f300c1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f301455e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f300c102ef0 0x7f300c1a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f301455e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f300c110fc0 con 0x7f300c102ef0 2024-09-19T06:52:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f301455e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f300c110e40 con 0x7f300c102570 2024-09-19T06:52:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f301455e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f300c111140 con 0x7f300c100e20 2024-09-19T06:52:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f3012afb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f300c102ef0 0x7f300c1a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f3012afb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f300c102ef0 0x7f300c1a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38948/0 (socket says 172.21.15.103:38948) 2024-09-19T06:52:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f3012afb640 1 -- 172.21.15.103:0/72365579 learned_addr learned my addr 172.21.15.103:0/72365579 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f30122fa640 1 --2- 172.21.15.103:0/72365579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f300c100e20 0x7f300c1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f3011af9640 1 --2- 172.21.15.103:0/72365579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f300c102570 0x7f300c1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f3012afb640 1 -- 172.21.15.103:0/72365579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f300c100e20 msgr2=0x7f300c1993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f3012afb640 1 --2- 172.21.15.103:0/72365579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f300c100e20 0x7f300c1993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f3012afb640 1 -- 172.21.15.103:0/72365579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f300c102570 msgr2=0x7f300c1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f3012afb640 1 --2- 172.21.15.103:0/72365579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f300c102570 0x7f300c1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f3012afb640 1 -- 172.21.15.103:0/72365579 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c1a4960 con 0x7f300c102ef0 2024-09-19T06:52:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f30122fa640 1 --2- 172.21.15.103:0/72365579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f300c100e20 0x7f300c1993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.596+0000 7f3011af9640 1 --2- 172.21.15.103:0/72365579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f300c102570 0x7f300c1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.597+0000 7f3012afb640 1 --2- 172.21.15.103:0/72365579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f300c102ef0 0x7f300c1a4420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f300400eaa0 tx=0x7f300400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.597+0000 7f2fff7fe640 1 -- 172.21.15.103:0/72365579 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300400cdb0 con 0x7f300c102ef0 2024-09-19T06:52:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.597+0000 7f2fff7fe640 1 -- 172.21.15.103:0/72365579 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3004005020 con 0x7f300c102ef0 2024-09-19T06:52:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.597+0000 7f2fff7fe640 1 -- 172.21.15.103:0/72365579 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30040116b0 con 0x7f300c102ef0 2024-09-19T06:52:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.597+0000 7f301455e640 1 -- 172.21.15.103:0/72365579 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300c1a4b30 con 0x7f300c102ef0 2024-09-19T06:52:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.597+0000 7f301455e640 1 -- 172.21.15.103:0/72365579 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300c1a5070 con 0x7f300c102ef0 2024-09-19T06:52:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.600+0000 7f2fff7fe640 1 -- 172.21.15.103:0/72365579 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3004005560 con 0x7f300c102ef0 2024-09-19T06:52:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.600+0000 7f3016937740 1 Processor -- start 2024-09-19T06:52:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.600+0000 7f3016937740 1 -- start start 2024-09-19T06:52:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.600+0000 7f3016937740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b584b40 0x55899b583090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.600+0000 7f3016937740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b5835d0 0x55899b583a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.600+0000 7f3016937740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b58e370 0x55899b590760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.600+0000 7f3016937740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55899b4e92f0 con 0x55899b58e370 2024-09-19T06:52:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.600+0000 7f3016937740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55899b490cf0 con 0x55899b584b40 2024-09-19T06:52:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.600+0000 7f3016937740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55899b4e52b0 con 0x55899b5835d0 2024-09-19T06:52:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.600+0000 7f3011af9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b58e370 0x55899b590760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.600+0000 7f3011af9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b58e370 0x55899b590760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38960/0 (socket says 172.21.15.103:38960) 2024-09-19T06:52:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.600+0000 7f3011af9640 1 -- 172.21.15.103:0/4154361750 learned_addr learned my addr 172.21.15.103:0/4154361750 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.600+0000 7f30122fa640 1 --2- 172.21.15.103:0/4154361750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b5835d0 0x55899b583a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.601+0000 7f3011af9640 1 --2- 172.21.15.103:0/4154361750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b584b40 0x55899b583090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.601+0000 7f2fff7fe640 1 --2- 172.21.15.103:0/72365579 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2ff80a3430 0x7f2ff80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.601+0000 7f3012afb640 1 --2- 172.21.15.103:0/72365579 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2ff80a3430 0x7f2ff80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.601+0000 7f30122fa640 1 -- 172.21.15.103:0/4154361750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b584b40 msgr2=0x55899b583090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.601+0000 7f30122fa640 1 --2- 172.21.15.103:0/4154361750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b584b40 0x55899b583090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.601+0000 7f30122fa640 1 -- 172.21.15.103:0/4154361750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b58e370 msgr2=0x55899b590760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.601+0000 7f30122fa640 1 --2- 172.21.15.103:0/4154361750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b58e370 0x55899b590760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.601+0000 7f30122fa640 1 -- 172.21.15.103:0/4154361750 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55899b4753a0 con 0x55899b5835d0 2024-09-19T06:52:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.601+0000 7f3011af9640 1 --2- 172.21.15.103:0/4154361750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b58e370 0x55899b590760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.601+0000 7f2fff7fe640 1 -- 172.21.15.103:0/72365579 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f30040cff70 con 0x7f300c102ef0 2024-09-19T06:52:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.601+0000 7f30122fa640 1 --2- 172.21.15.103:0/4154361750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b5835d0 0x55899b583a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3000009230 tx=0x7f3000004200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=661329944fd271ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.601+0000 7f3012afb640 1 --2- 172.21.15.103:0/72365579 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2ff80a3430 0x7f2ff80a58f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f30040d02a0 tx=0x7f300406dfe0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.601+0000 7f2ffcff9640 1 -- 172.21.15.103:0/4154361750 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000035030 con 0x55899b5835d0 2024-09-19T06:52:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.601+0000 7f2ffcff9640 1 -- 172.21.15.103:0/4154361750 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3000005520 con 0x55899b5835d0 2024-09-19T06:52:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.601+0000 7f2ffcff9640 1 -- 172.21.15.103:0/4154361750 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30000035f0 con 0x55899b5835d0 2024-09-19T06:52:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.602+0000 7f3016937740 1 -- 172.21.15.103:0/4154361750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b5835d0 msgr2=0x55899b583a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.602+0000 7f3016937740 1 --2- 172.21.15.103:0/4154361750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b5835d0 0x55899b583a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3000009230 tx=0x7f3000004200 comp rx=0 tx=0).stop 2024-09-19T06:52:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.602+0000 7f3016937740 1 -- 172.21.15.103:0/4154361750 shutdown_connections 2024-09-19T06:52:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.602+0000 7f3016937740 1 --2- 172.21.15.103:0/4154361750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b58e370 0x55899b590760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.602+0000 7f3016937740 1 --2- 172.21.15.103:0/4154361750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b5835d0 0x55899b583a10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.602+0000 7f3016937740 1 --2- 172.21.15.103:0/4154361750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b584b40 0x55899b583090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.602+0000 7f3016937740 1 -- 172.21.15.103:0/4154361750 >> 172.21.15.103:0/4154361750 conn(0x55899b54c310 msgr2=0x55899b57fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.602+0000 7f3016937740 1 -- 172.21.15.103:0/4154361750 shutdown_connections 2024-09-19T06:52:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.602+0000 7f3016937740 1 -- 172.21.15.103:0/4154361750 wait complete. 2024-09-19T06:52:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f3016937740 1 -- start start 2024-09-19T06:52:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f3016937740 1 Processor -- start 2024-09-19T06:52:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f3016937740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b5835d0 0x55899b58dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f3016937740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b584b40 0x55899b58b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f3016937740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b58e370 0x55899b58b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f3016937740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55899b591010 con 0x55899b584b40 2024-09-19T06:52:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f3016937740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55899b4a4060 con 0x55899b58e370 2024-09-19T06:52:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f3016937740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55899b591190 con 0x55899b5835d0 2024-09-19T06:52:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f3016937740 10 client.0 _ll_register_callbacks cb 0x55899b1f78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f30122fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b584b40 0x55899b58b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f30122fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b584b40 0x55899b58b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38968/0 (socket says 172.21.15.103:38968) 2024-09-19T06:52:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f30122fa640 1 -- 172.21.15.103:0/3293782353 learned_addr learned my addr 172.21.15.103:0/3293782353 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f3011af9640 1 --2- 172.21.15.103:0/3293782353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b5835d0 0x55899b58dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:54.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f3011af9640 1 --2- 172.21.15.103:0/3293782353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b58e370 0x55899b58b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:54.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f30122fa640 1 -- 172.21.15.103:0/3293782353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b5835d0 msgr2=0x55899b58dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:54.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f30122fa640 1 --2- 172.21.15.103:0/3293782353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b5835d0 0x55899b58dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f30122fa640 1 -- 172.21.15.103:0/3293782353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b58e370 msgr2=0x55899b58b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:54.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f30122fa640 1 --2- 172.21.15.103:0/3293782353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b58e370 0x55899b58b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f30122fa640 1 -- 172.21.15.103:0/3293782353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899b25a3c0 con 0x55899b584b40 2024-09-19T06:52:54.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.603+0000 7f3011af9640 1 --2- 172.21.15.103:0/3293782353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b58e370 0x55899b58b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:54.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.604+0000 7f30122fa640 1 --2- 172.21.15.103:0/3293782353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b584b40 0x55899b58b030 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f300002a9a0 tx=0x7f3000004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6434f0460e79112 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:54.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.604+0000 7f2ff5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:54.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.604+0000 7f2ff5ffb640 1 -- 172.21.15.103:0/3293782353 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000035070 con 0x55899b584b40 2024-09-19T06:52:54.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.604+0000 7f2ff5ffb640 1 -- 172.21.15.103:0/3293782353 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f3000004fe0 con 0x55899b584b40 2024-09-19T06:52:54.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.604+0000 7f3016937740 1 -- 172.21.15.103:0/3293782353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3000005520 con 0x55899b584b40 2024-09-19T06:52:54.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.604+0000 7f2ff5ffb640 1 -- 172.21.15.103:0/3293782353 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f30000051e0 con 0x55899b584b40 2024-09-19T06:52:54.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.604+0000 7f3016937740 10 client.16734 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:54.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.604+0000 7f3016937740 1 -- 172.21.15.103:0/3293782353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55899b59cae0 con 0x55899b584b40 2024-09-19T06:52:54.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.604+0000 7f2ff5ffb640 1 -- 172.21.15.103:0/3293782353 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3000005520 con 0x55899b584b40 2024-09-19T06:52:54.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.604+0000 7f2ff5ffb640 1 -- 172.21.15.103:0/3293782353 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f30000316d0 con 0x55899b584b40 2024-09-19T06:52:54.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.604+0000 7f3016937740 10 client.16734 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:54.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.604+0000 7f3016937740 10 client.16734 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:52:54.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.604+0000 7f3016937740 20 client.16734 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:54.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.604+0000 7f3016937740 1 --2- 172.21.15.103:0/3293782353 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55899b59de30 0x55899b5a0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.604+0000 7f3016937740 4 client.16734 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:54.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.604+0000 7f3016937740 1 -- 172.21.15.103:0/3293782353 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55899b4f6860 con 0x55899b59de30 2024-09-19T06:52:54.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.605+0000 7f3011af9640 1 --2- 172.21.15.103:0/3293782353 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55899b59de30 0x55899b5a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:54.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.605+0000 7f2ff5ffb640 10 client.16734 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:52:54.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.605+0000 7f3011af9640 1 --2- 172.21.15.103:0/3293782353 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55899b59de30 0x55899b5a0220 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=39a792c6ea8d00f6 server_cookie=83cbf22675e0055d in_seq=0 out_seq=0 2024-09-19T06:52:54.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.606+0000 7f2ff5ffb640 1 -- 172.21.15.103:0/3293782353 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55899b4f6860 con 0x55899b59de30 2024-09-19T06:52:54.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.606+0000 7f2ff5ffb640 10 client.16734 handle_command_reply: tid=0 2024-09-19T06:52:54.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.606+0000 7f3016937740 1 client.16734 shutdown 2024-09-19T06:52:54.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.606+0000 7f3016937740 1 -- 172.21.15.103:0/3293782353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b584b40 msgr2=0x55899b58b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:54.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.606+0000 7f3016937740 1 --2- 172.21.15.103:0/3293782353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b584b40 0x55899b58b030 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f300002a9a0 tx=0x7f3000004b80 comp rx=0 tx=0).stop 2024-09-19T06:52:54.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.606+0000 7f3016937740 1 -- 172.21.15.103:0/3293782353 shutdown_connections 2024-09-19T06:52:54.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.606+0000 7f3016937740 1 --2- 172.21.15.103:0/3293782353 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55899b59de30 0x55899b5a0220 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.606+0000 7f3016937740 1 --2- 172.21.15.103:0/3293782353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b58e370 0x55899b58b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.606+0000 7f3016937740 1 --2- 172.21.15.103:0/3293782353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b584b40 0x55899b58b030 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.606+0000 7f2ff5ffb640 0 client.16734 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:52:54.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.606+0000 7f3016937740 1 --2- 172.21.15.103:0/3293782353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b5835d0 0x55899b58dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.606+0000 7f3016937740 1 -- 172.21.15.103:0/3293782353 >> 172.21.15.103:0/3293782353 conn(0x55899b54c310 msgr2=0x55899b585960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:54.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.606+0000 7f3016937740 1 -- 172.21.15.103:0/3293782353 shutdown_connections 2024-09-19T06:52:54.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.606+0000 7f3016937740 1 -- 172.21.15.103:0/3293782353 wait complete. 2024-09-19T06:52:54.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.606+0000 7f3016937740 20 client.16734 trim_cache size 0 max 16384 2024-09-19T06:52:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.619+0000 7f3016937740 1 Processor -- start 2024-09-19T06:52:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.619+0000 7f3016937740 1 -- start start 2024-09-19T06:52:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.619+0000 7f3016937740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b5835d0 0x55899b59a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.619+0000 7f3016937740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b584b40 0x55899b58a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.619+0000 7f3016937740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b58e370 0x55899b586120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.619+0000 7f3016937740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1ad870 con 0x55899b584b40 2024-09-19T06:52:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.619+0000 7f3016937740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1af3b0 con 0x55899b58e370 2024-09-19T06:52:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.619+0000 7f3016937740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1adf40 con 0x55899b5835d0 2024-09-19T06:52:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.619+0000 7f30122fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b584b40 0x55899b58a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:54.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.619+0000 7f30122fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b584b40 0x55899b58a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38970/0 (socket says 172.21.15.103:38970) 2024-09-19T06:52:54.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.619+0000 7f30122fa640 1 -- 172.21.15.103:0/4271762695 learned_addr learned my addr 172.21.15.103:0/4271762695 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:54.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.620+0000 7f30122fa640 1 -- 172.21.15.103:0/4271762695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b5835d0 msgr2=0x55899b59a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:54.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.620+0000 7f30122fa640 1 --2- 172.21.15.103:0/4271762695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b5835d0 0x55899b59a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.620+0000 7f30122fa640 1 -- 172.21.15.103:0/4271762695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b58e370 msgr2=0x55899b586120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:54.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.620+0000 7f30122fa640 1 --2- 172.21.15.103:0/4271762695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b58e370 0x55899b586120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.620+0000 7f30122fa640 1 -- 172.21.15.103:0/4271762695 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55899b58bfd0 con 0x55899b584b40 2024-09-19T06:52:54.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.620+0000 7f30122fa640 1 --2- 172.21.15.103:0/4271762695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b584b40 0x55899b58a8f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3000009900 tx=0x7f30000097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c4231728a94ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:54.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.620+0000 7f2ff7fff640 1 -- 172.21.15.103:0/4271762695 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000035030 con 0x55899b584b40 2024-09-19T06:52:54.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.620+0000 7f2ff7fff640 1 -- 172.21.15.103:0/4271762695 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f300005b020 con 0x55899b584b40 2024-09-19T06:52:54.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.620+0000 7f2ff7fff640 1 -- 172.21.15.103:0/4271762695 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000005550 con 0x55899b584b40 2024-09-19T06:52:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 -- 172.21.15.103:0/4271762695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b584b40 msgr2=0x55899b58a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 --2- 172.21.15.103:0/4271762695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b584b40 0x55899b58a8f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3000009900 tx=0x7f30000097a0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 -- 172.21.15.103:0/4271762695 shutdown_connections 2024-09-19T06:52:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 --2- 172.21.15.103:0/4271762695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b58e370 0x55899b586120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 --2- 172.21.15.103:0/4271762695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b584b40 0x55899b58a8f0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 --2- 172.21.15.103:0/4271762695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b5835d0 0x55899b59a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 -- 172.21.15.103:0/4271762695 >> 172.21.15.103:0/4271762695 conn(0x55899b54c310 msgr2=0x55899b58a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 -- 172.21.15.103:0/4271762695 shutdown_connections 2024-09-19T06:52:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 -- 172.21.15.103:0/4271762695 wait complete. 2024-09-19T06:52:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 -- start start 2024-09-19T06:52:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 Processor -- start 2024-09-19T06:52:54.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b5835d0 0x55899b5a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b584b40 0x55899b5a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b58e370 0x55899b606760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1aece0 con 0x55899b5835d0 2024-09-19T06:52:54.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1ae610 con 0x55899b584b40 2024-09-19T06:52:54.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1ab830 con 0x55899b58e370 2024-09-19T06:52:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3016937740 10 client.0 _ll_register_callbacks cb 0x7f300800c810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3011af9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b5835d0 0x55899b5a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3011af9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b5835d0 0x55899b5a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38980/0 (socket says 172.21.15.103:38980) 2024-09-19T06:52:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.621+0000 7f3011af9640 1 -- 172.21.15.103:0/3507645934 learned_addr learned my addr 172.21.15.103:0/3507645934 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.622+0000 7f30122fa640 1 --2- 172.21.15.103:0/3507645934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b584b40 0x55899b5a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.622+0000 7f3011af9640 1 --2- 172.21.15.103:0/3507645934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b58e370 0x55899b606760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:54.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.622+0000 7f3011af9640 1 -- 172.21.15.103:0/3507645934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b58e370 msgr2=0x55899b606760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:54.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.622+0000 7f3011af9640 1 --2- 172.21.15.103:0/3507645934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b58e370 0x55899b606760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.622+0000 7f3011af9640 1 -- 172.21.15.103:0/3507645934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b584b40 msgr2=0x55899b5a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:54.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.622+0000 7f3011af9640 1 --2- 172.21.15.103:0/3507645934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b584b40 0x55899b5a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.622+0000 7f3011af9640 1 -- 172.21.15.103:0/3507645934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899b606ca0 con 0x55899b5835d0 2024-09-19T06:52:54.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.622+0000 7f3011af9640 1 --2- 172.21.15.103:0/3507645934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b5835d0 0x55899b5a06e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3008000c00 tx=0x7f3008006bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc786b5399bed764 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:54.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.622+0000 7f2ff5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:54.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.622+0000 7f2ff5ffb640 1 -- 172.21.15.103:0/3507645934 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300800f040 con 0x55899b5835d0 2024-09-19T06:52:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.622+0000 7f2ff5ffb640 1 -- 172.21.15.103:0/3507645934 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f30080187e0 con 0x55899b5835d0 2024-09-19T06:52:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.622+0000 7f3016937740 1 -- 172.21.15.103:0/3507645934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f300005b020 con 0x55899b5835d0 2024-09-19T06:52:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.622+0000 7f2ff5ffb640 1 -- 172.21.15.103:0/3507645934 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3008031050 con 0x55899b5835d0 2024-09-19T06:52:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.622+0000 7f3016937740 10 client.16740 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.622+0000 7f3016937740 1 -- 172.21.15.103:0/3507645934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55899b607fd0 con 0x55899b5835d0 2024-09-19T06:52:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.622+0000 7f2ff5ffb640 1 -- 172.21.15.103:0/3507645934 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f300005b020 con 0x55899b5835d0 2024-09-19T06:52:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.623+0000 7f2ff5ffb640 1 -- 172.21.15.103:0/3507645934 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3008015040 con 0x55899b5835d0 2024-09-19T06:52:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.623+0000 7f3016937740 10 client.16740 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.623+0000 7f3016937740 10 client.16740 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:52:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.623+0000 7f3016937740 20 client.16740 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.623+0000 7f3016937740 1 --2- 172.21.15.103:0/3507645934 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55899b6086d0 0x55899b60aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.623+0000 7f3016937740 4 client.16740 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:52:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.623+0000 7f3016937740 1 -- 172.21.15.103:0/3507645934 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55899b4f0d70 con 0x55899b6086d0 2024-09-19T06:52:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.623+0000 7f30122fa640 1 --2- 172.21.15.103:0/3507645934 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55899b6086d0 0x55899b60aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.623+0000 7f30122fa640 1 --2- 172.21.15.103:0/3507645934 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55899b6086d0 0x55899b60aaf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6089ea8cfae55d61 server_cookie=fe3092c05526c404 in_seq=0 out_seq=0 2024-09-19T06:52:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.623+0000 7f2ff5ffb640 10 client.16740 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:52:54.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.624+0000 7f2ff5ffb640 1 -- 172.21.15.103:0/3507645934 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55899b4f0d70 con 0x55899b6086d0 2024-09-19T06:52:54.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.624+0000 7f2ff5ffb640 10 client.16740 handle_command_reply: tid=0 2024-09-19T06:52:54.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.624+0000 7f3016937740 1 client.16740 shutdown 2024-09-19T06:52:54.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.624+0000 7f3016937740 1 -- 172.21.15.103:0/3507645934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b5835d0 msgr2=0x55899b5a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:54.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.624+0000 7f3016937740 1 --2- 172.21.15.103:0/3507645934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b5835d0 0x55899b5a06e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3008000c00 tx=0x7f3008006bf0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.624+0000 7f3016937740 1 -- 172.21.15.103:0/3507645934 shutdown_connections 2024-09-19T06:52:54.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.624+0000 7f3016937740 1 --2- 172.21.15.103:0/3507645934 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55899b6086d0 0x55899b60aaf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.624+0000 7f3016937740 1 --2- 172.21.15.103:0/3507645934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899b58e370 0x55899b606760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.624+0000 7f2ff5ffb640 0 client.16740 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:52:54.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.624+0000 7f3016937740 1 --2- 172.21.15.103:0/3507645934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899b584b40 0x55899b5a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.624+0000 7f3016937740 1 --2- 172.21.15.103:0/3507645934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899b5835d0 0x55899b5a06e0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.624+0000 7f3016937740 1 -- 172.21.15.103:0/3507645934 >> 172.21.15.103:0/3507645934 conn(0x55899b54c310 msgr2=0x55899b58a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:54.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.624+0000 7f3016937740 1 -- 172.21.15.103:0/3507645934 shutdown_connections 2024-09-19T06:52:54.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.624+0000 7f3016937740 1 -- 172.21.15.103:0/3507645934 wait complete. 2024-09-19T06:52:54.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.624+0000 7f3016937740 20 client.16740 trim_cache size 0 max 16384 2024-09-19T06:52:54.636 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:52:54.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.624+0000 7f301455e640 1 -- 172.21.15.103:0/72365579 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2ff80a3430 msgr2=0x7f2ff80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:54.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.625+0000 7f301455e640 1 --2- 172.21.15.103:0/72365579 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2ff80a3430 0x7f2ff80a58f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f30040d02a0 tx=0x7f300406dfe0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.625+0000 7f301455e640 1 -- 172.21.15.103:0/72365579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f300c102ef0 msgr2=0x7f300c1a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:54.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.625+0000 7f301455e640 1 --2- 172.21.15.103:0/72365579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f300c102ef0 0x7f300c1a4420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f300400eaa0 tx=0x7f300400ef60 comp rx=0 tx=0).stop 2024-09-19T06:52:54.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.625+0000 7f301455e640 1 -- 172.21.15.103:0/72365579 shutdown_connections 2024-09-19T06:52:54.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.625+0000 7f301455e640 1 --2- 172.21.15.103:0/72365579 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2ff80a3430 0x7f2ff80a58f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.625+0000 7f301455e640 1 --2- 172.21.15.103:0/72365579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f300c102ef0 0x7f300c1a4420 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.625+0000 7f301455e640 1 --2- 172.21.15.103:0/72365579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f300c102570 0x7f300c1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.625+0000 7f301455e640 1 --2- 172.21.15.103:0/72365579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f300c100e20 0x7f300c1993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:54.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.625+0000 7f301455e640 1 -- 172.21.15.103:0/72365579 >> 172.21.15.103:0/72365579 conn(0x7f300c0fc7d0 msgr2=0x7f300c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:54.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.625+0000 7f301455e640 1 -- 172.21.15.103:0/72365579 shutdown_connections 2024-09-19T06:52:54.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:54.625+0000 7f301455e640 1 -- 172.21.15.103:0/72365579 wait complete. 2024-09-19T06:52:55.323 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:52:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.527+0000 7f8bf880e640 1 -- 172.21.15.103:0/1789905214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf4101370 msgr2=0x7f8bf410d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.527+0000 7f8bf880e640 1 --2- 172.21.15.103:0/1789905214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf4101370 0x7f8bf410d8a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009cd0 tx=0x7f8bec02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.527+0000 7f8bf880e640 1 -- 172.21.15.103:0/1789905214 shutdown_connections 2024-09-19T06:52:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.527+0000 7f8bf880e640 1 --2- 172.21.15.103:0/1789905214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf4101370 0x7f8bf410d8a0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.527+0000 7f8bf880e640 1 --2- 172.21.15.103:0/1789905214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bf41009d0 0x7f8bf4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.527+0000 7f8bf880e640 1 --2- 172.21.15.103:0/1789905214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bf41069e0 0x7f8bf4106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.527+0000 7f8bf880e640 1 -- 172.21.15.103:0/1789905214 >> 172.21.15.103:0/1789905214 conn(0x7f8bf40fc6b0 msgr2=0x7f8bf40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.528+0000 7f8bf880e640 1 -- 172.21.15.103:0/1789905214 shutdown_connections 2024-09-19T06:52:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.528+0000 7f8bf880e640 1 -- 172.21.15.103:0/1789905214 wait complete. 2024-09-19T06:52:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.528+0000 7f8bf880e640 1 Processor -- start 2024-09-19T06:52:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.528+0000 7f8bf880e640 1 -- start start 2024-09-19T06:52:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.529+0000 7f8bf880e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bf41009d0 0x7f8bf4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.529+0000 7f8bf880e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bf4101370 0x7f8bf41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.529+0000 7f8bf880e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf41069e0 0x7f8bf41a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.529+0000 7f8bf880e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4110e20 con 0x7f8bf41069e0 2024-09-19T06:52:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.529+0000 7f8bf880e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4110ca0 con 0x7f8bf41009d0 2024-09-19T06:52:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.529+0000 7f8bf880e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4110fa0 con 0x7f8bf4101370 2024-09-19T06:52:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.529+0000 7f8bf2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf41069e0 0x7f8bf41a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.529+0000 7f8bf2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf41069e0 0x7f8bf41a4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39000/0 (socket says 172.21.15.103:39000) 2024-09-19T06:52:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.529+0000 7f8bf2d9d640 1 -- 172.21.15.103:0/216700424 learned_addr learned my addr 172.21.15.103:0/216700424 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.529+0000 7f8bf259c640 1 --2- 172.21.15.103:0/216700424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bf41009d0 0x7f8bf4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.529+0000 7f8bf1d9b640 1 --2- 172.21.15.103:0/216700424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bf4101370 0x7f8bf41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.529+0000 7f8bf2d9d640 1 -- 172.21.15.103:0/216700424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bf4101370 msgr2=0x7f8bf41996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.529+0000 7f8bf2d9d640 1 --2- 172.21.15.103:0/216700424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bf4101370 0x7f8bf41996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.529+0000 7f8bf2d9d640 1 -- 172.21.15.103:0/216700424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bf41009d0 msgr2=0x7f8bf4199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.529+0000 7f8bf2d9d640 1 --2- 172.21.15.103:0/216700424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bf41009d0 0x7f8bf4199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.529+0000 7f8bf2d9d640 1 -- 172.21.15.103:0/216700424 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf41a4750 con 0x7f8bf41069e0 2024-09-19T06:52:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.530+0000 7f8bf2d9d640 1 --2- 172.21.15.103:0/216700424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf41069e0 0x7f8bf41a4210 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8bec005cc0 tx=0x7f8bec003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.530+0000 7f8be37fe640 1 -- 172.21.15.103:0/216700424 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec004040 con 0x7f8bf41069e0 2024-09-19T06:52:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.530+0000 7f8be37fe640 1 -- 172.21.15.103:0/216700424 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bec0046c0 con 0x7f8bf41069e0 2024-09-19T06:52:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.530+0000 7f8bf880e640 1 -- 172.21.15.103:0/216700424 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf41a4920 con 0x7f8bf41069e0 2024-09-19T06:52:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.530+0000 7f8be37fe640 1 -- 172.21.15.103:0/216700424 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec046480 con 0x7f8bf41069e0 2024-09-19T06:52:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.530+0000 7f8bf880e640 1 -- 172.21.15.103:0/216700424 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf41a4de0 con 0x7f8bf41069e0 2024-09-19T06:52:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.532+0000 7f8be37fe640 1 -- 172.21.15.103:0/216700424 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8bec0305a0 con 0x7f8bf41069e0 2024-09-19T06:52:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.533+0000 7f8bfb685740 1 Processor -- start 2024-09-19T06:52:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.533+0000 7f8bfb685740 1 -- start start 2024-09-19T06:52:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.533+0000 7f8be37fe640 1 --2- 172.21.15.103:0/216700424 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8bdc0a3460 0x7f8bdc0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.533+0000 7f8bfb685740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56464095eaf0 0x56464095eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.533+0000 7f8bfb685740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x564640969770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.533+0000 7f8bfb685740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564640969cb0 0x56464096c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.533+0000 7f8bfb685740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5646408c32f0 con 0x56464095f410 2024-09-19T06:52:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.533+0000 7f8bfb685740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56464086b270 con 0x564640969cb0 2024-09-19T06:52:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.533+0000 7f8bfb685740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5646408bf2b0 con 0x56464095eaf0 2024-09-19T06:52:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.533+0000 7f8bf1d9b640 1 --2- 172.21.15.103:0/216700424 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8bdc0a3460 0x7f8bdc0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.533+0000 7f8bf1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x564640969770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.534+0000 7f8bf1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x564640969770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39014/0 (socket says 172.21.15.103:39014) 2024-09-19T06:52:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.534+0000 7f8bf1d9b640 1 -- 172.21.15.103:0/518536796 learned_addr learned my addr 172.21.15.103:0/518536796 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.534+0000 7f8be37fe640 1 -- 172.21.15.103:0/216700424 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f8bec02e030 con 0x7f8bf41069e0 2024-09-19T06:52:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.534+0000 7f8bf259c640 1 --2- 172.21.15.103:0/518536796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56464095eaf0 0x56464095eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.534+0000 7f8bf2d9d640 1 --2- 172.21.15.103:0/518536796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564640969cb0 0x56464096c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.534+0000 7f8bf1d9b640 1 -- 172.21.15.103:0/518536796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56464095eaf0 msgr2=0x56464095eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.534+0000 7f8bf1d9b640 1 --2- 172.21.15.103:0/518536796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56464095eaf0 0x56464095eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.534+0000 7f8bf1d9b640 1 -- 172.21.15.103:0/518536796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564640969cb0 msgr2=0x56464096c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.534+0000 7f8bf1d9b640 1 --2- 172.21.15.103:0/518536796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564640969cb0 0x56464096c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.534+0000 7f8bf1d9b640 1 -- 172.21.15.103:0/518536796 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56464086a340 con 0x56464095f410 2024-09-19T06:52:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.534+0000 7f8bf1d9b640 1 --2- 172.21.15.103:0/216700424 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8bdc0a3460 0x7f8bdc0a5920 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8be8005d00 tx=0x7f8be800bb90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.534+0000 7f8bf1d9b640 1 --2- 172.21.15.103:0/518536796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x564640969770 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8be80123f0 tx=0x7f8be801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8327ff6861445e23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.534+0000 7f8be0ff9640 1 -- 172.21.15.103:0/518536796 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8024070 con 0x56464095f410 2024-09-19T06:52:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.534+0000 7f8be0ff9640 1 -- 172.21.15.103:0/518536796 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8be8010e00 con 0x56464095f410 2024-09-19T06:52:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.534+0000 7f8be0ff9640 1 -- 172.21.15.103:0/518536796 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8004e00 con 0x56464095f410 2024-09-19T06:52:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.535+0000 7f8bfb685740 1 -- 172.21.15.103:0/518536796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 msgr2=0x564640969770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.535+0000 7f8bfb685740 1 --2- 172.21.15.103:0/518536796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x564640969770 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8be80123f0 tx=0x7f8be801c000 comp rx=0 tx=0).stop 2024-09-19T06:52:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.535+0000 7f8bfb685740 1 -- 172.21.15.103:0/518536796 shutdown_connections 2024-09-19T06:52:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.535+0000 7f8bfb685740 1 --2- 172.21.15.103:0/518536796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564640969cb0 0x56464096c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.535+0000 7f8bfb685740 1 --2- 172.21.15.103:0/518536796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x564640969770 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.535+0000 7f8bfb685740 1 --2- 172.21.15.103:0/518536796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56464095eaf0 0x56464095eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.535+0000 7f8bfb685740 1 -- 172.21.15.103:0/518536796 >> 172.21.15.103:0/518536796 conn(0x56464095b060 msgr2=0x56464095b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.535+0000 7f8bfb685740 1 -- 172.21.15.103:0/518536796 shutdown_connections 2024-09-19T06:52:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.535+0000 7f8bfb685740 1 -- 172.21.15.103:0/518536796 wait complete. 2024-09-19T06:52:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.535+0000 7f8bfb685740 1 -- start start 2024-09-19T06:52:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.536+0000 7f8bfb685740 1 Processor -- start 2024-09-19T06:52:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.536+0000 7f8bfb685740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56464095eaf0 0x564640966c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.536+0000 7f8bfb685740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56464095f410 0x5646409671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.536+0000 7f8bfb685740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564640969cb0 0x564640967710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.536+0000 7f8bfb685740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56464096c880 con 0x564640969cb0 2024-09-19T06:52:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.536+0000 7f8bfb685740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56464084c290 con 0x56464095eaf0 2024-09-19T06:52:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.536+0000 7f8bfb685740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56464096ca00 con 0x56464095f410 2024-09-19T06:52:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.536+0000 7f8bfb685740 10 client.0 _ll_register_callbacks cb 0x56464087d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.536+0000 7f8bf2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564640969cb0 0x564640967710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.536+0000 7f8bf2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564640969cb0 0x564640967710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39028/0 (socket says 172.21.15.103:39028) 2024-09-19T06:52:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.536+0000 7f8bf2d9d640 1 -- 172.21.15.103:0/2648031405 learned_addr learned my addr 172.21.15.103:0/2648031405 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.536+0000 7f8bf1d9b640 1 --2- 172.21.15.103:0/2648031405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56464095f410 0x5646409671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bf259c640 1 --2- 172.21.15.103:0/2648031405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56464095eaf0 0x564640966c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bf2d9d640 1 -- 172.21.15.103:0/2648031405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56464095f410 msgr2=0x5646409671d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bf2d9d640 1 --2- 172.21.15.103:0/2648031405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56464095f410 0x5646409671d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bf2d9d640 1 -- 172.21.15.103:0/2648031405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56464095eaf0 msgr2=0x564640966c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bf2d9d640 1 --2- 172.21.15.103:0/2648031405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56464095eaf0 0x564640966c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bf2d9d640 1 -- 172.21.15.103:0/2648031405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646405d6210 con 0x564640969cb0 2024-09-19T06:52:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bf1d9b640 1 --2- 172.21.15.103:0/2648031405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56464095f410 0x5646409671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bf2d9d640 1 --2- 172.21.15.103:0/2648031405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564640969cb0 0x564640967710 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8bec006010 tx=0x7f8bec0bd540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5200e77caab8fa33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bd9ffb640 1 -- 172.21.15.103:0/2648031405 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0bde70 con 0x564640969cb0 2024-09-19T06:52:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bd9ffb640 1 -- 172.21.15.103:0/2648031405 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f8bec0bcbc0 con 0x564640969cb0 2024-09-19T06:52:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bfb685740 1 -- 172.21.15.103:0/2648031405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8be8010e00 con 0x564640969cb0 2024-09-19T06:52:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bd9ffb640 1 -- 172.21.15.103:0/2648031405 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8bec0098b0 con 0x564640969cb0 2024-09-19T06:52:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bfb685740 10 client.16752 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bfb685740 1 -- 172.21.15.103:0/2648031405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5646406a2620 con 0x564640969cb0 2024-09-19T06:52:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bd9ffb640 1 -- 172.21.15.103:0/2648031405 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8be8010e00 con 0x564640969cb0 2024-09-19T06:52:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.537+0000 7f8bd9ffb640 1 -- 172.21.15.103:0/2648031405 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8bec04e070 con 0x564640969cb0 2024-09-19T06:52:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.538+0000 7f8bfb685740 10 client.16752 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.538+0000 7f8bfb685740 10 client.16752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.538+0000 7f8bfb685740 20 client.16752 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.538+0000 7f8bfb685740 1 --2- 172.21.15.103:0/2648031405 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564640979d30 0x56464097c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.538+0000 7f8bfb685740 4 client.16752 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.538+0000 7f8bfb685740 1 -- 172.21.15.103:0/2648031405 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646408d0a20 con 0x564640979d30 2024-09-19T06:52:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.538+0000 7f8bf259c640 1 --2- 172.21.15.103:0/2648031405 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564640979d30 0x56464097c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.538+0000 7f8bd9ffb640 10 client.16752 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.538+0000 7f8bf259c640 1 --2- 172.21.15.103:0/2648031405 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564640979d30 0x56464097c140 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f98566f567aeb2 server_cookie=44df933250f67cf in_seq=0 out_seq=0 2024-09-19T06:52:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.539+0000 7f8bd9ffb640 1 -- 172.21.15.103:0/2648031405 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646408d0a20 con 0x564640979d30 2024-09-19T06:52:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.539+0000 7f8bd9ffb640 10 client.16752 handle_command_reply: tid=0 2024-09-19T06:52:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.539+0000 7f8bfb685740 1 client.16752 shutdown 2024-09-19T06:52:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.539+0000 7f8bfb685740 1 -- 172.21.15.103:0/2648031405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564640969cb0 msgr2=0x564640967710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.539+0000 7f8bfb685740 1 --2- 172.21.15.103:0/2648031405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564640969cb0 0x564640967710 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8bec006010 tx=0x7f8bec0bd540 comp rx=0 tx=0).stop 2024-09-19T06:52:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.539+0000 7f8bfb685740 1 -- 172.21.15.103:0/2648031405 shutdown_connections 2024-09-19T06:52:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.539+0000 7f8bfb685740 1 --2- 172.21.15.103:0/2648031405 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564640979d30 0x56464097c140 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.539+0000 7f8bfb685740 1 --2- 172.21.15.103:0/2648031405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564640969cb0 0x564640967710 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.539+0000 7f8bfb685740 1 --2- 172.21.15.103:0/2648031405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56464095f410 0x5646409671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.539+0000 7f8bd9ffb640 0 client.16752 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.539+0000 7f8bfb685740 1 --2- 172.21.15.103:0/2648031405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56464095eaf0 0x564640966c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.539+0000 7f8bfb685740 1 -- 172.21.15.103:0/2648031405 >> 172.21.15.103:0/2648031405 conn(0x56464095b060 msgr2=0x564640960dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.539+0000 7f8bfb685740 1 -- 172.21.15.103:0/2648031405 shutdown_connections 2024-09-19T06:52:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.539+0000 7f8bfb685740 1 -- 172.21.15.103:0/2648031405 wait complete. 2024-09-19T06:52:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.539+0000 7f8bfb685740 20 client.16752 trim_cache size 0 max 16384 2024-09-19T06:52:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.554+0000 7f8bfb685740 1 Processor -- start 2024-09-19T06:52:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.554+0000 7f8bfb685740 1 -- start start 2024-09-19T06:52:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.554+0000 7f8bfb685740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56464095eaf0 0x564640967540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.554+0000 7f8bfb685740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x5646409615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.554+0000 7f8bfb685740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564640969cb0 0x5646409819f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.554+0000 7f8bfb685740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41ad6d0 con 0x56464095f410 2024-09-19T06:52:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.554+0000 7f8bfb685740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41af210 con 0x56464095eaf0 2024-09-19T06:52:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.554+0000 7f8bfb685740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41adda0 con 0x564640969cb0 2024-09-19T06:52:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.554+0000 7f8bf1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x5646409615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.554+0000 7f8bf1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x5646409615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39038/0 (socket says 172.21.15.103:39038) 2024-09-19T06:52:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.554+0000 7f8bf1d9b640 1 -- 172.21.15.103:0/1863814741 learned_addr learned my addr 172.21.15.103:0/1863814741 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.554+0000 7f8bf259c640 1 --2- 172.21.15.103:0/1863814741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56464095eaf0 0x564640967540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.554+0000 7f8bf2d9d640 1 --2- 172.21.15.103:0/1863814741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564640969cb0 0x5646409819f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.555+0000 7f8bf1d9b640 1 -- 172.21.15.103:0/1863814741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564640969cb0 msgr2=0x5646409819f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.555+0000 7f8bf1d9b640 1 --2- 172.21.15.103:0/1863814741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564640969cb0 0x5646409819f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.555+0000 7f8bf1d9b640 1 -- 172.21.15.103:0/1863814741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56464095eaf0 msgr2=0x564640967540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.555+0000 7f8bf1d9b640 1 --2- 172.21.15.103:0/1863814741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56464095eaf0 0x564640967540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.555+0000 7f8bf1d9b640 1 -- 172.21.15.103:0/1863814741 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564640967a80 con 0x56464095f410 2024-09-19T06:52:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.555+0000 7f8bf259c640 1 --2- 172.21.15.103:0/1863814741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56464095eaf0 0x564640967540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.555+0000 7f8bf1d9b640 1 --2- 172.21.15.103:0/1863814741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x5646409615b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8be800d720 tx=0x7f8be800b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fddce1bb6cf7a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.555+0000 7f8bdbfff640 1 -- 172.21.15.103:0/1863814741 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8024040 con 0x56464095f410 2024-09-19T06:52:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.555+0000 7f8bdbfff640 1 -- 172.21.15.103:0/1863814741 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8be800b690 con 0x56464095f410 2024-09-19T06:52:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.555+0000 7f8bdbfff640 1 -- 172.21.15.103:0/1863814741 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be80336e0 con 0x56464095f410 2024-09-19T06:52:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.556+0000 7f8bfb685740 1 -- 172.21.15.103:0/1863814741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 msgr2=0x5646409615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.556+0000 7f8bfb685740 1 --2- 172.21.15.103:0/1863814741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x5646409615b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8be800d720 tx=0x7f8be800b550 comp rx=0 tx=0).stop 2024-09-19T06:52:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.556+0000 7f8bfb685740 1 -- 172.21.15.103:0/1863814741 shutdown_connections 2024-09-19T06:52:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.556+0000 7f8bfb685740 1 --2- 172.21.15.103:0/1863814741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564640969cb0 0x5646409819f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.556+0000 7f8bfb685740 1 --2- 172.21.15.103:0/1863814741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x5646409615b0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.556+0000 7f8bfb685740 1 --2- 172.21.15.103:0/1863814741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56464095eaf0 0x564640967540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.556+0000 7f8bfb685740 1 -- 172.21.15.103:0/1863814741 >> 172.21.15.103:0/1863814741 conn(0x56464095b060 msgr2=0x564640968d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.556+0000 7f8bfb685740 1 -- 172.21.15.103:0/1863814741 shutdown_connections 2024-09-19T06:52:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.556+0000 7f8bfb685740 1 -- 172.21.15.103:0/1863814741 wait complete. 2024-09-19T06:52:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.556+0000 7f8bfb685740 1 -- start start 2024-09-19T06:52:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.556+0000 7f8bfb685740 1 Processor -- start 2024-09-19T06:52:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bfb685740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56464095eaf0 0x564640963b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bfb685740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x5646409640d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bfb685740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564640969cb0 0x564640964610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bfb685740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41aeb40 con 0x56464095f410 2024-09-19T06:52:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bfb685740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41ae470 con 0x56464095eaf0 2024-09-19T06:52:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bfb685740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41ab690 con 0x564640969cb0 2024-09-19T06:52:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bfb685740 10 client.0 _ll_register_callbacks cb 0x7f8bec005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bf1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x5646409640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bf1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x5646409640d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39052/0 (socket says 172.21.15.103:39052) 2024-09-19T06:52:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bf1d9b640 1 -- 172.21.15.103:0/326216431 learned_addr learned my addr 172.21.15.103:0/326216431 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bf2d9d640 1 --2- 172.21.15.103:0/326216431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564640969cb0 0x564640964610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bf259c640 1 --2- 172.21.15.103:0/326216431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56464095eaf0 0x564640963b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bf1d9b640 1 -- 172.21.15.103:0/326216431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564640969cb0 msgr2=0x564640964610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bf1d9b640 1 --2- 172.21.15.103:0/326216431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564640969cb0 0x564640964610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bf1d9b640 1 -- 172.21.15.103:0/326216431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56464095eaf0 msgr2=0x564640963b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bf1d9b640 1 --2- 172.21.15.103:0/326216431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56464095eaf0 0x564640963b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bf1d9b640 1 -- 172.21.15.103:0/326216431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646409e7dd0 con 0x56464095f410 2024-09-19T06:52:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bf1d9b640 1 --2- 172.21.15.103:0/326216431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x5646409640d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8be8005cd0 tx=0x7f8be80127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b33e6e74c350cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.557+0000 7f8bd9ffb640 1 -- 172.21.15.103:0/326216431 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8024040 con 0x56464095f410 2024-09-19T06:52:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.558+0000 7f8bd9ffb640 1 -- 172.21.15.103:0/326216431 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f8be800d750 con 0x56464095f410 2024-09-19T06:52:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.558+0000 7f8bfb685740 1 -- 172.21.15.103:0/326216431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8be800b690 con 0x56464095f410 2024-09-19T06:52:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.558+0000 7f8bd9ffb640 1 -- 172.21.15.103:0/326216431 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8be800d950 con 0x56464095f410 2024-09-19T06:52:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.558+0000 7f8bfb685740 10 client.16758 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.558+0000 7f8bfb685740 1 -- 172.21.15.103:0/326216431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5646409e8fc0 con 0x56464095f410 2024-09-19T06:52:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.558+0000 7f8bd9ffb640 1 -- 172.21.15.103:0/326216431 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8be800b690 con 0x56464095f410 2024-09-19T06:52:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.558+0000 7f8bd9ffb640 1 -- 172.21.15.103:0/326216431 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8be801f070 con 0x56464095f410 2024-09-19T06:52:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.558+0000 7f8bfb685740 10 client.16758 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.558+0000 7f8bfb685740 10 client.16758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.558+0000 7f8bfb685740 20 client.16758 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.558+0000 7f8bfb685740 1 --2- 172.21.15.103:0/326216431 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5646409e9490 0x5646409eb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.558+0000 7f8bfb685740 4 client.16758 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:52:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.558+0000 7f8bfb685740 1 -- 172.21.15.103:0/326216431 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646408cad70 con 0x5646409e9490 2024-09-19T06:52:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.558+0000 7f8bf259c640 1 --2- 172.21.15.103:0/326216431 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5646409e9490 0x5646409eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.559+0000 7f8bd9ffb640 10 client.16758 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.559+0000 7f8bf259c640 1 --2- 172.21.15.103:0/326216431 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5646409e9490 0x5646409eb880 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8c61fbbeb85bec7 server_cookie=97fed8735874b2 in_seq=0 out_seq=0 2024-09-19T06:52:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.571+0000 7f8bd9ffb640 1 -- 172.21.15.103:0/326216431 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646408cad70 con 0x5646409e9490 2024-09-19T06:52:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.571+0000 7f8bd9ffb640 10 client.16758 handle_command_reply: tid=0 2024-09-19T06:52:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.571+0000 7f8bfb685740 1 client.16758 shutdown 2024-09-19T06:52:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.572+0000 7f8bfb685740 1 -- 172.21.15.103:0/326216431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 msgr2=0x5646409640d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.572+0000 7f8bfb685740 1 --2- 172.21.15.103:0/326216431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x5646409640d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8be8005cd0 tx=0x7f8be80127c0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.572+0000 7f8bfb685740 1 -- 172.21.15.103:0/326216431 shutdown_connections 2024-09-19T06:52:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.572+0000 7f8bfb685740 1 --2- 172.21.15.103:0/326216431 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5646409e9490 0x5646409eb880 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.572+0000 7f8bfb685740 1 --2- 172.21.15.103:0/326216431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564640969cb0 0x564640964610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.572+0000 7f8bd9ffb640 0 client.16758 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.572+0000 7f8bfb685740 1 --2- 172.21.15.103:0/326216431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56464095f410 0x5646409640d0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.572+0000 7f8bfb685740 1 --2- 172.21.15.103:0/326216431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56464095eaf0 0x564640963b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.572+0000 7f8bfb685740 1 -- 172.21.15.103:0/326216431 >> 172.21.15.103:0/326216431 conn(0x56464095b060 msgr2=0x5646409599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.572+0000 7f8bfb685740 1 -- 172.21.15.103:0/326216431 shutdown_connections 2024-09-19T06:52:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.572+0000 7f8bfb685740 1 -- 172.21.15.103:0/326216431 wait complete. 2024-09-19T06:52:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.572+0000 7f8bfb685740 20 client.16758 trim_cache size 0 max 16384 2024-09-19T06:52:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:52:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:52:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:52:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:52:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.572+0000 7f8bf880e640 1 -- 172.21.15.103:0/216700424 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8bdc0a3460 msgr2=0x7f8bdc0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.572+0000 7f8bf880e640 1 --2- 172.21.15.103:0/216700424 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8bdc0a3460 0x7f8bdc0a5920 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8be8005d00 tx=0x7f8be800bb90 comp rx=0 tx=0).stop 2024-09-19T06:52:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.572+0000 7f8bf880e640 1 -- 172.21.15.103:0/216700424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf41069e0 msgr2=0x7f8bf41a4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.572+0000 7f8bf880e640 1 --2- 172.21.15.103:0/216700424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf41069e0 0x7f8bf41a4210 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8bec005cc0 tx=0x7f8bec003970 comp rx=0 tx=0).stop 2024-09-19T06:52:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.573+0000 7f8bf880e640 1 -- 172.21.15.103:0/216700424 shutdown_connections 2024-09-19T06:52:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.573+0000 7f8bf880e640 1 --2- 172.21.15.103:0/216700424 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8bdc0a3460 0x7f8bdc0a5920 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.573+0000 7f8bf880e640 1 --2- 172.21.15.103:0/216700424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bf41069e0 0x7f8bf41a4210 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.573+0000 7f8bf880e640 1 --2- 172.21.15.103:0/216700424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bf4101370 0x7f8bf41996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.573+0000 7f8bf880e640 1 --2- 172.21.15.103:0/216700424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bf41009d0 0x7f8bf4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.573+0000 7f8bf880e640 1 -- 172.21.15.103:0/216700424 >> 172.21.15.103:0/216700424 conn(0x7f8bf40fc6b0 msgr2=0x7f8bf40fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.573+0000 7f8bf880e640 1 -- 172.21.15.103:0/216700424 shutdown_connections 2024-09-19T06:52:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:52:55.573+0000 7f8bf880e640 1 -- 172.21.15.103:0/216700424 wait complete. 2024-09-19T06:52:55.576 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:56.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.014+0000 7fa5428c1640 1 -- 172.21.15.103:0/4089962582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa53c102ef0 msgr2=0x7fa53c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.014+0000 7fa5428c1640 1 --2- 172.21.15.103:0/4089962582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa53c102ef0 0x7fa53c10da40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa534009ca0 tx=0x7fa53402a950 comp rx=0 tx=0).stop 2024-09-19T06:52:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.014+0000 7fa5428c1640 1 -- 172.21.15.103:0/4089962582 shutdown_connections 2024-09-19T06:52:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.014+0000 7fa5428c1640 1 --2- 172.21.15.103:0/4089962582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa53c102ef0 0x7fa53c10da40 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.014+0000 7fa5428c1640 1 --2- 172.21.15.103:0/4089962582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa53c102570 0x7fa53c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.014+0000 7fa5428c1640 1 --2- 172.21.15.103:0/4089962582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa53c100e20 0x7fa53c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.014+0000 7fa5428c1640 1 -- 172.21.15.103:0/4089962582 >> 172.21.15.103:0/4089962582 conn(0x7fa53c0fc7d0 msgr2=0x7fa53c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.014+0000 7fa5428c1640 1 -- 172.21.15.103:0/4089962582 shutdown_connections 2024-09-19T06:52:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.014+0000 7fa5428c1640 1 -- 172.21.15.103:0/4089962582 wait complete. 2024-09-19T06:52:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.014+0000 7fa5428c1640 1 Processor -- start 2024-09-19T06:52:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.014+0000 7fa5428c1640 1 -- start start 2024-09-19T06:52:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.015+0000 7fa5428c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa53c100e20 0x7fa53c1a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.015+0000 7fa5428c1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa53c102570 0x7fa53c1a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.015+0000 7fa5428c1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa53c102ef0 0x7fa53c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.015+0000 7fa5428c1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c110fa0 con 0x7fa53c102570 2024-09-19T06:52:56.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.015+0000 7fa5428c1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c110e20 con 0x7fa53c100e20 2024-09-19T06:52:56.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.015+0000 7fa5428c1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c111120 con 0x7fa53c102ef0 2024-09-19T06:52:56.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.015+0000 7fa53b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa53c102570 0x7fa53c1a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.015+0000 7fa53b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa53c102570 0x7fa53c1a4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39072/0 (socket says 172.21.15.103:39072) 2024-09-19T06:52:56.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.015+0000 7fa53b7fe640 1 -- 172.21.15.103:0/3254122171 learned_addr learned my addr 172.21.15.103:0/3254122171 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:56.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.015+0000 7fa540e5e640 1 --2- 172.21.15.103:0/3254122171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa53c102ef0 0x7fa53c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.015+0000 7fa53bfff640 1 --2- 172.21.15.103:0/3254122171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa53c100e20 0x7fa53c1a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.015+0000 7fa53b7fe640 1 -- 172.21.15.103:0/3254122171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa53c102ef0 msgr2=0x7fa53c19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.015+0000 7fa53b7fe640 1 --2- 172.21.15.103:0/3254122171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa53c102ef0 0x7fa53c19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.015+0000 7fa53b7fe640 1 -- 172.21.15.103:0/3254122171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa53c100e20 msgr2=0x7fa53c1a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.015+0000 7fa53b7fe640 1 --2- 172.21.15.103:0/3254122171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa53c100e20 0x7fa53c1a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.015+0000 7fa53b7fe640 1 -- 172.21.15.103:0/3254122171 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53c19f540 con 0x7fa53c102570 2024-09-19T06:52:56.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.016+0000 7fa53b7fe640 1 --2- 172.21.15.103:0/3254122171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa53c102570 0x7fa53c1a4ce0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa52400c920 tx=0x7fa52400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:56.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.016+0000 7fa5397fa640 1 -- 172.21.15.103:0/3254122171 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524007c70 con 0x7fa53c102570 2024-09-19T06:52:56.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.016+0000 7fa5428c1640 1 -- 172.21.15.103:0/3254122171 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53c19f830 con 0x7fa53c102570 2024-09-19T06:52:56.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.016+0000 7fa5397fa640 1 -- 172.21.15.103:0/3254122171 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa524007e10 con 0x7fa53c102570 2024-09-19T06:52:56.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.016+0000 7fa5397fa640 1 -- 172.21.15.103:0/3254122171 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524018890 con 0x7fa53c102570 2024-09-19T06:52:56.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.016+0000 7fa5428c1640 1 -- 172.21.15.103:0/3254122171 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa53c1ab740 con 0x7fa53c102570 2024-09-19T06:52:56.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.018+0000 7fa5397fa640 1 -- 172.21.15.103:0/3254122171 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa524018a70 con 0x7fa53c102570 2024-09-19T06:52:56.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.018+0000 7fa544c9a740 1 Processor -- start 2024-09-19T06:52:56.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.018+0000 7fa544c9a740 1 -- start start 2024-09-19T06:52:56.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa544c9a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fbb40 0x55eddd4fa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa544c9a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd4fa5d0 0x55eddd4faa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa544c9a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd505370 0x55eddd507760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa544c9a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eddd4602f0 con 0x55eddd505370 2024-09-19T06:52:56.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa544c9a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55eddd407cf0 con 0x55eddd4fa5d0 2024-09-19T06:52:56.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa544c9a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eddd45c2b0 con 0x55eddd4fbb40 2024-09-19T06:52:56.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa53b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd505370 0x55eddd507760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa53b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd505370 0x55eddd507760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39080/0 (socket says 172.21.15.103:39080) 2024-09-19T06:52:56.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa53b7fe640 1 -- 172.21.15.103:0/3034212835 learned_addr learned my addr 172.21.15.103:0/3034212835 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:56.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa5397fa640 1 --2- 172.21.15.103:0/3254122171 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5280a33b0 0x7fa5280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa53bfff640 1 --2- 172.21.15.103:0/3034212835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd4fa5d0 0x55eddd4faa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa540e5e640 1 --2- 172.21.15.103:0/3034212835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fbb40 0x55eddd4fa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa540e5e640 1 --2- 172.21.15.103:0/3254122171 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5280a33b0 0x7fa5280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa5397fa640 1 -- 172.21.15.103:0/3254122171 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa5240cfe60 con 0x7fa53c102570 2024-09-19T06:52:56.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa53b7fe640 1 -- 172.21.15.103:0/3034212835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fbb40 msgr2=0x55eddd4fa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:56.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa53b7fe640 1 --2- 172.21.15.103:0/3034212835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fbb40 0x55eddd4fa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa53b7fe640 1 -- 172.21.15.103:0/3034212835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd4fa5d0 msgr2=0x55eddd4faa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:56.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa53b7fe640 1 --2- 172.21.15.103:0/3034212835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd4fa5d0 0x55eddd4faa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa53b7fe640 1 -- 172.21.15.103:0/3034212835 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eddd3ec3a0 con 0x55eddd505370 2024-09-19T06:52:56.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa53bfff640 1 --2- 172.21.15.103:0/3034212835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd4fa5d0 0x55eddd4faa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:56.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.019+0000 7fa540e5e640 1 --2- 172.21.15.103:0/3034212835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fbb40 0x55eddd4fa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:56.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.020+0000 7fa53b7fe640 1 --2- 172.21.15.103:0/3034212835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd505370 0x55eddd507760 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa5240cd500 tx=0x7fa52400ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f266ae849d19af7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:56.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.020+0000 7fa5327fc640 1 -- 172.21.15.103:0/3034212835 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52409c850 con 0x55eddd505370 2024-09-19T06:52:56.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.020+0000 7fa540e5e640 1 --2- 172.21.15.103:0/3254122171 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5280a33b0 0x7fa5280a5870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa5340098d0 tx=0x7fa53402a950 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:56.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.020+0000 7fa5327fc640 1 -- 172.21.15.103:0/3034212835 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa52409c150 con 0x55eddd505370 2024-09-19T06:52:56.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.020+0000 7fa5327fc640 1 -- 172.21.15.103:0/3034212835 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240bc110 con 0x55eddd505370 2024-09-19T06:52:56.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.020+0000 7fa544c9a740 1 -- 172.21.15.103:0/3034212835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd505370 msgr2=0x55eddd507760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.020+0000 7fa544c9a740 1 --2- 172.21.15.103:0/3034212835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd505370 0x55eddd507760 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa5240cd500 tx=0x7fa52400ff30 comp rx=0 tx=0).stop 2024-09-19T06:52:56.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.021+0000 7fa544c9a740 1 -- 172.21.15.103:0/3034212835 shutdown_connections 2024-09-19T06:52:56.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.021+0000 7fa544c9a740 1 --2- 172.21.15.103:0/3034212835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd505370 0x55eddd507760 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.021+0000 7fa544c9a740 1 --2- 172.21.15.103:0/3034212835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd4fa5d0 0x55eddd4faa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.021+0000 7fa544c9a740 1 --2- 172.21.15.103:0/3034212835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fbb40 0x55eddd4fa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.021+0000 7fa544c9a740 1 -- 172.21.15.103:0/3034212835 >> 172.21.15.103:0/3034212835 conn(0x55eddd4c3310 msgr2=0x55eddd4f6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:56.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.021+0000 7fa544c9a740 1 -- 172.21.15.103:0/3034212835 shutdown_connections 2024-09-19T06:52:56.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.021+0000 7fa544c9a740 1 -- 172.21.15.103:0/3034212835 wait complete. 2024-09-19T06:52:56.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.021+0000 7fa544c9a740 1 -- start start 2024-09-19T06:52:56.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.021+0000 7fa544c9a740 1 Processor -- start 2024-09-19T06:52:56.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.021+0000 7fa544c9a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fa5d0 0x55eddd504d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.021+0000 7fa544c9a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd4fbb40 0x55eddd502010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.021+0000 7fa544c9a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd505370 0x55eddd502600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.021+0000 7fa544c9a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eddd508070 con 0x55eddd4fa5d0 2024-09-19T06:52:56.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.021+0000 7fa544c9a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55eddd41b060 con 0x55eddd4fbb40 2024-09-19T06:52:56.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.021+0000 7fa544c9a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eddd5081f0 con 0x55eddd505370 2024-09-19T06:52:56.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.021+0000 7fa544c9a740 10 client.0 _ll_register_callbacks cb 0x55eddd16e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:56.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fa5d0 0x55eddd504d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fa5d0 0x55eddd504d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39088/0 (socket says 172.21.15.103:39088) 2024-09-19T06:52:56.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa53bfff640 1 -- 172.21.15.103:0/1666573808 learned_addr learned my addr 172.21.15.103:0/1666573808 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:56.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa53b7fe640 1 --2- 172.21.15.103:0/1666573808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd4fbb40 0x55eddd502010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa540e5e640 1 --2- 172.21.15.103:0/1666573808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd505370 0x55eddd502600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa53bfff640 1 -- 172.21.15.103:0/1666573808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd505370 msgr2=0x55eddd502600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:56.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa53bfff640 1 --2- 172.21.15.103:0/1666573808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd505370 0x55eddd502600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa53bfff640 1 -- 172.21.15.103:0/1666573808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd4fbb40 msgr2=0x55eddd502010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:56.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa53bfff640 1 --2- 172.21.15.103:0/1666573808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd4fbb40 0x55eddd502010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa53bfff640 1 -- 172.21.15.103:0/1666573808 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eddd1d13c0 con 0x55eddd4fa5d0 2024-09-19T06:52:56.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa540e5e640 1 --2- 172.21.15.103:0/1666573808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd505370 0x55eddd502600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:56.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa53bfff640 1 --2- 172.21.15.103:0/1666573808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fa5d0 0x55eddd504d60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00bfd0 tx=0x7fa52c00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ebe7c6f7ce8cef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:56.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa51b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:56.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa51b7fe640 1 -- 172.21.15.103:0/1666573808 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c018020 con 0x55eddd4fa5d0 2024-09-19T06:52:56.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa51b7fe640 1 -- 172.21.15.103:0/1666573808 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa52c014870 con 0x55eddd4fa5d0 2024-09-19T06:52:56.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa544c9a740 1 -- 172.21.15.103:0/1666573808 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa52409c150 con 0x55eddd4fa5d0 2024-09-19T06:52:56.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa51b7fe640 1 -- 172.21.15.103:0/1666573808 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa52c023050 con 0x55eddd4fa5d0 2024-09-19T06:52:56.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa544c9a740 10 client.16770 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:56.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.022+0000 7fa544c9a740 1 -- 172.21.15.103:0/1666573808 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eddd513ba0 con 0x55eddd4fa5d0 2024-09-19T06:52:56.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.023+0000 7fa51b7fe640 1 -- 172.21.15.103:0/1666573808 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa52409c150 con 0x55eddd4fa5d0 2024-09-19T06:52:56.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.023+0000 7fa51b7fe640 1 -- 172.21.15.103:0/1666573808 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa52c013af0 con 0x55eddd4fa5d0 2024-09-19T06:52:56.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.023+0000 7fa544c9a740 10 client.16770 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:56.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.023+0000 7fa544c9a740 10 client.16770 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:52:56.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.023+0000 7fa544c9a740 20 client.16770 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:56.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.023+0000 7fa544c9a740 1 --2- 172.21.15.103:0/1666573808 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55eddd515060 0x55eddd517450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.023+0000 7fa544c9a740 4 client.16770 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:56.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.023+0000 7fa544c9a740 1 -- 172.21.15.103:0/1666573808 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eddd46d860 con 0x55eddd515060 2024-09-19T06:52:56.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.023+0000 7fa53b7fe640 1 --2- 172.21.15.103:0/1666573808 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55eddd515060 0x55eddd517450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.023+0000 7fa53b7fe640 1 --2- 172.21.15.103:0/1666573808 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55eddd515060 0x55eddd517450 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aba9f9061992cb02 server_cookie=e142cd634c6cc0d0 in_seq=0 out_seq=0 2024-09-19T06:52:56.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.023+0000 7fa51b7fe640 10 client.16770 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:52:56.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.024+0000 7fa51b7fe640 1 -- 172.21.15.103:0/1666573808 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eddd46d860 con 0x55eddd515060 2024-09-19T06:52:56.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.024+0000 7fa51b7fe640 10 client.16770 handle_command_reply: tid=0 2024-09-19T06:52:56.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.024+0000 7fa544c9a740 1 client.16770 shutdown 2024-09-19T06:52:56.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.024+0000 7fa544c9a740 1 -- 172.21.15.103:0/1666573808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fa5d0 msgr2=0x55eddd504d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.024+0000 7fa544c9a740 1 --2- 172.21.15.103:0/1666573808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fa5d0 0x55eddd504d60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00bfd0 tx=0x7fa52c00f670 comp rx=0 tx=0).stop 2024-09-19T06:52:56.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.024+0000 7fa544c9a740 1 -- 172.21.15.103:0/1666573808 shutdown_connections 2024-09-19T06:52:56.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.024+0000 7fa544c9a740 1 --2- 172.21.15.103:0/1666573808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd505370 0x55eddd502600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.024+0000 7fa544c9a740 1 --2- 172.21.15.103:0/1666573808 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55eddd515060 0x55eddd517450 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.024+0000 7fa544c9a740 1 --2- 172.21.15.103:0/1666573808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd4fbb40 0x55eddd502010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.024+0000 7fa51b7fe640 0 client.16770 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:52:56.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.024+0000 7fa544c9a740 1 --2- 172.21.15.103:0/1666573808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fa5d0 0x55eddd504d60 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.024+0000 7fa544c9a740 1 -- 172.21.15.103:0/1666573808 >> 172.21.15.103:0/1666573808 conn(0x55eddd4c3310 msgr2=0x55eddd4fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:56.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.024+0000 7fa544c9a740 1 -- 172.21.15.103:0/1666573808 shutdown_connections 2024-09-19T06:52:56.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.024+0000 7fa544c9a740 1 -- 172.21.15.103:0/1666573808 wait complete. 2024-09-19T06:52:56.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.025+0000 7fa544c9a740 20 client.16770 trim_cache size 0 max 16384 2024-09-19T06:52:56.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa544c9a740 1 Processor -- start 2024-09-19T06:52:56.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa544c9a740 1 -- start start 2024-09-19T06:52:56.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa544c9a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fa5d0 0x55eddd511340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa544c9a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fbb40 0x55eddd503f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa544c9a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd505370 0x55eddd51ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa544c9a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1b3e50 con 0x55eddd4fa5d0 2024-09-19T06:52:56.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa544c9a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1b5990 con 0x55eddd505370 2024-09-19T06:52:56.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa544c9a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1b4520 con 0x55eddd4fbb40 2024-09-19T06:52:56.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fa5d0 0x55eddd511340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fa5d0 0x55eddd511340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39100/0 (socket says 172.21.15.103:39100) 2024-09-19T06:52:56.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa53bfff640 1 -- 172.21.15.103:0/3081715282 learned_addr learned my addr 172.21.15.103:0/3081715282 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:56.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa53b7fe640 1 --2- 172.21.15.103:0/3081715282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fbb40 0x55eddd503f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa540e5e640 1 --2- 172.21.15.103:0/3081715282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd505370 0x55eddd51ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa53bfff640 1 -- 172.21.15.103:0/3081715282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fbb40 msgr2=0x55eddd503f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:56.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa53bfff640 1 --2- 172.21.15.103:0/3081715282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fbb40 0x55eddd503f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa53bfff640 1 -- 172.21.15.103:0/3081715282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd505370 msgr2=0x55eddd51ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:56.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa53bfff640 1 --2- 172.21.15.103:0/3081715282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd505370 0x55eddd51ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa53bfff640 1 -- 172.21.15.103:0/3081715282 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eddd4f7960 con 0x55eddd4fa5d0 2024-09-19T06:52:56.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.038+0000 7fa53bfff640 1 --2- 172.21.15.103:0/3081715282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fa5d0 0x55eddd511340 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fa52c000c00 tx=0x7fa52c004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fe7fa938e4f8a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:56.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.039+0000 7fa531ffb640 1 -- 172.21.15.103:0/3081715282 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c018040 con 0x55eddd4fa5d0 2024-09-19T06:52:56.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.039+0000 7fa531ffb640 1 -- 172.21.15.103:0/3081715282 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa52c00fb70 con 0x55eddd4fa5d0 2024-09-19T06:52:56.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.039+0000 7fa531ffb640 1 -- 172.21.15.103:0/3081715282 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c01ca70 con 0x55eddd4fa5d0 2024-09-19T06:52:56.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.040+0000 7fa544c9a740 1 -- 172.21.15.103:0/3081715282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fa5d0 msgr2=0x55eddd511340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.040+0000 7fa544c9a740 1 --2- 172.21.15.103:0/3081715282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fa5d0 0x55eddd511340 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fa52c000c00 tx=0x7fa52c004640 comp rx=0 tx=0).stop 2024-09-19T06:52:56.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.040+0000 7fa544c9a740 1 -- 172.21.15.103:0/3081715282 shutdown_connections 2024-09-19T06:52:56.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.040+0000 7fa544c9a740 1 --2- 172.21.15.103:0/3081715282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd505370 0x55eddd51ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.040+0000 7fa544c9a740 1 --2- 172.21.15.103:0/3081715282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fbb40 0x55eddd503f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.040+0000 7fa544c9a740 1 --2- 172.21.15.103:0/3081715282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fa5d0 0x55eddd511340 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.040+0000 7fa544c9a740 1 -- 172.21.15.103:0/3081715282 >> 172.21.15.103:0/3081715282 conn(0x55eddd4c3310 msgr2=0x55eddd503870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:56.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.040+0000 7fa544c9a740 1 -- 172.21.15.103:0/3081715282 shutdown_connections 2024-09-19T06:52:56.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.040+0000 7fa544c9a740 1 -- 172.21.15.103:0/3081715282 wait complete. 2024-09-19T06:52:56.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.040+0000 7fa544c9a740 1 -- start start 2024-09-19T06:52:56.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.041+0000 7fa544c9a740 1 Processor -- start 2024-09-19T06:52:56.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.041+0000 7fa544c9a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fa5d0 0x55eddd509ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.041+0000 7fa544c9a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fbb40 0x55eddd50a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.041+0000 7fa544c9a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd505370 0x55eddd578d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.041+0000 7fa544c9a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1b52c0 con 0x55eddd4fbb40 2024-09-19T06:52:56.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.041+0000 7fa544c9a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1b4bf0 con 0x55eddd505370 2024-09-19T06:52:56.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.041+0000 7fa544c9a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1b1e10 con 0x55eddd4fa5d0 2024-09-19T06:52:56.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.041+0000 7fa544c9a740 10 client.0 _ll_register_callbacks cb 0x7fa52c00b2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:56.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.041+0000 7fa53b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fbb40 0x55eddd50a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.041+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fa5d0 0x55eddd509ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.041+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fa5d0 0x55eddd509ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:54774/0 (socket says 172.21.15.103:54774) 2024-09-19T06:52:56.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.041+0000 7fa53b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fbb40 0x55eddd50a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39106/0 (socket says 172.21.15.103:39106) 2024-09-19T06:52:56.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.041+0000 7fa53b7fe640 1 -- 172.21.15.103:0/2771317897 learned_addr learned my addr 172.21.15.103:0/2771317897 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:56.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.041+0000 7fa540e5e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd505370 0x55eddd578d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.042+0000 7fa53b7fe640 1 -- 172.21.15.103:0/2771317897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fa5d0 msgr2=0x55eddd509ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:56.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.042+0000 7fa53b7fe640 1 --2- 172.21.15.103:0/2771317897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fa5d0 0x55eddd509ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.042+0000 7fa53b7fe640 1 -- 172.21.15.103:0/2771317897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd505370 msgr2=0x55eddd578d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:56.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.042+0000 7fa53b7fe640 1 --2- 172.21.15.103:0/2771317897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd505370 0x55eddd578d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.042+0000 7fa53b7fe640 1 -- 172.21.15.103:0/2771317897 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eddd579340 con 0x55eddd4fbb40 2024-09-19T06:52:56.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.042+0000 7fa53bfff640 1 --2- 172.21.15.103:0/2771317897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fa5d0 0x55eddd509ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:56.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.042+0000 7fa53b7fe640 1 --2- 172.21.15.103:0/2771317897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fbb40 0x55eddd50a000 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa524000c00 tx=0x7fa5240d0130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974f5afdd882ae2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:56.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.042+0000 7fa51b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:56.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.042+0000 7fa51b7fe640 1 -- 172.21.15.103:0/2771317897 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240d3a40 con 0x55eddd4fbb40 2024-09-19T06:52:56.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.042+0000 7fa51b7fe640 1 -- 172.21.15.103:0/2771317897 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa52400fb30 con 0x55eddd4fbb40 2024-09-19T06:52:56.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.042+0000 7fa544c9a740 1 -- 172.21.15.103:0/2771317897 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa52c00fb70 con 0x55eddd4fbb40 2024-09-19T06:52:56.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.042+0000 7fa51b7fe640 1 -- 172.21.15.103:0/2771317897 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa524066050 con 0x55eddd4fbb40 2024-09-19T06:52:56.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.042+0000 7fa544c9a740 10 client.16776 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:56.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.042+0000 7fa544c9a740 1 -- 172.21.15.103:0/2771317897 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eddd5806f0 con 0x55eddd4fbb40 2024-09-19T06:52:56.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.042+0000 7fa51b7fe640 1 -- 172.21.15.103:0/2771317897 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa52409c850 con 0x55eddd4fbb40 2024-09-19T06:52:56.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.043+0000 7fa51b7fe640 1 -- 172.21.15.103:0/2771317897 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa5240987c0 con 0x55eddd4fbb40 2024-09-19T06:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.043+0000 7fa544c9a740 10 client.16776 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.043+0000 7fa544c9a740 10 client.16776 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.043+0000 7fa544c9a740 20 client.16776 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.043+0000 7fa544c9a740 1 --2- 172.21.15.103:0/2771317897 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55eddd515060 0x55eddd582990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.043+0000 7fa544c9a740 4 client.16776 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.043+0000 7fa544c9a740 1 -- 172.21.15.103:0/2771317897 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa52c00fb70 con 0x55eddd515060 2024-09-19T06:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.043+0000 7fa53bfff640 1 --2- 172.21.15.103:0/2771317897 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55eddd515060 0x55eddd582990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.043+0000 7fa53bfff640 1 --2- 172.21.15.103:0/2771317897 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55eddd515060 0x55eddd582990 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e6f69f36f6c6e9f2 server_cookie=438b2b533d674adf in_seq=0 out_seq=0 2024-09-19T06:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.043+0000 7fa51b7fe640 10 client.16776 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.043+0000 7fa51b7fe640 1 -- 172.21.15.103:0/2771317897 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa52c00fb70 con 0x55eddd515060 2024-09-19T06:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.044+0000 7fa51b7fe640 10 client.16776 handle_command_reply: tid=0 2024-09-19T06:52:56.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.044+0000 7fa544c9a740 1 client.16776 shutdown 2024-09-19T06:52:56.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.044+0000 7fa544c9a740 1 -- 172.21.15.103:0/2771317897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fbb40 msgr2=0x55eddd50a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.044+0000 7fa544c9a740 1 --2- 172.21.15.103:0/2771317897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fbb40 0x55eddd50a000 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa524000c00 tx=0x7fa5240d0130 comp rx=0 tx=0).stop 2024-09-19T06:52:56.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.044+0000 7fa544c9a740 1 -- 172.21.15.103:0/2771317897 shutdown_connections 2024-09-19T06:52:56.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.044+0000 7fa544c9a740 1 --2- 172.21.15.103:0/2771317897 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55eddd515060 0x55eddd582990 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.044+0000 7fa544c9a740 1 --2- 172.21.15.103:0/2771317897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd505370 0x55eddd578d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.044+0000 7fa544c9a740 1 --2- 172.21.15.103:0/2771317897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd4fbb40 0x55eddd50a000 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.044+0000 7fa544c9a740 1 --2- 172.21.15.103:0/2771317897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd4fa5d0 0x55eddd509ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.044+0000 7fa544c9a740 1 -- 172.21.15.103:0/2771317897 >> 172.21.15.103:0/2771317897 conn(0x55eddd4c3310 msgr2=0x55eddd504090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:56.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.044+0000 7fa51b7fe640 0 client.16776 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:52:56.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.044+0000 7fa544c9a740 1 -- 172.21.15.103:0/2771317897 shutdown_connections 2024-09-19T06:52:56.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.044+0000 7fa544c9a740 1 -- 172.21.15.103:0/2771317897 wait complete. 2024-09-19T06:52:56.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.044+0000 7fa544c9a740 20 client.16776 trim_cache size 0 max 16384 2024-09-19T06:52:56.063 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:52:56.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.045+0000 7fa5428c1640 1 -- 172.21.15.103:0/3254122171 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5280a33b0 msgr2=0x7fa5280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.045+0000 7fa5428c1640 1 --2- 172.21.15.103:0/3254122171 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5280a33b0 0x7fa5280a5870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa5340098d0 tx=0x7fa53402a950 comp rx=0 tx=0).stop 2024-09-19T06:52:56.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.045+0000 7fa5428c1640 1 -- 172.21.15.103:0/3254122171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa53c102570 msgr2=0x7fa53c1a4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.045+0000 7fa5428c1640 1 --2- 172.21.15.103:0/3254122171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa53c102570 0x7fa53c1a4ce0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa52400c920 tx=0x7fa52400cde0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.045+0000 7fa5428c1640 1 -- 172.21.15.103:0/3254122171 shutdown_connections 2024-09-19T06:52:56.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.045+0000 7fa5428c1640 1 --2- 172.21.15.103:0/3254122171 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5280a33b0 0x7fa5280a5870 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.045+0000 7fa5428c1640 1 --2- 172.21.15.103:0/3254122171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa53c102ef0 0x7fa53c19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.045+0000 7fa5428c1640 1 --2- 172.21.15.103:0/3254122171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa53c102570 0x7fa53c1a4ce0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.045+0000 7fa5428c1640 1 --2- 172.21.15.103:0/3254122171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa53c100e20 0x7fa53c1a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.045+0000 7fa5428c1640 1 -- 172.21.15.103:0/3254122171 >> 172.21.15.103:0/3254122171 conn(0x7fa53c0fc7d0 msgr2=0x7fa53c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:56.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.045+0000 7fa5428c1640 1 -- 172.21.15.103:0/3254122171 shutdown_connections 2024-09-19T06:52:56.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:56.045+0000 7fa5428c1640 1 -- 172.21.15.103:0/3254122171 wait complete. 2024-09-19T06:52:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:56 smithi174 ceph-mon[35637]: pgmap v114: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-19T06:52:56.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:56 smithi103 ceph-mon[31158]: pgmap v114: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-19T06:52:56.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:56 smithi134 ceph-mon[35356]: pgmap v114: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-19T06:52:56.716 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:52:56.962 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:57.327 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:52:57.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.374+0000 7f55e71de640 1 -- 172.21.15.103:0/2601522032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e0100e20 msgr2=0x7f55e0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:57.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.374+0000 7f55e71de640 1 --2- 172.21.15.103:0/2601522032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e0100e20 0x7f55e0101200 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009900 tx=0x7f55d802a480 comp rx=0 tx=0).stop 2024-09-19T06:52:57.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.374+0000 7f55e71de640 1 -- 172.21.15.103:0/2601522032 shutdown_connections 2024-09-19T06:52:57.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.374+0000 7f55e71de640 1 --2- 172.21.15.103:0/2601522032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55e0102ef0 0x7f55e010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.374+0000 7f55e71de640 1 --2- 172.21.15.103:0/2601522032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55e0102570 0x7f55e01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.374+0000 7f55e71de640 1 --2- 172.21.15.103:0/2601522032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e0100e20 0x7f55e0101200 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.374+0000 7f55e71de640 1 -- 172.21.15.103:0/2601522032 >> 172.21.15.103:0/2601522032 conn(0x7f55e00fc7d0 msgr2=0x7f55e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:57.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.374+0000 7f55e71de640 1 -- 172.21.15.103:0/2601522032 shutdown_connections 2024-09-19T06:52:57.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.374+0000 7f55e71de640 1 -- 172.21.15.103:0/2601522032 wait complete. 2024-09-19T06:52:57.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.374+0000 7f55e71de640 1 Processor -- start 2024-09-19T06:52:57.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.374+0000 7f55e71de640 1 -- start start 2024-09-19T06:52:57.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e71de640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e0100e20 0x7f55e0197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e71de640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55e0102570 0x7f55e0197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e71de640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55e0102ef0 0x7f55e019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e71de640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55e0110f50 con 0x7f55e0100e20 2024-09-19T06:52:57.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e71de640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55e0110dd0 con 0x7f55e0102570 2024-09-19T06:52:57.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e71de640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01110d0 con 0x7f55e0102ef0 2024-09-19T06:52:57.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e4f7a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e0100e20 0x7f55e0197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e4f7a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e0100e20 0x7f55e0197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33152/0 (socket says 172.21.15.103:33152) 2024-09-19T06:52:57.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e4f7a640 1 -- 172.21.15.103:0/1831283245 learned_addr learned my addr 172.21.15.103:0/1831283245 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:57.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e577b640 1 --2- 172.21.15.103:0/1831283245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55e0102ef0 0x7f55e019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55dffff640 1 --2- 172.21.15.103:0/1831283245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55e0102570 0x7f55e0197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e4f7a640 1 -- 172.21.15.103:0/1831283245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55e0102ef0 msgr2=0x7f55e019bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:57.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e4f7a640 1 --2- 172.21.15.103:0/1831283245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55e0102ef0 0x7f55e019bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e4f7a640 1 -- 172.21.15.103:0/1831283245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55e0102570 msgr2=0x7f55e0197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:57.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e4f7a640 1 --2- 172.21.15.103:0/1831283245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55e0102570 0x7f55e0197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e4f7a640 1 -- 172.21.15.103:0/1831283245 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e019c400 con 0x7f55e0100e20 2024-09-19T06:52:57.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55dffff640 1 --2- 172.21.15.103:0/1831283245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55e0102570 0x7f55e0197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:57.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e4f7a640 1 --2- 172.21.15.103:0/1831283245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e0100e20 0x7f55e0197450 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f55d8006010 tx=0x7f55d8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:57.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55ddffb640 1 -- 172.21.15.103:0/1831283245 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d803f070 con 0x7f55e0100e20 2024-09-19T06:52:57.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55ddffb640 1 -- 172.21.15.103:0/1831283245 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55d8031c30 con 0x7f55e0100e20 2024-09-19T06:52:57.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55e71de640 1 -- 172.21.15.103:0/1831283245 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e019c690 con 0x7f55e0100e20 2024-09-19T06:52:57.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.375+0000 7f55ddffb640 1 -- 172.21.15.103:0/1831283245 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8035030 con 0x7f55e0100e20 2024-09-19T06:52:57.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.376+0000 7f55e71de640 1 -- 172.21.15.103:0/1831283245 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e01a3fd0 con 0x7f55e0100e20 2024-09-19T06:52:57.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.378+0000 7f55ddffb640 1 -- 172.21.15.103:0/1831283245 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f55d8042070 con 0x7f55e0100e20 2024-09-19T06:52:57.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.378+0000 7f55e95b7740 1 Processor -- start 2024-09-19T06:52:57.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.378+0000 7f55e95b7740 1 -- start start 2024-09-19T06:52:57.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55e95b7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 0x55a30d7fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55e95b7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d7fd5d0 0x55a30d7fda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55e95b7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d808370 0x55a30d80a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55e95b7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a30d7632f0 con 0x55a30d7fd5d0 2024-09-19T06:52:57.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55e95b7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a30d70acf0 con 0x55a30d808370 2024-09-19T06:52:57.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55e95b7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a30d75f2b0 con 0x55a30d7feb40 2024-09-19T06:52:57.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55e4f7a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d7fd5d0 0x55a30d7fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55e577b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 0x55a30d7fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55e4f7a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d7fd5d0 0x55a30d7fda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33162/0 (socket says 172.21.15.103:33162) 2024-09-19T06:52:57.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55e4f7a640 1 -- 172.21.15.103:0/1917827700 learned_addr learned my addr 172.21.15.103:0/1917827700 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:57.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55ddffb640 1 --2- 172.21.15.103:0/1831283245 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55cc0a33b0 0x7f55cc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55dffff640 1 --2- 172.21.15.103:0/1917827700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d808370 0x55a30d80a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55e577b640 1 --2- 172.21.15.103:0/1831283245 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55cc0a33b0 0x7f55cc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55e577b640 1 -- 172.21.15.103:0/1917827700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d808370 msgr2=0x55a30d80a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:57.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55e577b640 1 --2- 172.21.15.103:0/1917827700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d808370 0x55a30d80a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55e577b640 1 -- 172.21.15.103:0/1917827700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d7fd5d0 msgr2=0x55a30d7fda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:57.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55e577b640 1 --2- 172.21.15.103:0/1917827700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d7fd5d0 0x55a30d7fda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55e577b640 1 -- 172.21.15.103:0/1917827700 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a30d6ef3a0 con 0x55a30d7feb40 2024-09-19T06:52:57.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.379+0000 7f55ddffb640 1 -- 172.21.15.103:0/1831283245 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f55d80f0600 con 0x7f55e0100e20 2024-09-19T06:52:57.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.380+0000 7f55e577b640 1 --2- 172.21.15.103:0/1917827700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 0x55a30d7fd090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f55c800bc30 tx=0x7f55c80039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32d727284b3bc2a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:57.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.380+0000 7f55e577b640 1 --2- 172.21.15.103:0/1831283245 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55cc0a33b0 0x7f55cc0a5870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f55c801b000 tx=0x7f55c801d000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:57.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.380+0000 7f55d6ffd640 1 -- 172.21.15.103:0/1917827700 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8012e60 con 0x55a30d7feb40 2024-09-19T06:52:57.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.380+0000 7f55d6ffd640 1 -- 172.21.15.103:0/1917827700 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55c80047a0 con 0x55a30d7feb40 2024-09-19T06:52:57.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.380+0000 7f55d6ffd640 1 -- 172.21.15.103:0/1917827700 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8010680 con 0x55a30d7feb40 2024-09-19T06:52:57.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.381+0000 7f55e95b7740 1 -- 172.21.15.103:0/1917827700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 msgr2=0x55a30d7fd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:57.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.381+0000 7f55e95b7740 1 --2- 172.21.15.103:0/1917827700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 0x55a30d7fd090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f55c800bc30 tx=0x7f55c80039c0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.381+0000 7f55e95b7740 1 -- 172.21.15.103:0/1917827700 shutdown_connections 2024-09-19T06:52:57.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.381+0000 7f55e95b7740 1 --2- 172.21.15.103:0/1917827700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d808370 0x55a30d80a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.381+0000 7f55e95b7740 1 --2- 172.21.15.103:0/1917827700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d7fd5d0 0x55a30d7fda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.381+0000 7f55e95b7740 1 --2- 172.21.15.103:0/1917827700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 0x55a30d7fd090 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.381+0000 7f55e95b7740 1 -- 172.21.15.103:0/1917827700 >> 172.21.15.103:0/1917827700 conn(0x55a30d7c6310 msgr2=0x55a30d7f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:57.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.381+0000 7f55e95b7740 1 -- 172.21.15.103:0/1917827700 shutdown_connections 2024-09-19T06:52:57.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.381+0000 7f55e95b7740 1 -- 172.21.15.103:0/1917827700 wait complete. 2024-09-19T06:52:57.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.381+0000 7f55e95b7740 1 -- start start 2024-09-19T06:52:57.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.381+0000 7f55e95b7740 1 Processor -- start 2024-09-19T06:52:57.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e95b7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d7fd5d0 0x55a30d807dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e95b7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 0x55a30d805070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e95b7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d808370 0x55a30d805800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e95b7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a30d80d1c0 con 0x55a30d808370 2024-09-19T06:52:57.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e95b7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a30d71e060 con 0x55a30d7fd5d0 2024-09-19T06:52:57.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e95b7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a30d80d340 con 0x55a30d7feb40 2024-09-19T06:52:57.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e95b7740 10 client.0 _ll_register_callbacks cb 0x55a30d4718c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:57.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e577b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d808370 0x55a30d805800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e577b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d808370 0x55a30d805800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33174/0 (socket says 172.21.15.103:33174) 2024-09-19T06:52:57.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e577b640 1 -- 172.21.15.103:0/1373160309 learned_addr learned my addr 172.21.15.103:0/1373160309 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:57.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55dffff640 1 --2- 172.21.15.103:0/1373160309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 0x55a30d805070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e4f7a640 1 --2- 172.21.15.103:0/1373160309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d7fd5d0 0x55a30d807dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e577b640 1 -- 172.21.15.103:0/1373160309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 msgr2=0x55a30d805070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:57.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e577b640 1 --2- 172.21.15.103:0/1373160309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 0x55a30d805070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e577b640 1 -- 172.21.15.103:0/1373160309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d7fd5d0 msgr2=0x55a30d807dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:57.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e577b640 1 --2- 172.21.15.103:0/1373160309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d7fd5d0 0x55a30d807dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e577b640 1 -- 172.21.15.103:0/1373160309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a30d4d43c0 con 0x55a30d808370 2024-09-19T06:52:57.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55dffff640 1 --2- 172.21.15.103:0/1373160309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 0x55a30d805070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:57.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e577b640 1 --2- 172.21.15.103:0/1373160309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d808370 0x55a30d805800 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f55c80059e0 tx=0x7f55c8010420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ee994043661169 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:57.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55bffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:57.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55bffff640 1 -- 172.21.15.103:0/1373160309 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c802b020 con 0x55a30d808370 2024-09-19T06:52:57.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55bffff640 1 -- 172.21.15.103:0/1373160309 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f55c8004b50 con 0x55a30d808370 2024-09-19T06:52:57.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.382+0000 7f55e95b7740 1 -- 172.21.15.103:0/1373160309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55c80047a0 con 0x55a30d808370 2024-09-19T06:52:57.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.383+0000 7f55bffff640 1 -- 172.21.15.103:0/1373160309 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f55c8036050 con 0x55a30d808370 2024-09-19T06:52:57.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.383+0000 7f55e95b7740 10 client.16788 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:57.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.383+0000 7f55e95b7740 1 -- 172.21.15.103:0/1373160309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a30d818d20 con 0x55a30d808370 2024-09-19T06:52:57.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.383+0000 7f55bffff640 1 -- 172.21.15.103:0/1373160309 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55c80047a0 con 0x55a30d808370 2024-09-19T06:52:57.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.383+0000 7f55bffff640 1 -- 172.21.15.103:0/1373160309 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f55c8027460 con 0x55a30d808370 2024-09-19T06:52:57.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.383+0000 7f55e95b7740 10 client.16788 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:57.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.383+0000 7f55e95b7740 10 client.16788 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:52:57.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.383+0000 7f55e95b7740 20 client.16788 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:57.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.383+0000 7f55e95b7740 1 --2- 172.21.15.103:0/1373160309 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55a30d81a1e0 0x55a30d81c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.383+0000 7f55e95b7740 4 client.16788 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:57.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.383+0000 7f55e95b7740 1 -- 172.21.15.103:0/1373160309 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a30d770860 con 0x55a30d81a1e0 2024-09-19T06:52:57.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.383+0000 7f55e4f7a640 1 --2- 172.21.15.103:0/1373160309 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55a30d81a1e0 0x55a30d81c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.384+0000 7f55bffff640 10 client.16788 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:52:57.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.384+0000 7f55e4f7a640 1 --2- 172.21.15.103:0/1373160309 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55a30d81a1e0 0x55a30d81c5d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc077ce4cf166952 server_cookie=118311d8a20545b2 in_seq=0 out_seq=0 2024-09-19T06:52:57.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.385+0000 7f55bffff640 1 -- 172.21.15.103:0/1373160309 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a30d770860 con 0x55a30d81a1e0 2024-09-19T06:52:57.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.385+0000 7f55bffff640 10 client.16788 handle_command_reply: tid=0 2024-09-19T06:52:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.385+0000 7f55e95b7740 1 client.16788 shutdown 2024-09-19T06:52:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.385+0000 7f55e95b7740 1 -- 172.21.15.103:0/1373160309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d808370 msgr2=0x55a30d805800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.385+0000 7f55e95b7740 1 --2- 172.21.15.103:0/1373160309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d808370 0x55a30d805800 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f55c80059e0 tx=0x7f55c8010420 comp rx=0 tx=0).stop 2024-09-19T06:52:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.385+0000 7f55e95b7740 1 -- 172.21.15.103:0/1373160309 shutdown_connections 2024-09-19T06:52:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.385+0000 7f55e95b7740 1 --2- 172.21.15.103:0/1373160309 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55a30d81a1e0 0x55a30d81c5d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.385+0000 7f55e95b7740 1 --2- 172.21.15.103:0/1373160309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d808370 0x55a30d805800 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.385+0000 7f55e95b7740 1 --2- 172.21.15.103:0/1373160309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 0x55a30d805070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.385+0000 7f55e95b7740 1 --2- 172.21.15.103:0/1373160309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d7fd5d0 0x55a30d807dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.385+0000 7f55bffff640 0 client.16788 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:52:57.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.385+0000 7f55e95b7740 1 -- 172.21.15.103:0/1373160309 >> 172.21.15.103:0/1373160309 conn(0x55a30d7c6310 msgr2=0x55a30d7ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:57.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.385+0000 7f55e95b7740 1 -- 172.21.15.103:0/1373160309 shutdown_connections 2024-09-19T06:52:57.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.385+0000 7f55e95b7740 1 -- 172.21.15.103:0/1373160309 wait complete. 2024-09-19T06:52:57.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.385+0000 7f55e95b7740 20 client.16788 trim_cache size 0 max 16384 2024-09-19T06:52:57.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.398+0000 7f55e95b7740 1 Processor -- start 2024-09-19T06:52:57.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.398+0000 7f55e95b7740 1 -- start start 2024-09-19T06:52:57.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.398+0000 7f55e95b7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d7fd5d0 0x55a30d805780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.398+0000 7f55e95b7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d7feb40 0x55a30d7f9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.398+0000 7f55e95b7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d808370 0x55a30d800120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.398+0000 7f55e95b7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01ab610 con 0x55a30d7feb40 2024-09-19T06:52:57.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.398+0000 7f55e95b7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01ad150 con 0x55a30d7fd5d0 2024-09-19T06:52:57.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.398+0000 7f55e95b7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01abce0 con 0x55a30d808370 2024-09-19T06:52:57.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.398+0000 7f55dffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d7feb40 0x55a30d7f9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.398+0000 7f55dffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d7feb40 0x55a30d7f9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33178/0 (socket says 172.21.15.103:33178) 2024-09-19T06:52:57.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.398+0000 7f55dffff640 1 -- 172.21.15.103:0/2997281246 learned_addr learned my addr 172.21.15.103:0/2997281246 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:57.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.398+0000 7f55e577b640 1 --2- 172.21.15.103:0/2997281246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d808370 0x55a30d800120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.399+0000 7f55e4f7a640 1 --2- 172.21.15.103:0/2997281246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d7fd5d0 0x55a30d805780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.399+0000 7f55dffff640 1 -- 172.21.15.103:0/2997281246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d808370 msgr2=0x55a30d800120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:57.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.399+0000 7f55dffff640 1 --2- 172.21.15.103:0/2997281246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d808370 0x55a30d800120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.399+0000 7f55dffff640 1 -- 172.21.15.103:0/2997281246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d7fd5d0 msgr2=0x55a30d805780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:57.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.399+0000 7f55dffff640 1 --2- 172.21.15.103:0/2997281246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d7fd5d0 0x55a30d805780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.399+0000 7f55dffff640 1 -- 172.21.15.103:0/2997281246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a30d81c940 con 0x55a30d7feb40 2024-09-19T06:52:57.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.399+0000 7f55dffff640 1 --2- 172.21.15.103:0/2997281246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d7feb40 0x55a30d7f9d70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f55d0005fd0 tx=0x7f55d00128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8211f656cd2455 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:57.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.399+0000 7f55d67fc640 1 -- 172.21.15.103:0/2997281246 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0003a30 con 0x55a30d7feb40 2024-09-19T06:52:57.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.399+0000 7f55d67fc640 1 -- 172.21.15.103:0/2997281246 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55d0003bd0 con 0x55a30d7feb40 2024-09-19T06:52:57.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.399+0000 7f55d67fc640 1 -- 172.21.15.103:0/2997281246 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0010890 con 0x55a30d7feb40 2024-09-19T06:52:57.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.400+0000 7f55e95b7740 1 -- 172.21.15.103:0/2997281246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d7feb40 msgr2=0x55a30d7f9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:57.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.400+0000 7f55e95b7740 1 --2- 172.21.15.103:0/2997281246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d7feb40 0x55a30d7f9d70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f55d0005fd0 tx=0x7f55d00128e0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.400+0000 7f55e95b7740 1 -- 172.21.15.103:0/2997281246 shutdown_connections 2024-09-19T06:52:57.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.400+0000 7f55e95b7740 1 --2- 172.21.15.103:0/2997281246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d808370 0x55a30d800120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.400+0000 7f55e95b7740 1 --2- 172.21.15.103:0/2997281246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d7feb40 0x55a30d7f9d70 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.400+0000 7f55e95b7740 1 --2- 172.21.15.103:0/2997281246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d7fd5d0 0x55a30d805780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.400+0000 7f55e95b7740 1 -- 172.21.15.103:0/2997281246 >> 172.21.15.103:0/2997281246 conn(0x55a30d7c6310 msgr2=0x55a30d8070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:57.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.400+0000 7f55e95b7740 1 -- 172.21.15.103:0/2997281246 shutdown_connections 2024-09-19T06:52:57.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.400+0000 7f55e95b7740 1 -- 172.21.15.103:0/2997281246 wait complete. 2024-09-19T06:52:57.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.400+0000 7f55e95b7740 1 -- start start 2024-09-19T06:52:57.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.400+0000 7f55e95b7740 1 Processor -- start 2024-09-19T06:52:57.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e95b7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d7fd5d0 0x55a30d812b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e95b7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 0x55a30d801c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e95b7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d808370 0x55a30d8021b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e95b7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01aca80 con 0x55a30d808370 2024-09-19T06:52:57.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e95b7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01ac3b0 con 0x55a30d7fd5d0 2024-09-19T06:52:57.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e95b7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01a95d0 con 0x55a30d7feb40 2024-09-19T06:52:57.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e95b7740 10 client.0 _ll_register_callbacks cb 0x7f55d80b84a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:57.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e577b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d808370 0x55a30d8021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e577b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d808370 0x55a30d8021b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33180/0 (socket says 172.21.15.103:33180) 2024-09-19T06:52:57.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e577b640 1 -- 172.21.15.103:0/1968354479 learned_addr learned my addr 172.21.15.103:0/1968354479 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:57.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55dffff640 1 --2- 172.21.15.103:0/1968354479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 0x55a30d801c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e4f7a640 1 --2- 172.21.15.103:0/1968354479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d7fd5d0 0x55a30d812b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e577b640 1 -- 172.21.15.103:0/1968354479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 msgr2=0x55a30d801c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:57.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e577b640 1 --2- 172.21.15.103:0/1968354479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 0x55a30d801c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e577b640 1 -- 172.21.15.103:0/1968354479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d7fd5d0 msgr2=0x55a30d812b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:57.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e577b640 1 --2- 172.21.15.103:0/1968354479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d7fd5d0 0x55a30d812b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e577b640 1 -- 172.21.15.103:0/1968354479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a30d802820 con 0x55a30d808370 2024-09-19T06:52:57.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e4f7a640 1 --2- 172.21.15.103:0/1968354479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d7fd5d0 0x55a30d812b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:57.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55e577b640 1 --2- 172.21.15.103:0/1968354479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d808370 0x55a30d8021b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f55c80041f0 tx=0x7f55c800ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bf6808a5921d98 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:57.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55bffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:57.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.401+0000 7f55bffff640 1 -- 172.21.15.103:0/1968354479 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c802b050 con 0x55a30d808370 2024-09-19T06:52:57.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.402+0000 7f55bffff640 1 -- 172.21.15.103:0/1968354479 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f55c8027c70 con 0x55a30d808370 2024-09-19T06:52:57.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.402+0000 7f55e95b7740 1 -- 172.21.15.103:0/1968354479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d0003bd0 con 0x55a30d808370 2024-09-19T06:52:57.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.402+0000 7f55bffff640 1 -- 172.21.15.103:0/1968354479 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f55c8048050 con 0x55a30d808370 2024-09-19T06:52:57.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.402+0000 7f55e95b7740 10 client.16794 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:57.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.402+0000 7f55e95b7740 1 -- 172.21.15.103:0/1968354479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a30d884c90 con 0x55a30d808370 2024-09-19T06:52:57.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.402+0000 7f55bffff640 1 -- 172.21.15.103:0/1968354479 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55d0003bd0 con 0x55a30d808370 2024-09-19T06:52:57.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.402+0000 7f55bffff640 1 -- 172.21.15.103:0/1968354479 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f55c803a550 con 0x55a30d808370 2024-09-19T06:52:57.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.402+0000 7f55e95b7740 10 client.16794 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:57.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.402+0000 7f55e95b7740 10 client.16794 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:52:57.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.402+0000 7f55e95b7740 20 client.16794 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:57.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.402+0000 7f55e95b7740 1 --2- 172.21.15.103:0/1968354479 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55a30d8853d0 0x55a30d8877f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:57.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.402+0000 7f55e95b7740 4 client.16794 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:52:57.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.402+0000 7f55e95b7740 1 -- 172.21.15.103:0/1968354479 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a30d76ad70 con 0x55a30d8853d0 2024-09-19T06:52:57.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55e4f7a640 1 --2- 172.21.15.103:0/1968354479 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55a30d8853d0 0x55a30d8877f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:57.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55bffff640 10 client.16794 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:52:57.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55e4f7a640 1 --2- 172.21.15.103:0/1968354479 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55a30d8853d0 0x55a30d8877f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af5aa7d114173679 server_cookie=d7725b564bcf17fd in_seq=0 out_seq=0 2024-09-19T06:52:57.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55bffff640 1 -- 172.21.15.103:0/1968354479 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a30d76ad70 con 0x55a30d8853d0 2024-09-19T06:52:57.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55bffff640 10 client.16794 handle_command_reply: tid=0 2024-09-19T06:52:57.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55e95b7740 1 client.16794 shutdown 2024-09-19T06:52:57.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55e95b7740 1 -- 172.21.15.103:0/1968354479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d808370 msgr2=0x55a30d8021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:57.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55e95b7740 1 --2- 172.21.15.103:0/1968354479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d808370 0x55a30d8021b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f55c80041f0 tx=0x7f55c800ba80 comp rx=0 tx=0).stop 2024-09-19T06:52:57.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55e95b7740 1 -- 172.21.15.103:0/1968354479 shutdown_connections 2024-09-19T06:52:57.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55e95b7740 1 --2- 172.21.15.103:0/1968354479 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55a30d8853d0 0x55a30d8877f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55e95b7740 1 --2- 172.21.15.103:0/1968354479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a30d808370 0x55a30d8021b0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55e95b7740 1 --2- 172.21.15.103:0/1968354479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a30d7feb40 0x55a30d801c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55e95b7740 1 --2- 172.21.15.103:0/1968354479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a30d7fd5d0 0x55a30d812b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55e95b7740 1 -- 172.21.15.103:0/1968354479 >> 172.21.15.103:0/1968354479 conn(0x55a30d7c6310 msgr2=0x55a30d815450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:57.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55bffff640 0 client.16794 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:52:57.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55e95b7740 1 -- 172.21.15.103:0/1968354479 shutdown_connections 2024-09-19T06:52:57.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.403+0000 7f55e95b7740 1 -- 172.21.15.103:0/1968354479 wait complete. 2024-09-19T06:52:57.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.404+0000 7f55e95b7740 20 client.16794 trim_cache size 0 max 16384 2024-09-19T06:52:57.418 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:52:57.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.404+0000 7f55e71de640 1 -- 172.21.15.103:0/1831283245 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55cc0a33b0 msgr2=0x7f55cc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:57.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.404+0000 7f55e71de640 1 --2- 172.21.15.103:0/1831283245 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55cc0a33b0 0x7f55cc0a5870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f55c801b000 tx=0x7f55c801d000 comp rx=0 tx=0).stop 2024-09-19T06:52:57.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.404+0000 7f55e71de640 1 -- 172.21.15.103:0/1831283245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e0100e20 msgr2=0x7f55e0197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:57.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.404+0000 7f55e71de640 1 --2- 172.21.15.103:0/1831283245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e0100e20 0x7f55e0197450 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f55d8006010 tx=0x7f55d8003c10 comp rx=0 tx=0).stop 2024-09-19T06:52:57.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.404+0000 7f55e71de640 1 -- 172.21.15.103:0/1831283245 shutdown_connections 2024-09-19T06:52:57.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.404+0000 7f55e71de640 1 --2- 172.21.15.103:0/1831283245 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55cc0a33b0 0x7f55cc0a5870 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.404+0000 7f55e71de640 1 --2- 172.21.15.103:0/1831283245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55e0102ef0 0x7f55e019bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.404+0000 7f55e71de640 1 --2- 172.21.15.103:0/1831283245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55e0102570 0x7f55e0197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.404+0000 7f55e71de640 1 --2- 172.21.15.103:0/1831283245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e0100e20 0x7f55e0197450 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:57.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.404+0000 7f55e71de640 1 -- 172.21.15.103:0/1831283245 >> 172.21.15.103:0/1831283245 conn(0x7f55e00fc7d0 msgr2=0x7f55e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:57.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.404+0000 7f55e71de640 1 -- 172.21.15.103:0/1831283245 shutdown_connections 2024-09-19T06:52:57.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:57.404+0000 7f55e71de640 1 -- 172.21.15.103:0/1831283245 wait complete. 2024-09-19T06:52:58.174 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:52:58.223 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:58.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:58 smithi174 ceph-mon[35637]: pgmap v115: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-09-19T06:52:58.344 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:58 smithi103 ceph-mon[31158]: pgmap v115: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-09-19T06:52:58.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:58 smithi134 ceph-mon[35356]: pgmap v115: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-09-19T06:52:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.666+0000 7f0de0796640 1 -- 172.21.15.103:0/2900535373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0dd8102790 msgr2=0x7f0dd81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.666+0000 7f0de0796640 1 --2- 172.21.15.103:0/2900535373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0dd8102790 0x7f0dd81067e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4009900 tx=0x7f0dd402a410 comp rx=0 tx=0).stop 2024-09-19T06:52:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.666+0000 7f0de0796640 1 -- 172.21.15.103:0/2900535373 shutdown_connections 2024-09-19T06:52:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.666+0000 7f0de0796640 1 --2- 172.21.15.103:0/2900535373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dd8106f10 0x7f0dd810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.666+0000 7f0de0796640 1 --2- 172.21.15.103:0/2900535373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0dd8102790 0x7f0dd81067e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.666+0000 7f0de0796640 1 --2- 172.21.15.103:0/2900535373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dd8101de0 0x7f0dd81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.666+0000 7f0de0796640 1 -- 172.21.15.103:0/2900535373 >> 172.21.15.103:0/2900535373 conn(0x7f0dd80fd8c0 msgr2=0x7f0dd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:58.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.666+0000 7f0de0796640 1 -- 172.21.15.103:0/2900535373 shutdown_connections 2024-09-19T06:52:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.666+0000 7f0de0796640 1 -- 172.21.15.103:0/2900535373 wait complete. 2024-09-19T06:52:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0de0796640 1 Processor -- start 2024-09-19T06:52:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0de0796640 1 -- start start 2024-09-19T06:52:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0de0796640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0dd8101de0 0x7f0dd81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0de0796640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dd8102790 0x7f0dd8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0de0796640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dd8106f10 0x7f0dd819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0de0796640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd8112010 con 0x7f0dd8101de0 2024-09-19T06:52:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0de0796640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd8111e90 con 0x7f0dd8106f10 2024-09-19T06:52:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0de0796640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd8112190 con 0x7f0dd8102790 2024-09-19T06:52:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0dde532640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0dd8101de0 0x7f0dd81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0dde532640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0dd8101de0 0x7f0dd81985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33202/0 (socket says 172.21.15.103:33202) 2024-09-19T06:52:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0dde532640 1 -- 172.21.15.103:0/3741125556 learned_addr learned my addr 172.21.15.103:0/3741125556 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0dded33640 1 --2- 172.21.15.103:0/3741125556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dd8106f10 0x7f0dd819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0dddd31640 1 --2- 172.21.15.103:0/3741125556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dd8102790 0x7f0dd8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0dde532640 1 -- 172.21.15.103:0/3741125556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dd8102790 msgr2=0x7f0dd8198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0dde532640 1 --2- 172.21.15.103:0/3741125556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dd8102790 0x7f0dd8198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0dde532640 1 -- 172.21.15.103:0/3741125556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dd8106f10 msgr2=0x7f0dd819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0dde532640 1 --2- 172.21.15.103:0/3741125556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dd8106f10 0x7f0dd819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0dde532640 1 -- 172.21.15.103:0/3741125556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd819d560 con 0x7f0dd8101de0 2024-09-19T06:52:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0dded33640 1 --2- 172.21.15.103:0/3741125556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dd8106f10 0x7f0dd819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.667+0000 7f0dddd31640 1 --2- 172.21.15.103:0/3741125556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dd8102790 0x7f0dd8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.668+0000 7f0dde532640 1 --2- 172.21.15.103:0/3741125556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0dd8101de0 0x7f0dd81985b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800b4a0 tx=0x7f0dc800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.668+0000 7f0dcf7fe640 1 -- 172.21.15.103:0/3741125556 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8014070 con 0x7f0dd8101de0 2024-09-19T06:52:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.668+0000 7f0dcf7fe640 1 -- 172.21.15.103:0/3741125556 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc8010af0 con 0x7f0dd8101de0 2024-09-19T06:52:58.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.668+0000 7f0de0796640 1 -- 172.21.15.103:0/3741125556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd819d850 con 0x7f0dd8101de0 2024-09-19T06:52:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.668+0000 7f0dcf7fe640 1 -- 172.21.15.103:0/3741125556 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8017d10 con 0x7f0dd8101de0 2024-09-19T06:52:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.668+0000 7f0de0796640 1 -- 172.21.15.103:0/3741125556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd81a5130 con 0x7f0dd8101de0 2024-09-19T06:52:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.669+0000 7f0de0796640 1 -- 172.21.15.103:0/3741125556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd8102240 con 0x7f0dd8101de0 2024-09-19T06:52:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.670+0000 7f0dcf7fe640 1 -- 172.21.15.103:0/3741125556 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0dc8017070 con 0x7f0dd8101de0 2024-09-19T06:52:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.670+0000 7f0dcf7fe640 1 --2- 172.21.15.103:0/3741125556 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0dc40a33b0 0x7f0dc40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.671+0000 7f0dddd31640 1 --2- 172.21.15.103:0/3741125556 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0dc40a33b0 0x7f0dc40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.671+0000 7f0dcf7fe640 1 -- 172.21.15.103:0/3741125556 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f0dc80cf410 con 0x7f0dd8101de0 2024-09-19T06:52:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.671+0000 7f0dddd31640 1 --2- 172.21.15.103:0/3741125556 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0dc40a33b0 0x7f0dc40a5870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4000c00 tx=0x7f0dd4032d50 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:58.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.674+0000 7f0dcf7fe640 1 -- 172.21.15.103:0/3741125556 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0dc809bf10 con 0x7f0dd8101de0 2024-09-19T06:52:58.736 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:58.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.896+0000 7f0de0796640 1 -- 172.21.15.103:0/3741125556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dd8199910 con 0x7f0dd8101de0 2024-09-19T06:52:58.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.897+0000 7f0dcf7fe640 1 -- 172.21.15.103:0/3741125556 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f0dc809b810 con 0x7f0dd8101de0 2024-09-19T06:52:58.898 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:52:58.898 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:52:58.898 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:52:58.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.899+0000 7f0de0796640 1 -- 172.21.15.103:0/3741125556 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0dc40a33b0 msgr2=0x7f0dc40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:58.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.899+0000 7f0de0796640 1 --2- 172.21.15.103:0/3741125556 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0dc40a33b0 0x7f0dc40a5870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4000c00 tx=0x7f0dd4032d50 comp rx=0 tx=0).stop 2024-09-19T06:52:58.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.899+0000 7f0de0796640 1 -- 172.21.15.103:0/3741125556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0dd8101de0 msgr2=0x7f0dd81985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:58.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.899+0000 7f0de0796640 1 --2- 172.21.15.103:0/3741125556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0dd8101de0 0x7f0dd81985b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800b4a0 tx=0x7f0dc800b960 comp rx=0 tx=0).stop 2024-09-19T06:52:58.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.899+0000 7f0de0796640 1 -- 172.21.15.103:0/3741125556 shutdown_connections 2024-09-19T06:52:58.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.899+0000 7f0de0796640 1 --2- 172.21.15.103:0/3741125556 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0dc40a33b0 0x7f0dc40a5870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.899+0000 7f0de0796640 1 --2- 172.21.15.103:0/3741125556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dd8106f10 0x7f0dd819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.899+0000 7f0de0796640 1 --2- 172.21.15.103:0/3741125556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dd8102790 0x7f0dd8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.899+0000 7f0de0796640 1 --2- 172.21.15.103:0/3741125556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0dd8101de0 0x7f0dd81985b0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.899+0000 7f0de0796640 1 -- 172.21.15.103:0/3741125556 >> 172.21.15.103:0/3741125556 conn(0x7f0dd80fd8c0 msgr2=0x7f0dd80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:58.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.899+0000 7f0de0796640 1 -- 172.21.15.103:0/3741125556 shutdown_connections 2024-09-19T06:52:58.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:58.899+0000 7f0de0796640 1 -- 172.21.15.103:0/3741125556 wait complete. 2024-09-19T06:52:59.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:52:59 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3741125556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:59.373 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:52:59 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3741125556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:59.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:52:59 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3741125556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:59.514 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 95fcbfa8 --release --await' 2024-09-19T06:52:59.515 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 95fcbfa8 --release --await 2024-09-19T06:52:59.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.721+0000 7fbb20f2e640 1 -- 172.21.15.103:0/2844991285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb1c1009d0 msgr2=0x7fbb1c100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.721+0000 7fbb20f2e640 1 --2- 172.21.15.103:0/2844991285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb1c1009d0 0x7fbb1c100db0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009900 tx=0x7fbb0c02a410 comp rx=0 tx=0).stop 2024-09-19T06:52:59.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.721+0000 7fbb20f2e640 1 -- 172.21.15.103:0/2844991285 shutdown_connections 2024-09-19T06:52:59.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.721+0000 7fbb20f2e640 1 --2- 172.21.15.103:0/2844991285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb1c10e4f0 0x7fbb1c110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.721+0000 7fbb20f2e640 1 --2- 172.21.15.103:0/2844991285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb1c101380 0x7fbb1c10dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.721+0000 7fbb20f2e640 1 --2- 172.21.15.103:0/2844991285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb1c1009d0 0x7fbb1c100db0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.721+0000 7fbb20f2e640 1 -- 172.21.15.103:0/2844991285 >> 172.21.15.103:0/2844991285 conn(0x7fbb1c0fc7d0 msgr2=0x7fbb1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:59.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.721+0000 7fbb20f2e640 1 -- 172.21.15.103:0/2844991285 shutdown_connections 2024-09-19T06:52:59.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.721+0000 7fbb20f2e640 1 -- 172.21.15.103:0/2844991285 wait complete. 2024-09-19T06:52:59.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.722+0000 7fbb20f2e640 1 Processor -- start 2024-09-19T06:52:59.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.722+0000 7fbb20f2e640 1 -- start start 2024-09-19T06:52:59.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb20f2e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb1c1009d0 0x7fbb1c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb20f2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb1c101380 0x7fbb1c1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb20f2e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb1c10e4f0 0x7fbb1c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb20f2e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c114340 con 0x7fbb1c101380 2024-09-19T06:52:59.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb20f2e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1141c0 con 0x7fbb1c10e4f0 2024-09-19T06:52:59.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb20f2e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1144c0 con 0x7fbb1c1009d0 2024-09-19T06:52:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb19d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb1c101380 0x7fbb1c1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb19d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb1c101380 0x7fbb1c1a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33218/0 (socket says 172.21.15.103:33218) 2024-09-19T06:52:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb19d9b640 1 -- 172.21.15.103:0/451845096 learned_addr learned my addr 172.21.15.103:0/451845096 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb1a59c640 1 --2- 172.21.15.103:0/451845096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb1c1009d0 0x7fbb1c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb1ad9d640 1 --2- 172.21.15.103:0/451845096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb1c10e4f0 0x7fbb1c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb19d9b640 1 -- 172.21.15.103:0/451845096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb1c1009d0 msgr2=0x7fbb1c1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb19d9b640 1 --2- 172.21.15.103:0/451845096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb1c1009d0 0x7fbb1c1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb19d9b640 1 -- 172.21.15.103:0/451845096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb1c10e4f0 msgr2=0x7fbb1c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb19d9b640 1 --2- 172.21.15.103:0/451845096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb1c10e4f0 0x7fbb1c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb19d9b640 1 -- 172.21.15.103:0/451845096 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1c19af50 con 0x7fbb1c101380 2024-09-19T06:52:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb1ad9d640 1 --2- 172.21.15.103:0/451845096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb1c10e4f0 0x7fbb1c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb19d9b640 1 --2- 172.21.15.103:0/451845096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb1c101380 0x7fbb1c1a0b50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400b4a0 tx=0x7fbb0400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.723+0000 7fbb137fe640 1 -- 172.21.15.103:0/451845096 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04014070 con 0x7fbb1c101380 2024-09-19T06:52:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.724+0000 7fbb137fe640 1 -- 172.21.15.103:0/451845096 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb04010af0 con 0x7fbb1c101380 2024-09-19T06:52:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.724+0000 7fbb137fe640 1 -- 172.21.15.103:0/451845096 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04017d10 con 0x7fbb1c101380 2024-09-19T06:52:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.724+0000 7fbb20f2e640 1 -- 172.21.15.103:0/451845096 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1c19b240 con 0x7fbb1c101380 2024-09-19T06:52:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.724+0000 7fbb20f2e640 1 -- 172.21.15.103:0/451845096 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1c1a73c0 con 0x7fbb1c101380 2024-09-19T06:52:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.726+0000 7fbb137fe640 1 -- 172.21.15.103:0/451845096 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fbb04017070 con 0x7fbb1c101380 2024-09-19T06:52:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.726+0000 7fbb23307740 1 Processor -- start 2024-09-19T06:52:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.726+0000 7fbb23307740 1 -- start start 2024-09-19T06:52:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb23307740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25feb40 0x562fa25fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb23307740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25fd5d0 0x562fa25fda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb23307740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa2608370 0x562fa260a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb23307740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562fa25632f0 con 0x562fa25feb40 2024-09-19T06:52:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb23307740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562fa250acf0 con 0x562fa2608370 2024-09-19T06:52:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb23307740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562fa255f2b0 con 0x562fa25fd5d0 2024-09-19T06:52:59.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb1ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25feb40 0x562fa25fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb1a59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25fd5d0 0x562fa25fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb1ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25feb40 0x562fa25fd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33226/0 (socket says 172.21.15.103:33226) 2024-09-19T06:52:59.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb1ad9d640 1 -- 172.21.15.103:0/845733140 learned_addr learned my addr 172.21.15.103:0/845733140 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:59.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb19d9b640 1 --2- 172.21.15.103:0/845733140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa2608370 0x562fa260a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb1ad9d640 1 -- 172.21.15.103:0/845733140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25fd5d0 msgr2=0x562fa25fda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb137fe640 1 --2- 172.21.15.103:0/451845096 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbb080a3430 0x7fbb080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb1ad9d640 1 --2- 172.21.15.103:0/845733140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25fd5d0 0x562fa25fda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb1ad9d640 1 -- 172.21.15.103:0/845733140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa2608370 msgr2=0x562fa260a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb1ad9d640 1 --2- 172.21.15.103:0/845733140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa2608370 0x562fa260a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb1ad9d640 1 -- 172.21.15.103:0/845733140 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa24ef3a0 con 0x562fa25feb40 2024-09-19T06:52:59.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb137fe640 1 -- 172.21.15.103:0/451845096 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fbb040cefa0 con 0x7fbb1c101380 2024-09-19T06:52:59.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb1ad9d640 1 --2- 172.21.15.103:0/845733140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25feb40 0x562fa25fd090 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbb14007f00 tx=0x7fbb1400c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6138e7cc3912d5f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:59.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.727+0000 7fbb1ad9d640 1 --2- 172.21.15.103:0/451845096 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbb080a3430 0x7fbb080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.728+0000 7fbb10ff9640 1 -- 172.21.15.103:0/845733140 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14014070 con 0x562fa25feb40 2024-09-19T06:52:59.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.728+0000 7fbb10ff9640 1 -- 172.21.15.103:0/845733140 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb14019930 con 0x562fa25feb40 2024-09-19T06:52:59.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.728+0000 7fbb10ff9640 1 -- 172.21.15.103:0/845733140 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400ece0 con 0x562fa25feb40 2024-09-19T06:52:59.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.728+0000 7fbb23307740 1 -- 172.21.15.103:0/845733140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25feb40 msgr2=0x562fa25fd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.728+0000 7fbb23307740 1 --2- 172.21.15.103:0/845733140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25feb40 0x562fa25fd090 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbb14007f00 tx=0x7fbb1400c690 comp rx=0 tx=0).stop 2024-09-19T06:52:59.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.728+0000 7fbb1ad9d640 1 --2- 172.21.15.103:0/451845096 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbb080a3430 0x7fbb080a58f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400efd0 tx=0x7fbb1400ef60 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:59.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.728+0000 7fbb23307740 1 -- 172.21.15.103:0/845733140 shutdown_connections 2024-09-19T06:52:59.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.728+0000 7fbb23307740 1 --2- 172.21.15.103:0/845733140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa2608370 0x562fa260a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.728+0000 7fbb23307740 1 --2- 172.21.15.103:0/845733140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25fd5d0 0x562fa25fda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.728+0000 7fbb23307740 1 --2- 172.21.15.103:0/845733140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25feb40 0x562fa25fd090 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.728+0000 7fbb23307740 1 -- 172.21.15.103:0/845733140 >> 172.21.15.103:0/845733140 conn(0x562fa25c6310 msgr2=0x562fa25f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:59.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.729+0000 7fbb23307740 1 -- 172.21.15.103:0/845733140 shutdown_connections 2024-09-19T06:52:59.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.729+0000 7fbb23307740 1 -- 172.21.15.103:0/845733140 wait complete. 2024-09-19T06:52:59.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.729+0000 7fbb23307740 1 -- start start 2024-09-19T06:52:59.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.729+0000 7fbb23307740 1 Processor -- start 2024-09-19T06:52:59.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.730+0000 7fbb23307740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25fd5d0 0x562fa2607cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.730+0000 7fbb23307740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25feb40 0x562fa2604f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.730+0000 7fbb23307740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa2608370 0x562fa26054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.730+0000 7fbb23307740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562fa260b070 con 0x562fa25fd5d0 2024-09-19T06:52:59.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.730+0000 7fbb23307740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562fa251e060 con 0x562fa2608370 2024-09-19T06:52:59.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.730+0000 7fbb23307740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562fa260b1f0 con 0x562fa25feb40 2024-09-19T06:52:59.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.730+0000 7fbb1a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25fd5d0 0x562fa2607cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.730+0000 7fbb1a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25fd5d0 0x562fa2607cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33228/0 (socket says 172.21.15.103:33228) 2024-09-19T06:52:59.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.730+0000 7fbb1a59c640 1 -- 172.21.15.103:0/4166696476 learned_addr learned my addr 172.21.15.103:0/4166696476 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:59.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.730+0000 7fbb1ad9d640 1 --2- 172.21.15.103:0/4166696476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa2608370 0x562fa26054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.730+0000 7fbb19d9b640 1 --2- 172.21.15.103:0/4166696476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25feb40 0x562fa2604f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.730+0000 7fbb23307740 10 client.0 _ll_register_callbacks cb 0x562fa22718c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:59.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.730+0000 7fbb1a59c640 1 -- 172.21.15.103:0/4166696476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25feb40 msgr2=0x562fa2604f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.730+0000 7fbb1a59c640 1 --2- 172.21.15.103:0/4166696476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25feb40 0x562fa2604f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.731+0000 7fbb1a59c640 1 -- 172.21.15.103:0/4166696476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa2608370 msgr2=0x562fa26054d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.731+0000 7fbb1a59c640 1 --2- 172.21.15.103:0/4166696476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa2608370 0x562fa26054d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.731+0000 7fbb1a59c640 1 -- 172.21.15.103:0/4166696476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa22d43c0 con 0x562fa25fd5d0 2024-09-19T06:52:59.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.731+0000 7fbb1ad9d640 1 --2- 172.21.15.103:0/4166696476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa2608370 0x562fa26054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:59.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.731+0000 7fbb1a59c640 1 --2- 172.21.15.103:0/4166696476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25fd5d0 0x562fa2607cc0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009900 tx=0x7fbb0c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9656811db950cbf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:59.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.731+0000 7fbb01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:59.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.731+0000 7fbb01ffb640 1 -- 172.21.15.103:0/4166696476 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c035040 con 0x562fa25fd5d0 2024-09-19T06:52:59.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.731+0000 7fbb01ffb640 1 -- 172.21.15.103:0/4166696476 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fbb0c005020 con 0x562fa25fd5d0 2024-09-19T06:52:59.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.731+0000 7fbb23307740 1 -- 172.21.15.103:0/4166696476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb14019930 con 0x562fa25fd5d0 2024-09-19T06:52:59.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.731+0000 7fbb01ffb640 1 -- 172.21.15.103:0/4166696476 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbb0c005d50 con 0x562fa25fd5d0 2024-09-19T06:52:59.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.731+0000 7fbb23307740 10 client.16812 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:59.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.731+0000 7fbb23307740 1 -- 172.21.15.103:0/4166696476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562fa2616a60 con 0x562fa25fd5d0 2024-09-19T06:52:59.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.732+0000 7fbb01ffb640 1 -- 172.21.15.103:0/4166696476 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb14019930 con 0x562fa25fd5d0 2024-09-19T06:52:59.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.732+0000 7fbb01ffb640 1 -- 172.21.15.103:0/4166696476 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbb0c003760 con 0x562fa25fd5d0 2024-09-19T06:52:59.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.732+0000 7fbb23307740 10 client.16812 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:59.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.732+0000 7fbb23307740 10 client.16812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:59.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.732+0000 7fbb23307740 20 client.16812 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:59.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.732+0000 7fbb23307740 1 --2- 172.21.15.103:0/4166696476 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562fa2617e70 0x562fa261a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.732+0000 7fbb23307740 4 client.16812 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:59.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.732+0000 7fbb23307740 1 -- 172.21.15.103:0/4166696476 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fa2570860 con 0x562fa2617e70 2024-09-19T06:52:59.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.732+0000 7fbb19d9b640 1 --2- 172.21.15.103:0/4166696476 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562fa2617e70 0x562fa261a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.733+0000 7fbb01ffb640 10 client.16812 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:59.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.733+0000 7fbb19d9b640 1 --2- 172.21.15.103:0/4166696476 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562fa2617e70 0x562fa261a2e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37bb2f07e52d6cc7 server_cookie=7d46d4bef55904f5 in_seq=0 out_seq=0 2024-09-19T06:52:59.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.733+0000 7fbb01ffb640 1 -- 172.21.15.103:0/4166696476 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562fa2570860 con 0x562fa2617e70 2024-09-19T06:52:59.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.733+0000 7fbb01ffb640 10 client.16812 handle_command_reply: tid=0 2024-09-19T06:52:59.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.734+0000 7fbb23307740 1 client.16812 shutdown 2024-09-19T06:52:59.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.734+0000 7fbb23307740 1 -- 172.21.15.103:0/4166696476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25fd5d0 msgr2=0x562fa2607cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.734+0000 7fbb23307740 1 --2- 172.21.15.103:0/4166696476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25fd5d0 0x562fa2607cc0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009900 tx=0x7fbb0c004270 comp rx=0 tx=0).stop 2024-09-19T06:52:59.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.734+0000 7fbb23307740 1 -- 172.21.15.103:0/4166696476 shutdown_connections 2024-09-19T06:52:59.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.734+0000 7fbb23307740 1 --2- 172.21.15.103:0/4166696476 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562fa2617e70 0x562fa261a2e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.734+0000 7fbb23307740 1 --2- 172.21.15.103:0/4166696476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa2608370 0x562fa26054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.734+0000 7fbb23307740 1 --2- 172.21.15.103:0/4166696476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25feb40 0x562fa2604f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.734+0000 7fbb23307740 1 --2- 172.21.15.103:0/4166696476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25fd5d0 0x562fa2607cc0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.734+0000 7fbb01ffb640 0 client.16812 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:59.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.734+0000 7fbb23307740 1 -- 172.21.15.103:0/4166696476 >> 172.21.15.103:0/4166696476 conn(0x562fa25c6310 msgr2=0x562fa25ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:59.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.734+0000 7fbb23307740 1 -- 172.21.15.103:0/4166696476 shutdown_connections 2024-09-19T06:52:59.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.734+0000 7fbb23307740 1 -- 172.21.15.103:0/4166696476 wait complete. 2024-09-19T06:52:59.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.734+0000 7fbb23307740 20 client.16812 trim_cache size 0 max 16384 2024-09-19T06:52:59.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.747+0000 7fbb23307740 1 Processor -- start 2024-09-19T06:52:59.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.747+0000 7fbb23307740 1 -- start start 2024-09-19T06:52:59.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.747+0000 7fbb23307740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25fd5d0 0x562fa2614310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.747+0000 7fbb23307740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25feb40 0x562fa261fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.747+0000 7fbb23307740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa2608370 0x562fa26200f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.747+0000 7fbb23307740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1afbf0 con 0x562fa25fd5d0 2024-09-19T06:52:59.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.747+0000 7fbb23307740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1b1730 con 0x562fa2608370 2024-09-19T06:52:59.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.747+0000 7fbb23307740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1b02c0 con 0x562fa25feb40 2024-09-19T06:52:59.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.747+0000 7fbb1a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25fd5d0 0x562fa2614310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.747+0000 7fbb1a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25fd5d0 0x562fa2614310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33238/0 (socket says 172.21.15.103:33238) 2024-09-19T06:52:59.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.747+0000 7fbb1a59c640 1 -- 172.21.15.103:0/2608050696 learned_addr learned my addr 172.21.15.103:0/2608050696 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:59.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.747+0000 7fbb19d9b640 1 --2- 172.21.15.103:0/2608050696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25feb40 0x562fa261fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.748+0000 7fbb1ad9d640 1 --2- 172.21.15.103:0/2608050696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa2608370 0x562fa26200f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.748+0000 7fbb1a59c640 1 -- 172.21.15.103:0/2608050696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25feb40 msgr2=0x562fa261fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.748+0000 7fbb1a59c640 1 --2- 172.21.15.103:0/2608050696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25feb40 0x562fa261fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.748+0000 7fbb1a59c640 1 -- 172.21.15.103:0/2608050696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa2608370 msgr2=0x562fa26200f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.748+0000 7fbb1a59c640 1 --2- 172.21.15.103:0/2608050696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa2608370 0x562fa26200f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.748+0000 7fbb1a59c640 1 -- 172.21.15.103:0/2608050696 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa25fad40 con 0x562fa25fd5d0 2024-09-19T06:52:59.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.748+0000 7fbb1a59c640 1 --2- 172.21.15.103:0/2608050696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25fd5d0 0x562fa2614310 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c000c00 tx=0x7fbb0c003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78503c87d667113 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:59.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.748+0000 7fbb03fff640 1 -- 172.21.15.103:0/2608050696 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c035040 con 0x562fa25fd5d0 2024-09-19T06:52:59.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.748+0000 7fbb03fff640 1 -- 172.21.15.103:0/2608050696 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb0c045640 con 0x562fa25fd5d0 2024-09-19T06:52:59.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.748+0000 7fbb03fff640 1 -- 172.21.15.103:0/2608050696 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0044e0 con 0x562fa25fd5d0 2024-09-19T06:52:59.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.749+0000 7fbb23307740 1 -- 172.21.15.103:0/2608050696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25fd5d0 msgr2=0x562fa2614310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.749+0000 7fbb23307740 1 --2- 172.21.15.103:0/2608050696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25fd5d0 0x562fa2614310 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c000c00 tx=0x7fbb0c003f50 comp rx=0 tx=0).stop 2024-09-19T06:52:59.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.749+0000 7fbb23307740 1 -- 172.21.15.103:0/2608050696 shutdown_connections 2024-09-19T06:52:59.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.749+0000 7fbb23307740 1 --2- 172.21.15.103:0/2608050696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa2608370 0x562fa26200f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.749+0000 7fbb23307740 1 --2- 172.21.15.103:0/2608050696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25feb40 0x562fa261fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.749+0000 7fbb23307740 1 --2- 172.21.15.103:0/2608050696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa25fd5d0 0x562fa2614310 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.749+0000 7fbb23307740 1 -- 172.21.15.103:0/2608050696 >> 172.21.15.103:0/2608050696 conn(0x562fa25c6310 msgr2=0x562fa2600310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:59.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.749+0000 7fbb23307740 1 -- 172.21.15.103:0/2608050696 shutdown_connections 2024-09-19T06:52:59.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.749+0000 7fbb23307740 1 -- 172.21.15.103:0/2608050696 wait complete. 2024-09-19T06:52:59.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.749+0000 7fbb23307740 1 -- start start 2024-09-19T06:52:59.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb23307740 1 Processor -- start 2024-09-19T06:52:59.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb23307740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa25fd5d0 0x562fa260c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb23307740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25feb40 0x562fa260c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb23307740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa2608370 0x562fa260ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb23307740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1b1060 con 0x562fa2608370 2024-09-19T06:52:59.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb23307740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1b0990 con 0x562fa25fd5d0 2024-09-19T06:52:59.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb23307740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1adbb0 con 0x562fa25feb40 2024-09-19T06:52:59.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb23307740 10 client.0 _ll_register_callbacks cb 0x7fbb0c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:59.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb1ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa2608370 0x562fa260ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb1ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa2608370 0x562fa260ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33248/0 (socket says 172.21.15.103:33248) 2024-09-19T06:52:59.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb1ad9d640 1 -- 172.21.15.103:0/397809946 learned_addr learned my addr 172.21.15.103:0/397809946 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:52:59.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb19d9b640 1 --2- 172.21.15.103:0/397809946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25feb40 0x562fa260c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb1a59c640 1 --2- 172.21.15.103:0/397809946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa25fd5d0 0x562fa260c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb1ad9d640 1 -- 172.21.15.103:0/397809946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25feb40 msgr2=0x562fa260c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb1ad9d640 1 --2- 172.21.15.103:0/397809946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25feb40 0x562fa260c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb1ad9d640 1 -- 172.21.15.103:0/397809946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa25fd5d0 msgr2=0x562fa260c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb1ad9d640 1 --2- 172.21.15.103:0/397809946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa25fd5d0 0x562fa260c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.750+0000 7fbb1ad9d640 1 -- 172.21.15.103:0/397809946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa2681d20 con 0x562fa2608370 2024-09-19T06:52:59.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.751+0000 7fbb1ad9d640 1 --2- 172.21.15.103:0/397809946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa2608370 0x562fa260ce00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fbb14007f00 tx=0x7fbb14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10eecca517fdb70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:59.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.751+0000 7fbb01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:52:59.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.751+0000 7fbb01ffb640 1 -- 172.21.15.103:0/397809946 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14036060 con 0x562fa2608370 2024-09-19T06:52:59.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.751+0000 7fbb01ffb640 1 -- 172.21.15.103:0/397809946 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fbb14017b50 con 0x562fa2608370 2024-09-19T06:52:59.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.751+0000 7fbb23307740 1 -- 172.21.15.103:0/397809946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb0c045640 con 0x562fa2608370 2024-09-19T06:52:59.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.751+0000 7fbb01ffb640 1 -- 172.21.15.103:0/397809946 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbb14014040 con 0x562fa2608370 2024-09-19T06:52:59.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.751+0000 7fbb23307740 10 client.16818 fetch_fsmap learned FSMap version 27 2024-09-19T06:52:59.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.751+0000 7fbb23307740 1 -- 172.21.15.103:0/397809946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562fa2683000 con 0x562fa2608370 2024-09-19T06:52:59.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.751+0000 7fbb01ffb640 1 -- 172.21.15.103:0/397809946 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb0c045640 con 0x562fa2608370 2024-09-19T06:52:59.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.752+0000 7fbb01ffb640 1 -- 172.21.15.103:0/397809946 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbb14030070 con 0x562fa2608370 2024-09-19T06:52:59.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.752+0000 7fbb23307740 10 client.16818 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:52:59.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.752+0000 7fbb23307740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:59.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.752+0000 7fbb23307740 20 client.16818 populate_metadata read hostname 'smithi103' 2024-09-19T06:52:59.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.752+0000 7fbb23307740 1 --2- 172.21.15.103:0/397809946 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562fa2683440 0x562fa2685860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.752+0000 7fbb23307740 4 client.16818 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:52:59.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.752+0000 7fbb23307740 1 -- 172.21.15.103:0/397809946 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562fa256ad70 con 0x562fa2683440 2024-09-19T06:52:59.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.752+0000 7fbb1a59c640 1 --2- 172.21.15.103:0/397809946 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562fa2683440 0x562fa2685860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.752+0000 7fbb1a59c640 1 --2- 172.21.15.103:0/397809946 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562fa2683440 0x562fa2685860 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5809d4867a9961b server_cookie=a1d6267d5d9bca53 in_seq=0 out_seq=0 2024-09-19T06:52:59.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.752+0000 7fbb01ffb640 10 client.16818 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:59.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.753+0000 7fbb01ffb640 1 -- 172.21.15.103:0/397809946 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562fa256ad70 con 0x562fa2683440 2024-09-19T06:52:59.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.753+0000 7fbb01ffb640 10 client.16818 handle_command_reply: tid=0 2024-09-19T06:52:59.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.753+0000 7fbb23307740 1 client.16818 shutdown 2024-09-19T06:52:59.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.753+0000 7fbb23307740 1 -- 172.21.15.103:0/397809946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa2608370 msgr2=0x562fa260ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.753+0000 7fbb23307740 1 --2- 172.21.15.103:0/397809946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa2608370 0x562fa260ce00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fbb14007f00 tx=0x7fbb14017000 comp rx=0 tx=0).stop 2024-09-19T06:52:59.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.753+0000 7fbb23307740 1 -- 172.21.15.103:0/397809946 shutdown_connections 2024-09-19T06:52:59.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.753+0000 7fbb23307740 1 --2- 172.21.15.103:0/397809946 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562fa2683440 0x562fa2685860 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.753+0000 7fbb23307740 1 --2- 172.21.15.103:0/397809946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fa2608370 0x562fa260ce00 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.753+0000 7fbb23307740 1 --2- 172.21.15.103:0/397809946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fa25feb40 0x562fa260c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.753+0000 7fbb01ffb640 0 client.16818 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:52:59.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.753+0000 7fbb23307740 1 --2- 172.21.15.103:0/397809946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fa25fd5d0 0x562fa260c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.753+0000 7fbb23307740 1 -- 172.21.15.103:0/397809946 >> 172.21.15.103:0/397809946 conn(0x562fa25c6310 msgr2=0x562fa25f8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:59.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.753+0000 7fbb23307740 1 -- 172.21.15.103:0/397809946 shutdown_connections 2024-09-19T06:52:59.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.753+0000 7fbb23307740 1 -- 172.21.15.103:0/397809946 wait complete. 2024-09-19T06:52:59.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.753+0000 7fbb23307740 20 client.16818 trim_cache size 0 max 16384 2024-09-19T06:52:59.760 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:52:59.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.754+0000 7fbb20f2e640 1 -- 172.21.15.103:0/451845096 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbb080a3430 msgr2=0x7fbb080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.754+0000 7fbb20f2e640 1 --2- 172.21.15.103:0/451845096 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbb080a3430 0x7fbb080a58f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400efd0 tx=0x7fbb1400ef60 comp rx=0 tx=0).stop 2024-09-19T06:52:59.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.754+0000 7fbb20f2e640 1 -- 172.21.15.103:0/451845096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb1c101380 msgr2=0x7fbb1c1a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.754+0000 7fbb20f2e640 1 --2- 172.21.15.103:0/451845096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb1c101380 0x7fbb1c1a0b50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400b4a0 tx=0x7fbb0400b960 comp rx=0 tx=0).stop 2024-09-19T06:52:59.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.754+0000 7fbb20f2e640 1 -- 172.21.15.103:0/451845096 shutdown_connections 2024-09-19T06:52:59.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.754+0000 7fbb20f2e640 1 --2- 172.21.15.103:0/451845096 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbb080a3430 0x7fbb080a58f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.754+0000 7fbb20f2e640 1 --2- 172.21.15.103:0/451845096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb1c10e4f0 0x7fbb1c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.754+0000 7fbb20f2e640 1 --2- 172.21.15.103:0/451845096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb1c101380 0x7fbb1c1a0b50 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.754+0000 7fbb20f2e640 1 --2- 172.21.15.103:0/451845096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb1c1009d0 0x7fbb1c1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.754+0000 7fbb20f2e640 1 -- 172.21.15.103:0/451845096 >> 172.21.15.103:0/451845096 conn(0x7fbb1c0fc7d0 msgr2=0x7fbb1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:59.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.754+0000 7fbb20f2e640 1 -- 172.21.15.103:0/451845096 shutdown_connections 2024-09-19T06:52:59.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:52:59.754+0000 7fbb20f2e640 1 -- 172.21.15.103:0/451845096 wait complete. 2024-09-19T06:53:00.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:00 smithi174 ceph-mon[35637]: pgmap v116: 97 pgs: 97 active+clean; 3.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-09-19T06:53:00.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:00 smithi103 ceph-mon[31158]: pgmap v116: 97 pgs: 97 active+clean; 3.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-09-19T06:53:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:00 smithi134 ceph-mon[35356]: pgmap v116: 97 pgs: 97 active+clean; 3.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-09-19T06:53:00.540 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:01.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.061+0000 7fd15308d640 1 -- 172.21.15.103:0/2080592790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd14c101420 msgr2=0x7fd14c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:01.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.061+0000 7fd15308d640 1 --2- 172.21.15.103:0/2080592790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd14c101420 0x7fd14c10d950 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd134009930 tx=0x7fd13402a420 comp rx=0 tx=0).stop 2024-09-19T06:53:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.061+0000 7fd15308d640 1 -- 172.21.15.103:0/2080592790 shutdown_connections 2024-09-19T06:53:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.061+0000 7fd15308d640 1 --2- 172.21.15.103:0/2080592790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd14c101420 0x7fd14c10d950 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.061+0000 7fd15308d640 1 --2- 172.21.15.103:0/2080592790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd14c100a80 0x7fd14c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.061+0000 7fd15308d640 1 --2- 172.21.15.103:0/2080592790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd14c106a80 0x7fd14c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.061+0000 7fd15308d640 1 -- 172.21.15.103:0/2080592790 >> 172.21.15.103:0/2080592790 conn(0x7fd14c0fc7a0 msgr2=0x7fd14c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.061+0000 7fd15308d640 1 -- 172.21.15.103:0/2080592790 shutdown_connections 2024-09-19T06:53:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.061+0000 7fd15308d640 1 -- 172.21.15.103:0/2080592790 wait complete. 2024-09-19T06:53:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.062+0000 7fd15308d640 1 Processor -- start 2024-09-19T06:53:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.062+0000 7fd15308d640 1 -- start start 2024-09-19T06:53:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.062+0000 7fd15308d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd14c100a80 0x7fd14c197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.062+0000 7fd15308d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd14c101420 0x7fd14c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.062+0000 7fd15308d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd14c106a80 0x7fd14c19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.062+0000 7fd15308d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c110f20 con 0x7fd14c100a80 2024-09-19T06:53:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.062+0000 7fd15308d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c110da0 con 0x7fd14c101420 2024-09-19T06:53:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.062+0000 7fd15308d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c1110a0 con 0x7fd14c106a80 2024-09-19T06:53:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd150e29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd14c100a80 0x7fd14c197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd150e29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd14c100a80 0x7fd14c197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33272/0 (socket says 172.21.15.103:33272) 2024-09-19T06:53:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd150e29640 1 -- 172.21.15.103:0/2004466533 learned_addr learned my addr 172.21.15.103:0/2004466533 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd15162a640 1 --2- 172.21.15.103:0/2004466533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd14c106a80 0x7fd14c19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd14bfff640 1 --2- 172.21.15.103:0/2004466533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd14c101420 0x7fd14c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd150e29640 1 -- 172.21.15.103:0/2004466533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd14c106a80 msgr2=0x7fd14c19bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd150e29640 1 --2- 172.21.15.103:0/2004466533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd14c106a80 0x7fd14c19bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd150e29640 1 -- 172.21.15.103:0/2004466533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd14c101420 msgr2=0x7fd14c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd150e29640 1 --2- 172.21.15.103:0/2004466533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd14c101420 0x7fd14c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd150e29640 1 -- 172.21.15.103:0/2004466533 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14c19c430 con 0x7fd14c100a80 2024-09-19T06:53:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd15162a640 1 --2- 172.21.15.103:0/2004466533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd14c106a80 0x7fd14c19bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd150e29640 1 --2- 172.21.15.103:0/2004466533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd14c100a80 0x7fd14c197480 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd14400b4a0 tx=0x7fd14400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd149ffb640 1 -- 172.21.15.103:0/2004466533 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144014070 con 0x7fd14c100a80 2024-09-19T06:53:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd15308d640 1 -- 172.21.15.103:0/2004466533 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14c19c720 con 0x7fd14c100a80 2024-09-19T06:53:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd149ffb640 1 -- 172.21.15.103:0/2004466533 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd144010af0 con 0x7fd14c100a80 2024-09-19T06:53:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd149ffb640 1 -- 172.21.15.103:0/2004466533 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144017d10 con 0x7fd14c100a80 2024-09-19T06:53:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.063+0000 7fd15308d640 1 -- 172.21.15.103:0/2004466533 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd14c1a4000 con 0x7fd14c100a80 2024-09-19T06:53:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.066+0000 7fd149ffb640 1 -- 172.21.15.103:0/2004466533 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd144017070 con 0x7fd14c100a80 2024-09-19T06:53:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.067+0000 7fd155466740 1 Processor -- start 2024-09-19T06:53:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.067+0000 7fd155466740 1 -- start start 2024-09-19T06:53:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.067+0000 7fd149ffb640 1 --2- 172.21.15.103:0/2004466533 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1380a33b0 0x7fd1380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.067+0000 7fd155466740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949c270 0x56333949c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.067+0000 7fd15162a640 1 --2- 172.21.15.103:0/2004466533 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1380a33b0 0x7fd1380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.067+0000 7fd150e29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949c270 0x56333949c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.067+0000 7fd150e29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949c270 0x56333949c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33280/0 (socket says 172.21.15.103:33280) 2024-09-19T06:53:01.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.067+0000 7fd149ffb640 1 -- 172.21.15.103:0/2004466533 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd1440cf080 con 0x7fd14c100a80 2024-09-19T06:53:01.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.067+0000 7fd150e29640 1 -- 172.21.15.103:0/904812133 learned_addr learned my addr 172.21.15.103:0/904812133 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.067+0000 7fd155466740 1 --2- 172.21.15.103:0/904812133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56333949cb90 0x5633394a6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.067+0000 7fd155466740 1 --2- 172.21.15.103:0/904812133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633394a7450 0x5633394a9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.067+0000 7fd155466740 1 -- 172.21.15.103:0/904812133 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5633393ff620 con 0x56333949c270 2024-09-19T06:53:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.068+0000 7fd155466740 1 -- 172.21.15.103:0/904812133 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5633393a6e50 con 0x56333949cb90 2024-09-19T06:53:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.068+0000 7fd155466740 1 -- 172.21.15.103:0/904812133 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5633393fb5e0 con 0x5633394a7450 2024-09-19T06:53:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.068+0000 7fd14bfff640 1 --2- 172.21.15.103:0/904812133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56333949cb90 0x5633394a6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.068+0000 7fd15162a640 1 --2- 172.21.15.103:0/904812133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633394a7450 0x5633394a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.068+0000 7fd15162a640 1 --2- 172.21.15.103:0/2004466533 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1380a33b0 0x7fd1380a5870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd134005ab0 tx=0x7fd134005a20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.068+0000 7fd150e29640 1 -- 172.21.15.103:0/904812133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633394a7450 msgr2=0x5633394a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.068+0000 7fd150e29640 1 --2- 172.21.15.103:0/904812133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633394a7450 0x5633394a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.068+0000 7fd150e29640 1 -- 172.21.15.103:0/904812133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56333949cb90 msgr2=0x5633394a6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.068+0000 7fd150e29640 1 --2- 172.21.15.103:0/904812133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56333949cb90 0x5633394a6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.068+0000 7fd150e29640 1 -- 172.21.15.103:0/904812133 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563339397d40 con 0x56333949c270 2024-09-19T06:53:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.068+0000 7fd15162a640 1 --2- 172.21.15.103:0/904812133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633394a7450 0x5633394a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.068+0000 7fd150e29640 1 --2- 172.21.15.103:0/904812133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949c270 0x56333949c650 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd144097e50 tx=0x7fd144009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd7ce31dae684d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.068+0000 7fd142ffd640 1 -- 172.21.15.103:0/904812133 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14401e070 con 0x56333949c270 2024-09-19T06:53:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.068+0000 7fd142ffd640 1 -- 172.21.15.103:0/904812133 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd14404a040 con 0x56333949c270 2024-09-19T06:53:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.068+0000 7fd142ffd640 1 -- 172.21.15.103:0/904812133 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144010ca0 con 0x56333949c270 2024-09-19T06:53:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.069+0000 7fd155466740 1 -- 172.21.15.103:0/904812133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949c270 msgr2=0x56333949c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.069+0000 7fd155466740 1 --2- 172.21.15.103:0/904812133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949c270 0x56333949c650 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd144097e50 tx=0x7fd144009390 comp rx=0 tx=0).stop 2024-09-19T06:53:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.069+0000 7fd155466740 1 -- 172.21.15.103:0/904812133 shutdown_connections 2024-09-19T06:53:01.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.069+0000 7fd155466740 1 --2- 172.21.15.103:0/904812133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633394a7450 0x5633394a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.069+0000 7fd155466740 1 --2- 172.21.15.103:0/904812133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56333949cb90 0x5633394a6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.069+0000 7fd155466740 1 --2- 172.21.15.103:0/904812133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949c270 0x56333949c650 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.069+0000 7fd155466740 1 -- 172.21.15.103:0/904812133 >> 172.21.15.103:0/904812133 conn(0x563339498800 msgr2=0x563339498be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:01.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.069+0000 7fd155466740 1 -- 172.21.15.103:0/904812133 shutdown_connections 2024-09-19T06:53:01.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.069+0000 7fd155466740 1 -- 172.21.15.103:0/904812133 wait complete. 2024-09-19T06:53:01.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.070+0000 7fd155466740 1 -- start start 2024-09-19T06:53:01.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.070+0000 7fd155466740 1 Processor -- start 2024-09-19T06:53:01.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.070+0000 7fd155466740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56333949c270 0x5633394a2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.070+0000 7fd155466740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949cb90 0x5633394a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.070+0000 7fd155466740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633394a7450 0x5633394b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.070+0000 7fd155466740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563339495fc0 con 0x56333949cb90 2024-09-19T06:53:01.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.070+0000 7fd155466740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563339387f10 con 0x56333949c270 2024-09-19T06:53:01.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.070+0000 7fd155466740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563339496140 con 0x5633394a7450 2024-09-19T06:53:01.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.070+0000 7fd155466740 10 client.0 _ll_register_callbacks cb 0x5633393abdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:01.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.070+0000 7fd14bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949cb90 0x5633394a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.070+0000 7fd14bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949cb90 0x5633394a2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33288/0 (socket says 172.21.15.103:33288) 2024-09-19T06:53:01.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.070+0000 7fd14bfff640 1 -- 172.21.15.103:0/1509607446 learned_addr learned my addr 172.21.15.103:0/1509607446 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:01.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.070+0000 7fd150e29640 1 --2- 172.21.15.103:0/1509607446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56333949c270 0x5633394a2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.071+0000 7fd15162a640 1 --2- 172.21.15.103:0/1509607446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633394a7450 0x5633394b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.071+0000 7fd14bfff640 1 -- 172.21.15.103:0/1509607446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633394a7450 msgr2=0x5633394b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:01.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.071+0000 7fd14bfff640 1 --2- 172.21.15.103:0/1509607446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633394a7450 0x5633394b51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.071+0000 7fd14bfff640 1 -- 172.21.15.103:0/1509607446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56333949c270 msgr2=0x5633394a2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:01.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.071+0000 7fd14bfff640 1 --2- 172.21.15.103:0/1509607446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56333949c270 0x5633394a2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.071+0000 7fd14bfff640 1 -- 172.21.15.103:0/1509607446 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56333910e580 con 0x56333949cb90 2024-09-19T06:53:01.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.071+0000 7fd150e29640 1 --2- 172.21.15.103:0/1509607446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56333949c270 0x5633394a2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:01.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.071+0000 7fd12bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:01.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.071+0000 7fd14bfff640 1 --2- 172.21.15.103:0/1509607446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949cb90 0x5633394a2e60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00be80 tx=0x7fd13c007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a12e75335d90239 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:01.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.071+0000 7fd12bfff640 1 -- 172.21.15.103:0/1509607446 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c009e20 con 0x56333949cb90 2024-09-19T06:53:01.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.071+0000 7fd12bfff640 1 -- 172.21.15.103:0/1509607446 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd13c007c90 con 0x56333949cb90 2024-09-19T06:53:01.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.071+0000 7fd155466740 1 -- 172.21.15.103:0/1509607446 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd14404a040 con 0x56333949cb90 2024-09-19T06:53:01.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.071+0000 7fd12bfff640 1 -- 172.21.15.103:0/1509607446 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd13c019410 con 0x56333949cb90 2024-09-19T06:53:01.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.071+0000 7fd155466740 10 client.16830 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:01.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.071+0000 7fd155466740 1 -- 172.21.15.103:0/1509607446 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633391d8960 con 0x56333949cb90 2024-09-19T06:53:01.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.071+0000 7fd12bfff640 1 -- 172.21.15.103:0/1509607446 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd14404a040 con 0x56333949cb90 2024-09-19T06:53:01.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.072+0000 7fd12bfff640 1 -- 172.21.15.103:0/1509607446 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd13c0216d0 con 0x56333949cb90 2024-09-19T06:53:01.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.072+0000 7fd155466740 10 client.16830 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:01.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.072+0000 7fd155466740 10 client.16830 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:53:01.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.072+0000 7fd155466740 20 client.16830 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:01.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.072+0000 7fd155466740 1 --2- 172.21.15.103:0/1509607446 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5633394b91d0 0x5633394bb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.072+0000 7fd155466740 4 client.16830 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:01.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.072+0000 7fd155466740 1 -- 172.21.15.103:0/1509607446 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56333940cb90 con 0x5633394b91d0 2024-09-19T06:53:01.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.072+0000 7fd150e29640 1 --2- 172.21.15.103:0/1509607446 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5633394b91d0 0x5633394bb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.073+0000 7fd12bfff640 10 client.16830 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:01.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.073+0000 7fd150e29640 1 --2- 172.21.15.103:0/1509607446 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5633394b91d0 0x5633394bb610 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3859fd4ad6a155c8 server_cookie=7d2f64f2f05d4fe5 in_seq=0 out_seq=0 2024-09-19T06:53:01.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.073+0000 7fd12bfff640 1 -- 172.21.15.103:0/1509607446 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56333940cb90 con 0x5633394b91d0 2024-09-19T06:53:01.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.073+0000 7fd12bfff640 10 client.16830 handle_command_reply: tid=0 2024-09-19T06:53:01.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.073+0000 7fd155466740 1 client.16830 shutdown 2024-09-19T06:53:01.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.074+0000 7fd155466740 1 -- 172.21.15.103:0/1509607446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949cb90 msgr2=0x5633394a2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:01.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.074+0000 7fd155466740 1 --2- 172.21.15.103:0/1509607446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949cb90 0x5633394a2e60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00be80 tx=0x7fd13c007680 comp rx=0 tx=0).stop 2024-09-19T06:53:01.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.074+0000 7fd155466740 1 -- 172.21.15.103:0/1509607446 shutdown_connections 2024-09-19T06:53:01.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.074+0000 7fd155466740 1 --2- 172.21.15.103:0/1509607446 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5633394b91d0 0x5633394bb610 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.074+0000 7fd155466740 1 --2- 172.21.15.103:0/1509607446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633394a7450 0x5633394b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.074+0000 7fd155466740 1 --2- 172.21.15.103:0/1509607446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949cb90 0x5633394a2e60 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.074+0000 7fd155466740 1 --2- 172.21.15.103:0/1509607446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56333949c270 0x5633394a2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.074+0000 7fd12bfff640 0 client.16830 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:01.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.074+0000 7fd155466740 1 -- 172.21.15.103:0/1509607446 >> 172.21.15.103:0/1509607446 conn(0x563339498800 msgr2=0x56333949e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:01.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.074+0000 7fd155466740 1 -- 172.21.15.103:0/1509607446 shutdown_connections 2024-09-19T06:53:01.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.074+0000 7fd155466740 1 -- 172.21.15.103:0/1509607446 wait complete. 2024-09-19T06:53:01.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.074+0000 7fd155466740 20 client.16830 trim_cache size 0 max 16384 2024-09-19T06:53:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.091+0000 7fd155466740 1 Processor -- start 2024-09-19T06:53:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.091+0000 7fd155466740 1 -- start start 2024-09-19T06:53:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.091+0000 7fd155466740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56333949c270 0x56333949fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.091+0000 7fd155466740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56333949cb90 0x563339497440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.091+0000 7fd155466740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633394a7450 0x563339497980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.091+0000 7fd155466740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c1ab810 con 0x5633394a7450 2024-09-19T06:53:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.091+0000 7fd155466740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c1ad350 con 0x56333949cb90 2024-09-19T06:53:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.091+0000 7fd155466740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c1abee0 con 0x56333949c270 2024-09-19T06:53:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.091+0000 7fd15162a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633394a7450 0x563339497980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.091+0000 7fd15162a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633394a7450 0x563339497980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33304/0 (socket says 172.21.15.103:33304) 2024-09-19T06:53:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.091+0000 7fd15162a640 1 -- 172.21.15.103:0/2912063550 learned_addr learned my addr 172.21.15.103:0/2912063550 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.091+0000 7fd150e29640 1 --2- 172.21.15.103:0/2912063550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56333949c270 0x56333949fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.092+0000 7fd14bfff640 1 --2- 172.21.15.103:0/2912063550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56333949cb90 0x563339497440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.092+0000 7fd15162a640 1 -- 172.21.15.103:0/2912063550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56333949c270 msgr2=0x56333949fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.092+0000 7fd15162a640 1 --2- 172.21.15.103:0/2912063550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56333949c270 0x56333949fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.092+0000 7fd15162a640 1 -- 172.21.15.103:0/2912063550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56333949cb90 msgr2=0x563339497440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.092+0000 7fd15162a640 1 --2- 172.21.15.103:0/2912063550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56333949cb90 0x563339497440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.092+0000 7fd15162a640 1 -- 172.21.15.103:0/2912063550 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563339494bb0 con 0x5633394a7450 2024-09-19T06:53:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.092+0000 7fd15162a640 1 --2- 172.21.15.103:0/2912063550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633394a7450 0x563339497980 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd134009230 tx=0x7fd13403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a74f7f3e0b3e219 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.092+0000 7fd1427fc640 1 -- 172.21.15.103:0/2912063550 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134035050 con 0x5633394a7450 2024-09-19T06:53:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.092+0000 7fd1427fc640 1 -- 172.21.15.103:0/2912063550 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd134003d90 con 0x5633394a7450 2024-09-19T06:53:01.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.092+0000 7fd1427fc640 1 -- 172.21.15.103:0/2912063550 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13403be00 con 0x5633394a7450 2024-09-19T06:53:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.093+0000 7fd155466740 1 -- 172.21.15.103:0/2912063550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633394a7450 msgr2=0x563339497980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.093+0000 7fd155466740 1 --2- 172.21.15.103:0/2912063550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633394a7450 0x563339497980 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd134009230 tx=0x7fd13403c000 comp rx=0 tx=0).stop 2024-09-19T06:53:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.093+0000 7fd155466740 1 -- 172.21.15.103:0/2912063550 shutdown_connections 2024-09-19T06:53:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.093+0000 7fd155466740 1 --2- 172.21.15.103:0/2912063550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633394a7450 0x563339497980 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.093+0000 7fd155466740 1 --2- 172.21.15.103:0/2912063550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56333949cb90 0x563339497440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.093+0000 7fd155466740 1 --2- 172.21.15.103:0/2912063550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56333949c270 0x56333949fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.093+0000 7fd155466740 1 -- 172.21.15.103:0/2912063550 >> 172.21.15.103:0/2912063550 conn(0x563339498800 msgr2=0x5633394a1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.093+0000 7fd155466740 1 -- 172.21.15.103:0/2912063550 shutdown_connections 2024-09-19T06:53:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.093+0000 7fd155466740 1 -- 172.21.15.103:0/2912063550 wait complete. 2024-09-19T06:53:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.093+0000 7fd155466740 1 -- start start 2024-09-19T06:53:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.093+0000 7fd155466740 1 Processor -- start 2024-09-19T06:53:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd155466740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56333949c270 0x5633394b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd155466740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949cb90 0x5633394b5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd155466740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633394a7450 0x5633394b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd155466740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c1acc80 con 0x56333949cb90 2024-09-19T06:53:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd155466740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c1ac5b0 con 0x5633394a7450 2024-09-19T06:53:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd155466740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c1a97d0 con 0x56333949c270 2024-09-19T06:53:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd155466740 10 client.0 _ll_register_callbacks cb 0x7fd13c00ba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd14bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949cb90 0x5633394b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd14bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949cb90 0x5633394b5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33316/0 (socket says 172.21.15.103:33316) 2024-09-19T06:53:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd150e29640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56333949c270 0x5633394b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd14bfff640 1 -- 172.21.15.103:0/2993826351 learned_addr learned my addr 172.21.15.103:0/2993826351 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd15162a640 1 --2- 172.21.15.103:0/2993826351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633394a7450 0x5633394b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd14bfff640 1 -- 172.21.15.103:0/2993826351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56333949c270 msgr2=0x5633394b51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd14bfff640 1 --2- 172.21.15.103:0/2993826351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56333949c270 0x5633394b51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd14bfff640 1 -- 172.21.15.103:0/2993826351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633394a7450 msgr2=0x5633394b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd14bfff640 1 --2- 172.21.15.103:0/2993826351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633394a7450 0x5633394b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd14bfff640 1 -- 172.21.15.103:0/2993826351 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633394b06e0 con 0x56333949cb90 2024-09-19T06:53:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd15162a640 1 --2- 172.21.15.103:0/2993826351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633394a7450 0x5633394b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd14bfff640 1 --2- 172.21.15.103:0/2993826351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949cb90 0x5633394b5730 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd13c000f80 tx=0x7fd13c009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180bf1a4101b1fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd12bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd12bfff640 1 -- 172.21.15.103:0/2993826351 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c02f060 con 0x56333949cb90 2024-09-19T06:53:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd12bfff640 1 -- 172.21.15.103:0/2993826351 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd13c021610 con 0x56333949cb90 2024-09-19T06:53:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd155466740 1 -- 172.21.15.103:0/2993826351 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd134003d90 con 0x56333949cb90 2024-09-19T06:53:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd12bfff640 1 -- 172.21.15.103:0/2993826351 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd13c021810 con 0x56333949cb90 2024-09-19T06:53:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.094+0000 7fd155466740 10 client.16836 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.095+0000 7fd155466740 1 -- 172.21.15.103:0/2993826351 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633395246d0 con 0x56333949cb90 2024-09-19T06:53:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.095+0000 7fd12bfff640 1 -- 172.21.15.103:0/2993826351 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd134003d90 con 0x56333949cb90 2024-09-19T06:53:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.095+0000 7fd12bfff640 1 -- 172.21.15.103:0/2993826351 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd13c02a5a0 con 0x56333949cb90 2024-09-19T06:53:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.095+0000 7fd155466740 10 client.16836 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.095+0000 7fd155466740 10 client.16836 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:53:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.095+0000 7fd155466740 20 client.16836 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.095+0000 7fd155466740 1 --2- 172.21.15.103:0/2993826351 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563339524e50 0x563339527270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.095+0000 7fd155466740 4 client.16836 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "95fcbfa8", "release": true, "await": true}] 2024-09-19T06:53:01.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.095+0000 7fd155466740 1 -- 172.21.15.103:0/2993826351 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "set_id": "95fcbfa8", "release": true, "await": true}) -- 0x5633394070a0 con 0x563339524e50 2024-09-19T06:53:01.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.095+0000 7fd150e29640 1 --2- 172.21.15.103:0/2993826351 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563339524e50 0x563339527270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:01.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.096+0000 7fd12bfff640 10 client.16836 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:01.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.096+0000 7fd150e29640 1 --2- 172.21.15.103:0/2993826351 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563339524e50 0x563339527270 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=328648cd5212eb07 server_cookie=8f17349c3ca36769 in_seq=0 out_seq=0 2024-09-19T06:53:01.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.099+0000 7fd12bfff640 1 -- 172.21.15.103:0/2993826351 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5633394070a0 con 0x563339524e50 2024-09-19T06:53:01.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.099+0000 7fd12bfff640 10 client.16836 handle_command_reply: tid=0 2024-09-19T06:53:01.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.099+0000 7fd155466740 1 client.16836 shutdown 2024-09-19T06:53:01.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.099+0000 7fd155466740 1 -- 172.21.15.103:0/2993826351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949cb90 msgr2=0x5633394b5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:01.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.099+0000 7fd155466740 1 --2- 172.21.15.103:0/2993826351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949cb90 0x5633394b5730 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd13c000f80 tx=0x7fd13c009830 comp rx=0 tx=0).stop 2024-09-19T06:53:01.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.099+0000 7fd155466740 1 -- 172.21.15.103:0/2993826351 shutdown_connections 2024-09-19T06:53:01.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.099+0000 7fd155466740 1 --2- 172.21.15.103:0/2993826351 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563339524e50 0x563339527270 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.099+0000 7fd155466740 1 --2- 172.21.15.103:0/2993826351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633394a7450 0x5633394b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.099+0000 7fd12bfff640 0 client.16836 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:01.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.099+0000 7fd155466740 1 --2- 172.21.15.103:0/2993826351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56333949cb90 0x5633394b5730 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.099+0000 7fd155466740 1 --2- 172.21.15.103:0/2993826351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56333949c270 0x5633394b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.099+0000 7fd155466740 1 -- 172.21.15.103:0/2993826351 >> 172.21.15.103:0/2993826351 conn(0x563339498800 msgr2=0x56333949dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:01.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.099+0000 7fd155466740 1 -- 172.21.15.103:0/2993826351 shutdown_connections 2024-09-19T06:53:01.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.099+0000 7fd155466740 1 -- 172.21.15.103:0/2993826351 wait complete. 2024-09-19T06:53:01.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.099+0000 7fd155466740 20 client.16836 trim_cache size 0 max 16384 2024-09-19T06:53:01.104 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:53:01.104 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:53:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:53:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 16, 2024-09-19T06:53:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:53:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "95fcbfa8": { 2024-09-19T06:53:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 16, 2024-09-19T06:53:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:53:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:53:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:53:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:53:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:53:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:53:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 136.6, 2024-09-19T06:53:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:53:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:53:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:53:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:53:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:53:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:53:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:53:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:53:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:53:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:53:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:53:01.107 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:53:01.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.100+0000 7fd15308d640 1 -- 172.21.15.103:0/2004466533 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1380a33b0 msgr2=0x7fd1380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:01.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.100+0000 7fd15308d640 1 --2- 172.21.15.103:0/2004466533 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1380a33b0 0x7fd1380a5870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd134005ab0 tx=0x7fd134005a20 comp rx=0 tx=0).stop 2024-09-19T06:53:01.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.100+0000 7fd15308d640 1 -- 172.21.15.103:0/2004466533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd14c100a80 msgr2=0x7fd14c197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:01.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.100+0000 7fd15308d640 1 --2- 172.21.15.103:0/2004466533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd14c100a80 0x7fd14c197480 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd14400b4a0 tx=0x7fd14400b960 comp rx=0 tx=0).stop 2024-09-19T06:53:01.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.100+0000 7fd15308d640 1 -- 172.21.15.103:0/2004466533 shutdown_connections 2024-09-19T06:53:01.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.100+0000 7fd15308d640 1 --2- 172.21.15.103:0/2004466533 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1380a33b0 0x7fd1380a5870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.100+0000 7fd15308d640 1 --2- 172.21.15.103:0/2004466533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd14c106a80 0x7fd14c19bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.100+0000 7fd15308d640 1 --2- 172.21.15.103:0/2004466533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd14c101420 0x7fd14c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.100+0000 7fd15308d640 1 --2- 172.21.15.103:0/2004466533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd14c100a80 0x7fd14c197480 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:01.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.100+0000 7fd15308d640 1 -- 172.21.15.103:0/2004466533 >> 172.21.15.103:0/2004466533 conn(0x7fd14c0fc7a0 msgr2=0x7fd14c0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:01.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.100+0000 7fd15308d640 1 -- 172.21.15.103:0/2004466533 shutdown_connections 2024-09-19T06:53:01.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:01.100+0000 7fd15308d640 1 -- 172.21.15.103:0/2004466533 wait complete. 2024-09-19T06:53:01.519 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:53:01.519 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:53:01.874 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '95fcbfa8', total seconds elapsed: 21.1 2024-09-19T06:53:01.953 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:02.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:02 smithi103 ceph-mon[31158]: pgmap v117: 97 pgs: 97 active+clean; 3.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.1 KiB/s wr, 11 op/s 2024-09-19T06:53:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:02 smithi174 ceph-mon[35637]: pgmap v117: 97 pgs: 97 active+clean; 3.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.1 KiB/s wr, 11 op/s 2024-09-19T06:53:02.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.374+0000 7f8bdecc1640 1 -- 172.21.15.103:0/2273606557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bd8101df0 msgr2=0x7f8bd81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:02.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.374+0000 7f8bdecc1640 1 --2- 172.21.15.103:0/2273606557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bd8101df0 0x7f8bd81021d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009900 tx=0x7f8bcc02a410 comp rx=0 tx=0).stop 2024-09-19T06:53:02.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.374+0000 7f8bdecc1640 1 -- 172.21.15.103:0/2273606557 shutdown_connections 2024-09-19T06:53:02.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.374+0000 7f8bdecc1640 1 --2- 172.21.15.103:0/2273606557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bd8106f20 0x7f8bd810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.374+0000 7f8bdecc1640 1 --2- 172.21.15.103:0/2273606557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bd81027a0 0x7f8bd81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.374+0000 7f8bdecc1640 1 --2- 172.21.15.103:0/2273606557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bd8101df0 0x7f8bd81021d0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.374+0000 7f8bdecc1640 1 -- 172.21.15.103:0/2273606557 >> 172.21.15.103:0/2273606557 conn(0x7f8bd80fd8b0 msgr2=0x7f8bd80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:02.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.374+0000 7f8bdecc1640 1 -- 172.21.15.103:0/2273606557 shutdown_connections 2024-09-19T06:53:02.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.374+0000 7f8bdecc1640 1 -- 172.21.15.103:0/2273606557 wait complete. 2024-09-19T06:53:02.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.374+0000 7f8bdecc1640 1 Processor -- start 2024-09-19T06:53:02.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdecc1640 1 -- start start 2024-09-19T06:53:02.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdecc1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bd8101df0 0x7f8bd80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdecc1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bd81027a0 0x7f8bd8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdecc1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bd8106f20 0x7f8bd8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdecc1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8111fc0 con 0x7f8bd8106f20 2024-09-19T06:53:02.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdecc1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8111e40 con 0x7f8bd81027a0 2024-09-19T06:53:02.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdecc1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8112140 con 0x7f8bd8101df0 2024-09-19T06:53:02.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdd25e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bd8106f20 0x7f8bd8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:02.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdca5d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bd8101df0 0x7f8bd80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:02.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdd25e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bd8106f20 0x7f8bd8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33334/0 (socket says 172.21.15.103:33334) 2024-09-19T06:53:02.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdd25e640 1 -- 172.21.15.103:0/1530394231 learned_addr learned my addr 172.21.15.103:0/1530394231 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:02.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdca5d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bd8101df0 0x7f8bd80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56508/0 (socket says 172.21.15.103:56508) 2024-09-19T06:53:02.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bd7fff640 1 --2- 172.21.15.103:0/1530394231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bd81027a0 0x7f8bd8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:02.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdd25e640 1 -- 172.21.15.103:0/1530394231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bd8101df0 msgr2=0x7f8bd80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:02.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdd25e640 1 --2- 172.21.15.103:0/1530394231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bd8101df0 0x7f8bd80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdd25e640 1 -- 172.21.15.103:0/1530394231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bd81027a0 msgr2=0x7f8bd8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:02.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdd25e640 1 --2- 172.21.15.103:0/1530394231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bd81027a0 0x7f8bd8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdd25e640 1 -- 172.21.15.103:0/1530394231 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd81a6950 con 0x7f8bd8106f20 2024-09-19T06:53:02.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.375+0000 7f8bdd25e640 1 --2- 172.21.15.103:0/1530394231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bd8106f20 0x7f8bd8074f70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000b970 tx=0x7f8bd000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:02.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.376+0000 7f8bd5ffb640 1 -- 172.21.15.103:0/1530394231 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0019070 con 0x7f8bd8106f20 2024-09-19T06:53:02.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.376+0000 7f8bdecc1640 1 -- 172.21.15.103:0/1530394231 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd81a6c40 con 0x7f8bd8106f20 2024-09-19T06:53:02.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.376+0000 7f8bd5ffb640 1 -- 172.21.15.103:0/1530394231 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd000cb40 con 0x7f8bd8106f20 2024-09-19T06:53:02.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.376+0000 7f8bd5ffb640 1 -- 172.21.15.103:0/1530394231 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0007650 con 0x7f8bd8106f20 2024-09-19T06:53:02.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.376+0000 7f8bdecc1640 1 -- 172.21.15.103:0/1530394231 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd81a7130 con 0x7f8bd8106f20 2024-09-19T06:53:02.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.379+0000 7f8bd5ffb640 1 -- 172.21.15.103:0/1530394231 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8bd001c070 con 0x7f8bd8106f20 2024-09-19T06:53:02.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.379+0000 7f8be109a740 1 Processor -- start 2024-09-19T06:53:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.379+0000 7f8be109a740 1 -- start start 2024-09-19T06:53:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.379+0000 7f8be109a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3cb60 0x557912e3cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.379+0000 7f8be109a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3d480 0x557912e47800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.379+0000 7f8be109a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e47d40 0x557912e4a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.379+0000 7f8be109a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557912da2280 con 0x557912e3d480 2024-09-19T06:53:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.379+0000 7f8be109a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557912d49b10 con 0x557912e3cb60 2024-09-19T06:53:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.379+0000 7f8be109a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557912d9e240 con 0x557912e47d40 2024-09-19T06:53:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.379+0000 7f8bd5ffb640 1 --2- 172.21.15.103:0/1530394231 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8bc80a33b0 0x7f8bc80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.379+0000 7f8bdca5d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3d480 0x557912e47800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:02.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.379+0000 7f8bdca5d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3d480 0x557912e47800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33342/0 (socket says 172.21.15.103:33342) 2024-09-19T06:53:02.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.379+0000 7f8bdca5d640 1 -- 172.21.15.103:0/2514126077 learned_addr learned my addr 172.21.15.103:0/2514126077 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:02.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.379+0000 7f8bdd25e640 1 --2- 172.21.15.103:0/1530394231 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8bc80a33b0 0x7f8bc80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:02.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.379+0000 7f8bd7fff640 1 --2- 172.21.15.103:0/2514126077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3cb60 0x557912e3cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:02.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.380+0000 7f8bd7fff640 1 --2- 172.21.15.103:0/2514126077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e47d40 0x557912e4a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:02.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.380+0000 7f8bd5ffb640 1 -- 172.21.15.103:0/1530394231 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f8bd00cf1e0 con 0x7f8bd8106f20 2024-09-19T06:53:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.380+0000 7f8bdca5d640 1 -- 172.21.15.103:0/2514126077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e47d40 msgr2=0x557912e4a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.380+0000 7f8bdca5d640 1 --2- 172.21.15.103:0/2514126077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e47d40 0x557912e4a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.380+0000 7f8bdca5d640 1 -- 172.21.15.103:0/2514126077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3cb60 msgr2=0x557912e3cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.380+0000 7f8bdca5d640 1 --2- 172.21.15.103:0/2514126077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3cb60 0x557912e3cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.380+0000 7f8bdca5d640 1 -- 172.21.15.103:0/2514126077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557912d561f0 con 0x557912e3d480 2024-09-19T06:53:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.380+0000 7f8bd7fff640 1 --2- 172.21.15.103:0/2514126077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3cb60 0x557912e3cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.380+0000 7f8bdd25e640 1 --2- 172.21.15.103:0/1530394231 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8bc80a33b0 0x7f8bc80a5870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8bd009b390 tx=0x7f8bd0097330 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.380+0000 7f8bdca5d640 1 --2- 172.21.15.103:0/2514126077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3d480 0x557912e47800 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009900 tx=0x7f8bcc031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eeaa15d6067dbab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.380+0000 7f8bc2ffd640 1 -- 172.21.15.103:0/2514126077 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc040070 con 0x557912e3d480 2024-09-19T06:53:02.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.380+0000 7f8bc2ffd640 1 -- 172.21.15.103:0/2514126077 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bcc005020 con 0x557912e3d480 2024-09-19T06:53:02.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.380+0000 7f8bc2ffd640 1 -- 172.21.15.103:0/2514126077 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc035040 con 0x557912e3d480 2024-09-19T06:53:02.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.381+0000 7f8be109a740 1 -- 172.21.15.103:0/2514126077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3d480 msgr2=0x557912e47800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:02.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.381+0000 7f8be109a740 1 --2- 172.21.15.103:0/2514126077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3d480 0x557912e47800 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009900 tx=0x7f8bcc031630 comp rx=0 tx=0).stop 2024-09-19T06:53:02.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.381+0000 7f8be109a740 1 -- 172.21.15.103:0/2514126077 shutdown_connections 2024-09-19T06:53:02.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.381+0000 7f8be109a740 1 --2- 172.21.15.103:0/2514126077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e47d40 0x557912e4a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.381+0000 7f8be109a740 1 --2- 172.21.15.103:0/2514126077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3d480 0x557912e47800 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.381+0000 7f8be109a740 1 --2- 172.21.15.103:0/2514126077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3cb60 0x557912e3cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.381+0000 7f8be109a740 1 -- 172.21.15.103:0/2514126077 >> 172.21.15.103:0/2514126077 conn(0x557912e390f0 msgr2=0x557912e394d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:02.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.381+0000 7f8be109a740 1 -- 172.21.15.103:0/2514126077 shutdown_connections 2024-09-19T06:53:02.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.381+0000 7f8be109a740 1 -- 172.21.15.103:0/2514126077 wait complete. 2024-09-19T06:53:02.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.381+0000 7f8be109a740 1 -- start start 2024-09-19T06:53:02.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.381+0000 7f8be109a740 1 Processor -- start 2024-09-19T06:53:02.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8be109a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3cb60 0x557912e47800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8be109a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3d480 0x557912e44b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8be109a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e47d40 0x557912e452e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8be109a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557912e4a910 con 0x557912e3cb60 2024-09-19T06:53:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8be109a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557912d5cf10 con 0x557912e3d480 2024-09-19T06:53:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8be109a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557912e4aa90 con 0x557912e47d40 2024-09-19T06:53:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8be109a740 10 client.0 _ll_register_callbacks cb 0x557912d25a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8bd7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3cb60 0x557912e47800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8bd7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3cb60 0x557912e47800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33346/0 (socket says 172.21.15.103:33346) 2024-09-19T06:53:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8bd7fff640 1 -- 172.21.15.103:0/2493835539 learned_addr learned my addr 172.21.15.103:0/2493835539 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8bdca5d640 1 --2- 172.21.15.103:0/2493835539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3d480 0x557912e44b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:02.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8bd7fff640 1 -- 172.21.15.103:0/2493835539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e47d40 msgr2=0x557912e452e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8bd7fff640 1 --2- 172.21.15.103:0/2493835539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e47d40 0x557912e452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8bd7fff640 1 -- 172.21.15.103:0/2493835539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3d480 msgr2=0x557912e44b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8bd7fff640 1 --2- 172.21.15.103:0/2493835539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3d480 0x557912e44b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8bd7fff640 1 -- 172.21.15.103:0/2493835539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557912b136a0 con 0x557912e3cb60 2024-09-19T06:53:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8bdca5d640 1 --2- 172.21.15.103:0/2493835539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3d480 0x557912e44b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8bd7fff640 1 --2- 172.21.15.103:0/2493835539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3cb60 0x557912e47800 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40057b0 tx=0x7f8bc4012d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856d581a05b319ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.382+0000 7f8bb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.383+0000 7f8bb7fff640 1 -- 172.21.15.103:0/2493835539 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc400fe10 con 0x557912e3cb60 2024-09-19T06:53:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.383+0000 7f8bb7fff640 1 -- 172.21.15.103:0/2493835539 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f8bc4010430 con 0x557912e3cb60 2024-09-19T06:53:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.383+0000 7f8be109a740 1 -- 172.21.15.103:0/2493835539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bcc005020 con 0x557912e3cb60 2024-09-19T06:53:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.383+0000 7f8bb7fff640 1 -- 172.21.15.103:0/2493835539 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8bc4010630 con 0x557912e3cb60 2024-09-19T06:53:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.383+0000 7f8be109a740 10 client.16848 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.383+0000 7f8be109a740 1 -- 172.21.15.103:0/2493835539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557912e564d0 con 0x557912e3cb60 2024-09-19T06:53:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.383+0000 7f8bb7fff640 1 -- 172.21.15.103:0/2493835539 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bcc005020 con 0x557912e3cb60 2024-09-19T06:53:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.383+0000 7f8bb7fff640 1 -- 172.21.15.103:0/2493835539 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8bc40269b0 con 0x557912e3cb60 2024-09-19T06:53:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.383+0000 7f8be109a740 10 client.16848 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:02.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.383+0000 7f8be109a740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:02.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.383+0000 7f8be109a740 20 client.16848 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:02.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.383+0000 7f8be109a740 1 --2- 172.21.15.103:0/2493835539 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557912e578e0 0x557912e59cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.383+0000 7f8be109a740 4 client.16848 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:02.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.383+0000 7f8be109a740 1 -- 172.21.15.103:0/2493835539 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557912daf7f0 con 0x557912e578e0 2024-09-19T06:53:02.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.383+0000 7f8bdca5d640 1 --2- 172.21.15.103:0/2493835539 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557912e578e0 0x557912e59cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:02.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.384+0000 7f8bb7fff640 10 client.16848 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:02.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.384+0000 7f8bdca5d640 1 --2- 172.21.15.103:0/2493835539 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557912e578e0 0x557912e59cd0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3772b46fb985e7 server_cookie=fd2a25733f5cc3b5 in_seq=0 out_seq=0 2024-09-19T06:53:02.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.385+0000 7f8bb7fff640 1 -- 172.21.15.103:0/2493835539 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557912daf7f0 con 0x557912e578e0 2024-09-19T06:53:02.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.385+0000 7f8bb7fff640 10 client.16848 handle_command_reply: tid=0 2024-09-19T06:53:02.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.385+0000 7f8be109a740 1 client.16848 shutdown 2024-09-19T06:53:02.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.385+0000 7f8be109a740 1 -- 172.21.15.103:0/2493835539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3cb60 msgr2=0x557912e47800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:02.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.385+0000 7f8be109a740 1 --2- 172.21.15.103:0/2493835539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3cb60 0x557912e47800 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40057b0 tx=0x7f8bc4012d90 comp rx=0 tx=0).stop 2024-09-19T06:53:02.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.385+0000 7f8be109a740 1 -- 172.21.15.103:0/2493835539 shutdown_connections 2024-09-19T06:53:02.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.385+0000 7f8be109a740 1 --2- 172.21.15.103:0/2493835539 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557912e578e0 0x557912e59cd0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.385+0000 7f8be109a740 1 --2- 172.21.15.103:0/2493835539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e47d40 0x557912e452e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.385+0000 7f8be109a740 1 --2- 172.21.15.103:0/2493835539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3d480 0x557912e44b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.385+0000 7f8be109a740 1 --2- 172.21.15.103:0/2493835539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3cb60 0x557912e47800 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.385+0000 7f8bb7fff640 0 client.16848 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:02.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.385+0000 7f8be109a740 1 -- 172.21.15.103:0/2493835539 >> 172.21.15.103:0/2493835539 conn(0x557912e390f0 msgr2=0x557912e38270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:02.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.385+0000 7f8be109a740 1 -- 172.21.15.103:0/2493835539 shutdown_connections 2024-09-19T06:53:02.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.385+0000 7f8be109a740 1 -- 172.21.15.103:0/2493835539 wait complete. 2024-09-19T06:53:02.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.385+0000 7f8be109a740 20 client.16848 trim_cache size 0 max 16384 2024-09-19T06:53:02.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8be109a740 1 Processor -- start 2024-09-19T06:53:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8be109a740 1 -- start start 2024-09-19T06:53:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8be109a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3cb60 0x557912e5f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8be109a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3d480 0x557912e5fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8be109a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e47d40 0x557912e41db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8be109a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81afa70 con 0x557912e3d480 2024-09-19T06:53:02.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8be109a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81b15b0 con 0x557912e3cb60 2024-09-19T06:53:02.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8be109a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81b0140 con 0x557912e47d40 2024-09-19T06:53:02.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8bdca5d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3d480 0x557912e5fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:02.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8bdca5d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3d480 0x557912e5fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33352/0 (socket says 172.21.15.103:33352) 2024-09-19T06:53:02.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8bdca5d640 1 -- 172.21.15.103:0/3464834995 learned_addr learned my addr 172.21.15.103:0/3464834995 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:02.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8bd7fff640 1 --2- 172.21.15.103:0/3464834995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3cb60 0x557912e5f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:02.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8bdca5d640 1 -- 172.21.15.103:0/3464834995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e47d40 msgr2=0x557912e41db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:02.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8bdca5d640 1 --2- 172.21.15.103:0/3464834995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e47d40 0x557912e41db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8bdca5d640 1 -- 172.21.15.103:0/3464834995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3cb60 msgr2=0x557912e5f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:02.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8bdca5d640 1 --2- 172.21.15.103:0/3464834995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3cb60 0x557912e5f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8bdca5d640 1 -- 172.21.15.103:0/3464834995 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557912e45650 con 0x557912e3d480 2024-09-19T06:53:02.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.402+0000 7f8bd7fff640 1 --2- 172.21.15.103:0/3464834995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3cb60 0x557912e5f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:02.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.403+0000 7f8bdca5d640 1 --2- 172.21.15.103:0/3464834995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3d480 0x557912e5fa90 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc005940 tx=0x7f8bcc0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9146760f0cca2de5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:02.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.403+0000 7f8bc27fc640 1 -- 172.21.15.103:0/3464834995 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc040040 con 0x557912e3d480 2024-09-19T06:53:02.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.403+0000 7f8bc27fc640 1 -- 172.21.15.103:0/3464834995 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bcc005450 con 0x557912e3d480 2024-09-19T06:53:02.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.403+0000 7f8bc27fc640 1 -- 172.21.15.103:0/3464834995 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc035040 con 0x557912e3d480 2024-09-19T06:53:02.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.403+0000 7f8be109a740 1 -- 172.21.15.103:0/3464834995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3d480 msgr2=0x557912e5fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:02.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.403+0000 7f8be109a740 1 --2- 172.21.15.103:0/3464834995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3d480 0x557912e5fa90 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc005940 tx=0x7f8bcc0058d0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.403+0000 7f8be109a740 1 -- 172.21.15.103:0/3464834995 shutdown_connections 2024-09-19T06:53:02.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.403+0000 7f8be109a740 1 --2- 172.21.15.103:0/3464834995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e47d40 0x557912e41db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.403+0000 7f8be109a740 1 --2- 172.21.15.103:0/3464834995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e3d480 0x557912e5fa90 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.403+0000 7f8be109a740 1 --2- 172.21.15.103:0/3464834995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3cb60 0x557912e5f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.403+0000 7f8be109a740 1 -- 172.21.15.103:0/3464834995 >> 172.21.15.103:0/3464834995 conn(0x557912e390f0 msgr2=0x557912e43420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:02.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.403+0000 7f8be109a740 1 -- 172.21.15.103:0/3464834995 shutdown_connections 2024-09-19T06:53:02.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.403+0000 7f8be109a740 1 -- 172.21.15.103:0/3464834995 wait complete. 2024-09-19T06:53:02.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.404+0000 7f8be109a740 1 -- start start 2024-09-19T06:53:02.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.404+0000 7f8be109a740 1 Processor -- start 2024-09-19T06:53:02.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.404+0000 7f8be109a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e3cb60 0x557912e4c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.404+0000 7f8be109a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3d480 0x557912e4c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.404+0000 7f8be109a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e47d40 0x557912ec1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.404+0000 7f8be109a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81b0ee0 con 0x557912e47d40 2024-09-19T06:53:02.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.404+0000 7f8be109a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81b0810 con 0x557912e3d480 2024-09-19T06:53:02.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.404+0000 7f8be109a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81ada30 con 0x557912e3cb60 2024-09-19T06:53:02.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.404+0000 7f8be109a740 10 client.0 _ll_register_callbacks cb 0x7f8bc4012540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:02.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.404+0000 7f8bd7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e47d40 0x557912ec1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:02.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.404+0000 7f8bd7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e47d40 0x557912ec1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33356/0 (socket says 172.21.15.103:33356) 2024-09-19T06:53:02.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.404+0000 7f8bd7fff640 1 -- 172.21.15.103:0/1967988017 learned_addr learned my addr 172.21.15.103:0/1967988017 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:02.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.404+0000 7f8bdca5d640 1 --2- 172.21.15.103:0/1967988017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3d480 0x557912e4c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:02.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.405+0000 7f8bd7fff640 1 --2- 172.21.15.103:0/1967988017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e3cb60 0x557912e4c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:02.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.405+0000 7f8bd7fff640 1 -- 172.21.15.103:0/1967988017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e3cb60 msgr2=0x557912e4c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:02.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.405+0000 7f8bd7fff640 1 --2- 172.21.15.103:0/1967988017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e3cb60 0x557912e4c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.405+0000 7f8bd7fff640 1 -- 172.21.15.103:0/1967988017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3d480 msgr2=0x557912e4c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:02.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.405+0000 7f8bd7fff640 1 --2- 172.21.15.103:0/1967988017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3d480 0x557912e4c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.405+0000 7f8bd7fff640 1 -- 172.21.15.103:0/1967988017 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557912e5a040 con 0x557912e47d40 2024-09-19T06:53:02.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.405+0000 7f8bd7fff640 1 --2- 172.21.15.103:0/1967988017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e47d40 0x557912ec1520 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400e620 tx=0x7f8bc4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd609758ab6220c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:02.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.405+0000 7f8bb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:02.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.405+0000 7f8bb7fff640 1 -- 172.21.15.103:0/1967988017 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4005240 con 0x557912e47d40 2024-09-19T06:53:02.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.405+0000 7f8bb7fff640 1 -- 172.21.15.103:0/1967988017 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f8bc40053e0 con 0x557912e47d40 2024-09-19T06:53:02.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.405+0000 7f8be109a740 1 -- 172.21.15.103:0/1967988017 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bcc005450 con 0x557912e47d40 2024-09-19T06:53:02.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.405+0000 7f8bb7fff640 1 -- 172.21.15.103:0/1967988017 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8bc400fe20 con 0x557912e47d40 2024-09-19T06:53:02.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.405+0000 7f8be109a740 10 client.16854 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:02.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.405+0000 7f8be109a740 1 -- 172.21.15.103:0/1967988017 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557912e47b70 con 0x557912e47d40 2024-09-19T06:53:02.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.405+0000 7f8bb7fff640 1 -- 172.21.15.103:0/1967988017 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bcc005450 con 0x557912e47d40 2024-09-19T06:53:02.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.405+0000 7f8bb7fff640 1 -- 172.21.15.103:0/1967988017 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8bc40266d0 con 0x557912e47d40 2024-09-19T06:53:02.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.406+0000 7f8be109a740 10 client.16854 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:02.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.406+0000 7f8be109a740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:02.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.406+0000 7f8be109a740 20 client.16854 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:02.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.406+0000 7f8be109a740 1 --2- 172.21.15.103:0/1967988017 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557912ec3100 0x557912ec5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:02.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.406+0000 7f8be109a740 4 client.16854 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:53:02.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.406+0000 7f8be109a740 1 -- 172.21.15.103:0/1967988017 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557912da9d00 con 0x557912ec3100 2024-09-19T06:53:02.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.406+0000 7f8bdca5d640 1 --2- 172.21.15.103:0/1967988017 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557912ec3100 0x557912ec5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:02.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.406+0000 7f8bb7fff640 10 client.16854 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:02.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.406+0000 7f8bdca5d640 1 --2- 172.21.15.103:0/1967988017 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557912ec3100 0x557912ec5520 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d63b978febcbebdd server_cookie=92b0f3c7f3e831d3 in_seq=0 out_seq=0 2024-09-19T06:53:02.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.407+0000 7f8bb7fff640 1 -- 172.21.15.103:0/1967988017 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557912da9d00 con 0x557912ec3100 2024-09-19T06:53:02.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.407+0000 7f8bb7fff640 10 client.16854 handle_command_reply: tid=0 2024-09-19T06:53:02.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.407+0000 7f8be109a740 1 client.16854 shutdown 2024-09-19T06:53:02.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.407+0000 7f8be109a740 1 -- 172.21.15.103:0/1967988017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e47d40 msgr2=0x557912ec1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:02.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.407+0000 7f8be109a740 1 --2- 172.21.15.103:0/1967988017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e47d40 0x557912ec1520 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400e620 tx=0x7f8bc4001e70 comp rx=0 tx=0).stop 2024-09-19T06:53:02.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.407+0000 7f8be109a740 1 -- 172.21.15.103:0/1967988017 shutdown_connections 2024-09-19T06:53:02.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.407+0000 7f8be109a740 1 --2- 172.21.15.103:0/1967988017 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557912ec3100 0x557912ec5520 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.407+0000 7f8be109a740 1 --2- 172.21.15.103:0/1967988017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557912e47d40 0x557912ec1520 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.407+0000 7f8be109a740 1 --2- 172.21.15.103:0/1967988017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557912e3d480 0x557912e4c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.407+0000 7f8be109a740 1 --2- 172.21.15.103:0/1967988017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557912e3cb60 0x557912e4c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.407+0000 7f8bb7fff640 0 client.16854 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:02.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.407+0000 7f8be109a740 1 -- 172.21.15.103:0/1967988017 >> 172.21.15.103:0/1967988017 conn(0x557912e390f0 msgr2=0x557912e43810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:02.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.408+0000 7f8be109a740 1 -- 172.21.15.103:0/1967988017 shutdown_connections 2024-09-19T06:53:02.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.408+0000 7f8be109a740 1 -- 172.21.15.103:0/1967988017 wait complete. 2024-09-19T06:53:02.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.408+0000 7f8be109a740 20 client.16854 trim_cache size 0 max 16384 2024-09-19T06:53:02.432 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:53:02.432 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T06:53:02.432 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "60b41e7b-0a92-45e0-bef7-1c665dd58b29", 2024-09-19T06:53:02.432 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T06:53:02.432 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:53:02.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.408+0000 7f8bdecc1640 1 -- 172.21.15.103:0/1530394231 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8bc80a33b0 msgr2=0x7f8bc80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:02.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.408+0000 7f8bdecc1640 1 --2- 172.21.15.103:0/1530394231 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8bc80a33b0 0x7f8bc80a5870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8bd009b390 tx=0x7f8bd0097330 comp rx=0 tx=0).stop 2024-09-19T06:53:02.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.408+0000 7f8bdecc1640 1 -- 172.21.15.103:0/1530394231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bd8106f20 msgr2=0x7f8bd8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:02.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.408+0000 7f8bdecc1640 1 --2- 172.21.15.103:0/1530394231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bd8106f20 0x7f8bd8074f70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000b970 tx=0x7f8bd000be30 comp rx=0 tx=0).stop 2024-09-19T06:53:02.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.408+0000 7f8bdecc1640 1 -- 172.21.15.103:0/1530394231 shutdown_connections 2024-09-19T06:53:02.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.408+0000 7f8bdecc1640 1 --2- 172.21.15.103:0/1530394231 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8bc80a33b0 0x7f8bc80a5870 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.408+0000 7f8bdecc1640 1 --2- 172.21.15.103:0/1530394231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bd8106f20 0x7f8bd8074f70 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.408+0000 7f8bdecc1640 1 --2- 172.21.15.103:0/1530394231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bd81027a0 0x7f8bd8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.408+0000 7f8bdecc1640 1 --2- 172.21.15.103:0/1530394231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bd8101df0 0x7f8bd80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:02.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.408+0000 7f8bdecc1640 1 -- 172.21.15.103:0/1530394231 >> 172.21.15.103:0/1530394231 conn(0x7f8bd80fd8b0 msgr2=0x7f8bd80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:02.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.408+0000 7f8bdecc1640 1 -- 172.21.15.103:0/1530394231 shutdown_connections 2024-09-19T06:53:02.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:02.408+0000 7f8bdecc1640 1 -- 172.21.15.103:0/1530394231 wait complete. 2024-09-19T06:53:02.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:02 smithi134 ceph-mon[35356]: pgmap v117: 97 pgs: 97 active+clean; 3.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.1 KiB/s wr, 11 op/s 2024-09-19T06:53:03.216 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:53:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:03 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T06:53:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:03 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:03 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T06:53:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:03 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:03.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:03 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T06:53:03.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:03 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:03.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:03 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T06:53:03.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:03 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:03.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:03 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T06:53:03.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:03 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:03.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:03 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T06:53:03.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:03 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:03.463 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:03.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.897+0000 7f6a48930640 1 -- 172.21.15.103:0/4198302387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a44108290 msgr2=0x7f6a4410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:03.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.897+0000 7f6a48930640 1 --2- 172.21.15.103:0/4198302387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a44108290 0x7f6a4410ec30 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009cd0 tx=0x7f6a3402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.898+0000 7f6a48930640 1 -- 172.21.15.103:0/4198302387 shutdown_connections 2024-09-19T06:53:03.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.898+0000 7f6a48930640 1 --2- 172.21.15.103:0/4198302387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a44108290 0x7f6a4410ec30 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.898+0000 7f6a48930640 1 --2- 172.21.15.103:0/4198302387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a44103740 0x7f6a44107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.898+0000 7f6a48930640 1 --2- 172.21.15.103:0/4198302387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a44102d90 0x7f6a44103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.898+0000 7f6a48930640 1 -- 172.21.15.103:0/4198302387 >> 172.21.15.103:0/4198302387 conn(0x7f6a440fc7d0 msgr2=0x7f6a440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:03.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.898+0000 7f6a48930640 1 -- 172.21.15.103:0/4198302387 shutdown_connections 2024-09-19T06:53:03.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.898+0000 7f6a48930640 1 -- 172.21.15.103:0/4198302387 wait complete. 2024-09-19T06:53:03.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.898+0000 7f6a48930640 1 Processor -- start 2024-09-19T06:53:03.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.898+0000 7f6a48930640 1 -- start start 2024-09-19T06:53:03.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a48930640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a44102d90 0x7f6a44198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a48930640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a44103740 0x7f6a44198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a48930640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a44108290 0x7f6a4419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a48930640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a44112090 con 0x7f6a44108290 2024-09-19T06:53:03.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a48930640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a44111f10 con 0x7f6a44103740 2024-09-19T06:53:03.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a48930640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6a44112210 con 0x7f6a44102d90 2024-09-19T06:53:03.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a42d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a44108290 0x7f6a4419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a42d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a44108290 0x7f6a4419ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33372/0 (socket says 172.21.15.103:33372) 2024-09-19T06:53:03.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a42d9d640 1 -- 172.21.15.103:0/3247155963 learned_addr learned my addr 172.21.15.103:0/3247155963 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:03.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a4259c640 1 --2- 172.21.15.103:0/3247155963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a44102d90 0x7f6a44198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a41d9b640 1 --2- 172.21.15.103:0/3247155963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a44103740 0x7f6a44198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a42d9d640 1 -- 172.21.15.103:0/3247155963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a44102d90 msgr2=0x7f6a44198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:03.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a42d9d640 1 --2- 172.21.15.103:0/3247155963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a44102d90 0x7f6a44198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a42d9d640 1 -- 172.21.15.103:0/3247155963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a44103740 msgr2=0x7f6a44198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:03.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a42d9d640 1 --2- 172.21.15.103:0/3247155963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a44103740 0x7f6a44198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a42d9d640 1 -- 172.21.15.103:0/3247155963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4419d4f0 con 0x7f6a44108290 2024-09-19T06:53:03.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a42d9d640 1 --2- 172.21.15.103:0/3247155963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a44108290 0x7f6a4419ce10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6a34005cc0 tx=0x7f6a34003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:03.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a337fe640 1 -- 172.21.15.103:0/3247155963 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34004040 con 0x7f6a44108290 2024-09-19T06:53:03.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a337fe640 1 -- 172.21.15.103:0/3247155963 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a340046c0 con 0x7f6a44108290 2024-09-19T06:53:03.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a337fe640 1 -- 172.21.15.103:0/3247155963 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34046480 con 0x7f6a44108290 2024-09-19T06:53:03.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.899+0000 7f6a48930640 1 -- 172.21.15.103:0/3247155963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4419d780 con 0x7f6a44108290 2024-09-19T06:53:03.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.900+0000 7f6a48930640 1 -- 172.21.15.103:0/3247155963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a441a50c0 con 0x7f6a44108290 2024-09-19T06:53:03.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.902+0000 7f6a337fe640 1 -- 172.21.15.103:0/3247155963 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f6a340305a0 con 0x7f6a44108290 2024-09-19T06:53:03.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.902+0000 7f6a4ad09740 1 Processor -- start 2024-09-19T06:53:03.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.902+0000 7f6a4ad09740 1 -- start start 2024-09-19T06:53:03.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a4ad09740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa07e4a0 0x55a5fa07e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a4ad09740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 0x55a5fa089140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a4ad09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa089680 0x55a5fa08ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a4ad09740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a5f9fe42e0 con 0x55a5fa07e4a0 2024-09-19T06:53:03.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a4ad09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a5f9f8bce0 con 0x55a5fa089680 2024-09-19T06:53:03.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a4ad09740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a5f9fe02a0 con 0x55a5fa07edc0 2024-09-19T06:53:03.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a337fe640 1 --2- 172.21.15.103:0/3247155963 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6a2c0a3430 0x7f6a2c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa07e4a0 0x55a5fa07e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa07e4a0 0x55a5fa07e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33376/0 (socket says 172.21.15.103:33376) 2024-09-19T06:53:03.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a41d9b640 1 -- 172.21.15.103:0/3837794126 learned_addr learned my addr 172.21.15.103:0/3837794126 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:03.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a42d9d640 1 --2- 172.21.15.103:0/3247155963 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6a2c0a3430 0x7f6a2c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a4259c640 1 --2- 172.21.15.103:0/3837794126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 0x55a5fa089140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a337fe640 1 -- 172.21.15.103:0/3247155963 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6a3402e030 con 0x7f6a44108290 2024-09-19T06:53:03.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a41d9b640 1 --2- 172.21.15.103:0/3837794126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa089680 0x55a5fa08ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a41d9b640 1 -- 172.21.15.103:0/3837794126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 msgr2=0x55a5fa089140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:03.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a41d9b640 1 --2- 172.21.15.103:0/3837794126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 0x55a5fa089140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a41d9b640 1 -- 172.21.15.103:0/3837794126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa089680 msgr2=0x55a5fa08ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:03.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a41d9b640 1 --2- 172.21.15.103:0/3837794126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa089680 0x55a5fa08ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a41d9b640 1 -- 172.21.15.103:0/3837794126 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5f9f70440 con 0x55a5fa07e4a0 2024-09-19T06:53:03.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a4259c640 1 --2- 172.21.15.103:0/3837794126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 0x55a5fa089140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:03.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a41d9b640 1 --2- 172.21.15.103:0/3837794126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa07e4a0 0x55a5fa07e880 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6a380058a0 tx=0x7f6a3800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebe14070733bcfc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:03.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a42d9d640 1 --2- 172.21.15.103:0/3247155963 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6a2c0a3430 0x7f6a2c0a58f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6a340bd6c0 tx=0x7f6a3408f4b0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:03.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.903+0000 7f6a30ff9640 1 -- 172.21.15.103:0/3837794126 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a380120d0 con 0x55a5fa07e4a0 2024-09-19T06:53:03.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.904+0000 7f6a30ff9640 1 -- 172.21.15.103:0/3837794126 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a38005020 con 0x55a5fa07e4a0 2024-09-19T06:53:03.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.904+0000 7f6a30ff9640 1 -- 172.21.15.103:0/3837794126 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38004a90 con 0x55a5fa07e4a0 2024-09-19T06:53:03.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.904+0000 7f6a4ad09740 1 -- 172.21.15.103:0/3837794126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa07e4a0 msgr2=0x55a5fa07e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:03.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.904+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/3837794126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa07e4a0 0x55a5fa07e880 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6a380058a0 tx=0x7f6a3800b780 comp rx=0 tx=0).stop 2024-09-19T06:53:03.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.904+0000 7f6a4ad09740 1 -- 172.21.15.103:0/3837794126 shutdown_connections 2024-09-19T06:53:03.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.904+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/3837794126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa089680 0x55a5fa08ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.904+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/3837794126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 0x55a5fa089140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.904+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/3837794126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa07e4a0 0x55a5fa07e880 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.904+0000 7f6a4ad09740 1 -- 172.21.15.103:0/3837794126 >> 172.21.15.103:0/3837794126 conn(0x55a5fa07aa30 msgr2=0x55a5fa07ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:03.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.904+0000 7f6a4ad09740 1 -- 172.21.15.103:0/3837794126 shutdown_connections 2024-09-19T06:53:03.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.904+0000 7f6a4ad09740 1 -- 172.21.15.103:0/3837794126 wait complete. 2024-09-19T06:53:03.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.905+0000 7f6a4ad09740 1 -- start start 2024-09-19T06:53:03.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.905+0000 7f6a4ad09740 1 Processor -- start 2024-09-19T06:53:03.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.905+0000 7f6a4ad09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa07e4a0 0x55a5fa089020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.905+0000 7f6a4ad09740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 0x55a5fa0862d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.905+0000 7f6a4ad09740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa089680 0x55a5fa086870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.905+0000 7f6a4ad09740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a5fa08c250 con 0x55a5fa089680 2024-09-19T06:53:03.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.905+0000 7f6a4ad09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a5f9f9f050 con 0x55a5fa07e4a0 2024-09-19T06:53:03.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.905+0000 7f6a4ad09740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a5fa08c3d0 con 0x55a5fa07edc0 2024-09-19T06:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.905+0000 7f6a4ad09740 10 client.0 _ll_register_callbacks cb 0x55a5f9fdf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.905+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa089680 0x55a5fa086870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.905+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa089680 0x55a5fa086870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33392/0 (socket says 172.21.15.103:33392) 2024-09-19T06:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.905+0000 7f6a41d9b640 1 -- 172.21.15.103:0/1681001671 learned_addr learned my addr 172.21.15.103:0/1681001671 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.905+0000 7f6a4259c640 1 --2- 172.21.15.103:0/1681001671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 0x55a5fa0862d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.905+0000 7f6a41d9b640 1 --2- 172.21.15.103:0/1681001671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa07e4a0 0x55a5fa089020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.906+0000 7f6a41d9b640 1 -- 172.21.15.103:0/1681001671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 msgr2=0x55a5fa0862d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.906+0000 7f6a41d9b640 1 --2- 172.21.15.103:0/1681001671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 0x55a5fa0862d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.906+0000 7f6a41d9b640 1 -- 172.21.15.103:0/1681001671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa07e4a0 msgr2=0x55a5fa089020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.906+0000 7f6a41d9b640 1 --2- 172.21.15.103:0/1681001671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa07e4a0 0x55a5fa089020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.906+0000 7f6a41d9b640 1 -- 172.21.15.103:0/1681001671 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5f9d553c0 con 0x55a5fa089680 2024-09-19T06:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.906+0000 7f6a4259c640 1 --2- 172.21.15.103:0/1681001671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 0x55a5fa0862d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.906+0000 7f6a41d9b640 1 --2- 172.21.15.103:0/1681001671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa089680 0x55a5fa086870 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f6a38005940 tx=0x7f6a380058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f10d27916916524 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.906+0000 7f6a29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.906+0000 7f6a29ffb640 1 -- 172.21.15.103:0/1681001671 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38004a10 con 0x55a5fa089680 2024-09-19T06:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.906+0000 7f6a29ffb640 1 -- 172.21.15.103:0/1681001671 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6a38004bb0 con 0x55a5fa089680 2024-09-19T06:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.906+0000 7f6a4ad09740 1 -- 172.21.15.103:0/1681001671 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a38005020 con 0x55a5fa089680 2024-09-19T06:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.906+0000 7f6a29ffb640 1 -- 172.21.15.103:0/1681001671 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a38032050 con 0x55a5fa089680 2024-09-19T06:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.906+0000 7f6a4ad09740 10 client.16866 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.906+0000 7f6a4ad09740 1 -- 172.21.15.103:0/1681001671 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a5fa097c60 con 0x55a5fa089680 2024-09-19T06:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.906+0000 7f6a29ffb640 1 -- 172.21.15.103:0/1681001671 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a38005020 con 0x55a5fa089680 2024-09-19T06:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.907+0000 7f6a29ffb640 1 -- 172.21.15.103:0/1681001671 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6a3801b6f0 con 0x55a5fa089680 2024-09-19T06:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.907+0000 7f6a4ad09740 10 client.16866 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.907+0000 7f6a4ad09740 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.907+0000 7f6a4ad09740 20 client.16866 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.907+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/1681001671 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a5fa099010 0x55a5fa09b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.907+0000 7f6a4ad09740 4 client.16866 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.907+0000 7f6a4ad09740 1 -- 172.21.15.103:0/1681001671 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5f9ff1850 con 0x55a5fa099010 2024-09-19T06:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.907+0000 7f6a4259c640 1 --2- 172.21.15.103:0/1681001671 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a5fa099010 0x55a5fa09b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.907+0000 7f6a29ffb640 10 client.16866 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.907+0000 7f6a4259c640 1 --2- 172.21.15.103:0/1681001671 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a5fa099010 0x55a5fa09b400 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e16df016eba6a35f server_cookie=11b176baed10a5c in_seq=0 out_seq=0 2024-09-19T06:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.908+0000 7f6a29ffb640 1 -- 172.21.15.103:0/1681001671 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5f9ff1850 con 0x55a5fa099010 2024-09-19T06:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.908+0000 7f6a29ffb640 10 client.16866 handle_command_reply: tid=0 2024-09-19T06:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.908+0000 7f6a4ad09740 1 client.16866 shutdown 2024-09-19T06:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.908+0000 7f6a4ad09740 1 -- 172.21.15.103:0/1681001671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa089680 msgr2=0x55a5fa086870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.908+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/1681001671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa089680 0x55a5fa086870 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f6a38005940 tx=0x7f6a380058d0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.908+0000 7f6a4ad09740 1 -- 172.21.15.103:0/1681001671 shutdown_connections 2024-09-19T06:53:03.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.908+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/1681001671 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a5fa099010 0x55a5fa09b400 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.908+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/1681001671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa089680 0x55a5fa086870 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.908+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/1681001671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 0x55a5fa0862d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.908+0000 7f6a29ffb640 0 client.16866 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:03.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.908+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/1681001671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa07e4a0 0x55a5fa089020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.908+0000 7f6a4ad09740 1 -- 172.21.15.103:0/1681001671 >> 172.21.15.103:0/1681001671 conn(0x55a5fa07aa30 msgr2=0x55a5fa080760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:03.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.908+0000 7f6a4ad09740 1 -- 172.21.15.103:0/1681001671 shutdown_connections 2024-09-19T06:53:03.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.908+0000 7f6a4ad09740 1 -- 172.21.15.103:0/1681001671 wait complete. 2024-09-19T06:53:03.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.908+0000 7f6a4ad09740 20 client.16866 trim_cache size 0 max 16384 2024-09-19T06:53:03.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.925+0000 7f6a4ad09740 1 Processor -- start 2024-09-19T06:53:03.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.925+0000 7f6a4ad09740 1 -- start start 2024-09-19T06:53:03.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.925+0000 7f6a4ad09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa07e4a0 0x55a5fa0954b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.925+0000 7f6a4ad09740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa07edc0 0x55a5fa079a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.925+0000 7f6a4ad09740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa089680 0x55a5fa0a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.925+0000 7f6a4ad09740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a441ad7b0 con 0x55a5fa07edc0 2024-09-19T06:53:03.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.925+0000 7f6a4ad09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a441af2f0 con 0x55a5fa07e4a0 2024-09-19T06:53:03.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.925+0000 7f6a4ad09740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6a441ade80 con 0x55a5fa089680 2024-09-19T06:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.925+0000 7f6a4259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa07edc0 0x55a5fa079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.925+0000 7f6a4259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa07edc0 0x55a5fa079a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33396/0 (socket says 172.21.15.103:33396) 2024-09-19T06:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.925+0000 7f6a4259c640 1 -- 172.21.15.103:0/1535737723 learned_addr learned my addr 172.21.15.103:0/1535737723 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.925+0000 7f6a41d9b640 1 --2- 172.21.15.103:0/1535737723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa089680 0x55a5fa0a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.926+0000 7f6a41d9b640 1 --2- 172.21.15.103:0/1535737723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa07e4a0 0x55a5fa0954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.926+0000 7f6a4259c640 1 -- 172.21.15.103:0/1535737723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa089680 msgr2=0x55a5fa0a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.926+0000 7f6a4259c640 1 --2- 172.21.15.103:0/1535737723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa089680 0x55a5fa0a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.926+0000 7f6a4259c640 1 -- 172.21.15.103:0/1535737723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa07e4a0 msgr2=0x55a5fa0954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.926+0000 7f6a4259c640 1 --2- 172.21.15.103:0/1535737723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa07e4a0 0x55a5fa0954b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.926+0000 7f6a4259c640 1 -- 172.21.15.103:0/1535737723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5fa089390 con 0x55a5fa07edc0 2024-09-19T06:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.926+0000 7f6a4259c640 1 --2- 172.21.15.103:0/1535737723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa07edc0 0x55a5fa079a80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x55a5fa085b60 tx=0x7f6a3c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529a7561eab86fa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:03.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.926+0000 7f6a2bfff640 1 -- 172.21.15.103:0/1535737723 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c022030 con 0x55a5fa07edc0 2024-09-19T06:53:03.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.926+0000 7f6a2bfff640 1 -- 172.21.15.103:0/1535737723 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a3c005500 con 0x55a5fa07edc0 2024-09-19T06:53:03.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.926+0000 7f6a2bfff640 1 -- 172.21.15.103:0/1535737723 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c01d680 con 0x55a5fa07edc0 2024-09-19T06:53:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 -- 172.21.15.103:0/1535737723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa07edc0 msgr2=0x55a5fa079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/1535737723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa07edc0 0x55a5fa079a80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x55a5fa085b60 tx=0x7f6a3c011810 comp rx=0 tx=0).stop 2024-09-19T06:53:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 -- 172.21.15.103:0/1535737723 shutdown_connections 2024-09-19T06:53:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/1535737723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa089680 0x55a5fa0a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/1535737723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa07edc0 0x55a5fa079a80 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/1535737723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa07e4a0 0x55a5fa0954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 -- 172.21.15.103:0/1535737723 >> 172.21.15.103:0/1535737723 conn(0x55a5fa07aa30 msgr2=0x55a5fa079370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 -- 172.21.15.103:0/1535737723 shutdown_connections 2024-09-19T06:53:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 -- 172.21.15.103:0/1535737723 wait complete. 2024-09-19T06:53:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 -- start start 2024-09-19T06:53:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 Processor -- start 2024-09-19T06:53:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa07e4a0 0x55a5fa091620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 0x55a5fa091b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa089680 0x55a5fa08e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a441aec20 con 0x55a5fa089680 2024-09-19T06:53:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a441ae550 con 0x55a5fa07e4a0 2024-09-19T06:53:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6a441ab770 con 0x55a5fa07edc0 2024-09-19T06:53:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.927+0000 7f6a4ad09740 10 client.0 _ll_register_callbacks cb 0x7f6a3800bcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa089680 0x55a5fa08e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa089680 0x55a5fa08e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33400/0 (socket says 172.21.15.103:33400) 2024-09-19T06:53:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a41d9b640 1 -- 172.21.15.103:0/2321121910 learned_addr learned my addr 172.21.15.103:0/2321121910 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a41d9b640 1 --2- 172.21.15.103:0/2321121910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa07e4a0 0x55a5fa091620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a4259c640 1 --2- 172.21.15.103:0/2321121910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 0x55a5fa091b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a41d9b640 1 -- 172.21.15.103:0/2321121910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 msgr2=0x55a5fa091b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a41d9b640 1 --2- 172.21.15.103:0/2321121910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 0x55a5fa091b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a41d9b640 1 -- 172.21.15.103:0/2321121910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa07e4a0 msgr2=0x55a5fa091620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a41d9b640 1 --2- 172.21.15.103:0/2321121910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa07e4a0 0x55a5fa091620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a41d9b640 1 -- 172.21.15.103:0/2321121910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5fa08eb30 con 0x55a5fa089680 2024-09-19T06:53:03.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a41d9b640 1 --2- 172.21.15.103:0/2321121910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa089680 0x55a5fa08e5f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f6a38012300 tx=0x7f6a380058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9254df19baf2fa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:03.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:03.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a29ffb640 1 -- 172.21.15.103:0/2321121910 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38012070 con 0x55a5fa089680 2024-09-19T06:53:03.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a29ffb640 1 -- 172.21.15.103:0/2321121910 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6a3801b3f0 con 0x55a5fa089680 2024-09-19T06:53:03.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a4ad09740 1 -- 172.21.15.103:0/2321121910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a3c005500 con 0x55a5fa089680 2024-09-19T06:53:03.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a29ffb640 1 -- 172.21.15.103:0/2321121910 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a3801b5f0 con 0x55a5fa089680 2024-09-19T06:53:03.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a4ad09740 10 client.16872 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:03.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.928+0000 7f6a4ad09740 1 -- 172.21.15.103:0/2321121910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a5fa104200 con 0x55a5fa089680 2024-09-19T06:53:03.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.929+0000 7f6a29ffb640 1 -- 172.21.15.103:0/2321121910 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a3c005500 con 0x55a5fa089680 2024-09-19T06:53:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.929+0000 7f6a29ffb640 1 -- 172.21.15.103:0/2321121910 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6a38043bf0 con 0x55a5fa089680 2024-09-19T06:53:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.929+0000 7f6a4ad09740 10 client.16872 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.929+0000 7f6a4ad09740 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.929+0000 7f6a4ad09740 20 client.16872 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.929+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/2321121910 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a5fa1048f0 0x55a5fa106d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.929+0000 7f6a4ad09740 4 client.16872 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:53:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.929+0000 7f6a4ad09740 1 -- 172.21.15.103:0/2321121910 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a5f9febd60 con 0x55a5fa1048f0 2024-09-19T06:53:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.929+0000 7f6a4259c640 1 --2- 172.21.15.103:0/2321121910 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a5fa1048f0 0x55a5fa106d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.929+0000 7f6a4259c640 1 --2- 172.21.15.103:0/2321121910 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a5fa1048f0 0x55a5fa106d10 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4b312275e11430e server_cookie=22bd5255ebdc3425 in_seq=0 out_seq=0 2024-09-19T06:53:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.929+0000 7f6a29ffb640 10 client.16872 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a29ffb640 1 -- 172.21.15.103:0/2321121910 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a5f9febd60 con 0x55a5fa1048f0 2024-09-19T06:53:03.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a29ffb640 10 client.16872 handle_command_reply: tid=0 2024-09-19T06:53:03.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a4ad09740 1 client.16872 shutdown 2024-09-19T06:53:03.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a4ad09740 1 -- 172.21.15.103:0/2321121910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa089680 msgr2=0x55a5fa08e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:03.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/2321121910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa089680 0x55a5fa08e5f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f6a38012300 tx=0x7f6a380058d0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a4ad09740 1 -- 172.21.15.103:0/2321121910 shutdown_connections 2024-09-19T06:53:03.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/2321121910 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a5fa1048f0 0x55a5fa106d10 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/2321121910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fa089680 0x55a5fa08e5f0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/2321121910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fa07edc0 0x55a5fa091b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a4ad09740 1 --2- 172.21.15.103:0/2321121910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fa07e4a0 0x55a5fa091620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a4ad09740 1 -- 172.21.15.103:0/2321121910 >> 172.21.15.103:0/2321121910 conn(0x55a5fa07aa30 msgr2=0x55a5fa079350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a29ffb640 0 client.16872 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a4ad09740 1 -- 172.21.15.103:0/2321121910 shutdown_connections 2024-09-19T06:53:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a4ad09740 1 -- 172.21.15.103:0/2321121910 wait complete. 2024-09-19T06:53:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a4ad09740 20 client.16872 trim_cache size 0 max 16384 2024-09-19T06:53:03.941 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:53:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:53:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T06:53:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "60b41e7b-0a92-45e0-bef7-1c665dd58b29": { 2024-09-19T06:53:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T06:53:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "60b41e7b-0a92-45e0-bef7-1c665dd58b29", 2024-09-19T06:53:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T06:53:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:53:03.942 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:53:03.942 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:53:03.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a48930640 1 -- 172.21.15.103:0/3247155963 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6a2c0a3430 msgr2=0x7f6a2c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:03.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a48930640 1 --2- 172.21.15.103:0/3247155963 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6a2c0a3430 0x7f6a2c0a58f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6a340bd6c0 tx=0x7f6a3408f4b0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a48930640 1 -- 172.21.15.103:0/3247155963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a44108290 msgr2=0x7f6a4419ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:03.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.930+0000 7f6a48930640 1 --2- 172.21.15.103:0/3247155963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a44108290 0x7f6a4419ce10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6a34005cc0 tx=0x7f6a34003970 comp rx=0 tx=0).stop 2024-09-19T06:53:03.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.931+0000 7f6a48930640 1 -- 172.21.15.103:0/3247155963 shutdown_connections 2024-09-19T06:53:03.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.931+0000 7f6a48930640 1 --2- 172.21.15.103:0/3247155963 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6a2c0a3430 0x7f6a2c0a58f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.931+0000 7f6a48930640 1 --2- 172.21.15.103:0/3247155963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a44108290 0x7f6a4419ce10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.931+0000 7f6a48930640 1 --2- 172.21.15.103:0/3247155963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a44103740 0x7f6a44198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.931+0000 7f6a48930640 1 --2- 172.21.15.103:0/3247155963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a44102d90 0x7f6a44198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:03.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.931+0000 7f6a48930640 1 -- 172.21.15.103:0/3247155963 >> 172.21.15.103:0/3247155963 conn(0x7f6a440fc7d0 msgr2=0x7f6a44106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:03.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.931+0000 7f6a48930640 1 -- 172.21.15.103:0/3247155963 shutdown_connections 2024-09-19T06:53:03.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:03.931+0000 7f6a48930640 1 -- 172.21.15.103:0/3247155963 wait complete. 2024-09-19T06:53:04.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:04 smithi174 ceph-mon[35637]: pgmap v118: 97 pgs: 97 active+clean; 3.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.1 KiB/s wr, 11 op/s 2024-09-19T06:53:04.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:04 smithi103 ceph-mon[31158]: pgmap v118: 97 pgs: 97 active+clean; 3.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.1 KiB/s wr, 11 op/s 2024-09-19T06:53:04.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:04 smithi134 ceph-mon[35356]: pgmap v118: 97 pgs: 97 active+clean; 3.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.1 KiB/s wr, 11 op/s 2024-09-19T06:53:04.770 INFO:tasks.cephfs.filesystem:scrub status for tag:60b41e7b-0a92-45e0-bef7-1c665dd58b29 - {'path': '/', 'tag': '60b41e7b-0a92-45e0-bef7-1c665dd58b29', 'options': 'recursive,force'} 2024-09-19T06:53:05.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:05 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T06:53:05.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:05 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T06:53:05.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:05 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T06:53:06.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:06 smithi174 ceph-mon[35637]: pgmap v119: 97 pgs: 97 active+clean; 3.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 KiB/s wr, 32 op/s 2024-09-19T06:53:06.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:06 smithi103 ceph-mon[31158]: pgmap v119: 97 pgs: 97 active+clean; 3.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 KiB/s wr, 32 op/s 2024-09-19T06:53:06.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:06 smithi134 ceph-mon[35356]: pgmap v119: 97 pgs: 97 active+clean; 3.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 KiB/s wr, 32 op/s 2024-09-19T06:53:08.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:08 smithi174 ceph-mon[35637]: pgmap v120: 97 pgs: 97 active+clean; 3.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 28 op/s 2024-09-19T06:53:08.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:08 smithi103 ceph-mon[31158]: pgmap v120: 97 pgs: 97 active+clean; 3.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 28 op/s 2024-09-19T06:53:08.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:08 smithi134 ceph-mon[35356]: pgmap v120: 97 pgs: 97 active+clean; 3.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 28 op/s 2024-09-19T06:53:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.670+0000 7f776a606640 1 -- 172.21.15.103:0/3229381827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77641009b0 msgr2=0x7f7764100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.670+0000 7f776a606640 1 --2- 172.21.15.103:0/3229381827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77641009b0 0x7f7764100e10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7758009930 tx=0x7f775802a420 comp rx=0 tx=0).stop 2024-09-19T06:53:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.670+0000 7f776a606640 1 -- 172.21.15.103:0/3229381827 shutdown_connections 2024-09-19T06:53:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.670+0000 7f776a606640 1 --2- 172.21.15.103:0/3229381827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7764101350 0x7f776410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.670+0000 7f776a606640 1 --2- 172.21.15.103:0/3229381827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77641009b0 0x7f7764100e10 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.670+0000 7f776a606640 1 --2- 172.21.15.103:0/3229381827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77641069c0 0x7f7764106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.670+0000 7f776a606640 1 -- 172.21.15.103:0/3229381827 >> 172.21.15.103:0/3229381827 conn(0x7f7764077990 msgr2=0x7f77640fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.670+0000 7f776a606640 1 -- 172.21.15.103:0/3229381827 shutdown_connections 2024-09-19T06:53:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.670+0000 7f776a606640 1 -- 172.21.15.103:0/3229381827 wait complete. 2024-09-19T06:53:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.671+0000 7f776a606640 1 Processor -- start 2024-09-19T06:53:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.671+0000 7f776a606640 1 -- start start 2024-09-19T06:53:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.671+0000 7f776a606640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77641009b0 0x7f77641991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.671+0000 7f776a606640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7764101350 0x7f77641996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.671+0000 7f776a606640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77641069c0 0x7f77641a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.671+0000 7f776a606640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7764110e00 con 0x7f77641069c0 2024-09-19T06:53:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.671+0000 7f776a606640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7764110c80 con 0x7f7764101350 2024-09-19T06:53:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.671+0000 7f776a606640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7764110f80 con 0x7f77641009b0 2024-09-19T06:53:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.671+0000 7f7768ba3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77641069c0 0x7f77641a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.671+0000 7f7768ba3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77641069c0 0x7f77641a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37622/0 (socket says 172.21.15.103:37622) 2024-09-19T06:53:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.671+0000 7f7768ba3640 1 -- 172.21.15.103:0/1319207983 learned_addr learned my addr 172.21.15.103:0/1319207983 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.671+0000 7f7763fff640 1 --2- 172.21.15.103:0/1319207983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77641009b0 0x7f77641991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.672+0000 7f77637fe640 1 --2- 172.21.15.103:0/1319207983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7764101350 0x7f77641996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.672+0000 7f7768ba3640 1 -- 172.21.15.103:0/1319207983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77641009b0 msgr2=0x7f77641991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.672+0000 7f7768ba3640 1 --2- 172.21.15.103:0/1319207983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77641009b0 0x7f77641991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.672+0000 7f7768ba3640 1 -- 172.21.15.103:0/1319207983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7764101350 msgr2=0x7f77641996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.672+0000 7f7768ba3640 1 --2- 172.21.15.103:0/1319207983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7764101350 0x7f77641996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.672+0000 7f7768ba3640 1 -- 172.21.15.103:0/1319207983 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77641a4760 con 0x7f77641069c0 2024-09-19T06:53:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.672+0000 7f7763fff640 1 --2- 172.21.15.103:0/1319207983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77641009b0 0x7f77641991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.672+0000 7f7768ba3640 1 --2- 172.21.15.103:0/1319207983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77641069c0 0x7f77641a4220 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f775c00cad0 tx=0x7f775c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.672+0000 7f77617fa640 1 -- 172.21.15.103:0/1319207983 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c00ae20 con 0x7f77641069c0 2024-09-19T06:53:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.672+0000 7f776a606640 1 -- 172.21.15.103:0/1319207983 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77641a4930 con 0x7f77641069c0 2024-09-19T06:53:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.672+0000 7f77617fa640 1 -- 172.21.15.103:0/1319207983 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f775c005020 con 0x7f77641069c0 2024-09-19T06:53:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.672+0000 7f77617fa640 1 -- 172.21.15.103:0/1319207983 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c00f6b0 con 0x7f77641069c0 2024-09-19T06:53:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.672+0000 7f776a606640 1 -- 172.21.15.103:0/1319207983 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77641a4ea0 con 0x7f77641069c0 2024-09-19T06:53:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.674+0000 7f77617fa640 1 -- 172.21.15.103:0/1319207983 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f775c0051d0 con 0x7f77641069c0 2024-09-19T06:53:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.675+0000 7f776d482740 1 Processor -- start 2024-09-19T06:53:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.675+0000 7f776d482740 1 -- start start 2024-09-19T06:53:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.675+0000 7f77617fa640 1 --2- 172.21.15.103:0/1319207983 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f774c0a3460 0x7f774c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.675+0000 7f776d482740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590d9af0 0x55d1590d9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.675+0000 7f776d482740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590da410 0x55d1590e4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.675+0000 7f776d482740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590e4cb0 0x55d1590e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.675+0000 7f776d482740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d15903e2f0 con 0x55d1590e4cb0 2024-09-19T06:53:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.675+0000 7f776d482740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d158fe6270 con 0x55d1590da410 2024-09-19T06:53:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.675+0000 7f776d482740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d15903a2b0 con 0x55d1590d9af0 2024-09-19T06:53:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.675+0000 7f77617fa640 1 -- 172.21.15.103:0/1319207983 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f775c0d0150 con 0x7f77641069c0 2024-09-19T06:53:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.675+0000 7f77637fe640 1 --2- 172.21.15.103:0/1319207983 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f774c0a3460 0x7f774c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.675+0000 7f7768ba3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590e4cb0 0x55d1590e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.675+0000 7f7768ba3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590e4cb0 0x55d1590e70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37624/0 (socket says 172.21.15.103:37624) 2024-09-19T06:53:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.675+0000 7f7768ba3640 1 -- 172.21.15.103:0/2349499912 learned_addr learned my addr 172.21.15.103:0/2349499912 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.675+0000 7f77637fe640 1 --2- 172.21.15.103:0/2349499912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590da410 0x55d1590e4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.675+0000 7f7763fff640 1 --2- 172.21.15.103:0/2349499912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590d9af0 0x55d1590d9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.676+0000 7f7768ba3640 1 -- 172.21.15.103:0/2349499912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590d9af0 msgr2=0x55d1590d9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.676+0000 7f7768ba3640 1 --2- 172.21.15.103:0/2349499912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590d9af0 0x55d1590d9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.676+0000 7f7768ba3640 1 -- 172.21.15.103:0/2349499912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590da410 msgr2=0x55d1590e4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.676+0000 7f7768ba3640 1 --2- 172.21.15.103:0/2349499912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590da410 0x55d1590e4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.676+0000 7f7768ba3640 1 -- 172.21.15.103:0/2349499912 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d158fe5340 con 0x55d1590e4cb0 2024-09-19T06:53:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.676+0000 7f77637fe640 1 --2- 172.21.15.103:0/1319207983 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f774c0a3460 0x7f774c0a5920 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7758000c00 tx=0x7f7758031690 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.676+0000 7f7768ba3640 1 --2- 172.21.15.103:0/2349499912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590e4cb0 0x55d1590e70a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f775c098e10 tx=0x7f775c09cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcb731198b28df4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.676+0000 7f774a7fc640 1 -- 172.21.15.103:0/2349499912 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c01d040 con 0x55d1590e4cb0 2024-09-19T06:53:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.676+0000 7f774a7fc640 1 -- 172.21.15.103:0/2349499912 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f775c0a50b0 con 0x55d1590e4cb0 2024-09-19T06:53:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.676+0000 7f774a7fc640 1 -- 172.21.15.103:0/2349499912 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c0a4c20 con 0x55d1590e4cb0 2024-09-19T06:53:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.677+0000 7f776d482740 1 -- 172.21.15.103:0/2349499912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590e4cb0 msgr2=0x55d1590e70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.677+0000 7f776d482740 1 --2- 172.21.15.103:0/2349499912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590e4cb0 0x55d1590e70a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f775c098e10 tx=0x7f775c09cb40 comp rx=0 tx=0).stop 2024-09-19T06:53:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.677+0000 7f776d482740 1 -- 172.21.15.103:0/2349499912 shutdown_connections 2024-09-19T06:53:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.677+0000 7f776d482740 1 --2- 172.21.15.103:0/2349499912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590e4cb0 0x55d1590e70a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.677+0000 7f776d482740 1 --2- 172.21.15.103:0/2349499912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590da410 0x55d1590e4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.677+0000 7f776d482740 1 --2- 172.21.15.103:0/2349499912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590d9af0 0x55d1590d9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.677+0000 7f776d482740 1 -- 172.21.15.103:0/2349499912 >> 172.21.15.103:0/2349499912 conn(0x55d1590d6060 msgr2=0x55d1590d6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.677+0000 7f776d482740 1 -- 172.21.15.103:0/2349499912 shutdown_connections 2024-09-19T06:53:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.677+0000 7f776d482740 1 -- 172.21.15.103:0/2349499912 wait complete. 2024-09-19T06:53:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.677+0000 7f776d482740 1 -- start start 2024-09-19T06:53:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.677+0000 7f776d482740 1 Processor -- start 2024-09-19T06:53:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f776d482740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590d9af0 0x55d1590e1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f776d482740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590da410 0x55d1590e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f776d482740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590e4cb0 0x55d1590e25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f776d482740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d1590e7880 con 0x55d1590e4cb0 2024-09-19T06:53:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f776d482740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d158fc7290 con 0x55d1590d9af0 2024-09-19T06:53:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f776d482740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d1590e7a00 con 0x55d1590da410 2024-09-19T06:53:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f776d482740 10 client.0 _ll_register_callbacks cb 0x55d158ff8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f7768ba3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590e4cb0 0x55d1590e25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f7768ba3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590e4cb0 0x55d1590e25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37626/0 (socket says 172.21.15.103:37626) 2024-09-19T06:53:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f7768ba3640 1 -- 172.21.15.103:0/1193861506 learned_addr learned my addr 172.21.15.103:0/1193861506 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f7763fff640 1 --2- 172.21.15.103:0/1193861506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590d9af0 0x55d1590e1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f7768ba3640 1 -- 172.21.15.103:0/1193861506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590da410 msgr2=0x55d1590e2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f7768ba3640 1 --2- 172.21.15.103:0/1193861506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590da410 0x55d1590e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f7768ba3640 1 -- 172.21.15.103:0/1193861506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590d9af0 msgr2=0x55d1590e1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f7768ba3640 1 --2- 172.21.15.103:0/1193861506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590d9af0 0x55d1590e1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f7768ba3640 1 -- 172.21.15.103:0/1193861506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d158d51210 con 0x55d1590e4cb0 2024-09-19T06:53:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f7768ba3640 1 --2- 172.21.15.103:0/1193861506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590e4cb0 0x55d1590e25d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f775c09ccf0 tx=0x7f775c005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6e2de6b957ef98 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f77437fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f77437fe640 1 -- 172.21.15.103:0/1193861506 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c0b9070 con 0x55d1590e4cb0 2024-09-19T06:53:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f77437fe640 1 -- 172.21.15.103:0/1193861506 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f775c0b5260 con 0x55d1590e4cb0 2024-09-19T06:53:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.678+0000 7f776d482740 1 -- 172.21.15.103:0/1193861506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f775c0a50b0 con 0x55d1590e4cb0 2024-09-19T06:53:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.679+0000 7f77437fe640 1 -- 172.21.15.103:0/1193861506 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f775c01d070 con 0x55d1590e4cb0 2024-09-19T06:53:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.679+0000 7f776d482740 10 client.16884 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.679+0000 7f776d482740 1 -- 172.21.15.103:0/1193861506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d158e1d620 con 0x55d1590e4cb0 2024-09-19T06:53:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.679+0000 7f77437fe640 1 -- 172.21.15.103:0/1193861506 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f775c0a50b0 con 0x55d1590e4cb0 2024-09-19T06:53:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.679+0000 7f77437fe640 1 -- 172.21.15.103:0/1193861506 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f775c09b920 con 0x55d1590e4cb0 2024-09-19T06:53:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.679+0000 7f776d482740 10 client.16884 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.679+0000 7f776d482740 10 client.16884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.679+0000 7f776d482740 20 client.16884 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.679+0000 7f776d482740 1 --2- 172.21.15.103:0/1193861506 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d1590f4a50 0x55d1590f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.679+0000 7f776d482740 4 client.16884 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.679+0000 7f776d482740 1 -- 172.21.15.103:0/1193861506 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d15904ba20 con 0x55d1590f4a50 2024-09-19T06:53:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.679+0000 7f7763fff640 1 --2- 172.21.15.103:0/1193861506 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d1590f4a50 0x55d1590f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.680+0000 7f7763fff640 1 --2- 172.21.15.103:0/1193861506 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d1590f4a50 0x55d1590f6e40 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23afb588baaffadc server_cookie=8be240a4367cf66f in_seq=0 out_seq=0 2024-09-19T06:53:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.680+0000 7f77437fe640 10 client.16884 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.680+0000 7f77437fe640 1 -- 172.21.15.103:0/1193861506 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d15904ba20 con 0x55d1590f4a50 2024-09-19T06:53:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.680+0000 7f77437fe640 10 client.16884 handle_command_reply: tid=0 2024-09-19T06:53:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.680+0000 7f776d482740 1 client.16884 shutdown 2024-09-19T06:53:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.681+0000 7f776d482740 1 -- 172.21.15.103:0/1193861506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590e4cb0 msgr2=0x55d1590e25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.681+0000 7f776d482740 1 --2- 172.21.15.103:0/1193861506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590e4cb0 0x55d1590e25d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f775c09ccf0 tx=0x7f775c005790 comp rx=0 tx=0).stop 2024-09-19T06:53:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.681+0000 7f776d482740 1 -- 172.21.15.103:0/1193861506 shutdown_connections 2024-09-19T06:53:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.681+0000 7f776d482740 1 --2- 172.21.15.103:0/1193861506 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d1590f4a50 0x55d1590f6e40 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.681+0000 7f776d482740 1 --2- 172.21.15.103:0/1193861506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590e4cb0 0x55d1590e25d0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.681+0000 7f77437fe640 0 client.16884 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.681+0000 7f776d482740 1 --2- 172.21.15.103:0/1193861506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590da410 0x55d1590e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.681+0000 7f776d482740 1 --2- 172.21.15.103:0/1193861506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590d9af0 0x55d1590e1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.681+0000 7f776d482740 1 -- 172.21.15.103:0/1193861506 >> 172.21.15.103:0/1193861506 conn(0x55d1590d6060 msgr2=0x55d1590dbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.681+0000 7f776d482740 1 -- 172.21.15.103:0/1193861506 shutdown_connections 2024-09-19T06:53:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.681+0000 7f776d482740 1 -- 172.21.15.103:0/1193861506 wait complete. 2024-09-19T06:53:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.681+0000 7f776d482740 20 client.16884 trim_cache size 0 max 16384 2024-09-19T06:53:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.695+0000 7f776d482740 1 Processor -- start 2024-09-19T06:53:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.695+0000 7f776d482740 1 -- start start 2024-09-19T06:53:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.695+0000 7f776d482740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590d9af0 0x55d1590e2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.695+0000 7f776d482740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590da410 0x55d1590e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.695+0000 7f776d482740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590e4cb0 0x55d1590ed070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.695+0000 7f776d482740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77641ad810 con 0x55d1590da410 2024-09-19T06:53:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.695+0000 7f776d482740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f77641af350 con 0x55d1590e4cb0 2024-09-19T06:53:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.695+0000 7f776d482740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f77641adee0 con 0x55d1590d9af0 2024-09-19T06:53:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.695+0000 7f77637fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590da410 0x55d1590e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.695+0000 7f77637fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590da410 0x55d1590e3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37634/0 (socket says 172.21.15.103:37634) 2024-09-19T06:53:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.695+0000 7f77637fe640 1 -- 172.21.15.103:0/2307489237 learned_addr learned my addr 172.21.15.103:0/2307489237 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.696+0000 7f7763fff640 1 --2- 172.21.15.103:0/2307489237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590d9af0 0x55d1590e2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.696+0000 7f77637fe640 1 -- 172.21.15.103:0/2307489237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590d9af0 msgr2=0x55d1590e2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.696+0000 7f7768ba3640 1 --2- 172.21.15.103:0/2307489237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590e4cb0 0x55d1590ed070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.696+0000 7f77637fe640 1 --2- 172.21.15.103:0/2307489237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590d9af0 0x55d1590e2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.696+0000 7f77637fe640 1 -- 172.21.15.103:0/2307489237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590e4cb0 msgr2=0x55d1590ed070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.696+0000 7f77637fe640 1 --2- 172.21.15.103:0/2307489237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590e4cb0 0x55d1590ed070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.696+0000 7f77637fe640 1 -- 172.21.15.103:0/2307489237 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1590d5e90 con 0x55d1590da410 2024-09-19T06:53:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.696+0000 7f7763fff640 1 --2- 172.21.15.103:0/2307489237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590d9af0 0x55d1590e2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.696+0000 7f7768ba3640 1 --2- 172.21.15.103:0/2307489237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590e4cb0 0x55d1590ed070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.696+0000 7f77637fe640 1 --2- 172.21.15.103:0/2307489237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590da410 0x55d1590e3ac0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7758009930 tx=0x7f7758043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613f0674dc87272b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.696+0000 7f7749ffb640 1 -- 172.21.15.103:0/2307489237 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775804d070 con 0x55d1590da410 2024-09-19T06:53:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.696+0000 7f7749ffb640 1 -- 172.21.15.103:0/2307489237 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77580046c0 con 0x55d1590da410 2024-09-19T06:53:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.696+0000 7f7749ffb640 1 -- 172.21.15.103:0/2307489237 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758035050 con 0x55d1590da410 2024-09-19T06:53:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 -- 172.21.15.103:0/2307489237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590da410 msgr2=0x55d1590e3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 --2- 172.21.15.103:0/2307489237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590da410 0x55d1590e3ac0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7758009930 tx=0x7f7758043000 comp rx=0 tx=0).stop 2024-09-19T06:53:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 -- 172.21.15.103:0/2307489237 shutdown_connections 2024-09-19T06:53:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 --2- 172.21.15.103:0/2307489237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590e4cb0 0x55d1590ed070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 --2- 172.21.15.103:0/2307489237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590da410 0x55d1590e3ac0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 --2- 172.21.15.103:0/2307489237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590d9af0 0x55d1590e2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 -- 172.21.15.103:0/2307489237 >> 172.21.15.103:0/2307489237 conn(0x55d1590d6060 msgr2=0x55d1590e33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 -- 172.21.15.103:0/2307489237 shutdown_connections 2024-09-19T06:53:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 -- 172.21.15.103:0/2307489237 wait complete. 2024-09-19T06:53:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 -- start start 2024-09-19T06:53:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 Processor -- start 2024-09-19T06:53:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590d9af0 0x55d1590df6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590da410 0x55d1590fbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590e4cb0 0x55d1590fc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77641aec80 con 0x55d1590da410 2024-09-19T06:53:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f77641ae5b0 con 0x55d1590e4cb0 2024-09-19T06:53:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.697+0000 7f776d482740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f77641ab7d0 con 0x55d1590d9af0 2024-09-19T06:53:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f776d482740 10 client.0 _ll_register_callbacks cb 0x7f7758009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f77637fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590da410 0x55d1590fbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f77637fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590da410 0x55d1590fbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37638/0 (socket says 172.21.15.103:37638) 2024-09-19T06:53:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f77637fe640 1 -- 172.21.15.103:0/2353038919 learned_addr learned my addr 172.21.15.103:0/2353038919 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f7768ba3640 1 --2- 172.21.15.103:0/2353038919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590e4cb0 0x55d1590fc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f7763fff640 1 --2- 172.21.15.103:0/2353038919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590d9af0 0x55d1590df6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f77637fe640 1 -- 172.21.15.103:0/2353038919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590d9af0 msgr2=0x55d1590df6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f77637fe640 1 --2- 172.21.15.103:0/2353038919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590d9af0 0x55d1590df6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f77637fe640 1 -- 172.21.15.103:0/2353038919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590e4cb0 msgr2=0x55d1590fc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f77637fe640 1 --2- 172.21.15.103:0/2353038919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590e4cb0 0x55d1590fc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f77637fe640 1 -- 172.21.15.103:0/2353038919 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1590fcc80 con 0x55d1590da410 2024-09-19T06:53:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f77637fe640 1 --2- 172.21.15.103:0/2353038919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590da410 0x55d1590fbf10 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f7758031770 tx=0x7f7758005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f839f3bbe87d7919 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f77437fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f77437fe640 1 -- 172.21.15.103:0/2353038919 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758055030 con 0x55d1590da410 2024-09-19T06:53:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f77437fe640 1 -- 172.21.15.103:0/2353038919 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f7758004fe0 con 0x55d1590da410 2024-09-19T06:53:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f776d482740 1 -- 172.21.15.103:0/2353038919 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77580046c0 con 0x55d1590da410 2024-09-19T06:53:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f77437fe640 1 -- 172.21.15.103:0/2353038919 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f775804d050 con 0x55d1590da410 2024-09-19T06:53:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f776d482740 10 client.16890 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.698+0000 7f776d482740 1 -- 172.21.15.103:0/2353038919 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d159163410 con 0x55d1590da410 2024-09-19T06:53:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.699+0000 7f77437fe640 1 -- 172.21.15.103:0/2353038919 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77580046c0 con 0x55d1590da410 2024-09-19T06:53:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.699+0000 7f77437fe640 1 -- 172.21.15.103:0/2353038919 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7758003b80 con 0x55d1590da410 2024-09-19T06:53:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.699+0000 7f776d482740 10 client.16890 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.699+0000 7f776d482740 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.699+0000 7f776d482740 20 client.16890 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.699+0000 7f776d482740 1 --2- 172.21.15.103:0/2353038919 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d159163b00 0x55d159165f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.699+0000 7f776d482740 4 client.16890 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:53:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.699+0000 7f776d482740 1 -- 172.21.15.103:0/2353038919 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d159045d70 con 0x55d159163b00 2024-09-19T06:53:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.699+0000 7f7763fff640 1 --2- 172.21.15.103:0/2353038919 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d159163b00 0x55d159165f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.699+0000 7f7763fff640 1 --2- 172.21.15.103:0/2353038919 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d159163b00 0x55d159165f20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e2ac119592ca0f0 server_cookie=7a842d750d7a0a7 in_seq=0 out_seq=0 2024-09-19T06:53:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.699+0000 7f77437fe640 10 client.16890 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f77437fe640 1 -- 172.21.15.103:0/2353038919 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d159045d70 con 0x55d159163b00 2024-09-19T06:53:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f77437fe640 10 client.16890 handle_command_reply: tid=0 2024-09-19T06:53:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776d482740 1 client.16890 shutdown 2024-09-19T06:53:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776d482740 1 -- 172.21.15.103:0/2353038919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590da410 msgr2=0x55d1590fbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776d482740 1 --2- 172.21.15.103:0/2353038919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590da410 0x55d1590fbf10 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f7758031770 tx=0x7f7758005fd0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776d482740 1 -- 172.21.15.103:0/2353038919 shutdown_connections 2024-09-19T06:53:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776d482740 1 --2- 172.21.15.103:0/2353038919 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d159163b00 0x55d159165f20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776d482740 1 --2- 172.21.15.103:0/2353038919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1590e4cb0 0x55d1590fc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776d482740 1 --2- 172.21.15.103:0/2353038919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1590da410 0x55d1590fbf10 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f77437fe640 0 client.16890 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776d482740 1 --2- 172.21.15.103:0/2353038919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1590d9af0 0x55d1590df6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776d482740 1 -- 172.21.15.103:0/2353038919 >> 172.21.15.103:0/2353038919 conn(0x55d1590d6060 msgr2=0x55d1590d51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776d482740 1 -- 172.21.15.103:0/2353038919 shutdown_connections 2024-09-19T06:53:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776d482740 1 -- 172.21.15.103:0/2353038919 wait complete. 2024-09-19T06:53:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776d482740 20 client.16890 trim_cache size 0 max 16384 2024-09-19T06:53:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:53:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:53:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:53:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:53:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776a606640 1 -- 172.21.15.103:0/1319207983 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f774c0a3460 msgr2=0x7f774c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776a606640 1 --2- 172.21.15.103:0/1319207983 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f774c0a3460 0x7f774c0a5920 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7758000c00 tx=0x7f7758031690 comp rx=0 tx=0).stop 2024-09-19T06:53:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776a606640 1 -- 172.21.15.103:0/1319207983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77641069c0 msgr2=0x7f77641a4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776a606640 1 --2- 172.21.15.103:0/1319207983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77641069c0 0x7f77641a4220 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f775c00cad0 tx=0x7f775c00cf90 comp rx=0 tx=0).stop 2024-09-19T06:53:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776a606640 1 -- 172.21.15.103:0/1319207983 shutdown_connections 2024-09-19T06:53:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776a606640 1 --2- 172.21.15.103:0/1319207983 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f774c0a3460 0x7f774c0a5920 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776a606640 1 --2- 172.21.15.103:0/1319207983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77641069c0 0x7f77641a4220 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776a606640 1 --2- 172.21.15.103:0/1319207983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7764101350 0x7f77641996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776a606640 1 --2- 172.21.15.103:0/1319207983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77641009b0 0x7f77641991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776a606640 1 -- 172.21.15.103:0/1319207983 >> 172.21.15.103:0/1319207983 conn(0x7f7764077990 msgr2=0x7f77641037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776a606640 1 -- 172.21.15.103:0/1319207983 shutdown_connections 2024-09-19T06:53:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:08.712+0000 7f776a606640 1 -- 172.21.15.103:0/1319207983 wait complete. 2024-09-19T06:53:10.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:10 smithi174 ceph-mon[35637]: pgmap v121: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-09-19T06:53:10.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:10 smithi103 ceph-mon[31158]: pgmap v121: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-09-19T06:53:10.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:10 smithi134 ceph-mon[35356]: pgmap v121: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-09-19T06:53:12.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:12 smithi174 ceph-mon[35637]: pgmap v122: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-09-19T06:53:12.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:12 smithi103 ceph-mon[31158]: pgmap v122: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-09-19T06:53:12.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:12 smithi134 ceph-mon[35356]: pgmap v122: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-09-19T06:53:14.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:14 smithi174 ceph-mon[35637]: pgmap v123: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-09-19T06:53:14.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:14 smithi103 ceph-mon[31158]: pgmap v123: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-09-19T06:53:14.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:14 smithi134 ceph-mon[35356]: pgmap v123: 97 pgs: 97 active+clean; 3.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-09-19T06:53:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:16 smithi174 ceph-mon[35637]: pgmap v124: 97 pgs: 97 active+clean; 3.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 63 KiB/s wr, 37 op/s 2024-09-19T06:53:16.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:16 smithi103 ceph-mon[31158]: pgmap v124: 97 pgs: 97 active+clean; 3.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 63 KiB/s wr, 37 op/s 2024-09-19T06:53:16.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:16 smithi134 ceph-mon[35356]: pgmap v124: 97 pgs: 97 active+clean; 3.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 63 KiB/s wr, 37 op/s 2024-09-19T06:53:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:18 smithi174 ceph-mon[35637]: pgmap v125: 97 pgs: 97 active+clean; 3.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-09-19T06:53:18.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:18 smithi103 ceph-mon[31158]: pgmap v125: 97 pgs: 97 active+clean; 3.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-09-19T06:53:18.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:18 smithi134 ceph-mon[35356]: pgmap v125: 97 pgs: 97 active+clean; 3.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-09-19T06:53:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:20 smithi174 ceph-mon[35637]: pgmap v126: 97 pgs: 97 active+clean; 3.9 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 21 op/s 2024-09-19T06:53:20.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:20 smithi103 ceph-mon[31158]: pgmap v126: 97 pgs: 97 active+clean; 3.9 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 21 op/s 2024-09-19T06:53:20.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:20 smithi134 ceph-mon[35356]: pgmap v126: 97 pgs: 97 active+clean; 3.9 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 21 op/s 2024-09-19T06:53:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.809+0000 7f0308fd6640 1 -- 172.21.15.103:0/3612924640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03041069d0 msgr2=0x7f0304106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.809+0000 7f0308fd6640 1 --2- 172.21.15.103:0/3612924640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03041069d0 0x7f0304106db0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009930 tx=0x7f02f402a420 comp rx=0 tx=0).stop 2024-09-19T06:53:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.810+0000 7f0308fd6640 1 -- 172.21.15.103:0/3612924640 shutdown_connections 2024-09-19T06:53:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.810+0000 7f0308fd6640 1 --2- 172.21.15.103:0/3612924640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0304101360 0x7f030410d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.810+0000 7f0308fd6640 1 --2- 172.21.15.103:0/3612924640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03041009c0 0x7f0304100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.810+0000 7f0308fd6640 1 --2- 172.21.15.103:0/3612924640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03041069d0 0x7f0304106db0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.810+0000 7f0308fd6640 1 -- 172.21.15.103:0/3612924640 >> 172.21.15.103:0/3612924640 conn(0x7f03040fc700 msgr2=0x7f03040feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.810+0000 7f0308fd6640 1 -- 172.21.15.103:0/3612924640 shutdown_connections 2024-09-19T06:53:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.810+0000 7f0308fd6640 1 -- 172.21.15.103:0/3612924640 wait complete. 2024-09-19T06:53:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.810+0000 7f0308fd6640 1 Processor -- start 2024-09-19T06:53:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0308fd6640 1 -- start start 2024-09-19T06:53:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0308fd6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03041009c0 0x7f030410e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0308fd6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0304101360 0x7f030410e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0308fd6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03041069d0 0x7f030410eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0308fd6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0304111f00 con 0x7f0304101360 2024-09-19T06:53:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0308fd6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0304111d80 con 0x7f03041009c0 2024-09-19T06:53:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0308fd6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0304112080 con 0x7f03041069d0 2024-09-19T06:53:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0301d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0304101360 0x7f030410e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0301d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0304101360 0x7f030410e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45068/0 (socket says 172.21.15.103:45068) 2024-09-19T06:53:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0301d9b640 1 -- 172.21.15.103:0/777208461 learned_addr learned my addr 172.21.15.103:0/777208461 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0302d9d640 1 --2- 172.21.15.103:0/777208461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03041069d0 0x7f030410eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f030259c640 1 --2- 172.21.15.103:0/777208461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03041009c0 0x7f030410e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0301d9b640 1 -- 172.21.15.103:0/777208461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03041069d0 msgr2=0x7f030410eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0301d9b640 1 --2- 172.21.15.103:0/777208461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03041069d0 0x7f030410eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0301d9b640 1 -- 172.21.15.103:0/777208461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03041009c0 msgr2=0x7f030410e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0301d9b640 1 --2- 172.21.15.103:0/777208461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03041009c0 0x7f030410e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0301d9b640 1 -- 172.21.15.103:0/777208461 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03041a6910 con 0x7f0304101360 2024-09-19T06:53:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f030259c640 1 --2- 172.21.15.103:0/777208461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03041009c0 0x7f030410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.811+0000 7f0301d9b640 1 --2- 172.21.15.103:0/777208461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0304101360 0x7f030410e860 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f02f800d950 tx=0x7f02f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.812+0000 7f02f37fe640 1 -- 172.21.15.103:0/777208461 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8017070 con 0x7f0304101360 2024-09-19T06:53:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.812+0000 7f0308fd6640 1 -- 172.21.15.103:0/777208461 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03041a6c00 con 0x7f0304101360 2024-09-19T06:53:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.812+0000 7f02f37fe640 1 -- 172.21.15.103:0/777208461 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f8009c30 con 0x7f0304101360 2024-09-19T06:53:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.812+0000 7f02f37fe640 1 -- 172.21.15.103:0/777208461 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f800f6b0 con 0x7f0304101360 2024-09-19T06:53:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.812+0000 7f0308fd6640 1 -- 172.21.15.103:0/777208461 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03041a71a0 con 0x7f0304101360 2024-09-19T06:53:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.814+0000 7f02f37fe640 1 -- 172.21.15.103:0/777208461 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f02f80109d0 con 0x7f0304101360 2024-09-19T06:53:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.814+0000 7f030be51740 1 Processor -- start 2024-09-19T06:53:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f02f37fe640 1 --2- 172.21.15.103:0/777208461 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f02ec0a3460 0x7f02ec0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f030be51740 1 -- start start 2024-09-19T06:53:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f02f37fe640 1 -- 172.21.15.103:0/777208461 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f02f80cf920 con 0x7f0304101360 2024-09-19T06:53:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f0302d9d640 1 --2- 172.21.15.103:0/777208461 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f02ec0a3460 0x7f02ec0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f030be51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb523af0 0x5618bb523ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f030be51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb524410 0x5618bb52e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f030be51740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb52ecb0 0x5618bb5310a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f030be51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5618bb4882f0 con 0x5618bb524410 2024-09-19T06:53:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f030be51740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5618bb430270 con 0x5618bb52ecb0 2024-09-19T06:53:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f030be51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5618bb4842b0 con 0x5618bb523af0 2024-09-19T06:53:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f0301d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb524410 0x5618bb52e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f0301d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb524410 0x5618bb52e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45080/0 (socket says 172.21.15.103:45080) 2024-09-19T06:53:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f0301d9b640 1 -- 172.21.15.103:0/3611264712 learned_addr learned my addr 172.21.15.103:0/3611264712 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f0302d9d640 1 --2- 172.21.15.103:0/777208461 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f02ec0a3460 0x7f02ec0a5920 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009fb0 tx=0x7f02fc00d040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f0302d9d640 1 --2- 172.21.15.103:0/3611264712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb52ecb0 0x5618bb5310a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f030259c640 1 --2- 172.21.15.103:0/3611264712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb523af0 0x5618bb523ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.815+0000 7f0301d9b640 1 -- 172.21.15.103:0/3611264712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb523af0 msgr2=0x5618bb523ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.816+0000 7f0301d9b640 1 --2- 172.21.15.103:0/3611264712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb523af0 0x5618bb523ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.816+0000 7f0301d9b640 1 -- 172.21.15.103:0/3611264712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb52ecb0 msgr2=0x5618bb5310a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.816+0000 7f0301d9b640 1 --2- 172.21.15.103:0/3611264712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb52ecb0 0x5618bb5310a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.816+0000 7f0301d9b640 1 -- 172.21.15.103:0/3611264712 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618bb42f340 con 0x5618bb524410 2024-09-19T06:53:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.816+0000 7f0302d9d640 1 --2- 172.21.15.103:0/3611264712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb52ecb0 0x5618bb5310a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.816+0000 7f030259c640 1 --2- 172.21.15.103:0/3611264712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb523af0 0x5618bb523ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.816+0000 7f0301d9b640 1 --2- 172.21.15.103:0/3611264712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb524410 0x5618bb52e770 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f02f80a4350 tx=0x7f02f809b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be26e57bdeaf9a15 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.816+0000 7f02f0ff9640 1 -- 172.21.15.103:0/3611264712 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8020040 con 0x5618bb524410 2024-09-19T06:53:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.816+0000 7f02f0ff9640 1 -- 172.21.15.103:0/3611264712 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f809ffc0 con 0x5618bb524410 2024-09-19T06:53:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.816+0000 7f02f0ff9640 1 -- 172.21.15.103:0/3611264712 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f809b2a0 con 0x5618bb524410 2024-09-19T06:53:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 -- 172.21.15.103:0/3611264712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb524410 msgr2=0x5618bb52e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 --2- 172.21.15.103:0/3611264712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb524410 0x5618bb52e770 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f02f80a4350 tx=0x7f02f809b6a0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 -- 172.21.15.103:0/3611264712 shutdown_connections 2024-09-19T06:53:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 --2- 172.21.15.103:0/3611264712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb52ecb0 0x5618bb5310a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 --2- 172.21.15.103:0/3611264712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb524410 0x5618bb52e770 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 --2- 172.21.15.103:0/3611264712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb523af0 0x5618bb523ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 -- 172.21.15.103:0/3611264712 >> 172.21.15.103:0/3611264712 conn(0x5618bb520060 msgr2=0x5618bb520440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 -- 172.21.15.103:0/3611264712 shutdown_connections 2024-09-19T06:53:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 -- 172.21.15.103:0/3611264712 wait complete. 2024-09-19T06:53:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 -- start start 2024-09-19T06:53:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 Processor -- start 2024-09-19T06:53:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb523af0 0x5618bb52bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb524410 0x5618bb52c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb52ecb0 0x5618bb52c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5618bb531880 con 0x5618bb523af0 2024-09-19T06:53:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5618bb411290 con 0x5618bb524410 2024-09-19T06:53:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.817+0000 7f030be51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5618bb531a00 con 0x5618bb52ecb0 2024-09-19T06:53:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f030be51740 10 client.0 _ll_register_callbacks cb 0x5618bb442140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f030259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb523af0 0x5618bb52bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f030259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb523af0 0x5618bb52bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45096/0 (socket says 172.21.15.103:45096) 2024-09-19T06:53:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f030259c640 1 -- 172.21.15.103:0/3829148151 learned_addr learned my addr 172.21.15.103:0/3829148151 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f0302d9d640 1 --2- 172.21.15.103:0/3829148151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb52ecb0 0x5618bb52c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f0301d9b640 1 --2- 172.21.15.103:0/3829148151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb524410 0x5618bb52c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f030259c640 1 -- 172.21.15.103:0/3829148151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb52ecb0 msgr2=0x5618bb52c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f030259c640 1 --2- 172.21.15.103:0/3829148151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb52ecb0 0x5618bb52c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f030259c640 1 -- 172.21.15.103:0/3829148151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb524410 msgr2=0x5618bb52c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f030259c640 1 --2- 172.21.15.103:0/3829148151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb524410 0x5618bb52c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f030259c640 1 -- 172.21.15.103:0/3829148151 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618bb19b210 con 0x5618bb523af0 2024-09-19T06:53:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f0301d9b640 1 --2- 172.21.15.103:0/3829148151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb524410 0x5618bb52c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f030259c640 1 --2- 172.21.15.103:0/3829148151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb523af0 0x5618bb52bb90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f02f4000c00 tx=0x7f02f4031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e66a689d0e522ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f02e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f02e9ffb640 1 -- 172.21.15.103:0/3829148151 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4035030 con 0x5618bb523af0 2024-09-19T06:53:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f02e9ffb640 1 -- 172.21.15.103:0/3829148151 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f02f4004250 con 0x5618bb523af0 2024-09-19T06:53:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f030be51740 1 -- 172.21.15.103:0/3829148151 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02f809ffc0 con 0x5618bb523af0 2024-09-19T06:53:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f02e9ffb640 1 -- 172.21.15.103:0/3829148151 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f02f404a050 con 0x5618bb523af0 2024-09-19T06:53:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f030be51740 10 client.16902 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.818+0000 7f030be51740 1 -- 172.21.15.103:0/3829148151 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618bb267620 con 0x5618bb523af0 2024-09-19T06:53:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.819+0000 7f02e9ffb640 1 -- 172.21.15.103:0/3829148151 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f809ffc0 con 0x5618bb523af0 2024-09-19T06:53:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.819+0000 7f02e9ffb640 1 -- 172.21.15.103:0/3829148151 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f02f4004990 con 0x5618bb523af0 2024-09-19T06:53:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.819+0000 7f030be51740 10 client.16902 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.819+0000 7f030be51740 10 client.16902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.819+0000 7f030be51740 20 client.16902 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.819+0000 7f030be51740 1 --2- 172.21.15.103:0/3829148151 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5618bb53eb40 0x5618bb540fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.819+0000 7f030be51740 4 client.16902 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.819+0000 7f030be51740 1 -- 172.21.15.103:0/3829148151 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618bb495a20 con 0x5618bb53eb40 2024-09-19T06:53:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.819+0000 7f0301d9b640 1 --2- 172.21.15.103:0/3829148151 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5618bb53eb40 0x5618bb540fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.820+0000 7f0301d9b640 1 --2- 172.21.15.103:0/3829148151 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5618bb53eb40 0x5618bb540fb0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f58920282523016 server_cookie=e85f21acd9bec1da in_seq=0 out_seq=0 2024-09-19T06:53:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.820+0000 7f02e9ffb640 10 client.16902 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.820+0000 7f02e9ffb640 1 -- 172.21.15.103:0/3829148151 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618bb495a20 con 0x5618bb53eb40 2024-09-19T06:53:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.820+0000 7f02e9ffb640 10 client.16902 handle_command_reply: tid=0 2024-09-19T06:53:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.820+0000 7f030be51740 1 client.16902 shutdown 2024-09-19T06:53:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.821+0000 7f030be51740 1 -- 172.21.15.103:0/3829148151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb523af0 msgr2=0x5618bb52bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.821+0000 7f030be51740 1 --2- 172.21.15.103:0/3829148151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb523af0 0x5618bb52bb90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f02f4000c00 tx=0x7f02f4031630 comp rx=0 tx=0).stop 2024-09-19T06:53:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.821+0000 7f030be51740 1 -- 172.21.15.103:0/3829148151 shutdown_connections 2024-09-19T06:53:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.821+0000 7f030be51740 1 --2- 172.21.15.103:0/3829148151 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5618bb53eb40 0x5618bb540fb0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.821+0000 7f030be51740 1 --2- 172.21.15.103:0/3829148151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb52ecb0 0x5618bb52c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.821+0000 7f030be51740 1 --2- 172.21.15.103:0/3829148151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb524410 0x5618bb52c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.821+0000 7f02e9ffb640 0 client.16902 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.821+0000 7f030be51740 1 --2- 172.21.15.103:0/3829148151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb523af0 0x5618bb52bb90 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.821+0000 7f030be51740 1 -- 172.21.15.103:0/3829148151 >> 172.21.15.103:0/3829148151 conn(0x5618bb520060 msgr2=0x5618bb525d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.821+0000 7f030be51740 1 -- 172.21.15.103:0/3829148151 shutdown_connections 2024-09-19T06:53:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.821+0000 7f030be51740 1 -- 172.21.15.103:0/3829148151 wait complete. 2024-09-19T06:53:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.821+0000 7f030be51740 20 client.16902 trim_cache size 0 max 16384 2024-09-19T06:53:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.835+0000 7f030be51740 1 Processor -- start 2024-09-19T06:53:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.835+0000 7f030be51740 1 -- start start 2024-09-19T06:53:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.835+0000 7f030be51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb523af0 0x5618bb52c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.835+0000 7f030be51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb524410 0x5618bb5468c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.835+0000 7f030be51740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb52ecb0 0x5618bb546e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.835+0000 7f030be51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03041afa90 con 0x5618bb523af0 2024-09-19T06:53:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.835+0000 7f030be51740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f03041b15d0 con 0x5618bb52ecb0 2024-09-19T06:53:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.835+0000 7f030be51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f03041b0160 con 0x5618bb524410 2024-09-19T06:53:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.835+0000 7f030259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb523af0 0x5618bb52c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.835+0000 7f030259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb523af0 0x5618bb52c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45098/0 (socket says 172.21.15.103:45098) 2024-09-19T06:53:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.835+0000 7f030259c640 1 -- 172.21.15.103:0/219160974 learned_addr learned my addr 172.21.15.103:0/219160974 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.836+0000 7f030259c640 1 -- 172.21.15.103:0/219160974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb524410 msgr2=0x5618bb5468c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.836+0000 7f030259c640 1 --2- 172.21.15.103:0/219160974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb524410 0x5618bb5468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.836+0000 7f030259c640 1 -- 172.21.15.103:0/219160974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb52ecb0 msgr2=0x5618bb546e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.836+0000 7f030259c640 1 --2- 172.21.15.103:0/219160974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb52ecb0 0x5618bb546e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.836+0000 7f030259c640 1 -- 172.21.15.103:0/219160974 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618bb51fe90 con 0x5618bb523af0 2024-09-19T06:53:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.836+0000 7f030259c640 1 --2- 172.21.15.103:0/219160974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb523af0 0x5618bb52c440 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f02f4005ab0 tx=0x7f02f402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3deb2266b8936996 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.836+0000 7f02ebfff640 1 -- 172.21.15.103:0/219160974 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4035070 con 0x5618bb523af0 2024-09-19T06:53:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.836+0000 7f02ebfff640 1 -- 172.21.15.103:0/219160974 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f4031c30 con 0x5618bb523af0 2024-09-19T06:53:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.836+0000 7f02ebfff640 1 -- 172.21.15.103:0/219160974 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f40052d0 con 0x5618bb523af0 2024-09-19T06:53:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 -- 172.21.15.103:0/219160974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb523af0 msgr2=0x5618bb52c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 --2- 172.21.15.103:0/219160974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb523af0 0x5618bb52c440 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f02f4005ab0 tx=0x7f02f402a420 comp rx=0 tx=0).stop 2024-09-19T06:53:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 -- 172.21.15.103:0/219160974 shutdown_connections 2024-09-19T06:53:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 --2- 172.21.15.103:0/219160974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb52ecb0 0x5618bb546e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 --2- 172.21.15.103:0/219160974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb524410 0x5618bb5468c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 --2- 172.21.15.103:0/219160974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb523af0 0x5618bb52c440 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 -- 172.21.15.103:0/219160974 >> 172.21.15.103:0/219160974 conn(0x5618bb520060 msgr2=0x5618bb52db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 -- 172.21.15.103:0/219160974 shutdown_connections 2024-09-19T06:53:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 -- 172.21.15.103:0/219160974 wait complete. 2024-09-19T06:53:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 -- start start 2024-09-19T06:53:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 Processor -- start 2024-09-19T06:53:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb523af0 0x5618bb536eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb524410 0x5618bb5373f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb52ecb0 0x5618bb533c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03041b0f00 con 0x5618bb524410 2024-09-19T06:53:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f03041b0830 con 0x5618bb52ecb0 2024-09-19T06:53:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f03041ada50 con 0x5618bb523af0 2024-09-19T06:53:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f030be51740 10 client.0 _ll_register_callbacks cb 0x7f02f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f0301d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb524410 0x5618bb5373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f0301d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb524410 0x5618bb5373f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45114/0 (socket says 172.21.15.103:45114) 2024-09-19T06:53:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.837+0000 7f0301d9b640 1 -- 172.21.15.103:0/3899388100 learned_addr learned my addr 172.21.15.103:0/3899388100 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f0302d9d640 1 --2- 172.21.15.103:0/3899388100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb52ecb0 0x5618bb533c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f030259c640 1 --2- 172.21.15.103:0/3899388100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb523af0 0x5618bb536eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f0301d9b640 1 -- 172.21.15.103:0/3899388100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb523af0 msgr2=0x5618bb536eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f0301d9b640 1 --2- 172.21.15.103:0/3899388100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb523af0 0x5618bb536eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f0301d9b640 1 -- 172.21.15.103:0/3899388100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb52ecb0 msgr2=0x5618bb533c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f0301d9b640 1 --2- 172.21.15.103:0/3899388100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb52ecb0 0x5618bb533c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f0301d9b640 1 -- 172.21.15.103:0/3899388100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618bb520cf0 con 0x5618bb524410 2024-09-19T06:53:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f0302d9d640 1 --2- 172.21.15.103:0/3899388100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb52ecb0 0x5618bb533c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f030259c640 1 --2- 172.21.15.103:0/3899388100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb523af0 0x5618bb536eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f0301d9b640 1 --2- 172.21.15.103:0/3899388100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb524410 0x5618bb5373f0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f02f809bba0 tx=0x7f02f800d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fbf6be0219d734 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f02e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f02e9ffb640 1 -- 172.21.15.103:0/3899388100 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8020040 con 0x5618bb524410 2024-09-19T06:53:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f02e9ffb640 1 -- 172.21.15.103:0/3899388100 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f02f8098280 con 0x5618bb524410 2024-09-19T06:53:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f030be51740 1 -- 172.21.15.103:0/3899388100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02f4031c30 con 0x5618bb524410 2024-09-19T06:53:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f02e9ffb640 1 -- 172.21.15.103:0/3899388100 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f02f809ffc0 con 0x5618bb524410 2024-09-19T06:53:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f030be51740 10 client.16908 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f030be51740 1 -- 172.21.15.103:0/3899388100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618bb523920 con 0x5618bb524410 2024-09-19T06:53:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f02e9ffb640 1 -- 172.21.15.103:0/3899388100 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f4031c30 con 0x5618bb524410 2024-09-19T06:53:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f02e9ffb640 1 -- 172.21.15.103:0/3899388100 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f02f809b0f0 con 0x5618bb524410 2024-09-19T06:53:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f030be51740 10 client.16908 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f030be51740 10 client.16908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f030be51740 20 client.16908 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f030be51740 1 --2- 172.21.15.103:0/3899388100 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5618bb5aa280 0x5618bb5ac670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f030be51740 4 client.16908 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:53:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.838+0000 7f030be51740 1 -- 172.21.15.103:0/3899388100 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5618bb48fd70 con 0x5618bb5aa280 2024-09-19T06:53:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.839+0000 7f030259c640 1 --2- 172.21.15.103:0/3899388100 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5618bb5aa280 0x5618bb5ac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.839+0000 7f030259c640 1 --2- 172.21.15.103:0/3899388100 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5618bb5aa280 0x5618bb5ac670 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be8f3db8e2005322 server_cookie=f84425d2a311b8ca in_seq=0 out_seq=0 2024-09-19T06:53:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.839+0000 7f02e9ffb640 10 client.16908 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.852+0000 7f02e9ffb640 1 -- 172.21.15.103:0/3899388100 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5618bb48fd70 con 0x5618bb5aa280 2024-09-19T06:53:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.852+0000 7f02e9ffb640 10 client.16908 handle_command_reply: tid=0 2024-09-19T06:53:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.852+0000 7f030be51740 1 client.16908 shutdown 2024-09-19T06:53:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.852+0000 7f030be51740 1 -- 172.21.15.103:0/3899388100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb524410 msgr2=0x5618bb5373f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.852+0000 7f030be51740 1 --2- 172.21.15.103:0/3899388100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb524410 0x5618bb5373f0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f02f809bba0 tx=0x7f02f800d8d0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.852+0000 7f030be51740 1 -- 172.21.15.103:0/3899388100 shutdown_connections 2024-09-19T06:53:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.852+0000 7f030be51740 1 --2- 172.21.15.103:0/3899388100 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5618bb5aa280 0x5618bb5ac670 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.852+0000 7f030be51740 1 --2- 172.21.15.103:0/3899388100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618bb52ecb0 0x5618bb533c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.852+0000 7f030be51740 1 --2- 172.21.15.103:0/3899388100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618bb524410 0x5618bb5373f0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.852+0000 7f02e9ffb640 0 client.16908 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.852+0000 7f030be51740 1 --2- 172.21.15.103:0/3899388100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618bb523af0 0x5618bb536eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.852+0000 7f030be51740 1 -- 172.21.15.103:0/3899388100 >> 172.21.15.103:0/3899388100 conn(0x5618bb520060 msgr2=0x5618bb525d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.852+0000 7f030be51740 1 -- 172.21.15.103:0/3899388100 shutdown_connections 2024-09-19T06:53:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.852+0000 7f030be51740 1 -- 172.21.15.103:0/3899388100 wait complete. 2024-09-19T06:53:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.852+0000 7f030be51740 20 client.16908 trim_cache size 0 max 16384 2024-09-19T06:53:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:53:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:53:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:53:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:53:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.853+0000 7f0308fd6640 1 -- 172.21.15.103:0/777208461 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f02ec0a3460 msgr2=0x7f02ec0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.853+0000 7f0308fd6640 1 --2- 172.21.15.103:0/777208461 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f02ec0a3460 0x7f02ec0a5920 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009fb0 tx=0x7f02fc00d040 comp rx=0 tx=0).stop 2024-09-19T06:53:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.853+0000 7f0308fd6640 1 -- 172.21.15.103:0/777208461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0304101360 msgr2=0x7f030410e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.853+0000 7f0308fd6640 1 --2- 172.21.15.103:0/777208461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0304101360 0x7f030410e860 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f02f800d950 tx=0x7f02f800de10 comp rx=0 tx=0).stop 2024-09-19T06:53:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.853+0000 7f0308fd6640 1 -- 172.21.15.103:0/777208461 shutdown_connections 2024-09-19T06:53:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.853+0000 7f0308fd6640 1 --2- 172.21.15.103:0/777208461 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f02ec0a3460 0x7f02ec0a5920 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.853+0000 7f0308fd6640 1 --2- 172.21.15.103:0/777208461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03041069d0 0x7f030410eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.853+0000 7f0308fd6640 1 --2- 172.21.15.103:0/777208461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0304101360 0x7f030410e860 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.853+0000 7f0308fd6640 1 --2- 172.21.15.103:0/777208461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03041009c0 0x7f030410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.853+0000 7f0308fd6640 1 -- 172.21.15.103:0/777208461 >> 172.21.15.103:0/777208461 conn(0x7f03040fc700 msgr2=0x7f03040fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.853+0000 7f0308fd6640 1 -- 172.21.15.103:0/777208461 shutdown_connections 2024-09-19T06:53:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:21.853+0000 7f0308fd6640 1 -- 172.21.15.103:0/777208461 wait complete. 2024-09-19T06:53:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:22 smithi174 ceph-mon[35637]: pgmap v127: 97 pgs: 97 active+clean; 3.9 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-19T06:53:22.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:22 smithi103 ceph-mon[31158]: pgmap v127: 97 pgs: 97 active+clean; 3.9 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-19T06:53:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:22 smithi134 ceph-mon[35356]: pgmap v127: 97 pgs: 97 active+clean; 3.9 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-19T06:53:24.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:24 smithi174 ceph-mon[35637]: pgmap v128: 97 pgs: 97 active+clean; 3.9 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-19T06:53:24.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:24 smithi103 ceph-mon[31158]: pgmap v128: 97 pgs: 97 active+clean; 3.9 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-19T06:53:24.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:24 smithi134 ceph-mon[35356]: pgmap v128: 97 pgs: 97 active+clean; 3.9 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-19T06:53:26.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:26 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:53:26.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:26 smithi174 ceph-mon[35637]: pgmap v129: 97 pgs: 97 active+clean; 4.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 21 op/s 2024-09-19T06:53:26.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:26 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:53:26.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:26 smithi103 ceph-mon[31158]: pgmap v129: 97 pgs: 97 active+clean; 4.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 21 op/s 2024-09-19T06:53:26.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:26 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:53:26.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:26 smithi134 ceph-mon[35356]: pgmap v129: 97 pgs: 97 active+clean; 4.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 21 op/s 2024-09-19T06:53:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:27 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:53:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:27 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:53:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:27 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:53:27.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:27 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:53:27.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:27 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:53:27.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:27 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:53:27.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:27 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:53:27.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:27 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:53:27.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:27 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:53:28.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:28 smithi174 ceph-mon[35637]: pgmap v130: 97 pgs: 97 active+clean; 4.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-09-19T06:53:28.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:28 smithi103 ceph-mon[31158]: pgmap v130: 97 pgs: 97 active+clean; 4.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-09-19T06:53:28.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:28 smithi134 ceph-mon[35356]: pgmap v130: 97 pgs: 97 active+clean; 4.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-09-19T06:53:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:30 smithi174 ceph-mon[35637]: pgmap v131: 97 pgs: 97 active+clean; 4.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 21 op/s 2024-09-19T06:53:30.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:30 smithi103 ceph-mon[31158]: pgmap v131: 97 pgs: 97 active+clean; 4.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 21 op/s 2024-09-19T06:53:30.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:30 smithi134 ceph-mon[35356]: pgmap v131: 97 pgs: 97 active+clean; 4.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 21 op/s 2024-09-19T06:53:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:32 smithi174 ceph-mon[35637]: pgmap v132: 97 pgs: 97 active+clean; 4.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-09-19T06:53:32.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:32 smithi103 ceph-mon[31158]: pgmap v132: 97 pgs: 97 active+clean; 4.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-09-19T06:53:32.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:32 smithi134 ceph-mon[35356]: pgmap v132: 97 pgs: 97 active+clean; 4.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-09-19T06:53:32.704 INFO:tasks.quiescer.fs.[cephfs]:Total duration (59.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T06:53:32.704 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-09-19T06:53:32.705 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:53:32.971 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:33.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.408+0000 7f73f8a2a640 1 -- 172.21.15.103:0/3453502844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73f4102dd0 msgr2=0x7f73f41031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:33.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.408+0000 7f73f8a2a640 1 --2- 172.21.15.103:0/3453502844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73f4102dd0 0x7f73f41031b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009cd0 tx=0x7f73e402a8d0 comp rx=0 tx=0).stop 2024-09-19T06:53:33.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.408+0000 7f73f8a2a640 1 -- 172.21.15.103:0/3453502844 shutdown_connections 2024-09-19T06:53:33.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.408+0000 7f73f8a2a640 1 --2- 172.21.15.103:0/3453502844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73f4108300 0x7f73f410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:33.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.408+0000 7f73f8a2a640 1 --2- 172.21.15.103:0/3453502844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73f4103780 0x7f73f4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:33.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.408+0000 7f73f8a2a640 1 --2- 172.21.15.103:0/3453502844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73f4102dd0 0x7f73f41031b0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:33.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.408+0000 7f73f8a2a640 1 -- 172.21.15.103:0/3453502844 >> 172.21.15.103:0/3453502844 conn(0x7f73f40fc7d0 msgr2=0x7f73f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:33.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.408+0000 7f73f8a2a640 1 -- 172.21.15.103:0/3453502844 shutdown_connections 2024-09-19T06:53:33.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.408+0000 7f73f8a2a640 1 -- 172.21.15.103:0/3453502844 wait complete. 2024-09-19T06:53:33.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f8a2a640 1 Processor -- start 2024-09-19T06:53:33.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f8a2a640 1 -- start start 2024-09-19T06:53:33.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f8a2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73f4102dd0 0x7f73f41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:33.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f8a2a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73f4103780 0x7f73f4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:33.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f8a2a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73f4108300 0x7f73f419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:33.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f8a2a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f73f41120b0 con 0x7f73f4102dd0 2024-09-19T06:53:33.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f8a2a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f73f4111f30 con 0x7f73f4108300 2024-09-19T06:53:33.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f8a2a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f73f4112230 con 0x7f73f4103780 2024-09-19T06:53:33.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73f4102dd0 0x7f73f41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:33.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73f4102dd0 0x7f73f41985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51850/0 (socket says 172.21.15.103:51850) 2024-09-19T06:53:33.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f259c640 1 -- 172.21.15.103:0/1547570146 learned_addr learned my addr 172.21.15.103:0/1547570146 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:33.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f1d9b640 1 --2- 172.21.15.103:0/1547570146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73f4103780 0x7f73f4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:33.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f259c640 1 -- 172.21.15.103:0/1547570146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73f4103780 msgr2=0x7f73f4198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:33.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f259c640 1 --2- 172.21.15.103:0/1547570146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73f4103780 0x7f73f4198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:33.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f2d9d640 1 --2- 172.21.15.103:0/1547570146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73f4108300 0x7f73f419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:33.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f259c640 1 -- 172.21.15.103:0/1547570146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73f4108300 msgr2=0x7f73f419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:33.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f259c640 1 --2- 172.21.15.103:0/1547570146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73f4108300 0x7f73f419cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:33.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f259c640 1 -- 172.21.15.103:0/1547570146 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f419d5a0 con 0x7f73f4102dd0 2024-09-19T06:53:33.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.409+0000 7f73f259c640 1 --2- 172.21.15.103:0/1547570146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73f4102dd0 0x7f73f41985f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009df0 tx=0x7f73e4031420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:33.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.410+0000 7f73e37fe640 1 -- 172.21.15.103:0/1547570146 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4031840 con 0x7f73f4102dd0 2024-09-19T06:53:33.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.410+0000 7f73f8a2a640 1 -- 172.21.15.103:0/1547570146 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f419d830 con 0x7f73f4102dd0 2024-09-19T06:53:33.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.410+0000 7f73e37fe640 1 -- 172.21.15.103:0/1547570146 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e4003e50 con 0x7f73f4102dd0 2024-09-19T06:53:33.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.410+0000 7f73e37fe640 1 -- 172.21.15.103:0/1547570146 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e40307f0 con 0x7f73f4102dd0 2024-09-19T06:53:33.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.410+0000 7f73f8a2a640 1 -- 172.21.15.103:0/1547570146 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f41a5170 con 0x7f73f4102dd0 2024-09-19T06:53:33.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.411+0000 7f73f8a2a640 1 -- 172.21.15.103:0/1547570146 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f41031b0 con 0x7f73f4102dd0 2024-09-19T06:53:33.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.412+0000 7f73e37fe640 1 -- 172.21.15.103:0/1547570146 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f73e4004340 con 0x7f73f4102dd0 2024-09-19T06:53:33.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.412+0000 7f73e37fe640 1 --2- 172.21.15.103:0/1547570146 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f73dc0a33b0 0x7f73dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:33.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.413+0000 7f73f1d9b640 1 --2- 172.21.15.103:0/1547570146 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f73dc0a33b0 0x7f73dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:33.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.413+0000 7f73e37fe640 1 -- 172.21.15.103:0/1547570146 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f73e40f0cb0 con 0x7f73f4102dd0 2024-09-19T06:53:33.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.413+0000 7f73f1d9b640 1 --2- 172.21.15.103:0/1547570146 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f73dc0a33b0 0x7f73dc0a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73f40feef0 tx=0x7f73e800a250 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:33.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.416+0000 7f73e37fe640 1 -- 172.21.15.103:0/1547570146 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f73e40bd7b0 con 0x7f73f4102dd0 2024-09-19T06:53:33.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.615+0000 7f73f8a2a640 1 -- 172.21.15.103:0/1547570146 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73f410a290 con 0x7f73f4102dd0 2024-09-19T06:53:33.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.616+0000 7f73e37fe640 1 -- 172.21.15.103:0/1547570146 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f73e40bd0b0 con 0x7f73f4102dd0 2024-09-19T06:53:33.617 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:53:33.618 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:53:33.618 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:53:33.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.618+0000 7f73f8a2a640 1 -- 172.21.15.103:0/1547570146 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f73dc0a33b0 msgr2=0x7f73dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:33.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.618+0000 7f73f8a2a640 1 --2- 172.21.15.103:0/1547570146 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f73dc0a33b0 0x7f73dc0a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73f40feef0 tx=0x7f73e800a250 comp rx=0 tx=0).stop 2024-09-19T06:53:33.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.618+0000 7f73f8a2a640 1 -- 172.21.15.103:0/1547570146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73f4102dd0 msgr2=0x7f73f41985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:33.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.618+0000 7f73f8a2a640 1 --2- 172.21.15.103:0/1547570146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73f4102dd0 0x7f73f41985f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009df0 tx=0x7f73e4031420 comp rx=0 tx=0).stop 2024-09-19T06:53:33.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.618+0000 7f73f8a2a640 1 -- 172.21.15.103:0/1547570146 shutdown_connections 2024-09-19T06:53:33.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.618+0000 7f73f8a2a640 1 --2- 172.21.15.103:0/1547570146 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f73dc0a33b0 0x7f73dc0a5870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:33.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.618+0000 7f73f8a2a640 1 --2- 172.21.15.103:0/1547570146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73f4108300 0x7f73f419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:33.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.618+0000 7f73f8a2a640 1 --2- 172.21.15.103:0/1547570146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73f4103780 0x7f73f4198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:33.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.618+0000 7f73f8a2a640 1 --2- 172.21.15.103:0/1547570146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73f4102dd0 0x7f73f41985f0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:33.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.618+0000 7f73f8a2a640 1 -- 172.21.15.103:0/1547570146 >> 172.21.15.103:0/1547570146 conn(0x7f73f40fc7d0 msgr2=0x7f73f4105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:33.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.618+0000 7f73f8a2a640 1 -- 172.21.15.103:0/1547570146 shutdown_connections 2024-09-19T06:53:33.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:33.618+0000 7f73f8a2a640 1 -- 172.21.15.103:0/1547570146 wait complete. 2024-09-19T06:53:33.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:33 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1547570146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:53:34.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:33 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1547570146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:53:34.112 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:33 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1547570146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:53:34.313 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-09-19T06:53:34.313 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-09-19T06:53:34.560 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:34.772 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:53:34.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:34 smithi134 ceph-mon[35356]: pgmap v133: 97 pgs: 97 active+clean; 4.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-09-19T06:53:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.942+0000 7fc386853640 1 -- 172.21.15.103:0/1926125410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc38010c0b0 msgr2=0x7fc38010e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.942+0000 7fc386853640 1 --2- 172.21.15.103:0/1926125410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc38010c0b0 0x7fc38010e550 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc378009ca0 tx=0x7fc37802a950 comp rx=0 tx=0).stop 2024-09-19T06:53:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.942+0000 7fc386853640 1 -- 172.21.15.103:0/1926125410 shutdown_connections 2024-09-19T06:53:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.942+0000 7fc386853640 1 --2- 172.21.15.103:0/1926125410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc38010c0b0 0x7fc38010e550 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.942+0000 7fc386853640 1 --2- 172.21.15.103:0/1926125410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc380103600 0x7fc38010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.942+0000 7fc386853640 1 --2- 172.21.15.103:0/1926125410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc380102ce0 0x7fc3801030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.942+0000 7fc386853640 1 -- 172.21.15.103:0/1926125410 >> 172.21.15.103:0/1926125410 conn(0x7fc380077990 msgr2=0x7fc3800fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.942+0000 7fc386853640 1 -- 172.21.15.103:0/1926125410 shutdown_connections 2024-09-19T06:53:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.942+0000 7fc386853640 1 -- 172.21.15.103:0/1926125410 wait complete. 2024-09-19T06:53:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.942+0000 7fc386853640 1 Processor -- start 2024-09-19T06:53:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.942+0000 7fc386853640 1 -- start start 2024-09-19T06:53:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc386853640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc380102ce0 0x7fc3801982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc386853640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc380103600 0x7fc380198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc386853640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc38010c0b0 0x7fc38019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc386853640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc380111e90 con 0x7fc380102ce0 2024-09-19T06:53:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc386853640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc380111d10 con 0x7fc38010c0b0 2024-09-19T06:53:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc386853640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc380112010 con 0x7fc380103600 2024-09-19T06:53:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc37ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc380102ce0 0x7fc3801982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc37ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc380102ce0 0x7fc3801982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51866/0 (socket says 172.21.15.103:51866) 2024-09-19T06:53:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc37ffff640 1 -- 172.21.15.103:0/929035723 learned_addr learned my addr 172.21.15.103:0/929035723 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc384df0640 1 --2- 172.21.15.103:0/929035723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc38010c0b0 0x7fc38019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc37f7fe640 1 --2- 172.21.15.103:0/929035723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc380103600 0x7fc380198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc37ffff640 1 -- 172.21.15.103:0/929035723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc380103600 msgr2=0x7fc380198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc37ffff640 1 --2- 172.21.15.103:0/929035723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc380103600 0x7fc380198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc37ffff640 1 -- 172.21.15.103:0/929035723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc38010c0b0 msgr2=0x7fc38019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc37ffff640 1 --2- 172.21.15.103:0/929035723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc38010c0b0 0x7fc38019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc37ffff640 1 -- 172.21.15.103:0/929035723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38019d2b0 con 0x7fc380102ce0 2024-09-19T06:53:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc37ffff640 1 --2- 172.21.15.103:0/929035723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc380102ce0 0x7fc3801982c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc37000d950 tx=0x7fc37000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc37d7fa640 1 -- 172.21.15.103:0/929035723 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370017030 con 0x7fc380102ce0 2024-09-19T06:53:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc386853640 1 -- 172.21.15.103:0/929035723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc38019d5a0 con 0x7fc380102ce0 2024-09-19T06:53:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc37d7fa640 1 -- 172.21.15.103:0/929035723 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc370004020 con 0x7fc380102ce0 2024-09-19T06:53:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc37d7fa640 1 -- 172.21.15.103:0/929035723 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3700047a0 con 0x7fc380102ce0 2024-09-19T06:53:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.943+0000 7fc386853640 1 -- 172.21.15.103:0/929035723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3801a4e30 con 0x7fc380102ce0 2024-09-19T06:53:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.946+0000 7fc3895eb740 1 Processor -- start 2024-09-19T06:53:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.946+0000 7fc37d7fa640 1 -- 172.21.15.103:0/929035723 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc3700041d0 con 0x7fc380102ce0 2024-09-19T06:53:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.946+0000 7fc3895eb740 1 -- start start 2024-09-19T06:53:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.946+0000 7fc3895eb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e529b0 0x562f66e52d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.946+0000 7fc3895eb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e532d0 0x562f66e61b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.946+0000 7fc3895eb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e620c0 0x562f66e644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.946+0000 7fc3895eb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562f66db72f0 con 0x562f66e620c0 2024-09-19T06:53:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.946+0000 7fc3895eb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562f66d5f270 con 0x562f66e532d0 2024-09-19T06:53:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.946+0000 7fc3895eb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562f66db32b0 con 0x562f66e529b0 2024-09-19T06:53:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.947+0000 7fc37f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e620c0 0x562f66e644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.947+0000 7fc37f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e620c0 0x562f66e644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51870/0 (socket says 172.21.15.103:51870) 2024-09-19T06:53:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.947+0000 7fc37f7fe640 1 -- 172.21.15.103:0/3713322173 learned_addr learned my addr 172.21.15.103:0/3713322173 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.947+0000 7fc384df0640 1 --2- 172.21.15.103:0/3713322173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e529b0 0x562f66e52d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.947+0000 7fc37ffff640 1 --2- 172.21.15.103:0/3713322173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e532d0 0x562f66e61b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.947+0000 7fc37f7fe640 1 -- 172.21.15.103:0/3713322173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e529b0 msgr2=0x562f66e52d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.947+0000 7fc37f7fe640 1 --2- 172.21.15.103:0/3713322173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e529b0 0x562f66e52d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.947+0000 7fc37f7fe640 1 -- 172.21.15.103:0/3713322173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e532d0 msgr2=0x562f66e61b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.947+0000 7fc37f7fe640 1 --2- 172.21.15.103:0/3713322173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e532d0 0x562f66e61b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.947+0000 7fc37f7fe640 1 -- 172.21.15.103:0/3713322173 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f66d5e340 con 0x562f66e620c0 2024-09-19T06:53:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.947+0000 7fc37d7fa640 1 --2- 172.21.15.103:0/929035723 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3680a3460 0x7fc3680a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.947+0000 7fc37f7fe640 1 --2- 172.21.15.103:0/3713322173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e620c0 0x562f66e644b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc374009ae0 tx=0x7fc37400e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c079510fb792851 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.947+0000 7fc384df0640 1 --2- 172.21.15.103:0/929035723 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3680a3460 0x7fc3680a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.947+0000 7fc37d7fa640 1 -- 172.21.15.103:0/929035723 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc3700cf790 con 0x7fc380102ce0 2024-09-19T06:53:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.947+0000 7fc3667fc640 1 -- 172.21.15.103:0/3713322173 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374016070 con 0x562f66e620c0 2024-09-19T06:53:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.948+0000 7fc3667fc640 1 -- 172.21.15.103:0/3713322173 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc37400ecc0 con 0x562f66e620c0 2024-09-19T06:53:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.948+0000 7fc3667fc640 1 -- 172.21.15.103:0/3713322173 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374010750 con 0x562f66e620c0 2024-09-19T06:53:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.948+0000 7fc3895eb740 1 -- 172.21.15.103:0/3713322173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e620c0 msgr2=0x562f66e644b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.948+0000 7fc384df0640 1 --2- 172.21.15.103:0/929035723 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3680a3460 0x7fc3680a5920 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc378004af0 tx=0x7fc378005d80 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.948+0000 7fc3895eb740 1 --2- 172.21.15.103:0/3713322173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e620c0 0x562f66e644b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc374009ae0 tx=0x7fc37400e2f0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.948+0000 7fc3895eb740 1 -- 172.21.15.103:0/3713322173 shutdown_connections 2024-09-19T06:53:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.948+0000 7fc3895eb740 1 --2- 172.21.15.103:0/3713322173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e620c0 0x562f66e644b0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.948+0000 7fc3895eb740 1 --2- 172.21.15.103:0/3713322173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e532d0 0x562f66e61b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.948+0000 7fc3895eb740 1 --2- 172.21.15.103:0/3713322173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e529b0 0x562f66e52d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.949+0000 7fc3895eb740 1 -- 172.21.15.103:0/3713322173 >> 172.21.15.103:0/3713322173 conn(0x562f66e4f060 msgr2=0x562f66e4f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.949+0000 7fc3895eb740 1 -- 172.21.15.103:0/3713322173 shutdown_connections 2024-09-19T06:53:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.949+0000 7fc3895eb740 1 -- 172.21.15.103:0/3713322173 wait complete. 2024-09-19T06:53:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.949+0000 7fc3895eb740 1 -- start start 2024-09-19T06:53:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.949+0000 7fc3895eb740 1 Processor -- start 2024-09-19T06:53:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.949+0000 7fc3895eb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e529b0 0x562f66e5efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.949+0000 7fc3895eb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e532d0 0x562f66e5f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.949+0000 7fc3895eb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e620c0 0x562f66e5fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.949+0000 7fc3895eb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562f66e64c90 con 0x562f66e532d0 2024-09-19T06:53:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.949+0000 7fc3895eb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562f66d40290 con 0x562f66e529b0 2024-09-19T06:53:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.949+0000 7fc3895eb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562f66e64e10 con 0x562f66e620c0 2024-09-19T06:53:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.949+0000 7fc3895eb740 10 client.0 _ll_register_callbacks cb 0x562f66d71140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc37f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e532d0 0x562f66e5f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc37ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e529b0 0x562f66e5efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc37f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e532d0 0x562f66e5f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51874/0 (socket says 172.21.15.103:51874) 2024-09-19T06:53:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc37f7fe640 1 -- 172.21.15.103:0/1453438306 learned_addr learned my addr 172.21.15.103:0/1453438306 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc384df0640 1 --2- 172.21.15.103:0/1453438306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e620c0 0x562f66e5fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc37ffff640 1 -- 172.21.15.103:0/1453438306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e620c0 msgr2=0x562f66e5fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc37ffff640 1 --2- 172.21.15.103:0/1453438306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e620c0 0x562f66e5fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc37ffff640 1 -- 172.21.15.103:0/1453438306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e532d0 msgr2=0x562f66e5f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc37ffff640 1 --2- 172.21.15.103:0/1453438306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e532d0 0x562f66e5f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc37ffff640 1 -- 172.21.15.103:0/1453438306 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f66aca210 con 0x562f66e529b0 2024-09-19T06:53:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc37ffff640 1 --2- 172.21.15.103:0/1453438306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e529b0 0x562f66e5efc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3700ccec0 tx=0x7fc37000b530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f975b6acd8852707 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc35f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:53:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc35f7fe640 1 -- 172.21.15.103:0/1453438306 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37009b5c0 con 0x562f66e529b0 2024-09-19T06:53:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc3895eb740 1 -- 172.21.15.103:0/1453438306 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc37400ecc0 con 0x562f66e529b0 2024-09-19T06:53:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc35f7fe640 1 -- 172.21.15.103:0/1453438306 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc3700cd170 con 0x562f66e529b0 2024-09-19T06:53:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc35f7fe640 1 -- 172.21.15.103:0/1453438306 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc37009b3e0 con 0x562f66e529b0 2024-09-19T06:53:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc3895eb740 10 client.25961 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.950+0000 7fc3895eb740 1 -- 172.21.15.103:0/1453438306 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562f66b96620 con 0x562f66e529b0 2024-09-19T06:53:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.951+0000 7fc35f7fe640 1 -- 172.21.15.103:0/1453438306 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc37400ecc0 con 0x562f66e529b0 2024-09-19T06:53:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.951+0000 7fc35f7fe640 1 -- 172.21.15.103:0/1453438306 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc3700a4260 con 0x562f66e529b0 2024-09-19T06:53:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.951+0000 7fc3895eb740 10 client.25961 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.951+0000 7fc3895eb740 10 client.25961 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.951+0000 7fc3895eb740 20 client.25961 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.951+0000 7fc3895eb740 1 --2- 172.21.15.103:0/1453438306 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562f66e720c0 0x562f66e74510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.951+0000 7fc3895eb740 4 client.25961 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.951+0000 7fc3895eb740 1 -- 172.21.15.103:0/1453438306 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f66dc4a20 con 0x562f66e720c0 2024-09-19T06:53:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.951+0000 7fc37f7fe640 1 --2- 172.21.15.103:0/1453438306 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562f66e720c0 0x562f66e74510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.952+0000 7fc37f7fe640 1 --2- 172.21.15.103:0/1453438306 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562f66e720c0 0x562f66e74510 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8969858737d70684 server_cookie=ce5e1d8a5055cdb5 in_seq=0 out_seq=0 2024-09-19T06:53:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.952+0000 7fc35f7fe640 10 client.25961 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.952+0000 7fc35f7fe640 1 -- 172.21.15.103:0/1453438306 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f66dc4a20 con 0x562f66e720c0 2024-09-19T06:53:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.952+0000 7fc35f7fe640 10 client.25961 handle_command_reply: tid=0 2024-09-19T06:53:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.952+0000 7fc3895eb740 1 client.25961 shutdown 2024-09-19T06:53:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.952+0000 7fc3895eb740 1 -- 172.21.15.103:0/1453438306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e529b0 msgr2=0x562f66e5efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.952+0000 7fc3895eb740 1 --2- 172.21.15.103:0/1453438306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e529b0 0x562f66e5efc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3700ccec0 tx=0x7fc37000b530 comp rx=0 tx=0).stop 2024-09-19T06:53:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.953+0000 7fc3895eb740 1 -- 172.21.15.103:0/1453438306 shutdown_connections 2024-09-19T06:53:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.953+0000 7fc3895eb740 1 --2- 172.21.15.103:0/1453438306 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562f66e720c0 0x562f66e74510 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.953+0000 7fc3895eb740 1 --2- 172.21.15.103:0/1453438306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e620c0 0x562f66e5fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.953+0000 7fc35f7fe640 0 client.25961 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.953+0000 7fc3895eb740 1 --2- 172.21.15.103:0/1453438306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e532d0 0x562f66e5f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.953+0000 7fc3895eb740 1 --2- 172.21.15.103:0/1453438306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e529b0 0x562f66e5efc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.953+0000 7fc3895eb740 1 -- 172.21.15.103:0/1453438306 >> 172.21.15.103:0/1453438306 conn(0x562f66e4f060 msgr2=0x562f66e54dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.953+0000 7fc3895eb740 1 -- 172.21.15.103:0/1453438306 shutdown_connections 2024-09-19T06:53:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.953+0000 7fc3895eb740 1 -- 172.21.15.103:0/1453438306 wait complete. 2024-09-19T06:53:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.953+0000 7fc3895eb740 20 client.25961 trim_cache size 0 max 16384 2024-09-19T06:53:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.976+0000 7fc3895eb740 1 Processor -- start 2024-09-19T06:53:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.976+0000 7fc3895eb740 1 -- start start 2024-09-19T06:53:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.976+0000 7fc3895eb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e529b0 0x562f66e60f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.976+0000 7fc3895eb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e532d0 0x562f66e79e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.976+0000 7fc3895eb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e620c0 0x562f66e7a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.976+0000 7fc3895eb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3801ab560 con 0x562f66e532d0 2024-09-19T06:53:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.976+0000 7fc3895eb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3801ad0a0 con 0x562f66e529b0 2024-09-19T06:53:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.976+0000 7fc3895eb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3801abc30 con 0x562f66e620c0 2024-09-19T06:53:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.977+0000 7fc37f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e532d0 0x562f66e79e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.977+0000 7fc37f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e532d0 0x562f66e79e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51884/0 (socket says 172.21.15.103:51884) 2024-09-19T06:53:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.977+0000 7fc37f7fe640 1 -- 172.21.15.103:0/1969798255 learned_addr learned my addr 172.21.15.103:0/1969798255 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.977+0000 7fc37ffff640 1 --2- 172.21.15.103:0/1969798255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e529b0 0x562f66e60f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.977+0000 7fc384df0640 1 --2- 172.21.15.103:0/1969798255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e620c0 0x562f66e7a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.977+0000 7fc37f7fe640 1 -- 172.21.15.103:0/1969798255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e620c0 msgr2=0x562f66e7a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.977+0000 7fc37f7fe640 1 --2- 172.21.15.103:0/1969798255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e620c0 0x562f66e7a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.977+0000 7fc37f7fe640 1 -- 172.21.15.103:0/1969798255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e529b0 msgr2=0x562f66e60f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.977+0000 7fc37f7fe640 1 --2- 172.21.15.103:0/1969798255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e529b0 0x562f66e60f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.977+0000 7fc37f7fe640 1 -- 172.21.15.103:0/1969798255 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f66e5d4f0 con 0x562f66e532d0 2024-09-19T06:53:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.977+0000 7fc384df0640 1 --2- 172.21.15.103:0/1969798255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e620c0 0x562f66e7a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.977+0000 7fc37f7fe640 1 --2- 172.21.15.103:0/1969798255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e532d0 0x562f66e79e20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc374011f90 tx=0x7fc3740058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7db6384696fa47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.977+0000 7fc365ffb640 1 -- 172.21.15.103:0/1969798255 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374016040 con 0x562f66e532d0 2024-09-19T06:53:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.977+0000 7fc365ffb640 1 -- 172.21.15.103:0/1969798255 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc37400ec50 con 0x562f66e532d0 2024-09-19T06:53:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.977+0000 7fc365ffb640 1 -- 172.21.15.103:0/1969798255 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374004bf0 con 0x562f66e532d0 2024-09-19T06:53:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.978+0000 7fc3895eb740 1 -- 172.21.15.103:0/1969798255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e532d0 msgr2=0x562f66e79e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.978+0000 7fc3895eb740 1 --2- 172.21.15.103:0/1969798255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e532d0 0x562f66e79e20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc374011f90 tx=0x7fc3740058c0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.978+0000 7fc3895eb740 1 -- 172.21.15.103:0/1969798255 shutdown_connections 2024-09-19T06:53:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.978+0000 7fc3895eb740 1 --2- 172.21.15.103:0/1969798255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e620c0 0x562f66e7a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.978+0000 7fc3895eb740 1 --2- 172.21.15.103:0/1969798255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e532d0 0x562f66e79e20 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.978+0000 7fc3895eb740 1 --2- 172.21.15.103:0/1969798255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e529b0 0x562f66e60f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.978+0000 7fc3895eb740 1 -- 172.21.15.103:0/1969798255 >> 172.21.15.103:0/1969798255 conn(0x562f66e4f060 msgr2=0x562f66e60820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.978+0000 7fc3895eb740 1 -- 172.21.15.103:0/1969798255 shutdown_connections 2024-09-19T06:53:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.978+0000 7fc3895eb740 1 -- 172.21.15.103:0/1969798255 wait complete. 2024-09-19T06:53:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.978+0000 7fc3895eb740 1 -- start start 2024-09-19T06:53:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc3895eb740 1 Processor -- start 2024-09-19T06:53:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc3895eb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e529b0 0x562f66e6a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc3895eb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e532d0 0x562f66e5bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc3895eb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e620c0 0x562f66e5c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc3895eb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3801ac9d0 con 0x562f66e529b0 2024-09-19T06:53:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc3895eb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3801ac300 con 0x562f66e620c0 2024-09-19T06:53:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc3895eb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3801a9520 con 0x562f66e532d0 2024-09-19T06:53:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc37ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e529b0 0x562f66e6a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc37ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e529b0 0x562f66e6a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51892/0 (socket says 172.21.15.103:51892) 2024-09-19T06:53:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc37ffff640 1 -- 172.21.15.103:0/1705981179 learned_addr learned my addr 172.21.15.103:0/1705981179 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc3895eb740 10 client.0 _ll_register_callbacks cb 0x7fc370004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc384df0640 1 --2- 172.21.15.103:0/1705981179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e620c0 0x562f66e5c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc37f7fe640 1 --2- 172.21.15.103:0/1705981179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e532d0 0x562f66e5bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc37ffff640 1 -- 172.21.15.103:0/1705981179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e532d0 msgr2=0x562f66e5bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc37ffff640 1 --2- 172.21.15.103:0/1705981179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e532d0 0x562f66e5bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc37ffff640 1 -- 172.21.15.103:0/1705981179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e620c0 msgr2=0x562f66e5c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc37ffff640 1 --2- 172.21.15.103:0/1705981179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e620c0 0x562f66e5c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc37ffff640 1 -- 172.21.15.103:0/1705981179 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f66e5ca80 con 0x562f66e529b0 2024-09-19T06:53:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.979+0000 7fc37f7fe640 1 --2- 172.21.15.103:0/1705981179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e532d0 0x562f66e5bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.980+0000 7fc37ffff640 1 --2- 172.21.15.103:0/1705981179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e529b0 0x562f66e6a8c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc370097ee0 tx=0x7fc370098320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe08f8ae05e6400 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.980+0000 7fc35f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.980+0000 7fc35f7fe640 1 -- 172.21.15.103:0/1705981179 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37009b5c0 con 0x562f66e529b0 2024-09-19T06:53:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.980+0000 7fc35f7fe640 1 -- 172.21.15.103:0/1705981179 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc37009aed0 con 0x562f66e529b0 2024-09-19T06:53:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.980+0000 7fc3895eb740 1 -- 172.21.15.103:0/1705981179 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc37400ec50 con 0x562f66e529b0 2024-09-19T06:53:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.980+0000 7fc35f7fe640 1 -- 172.21.15.103:0/1705981179 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc3700cd380 con 0x562f66e529b0 2024-09-19T06:53:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.980+0000 7fc3895eb740 10 client.16932 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.980+0000 7fc3895eb740 1 -- 172.21.15.103:0/1705981179 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562f66edcd80 con 0x562f66e529b0 2024-09-19T06:53:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.980+0000 7fc35f7fe640 1 -- 172.21.15.103:0/1705981179 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc37400ec50 con 0x562f66e529b0 2024-09-19T06:53:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.980+0000 7fc35f7fe640 1 -- 172.21.15.103:0/1705981179 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc37009a940 con 0x562f66e529b0 2024-09-19T06:53:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.980+0000 7fc3895eb740 10 client.16932 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.980+0000 7fc3895eb740 10 client.16932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.980+0000 7fc3895eb740 20 client.16932 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.980+0000 7fc3895eb740 1 --2- 172.21.15.103:0/1705981179 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562f66edd340 0x562f66edf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.980+0000 7fc3895eb740 4 client.16932 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:53:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.980+0000 7fc3895eb740 1 -- 172.21.15.103:0/1705981179 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f66dbed70 con 0x562f66edd340 2024-09-19T06:53:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.981+0000 7fc37f7fe640 1 --2- 172.21.15.103:0/1705981179 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562f66edd340 0x562f66edf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.981+0000 7fc37f7fe640 1 --2- 172.21.15.103:0/1705981179 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562f66edd340 0x562f66edf760 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcf1d783df33a728 server_cookie=b726b0053d447b26 in_seq=0 out_seq=0 2024-09-19T06:53:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.981+0000 7fc35f7fe640 10 client.16932 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:34.991 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:34 smithi103 ceph-mon[31158]: pgmap v133: 97 pgs: 97 active+clean; 4.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-09-19T06:53:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.994+0000 7fc35f7fe640 1 -- 172.21.15.103:0/1705981179 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f66dbed70 con 0x562f66edd340 2024-09-19T06:53:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.994+0000 7fc35f7fe640 10 client.16932 handle_command_reply: tid=0 2024-09-19T06:53:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.994+0000 7fc3895eb740 1 client.16932 shutdown 2024-09-19T06:53:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.994+0000 7fc3895eb740 1 -- 172.21.15.103:0/1705981179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e529b0 msgr2=0x562f66e6a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.994+0000 7fc3895eb740 1 --2- 172.21.15.103:0/1705981179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e529b0 0x562f66e6a8c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc370097ee0 tx=0x7fc370098320 comp rx=0 tx=0).stop 2024-09-19T06:53:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.994+0000 7fc3895eb740 1 -- 172.21.15.103:0/1705981179 shutdown_connections 2024-09-19T06:53:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.994+0000 7fc3895eb740 1 --2- 172.21.15.103:0/1705981179 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562f66edd340 0x562f66edf760 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.994+0000 7fc3895eb740 1 --2- 172.21.15.103:0/1705981179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562f66e620c0 0x562f66e5c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.994+0000 7fc3895eb740 1 --2- 172.21.15.103:0/1705981179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f66e532d0 0x562f66e5bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.994+0000 7fc35f7fe640 0 client.16932 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.994+0000 7fc3895eb740 1 --2- 172.21.15.103:0/1705981179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562f66e529b0 0x562f66e6a8c0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.994+0000 7fc3895eb740 1 -- 172.21.15.103:0/1705981179 >> 172.21.15.103:0/1705981179 conn(0x562f66e4f060 msgr2=0x562f66e54c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.994+0000 7fc3895eb740 1 -- 172.21.15.103:0/1705981179 shutdown_connections 2024-09-19T06:53:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.994+0000 7fc3895eb740 1 -- 172.21.15.103:0/1705981179 wait complete. 2024-09-19T06:53:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.994+0000 7fc3895eb740 20 client.16932 trim_cache size 0 max 16384 2024-09-19T06:53:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:53:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:53:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:53:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:53:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.995+0000 7fc386853640 1 -- 172.21.15.103:0/929035723 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3680a3460 msgr2=0x7fc3680a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.995+0000 7fc386853640 1 --2- 172.21.15.103:0/929035723 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3680a3460 0x7fc3680a5920 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc378004af0 tx=0x7fc378005d80 comp rx=0 tx=0).stop 2024-09-19T06:53:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.995+0000 7fc386853640 1 -- 172.21.15.103:0/929035723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc380102ce0 msgr2=0x7fc3801982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.995+0000 7fc386853640 1 --2- 172.21.15.103:0/929035723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc380102ce0 0x7fc3801982c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc37000d950 tx=0x7fc37000de10 comp rx=0 tx=0).stop 2024-09-19T06:53:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.995+0000 7fc386853640 1 -- 172.21.15.103:0/929035723 shutdown_connections 2024-09-19T06:53:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.995+0000 7fc386853640 1 --2- 172.21.15.103:0/929035723 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3680a3460 0x7fc3680a5920 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.995+0000 7fc386853640 1 --2- 172.21.15.103:0/929035723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc38010c0b0 0x7fc38019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.995+0000 7fc386853640 1 --2- 172.21.15.103:0/929035723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc380103600 0x7fc380198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.995+0000 7fc386853640 1 --2- 172.21.15.103:0/929035723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc380102ce0 0x7fc3801982c0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.995+0000 7fc386853640 1 -- 172.21.15.103:0/929035723 >> 172.21.15.103:0/929035723 conn(0x7fc380077990 msgr2=0x7fc3800fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.995+0000 7fc386853640 1 -- 172.21.15.103:0/929035723 shutdown_connections 2024-09-19T06:53:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:34.995+0000 7fc386853640 1 -- 172.21.15.103:0/929035723 wait complete. 2024-09-19T06:53:35.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:34 smithi174 ceph-mon[35637]: pgmap v133: 97 pgs: 97 active+clean; 4.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-09-19T06:53:35.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.080+0000 7f67f5119640 1 -- 172.21.15.103:0/806315872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67f0102790 msgr2=0x7f67f01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.080+0000 7f67f5119640 1 --2- 172.21.15.103:0/806315872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67f0102790 0x7f67f01067e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009900 tx=0x7f67d802a410 comp rx=0 tx=0).stop 2024-09-19T06:53:35.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.080+0000 7f67f5119640 1 -- 172.21.15.103:0/806315872 shutdown_connections 2024-09-19T06:53:35.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.080+0000 7f67f5119640 1 --2- 172.21.15.103:0/806315872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67f0106f10 0x7f67f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.080+0000 7f67f5119640 1 --2- 172.21.15.103:0/806315872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67f0102790 0x7f67f01067e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.080+0000 7f67f5119640 1 --2- 172.21.15.103:0/806315872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67f0101de0 0x7f67f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.080+0000 7f67f5119640 1 -- 172.21.15.103:0/806315872 >> 172.21.15.103:0/806315872 conn(0x7f67f00fd8c0 msgr2=0x7f67f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:35.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.080+0000 7f67f5119640 1 -- 172.21.15.103:0/806315872 shutdown_connections 2024-09-19T06:53:35.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.080+0000 7f67f5119640 1 -- 172.21.15.103:0/806315872 wait complete. 2024-09-19T06:53:35.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.081+0000 7f67f5119640 1 Processor -- start 2024-09-19T06:53:35.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.081+0000 7f67f5119640 1 -- start start 2024-09-19T06:53:35.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.081+0000 7f67f5119640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67f0101de0 0x7f67f01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.081+0000 7f67f5119640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67f0102790 0x7f67f0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.081+0000 7f67f5119640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67f0106f10 0x7f67f019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.081+0000 7f67f5119640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f67f0112010 con 0x7f67f0106f10 2024-09-19T06:53:35.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.081+0000 7f67f5119640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67f0111e90 con 0x7f67f0102790 2024-09-19T06:53:35.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.081+0000 7f67f5119640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f67f0112190 con 0x7f67f0101de0 2024-09-19T06:53:35.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.081+0000 7f67ef59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67f0106f10 0x7f67f019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.081+0000 7f67ef59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67f0106f10 0x7f67f019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51906/0 (socket says 172.21.15.103:51906) 2024-09-19T06:53:35.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.081+0000 7f67ef59e640 1 -- 172.21.15.103:0/1594116902 learned_addr learned my addr 172.21.15.103:0/1594116902 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:35.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.081+0000 7f67eed9d640 1 --2- 172.21.15.103:0/1594116902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67f0101de0 0x7f67f01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.082+0000 7f67ee59c640 1 --2- 172.21.15.103:0/1594116902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67f0102790 0x7f67f0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.082+0000 7f67ef59e640 1 -- 172.21.15.103:0/1594116902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67f0101de0 msgr2=0x7f67f01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.082+0000 7f67ef59e640 1 --2- 172.21.15.103:0/1594116902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67f0101de0 0x7f67f01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.082+0000 7f67ef59e640 1 -- 172.21.15.103:0/1594116902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67f0102790 msgr2=0x7f67f0198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.082+0000 7f67ef59e640 1 --2- 172.21.15.103:0/1594116902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67f0102790 0x7f67f0198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.082+0000 7f67ef59e640 1 -- 172.21.15.103:0/1594116902 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f019d560 con 0x7f67f0106f10 2024-09-19T06:53:35.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.082+0000 7f67ee59c640 1 --2- 172.21.15.103:0/1594116902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67f0102790 0x7f67f0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:35.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.082+0000 7f67ef59e640 1 --2- 172.21.15.103:0/1594116902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67f0106f10 0x7f67f019ce80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f67e800b970 tx=0x7f67e800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:35.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.082+0000 7f67e7fff640 1 -- 172.21.15.103:0/1594116902 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8019070 con 0x7f67f0106f10 2024-09-19T06:53:35.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.082+0000 7f67f5119640 1 -- 172.21.15.103:0/1594116902 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f019d850 con 0x7f67f0106f10 2024-09-19T06:53:35.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.082+0000 7f67e7fff640 1 -- 172.21.15.103:0/1594116902 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e800cb40 con 0x7f67f0106f10 2024-09-19T06:53:35.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.082+0000 7f67e7fff640 1 -- 172.21.15.103:0/1594116902 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8007650 con 0x7f67f0106f10 2024-09-19T06:53:35.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.082+0000 7f67f5119640 1 -- 172.21.15.103:0/1594116902 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f01a5130 con 0x7f67f0106f10 2024-09-19T06:53:35.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.085+0000 7f67f74f2740 1 Processor -- start 2024-09-19T06:53:35.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.085+0000 7f67e7fff640 1 -- 172.21.15.103:0/1594116902 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f67e801c070 con 0x7f67f0106f10 2024-09-19T06:53:35.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.085+0000 7f67f74f2740 1 -- start start 2024-09-19T06:53:35.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67f74f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e4e760 0x55d7e8e4eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67f74f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e59400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67f74f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e59940 0x55d7e8e5bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67f74f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d7e8db42a0 con 0x55d7e8e4f080 2024-09-19T06:53:35.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67f74f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d7e8d5bb10 con 0x55d7e8e4e760 2024-09-19T06:53:35.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67f74f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d7e8db0260 con 0x55d7e8e59940 2024-09-19T06:53:35.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67eed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e59400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67eed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e59400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51914/0 (socket says 172.21.15.103:51914) 2024-09-19T06:53:35.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67eed9d640 1 -- 172.21.15.103:0/2241223976 learned_addr learned my addr 172.21.15.103:0/2241223976 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:35.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67ee59c640 1 --2- 172.21.15.103:0/2241223976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e4e760 0x55d7e8e4eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67ee59c640 1 --2- 172.21.15.103:0/2241223976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e59940 0x55d7e8e5bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67eed9d640 1 -- 172.21.15.103:0/2241223976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e59940 msgr2=0x55d7e8e5bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:35.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67eed9d640 1 --2- 172.21.15.103:0/2241223976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e59940 0x55d7e8e5bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67eed9d640 1 -- 172.21.15.103:0/2241223976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e4e760 msgr2=0x55d7e8e4eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:35.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67eed9d640 1 --2- 172.21.15.103:0/2241223976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e4e760 0x55d7e8e4eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67eed9d640 1 -- 172.21.15.103:0/2241223976 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7e8b25b90 con 0x55d7e8e4f080 2024-09-19T06:53:35.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67ee59c640 1 --2- 172.21.15.103:0/2241223976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e59940 0x55d7e8e5bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:35.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67e7fff640 1 --2- 172.21.15.103:0/1594116902 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f67dc0a33b0 0x7f67dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67eed9d640 1 --2- 172.21.15.103:0/2241223976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e59400 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f67e000b820 tx=0x7f67e000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980132fa0d7111ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:35.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67ee59c640 1 --2- 172.21.15.103:0/1594116902 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f67dc0a33b0 0x7f67dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.087+0000 7f67e7fff640 1 -- 172.21.15.103:0/1594116902 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f67e80cf2f0 con 0x7f67f0106f10 2024-09-19T06:53:35.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.086+0000 7f67e57fa640 1 -- 172.21.15.103:0/2241223976 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e000d950 con 0x55d7e8e4f080 2024-09-19T06:53:35.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.087+0000 7f67e57fa640 1 -- 172.21.15.103:0/2241223976 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e0019070 con 0x55d7e8e4f080 2024-09-19T06:53:35.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.087+0000 7f67e57fa640 1 -- 172.21.15.103:0/2241223976 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e00123e0 con 0x55d7e8e4f080 2024-09-19T06:53:35.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.087+0000 7f67f74f2740 1 -- 172.21.15.103:0/2241223976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 msgr2=0x55d7e8e59400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.087+0000 7f67ee59c640 1 --2- 172.21.15.103:0/1594116902 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f67dc0a33b0 0x7f67dc0a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f67f00fdd50 tx=0x7f67d802a410 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:35.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.087+0000 7f67f74f2740 1 --2- 172.21.15.103:0/2241223976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e59400 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f67e000b820 tx=0x7f67e000bce0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.087+0000 7f67f74f2740 1 -- 172.21.15.103:0/2241223976 shutdown_connections 2024-09-19T06:53:35.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.087+0000 7f67f74f2740 1 --2- 172.21.15.103:0/2241223976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e59940 0x55d7e8e5bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.087+0000 7f67f74f2740 1 --2- 172.21.15.103:0/2241223976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e59400 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.087+0000 7f67f74f2740 1 --2- 172.21.15.103:0/2241223976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e4e760 0x55d7e8e4eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.087+0000 7f67f74f2740 1 -- 172.21.15.103:0/2241223976 >> 172.21.15.103:0/2241223976 conn(0x55d7e8e4acf0 msgr2=0x55d7e8e4b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:35.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.087+0000 7f67f74f2740 1 -- 172.21.15.103:0/2241223976 shutdown_connections 2024-09-19T06:53:35.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.087+0000 7f67f74f2740 1 -- 172.21.15.103:0/2241223976 wait complete. 2024-09-19T06:53:35.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.088+0000 7f67f74f2740 1 -- start start 2024-09-19T06:53:35.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.088+0000 7f67f74f2740 1 Processor -- start 2024-09-19T06:53:35.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.088+0000 7f67f74f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4e760 0x55d7e8e56820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.088+0000 7f67f74f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e4f080 0x55d7e8e56d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.088+0000 7f67f74f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e59940 0x55d7e8e572a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.088+0000 7f67f74f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d7e8e5c510 con 0x55d7e8e4e760 2024-09-19T06:53:35.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.088+0000 7f67f74f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d7e8d3c6e0 con 0x55d7e8e4f080 2024-09-19T06:53:35.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.088+0000 7f67f74f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d7e8e5c690 con 0x55d7e8e59940 2024-09-19T06:53:35.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.088+0000 7f67f74f2740 10 client.0 _ll_register_callbacks cb 0x55d7e8d60a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:35.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.088+0000 7f67eed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4e760 0x55d7e8e56820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.088+0000 7f67eed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4e760 0x55d7e8e56820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51924/0 (socket says 172.21.15.103:51924) 2024-09-19T06:53:35.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.088+0000 7f67eed9d640 1 -- 172.21.15.103:0/2020052396 learned_addr learned my addr 172.21.15.103:0/2020052396 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:35.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.088+0000 7f67ee59c640 1 --2- 172.21.15.103:0/2020052396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e4f080 0x55d7e8e56d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.089+0000 7f67eed9d640 1 -- 172.21.15.103:0/2020052396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e59940 msgr2=0x55d7e8e572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:35.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.089+0000 7f67eed9d640 1 --2- 172.21.15.103:0/2020052396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e59940 0x55d7e8e572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.089+0000 7f67eed9d640 1 -- 172.21.15.103:0/2020052396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e4f080 msgr2=0x55d7e8e56d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:35.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.089+0000 7f67eed9d640 1 --2- 172.21.15.103:0/2020052396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e4f080 0x55d7e8e56d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.089+0000 7f67eed9d640 1 -- 172.21.15.103:0/2020052396 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7e8e65a00 con 0x55d7e8e4e760 2024-09-19T06:53:35.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.089+0000 7f67eed9d640 1 --2- 172.21.15.103:0/2020052396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4e760 0x55d7e8e56820 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f67e0006aa0 tx=0x7f67e0014660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88efcd46cb5ec1e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:35.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.089+0000 7f67d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:35.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.089+0000 7f67d67fc640 1 -- 172.21.15.103:0/2020052396 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0023070 con 0x55d7e8e4e760 2024-09-19T06:53:35.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.089+0000 7f67d67fc640 1 -- 172.21.15.103:0/2020052396 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f67e000dd60 con 0x55d7e8e4e760 2024-09-19T06:53:35.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.089+0000 7f67f74f2740 1 -- 172.21.15.103:0/2020052396 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67e0019070 con 0x55d7e8e4e760 2024-09-19T06:53:35.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.089+0000 7f67d67fc640 1 -- 172.21.15.103:0/2020052396 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f67e002f050 con 0x55d7e8e4e760 2024-09-19T06:53:35.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.089+0000 7f67f74f2740 10 client.16944 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:35.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.089+0000 7f67f74f2740 1 -- 172.21.15.103:0/2020052396 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d7e8e68300 con 0x55d7e8e4e760 2024-09-19T06:53:35.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.089+0000 7f67d67fc640 1 -- 172.21.15.103:0/2020052396 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e0019070 con 0x55d7e8e4e760 2024-09-19T06:53:35.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.090+0000 7f67d67fc640 1 -- 172.21.15.103:0/2020052396 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f67e0013070 con 0x55d7e8e4e760 2024-09-19T06:53:35.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.090+0000 7f67f74f2740 10 client.16944 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:35.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.090+0000 7f67f74f2740 10 client.16944 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:53:35.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.090+0000 7f67f74f2740 20 client.16944 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:35.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.090+0000 7f67f74f2740 1 --2- 172.21.15.103:0/2020052396 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d7e8e69650 0x55d7e8e6ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.090+0000 7f67f74f2740 4 client.16944 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:35.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.090+0000 7f67f74f2740 1 -- 172.21.15.103:0/2020052396 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7e8dc19d0 con 0x55d7e8e69650 2024-09-19T06:53:35.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.090+0000 7f67ee59c640 1 --2- 172.21.15.103:0/2020052396 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d7e8e69650 0x55d7e8e6ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.090+0000 7f67d67fc640 10 client.16944 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:35.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.090+0000 7f67ee59c640 1 --2- 172.21.15.103:0/2020052396 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d7e8e69650 0x55d7e8e6ba40 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7df32a1bd700091 server_cookie=2f3feeb65fa31805 in_seq=0 out_seq=0 2024-09-19T06:53:35.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.091+0000 7f67d67fc640 1 -- 172.21.15.103:0/2020052396 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7e8dc19d0 con 0x55d7e8e69650 2024-09-19T06:53:35.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.091+0000 7f67d67fc640 10 client.16944 handle_command_reply: tid=0 2024-09-19T06:53:35.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.091+0000 7f67f74f2740 1 client.16944 shutdown 2024-09-19T06:53:35.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.091+0000 7f67f74f2740 1 -- 172.21.15.103:0/2020052396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4e760 msgr2=0x55d7e8e56820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.091+0000 7f67f74f2740 1 --2- 172.21.15.103:0/2020052396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4e760 0x55d7e8e56820 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f67e0006aa0 tx=0x7f67e0014660 comp rx=0 tx=0).stop 2024-09-19T06:53:35.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.091+0000 7f67f74f2740 1 -- 172.21.15.103:0/2020052396 shutdown_connections 2024-09-19T06:53:35.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.091+0000 7f67f74f2740 1 --2- 172.21.15.103:0/2020052396 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d7e8e69650 0x55d7e8e6ba40 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.091+0000 7f67f74f2740 1 --2- 172.21.15.103:0/2020052396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e59940 0x55d7e8e572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.091+0000 7f67f74f2740 1 --2- 172.21.15.103:0/2020052396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e4f080 0x55d7e8e56d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.091+0000 7f67f74f2740 1 --2- 172.21.15.103:0/2020052396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4e760 0x55d7e8e56820 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.091+0000 7f67f74f2740 1 -- 172.21.15.103:0/2020052396 >> 172.21.15.103:0/2020052396 conn(0x55d7e8e4acf0 msgr2=0x55d7e8e49e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:35.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.091+0000 7f67d67fc640 0 client.16944 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:35.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.091+0000 7f67f74f2740 1 -- 172.21.15.103:0/2020052396 shutdown_connections 2024-09-19T06:53:35.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.091+0000 7f67f74f2740 1 -- 172.21.15.103:0/2020052396 wait complete. 2024-09-19T06:53:35.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.092+0000 7f67f74f2740 20 client.16944 trim_cache size 0 max 16384 2024-09-19T06:53:35.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.113+0000 7f67f74f2740 1 Processor -- start 2024-09-19T06:53:35.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.113+0000 7f67f74f2740 1 -- start start 2024-09-19T06:53:35.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67f74f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e4e760 0x55d7e8e55130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67f74f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67f74f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e59940 0x55d7e8e71610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67f74f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01ada70 con 0x55d7e8e4f080 2024-09-19T06:53:35.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67f74f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01af5b0 con 0x55d7e8e4e760 2024-09-19T06:53:35.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67f74f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01ae140 con 0x55d7e8e59940 2024-09-19T06:53:35.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67ee59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67ee59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51934/0 (socket says 172.21.15.103:51934) 2024-09-19T06:53:35.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67ee59c640 1 -- 172.21.15.103:0/2092455738 learned_addr learned my addr 172.21.15.103:0/2092455738 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:35.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67ef59e640 1 --2- 172.21.15.103:0/2092455738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e59940 0x55d7e8e71610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67eed9d640 1 --2- 172.21.15.103:0/2092455738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e4e760 0x55d7e8e55130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67ee59c640 1 -- 172.21.15.103:0/2092455738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e59940 msgr2=0x55d7e8e71610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:35.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67ee59c640 1 --2- 172.21.15.103:0/2092455738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e59940 0x55d7e8e71610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67ee59c640 1 -- 172.21.15.103:0/2092455738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e4e760 msgr2=0x55d7e8e55130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:35.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67ee59c640 1 --2- 172.21.15.103:0/2092455738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e4e760 0x55d7e8e55130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67ee59c640 1 -- 172.21.15.103:0/2092455738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7e8e46260 con 0x55d7e8e4f080 2024-09-19T06:53:35.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.114+0000 7f67eed9d640 1 --2- 172.21.15.103:0/2092455738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e4e760 0x55d7e8e55130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:35.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.115+0000 7f67ee59c640 1 --2- 172.21.15.103:0/2092455738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e710d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f67d802a930 tx=0x7f67d8042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662350c2188a4f17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:35.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.115+0000 7f67e4ff9640 1 -- 172.21.15.103:0/2092455738 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8035040 con 0x55d7e8e4f080 2024-09-19T06:53:35.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.115+0000 7f67e4ff9640 1 -- 172.21.15.103:0/2092455738 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d8005020 con 0x55d7e8e4f080 2024-09-19T06:53:35.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.115+0000 7f67e4ff9640 1 -- 172.21.15.103:0/2092455738 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8004b00 con 0x55d7e8e4f080 2024-09-19T06:53:35.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.116+0000 7f67f74f2740 1 -- 172.21.15.103:0/2092455738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 msgr2=0x55d7e8e710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.116+0000 7f67f74f2740 1 --2- 172.21.15.103:0/2092455738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e710d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f67d802a930 tx=0x7f67d8042000 comp rx=0 tx=0).stop 2024-09-19T06:53:35.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.116+0000 7f67f74f2740 1 -- 172.21.15.103:0/2092455738 shutdown_connections 2024-09-19T06:53:35.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.116+0000 7f67f74f2740 1 --2- 172.21.15.103:0/2092455738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e59940 0x55d7e8e71610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.116+0000 7f67f74f2740 1 --2- 172.21.15.103:0/2092455738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e710d0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.116+0000 7f67f74f2740 1 --2- 172.21.15.103:0/2092455738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e4e760 0x55d7e8e55130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.116+0000 7f67f74f2740 1 -- 172.21.15.103:0/2092455738 >> 172.21.15.103:0/2092455738 conn(0x55d7e8e4acf0 msgr2=0x55d7e8e588a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:35.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.116+0000 7f67f74f2740 1 -- 172.21.15.103:0/2092455738 shutdown_connections 2024-09-19T06:53:35.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.116+0000 7f67f74f2740 1 -- 172.21.15.103:0/2092455738 wait complete. 2024-09-19T06:53:35.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.116+0000 7f67f74f2740 1 -- start start 2024-09-19T06:53:35.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.116+0000 7f67f74f2740 1 Processor -- start 2024-09-19T06:53:35.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67f74f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e4e760 0x55d7e8e537c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67f74f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e53d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67f74f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e59940 0x55d7e8e54240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67f74f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01aeee0 con 0x55d7e8e4f080 2024-09-19T06:53:35.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67f74f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01ae810 con 0x55d7e8e59940 2024-09-19T06:53:35.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67f74f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01aba30 con 0x55d7e8e4e760 2024-09-19T06:53:35.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67ee59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e53d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67f74f2740 10 client.0 _ll_register_callbacks cb 0x7f67e0006d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67ee59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e53d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51938/0 (socket says 172.21.15.103:51938) 2024-09-19T06:53:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67ee59c640 1 -- 172.21.15.103:0/3819680820 learned_addr learned my addr 172.21.15.103:0/3819680820 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67eed9d640 1 --2- 172.21.15.103:0/3819680820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e4e760 0x55d7e8e537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67ef59e640 1 --2- 172.21.15.103:0/3819680820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e59940 0x55d7e8e54240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67ee59c640 1 -- 172.21.15.103:0/3819680820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e4e760 msgr2=0x55d7e8e537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67ee59c640 1 --2- 172.21.15.103:0/3819680820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e4e760 0x55d7e8e537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67ee59c640 1 -- 172.21.15.103:0/3819680820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e59940 msgr2=0x55d7e8e54240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67ee59c640 1 --2- 172.21.15.103:0/3819680820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e59940 0x55d7e8e54240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67ee59c640 1 -- 172.21.15.103:0/3819680820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7e8ed30a0 con 0x55d7e8e4f080 2024-09-19T06:53:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67eed9d640 1 --2- 172.21.15.103:0/3819680820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e4e760 0x55d7e8e537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.117+0000 7f67ef59e640 1 --2- 172.21.15.103:0/3819680820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e59940 0x55d7e8e54240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.118+0000 7f67ee59c640 1 --2- 172.21.15.103:0/3819680820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e53d00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f67d8032d50 tx=0x7f67d8032d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabc12d838060f69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.118+0000 7f67d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.118+0000 7f67d67fc640 1 -- 172.21.15.103:0/3819680820 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d805f020 con 0x55d7e8e4f080 2024-09-19T06:53:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.118+0000 7f67d67fc640 1 -- 172.21.15.103:0/3819680820 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f67d8005520 con 0x55d7e8e4f080 2024-09-19T06:53:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.118+0000 7f67f74f2740 1 -- 172.21.15.103:0/3819680820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d8005020 con 0x55d7e8e4f080 2024-09-19T06:53:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.118+0000 7f67d67fc640 1 -- 172.21.15.103:0/3819680820 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f67d8004880 con 0x55d7e8e4f080 2024-09-19T06:53:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.118+0000 7f67f74f2740 10 client.16950 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.118+0000 7f67f74f2740 1 -- 172.21.15.103:0/3819680820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d7e8ed4260 con 0x55d7e8e4f080 2024-09-19T06:53:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.118+0000 7f67d67fc640 1 -- 172.21.15.103:0/3819680820 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d8005020 con 0x55d7e8e4f080 2024-09-19T06:53:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.119+0000 7f67d67fc640 1 -- 172.21.15.103:0/3819680820 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f67d803a040 con 0x55d7e8e4f080 2024-09-19T06:53:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.119+0000 7f67f74f2740 10 client.16950 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.119+0000 7f67f74f2740 10 client.16950 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:53:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.119+0000 7f67f74f2740 20 client.16950 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.119+0000 7f67f74f2740 1 --2- 172.21.15.103:0/3819680820 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d7e8ed4990 0x55d7e8ed6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.119+0000 7f67f74f2740 4 client.16950 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-09-19T06:53:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.119+0000 7f67f74f2740 1 -- 172.21.15.103:0/3819680820 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x55d7e8dbbd20 con 0x55d7e8ed4990 2024-09-19T06:53:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.119+0000 7f67eed9d640 1 --2- 172.21.15.103:0/3819680820 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d7e8ed4990 0x55d7e8ed6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.119+0000 7f67d67fc640 10 client.16950 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:35.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.119+0000 7f67eed9d640 1 --2- 172.21.15.103:0/3819680820 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d7e8ed4990 0x55d7e8ed6db0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5e12ffcceaa247b server_cookie=1a74e3c9f65a932e in_seq=0 out_seq=0 2024-09-19T06:53:35.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.130+0000 7f67d67fc640 1 -- 172.21.15.103:0/3819680820 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d7e8dbbd20 con 0x55d7e8ed4990 2024-09-19T06:53:35.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.130+0000 7f67d67fc640 10 client.16950 handle_command_reply: tid=0 2024-09-19T06:53:35.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.130+0000 7f67f74f2740 1 client.16950 shutdown 2024-09-19T06:53:35.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.131+0000 7f67f74f2740 1 -- 172.21.15.103:0/3819680820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 msgr2=0x55d7e8e53d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.131+0000 7f67f74f2740 1 --2- 172.21.15.103:0/3819680820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e53d00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f67d8032d50 tx=0x7f67d8032d80 comp rx=0 tx=0).stop 2024-09-19T06:53:35.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.131+0000 7f67f74f2740 1 -- 172.21.15.103:0/3819680820 shutdown_connections 2024-09-19T06:53:35.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.131+0000 7f67f74f2740 1 --2- 172.21.15.103:0/3819680820 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d7e8ed4990 0x55d7e8ed6db0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.131+0000 7f67f74f2740 1 --2- 172.21.15.103:0/3819680820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e8e59940 0x55d7e8e54240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.131+0000 7f67f74f2740 1 --2- 172.21.15.103:0/3819680820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7e8e4f080 0x55d7e8e53d00 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.131+0000 7f67f74f2740 1 --2- 172.21.15.103:0/3819680820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e8e4e760 0x55d7e8e537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.131+0000 7f67f74f2740 1 -- 172.21.15.103:0/3819680820 >> 172.21.15.103:0/3819680820 conn(0x55d7e8e4acf0 msgr2=0x55d7e8e58080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:35.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.131+0000 7f67d67fc640 0 client.16950 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:35.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.131+0000 7f67f74f2740 1 -- 172.21.15.103:0/3819680820 shutdown_connections 2024-09-19T06:53:35.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.131+0000 7f67f74f2740 1 -- 172.21.15.103:0/3819680820 wait complete. 2024-09-19T06:53:35.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.131+0000 7f67f74f2740 20 client.16950 trim_cache size 0 max 16384 2024-09-19T06:53:35.133 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:53:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:53:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:53:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 18, 2024-09-19T06:53:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:53:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "3237e9a8": { 2024-09-19T06:53:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 18, 2024-09-19T06:53:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:53:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:53:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:53:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:53:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:53:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:53:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 149.8, 2024-09-19T06:53:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:53:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:53:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:53:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:53:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:53:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:53:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:53:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:53:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:53:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:53:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:53:35.135 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:53:35.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.132+0000 7f67f5119640 1 -- 172.21.15.103:0/1594116902 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f67dc0a33b0 msgr2=0x7f67dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.132+0000 7f67f5119640 1 --2- 172.21.15.103:0/1594116902 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f67dc0a33b0 0x7f67dc0a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f67f00fdd50 tx=0x7f67d802a410 comp rx=0 tx=0).stop 2024-09-19T06:53:35.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.132+0000 7f67f5119640 1 -- 172.21.15.103:0/1594116902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67f0106f10 msgr2=0x7f67f019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.132+0000 7f67f5119640 1 --2- 172.21.15.103:0/1594116902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67f0106f10 0x7f67f019ce80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f67e800b970 tx=0x7f67e800be30 comp rx=0 tx=0).stop 2024-09-19T06:53:35.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.132+0000 7f67f5119640 1 -- 172.21.15.103:0/1594116902 shutdown_connections 2024-09-19T06:53:35.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.132+0000 7f67f5119640 1 --2- 172.21.15.103:0/1594116902 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f67dc0a33b0 0x7f67dc0a5870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.132+0000 7f67f5119640 1 --2- 172.21.15.103:0/1594116902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67f0106f10 0x7f67f019ce80 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.132+0000 7f67f5119640 1 --2- 172.21.15.103:0/1594116902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67f0102790 0x7f67f0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.132+0000 7f67f5119640 1 --2- 172.21.15.103:0/1594116902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67f0101de0 0x7f67f01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.132+0000 7f67f5119640 1 -- 172.21.15.103:0/1594116902 >> 172.21.15.103:0/1594116902 conn(0x7f67f00fd8c0 msgr2=0x7f67f00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:35.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.132+0000 7f67f5119640 1 -- 172.21.15.103:0/1594116902 shutdown_connections 2024-09-19T06:53:35.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:35.132+0000 7f67f5119640 1 -- 172.21.15.103:0/1594116902 wait complete. 2024-09-19T06:53:35.140 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:35.949 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3237e9a8', quiesce took 3.2 seconds. Will release after: 26.6 2024-09-19T06:53:36.174 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:36 smithi103 ceph-mon[31158]: pgmap v134: 97 pgs: 97 active+clean; 4.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 21 op/s 2024-09-19T06:53:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:36 smithi174 ceph-mon[35637]: pgmap v134: 97 pgs: 97 active+clean; 4.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 21 op/s 2024-09-19T06:53:36.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.370+0000 7f9d5e2c9640 1 -- 172.21.15.103:0/3681981853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d58102d40 msgr2=0x7f9d58103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:36.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.370+0000 7f9d5e2c9640 1 --2- 172.21.15.103:0/3681981853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d58102d40 0x7f9d58103120 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009900 tx=0x7f9d4802a480 comp rx=0 tx=0).stop 2024-09-19T06:53:36.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.370+0000 7f9d5e2c9640 1 -- 172.21.15.103:0/3681981853 shutdown_connections 2024-09-19T06:53:36.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.370+0000 7f9d5e2c9640 1 --2- 172.21.15.103:0/3681981853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d5810c170 0x7f9d5810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.370+0000 7f9d5e2c9640 1 --2- 172.21.15.103:0/3681981853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d58103660 0x7f9d5810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.370+0000 7f9d5e2c9640 1 --2- 172.21.15.103:0/3681981853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d58102d40 0x7f9d58103120 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.370+0000 7f9d5e2c9640 1 -- 172.21.15.103:0/3681981853 >> 172.21.15.103:0/3681981853 conn(0x7f9d580fc7a0 msgr2=0x7f9d580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:36.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.370+0000 7f9d5e2c9640 1 -- 172.21.15.103:0/3681981853 shutdown_connections 2024-09-19T06:53:36.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.370+0000 7f9d5e2c9640 1 -- 172.21.15.103:0/3681981853 wait complete. 2024-09-19T06:53:36.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.371+0000 7f9d5e2c9640 1 Processor -- start 2024-09-19T06:53:36.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.371+0000 7f9d5e2c9640 1 -- start start 2024-09-19T06:53:36.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.371+0000 7f9d5e2c9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d58102d40 0x7f9d58198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.371+0000 7f9d5e2c9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d58103660 0x7f9d581989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.371+0000 7f9d5e2c9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d5810c170 0x7f9d5819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.371+0000 7f9d5e2c9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d58111f70 con 0x7f9d58103660 2024-09-19T06:53:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.371+0000 7f9d5e2c9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9d58111df0 con 0x7f9d5810c170 2024-09-19T06:53:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.371+0000 7f9d5e2c9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581120f0 con 0x7f9d58102d40 2024-09-19T06:53:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.371+0000 7f9d577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d58103660 0x7f9d581989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.371+0000 7f9d577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d58103660 0x7f9d581989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51970/0 (socket says 172.21.15.103:51970) 2024-09-19T06:53:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.371+0000 7f9d577fe640 1 -- 172.21.15.103:0/87342304 learned_addr learned my addr 172.21.15.103:0/87342304 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.371+0000 7f9d57fff640 1 --2- 172.21.15.103:0/87342304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d58102d40 0x7f9d58198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.372+0000 7f9d5c866640 1 --2- 172.21.15.103:0/87342304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d5810c170 0x7f9d5819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.372+0000 7f9d577fe640 1 -- 172.21.15.103:0/87342304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d58102d40 msgr2=0x7f9d58198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.372+0000 7f9d577fe640 1 --2- 172.21.15.103:0/87342304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d58102d40 0x7f9d58198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.372+0000 7f9d577fe640 1 -- 172.21.15.103:0/87342304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d5810c170 msgr2=0x7f9d5819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.372+0000 7f9d577fe640 1 --2- 172.21.15.103:0/87342304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d5810c170 0x7f9d5819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.372+0000 7f9d577fe640 1 -- 172.21.15.103:0/87342304 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5819d420 con 0x7f9d58103660 2024-09-19T06:53:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.372+0000 7f9d577fe640 1 --2- 172.21.15.103:0/87342304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d58103660 0x7f9d581989b0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000d750 tx=0x7f9d4000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.372+0000 7f9d557fa640 1 -- 172.21.15.103:0/87342304 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40017070 con 0x7f9d58103660 2024-09-19T06:53:36.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.372+0000 7f9d557fa640 1 -- 172.21.15.103:0/87342304 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d40005020 con 0x7f9d58103660 2024-09-19T06:53:36.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.372+0000 7f9d5e2c9640 1 -- 172.21.15.103:0/87342304 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5819d710 con 0x7f9d58103660 2024-09-19T06:53:36.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.372+0000 7f9d557fa640 1 -- 172.21.15.103:0/87342304 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40004d00 con 0x7f9d58103660 2024-09-19T06:53:36.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.372+0000 7f9d5e2c9640 1 -- 172.21.15.103:0/87342304 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d581a4ff0 con 0x7f9d58103660 2024-09-19T06:53:36.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.374+0000 7f9d557fa640 1 -- 172.21.15.103:0/87342304 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9d40009a30 con 0x7f9d58103660 2024-09-19T06:53:36.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.375+0000 7f9d606a2740 1 Processor -- start 2024-09-19T06:53:36.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.375+0000 7f9d606a2740 1 -- start start 2024-09-19T06:53:36.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.375+0000 7f9d606a2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 0x55fdc5ebc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.375+0000 7f9d606a2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ebcca0 0x55fdc5ecb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.375+0000 7f9d606a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5ecde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.375+0000 7f9d606a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fdc5e222e0 con 0x55fdc5ecba90 2024-09-19T06:53:36.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.375+0000 7f9d606a2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fdc5dc9ce0 con 0x55fdc5ebc380 2024-09-19T06:53:36.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.375+0000 7f9d606a2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fdc5e1e2a0 con 0x55fdc5ebcca0 2024-09-19T06:53:36.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.375+0000 7f9d557fa640 1 --2- 172.21.15.103:0/87342304 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9d440a33b0 0x7f9d440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.375+0000 7f9d577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5ecde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:36.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.375+0000 7f9d5c866640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 0x55fdc5ebc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:36.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.375+0000 7f9d577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5ecde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51980/0 (socket says 172.21.15.103:51980) 2024-09-19T06:53:36.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.375+0000 7f9d577fe640 1 -- 172.21.15.103:0/2146279987 learned_addr learned my addr 172.21.15.103:0/2146279987 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.375+0000 7f9d57fff640 1 --2- 172.21.15.103:0/2146279987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ebcca0 0x55fdc5ecb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.375+0000 7f9d5c866640 1 --2- 172.21.15.103:0/87342304 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9d440a33b0 0x7f9d440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.375+0000 7f9d557fa640 1 -- 172.21.15.103:0/87342304 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f9d400cf7c0 con 0x7f9d58103660 2024-09-19T06:53:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.376+0000 7f9d5c866640 1 -- 172.21.15.103:0/2146279987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ebcca0 msgr2=0x55fdc5ecb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.376+0000 7f9d5c866640 1 --2- 172.21.15.103:0/2146279987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ebcca0 0x55fdc5ecb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.376+0000 7f9d5c866640 1 -- 172.21.15.103:0/2146279987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 msgr2=0x55fdc5ecde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.376+0000 7f9d5c866640 1 --2- 172.21.15.103:0/2146279987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5ecde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.376+0000 7f9d5c866640 1 -- 172.21.15.103:0/2146279987 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdc5dae440 con 0x55fdc5ebc380 2024-09-19T06:53:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.376+0000 7f9d577fe640 1 --2- 172.21.15.103:0/2146279987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5ecde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.376+0000 7f9d5c866640 1 --2- 172.21.15.103:0/87342304 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9d440a33b0 0x7f9d440a5870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d50005c40 tx=0x7f9d50017000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.376+0000 7f9d5c866640 1 --2- 172.21.15.103:0/2146279987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 0x55fdc5ebc760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9d50012c40 tx=0x7f9d500107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=711348338d86f59e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.376+0000 7f9d4e7fc640 1 -- 172.21.15.103:0/2146279987 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50012df0 con 0x55fdc5ebc380 2024-09-19T06:53:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.376+0000 7f9d4e7fc640 1 -- 172.21.15.103:0/2146279987 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d50012750 con 0x55fdc5ebc380 2024-09-19T06:53:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.376+0000 7f9d4e7fc640 1 -- 172.21.15.103:0/2146279987 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50003e10 con 0x55fdc5ebc380 2024-09-19T06:53:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.377+0000 7f9d606a2740 1 -- 172.21.15.103:0/2146279987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 msgr2=0x55fdc5ebc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.377+0000 7f9d606a2740 1 --2- 172.21.15.103:0/2146279987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 0x55fdc5ebc760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9d50012c40 tx=0x7f9d500107a0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.377+0000 7f9d606a2740 1 -- 172.21.15.103:0/2146279987 shutdown_connections 2024-09-19T06:53:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.377+0000 7f9d606a2740 1 --2- 172.21.15.103:0/2146279987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5ecde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.377+0000 7f9d606a2740 1 --2- 172.21.15.103:0/2146279987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ebcca0 0x55fdc5ecb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.377+0000 7f9d606a2740 1 --2- 172.21.15.103:0/2146279987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 0x55fdc5ebc760 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.377+0000 7f9d606a2740 1 -- 172.21.15.103:0/2146279987 >> 172.21.15.103:0/2146279987 conn(0x55fdc5eb8a30 msgr2=0x55fdc5eb8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.377+0000 7f9d606a2740 1 -- 172.21.15.103:0/2146279987 shutdown_connections 2024-09-19T06:53:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.377+0000 7f9d606a2740 1 -- 172.21.15.103:0/2146279987 wait complete. 2024-09-19T06:53:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.377+0000 7f9d606a2740 1 -- start start 2024-09-19T06:53:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.377+0000 7f9d606a2740 1 Processor -- start 2024-09-19T06:53:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d606a2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 0x55fdc5ecb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d606a2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ebcca0 0x55fdc5ec86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d606a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5ec8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d606a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fdc5ece600 con 0x55fdc5ecba90 2024-09-19T06:53:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d606a2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fdc5ddd050 con 0x55fdc5ebc380 2024-09-19T06:53:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d606a2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fdc5ece780 con 0x55fdc5ebcca0 2024-09-19T06:53:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d606a2740 10 client.0 _ll_register_callbacks cb 0x55fdc5e1d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d5c866640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5ec8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d5c866640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5ec8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51988/0 (socket says 172.21.15.103:51988) 2024-09-19T06:53:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d5c866640 1 -- 172.21.15.103:0/1115429249 learned_addr learned my addr 172.21.15.103:0/1115429249 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d57fff640 1 --2- 172.21.15.103:0/1115429249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 0x55fdc5ecb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d577fe640 1 --2- 172.21.15.103:0/1115429249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ebcca0 0x55fdc5ec86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d5c866640 1 -- 172.21.15.103:0/1115429249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ebcca0 msgr2=0x55fdc5ec86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d5c866640 1 --2- 172.21.15.103:0/1115429249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ebcca0 0x55fdc5ec86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d5c866640 1 -- 172.21.15.103:0/1115429249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 msgr2=0x55fdc5ecb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d5c866640 1 --2- 172.21.15.103:0/1115429249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 0x55fdc5ecb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d5c866640 1 -- 172.21.15.103:0/1115429249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdc5b933c0 con 0x55fdc5ecba90 2024-09-19T06:53:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d57fff640 1 --2- 172.21.15.103:0/1115429249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 0x55fdc5ecb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d5c866640 1 --2- 172.21.15.103:0/1115429249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5ec8c50 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9d500039b0 tx=0x7f9d50012e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b494ac234712b32f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d377fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d377fe640 1 -- 172.21.15.103:0/1115429249 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50003d90 con 0x55fdc5ecba90 2024-09-19T06:53:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.378+0000 7f9d377fe640 1 -- 172.21.15.103:0/1115429249 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f9d50005180 con 0x55fdc5ecba90 2024-09-19T06:53:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.379+0000 7f9d606a2740 1 -- 172.21.15.103:0/1115429249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d50012750 con 0x55fdc5ecba90 2024-09-19T06:53:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.379+0000 7f9d377fe640 1 -- 172.21.15.103:0/1115429249 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9d50035050 con 0x55fdc5ecba90 2024-09-19T06:53:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.379+0000 7f9d606a2740 10 client.16962 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.379+0000 7f9d606a2740 1 -- 172.21.15.103:0/1115429249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fdc5eda080 con 0x55fdc5ecba90 2024-09-19T06:53:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.379+0000 7f9d377fe640 1 -- 172.21.15.103:0/1115429249 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d50012750 con 0x55fdc5ecba90 2024-09-19T06:53:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.379+0000 7f9d377fe640 1 -- 172.21.15.103:0/1115429249 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9d500276b0 con 0x55fdc5ecba90 2024-09-19T06:53:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.379+0000 7f9d606a2740 10 client.16962 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.379+0000 7f9d606a2740 10 client.16962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.379+0000 7f9d606a2740 20 client.16962 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.379+0000 7f9d606a2740 1 --2- 172.21.15.103:0/1115429249 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fdc5edb540 0x55fdc5edd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.379+0000 7f9d606a2740 4 client.16962 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.379+0000 7f9d606a2740 1 -- 172.21.15.103:0/1115429249 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdc5e2f850 con 0x55fdc5edb540 2024-09-19T06:53:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.379+0000 7f9d57fff640 1 --2- 172.21.15.103:0/1115429249 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fdc5edb540 0x55fdc5edd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.380+0000 7f9d57fff640 1 --2- 172.21.15.103:0/1115429249 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fdc5edb540 0x55fdc5edd930 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54613e41a6f8e095 server_cookie=700c4e02af7b8f66 in_seq=0 out_seq=0 2024-09-19T06:53:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.380+0000 7f9d377fe640 10 client.16962 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.380+0000 7f9d377fe640 1 -- 172.21.15.103:0/1115429249 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fdc5e2f850 con 0x55fdc5edb540 2024-09-19T06:53:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.380+0000 7f9d377fe640 10 client.16962 handle_command_reply: tid=0 2024-09-19T06:53:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.380+0000 7f9d606a2740 1 client.16962 shutdown 2024-09-19T06:53:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.381+0000 7f9d606a2740 1 -- 172.21.15.103:0/1115429249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 msgr2=0x55fdc5ec8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:36.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.381+0000 7f9d606a2740 1 --2- 172.21.15.103:0/1115429249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5ec8c50 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9d500039b0 tx=0x7f9d50012e10 comp rx=0 tx=0).stop 2024-09-19T06:53:36.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.381+0000 7f9d606a2740 1 -- 172.21.15.103:0/1115429249 shutdown_connections 2024-09-19T06:53:36.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.381+0000 7f9d606a2740 1 --2- 172.21.15.103:0/1115429249 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fdc5edb540 0x55fdc5edd930 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.381+0000 7f9d606a2740 1 --2- 172.21.15.103:0/1115429249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5ec8c50 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.381+0000 7f9d377fe640 0 client.16962 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:36.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.381+0000 7f9d606a2740 1 --2- 172.21.15.103:0/1115429249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ebcca0 0x55fdc5ec86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.381+0000 7f9d606a2740 1 --2- 172.21.15.103:0/1115429249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 0x55fdc5ecb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.381+0000 7f9d606a2740 1 -- 172.21.15.103:0/1115429249 >> 172.21.15.103:0/1115429249 conn(0x55fdc5eb8a30 msgr2=0x55fdc5ebe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:36.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.381+0000 7f9d606a2740 1 -- 172.21.15.103:0/1115429249 shutdown_connections 2024-09-19T06:53:36.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.381+0000 7f9d606a2740 1 -- 172.21.15.103:0/1115429249 wait complete. 2024-09-19T06:53:36.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.381+0000 7f9d606a2740 20 client.16962 trim_cache size 0 max 16384 2024-09-19T06:53:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.398+0000 7f9d606a2740 1 Processor -- start 2024-09-19T06:53:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.398+0000 7f9d606a2740 1 -- start start 2024-09-19T06:53:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.398+0000 7f9d606a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ebc380 0x55fdc5ed78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.398+0000 7f9d606a2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebcca0 0x55fdc5eca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.398+0000 7f9d606a2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ecba90 0x55fdc5ee2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.398+0000 7f9d606a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581ab5b0 con 0x55fdc5ebc380 2024-09-19T06:53:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.398+0000 7f9d606a2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581ad0f0 con 0x55fdc5ebcca0 2024-09-19T06:53:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.398+0000 7f9d606a2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581abc80 con 0x55fdc5ecba90 2024-09-19T06:53:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.398+0000 7f9d57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ebc380 0x55fdc5ed78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.398+0000 7f9d57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ebc380 0x55fdc5ed78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51990/0 (socket says 172.21.15.103:51990) 2024-09-19T06:53:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.398+0000 7f9d57fff640 1 -- 172.21.15.103:0/2557198285 learned_addr learned my addr 172.21.15.103:0/2557198285 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.398+0000 7f9d57fff640 1 -- 172.21.15.103:0/2557198285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ecba90 msgr2=0x55fdc5ee2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.398+0000 7f9d57fff640 1 --2- 172.21.15.103:0/2557198285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ecba90 0x55fdc5ee2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.398+0000 7f9d57fff640 1 -- 172.21.15.103:0/2557198285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebcca0 msgr2=0x55fdc5eca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.398+0000 7f9d57fff640 1 --2- 172.21.15.103:0/2557198285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebcca0 0x55fdc5eca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.398+0000 7f9d57fff640 1 -- 172.21.15.103:0/2557198285 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdc5eb84d0 con 0x55fdc5ebc380 2024-09-19T06:53:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.399+0000 7f9d57fff640 1 --2- 172.21.15.103:0/2557198285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ebc380 0x55fdc5ed78a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009230 tx=0x7f9d480058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50cea1e518bbd29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.399+0000 7f9d4dffb640 1 -- 172.21.15.103:0/2557198285 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4804a070 con 0x55fdc5ebc380 2024-09-19T06:53:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.399+0000 7f9d4dffb640 1 -- 172.21.15.103:0/2557198285 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d48005500 con 0x55fdc5ebc380 2024-09-19T06:53:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.399+0000 7f9d4dffb640 1 -- 172.21.15.103:0/2557198285 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48035040 con 0x55fdc5ebc380 2024-09-19T06:53:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.400+0000 7f9d606a2740 1 -- 172.21.15.103:0/2557198285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ebc380 msgr2=0x55fdc5ed78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.400+0000 7f9d606a2740 1 --2- 172.21.15.103:0/2557198285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ebc380 0x55fdc5ed78a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009230 tx=0x7f9d480058d0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.400+0000 7f9d606a2740 1 -- 172.21.15.103:0/2557198285 shutdown_connections 2024-09-19T06:53:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.400+0000 7f9d606a2740 1 --2- 172.21.15.103:0/2557198285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ecba90 0x55fdc5ee2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.400+0000 7f9d606a2740 1 --2- 172.21.15.103:0/2557198285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebcca0 0x55fdc5eca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.400+0000 7f9d606a2740 1 --2- 172.21.15.103:0/2557198285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ebc380 0x55fdc5ed78a0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.400+0000 7f9d606a2740 1 -- 172.21.15.103:0/2557198285 >> 172.21.15.103:0/2557198285 conn(0x55fdc5eb8a30 msgr2=0x55fdc5ec9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.400+0000 7f9d606a2740 1 -- 172.21.15.103:0/2557198285 shutdown_connections 2024-09-19T06:53:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.400+0000 7f9d606a2740 1 -- 172.21.15.103:0/2557198285 wait complete. 2024-09-19T06:53:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.400+0000 7f9d606a2740 1 -- start start 2024-09-19T06:53:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.400+0000 7f9d606a2740 1 Processor -- start 2024-09-19T06:53:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.400+0000 7f9d606a2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 0x55fdc5ecfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d606a2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ebcca0 0x55fdc5ed03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d606a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5f44f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d606a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581aca20 con 0x55fdc5ecba90 2024-09-19T06:53:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d606a2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581ac350 con 0x55fdc5ebc380 2024-09-19T06:53:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d606a2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581a9570 con 0x55fdc5ebcca0 2024-09-19T06:53:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d606a2740 10 client.0 _ll_register_callbacks cb 0x7f9d40097670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d5c866640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5f44f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d5c866640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5f44f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51994/0 (socket says 172.21.15.103:51994) 2024-09-19T06:53:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d5c866640 1 -- 172.21.15.103:0/1117506409 learned_addr learned my addr 172.21.15.103:0/1117506409 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d577fe640 1 --2- 172.21.15.103:0/1117506409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ebcca0 0x55fdc5ed03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d57fff640 1 --2- 172.21.15.103:0/1117506409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 0x55fdc5ecfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d5c866640 1 -- 172.21.15.103:0/1117506409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ebcca0 msgr2=0x55fdc5ed03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d5c866640 1 --2- 172.21.15.103:0/1117506409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ebcca0 0x55fdc5ed03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d5c866640 1 -- 172.21.15.103:0/1117506409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 msgr2=0x55fdc5ecfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d5c866640 1 --2- 172.21.15.103:0/1117506409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 0x55fdc5ecfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d5c866640 1 -- 172.21.15.103:0/1117506409 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdc5f45470 con 0x55fdc5ecba90 2024-09-19T06:53:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d5c866640 1 --2- 172.21.15.103:0/1117506409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5f44f30 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9d50002230 tx=0x7f9d50010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313753230989a6e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d377fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d377fe640 1 -- 172.21.15.103:0/1117506409 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5003d030 con 0x55fdc5ecba90 2024-09-19T06:53:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d377fe640 1 -- 172.21.15.103:0/1117506409 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f9d5000ddb0 con 0x55fdc5ecba90 2024-09-19T06:53:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d606a2740 1 -- 172.21.15.103:0/1117506409 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d48005500 con 0x55fdc5ecba90 2024-09-19T06:53:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.401+0000 7f9d377fe640 1 -- 172.21.15.103:0/1117506409 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9d50003a80 con 0x55fdc5ecba90 2024-09-19T06:53:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.402+0000 7f9d606a2740 10 client.16968 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.402+0000 7f9d606a2740 1 -- 172.21.15.103:0/1117506409 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fdc5f46720 con 0x55fdc5ecba90 2024-09-19T06:53:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.402+0000 7f9d377fe640 1 -- 172.21.15.103:0/1117506409 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d48005500 con 0x55fdc5ecba90 2024-09-19T06:53:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.402+0000 7f9d377fe640 1 -- 172.21.15.103:0/1117506409 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9d500053c0 con 0x55fdc5ecba90 2024-09-19T06:53:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.402+0000 7f9d606a2740 10 client.16968 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.402+0000 7f9d606a2740 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.402+0000 7f9d606a2740 20 client.16968 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.402+0000 7f9d606a2740 1 --2- 172.21.15.103:0/1117506409 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fdc5edb540 0x55fdc5f48d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.402+0000 7f9d606a2740 4 client.16968 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:53:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.402+0000 7f9d606a2740 1 -- 172.21.15.103:0/1117506409 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fdc5e29d60 con 0x55fdc5edb540 2024-09-19T06:53:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.402+0000 7f9d57fff640 1 --2- 172.21.15.103:0/1117506409 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fdc5edb540 0x55fdc5f48d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.402+0000 7f9d377fe640 10 client.16968 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.403+0000 7f9d57fff640 1 --2- 172.21.15.103:0/1117506409 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fdc5edb540 0x55fdc5f48d90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13322e5aba3a4ca6 server_cookie=b91681a204da29df in_seq=0 out_seq=0 2024-09-19T06:53:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.403+0000 7f9d377fe640 1 -- 172.21.15.103:0/1117506409 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fdc5e29d60 con 0x55fdc5edb540 2024-09-19T06:53:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.403+0000 7f9d377fe640 10 client.16968 handle_command_reply: tid=0 2024-09-19T06:53:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.403+0000 7f9d606a2740 1 client.16968 shutdown 2024-09-19T06:53:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.403+0000 7f9d606a2740 1 -- 172.21.15.103:0/1117506409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 msgr2=0x55fdc5f44f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.403+0000 7f9d606a2740 1 --2- 172.21.15.103:0/1117506409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5f44f30 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9d50002230 tx=0x7f9d50010d20 comp rx=0 tx=0).stop 2024-09-19T06:53:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.403+0000 7f9d606a2740 1 -- 172.21.15.103:0/1117506409 shutdown_connections 2024-09-19T06:53:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.403+0000 7f9d606a2740 1 --2- 172.21.15.103:0/1117506409 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fdc5edb540 0x55fdc5f48d90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.403+0000 7f9d377fe640 0 client.16968 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:36.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.403+0000 7f9d606a2740 1 --2- 172.21.15.103:0/1117506409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fdc5ecba90 0x55fdc5f44f30 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.403+0000 7f9d606a2740 1 --2- 172.21.15.103:0/1117506409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdc5ebcca0 0x55fdc5ed03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.403+0000 7f9d606a2740 1 --2- 172.21.15.103:0/1117506409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc5ebc380 0x55fdc5ecfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.403+0000 7f9d606a2740 1 -- 172.21.15.103:0/1117506409 >> 172.21.15.103:0/1117506409 conn(0x55fdc5eb8a30 msgr2=0x55fdc5eca750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:36.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.403+0000 7f9d606a2740 1 -- 172.21.15.103:0/1117506409 shutdown_connections 2024-09-19T06:53:36.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.403+0000 7f9d606a2740 1 -- 172.21.15.103:0/1117506409 wait complete. 2024-09-19T06:53:36.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.403+0000 7f9d606a2740 20 client.16968 trim_cache size 0 max 16384 2024-09-19T06:53:36.409 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:53:36.409 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T06:53:36.409 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T06:53:36.410 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:53:36.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.404+0000 7f9d5e2c9640 1 -- 172.21.15.103:0/87342304 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9d440a33b0 msgr2=0x7f9d440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:36.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.404+0000 7f9d5e2c9640 1 --2- 172.21.15.103:0/87342304 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9d440a33b0 0x7f9d440a5870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d50005c40 tx=0x7f9d50017000 comp rx=0 tx=0).stop 2024-09-19T06:53:36.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.404+0000 7f9d5e2c9640 1 -- 172.21.15.103:0/87342304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d58103660 msgr2=0x7f9d581989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:36.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.404+0000 7f9d5e2c9640 1 --2- 172.21.15.103:0/87342304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d58103660 0x7f9d581989b0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000d750 tx=0x7f9d4000dc10 comp rx=0 tx=0).stop 2024-09-19T06:53:36.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.404+0000 7f9d5e2c9640 1 -- 172.21.15.103:0/87342304 shutdown_connections 2024-09-19T06:53:36.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.404+0000 7f9d5e2c9640 1 --2- 172.21.15.103:0/87342304 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9d440a33b0 0x7f9d440a5870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.404+0000 7f9d5e2c9640 1 --2- 172.21.15.103:0/87342304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d5810c170 0x7f9d5819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.404+0000 7f9d5e2c9640 1 --2- 172.21.15.103:0/87342304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d58103660 0x7f9d581989b0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.404+0000 7f9d5e2c9640 1 --2- 172.21.15.103:0/87342304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d58102d40 0x7f9d58198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:36.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.404+0000 7f9d5e2c9640 1 -- 172.21.15.103:0/87342304 >> 172.21.15.103:0/87342304 conn(0x7f9d580fc7a0 msgr2=0x7f9d580fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:36.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.404+0000 7f9d5e2c9640 1 -- 172.21.15.103:0/87342304 shutdown_connections 2024-09-19T06:53:36.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:36.404+0000 7f9d5e2c9640 1 -- 172.21.15.103:0/87342304 wait complete. 2024-09-19T06:53:36.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:36 smithi134 ceph-mon[35356]: pgmap v134: 97 pgs: 97 active+clean; 4.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 21 op/s 2024-09-19T06:53:37.107 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:53:37.107 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:53:37.355 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:37.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.753+0000 7f7d75493640 1 -- 172.21.15.103:0/359456330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d70102790 msgr2=0x7f7d701067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:37.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.753+0000 7f7d75493640 1 --2- 172.21.15.103:0/359456330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d70102790 0x7f7d701067e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009900 tx=0x7f7d5802a410 comp rx=0 tx=0).stop 2024-09-19T06:53:37.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.753+0000 7f7d75493640 1 -- 172.21.15.103:0/359456330 shutdown_connections 2024-09-19T06:53:37.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.753+0000 7f7d75493640 1 --2- 172.21.15.103:0/359456330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d70106f10 0x7f7d7010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.753+0000 7f7d75493640 1 --2- 172.21.15.103:0/359456330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d70102790 0x7f7d701067e0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.753+0000 7f7d75493640 1 --2- 172.21.15.103:0/359456330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d70101de0 0x7f7d701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.753+0000 7f7d75493640 1 -- 172.21.15.103:0/359456330 >> 172.21.15.103:0/359456330 conn(0x7f7d700fd8c0 msgr2=0x7f7d700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:37.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.753+0000 7f7d75493640 1 -- 172.21.15.103:0/359456330 shutdown_connections 2024-09-19T06:53:37.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.753+0000 7f7d75493640 1 -- 172.21.15.103:0/359456330 wait complete. 2024-09-19T06:53:37.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.753+0000 7f7d75493640 1 Processor -- start 2024-09-19T06:53:37.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d75493640 1 -- start start 2024-09-19T06:53:37.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d75493640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d70101de0 0x7f7d7019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d75493640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d70102790 0x7f7d7019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d75493640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d70106f10 0x7f7d7019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d75493640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d70111f70 con 0x7f7d70102790 2024-09-19T06:53:37.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d75493640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d70111df0 con 0x7f7d70106f10 2024-09-19T06:53:37.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d75493640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d701120f0 con 0x7f7d70101de0 2024-09-19T06:53:37.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d6e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d70102790 0x7f7d7019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d6e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d70102790 0x7f7d7019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56700/0 (socket says 172.21.15.103:56700) 2024-09-19T06:53:37.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d6e7fc640 1 -- 172.21.15.103:0/3244981488 learned_addr learned my addr 172.21.15.103:0/3244981488 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:37.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d6effd640 1 --2- 172.21.15.103:0/3244981488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d70101de0 0x7f7d7019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d6f7fe640 1 --2- 172.21.15.103:0/3244981488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d70106f10 0x7f7d7019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d6e7fc640 1 -- 172.21.15.103:0/3244981488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d70101de0 msgr2=0x7f7d7019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:37.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d6e7fc640 1 --2- 172.21.15.103:0/3244981488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d70101de0 0x7f7d7019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d6e7fc640 1 -- 172.21.15.103:0/3244981488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d70106f10 msgr2=0x7f7d7019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:37.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d6e7fc640 1 --2- 172.21.15.103:0/3244981488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d70106f10 0x7f7d7019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d6e7fc640 1 -- 172.21.15.103:0/3244981488 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7019f600 con 0x7f7d70102790 2024-09-19T06:53:37.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d6effd640 1 --2- 172.21.15.103:0/3244981488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d70101de0 0x7f7d7019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:37.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.754+0000 7f7d6e7fc640 1 --2- 172.21.15.103:0/3244981488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d70102790 0x7f7d7019ab90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7d58006010 tx=0x7f7d580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:37.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.755+0000 7f7d67fff640 1 -- 172.21.15.103:0/3244981488 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5803f030 con 0x7f7d70102790 2024-09-19T06:53:37.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.755+0000 7f7d75493640 1 -- 172.21.15.103:0/3244981488 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7019f890 con 0x7f7d70102790 2024-09-19T06:53:37.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.755+0000 7f7d67fff640 1 -- 172.21.15.103:0/3244981488 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d58003ce0 con 0x7f7d70102790 2024-09-19T06:53:37.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.755+0000 7f7d67fff640 1 -- 172.21.15.103:0/3244981488 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58035070 con 0x7f7d70102790 2024-09-19T06:53:37.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.755+0000 7f7d75493640 1 -- 172.21.15.103:0/3244981488 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d701a71d0 con 0x7f7d70102790 2024-09-19T06:53:37.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.756+0000 7f7d75493640 1 -- 172.21.15.103:0/3244981488 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d701021c0 con 0x7f7d70102790 2024-09-19T06:53:37.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.757+0000 7f7d67fff640 1 -- 172.21.15.103:0/3244981488 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7d58042030 con 0x7f7d70102790 2024-09-19T06:53:37.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.758+0000 7f7d67fff640 1 --2- 172.21.15.103:0/3244981488 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d5c0a33b0 0x7f7d5c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.758+0000 7f7d6effd640 1 --2- 172.21.15.103:0/3244981488 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d5c0a33b0 0x7f7d5c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.758+0000 7f7d67fff640 1 -- 172.21.15.103:0/3244981488 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f7d580f0580 con 0x7f7d70102790 2024-09-19T06:53:37.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.758+0000 7f7d6effd640 1 --2- 172.21.15.103:0/3244981488 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d5c0a33b0 0x7f7d5c0a5870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009c20 tx=0x7f7d68005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:37.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.761+0000 7f7d67fff640 1 -- 172.21.15.103:0/3244981488 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7d580bd030 con 0x7f7d70102790 2024-09-19T06:53:37.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.961+0000 7f7d75493640 1 -- 172.21.15.103:0/3244981488 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d7019b9b0 con 0x7f7d70102790 2024-09-19T06:53:37.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.962+0000 7f7d67fff640 1 -- 172.21.15.103:0/3244981488 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f7d7019b9b0 con 0x7f7d70102790 2024-09-19T06:53:37.963 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:53:37.963 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:53:37.963 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:53:37.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.964+0000 7f7d75493640 1 -- 172.21.15.103:0/3244981488 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d5c0a33b0 msgr2=0x7f7d5c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:37.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.964+0000 7f7d75493640 1 --2- 172.21.15.103:0/3244981488 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d5c0a33b0 0x7f7d5c0a5870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009c20 tx=0x7f7d68005f90 comp rx=0 tx=0).stop 2024-09-19T06:53:37.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.964+0000 7f7d75493640 1 -- 172.21.15.103:0/3244981488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d70102790 msgr2=0x7f7d7019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:37.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.964+0000 7f7d75493640 1 --2- 172.21.15.103:0/3244981488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d70102790 0x7f7d7019ab90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7d58006010 tx=0x7f7d580042f0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.964+0000 7f7d75493640 1 -- 172.21.15.103:0/3244981488 shutdown_connections 2024-09-19T06:53:37.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.964+0000 7f7d75493640 1 --2- 172.21.15.103:0/3244981488 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d5c0a33b0 0x7f7d5c0a5870 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.964+0000 7f7d75493640 1 --2- 172.21.15.103:0/3244981488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d70106f10 0x7f7d7019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.964+0000 7f7d75493640 1 --2- 172.21.15.103:0/3244981488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d70102790 0x7f7d7019ab90 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.964+0000 7f7d75493640 1 --2- 172.21.15.103:0/3244981488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d70101de0 0x7f7d7019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.964+0000 7f7d75493640 1 -- 172.21.15.103:0/3244981488 >> 172.21.15.103:0/3244981488 conn(0x7f7d700fd8c0 msgr2=0x7f7d700fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:37.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.964+0000 7f7d75493640 1 -- 172.21.15.103:0/3244981488 shutdown_connections 2024-09-19T06:53:37.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:37.964+0000 7f7d75493640 1 -- 172.21.15.103:0/3244981488 wait complete. 2024-09-19T06:53:38.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:38 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3244981488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:53:38.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:38 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3244981488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:53:38.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:38 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3244981488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:53:38.701 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:53:38.952 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:39.326 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:39 smithi103 ceph-mon[31158]: pgmap v135: 97 pgs: 97 active+clean; 4.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-19T06:53:39.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:39 smithi174 ceph-mon[35637]: pgmap v135: 97 pgs: 97 active+clean; 4.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-19T06:53:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.407+0000 7f97149a1640 1 -- 172.21.15.103:0/1659821614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9710100c70 msgr2=0x7f9710104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.407+0000 7f97149a1640 1 --2- 172.21.15.103:0/1659821614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9710100c70 0x7f9710104dd0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9704009900 tx=0x7f970402a410 comp rx=0 tx=0).stop 2024-09-19T06:53:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.407+0000 7f97149a1640 1 -- 172.21.15.103:0/1659821614 shutdown_connections 2024-09-19T06:53:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.407+0000 7f97149a1640 1 --2- 172.21.15.103:0/1659821614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9710105500 0x7f971010d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.407+0000 7f97149a1640 1 --2- 172.21.15.103:0/1659821614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9710100c70 0x7f9710104dd0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.407+0000 7f97149a1640 1 --2- 172.21.15.103:0/1659821614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97101002c0 0x7f97101006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.407+0000 7f97149a1640 1 -- 172.21.15.103:0/1659821614 >> 172.21.15.103:0/1659821614 conn(0x7f9710077010 msgr2=0x7f97100fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.407+0000 7f97149a1640 1 -- 172.21.15.103:0/1659821614 shutdown_connections 2024-09-19T06:53:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.407+0000 7f97149a1640 1 -- 172.21.15.103:0/1659821614 wait complete. 2024-09-19T06:53:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.407+0000 7f97149a1640 1 Processor -- start 2024-09-19T06:53:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.407+0000 7f97149a1640 1 -- start start 2024-09-19T06:53:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f97149a1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97101002c0 0x7f97101982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f97149a1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9710100c70 0x7f9710198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f97149a1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9710105500 0x7f971019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f97149a1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9710111ea0 con 0x7f9710100c70 2024-09-19T06:53:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f97149a1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9710111d20 con 0x7f97101002c0 2024-09-19T06:53:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f97149a1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9710112020 con 0x7f9710105500 2024-09-19T06:53:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f970effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9710100c70 0x7f9710198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f970effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9710100c70 0x7f9710198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56716/0 (socket says 172.21.15.103:56716) 2024-09-19T06:53:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f970effd640 1 -- 172.21.15.103:0/3553491585 learned_addr learned my addr 172.21.15.103:0/3553491585 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f970f7fe640 1 --2- 172.21.15.103:0/3553491585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97101002c0 0x7f97101982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f970ffff640 1 --2- 172.21.15.103:0/3553491585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9710105500 0x7f971019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f970effd640 1 -- 172.21.15.103:0/3553491585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9710105500 msgr2=0x7f971019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f970effd640 1 --2- 172.21.15.103:0/3553491585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9710105500 0x7f971019cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f970effd640 1 -- 172.21.15.103:0/3553491585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97101002c0 msgr2=0x7f97101982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f970effd640 1 --2- 172.21.15.103:0/3553491585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97101002c0 0x7f97101982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f970effd640 1 -- 172.21.15.103:0/3553491585 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971019d270 con 0x7f9710100c70 2024-09-19T06:53:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f970effd640 1 --2- 172.21.15.103:0/3553491585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9710100c70 0x7f9710198800 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9704006010 tx=0x7f97040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.408+0000 7f970cff9640 1 -- 172.21.15.103:0/3553491585 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970403f030 con 0x7f9710100c70 2024-09-19T06:53:39.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.409+0000 7f970cff9640 1 -- 172.21.15.103:0/3553491585 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9704003ce0 con 0x7f9710100c70 2024-09-19T06:53:39.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.409+0000 7f970cff9640 1 -- 172.21.15.103:0/3553491585 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704035070 con 0x7f9710100c70 2024-09-19T06:53:39.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.409+0000 7f97149a1640 1 -- 172.21.15.103:0/3553491585 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971019d500 con 0x7f9710100c70 2024-09-19T06:53:39.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.409+0000 7f97149a1640 1 -- 172.21.15.103:0/3553491585 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97101a4e40 con 0x7f9710100c70 2024-09-19T06:53:39.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.411+0000 7f970cff9640 1 -- 172.21.15.103:0/3553491585 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9704042030 con 0x7f9710100c70 2024-09-19T06:53:39.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.411+0000 7f9717fdc740 1 Processor -- start 2024-09-19T06:53:39.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.411+0000 7f9717fdc740 1 -- start start 2024-09-19T06:53:39.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.411+0000 7f9717fdc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde2d20 0x56065bde1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.411+0000 7f9717fdc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bde17b0 0x56065bde1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.411+0000 7f9717fdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bdec550 0x56065bdee940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.411+0000 7f9717fdc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56065bd472f0 con 0x56065bde2d20 2024-09-19T06:53:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.411+0000 7f9717fdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56065bceecf0 con 0x56065bdec550 2024-09-19T06:53:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.411+0000 7f9717fdc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56065bd432b0 con 0x56065bde17b0 2024-09-19T06:53:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.411+0000 7f970ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde2d20 0x56065bde1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.411+0000 7f970ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde2d20 0x56065bde1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56726/0 (socket says 172.21.15.103:56726) 2024-09-19T06:53:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.411+0000 7f970ffff640 1 -- 172.21.15.103:0/2466287927 learned_addr learned my addr 172.21.15.103:0/2466287927 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.412+0000 7f970effd640 1 --2- 172.21.15.103:0/2466287927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bdec550 0x56065bdee940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.412+0000 7f970f7fe640 1 --2- 172.21.15.103:0/2466287927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bde17b0 0x56065bde1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.412+0000 7f970ffff640 1 -- 172.21.15.103:0/2466287927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bde17b0 msgr2=0x56065bde1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.412+0000 7f970ffff640 1 --2- 172.21.15.103:0/2466287927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bde17b0 0x56065bde1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.412+0000 7f970ffff640 1 -- 172.21.15.103:0/2466287927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bdec550 msgr2=0x56065bdee940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.412+0000 7f970ffff640 1 --2- 172.21.15.103:0/2466287927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bdec550 0x56065bdee940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.412+0000 7f970ffff640 1 -- 172.21.15.103:0/2466287927 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56065bcd33a0 con 0x56065bde2d20 2024-09-19T06:53:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.412+0000 7f970cff9640 1 --2- 172.21.15.103:0/3553491585 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96f80a3430 0x7f96f80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.412+0000 7f970f7fe640 1 --2- 172.21.15.103:0/2466287927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bde17b0 0x56065bde1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.412+0000 7f970f7fe640 1 --2- 172.21.15.103:0/3553491585 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96f80a3430 0x7f96f80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.412+0000 7f970ffff640 1 --2- 172.21.15.103:0/2466287927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde2d20 0x56065bde1270 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9708009ab0 tx=0x7f9708009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5caf3ffdf868df17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.412+0000 7f970cff9640 1 -- 172.21.15.103:0/3553491585 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f97040f02e0 con 0x7f9710100c70 2024-09-19T06:53:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.412+0000 7f96f5ffb640 1 -- 172.21.15.103:0/2466287927 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708016070 con 0x56065bde2d20 2024-09-19T06:53:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.412+0000 7f96f5ffb640 1 -- 172.21.15.103:0/2466287927 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970800ecb0 con 0x56065bde2d20 2024-09-19T06:53:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.412+0000 7f96f5ffb640 1 -- 172.21.15.103:0/2466287927 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708010700 con 0x56065bde2d20 2024-09-19T06:53:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.413+0000 7f9717fdc740 1 -- 172.21.15.103:0/2466287927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde2d20 msgr2=0x56065bde1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.413+0000 7f970f7fe640 1 --2- 172.21.15.103:0/3553491585 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96f80a3430 0x7f96f80a58f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x56065bddeca0 tx=0x7f97000075b0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.413+0000 7f9717fdc740 1 --2- 172.21.15.103:0/2466287927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde2d20 0x56065bde1270 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9708009ab0 tx=0x7f9708009f70 comp rx=0 tx=0).stop 2024-09-19T06:53:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.413+0000 7f9717fdc740 1 -- 172.21.15.103:0/2466287927 shutdown_connections 2024-09-19T06:53:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.413+0000 7f9717fdc740 1 --2- 172.21.15.103:0/2466287927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bdec550 0x56065bdee940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.413+0000 7f9717fdc740 1 --2- 172.21.15.103:0/2466287927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bde17b0 0x56065bde1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.413+0000 7f9717fdc740 1 --2- 172.21.15.103:0/2466287927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde2d20 0x56065bde1270 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.413+0000 7f9717fdc740 1 -- 172.21.15.103:0/2466287927 >> 172.21.15.103:0/2466287927 conn(0x56065bdaa310 msgr2=0x56065bddde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.413+0000 7f9717fdc740 1 -- 172.21.15.103:0/2466287927 shutdown_connections 2024-09-19T06:53:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.413+0000 7f9717fdc740 1 -- 172.21.15.103:0/2466287927 wait complete. 2024-09-19T06:53:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.413+0000 7f9717fdc740 1 -- start start 2024-09-19T06:53:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f9717fdc740 1 Processor -- start 2024-09-19T06:53:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f9717fdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bde17b0 0x56065bdebe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f9717fdc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bde2d20 0x56065bde9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f9717fdc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bdec550 0x56065bde9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f9717fdc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56065bdef250 con 0x56065bdec550 2024-09-19T06:53:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f9717fdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56065bd02060 con 0x56065bde17b0 2024-09-19T06:53:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f9717fdc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56065bdef3d0 con 0x56065bde2d20 2024-09-19T06:53:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f9717fdc740 10 client.0 _ll_register_callbacks cb 0x56065ba558c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f970ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bdec550 0x56065bde9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f970ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bdec550 0x56065bde9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56742/0 (socket says 172.21.15.103:56742) 2024-09-19T06:53:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f970ffff640 1 -- 172.21.15.103:0/848034957 learned_addr learned my addr 172.21.15.103:0/848034957 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f970f7fe640 1 --2- 172.21.15.103:0/848034957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bde17b0 0x56065bdebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f970effd640 1 --2- 172.21.15.103:0/848034957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bde2d20 0x56065bde9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f970ffff640 1 -- 172.21.15.103:0/848034957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bde2d20 msgr2=0x56065bde9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f970ffff640 1 --2- 172.21.15.103:0/848034957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bde2d20 0x56065bde9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f970ffff640 1 -- 172.21.15.103:0/848034957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bde17b0 msgr2=0x56065bdebe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f970ffff640 1 --2- 172.21.15.103:0/848034957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bde17b0 0x56065bdebe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f970ffff640 1 -- 172.21.15.103:0/848034957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56065bab83c0 con 0x56065bdec550 2024-09-19T06:53:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f970effd640 1 --2- 172.21.15.103:0/848034957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bde2d20 0x56065bde9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f970ffff640 1 --2- 172.21.15.103:0/848034957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bdec550 0x56065bde9680 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9708011f10 tx=0x7f9708011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b54ca0f49f4808 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.414+0000 7f96eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.415+0000 7f96eeffd640 1 -- 172.21.15.103:0/848034957 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708016050 con 0x56065bdec550 2024-09-19T06:53:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.415+0000 7f96eeffd640 1 -- 172.21.15.103:0/848034957 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f9708011680 con 0x56065bdec550 2024-09-19T06:53:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.415+0000 7f9717fdc740 1 -- 172.21.15.103:0/848034957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f970800ecb0 con 0x56065bdec550 2024-09-19T06:53:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.415+0000 7f96eeffd640 1 -- 172.21.15.103:0/848034957 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f97080096b0 con 0x56065bdec550 2024-09-19T06:53:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.415+0000 7f9717fdc740 10 client.16986 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.415+0000 7f9717fdc740 1 -- 172.21.15.103:0/848034957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56065bdfabb0 con 0x56065bdec550 2024-09-19T06:53:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.415+0000 7f96eeffd640 1 -- 172.21.15.103:0/848034957 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970800ecb0 con 0x56065bdec550 2024-09-19T06:53:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.415+0000 7f96eeffd640 1 -- 172.21.15.103:0/848034957 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f970802b030 con 0x56065bdec550 2024-09-19T06:53:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.415+0000 7f9717fdc740 10 client.16986 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.415+0000 7f9717fdc740 10 client.16986 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:53:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.415+0000 7f9717fdc740 20 client.16986 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.415+0000 7f9717fdc740 1 --2- 172.21.15.103:0/848034957 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x56065bdfbfb0 0x56065bdfe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.415+0000 7f9717fdc740 4 client.16986 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.415+0000 7f9717fdc740 1 -- 172.21.15.103:0/848034957 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56065bd54860 con 0x56065bdfbfb0 2024-09-19T06:53:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.416+0000 7f970f7fe640 1 --2- 172.21.15.103:0/848034957 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x56065bdfbfb0 0x56065bdfe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.416+0000 7f970f7fe640 1 --2- 172.21.15.103:0/848034957 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x56065bdfbfb0 0x56065bdfe3a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e097c95697ed3c4 server_cookie=8da067050517b343 in_seq=0 out_seq=0 2024-09-19T06:53:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.416+0000 7f96eeffd640 10 client.16986 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:53:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.417+0000 7f96eeffd640 1 -- 172.21.15.103:0/848034957 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56065bd54860 con 0x56065bdfbfb0 2024-09-19T06:53:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.417+0000 7f96eeffd640 10 client.16986 handle_command_reply: tid=0 2024-09-19T06:53:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.417+0000 7f9717fdc740 1 client.16986 shutdown 2024-09-19T06:53:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.417+0000 7f9717fdc740 1 -- 172.21.15.103:0/848034957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bdec550 msgr2=0x56065bde9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.417+0000 7f9717fdc740 1 --2- 172.21.15.103:0/848034957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bdec550 0x56065bde9680 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9708011f10 tx=0x7f9708011c90 comp rx=0 tx=0).stop 2024-09-19T06:53:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.417+0000 7f9717fdc740 1 -- 172.21.15.103:0/848034957 shutdown_connections 2024-09-19T06:53:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.417+0000 7f9717fdc740 1 --2- 172.21.15.103:0/848034957 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x56065bdfbfb0 0x56065bdfe3a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.417+0000 7f96eeffd640 0 client.16986 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:53:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.417+0000 7f9717fdc740 1 --2- 172.21.15.103:0/848034957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bdec550 0x56065bde9680 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.417+0000 7f9717fdc740 1 --2- 172.21.15.103:0/848034957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bde2d20 0x56065bde9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.417+0000 7f9717fdc740 1 --2- 172.21.15.103:0/848034957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bde17b0 0x56065bdebe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.417+0000 7f9717fdc740 1 -- 172.21.15.103:0/848034957 >> 172.21.15.103:0/848034957 conn(0x56065bdaa310 msgr2=0x56065bde3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.417+0000 7f9717fdc740 1 -- 172.21.15.103:0/848034957 shutdown_connections 2024-09-19T06:53:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.417+0000 7f9717fdc740 1 -- 172.21.15.103:0/848034957 wait complete. 2024-09-19T06:53:39.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.417+0000 7f9717fdc740 20 client.16986 trim_cache size 0 max 16384 2024-09-19T06:53:39.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f9717fdc740 1 Processor -- start 2024-09-19T06:53:39.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f9717fdc740 1 -- start start 2024-09-19T06:53:39.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f9717fdc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde17b0 0x56065bdf84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f9717fdc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bde2d20 0x56065bde8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f9717fdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bdec550 0x56065be039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f9717fdc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f97101ad600 con 0x56065bde17b0 2024-09-19T06:53:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f9717fdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f97101af140 con 0x56065bdec550 2024-09-19T06:53:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f9717fdc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97101adcd0 con 0x56065bde2d20 2024-09-19T06:53:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f970f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde17b0 0x56065bdf84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f970f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde17b0 0x56065bdf84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56752/0 (socket says 172.21.15.103:56752) 2024-09-19T06:53:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f970f7fe640 1 -- 172.21.15.103:0/3889299890 learned_addr learned my addr 172.21.15.103:0/3889299890 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f970f7fe640 1 -- 172.21.15.103:0/3889299890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bde2d20 msgr2=0x56065bde8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f970f7fe640 1 --2- 172.21.15.103:0/3889299890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bde2d20 0x56065bde8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f970f7fe640 1 -- 172.21.15.103:0/3889299890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bdec550 msgr2=0x56065be039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f970f7fe640 1 --2- 172.21.15.103:0/3889299890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bdec550 0x56065be039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f970f7fe640 1 -- 172.21.15.103:0/3889299890 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56065bde7690 con 0x56065bde17b0 2024-09-19T06:53:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.430+0000 7f970f7fe640 1 --2- 172.21.15.103:0/3889299890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde17b0 0x56065bdf84f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9700007270 tx=0x7f9700013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a217e4db95072d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.431+0000 7f96f57fa640 1 -- 172.21.15.103:0/3889299890 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700020110 con 0x56065bde17b0 2024-09-19T06:53:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.431+0000 7f96f57fa640 1 -- 172.21.15.103:0/3889299890 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9700020940 con 0x56065bde17b0 2024-09-19T06:53:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.431+0000 7f96f57fa640 1 -- 172.21.15.103:0/3889299890 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97000054e0 con 0x56065bde17b0 2024-09-19T06:53:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.431+0000 7f9717fdc740 1 -- 172.21.15.103:0/3889299890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde17b0 msgr2=0x56065bdf84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.431+0000 7f9717fdc740 1 --2- 172.21.15.103:0/3889299890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde17b0 0x56065bdf84f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9700007270 tx=0x7f9700013000 comp rx=0 tx=0).stop 2024-09-19T06:53:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.431+0000 7f9717fdc740 1 -- 172.21.15.103:0/3889299890 shutdown_connections 2024-09-19T06:53:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.431+0000 7f9717fdc740 1 --2- 172.21.15.103:0/3889299890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bdec550 0x56065be039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.431+0000 7f9717fdc740 1 --2- 172.21.15.103:0/3889299890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bde2d20 0x56065bde8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.431+0000 7f9717fdc740 1 --2- 172.21.15.103:0/3889299890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde17b0 0x56065bdf84f0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.431+0000 7f9717fdc740 1 -- 172.21.15.103:0/3889299890 >> 172.21.15.103:0/3889299890 conn(0x56065bdaa310 msgr2=0x56065bde9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.431+0000 7f9717fdc740 1 -- 172.21.15.103:0/3889299890 shutdown_connections 2024-09-19T06:53:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.431+0000 7f9717fdc740 1 -- 172.21.15.103:0/3889299890 wait complete. 2024-09-19T06:53:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.432+0000 7f9717fdc740 1 -- start start 2024-09-19T06:53:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.432+0000 7f9717fdc740 1 Processor -- start 2024-09-19T06:53:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.432+0000 7f9717fdc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde17b0 0x56065bdefda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.432+0000 7f9717fdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bde2d20 0x56065bdf02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.432+0000 7f9717fdc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bdec550 0x56065be5e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.432+0000 7f9717fdc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f97101aea70 con 0x56065bde17b0 2024-09-19T06:53:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.432+0000 7f9717fdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f97101ae3a0 con 0x56065bde2d20 2024-09-19T06:53:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.432+0000 7f9717fdc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97101ab5c0 con 0x56065bdec550 2024-09-19T06:53:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.432+0000 7f9717fdc740 10 client.0 _ll_register_callbacks cb 0x7f97040f3c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.432+0000 7f970f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde17b0 0x56065bdefda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.432+0000 7f970f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde17b0 0x56065bdefda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56768/0 (socket says 172.21.15.103:56768) 2024-09-19T06:53:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.432+0000 7f970f7fe640 1 -- 172.21.15.103:0/3165682962 learned_addr learned my addr 172.21.15.103:0/3165682962 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.432+0000 7f970ffff640 1 --2- 172.21.15.103:0/3165682962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bdec550 0x56065be5e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f970effd640 1 --2- 172.21.15.103:0/3165682962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bde2d20 0x56065bdf02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f970f7fe640 1 -- 172.21.15.103:0/3165682962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bdec550 msgr2=0x56065be5e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f970f7fe640 1 --2- 172.21.15.103:0/3165682962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bdec550 0x56065be5e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f970f7fe640 1 -- 172.21.15.103:0/3165682962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bde2d20 msgr2=0x56065bdf02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f970f7fe640 1 --2- 172.21.15.103:0/3165682962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bde2d20 0x56065bdf02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f970f7fe640 1 -- 172.21.15.103:0/3165682962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56065be5efa0 con 0x56065bde17b0 2024-09-19T06:53:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f970effd640 1 --2- 172.21.15.103:0/3165682962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bde2d20 0x56065bdf02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f970f7fe640 1 --2- 172.21.15.103:0/3165682962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde17b0 0x56065bdefda0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9700011860 tx=0x7f9700020340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4c09d24b0554eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f96eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f96eeffd640 1 -- 172.21.15.103:0/3165682962 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700035020 con 0x56065bde17b0 2024-09-19T06:53:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f9717fdc740 1 -- 172.21.15.103:0/3165682962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9700020940 con 0x56065bde17b0 2024-09-19T06:53:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f96eeffd640 1 -- 172.21.15.103:0/3165682962 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f970002e040 con 0x56065bde17b0 2024-09-19T06:53:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f96eeffd640 1 -- 172.21.15.103:0/3165682962 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9700020e00 con 0x56065bde17b0 2024-09-19T06:53:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f9717fdc740 10 client.16992 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f9717fdc740 1 -- 172.21.15.103:0/3165682962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56065be66300 con 0x56065bde17b0 2024-09-19T06:53:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f96eeffd640 1 -- 172.21.15.103:0/3165682962 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9700020940 con 0x56065bde17b0 2024-09-19T06:53:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.433+0000 7f96eeffd640 1 -- 172.21.15.103:0/3165682962 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f970001e420 con 0x56065bde17b0 2024-09-19T06:53:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.434+0000 7f9717fdc740 10 client.16992 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.434+0000 7f9717fdc740 10 client.16992 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:53:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.434+0000 7f9717fdc740 20 client.16992 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.434+0000 7f9717fdc740 1 --2- 172.21.15.103:0/3165682962 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x56065bdfbfb0 0x56065be68a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.434+0000 7f9717fdc740 4 client.16992 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:53:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.434+0000 7f9717fdc740 1 -- 172.21.15.103:0/3165682962 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56065bd4ed70 con 0x56065bdfbfb0 2024-09-19T06:53:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.434+0000 7f970effd640 1 --2- 172.21.15.103:0/3165682962 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x56065bdfbfb0 0x56065be68a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.434+0000 7f970effd640 1 --2- 172.21.15.103:0/3165682962 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x56065bdfbfb0 0x56065be68a80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=159d06b599d42f7 server_cookie=d4481407bf194c8d in_seq=0 out_seq=0 2024-09-19T06:53:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.434+0000 7f96eeffd640 10 client.16992 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:53:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.434+0000 7f96eeffd640 1 -- 172.21.15.103:0/3165682962 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56065bd4ed70 con 0x56065bdfbfb0 2024-09-19T06:53:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.434+0000 7f96eeffd640 10 client.16992 handle_command_reply: tid=0 2024-09-19T06:53:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.434+0000 7f9717fdc740 1 client.16992 shutdown 2024-09-19T06:53:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.435+0000 7f9717fdc740 1 -- 172.21.15.103:0/3165682962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde17b0 msgr2=0x56065bdefda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.435+0000 7f9717fdc740 1 --2- 172.21.15.103:0/3165682962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde17b0 0x56065bdefda0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9700011860 tx=0x7f9700020340 comp rx=0 tx=0).stop 2024-09-19T06:53:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.435+0000 7f9717fdc740 1 -- 172.21.15.103:0/3165682962 shutdown_connections 2024-09-19T06:53:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.435+0000 7f9717fdc740 1 --2- 172.21.15.103:0/3165682962 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x56065bdfbfb0 0x56065be68a80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.435+0000 7f9717fdc740 1 --2- 172.21.15.103:0/3165682962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56065bdec550 0x56065be5e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.435+0000 7f9717fdc740 1 --2- 172.21.15.103:0/3165682962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56065bde2d20 0x56065bdf02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.435+0000 7f9717fdc740 1 --2- 172.21.15.103:0/3165682962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56065bde17b0 0x56065bdefda0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.435+0000 7f9717fdc740 1 -- 172.21.15.103:0/3165682962 >> 172.21.15.103:0/3165682962 conn(0x56065bdaa310 msgr2=0x56065bdeb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.435+0000 7f96eeffd640 0 client.16992 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:53:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.435+0000 7f9717fdc740 1 -- 172.21.15.103:0/3165682962 shutdown_connections 2024-09-19T06:53:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.435+0000 7f9717fdc740 1 -- 172.21.15.103:0/3165682962 wait complete. 2024-09-19T06:53:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.435+0000 7f9717fdc740 20 client.16992 trim_cache size 0 max 16384 2024-09-19T06:53:39.445 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:53:39.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:39 smithi134 ceph-mon[35356]: pgmap v135: 97 pgs: 97 active+clean; 4.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-19T06:53:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.435+0000 7f97149a1640 1 -- 172.21.15.103:0/3553491585 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96f80a3430 msgr2=0x7f96f80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.435+0000 7f97149a1640 1 --2- 172.21.15.103:0/3553491585 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96f80a3430 0x7f96f80a58f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x56065bddeca0 tx=0x7f97000075b0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.435+0000 7f97149a1640 1 -- 172.21.15.103:0/3553491585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9710100c70 msgr2=0x7f9710198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.435+0000 7f97149a1640 1 --2- 172.21.15.103:0/3553491585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9710100c70 0x7f9710198800 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9704006010 tx=0x7f97040042f0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.436+0000 7f97149a1640 1 -- 172.21.15.103:0/3553491585 shutdown_connections 2024-09-19T06:53:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.436+0000 7f97149a1640 1 --2- 172.21.15.103:0/3553491585 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96f80a3430 0x7f96f80a58f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.436+0000 7f97149a1640 1 --2- 172.21.15.103:0/3553491585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9710105500 0x7f971019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.436+0000 7f97149a1640 1 --2- 172.21.15.103:0/3553491585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9710100c70 0x7f9710198800 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.436+0000 7f97149a1640 1 --2- 172.21.15.103:0/3553491585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97101002c0 0x7f97101982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.436+0000 7f97149a1640 1 -- 172.21.15.103:0/3553491585 >> 172.21.15.103:0/3553491585 conn(0x7f9710077010 msgr2=0x7f9710103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.436+0000 7f97149a1640 1 -- 172.21.15.103:0/3553491585 shutdown_connections 2024-09-19T06:53:39.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:39.436+0000 7f97149a1640 1 -- 172.21.15.103:0/3553491585 wait complete. 2024-09-19T06:53:40.227 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:53:40.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:40 smithi174 ceph-mon[35637]: pgmap v136: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-09-19T06:53:40.340 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:40 smithi134 ceph-mon[35356]: pgmap v136: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-09-19T06:53:40.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:40 smithi103 ceph-mon[31158]: pgmap v136: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-09-19T06:53:40.475 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:40.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.893+0000 7f62b3b5d640 1 -- 172.21.15.103:0/4017103034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62ac101de0 msgr2=0x7f62ac1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:40.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.893+0000 7f62b3b5d640 1 --2- 172.21.15.103:0/4017103034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62ac101de0 0x7f62ac1021c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009900 tx=0x7f62a802a410 comp rx=0 tx=0).stop 2024-09-19T06:53:40.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.893+0000 7f62b3b5d640 1 -- 172.21.15.103:0/4017103034 shutdown_connections 2024-09-19T06:53:40.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.893+0000 7f62b3b5d640 1 --2- 172.21.15.103:0/4017103034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ac106f10 0x7f62ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.893+0000 7f62b3b5d640 1 --2- 172.21.15.103:0/4017103034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac102790 0x7f62ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.893+0000 7f62b3b5d640 1 --2- 172.21.15.103:0/4017103034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62ac101de0 0x7f62ac1021c0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.893+0000 7f62b3b5d640 1 -- 172.21.15.103:0/4017103034 >> 172.21.15.103:0/4017103034 conn(0x7f62ac0fd8c0 msgr2=0x7f62ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:40.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.893+0000 7f62b3b5d640 1 -- 172.21.15.103:0/4017103034 shutdown_connections 2024-09-19T06:53:40.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.893+0000 7f62b3b5d640 1 -- 172.21.15.103:0/4017103034 wait complete. 2024-09-19T06:53:40.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.894+0000 7f62b3b5d640 1 Processor -- start 2024-09-19T06:53:40.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.894+0000 7f62b3b5d640 1 -- start start 2024-09-19T06:53:40.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b3b5d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac101de0 0x7f62ac198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b3b5d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62ac102790 0x7f62ac198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b3b5d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ac106f10 0x7f62ac19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b3b5d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac111f70 con 0x7f62ac102790 2024-09-19T06:53:40.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b3b5d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac111df0 con 0x7f62ac101de0 2024-09-19T06:53:40.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b3b5d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac1120f0 con 0x7f62ac106f10 2024-09-19T06:53:40.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b10f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62ac102790 0x7f62ac198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b10f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62ac102790 0x7f62ac198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56796/0 (socket says 172.21.15.103:56796) 2024-09-19T06:53:40.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b10f8640 1 -- 172.21.15.103:0/4261803661 learned_addr learned my addr 172.21.15.103:0/4261803661 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:40.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b20fa640 1 --2- 172.21.15.103:0/4261803661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ac106f10 0x7f62ac19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b18f9640 1 --2- 172.21.15.103:0/4261803661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac101de0 0x7f62ac198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b10f8640 1 -- 172.21.15.103:0/4261803661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ac106f10 msgr2=0x7f62ac19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:40.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b10f8640 1 --2- 172.21.15.103:0/4261803661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ac106f10 0x7f62ac19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b10f8640 1 -- 172.21.15.103:0/4261803661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac101de0 msgr2=0x7f62ac198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:40.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b10f8640 1 --2- 172.21.15.103:0/4261803661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac101de0 0x7f62ac198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b10f8640 1 -- 172.21.15.103:0/4261803661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ac19d3e0 con 0x7f62ac102790 2024-09-19T06:53:40.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b10f8640 1 --2- 172.21.15.103:0/4261803661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62ac102790 0x7f62ac198970 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c890 tx=0x7f629c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:40.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62a2ffd640 1 -- 172.21.15.103:0/4261803661 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c004220 con 0x7f62ac102790 2024-09-19T06:53:40.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62a2ffd640 1 -- 172.21.15.103:0/4261803661 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f629c0043c0 con 0x7f62ac102790 2024-09-19T06:53:40.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b3b5d640 1 -- 172.21.15.103:0/4261803661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ac19d6d0 con 0x7f62ac102790 2024-09-19T06:53:40.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62a2ffd640 1 -- 172.21.15.103:0/4261803661 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c0049e0 con 0x7f62ac102790 2024-09-19T06:53:40.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.895+0000 7f62b3b5d640 1 -- 172.21.15.103:0/4261803661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ac1a4fb0 con 0x7f62ac102790 2024-09-19T06:53:40.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.898+0000 7f62a2ffd640 1 -- 172.21.15.103:0/4261803661 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f629c0079b0 con 0x7f62ac102790 2024-09-19T06:53:40.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.898+0000 7f62b5f36740 1 Processor -- start 2024-09-19T06:53:40.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.899+0000 7f62b5f36740 1 -- start start 2024-09-19T06:53:40.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.899+0000 7f62b5f36740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1b4b40 0x560b7f1b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.899+0000 7f62b5f36740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b35d0 0x560b7f1b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.899+0000 7f62b5f36740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.899+0000 7f62b5f36740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560b7f1192f0 con 0x560b7f1c2780 2024-09-19T06:53:40.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.899+0000 7f62b5f36740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560b7f0c0cf0 con 0x560b7f1b35d0 2024-09-19T06:53:40.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.899+0000 7f62b5f36740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560b7f1152b0 con 0x560b7f1b4b40 2024-09-19T06:53:40.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.899+0000 7f62b10f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.899+0000 7f62b10f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56806/0 (socket says 172.21.15.103:56806) 2024-09-19T06:53:40.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.899+0000 7f62b10f8640 1 -- 172.21.15.103:0/1437597071 learned_addr learned my addr 172.21.15.103:0/1437597071 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:40.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.899+0000 7f62b20fa640 1 --2- 172.21.15.103:0/1437597071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1b4b40 0x560b7f1b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.899+0000 7f62b18f9640 1 --2- 172.21.15.103:0/1437597071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b35d0 0x560b7f1b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.899+0000 7f62a2ffd640 1 --2- 172.21.15.103:0/4261803661 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62980a33b0 0x7f62980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.899+0000 7f62b20fa640 1 --2- 172.21.15.103:0/4261803661 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62980a33b0 0x7f62980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.900+0000 7f62b20fa640 1 -- 172.21.15.103:0/1437597071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b35d0 msgr2=0x560b7f1b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:40.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.900+0000 7f62b20fa640 1 --2- 172.21.15.103:0/1437597071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b35d0 0x560b7f1b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.900+0000 7f62b20fa640 1 -- 172.21.15.103:0/1437597071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 msgr2=0x560b7f1c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:40.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.900+0000 7f62b20fa640 1 --2- 172.21.15.103:0/1437597071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1c4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.900+0000 7f62b20fa640 1 -- 172.21.15.103:0/1437597071 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b7f0a53a0 con 0x560b7f1b4b40 2024-09-19T06:53:40.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.900+0000 7f62a2ffd640 1 -- 172.21.15.103:0/4261803661 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f629c0cfca0 con 0x7f62ac102790 2024-09-19T06:53:40.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.900+0000 7f62b20fa640 1 --2- 172.21.15.103:0/1437597071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1b4b40 0x560b7f1b3090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f62a40149a0 tx=0x7f62a4014e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ace996fa83ca9277 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:40.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.900+0000 7f62b20fa640 1 --2- 172.21.15.103:0/4261803661 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62980a33b0 0x7f62980a5870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f62a401c000 tx=0x7f62a401e000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:40.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.900+0000 7f6297fff640 1 -- 172.21.15.103:0/1437597071 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4023070 con 0x560b7f1b4b40 2024-09-19T06:53:40.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.900+0000 7f6297fff640 1 -- 172.21.15.103:0/1437597071 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a4014030 con 0x560b7f1b4b40 2024-09-19T06:53:40.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.900+0000 7f6297fff640 1 -- 172.21.15.103:0/1437597071 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4013900 con 0x560b7f1b4b40 2024-09-19T06:53:40.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.901+0000 7f62b5f36740 1 -- 172.21.15.103:0/1437597071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1b4b40 msgr2=0x560b7f1b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:40.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.901+0000 7f62b5f36740 1 --2- 172.21.15.103:0/1437597071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1b4b40 0x560b7f1b3090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f62a40149a0 tx=0x7f62a4014e60 comp rx=0 tx=0).stop 2024-09-19T06:53:40.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.901+0000 7f62b5f36740 1 -- 172.21.15.103:0/1437597071 shutdown_connections 2024-09-19T06:53:40.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.901+0000 7f62b5f36740 1 --2- 172.21.15.103:0/1437597071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.901+0000 7f62b5f36740 1 --2- 172.21.15.103:0/1437597071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b35d0 0x560b7f1b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.901+0000 7f62b5f36740 1 --2- 172.21.15.103:0/1437597071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1b4b40 0x560b7f1b3090 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.901+0000 7f62b5f36740 1 -- 172.21.15.103:0/1437597071 >> 172.21.15.103:0/1437597071 conn(0x560b7f17c310 msgr2=0x560b7f1afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:40.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.901+0000 7f62b5f36740 1 -- 172.21.15.103:0/1437597071 shutdown_connections 2024-09-19T06:53:40.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.901+0000 7f62b5f36740 1 -- 172.21.15.103:0/1437597071 wait complete. 2024-09-19T06:53:40.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.901+0000 7f62b5f36740 1 -- start start 2024-09-19T06:53:40.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.901+0000 7f62b5f36740 1 Processor -- start 2024-09-19T06:53:40.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b5f36740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b35d0 0x560b7f1c2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b5f36740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1b4b40 0x560b7f1bf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b5f36740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1c2780 0x560b7f1bfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b5f36740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560b7f1c5480 con 0x560b7f1b4b40 2024-09-19T06:53:40.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b5f36740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560b7f0d4060 con 0x560b7f1b35d0 2024-09-19T06:53:40.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b5f36740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560b7f1c5600 con 0x560b7f1c2780 2024-09-19T06:53:40.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b5f36740 10 client.0 _ll_register_callbacks cb 0x560b7ee278c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:40.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b10f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1b4b40 0x560b7f1bf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b10f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1b4b40 0x560b7f1bf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56816/0 (socket says 172.21.15.103:56816) 2024-09-19T06:53:40.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b10f8640 1 -- 172.21.15.103:0/2714388712 learned_addr learned my addr 172.21.15.103:0/2714388712 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:40.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b18f9640 1 --2- 172.21.15.103:0/2714388712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b35d0 0x560b7f1c2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b20fa640 1 --2- 172.21.15.103:0/2714388712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1c2780 0x560b7f1bfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b10f8640 1 -- 172.21.15.103:0/2714388712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1c2780 msgr2=0x560b7f1bfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:40.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b10f8640 1 --2- 172.21.15.103:0/2714388712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1c2780 0x560b7f1bfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b10f8640 1 -- 172.21.15.103:0/2714388712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b35d0 msgr2=0x560b7f1c2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:40.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b10f8640 1 --2- 172.21.15.103:0/2714388712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b35d0 0x560b7f1c2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b10f8640 1 -- 172.21.15.103:0/2714388712 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b7ee8a3c0 con 0x560b7f1b4b40 2024-09-19T06:53:40.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62b10f8640 1 --2- 172.21.15.103:0/2714388712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1b4b40 0x560b7f1bf450 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f629c005fd0 tx=0x7f629c0d32c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf26c050359fec19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:40.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.902+0000 7f62957fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:40.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.903+0000 7f62957fa640 1 -- 172.21.15.103:0/2714388712 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c01d040 con 0x560b7f1b4b40 2024-09-19T06:53:40.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.903+0000 7f62957fa640 1 -- 172.21.15.103:0/2714388712 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f629c0babd0 con 0x560b7f1b4b40 2024-09-19T06:53:40.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.903+0000 7f62b5f36740 1 -- 172.21.15.103:0/2714388712 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62a4014030 con 0x560b7f1b4b40 2024-09-19T06:53:40.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.903+0000 7f62957fa640 1 -- 172.21.15.103:0/2714388712 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f629c09bb80 con 0x560b7f1b4b40 2024-09-19T06:53:40.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.903+0000 7f62b5f36740 10 client.17004 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:40.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.903+0000 7f62b5f36740 1 -- 172.21.15.103:0/2714388712 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560b7f1d0f30 con 0x560b7f1b4b40 2024-09-19T06:53:40.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.903+0000 7f62957fa640 1 -- 172.21.15.103:0/2714388712 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a4014030 con 0x560b7f1b4b40 2024-09-19T06:53:40.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.903+0000 7f62957fa640 1 -- 172.21.15.103:0/2714388712 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f629c0a4770 con 0x560b7f1b4b40 2024-09-19T06:53:40.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.903+0000 7f62b5f36740 10 client.17004 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:40.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.903+0000 7f62b5f36740 10 client.17004 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:53:40.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.903+0000 7f62b5f36740 20 client.17004 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:40.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.903+0000 7f62b5f36740 1 --2- 172.21.15.103:0/2714388712 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x560b7f1d2340 0x560b7f1d4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.903+0000 7f62b5f36740 4 client.17004 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:40.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.903+0000 7f62b5f36740 1 -- 172.21.15.103:0/2714388712 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b7f126860 con 0x560b7f1d2340 2024-09-19T06:53:40.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.903+0000 7f62b18f9640 1 --2- 172.21.15.103:0/2714388712 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x560b7f1d2340 0x560b7f1d4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.904+0000 7f62b18f9640 1 --2- 172.21.15.103:0/2714388712 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x560b7f1d2340 0x560b7f1d4750 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c9212261182ef1d6 server_cookie=44e89b8772d498f3 in_seq=0 out_seq=0 2024-09-19T06:53:40.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.904+0000 7f62957fa640 10 client.17004 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:53:40.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.904+0000 7f62957fa640 1 -- 172.21.15.103:0/2714388712 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b7f126860 con 0x560b7f1d2340 2024-09-19T06:53:40.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.904+0000 7f62957fa640 10 client.17004 handle_command_reply: tid=0 2024-09-19T06:53:40.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.904+0000 7f62b5f36740 1 client.17004 shutdown 2024-09-19T06:53:40.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.905+0000 7f62b5f36740 1 -- 172.21.15.103:0/2714388712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1b4b40 msgr2=0x560b7f1bf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:40.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.905+0000 7f62b5f36740 1 --2- 172.21.15.103:0/2714388712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1b4b40 0x560b7f1bf450 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f629c005fd0 tx=0x7f629c0d32c0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.905+0000 7f62b5f36740 1 -- 172.21.15.103:0/2714388712 shutdown_connections 2024-09-19T06:53:40.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.905+0000 7f62b5f36740 1 --2- 172.21.15.103:0/2714388712 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x560b7f1d2340 0x560b7f1d4750 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.905+0000 7f62b5f36740 1 --2- 172.21.15.103:0/2714388712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1c2780 0x560b7f1bfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.905+0000 7f62b5f36740 1 --2- 172.21.15.103:0/2714388712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1b4b40 0x560b7f1bf450 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.905+0000 7f62957fa640 0 client.17004 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:53:40.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.905+0000 7f62b5f36740 1 --2- 172.21.15.103:0/2714388712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b35d0 0x560b7f1c2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.905+0000 7f62b5f36740 1 -- 172.21.15.103:0/2714388712 >> 172.21.15.103:0/2714388712 conn(0x560b7f17c310 msgr2=0x560b7f1b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:40.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.905+0000 7f62b5f36740 1 -- 172.21.15.103:0/2714388712 shutdown_connections 2024-09-19T06:53:40.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.905+0000 7f62b5f36740 1 -- 172.21.15.103:0/2714388712 wait complete. 2024-09-19T06:53:40.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.905+0000 7f62b5f36740 20 client.17004 trim_cache size 0 max 16384 2024-09-19T06:53:40.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b5f36740 1 Processor -- start 2024-09-19T06:53:40.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b5f36740 1 -- start start 2024-09-19T06:53:40.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b5f36740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1b35d0 0x560b7f1ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b5f36740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b4b40 0x560b7f1aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b5f36740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1d9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b5f36740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac1ab730 con 0x560b7f1c2780 2024-09-19T06:53:40.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b5f36740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac1ad270 con 0x560b7f1b4b40 2024-09-19T06:53:40.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b5f36740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac1abe00 con 0x560b7f1b35d0 2024-09-19T06:53:40.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b20fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1d9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b20fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1d9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56820/0 (socket says 172.21.15.103:56820) 2024-09-19T06:53:40.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b20fa640 1 -- 172.21.15.103:0/1535121705 learned_addr learned my addr 172.21.15.103:0/1535121705 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:40.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b10f8640 1 --2- 172.21.15.103:0/1535121705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b4b40 0x560b7f1aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b18f9640 1 --2- 172.21.15.103:0/1535121705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1b35d0 0x560b7f1ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b20fa640 1 -- 172.21.15.103:0/1535121705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1b35d0 msgr2=0x560b7f1ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:40.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b20fa640 1 --2- 172.21.15.103:0/1535121705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1b35d0 0x560b7f1ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b20fa640 1 -- 172.21.15.103:0/1535121705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b4b40 msgr2=0x560b7f1aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:40.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b20fa640 1 --2- 172.21.15.103:0/1535121705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b4b40 0x560b7f1aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b20fa640 1 -- 172.21.15.103:0/1535121705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b7f1bda00 con 0x560b7f1c2780 2024-09-19T06:53:40.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.918+0000 7f62b20fa640 1 --2- 172.21.15.103:0/1535121705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1d9ff0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f62a40149a0 tx=0x7f62a40093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be432daf5fd74a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:40.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.919+0000 7f62977fe640 1 -- 172.21.15.103:0/1535121705 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4023030 con 0x560b7f1c2780 2024-09-19T06:53:40.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.919+0000 7f62977fe640 1 -- 172.21.15.103:0/1535121705 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a4003e50 con 0x560b7f1c2780 2024-09-19T06:53:40.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.919+0000 7f62977fe640 1 -- 172.21.15.103:0/1535121705 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a40052a0 con 0x560b7f1c2780 2024-09-19T06:53:40.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.919+0000 7f62b5f36740 1 -- 172.21.15.103:0/1535121705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 msgr2=0x560b7f1d9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:40.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.919+0000 7f62b5f36740 1 --2- 172.21.15.103:0/1535121705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1d9ff0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f62a40149a0 tx=0x7f62a40093e0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.919+0000 7f62b5f36740 1 -- 172.21.15.103:0/1535121705 shutdown_connections 2024-09-19T06:53:40.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.919+0000 7f62b5f36740 1 --2- 172.21.15.103:0/1535121705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1d9ff0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.919+0000 7f62b5f36740 1 --2- 172.21.15.103:0/1535121705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b4b40 0x560b7f1aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.919+0000 7f62b5f36740 1 --2- 172.21.15.103:0/1535121705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1b35d0 0x560b7f1ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.919+0000 7f62b5f36740 1 -- 172.21.15.103:0/1535121705 >> 172.21.15.103:0/1535121705 conn(0x560b7f17c310 msgr2=0x560b7f1ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:40.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.919+0000 7f62b5f36740 1 -- 172.21.15.103:0/1535121705 shutdown_connections 2024-09-19T06:53:40.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.919+0000 7f62b5f36740 1 -- 172.21.15.103:0/1535121705 wait complete. 2024-09-19T06:53:40.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.920+0000 7f62b5f36740 1 -- start start 2024-09-19T06:53:40.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.920+0000 7f62b5f36740 1 Processor -- start 2024-09-19T06:53:40.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.920+0000 7f62b5f36740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b35d0 0x560b7f1ca860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.920+0000 7f62b5f36740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1b4b40 0x560b7f1cada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.920+0000 7f62b5f36740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.920+0000 7f62b5f36740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac1acba0 con 0x560b7f1c2780 2024-09-19T06:53:40.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.920+0000 7f62b5f36740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac1ac4d0 con 0x560b7f1b35d0 2024-09-19T06:53:40.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.920+0000 7f62b5f36740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac1a96f0 con 0x560b7f1b4b40 2024-09-19T06:53:40.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.920+0000 7f62b5f36740 10 client.0 _ll_register_callbacks cb 0x7f629c0057c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:40.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.920+0000 7f62b20fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.920+0000 7f62b20fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56834/0 (socket says 172.21.15.103:56834) 2024-09-19T06:53:40.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.920+0000 7f62b20fa640 1 -- 172.21.15.103:0/4074675758 learned_addr learned my addr 172.21.15.103:0/4074675758 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:40.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.920+0000 7f62b18f9640 1 --2- 172.21.15.103:0/4074675758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b35d0 0x560b7f1ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.921+0000 7f62b10f8640 1 --2- 172.21.15.103:0/4074675758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1b4b40 0x560b7f1cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.921+0000 7f62b20fa640 1 -- 172.21.15.103:0/4074675758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1b4b40 msgr2=0x560b7f1cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:40.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.921+0000 7f62b20fa640 1 --2- 172.21.15.103:0/4074675758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1b4b40 0x560b7f1cada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.921+0000 7f62b20fa640 1 -- 172.21.15.103:0/4074675758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b35d0 msgr2=0x560b7f1ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:40.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.921+0000 7f62b20fa640 1 --2- 172.21.15.103:0/4074675758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b35d0 0x560b7f1ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.921+0000 7f62b20fa640 1 -- 172.21.15.103:0/4074675758 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b7f1c7d70 con 0x560b7f1c2780 2024-09-19T06:53:40.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.921+0000 7f62b20fa640 1 --2- 172.21.15.103:0/4074675758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1c7830 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f62a4014ea0 tx=0x7f62a4012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85b58b068cdb4ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:40.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.921+0000 7f62957fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:40.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.921+0000 7f62957fa640 1 -- 172.21.15.103:0/4074675758 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4023030 con 0x560b7f1c2780 2024-09-19T06:53:40.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.921+0000 7f62957fa640 1 -- 172.21.15.103:0/4074675758 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f62a4038070 con 0x560b7f1c2780 2024-09-19T06:53:40.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.921+0000 7f62b5f36740 1 -- 172.21.15.103:0/4074675758 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62a4003e50 con 0x560b7f1c2780 2024-09-19T06:53:40.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.921+0000 7f62957fa640 1 -- 172.21.15.103:0/4074675758 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f62a4009a00 con 0x560b7f1c2780 2024-09-19T06:53:40.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.921+0000 7f62b5f36740 10 client.17010 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:40.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.921+0000 7f62b5f36740 1 -- 172.21.15.103:0/4074675758 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560b7f23d7f0 con 0x560b7f1c2780 2024-09-19T06:53:40.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.921+0000 7f62957fa640 1 -- 172.21.15.103:0/4074675758 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a4003e50 con 0x560b7f1c2780 2024-09-19T06:53:40.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.922+0000 7f62957fa640 1 -- 172.21.15.103:0/4074675758 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f62a400f040 con 0x560b7f1c2780 2024-09-19T06:53:40.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.922+0000 7f62b5f36740 10 client.17010 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:40.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.922+0000 7f62b5f36740 10 client.17010 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:53:40.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.922+0000 7f62b5f36740 20 client.17010 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:40.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.922+0000 7f62b5f36740 1 --2- 172.21.15.103:0/4074675758 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x560b7f23dd90 0x560b7f2401b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:40.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.922+0000 7f62b5f36740 4 client.17010 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:53:40.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.922+0000 7f62b5f36740 1 -- 172.21.15.103:0/4074675758 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b7f120d70 con 0x560b7f23dd90 2024-09-19T06:53:40.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.922+0000 7f62b18f9640 1 --2- 172.21.15.103:0/4074675758 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x560b7f23dd90 0x560b7f2401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:40.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.922+0000 7f62b18f9640 1 --2- 172.21.15.103:0/4074675758 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x560b7f23dd90 0x560b7f2401b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fb9bc0836e8cc4bc server_cookie=f076e6435a36e2ce in_seq=0 out_seq=0 2024-09-19T06:53:40.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.922+0000 7f62957fa640 10 client.17010 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:53:40.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.922+0000 7f62957fa640 1 -- 172.21.15.103:0/4074675758 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b7f120d70 con 0x560b7f23dd90 2024-09-19T06:53:40.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.922+0000 7f62957fa640 10 client.17010 handle_command_reply: tid=0 2024-09-19T06:53:40.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.922+0000 7f62b5f36740 1 client.17010 shutdown 2024-09-19T06:53:40.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b5f36740 1 -- 172.21.15.103:0/4074675758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 msgr2=0x560b7f1c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:40.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b5f36740 1 --2- 172.21.15.103:0/4074675758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1c7830 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f62a4014ea0 tx=0x7f62a4012c90 comp rx=0 tx=0).stop 2024-09-19T06:53:40.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b5f36740 1 -- 172.21.15.103:0/4074675758 shutdown_connections 2024-09-19T06:53:40.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b5f36740 1 --2- 172.21.15.103:0/4074675758 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x560b7f23dd90 0x560b7f2401b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b5f36740 1 --2- 172.21.15.103:0/4074675758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b7f1c2780 0x560b7f1c7830 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b5f36740 1 --2- 172.21.15.103:0/4074675758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b7f1b4b40 0x560b7f1cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b5f36740 1 --2- 172.21.15.103:0/4074675758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b7f1b35d0 0x560b7f1ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62957fa640 0 client.17010 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:53:40.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b5f36740 1 -- 172.21.15.103:0/4074675758 >> 172.21.15.103:0/4074675758 conn(0x560b7f17c310 msgr2=0x560b7f1c14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:40.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b5f36740 1 -- 172.21.15.103:0/4074675758 shutdown_connections 2024-09-19T06:53:40.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b5f36740 1 -- 172.21.15.103:0/4074675758 wait complete. 2024-09-19T06:53:40.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b5f36740 20 client.17010 trim_cache size 0 max 16384 2024-09-19T06:53:40.932 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:53:40.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b3b5d640 1 -- 172.21.15.103:0/4261803661 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62980a33b0 msgr2=0x7f62980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:40.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b3b5d640 1 --2- 172.21.15.103:0/4261803661 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62980a33b0 0x7f62980a5870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f62a401c000 tx=0x7f62a401e000 comp rx=0 tx=0).stop 2024-09-19T06:53:40.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b3b5d640 1 -- 172.21.15.103:0/4261803661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62ac102790 msgr2=0x7f62ac198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:40.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b3b5d640 1 --2- 172.21.15.103:0/4261803661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62ac102790 0x7f62ac198970 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c890 tx=0x7f629c00cd50 comp rx=0 tx=0).stop 2024-09-19T06:53:40.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b3b5d640 1 -- 172.21.15.103:0/4261803661 shutdown_connections 2024-09-19T06:53:40.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b3b5d640 1 --2- 172.21.15.103:0/4261803661 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62980a33b0 0x7f62980a5870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b3b5d640 1 --2- 172.21.15.103:0/4261803661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ac106f10 0x7f62ac19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b3b5d640 1 --2- 172.21.15.103:0/4261803661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62ac102790 0x7f62ac198970 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b3b5d640 1 --2- 172.21.15.103:0/4261803661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac101de0 0x7f62ac198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:40.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.923+0000 7f62b3b5d640 1 -- 172.21.15.103:0/4261803661 >> 172.21.15.103:0/4261803661 conn(0x7f62ac0fd8c0 msgr2=0x7f62ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:40.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.924+0000 7f62b3b5d640 1 -- 172.21.15.103:0/4261803661 shutdown_connections 2024-09-19T06:53:40.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:40.924+0000 7f62b3b5d640 1 -- 172.21.15.103:0/4261803661 wait complete. 2024-09-19T06:53:41.844 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:53:42.089 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:42.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:42 smithi103 ceph-mon[31158]: pgmap v137: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-19T06:53:42.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:42 smithi174 ceph-mon[35637]: pgmap v137: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-19T06:53:42.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:42 smithi134 ceph-mon[35356]: pgmap v137: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-19T06:53:42.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 -- 172.21.15.103:0/807147527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc100e30 msgr2=0x7f5bbc104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:42.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 --2- 172.21.15.103:0/807147527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc100e30 0x7f5bbc104e80 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009900 tx=0x7f5ba402a410 comp rx=0 tx=0).stop 2024-09-19T06:53:42.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 -- 172.21.15.103:0/807147527 shutdown_connections 2024-09-19T06:53:42.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 --2- 172.21.15.103:0/807147527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bbc1055b0 0x7f5bbc10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 --2- 172.21.15.103:0/807147527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc100e30 0x7f5bbc104e80 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 --2- 172.21.15.103:0/807147527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bbc100480 0x7f5bbc100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 -- 172.21.15.103:0/807147527 >> 172.21.15.103:0/807147527 conn(0x7f5bbc077210 msgr2=0x7f5bbc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:42.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 -- 172.21.15.103:0/807147527 shutdown_connections 2024-09-19T06:53:42.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 -- 172.21.15.103:0/807147527 wait complete. 2024-09-19T06:53:42.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 Processor -- start 2024-09-19T06:53:42.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 -- start start 2024-09-19T06:53:42.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bbc100480 0x7f5bbc1a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc100e30 0x7f5bbc1a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bbc1055b0 0x7f5bbc19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc10fe30 con 0x7f5bbc100e30 2024-09-19T06:53:42.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc10fcb0 con 0x7f5bbc100480 2024-09-19T06:53:42.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.560+0000 7f5bc2fd9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc10ffb0 con 0x7f5bbc1055b0 2024-09-19T06:53:42.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc100e30 0x7f5bbc1a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:42.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc100e30 0x7f5bbc1a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56850/0 (socket says 172.21.15.103:56850) 2024-09-19T06:53:42.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bbbfff640 1 -- 172.21.15.103:0/2952553994 learned_addr learned my addr 172.21.15.103:0/2952553994 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:42.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bc0d75640 1 --2- 172.21.15.103:0/2952553994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bbc100480 0x7f5bbc1a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:42.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bbbfff640 1 -- 172.21.15.103:0/2952553994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bbc1055b0 msgr2=0x7f5bbc19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:42.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bc1576640 1 --2- 172.21.15.103:0/2952553994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bbc1055b0 0x7f5bbc19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:42.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bbbfff640 1 --2- 172.21.15.103:0/2952553994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bbc1055b0 0x7f5bbc19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bbbfff640 1 -- 172.21.15.103:0/2952553994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bbc100480 msgr2=0x7f5bbc1a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:42.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bbbfff640 1 --2- 172.21.15.103:0/2952553994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bbc100480 0x7f5bbc1a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bbbfff640 1 -- 172.21.15.103:0/2952553994 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc19aea0 con 0x7f5bbc100e30 2024-09-19T06:53:42.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bc0d75640 1 --2- 172.21.15.103:0/2952553994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bbc100480 0x7f5bbc1a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:42.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bc1576640 1 --2- 172.21.15.103:0/2952553994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bbc1055b0 0x7f5bbc19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:42.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bbbfff640 1 --2- 172.21.15.103:0/2952553994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc100e30 0x7f5bbc1a0aa0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4005ab0 tx=0x7f5ba40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:42.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bb9ffb640 1 -- 172.21.15.103:0/2952553994 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba403f030 con 0x7f5bbc100e30 2024-09-19T06:53:42.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bc2fd9640 1 -- 172.21.15.103:0/2952553994 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bbc19b130 con 0x7f5bbc100e30 2024-09-19T06:53:42.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bb9ffb640 1 -- 172.21.15.103:0/2952553994 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ba4003ce0 con 0x7f5bbc100e30 2024-09-19T06:53:42.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bb9ffb640 1 -- 172.21.15.103:0/2952553994 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4035070 con 0x7f5bbc100e30 2024-09-19T06:53:42.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.561+0000 7f5bc2fd9640 1 -- 172.21.15.103:0/2952553994 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bbc1a7310 con 0x7f5bbc100e30 2024-09-19T06:53:42.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.564+0000 7f5bb9ffb640 1 -- 172.21.15.103:0/2952553994 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f5ba4042030 con 0x7f5bbc100e30 2024-09-19T06:53:42.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.564+0000 7f5bc53b2740 1 Processor -- start 2024-09-19T06:53:42.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.564+0000 7f5bc53b2740 1 -- start start 2024-09-19T06:53:42.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc53b2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00bb40 0x55ef9f00a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc53b2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00a5d0 0x55ef9f00aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc53b2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 0x55ef9f017760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc53b2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef9ef702f0 con 0x55ef9f00a5d0 2024-09-19T06:53:42.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc53b2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ef9ef17cf0 con 0x55ef9f015370 2024-09-19T06:53:42.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc53b2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef9ef6c2b0 con 0x55ef9f00bb40 2024-09-19T06:53:42.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc0d75640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00a5d0 0x55ef9f00aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:42.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc0d75640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00a5d0 0x55ef9f00aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56854/0 (socket says 172.21.15.103:56854) 2024-09-19T06:53:42.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc0d75640 1 -- 172.21.15.103:0/3452424850 learned_addr learned my addr 172.21.15.103:0/3452424850 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:42.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc1576640 1 --2- 172.21.15.103:0/3452424850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00bb40 0x55ef9f00a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:42.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bb9ffb640 1 --2- 172.21.15.103:0/2952553994 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5ba80a33a0 0x7f5ba80a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bbbfff640 1 --2- 172.21.15.103:0/3452424850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 0x55ef9f017760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:42.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc0d75640 1 -- 172.21.15.103:0/3452424850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00bb40 msgr2=0x55ef9f00a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:42.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc1576640 1 --2- 172.21.15.103:0/2952553994 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5ba80a33a0 0x7f5ba80a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:42.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc0d75640 1 --2- 172.21.15.103:0/3452424850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00bb40 0x55ef9f00a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc0d75640 1 -- 172.21.15.103:0/3452424850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 msgr2=0x55ef9f017760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:42.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc0d75640 1 --2- 172.21.15.103:0/3452424850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 0x55ef9f017760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc0d75640 1 -- 172.21.15.103:0/3452424850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef9eefc3a0 con 0x55ef9f00a5d0 2024-09-19T06:53:42.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bb9ffb640 1 -- 172.21.15.103:0/2952553994 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5ba40f02b0 con 0x7f5bbc100e30 2024-09-19T06:53:42.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.565+0000 7f5bc0d75640 1 --2- 172.21.15.103:0/3452424850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00a5d0 0x55ef9f00aa10 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00aa10 tx=0x7f5bac00aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147ee4f4dbd7e47f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:42.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.566+0000 7f5bb2ffd640 1 -- 172.21.15.103:0/3452424850 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac015070 con 0x55ef9f00a5d0 2024-09-19T06:53:42.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.566+0000 7f5bc1576640 1 --2- 172.21.15.103:0/2952553994 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5ba80a33a0 0x7f5ba80a5860 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4010500 tx=0x7f5bb4004a40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:42.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.566+0000 7f5bb2ffd640 1 -- 172.21.15.103:0/3452424850 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bac00cc60 con 0x55ef9f00a5d0 2024-09-19T06:53:42.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.566+0000 7f5bb2ffd640 1 -- 172.21.15.103:0/3452424850 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0106c0 con 0x55ef9f00a5d0 2024-09-19T06:53:42.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.566+0000 7f5bc53b2740 1 -- 172.21.15.103:0/3452424850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00a5d0 msgr2=0x55ef9f00aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:42.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.566+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/3452424850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00a5d0 0x55ef9f00aa10 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00aa10 tx=0x7f5bac00aed0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.566+0000 7f5bc53b2740 1 -- 172.21.15.103:0/3452424850 shutdown_connections 2024-09-19T06:53:42.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.566+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/3452424850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 0x55ef9f017760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.566+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/3452424850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00a5d0 0x55ef9f00aa10 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.566+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/3452424850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00bb40 0x55ef9f00a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.566+0000 7f5bc53b2740 1 -- 172.21.15.103:0/3452424850 >> 172.21.15.103:0/3452424850 conn(0x55ef9efd3310 msgr2=0x55ef9f006c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:42.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.566+0000 7f5bc53b2740 1 -- 172.21.15.103:0/3452424850 shutdown_connections 2024-09-19T06:53:42.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.566+0000 7f5bc53b2740 1 -- 172.21.15.103:0/3452424850 wait complete. 2024-09-19T06:53:42.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.567+0000 7f5bc53b2740 1 -- start start 2024-09-19T06:53:42.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.567+0000 7f5bc53b2740 1 Processor -- start 2024-09-19T06:53:42.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.567+0000 7f5bc53b2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00a5d0 0x55ef9f014cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.567+0000 7f5bc53b2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f011f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.567+0000 7f5bc53b2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 0x55ef9f0124e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.567+0000 7f5bc53b2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef9f018070 con 0x55ef9f00bb40 2024-09-19T06:53:42.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.567+0000 7f5bc53b2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ef9ef2b060 con 0x55ef9f015370 2024-09-19T06:53:42.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.567+0000 7f5bc53b2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef9f0181f0 con 0x55ef9f00a5d0 2024-09-19T06:53:42.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.567+0000 7f5bc53b2740 10 client.0 _ll_register_callbacks cb 0x55ef9ec7e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:42.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.567+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f011f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:42.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.567+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f011f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56868/0 (socket says 172.21.15.103:56868) 2024-09-19T06:53:42.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.567+0000 7f5bbbfff640 1 -- 172.21.15.103:0/2300224271 learned_addr learned my addr 172.21.15.103:0/2300224271 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:42.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.567+0000 7f5bc1576640 1 --2- 172.21.15.103:0/2300224271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 0x55ef9f0124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:42.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5bbbfff640 1 -- 172.21.15.103:0/2300224271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00a5d0 msgr2=0x55ef9f014cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:42.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5bc0d75640 1 --2- 172.21.15.103:0/2300224271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00a5d0 0x55ef9f014cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:42.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5bbbfff640 1 --2- 172.21.15.103:0/2300224271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00a5d0 0x55ef9f014cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5bbbfff640 1 -- 172.21.15.103:0/2300224271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 msgr2=0x55ef9f0124e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:42.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5bbbfff640 1 --2- 172.21.15.103:0/2300224271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 0x55ef9f0124e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5bbbfff640 1 -- 172.21.15.103:0/2300224271 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef9ece13c0 con 0x55ef9f00bb40 2024-09-19T06:53:42.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5bc0d75640 1 --2- 172.21.15.103:0/2300224271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00a5d0 0x55ef9f014cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:42.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5bbbfff640 1 --2- 172.21.15.103:0/2300224271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f011f80 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4000c00 tx=0x7f5ba40bc1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe128e4ef0336696 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:42.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5b9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:42.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5b9bfff640 1 -- 172.21.15.103:0/2300224271 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4049050 con 0x55ef9f00bb40 2024-09-19T06:53:42.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5b9bfff640 1 -- 172.21.15.103:0/2300224271 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5ba403b460 con 0x55ef9f00bb40 2024-09-19T06:53:42.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5bc53b2740 1 -- 172.21.15.103:0/2300224271 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bac00cc60 con 0x55ef9f00bb40 2024-09-19T06:53:42.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5b9bfff640 1 -- 172.21.15.103:0/2300224271 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ba40bcca0 con 0x55ef9f00bb40 2024-09-19T06:53:42.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5bc53b2740 10 client.17022 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:42.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5bc53b2740 1 -- 172.21.15.103:0/2300224271 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ef9f023ab0 con 0x55ef9f00bb40 2024-09-19T06:53:42.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5b9bfff640 1 -- 172.21.15.103:0/2300224271 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bac00cc60 con 0x55ef9f00bb40 2024-09-19T06:53:42.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5b9bfff640 1 -- 172.21.15.103:0/2300224271 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5ba40edc90 con 0x55ef9f00bb40 2024-09-19T06:53:42.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5bc53b2740 10 client.17022 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:42.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5bc53b2740 10 client.17022 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:53:42.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5bc53b2740 20 client.17022 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:42.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.568+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/2300224271 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55ef9f024e00 0x55ef9f027270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.569+0000 7f5bc53b2740 4 client.17022 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:42.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.569+0000 7f5bc53b2740 1 -- 172.21.15.103:0/2300224271 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef9ef7d860 con 0x55ef9f024e00 2024-09-19T06:53:42.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.569+0000 7f5bc0d75640 1 --2- 172.21.15.103:0/2300224271 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55ef9f024e00 0x55ef9f027270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:42.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.569+0000 7f5bc0d75640 1 --2- 172.21.15.103:0/2300224271 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55ef9f024e00 0x55ef9f027270 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c601ae9018db32b server_cookie=cee123b4025cd2ba in_seq=0 out_seq=0 2024-09-19T06:53:42.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.569+0000 7f5b9bfff640 10 client.17022 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:53:42.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.570+0000 7f5b9bfff640 1 -- 172.21.15.103:0/2300224271 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef9ef7d860 con 0x55ef9f024e00 2024-09-19T06:53:42.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.570+0000 7f5b9bfff640 10 client.17022 handle_command_reply: tid=0 2024-09-19T06:53:42.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.570+0000 7f5bc53b2740 1 client.17022 shutdown 2024-09-19T06:53:42.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.570+0000 7f5bc53b2740 1 -- 172.21.15.103:0/2300224271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 msgr2=0x55ef9f011f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:42.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.570+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/2300224271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f011f80 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4000c00 tx=0x7f5ba40bc1a0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.570+0000 7f5bc53b2740 1 -- 172.21.15.103:0/2300224271 shutdown_connections 2024-09-19T06:53:42.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.570+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/2300224271 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55ef9f024e00 0x55ef9f027270 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.570+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/2300224271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 0x55ef9f0124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.570+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/2300224271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f011f80 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.570+0000 7f5b9bfff640 0 client.17022 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:53:42.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.570+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/2300224271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00a5d0 0x55ef9f014cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.570+0000 7f5bc53b2740 1 -- 172.21.15.103:0/2300224271 >> 172.21.15.103:0/2300224271 conn(0x55ef9efd3310 msgr2=0x55ef9f00c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:42.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.570+0000 7f5bc53b2740 1 -- 172.21.15.103:0/2300224271 shutdown_connections 2024-09-19T06:53:42.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.570+0000 7f5bc53b2740 1 -- 172.21.15.103:0/2300224271 wait complete. 2024-09-19T06:53:42.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.570+0000 7f5bc53b2740 20 client.17022 trim_cache size 0 max 16384 2024-09-19T06:53:42.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bc53b2740 1 Processor -- start 2024-09-19T06:53:42.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bc53b2740 1 -- start start 2024-09-19T06:53:42.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bc53b2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00a5d0 0x55ef9f021340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bc53b2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f016760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bc53b2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 0x55ef9f016ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bc53b2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc1afbc0 con 0x55ef9f00bb40 2024-09-19T06:53:42.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bc53b2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc1b1700 con 0x55ef9f015370 2024-09-19T06:53:42.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bc53b2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc1b0290 con 0x55ef9f00a5d0 2024-09-19T06:53:42.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f016760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:42.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f016760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56884/0 (socket says 172.21.15.103:56884) 2024-09-19T06:53:42.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bbbfff640 1 -- 172.21.15.103:0/2979352542 learned_addr learned my addr 172.21.15.103:0/2979352542 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:42.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bbbfff640 1 -- 172.21.15.103:0/2979352542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00a5d0 msgr2=0x55ef9f021340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:42.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bbbfff640 1 --2- 172.21.15.103:0/2979352542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00a5d0 0x55ef9f021340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bbbfff640 1 -- 172.21.15.103:0/2979352542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 msgr2=0x55ef9f016ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:42.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bbbfff640 1 --2- 172.21.15.103:0/2979352542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 0x55ef9f016ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bbbfff640 1 -- 172.21.15.103:0/2979352542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef9f0104d0 con 0x55ef9f00bb40 2024-09-19T06:53:42.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.583+0000 7f5bbbfff640 1 --2- 172.21.15.103:0/2979352542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f016760 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40bc2a0 tx=0x7f5ba40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174fd149952b754f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:42.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.584+0000 7f5bb27fc640 1 -- 172.21.15.103:0/2979352542 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4049070 con 0x55ef9f00bb40 2024-09-19T06:53:42.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.584+0000 7f5bb27fc640 1 -- 172.21.15.103:0/2979352542 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ba40c0950 con 0x55ef9f00bb40 2024-09-19T06:53:42.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.584+0000 7f5bb27fc640 1 -- 172.21.15.103:0/2979352542 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba403b460 con 0x55ef9f00bb40 2024-09-19T06:53:42.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.584+0000 7f5bc53b2740 1 -- 172.21.15.103:0/2979352542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 msgr2=0x55ef9f016760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:42.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.584+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/2979352542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f016760 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40bc2a0 tx=0x7f5ba40045e0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.584+0000 7f5bc53b2740 1 -- 172.21.15.103:0/2979352542 shutdown_connections 2024-09-19T06:53:42.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.584+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/2979352542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 0x55ef9f016ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.584+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/2979352542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f016760 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.584+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/2979352542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00a5d0 0x55ef9f021340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.584+0000 7f5bc53b2740 1 -- 172.21.15.103:0/2979352542 >> 172.21.15.103:0/2979352542 conn(0x55ef9efd3310 msgr2=0x55ef9f011070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:42.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.584+0000 7f5bc53b2740 1 -- 172.21.15.103:0/2979352542 shutdown_connections 2024-09-19T06:53:42.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.584+0000 7f5bc53b2740 1 -- 172.21.15.103:0/2979352542 wait complete. 2024-09-19T06:53:42.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bc53b2740 1 -- start start 2024-09-19T06:53:42.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bc53b2740 1 Processor -- start 2024-09-19T06:53:42.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bc53b2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00a5d0 0x55ef9f02c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bc53b2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f02cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bc53b2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 0x55ef9f02d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bc53b2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc1b1030 con 0x55ef9f00bb40 2024-09-19T06:53:42.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bc53b2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc1b0960 con 0x55ef9f015370 2024-09-19T06:53:42.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bc53b2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc1adb80 con 0x55ef9f00a5d0 2024-09-19T06:53:42.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bc53b2740 10 client.0 _ll_register_callbacks cb 0x7f5ba40b8910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:42.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f02cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:42.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f02cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56898/0 (socket says 172.21.15.103:56898) 2024-09-19T06:53:42.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bbbfff640 1 -- 172.21.15.103:0/1877862875 learned_addr learned my addr 172.21.15.103:0/1877862875 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:42.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bc0d75640 1 --2- 172.21.15.103:0/1877862875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00a5d0 0x55ef9f02c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:42.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bbbfff640 1 -- 172.21.15.103:0/1877862875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00a5d0 msgr2=0x55ef9f02c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:42.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bbbfff640 1 --2- 172.21.15.103:0/1877862875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00a5d0 0x55ef9f02c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bbbfff640 1 -- 172.21.15.103:0/1877862875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 msgr2=0x55ef9f02d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:42.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bbbfff640 1 --2- 172.21.15.103:0/1877862875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 0x55ef9f02d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bbbfff640 1 -- 172.21.15.103:0/1877862875 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef9f01b8f0 con 0x55ef9f00bb40 2024-09-19T06:53:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.585+0000 7f5bc0d75640 1 --2- 172.21.15.103:0/1877862875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00a5d0 0x55ef9f02c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.586+0000 7f5bbbfff640 1 --2- 172.21.15.103:0/1877862875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f02cc00 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4000c00 tx=0x7f5ba40b8180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36b3b26bc47da19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.586+0000 7f5b9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.586+0000 7f5b9bfff640 1 -- 172.21.15.103:0/1877862875 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba40d9020 con 0x55ef9f00bb40 2024-09-19T06:53:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.586+0000 7f5b9bfff640 1 -- 172.21.15.103:0/1877862875 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5ba403b460 con 0x55ef9f00bb40 2024-09-19T06:53:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.586+0000 7f5bc53b2740 1 -- 172.21.15.103:0/1877862875 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ba40c0950 con 0x55ef9f00bb40 2024-09-19T06:53:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.586+0000 7f5b9bfff640 1 -- 172.21.15.103:0/1877862875 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ba4049040 con 0x55ef9f00bb40 2024-09-19T06:53:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.586+0000 7f5bc53b2740 10 client.17028 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.586+0000 7f5bc53b2740 1 -- 172.21.15.103:0/1877862875 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ef9f01b8f0 con 0x55ef9f00bb40 2024-09-19T06:53:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.586+0000 7f5b9bfff640 1 -- 172.21.15.103:0/1877862875 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ba40c0950 con 0x55ef9f00bb40 2024-09-19T06:53:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.586+0000 7f5b9bfff640 1 -- 172.21.15.103:0/1877862875 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5ba4087290 con 0x55ef9f00bb40 2024-09-19T06:53:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.586+0000 7f5bc53b2740 10 client.17028 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.586+0000 7f5bc53b2740 10 client.17028 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:53:42.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.586+0000 7f5bc53b2740 20 client.17028 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:42.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.586+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/1877862875 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55ef9f08ffd0 0x55ef9f0923c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:42.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.586+0000 7f5bc53b2740 4 client.17028 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:53:42.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.586+0000 7f5bc53b2740 1 -- 172.21.15.103:0/1877862875 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef9ef77d70 con 0x55ef9f08ffd0 2024-09-19T06:53:42.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.587+0000 7f5bc0d75640 1 --2- 172.21.15.103:0/1877862875 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55ef9f08ffd0 0x55ef9f0923c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:42.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.587+0000 7f5b9bfff640 10 client.17028 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:53:42.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.587+0000 7f5bc0d75640 1 --2- 172.21.15.103:0/1877862875 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55ef9f08ffd0 0x55ef9f0923c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ce59c9d618403e03 server_cookie=26684bd9f8f725b6 in_seq=0 out_seq=0 2024-09-19T06:53:42.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.587+0000 7f5b9bfff640 1 -- 172.21.15.103:0/1877862875 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef9ef77d70 con 0x55ef9f08ffd0 2024-09-19T06:53:42.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.587+0000 7f5b9bfff640 10 client.17028 handle_command_reply: tid=0 2024-09-19T06:53:42.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.587+0000 7f5bc53b2740 1 client.17028 shutdown 2024-09-19T06:53:42.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.587+0000 7f5bc53b2740 1 -- 172.21.15.103:0/1877862875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 msgr2=0x55ef9f02cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:42.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.587+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/1877862875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f02cc00 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4000c00 tx=0x7f5ba40b8180 comp rx=0 tx=0).stop 2024-09-19T06:53:42.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.587+0000 7f5bc53b2740 1 -- 172.21.15.103:0/1877862875 shutdown_connections 2024-09-19T06:53:42.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.587+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/1877862875 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55ef9f08ffd0 0x55ef9f0923c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.587+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/1877862875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef9f015370 0x55ef9f02d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.587+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/1877862875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9f00bb40 0x55ef9f02cc00 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.587+0000 7f5bc53b2740 1 --2- 172.21.15.103:0/1877862875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9f00a5d0 0x55ef9f02c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.587+0000 7f5bc53b2740 1 -- 172.21.15.103:0/1877862875 >> 172.21.15.103:0/1877862875 conn(0x55ef9efd3310 msgr2=0x55ef9f0137e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:42.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.587+0000 7f5b9bfff640 0 client.17028 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:53:42.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.588+0000 7f5bc53b2740 1 -- 172.21.15.103:0/1877862875 shutdown_connections 2024-09-19T06:53:42.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.588+0000 7f5bc53b2740 1 -- 172.21.15.103:0/1877862875 wait complete. 2024-09-19T06:53:42.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.588+0000 7f5bc53b2740 20 client.17028 trim_cache size 0 max 16384 2024-09-19T06:53:42.605 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:53:42.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.588+0000 7f5bc2fd9640 1 -- 172.21.15.103:0/2952553994 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5ba80a33a0 msgr2=0x7f5ba80a5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:42.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.588+0000 7f5bc2fd9640 1 --2- 172.21.15.103:0/2952553994 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5ba80a33a0 0x7f5ba80a5860 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4010500 tx=0x7f5bb4004a40 comp rx=0 tx=0).stop 2024-09-19T06:53:42.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.588+0000 7f5bc2fd9640 1 -- 172.21.15.103:0/2952553994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc100e30 msgr2=0x7f5bbc1a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:42.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.588+0000 7f5bc2fd9640 1 --2- 172.21.15.103:0/2952553994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc100e30 0x7f5bbc1a0aa0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4005ab0 tx=0x7f5ba40042f0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.588+0000 7f5bc2fd9640 1 -- 172.21.15.103:0/2952553994 shutdown_connections 2024-09-19T06:53:42.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.588+0000 7f5bc2fd9640 1 --2- 172.21.15.103:0/2952553994 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5ba80a33a0 0x7f5ba80a5860 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.588+0000 7f5bc2fd9640 1 --2- 172.21.15.103:0/2952553994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bbc1055b0 0x7f5bbc19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.588+0000 7f5bc2fd9640 1 --2- 172.21.15.103:0/2952553994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc100e30 0x7f5bbc1a0aa0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.588+0000 7f5bc2fd9640 1 --2- 172.21.15.103:0/2952553994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bbc100480 0x7f5bbc1a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:42.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.588+0000 7f5bc2fd9640 1 -- 172.21.15.103:0/2952553994 >> 172.21.15.103:0/2952553994 conn(0x7f5bbc077210 msgr2=0x7f5bbc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:42.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.588+0000 7f5bc2fd9640 1 -- 172.21.15.103:0/2952553994 shutdown_connections 2024-09-19T06:53:42.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:42.588+0000 7f5bc2fd9640 1 -- 172.21.15.103:0/2952553994 wait complete. 2024-09-19T06:53:43.406 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:53:43.653 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:44.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.111+0000 7f0a47a91640 1 -- 172.21.15.103:0/4005079197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a40101430 msgr2=0x7f0a4010d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:44.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.111+0000 7f0a47a91640 1 --2- 172.21.15.103:0/4005079197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a40101430 0x7f0a4010d960 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009cd0 tx=0x7f0a3802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.111+0000 7f0a47a91640 1 -- 172.21.15.103:0/4005079197 shutdown_connections 2024-09-19T06:53:44.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.111+0000 7f0a47a91640 1 --2- 172.21.15.103:0/4005079197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a40101430 0x7f0a4010d960 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.111+0000 7f0a47a91640 1 --2- 172.21.15.103:0/4005079197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a40100a90 0x7f0a40100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.111+0000 7f0a47a91640 1 --2- 172.21.15.103:0/4005079197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a40106a90 0x7f0a40106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.111+0000 7f0a47a91640 1 -- 172.21.15.103:0/4005079197 >> 172.21.15.103:0/4005079197 conn(0x7f0a400fc7d0 msgr2=0x7f0a400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:44.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.111+0000 7f0a47a91640 1 -- 172.21.15.103:0/4005079197 shutdown_connections 2024-09-19T06:53:44.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.111+0000 7f0a47a91640 1 -- 172.21.15.103:0/4005079197 wait complete. 2024-09-19T06:53:44.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a47a91640 1 Processor -- start 2024-09-19T06:53:44.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a47a91640 1 -- start start 2024-09-19T06:53:44.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a47a91640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a40100a90 0x7f0a40198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a47a91640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a40101430 0x7f0a401989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a47a91640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a40106a90 0x7f0a4019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a47a91640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a40110eb0 con 0x7f0a40101430 2024-09-19T06:53:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a47a91640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0a40110d30 con 0x7f0a40100a90 2024-09-19T06:53:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a47a91640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a40111030 con 0x7f0a40106a90 2024-09-19T06:53:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a4502c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a40101430 0x7f0a401989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a4502c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a40101430 0x7f0a401989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56914/0 (socket says 172.21.15.103:56914) 2024-09-19T06:53:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a4502c640 1 -- 172.21.15.103:0/3030379677 learned_addr learned my addr 172.21.15.103:0/3030379677 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a4602e640 1 --2- 172.21.15.103:0/3030379677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a40106a90 0x7f0a4019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a4502c640 1 -- 172.21.15.103:0/3030379677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a40106a90 msgr2=0x7f0a4019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a4582d640 1 --2- 172.21.15.103:0/3030379677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a40100a90 0x7f0a40198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a4502c640 1 --2- 172.21.15.103:0/3030379677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a40106a90 0x7f0a4019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a4502c640 1 -- 172.21.15.103:0/3030379677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a40100a90 msgr2=0x7f0a40198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.112+0000 7f0a4502c640 1 --2- 172.21.15.103:0/3030379677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a40100a90 0x7f0a40198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.113+0000 7f0a4502c640 1 -- 172.21.15.103:0/3030379677 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4019d440 con 0x7f0a40101430 2024-09-19T06:53:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.113+0000 7f0a4602e640 1 --2- 172.21.15.103:0/3030379677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a40106a90 0x7f0a4019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.113+0000 7f0a4582d640 1 --2- 172.21.15.103:0/3030379677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a40100a90 0x7f0a40198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.113+0000 7f0a4502c640 1 --2- 172.21.15.103:0/3030379677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a40101430 0x7f0a401989d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000c890 tx=0x7f0a3000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.113+0000 7f0a36ffd640 1 -- 172.21.15.103:0/3030379677 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30004220 con 0x7f0a40101430 2024-09-19T06:53:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.113+0000 7f0a47a91640 1 -- 172.21.15.103:0/3030379677 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4019d730 con 0x7f0a40101430 2024-09-19T06:53:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.113+0000 7f0a36ffd640 1 -- 172.21.15.103:0/3030379677 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a300043c0 con 0x7f0a40101430 2024-09-19T06:53:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.113+0000 7f0a36ffd640 1 -- 172.21.15.103:0/3030379677 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30004a00 con 0x7f0a40101430 2024-09-19T06:53:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.113+0000 7f0a47a91640 1 -- 172.21.15.103:0/3030379677 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a401a5010 con 0x7f0a40101430 2024-09-19T06:53:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.116+0000 7f0a49e6a740 1 Processor -- start 2024-09-19T06:53:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.116+0000 7f0a36ffd640 1 -- 172.21.15.103:0/3030379677 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0a300079b0 con 0x7f0a40101430 2024-09-19T06:53:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.116+0000 7f0a49e6a740 1 -- start start 2024-09-19T06:53:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.116+0000 7f0a49e6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc215bb40 0x560bc215a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.116+0000 7f0a49e6a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc215aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.116+0000 7f0a49e6a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 0x560bc216bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.116+0000 7f0a49e6a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560bc20c02f0 con 0x560bc215a5d0 2024-09-19T06:53:44.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.116+0000 7f0a49e6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560bc2067cf0 con 0x560bc215bb40 2024-09-19T06:53:44.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.116+0000 7f0a49e6a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560bc20bc2b0 con 0x560bc2169780 2024-09-19T06:53:44.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.116+0000 7f0a4582d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc215aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a4582d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc215aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56922/0 (socket says 172.21.15.103:56922) 2024-09-19T06:53:44.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a4582d640 1 -- 172.21.15.103:0/1376541257 learned_addr learned my addr 172.21.15.103:0/1376541257 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:44.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a4602e640 1 --2- 172.21.15.103:0/1376541257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc215bb40 0x560bc215a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a4502c640 1 --2- 172.21.15.103:0/1376541257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 0x560bc216bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a36ffd640 1 --2- 172.21.15.103:0/3030379677 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0a2c0a33b0 0x7f0a2c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a4582d640 1 -- 172.21.15.103:0/1376541257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 msgr2=0x560bc216bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:44.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a4582d640 1 --2- 172.21.15.103:0/1376541257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 0x560bc216bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a4582d640 1 -- 172.21.15.103:0/1376541257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc215bb40 msgr2=0x560bc215a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:44.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a4582d640 1 --2- 172.21.15.103:0/1376541257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc215bb40 0x560bc215a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a4582d640 1 -- 172.21.15.103:0/1376541257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bc204c3a0 con 0x560bc215a5d0 2024-09-19T06:53:44.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a4502c640 1 --2- 172.21.15.103:0/1376541257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 0x560bc216bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:44.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a4602e640 1 --2- 172.21.15.103:0/1376541257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc215bb40 0x560bc215a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:44.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a4602e640 1 --2- 172.21.15.103:0/3030379677 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0a2c0a33b0 0x7f0a2c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a36ffd640 1 -- 172.21.15.103:0/3030379677 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f0a300cfac0 con 0x7f0a40101430 2024-09-19T06:53:44.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a4582d640 1 --2- 172.21.15.103:0/1376541257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc215aa10 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c00aa10 tx=0x7f0a3c00aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79224ab3902536e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:44.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a2bfff640 1 -- 172.21.15.103:0/1376541257 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c015070 con 0x560bc215a5d0 2024-09-19T06:53:44.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a2bfff640 1 -- 172.21.15.103:0/1376541257 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3c00cc60 con 0x560bc215a5d0 2024-09-19T06:53:44.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a2bfff640 1 -- 172.21.15.103:0/1376541257 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c018690 con 0x560bc215a5d0 2024-09-19T06:53:44.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.117+0000 7f0a4602e640 1 --2- 172.21.15.103:0/3030379677 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0a2c0a33b0 0x7f0a2c0a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009430 tx=0x7f0a3803e040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:44.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.118+0000 7f0a49e6a740 1 -- 172.21.15.103:0/1376541257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 msgr2=0x560bc215aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:44.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.118+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/1376541257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc215aa10 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c00aa10 tx=0x7f0a3c00aed0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.118+0000 7f0a49e6a740 1 -- 172.21.15.103:0/1376541257 shutdown_connections 2024-09-19T06:53:44.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.118+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/1376541257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 0x560bc216bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.118+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/1376541257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc215aa10 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.118+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/1376541257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc215bb40 0x560bc215a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.118+0000 7f0a49e6a740 1 -- 172.21.15.103:0/1376541257 >> 172.21.15.103:0/1376541257 conn(0x560bc2123310 msgr2=0x560bc2156c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:44.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.118+0000 7f0a49e6a740 1 -- 172.21.15.103:0/1376541257 shutdown_connections 2024-09-19T06:53:44.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.118+0000 7f0a49e6a740 1 -- 172.21.15.103:0/1376541257 wait complete. 2024-09-19T06:53:44.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a49e6a740 1 -- start start 2024-09-19T06:53:44.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a49e6a740 1 Processor -- start 2024-09-19T06:53:44.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a49e6a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc21690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a49e6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc215bb40 0x560bc2166390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.155 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:53:44.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a49e6a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 0x560bc21668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a49e6a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560bc216c480 con 0x560bc215a5d0 2024-09-19T06:53:44.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a49e6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560bc207b060 con 0x560bc215bb40 2024-09-19T06:53:44.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a49e6a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560bc216c600 con 0x560bc2169780 2024-09-19T06:53:44.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a49e6a740 10 client.0 _ll_register_callbacks cb 0x560bc1dce8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:44.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a4582d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc21690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a4582d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc21690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56930/0 (socket says 172.21.15.103:56930) 2024-09-19T06:53:44.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a4582d640 1 -- 172.21.15.103:0/3074695213 learned_addr learned my addr 172.21.15.103:0/3074695213 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:44.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a4502c640 1 --2- 172.21.15.103:0/3074695213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc215bb40 0x560bc2166390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a4602e640 1 --2- 172.21.15.103:0/3074695213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 0x560bc21668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a4582d640 1 -- 172.21.15.103:0/3074695213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 msgr2=0x560bc21668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:44.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a4582d640 1 --2- 172.21.15.103:0/3074695213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 0x560bc21668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a4582d640 1 -- 172.21.15.103:0/3074695213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc215bb40 msgr2=0x560bc2166390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:44.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a4582d640 1 --2- 172.21.15.103:0/3074695213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc215bb40 0x560bc2166390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a4582d640 1 -- 172.21.15.103:0/3074695213 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bc1e313c0 con 0x560bc215a5d0 2024-09-19T06:53:44.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a4602e640 1 --2- 172.21.15.103:0/3074695213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 0x560bc21668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:44.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.119+0000 7f0a4502c640 1 --2- 172.21.15.103:0/3074695213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc215bb40 0x560bc2166390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:44.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.120+0000 7f0a4582d640 1 --2- 172.21.15.103:0/3074695213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc21690e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c006aa0 tx=0x7f0a3c011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93357288e745a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:44.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.120+0000 7f0a297fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:44.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.120+0000 7f0a297fa640 1 -- 172.21.15.103:0/3074695213 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c015040 con 0x560bc215a5d0 2024-09-19T06:53:44.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.120+0000 7f0a49e6a740 1 -- 172.21.15.103:0/3074695213 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a3c00cc60 con 0x560bc215a5d0 2024-09-19T06:53:44.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.120+0000 7f0a297fa640 1 -- 172.21.15.103:0/3074695213 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f0a3c01a660 con 0x560bc215a5d0 2024-09-19T06:53:44.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.120+0000 7f0a297fa640 1 -- 172.21.15.103:0/3074695213 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a3c00ce10 con 0x560bc215a5d0 2024-09-19T06:53:44.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.120+0000 7f0a49e6a740 10 client.17040 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:44.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.120+0000 7f0a49e6a740 1 -- 172.21.15.103:0/3074695213 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560bc2177e90 con 0x560bc215a5d0 2024-09-19T06:53:44.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.120+0000 7f0a297fa640 1 -- 172.21.15.103:0/3074695213 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3c00cc60 con 0x560bc215a5d0 2024-09-19T06:53:44.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.120+0000 7f0a297fa640 1 -- 172.21.15.103:0/3074695213 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0a3c025a50 con 0x560bc215a5d0 2024-09-19T06:53:44.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.120+0000 7f0a49e6a740 10 client.17040 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:44.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.120+0000 7f0a49e6a740 10 client.17040 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:53:44.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.120+0000 7f0a49e6a740 20 client.17040 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:44.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.120+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/3074695213 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x560bc2179240 0x560bc217b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.120+0000 7f0a49e6a740 4 client.17040 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:44.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.121+0000 7f0a49e6a740 1 -- 172.21.15.103:0/3074695213 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bc20cd860 con 0x560bc2179240 2024-09-19T06:53:44.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.121+0000 7f0a4502c640 1 --2- 172.21.15.103:0/3074695213 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x560bc2179240 0x560bc217b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.121+0000 7f0a297fa640 10 client.17040 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:53:44.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.121+0000 7f0a4502c640 1 --2- 172.21.15.103:0/3074695213 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x560bc2179240 0x560bc217b630 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0011e19f2bcc33e server_cookie=f684577b7b094b33 in_seq=0 out_seq=0 2024-09-19T06:53:44.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.122+0000 7f0a297fa640 1 -- 172.21.15.103:0/3074695213 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560bc20cd860 con 0x560bc2179240 2024-09-19T06:53:44.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.122+0000 7f0a297fa640 10 client.17040 handle_command_reply: tid=0 2024-09-19T06:53:44.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.122+0000 7f0a49e6a740 1 client.17040 shutdown 2024-09-19T06:53:44.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.122+0000 7f0a49e6a740 1 -- 172.21.15.103:0/3074695213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 msgr2=0x560bc21690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:44.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.122+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/3074695213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc21690e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c006aa0 tx=0x7f0a3c011c50 comp rx=0 tx=0).stop 2024-09-19T06:53:44.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.122+0000 7f0a49e6a740 1 -- 172.21.15.103:0/3074695213 shutdown_connections 2024-09-19T06:53:44.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.122+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/3074695213 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x560bc2179240 0x560bc217b630 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.122+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/3074695213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 0x560bc21668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.122+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/3074695213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc215bb40 0x560bc2166390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.122+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/3074695213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc21690e0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.122+0000 7f0a297fa640 0 client.17040 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:53:44.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.122+0000 7f0a49e6a740 1 -- 172.21.15.103:0/3074695213 >> 172.21.15.103:0/3074695213 conn(0x560bc2123310 msgr2=0x560bc215c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:44.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.122+0000 7f0a49e6a740 1 -- 172.21.15.103:0/3074695213 shutdown_connections 2024-09-19T06:53:44.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.122+0000 7f0a49e6a740 1 -- 172.21.15.103:0/3074695213 wait complete. 2024-09-19T06:53:44.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.122+0000 7f0a49e6a740 20 client.17040 trim_cache size 0 max 16384 2024-09-19T06:53:44.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a49e6a740 1 Processor -- start 2024-09-19T06:53:44.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a49e6a740 1 -- start start 2024-09-19T06:53:44.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a49e6a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc2175720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a49e6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc215bb40 0x560bc2168300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a49e6a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 0x560bc2180c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a49e6a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a401ab700 con 0x560bc215a5d0 2024-09-19T06:53:44.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a49e6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0a401ad240 con 0x560bc215bb40 2024-09-19T06:53:44.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a49e6a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a401abdd0 con 0x560bc2169780 2024-09-19T06:53:44.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a4582d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc2175720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a4582d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc2175720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56934/0 (socket says 172.21.15.103:56934) 2024-09-19T06:53:44.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a4582d640 1 -- 172.21.15.103:0/1104532653 learned_addr learned my addr 172.21.15.103:0/1104532653 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:44.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a4602e640 1 --2- 172.21.15.103:0/1104532653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 0x560bc2180c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a4582d640 1 -- 172.21.15.103:0/1104532653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 msgr2=0x560bc2180c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:44.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a4582d640 1 --2- 172.21.15.103:0/1104532653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 0x560bc2180c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a4582d640 1 -- 172.21.15.103:0/1104532653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc215bb40 msgr2=0x560bc2168300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:44.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a4582d640 1 --2- 172.21.15.103:0/1104532653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc215bb40 0x560bc2168300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.135+0000 7f0a4582d640 1 -- 172.21.15.103:0/1104532653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bc216c5f0 con 0x560bc215a5d0 2024-09-19T06:53:44.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.136+0000 7f0a4602e640 1 --2- 172.21.15.103:0/1104532653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 0x560bc2180c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:44.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.136+0000 7f0a4582d640 1 --2- 172.21.15.103:0/1104532653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc2175720 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c025e70 tx=0x7f0a3c0118c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2727be0627baab32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:44.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.136+0000 7f0a2b7fe640 1 -- 172.21.15.103:0/1104532653 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c032070 con 0x560bc215a5d0 2024-09-19T06:53:44.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.136+0000 7f0a2b7fe640 1 -- 172.21.15.103:0/1104532653 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3c006ba0 con 0x560bc215a5d0 2024-09-19T06:53:44.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.136+0000 7f0a2b7fe640 1 -- 172.21.15.103:0/1104532653 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c015040 con 0x560bc215a5d0 2024-09-19T06:53:44.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.136+0000 7f0a49e6a740 1 -- 172.21.15.103:0/1104532653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 msgr2=0x560bc2175720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:44.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.136+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/1104532653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc2175720 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c025e70 tx=0x7f0a3c0118c0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.136+0000 7f0a49e6a740 1 -- 172.21.15.103:0/1104532653 shutdown_connections 2024-09-19T06:53:44.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.136+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/1104532653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc2169780 0x560bc2180c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.136+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/1104532653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc215bb40 0x560bc2168300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.136+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/1104532653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215a5d0 0x560bc2175720 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.136+0000 7f0a49e6a740 1 -- 172.21.15.103:0/1104532653 >> 172.21.15.103:0/1104532653 conn(0x560bc2123310 msgr2=0x560bc2167bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:44.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.137+0000 7f0a49e6a740 1 -- 172.21.15.103:0/1104532653 shutdown_connections 2024-09-19T06:53:44.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.137+0000 7f0a49e6a740 1 -- 172.21.15.103:0/1104532653 wait complete. 2024-09-19T06:53:44.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.137+0000 7f0a49e6a740 1 -- start start 2024-09-19T06:53:44.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.137+0000 7f0a49e6a740 1 Processor -- start 2024-09-19T06:53:44.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.137+0000 7f0a49e6a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc215a5d0 0x560bc2171ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.137+0000 7f0a49e6a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215bb40 0x560bc2172020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.137+0000 7f0a49e6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc2169780 0x560bc2162c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.137+0000 7f0a49e6a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a401acb70 con 0x560bc215bb40 2024-09-19T06:53:44.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.137+0000 7f0a49e6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0a401ac4a0 con 0x560bc2169780 2024-09-19T06:53:44.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.137+0000 7f0a49e6a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a401a96e0 con 0x560bc215a5d0 2024-09-19T06:53:44.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a49e6a740 10 client.0 _ll_register_callbacks cb 0x7f0a3c00a860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:44.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a4502c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215bb40 0x560bc2172020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a4502c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215bb40 0x560bc2172020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56946/0 (socket says 172.21.15.103:56946) 2024-09-19T06:53:44.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a4502c640 1 -- 172.21.15.103:0/3985219793 learned_addr learned my addr 172.21.15.103:0/3985219793 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:44.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a4582d640 1 --2- 172.21.15.103:0/3985219793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc215a5d0 0x560bc2171ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a4602e640 1 --2- 172.21.15.103:0/3985219793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc2169780 0x560bc2162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a4502c640 1 -- 172.21.15.103:0/3985219793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc215a5d0 msgr2=0x560bc2171ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:44.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a4502c640 1 --2- 172.21.15.103:0/3985219793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc215a5d0 0x560bc2171ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a4502c640 1 -- 172.21.15.103:0/3985219793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc2169780 msgr2=0x560bc2162c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:44.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a4502c640 1 --2- 172.21.15.103:0/3985219793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc2169780 0x560bc2162c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a4502c640 1 -- 172.21.15.103:0/3985219793 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bc2163400 con 0x560bc215bb40 2024-09-19T06:53:44.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a4602e640 1 --2- 172.21.15.103:0/3985219793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc2169780 0x560bc2162c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:44.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a4502c640 1 --2- 172.21.15.103:0/3985219793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215bb40 0x560bc2172020 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f0a30005fd0 tx=0x7f0a300054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c2bee885fc82bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:44.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a297fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:44.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a297fa640 1 -- 172.21.15.103:0/3985219793 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3001d040 con 0x560bc215bb40 2024-09-19T06:53:44.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a297fa640 1 -- 172.21.15.103:0/3985219793 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f0a30007ca0 con 0x560bc215bb40 2024-09-19T06:53:44.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a49e6a740 1 -- 172.21.15.103:0/3985219793 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a3c006ba0 con 0x560bc215bb40 2024-09-19T06:53:44.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a297fa640 1 -- 172.21.15.103:0/3985219793 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a300d7040 con 0x560bc215bb40 2024-09-19T06:53:44.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a49e6a740 10 client.17046 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:44.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a49e6a740 1 -- 172.21.15.103:0/3985219793 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560bc21e36d0 con 0x560bc215bb40 2024-09-19T06:53:44.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.138+0000 7f0a297fa640 1 -- 172.21.15.103:0/3985219793 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3c006ba0 con 0x560bc215bb40 2024-09-19T06:53:44.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.139+0000 7f0a297fa640 1 -- 172.21.15.103:0/3985219793 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0a300cd530 con 0x560bc215bb40 2024-09-19T06:53:44.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.139+0000 7f0a49e6a740 10 client.17046 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:44.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.139+0000 7f0a49e6a740 10 client.17046 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:53:44.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.139+0000 7f0a49e6a740 20 client.17046 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:44.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.139+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/3985219793 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x560bc21e3ed0 0x560bc21e62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:44.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.139+0000 7f0a49e6a740 4 client.17046 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:53:44.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.139+0000 7f0a49e6a740 1 -- 172.21.15.103:0/3985219793 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560bc20c7d70 con 0x560bc21e3ed0 2024-09-19T06:53:44.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.139+0000 7f0a4582d640 1 --2- 172.21.15.103:0/3985219793 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x560bc21e3ed0 0x560bc21e62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:44.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.139+0000 7f0a4582d640 1 --2- 172.21.15.103:0/3985219793 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x560bc21e3ed0 0x560bc21e62c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8f3c61e34dcb1d1 server_cookie=8874d80fbc58ba7e in_seq=0 out_seq=0 2024-09-19T06:53:44.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.139+0000 7f0a297fa640 10 client.17046 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:53:44.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.139+0000 7f0a297fa640 1 -- 172.21.15.103:0/3985219793 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560bc20c7d70 con 0x560bc21e3ed0 2024-09-19T06:53:44.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.139+0000 7f0a297fa640 10 client.17046 handle_command_reply: tid=0 2024-09-19T06:53:44.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.139+0000 7f0a49e6a740 1 client.17046 shutdown 2024-09-19T06:53:44.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a49e6a740 1 -- 172.21.15.103:0/3985219793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215bb40 msgr2=0x560bc2172020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:44.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/3985219793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215bb40 0x560bc2172020 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f0a30005fd0 tx=0x7f0a300054a0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a49e6a740 1 -- 172.21.15.103:0/3985219793 shutdown_connections 2024-09-19T06:53:44.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/3985219793 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x560bc21e3ed0 0x560bc21e62c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/3985219793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bc2169780 0x560bc2162c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/3985219793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bc215bb40 0x560bc2172020 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a297fa640 0 client.17046 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:53:44.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a49e6a740 1 --2- 172.21.15.103:0/3985219793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bc215a5d0 0x560bc2171ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a49e6a740 1 -- 172.21.15.103:0/3985219793 >> 172.21.15.103:0/3985219793 conn(0x560bc2123310 msgr2=0x560bc2168410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:44.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a49e6a740 1 -- 172.21.15.103:0/3985219793 shutdown_connections 2024-09-19T06:53:44.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a49e6a740 1 -- 172.21.15.103:0/3985219793 wait complete. 2024-09-19T06:53:44.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a49e6a740 20 client.17046 trim_cache size 0 max 16384 2024-09-19T06:53:44.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a47a91640 1 -- 172.21.15.103:0/3030379677 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0a2c0a33b0 msgr2=0x7f0a2c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:44.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a47a91640 1 --2- 172.21.15.103:0/3030379677 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0a2c0a33b0 0x7f0a2c0a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009430 tx=0x7f0a3803e040 comp rx=0 tx=0).stop 2024-09-19T06:53:44.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a47a91640 1 -- 172.21.15.103:0/3030379677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a40101430 msgr2=0x7f0a401989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:44.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a47a91640 1 --2- 172.21.15.103:0/3030379677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a40101430 0x7f0a401989d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000c890 tx=0x7f0a3000cd50 comp rx=0 tx=0).stop 2024-09-19T06:53:44.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a47a91640 1 -- 172.21.15.103:0/3030379677 shutdown_connections 2024-09-19T06:53:44.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a47a91640 1 --2- 172.21.15.103:0/3030379677 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0a2c0a33b0 0x7f0a2c0a5870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a47a91640 1 --2- 172.21.15.103:0/3030379677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a40106a90 0x7f0a4019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a47a91640 1 --2- 172.21.15.103:0/3030379677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a40101430 0x7f0a401989d0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a47a91640 1 --2- 172.21.15.103:0/3030379677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a40100a90 0x7f0a40198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:44.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.140+0000 7f0a47a91640 1 -- 172.21.15.103:0/3030379677 >> 172.21.15.103:0/3030379677 conn(0x7f0a400fc7d0 msgr2=0x7f0a4006a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:44.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.141+0000 7f0a47a91640 1 -- 172.21.15.103:0/3030379677 shutdown_connections 2024-09-19T06:53:44.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:44.141+0000 7f0a47a91640 1 -- 172.21.15.103:0/3030379677 wait complete. 2024-09-19T06:53:44.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:44 smithi174 ceph-mon[35637]: pgmap v138: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-19T06:53:44.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:44 smithi103 ceph-mon[31158]: pgmap v138: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-19T06:53:44.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:44 smithi134 ceph-mon[35356]: pgmap v138: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-19T06:53:44.865 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:53:45.113 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:45.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.555+0000 7f6c7b278640 1 -- 172.21.15.103:0/1648205596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c74102790 msgr2=0x7f6c741067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:45.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.555+0000 7f6c7b278640 1 --2- 172.21.15.103:0/1648205596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c74102790 0x7f6c741067e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009900 tx=0x7f6c6402a410 comp rx=0 tx=0).stop 2024-09-19T06:53:45.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.555+0000 7f6c7b278640 1 -- 172.21.15.103:0/1648205596 shutdown_connections 2024-09-19T06:53:45.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.555+0000 7f6c7b278640 1 --2- 172.21.15.103:0/1648205596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c74106f10 0x7f6c7410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.555+0000 7f6c7b278640 1 --2- 172.21.15.103:0/1648205596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c74102790 0x7f6c741067e0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.555+0000 7f6c7b278640 1 --2- 172.21.15.103:0/1648205596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c74101de0 0x7f6c741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.555+0000 7f6c7b278640 1 -- 172.21.15.103:0/1648205596 >> 172.21.15.103:0/1648205596 conn(0x7f6c740fd8c0 msgr2=0x7f6c740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:45.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.555+0000 7f6c7b278640 1 -- 172.21.15.103:0/1648205596 shutdown_connections 2024-09-19T06:53:45.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.555+0000 7f6c7b278640 1 -- 172.21.15.103:0/1648205596 wait complete. 2024-09-19T06:53:45.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.556+0000 7f6c7b278640 1 Processor -- start 2024-09-19T06:53:45.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.556+0000 7f6c7b278640 1 -- start start 2024-09-19T06:53:45.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.556+0000 7f6c7b278640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c74101de0 0x7f6c740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.556+0000 7f6c7b278640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c74102790 0x7f6c74074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.556+0000 7f6c7b278640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c74106f10 0x7f6c74074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.556+0000 7f6c7b278640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74112010 con 0x7f6c74106f10 2024-09-19T06:53:45.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.556+0000 7f6c7b278640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74111e90 con 0x7f6c74101de0 2024-09-19T06:53:45.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.556+0000 7f6c7b278640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74112190 con 0x7f6c74102790 2024-09-19T06:53:45.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.556+0000 7f6c79815640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c74106f10 0x7f6c74074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.556+0000 7f6c79815640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c74106f10 0x7f6c74074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56966/0 (socket says 172.21.15.103:56966) 2024-09-19T06:53:45.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.556+0000 7f6c79815640 1 -- 172.21.15.103:0/209547647 learned_addr learned my addr 172.21.15.103:0/209547647 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:45.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.556+0000 7f6c79014640 1 --2- 172.21.15.103:0/209547647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c74101de0 0x7f6c740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.556+0000 7f6c78813640 1 --2- 172.21.15.103:0/209547647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c74102790 0x7f6c74074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.556+0000 7f6c79815640 1 -- 172.21.15.103:0/209547647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c74102790 msgr2=0x7f6c74074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:45.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.556+0000 7f6c79815640 1 --2- 172.21.15.103:0/209547647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c74102790 0x7f6c74074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.557+0000 7f6c79815640 1 -- 172.21.15.103:0/209547647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c74101de0 msgr2=0x7f6c740744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:45.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.557+0000 7f6c79815640 1 --2- 172.21.15.103:0/209547647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c74101de0 0x7f6c740744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.557+0000 7f6c79815640 1 -- 172.21.15.103:0/209547647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c741a6a70 con 0x7f6c74106f10 2024-09-19T06:53:45.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.557+0000 7f6c79014640 1 --2- 172.21.15.103:0/209547647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c74101de0 0x7f6c740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:45.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.557+0000 7f6c78813640 1 --2- 172.21.15.103:0/209547647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c74102790 0x7f6c74074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:45.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.557+0000 7f6c79815640 1 --2- 172.21.15.103:0/209547647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c74106f10 0x7f6c74074f70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00b970 tx=0x7f6c6c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:45.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.557+0000 7f6c6a7fc640 1 -- 172.21.15.103:0/209547647 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c019070 con 0x7f6c74106f10 2024-09-19T06:53:45.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.557+0000 7f6c6a7fc640 1 -- 172.21.15.103:0/209547647 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c00cb40 con 0x7f6c74106f10 2024-09-19T06:53:45.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.557+0000 7f6c7b278640 1 -- 172.21.15.103:0/209547647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c741a6d60 con 0x7f6c74106f10 2024-09-19T06:53:45.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.557+0000 7f6c6a7fc640 1 -- 172.21.15.103:0/209547647 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c007650 con 0x7f6c74106f10 2024-09-19T06:53:45.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.557+0000 7f6c7b278640 1 -- 172.21.15.103:0/209547647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c741a72a0 con 0x7f6c74106f10 2024-09-19T06:53:45.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.559+0000 7f6c6a7fc640 1 -- 172.21.15.103:0/209547647 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f6c6c01c070 con 0x7f6c74106f10 2024-09-19T06:53:45.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.560+0000 7f6c6a7fc640 1 --2- 172.21.15.103:0/209547647 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6c600a33b0 0x7f6c600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.560+0000 7f6c7d651740 1 Processor -- start 2024-09-19T06:53:45.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.560+0000 7f6c7d651740 1 -- start start 2024-09-19T06:53:45.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.560+0000 7f6c79014640 1 --2- 172.21.15.103:0/209547647 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6c600a33b0 0x7f6c600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.560+0000 7f6c6a7fc640 1 -- 172.21.15.103:0/209547647 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6c6c0cf1f0 con 0x7f6c74106f10 2024-09-19T06:53:45.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.560+0000 7f6c7d651740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be7029b40 0x558be7028090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.560+0000 7f6c7d651740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be70285d0 0x558be7028a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.560+0000 7f6c7d651740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be7033370 0x558be7035760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.560+0000 7f6c7d651740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558be6f8e2f0 con 0x558be70285d0 2024-09-19T06:53:45.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.560+0000 7f6c7d651740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558be6f35cf0 con 0x558be7033370 2024-09-19T06:53:45.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.560+0000 7f6c7d651740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558be6f8a2b0 con 0x558be7029b40 2024-09-19T06:53:45.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.561+0000 7f6c78813640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be70285d0 0x558be7028a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.561+0000 7f6c78813640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be70285d0 0x558be7028a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56974/0 (socket says 172.21.15.103:56974) 2024-09-19T06:53:45.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.561+0000 7f6c78813640 1 -- 172.21.15.103:0/1060461508 learned_addr learned my addr 172.21.15.103:0/1060461508 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:45.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.561+0000 7f6c79815640 1 --2- 172.21.15.103:0/1060461508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be7033370 0x558be7035760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.561+0000 7f6c79014640 1 --2- 172.21.15.103:0/1060461508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be7029b40 0x558be7028090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.561+0000 7f6c79014640 1 --2- 172.21.15.103:0/209547647 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6c600a33b0 0x7f6c600a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c70006fd0 tx=0x7f6c70008040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:45.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.561+0000 7f6c78813640 1 -- 172.21.15.103:0/1060461508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be7029b40 msgr2=0x558be7028090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:45.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.561+0000 7f6c78813640 1 --2- 172.21.15.103:0/1060461508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be7029b40 0x558be7028090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.561+0000 7f6c78813640 1 -- 172.21.15.103:0/1060461508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be7033370 msgr2=0x558be7035760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:45.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.561+0000 7f6c78813640 1 --2- 172.21.15.103:0/1060461508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be7033370 0x558be7035760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.561+0000 7f6c78813640 1 -- 172.21.15.103:0/1060461508 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be6f1a3a0 con 0x558be70285d0 2024-09-19T06:53:45.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.561+0000 7f6c78813640 1 --2- 172.21.15.103:0/1060461508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be70285d0 0x558be7028a10 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f6c64005ab0 tx=0x7f6c64009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba3840849d4f25f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:45.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.561+0000 7f6c5f7fe640 1 -- 172.21.15.103:0/1060461508 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6403f070 con 0x558be70285d0 2024-09-19T06:53:45.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.561+0000 7f6c5f7fe640 1 -- 172.21.15.103:0/1060461508 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c640317b0 con 0x558be70285d0 2024-09-19T06:53:45.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.561+0000 7f6c5f7fe640 1 -- 172.21.15.103:0/1060461508 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64035070 con 0x558be70285d0 2024-09-19T06:53:45.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.562+0000 7f6c7d651740 1 -- 172.21.15.103:0/1060461508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be70285d0 msgr2=0x558be7028a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:45.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.562+0000 7f6c7d651740 1 --2- 172.21.15.103:0/1060461508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be70285d0 0x558be7028a10 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f6c64005ab0 tx=0x7f6c64009680 comp rx=0 tx=0).stop 2024-09-19T06:53:45.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.562+0000 7f6c7d651740 1 -- 172.21.15.103:0/1060461508 shutdown_connections 2024-09-19T06:53:45.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.562+0000 7f6c7d651740 1 --2- 172.21.15.103:0/1060461508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be7033370 0x558be7035760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.562+0000 7f6c7d651740 1 --2- 172.21.15.103:0/1060461508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be70285d0 0x558be7028a10 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.562+0000 7f6c7d651740 1 --2- 172.21.15.103:0/1060461508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be7029b40 0x558be7028090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.562+0000 7f6c7d651740 1 -- 172.21.15.103:0/1060461508 >> 172.21.15.103:0/1060461508 conn(0x558be6ff1310 msgr2=0x558be7024c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:45.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.562+0000 7f6c7d651740 1 -- 172.21.15.103:0/1060461508 shutdown_connections 2024-09-19T06:53:45.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.562+0000 7f6c7d651740 1 -- 172.21.15.103:0/1060461508 wait complete. 2024-09-19T06:53:45.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.562+0000 7f6c7d651740 1 -- start start 2024-09-19T06:53:45.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c7d651740 1 Processor -- start 2024-09-19T06:53:45.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c7d651740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be70285d0 0x558be7032cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c7d651740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7029b40 0x558be702ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c7d651740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be7033370 0x558be70304e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c7d651740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558be7036010 con 0x558be7029b40 2024-09-19T06:53:45.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c7d651740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558be6f49060 con 0x558be70285d0 2024-09-19T06:53:45.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c7d651740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558be7036190 con 0x558be7033370 2024-09-19T06:53:45.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c7d651740 10 client.0 _ll_register_callbacks cb 0x558be6c9c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:45.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c78813640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7029b40 0x558be702ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c78813640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7029b40 0x558be702ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56982/0 (socket says 172.21.15.103:56982) 2024-09-19T06:53:45.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c78813640 1 -- 172.21.15.103:0/1823849916 learned_addr learned my addr 172.21.15.103:0/1823849916 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:45.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c79014640 1 --2- 172.21.15.103:0/1823849916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be70285d0 0x558be7032cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c79815640 1 --2- 172.21.15.103:0/1823849916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be7033370 0x558be70304e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c78813640 1 -- 172.21.15.103:0/1823849916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be7033370 msgr2=0x558be70304e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:45.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c78813640 1 --2- 172.21.15.103:0/1823849916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be7033370 0x558be70304e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c78813640 1 -- 172.21.15.103:0/1823849916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be70285d0 msgr2=0x558be7032cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:45.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c78813640 1 --2- 172.21.15.103:0/1823849916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be70285d0 0x558be7032cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.563+0000 7f6c78813640 1 -- 172.21.15.103:0/1823849916 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be6cff3c0 con 0x558be7029b40 2024-09-19T06:53:45.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.564+0000 7f6c78813640 1 --2- 172.21.15.103:0/1823849916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7029b40 0x558be702ff80 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009900 tx=0x7f6c6402aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f2ec2753123919 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:45.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.564+0000 7f6c5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:45.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.564+0000 7f6c5cff9640 1 -- 172.21.15.103:0/1823849916 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6403f050 con 0x558be7029b40 2024-09-19T06:53:45.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.564+0000 7f6c5cff9640 1 -- 172.21.15.103:0/1823849916 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6c64004240 con 0x558be7029b40 2024-09-19T06:53:45.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.564+0000 7f6c7d651740 1 -- 172.21.15.103:0/1823849916 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c640317b0 con 0x558be7029b40 2024-09-19T06:53:45.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.564+0000 7f6c5cff9640 1 -- 172.21.15.103:0/1823849916 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c64035040 con 0x558be7029b40 2024-09-19T06:53:45.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.564+0000 7f6c7d651740 10 client.17058 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:45.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.564+0000 7f6c7d651740 1 -- 172.21.15.103:0/1823849916 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558be7041a20 con 0x558be7029b40 2024-09-19T06:53:45.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.564+0000 7f6c5cff9640 1 -- 172.21.15.103:0/1823849916 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c640317b0 con 0x558be7029b40 2024-09-19T06:53:45.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.564+0000 7f6c5cff9640 1 -- 172.21.15.103:0/1823849916 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6c640047e0 con 0x558be7029b40 2024-09-19T06:53:45.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.564+0000 7f6c7d651740 10 client.17058 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:45.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.564+0000 7f6c7d651740 10 client.17058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:45.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.564+0000 7f6c7d651740 20 client.17058 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:45.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.564+0000 7f6c7d651740 1 --2- 172.21.15.103:0/1823849916 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558be7042d70 0x558be7045160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.564+0000 7f6c7d651740 4 client.17058 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:45.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.564+0000 7f6c7d651740 1 -- 172.21.15.103:0/1823849916 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558be6f9b860 con 0x558be7042d70 2024-09-19T06:53:45.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.565+0000 7f6c79014640 1 --2- 172.21.15.103:0/1823849916 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558be7042d70 0x558be7045160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.565+0000 7f6c79014640 1 --2- 172.21.15.103:0/1823849916 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558be7042d70 0x558be7045160 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfc29dc72501d61f server_cookie=fcb3e7091b3e7a98 in_seq=0 out_seq=0 2024-09-19T06:53:45.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.565+0000 7f6c5cff9640 10 client.17058 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:45.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.566+0000 7f6c5cff9640 1 -- 172.21.15.103:0/1823849916 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558be6f9b860 con 0x558be7042d70 2024-09-19T06:53:45.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.566+0000 7f6c5cff9640 10 client.17058 handle_command_reply: tid=0 2024-09-19T06:53:45.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.566+0000 7f6c7d651740 1 client.17058 shutdown 2024-09-19T06:53:45.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.566+0000 7f6c7d651740 1 -- 172.21.15.103:0/1823849916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7029b40 msgr2=0x558be702ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:45.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.566+0000 7f6c7d651740 1 --2- 172.21.15.103:0/1823849916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7029b40 0x558be702ff80 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009900 tx=0x7f6c6402aa90 comp rx=0 tx=0).stop 2024-09-19T06:53:45.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.566+0000 7f6c7d651740 1 -- 172.21.15.103:0/1823849916 shutdown_connections 2024-09-19T06:53:45.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.566+0000 7f6c7d651740 1 --2- 172.21.15.103:0/1823849916 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558be7042d70 0x558be7045160 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.566+0000 7f6c7d651740 1 --2- 172.21.15.103:0/1823849916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be7033370 0x558be70304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.566+0000 7f6c7d651740 1 --2- 172.21.15.103:0/1823849916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7029b40 0x558be702ff80 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.566+0000 7f6c7d651740 1 --2- 172.21.15.103:0/1823849916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be70285d0 0x558be7032cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.566+0000 7f6c5cff9640 0 client.17058 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:45.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.566+0000 7f6c7d651740 1 -- 172.21.15.103:0/1823849916 >> 172.21.15.103:0/1823849916 conn(0x558be6ff1310 msgr2=0x558be702a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:45.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.566+0000 7f6c7d651740 1 -- 172.21.15.103:0/1823849916 shutdown_connections 2024-09-19T06:53:45.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.566+0000 7f6c7d651740 1 -- 172.21.15.103:0/1823849916 wait complete. 2024-09-19T06:53:45.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.566+0000 7f6c7d651740 20 client.17058 trim_cache size 0 max 16384 2024-09-19T06:53:45.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c7d651740 1 Processor -- start 2024-09-19T06:53:45.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c7d651740 1 -- start start 2024-09-19T06:53:45.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c7d651740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be70285d0 0x558be7034760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c7d651740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be7029b40 0x558be7034ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c7d651740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7033370 0x558be702f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c7d651740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c741afbe0 con 0x558be7033370 2024-09-19T06:53:45.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c7d651740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c741b1720 con 0x558be7029b40 2024-09-19T06:53:45.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c7d651740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6c741b02b0 con 0x558be70285d0 2024-09-19T06:53:45.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c79815640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7033370 0x558be702f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c79815640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7033370 0x558be702f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56996/0 (socket says 172.21.15.103:56996) 2024-09-19T06:53:45.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c79815640 1 -- 172.21.15.103:0/1877039261 learned_addr learned my addr 172.21.15.103:0/1877039261 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:45.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c78813640 1 --2- 172.21.15.103:0/1877039261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be7029b40 0x558be7034ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c79014640 1 --2- 172.21.15.103:0/1877039261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be70285d0 0x558be7034760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c79815640 1 -- 172.21.15.103:0/1877039261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be70285d0 msgr2=0x558be7034760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:45.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c79815640 1 --2- 172.21.15.103:0/1877039261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be70285d0 0x558be7034760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c79815640 1 -- 172.21.15.103:0/1877039261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be7029b40 msgr2=0x558be7034ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:45.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c79815640 1 --2- 172.21.15.103:0/1877039261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be7029b40 0x558be7034ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.579+0000 7f6c79815640 1 -- 172.21.15.103:0/1877039261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be7025960 con 0x558be7033370 2024-09-19T06:53:45.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.580+0000 7f6c79815640 1 --2- 172.21.15.103:0/1877039261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7033370 0x558be702f840 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c097fc0 tx=0x7f6c6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d55cc3e2251c7c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:45.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.580+0000 7f6c5effd640 1 -- 172.21.15.103:0/1877039261 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c09b5f0 con 0x558be7033370 2024-09-19T06:53:45.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.580+0000 7f6c5effd640 1 -- 172.21.15.103:0/1877039261 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c09f9a0 con 0x558be7033370 2024-09-19T06:53:45.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.580+0000 7f6c5effd640 1 -- 172.21.15.103:0/1877039261 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c012340 con 0x558be7033370 2024-09-19T06:53:45.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 -- 172.21.15.103:0/1877039261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7033370 msgr2=0x558be702f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:45.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 --2- 172.21.15.103:0/1877039261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7033370 0x558be702f840 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c097fc0 tx=0x7f6c6c004a40 comp rx=0 tx=0).stop 2024-09-19T06:53:45.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 -- 172.21.15.103:0/1877039261 shutdown_connections 2024-09-19T06:53:45.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 --2- 172.21.15.103:0/1877039261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7033370 0x558be702f840 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 --2- 172.21.15.103:0/1877039261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be7029b40 0x558be7034ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 --2- 172.21.15.103:0/1877039261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be70285d0 0x558be7034760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 -- 172.21.15.103:0/1877039261 >> 172.21.15.103:0/1877039261 conn(0x558be6ff1310 msgr2=0x558be702e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:45.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 -- 172.21.15.103:0/1877039261 shutdown_connections 2024-09-19T06:53:45.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 -- 172.21.15.103:0/1877039261 wait complete. 2024-09-19T06:53:45.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 -- start start 2024-09-19T06:53:45.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 Processor -- start 2024-09-19T06:53:45.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be70285d0 0x558be70a5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be7029b40 0x558be70a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7033370 0x558be70436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c741b1050 con 0x558be7033370 2024-09-19T06:53:45.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c741b0980 con 0x558be7029b40 2024-09-19T06:53:45.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6c741adba0 con 0x558be70285d0 2024-09-19T06:53:45.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.581+0000 7f6c7d651740 10 client.0 _ll_register_callbacks cb 0x7f6c6c014c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:45.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c79815640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7033370 0x558be70436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c79815640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7033370 0x558be70436f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57000/0 (socket says 172.21.15.103:57000) 2024-09-19T06:53:45.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c79815640 1 -- 172.21.15.103:0/2248415679 learned_addr learned my addr 172.21.15.103:0/2248415679 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:45.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c79014640 1 --2- 172.21.15.103:0/2248415679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be70285d0 0x558be70a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c78813640 1 --2- 172.21.15.103:0/2248415679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be7029b40 0x558be70a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c79815640 1 -- 172.21.15.103:0/2248415679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be70285d0 msgr2=0x558be70a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:45.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c79815640 1 --2- 172.21.15.103:0/2248415679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be70285d0 0x558be70a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c79815640 1 -- 172.21.15.103:0/2248415679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be7029b40 msgr2=0x558be70a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:45.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c79815640 1 --2- 172.21.15.103:0/2248415679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be7029b40 0x558be70a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c79815640 1 -- 172.21.15.103:0/2248415679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be7043f20 con 0x558be7033370 2024-09-19T06:53:45.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c78813640 1 --2- 172.21.15.103:0/2248415679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be7029b40 0x558be70a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:45.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c79815640 1 --2- 172.21.15.103:0/2248415679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7033370 0x558be70436f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0a3d30 tx=0x7f6c6c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e0b2231a1cb357 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:45.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:45.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c5cff9640 1 -- 172.21.15.103:0/2248415679 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c09b5f0 con 0x558be7033370 2024-09-19T06:53:45.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c5cff9640 1 -- 172.21.15.103:0/2248415679 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6c6c0998f0 con 0x558be7033370 2024-09-19T06:53:45.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c7d651740 1 -- 172.21.15.103:0/2248415679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c6c09f9a0 con 0x558be7033370 2024-09-19T06:53:45.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c5cff9640 1 -- 172.21.15.103:0/2248415679 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c6c0a4260 con 0x558be7033370 2024-09-19T06:53:45.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c7d651740 10 client.17064 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:45.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.582+0000 7f6c7d651740 1 -- 172.21.15.103:0/2248415679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558be70ad1a0 con 0x558be7033370 2024-09-19T06:53:45.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.583+0000 7f6c5cff9640 1 -- 172.21.15.103:0/2248415679 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c09f9a0 con 0x558be7033370 2024-09-19T06:53:45.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.583+0000 7f6c5cff9640 1 -- 172.21.15.103:0/2248415679 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6c6c0afb10 con 0x558be7033370 2024-09-19T06:53:45.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.583+0000 7f6c7d651740 10 client.17064 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:45.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.583+0000 7f6c7d651740 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:45.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.583+0000 7f6c7d651740 20 client.17064 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:45.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.583+0000 7f6c7d651740 1 --2- 172.21.15.103:0/2248415679 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558be70ad8c0 0x558be70afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:45.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.583+0000 7f6c7d651740 4 client.17064 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:53:45.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.583+0000 7f6c7d651740 1 -- 172.21.15.103:0/2248415679 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558be6f95d70 con 0x558be70ad8c0 2024-09-19T06:53:45.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.583+0000 7f6c79014640 1 --2- 172.21.15.103:0/2248415679 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558be70ad8c0 0x558be70afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:45.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c5cff9640 10 client.17064 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:45.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c79014640 1 --2- 172.21.15.103:0/2248415679 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558be70ad8c0 0x558be70afcb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fab0b218e23a1982 server_cookie=9e677b713e173156 in_seq=0 out_seq=0 2024-09-19T06:53:45.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c5cff9640 1 -- 172.21.15.103:0/2248415679 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558be6f95d70 con 0x558be70ad8c0 2024-09-19T06:53:45.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c5cff9640 10 client.17064 handle_command_reply: tid=0 2024-09-19T06:53:45.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c7d651740 1 client.17064 shutdown 2024-09-19T06:53:45.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c7d651740 1 -- 172.21.15.103:0/2248415679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7033370 msgr2=0x558be70436f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:45.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c7d651740 1 --2- 172.21.15.103:0/2248415679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7033370 0x558be70436f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0a3d30 tx=0x7f6c6c004b50 comp rx=0 tx=0).stop 2024-09-19T06:53:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c7d651740 1 -- 172.21.15.103:0/2248415679 shutdown_connections 2024-09-19T06:53:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c7d651740 1 --2- 172.21.15.103:0/2248415679 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558be70ad8c0 0x558be70afcb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c7d651740 1 --2- 172.21.15.103:0/2248415679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be7033370 0x558be70436f0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c7d651740 1 --2- 172.21.15.103:0/2248415679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be7029b40 0x558be70a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c5cff9640 0 client.17064 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c7d651740 1 --2- 172.21.15.103:0/2248415679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558be70285d0 0x558be70a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c7d651740 1 -- 172.21.15.103:0/2248415679 >> 172.21.15.103:0/2248415679 conn(0x558be6ff1310 msgr2=0x558be702e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c7d651740 1 -- 172.21.15.103:0/2248415679 shutdown_connections 2024-09-19T06:53:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c7d651740 1 -- 172.21.15.103:0/2248415679 wait complete. 2024-09-19T06:53:45.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.584+0000 7f6c7d651740 20 client.17064 trim_cache size 0 max 16384 2024-09-19T06:53:45.595 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:53:45.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.585+0000 7f6c7b278640 1 -- 172.21.15.103:0/209547647 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6c600a33b0 msgr2=0x7f6c600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:45.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.585+0000 7f6c7b278640 1 --2- 172.21.15.103:0/209547647 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6c600a33b0 0x7f6c600a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c70006fd0 tx=0x7f6c70008040 comp rx=0 tx=0).stop 2024-09-19T06:53:45.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.585+0000 7f6c7b278640 1 -- 172.21.15.103:0/209547647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c74106f10 msgr2=0x7f6c74074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:45.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.585+0000 7f6c7b278640 1 --2- 172.21.15.103:0/209547647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c74106f10 0x7f6c74074f70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00b970 tx=0x7f6c6c00be30 comp rx=0 tx=0).stop 2024-09-19T06:53:45.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.585+0000 7f6c7b278640 1 -- 172.21.15.103:0/209547647 shutdown_connections 2024-09-19T06:53:45.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.585+0000 7f6c7b278640 1 --2- 172.21.15.103:0/209547647 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6c600a33b0 0x7f6c600a5870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.585+0000 7f6c7b278640 1 --2- 172.21.15.103:0/209547647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c74106f10 0x7f6c74074f70 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.585+0000 7f6c7b278640 1 --2- 172.21.15.103:0/209547647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c74102790 0x7f6c74074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.585+0000 7f6c7b278640 1 --2- 172.21.15.103:0/209547647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c74101de0 0x7f6c740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:45.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.585+0000 7f6c7b278640 1 -- 172.21.15.103:0/209547647 >> 172.21.15.103:0/209547647 conn(0x7f6c740fd8c0 msgr2=0x7f6c740fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:45.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.585+0000 7f6c7b278640 1 -- 172.21.15.103:0/209547647 shutdown_connections 2024-09-19T06:53:45.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:45.585+0000 7f6c7b278640 1 -- 172.21.15.103:0/209547647 wait complete. 2024-09-19T06:53:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:46 smithi174 ceph-mon[35637]: pgmap v139: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-09-19T06:53:46.341 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:46 smithi103 ceph-mon[31158]: pgmap v139: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-09-19T06:53:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:46 smithi134 ceph-mon[35356]: pgmap v139: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-09-19T06:53:47.343 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:53:47.344 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:53:47.604 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.090+0000 7f19a3d49640 1 -- 172.21.15.103:0/1073828319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f199c1069e0 msgr2=0x7f199c106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.090+0000 7f19a3d49640 1 --2- 172.21.15.103:0/1073828319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f199c1069e0 0x7f199c106dc0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f1998009930 tx=0x7f199802a420 comp rx=0 tx=0).stop 2024-09-19T06:53:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.090+0000 7f19a3d49640 1 -- 172.21.15.103:0/1073828319 shutdown_connections 2024-09-19T06:53:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.090+0000 7f19a3d49640 1 --2- 172.21.15.103:0/1073828319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f199c101370 0x7f199c10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.090+0000 7f19a3d49640 1 --2- 172.21.15.103:0/1073828319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f199c1009d0 0x7f199c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.090+0000 7f19a3d49640 1 --2- 172.21.15.103:0/1073828319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f199c1069e0 0x7f199c106dc0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.090+0000 7f19a3d49640 1 -- 172.21.15.103:0/1073828319 >> 172.21.15.103:0/1073828319 conn(0x7f199c0fc6b0 msgr2=0x7f199c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.090+0000 7f19a3d49640 1 -- 172.21.15.103:0/1073828319 shutdown_connections 2024-09-19T06:53:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.090+0000 7f19a3d49640 1 -- 172.21.15.103:0/1073828319 wait complete. 2024-09-19T06:53:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.090+0000 7f19a3d49640 1 Processor -- start 2024-09-19T06:53:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.090+0000 7f19a3d49640 1 -- start start 2024-09-19T06:53:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a3d49640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f199c1009d0 0x7f199c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a3d49640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f199c101370 0x7f199c1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a3d49640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f199c1069e0 0x7f199c19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a3d49640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f199c110e20 con 0x7f199c1069e0 2024-09-19T06:53:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a3d49640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f199c110ca0 con 0x7f199c1009d0 2024-09-19T06:53:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a3d49640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f199c110fa0 con 0x7f199c101370 2024-09-19T06:53:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a22e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f199c1069e0 0x7f199c19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a22e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f199c1069e0 0x7f199c19bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40868/0 (socket says 172.21.15.103:40868) 2024-09-19T06:53:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a22e6640 1 -- 172.21.15.103:0/867782465 learned_addr learned my addr 172.21.15.103:0/867782465 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a1ae5640 1 --2- 172.21.15.103:0/867782465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f199c1009d0 0x7f199c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a12e4640 1 --2- 172.21.15.103:0/867782465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f199c101370 0x7f199c1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a22e6640 1 -- 172.21.15.103:0/867782465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f199c101370 msgr2=0x7f199c1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a22e6640 1 --2- 172.21.15.103:0/867782465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f199c101370 0x7f199c1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a22e6640 1 -- 172.21.15.103:0/867782465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f199c1009d0 msgr2=0x7f199c1972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a22e6640 1 --2- 172.21.15.103:0/867782465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f199c1009d0 0x7f199c1972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a22e6640 1 -- 172.21.15.103:0/867782465 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199c19c2a0 con 0x7f199c1069e0 2024-09-19T06:53:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a22e6640 1 --2- 172.21.15.103:0/867782465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f199c1069e0 0x7f199c19bb70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f199400eca0 tx=0x7f199400b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f1992ffd640 1 -- 172.21.15.103:0/867782465 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994004b20 con 0x7f199c1069e0 2024-09-19T06:53:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f1992ffd640 1 -- 172.21.15.103:0/867782465 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1994004cc0 con 0x7f199c1069e0 2024-09-19T06:53:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f19a3d49640 1 -- 172.21.15.103:0/867782465 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199c19c590 con 0x7f199c1069e0 2024-09-19T06:53:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.091+0000 7f1992ffd640 1 -- 172.21.15.103:0/867782465 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994017650 con 0x7f199c1069e0 2024-09-19T06:53:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.092+0000 7f19a3d49640 1 -- 172.21.15.103:0/867782465 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199c06a0c0 con 0x7f199c1069e0 2024-09-19T06:53:48.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.073+0000 7f5145244640 1 -- 172.21.15.103:0/1165916339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5140102570 msgr2=0x7f51401029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.073+0000 7f5145244640 1 --2- 172.21.15.103:0/1165916339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5140102570 0x7f51401029b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f5134009900 tx=0x7f513402a480 comp rx=0 tx=0).stop 2024-09-19T06:53:48.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.073+0000 7f5145244640 1 -- 172.21.15.103:0/1165916339 shutdown_connections 2024-09-19T06:53:48.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.073+0000 7f5145244640 1 --2- 172.21.15.103:0/1165916339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5140102ef0 0x7f514010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.073+0000 7f5145244640 1 --2- 172.21.15.103:0/1165916339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5140102570 0x7f51401029b0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.073+0000 7f5145244640 1 --2- 172.21.15.103:0/1165916339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5140100e20 0x7f5140101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.073+0000 7f5145244640 1 -- 172.21.15.103:0/1165916339 >> 172.21.15.103:0/1165916339 conn(0x7f51400fc7d0 msgr2=0x7f51400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:48.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.073+0000 7f5145244640 1 -- 172.21.15.103:0/1165916339 shutdown_connections 2024-09-19T06:53:48.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.073+0000 7f5145244640 1 -- 172.21.15.103:0/1165916339 wait complete. 2024-09-19T06:53:48.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.074+0000 7f5145244640 1 Processor -- start 2024-09-19T06:53:48.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.074+0000 7f5145244640 1 -- start start 2024-09-19T06:53:48.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.074+0000 7f5145244640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5140100e20 0x7f5140199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f5145244640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5140102570 0x7f5140199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f5145244640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5140102ef0 0x7f51401a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f5145244640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5140110f80 con 0x7f5140102ef0 2024-09-19T06:53:48.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f5145244640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5140110e00 con 0x7f5140102570 2024-09-19T06:53:48.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f5145244640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5140111100 con 0x7f5140100e20 2024-09-19T06:53:48.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f513f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5140102ef0 0x7f51401a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f513f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5140102ef0 0x7f51401a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40830/0 (socket says 172.21.15.103:40830) 2024-09-19T06:53:48.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f513f59e640 1 -- 172.21.15.103:0/1488874751 learned_addr learned my addr 172.21.15.103:0/1488874751 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:48.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f513e59c640 1 --2- 172.21.15.103:0/1488874751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5140102570 0x7f5140199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f513ed9d640 1 --2- 172.21.15.103:0/1488874751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5140100e20 0x7f5140199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f513f59e640 1 -- 172.21.15.103:0/1488874751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5140100e20 msgr2=0x7f5140199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f513f59e640 1 --2- 172.21.15.103:0/1488874751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5140100e20 0x7f5140199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f513f59e640 1 -- 172.21.15.103:0/1488874751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5140102570 msgr2=0x7f5140199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f513f59e640 1 --2- 172.21.15.103:0/1488874751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5140102570 0x7f5140199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f513f59e640 1 -- 172.21.15.103:0/1488874751 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51401a4910 con 0x7f5140102ef0 2024-09-19T06:53:48.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f513ed9d640 1 --2- 172.21.15.103:0/1488874751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5140100e20 0x7f5140199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:48.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f513f59e640 1 --2- 172.21.15.103:0/1488874751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5140102ef0 0x7f51401a43d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f513800eaa0 tx=0x7f513800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:48.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f5127fff640 1 -- 172.21.15.103:0/1488874751 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513800cdb0 con 0x7f5140102ef0 2024-09-19T06:53:48.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f5145244640 1 -- 172.21.15.103:0/1488874751 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5140197140 con 0x7f5140102ef0 2024-09-19T06:53:48.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f5127fff640 1 -- 172.21.15.103:0/1488874751 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5138005020 con 0x7f5140102ef0 2024-09-19T06:53:48.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.075+0000 7f5127fff640 1 -- 172.21.15.103:0/1488874751 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51380116b0 con 0x7f5140102ef0 2024-09-19T06:53:48.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.076+0000 7f5145244640 1 -- 172.21.15.103:0/1488874751 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5140197600 con 0x7f5140102ef0 2024-09-19T06:53:48.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.078+0000 7f5127fff640 1 -- 172.21.15.103:0/1488874751 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f5138005560 con 0x7f5140102ef0 2024-09-19T06:53:48.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.079+0000 7f514761d740 1 Processor -- start 2024-09-19T06:53:48.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.079+0000 7f514761d740 1 -- start start 2024-09-19T06:53:48.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.079+0000 7f5127fff640 1 --2- 172.21.15.103:0/1488874751 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f51280a33b0 0x7f51280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.079+0000 7f514761d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e04f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.079+0000 7f514761d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 0x561ee6e0f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.079+0000 7f514761d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e0fd40 0x561ee6e12130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.079+0000 7f514761d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561ee6d6a280 con 0x561ee6e04b60 2024-09-19T06:53:48.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.079+0000 7f514761d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561ee6d11b10 con 0x561ee6e05480 2024-09-19T06:53:48.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.079+0000 7f514761d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561ee6d66240 con 0x561ee6e0fd40 2024-09-19T06:53:48.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.079+0000 7f513e59c640 1 --2- 172.21.15.103:0/1488874751 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f51280a33b0 0x7f51280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.079+0000 7f513ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e04f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.079+0000 7f513ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e04f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40842/0 (socket says 172.21.15.103:40842) 2024-09-19T06:53:48.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.079+0000 7f513ed9d640 1 -- 172.21.15.103:0/4114998218 learned_addr learned my addr 172.21.15.103:0/4114998218 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:48.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.079+0000 7f5127fff640 1 -- 172.21.15.103:0/1488874751 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f51380cff50 con 0x7f5140102ef0 2024-09-19T06:53:48.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.079+0000 7f513f59e640 1 --2- 172.21.15.103:0/4114998218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e0fd40 0x561ee6e12130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.080+0000 7f513e59c640 1 --2- 172.21.15.103:0/4114998218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 0x561ee6e0f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.080+0000 7f513ed9d640 1 -- 172.21.15.103:0/4114998218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e0fd40 msgr2=0x561ee6e12130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:48.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.080+0000 7f513ed9d640 1 --2- 172.21.15.103:0/4114998218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e0fd40 0x561ee6e12130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.080+0000 7f513ed9d640 1 -- 172.21.15.103:0/4114998218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 msgr2=0x561ee6e0f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:48.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.080+0000 7f513ed9d640 1 --2- 172.21.15.103:0/4114998218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 0x561ee6e0f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.080+0000 7f513ed9d640 1 -- 172.21.15.103:0/4114998218 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ee6d1e1f0 con 0x561ee6e04b60 2024-09-19T06:53:48.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.080+0000 7f513e59c640 1 --2- 172.21.15.103:0/1488874751 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f51280a33b0 0x7f51280a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5134009230 tx=0x7f51340042b0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:48.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.080+0000 7f513ed9d640 1 --2- 172.21.15.103:0/4114998218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e04f40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f51300092c0 tx=0x7f5130009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba7669f87e9a366 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:48.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.080+0000 7f51257fa640 1 -- 172.21.15.103:0/4114998218 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513001e070 con 0x561ee6e04b60 2024-09-19T06:53:48.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.080+0000 7f51257fa640 1 -- 172.21.15.103:0/4114998218 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5130019ad0 con 0x561ee6e04b60 2024-09-19T06:53:48.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.080+0000 7f51257fa640 1 -- 172.21.15.103:0/4114998218 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130018b80 con 0x561ee6e04b60 2024-09-19T06:53:48.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.081+0000 7f514761d740 1 -- 172.21.15.103:0/4114998218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 msgr2=0x561ee6e04f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.081+0000 7f514761d740 1 --2- 172.21.15.103:0/4114998218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e04f40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f51300092c0 tx=0x7f5130009780 comp rx=0 tx=0).stop 2024-09-19T06:53:48.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.081+0000 7f514761d740 1 -- 172.21.15.103:0/4114998218 shutdown_connections 2024-09-19T06:53:48.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.081+0000 7f514761d740 1 --2- 172.21.15.103:0/4114998218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e0fd40 0x561ee6e12130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.081+0000 7f514761d740 1 --2- 172.21.15.103:0/4114998218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 0x561ee6e0f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.081+0000 7f514761d740 1 --2- 172.21.15.103:0/4114998218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e04f40 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.081+0000 7f514761d740 1 -- 172.21.15.103:0/4114998218 >> 172.21.15.103:0/4114998218 conn(0x561ee6e010f0 msgr2=0x561ee6e014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:48.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.081+0000 7f514761d740 1 -- 172.21.15.103:0/4114998218 shutdown_connections 2024-09-19T06:53:48.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.081+0000 7f514761d740 1 -- 172.21.15.103:0/4114998218 wait complete. 2024-09-19T06:53:48.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.081+0000 7f514761d740 1 -- start start 2024-09-19T06:53:48.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f514761d740 1 Processor -- start 2024-09-19T06:53:48.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f514761d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e04b60 0x561ee6e0cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f514761d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 0x561ee6e0d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f514761d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e0fd40 0x561ee6e0d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f514761d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561ee6e12910 con 0x561ee6e0fd40 2024-09-19T06:53:48.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f514761d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561ee6d24f10 con 0x561ee6e05480 2024-09-19T06:53:48.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f514761d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561ee6e12a90 con 0x561ee6e04b60 2024-09-19T06:53:48.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f513f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e0fd40 0x561ee6e0d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f513f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e0fd40 0x561ee6e0d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40852/0 (socket says 172.21.15.103:40852) 2024-09-19T06:53:48.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f513f59e640 1 -- 172.21.15.103:0/2216673506 learned_addr learned my addr 172.21.15.103:0/2216673506 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:48.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f514761d740 10 client.0 _ll_register_callbacks cb 0x561ee6ceda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:48.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f513ed9d640 1 --2- 172.21.15.103:0/2216673506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e04b60 0x561ee6e0cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f513f59e640 1 -- 172.21.15.103:0/2216673506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e04b60 msgr2=0x561ee6e0cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:48.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f513f59e640 1 --2- 172.21.15.103:0/2216673506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e04b60 0x561ee6e0cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f513f59e640 1 -- 172.21.15.103:0/2216673506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 msgr2=0x561ee6e0d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:48.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f513f59e640 1 --2- 172.21.15.103:0/2216673506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 0x561ee6e0d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f513f59e640 1 -- 172.21.15.103:0/2216673506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ee6adb6a0 con 0x561ee6e0fd40 2024-09-19T06:53:48.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f513ed9d640 1 --2- 172.21.15.103:0/2216673506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e04b60 0x561ee6e0cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:48.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f513f59e640 1 --2- 172.21.15.103:0/2216673506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e0fd40 0x561ee6e0d6f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5138098bd0 tx=0x7f51380794f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff91bc05735b0260 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:48.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.082+0000 7f5122ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:48.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.083+0000 7f5122ff5640 1 -- 172.21.15.103:0/2216673506 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513801f040 con 0x561ee6e0fd40 2024-09-19T06:53:48.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.083+0000 7f5122ff5640 1 -- 172.21.15.103:0/2216673506 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5138095f00 con 0x561ee6e0fd40 2024-09-19T06:53:48.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.083+0000 7f514761d740 1 -- 172.21.15.103:0/2216673506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5130019ad0 con 0x561ee6e0fd40 2024-09-19T06:53:48.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.083+0000 7f5122ff5640 1 -- 172.21.15.103:0/2216673506 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f51380a05b0 con 0x561ee6e0fd40 2024-09-19T06:53:48.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.083+0000 7f514761d740 10 client.17076 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:48.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.083+0000 7f514761d740 1 -- 172.21.15.103:0/2216673506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ee6e1e5b0 con 0x561ee6e0fd40 2024-09-19T06:53:48.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.083+0000 7f5122ff5640 1 -- 172.21.15.103:0/2216673506 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5130019ad0 con 0x561ee6e0fd40 2024-09-19T06:53:48.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.083+0000 7f5122ff5640 1 -- 172.21.15.103:0/2216673506 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f513800f040 con 0x561ee6e0fd40 2024-09-19T06:53:48.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.083+0000 7f514761d740 10 client.17076 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:48.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.083+0000 7f514761d740 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:48.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.083+0000 7f514761d740 20 client.17076 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:48.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.083+0000 7f514761d740 1 --2- 172.21.15.103:0/2216673506 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561ee6e1f9c0 0x561ee6e21e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.083+0000 7f514761d740 4 client.17076 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:48.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.083+0000 7f514761d740 1 -- 172.21.15.103:0/2216673506 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ee6d777f0 con 0x561ee6e1f9c0 2024-09-19T06:53:48.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.084+0000 7f513ed9d640 1 --2- 172.21.15.103:0/2216673506 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561ee6e1f9c0 0x561ee6e21e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.084+0000 7f5122ff5640 10 client.17076 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:48.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.084+0000 7f513ed9d640 1 --2- 172.21.15.103:0/2216673506 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561ee6e1f9c0 0x561ee6e21e30 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10fad098c13eea4c server_cookie=e4fe2c98c24eaa9b in_seq=0 out_seq=0 2024-09-19T06:53:48.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.085+0000 7f5122ff5640 1 -- 172.21.15.103:0/2216673506 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ee6d777f0 con 0x561ee6e1f9c0 2024-09-19T06:53:48.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.085+0000 7f5122ff5640 10 client.17076 handle_command_reply: tid=0 2024-09-19T06:53:48.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.085+0000 7f514761d740 1 client.17076 shutdown 2024-09-19T06:53:48.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.085+0000 7f514761d740 1 -- 172.21.15.103:0/2216673506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e0fd40 msgr2=0x561ee6e0d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.085+0000 7f514761d740 1 --2- 172.21.15.103:0/2216673506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e0fd40 0x561ee6e0d6f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5138098bd0 tx=0x7f51380794f0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.085+0000 7f514761d740 1 -- 172.21.15.103:0/2216673506 shutdown_connections 2024-09-19T06:53:48.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.085+0000 7f514761d740 1 --2- 172.21.15.103:0/2216673506 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561ee6e1f9c0 0x561ee6e21e30 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.085+0000 7f514761d740 1 --2- 172.21.15.103:0/2216673506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e0fd40 0x561ee6e0d6f0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.085+0000 7f514761d740 1 --2- 172.21.15.103:0/2216673506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 0x561ee6e0d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.085+0000 7f514761d740 1 --2- 172.21.15.103:0/2216673506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e04b60 0x561ee6e0cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.085+0000 7f514761d740 1 -- 172.21.15.103:0/2216673506 >> 172.21.15.103:0/2216673506 conn(0x561ee6e010f0 msgr2=0x561ee6e00270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:48.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.085+0000 7f5122ff5640 0 client.17076 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:48.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.085+0000 7f514761d740 1 -- 172.21.15.103:0/2216673506 shutdown_connections 2024-09-19T06:53:48.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.085+0000 7f514761d740 1 -- 172.21.15.103:0/2216673506 wait complete. 2024-09-19T06:53:48.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.085+0000 7f514761d740 20 client.17076 trim_cache size 0 max 16384 2024-09-19T06:53:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.094+0000 7f19a6bbd740 1 Processor -- start 2024-09-19T06:53:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.095+0000 7f1992ffd640 1 -- 172.21.15.103:0/867782465 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1994005020 con 0x7f199c1069e0 2024-09-19T06:53:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a6bbd740 1 -- start start 2024-09-19T06:53:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a6bbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def629d0 0x5585def62db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a6bbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def632f0 0x5585def71b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a6bbd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def720c0 0x5585def744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a6bbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5585deec72f0 con 0x5585def632f0 2024-09-19T06:53:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a6bbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5585dee6f270 con 0x5585def629d0 2024-09-19T06:53:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a6bbd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5585deec32b0 con 0x5585def720c0 2024-09-19T06:53:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a1ae5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def632f0 0x5585def71b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a1ae5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def632f0 0x5585def71b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40870/0 (socket says 172.21.15.103:40870) 2024-09-19T06:53:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a1ae5640 1 -- 172.21.15.103:0/503475657 learned_addr learned my addr 172.21.15.103:0/503475657 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a12e4640 1 --2- 172.21.15.103:0/503475657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def629d0 0x5585def62db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a12e4640 1 --2- 172.21.15.103:0/503475657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def720c0 0x5585def744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a1ae5640 1 -- 172.21.15.103:0/503475657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def720c0 msgr2=0x5585def744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a1ae5640 1 --2- 172.21.15.103:0/503475657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def720c0 0x5585def744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a1ae5640 1 -- 172.21.15.103:0/503475657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def629d0 msgr2=0x5585def62db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a1ae5640 1 --2- 172.21.15.103:0/503475657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def629d0 0x5585def62db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a1ae5640 1 -- 172.21.15.103:0/503475657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585dee6e340 con 0x5585def632f0 2024-09-19T06:53:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a12e4640 1 --2- 172.21.15.103:0/503475657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def629d0 0x5585def62db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f1992ffd640 1 --2- 172.21.15.103:0/867782465 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19880a3460 0x7f19880a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.096+0000 7f19a12e4640 1 --2- 172.21.15.103:0/867782465 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19880a3460 0x7f19880a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.097+0000 7f19a1ae5640 1 --2- 172.21.15.103:0/503475657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def632f0 0x5585def71b80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1998009930 tx=0x7f1998005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54df0f242d3b6bb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.097+0000 7f1992ffd640 1 -- 172.21.15.103:0/867782465 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f19940d1750 con 0x7f199c1069e0 2024-09-19T06:53:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.097+0000 7f1987fff640 1 -- 172.21.15.103:0/503475657 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998040070 con 0x5585def632f0 2024-09-19T06:53:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.097+0000 7f1987fff640 1 -- 172.21.15.103:0/503475657 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1998031870 con 0x5585def632f0 2024-09-19T06:53:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.097+0000 7f1987fff640 1 -- 172.21.15.103:0/503475657 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998035040 con 0x5585def632f0 2024-09-19T06:53:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.098+0000 7f19a6bbd740 1 -- 172.21.15.103:0/503475657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def632f0 msgr2=0x5585def71b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.098+0000 7f19a12e4640 1 --2- 172.21.15.103:0/867782465 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19880a3460 0x7f19880a5920 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5585def603e0 tx=0x7f198c0096f0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.098+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/503475657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def632f0 0x5585def71b80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1998009930 tx=0x7f1998005bd0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.098+0000 7f19a6bbd740 1 -- 172.21.15.103:0/503475657 shutdown_connections 2024-09-19T06:53:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.098+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/503475657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def720c0 0x5585def744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.098+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/503475657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def632f0 0x5585def71b80 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.098+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/503475657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def629d0 0x5585def62db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.098+0000 7f19a6bbd740 1 -- 172.21.15.103:0/503475657 >> 172.21.15.103:0/503475657 conn(0x5585def5f060 msgr2=0x5585def5f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.098+0000 7f19a6bbd740 1 -- 172.21.15.103:0/503475657 shutdown_connections 2024-09-19T06:53:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.098+0000 7f19a6bbd740 1 -- 172.21.15.103:0/503475657 wait complete. 2024-09-19T06:53:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.098+0000 7f19a6bbd740 1 -- start start 2024-09-19T06:53:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.098+0000 7f19a6bbd740 1 Processor -- start 2024-09-19T06:53:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a6bbd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def629d0 0x5585def6f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a6bbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def632f0 0x5585def6f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a6bbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def720c0 0x5585def6fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a6bbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5585def74c90 con 0x5585def632f0 2024-09-19T06:53:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a6bbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5585dee50290 con 0x5585def720c0 2024-09-19T06:53:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a12e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def632f0 0x5585def6f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a6bbd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5585def74e10 con 0x5585def629d0 2024-09-19T06:53:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a6bbd740 10 client.0 _ll_register_callbacks cb 0x5585dee81140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a12e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def632f0 0x5585def6f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40884/0 (socket says 172.21.15.103:40884) 2024-09-19T06:53:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a12e4640 1 -- 172.21.15.103:0/2725662942 learned_addr learned my addr 172.21.15.103:0/2725662942 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a1ae5640 1 --2- 172.21.15.103:0/2725662942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def629d0 0x5585def6f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a22e6640 1 --2- 172.21.15.103:0/2725662942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def720c0 0x5585def6fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a12e4640 1 -- 172.21.15.103:0/2725662942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def629d0 msgr2=0x5585def6f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a12e4640 1 --2- 172.21.15.103:0/2725662942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def629d0 0x5585def6f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a12e4640 1 -- 172.21.15.103:0/2725662942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def720c0 msgr2=0x5585def6fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a12e4640 1 --2- 172.21.15.103:0/2725662942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def720c0 0x5585def6fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a12e4640 1 -- 172.21.15.103:0/2725662942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585debda210 con 0x5585def632f0 2024-09-19T06:53:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a22e6640 1 --2- 172.21.15.103:0/2725662942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def720c0 0x5585def6fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.099+0000 7f19a1ae5640 1 --2- 172.21.15.103:0/2725662942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def629d0 0x5585def6f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.100+0000 7f19a12e4640 1 --2- 172.21.15.103:0/2725662942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def632f0 0x5585def6f570 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f198c004590 tx=0x7f198c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236036cef5dc6c6c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.100+0000 7f19857fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.100+0000 7f19857fa640 1 -- 172.21.15.103:0/2725662942 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c010490 con 0x5585def632f0 2024-09-19T06:53:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.100+0000 7f19857fa640 1 -- 172.21.15.103:0/2725662942 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f198c010630 con 0x5585def632f0 2024-09-19T06:53:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.100+0000 7f19a6bbd740 1 -- 172.21.15.103:0/2725662942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1998031870 con 0x5585def632f0 2024-09-19T06:53:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.100+0000 7f19857fa640 1 -- 172.21.15.103:0/2725662942 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f198c010830 con 0x5585def632f0 2024-09-19T06:53:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.100+0000 7f19a6bbd740 10 client.17088 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.100+0000 7f19a6bbd740 1 -- 172.21.15.103:0/2725662942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5585deca6620 con 0x5585def632f0 2024-09-19T06:53:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.100+0000 7f19857fa640 1 -- 172.21.15.103:0/2725662942 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1998031870 con 0x5585def632f0 2024-09-19T06:53:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.100+0000 7f19857fa640 1 -- 172.21.15.103:0/2725662942 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f198c02acc0 con 0x5585def632f0 2024-09-19T06:53:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.100+0000 7f19a6bbd740 10 client.17088 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.100+0000 7f19a6bbd740 10 client.17088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.100+0000 7f19a6bbd740 20 client.17088 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.100+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/2725662942 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5585def82130 0x5585def84520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.100+0000 7f19a6bbd740 4 client.17088 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.100+0000 7f19a6bbd740 1 -- 172.21.15.103:0/2725662942 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585deed4a20 con 0x5585def82130 2024-09-19T06:53:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.101+0000 7f19a1ae5640 1 --2- 172.21.15.103:0/2725662942 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5585def82130 0x5585def84520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.101+0000 7f19a1ae5640 1 --2- 172.21.15.103:0/2725662942 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5585def82130 0x5585def84520 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4a6f5e48819b7a2 server_cookie=911124dd34d41f3a in_seq=0 out_seq=0 2024-09-19T06:53:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.101+0000 7f19857fa640 10 client.17088 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.102+0000 7f19857fa640 1 -- 172.21.15.103:0/2725662942 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585deed4a20 con 0x5585def82130 2024-09-19T06:53:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.102+0000 7f19857fa640 10 client.17088 handle_command_reply: tid=0 2024-09-19T06:53:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.102+0000 7f19a6bbd740 1 client.17088 shutdown 2024-09-19T06:53:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.102+0000 7f19a6bbd740 1 -- 172.21.15.103:0/2725662942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def632f0 msgr2=0x5585def6f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.102+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/2725662942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def632f0 0x5585def6f570 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f198c004590 tx=0x7f198c005f90 comp rx=0 tx=0).stop 2024-09-19T06:53:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.102+0000 7f19a6bbd740 1 -- 172.21.15.103:0/2725662942 shutdown_connections 2024-09-19T06:53:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.102+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/2725662942 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5585def82130 0x5585def84520 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.102+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/2725662942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def720c0 0x5585def6fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.102+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/2725662942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def632f0 0x5585def6f570 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.102+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/2725662942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def629d0 0x5585def6f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.102+0000 7f19a6bbd740 1 -- 172.21.15.103:0/2725662942 >> 172.21.15.103:0/2725662942 conn(0x5585def5f060 msgr2=0x5585def64d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.102+0000 7f19857fa640 0 client.17088 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.102+0000 7f19a6bbd740 1 -- 172.21.15.103:0/2725662942 shutdown_connections 2024-09-19T06:53:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.102+0000 7f19a6bbd740 1 -- 172.21.15.103:0/2725662942 wait complete. 2024-09-19T06:53:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.102+0000 7f19a6bbd740 20 client.17088 trim_cache size 0 max 16384 2024-09-19T06:53:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f514761d740 1 Processor -- start 2024-09-19T06:53:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f514761d740 1 -- start start 2024-09-19T06:53:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f514761d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e0ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f514761d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 0x561ee6e27700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f514761d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e0fd40 0x561ee6e27c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f514761d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f51401ab650 con 0x561ee6e04b60 2024-09-19T06:53:48.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f514761d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51401ad190 con 0x561ee6e05480 2024-09-19T06:53:48.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f514761d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f51401abd20 con 0x561ee6e0fd40 2024-09-19T06:53:48.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f513ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e0ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f513ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e0ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40890/0 (socket says 172.21.15.103:40890) 2024-09-19T06:53:48.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f513ed9d640 1 -- 172.21.15.103:0/314113933 learned_addr learned my addr 172.21.15.103:0/314113933 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:48.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f513e59c640 1 --2- 172.21.15.103:0/314113933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 0x561ee6e27700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f513ed9d640 1 -- 172.21.15.103:0/314113933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e0fd40 msgr2=0x561ee6e27c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:48.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f513ed9d640 1 --2- 172.21.15.103:0/314113933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e0fd40 0x561ee6e27c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f513ed9d640 1 -- 172.21.15.103:0/314113933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 msgr2=0x561ee6e27700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:48.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f513ed9d640 1 --2- 172.21.15.103:0/314113933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 0x561ee6e27700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.106+0000 7f513ed9d640 1 -- 172.21.15.103:0/314113933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ee6e052b0 con 0x561ee6e04b60 2024-09-19T06:53:48.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.107+0000 7f513ed9d640 1 --2- 172.21.15.103:0/314113933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e0ead0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5130009c80 tx=0x7f513000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae6754e94a874ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:48.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.107+0000 7f5124ff9640 1 -- 172.21.15.103:0/314113933 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513001e040 con 0x561ee6e04b60 2024-09-19T06:53:48.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.107+0000 7f5124ff9640 1 -- 172.21.15.103:0/314113933 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f513002a030 con 0x561ee6e04b60 2024-09-19T06:53:48.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.107+0000 7f5124ff9640 1 -- 172.21.15.103:0/314113933 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130019470 con 0x561ee6e04b60 2024-09-19T06:53:48.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 -- 172.21.15.103:0/314113933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 msgr2=0x561ee6e0ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 --2- 172.21.15.103:0/314113933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e0ead0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5130009c80 tx=0x7f513000ec40 comp rx=0 tx=0).stop 2024-09-19T06:53:48.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 -- 172.21.15.103:0/314113933 shutdown_connections 2024-09-19T06:53:48.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 --2- 172.21.15.103:0/314113933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e0fd40 0x561ee6e27c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 --2- 172.21.15.103:0/314113933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 0x561ee6e27700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 --2- 172.21.15.103:0/314113933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e0ead0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 -- 172.21.15.103:0/314113933 >> 172.21.15.103:0/314113933 conn(0x561ee6e010f0 msgr2=0x561ee6e0e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:48.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 -- 172.21.15.103:0/314113933 shutdown_connections 2024-09-19T06:53:48.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 -- 172.21.15.103:0/314113933 wait complete. 2024-09-19T06:53:48.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 -- start start 2024-09-19T06:53:48.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 Processor -- start 2024-09-19T06:53:48.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e18520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 0x561ee6e13850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e0fd40 0x561ee6e13e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f51401acac0 con 0x561ee6e04b60 2024-09-19T06:53:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51401ac3f0 con 0x561ee6e05480 2024-09-19T06:53:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f51401a9610 con 0x561ee6e0fd40 2024-09-19T06:53:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f514761d740 10 client.0 _ll_register_callbacks cb 0x7f513809be10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f513ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e18520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f513ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e18520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40892/0 (socket says 172.21.15.103:40892) 2024-09-19T06:53:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.108+0000 7f513ed9d640 1 -- 172.21.15.103:0/2687676162 learned_addr learned my addr 172.21.15.103:0/2687676162 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.109+0000 7f513e59c640 1 --2- 172.21.15.103:0/2687676162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 0x561ee6e13850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.109+0000 7f513f59e640 1 --2- 172.21.15.103:0/2687676162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e0fd40 0x561ee6e13e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.109+0000 7f513ed9d640 1 -- 172.21.15.103:0/2687676162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e0fd40 msgr2=0x561ee6e13e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.109+0000 7f513ed9d640 1 --2- 172.21.15.103:0/2687676162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e0fd40 0x561ee6e13e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.109+0000 7f513ed9d640 1 -- 172.21.15.103:0/2687676162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 msgr2=0x561ee6e13850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.109+0000 7f513ed9d640 1 --2- 172.21.15.103:0/2687676162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 0x561ee6e13850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.109+0000 7f513ed9d640 1 -- 172.21.15.103:0/2687676162 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ee6e14640 con 0x561ee6e04b60 2024-09-19T06:53:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.109+0000 7f513f59e640 1 --2- 172.21.15.103:0/2687676162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e0fd40 0x561ee6e13e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.109+0000 7f513ed9d640 1 --2- 172.21.15.103:0/2687676162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e18520 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f51300053f0 tx=0x7f513000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52570b7ed80caac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.109+0000 7f5122ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.109+0000 7f5122ff5640 1 -- 172.21.15.103:0/2687676162 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130036030 con 0x561ee6e04b60 2024-09-19T06:53:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.109+0000 7f514761d740 1 -- 172.21.15.103:0/2687676162 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f513002a030 con 0x561ee6e04b60 2024-09-19T06:53:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.109+0000 7f5122ff5640 1 -- 172.21.15.103:0/2687676162 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5130018390 con 0x561ee6e04b60 2024-09-19T06:53:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.109+0000 7f5122ff5640 1 -- 172.21.15.103:0/2687676162 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f513001e050 con 0x561ee6e04b60 2024-09-19T06:53:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.109+0000 7f514761d740 10 client.17094 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.109+0000 7f514761d740 1 -- 172.21.15.103:0/2687676162 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ee6e8a7f0 con 0x561ee6e04b60 2024-09-19T06:53:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.110+0000 7f5122ff5640 1 -- 172.21.15.103:0/2687676162 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f513002a030 con 0x561ee6e04b60 2024-09-19T06:53:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.110+0000 7f5122ff5640 1 -- 172.21.15.103:0/2687676162 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5130018070 con 0x561ee6e04b60 2024-09-19T06:53:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.110+0000 7f514761d740 10 client.17094 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.110+0000 7f514761d740 10 client.17094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.110+0000 7f514761d740 20 client.17094 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.110+0000 7f514761d740 1 --2- 172.21.15.103:0/2687676162 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561ee6e8acc0 0x561ee6e8d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.110+0000 7f514761d740 4 client.17094 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:53:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.110+0000 7f514761d740 1 -- 172.21.15.103:0/2687676162 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561ee6d71d00 con 0x561ee6e8acc0 2024-09-19T06:53:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.110+0000 7f513e59c640 1 --2- 172.21.15.103:0/2687676162 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561ee6e8acc0 0x561ee6e8d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.110+0000 7f5122ff5640 10 client.17094 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.110+0000 7f513e59c640 1 --2- 172.21.15.103:0/2687676162 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561ee6e8acc0 0x561ee6e8d0b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3a0224addd9101f server_cookie=b6679e57bffd95bb in_seq=0 out_seq=0 2024-09-19T06:53:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.112+0000 7f5122ff5640 1 -- 172.21.15.103:0/2687676162 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561ee6d71d00 con 0x561ee6e8acc0 2024-09-19T06:53:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.112+0000 7f5122ff5640 10 client.17094 handle_command_reply: tid=0 2024-09-19T06:53:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.112+0000 7f514761d740 1 client.17094 shutdown 2024-09-19T06:53:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.112+0000 7f514761d740 1 -- 172.21.15.103:0/2687676162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 msgr2=0x561ee6e18520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.112+0000 7f514761d740 1 --2- 172.21.15.103:0/2687676162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e18520 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f51300053f0 tx=0x7f513000ea40 comp rx=0 tx=0).stop 2024-09-19T06:53:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.112+0000 7f514761d740 1 -- 172.21.15.103:0/2687676162 shutdown_connections 2024-09-19T06:53:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.112+0000 7f514761d740 1 --2- 172.21.15.103:0/2687676162 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561ee6e8acc0 0x561ee6e8d0b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.112+0000 7f514761d740 1 --2- 172.21.15.103:0/2687676162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee6e0fd40 0x561ee6e13e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.112+0000 7f514761d740 1 --2- 172.21.15.103:0/2687676162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ee6e05480 0x561ee6e13850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.112+0000 7f5122ff5640 0 client.17094 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.112+0000 7f514761d740 1 --2- 172.21.15.103:0/2687676162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee6e04b60 0x561ee6e18520 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.112+0000 7f514761d740 1 -- 172.21.15.103:0/2687676162 >> 172.21.15.103:0/2687676162 conn(0x561ee6e010f0 msgr2=0x561ee6e0d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.112+0000 7f514761d740 1 -- 172.21.15.103:0/2687676162 shutdown_connections 2024-09-19T06:53:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.112+0000 7f514761d740 1 -- 172.21.15.103:0/2687676162 wait complete. 2024-09-19T06:53:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.112+0000 7f514761d740 20 client.17094 trim_cache size 0 max 16384 2024-09-19T06:53:48.138 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:53:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T06:53:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "459a3fef-4dfc-4143-8840-c598e5939b31", 2024-09-19T06:53:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T06:53:48.138 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:53:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.113+0000 7f5145244640 1 -- 172.21.15.103:0/1488874751 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f51280a33b0 msgr2=0x7f51280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.113+0000 7f5145244640 1 --2- 172.21.15.103:0/1488874751 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f51280a33b0 0x7f51280a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5134009230 tx=0x7f51340042b0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.113+0000 7f5145244640 1 -- 172.21.15.103:0/1488874751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5140102ef0 msgr2=0x7f51401a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.113+0000 7f5145244640 1 --2- 172.21.15.103:0/1488874751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5140102ef0 0x7f51401a43d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f513800eaa0 tx=0x7f513800ef60 comp rx=0 tx=0).stop 2024-09-19T06:53:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.113+0000 7f5145244640 1 -- 172.21.15.103:0/1488874751 shutdown_connections 2024-09-19T06:53:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.113+0000 7f5145244640 1 --2- 172.21.15.103:0/1488874751 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f51280a33b0 0x7f51280a5870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.113+0000 7f5145244640 1 --2- 172.21.15.103:0/1488874751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5140102ef0 0x7f51401a43d0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.113+0000 7f5145244640 1 --2- 172.21.15.103:0/1488874751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5140102570 0x7f5140199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.113+0000 7f5145244640 1 --2- 172.21.15.103:0/1488874751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5140100e20 0x7f5140199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.113+0000 7f5145244640 1 -- 172.21.15.103:0/1488874751 >> 172.21.15.103:0/1488874751 conn(0x7f51400fc7d0 msgr2=0x7f51400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.113+0000 7f5145244640 1 -- 172.21.15.103:0/1488874751 shutdown_connections 2024-09-19T06:53:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:48.113+0000 7f5145244640 1 -- 172.21.15.103:0/1488874751 wait complete. 2024-09-19T06:53:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.118+0000 7f19a6bbd740 1 Processor -- start 2024-09-19T06:53:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.118+0000 7f19a6bbd740 1 -- start start 2024-09-19T06:53:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a6bbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def629d0 0x5585def70fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a6bbd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def632f0 0x5585def89df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a6bbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def720c0 0x5585def8a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a6bbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f199c1ab650 con 0x5585def629d0 2024-09-19T06:53:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a6bbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f199c1ad190 con 0x5585def720c0 2024-09-19T06:53:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a6bbd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f199c1abd20 con 0x5585def632f0 2024-09-19T06:53:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a1ae5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def629d0 0x5585def70fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a12e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def632f0 0x5585def89df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a1ae5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def629d0 0x5585def70fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40898/0 (socket says 172.21.15.103:40898) 2024-09-19T06:53:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a1ae5640 1 -- 172.21.15.103:0/4238620597 learned_addr learned my addr 172.21.15.103:0/4238620597 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a22e6640 1 --2- 172.21.15.103:0/4238620597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def720c0 0x5585def8a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a1ae5640 1 -- 172.21.15.103:0/4238620597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def632f0 msgr2=0x5585def89df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a1ae5640 1 --2- 172.21.15.103:0/4238620597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def632f0 0x5585def89df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a1ae5640 1 -- 172.21.15.103:0/4238620597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def720c0 msgr2=0x5585def8a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a1ae5640 1 --2- 172.21.15.103:0/4238620597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def720c0 0x5585def8a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a1ae5640 1 -- 172.21.15.103:0/4238620597 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585def5d650 con 0x5585def629d0 2024-09-19T06:53:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a22e6640 1 --2- 172.21.15.103:0/4238620597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def720c0 0x5585def8a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19a1ae5640 1 --2- 172.21.15.103:0/4238620597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def629d0 0x5585def70fa0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1998009900 tx=0x7f1998000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55871ab0dbe1339 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.119+0000 7f19877fe640 1 -- 172.21.15.103:0/4238620597 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998040070 con 0x5585def629d0 2024-09-19T06:53:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.120+0000 7f19877fe640 1 -- 172.21.15.103:0/4238620597 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1998031870 con 0x5585def629d0 2024-09-19T06:53:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.120+0000 7f19877fe640 1 -- 172.21.15.103:0/4238620597 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998035040 con 0x5585def629d0 2024-09-19T06:53:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.120+0000 7f19a6bbd740 1 -- 172.21.15.103:0/4238620597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def629d0 msgr2=0x5585def70fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.120+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/4238620597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def629d0 0x5585def70fa0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1998009900 tx=0x7f1998000bc0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.120+0000 7f19a6bbd740 1 -- 172.21.15.103:0/4238620597 shutdown_connections 2024-09-19T06:53:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.120+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/4238620597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def720c0 0x5585def8a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.120+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/4238620597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def632f0 0x5585def89df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.120+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/4238620597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def629d0 0x5585def70fa0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.120+0000 7f19a6bbd740 1 -- 172.21.15.103:0/4238620597 >> 172.21.15.103:0/4238620597 conn(0x5585def5f060 msgr2=0x5585def70890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.120+0000 7f19a6bbd740 1 -- 172.21.15.103:0/4238620597 shutdown_connections 2024-09-19T06:53:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.120+0000 7f19a6bbd740 1 -- 172.21.15.103:0/4238620597 wait complete. 2024-09-19T06:53:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.121+0000 7f19a6bbd740 1 -- start start 2024-09-19T06:53:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.121+0000 7f19a6bbd740 1 Processor -- start 2024-09-19T06:53:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.121+0000 7f19a6bbd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def629d0 0x5585def7a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.121+0000 7f19a6bbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def632f0 0x5585def7a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.121+0000 7f19a6bbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def720c0 0x5585def6b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.121+0000 7f19a6bbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f199c1acac0 con 0x5585def720c0 2024-09-19T06:53:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.121+0000 7f19a6bbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f199c1ac3f0 con 0x5585def632f0 2024-09-19T06:53:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.121+0000 7f19a6bbd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f199c1a9610 con 0x5585def629d0 2024-09-19T06:53:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.121+0000 7f19a6bbd740 10 client.0 _ll_register_callbacks cb 0x7f198c009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.121+0000 7f19a22e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def720c0 0x5585def6b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.121+0000 7f19a22e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def720c0 0x5585def6b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40910/0 (socket says 172.21.15.103:40910) 2024-09-19T06:53:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.121+0000 7f19a22e6640 1 -- 172.21.15.103:0/3322222801 learned_addr learned my addr 172.21.15.103:0/3322222801 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.121+0000 7f19a1ae5640 1 --2- 172.21.15.103:0/3322222801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def629d0 0x5585def7a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.121+0000 7f19a12e4640 1 --2- 172.21.15.103:0/3322222801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def632f0 0x5585def7a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19a22e6640 1 -- 172.21.15.103:0/3322222801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def629d0 msgr2=0x5585def7a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19a22e6640 1 --2- 172.21.15.103:0/3322222801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def629d0 0x5585def7a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19a22e6640 1 -- 172.21.15.103:0/3322222801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def632f0 msgr2=0x5585def7a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19a22e6640 1 --2- 172.21.15.103:0/3322222801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def632f0 0x5585def7a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19a22e6640 1 -- 172.21.15.103:0/3322222801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585def6bc10 con 0x5585def720c0 2024-09-19T06:53:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19a12e4640 1 --2- 172.21.15.103:0/3322222801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def632f0 0x5585def7a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19a22e6640 1 --2- 172.21.15.103:0/3322222801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def720c0 0x5585def6b650 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f199400bb90 tx=0x7f199409d640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8699879078c276e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19857fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19857fa640 1 -- 172.21.15.103:0/3322222801 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19940a1df0 con 0x5585def720c0 2024-09-19T06:53:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19857fa640 1 -- 172.21.15.103:0/3322222801 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f199409a0b0 con 0x5585def720c0 2024-09-19T06:53:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19a6bbd740 1 -- 172.21.15.103:0/3322222801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1998031870 con 0x5585def720c0 2024-09-19T06:53:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19857fa640 1 -- 172.21.15.103:0/3322222801 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f19940a66b0 con 0x5585def720c0 2024-09-19T06:53:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19a6bbd740 10 client.17100 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19a6bbd740 1 -- 172.21.15.103:0/3322222801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5585defec650 con 0x5585def720c0 2024-09-19T06:53:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19857fa640 1 -- 172.21.15.103:0/3322222801 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1998031870 con 0x5585def720c0 2024-09-19T06:53:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19857fa640 1 -- 172.21.15.103:0/3322222801 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f199409ce90 con 0x5585def720c0 2024-09-19T06:53:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19a6bbd740 10 client.17100 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19a6bbd740 10 client.17100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19a6bbd740 20 client.17100 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/3322222801 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5585defecd70 0x5585defef190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19a6bbd740 4 client.17100 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:53:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.122+0000 7f19a6bbd740 1 -- 172.21.15.103:0/3322222801 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585deeced70 con 0x5585defecd70 2024-09-19T06:53:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.123+0000 7f19a1ae5640 1 --2- 172.21.15.103:0/3322222801 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5585defecd70 0x5585defef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.123+0000 7f19a1ae5640 1 --2- 172.21.15.103:0/3322222801 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5585defecd70 0x5585defef190 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30360a5796ea788b server_cookie=f9d2812f6ef8789f in_seq=0 out_seq=0 2024-09-19T06:53:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.123+0000 7f19857fa640 10 client.17100 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.133+0000 7f19857fa640 1 -- 172.21.15.103:0/3322222801 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5585deeced70 con 0x5585defecd70 2024-09-19T06:53:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.133+0000 7f19857fa640 10 client.17100 handle_command_reply: tid=0 2024-09-19T06:53:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.133+0000 7f19a6bbd740 1 client.17100 shutdown 2024-09-19T06:53:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.134+0000 7f19a6bbd740 1 -- 172.21.15.103:0/3322222801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def720c0 msgr2=0x5585def6b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.134+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/3322222801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def720c0 0x5585def6b650 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f199400bb90 tx=0x7f199409d640 comp rx=0 tx=0).stop 2024-09-19T06:53:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.134+0000 7f19a6bbd740 1 -- 172.21.15.103:0/3322222801 shutdown_connections 2024-09-19T06:53:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.134+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/3322222801 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5585defecd70 0x5585defef190 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.134+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/3322222801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585def720c0 0x5585def6b650 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.134+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/3322222801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585def632f0 0x5585def7a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.134+0000 7f19857fa640 0 client.17100 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.134+0000 7f19a6bbd740 1 --2- 172.21.15.103:0/3322222801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585def629d0 0x5585def7a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.134+0000 7f19a6bbd740 1 -- 172.21.15.103:0/3322222801 >> 172.21.15.103:0/3322222801 conn(0x5585def5f060 msgr2=0x5585def5e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.134+0000 7f19a6bbd740 1 -- 172.21.15.103:0/3322222801 shutdown_connections 2024-09-19T06:53:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.134+0000 7f19a6bbd740 1 -- 172.21.15.103:0/3322222801 wait complete. 2024-09-19T06:53:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.134+0000 7f19a6bbd740 20 client.17100 trim_cache size 0 max 16384 2024-09-19T06:53:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:53:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:53:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:53:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:53:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.134+0000 7f19a3d49640 1 -- 172.21.15.103:0/867782465 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19880a3460 msgr2=0x7f19880a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.134+0000 7f19a3d49640 1 --2- 172.21.15.103:0/867782465 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19880a3460 0x7f19880a5920 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5585def603e0 tx=0x7f198c0096f0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.134+0000 7f19a3d49640 1 -- 172.21.15.103:0/867782465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f199c1069e0 msgr2=0x7f199c19bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.134+0000 7f19a3d49640 1 --2- 172.21.15.103:0/867782465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f199c1069e0 0x7f199c19bb70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f199400eca0 tx=0x7f199400b690 comp rx=0 tx=0).stop 2024-09-19T06:53:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.135+0000 7f19a3d49640 1 -- 172.21.15.103:0/867782465 shutdown_connections 2024-09-19T06:53:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.135+0000 7f19a3d49640 1 --2- 172.21.15.103:0/867782465 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19880a3460 0x7f19880a5920 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.135+0000 7f19a3d49640 1 --2- 172.21.15.103:0/867782465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f199c1069e0 0x7f199c19bb70 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.135+0000 7f19a3d49640 1 --2- 172.21.15.103:0/867782465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f199c101370 0x7f199c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.135+0000 7f19a3d49640 1 --2- 172.21.15.103:0/867782465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f199c1009d0 0x7f199c1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.135+0000 7f19a3d49640 1 -- 172.21.15.103:0/867782465 >> 172.21.15.103:0/867782465 conn(0x7f199c0fc6b0 msgr2=0x7f199c0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.135+0000 7f19a3d49640 1 -- 172.21.15.103:0/867782465 shutdown_connections 2024-09-19T06:53:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:53:48.135+0000 7f19a3d49640 1 -- 172.21.15.103:0/867782465 wait complete. 2024-09-19T06:53:48.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:48 smithi174 ceph-mon[35637]: pgmap v140: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-09-19T06:53:48.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:48 smithi103 ceph-mon[31158]: pgmap v140: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-09-19T06:53:48.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:48 smithi134 ceph-mon[35356]: pgmap v140: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-09-19T06:53:48.979 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:53:49.266 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:49.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:49 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T06:53:49.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:49 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:49.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:49 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T06:53:49.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:49 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:49.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:49 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T06:53:49.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:49 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:49.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:49 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T06:53:49.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:49 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:49.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:49 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T06:53:49.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:49 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:49.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:49 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T06:53:49.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:49 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:49.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.726+0000 7f7248f16640 1 -- 172.21.15.103:0/918049617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7244102ef0 msgr2=0x7f724410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:49.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.726+0000 7f7248f16640 1 --2- 172.21.15.103:0/918049617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7244102ef0 0x7f724410da40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f723c009ca0 tx=0x7f723c02a950 comp rx=0 tx=0).stop 2024-09-19T06:53:49.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.726+0000 7f7248f16640 1 -- 172.21.15.103:0/918049617 shutdown_connections 2024-09-19T06:53:49.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.726+0000 7f7248f16640 1 --2- 172.21.15.103:0/918049617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7244102ef0 0x7f724410da40 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.726+0000 7f7248f16640 1 --2- 172.21.15.103:0/918049617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7244102570 0x7f72441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.726+0000 7f7248f16640 1 --2- 172.21.15.103:0/918049617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7244100e20 0x7f7244101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.726+0000 7f7248f16640 1 -- 172.21.15.103:0/918049617 >> 172.21.15.103:0/918049617 conn(0x7f72440fc7d0 msgr2=0x7f72440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:49.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.726+0000 7f7248f16640 1 -- 172.21.15.103:0/918049617 shutdown_connections 2024-09-19T06:53:49.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.726+0000 7f7248f16640 1 -- 172.21.15.103:0/918049617 wait complete. 2024-09-19T06:53:49.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.727+0000 7f7248f16640 1 Processor -- start 2024-09-19T06:53:49.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.727+0000 7f7248f16640 1 -- start start 2024-09-19T06:53:49.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.727+0000 7f7248f16640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7244100e20 0x7f7244197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.727+0000 7f7248f16640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7244102570 0x7f7244197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.727+0000 7f7248f16640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7244102ef0 0x7f724419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.727+0000 7f7248f16640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7244110f80 con 0x7f7244102ef0 2024-09-19T06:53:49.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.727+0000 7f7248f16640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7244110e00 con 0x7f7244100e20 2024-09-19T06:53:49.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.727+0000 7f7248f16640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7244111100 con 0x7f7244102570 2024-09-19T06:53:49.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.728+0000 7f7242d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7244102ef0 0x7f724419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:49.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.728+0000 7f7242d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7244102ef0 0x7f724419bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40928/0 (socket says 172.21.15.103:40928) 2024-09-19T06:53:49.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.728+0000 7f7242d9d640 1 -- 172.21.15.103:0/189533978 learned_addr learned my addr 172.21.15.103:0/189533978 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:49.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.728+0000 7f7241d9b640 1 --2- 172.21.15.103:0/189533978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7244102570 0x7f7244197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:49.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.728+0000 7f724259c640 1 --2- 172.21.15.103:0/189533978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7244100e20 0x7f7244197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:49.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.728+0000 7f7242d9d640 1 -- 172.21.15.103:0/189533978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7244102570 msgr2=0x7f7244197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:49.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.728+0000 7f7242d9d640 1 --2- 172.21.15.103:0/189533978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7244102570 0x7f7244197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.728+0000 7f7242d9d640 1 -- 172.21.15.103:0/189533978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7244100e20 msgr2=0x7f7244197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:49.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.728+0000 7f7242d9d640 1 --2- 172.21.15.103:0/189533978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7244100e20 0x7f7244197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.728+0000 7f7242d9d640 1 -- 172.21.15.103:0/189533978 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724419c3b0 con 0x7f7244102ef0 2024-09-19T06:53:49.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.728+0000 7f7242d9d640 1 --2- 172.21.15.103:0/189533978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7244102ef0 0x7f724419bcd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f723c02ae70 tx=0x7f723c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:49.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.728+0000 7f72337fe640 1 -- 172.21.15.103:0/189533978 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c034040 con 0x7f7244102ef0 2024-09-19T06:53:49.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.728+0000 7f72337fe640 1 -- 172.21.15.103:0/189533978 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f723c030920 con 0x7f7244102ef0 2024-09-19T06:53:49.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.728+0000 7f7248f16640 1 -- 172.21.15.103:0/189533978 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f724419c640 con 0x7f7244102ef0 2024-09-19T06:53:49.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.728+0000 7f72337fe640 1 -- 172.21.15.103:0/189533978 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c004040 con 0x7f7244102ef0 2024-09-19T06:53:49.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.728+0000 7f7248f16640 1 -- 172.21.15.103:0/189533978 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7244069f70 con 0x7f7244102ef0 2024-09-19T06:53:49.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.731+0000 7f72337fe640 1 -- 172.21.15.103:0/189533978 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f723c037020 con 0x7f7244102ef0 2024-09-19T06:53:49.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.731+0000 7f724b2ef740 1 Processor -- start 2024-09-19T06:53:49.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.731+0000 7f724b2ef740 1 -- start start 2024-09-19T06:53:49.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.731+0000 7f724b2ef740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b047740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.731+0000 7f724b2ef740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047c80 0x563b9b056550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.731+0000 7f724b2ef740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b056a90 0x563b9b058e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.731+0000 7f724b2ef740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563b9afad2e0 con 0x563b9b047360 2024-09-19T06:53:49.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.731+0000 7f724b2ef740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563b9af54ce0 con 0x563b9b047c80 2024-09-19T06:53:49.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.731+0000 7f724b2ef740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b9afa92a0 con 0x563b9b056a90 2024-09-19T06:53:49.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f72337fe640 1 --2- 172.21.15.103:0/189533978 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f722c0a33b0 0x7f722c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f7241d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b047740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:49.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f7241d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b047740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40940/0 (socket says 172.21.15.103:40940) 2024-09-19T06:53:49.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f7241d9b640 1 -- 172.21.15.103:0/2365308124 learned_addr learned my addr 172.21.15.103:0/2365308124 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:49.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f7241d9b640 1 --2- 172.21.15.103:0/2365308124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b056a90 0x563b9b058e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:49.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f724259c640 1 --2- 172.21.15.103:0/2365308124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047c80 0x563b9b056550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:49.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f72337fe640 1 -- 172.21.15.103:0/189533978 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f723c0f2c40 con 0x7f7244102ef0 2024-09-19T06:53:49.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f7242d9d640 1 --2- 172.21.15.103:0/189533978 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f722c0a33b0 0x7f722c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:49.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f7241d9b640 1 -- 172.21.15.103:0/2365308124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b056a90 msgr2=0x563b9b058e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:49.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f7241d9b640 1 --2- 172.21.15.103:0/2365308124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b056a90 0x563b9b058e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f7241d9b640 1 -- 172.21.15.103:0/2365308124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047c80 msgr2=0x563b9b056550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:49.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f7241d9b640 1 --2- 172.21.15.103:0/2365308124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047c80 0x563b9b056550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f7241d9b640 1 -- 172.21.15.103:0/2365308124 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b9af39440 con 0x563b9b047360 2024-09-19T06:53:49.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f7241d9b640 1 --2- 172.21.15.103:0/2365308124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b047740 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f723800bde0 tx=0x7f72380041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aeb62783b6e7d99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:49.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f7242d9d640 1 --2- 172.21.15.103:0/189533978 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f722c0a33b0 0x7f722c0a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f723c0f0510 tx=0x7f723c0095c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:49.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f7230ff9640 1 -- 172.21.15.103:0/2365308124 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238003e80 con 0x563b9b047360 2024-09-19T06:53:49.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f7230ff9640 1 -- 172.21.15.103:0/2365308124 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7238005500 con 0x563b9b047360 2024-09-19T06:53:49.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.732+0000 7f7230ff9640 1 -- 172.21.15.103:0/2365308124 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238012450 con 0x563b9b047360 2024-09-19T06:53:49.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.733+0000 7f724b2ef740 1 -- 172.21.15.103:0/2365308124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 msgr2=0x563b9b047740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:49.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.733+0000 7f724b2ef740 1 --2- 172.21.15.103:0/2365308124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b047740 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f723800bde0 tx=0x7f72380041a0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.733+0000 7f724b2ef740 1 -- 172.21.15.103:0/2365308124 shutdown_connections 2024-09-19T06:53:49.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.733+0000 7f724b2ef740 1 --2- 172.21.15.103:0/2365308124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b056a90 0x563b9b058e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.733+0000 7f724b2ef740 1 --2- 172.21.15.103:0/2365308124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047c80 0x563b9b056550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.733+0000 7f724b2ef740 1 --2- 172.21.15.103:0/2365308124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b047740 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.733+0000 7f724b2ef740 1 -- 172.21.15.103:0/2365308124 >> 172.21.15.103:0/2365308124 conn(0x563b9b043a30 msgr2=0x563b9b043e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:49.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.733+0000 7f724b2ef740 1 -- 172.21.15.103:0/2365308124 shutdown_connections 2024-09-19T06:53:49.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.733+0000 7f724b2ef740 1 -- 172.21.15.103:0/2365308124 wait complete. 2024-09-19T06:53:49.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.734+0000 7f724b2ef740 1 -- start start 2024-09-19T06:53:49.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.734+0000 7f724b2ef740 1 Processor -- start 2024-09-19T06:53:49.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.734+0000 7f724b2ef740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047360 0x563b9b0563c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.734+0000 7f724b2ef740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b047c80 0x563b9b053670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.734+0000 7f724b2ef740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b056a90 0x563b9b053bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.734+0000 7f724b2ef740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563b9b0595a0 con 0x563b9b056a90 2024-09-19T06:53:49.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.734+0000 7f724b2ef740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563b9af68050 con 0x563b9b047360 2024-09-19T06:53:49.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.734+0000 7f724b2ef740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b9b059720 con 0x563b9b047c80 2024-09-19T06:53:49.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.734+0000 7f724b2ef740 10 client.0 _ll_register_callbacks cb 0x563b9afa8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:49.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.734+0000 7f7241d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b056a90 0x563b9b053bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:49.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.734+0000 7f7241d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b056a90 0x563b9b053bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40944/0 (socket says 172.21.15.103:40944) 2024-09-19T06:53:49.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.734+0000 7f7241d9b640 1 -- 172.21.15.103:0/1271133522 learned_addr learned my addr 172.21.15.103:0/1271133522 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:49.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.734+0000 7f724259c640 1 --2- 172.21.15.103:0/1271133522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b047c80 0x563b9b053670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:49.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.734+0000 7f7241d9b640 1 -- 172.21.15.103:0/1271133522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b047c80 msgr2=0x563b9b053670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:49.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.734+0000 7f7241d9b640 1 --2- 172.21.15.103:0/1271133522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b047c80 0x563b9b053670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f7241d9b640 1 -- 172.21.15.103:0/1271133522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047360 msgr2=0x563b9b0563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:49.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f7241d9b640 1 --2- 172.21.15.103:0/1271133522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047360 0x563b9b0563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f7241d9b640 1 -- 172.21.15.103:0/1271133522 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b9ad1e3c0 con 0x563b9b056a90 2024-09-19T06:53:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f7241d9b640 1 --2- 172.21.15.103:0/1271133522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b056a90 0x563b9b053bb0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f723800b400 tx=0x7f723800b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85466f3d917587 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f7229ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f7229ffb640 1 -- 172.21.15.103:0/1271133522 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238003e80 con 0x563b9b056a90 2024-09-19T06:53:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f7229ffb640 1 -- 172.21.15.103:0/1271133522 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f7238005000 con 0x563b9b056a90 2024-09-19T06:53:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f724b2ef740 1 -- 172.21.15.103:0/1271133522 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7238005500 con 0x563b9b056a90 2024-09-19T06:53:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f7229ffb640 1 -- 172.21.15.103:0/1271133522 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f723802f050 con 0x563b9b056a90 2024-09-19T06:53:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f724b2ef740 10 client.17112 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f724b2ef740 1 -- 172.21.15.103:0/1271133522 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563b9b064f90 con 0x563b9b056a90 2024-09-19T06:53:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f7229ffb640 1 -- 172.21.15.103:0/1271133522 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7238005500 con 0x563b9b056a90 2024-09-19T06:53:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f7229ffb640 1 -- 172.21.15.103:0/1271133522 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7238012450 con 0x563b9b056a90 2024-09-19T06:53:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f724b2ef740 10 client.17112 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f724b2ef740 10 client.17112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f724b2ef740 20 client.17112 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f724b2ef740 1 --2- 172.21.15.103:0/1271133522 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563b9b066330 0x563b9b068720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f724b2ef740 4 client.17112 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.735+0000 7f724b2ef740 1 -- 172.21.15.103:0/1271133522 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b9afba850 con 0x563b9b066330 2024-09-19T06:53:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.736+0000 7f724259c640 1 --2- 172.21.15.103:0/1271133522 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563b9b066330 0x563b9b068720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:49.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.736+0000 7f7229ffb640 10 client.17112 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:49.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.736+0000 7f724259c640 1 --2- 172.21.15.103:0/1271133522 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563b9b066330 0x563b9b068720 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d386695eecc97bec server_cookie=6be67f2223b0dcc0 in_seq=0 out_seq=0 2024-09-19T06:53:49.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.737+0000 7f7229ffb640 1 -- 172.21.15.103:0/1271133522 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b9afba850 con 0x563b9b066330 2024-09-19T06:53:49.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.737+0000 7f7229ffb640 10 client.17112 handle_command_reply: tid=0 2024-09-19T06:53:49.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.737+0000 7f724b2ef740 1 client.17112 shutdown 2024-09-19T06:53:49.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.737+0000 7f724b2ef740 1 -- 172.21.15.103:0/1271133522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b056a90 msgr2=0x563b9b053bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:49.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.737+0000 7f724b2ef740 1 --2- 172.21.15.103:0/1271133522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b056a90 0x563b9b053bb0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f723800b400 tx=0x7f723800b430 comp rx=0 tx=0).stop 2024-09-19T06:53:49.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.737+0000 7f724b2ef740 1 -- 172.21.15.103:0/1271133522 shutdown_connections 2024-09-19T06:53:49.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.737+0000 7f724b2ef740 1 --2- 172.21.15.103:0/1271133522 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563b9b066330 0x563b9b068720 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.737+0000 7f724b2ef740 1 --2- 172.21.15.103:0/1271133522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b056a90 0x563b9b053bb0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.737+0000 7f724b2ef740 1 --2- 172.21.15.103:0/1271133522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b047c80 0x563b9b053670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.737+0000 7f724b2ef740 1 --2- 172.21.15.103:0/1271133522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047360 0x563b9b0563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.737+0000 7f724b2ef740 1 -- 172.21.15.103:0/1271133522 >> 172.21.15.103:0/1271133522 conn(0x563b9b043a30 msgr2=0x563b9b042b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:49.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.737+0000 7f7229ffb640 0 client.17112 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:49.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.737+0000 7f724b2ef740 1 -- 172.21.15.103:0/1271133522 shutdown_connections 2024-09-19T06:53:49.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.737+0000 7f724b2ef740 1 -- 172.21.15.103:0/1271133522 wait complete. 2024-09-19T06:53:49.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.737+0000 7f724b2ef740 20 client.17112 trim_cache size 0 max 16384 2024-09-19T06:53:49.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.754+0000 7f724b2ef740 1 Processor -- start 2024-09-19T06:53:49.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.754+0000 7f724b2ef740 1 -- start start 2024-09-19T06:53:49.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.754+0000 7f724b2ef740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b043a30 0x563b9b062840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.754+0000 7f724b2ef740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b051f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.754+0000 7f724b2ef740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047c80 0x563b9b0555e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.754+0000 7f724b2ef740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72441ab5a0 con 0x563b9b047360 2024-09-19T06:53:49.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.754+0000 7f724b2ef740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f72441ad0e0 con 0x563b9b047c80 2024-09-19T06:53:49.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.754+0000 7f724b2ef740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f72441abc70 con 0x563b9b043a30 2024-09-19T06:53:49.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.755+0000 7f724259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b051f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:49.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.755+0000 7f724259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b051f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40954/0 (socket says 172.21.15.103:40954) 2024-09-19T06:53:49.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.755+0000 7f724259c640 1 -- 172.21.15.103:0/2002542919 learned_addr learned my addr 172.21.15.103:0/2002542919 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:49.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.755+0000 7f7241d9b640 1 --2- 172.21.15.103:0/2002542919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047c80 0x563b9b0555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:49.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.755+0000 7f7241d9b640 1 --2- 172.21.15.103:0/2002542919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b043a30 0x563b9b062840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:49.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.755+0000 7f724259c640 1 -- 172.21.15.103:0/2002542919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b043a30 msgr2=0x563b9b062840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:49.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.755+0000 7f724259c640 1 --2- 172.21.15.103:0/2002542919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b043a30 0x563b9b062840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.755+0000 7f724259c640 1 -- 172.21.15.103:0/2002542919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047c80 msgr2=0x563b9b0555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:49.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.755+0000 7f724259c640 1 --2- 172.21.15.103:0/2002542919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047c80 0x563b9b0555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.755+0000 7f724259c640 1 -- 172.21.15.103:0/2002542919 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b9b052620 con 0x563b9b047360 2024-09-19T06:53:49.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.755+0000 7f724259c640 1 --2- 172.21.15.103:0/2002542919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b051f40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f7234000c00 tx=0x7f723400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6a39003be2dd31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:49.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.755+0000 7f722bfff640 1 -- 172.21.15.103:0/2002542919 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7234003d20 con 0x563b9b047360 2024-09-19T06:53:49.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.755+0000 7f722bfff640 1 -- 172.21.15.103:0/2002542919 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7234003ec0 con 0x563b9b047360 2024-09-19T06:53:49.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.755+0000 7f722bfff640 1 -- 172.21.15.103:0/2002542919 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7234011870 con 0x563b9b047360 2024-09-19T06:53:49.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.756+0000 7f724b2ef740 1 -- 172.21.15.103:0/2002542919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 msgr2=0x563b9b051f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:49.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.756+0000 7f724b2ef740 1 --2- 172.21.15.103:0/2002542919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b051f40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f7234000c00 tx=0x7f723400b390 comp rx=0 tx=0).stop 2024-09-19T06:53:49.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.756+0000 7f724b2ef740 1 -- 172.21.15.103:0/2002542919 shutdown_connections 2024-09-19T06:53:49.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.756+0000 7f724b2ef740 1 --2- 172.21.15.103:0/2002542919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047c80 0x563b9b0555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.756+0000 7f724b2ef740 1 --2- 172.21.15.103:0/2002542919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b051f40 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.756+0000 7f724b2ef740 1 --2- 172.21.15.103:0/2002542919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b043a30 0x563b9b062840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.756+0000 7f724b2ef740 1 -- 172.21.15.103:0/2002542919 >> 172.21.15.103:0/2002542919 conn(0x563b9b053d10 msgr2=0x563b9b054ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:49.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.756+0000 7f724b2ef740 1 -- 172.21.15.103:0/2002542919 shutdown_connections 2024-09-19T06:53:49.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.756+0000 7f724b2ef740 1 -- 172.21.15.103:0/2002542919 wait complete. 2024-09-19T06:53:49.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.756+0000 7f724b2ef740 1 -- start start 2024-09-19T06:53:49.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.756+0000 7f724b2ef740 1 Processor -- start 2024-09-19T06:53:49.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.756+0000 7f724b2ef740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b043a30 0x563b9b05a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.756+0000 7f724b2ef740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b05ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.756+0000 7f724b2ef740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047c80 0x563b9b05b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f724b2ef740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72441aca10 con 0x563b9b047360 2024-09-19T06:53:49.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f724b2ef740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f72441ac340 con 0x563b9b047c80 2024-09-19T06:53:49.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f724b2ef740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f72441a95a0 con 0x563b9b043a30 2024-09-19T06:53:49.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f724b2ef740 10 client.0 _ll_register_callbacks cb 0x7f723400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:49.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f724259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b05ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:49.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f724259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b05ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40968/0 (socket says 172.21.15.103:40968) 2024-09-19T06:53:49.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f724259c640 1 -- 172.21.15.103:0/544518879 learned_addr learned my addr 172.21.15.103:0/544518879 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:49.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f7241d9b640 1 --2- 172.21.15.103:0/544518879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b043a30 0x563b9b05a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:49.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f724259c640 1 -- 172.21.15.103:0/544518879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b043a30 msgr2=0x563b9b05a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f724259c640 1 --2- 172.21.15.103:0/544518879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b043a30 0x563b9b05a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f724259c640 1 -- 172.21.15.103:0/544518879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047c80 msgr2=0x563b9b05b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f724259c640 1 --2- 172.21.15.103:0/544518879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047c80 0x563b9b05b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f724259c640 1 -- 172.21.15.103:0/544518879 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b9b0cfeb0 con 0x563b9b047360 2024-09-19T06:53:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f7241d9b640 1 --2- 172.21.15.103:0/544518879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b043a30 0x563b9b05a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f724259c640 1 --2- 172.21.15.103:0/544518879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b05ac70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f723400b740 tx=0x7f7234003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea30a6e097b66c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f7229ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f7229ffb640 1 -- 172.21.15.103:0/544518879 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7234005440 con 0x563b9b047360 2024-09-19T06:53:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f7229ffb640 1 -- 172.21.15.103:0/544518879 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f7234004730 con 0x563b9b047360 2024-09-19T06:53:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f724b2ef740 1 -- 172.21.15.103:0/544518879 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7234003ec0 con 0x563b9b047360 2024-09-19T06:53:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f7229ffb640 1 -- 172.21.15.103:0/544518879 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7234003d10 con 0x563b9b047360 2024-09-19T06:53:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f724b2ef740 10 client.17118 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.757+0000 7f724b2ef740 1 -- 172.21.15.103:0/544518879 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563b9b0d1180 con 0x563b9b047360 2024-09-19T06:53:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.758+0000 7f7229ffb640 1 -- 172.21.15.103:0/544518879 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7234003ec0 con 0x563b9b047360 2024-09-19T06:53:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.758+0000 7f7229ffb640 1 -- 172.21.15.103:0/544518879 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7234024020 con 0x563b9b047360 2024-09-19T06:53:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.758+0000 7f724b2ef740 10 client.17118 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.758+0000 7f724b2ef740 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.758+0000 7f724b2ef740 20 client.17118 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.758+0000 7f724b2ef740 1 --2- 172.21.15.103:0/544518879 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563b9b056a90 0x563b9b0d3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.758+0000 7f724b2ef740 4 client.17118 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:53:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.758+0000 7f724b2ef740 1 -- 172.21.15.103:0/544518879 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b9afb4d60 con 0x563b9b056a90 2024-09-19T06:53:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.758+0000 7f7241d9b640 1 --2- 172.21.15.103:0/544518879 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563b9b056a90 0x563b9b0d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f7229ffb640 10 client.17118 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f7241d9b640 1 --2- 172.21.15.103:0/544518879 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563b9b056a90 0x563b9b0d3900 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf9dca0b221f0f4 server_cookie=f86d8083586e4179 in_seq=0 out_seq=0 2024-09-19T06:53:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f7229ffb640 1 -- 172.21.15.103:0/544518879 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563b9afb4d60 con 0x563b9b056a90 2024-09-19T06:53:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f7229ffb640 10 client.17118 handle_command_reply: tid=0 2024-09-19T06:53:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f724b2ef740 1 client.17118 shutdown 2024-09-19T06:53:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f724b2ef740 1 -- 172.21.15.103:0/544518879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 msgr2=0x563b9b05ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f724b2ef740 1 --2- 172.21.15.103:0/544518879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b05ac70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f723400b740 tx=0x7f7234003b20 comp rx=0 tx=0).stop 2024-09-19T06:53:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f724b2ef740 1 -- 172.21.15.103:0/544518879 shutdown_connections 2024-09-19T06:53:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f724b2ef740 1 --2- 172.21.15.103:0/544518879 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563b9b056a90 0x563b9b0d3900 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f724b2ef740 1 --2- 172.21.15.103:0/544518879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b9b047c80 0x563b9b05b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f724b2ef740 1 --2- 172.21.15.103:0/544518879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b9b047360 0x563b9b05ac70 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f724b2ef740 1 --2- 172.21.15.103:0/544518879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b043a30 0x563b9b05a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f724b2ef740 1 -- 172.21.15.103:0/544518879 >> 172.21.15.103:0/544518879 conn(0x563b9b053d10 msgr2=0x563b9b054ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:49.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f7229ffb640 0 client.17118 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:49.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f724b2ef740 1 -- 172.21.15.103:0/544518879 shutdown_connections 2024-09-19T06:53:49.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f724b2ef740 1 -- 172.21.15.103:0/544518879 wait complete. 2024-09-19T06:53:49.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.759+0000 7f724b2ef740 20 client.17118 trim_cache size 0 max 16384 2024-09-19T06:53:49.767 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:53:49.767 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T06:53:49.767 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T06:53:49.767 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:53:49.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.760+0000 7f7248f16640 1 -- 172.21.15.103:0/189533978 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f722c0a33b0 msgr2=0x7f722c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:49.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.760+0000 7f7248f16640 1 --2- 172.21.15.103:0/189533978 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f722c0a33b0 0x7f722c0a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f723c0f0510 tx=0x7f723c0095c0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.760+0000 7f7248f16640 1 -- 172.21.15.103:0/189533978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7244102ef0 msgr2=0x7f724419bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.760+0000 7f7248f16640 1 --2- 172.21.15.103:0/189533978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7244102ef0 0x7f724419bcd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f723c02ae70 tx=0x7f723c02f5d0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.760+0000 7f7248f16640 1 -- 172.21.15.103:0/189533978 shutdown_connections 2024-09-19T06:53:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.760+0000 7f7248f16640 1 --2- 172.21.15.103:0/189533978 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f722c0a33b0 0x7f722c0a5870 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.760+0000 7f7248f16640 1 --2- 172.21.15.103:0/189533978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7244102ef0 0x7f724419bcd0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.760+0000 7f7248f16640 1 --2- 172.21.15.103:0/189533978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7244102570 0x7f7244197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.760+0000 7f7248f16640 1 --2- 172.21.15.103:0/189533978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7244100e20 0x7f7244197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.760+0000 7f7248f16640 1 -- 172.21.15.103:0/189533978 >> 172.21.15.103:0/189533978 conn(0x7f72440fc7d0 msgr2=0x7f7244106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.760+0000 7f7248f16640 1 -- 172.21.15.103:0/189533978 shutdown_connections 2024-09-19T06:53:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:49.760+0000 7f7248f16640 1 -- 172.21.15.103:0/189533978 wait complete. 2024-09-19T06:53:50.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:50 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T06:53:50.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:50 smithi174 ceph-mon[35637]: pgmap v141: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 19 op/s 2024-09-19T06:53:50.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:50 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T06:53:50.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:50 smithi103 ceph-mon[31158]: pgmap v141: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 19 op/s 2024-09-19T06:53:50.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:50 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T06:53:50.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:50 smithi134 ceph-mon[35356]: pgmap v141: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 19 op/s 2024-09-19T06:53:50.505 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:53:50.506 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:53:50.750 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:51.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.210+0000 7f75e26a5640 1 -- 172.21.15.103:0/1204139488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75dc106a80 msgr2=0x7f75dc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:51.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.210+0000 7f75e26a5640 1 --2- 172.21.15.103:0/1204139488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75dc106a80 0x7f75dc106e60 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009930 tx=0x7f75cc02a420 comp rx=0 tx=0).stop 2024-09-19T06:53:51.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.210+0000 7f75e26a5640 1 -- 172.21.15.103:0/1204139488 shutdown_connections 2024-09-19T06:53:51.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.210+0000 7f75e26a5640 1 --2- 172.21.15.103:0/1204139488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75dc101420 0x7f75dc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:51.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.210+0000 7f75e26a5640 1 --2- 172.21.15.103:0/1204139488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75dc100a80 0x7f75dc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:51.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.210+0000 7f75e26a5640 1 --2- 172.21.15.103:0/1204139488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75dc106a80 0x7f75dc106e60 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:51.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.210+0000 7f75e26a5640 1 -- 172.21.15.103:0/1204139488 >> 172.21.15.103:0/1204139488 conn(0x7f75dc0fc7a0 msgr2=0x7f75dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:51.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.210+0000 7f75e26a5640 1 -- 172.21.15.103:0/1204139488 shutdown_connections 2024-09-19T06:53:51.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.210+0000 7f75e26a5640 1 -- 172.21.15.103:0/1204139488 wait complete. 2024-09-19T06:53:51.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.210+0000 7f75e26a5640 1 Processor -- start 2024-09-19T06:53:51.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.210+0000 7f75e26a5640 1 -- start start 2024-09-19T06:53:51.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75e26a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75dc100a80 0x7f75dc199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:51.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75e26a5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75dc101420 0x7f75dc1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:51.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75e26a5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75dc106a80 0x7f75dc1a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:51.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75e26a5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc110f20 con 0x7f75dc106a80 2024-09-19T06:53:51.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75e26a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc110da0 con 0x7f75dc100a80 2024-09-19T06:53:51.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75e26a5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc1110a0 con 0x7f75dc101420 2024-09-19T06:53:51.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75e0c42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75dc106a80 0x7f75dc1a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:51.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75e0c42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75dc106a80 0x7f75dc1a44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40986/0 (socket says 172.21.15.103:40986) 2024-09-19T06:53:51.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75e0c42640 1 -- 172.21.15.103:0/3143749673 learned_addr learned my addr 172.21.15.103:0/3143749673 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:51.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75db7fe640 1 --2- 172.21.15.103:0/3143749673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75dc101420 0x7f75dc1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:51.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75dbfff640 1 --2- 172.21.15.103:0/3143749673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75dc100a80 0x7f75dc199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:51.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75e0c42640 1 -- 172.21.15.103:0/3143749673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75dc101420 msgr2=0x7f75dc1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:51.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75e0c42640 1 --2- 172.21.15.103:0/3143749673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75dc101420 0x7f75dc1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:51.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75e0c42640 1 -- 172.21.15.103:0/3143749673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75dc100a80 msgr2=0x7f75dc199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:51.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75e0c42640 1 --2- 172.21.15.103:0/3143749673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75dc100a80 0x7f75dc199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:51.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75e0c42640 1 -- 172.21.15.103:0/3143749673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc1a4a20 con 0x7f75dc106a80 2024-09-19T06:53:51.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75dbfff640 1 --2- 172.21.15.103:0/3143749673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75dc100a80 0x7f75dc199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:51.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75e0c42640 1 --2- 172.21.15.103:0/3143749673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75dc106a80 0x7f75dc1a44e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f75d400b970 tx=0x7f75d400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:51.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75d97fa640 1 -- 172.21.15.103:0/3143749673 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d4019070 con 0x7f75dc106a80 2024-09-19T06:53:51.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.211+0000 7f75e26a5640 1 -- 172.21.15.103:0/3143749673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75dc1a4bf0 con 0x7f75dc106a80 2024-09-19T06:53:51.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.212+0000 7f75d97fa640 1 -- 172.21.15.103:0/3143749673 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75d4014440 con 0x7f75dc106a80 2024-09-19T06:53:51.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.212+0000 7f75d97fa640 1 -- 172.21.15.103:0/3143749673 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d40075d0 con 0x7f75dc106a80 2024-09-19T06:53:51.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.212+0000 7f75e26a5640 1 -- 172.21.15.103:0/3143749673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75dc1a5130 con 0x7f75dc106a80 2024-09-19T06:53:51.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.213+0000 7f75e26a5640 1 -- 172.21.15.103:0/3143749673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75dc100f60 con 0x7f75dc106a80 2024-09-19T06:53:51.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.214+0000 7f75d97fa640 1 -- 172.21.15.103:0/3143749673 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f75d401c070 con 0x7f75dc106a80 2024-09-19T06:53:51.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.214+0000 7f75d97fa640 1 --2- 172.21.15.103:0/3143749673 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75c40a33b0 0x7f75c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:51.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.215+0000 7f75dbfff640 1 --2- 172.21.15.103:0/3143749673 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75c40a33b0 0x7f75c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:51.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.215+0000 7f75d97fa640 1 -- 172.21.15.103:0/3143749673 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f75d40cf6e0 con 0x7f75dc106a80 2024-09-19T06:53:51.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.215+0000 7f75dbfff640 1 --2- 172.21.15.103:0/3143749673 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75c40a33b0 0x7f75c40a5870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f75cc000c00 tx=0x7f75cc032d50 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:51.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.218+0000 7f75d97fa640 1 -- 172.21.15.103:0/3143749673 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f75d409c1a0 con 0x7f75dc106a80 2024-09-19T06:53:51.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.420+0000 7f75e26a5640 1 -- 172.21.15.103:0/3143749673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75dc109050 con 0x7f75dc106a80 2024-09-19T06:53:51.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.421+0000 7f75d97fa640 1 -- 172.21.15.103:0/3143749673 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f75d409baa0 con 0x7f75dc106a80 2024-09-19T06:53:51.422 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:53:51.422 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:53:51.422 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:53:51.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.423+0000 7f75e26a5640 1 -- 172.21.15.103:0/3143749673 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75c40a33b0 msgr2=0x7f75c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:51.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.423+0000 7f75e26a5640 1 --2- 172.21.15.103:0/3143749673 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75c40a33b0 0x7f75c40a5870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f75cc000c00 tx=0x7f75cc032d50 comp rx=0 tx=0).stop 2024-09-19T06:53:51.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.423+0000 7f75e26a5640 1 -- 172.21.15.103:0/3143749673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75dc106a80 msgr2=0x7f75dc1a44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:51.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.423+0000 7f75e26a5640 1 --2- 172.21.15.103:0/3143749673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75dc106a80 0x7f75dc1a44e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f75d400b970 tx=0x7f75d400bd70 comp rx=0 tx=0).stop 2024-09-19T06:53:51.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.423+0000 7f75e26a5640 1 -- 172.21.15.103:0/3143749673 shutdown_connections 2024-09-19T06:53:51.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.423+0000 7f75e26a5640 1 --2- 172.21.15.103:0/3143749673 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75c40a33b0 0x7f75c40a5870 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:51.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.423+0000 7f75e26a5640 1 --2- 172.21.15.103:0/3143749673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75dc106a80 0x7f75dc1a44e0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:51.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.423+0000 7f75e26a5640 1 --2- 172.21.15.103:0/3143749673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75dc101420 0x7f75dc1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:51.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.423+0000 7f75e26a5640 1 --2- 172.21.15.103:0/3143749673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75dc100a80 0x7f75dc199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:51.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.423+0000 7f75e26a5640 1 -- 172.21.15.103:0/3143749673 >> 172.21.15.103:0/3143749673 conn(0x7f75dc0fc7a0 msgr2=0x7f75dc0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:51.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.423+0000 7f75e26a5640 1 -- 172.21.15.103:0/3143749673 shutdown_connections 2024-09-19T06:53:51.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:51.423+0000 7f75e26a5640 1 -- 172.21.15.103:0/3143749673 wait complete. 2024-09-19T06:53:51.717 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:51 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3143749673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:53:51.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:51 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3143749673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:53:51.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:51 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3143749673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:53:52.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:53:52.460 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:52.732 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:52 smithi103 ceph-mon[31158]: pgmap v142: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.5 KiB/s wr, 12 op/s 2024-09-19T06:53:52.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:52 smithi174 ceph-mon[35637]: pgmap v142: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.5 KiB/s wr, 12 op/s 2024-09-19T06:53:52.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.897+0000 7fde706c4640 1 -- 172.21.15.103:0/2172506906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde68101de0 msgr2=0x7fde681021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.897+0000 7fde706c4640 1 --2- 172.21.15.103:0/2172506906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde68101de0 0x7fde681021c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fde60009900 tx=0x7fde6002a410 comp rx=0 tx=0).stop 2024-09-19T06:53:52.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.897+0000 7fde706c4640 1 -- 172.21.15.103:0/2172506906 shutdown_connections 2024-09-19T06:53:52.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.897+0000 7fde706c4640 1 --2- 172.21.15.103:0/2172506906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde68106f10 0x7fde6810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.897+0000 7fde706c4640 1 --2- 172.21.15.103:0/2172506906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde68102790 0x7fde681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.897+0000 7fde706c4640 1 --2- 172.21.15.103:0/2172506906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde68101de0 0x7fde681021c0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.897+0000 7fde706c4640 1 -- 172.21.15.103:0/2172506906 >> 172.21.15.103:0/2172506906 conn(0x7fde680fd8c0 msgr2=0x7fde680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:52.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.897+0000 7fde706c4640 1 -- 172.21.15.103:0/2172506906 shutdown_connections 2024-09-19T06:53:52.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.897+0000 7fde706c4640 1 -- 172.21.15.103:0/2172506906 wait complete. 2024-09-19T06:53:52.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.897+0000 7fde706c4640 1 Processor -- start 2024-09-19T06:53:52.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.897+0000 7fde706c4640 1 -- start start 2024-09-19T06:53:52.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde706c4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde68101de0 0x7fde680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde706c4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde68102790 0x7fde68074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde706c4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde68106f10 0x7fde68074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde706c4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde68111fb0 con 0x7fde68102790 2024-09-19T06:53:52.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde706c4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde68111e30 con 0x7fde68101de0 2024-09-19T06:53:52.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde706c4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde68112130 con 0x7fde68106f10 2024-09-19T06:53:52.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde6dc5f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde68102790 0x7fde68074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde6dc5f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde68102790 0x7fde68074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41010/0 (socket says 172.21.15.103:41010) 2024-09-19T06:53:52.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde6dc5f640 1 -- 172.21.15.103:0/312314630 learned_addr learned my addr 172.21.15.103:0/312314630 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:52.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde6ec61640 1 --2- 172.21.15.103:0/312314630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde68106f10 0x7fde68074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde6e460640 1 --2- 172.21.15.103:0/312314630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde68101de0 0x7fde680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde6dc5f640 1 -- 172.21.15.103:0/312314630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde68106f10 msgr2=0x7fde68074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde6dc5f640 1 --2- 172.21.15.103:0/312314630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde68106f10 0x7fde68074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde6dc5f640 1 -- 172.21.15.103:0/312314630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde68101de0 msgr2=0x7fde680744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde6dc5f640 1 --2- 172.21.15.103:0/312314630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde68101de0 0x7fde680744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde6dc5f640 1 -- 172.21.15.103:0/312314630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde681a6960 con 0x7fde68102790 2024-09-19T06:53:52.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde6dc5f640 1 --2- 172.21.15.103:0/312314630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde68102790 0x7fde68074a30 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00d770 tx=0x7fde5c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:52.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde677fe640 1 -- 172.21.15.103:0/312314630 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c017070 con 0x7fde68102790 2024-09-19T06:53:52.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.898+0000 7fde706c4640 1 -- 172.21.15.103:0/312314630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde681a6c50 con 0x7fde68102790 2024-09-19T06:53:52.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.899+0000 7fde677fe640 1 -- 172.21.15.103:0/312314630 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde5c005500 con 0x7fde68102790 2024-09-19T06:53:52.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.899+0000 7fde677fe640 1 -- 172.21.15.103:0/312314630 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c004e00 con 0x7fde68102790 2024-09-19T06:53:52.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.899+0000 7fde706c4640 1 -- 172.21.15.103:0/312314630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde681a7190 con 0x7fde68102790 2024-09-19T06:53:52.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.901+0000 7fde677fe640 1 -- 172.21.15.103:0/312314630 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fde5c004fa0 con 0x7fde68102790 2024-09-19T06:53:52.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.901+0000 7fde72a9d740 1 Processor -- start 2024-09-19T06:53:52.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.901+0000 7fde72a9d740 1 -- start start 2024-09-19T06:53:52.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.901+0000 7fde72a9d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 0x55f2d480a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.901+0000 7fde72a9d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d480a5d0 0x55f2d480aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.901+0000 7fde72a9d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4815370 0x55f2d4817760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.901+0000 7fde72a9d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f2d47702f0 con 0x55f2d4815370 2024-09-19T06:53:52.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.901+0000 7fde72a9d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f2d4717cf0 con 0x55f2d480a5d0 2024-09-19T06:53:52.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.901+0000 7fde72a9d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f2d476c2b0 con 0x55f2d480bb40 2024-09-19T06:53:52.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.901+0000 7fde6dc5f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4815370 0x55f2d4817760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde6dc5f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4815370 0x55f2d4817760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41022/0 (socket says 172.21.15.103:41022) 2024-09-19T06:53:52.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde6dc5f640 1 -- 172.21.15.103:0/798082752 learned_addr learned my addr 172.21.15.103:0/798082752 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:52.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde6ec61640 1 --2- 172.21.15.103:0/798082752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 0x55f2d480a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde6e460640 1 --2- 172.21.15.103:0/798082752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d480a5d0 0x55f2d480aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde6dc5f640 1 -- 172.21.15.103:0/798082752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 msgr2=0x55f2d480a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:52.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde6dc5f640 1 --2- 172.21.15.103:0/798082752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 0x55f2d480a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde6dc5f640 1 -- 172.21.15.103:0/798082752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d480a5d0 msgr2=0x55f2d480aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:52.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde6dc5f640 1 --2- 172.21.15.103:0/798082752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d480a5d0 0x55f2d480aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde6dc5f640 1 -- 172.21.15.103:0/798082752 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2d46fc3a0 con 0x55f2d4815370 2024-09-19T06:53:52.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde6e460640 1 --2- 172.21.15.103:0/798082752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d480a5d0 0x55f2d480aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:52.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde677fe640 1 --2- 172.21.15.103:0/312314630 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fde500a33b0 0x7fde500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde6ec61640 1 --2- 172.21.15.103:0/798082752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 0x55f2d480a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:52.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde6ec61640 1 --2- 172.21.15.103:0/312314630 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fde500a33b0 0x7fde500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde6dc5f640 1 --2- 172.21.15.103:0/798082752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4815370 0x55f2d4817760 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0ccd80 tx=0x7fde5c0d7670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca43cb4728a1777c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:52.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde677fe640 1 -- 172.21.15.103:0/312314630 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fde5c0cf6e0 con 0x7fde68102790 2024-09-19T06:53:52.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde64ff9640 1 -- 172.21.15.103:0/798082752 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c020020 con 0x55f2d4815370 2024-09-19T06:53:52.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde64ff9640 1 -- 172.21.15.103:0/798082752 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde5c0046c0 con 0x55f2d4815370 2024-09-19T06:53:52.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.902+0000 7fde64ff9640 1 -- 172.21.15.103:0/798082752 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c06e070 con 0x55f2d4815370 2024-09-19T06:53:52.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.903+0000 7fde72a9d740 1 -- 172.21.15.103:0/798082752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4815370 msgr2=0x55f2d4817760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.903+0000 7fde72a9d740 1 --2- 172.21.15.103:0/798082752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4815370 0x55f2d4817760 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0ccd80 tx=0x7fde5c0d7670 comp rx=0 tx=0).stop 2024-09-19T06:53:52.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.903+0000 7fde6ec61640 1 --2- 172.21.15.103:0/312314630 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fde500a33b0 0x7fde500a5870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fde58009fd0 tx=0x7fde5800d040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:52.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.903+0000 7fde72a9d740 1 -- 172.21.15.103:0/798082752 shutdown_connections 2024-09-19T06:53:52.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.903+0000 7fde72a9d740 1 --2- 172.21.15.103:0/798082752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4815370 0x55f2d4817760 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.903+0000 7fde72a9d740 1 --2- 172.21.15.103:0/798082752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d480a5d0 0x55f2d480aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.903+0000 7fde72a9d740 1 --2- 172.21.15.103:0/798082752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 0x55f2d480a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.903+0000 7fde72a9d740 1 -- 172.21.15.103:0/798082752 >> 172.21.15.103:0/798082752 conn(0x55f2d47d3310 msgr2=0x55f2d4806c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:52.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.903+0000 7fde72a9d740 1 -- 172.21.15.103:0/798082752 shutdown_connections 2024-09-19T06:53:52.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.903+0000 7fde72a9d740 1 -- 172.21.15.103:0/798082752 wait complete. 2024-09-19T06:53:52.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde72a9d740 1 -- start start 2024-09-19T06:53:52.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde72a9d740 1 Processor -- start 2024-09-19T06:53:52.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde72a9d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d4814d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde72a9d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 0x55f2d4812030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde72a9d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d4815370 0x55f2d4812640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde72a9d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f2d4818010 con 0x55f2d480a5d0 2024-09-19T06:53:52.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde72a9d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f2d472b060 con 0x55f2d4815370 2024-09-19T06:53:52.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde72a9d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f2d4818190 con 0x55f2d480bb40 2024-09-19T06:53:52.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde72a9d740 10 client.0 _ll_register_callbacks cb 0x55f2d447e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:52.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde6e460640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d4814d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde6e460640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d4814d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41030/0 (socket says 172.21.15.103:41030) 2024-09-19T06:53:52.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde6e460640 1 -- 172.21.15.103:0/3070100318 learned_addr learned my addr 172.21.15.103:0/3070100318 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:52.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde6dc5f640 1 --2- 172.21.15.103:0/3070100318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 0x55f2d4812030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde6e460640 1 -- 172.21.15.103:0/3070100318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 msgr2=0x55f2d4812030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:52.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde6e460640 1 --2- 172.21.15.103:0/3070100318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 0x55f2d4812030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde6ec61640 1 --2- 172.21.15.103:0/3070100318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d4815370 0x55f2d4812640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde6e460640 1 -- 172.21.15.103:0/3070100318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d4815370 msgr2=0x55f2d4812640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:52.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde6e460640 1 --2- 172.21.15.103:0/3070100318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d4815370 0x55f2d4812640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.904+0000 7fde6e460640 1 -- 172.21.15.103:0/3070100318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2d44e13c0 con 0x55f2d480a5d0 2024-09-19T06:53:52.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.905+0000 7fde6e460640 1 --2- 172.21.15.103:0/3070100318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d4814d80 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fde6002a930 tx=0x7fde600318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef53f6a5257f8252 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:52.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.905+0000 7fde4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:52.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.905+0000 7fde4dffb640 1 -- 172.21.15.103:0/3070100318 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60035040 con 0x55f2d480a5d0 2024-09-19T06:53:52.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.905+0000 7fde4dffb640 1 -- 172.21.15.103:0/3070100318 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fde60004240 con 0x55f2d480a5d0 2024-09-19T06:53:52.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.905+0000 7fde72a9d740 1 -- 172.21.15.103:0/3070100318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f2d477d860 con 0x55f2d480a5d0 2024-09-19T06:53:52.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.905+0000 7fde4dffb640 1 -- 172.21.15.103:0/3070100318 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fde60004440 con 0x55f2d480a5d0 2024-09-19T06:53:52.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.905+0000 7fde72a9d740 10 client.17136 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:52.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.905+0000 7fde72a9d740 1 -- 172.21.15.103:0/3070100318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f2d4823b30 con 0x55f2d480a5d0 2024-09-19T06:53:52.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.905+0000 7fde4dffb640 1 -- 172.21.15.103:0/3070100318 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f2d477d860 con 0x55f2d480a5d0 2024-09-19T06:53:52.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.905+0000 7fde4dffb640 1 -- 172.21.15.103:0/3070100318 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fde600035f0 con 0x55f2d480a5d0 2024-09-19T06:53:52.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.905+0000 7fde72a9d740 10 client.17136 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:52.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.905+0000 7fde72a9d740 10 client.17136 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:53:52.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.905+0000 7fde72a9d740 20 client.17136 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:52.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.905+0000 7fde72a9d740 1 --2- 172.21.15.103:0/3070100318 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55f2d4824f40 0x55f2d4827370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.905+0000 7fde72a9d740 4 client.17136 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:52.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.905+0000 7fde72a9d740 1 -- 172.21.15.103:0/3070100318 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2d4777d70 con 0x55f2d4824f40 2024-09-19T06:53:52.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.906+0000 7fde6dc5f640 1 --2- 172.21.15.103:0/3070100318 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55f2d4824f40 0x55f2d4827370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.906+0000 7fde6dc5f640 1 --2- 172.21.15.103:0/3070100318 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55f2d4824f40 0x55f2d4827370 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6c3779f1ab718b8e server_cookie=b44367c019960398 in_seq=0 out_seq=0 2024-09-19T06:53:52.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.906+0000 7fde4dffb640 10 client.17136 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:53:52.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.907+0000 7fde4dffb640 1 -- 172.21.15.103:0/3070100318 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2d4777d70 con 0x55f2d4824f40 2024-09-19T06:53:52.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.907+0000 7fde4dffb640 10 client.17136 handle_command_reply: tid=0 2024-09-19T06:53:52.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.907+0000 7fde72a9d740 1 client.17136 shutdown 2024-09-19T06:53:52.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.907+0000 7fde72a9d740 1 -- 172.21.15.103:0/3070100318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 msgr2=0x55f2d4814d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.907+0000 7fde72a9d740 1 --2- 172.21.15.103:0/3070100318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d4814d80 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fde6002a930 tx=0x7fde600318d0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.907+0000 7fde72a9d740 1 -- 172.21.15.103:0/3070100318 shutdown_connections 2024-09-19T06:53:52.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.907+0000 7fde72a9d740 1 --2- 172.21.15.103:0/3070100318 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55f2d4824f40 0x55f2d4827370 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.907+0000 7fde72a9d740 1 --2- 172.21.15.103:0/3070100318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d4815370 0x55f2d4812640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.907+0000 7fde72a9d740 1 --2- 172.21.15.103:0/3070100318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 0x55f2d4812030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.907+0000 7fde72a9d740 1 --2- 172.21.15.103:0/3070100318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d4814d80 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.907+0000 7fde4dffb640 0 client.17136 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:53:52.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.907+0000 7fde72a9d740 1 -- 172.21.15.103:0/3070100318 >> 172.21.15.103:0/3070100318 conn(0x55f2d47d3310 msgr2=0x55f2d480c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:52.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.907+0000 7fde72a9d740 1 -- 172.21.15.103:0/3070100318 shutdown_connections 2024-09-19T06:53:52.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.907+0000 7fde72a9d740 1 -- 172.21.15.103:0/3070100318 wait complete. 2024-09-19T06:53:52.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.907+0000 7fde72a9d740 20 client.17136 trim_cache size 0 max 16384 2024-09-19T06:53:52.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.920+0000 7fde72a9d740 1 Processor -- start 2024-09-19T06:53:52.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.920+0000 7fde72a9d740 1 -- start start 2024-09-19T06:53:52.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.920+0000 7fde72a9d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d48212b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.920+0000 7fde72a9d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 0x55f2d480c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.920+0000 7fde72a9d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d4815370 0x55f2d4816760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.920+0000 7fde72a9d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde681afa50 con 0x55f2d480a5d0 2024-09-19T06:53:52.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.920+0000 7fde72a9d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde681b1590 con 0x55f2d4815370 2024-09-19T06:53:52.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.920+0000 7fde72a9d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde681b0120 con 0x55f2d480bb40 2024-09-19T06:53:52.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.920+0000 7fde6e460640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d48212b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.920+0000 7fde6e460640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d48212b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41032/0 (socket says 172.21.15.103:41032) 2024-09-19T06:53:52.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.920+0000 7fde6e460640 1 -- 172.21.15.103:0/1474565362 learned_addr learned my addr 172.21.15.103:0/1474565362 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:52.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.921+0000 7fde6dc5f640 1 --2- 172.21.15.103:0/1474565362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 0x55f2d480c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.921+0000 7fde6e460640 1 -- 172.21.15.103:0/1474565362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 msgr2=0x55f2d480c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:52.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.921+0000 7fde6e460640 1 --2- 172.21.15.103:0/1474565362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 0x55f2d480c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.921+0000 7fde6e460640 1 -- 172.21.15.103:0/1474565362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d4815370 msgr2=0x55f2d4816760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:52.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.921+0000 7fde6ec61640 1 --2- 172.21.15.103:0/1474565362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d4815370 0x55f2d4816760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.921+0000 7fde6e460640 1 --2- 172.21.15.103:0/1474565362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d4815370 0x55f2d4816760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.921+0000 7fde6e460640 1 -- 172.21.15.103:0/1474565362 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2d48129b0 con 0x55f2d480a5d0 2024-09-19T06:53:52.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.921+0000 7fde6dc5f640 1 --2- 172.21.15.103:0/1474565362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 0x55f2d480c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:52.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.921+0000 7fde6e460640 1 --2- 172.21.15.103:0/1474565362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d48212b0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fde60009850 tx=0x7fde6002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0c15274a52b8ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:52.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.921+0000 7fde4ffff640 1 -- 172.21.15.103:0/1474565362 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60035070 con 0x55f2d480a5d0 2024-09-19T06:53:52.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.921+0000 7fde4ffff640 1 -- 172.21.15.103:0/1474565362 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde60005030 con 0x55f2d480a5d0 2024-09-19T06:53:52.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.921+0000 7fde4ffff640 1 -- 172.21.15.103:0/1474565362 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde600049b0 con 0x55f2d480a5d0 2024-09-19T06:53:52.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 -- 172.21.15.103:0/1474565362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 msgr2=0x55f2d48212b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 --2- 172.21.15.103:0/1474565362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d48212b0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fde60009850 tx=0x7fde6002a410 comp rx=0 tx=0).stop 2024-09-19T06:53:52.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 -- 172.21.15.103:0/1474565362 shutdown_connections 2024-09-19T06:53:52.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 --2- 172.21.15.103:0/1474565362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d4815370 0x55f2d4816760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 --2- 172.21.15.103:0/1474565362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d480bb40 0x55f2d480c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 --2- 172.21.15.103:0/1474565362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d48212b0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 -- 172.21.15.103:0/1474565362 >> 172.21.15.103:0/1474565362 conn(0x55f2d47d3310 msgr2=0x55f2d480c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:52.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 -- 172.21.15.103:0/1474565362 shutdown_connections 2024-09-19T06:53:52.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 -- 172.21.15.103:0/1474565362 wait complete. 2024-09-19T06:53:52.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 -- start start 2024-09-19T06:53:52.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 Processor -- start 2024-09-19T06:53:52.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d482c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d480bb40 0x55f2d482cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4815370 0x55f2d482d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde681b0ec0 con 0x55f2d480a5d0 2024-09-19T06:53:52.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde681b07f0 con 0x55f2d480bb40 2024-09-19T06:53:52.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde681ada10 con 0x55f2d4815370 2024-09-19T06:53:52.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.922+0000 7fde72a9d740 10 client.0 _ll_register_callbacks cb 0x7fde60009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:52.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde6e460640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d482c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde6e460640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d482c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41042/0 (socket says 172.21.15.103:41042) 2024-09-19T06:53:52.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde6e460640 1 -- 172.21.15.103:0/1971419785 learned_addr learned my addr 172.21.15.103:0/1971419785 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:52.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde6ec61640 1 --2- 172.21.15.103:0/1971419785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4815370 0x55f2d482d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde6dc5f640 1 --2- 172.21.15.103:0/1971419785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d480bb40 0x55f2d482cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde6e460640 1 -- 172.21.15.103:0/1971419785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4815370 msgr2=0x55f2d482d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:52.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde6e460640 1 --2- 172.21.15.103:0/1971419785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4815370 0x55f2d482d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde6e460640 1 -- 172.21.15.103:0/1971419785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d480bb40 msgr2=0x55f2d482cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:52.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde6e460640 1 --2- 172.21.15.103:0/1971419785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d480bb40 0x55f2d482cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde6e460640 1 -- 172.21.15.103:0/1971419785 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2d481b930 con 0x55f2d480a5d0 2024-09-19T06:53:52.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde6ec61640 1 --2- 172.21.15.103:0/1971419785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4815370 0x55f2d482d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:52.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde6dc5f640 1 --2- 172.21.15.103:0/1971419785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d480bb40 0x55f2d482cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:52.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde6e460640 1 --2- 172.21.15.103:0/1971419785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d482c840 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fde60003960 tx=0x7fde60005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e57c7b1b090303b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:52.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:52.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde4dffb640 1 -- 172.21.15.103:0/1971419785 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60035050 con 0x55f2d480a5d0 2024-09-19T06:53:52.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde4dffb640 1 -- 172.21.15.103:0/1971419785 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fde60003b50 con 0x55f2d480a5d0 2024-09-19T06:53:52.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde72a9d740 1 -- 172.21.15.103:0/1971419785 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde60005030 con 0x55f2d480a5d0 2024-09-19T06:53:52.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde4dffb640 1 -- 172.21.15.103:0/1971419785 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fde60067050 con 0x55f2d480a5d0 2024-09-19T06:53:52.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde72a9d740 10 client.17142 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:52.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.923+0000 7fde72a9d740 1 -- 172.21.15.103:0/1971419785 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f2d488fe90 con 0x55f2d480a5d0 2024-09-19T06:53:52.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.924+0000 7fde4dffb640 1 -- 172.21.15.103:0/1971419785 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde60005030 con 0x55f2d480a5d0 2024-09-19T06:53:52.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.924+0000 7fde4dffb640 1 -- 172.21.15.103:0/1971419785 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fde60045480 con 0x55f2d480a5d0 2024-09-19T06:53:52.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.924+0000 7fde72a9d740 10 client.17142 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:52.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.924+0000 7fde72a9d740 10 client.17142 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:53:52.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.924+0000 7fde72a9d740 20 client.17142 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:52.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.924+0000 7fde72a9d740 1 --2- 172.21.15.103:0/1971419785 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55f2d48904a0 0x55f2d48928c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.924+0000 7fde72a9d740 4 client.17142 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:53:52.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.924+0000 7fde72a9d740 1 -- 172.21.15.103:0/1971419785 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2d46f4170 con 0x55f2d48904a0 2024-09-19T06:53:52.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.924+0000 7fde6dc5f640 1 --2- 172.21.15.103:0/1971419785 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55f2d48904a0 0x55f2d48928c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde6dc5f640 1 --2- 172.21.15.103:0/1971419785 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55f2d48904a0 0x55f2d48928c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c02734b79b527922 server_cookie=4c2bdebff7abf83b in_seq=0 out_seq=0 2024-09-19T06:53:52.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde4dffb640 10 client.17142 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:53:52.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde4dffb640 1 -- 172.21.15.103:0/1971419785 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2d46f4170 con 0x55f2d48904a0 2024-09-19T06:53:52.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde4dffb640 10 client.17142 handle_command_reply: tid=0 2024-09-19T06:53:52.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde72a9d740 1 client.17142 shutdown 2024-09-19T06:53:52.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde72a9d740 1 -- 172.21.15.103:0/1971419785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 msgr2=0x55f2d482c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde72a9d740 1 --2- 172.21.15.103:0/1971419785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d482c840 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fde60003960 tx=0x7fde60005a20 comp rx=0 tx=0).stop 2024-09-19T06:53:52.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde72a9d740 1 -- 172.21.15.103:0/1971419785 shutdown_connections 2024-09-19T06:53:52.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde72a9d740 1 --2- 172.21.15.103:0/1971419785 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55f2d48904a0 0x55f2d48928c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde72a9d740 1 --2- 172.21.15.103:0/1971419785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4815370 0x55f2d482d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde72a9d740 1 --2- 172.21.15.103:0/1971419785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2d480bb40 0x55f2d482cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde4dffb640 0 client.17142 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:53:52.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde72a9d740 1 --2- 172.21.15.103:0/1971419785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d480a5d0 0x55f2d482c840 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde72a9d740 1 -- 172.21.15.103:0/1971419785 >> 172.21.15.103:0/1971419785 conn(0x55f2d47d3310 msgr2=0x55f2d480c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:52.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde72a9d740 1 -- 172.21.15.103:0/1971419785 shutdown_connections 2024-09-19T06:53:52.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde72a9d740 1 -- 172.21.15.103:0/1971419785 wait complete. 2024-09-19T06:53:52.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.925+0000 7fde72a9d740 20 client.17142 trim_cache size 0 max 16384 2024-09-19T06:53:52.933 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:53:52.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.926+0000 7fde706c4640 1 -- 172.21.15.103:0/312314630 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fde500a33b0 msgr2=0x7fde500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.926+0000 7fde706c4640 1 --2- 172.21.15.103:0/312314630 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fde500a33b0 0x7fde500a5870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fde58009fd0 tx=0x7fde5800d040 comp rx=0 tx=0).stop 2024-09-19T06:53:52.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.926+0000 7fde706c4640 1 -- 172.21.15.103:0/312314630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde68102790 msgr2=0x7fde68074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.926+0000 7fde706c4640 1 --2- 172.21.15.103:0/312314630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde68102790 0x7fde68074a30 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00d770 tx=0x7fde5c00dc30 comp rx=0 tx=0).stop 2024-09-19T06:53:52.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.926+0000 7fde706c4640 1 -- 172.21.15.103:0/312314630 shutdown_connections 2024-09-19T06:53:52.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.926+0000 7fde706c4640 1 --2- 172.21.15.103:0/312314630 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fde500a33b0 0x7fde500a5870 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.926+0000 7fde706c4640 1 --2- 172.21.15.103:0/312314630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde68106f10 0x7fde68074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.926+0000 7fde706c4640 1 --2- 172.21.15.103:0/312314630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde68102790 0x7fde68074a30 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.926+0000 7fde706c4640 1 --2- 172.21.15.103:0/312314630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde68101de0 0x7fde680744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.926+0000 7fde706c4640 1 -- 172.21.15.103:0/312314630 >> 172.21.15.103:0/312314630 conn(0x7fde680fd8c0 msgr2=0x7fde680febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:52.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.926+0000 7fde706c4640 1 -- 172.21.15.103:0/312314630 shutdown_connections 2024-09-19T06:53:52.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:52.926+0000 7fde706c4640 1 -- 172.21.15.103:0/312314630 wait complete. 2024-09-19T06:53:52.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:52 smithi134 ceph-mon[35356]: pgmap v142: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.5 KiB/s wr, 12 op/s 2024-09-19T06:53:53.709 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:53:53.968 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:54.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:54 smithi174 ceph-mon[35637]: pgmap v143: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.5 KiB/s wr, 12 op/s 2024-09-19T06:53:54.358 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:54 smithi103 ceph-mon[31158]: pgmap v143: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.5 KiB/s wr, 12 op/s 2024-09-19T06:53:54.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:54 smithi134 ceph-mon[35356]: pgmap v143: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.5 KiB/s wr, 12 op/s 2024-09-19T06:53:54.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.439+0000 7f21f57c2640 1 -- 172.21.15.103:0/90473809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21f0101430 msgr2=0x7f21f010d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:54.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.439+0000 7f21f57c2640 1 --2- 172.21.15.103:0/90473809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21f0101430 0x7f21f010d960 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009cd0 tx=0x7f21e802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.439+0000 7f21f57c2640 1 -- 172.21.15.103:0/90473809 shutdown_connections 2024-09-19T06:53:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.439+0000 7f21f57c2640 1 --2- 172.21.15.103:0/90473809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21f0101430 0x7f21f010d960 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.439+0000 7f21f57c2640 1 --2- 172.21.15.103:0/90473809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21f0100a90 0x7f21f0100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.439+0000 7f21f57c2640 1 --2- 172.21.15.103:0/90473809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21f0106a90 0x7f21f0106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.439+0000 7f21f57c2640 1 -- 172.21.15.103:0/90473809 >> 172.21.15.103:0/90473809 conn(0x7f21f00fc7d0 msgr2=0x7f21f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.439+0000 7f21f57c2640 1 -- 172.21.15.103:0/90473809 shutdown_connections 2024-09-19T06:53:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.439+0000 7f21f57c2640 1 -- 172.21.15.103:0/90473809 wait complete. 2024-09-19T06:53:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.440+0000 7f21f57c2640 1 Processor -- start 2024-09-19T06:53:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.440+0000 7f21f57c2640 1 -- start start 2024-09-19T06:53:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.440+0000 7f21f57c2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21f0100a90 0x7f21f0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.440+0000 7f21f57c2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21f0101430 0x7f21f0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.440+0000 7f21f57c2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21f0106a90 0x7f21f019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.440+0000 7f21f57c2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21f0110eb0 con 0x7f21f0101430 2024-09-19T06:53:54.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.440+0000 7f21f57c2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21f0110d30 con 0x7f21f0106a90 2024-09-19T06:53:54.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.440+0000 7f21f57c2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21f0111030 con 0x7f21f0100a90 2024-09-19T06:53:54.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.440+0000 7f21ee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21f0101430 0x7f21f0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.440+0000 7f21ee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21f0101430 0x7f21f0198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41068/0 (socket says 172.21.15.103:41068) 2024-09-19T06:53:54.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.440+0000 7f21ee7fc640 1 -- 172.21.15.103:0/1812586969 learned_addr learned my addr 172.21.15.103:0/1812586969 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:54.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.440+0000 7f21ef7fe640 1 --2- 172.21.15.103:0/1812586969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21f0106a90 0x7f21f019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.440+0000 7f21eeffd640 1 --2- 172.21.15.103:0/1812586969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21f0100a90 0x7f21f0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.441+0000 7f21ee7fc640 1 -- 172.21.15.103:0/1812586969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21f0100a90 msgr2=0x7f21f0198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:54.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.441+0000 7f21ee7fc640 1 --2- 172.21.15.103:0/1812586969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21f0100a90 0x7f21f0198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.441+0000 7f21ee7fc640 1 -- 172.21.15.103:0/1812586969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21f0106a90 msgr2=0x7f21f019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:54.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.441+0000 7f21ee7fc640 1 --2- 172.21.15.103:0/1812586969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21f0106a90 0x7f21f019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.441+0000 7f21ee7fc640 1 -- 172.21.15.103:0/1812586969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f019d3f0 con 0x7f21f0101430 2024-09-19T06:53:54.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.441+0000 7f21ef7fe640 1 --2- 172.21.15.103:0/1812586969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21f0106a90 0x7f21f019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:54.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.441+0000 7f21ee7fc640 1 --2- 172.21.15.103:0/1812586969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21f0101430 0x7f21f0198980 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f21e400b4a0 tx=0x7f21e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:54.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.441+0000 7f21d7fff640 1 -- 172.21.15.103:0/1812586969 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4014070 con 0x7f21f0101430 2024-09-19T06:53:54.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.441+0000 7f21f57c2640 1 -- 172.21.15.103:0/1812586969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f019d6e0 con 0x7f21f0101430 2024-09-19T06:53:54.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.441+0000 7f21d7fff640 1 -- 172.21.15.103:0/1812586969 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e4010af0 con 0x7f21f0101430 2024-09-19T06:53:54.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.441+0000 7f21d7fff640 1 -- 172.21.15.103:0/1812586969 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4017d10 con 0x7f21f0101430 2024-09-19T06:53:54.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.441+0000 7f21f57c2640 1 -- 172.21.15.103:0/1812586969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f01a4fc0 con 0x7f21f0101430 2024-09-19T06:53:54.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.443+0000 7f21d7fff640 1 -- 172.21.15.103:0/1812586969 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f21e4017070 con 0x7f21f0101430 2024-09-19T06:53:54.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21f7b9b740 1 Processor -- start 2024-09-19T06:53:54.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21f7b9b740 1 -- start start 2024-09-19T06:53:54.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21f7b9b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e045b40 0x55bf4e044090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21f7b9b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 0x55bf4e044a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21f7b9b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e04f370 0x55bf4e051760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21f7b9b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bf4dfaa2f0 con 0x55bf4e04f370 2024-09-19T06:53:54.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21f7b9b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bf4df51cf0 con 0x55bf4e0445d0 2024-09-19T06:53:54.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21f7b9b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf4dfa62b0 con 0x55bf4e045b40 2024-09-19T06:53:54.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21d7fff640 1 --2- 172.21.15.103:0/1812586969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21d80a33b0 0x7f21d80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21ee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e04f370 0x55bf4e051760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21ee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e04f370 0x55bf4e051760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41072/0 (socket says 172.21.15.103:41072) 2024-09-19T06:53:54.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21ee7fc640 1 -- 172.21.15.103:0/1340517976 learned_addr learned my addr 172.21.15.103:0/1340517976 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:54.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21ef7fe640 1 --2- 172.21.15.103:0/1812586969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21d80a33b0 0x7f21d80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21eeffd640 1 --2- 172.21.15.103:0/1340517976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 0x55bf4e044a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21ef7fe640 1 --2- 172.21.15.103:0/1340517976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e045b40 0x55bf4e044090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21d7fff640 1 -- 172.21.15.103:0/1812586969 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f21e40cf0c0 con 0x7f21f0101430 2024-09-19T06:53:54.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21ee7fc640 1 -- 172.21.15.103:0/1340517976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e045b40 msgr2=0x55bf4e044090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:54.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21ee7fc640 1 --2- 172.21.15.103:0/1340517976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e045b40 0x55bf4e044090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21ee7fc640 1 -- 172.21.15.103:0/1340517976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 msgr2=0x55bf4e044a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:54.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21ee7fc640 1 --2- 172.21.15.103:0/1340517976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 0x55bf4e044a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.444+0000 7f21ee7fc640 1 -- 172.21.15.103:0/1340517976 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf4df363a0 con 0x55bf4e04f370 2024-09-19T06:53:54.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.445+0000 7f21ef7fe640 1 --2- 172.21.15.103:0/1340517976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e045b40 0x55bf4e044090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:54.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.445+0000 7f21ee7fc640 1 --2- 172.21.15.103:0/1340517976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e04f370 0x55bf4e051760 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f21e409b640 tx=0x7f21e40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d14910b0e32af9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:54.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.445+0000 7f21d57fa640 1 -- 172.21.15.103:0/1340517976 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e401e050 con 0x55bf4e04f370 2024-09-19T06:53:54.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.445+0000 7f21d57fa640 1 -- 172.21.15.103:0/1340517976 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e409bb80 con 0x55bf4e04f370 2024-09-19T06:53:54.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.445+0000 7f21d57fa640 1 -- 172.21.15.103:0/1340517976 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4097af0 con 0x55bf4e04f370 2024-09-19T06:53:54.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.445+0000 7f21f7b9b740 1 -- 172.21.15.103:0/1340517976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e04f370 msgr2=0x55bf4e051760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:54.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/1340517976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e04f370 0x55bf4e051760 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f21e409b640 tx=0x7f21e40047d0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.445+0000 7f21ef7fe640 1 --2- 172.21.15.103:0/1812586969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21d80a33b0 0x7f21d80a5870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009430 tx=0x7f21e8009360 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:54.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 1 -- 172.21.15.103:0/1340517976 shutdown_connections 2024-09-19T06:53:54.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/1340517976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e04f370 0x55bf4e051760 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/1340517976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 0x55bf4e044a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/1340517976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e045b40 0x55bf4e044090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 1 -- 172.21.15.103:0/1340517976 >> 172.21.15.103:0/1340517976 conn(0x55bf4e00d310 msgr2=0x55bf4e040c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:54.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 1 -- 172.21.15.103:0/1340517976 shutdown_connections 2024-09-19T06:53:54.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 1 -- 172.21.15.103:0/1340517976 wait complete. 2024-09-19T06:53:54.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 1 -- start start 2024-09-19T06:53:54.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 1 Processor -- start 2024-09-19T06:53:54.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 0x55bf4e04edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e045b40 0x55bf4e04c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e04f370 0x55bf4e04c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bf4e052070 con 0x55bf4e04f370 2024-09-19T06:53:54.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bf4df65060 con 0x55bf4e0445d0 2024-09-19T06:53:54.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf4e0521f0 con 0x55bf4e045b40 2024-09-19T06:53:54.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21f7b9b740 10 client.0 _ll_register_callbacks cb 0x55bf4dcb88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:54.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21ef7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e04f370 0x55bf4e04c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21ef7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e04f370 0x55bf4e04c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41080/0 (socket says 172.21.15.103:41080) 2024-09-19T06:53:54.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.446+0000 7f21ef7fe640 1 -- 172.21.15.103:0/3929109069 learned_addr learned my addr 172.21.15.103:0/3929109069 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:54.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21ee7fc640 1 --2- 172.21.15.103:0/3929109069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e045b40 0x55bf4e04c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21eeffd640 1 --2- 172.21.15.103:0/3929109069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 0x55bf4e04edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21ef7fe640 1 -- 172.21.15.103:0/3929109069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e045b40 msgr2=0x55bf4e04c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:54.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21ef7fe640 1 --2- 172.21.15.103:0/3929109069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e045b40 0x55bf4e04c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21ef7fe640 1 -- 172.21.15.103:0/3929109069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 msgr2=0x55bf4e04edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:54.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21ef7fe640 1 --2- 172.21.15.103:0/3929109069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 0x55bf4e04edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21ef7fe640 1 -- 172.21.15.103:0/3929109069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf4dd1b3c0 con 0x55bf4e04f370 2024-09-19T06:53:54.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21eeffd640 1 --2- 172.21.15.103:0/3929109069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 0x55bf4e04edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:54.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21ef7fe640 1 --2- 172.21.15.103:0/3929109069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e04f370 0x55bf4e04c810 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009cd0 tx=0x7f21e802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb82dfdf2c67e93b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:54.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:54.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21d2ff5640 1 -- 172.21.15.103:0/3929109069 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8004790 con 0x55bf4e04f370 2024-09-19T06:53:54.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21d2ff5640 1 -- 172.21.15.103:0/3929109069 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f21e80305a0 con 0x55bf4e04f370 2024-09-19T06:53:54.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21f7b9b740 1 -- 172.21.15.103:0/3929109069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e409bb80 con 0x55bf4e04f370 2024-09-19T06:53:54.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21d2ff5640 1 -- 172.21.15.103:0/3929109069 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f21e80307a0 con 0x55bf4e04f370 2024-09-19T06:53:54.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21f7b9b740 10 client.17154 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:54.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21f7b9b740 1 -- 172.21.15.103:0/3929109069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf4e05db00 con 0x55bf4e04f370 2024-09-19T06:53:54.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.447+0000 7f21d2ff5640 1 -- 172.21.15.103:0/3929109069 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e409bb80 con 0x55bf4e04f370 2024-09-19T06:53:54.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.448+0000 7f21d2ff5640 1 -- 172.21.15.103:0/3929109069 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f21e8044070 con 0x55bf4e04f370 2024-09-19T06:53:54.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.448+0000 7f21f7b9b740 10 client.17154 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:54.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.448+0000 7f21f7b9b740 10 client.17154 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:53:54.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.448+0000 7f21f7b9b740 20 client.17154 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:54.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.448+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/3929109069 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55bf4e05ef10 0x55bf4e0613b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.448+0000 7f21f7b9b740 4 client.17154 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:54.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.448+0000 7f21f7b9b740 1 -- 172.21.15.103:0/3929109069 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf4dfb7860 con 0x55bf4e05ef10 2024-09-19T06:53:54.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.448+0000 7f21eeffd640 1 --2- 172.21.15.103:0/3929109069 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55bf4e05ef10 0x55bf4e0613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.448+0000 7f21d2ff5640 10 client.17154 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:53:54.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.448+0000 7f21eeffd640 1 --2- 172.21.15.103:0/3929109069 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55bf4e05ef10 0x55bf4e0613b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f384b4b54cd8e8e4 server_cookie=2b637b4c72da796b in_seq=0 out_seq=0 2024-09-19T06:53:54.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.449+0000 7f21d2ff5640 1 -- 172.21.15.103:0/3929109069 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf4dfb7860 con 0x55bf4e05ef10 2024-09-19T06:53:54.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.449+0000 7f21d2ff5640 10 client.17154 handle_command_reply: tid=0 2024-09-19T06:53:54.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.449+0000 7f21f7b9b740 1 client.17154 shutdown 2024-09-19T06:53:54.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.449+0000 7f21f7b9b740 1 -- 172.21.15.103:0/3929109069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e04f370 msgr2=0x55bf4e04c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:54.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.449+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/3929109069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e04f370 0x55bf4e04c810 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009cd0 tx=0x7f21e802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.449+0000 7f21f7b9b740 1 -- 172.21.15.103:0/3929109069 shutdown_connections 2024-09-19T06:53:54.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.449+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/3929109069 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55bf4e05ef10 0x55bf4e0613b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.449+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/3929109069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e04f370 0x55bf4e04c810 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.449+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/3929109069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e045b40 0x55bf4e04c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.449+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/3929109069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 0x55bf4e04edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.449+0000 7f21f7b9b740 1 -- 172.21.15.103:0/3929109069 >> 172.21.15.103:0/3929109069 conn(0x55bf4e00d310 msgr2=0x55bf4e046990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:54.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.449+0000 7f21d2ff5640 0 client.17154 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:53:54.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.450+0000 7f21f7b9b740 1 -- 172.21.15.103:0/3929109069 shutdown_connections 2024-09-19T06:53:54.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.450+0000 7f21f7b9b740 1 -- 172.21.15.103:0/3929109069 wait complete. 2024-09-19T06:53:54.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.450+0000 7f21f7b9b740 20 client.17154 trim_cache size 0 max 16384 2024-09-19T06:53:54.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.462+0000 7f21f7b9b740 1 Processor -- start 2024-09-19T06:53:54.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.462+0000 7f21f7b9b740 1 -- start start 2024-09-19T06:53:54.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21f7b9b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 0x55bf4e040e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21f7b9b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e045b40 0x55bf4e0453a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21f7b9b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e04f370 0x55bf4e04b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21f7b9b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21f01ad850 con 0x55bf4e045b40 2024-09-19T06:53:54.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21f7b9b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21f01af390 con 0x55bf4e0445d0 2024-09-19T06:53:54.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21f7b9b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21f01adf20 con 0x55bf4e04f370 2024-09-19T06:53:54.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21ee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e045b40 0x55bf4e0453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21ee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e045b40 0x55bf4e0453a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41086/0 (socket says 172.21.15.103:41086) 2024-09-19T06:53:54.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21ee7fc640 1 -- 172.21.15.103:0/2144800919 learned_addr learned my addr 172.21.15.103:0/2144800919 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:54.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21eeffd640 1 --2- 172.21.15.103:0/2144800919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 0x55bf4e040e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21ee7fc640 1 -- 172.21.15.103:0/2144800919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e04f370 msgr2=0x55bf4e04b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:54.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21ef7fe640 1 --2- 172.21.15.103:0/2144800919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e04f370 0x55bf4e04b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21ee7fc640 1 --2- 172.21.15.103:0/2144800919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e04f370 0x55bf4e04b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21ee7fc640 1 -- 172.21.15.103:0/2144800919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 msgr2=0x55bf4e040e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:54.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21ee7fc640 1 --2- 172.21.15.103:0/2144800919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 0x55bf4e040e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21ee7fc640 1 -- 172.21.15.103:0/2144800919 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf4e040320 con 0x55bf4e045b40 2024-09-19T06:53:54.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21ef7fe640 1 --2- 172.21.15.103:0/2144800919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e04f370 0x55bf4e04b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:54.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21eeffd640 1 --2- 172.21.15.103:0/2144800919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 0x55bf4e040e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:54.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21ee7fc640 1 --2- 172.21.15.103:0/2144800919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e045b40 0x55bf4e0453a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f21e40cc950 tx=0x7f21e400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314ccce88d0eea93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:54.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21d4ff9640 1 -- 172.21.15.103:0/2144800919 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e409f830 con 0x55bf4e045b40 2024-09-19T06:53:54.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21d4ff9640 1 -- 172.21.15.103:0/2144800919 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e4017360 con 0x55bf4e045b40 2024-09-19T06:53:54.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.463+0000 7f21d4ff9640 1 -- 172.21.15.103:0/2144800919 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4010e40 con 0x55bf4e045b40 2024-09-19T06:53:54.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.464+0000 7f21f7b9b740 1 -- 172.21.15.103:0/2144800919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e045b40 msgr2=0x55bf4e0453a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:54.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.464+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/2144800919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e045b40 0x55bf4e0453a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f21e40cc950 tx=0x7f21e400bf90 comp rx=0 tx=0).stop 2024-09-19T06:53:54.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.464+0000 7f21f7b9b740 1 -- 172.21.15.103:0/2144800919 shutdown_connections 2024-09-19T06:53:54.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.464+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/2144800919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e04f370 0x55bf4e04b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.464+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/2144800919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e045b40 0x55bf4e0453a0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.464+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/2144800919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e0445d0 0x55bf4e040e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.464+0000 7f21f7b9b740 1 -- 172.21.15.103:0/2144800919 >> 172.21.15.103:0/2144800919 conn(0x55bf4e00d310 msgr2=0x55bf4e040700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:54.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.464+0000 7f21f7b9b740 1 -- 172.21.15.103:0/2144800919 shutdown_connections 2024-09-19T06:53:54.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.464+0000 7f21f7b9b740 1 -- 172.21.15.103:0/2144800919 wait complete. 2024-09-19T06:53:54.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21f7b9b740 1 -- start start 2024-09-19T06:53:54.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21f7b9b740 1 Processor -- start 2024-09-19T06:53:54.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21f7b9b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e0445d0 0x55bf4e057480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21f7b9b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e045b40 0x55bf4e0579c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21f7b9b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e04f370 0x55bf4e0484b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21f7b9b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21f01aecc0 con 0x55bf4e045b40 2024-09-19T06:53:54.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21f7b9b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21f01ae5f0 con 0x55bf4e04f370 2024-09-19T06:53:54.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21f7b9b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21f01ab810 con 0x55bf4e0445d0 2024-09-19T06:53:54.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21f7b9b740 10 client.0 _ll_register_callbacks cb 0x7f21e8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:54.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21ee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e045b40 0x55bf4e0579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21ee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e045b40 0x55bf4e0579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41102/0 (socket says 172.21.15.103:41102) 2024-09-19T06:53:54.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21ee7fc640 1 -- 172.21.15.103:0/4008758954 learned_addr learned my addr 172.21.15.103:0/4008758954 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:54.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21ef7fe640 1 --2- 172.21.15.103:0/4008758954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e04f370 0x55bf4e0484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21eeffd640 1 --2- 172.21.15.103:0/4008758954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e0445d0 0x55bf4e057480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21ee7fc640 1 -- 172.21.15.103:0/4008758954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e0445d0 msgr2=0x55bf4e057480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:54.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21ee7fc640 1 --2- 172.21.15.103:0/4008758954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e0445d0 0x55bf4e057480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21ee7fc640 1 -- 172.21.15.103:0/4008758954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e04f370 msgr2=0x55bf4e0484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:54.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21ee7fc640 1 --2- 172.21.15.103:0/4008758954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e04f370 0x55bf4e0484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21ee7fc640 1 -- 172.21.15.103:0/4008758954 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf4e0489f0 con 0x55bf4e045b40 2024-09-19T06:53:54.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21eeffd640 1 --2- 172.21.15.103:0/4008758954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e0445d0 0x55bf4e057480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:54.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.465+0000 7f21ef7fe640 1 --2- 172.21.15.103:0/4008758954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e04f370 0x55bf4e0484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:54.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.466+0000 7f21ee7fc640 1 --2- 172.21.15.103:0/4008758954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e045b40 0x55bf4e0579c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f21e409b410 tx=0x7f21e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbed61236b92e3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:54.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.466+0000 7f21d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:54.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.466+0000 7f21d2ff5640 1 -- 172.21.15.103:0/4008758954 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e40b3060 con 0x55bf4e045b40 2024-09-19T06:53:54.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.466+0000 7f21d2ff5640 1 -- 172.21.15.103:0/4008758954 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f21e40cc9d0 con 0x55bf4e045b40 2024-09-19T06:53:54.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.466+0000 7f21f7b9b740 1 -- 172.21.15.103:0/4008758954 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e4017360 con 0x55bf4e045b40 2024-09-19T06:53:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.466+0000 7f21d2ff5640 1 -- 172.21.15.103:0/4008758954 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f21e401e070 con 0x55bf4e045b40 2024-09-19T06:53:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.466+0000 7f21f7b9b740 10 client.17160 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.466+0000 7f21f7b9b740 1 -- 172.21.15.103:0/4008758954 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf4e0c9350 con 0x55bf4e045b40 2024-09-19T06:53:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.466+0000 7f21d2ff5640 1 -- 172.21.15.103:0/4008758954 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e4017360 con 0x55bf4e045b40 2024-09-19T06:53:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.466+0000 7f21d2ff5640 1 -- 172.21.15.103:0/4008758954 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f21e40b4250 con 0x55bf4e045b40 2024-09-19T06:53:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.466+0000 7f21f7b9b740 10 client.17160 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.466+0000 7f21f7b9b740 10 client.17160 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:53:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.466+0000 7f21f7b9b740 20 client.17160 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.466+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/4008758954 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55bf4e0c9520 0x55bf4e0cb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.466+0000 7f21f7b9b740 4 client.17160 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:53:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.466+0000 7f21f7b9b740 1 -- 172.21.15.103:0/4008758954 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf4dfb1d70 con 0x55bf4e0c9520 2024-09-19T06:53:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.467+0000 7f21eeffd640 1 --2- 172.21.15.103:0/4008758954 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55bf4e0c9520 0x55bf4e0cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:54.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.467+0000 7f21eeffd640 1 --2- 172.21.15.103:0/4008758954 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55bf4e0c9520 0x55bf4e0cb910 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c1d89e61e1adc90 server_cookie=735af5f9e42bc6d6 in_seq=0 out_seq=0 2024-09-19T06:53:54.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.467+0000 7f21d2ff5640 10 client.17160 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:53:54.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.467+0000 7f21d2ff5640 1 -- 172.21.15.103:0/4008758954 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf4dfb1d70 con 0x55bf4e0c9520 2024-09-19T06:53:54.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.467+0000 7f21d2ff5640 10 client.17160 handle_command_reply: tid=0 2024-09-19T06:53:54.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.467+0000 7f21f7b9b740 1 client.17160 shutdown 2024-09-19T06:53:54.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.467+0000 7f21f7b9b740 1 -- 172.21.15.103:0/4008758954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e045b40 msgr2=0x55bf4e0579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:54.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.467+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/4008758954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e045b40 0x55bf4e0579c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f21e409b410 tx=0x7f21e40046c0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.467+0000 7f21f7b9b740 1 -- 172.21.15.103:0/4008758954 shutdown_connections 2024-09-19T06:53:54.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.467+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/4008758954 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55bf4e0c9520 0x55bf4e0cb910 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.467+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/4008758954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf4e04f370 0x55bf4e0484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.467+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/4008758954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf4e045b40 0x55bf4e0579c0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.467+0000 7f21d2ff5640 0 client.17160 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:53:54.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.467+0000 7f21f7b9b740 1 --2- 172.21.15.103:0/4008758954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf4e0445d0 0x55bf4e057480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.467+0000 7f21f7b9b740 1 -- 172.21.15.103:0/4008758954 >> 172.21.15.103:0/4008758954 conn(0x55bf4e00d310 msgr2=0x55bf4e04d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:54.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.468+0000 7f21f7b9b740 1 -- 172.21.15.103:0/4008758954 shutdown_connections 2024-09-19T06:53:54.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.468+0000 7f21f7b9b740 1 -- 172.21.15.103:0/4008758954 wait complete. 2024-09-19T06:53:54.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.468+0000 7f21f7b9b740 20 client.17160 trim_cache size 0 max 16384 2024-09-19T06:53:54.476 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:53:54.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.468+0000 7f21f57c2640 1 -- 172.21.15.103:0/1812586969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21d80a33b0 msgr2=0x7f21d80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:54.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.468+0000 7f21f57c2640 1 --2- 172.21.15.103:0/1812586969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21d80a33b0 0x7f21d80a5870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009430 tx=0x7f21e8009360 comp rx=0 tx=0).stop 2024-09-19T06:53:54.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.468+0000 7f21f57c2640 1 -- 172.21.15.103:0/1812586969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21f0101430 msgr2=0x7f21f0198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:54.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.468+0000 7f21f57c2640 1 --2- 172.21.15.103:0/1812586969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21f0101430 0x7f21f0198980 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f21e400b4a0 tx=0x7f21e400b960 comp rx=0 tx=0).stop 2024-09-19T06:53:54.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.468+0000 7f21f57c2640 1 -- 172.21.15.103:0/1812586969 shutdown_connections 2024-09-19T06:53:54.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.468+0000 7f21f57c2640 1 --2- 172.21.15.103:0/1812586969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21d80a33b0 0x7f21d80a5870 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.468+0000 7f21f57c2640 1 --2- 172.21.15.103:0/1812586969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21f0106a90 0x7f21f019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.468+0000 7f21f57c2640 1 --2- 172.21.15.103:0/1812586969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21f0101430 0x7f21f0198980 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.468+0000 7f21f57c2640 1 --2- 172.21.15.103:0/1812586969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21f0100a90 0x7f21f0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:54.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.468+0000 7f21f57c2640 1 -- 172.21.15.103:0/1812586969 >> 172.21.15.103:0/1812586969 conn(0x7f21f00fc7d0 msgr2=0x7f21f0104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:54.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.468+0000 7f21f57c2640 1 -- 172.21.15.103:0/1812586969 shutdown_connections 2024-09-19T06:53:54.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:54.468+0000 7f21f57c2640 1 -- 172.21.15.103:0/1812586969 wait complete. 2024-09-19T06:53:55.191 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:53:55.438 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:55.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.880+0000 7f4dfe6f6640 1 -- 172.21.15.103:0/2224295014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4df8100e30 msgr2=0x7f4df8104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:55.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.880+0000 7f4dfe6f6640 1 --2- 172.21.15.103:0/2224295014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4df8100e30 0x7f4df8104e80 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00aa10 tx=0x7f4dec02b720 comp rx=0 tx=0).stop 2024-09-19T06:53:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.880+0000 7f4dfe6f6640 1 -- 172.21.15.103:0/2224295014 shutdown_connections 2024-09-19T06:53:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.880+0000 7f4dfe6f6640 1 --2- 172.21.15.103:0/2224295014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df81055b0 0x7f4df810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.880+0000 7f4dfe6f6640 1 --2- 172.21.15.103:0/2224295014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4df8100e30 0x7f4df8104e80 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.880+0000 7f4dfe6f6640 1 --2- 172.21.15.103:0/2224295014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4df8100480 0x7f4df8100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.880+0000 7f4dfe6f6640 1 -- 172.21.15.103:0/2224295014 >> 172.21.15.103:0/2224295014 conn(0x7f4df8077210 msgr2=0x7f4df80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.880+0000 7f4dfe6f6640 1 -- 172.21.15.103:0/2224295014 shutdown_connections 2024-09-19T06:53:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.880+0000 7f4dfe6f6640 1 -- 172.21.15.103:0/2224295014 wait complete. 2024-09-19T06:53:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.881+0000 7f4dfe6f6640 1 Processor -- start 2024-09-19T06:53:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.881+0000 7f4dfe6f6640 1 -- start start 2024-09-19T06:53:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4dfe6f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4df8100480 0x7f4df81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4dfe6f6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4df8100e30 0x7f4df81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4dfe6f6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df81055b0 0x7f4df819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4dfe6f6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4df810fdb0 con 0x7f4df8100480 2024-09-19T06:53:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4dfe6f6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4df810fc30 con 0x7f4df8100e30 2024-09-19T06:53:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4dfe6f6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4df810ff30 con 0x7f4df81055b0 2024-09-19T06:53:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4df8100480 0x7f4df81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4df8100480 0x7f4df81984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41124/0 (socket says 172.21.15.103:41124) 2024-09-19T06:53:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4df7fff640 1 -- 172.21.15.103:0/2574973122 learned_addr learned my addr 172.21.15.103:0/2574973122 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4dfcc93640 1 --2- 172.21.15.103:0/2574973122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df81055b0 0x7f4df819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4df77fe640 1 --2- 172.21.15.103:0/2574973122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4df8100e30 0x7f4df81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4df7fff640 1 -- 172.21.15.103:0/2574973122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df81055b0 msgr2=0x7f4df819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4df7fff640 1 --2- 172.21.15.103:0/2574973122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df81055b0 0x7f4df819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4df7fff640 1 -- 172.21.15.103:0/2574973122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4df8100e30 msgr2=0x7f4df81989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4df7fff640 1 --2- 172.21.15.103:0/2574973122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4df8100e30 0x7f4df81989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4df7fff640 1 -- 172.21.15.103:0/2574973122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df819d460 con 0x7f4df8100480 2024-09-19T06:53:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4df77fe640 1 --2- 172.21.15.103:0/2574973122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4df8100e30 0x7f4df81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:53:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4df7fff640 1 --2- 172.21.15.103:0/2574973122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4df8100480 0x7f4df81984b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f4de800c920 tx=0x7f4de800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4df57fa640 1 -- 172.21.15.103:0/2574973122 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8007c70 con 0x7f4df8100480 2024-09-19T06:53:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4dfe6f6640 1 -- 172.21.15.103:0/2574973122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df819d750 con 0x7f4df8100480 2024-09-19T06:53:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4df57fa640 1 -- 172.21.15.103:0/2574973122 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de8007e10 con 0x7f4df8100480 2024-09-19T06:53:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4df57fa640 1 -- 172.21.15.103:0/2574973122 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8018890 con 0x7f4df8100480 2024-09-19T06:53:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.882+0000 7f4dfe6f6640 1 -- 172.21.15.103:0/2574973122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df81a5030 con 0x7f4df8100480 2024-09-19T06:53:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.885+0000 7f4df57fa640 1 -- 172.21.15.103:0/2574973122 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4de8018a70 con 0x7f4df8100480 2024-09-19T06:53:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.885+0000 7f4e00acf740 1 Processor -- start 2024-09-19T06:53:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.885+0000 7f4e00acf740 1 -- start start 2024-09-19T06:53:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.885+0000 7f4e00acf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac316b40 0x563dac315090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.885+0000 7f4df57fa640 1 --2- 172.21.15.103:0/2574973122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4de00a33b0 0x7f4de00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.885+0000 7f4e00acf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac315a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.885+0000 7f4e00acf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac324780 0x563dac326b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.885+0000 7f4e00acf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563dac27b2f0 con 0x563dac3155d0 2024-09-19T06:53:55.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.885+0000 7f4e00acf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563dac222cf0 con 0x563dac324780 2024-09-19T06:53:55.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.885+0000 7f4e00acf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563dac2772b0 con 0x563dac316b40 2024-09-19T06:53:55.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.885+0000 7f4df77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac315a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.885+0000 7f4df77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac315a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41128/0 (socket says 172.21.15.103:41128) 2024-09-19T06:53:55.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.885+0000 7f4df77fe640 1 -- 172.21.15.103:0/3545718663 learned_addr learned my addr 172.21.15.103:0/3545718663 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:55.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.885+0000 7f4df7fff640 1 --2- 172.21.15.103:0/2574973122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4de00a33b0 0x7f4de00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.885+0000 7f4dfcc93640 1 --2- 172.21.15.103:0/3545718663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac316b40 0x563dac315090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.885+0000 7f4dfcc93640 1 --2- 172.21.15.103:0/3545718663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac324780 0x563dac326b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.886+0000 7f4df57fa640 1 -- 172.21.15.103:0/2574973122 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f4de80cfe60 con 0x7f4df8100480 2024-09-19T06:53:55.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.886+0000 7f4df77fe640 1 -- 172.21.15.103:0/3545718663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac316b40 msgr2=0x563dac315090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:55.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.886+0000 7f4df77fe640 1 --2- 172.21.15.103:0/3545718663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac316b40 0x563dac315090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.886+0000 7f4df77fe640 1 -- 172.21.15.103:0/3545718663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac324780 msgr2=0x563dac326b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:55.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.886+0000 7f4df77fe640 1 --2- 172.21.15.103:0/3545718663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac324780 0x563dac326b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.886+0000 7f4df77fe640 1 -- 172.21.15.103:0/3545718663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dac2073a0 con 0x563dac3155d0 2024-09-19T06:53:55.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.886+0000 7f4df77fe640 1 --2- 172.21.15.103:0/3545718663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac315a10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4dec005ab0 tx=0x7f4dec00a790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e623f2c0ef2d3e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:55.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.886+0000 7f4df7fff640 1 --2- 172.21.15.103:0/2574973122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4de00a33b0 0x7f4de00a5870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4de809b010 tx=0x7f4de800ff30 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:55.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.886+0000 7f4dde7fc640 1 -- 172.21.15.103:0/3545718663 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec036040 con 0x563dac3155d0 2024-09-19T06:53:55.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.886+0000 7f4dde7fc640 1 -- 172.21.15.103:0/3545718663 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dec0308c0 con 0x563dac3155d0 2024-09-19T06:53:55.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.886+0000 7f4dde7fc640 1 -- 172.21.15.103:0/3545718663 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec0048c0 con 0x563dac3155d0 2024-09-19T06:53:55.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.887+0000 7f4e00acf740 1 -- 172.21.15.103:0/3545718663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 msgr2=0x563dac315a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:55.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.887+0000 7f4e00acf740 1 --2- 172.21.15.103:0/3545718663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac315a10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4dec005ab0 tx=0x7f4dec00a790 comp rx=0 tx=0).stop 2024-09-19T06:53:55.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.887+0000 7f4e00acf740 1 -- 172.21.15.103:0/3545718663 shutdown_connections 2024-09-19T06:53:55.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.887+0000 7f4e00acf740 1 --2- 172.21.15.103:0/3545718663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac324780 0x563dac326b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.887+0000 7f4e00acf740 1 --2- 172.21.15.103:0/3545718663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac315a10 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.887+0000 7f4e00acf740 1 --2- 172.21.15.103:0/3545718663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac316b40 0x563dac315090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.887+0000 7f4e00acf740 1 -- 172.21.15.103:0/3545718663 >> 172.21.15.103:0/3545718663 conn(0x563dac2de310 msgr2=0x563dac311c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:55.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.887+0000 7f4e00acf740 1 -- 172.21.15.103:0/3545718663 shutdown_connections 2024-09-19T06:53:55.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.887+0000 7f4e00acf740 1 -- 172.21.15.103:0/3545718663 wait complete. 2024-09-19T06:53:55.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.887+0000 7f4e00acf740 1 -- start start 2024-09-19T06:53:55.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.887+0000 7f4e00acf740 1 Processor -- start 2024-09-19T06:53:55.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4e00acf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac3155d0 0x563dac324130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4e00acf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac316b40 0x563dac3213e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4e00acf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac324780 0x563dac321980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4e00acf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563dac327420 con 0x563dac324780 2024-09-19T06:53:55.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4e00acf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563dac236060 con 0x563dac316b40 2024-09-19T06:53:55.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4e00acf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563dac3275a0 con 0x563dac3155d0 2024-09-19T06:53:55.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4e00acf740 10 client.0 _ll_register_callbacks cb 0x563dabf898c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:55.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4dfcc93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac324780 0x563dac321980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4dfcc93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac324780 0x563dac321980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41134/0 (socket says 172.21.15.103:41134) 2024-09-19T06:53:55.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4dfcc93640 1 -- 172.21.15.103:0/934107299 learned_addr learned my addr 172.21.15.103:0/934107299 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:55.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4dfcc93640 1 --2- 172.21.15.103:0/934107299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac3155d0 0x563dac324130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4df77fe640 1 --2- 172.21.15.103:0/934107299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac316b40 0x563dac3213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4dfcc93640 1 -- 172.21.15.103:0/934107299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac3155d0 msgr2=0x563dac324130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:55.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4dfcc93640 1 --2- 172.21.15.103:0/934107299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac3155d0 0x563dac324130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4dfcc93640 1 -- 172.21.15.103:0/934107299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac316b40 msgr2=0x563dac3213e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:55.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4dfcc93640 1 --2- 172.21.15.103:0/934107299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac316b40 0x563dac3213e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4dfcc93640 1 -- 172.21.15.103:0/934107299 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dabfec3c0 con 0x563dac324780 2024-09-19T06:53:55.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4dfcc93640 1 --2- 172.21.15.103:0/934107299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac324780 0x563dac321980 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4df000bf10 tx=0x7f4df0009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2444a3db9d1922 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:55.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.888+0000 7f4dd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:55.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.889+0000 7f4dd77fe640 1 -- 172.21.15.103:0/934107299 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0014040 con 0x563dac324780 2024-09-19T06:53:55.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.889+0000 7f4e00acf740 1 -- 172.21.15.103:0/934107299 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dec0308c0 con 0x563dac324780 2024-09-19T06:53:55.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.889+0000 7f4dd77fe640 1 -- 172.21.15.103:0/934107299 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f4df000b230 con 0x563dac324780 2024-09-19T06:53:55.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.889+0000 7f4dd77fe640 1 -- 172.21.15.103:0/934107299 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4df002b050 con 0x563dac324780 2024-09-19T06:53:55.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.889+0000 7f4e00acf740 10 client.17172 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:55.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.889+0000 7f4e00acf740 1 -- 172.21.15.103:0/934107299 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563dac332e70 con 0x563dac324780 2024-09-19T06:53:55.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.889+0000 7f4dd77fe640 1 -- 172.21.15.103:0/934107299 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dec0308c0 con 0x563dac324780 2024-09-19T06:53:55.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.889+0000 7f4dd77fe640 1 -- 172.21.15.103:0/934107299 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4df001d720 con 0x563dac324780 2024-09-19T06:53:55.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.889+0000 7f4e00acf740 10 client.17172 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:55.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.889+0000 7f4e00acf740 10 client.17172 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:53:55.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.889+0000 7f4e00acf740 20 client.17172 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:55.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.889+0000 7f4e00acf740 1 --2- 172.21.15.103:0/934107299 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x563dac334330 0x563dac336760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.889+0000 7f4e00acf740 4 client.17172 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:55.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.889+0000 7f4e00acf740 1 -- 172.21.15.103:0/934107299 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dac288860 con 0x563dac334330 2024-09-19T06:53:55.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.889+0000 7f4df77fe640 1 --2- 172.21.15.103:0/934107299 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x563dac334330 0x563dac336760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.890+0000 7f4dd77fe640 10 client.17172 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:53:55.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.890+0000 7f4df77fe640 1 --2- 172.21.15.103:0/934107299 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x563dac334330 0x563dac336760 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a5feef7b7055efd server_cookie=d7f1c67a24777ce8 in_seq=0 out_seq=0 2024-09-19T06:53:55.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.890+0000 7f4dd77fe640 1 -- 172.21.15.103:0/934107299 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563dac288860 con 0x563dac334330 2024-09-19T06:53:55.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.890+0000 7f4dd77fe640 10 client.17172 handle_command_reply: tid=0 2024-09-19T06:53:55.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.890+0000 7f4e00acf740 1 client.17172 shutdown 2024-09-19T06:53:55.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.891+0000 7f4e00acf740 1 -- 172.21.15.103:0/934107299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac324780 msgr2=0x563dac321980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:55.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.891+0000 7f4e00acf740 1 --2- 172.21.15.103:0/934107299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac324780 0x563dac321980 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4df000bf10 tx=0x7f4df0009650 comp rx=0 tx=0).stop 2024-09-19T06:53:55.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.891+0000 7f4e00acf740 1 -- 172.21.15.103:0/934107299 shutdown_connections 2024-09-19T06:53:55.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.891+0000 7f4e00acf740 1 --2- 172.21.15.103:0/934107299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac324780 0x563dac321980 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.891+0000 7f4e00acf740 1 --2- 172.21.15.103:0/934107299 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x563dac334330 0x563dac336760 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.891+0000 7f4e00acf740 1 --2- 172.21.15.103:0/934107299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac316b40 0x563dac3213e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.891+0000 7f4e00acf740 1 --2- 172.21.15.103:0/934107299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac3155d0 0x563dac324130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.891+0000 7f4e00acf740 1 -- 172.21.15.103:0/934107299 >> 172.21.15.103:0/934107299 conn(0x563dac2de310 msgr2=0x563dac317960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:55.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.891+0000 7f4dd77fe640 0 client.17172 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:53:55.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.891+0000 7f4e00acf740 1 -- 172.21.15.103:0/934107299 shutdown_connections 2024-09-19T06:53:55.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.891+0000 7f4e00acf740 1 -- 172.21.15.103:0/934107299 wait complete. 2024-09-19T06:53:55.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.891+0000 7f4e00acf740 20 client.17172 trim_cache size 0 max 16384 2024-09-19T06:53:55.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.903+0000 7f4e00acf740 1 Processor -- start 2024-09-19T06:53:55.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.903+0000 7f4e00acf740 1 -- start start 2024-09-19T06:53:55.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4e00acf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac3306c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4e00acf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac316b40 0x563dac325b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4e00acf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac324780 0x563dac3260b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4e00acf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4df81ab650 con 0x563dac3155d0 2024-09-19T06:53:55.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4e00acf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4df81ad190 con 0x563dac316b40 2024-09-19T06:53:55.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4e00acf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4df81abd20 con 0x563dac324780 2024-09-19T06:53:55.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4dfcc93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac3306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4dfcc93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac3306c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41146/0 (socket says 172.21.15.103:41146) 2024-09-19T06:53:55.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4dfcc93640 1 -- 172.21.15.103:0/2393191417 learned_addr learned my addr 172.21.15.103:0/2393191417 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:55.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4df77fe640 1 --2- 172.21.15.103:0/2393191417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac316b40 0x563dac325b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4dfcc93640 1 -- 172.21.15.103:0/2393191417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac324780 msgr2=0x563dac3260b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:55.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4dfcc93640 1 --2- 172.21.15.103:0/2393191417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac324780 0x563dac3260b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4dfcc93640 1 -- 172.21.15.103:0/2393191417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac316b40 msgr2=0x563dac325b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:55.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4dfcc93640 1 --2- 172.21.15.103:0/2393191417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac316b40 0x563dac325b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4dfcc93640 1 -- 172.21.15.103:0/2393191417 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dac311ed0 con 0x563dac3155d0 2024-09-19T06:53:55.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4df77fe640 1 --2- 172.21.15.103:0/2393191417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac316b40 0x563dac325b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:55.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4dfcc93640 1 --2- 172.21.15.103:0/2393191417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac3306c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009730 tx=0x7f4df0009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919b45fe0bb59652 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:55.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4dddffb640 1 -- 172.21.15.103:0/2393191417 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df001ce60 con 0x563dac3155d0 2024-09-19T06:53:55.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4dddffb640 1 -- 172.21.15.103:0/2393191417 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df0022070 con 0x563dac3155d0 2024-09-19T06:53:55.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.904+0000 7f4dddffb640 1 -- 172.21.15.103:0/2393191417 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df001ab10 con 0x563dac3155d0 2024-09-19T06:53:55.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.905+0000 7f4e00acf740 1 -- 172.21.15.103:0/2393191417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 msgr2=0x563dac3306c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:55.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.905+0000 7f4e00acf740 1 --2- 172.21.15.103:0/2393191417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac3306c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009730 tx=0x7f4df0009d10 comp rx=0 tx=0).stop 2024-09-19T06:53:55.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.905+0000 7f4e00acf740 1 -- 172.21.15.103:0/2393191417 shutdown_connections 2024-09-19T06:53:55.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.905+0000 7f4e00acf740 1 --2- 172.21.15.103:0/2393191417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac324780 0x563dac3260b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.905+0000 7f4e00acf740 1 --2- 172.21.15.103:0/2393191417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac316b40 0x563dac325b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.905+0000 7f4e00acf740 1 --2- 172.21.15.103:0/2393191417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac3306c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.905+0000 7f4e00acf740 1 -- 172.21.15.103:0/2393191417 >> 172.21.15.103:0/2393191417 conn(0x563dac2de310 msgr2=0x563dac31fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:55.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.905+0000 7f4e00acf740 1 -- 172.21.15.103:0/2393191417 shutdown_connections 2024-09-19T06:53:55.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.905+0000 7f4e00acf740 1 -- 172.21.15.103:0/2393191417 wait complete. 2024-09-19T06:53:55.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4e00acf740 1 -- start start 2024-09-19T06:53:55.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4e00acf740 1 Processor -- start 2024-09-19T06:53:55.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4e00acf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac31ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4e00acf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac316b40 0x563dac32c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4e00acf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac324780 0x563dac32cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4e00acf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4df81acac0 con 0x563dac3155d0 2024-09-19T06:53:55.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4e00acf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4df81ac3f0 con 0x563dac316b40 2024-09-19T06:53:55.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4e00acf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4df81a9610 con 0x563dac324780 2024-09-19T06:53:55.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4e00acf740 10 client.0 _ll_register_callbacks cb 0x7f4df000b8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:55.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4dfcc93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac31ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4dfcc93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac31ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41150/0 (socket says 172.21.15.103:41150) 2024-09-19T06:53:55.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4dfcc93640 1 -- 172.21.15.103:0/1490894455 learned_addr learned my addr 172.21.15.103:0/1490894455 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:55.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4df77fe640 1 --2- 172.21.15.103:0/1490894455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac316b40 0x563dac32c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4dfcc93640 1 --2- 172.21.15.103:0/1490894455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac324780 0x563dac32cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4dfcc93640 1 -- 172.21.15.103:0/1490894455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac324780 msgr2=0x563dac32cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:55.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4dfcc93640 1 --2- 172.21.15.103:0/1490894455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac324780 0x563dac32cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4dfcc93640 1 -- 172.21.15.103:0/1490894455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac316b40 msgr2=0x563dac32c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:55.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4dfcc93640 1 --2- 172.21.15.103:0/1490894455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac316b40 0x563dac32c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.906+0000 7f4dfcc93640 1 -- 172.21.15.103:0/1490894455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dac32d460 con 0x563dac3155d0 2024-09-19T06:53:55.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.907+0000 7f4dfcc93640 1 --2- 172.21.15.103:0/1490894455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac31ee10 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f4df0004b90 tx=0x7f4df001a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d6bba2b01711a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:55.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.907+0000 7f4dd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:55.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.907+0000 7f4dd77fe640 1 -- 172.21.15.103:0/1490894455 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df001d650 con 0x563dac3155d0 2024-09-19T06:53:55.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.907+0000 7f4dd77fe640 1 -- 172.21.15.103:0/1490894455 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f4df001d7f0 con 0x563dac3155d0 2024-09-19T06:53:55.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.907+0000 7f4e00acf740 1 -- 172.21.15.103:0/1490894455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4df0022070 con 0x563dac3155d0 2024-09-19T06:53:55.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.907+0000 7f4dd77fe640 1 -- 172.21.15.103:0/1490894455 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4df0007000 con 0x563dac3155d0 2024-09-19T06:53:55.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.907+0000 7f4e00acf740 10 client.17178 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:55.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.907+0000 7f4e00acf740 1 -- 172.21.15.103:0/1490894455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563dac39f0c0 con 0x563dac3155d0 2024-09-19T06:53:55.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.907+0000 7f4dd77fe640 1 -- 172.21.15.103:0/1490894455 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df0022070 con 0x563dac3155d0 2024-09-19T06:53:55.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.907+0000 7f4dd77fe640 1 -- 172.21.15.103:0/1490894455 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4df004e9a0 con 0x563dac3155d0 2024-09-19T06:53:55.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.907+0000 7f4e00acf740 10 client.17178 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:55.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.907+0000 7f4e00acf740 10 client.17178 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:53:55.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.907+0000 7f4e00acf740 20 client.17178 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:55.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.907+0000 7f4e00acf740 1 --2- 172.21.15.103:0/1490894455 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x563dac39f720 0x563dac3a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:55.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.907+0000 7f4e00acf740 4 client.17178 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:53:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.907+0000 7f4e00acf740 1 -- 172.21.15.103:0/1490894455 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563dac282d70 con 0x563dac39f720 2024-09-19T06:53:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.908+0000 7f4df77fe640 1 --2- 172.21.15.103:0/1490894455 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x563dac39f720 0x563dac3a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.908+0000 7f4df77fe640 1 --2- 172.21.15.103:0/1490894455 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x563dac39f720 0x563dac3a1b10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52196a8bbcdfcb48 server_cookie=559ea0366bd791ad in_seq=0 out_seq=0 2024-09-19T06:53:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.908+0000 7f4dd77fe640 10 client.17178 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:53:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.908+0000 7f4dd77fe640 1 -- 172.21.15.103:0/1490894455 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563dac282d70 con 0x563dac39f720 2024-09-19T06:53:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.908+0000 7f4dd77fe640 10 client.17178 handle_command_reply: tid=0 2024-09-19T06:53:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.908+0000 7f4e00acf740 1 client.17178 shutdown 2024-09-19T06:53:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.908+0000 7f4e00acf740 1 -- 172.21.15.103:0/1490894455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 msgr2=0x563dac31ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.908+0000 7f4e00acf740 1 --2- 172.21.15.103:0/1490894455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac31ee10 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f4df0004b90 tx=0x7f4df001a7d0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.908+0000 7f4e00acf740 1 -- 172.21.15.103:0/1490894455 shutdown_connections 2024-09-19T06:53:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.908+0000 7f4e00acf740 1 --2- 172.21.15.103:0/1490894455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563dac324780 0x563dac32cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.908+0000 7f4e00acf740 1 --2- 172.21.15.103:0/1490894455 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x563dac39f720 0x563dac3a1b10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.908+0000 7f4e00acf740 1 --2- 172.21.15.103:0/1490894455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563dac316b40 0x563dac32c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.908+0000 7f4e00acf740 1 --2- 172.21.15.103:0/1490894455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563dac3155d0 0x563dac31ee10 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.908+0000 7f4dd77fe640 0 client.17178 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:53:55.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.909+0000 7f4e00acf740 1 -- 172.21.15.103:0/1490894455 >> 172.21.15.103:0/1490894455 conn(0x563dac2de310 msgr2=0x563dac317930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:55.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.909+0000 7f4e00acf740 1 -- 172.21.15.103:0/1490894455 shutdown_connections 2024-09-19T06:53:55.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.909+0000 7f4e00acf740 1 -- 172.21.15.103:0/1490894455 wait complete. 2024-09-19T06:53:55.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.909+0000 7f4e00acf740 20 client.17178 trim_cache size 0 max 16384 2024-09-19T06:53:55.932 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:53:55.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.909+0000 7f4dfe6f6640 1 -- 172.21.15.103:0/2574973122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4de00a33b0 msgr2=0x7f4de00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:55.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.909+0000 7f4dfe6f6640 1 --2- 172.21.15.103:0/2574973122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4de00a33b0 0x7f4de00a5870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4de809b010 tx=0x7f4de800ff30 comp rx=0 tx=0).stop 2024-09-19T06:53:55.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.909+0000 7f4dfe6f6640 1 -- 172.21.15.103:0/2574973122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4df8100480 msgr2=0x7f4df81984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:55.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.909+0000 7f4dfe6f6640 1 --2- 172.21.15.103:0/2574973122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4df8100480 0x7f4df81984b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f4de800c920 tx=0x7f4de800cde0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.909+0000 7f4dfe6f6640 1 -- 172.21.15.103:0/2574973122 shutdown_connections 2024-09-19T06:53:55.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.909+0000 7f4dfe6f6640 1 --2- 172.21.15.103:0/2574973122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4de00a33b0 0x7f4de00a5870 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.909+0000 7f4dfe6f6640 1 --2- 172.21.15.103:0/2574973122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df81055b0 0x7f4df819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.909+0000 7f4dfe6f6640 1 --2- 172.21.15.103:0/2574973122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4df8100e30 0x7f4df81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.909+0000 7f4dfe6f6640 1 --2- 172.21.15.103:0/2574973122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4df8100480 0x7f4df81984b0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:55.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.909+0000 7f4dfe6f6640 1 -- 172.21.15.103:0/2574973122 >> 172.21.15.103:0/2574973122 conn(0x7f4df8077210 msgr2=0x7f4df8103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:55.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.909+0000 7f4dfe6f6640 1 -- 172.21.15.103:0/2574973122 shutdown_connections 2024-09-19T06:53:55.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:55.909+0000 7f4dfe6f6640 1 -- 172.21.15.103:0/2574973122 wait complete. 2024-09-19T06:53:56.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:56 smithi103 ceph-mon[31158]: pgmap v144: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-09-19T06:53:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:56 smithi174 ceph-mon[35637]: pgmap v144: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-09-19T06:53:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:56 smithi134 ceph-mon[35356]: pgmap v144: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-09-19T06:53:56.607 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:53:56.863 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:57.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.315+0000 7f423ec72640 1 -- 172.21.15.103:0/395501513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4238102790 msgr2=0x7f42381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:57.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.315+0000 7f423ec72640 1 --2- 172.21.15.103:0/395501513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4238102790 0x7f42381067e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f4224009900 tx=0x7f422402a410 comp rx=0 tx=0).stop 2024-09-19T06:53:57.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.315+0000 7f423ec72640 1 -- 172.21.15.103:0/395501513 shutdown_connections 2024-09-19T06:53:57.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.315+0000 7f423ec72640 1 --2- 172.21.15.103:0/395501513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4238106f10 0x7f423810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.315+0000 7f423ec72640 1 --2- 172.21.15.103:0/395501513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4238102790 0x7f42381067e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.315+0000 7f423ec72640 1 --2- 172.21.15.103:0/395501513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4238101de0 0x7f42381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.315+0000 7f423ec72640 1 -- 172.21.15.103:0/395501513 >> 172.21.15.103:0/395501513 conn(0x7f42380fd8c0 msgr2=0x7f42380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:57.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.315+0000 7f423ec72640 1 -- 172.21.15.103:0/395501513 shutdown_connections 2024-09-19T06:53:57.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.315+0000 7f423ec72640 1 -- 172.21.15.103:0/395501513 wait complete. 2024-09-19T06:53:57.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.315+0000 7f423ec72640 1 Processor -- start 2024-09-19T06:53:57.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.315+0000 7f423ec72640 1 -- start start 2024-09-19T06:53:57.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423ec72640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4238101de0 0x7f42381985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423ec72640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4238102790 0x7f4238198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423ec72640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4238106f10 0x7f423819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423ec72640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4238112010 con 0x7f4238106f10 2024-09-19T06:53:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423ec72640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4238111e90 con 0x7f4238102790 2024-09-19T06:53:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423ec72640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4238112190 con 0x7f4238101de0 2024-09-19T06:53:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423d20f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4238106f10 0x7f423819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423d20f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4238106f10 0x7f423819ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54764/0 (socket says 172.21.15.103:54764) 2024-09-19T06:53:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423d20f640 1 -- 172.21.15.103:0/1342942195 learned_addr learned my addr 172.21.15.103:0/1342942195 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423ca0e640 1 --2- 172.21.15.103:0/1342942195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4238101de0 0x7f42381985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f4237fff640 1 --2- 172.21.15.103:0/1342942195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4238102790 0x7f4238198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423d20f640 1 -- 172.21.15.103:0/1342942195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4238101de0 msgr2=0x7f42381985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423d20f640 1 --2- 172.21.15.103:0/1342942195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4238101de0 0x7f42381985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423d20f640 1 -- 172.21.15.103:0/1342942195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4238102790 msgr2=0x7f4238198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423d20f640 1 --2- 172.21.15.103:0/1342942195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4238102790 0x7f4238198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423d20f640 1 -- 172.21.15.103:0/1342942195 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423819d560 con 0x7f4238106f10 2024-09-19T06:53:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423d20f640 1 --2- 172.21.15.103:0/1342942195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4238106f10 0x7f423819ce80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f423000b970 tx=0x7f423000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f4235ffb640 1 -- 172.21.15.103:0/1342942195 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230019070 con 0x7f4238106f10 2024-09-19T06:53:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f423ec72640 1 -- 172.21.15.103:0/1342942195 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423819d850 con 0x7f4238106f10 2024-09-19T06:53:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.316+0000 7f4235ffb640 1 -- 172.21.15.103:0/1342942195 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f423000cb40 con 0x7f4238106f10 2024-09-19T06:53:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.317+0000 7f4235ffb640 1 -- 172.21.15.103:0/1342942195 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230007610 con 0x7f4238106f10 2024-09-19T06:53:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.317+0000 7f423ec72640 1 -- 172.21.15.103:0/1342942195 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42381a5130 con 0x7f4238106f10 2024-09-19T06:53:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.319+0000 7f4235ffb640 1 -- 172.21.15.103:0/1342942195 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f423001c070 con 0x7f4238106f10 2024-09-19T06:53:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.319+0000 7f424104b740 1 Processor -- start 2024-09-19T06:53:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.319+0000 7f424104b740 1 -- start start 2024-09-19T06:53:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.319+0000 7f424104b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a663b40 0x56009a662090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.319+0000 7f424104b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a6625d0 0x56009a662a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.319+0000 7f424104b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a66d370 0x56009a66f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.319+0000 7f424104b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56009a5c82f0 con 0x56009a66d370 2024-09-19T06:53:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.319+0000 7f424104b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56009a56fcf0 con 0x56009a6625d0 2024-09-19T06:53:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.319+0000 7f424104b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56009a5c42b0 con 0x56009a663b40 2024-09-19T06:53:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.319+0000 7f4235ffb640 1 --2- 172.21.15.103:0/1342942195 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f42280a33b0 0x7f42280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f4237fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a66d370 0x56009a66f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f4237fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a66d370 0x56009a66f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54780/0 (socket says 172.21.15.103:54780) 2024-09-19T06:53:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f4237fff640 1 -- 172.21.15.103:0/3218082532 learned_addr learned my addr 172.21.15.103:0/3218082532 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f423ca0e640 1 --2- 172.21.15.103:0/3218082532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a6625d0 0x56009a662a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f423d20f640 1 --2- 172.21.15.103:0/1342942195 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f42280a33b0 0x7f42280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f4237fff640 1 --2- 172.21.15.103:0/3218082532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a663b40 0x56009a662090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f4235ffb640 1 -- 172.21.15.103:0/1342942195 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f42300cf240 con 0x7f4238106f10 2024-09-19T06:53:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f4237fff640 1 -- 172.21.15.103:0/3218082532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a663b40 msgr2=0x56009a662090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f4237fff640 1 --2- 172.21.15.103:0/3218082532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a663b40 0x56009a662090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f4237fff640 1 -- 172.21.15.103:0/3218082532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a6625d0 msgr2=0x56009a662a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f4237fff640 1 --2- 172.21.15.103:0/3218082532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a6625d0 0x56009a662a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f4237fff640 1 -- 172.21.15.103:0/3218082532 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009a5543a0 con 0x56009a66d370 2024-09-19T06:53:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f423ca0e640 1 --2- 172.21.15.103:0/3218082532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a6625d0 0x56009a662a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f4237fff640 1 --2- 172.21.15.103:0/3218082532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a66d370 0x56009a66f760 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4224005940 tx=0x7f42240058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fcbe7730889a70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f423d20f640 1 --2- 172.21.15.103:0/1342942195 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f42280a33b0 0x7f42280a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4230097aa0 tx=0x7f4230090ad0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f4222ffd640 1 -- 172.21.15.103:0/3218082532 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224043030 con 0x56009a66d370 2024-09-19T06:53:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f4222ffd640 1 -- 172.21.15.103:0/3218082532 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4224031750 con 0x56009a66d370 2024-09-19T06:53:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.320+0000 7f4222ffd640 1 -- 172.21.15.103:0/3218082532 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224035040 con 0x56009a66d370 2024-09-19T06:53:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.321+0000 7f424104b740 1 -- 172.21.15.103:0/3218082532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a66d370 msgr2=0x56009a66f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.321+0000 7f424104b740 1 --2- 172.21.15.103:0/3218082532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a66d370 0x56009a66f760 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4224005940 tx=0x7f42240058d0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.321+0000 7f424104b740 1 -- 172.21.15.103:0/3218082532 shutdown_connections 2024-09-19T06:53:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.321+0000 7f424104b740 1 --2- 172.21.15.103:0/3218082532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a66d370 0x56009a66f760 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.321+0000 7f424104b740 1 --2- 172.21.15.103:0/3218082532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a6625d0 0x56009a662a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.321+0000 7f424104b740 1 --2- 172.21.15.103:0/3218082532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a663b40 0x56009a662090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.321+0000 7f424104b740 1 -- 172.21.15.103:0/3218082532 >> 172.21.15.103:0/3218082532 conn(0x56009a62b310 msgr2=0x56009a65ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.321+0000 7f424104b740 1 -- 172.21.15.103:0/3218082532 shutdown_connections 2024-09-19T06:53:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.321+0000 7f424104b740 1 -- 172.21.15.103:0/3218082532 wait complete. 2024-09-19T06:53:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.321+0000 7f424104b740 1 -- start start 2024-09-19T06:53:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.321+0000 7f424104b740 1 Processor -- start 2024-09-19T06:53:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f424104b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a6625d0 0x56009a66cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f424104b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a663b40 0x56009a66a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f424104b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a66d370 0x56009a66a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f424104b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56009a670010 con 0x56009a66d370 2024-09-19T06:53:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f424104b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56009a583060 con 0x56009a6625d0 2024-09-19T06:53:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f424104b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56009a670190 con 0x56009a663b40 2024-09-19T06:53:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f424104b740 10 client.0 _ll_register_callbacks cb 0x56009a2d68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f4237fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a66d370 0x56009a66a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f4237fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a66d370 0x56009a66a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54790/0 (socket says 172.21.15.103:54790) 2024-09-19T06:53:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f4237fff640 1 -- 172.21.15.103:0/4050186596 learned_addr learned my addr 172.21.15.103:0/4050186596 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f423ca0e640 1 --2- 172.21.15.103:0/4050186596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a663b40 0x56009a66a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f4237fff640 1 -- 172.21.15.103:0/4050186596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a663b40 msgr2=0x56009a66a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f4237fff640 1 --2- 172.21.15.103:0/4050186596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a663b40 0x56009a66a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f4237fff640 1 -- 172.21.15.103:0/4050186596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a6625d0 msgr2=0x56009a66cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f4237fff640 1 --2- 172.21.15.103:0/4050186596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a6625d0 0x56009a66cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f4237fff640 1 -- 172.21.15.103:0/4050186596 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56009a3393c0 con 0x56009a66d370 2024-09-19T06:53:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f4237fff640 1 --2- 172.21.15.103:0/4050186596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a66d370 0x56009a66a610 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4224006010 tx=0x7f4224031b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5082e320425d2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.322+0000 7f4217fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.323+0000 7f4217fff640 1 -- 172.21.15.103:0/4050186596 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224035030 con 0x56009a66d370 2024-09-19T06:53:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.323+0000 7f4217fff640 1 -- 172.21.15.103:0/4050186596 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f4224004310 con 0x56009a66d370 2024-09-19T06:53:57.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.323+0000 7f424104b740 1 -- 172.21.15.103:0/4050186596 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4224031750 con 0x56009a66d370 2024-09-19T06:53:57.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.323+0000 7f4217fff640 1 -- 172.21.15.103:0/4050186596 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4224058050 con 0x56009a66d370 2024-09-19T06:53:57.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.323+0000 7f424104b740 10 client.17190 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:57.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.323+0000 7f424104b740 1 -- 172.21.15.103:0/4050186596 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56009a67bab0 con 0x56009a66d370 2024-09-19T06:53:57.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.323+0000 7f4217fff640 1 -- 172.21.15.103:0/4050186596 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4224031750 con 0x56009a66d370 2024-09-19T06:53:57.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.323+0000 7f4217fff640 1 -- 172.21.15.103:0/4050186596 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f422403a040 con 0x56009a66d370 2024-09-19T06:53:57.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.323+0000 7f424104b740 10 client.17190 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:57.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.323+0000 7f424104b740 10 client.17190 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:53:57.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.323+0000 7f424104b740 20 client.17190 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:57.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.323+0000 7f424104b740 1 --2- 172.21.15.103:0/4050186596 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56009a67ce00 0x56009a67f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.323+0000 7f424104b740 4 client.17190 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:57.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.323+0000 7f424104b740 1 -- 172.21.15.103:0/4050186596 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56009a5d5860 con 0x56009a67ce00 2024-09-19T06:53:57.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.324+0000 7f423ca0e640 1 --2- 172.21.15.103:0/4050186596 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56009a67ce00 0x56009a67f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.324+0000 7f4217fff640 10 client.17190 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:53:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.324+0000 7f423ca0e640 1 --2- 172.21.15.103:0/4050186596 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56009a67ce00 0x56009a67f1f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b865ac93d4b6691e server_cookie=68e85d783b01208a in_seq=0 out_seq=0 2024-09-19T06:53:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.325+0000 7f4217fff640 1 -- 172.21.15.103:0/4050186596 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56009a5d5860 con 0x56009a67ce00 2024-09-19T06:53:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.325+0000 7f4217fff640 10 client.17190 handle_command_reply: tid=0 2024-09-19T06:53:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.325+0000 7f424104b740 1 client.17190 shutdown 2024-09-19T06:53:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.325+0000 7f424104b740 1 -- 172.21.15.103:0/4050186596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a66d370 msgr2=0x56009a66a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.325+0000 7f424104b740 1 --2- 172.21.15.103:0/4050186596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a66d370 0x56009a66a610 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4224006010 tx=0x7f4224031b30 comp rx=0 tx=0).stop 2024-09-19T06:53:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.325+0000 7f424104b740 1 -- 172.21.15.103:0/4050186596 shutdown_connections 2024-09-19T06:53:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.325+0000 7f424104b740 1 --2- 172.21.15.103:0/4050186596 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56009a67ce00 0x56009a67f1f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.325+0000 7f424104b740 1 --2- 172.21.15.103:0/4050186596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a66d370 0x56009a66a610 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.325+0000 7f424104b740 1 --2- 172.21.15.103:0/4050186596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a663b40 0x56009a66a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.325+0000 7f424104b740 1 --2- 172.21.15.103:0/4050186596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a6625d0 0x56009a66cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.325+0000 7f424104b740 1 -- 172.21.15.103:0/4050186596 >> 172.21.15.103:0/4050186596 conn(0x56009a62b310 msgr2=0x56009a664960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:57.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.325+0000 7f4217fff640 0 client.17190 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:53:57.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.325+0000 7f424104b740 1 -- 172.21.15.103:0/4050186596 shutdown_connections 2024-09-19T06:53:57.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.325+0000 7f424104b740 1 -- 172.21.15.103:0/4050186596 wait complete. 2024-09-19T06:53:57.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.325+0000 7f424104b740 20 client.17190 trim_cache size 0 max 16384 2024-09-19T06:53:57.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f424104b740 1 Processor -- start 2024-09-19T06:53:57.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f424104b740 1 -- start start 2024-09-19T06:53:57.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f424104b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a6625d0 0x56009a6792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f424104b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a663b40 0x56009a65e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f424104b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a66d370 0x56009a65ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f424104b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42381ad880 con 0x56009a663b40 2024-09-19T06:53:57.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f424104b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42381af3c0 con 0x56009a66d370 2024-09-19T06:53:57.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f424104b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42381adf50 con 0x56009a6625d0 2024-09-19T06:53:57.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f423ca0e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a663b40 0x56009a65e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f423ca0e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a663b40 0x56009a65e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54792/0 (socket says 172.21.15.103:54792) 2024-09-19T06:53:57.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f423ca0e640 1 -- 172.21.15.103:0/3336693331 learned_addr learned my addr 172.21.15.103:0/3336693331 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:57.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f4237fff640 1 --2- 172.21.15.103:0/3336693331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a66d370 0x56009a65ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f4237fff640 1 --2- 172.21.15.103:0/3336693331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a6625d0 0x56009a6792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f423ca0e640 1 -- 172.21.15.103:0/3336693331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a6625d0 msgr2=0x56009a6792b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:57.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f423ca0e640 1 --2- 172.21.15.103:0/3336693331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a6625d0 0x56009a6792b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f423ca0e640 1 -- 172.21.15.103:0/3336693331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a66d370 msgr2=0x56009a65ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:57.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f423ca0e640 1 --2- 172.21.15.103:0/3336693331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a66d370 0x56009a65ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f423ca0e640 1 -- 172.21.15.103:0/3336693331 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009a6622d0 con 0x56009a663b40 2024-09-19T06:53:57.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.338+0000 7f4237fff640 1 --2- 172.21.15.103:0/3336693331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a66d370 0x56009a65ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:57.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.339+0000 7f423ca0e640 1 --2- 172.21.15.103:0/3336693331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a663b40 0x56009a65e6e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f422c00b5b0 tx=0x7f422c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b858646d84f3da6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:57.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.339+0000 7f42227fc640 1 -- 172.21.15.103:0/3336693331 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c028070 con 0x56009a663b40 2024-09-19T06:53:57.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.339+0000 7f42227fc640 1 -- 172.21.15.103:0/3336693331 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f422c018c30 con 0x56009a663b40 2024-09-19T06:53:57.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.339+0000 7f42227fc640 1 -- 172.21.15.103:0/3336693331 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c019650 con 0x56009a663b40 2024-09-19T06:53:57.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.339+0000 7f424104b740 1 -- 172.21.15.103:0/3336693331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a663b40 msgr2=0x56009a65e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:57.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.339+0000 7f424104b740 1 --2- 172.21.15.103:0/3336693331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a663b40 0x56009a65e6e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f422c00b5b0 tx=0x7f422c008040 comp rx=0 tx=0).stop 2024-09-19T06:53:57.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.339+0000 7f424104b740 1 -- 172.21.15.103:0/3336693331 shutdown_connections 2024-09-19T06:53:57.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.339+0000 7f424104b740 1 --2- 172.21.15.103:0/3336693331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a66d370 0x56009a65ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.339+0000 7f424104b740 1 --2- 172.21.15.103:0/3336693331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a663b40 0x56009a65e6e0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.339+0000 7f424104b740 1 --2- 172.21.15.103:0/3336693331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a6625d0 0x56009a6792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.339+0000 7f424104b740 1 -- 172.21.15.103:0/3336693331 >> 172.21.15.103:0/3336693331 conn(0x56009a62b310 msgr2=0x56009a669fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:57.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.340+0000 7f424104b740 1 -- 172.21.15.103:0/3336693331 shutdown_connections 2024-09-19T06:53:57.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.340+0000 7f424104b740 1 -- 172.21.15.103:0/3336693331 wait complete. 2024-09-19T06:53:57.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.340+0000 7f424104b740 1 -- start start 2024-09-19T06:53:57.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.340+0000 7f424104b740 1 Processor -- start 2024-09-19T06:53:57.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.340+0000 7f424104b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a6625d0 0x56009a6e29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.340+0000 7f424104b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a663b40 0x56009a6e2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.340+0000 7f424104b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a66d370 0x56009a675420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.340+0000 7f424104b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42381aecf0 con 0x56009a663b40 2024-09-19T06:53:57.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.340+0000 7f424104b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42381ae620 con 0x56009a6625d0 2024-09-19T06:53:57.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.340+0000 7f424104b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42381ab840 con 0x56009a66d370 2024-09-19T06:53:57.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f424104b740 10 client.0 _ll_register_callbacks cb 0x7f422c00b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:57.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f423ca0e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a663b40 0x56009a6e2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f4237fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a6625d0 0x56009a6e29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f4237fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a6625d0 0x56009a6e29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:34412/0 (socket says 172.21.15.103:34412) 2024-09-19T06:53:57.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f4237fff640 1 -- 172.21.15.103:0/1528759607 learned_addr learned my addr 172.21.15.103:0/1528759607 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:57.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f4237fff640 1 --2- 172.21.15.103:0/1528759607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a66d370 0x56009a675420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f423ca0e640 1 -- 172.21.15.103:0/1528759607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a66d370 msgr2=0x56009a675420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:57.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f423ca0e640 1 --2- 172.21.15.103:0/1528759607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a66d370 0x56009a675420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f423ca0e640 1 -- 172.21.15.103:0/1528759607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a6625d0 msgr2=0x56009a6e29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:57.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f423ca0e640 1 --2- 172.21.15.103:0/1528759607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a6625d0 0x56009a6e29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f423ca0e640 1 -- 172.21.15.103:0/1528759607 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56009a675b90 con 0x56009a663b40 2024-09-19T06:53:57.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f423ca0e640 1 --2- 172.21.15.103:0/1528759607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a663b40 0x56009a6e2f30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x56009a65a260 tx=0x7f422c018aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbac60f1b5734dc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:57.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f4217fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:57.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f4217fff640 1 -- 172.21.15.103:0/1528759607 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c033060 con 0x56009a663b40 2024-09-19T06:53:57.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f4217fff640 1 -- 172.21.15.103:0/1528759607 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f422c023660 con 0x56009a663b40 2024-09-19T06:53:57.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f424104b740 1 -- 172.21.15.103:0/1528759607 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f422c018c30 con 0x56009a663b40 2024-09-19T06:53:57.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f4217fff640 1 -- 172.21.15.103:0/1528759607 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f422c028040 con 0x56009a663b40 2024-09-19T06:53:57.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f424104b740 10 client.17196 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:57.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.341+0000 7f424104b740 1 -- 172.21.15.103:0/1528759607 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56009a667560 con 0x56009a663b40 2024-09-19T06:53:57.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.342+0000 7f4217fff640 1 -- 172.21.15.103:0/1528759607 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f422c018c30 con 0x56009a663b40 2024-09-19T06:53:57.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.342+0000 7f4217fff640 1 -- 172.21.15.103:0/1528759607 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f422c016050 con 0x56009a663b40 2024-09-19T06:53:57.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.342+0000 7f424104b740 10 client.17196 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:57.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.342+0000 7f424104b740 10 client.17196 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:53:57.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.342+0000 7f424104b740 20 client.17196 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:57.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.342+0000 7f424104b740 1 --2- 172.21.15.103:0/1528759607 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56009a6ebe20 0x56009a6ee240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:57.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.342+0000 7f424104b740 4 client.17196 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:53:57.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.342+0000 7f424104b740 1 -- 172.21.15.103:0/1528759607 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56009a5cfd70 con 0x56009a6ebe20 2024-09-19T06:53:57.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.342+0000 7f4237fff640 1 --2- 172.21.15.103:0/1528759607 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56009a6ebe20 0x56009a6ee240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:57.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f4237fff640 1 --2- 172.21.15.103:0/1528759607 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56009a6ebe20 0x56009a6ee240 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db4073835bc11fe6 server_cookie=1d9bdad1d00716ba in_seq=0 out_seq=0 2024-09-19T06:53:57.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f4217fff640 10 client.17196 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:53:57.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f4217fff640 1 -- 172.21.15.103:0/1528759607 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56009a5cfd70 con 0x56009a6ebe20 2024-09-19T06:53:57.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f4217fff640 10 client.17196 handle_command_reply: tid=0 2024-09-19T06:53:57.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f424104b740 1 client.17196 shutdown 2024-09-19T06:53:57.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f424104b740 1 -- 172.21.15.103:0/1528759607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a663b40 msgr2=0x56009a6e2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:57.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f424104b740 1 --2- 172.21.15.103:0/1528759607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a663b40 0x56009a6e2f30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x56009a65a260 tx=0x7f422c018aa0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f424104b740 1 -- 172.21.15.103:0/1528759607 shutdown_connections 2024-09-19T06:53:57.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f424104b740 1 --2- 172.21.15.103:0/1528759607 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56009a6ebe20 0x56009a6ee240 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f424104b740 1 --2- 172.21.15.103:0/1528759607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009a66d370 0x56009a675420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f424104b740 1 --2- 172.21.15.103:0/1528759607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56009a663b40 0x56009a6e2f30 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f4217fff640 0 client.17196 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:53:57.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f424104b740 1 --2- 172.21.15.103:0/1528759607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56009a6625d0 0x56009a6e29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f424104b740 1 -- 172.21.15.103:0/1528759607 >> 172.21.15.103:0/1528759607 conn(0x56009a62b310 msgr2=0x56009a65e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:57.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f424104b740 1 -- 172.21.15.103:0/1528759607 shutdown_connections 2024-09-19T06:53:57.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f424104b740 1 -- 172.21.15.103:0/1528759607 wait complete. 2024-09-19T06:53:57.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.343+0000 7f424104b740 20 client.17196 trim_cache size 0 max 16384 2024-09-19T06:53:57.351 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:53:57.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.344+0000 7f423ec72640 1 -- 172.21.15.103:0/1342942195 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f42280a33b0 msgr2=0x7f42280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:57.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.344+0000 7f423ec72640 1 --2- 172.21.15.103:0/1342942195 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f42280a33b0 0x7f42280a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4230097aa0 tx=0x7f4230090ad0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.344+0000 7f423ec72640 1 -- 172.21.15.103:0/1342942195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4238106f10 msgr2=0x7f423819ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:57.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.344+0000 7f423ec72640 1 --2- 172.21.15.103:0/1342942195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4238106f10 0x7f423819ce80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f423000b970 tx=0x7f423000be30 comp rx=0 tx=0).stop 2024-09-19T06:53:57.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.344+0000 7f423ec72640 1 -- 172.21.15.103:0/1342942195 shutdown_connections 2024-09-19T06:53:57.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.344+0000 7f423ec72640 1 --2- 172.21.15.103:0/1342942195 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f42280a33b0 0x7f42280a5870 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.344+0000 7f423ec72640 1 --2- 172.21.15.103:0/1342942195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4238106f10 0x7f423819ce80 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.344+0000 7f423ec72640 1 --2- 172.21.15.103:0/1342942195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4238102790 0x7f4238198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.344+0000 7f423ec72640 1 --2- 172.21.15.103:0/1342942195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4238101de0 0x7f42381985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:57.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.344+0000 7f423ec72640 1 -- 172.21.15.103:0/1342942195 >> 172.21.15.103:0/1342942195 conn(0x7f42380fd8c0 msgr2=0x7f42380fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:57.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.344+0000 7f423ec72640 1 -- 172.21.15.103:0/1342942195 shutdown_connections 2024-09-19T06:53:57.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:57.344+0000 7f423ec72640 1 -- 172.21.15.103:0/1342942195 wait complete. 2024-09-19T06:53:57.968 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:53:58.216 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:58.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:53:58 smithi174 ceph-mon[35637]: pgmap v145: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2024-09-19T06:53:58.336 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:53:58 smithi103 ceph-mon[31158]: pgmap v145: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2024-09-19T06:53:58.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:53:58 smithi134 ceph-mon[35356]: pgmap v145: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2024-09-19T06:53:58.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.683+0000 7fbfc7a67640 1 -- 172.21.15.103:0/997779197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfc0106f20 msgr2=0x7fbfc010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:58.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.683+0000 7fbfc7a67640 1 --2- 172.21.15.103:0/997779197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfc0106f20 0x7fbfc010eaa0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009900 tx=0x7fbfb802a410 comp rx=0 tx=0).stop 2024-09-19T06:53:58.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.683+0000 7fbfc7a67640 1 -- 172.21.15.103:0/997779197 shutdown_connections 2024-09-19T06:53:58.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.683+0000 7fbfc7a67640 1 --2- 172.21.15.103:0/997779197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfc0106f20 0x7fbfc010eaa0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.683+0000 7fbfc7a67640 1 --2- 172.21.15.103:0/997779197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc01027a0 0x7fbfc01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.683+0000 7fbfc7a67640 1 --2- 172.21.15.103:0/997779197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc0101df0 0x7fbfc01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.683+0000 7fbfc7a67640 1 -- 172.21.15.103:0/997779197 >> 172.21.15.103:0/997779197 conn(0x7fbfc00fd8f0 msgr2=0x7fbfc00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:58.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.683+0000 7fbfc7a67640 1 -- 172.21.15.103:0/997779197 shutdown_connections 2024-09-19T06:53:58.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.683+0000 7fbfc7a67640 1 -- 172.21.15.103:0/997779197 wait complete. 2024-09-19T06:53:58.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.684+0000 7fbfc7a67640 1 Processor -- start 2024-09-19T06:53:58.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.684+0000 7fbfc7a67640 1 -- start start 2024-09-19T06:53:58.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.684+0000 7fbfc7a67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfc0101df0 0x7fbfc01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.684+0000 7fbfc7a67640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc01027a0 0x7fbfc0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.684+0000 7fbfc7a67640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc0106f20 0x7fbfc019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.684+0000 7fbfc7a67640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc0111fd0 con 0x7fbfc0101df0 2024-09-19T06:53:58.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.684+0000 7fbfc7a67640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc0111e50 con 0x7fbfc0106f20 2024-09-19T06:53:58.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.684+0000 7fbfc7a67640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc0112150 con 0x7fbfc01027a0 2024-09-19T06:53:58.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.684+0000 7fbfc5803640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfc0101df0 0x7fbfc01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:58.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.684+0000 7fbfc5803640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfc0101df0 0x7fbfc01983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54820/0 (socket says 172.21.15.103:54820) 2024-09-19T06:53:58.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.684+0000 7fbfc5803640 1 -- 172.21.15.103:0/107829505 learned_addr learned my addr 172.21.15.103:0/107829505 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:58.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.684+0000 7fbfc5002640 1 --2- 172.21.15.103:0/107829505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc01027a0 0x7fbfc0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:58.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.684+0000 7fbfc6004640 1 --2- 172.21.15.103:0/107829505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc0106f20 0x7fbfc019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:58.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.685+0000 7fbfc5803640 1 -- 172.21.15.103:0/107829505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc01027a0 msgr2=0x7fbfc0198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:58.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.685+0000 7fbfc5803640 1 --2- 172.21.15.103:0/107829505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc01027a0 0x7fbfc0198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.685+0000 7fbfc5803640 1 -- 172.21.15.103:0/107829505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc0106f20 msgr2=0x7fbfc019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:58.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.685+0000 7fbfc5803640 1 --2- 172.21.15.103:0/107829505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc0106f20 0x7fbfc019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.685+0000 7fbfc5803640 1 -- 172.21.15.103:0/107829505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc019d3a0 con 0x7fbfc0101df0 2024-09-19T06:53:58.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.685+0000 7fbfc6004640 1 --2- 172.21.15.103:0/107829505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc0106f20 0x7fbfc019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:58.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.685+0000 7fbfc5002640 1 --2- 172.21.15.103:0/107829505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc01027a0 0x7fbfc0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:58.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.685+0000 7fbfc5803640 1 --2- 172.21.15.103:0/107829505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfc0101df0 0x7fbfc01983f0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00db80 tx=0x7fbfbc00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:58.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.685+0000 7fbfb6ffd640 1 -- 172.21.15.103:0/107829505 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc017070 con 0x7fbfc0101df0 2024-09-19T06:53:58.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.685+0000 7fbfc7a67640 1 -- 172.21.15.103:0/107829505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc019d690 con 0x7fbfc0101df0 2024-09-19T06:53:58.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.685+0000 7fbfb6ffd640 1 -- 172.21.15.103:0/107829505 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfbc003b70 con 0x7fbfc0101df0 2024-09-19T06:53:58.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.685+0000 7fbfb6ffd640 1 -- 172.21.15.103:0/107829505 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0106c0 con 0x7fbfc0101df0 2024-09-19T06:53:58.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.685+0000 7fbfc7a67640 1 -- 172.21.15.103:0/107829505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc01a4f70 con 0x7fbfc0101df0 2024-09-19T06:53:58.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.687+0000 7fbfb6ffd640 1 -- 172.21.15.103:0/107829505 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fbfbc004020 con 0x7fbfc0101df0 2024-09-19T06:53:58.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.687+0000 7fbfc9e40740 1 Processor -- start 2024-09-19T06:53:58.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.688+0000 7fbfc9e40740 1 -- start start 2024-09-19T06:53:58.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.688+0000 7fbfc9e40740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face2b40 0x5628face1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.688+0000 7fbfc9e40740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628face15d0 0x5628face1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.688+0000 7fbfc9e40740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628facec370 0x5628facee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.688+0000 7fbfc9e40740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5628fac472f0 con 0x5628facec370 2024-09-19T06:53:58.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.688+0000 7fbfc9e40740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5628fabeecf0 con 0x5628face2b40 2024-09-19T06:53:58.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.688+0000 7fbfc9e40740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5628fac432b0 con 0x5628face15d0 2024-09-19T06:53:58.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.688+0000 7fbfb6ffd640 1 --2- 172.21.15.103:0/107829505 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbfac0a33b0 0x7fbfac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.688+0000 7fbfc5002640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628facec370 0x5628facee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:58.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.688+0000 7fbfc5002640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628facec370 0x5628facee760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54836/0 (socket says 172.21.15.103:54836) 2024-09-19T06:53:58.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.688+0000 7fbfc5002640 1 -- 172.21.15.103:0/2135668340 learned_addr learned my addr 172.21.15.103:0/2135668340 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:58.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.688+0000 7fbfc6004640 1 --2- 172.21.15.103:0/107829505 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbfac0a33b0 0x7fbfac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:58.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.688+0000 7fbfc5803640 1 --2- 172.21.15.103:0/2135668340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628face15d0 0x5628face1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:58.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.688+0000 7fbfc6004640 1 --2- 172.21.15.103:0/2135668340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face2b40 0x5628face1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:58.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.688+0000 7fbfb6ffd640 1 -- 172.21.15.103:0/107829505 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fbfbc0cfbb0 con 0x7fbfc0101df0 2024-09-19T06:53:58.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.689+0000 7fbfc5002640 1 -- 172.21.15.103:0/2135668340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628face15d0 msgr2=0x5628face1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:58.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.689+0000 7fbfc5002640 1 --2- 172.21.15.103:0/2135668340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628face15d0 0x5628face1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.689+0000 7fbfc5002640 1 -- 172.21.15.103:0/2135668340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face2b40 msgr2=0x5628face1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:58.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.689+0000 7fbfc5002640 1 --2- 172.21.15.103:0/2135668340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face2b40 0x5628face1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.689+0000 7fbfc5002640 1 -- 172.21.15.103:0/2135668340 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628fabd33a0 con 0x5628facec370 2024-09-19T06:53:58.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.689+0000 7fbfc5002640 1 --2- 172.21.15.103:0/2135668340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628facec370 0x5628facee760 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009c60 tx=0x7fbfb00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b365cb2072611e1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:58.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.689+0000 7fbfc6004640 1 --2- 172.21.15.103:0/107829505 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbfac0a33b0 0x7fbfac0a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005c40 tx=0x7fbfb8004370 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:58.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.689+0000 7fbfabfff640 1 -- 172.21.15.103:0/2135668340 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0012450 con 0x5628facec370 2024-09-19T06:53:58.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.689+0000 7fbfabfff640 1 -- 172.21.15.103:0/2135668340 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfb0012ad0 con 0x5628facec370 2024-09-19T06:53:58.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.689+0000 7fbfabfff640 1 -- 172.21.15.103:0/2135668340 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb000fbb0 con 0x5628facec370 2024-09-19T06:53:58.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.690+0000 7fbfc9e40740 1 -- 172.21.15.103:0/2135668340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628facec370 msgr2=0x5628facee760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:58.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.690+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/2135668340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628facec370 0x5628facee760 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009c60 tx=0x7fbfb00117b0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.690+0000 7fbfc9e40740 1 -- 172.21.15.103:0/2135668340 shutdown_connections 2024-09-19T06:53:58.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.690+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/2135668340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628facec370 0x5628facee760 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.690+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/2135668340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628face15d0 0x5628face1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.690+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/2135668340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face2b40 0x5628face1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.690+0000 7fbfc9e40740 1 -- 172.21.15.103:0/2135668340 >> 172.21.15.103:0/2135668340 conn(0x5628facaa310 msgr2=0x5628facddc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:58.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.690+0000 7fbfc9e40740 1 -- 172.21.15.103:0/2135668340 shutdown_connections 2024-09-19T06:53:58.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.690+0000 7fbfc9e40740 1 -- 172.21.15.103:0/2135668340 wait complete. 2024-09-19T06:53:58.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.690+0000 7fbfc9e40740 1 -- start start 2024-09-19T06:53:58.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.690+0000 7fbfc9e40740 1 Processor -- start 2024-09-19T06:53:58.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc9e40740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628face15d0 0x5628facebd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc9e40740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face2b40 0x5628face8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc9e40740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628facec370 0x5628face95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc9e40740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5628facf3480 con 0x5628face15d0 2024-09-19T06:53:58.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc9e40740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5628fac02060 con 0x5628face2b40 2024-09-19T06:53:58.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc9e40740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5628facf3600 con 0x5628facec370 2024-09-19T06:53:58.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc9e40740 10 client.0 _ll_register_callbacks cb 0x5628fa9558c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:58.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc5803640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628face15d0 0x5628facebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:58.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc5803640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628face15d0 0x5628facebd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54844/0 (socket says 172.21.15.103:54844) 2024-09-19T06:53:58.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc5803640 1 -- 172.21.15.103:0/1794809339 learned_addr learned my addr 172.21.15.103:0/1794809339 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:58.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc5002640 1 --2- 172.21.15.103:0/1794809339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face2b40 0x5628face8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:58.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc6004640 1 --2- 172.21.15.103:0/1794809339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628facec370 0x5628face95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:58.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc5803640 1 -- 172.21.15.103:0/1794809339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628facec370 msgr2=0x5628face95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:58.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc5803640 1 --2- 172.21.15.103:0/1794809339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628facec370 0x5628face95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc5803640 1 -- 172.21.15.103:0/1794809339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face2b40 msgr2=0x5628face8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:58.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc5803640 1 --2- 172.21.15.103:0/1794809339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face2b40 0x5628face8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc5803640 1 -- 172.21.15.103:0/1794809339 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628fa9b83c0 con 0x5628face15d0 2024-09-19T06:53:58.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc5002640 1 --2- 172.21.15.103:0/1794809339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face2b40 0x5628face8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:58.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfc5803640 1 --2- 172.21.15.103:0/1794809339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628face15d0 0x5628facebd30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0982c0 tx=0x7fbfbc09b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c23747e470e867 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:58.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.691+0000 7fbfa97fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:58.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.692+0000 7fbfa97fa640 1 -- 172.21.15.103:0/1794809339 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0a0210 con 0x5628face15d0 2024-09-19T06:53:58.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.692+0000 7fbfa97fa640 1 -- 172.21.15.103:0/1794809339 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fbfbc0984d0 con 0x5628face15d0 2024-09-19T06:53:58.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.692+0000 7fbfc9e40740 1 -- 172.21.15.103:0/1794809339 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb0012ad0 con 0x5628face15d0 2024-09-19T06:53:58.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.692+0000 7fbfa97fa640 1 -- 172.21.15.103:0/1794809339 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbfbc0a4ad0 con 0x5628face15d0 2024-09-19T06:53:58.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.692+0000 7fbfc9e40740 10 client.17208 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:58.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.692+0000 7fbfc9e40740 1 -- 172.21.15.103:0/1794809339 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5628facfef40 con 0x5628face15d0 2024-09-19T06:53:58.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.692+0000 7fbfa97fa640 1 -- 172.21.15.103:0/1794809339 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfb0012ad0 con 0x5628face15d0 2024-09-19T06:53:58.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.692+0000 7fbfa97fa640 1 -- 172.21.15.103:0/1794809339 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbfbc09b2b0 con 0x5628face15d0 2024-09-19T06:53:58.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.692+0000 7fbfc9e40740 10 client.17208 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:58.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.692+0000 7fbfc9e40740 10 client.17208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:58.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.692+0000 7fbfc9e40740 20 client.17208 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:58.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.692+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/1794809339 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5628fad00400 0x5628fad02870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.692+0000 7fbfc9e40740 4 client.17208 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:58.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.692+0000 7fbfc9e40740 1 -- 172.21.15.103:0/1794809339 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628fac54860 con 0x5628fad00400 2024-09-19T06:53:58.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.693+0000 7fbfc5002640 1 --2- 172.21.15.103:0/1794809339 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5628fad00400 0x5628fad02870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:58.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.693+0000 7fbfc5002640 1 --2- 172.21.15.103:0/1794809339 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5628fad00400 0x5628fad02870 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a46afe1146f2d9c1 server_cookie=64d39c8bee489423 in_seq=0 out_seq=0 2024-09-19T06:53:58.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.693+0000 7fbfa97fa640 10 client.17208 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:58.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.694+0000 7fbfa97fa640 1 -- 172.21.15.103:0/1794809339 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628fac54860 con 0x5628fad00400 2024-09-19T06:53:58.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.694+0000 7fbfa97fa640 10 client.17208 handle_command_reply: tid=0 2024-09-19T06:53:58.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.694+0000 7fbfc9e40740 1 client.17208 shutdown 2024-09-19T06:53:58.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.694+0000 7fbfc9e40740 1 -- 172.21.15.103:0/1794809339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628face15d0 msgr2=0x5628facebd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:58.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.694+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/1794809339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628face15d0 0x5628facebd30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0982c0 tx=0x7fbfbc09b7c0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.694+0000 7fbfc9e40740 1 -- 172.21.15.103:0/1794809339 shutdown_connections 2024-09-19T06:53:58.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.694+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/1794809339 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5628fad00400 0x5628fad02870 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.694+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/1794809339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628facec370 0x5628face95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.694+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/1794809339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face2b40 0x5628face8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.694+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/1794809339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628face15d0 0x5628facebd30 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.694+0000 7fbfc9e40740 1 -- 172.21.15.103:0/1794809339 >> 172.21.15.103:0/1794809339 conn(0x5628facaa310 msgr2=0x5628face3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:58.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.694+0000 7fbfa97fa640 0 client.17208 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:58.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.694+0000 7fbfc9e40740 1 -- 172.21.15.103:0/1794809339 shutdown_connections 2024-09-19T06:53:58.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.694+0000 7fbfc9e40740 1 -- 172.21.15.103:0/1794809339 wait complete. 2024-09-19T06:53:58.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.694+0000 7fbfc9e40740 20 client.17208 trim_cache size 0 max 16384 2024-09-19T06:53:58.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc9e40740 1 Processor -- start 2024-09-19T06:53:58.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc9e40740 1 -- start start 2024-09-19T06:53:58.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc9e40740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628face15d0 0x5628facfc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc9e40740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face2b40 0x5628faced760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc9e40740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628facec370 0x5628facedca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc9e40740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc01ab630 con 0x5628facec370 2024-09-19T06:53:58.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc9e40740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc01ad170 con 0x5628face2b40 2024-09-19T06:53:58.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc9e40740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc01abd00 con 0x5628face15d0 2024-09-19T06:53:58.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc6004640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628facec370 0x5628facedca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:58.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc6004640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628facec370 0x5628facedca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54850/0 (socket says 172.21.15.103:54850) 2024-09-19T06:53:58.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc6004640 1 -- 172.21.15.103:0/1255230745 learned_addr learned my addr 172.21.15.103:0/1255230745 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:58.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc5002640 1 --2- 172.21.15.103:0/1255230745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face2b40 0x5628faced760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:58.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc5803640 1 --2- 172.21.15.103:0/1255230745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628face15d0 0x5628facfc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:58.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc6004640 1 -- 172.21.15.103:0/1255230745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628face15d0 msgr2=0x5628facfc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:58.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc6004640 1 --2- 172.21.15.103:0/1255230745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628face15d0 0x5628facfc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc6004640 1 -- 172.21.15.103:0/1255230745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face2b40 msgr2=0x5628faced760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:58.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc6004640 1 --2- 172.21.15.103:0/1255230745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face2b40 0x5628faced760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.707+0000 7fbfc6004640 1 -- 172.21.15.103:0/1255230745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628facdd4f0 con 0x5628facec370 2024-09-19T06:53:58.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.708+0000 7fbfc6004640 1 --2- 172.21.15.103:0/1255230745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628facec370 0x5628facedca0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009900 tx=0x7fbfb8039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c91212785f5f02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:58.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.708+0000 7fbfab7fe640 1 -- 172.21.15.103:0/1255230745 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8035040 con 0x5628facec370 2024-09-19T06:53:58.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.708+0000 7fbfab7fe640 1 -- 172.21.15.103:0/1255230745 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfb8031c40 con 0x5628facec370 2024-09-19T06:53:58.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.708+0000 7fbfab7fe640 1 -- 172.21.15.103:0/1255230745 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb803d650 con 0x5628facec370 2024-09-19T06:53:58.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.708+0000 7fbfc9e40740 1 -- 172.21.15.103:0/1255230745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628facec370 msgr2=0x5628facedca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:58.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.708+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/1255230745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628facec370 0x5628facedca0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009900 tx=0x7fbfb8039000 comp rx=0 tx=0).stop 2024-09-19T06:53:58.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.708+0000 7fbfc9e40740 1 -- 172.21.15.103:0/1255230745 shutdown_connections 2024-09-19T06:53:58.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.708+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/1255230745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628facec370 0x5628facedca0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.708+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/1255230745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face2b40 0x5628faced760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.708+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/1255230745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628face15d0 0x5628facfc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.708+0000 7fbfc9e40740 1 -- 172.21.15.103:0/1255230745 >> 172.21.15.103:0/1255230745 conn(0x5628facaa310 msgr2=0x5628face78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:58.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.708+0000 7fbfc9e40740 1 -- 172.21.15.103:0/1255230745 shutdown_connections 2024-09-19T06:53:58.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.708+0000 7fbfc9e40740 1 -- 172.21.15.103:0/1255230745 wait complete. 2024-09-19T06:53:58.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.709+0000 7fbfc9e40740 1 -- start start 2024-09-19T06:53:58.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.709+0000 7fbfc9e40740 1 Processor -- start 2024-09-19T06:53:58.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.709+0000 7fbfc9e40740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face15d0 0x5628facf8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.709+0000 7fbfc9e40740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628face2b40 0x5628facf9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.709+0000 7fbfc9e40740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628facec370 0x5628fad07710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.709+0000 7fbfc9e40740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc01acaa0 con 0x5628face2b40 2024-09-19T06:53:58.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.709+0000 7fbfc9e40740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc01ac3d0 con 0x5628face15d0 2024-09-19T06:53:58.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.709+0000 7fbfc9e40740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc01a95f0 con 0x5628facec370 2024-09-19T06:53:58.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.709+0000 7fbfc9e40740 10 client.0 _ll_register_callbacks cb 0x7fbfbc005ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:58.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.709+0000 7fbfc5002640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628face2b40 0x5628facf9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:58.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.709+0000 7fbfc5002640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628face2b40 0x5628facf9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54852/0 (socket says 172.21.15.103:54852) 2024-09-19T06:53:58.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.709+0000 7fbfc5002640 1 -- 172.21.15.103:0/2037496781 learned_addr learned my addr 172.21.15.103:0/2037496781 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:53:58.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.710+0000 7fbfc5002640 1 -- 172.21.15.103:0/2037496781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628facec370 msgr2=0x5628fad07710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:58.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.710+0000 7fbfc5002640 1 --2- 172.21.15.103:0/2037496781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628facec370 0x5628fad07710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.710+0000 7fbfc5002640 1 -- 172.21.15.103:0/2037496781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face15d0 msgr2=0x5628facf8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:58.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.710+0000 7fbfc5002640 1 --2- 172.21.15.103:0/2037496781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face15d0 0x5628facf8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.710+0000 7fbfc5002640 1 -- 172.21.15.103:0/2037496781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628fad07f40 con 0x5628face2b40 2024-09-19T06:53:58.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.710+0000 7fbfc5002640 1 --2- 172.21.15.103:0/2037496781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628face2b40 0x5628facf9070 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009c60 tx=0x7fbfb0009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8459d02a37b5a33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:58.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.710+0000 7fbfa97fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:53:58.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.710+0000 7fbfa97fa640 1 -- 172.21.15.103:0/2037496781 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0012650 con 0x5628face2b40 2024-09-19T06:53:58.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.710+0000 7fbfa97fa640 1 -- 172.21.15.103:0/2037496781 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fbfb00127f0 con 0x5628face2b40 2024-09-19T06:53:58.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.710+0000 7fbfc9e40740 1 -- 172.21.15.103:0/2037496781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb8031c40 con 0x5628face2b40 2024-09-19T06:53:58.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.710+0000 7fbfa97fa640 1 -- 172.21.15.103:0/2037496781 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbfb003f050 con 0x5628face2b40 2024-09-19T06:53:58.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.710+0000 7fbfc9e40740 10 client.17214 fetch_fsmap learned FSMap version 27 2024-09-19T06:53:58.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.710+0000 7fbfc9e40740 1 -- 172.21.15.103:0/2037496781 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5628facf83d0 con 0x5628face2b40 2024-09-19T06:53:58.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.710+0000 7fbfa97fa640 1 -- 172.21.15.103:0/2037496781 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfb8031c40 con 0x5628face2b40 2024-09-19T06:53:58.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.710+0000 7fbfa97fa640 1 -- 172.21.15.103:0/2037496781 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbfb000f990 con 0x5628face2b40 2024-09-19T06:53:58.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.711+0000 7fbfc9e40740 10 client.17214 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:53:58.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.711+0000 7fbfc9e40740 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:58.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.711+0000 7fbfc9e40740 20 client.17214 populate_metadata read hostname 'smithi103' 2024-09-19T06:53:58.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.711+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/2037496781 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5628fad6b180 0x5628fad6d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:58.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.711+0000 7fbfc9e40740 4 client.17214 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:53:58.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.711+0000 7fbfc9e40740 1 -- 172.21.15.103:0/2037496781 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628fac4ed70 con 0x5628fad6b180 2024-09-19T06:53:58.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.711+0000 7fbfc5803640 1 --2- 172.21.15.103:0/2037496781 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5628fad6b180 0x5628fad6d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:58.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.711+0000 7fbfa97fa640 10 client.17214 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:58.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.711+0000 7fbfc5803640 1 --2- 172.21.15.103:0/2037496781 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5628fad6b180 0x5628fad6d590 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4df78a343548ec4 server_cookie=ca9c56e091a95eb6 in_seq=0 out_seq=0 2024-09-19T06:53:58.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfa97fa640 1 -- 172.21.15.103:0/2037496781 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628fac4ed70 con 0x5628fad6b180 2024-09-19T06:53:58.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfa97fa640 10 client.17214 handle_command_reply: tid=0 2024-09-19T06:53:58.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfc9e40740 1 client.17214 shutdown 2024-09-19T06:53:58.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfc9e40740 1 -- 172.21.15.103:0/2037496781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628face2b40 msgr2=0x5628facf9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:58.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/2037496781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628face2b40 0x5628facf9070 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009c60 tx=0x7fbfb0009e50 comp rx=0 tx=0).stop 2024-09-19T06:53:58.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfc9e40740 1 -- 172.21.15.103:0/2037496781 shutdown_connections 2024-09-19T06:53:58.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/2037496781 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5628fad6b180 0x5628fad6d590 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/2037496781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628facec370 0x5628fad07710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/2037496781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628face2b40 0x5628facf9070 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfa97fa640 0 client.17214 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:53:58.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfc9e40740 1 --2- 172.21.15.103:0/2037496781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628face15d0 0x5628facf8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfc9e40740 1 -- 172.21.15.103:0/2037496781 >> 172.21.15.103:0/2037496781 conn(0x5628facaa310 msgr2=0x5628facdc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:58.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfc9e40740 1 -- 172.21.15.103:0/2037496781 shutdown_connections 2024-09-19T06:53:58.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfc9e40740 1 -- 172.21.15.103:0/2037496781 wait complete. 2024-09-19T06:53:58.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfc9e40740 20 client.17214 trim_cache size 0 max 16384 2024-09-19T06:53:58.740 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:53:58.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfc7a67640 1 -- 172.21.15.103:0/107829505 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbfac0a33b0 msgr2=0x7fbfac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:58.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfc7a67640 1 --2- 172.21.15.103:0/107829505 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbfac0a33b0 0x7fbfac0a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005c40 tx=0x7fbfb8004370 comp rx=0 tx=0).stop 2024-09-19T06:53:58.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfc7a67640 1 -- 172.21.15.103:0/107829505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfc0101df0 msgr2=0x7fbfc01983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:58.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.712+0000 7fbfc7a67640 1 --2- 172.21.15.103:0/107829505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfc0101df0 0x7fbfc01983f0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00db80 tx=0x7fbfbc00b630 comp rx=0 tx=0).stop 2024-09-19T06:53:58.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.713+0000 7fbfc7a67640 1 -- 172.21.15.103:0/107829505 shutdown_connections 2024-09-19T06:53:58.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.713+0000 7fbfc7a67640 1 --2- 172.21.15.103:0/107829505 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbfac0a33b0 0x7fbfac0a5870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.713+0000 7fbfc7a67640 1 --2- 172.21.15.103:0/107829505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc0106f20 0x7fbfc019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.713+0000 7fbfc7a67640 1 --2- 172.21.15.103:0/107829505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc01027a0 0x7fbfc0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.713+0000 7fbfc7a67640 1 --2- 172.21.15.103:0/107829505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfc0101df0 0x7fbfc01983f0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:58.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.713+0000 7fbfc7a67640 1 -- 172.21.15.103:0/107829505 >> 172.21.15.103:0/107829505 conn(0x7fbfc00fd8f0 msgr2=0x7fbfc00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:58.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.713+0000 7fbfc7a67640 1 -- 172.21.15.103:0/107829505 shutdown_connections 2024-09-19T06:53:58.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:53:58.713+0000 7fbfc7a67640 1 -- 172.21.15.103:0/107829505 wait complete. 2024-09-19T06:54:00.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:00 smithi174 ceph-mon[35637]: pgmap v146: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.5 KiB/s wr, 16 op/s 2024-09-19T06:54:00.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:00 smithi103 ceph-mon[31158]: pgmap v146: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.5 KiB/s wr, 16 op/s 2024-09-19T06:54:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:00 smithi134 ceph-mon[35356]: pgmap v146: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.5 KiB/s wr, 16 op/s 2024-09-19T06:54:00.458 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:54:00.459 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:54:00.713 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:01.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.154+0000 7f5dfd2da640 1 -- 172.21.15.103:0/3662861573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5df81082a0 msgr2=0x7f5df810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.154+0000 7f5dfd2da640 1 --2- 172.21.15.103:0/3662861573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5df81082a0 0x7f5df810ec40 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009cd0 tx=0x7f5df002a910 comp rx=0 tx=0).stop 2024-09-19T06:54:01.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.154+0000 7f5dfd2da640 1 -- 172.21.15.103:0/3662861573 shutdown_connections 2024-09-19T06:54:01.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.154+0000 7f5dfd2da640 1 --2- 172.21.15.103:0/3662861573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5df81082a0 0x7f5df810ec40 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.154+0000 7f5dfd2da640 1 --2- 172.21.15.103:0/3662861573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5df81036e0 0x7f5df8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.154+0000 7f5dfd2da640 1 --2- 172.21.15.103:0/3662861573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df8102d30 0x7f5df8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.154+0000 7f5dfd2da640 1 -- 172.21.15.103:0/3662861573 >> 172.21.15.103:0/3662861573 conn(0x7f5df80fc7d0 msgr2=0x7f5df80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:01.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.154+0000 7f5dfd2da640 1 -- 172.21.15.103:0/3662861573 shutdown_connections 2024-09-19T06:54:01.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.154+0000 7f5dfd2da640 1 -- 172.21.15.103:0/3662861573 wait complete. 2024-09-19T06:54:01.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.155+0000 7f5dfd2da640 1 Processor -- start 2024-09-19T06:54:01.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.155+0000 7f5dfd2da640 1 -- start start 2024-09-19T06:54:01.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.155+0000 7f5dfd2da640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5df8102d30 0x7f5df81986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.155+0000 7f5dfd2da640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5df81036e0 0x7f5df8198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.155+0000 7f5dfd2da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df81082a0 0x7f5df819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.155+0000 7f5dfd2da640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5df8112160 con 0x7f5df8102d30 2024-09-19T06:54:01.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.155+0000 7f5dfd2da640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5df8111fe0 con 0x7f5df81082a0 2024-09-19T06:54:01.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.155+0000 7f5dfd2da640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5df81122e0 con 0x7f5df81036e0 2024-09-19T06:54:01.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.155+0000 7f5df6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5df8102d30 0x7f5df81986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.155+0000 7f5df6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5df8102d30 0x7f5df81986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54858/0 (socket says 172.21.15.103:54858) 2024-09-19T06:54:01.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.155+0000 7f5df6ffd640 1 -- 172.21.15.103:0/107238 learned_addr learned my addr 172.21.15.103:0/107238 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:01.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.155+0000 7f5df77fe640 1 --2- 172.21.15.103:0/107238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df81082a0 0x7f5df819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.155+0000 7f5deffff640 1 --2- 172.21.15.103:0/107238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5df81036e0 0x7f5df8198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.156+0000 7f5df6ffd640 1 -- 172.21.15.103:0/107238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5df81036e0 msgr2=0x7f5df8198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.156+0000 7f5df6ffd640 1 --2- 172.21.15.103:0/107238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5df81036e0 0x7f5df8198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.156+0000 7f5df6ffd640 1 -- 172.21.15.103:0/107238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df81082a0 msgr2=0x7f5df819cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.156+0000 7f5df6ffd640 1 --2- 172.21.15.103:0/107238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df81082a0 0x7f5df819cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.156+0000 7f5df6ffd640 1 -- 172.21.15.103:0/107238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df819d670 con 0x7f5df8102d30 2024-09-19T06:54:01.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.156+0000 7f5df77fe640 1 --2- 172.21.15.103:0/107238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df81082a0 0x7f5df819cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:01.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.156+0000 7f5df6ffd640 1 --2- 172.21.15.103:0/107238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5df8102d30 0x7f5df81986c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f5de800b4a0 tx=0x7f5de800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:01.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.156+0000 7f5df4ff9640 1 -- 172.21.15.103:0/107238 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8015020 con 0x7f5df8102d30 2024-09-19T06:54:01.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.156+0000 7f5dfd2da640 1 -- 172.21.15.103:0/107238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df819d960 con 0x7f5df8102d30 2024-09-19T06:54:01.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.156+0000 7f5df4ff9640 1 -- 172.21.15.103:0/107238 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de8011d10 con 0x7f5df8102d30 2024-09-19T06:54:01.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.156+0000 7f5df4ff9640 1 -- 172.21.15.103:0/107238 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de800f2e0 con 0x7f5df8102d30 2024-09-19T06:54:01.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.156+0000 7f5dfd2da640 1 -- 172.21.15.103:0/107238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df81a5240 con 0x7f5df8102d30 2024-09-19T06:54:01.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.158+0000 7f5df4ff9640 1 -- 172.21.15.103:0/107238 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f5de8018020 con 0x7f5df8102d30 2024-09-19T06:54:01.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.159+0000 7f5dff6b3740 1 Processor -- start 2024-09-19T06:54:01.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.159+0000 7f5dff6b3740 1 -- start start 2024-09-19T06:54:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.159+0000 7f5dff6b3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.159+0000 7f5dff6b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24af480 0x55ded24b9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5dff6b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24b9d40 0x55ded24bc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5dff6b3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ded2414280 con 0x55ded24aeb60 2024-09-19T06:54:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5dff6b3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ded23bbb10 con 0x55ded24b9d40 2024-09-19T06:54:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5dff6b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ded2410240 con 0x55ded24af480 2024-09-19T06:54:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5df4ff9640 1 --2- 172.21.15.103:0/107238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5de40a33b0 0x7f5de40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5df77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5df77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54874/0 (socket says 172.21.15.103:54874) 2024-09-19T06:54:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5df77fe640 1 -- 172.21.15.103:0/1751917125 learned_addr learned my addr 172.21.15.103:0/1751917125 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5df4ff9640 1 -- 172.21.15.103:0/107238 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5de80cf490 con 0x7f5df8102d30 2024-09-19T06:54:01.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5deffff640 1 --2- 172.21.15.103:0/1751917125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24b9d40 0x55ded24bc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5df6ffd640 1 --2- 172.21.15.103:0/1751917125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24af480 0x55ded24b9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5df77fe640 1 --2- 172.21.15.103:0/107238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5de40a33b0 0x7f5de40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5df77fe640 1 -- 172.21.15.103:0/1751917125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24af480 msgr2=0x55ded24b9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5df77fe640 1 --2- 172.21.15.103:0/1751917125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24af480 0x55ded24b9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5df77fe640 1 -- 172.21.15.103:0/1751917125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24b9d40 msgr2=0x55ded24bc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5df77fe640 1 --2- 172.21.15.103:0/1751917125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24b9d40 0x55ded24bc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5df77fe640 1 -- 172.21.15.103:0/1751917125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ded23c81f0 con 0x55ded24aeb60 2024-09-19T06:54:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.160+0000 7f5df77fe640 1 --2- 172.21.15.103:0/1751917125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24aef40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5df0035000 tx=0x7f5df0003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37491fc3e66536aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.161+0000 7f5ded7fa640 1 -- 172.21.15.103:0/1751917125 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df0031830 con 0x55ded24aeb60 2024-09-19T06:54:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.161+0000 7f5ded7fa640 1 -- 172.21.15.103:0/1751917125 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df0036040 con 0x55ded24aeb60 2024-09-19T06:54:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.161+0000 7f5ded7fa640 1 -- 172.21.15.103:0/1751917125 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df002f870 con 0x55ded24aeb60 2024-09-19T06:54:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.161+0000 7f5dff6b3740 1 -- 172.21.15.103:0/1751917125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 msgr2=0x55ded24aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.161+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/1751917125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24aef40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5df0035000 tx=0x7f5df0003bb0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.161+0000 7f5df77fe640 1 --2- 172.21.15.103:0/107238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5de40a33b0 0x7f5de40a5870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5df004e000 tx=0x7f5df004b000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:01.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5dff6b3740 1 -- 172.21.15.103:0/1751917125 shutdown_connections 2024-09-19T06:54:01.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/1751917125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24b9d40 0x55ded24bc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/1751917125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24af480 0x55ded24b9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/1751917125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24aef40 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5dff6b3740 1 -- 172.21.15.103:0/1751917125 >> 172.21.15.103:0/1751917125 conn(0x55ded24ab0f0 msgr2=0x55ded24ab4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:01.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5dff6b3740 1 -- 172.21.15.103:0/1751917125 shutdown_connections 2024-09-19T06:54:01.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5dff6b3740 1 -- 172.21.15.103:0/1751917125 wait complete. 2024-09-19T06:54:01.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5dff6b3740 1 -- start start 2024-09-19T06:54:01.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5dff6b3740 1 Processor -- start 2024-09-19T06:54:01.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5dff6b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24aeb60 0x55ded24b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5dff6b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24af480 0x55ded24b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5dff6b3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24b9d40 0x55ded24b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5dff6b3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ded24bc910 con 0x55ded24b9d40 2024-09-19T06:54:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5dff6b3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ded23cef10 con 0x55ded24aeb60 2024-09-19T06:54:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5dff6b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ded24bca90 con 0x55ded24af480 2024-09-19T06:54:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5dff6b3740 10 client.0 _ll_register_callbacks cb 0x55ded2397a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.162+0000 7f5df77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24b9d40 0x55ded24b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5df77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24b9d40 0x55ded24b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54876/0 (socket says 172.21.15.103:54876) 2024-09-19T06:54:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5df77fe640 1 -- 172.21.15.103:0/2445056340 learned_addr learned my addr 172.21.15.103:0/2445056340 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5df6ffd640 1 --2- 172.21.15.103:0/2445056340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24aeb60 0x55ded24b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5deffff640 1 --2- 172.21.15.103:0/2445056340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24af480 0x55ded24b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5df77fe640 1 -- 172.21.15.103:0/2445056340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24af480 msgr2=0x55ded24b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5df77fe640 1 --2- 172.21.15.103:0/2445056340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24af480 0x55ded24b6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5df77fe640 1 -- 172.21.15.103:0/2445056340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24aeb60 msgr2=0x55ded24b97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5df77fe640 1 --2- 172.21.15.103:0/2445056340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24aeb60 0x55ded24b97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5df77fe640 1 -- 172.21.15.103:0/2445056340 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ded21856a0 con 0x55ded24b9d40 2024-09-19T06:54:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5deffff640 1 --2- 172.21.15.103:0/2445056340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24af480 0x55ded24b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5df77fe640 1 --2- 172.21.15.103:0/2445056340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24b9d40 0x55ded24b72d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5df0005f30 tx=0x7f5df0031ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce788c335ebad71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5dde7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5dde7fc640 1 -- 172.21.15.103:0/2445056340 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df0060060 con 0x55ded24b9d40 2024-09-19T06:54:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5dde7fc640 1 -- 172.21.15.103:0/2445056340 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5df002fb10 con 0x55ded24b9d40 2024-09-19T06:54:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5dff6b3740 1 -- 172.21.15.103:0/2445056340 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5df0036040 con 0x55ded24b9d40 2024-09-19T06:54:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5dde7fc640 1 -- 172.21.15.103:0/2445056340 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5df002f880 con 0x55ded24b9d40 2024-09-19T06:54:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.163+0000 7f5dff6b3740 10 client.17226 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.164+0000 7f5dff6b3740 1 -- 172.21.15.103:0/2445056340 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ded24bc910 con 0x55ded24b9d40 2024-09-19T06:54:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.164+0000 7f5dde7fc640 1 -- 172.21.15.103:0/2445056340 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df0036040 con 0x55ded24b9d40 2024-09-19T06:54:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.164+0000 7f5dde7fc640 1 -- 172.21.15.103:0/2445056340 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5df00305d0 con 0x55ded24b9d40 2024-09-19T06:54:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.164+0000 7f5dff6b3740 10 client.17226 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.164+0000 7f5dff6b3740 10 client.17226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.164+0000 7f5dff6b3740 20 client.17226 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.164+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/2445056340 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ded24c95c0 0x55ded24cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.164+0000 7f5dff6b3740 4 client.17226 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:01.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.164+0000 7f5dff6b3740 1 -- 172.21.15.103:0/2445056340 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ded24217f0 con 0x55ded24c95c0 2024-09-19T06:54:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.164+0000 7f5df6ffd640 1 --2- 172.21.15.103:0/2445056340 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ded24c95c0 0x55ded24cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.165+0000 7f5dde7fc640 10 client.17226 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.165+0000 7f5df6ffd640 1 --2- 172.21.15.103:0/2445056340 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ded24c95c0 0x55ded24cb9b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=986f3c50fc074a51 server_cookie=a9616f83523cffcc in_seq=0 out_seq=0 2024-09-19T06:54:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.165+0000 7f5dde7fc640 1 -- 172.21.15.103:0/2445056340 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ded24217f0 con 0x55ded24c95c0 2024-09-19T06:54:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.165+0000 7f5dde7fc640 10 client.17226 handle_command_reply: tid=0 2024-09-19T06:54:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.166+0000 7f5dff6b3740 1 client.17226 shutdown 2024-09-19T06:54:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.166+0000 7f5dff6b3740 1 -- 172.21.15.103:0/2445056340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24b9d40 msgr2=0x55ded24b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.166+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/2445056340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24b9d40 0x55ded24b72d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5df0005f30 tx=0x7f5df0031ae0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.166+0000 7f5dff6b3740 1 -- 172.21.15.103:0/2445056340 shutdown_connections 2024-09-19T06:54:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.166+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/2445056340 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ded24c95c0 0x55ded24cb9b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.166+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/2445056340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24b9d40 0x55ded24b72d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.166+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/2445056340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24af480 0x55ded24b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.166+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/2445056340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24aeb60 0x55ded24b97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.166+0000 7f5dff6b3740 1 -- 172.21.15.103:0/2445056340 >> 172.21.15.103:0/2445056340 conn(0x55ded24ab0f0 msgr2=0x55ded24aa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:01.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.166+0000 7f5dde7fc640 0 client.17226 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:01.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.166+0000 7f5dff6b3740 1 -- 172.21.15.103:0/2445056340 shutdown_connections 2024-09-19T06:54:01.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.166+0000 7f5dff6b3740 1 -- 172.21.15.103:0/2445056340 wait complete. 2024-09-19T06:54:01.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.166+0000 7f5dff6b3740 20 client.17226 trim_cache size 0 max 16384 2024-09-19T06:54:01.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.183+0000 7f5dff6b3740 1 Processor -- start 2024-09-19T06:54:01.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.183+0000 7f5dff6b3740 1 -- start start 2024-09-19T06:54:01.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.183+0000 7f5dff6b3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.183+0000 7f5dff6b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24af480 0x55ded24b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.183+0000 7f5dff6b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24b9d40 0x55ded24d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.183+0000 7f5dff6b3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5df81afc10 con 0x55ded24aeb60 2024-09-19T06:54:01.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.183+0000 7f5dff6b3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5df81b1750 con 0x55ded24af480 2024-09-19T06:54:01.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.183+0000 7f5dff6b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5df81b02e0 con 0x55ded24b9d40 2024-09-19T06:54:01.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.184+0000 7f5df6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.184+0000 7f5df6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54880/0 (socket says 172.21.15.103:54880) 2024-09-19T06:54:01.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.184+0000 7f5df6ffd640 1 -- 172.21.15.103:0/1433767339 learned_addr learned my addr 172.21.15.103:0/1433767339 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:01.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.184+0000 7f5df77fe640 1 --2- 172.21.15.103:0/1433767339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24b9d40 0x55ded24d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.184+0000 7f5deffff640 1 --2- 172.21.15.103:0/1433767339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24af480 0x55ded24b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.184+0000 7f5df6ffd640 1 -- 172.21.15.103:0/1433767339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24b9d40 msgr2=0x55ded24d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:01.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.184+0000 7f5df6ffd640 1 --2- 172.21.15.103:0/1433767339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24b9d40 0x55ded24d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.184+0000 7f5df6ffd640 1 -- 172.21.15.103:0/1433767339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24af480 msgr2=0x55ded24b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:01.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.184+0000 7f5df6ffd640 1 --2- 172.21.15.103:0/1433767339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24af480 0x55ded24b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.184+0000 7f5df6ffd640 1 -- 172.21.15.103:0/1433767339 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ded24af2b0 con 0x55ded24aeb60 2024-09-19T06:54:01.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.184+0000 7f5df6ffd640 1 --2- 172.21.15.103:0/1433767339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24b15c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f5de80ccd20 tx=0x7f5de80c6620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298ed3c799a8fc18 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:01.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.184+0000 7f5decff9640 1 -- 172.21.15.103:0/1433767339 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de809b850 con 0x55ded24aeb60 2024-09-19T06:54:01.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.184+0000 7f5decff9640 1 -- 172.21.15.103:0/1433767339 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de809fc00 con 0x55ded24aeb60 2024-09-19T06:54:01.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.184+0000 7f5decff9640 1 -- 172.21.15.103:0/1433767339 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de80cce30 con 0x55ded24aeb60 2024-09-19T06:54:01.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.185+0000 7f5dff6b3740 1 -- 172.21.15.103:0/1433767339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 msgr2=0x55ded24b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.185+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/1433767339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24b15c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f5de80ccd20 tx=0x7f5de80c6620 comp rx=0 tx=0).stop 2024-09-19T06:54:01.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.185+0000 7f5dff6b3740 1 -- 172.21.15.103:0/1433767339 shutdown_connections 2024-09-19T06:54:01.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.185+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/1433767339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24b9d40 0x55ded24d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.185+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/1433767339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24af480 0x55ded24b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.185+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/1433767339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24b15c0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.185+0000 7f5dff6b3740 1 -- 172.21.15.103:0/1433767339 >> 172.21.15.103:0/1433767339 conn(0x55ded24ab0f0 msgr2=0x55ded24b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:01.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.185+0000 7f5dff6b3740 1 -- 172.21.15.103:0/1433767339 shutdown_connections 2024-09-19T06:54:01.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.185+0000 7f5dff6b3740 1 -- 172.21.15.103:0/1433767339 wait complete. 2024-09-19T06:54:01.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.186+0000 7f5dff6b3740 1 -- start start 2024-09-19T06:54:01.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.186+0000 7f5dff6b3740 1 Processor -- start 2024-09-19T06:54:01.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.186+0000 7f5dff6b3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.186+0000 7f5dff6b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24af480 0x55ded24bd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.186+0000 7f5dff6b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24b9d40 0x55ded24bda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.186+0000 7f5dff6b3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5df81b1080 con 0x55ded24aeb60 2024-09-19T06:54:01.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.186+0000 7f5dff6b3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5df81b09b0 con 0x55ded24af480 2024-09-19T06:54:01.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.186+0000 7f5dff6b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5df81adbd0 con 0x55ded24b9d40 2024-09-19T06:54:01.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.186+0000 7f5dff6b3740 10 client.0 _ll_register_callbacks cb 0x7f5df0003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:01.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.186+0000 7f5df6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.186+0000 7f5df6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24c21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54894/0 (socket says 172.21.15.103:54894) 2024-09-19T06:54:01.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.186+0000 7f5df6ffd640 1 -- 172.21.15.103:0/3921631317 learned_addr learned my addr 172.21.15.103:0/3921631317 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:01.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.186+0000 7f5deffff640 1 --2- 172.21.15.103:0/3921631317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24af480 0x55ded24bd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.186+0000 7f5df77fe640 1 --2- 172.21.15.103:0/3921631317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24b9d40 0x55ded24bda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.186+0000 7f5df6ffd640 1 -- 172.21.15.103:0/3921631317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24b9d40 msgr2=0x55ded24bda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:01.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5df6ffd640 1 --2- 172.21.15.103:0/3921631317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24b9d40 0x55ded24bda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5df6ffd640 1 -- 172.21.15.103:0/3921631317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24af480 msgr2=0x55ded24bd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:01.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5df6ffd640 1 --2- 172.21.15.103:0/3921631317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24af480 0x55ded24bd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5df6ffd640 1 -- 172.21.15.103:0/3921631317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ded24bdff0 con 0x55ded24aeb60 2024-09-19T06:54:01.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5deffff640 1 --2- 172.21.15.103:0/3921631317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24af480 0x55ded24bd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:01.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5df6ffd640 1 --2- 172.21.15.103:0/3921631317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24c21c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f5de80a3f90 tx=0x7f5de8010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45baa9afaccb295a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:01.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5dde7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:01.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5dde7fc640 1 -- 172.21.15.103:0/3921631317 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de80b9060 con 0x55ded24aeb60 2024-09-19T06:54:01.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5dde7fc640 1 -- 172.21.15.103:0/3921631317 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5de8010510 con 0x55ded24aeb60 2024-09-19T06:54:01.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5dff6b3740 1 -- 172.21.15.103:0/3921631317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de809fc00 con 0x55ded24aeb60 2024-09-19T06:54:01.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5dde7fc640 1 -- 172.21.15.103:0/3921631317 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5de80ba250 con 0x55ded24aeb60 2024-09-19T06:54:01.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5dff6b3740 10 client.17232 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:01.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5dff6b3740 1 -- 172.21.15.103:0/3921631317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ded2533f30 con 0x55ded24aeb60 2024-09-19T06:54:01.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5dde7fc640 1 -- 172.21.15.103:0/3921631317 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de809fc00 con 0x55ded24aeb60 2024-09-19T06:54:01.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5dde7fc640 1 -- 172.21.15.103:0/3921631317 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5de8084b50 con 0x55ded24aeb60 2024-09-19T06:54:01.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5dff6b3740 10 client.17232 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:01.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5dff6b3740 10 client.17232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:01.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.187+0000 7f5dff6b3740 20 client.17232 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:01.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.188+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/3921631317 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ded25345e0 0x55ded2536a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.188+0000 7f5dff6b3740 4 client.17232 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:54:01.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.188+0000 7f5dff6b3740 1 -- 172.21.15.103:0/3921631317 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ded241bd00 con 0x55ded25345e0 2024-09-19T06:54:01.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.188+0000 7f5deffff640 1 --2- 172.21.15.103:0/3921631317 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ded25345e0 0x55ded2536a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.188+0000 7f5dde7fc640 10 client.17232 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:01.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.188+0000 7f5deffff640 1 --2- 172.21.15.103:0/3921631317 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ded25345e0 0x55ded2536a00 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20f684f42d4befa4 server_cookie=24035f1ba2a538b0 in_seq=0 out_seq=0 2024-09-19T06:54:01.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.190+0000 7f5dde7fc640 1 -- 172.21.15.103:0/3921631317 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ded241bd00 con 0x55ded25345e0 2024-09-19T06:54:01.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.190+0000 7f5dde7fc640 10 client.17232 handle_command_reply: tid=0 2024-09-19T06:54:01.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.190+0000 7f5dff6b3740 1 client.17232 shutdown 2024-09-19T06:54:01.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.190+0000 7f5dff6b3740 1 -- 172.21.15.103:0/3921631317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 msgr2=0x55ded24c21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.190+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/3921631317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24c21c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f5de80a3f90 tx=0x7f5de8010910 comp rx=0 tx=0).stop 2024-09-19T06:54:01.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.190+0000 7f5dff6b3740 1 -- 172.21.15.103:0/3921631317 shutdown_connections 2024-09-19T06:54:01.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.190+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/3921631317 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ded25345e0 0x55ded2536a00 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.190+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/3921631317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ded24b9d40 0x55ded24bda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.190+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/3921631317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ded24af480 0x55ded24bd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.190+0000 7f5dff6b3740 1 --2- 172.21.15.103:0/3921631317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ded24aeb60 0x55ded24c21c0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.190+0000 7f5dff6b3740 1 -- 172.21.15.103:0/3921631317 >> 172.21.15.103:0/3921631317 conn(0x55ded24ab0f0 msgr2=0x55ded24b5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:01.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.190+0000 7f5dde7fc640 0 client.17232 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:01.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.190+0000 7f5dff6b3740 1 -- 172.21.15.103:0/3921631317 shutdown_connections 2024-09-19T06:54:01.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.190+0000 7f5dff6b3740 1 -- 172.21.15.103:0/3921631317 wait complete. 2024-09-19T06:54:01.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.190+0000 7f5dff6b3740 20 client.17232 trim_cache size 0 max 16384 2024-09-19T06:54:01.196 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:54:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T06:54:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "c340cefb-a68f-48bb-9676-a700d991504b", 2024-09-19T06:54:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T06:54:01.196 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:54:01.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.191+0000 7f5dfd2da640 1 -- 172.21.15.103:0/107238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5de40a33b0 msgr2=0x7f5de40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.191+0000 7f5dfd2da640 1 --2- 172.21.15.103:0/107238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5de40a33b0 0x7f5de40a5870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5df004e000 tx=0x7f5df004b000 comp rx=0 tx=0).stop 2024-09-19T06:54:01.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.191+0000 7f5dfd2da640 1 -- 172.21.15.103:0/107238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5df8102d30 msgr2=0x7f5df81986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.191+0000 7f5dfd2da640 1 --2- 172.21.15.103:0/107238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5df8102d30 0x7f5df81986c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f5de800b4a0 tx=0x7f5de800b8a0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.191+0000 7f5dfd2da640 1 -- 172.21.15.103:0/107238 shutdown_connections 2024-09-19T06:54:01.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.191+0000 7f5dfd2da640 1 --2- 172.21.15.103:0/107238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5de40a33b0 0x7f5de40a5870 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.191+0000 7f5dfd2da640 1 --2- 172.21.15.103:0/107238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df81082a0 0x7f5df819cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.191+0000 7f5dfd2da640 1 --2- 172.21.15.103:0/107238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5df81036e0 0x7f5df8198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.191+0000 7f5dfd2da640 1 --2- 172.21.15.103:0/107238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5df8102d30 0x7f5df81986c0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.191+0000 7f5dfd2da640 1 -- 172.21.15.103:0/107238 >> 172.21.15.103:0/107238 conn(0x7f5df80fc7d0 msgr2=0x7f5df8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:01.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.191+0000 7f5dfd2da640 1 -- 172.21.15.103:0/107238 shutdown_connections 2024-09-19T06:54:01.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:01.191+0000 7f5dfd2da640 1 -- 172.21.15.103:0/107238 wait complete. 2024-09-19T06:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.229+0000 7f6d4256d640 1 -- 172.21.15.103:0/973681101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c101370 msgr2=0x7f6d3c10d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.229+0000 7f6d4256d640 1 --2- 172.21.15.103:0/973681101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c101370 0x7f6d3c10d8a0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009cd0 tx=0x7f6d3402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.229+0000 7f6d4256d640 1 -- 172.21.15.103:0/973681101 shutdown_connections 2024-09-19T06:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.229+0000 7f6d4256d640 1 --2- 172.21.15.103:0/973681101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c101370 0x7f6d3c10d8a0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.229+0000 7f6d4256d640 1 --2- 172.21.15.103:0/973681101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d3c1009d0 0x7f6d3c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.229+0000 7f6d4256d640 1 --2- 172.21.15.103:0/973681101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c1069e0 0x7f6d3c106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.229+0000 7f6d4256d640 1 -- 172.21.15.103:0/973681101 >> 172.21.15.103:0/973681101 conn(0x7f6d3c0fc6b0 msgr2=0x7f6d3c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.229+0000 7f6d4256d640 1 -- 172.21.15.103:0/973681101 shutdown_connections 2024-09-19T06:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.230+0000 7f6d4256d640 1 -- 172.21.15.103:0/973681101 wait complete. 2024-09-19T06:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.230+0000 7f6d4256d640 1 Processor -- start 2024-09-19T06:54:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.230+0000 7f6d4256d640 1 -- start start 2024-09-19T06:54:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.230+0000 7f6d4256d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c1009d0 0x7f6d3c1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.230+0000 7f6d4256d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d3c101370 0x7f6d3c199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.230+0000 7f6d4256d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c1069e0 0x7f6d3c1a4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.230+0000 7f6d4256d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c110e20 con 0x7f6d3c1069e0 2024-09-19T06:54:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.230+0000 7f6d4256d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c110ca0 con 0x7f6d3c1009d0 2024-09-19T06:54:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.230+0000 7f6d4256d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c110fa0 con 0x7f6d3c101370 2024-09-19T06:54:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.231+0000 7f6d40b0a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c1069e0 0x7f6d3c1a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.231+0000 7f6d40b0a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c1069e0 0x7f6d3c1a4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54908/0 (socket says 172.21.15.103:54908) 2024-09-19T06:54:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.231+0000 7f6d40b0a640 1 -- 172.21.15.103:0/3226997397 learned_addr learned my addr 172.21.15.103:0/3226997397 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.231+0000 7f6d3b7fe640 1 --2- 172.21.15.103:0/3226997397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d3c101370 0x7f6d3c199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.231+0000 7f6d3bfff640 1 --2- 172.21.15.103:0/3226997397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c1009d0 0x7f6d3c1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.231+0000 7f6d40b0a640 1 -- 172.21.15.103:0/3226997397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d3c101370 msgr2=0x7f6d3c199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.231+0000 7f6d40b0a640 1 --2- 172.21.15.103:0/3226997397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d3c101370 0x7f6d3c199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.231+0000 7f6d40b0a640 1 -- 172.21.15.103:0/3226997397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c1009d0 msgr2=0x7f6d3c1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.231+0000 7f6d40b0a640 1 --2- 172.21.15.103:0/3226997397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c1009d0 0x7f6d3c1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.231+0000 7f6d40b0a640 1 -- 172.21.15.103:0/3226997397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c1a46b0 con 0x7f6d3c1069e0 2024-09-19T06:54:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.231+0000 7f6d40b0a640 1 --2- 172.21.15.103:0/3226997397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c1069e0 0x7f6d3c1a4170 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f6d34005cc0 tx=0x7f6d34003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.231+0000 7f6d397fa640 1 -- 172.21.15.103:0/3226997397 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34004040 con 0x7f6d3c1069e0 2024-09-19T06:54:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.231+0000 7f6d397fa640 1 -- 172.21.15.103:0/3226997397 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d340046c0 con 0x7f6d3c1069e0 2024-09-19T06:54:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.231+0000 7f6d4256d640 1 -- 172.21.15.103:0/3226997397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3c1a4880 con 0x7f6d3c1069e0 2024-09-19T06:54:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.231+0000 7f6d397fa640 1 -- 172.21.15.103:0/3226997397 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34030e70 con 0x7f6d3c1069e0 2024-09-19T06:54:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.231+0000 7f6d4256d640 1 -- 172.21.15.103:0/3226997397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d3c1a4d90 con 0x7f6d3c1069e0 2024-09-19T06:54:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.234+0000 7f6d397fa640 1 -- 172.21.15.103:0/3226997397 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f6d340305a0 con 0x7f6d3c1069e0 2024-09-19T06:54:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.234+0000 7f6d453e9740 1 Processor -- start 2024-09-19T06:54:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.234+0000 7f6d453e9740 1 -- start start 2024-09-19T06:54:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.234+0000 7f6d453e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1afaf0 0x5573ce1afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.234+0000 7f6d453e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1b0410 0x5573ce1ba770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.234+0000 7f6d453e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1bacb0 0x5573ce1bd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.234+0000 7f6d453e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5573ce1142f0 con 0x5573ce1afaf0 2024-09-19T06:54:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.234+0000 7f6d453e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5573ce0bc270 con 0x5573ce1b0410 2024-09-19T06:54:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.234+0000 7f6d453e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5573ce1102b0 con 0x5573ce1bacb0 2024-09-19T06:54:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.234+0000 7f6d3b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1afaf0 0x5573ce1afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.234+0000 7f6d397fa640 1 --2- 172.21.15.103:0/3226997397 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6d240a34e0 0x7f6d240a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.234+0000 7f6d3b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1afaf0 0x5573ce1afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54916/0 (socket says 172.21.15.103:54916) 2024-09-19T06:54:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.234+0000 7f6d3b7fe640 1 -- 172.21.15.103:0/3252853626 learned_addr learned my addr 172.21.15.103:0/3252853626 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.234+0000 7f6d40b0a640 1 --2- 172.21.15.103:0/3226997397 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6d240a34e0 0x7f6d240a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.235+0000 7f6d3bfff640 1 --2- 172.21.15.103:0/3252853626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1b0410 0x5573ce1ba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.235+0000 7f6d397fa640 1 -- 172.21.15.103:0/3226997397 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6d3402e030 con 0x7f6d3c1069e0 2024-09-19T06:54:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.235+0000 7f6d3b7fe640 1 -- 172.21.15.103:0/3252853626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1bacb0 msgr2=0x5573ce1bd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.235+0000 7f6d3b7fe640 1 --2- 172.21.15.103:0/3252853626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1bacb0 0x5573ce1bd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.235+0000 7f6d3b7fe640 1 -- 172.21.15.103:0/3252853626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1b0410 msgr2=0x5573ce1ba770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.235+0000 7f6d3b7fe640 1 --2- 172.21.15.103:0/3252853626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1b0410 0x5573ce1ba770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.235+0000 7f6d3b7fe640 1 -- 172.21.15.103:0/3252853626 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573ce0bb340 con 0x5573ce1afaf0 2024-09-19T06:54:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.235+0000 7f6d3b7fe640 1 --2- 172.21.15.103:0/3252853626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1afaf0 0x5573ce1afed0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000b7c0 tx=0x7f6d3000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6374c621812825b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.235+0000 7f6d40b0a640 1 --2- 172.21.15.103:0/3226997397 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6d240a34e0 0x7f6d240a59a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6d340b99b0 tx=0x7f6d340b2470 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.235+0000 7f6d227fc640 1 -- 172.21.15.103:0/3252853626 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30003dd0 con 0x5573ce1afaf0 2024-09-19T06:54:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.235+0000 7f6d227fc640 1 -- 172.21.15.103:0/3252853626 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d30016030 con 0x5573ce1afaf0 2024-09-19T06:54:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.235+0000 7f6d227fc640 1 -- 172.21.15.103:0/3252853626 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30004da0 con 0x5573ce1afaf0 2024-09-19T06:54:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.236+0000 7f6d453e9740 1 -- 172.21.15.103:0/3252853626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1afaf0 msgr2=0x5573ce1afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.236+0000 7f6d453e9740 1 --2- 172.21.15.103:0/3252853626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1afaf0 0x5573ce1afed0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000b7c0 tx=0x7f6d3000bc80 comp rx=0 tx=0).stop 2024-09-19T06:54:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.236+0000 7f6d453e9740 1 -- 172.21.15.103:0/3252853626 shutdown_connections 2024-09-19T06:54:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.236+0000 7f6d453e9740 1 --2- 172.21.15.103:0/3252853626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1bacb0 0x5573ce1bd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.236+0000 7f6d453e9740 1 --2- 172.21.15.103:0/3252853626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1b0410 0x5573ce1ba770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.236+0000 7f6d453e9740 1 --2- 172.21.15.103:0/3252853626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1afaf0 0x5573ce1afed0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.236+0000 7f6d453e9740 1 -- 172.21.15.103:0/3252853626 >> 172.21.15.103:0/3252853626 conn(0x5573ce1ac060 msgr2=0x5573ce1ac440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.236+0000 7f6d453e9740 1 -- 172.21.15.103:0/3252853626 shutdown_connections 2024-09-19T06:54:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.236+0000 7f6d453e9740 1 -- 172.21.15.103:0/3252853626 wait complete. 2024-09-19T06:54:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.236+0000 7f6d453e9740 1 -- start start 2024-09-19T06:54:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.236+0000 7f6d453e9740 1 Processor -- start 2024-09-19T06:54:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d453e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1afaf0 0x5573ce1b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d453e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1b0410 0x5573ce1b8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d453e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d453e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5573ce1bd880 con 0x5573ce1bacb0 2024-09-19T06:54:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d453e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5573ce09d290 con 0x5573ce1b0410 2024-09-19T06:54:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d453e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5573ce1bda00 con 0x5573ce1afaf0 2024-09-19T06:54:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d453e9740 10 client.0 _ll_register_callbacks cb 0x5573ce0ce140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d3b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d3b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54918/0 (socket says 172.21.15.103:54918) 2024-09-19T06:54:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d3b7fe640 1 -- 172.21.15.103:0/1750958468 learned_addr learned my addr 172.21.15.103:0/1750958468 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d3b7fe640 1 --2- 172.21.15.103:0/1750958468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1afaf0 0x5573ce1b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d3bfff640 1 --2- 172.21.15.103:0/1750958468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1b0410 0x5573ce1b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d3b7fe640 1 -- 172.21.15.103:0/1750958468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1afaf0 msgr2=0x5573ce1b7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d3b7fe640 1 --2- 172.21.15.103:0/1750958468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1afaf0 0x5573ce1b7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d3b7fe640 1 -- 172.21.15.103:0/1750958468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1b0410 msgr2=0x5573ce1b8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d3b7fe640 1 --2- 172.21.15.103:0/1750958468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1b0410 0x5573ce1b8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d3b7fe640 1 -- 172.21.15.103:0/1750958468 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573cde27210 con 0x5573ce1bacb0 2024-09-19T06:54:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d3bfff640 1 --2- 172.21.15.103:0/1750958468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1b0410 0x5573ce1b8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d3b7fe640 1 --2- 172.21.15.103:0/1750958468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b8540 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f6d30005fd0 tx=0x7f6d30004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3ce3adfe1ea685 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d1b7fe640 1 -- 172.21.15.103:0/1750958468 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3001b470 con 0x5573ce1bacb0 2024-09-19T06:54:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d1b7fe640 1 -- 172.21.15.103:0/1750958468 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6d3001b610 con 0x5573ce1bacb0 2024-09-19T06:54:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.237+0000 7f6d453e9740 1 -- 172.21.15.103:0/1750958468 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d30016030 con 0x5573ce1bacb0 2024-09-19T06:54:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.238+0000 7f6d1b7fe640 1 -- 172.21.15.103:0/1750958468 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6d3002f050 con 0x5573ce1bacb0 2024-09-19T06:54:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.238+0000 7f6d453e9740 10 client.17244 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.238+0000 7f6d453e9740 1 -- 172.21.15.103:0/1750958468 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573cdef3620 con 0x5573ce1bacb0 2024-09-19T06:54:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.238+0000 7f6d1b7fe640 1 -- 172.21.15.103:0/1750958468 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d30016030 con 0x5573ce1bacb0 2024-09-19T06:54:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.238+0000 7f6d1b7fe640 1 -- 172.21.15.103:0/1750958468 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6d30010a30 con 0x5573ce1bacb0 2024-09-19T06:54:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.238+0000 7f6d453e9740 10 client.17244 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.238+0000 7f6d453e9740 10 client.17244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.238+0000 7f6d453e9740 20 client.17244 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.238+0000 7f6d453e9740 1 --2- 172.21.15.103:0/1750958468 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5573ce1caaa0 0x5573ce1cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.238+0000 7f6d453e9740 4 client.17244 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.238+0000 7f6d453e9740 1 -- 172.21.15.103:0/1750958468 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573ce121a20 con 0x5573ce1caaa0 2024-09-19T06:54:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.238+0000 7f6d3bfff640 1 --2- 172.21.15.103:0/1750958468 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5573ce1caaa0 0x5573ce1cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.239+0000 7f6d1b7fe640 10 client.17244 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.239+0000 7f6d3bfff640 1 --2- 172.21.15.103:0/1750958468 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5573ce1caaa0 0x5573ce1cce90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf3e697538046dd3 server_cookie=4c57f36992fcade7 in_seq=0 out_seq=0 2024-09-19T06:54:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.239+0000 7f6d1b7fe640 1 -- 172.21.15.103:0/1750958468 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573ce121a20 con 0x5573ce1caaa0 2024-09-19T06:54:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.239+0000 7f6d1b7fe640 10 client.17244 handle_command_reply: tid=0 2024-09-19T06:54:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.239+0000 7f6d453e9740 1 client.17244 shutdown 2024-09-19T06:54:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.239+0000 7f6d453e9740 1 -- 172.21.15.103:0/1750958468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 msgr2=0x5573ce1b8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.239+0000 7f6d453e9740 1 --2- 172.21.15.103:0/1750958468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b8540 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f6d30005fd0 tx=0x7f6d30004a60 comp rx=0 tx=0).stop 2024-09-19T06:54:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.239+0000 7f6d453e9740 1 -- 172.21.15.103:0/1750958468 shutdown_connections 2024-09-19T06:54:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.239+0000 7f6d453e9740 1 --2- 172.21.15.103:0/1750958468 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5573ce1caaa0 0x5573ce1cce90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.239+0000 7f6d1b7fe640 0 client.17244 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.239+0000 7f6d453e9740 1 --2- 172.21.15.103:0/1750958468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b8540 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.239+0000 7f6d453e9740 1 --2- 172.21.15.103:0/1750958468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1b0410 0x5573ce1b8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.239+0000 7f6d453e9740 1 --2- 172.21.15.103:0/1750958468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1afaf0 0x5573ce1b7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.239+0000 7f6d453e9740 1 -- 172.21.15.103:0/1750958468 >> 172.21.15.103:0/1750958468 conn(0x5573ce1ac060 msgr2=0x5573ce1b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.240+0000 7f6d453e9740 1 -- 172.21.15.103:0/1750958468 shutdown_connections 2024-09-19T06:54:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.240+0000 7f6d453e9740 1 -- 172.21.15.103:0/1750958468 wait complete. 2024-09-19T06:54:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.240+0000 7f6d453e9740 20 client.17244 trim_cache size 0 max 16384 2024-09-19T06:54:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d453e9740 1 Processor -- start 2024-09-19T06:54:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d453e9740 1 -- start start 2024-09-19T06:54:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d453e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1afaf0 0x5573ce1b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d453e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1b0410 0x5573ce1b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d453e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d453e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c1ad680 con 0x5573ce1bacb0 2024-09-19T06:54:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d453e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c1af1c0 con 0x5573ce1afaf0 2024-09-19T06:54:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d453e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c1add50 con 0x5573ce1b0410 2024-09-19T06:54:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d3b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d3b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54926/0 (socket says 172.21.15.103:54926) 2024-09-19T06:54:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d3b7fe640 1 -- 172.21.15.103:0/3832801482 learned_addr learned my addr 172.21.15.103:0/3832801482 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d3b7fe640 1 -- 172.21.15.103:0/3832801482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1b0410 msgr2=0x5573ce1b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d3b7fe640 1 --2- 172.21.15.103:0/3832801482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1b0410 0x5573ce1b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d3b7fe640 1 -- 172.21.15.103:0/3832801482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1afaf0 msgr2=0x5573ce1b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d3b7fe640 1 --2- 172.21.15.103:0/3832801482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1afaf0 0x5573ce1b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d3b7fe640 1 -- 172.21.15.103:0/3832801482 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573ce1b0240 con 0x5573ce1bacb0 2024-09-19T06:54:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.254+0000 7f6d3b7fe640 1 --2- 172.21.15.103:0/3832801482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b8370 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f6d30010e00 tx=0x7f6d3000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906ac207e0288dfe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.255+0000 7f6d21ffb640 1 -- 172.21.15.103:0/3832801482 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30037020 con 0x5573ce1bacb0 2024-09-19T06:54:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.255+0000 7f6d21ffb640 1 -- 172.21.15.103:0/3832801482 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d30016070 con 0x5573ce1bacb0 2024-09-19T06:54:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.255+0000 7f6d21ffb640 1 -- 172.21.15.103:0/3832801482 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3002aca0 con 0x5573ce1bacb0 2024-09-19T06:54:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.255+0000 7f6d453e9740 1 -- 172.21.15.103:0/3832801482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 msgr2=0x5573ce1b8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.255+0000 7f6d453e9740 1 --2- 172.21.15.103:0/3832801482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b8370 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f6d30010e00 tx=0x7f6d3000bc80 comp rx=0 tx=0).stop 2024-09-19T06:54:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.255+0000 7f6d453e9740 1 -- 172.21.15.103:0/3832801482 shutdown_connections 2024-09-19T06:54:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.255+0000 7f6d453e9740 1 --2- 172.21.15.103:0/3832801482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b8370 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.255+0000 7f6d453e9740 1 --2- 172.21.15.103:0/3832801482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1b0410 0x5573ce1b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.255+0000 7f6d453e9740 1 --2- 172.21.15.103:0/3832801482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1afaf0 0x5573ce1b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.255+0000 7f6d453e9740 1 -- 172.21.15.103:0/3832801482 >> 172.21.15.103:0/3832801482 conn(0x5573ce1ac060 msgr2=0x5573ce1b9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.255+0000 7f6d453e9740 1 -- 172.21.15.103:0/3832801482 shutdown_connections 2024-09-19T06:54:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.255+0000 7f6d453e9740 1 -- 172.21.15.103:0/3832801482 wait complete. 2024-09-19T06:54:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.255+0000 7f6d453e9740 1 -- start start 2024-09-19T06:54:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d453e9740 1 Processor -- start 2024-09-19T06:54:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d453e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1afaf0 0x5573ce1c2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d453e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1b0410 0x5573ce1c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d453e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d453e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c1aeaf0 con 0x5573ce1bacb0 2024-09-19T06:54:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d453e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c1ae420 con 0x5573ce1b0410 2024-09-19T06:54:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d453e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c1ab640 con 0x5573ce1afaf0 2024-09-19T06:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d453e9740 10 client.0 _ll_register_callbacks cb 0x7f6d2c00a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d3b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d3b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54932/0 (socket says 172.21.15.103:54932) 2024-09-19T06:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d3b7fe640 1 -- 172.21.15.103:0/820620117 learned_addr learned my addr 172.21.15.103:0/820620117 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d3bfff640 1 --2- 172.21.15.103:0/820620117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1b0410 0x5573ce1c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d3b7fe640 1 -- 172.21.15.103:0/820620117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1afaf0 msgr2=0x5573ce1c2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d3b7fe640 1 --2- 172.21.15.103:0/820620117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1afaf0 0x5573ce1c2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d3b7fe640 1 -- 172.21.15.103:0/820620117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1b0410 msgr2=0x5573ce1c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d3b7fe640 1 --2- 172.21.15.103:0/820620117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1b0410 0x5573ce1c31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d3b7fe640 1 -- 172.21.15.103:0/820620117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573ce1b4640 con 0x5573ce1bacb0 2024-09-19T06:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d3b7fe640 1 --2- 172.21.15.103:0/820620117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b4100 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000b3b0 tx=0x7f6d3002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa897a42254fd932 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.256+0000 7f6d1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.257+0000 7f6d1b7fe640 1 -- 172.21.15.103:0/820620117 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30037040 con 0x5573ce1bacb0 2024-09-19T06:54:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.257+0000 7f6d1b7fe640 1 -- 172.21.15.103:0/820620117 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6d30004700 con 0x5573ce1bacb0 2024-09-19T06:54:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.257+0000 7f6d453e9740 1 -- 172.21.15.103:0/820620117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d30016070 con 0x5573ce1bacb0 2024-09-19T06:54:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.257+0000 7f6d1b7fe640 1 -- 172.21.15.103:0/820620117 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6d3002a9d0 con 0x5573ce1bacb0 2024-09-19T06:54:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.257+0000 7f6d453e9740 10 client.17250 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.257+0000 7f6d453e9740 1 -- 172.21.15.103:0/820620117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573ce234df0 con 0x5573ce1bacb0 2024-09-19T06:54:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.257+0000 7f6d1b7fe640 1 -- 172.21.15.103:0/820620117 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d30016070 con 0x5573ce1bacb0 2024-09-19T06:54:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.257+0000 7f6d1b7fe640 1 -- 172.21.15.103:0/820620117 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6d3003c040 con 0x5573ce1bacb0 2024-09-19T06:54:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.257+0000 7f6d453e9740 10 client.17250 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.257+0000 7f6d453e9740 10 client.17250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.257+0000 7f6d453e9740 20 client.17250 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.257+0000 7f6d453e9740 1 --2- 172.21.15.103:0/820620117 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5573ce1caaa0 0x5573ce2373b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.257+0000 7f6d453e9740 4 client.17250 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:54:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.257+0000 7f6d453e9740 1 -- 172.21.15.103:0/820620117 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5573ce11bd70 con 0x5573ce1caaa0 2024-09-19T06:54:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.257+0000 7f6d3bfff640 1 --2- 172.21.15.103:0/820620117 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5573ce1caaa0 0x5573ce2373b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.258+0000 7f6d3bfff640 1 --2- 172.21.15.103:0/820620117 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5573ce1caaa0 0x5573ce2373b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b4cc2c96bdf3177 server_cookie=19a1938597325108 in_seq=0 out_seq=0 2024-09-19T06:54:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.258+0000 7f6d1b7fe640 10 client.17250 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.267+0000 7f6d1b7fe640 1 -- 172.21.15.103:0/820620117 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5573ce11bd70 con 0x5573ce1caaa0 2024-09-19T06:54:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.267+0000 7f6d1b7fe640 10 client.17250 handle_command_reply: tid=0 2024-09-19T06:54:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.267+0000 7f6d453e9740 1 client.17250 shutdown 2024-09-19T06:54:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.267+0000 7f6d453e9740 1 -- 172.21.15.103:0/820620117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 msgr2=0x5573ce1b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.267+0000 7f6d453e9740 1 --2- 172.21.15.103:0/820620117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b4100 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000b3b0 tx=0x7f6d3002a410 comp rx=0 tx=0).stop 2024-09-19T06:54:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.267+0000 7f6d453e9740 1 -- 172.21.15.103:0/820620117 shutdown_connections 2024-09-19T06:54:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.267+0000 7f6d453e9740 1 --2- 172.21.15.103:0/820620117 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5573ce1caaa0 0x5573ce2373b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.267+0000 7f6d453e9740 1 --2- 172.21.15.103:0/820620117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573ce1bacb0 0x5573ce1b4100 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.267+0000 7f6d453e9740 1 --2- 172.21.15.103:0/820620117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573ce1b0410 0x5573ce1c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.267+0000 7f6d1b7fe640 0 client.17250 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.267+0000 7f6d453e9740 1 --2- 172.21.15.103:0/820620117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573ce1afaf0 0x5573ce1c2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.267+0000 7f6d453e9740 1 -- 172.21.15.103:0/820620117 >> 172.21.15.103:0/820620117 conn(0x5573ce1ac060 msgr2=0x5573ce1aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.267+0000 7f6d453e9740 1 -- 172.21.15.103:0/820620117 shutdown_connections 2024-09-19T06:54:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.267+0000 7f6d453e9740 1 -- 172.21.15.103:0/820620117 wait complete. 2024-09-19T06:54:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.267+0000 7f6d453e9740 20 client.17250 trim_cache size 0 max 16384 2024-09-19T06:54:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:54:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:54:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:54:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:54:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.268+0000 7f6d4256d640 1 -- 172.21.15.103:0/3226997397 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6d240a34e0 msgr2=0x7f6d240a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.268+0000 7f6d4256d640 1 --2- 172.21.15.103:0/3226997397 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6d240a34e0 0x7f6d240a59a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6d340b99b0 tx=0x7f6d340b2470 comp rx=0 tx=0).stop 2024-09-19T06:54:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.268+0000 7f6d4256d640 1 -- 172.21.15.103:0/3226997397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c1069e0 msgr2=0x7f6d3c1a4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.268+0000 7f6d4256d640 1 --2- 172.21.15.103:0/3226997397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c1069e0 0x7f6d3c1a4170 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f6d34005cc0 tx=0x7f6d34003970 comp rx=0 tx=0).stop 2024-09-19T06:54:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.268+0000 7f6d4256d640 1 -- 172.21.15.103:0/3226997397 shutdown_connections 2024-09-19T06:54:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.268+0000 7f6d4256d640 1 --2- 172.21.15.103:0/3226997397 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6d240a34e0 0x7f6d240a59a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.268+0000 7f6d4256d640 1 --2- 172.21.15.103:0/3226997397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c1069e0 0x7f6d3c1a4170 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.268+0000 7f6d4256d640 1 --2- 172.21.15.103:0/3226997397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d3c101370 0x7f6d3c199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.268+0000 7f6d4256d640 1 --2- 172.21.15.103:0/3226997397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c1009d0 0x7f6d3c1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.268+0000 7f6d4256d640 1 -- 172.21.15.103:0/3226997397 >> 172.21.15.103:0/3226997397 conn(0x7f6d3c0fc6b0 msgr2=0x7f6d3c0fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.268+0000 7f6d4256d640 1 -- 172.21.15.103:0/3226997397 shutdown_connections 2024-09-19T06:54:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:01.268+0000 7f6d4256d640 1 -- 172.21.15.103:0/3226997397 wait complete. 2024-09-19T06:54:01.906 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:54:02.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:01 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T06:54:02.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:01 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:02.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:01 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T06:54:02.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:01 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:02.156 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:02.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:01 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T06:54:02.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:01 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:02.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:01 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T06:54:02.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:01 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:01 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T06:54:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:01 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:01 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T06:54:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:01 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:02.550 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:54:02.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.613+0000 7efdb0c1c640 1 -- 172.21.15.103:0/1191012965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdac106f20 msgr2=0x7efdac10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:02.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.613+0000 7efdb0c1c640 1 --2- 172.21.15.103:0/1191012965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdac106f20 0x7efdac10eaa0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7efda4009ca0 tx=0x7efda402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.614+0000 7efdb0c1c640 1 -- 172.21.15.103:0/1191012965 shutdown_connections 2024-09-19T06:54:02.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.614+0000 7efdb0c1c640 1 --2- 172.21.15.103:0/1191012965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdac106f20 0x7efdac10eaa0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.614+0000 7efdb0c1c640 1 --2- 172.21.15.103:0/1191012965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdac1027a0 0x7efdac1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.614+0000 7efdb0c1c640 1 --2- 172.21.15.103:0/1191012965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdac101df0 0x7efdac1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.614+0000 7efdb0c1c640 1 -- 172.21.15.103:0/1191012965 >> 172.21.15.103:0/1191012965 conn(0x7efdac0fd8f0 msgr2=0x7efdac0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:02.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.614+0000 7efdb0c1c640 1 -- 172.21.15.103:0/1191012965 shutdown_connections 2024-09-19T06:54:02.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.614+0000 7efdb0c1c640 1 -- 172.21.15.103:0/1191012965 wait complete. 2024-09-19T06:54:02.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.614+0000 7efdb0c1c640 1 Processor -- start 2024-09-19T06:54:02.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.614+0000 7efdb0c1c640 1 -- start start 2024-09-19T06:54:02.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efdb0c1c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdac101df0 0x7efdac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efdb0c1c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdac1027a0 0x7efdac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efdb0c1c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdac106f20 0x7efdac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efdb0c1c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efdac111f80 con 0x7efdac101df0 2024-09-19T06:54:02.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efdb0c1c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efdac111e00 con 0x7efdac106f20 2024-09-19T06:54:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efdb0c1c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efdac112100 con 0x7efdac1027a0 2024-09-19T06:54:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efdaa59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdac101df0 0x7efdac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efdaa59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdac101df0 0x7efdac0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54944/0 (socket says 172.21.15.103:54944) 2024-09-19T06:54:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efdaa59c640 1 -- 172.21.15.103:0/2189557410 learned_addr learned my addr 172.21.15.103:0/2189557410 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efda9d9b640 1 --2- 172.21.15.103:0/2189557410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdac1027a0 0x7efdac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efdaad9d640 1 --2- 172.21.15.103:0/2189557410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdac106f20 0x7efdac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efdaa59c640 1 -- 172.21.15.103:0/2189557410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdac1027a0 msgr2=0x7efdac074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efdaa59c640 1 --2- 172.21.15.103:0/2189557410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdac1027a0 0x7efdac074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efdaa59c640 1 -- 172.21.15.103:0/2189557410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdac106f20 msgr2=0x7efdac074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efdaa59c640 1 --2- 172.21.15.103:0/2189557410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdac106f20 0x7efdac074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efdaa59c640 1 -- 172.21.15.103:0/2189557410 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac1a6900 con 0x7efdac101df0 2024-09-19T06:54:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efdaa59c640 1 --2- 172.21.15.103:0/2189557410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdac101df0 0x7efdac0744f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00d950 tx=0x7efd9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.615+0000 7efd9b7fe640 1 -- 172.21.15.103:0/2189557410 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c017070 con 0x7efdac101df0 2024-09-19T06:54:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.616+0000 7efdb0c1c640 1 -- 172.21.15.103:0/2189557410 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdac1a6bf0 con 0x7efdac101df0 2024-09-19T06:54:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.616+0000 7efd9b7fe640 1 -- 172.21.15.103:0/2189557410 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd9c009c30 con 0x7efdac101df0 2024-09-19T06:54:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.616+0000 7efd9b7fe640 1 -- 172.21.15.103:0/2189557410 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00f6b0 con 0x7efdac101df0 2024-09-19T06:54:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.616+0000 7efdb0c1c640 1 -- 172.21.15.103:0/2189557410 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdac1a7130 con 0x7efdac101df0 2024-09-19T06:54:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.618+0000 7efd9b7fe640 1 -- 172.21.15.103:0/2189557410 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7efd9c0109d0 con 0x7efdac101df0 2024-09-19T06:54:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.618+0000 7efdb2ff5740 1 Processor -- start 2024-09-19T06:54:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.618+0000 7efdb2ff5740 1 -- start start 2024-09-19T06:54:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdb2ff5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755e34a0 0x55cb755e3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdb2ff5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755e3dc0 0x55cb755ee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efd9b7fe640 1 --2- 172.21.15.103:0/2189557410 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efd940a33b0 0x7efd940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdb2ff5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755ee680 0x55cb755f0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdb2ff5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cb755492e0 con 0x55cb755ee680 2024-09-19T06:54:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdb2ff5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cb754f0ce0 con 0x55cb755e3dc0 2024-09-19T06:54:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdb2ff5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cb755452a0 con 0x55cb755e34a0 2024-09-19T06:54:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdaad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755ee680 0x55cb755f0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efda9d9b640 1 --2- 172.21.15.103:0/2189557410 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efd940a33b0 0x7efd940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdaad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755ee680 0x55cb755f0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54960/0 (socket says 172.21.15.103:54960) 2024-09-19T06:54:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdaad9d640 1 -- 172.21.15.103:0/2640550312 learned_addr learned my addr 172.21.15.103:0/2640550312 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdaad9d640 1 --2- 172.21.15.103:0/2640550312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755e34a0 0x55cb755e3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efd9b7fe640 1 -- 172.21.15.103:0/2189557410 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7efd9c0cf920 con 0x7efdac101df0 2024-09-19T06:54:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdaa59c640 1 --2- 172.21.15.103:0/2640550312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755e3dc0 0x55cb755ee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdaad9d640 1 -- 172.21.15.103:0/2640550312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755e34a0 msgr2=0x55cb755e3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdaad9d640 1 --2- 172.21.15.103:0/2640550312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755e34a0 0x55cb755e3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdaad9d640 1 -- 172.21.15.103:0/2640550312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755e3dc0 msgr2=0x55cb755ee140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdaad9d640 1 --2- 172.21.15.103:0/2640550312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755e3dc0 0x55cb755ee140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdaad9d640 1 -- 172.21.15.103:0/2640550312 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb754d5440 con 0x55cb755ee680 2024-09-19T06:54:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efda9d9b640 1 --2- 172.21.15.103:0/2189557410 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efd940a33b0 0x7efd940a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efdac075f80 tx=0x7efda000b390 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efdaad9d640 1 --2- 172.21.15.103:0/2640550312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755ee680 0x55cb755f0a70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7efda4005e30 tx=0x7efda4005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e925867998accce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.619+0000 7efd98ff9640 1 -- 172.21.15.103:0/2640550312 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4031e00 con 0x55cb755ee680 2024-09-19T06:54:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.620+0000 7efd98ff9640 1 -- 172.21.15.103:0/2640550312 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda4036040 con 0x55cb755ee680 2024-09-19T06:54:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.620+0000 7efd98ff9640 1 -- 172.21.15.103:0/2640550312 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4031730 con 0x55cb755ee680 2024-09-19T06:54:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.620+0000 7efdb2ff5740 1 -- 172.21.15.103:0/2640550312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755ee680 msgr2=0x55cb755f0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.620+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/2640550312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755ee680 0x55cb755f0a70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7efda4005e30 tx=0x7efda4005d80 comp rx=0 tx=0).stop 2024-09-19T06:54:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.621+0000 7efdb2ff5740 1 -- 172.21.15.103:0/2640550312 shutdown_connections 2024-09-19T06:54:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.621+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/2640550312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755ee680 0x55cb755f0a70 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.621+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/2640550312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755e3dc0 0x55cb755ee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.621+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/2640550312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755e34a0 0x55cb755e3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.621+0000 7efdb2ff5740 1 -- 172.21.15.103:0/2640550312 >> 172.21.15.103:0/2640550312 conn(0x55cb755dfa30 msgr2=0x55cb755dfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.621+0000 7efdb2ff5740 1 -- 172.21.15.103:0/2640550312 shutdown_connections 2024-09-19T06:54:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.621+0000 7efdb2ff5740 1 -- 172.21.15.103:0/2640550312 wait complete. 2024-09-19T06:54:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.621+0000 7efdb2ff5740 1 -- start start 2024-09-19T06:54:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.621+0000 7efdb2ff5740 1 Processor -- start 2024-09-19T06:54:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.621+0000 7efdb2ff5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755e34a0 0x55cb755edf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.621+0000 7efdb2ff5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755e3dc0 0x55cb755eb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.621+0000 7efdb2ff5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755ee680 0x55cb755eb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.621+0000 7efdb2ff5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cb755f10d0 con 0x55cb755ee680 2024-09-19T06:54:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.621+0000 7efdb2ff5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cb75504050 con 0x55cb755e34a0 2024-09-19T06:54:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.621+0000 7efdb2ff5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cb755f1250 con 0x55cb755e3dc0 2024-09-19T06:54:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.621+0000 7efdb2ff5740 10 client.0 _ll_register_callbacks cb 0x55cb75544160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efdaad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755ee680 0x55cb755eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efdaa59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755e34a0 0x55cb755edf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efdaad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755ee680 0x55cb755eb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54976/0 (socket says 172.21.15.103:54976) 2024-09-19T06:54:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efdaad9d640 1 -- 172.21.15.103:0/4159408853 learned_addr learned my addr 172.21.15.103:0/4159408853 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efda9d9b640 1 --2- 172.21.15.103:0/4159408853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755e3dc0 0x55cb755eb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efdaad9d640 1 -- 172.21.15.103:0/4159408853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755e3dc0 msgr2=0x55cb755eb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efdaad9d640 1 --2- 172.21.15.103:0/4159408853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755e3dc0 0x55cb755eb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efdaad9d640 1 -- 172.21.15.103:0/4159408853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755e34a0 msgr2=0x55cb755edf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efdaad9d640 1 --2- 172.21.15.103:0/4159408853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755e34a0 0x55cb755edf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efdaad9d640 1 -- 172.21.15.103:0/4159408853 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb752ba3c0 con 0x55cb755ee680 2024-09-19T06:54:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efda9d9b640 1 --2- 172.21.15.103:0/4159408853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755e3dc0 0x55cb755eb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efdaa59c640 1 --2- 172.21.15.103:0/4159408853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755e34a0 0x55cb755edf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efdaad9d640 1 --2- 172.21.15.103:0/4159408853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755ee680 0x55cb755eb740 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7efda402f540 tx=0x7efda402f480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd74eaec3842113 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efd91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efd91ffb640 1 -- 172.21.15.103:0/4159408853 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4058070 con 0x55cb755ee680 2024-09-19T06:54:02.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efd91ffb640 1 -- 172.21.15.103:0/4159408853 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7efda4031740 con 0x55cb755ee680 2024-09-19T06:54:02.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efdb2ff5740 1 -- 172.21.15.103:0/4159408853 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda4036040 con 0x55cb755ee680 2024-09-19T06:54:02.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efd91ffb640 1 -- 172.21.15.103:0/4159408853 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efda4060050 con 0x55cb755ee680 2024-09-19T06:54:02.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efdb2ff5740 10 client.17262 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:02.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.622+0000 7efdb2ff5740 1 -- 172.21.15.103:0/4159408853 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cb755f10d0 con 0x55cb755ee680 2024-09-19T06:54:02.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.623+0000 7efd91ffb640 1 -- 172.21.15.103:0/4159408853 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda4036040 con 0x55cb755ee680 2024-09-19T06:54:02.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.623+0000 7efd91ffb640 1 -- 172.21.15.103:0/4159408853 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7efda405b030 con 0x55cb755ee680 2024-09-19T06:54:02.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.623+0000 7efdb2ff5740 10 client.17262 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:02.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.623+0000 7efdb2ff5740 10 client.17262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:02.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.623+0000 7efdb2ff5740 20 client.17262 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:02.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.623+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/4159408853 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cb755fdba0 0x55cb755fff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.623+0000 7efdb2ff5740 4 client.17262 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:02.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.623+0000 7efdb2ff5740 1 -- 172.21.15.103:0/4159408853 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb75556850 con 0x55cb755fdba0 2024-09-19T06:54:02.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.623+0000 7efdaa59c640 1 --2- 172.21.15.103:0/4159408853 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cb755fdba0 0x55cb755fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.624+0000 7efdaa59c640 1 --2- 172.21.15.103:0/4159408853 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cb755fdba0 0x55cb755fff90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef4b07ef90c3a152 server_cookie=d13c0ad1b4e7bc2b in_seq=0 out_seq=0 2024-09-19T06:54:02.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.624+0000 7efd91ffb640 10 client.17262 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:02.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.624+0000 7efd91ffb640 1 -- 172.21.15.103:0/4159408853 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb75556850 con 0x55cb755fdba0 2024-09-19T06:54:02.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.624+0000 7efd91ffb640 10 client.17262 handle_command_reply: tid=0 2024-09-19T06:54:02.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.624+0000 7efdb2ff5740 1 client.17262 shutdown 2024-09-19T06:54:02.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.625+0000 7efdb2ff5740 1 -- 172.21.15.103:0/4159408853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755ee680 msgr2=0x55cb755eb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:02.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.625+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/4159408853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755ee680 0x55cb755eb740 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7efda402f540 tx=0x7efda402f480 comp rx=0 tx=0).stop 2024-09-19T06:54:02.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.625+0000 7efdb2ff5740 1 -- 172.21.15.103:0/4159408853 shutdown_connections 2024-09-19T06:54:02.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.625+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/4159408853 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cb755fdba0 0x55cb755fff90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.625+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/4159408853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755ee680 0x55cb755eb740 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.625+0000 7efd91ffb640 0 client.17262 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:02.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.625+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/4159408853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755e3dc0 0x55cb755eb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.625+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/4159408853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755e34a0 0x55cb755edf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.625+0000 7efdb2ff5740 1 -- 172.21.15.103:0/4159408853 >> 172.21.15.103:0/4159408853 conn(0x55cb755dfa30 msgr2=0x55cb755e5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:02.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.625+0000 7efdb2ff5740 1 -- 172.21.15.103:0/4159408853 shutdown_connections 2024-09-19T06:54:02.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.625+0000 7efdb2ff5740 1 -- 172.21.15.103:0/4159408853 wait complete. 2024-09-19T06:54:02.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.625+0000 7efdb2ff5740 20 client.17262 trim_cache size 0 max 16384 2024-09-19T06:54:02.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdb2ff5740 1 Processor -- start 2024-09-19T06:54:02.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdb2ff5740 1 -- start start 2024-09-19T06:54:02.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdb2ff5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755e34a0 0x55cb755fa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdb2ff5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755e3dc0 0x55cb756057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdb2ff5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755ee680 0x55cb75605ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdb2ff5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1ab600 con 0x55cb755e34a0 2024-09-19T06:54:02.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdb2ff5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1ad140 con 0x55cb755e3dc0 2024-09-19T06:54:02.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdb2ff5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1abcd0 con 0x55cb755ee680 2024-09-19T06:54:02.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdaa59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755e34a0 0x55cb755fa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdaa59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755e34a0 0x55cb755fa330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54990/0 (socket says 172.21.15.103:54990) 2024-09-19T06:54:02.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdaa59c640 1 -- 172.21.15.103:0/1051230427 learned_addr learned my addr 172.21.15.103:0/1051230427 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:02.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdaad9d640 1 --2- 172.21.15.103:0/1051230427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755ee680 0x55cb75605ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efda9d9b640 1 --2- 172.21.15.103:0/1051230427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755e3dc0 0x55cb756057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdaa59c640 1 -- 172.21.15.103:0/1051230427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755ee680 msgr2=0x55cb75605ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:02.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdaa59c640 1 --2- 172.21.15.103:0/1051230427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755ee680 0x55cb75605ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdaa59c640 1 -- 172.21.15.103:0/1051230427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755e3dc0 msgr2=0x55cb756057a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:02.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdaa59c640 1 --2- 172.21.15.103:0/1051230427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755e3dc0 0x55cb756057a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.642+0000 7efdaa59c640 1 -- 172.21.15.103:0/1051230427 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb755ee2c0 con 0x55cb755e34a0 2024-09-19T06:54:02.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.643+0000 7efdaa59c640 1 --2- 172.21.15.103:0/1051230427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755e34a0 0x55cb755fa330 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7efd9c09b910 tx=0x7efd9c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78813abcf66bb73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:02.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.643+0000 7efd93fff640 1 -- 172.21.15.103:0/1051230427 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c020040 con 0x55cb755e34a0 2024-09-19T06:54:02.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.643+0000 7efd93fff640 1 -- 172.21.15.103:0/1051230427 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd9c015070 con 0x55cb755e34a0 2024-09-19T06:54:02.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.643+0000 7efd93fff640 1 -- 172.21.15.103:0/1051230427 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0cd300 con 0x55cb755e34a0 2024-09-19T06:54:02.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.643+0000 7efdb2ff5740 1 -- 172.21.15.103:0/1051230427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755e34a0 msgr2=0x55cb755fa330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:02.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.643+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/1051230427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755e34a0 0x55cb755fa330 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7efd9c09b910 tx=0x7efd9c086040 comp rx=0 tx=0).stop 2024-09-19T06:54:02.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.643+0000 7efdb2ff5740 1 -- 172.21.15.103:0/1051230427 shutdown_connections 2024-09-19T06:54:02.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.643+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/1051230427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755ee680 0x55cb75605ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.643+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/1051230427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755e3dc0 0x55cb756057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.643+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/1051230427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755e34a0 0x55cb755fa330 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.644+0000 7efdb2ff5740 1 -- 172.21.15.103:0/1051230427 >> 172.21.15.103:0/1051230427 conn(0x55cb755dfa30 msgr2=0x55cb755ed280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:02.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.644+0000 7efdb2ff5740 1 -- 172.21.15.103:0/1051230427 shutdown_connections 2024-09-19T06:54:02.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.644+0000 7efdb2ff5740 1 -- 172.21.15.103:0/1051230427 wait complete. 2024-09-19T06:54:02.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.644+0000 7efdb2ff5740 1 -- start start 2024-09-19T06:54:02.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.644+0000 7efdb2ff5740 1 Processor -- start 2024-09-19T06:54:02.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.644+0000 7efdb2ff5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755e34a0 0x55cb755f2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.644+0000 7efdb2ff5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755e3dc0 0x55cb755f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.644+0000 7efdb2ff5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755ee680 0x55cb755f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.644+0000 7efdb2ff5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1aca70 con 0x55cb755e34a0 2024-09-19T06:54:02.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.644+0000 7efdb2ff5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1ac3a0 con 0x55cb755ee680 2024-09-19T06:54:02.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.644+0000 7efdb2ff5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1a95e0 con 0x55cb755e3dc0 2024-09-19T06:54:02.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.644+0000 7efdb2ff5740 10 client.0 _ll_register_callbacks cb 0x7efd9c00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:02.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.644+0000 7efdaa59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755e34a0 0x55cb755f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.644+0000 7efdaa59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755e34a0 0x55cb755f2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54996/0 (socket says 172.21.15.103:54996) 2024-09-19T06:54:02.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.644+0000 7efdaa59c640 1 -- 172.21.15.103:0/2540255872 learned_addr learned my addr 172.21.15.103:0/2540255872 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:02.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.645+0000 7efda9d9b640 1 --2- 172.21.15.103:0/2540255872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755e3dc0 0x55cb755f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.645+0000 7efdaad9d640 1 --2- 172.21.15.103:0/2540255872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755ee680 0x55cb755f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.645+0000 7efdaa59c640 1 -- 172.21.15.103:0/2540255872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755e3dc0 msgr2=0x55cb755f28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:02.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.645+0000 7efdaa59c640 1 --2- 172.21.15.103:0/2540255872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755e3dc0 0x55cb755f28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.645+0000 7efdaa59c640 1 -- 172.21.15.103:0/2540255872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755ee680 msgr2=0x55cb755f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:02.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.645+0000 7efdaa59c640 1 --2- 172.21.15.103:0/2540255872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755ee680 0x55cb755f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.645+0000 7efdaa59c640 1 -- 172.21.15.103:0/2540255872 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb756678e0 con 0x55cb755e34a0 2024-09-19T06:54:02.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.645+0000 7efdaa59c640 1 --2- 172.21.15.103:0/2540255872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755e34a0 0x55cb755f2360 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0a4350 tx=0x7efd9c097f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abe01eb55e963d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:02.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.645+0000 7efd91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:02.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.645+0000 7efd91ffb640 1 -- 172.21.15.103:0/2540255872 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0c0060 con 0x55cb755e34a0 2024-09-19T06:54:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.645+0000 7efd91ffb640 1 -- 172.21.15.103:0/2540255872 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7efd9c0cd300 con 0x55cb755e34a0 2024-09-19T06:54:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.645+0000 7efdb2ff5740 1 -- 172.21.15.103:0/2540255872 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd9c015070 con 0x55cb755e34a0 2024-09-19T06:54:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.645+0000 7efd91ffb640 1 -- 172.21.15.103:0/2540255872 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efd9c020040 con 0x55cb755e34a0 2024-09-19T06:54:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.645+0000 7efdb2ff5740 10 client.17268 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.645+0000 7efdb2ff5740 1 -- 172.21.15.103:0/2540255872 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cb75668bb0 con 0x55cb755e34a0 2024-09-19T06:54:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.646+0000 7efd91ffb640 1 -- 172.21.15.103:0/2540255872 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd9c015070 con 0x55cb755e34a0 2024-09-19T06:54:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.646+0000 7efd91ffb640 1 -- 172.21.15.103:0/2540255872 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7efd9c0c1250 con 0x55cb755e34a0 2024-09-19T06:54:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.646+0000 7efdb2ff5740 10 client.17268 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.646+0000 7efdb2ff5740 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.646+0000 7efdb2ff5740 20 client.17268 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.646+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/2540255872 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cb755fdba0 0x55cb7566b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.646+0000 7efdb2ff5740 4 client.17268 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:54:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.646+0000 7efdb2ff5740 1 -- 172.21.15.103:0/2540255872 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cb75550d60 con 0x55cb755fdba0 2024-09-19T06:54:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.646+0000 7efda9d9b640 1 --2- 172.21.15.103:0/2540255872 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cb755fdba0 0x55cb7566b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efda9d9b640 1 --2- 172.21.15.103:0/2540255872 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cb755fdba0 0x55cb7566b1b0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6fd7c3a7bda90e4 server_cookie=c09c14510b86a879 in_seq=0 out_seq=0 2024-09-19T06:54:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efd91ffb640 10 client.17268 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efd91ffb640 1 -- 172.21.15.103:0/2540255872 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cb75550d60 con 0x55cb755fdba0 2024-09-19T06:54:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efd91ffb640 10 client.17268 handle_command_reply: tid=0 2024-09-19T06:54:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efdb2ff5740 1 client.17268 shutdown 2024-09-19T06:54:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efdb2ff5740 1 -- 172.21.15.103:0/2540255872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755e34a0 msgr2=0x55cb755f2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/2540255872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755e34a0 0x55cb755f2360 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0a4350 tx=0x7efd9c097f30 comp rx=0 tx=0).stop 2024-09-19T06:54:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efdb2ff5740 1 -- 172.21.15.103:0/2540255872 shutdown_connections 2024-09-19T06:54:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/2540255872 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cb755fdba0 0x55cb7566b1b0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/2540255872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb755ee680 0x55cb755f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/2540255872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb755e3dc0 0x55cb755f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efdb2ff5740 1 --2- 172.21.15.103:0/2540255872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb755e34a0 0x55cb755f2360 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efd91ffb640 0 client.17268 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efdb2ff5740 1 -- 172.21.15.103:0/2540255872 >> 172.21.15.103:0/2540255872 conn(0x55cb755dfa30 msgr2=0x55cb755de350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efdb2ff5740 1 -- 172.21.15.103:0/2540255872 shutdown_connections 2024-09-19T06:54:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efdb2ff5740 1 -- 172.21.15.103:0/2540255872 wait complete. 2024-09-19T06:54:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.647+0000 7efdb2ff5740 20 client.17268 trim_cache size 0 max 16384 2024-09-19T06:54:02.657 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:54:02.658 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:54:02.658 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T06:54:02.658 INFO:teuthology.orchestra.run.smithi103.stdout: "c340cefb-a68f-48bb-9676-a700d991504b": { 2024-09-19T06:54:02.658 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T06:54:02.658 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "c340cefb-a68f-48bb-9676-a700d991504b", 2024-09-19T06:54:02.658 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T06:54:02.658 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:02.658 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:02.658 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:54:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.648+0000 7efdb0c1c640 1 -- 172.21.15.103:0/2189557410 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efd940a33b0 msgr2=0x7efd940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.648+0000 7efdb0c1c640 1 --2- 172.21.15.103:0/2189557410 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efd940a33b0 0x7efd940a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efdac075f80 tx=0x7efda000b390 comp rx=0 tx=0).stop 2024-09-19T06:54:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.648+0000 7efdb0c1c640 1 -- 172.21.15.103:0/2189557410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdac101df0 msgr2=0x7efdac0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.648+0000 7efdb0c1c640 1 --2- 172.21.15.103:0/2189557410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdac101df0 0x7efdac0744f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00d950 tx=0x7efd9c00de10 comp rx=0 tx=0).stop 2024-09-19T06:54:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.648+0000 7efdb0c1c640 1 -- 172.21.15.103:0/2189557410 shutdown_connections 2024-09-19T06:54:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.648+0000 7efdb0c1c640 1 --2- 172.21.15.103:0/2189557410 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efd940a33b0 0x7efd940a5870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.648+0000 7efdb0c1c640 1 --2- 172.21.15.103:0/2189557410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdac106f20 0x7efdac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.648+0000 7efdb0c1c640 1 --2- 172.21.15.103:0/2189557410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdac1027a0 0x7efdac074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.648+0000 7efdb0c1c640 1 --2- 172.21.15.103:0/2189557410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdac101df0 0x7efdac0744f0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.648+0000 7efdb0c1c640 1 -- 172.21.15.103:0/2189557410 >> 172.21.15.103:0/2189557410 conn(0x7efdac0fd8f0 msgr2=0x7efdac0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.648+0000 7efdb0c1c640 1 -- 172.21.15.103:0/2189557410 shutdown_connections 2024-09-19T06:54:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:02.648+0000 7efdb0c1c640 1 -- 172.21.15.103:0/2189557410 wait complete. 2024-09-19T06:54:03.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:02 smithi103 ceph-mon[31158]: pgmap v147: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T06:54:03.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:02 smithi134 ceph-mon[35356]: pgmap v147: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T06:54:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:02 smithi174 ceph-mon[35637]: pgmap v147: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T06:54:03.354 INFO:tasks.cephfs.filesystem:scrub status for tag:c340cefb-a68f-48bb-9676-a700d991504b - {'path': '/', 'tag': 'c340cefb-a68f-48bb-9676-a700d991504b', 'options': 'recursive,force'} 2024-09-19T06:54:03.447 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:04.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.009+0000 7f4f46471640 1 -- 172.21.15.103:0/2784875362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f40102790 msgr2=0x7f4f401067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.009+0000 7f4f46471640 1 --2- 172.21.15.103:0/2784875362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f40102790 0x7f4f401067e0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009900 tx=0x7f4f2802a410 comp rx=0 tx=0).stop 2024-09-19T06:54:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.009+0000 7f4f46471640 1 -- 172.21.15.103:0/2784875362 shutdown_connections 2024-09-19T06:54:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.009+0000 7f4f46471640 1 --2- 172.21.15.103:0/2784875362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f40106f10 0x7f4f4010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.009+0000 7f4f46471640 1 --2- 172.21.15.103:0/2784875362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f40102790 0x7f4f401067e0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.009+0000 7f4f46471640 1 --2- 172.21.15.103:0/2784875362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f40101de0 0x7f4f401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.009+0000 7f4f46471640 1 -- 172.21.15.103:0/2784875362 >> 172.21.15.103:0/2784875362 conn(0x7f4f400fd8c0 msgr2=0x7f4f400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.009+0000 7f4f46471640 1 -- 172.21.15.103:0/2784875362 shutdown_connections 2024-09-19T06:54:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.009+0000 7f4f46471640 1 -- 172.21.15.103:0/2784875362 wait complete. 2024-09-19T06:54:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.009+0000 7f4f46471640 1 Processor -- start 2024-09-19T06:54:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.009+0000 7f4f46471640 1 -- start start 2024-09-19T06:54:04.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f46471640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f40101de0 0x7f4f401a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:04.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f46471640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f40102790 0x7f4f401a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:04.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f46471640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f40106f10 0x7f4f4019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:04.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f46471640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40112010 con 0x7f4f40106f10 2024-09-19T06:54:04.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f46471640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40111e90 con 0x7f4f40102790 2024-09-19T06:54:04.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f46471640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40112190 con 0x7f4f40101de0 2024-09-19T06:54:04.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f44a0e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f40106f10 0x7f4f4019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:04.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f44a0e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f40106f10 0x7f4f4019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55004/0 (socket says 172.21.15.103:55004) 2024-09-19T06:54:04.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f44a0e640 1 -- 172.21.15.103:0/1429814269 learned_addr learned my addr 172.21.15.103:0/1429814269 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:04.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f3ffff640 1 --2- 172.21.15.103:0/1429814269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f40101de0 0x7f4f401a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:04.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f3f7fe640 1 --2- 172.21.15.103:0/1429814269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f40102790 0x7f4f401a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:04.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f44a0e640 1 -- 172.21.15.103:0/1429814269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f40101de0 msgr2=0x7f4f401a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:04.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f44a0e640 1 --2- 172.21.15.103:0/1429814269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f40101de0 0x7f4f401a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:04.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f44a0e640 1 -- 172.21.15.103:0/1429814269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f40102790 msgr2=0x7f4f401a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:04.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f44a0e640 1 --2- 172.21.15.103:0/1429814269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f40102790 0x7f4f401a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:04.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f44a0e640 1 -- 172.21.15.103:0/1429814269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4019ad00 con 0x7f4f40106f10 2024-09-19T06:54:04.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f3f7fe640 1 --2- 172.21.15.103:0/1429814269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f40102790 0x7f4f401a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:04.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.010+0000 7f4f3ffff640 1 --2- 172.21.15.103:0/1429814269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f40101de0 0x7f4f401a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:04.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.011+0000 7f4f44a0e640 1 --2- 172.21.15.103:0/1429814269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f40106f10 0x7f4f4019a5d0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800b970 tx=0x7f4f3800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:04.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.011+0000 7f4f3d7fa640 1 -- 172.21.15.103:0/1429814269 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38019070 con 0x7f4f40106f10 2024-09-19T06:54:04.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.011+0000 7f4f3d7fa640 1 -- 172.21.15.103:0/1429814269 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f3800cb40 con 0x7f4f40106f10 2024-09-19T06:54:04.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.011+0000 7f4f46471640 1 -- 172.21.15.103:0/1429814269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4019aff0 con 0x7f4f40106f10 2024-09-19T06:54:04.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.011+0000 7f4f3d7fa640 1 -- 172.21.15.103:0/1429814269 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38007650 con 0x7f4f40106f10 2024-09-19T06:54:04.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.011+0000 7f4f46471640 1 -- 172.21.15.103:0/1429814269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f401a72e0 con 0x7f4f40106f10 2024-09-19T06:54:04.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.012+0000 7f4f46471640 1 -- 172.21.15.103:0/1429814269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f40102240 con 0x7f4f40106f10 2024-09-19T06:54:04.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.013+0000 7f4f3d7fa640 1 -- 172.21.15.103:0/1429814269 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4f3801c070 con 0x7f4f40106f10 2024-09-19T06:54:04.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.014+0000 7f4f3d7fa640 1 --2- 172.21.15.103:0/1429814269 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4f2c0a33b0 0x7f4f2c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:04.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.014+0000 7f4f3ffff640 1 --2- 172.21.15.103:0/1429814269 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4f2c0a33b0 0x7f4f2c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:04.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.014+0000 7f4f3d7fa640 1 -- 172.21.15.103:0/1429814269 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f4f380cf700 con 0x7f4f40106f10 2024-09-19T06:54:04.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.014+0000 7f4f3ffff640 1 --2- 172.21.15.103:0/1429814269 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4f2c0a33b0 0x7f4f2c0a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009930 tx=0x7f4f30008040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:04.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.017+0000 7f4f3d7fa640 1 -- 172.21.15.103:0/1429814269 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4f3809c200 con 0x7f4f40106f10 2024-09-19T06:54:04.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.224+0000 7f4f46471640 1 -- 172.21.15.103:0/1429814269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f4019bb00 con 0x7f4f40106f10 2024-09-19T06:54:04.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.224+0000 7f4f3d7fa640 1 -- 172.21.15.103:0/1429814269 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f4f4019bb00 con 0x7f4f40106f10 2024-09-19T06:54:04.227 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:54:04.227 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:54:04.227 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:54:04.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.227+0000 7f4f46471640 1 -- 172.21.15.103:0/1429814269 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4f2c0a33b0 msgr2=0x7f4f2c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:04.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.227+0000 7f4f46471640 1 --2- 172.21.15.103:0/1429814269 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4f2c0a33b0 0x7f4f2c0a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009930 tx=0x7f4f30008040 comp rx=0 tx=0).stop 2024-09-19T06:54:04.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.227+0000 7f4f46471640 1 -- 172.21.15.103:0/1429814269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f40106f10 msgr2=0x7f4f4019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:04.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.227+0000 7f4f46471640 1 --2- 172.21.15.103:0/1429814269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f40106f10 0x7f4f4019a5d0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800b970 tx=0x7f4f3800be30 comp rx=0 tx=0).stop 2024-09-19T06:54:04.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.227+0000 7f4f46471640 1 -- 172.21.15.103:0/1429814269 shutdown_connections 2024-09-19T06:54:04.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.227+0000 7f4f46471640 1 --2- 172.21.15.103:0/1429814269 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4f2c0a33b0 0x7f4f2c0a5870 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:04.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.227+0000 7f4f46471640 1 --2- 172.21.15.103:0/1429814269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f40106f10 0x7f4f4019a5d0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:04.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.227+0000 7f4f46471640 1 --2- 172.21.15.103:0/1429814269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f40102790 0x7f4f401a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:04.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.227+0000 7f4f46471640 1 --2- 172.21.15.103:0/1429814269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f40101de0 0x7f4f401a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:04.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.227+0000 7f4f46471640 1 -- 172.21.15.103:0/1429814269 >> 172.21.15.103:0/1429814269 conn(0x7f4f400fd8c0 msgr2=0x7f4f400fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:04.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.227+0000 7f4f46471640 1 -- 172.21.15.103:0/1429814269 shutdown_connections 2024-09-19T06:54:04.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:04.227+0000 7f4f46471640 1 -- 172.21.15.103:0/1429814269 wait complete. 2024-09-19T06:54:04.267 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:04 smithi103 ceph-mon[31158]: pgmap v148: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T06:54:04.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:04 smithi174 ceph-mon[35637]: pgmap v148: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T06:54:04.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:04 smithi134 ceph-mon[35356]: pgmap v148: 97 pgs: 97 active+clean; 4.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T06:54:04.848 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 3237e9a8 --release --await' 2024-09-19T06:54:04.849 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 3237e9a8 --release --await 2024-09-19T06:54:05.097 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:05.331 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:05 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1429814269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:05.331 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:05 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T06:54:05.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:05 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1429814269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:05.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:05 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T06:54:05.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:05 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1429814269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:05.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:05 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T06:54:05.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.533+0000 7f108568c640 1 -- 172.21.15.103:0/801748820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10801027c0 msgr2=0x7f1080106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:05.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.533+0000 7f108568c640 1 --2- 172.21.15.103:0/801748820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10801027c0 0x7f1080106810 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1068009900 tx=0x7f106802a410 comp rx=0 tx=0).stop 2024-09-19T06:54:05.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.533+0000 7f108568c640 1 -- 172.21.15.103:0/801748820 shutdown_connections 2024-09-19T06:54:05.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.533+0000 7f108568c640 1 --2- 172.21.15.103:0/801748820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1080106f40 0x7f108010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.533+0000 7f108568c640 1 --2- 172.21.15.103:0/801748820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10801027c0 0x7f1080106810 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.533+0000 7f108568c640 1 --2- 172.21.15.103:0/801748820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1080101e10 0x7f10801021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.533+0000 7f108568c640 1 -- 172.21.15.103:0/801748820 >> 172.21.15.103:0/801748820 conn(0x7f10800fd8d0 msgr2=0x7f10800ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:05.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.533+0000 7f108568c640 1 -- 172.21.15.103:0/801748820 shutdown_connections 2024-09-19T06:54:05.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.533+0000 7f108568c640 1 -- 172.21.15.103:0/801748820 wait complete. 2024-09-19T06:54:05.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f108568c640 1 Processor -- start 2024-09-19T06:54:05.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f108568c640 1 -- start start 2024-09-19T06:54:05.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f108568c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1080101e10 0x7f1080198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f108568c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10801027c0 0x7f10801988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f108568c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1080106f40 0x7f108019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f108568c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1080111fa0 con 0x7f1080106f40 2024-09-19T06:54:05.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f108568c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1080111e20 con 0x7f1080101e10 2024-09-19T06:54:05.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f108568c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1080112120 con 0x7f10801027c0 2024-09-19T06:54:05.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f107f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1080106f40 0x7f108019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f107f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1080106f40 0x7f108019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55020/0 (socket says 172.21.15.103:55020) 2024-09-19T06:54:05.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f107f7fe640 1 -- 172.21.15.103:0/309085514 learned_addr learned my addr 172.21.15.103:0/309085514 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:05.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f107effd640 1 --2- 172.21.15.103:0/309085514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1080101e10 0x7f1080198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f107f7fe640 1 -- 172.21.15.103:0/309085514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10801027c0 msgr2=0x7f10801988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:05.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f1077fff640 1 --2- 172.21.15.103:0/309085514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10801027c0 0x7f10801988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f107f7fe640 1 --2- 172.21.15.103:0/309085514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10801027c0 0x7f10801988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f107f7fe640 1 -- 172.21.15.103:0/309085514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1080101e10 msgr2=0x7f1080198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:05.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f107f7fe640 1 --2- 172.21.15.103:0/309085514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1080101e10 0x7f1080198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f107f7fe640 1 -- 172.21.15.103:0/309085514 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108019d310 con 0x7f1080106f40 2024-09-19T06:54:05.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.534+0000 7f107effd640 1 --2- 172.21.15.103:0/309085514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1080101e10 0x7f1080198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:05.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.535+0000 7f107f7fe640 1 --2- 172.21.15.103:0/309085514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1080106f40 0x7f108019cc30 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f107800eaa0 tx=0x7f107800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:05.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.535+0000 7f107cff9640 1 -- 172.21.15.103:0/309085514 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107800cdb0 con 0x7f1080106f40 2024-09-19T06:54:05.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.535+0000 7f107cff9640 1 -- 172.21.15.103:0/309085514 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1078005020 con 0x7f1080106f40 2024-09-19T06:54:05.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.535+0000 7f108568c640 1 -- 172.21.15.103:0/309085514 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108019d600 con 0x7f1080106f40 2024-09-19T06:54:05.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.535+0000 7f107cff9640 1 -- 172.21.15.103:0/309085514 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10780116b0 con 0x7f1080106f40 2024-09-19T06:54:05.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.535+0000 7f108568c640 1 -- 172.21.15.103:0/309085514 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10801a4ee0 con 0x7f1080106f40 2024-09-19T06:54:05.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.537+0000 7f107cff9640 1 -- 172.21.15.103:0/309085514 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1078005560 con 0x7f1080106f40 2024-09-19T06:54:05.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.537+0000 7f1087a65740 1 Processor -- start 2024-09-19T06:54:05.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.538+0000 7f1087a65740 1 -- start start 2024-09-19T06:54:05.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.538+0000 7f1087a65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479686060 0x560479686440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.538+0000 7f1087a65740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x560479690d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.538+0000 7f1087a65740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479691240 0x560479693630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.538+0000 7f1087a65740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5604795eb620 con 0x560479686980 2024-09-19T06:54:05.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.538+0000 7f1087a65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560479592e50 con 0x560479686060 2024-09-19T06:54:05.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.538+0000 7f1087a65740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5604795e75e0 con 0x560479691240 2024-09-19T06:54:05.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.538+0000 7f107cff9640 1 --2- 172.21.15.103:0/309085514 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f106c0a3430 0x7f106c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.538+0000 7f107effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x560479690d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.538+0000 7f107effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x560479690d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55022/0 (socket says 172.21.15.103:55022) 2024-09-19T06:54:05.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.538+0000 7f107effd640 1 -- 172.21.15.103:0/1631665159 learned_addr learned my addr 172.21.15.103:0/1631665159 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:05.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.538+0000 7f107f7fe640 1 --2- 172.21.15.103:0/309085514 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f106c0a3430 0x7f106c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.538+0000 7f1077fff640 1 --2- 172.21.15.103:0/1631665159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479686060 0x560479686440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.538+0000 7f1077fff640 1 --2- 172.21.15.103:0/1631665159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479691240 0x560479693630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.538+0000 7f107cff9640 1 -- 172.21.15.103:0/309085514 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f10780cff00 con 0x7f1080106f40 2024-09-19T06:54:05.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.539+0000 7f107effd640 1 -- 172.21.15.103:0/1631665159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479691240 msgr2=0x560479693630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:05.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.539+0000 7f107effd640 1 --2- 172.21.15.103:0/1631665159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479691240 0x560479693630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.539+0000 7f107effd640 1 -- 172.21.15.103:0/1631665159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479686060 msgr2=0x560479686440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:05.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.539+0000 7f107effd640 1 --2- 172.21.15.103:0/1631665159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479686060 0x560479686440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.539+0000 7f107effd640 1 -- 172.21.15.103:0/1631665159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560479583d40 con 0x560479686980 2024-09-19T06:54:05.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.539+0000 7f107f7fe640 1 --2- 172.21.15.103:0/309085514 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f106c0a3430 0x7f106c0a58f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f107809bef0 tx=0x7f107806f020 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:05.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.539+0000 7f107effd640 1 --2- 172.21.15.103:0/1631665159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x560479690d00 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f107000bc20 tx=0x7f107000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5af8b3358fc5ef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:05.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.539+0000 7f10757fa640 1 -- 172.21.15.103:0/1631665159 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070017620 con 0x560479686980 2024-09-19T06:54:05.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.539+0000 7f10757fa640 1 -- 172.21.15.103:0/1631665159 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1070017ca0 con 0x560479686980 2024-09-19T06:54:05.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.539+0000 7f10757fa640 1 -- 172.21.15.103:0/1631665159 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107000fce0 con 0x560479686980 2024-09-19T06:54:05.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.540+0000 7f1087a65740 1 -- 172.21.15.103:0/1631665159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 msgr2=0x560479690d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:05.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.540+0000 7f1087a65740 1 --2- 172.21.15.103:0/1631665159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x560479690d00 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f107000bc20 tx=0x7f107000d710 comp rx=0 tx=0).stop 2024-09-19T06:54:05.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.540+0000 7f1087a65740 1 -- 172.21.15.103:0/1631665159 shutdown_connections 2024-09-19T06:54:05.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.540+0000 7f1087a65740 1 --2- 172.21.15.103:0/1631665159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479691240 0x560479693630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.540+0000 7f1087a65740 1 --2- 172.21.15.103:0/1631665159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x560479690d00 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.540+0000 7f1087a65740 1 --2- 172.21.15.103:0/1631665159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479686060 0x560479686440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.540+0000 7f1087a65740 1 -- 172.21.15.103:0/1631665159 >> 172.21.15.103:0/1631665159 conn(0x5604796825f0 msgr2=0x5604796829d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:05.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.540+0000 7f1087a65740 1 -- 172.21.15.103:0/1631665159 shutdown_connections 2024-09-19T06:54:05.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.540+0000 7f1087a65740 1 -- 172.21.15.103:0/1631665159 wait complete. 2024-09-19T06:54:05.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.540+0000 7f1087a65740 1 -- start start 2024-09-19T06:54:05.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.540+0000 7f1087a65740 1 Processor -- start 2024-09-19T06:54:05.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f1087a65740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479686060 0x56047968e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f1087a65740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047968e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f1087a65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479691240 0x56047968eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f1087a65740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560479698220 con 0x560479686980 2024-09-19T06:54:05.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f1087a65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560479573f10 con 0x560479691240 2024-09-19T06:54:05.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f1087a65740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5604796983a0 con 0x560479686060 2024-09-19T06:54:05.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f1087a65740 10 client.0 _ll_register_callbacks cb 0x560479597db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:05.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f107effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047968e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f107effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047968e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55036/0 (socket says 172.21.15.103:55036) 2024-09-19T06:54:05.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f107effd640 1 -- 172.21.15.103:0/1293065643 learned_addr learned my addr 172.21.15.103:0/1293065643 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:05.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f1077fff640 1 --2- 172.21.15.103:0/1293065643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479686060 0x56047968e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f1077fff640 1 --2- 172.21.15.103:0/1293065643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479691240 0x56047968eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f107effd640 1 -- 172.21.15.103:0/1293065643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479686060 msgr2=0x56047968e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:05.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f107effd640 1 --2- 172.21.15.103:0/1293065643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479686060 0x56047968e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f107effd640 1 -- 172.21.15.103:0/1293065643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479691240 msgr2=0x56047968eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:05.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f107effd640 1 --2- 172.21.15.103:0/1293065643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479691240 0x56047968eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f107effd640 1 -- 172.21.15.103:0/1293065643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604792fa580 con 0x560479686980 2024-09-19T06:54:05.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f1077fff640 1 --2- 172.21.15.103:0/1293065643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479691240 0x56047968eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:05.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f107effd640 1 --2- 172.21.15.103:0/1293065643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047968e5c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f107000b390 tx=0x7f1070005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac72794c6d31d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:05.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f10667fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:05.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f10667fc640 1 -- 172.21.15.103:0/1293065643 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070025020 con 0x560479686980 2024-09-19T06:54:05.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f10667fc640 1 -- 172.21.15.103:0/1293065643 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1070005050 con 0x560479686980 2024-09-19T06:54:05.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.541+0000 7f1087a65740 1 -- 172.21.15.103:0/1293065643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1070017ca0 con 0x560479686980 2024-09-19T06:54:05.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.542+0000 7f10667fc640 1 -- 172.21.15.103:0/1293065643 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1070030050 con 0x560479686980 2024-09-19T06:54:05.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.542+0000 7f1087a65740 10 client.17286 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:05.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.542+0000 7f1087a65740 1 -- 172.21.15.103:0/1293065643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5604793c4960 con 0x560479686980 2024-09-19T06:54:05.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.542+0000 7f10667fc640 1 -- 172.21.15.103:0/1293065643 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1070017ca0 con 0x560479686980 2024-09-19T06:54:05.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.542+0000 7f10667fc640 1 -- 172.21.15.103:0/1293065643 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f10700178d0 con 0x560479686980 2024-09-19T06:54:05.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.542+0000 7f1087a65740 10 client.17286 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:05.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.542+0000 7f1087a65740 10 client.17286 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:54:05.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.542+0000 7f1087a65740 20 client.17286 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:05.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.542+0000 7f1087a65740 1 --2- 172.21.15.103:0/1293065643 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5604796a50b0 0x5604796a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.542+0000 7f1087a65740 4 client.17286 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:05.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.542+0000 7f1087a65740 1 -- 172.21.15.103:0/1293065643 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604795f8b90 con 0x5604796a50b0 2024-09-19T06:54:05.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.542+0000 7f1077fff640 1 --2- 172.21.15.103:0/1293065643 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5604796a50b0 0x5604796a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.543+0000 7f1077fff640 1 --2- 172.21.15.103:0/1293065643 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5604796a50b0 0x5604796a74a0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12d2fc62e823f294 server_cookie=fcfa775d4b7f2f28 in_seq=0 out_seq=0 2024-09-19T06:54:05.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.543+0000 7f10667fc640 10 client.17286 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:05.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.544+0000 7f10667fc640 1 -- 172.21.15.103:0/1293065643 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604795f8b90 con 0x5604796a50b0 2024-09-19T06:54:05.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.544+0000 7f10667fc640 10 client.17286 handle_command_reply: tid=0 2024-09-19T06:54:05.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.544+0000 7f1087a65740 1 client.17286 shutdown 2024-09-19T06:54:05.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.544+0000 7f1087a65740 1 -- 172.21.15.103:0/1293065643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 msgr2=0x56047968e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:05.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.544+0000 7f1087a65740 1 --2- 172.21.15.103:0/1293065643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047968e5c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f107000b390 tx=0x7f1070005cf0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.544+0000 7f1087a65740 1 -- 172.21.15.103:0/1293065643 shutdown_connections 2024-09-19T06:54:05.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.544+0000 7f1087a65740 1 --2- 172.21.15.103:0/1293065643 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5604796a50b0 0x5604796a74a0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.544+0000 7f1087a65740 1 --2- 172.21.15.103:0/1293065643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479691240 0x56047968eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.544+0000 7f1087a65740 1 --2- 172.21.15.103:0/1293065643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047968e5c0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.544+0000 7f10667fc640 0 client.17286 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:05.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.544+0000 7f1087a65740 1 --2- 172.21.15.103:0/1293065643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479686060 0x56047968e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.544+0000 7f1087a65740 1 -- 172.21.15.103:0/1293065643 >> 172.21.15.103:0/1293065643 conn(0x5604796825f0 msgr2=0x560479681750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:05.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.544+0000 7f1087a65740 1 -- 172.21.15.103:0/1293065643 shutdown_connections 2024-09-19T06:54:05.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.544+0000 7f1087a65740 1 -- 172.21.15.103:0/1293065643 wait complete. 2024-09-19T06:54:05.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.544+0000 7f1087a65740 20 client.17286 trim_cache size 0 max 16384 2024-09-19T06:54:05.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.561+0000 7f1087a65740 1 Processor -- start 2024-09-19T06:54:05.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.561+0000 7f1087a65740 1 -- start start 2024-09-19T06:54:05.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.561+0000 7f1087a65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479686060 0x560479681a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.561+0000 7f1087a65740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047968e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.561+0000 7f1087a65740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479691240 0x56047968ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.561+0000 7f1087a65740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10801ab690 con 0x560479686980 2024-09-19T06:54:05.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.561+0000 7f1087a65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10801ad1d0 con 0x560479686060 2024-09-19T06:54:05.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.561+0000 7f1087a65740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10801abd60 con 0x560479691240 2024-09-19T06:54:05.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.561+0000 7f107effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047968e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.561+0000 7f107effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047968e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55048/0 (socket says 172.21.15.103:55048) 2024-09-19T06:54:05.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.561+0000 7f107effd640 1 -- 172.21.15.103:0/3240779518 learned_addr learned my addr 172.21.15.103:0/3240779518 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:05.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.562+0000 7f107effd640 1 -- 172.21.15.103:0/3240779518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479691240 msgr2=0x56047968ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:05.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.562+0000 7f107effd640 1 --2- 172.21.15.103:0/3240779518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479691240 0x56047968ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.562+0000 7f107effd640 1 -- 172.21.15.103:0/3240779518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479686060 msgr2=0x560479681a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:05.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.562+0000 7f1077fff640 1 --2- 172.21.15.103:0/3240779518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479686060 0x560479681a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.562+0000 7f107effd640 1 --2- 172.21.15.103:0/3240779518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479686060 0x560479681a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.562+0000 7f107effd640 1 -- 172.21.15.103:0/3240779518 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560479682420 con 0x560479686980 2024-09-19T06:54:05.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.562+0000 7f107effd640 1 --2- 172.21.15.103:0/3240779518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047968e750 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1070004710 tx=0x7f107000f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c3620e13a93df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:05.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.562+0000 7f1074ff9640 1 -- 172.21.15.103:0/3240779518 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070039070 con 0x560479686980 2024-09-19T06:54:05.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.562+0000 7f1074ff9640 1 -- 172.21.15.103:0/3240779518 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f107000b3c0 con 0x560479686980 2024-09-19T06:54:05.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.562+0000 7f1074ff9640 1 -- 172.21.15.103:0/3240779518 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070025040 con 0x560479686980 2024-09-19T06:54:05.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 -- 172.21.15.103:0/3240779518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 msgr2=0x56047968e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:05.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 --2- 172.21.15.103:0/3240779518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047968e750 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1070004710 tx=0x7f107000f9b0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 -- 172.21.15.103:0/3240779518 shutdown_connections 2024-09-19T06:54:05.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 --2- 172.21.15.103:0/3240779518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479691240 0x56047968ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 --2- 172.21.15.103:0/3240779518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047968e750 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 --2- 172.21.15.103:0/3240779518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479686060 0x560479681a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 -- 172.21.15.103:0/3240779518 >> 172.21.15.103:0/3240779518 conn(0x5604796825f0 msgr2=0x56047968e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:05.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 -- 172.21.15.103:0/3240779518 shutdown_connections 2024-09-19T06:54:05.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 -- 172.21.15.103:0/3240779518 wait complete. 2024-09-19T06:54:05.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 -- start start 2024-09-19T06:54:05.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 Processor -- start 2024-09-19T06:54:05.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479686060 0x56047969d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047969db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479691240 0x5604796985b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10801acb00 con 0x560479686980 2024-09-19T06:54:05.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10801ac430 con 0x560479686060 2024-09-19T06:54:05.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10801a9650 con 0x560479691240 2024-09-19T06:54:05.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f1087a65740 10 client.0 _ll_register_callbacks cb 0x7f107000fdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:05.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f107effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047969db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f107effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047969db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55054/0 (socket says 172.21.15.103:55054) 2024-09-19T06:54:05.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.563+0000 7f107effd640 1 -- 172.21.15.103:0/1132528454 learned_addr learned my addr 172.21.15.103:0/1132528454 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:05.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f1077fff640 1 --2- 172.21.15.103:0/1132528454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479691240 0x5604796985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f1077fff640 1 --2- 172.21.15.103:0/1132528454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479686060 0x56047969d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f107effd640 1 -- 172.21.15.103:0/1132528454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479691240 msgr2=0x5604796985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:05.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f107effd640 1 --2- 172.21.15.103:0/1132528454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479691240 0x5604796985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f107effd640 1 -- 172.21.15.103:0/1132528454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479686060 msgr2=0x56047969d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:05.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f107effd640 1 --2- 172.21.15.103:0/1132528454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479686060 0x56047969d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f107effd640 1 -- 172.21.15.103:0/1132528454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560479698af0 con 0x560479686980 2024-09-19T06:54:05.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f1077fff640 1 --2- 172.21.15.103:0/1132528454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479686060 0x56047969d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:05.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f107effd640 1 --2- 172.21.15.103:0/1132528454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047969db40 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f1070037040 tx=0x7f1070004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509d6d1adc57b418 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:05.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f10667fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:05.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f10667fc640 1 -- 172.21.15.103:0/1132528454 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070039030 con 0x560479686980 2024-09-19T06:54:05.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f10667fc640 1 -- 172.21.15.103:0/1132528454 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1070017630 con 0x560479686980 2024-09-19T06:54:05.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f1087a65740 1 -- 172.21.15.103:0/1132528454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f107000b3c0 con 0x560479686980 2024-09-19T06:54:05.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f10667fc640 1 -- 172.21.15.103:0/1132528454 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1070025040 con 0x560479686980 2024-09-19T06:54:05.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f1087a65740 10 client.17292 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:05.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f1087a65740 1 -- 172.21.15.103:0/1132528454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5604797101f0 con 0x560479686980 2024-09-19T06:54:05.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.564+0000 7f10667fc640 1 -- 172.21.15.103:0/1132528454 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f107000b3c0 con 0x560479686980 2024-09-19T06:54:05.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.565+0000 7f10667fc640 1 -- 172.21.15.103:0/1132528454 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f10700343f0 con 0x560479686980 2024-09-19T06:54:05.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.565+0000 7f1087a65740 10 client.17292 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:05.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.565+0000 7f1087a65740 10 client.17292 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:54:05.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.565+0000 7f1087a65740 20 client.17292 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:05.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.565+0000 7f1087a65740 1 --2- 172.21.15.103:0/1132528454 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560479710850 0x560479712c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.565+0000 7f1087a65740 4 client.17292 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3237e9a8", "release": true, "await": true}] 2024-09-19T06:54:05.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.565+0000 7f1087a65740 1 -- 172.21.15.103:0/1132528454 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3237e9a8", "release": true, "await": true}) -- 0x56047967d950 con 0x560479710850 2024-09-19T06:54:05.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.565+0000 7f1077fff640 1 --2- 172.21.15.103:0/1132528454 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560479710850 0x560479712c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.565+0000 7f1077fff640 1 --2- 172.21.15.103:0/1132528454 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560479710850 0x560479712c70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28a230dc3de36092 server_cookie=2f71942955a6c476 in_seq=0 out_seq=0 2024-09-19T06:54:05.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.565+0000 7f10667fc640 10 client.17292 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:05.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f10667fc640 1 -- 172.21.15.103:0/1132528454 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56047967d950 con 0x560479710850 2024-09-19T06:54:05.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f10667fc640 10 client.17292 handle_command_reply: tid=0 2024-09-19T06:54:05.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f1087a65740 1 client.17292 shutdown 2024-09-19T06:54:05.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f1087a65740 1 -- 172.21.15.103:0/1132528454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 msgr2=0x56047969db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:05.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f1087a65740 1 --2- 172.21.15.103:0/1132528454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047969db40 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f1070037040 tx=0x7f1070004900 comp rx=0 tx=0).stop 2024-09-19T06:54:05.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f1087a65740 1 -- 172.21.15.103:0/1132528454 shutdown_connections 2024-09-19T06:54:05.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f1087a65740 1 --2- 172.21.15.103:0/1132528454 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560479710850 0x560479712c70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f1087a65740 1 --2- 172.21.15.103:0/1132528454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560479691240 0x5604796985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f10667fc640 0 client.17292 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:05.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f1087a65740 1 --2- 172.21.15.103:0/1132528454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479686980 0x56047969db40 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f1087a65740 1 --2- 172.21.15.103:0/1132528454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560479686060 0x56047969d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f1087a65740 1 -- 172.21.15.103:0/1132528454 >> 172.21.15.103:0/1132528454 conn(0x5604796825f0 msgr2=0x560479690450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:05.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f1087a65740 1 -- 172.21.15.103:0/1132528454 shutdown_connections 2024-09-19T06:54:05.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f1087a65740 1 -- 172.21.15.103:0/1132528454 wait complete. 2024-09-19T06:54:05.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f1087a65740 20 client.17292 trim_cache size 0 max 16384 2024-09-19T06:54:05.575 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:54:05.575 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:54:05.576 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:54:05.576 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 20, 2024-09-19T06:54:05.576 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:54:05.576 INFO:teuthology.orchestra.run.smithi103.stdout: "3237e9a8": { 2024-09-19T06:54:05.576 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 20, 2024-09-19T06:54:05.576 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:54:05.576 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:54:05.576 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:54:05.576 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:54:05.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:54:05.576 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:54:05.576 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 149.8, 2024-09-19T06:54:05.576 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:54:05.577 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:54:05.577 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:54:05.577 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:54:05.577 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:54:05.577 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:54:05.577 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:05.577 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:05.577 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:05.577 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:05.577 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:05.577 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:54:05.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f108568c640 1 -- 172.21.15.103:0/309085514 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f106c0a3430 msgr2=0x7f106c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:05.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f108568c640 1 --2- 172.21.15.103:0/309085514 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f106c0a3430 0x7f106c0a58f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f107809bef0 tx=0x7f107806f020 comp rx=0 tx=0).stop 2024-09-19T06:54:05.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f108568c640 1 -- 172.21.15.103:0/309085514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1080106f40 msgr2=0x7f108019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:05.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f108568c640 1 --2- 172.21.15.103:0/309085514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1080106f40 0x7f108019cc30 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f107800eaa0 tx=0x7f107800ef60 comp rx=0 tx=0).stop 2024-09-19T06:54:05.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f108568c640 1 -- 172.21.15.103:0/309085514 shutdown_connections 2024-09-19T06:54:05.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f108568c640 1 --2- 172.21.15.103:0/309085514 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f106c0a3430 0x7f106c0a58f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f108568c640 1 --2- 172.21.15.103:0/309085514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1080106f40 0x7f108019cc30 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f108568c640 1 --2- 172.21.15.103:0/309085514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10801027c0 0x7f10801988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f108568c640 1 --2- 172.21.15.103:0/309085514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1080101e10 0x7f1080198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.569+0000 7f108568c640 1 -- 172.21.15.103:0/309085514 >> 172.21.15.103:0/309085514 conn(0x7f10800fd8d0 msgr2=0x7f10800feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:05.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.570+0000 7f108568c640 1 -- 172.21.15.103:0/309085514 shutdown_connections 2024-09-19T06:54:05.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:05.570+0000 7f108568c640 1 -- 172.21.15.103:0/309085514 wait complete. 2024-09-19T06:54:06.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:06 smithi174 ceph-mon[35637]: pgmap v149: 97 pgs: 97 active+clean; 4.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 30 op/s 2024-09-19T06:54:06.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:06 smithi103 ceph-mon[31158]: pgmap v149: 97 pgs: 97 active+clean; 4.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 30 op/s 2024-09-19T06:54:06.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:06 smithi134 ceph-mon[35356]: pgmap v149: 97 pgs: 97 active+clean; 4.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 30 op/s 2024-09-19T06:54:06.475 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3237e9a8', total seconds elapsed: 33.8 2024-09-19T06:54:06.475 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-09-19T06:54:06.475 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:54:06.722 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:07.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.195+0000 7f310b051640 1 -- 172.21.15.103:0/2235896775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3104101420 msgr2=0x7f310410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:07.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.195+0000 7f310b051640 1 --2- 172.21.15.103:0/2235896775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3104101420 0x7f310410d950 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009cd0 tx=0x7f30fc02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:54:07.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.195+0000 7f310b051640 1 -- 172.21.15.103:0/2235896775 shutdown_connections 2024-09-19T06:54:07.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.195+0000 7f310b051640 1 --2- 172.21.15.103:0/2235896775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3104101420 0x7f310410d950 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:07.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.195+0000 7f310b051640 1 --2- 172.21.15.103:0/2235896775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3104100a80 0x7f3104100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:07.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.195+0000 7f310b051640 1 --2- 172.21.15.103:0/2235896775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3104106a80 0x7f3104106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:07.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.195+0000 7f310b051640 1 -- 172.21.15.103:0/2235896775 >> 172.21.15.103:0/2235896775 conn(0x7f31040fc7a0 msgr2=0x7f31040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:07.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.195+0000 7f310b051640 1 -- 172.21.15.103:0/2235896775 shutdown_connections 2024-09-19T06:54:07.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.195+0000 7f310b051640 1 -- 172.21.15.103:0/2235896775 wait complete. 2024-09-19T06:54:07.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.196+0000 7f310b051640 1 Processor -- start 2024-09-19T06:54:07.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.196+0000 7f310b051640 1 -- start start 2024-09-19T06:54:07.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.196+0000 7f310b051640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3104100a80 0x7f3104197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:07.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.196+0000 7f310b051640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3104101420 0x7f3104197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:07.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.196+0000 7f310b051640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3104106a80 0x7f310419bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:07.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f310b051640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3104110e80 con 0x7f3104101420 2024-09-19T06:54:07.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f310b051640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3104110d00 con 0x7f3104100a80 2024-09-19T06:54:07.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f310b051640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3104111000 con 0x7f3104106a80 2024-09-19T06:54:07.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f3103fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3104101420 0x7f3104197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:07.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f3103fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3104101420 0x7f3104197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55076/0 (socket says 172.21.15.103:55076) 2024-09-19T06:54:07.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f3103fff640 1 -- 172.21.15.103:0/1629779204 learned_addr learned my addr 172.21.15.103:0/1629779204 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:07.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f3108ded640 1 --2- 172.21.15.103:0/1629779204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3104100a80 0x7f3104197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:07.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f31095ee640 1 --2- 172.21.15.103:0/1629779204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3104106a80 0x7f310419bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:07.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f3103fff640 1 -- 172.21.15.103:0/1629779204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3104106a80 msgr2=0x7f310419bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:07.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f3103fff640 1 --2- 172.21.15.103:0/1629779204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3104106a80 0x7f310419bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:07.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f3103fff640 1 -- 172.21.15.103:0/1629779204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3104100a80 msgr2=0x7f3104197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:07.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f3103fff640 1 --2- 172.21.15.103:0/1629779204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3104100a80 0x7f3104197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:07.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f3103fff640 1 -- 172.21.15.103:0/1629779204 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310419c1f0 con 0x7f3104101420 2024-09-19T06:54:07.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f3103fff640 1 --2- 172.21.15.103:0/1629779204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3104101420 0x7f3104197780 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f30f000c920 tx=0x7f30f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:07.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f3101ffb640 1 -- 172.21.15.103:0/1629779204 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0007c70 con 0x7f3104101420 2024-09-19T06:54:07.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f310b051640 1 -- 172.21.15.103:0/1629779204 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310419c4e0 con 0x7f3104101420 2024-09-19T06:54:07.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f3101ffb640 1 -- 172.21.15.103:0/1629779204 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f0007e10 con 0x7f3104101420 2024-09-19T06:54:07.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.197+0000 7f3101ffb640 1 -- 172.21.15.103:0/1629779204 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0018890 con 0x7f3104101420 2024-09-19T06:54:07.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.198+0000 7f310b051640 1 -- 172.21.15.103:0/1629779204 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31041a3dc0 con 0x7f3104101420 2024-09-19T06:54:07.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.198+0000 7f310b051640 1 -- 172.21.15.103:0/1629779204 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3104100ee0 con 0x7f3104101420 2024-09-19T06:54:07.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.199+0000 7f3101ffb640 1 -- 172.21.15.103:0/1629779204 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f30f0018a70 con 0x7f3104101420 2024-09-19T06:54:07.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.200+0000 7f3101ffb640 1 --2- 172.21.15.103:0/1629779204 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f30f40a33b0 0x7f30f40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:07.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.200+0000 7f3101ffb640 1 -- 172.21.15.103:0/1629779204 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f30f00cff10 con 0x7f3104101420 2024-09-19T06:54:07.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.200+0000 7f3108ded640 1 --2- 172.21.15.103:0/1629779204 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f30f40a33b0 0x7f30f40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:07.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.201+0000 7f3108ded640 1 --2- 172.21.15.103:0/1629779204 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f30f40a33b0 0x7f30f40a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f30f800cac0 tx=0x7f30f800c4a0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:07.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.204+0000 7f3101ffb640 1 -- 172.21.15.103:0/1629779204 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f30f0014030 con 0x7f3104101420 2024-09-19T06:54:07.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.406+0000 7f310b051640 1 -- 172.21.15.103:0/1629779204 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31041a4020 con 0x7f3104101420 2024-09-19T06:54:07.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.407+0000 7f3101ffb640 1 -- 172.21.15.103:0/1629779204 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f30f000de20 con 0x7f3104101420 2024-09-19T06:54:07.409 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:54:07.409 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:54:07.409 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:54:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.409+0000 7f310b051640 1 -- 172.21.15.103:0/1629779204 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f30f40a33b0 msgr2=0x7f30f40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.409+0000 7f310b051640 1 --2- 172.21.15.103:0/1629779204 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f30f40a33b0 0x7f30f40a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f30f800cac0 tx=0x7f30f800c4a0 comp rx=0 tx=0).stop 2024-09-19T06:54:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.409+0000 7f310b051640 1 -- 172.21.15.103:0/1629779204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3104101420 msgr2=0x7f3104197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.409+0000 7f310b051640 1 --2- 172.21.15.103:0/1629779204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3104101420 0x7f3104197780 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f30f000c920 tx=0x7f30f000cde0 comp rx=0 tx=0).stop 2024-09-19T06:54:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.409+0000 7f310b051640 1 -- 172.21.15.103:0/1629779204 shutdown_connections 2024-09-19T06:54:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.409+0000 7f310b051640 1 --2- 172.21.15.103:0/1629779204 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f30f40a33b0 0x7f30f40a5870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.409+0000 7f310b051640 1 --2- 172.21.15.103:0/1629779204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3104106a80 0x7f310419bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.409+0000 7f310b051640 1 --2- 172.21.15.103:0/1629779204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3104101420 0x7f3104197780 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.409+0000 7f310b051640 1 --2- 172.21.15.103:0/1629779204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3104100a80 0x7f3104197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.409+0000 7f310b051640 1 -- 172.21.15.103:0/1629779204 >> 172.21.15.103:0/1629779204 conn(0x7f31040fc7a0 msgr2=0x7f31040fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:07.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.409+0000 7f310b051640 1 -- 172.21.15.103:0/1629779204 shutdown_connections 2024-09-19T06:54:07.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:07.409+0000 7f310b051640 1 -- 172.21.15.103:0/1629779204 wait complete. 2024-09-19T06:54:07.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:07 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1629779204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:07.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:07 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1629779204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:07.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:07 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1629779204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:08.034 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-09-19T06:54:08.034 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-09-19T06:54:08.280 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:08.677 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:08 smithi103 ceph-mon[31158]: pgmap v150: 97 pgs: 97 active+clean; 4.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-09-19T06:54:08.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.764+0000 7feec6a78640 1 -- 172.21.15.103:0/4104981982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feec0100e20 msgr2=0x7feec0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:08.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.764+0000 7feec6a78640 1 --2- 172.21.15.103:0/4104981982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feec0100e20 0x7feec0101200 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009900 tx=0x7feeb402a480 comp rx=0 tx=0).stop 2024-09-19T06:54:08.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.764+0000 7feec6a78640 1 -- 172.21.15.103:0/4104981982 shutdown_connections 2024-09-19T06:54:08.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.764+0000 7feec6a78640 1 --2- 172.21.15.103:0/4104981982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feec0102ef0 0x7feec010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.764+0000 7feec6a78640 1 --2- 172.21.15.103:0/4104981982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec0102570 0x7feec01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.764+0000 7feec6a78640 1 --2- 172.21.15.103:0/4104981982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feec0100e20 0x7feec0101200 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.764+0000 7feec6a78640 1 -- 172.21.15.103:0/4104981982 >> 172.21.15.103:0/4104981982 conn(0x7feec00fc7d0 msgr2=0x7feec00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:08.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.764+0000 7feec6a78640 1 -- 172.21.15.103:0/4104981982 shutdown_connections 2024-09-19T06:54:08.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.764+0000 7feec6a78640 1 -- 172.21.15.103:0/4104981982 wait complete. 2024-09-19T06:54:08.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.764+0000 7feec6a78640 1 Processor -- start 2024-09-19T06:54:08.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.764+0000 7feec6a78640 1 -- start start 2024-09-19T06:54:08.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.764+0000 7feec6a78640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feec0100e20 0x7feec0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feec6a78640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feec0102570 0x7feec0198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feec6a78640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec0102ef0 0x7feec019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feec6a78640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feec0110fa0 con 0x7feec0100e20 2024-09-19T06:54:08.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feec6a78640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feec0110e20 con 0x7feec0102ef0 2024-09-19T06:54:08.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feec6a78640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feec0111120 con 0x7feec0102570 2024-09-19T06:54:08.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feec4814640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feec0100e20 0x7feec0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feec4814640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feec0100e20 0x7feec0198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54678/0 (socket says 172.21.15.103:54678) 2024-09-19T06:54:08.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feec4814640 1 -- 172.21.15.103:0/2247668543 learned_addr learned my addr 172.21.15.103:0/2247668543 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:08.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feec5015640 1 --2- 172.21.15.103:0/2247668543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec0102ef0 0x7feec019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feebffff640 1 --2- 172.21.15.103:0/2247668543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feec0102570 0x7feec0198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feec4814640 1 -- 172.21.15.103:0/2247668543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feec0102570 msgr2=0x7feec0198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:08.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feec4814640 1 --2- 172.21.15.103:0/2247668543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feec0102570 0x7feec0198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feec4814640 1 -- 172.21.15.103:0/2247668543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec0102ef0 msgr2=0x7feec019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:08.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feec4814640 1 --2- 172.21.15.103:0/2247668543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec0102ef0 0x7feec019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feec4814640 1 -- 172.21.15.103:0/2247668543 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec019d5a0 con 0x7feec0100e20 2024-09-19T06:54:08.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feec4814640 1 --2- 172.21.15.103:0/2247668543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feec0100e20 0x7feec0198640 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7feeb4006010 tx=0x7feeb4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:08.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feebdffb640 1 -- 172.21.15.103:0/2247668543 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb403f070 con 0x7feec0100e20 2024-09-19T06:54:08.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feebdffb640 1 -- 172.21.15.103:0/2247668543 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb4031e30 con 0x7feec0100e20 2024-09-19T06:54:08.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feec6a78640 1 -- 172.21.15.103:0/2247668543 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec019d830 con 0x7feec0100e20 2024-09-19T06:54:08.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.765+0000 7feebdffb640 1 -- 172.21.15.103:0/2247668543 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4035030 con 0x7feec0100e20 2024-09-19T06:54:08.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.766+0000 7feec6a78640 1 -- 172.21.15.103:0/2247668543 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec01a5170 con 0x7feec0100e20 2024-09-19T06:54:08.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.768+0000 7feebdffb640 1 -- 172.21.15.103:0/2247668543 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7feeb4042070 con 0x7feec0100e20 2024-09-19T06:54:08.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.768+0000 7feec8e51740 1 Processor -- start 2024-09-19T06:54:08.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.768+0000 7feec8e51740 1 -- start start 2024-09-19T06:54:08.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.768+0000 7feec8e51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa87619760 0x55aa87619b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.768+0000 7feec8e51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa8761a080 0x55aa87624400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.768+0000 7feec8e51740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87624940 0x55aa87626d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.768+0000 7feec8e51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aa8757f2a0 con 0x55aa8761a080 2024-09-19T06:54:08.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.768+0000 7feec8e51740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55aa87526b10 con 0x55aa87624940 2024-09-19T06:54:08.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.768+0000 7feec8e51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aa8757b260 con 0x55aa87619760 2024-09-19T06:54:08.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.769+0000 7feec4814640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa8761a080 0x55aa87624400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.769+0000 7feebdffb640 1 --2- 172.21.15.103:0/2247668543 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feeb00a33b0 0x7feeb00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.769+0000 7feec4814640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa8761a080 0x55aa87624400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54688/0 (socket says 172.21.15.103:54688) 2024-09-19T06:54:08.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.769+0000 7feec4814640 1 -- 172.21.15.103:0/205992859 learned_addr learned my addr 172.21.15.103:0/205992859 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:08.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.769+0000 7feebffff640 1 --2- 172.21.15.103:0/205992859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87624940 0x55aa87626d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.769+0000 7feec5015640 1 --2- 172.21.15.103:0/205992859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa87619760 0x55aa87619b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.769+0000 7feec5015640 1 --2- 172.21.15.103:0/2247668543 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feeb00a33b0 0x7feeb00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.769+0000 7feebffff640 1 -- 172.21.15.103:0/205992859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa87619760 msgr2=0x55aa87619b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:08.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.769+0000 7feebffff640 1 --2- 172.21.15.103:0/205992859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa87619760 0x55aa87619b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.769+0000 7feebffff640 1 -- 172.21.15.103:0/205992859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa8761a080 msgr2=0x55aa87624400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:08.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.769+0000 7feebffff640 1 --2- 172.21.15.103:0/205992859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa8761a080 0x55aa87624400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.769+0000 7feebffff640 1 -- 172.21.15.103:0/205992859 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa872f0b90 con 0x55aa87624940 2024-09-19T06:54:08.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.769+0000 7feebdffb640 1 -- 172.21.15.103:0/2247668543 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7feeb40f0710 con 0x7feec0100e20 2024-09-19T06:54:08.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.769+0000 7feebffff640 1 --2- 172.21.15.103:0/205992859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87624940 0x55aa87626d30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feeac0058c0 tx=0x7feeac00be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd2fd569aa0cc47c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:08.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.769+0000 7feec5015640 1 --2- 172.21.15.103:0/2247668543 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feeb00a33b0 0x7feeb00a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feeb80070b0 tx=0x7feeb80093d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:08.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.769+0000 7feeaaffd640 1 -- 172.21.15.103:0/205992859 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac00ddf0 con 0x55aa87624940 2024-09-19T06:54:08.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.770+0000 7feeaaffd640 1 -- 172.21.15.103:0/205992859 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeac017900 con 0x55aa87624940 2024-09-19T06:54:08.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.770+0000 7feeaaffd640 1 -- 172.21.15.103:0/205992859 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac00fac0 con 0x55aa87624940 2024-09-19T06:54:08.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.770+0000 7feec8e51740 1 -- 172.21.15.103:0/205992859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87624940 msgr2=0x55aa87626d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:08.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.770+0000 7feec8e51740 1 --2- 172.21.15.103:0/205992859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87624940 0x55aa87626d30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feeac0058c0 tx=0x7feeac00be70 comp rx=0 tx=0).stop 2024-09-19T06:54:08.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.770+0000 7feec8e51740 1 -- 172.21.15.103:0/205992859 shutdown_connections 2024-09-19T06:54:08.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.770+0000 7feec8e51740 1 --2- 172.21.15.103:0/205992859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87624940 0x55aa87626d30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.770+0000 7feec8e51740 1 --2- 172.21.15.103:0/205992859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa8761a080 0x55aa87624400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.770+0000 7feec8e51740 1 --2- 172.21.15.103:0/205992859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa87619760 0x55aa87619b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.770+0000 7feec8e51740 1 -- 172.21.15.103:0/205992859 >> 172.21.15.103:0/205992859 conn(0x55aa87615cf0 msgr2=0x55aa876160d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:08.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.770+0000 7feec8e51740 1 -- 172.21.15.103:0/205992859 shutdown_connections 2024-09-19T06:54:08.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.770+0000 7feec8e51740 1 -- 172.21.15.103:0/205992859 wait complete. 2024-09-19T06:54:08.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.771+0000 7feec8e51740 1 -- start start 2024-09-19T06:54:08.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.771+0000 7feec8e51740 1 Processor -- start 2024-09-19T06:54:08.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.771+0000 7feec8e51740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87619760 0x55aa87621750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.771+0000 7feec8e51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa8761a080 0x55aa87621c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.771+0000 7feec8e51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa87624940 0x55aa876221d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.771+0000 7feec8e51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aa87627510 con 0x55aa87624940 2024-09-19T06:54:08.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.771+0000 7feec8e51740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55aa875076e0 con 0x55aa87619760 2024-09-19T06:54:08.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.771+0000 7feec8e51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aa87627690 con 0x55aa8761a080 2024-09-19T06:54:08.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.771+0000 7feec8e51740 10 client.0 _ll_register_callbacks cb 0x55aa8752ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:08.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.771+0000 7feec5015640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa87624940 0x55aa876221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.771+0000 7feec4814640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87619760 0x55aa87621750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.771+0000 7feec5015640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa87624940 0x55aa876221d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54704/0 (socket says 172.21.15.103:54704) 2024-09-19T06:54:08.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.771+0000 7feec5015640 1 -- 172.21.15.103:0/2278349414 learned_addr learned my addr 172.21.15.103:0/2278349414 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:08.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7feebffff640 1 --2- 172.21.15.103:0/2278349414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa8761a080 0x55aa87621c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7feec4814640 1 -- 172.21.15.103:0/2278349414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa8761a080 msgr2=0x55aa87621c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:08.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7feec4814640 1 --2- 172.21.15.103:0/2278349414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa8761a080 0x55aa87621c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7feec4814640 1 -- 172.21.15.103:0/2278349414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa87624940 msgr2=0x55aa876221d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:08.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7feec4814640 1 --2- 172.21.15.103:0/2278349414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa87624940 0x55aa876221d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7feec4814640 1 -- 172.21.15.103:0/2278349414 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa87630980 con 0x55aa87619760 2024-09-19T06:54:08.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7feec5015640 1 --2- 172.21.15.103:0/2278349414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa87624940 0x55aa876221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:08.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7feebffff640 1 --2- 172.21.15.103:0/2278349414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa8761a080 0x55aa87621c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:08.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7feec4814640 1 --2- 172.21.15.103:0/2278349414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87619760 0x55aa87621750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feeb40c55e0 tx=0x7feeb40baf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bd1b094f7f9fa51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:08.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7fee9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:54:08.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7fee9ffff640 1 -- 172.21.15.103:0/2278349414 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4049070 con 0x55aa87619760 2024-09-19T06:54:08.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7feec8e51740 1 -- 172.21.15.103:0/2278349414 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeac017900 con 0x55aa87619760 2024-09-19T06:54:08.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7fee9ffff640 1 -- 172.21.15.103:0/2278349414 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7feeb403b5f0 con 0x55aa87619760 2024-09-19T06:54:08.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7fee9ffff640 1 -- 172.21.15.103:0/2278349414 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feeb40bc360 con 0x55aa87619760 2024-09-19T06:54:08.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7feec8e51740 10 client.26213 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:08.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7feec8e51740 1 -- 172.21.15.103:0/2278349414 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa876332b0 con 0x55aa87619760 2024-09-19T06:54:08.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.772+0000 7fee9ffff640 1 -- 172.21.15.103:0/2278349414 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeac017900 con 0x55aa87619760 2024-09-19T06:54:08.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.773+0000 7fee9ffff640 1 -- 172.21.15.103:0/2278349414 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feeb40ee0f0 con 0x55aa87619760 2024-09-19T06:54:08.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.773+0000 7feec8e51740 10 client.26213 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:08.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.773+0000 7feec8e51740 10 client.26213 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:54:08.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.773+0000 7feec8e51740 20 client.26213 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:08.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.773+0000 7feec8e51740 1 --2- 172.21.15.103:0/2278349414 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aa87634760 0x55aa87636c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.773+0000 7feec8e51740 4 client.26213 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:08.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.773+0000 7feec8e51740 1 -- 172.21.15.103:0/2278349414 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa8758c9d0 con 0x55aa87634760 2024-09-19T06:54:08.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.773+0000 7feebffff640 1 --2- 172.21.15.103:0/2278349414 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aa87634760 0x55aa87636c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.773+0000 7fee9ffff640 10 client.26213 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:08.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.773+0000 7feebffff640 1 --2- 172.21.15.103:0/2278349414 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aa87634760 0x55aa87636c20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c31857c859f8106b server_cookie=c50de993148f181c in_seq=0 out_seq=0 2024-09-19T06:54:08.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.774+0000 7fee9ffff640 1 -- 172.21.15.103:0/2278349414 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa8758c9d0 con 0x55aa87634760 2024-09-19T06:54:08.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.774+0000 7fee9ffff640 10 client.26213 handle_command_reply: tid=0 2024-09-19T06:54:08.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.774+0000 7feec8e51740 1 client.26213 shutdown 2024-09-19T06:54:08.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.774+0000 7feec8e51740 1 -- 172.21.15.103:0/2278349414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87619760 msgr2=0x55aa87621750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:08.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.774+0000 7feec8e51740 1 --2- 172.21.15.103:0/2278349414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87619760 0x55aa87621750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feeb40c55e0 tx=0x7feeb40baf10 comp rx=0 tx=0).stop 2024-09-19T06:54:08.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.774+0000 7feec8e51740 1 -- 172.21.15.103:0/2278349414 shutdown_connections 2024-09-19T06:54:08.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.774+0000 7feec8e51740 1 --2- 172.21.15.103:0/2278349414 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aa87634760 0x55aa87636c20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.774+0000 7feec8e51740 1 --2- 172.21.15.103:0/2278349414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa87624940 0x55aa876221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.774+0000 7feec8e51740 1 --2- 172.21.15.103:0/2278349414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa8761a080 0x55aa87621c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.774+0000 7fee9ffff640 0 client.26213 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:08.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.774+0000 7feec8e51740 1 --2- 172.21.15.103:0/2278349414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87619760 0x55aa87621750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.774+0000 7feec8e51740 1 -- 172.21.15.103:0/2278349414 >> 172.21.15.103:0/2278349414 conn(0x55aa87615cf0 msgr2=0x55aa8761ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:08.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.774+0000 7feec8e51740 1 -- 172.21.15.103:0/2278349414 shutdown_connections 2024-09-19T06:54:08.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.775+0000 7feec8e51740 1 -- 172.21.15.103:0/2278349414 wait complete. 2024-09-19T06:54:08.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.775+0000 7feec8e51740 20 client.26213 trim_cache size 0 max 16384 2024-09-19T06:54:08.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.791+0000 7feec8e51740 1 Processor -- start 2024-09-19T06:54:08.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.791+0000 7feec8e51740 1 -- start start 2024-09-19T06:54:08.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.791+0000 7feec8e51740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87619760 0x55aa87620600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.791+0000 7feec8e51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa8761a080 0x55aa8763c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.791+0000 7feec8e51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa87624940 0x55aa8763ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.791+0000 7feec8e51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feec01ab710 con 0x55aa8761a080 2024-09-19T06:54:08.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.791+0000 7feec8e51740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feec01ad250 con 0x55aa87619760 2024-09-19T06:54:08.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.791+0000 7feec8e51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feec01abde0 con 0x55aa87624940 2024-09-19T06:54:08.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.792+0000 7feebffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa8761a080 0x55aa8763c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.792+0000 7feebffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa8761a080 0x55aa8763c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54710/0 (socket says 172.21.15.103:54710) 2024-09-19T06:54:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.792+0000 7feebffff640 1 -- 172.21.15.103:0/1035692561 learned_addr learned my addr 172.21.15.103:0/1035692561 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.792+0000 7feebffff640 1 -- 172.21.15.103:0/1035692561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa87624940 msgr2=0x55aa8763ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.792+0000 7feebffff640 1 --2- 172.21.15.103:0/1035692561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa87624940 0x55aa8763ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.792+0000 7feec4814640 1 --2- 172.21.15.103:0/1035692561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87619760 0x55aa87620600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.792+0000 7feebffff640 1 -- 172.21.15.103:0/1035692561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87619760 msgr2=0x55aa87620600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.792+0000 7feebffff640 1 --2- 172.21.15.103:0/1035692561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87619760 0x55aa87620600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.792+0000 7feebffff640 1 -- 172.21.15.103:0/1035692561 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa87619eb0 con 0x55aa8761a080 2024-09-19T06:54:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.792+0000 7feebffff640 1 --2- 172.21.15.103:0/1035692561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa8761a080 0x55aa8763c4f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7feeac019040 tx=0x7feeac00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0fc6259af82927 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.792+0000 7feeaa7fc640 1 -- 172.21.15.103:0/1035692561 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac031070 con 0x55aa8761a080 2024-09-19T06:54:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.792+0000 7feeaa7fc640 1 -- 172.21.15.103:0/1035692561 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeac005e00 con 0x55aa8761a080 2024-09-19T06:54:08.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.792+0000 7feeaa7fc640 1 -- 172.21.15.103:0/1035692561 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac004660 con 0x55aa8761a080 2024-09-19T06:54:08.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.793+0000 7feec8e51740 1 -- 172.21.15.103:0/1035692561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa8761a080 msgr2=0x55aa8763c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:08.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.793+0000 7feec8e51740 1 --2- 172.21.15.103:0/1035692561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa8761a080 0x55aa8763c4f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7feeac019040 tx=0x7feeac00b910 comp rx=0 tx=0).stop 2024-09-19T06:54:08.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.793+0000 7feec8e51740 1 -- 172.21.15.103:0/1035692561 shutdown_connections 2024-09-19T06:54:08.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.793+0000 7feec8e51740 1 --2- 172.21.15.103:0/1035692561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa87624940 0x55aa8763ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.793+0000 7feec8e51740 1 --2- 172.21.15.103:0/1035692561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa8761a080 0x55aa8763c4f0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.793+0000 7feec8e51740 1 --2- 172.21.15.103:0/1035692561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87619760 0x55aa87620600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.793+0000 7feec8e51740 1 -- 172.21.15.103:0/1035692561 >> 172.21.15.103:0/1035692561 conn(0x55aa87615cf0 msgr2=0x55aa8761fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:08.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.793+0000 7feec8e51740 1 -- 172.21.15.103:0/1035692561 shutdown_connections 2024-09-19T06:54:08.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.793+0000 7feec8e51740 1 -- 172.21.15.103:0/1035692561 wait complete. 2024-09-19T06:54:08.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.793+0000 7feec8e51740 1 -- start start 2024-09-19T06:54:08.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.793+0000 7feec8e51740 1 Processor -- start 2024-09-19T06:54:08.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec8e51740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87619760 0x55aa8762d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec8e51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa8761a080 0x55aa876284e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec8e51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa87624940 0x55aa87628c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec8e51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feec01acb80 con 0x55aa87624940 2024-09-19T06:54:08.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec8e51740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feec01ac4b0 con 0x55aa87619760 2024-09-19T06:54:08.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec8e51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feec01a96d0 con 0x55aa8761a080 2024-09-19T06:54:08.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec8e51740 10 client.0 _ll_register_callbacks cb 0x7feeb4031400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:08.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec5015640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa87624940 0x55aa87628c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec5015640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa87624940 0x55aa87628c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54720/0 (socket says 172.21.15.103:54720) 2024-09-19T06:54:08.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec5015640 1 -- 172.21.15.103:0/3588293990 learned_addr learned my addr 172.21.15.103:0/3588293990 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:08.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feebffff640 1 --2- 172.21.15.103:0/3588293990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa8761a080 0x55aa876284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec4814640 1 --2- 172.21.15.103:0/3588293990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87619760 0x55aa8762d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec5015640 1 -- 172.21.15.103:0/3588293990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa8761a080 msgr2=0x55aa876284e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:08.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec5015640 1 --2- 172.21.15.103:0/3588293990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa8761a080 0x55aa876284e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec5015640 1 -- 172.21.15.103:0/3588293990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87619760 msgr2=0x55aa8762d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:08.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec5015640 1 --2- 172.21.15.103:0/3588293990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87619760 0x55aa8762d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec5015640 1 -- 172.21.15.103:0/3588293990 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa8769e4c0 con 0x55aa87624940 2024-09-19T06:54:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7feec5015640 1 --2- 172.21.15.103:0/3588293990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa87624940 0x55aa87628c70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7feeb800afa0 tx=0x7feeb8004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96d06d00403c73a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.794+0000 7fee9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.795+0000 7fee9ffff640 1 -- 172.21.15.103:0/3588293990 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8013040 con 0x55aa87624940 2024-09-19T06:54:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.795+0000 7fee9ffff640 1 -- 172.21.15.103:0/3588293990 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7feeb8018d20 con 0x55aa87624940 2024-09-19T06:54:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.795+0000 7feec8e51740 1 -- 172.21.15.103:0/3588293990 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeac005e00 con 0x55aa87624940 2024-09-19T06:54:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.795+0000 7fee9ffff640 1 -- 172.21.15.103:0/3588293990 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feeb8037050 con 0x55aa87624940 2024-09-19T06:54:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.795+0000 7feec8e51740 10 client.17316 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.795+0000 7feec8e51740 1 -- 172.21.15.103:0/3588293990 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa8769f630 con 0x55aa87624940 2024-09-19T06:54:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.795+0000 7fee9ffff640 1 -- 172.21.15.103:0/3588293990 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeac005e00 con 0x55aa87624940 2024-09-19T06:54:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.795+0000 7fee9ffff640 1 -- 172.21.15.103:0/3588293990 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feeb8032440 con 0x55aa87624940 2024-09-19T06:54:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.795+0000 7feec8e51740 10 client.17316 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.795+0000 7feec8e51740 10 client.17316 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:54:08.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.795+0000 7feec8e51740 20 client.17316 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:08.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.795+0000 7feec8e51740 1 --2- 172.21.15.103:0/3588293990 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aa8769fe70 0x55aa876a2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:08.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.795+0000 7feec8e51740 4 client.17316 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-09-19T06:54:08.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.795+0000 7feec8e51740 1 -- 172.21.15.103:0/3588293990 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x55aa87586d20 con 0x55aa8769fe70 2024-09-19T06:54:08.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.795+0000 7feec4814640 1 --2- 172.21.15.103:0/3588293990 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aa8769fe70 0x55aa876a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.796+0000 7feec4814640 1 --2- 172.21.15.103:0/3588293990 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aa8769fe70 0x55aa876a2290 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a14de8857e8439bd server_cookie=324a550692ff9e43 in_seq=0 out_seq=0 2024-09-19T06:54:08.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.796+0000 7fee9ffff640 10 client.17316 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:08.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7fee9ffff640 1 -- 172.21.15.103:0/3588293990 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55aa87586d20 con 0x55aa8769fe70 2024-09-19T06:54:08.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7fee9ffff640 10 client.17316 handle_command_reply: tid=0 2024-09-19T06:54:08.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec8e51740 1 client.17316 shutdown 2024-09-19T06:54:08.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec8e51740 1 -- 172.21.15.103:0/3588293990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa87624940 msgr2=0x55aa87628c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:08.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec8e51740 1 --2- 172.21.15.103:0/3588293990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa87624940 0x55aa87628c70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7feeb800afa0 tx=0x7feeb8004b50 comp rx=0 tx=0).stop 2024-09-19T06:54:08.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec8e51740 1 -- 172.21.15.103:0/3588293990 shutdown_connections 2024-09-19T06:54:08.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec8e51740 1 --2- 172.21.15.103:0/3588293990 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aa8769fe70 0x55aa876a2290 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec8e51740 1 --2- 172.21.15.103:0/3588293990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa87624940 0x55aa87628c70 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec8e51740 1 --2- 172.21.15.103:0/3588293990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa8761a080 0x55aa876284e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec8e51740 1 --2- 172.21.15.103:0/3588293990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa87619760 0x55aa8762d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7fee9ffff640 0 client.17316 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:08.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec8e51740 1 -- 172.21.15.103:0/3588293990 >> 172.21.15.103:0/3588293990 conn(0x55aa87615cf0 msgr2=0x55aa87621f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:08.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec8e51740 1 -- 172.21.15.103:0/3588293990 shutdown_connections 2024-09-19T06:54:08.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec8e51740 1 -- 172.21.15.103:0/3588293990 wait complete. 2024-09-19T06:54:08.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec8e51740 20 client.17316 trim_cache size 0 max 16384 2024-09-19T06:54:08.810 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:54:08.810 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:54:08.810 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:54:08.810 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 22, 2024-09-19T06:54:08.811 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:54:08.811 INFO:teuthology.orchestra.run.smithi103.stdout: "b06a23fc": { 2024-09-19T06:54:08.811 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 22, 2024-09-19T06:54:08.811 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:54:08.811 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:54:08.811 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:54:08.811 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:54:08.811 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:54:08.811 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:54:08.811 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 149.8, 2024-09-19T06:54:08.811 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:54:08.811 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:54:08.811 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:54:08.812 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:54:08.812 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:54:08.812 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:54:08.812 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:08.812 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:08.812 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:08.812 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:08.812 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:08.812 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:54:08.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec6a78640 1 -- 172.21.15.103:0/2247668543 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feeb00a33b0 msgr2=0x7feeb00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:08.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec6a78640 1 --2- 172.21.15.103:0/2247668543 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feeb00a33b0 0x7feeb00a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feeb80070b0 tx=0x7feeb80093d0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec6a78640 1 -- 172.21.15.103:0/2247668543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feec0100e20 msgr2=0x7feec0198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:08.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec6a78640 1 --2- 172.21.15.103:0/2247668543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feec0100e20 0x7feec0198640 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7feeb4006010 tx=0x7feeb4003c80 comp rx=0 tx=0).stop 2024-09-19T06:54:08.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec6a78640 1 -- 172.21.15.103:0/2247668543 shutdown_connections 2024-09-19T06:54:08.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec6a78640 1 --2- 172.21.15.103:0/2247668543 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feeb00a33b0 0x7feeb00a5870 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec6a78640 1 --2- 172.21.15.103:0/2247668543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec0102ef0 0x7feec019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec6a78640 1 --2- 172.21.15.103:0/2247668543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feec0102570 0x7feec0198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec6a78640 1 --2- 172.21.15.103:0/2247668543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feec0100e20 0x7feec0198640 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:08.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.807+0000 7feec6a78640 1 -- 172.21.15.103:0/2247668543 >> 172.21.15.103:0/2247668543 conn(0x7feec00fc7d0 msgr2=0x7feec0069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:08.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.808+0000 7feec6a78640 1 -- 172.21.15.103:0/2247668543 shutdown_connections 2024-09-19T06:54:08.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:08.808+0000 7feec6a78640 1 -- 172.21.15.103:0/2247668543 wait complete. 2024-09-19T06:54:08.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:08 smithi174 ceph-mon[35637]: pgmap v150: 97 pgs: 97 active+clean; 4.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-09-19T06:54:08.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:08 smithi134 ceph-mon[35356]: pgmap v150: 97 pgs: 97 active+clean; 4.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-09-19T06:54:09.518 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b06a23fc', quiesce took 3.0 seconds. Will release after: 26.8 2024-09-19T06:54:10.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:10 smithi174 ceph-mon[35637]: pgmap v151: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 KiB/s wr, 32 op/s 2024-09-19T06:54:10.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:10 smithi103 ceph-mon[31158]: pgmap v151: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 KiB/s wr, 32 op/s 2024-09-19T06:54:10.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:10 smithi134 ceph-mon[35356]: pgmap v151: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 KiB/s wr, 32 op/s 2024-09-19T06:54:12.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:12 smithi174 ceph-mon[35637]: pgmap v152: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 KiB/s wr, 28 op/s 2024-09-19T06:54:12.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:12 smithi103 ceph-mon[31158]: pgmap v152: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 KiB/s wr, 28 op/s 2024-09-19T06:54:12.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:12 smithi134 ceph-mon[35356]: pgmap v152: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 KiB/s wr, 28 op/s 2024-09-19T06:54:14.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:14 smithi174 ceph-mon[35637]: pgmap v153: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 KiB/s wr, 28 op/s 2024-09-19T06:54:14.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:14 smithi103 ceph-mon[31158]: pgmap v153: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 KiB/s wr, 28 op/s 2024-09-19T06:54:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.366+0000 7fbc13ed0640 1 -- 172.21.15.103:0/1667863830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc0c1069c0 msgr2=0x7fbc0c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.366+0000 7fbc13ed0640 1 --2- 172.21.15.103:0/1667863830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc0c1069c0 0x7fbc0c106da0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009930 tx=0x7fbc0002a420 comp rx=0 tx=0).stop 2024-09-19T06:54:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.366+0000 7fbc13ed0640 1 -- 172.21.15.103:0/1667863830 shutdown_connections 2024-09-19T06:54:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.366+0000 7fbc13ed0640 1 --2- 172.21.15.103:0/1667863830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc0c101350 0x7fbc0c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.366+0000 7fbc13ed0640 1 --2- 172.21.15.103:0/1667863830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc0c1009b0 0x7fbc0c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.366+0000 7fbc13ed0640 1 --2- 172.21.15.103:0/1667863830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc0c1069c0 0x7fbc0c106da0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.366+0000 7fbc13ed0640 1 -- 172.21.15.103:0/1667863830 >> 172.21.15.103:0/1667863830 conn(0x7fbc0c077990 msgr2=0x7fbc0c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.366+0000 7fbc13ed0640 1 -- 172.21.15.103:0/1667863830 shutdown_connections 2024-09-19T06:54:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.366+0000 7fbc13ed0640 1 -- 172.21.15.103:0/1667863830 wait complete. 2024-09-19T06:54:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.366+0000 7fbc13ed0640 1 Processor -- start 2024-09-19T06:54:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc13ed0640 1 -- start start 2024-09-19T06:54:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc13ed0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc0c1009b0 0x7fbc0c1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc13ed0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc0c101350 0x7fbc0c1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc13ed0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc0c1069c0 0x7fbc0c1a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc13ed0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c110e50 con 0x7fbc0c101350 2024-09-19T06:54:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc13ed0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c110cd0 con 0x7fbc0c1069c0 2024-09-19T06:54:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc13ed0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c110fd0 con 0x7fbc0c1009b0 2024-09-19T06:54:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc1146b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc0c101350 0x7fbc0c1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc1146b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc0c101350 0x7fbc0c1996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54744/0 (socket says 172.21.15.103:54744) 2024-09-19T06:54:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc1146b640 1 -- 172.21.15.103:0/763118779 learned_addr learned my addr 172.21.15.103:0/763118779 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc11c6c640 1 --2- 172.21.15.103:0/763118779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc0c1009b0 0x7fbc0c1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc1146b640 1 -- 172.21.15.103:0/763118779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc0c1009b0 msgr2=0x7fbc0c1991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc1246d640 1 --2- 172.21.15.103:0/763118779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc0c1069c0 0x7fbc0c1a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc1146b640 1 --2- 172.21.15.103:0/763118779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc0c1009b0 0x7fbc0c1991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc1146b640 1 -- 172.21.15.103:0/763118779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc0c1069c0 msgr2=0x7fbc0c1a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc1146b640 1 --2- 172.21.15.103:0/763118779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc0c1069c0 0x7fbc0c1a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc1146b640 1 -- 172.21.15.103:0/763118779 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0c1a4760 con 0x7fbc0c101350 2024-09-19T06:54:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbc1146b640 1 --2- 172.21.15.103:0/763118779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc0c101350 0x7fbc0c1996e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800d950 tx=0x7fbc0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.367+0000 7fbbfeffd640 1 -- 172.21.15.103:0/763118779 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc08017070 con 0x7fbc0c101350 2024-09-19T06:54:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.368+0000 7fbbfeffd640 1 -- 172.21.15.103:0/763118779 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc08009c30 con 0x7fbc0c101350 2024-09-19T06:54:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.368+0000 7fbc13ed0640 1 -- 172.21.15.103:0/763118779 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0c1a4930 con 0x7fbc0c101350 2024-09-19T06:54:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.368+0000 7fbbfeffd640 1 -- 172.21.15.103:0/763118779 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0800f6b0 con 0x7fbc0c101350 2024-09-19T06:54:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.368+0000 7fbc13ed0640 1 -- 172.21.15.103:0/763118779 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0c1a4e50 con 0x7fbc0c101350 2024-09-19T06:54:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.370+0000 7fbbfeffd640 1 -- 172.21.15.103:0/763118779 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fbc080109d0 con 0x7fbc0c101350 2024-09-19T06:54:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.370+0000 7fbc16c65740 1 Processor -- start 2024-09-19T06:54:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.370+0000 7fbc16c65740 1 -- start start 2024-09-19T06:54:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.370+0000 7fbc16c65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f982aaf0 0x55e0f982aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.370+0000 7fbc16c65740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f982b410 0x55e0f9835770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.370+0000 7fbc16c65740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f9835cb0 0x55e0f98380a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.370+0000 7fbc16c65740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e0f978f2f0 con 0x55e0f9835cb0 2024-09-19T06:54:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.370+0000 7fbc16c65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e0f9737270 con 0x55e0f982aaf0 2024-09-19T06:54:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.370+0000 7fbc16c65740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e0f978b2b0 con 0x55e0f982b410 2024-09-19T06:54:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbbfeffd640 1 --2- 172.21.15.103:0/763118779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbbf80a3460 0x7fbbf80a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbc1146b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f9835cb0 0x55e0f98380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbc1146b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f9835cb0 0x55e0f98380a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54756/0 (socket says 172.21.15.103:54756) 2024-09-19T06:54:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbc1146b640 1 -- 172.21.15.103:0/1737160142 learned_addr learned my addr 172.21.15.103:0/1737160142 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbc1246d640 1 --2- 172.21.15.103:0/1737160142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f982aaf0 0x55e0f982aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbc11c6c640 1 --2- 172.21.15.103:0/1737160142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f982b410 0x55e0f9835770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbc1246d640 1 --2- 172.21.15.103:0/763118779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbbf80a3460 0x7fbbf80a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbbfeffd640 1 -- 172.21.15.103:0/763118779 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fbc080cf960 con 0x7fbc0c101350 2024-09-19T06:54:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbc1146b640 1 -- 172.21.15.103:0/1737160142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f982b410 msgr2=0x55e0f9835770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbc1146b640 1 --2- 172.21.15.103:0/1737160142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f982b410 0x55e0f9835770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbc1146b640 1 -- 172.21.15.103:0/1737160142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f982aaf0 msgr2=0x55e0f982aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbc1146b640 1 --2- 172.21.15.103:0/1737160142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f982aaf0 0x55e0f982aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbc1146b640 1 -- 172.21.15.103:0/1737160142 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0f9736340 con 0x55e0f9835cb0 2024-09-19T06:54:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbc1246d640 1 --2- 172.21.15.103:0/763118779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbbf80a3460 0x7fbbf80a5920 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009fd0 tx=0x7fbc04009820 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbc1146b640 1 --2- 172.21.15.103:0/1737160142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f9835cb0 0x55e0f98380a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fbc08098620 tx=0x7fbc0800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c90746e4d56a2ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbbf7fff640 1 -- 172.21.15.103:0/1737160142 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc08020070 con 0x55e0f9835cb0 2024-09-19T06:54:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbbf7fff640 1 -- 172.21.15.103:0/1737160142 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc0809bc50 con 0x55e0f9835cb0 2024-09-19T06:54:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.371+0000 7fbbf7fff640 1 -- 172.21.15.103:0/1737160142 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc080b1040 con 0x55e0f9835cb0 2024-09-19T06:54:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.372+0000 7fbc16c65740 1 -- 172.21.15.103:0/1737160142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f9835cb0 msgr2=0x55e0f98380a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.372+0000 7fbc16c65740 1 --2- 172.21.15.103:0/1737160142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f9835cb0 0x55e0f98380a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fbc08098620 tx=0x7fbc0800bf60 comp rx=0 tx=0).stop 2024-09-19T06:54:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.372+0000 7fbc16c65740 1 -- 172.21.15.103:0/1737160142 shutdown_connections 2024-09-19T06:54:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.372+0000 7fbc16c65740 1 --2- 172.21.15.103:0/1737160142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f9835cb0 0x55e0f98380a0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.372+0000 7fbc16c65740 1 --2- 172.21.15.103:0/1737160142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f982b410 0x55e0f9835770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.372+0000 7fbc16c65740 1 --2- 172.21.15.103:0/1737160142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f982aaf0 0x55e0f982aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.372+0000 7fbc16c65740 1 -- 172.21.15.103:0/1737160142 >> 172.21.15.103:0/1737160142 conn(0x55e0f9827060 msgr2=0x55e0f9827440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.372+0000 7fbc16c65740 1 -- 172.21.15.103:0/1737160142 shutdown_connections 2024-09-19T06:54:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.372+0000 7fbc16c65740 1 -- 172.21.15.103:0/1737160142 wait complete. 2024-09-19T06:54:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.372+0000 7fbc16c65740 1 -- start start 2024-09-19T06:54:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc16c65740 1 Processor -- start 2024-09-19T06:54:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc16c65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f982aaf0 0x55e0f9832bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc16c65740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f982b410 0x55e0f9833100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc16c65740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f9835cb0 0x55e0f9833640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc16c65740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e0f9838820 con 0x55e0f9835cb0 2024-09-19T06:54:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc16c65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e0f9718290 con 0x55e0f982aaf0 2024-09-19T06:54:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc16c65740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e0f98389a0 con 0x55e0f982b410 2024-09-19T06:54:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc16c65740 10 client.0 _ll_register_callbacks cb 0x55e0f9749140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc1246d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f9835cb0 0x55e0f9833640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc1246d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f9835cb0 0x55e0f9833640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54768/0 (socket says 172.21.15.103:54768) 2024-09-19T06:54:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc1246d640 1 -- 172.21.15.103:0/2195271740 learned_addr learned my addr 172.21.15.103:0/2195271740 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc1146b640 1 --2- 172.21.15.103:0/2195271740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f982b410 0x55e0f9833100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc1246d640 1 -- 172.21.15.103:0/2195271740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f982b410 msgr2=0x55e0f9833100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc1246d640 1 --2- 172.21.15.103:0/2195271740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f982b410 0x55e0f9833100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc11c6c640 1 --2- 172.21.15.103:0/2195271740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f982aaf0 0x55e0f9832bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc1246d640 1 -- 172.21.15.103:0/2195271740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f982aaf0 msgr2=0x55e0f9832bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc1246d640 1 --2- 172.21.15.103:0/2195271740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f982aaf0 0x55e0f9832bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc1246d640 1 -- 172.21.15.103:0/2195271740 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0f94a2210 con 0x55e0f9835cb0 2024-09-19T06:54:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc1146b640 1 --2- 172.21.15.103:0/2195271740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f982b410 0x55e0f9833100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.373+0000 7fbc11c6c640 1 --2- 172.21.15.103:0/2195271740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f982aaf0 0x55e0f9832bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.374+0000 7fbc1246d640 1 --2- 172.21.15.103:0/2195271740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f9835cb0 0x55e0f9833640 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fbc04011c00 tx=0x7fbc040095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de548a59873f8090 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.374+0000 7fbbf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.374+0000 7fbbf57fa640 1 -- 172.21.15.103:0/2195271740 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04024030 con 0x55e0f9835cb0 2024-09-19T06:54:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.374+0000 7fbbf57fa640 1 -- 172.21.15.103:0/2195271740 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fbc04004b20 con 0x55e0f9835cb0 2024-09-19T06:54:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.374+0000 7fbc16c65740 1 -- 172.21.15.103:0/2195271740 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e0f979ca20 con 0x55e0f9835cb0 2024-09-19T06:54:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.374+0000 7fbbf57fa640 1 -- 172.21.15.103:0/2195271740 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbc04030050 con 0x55e0f9835cb0 2024-09-19T06:54:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.374+0000 7fbc16c65740 10 client.17328 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.374+0000 7fbc16c65740 1 -- 172.21.15.103:0/2195271740 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e0f956e620 con 0x55e0f9835cb0 2024-09-19T06:54:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.374+0000 7fbbf57fa640 1 -- 172.21.15.103:0/2195271740 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e0f979ca20 con 0x55e0f9835cb0 2024-09-19T06:54:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.374+0000 7fbbf57fa640 1 -- 172.21.15.103:0/2195271740 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbc04005440 con 0x55e0f9835cb0 2024-09-19T06:54:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.374+0000 7fbc16c65740 10 client.17328 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.374+0000 7fbc16c65740 10 client.17328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.374+0000 7fbc16c65740 20 client.17328 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.374+0000 7fbc16c65740 1 --2- 172.21.15.103:0/2195271740 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e0f9845b70 0x55e0f9848000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.374+0000 7fbc16c65740 4 client.17328 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.374+0000 7fbc16c65740 1 -- 172.21.15.103:0/2195271740 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0f9796d70 con 0x55e0f9845b70 2024-09-19T06:54:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.375+0000 7fbc11c6c640 1 --2- 172.21.15.103:0/2195271740 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e0f9845b70 0x55e0f9848000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.375+0000 7fbbf57fa640 10 client.17328 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.375+0000 7fbc11c6c640 1 --2- 172.21.15.103:0/2195271740 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e0f9845b70 0x55e0f9848000 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a4299990bdb03ff server_cookie=107f874f871b1e0e in_seq=0 out_seq=0 2024-09-19T06:54:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.376+0000 7fbbf57fa640 1 -- 172.21.15.103:0/2195271740 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0f9796d70 con 0x55e0f9845b70 2024-09-19T06:54:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.376+0000 7fbbf57fa640 10 client.17328 handle_command_reply: tid=0 2024-09-19T06:54:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.376+0000 7fbc16c65740 1 client.17328 shutdown 2024-09-19T06:54:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.376+0000 7fbc16c65740 1 -- 172.21.15.103:0/2195271740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f9835cb0 msgr2=0x55e0f9833640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.376+0000 7fbc16c65740 1 --2- 172.21.15.103:0/2195271740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f9835cb0 0x55e0f9833640 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fbc04011c00 tx=0x7fbc040095a0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.376+0000 7fbc16c65740 1 -- 172.21.15.103:0/2195271740 shutdown_connections 2024-09-19T06:54:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.376+0000 7fbc16c65740 1 --2- 172.21.15.103:0/2195271740 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e0f9845b70 0x55e0f9848000 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.376+0000 7fbc16c65740 1 --2- 172.21.15.103:0/2195271740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f9835cb0 0x55e0f9833640 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.376+0000 7fbc16c65740 1 --2- 172.21.15.103:0/2195271740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f982b410 0x55e0f9833100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.376+0000 7fbbf57fa640 0 client.17328 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.376+0000 7fbc16c65740 1 --2- 172.21.15.103:0/2195271740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f982aaf0 0x55e0f9832bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.376+0000 7fbc16c65740 1 -- 172.21.15.103:0/2195271740 >> 172.21.15.103:0/2195271740 conn(0x55e0f9827060 msgr2=0x55e0f982cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.376+0000 7fbc16c65740 1 -- 172.21.15.103:0/2195271740 shutdown_connections 2024-09-19T06:54:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.376+0000 7fbc16c65740 1 -- 172.21.15.103:0/2195271740 wait complete. 2024-09-19T06:54:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.376+0000 7fbc16c65740 20 client.17328 trim_cache size 0 max 16384 2024-09-19T06:54:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.390+0000 7fbc16c65740 1 Processor -- start 2024-09-19T06:54:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.390+0000 7fbc16c65740 1 -- start start 2024-09-19T06:54:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbc16c65740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f982aaf0 0x55e0f9833470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbc16c65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f982b410 0x55e0f984d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbc16c65740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f9835cb0 0x55e0f984de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbc16c65740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c1ad7c0 con 0x55e0f982aaf0 2024-09-19T06:54:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbc16c65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c1af300 con 0x55e0f982b410 2024-09-19T06:54:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbc16c65740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c1ade90 con 0x55e0f9835cb0 2024-09-19T06:54:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbc11c6c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f982aaf0 0x55e0f9833470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbc11c6c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f982aaf0 0x55e0f9833470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54776/0 (socket says 172.21.15.103:54776) 2024-09-19T06:54:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbc11c6c640 1 -- 172.21.15.103:0/2403278928 learned_addr learned my addr 172.21.15.103:0/2403278928 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbc11c6c640 1 -- 172.21.15.103:0/2403278928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f9835cb0 msgr2=0x55e0f984de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbc11c6c640 1 --2- 172.21.15.103:0/2403278928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f9835cb0 0x55e0f984de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbc11c6c640 1 -- 172.21.15.103:0/2403278928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f982b410 msgr2=0x55e0f984d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbc11c6c640 1 --2- 172.21.15.103:0/2403278928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f982b410 0x55e0f984d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbc11c6c640 1 -- 172.21.15.103:0/2403278928 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0f98212f0 con 0x55e0f982aaf0 2024-09-19T06:54:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbc11c6c640 1 --2- 172.21.15.103:0/2403278928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f982aaf0 0x55e0f9833470 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009930 tx=0x7fbc00000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ba8abe1aebaff0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbbf77fe640 1 -- 172.21.15.103:0/2403278928 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00035040 con 0x55e0f982aaf0 2024-09-19T06:54:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbbf77fe640 1 -- 172.21.15.103:0/2403278928 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc00049070 con 0x55e0f982aaf0 2024-09-19T06:54:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.391+0000 7fbbf77fe640 1 -- 172.21.15.103:0/2403278928 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00004e10 con 0x55e0f982aaf0 2024-09-19T06:54:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.392+0000 7fbc16c65740 1 -- 172.21.15.103:0/2403278928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f982aaf0 msgr2=0x55e0f9833470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.392+0000 7fbc16c65740 1 --2- 172.21.15.103:0/2403278928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f982aaf0 0x55e0f9833470 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009930 tx=0x7fbc00000bc0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.392+0000 7fbc16c65740 1 -- 172.21.15.103:0/2403278928 shutdown_connections 2024-09-19T06:54:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.392+0000 7fbc16c65740 1 --2- 172.21.15.103:0/2403278928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f9835cb0 0x55e0f984de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.392+0000 7fbc16c65740 1 --2- 172.21.15.103:0/2403278928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f982b410 0x55e0f984d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.392+0000 7fbc16c65740 1 --2- 172.21.15.103:0/2403278928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f982aaf0 0x55e0f9833470 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.392+0000 7fbc16c65740 1 -- 172.21.15.103:0/2403278928 >> 172.21.15.103:0/2403278928 conn(0x55e0f9827060 msgr2=0x55e0f9834c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.392+0000 7fbc16c65740 1 -- 172.21.15.103:0/2403278928 shutdown_connections 2024-09-19T06:54:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.392+0000 7fbc16c65740 1 -- 172.21.15.103:0/2403278928 wait complete. 2024-09-19T06:54:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.392+0000 7fbc16c65740 1 -- start start 2024-09-19T06:54:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.392+0000 7fbc16c65740 1 Processor -- start 2024-09-19T06:54:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc16c65740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f982aaf0 0x55e0f983e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc16c65740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f982b410 0x55e0f9839610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc16c65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f9835cb0 0x55e0f9839b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc16c65740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c1aec30 con 0x55e0f982aaf0 2024-09-19T06:54:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc16c65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c1ae560 con 0x55e0f9835cb0 2024-09-19T06:54:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc16c65740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c1ab780 con 0x55e0f982b410 2024-09-19T06:54:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc16c65740 10 client.0 _ll_register_callbacks cb 0x7fbc040094c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc11c6c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f982aaf0 0x55e0f983e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc11c6c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f982aaf0 0x55e0f983e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54778/0 (socket says 172.21.15.103:54778) 2024-09-19T06:54:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc11c6c640 1 -- 172.21.15.103:0/916410317 learned_addr learned my addr 172.21.15.103:0/916410317 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc1146b640 1 --2- 172.21.15.103:0/916410317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f982b410 0x55e0f9839610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc1246d640 1 --2- 172.21.15.103:0/916410317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f9835cb0 0x55e0f9839b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc11c6c640 1 -- 172.21.15.103:0/916410317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f982b410 msgr2=0x55e0f9839610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc11c6c640 1 --2- 172.21.15.103:0/916410317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f982b410 0x55e0f9839610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc11c6c640 1 -- 172.21.15.103:0/916410317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f9835cb0 msgr2=0x55e0f9839b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc11c6c640 1 --2- 172.21.15.103:0/916410317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f9835cb0 0x55e0f9839b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc11c6c640 1 -- 172.21.15.103:0/916410317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0f983a360 con 0x55e0f982aaf0 2024-09-19T06:54:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc11c6c640 1 --2- 172.21.15.103:0/916410317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f982aaf0 0x55e0f983e2f0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fbc000047b0 tx=0x7fbc000047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c3bf2be96b1d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbbf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbbf57fa640 1 -- 172.21.15.103:0/916410317 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00035040 con 0x55e0f982aaf0 2024-09-19T06:54:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbbf57fa640 1 -- 172.21.15.103:0/916410317 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fbc00031880 con 0x55e0f982aaf0 2024-09-19T06:54:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc16c65740 1 -- 172.21.15.103:0/916410317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc00049070 con 0x55e0f982aaf0 2024-09-19T06:54:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbbf57fa640 1 -- 172.21.15.103:0/916410317 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbc00031ac0 con 0x55e0f982aaf0 2024-09-19T06:54:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc16c65740 10 client.17334 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.393+0000 7fbc16c65740 1 -- 172.21.15.103:0/916410317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e0f98b0860 con 0x55e0f982aaf0 2024-09-19T06:54:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.394+0000 7fbbf57fa640 1 -- 172.21.15.103:0/916410317 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc00049070 con 0x55e0f982aaf0 2024-09-19T06:54:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.394+0000 7fbbf57fa640 1 -- 172.21.15.103:0/916410317 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbc0004d4e0 con 0x55e0f982aaf0 2024-09-19T06:54:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.394+0000 7fbc16c65740 10 client.17334 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.394+0000 7fbc16c65740 10 client.17334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.394+0000 7fbc16c65740 20 client.17334 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.394+0000 7fbc16c65740 1 --2- 172.21.15.103:0/916410317 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e0f98b0f70 0x55e0f98b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.394+0000 7fbc16c65740 4 client.17334 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:54:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.394+0000 7fbc16c65740 1 -- 172.21.15.103:0/916410317 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0f9713900 con 0x55e0f98b0f70 2024-09-19T06:54:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.394+0000 7fbc1146b640 1 --2- 172.21.15.103:0/916410317 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e0f98b0f70 0x55e0f98b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.395+0000 7fbbf57fa640 10 client.17334 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.395+0000 7fbc1146b640 1 --2- 172.21.15.103:0/916410317 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e0f98b0f70 0x55e0f98b3390 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81938665726cd8a6 server_cookie=31aac4e53e1f1116 in_seq=0 out_seq=0 2024-09-19T06:54:14.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.404+0000 7fbbf57fa640 1 -- 172.21.15.103:0/916410317 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e0f9713900 con 0x55e0f98b0f70 2024-09-19T06:54:14.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.404+0000 7fbbf57fa640 10 client.17334 handle_command_reply: tid=0 2024-09-19T06:54:14.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.404+0000 7fbc16c65740 1 client.17334 shutdown 2024-09-19T06:54:14.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.404+0000 7fbc16c65740 1 -- 172.21.15.103:0/916410317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f982aaf0 msgr2=0x55e0f983e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.404+0000 7fbc16c65740 1 --2- 172.21.15.103:0/916410317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f982aaf0 0x55e0f983e2f0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fbc000047b0 tx=0x7fbc000047e0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.404+0000 7fbc16c65740 1 -- 172.21.15.103:0/916410317 shutdown_connections 2024-09-19T06:54:14.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.404+0000 7fbc16c65740 1 --2- 172.21.15.103:0/916410317 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e0f98b0f70 0x55e0f98b3390 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.404+0000 7fbc16c65740 1 --2- 172.21.15.103:0/916410317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f9835cb0 0x55e0f9839b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.404+0000 7fbbf57fa640 0 client.17334 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:14.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.404+0000 7fbc16c65740 1 --2- 172.21.15.103:0/916410317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0f982b410 0x55e0f9839610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.404+0000 7fbc16c65740 1 --2- 172.21.15.103:0/916410317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0f982aaf0 0x55e0f983e2f0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.404+0000 7fbc16c65740 1 -- 172.21.15.103:0/916410317 >> 172.21.15.103:0/916410317 conn(0x55e0f9827060 msgr2=0x55e0f98259a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.404+0000 7fbc16c65740 1 -- 172.21.15.103:0/916410317 shutdown_connections 2024-09-19T06:54:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.405+0000 7fbc16c65740 1 -- 172.21.15.103:0/916410317 wait complete. 2024-09-19T06:54:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.405+0000 7fbc16c65740 20 client.17334 trim_cache size 0 max 16384 2024-09-19T06:54:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:54:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:54:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:54:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:54:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.405+0000 7fbc13ed0640 1 -- 172.21.15.103:0/763118779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbbf80a3460 msgr2=0x7fbbf80a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.405+0000 7fbc13ed0640 1 --2- 172.21.15.103:0/763118779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbbf80a3460 0x7fbbf80a5920 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009fd0 tx=0x7fbc04009820 comp rx=0 tx=0).stop 2024-09-19T06:54:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.405+0000 7fbc13ed0640 1 -- 172.21.15.103:0/763118779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc0c101350 msgr2=0x7fbc0c1996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.405+0000 7fbc13ed0640 1 --2- 172.21.15.103:0/763118779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc0c101350 0x7fbc0c1996e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800d950 tx=0x7fbc0800de10 comp rx=0 tx=0).stop 2024-09-19T06:54:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.405+0000 7fbc13ed0640 1 -- 172.21.15.103:0/763118779 shutdown_connections 2024-09-19T06:54:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.405+0000 7fbc13ed0640 1 --2- 172.21.15.103:0/763118779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbbf80a3460 0x7fbbf80a5920 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.405+0000 7fbc13ed0640 1 --2- 172.21.15.103:0/763118779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc0c1069c0 0x7fbc0c1a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.405+0000 7fbc13ed0640 1 --2- 172.21.15.103:0/763118779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc0c101350 0x7fbc0c1996e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.405+0000 7fbc13ed0640 1 --2- 172.21.15.103:0/763118779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc0c1009b0 0x7fbc0c1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.405+0000 7fbc13ed0640 1 -- 172.21.15.103:0/763118779 >> 172.21.15.103:0/763118779 conn(0x7fbc0c077990 msgr2=0x7fbc0c103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.405+0000 7fbc13ed0640 1 -- 172.21.15.103:0/763118779 shutdown_connections 2024-09-19T06:54:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:14.405+0000 7fbc13ed0640 1 -- 172.21.15.103:0/763118779 wait complete. 2024-09-19T06:54:14.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:14 smithi134 ceph-mon[35356]: pgmap v153: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 KiB/s wr, 28 op/s 2024-09-19T06:54:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:16 smithi174 ceph-mon[35637]: pgmap v154: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 41 KiB/s wr, 34 op/s 2024-09-19T06:54:16.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:16 smithi103 ceph-mon[31158]: pgmap v154: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 41 KiB/s wr, 34 op/s 2024-09-19T06:54:16.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:16 smithi134 ceph-mon[35356]: pgmap v154: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 41 KiB/s wr, 34 op/s 2024-09-19T06:54:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:18 smithi174 ceph-mon[35637]: pgmap v155: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-19T06:54:18.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:18 smithi103 ceph-mon[31158]: pgmap v155: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-19T06:54:18.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:18 smithi134 ceph-mon[35356]: pgmap v155: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-19T06:54:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:20 smithi174 ceph-mon[35637]: pgmap v156: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-19T06:54:20.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:20 smithi103 ceph-mon[31158]: pgmap v156: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-19T06:54:20.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:20 smithi134 ceph-mon[35356]: pgmap v156: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-19T06:54:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:22 smithi174 ceph-mon[35637]: pgmap v157: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:54:22.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:22 smithi103 ceph-mon[31158]: pgmap v157: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:54:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:22 smithi134 ceph-mon[35356]: pgmap v157: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:54:24.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:24 smithi174 ceph-mon[35637]: pgmap v158: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:54:24.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:24 smithi103 ceph-mon[31158]: pgmap v158: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:54:24.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:24 smithi134 ceph-mon[35356]: pgmap v158: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:54:26.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:26 smithi174 ceph-mon[35637]: pgmap v159: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-09-19T06:54:26.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:26 smithi103 ceph-mon[31158]: pgmap v159: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-09-19T06:54:26.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:26 smithi134 ceph-mon[35356]: pgmap v159: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-09-19T06:54:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:27 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:54:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:27 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:54:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:27 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:54:27.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:27 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:54:27.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:27 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:54:27.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:27 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:54:27.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:27 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:54:27.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:27 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:54:27.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:27 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:54:27.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:27 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:54:27.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:27 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:54:27.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:27 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:54:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.492+0000 7fd86c642640 1 -- 172.21.15.103:0/3081471904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8641009d0 msgr2=0x7fd864100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.492+0000 7fd86c642640 1 --2- 172.21.15.103:0/3081471904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8641009d0 0x7fd864100e30 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd860009930 tx=0x7fd86002a420 comp rx=0 tx=0).stop 2024-09-19T06:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.492+0000 7fd86c642640 1 -- 172.21.15.103:0/3081471904 shutdown_connections 2024-09-19T06:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.492+0000 7fd86c642640 1 --2- 172.21.15.103:0/3081471904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd864101370 0x7fd86410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.492+0000 7fd86c642640 1 --2- 172.21.15.103:0/3081471904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8641009d0 0x7fd864100e30 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.492+0000 7fd86c642640 1 --2- 172.21.15.103:0/3081471904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8641069e0 0x7fd864106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.492+0000 7fd86c642640 1 -- 172.21.15.103:0/3081471904 >> 172.21.15.103:0/3081471904 conn(0x7fd8640fc6f0 msgr2=0x7fd8640feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.492+0000 7fd86c642640 1 -- 172.21.15.103:0/3081471904 shutdown_connections 2024-09-19T06:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.492+0000 7fd86c642640 1 -- 172.21.15.103:0/3081471904 wait complete. 2024-09-19T06:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.493+0000 7fd86c642640 1 Processor -- start 2024-09-19T06:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.493+0000 7fd86c642640 1 -- start start 2024-09-19T06:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.493+0000 7fd86c642640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8641009d0 0x7fd8641983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.493+0000 7fd86c642640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd864101370 0x7fd864198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.493+0000 7fd86c642640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8641069e0 0x7fd86419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.493+0000 7fd86c642640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd864111f30 con 0x7fd8641009d0 2024-09-19T06:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.493+0000 7fd86c642640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd864111db0 con 0x7fd864101370 2024-09-19T06:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.493+0000 7fd86c642640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8641120b0 con 0x7fd8641069e0 2024-09-19T06:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.493+0000 7fd86a3de640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8641009d0 0x7fd8641983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.493+0000 7fd86a3de640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8641009d0 0x7fd8641983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49150/0 (socket says 172.21.15.103:49150) 2024-09-19T06:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.493+0000 7fd86a3de640 1 -- 172.21.15.103:0/694132355 learned_addr learned my addr 172.21.15.103:0/694132355 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.493+0000 7fd86abdf640 1 --2- 172.21.15.103:0/694132355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8641069e0 0x7fd86419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.493+0000 7fd869bdd640 1 --2- 172.21.15.103:0/694132355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd864101370 0x7fd864198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.494+0000 7fd86a3de640 1 -- 172.21.15.103:0/694132355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8641069e0 msgr2=0x7fd86419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.494+0000 7fd86a3de640 1 --2- 172.21.15.103:0/694132355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8641069e0 0x7fd86419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.494+0000 7fd86a3de640 1 -- 172.21.15.103:0/694132355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd864101370 msgr2=0x7fd864198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.494+0000 7fd86a3de640 1 --2- 172.21.15.103:0/694132355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd864101370 0x7fd864198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.494+0000 7fd86a3de640 1 -- 172.21.15.103:0/694132355 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86419d3d0 con 0x7fd8641009d0 2024-09-19T06:54:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.494+0000 7fd869bdd640 1 --2- 172.21.15.103:0/694132355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd864101370 0x7fd864198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.494+0000 7fd86a3de640 1 --2- 172.21.15.103:0/694132355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8641009d0 0x7fd8641983e0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd85800b4a0 tx=0x7fd85800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.494+0000 7fd8577fe640 1 -- 172.21.15.103:0/694132355 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858014070 con 0x7fd8641009d0 2024-09-19T06:54:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.494+0000 7fd8577fe640 1 -- 172.21.15.103:0/694132355 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd858010af0 con 0x7fd8641009d0 2024-09-19T06:54:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.494+0000 7fd86c642640 1 -- 172.21.15.103:0/694132355 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86419d6c0 con 0x7fd8641009d0 2024-09-19T06:54:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.494+0000 7fd8577fe640 1 -- 172.21.15.103:0/694132355 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858017d10 con 0x7fd8641009d0 2024-09-19T06:54:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.494+0000 7fd86c642640 1 -- 172.21.15.103:0/694132355 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8641a4f50 con 0x7fd8641009d0 2024-09-19T06:54:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.496+0000 7fd8577fe640 1 -- 172.21.15.103:0/694132355 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd858017070 con 0x7fd8641009d0 2024-09-19T06:54:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.496+0000 7fd86f3d3740 1 Processor -- start 2024-09-19T06:54:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd86f3d3740 1 -- start start 2024-09-19T06:54:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd86f3d3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d56af0 0x559f81d56ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd86f3d3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d61770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd86f3d3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 0x559f81d640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd86f3d3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559f81cbb2f0 con 0x559f81d57410 2024-09-19T06:54:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd86f3d3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f81c63270 con 0x559f81d56af0 2024-09-19T06:54:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd86f3d3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559f81cb72b0 con 0x559f81d61cb0 2024-09-19T06:54:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd86a3de640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d61770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd8577fe640 1 --2- 172.21.15.103:0/694132355 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd8500a34e0 0x7fd8500a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd86a3de640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d61770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49158/0 (socket says 172.21.15.103:49158) 2024-09-19T06:54:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd86a3de640 1 -- 172.21.15.103:0/714635169 learned_addr learned my addr 172.21.15.103:0/714635169 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd869bdd640 1 --2- 172.21.15.103:0/714635169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 0x559f81d640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd86abdf640 1 --2- 172.21.15.103:0/694132355 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd8500a34e0 0x7fd8500a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd86abdf640 1 --2- 172.21.15.103:0/714635169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d56af0 0x559f81d56ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd8577fe640 1 -- 172.21.15.103:0/694132355 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd8580ceef0 con 0x7fd8641009d0 2024-09-19T06:54:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd869bdd640 1 -- 172.21.15.103:0/714635169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d56af0 msgr2=0x559f81d56ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd869bdd640 1 --2- 172.21.15.103:0/714635169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d56af0 0x559f81d56ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd869bdd640 1 -- 172.21.15.103:0/714635169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 msgr2=0x559f81d61770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd869bdd640 1 --2- 172.21.15.103:0/714635169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d61770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd869bdd640 1 -- 172.21.15.103:0/714635169 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f81c62340 con 0x559f81d61cb0 2024-09-19T06:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.497+0000 7fd86a3de640 1 --2- 172.21.15.103:0/714635169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d61770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.498+0000 7fd869bdd640 1 --2- 172.21.15.103:0/714635169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 0x559f81d640a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd86002a940 tx=0x7fd860009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcc6456646d6f614 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.498+0000 7fd86abdf640 1 --2- 172.21.15.103:0/694132355 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd8500a34e0 0x7fd8500a59a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0098b0 tx=0x7fd85c009840 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.498+0000 7fd854bfe640 1 -- 172.21.15.103:0/714635169 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86003f070 con 0x559f81d61cb0 2024-09-19T06:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.498+0000 7fd854bfe640 1 -- 172.21.15.103:0/714635169 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8600317b0 con 0x559f81d61cb0 2024-09-19T06:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.498+0000 7fd854bfe640 1 -- 172.21.15.103:0/714635169 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860035070 con 0x559f81d61cb0 2024-09-19T06:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 -- 172.21.15.103:0/714635169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 msgr2=0x559f81d640a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/714635169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 0x559f81d640a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd86002a940 tx=0x7fd860009680 comp rx=0 tx=0).stop 2024-09-19T06:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 -- 172.21.15.103:0/714635169 shutdown_connections 2024-09-19T06:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/714635169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 0x559f81d640a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/714635169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d61770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/714635169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d56af0 0x559f81d56ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 -- 172.21.15.103:0/714635169 >> 172.21.15.103:0/714635169 conn(0x559f81d53060 msgr2=0x559f81d53440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 -- 172.21.15.103:0/714635169 shutdown_connections 2024-09-19T06:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 -- 172.21.15.103:0/714635169 wait complete. 2024-09-19T06:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 -- start start 2024-09-19T06:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 Processor -- start 2024-09-19T06:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d56af0 0x559f81d5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d5f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d61cb0 0x559f81d5f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559f81d64880 con 0x559f81d57410 2024-09-19T06:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f81c44290 con 0x559f81d61cb0 2024-09-19T06:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559f81d64a00 con 0x559f81d56af0 2024-09-19T06:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86f3d3740 10 client.0 _ll_register_callbacks cb 0x559f81c75140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd869bdd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d5f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd869bdd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d5f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49160/0 (socket says 172.21.15.103:49160) 2024-09-19T06:54:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd869bdd640 1 -- 172.21.15.103:0/3892289373 learned_addr learned my addr 172.21.15.103:0/3892289373 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.499+0000 7fd86a3de640 1 --2- 172.21.15.103:0/3892289373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d56af0 0x559f81d5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd86abdf640 1 --2- 172.21.15.103:0/3892289373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d61cb0 0x559f81d5f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd869bdd640 1 -- 172.21.15.103:0/3892289373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d56af0 msgr2=0x559f81d5eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd869bdd640 1 --2- 172.21.15.103:0/3892289373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d56af0 0x559f81d5eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd869bdd640 1 -- 172.21.15.103:0/3892289373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d61cb0 msgr2=0x559f81d5f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd869bdd640 1 --2- 172.21.15.103:0/3892289373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d61cb0 0x559f81d5f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd869bdd640 1 -- 172.21.15.103:0/3892289373 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f819ce210 con 0x559f81d57410 2024-09-19T06:54:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd86abdf640 1 --2- 172.21.15.103:0/3892289373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d61cb0 0x559f81d5f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd869bdd640 1 --2- 172.21.15.103:0/3892289373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d5f040 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fd86002ab50 tx=0x7fd86002ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4dff8ccc51146ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd84dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd84dffb640 1 -- 172.21.15.103:0/3892289373 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86003f030 con 0x559f81d57410 2024-09-19T06:54:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd84dffb640 1 -- 172.21.15.103:0/3892289373 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd860004250 con 0x559f81d57410 2024-09-19T06:54:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd86f3d3740 1 -- 172.21.15.103:0/3892289373 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8600317b0 con 0x559f81d57410 2024-09-19T06:54:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd84dffb640 1 -- 172.21.15.103:0/3892289373 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd860054050 con 0x559f81d57410 2024-09-19T06:54:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd86f3d3740 10 client.17346 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd86f3d3740 1 -- 172.21.15.103:0/3892289373 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f81a9a620 con 0x559f81d57410 2024-09-19T06:54:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd84dffb640 1 -- 172.21.15.103:0/3892289373 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8600317b0 con 0x559f81d57410 2024-09-19T06:54:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.500+0000 7fd84dffb640 1 -- 172.21.15.103:0/3892289373 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd86003b540 con 0x559f81d57410 2024-09-19T06:54:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.501+0000 7fd86f3d3740 10 client.17346 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.501+0000 7fd86f3d3740 10 client.17346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.501+0000 7fd86f3d3740 20 client.17346 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.501+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/3892289373 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559f81d71a00 0x559f81d73df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.501+0000 7fd86f3d3740 4 client.17346 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.501+0000 7fd86f3d3740 1 -- 172.21.15.103:0/3892289373 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f81cc8a20 con 0x559f81d71a00 2024-09-19T06:54:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.501+0000 7fd86a3de640 1 --2- 172.21.15.103:0/3892289373 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559f81d71a00 0x559f81d73df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.501+0000 7fd84dffb640 10 client.17346 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.501+0000 7fd86a3de640 1 --2- 172.21.15.103:0/3892289373 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559f81d71a00 0x559f81d73df0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83892aa3367a0e95 server_cookie=d00d45b4ee852b38 in_seq=0 out_seq=0 2024-09-19T06:54:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.502+0000 7fd84dffb640 1 -- 172.21.15.103:0/3892289373 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f81cc8a20 con 0x559f81d71a00 2024-09-19T06:54:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.502+0000 7fd84dffb640 10 client.17346 handle_command_reply: tid=0 2024-09-19T06:54:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.502+0000 7fd86f3d3740 1 client.17346 shutdown 2024-09-19T06:54:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.502+0000 7fd86f3d3740 1 -- 172.21.15.103:0/3892289373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 msgr2=0x559f81d5f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.502+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/3892289373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d5f040 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fd86002ab50 tx=0x7fd86002ab80 comp rx=0 tx=0).stop 2024-09-19T06:54:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.502+0000 7fd86f3d3740 1 -- 172.21.15.103:0/3892289373 shutdown_connections 2024-09-19T06:54:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.502+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/3892289373 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559f81d71a00 0x559f81d73df0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.502+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/3892289373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d61cb0 0x559f81d5f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.502+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/3892289373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d5f040 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.502+0000 7fd84dffb640 0 client.17346 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.502+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/3892289373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d56af0 0x559f81d5eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.502+0000 7fd86f3d3740 1 -- 172.21.15.103:0/3892289373 >> 172.21.15.103:0/3892289373 conn(0x559f81d53060 msgr2=0x559f81d58dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.502+0000 7fd86f3d3740 1 -- 172.21.15.103:0/3892289373 shutdown_connections 2024-09-19T06:54:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.502+0000 7fd86f3d3740 1 -- 172.21.15.103:0/3892289373 wait complete. 2024-09-19T06:54:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.502+0000 7fd86f3d3740 20 client.17346 trim_cache size 0 max 16384 2024-09-19T06:54:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.516+0000 7fd86f3d3740 1 Processor -- start 2024-09-19T06:54:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.516+0000 7fd86f3d3740 1 -- start start 2024-09-19T06:54:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.516+0000 7fd86f3d3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d56af0 0x559f81d59560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.516+0000 7fd86f3d3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d57410 0x559f81d5f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.516+0000 7fd86f3d3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 0x559f81d79510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.516+0000 7fd86f3d3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8641ad8b0 con 0x559f81d56af0 2024-09-19T06:54:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.516+0000 7fd86f3d3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8641af3f0 con 0x559f81d57410 2024-09-19T06:54:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.516+0000 7fd86f3d3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8641adf80 con 0x559f81d61cb0 2024-09-19T06:54:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.516+0000 7fd86a3de640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d56af0 0x559f81d59560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.516+0000 7fd86a3de640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d56af0 0x559f81d59560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49170/0 (socket says 172.21.15.103:49170) 2024-09-19T06:54:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.516+0000 7fd86a3de640 1 -- 172.21.15.103:0/2576586064 learned_addr learned my addr 172.21.15.103:0/2576586064 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.517+0000 7fd869bdd640 1 --2- 172.21.15.103:0/2576586064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d57410 0x559f81d5f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.517+0000 7fd86abdf640 1 --2- 172.21.15.103:0/2576586064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 0x559f81d79510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.517+0000 7fd86a3de640 1 -- 172.21.15.103:0/2576586064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 msgr2=0x559f81d79510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.517+0000 7fd86a3de640 1 --2- 172.21.15.103:0/2576586064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 0x559f81d79510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.517+0000 7fd86a3de640 1 -- 172.21.15.103:0/2576586064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d57410 msgr2=0x559f81d5f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.517+0000 7fd86a3de640 1 --2- 172.21.15.103:0/2576586064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d57410 0x559f81d5f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.517+0000 7fd86a3de640 1 -- 172.21.15.103:0/2576586064 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f81d52e90 con 0x559f81d56af0 2024-09-19T06:54:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.517+0000 7fd869bdd640 1 --2- 172.21.15.103:0/2576586064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d57410 0x559f81d5f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.517+0000 7fd86abdf640 1 --2- 172.21.15.103:0/2576586064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 0x559f81d79510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.517+0000 7fd86a3de640 1 --2- 172.21.15.103:0/2576586064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d56af0 0x559f81d59560 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd85800be60 tx=0x7fd85809e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242f27a896ed589f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.517+0000 7fd84ffff640 1 -- 172.21.15.103:0/2576586064 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85801e040 con 0x559f81d56af0 2024-09-19T06:54:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.517+0000 7fd84ffff640 1 -- 172.21.15.103:0/2576586064 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd85809b2f0 con 0x559f81d56af0 2024-09-19T06:54:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.517+0000 7fd84ffff640 1 -- 172.21.15.103:0/2576586064 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858010dc0 con 0x559f81d56af0 2024-09-19T06:54:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 -- 172.21.15.103:0/2576586064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d56af0 msgr2=0x559f81d59560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/2576586064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d56af0 0x559f81d59560 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd85800be60 tx=0x7fd85809e040 comp rx=0 tx=0).stop 2024-09-19T06:54:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 -- 172.21.15.103:0/2576586064 shutdown_connections 2024-09-19T06:54:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/2576586064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 0x559f81d79510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/2576586064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d57410 0x559f81d5f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/2576586064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d56af0 0x559f81d59560 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 -- 172.21.15.103:0/2576586064 >> 172.21.15.103:0/2576586064 conn(0x559f81d53060 msgr2=0x559f81d60b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 -- 172.21.15.103:0/2576586064 shutdown_connections 2024-09-19T06:54:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 -- 172.21.15.103:0/2576586064 wait complete. 2024-09-19T06:54:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 -- start start 2024-09-19T06:54:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 Processor -- start 2024-09-19T06:54:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d56af0 0x559f81d65c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d661d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 0x559f81d66710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8641aed20 con 0x559f81d57410 2024-09-19T06:54:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8641ae650 con 0x559f81d56af0 2024-09-19T06:54:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8641ab870 con 0x559f81d61cb0 2024-09-19T06:54:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86f3d3740 10 client.0 _ll_register_callbacks cb 0x7fd85809b0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd869bdd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd869bdd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d661d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49186/0 (socket says 172.21.15.103:49186) 2024-09-19T06:54:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd869bdd640 1 -- 172.21.15.103:0/883764750 learned_addr learned my addr 172.21.15.103:0/883764750 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.518+0000 7fd86a3de640 1 --2- 172.21.15.103:0/883764750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d56af0 0x559f81d65c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd869bdd640 1 -- 172.21.15.103:0/883764750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 msgr2=0x559f81d66710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd86abdf640 1 --2- 172.21.15.103:0/883764750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 0x559f81d66710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd869bdd640 1 --2- 172.21.15.103:0/883764750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 0x559f81d66710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd869bdd640 1 -- 172.21.15.103:0/883764750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d56af0 msgr2=0x559f81d65c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd869bdd640 1 --2- 172.21.15.103:0/883764750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d56af0 0x559f81d65c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd869bdd640 1 -- 172.21.15.103:0/883764750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f81d4e2d0 con 0x559f81d57410 2024-09-19T06:54:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd86abdf640 1 --2- 172.21.15.103:0/883764750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 0x559f81d66710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd869bdd640 1 --2- 172.21.15.103:0/883764750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d661d0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x559f81d60140 tx=0x7fd8600058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd8c29a00407434 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd84dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd84dffb640 1 -- 172.21.15.103:0/883764750 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86005d060 con 0x559f81d57410 2024-09-19T06:54:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd84dffb640 1 -- 172.21.15.103:0/883764750 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd86003b460 con 0x559f81d57410 2024-09-19T06:54:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd86f3d3740 1 -- 172.21.15.103:0/883764750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd85809b2f0 con 0x559f81d57410 2024-09-19T06:54:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd84dffb640 1 -- 172.21.15.103:0/883764750 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd86003f040 con 0x559f81d57410 2024-09-19T06:54:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd86f3d3740 10 client.17352 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd86f3d3740 1 -- 172.21.15.103:0/883764750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f81d53cf0 con 0x559f81d57410 2024-09-19T06:54:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd84dffb640 1 -- 172.21.15.103:0/883764750 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd85809b2f0 con 0x559f81d57410 2024-09-19T06:54:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd84dffb640 1 -- 172.21.15.103:0/883764750 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd860044930 con 0x559f81d57410 2024-09-19T06:54:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd86f3d3740 10 client.17352 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd86f3d3740 10 client.17352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd86f3d3740 20 client.17352 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/883764750 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559f81ddcd60 0x559f81ddf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd86f3d3740 4 client.17352 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:54:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.519+0000 7fd86f3d3740 1 -- 172.21.15.103:0/883764750 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f81cc2d70 con 0x559f81ddcd60 2024-09-19T06:54:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.520+0000 7fd86a3de640 1 --2- 172.21.15.103:0/883764750 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559f81ddcd60 0x559f81ddf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.520+0000 7fd84dffb640 10 client.17352 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.520+0000 7fd86a3de640 1 --2- 172.21.15.103:0/883764750 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559f81ddcd60 0x559f81ddf180 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c829426af8f3795 server_cookie=bb4392dd35b299ce in_seq=0 out_seq=0 2024-09-19T06:54:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.526+0000 7fd84dffb640 1 -- 172.21.15.103:0/883764750 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f81cc2d70 con 0x559f81ddcd60 2024-09-19T06:54:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.526+0000 7fd84dffb640 10 client.17352 handle_command_reply: tid=0 2024-09-19T06:54:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.526+0000 7fd86f3d3740 1 client.17352 shutdown 2024-09-19T06:54:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.526+0000 7fd86f3d3740 1 -- 172.21.15.103:0/883764750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 msgr2=0x559f81d661d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.526+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/883764750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d661d0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x559f81d60140 tx=0x7fd8600058b0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.526+0000 7fd86f3d3740 1 -- 172.21.15.103:0/883764750 shutdown_connections 2024-09-19T06:54:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.526+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/883764750 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559f81ddcd60 0x559f81ddf180 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.526+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/883764750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f81d61cb0 0x559f81d66710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.526+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/883764750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f81d57410 0x559f81d661d0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.526+0000 7fd86f3d3740 1 --2- 172.21.15.103:0/883764750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f81d56af0 0x559f81d65c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.526+0000 7fd84dffb640 0 client.17352 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.526+0000 7fd86f3d3740 1 -- 172.21.15.103:0/883764750 >> 172.21.15.103:0/883764750 conn(0x559f81d53060 msgr2=0x559f81d519a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.526+0000 7fd86f3d3740 1 -- 172.21.15.103:0/883764750 shutdown_connections 2024-09-19T06:54:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.526+0000 7fd86f3d3740 1 -- 172.21.15.103:0/883764750 wait complete. 2024-09-19T06:54:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.527+0000 7fd86f3d3740 20 client.17352 trim_cache size 0 max 16384 2024-09-19T06:54:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:54:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:54:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:54:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:54:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.527+0000 7fd86c642640 1 -- 172.21.15.103:0/694132355 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd8500a34e0 msgr2=0x7fd8500a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.527+0000 7fd86c642640 1 --2- 172.21.15.103:0/694132355 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd8500a34e0 0x7fd8500a59a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0098b0 tx=0x7fd85c009840 comp rx=0 tx=0).stop 2024-09-19T06:54:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.527+0000 7fd86c642640 1 -- 172.21.15.103:0/694132355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8641009d0 msgr2=0x7fd8641983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.527+0000 7fd86c642640 1 --2- 172.21.15.103:0/694132355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8641009d0 0x7fd8641983e0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd85800b4a0 tx=0x7fd85800b960 comp rx=0 tx=0).stop 2024-09-19T06:54:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.527+0000 7fd86c642640 1 -- 172.21.15.103:0/694132355 shutdown_connections 2024-09-19T06:54:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.527+0000 7fd86c642640 1 --2- 172.21.15.103:0/694132355 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd8500a34e0 0x7fd8500a59a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.527+0000 7fd86c642640 1 --2- 172.21.15.103:0/694132355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8641069e0 0x7fd86419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.527+0000 7fd86c642640 1 --2- 172.21.15.103:0/694132355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd864101370 0x7fd864198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.527+0000 7fd86c642640 1 --2- 172.21.15.103:0/694132355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8641009d0 0x7fd8641983e0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.527+0000 7fd86c642640 1 -- 172.21.15.103:0/694132355 >> 172.21.15.103:0/694132355 conn(0x7fd8640fc6f0 msgr2=0x7fd8640fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.527+0000 7fd86c642640 1 -- 172.21.15.103:0/694132355 shutdown_connections 2024-09-19T06:54:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:27.527+0000 7fd86c642640 1 -- 172.21.15.103:0/694132355 wait complete. 2024-09-19T06:54:28.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:28 smithi174 ceph-mon[35637]: pgmap v160: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8 op/s 2024-09-19T06:54:28.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:28 smithi103 ceph-mon[31158]: pgmap v160: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8 op/s 2024-09-19T06:54:28.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:28 smithi134 ceph-mon[35356]: pgmap v160: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8 op/s 2024-09-19T06:54:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:30 smithi174 ceph-mon[35637]: pgmap v161: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-09-19T06:54:30.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:30 smithi103 ceph-mon[31158]: pgmap v161: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-09-19T06:54:30.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:30 smithi134 ceph-mon[35356]: pgmap v161: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-09-19T06:54:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:32 smithi174 ceph-mon[35637]: pgmap v162: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-19T06:54:32.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:32 smithi103 ceph-mon[31158]: pgmap v162: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-19T06:54:32.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:32 smithi134 ceph-mon[35356]: pgmap v162: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-19T06:54:33.355 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:54:33.619 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.044+0000 7f70b35d4640 1 -- 172.21.15.103:0/33171947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70ac106f00 msgr2=0x7f70ac10ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.044+0000 7f70b35d4640 1 --2- 172.21.15.103:0/33171947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70ac106f00 0x7f70ac10ea80 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009ca0 tx=0x7f70a402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.044+0000 7f70b35d4640 1 -- 172.21.15.103:0/33171947 shutdown_connections 2024-09-19T06:54:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.044+0000 7f70b35d4640 1 --2- 172.21.15.103:0/33171947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70ac106f00 0x7f70ac10ea80 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.044+0000 7f70b35d4640 1 --2- 172.21.15.103:0/33171947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70ac102780 0x7f70ac1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.044+0000 7f70b35d4640 1 --2- 172.21.15.103:0/33171947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70ac101dd0 0x7f70ac1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.044+0000 7f70b35d4640 1 -- 172.21.15.103:0/33171947 >> 172.21.15.103:0/33171947 conn(0x7f70ac0fd8b0 msgr2=0x7f70ac0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.044+0000 7f70b35d4640 1 -- 172.21.15.103:0/33171947 shutdown_connections 2024-09-19T06:54:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.044+0000 7f70b35d4640 1 -- 172.21.15.103:0/33171947 wait complete. 2024-09-19T06:54:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.045+0000 7f70b35d4640 1 Processor -- start 2024-09-19T06:54:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.045+0000 7f70b35d4640 1 -- start start 2024-09-19T06:54:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.045+0000 7f70b35d4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70ac101dd0 0x7f70ac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.045+0000 7f70b35d4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70ac102780 0x7f70ac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.045+0000 7f70b35d4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70ac106f00 0x7f70ac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.045+0000 7f70b35d4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac111f80 con 0x7f70ac101dd0 2024-09-19T06:54:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.045+0000 7f70b35d4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac111e00 con 0x7f70ac106f00 2024-09-19T06:54:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.045+0000 7f70b35d4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac112100 con 0x7f70ac102780 2024-09-19T06:54:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.046+0000 7f70b1370640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70ac101dd0 0x7f70ac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.046+0000 7f70b1370640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70ac101dd0 0x7f70ac0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49198/0 (socket says 172.21.15.103:49198) 2024-09-19T06:54:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.046+0000 7f70b1370640 1 -- 172.21.15.103:0/4177212710 learned_addr learned my addr 172.21.15.103:0/4177212710 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.046+0000 7f70b1b71640 1 --2- 172.21.15.103:0/4177212710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70ac106f00 0x7f70ac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.046+0000 7f70b0b6f640 1 --2- 172.21.15.103:0/4177212710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70ac102780 0x7f70ac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.046+0000 7f70b1370640 1 -- 172.21.15.103:0/4177212710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70ac102780 msgr2=0x7f70ac074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.046+0000 7f70b1370640 1 --2- 172.21.15.103:0/4177212710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70ac102780 0x7f70ac074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.046+0000 7f70b1370640 1 -- 172.21.15.103:0/4177212710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70ac106f00 msgr2=0x7f70ac074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.046+0000 7f70b1370640 1 --2- 172.21.15.103:0/4177212710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70ac106f00 0x7f70ac074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.046+0000 7f70b1370640 1 -- 172.21.15.103:0/4177212710 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac1a6920 con 0x7f70ac101dd0 2024-09-19T06:54:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.046+0000 7f70b1370640 1 --2- 172.21.15.103:0/4177212710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70ac101dd0 0x7f70ac0744f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f70a800c890 tx=0x7f70a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.046+0000 7f70a27fc640 1 -- 172.21.15.103:0/4177212710 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8004220 con 0x7f70ac101dd0 2024-09-19T06:54:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.046+0000 7f70a27fc640 1 -- 172.21.15.103:0/4177212710 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a80043c0 con 0x7f70ac101dd0 2024-09-19T06:54:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.046+0000 7f70b35d4640 1 -- 172.21.15.103:0/4177212710 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ac1a6c10 con 0x7f70ac101dd0 2024-09-19T06:54:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.046+0000 7f70a27fc640 1 -- 172.21.15.103:0/4177212710 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a80049e0 con 0x7f70ac101dd0 2024-09-19T06:54:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.046+0000 7f70b35d4640 1 -- 172.21.15.103:0/4177212710 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ac1a7150 con 0x7f70ac101dd0 2024-09-19T06:54:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.049+0000 7f70a27fc640 1 -- 172.21.15.103:0/4177212710 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f70a8004c40 con 0x7f70ac101dd0 2024-09-19T06:54:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.049+0000 7f70b59ad740 1 Processor -- start 2024-09-19T06:54:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.049+0000 7f70b59ad740 1 -- start start 2024-09-19T06:54:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.049+0000 7f70b59ad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f743c4a0 0x5584f743c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.049+0000 7f70b59ad740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f743cdc0 0x5584f7447140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.049+0000 7f70b59ad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f7447680 0x5584f7449a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.049+0000 7f70b59ad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5584f73a22e0 con 0x5584f7447680 2024-09-19T06:54:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.049+0000 7f70b59ad740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5584f7349ce0 con 0x5584f743cdc0 2024-09-19T06:54:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.049+0000 7f70b59ad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5584f739e2a0 con 0x5584f743c4a0 2024-09-19T06:54:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.049+0000 7f70b0b6f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f7447680 0x5584f7449a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.049+0000 7f70b0b6f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f7447680 0x5584f7449a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49200/0 (socket says 172.21.15.103:49200) 2024-09-19T06:54:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.049+0000 7f70b0b6f640 1 -- 172.21.15.103:0/2391643868 learned_addr learned my addr 172.21.15.103:0/2391643868 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.049+0000 7f70a27fc640 1 --2- 172.21.15.103:0/4177212710 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70980a33b0 0x7f70980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.050+0000 7f70b1370640 1 --2- 172.21.15.103:0/2391643868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f743cdc0 0x5584f7447140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.050+0000 7f70b1b71640 1 --2- 172.21.15.103:0/2391643868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f743c4a0 0x5584f743c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.050+0000 7f70b1b71640 1 --2- 172.21.15.103:0/4177212710 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70980a33b0 0x7f70980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.050+0000 7f70a27fc640 1 -- 172.21.15.103:0/4177212710 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f70a80cfe00 con 0x7f70ac101dd0 2024-09-19T06:54:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.050+0000 7f70b0b6f640 1 -- 172.21.15.103:0/2391643868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f743c4a0 msgr2=0x5584f743c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.050+0000 7f70b0b6f640 1 --2- 172.21.15.103:0/2391643868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f743c4a0 0x5584f743c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.050+0000 7f70b0b6f640 1 -- 172.21.15.103:0/2391643868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f743cdc0 msgr2=0x5584f7447140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.050+0000 7f70b0b6f640 1 --2- 172.21.15.103:0/2391643868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f743cdc0 0x5584f7447140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.050+0000 7f70b0b6f640 1 -- 172.21.15.103:0/2391643868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584f732e440 con 0x5584f7447680 2024-09-19T06:54:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.050+0000 7f70b1b71640 1 --2- 172.21.15.103:0/2391643868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f743c4a0 0x5584f743c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.050+0000 7f70b0b6f640 1 --2- 172.21.15.103:0/2391643868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f7447680 0x5584f7449a70 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f709c00a8e0 tx=0x7f709c00ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7b73c747efa192 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.050+0000 7f70b1b71640 1 --2- 172.21.15.103:0/4177212710 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70980a33b0 0x7f70980a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009a60 tx=0x7f70a40095c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.050+0000 7f70977fe640 1 -- 172.21.15.103:0/2391643868 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c015070 con 0x5584f7447680 2024-09-19T06:54:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.050+0000 7f70977fe640 1 -- 172.21.15.103:0/2391643868 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f709c00cb40 con 0x5584f7447680 2024-09-19T06:54:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.050+0000 7f70977fe640 1 -- 172.21.15.103:0/2391643868 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c0185f0 con 0x5584f7447680 2024-09-19T06:54:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.051+0000 7f70b59ad740 1 -- 172.21.15.103:0/2391643868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f7447680 msgr2=0x5584f7449a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.051+0000 7f70b59ad740 1 --2- 172.21.15.103:0/2391643868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f7447680 0x5584f7449a70 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f709c00a8e0 tx=0x7f709c00ada0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.051+0000 7f70b59ad740 1 -- 172.21.15.103:0/2391643868 shutdown_connections 2024-09-19T06:54:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.051+0000 7f70b59ad740 1 --2- 172.21.15.103:0/2391643868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f7447680 0x5584f7449a70 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.051+0000 7f70b59ad740 1 --2- 172.21.15.103:0/2391643868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f743cdc0 0x5584f7447140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.051+0000 7f70b59ad740 1 --2- 172.21.15.103:0/2391643868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f743c4a0 0x5584f743c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.051+0000 7f70b59ad740 1 -- 172.21.15.103:0/2391643868 >> 172.21.15.103:0/2391643868 conn(0x5584f7438a30 msgr2=0x5584f7438e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.051+0000 7f70b59ad740 1 -- 172.21.15.103:0/2391643868 shutdown_connections 2024-09-19T06:54:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.051+0000 7f70b59ad740 1 -- 172.21.15.103:0/2391643868 wait complete. 2024-09-19T06:54:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.051+0000 7f70b59ad740 1 -- start start 2024-09-19T06:54:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b59ad740 1 Processor -- start 2024-09-19T06:54:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b59ad740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f743c4a0 0x5584f7446ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b59ad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743cdc0 0x5584f74441a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b59ad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f7447680 0x5584f74446e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b59ad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5584f744a250 con 0x5584f743cdc0 2024-09-19T06:54:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b59ad740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5584f735d050 con 0x5584f743c4a0 2024-09-19T06:54:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b59ad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5584f744a3d0 con 0x5584f7447680 2024-09-19T06:54:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b59ad740 10 client.0 _ll_register_callbacks cb 0x5584f739d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b0b6f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743cdc0 0x5584f74441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b0b6f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743cdc0 0x5584f74441a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49206/0 (socket says 172.21.15.103:49206) 2024-09-19T06:54:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b0b6f640 1 -- 172.21.15.103:0/4088462933 learned_addr learned my addr 172.21.15.103:0/4088462933 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b1370640 1 --2- 172.21.15.103:0/4088462933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f743c4a0 0x5584f7446ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b0b6f640 1 -- 172.21.15.103:0/4088462933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f7447680 msgr2=0x5584f74446e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b0b6f640 1 --2- 172.21.15.103:0/4088462933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f7447680 0x5584f74446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b0b6f640 1 -- 172.21.15.103:0/4088462933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f743c4a0 msgr2=0x5584f7446ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b0b6f640 1 --2- 172.21.15.103:0/4088462933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f743c4a0 0x5584f7446ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b0b6f640 1 -- 172.21.15.103:0/4088462933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584f71133c0 con 0x5584f743cdc0 2024-09-19T06:54:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.052+0000 7f70b1370640 1 --2- 172.21.15.103:0/4088462933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f743c4a0 0x5584f7446ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.053+0000 7f70b0b6f640 1 --2- 172.21.15.103:0/4088462933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743cdc0 0x5584f74441a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f709c00a120 tx=0x7f709c0116a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2480b8de6bcfb2e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.053+0000 7f7094ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.053+0000 7f7094ff9640 1 -- 172.21.15.103:0/4088462933 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c020070 con 0x5584f743cdc0 2024-09-19T06:54:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.053+0000 7f7094ff9640 1 -- 172.21.15.103:0/4088462933 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f709c01a450 con 0x5584f743cdc0 2024-09-19T06:54:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.053+0000 7f70b59ad740 1 -- 172.21.15.103:0/4088462933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f709c00cb40 con 0x5584f743cdc0 2024-09-19T06:54:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.053+0000 7f7094ff9640 1 -- 172.21.15.103:0/4088462933 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f709c015040 con 0x5584f743cdc0 2024-09-19T06:54:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.053+0000 7f70b59ad740 10 client.17364 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.053+0000 7f70b59ad740 1 -- 172.21.15.103:0/4088462933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5584f7455b40 con 0x5584f743cdc0 2024-09-19T06:54:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.053+0000 7f7094ff9640 1 -- 172.21.15.103:0/4088462933 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f709c00cb40 con 0x5584f743cdc0 2024-09-19T06:54:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.053+0000 7f7094ff9640 1 -- 172.21.15.103:0/4088462933 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f709c02c5a0 con 0x5584f743cdc0 2024-09-19T06:54:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.053+0000 7f70b59ad740 10 client.17364 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.053+0000 7f70b59ad740 10 client.17364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.053+0000 7f70b59ad740 20 client.17364 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.053+0000 7f70b59ad740 1 --2- 172.21.15.103:0/4088462933 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5584f7456fa0 0x5584f7459390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.053+0000 7f70b59ad740 4 client.17364 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.053+0000 7f70b59ad740 1 -- 172.21.15.103:0/4088462933 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584f73af850 con 0x5584f7456fa0 2024-09-19T06:54:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.054+0000 7f70b1370640 1 --2- 172.21.15.103:0/4088462933 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5584f7456fa0 0x5584f7459390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.054+0000 7f7094ff9640 10 client.17364 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.054+0000 7f70b1370640 1 --2- 172.21.15.103:0/4088462933 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5584f7456fa0 0x5584f7459390 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b70f2357a992ad4 server_cookie=7fa8e8c855f3aea6 in_seq=0 out_seq=0 2024-09-19T06:54:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.055+0000 7f7094ff9640 1 -- 172.21.15.103:0/4088462933 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5584f73af850 con 0x5584f7456fa0 2024-09-19T06:54:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.055+0000 7f7094ff9640 10 client.17364 handle_command_reply: tid=0 2024-09-19T06:54:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.055+0000 7f70b59ad740 1 client.17364 shutdown 2024-09-19T06:54:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.055+0000 7f70b59ad740 1 -- 172.21.15.103:0/4088462933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743cdc0 msgr2=0x5584f74441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.055+0000 7f70b59ad740 1 --2- 172.21.15.103:0/4088462933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743cdc0 0x5584f74441a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f709c00a120 tx=0x7f709c0116a0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.055+0000 7f70b59ad740 1 -- 172.21.15.103:0/4088462933 shutdown_connections 2024-09-19T06:54:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.055+0000 7f70b59ad740 1 --2- 172.21.15.103:0/4088462933 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5584f7456fa0 0x5584f7459390 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.055+0000 7f70b59ad740 1 --2- 172.21.15.103:0/4088462933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f7447680 0x5584f74446e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.055+0000 7f70b59ad740 1 --2- 172.21.15.103:0/4088462933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743cdc0 0x5584f74441a0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.055+0000 7f70b59ad740 1 --2- 172.21.15.103:0/4088462933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f743c4a0 0x5584f7446ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.055+0000 7f7094ff9640 0 client.17364 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.055+0000 7f70b59ad740 1 -- 172.21.15.103:0/4088462933 >> 172.21.15.103:0/4088462933 conn(0x5584f7438a30 msgr2=0x5584f743e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.055+0000 7f70b59ad740 1 -- 172.21.15.103:0/4088462933 shutdown_connections 2024-09-19T06:54:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.055+0000 7f70b59ad740 1 -- 172.21.15.103:0/4088462933 wait complete. 2024-09-19T06:54:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.055+0000 7f70b59ad740 20 client.17364 trim_cache size 0 max 16384 2024-09-19T06:54:34.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.077+0000 7f70b59ad740 1 Processor -- start 2024-09-19T06:54:34.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.077+0000 7f70b59ad740 1 -- start start 2024-09-19T06:54:34.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.077+0000 7f70b59ad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743c4a0 0x5584f743f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.077+0000 7f70b59ad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f743cdc0 0x5584f74431a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.077+0000 7f70b59ad740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f7447680 0x5584f745eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.077+0000 7f70b59ad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1afb30 con 0x5584f743c4a0 2024-09-19T06:54:34.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.077+0000 7f70b59ad740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1b1670 con 0x5584f7447680 2024-09-19T06:54:34.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.077+0000 7f70b59ad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1b0200 con 0x5584f743cdc0 2024-09-19T06:54:34.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.077+0000 7f70b1370640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743c4a0 0x5584f743f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.077+0000 7f70b1370640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743c4a0 0x5584f743f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49218/0 (socket says 172.21.15.103:49218) 2024-09-19T06:54:34.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.077+0000 7f70b1370640 1 -- 172.21.15.103:0/3087728691 learned_addr learned my addr 172.21.15.103:0/3087728691 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:34.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.077+0000 7f70b1b71640 1 --2- 172.21.15.103:0/3087728691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f7447680 0x5584f745eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.077+0000 7f70b0b6f640 1 --2- 172.21.15.103:0/3087728691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f743cdc0 0x5584f74431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.078+0000 7f70b1370640 1 -- 172.21.15.103:0/3087728691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f743cdc0 msgr2=0x5584f74431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:34.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.078+0000 7f70b1370640 1 --2- 172.21.15.103:0/3087728691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f743cdc0 0x5584f74431a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.078+0000 7f70b1370640 1 -- 172.21.15.103:0/3087728691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f7447680 msgr2=0x5584f745eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:34.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.078+0000 7f70b1370640 1 --2- 172.21.15.103:0/3087728691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f7447680 0x5584f745eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.078+0000 7f70b1370640 1 -- 172.21.15.103:0/3087728691 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584f7438860 con 0x5584f743c4a0 2024-09-19T06:54:34.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.078+0000 7f70b1370640 1 --2- 172.21.15.103:0/3087728691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743c4a0 0x5584f743f900 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f70a8007590 tx=0x7f70a8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d83cfea1bb3985 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:34.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.078+0000 7f7096ffd640 1 -- 172.21.15.103:0/3087728691 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a801d040 con 0x5584f743c4a0 2024-09-19T06:54:34.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.078+0000 7f7096ffd640 1 -- 172.21.15.103:0/3087728691 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a80a04a0 con 0x5584f743c4a0 2024-09-19T06:54:34.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.078+0000 7f7096ffd640 1 -- 172.21.15.103:0/3087728691 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8098760 con 0x5584f743c4a0 2024-09-19T06:54:34.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.079+0000 7f70b59ad740 1 -- 172.21.15.103:0/3087728691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743c4a0 msgr2=0x5584f743f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:34.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.079+0000 7f70b59ad740 1 --2- 172.21.15.103:0/3087728691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743c4a0 0x5584f743f900 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f70a8007590 tx=0x7f70a8015000 comp rx=0 tx=0).stop 2024-09-19T06:54:34.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.079+0000 7f70b59ad740 1 -- 172.21.15.103:0/3087728691 shutdown_connections 2024-09-19T06:54:34.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.079+0000 7f70b59ad740 1 --2- 172.21.15.103:0/3087728691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f7447680 0x5584f745eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.079+0000 7f70b59ad740 1 --2- 172.21.15.103:0/3087728691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f743cdc0 0x5584f74431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.079+0000 7f70b59ad740 1 --2- 172.21.15.103:0/3087728691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743c4a0 0x5584f743f900 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.079+0000 7f70b59ad740 1 -- 172.21.15.103:0/3087728691 >> 172.21.15.103:0/3087728691 conn(0x5584f7438a30 msgr2=0x5584f7442a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:34.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.079+0000 7f70b59ad740 1 -- 172.21.15.103:0/3087728691 shutdown_connections 2024-09-19T06:54:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.079+0000 7f70b59ad740 1 -- 172.21.15.103:0/3087728691 wait complete. 2024-09-19T06:54:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.079+0000 7f70b59ad740 1 -- start start 2024-09-19T06:54:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.079+0000 7f70b59ad740 1 Processor -- start 2024-09-19T06:54:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b59ad740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f743c4a0 0x5584f744ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b59ad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743cdc0 0x5584f744b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b59ad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f7447680 0x5584f744b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b59ad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1b0fa0 con 0x5584f743cdc0 2024-09-19T06:54:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b59ad740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1b08d0 con 0x5584f743c4a0 2024-09-19T06:54:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b59ad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1adaf0 con 0x5584f7447680 2024-09-19T06:54:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b59ad740 10 client.0 _ll_register_callbacks cb 0x7f70a8098390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b0b6f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743cdc0 0x5584f744b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b0b6f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743cdc0 0x5584f744b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49232/0 (socket says 172.21.15.103:49232) 2024-09-19T06:54:34.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b0b6f640 1 -- 172.21.15.103:0/1842218787 learned_addr learned my addr 172.21.15.103:0/1842218787 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b1370640 1 --2- 172.21.15.103:0/1842218787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f743c4a0 0x5584f744ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b1b71640 1 --2- 172.21.15.103:0/1842218787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f7447680 0x5584f744b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b0b6f640 1 -- 172.21.15.103:0/1842218787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f7447680 msgr2=0x5584f744b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b0b6f640 1 --2- 172.21.15.103:0/1842218787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f7447680 0x5584f744b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b0b6f640 1 -- 172.21.15.103:0/1842218787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f743c4a0 msgr2=0x5584f744ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b0b6f640 1 --2- 172.21.15.103:0/1842218787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f743c4a0 0x5584f744ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b0b6f640 1 -- 172.21.15.103:0/1842218787 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584f74c0a80 con 0x5584f743cdc0 2024-09-19T06:54:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f70b0b6f640 1 --2- 172.21.15.103:0/1842218787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743cdc0 0x5584f744b240 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f709c001eb0 tx=0x7f709c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0bcebd6234e6685 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.080+0000 7f7094ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.081+0000 7f7094ff9640 1 -- 172.21.15.103:0/1842218787 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c020040 con 0x5584f743cdc0 2024-09-19T06:54:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.081+0000 7f7094ff9640 1 -- 172.21.15.103:0/1842218787 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f709c006a70 con 0x5584f743cdc0 2024-09-19T06:54:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.081+0000 7f70b59ad740 1 -- 172.21.15.103:0/1842218787 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a80a04a0 con 0x5584f743cdc0 2024-09-19T06:54:34.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.081+0000 7f7094ff9640 1 -- 172.21.15.103:0/1842218787 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f709c03c050 con 0x5584f743cdc0 2024-09-19T06:54:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.081+0000 7f70b59ad740 10 client.17370 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.081+0000 7f70b59ad740 1 -- 172.21.15.103:0/1842218787 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5584f74c1c30 con 0x5584f743cdc0 2024-09-19T06:54:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.081+0000 7f7094ff9640 1 -- 172.21.15.103:0/1842218787 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a80a04a0 con 0x5584f743cdc0 2024-09-19T06:54:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.081+0000 7f7094ff9640 1 -- 172.21.15.103:0/1842218787 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f709c033070 con 0x5584f743cdc0 2024-09-19T06:54:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.081+0000 7f70b59ad740 10 client.17370 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.081+0000 7f70b59ad740 10 client.17370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.081+0000 7f70b59ad740 20 client.17370 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.081+0000 7f70b59ad740 1 --2- 172.21.15.103:0/1842218787 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5584f74c2350 0x5584f74c4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.081+0000 7f70b59ad740 4 client.17370 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:54:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.081+0000 7f70b59ad740 1 -- 172.21.15.103:0/1842218787 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5584f73a9d60 con 0x5584f74c2350 2024-09-19T06:54:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.082+0000 7f70b1370640 1 --2- 172.21.15.103:0/1842218787 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5584f74c2350 0x5584f74c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.082+0000 7f7094ff9640 10 client.17370 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:34.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.082+0000 7f70b1370640 1 --2- 172.21.15.103:0/1842218787 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5584f74c2350 0x5584f74c4770 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=788399132d60fe00 server_cookie=5c28968968c54560 in_seq=0 out_seq=0 2024-09-19T06:54:34.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.082+0000 7f7094ff9640 1 -- 172.21.15.103:0/1842218787 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5584f73a9d60 con 0x5584f74c2350 2024-09-19T06:54:34.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.082+0000 7f7094ff9640 10 client.17370 handle_command_reply: tid=0 2024-09-19T06:54:34.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.082+0000 7f70b59ad740 1 client.17370 shutdown 2024-09-19T06:54:34.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.082+0000 7f70b59ad740 1 -- 172.21.15.103:0/1842218787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743cdc0 msgr2=0x5584f744b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:34.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.082+0000 7f70b59ad740 1 --2- 172.21.15.103:0/1842218787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743cdc0 0x5584f744b240 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f709c001eb0 tx=0x7f709c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.083+0000 7f70b59ad740 1 -- 172.21.15.103:0/1842218787 shutdown_connections 2024-09-19T06:54:34.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.083+0000 7f70b59ad740 1 --2- 172.21.15.103:0/1842218787 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5584f74c2350 0x5584f74c4770 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.083+0000 7f70b59ad740 1 --2- 172.21.15.103:0/1842218787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584f7447680 0x5584f744b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.083+0000 7f70b59ad740 1 --2- 172.21.15.103:0/1842218787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584f743cdc0 0x5584f744b240 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.083+0000 7f70b59ad740 1 --2- 172.21.15.103:0/1842218787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584f743c4a0 0x5584f744ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.083+0000 7f70b59ad740 1 -- 172.21.15.103:0/1842218787 >> 172.21.15.103:0/1842218787 conn(0x5584f7438a30 msgr2=0x5584f7445a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:34.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.083+0000 7f7094ff9640 0 client.17370 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:34.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.083+0000 7f70b59ad740 1 -- 172.21.15.103:0/1842218787 shutdown_connections 2024-09-19T06:54:34.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.083+0000 7f70b59ad740 1 -- 172.21.15.103:0/1842218787 wait complete. 2024-09-19T06:54:34.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.083+0000 7f70b59ad740 20 client.17370 trim_cache size 0 max 16384 2024-09-19T06:54:34.094 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:54:34.094 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T06:54:34.094 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T06:54:34.094 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:54:34.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.083+0000 7f70b35d4640 1 -- 172.21.15.103:0/4177212710 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70980a33b0 msgr2=0x7f70980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:34.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.083+0000 7f70b35d4640 1 --2- 172.21.15.103:0/4177212710 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70980a33b0 0x7f70980a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009a60 tx=0x7f70a40095c0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.083+0000 7f70b35d4640 1 -- 172.21.15.103:0/4177212710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70ac101dd0 msgr2=0x7f70ac0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:34.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.083+0000 7f70b35d4640 1 --2- 172.21.15.103:0/4177212710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70ac101dd0 0x7f70ac0744f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f70a800c890 tx=0x7f70a800cd50 comp rx=0 tx=0).stop 2024-09-19T06:54:34.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.084+0000 7f70b35d4640 1 -- 172.21.15.103:0/4177212710 shutdown_connections 2024-09-19T06:54:34.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.084+0000 7f70b35d4640 1 --2- 172.21.15.103:0/4177212710 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70980a33b0 0x7f70980a5870 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.084+0000 7f70b35d4640 1 --2- 172.21.15.103:0/4177212710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70ac106f00 0x7f70ac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.084+0000 7f70b35d4640 1 --2- 172.21.15.103:0/4177212710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70ac102780 0x7f70ac074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.084+0000 7f70b35d4640 1 --2- 172.21.15.103:0/4177212710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70ac101dd0 0x7f70ac0744f0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:34.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.084+0000 7f70b35d4640 1 -- 172.21.15.103:0/4177212710 >> 172.21.15.103:0/4177212710 conn(0x7f70ac0fd8b0 msgr2=0x7f70ac0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:34.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.084+0000 7f70b35d4640 1 -- 172.21.15.103:0/4177212710 shutdown_connections 2024-09-19T06:54:34.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:34.084+0000 7f70b35d4640 1 -- 172.21.15.103:0/4177212710 wait complete. 2024-09-19T06:54:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:34 smithi174 ceph-mon[35637]: pgmap v163: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-19T06:54:34.357 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:34 smithi103 ceph-mon[31158]: pgmap v163: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-19T06:54:34.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:34 smithi134 ceph-mon[35356]: pgmap v163: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-19T06:54:34.814 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:54:34.814 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:54:35.064 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.534+0000 7f8d2b40d640 1 -- 172.21.15.103:0/1882959182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d24102790 msgr2=0x7f8d241067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.534+0000 7f8d2b40d640 1 --2- 172.21.15.103:0/1882959182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d24102790 0x7f8d241067e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009ca0 tx=0x7f8d1802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:54:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.535+0000 7f8d2b40d640 1 -- 172.21.15.103:0/1882959182 shutdown_connections 2024-09-19T06:54:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.535+0000 7f8d2b40d640 1 --2- 172.21.15.103:0/1882959182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d24106f10 0x7f8d2410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.535+0000 7f8d2b40d640 1 --2- 172.21.15.103:0/1882959182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d24102790 0x7f8d241067e0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.535+0000 7f8d2b40d640 1 --2- 172.21.15.103:0/1882959182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d24101de0 0x7f8d241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.535+0000 7f8d2b40d640 1 -- 172.21.15.103:0/1882959182 >> 172.21.15.103:0/1882959182 conn(0x7f8d240fd8c0 msgr2=0x7f8d240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.535+0000 7f8d2b40d640 1 -- 172.21.15.103:0/1882959182 shutdown_connections 2024-09-19T06:54:35.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.535+0000 7f8d2b40d640 1 -- 172.21.15.103:0/1882959182 wait complete. 2024-09-19T06:54:35.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.535+0000 7f8d2b40d640 1 Processor -- start 2024-09-19T06:54:35.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.535+0000 7f8d2b40d640 1 -- start start 2024-09-19T06:54:35.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.535+0000 7f8d2b40d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d24101de0 0x7f8d24198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:35.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.535+0000 7f8d2b40d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d24106f10 0x7f8d24198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:35.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.535+0000 7f8d2b40d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d2419cf20 0x7f8d2419d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:35.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.535+0000 7f8d2b40d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d24111f90 con 0x7f8d24106f10 2024-09-19T06:54:35.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.535+0000 7f8d2b40d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d24111e10 con 0x7f8d2419cf20 2024-09-19T06:54:35.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.535+0000 7f8d2b40d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d24112110 con 0x7f8d24101de0 2024-09-19T06:54:35.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d289a8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d24106f10 0x7f8d24198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:35.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d291a9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d24101de0 0x7f8d24198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:35.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d289a8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d24106f10 0x7f8d24198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49242/0 (socket says 172.21.15.103:49242) 2024-09-19T06:54:35.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d289a8640 1 -- 172.21.15.103:0/3901401623 learned_addr learned my addr 172.21.15.103:0/3901401623 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:35.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d299aa640 1 --2- 172.21.15.103:0/3901401623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d2419cf20 0x7f8d2419d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:35.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d289a8640 1 -- 172.21.15.103:0/3901401623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d24101de0 msgr2=0x7f8d24198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:35.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d289a8640 1 --2- 172.21.15.103:0/3901401623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d24101de0 0x7f8d24198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:35.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d289a8640 1 -- 172.21.15.103:0/3901401623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d2419cf20 msgr2=0x7f8d2419d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:35.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d289a8640 1 --2- 172.21.15.103:0/3901401623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d2419cf20 0x7f8d2419d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:35.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d289a8640 1 -- 172.21.15.103:0/3901401623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2419d9e0 con 0x7f8d24106f10 2024-09-19T06:54:35.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d299aa640 1 --2- 172.21.15.103:0/3901401623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d2419cf20 0x7f8d2419d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:35.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d289a8640 1 --2- 172.21.15.103:0/3901401623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d24106f10 0x7f8d24198b90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009bf0 tx=0x7f8d18030f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:35.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d227fc640 1 -- 172.21.15.103:0/3901401623 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18031960 con 0x7f8d24106f10 2024-09-19T06:54:35.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d227fc640 1 -- 172.21.15.103:0/3901401623 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d18046960 con 0x7f8d24106f10 2024-09-19T06:54:35.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d2b40d640 1 -- 172.21.15.103:0/3901401623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2419dc70 con 0x7f8d24106f10 2024-09-19T06:54:35.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d227fc640 1 -- 172.21.15.103:0/3901401623 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1803a9d0 con 0x7f8d24106f10 2024-09-19T06:54:35.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.536+0000 7f8d2b40d640 1 -- 172.21.15.103:0/3901401623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d241a5450 con 0x7f8d24106f10 2024-09-19T06:54:35.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.537+0000 7f8d2b40d640 1 -- 172.21.15.103:0/3901401623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d241067e0 con 0x7f8d24106f10 2024-09-19T06:54:35.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.538+0000 7f8d227fc640 1 -- 172.21.15.103:0/3901401623 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8d18046b10 con 0x7f8d24106f10 2024-09-19T06:54:35.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.539+0000 7f8d227fc640 1 --2- 172.21.15.103:0/3901401623 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8d0c0a33b0 0x7f8d0c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:35.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.539+0000 7f8d291a9640 1 --2- 172.21.15.103:0/3901401623 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8d0c0a33b0 0x7f8d0c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:35.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.539+0000 7f8d227fc640 1 -- 172.21.15.103:0/3901401623 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f8d180f1420 con 0x7f8d24106f10 2024-09-19T06:54:35.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.540+0000 7f8d291a9640 1 --2- 172.21.15.103:0/3901401623 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8d0c0a33b0 0x7f8d0c0a5870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00abe0 tx=0x7f8d1c00a2d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:35.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.542+0000 7f8d227fc640 1 -- 172.21.15.103:0/3901401623 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8d180bdf20 con 0x7f8d24106f10 2024-09-19T06:54:35.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.742+0000 7f8d2b40d640 1 -- 172.21.15.103:0/3901401623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d241999b0 con 0x7f8d24106f10 2024-09-19T06:54:35.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.742+0000 7f8d227fc640 1 -- 172.21.15.103:0/3901401623 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f8d241999b0 con 0x7f8d24106f10 2024-09-19T06:54:35.744 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:54:35.744 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:54:35.744 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:54:35.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.745+0000 7f8d2b40d640 1 -- 172.21.15.103:0/3901401623 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8d0c0a33b0 msgr2=0x7f8d0c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:35.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.745+0000 7f8d2b40d640 1 --2- 172.21.15.103:0/3901401623 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8d0c0a33b0 0x7f8d0c0a5870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00abe0 tx=0x7f8d1c00a2d0 comp rx=0 tx=0).stop 2024-09-19T06:54:35.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.745+0000 7f8d2b40d640 1 -- 172.21.15.103:0/3901401623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d24106f10 msgr2=0x7f8d24198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:35.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.745+0000 7f8d2b40d640 1 --2- 172.21.15.103:0/3901401623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d24106f10 0x7f8d24198b90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009bf0 tx=0x7f8d18030f80 comp rx=0 tx=0).stop 2024-09-19T06:54:35.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.745+0000 7f8d2b40d640 1 -- 172.21.15.103:0/3901401623 shutdown_connections 2024-09-19T06:54:35.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.745+0000 7f8d2b40d640 1 --2- 172.21.15.103:0/3901401623 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8d0c0a33b0 0x7f8d0c0a5870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:35.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.745+0000 7f8d2b40d640 1 --2- 172.21.15.103:0/3901401623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d2419cf20 0x7f8d2419d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:35.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.745+0000 7f8d2b40d640 1 --2- 172.21.15.103:0/3901401623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d24106f10 0x7f8d24198b90 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:35.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.745+0000 7f8d2b40d640 1 --2- 172.21.15.103:0/3901401623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d24101de0 0x7f8d24198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:35.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.745+0000 7f8d2b40d640 1 -- 172.21.15.103:0/3901401623 >> 172.21.15.103:0/3901401623 conn(0x7f8d240fd8c0 msgr2=0x7f8d240fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:35.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.745+0000 7f8d2b40d640 1 -- 172.21.15.103:0/3901401623 shutdown_connections 2024-09-19T06:54:35.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:35.745+0000 7f8d2b40d640 1 -- 172.21.15.103:0/3901401623 wait complete. 2024-09-19T06:54:36.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:35 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3901401623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:36.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:35 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3901401623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:36.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:35 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3901401623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:36.320 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:54:36.447 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:54:36.589 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:36.990 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:36 smithi103 ceph-mon[31158]: pgmap v164: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-09-19T06:54:37.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.069+0000 7f9c67277640 1 -- 172.21.15.103:0/3575839778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c60102d30 msgr2=0x7f9c60103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:37.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.069+0000 7f9c67277640 1 --2- 172.21.15.103:0/3575839778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c60102d30 0x7f9c60103110 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009930 tx=0x7f9c5002a420 comp rx=0 tx=0).stop 2024-09-19T06:54:37.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.069+0000 7f9c67277640 1 -- 172.21.15.103:0/3575839778 shutdown_connections 2024-09-19T06:54:37.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.069+0000 7f9c67277640 1 --2- 172.21.15.103:0/3575839778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c601082a0 0x7f9c6010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.069+0000 7f9c67277640 1 --2- 172.21.15.103:0/3575839778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c601036e0 0x7f9c60107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.069+0000 7f9c67277640 1 --2- 172.21.15.103:0/3575839778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c60102d30 0x7f9c60103110 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.069+0000 7f9c67277640 1 -- 172.21.15.103:0/3575839778 >> 172.21.15.103:0/3575839778 conn(0x7f9c600fc7d0 msgr2=0x7f9c600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:37.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.069+0000 7f9c67277640 1 -- 172.21.15.103:0/3575839778 shutdown_connections 2024-09-19T06:54:37.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.069+0000 7f9c67277640 1 -- 172.21.15.103:0/3575839778 wait complete. 2024-09-19T06:54:37.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.070+0000 7f9c67277640 1 Processor -- start 2024-09-19T06:54:37.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.070+0000 7f9c67277640 1 -- start start 2024-09-19T06:54:37.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.070+0000 7f9c67277640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c60102d30 0x7f9c60100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.070+0000 7f9c67277640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c601036e0 0x7f9c60100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.070+0000 7f9c67277640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c601082a0 0x7f9c60101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.070+0000 7f9c67277640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9c601120a0 con 0x7f9c601036e0 2024-09-19T06:54:37.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.070+0000 7f9c67277640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9c60111f20 con 0x7f9c601082a0 2024-09-19T06:54:37.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.070+0000 7f9c67277640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9c60112220 con 0x7f9c60102d30 2024-09-19T06:54:37.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.070+0000 7f9c64812640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c601036e0 0x7f9c60100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.071+0000 7f9c64812640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c601036e0 0x7f9c60100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49252/0 (socket says 172.21.15.103:49252) 2024-09-19T06:54:37.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.071+0000 7f9c64812640 1 -- 172.21.15.103:0/3330929245 learned_addr learned my addr 172.21.15.103:0/3330929245 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:37.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.071+0000 7f9c65814640 1 --2- 172.21.15.103:0/3330929245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c601082a0 0x7f9c60101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.071+0000 7f9c65013640 1 --2- 172.21.15.103:0/3330929245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c60102d30 0x7f9c60100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.071+0000 7f9c64812640 1 -- 172.21.15.103:0/3330929245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c60102d30 msgr2=0x7f9c60100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:37.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.071+0000 7f9c64812640 1 --2- 172.21.15.103:0/3330929245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c60102d30 0x7f9c60100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.071+0000 7f9c64812640 1 -- 172.21.15.103:0/3330929245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c601082a0 msgr2=0x7f9c60101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:37.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.071+0000 7f9c64812640 1 --2- 172.21.15.103:0/3330929245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c601082a0 0x7f9c60101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.071+0000 7f9c64812640 1 -- 172.21.15.103:0/3330929245 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c601a6a60 con 0x7f9c601036e0 2024-09-19T06:54:37.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.071+0000 7f9c64812640 1 --2- 172.21.15.103:0/3330929245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c601036e0 0x7f9c60100ed0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00c8c0 tx=0x7f9c5c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:37.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.071+0000 7f9c567fc640 1 -- 172.21.15.103:0/3330929245 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c007bb0 con 0x7f9c601036e0 2024-09-19T06:54:37.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.071+0000 7f9c67277640 1 -- 172.21.15.103:0/3330929245 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c601a6d50 con 0x7f9c601036e0 2024-09-19T06:54:37.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.071+0000 7f9c567fc640 1 -- 172.21.15.103:0/3330929245 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c5c007d50 con 0x7f9c601036e0 2024-09-19T06:54:37.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.071+0000 7f9c567fc640 1 -- 172.21.15.103:0/3330929245 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c018830 con 0x7f9c601036e0 2024-09-19T06:54:37.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.071+0000 7f9c67277640 1 -- 172.21.15.103:0/3330929245 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c601a7200 con 0x7f9c601036e0 2024-09-19T06:54:37.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.072+0000 7f9c67277640 1 -- 172.21.15.103:0/3330929245 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c6006a850 con 0x7f9c601036e0 2024-09-19T06:54:37.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.073+0000 7f9c567fc640 1 -- 172.21.15.103:0/3330929245 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9c5c0189f0 con 0x7f9c601036e0 2024-09-19T06:54:37.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.074+0000 7f9c567fc640 1 --2- 172.21.15.103:0/3330929245 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9c4c0a33b0 0x7f9c4c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.074+0000 7f9c567fc640 1 -- 172.21.15.103:0/3330929245 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f9c5c0cfe40 con 0x7f9c601036e0 2024-09-19T06:54:37.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.074+0000 7f9c65013640 1 --2- 172.21.15.103:0/3330929245 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9c4c0a33b0 0x7f9c4c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.075+0000 7f9c65013640 1 --2- 172.21.15.103:0/3330929245 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9c4c0a33b0 0x7f9c4c0a5870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9c50000c00 tx=0x7f9c50005a20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:37.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.079+0000 7f9c567fc640 1 -- 172.21.15.103:0/3330929245 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9c5c014030 con 0x7f9c601036e0 2024-09-19T06:54:37.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:36 smithi174 ceph-mon[35637]: pgmap v164: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-09-19T06:54:37.094 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:36 smithi134 ceph-mon[35356]: pgmap v164: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-09-19T06:54:37.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.287+0000 7f9c67277640 1 -- 172.21.15.103:0/3330929245 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c6002d7c0 con 0x7f9c601036e0 2024-09-19T06:54:37.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.288+0000 7f9c567fc640 1 -- 172.21.15.103:0/3330929245 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f9c5c09c7f0 con 0x7f9c601036e0 2024-09-19T06:54:37.289 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:54:37.289 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:54:37.290 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:54:37.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.290+0000 7f9c67277640 1 -- 172.21.15.103:0/3330929245 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9c4c0a33b0 msgr2=0x7f9c4c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:37.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.290+0000 7f9c67277640 1 --2- 172.21.15.103:0/3330929245 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9c4c0a33b0 0x7f9c4c0a5870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9c50000c00 tx=0x7f9c50005a20 comp rx=0 tx=0).stop 2024-09-19T06:54:37.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.290+0000 7f9c67277640 1 -- 172.21.15.103:0/3330929245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c601036e0 msgr2=0x7f9c60100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:37.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.290+0000 7f9c67277640 1 --2- 172.21.15.103:0/3330929245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c601036e0 0x7f9c60100ed0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00c8c0 tx=0x7f9c5c00cd80 comp rx=0 tx=0).stop 2024-09-19T06:54:37.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.290+0000 7f9c67277640 1 -- 172.21.15.103:0/3330929245 shutdown_connections 2024-09-19T06:54:37.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.290+0000 7f9c67277640 1 --2- 172.21.15.103:0/3330929245 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9c4c0a33b0 0x7f9c4c0a5870 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.290+0000 7f9c67277640 1 --2- 172.21.15.103:0/3330929245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c601082a0 0x7f9c60101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.290+0000 7f9c67277640 1 --2- 172.21.15.103:0/3330929245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c601036e0 0x7f9c60100ed0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.290+0000 7f9c67277640 1 --2- 172.21.15.103:0/3330929245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c60102d30 0x7f9c60100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.290+0000 7f9c67277640 1 -- 172.21.15.103:0/3330929245 >> 172.21.15.103:0/3330929245 conn(0x7f9c600fc7d0 msgr2=0x7f9c60105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:37.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.291+0000 7f9c67277640 1 -- 172.21.15.103:0/3330929245 shutdown_connections 2024-09-19T06:54:37.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.291+0000 7f9c67277640 1 -- 172.21.15.103:0/3330929245 wait complete. 2024-09-19T06:54:37.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.682+0000 7f9344243640 1 -- 172.21.15.103:0/536988004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f933c101de0 msgr2=0x7f933c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:37.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.682+0000 7f9344243640 1 --2- 172.21.15.103:0/536988004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f933c101de0 0x7f933c1021c0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9338009900 tx=0x7f933802a410 comp rx=0 tx=0).stop 2024-09-19T06:54:37.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.682+0000 7f9344243640 1 -- 172.21.15.103:0/536988004 shutdown_connections 2024-09-19T06:54:37.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.682+0000 7f9344243640 1 --2- 172.21.15.103:0/536988004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f933c106f10 0x7f933c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.682+0000 7f9344243640 1 --2- 172.21.15.103:0/536988004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f933c102790 0x7f933c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.682+0000 7f9344243640 1 --2- 172.21.15.103:0/536988004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f933c101de0 0x7f933c1021c0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.682+0000 7f9344243640 1 -- 172.21.15.103:0/536988004 >> 172.21.15.103:0/536988004 conn(0x7f933c0fd8c0 msgr2=0x7f933c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:37.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.682+0000 7f9344243640 1 -- 172.21.15.103:0/536988004 shutdown_connections 2024-09-19T06:54:37.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.682+0000 7f9344243640 1 -- 172.21.15.103:0/536988004 wait complete. 2024-09-19T06:54:37.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f9344243640 1 Processor -- start 2024-09-19T06:54:37.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f9344243640 1 -- start start 2024-09-19T06:54:37.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f9344243640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f933c101de0 0x7f933c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f9344243640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f933c102790 0x7f933c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f9344243640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f933c106f10 0x7f933c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f9344243640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f933c111f70 con 0x7f933c102790 2024-09-19T06:54:37.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f9344243640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f933c111df0 con 0x7f933c101de0 2024-09-19T06:54:37.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f9344243640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1120f0 con 0x7f933c106f10 2024-09-19T06:54:37.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f93417de640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f933c102790 0x7f933c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f93417de640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f933c102790 0x7f933c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48676/0 (socket says 172.21.15.103:48676) 2024-09-19T06:54:37.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f93417de640 1 -- 172.21.15.103:0/2615530838 learned_addr learned my addr 172.21.15.103:0/2615530838 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:37.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f93427e0640 1 --2- 172.21.15.103:0/2615530838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f933c106f10 0x7f933c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f9341fdf640 1 --2- 172.21.15.103:0/2615530838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f933c101de0 0x7f933c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f93417de640 1 -- 172.21.15.103:0/2615530838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f933c106f10 msgr2=0x7f933c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:37.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f93417de640 1 --2- 172.21.15.103:0/2615530838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f933c106f10 0x7f933c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f93417de640 1 -- 172.21.15.103:0/2615530838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f933c101de0 msgr2=0x7f933c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:37.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f93417de640 1 --2- 172.21.15.103:0/2615530838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f933c101de0 0x7f933c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.683+0000 7f93417de640 1 -- 172.21.15.103:0/2615530838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933c19d3e0 con 0x7f933c102790 2024-09-19T06:54:37.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.684+0000 7f93417de640 1 --2- 172.21.15.103:0/2615530838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f933c102790 0x7f933c198970 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f932c00d950 tx=0x7f932c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:37.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.684+0000 7f9332ffd640 1 -- 172.21.15.103:0/2615530838 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c017030 con 0x7f933c102790 2024-09-19T06:54:37.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.684+0000 7f9344243640 1 -- 172.21.15.103:0/2615530838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933c19d6d0 con 0x7f933c102790 2024-09-19T06:54:37.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.684+0000 7f9332ffd640 1 -- 172.21.15.103:0/2615530838 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f932c004020 con 0x7f933c102790 2024-09-19T06:54:37.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.684+0000 7f9332ffd640 1 -- 172.21.15.103:0/2615530838 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c0047a0 con 0x7f933c102790 2024-09-19T06:54:37.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.684+0000 7f9344243640 1 -- 172.21.15.103:0/2615530838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f933c1a4fb0 con 0x7f933c102790 2024-09-19T06:54:37.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.686+0000 7f9332ffd640 1 -- 172.21.15.103:0/2615530838 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f932c0041d0 con 0x7f933c102790 2024-09-19T06:54:37.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.686+0000 7f934661c740 1 Processor -- start 2024-09-19T06:54:37.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f934661c740 1 -- start start 2024-09-19T06:54:37.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f934661c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f8fb40 0x556ca9f8e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f934661c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8e5d0 0x556ca9f8ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f934661c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f9d780 0x556ca9f9fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f934661c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556ca9ef42f0 con 0x556ca9f8fb40 2024-09-19T06:54:37.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f934661c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556ca9e9bcf0 con 0x556ca9f9d780 2024-09-19T06:54:37.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f934661c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556ca9ef02b0 con 0x556ca9f8e5d0 2024-09-19T06:54:37.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f93427e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f8fb40 0x556ca9f8e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f93427e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f8fb40 0x556ca9f8e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48682/0 (socket says 172.21.15.103:48682) 2024-09-19T06:54:37.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f93427e0640 1 -- 172.21.15.103:0/1040016471 learned_addr learned my addr 172.21.15.103:0/1040016471 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:37.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f9332ffd640 1 --2- 172.21.15.103:0/2615530838 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f93280a33b0 0x7f93280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f9341fdf640 1 --2- 172.21.15.103:0/1040016471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8e5d0 0x556ca9f8ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f93427e0640 1 --2- 172.21.15.103:0/2615530838 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f93280a33b0 0x7f93280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f9332ffd640 1 -- 172.21.15.103:0/2615530838 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f932c0cf670 con 0x7f933c102790 2024-09-19T06:54:37.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f93427e0640 1 -- 172.21.15.103:0/1040016471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8e5d0 msgr2=0x556ca9f8ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:37.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f93427e0640 1 --2- 172.21.15.103:0/1040016471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8e5d0 0x556ca9f8ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f93427e0640 1 -- 172.21.15.103:0/1040016471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f9d780 msgr2=0x556ca9f9fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:37.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.687+0000 7f93427e0640 1 --2- 172.21.15.103:0/1040016471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f9d780 0x556ca9f9fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.688+0000 7f93427e0640 1 -- 172.21.15.103:0/1040016471 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ca9e803a0 con 0x556ca9f8fb40 2024-09-19T06:54:37.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.688+0000 7f93427e0640 1 --2- 172.21.15.103:0/1040016471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f8fb40 0x556ca9f8e090 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f9334014a60 tx=0x7f9334014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d25be0e9d2bb16c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:37.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.688+0000 7f9327fff640 1 -- 172.21.15.103:0/1040016471 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933401ec50 con 0x556ca9f8fb40 2024-09-19T06:54:37.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.688+0000 7f9327fff640 1 -- 172.21.15.103:0/1040016471 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9334023070 con 0x556ca9f8fb40 2024-09-19T06:54:37.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.688+0000 7f9327fff640 1 -- 172.21.15.103:0/1040016471 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933401de00 con 0x556ca9f8fb40 2024-09-19T06:54:37.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.689+0000 7f934661c740 1 -- 172.21.15.103:0/1040016471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f8fb40 msgr2=0x556ca9f8e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:37.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.689+0000 7f93427e0640 1 --2- 172.21.15.103:0/2615530838 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f93280a33b0 0x7f93280a5870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9334022000 tx=0x7f933401e000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:37.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.689+0000 7f934661c740 1 --2- 172.21.15.103:0/1040016471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f8fb40 0x556ca9f8e090 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f9334014a60 tx=0x7f9334014f20 comp rx=0 tx=0).stop 2024-09-19T06:54:37.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.689+0000 7f934661c740 1 -- 172.21.15.103:0/1040016471 shutdown_connections 2024-09-19T06:54:37.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.689+0000 7f934661c740 1 --2- 172.21.15.103:0/1040016471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f9d780 0x556ca9f9fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.689+0000 7f934661c740 1 --2- 172.21.15.103:0/1040016471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8e5d0 0x556ca9f8ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.689+0000 7f934661c740 1 --2- 172.21.15.103:0/1040016471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f8fb40 0x556ca9f8e090 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.689+0000 7f934661c740 1 -- 172.21.15.103:0/1040016471 >> 172.21.15.103:0/1040016471 conn(0x556ca9f57310 msgr2=0x556ca9f8ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:37.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.689+0000 7f934661c740 1 -- 172.21.15.103:0/1040016471 shutdown_connections 2024-09-19T06:54:37.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.689+0000 7f934661c740 1 -- 172.21.15.103:0/1040016471 wait complete. 2024-09-19T06:54:37.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.689+0000 7f934661c740 1 -- start start 2024-09-19T06:54:37.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.689+0000 7f934661c740 1 Processor -- start 2024-09-19T06:54:37.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f934661c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8e5d0 0x556ca9f9d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f934661c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8fb40 0x556ca9f9a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f934661c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9f9ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f934661c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556ca9fa0480 con 0x556ca9f9d780 2024-09-19T06:54:37.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f934661c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556ca9eaf060 con 0x556ca9f8e5d0 2024-09-19T06:54:37.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f934661c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556ca9fa0600 con 0x556ca9f8fb40 2024-09-19T06:54:37.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f934661c740 10 client.0 _ll_register_callbacks cb 0x556ca9c028c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:37.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f93427e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9f9ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f93427e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9f9ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48694/0 (socket says 172.21.15.103:48694) 2024-09-19T06:54:37.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f93427e0640 1 -- 172.21.15.103:0/170986785 learned_addr learned my addr 172.21.15.103:0/170986785 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:37.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f9341fdf640 1 --2- 172.21.15.103:0/170986785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8e5d0 0x556ca9f9d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f93417de640 1 --2- 172.21.15.103:0/170986785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8fb40 0x556ca9f9a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f93427e0640 1 -- 172.21.15.103:0/170986785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8fb40 msgr2=0x556ca9f9a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:37.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f93427e0640 1 --2- 172.21.15.103:0/170986785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8fb40 0x556ca9f9a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f93427e0640 1 -- 172.21.15.103:0/170986785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8e5d0 msgr2=0x556ca9f9d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:37.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f93427e0640 1 --2- 172.21.15.103:0/170986785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8e5d0 0x556ca9f9d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f93427e0640 1 -- 172.21.15.103:0/170986785 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ca9c653c0 con 0x556ca9f9d780 2024-09-19T06:54:37.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f93427e0640 1 --2- 172.21.15.103:0/170986785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9f9ac10 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f9334000f80 tx=0x7f9334009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9ded15d5fa926e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:37.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f93257fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:37.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f93257fa640 1 -- 172.21.15.103:0/170986785 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334036060 con 0x556ca9f9d780 2024-09-19T06:54:37.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f93257fa640 1 -- 172.21.15.103:0/170986785 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f9334012c70 con 0x556ca9f9d780 2024-09-19T06:54:37.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.690+0000 7f934661c740 1 -- 172.21.15.103:0/170986785 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9334023070 con 0x556ca9f9d780 2024-09-19T06:54:37.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.691+0000 7f93257fa640 1 -- 172.21.15.103:0/170986785 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f933401db80 con 0x556ca9f9d780 2024-09-19T06:54:37.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.691+0000 7f934661c740 10 client.17394 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:37.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.691+0000 7f934661c740 1 -- 172.21.15.103:0/170986785 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556ca9fabf90 con 0x556ca9f9d780 2024-09-19T06:54:37.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.691+0000 7f93257fa640 1 -- 172.21.15.103:0/170986785 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9334023070 con 0x556ca9f9d780 2024-09-19T06:54:37.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.691+0000 7f93257fa640 1 -- 172.21.15.103:0/170986785 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f933400f040 con 0x556ca9f9d780 2024-09-19T06:54:37.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.691+0000 7f934661c740 10 client.17394 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:37.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.691+0000 7f934661c740 10 client.17394 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:54:37.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.691+0000 7f934661c740 20 client.17394 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:37.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.691+0000 7f934661c740 1 --2- 172.21.15.103:0/170986785 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x556ca9fad3f0 0x556ca9faf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.691+0000 7f934661c740 4 client.17394 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:37.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.691+0000 7f934661c740 1 -- 172.21.15.103:0/170986785 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ca9f01860 con 0x556ca9fad3f0 2024-09-19T06:54:37.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.691+0000 7f9341fdf640 1 --2- 172.21.15.103:0/170986785 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x556ca9fad3f0 0x556ca9faf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.692+0000 7f93257fa640 10 client.17394 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:54:37.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.692+0000 7f9341fdf640 1 --2- 172.21.15.103:0/170986785 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x556ca9fad3f0 0x556ca9faf7e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d6f0424b93633949 server_cookie=1165f18432107b4 in_seq=0 out_seq=0 2024-09-19T06:54:37.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.693+0000 7f93257fa640 1 -- 172.21.15.103:0/170986785 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ca9f01860 con 0x556ca9fad3f0 2024-09-19T06:54:37.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.693+0000 7f93257fa640 10 client.17394 handle_command_reply: tid=0 2024-09-19T06:54:37.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.693+0000 7f934661c740 1 client.17394 shutdown 2024-09-19T06:54:37.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.693+0000 7f934661c740 1 -- 172.21.15.103:0/170986785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 msgr2=0x556ca9f9ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:37.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.693+0000 7f934661c740 1 --2- 172.21.15.103:0/170986785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9f9ac10 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f9334000f80 tx=0x7f9334009b10 comp rx=0 tx=0).stop 2024-09-19T06:54:37.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.693+0000 7f934661c740 1 -- 172.21.15.103:0/170986785 shutdown_connections 2024-09-19T06:54:37.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.693+0000 7f934661c740 1 --2- 172.21.15.103:0/170986785 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x556ca9fad3f0 0x556ca9faf7e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.693+0000 7f934661c740 1 --2- 172.21.15.103:0/170986785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9f9ac10 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.693+0000 7f934661c740 1 --2- 172.21.15.103:0/170986785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8fb40 0x556ca9f9a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.693+0000 7f934661c740 1 --2- 172.21.15.103:0/170986785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8e5d0 0x556ca9f9d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.693+0000 7f93257fa640 0 client.17394 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:54:37.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.693+0000 7f934661c740 1 -- 172.21.15.103:0/170986785 >> 172.21.15.103:0/170986785 conn(0x556ca9f57310 msgr2=0x556ca9f90960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:37.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.693+0000 7f934661c740 1 -- 172.21.15.103:0/170986785 shutdown_connections 2024-09-19T06:54:37.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.693+0000 7f934661c740 1 -- 172.21.15.103:0/170986785 wait complete. 2024-09-19T06:54:37.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.693+0000 7f934661c740 20 client.17394 trim_cache size 0 max 16384 2024-09-19T06:54:37.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f934661c740 1 Processor -- start 2024-09-19T06:54:37.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f934661c740 1 -- start start 2024-09-19T06:54:37.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f934661c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8e5d0 0x556ca9f8adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f934661c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8fb40 0x556ca9f9a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f934661c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9f99460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f934661c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1ab7c0 con 0x556ca9f9d780 2024-09-19T06:54:37.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f934661c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1ad300 con 0x556ca9f8fb40 2024-09-19T06:54:37.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f934661c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1abe90 con 0x556ca9f8e5d0 2024-09-19T06:54:37.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f93427e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9f99460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f93427e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9f99460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48708/0 (socket says 172.21.15.103:48708) 2024-09-19T06:54:37.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f93427e0640 1 -- 172.21.15.103:0/2560967383 learned_addr learned my addr 172.21.15.103:0/2560967383 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:37.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f93417de640 1 --2- 172.21.15.103:0/2560967383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8fb40 0x556ca9f9a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f93427e0640 1 -- 172.21.15.103:0/2560967383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8e5d0 msgr2=0x556ca9f8adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:37.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f93427e0640 1 --2- 172.21.15.103:0/2560967383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8e5d0 0x556ca9f8adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f93427e0640 1 -- 172.21.15.103:0/2560967383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8fb40 msgr2=0x556ca9f9a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:37.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f93427e0640 1 --2- 172.21.15.103:0/2560967383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8fb40 0x556ca9f9a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f93427e0640 1 -- 172.21.15.103:0/2560967383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ca9f91120 con 0x556ca9f9d780 2024-09-19T06:54:37.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.706+0000 7f93417de640 1 --2- 172.21.15.103:0/2560967383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8fb40 0x556ca9f9a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:37.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.707+0000 7f93427e0640 1 --2- 172.21.15.103:0/2560967383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9f99460 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f9334005d70 tx=0x7f9334014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f24506535a4a671 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:37.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.707+0000 7f93277fe640 1 -- 172.21.15.103:0/2560967383 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334036070 con 0x556ca9f9d780 2024-09-19T06:54:37.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.707+0000 7f93277fe640 1 -- 172.21.15.103:0/2560967383 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9334023040 con 0x556ca9f9d780 2024-09-19T06:54:37.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.707+0000 7f93277fe640 1 -- 172.21.15.103:0/2560967383 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933401ee60 con 0x556ca9f9d780 2024-09-19T06:54:37.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.707+0000 7f934661c740 1 -- 172.21.15.103:0/2560967383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 msgr2=0x556ca9f99460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:37.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.707+0000 7f934661c740 1 --2- 172.21.15.103:0/2560967383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9f99460 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f9334005d70 tx=0x7f9334014f20 comp rx=0 tx=0).stop 2024-09-19T06:54:37.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.707+0000 7f934661c740 1 -- 172.21.15.103:0/2560967383 shutdown_connections 2024-09-19T06:54:37.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.707+0000 7f934661c740 1 --2- 172.21.15.103:0/2560967383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9f99460 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.707+0000 7f934661c740 1 --2- 172.21.15.103:0/2560967383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8fb40 0x556ca9f9a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.707+0000 7f934661c740 1 --2- 172.21.15.103:0/2560967383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8e5d0 0x556ca9f8adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.707+0000 7f934661c740 1 -- 172.21.15.103:0/2560967383 >> 172.21.15.103:0/2560967383 conn(0x556ca9f57310 msgr2=0x556ca9f98d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:37.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.707+0000 7f934661c740 1 -- 172.21.15.103:0/2560967383 shutdown_connections 2024-09-19T06:54:37.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.707+0000 7f934661c740 1 -- 172.21.15.103:0/2560967383 wait complete. 2024-09-19T06:54:37.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.708+0000 7f934661c740 1 -- start start 2024-09-19T06:54:37.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.708+0000 7f934661c740 1 Processor -- start 2024-09-19T06:54:37.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.708+0000 7f934661c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8e5d0 0x556ca9fa5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.708+0000 7f934661c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8fb40 0x556ca9fa1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.708+0000 7f934661c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9fa16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.708+0000 7f934661c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1acc30 con 0x556ca9f9d780 2024-09-19T06:54:37.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.708+0000 7f934661c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1ac560 con 0x556ca9f8fb40 2024-09-19T06:54:37.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.708+0000 7f934661c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1a9780 con 0x556ca9f8e5d0 2024-09-19T06:54:37.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.708+0000 7f934661c740 10 client.0 _ll_register_callbacks cb 0x7f9334014860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:37.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.708+0000 7f93427e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9fa16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.708+0000 7f93427e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9fa16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48724/0 (socket says 172.21.15.103:48724) 2024-09-19T06:54:37.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.708+0000 7f93427e0640 1 -- 172.21.15.103:0/539077497 learned_addr learned my addr 172.21.15.103:0/539077497 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:37.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f93417de640 1 --2- 172.21.15.103:0/539077497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8fb40 0x556ca9fa1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f9341fdf640 1 --2- 172.21.15.103:0/539077497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8e5d0 0x556ca9fa5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f93427e0640 1 -- 172.21.15.103:0/539077497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8e5d0 msgr2=0x556ca9fa5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:37.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f93427e0640 1 --2- 172.21.15.103:0/539077497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8e5d0 0x556ca9fa5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f93427e0640 1 -- 172.21.15.103:0/539077497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8fb40 msgr2=0x556ca9fa1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:37.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f93427e0640 1 --2- 172.21.15.103:0/539077497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8fb40 0x556ca9fa1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f93427e0640 1 -- 172.21.15.103:0/539077497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ca9fa1d70 con 0x556ca9f9d780 2024-09-19T06:54:37.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f93417de640 1 --2- 172.21.15.103:0/539077497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8fb40 0x556ca9fa1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:37.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f93427e0640 1 --2- 172.21.15.103:0/539077497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9fa16d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f9334014590 tx=0x7f933400e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99d91da8932aa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:37.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f93257fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:37.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f93257fa640 1 -- 172.21.15.103:0/539077497 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933401ee60 con 0x556ca9f9d780 2024-09-19T06:54:37.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f93257fa640 1 -- 172.21.15.103:0/539077497 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f9334002030 con 0x556ca9f9d780 2024-09-19T06:54:37.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f934661c740 1 -- 172.21.15.103:0/539077497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9334023040 con 0x556ca9f9d780 2024-09-19T06:54:37.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f93257fa640 1 -- 172.21.15.103:0/539077497 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9334036040 con 0x556ca9f9d780 2024-09-19T06:54:37.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f934661c740 10 client.17400 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:37.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f934661c740 1 -- 172.21.15.103:0/539077497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556caa017cb0 con 0x556ca9f9d780 2024-09-19T06:54:37.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.709+0000 7f93257fa640 1 -- 172.21.15.103:0/539077497 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9334023040 con 0x556ca9f9d780 2024-09-19T06:54:37.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.710+0000 7f93257fa640 1 -- 172.21.15.103:0/539077497 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f933400e670 con 0x556ca9f9d780 2024-09-19T06:54:37.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.710+0000 7f934661c740 10 client.17400 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:37.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.710+0000 7f934661c740 10 client.17400 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:54:37.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.710+0000 7f934661c740 20 client.17400 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:37.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.710+0000 7f934661c740 1 --2- 172.21.15.103:0/539077497 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x556caa018260 0x556caa01a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:37.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.710+0000 7f934661c740 4 client.17400 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:54:37.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.710+0000 7f934661c740 1 -- 172.21.15.103:0/539077497 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ca9efbd70 con 0x556caa018260 2024-09-19T06:54:37.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.710+0000 7f9341fdf640 1 --2- 172.21.15.103:0/539077497 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x556caa018260 0x556caa01a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:37.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.710+0000 7f9341fdf640 1 --2- 172.21.15.103:0/539077497 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x556caa018260 0x556caa01a650 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=982a6bb331626404 server_cookie=9f30e8858b223792 in_seq=0 out_seq=0 2024-09-19T06:54:37.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.710+0000 7f93257fa640 10 client.17400 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:54:37.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f93257fa640 1 -- 172.21.15.103:0/539077497 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ca9efbd70 con 0x556caa018260 2024-09-19T06:54:37.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f93257fa640 10 client.17400 handle_command_reply: tid=0 2024-09-19T06:54:37.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f934661c740 1 client.17400 shutdown 2024-09-19T06:54:37.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f934661c740 1 -- 172.21.15.103:0/539077497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 msgr2=0x556ca9fa16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:37.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f934661c740 1 --2- 172.21.15.103:0/539077497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9fa16d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f9334014590 tx=0x7f933400e5d0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f934661c740 1 -- 172.21.15.103:0/539077497 shutdown_connections 2024-09-19T06:54:37.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f934661c740 1 --2- 172.21.15.103:0/539077497 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x556caa018260 0x556caa01a650 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f934661c740 1 --2- 172.21.15.103:0/539077497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ca9f9d780 0x556ca9fa16d0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f934661c740 1 --2- 172.21.15.103:0/539077497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca9f8fb40 0x556ca9fa1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f93257fa640 0 client.17400 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:54:37.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f934661c740 1 --2- 172.21.15.103:0/539077497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ca9f8e5d0 0x556ca9fa5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f934661c740 1 -- 172.21.15.103:0/539077497 >> 172.21.15.103:0/539077497 conn(0x556ca9f57310 msgr2=0x556ca9f99570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:37.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f934661c740 1 -- 172.21.15.103:0/539077497 shutdown_connections 2024-09-19T06:54:37.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f934661c740 1 -- 172.21.15.103:0/539077497 wait complete. 2024-09-19T06:54:37.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f934661c740 20 client.17400 trim_cache size 0 max 16384 2024-09-19T06:54:37.718 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:54:37.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f9344243640 1 -- 172.21.15.103:0/2615530838 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f93280a33b0 msgr2=0x7f93280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:37.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f9344243640 1 --2- 172.21.15.103:0/2615530838 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f93280a33b0 0x7f93280a5870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9334022000 tx=0x7f933401e000 comp rx=0 tx=0).stop 2024-09-19T06:54:37.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f9344243640 1 -- 172.21.15.103:0/2615530838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f933c102790 msgr2=0x7f933c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:37.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.711+0000 7f9344243640 1 --2- 172.21.15.103:0/2615530838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f933c102790 0x7f933c198970 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f932c00d950 tx=0x7f932c00de10 comp rx=0 tx=0).stop 2024-09-19T06:54:37.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.712+0000 7f9344243640 1 -- 172.21.15.103:0/2615530838 shutdown_connections 2024-09-19T06:54:37.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.712+0000 7f9344243640 1 --2- 172.21.15.103:0/2615530838 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f93280a33b0 0x7f93280a5870 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.712+0000 7f9344243640 1 --2- 172.21.15.103:0/2615530838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f933c106f10 0x7f933c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.712+0000 7f9344243640 1 --2- 172.21.15.103:0/2615530838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f933c102790 0x7f933c198970 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.712+0000 7f9344243640 1 --2- 172.21.15.103:0/2615530838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f933c101de0 0x7f933c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:37.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.712+0000 7f9344243640 1 -- 172.21.15.103:0/2615530838 >> 172.21.15.103:0/2615530838 conn(0x7f933c0fd8c0 msgr2=0x7f933c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:37.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.712+0000 7f9344243640 1 -- 172.21.15.103:0/2615530838 shutdown_connections 2024-09-19T06:54:37.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:37.712+0000 7f9344243640 1 -- 172.21.15.103:0/2615530838 wait complete. 2024-09-19T06:54:37.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:37 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3330929245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:38.084 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:37 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3330929245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:38.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:37 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3330929245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:38.452 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id b06a23fc --release --await' 2024-09-19T06:54:38.452 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id b06a23fc --release --await 2024-09-19T06:54:38.861 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:54:38.937 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:39.060 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:38 smithi103 ceph-mon[31158]: pgmap v165: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-19T06:54:39.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:38 smithi174 ceph-mon[35637]: pgmap v165: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-19T06:54:39.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:38 smithi134 ceph-mon[35356]: pgmap v165: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-19T06:54:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.399+0000 7f3d0cf1f640 1 -- 172.21.15.103:0/1278127208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d081082a0 msgr2=0x7f3d0810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.399+0000 7f3d0cf1f640 1 --2- 172.21.15.103:0/1278127208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d081082a0 0x7f3d0810ec40 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009cd0 tx=0x7f3d0002a910 comp rx=0 tx=0).stop 2024-09-19T06:54:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.399+0000 7f3d0cf1f640 1 -- 172.21.15.103:0/1278127208 shutdown_connections 2024-09-19T06:54:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.399+0000 7f3d0cf1f640 1 --2- 172.21.15.103:0/1278127208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d081082a0 0x7f3d0810ec40 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.399+0000 7f3d0cf1f640 1 --2- 172.21.15.103:0/1278127208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d081036e0 0x7f3d08107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.399+0000 7f3d0cf1f640 1 --2- 172.21.15.103:0/1278127208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d08102d30 0x7f3d08103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.399+0000 7f3d0cf1f640 1 -- 172.21.15.103:0/1278127208 >> 172.21.15.103:0/1278127208 conn(0x7f3d080fc7d0 msgr2=0x7f3d080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.399+0000 7f3d0cf1f640 1 -- 172.21.15.103:0/1278127208 shutdown_connections 2024-09-19T06:54:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.400+0000 7f3d0cf1f640 1 -- 172.21.15.103:0/1278127208 wait complete. 2024-09-19T06:54:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.400+0000 7f3d0cf1f640 1 Processor -- start 2024-09-19T06:54:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.400+0000 7f3d0cf1f640 1 -- start start 2024-09-19T06:54:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.400+0000 7f3d0cf1f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d08102d30 0x7f3d081985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.400+0000 7f3d0cf1f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d081036e0 0x7f3d08198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.400+0000 7f3d0cf1f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d081082a0 0x7f3d0819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.400+0000 7f3d0cf1f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081120e0 con 0x7f3d081036e0 2024-09-19T06:54:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.400+0000 7f3d0cf1f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d08111f60 con 0x7f3d081082a0 2024-09-19T06:54:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.400+0000 7f3d0cf1f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3d08112260 con 0x7f3d08102d30 2024-09-19T06:54:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.401+0000 7f3d05d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d081036e0 0x7f3d08198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.401+0000 7f3d05d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d081036e0 0x7f3d08198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48730/0 (socket says 172.21.15.103:48730) 2024-09-19T06:54:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.401+0000 7f3d05d9b640 1 -- 172.21.15.103:0/2734705144 learned_addr learned my addr 172.21.15.103:0/2734705144 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.401+0000 7f3d0659c640 1 --2- 172.21.15.103:0/2734705144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d08102d30 0x7f3d081985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.401+0000 7f3d06d9d640 1 --2- 172.21.15.103:0/2734705144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d081082a0 0x7f3d0819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.401+0000 7f3d05d9b640 1 -- 172.21.15.103:0/2734705144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d08102d30 msgr2=0x7f3d081985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.401+0000 7f3d05d9b640 1 --2- 172.21.15.103:0/2734705144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d08102d30 0x7f3d081985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.401+0000 7f3d05d9b640 1 -- 172.21.15.103:0/2734705144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d081082a0 msgr2=0x7f3d0819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.401+0000 7f3d05d9b640 1 --2- 172.21.15.103:0/2734705144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d081082a0 0x7f3d0819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.401+0000 7f3d05d9b640 1 -- 172.21.15.103:0/2734705144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0819d590 con 0x7f3d081036e0 2024-09-19T06:54:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.401+0000 7f3d05d9b640 1 --2- 172.21.15.103:0/2734705144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d081036e0 0x7f3d08198b20 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc006fd0 tx=0x7f3cfc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.401+0000 7f3cf77fe640 1 -- 172.21.15.103:0/2734705144 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc011830 con 0x7f3d081036e0 2024-09-19T06:54:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.401+0000 7f3cf77fe640 1 -- 172.21.15.103:0/2734705144 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cfc016070 con 0x7f3d081036e0 2024-09-19T06:54:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.401+0000 7f3cf77fe640 1 -- 172.21.15.103:0/2734705144 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00f2e0 con 0x7f3d081036e0 2024-09-19T06:54:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.401+0000 7f3d0cf1f640 1 -- 172.21.15.103:0/2734705144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0819d880 con 0x7f3d081036e0 2024-09-19T06:54:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.401+0000 7f3d0cf1f640 1 -- 172.21.15.103:0/2734705144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d081a5160 con 0x7f3d081036e0 2024-09-19T06:54:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.404+0000 7f3cf77fe640 1 -- 172.21.15.103:0/2734705144 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3cfc01b460 con 0x7f3d081036e0 2024-09-19T06:54:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.404+0000 7f3d0f2f8740 1 Processor -- start 2024-09-19T06:54:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.404+0000 7f3d0f2f8740 1 -- start start 2024-09-19T06:54:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.404+0000 7f3cf77fe640 1 --2- 172.21.15.103:0/2734705144 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cf00a33b0 0x7f3cf00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.404+0000 7f3d0f2f8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9cc0060 0x55c6c9cc0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.404+0000 7f3d0f2f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9ccad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.404+0000 7f3d0f2f8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9ccb240 0x55c6c9ccd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.404+0000 7f3d0f2f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c6c9c25620 con 0x55c6c9cc0980 2024-09-19T06:54:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.404+0000 7f3d0f2f8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c6c9bcce50 con 0x55c6c9ccb240 2024-09-19T06:54:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.404+0000 7f3d0f2f8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c6c9c215e0 con 0x55c6c9cc0060 2024-09-19T06:54:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.405+0000 7f3d05d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9ccad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.405+0000 7f3d06d9d640 1 --2- 172.21.15.103:0/2734705144 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cf00a33b0 0x7f3cf00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.405+0000 7f3d05d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9ccad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48736/0 (socket says 172.21.15.103:48736) 2024-09-19T06:54:39.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.405+0000 7f3d05d9b640 1 -- 172.21.15.103:0/4129549986 learned_addr learned my addr 172.21.15.103:0/4129549986 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:39.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.405+0000 7f3d0659c640 1 --2- 172.21.15.103:0/4129549986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9cc0060 0x55c6c9cc0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:39.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.405+0000 7f3cf77fe640 1 -- 172.21.15.103:0/2734705144 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f3cfc0cf2a0 con 0x7f3d081036e0 2024-09-19T06:54:39.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.405+0000 7f3d06d9d640 1 --2- 172.21.15.103:0/4129549986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9ccb240 0x55c6c9ccd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:39.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.405+0000 7f3d0659c640 1 -- 172.21.15.103:0/4129549986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9ccb240 msgr2=0x55c6c9ccd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:39.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.405+0000 7f3d0659c640 1 --2- 172.21.15.103:0/4129549986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9ccb240 0x55c6c9ccd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.405+0000 7f3d0659c640 1 -- 172.21.15.103:0/4129549986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 msgr2=0x55c6c9ccad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:39.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.405+0000 7f3d0659c640 1 --2- 172.21.15.103:0/4129549986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9ccad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.405+0000 7f3d0659c640 1 -- 172.21.15.103:0/4129549986 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c9bbdd40 con 0x55c6c9cc0060 2024-09-19T06:54:39.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.405+0000 7f3d05d9b640 1 --2- 172.21.15.103:0/4129549986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9ccad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:39.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.405+0000 7f3d06d9d640 1 --2- 172.21.15.103:0/2734705144 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cf00a33b0 0x7f3cf00a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d00005fe0 tx=0x7f3d00031b80 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.405+0000 7f3d0659c640 1 --2- 172.21.15.103:0/4129549986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9cc0060 0x55c6c9cc0440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800bb70 tx=0x7f3cf80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5d346d60c55b9bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.405+0000 7f3cf4ff9640 1 -- 172.21.15.103:0/4129549986 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8004030 con 0x55c6c9cc0060 2024-09-19T06:54:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.406+0000 7f3cf4ff9640 1 -- 172.21.15.103:0/4129549986 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cf80041d0 con 0x55c6c9cc0060 2024-09-19T06:54:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.406+0000 7f3cf4ff9640 1 -- 172.21.15.103:0/4129549986 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf80116a0 con 0x55c6c9cc0060 2024-09-19T06:54:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.406+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/4129549986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9cc0060 msgr2=0x55c6c9cc0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.406+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/4129549986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9cc0060 0x55c6c9cc0440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800bb70 tx=0x7f3cf80107a0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.406+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/4129549986 shutdown_connections 2024-09-19T06:54:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.406+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/4129549986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9ccb240 0x55c6c9ccd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.406+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/4129549986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9ccad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.406+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/4129549986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9cc0060 0x55c6c9cc0440 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.406+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/4129549986 >> 172.21.15.103:0/4129549986 conn(0x55c6c9cbc5f0 msgr2=0x55c6c9cbc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:39.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.406+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/4129549986 shutdown_connections 2024-09-19T06:54:39.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.406+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/4129549986 wait complete. 2024-09-19T06:54:39.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.407+0000 7f3d0f2f8740 1 -- start start 2024-09-19T06:54:39.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.407+0000 7f3d0f2f8740 1 Processor -- start 2024-09-19T06:54:39.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.407+0000 7f3d0f2f8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9cc0060 0x55c6c9cc8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.407+0000 7f3d0f2f8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9cc0980 0x55c6c9cc8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.407+0000 7f3d0f2f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9ccb240 0x55c6c9cc8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.407+0000 7f3d0f2f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c6c9ccde10 con 0x55c6c9ccb240 2024-09-19T06:54:39.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.407+0000 7f3d0f2f8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c6c9badf10 con 0x55c6c9cc0060 2024-09-19T06:54:39.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.407+0000 7f3d0f2f8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c6c9ccdf90 con 0x55c6c9cc0980 2024-09-19T06:54:39.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.407+0000 7f3d0f2f8740 10 client.0 _ll_register_callbacks cb 0x55c6c9bd1db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:39.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.407+0000 7f3d06d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9ccb240 0x55c6c9cc8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:39.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.407+0000 7f3d06d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9ccb240 0x55c6c9cc8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48738/0 (socket says 172.21.15.103:48738) 2024-09-19T06:54:39.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.407+0000 7f3d06d9d640 1 -- 172.21.15.103:0/3012463647 learned_addr learned my addr 172.21.15.103:0/3012463647 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:39.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.407+0000 7f3d05d9b640 1 --2- 172.21.15.103:0/3012463647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9cc0980 0x55c6c9cc8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:39.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.407+0000 7f3d0659c640 1 --2- 172.21.15.103:0/3012463647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9cc0060 0x55c6c9cc8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:39.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.408+0000 7f3d06d9d640 1 -- 172.21.15.103:0/3012463647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9cc0980 msgr2=0x55c6c9cc8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:39.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.408+0000 7f3d06d9d640 1 --2- 172.21.15.103:0/3012463647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9cc0980 0x55c6c9cc8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.408+0000 7f3d06d9d640 1 -- 172.21.15.103:0/3012463647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9cc0060 msgr2=0x55c6c9cc8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:39.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.408+0000 7f3d06d9d640 1 --2- 172.21.15.103:0/3012463647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9cc0060 0x55c6c9cc8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.408+0000 7f3d06d9d640 1 -- 172.21.15.103:0/3012463647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c9934580 con 0x55c6c9ccb240 2024-09-19T06:54:39.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.408+0000 7f3d06d9d640 1 --2- 172.21.15.103:0/3012463647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9ccb240 0x55c6c9cc8bd0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009cd0 tx=0x7f3d0002a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60fd8c38c80d1d90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:39.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.408+0000 7f3cedffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:39.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.408+0000 7f3cedffb640 1 -- 172.21.15.103:0/3012463647 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00031e10 con 0x55c6c9ccb240 2024-09-19T06:54:39.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.408+0000 7f3cedffb640 1 -- 172.21.15.103:0/3012463647 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f3d00004b40 con 0x55c6c9ccb240 2024-09-19T06:54:39.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.408+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/3012463647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cf80041d0 con 0x55c6c9ccb240 2024-09-19T06:54:39.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.408+0000 7f3cedffb640 1 -- 172.21.15.103:0/3012463647 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d00058050 con 0x55c6c9ccb240 2024-09-19T06:54:39.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.408+0000 7f3d0f2f8740 10 client.17412 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:39.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.408+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/3012463647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c6c99fe960 con 0x55c6c9ccb240 2024-09-19T06:54:39.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.408+0000 7f3cedffb640 1 -- 172.21.15.103:0/3012463647 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cf80041d0 con 0x55c6c9ccb240 2024-09-19T06:54:39.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.408+0000 7f3cedffb640 1 -- 172.21.15.103:0/3012463647 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3d0002fc20 con 0x55c6c9ccb240 2024-09-19T06:54:39.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.409+0000 7f3d0f2f8740 10 client.17412 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:39.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.409+0000 7f3d0f2f8740 10 client.17412 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:54:39.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.409+0000 7f3d0f2f8740 20 client.17412 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:39.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.409+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/3012463647 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c6c9cdade0 0x55c6c9cdd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.409+0000 7f3d0f2f8740 4 client.17412 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:39.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.409+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/3012463647 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6c9c32b90 con 0x55c6c9cdade0 2024-09-19T06:54:39.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.409+0000 7f3d0659c640 1 --2- 172.21.15.103:0/3012463647 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c6c9cdade0 0x55c6c9cdd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:39.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.409+0000 7f3cedffb640 10 client.17412 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:39.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.409+0000 7f3d0659c640 1 --2- 172.21.15.103:0/3012463647 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c6c9cdade0 0x55c6c9cdd280 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54ddccb407dea5ae server_cookie=56e885a01b98dffe in_seq=0 out_seq=0 2024-09-19T06:54:39.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.410+0000 7f3cedffb640 1 -- 172.21.15.103:0/3012463647 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c6c9c32b90 con 0x55c6c9cdade0 2024-09-19T06:54:39.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.410+0000 7f3cedffb640 10 client.17412 handle_command_reply: tid=0 2024-09-19T06:54:39.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.410+0000 7f3d0f2f8740 1 client.17412 shutdown 2024-09-19T06:54:39.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.410+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/3012463647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9ccb240 msgr2=0x55c6c9cc8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:39.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.410+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/3012463647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9ccb240 0x55c6c9cc8bd0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009cd0 tx=0x7f3d0002a910 comp rx=0 tx=0).stop 2024-09-19T06:54:39.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.410+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/3012463647 shutdown_connections 2024-09-19T06:54:39.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.410+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/3012463647 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c6c9cdade0 0x55c6c9cdd280 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.410+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/3012463647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9ccb240 0x55c6c9cc8bd0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.410+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/3012463647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9cc0980 0x55c6c9cc8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.410+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/3012463647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9cc0060 0x55c6c9cc8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.410+0000 7f3cedffb640 0 client.17412 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:39.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.410+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/3012463647 >> 172.21.15.103:0/3012463647 conn(0x55c6c9cbc5f0 msgr2=0x55c6c9cbb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:39.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.410+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/3012463647 shutdown_connections 2024-09-19T06:54:39.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.410+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/3012463647 wait complete. 2024-09-19T06:54:39.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.410+0000 7f3d0f2f8740 20 client.17412 trim_cache size 0 max 16384 2024-09-19T06:54:39.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d0f2f8740 1 Processor -- start 2024-09-19T06:54:39.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d0f2f8740 1 -- start start 2024-09-19T06:54:39.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d0f2f8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9cc0060 0x55c6c9cc2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d0f2f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9cbb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d0f2f8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9ccb240 0x55c6c9ce2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d0f2f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081ad9a0 con 0x55c6c9cc0980 2024-09-19T06:54:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d0f2f8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081af4e0 con 0x55c6c9ccb240 2024-09-19T06:54:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d0f2f8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081ae070 con 0x55c6c9cc0060 2024-09-19T06:54:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d05d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9cbb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d05d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9cbb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48752/0 (socket says 172.21.15.103:48752) 2024-09-19T06:54:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d05d9b640 1 -- 172.21.15.103:0/650736056 learned_addr learned my addr 172.21.15.103:0/650736056 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d05d9b640 1 -- 172.21.15.103:0/650736056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9cc0060 msgr2=0x55c6c9cc2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d05d9b640 1 --2- 172.21.15.103:0/650736056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9cc0060 0x55c6c9cc2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d05d9b640 1 -- 172.21.15.103:0/650736056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9ccb240 msgr2=0x55c6c9ce2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:39.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d05d9b640 1 --2- 172.21.15.103:0/650736056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9ccb240 0x55c6c9ce2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d05d9b640 1 -- 172.21.15.103:0/650736056 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c9ccaac0 con 0x55c6c9cc0980 2024-09-19T06:54:39.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.427+0000 7f3d05d9b640 1 --2- 172.21.15.103:0/650736056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9cbb640 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c00 tx=0x7f3cfc0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f742fb5ec002163a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:39.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.428+0000 7f3ceffff640 1 -- 172.21.15.103:0/650736056 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc0ae3c0 con 0x55c6c9cc0980 2024-09-19T06:54:39.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.428+0000 7f3ceffff640 1 -- 172.21.15.103:0/650736056 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cfc09bda0 con 0x55c6c9cc0980 2024-09-19T06:54:39.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.428+0000 7f3ceffff640 1 -- 172.21.15.103:0/650736056 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc0ae1e0 con 0x55c6c9cc0980 2024-09-19T06:54:39.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.428+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/650736056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 msgr2=0x55c6c9cbb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:39.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.428+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/650736056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9cbb640 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c00 tx=0x7f3cfc0108e0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.428+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/650736056 shutdown_connections 2024-09-19T06:54:39.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.428+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/650736056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9ccb240 0x55c6c9ce2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.428+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/650736056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9cbb640 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.428+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/650736056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9cc0060 0x55c6c9cc2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.428+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/650736056 >> 172.21.15.103:0/650736056 conn(0x55c6c9cbc5f0 msgr2=0x55c6c9cbaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:39.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.428+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/650736056 shutdown_connections 2024-09-19T06:54:39.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.428+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/650736056 wait complete. 2024-09-19T06:54:39.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d0f2f8740 1 -- start start 2024-09-19T06:54:39.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d0f2f8740 1 Processor -- start 2024-09-19T06:54:39.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d0f2f8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9cc0060 0x55c6c9cd3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d0f2f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9ccfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d0f2f8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9ccb240 0x55c6c9cd01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d0f2f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081aee10 con 0x55c6c9cc0980 2024-09-19T06:54:39.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d0f2f8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081ae740 con 0x55c6c9cc0060 2024-09-19T06:54:39.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d0f2f8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081ab960 con 0x55c6c9ccb240 2024-09-19T06:54:39.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d0f2f8740 10 client.0 _ll_register_callbacks cb 0x7f3d00003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:39.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d05d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9ccfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:39.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d05d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9ccfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48764/0 (socket says 172.21.15.103:48764) 2024-09-19T06:54:39.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d05d9b640 1 -- 172.21.15.103:0/96004358 learned_addr learned my addr 172.21.15.103:0/96004358 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:39.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d0659c640 1 --2- 172.21.15.103:0/96004358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9cc0060 0x55c6c9cd3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:39.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d06d9d640 1 --2- 172.21.15.103:0/96004358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9ccb240 0x55c6c9cd01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:39.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d05d9b640 1 -- 172.21.15.103:0/96004358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9ccb240 msgr2=0x55c6c9cd01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:39.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d05d9b640 1 --2- 172.21.15.103:0/96004358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9ccb240 0x55c6c9cd01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d05d9b640 1 -- 172.21.15.103:0/96004358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9cc0060 msgr2=0x55c6c9cd3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:39.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d05d9b640 1 --2- 172.21.15.103:0/96004358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9cc0060 0x55c6c9cd3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d05d9b640 1 -- 172.21.15.103:0/96004358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c9cd09d0 con 0x55c6c9cc0980 2024-09-19T06:54:39.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d06d9d640 1 --2- 172.21.15.103:0/96004358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9ccb240 0x55c6c9cd01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:39.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3d05d9b640 1 --2- 172.21.15.103:0/96004358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9ccfc60 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00bde0 tx=0x7f3cfc0ae0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6e045f380f72c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:39.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.429+0000 7f3cedffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:39.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.430+0000 7f3cedffb640 1 -- 172.21.15.103:0/96004358 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc0ac060 con 0x55c6c9cc0980 2024-09-19T06:54:39.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.430+0000 7f3cedffb640 1 -- 172.21.15.103:0/96004358 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f3cfc0ad3d0 con 0x55c6c9cc0980 2024-09-19T06:54:39.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.430+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/96004358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cfc09bda0 con 0x55c6c9cc0980 2024-09-19T06:54:39.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.430+0000 7f3cedffb640 1 -- 172.21.15.103:0/96004358 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3cfc0a4310 con 0x55c6c9cc0980 2024-09-19T06:54:39.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.430+0000 7f3d0f2f8740 10 client.17418 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:39.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.430+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/96004358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c6c9d46bb0 con 0x55c6c9cc0980 2024-09-19T06:54:39.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.430+0000 7f3cedffb640 1 -- 172.21.15.103:0/96004358 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cfc09bda0 con 0x55c6c9cc0980 2024-09-19T06:54:39.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.430+0000 7f3cedffb640 1 -- 172.21.15.103:0/96004358 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3cfc0a8ce0 con 0x55c6c9cc0980 2024-09-19T06:54:39.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.430+0000 7f3d0f2f8740 10 client.17418 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:39.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.430+0000 7f3d0f2f8740 10 client.17418 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:54:39.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.430+0000 7f3d0f2f8740 20 client.17418 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:39.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.430+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/96004358 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c6c9d470c0 0x55c6c9d494e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:39.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.430+0000 7f3d0f2f8740 4 client.17418 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b06a23fc", "release": true, "await": true}] 2024-09-19T06:54:39.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.430+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/96004358 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b06a23fc", "release": true, "await": true}) -- 0x55c6c9c2d0a0 con 0x55c6c9d470c0 2024-09-19T06:54:39.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.431+0000 7f3d0659c640 1 --2- 172.21.15.103:0/96004358 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c6c9d470c0 0x55c6c9d494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:39.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.431+0000 7f3cedffb640 10 client.17418 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:39.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.431+0000 7f3d0659c640 1 --2- 172.21.15.103:0/96004358 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c6c9d470c0 0x55c6c9d494e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbc6ef7d341192a5 server_cookie=ad5b3851ca32bb11 in_seq=0 out_seq=0 2024-09-19T06:54:39.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.434+0000 7f3cedffb640 1 -- 172.21.15.103:0/96004358 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c6c9c2d0a0 con 0x55c6c9d470c0 2024-09-19T06:54:39.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.434+0000 7f3cedffb640 10 client.17418 handle_command_reply: tid=0 2024-09-19T06:54:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.434+0000 7f3d0f2f8740 1 client.17418 shutdown 2024-09-19T06:54:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.434+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/96004358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 msgr2=0x55c6c9ccfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.434+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/96004358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9ccfc60 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00bde0 tx=0x7f3cfc0ae0f0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.434+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/96004358 shutdown_connections 2024-09-19T06:54:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.434+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/96004358 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c6c9d470c0 0x55c6c9d494e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.434+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/96004358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6c9ccb240 0x55c6c9cd01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.434+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/96004358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6c9cc0980 0x55c6c9ccfc60 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.434+0000 7f3cedffb640 0 client.17418 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.434+0000 7f3d0f2f8740 1 --2- 172.21.15.103:0/96004358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6c9cc0060 0x55c6c9cd3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.434+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/96004358 >> 172.21.15.103:0/96004358 conn(0x55c6c9cbc5f0 msgr2=0x55c6c9cc8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.434+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/96004358 shutdown_connections 2024-09-19T06:54:39.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.435+0000 7f3d0f2f8740 1 -- 172.21.15.103:0/96004358 wait complete. 2024-09-19T06:54:39.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.435+0000 7f3d0f2f8740 20 client.17418 trim_cache size 0 max 16384 2024-09-19T06:54:39.459 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:54:39.459 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:54:39.459 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:54:39.459 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 24, 2024-09-19T06:54:39.459 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:54:39.459 INFO:teuthology.orchestra.run.smithi103.stdout: "b06a23fc": { 2024-09-19T06:54:39.460 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 24, 2024-09-19T06:54:39.460 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:54:39.460 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:54:39.460 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:54:39.460 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:54:39.460 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:54:39.460 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:54:39.460 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 149.8, 2024-09-19T06:54:39.460 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:54:39.460 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:54:39.460 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:54:39.460 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:54:39.461 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:54:39.461 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:54:39.461 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:39.461 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:39.461 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:39.461 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:39.461 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:39.461 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:54:39.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.435+0000 7f3d0cf1f640 1 -- 172.21.15.103:0/2734705144 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cf00a33b0 msgr2=0x7f3cf00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:39.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.435+0000 7f3d0cf1f640 1 --2- 172.21.15.103:0/2734705144 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cf00a33b0 0x7f3cf00a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d00005fe0 tx=0x7f3d00031b80 comp rx=0 tx=0).stop 2024-09-19T06:54:39.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.435+0000 7f3d0cf1f640 1 -- 172.21.15.103:0/2734705144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d081036e0 msgr2=0x7f3d08198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:39.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.435+0000 7f3d0cf1f640 1 --2- 172.21.15.103:0/2734705144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d081036e0 0x7f3d08198b20 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc006fd0 tx=0x7f3cfc00b8e0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.435+0000 7f3d0cf1f640 1 -- 172.21.15.103:0/2734705144 shutdown_connections 2024-09-19T06:54:39.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.435+0000 7f3d0cf1f640 1 --2- 172.21.15.103:0/2734705144 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cf00a33b0 0x7f3cf00a5870 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.435+0000 7f3d0cf1f640 1 --2- 172.21.15.103:0/2734705144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d081082a0 0x7f3d0819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.435+0000 7f3d0cf1f640 1 --2- 172.21.15.103:0/2734705144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d081036e0 0x7f3d08198b20 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.435+0000 7f3d0cf1f640 1 --2- 172.21.15.103:0/2734705144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d08102d30 0x7f3d081985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:39.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.435+0000 7f3d0cf1f640 1 -- 172.21.15.103:0/2734705144 >> 172.21.15.103:0/2734705144 conn(0x7f3d080fc7d0 msgr2=0x7f3d08106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:39.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.435+0000 7f3d0cf1f640 1 -- 172.21.15.103:0/2734705144 shutdown_connections 2024-09-19T06:54:39.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:39.435+0000 7f3d0cf1f640 1 -- 172.21.15.103:0/2734705144 wait complete. 2024-09-19T06:54:39.664 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:40.155 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b06a23fc', total seconds elapsed: 33.7 2024-09-19T06:54:40.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:40 smithi174 ceph-mon[35637]: pgmap v166: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-09-19T06:54:40.336 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:40 smithi103 ceph-mon[31158]: pgmap v166: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-09-19T06:54:40.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:40 smithi134 ceph-mon[35356]: pgmap v166: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-09-19T06:54:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.616+0000 7f5dd23f6640 1 -- 172.21.15.103:0/3976857666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dcc1009b0 msgr2=0x7f5dcc100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.616+0000 7f5dd23f6640 1 --2- 172.21.15.103:0/3976857666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dcc1009b0 0x7f5dcc100e10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009930 tx=0x7f5dc002a420 comp rx=0 tx=0).stop 2024-09-19T06:54:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.616+0000 7f5dd23f6640 1 -- 172.21.15.103:0/3976857666 shutdown_connections 2024-09-19T06:54:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.616+0000 7f5dd23f6640 1 --2- 172.21.15.103:0/3976857666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dcc101350 0x7f5dcc10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.616+0000 7f5dd23f6640 1 --2- 172.21.15.103:0/3976857666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dcc1009b0 0x7f5dcc100e10 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.616+0000 7f5dd23f6640 1 --2- 172.21.15.103:0/3976857666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dcc1069c0 0x7f5dcc106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.616+0000 7f5dd23f6640 1 -- 172.21.15.103:0/3976857666 >> 172.21.15.103:0/3976857666 conn(0x7f5dcc077990 msgr2=0x7f5dcc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.616+0000 7f5dd23f6640 1 -- 172.21.15.103:0/3976857666 shutdown_connections 2024-09-19T06:54:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.616+0000 7f5dd23f6640 1 -- 172.21.15.103:0/3976857666 wait complete. 2024-09-19T06:54:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.617+0000 7f5dd23f6640 1 Processor -- start 2024-09-19T06:54:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.617+0000 7f5dd23f6640 1 -- start start 2024-09-19T06:54:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.617+0000 7f5dd23f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dcc1009b0 0x7f5dcc197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.617+0000 7f5dd23f6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dcc101350 0x7f5dcc197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.617+0000 7f5dd23f6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dcc1069c0 0x7f5dcc19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.617+0000 7f5dd23f6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc110e50 con 0x7f5dcc1009b0 2024-09-19T06:54:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.617+0000 7f5dd23f6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc110cd0 con 0x7f5dcc101350 2024-09-19T06:54:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.617+0000 7f5dd23f6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc110fd0 con 0x7f5dcc1069c0 2024-09-19T06:54:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.618+0000 7f5dcbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dcc1009b0 0x7f5dcc197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.618+0000 7f5dcbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dcc1009b0 0x7f5dcc197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48782/0 (socket says 172.21.15.103:48782) 2024-09-19T06:54:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.618+0000 7f5dcbfff640 1 -- 172.21.15.103:0/41439023 learned_addr learned my addr 172.21.15.103:0/41439023 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.618+0000 7f5dcb7fe640 1 --2- 172.21.15.103:0/41439023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dcc101350 0x7f5dcc197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.618+0000 7f5dd0993640 1 --2- 172.21.15.103:0/41439023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dcc1069c0 0x7f5dcc19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.618+0000 7f5dcbfff640 1 -- 172.21.15.103:0/41439023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dcc1069c0 msgr2=0x7f5dcc19bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.618+0000 7f5dcbfff640 1 --2- 172.21.15.103:0/41439023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dcc1069c0 0x7f5dcc19bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.618+0000 7f5dcbfff640 1 -- 172.21.15.103:0/41439023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dcc101350 msgr2=0x7f5dcc197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.618+0000 7f5dcbfff640 1 --2- 172.21.15.103:0/41439023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dcc101350 0x7f5dcc197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.618+0000 7f5dcbfff640 1 -- 172.21.15.103:0/41439023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dcc19c310 con 0x7f5dcc1009b0 2024-09-19T06:54:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.618+0000 7f5dcbfff640 1 --2- 172.21.15.103:0/41439023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dcc1009b0 0x7f5dcc197320 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00d950 tx=0x7f5dbc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.618+0000 7f5dc97fa640 1 -- 172.21.15.103:0/41439023 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc017070 con 0x7f5dcc1009b0 2024-09-19T06:54:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.618+0000 7f5dd23f6640 1 -- 172.21.15.103:0/41439023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dcc19c600 con 0x7f5dcc1009b0 2024-09-19T06:54:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.618+0000 7f5dc97fa640 1 -- 172.21.15.103:0/41439023 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dbc009c30 con 0x7f5dcc1009b0 2024-09-19T06:54:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.618+0000 7f5dc97fa640 1 -- 172.21.15.103:0/41439023 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc00f6b0 con 0x7f5dcc1009b0 2024-09-19T06:54:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.618+0000 7f5dd23f6640 1 -- 172.21.15.103:0/41439023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dcc1a3e90 con 0x7f5dcc1009b0 2024-09-19T06:54:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dd526c740 1 Processor -- start 2024-09-19T06:54:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dd526c740 1 -- start start 2024-09-19T06:54:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dd526c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455e4a10 0x5630455e4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dd526c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455e5330 0x5630455f1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dd526c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455f1ec0 0x5630455f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dd526c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5630455492f0 con 0x5630455e4a10 2024-09-19T06:54:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dd526c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5630454f1270 con 0x5630455e5330 2024-09-19T06:54:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dd526c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5630455452b0 con 0x5630455f1ec0 2024-09-19T06:54:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dd0993640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455e4a10 0x5630455e4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dd0993640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455e4a10 0x5630455e4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48798/0 (socket says 172.21.15.103:48798) 2024-09-19T06:54:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dd0993640 1 -- 172.21.15.103:0/1561136878 learned_addr learned my addr 172.21.15.103:0/1561136878 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dc97fa640 1 -- 172.21.15.103:0/41439023 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f5dbc00f850 con 0x7f5dcc1009b0 2024-09-19T06:54:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dd0993640 1 -- 172.21.15.103:0/1561136878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455f1ec0 msgr2=0x5630455f42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dd0993640 1 --2- 172.21.15.103:0/1561136878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455f1ec0 0x5630455f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dd0993640 1 -- 172.21.15.103:0/1561136878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455e5330 msgr2=0x5630455f1980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:54:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dd0993640 1 --2- 172.21.15.103:0/1561136878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455e5330 0x5630455f1980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.621+0000 7f5dd0993640 1 -- 172.21.15.103:0/1561136878 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630454f0340 con 0x5630455e4a10 2024-09-19T06:54:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.622+0000 7f5dd0993640 1 --2- 172.21.15.103:0/1561136878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455e4a10 0x5630455e4df0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4010370 tx=0x7f5dc4010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e1f8e04db03a2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.622+0000 7f5db27fc640 1 -- 172.21.15.103:0/1561136878 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4016070 con 0x5630455e4a10 2024-09-19T06:54:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.622+0000 7f5db27fc640 1 -- 172.21.15.103:0/1561136878 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dc4004d20 con 0x5630455e4a10 2024-09-19T06:54:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.622+0000 7f5db27fc640 1 -- 172.21.15.103:0/1561136878 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40053e0 con 0x5630455e4a10 2024-09-19T06:54:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.622+0000 7f5dc97fa640 1 --2- 172.21.15.103:0/41439023 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5db40a3460 0x7f5db40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.622+0000 7f5dcbfff640 1 --2- 172.21.15.103:0/41439023 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5db40a3460 0x7f5db40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.622+0000 7f5dc97fa640 1 -- 172.21.15.103:0/41439023 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5dbc0cf810 con 0x7f5dcc1009b0 2024-09-19T06:54:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.623+0000 7f5dd526c740 1 -- 172.21.15.103:0/1561136878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455e4a10 msgr2=0x5630455e4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.623+0000 7f5dd526c740 1 --2- 172.21.15.103:0/1561136878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455e4a10 0x5630455e4df0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4010370 tx=0x7f5dc4010830 comp rx=0 tx=0).stop 2024-09-19T06:54:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.623+0000 7f5dcbfff640 1 --2- 172.21.15.103:0/41439023 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5db40a3460 0x7f5db40a5920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc098580 tx=0x7f5dbc086040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.623+0000 7f5dd526c740 1 -- 172.21.15.103:0/1561136878 shutdown_connections 2024-09-19T06:54:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.623+0000 7f5dd526c740 1 --2- 172.21.15.103:0/1561136878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455f1ec0 0x5630455f42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.623+0000 7f5dd526c740 1 --2- 172.21.15.103:0/1561136878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455e5330 0x5630455f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.623+0000 7f5dd526c740 1 --2- 172.21.15.103:0/1561136878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455e4a10 0x5630455e4df0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.623+0000 7f5dd526c740 1 -- 172.21.15.103:0/1561136878 >> 172.21.15.103:0/1561136878 conn(0x5630455e1060 msgr2=0x5630455e1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.623+0000 7f5dd526c740 1 -- 172.21.15.103:0/1561136878 shutdown_connections 2024-09-19T06:54:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.623+0000 7f5dd526c740 1 -- 172.21.15.103:0/1561136878 wait complete. 2024-09-19T06:54:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.623+0000 7f5dd526c740 1 -- start start 2024-09-19T06:54:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.623+0000 7f5dd526c740 1 Processor -- start 2024-09-19T06:54:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dd526c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455e4a10 0x5630455eee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dd526c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455e5330 0x5630455ef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dd526c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455f1ec0 0x5630455ef890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dd526c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5630455f4a90 con 0x5630455f1ec0 2024-09-19T06:54:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dd526c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5630454d2290 con 0x5630455e5330 2024-09-19T06:54:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dd526c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5630455f4c10 con 0x5630455e4a10 2024-09-19T06:54:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dd526c740 10 client.0 _ll_register_callbacks cb 0x563045503140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dd0993640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455f1ec0 0x5630455ef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dcb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455e5330 0x5630455ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dd0993640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455f1ec0 0x5630455ef890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48808/0 (socket says 172.21.15.103:48808) 2024-09-19T06:54:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dd0993640 1 -- 172.21.15.103:0/339530794 learned_addr learned my addr 172.21.15.103:0/339530794 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dd0993640 1 --2- 172.21.15.103:0/339530794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455e4a10 0x5630455eee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dcb7fe640 1 -- 172.21.15.103:0/339530794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455e4a10 msgr2=0x5630455eee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dcb7fe640 1 --2- 172.21.15.103:0/339530794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455e4a10 0x5630455eee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dcb7fe640 1 -- 172.21.15.103:0/339530794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455f1ec0 msgr2=0x5630455ef890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dcb7fe640 1 --2- 172.21.15.103:0/339530794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455f1ec0 0x5630455ef890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dcb7fe640 1 -- 172.21.15.103:0/339530794 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56304525c210 con 0x5630455e5330 2024-09-19T06:54:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dcb7fe640 1 --2- 172.21.15.103:0/339530794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455e5330 0x5630455ef350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009230 tx=0x7f5dc00042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f75245ea97a9963 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.624+0000 7f5dab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:54:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.625+0000 7f5dab7fe640 1 -- 172.21.15.103:0/339530794 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0040070 con 0x5630455e5330 2024-09-19T06:54:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.625+0000 7f5dab7fe640 1 -- 172.21.15.103:0/339530794 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5dc0005460 con 0x5630455e5330 2024-09-19T06:54:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.625+0000 7f5dd526c740 1 -- 172.21.15.103:0/339530794 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dc4004d20 con 0x5630455e5330 2024-09-19T06:54:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.625+0000 7f5dab7fe640 1 -- 172.21.15.103:0/339530794 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5dc004c050 con 0x5630455e5330 2024-09-19T06:54:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.625+0000 7f5dd526c740 10 client.26291 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.625+0000 7f5dd526c740 1 -- 172.21.15.103:0/339530794 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563045328620 con 0x5630455e5330 2024-09-19T06:54:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.625+0000 7f5dab7fe640 1 -- 172.21.15.103:0/339530794 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dc4004d20 con 0x5630455e5330 2024-09-19T06:54:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.625+0000 7f5dab7fe640 1 -- 172.21.15.103:0/339530794 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5dc0003850 con 0x5630455e5330 2024-09-19T06:54:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.625+0000 7f5dd526c740 10 client.26291 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.625+0000 7f5dd526c740 10 client.26291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.625+0000 7f5dd526c740 20 client.26291 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.625+0000 7f5dd526c740 1 --2- 172.21.15.103:0/339530794 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563045601eb0 0x5630456042a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.625+0000 7f5dd526c740 4 client.26291 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.625+0000 7f5dd526c740 1 -- 172.21.15.103:0/339530794 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563045556a20 con 0x563045601eb0 2024-09-19T06:54:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.626+0000 7f5dd0993640 1 --2- 172.21.15.103:0/339530794 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563045601eb0 0x5630456042a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.626+0000 7f5dd0993640 1 --2- 172.21.15.103:0/339530794 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563045601eb0 0x5630456042a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6baa9a956c6ce666 server_cookie=9447df392403151e in_seq=0 out_seq=0 2024-09-19T06:54:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.626+0000 7f5dab7fe640 10 client.26291 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.627+0000 7f5dab7fe640 1 -- 172.21.15.103:0/339530794 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563045556a20 con 0x563045601eb0 2024-09-19T06:54:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.627+0000 7f5dab7fe640 10 client.26291 handle_command_reply: tid=0 2024-09-19T06:54:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.627+0000 7f5dd526c740 1 client.26291 shutdown 2024-09-19T06:54:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.627+0000 7f5dd526c740 1 -- 172.21.15.103:0/339530794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455e5330 msgr2=0x5630455ef350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.627+0000 7f5dd526c740 1 --2- 172.21.15.103:0/339530794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455e5330 0x5630455ef350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009230 tx=0x7f5dc00042e0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.627+0000 7f5dd526c740 1 -- 172.21.15.103:0/339530794 shutdown_connections 2024-09-19T06:54:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.627+0000 7f5dd526c740 1 --2- 172.21.15.103:0/339530794 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563045601eb0 0x5630456042a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.627+0000 7f5dd526c740 1 --2- 172.21.15.103:0/339530794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455f1ec0 0x5630455ef890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.627+0000 7f5dd526c740 1 --2- 172.21.15.103:0/339530794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455e5330 0x5630455ef350 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.627+0000 7f5dab7fe640 0 client.26291 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.627+0000 7f5dd526c740 1 --2- 172.21.15.103:0/339530794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455e4a10 0x5630455eee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.627+0000 7f5dd526c740 1 -- 172.21.15.103:0/339530794 >> 172.21.15.103:0/339530794 conn(0x5630455e1060 msgr2=0x5630455e6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.627+0000 7f5dd526c740 1 -- 172.21.15.103:0/339530794 shutdown_connections 2024-09-19T06:54:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.627+0000 7f5dd526c740 1 -- 172.21.15.103:0/339530794 wait complete. 2024-09-19T06:54:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.627+0000 7f5dd526c740 20 client.26291 trim_cache size 0 max 16384 2024-09-19T06:54:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.641+0000 7f5dd526c740 1 Processor -- start 2024-09-19T06:54:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.641+0000 7f5dd526c740 1 -- start start 2024-09-19T06:54:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.641+0000 7f5dd526c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455e4a10 0x5630455ef6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.641+0000 7f5dd526c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455e5330 0x563045609b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.641+0000 7f5dd526c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455f1ec0 0x56304560a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.641+0000 7f5dd526c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc1ab6d0 con 0x5630455e4a10 2024-09-19T06:54:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.641+0000 7f5dd526c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc1ad210 con 0x5630455f1ec0 2024-09-19T06:54:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.641+0000 7f5dd526c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc1abda0 con 0x5630455e5330 2024-09-19T06:54:40.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.642+0000 7f5dd0993640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455e4a10 0x5630455ef6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.642+0000 7f5dd0993640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455e4a10 0x5630455ef6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48824/0 (socket says 172.21.15.103:48824) 2024-09-19T06:54:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.642+0000 7f5dd0993640 1 -- 172.21.15.103:0/515181613 learned_addr learned my addr 172.21.15.103:0/515181613 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.642+0000 7f5dcb7fe640 1 --2- 172.21.15.103:0/515181613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455e5330 0x563045609b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.642+0000 7f5dd0993640 1 --2- 172.21.15.103:0/515181613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455f1ec0 0x56304560a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.642+0000 7f5dd0993640 1 -- 172.21.15.103:0/515181613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455e5330 msgr2=0x563045609b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.642+0000 7f5dd0993640 1 --2- 172.21.15.103:0/515181613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455e5330 0x563045609b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.642+0000 7f5dd0993640 1 -- 172.21.15.103:0/515181613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455f1ec0 msgr2=0x56304560a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.642+0000 7f5dd0993640 1 --2- 172.21.15.103:0/515181613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455f1ec0 0x56304560a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.642+0000 7f5dd0993640 1 -- 172.21.15.103:0/515181613 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630455e0e90 con 0x5630455e4a10 2024-09-19T06:54:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.642+0000 7f5dcb7fe640 1 --2- 172.21.15.103:0/515181613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455e5330 0x563045609b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.642+0000 7f5dd0993640 1 --2- 172.21.15.103:0/515181613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455e4a10 0x5630455ef6c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4010d30 tx=0x7f5dc4009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c114c93b731a64a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.642+0000 7f5db1ffb640 1 -- 172.21.15.103:0/515181613 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4016040 con 0x5630455e4a10 2024-09-19T06:54:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.643+0000 7f5db1ffb640 1 -- 172.21.15.103:0/515181613 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dc4004d20 con 0x5630455e4a10 2024-09-19T06:54:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.643+0000 7f5db1ffb640 1 -- 172.21.15.103:0/515181613 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4023040 con 0x5630455e4a10 2024-09-19T06:54:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.643+0000 7f5dd526c740 1 -- 172.21.15.103:0/515181613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455e4a10 msgr2=0x5630455ef6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.643+0000 7f5dd526c740 1 --2- 172.21.15.103:0/515181613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455e4a10 0x5630455ef6c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4010d30 tx=0x7f5dc4009600 comp rx=0 tx=0).stop 2024-09-19T06:54:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.643+0000 7f5dd526c740 1 -- 172.21.15.103:0/515181613 shutdown_connections 2024-09-19T06:54:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.643+0000 7f5dd526c740 1 --2- 172.21.15.103:0/515181613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455f1ec0 0x56304560a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.643+0000 7f5dd526c740 1 --2- 172.21.15.103:0/515181613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455e5330 0x563045609b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.643+0000 7f5dd526c740 1 --2- 172.21.15.103:0/515181613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455e4a10 0x5630455ef6c0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.643+0000 7f5dd526c740 1 -- 172.21.15.103:0/515181613 >> 172.21.15.103:0/515181613 conn(0x5630455e1060 msgr2=0x5630455e01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.643+0000 7f5dd526c740 1 -- 172.21.15.103:0/515181613 shutdown_connections 2024-09-19T06:54:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.643+0000 7f5dd526c740 1 -- 172.21.15.103:0/515181613 wait complete. 2024-09-19T06:54:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.644+0000 7f5dd526c740 1 -- start start 2024-09-19T06:54:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.644+0000 7f5dd526c740 1 Processor -- start 2024-09-19T06:54:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.644+0000 7f5dd526c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455e4a10 0x5630455fa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.644+0000 7f5dd526c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455e5330 0x5630455f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.644+0000 7f5dd526c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455f1ec0 0x5630455f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.644+0000 7f5dd526c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc1acb40 con 0x5630455f1ec0 2024-09-19T06:54:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.644+0000 7f5dd526c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc1ac470 con 0x5630455e5330 2024-09-19T06:54:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.644+0000 7f5dd526c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc1a96b0 con 0x5630455e4a10 2024-09-19T06:54:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.644+0000 7f5dd526c740 10 client.0 _ll_register_callbacks cb 0x7f5dc0009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.644+0000 7f5dd0993640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455f1ec0 0x5630455f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.644+0000 7f5dd0993640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455f1ec0 0x5630455f5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48828/0 (socket says 172.21.15.103:48828) 2024-09-19T06:54:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.644+0000 7f5dd0993640 1 -- 172.21.15.103:0/1413088935 learned_addr learned my addr 172.21.15.103:0/1413088935 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.644+0000 7f5dcb7fe640 1 --2- 172.21.15.103:0/1413088935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455e5330 0x5630455f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.645+0000 7f5dd0993640 1 --2- 172.21.15.103:0/1413088935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455e4a10 0x5630455fa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.645+0000 7f5dd0993640 1 -- 172.21.15.103:0/1413088935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455e4a10 msgr2=0x5630455fa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.645+0000 7f5dd0993640 1 --2- 172.21.15.103:0/1413088935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455e4a10 0x5630455fa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.645+0000 7f5dd0993640 1 -- 172.21.15.103:0/1413088935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455e5330 msgr2=0x5630455f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.645+0000 7f5dd0993640 1 --2- 172.21.15.103:0/1413088935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455e5330 0x5630455f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.645+0000 7f5dd0993640 1 -- 172.21.15.103:0/1413088935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630455f6160 con 0x5630455f1ec0 2024-09-19T06:54:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.645+0000 7f5dcb7fe640 1 --2- 172.21.15.103:0/1413088935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455e5330 0x5630455f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.645+0000 7f5dd0993640 1 --2- 172.21.15.103:0/1413088935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455f1ec0 0x5630455f5c20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4014010 tx=0x7f5dc4012b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab567e88c9d6659 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.645+0000 7f5dab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.645+0000 7f5dab7fe640 1 -- 172.21.15.103:0/1413088935 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4016040 con 0x5630455f1ec0 2024-09-19T06:54:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.645+0000 7f5dab7fe640 1 -- 172.21.15.103:0/1413088935 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f5dc4012510 con 0x5630455f1ec0 2024-09-19T06:54:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.645+0000 7f5dd526c740 1 -- 172.21.15.103:0/1413088935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dc4004d20 con 0x5630455f1ec0 2024-09-19T06:54:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.645+0000 7f5dab7fe640 1 -- 172.21.15.103:0/1413088935 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5dc4012710 con 0x5630455f1ec0 2024-09-19T06:54:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.645+0000 7f5dd526c740 10 client.17436 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.645+0000 7f5dd526c740 1 -- 172.21.15.103:0/1413088935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56304566c850 con 0x5630455f1ec0 2024-09-19T06:54:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.645+0000 7f5dab7fe640 1 -- 172.21.15.103:0/1413088935 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dc4004d20 con 0x5630455f1ec0 2024-09-19T06:54:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.646+0000 7f5dab7fe640 1 -- 172.21.15.103:0/1413088935 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5dc4029040 con 0x5630455f1ec0 2024-09-19T06:54:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.646+0000 7f5dd526c740 10 client.17436 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.646+0000 7f5dd526c740 10 client.17436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.646+0000 7f5dd526c740 20 client.17436 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.646+0000 7f5dd526c740 1 --2- 172.21.15.103:0/1413088935 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56304566ccc0 0x56304566f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.646+0000 7f5dd526c740 4 client.17436 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:54:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.646+0000 7f5dd526c740 1 -- 172.21.15.103:0/1413088935 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563045550d70 con 0x56304566ccc0 2024-09-19T06:54:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.646+0000 7f5dcb7fe640 1 --2- 172.21.15.103:0/1413088935 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56304566ccc0 0x56304566f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.646+0000 7f5dab7fe640 10 client.17436 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.646+0000 7f5dcb7fe640 1 --2- 172.21.15.103:0/1413088935 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56304566ccc0 0x56304566f0b0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20f0a5f52a7cfce8 server_cookie=6f330eb045484ad4 in_seq=0 out_seq=0 2024-09-19T06:54:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.655+0000 7f5dab7fe640 1 -- 172.21.15.103:0/1413088935 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563045550d70 con 0x56304566ccc0 2024-09-19T06:54:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.655+0000 7f5dab7fe640 10 client.17436 handle_command_reply: tid=0 2024-09-19T06:54:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.655+0000 7f5dd526c740 1 client.17436 shutdown 2024-09-19T06:54:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.655+0000 7f5dd526c740 1 -- 172.21.15.103:0/1413088935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455f1ec0 msgr2=0x5630455f5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.655+0000 7f5dd526c740 1 --2- 172.21.15.103:0/1413088935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455f1ec0 0x5630455f5c20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4014010 tx=0x7f5dc4012b10 comp rx=0 tx=0).stop 2024-09-19T06:54:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.655+0000 7f5dd526c740 1 -- 172.21.15.103:0/1413088935 shutdown_connections 2024-09-19T06:54:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.655+0000 7f5dd526c740 1 --2- 172.21.15.103:0/1413088935 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56304566ccc0 0x56304566f0b0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.655+0000 7f5dd526c740 1 --2- 172.21.15.103:0/1413088935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630455f1ec0 0x5630455f5c20 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.655+0000 7f5dd526c740 1 --2- 172.21.15.103:0/1413088935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630455e5330 0x5630455f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.655+0000 7f5dab7fe640 0 client.17436 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.655+0000 7f5dd526c740 1 --2- 172.21.15.103:0/1413088935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630455e4a10 0x5630455fa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.655+0000 7f5dd526c740 1 -- 172.21.15.103:0/1413088935 >> 172.21.15.103:0/1413088935 conn(0x5630455e1060 msgr2=0x5630455e64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.655+0000 7f5dd526c740 1 -- 172.21.15.103:0/1413088935 shutdown_connections 2024-09-19T06:54:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.655+0000 7f5dd526c740 1 -- 172.21.15.103:0/1413088935 wait complete. 2024-09-19T06:54:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.655+0000 7f5dd526c740 20 client.17436 trim_cache size 0 max 16384 2024-09-19T06:54:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:54:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:54:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:54:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:54:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.656+0000 7f5dd23f6640 1 -- 172.21.15.103:0/41439023 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5db40a3460 msgr2=0x7f5db40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.656+0000 7f5dd23f6640 1 --2- 172.21.15.103:0/41439023 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5db40a3460 0x7f5db40a5920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc098580 tx=0x7f5dbc086040 comp rx=0 tx=0).stop 2024-09-19T06:54:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.656+0000 7f5dd23f6640 1 -- 172.21.15.103:0/41439023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dcc1009b0 msgr2=0x7f5dcc197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.656+0000 7f5dd23f6640 1 --2- 172.21.15.103:0/41439023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dcc1009b0 0x7f5dcc197320 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00d950 tx=0x7f5dbc00de10 comp rx=0 tx=0).stop 2024-09-19T06:54:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.656+0000 7f5dd23f6640 1 -- 172.21.15.103:0/41439023 shutdown_connections 2024-09-19T06:54:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.656+0000 7f5dd23f6640 1 --2- 172.21.15.103:0/41439023 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5db40a3460 0x7f5db40a5920 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.656+0000 7f5dd23f6640 1 --2- 172.21.15.103:0/41439023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dcc1069c0 0x7f5dcc19bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.656+0000 7f5dd23f6640 1 --2- 172.21.15.103:0/41439023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dcc101350 0x7f5dcc197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.656+0000 7f5dd23f6640 1 --2- 172.21.15.103:0/41439023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dcc1009b0 0x7f5dcc197320 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.656+0000 7f5dd23f6640 1 -- 172.21.15.103:0/41439023 >> 172.21.15.103:0/41439023 conn(0x7f5dcc077990 msgr2=0x7f5dcc103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.656+0000 7f5dd23f6640 1 -- 172.21.15.103:0/41439023 shutdown_connections 2024-09-19T06:54:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:40.656+0000 7f5dd23f6640 1 -- 172.21.15.103:0/41439023 wait complete. 2024-09-19T06:54:40.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.660+0000 7f2a5b7e8640 1 -- 172.21.15.103:0/4100030526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a54102570 msgr2=0x7f2a541029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.660+0000 7f2a5b7e8640 1 --2- 172.21.15.103:0/4100030526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a54102570 0x7f2a541029b0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009900 tx=0x7f2a4402a480 comp rx=0 tx=0).stop 2024-09-19T06:54:40.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.660+0000 7f2a5b7e8640 1 -- 172.21.15.103:0/4100030526 shutdown_connections 2024-09-19T06:54:40.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.660+0000 7f2a5b7e8640 1 --2- 172.21.15.103:0/4100030526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a54102ef0 0x7f2a5410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.660+0000 7f2a5b7e8640 1 --2- 172.21.15.103:0/4100030526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a54102570 0x7f2a541029b0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.660+0000 7f2a5b7e8640 1 --2- 172.21.15.103:0/4100030526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a54100e20 0x7f2a54101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.660+0000 7f2a5b7e8640 1 -- 172.21.15.103:0/4100030526 >> 172.21.15.103:0/4100030526 conn(0x7f2a540fc7d0 msgr2=0x7f2a540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:40.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.660+0000 7f2a5b7e8640 1 -- 172.21.15.103:0/4100030526 shutdown_connections 2024-09-19T06:54:40.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.660+0000 7f2a5b7e8640 1 -- 172.21.15.103:0/4100030526 wait complete. 2024-09-19T06:54:40.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.661+0000 7f2a5b7e8640 1 Processor -- start 2024-09-19T06:54:40.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.661+0000 7f2a5b7e8640 1 -- start start 2024-09-19T06:54:40.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.661+0000 7f2a5b7e8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a54100e20 0x7f2a54197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.661+0000 7f2a5b7e8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a54102570 0x7f2a541978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.661+0000 7f2a5b7e8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a54102ef0 0x7f2a5419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.661+0000 7f2a5b7e8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a54110f80 con 0x7f2a54100e20 2024-09-19T06:54:40.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.661+0000 7f2a5b7e8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a54110e00 con 0x7f2a54102ef0 2024-09-19T06:54:40.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.661+0000 7f2a5b7e8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a54111100 con 0x7f2a54102570 2024-09-19T06:54:40.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.661+0000 7f2a59584640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a54100e20 0x7f2a54197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.662+0000 7f2a59584640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a54100e20 0x7f2a54197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48842/0 (socket says 172.21.15.103:48842) 2024-09-19T06:54:40.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.662+0000 7f2a59584640 1 -- 172.21.15.103:0/95093454 learned_addr learned my addr 172.21.15.103:0/95093454 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:40.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.662+0000 7f2a59d85640 1 --2- 172.21.15.103:0/95093454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a54102ef0 0x7f2a5419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.662+0000 7f2a58d83640 1 --2- 172.21.15.103:0/95093454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a54102570 0x7f2a541978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.662+0000 7f2a59584640 1 -- 172.21.15.103:0/95093454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a54102570 msgr2=0x7f2a541978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.662+0000 7f2a59584640 1 --2- 172.21.15.103:0/95093454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a54102570 0x7f2a541978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.662+0000 7f2a59584640 1 -- 172.21.15.103:0/95093454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a54102ef0 msgr2=0x7f2a5419bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.662+0000 7f2a59584640 1 --2- 172.21.15.103:0/95093454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a54102ef0 0x7f2a5419bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.662+0000 7f2a59584640 1 -- 172.21.15.103:0/95093454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5419c310 con 0x7f2a54100e20 2024-09-19T06:54:40.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.662+0000 7f2a59584640 1 --2- 172.21.15.103:0/95093454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a54100e20 0x7f2a54197360 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009900 tx=0x7f2a50009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:40.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.662+0000 7f2a4a7fc640 1 -- 172.21.15.103:0/95093454 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5000d030 con 0x7f2a54100e20 2024-09-19T06:54:40.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.662+0000 7f2a4a7fc640 1 -- 172.21.15.103:0/95093454 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5000d1d0 con 0x7f2a54100e20 2024-09-19T06:54:40.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.662+0000 7f2a5b7e8640 1 -- 172.21.15.103:0/95093454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5419c600 con 0x7f2a54100e20 2024-09-19T06:54:40.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.662+0000 7f2a4a7fc640 1 -- 172.21.15.103:0/95093454 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a50005260 con 0x7f2a54100e20 2024-09-19T06:54:40.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.662+0000 7f2a5b7e8640 1 -- 172.21.15.103:0/95093454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a54069f70 con 0x7f2a54100e20 2024-09-19T06:54:40.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.665+0000 7f2a4a7fc640 1 -- 172.21.15.103:0/95093454 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f2a500078f0 con 0x7f2a54100e20 2024-09-19T06:54:40.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.665+0000 7f2a5dbc1740 1 Processor -- start 2024-09-19T06:54:40.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.665+0000 7f2a5dbc1740 1 -- start start 2024-09-19T06:54:40.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.665+0000 7f2a5dbc1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d14660b40 0x558d1465f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.665+0000 7f2a5dbc1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1465f5d0 0x558d1465fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.665+0000 7f2a5dbc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1466a370 0x558d1466c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.665+0000 7f2a5dbc1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558d145c52f0 con 0x558d1465f5d0 2024-09-19T06:54:40.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.665+0000 7f2a5dbc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558d1456ccf0 con 0x558d1466a370 2024-09-19T06:54:40.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.665+0000 7f2a5dbc1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558d145c12b0 con 0x558d14660b40 2024-09-19T06:54:40.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.666+0000 7f2a59584640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1465f5d0 0x558d1465fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.666+0000 7f2a59584640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1465f5d0 0x558d1465fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48852/0 (socket says 172.21.15.103:48852) 2024-09-19T06:54:40.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.666+0000 7f2a59584640 1 -- 172.21.15.103:0/1996004589 learned_addr learned my addr 172.21.15.103:0/1996004589 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:40.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.666+0000 7f2a59d85640 1 --2- 172.21.15.103:0/1996004589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d14660b40 0x558d1465f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.666+0000 7f2a4a7fc640 1 --2- 172.21.15.103:0/95093454 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2a400a33b0 0x7f2a400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.666+0000 7f2a59d85640 1 -- 172.21.15.103:0/1996004589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1466a370 msgr2=0x558d1466c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:40.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.666+0000 7f2a59d85640 1 --2- 172.21.15.103:0/1996004589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1466a370 0x558d1466c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.666+0000 7f2a59d85640 1 -- 172.21.15.103:0/1996004589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1465f5d0 msgr2=0x558d1465fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:40.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.666+0000 7f2a59d85640 1 --2- 172.21.15.103:0/1996004589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1465f5d0 0x558d1465fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.666+0000 7f2a59d85640 1 -- 172.21.15.103:0/1996004589 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d145513a0 con 0x558d14660b40 2024-09-19T06:54:40.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.666+0000 7f2a59584640 1 --2- 172.21.15.103:0/1996004589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1465f5d0 0x558d1465fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:40.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.666+0000 7f2a59d85640 1 --2- 172.21.15.103:0/95093454 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2a400a33b0 0x7f2a400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.666+0000 7f2a4a7fc640 1 -- 172.21.15.103:0/95093454 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f2a500d10c0 con 0x7f2a54100e20 2024-09-19T06:54:40.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.666+0000 7f2a59d85640 1 --2- 172.21.15.103:0/1996004589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d14660b40 0x558d1465f090 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c007ec0 tx=0x7f2a4c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac4d7ea876399423 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:40.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.667+0000 7f2a3f7fe640 1 -- 172.21.15.103:0/1996004589 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c019760 con 0x558d14660b40 2024-09-19T06:54:40.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.667+0000 7f2a3f7fe640 1 -- 172.21.15.103:0/1996004589 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a4c019de0 con 0x558d14660b40 2024-09-19T06:54:40.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.667+0000 7f2a3f7fe640 1 -- 172.21.15.103:0/1996004589 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c01abf0 con 0x558d14660b40 2024-09-19T06:54:40.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.667+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/1996004589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d14660b40 msgr2=0x558d1465f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.667+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/1996004589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d14660b40 0x558d1465f090 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c007ec0 tx=0x7f2a4c00f790 comp rx=0 tx=0).stop 2024-09-19T06:54:40.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.667+0000 7f2a59d85640 1 --2- 172.21.15.103:0/95093454 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2a400a33b0 0x7f2a400a5870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c020000 tx=0x7f2a4c01d000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:40.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.667+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/1996004589 shutdown_connections 2024-09-19T06:54:40.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.667+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/1996004589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1466a370 0x558d1466c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.667+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/1996004589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1465f5d0 0x558d1465fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.667+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/1996004589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d14660b40 0x558d1465f090 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.667+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/1996004589 >> 172.21.15.103:0/1996004589 conn(0x558d14628310 msgr2=0x558d1465bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:40.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.668+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/1996004589 shutdown_connections 2024-09-19T06:54:40.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.668+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/1996004589 wait complete. 2024-09-19T06:54:40.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.668+0000 7f2a5dbc1740 1 -- start start 2024-09-19T06:54:40.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.668+0000 7f2a5dbc1740 1 Processor -- start 2024-09-19T06:54:40.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.668+0000 7f2a5dbc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1465f5d0 0x558d14669d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.668+0000 7f2a5dbc1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d14660b40 0x558d14667020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.668+0000 7f2a5dbc1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d1466a370 0x558d14667630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.668+0000 7f2a5dbc1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558d14671480 con 0x558d14660b40 2024-09-19T06:54:40.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.668+0000 7f2a5dbc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558d14580060 con 0x558d1465f5d0 2024-09-19T06:54:40.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.668+0000 7f2a5dbc1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558d14671600 con 0x558d1466a370 2024-09-19T06:54:40.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.668+0000 7f2a5dbc1740 10 client.0 _ll_register_callbacks cb 0x558d142d38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:40.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.669+0000 7f2a58d83640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d14660b40 0x558d14667020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.669+0000 7f2a58d83640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d14660b40 0x558d14667020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48858/0 (socket says 172.21.15.103:48858) 2024-09-19T06:54:40.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.669+0000 7f2a58d83640 1 -- 172.21.15.103:0/2711093590 learned_addr learned my addr 172.21.15.103:0/2711093590 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:40.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.669+0000 7f2a59584640 1 --2- 172.21.15.103:0/2711093590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1465f5d0 0x558d14669d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.669+0000 7f2a59d85640 1 --2- 172.21.15.103:0/2711093590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d1466a370 0x558d14667630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.669+0000 7f2a59584640 1 -- 172.21.15.103:0/2711093590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d1466a370 msgr2=0x558d14667630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:40.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.669+0000 7f2a59584640 1 --2- 172.21.15.103:0/2711093590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d1466a370 0x558d14667630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.669+0000 7f2a59584640 1 -- 172.21.15.103:0/2711093590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d14660b40 msgr2=0x558d14667020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:40.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.669+0000 7f2a59584640 1 --2- 172.21.15.103:0/2711093590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d14660b40 0x558d14667020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.669+0000 7f2a59584640 1 -- 172.21.15.103:0/2711093590 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d143363c0 con 0x558d1465f5d0 2024-09-19T06:54:40.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.669+0000 7f2a58d83640 1 --2- 172.21.15.103:0/2711093590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d14660b40 0x558d14667020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:40.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.669+0000 7f2a59584640 1 --2- 172.21.15.103:0/2711093590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1465f5d0 0x558d14669d20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2a50007590 tx=0x7f2a5009b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a73e30fd2b1e3f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:40.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.669+0000 7f2a3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:54:40.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.669+0000 7f2a3cff9640 1 -- 172.21.15.103:0/2711093590 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5001e040 con 0x558d1465f5d0 2024-09-19T06:54:40.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.669+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/2711093590 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a4c019de0 con 0x558d1465f5d0 2024-09-19T06:54:40.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.669+0000 7f2a3cff9640 1 -- 172.21.15.103:0/2711093590 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f2a500d30d0 con 0x558d1465f5d0 2024-09-19T06:54:40.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.670+0000 7f2a3cff9640 1 -- 172.21.15.103:0/2711093590 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a5009cd80 con 0x558d1465f5d0 2024-09-19T06:54:40.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.670+0000 7f2a5dbc1740 10 client.26306 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:40.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.670+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/2711093590 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d1467cf70 con 0x558d1465f5d0 2024-09-19T06:54:40.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.670+0000 7f2a3cff9640 1 -- 172.21.15.103:0/2711093590 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a4c019de0 con 0x558d1465f5d0 2024-09-19T06:54:40.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.670+0000 7f2a3cff9640 1 -- 172.21.15.103:0/2711093590 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2a500d5110 con 0x558d1465f5d0 2024-09-19T06:54:40.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.670+0000 7f2a5dbc1740 10 client.26306 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:40.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.670+0000 7f2a5dbc1740 10 client.26306 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:54:40.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.670+0000 7f2a5dbc1740 20 client.26306 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:40.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.670+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/2711093590 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x558d1467e430 0x558d14680820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.670+0000 7f2a5dbc1740 4 client.26306 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:40.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.670+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/2711093590 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d145d2860 con 0x558d1467e430 2024-09-19T06:54:40.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.670+0000 7f2a58d83640 1 --2- 172.21.15.103:0/2711093590 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x558d1467e430 0x558d14680820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.671+0000 7f2a3cff9640 10 client.26306 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:54:40.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.671+0000 7f2a58d83640 1 --2- 172.21.15.103:0/2711093590 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x558d1467e430 0x558d14680820 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a1e1b00748532a43 server_cookie=c40969517ba07fa2 in_seq=0 out_seq=0 2024-09-19T06:54:40.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.672+0000 7f2a3cff9640 1 -- 172.21.15.103:0/2711093590 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d145d2860 con 0x558d1467e430 2024-09-19T06:54:40.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.672+0000 7f2a3cff9640 10 client.26306 handle_command_reply: tid=0 2024-09-19T06:54:40.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.672+0000 7f2a5dbc1740 1 client.26306 shutdown 2024-09-19T06:54:40.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.672+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/2711093590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1465f5d0 msgr2=0x558d14669d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.672+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/2711093590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1465f5d0 0x558d14669d20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2a50007590 tx=0x7f2a5009b930 comp rx=0 tx=0).stop 2024-09-19T06:54:40.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.672+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/2711093590 shutdown_connections 2024-09-19T06:54:40.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.672+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/2711093590 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x558d1467e430 0x558d14680820 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.672+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/2711093590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d1466a370 0x558d14667630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.672+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/2711093590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d14660b40 0x558d14667020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.672+0000 7f2a3cff9640 0 client.26306 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:54:40.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.672+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/2711093590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1465f5d0 0x558d14669d20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.672+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/2711093590 >> 172.21.15.103:0/2711093590 conn(0x558d14628310 msgr2=0x558d14661960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:40.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.672+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/2711093590 shutdown_connections 2024-09-19T06:54:40.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.672+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/2711093590 wait complete. 2024-09-19T06:54:40.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.672+0000 7f2a5dbc1740 20 client.26306 trim_cache size 0 max 16384 2024-09-19T06:54:40.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.685+0000 7f2a5dbc1740 1 Processor -- start 2024-09-19T06:54:40.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.685+0000 7f2a5dbc1740 1 -- start start 2024-09-19T06:54:40.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.685+0000 7f2a5dbc1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d1465f5d0 0x558d14662120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.685+0000 7f2a5dbc1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d14660b40 0x558d14662660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.685+0000 7f2a5dbc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1466a370 0x558d146860f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.685+0000 7f2a5dbc1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a541ab600 con 0x558d14660b40 2024-09-19T06:54:40.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.685+0000 7f2a5dbc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a541ad140 con 0x558d1466a370 2024-09-19T06:54:40.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.685+0000 7f2a5dbc1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a541abcd0 con 0x558d1465f5d0 2024-09-19T06:54:40.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.685+0000 7f2a58d83640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d14660b40 0x558d14662660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.685+0000 7f2a58d83640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d14660b40 0x558d14662660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48864/0 (socket says 172.21.15.103:48864) 2024-09-19T06:54:40.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.685+0000 7f2a58d83640 1 -- 172.21.15.103:0/2804663784 learned_addr learned my addr 172.21.15.103:0/2804663784 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:40.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.685+0000 7f2a58d83640 1 -- 172.21.15.103:0/2804663784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d1465f5d0 msgr2=0x558d14662120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:40.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.685+0000 7f2a58d83640 1 --2- 172.21.15.103:0/2804663784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d1465f5d0 0x558d14662120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.685+0000 7f2a58d83640 1 -- 172.21.15.103:0/2804663784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1466a370 msgr2=0x558d146860f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:40.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.685+0000 7f2a58d83640 1 --2- 172.21.15.103:0/2804663784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1466a370 0x558d146860f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.685+0000 7f2a58d83640 1 -- 172.21.15.103:0/2804663784 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d14680b50 con 0x558d14660b40 2024-09-19T06:54:40.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.686+0000 7f2a58d83640 1 --2- 172.21.15.103:0/2804663784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d14660b40 0x558d14662660 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009230 tx=0x7f2a44000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fd5f92e5269118 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:40.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.686+0000 7f2a3effd640 1 -- 172.21.15.103:0/2804663784 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44035040 con 0x558d14660b40 2024-09-19T06:54:40.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.686+0000 7f2a3effd640 1 -- 172.21.15.103:0/2804663784 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a4404a070 con 0x558d14660b40 2024-09-19T06:54:40.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.686+0000 7f2a3effd640 1 -- 172.21.15.103:0/2804663784 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44003850 con 0x558d14660b40 2024-09-19T06:54:40.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.686+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/2804663784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d14660b40 msgr2=0x558d14662660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.686+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/2804663784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d14660b40 0x558d14662660 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009230 tx=0x7f2a44000bc0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.686+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/2804663784 shutdown_connections 2024-09-19T06:54:40.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.686+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/2804663784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1466a370 0x558d146860f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.686+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/2804663784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d14660b40 0x558d14662660 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.686+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/2804663784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d1465f5d0 0x558d14662120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.686+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/2804663784 >> 172.21.15.103:0/2804663784 conn(0x558d14628310 msgr2=0x558d1465ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:40.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.687+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/2804663784 shutdown_connections 2024-09-19T06:54:40.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.687+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/2804663784 wait complete. 2024-09-19T06:54:40.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.687+0000 7f2a5dbc1740 1 -- start start 2024-09-19T06:54:40.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.687+0000 7f2a5dbc1740 1 Processor -- start 2024-09-19T06:54:40.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.687+0000 7f2a5dbc1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1465f5d0 0x558d146728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.687+0000 7f2a5dbc1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d14660b40 0x558d14672e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.687+0000 7f2a5dbc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1466a370 0x558d14673360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.687+0000 7f2a5dbc1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a541aca70 con 0x558d1465f5d0 2024-09-19T06:54:40.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.687+0000 7f2a5dbc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a541ac3a0 con 0x558d1466a370 2024-09-19T06:54:40.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.687+0000 7f2a5dbc1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a541a95c0 con 0x558d14660b40 2024-09-19T06:54:40.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.687+0000 7f2a5dbc1740 10 client.0 _ll_register_callbacks cb 0x7f2a50009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:40.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.687+0000 7f2a59584640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1465f5d0 0x558d146728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a59584640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1465f5d0 0x558d146728c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48878/0 (socket says 172.21.15.103:48878) 2024-09-19T06:54:40.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a59584640 1 -- 172.21.15.103:0/1865784960 learned_addr learned my addr 172.21.15.103:0/1865784960 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:40.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a58d83640 1 --2- 172.21.15.103:0/1865784960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d14660b40 0x558d14672e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a59d85640 1 --2- 172.21.15.103:0/1865784960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1466a370 0x558d14673360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a59584640 1 -- 172.21.15.103:0/1865784960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d14660b40 msgr2=0x558d14672e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:40.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a59584640 1 --2- 172.21.15.103:0/1865784960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d14660b40 0x558d14672e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a59584640 1 -- 172.21.15.103:0/1865784960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1466a370 msgr2=0x558d14673360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:40.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a59584640 1 --2- 172.21.15.103:0/1865784960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1466a370 0x558d14673360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a59584640 1 -- 172.21.15.103:0/1865784960 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d146e8390 con 0x558d1465f5d0 2024-09-19T06:54:40.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a59584640 1 --2- 172.21.15.103:0/1865784960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1465f5d0 0x558d146728c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f2a50007d30 tx=0x7f2a5000a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69ca30150b45ff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:40.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:40.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a3cff9640 1 -- 172.21.15.103:0/1865784960 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5001e040 con 0x558d1465f5d0 2024-09-19T06:54:40.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a3cff9640 1 -- 172.21.15.103:0/1865784960 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f2a50004020 con 0x558d1465f5d0 2024-09-19T06:54:40.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/1865784960 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a4404a070 con 0x558d1465f5d0 2024-09-19T06:54:40.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a3cff9640 1 -- 172.21.15.103:0/1865784960 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a500cec60 con 0x558d1465f5d0 2024-09-19T06:54:40.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a5dbc1740 10 client.17454 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:40.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.688+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/1865784960 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d146e9690 con 0x558d1465f5d0 2024-09-19T06:54:40.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.689+0000 7f2a3cff9640 1 -- 172.21.15.103:0/1865784960 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a4404a070 con 0x558d1465f5d0 2024-09-19T06:54:40.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.689+0000 7f2a3cff9640 1 -- 172.21.15.103:0/1865784960 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2a5009c870 con 0x558d1465f5d0 2024-09-19T06:54:40.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.689+0000 7f2a5dbc1740 10 client.17454 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:40.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.689+0000 7f2a5dbc1740 10 client.17454 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:54:40.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.689+0000 7f2a5dbc1740 20 client.17454 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:40.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.689+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/1865784960 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x558d146e9d00 0x558d146ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:40.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.689+0000 7f2a5dbc1740 4 client.17454 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:54:40.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.689+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/1865784960 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d145ccd70 con 0x558d146e9d00 2024-09-19T06:54:40.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.689+0000 7f2a58d83640 1 --2- 172.21.15.103:0/1865784960 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x558d146e9d00 0x558d146ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:40.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a3cff9640 10 client.17454 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:54:40.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a58d83640 1 --2- 172.21.15.103:0/1865784960 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x558d146e9d00 0x558d146ec0f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d9c28250faec8e45 server_cookie=1313f809bbb3261d in_seq=0 out_seq=0 2024-09-19T06:54:40.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a3cff9640 1 -- 172.21.15.103:0/1865784960 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d145ccd70 con 0x558d146e9d00 2024-09-19T06:54:40.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a3cff9640 10 client.17454 handle_command_reply: tid=0 2024-09-19T06:54:40.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a5dbc1740 1 client.17454 shutdown 2024-09-19T06:54:40.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/1865784960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1465f5d0 msgr2=0x558d146728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/1865784960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1465f5d0 0x558d146728c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f2a50007d30 tx=0x7f2a5000a250 comp rx=0 tx=0).stop 2024-09-19T06:54:40.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/1865784960 shutdown_connections 2024-09-19T06:54:40.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/1865784960 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x558d146e9d00 0x558d146ec0f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/1865784960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d1466a370 0x558d14673360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/1865784960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d14660b40 0x558d14672e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a3cff9640 0 client.17454 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:54:40.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a5dbc1740 1 --2- 172.21.15.103:0/1865784960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d1465f5d0 0x558d146728c0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/1865784960 >> 172.21.15.103:0/1865784960 conn(0x558d14628310 msgr2=0x558d14668880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:40.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/1865784960 shutdown_connections 2024-09-19T06:54:40.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a5dbc1740 1 -- 172.21.15.103:0/1865784960 wait complete. 2024-09-19T06:54:40.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.690+0000 7f2a5dbc1740 20 client.17454 trim_cache size 0 max 16384 2024-09-19T06:54:40.698 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:54:40.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.691+0000 7f2a5b7e8640 1 -- 172.21.15.103:0/95093454 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2a400a33b0 msgr2=0x7f2a400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.691+0000 7f2a5b7e8640 1 --2- 172.21.15.103:0/95093454 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2a400a33b0 0x7f2a400a5870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c020000 tx=0x7f2a4c01d000 comp rx=0 tx=0).stop 2024-09-19T06:54:40.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.691+0000 7f2a5b7e8640 1 -- 172.21.15.103:0/95093454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a54100e20 msgr2=0x7f2a54197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:40.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.691+0000 7f2a5b7e8640 1 --2- 172.21.15.103:0/95093454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a54100e20 0x7f2a54197360 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009900 tx=0x7f2a50009dc0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.691+0000 7f2a5b7e8640 1 -- 172.21.15.103:0/95093454 shutdown_connections 2024-09-19T06:54:40.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.691+0000 7f2a5b7e8640 1 --2- 172.21.15.103:0/95093454 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2a400a33b0 0x7f2a400a5870 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.691+0000 7f2a5b7e8640 1 --2- 172.21.15.103:0/95093454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a54102ef0 0x7f2a5419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.691+0000 7f2a5b7e8640 1 --2- 172.21.15.103:0/95093454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a54102570 0x7f2a541978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.691+0000 7f2a5b7e8640 1 --2- 172.21.15.103:0/95093454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a54100e20 0x7f2a54197360 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:40.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.691+0000 7f2a5b7e8640 1 -- 172.21.15.103:0/95093454 >> 172.21.15.103:0/95093454 conn(0x7f2a540fc7d0 msgr2=0x7f2a54105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:40.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.691+0000 7f2a5b7e8640 1 -- 172.21.15.103:0/95093454 shutdown_connections 2024-09-19T06:54:40.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:40.691+0000 7f2a5b7e8640 1 -- 172.21.15.103:0/95093454 wait complete. 2024-09-19T06:54:41.438 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:54:41.724 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:42.107 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:42 smithi103 ceph-mon[31158]: pgmap v167: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-09-19T06:54:42.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.188+0000 7fc3579c0640 1 -- 172.21.15.103:0/333575307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc350102790 msgr2=0x7fc3501067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:42.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.188+0000 7fc3579c0640 1 --2- 172.21.15.103:0/333575307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc350102790 0x7fc3501067e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc340009900 tx=0x7fc34002a410 comp rx=0 tx=0).stop 2024-09-19T06:54:42.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.188+0000 7fc3579c0640 1 -- 172.21.15.103:0/333575307 shutdown_connections 2024-09-19T06:54:42.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.188+0000 7fc3579c0640 1 --2- 172.21.15.103:0/333575307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc350106f10 0x7fc35010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.188+0000 7fc3579c0640 1 --2- 172.21.15.103:0/333575307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc350102790 0x7fc3501067e0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.188+0000 7fc3579c0640 1 --2- 172.21.15.103:0/333575307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc350101de0 0x7fc3501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.188+0000 7fc3579c0640 1 -- 172.21.15.103:0/333575307 >> 172.21.15.103:0/333575307 conn(0x7fc3500fd8c0 msgr2=0x7fc3500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.188+0000 7fc3579c0640 1 -- 172.21.15.103:0/333575307 shutdown_connections 2024-09-19T06:54:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.188+0000 7fc3579c0640 1 -- 172.21.15.103:0/333575307 wait complete. 2024-09-19T06:54:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc3579c0640 1 Processor -- start 2024-09-19T06:54:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc3579c0640 1 -- start start 2024-09-19T06:54:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc3579c0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc350101de0 0x7fc3500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc3579c0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc350102790 0x7fc350074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc3579c0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc350106f10 0x7fc350074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc3579c0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc350111f70 con 0x7fc350106f10 2024-09-19T06:54:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc3579c0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc350111df0 con 0x7fc350101de0 2024-09-19T06:54:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc3579c0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501120f0 con 0x7fc350102790 2024-09-19T06:54:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc355f5d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc350106f10 0x7fc350074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc355f5d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc350106f10 0x7fc350074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48890/0 (socket says 172.21.15.103:48890) 2024-09-19T06:54:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc355f5d640 1 -- 172.21.15.103:0/1168264422 learned_addr learned my addr 172.21.15.103:0/1168264422 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc35575c640 1 --2- 172.21.15.103:0/1168264422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc350101de0 0x7fc3500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc354f5b640 1 --2- 172.21.15.103:0/1168264422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc350102790 0x7fc350074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc355f5d640 1 -- 172.21.15.103:0/1168264422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc350102790 msgr2=0x7fc350074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc355f5d640 1 --2- 172.21.15.103:0/1168264422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc350102790 0x7fc350074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc355f5d640 1 -- 172.21.15.103:0/1168264422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc350101de0 msgr2=0x7fc3500744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc355f5d640 1 --2- 172.21.15.103:0/1168264422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc350101de0 0x7fc3500744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc355f5d640 1 -- 172.21.15.103:0/1168264422 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3501a6970 con 0x7fc350106f10 2024-09-19T06:54:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.189+0000 7fc355f5d640 1 --2- 172.21.15.103:0/1168264422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc350106f10 0x7fc350074f70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc34800eaa0 tx=0x7fc34800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.190+0000 7fc3467fc640 1 -- 172.21.15.103:0/1168264422 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34800cdb0 con 0x7fc350106f10 2024-09-19T06:54:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.190+0000 7fc3467fc640 1 -- 172.21.15.103:0/1168264422 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc348005020 con 0x7fc350106f10 2024-09-19T06:54:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.190+0000 7fc3467fc640 1 -- 172.21.15.103:0/1168264422 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3480116b0 con 0x7fc350106f10 2024-09-19T06:54:42.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.190+0000 7fc3579c0640 1 -- 172.21.15.103:0/1168264422 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3501a6c60 con 0x7fc350106f10 2024-09-19T06:54:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.190+0000 7fc3579c0640 1 -- 172.21.15.103:0/1168264422 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3501a7150 con 0x7fc350106f10 2024-09-19T06:54:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.192+0000 7fc3467fc640 1 -- 172.21.15.103:0/1168264422 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc348005560 con 0x7fc350106f10 2024-09-19T06:54:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.192+0000 7fc359d99740 1 Processor -- start 2024-09-19T06:54:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.192+0000 7fc359d99740 1 -- start start 2024-09-19T06:54:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc359d99740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687d99090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc359d99740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687d995d0 0x558687d99a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc359d99740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687da4370 0x558687da6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc359d99740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558687cff2f0 con 0x558687d9ab40 2024-09-19T06:54:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc359d99740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558687ca6cf0 con 0x558687da4370 2024-09-19T06:54:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc359d99740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558687cfb2b0 con 0x558687d995d0 2024-09-19T06:54:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc3467fc640 1 --2- 172.21.15.103:0/1168264422 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc33c0a33b0 0x7fc33c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc354f5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687d99090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc354f5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687d99090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48892/0 (socket says 172.21.15.103:48892) 2024-09-19T06:54:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc354f5b640 1 -- 172.21.15.103:0/2637123557 learned_addr learned my addr 172.21.15.103:0/2637123557 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc35575c640 1 --2- 172.21.15.103:0/2637123557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687d995d0 0x558687d99a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc355f5d640 1 --2- 172.21.15.103:0/1168264422 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc33c0a33b0 0x7fc33c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc354f5b640 1 --2- 172.21.15.103:0/2637123557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687da4370 0x558687da6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc354f5b640 1 -- 172.21.15.103:0/2637123557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687d995d0 msgr2=0x558687d99a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc354f5b640 1 --2- 172.21.15.103:0/2637123557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687d995d0 0x558687d99a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc354f5b640 1 -- 172.21.15.103:0/2637123557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687da4370 msgr2=0x558687da6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc354f5b640 1 --2- 172.21.15.103:0/2637123557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687da4370 0x558687da6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc3467fc640 1 -- 172.21.15.103:0/1168264422 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc3480cff20 con 0x7fc350106f10 2024-09-19T06:54:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc354f5b640 1 -- 172.21.15.103:0/2637123557 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558687c8b3a0 con 0x558687d9ab40 2024-09-19T06:54:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc354f5b640 1 --2- 172.21.15.103:0/2637123557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687d99090 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc34002a930 tx=0x7fc340005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852ce6e1792275d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc355f5d640 1 --2- 172.21.15.103:0/1168264422 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc33c0a33b0 0x7fc33c0a5870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc34800efd0 tx=0x7fc34806f040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:42.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc33b7fe640 1 -- 172.21.15.103:0/2637123557 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340043020 con 0x558687d9ab40 2024-09-19T06:54:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.193+0000 7fc33b7fe640 1 -- 172.21.15.103:0/2637123557 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc340031980 con 0x558687d9ab40 2024-09-19T06:54:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.194+0000 7fc33b7fe640 1 -- 172.21.15.103:0/2637123557 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340035050 con 0x558687d9ab40 2024-09-19T06:54:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.194+0000 7fc359d99740 1 -- 172.21.15.103:0/2637123557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 msgr2=0x558687d99090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.194+0000 7fc359d99740 1 --2- 172.21.15.103:0/2637123557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687d99090 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc34002a930 tx=0x7fc340005d70 comp rx=0 tx=0).stop 2024-09-19T06:54:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.194+0000 7fc359d99740 1 -- 172.21.15.103:0/2637123557 shutdown_connections 2024-09-19T06:54:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.194+0000 7fc359d99740 1 --2- 172.21.15.103:0/2637123557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687da4370 0x558687da6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.194+0000 7fc359d99740 1 --2- 172.21.15.103:0/2637123557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687d995d0 0x558687d99a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.194+0000 7fc359d99740 1 --2- 172.21.15.103:0/2637123557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687d99090 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.194+0000 7fc359d99740 1 -- 172.21.15.103:0/2637123557 >> 172.21.15.103:0/2637123557 conn(0x558687d62310 msgr2=0x558687d95c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.194+0000 7fc359d99740 1 -- 172.21.15.103:0/2637123557 shutdown_connections 2024-09-19T06:54:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.194+0000 7fc359d99740 1 -- 172.21.15.103:0/2637123557 wait complete. 2024-09-19T06:54:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.195+0000 7fc359d99740 1 -- start start 2024-09-19T06:54:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.195+0000 7fc359d99740 1 Processor -- start 2024-09-19T06:54:42.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.195+0000 7fc359d99740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687d995d0 0x558687da3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.195+0000 7fc359d99740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687d9ab40 0x558687da0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.195+0000 7fc359d99740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687da4370 0x558687da1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.195+0000 7fc359d99740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558687da7010 con 0x558687da4370 2024-09-19T06:54:42.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.195+0000 7fc359d99740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558687cba060 con 0x558687d995d0 2024-09-19T06:54:42.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.195+0000 7fc359d99740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558687da7190 con 0x558687d9ab40 2024-09-19T06:54:42.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.195+0000 7fc359d99740 10 client.0 _ll_register_callbacks cb 0x558687a0d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:42.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.195+0000 7fc354f5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687da4370 0x558687da1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.195+0000 7fc354f5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687da4370 0x558687da1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48894/0 (socket says 172.21.15.103:48894) 2024-09-19T06:54:42.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.195+0000 7fc354f5b640 1 -- 172.21.15.103:0/3481108024 learned_addr learned my addr 172.21.15.103:0/3481108024 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:42.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.195+0000 7fc354f5b640 1 --2- 172.21.15.103:0/3481108024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687d995d0 0x558687da3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.195+0000 7fc35575c640 1 --2- 172.21.15.103:0/3481108024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687d9ab40 0x558687da0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc354f5b640 1 -- 172.21.15.103:0/3481108024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687d9ab40 msgr2=0x558687da0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:42.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc354f5b640 1 --2- 172.21.15.103:0/3481108024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687d9ab40 0x558687da0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc354f5b640 1 -- 172.21.15.103:0/3481108024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687d995d0 msgr2=0x558687da3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc354f5b640 1 --2- 172.21.15.103:0/3481108024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687d995d0 0x558687da3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc354f5b640 1 -- 172.21.15.103:0/3481108024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558687a703c0 con 0x558687da4370 2024-09-19T06:54:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc354f5b640 1 --2- 172.21.15.103:0/3481108024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687da4370 0x558687da1560 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc340006010 tx=0x7fc340005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfaa87b84ac352a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc338ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc338ff9640 1 -- 172.21.15.103:0/3481108024 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340035070 con 0x558687da4370 2024-09-19T06:54:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc338ff9640 1 -- 172.21.15.103:0/3481108024 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc340004b00 con 0x558687da4370 2024-09-19T06:54:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc359d99740 1 -- 172.21.15.103:0/3481108024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc340031980 con 0x558687da4370 2024-09-19T06:54:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc338ff9640 1 -- 172.21.15.103:0/3481108024 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc340043040 con 0x558687da4370 2024-09-19T06:54:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc359d99740 10 client.17466 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc359d99740 1 -- 172.21.15.103:0/3481108024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558687db2a90 con 0x558687da4370 2024-09-19T06:54:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc338ff9640 1 -- 172.21.15.103:0/3481108024 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc340031980 con 0x558687da4370 2024-09-19T06:54:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc338ff9640 1 -- 172.21.15.103:0/3481108024 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc340003890 con 0x558687da4370 2024-09-19T06:54:42.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc359d99740 10 client.17466 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc359d99740 10 client.17466 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:54:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.196+0000 7fc359d99740 20 client.17466 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.197+0000 7fc359d99740 1 --2- 172.21.15.103:0/3481108024 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558687db3de0 0x558687db61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.197+0000 7fc359d99740 4 client.17466 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.197+0000 7fc359d99740 1 -- 172.21.15.103:0/3481108024 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558687d0c860 con 0x558687db3de0 2024-09-19T06:54:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.197+0000 7fc35575c640 1 --2- 172.21.15.103:0/3481108024 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558687db3de0 0x558687db61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.197+0000 7fc338ff9640 10 client.17466 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:54:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.197+0000 7fc35575c640 1 --2- 172.21.15.103:0/3481108024 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558687db3de0 0x558687db61d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=415f52a32cd895c5 server_cookie=60311a73eebc6acc in_seq=0 out_seq=0 2024-09-19T06:54:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.198+0000 7fc338ff9640 1 -- 172.21.15.103:0/3481108024 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558687d0c860 con 0x558687db3de0 2024-09-19T06:54:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.198+0000 7fc338ff9640 10 client.17466 handle_command_reply: tid=0 2024-09-19T06:54:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.198+0000 7fc359d99740 1 client.17466 shutdown 2024-09-19T06:54:42.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.198+0000 7fc359d99740 1 -- 172.21.15.103:0/3481108024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687da4370 msgr2=0x558687da1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:42.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.198+0000 7fc359d99740 1 --2- 172.21.15.103:0/3481108024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687da4370 0x558687da1560 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc340006010 tx=0x7fc340005570 comp rx=0 tx=0).stop 2024-09-19T06:54:42.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.198+0000 7fc359d99740 1 -- 172.21.15.103:0/3481108024 shutdown_connections 2024-09-19T06:54:42.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.198+0000 7fc359d99740 1 --2- 172.21.15.103:0/3481108024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687da4370 0x558687da1560 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.198+0000 7fc359d99740 1 --2- 172.21.15.103:0/3481108024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687d9ab40 0x558687da0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.198+0000 7fc359d99740 1 --2- 172.21.15.103:0/3481108024 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558687db3de0 0x558687db61d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.198+0000 7fc359d99740 1 --2- 172.21.15.103:0/3481108024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687d995d0 0x558687da3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.198+0000 7fc359d99740 1 -- 172.21.15.103:0/3481108024 >> 172.21.15.103:0/3481108024 conn(0x558687d62310 msgr2=0x558687d9b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:42.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.198+0000 7fc338ff9640 0 client.17466 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:54:42.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.198+0000 7fc359d99740 1 -- 172.21.15.103:0/3481108024 shutdown_connections 2024-09-19T06:54:42.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.198+0000 7fc359d99740 1 -- 172.21.15.103:0/3481108024 wait complete. 2024-09-19T06:54:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.198+0000 7fc359d99740 20 client.17466 trim_cache size 0 max 16384 2024-09-19T06:54:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.211+0000 7fc359d99740 1 Processor -- start 2024-09-19T06:54:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.211+0000 7fc359d99740 1 -- start start 2024-09-19T06:54:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc359d99740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687d995d0 0x558687d9c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc359d99740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687d9c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc359d99740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687da4370 0x558687db8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc359d99740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501af9b0 con 0x558687d9ab40 2024-09-19T06:54:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc359d99740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501b14f0 con 0x558687d995d0 2024-09-19T06:54:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc359d99740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501b0080 con 0x558687da4370 2024-09-19T06:54:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc35575c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687d9c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc35575c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687d9c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48896/0 (socket says 172.21.15.103:48896) 2024-09-19T06:54:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc35575c640 1 -- 172.21.15.103:0/6381458 learned_addr learned my addr 172.21.15.103:0/6381458 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc354f5b640 1 --2- 172.21.15.103:0/6381458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687da4370 0x558687db8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc35575c640 1 -- 172.21.15.103:0/6381458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687da4370 msgr2=0x558687db8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc35575c640 1 --2- 172.21.15.103:0/6381458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687da4370 0x558687db8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc35575c640 1 -- 172.21.15.103:0/6381458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687d995d0 msgr2=0x558687d9c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc35575c640 1 --2- 172.21.15.103:0/6381458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687d995d0 0x558687d9c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc35575c640 1 -- 172.21.15.103:0/6381458 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558687da6cc0 con 0x558687d9ab40 2024-09-19T06:54:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc35575c640 1 --2- 172.21.15.103:0/6381458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687d9c660 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x558687da0850 tx=0x7fc34c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89076d2be9056ad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc33affd640 1 -- 172.21.15.103:0/6381458 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c004220 con 0x558687d9ab40 2024-09-19T06:54:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc33affd640 1 -- 172.21.15.103:0/6381458 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc34c0043c0 con 0x558687d9ab40 2024-09-19T06:54:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.212+0000 7fc33affd640 1 -- 172.21.15.103:0/6381458 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c025ad0 con 0x558687d9ab40 2024-09-19T06:54:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.213+0000 7fc359d99740 1 -- 172.21.15.103:0/6381458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 msgr2=0x558687d9c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.213+0000 7fc359d99740 1 --2- 172.21.15.103:0/6381458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687d9c660 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x558687da0850 tx=0x7fc34c010800 comp rx=0 tx=0).stop 2024-09-19T06:54:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.213+0000 7fc359d99740 1 -- 172.21.15.103:0/6381458 shutdown_connections 2024-09-19T06:54:42.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.213+0000 7fc359d99740 1 --2- 172.21.15.103:0/6381458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687da4370 0x558687db8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.213+0000 7fc359d99740 1 --2- 172.21.15.103:0/6381458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687d9c660 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.213+0000 7fc359d99740 1 --2- 172.21.15.103:0/6381458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687d995d0 0x558687d9c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.213+0000 7fc359d99740 1 -- 172.21.15.103:0/6381458 >> 172.21.15.103:0/6381458 conn(0x558687d62310 msgr2=0x558687da00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:42.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.213+0000 7fc359d99740 1 -- 172.21.15.103:0/6381458 shutdown_connections 2024-09-19T06:54:42.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.213+0000 7fc359d99740 1 -- 172.21.15.103:0/6381458 wait complete. 2024-09-19T06:54:42.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.213+0000 7fc359d99740 1 -- start start 2024-09-19T06:54:42.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.213+0000 7fc359d99740 1 Processor -- start 2024-09-19T06:54:42.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc359d99740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687d995d0 0x558687db93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc359d99740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687db6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc359d99740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687da4370 0x558687db6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc359d99740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501b0e20 con 0x558687d9ab40 2024-09-19T06:54:42.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc359d99740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501b0750 con 0x558687d995d0 2024-09-19T06:54:42.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc359d99740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501ad970 con 0x558687da4370 2024-09-19T06:54:42.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc359d99740 10 client.0 _ll_register_callbacks cb 0x7fc34c00a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:42.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc35575c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687db6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc35575c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687db6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48912/0 (socket says 172.21.15.103:48912) 2024-09-19T06:54:42.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc35575c640 1 -- 172.21.15.103:0/1777506956 learned_addr learned my addr 172.21.15.103:0/1777506956 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:42.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc354f5b640 1 --2- 172.21.15.103:0/1777506956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687da4370 0x558687db6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc354f5b640 1 --2- 172.21.15.103:0/1777506956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687d995d0 0x558687db93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc35575c640 1 -- 172.21.15.103:0/1777506956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687da4370 msgr2=0x558687db6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:42.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc35575c640 1 --2- 172.21.15.103:0/1777506956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687da4370 0x558687db6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc35575c640 1 -- 172.21.15.103:0/1777506956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687d995d0 msgr2=0x558687db93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:42.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc35575c640 1 --2- 172.21.15.103:0/1777506956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687d995d0 0x558687db93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.214+0000 7fc35575c640 1 -- 172.21.15.103:0/1777506956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558687db7320 con 0x558687d9ab40 2024-09-19T06:54:42.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.215+0000 7fc35575c640 1 --2- 172.21.15.103:0/1777506956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687db6740 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fc34c024000 tx=0x7fc34c004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b8aff8096f8017 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:42.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.215+0000 7fc338ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:42.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.215+0000 7fc338ff9640 1 -- 172.21.15.103:0/1777506956 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c025400 con 0x558687d9ab40 2024-09-19T06:54:42.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.215+0000 7fc338ff9640 1 -- 172.21.15.103:0/1777506956 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc34c005260 con 0x558687d9ab40 2024-09-19T06:54:42.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.215+0000 7fc359d99740 1 -- 172.21.15.103:0/1777506956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc34c0043c0 con 0x558687d9ab40 2024-09-19T06:54:42.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.215+0000 7fc338ff9640 1 -- 172.21.15.103:0/1777506956 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc34c005460 con 0x558687d9ab40 2024-09-19T06:54:42.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.215+0000 7fc359d99740 10 client.17472 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:42.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.215+0000 7fc359d99740 1 -- 172.21.15.103:0/1777506956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558687e1e590 con 0x558687d9ab40 2024-09-19T06:54:42.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.215+0000 7fc338ff9640 1 -- 172.21.15.103:0/1777506956 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc34c0043c0 con 0x558687d9ab40 2024-09-19T06:54:42.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.215+0000 7fc338ff9640 1 -- 172.21.15.103:0/1777506956 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc34c004b40 con 0x558687d9ab40 2024-09-19T06:54:42.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.215+0000 7fc359d99740 10 client.17472 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:42.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.215+0000 7fc359d99740 10 client.17472 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:54:42.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.215+0000 7fc359d99740 20 client.17472 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:42.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.215+0000 7fc359d99740 1 --2- 172.21.15.103:0/1777506956 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558687e1ec60 0x558687e21080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:42.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.215+0000 7fc359d99740 4 client.17472 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:54:42.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.215+0000 7fc359d99740 1 -- 172.21.15.103:0/1777506956 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558687d06d70 con 0x558687e1ec60 2024-09-19T06:54:42.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.216+0000 7fc354f5b640 1 --2- 172.21.15.103:0/1777506956 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558687e1ec60 0x558687e21080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:42.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.216+0000 7fc338ff9640 10 client.17472 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:54:42.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.216+0000 7fc354f5b640 1 --2- 172.21.15.103:0/1777506956 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558687e1ec60 0x558687e21080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c47fe8a7410a0f52 server_cookie=ceaf3ca86ab526b4 in_seq=0 out_seq=0 2024-09-19T06:54:42.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.216+0000 7fc338ff9640 1 -- 172.21.15.103:0/1777506956 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558687d06d70 con 0x558687e1ec60 2024-09-19T06:54:42.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.216+0000 7fc338ff9640 10 client.17472 handle_command_reply: tid=0 2024-09-19T06:54:42.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.216+0000 7fc359d99740 1 client.17472 shutdown 2024-09-19T06:54:42.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.216+0000 7fc359d99740 1 -- 172.21.15.103:0/1777506956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 msgr2=0x558687db6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:42.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.216+0000 7fc359d99740 1 --2- 172.21.15.103:0/1777506956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687db6740 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fc34c024000 tx=0x7fc34c004250 comp rx=0 tx=0).stop 2024-09-19T06:54:42.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.216+0000 7fc359d99740 1 -- 172.21.15.103:0/1777506956 shutdown_connections 2024-09-19T06:54:42.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.216+0000 7fc359d99740 1 --2- 172.21.15.103:0/1777506956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558687da4370 0x558687db6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.216+0000 7fc359d99740 1 --2- 172.21.15.103:0/1777506956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558687d9ab40 0x558687db6740 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.216+0000 7fc359d99740 1 --2- 172.21.15.103:0/1777506956 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558687e1ec60 0x558687e21080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.216+0000 7fc359d99740 1 --2- 172.21.15.103:0/1777506956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558687d995d0 0x558687db93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.216+0000 7fc359d99740 1 -- 172.21.15.103:0/1777506956 >> 172.21.15.103:0/1777506956 conn(0x558687d62310 msgr2=0x558687da2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:42.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.216+0000 7fc338ff9640 0 client.17472 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:54:42.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.216+0000 7fc359d99740 1 -- 172.21.15.103:0/1777506956 shutdown_connections 2024-09-19T06:54:42.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.217+0000 7fc359d99740 1 -- 172.21.15.103:0/1777506956 wait complete. 2024-09-19T06:54:42.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.217+0000 7fc359d99740 20 client.17472 trim_cache size 0 max 16384 2024-09-19T06:54:42.235 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:54:42.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.217+0000 7fc3579c0640 1 -- 172.21.15.103:0/1168264422 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc33c0a33b0 msgr2=0x7fc33c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:42.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.217+0000 7fc3579c0640 1 --2- 172.21.15.103:0/1168264422 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc33c0a33b0 0x7fc33c0a5870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc34800efd0 tx=0x7fc34806f040 comp rx=0 tx=0).stop 2024-09-19T06:54:42.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.217+0000 7fc3579c0640 1 -- 172.21.15.103:0/1168264422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc350106f10 msgr2=0x7fc350074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:42.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.217+0000 7fc3579c0640 1 --2- 172.21.15.103:0/1168264422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc350106f10 0x7fc350074f70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc34800eaa0 tx=0x7fc34800ef60 comp rx=0 tx=0).stop 2024-09-19T06:54:42.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.217+0000 7fc3579c0640 1 -- 172.21.15.103:0/1168264422 shutdown_connections 2024-09-19T06:54:42.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.217+0000 7fc3579c0640 1 --2- 172.21.15.103:0/1168264422 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc33c0a33b0 0x7fc33c0a5870 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.217+0000 7fc3579c0640 1 --2- 172.21.15.103:0/1168264422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc350106f10 0x7fc350074f70 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.217+0000 7fc3579c0640 1 --2- 172.21.15.103:0/1168264422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc350102790 0x7fc350074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.217+0000 7fc3579c0640 1 --2- 172.21.15.103:0/1168264422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc350101de0 0x7fc3500744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:42.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.217+0000 7fc3579c0640 1 -- 172.21.15.103:0/1168264422 >> 172.21.15.103:0/1168264422 conn(0x7fc3500fd8c0 msgr2=0x7fc3500fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:42.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.217+0000 7fc3579c0640 1 -- 172.21.15.103:0/1168264422 shutdown_connections 2024-09-19T06:54:42.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:42.217+0000 7fc3579c0640 1 -- 172.21.15.103:0/1168264422 wait complete. 2024-09-19T06:54:42.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:42 smithi134 ceph-mon[35356]: pgmap v167: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-09-19T06:54:42.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:42 smithi174 ceph-mon[35637]: pgmap v167: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-09-19T06:54:42.841 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:54:43.088 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:43.464 INFO:tasks.workunit.client.0.smithi103.stdout:64/600: open fd = 3 2024-09-19T06:54:43.465 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.465 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.465 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.465 INFO:tasks.workunit.client.0.smithi103.stdout:65/600: open fd = 3 2024-09-19T06:54:43.465 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.465 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.465 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.465 INFO:tasks.workunit.client.0.smithi103.stdout:66/600: open fd = 3 2024-09-19T06:54:43.465 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.465 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.465 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.465 INFO:tasks.workunit.client.0.smithi103.stdout:67/600: open fd = 3 2024-09-19T06:54:43.465 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.466 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.466 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.466 INFO:tasks.workunit.client.0.smithi103.stdout:68/600: open fd = 3 2024-09-19T06:54:43.466 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.466 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.466 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.466 INFO:tasks.workunit.client.0.smithi103.stdout:69/600: open fd = 3 2024-09-19T06:54:43.466 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.466 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.466 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.466 INFO:tasks.workunit.client.0.smithi103.stdout:70/600: open fd = 3 2024-09-19T06:54:43.466 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.466 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.466 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.467 INFO:tasks.workunit.client.0.smithi103.stdout:71/600: open fd = 3 2024-09-19T06:54:43.467 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.467 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.467 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.467 INFO:tasks.workunit.client.0.smithi103.stdout:72/600: open fd = 3 2024-09-19T06:54:43.467 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.467 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.467 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.467 INFO:tasks.workunit.client.0.smithi103.stdout:73/600: open fd = 3 2024-09-19T06:54:43.467 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.467 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.467 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.467 INFO:tasks.workunit.client.0.smithi103.stdout:74/600: open fd = 3 2024-09-19T06:54:43.467 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.468 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.468 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.468 INFO:tasks.workunit.client.0.smithi103.stdout:75/600: open fd = 3 2024-09-19T06:54:43.468 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.468 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.468 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.468 INFO:tasks.workunit.client.0.smithi103.stdout:76/600: open fd = 3 2024-09-19T06:54:43.468 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.468 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.468 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.468 INFO:tasks.workunit.client.0.smithi103.stdout:77/600: open fd = 3 2024-09-19T06:54:43.468 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.468 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.468 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.469 INFO:tasks.workunit.client.0.smithi103.stdout:78/600: open fd = 3 2024-09-19T06:54:43.469 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.469 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.469 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.469 INFO:tasks.workunit.client.0.smithi103.stdout:79/600: open fd = 3 2024-09-19T06:54:43.469 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.469 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.469 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.469 INFO:tasks.workunit.client.0.smithi103.stdout:80/600: open fd = 3 2024-09-19T06:54:43.469 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.469 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.469 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.469 INFO:tasks.workunit.client.0.smithi103.stdout:81/600: open fd = 3 2024-09-19T06:54:43.469 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.470 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.470 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.470 INFO:tasks.workunit.client.0.smithi103.stdout:82/600: open fd = 3 2024-09-19T06:54:43.470 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.470 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.470 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.470 INFO:tasks.workunit.client.0.smithi103.stdout:83/600: open fd = 3 2024-09-19T06:54:43.470 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.470 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.470 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.470 INFO:tasks.workunit.client.0.smithi103.stdout:84/600: open fd = 3 2024-09-19T06:54:43.470 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.470 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.470 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.471 INFO:tasks.workunit.client.0.smithi103.stdout:85/600: open fd = 3 2024-09-19T06:54:43.471 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.471 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.471 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.471 INFO:tasks.workunit.client.0.smithi103.stdout:86/600: open fd = 3 2024-09-19T06:54:43.471 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.471 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.471 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.471 INFO:tasks.workunit.client.0.smithi103.stdout:87/600: open fd = 3 2024-09-19T06:54:43.471 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.471 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.471 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.471 INFO:tasks.workunit.client.0.smithi103.stdout:88/600: open fd = 3 2024-09-19T06:54:43.471 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.472 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.472 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.472 INFO:tasks.workunit.client.0.smithi103.stdout:89/600: open fd = 3 2024-09-19T06:54:43.472 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.472 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.472 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.472 INFO:tasks.workunit.client.0.smithi103.stdout:90/600: open fd = 3 2024-09-19T06:54:43.472 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.472 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.473 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.473 INFO:tasks.workunit.client.0.smithi103.stdout:91/600: open fd = 3 2024-09-19T06:54:43.473 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.473 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.473 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.473 INFO:tasks.workunit.client.0.smithi103.stdout:92/600: open fd = 3 2024-09-19T06:54:43.473 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.473 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.473 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.473 INFO:tasks.workunit.client.0.smithi103.stdout:93/600: open fd = 3 2024-09-19T06:54:43.473 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.473 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.473 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.473 INFO:tasks.workunit.client.0.smithi103.stdout:94/600: open fd = 3 2024-09-19T06:54:43.474 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.474 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.474 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.474 INFO:tasks.workunit.client.0.smithi103.stdout:95/600: open fd = 3 2024-09-19T06:54:43.474 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.474 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.474 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.474 INFO:tasks.workunit.client.0.smithi103.stdout:96/600: open fd = 3 2024-09-19T06:54:43.474 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.474 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.474 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.474 INFO:tasks.workunit.client.0.smithi103.stdout:97/600: open fd = 3 2024-09-19T06:54:43.474 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.474 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.474 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.475 INFO:tasks.workunit.client.0.smithi103.stdout:98/600: open fd = 3 2024-09-19T06:54:43.475 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.475 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.475 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.475 INFO:tasks.workunit.client.0.smithi103.stdout:99/600: open fd = 3 2024-09-19T06:54:43.475 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.475 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.475 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.475 INFO:tasks.workunit.client.0.smithi103.stdout:100/600: open fd = 3 2024-09-19T06:54:43.475 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.475 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.475 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.475 INFO:tasks.workunit.client.0.smithi103.stdout:101/600: open fd = 3 2024-09-19T06:54:43.475 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.476 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.476 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.476 INFO:tasks.workunit.client.0.smithi103.stdout:102/600: open fd = 3 2024-09-19T06:54:43.476 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.476 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.476 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.476 INFO:tasks.workunit.client.0.smithi103.stdout:103/600: open fd = 3 2024-09-19T06:54:43.476 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.476 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.476 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.476 INFO:tasks.workunit.client.0.smithi103.stdout:104/600: open fd = 3 2024-09-19T06:54:43.476 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.476 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.477 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.477 INFO:tasks.workunit.client.0.smithi103.stdout:105/600: open fd = 3 2024-09-19T06:54:43.477 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.477 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.477 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.477 INFO:tasks.workunit.client.0.smithi103.stdout:106/600: open fd = 3 2024-09-19T06:54:43.477 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.477 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.477 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.477 INFO:tasks.workunit.client.0.smithi103.stdout:107/600: open fd = 3 2024-09-19T06:54:43.477 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.477 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.478 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.478 INFO:tasks.workunit.client.0.smithi103.stdout:108/600: open fd = 3 2024-09-19T06:54:43.478 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.478 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.478 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.478 INFO:tasks.workunit.client.0.smithi103.stdout:109/600: open fd = 3 2024-09-19T06:54:43.478 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.478 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.478 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.478 INFO:tasks.workunit.client.0.smithi103.stdout:110/600: open fd = 3 2024-09-19T06:54:43.478 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.478 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.479 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.479 INFO:tasks.workunit.client.0.smithi103.stdout:111/600: open fd = 3 2024-09-19T06:54:43.479 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.479 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.479 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.479 INFO:tasks.workunit.client.0.smithi103.stdout:112/600: open fd = 3 2024-09-19T06:54:43.479 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.479 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.479 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.479 INFO:tasks.workunit.client.0.smithi103.stdout:113/600: open fd = 3 2024-09-19T06:54:43.479 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.479 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.479 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.479 INFO:tasks.workunit.client.0.smithi103.stdout:114/600: open fd = 3 2024-09-19T06:54:43.480 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.480 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.480 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.480 INFO:tasks.workunit.client.0.smithi103.stdout:115/600: open fd = 3 2024-09-19T06:54:43.480 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.480 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.480 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.480 INFO:tasks.workunit.client.0.smithi103.stdout:116/600: open fd = 3 2024-09-19T06:54:43.480 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.480 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.480 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.480 INFO:tasks.workunit.client.0.smithi103.stdout:117/600: open fd = 3 2024-09-19T06:54:43.480 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.481 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.481 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.481 INFO:tasks.workunit.client.0.smithi103.stdout:118/600: open fd = 3 2024-09-19T06:54:43.481 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.481 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.481 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.481 INFO:tasks.workunit.client.0.smithi103.stdout:119/600: open fd = 3 2024-09-19T06:54:43.481 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.481 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.482 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.482 INFO:tasks.workunit.client.0.smithi103.stdout:120/600: open fd = 3 2024-09-19T06:54:43.482 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.482 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.482 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.482 INFO:tasks.workunit.client.0.smithi103.stdout:121/600: open fd = 3 2024-09-19T06:54:43.482 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.482 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.482 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.482 INFO:tasks.workunit.client.0.smithi103.stdout:122/600: open fd = 3 2024-09-19T06:54:43.482 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.482 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.482 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.482 INFO:tasks.workunit.client.0.smithi103.stdout:123/600: open fd = 3 2024-09-19T06:54:43.483 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.483 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.483 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.483 INFO:tasks.workunit.client.0.smithi103.stdout:124/600: open fd = 3 2024-09-19T06:54:43.483 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.483 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.483 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.483 INFO:tasks.workunit.client.0.smithi103.stdout:125/600: open fd = 3 2024-09-19T06:54:43.483 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.483 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.483 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:54:43.483 INFO:tasks.workunit.client.0.smithi103.stdout:126/600: open fd = 3 2024-09-19T06:54:43.483 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:54:43.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.528+0000 7fa7b04a0640 1 -- 172.21.15.103:0/1080660016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7a810e430 msgr2=0x7fa7a8110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.528+0000 7fa7b04a0640 1 --2- 172.21.15.103:0/1080660016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7a810e430 0x7fa7a8110820 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009ca0 tx=0x7fa7a002a950 comp rx=0 tx=0).stop 2024-09-19T06:54:43.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.528+0000 7fa7b04a0640 1 -- 172.21.15.103:0/1080660016 shutdown_connections 2024-09-19T06:54:43.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.528+0000 7fa7b04a0640 1 --2- 172.21.15.103:0/1080660016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7a810e430 0x7fa7a8110820 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.528+0000 7fa7b04a0640 1 --2- 172.21.15.103:0/1080660016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7a8074e30 0x7fa7a810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.528+0000 7fa7b04a0640 1 --2- 172.21.15.103:0/1080660016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7a8074510 0x7fa7a80748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.528+0000 7fa7b04a0640 1 -- 172.21.15.103:0/1080660016 >> 172.21.15.103:0/1080660016 conn(0x7fa7a80fd8b0 msgr2=0x7fa7a80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:43.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.528+0000 7fa7b04a0640 1 -- 172.21.15.103:0/1080660016 shutdown_connections 2024-09-19T06:54:43.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.528+0000 7fa7b04a0640 1 -- 172.21.15.103:0/1080660016 wait complete. 2024-09-19T06:54:43.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7b04a0640 1 Processor -- start 2024-09-19T06:54:43.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7b04a0640 1 -- start start 2024-09-19T06:54:43.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7b04a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7a8074510 0x7fa7a819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7b04a0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7a8074e30 0x7fa7a819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7b04a0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7a810e430 0x7fa7a819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7b04a0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81141a0 con 0x7fa7a8074510 2024-09-19T06:54:43.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7b04a0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a8114020 con 0x7fa7a810e430 2024-09-19T06:54:43.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7b04a0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a8114320 con 0x7fa7a8074e30 2024-09-19T06:54:43.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7ae23c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7a8074510 0x7fa7a819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7ae23c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7a8074510 0x7fa7a819a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48924/0 (socket says 172.21.15.103:48924) 2024-09-19T06:54:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7ae23c640 1 -- 172.21.15.103:0/561161581 learned_addr learned my addr 172.21.15.103:0/561161581 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7ada3b640 1 --2- 172.21.15.103:0/561161581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7a8074e30 0x7fa7a819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7aea3d640 1 --2- 172.21.15.103:0/561161581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7a810e430 0x7fa7a819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7ae23c640 1 -- 172.21.15.103:0/561161581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7a8074e30 msgr2=0x7fa7a819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7ae23c640 1 --2- 172.21.15.103:0/561161581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7a8074e30 0x7fa7a819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7ae23c640 1 -- 172.21.15.103:0/561161581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7a810e430 msgr2=0x7fa7a819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7ae23c640 1 --2- 172.21.15.103:0/561161581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7a810e430 0x7fa7a819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7ae23c640 1 -- 172.21.15.103:0/561161581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a819f6f0 con 0x7fa7a8074510 2024-09-19T06:54:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.529+0000 7fa7ae23c640 1 --2- 172.21.15.103:0/561161581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7a8074510 0x7fa7a819a740 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400d950 tx=0x7fa7a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.530+0000 7fa79f7fe640 1 -- 172.21.15.103:0/561161581 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4017070 con 0x7fa7a8074510 2024-09-19T06:54:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.530+0000 7fa7b04a0640 1 -- 172.21.15.103:0/561161581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a819f9e0 con 0x7fa7a8074510 2024-09-19T06:54:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.530+0000 7fa79f7fe640 1 -- 172.21.15.103:0/561161581 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a4009c30 con 0x7fa7a8074510 2024-09-19T06:54:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.530+0000 7fa79f7fe640 1 -- 172.21.15.103:0/561161581 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a400f6b0 con 0x7fa7a8074510 2024-09-19T06:54:43.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.530+0000 7fa7b04a0640 1 -- 172.21.15.103:0/561161581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a81a72c0 con 0x7fa7a8074510 2024-09-19T06:54:43.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.532+0000 7fa7b2879740 1 Processor -- start 2024-09-19T06:54:43.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.532+0000 7fa7b2879740 1 -- start start 2024-09-19T06:54:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.532+0000 7fa79f7fe640 1 -- 172.21.15.103:0/561161581 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa7a40109d0 con 0x7fa7a8074510 2024-09-19T06:54:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.532+0000 7fa7b2879740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c294090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.532+0000 7fa7b2879740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c2945d0 0x560d4c294a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.532+0000 7fa7b2879740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c29f370 0x560d4c2a1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.532+0000 7fa7b2879740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560d4c1fa2f0 con 0x560d4c295b40 2024-09-19T06:54:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.532+0000 7fa7b2879740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560d4c1a1cf0 con 0x560d4c29f370 2024-09-19T06:54:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.532+0000 7fa7b2879740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560d4c1f62b0 con 0x560d4c2945d0 2024-09-19T06:54:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.533+0000 7fa7aea3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c294090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.533+0000 7fa7aea3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c294090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48932/0 (socket says 172.21.15.103:48932) 2024-09-19T06:54:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.533+0000 7fa7aea3d640 1 -- 172.21.15.103:0/2602583123 learned_addr learned my addr 172.21.15.103:0/2602583123 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.533+0000 7fa7ae23c640 1 --2- 172.21.15.103:0/2602583123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c2945d0 0x560d4c294a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.533+0000 7fa7ada3b640 1 --2- 172.21.15.103:0/2602583123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c29f370 0x560d4c2a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.533+0000 7fa7aea3d640 1 -- 172.21.15.103:0/2602583123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c2945d0 msgr2=0x560d4c294a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.533+0000 7fa7aea3d640 1 --2- 172.21.15.103:0/2602583123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c2945d0 0x560d4c294a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.533+0000 7fa7aea3d640 1 -- 172.21.15.103:0/2602583123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c29f370 msgr2=0x560d4c2a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.533+0000 7fa7aea3d640 1 --2- 172.21.15.103:0/2602583123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c29f370 0x560d4c2a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.533+0000 7fa7aea3d640 1 -- 172.21.15.103:0/2602583123 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d4c1863a0 con 0x560d4c295b40 2024-09-19T06:54:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.533+0000 7fa7aea3d640 1 --2- 172.21.15.103:0/2602583123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c294090 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0035000 tx=0x7fa7a0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74538387dc745ce1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.533+0000 7fa79f7fe640 1 --2- 172.21.15.103:0/561161581 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7940a33b0 0x7fa7940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.533+0000 7fa7ae23c640 1 --2- 172.21.15.103:0/561161581 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7940a33b0 0x7fa7940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.533+0000 7fa79cff9640 1 -- 172.21.15.103:0/2602583123 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0034040 con 0x560d4c295b40 2024-09-19T06:54:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.533+0000 7fa79cff9640 1 -- 172.21.15.103:0/2602583123 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a0003970 con 0x560d4c295b40 2024-09-19T06:54:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.533+0000 7fa79cff9640 1 -- 172.21.15.103:0/2602583123 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0004610 con 0x560d4c295b40 2024-09-19T06:54:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.534+0000 7fa79f7fe640 1 -- 172.21.15.103:0/561161581 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa7a40cfa50 con 0x7fa7a8074510 2024-09-19T06:54:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.534+0000 7fa7b2879740 1 -- 172.21.15.103:0/2602583123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 msgr2=0x560d4c294090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.534+0000 7fa7ae23c640 1 --2- 172.21.15.103:0/561161581 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7940a33b0 0x7fa7940a5870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x560d4c291ac0 tx=0x7fa7a4086040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.534+0000 7fa7b2879740 1 --2- 172.21.15.103:0/2602583123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c294090 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0035000 tx=0x7fa7a0009a20 comp rx=0 tx=0).stop 2024-09-19T06:54:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.534+0000 7fa7b2879740 1 -- 172.21.15.103:0/2602583123 shutdown_connections 2024-09-19T06:54:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.534+0000 7fa7b2879740 1 --2- 172.21.15.103:0/2602583123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c29f370 0x560d4c2a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.534+0000 7fa7b2879740 1 --2- 172.21.15.103:0/2602583123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c2945d0 0x560d4c294a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.534+0000 7fa7b2879740 1 --2- 172.21.15.103:0/2602583123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c294090 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.534+0000 7fa7b2879740 1 -- 172.21.15.103:0/2602583123 >> 172.21.15.103:0/2602583123 conn(0x560d4c25d310 msgr2=0x560d4c290c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.534+0000 7fa7b2879740 1 -- 172.21.15.103:0/2602583123 shutdown_connections 2024-09-19T06:54:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.534+0000 7fa7b2879740 1 -- 172.21.15.103:0/2602583123 wait complete. 2024-09-19T06:54:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.535+0000 7fa7b2879740 1 -- start start 2024-09-19T06:54:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.535+0000 7fa7b2879740 1 Processor -- start 2024-09-19T06:54:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.535+0000 7fa7b2879740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c2945d0 0x560d4c29ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.535+0000 7fa7b2879740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c29c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.535+0000 7fa7b2879740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c29f370 0x560d4c29c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.535+0000 7fa7b2879740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560d4c2a2010 con 0x560d4c295b40 2024-09-19T06:54:43.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.535+0000 7fa7b2879740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560d4c1b5060 con 0x560d4c2945d0 2024-09-19T06:54:43.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.535+0000 7fa7b2879740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560d4c2a2190 con 0x560d4c29f370 2024-09-19T06:54:43.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.535+0000 7fa7b2879740 10 client.0 _ll_register_callbacks cb 0x560d4bf088c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:43.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.535+0000 7fa7ada3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c29c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.535+0000 7fa7ada3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c29c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48942/0 (socket says 172.21.15.103:48942) 2024-09-19T06:54:43.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.535+0000 7fa7ada3b640 1 -- 172.21.15.103:0/111604760 learned_addr learned my addr 172.21.15.103:0/111604760 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:43.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.535+0000 7fa7aea3d640 1 --2- 172.21.15.103:0/111604760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c2945d0 0x560d4c29ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.535+0000 7fa7aea3d640 1 --2- 172.21.15.103:0/111604760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c29f370 0x560d4c29c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.536+0000 7fa7ada3b640 1 -- 172.21.15.103:0/111604760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c29f370 msgr2=0x560d4c29c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:43.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.536+0000 7fa7ada3b640 1 --2- 172.21.15.103:0/111604760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c29f370 0x560d4c29c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.536+0000 7fa7ada3b640 1 -- 172.21.15.103:0/111604760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c2945d0 msgr2=0x560d4c29ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:43.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.536+0000 7fa7ada3b640 1 --2- 172.21.15.103:0/111604760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c2945d0 0x560d4c29ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.536+0000 7fa7ada3b640 1 -- 172.21.15.103:0/111604760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d4bf6b3c0 con 0x560d4c295b40 2024-09-19T06:54:43.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.536+0000 7fa7aea3d640 1 --2- 172.21.15.103:0/111604760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c2945d0 0x560d4c29ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:43.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.536+0000 7fa7ada3b640 1 --2- 172.21.15.103:0/111604760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c29c000 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa79800bf90 tx=0x7fa79800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a112550af43a44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:43.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.536+0000 7fa791ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:43.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.536+0000 7fa791ffb640 1 -- 172.21.15.103:0/111604760 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798018020 con 0x560d4c295b40 2024-09-19T06:54:43.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.536+0000 7fa791ffb640 1 -- 172.21.15.103:0/111604760 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa798014870 con 0x560d4c295b40 2024-09-19T06:54:43.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.536+0000 7fa7b2879740 1 -- 172.21.15.103:0/111604760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a0003970 con 0x560d4c295b40 2024-09-19T06:54:43.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.536+0000 7fa791ffb640 1 -- 172.21.15.103:0/111604760 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa798023050 con 0x560d4c295b40 2024-09-19T06:54:43.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.536+0000 7fa7b2879740 10 client.17484 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:43.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.536+0000 7fa7b2879740 1 -- 172.21.15.103:0/111604760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560d4c2adaf0 con 0x560d4c295b40 2024-09-19T06:54:43.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.536+0000 7fa791ffb640 1 -- 172.21.15.103:0/111604760 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a0003970 con 0x560d4c295b40 2024-09-19T06:54:43.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.536+0000 7fa791ffb640 1 -- 172.21.15.103:0/111604760 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa798013b20 con 0x560d4c295b40 2024-09-19T06:54:43.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.537+0000 7fa7b2879740 10 client.17484 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:43.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.537+0000 7fa7b2879740 10 client.17484 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:54:43.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.537+0000 7fa7b2879740 20 client.17484 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:43.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.537+0000 7fa7b2879740 1 --2- 172.21.15.103:0/111604760 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x560d4c2af050 0x560d4c2b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.537+0000 7fa7b2879740 4 client.17484 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:43.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.537+0000 7fa7b2879740 1 -- 172.21.15.103:0/111604760 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d4c207860 con 0x560d4c2af050 2024-09-19T06:54:43.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.537+0000 7fa7aea3d640 1 --2- 172.21.15.103:0/111604760 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x560d4c2af050 0x560d4c2b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.537+0000 7fa7aea3d640 1 --2- 172.21.15.103:0/111604760 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x560d4c2af050 0x560d4c2b1470 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b5924c805985e1c server_cookie=2facb5ab28e5c5e in_seq=0 out_seq=0 2024-09-19T06:54:43.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.537+0000 7fa791ffb640 10 client.17484 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:54:43.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.538+0000 7fa791ffb640 1 -- 172.21.15.103:0/111604760 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d4c207860 con 0x560d4c2af050 2024-09-19T06:54:43.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.538+0000 7fa791ffb640 10 client.17484 handle_command_reply: tid=0 2024-09-19T06:54:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.538+0000 7fa7b2879740 1 client.17484 shutdown 2024-09-19T06:54:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.538+0000 7fa7b2879740 1 -- 172.21.15.103:0/111604760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 msgr2=0x560d4c29c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.538+0000 7fa7b2879740 1 --2- 172.21.15.103:0/111604760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c29c000 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa79800bf90 tx=0x7fa79800f670 comp rx=0 tx=0).stop 2024-09-19T06:54:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.539+0000 7fa7b2879740 1 -- 172.21.15.103:0/111604760 shutdown_connections 2024-09-19T06:54:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.539+0000 7fa7b2879740 1 --2- 172.21.15.103:0/111604760 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x560d4c2af050 0x560d4c2b1470 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.539+0000 7fa7b2879740 1 --2- 172.21.15.103:0/111604760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c29f370 0x560d4c29c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.539+0000 7fa7b2879740 1 --2- 172.21.15.103:0/111604760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c29c000 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.539+0000 7fa791ffb640 0 client.17484 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:54:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.539+0000 7fa7b2879740 1 --2- 172.21.15.103:0/111604760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c2945d0 0x560d4c29ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.539+0000 7fa7b2879740 1 -- 172.21.15.103:0/111604760 >> 172.21.15.103:0/111604760 conn(0x560d4c25d310 msgr2=0x560d4c296960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:43.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.539+0000 7fa7b2879740 1 -- 172.21.15.103:0/111604760 shutdown_connections 2024-09-19T06:54:43.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.539+0000 7fa7b2879740 1 -- 172.21.15.103:0/111604760 wait complete. 2024-09-19T06:54:43.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.539+0000 7fa7b2879740 20 client.17484 trim_cache size 0 max 16384 2024-09-19T06:54:43.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.551+0000 7fa7b2879740 1 Processor -- start 2024-09-19T06:54:43.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.551+0000 7fa7b2879740 1 -- start start 2024-09-19T06:54:43.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7b2879740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c2945d0 0x560d4c297770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7b2879740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c2ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7b2879740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c29f370 0x560d4c29df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7b2879740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81afc20 con 0x560d4c295b40 2024-09-19T06:54:43.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7b2879740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81b1760 con 0x560d4c2945d0 2024-09-19T06:54:43.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7b2879740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81b02f0 con 0x560d4c29f370 2024-09-19T06:54:43.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7ada3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c2ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7ada3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c2ab2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48958/0 (socket says 172.21.15.103:48958) 2024-09-19T06:54:43.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7ada3b640 1 -- 172.21.15.103:0/3099431388 learned_addr learned my addr 172.21.15.103:0/3099431388 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:43.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7aea3d640 1 --2- 172.21.15.103:0/3099431388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c2945d0 0x560d4c297770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7aea3d640 1 --2- 172.21.15.103:0/3099431388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c29f370 0x560d4c29df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7ada3b640 1 -- 172.21.15.103:0/3099431388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c29f370 msgr2=0x560d4c29df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:43.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7ada3b640 1 --2- 172.21.15.103:0/3099431388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c29f370 0x560d4c29df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7ada3b640 1 -- 172.21.15.103:0/3099431388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c2945d0 msgr2=0x560d4c297770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:43.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7ada3b640 1 --2- 172.21.15.103:0/3099431388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c2945d0 0x560d4c297770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7ada3b640 1 -- 172.21.15.103:0/3099431388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d4c29a550 con 0x560d4c295b40 2024-09-19T06:54:43.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7aea3d640 1 --2- 172.21.15.103:0/3099431388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c29f370 0x560d4c29df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:43.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa7ada3b640 1 --2- 172.21.15.103:0/3099431388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c2ab2b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa798004810 tx=0x7fa798014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d01d024d009791a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:43.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa793fff640 1 -- 172.21.15.103:0/3099431388 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798018040 con 0x560d4c295b40 2024-09-19T06:54:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa793fff640 1 -- 172.21.15.103:0/3099431388 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79800fb70 con 0x560d4c295b40 2024-09-19T06:54:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.552+0000 7fa793fff640 1 -- 172.21.15.103:0/3099431388 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7980149f0 con 0x560d4c295b40 2024-09-19T06:54:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.553+0000 7fa7b2879740 1 -- 172.21.15.103:0/3099431388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 msgr2=0x560d4c2ab2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.553+0000 7fa7b2879740 1 --2- 172.21.15.103:0/3099431388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c2ab2b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa798004810 tx=0x7fa798014000 comp rx=0 tx=0).stop 2024-09-19T06:54:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.553+0000 7fa7b2879740 1 -- 172.21.15.103:0/3099431388 shutdown_connections 2024-09-19T06:54:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.553+0000 7fa7b2879740 1 --2- 172.21.15.103:0/3099431388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c29f370 0x560d4c29df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.553+0000 7fa7b2879740 1 --2- 172.21.15.103:0/3099431388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c2ab2b0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.553+0000 7fa7b2879740 1 --2- 172.21.15.103:0/3099431388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c2945d0 0x560d4c297770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.553+0000 7fa7b2879740 1 -- 172.21.15.103:0/3099431388 >> 172.21.15.103:0/3099431388 conn(0x560d4c25d310 msgr2=0x560d4c29d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:43.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.553+0000 7fa7b2879740 1 -- 172.21.15.103:0/3099431388 shutdown_connections 2024-09-19T06:54:43.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.553+0000 7fa7b2879740 1 -- 172.21.15.103:0/3099431388 wait complete. 2024-09-19T06:54:43.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.553+0000 7fa7b2879740 1 -- start start 2024-09-19T06:54:43.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.553+0000 7fa7b2879740 1 Processor -- start 2024-09-19T06:54:43.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7b2879740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c2945d0 0x560d4c2a7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7b2879740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c2a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7b2879740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c29f370 0x560d4c2a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7b2879740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81b1090 con 0x560d4c295b40 2024-09-19T06:54:43.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7b2879740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81b09c0 con 0x560d4c2945d0 2024-09-19T06:54:43.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7b2879740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81adbe0 con 0x560d4c29f370 2024-09-19T06:54:43.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7b2879740 10 client.0 _ll_register_callbacks cb 0x7fa79800b2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:43.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7ada3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c2a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7ada3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c2a2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48970/0 (socket says 172.21.15.103:48970) 2024-09-19T06:54:43.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7ada3b640 1 -- 172.21.15.103:0/2661764107 learned_addr learned my addr 172.21.15.103:0/2661764107 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:43.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7aea3d640 1 --2- 172.21.15.103:0/2661764107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c29f370 0x560d4c2a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7ada3b640 1 -- 172.21.15.103:0/2661764107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c29f370 msgr2=0x560d4c2a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:43.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7ada3b640 1 --2- 172.21.15.103:0/2661764107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c29f370 0x560d4c2a35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7aea3d640 1 --2- 172.21.15.103:0/2661764107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c2945d0 0x560d4c2a7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7ada3b640 1 -- 172.21.15.103:0/2661764107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c2945d0 msgr2=0x560d4c2a7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:43.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7ada3b640 1 --2- 172.21.15.103:0/2661764107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c2945d0 0x560d4c2a7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7ada3b640 1 -- 172.21.15.103:0/2661764107 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d4c2a3e20 con 0x560d4c295b40 2024-09-19T06:54:43.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7ada3b640 1 --2- 172.21.15.103:0/2661764107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c2a2fe0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x560d4c2ab870 tx=0x7fa79801c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e57f7904948bbf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:43.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa791ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:43.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa791ffb640 1 -- 172.21.15.103:0/2661764107 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798018050 con 0x560d4c295b40 2024-09-19T06:54:43.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa791ffb640 1 -- 172.21.15.103:0/2661764107 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fa79801cc30 con 0x560d4c295b40 2024-09-19T06:54:43.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.554+0000 7fa7b2879740 1 -- 172.21.15.103:0/2661764107 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa79800fb70 con 0x560d4c295b40 2024-09-19T06:54:43.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.555+0000 7fa791ffb640 1 -- 172.21.15.103:0/2661764107 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa79801ce30 con 0x560d4c295b40 2024-09-19T06:54:43.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.555+0000 7fa7b2879740 10 client.17490 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:43.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.555+0000 7fa7b2879740 1 -- 172.21.15.103:0/2661764107 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560d4c319db0 con 0x560d4c295b40 2024-09-19T06:54:43.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.555+0000 7fa791ffb640 1 -- 172.21.15.103:0/2661764107 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79800fb70 con 0x560d4c295b40 2024-09-19T06:54:43.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.555+0000 7fa791ffb640 1 -- 172.21.15.103:0/2661764107 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa798006020 con 0x560d4c295b40 2024-09-19T06:54:43.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.555+0000 7fa7b2879740 10 client.17490 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:43.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.555+0000 7fa7b2879740 10 client.17490 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:54:43.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.555+0000 7fa7b2879740 20 client.17490 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:43.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.555+0000 7fa7b2879740 1 --2- 172.21.15.103:0/2661764107 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x560d4c31a4c0 0x560d4c31c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.555+0000 7fa7b2879740 4 client.17490 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:54:43.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.555+0000 7fa7b2879740 1 -- 172.21.15.103:0/2661764107 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d4c201d70 con 0x560d4c31a4c0 2024-09-19T06:54:43.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.555+0000 7fa7aea3d640 1 --2- 172.21.15.103:0/2661764107 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x560d4c31a4c0 0x560d4c31c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa791ffb640 10 client.17490 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:54:43.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa7aea3d640 1 --2- 172.21.15.103:0/2661764107 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x560d4c31a4c0 0x560d4c31c8e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4b3d79a2352d6a6 server_cookie=4784e339acb16b14 in_seq=0 out_seq=0 2024-09-19T06:54:43.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa791ffb640 1 -- 172.21.15.103:0/2661764107 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d4c201d70 con 0x560d4c31a4c0 2024-09-19T06:54:43.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa791ffb640 10 client.17490 handle_command_reply: tid=0 2024-09-19T06:54:43.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa7b2879740 1 client.17490 shutdown 2024-09-19T06:54:43.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa7b2879740 1 -- 172.21.15.103:0/2661764107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 msgr2=0x560d4c2a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa7b2879740 1 --2- 172.21.15.103:0/2661764107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c2a2fe0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x560d4c2ab870 tx=0x7fa79801c6f0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa7b2879740 1 -- 172.21.15.103:0/2661764107 shutdown_connections 2024-09-19T06:54:43.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa7b2879740 1 --2- 172.21.15.103:0/2661764107 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x560d4c31a4c0 0x560d4c31c8e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa7b2879740 1 --2- 172.21.15.103:0/2661764107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d4c29f370 0x560d4c2a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa7b2879740 1 --2- 172.21.15.103:0/2661764107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d4c295b40 0x560d4c2a2fe0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa791ffb640 0 client.17490 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:54:43.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa7b2879740 1 --2- 172.21.15.103:0/2661764107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d4c2945d0 0x560d4c2a7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa7b2879740 1 -- 172.21.15.103:0/2661764107 >> 172.21.15.103:0/2661764107 conn(0x560d4c25d310 msgr2=0x560d4c29e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:43.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa7b2879740 1 -- 172.21.15.103:0/2661764107 shutdown_connections 2024-09-19T06:54:43.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa7b2879740 1 -- 172.21.15.103:0/2661764107 wait complete. 2024-09-19T06:54:43.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.556+0000 7fa7b2879740 20 client.17490 trim_cache size 0 max 16384 2024-09-19T06:54:43.568 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:54:43.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.557+0000 7fa7b04a0640 1 -- 172.21.15.103:0/561161581 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7940a33b0 msgr2=0x7fa7940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.557+0000 7fa7b04a0640 1 --2- 172.21.15.103:0/561161581 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7940a33b0 0x7fa7940a5870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x560d4c291ac0 tx=0x7fa7a4086040 comp rx=0 tx=0).stop 2024-09-19T06:54:43.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.557+0000 7fa7b04a0640 1 -- 172.21.15.103:0/561161581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7a8074510 msgr2=0x7fa7a819a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.557+0000 7fa7b04a0640 1 --2- 172.21.15.103:0/561161581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7a8074510 0x7fa7a819a740 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400d950 tx=0x7fa7a400de10 comp rx=0 tx=0).stop 2024-09-19T06:54:43.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.557+0000 7fa7b04a0640 1 -- 172.21.15.103:0/561161581 shutdown_connections 2024-09-19T06:54:43.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.557+0000 7fa7b04a0640 1 --2- 172.21.15.103:0/561161581 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7940a33b0 0x7fa7940a5870 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.557+0000 7fa7b04a0640 1 --2- 172.21.15.103:0/561161581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7a810e430 0x7fa7a819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.557+0000 7fa7b04a0640 1 --2- 172.21.15.103:0/561161581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7a8074e30 0x7fa7a819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.557+0000 7fa7b04a0640 1 --2- 172.21.15.103:0/561161581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7a8074510 0x7fa7a819a740 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.557+0000 7fa7b04a0640 1 -- 172.21.15.103:0/561161581 >> 172.21.15.103:0/561161581 conn(0x7fa7a80fd8b0 msgr2=0x7fa7a80fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:43.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.557+0000 7fa7b04a0640 1 -- 172.21.15.103:0/561161581 shutdown_connections 2024-09-19T06:54:43.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:43.557+0000 7fa7b04a0640 1 -- 172.21.15.103:0/561161581 wait complete. 2024-09-19T06:54:44.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:44 smithi103 ceph-mon[31158]: pgmap v168: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-09-19T06:54:44.399 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:54:44.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:44 smithi134 ceph-mon[35356]: pgmap v168: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-09-19T06:54:44.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:44 smithi174 ceph-mon[35637]: pgmap v168: 97 pgs: 97 active+clean; 4.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-09-19T06:54:44.648 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:45.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.080+0000 7f28d4589640 1 -- 172.21.15.103:0/461177859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28cc100ca0 msgr2=0x7f28cc106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.080+0000 7f28d4589640 1 --2- 172.21.15.103:0/461177859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28cc100ca0 0x7f28cc106840 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009930 tx=0x7f28bc02a420 comp rx=0 tx=0).stop 2024-09-19T06:54:45.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.080+0000 7f28d4589640 1 -- 172.21.15.103:0/461177859 shutdown_connections 2024-09-19T06:54:45.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.080+0000 7f28d4589640 1 --2- 172.21.15.103:0/461177859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28cc106f50 0x7f28cc10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.080+0000 7f28d4589640 1 --2- 172.21.15.103:0/461177859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28cc100ca0 0x7f28cc106840 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.080+0000 7f28d4589640 1 --2- 172.21.15.103:0/461177859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28cc1002f0 0x7f28cc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.080+0000 7f28d4589640 1 -- 172.21.15.103:0/461177859 >> 172.21.15.103:0/461177859 conn(0x7f28cc077210 msgr2=0x7f28cc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:45.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.080+0000 7f28d4589640 1 -- 172.21.15.103:0/461177859 shutdown_connections 2024-09-19T06:54:45.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.080+0000 7f28d4589640 1 -- 172.21.15.103:0/461177859 wait complete. 2024-09-19T06:54:45.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.081+0000 7f28d4589640 1 Processor -- start 2024-09-19T06:54:45.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.081+0000 7f28d4589640 1 -- start start 2024-09-19T06:54:45.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.081+0000 7f28d4589640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28cc1002f0 0x7f28cc19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.081+0000 7f28d4589640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28cc100ca0 0x7f28cc19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.081+0000 7f28d4589640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28cc106f50 0x7f28cc19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.081+0000 7f28d4589640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc111fe0 con 0x7f28cc1002f0 2024-09-19T06:54:45.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.081+0000 7f28d4589640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc111e60 con 0x7f28cc100ca0 2024-09-19T06:54:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.081+0000 7f28d4589640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc112160 con 0x7f28cc106f50 2024-09-19T06:54:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.081+0000 7f28d2325640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28cc1002f0 0x7f28cc19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.081+0000 7f28d2325640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28cc1002f0 0x7f28cc19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48996/0 (socket says 172.21.15.103:48996) 2024-09-19T06:54:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.081+0000 7f28d2325640 1 -- 172.21.15.103:0/488981942 learned_addr learned my addr 172.21.15.103:0/488981942 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.082+0000 7f28d1b24640 1 --2- 172.21.15.103:0/488981942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28cc100ca0 0x7f28cc19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.082+0000 7f28d2b26640 1 --2- 172.21.15.103:0/488981942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28cc106f50 0x7f28cc19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.082+0000 7f28d2325640 1 -- 172.21.15.103:0/488981942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28cc106f50 msgr2=0x7f28cc19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.082+0000 7f28d2325640 1 --2- 172.21.15.103:0/488981942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28cc106f50 0x7f28cc19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.082+0000 7f28d2325640 1 -- 172.21.15.103:0/488981942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28cc100ca0 msgr2=0x7f28cc19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.082+0000 7f28d2325640 1 --2- 172.21.15.103:0/488981942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28cc100ca0 0x7f28cc19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.082+0000 7f28d2325640 1 -- 172.21.15.103:0/488981942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc19f6b0 con 0x7f28cc1002f0 2024-09-19T06:54:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.082+0000 7f28d1b24640 1 --2- 172.21.15.103:0/488981942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28cc100ca0 0x7f28cc19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.082+0000 7f28d2b26640 1 --2- 172.21.15.103:0/488981942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28cc106f50 0x7f28cc19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.082+0000 7f28d2325640 1 --2- 172.21.15.103:0/488981942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28cc1002f0 0x7f28cc19a700 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f28c800d950 tx=0x7f28c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.082+0000 7f28c37fe640 1 -- 172.21.15.103:0/488981942 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8017070 con 0x7f28cc1002f0 2024-09-19T06:54:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.082+0000 7f28d4589640 1 -- 172.21.15.103:0/488981942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28cc19f9a0 con 0x7f28cc1002f0 2024-09-19T06:54:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.082+0000 7f28c37fe640 1 -- 172.21.15.103:0/488981942 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c8009c30 con 0x7f28cc1002f0 2024-09-19T06:54:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.082+0000 7f28c37fe640 1 -- 172.21.15.103:0/488981942 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c800f6b0 con 0x7f28cc1002f0 2024-09-19T06:54:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.082+0000 7f28d4589640 1 -- 172.21.15.103:0/488981942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28cc1a7280 con 0x7f28cc1002f0 2024-09-19T06:54:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.084+0000 7f28c37fe640 1 -- 172.21.15.103:0/488981942 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f28c80109d0 con 0x7f28cc1002f0 2024-09-19T06:54:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.085+0000 7f28d6962740 1 Processor -- start 2024-09-19T06:54:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.085+0000 7f28d6962740 1 -- start start 2024-09-19T06:54:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.085+0000 7f28d6962740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 0x55e1150e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.085+0000 7f28d6962740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150e25d0 0x55e1150e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d6962740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150ed370 0x55e1150ef760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d6962740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e1150482f0 con 0x55e1150ed370 2024-09-19T06:54:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d6962740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e114fefcf0 con 0x55e1150e3b40 2024-09-19T06:54:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d6962740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e1150442b0 con 0x55e1150e25d0 2024-09-19T06:54:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28c37fe640 1 --2- 172.21.15.103:0/488981942 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28b80a33b0 0x7f28b80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d1b24640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150ed370 0x55e1150ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d2325640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150e25d0 0x55e1150e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d1b24640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150ed370 0x55e1150ef760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49008/0 (socket says 172.21.15.103:49008) 2024-09-19T06:54:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d2b26640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 0x55e1150e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d1b24640 1 -- 172.21.15.103:0/445091425 learned_addr learned my addr 172.21.15.103:0/445091425 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d2b26640 1 --2- 172.21.15.103:0/488981942 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28b80a33b0 0x7f28b80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28c37fe640 1 -- 172.21.15.103:0/488981942 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f28c80cfa50 con 0x7f28cc1002f0 2024-09-19T06:54:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d1b24640 1 -- 172.21.15.103:0/445091425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150e25d0 msgr2=0x55e1150e2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d1b24640 1 --2- 172.21.15.103:0/445091425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150e25d0 0x55e1150e2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d1b24640 1 -- 172.21.15.103:0/445091425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 msgr2=0x55e1150e2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d1b24640 1 --2- 172.21.15.103:0/445091425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 0x55e1150e2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d1b24640 1 -- 172.21.15.103:0/445091425 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e114fd43a0 con 0x55e1150ed370 2024-09-19T06:54:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d1b24640 1 --2- 172.21.15.103:0/445091425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150ed370 0x55e1150ef760 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f28cc06a100 tx=0x7f28bc032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2827d1b87bd4b552 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28d2b26640 1 --2- 172.21.15.103:0/488981942 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28b80a33b0 0x7f28b80a5870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f28c4014b80 tx=0x7f28c400a8f0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28c0ff9640 1 -- 172.21.15.103:0/445091425 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc03f070 con 0x55e1150ed370 2024-09-19T06:54:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.086+0000 7f28c0ff9640 1 -- 172.21.15.103:0/445091425 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28bc0317b0 con 0x55e1150ed370 2024-09-19T06:54:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.087+0000 7f28c0ff9640 1 -- 172.21.15.103:0/445091425 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc035070 con 0x55e1150ed370 2024-09-19T06:54:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.087+0000 7f28d6962740 1 -- 172.21.15.103:0/445091425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150ed370 msgr2=0x55e1150ef760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.087+0000 7f28d6962740 1 --2- 172.21.15.103:0/445091425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150ed370 0x55e1150ef760 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f28cc06a100 tx=0x7f28bc032d60 comp rx=0 tx=0).stop 2024-09-19T06:54:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.087+0000 7f28d6962740 1 -- 172.21.15.103:0/445091425 shutdown_connections 2024-09-19T06:54:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.087+0000 7f28d6962740 1 --2- 172.21.15.103:0/445091425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150ed370 0x55e1150ef760 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.087+0000 7f28d6962740 1 --2- 172.21.15.103:0/445091425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150e25d0 0x55e1150e2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.087+0000 7f28d6962740 1 --2- 172.21.15.103:0/445091425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 0x55e1150e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.087+0000 7f28d6962740 1 -- 172.21.15.103:0/445091425 >> 172.21.15.103:0/445091425 conn(0x55e1150ab310 msgr2=0x55e1150dec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:45.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.087+0000 7f28d6962740 1 -- 172.21.15.103:0/445091425 shutdown_connections 2024-09-19T06:54:45.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.087+0000 7f28d6962740 1 -- 172.21.15.103:0/445091425 wait complete. 2024-09-19T06:54:45.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d6962740 1 -- start start 2024-09-19T06:54:45.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d6962740 1 Processor -- start 2024-09-19T06:54:45.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d6962740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e25d0 0x55e1150ecd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d6962740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150e3b40 0x55e1150e9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d6962740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150ed370 0x55e1150ea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d6962740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e1150f0010 con 0x55e1150e3b40 2024-09-19T06:54:45.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d6962740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e115003060 con 0x55e1150e25d0 2024-09-19T06:54:45.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d6962740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e1150f0190 con 0x55e1150ed370 2024-09-19T06:54:45.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d6962740 10 client.0 _ll_register_callbacks cb 0x55e114d568c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d1b24640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150e3b40 0x55e1150e9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d1b24640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150e3b40 0x55e1150e9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49016/0 (socket says 172.21.15.103:49016) 2024-09-19T06:54:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d1b24640 1 -- 172.21.15.103:0/2848756254 learned_addr learned my addr 172.21.15.103:0/2848756254 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d2325640 1 --2- 172.21.15.103:0/2848756254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e25d0 0x55e1150ecd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d1b24640 1 -- 172.21.15.103:0/2848756254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150ed370 msgr2=0x55e1150ea5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d1b24640 1 --2- 172.21.15.103:0/2848756254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150ed370 0x55e1150ea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d1b24640 1 -- 172.21.15.103:0/2848756254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e25d0 msgr2=0x55e1150ecd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d1b24640 1 --2- 172.21.15.103:0/2848756254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e25d0 0x55e1150ecd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.088+0000 7f28d1b24640 1 -- 172.21.15.103:0/2848756254 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e114db93c0 con 0x55e1150e3b40 2024-09-19T06:54:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.089+0000 7f28d1b24640 1 --2- 172.21.15.103:0/2848756254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150e3b40 0x55e1150e9fe0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f28bc031550 tx=0x7f28bc0092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d7ac48c3833a43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.089+0000 7f28b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:45.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.089+0000 7f28b5ffb640 1 -- 172.21.15.103:0/2848756254 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc03f070 con 0x55e1150e3b40 2024-09-19T06:54:45.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.089+0000 7f28b5ffb640 1 -- 172.21.15.103:0/2848756254 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f28bc0035f0 con 0x55e1150e3b40 2024-09-19T06:54:45.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.089+0000 7f28d6962740 1 -- 172.21.15.103:0/2848756254 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28bc0317b0 con 0x55e1150e3b40 2024-09-19T06:54:45.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.089+0000 7f28b5ffb640 1 -- 172.21.15.103:0/2848756254 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f28bc054050 con 0x55e1150e3b40 2024-09-19T06:54:45.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.089+0000 7f28d6962740 10 client.17502 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:45.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.089+0000 7f28d6962740 1 -- 172.21.15.103:0/2848756254 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e1150fba70 con 0x55e1150e3b40 2024-09-19T06:54:45.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.089+0000 7f28b5ffb640 1 -- 172.21.15.103:0/2848756254 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28bc0317b0 con 0x55e1150e3b40 2024-09-19T06:54:45.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.089+0000 7f28b5ffb640 1 -- 172.21.15.103:0/2848756254 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f28bc004b00 con 0x55e1150e3b40 2024-09-19T06:54:45.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.089+0000 7f28d6962740 10 client.17502 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:45.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.089+0000 7f28d6962740 10 client.17502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:45.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.089+0000 7f28d6962740 20 client.17502 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:45.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.089+0000 7f28d6962740 1 --2- 172.21.15.103:0/2848756254 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e1150fcdc0 0x55e1150ff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.089+0000 7f28d6962740 4 client.17502 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.089+0000 7f28d6962740 1 -- 172.21.15.103:0/2848756254 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e115055860 con 0x55e1150fcdc0 2024-09-19T06:54:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.090+0000 7f28d2325640 1 --2- 172.21.15.103:0/2848756254 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e1150fcdc0 0x55e1150ff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.090+0000 7f28b5ffb640 10 client.17502 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.090+0000 7f28d2325640 1 --2- 172.21.15.103:0/2848756254 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e1150fcdc0 0x55e1150ff1b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=facfdd175e3cc2b3 server_cookie=62295e978b619f1 in_seq=0 out_seq=0 2024-09-19T06:54:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.091+0000 7f28b5ffb640 1 -- 172.21.15.103:0/2848756254 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e115055860 con 0x55e1150fcdc0 2024-09-19T06:54:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.091+0000 7f28b5ffb640 10 client.17502 handle_command_reply: tid=0 2024-09-19T06:54:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.091+0000 7f28d6962740 1 client.17502 shutdown 2024-09-19T06:54:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.091+0000 7f28d6962740 1 -- 172.21.15.103:0/2848756254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150e3b40 msgr2=0x55e1150e9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.091+0000 7f28d6962740 1 --2- 172.21.15.103:0/2848756254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150e3b40 0x55e1150e9fe0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f28bc031550 tx=0x7f28bc0092b0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.091+0000 7f28d6962740 1 -- 172.21.15.103:0/2848756254 shutdown_connections 2024-09-19T06:54:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.091+0000 7f28d6962740 1 --2- 172.21.15.103:0/2848756254 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e1150fcdc0 0x55e1150ff1b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.091+0000 7f28d6962740 1 --2- 172.21.15.103:0/2848756254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150ed370 0x55e1150ea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.091+0000 7f28d6962740 1 --2- 172.21.15.103:0/2848756254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150e3b40 0x55e1150e9fe0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.091+0000 7f28d6962740 1 --2- 172.21.15.103:0/2848756254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e25d0 0x55e1150ecd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.091+0000 7f28d6962740 1 -- 172.21.15.103:0/2848756254 >> 172.21.15.103:0/2848756254 conn(0x55e1150ab310 msgr2=0x55e1150e4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:45.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.091+0000 7f28b5ffb640 0 client.17502 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:45.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.091+0000 7f28d6962740 1 -- 172.21.15.103:0/2848756254 shutdown_connections 2024-09-19T06:54:45.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.091+0000 7f28d6962740 1 -- 172.21.15.103:0/2848756254 wait complete. 2024-09-19T06:54:45.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.091+0000 7f28d6962740 20 client.17502 trim_cache size 0 max 16384 2024-09-19T06:54:45.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d6962740 1 Processor -- start 2024-09-19T06:54:45.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d6962740 1 -- start start 2024-09-19T06:54:45.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d6962740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150e25d0 0x55e1150f92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d6962740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 0x55e1150ebf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d6962740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150ed370 0x55e1150ee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d6962740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc1afb10 con 0x55e1150ed370 2024-09-19T06:54:45.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d6962740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc1b1650 con 0x55e1150e3b40 2024-09-19T06:54:45.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d6962740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc1b01e0 con 0x55e1150e25d0 2024-09-19T06:54:45.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d2b26640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150ed370 0x55e1150ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d2b26640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150ed370 0x55e1150ee760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49032/0 (socket says 172.21.15.103:49032) 2024-09-19T06:54:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d2b26640 1 -- 172.21.15.103:0/3441842249 learned_addr learned my addr 172.21.15.103:0/3441842249 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d2325640 1 --2- 172.21.15.103:0/3441842249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150e25d0 0x55e1150f92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d1b24640 1 --2- 172.21.15.103:0/3441842249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 0x55e1150ebf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d2b26640 1 -- 172.21.15.103:0/3441842249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150e25d0 msgr2=0x55e1150f92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d2b26640 1 --2- 172.21.15.103:0/3441842249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150e25d0 0x55e1150f92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d2b26640 1 -- 172.21.15.103:0/3441842249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 msgr2=0x55e1150ebf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d2b26640 1 --2- 172.21.15.103:0/3441842249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 0x55e1150ebf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d2b26640 1 -- 172.21.15.103:0/3441842249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1150deed0 con 0x55e1150ed370 2024-09-19T06:54:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.104+0000 7f28d1b24640 1 --2- 172.21.15.103:0/3441842249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 0x55e1150ebf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.105+0000 7f28d2b26640 1 --2- 172.21.15.103:0/3441842249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150ed370 0x55e1150ee760 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f28c400a3c0 tx=0x7f28c4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30b6e62facd460d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.105+0000 7f28b7fff640 1 -- 172.21.15.103:0/3441842249 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4023030 con 0x55e1150ed370 2024-09-19T06:54:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.105+0000 7f28b7fff640 1 -- 172.21.15.103:0/3441842249 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c4013990 con 0x55e1150ed370 2024-09-19T06:54:45.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.105+0000 7f28b7fff640 1 -- 172.21.15.103:0/3441842249 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c401f910 con 0x55e1150ed370 2024-09-19T06:54:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.106+0000 7f28d6962740 1 -- 172.21.15.103:0/3441842249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150ed370 msgr2=0x55e1150ee760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.106+0000 7f28d6962740 1 --2- 172.21.15.103:0/3441842249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150ed370 0x55e1150ee760 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f28c400a3c0 tx=0x7f28c4019000 comp rx=0 tx=0).stop 2024-09-19T06:54:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.106+0000 7f28d6962740 1 -- 172.21.15.103:0/3441842249 shutdown_connections 2024-09-19T06:54:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.106+0000 7f28d6962740 1 --2- 172.21.15.103:0/3441842249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150ed370 0x55e1150ee760 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.106+0000 7f28d6962740 1 --2- 172.21.15.103:0/3441842249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 0x55e1150ebf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.106+0000 7f28d6962740 1 --2- 172.21.15.103:0/3441842249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150e25d0 0x55e1150f92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.106+0000 7f28d6962740 1 -- 172.21.15.103:0/3441842249 >> 172.21.15.103:0/3441842249 conn(0x55e1150ab310 msgr2=0x55e1150eb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.106+0000 7f28d6962740 1 -- 172.21.15.103:0/3441842249 shutdown_connections 2024-09-19T06:54:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.106+0000 7f28d6962740 1 -- 172.21.15.103:0/3441842249 wait complete. 2024-09-19T06:54:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.106+0000 7f28d6962740 1 -- start start 2024-09-19T06:54:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.106+0000 7f28d6962740 1 Processor -- start 2024-09-19T06:54:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d6962740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150e25d0 0x55e1150f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d6962740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 0x55e1150ee9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d6962740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150ed370 0x55e1150ef000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d6962740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc1b0f80 con 0x55e1150e25d0 2024-09-19T06:54:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d6962740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc1b08b0 con 0x55e1150e3b40 2024-09-19T06:54:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d2325640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150e25d0 0x55e1150f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d6962740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc1adad0 con 0x55e1150ed370 2024-09-19T06:54:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d6962740 10 client.0 _ll_register_callbacks cb 0x7f28c400ad80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d2325640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150e25d0 0x55e1150f5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49042/0 (socket says 172.21.15.103:49042) 2024-09-19T06:54:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d2325640 1 -- 172.21.15.103:0/2648506227 learned_addr learned my addr 172.21.15.103:0/2648506227 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d2b26640 1 --2- 172.21.15.103:0/2648506227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150ed370 0x55e1150ef000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d1b24640 1 --2- 172.21.15.103:0/2648506227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 0x55e1150ee9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d2325640 1 -- 172.21.15.103:0/2648506227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150ed370 msgr2=0x55e1150ef000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d2325640 1 --2- 172.21.15.103:0/2648506227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150ed370 0x55e1150ef000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d2325640 1 -- 172.21.15.103:0/2648506227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 msgr2=0x55e1150ee9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:45.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d2325640 1 --2- 172.21.15.103:0/2648506227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 0x55e1150ee9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d2325640 1 -- 172.21.15.103:0/2648506227 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1150e3790 con 0x55e1150e25d0 2024-09-19T06:54:45.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.107+0000 7f28d1b24640 1 --2- 172.21.15.103:0/2648506227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 0x55e1150ee9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:45.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.108+0000 7f28d2325640 1 --2- 172.21.15.103:0/2648506227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150e25d0 0x55e1150f5c50 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f28c809bc90 tx=0x7f28c806b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2c7854506fbe28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:45.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.108+0000 7f28b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:45.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.108+0000 7f28b5ffb640 1 -- 172.21.15.103:0/2648506227 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8020040 con 0x55e1150e25d0 2024-09-19T06:54:45.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.108+0000 7f28b5ffb640 1 -- 172.21.15.103:0/2648506227 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f28c80cd3f0 con 0x55e1150e25d0 2024-09-19T06:54:45.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.108+0000 7f28d6962740 1 -- 172.21.15.103:0/2648506227 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28c4013990 con 0x55e1150e25d0 2024-09-19T06:54:45.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.108+0000 7f28b5ffb640 1 -- 172.21.15.103:0/2648506227 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f28c80cd5f0 con 0x55e1150e25d0 2024-09-19T06:54:45.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.108+0000 7f28d6962740 10 client.17508 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:45.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.108+0000 7f28d6962740 1 -- 172.21.15.103:0/2648506227 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e115167720 con 0x55e1150e25d0 2024-09-19T06:54:45.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.108+0000 7f28b5ffb640 1 -- 172.21.15.103:0/2648506227 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c4013990 con 0x55e1150e25d0 2024-09-19T06:54:45.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.108+0000 7f28b5ffb640 1 -- 172.21.15.103:0/2648506227 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f28c809b150 con 0x55e1150e25d0 2024-09-19T06:54:45.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.108+0000 7f28d6962740 10 client.17508 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.108+0000 7f28d6962740 10 client.17508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.108+0000 7f28d6962740 20 client.17508 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.108+0000 7f28d6962740 1 --2- 172.21.15.103:0/2648506227 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e115167c80 0x55e11516a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.108+0000 7f28d6962740 4 client.17508 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:54:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.108+0000 7f28d6962740 1 -- 172.21.15.103:0/2648506227 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e11504fd70 con 0x55e115167c80 2024-09-19T06:54:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.109+0000 7f28d1b24640 1 --2- 172.21.15.103:0/2648506227 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e115167c80 0x55e11516a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.109+0000 7f28b5ffb640 10 client.17508 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.109+0000 7f28d1b24640 1 --2- 172.21.15.103:0/2648506227 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e115167c80 0x55e11516a070 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11487d60971cf2c6 server_cookie=5ef30981d01fad92 in_seq=0 out_seq=0 2024-09-19T06:54:45.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.109+0000 7f28b5ffb640 1 -- 172.21.15.103:0/2648506227 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e11504fd70 con 0x55e115167c80 2024-09-19T06:54:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.109+0000 7f28b5ffb640 10 client.17508 handle_command_reply: tid=0 2024-09-19T06:54:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.109+0000 7f28d6962740 1 client.17508 shutdown 2024-09-19T06:54:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.109+0000 7f28d6962740 1 -- 172.21.15.103:0/2648506227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150e25d0 msgr2=0x55e1150f5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.109+0000 7f28d6962740 1 --2- 172.21.15.103:0/2648506227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150e25d0 0x55e1150f5c50 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f28c809bc90 tx=0x7f28c806b920 comp rx=0 tx=0).stop 2024-09-19T06:54:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.109+0000 7f28d6962740 1 -- 172.21.15.103:0/2648506227 shutdown_connections 2024-09-19T06:54:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.109+0000 7f28d6962740 1 --2- 172.21.15.103:0/2648506227 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e115167c80 0x55e11516a070 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d6962740 1 --2- 172.21.15.103:0/2648506227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1150ed370 0x55e1150ef000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d6962740 1 --2- 172.21.15.103:0/2648506227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1150e3b40 0x55e1150ee9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d6962740 1 --2- 172.21.15.103:0/2648506227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1150e25d0 0x55e1150f5c50 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d6962740 1 -- 172.21.15.103:0/2648506227 >> 172.21.15.103:0/2648506227 conn(0x55e1150ab310 msgr2=0x55e1150eb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28b5ffb640 0 client.17508 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d6962740 1 -- 172.21.15.103:0/2648506227 shutdown_connections 2024-09-19T06:54:45.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d6962740 1 -- 172.21.15.103:0/2648506227 wait complete. 2024-09-19T06:54:45.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d6962740 20 client.17508 trim_cache size 0 max 16384 2024-09-19T06:54:45.125 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:54:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d4589640 1 -- 172.21.15.103:0/488981942 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28b80a33b0 msgr2=0x7f28b80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d4589640 1 --2- 172.21.15.103:0/488981942 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28b80a33b0 0x7f28b80a5870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f28c4014b80 tx=0x7f28c400a8f0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d4589640 1 -- 172.21.15.103:0/488981942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28cc1002f0 msgr2=0x7f28cc19a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d4589640 1 --2- 172.21.15.103:0/488981942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28cc1002f0 0x7f28cc19a700 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f28c800d950 tx=0x7f28c800de10 comp rx=0 tx=0).stop 2024-09-19T06:54:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d4589640 1 -- 172.21.15.103:0/488981942 shutdown_connections 2024-09-19T06:54:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d4589640 1 --2- 172.21.15.103:0/488981942 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28b80a33b0 0x7f28b80a5870 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d4589640 1 --2- 172.21.15.103:0/488981942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28cc106f50 0x7f28cc19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d4589640 1 --2- 172.21.15.103:0/488981942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28cc100ca0 0x7f28cc19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d4589640 1 --2- 172.21.15.103:0/488981942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28cc1002f0 0x7f28cc19a700 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d4589640 1 -- 172.21.15.103:0/488981942 >> 172.21.15.103:0/488981942 conn(0x7f28cc077210 msgr2=0x7f28cc1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.110+0000 7f28d4589640 1 -- 172.21.15.103:0/488981942 shutdown_connections 2024-09-19T06:54:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:45.111+0000 7f28d4589640 1 -- 172.21.15.103:0/488981942 wait complete. 2024-09-19T06:54:46.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:46 smithi103 ceph-mon[31158]: pgmap v169: 97 pgs: 97 active+clean; 4.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-19T06:54:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:46 smithi134 ceph-mon[35356]: pgmap v169: 97 pgs: 97 active+clean; 4.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-19T06:54:46.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:46 smithi174 ceph-mon[35637]: pgmap v169: 97 pgs: 97 active+clean; 4.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-19T06:54:46.742 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:54:46.742 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:54:47.003 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:47.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.434+0000 7f54c1f21640 1 -- 172.21.15.103:0/3783757750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc106f20 msgr2=0x7f54bc10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:47.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.434+0000 7f54c1f21640 1 --2- 172.21.15.103:0/3783757750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc106f20 0x7f54bc10eaa0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009900 tx=0x7f54b402a410 comp rx=0 tx=0).stop 2024-09-19T06:54:47.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.434+0000 7f54c1f21640 1 -- 172.21.15.103:0/3783757750 shutdown_connections 2024-09-19T06:54:47.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.434+0000 7f54c1f21640 1 --2- 172.21.15.103:0/3783757750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc106f20 0x7f54bc10eaa0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.434+0000 7f54c1f21640 1 --2- 172.21.15.103:0/3783757750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54bc1027a0 0x7f54bc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.434+0000 7f54c1f21640 1 --2- 172.21.15.103:0/3783757750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc101df0 0x7f54bc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.434+0000 7f54c1f21640 1 -- 172.21.15.103:0/3783757750 >> 172.21.15.103:0/3783757750 conn(0x7f54bc0fd8f0 msgr2=0x7f54bc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:47.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.434+0000 7f54c1f21640 1 -- 172.21.15.103:0/3783757750 shutdown_connections 2024-09-19T06:54:47.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.434+0000 7f54c1f21640 1 -- 172.21.15.103:0/3783757750 wait complete. 2024-09-19T06:54:47.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.434+0000 7f54c1f21640 1 Processor -- start 2024-09-19T06:54:47.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54c1f21640 1 -- start start 2024-09-19T06:54:47.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54c1f21640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc101df0 0x7f54bc198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54c1f21640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc1027a0 0x7f54bc198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54c1f21640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54bc106f20 0x7f54bc19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54c1f21640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc112020 con 0x7f54bc1027a0 2024-09-19T06:54:47.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54c1f21640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc111ea0 con 0x7f54bc106f20 2024-09-19T06:54:47.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54c1f21640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc1121a0 con 0x7f54bc101df0 2024-09-19T06:54:47.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54baffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc1027a0 0x7f54bc198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54baffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc1027a0 0x7f54bc198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34774/0 (socket says 172.21.15.103:34774) 2024-09-19T06:54:47.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54baffd640 1 -- 172.21.15.103:0/1101394274 learned_addr learned my addr 172.21.15.103:0/1101394274 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:47.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54bb7fe640 1 --2- 172.21.15.103:0/1101394274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc101df0 0x7f54bc198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54bbfff640 1 --2- 172.21.15.103:0/1101394274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54bc106f20 0x7f54bc19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54baffd640 1 -- 172.21.15.103:0/1101394274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc101df0 msgr2=0x7f54bc198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:47.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54baffd640 1 --2- 172.21.15.103:0/1101394274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc101df0 0x7f54bc198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54baffd640 1 -- 172.21.15.103:0/1101394274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54bc106f20 msgr2=0x7f54bc19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:47.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54baffd640 1 --2- 172.21.15.103:0/1101394274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54bc106f20 0x7f54bc19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.435+0000 7f54baffd640 1 -- 172.21.15.103:0/1101394274 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54bc19d4d0 con 0x7f54bc1027a0 2024-09-19T06:54:47.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.436+0000 7f54baffd640 1 --2- 172.21.15.103:0/1101394274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc1027a0 0x7f54bc198a60 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f54b000b4a0 tx=0x7f54b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:47.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.436+0000 7f54b8ff9640 1 -- 172.21.15.103:0/1101394274 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0014070 con 0x7f54bc1027a0 2024-09-19T06:54:47.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.436+0000 7f54c1f21640 1 -- 172.21.15.103:0/1101394274 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54bc19d7c0 con 0x7f54bc1027a0 2024-09-19T06:54:47.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.436+0000 7f54b8ff9640 1 -- 172.21.15.103:0/1101394274 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b0010af0 con 0x7f54bc1027a0 2024-09-19T06:54:47.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.436+0000 7f54b8ff9640 1 -- 172.21.15.103:0/1101394274 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0017d10 con 0x7f54bc1027a0 2024-09-19T06:54:47.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.436+0000 7f54c1f21640 1 -- 172.21.15.103:0/1101394274 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54bc1a50a0 con 0x7f54bc1027a0 2024-09-19T06:54:47.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.438+0000 7f54b8ff9640 1 -- 172.21.15.103:0/1101394274 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f54b0017070 con 0x7f54bc1027a0 2024-09-19T06:54:47.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54c42fa740 1 Processor -- start 2024-09-19T06:54:47.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54c42fa740 1 -- start start 2024-09-19T06:54:47.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54b8ff9640 1 --2- 172.21.15.103:0/1101394274 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f54a40a33b0 0x7f54a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54c42fa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae212b60 0x55c3ae212f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54c42fa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae21d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54c42fa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae21dd40 0x55c3ae220130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54c42fa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c3ae178280 con 0x55c3ae212b60 2024-09-19T06:54:47.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54c42fa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c3ae11fb10 con 0x55c3ae21dd40 2024-09-19T06:54:47.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54c42fa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c3ae174240 con 0x55c3ae213480 2024-09-19T06:54:47.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54b8ff9640 1 -- 172.21.15.103:0/1101394274 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f54b00cef20 con 0x7f54bc1027a0 2024-09-19T06:54:47.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54bbfff640 1 --2- 172.21.15.103:0/1101394274 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f54a40a33b0 0x7f54a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54bb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae212b60 0x55c3ae212f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54bb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae212b60 0x55c3ae212f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34782/0 (socket says 172.21.15.103:34782) 2024-09-19T06:54:47.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54bb7fe640 1 -- 172.21.15.103:0/489189542 learned_addr learned my addr 172.21.15.103:0/489189542 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:47.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54baffd640 1 --2- 172.21.15.103:0/489189542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae21d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54bbfff640 1 --2- 172.21.15.103:0/489189542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae21dd40 0x55c3ae220130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54bb7fe640 1 -- 172.21.15.103:0/489189542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 msgr2=0x55c3ae21d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:47.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54bb7fe640 1 --2- 172.21.15.103:0/489189542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae21d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54bb7fe640 1 -- 172.21.15.103:0/489189542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae21dd40 msgr2=0x55c3ae220130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:47.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54bb7fe640 1 --2- 172.21.15.103:0/489189542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae21dd40 0x55c3ae220130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54bb7fe640 1 -- 172.21.15.103:0/489189542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3ae12c1f0 con 0x55c3ae212b60 2024-09-19T06:54:47.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54baffd640 1 --2- 172.21.15.103:0/489189542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae21d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:47.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.439+0000 7f54bbfff640 1 --2- 172.21.15.103:0/489189542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae21dd40 0x55c3ae220130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:47.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.440+0000 7f54bb7fe640 1 --2- 172.21.15.103:0/489189542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae212b60 0x55c3ae212f40 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00a4b0 tx=0x7f54ac0096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6565a42ed266f131 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:47.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.440+0000 7f54bbfff640 1 --2- 172.21.15.103:0/1101394274 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f54a40a33b0 0x7f54a40a5870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55c3ae210470 tx=0x7f54b40095d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:47.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.440+0000 7f54a1ffb640 1 -- 172.21.15.103:0/489189542 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac018020 con 0x55c3ae212b60 2024-09-19T06:54:47.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.440+0000 7f54a1ffb640 1 -- 172.21.15.103:0/489189542 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ac014cf0 con 0x55c3ae212b60 2024-09-19T06:54:47.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.440+0000 7f54a1ffb640 1 -- 172.21.15.103:0/489189542 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac013de0 con 0x55c3ae212b60 2024-09-19T06:54:47.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.440+0000 7f54c42fa740 1 -- 172.21.15.103:0/489189542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae212b60 msgr2=0x55c3ae212f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:47.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.440+0000 7f54c42fa740 1 --2- 172.21.15.103:0/489189542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae212b60 0x55c3ae212f40 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00a4b0 tx=0x7f54ac0096d0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.441+0000 7f54c42fa740 1 -- 172.21.15.103:0/489189542 shutdown_connections 2024-09-19T06:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.441+0000 7f54c42fa740 1 --2- 172.21.15.103:0/489189542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae21dd40 0x55c3ae220130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.441+0000 7f54c42fa740 1 --2- 172.21.15.103:0/489189542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae21d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.441+0000 7f54c42fa740 1 --2- 172.21.15.103:0/489189542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae212b60 0x55c3ae212f40 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.441+0000 7f54c42fa740 1 -- 172.21.15.103:0/489189542 >> 172.21.15.103:0/489189542 conn(0x55c3ae20f0f0 msgr2=0x55c3ae20f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.441+0000 7f54c42fa740 1 -- 172.21.15.103:0/489189542 shutdown_connections 2024-09-19T06:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.441+0000 7f54c42fa740 1 -- 172.21.15.103:0/489189542 wait complete. 2024-09-19T06:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.441+0000 7f54c42fa740 1 -- start start 2024-09-19T06:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.441+0000 7f54c42fa740 1 Processor -- start 2024-09-19T06:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.441+0000 7f54c42fa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae212b60 0x55c3ae21d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.441+0000 7f54c42fa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae21ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.441+0000 7f54c42fa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae21dd40 0x55c3ae21b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.441+0000 7f54c42fa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c3ae220910 con 0x55c3ae212b60 2024-09-19T06:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.441+0000 7f54c42fa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c3ae132f10 con 0x55c3ae21dd40 2024-09-19T06:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.441+0000 7f54c42fa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c3ae220a90 con 0x55c3ae213480 2024-09-19T06:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f54c42fa740 10 client.0 _ll_register_callbacks cb 0x55c3ae0fba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f54bb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae212b60 0x55c3ae21d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f54bb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae212b60 0x55c3ae21d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34790/0 (socket says 172.21.15.103:34790) 2024-09-19T06:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f54bb7fe640 1 -- 172.21.15.103:0/2117719729 learned_addr learned my addr 172.21.15.103:0/2117719729 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f54baffd640 1 --2- 172.21.15.103:0/2117719729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae21ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f54bbfff640 1 --2- 172.21.15.103:0/2117719729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae21dd40 0x55c3ae21b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f54bb7fe640 1 -- 172.21.15.103:0/2117719729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 msgr2=0x55c3ae21ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f54bb7fe640 1 --2- 172.21.15.103:0/2117719729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae21ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f54bb7fe640 1 -- 172.21.15.103:0/2117719729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae21dd40 msgr2=0x55c3ae21b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f54bb7fe640 1 --2- 172.21.15.103:0/2117719729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae21dd40 0x55c3ae21b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f54bb7fe640 1 -- 172.21.15.103:0/2117719729 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3adee96a0 con 0x55c3ae212b60 2024-09-19T06:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f54bb7fe640 1 --2- 172.21.15.103:0/2117719729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae212b60 0x55c3ae21d7f0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f54ac013a00 tx=0x7f54ac0137f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5093aa435ad11b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f549affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f549affd640 1 -- 172.21.15.103:0/2117719729 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac02c060 con 0x55c3ae212b60 2024-09-19T06:54:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f549affd640 1 -- 172.21.15.103:0/2117719729 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f54ac014610 con 0x55c3ae212b60 2024-09-19T06:54:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f54c42fa740 1 -- 172.21.15.103:0/2117719729 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54ac014cf0 con 0x55c3ae212b60 2024-09-19T06:54:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f549affd640 1 -- 172.21.15.103:0/2117719729 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54ac014810 con 0x55c3ae212b60 2024-09-19T06:54:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.442+0000 7f54c42fa740 10 client.17520 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.443+0000 7f54c42fa740 1 -- 172.21.15.103:0/2117719729 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c3ae22c3b0 con 0x55c3ae212b60 2024-09-19T06:54:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.443+0000 7f549affd640 1 -- 172.21.15.103:0/2117719729 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ac014cf0 con 0x55c3ae212b60 2024-09-19T06:54:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.443+0000 7f549affd640 1 -- 172.21.15.103:0/2117719729 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f54ac027420 con 0x55c3ae212b60 2024-09-19T06:54:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.443+0000 7f54c42fa740 10 client.17520 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.443+0000 7f54c42fa740 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.443+0000 7f54c42fa740 20 client.17520 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.443+0000 7f54c42fa740 1 --2- 172.21.15.103:0/2117719729 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c3ae22d870 0x55c3ae22fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.443+0000 7f54c42fa740 4 client.17520 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.443+0000 7f54c42fa740 1 -- 172.21.15.103:0/2117719729 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3ae1857f0 con 0x55c3ae22d870 2024-09-19T06:54:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.443+0000 7f54baffd640 1 --2- 172.21.15.103:0/2117719729 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c3ae22d870 0x55c3ae22fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.444+0000 7f549affd640 10 client.17520 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.444+0000 7f54baffd640 1 --2- 172.21.15.103:0/2117719729 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c3ae22d870 0x55c3ae22fc60 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a0e7adc111091a server_cookie=8122f3fa36f70c9 in_seq=0 out_seq=0 2024-09-19T06:54:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.444+0000 7f549affd640 1 -- 172.21.15.103:0/2117719729 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c3ae1857f0 con 0x55c3ae22d870 2024-09-19T06:54:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.444+0000 7f549affd640 10 client.17520 handle_command_reply: tid=0 2024-09-19T06:54:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.445+0000 7f54c42fa740 1 client.17520 shutdown 2024-09-19T06:54:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.445+0000 7f54c42fa740 1 -- 172.21.15.103:0/2117719729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae212b60 msgr2=0x55c3ae21d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.445+0000 7f54c42fa740 1 --2- 172.21.15.103:0/2117719729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae212b60 0x55c3ae21d7f0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f54ac013a00 tx=0x7f54ac0137f0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.445+0000 7f54c42fa740 1 -- 172.21.15.103:0/2117719729 shutdown_connections 2024-09-19T06:54:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.445+0000 7f54c42fa740 1 --2- 172.21.15.103:0/2117719729 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c3ae22d870 0x55c3ae22fc60 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.445+0000 7f54c42fa740 1 --2- 172.21.15.103:0/2117719729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae21dd40 0x55c3ae21b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.445+0000 7f54c42fa740 1 --2- 172.21.15.103:0/2117719729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae21ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.445+0000 7f54c42fa740 1 --2- 172.21.15.103:0/2117719729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae212b60 0x55c3ae21d7f0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.445+0000 7f54c42fa740 1 -- 172.21.15.103:0/2117719729 >> 172.21.15.103:0/2117719729 conn(0x55c3ae20f0f0 msgr2=0x55c3ae20e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.445+0000 7f549affd640 0 client.17520 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.445+0000 7f54c42fa740 1 -- 172.21.15.103:0/2117719729 shutdown_connections 2024-09-19T06:54:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.445+0000 7f54c42fa740 1 -- 172.21.15.103:0/2117719729 wait complete. 2024-09-19T06:54:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.445+0000 7f54c42fa740 20 client.17520 trim_cache size 0 max 16384 2024-09-19T06:54:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54c42fa740 1 Processor -- start 2024-09-19T06:54:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54c42fa740 1 -- start start 2024-09-19T06:54:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54c42fa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae212b60 0x55c3ae2155c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54c42fa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae219b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54c42fa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae21dd40 0x55c3ae2353a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54c42fa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc1ada00 con 0x55c3ae21dd40 2024-09-19T06:54:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54c42fa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc1af540 con 0x55c3ae212b60 2024-09-19T06:54:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54c42fa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc1ae0d0 con 0x55c3ae213480 2024-09-19T06:54:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54bbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae21dd40 0x55c3ae2353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54bbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae21dd40 0x55c3ae2353a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34800/0 (socket says 172.21.15.103:34800) 2024-09-19T06:54:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54bbfff640 1 -- 172.21.15.103:0/3518894221 learned_addr learned my addr 172.21.15.103:0/3518894221 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54baffd640 1 --2- 172.21.15.103:0/3518894221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae219b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54bb7fe640 1 --2- 172.21.15.103:0/3518894221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae212b60 0x55c3ae2155c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54bbfff640 1 -- 172.21.15.103:0/3518894221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 msgr2=0x55c3ae219b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54bbfff640 1 --2- 172.21.15.103:0/3518894221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae219b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54bbfff640 1 -- 172.21.15.103:0/3518894221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae212b60 msgr2=0x55c3ae2155c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54bbfff640 1 --2- 172.21.15.103:0/3518894221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae212b60 0x55c3ae2155c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.462+0000 7f54bbfff640 1 -- 172.21.15.103:0/3518894221 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3ae20ef20 con 0x55c3ae21dd40 2024-09-19T06:54:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.463+0000 7f54bb7fe640 1 --2- 172.21.15.103:0/3518894221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae212b60 0x55c3ae2155c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.463+0000 7f54bbfff640 1 --2- 172.21.15.103:0/3518894221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae21dd40 0x55c3ae2353a0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009900 tx=0x7f54b403a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681d0c4175acc279 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.463+0000 7f54a17fa640 1 -- 172.21.15.103:0/3518894221 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4045070 con 0x55c3ae21dd40 2024-09-19T06:54:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.463+0000 7f54a17fa640 1 -- 172.21.15.103:0/3518894221 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b4051070 con 0x55c3ae21dd40 2024-09-19T06:54:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.463+0000 7f54a17fa640 1 -- 172.21.15.103:0/3518894221 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4035040 con 0x55c3ae21dd40 2024-09-19T06:54:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.463+0000 7f54c42fa740 1 -- 172.21.15.103:0/3518894221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae21dd40 msgr2=0x55c3ae2353a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.463+0000 7f54c42fa740 1 --2- 172.21.15.103:0/3518894221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae21dd40 0x55c3ae2353a0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009900 tx=0x7f54b403a000 comp rx=0 tx=0).stop 2024-09-19T06:54:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.463+0000 7f54c42fa740 1 -- 172.21.15.103:0/3518894221 shutdown_connections 2024-09-19T06:54:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.463+0000 7f54c42fa740 1 --2- 172.21.15.103:0/3518894221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae21dd40 0x55c3ae2353a0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.463+0000 7f54c42fa740 1 --2- 172.21.15.103:0/3518894221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae219b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.463+0000 7f54c42fa740 1 --2- 172.21.15.103:0/3518894221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae212b60 0x55c3ae2155c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.463+0000 7f54c42fa740 1 -- 172.21.15.103:0/3518894221 >> 172.21.15.103:0/3518894221 conn(0x55c3ae20f0f0 msgr2=0x55c3ae219410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.464+0000 7f54c42fa740 1 -- 172.21.15.103:0/3518894221 shutdown_connections 2024-09-19T06:54:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.464+0000 7f54c42fa740 1 -- 172.21.15.103:0/3518894221 wait complete. 2024-09-19T06:54:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.464+0000 7f54c42fa740 1 -- start start 2024-09-19T06:54:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.464+0000 7f54c42fa740 1 Processor -- start 2024-09-19T06:54:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.464+0000 7f54c42fa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae212b60 0x55c3ae225db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.464+0000 7f54c42fa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae2262f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.464+0000 7f54c42fa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae21dd40 0x55c3ae221240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.464+0000 7f54c42fa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc1aee70 con 0x55c3ae21dd40 2024-09-19T06:54:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.464+0000 7f54c42fa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc1ae7a0 con 0x55c3ae212b60 2024-09-19T06:54:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.464+0000 7f54c42fa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc1ab9c0 con 0x55c3ae213480 2024-09-19T06:54:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.464+0000 7f54c42fa740 10 client.0 _ll_register_callbacks cb 0x7f54ac009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.464+0000 7f54bbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae21dd40 0x55c3ae221240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.464+0000 7f54bbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae21dd40 0x55c3ae221240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34804/0 (socket says 172.21.15.103:34804) 2024-09-19T06:54:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.464+0000 7f54bbfff640 1 -- 172.21.15.103:0/2780702759 learned_addr learned my addr 172.21.15.103:0/2780702759 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.465+0000 7f54bb7fe640 1 --2- 172.21.15.103:0/2780702759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae212b60 0x55c3ae225db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.465+0000 7f54baffd640 1 --2- 172.21.15.103:0/2780702759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae2262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.465+0000 7f54bbfff640 1 -- 172.21.15.103:0/2780702759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 msgr2=0x55c3ae2262f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.465+0000 7f54bbfff640 1 --2- 172.21.15.103:0/2780702759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae2262f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.465+0000 7f54bbfff640 1 -- 172.21.15.103:0/2780702759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae212b60 msgr2=0x55c3ae225db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.465+0000 7f54bbfff640 1 --2- 172.21.15.103:0/2780702759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae212b60 0x55c3ae225db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.465+0000 7f54bbfff640 1 -- 172.21.15.103:0/2780702759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3ae2139f0 con 0x55c3ae21dd40 2024-09-19T06:54:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.465+0000 7f54bbfff640 1 --2- 172.21.15.103:0/2780702759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae21dd40 0x55c3ae221240 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f54b402a930 tx=0x7f54b4003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23876b1e90f63a0f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.465+0000 7f549affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.465+0000 7f549affd640 1 -- 172.21.15.103:0/2780702759 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4059070 con 0x55c3ae21dd40 2024-09-19T06:54:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.465+0000 7f549affd640 1 -- 172.21.15.103:0/2780702759 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f54b404c5f0 con 0x55c3ae21dd40 2024-09-19T06:54:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.465+0000 7f54c42fa740 1 -- 172.21.15.103:0/2780702759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54b4051070 con 0x55c3ae21dd40 2024-09-19T06:54:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.465+0000 7f549affd640 1 -- 172.21.15.103:0/2780702759 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54b4035070 con 0x55c3ae21dd40 2024-09-19T06:54:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.465+0000 7f54c42fa740 10 client.17526 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.465+0000 7f54c42fa740 1 -- 172.21.15.103:0/2780702759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c3ae297c40 con 0x55c3ae21dd40 2024-09-19T06:54:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.465+0000 7f549affd640 1 -- 172.21.15.103:0/2780702759 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b4051070 con 0x55c3ae21dd40 2024-09-19T06:54:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.466+0000 7f549affd640 1 -- 172.21.15.103:0/2780702759 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f54b4066840 con 0x55c3ae21dd40 2024-09-19T06:54:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.466+0000 7f54c42fa740 10 client.17526 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.466+0000 7f54c42fa740 10 client.17526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.466+0000 7f54c42fa740 20 client.17526 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.466+0000 7f54c42fa740 1 --2- 172.21.15.103:0/2780702759 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c3ae298340 0x55c3ae29a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.466+0000 7f54c42fa740 4 client.17526 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:54:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.466+0000 7f54c42fa740 1 -- 172.21.15.103:0/2780702759 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c3ae17fd00 con 0x55c3ae298340 2024-09-19T06:54:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.466+0000 7f54bb7fe640 1 --2- 172.21.15.103:0/2780702759 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c3ae298340 0x55c3ae29a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.466+0000 7f54bb7fe640 1 --2- 172.21.15.103:0/2780702759 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c3ae298340 0x55c3ae29a760 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dba1d329cbd8f9eb server_cookie=688f8a1f2d2d7d75 in_seq=0 out_seq=0 2024-09-19T06:54:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.466+0000 7f549affd640 10 client.17526 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.468+0000 7f549affd640 1 -- 172.21.15.103:0/2780702759 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c3ae17fd00 con 0x55c3ae298340 2024-09-19T06:54:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.468+0000 7f549affd640 10 client.17526 handle_command_reply: tid=0 2024-09-19T06:54:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.468+0000 7f54c42fa740 1 client.17526 shutdown 2024-09-19T06:54:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.468+0000 7f54c42fa740 1 -- 172.21.15.103:0/2780702759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae21dd40 msgr2=0x55c3ae221240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.468+0000 7f54c42fa740 1 --2- 172.21.15.103:0/2780702759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae21dd40 0x55c3ae221240 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f54b402a930 tx=0x7f54b4003f40 comp rx=0 tx=0).stop 2024-09-19T06:54:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.468+0000 7f54c42fa740 1 -- 172.21.15.103:0/2780702759 shutdown_connections 2024-09-19T06:54:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.468+0000 7f54c42fa740 1 --2- 172.21.15.103:0/2780702759 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c3ae298340 0x55c3ae29a760 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.468+0000 7f54c42fa740 1 --2- 172.21.15.103:0/2780702759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3ae21dd40 0x55c3ae221240 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.468+0000 7f54c42fa740 1 --2- 172.21.15.103:0/2780702759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3ae213480 0x55c3ae2262f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.468+0000 7f549affd640 0 client.17526 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:47.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.468+0000 7f54c42fa740 1 --2- 172.21.15.103:0/2780702759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3ae212b60 0x55c3ae225db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.468+0000 7f54c42fa740 1 -- 172.21.15.103:0/2780702759 >> 172.21.15.103:0/2780702759 conn(0x55c3ae20f0f0 msgr2=0x55c3ae219c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:47.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.468+0000 7f54c42fa740 1 -- 172.21.15.103:0/2780702759 shutdown_connections 2024-09-19T06:54:47.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.468+0000 7f54c42fa740 1 -- 172.21.15.103:0/2780702759 wait complete. 2024-09-19T06:54:47.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.468+0000 7f54c42fa740 20 client.17526 trim_cache size 0 max 16384 2024-09-19T06:54:47.474 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:54:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T06:54:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "7d298d5c-20a0-47b1-82c3-56ef27d5942f", 2024-09-19T06:54:47.474 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T06:54:47.474 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:54:47.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.469+0000 7f54c1f21640 1 -- 172.21.15.103:0/1101394274 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f54a40a33b0 msgr2=0x7f54a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:47.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.469+0000 7f54c1f21640 1 --2- 172.21.15.103:0/1101394274 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f54a40a33b0 0x7f54a40a5870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55c3ae210470 tx=0x7f54b40095d0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.469+0000 7f54c1f21640 1 -- 172.21.15.103:0/1101394274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc1027a0 msgr2=0x7f54bc198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:47.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.469+0000 7f54c1f21640 1 --2- 172.21.15.103:0/1101394274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc1027a0 0x7f54bc198a60 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f54b000b4a0 tx=0x7f54b000b960 comp rx=0 tx=0).stop 2024-09-19T06:54:47.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.469+0000 7f54c1f21640 1 -- 172.21.15.103:0/1101394274 shutdown_connections 2024-09-19T06:54:47.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.469+0000 7f54c1f21640 1 --2- 172.21.15.103:0/1101394274 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f54a40a33b0 0x7f54a40a5870 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.469+0000 7f54c1f21640 1 --2- 172.21.15.103:0/1101394274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54bc106f20 0x7f54bc19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.469+0000 7f54c1f21640 1 --2- 172.21.15.103:0/1101394274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc1027a0 0x7f54bc198a60 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.469+0000 7f54c1f21640 1 --2- 172.21.15.103:0/1101394274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc101df0 0x7f54bc198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:47.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.469+0000 7f54c1f21640 1 -- 172.21.15.103:0/1101394274 >> 172.21.15.103:0/1101394274 conn(0x7f54bc0fd8f0 msgr2=0x7f54bc0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:47.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.469+0000 7f54c1f21640 1 -- 172.21.15.103:0/1101394274 shutdown_connections 2024-09-19T06:54:47.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:47.469+0000 7f54c1f21640 1 -- 172.21.15.103:0/1101394274 wait complete. 2024-09-19T06:54:48.121 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:47 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T06:54:48.121 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:47 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:48.121 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:47 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T06:54:48.122 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:47 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:48.123 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:54:48.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:47 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T06:54:48.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:47 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:48.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:47 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T06:54:48.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:47 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:48.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:47 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T06:54:48.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:47 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:48.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:47 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T06:54:48.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:47 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:48.369 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:48.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.820+0000 7f1299c50640 1 -- 172.21.15.103:0/3054641451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1294100e20 msgr2=0x7f1294101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:48.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.820+0000 7f1299c50640 1 --2- 172.21.15.103:0/3054641451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1294100e20 0x7f1294101200 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f1284009900 tx=0x7f128402a480 comp rx=0 tx=0).stop 2024-09-19T06:54:48.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.820+0000 7f1299c50640 1 -- 172.21.15.103:0/3054641451 shutdown_connections 2024-09-19T06:54:48.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.820+0000 7f1299c50640 1 --2- 172.21.15.103:0/3054641451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1294102ef0 0x7f129410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.820+0000 7f1299c50640 1 --2- 172.21.15.103:0/3054641451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1294102570 0x7f12941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.820+0000 7f1299c50640 1 --2- 172.21.15.103:0/3054641451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1294100e20 0x7f1294101200 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.820+0000 7f1299c50640 1 -- 172.21.15.103:0/3054641451 >> 172.21.15.103:0/3054641451 conn(0x7f12940fc7d0 msgr2=0x7f12940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:48.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.820+0000 7f1299c50640 1 -- 172.21.15.103:0/3054641451 shutdown_connections 2024-09-19T06:54:48.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.820+0000 7f1299c50640 1 -- 172.21.15.103:0/3054641451 wait complete. 2024-09-19T06:54:48.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f1299c50640 1 Processor -- start 2024-09-19T06:54:48.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f1299c50640 1 -- start start 2024-09-19T06:54:48.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f1299c50640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1294100e20 0x7f129419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f1299c50640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1294102570 0x7f129419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f1299c50640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1294102ef0 0x7f129419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f1299c50640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1294110fa0 con 0x7f1294102570 2024-09-19T06:54:48.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f1299c50640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1294110e20 con 0x7f1294102ef0 2024-09-19T06:54:48.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f1299c50640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1294111120 con 0x7f1294100e20 2024-09-19T06:54:48.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f128affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1294102570 0x7f129419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f128affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1294102570 0x7f129419ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34820/0 (socket says 172.21.15.103:34820) 2024-09-19T06:54:48.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f128affd640 1 -- 172.21.15.103:0/1039213609 learned_addr learned my addr 172.21.15.103:0/1039213609 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:48.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f1293fff640 1 --2- 172.21.15.103:0/1039213609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1294102ef0 0x7f129419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f12937fe640 1 --2- 172.21.15.103:0/1039213609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1294100e20 0x7f129419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f128affd640 1 -- 172.21.15.103:0/1039213609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1294100e20 msgr2=0x7f129419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:48.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f128affd640 1 --2- 172.21.15.103:0/1039213609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1294100e20 0x7f129419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f128affd640 1 -- 172.21.15.103:0/1039213609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1294102ef0 msgr2=0x7f129419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:48.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f128affd640 1 --2- 172.21.15.103:0/1039213609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1294102ef0 0x7f129419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f128affd640 1 -- 172.21.15.103:0/1039213609 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129419f680 con 0x7f1294102570 2024-09-19T06:54:48.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.821+0000 7f12937fe640 1 --2- 172.21.15.103:0/1039213609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1294100e20 0x7f129419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:48.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.822+0000 7f128affd640 1 --2- 172.21.15.103:0/1039213609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1294102570 0x7f129419ac10 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f127c00c920 tx=0x7f127c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:48.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.822+0000 7f12917fa640 1 -- 172.21.15.103:0/1039213609 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c007c70 con 0x7f1294102570 2024-09-19T06:54:48.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.822+0000 7f12917fa640 1 -- 172.21.15.103:0/1039213609 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f127c007e10 con 0x7f1294102570 2024-09-19T06:54:48.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.822+0000 7f1299c50640 1 -- 172.21.15.103:0/1039213609 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129419f970 con 0x7f1294102570 2024-09-19T06:54:48.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.822+0000 7f12917fa640 1 -- 172.21.15.103:0/1039213609 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c018890 con 0x7f1294102570 2024-09-19T06:54:48.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.822+0000 7f1299c50640 1 -- 172.21.15.103:0/1039213609 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12941a7250 con 0x7f1294102570 2024-09-19T06:54:48.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.824+0000 7f12917fa640 1 -- 172.21.15.103:0/1039213609 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f127c018a70 con 0x7f1294102570 2024-09-19T06:54:48.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.824+0000 7f129c029740 1 Processor -- start 2024-09-19T06:54:48.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.824+0000 7f129c029740 1 -- start start 2024-09-19T06:54:48.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f129c029740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a324a0 0x55ddb0a32880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f129c029740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a3d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f129c029740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a3fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f129c029740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ddb09982e0 con 0x55ddb0a3d680 2024-09-19T06:54:48.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f129c029740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ddb093fce0 con 0x55ddb0a32dc0 2024-09-19T06:54:48.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f129c029740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ddb09942a0 con 0x55ddb0a324a0 2024-09-19T06:54:48.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f12917fa640 1 --2- 172.21.15.103:0/1039213609 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f12800a33b0 0x7f12800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f1293fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a324a0 0x55ddb0a32880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f1293fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a324a0 0x55ddb0a32880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:33950/0 (socket says 172.21.15.103:33950) 2024-09-19T06:54:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f1293fff640 1 -- 172.21.15.103:0/2776904615 learned_addr learned my addr 172.21.15.103:0/2776904615 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f128affd640 1 --2- 172.21.15.103:0/2776904615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a3fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f12937fe640 1 --2- 172.21.15.103:0/2776904615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a3d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f1293fff640 1 --2- 172.21.15.103:0/1039213609 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f12800a33b0 0x7f12800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f12917fa640 1 -- 172.21.15.103:0/1039213609 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f127c0cff50 con 0x7f1294102570 2024-09-19T06:54:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f128affd640 1 -- 172.21.15.103:0/2776904615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a324a0 msgr2=0x55ddb0a32880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f128affd640 1 --2- 172.21.15.103:0/2776904615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a324a0 0x55ddb0a32880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f128affd640 1 -- 172.21.15.103:0/2776904615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a32dc0 msgr2=0x55ddb0a3d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f128affd640 1 --2- 172.21.15.103:0/2776904615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a3d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.825+0000 7f128affd640 1 -- 172.21.15.103:0/2776904615 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb0924440 con 0x55ddb0a3d680 2024-09-19T06:54:48.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.826+0000 7f128affd640 1 --2- 172.21.15.103:0/2776904615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a3fa70 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f127c0cd670 tx=0x7f127c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b92f3758b10d495 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.826+0000 7f1293fff640 1 --2- 172.21.15.103:0/1039213609 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f12800a33b0 0x7f12800a5870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f128c009fd0 tx=0x7f128c0093d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.826+0000 7f1289ffb640 1 -- 172.21.15.103:0/2776904615 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c09c900 con 0x55ddb0a3d680 2024-09-19T06:54:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.826+0000 7f1289ffb640 1 -- 172.21.15.103:0/2776904615 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f127c00de20 con 0x55ddb0a3d680 2024-09-19T06:54:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.826+0000 7f1289ffb640 1 -- 172.21.15.103:0/2776904615 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c098870 con 0x55ddb0a3d680 2024-09-19T06:54:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.826+0000 7f129c029740 1 -- 172.21.15.103:0/2776904615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 msgr2=0x55ddb0a3fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.826+0000 7f129c029740 1 --2- 172.21.15.103:0/2776904615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a3fa70 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f127c0cd670 tx=0x7f127c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.826+0000 7f129c029740 1 -- 172.21.15.103:0/2776904615 shutdown_connections 2024-09-19T06:54:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.826+0000 7f129c029740 1 --2- 172.21.15.103:0/2776904615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a3fa70 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.826+0000 7f129c029740 1 --2- 172.21.15.103:0/2776904615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a3d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.826+0000 7f129c029740 1 --2- 172.21.15.103:0/2776904615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a324a0 0x55ddb0a32880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.826+0000 7f129c029740 1 -- 172.21.15.103:0/2776904615 >> 172.21.15.103:0/2776904615 conn(0x55ddb0a2ea30 msgr2=0x55ddb0a2ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.827+0000 7f129c029740 1 -- 172.21.15.103:0/2776904615 shutdown_connections 2024-09-19T06:54:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.827+0000 7f129c029740 1 -- 172.21.15.103:0/2776904615 wait complete. 2024-09-19T06:54:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.827+0000 7f129c029740 1 -- start start 2024-09-19T06:54:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.827+0000 7f129c029740 1 Processor -- start 2024-09-19T06:54:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.827+0000 7f129c029740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a324a0 0x55ddb0a3cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.827+0000 7f129c029740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a3a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.827+0000 7f129c029740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a3d680 0x55ddb0a3a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.827+0000 7f129c029740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ddb0a40250 con 0x55ddb0a32dc0 2024-09-19T06:54:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.827+0000 7f129c029740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ddb0953050 con 0x55ddb0a3d680 2024-09-19T06:54:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.827+0000 7f129c029740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ddb0a403d0 con 0x55ddb0a324a0 2024-09-19T06:54:48.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.827+0000 7f129c029740 10 client.0 _ll_register_callbacks cb 0x55ddb0993160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.827+0000 7f128affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a3a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f128affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a3a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34844/0 (socket says 172.21.15.103:34844) 2024-09-19T06:54:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f128affd640 1 -- 172.21.15.103:0/2265882446 learned_addr learned my addr 172.21.15.103:0/2265882446 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f1293fff640 1 --2- 172.21.15.103:0/2265882446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a3d680 0x55ddb0a3a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f12937fe640 1 --2- 172.21.15.103:0/2265882446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a324a0 0x55ddb0a3cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f128affd640 1 -- 172.21.15.103:0/2265882446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a324a0 msgr2=0x55ddb0a3cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f128affd640 1 --2- 172.21.15.103:0/2265882446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a324a0 0x55ddb0a3cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f128affd640 1 -- 172.21.15.103:0/2265882446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a3d680 msgr2=0x55ddb0a3a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f128affd640 1 --2- 172.21.15.103:0/2265882446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a3d680 0x55ddb0a3a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f128affd640 1 -- 172.21.15.103:0/2265882446 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddb07093c0 con 0x55ddb0a32dc0 2024-09-19T06:54:48.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f128affd640 1 --2- 172.21.15.103:0/2265882446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a3a250 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f127c09c190 tx=0x7f127c09c200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48903a71479a08b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f1272ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f1272ffd640 1 -- 172.21.15.103:0/2265882446 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c0cdaf0 con 0x55ddb0a32dc0 2024-09-19T06:54:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f1272ffd640 1 -- 172.21.15.103:0/2265882446 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f127c0bc0d0 con 0x55ddb0a32dc0 2024-09-19T06:54:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f129c029740 1 -- 172.21.15.103:0/2265882446 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f127c00de20 con 0x55ddb0a32dc0 2024-09-19T06:54:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f1272ffd640 1 -- 172.21.15.103:0/2265882446 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f127c0a05b0 con 0x55ddb0a32dc0 2024-09-19T06:54:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f129c029740 10 client.17538 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.828+0000 7f129c029740 1 -- 172.21.15.103:0/2265882446 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ddb0a4bbd0 con 0x55ddb0a32dc0 2024-09-19T06:54:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.829+0000 7f1272ffd640 1 -- 172.21.15.103:0/2265882446 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f127c00de20 con 0x55ddb0a32dc0 2024-09-19T06:54:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.829+0000 7f1272ffd640 1 -- 172.21.15.103:0/2265882446 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f127c0a49e0 con 0x55ddb0a32dc0 2024-09-19T06:54:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.829+0000 7f129c029740 10 client.17538 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.829+0000 7f129c029740 10 client.17538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:48.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.829+0000 7f129c029740 20 client.17538 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.829+0000 7f129c029740 1 --2- 172.21.15.103:0/2265882446 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ddb0a4cf20 0x55ddb0a4f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.829+0000 7f129c029740 4 client.17538 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.829+0000 7f129c029740 1 -- 172.21.15.103:0/2265882446 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddb09a5850 con 0x55ddb0a4cf20 2024-09-19T06:54:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.829+0000 7f12937fe640 1 --2- 172.21.15.103:0/2265882446 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ddb0a4cf20 0x55ddb0a4f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.830+0000 7f12937fe640 1 --2- 172.21.15.103:0/2265882446 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ddb0a4cf20 0x55ddb0a4f310 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a5ac550113555cf server_cookie=6f6eca65679a2c63 in_seq=0 out_seq=0 2024-09-19T06:54:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.830+0000 7f1272ffd640 10 client.17538 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.830+0000 7f1272ffd640 1 -- 172.21.15.103:0/2265882446 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ddb09a5850 con 0x55ddb0a4cf20 2024-09-19T06:54:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.830+0000 7f1272ffd640 10 client.17538 handle_command_reply: tid=0 2024-09-19T06:54:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.830+0000 7f129c029740 1 client.17538 shutdown 2024-09-19T06:54:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.830+0000 7f129c029740 1 -- 172.21.15.103:0/2265882446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a32dc0 msgr2=0x55ddb0a3a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.830+0000 7f129c029740 1 --2- 172.21.15.103:0/2265882446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a3a250 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f127c09c190 tx=0x7f127c09c200 comp rx=0 tx=0).stop 2024-09-19T06:54:48.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.830+0000 7f129c029740 1 -- 172.21.15.103:0/2265882446 shutdown_connections 2024-09-19T06:54:48.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.830+0000 7f129c029740 1 --2- 172.21.15.103:0/2265882446 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ddb0a4cf20 0x55ddb0a4f310 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.830+0000 7f129c029740 1 --2- 172.21.15.103:0/2265882446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a3d680 0x55ddb0a3a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.830+0000 7f129c029740 1 --2- 172.21.15.103:0/2265882446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a3a250 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.830+0000 7f129c029740 1 --2- 172.21.15.103:0/2265882446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a324a0 0x55ddb0a3cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.830+0000 7f129c029740 1 -- 172.21.15.103:0/2265882446 >> 172.21.15.103:0/2265882446 conn(0x55ddb0a2ea30 msgr2=0x55ddb0a34760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:48.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.830+0000 7f1272ffd640 0 client.17538 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:48.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.831+0000 7f129c029740 1 -- 172.21.15.103:0/2265882446 shutdown_connections 2024-09-19T06:54:48.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.831+0000 7f129c029740 1 -- 172.21.15.103:0/2265882446 wait complete. 2024-09-19T06:54:48.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.831+0000 7f129c029740 20 client.17538 trim_cache size 0 max 16384 2024-09-19T06:54:48.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.847+0000 7f129c029740 1 Processor -- start 2024-09-19T06:54:48.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.847+0000 7f129c029740 1 -- start start 2024-09-19T06:54:48.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.847+0000 7f129c029740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a324a0 0x55ddb0a494b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.847+0000 7f129c029740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a3bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.847+0000 7f129c029740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.847+0000 7f129c029740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f12941afae0 con 0x55ddb0a3d680 2024-09-19T06:54:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.847+0000 7f129c029740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f12941b1620 con 0x55ddb0a324a0 2024-09-19T06:54:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.847+0000 7f129c029740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12941b01b0 con 0x55ddb0a32dc0 2024-09-19T06:54:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.848+0000 7f1293fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.848+0000 7f1293fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34850/0 (socket says 172.21.15.103:34850) 2024-09-19T06:54:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.848+0000 7f1293fff640 1 -- 172.21.15.103:0/760727688 learned_addr learned my addr 172.21.15.103:0/760727688 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.848+0000 7f12937fe640 1 --2- 172.21.15.103:0/760727688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a324a0 0x55ddb0a494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.848+0000 7f1293fff640 1 -- 172.21.15.103:0/760727688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a32dc0 msgr2=0x55ddb0a3bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.848+0000 7f128affd640 1 --2- 172.21.15.103:0/760727688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a3bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.848+0000 7f1293fff640 1 --2- 172.21.15.103:0/760727688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a3bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.848+0000 7f1293fff640 1 -- 172.21.15.103:0/760727688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a324a0 msgr2=0x55ddb0a494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.848+0000 7f1293fff640 1 --2- 172.21.15.103:0/760727688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a324a0 0x55ddb0a494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.848+0000 7f1293fff640 1 -- 172.21.15.103:0/760727688 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb0a2e4d0 con 0x55ddb0a3d680 2024-09-19T06:54:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.848+0000 7f1293fff640 1 --2- 172.21.15.103:0/760727688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a548b0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f128c009780 tx=0x7f128c00c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1172f4085e1b3782 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.848+0000 7f12897fa640 1 -- 172.21.15.103:0/760727688 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c02a070 con 0x55ddb0a3d680 2024-09-19T06:54:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.848+0000 7f12897fa640 1 -- 172.21.15.103:0/760727688 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f128c01bdf0 con 0x55ddb0a3d680 2024-09-19T06:54:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.848+0000 7f12897fa640 1 -- 172.21.15.103:0/760727688 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c013040 con 0x55ddb0a3d680 2024-09-19T06:54:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.849+0000 7f129c029740 1 -- 172.21.15.103:0/760727688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 msgr2=0x55ddb0a548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.849+0000 7f129c029740 1 --2- 172.21.15.103:0/760727688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a548b0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f128c009780 tx=0x7f128c00c070 comp rx=0 tx=0).stop 2024-09-19T06:54:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.849+0000 7f129c029740 1 -- 172.21.15.103:0/760727688 shutdown_connections 2024-09-19T06:54:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.849+0000 7f129c029740 1 --2- 172.21.15.103:0/760727688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a548b0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.849+0000 7f129c029740 1 --2- 172.21.15.103:0/760727688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a3bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.849+0000 7f129c029740 1 --2- 172.21.15.103:0/760727688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a324a0 0x55ddb0a494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.849+0000 7f129c029740 1 -- 172.21.15.103:0/760727688 >> 172.21.15.103:0/760727688 conn(0x55ddb0a2ea30 msgr2=0x55ddb0a39210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.849+0000 7f129c029740 1 -- 172.21.15.103:0/760727688 shutdown_connections 2024-09-19T06:54:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.849+0000 7f129c029740 1 -- 172.21.15.103:0/760727688 wait complete. 2024-09-19T06:54:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.849+0000 7f129c029740 1 -- start start 2024-09-19T06:54:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.849+0000 7f129c029740 1 Processor -- start 2024-09-19T06:54:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f129c029740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a324a0 0x55ddb0a41580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f129c029740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a41ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f129c029740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a42000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f129c029740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f12941b0f50 con 0x55ddb0a3d680 2024-09-19T06:54:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f129c029740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f12941b0880 con 0x55ddb0a324a0 2024-09-19T06:54:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f129c029740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12941adaa0 con 0x55ddb0a32dc0 2024-09-19T06:54:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f129c029740 10 client.0 _ll_register_callbacks cb 0x7f1284009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f1293fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a42000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f1293fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a42000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34852/0 (socket says 172.21.15.103:34852) 2024-09-19T06:54:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f1293fff640 1 -- 172.21.15.103:0/2754603199 learned_addr learned my addr 172.21.15.103:0/2754603199 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f128affd640 1 --2- 172.21.15.103:0/2754603199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a41ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f12937fe640 1 --2- 172.21.15.103:0/2754603199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a324a0 0x55ddb0a41580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f1293fff640 1 -- 172.21.15.103:0/2754603199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a32dc0 msgr2=0x55ddb0a41ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:48.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f1293fff640 1 --2- 172.21.15.103:0/2754603199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a41ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f1293fff640 1 -- 172.21.15.103:0/2754603199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a324a0 msgr2=0x55ddb0a41580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:48.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f1293fff640 1 --2- 172.21.15.103:0/2754603199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a324a0 0x55ddb0a41580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f1293fff640 1 -- 172.21.15.103:0/2754603199 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddb0ab6a80 con 0x55ddb0a3d680 2024-09-19T06:54:48.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f12937fe640 1 --2- 172.21.15.103:0/2754603199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a324a0 0x55ddb0a41580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:48.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f1293fff640 1 --2- 172.21.15.103:0/2754603199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a42000 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f128c00ab50 tx=0x7f128c00a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda66598f81a7f8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:48.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f1272ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:48.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f1272ffd640 1 -- 172.21.15.103:0/2754603199 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c033060 con 0x55ddb0a3d680 2024-09-19T06:54:48.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f1272ffd640 1 -- 172.21.15.103:0/2754603199 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f128c01b910 con 0x55ddb0a3d680 2024-09-19T06:54:48.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.850+0000 7f129c029740 1 -- 172.21.15.103:0/2754603199 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f128c01bdf0 con 0x55ddb0a3d680 2024-09-19T06:54:48.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.851+0000 7f1272ffd640 1 -- 172.21.15.103:0/2754603199 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f128c02a050 con 0x55ddb0a3d680 2024-09-19T06:54:48.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.851+0000 7f129c029740 10 client.17544 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:48.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.851+0000 7f129c029740 1 -- 172.21.15.103:0/2754603199 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ddb0ab7d70 con 0x55ddb0a3d680 2024-09-19T06:54:48.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.851+0000 7f1272ffd640 1 -- 172.21.15.103:0/2754603199 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f128c01bdf0 con 0x55ddb0a3d680 2024-09-19T06:54:48.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.851+0000 7f1272ffd640 1 -- 172.21.15.103:0/2754603199 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f128c02e730 con 0x55ddb0a3d680 2024-09-19T06:54:48.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.851+0000 7f129c029740 10 client.17544 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:48.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.851+0000 7f129c029740 10 client.17544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:48.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.851+0000 7f129c029740 20 client.17544 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:48.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.851+0000 7f129c029740 1 --2- 172.21.15.103:0/2754603199 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ddb0ab8370 0x55ddb0aba790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:48.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.851+0000 7f129c029740 4 client.17544 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:54:48.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.851+0000 7f129c029740 1 -- 172.21.15.103:0/2754603199 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ddb099fd60 con 0x55ddb0ab8370 2024-09-19T06:54:48.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.851+0000 7f12937fe640 1 --2- 172.21.15.103:0/2754603199 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ddb0ab8370 0x55ddb0aba790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:48.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f12937fe640 1 --2- 172.21.15.103:0/2754603199 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ddb0ab8370 0x55ddb0aba790 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3f8bc90ae6728a7 server_cookie=297270d48ad208bd in_seq=0 out_seq=0 2024-09-19T06:54:48.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f1272ffd640 10 client.17544 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:48.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f1272ffd640 1 -- 172.21.15.103:0/2754603199 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ddb099fd60 con 0x55ddb0ab8370 2024-09-19T06:54:48.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f1272ffd640 10 client.17544 handle_command_reply: tid=0 2024-09-19T06:54:48.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f129c029740 1 client.17544 shutdown 2024-09-19T06:54:48.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f129c029740 1 -- 172.21.15.103:0/2754603199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 msgr2=0x55ddb0a42000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:48.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f129c029740 1 --2- 172.21.15.103:0/2754603199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a42000 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f128c00ab50 tx=0x7f128c00a5b0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f129c029740 1 -- 172.21.15.103:0/2754603199 shutdown_connections 2024-09-19T06:54:48.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f129c029740 1 --2- 172.21.15.103:0/2754603199 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ddb0ab8370 0x55ddb0aba790 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f1272ffd640 0 client.17544 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:48.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f129c029740 1 --2- 172.21.15.103:0/2754603199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddb0a3d680 0x55ddb0a42000 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f129c029740 1 --2- 172.21.15.103:0/2754603199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddb0a32dc0 0x55ddb0a41ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f129c029740 1 --2- 172.21.15.103:0/2754603199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ddb0a324a0 0x55ddb0a41580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f129c029740 1 -- 172.21.15.103:0/2754603199 >> 172.21.15.103:0/2754603199 conn(0x55ddb0a2ea30 msgr2=0x55ddb0a2d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:48.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f129c029740 1 -- 172.21.15.103:0/2754603199 shutdown_connections 2024-09-19T06:54:48.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f129c029740 1 -- 172.21.15.103:0/2754603199 wait complete. 2024-09-19T06:54:48.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.852+0000 7f129c029740 20 client.17544 trim_cache size 0 max 16384 2024-09-19T06:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T06:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: "7d298d5c-20a0-47b1-82c3-56ef27d5942f": { 2024-09-19T06:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T06:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "7d298d5c-20a0-47b1-82c3-56ef27d5942f", 2024-09-19T06:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T06:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:54:48.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.853+0000 7f1299c50640 1 -- 172.21.15.103:0/1039213609 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f12800a33b0 msgr2=0x7f12800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:48.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.853+0000 7f1299c50640 1 --2- 172.21.15.103:0/1039213609 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f12800a33b0 0x7f12800a5870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f128c009fd0 tx=0x7f128c0093d0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.853+0000 7f1299c50640 1 -- 172.21.15.103:0/1039213609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1294102570 msgr2=0x7f129419ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:48.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.853+0000 7f1299c50640 1 --2- 172.21.15.103:0/1039213609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1294102570 0x7f129419ac10 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f127c00c920 tx=0x7f127c00cde0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.853+0000 7f1299c50640 1 -- 172.21.15.103:0/1039213609 shutdown_connections 2024-09-19T06:54:48.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.853+0000 7f1299c50640 1 --2- 172.21.15.103:0/1039213609 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f12800a33b0 0x7f12800a5870 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.853+0000 7f1299c50640 1 --2- 172.21.15.103:0/1039213609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1294102ef0 0x7f129419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.853+0000 7f1299c50640 1 --2- 172.21.15.103:0/1039213609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1294102570 0x7f129419ac10 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.853+0000 7f1299c50640 1 --2- 172.21.15.103:0/1039213609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1294100e20 0x7f129419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:48.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.853+0000 7f1299c50640 1 -- 172.21.15.103:0/1039213609 >> 172.21.15.103:0/1039213609 conn(0x7f12940fc7d0 msgr2=0x7f1294069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:48.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.853+0000 7f1299c50640 1 -- 172.21.15.103:0/1039213609 shutdown_connections 2024-09-19T06:54:48.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:54:48.853+0000 7f1299c50640 1 -- 172.21.15.103:0/1039213609 wait complete. 2024-09-19T06:54:49.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:48 smithi103 ceph-mon[31158]: pgmap v170: 97 pgs: 97 active+clean; 4.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-09-19T06:54:49.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:48 smithi134 ceph-mon[35356]: pgmap v170: 97 pgs: 97 active+clean; 4.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-09-19T06:54:49.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:48 smithi174 ceph-mon[35637]: pgmap v170: 97 pgs: 97 active+clean; 4.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-09-19T06:54:49.515 INFO:tasks.cephfs.filesystem:scrub status for tag:7d298d5c-20a0-47b1-82c3-56ef27d5942f - {'path': '/', 'tag': '7d298d5c-20a0-47b1-82c3-56ef27d5942f', 'options': 'recursive,force'} 2024-09-19T06:54:50.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:49 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T06:54:50.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:49 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T06:54:50.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:49 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T06:54:51.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:50 smithi103 ceph-mon[31158]: pgmap v171: 97 pgs: 97 active+clean; 5.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 21 op/s 2024-09-19T06:54:51.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:50 smithi134 ceph-mon[35356]: pgmap v171: 97 pgs: 97 active+clean; 5.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 21 op/s 2024-09-19T06:54:51.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:50 smithi174 ceph-mon[35637]: pgmap v171: 97 pgs: 97 active+clean; 5.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 21 op/s 2024-09-19T06:54:52.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:52 smithi103 ceph-mon[31158]: pgmap v172: 97 pgs: 97 active+clean; 5.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-09-19T06:54:52.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:52 smithi134 ceph-mon[35356]: pgmap v172: 97 pgs: 97 active+clean; 5.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-09-19T06:54:52.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:52 smithi174 ceph-mon[35637]: pgmap v172: 97 pgs: 97 active+clean; 5.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-09-19T06:54:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.754+0000 7f700ae3c640 1 -- 172.21.15.103:0/2066668902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7004102cc0 msgr2=0x7f70041030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.754+0000 7f700ae3c640 1 --2- 172.21.15.103:0/2066668902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7004102cc0 0x7f70041030a0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f7000009900 tx=0x7f700002a480 comp rx=0 tx=0).stop 2024-09-19T06:54:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.754+0000 7f700ae3c640 1 -- 172.21.15.103:0/2066668902 shutdown_connections 2024-09-19T06:54:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.754+0000 7f700ae3c640 1 --2- 172.21.15.103:0/2066668902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f700410c110 0x7f700410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.754+0000 7f700ae3c640 1 --2- 172.21.15.103:0/2066668902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70041035e0 0x7f700410bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.754+0000 7f700ae3c640 1 --2- 172.21.15.103:0/2066668902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7004102cc0 0x7f70041030a0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.754+0000 7f700ae3c640 1 -- 172.21.15.103:0/2066668902 >> 172.21.15.103:0/2066668902 conn(0x7f70040fc6b0 msgr2=0x7f70040fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.754+0000 7f700ae3c640 1 -- 172.21.15.103:0/2066668902 shutdown_connections 2024-09-19T06:54:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.754+0000 7f700ae3c640 1 -- 172.21.15.103:0/2066668902 wait complete. 2024-09-19T06:54:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.755+0000 7f700ae3c640 1 Processor -- start 2024-09-19T06:54:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.755+0000 7f700ae3c640 1 -- start start 2024-09-19T06:54:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.755+0000 7f700ae3c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7004102cc0 0x7f7004198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.755+0000 7f700ae3c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70041035e0 0x7f7004198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.755+0000 7f700ae3c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f700410c110 0x7f700419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.755+0000 7f700ae3c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7004111ef0 con 0x7f7004102cc0 2024-09-19T06:54:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.755+0000 7f700ae3c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7004111d70 con 0x7f70041035e0 2024-09-19T06:54:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.755+0000 7f700ae3c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7004112070 con 0x7f700410c110 2024-09-19T06:54:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.755+0000 7f7008bd8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7004102cc0 0x7f7004198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.755+0000 7f7008bd8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7004102cc0 0x7f7004198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34876/0 (socket says 172.21.15.103:34876) 2024-09-19T06:54:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.755+0000 7f7008bd8640 1 -- 172.21.15.103:0/789367024 learned_addr learned my addr 172.21.15.103:0/789367024 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.755+0000 7f6ffbfff640 1 --2- 172.21.15.103:0/789367024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70041035e0 0x7f7004198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.756+0000 7f70093d9640 1 --2- 172.21.15.103:0/789367024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f700410c110 0x7f700419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.756+0000 7f7008bd8640 1 -- 172.21.15.103:0/789367024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f700410c110 msgr2=0x7f700419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.756+0000 7f7008bd8640 1 --2- 172.21.15.103:0/789367024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f700410c110 0x7f700419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.756+0000 7f7008bd8640 1 -- 172.21.15.103:0/789367024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70041035e0 msgr2=0x7f7004198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.756+0000 7f7008bd8640 1 --2- 172.21.15.103:0/789367024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70041035e0 0x7f7004198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.756+0000 7f7008bd8640 1 -- 172.21.15.103:0/789367024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700419d320 con 0x7f7004102cc0 2024-09-19T06:54:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.756+0000 7f70093d9640 1 --2- 172.21.15.103:0/789367024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f700410c110 0x7f700419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.756+0000 7f7008bd8640 1 --2- 172.21.15.103:0/789367024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7004102cc0 0x7f7004198330 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7000006010 tx=0x7f70000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.756+0000 7f6ff9ffb640 1 -- 172.21.15.103:0/789367024 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700003f030 con 0x7f7004102cc0 2024-09-19T06:54:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.756+0000 7f6ff9ffb640 1 -- 172.21.15.103:0/789367024 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7000003ce0 con 0x7f7004102cc0 2024-09-19T06:54:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.756+0000 7f700ae3c640 1 -- 172.21.15.103:0/789367024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700419d5b0 con 0x7f7004102cc0 2024-09-19T06:54:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.756+0000 7f6ff9ffb640 1 -- 172.21.15.103:0/789367024 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000035070 con 0x7f7004102cc0 2024-09-19T06:54:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.756+0000 7f700ae3c640 1 -- 172.21.15.103:0/789367024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70041a4ea0 con 0x7f7004102cc0 2024-09-19T06:54:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.758+0000 7f6ff9ffb640 1 -- 172.21.15.103:0/789367024 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7000042030 con 0x7f7004102cc0 2024-09-19T06:54:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.759+0000 7f700dbd6740 1 Processor -- start 2024-09-19T06:54:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.759+0000 7f700dbd6740 1 -- start start 2024-09-19T06:54:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.759+0000 7f6ff9ffb640 1 --2- 172.21.15.103:0/789367024 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6fec0a3460 0x7f6fec0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.759+0000 7f700dbd6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331bfbaf0 0x55c331bfbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.759+0000 7f700dbd6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c06770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.759+0000 7f700dbd6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331c06cb0 0x55c331c090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.759+0000 7f700dbd6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c331b602f0 con 0x55c331bfc410 2024-09-19T06:54:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.759+0000 7f700dbd6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c331b08270 con 0x55c331c06cb0 2024-09-19T06:54:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.759+0000 7f700dbd6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c331b5c2b0 con 0x55c331bfbaf0 2024-09-19T06:54:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.759+0000 7f6ff9ffb640 1 -- 172.21.15.103:0/789367024 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f70000f0330 con 0x7f7004102cc0 2024-09-19T06:54:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.759+0000 7f70093d9640 1 --2- 172.21.15.103:0/789367024 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6fec0a3460 0x7f6fec0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.759+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c06770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.759+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c06770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34886/0 (socket says 172.21.15.103:34886) 2024-09-19T06:54:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.759+0000 7f6ffbfff640 1 -- 172.21.15.103:0/2298958789 learned_addr learned my addr 172.21.15.103:0/2298958789 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.759+0000 7f70093d9640 1 --2- 172.21.15.103:0/2298958789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331c06cb0 0x55c331c090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.759+0000 7f7008bd8640 1 --2- 172.21.15.103:0/2298958789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331bfbaf0 0x55c331bfbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.760+0000 7f6ffbfff640 1 -- 172.21.15.103:0/2298958789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331bfbaf0 msgr2=0x55c331bfbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.760+0000 7f6ffbfff640 1 --2- 172.21.15.103:0/2298958789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331bfbaf0 0x55c331bfbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.760+0000 7f6ffbfff640 1 -- 172.21.15.103:0/2298958789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331c06cb0 msgr2=0x55c331c090a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.760+0000 7f6ffbfff640 1 --2- 172.21.15.103:0/2298958789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331c06cb0 0x55c331c090a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.760+0000 7f70093d9640 1 --2- 172.21.15.103:0/789367024 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6fec0a3460 0x7f6fec0a5920 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0094d0 tx=0x7f6ffc013b20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.760+0000 7f6ffbfff640 1 -- 172.21.15.103:0/2298958789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c331b07340 con 0x55c331bfc410 2024-09-19T06:54:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.760+0000 7f6ffbfff640 1 --2- 172.21.15.103:0/2298958789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c06770 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400bca0 tx=0x7f6ff40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55df2a4405246256 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.760+0000 7f6feaffd640 1 -- 172.21.15.103:0/2298958789 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4004030 con 0x55c331bfc410 2024-09-19T06:54:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.760+0000 7f6feaffd640 1 -- 172.21.15.103:0/2298958789 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff40041d0 con 0x55c331bfc410 2024-09-19T06:54:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.760+0000 7f6feaffd640 1 -- 172.21.15.103:0/2298958789 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4011640 con 0x55c331bfc410 2024-09-19T06:54:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.761+0000 7f700dbd6740 1 -- 172.21.15.103:0/2298958789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 msgr2=0x55c331c06770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.761+0000 7f700dbd6740 1 --2- 172.21.15.103:0/2298958789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c06770 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400bca0 tx=0x7f6ff40107a0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.761+0000 7f700dbd6740 1 -- 172.21.15.103:0/2298958789 shutdown_connections 2024-09-19T06:54:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.761+0000 7f700dbd6740 1 --2- 172.21.15.103:0/2298958789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331c06cb0 0x55c331c090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.761+0000 7f700dbd6740 1 --2- 172.21.15.103:0/2298958789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c06770 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.761+0000 7f700dbd6740 1 --2- 172.21.15.103:0/2298958789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331bfbaf0 0x55c331bfbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.761+0000 7f700dbd6740 1 -- 172.21.15.103:0/2298958789 >> 172.21.15.103:0/2298958789 conn(0x55c331bf8060 msgr2=0x55c331bf8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.761+0000 7f700dbd6740 1 -- 172.21.15.103:0/2298958789 shutdown_connections 2024-09-19T06:54:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.761+0000 7f700dbd6740 1 -- 172.21.15.103:0/2298958789 wait complete. 2024-09-19T06:54:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.761+0000 7f700dbd6740 1 -- start start 2024-09-19T06:54:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.761+0000 7f700dbd6740 1 Processor -- start 2024-09-19T06:54:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f700dbd6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfbaf0 0x55c331c03b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f700dbd6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331bfc410 0x55c331c040d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f700dbd6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331c06cb0 0x55c331c04610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f700dbd6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c331c09880 con 0x55c331bfbaf0 2024-09-19T06:54:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f700dbd6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c331ae9290 con 0x55c331c06cb0 2024-09-19T06:54:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f700dbd6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c331c09a00 con 0x55c331bfc410 2024-09-19T06:54:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f700dbd6740 10 client.0 _ll_register_callbacks cb 0x55c331b1a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f7008bd8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfbaf0 0x55c331c03b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f7008bd8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfbaf0 0x55c331c03b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34898/0 (socket says 172.21.15.103:34898) 2024-09-19T06:54:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f7008bd8640 1 -- 172.21.15.103:0/2070684847 learned_addr learned my addr 172.21.15.103:0/2070684847 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f70093d9640 1 --2- 172.21.15.103:0/2070684847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331c06cb0 0x55c331c04610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f7008bd8640 1 -- 172.21.15.103:0/2070684847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331bfc410 msgr2=0x55c331c040d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f6ffbfff640 1 --2- 172.21.15.103:0/2070684847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331bfc410 0x55c331c040d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f7008bd8640 1 --2- 172.21.15.103:0/2070684847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331bfc410 0x55c331c040d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f7008bd8640 1 -- 172.21.15.103:0/2070684847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331c06cb0 msgr2=0x55c331c04610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f7008bd8640 1 --2- 172.21.15.103:0/2070684847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331c06cb0 0x55c331c04610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f7008bd8640 1 -- 172.21.15.103:0/2070684847 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c331873210 con 0x55c331bfbaf0 2024-09-19T06:54:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f70093d9640 1 --2- 172.21.15.103:0/2070684847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331c06cb0 0x55c331c04610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f7008bd8640 1 --2- 172.21.15.103:0/2070684847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfbaf0 0x55c331c03b90 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f70000b8b00 tx=0x7f70000b8940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d522df4e746fdbb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.762+0000 7f6fe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.763+0000 7f6fe3fff640 1 -- 172.21.15.103:0/2070684847 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000049070 con 0x55c331bfbaf0 2024-09-19T06:54:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.763+0000 7f6fe3fff640 1 -- 172.21.15.103:0/2070684847 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f70000edcd0 con 0x55c331bfbaf0 2024-09-19T06:54:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.763+0000 7f700dbd6740 1 -- 172.21.15.103:0/2070684847 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ff40041d0 con 0x55c331bfbaf0 2024-09-19T06:54:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.763+0000 7f6fe3fff640 1 -- 172.21.15.103:0/2070684847 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f70000bc1e0 con 0x55c331bfbaf0 2024-09-19T06:54:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.763+0000 7f700dbd6740 10 client.17556 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.763+0000 7f700dbd6740 1 -- 172.21.15.103:0/2070684847 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c33193f620 con 0x55c331bfbaf0 2024-09-19T06:54:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.763+0000 7f6fe3fff640 1 -- 172.21.15.103:0/2070684847 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff40041d0 con 0x55c331bfbaf0 2024-09-19T06:54:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.763+0000 7f6fe3fff640 1 -- 172.21.15.103:0/2070684847 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f70000c4e80 con 0x55c331bfbaf0 2024-09-19T06:54:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.763+0000 7f700dbd6740 10 client.17556 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.763+0000 7f700dbd6740 10 client.17556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.763+0000 7f700dbd6740 20 client.17556 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.763+0000 7f700dbd6740 1 --2- 172.21.15.103:0/2070684847 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c331c16c40 0x55c331c190b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.763+0000 7f700dbd6740 4 client.17556 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.763+0000 7f700dbd6740 1 -- 172.21.15.103:0/2070684847 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c331b6da20 con 0x55c331c16c40 2024-09-19T06:54:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.764+0000 7f6ffbfff640 1 --2- 172.21.15.103:0/2070684847 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c331c16c40 0x55c331c190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.764+0000 7f6ffbfff640 1 --2- 172.21.15.103:0/2070684847 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c331c16c40 0x55c331c190b0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f37e760798f7c091 server_cookie=9dc0e52a4f096b25 in_seq=0 out_seq=0 2024-09-19T06:54:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.764+0000 7f6fe3fff640 10 client.17556 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.765+0000 7f6fe3fff640 1 -- 172.21.15.103:0/2070684847 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c331b6da20 con 0x55c331c16c40 2024-09-19T06:54:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.765+0000 7f6fe3fff640 10 client.17556 handle_command_reply: tid=0 2024-09-19T06:54:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.765+0000 7f700dbd6740 1 client.17556 shutdown 2024-09-19T06:54:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.765+0000 7f700dbd6740 1 -- 172.21.15.103:0/2070684847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfbaf0 msgr2=0x55c331c03b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.765+0000 7f700dbd6740 1 --2- 172.21.15.103:0/2070684847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfbaf0 0x55c331c03b90 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f70000b8b00 tx=0x7f70000b8940 comp rx=0 tx=0).stop 2024-09-19T06:54:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.765+0000 7f700dbd6740 1 -- 172.21.15.103:0/2070684847 shutdown_connections 2024-09-19T06:54:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.765+0000 7f700dbd6740 1 --2- 172.21.15.103:0/2070684847 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c331c16c40 0x55c331c190b0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.765+0000 7f700dbd6740 1 --2- 172.21.15.103:0/2070684847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331c06cb0 0x55c331c04610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.765+0000 7f6fe3fff640 0 client.17556 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.765+0000 7f700dbd6740 1 --2- 172.21.15.103:0/2070684847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331bfc410 0x55c331c040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.765+0000 7f700dbd6740 1 --2- 172.21.15.103:0/2070684847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfbaf0 0x55c331c03b90 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.765+0000 7f700dbd6740 1 -- 172.21.15.103:0/2070684847 >> 172.21.15.103:0/2070684847 conn(0x55c331bf8060 msgr2=0x55c331bfdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.765+0000 7f700dbd6740 1 -- 172.21.15.103:0/2070684847 shutdown_connections 2024-09-19T06:54:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.765+0000 7f700dbd6740 1 -- 172.21.15.103:0/2070684847 wait complete. 2024-09-19T06:54:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.765+0000 7f700dbd6740 20 client.17556 trim_cache size 0 max 16384 2024-09-19T06:54:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.779+0000 7f700dbd6740 1 Processor -- start 2024-09-19T06:54:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.779+0000 7f700dbd6740 1 -- start start 2024-09-19T06:54:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.779+0000 7f700dbd6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331bfbaf0 0x55c331c04440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.779+0000 7f700dbd6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c1e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.779+0000 7f700dbd6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331c06cb0 0x55c331c1eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.779+0000 7f700dbd6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70041ad720 con 0x55c331bfc410 2024-09-19T06:54:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.779+0000 7f700dbd6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70041af260 con 0x55c331bfbaf0 2024-09-19T06:54:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.779+0000 7f700dbd6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70041addf0 con 0x55c331c06cb0 2024-09-19T06:54:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.780+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c1e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.780+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c1e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34906/0 (socket says 172.21.15.103:34906) 2024-09-19T06:54:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.780+0000 7f6ffbfff640 1 -- 172.21.15.103:0/866152262 learned_addr learned my addr 172.21.15.103:0/866152262 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.780+0000 7f7008bd8640 1 --2- 172.21.15.103:0/866152262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331bfbaf0 0x55c331c04440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.780+0000 7f70093d9640 1 --2- 172.21.15.103:0/866152262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331c06cb0 0x55c331c1eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.780+0000 7f6ffbfff640 1 -- 172.21.15.103:0/866152262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331c06cb0 msgr2=0x55c331c1eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.780+0000 7f6ffbfff640 1 --2- 172.21.15.103:0/866152262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331c06cb0 0x55c331c1eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.780+0000 7f6ffbfff640 1 -- 172.21.15.103:0/866152262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331bfbaf0 msgr2=0x55c331c04440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.780+0000 7f6ffbfff640 1 --2- 172.21.15.103:0/866152262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331bfbaf0 0x55c331c04440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.780+0000 7f6ffbfff640 1 -- 172.21.15.103:0/866152262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c331bfb870 con 0x55c331bfc410 2024-09-19T06:54:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.780+0000 7f6ffbfff640 1 --2- 172.21.15.103:0/866152262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c1e990 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400bca0 tx=0x7f6ff400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd49e03a25425a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.780+0000 7f6fea7fc640 1 -- 172.21.15.103:0/866152262 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff40037f0 con 0x55c331bfc410 2024-09-19T06:54:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.780+0000 7f6fea7fc640 1 -- 172.21.15.103:0/866152262 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff4003e50 con 0x55c331bfc410 2024-09-19T06:54:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.780+0000 7f6fea7fc640 1 -- 172.21.15.103:0/866152262 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4028430 con 0x55c331bfc410 2024-09-19T06:54:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.781+0000 7f700dbd6740 1 -- 172.21.15.103:0/866152262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 msgr2=0x55c331c1e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.781+0000 7f700dbd6740 1 --2- 172.21.15.103:0/866152262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c1e990 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400bca0 tx=0x7f6ff400be60 comp rx=0 tx=0).stop 2024-09-19T06:54:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.781+0000 7f700dbd6740 1 -- 172.21.15.103:0/866152262 shutdown_connections 2024-09-19T06:54:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.781+0000 7f700dbd6740 1 --2- 172.21.15.103:0/866152262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331c06cb0 0x55c331c1eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.781+0000 7f700dbd6740 1 --2- 172.21.15.103:0/866152262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c1e990 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.781+0000 7f700dbd6740 1 --2- 172.21.15.103:0/866152262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331bfbaf0 0x55c331c04440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.781+0000 7f700dbd6740 1 -- 172.21.15.103:0/866152262 >> 172.21.15.103:0/866152262 conn(0x55c331bf8060 msgr2=0x55c331c05c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.781+0000 7f700dbd6740 1 -- 172.21.15.103:0/866152262 shutdown_connections 2024-09-19T06:54:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.781+0000 7f700dbd6740 1 -- 172.21.15.103:0/866152262 wait complete. 2024-09-19T06:54:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.781+0000 7f700dbd6740 1 -- start start 2024-09-19T06:54:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.781+0000 7f700dbd6740 1 Processor -- start 2024-09-19T06:54:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f700dbd6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331bfbaf0 0x55c331c0f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f700dbd6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c0a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f700dbd6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331c06cb0 0x55c331c0b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f700dbd6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70041aeb90 con 0x55c331bfc410 2024-09-19T06:54:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f700dbd6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70041ae4c0 con 0x55c331c06cb0 2024-09-19T06:54:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f700dbd6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70041ab6e0 con 0x55c331bfbaf0 2024-09-19T06:54:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f700dbd6740 10 client.0 _ll_register_callbacks cb 0x7f70000b8280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c0a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c0a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34918/0 (socket says 172.21.15.103:34918) 2024-09-19T06:54:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f6ffbfff640 1 -- 172.21.15.103:0/377093080 learned_addr learned my addr 172.21.15.103:0/377093080 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:54:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f7008bd8640 1 --2- 172.21.15.103:0/377093080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331bfbaf0 0x55c331c0f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f6ffbfff640 1 -- 172.21.15.103:0/377093080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331bfbaf0 msgr2=0x55c331c0f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f6ffbfff640 1 --2- 172.21.15.103:0/377093080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331bfbaf0 0x55c331c0f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f6ffbfff640 1 -- 172.21.15.103:0/377093080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331c06cb0 msgr2=0x55c331c0b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f6ffbfff640 1 --2- 172.21.15.103:0/377093080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331c06cb0 0x55c331c0b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f6ffbfff640 1 -- 172.21.15.103:0/377093080 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c331c80960 con 0x55c331bfc410 2024-09-19T06:54:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f6ffbfff640 1 --2- 172.21.15.103:0/377093080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c0a8a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400b570 tx=0x7f6ff4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a5046cb0f779da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f6fe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:54:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f6fe3fff640 1 -- 172.21.15.103:0/377093080 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4036020 con 0x55c331bfc410 2024-09-19T06:54:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f6fe3fff640 1 -- 172.21.15.103:0/377093080 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f6ff4005050 con 0x55c331bfc410 2024-09-19T06:54:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.782+0000 7f700dbd6740 1 -- 172.21.15.103:0/377093080 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ff4003e50 con 0x55c331bfc410 2024-09-19T06:54:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.783+0000 7f6fe3fff640 1 -- 172.21.15.103:0/377093080 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6ff4005250 con 0x55c331bfc410 2024-09-19T06:54:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.783+0000 7f700dbd6740 10 client.17562 fetch_fsmap learned FSMap version 27 2024-09-19T06:54:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.783+0000 7f700dbd6740 1 -- 172.21.15.103:0/377093080 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c331c81af0 con 0x55c331bfc410 2024-09-19T06:54:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.783+0000 7f6fe3fff640 1 -- 172.21.15.103:0/377093080 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff4003e50 con 0x55c331bfc410 2024-09-19T06:54:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.783+0000 7f6fe3fff640 1 -- 172.21.15.103:0/377093080 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6ff40289d0 con 0x55c331bfc410 2024-09-19T06:54:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.783+0000 7f700dbd6740 10 client.17562 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:54:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.783+0000 7f700dbd6740 10 client.17562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.783+0000 7f700dbd6740 20 client.17562 populate_metadata read hostname 'smithi103' 2024-09-19T06:54:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.783+0000 7f700dbd6740 1 --2- 172.21.15.103:0/377093080 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c331c82030 0x55c331c84450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.783+0000 7f700dbd6740 4 client.17562 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:54:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.783+0000 7f700dbd6740 1 -- 172.21.15.103:0/377093080 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c331b67d70 con 0x55c331c82030 2024-09-19T06:54:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.783+0000 7f7008bd8640 1 --2- 172.21.15.103:0/377093080 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c331c82030 0x55c331c84450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.784+0000 7f6fe3fff640 10 client.17562 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.784+0000 7f7008bd8640 1 --2- 172.21.15.103:0/377093080 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c331c82030 0x55c331c84450 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ed7e10202c5b88 server_cookie=e9a20317706dad14 in_seq=0 out_seq=0 2024-09-19T06:54:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.796+0000 7f6fe3fff640 1 -- 172.21.15.103:0/377093080 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c331b67d70 con 0x55c331c82030 2024-09-19T06:54:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.796+0000 7f6fe3fff640 10 client.17562 handle_command_reply: tid=0 2024-09-19T06:54:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.796+0000 7f700dbd6740 1 client.17562 shutdown 2024-09-19T06:54:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.796+0000 7f700dbd6740 1 -- 172.21.15.103:0/377093080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 msgr2=0x55c331c0a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.796+0000 7f700dbd6740 1 --2- 172.21.15.103:0/377093080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c0a8a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400b570 tx=0x7f6ff4005b00 comp rx=0 tx=0).stop 2024-09-19T06:54:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.796+0000 7f700dbd6740 1 -- 172.21.15.103:0/377093080 shutdown_connections 2024-09-19T06:54:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.796+0000 7f700dbd6740 1 --2- 172.21.15.103:0/377093080 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c331c82030 0x55c331c84450 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.796+0000 7f700dbd6740 1 --2- 172.21.15.103:0/377093080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c331c06cb0 0x55c331c0b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.796+0000 7f700dbd6740 1 --2- 172.21.15.103:0/377093080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c331bfc410 0x55c331c0a8a0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.796+0000 7f700dbd6740 1 --2- 172.21.15.103:0/377093080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c331bfbaf0 0x55c331c0f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.796+0000 7f6fe3fff640 0 client.17562 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:54:53.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.796+0000 7f700dbd6740 1 -- 172.21.15.103:0/377093080 >> 172.21.15.103:0/377093080 conn(0x55c331bf8060 msgr2=0x55c331bfd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.797+0000 7f700dbd6740 1 -- 172.21.15.103:0/377093080 shutdown_connections 2024-09-19T06:54:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.797+0000 7f700dbd6740 1 -- 172.21.15.103:0/377093080 wait complete. 2024-09-19T06:54:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.797+0000 7f700dbd6740 20 client.17562 trim_cache size 0 max 16384 2024-09-19T06:54:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:54:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:54:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:54:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:54:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.797+0000 7f700ae3c640 1 -- 172.21.15.103:0/789367024 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6fec0a3460 msgr2=0x7f6fec0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.797+0000 7f700ae3c640 1 --2- 172.21.15.103:0/789367024 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6fec0a3460 0x7f6fec0a5920 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0094d0 tx=0x7f6ffc013b20 comp rx=0 tx=0).stop 2024-09-19T06:54:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.797+0000 7f700ae3c640 1 -- 172.21.15.103:0/789367024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7004102cc0 msgr2=0x7f7004198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.797+0000 7f700ae3c640 1 --2- 172.21.15.103:0/789367024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7004102cc0 0x7f7004198330 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7000006010 tx=0x7f70000042f0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.797+0000 7f700ae3c640 1 -- 172.21.15.103:0/789367024 shutdown_connections 2024-09-19T06:54:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.797+0000 7f700ae3c640 1 --2- 172.21.15.103:0/789367024 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6fec0a3460 0x7f6fec0a5920 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.797+0000 7f700ae3c640 1 --2- 172.21.15.103:0/789367024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f700410c110 0x7f700419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.797+0000 7f700ae3c640 1 --2- 172.21.15.103:0/789367024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70041035e0 0x7f7004198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.797+0000 7f700ae3c640 1 --2- 172.21.15.103:0/789367024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7004102cc0 0x7f7004198330 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.797+0000 7f700ae3c640 1 -- 172.21.15.103:0/789367024 >> 172.21.15.103:0/789367024 conn(0x7f70040fc6b0 msgr2=0x7f70040fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.797+0000 7f700ae3c640 1 -- 172.21.15.103:0/789367024 shutdown_connections 2024-09-19T06:54:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:54:53.797+0000 7f700ae3c640 1 -- 172.21.15.103:0/789367024 wait complete. 2024-09-19T06:54:54.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:54 smithi103 ceph-mon[31158]: pgmap v173: 97 pgs: 97 active+clean; 5.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-09-19T06:54:54.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:54 smithi134 ceph-mon[35356]: pgmap v173: 97 pgs: 97 active+clean; 5.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-09-19T06:54:54.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:54 smithi174 ceph-mon[35637]: pgmap v173: 97 pgs: 97 active+clean; 5.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-09-19T06:54:56.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:56 smithi103 ceph-mon[31158]: pgmap v174: 97 pgs: 97 active+clean; 5.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-09-19T06:54:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:56 smithi134 ceph-mon[35356]: pgmap v174: 97 pgs: 97 active+clean; 5.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-09-19T06:54:56.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:56 smithi174 ceph-mon[35637]: pgmap v174: 97 pgs: 97 active+clean; 5.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-09-19T06:54:58.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:54:58 smithi103 ceph-mon[31158]: pgmap v175: 97 pgs: 97 active+clean; 5.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-09-19T06:54:58.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:54:58 smithi134 ceph-mon[35356]: pgmap v175: 97 pgs: 97 active+clean; 5.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-09-19T06:54:58.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:54:58 smithi174 ceph-mon[35637]: pgmap v175: 97 pgs: 97 active+clean; 5.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-09-19T06:55:00.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:00 smithi103 ceph-mon[31158]: pgmap v176: 97 pgs: 97 active+clean; 5.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 23 op/s 2024-09-19T06:55:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:00 smithi134 ceph-mon[35356]: pgmap v176: 97 pgs: 97 active+clean; 5.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 23 op/s 2024-09-19T06:55:00.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:00 smithi174 ceph-mon[35637]: pgmap v176: 97 pgs: 97 active+clean; 5.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 23 op/s 2024-09-19T06:55:02.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:02 smithi103 ceph-mon[31158]: pgmap v177: 97 pgs: 97 active+clean; 5.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-19T06:55:02.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:02 smithi134 ceph-mon[35356]: pgmap v177: 97 pgs: 97 active+clean; 5.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-19T06:55:02.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:02 smithi174 ceph-mon[35637]: pgmap v177: 97 pgs: 97 active+clean; 5.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-19T06:55:04.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:04 smithi103 ceph-mon[31158]: pgmap v178: 97 pgs: 97 active+clean; 5.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-19T06:55:04.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:04 smithi134 ceph-mon[35356]: pgmap v178: 97 pgs: 97 active+clean; 5.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-19T06:55:04.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:04 smithi174 ceph-mon[35637]: pgmap v178: 97 pgs: 97 active+clean; 5.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-19T06:55:06.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:06 smithi103 ceph-mon[31158]: pgmap v179: 97 pgs: 97 active+clean; 5.5 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 73 KiB/s wr, 37 op/s 2024-09-19T06:55:06.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:06 smithi134 ceph-mon[35356]: pgmap v179: 97 pgs: 97 active+clean; 5.5 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 73 KiB/s wr, 37 op/s 2024-09-19T06:55:06.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:06 smithi174 ceph-mon[35637]: pgmap v179: 97 pgs: 97 active+clean; 5.5 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 73 KiB/s wr, 37 op/s 2024-09-19T06:55:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.897+0000 7f80507d8640 1 -- 172.21.15.103:0/1650323968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804810e470 msgr2=0x7f8048110860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.897+0000 7f80507d8640 1 --2- 172.21.15.103:0/1650323968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804810e470 0x7f8048110860 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f8040009ca0 tx=0x7f804002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.897+0000 7f80507d8640 1 -- 172.21.15.103:0/1650323968 shutdown_connections 2024-09-19T06:55:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.897+0000 7f80507d8640 1 --2- 172.21.15.103:0/1650323968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804810e470 0x7f8048110860 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.897+0000 7f80507d8640 1 --2- 172.21.15.103:0/1650323968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8048101270 0x7f804810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.897+0000 7f80507d8640 1 --2- 172.21.15.103:0/1650323968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80481008c0 0x7f8048100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.897+0000 7f80507d8640 1 -- 172.21.15.103:0/1650323968 >> 172.21.15.103:0/1650323968 conn(0x7f80480fc6b0 msgr2=0x7f80480fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.897+0000 7f80507d8640 1 -- 172.21.15.103:0/1650323968 shutdown_connections 2024-09-19T06:55:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.897+0000 7f80507d8640 1 -- 172.21.15.103:0/1650323968 wait complete. 2024-09-19T06:55:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.898+0000 7f80507d8640 1 Processor -- start 2024-09-19T06:55:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.898+0000 7f80507d8640 1 -- start start 2024-09-19T06:55:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.898+0000 7f80507d8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80481008c0 0x7f804819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.898+0000 7f80507d8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8048101270 0x7f804819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.898+0000 7f80507d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804810e470 0x7f804819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.898+0000 7f80507d8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80481140f0 con 0x7f804810e470 2024-09-19T06:55:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.898+0000 7f80507d8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8048113f70 con 0x7f8048101270 2024-09-19T06:55:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.898+0000 7f80507d8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8048114270 con 0x7f80481008c0 2024-09-19T06:55:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.898+0000 7f804ed75640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804810e470 0x7f804819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.898+0000 7f804ed75640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804810e470 0x7f804819ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35748/0 (socket says 172.21.15.103:35748) 2024-09-19T06:55:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.898+0000 7f804ed75640 1 -- 172.21.15.103:0/1734539383 learned_addr learned my addr 172.21.15.103:0/1734539383 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.899+0000 7f804e574640 1 --2- 172.21.15.103:0/1734539383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80481008c0 0x7f804819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.899+0000 7f804dd73640 1 --2- 172.21.15.103:0/1734539383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8048101270 0x7f804819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.899+0000 7f804ed75640 1 -- 172.21.15.103:0/1734539383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80481008c0 msgr2=0x7f804819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.899+0000 7f804ed75640 1 --2- 172.21.15.103:0/1734539383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80481008c0 0x7f804819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.899+0000 7f804ed75640 1 -- 172.21.15.103:0/1734539383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8048101270 msgr2=0x7f804819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.899+0000 7f804ed75640 1 --2- 172.21.15.103:0/1734539383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8048101270 0x7f804819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.899+0000 7f804ed75640 1 -- 172.21.15.103:0/1734539383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804819f570 con 0x7f804810e470 2024-09-19T06:55:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.899+0000 7f804ed75640 1 --2- 172.21.15.103:0/1734539383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804810e470 0x7f804819ee40 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f804002ade0 tx=0x7f80400319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.899+0000 7f803f7fe640 1 -- 172.21.15.103:0/1734539383 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040003c30 con 0x7f804810e470 2024-09-19T06:55:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.899+0000 7f803f7fe640 1 -- 172.21.15.103:0/1734539383 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8040003dd0 con 0x7f804810e470 2024-09-19T06:55:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.899+0000 7f803f7fe640 1 -- 172.21.15.103:0/1734539383 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804002fd60 con 0x7f804810e470 2024-09-19T06:55:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.899+0000 7f80507d8640 1 -- 172.21.15.103:0/1734539383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804819f800 con 0x7f804810e470 2024-09-19T06:55:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.899+0000 7f80507d8640 1 -- 172.21.15.103:0/1734539383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80481a70f0 con 0x7f804810e470 2024-09-19T06:55:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.902+0000 7f803f7fe640 1 -- 172.21.15.103:0/1734539383 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8040004050 con 0x7f804810e470 2024-09-19T06:55:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.902+0000 7f805364c740 1 Processor -- start 2024-09-19T06:55:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f805364c740 1 -- start start 2024-09-19T06:55:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f805364c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8caf0 0x56193db8ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f805364c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db8d410 0x56193db97770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f805364c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db97cb0 0x56193db9a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f805364c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56193daf12f0 con 0x56193db97cb0 2024-09-19T06:55:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f805364c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56193da99270 con 0x56193db8caf0 2024-09-19T06:55:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f805364c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56193daed2b0 con 0x56193db8d410 2024-09-19T06:55:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f804dd73640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db97cb0 0x56193db9a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f804dd73640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db97cb0 0x56193db9a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35752/0 (socket says 172.21.15.103:35752) 2024-09-19T06:55:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f804dd73640 1 -- 172.21.15.103:0/2410981525 learned_addr learned my addr 172.21.15.103:0/2410981525 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f804e574640 1 --2- 172.21.15.103:0/2410981525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db8d410 0x56193db97770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f804dd73640 1 --2- 172.21.15.103:0/2410981525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8caf0 0x56193db8ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f803f7fe640 1 --2- 172.21.15.103:0/1734539383 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80340a34e0 0x7f80340a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f804dd73640 1 -- 172.21.15.103:0/2410981525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db8d410 msgr2=0x56193db97770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f804dd73640 1 --2- 172.21.15.103:0/2410981525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db8d410 0x56193db97770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f804dd73640 1 -- 172.21.15.103:0/2410981525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8caf0 msgr2=0x56193db8ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f804dd73640 1 --2- 172.21.15.103:0/2410981525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8caf0 0x56193db8ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f804dd73640 1 -- 172.21.15.103:0/2410981525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56193da98340 con 0x56193db97cb0 2024-09-19T06:55:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.903+0000 7f804ed75640 1 --2- 172.21.15.103:0/1734539383 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80340a34e0 0x7f80340a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.904+0000 7f803f7fe640 1 -- 172.21.15.103:0/1734539383 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f80400f0fd0 con 0x7f804810e470 2024-09-19T06:55:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.904+0000 7f804dd73640 1 --2- 172.21.15.103:0/2410981525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db97cb0 0x56193db9a0a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f80380116c0 tx=0x7f803800a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84d8719d28eda4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.904+0000 7f803cff9640 1 -- 172.21.15.103:0/2410981525 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038011260 con 0x56193db97cb0 2024-09-19T06:55:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.904+0000 7f803cff9640 1 -- 172.21.15.103:0/2410981525 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8038005020 con 0x56193db97cb0 2024-09-19T06:55:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.904+0000 7f803cff9640 1 -- 172.21.15.103:0/2410981525 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038004a00 con 0x56193db97cb0 2024-09-19T06:55:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 -- 172.21.15.103:0/2410981525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db97cb0 msgr2=0x56193db9a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f804ed75640 1 --2- 172.21.15.103:0/1734539383 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80340a34e0 0x7f80340a59a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f80400b99e0 tx=0x7f804008f420 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 --2- 172.21.15.103:0/2410981525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db97cb0 0x56193db9a0a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f80380116c0 tx=0x7f803800a6d0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 -- 172.21.15.103:0/2410981525 shutdown_connections 2024-09-19T06:55:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 --2- 172.21.15.103:0/2410981525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db97cb0 0x56193db9a0a0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 --2- 172.21.15.103:0/2410981525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db8d410 0x56193db97770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 --2- 172.21.15.103:0/2410981525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8caf0 0x56193db8ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 -- 172.21.15.103:0/2410981525 >> 172.21.15.103:0/2410981525 conn(0x56193db89060 msgr2=0x56193db89440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 -- 172.21.15.103:0/2410981525 shutdown_connections 2024-09-19T06:55:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 -- 172.21.15.103:0/2410981525 wait complete. 2024-09-19T06:55:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 -- start start 2024-09-19T06:55:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 Processor -- start 2024-09-19T06:55:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db8caf0 0x56193db94be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8d410 0x56193db95120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db97cb0 0x56193db95660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56193db9a880 con 0x56193db8caf0 2024-09-19T06:55:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56193da7a290 con 0x56193db8d410 2024-09-19T06:55:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.905+0000 7f805364c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56193db9aa00 con 0x56193db97cb0 2024-09-19T06:55:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f805364c740 10 client.0 _ll_register_callbacks cb 0x56193daab140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f804dd73640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db8caf0 0x56193db94be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f804dd73640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db8caf0 0x56193db94be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35760/0 (socket says 172.21.15.103:35760) 2024-09-19T06:55:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f804dd73640 1 -- 172.21.15.103:0/1058037095 learned_addr learned my addr 172.21.15.103:0/1058037095 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f804e574640 1 --2- 172.21.15.103:0/1058037095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8d410 0x56193db95120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f804dd73640 1 -- 172.21.15.103:0/1058037095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db97cb0 msgr2=0x56193db95660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f804dd73640 1 --2- 172.21.15.103:0/1058037095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db97cb0 0x56193db95660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f804dd73640 1 -- 172.21.15.103:0/1058037095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8d410 msgr2=0x56193db95120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f804dd73640 1 --2- 172.21.15.103:0/1058037095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8d410 0x56193db95120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f804dd73640 1 -- 172.21.15.103:0/1058037095 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56193d804210 con 0x56193db8caf0 2024-09-19T06:55:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f804dd73640 1 --2- 172.21.15.103:0/1058037095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db8caf0 0x56193db94be0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f8038011690 tx=0x7f8038005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1aeb5b303b4e3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f8031ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f8031ffb640 1 -- 172.21.15.103:0/1058037095 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803800fc60 con 0x56193db8caf0 2024-09-19T06:55:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f805364c740 1 -- 172.21.15.103:0/1058037095 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8038005020 con 0x56193db8caf0 2024-09-19T06:55:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f8031ffb640 1 -- 172.21.15.103:0/1058037095 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f803800fe00 con 0x56193db8caf0 2024-09-19T06:55:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f8031ffb640 1 -- 172.21.15.103:0/1058037095 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80380046c0 con 0x56193db8caf0 2024-09-19T06:55:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f805364c740 10 client.17574 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.906+0000 7f805364c740 1 -- 172.21.15.103:0/1058037095 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56193d8d0620 con 0x56193db8caf0 2024-09-19T06:55:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.907+0000 7f8031ffb640 1 -- 172.21.15.103:0/1058037095 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8038005020 con 0x56193db8caf0 2024-09-19T06:55:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.907+0000 7f8031ffb640 1 -- 172.21.15.103:0/1058037095 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8038034aa0 con 0x56193db8caf0 2024-09-19T06:55:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.907+0000 7f805364c740 10 client.17574 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.907+0000 7f805364c740 10 client.17574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.907+0000 7f805364c740 20 client.17574 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.907+0000 7f805364c740 1 --2- 172.21.15.103:0/1058037095 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56193dba7b60 0x56193dba9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.907+0000 7f805364c740 4 client.17574 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.907+0000 7f805364c740 1 -- 172.21.15.103:0/1058037095 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56193dafea20 con 0x56193dba7b60 2024-09-19T06:55:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.907+0000 7f804e574640 1 --2- 172.21.15.103:0/1058037095 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56193dba7b60 0x56193dba9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.907+0000 7f804e574640 1 --2- 172.21.15.103:0/1058037095 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56193dba7b60 0x56193dba9f50 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c73e6659066c7d97 server_cookie=eeafe3030005eed6 in_seq=0 out_seq=0 2024-09-19T06:55:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.907+0000 7f8031ffb640 10 client.17574 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.908+0000 7f8031ffb640 1 -- 172.21.15.103:0/1058037095 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56193dafea20 con 0x56193dba7b60 2024-09-19T06:55:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.908+0000 7f8031ffb640 10 client.17574 handle_command_reply: tid=0 2024-09-19T06:55:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.908+0000 7f805364c740 1 client.17574 shutdown 2024-09-19T06:55:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.908+0000 7f805364c740 1 -- 172.21.15.103:0/1058037095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db8caf0 msgr2=0x56193db94be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.908+0000 7f805364c740 1 --2- 172.21.15.103:0/1058037095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db8caf0 0x56193db94be0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f8038011690 tx=0x7f8038005da0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.908+0000 7f805364c740 1 -- 172.21.15.103:0/1058037095 shutdown_connections 2024-09-19T06:55:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.908+0000 7f805364c740 1 --2- 172.21.15.103:0/1058037095 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56193dba7b60 0x56193dba9f50 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.908+0000 7f805364c740 1 --2- 172.21.15.103:0/1058037095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db97cb0 0x56193db95660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.908+0000 7f805364c740 1 --2- 172.21.15.103:0/1058037095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8d410 0x56193db95120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.908+0000 7f8031ffb640 0 client.17574 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.908+0000 7f805364c740 1 --2- 172.21.15.103:0/1058037095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db8caf0 0x56193db94be0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.908+0000 7f805364c740 1 -- 172.21.15.103:0/1058037095 >> 172.21.15.103:0/1058037095 conn(0x56193db89060 msgr2=0x56193db8ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.908+0000 7f805364c740 1 -- 172.21.15.103:0/1058037095 shutdown_connections 2024-09-19T06:55:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.908+0000 7f805364c740 1 -- 172.21.15.103:0/1058037095 wait complete. 2024-09-19T06:55:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.908+0000 7f805364c740 20 client.17574 trim_cache size 0 max 16384 2024-09-19T06:55:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f805364c740 1 Processor -- start 2024-09-19T06:55:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f805364c740 1 -- start start 2024-09-19T06:55:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f805364c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db8caf0 0x56193db93490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f805364c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8d410 0x56193db95490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f805364c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db97cb0 0x56193dbaf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f805364c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80481af9b0 con 0x56193db8caf0 2024-09-19T06:55:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f805364c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f80481b14f0 con 0x56193db8d410 2024-09-19T06:55:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f805364c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80481b0080 con 0x56193db97cb0 2024-09-19T06:55:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f804dd73640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db8caf0 0x56193db93490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f804dd73640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db8caf0 0x56193db93490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35770/0 (socket says 172.21.15.103:35770) 2024-09-19T06:55:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f804dd73640 1 -- 172.21.15.103:0/4265210509 learned_addr learned my addr 172.21.15.103:0/4265210509 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f804dd73640 1 -- 172.21.15.103:0/4265210509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db97cb0 msgr2=0x56193dbaf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f804dd73640 1 --2- 172.21.15.103:0/4265210509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db97cb0 0x56193dbaf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f804dd73640 1 -- 172.21.15.103:0/4265210509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8d410 msgr2=0x56193db95490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f804dd73640 1 --2- 172.21.15.103:0/4265210509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8d410 0x56193db95490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f804dd73640 1 -- 172.21.15.103:0/4265210509 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56193db88e90 con 0x56193db8caf0 2024-09-19T06:55:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.923+0000 7f804dd73640 1 --2- 172.21.15.103:0/4265210509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db8caf0 0x56193db93490 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f8038005c90 tx=0x7f8038011620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec24cb08968bce51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.924+0000 7f8033fff640 1 -- 172.21.15.103:0/4265210509 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038039070 con 0x56193db8caf0 2024-09-19T06:55:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.924+0000 7f8033fff640 1 -- 172.21.15.103:0/4265210509 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8038021040 con 0x56193db8caf0 2024-09-19T06:55:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.924+0000 7f8033fff640 1 -- 172.21.15.103:0/4265210509 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038004280 con 0x56193db8caf0 2024-09-19T06:55:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.924+0000 7f805364c740 1 -- 172.21.15.103:0/4265210509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db8caf0 msgr2=0x56193db93490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.924+0000 7f805364c740 1 --2- 172.21.15.103:0/4265210509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db8caf0 0x56193db93490 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f8038005c90 tx=0x7f8038011620 comp rx=0 tx=0).stop 2024-09-19T06:55:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.924+0000 7f805364c740 1 -- 172.21.15.103:0/4265210509 shutdown_connections 2024-09-19T06:55:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.924+0000 7f805364c740 1 --2- 172.21.15.103:0/4265210509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db97cb0 0x56193dbaf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.924+0000 7f805364c740 1 --2- 172.21.15.103:0/4265210509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8d410 0x56193db95490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.924+0000 7f805364c740 1 --2- 172.21.15.103:0/4265210509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db8caf0 0x56193db93490 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.924+0000 7f805364c740 1 -- 172.21.15.103:0/4265210509 >> 172.21.15.103:0/4265210509 conn(0x56193db89060 msgr2=0x56193db96c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.924+0000 7f805364c740 1 -- 172.21.15.103:0/4265210509 shutdown_connections 2024-09-19T06:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.924+0000 7f805364c740 1 -- 172.21.15.103:0/4265210509 wait complete. 2024-09-19T06:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f805364c740 1 -- start start 2024-09-19T06:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f805364c740 1 Processor -- start 2024-09-19T06:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f805364c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8caf0 0x56193db9ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f805364c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db8d410 0x56193dba0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f805364c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db97cb0 0x56193db91410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f805364c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80481b0e20 con 0x56193db97cb0 2024-09-19T06:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f805364c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f80481b0750 con 0x56193db8caf0 2024-09-19T06:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f805364c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80481ad970 con 0x56193db8d410 2024-09-19T06:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f805364c740 10 client.0 _ll_register_callbacks cb 0x7f803800ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f804dd73640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db97cb0 0x56193db91410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f804dd73640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db97cb0 0x56193db91410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35776/0 (socket says 172.21.15.103:35776) 2024-09-19T06:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f804dd73640 1 -- 172.21.15.103:0/1847191100 learned_addr learned my addr 172.21.15.103:0/1847191100 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f804e574640 1 --2- 172.21.15.103:0/1847191100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db8d410 0x56193dba0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f804dd73640 1 --2- 172.21.15.103:0/1847191100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8caf0 0x56193db9ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f804dd73640 1 -- 172.21.15.103:0/1847191100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db8d410 msgr2=0x56193dba0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f804dd73640 1 --2- 172.21.15.103:0/1847191100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db8d410 0x56193dba0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f804dd73640 1 -- 172.21.15.103:0/1847191100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8caf0 msgr2=0x56193db9ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f804dd73640 1 --2- 172.21.15.103:0/1847191100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8caf0 0x56193db9ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f804dd73640 1 -- 172.21.15.103:0/1847191100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56193db91c10 con 0x56193db97cb0 2024-09-19T06:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f804dd73640 1 --2- 172.21.15.103:0/1847191100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db97cb0 0x56193db91410 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8038005940 tx=0x7f803802d250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb140b43d5921c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.925+0000 7f8031ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.926+0000 7f8031ffb640 1 -- 172.21.15.103:0/1847191100 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038039070 con 0x56193db97cb0 2024-09-19T06:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.926+0000 7f8031ffb640 1 -- 172.21.15.103:0/1847191100 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f803800fc60 con 0x56193db97cb0 2024-09-19T06:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.926+0000 7f805364c740 1 -- 172.21.15.103:0/1847191100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8038021040 con 0x56193db97cb0 2024-09-19T06:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.926+0000 7f8031ffb640 1 -- 172.21.15.103:0/1847191100 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80380054e0 con 0x56193db97cb0 2024-09-19T06:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.926+0000 7f805364c740 10 client.17580 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.926+0000 7f805364c740 1 -- 172.21.15.103:0/1847191100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56193dc12150 con 0x56193db97cb0 2024-09-19T06:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.926+0000 7f8031ffb640 1 -- 172.21.15.103:0/1847191100 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8038021040 con 0x56193db97cb0 2024-09-19T06:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.926+0000 7f8031ffb640 1 -- 172.21.15.103:0/1847191100 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8038048050 con 0x56193db97cb0 2024-09-19T06:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.926+0000 7f805364c740 10 client.17580 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.926+0000 7f805364c740 10 client.17580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.926+0000 7f805364c740 20 client.17580 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.926+0000 7f805364c740 1 --2- 172.21.15.103:0/1847191100 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56193dc12660 0x56193dc14a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.926+0000 7f805364c740 4 client.17580 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.926+0000 7f805364c740 1 -- 172.21.15.103:0/1847191100 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56193daf8d70 con 0x56193dc12660 2024-09-19T06:55:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.926+0000 7f804e574640 1 --2- 172.21.15.103:0/1847191100 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56193dc12660 0x56193dc14a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.927+0000 7f8031ffb640 10 client.17580 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.927+0000 7f804e574640 1 --2- 172.21.15.103:0/1847191100 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56193dc12660 0x56193dc14a80 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25c0574e5646d091 server_cookie=266703f450cdab38 in_seq=0 out_seq=0 2024-09-19T06:55:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.939+0000 7f8031ffb640 1 -- 172.21.15.103:0/1847191100 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56193daf8d70 con 0x56193dc12660 2024-09-19T06:55:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.939+0000 7f8031ffb640 10 client.17580 handle_command_reply: tid=0 2024-09-19T06:55:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.939+0000 7f805364c740 1 client.17580 shutdown 2024-09-19T06:55:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.939+0000 7f805364c740 1 -- 172.21.15.103:0/1847191100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db97cb0 msgr2=0x56193db91410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.939+0000 7f805364c740 1 --2- 172.21.15.103:0/1847191100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db97cb0 0x56193db91410 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8038005940 tx=0x7f803802d250 comp rx=0 tx=0).stop 2024-09-19T06:55:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.939+0000 7f805364c740 1 -- 172.21.15.103:0/1847191100 shutdown_connections 2024-09-19T06:55:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.939+0000 7f805364c740 1 --2- 172.21.15.103:0/1847191100 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56193dc12660 0x56193dc14a80 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.939+0000 7f805364c740 1 --2- 172.21.15.103:0/1847191100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56193db97cb0 0x56193db91410 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.939+0000 7f805364c740 1 --2- 172.21.15.103:0/1847191100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193db8d410 0x56193dba0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.939+0000 7f8031ffb640 0 client.17580 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.939+0000 7f805364c740 1 --2- 172.21.15.103:0/1847191100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56193db8caf0 0x56193db9ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.939+0000 7f805364c740 1 -- 172.21.15.103:0/1847191100 >> 172.21.15.103:0/1847191100 conn(0x56193db89060 msgr2=0x56193db96440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.939+0000 7f805364c740 1 -- 172.21.15.103:0/1847191100 shutdown_connections 2024-09-19T06:55:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.939+0000 7f805364c740 1 -- 172.21.15.103:0/1847191100 wait complete. 2024-09-19T06:55:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.939+0000 7f805364c740 20 client.17580 trim_cache size 0 max 16384 2024-09-19T06:55:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:55:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:55:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:55:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.940+0000 7f80507d8640 1 -- 172.21.15.103:0/1734539383 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80340a34e0 msgr2=0x7f80340a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.940+0000 7f80507d8640 1 --2- 172.21.15.103:0/1734539383 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80340a34e0 0x7f80340a59a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f80400b99e0 tx=0x7f804008f420 comp rx=0 tx=0).stop 2024-09-19T06:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.940+0000 7f80507d8640 1 -- 172.21.15.103:0/1734539383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804810e470 msgr2=0x7f804819ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.940+0000 7f80507d8640 1 --2- 172.21.15.103:0/1734539383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804810e470 0x7f804819ee40 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f804002ade0 tx=0x7f80400319f0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.940+0000 7f80507d8640 1 -- 172.21.15.103:0/1734539383 shutdown_connections 2024-09-19T06:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.940+0000 7f80507d8640 1 --2- 172.21.15.103:0/1734539383 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80340a34e0 0x7f80340a59a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.940+0000 7f80507d8640 1 --2- 172.21.15.103:0/1734539383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804810e470 0x7f804819ee40 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.940+0000 7f80507d8640 1 --2- 172.21.15.103:0/1734539383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8048101270 0x7f804819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.940+0000 7f80507d8640 1 --2- 172.21.15.103:0/1734539383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80481008c0 0x7f804819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.940+0000 7f80507d8640 1 -- 172.21.15.103:0/1734539383 >> 172.21.15.103:0/1734539383 conn(0x7f80480fc6b0 msgr2=0x7f80480feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.940+0000 7f80507d8640 1 -- 172.21.15.103:0/1734539383 shutdown_connections 2024-09-19T06:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:06.940+0000 7f80507d8640 1 -- 172.21.15.103:0/1734539383 wait complete. 2024-09-19T06:55:08.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:08 smithi103 ceph-mon[31158]: pgmap v180: 97 pgs: 97 active+clean; 5.5 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-09-19T06:55:08.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:08 smithi134 ceph-mon[35356]: pgmap v180: 97 pgs: 97 active+clean; 5.5 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-09-19T06:55:08.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:08 smithi174 ceph-mon[35637]: pgmap v180: 97 pgs: 97 active+clean; 5.5 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-09-19T06:55:10.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:10 smithi103 ceph-mon[31158]: pgmap v181: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 76 KiB/s wr, 38 op/s 2024-09-19T06:55:10.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:10 smithi134 ceph-mon[35356]: pgmap v181: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 76 KiB/s wr, 38 op/s 2024-09-19T06:55:10.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:10 smithi174 ceph-mon[35637]: pgmap v181: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 76 KiB/s wr, 38 op/s 2024-09-19T06:55:12.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:12 smithi103 ceph-mon[31158]: pgmap v182: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 60 KiB/s wr, 31 op/s 2024-09-19T06:55:12.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:12 smithi134 ceph-mon[35356]: pgmap v182: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 60 KiB/s wr, 31 op/s 2024-09-19T06:55:12.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:12 smithi174 ceph-mon[35637]: pgmap v182: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 60 KiB/s wr, 31 op/s 2024-09-19T06:55:14.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:14 smithi103 ceph-mon[31158]: pgmap v183: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 60 KiB/s wr, 31 op/s 2024-09-19T06:55:14.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:14 smithi134 ceph-mon[35356]: pgmap v183: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 60 KiB/s wr, 31 op/s 2024-09-19T06:55:14.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:14 smithi174 ceph-mon[35637]: pgmap v183: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 60 KiB/s wr, 31 op/s 2024-09-19T06:55:16.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:16 smithi103 ceph-mon[31158]: pgmap v184: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 79 KiB/s wr, 37 op/s 2024-09-19T06:55:16.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:16 smithi134 ceph-mon[35356]: pgmap v184: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 79 KiB/s wr, 37 op/s 2024-09-19T06:55:16.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:16 smithi174 ceph-mon[35637]: pgmap v184: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 79 KiB/s wr, 37 op/s 2024-09-19T06:55:18.385 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:18 smithi174 ceph-mon[35637]: pgmap v185: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-09-19T06:55:18.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:18 smithi103 ceph-mon[31158]: pgmap v185: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-09-19T06:55:18.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:18 smithi134 ceph-mon[35356]: pgmap v185: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-09-19T06:55:19.516 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:55:19.780 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.030+0000 7fcca0ca6640 1 -- 172.21.15.103:0/2319128102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc9c101ad0 msgr2=0x7fcc9c101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.030+0000 7fcca0ca6640 1 --2- 172.21.15.103:0/2319128102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc9c101ad0 0x7fcc9c101f30 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009cd0 tx=0x7fcc9002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.030+0000 7fcca0ca6640 1 -- 172.21.15.103:0/2319128102 shutdown_connections 2024-09-19T06:55:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.030+0000 7fcca0ca6640 1 --2- 172.21.15.103:0/2319128102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc9c102470 0x7fcc9c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.030+0000 7fcca0ca6640 1 --2- 172.21.15.103:0/2319128102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc9c101ad0 0x7fcc9c101f30 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.030+0000 7fcca0ca6640 1 --2- 172.21.15.103:0/2319128102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc9c107b70 0x7fcc9c107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.030+0000 7fcca0ca6640 1 -- 172.21.15.103:0/2319128102 >> 172.21.15.103:0/2319128102 conn(0x7fcc9c0fd7d0 msgr2=0x7fcc9c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.030+0000 7fcca0ca6640 1 -- 172.21.15.103:0/2319128102 shutdown_connections 2024-09-19T06:55:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.030+0000 7fcca0ca6640 1 -- 172.21.15.103:0/2319128102 wait complete. 2024-09-19T06:55:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcca0ca6640 1 Processor -- start 2024-09-19T06:55:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcca0ca6640 1 -- start start 2024-09-19T06:55:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcca0ca6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc9c101ad0 0x7fcc9c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcca0ca6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc9c102470 0x7fcc9c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcca0ca6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc9c107b70 0x7fcc9c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcca0ca6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c111f20 con 0x7fcc9c101ad0 2024-09-19T06:55:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcca0ca6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c111da0 con 0x7fcc9c107b70 2024-09-19T06:55:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcca0ca6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1120a0 con 0x7fcc9c102470 2024-09-19T06:55:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcc9a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc9c101ad0 0x7fcc9c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcc9a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc9c101ad0 0x7fcc9c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53178/0 (socket says 172.21.15.103:53178) 2024-09-19T06:55:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcc9a59c640 1 -- 172.21.15.103:0/359086463 learned_addr learned my addr 172.21.15.103:0/359086463 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcc99d9b640 1 --2- 172.21.15.103:0/359086463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc9c102470 0x7fcc9c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcc9a59c640 1 -- 172.21.15.103:0/359086463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc9c102470 msgr2=0x7fcc9c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcc9a59c640 1 --2- 172.21.15.103:0/359086463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc9c102470 0x7fcc9c198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcc9a59c640 1 -- 172.21.15.103:0/359086463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc9c107b70 msgr2=0x7fcc9c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcc9a59c640 1 --2- 172.21.15.103:0/359086463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc9c107b70 0x7fcc9c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcc9a59c640 1 -- 172.21.15.103:0/359086463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9c19d320 con 0x7fcc9c101ad0 2024-09-19T06:55:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcc9a59c640 1 --2- 172.21.15.103:0/359086463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc9c101ad0 0x7fcc9c198330 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400c890 tx=0x7fcc9400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcc8b7fe640 1 -- 172.21.15.103:0/359086463 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94004220 con 0x7fcc9c101ad0 2024-09-19T06:55:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.031+0000 7fcc8b7fe640 1 -- 172.21.15.103:0/359086463 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc940043c0 con 0x7fcc9c101ad0 2024-09-19T06:55:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.032+0000 7fcca0ca6640 1 -- 172.21.15.103:0/359086463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9c19d610 con 0x7fcc9c101ad0 2024-09-19T06:55:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.032+0000 7fcc8b7fe640 1 -- 172.21.15.103:0/359086463 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc940049e0 con 0x7fcc9c101ad0 2024-09-19T06:55:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.032+0000 7fcca0ca6640 1 -- 172.21.15.103:0/359086463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc9c1a4ea0 con 0x7fcc9c101ad0 2024-09-19T06:55:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.034+0000 7fcc8b7fe640 1 -- 172.21.15.103:0/359086463 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fcc940079b0 con 0x7fcc9c101ad0 2024-09-19T06:55:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.034+0000 7fcca3a3c740 1 Processor -- start 2024-09-19T06:55:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.034+0000 7fcca3a3c740 1 -- start start 2024-09-19T06:55:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.034+0000 7fcc8b7fe640 1 --2- 172.21.15.103:0/359086463 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcc840a3460 0x7fcc840a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.034+0000 7fcca3a3c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6b9daf0 0x5592b6b9ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.034+0000 7fcca3a3c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 0x5592b6ba8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.034+0000 7fcca3a3c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6ba8cb0 0x5592b6bab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcca3a3c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5592b6b022f0 con 0x5592b6ba8cb0 2024-09-19T06:55:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcca3a3c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5592b6aaa270 con 0x5592b6b9daf0 2024-09-19T06:55:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcca3a3c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592b6afe2b0 con 0x5592b6b9e410 2024-09-19T06:55:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc8b7fe640 1 -- 172.21.15.103:0/359086463 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fcc940cfde0 con 0x7fcc9c101ad0 2024-09-19T06:55:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc9ad9d640 1 --2- 172.21.15.103:0/359086463 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcc840a3460 0x7fcc840a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc9ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6ba8cb0 0x5592b6bab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc99d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 0x5592b6ba8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc99d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 0x5592b6ba8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:38188/0 (socket says 172.21.15.103:38188) 2024-09-19T06:55:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc9ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6ba8cb0 0x5592b6bab0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53192/0 (socket says 172.21.15.103:53192) 2024-09-19T06:55:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc99d9b640 1 -- 172.21.15.103:0/2175926617 learned_addr learned my addr 172.21.15.103:0/2175926617 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc9a59c640 1 --2- 172.21.15.103:0/2175926617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6b9daf0 0x5592b6b9ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc99d9b640 1 -- 172.21.15.103:0/2175926617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6b9daf0 msgr2=0x5592b6b9ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc99d9b640 1 --2- 172.21.15.103:0/2175926617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6b9daf0 0x5592b6b9ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc99d9b640 1 -- 172.21.15.103:0/2175926617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6ba8cb0 msgr2=0x5592b6bab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc99d9b640 1 --2- 172.21.15.103:0/2175926617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6ba8cb0 0x5592b6bab0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc99d9b640 1 -- 172.21.15.103:0/2175926617 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b6aa9340 con 0x5592b6b9e410 2024-09-19T06:55:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc9ad9d640 1 --2- 172.21.15.103:0/2175926617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6ba8cb0 0x5592b6bab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc9ad9d640 1 --2- 172.21.15.103:0/359086463 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcc840a3460 0x7fcc840a5920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00afd0 tx=0x7fcc8c00a930 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc99d9b640 1 --2- 172.21.15.103:0/2175926617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 0x5592b6ba8770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcc90005cc0 tx=0x7fcc90031580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93584ae045d8e914 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.035+0000 7fcc88ff9640 1 -- 172.21.15.103:0/2175926617 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90031ac0 con 0x5592b6b9e410 2024-09-19T06:55:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.036+0000 7fcc88ff9640 1 -- 172.21.15.103:0/2175926617 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc90036040 con 0x5592b6b9e410 2024-09-19T06:55:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.036+0000 7fcc88ff9640 1 -- 172.21.15.103:0/2175926617 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90004720 con 0x5592b6b9e410 2024-09-19T06:55:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.036+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2175926617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 msgr2=0x5592b6ba8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.036+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/2175926617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 0x5592b6ba8770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcc90005cc0 tx=0x7fcc90031580 comp rx=0 tx=0).stop 2024-09-19T06:55:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.036+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2175926617 shutdown_connections 2024-09-19T06:55:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.036+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/2175926617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6ba8cb0 0x5592b6bab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.036+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/2175926617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 0x5592b6ba8770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.036+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/2175926617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6b9daf0 0x5592b6b9ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.036+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2175926617 >> 172.21.15.103:0/2175926617 conn(0x5592b6b9a060 msgr2=0x5592b6b9a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.036+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2175926617 shutdown_connections 2024-09-19T06:55:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.036+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2175926617 wait complete. 2024-09-19T06:55:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.037+0000 7fcca3a3c740 1 -- start start 2024-09-19T06:55:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.037+0000 7fcca3a3c740 1 Processor -- start 2024-09-19T06:55:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.037+0000 7fcca3a3c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9daf0 0x5592b6ba5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.037+0000 7fcca3a3c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9e410 0x5592b6ba6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.037+0000 7fcca3a3c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 0x5592b6ba66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.037+0000 7fcca3a3c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5592b6bab880 con 0x5592b6b9e410 2024-09-19T06:55:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.037+0000 7fcca3a3c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5592b6a8b290 con 0x5592b6ba8cb0 2024-09-19T06:55:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.037+0000 7fcca3a3c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592b6baba00 con 0x5592b6b9daf0 2024-09-19T06:55:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.037+0000 7fcca3a3c740 10 client.0 _ll_register_callbacks cb 0x5592b6abc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.037+0000 7fcc99d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9e410 0x5592b6ba6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.037+0000 7fcc99d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9e410 0x5592b6ba6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53198/0 (socket says 172.21.15.103:53198) 2024-09-19T06:55:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.037+0000 7fcc9a59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9daf0 0x5592b6ba5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.037+0000 7fcc99d9b640 1 -- 172.21.15.103:0/2527489121 learned_addr learned my addr 172.21.15.103:0/2527489121 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.037+0000 7fcc9ad9d640 1 --2- 172.21.15.103:0/2527489121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 0x5592b6ba66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.038+0000 7fcc99d9b640 1 -- 172.21.15.103:0/2527489121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9daf0 msgr2=0x5592b6ba5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.038+0000 7fcc99d9b640 1 --2- 172.21.15.103:0/2527489121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9daf0 0x5592b6ba5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.038+0000 7fcc99d9b640 1 -- 172.21.15.103:0/2527489121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 msgr2=0x5592b6ba66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.038+0000 7fcc99d9b640 1 --2- 172.21.15.103:0/2527489121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 0x5592b6ba66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.038+0000 7fcc99d9b640 1 -- 172.21.15.103:0/2527489121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b6815210 con 0x5592b6b9e410 2024-09-19T06:55:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.038+0000 7fcc9a59c640 1 --2- 172.21.15.103:0/2527489121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9daf0 0x5592b6ba5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.038+0000 7fcc9ad9d640 1 --2- 172.21.15.103:0/2527489121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 0x5592b6ba66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.038+0000 7fcc99d9b640 1 --2- 172.21.15.103:0/2527489121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9e410 0x5592b6ba6170 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fcc9002a920 tx=0x7fcc90031b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5bde794968e99a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.038+0000 7fcc81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.038+0000 7fcc81ffb640 1 -- 172.21.15.103:0/2527489121 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90055060 con 0x5592b6b9e410 2024-09-19T06:55:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.038+0000 7fcc81ffb640 1 -- 172.21.15.103:0/2527489121 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fcc90003b80 con 0x5592b6b9e410 2024-09-19T06:55:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.038+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2527489121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc90036040 con 0x5592b6b9e410 2024-09-19T06:55:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.038+0000 7fcc81ffb640 1 -- 172.21.15.103:0/2527489121 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc90004860 con 0x5592b6b9e410 2024-09-19T06:55:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.038+0000 7fcca3a3c740 10 client.17592 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.038+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2527489121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5592b68e1620 con 0x5592b6b9e410 2024-09-19T06:55:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.038+0000 7fcc81ffb640 1 -- 172.21.15.103:0/2527489121 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc90036040 con 0x5592b6b9e410 2024-09-19T06:55:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.039+0000 7fcc81ffb640 1 -- 172.21.15.103:0/2527489121 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcc90030070 con 0x5592b6b9e410 2024-09-19T06:55:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.039+0000 7fcca3a3c740 10 client.17592 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.039+0000 7fcca3a3c740 10 client.17592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.039+0000 7fcca3a3c740 20 client.17592 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.039+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/2527489121 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5592b6bb8b50 0x5592b6bbaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.039+0000 7fcca3a3c740 4 client.17592 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.039+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2527489121 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592b6b0fa20 con 0x5592b6bb8b50 2024-09-19T06:55:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.039+0000 7fcc9a59c640 1 --2- 172.21.15.103:0/2527489121 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5592b6bb8b50 0x5592b6bbaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.039+0000 7fcc81ffb640 10 client.17592 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.039+0000 7fcc9a59c640 1 --2- 172.21.15.103:0/2527489121 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5592b6bb8b50 0x5592b6bbaf40 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcee006ce706b57e server_cookie=952254c32d4eef88 in_seq=0 out_seq=0 2024-09-19T06:55:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.040+0000 7fcc81ffb640 1 -- 172.21.15.103:0/2527489121 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592b6b0fa20 con 0x5592b6bb8b50 2024-09-19T06:55:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.040+0000 7fcc81ffb640 10 client.17592 handle_command_reply: tid=0 2024-09-19T06:55:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.040+0000 7fcca3a3c740 1 client.17592 shutdown 2024-09-19T06:55:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.040+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2527489121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9e410 msgr2=0x5592b6ba6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.040+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/2527489121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9e410 0x5592b6ba6170 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fcc9002a920 tx=0x7fcc90031b70 comp rx=0 tx=0).stop 2024-09-19T06:55:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.040+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2527489121 shutdown_connections 2024-09-19T06:55:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.040+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/2527489121 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5592b6bb8b50 0x5592b6bbaf40 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.040+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/2527489121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 0x5592b6ba66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.040+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/2527489121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9e410 0x5592b6ba6170 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.040+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/2527489121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9daf0 0x5592b6ba5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.040+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2527489121 >> 172.21.15.103:0/2527489121 conn(0x5592b6b9a060 msgr2=0x5592b6b9fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.040+0000 7fcc81ffb640 0 client.17592 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.041+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2527489121 shutdown_connections 2024-09-19T06:55:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.041+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2527489121 wait complete. 2024-09-19T06:55:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.041+0000 7fcca3a3c740 20 client.17592 trim_cache size 0 max 16384 2024-09-19T06:55:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.055+0000 7fcca3a3c740 1 Processor -- start 2024-09-19T06:55:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.055+0000 7fcca3a3c740 1 -- start start 2024-09-19T06:55:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.055+0000 7fcca3a3c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9daf0 0x5592b6ba64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.055+0000 7fcca3a3c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 0x5592b6bb43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.055+0000 7fcca3a3c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 0x5592b6ba2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.055+0000 7fcca3a3c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1ad7e0 con 0x5592b6b9daf0 2024-09-19T06:55:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.055+0000 7fcca3a3c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1af320 con 0x5592b6ba8cb0 2024-09-19T06:55:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.055+0000 7fcca3a3c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1adeb0 con 0x5592b6b9e410 2024-09-19T06:55:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.056+0000 7fcc9a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9daf0 0x5592b6ba64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.056+0000 7fcc9a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9daf0 0x5592b6ba64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53206/0 (socket says 172.21.15.103:53206) 2024-09-19T06:55:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.056+0000 7fcc9a59c640 1 -- 172.21.15.103:0/3420884206 learned_addr learned my addr 172.21.15.103:0/3420884206 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.056+0000 7fcc9ad9d640 1 --2- 172.21.15.103:0/3420884206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 0x5592b6ba2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.056+0000 7fcc9a59c640 1 -- 172.21.15.103:0/3420884206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 msgr2=0x5592b6bb43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.056+0000 7fcc99d9b640 1 --2- 172.21.15.103:0/3420884206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 0x5592b6bb43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.056+0000 7fcc9a59c640 1 --2- 172.21.15.103:0/3420884206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 0x5592b6bb43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.056+0000 7fcc9a59c640 1 -- 172.21.15.103:0/3420884206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 msgr2=0x5592b6ba2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.056+0000 7fcc9a59c640 1 --2- 172.21.15.103:0/3420884206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 0x5592b6ba2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.056+0000 7fcc9a59c640 1 -- 172.21.15.103:0/3420884206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b6b9d920 con 0x5592b6b9daf0 2024-09-19T06:55:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.056+0000 7fcc9ad9d640 1 --2- 172.21.15.103:0/3420884206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 0x5592b6ba2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.056+0000 7fcc9a59c640 1 --2- 172.21.15.103:0/3420884206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9daf0 0x5592b6ba64e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fcc940cd5a0 tx=0x7fcc940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e54c07302fbd130 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.056+0000 7fcc83fff640 1 -- 172.21.15.103:0/3420884206 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9401d040 con 0x5592b6b9daf0 2024-09-19T06:55:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.056+0000 7fcc83fff640 1 -- 172.21.15.103:0/3420884206 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc940a4d40 con 0x5592b6b9daf0 2024-09-19T06:55:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.056+0000 7fcc83fff640 1 -- 172.21.15.103:0/3420884206 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc940a0240 con 0x5592b6b9daf0 2024-09-19T06:55:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.057+0000 7fcca3a3c740 1 -- 172.21.15.103:0/3420884206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9daf0 msgr2=0x5592b6ba64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.057+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/3420884206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9daf0 0x5592b6ba64e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fcc940cd5a0 tx=0x7fcc940046c0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.057+0000 7fcca3a3c740 1 -- 172.21.15.103:0/3420884206 shutdown_connections 2024-09-19T06:55:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.057+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/3420884206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 0x5592b6ba2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.057+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/3420884206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 0x5592b6bb43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.057+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/3420884206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9daf0 0x5592b6ba64e0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.057+0000 7fcca3a3c740 1 -- 172.21.15.103:0/3420884206 >> 172.21.15.103:0/3420884206 conn(0x5592b6b9a060 msgr2=0x5592b6bb3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.057+0000 7fcca3a3c740 1 -- 172.21.15.103:0/3420884206 shutdown_connections 2024-09-19T06:55:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.057+0000 7fcca3a3c740 1 -- 172.21.15.103:0/3420884206 wait complete. 2024-09-19T06:55:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.057+0000 7fcca3a3c740 1 -- start start 2024-09-19T06:55:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.057+0000 7fcca3a3c740 1 Processor -- start 2024-09-19T06:55:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcca3a3c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9daf0 0x5592b6bbc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcca3a3c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 0x5592b6bbc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcca3a3c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 0x5592b6c22490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcca3a3c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1aec50 con 0x5592b6b9daf0 2024-09-19T06:55:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcca3a3c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1ae580 con 0x5592b6ba8cb0 2024-09-19T06:55:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcca3a3c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c1ab7a0 con 0x5592b6b9e410 2024-09-19T06:55:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcca3a3c740 10 client.0 _ll_register_callbacks cb 0x7fcc8c00ae00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcc9a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9daf0 0x5592b6bbc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcc9a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9daf0 0x5592b6bbc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53222/0 (socket says 172.21.15.103:53222) 2024-09-19T06:55:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcc9a59c640 1 -- 172.21.15.103:0/2861415850 learned_addr learned my addr 172.21.15.103:0/2861415850 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcc99d9b640 1 --2- 172.21.15.103:0/2861415850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 0x5592b6bbc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcc9ad9d640 1 --2- 172.21.15.103:0/2861415850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 0x5592b6c22490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcc9a59c640 1 -- 172.21.15.103:0/2861415850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 msgr2=0x5592b6bbc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcc9a59c640 1 --2- 172.21.15.103:0/2861415850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 0x5592b6bbc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcc9a59c640 1 -- 172.21.15.103:0/2861415850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 msgr2=0x5592b6c22490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcc9a59c640 1 --2- 172.21.15.103:0/2861415850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 0x5592b6c22490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcc9a59c640 1 -- 172.21.15.103:0/2861415850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b6c229d0 con 0x5592b6b9daf0 2024-09-19T06:55:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcc99d9b640 1 --2- 172.21.15.103:0/2861415850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 0x5592b6bbc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcc9ad9d640 1 --2- 172.21.15.103:0/2861415850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 0x5592b6c22490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcc9a59c640 1 --2- 172.21.15.103:0/2861415850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9daf0 0x5592b6bbc110 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fcc94098530 tx=0x7fcc9409c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfeb955c47c6b88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcc81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.058+0000 7fcc81ffb640 1 -- 172.21.15.103:0/2861415850 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9401d070 con 0x5592b6b9daf0 2024-09-19T06:55:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.059+0000 7fcc81ffb640 1 -- 172.21.15.103:0/2861415850 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fcc940bc0d0 con 0x5592b6b9daf0 2024-09-19T06:55:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.059+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2861415850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc940a4d40 con 0x5592b6b9daf0 2024-09-19T06:55:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.059+0000 7fcc81ffb640 1 -- 172.21.15.103:0/2861415850 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc94007ca0 con 0x5592b6b9daf0 2024-09-19T06:55:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.059+0000 7fcca3a3c740 10 client.17598 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.059+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2861415850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5592b6c27f40 con 0x5592b6b9daf0 2024-09-19T06:55:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.059+0000 7fcc81ffb640 1 -- 172.21.15.103:0/2861415850 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc940a4d40 con 0x5592b6b9daf0 2024-09-19T06:55:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.059+0000 7fcc81ffb640 1 -- 172.21.15.103:0/2861415850 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcc940a4940 con 0x5592b6b9daf0 2024-09-19T06:55:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.059+0000 7fcca3a3c740 10 client.17598 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.059+0000 7fcca3a3c740 10 client.17598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.059+0000 7fcca3a3c740 20 client.17598 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.059+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/2861415850 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5592b6c28550 0x5592b6c2a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.059+0000 7fcca3a3c740 4 client.17598 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:55:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.059+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2861415850 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592b6b09d70 con 0x5592b6c28550 2024-09-19T06:55:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.059+0000 7fcc99d9b640 1 --2- 172.21.15.103:0/2861415850 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5592b6c28550 0x5592b6c2a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.060+0000 7fcc81ffb640 10 client.17598 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.060+0000 7fcc99d9b640 1 --2- 172.21.15.103:0/2861415850 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5592b6c28550 0x5592b6c2a970 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=185eb803038b1484 server_cookie=660050ba256834a3 in_seq=0 out_seq=0 2024-09-19T06:55:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.085+0000 7fcc81ffb640 1 -- 172.21.15.103:0/2861415850 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592b6b09d70 con 0x5592b6c28550 2024-09-19T06:55:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.085+0000 7fcc81ffb640 10 client.17598 handle_command_reply: tid=0 2024-09-19T06:55:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.085+0000 7fcca3a3c740 1 client.17598 shutdown 2024-09-19T06:55:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.085+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2861415850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9daf0 msgr2=0x5592b6bbc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.085+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/2861415850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9daf0 0x5592b6bbc110 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fcc94098530 tx=0x7fcc9409c8c0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.085+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2861415850 shutdown_connections 2024-09-19T06:55:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.085+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/2861415850 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5592b6c28550 0x5592b6c2a970 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.085+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/2861415850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592b6ba8cb0 0x5592b6c22490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.085+0000 7fcc81ffb640 0 client.17598 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.085+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/2861415850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592b6b9e410 0x5592b6bbc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.085+0000 7fcca3a3c740 1 --2- 172.21.15.103:0/2861415850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5592b6b9daf0 0x5592b6bbc110 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.085+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2861415850 >> 172.21.15.103:0/2861415850 conn(0x5592b6b9a060 msgr2=0x5592b6bb44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.085+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2861415850 shutdown_connections 2024-09-19T06:55:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.085+0000 7fcca3a3c740 1 -- 172.21.15.103:0/2861415850 wait complete. 2024-09-19T06:55:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.086+0000 7fcca3a3c740 20 client.17598 trim_cache size 0 max 16384 2024-09-19T06:55:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:55:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:55:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:55:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:55:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.086+0000 7fcca0ca6640 1 -- 172.21.15.103:0/359086463 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcc840a3460 msgr2=0x7fcc840a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.086+0000 7fcca0ca6640 1 --2- 172.21.15.103:0/359086463 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcc840a3460 0x7fcc840a5920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00afd0 tx=0x7fcc8c00a930 comp rx=0 tx=0).stop 2024-09-19T06:55:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.086+0000 7fcca0ca6640 1 -- 172.21.15.103:0/359086463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc9c101ad0 msgr2=0x7fcc9c198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.086+0000 7fcca0ca6640 1 --2- 172.21.15.103:0/359086463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc9c101ad0 0x7fcc9c198330 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400c890 tx=0x7fcc9400cd50 comp rx=0 tx=0).stop 2024-09-19T06:55:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.086+0000 7fcca0ca6640 1 -- 172.21.15.103:0/359086463 shutdown_connections 2024-09-19T06:55:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.086+0000 7fcca0ca6640 1 --2- 172.21.15.103:0/359086463 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcc840a3460 0x7fcc840a5920 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.086+0000 7fcca0ca6640 1 --2- 172.21.15.103:0/359086463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc9c107b70 0x7fcc9c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.086+0000 7fcca0ca6640 1 --2- 172.21.15.103:0/359086463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc9c102470 0x7fcc9c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.086+0000 7fcca0ca6640 1 --2- 172.21.15.103:0/359086463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc9c101ad0 0x7fcc9c198330 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.086+0000 7fcca0ca6640 1 -- 172.21.15.103:0/359086463 >> 172.21.15.103:0/359086463 conn(0x7fcc9c0fd7d0 msgr2=0x7fcc9c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.086+0000 7fcca0ca6640 1 -- 172.21.15.103:0/359086463 shutdown_connections 2024-09-19T06:55:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:20.086+0000 7fcca0ca6640 1 -- 172.21.15.103:0/359086463 wait complete. 2024-09-19T06:55:20.174 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:20 smithi103 ceph-mon[31158]: pgmap v186: 97 pgs: 97 active+clean; 2.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-09-19T06:55:20.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.259+0000 7f3e77bd6640 1 -- 172.21.15.103:0/476310859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e70102dd0 msgr2=0x7f3e701031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.259+0000 7f3e77bd6640 1 --2- 172.21.15.103:0/476310859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e70102dd0 0x7f3e701031b0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009930 tx=0x7f3e6002a420 comp rx=0 tx=0).stop 2024-09-19T06:55:20.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.259+0000 7f3e77bd6640 1 -- 172.21.15.103:0/476310859 shutdown_connections 2024-09-19T06:55:20.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.259+0000 7f3e77bd6640 1 --2- 172.21.15.103:0/476310859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e70108300 0x7f3e7010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.259+0000 7f3e77bd6640 1 --2- 172.21.15.103:0/476310859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e70103780 0x7f3e70107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.259+0000 7f3e77bd6640 1 --2- 172.21.15.103:0/476310859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e70102dd0 0x7f3e701031b0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.259+0000 7f3e77bd6640 1 -- 172.21.15.103:0/476310859 >> 172.21.15.103:0/476310859 conn(0x7f3e700fc7d0 msgr2=0x7f3e700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:20.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.259+0000 7f3e77bd6640 1 -- 172.21.15.103:0/476310859 shutdown_connections 2024-09-19T06:55:20.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.259+0000 7f3e77bd6640 1 -- 172.21.15.103:0/476310859 wait complete. 2024-09-19T06:55:20.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e77bd6640 1 Processor -- start 2024-09-19T06:55:20.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e77bd6640 1 -- start start 2024-09-19T06:55:20.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e77bd6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e70102dd0 0x7f3e70100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e77bd6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e70103780 0x7f3e70100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e77bd6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e70108300 0x7f3e701014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e77bd6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e701120b0 con 0x7f3e70103780 2024-09-19T06:55:20.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e77bd6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e70111f30 con 0x7f3e70102dd0 2024-09-19T06:55:20.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e77bd6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3e70112230 con 0x7f3e70108300 2024-09-19T06:55:20.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e75171640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e70103780 0x7f3e70100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e75171640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e70103780 0x7f3e70100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53246/0 (socket says 172.21.15.103:53246) 2024-09-19T06:55:20.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e75171640 1 -- 172.21.15.103:0/2747147298 learned_addr learned my addr 172.21.15.103:0/2747147298 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:20.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e75972640 1 --2- 172.21.15.103:0/2747147298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e70102dd0 0x7f3e70100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e76173640 1 --2- 172.21.15.103:0/2747147298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e70108300 0x7f3e701014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e75171640 1 -- 172.21.15.103:0/2747147298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e70108300 msgr2=0x7f3e701014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e75171640 1 --2- 172.21.15.103:0/2747147298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e70108300 0x7f3e701014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e75171640 1 -- 172.21.15.103:0/2747147298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e70102dd0 msgr2=0x7f3e70100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e75171640 1 --2- 172.21.15.103:0/2747147298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e70102dd0 0x7f3e70100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e75171640 1 -- 172.21.15.103:0/2747147298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e701a6b30 con 0x7f3e70103780 2024-09-19T06:55:20.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.260+0000 7f3e76173640 1 --2- 172.21.15.103:0/2747147298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e70108300 0x7f3e701014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:20.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.261+0000 7f3e75171640 1 --2- 172.21.15.103:0/2747147298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e70103780 0x7f3e70100f70 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00d950 tx=0x7f3e6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:20.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.261+0000 7f3e66ffd640 1 -- 172.21.15.103:0/2747147298 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c017070 con 0x7f3e70103780 2024-09-19T06:55:20.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.261+0000 7f3e77bd6640 1 -- 172.21.15.103:0/2747147298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e701a6e20 con 0x7f3e70103780 2024-09-19T06:55:20.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.261+0000 7f3e66ffd640 1 -- 172.21.15.103:0/2747147298 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e6c009c30 con 0x7f3e70103780 2024-09-19T06:55:20.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.261+0000 7f3e66ffd640 1 -- 172.21.15.103:0/2747147298 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0047a0 con 0x7f3e70103780 2024-09-19T06:55:20.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.261+0000 7f3e77bd6640 1 -- 172.21.15.103:0/2747147298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e701a72d0 con 0x7f3e70103780 2024-09-19T06:55:20.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.263+0000 7f3e66ffd640 1 -- 172.21.15.103:0/2747147298 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3e6c004020 con 0x7f3e70103780 2024-09-19T06:55:20.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.264+0000 7f3e79faf740 1 Processor -- start 2024-09-19T06:55:20.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.264+0000 7f3e79faf740 1 -- start start 2024-09-19T06:55:20.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.264+0000 7f3e66ffd640 1 --2- 172.21.15.103:0/2747147298 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3e5c0a33b0 0x7f3e5c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.264+0000 7f3e79faf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44bfb4a0 0x55db44bfb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.264+0000 7f3e79faf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44bfbdc0 0x55db44c06140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.264+0000 7f3e66ffd640 1 -- 172.21.15.103:0/2747147298 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f3e6c0cfa50 con 0x7f3e70103780 2024-09-19T06:55:20.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.264+0000 7f3e79faf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44c06680 0x55db44c08a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.264+0000 7f3e79faf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55db44b612e0 con 0x55db44c06680 2024-09-19T06:55:20.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.264+0000 7f3e76173640 1 --2- 172.21.15.103:0/2747147298 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3e5c0a33b0 0x7f3e5c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.264+0000 7f3e79faf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55db44b08ce0 con 0x55db44bfbdc0 2024-09-19T06:55:20.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.264+0000 7f3e79faf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55db44b5d2a0 con 0x55db44bfb4a0 2024-09-19T06:55:20.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.264+0000 7f3e76173640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44c06680 0x55db44c08a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.264+0000 7f3e76173640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44c06680 0x55db44c08a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53254/0 (socket says 172.21.15.103:53254) 2024-09-19T06:55:20.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.264+0000 7f3e76173640 1 -- 172.21.15.103:0/2244892137 learned_addr learned my addr 172.21.15.103:0/2244892137 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:20.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.264+0000 7f3e75972640 1 --2- 172.21.15.103:0/2244892137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44bfb4a0 0x55db44bfb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.264+0000 7f3e75171640 1 --2- 172.21.15.103:0/2244892137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44bfbdc0 0x55db44c06140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.265+0000 7f3e76173640 1 -- 172.21.15.103:0/2244892137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44bfb4a0 msgr2=0x55db44bfb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:20.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.265+0000 7f3e76173640 1 --2- 172.21.15.103:0/2244892137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44bfb4a0 0x55db44bfb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.265+0000 7f3e76173640 1 -- 172.21.15.103:0/2244892137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44bfbdc0 msgr2=0x55db44c06140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:20.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.265+0000 7f3e76173640 1 --2- 172.21.15.103:0/2244892137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44bfbdc0 0x55db44c06140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.265+0000 7f3e76173640 1 -- 172.21.15.103:0/2244892137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db44aed440 con 0x55db44c06680 2024-09-19T06:55:20.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.265+0000 7f3e76173640 1 --2- 172.21.15.103:0/2244892137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44c06680 0x55db44c08a70 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f3e68013500 tx=0x7f3e680139c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1ab4bb74358663 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:20.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.265+0000 7f3e5bfff640 1 -- 172.21.15.103:0/2244892137 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68024070 con 0x55db44c06680 2024-09-19T06:55:20.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.265+0000 7f3e5bfff640 1 -- 172.21.15.103:0/2244892137 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e6800eda0 con 0x55db44c06680 2024-09-19T06:55:20.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.265+0000 7f3e5bfff640 1 -- 172.21.15.103:0/2244892137 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6801ed40 con 0x55db44c06680 2024-09-19T06:55:20.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.265+0000 7f3e76173640 1 --2- 172.21.15.103:0/2747147298 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3e5c0a33b0 0x7f3e5c0a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3e68021000 tx=0x7f3e6801a000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:20.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.266+0000 7f3e79faf740 1 -- 172.21.15.103:0/2244892137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44c06680 msgr2=0x55db44c08a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.266+0000 7f3e79faf740 1 --2- 172.21.15.103:0/2244892137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44c06680 0x55db44c08a70 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f3e68013500 tx=0x7f3e680139c0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.266+0000 7f3e79faf740 1 -- 172.21.15.103:0/2244892137 shutdown_connections 2024-09-19T06:55:20.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.266+0000 7f3e79faf740 1 --2- 172.21.15.103:0/2244892137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44c06680 0x55db44c08a70 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.266+0000 7f3e79faf740 1 --2- 172.21.15.103:0/2244892137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44bfbdc0 0x55db44c06140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.266+0000 7f3e79faf740 1 --2- 172.21.15.103:0/2244892137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44bfb4a0 0x55db44bfb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.266+0000 7f3e79faf740 1 -- 172.21.15.103:0/2244892137 >> 172.21.15.103:0/2244892137 conn(0x55db44bf7a30 msgr2=0x55db44bf7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:20.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.266+0000 7f3e79faf740 1 -- 172.21.15.103:0/2244892137 shutdown_connections 2024-09-19T06:55:20.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.266+0000 7f3e79faf740 1 -- 172.21.15.103:0/2244892137 wait complete. 2024-09-19T06:55:20.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.266+0000 7f3e79faf740 1 -- start start 2024-09-19T06:55:20.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.266+0000 7f3e79faf740 1 Processor -- start 2024-09-19T06:55:20.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e79faf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44bfb4a0 0x55db44c05fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e79faf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfbdc0 0x55db44c03250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e79faf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44c06680 0x55db44c03790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e79faf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55db44c09250 con 0x55db44bfbdc0 2024-09-19T06:55:20.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e79faf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55db44b1c050 con 0x55db44c06680 2024-09-19T06:55:20.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e79faf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55db44c093d0 con 0x55db44bfb4a0 2024-09-19T06:55:20.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e79faf740 10 client.0 _ll_register_callbacks cb 0x55db44b5c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:20.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e75171640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfbdc0 0x55db44c03250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e75171640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfbdc0 0x55db44c03250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53260/0 (socket says 172.21.15.103:53260) 2024-09-19T06:55:20.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e75171640 1 -- 172.21.15.103:0/4112277540 learned_addr learned my addr 172.21.15.103:0/4112277540 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:20.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e75972640 1 --2- 172.21.15.103:0/4112277540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44bfb4a0 0x55db44c05fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e76173640 1 --2- 172.21.15.103:0/4112277540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44c06680 0x55db44c03790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e75171640 1 -- 172.21.15.103:0/4112277540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44bfb4a0 msgr2=0x55db44c05fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:20.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e75171640 1 --2- 172.21.15.103:0/4112277540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44bfb4a0 0x55db44c05fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e75171640 1 -- 172.21.15.103:0/4112277540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44c06680 msgr2=0x55db44c03790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:20.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e75171640 1 --2- 172.21.15.103:0/4112277540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44c06680 0x55db44c03790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.267+0000 7f3e75171640 1 -- 172.21.15.103:0/4112277540 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db448d23c0 con 0x55db44bfbdc0 2024-09-19T06:55:20.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.268+0000 7f3e75171640 1 --2- 172.21.15.103:0/4112277540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfbdc0 0x55db44c03250 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c09ba00 tx=0x7f3e6c09c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e6159771901297 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:20.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.268+0000 7f3e597fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:20.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.268+0000 7f3e597fa640 1 -- 172.21.15.103:0/4112277540 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0a4970 con 0x55db44bfbdc0 2024-09-19T06:55:20.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.268+0000 7f3e597fa640 1 -- 172.21.15.103:0/4112277540 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f3e6c098370 con 0x55db44bfbdc0 2024-09-19T06:55:20.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.268+0000 7f3e79faf740 1 -- 172.21.15.103:0/4112277540 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e6800eda0 con 0x55db44bfbdc0 2024-09-19T06:55:20.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.268+0000 7f3e597fa640 1 -- 172.21.15.103:0/4112277540 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3e6c00fa40 con 0x55db44bfbdc0 2024-09-19T06:55:20.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.268+0000 7f3e79faf740 10 client.17610 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:20.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.268+0000 7f3e79faf740 1 -- 172.21.15.103:0/4112277540 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55db44c14c20 con 0x55db44bfbdc0 2024-09-19T06:55:20.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.268+0000 7f3e597fa640 1 -- 172.21.15.103:0/4112277540 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e6800eda0 con 0x55db44bfbdc0 2024-09-19T06:55:20.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.268+0000 7f3e597fa640 1 -- 172.21.15.103:0/4112277540 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3e6c09b2c0 con 0x55db44bfbdc0 2024-09-19T06:55:20.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.268+0000 7f3e79faf740 10 client.17610 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:20.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.268+0000 7f3e79faf740 10 client.17610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:20.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.268+0000 7f3e79faf740 20 client.17610 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:20.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.268+0000 7f3e79faf740 1 --2- 172.21.15.103:0/4112277540 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55db44c16080 0x55db44c184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.268+0000 7f3e79faf740 4 client.17610 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:20.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.268+0000 7f3e79faf740 1 -- 172.21.15.103:0/4112277540 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db44b6e850 con 0x55db44c16080 2024-09-19T06:55:20.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.269+0000 7f3e75972640 1 --2- 172.21.15.103:0/4112277540 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55db44c16080 0x55db44c184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.269+0000 7f3e597fa640 10 client.17610 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:20.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.269+0000 7f3e75972640 1 --2- 172.21.15.103:0/4112277540 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55db44c16080 0x55db44c184d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e026aa9c94a305a server_cookie=e055308070753d3e in_seq=0 out_seq=0 2024-09-19T06:55:20.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.270+0000 7f3e597fa640 1 -- 172.21.15.103:0/4112277540 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db44b6e850 con 0x55db44c16080 2024-09-19T06:55:20.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.270+0000 7f3e597fa640 10 client.17610 handle_command_reply: tid=0 2024-09-19T06:55:20.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.270+0000 7f3e79faf740 1 client.17610 shutdown 2024-09-19T06:55:20.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.270+0000 7f3e79faf740 1 -- 172.21.15.103:0/4112277540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfbdc0 msgr2=0x55db44c03250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.270+0000 7f3e79faf740 1 --2- 172.21.15.103:0/4112277540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfbdc0 0x55db44c03250 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c09ba00 tx=0x7f3e6c09c530 comp rx=0 tx=0).stop 2024-09-19T06:55:20.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.270+0000 7f3e79faf740 1 -- 172.21.15.103:0/4112277540 shutdown_connections 2024-09-19T06:55:20.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.270+0000 7f3e79faf740 1 --2- 172.21.15.103:0/4112277540 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55db44c16080 0x55db44c184d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.270+0000 7f3e79faf740 1 --2- 172.21.15.103:0/4112277540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44c06680 0x55db44c03790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.270+0000 7f3e79faf740 1 --2- 172.21.15.103:0/4112277540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfbdc0 0x55db44c03250 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.270+0000 7f3e79faf740 1 --2- 172.21.15.103:0/4112277540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44bfb4a0 0x55db44c05fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.270+0000 7f3e597fa640 0 client.17610 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:20.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.270+0000 7f3e79faf740 1 -- 172.21.15.103:0/4112277540 >> 172.21.15.103:0/4112277540 conn(0x55db44bf7a30 msgr2=0x55db44bfd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:20.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.270+0000 7f3e79faf740 1 -- 172.21.15.103:0/4112277540 shutdown_connections 2024-09-19T06:55:20.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.270+0000 7f3e79faf740 1 -- 172.21.15.103:0/4112277540 wait complete. 2024-09-19T06:55:20.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.270+0000 7f3e79faf740 20 client.17610 trim_cache size 0 max 16384 2024-09-19T06:55:20.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.287+0000 7f3e79faf740 1 Processor -- start 2024-09-19T06:55:20.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.287+0000 7f3e79faf740 1 -- start start 2024-09-19T06:55:20.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e79faf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfb4a0 0x55db44c124b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e79faf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44bfbdc0 0x55db44c1dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e79faf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44c06680 0x55db44c1e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e79faf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e701afb40 con 0x55db44bfb4a0 2024-09-19T06:55:20.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e79faf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e701b1680 con 0x55db44bfbdc0 2024-09-19T06:55:20.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e79faf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3e701b0210 con 0x55db44c06680 2024-09-19T06:55:20.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e75972640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfb4a0 0x55db44c124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e75972640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfb4a0 0x55db44c124b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53262/0 (socket says 172.21.15.103:53262) 2024-09-19T06:55:20.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e75972640 1 -- 172.21.15.103:0/3269020007 learned_addr learned my addr 172.21.15.103:0/3269020007 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:20.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e75171640 1 --2- 172.21.15.103:0/3269020007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44bfbdc0 0x55db44c1dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e76173640 1 --2- 172.21.15.103:0/3269020007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44c06680 0x55db44c1e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e75972640 1 -- 172.21.15.103:0/3269020007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44c06680 msgr2=0x55db44c1e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:20.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e75972640 1 --2- 172.21.15.103:0/3269020007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44c06680 0x55db44c1e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e75972640 1 -- 172.21.15.103:0/3269020007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44bfbdc0 msgr2=0x55db44c1dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:20.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e75972640 1 --2- 172.21.15.103:0/3269020007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44bfbdc0 0x55db44c1dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e75972640 1 -- 172.21.15.103:0/3269020007 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db44c017a0 con 0x55db44bfb4a0 2024-09-19T06:55:20.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.288+0000 7f3e75972640 1 --2- 172.21.15.103:0/3269020007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfb4a0 0x55db44c124b0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f3e60005940 tx=0x7f3e600058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e438b444ff2b13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:20.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.289+0000 7f3e5b7fe640 1 -- 172.21.15.103:0/3269020007 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60049070 con 0x55db44bfb4a0 2024-09-19T06:55:20.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.289+0000 7f3e5b7fe640 1 -- 172.21.15.103:0/3269020007 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e60005520 con 0x55db44bfb4a0 2024-09-19T06:55:20.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.289+0000 7f3e5b7fe640 1 -- 172.21.15.103:0/3269020007 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60035040 con 0x55db44bfb4a0 2024-09-19T06:55:20.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.289+0000 7f3e79faf740 1 -- 172.21.15.103:0/3269020007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfb4a0 msgr2=0x55db44c124b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.289+0000 7f3e79faf740 1 --2- 172.21.15.103:0/3269020007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfb4a0 0x55db44c124b0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f3e60005940 tx=0x7f3e600058b0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.289+0000 7f3e79faf740 1 -- 172.21.15.103:0/3269020007 shutdown_connections 2024-09-19T06:55:20.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.289+0000 7f3e79faf740 1 --2- 172.21.15.103:0/3269020007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44c06680 0x55db44c1e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.289+0000 7f3e79faf740 1 --2- 172.21.15.103:0/3269020007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44bfbdc0 0x55db44c1dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.289+0000 7f3e79faf740 1 --2- 172.21.15.103:0/3269020007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfb4a0 0x55db44c124b0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.289+0000 7f3e79faf740 1 -- 172.21.15.103:0/3269020007 >> 172.21.15.103:0/3269020007 conn(0x55db44bf7a30 msgr2=0x55db44bfe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:20.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.289+0000 7f3e79faf740 1 -- 172.21.15.103:0/3269020007 shutdown_connections 2024-09-19T06:55:20.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.289+0000 7f3e79faf740 1 -- 172.21.15.103:0/3269020007 wait complete. 2024-09-19T06:55:20.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.290+0000 7f3e79faf740 1 -- start start 2024-09-19T06:55:20.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.290+0000 7f3e79faf740 1 Processor -- start 2024-09-19T06:55:20.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.290+0000 7f3e79faf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfb4a0 0x55db44c0ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e79faf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44bfbdc0 0x55db44c0a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e79faf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44c06680 0x55db44c0a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e79faf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e701b0fb0 con 0x55db44bfb4a0 2024-09-19T06:55:20.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e79faf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e701b08e0 con 0x55db44c06680 2024-09-19T06:55:20.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e79faf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3e701adb00 con 0x55db44bfbdc0 2024-09-19T06:55:20.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e79faf740 10 client.0 _ll_register_callbacks cb 0x7f3e6c00b680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:20.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e75972640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfb4a0 0x55db44c0ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e75972640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfb4a0 0x55db44c0ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53272/0 (socket says 172.21.15.103:53272) 2024-09-19T06:55:20.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e75171640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44bfbdc0 0x55db44c0a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e75972640 1 -- 172.21.15.103:0/777405232 learned_addr learned my addr 172.21.15.103:0/777405232 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:20.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e75972640 1 -- 172.21.15.103:0/777405232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44bfbdc0 msgr2=0x55db44c0a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:20.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e75972640 1 --2- 172.21.15.103:0/777405232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44bfbdc0 0x55db44c0a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e75972640 1 -- 172.21.15.103:0/777405232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44c06680 msgr2=0x55db44c0a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:20.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e75972640 1 --2- 172.21.15.103:0/777405232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44c06680 0x55db44c0a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e75972640 1 -- 172.21.15.103:0/777405232 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db44c0af40 con 0x55db44bfb4a0 2024-09-19T06:55:20.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e75972640 1 --2- 172.21.15.103:0/777405232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfb4a0 0x55db44c0ee40 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f3e60006010 tx=0x7f3e60032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689b29e12349f195 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:20.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e597fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:20.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e597fa640 1 -- 172.21.15.103:0/777405232 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60052060 con 0x55db44bfb4a0 2024-09-19T06:55:20.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e597fa640 1 -- 172.21.15.103:0/777405232 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f3e60005040 con 0x55db44bfb4a0 2024-09-19T06:55:20.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.291+0000 7f3e79faf740 1 -- 172.21.15.103:0/777405232 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e60005520 con 0x55db44bfb4a0 2024-09-19T06:55:20.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.292+0000 7f3e597fa640 1 -- 172.21.15.103:0/777405232 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3e60049040 con 0x55db44bfb4a0 2024-09-19T06:55:20.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.292+0000 7f3e79faf740 10 client.17616 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:20.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.292+0000 7f3e79faf740 1 -- 172.21.15.103:0/777405232 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55db44c80e50 con 0x55db44bfb4a0 2024-09-19T06:55:20.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.292+0000 7f3e597fa640 1 -- 172.21.15.103:0/777405232 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e60005520 con 0x55db44bfb4a0 2024-09-19T06:55:20.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.292+0000 7f3e597fa640 1 -- 172.21.15.103:0/777405232 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3e60031a00 con 0x55db44bfb4a0 2024-09-19T06:55:20.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.292+0000 7f3e79faf740 10 client.17616 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:20.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.292+0000 7f3e79faf740 10 client.17616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:20.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.292+0000 7f3e79faf740 20 client.17616 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:20.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.292+0000 7f3e79faf740 1 --2- 172.21.15.103:0/777405232 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55db44c81510 0x55db44c83930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:20.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.292+0000 7f3e79faf740 4 client.17616 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:55:20.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.292+0000 7f3e79faf740 1 -- 172.21.15.103:0/777405232 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55db44b68d60 con 0x55db44c81510 2024-09-19T06:55:20.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.292+0000 7f3e75171640 1 --2- 172.21.15.103:0/777405232 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55db44c81510 0x55db44c83930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:20.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.293+0000 7f3e597fa640 10 client.17616 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:20.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.293+0000 7f3e75171640 1 --2- 172.21.15.103:0/777405232 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55db44c81510 0x55db44c83930 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5eb8cb77f1d99d server_cookie=dc020484a0ebe37 in_seq=0 out_seq=0 2024-09-19T06:55:20.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.293+0000 7f3e597fa640 1 -- 172.21.15.103:0/777405232 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55db44b68d60 con 0x55db44c81510 2024-09-19T06:55:20.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.293+0000 7f3e597fa640 10 client.17616 handle_command_reply: tid=0 2024-09-19T06:55:20.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.293+0000 7f3e79faf740 1 client.17616 shutdown 2024-09-19T06:55:20.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.293+0000 7f3e79faf740 1 -- 172.21.15.103:0/777405232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfb4a0 msgr2=0x55db44c0ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.293+0000 7f3e79faf740 1 --2- 172.21.15.103:0/777405232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfb4a0 0x55db44c0ee40 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f3e60006010 tx=0x7f3e60032eb0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.293+0000 7f3e79faf740 1 -- 172.21.15.103:0/777405232 shutdown_connections 2024-09-19T06:55:20.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.293+0000 7f3e79faf740 1 --2- 172.21.15.103:0/777405232 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55db44c81510 0x55db44c83930 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.293+0000 7f3e79faf740 1 --2- 172.21.15.103:0/777405232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db44c06680 0x55db44c0a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.293+0000 7f3e79faf740 1 --2- 172.21.15.103:0/777405232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db44bfbdc0 0x55db44c0a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.293+0000 7f3e79faf740 1 --2- 172.21.15.103:0/777405232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db44bfb4a0 0x55db44c0ee40 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.293+0000 7f3e597fa640 0 client.17616 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:20.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.293+0000 7f3e79faf740 1 -- 172.21.15.103:0/777405232 >> 172.21.15.103:0/777405232 conn(0x55db44bf7a30 msgr2=0x55db44c04ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:20.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.293+0000 7f3e79faf740 1 -- 172.21.15.103:0/777405232 shutdown_connections 2024-09-19T06:55:20.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.293+0000 7f3e79faf740 1 -- 172.21.15.103:0/777405232 wait complete. 2024-09-19T06:55:20.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.294+0000 7f3e79faf740 20 client.17616 trim_cache size 0 max 16384 2024-09-19T06:55:20.299 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:55:20.299 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T06:55:20.299 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T06:55:20.299 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:55:20.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.294+0000 7f3e77bd6640 1 -- 172.21.15.103:0/2747147298 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3e5c0a33b0 msgr2=0x7f3e5c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.294+0000 7f3e77bd6640 1 --2- 172.21.15.103:0/2747147298 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3e5c0a33b0 0x7f3e5c0a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3e68021000 tx=0x7f3e6801a000 comp rx=0 tx=0).stop 2024-09-19T06:55:20.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.294+0000 7f3e77bd6640 1 -- 172.21.15.103:0/2747147298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e70103780 msgr2=0x7f3e70100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:20.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.294+0000 7f3e77bd6640 1 --2- 172.21.15.103:0/2747147298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e70103780 0x7f3e70100f70 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00d950 tx=0x7f3e6c00de10 comp rx=0 tx=0).stop 2024-09-19T06:55:20.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.294+0000 7f3e77bd6640 1 -- 172.21.15.103:0/2747147298 shutdown_connections 2024-09-19T06:55:20.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.294+0000 7f3e77bd6640 1 --2- 172.21.15.103:0/2747147298 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3e5c0a33b0 0x7f3e5c0a5870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.294+0000 7f3e77bd6640 1 --2- 172.21.15.103:0/2747147298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e70108300 0x7f3e701014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.294+0000 7f3e77bd6640 1 --2- 172.21.15.103:0/2747147298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e70103780 0x7f3e70100f70 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.294+0000 7f3e77bd6640 1 --2- 172.21.15.103:0/2747147298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e70102dd0 0x7f3e70100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:20.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.294+0000 7f3e77bd6640 1 -- 172.21.15.103:0/2747147298 >> 172.21.15.103:0/2747147298 conn(0x7f3e700fc7d0 msgr2=0x7f3e70105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:20.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.294+0000 7f3e77bd6640 1 -- 172.21.15.103:0/2747147298 shutdown_connections 2024-09-19T06:55:20.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:20.295+0000 7f3e77bd6640 1 -- 172.21.15.103:0/2747147298 wait complete. 2024-09-19T06:55:20.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:20 smithi134 ceph-mon[35356]: pgmap v186: 97 pgs: 97 active+clean; 2.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-09-19T06:55:20.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:20 smithi174 ceph-mon[35637]: pgmap v186: 97 pgs: 97 active+clean; 2.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-09-19T06:55:21.075 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:55:21.076 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:55:21.332 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.787+0000 7f49a558d640 1 -- 172.21.15.103:0/2094174586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a0102570 msgr2=0x7f49a01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.787+0000 7f49a558d640 1 --2- 172.21.15.103:0/2094174586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a0102570 0x7f49a01029b0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f4994009900 tx=0x7f499402a480 comp rx=0 tx=0).stop 2024-09-19T06:55:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.788+0000 7f49a558d640 1 -- 172.21.15.103:0/2094174586 shutdown_connections 2024-09-19T06:55:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.788+0000 7f49a558d640 1 --2- 172.21.15.103:0/2094174586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a0102ef0 0x7f49a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.788+0000 7f49a558d640 1 --2- 172.21.15.103:0/2094174586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a0102570 0x7f49a01029b0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.788+0000 7f49a558d640 1 --2- 172.21.15.103:0/2094174586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49a0100e20 0x7f49a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.788+0000 7f49a558d640 1 -- 172.21.15.103:0/2094174586 >> 172.21.15.103:0/2094174586 conn(0x7f49a00fc7d0 msgr2=0x7f49a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:21.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.788+0000 7f49a558d640 1 -- 172.21.15.103:0/2094174586 shutdown_connections 2024-09-19T06:55:21.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.788+0000 7f49a558d640 1 -- 172.21.15.103:0/2094174586 wait complete. 2024-09-19T06:55:21.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.788+0000 7f49a558d640 1 Processor -- start 2024-09-19T06:55:21.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.788+0000 7f49a558d640 1 -- start start 2024-09-19T06:55:21.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f49a558d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a0100e20 0x7f49a0198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:21.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f49a558d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49a0102570 0x7f49a0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:21.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f49a558d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a0102ef0 0x7f49a019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:21.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f49a558d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f49a0110f80 con 0x7f49a0100e20 2024-09-19T06:55:21.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f49a558d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f49a0110e00 con 0x7f49a0102ef0 2024-09-19T06:55:21.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f49a558d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f49a0111100 con 0x7f49a0102570 2024-09-19T06:55:21.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f499effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a0100e20 0x7f49a0198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:21.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f499effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a0100e20 0x7f49a0198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53286/0 (socket says 172.21.15.103:53286) 2024-09-19T06:55:21.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f499effd640 1 -- 172.21.15.103:0/2595675962 learned_addr learned my addr 172.21.15.103:0/2595675962 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:21.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f499e7fc640 1 --2- 172.21.15.103:0/2595675962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49a0102570 0x7f49a0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:21.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f499f7fe640 1 --2- 172.21.15.103:0/2595675962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a0102ef0 0x7f49a019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:21.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f499effd640 1 -- 172.21.15.103:0/2595675962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49a0102570 msgr2=0x7f49a0198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:21.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f499effd640 1 --2- 172.21.15.103:0/2595675962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49a0102570 0x7f49a0198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:21.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f499effd640 1 -- 172.21.15.103:0/2595675962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a0102ef0 msgr2=0x7f49a019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:21.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f499effd640 1 --2- 172.21.15.103:0/2595675962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a0102ef0 0x7f49a019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:21.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f499effd640 1 -- 172.21.15.103:0/2595675962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a019d520 con 0x7f49a0100e20 2024-09-19T06:55:21.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f499f7fe640 1 --2- 172.21.15.103:0/2595675962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a0102ef0 0x7f49a019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:21.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f499effd640 1 --2- 172.21.15.103:0/2595675962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a0100e20 0x7f49a0198570 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f499000eb90 tx=0x7f499000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:21.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f4987fff640 1 -- 172.21.15.103:0/2595675962 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990004d20 con 0x7f49a0100e20 2024-09-19T06:55:21.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f4987fff640 1 -- 172.21.15.103:0/2595675962 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f499000ee90 con 0x7f49a0100e20 2024-09-19T06:55:21.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f49a558d640 1 -- 172.21.15.103:0/2595675962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a019d810 con 0x7f49a0100e20 2024-09-19T06:55:21.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f4987fff640 1 -- 172.21.15.103:0/2595675962 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990010830 con 0x7f49a0100e20 2024-09-19T06:55:21.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.789+0000 7f49a558d640 1 -- 172.21.15.103:0/2595675962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a01a50f0 con 0x7f49a0100e20 2024-09-19T06:55:21.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.790+0000 7f49a558d640 1 -- 172.21.15.103:0/2595675962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a0101200 con 0x7f49a0100e20 2024-09-19T06:55:21.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.792+0000 7f4987fff640 1 -- 172.21.15.103:0/2595675962 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4990005000 con 0x7f49a0100e20 2024-09-19T06:55:21.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.792+0000 7f4987fff640 1 --2- 172.21.15.103:0/2595675962 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f49880a33b0 0x7f49880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:21.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.792+0000 7f499e7fc640 1 --2- 172.21.15.103:0/2595675962 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f49880a33b0 0x7f49880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:21.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.792+0000 7f4987fff640 1 -- 172.21.15.103:0/2595675962 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f49900d05f0 con 0x7f49a0100e20 2024-09-19T06:55:21.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.793+0000 7f499e7fc640 1 --2- 172.21.15.103:0/2595675962 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f49880a33b0 0x7f49880a5870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4994009230 tx=0x7f49940058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:21.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.795+0000 7f4987fff640 1 -- 172.21.15.103:0/2595675962 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4990054210 con 0x7f49a0100e20 2024-09-19T06:55:21.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.997+0000 7f49a558d640 1 -- 172.21.15.103:0/2595675962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49a01998d0 con 0x7f49a0100e20 2024-09-19T06:55:21.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:21.998+0000 7f4987fff640 1 -- 172.21.15.103:0/2595675962 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f499009cfe0 con 0x7f49a0100e20 2024-09-19T06:55:21.999 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:55:21.999 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:55:21.999 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:55:22.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:22.000+0000 7f49a558d640 1 -- 172.21.15.103:0/2595675962 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f49880a33b0 msgr2=0x7f49880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:22.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:22.000+0000 7f49a558d640 1 --2- 172.21.15.103:0/2595675962 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f49880a33b0 0x7f49880a5870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4994009230 tx=0x7f49940058d0 comp rx=0 tx=0).stop 2024-09-19T06:55:22.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:22.000+0000 7f49a558d640 1 -- 172.21.15.103:0/2595675962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a0100e20 msgr2=0x7f49a0198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:22.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:22.000+0000 7f49a558d640 1 --2- 172.21.15.103:0/2595675962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a0100e20 0x7f49a0198570 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f499000eb90 tx=0x7f499000c6a0 comp rx=0 tx=0).stop 2024-09-19T06:55:22.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:22.000+0000 7f49a558d640 1 -- 172.21.15.103:0/2595675962 shutdown_connections 2024-09-19T06:55:22.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:22.000+0000 7f49a558d640 1 --2- 172.21.15.103:0/2595675962 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f49880a33b0 0x7f49880a5870 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:22.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:22.000+0000 7f49a558d640 1 --2- 172.21.15.103:0/2595675962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a0102ef0 0x7f49a019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:22.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:22.000+0000 7f49a558d640 1 --2- 172.21.15.103:0/2595675962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49a0102570 0x7f49a0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:22.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:22.000+0000 7f49a558d640 1 --2- 172.21.15.103:0/2595675962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49a0100e20 0x7f49a0198570 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:22.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:22.000+0000 7f49a558d640 1 -- 172.21.15.103:0/2595675962 >> 172.21.15.103:0/2595675962 conn(0x7f49a00fc7d0 msgr2=0x7f49a006a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:22.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:22.000+0000 7f49a558d640 1 -- 172.21.15.103:0/2595675962 shutdown_connections 2024-09-19T06:55:22.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:22.000+0000 7f49a558d640 1 -- 172.21.15.103:0/2595675962 wait complete. 2024-09-19T06:55:22.328 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:22 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2595675962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:22.328 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:22 smithi103 ceph-mon[31158]: pgmap v187: 97 pgs: 97 active+clean; 2.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-19T06:55:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:22 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2595675962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:22 smithi174 ceph-mon[35637]: pgmap v187: 97 pgs: 97 active+clean; 2.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-19T06:55:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:22 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2595675962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:22 smithi134 ceph-mon[35356]: pgmap v187: 97 pgs: 97 active+clean; 2.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-19T06:55:22.718 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:55:22.968 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:23.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.422+0000 7f711674c640 1 -- 172.21.15.103:0/3122091816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7110106f10 msgr2=0x7f711010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:23.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.422+0000 7f711674c640 1 --2- 172.21.15.103:0/3122091816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7110106f10 0x7f711010ea90 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7108009ca0 tx=0x7f710802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.422+0000 7f711674c640 1 -- 172.21.15.103:0/3122091816 shutdown_connections 2024-09-19T06:55:23.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.422+0000 7f711674c640 1 --2- 172.21.15.103:0/3122091816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7110106f10 0x7f711010ea90 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.422+0000 7f711674c640 1 --2- 172.21.15.103:0/3122091816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7110102790 0x7f71101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.422+0000 7f711674c640 1 --2- 172.21.15.103:0/3122091816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7110101de0 0x7f71101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.422+0000 7f711674c640 1 -- 172.21.15.103:0/3122091816 >> 172.21.15.103:0/3122091816 conn(0x7f71100fd8c0 msgr2=0x7f71100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:23.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.422+0000 7f711674c640 1 -- 172.21.15.103:0/3122091816 shutdown_connections 2024-09-19T06:55:23.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.422+0000 7f711674c640 1 -- 172.21.15.103:0/3122091816 wait complete. 2024-09-19T06:55:23.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.422+0000 7f711674c640 1 Processor -- start 2024-09-19T06:55:23.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.422+0000 7f711674c640 1 -- start start 2024-09-19T06:55:23.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.422+0000 7f711674c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7110101de0 0x7f711019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.422+0000 7f711674c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7110102790 0x7f711019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.422+0000 7f711674c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7110106f10 0x7f711019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.422+0000 7f711674c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7110111f70 con 0x7f7110106f10 2024-09-19T06:55:23.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.422+0000 7f711674c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7110111df0 con 0x7f7110102790 2024-09-19T06:55:23.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f711674c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f71101120f0 con 0x7f7110101de0 2024-09-19T06:55:23.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f7114ce9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7110106f10 0x7f711019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f7114ce9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7110106f10 0x7f711019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53296/0 (socket says 172.21.15.103:53296) 2024-09-19T06:55:23.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f7114ce9640 1 -- 172.21.15.103:0/4073825737 learned_addr learned my addr 172.21.15.103:0/4073825737 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:23.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f710ffff640 1 --2- 172.21.15.103:0/4073825737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7110101de0 0x7f711019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f7107fff640 1 --2- 172.21.15.103:0/4073825737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7110102790 0x7f711019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f7114ce9640 1 -- 172.21.15.103:0/4073825737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7110101de0 msgr2=0x7f711019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:23.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f7114ce9640 1 --2- 172.21.15.103:0/4073825737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7110101de0 0x7f711019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f7114ce9640 1 -- 172.21.15.103:0/4073825737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7110102790 msgr2=0x7f711019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:23.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f7114ce9640 1 --2- 172.21.15.103:0/4073825737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7110102790 0x7f711019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f7114ce9640 1 -- 172.21.15.103:0/4073825737 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711019f600 con 0x7f7110106f10 2024-09-19T06:55:23.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f7107fff640 1 --2- 172.21.15.103:0/4073825737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7110102790 0x7f711019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:23.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f7114ce9640 1 --2- 172.21.15.103:0/4073825737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7110106f10 0x7f711019ef20 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7108031a20 tx=0x7f7108031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:23.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f710dffb640 1 -- 172.21.15.103:0/4073825737 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108003c30 con 0x7f7110106f10 2024-09-19T06:55:23.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f711674c640 1 -- 172.21.15.103:0/4073825737 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711019f890 con 0x7f7110106f10 2024-09-19T06:55:23.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f710dffb640 1 -- 172.21.15.103:0/4073825737 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7108003dd0 con 0x7f7110106f10 2024-09-19T06:55:23.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f710dffb640 1 -- 172.21.15.103:0/4073825737 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710802fcf0 con 0x7f7110106f10 2024-09-19T06:55:23.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.423+0000 7f711674c640 1 -- 172.21.15.103:0/4073825737 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71101a71d0 con 0x7f7110106f10 2024-09-19T06:55:23.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.426+0000 7f710dffb640 1 -- 172.21.15.103:0/4073825737 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7108004050 con 0x7f7110106f10 2024-09-19T06:55:23.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.426+0000 7f7118b25740 1 Processor -- start 2024-09-19T06:55:23.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.426+0000 7f7118b25740 1 -- start start 2024-09-19T06:55:23.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.426+0000 7f7118b25740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e24cb40 0x55ef0e24b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.426+0000 7f7118b25740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 0x55ef0e24ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.426+0000 7f7118b25740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e256370 0x55ef0e258760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.426+0000 7f7118b25740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef0e1b12f0 con 0x55ef0e24cb40 2024-09-19T06:55:23.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.426+0000 7f7118b25740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ef0e158cf0 con 0x55ef0e256370 2024-09-19T06:55:23.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.426+0000 7f7118b25740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef0e1ad2b0 con 0x55ef0e24b5d0 2024-09-19T06:55:23.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.426+0000 7f7107fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e24cb40 0x55ef0e24b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.426+0000 7f7107fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e24cb40 0x55ef0e24b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53308/0 (socket says 172.21.15.103:53308) 2024-09-19T06:55:23.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.426+0000 7f7107fff640 1 -- 172.21.15.103:0/1315805915 learned_addr learned my addr 172.21.15.103:0/1315805915 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:23.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.426+0000 7f710dffb640 1 --2- 172.21.15.103:0/4073825737 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70fc0a33b0 0x7f70fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.427+0000 7f710ffff640 1 --2- 172.21.15.103:0/1315805915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 0x55ef0e24ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.427+0000 7f7107fff640 1 --2- 172.21.15.103:0/1315805915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e256370 0x55ef0e258760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.427+0000 7f7114ce9640 1 --2- 172.21.15.103:0/4073825737 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70fc0a33b0 0x7f70fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.427+0000 7f7107fff640 1 -- 172.21.15.103:0/1315805915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 msgr2=0x55ef0e24ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:23.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.427+0000 7f7107fff640 1 --2- 172.21.15.103:0/1315805915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 0x55ef0e24ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.427+0000 7f7107fff640 1 -- 172.21.15.103:0/1315805915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e256370 msgr2=0x55ef0e258760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:23.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.427+0000 7f7107fff640 1 --2- 172.21.15.103:0/1315805915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e256370 0x55ef0e258760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.427+0000 7f7107fff640 1 -- 172.21.15.103:0/1315805915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef0e13d3a0 con 0x55ef0e24cb40 2024-09-19T06:55:23.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.427+0000 7f710dffb640 1 -- 172.21.15.103:0/4073825737 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f71080f0f50 con 0x7f7110106f10 2024-09-19T06:55:23.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.427+0000 7f7107fff640 1 --2- 172.21.15.103:0/1315805915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e24cb40 0x55ef0e24b090 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f70f8005940 tx=0x7f70f800c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac21fff5a7d15b38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:23.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.427+0000 7f71067fc640 1 -- 172.21.15.103:0/1315805915 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8018030 con 0x55ef0e24cb40 2024-09-19T06:55:23.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.427+0000 7f7114ce9640 1 --2- 172.21.15.103:0/4073825737 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70fc0a33b0 0x7f70fc0a5870 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f71080b9840 tx=0x7f71080dbf90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:23.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.427+0000 7f71067fc640 1 -- 172.21.15.103:0/1315805915 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f8005500 con 0x55ef0e24cb40 2024-09-19T06:55:23.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.427+0000 7f71067fc640 1 -- 172.21.15.103:0/1315805915 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8010860 con 0x55ef0e24cb40 2024-09-19T06:55:23.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.428+0000 7f7118b25740 1 -- 172.21.15.103:0/1315805915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e24cb40 msgr2=0x55ef0e24b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:23.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.428+0000 7f7118b25740 1 --2- 172.21.15.103:0/1315805915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e24cb40 0x55ef0e24b090 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f70f8005940 tx=0x7f70f800c620 comp rx=0 tx=0).stop 2024-09-19T06:55:23.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.428+0000 7f7118b25740 1 -- 172.21.15.103:0/1315805915 shutdown_connections 2024-09-19T06:55:23.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.428+0000 7f7118b25740 1 --2- 172.21.15.103:0/1315805915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e256370 0x55ef0e258760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.428+0000 7f7118b25740 1 --2- 172.21.15.103:0/1315805915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 0x55ef0e24ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.428+0000 7f7118b25740 1 --2- 172.21.15.103:0/1315805915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e24cb40 0x55ef0e24b090 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.428+0000 7f7118b25740 1 -- 172.21.15.103:0/1315805915 >> 172.21.15.103:0/1315805915 conn(0x55ef0e214310 msgr2=0x55ef0e247c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:23.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.428+0000 7f7118b25740 1 -- 172.21.15.103:0/1315805915 shutdown_connections 2024-09-19T06:55:23.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.428+0000 7f7118b25740 1 -- 172.21.15.103:0/1315805915 wait complete. 2024-09-19T06:55:23.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.428+0000 7f7118b25740 1 -- start start 2024-09-19T06:55:23.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.428+0000 7f7118b25740 1 Processor -- start 2024-09-19T06:55:23.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7118b25740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 0x55ef0e255d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7118b25740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 0x55ef0e252fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7118b25740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e2535d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7118b25740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef0e259070 con 0x55ef0e256370 2024-09-19T06:55:23.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7118b25740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ef0e16c060 con 0x55ef0e24cb40 2024-09-19T06:55:23.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7118b25740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef0e2591f0 con 0x55ef0e24b5d0 2024-09-19T06:55:23.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7118b25740 10 client.0 _ll_register_callbacks cb 0x55ef0debf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:23.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7107fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e2535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7107fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e2535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53320/0 (socket says 172.21.15.103:53320) 2024-09-19T06:55:23.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7107fff640 1 -- 172.21.15.103:0/3806144868 learned_addr learned my addr 172.21.15.103:0/3806144868 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:23.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7107fff640 1 --2- 172.21.15.103:0/3806144868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 0x55ef0e255d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f710ffff640 1 --2- 172.21.15.103:0/3806144868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 0x55ef0e252fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7107fff640 1 -- 172.21.15.103:0/3806144868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 msgr2=0x55ef0e255d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:23.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7107fff640 1 --2- 172.21.15.103:0/3806144868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 0x55ef0e255d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7107fff640 1 -- 172.21.15.103:0/3806144868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 msgr2=0x55ef0e252fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:23.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7107fff640 1 --2- 172.21.15.103:0/3806144868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 0x55ef0e252fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7107fff640 1 -- 172.21.15.103:0/3806144868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef0df223c0 con 0x55ef0e256370 2024-09-19T06:55:23.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f710ffff640 1 --2- 172.21.15.103:0/3806144868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 0x55ef0e252fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:23.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f7107fff640 1 --2- 172.21.15.103:0/3806144868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e2535d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f70f8004bb0 tx=0x7f70f8004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1419b6faf97aff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:23.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f70ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:23.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.429+0000 7f70ef7fe640 1 -- 172.21.15.103:0/3806144868 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8018070 con 0x55ef0e256370 2024-09-19T06:55:23.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.430+0000 7f70ef7fe640 1 -- 172.21.15.103:0/3806144868 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f70f8010840 con 0x55ef0e256370 2024-09-19T06:55:23.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.430+0000 7f7118b25740 1 -- 172.21.15.103:0/3806144868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70f8005500 con 0x55ef0e256370 2024-09-19T06:55:23.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.430+0000 7f70ef7fe640 1 -- 172.21.15.103:0/3806144868 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f70f8033050 con 0x55ef0e256370 2024-09-19T06:55:23.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.430+0000 7f7118b25740 10 client.17634 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:23.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.430+0000 7f7118b25740 1 -- 172.21.15.103:0/3806144868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ef0e264ad0 con 0x55ef0e256370 2024-09-19T06:55:23.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.430+0000 7f70ef7fe640 1 -- 172.21.15.103:0/3806144868 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f8005500 con 0x55ef0e256370 2024-09-19T06:55:23.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.430+0000 7f70ef7fe640 1 -- 172.21.15.103:0/3806144868 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f70f801d930 con 0x55ef0e256370 2024-09-19T06:55:23.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.430+0000 7f7118b25740 10 client.17634 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:23.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.430+0000 7f7118b25740 10 client.17634 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:55:23.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.430+0000 7f7118b25740 20 client.17634 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:23.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.430+0000 7f7118b25740 1 --2- 172.21.15.103:0/3806144868 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55ef0e265e20 0x55ef0e268210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.430+0000 7f7118b25740 4 client.17634 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:23.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.430+0000 7f7118b25740 1 -- 172.21.15.103:0/3806144868 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef0e1be860 con 0x55ef0e265e20 2024-09-19T06:55:23.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.431+0000 7f710ffff640 1 --2- 172.21.15.103:0/3806144868 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55ef0e265e20 0x55ef0e268210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.431+0000 7f70ef7fe640 10 client.17634 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:55:23.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.431+0000 7f710ffff640 1 --2- 172.21.15.103:0/3806144868 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55ef0e265e20 0x55ef0e268210 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=562efcee75048771 server_cookie=f451bd8c73a690b in_seq=0 out_seq=0 2024-09-19T06:55:23.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.432+0000 7f70ef7fe640 1 -- 172.21.15.103:0/3806144868 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef0e1be860 con 0x55ef0e265e20 2024-09-19T06:55:23.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.432+0000 7f70ef7fe640 10 client.17634 handle_command_reply: tid=0 2024-09-19T06:55:23.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.432+0000 7f7118b25740 1 client.17634 shutdown 2024-09-19T06:55:23.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.432+0000 7f7118b25740 1 -- 172.21.15.103:0/3806144868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 msgr2=0x55ef0e2535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:23.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.432+0000 7f7118b25740 1 --2- 172.21.15.103:0/3806144868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e2535d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f70f8004bb0 tx=0x7f70f8004da0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.432+0000 7f7118b25740 1 -- 172.21.15.103:0/3806144868 shutdown_connections 2024-09-19T06:55:23.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.432+0000 7f7118b25740 1 --2- 172.21.15.103:0/3806144868 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55ef0e265e20 0x55ef0e268210 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.432+0000 7f7118b25740 1 --2- 172.21.15.103:0/3806144868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e2535d0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.432+0000 7f7118b25740 1 --2- 172.21.15.103:0/3806144868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 0x55ef0e252fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.432+0000 7f70ef7fe640 0 client.17634 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:55:23.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.432+0000 7f7118b25740 1 --2- 172.21.15.103:0/3806144868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 0x55ef0e255d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.432+0000 7f7118b25740 1 -- 172.21.15.103:0/3806144868 >> 172.21.15.103:0/3806144868 conn(0x55ef0e214310 msgr2=0x55ef0e24d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:23.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.432+0000 7f7118b25740 1 -- 172.21.15.103:0/3806144868 shutdown_connections 2024-09-19T06:55:23.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.432+0000 7f7118b25740 1 -- 172.21.15.103:0/3806144868 wait complete. 2024-09-19T06:55:23.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.432+0000 7f7118b25740 20 client.17634 trim_cache size 0 max 16384 2024-09-19T06:55:23.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7118b25740 1 Processor -- start 2024-09-19T06:55:23.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7118b25740 1 -- start start 2024-09-19T06:55:23.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7118b25740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 0x55ef0e24e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7118b25740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 0x55ef0e24e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7118b25740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e26d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7118b25740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71101afa00 con 0x55ef0e256370 2024-09-19T06:55:23.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7118b25740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71101b1540 con 0x55ef0e24cb40 2024-09-19T06:55:23.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7118b25740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f71101b00d0 con 0x55ef0e24b5d0 2024-09-19T06:55:23.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7107fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e26d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7107fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e26d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53334/0 (socket says 172.21.15.103:53334) 2024-09-19T06:55:23.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7107fff640 1 -- 172.21.15.103:0/2708555931 learned_addr learned my addr 172.21.15.103:0/2708555931 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:23.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f710ffff640 1 --2- 172.21.15.103:0/2708555931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 0x55ef0e24e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7107fff640 1 -- 172.21.15.103:0/2708555931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 msgr2=0x55ef0e24e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:23.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7107fff640 1 --2- 172.21.15.103:0/2708555931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 0x55ef0e24e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7107fff640 1 -- 172.21.15.103:0/2708555931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 msgr2=0x55ef0e24e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:23.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7107fff640 1 --2- 172.21.15.103:0/2708555931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 0x55ef0e24e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7107fff640 1 -- 172.21.15.103:0/2708555931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef0e24c750 con 0x55ef0e256370 2024-09-19T06:55:23.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f710ffff640 1 --2- 172.21.15.103:0/2708555931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 0x55ef0e24e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:23.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7107fff640 1 --2- 172.21.15.103:0/2708555931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e26d870 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f70f800c500 tx=0x7f70f8005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b533df1f6c2e64aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:23.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.445+0000 7f7105ffb640 1 -- 172.21.15.103:0/2708555931 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8030070 con 0x55ef0e256370 2024-09-19T06:55:23.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.446+0000 7f7105ffb640 1 -- 172.21.15.103:0/2708555931 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f8027040 con 0x55ef0e256370 2024-09-19T06:55:23.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.446+0000 7f7105ffb640 1 -- 172.21.15.103:0/2708555931 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8018070 con 0x55ef0e256370 2024-09-19T06:55:23.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.446+0000 7f7118b25740 1 -- 172.21.15.103:0/2708555931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 msgr2=0x55ef0e26d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:23.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.446+0000 7f7118b25740 1 --2- 172.21.15.103:0/2708555931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e26d870 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f70f800c500 tx=0x7f70f8005a10 comp rx=0 tx=0).stop 2024-09-19T06:55:23.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.446+0000 7f7118b25740 1 -- 172.21.15.103:0/2708555931 shutdown_connections 2024-09-19T06:55:23.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.446+0000 7f7118b25740 1 --2- 172.21.15.103:0/2708555931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e26d870 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.446+0000 7f7118b25740 1 --2- 172.21.15.103:0/2708555931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 0x55ef0e24e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.446+0000 7f7118b25740 1 --2- 172.21.15.103:0/2708555931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 0x55ef0e24e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.446+0000 7f7118b25740 1 -- 172.21.15.103:0/2708555931 >> 172.21.15.103:0/2708555931 conn(0x55ef0e214310 msgr2=0x55ef0e246d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:23.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.446+0000 7f7118b25740 1 -- 172.21.15.103:0/2708555931 shutdown_connections 2024-09-19T06:55:23.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.446+0000 7f7118b25740 1 -- 172.21.15.103:0/2708555931 wait complete. 2024-09-19T06:55:23.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.446+0000 7f7118b25740 1 -- start start 2024-09-19T06:55:23.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7118b25740 1 Processor -- start 2024-09-19T06:55:23.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7118b25740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 0x55ef0e25a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7118b25740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 0x55ef0e25aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7118b25740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e25af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7118b25740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71101b0e70 con 0x55ef0e256370 2024-09-19T06:55:23.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7118b25740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71101b07a0 con 0x55ef0e24cb40 2024-09-19T06:55:23.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7118b25740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f71101ad9c0 con 0x55ef0e24b5d0 2024-09-19T06:55:23.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7118b25740 10 client.0 _ll_register_callbacks cb 0x7f70f80123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:23.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7107fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e25af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7107fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e25af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53336/0 (socket says 172.21.15.103:53336) 2024-09-19T06:55:23.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7107fff640 1 -- 172.21.15.103:0/3419746068 learned_addr learned my addr 172.21.15.103:0/3419746068 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:23.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7107fff640 1 --2- 172.21.15.103:0/3419746068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 0x55ef0e25a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f710ffff640 1 --2- 172.21.15.103:0/3419746068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 0x55ef0e25aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7107fff640 1 -- 172.21.15.103:0/3419746068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 msgr2=0x55ef0e25a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:23.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7107fff640 1 --2- 172.21.15.103:0/3419746068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 0x55ef0e25a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7107fff640 1 -- 172.21.15.103:0/3419746068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 msgr2=0x55ef0e25aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:23.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7107fff640 1 --2- 172.21.15.103:0/3419746068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 0x55ef0e25aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7107fff640 1 -- 172.21.15.103:0/3419746068 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef0e2cfae0 con 0x55ef0e256370 2024-09-19T06:55:23.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f710ffff640 1 --2- 172.21.15.103:0/3419746068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 0x55ef0e25aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:23.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f7107fff640 1 --2- 172.21.15.103:0/3419746068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e25af60 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f70f801df70 tx=0x7f70f801f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810c9badc1511891 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:23.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.447+0000 7f70ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:23.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.448+0000 7f70ef7fe640 1 -- 172.21.15.103:0/3419746068 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8018070 con 0x55ef0e256370 2024-09-19T06:55:23.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.448+0000 7f70ef7fe640 1 -- 172.21.15.103:0/3419746068 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f70f800c580 con 0x55ef0e256370 2024-09-19T06:55:23.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.448+0000 7f7118b25740 1 -- 172.21.15.103:0/3419746068 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70f8027040 con 0x55ef0e256370 2024-09-19T06:55:23.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.448+0000 7f70ef7fe640 1 -- 172.21.15.103:0/3419746068 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f70f8030030 con 0x55ef0e256370 2024-09-19T06:55:23.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.448+0000 7f7118b25740 10 client.17640 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:23.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.448+0000 7f7118b25740 1 -- 172.21.15.103:0/3419746068 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ef0e2d0d50 con 0x55ef0e256370 2024-09-19T06:55:23.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.448+0000 7f70ef7fe640 1 -- 172.21.15.103:0/3419746068 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f8027040 con 0x55ef0e256370 2024-09-19T06:55:23.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.448+0000 7f70ef7fe640 1 -- 172.21.15.103:0/3419746068 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f70f801b640 con 0x55ef0e256370 2024-09-19T06:55:23.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.448+0000 7f7118b25740 10 client.17640 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:23.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.448+0000 7f7118b25740 10 client.17640 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:55:23.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.448+0000 7f7118b25740 20 client.17640 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.448+0000 7f7118b25740 1 --2- 172.21.15.103:0/3419746068 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55ef0e2d13e0 0x55ef0e2d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.448+0000 7f7118b25740 4 client.17640 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:55:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.448+0000 7f7118b25740 1 -- 172.21.15.103:0/3419746068 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef0e1b8d70 con 0x55ef0e2d13e0 2024-09-19T06:55:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.448+0000 7f710ffff640 1 --2- 172.21.15.103:0/3419746068 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55ef0e2d13e0 0x55ef0e2d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f710ffff640 1 --2- 172.21.15.103:0/3419746068 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55ef0e2d13e0 0x55ef0e2d3800 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=20d0671f210350eb server_cookie=e44efba6699e6c4e in_seq=0 out_seq=0 2024-09-19T06:55:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f70ef7fe640 10 client.17640 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:55:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f70ef7fe640 1 -- 172.21.15.103:0/3419746068 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef0e1b8d70 con 0x55ef0e2d13e0 2024-09-19T06:55:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f70ef7fe640 10 client.17640 handle_command_reply: tid=0 2024-09-19T06:55:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f7118b25740 1 client.17640 shutdown 2024-09-19T06:55:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f7118b25740 1 -- 172.21.15.103:0/3419746068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 msgr2=0x55ef0e25af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f7118b25740 1 --2- 172.21.15.103:0/3419746068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e25af60 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f70f801df70 tx=0x7f70f801f040 comp rx=0 tx=0).stop 2024-09-19T06:55:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f7118b25740 1 -- 172.21.15.103:0/3419746068 shutdown_connections 2024-09-19T06:55:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f7118b25740 1 --2- 172.21.15.103:0/3419746068 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55ef0e2d13e0 0x55ef0e2d3800 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f70ef7fe640 0 client.17640 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:55:23.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f7118b25740 1 --2- 172.21.15.103:0/3419746068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef0e256370 0x55ef0e25af60 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f7118b25740 1 --2- 172.21.15.103:0/3419746068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef0e24cb40 0x55ef0e25aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f7118b25740 1 --2- 172.21.15.103:0/3419746068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef0e24b5d0 0x55ef0e25a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f7118b25740 1 -- 172.21.15.103:0/3419746068 >> 172.21.15.103:0/3419746068 conn(0x55ef0e214310 msgr2=0x55ef0e246310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:23.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f7118b25740 1 -- 172.21.15.103:0/3419746068 shutdown_connections 2024-09-19T06:55:23.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f7118b25740 1 -- 172.21.15.103:0/3419746068 wait complete. 2024-09-19T06:55:23.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.449+0000 7f7118b25740 20 client.17640 trim_cache size 0 max 16384 2024-09-19T06:55:23.471 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:55:23.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.450+0000 7f711674c640 1 -- 172.21.15.103:0/4073825737 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70fc0a33b0 msgr2=0x7f70fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:23.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.450+0000 7f711674c640 1 --2- 172.21.15.103:0/4073825737 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70fc0a33b0 0x7f70fc0a5870 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f71080b9840 tx=0x7f71080dbf90 comp rx=0 tx=0).stop 2024-09-19T06:55:23.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.450+0000 7f711674c640 1 -- 172.21.15.103:0/4073825737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7110106f10 msgr2=0x7f711019ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:23.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.450+0000 7f711674c640 1 --2- 172.21.15.103:0/4073825737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7110106f10 0x7f711019ef20 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7108031a20 tx=0x7f7108031a50 comp rx=0 tx=0).stop 2024-09-19T06:55:23.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.450+0000 7f711674c640 1 -- 172.21.15.103:0/4073825737 shutdown_connections 2024-09-19T06:55:23.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.450+0000 7f711674c640 1 --2- 172.21.15.103:0/4073825737 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70fc0a33b0 0x7f70fc0a5870 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.450+0000 7f711674c640 1 --2- 172.21.15.103:0/4073825737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7110106f10 0x7f711019ef20 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.450+0000 7f711674c640 1 --2- 172.21.15.103:0/4073825737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7110102790 0x7f711019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.450+0000 7f711674c640 1 --2- 172.21.15.103:0/4073825737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7110101de0 0x7f711019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:23.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.450+0000 7f711674c640 1 -- 172.21.15.103:0/4073825737 >> 172.21.15.103:0/4073825737 conn(0x7f71100fd8c0 msgr2=0x7f71100feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:23.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.450+0000 7f711674c640 1 -- 172.21.15.103:0/4073825737 shutdown_connections 2024-09-19T06:55:23.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:23.450+0000 7f711674c640 1 -- 172.21.15.103:0/4073825737 wait complete. 2024-09-19T06:55:24.224 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:55:24.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:24 smithi103 ceph-mon[31158]: pgmap v188: 97 pgs: 97 active+clean; 2.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-19T06:55:24.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:24 smithi134 ceph-mon[35356]: pgmap v188: 97 pgs: 97 active+clean; 2.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-19T06:55:24.470 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:24.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:24 smithi174 ceph-mon[35637]: pgmap v188: 97 pgs: 97 active+clean; 2.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-19T06:55:24.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.918+0000 7f9a66c95640 1 -- 172.21.15.103:0/388655829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a60102570 msgr2=0x7f9a601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:24.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.918+0000 7f9a66c95640 1 --2- 172.21.15.103:0/388655829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a60102570 0x7f9a601029b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009900 tx=0x7f9a4c02a480 comp rx=0 tx=0).stop 2024-09-19T06:55:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.918+0000 7f9a66c95640 1 -- 172.21.15.103:0/388655829 shutdown_connections 2024-09-19T06:55:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.918+0000 7f9a66c95640 1 --2- 172.21.15.103:0/388655829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a60102ef0 0x7f9a6010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.918+0000 7f9a66c95640 1 --2- 172.21.15.103:0/388655829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a60102570 0x7f9a601029b0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.918+0000 7f9a66c95640 1 --2- 172.21.15.103:0/388655829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a60100e20 0x7f9a60101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.918+0000 7f9a66c95640 1 -- 172.21.15.103:0/388655829 >> 172.21.15.103:0/388655829 conn(0x7f9a600fc7d0 msgr2=0x7f9a600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.918+0000 7f9a66c95640 1 -- 172.21.15.103:0/388655829 shutdown_connections 2024-09-19T06:55:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.918+0000 7f9a66c95640 1 -- 172.21.15.103:0/388655829 wait complete. 2024-09-19T06:55:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a66c95640 1 Processor -- start 2024-09-19T06:55:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a66c95640 1 -- start start 2024-09-19T06:55:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a66c95640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a60100e20 0x7f9a601973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a66c95640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a60102570 0x7f9a601978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a66c95640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a60102ef0 0x7f9a6019bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a66c95640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a60110f80 con 0x7f9a60102570 2024-09-19T06:55:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a66c95640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9a60110e00 con 0x7f9a60102ef0 2024-09-19T06:55:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a66c95640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9a60111100 con 0x7f9a60100e20 2024-09-19T06:55:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a5ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a60102570 0x7f9a601978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a5ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a60102570 0x7f9a601978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53354/0 (socket says 172.21.15.103:53354) 2024-09-19T06:55:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a5ffff640 1 -- 172.21.15.103:0/2970640115 learned_addr learned my addr 172.21.15.103:0/2970640115 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a64a31640 1 --2- 172.21.15.103:0/2970640115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a60100e20 0x7f9a601973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a65232640 1 --2- 172.21.15.103:0/2970640115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a60102ef0 0x7f9a6019bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a5ffff640 1 -- 172.21.15.103:0/2970640115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a60100e20 msgr2=0x7f9a601973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a5ffff640 1 --2- 172.21.15.103:0/2970640115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a60100e20 0x7f9a601973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a5ffff640 1 -- 172.21.15.103:0/2970640115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a60102ef0 msgr2=0x7f9a6019bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.919+0000 7f9a5ffff640 1 --2- 172.21.15.103:0/2970640115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a60102ef0 0x7f9a6019bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.920+0000 7f9a5ffff640 1 -- 172.21.15.103:0/2970640115 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6019c350 con 0x7f9a60102570 2024-09-19T06:55:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.920+0000 7f9a5ffff640 1 --2- 172.21.15.103:0/2970640115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a60102570 0x7f9a601978e0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c006010 tx=0x7f9a4c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.920+0000 7f9a5dffb640 1 -- 172.21.15.103:0/2970640115 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c03f070 con 0x7f9a60102570 2024-09-19T06:55:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.920+0000 7f9a5dffb640 1 -- 172.21.15.103:0/2970640115 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a4c031c30 con 0x7f9a60102570 2024-09-19T06:55:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.920+0000 7f9a66c95640 1 -- 172.21.15.103:0/2970640115 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a6019c5e0 con 0x7f9a60102570 2024-09-19T06:55:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.920+0000 7f9a5dffb640 1 -- 172.21.15.103:0/2970640115 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c035030 con 0x7f9a60102570 2024-09-19T06:55:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.920+0000 7f9a66c95640 1 -- 172.21.15.103:0/2970640115 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a601a3f20 con 0x7f9a60102570 2024-09-19T06:55:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.922+0000 7f9a5dffb640 1 -- 172.21.15.103:0/2970640115 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9a4c042070 con 0x7f9a60102570 2024-09-19T06:55:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.923+0000 7f9a6906e740 1 Processor -- start 2024-09-19T06:55:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.923+0000 7f9a6906e740 1 -- start start 2024-09-19T06:55:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.923+0000 7f9a6906e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.923+0000 7f9a6906e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69c1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.923+0000 7f9a6906e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 0x5644e69ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.923+0000 7f9a6906e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5644e69252f0 con 0x5644e69c2d50 2024-09-19T06:55:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.923+0000 7f9a6906e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5644e68cccf0 con 0x5644e69c17e0 2024-09-19T06:55:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.923+0000 7f9a6906e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5644e69212b0 con 0x5644e69cc580 2024-09-19T06:55:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.923+0000 7f9a5dffb640 1 --2- 172.21.15.103:0/2970640115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9a500a33b0 0x7f9a500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.923+0000 7f9a65232640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.923+0000 7f9a5ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 0x5644e69ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.923+0000 7f9a64a31640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69c1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.923+0000 7f9a64a31640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69c1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45914/0 (socket says 172.21.15.103:45914) 2024-09-19T06:55:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.923+0000 7f9a64a31640 1 -- 172.21.15.103:0/2035616429 learned_addr learned my addr 172.21.15.103:0/2035616429 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.924+0000 7f9a65232640 1 --2- 172.21.15.103:0/2970640115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9a500a33b0 0x7f9a500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:24.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.924+0000 7f9a5dffb640 1 -- 172.21.15.103:0/2970640115 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f9a4c0f0570 con 0x7f9a60102570 2024-09-19T06:55:24.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.924+0000 7f9a65232640 1 -- 172.21.15.103:0/2035616429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 msgr2=0x5644e69ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:24.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.924+0000 7f9a65232640 1 --2- 172.21.15.103:0/2035616429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 0x5644e69ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.924+0000 7f9a65232640 1 -- 172.21.15.103:0/2035616429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 msgr2=0x5644e69c1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:24.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.924+0000 7f9a65232640 1 --2- 172.21.15.103:0/2035616429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69c1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.924+0000 7f9a65232640 1 -- 172.21.15.103:0/2035616429 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644e68b13a0 con 0x5644e69c2d50 2024-09-19T06:55:24.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.924+0000 7f9a64a31640 1 --2- 172.21.15.103:0/2035616429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69c1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:24.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.924+0000 7f9a65232640 1 --2- 172.21.15.103:0/2035616429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c12a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f9a58014900 tx=0x7f9a58014dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4842a35d005f2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:24.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.924+0000 7f9a65232640 1 --2- 172.21.15.103:0/2970640115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9a500a33b0 0x7f9a500a5870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9a58010000 tx=0x7f9a5801e000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:24.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.924+0000 7f9a4affd640 1 -- 172.21.15.103:0/2035616429 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58023070 con 0x5644e69c2d50 2024-09-19T06:55:24.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.924+0000 7f9a4affd640 1 -- 172.21.15.103:0/2035616429 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a58010c70 con 0x5644e69c2d50 2024-09-19T06:55:24.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.924+0000 7f9a4affd640 1 -- 172.21.15.103:0/2035616429 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5801ea90 con 0x5644e69c2d50 2024-09-19T06:55:24.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.925+0000 7f9a6906e740 1 -- 172.21.15.103:0/2035616429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 msgr2=0x5644e69c12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:24.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.925+0000 7f9a6906e740 1 --2- 172.21.15.103:0/2035616429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c12a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f9a58014900 tx=0x7f9a58014dc0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.925+0000 7f9a6906e740 1 -- 172.21.15.103:0/2035616429 shutdown_connections 2024-09-19T06:55:24.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.925+0000 7f9a6906e740 1 --2- 172.21.15.103:0/2035616429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 0x5644e69ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.925+0000 7f9a6906e740 1 --2- 172.21.15.103:0/2035616429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69c1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.925+0000 7f9a6906e740 1 --2- 172.21.15.103:0/2035616429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c12a0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.925+0000 7f9a6906e740 1 -- 172.21.15.103:0/2035616429 >> 172.21.15.103:0/2035616429 conn(0x5644e6988310 msgr2=0x5644e69bde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:24.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.925+0000 7f9a6906e740 1 -- 172.21.15.103:0/2035616429 shutdown_connections 2024-09-19T06:55:24.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.925+0000 7f9a6906e740 1 -- 172.21.15.103:0/2035616429 wait complete. 2024-09-19T06:55:24.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.925+0000 7f9a6906e740 1 -- start start 2024-09-19T06:55:24.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.925+0000 7f9a6906e740 1 Processor -- start 2024-09-19T06:55:24.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a6906e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a6906e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a6906e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 0x5644e69c9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a6906e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5644e69cf220 con 0x5644e69c2d50 2024-09-19T06:55:24.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a6906e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5644e68e0060 con 0x5644e69c17e0 2024-09-19T06:55:24.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a6906e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5644e69cf3a0 con 0x5644e69cc580 2024-09-19T06:55:24.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a6906e740 10 client.0 _ll_register_callbacks cb 0x5644e66338c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:24.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a5ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:24.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a5ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53368/0 (socket says 172.21.15.103:53368) 2024-09-19T06:55:24.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a5ffff640 1 -- 172.21.15.103:0/1931876470 learned_addr learned my addr 172.21.15.103:0/1931876470 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:24.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a65232640 1 --2- 172.21.15.103:0/1931876470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 0x5644e69c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:24.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a64a31640 1 --2- 172.21.15.103:0/1931876470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:24.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a5ffff640 1 -- 172.21.15.103:0/1931876470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 msgr2=0x5644e69c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:24.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a5ffff640 1 --2- 172.21.15.103:0/1931876470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 0x5644e69c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a5ffff640 1 -- 172.21.15.103:0/1931876470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 msgr2=0x5644e69cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:24.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a5ffff640 1 --2- 172.21.15.103:0/1931876470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a5ffff640 1 -- 172.21.15.103:0/1931876470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e66963c0 con 0x5644e69c2d50 2024-09-19T06:55:24.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a65232640 1 --2- 172.21.15.103:0/1931876470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 0x5644e69c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:24.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a5ffff640 1 --2- 172.21.15.103:0/1931876470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c9230 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c000c00 tx=0x7f9a4c0bc4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a6070c2f9be078 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:24.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:24.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.926+0000 7f9a3ffff640 1 -- 172.21.15.103:0/1931876470 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c049040 con 0x5644e69c2d50 2024-09-19T06:55:24.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.927+0000 7f9a3ffff640 1 -- 172.21.15.103:0/1931876470 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f9a4c0edf50 con 0x5644e69c2d50 2024-09-19T06:55:24.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.927+0000 7f9a6906e740 1 -- 172.21.15.103:0/1931876470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a58010c70 con 0x5644e69c2d50 2024-09-19T06:55:24.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.927+0000 7f9a3ffff640 1 -- 172.21.15.103:0/1931876470 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9a4c0ee150 con 0x5644e69c2d50 2024-09-19T06:55:24.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.927+0000 7f9a6906e740 10 client.17652 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:24.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.927+0000 7f9a6906e740 1 -- 172.21.15.103:0/1931876470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644e69dacc0 con 0x5644e69c2d50 2024-09-19T06:55:24.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.927+0000 7f9a3ffff640 1 -- 172.21.15.103:0/1931876470 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a58010c70 con 0x5644e69c2d50 2024-09-19T06:55:24.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.927+0000 7f9a3ffff640 1 -- 172.21.15.103:0/1931876470 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9a4c0bcac0 con 0x5644e69c2d50 2024-09-19T06:55:24.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.927+0000 7f9a6906e740 10 client.17652 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:24.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.927+0000 7f9a6906e740 10 client.17652 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:55:24.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.927+0000 7f9a6906e740 20 client.17652 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:24.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.927+0000 7f9a6906e740 1 --2- 172.21.15.103:0/1931876470 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5644e69dc170 0x5644e69de560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.927+0000 7f9a6906e740 4 client.17652 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:24.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.927+0000 7f9a6906e740 1 -- 172.21.15.103:0/1931876470 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644e6932860 con 0x5644e69dc170 2024-09-19T06:55:24.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.927+0000 7f9a64a31640 1 --2- 172.21.15.103:0/1931876470 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5644e69dc170 0x5644e69de560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:24.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.928+0000 7f9a64a31640 1 --2- 172.21.15.103:0/1931876470 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5644e69dc170 0x5644e69de560 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3327da362e2f6d61 server_cookie=849ffb9533fe4ba7 in_seq=0 out_seq=0 2024-09-19T06:55:24.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.928+0000 7f9a3ffff640 10 client.17652 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:55:24.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.929+0000 7f9a3ffff640 1 -- 172.21.15.103:0/1931876470 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644e6932860 con 0x5644e69dc170 2024-09-19T06:55:24.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.929+0000 7f9a3ffff640 10 client.17652 handle_command_reply: tid=0 2024-09-19T06:55:24.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.929+0000 7f9a6906e740 1 client.17652 shutdown 2024-09-19T06:55:24.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.929+0000 7f9a6906e740 1 -- 172.21.15.103:0/1931876470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 msgr2=0x5644e69c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:24.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.929+0000 7f9a6906e740 1 --2- 172.21.15.103:0/1931876470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c9230 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c000c00 tx=0x7f9a4c0bc4b0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.929+0000 7f9a6906e740 1 -- 172.21.15.103:0/1931876470 shutdown_connections 2024-09-19T06:55:24.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.929+0000 7f9a6906e740 1 --2- 172.21.15.103:0/1931876470 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5644e69dc170 0x5644e69de560 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.929+0000 7f9a3ffff640 0 client.17652 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:55:24.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.929+0000 7f9a6906e740 1 --2- 172.21.15.103:0/1931876470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 0x5644e69c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.929+0000 7f9a6906e740 1 --2- 172.21.15.103:0/1931876470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c9230 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.929+0000 7f9a6906e740 1 --2- 172.21.15.103:0/1931876470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.929+0000 7f9a6906e740 1 -- 172.21.15.103:0/1931876470 >> 172.21.15.103:0/1931876470 conn(0x5644e6988310 msgr2=0x5644e69c3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:24.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.929+0000 7f9a6906e740 1 -- 172.21.15.103:0/1931876470 shutdown_connections 2024-09-19T06:55:24.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.929+0000 7f9a6906e740 1 -- 172.21.15.103:0/1931876470 wait complete. 2024-09-19T06:55:24.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.929+0000 7f9a6906e740 20 client.17652 trim_cache size 0 max 16384 2024-09-19T06:55:24.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a6906e740 1 Processor -- start 2024-09-19T06:55:24.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a6906e740 1 -- start start 2024-09-19T06:55:24.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a6906e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a6906e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a6906e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 0x5644e69c4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a6906e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601ab570 con 0x5644e69c2d50 2024-09-19T06:55:24.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a6906e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601ad0b0 con 0x5644e69c17e0 2024-09-19T06:55:24.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a6906e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601abc40 con 0x5644e69cc580 2024-09-19T06:55:24.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a5ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:24.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a5ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53382/0 (socket says 172.21.15.103:53382) 2024-09-19T06:55:24.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a5ffff640 1 -- 172.21.15.103:0/4293464537 learned_addr learned my addr 172.21.15.103:0/4293464537 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:24.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a64a31640 1 --2- 172.21.15.103:0/4293464537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:24.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a5ffff640 1 -- 172.21.15.103:0/4293464537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 msgr2=0x5644e69c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:24.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a5ffff640 1 --2- 172.21.15.103:0/4293464537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 0x5644e69c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a5ffff640 1 -- 172.21.15.103:0/4293464537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 msgr2=0x5644e69cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:24.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a5ffff640 1 --2- 172.21.15.103:0/4293464537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a5ffff640 1 -- 172.21.15.103:0/4293464537 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644e69c14e0 con 0x5644e69c2d50 2024-09-19T06:55:24.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.942+0000 7f9a5ffff640 1 --2- 172.21.15.103:0/4293464537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c4330 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0c4fa0 tx=0x7f9a4c0b8b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18075e45b75f6c89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:24.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.943+0000 7f9a4a7fc640 1 -- 172.21.15.103:0/4293464537 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0c54d0 con 0x5644e69c2d50 2024-09-19T06:55:24.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.943+0000 7f9a4a7fc640 1 -- 172.21.15.103:0/4293464537 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a4c0c0c10 con 0x5644e69c2d50 2024-09-19T06:55:24.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.943+0000 7f9a4a7fc640 1 -- 172.21.15.103:0/4293464537 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0bdb90 con 0x5644e69c2d50 2024-09-19T06:55:24.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.943+0000 7f9a6906e740 1 -- 172.21.15.103:0/4293464537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 msgr2=0x5644e69c4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:24.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.943+0000 7f9a6906e740 1 --2- 172.21.15.103:0/4293464537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c4330 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0c4fa0 tx=0x7f9a4c0b8b00 comp rx=0 tx=0).stop 2024-09-19T06:55:24.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.943+0000 7f9a6906e740 1 -- 172.21.15.103:0/4293464537 shutdown_connections 2024-09-19T06:55:24.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.943+0000 7f9a6906e740 1 --2- 172.21.15.103:0/4293464537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 0x5644e69c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.943+0000 7f9a6906e740 1 --2- 172.21.15.103:0/4293464537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69c4330 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.943+0000 7f9a6906e740 1 --2- 172.21.15.103:0/4293464537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.943+0000 7f9a6906e740 1 -- 172.21.15.103:0/4293464537 >> 172.21.15.103:0/4293464537 conn(0x5644e6988310 msgr2=0x5644e69caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:24.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.943+0000 7f9a6906e740 1 -- 172.21.15.103:0/4293464537 shutdown_connections 2024-09-19T06:55:24.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.943+0000 7f9a6906e740 1 -- 172.21.15.103:0/4293464537 wait complete. 2024-09-19T06:55:24.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a6906e740 1 -- start start 2024-09-19T06:55:24.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a6906e740 1 Processor -- start 2024-09-19T06:55:24.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a6906e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69bc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a6906e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69bccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a6906e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 0x5644e69bd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a6906e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601ac9e0 con 0x5644e69c2d50 2024-09-19T06:55:24.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a6906e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601ac310 con 0x5644e69c17e0 2024-09-19T06:55:24.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a6906e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601a9530 con 0x5644e69cc580 2024-09-19T06:55:24.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a6906e740 10 client.0 _ll_register_callbacks cb 0x7f9a4c0bc6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:24.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a5ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69bccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:24.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a5ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69bccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53390/0 (socket says 172.21.15.103:53390) 2024-09-19T06:55:24.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a5ffff640 1 -- 172.21.15.103:0/2141716449 learned_addr learned my addr 172.21.15.103:0/2141716449 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:24.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a64a31640 1 --2- 172.21.15.103:0/2141716449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69bc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:24.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a5ffff640 1 -- 172.21.15.103:0/2141716449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 msgr2=0x5644e69bd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:24.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a5ffff640 1 --2- 172.21.15.103:0/2141716449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 0x5644e69bd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a5ffff640 1 -- 172.21.15.103:0/2141716449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 msgr2=0x5644e69bc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:24.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a5ffff640 1 --2- 172.21.15.103:0/2141716449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69bc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a5ffff640 1 -- 172.21.15.103:0/2141716449 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e69d4620 con 0x5644e69c2d50 2024-09-19T06:55:24.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.944+0000 7f9a64a31640 1 --2- 172.21.15.103:0/2141716449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69bc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:24.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.945+0000 7f9a5ffff640 1 --2- 172.21.15.103:0/2141716449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69bccf0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0096c0 tx=0x7f9a4c0b8c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf3961c443abb8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:24.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.945+0000 7f9a3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:24.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.945+0000 7f9a3ffff640 1 -- 172.21.15.103:0/2141716449 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0ee150 con 0x5644e69c2d50 2024-09-19T06:55:24.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.945+0000 7f9a3ffff640 1 -- 172.21.15.103:0/2141716449 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f9a4c0bdb90 con 0x5644e69c2d50 2024-09-19T06:55:24.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.945+0000 7f9a6906e740 1 -- 172.21.15.103:0/2141716449 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a4c0c0c10 con 0x5644e69c2d50 2024-09-19T06:55:24.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.945+0000 7f9a3ffff640 1 -- 172.21.15.103:0/2141716449 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9a4c0c54d0 con 0x5644e69c2d50 2024-09-19T06:55:24.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.945+0000 7f9a6906e740 10 client.17658 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:24.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.945+0000 7f9a6906e740 1 -- 172.21.15.103:0/2141716449 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644e6a495c0 con 0x5644e69c2d50 2024-09-19T06:55:24.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.945+0000 7f9a3ffff640 1 -- 172.21.15.103:0/2141716449 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a4c0c0c10 con 0x5644e69c2d50 2024-09-19T06:55:24.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.945+0000 7f9a3ffff640 1 -- 172.21.15.103:0/2141716449 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9a4c0bbdd0 con 0x5644e69c2d50 2024-09-19T06:55:24.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.945+0000 7f9a6906e740 10 client.17658 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:24.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.945+0000 7f9a6906e740 10 client.17658 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:55:24.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.945+0000 7f9a6906e740 20 client.17658 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:24.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.945+0000 7f9a6906e740 1 --2- 172.21.15.103:0/2141716449 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5644e69dc170 0x5644e6a4ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:24.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.945+0000 7f9a6906e740 4 client.17658 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:55:24.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.945+0000 7f9a6906e740 1 -- 172.21.15.103:0/2141716449 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644e692cd70 con 0x5644e69dc170 2024-09-19T06:55:24.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.946+0000 7f9a64a31640 1 --2- 172.21.15.103:0/2141716449 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5644e69dc170 0x5644e6a4ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:24.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.946+0000 7f9a64a31640 1 --2- 172.21.15.103:0/2141716449 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5644e69dc170 0x5644e6a4ba10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=675d385394c6b4e8 server_cookie=16e086c881dfe893 in_seq=0 out_seq=0 2024-09-19T06:55:24.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.946+0000 7f9a3ffff640 10 client.17658 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:55:24.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.946+0000 7f9a3ffff640 1 -- 172.21.15.103:0/2141716449 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644e692cd70 con 0x5644e69dc170 2024-09-19T06:55:24.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.946+0000 7f9a3ffff640 10 client.17658 handle_command_reply: tid=0 2024-09-19T06:55:24.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.946+0000 7f9a6906e740 1 client.17658 shutdown 2024-09-19T06:55:24.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.946+0000 7f9a6906e740 1 -- 172.21.15.103:0/2141716449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 msgr2=0x5644e69bccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:24.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.946+0000 7f9a6906e740 1 --2- 172.21.15.103:0/2141716449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69bccf0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0096c0 tx=0x7f9a4c0b8c30 comp rx=0 tx=0).stop 2024-09-19T06:55:24.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.946+0000 7f9a6906e740 1 -- 172.21.15.103:0/2141716449 shutdown_connections 2024-09-19T06:55:24.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.946+0000 7f9a6906e740 1 --2- 172.21.15.103:0/2141716449 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5644e69dc170 0x5644e6a4ba10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.946+0000 7f9a6906e740 1 --2- 172.21.15.103:0/2141716449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644e69cc580 0x5644e69bd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.946+0000 7f9a6906e740 1 --2- 172.21.15.103:0/2141716449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644e69c2d50 0x5644e69bccf0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.946+0000 7f9a6906e740 1 --2- 172.21.15.103:0/2141716449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e69c17e0 0x5644e69bc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.946+0000 7f9a6906e740 1 -- 172.21.15.103:0/2141716449 >> 172.21.15.103:0/2141716449 conn(0x5644e6988310 msgr2=0x5644e69cb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:24.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.947+0000 7f9a3ffff640 0 client.17658 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:55:24.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.947+0000 7f9a6906e740 1 -- 172.21.15.103:0/2141716449 shutdown_connections 2024-09-19T06:55:24.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.947+0000 7f9a6906e740 1 -- 172.21.15.103:0/2141716449 wait complete. 2024-09-19T06:55:24.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.947+0000 7f9a6906e740 20 client.17658 trim_cache size 0 max 16384 2024-09-19T06:55:24.956 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:55:24.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.947+0000 7f9a66c95640 1 -- 172.21.15.103:0/2970640115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9a500a33b0 msgr2=0x7f9a500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:24.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.947+0000 7f9a66c95640 1 --2- 172.21.15.103:0/2970640115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9a500a33b0 0x7f9a500a5870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9a58010000 tx=0x7f9a5801e000 comp rx=0 tx=0).stop 2024-09-19T06:55:24.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.947+0000 7f9a66c95640 1 -- 172.21.15.103:0/2970640115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a60102570 msgr2=0x7f9a601978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:24.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.947+0000 7f9a66c95640 1 --2- 172.21.15.103:0/2970640115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a60102570 0x7f9a601978e0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c006010 tx=0x7f9a4c003c10 comp rx=0 tx=0).stop 2024-09-19T06:55:24.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.947+0000 7f9a66c95640 1 -- 172.21.15.103:0/2970640115 shutdown_connections 2024-09-19T06:55:24.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.947+0000 7f9a66c95640 1 --2- 172.21.15.103:0/2970640115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9a500a33b0 0x7f9a500a5870 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.947+0000 7f9a66c95640 1 --2- 172.21.15.103:0/2970640115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a60102ef0 0x7f9a6019bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.947+0000 7f9a66c95640 1 --2- 172.21.15.103:0/2970640115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a60102570 0x7f9a601978e0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.947+0000 7f9a66c95640 1 --2- 172.21.15.103:0/2970640115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a60100e20 0x7f9a601973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:24.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.947+0000 7f9a66c95640 1 -- 172.21.15.103:0/2970640115 >> 172.21.15.103:0/2970640115 conn(0x7f9a600fc7d0 msgr2=0x7f9a600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:24.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.948+0000 7f9a66c95640 1 -- 172.21.15.103:0/2970640115 shutdown_connections 2024-09-19T06:55:24.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:24.948+0000 7f9a66c95640 1 -- 172.21.15.103:0/2970640115 wait complete. 2024-09-19T06:55:25.893 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:55:26.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:26 smithi103 ceph-mon[31158]: pgmap v189: 97 pgs: 97 active+clean; 2.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-09-19T06:55:26.144 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:26.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:26 smithi134 ceph-mon[35356]: pgmap v189: 97 pgs: 97 active+clean; 2.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-09-19T06:55:26.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:26 smithi174 ceph-mon[35637]: pgmap v189: 97 pgs: 97 active+clean; 2.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-09-19T06:55:26.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.624+0000 7fd09277b640 1 -- 172.21.15.103:0/337683510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd08c102ef0 msgr2=0x7fd08c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:26.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.624+0000 7fd09277b640 1 --2- 172.21.15.103:0/337683510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd08c102ef0 0x7fd08c10da40 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fd084009ca0 tx=0x7fd08402a950 comp rx=0 tx=0).stop 2024-09-19T06:55:26.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.624+0000 7fd09277b640 1 -- 172.21.15.103:0/337683510 shutdown_connections 2024-09-19T06:55:26.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.624+0000 7fd09277b640 1 --2- 172.21.15.103:0/337683510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd08c102ef0 0x7fd08c10da40 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.624+0000 7fd09277b640 1 --2- 172.21.15.103:0/337683510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd08c102570 0x7fd08c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.624+0000 7fd09277b640 1 --2- 172.21.15.103:0/337683510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd08c100e20 0x7fd08c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.624+0000 7fd09277b640 1 -- 172.21.15.103:0/337683510 >> 172.21.15.103:0/337683510 conn(0x7fd08c0fc7d0 msgr2=0x7fd08c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:26.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.624+0000 7fd09277b640 1 -- 172.21.15.103:0/337683510 shutdown_connections 2024-09-19T06:55:26.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.624+0000 7fd09277b640 1 -- 172.21.15.103:0/337683510 wait complete. 2024-09-19T06:55:26.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd09277b640 1 Processor -- start 2024-09-19T06:55:26.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd09277b640 1 -- start start 2024-09-19T06:55:26.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd09277b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd08c100e20 0x7fd08c10e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd09277b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd08c102570 0x7fd08c10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd09277b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd08c102ef0 0x7fd08c10ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd09277b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1120a0 con 0x7fd08c100e20 2024-09-19T06:55:26.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd09277b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c111f20 con 0x7fd08c102570 2024-09-19T06:55:26.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd09277b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c112220 con 0x7fd08c102ef0 2024-09-19T06:55:26.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd08bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd08c100e20 0x7fd08c10e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:26.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd08bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd08c100e20 0x7fd08c10e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53414/0 (socket says 172.21.15.103:53414) 2024-09-19T06:55:26.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd08bfff640 1 -- 172.21.15.103:0/1278768779 learned_addr learned my addr 172.21.15.103:0/1278768779 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:26.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd090d18640 1 --2- 172.21.15.103:0/1278768779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd08c102ef0 0x7fd08c10ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:26.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd083fff640 1 --2- 172.21.15.103:0/1278768779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd08c102570 0x7fd08c10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:26.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd08bfff640 1 -- 172.21.15.103:0/1278768779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd08c102ef0 msgr2=0x7fd08c10ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:26.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd08bfff640 1 --2- 172.21.15.103:0/1278768779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd08c102ef0 0x7fd08c10ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd08bfff640 1 -- 172.21.15.103:0/1278768779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd08c102570 msgr2=0x7fd08c10ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:26.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd08bfff640 1 --2- 172.21.15.103:0/1278768779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd08c102570 0x7fd08c10ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd08bfff640 1 -- 172.21.15.103:0/1278768779 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08c1a6920 con 0x7fd08c100e20 2024-09-19T06:55:26.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd083fff640 1 --2- 172.21.15.103:0/1278768779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd08c102570 0x7fd08c10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:26.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.625+0000 7fd08bfff640 1 --2- 172.21.15.103:0/1278768779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd08c100e20 0x7fd08c10e4d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00d950 tx=0x7fd07c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:26.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.626+0000 7fd089ffb640 1 -- 172.21.15.103:0/1278768779 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c017070 con 0x7fd08c100e20 2024-09-19T06:55:26.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.626+0000 7fd089ffb640 1 -- 172.21.15.103:0/1278768779 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07c009c30 con 0x7fd08c100e20 2024-09-19T06:55:26.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.626+0000 7fd09277b640 1 -- 172.21.15.103:0/1278768779 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08c1a6c10 con 0x7fd08c100e20 2024-09-19T06:55:26.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.626+0000 7fd089ffb640 1 -- 172.21.15.103:0/1278768779 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c00f6b0 con 0x7fd08c100e20 2024-09-19T06:55:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.626+0000 7fd09277b640 1 -- 172.21.15.103:0/1278768779 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08c1a71a0 con 0x7fd08c100e20 2024-09-19T06:55:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.628+0000 7fd089ffb640 1 -- 172.21.15.103:0/1278768779 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd07c0109d0 con 0x7fd08c100e20 2024-09-19T06:55:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.628+0000 7fd094b54740 1 Processor -- start 2024-09-19T06:55:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.628+0000 7fd094b54740 1 -- start start 2024-09-19T06:55:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd089ffb640 1 --2- 172.21.15.103:0/1278768779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd0780a3430 0x7fd0780a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd094b54740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca18b40 0x55b88ca17090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd094b54740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca175d0 0x55b88ca17a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd094b54740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca22370 0x55b88ca24760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd094b54740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b88c97d2f0 con 0x55b88ca18b40 2024-09-19T06:55:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd094b54740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b88c924cf0 con 0x55b88ca175d0 2024-09-19T06:55:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd094b54740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b88c9792b0 con 0x55b88ca22370 2024-09-19T06:55:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd090d18640 1 --2- 172.21.15.103:0/1278768779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd0780a3430 0x7fd0780a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd08bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca18b40 0x55b88ca17090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd089ffb640 1 -- 172.21.15.103:0/1278768779 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd07c0cf990 con 0x7fd08c100e20 2024-09-19T06:55:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd08bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca18b40 0x55b88ca17090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53418/0 (socket says 172.21.15.103:53418) 2024-09-19T06:55:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd08bfff640 1 -- 172.21.15.103:0/4020216917 learned_addr learned my addr 172.21.15.103:0/4020216917 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd090d18640 1 --2- 172.21.15.103:0/4020216917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca22370 0x55b88ca24760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd083fff640 1 --2- 172.21.15.103:0/4020216917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca175d0 0x55b88ca17a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd08bfff640 1 -- 172.21.15.103:0/4020216917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca22370 msgr2=0x55b88ca24760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd08bfff640 1 --2- 172.21.15.103:0/4020216917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca22370 0x55b88ca24760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd08bfff640 1 -- 172.21.15.103:0/4020216917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca175d0 msgr2=0x55b88ca17a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd08bfff640 1 --2- 172.21.15.103:0/4020216917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca175d0 0x55b88ca17a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd08bfff640 1 -- 172.21.15.103:0/4020216917 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b88c9093a0 con 0x55b88ca18b40 2024-09-19T06:55:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd083fff640 1 --2- 172.21.15.103:0/4020216917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca175d0 0x55b88ca17a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd090d18640 1 --2- 172.21.15.103:0/1278768779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd0780a3430 0x7fd0780a58f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd084005fa0 tx=0x7fd08402fa10 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd08bfff640 1 --2- 172.21.15.103:0/4020216917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca18b40 0x55b88ca17090 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009610 tx=0x7fd07c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40fbe139ad4b89e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd0827fc640 1 -- 172.21.15.103:0/4020216917 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c020020 con 0x55b88ca18b40 2024-09-19T06:55:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd0827fc640 1 -- 172.21.15.103:0/4020216917 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07c09c340 con 0x55b88ca18b40 2024-09-19T06:55:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.629+0000 7fd0827fc640 1 -- 172.21.15.103:0/4020216917 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0bd0b0 con 0x55b88ca18b40 2024-09-19T06:55:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.630+0000 7fd094b54740 1 -- 172.21.15.103:0/4020216917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca18b40 msgr2=0x55b88ca17090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.630+0000 7fd094b54740 1 --2- 172.21.15.103:0/4020216917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca18b40 0x55b88ca17090 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009610 tx=0x7fd07c005f90 comp rx=0 tx=0).stop 2024-09-19T06:55:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.630+0000 7fd094b54740 1 -- 172.21.15.103:0/4020216917 shutdown_connections 2024-09-19T06:55:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.630+0000 7fd094b54740 1 --2- 172.21.15.103:0/4020216917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca22370 0x55b88ca24760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.630+0000 7fd094b54740 1 --2- 172.21.15.103:0/4020216917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca175d0 0x55b88ca17a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.630+0000 7fd094b54740 1 --2- 172.21.15.103:0/4020216917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca18b40 0x55b88ca17090 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.630+0000 7fd094b54740 1 -- 172.21.15.103:0/4020216917 >> 172.21.15.103:0/4020216917 conn(0x55b88c9e0310 msgr2=0x55b88ca13c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.630+0000 7fd094b54740 1 -- 172.21.15.103:0/4020216917 shutdown_connections 2024-09-19T06:55:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.630+0000 7fd094b54740 1 -- 172.21.15.103:0/4020216917 wait complete. 2024-09-19T06:55:26.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.631+0000 7fd094b54740 1 -- start start 2024-09-19T06:55:26.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.631+0000 7fd094b54740 1 Processor -- start 2024-09-19T06:55:26.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.631+0000 7fd094b54740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca175d0 0x55b88ca21ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.631+0000 7fd094b54740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca18b40 0x55b88ca1f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.631+0000 7fd094b54740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca22370 0x55b88ca1f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.631+0000 7fd094b54740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b88ca24fb0 con 0x55b88ca175d0 2024-09-19T06:55:26.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.631+0000 7fd094b54740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b88c938060 con 0x55b88ca22370 2024-09-19T06:55:26.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.631+0000 7fd094b54740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b88ca25130 con 0x55b88ca18b40 2024-09-19T06:55:26.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.631+0000 7fd094b54740 10 client.0 _ll_register_callbacks cb 0x55b88c68b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:26.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.631+0000 7fd08bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca175d0 0x55b88ca21ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:26.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.631+0000 7fd08bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca175d0 0x55b88ca21ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53426/0 (socket says 172.21.15.103:53426) 2024-09-19T06:55:26.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.631+0000 7fd08bfff640 1 -- 172.21.15.103:0/3284851807 learned_addr learned my addr 172.21.15.103:0/3284851807 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:26.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.631+0000 7fd08bfff640 1 -- 172.21.15.103:0/3284851807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca18b40 msgr2=0x55b88ca1f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.631+0000 7fd08bfff640 1 --2- 172.21.15.103:0/3284851807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca18b40 0x55b88ca1f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.631+0000 7fd08bfff640 1 -- 172.21.15.103:0/3284851807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca22370 msgr2=0x55b88ca1f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.631+0000 7fd08bfff640 1 --2- 172.21.15.103:0/3284851807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca22370 0x55b88ca1f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd08bfff640 1 -- 172.21.15.103:0/3284851807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b88c6ee3c0 con 0x55b88ca175d0 2024-09-19T06:55:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd08bfff640 1 --2- 172.21.15.103:0/3284851807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca175d0 0x55b88ca21ce0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fd07c09bbd0 tx=0x7fd07c098560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59c8becdae138f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd06b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd06b7fe640 1 -- 172.21.15.103:0/3284851807 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c020070 con 0x55b88ca175d0 2024-09-19T06:55:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd06b7fe640 1 -- 172.21.15.103:0/3284851807 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd07c052130 con 0x55b88ca175d0 2024-09-19T06:55:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd094b54740 1 -- 172.21.15.103:0/3284851807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd07c09c340 con 0x55b88ca175d0 2024-09-19T06:55:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd06b7fe640 1 -- 172.21.15.103:0/3284851807 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd07c0cd540 con 0x55b88ca175d0 2024-09-19T06:55:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd094b54740 10 client.17670 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd094b54740 1 -- 172.21.15.103:0/3284851807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b88ca309f0 con 0x55b88ca175d0 2024-09-19T06:55:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd06b7fe640 1 -- 172.21.15.103:0/3284851807 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07c09c340 con 0x55b88ca175d0 2024-09-19T06:55:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd06b7fe640 1 -- 172.21.15.103:0/3284851807 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd07c09b090 con 0x55b88ca175d0 2024-09-19T06:55:26.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd094b54740 10 client.17670 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:26.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd094b54740 10 client.17670 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:55:26.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd094b54740 20 client.17670 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:26.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd094b54740 1 --2- 172.21.15.103:0/3284851807 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55b88ca31d40 0x55b88ca34130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd094b54740 4 client.17670 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:26.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.632+0000 7fd094b54740 1 -- 172.21.15.103:0/3284851807 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b88c98a860 con 0x55b88ca31d40 2024-09-19T06:55:26.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.633+0000 7fd083fff640 1 --2- 172.21.15.103:0/3284851807 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55b88ca31d40 0x55b88ca34130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:26.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.633+0000 7fd06b7fe640 10 client.17670 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:55:26.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.633+0000 7fd083fff640 1 --2- 172.21.15.103:0/3284851807 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55b88ca31d40 0x55b88ca34130 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ece2bff381c8dc02 server_cookie=fb81842ffa1899fe in_seq=0 out_seq=0 2024-09-19T06:55:26.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.634+0000 7fd06b7fe640 1 -- 172.21.15.103:0/3284851807 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b88c98a860 con 0x55b88ca31d40 2024-09-19T06:55:26.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.634+0000 7fd06b7fe640 10 client.17670 handle_command_reply: tid=0 2024-09-19T06:55:26.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.634+0000 7fd094b54740 1 client.17670 shutdown 2024-09-19T06:55:26.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.634+0000 7fd094b54740 1 -- 172.21.15.103:0/3284851807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca175d0 msgr2=0x55b88ca21ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:26.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.634+0000 7fd094b54740 1 --2- 172.21.15.103:0/3284851807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca175d0 0x55b88ca21ce0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fd07c09bbd0 tx=0x7fd07c098560 comp rx=0 tx=0).stop 2024-09-19T06:55:26.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.634+0000 7fd094b54740 1 -- 172.21.15.103:0/3284851807 shutdown_connections 2024-09-19T06:55:26.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.634+0000 7fd094b54740 1 --2- 172.21.15.103:0/3284851807 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55b88ca31d40 0x55b88ca34130 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.634+0000 7fd094b54740 1 --2- 172.21.15.103:0/3284851807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca22370 0x55b88ca1f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.634+0000 7fd094b54740 1 --2- 172.21.15.103:0/3284851807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca18b40 0x55b88ca1f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.634+0000 7fd094b54740 1 --2- 172.21.15.103:0/3284851807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca175d0 0x55b88ca21ce0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.634+0000 7fd094b54740 1 -- 172.21.15.103:0/3284851807 >> 172.21.15.103:0/3284851807 conn(0x55b88c9e0310 msgr2=0x55b88ca19990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:26.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.634+0000 7fd06b7fe640 0 client.17670 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:55:26.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.634+0000 7fd094b54740 1 -- 172.21.15.103:0/3284851807 shutdown_connections 2024-09-19T06:55:26.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.634+0000 7fd094b54740 1 -- 172.21.15.103:0/3284851807 wait complete. 2024-09-19T06:55:26.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.634+0000 7fd094b54740 20 client.17670 trim_cache size 0 max 16384 2024-09-19T06:55:26.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd094b54740 1 Processor -- start 2024-09-19T06:55:26.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd094b54740 1 -- start start 2024-09-19T06:55:26.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd094b54740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca175d0 0x55b88ca2e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd094b54740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca18b40 0x55b88ca1a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd094b54740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca22370 0x55b88ca1a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd094b54740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1afac0 con 0x55b88ca175d0 2024-09-19T06:55:26.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd094b54740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1b1600 con 0x55b88ca22370 2024-09-19T06:55:26.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd094b54740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1b0190 con 0x55b88ca18b40 2024-09-19T06:55:26.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd08bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca175d0 0x55b88ca2e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:26.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd08bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca175d0 0x55b88ca2e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53438/0 (socket says 172.21.15.103:53438) 2024-09-19T06:55:26.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd08bfff640 1 -- 172.21.15.103:0/3703026615 learned_addr learned my addr 172.21.15.103:0/3703026615 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:26.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd083fff640 1 --2- 172.21.15.103:0/3703026615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca18b40 0x55b88ca1a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:26.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd090d18640 1 --2- 172.21.15.103:0/3703026615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca22370 0x55b88ca1a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:26.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd08bfff640 1 -- 172.21.15.103:0/3703026615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca18b40 msgr2=0x55b88ca1a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:26.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd08bfff640 1 --2- 172.21.15.103:0/3703026615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca18b40 0x55b88ca1a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd08bfff640 1 -- 172.21.15.103:0/3703026615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca22370 msgr2=0x55b88ca1a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:26.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd08bfff640 1 --2- 172.21.15.103:0/3703026615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca22370 0x55b88ca1a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.647+0000 7fd08bfff640 1 -- 172.21.15.103:0/3703026615 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b88ca1d580 con 0x55b88ca175d0 2024-09-19T06:55:26.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.648+0000 7fd090d18640 1 --2- 172.21.15.103:0/3703026615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca22370 0x55b88ca1a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:26.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.648+0000 7fd08bfff640 1 --2- 172.21.15.103:0/3703026615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca175d0 0x55b88ca2e280 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd07c098610 tx=0x7fd07c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81438cc2eeba97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:26.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.648+0000 7fd081ffb640 1 -- 172.21.15.103:0/3703026615 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c020070 con 0x55b88ca175d0 2024-09-19T06:55:26.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.648+0000 7fd081ffb640 1 -- 172.21.15.103:0/3703026615 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07c0c6e50 con 0x55b88ca175d0 2024-09-19T06:55:26.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.648+0000 7fd081ffb640 1 -- 172.21.15.103:0/3703026615 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c086310 con 0x55b88ca175d0 2024-09-19T06:55:26.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.648+0000 7fd094b54740 1 -- 172.21.15.103:0/3703026615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca175d0 msgr2=0x55b88ca2e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:26.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.648+0000 7fd094b54740 1 --2- 172.21.15.103:0/3703026615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca175d0 0x55b88ca2e280 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd07c098610 tx=0x7fd07c009f50 comp rx=0 tx=0).stop 2024-09-19T06:55:26.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.649+0000 7fd094b54740 1 -- 172.21.15.103:0/3703026615 shutdown_connections 2024-09-19T06:55:26.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.649+0000 7fd094b54740 1 --2- 172.21.15.103:0/3703026615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca22370 0x55b88ca1a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.649+0000 7fd094b54740 1 --2- 172.21.15.103:0/3703026615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca18b40 0x55b88ca1a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.649+0000 7fd094b54740 1 --2- 172.21.15.103:0/3703026615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca175d0 0x55b88ca2e280 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.649+0000 7fd094b54740 1 -- 172.21.15.103:0/3703026615 >> 172.21.15.103:0/3703026615 conn(0x55b88c9e0310 msgr2=0x55b88ca1e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:26.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.649+0000 7fd094b54740 1 -- 172.21.15.103:0/3703026615 shutdown_connections 2024-09-19T06:55:26.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.649+0000 7fd094b54740 1 -- 172.21.15.103:0/3703026615 wait complete. 2024-09-19T06:55:26.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.649+0000 7fd094b54740 1 -- start start 2024-09-19T06:55:26.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.649+0000 7fd094b54740 1 Processor -- start 2024-09-19T06:55:26.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.649+0000 7fd094b54740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca175d0 0x55b88ca25b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.649+0000 7fd094b54740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca18b40 0x55b88ca26070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.649+0000 7fd094b54740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca22370 0x55b88ca9a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.649+0000 7fd094b54740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1b0f30 con 0x55b88ca22370 2024-09-19T06:55:26.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.649+0000 7fd094b54740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1b0860 con 0x55b88ca18b40 2024-09-19T06:55:26.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.649+0000 7fd094b54740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c1ada80 con 0x55b88ca175d0 2024-09-19T06:55:26.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.649+0000 7fd094b54740 10 client.0 _ll_register_callbacks cb 0x7fd07c09b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:26.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd090d18640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca22370 0x55b88ca9a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:26.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd083fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca18b40 0x55b88ca26070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:26.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd090d18640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca22370 0x55b88ca9a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53444/0 (socket says 172.21.15.103:53444) 2024-09-19T06:55:26.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd090d18640 1 -- 172.21.15.103:0/2603179083 learned_addr learned my addr 172.21.15.103:0/2603179083 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:26.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd08bfff640 1 --2- 172.21.15.103:0/2603179083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca175d0 0x55b88ca25b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:26.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd090d18640 1 -- 172.21.15.103:0/2603179083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca175d0 msgr2=0x55b88ca25b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:26.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd090d18640 1 --2- 172.21.15.103:0/2603179083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca175d0 0x55b88ca25b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd090d18640 1 -- 172.21.15.103:0/2603179083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca18b40 msgr2=0x55b88ca26070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:26.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd090d18640 1 --2- 172.21.15.103:0/2603179083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca18b40 0x55b88ca26070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd090d18640 1 -- 172.21.15.103:0/2603179083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b88ca9ae70 con 0x55b88ca22370 2024-09-19T06:55:26.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd090d18640 1 --2- 172.21.15.103:0/2603179083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca22370 0x55b88ca9a780 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fd084009ca0 tx=0x7fd0840095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347151f9a5b517b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:26.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd06b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:26.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd06b7fe640 1 -- 172.21.15.103:0/2603179083 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08404e020 con 0x55b88ca22370 2024-09-19T06:55:26.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd094b54740 1 -- 172.21.15.103:0/2603179083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd07c0c6e50 con 0x55b88ca22370 2024-09-19T06:55:26.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd06b7fe640 1 -- 172.21.15.103:0/2603179083 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fd084004230 con 0x55b88ca22370 2024-09-19T06:55:26.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd06b7fe640 1 -- 172.21.15.103:0/2603179083 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd084059050 con 0x55b88ca22370 2024-09-19T06:55:26.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd094b54740 10 client.17676 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:26.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.650+0000 7fd094b54740 1 -- 172.21.15.103:0/2603179083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b88ca9c0c0 con 0x55b88ca22370 2024-09-19T06:55:26.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.651+0000 7fd06b7fe640 1 -- 172.21.15.103:0/2603179083 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07c0c6e50 con 0x55b88ca22370 2024-09-19T06:55:26.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.651+0000 7fd06b7fe640 1 -- 172.21.15.103:0/2603179083 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd08403da30 con 0x55b88ca22370 2024-09-19T06:55:26.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.651+0000 7fd094b54740 10 client.17676 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:26.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.651+0000 7fd094b54740 10 client.17676 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:55:26.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.651+0000 7fd094b54740 20 client.17676 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:26.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.651+0000 7fd094b54740 1 --2- 172.21.15.103:0/2603179083 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55b88ca9c890 0x55b88ca9ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:26.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.651+0000 7fd094b54740 4 client.17676 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:55:26.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.651+0000 7fd094b54740 1 -- 172.21.15.103:0/2603179083 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b88c984d70 con 0x55b88ca9c890 2024-09-19T06:55:26.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.651+0000 7fd08bfff640 1 --2- 172.21.15.103:0/2603179083 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55b88ca9c890 0x55b88ca9ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:26.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.652+0000 7fd06b7fe640 10 client.17676 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:55:26.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.651+0000 7fd08bfff640 1 --2- 172.21.15.103:0/2603179083 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55b88ca9c890 0x55b88ca9ecb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc8dca56cf8a74df server_cookie=85e58d4a2baa0a47 in_seq=0 out_seq=0 2024-09-19T06:55:26.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.652+0000 7fd06b7fe640 1 -- 172.21.15.103:0/2603179083 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b88c984d70 con 0x55b88ca9c890 2024-09-19T06:55:26.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.652+0000 7fd06b7fe640 10 client.17676 handle_command_reply: tid=0 2024-09-19T06:55:26.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.652+0000 7fd094b54740 1 client.17676 shutdown 2024-09-19T06:55:26.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.652+0000 7fd094b54740 1 -- 172.21.15.103:0/2603179083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca22370 msgr2=0x55b88ca9a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:26.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.652+0000 7fd094b54740 1 --2- 172.21.15.103:0/2603179083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca22370 0x55b88ca9a780 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fd084009ca0 tx=0x7fd0840095c0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.652+0000 7fd094b54740 1 -- 172.21.15.103:0/2603179083 shutdown_connections 2024-09-19T06:55:26.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.652+0000 7fd094b54740 1 --2- 172.21.15.103:0/2603179083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b88ca22370 0x55b88ca9a780 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.652+0000 7fd094b54740 1 --2- 172.21.15.103:0/2603179083 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55b88ca9c890 0x55b88ca9ecb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.652+0000 7fd094b54740 1 --2- 172.21.15.103:0/2603179083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b88ca18b40 0x55b88ca26070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.652+0000 7fd094b54740 1 --2- 172.21.15.103:0/2603179083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b88ca175d0 0x55b88ca25b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.652+0000 7fd06b7fe640 0 client.17676 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:55:26.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.652+0000 7fd094b54740 1 -- 172.21.15.103:0/2603179083 >> 172.21.15.103:0/2603179083 conn(0x55b88c9e0310 msgr2=0x55b88ca124e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:26.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.652+0000 7fd094b54740 1 -- 172.21.15.103:0/2603179083 shutdown_connections 2024-09-19T06:55:26.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.652+0000 7fd094b54740 1 -- 172.21.15.103:0/2603179083 wait complete. 2024-09-19T06:55:26.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.652+0000 7fd094b54740 20 client.17676 trim_cache size 0 max 16384 2024-09-19T06:55:26.664 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:55:26.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.653+0000 7fd09277b640 1 -- 172.21.15.103:0/1278768779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd0780a3430 msgr2=0x7fd0780a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:26.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.653+0000 7fd09277b640 1 --2- 172.21.15.103:0/1278768779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd0780a3430 0x7fd0780a58f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd084005fa0 tx=0x7fd08402fa10 comp rx=0 tx=0).stop 2024-09-19T06:55:26.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.653+0000 7fd09277b640 1 -- 172.21.15.103:0/1278768779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd08c100e20 msgr2=0x7fd08c10e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:26.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.653+0000 7fd09277b640 1 --2- 172.21.15.103:0/1278768779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd08c100e20 0x7fd08c10e4d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00d950 tx=0x7fd07c00de10 comp rx=0 tx=0).stop 2024-09-19T06:55:26.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.653+0000 7fd09277b640 1 -- 172.21.15.103:0/1278768779 shutdown_connections 2024-09-19T06:55:26.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.653+0000 7fd09277b640 1 --2- 172.21.15.103:0/1278768779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd0780a3430 0x7fd0780a58f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.653+0000 7fd09277b640 1 --2- 172.21.15.103:0/1278768779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd08c102ef0 0x7fd08c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.653+0000 7fd09277b640 1 --2- 172.21.15.103:0/1278768779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd08c102570 0x7fd08c10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.653+0000 7fd09277b640 1 --2- 172.21.15.103:0/1278768779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd08c100e20 0x7fd08c10e4d0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:26.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.653+0000 7fd09277b640 1 -- 172.21.15.103:0/1278768779 >> 172.21.15.103:0/1278768779 conn(0x7fd08c0fc7d0 msgr2=0x7fd08c1068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:26.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.653+0000 7fd09277b640 1 -- 172.21.15.103:0/1278768779 shutdown_connections 2024-09-19T06:55:26.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:26.653+0000 7fd09277b640 1 -- 172.21.15.103:0/1278768779 wait complete. 2024-09-19T06:55:27.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:27 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:55:27.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:27 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:55:27.493 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:55:27.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:27 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:55:27.761 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:28.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:28 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:55:28.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:28 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:55:28.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:28 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:55:28.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:28 smithi103 ceph-mon[31158]: pgmap v190: 97 pgs: 97 active+clean; 2.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-09-19T06:55:28.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.250+0000 7f86fb3a0640 1 -- 172.21.15.103:0/3761223323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86f4102790 msgr2=0x7f86f41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:28.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.250+0000 7f86fb3a0640 1 --2- 172.21.15.103:0/3761223323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86f4102790 0x7f86f41067e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009900 tx=0x7f86e402a410 comp rx=0 tx=0).stop 2024-09-19T06:55:28.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.251+0000 7f86fb3a0640 1 -- 172.21.15.103:0/3761223323 shutdown_connections 2024-09-19T06:55:28.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.251+0000 7f86fb3a0640 1 --2- 172.21.15.103:0/3761223323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86f4106f10 0x7f86f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.251+0000 7f86fb3a0640 1 --2- 172.21.15.103:0/3761223323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86f4102790 0x7f86f41067e0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.251+0000 7f86fb3a0640 1 --2- 172.21.15.103:0/3761223323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f4101de0 0x7f86f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.251+0000 7f86fb3a0640 1 -- 172.21.15.103:0/3761223323 >> 172.21.15.103:0/3761223323 conn(0x7f86f40fd8c0 msgr2=0x7f86f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:28.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.251+0000 7f86fb3a0640 1 -- 172.21.15.103:0/3761223323 shutdown_connections 2024-09-19T06:55:28.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.251+0000 7f86fb3a0640 1 -- 172.21.15.103:0/3761223323 wait complete. 2024-09-19T06:55:28.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.251+0000 7f86fb3a0640 1 Processor -- start 2024-09-19T06:55:28.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.251+0000 7f86fb3a0640 1 -- start start 2024-09-19T06:55:28.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.251+0000 7f86fb3a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86f4101de0 0x7f86f41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.251+0000 7f86fb3a0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86f4102790 0x7f86f4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.251+0000 7f86fb3a0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f4106f10 0x7f86f419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.251+0000 7f86fb3a0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86f4112010 con 0x7f86f4101de0 2024-09-19T06:55:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.251+0000 7f86fb3a0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86f4111e90 con 0x7f86f4106f10 2024-09-19T06:55:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.251+0000 7f86fb3a0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86f4112190 con 0x7f86f4102790 2024-09-19T06:55:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.252+0000 7f86f913c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86f4101de0 0x7f86f41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.252+0000 7f86f913c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86f4101de0 0x7f86f41985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34854/0 (socket says 172.21.15.103:34854) 2024-09-19T06:55:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.252+0000 7f86f913c640 1 -- 172.21.15.103:0/3197192335 learned_addr learned my addr 172.21.15.103:0/3197192335 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.252+0000 7f86f893b640 1 --2- 172.21.15.103:0/3197192335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86f4102790 0x7f86f4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.252+0000 7f86f913c640 1 -- 172.21.15.103:0/3197192335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86f4102790 msgr2=0x7f86f4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.252+0000 7f86f913c640 1 --2- 172.21.15.103:0/3197192335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86f4102790 0x7f86f4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.252+0000 7f86f993d640 1 --2- 172.21.15.103:0/3197192335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f4106f10 0x7f86f419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.252+0000 7f86f913c640 1 -- 172.21.15.103:0/3197192335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f4106f10 msgr2=0x7f86f419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.252+0000 7f86f913c640 1 --2- 172.21.15.103:0/3197192335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f4106f10 0x7f86f419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.252+0000 7f86f913c640 1 -- 172.21.15.103:0/3197192335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f419d560 con 0x7f86f4101de0 2024-09-19T06:55:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.252+0000 7f86f913c640 1 --2- 172.21.15.103:0/3197192335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86f4101de0 0x7f86f41985b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f86f000b4a0 tx=0x7f86f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.252+0000 7f86ea7fc640 1 -- 172.21.15.103:0/3197192335 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0014070 con 0x7f86f4101de0 2024-09-19T06:55:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.252+0000 7f86ea7fc640 1 -- 172.21.15.103:0/3197192335 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f0010af0 con 0x7f86f4101de0 2024-09-19T06:55:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.252+0000 7f86fb3a0640 1 -- 172.21.15.103:0/3197192335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f419d850 con 0x7f86f4101de0 2024-09-19T06:55:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.252+0000 7f86ea7fc640 1 -- 172.21.15.103:0/3197192335 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0017d10 con 0x7f86f4101de0 2024-09-19T06:55:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.252+0000 7f86fb3a0640 1 -- 172.21.15.103:0/3197192335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f41a5130 con 0x7f86f4101de0 2024-09-19T06:55:28.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.254+0000 7f86ea7fc640 1 -- 172.21.15.103:0/3197192335 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f86f0017070 con 0x7f86f4101de0 2024-09-19T06:55:28.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.255+0000 7f86fd779740 1 Processor -- start 2024-09-19T06:55:28.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.255+0000 7f86fd779740 1 -- start start 2024-09-19T06:55:28.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.255+0000 7f86fd779740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb60b40 0x56051fb5f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.255+0000 7f86fd779740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb5f5d0 0x56051fb5fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.255+0000 7f86fd779740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb6a370 0x56051fb6c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.255+0000 7f86fd779740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56051fac52f0 con 0x56051fb6a370 2024-09-19T06:55:28.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.255+0000 7f86fd779740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56051fa6ccf0 con 0x56051fb60b40 2024-09-19T06:55:28.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.255+0000 7f86fd779740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56051fac12b0 con 0x56051fb5f5d0 2024-09-19T06:55:28.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.255+0000 7f86ea7fc640 1 --2- 172.21.15.103:0/3197192335 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f86e00a3430 0x7f86e00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.255+0000 7f86f893b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb6a370 0x56051fb6c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:28.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.255+0000 7f86f893b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb6a370 0x56051fb6c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34864/0 (socket says 172.21.15.103:34864) 2024-09-19T06:55:28.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.256+0000 7f86f893b640 1 -- 172.21.15.103:0/4251454579 learned_addr learned my addr 172.21.15.103:0/4251454579 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:28.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.256+0000 7f86f993d640 1 --2- 172.21.15.103:0/4251454579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb60b40 0x56051fb5f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:28.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.256+0000 7f86f913c640 1 --2- 172.21.15.103:0/4251454579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb5f5d0 0x56051fb5fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:28.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.256+0000 7f86f993d640 1 --2- 172.21.15.103:0/3197192335 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f86e00a3430 0x7f86e00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:28.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.256+0000 7f86f893b640 1 -- 172.21.15.103:0/4251454579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb5f5d0 msgr2=0x56051fb5fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:28.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.256+0000 7f86ea7fc640 1 -- 172.21.15.103:0/3197192335 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f86f00ceef0 con 0x7f86f4101de0 2024-09-19T06:55:28.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.256+0000 7f86f893b640 1 --2- 172.21.15.103:0/4251454579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb5f5d0 0x56051fb5fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.256+0000 7f86f893b640 1 -- 172.21.15.103:0/4251454579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb60b40 msgr2=0x56051fb5f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:28.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.256+0000 7f86f893b640 1 --2- 172.21.15.103:0/4251454579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb60b40 0x56051fb5f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.256+0000 7f86f893b640 1 -- 172.21.15.103:0/4251454579 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56051fa513a0 con 0x56051fb6a370 2024-09-19T06:55:28.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.256+0000 7f86f893b640 1 --2- 172.21.15.103:0/4251454579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb6a370 0x56051fb6c760 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009900 tx=0x7f86e40316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c1fc0d4c0ed3fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:28.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.256+0000 7f86f993d640 1 --2- 172.21.15.103:0/3197192335 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f86e00a3430 0x7f86e00a58f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f86ec014960 tx=0x7f86ec005df0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:28.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.256+0000 7f86df7fe640 1 -- 172.21.15.103:0/4251454579 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4040070 con 0x56051fb6a370 2024-09-19T06:55:28.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.256+0000 7f86df7fe640 1 -- 172.21.15.103:0/4251454579 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e4005020 con 0x56051fb6a370 2024-09-19T06:55:28.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.256+0000 7f86df7fe640 1 -- 172.21.15.103:0/4251454579 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4035040 con 0x56051fb6a370 2024-09-19T06:55:28.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.257+0000 7f86fd779740 1 -- 172.21.15.103:0/4251454579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb6a370 msgr2=0x56051fb6c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:28.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.257+0000 7f86fd779740 1 --2- 172.21.15.103:0/4251454579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb6a370 0x56051fb6c760 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009900 tx=0x7f86e40316e0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.257+0000 7f86fd779740 1 -- 172.21.15.103:0/4251454579 shutdown_connections 2024-09-19T06:55:28.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.257+0000 7f86fd779740 1 --2- 172.21.15.103:0/4251454579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb6a370 0x56051fb6c760 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.257+0000 7f86fd779740 1 --2- 172.21.15.103:0/4251454579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb5f5d0 0x56051fb5fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.257+0000 7f86fd779740 1 --2- 172.21.15.103:0/4251454579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb60b40 0x56051fb5f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.257+0000 7f86fd779740 1 -- 172.21.15.103:0/4251454579 >> 172.21.15.103:0/4251454579 conn(0x56051fb28310 msgr2=0x56051fb5bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:28.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.257+0000 7f86fd779740 1 -- 172.21.15.103:0/4251454579 shutdown_connections 2024-09-19T06:55:28.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.257+0000 7f86fd779740 1 -- 172.21.15.103:0/4251454579 wait complete. 2024-09-19T06:55:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.257+0000 7f86fd779740 1 -- start start 2024-09-19T06:55:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86fd779740 1 Processor -- start 2024-09-19T06:55:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86fd779740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb5f5d0 0x56051fb69ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86fd779740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb60b40 0x56051fb66f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86fd779740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb6a370 0x56051fb674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86fd779740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56051fb6d010 con 0x56051fb6a370 2024-09-19T06:55:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86fd779740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56051fa80060 con 0x56051fb5f5d0 2024-09-19T06:55:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86fd779740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56051fb6d190 con 0x56051fb60b40 2024-09-19T06:55:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86fd779740 10 client.0 _ll_register_callbacks cb 0x56051f7d38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86f993d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb6a370 0x56051fb674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86f993d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb6a370 0x56051fb674f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34878/0 (socket says 172.21.15.103:34878) 2024-09-19T06:55:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86f993d640 1 -- 172.21.15.103:0/3545894513 learned_addr learned my addr 172.21.15.103:0/3545894513 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86f913c640 1 --2- 172.21.15.103:0/3545894513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb5f5d0 0x56051fb69ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86f893b640 1 --2- 172.21.15.103:0/3545894513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb60b40 0x56051fb66f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86f993d640 1 -- 172.21.15.103:0/3545894513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb60b40 msgr2=0x56051fb66f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86f993d640 1 --2- 172.21.15.103:0/3545894513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb60b40 0x56051fb66f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86f993d640 1 -- 172.21.15.103:0/3545894513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb5f5d0 msgr2=0x56051fb69ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86f993d640 1 --2- 172.21.15.103:0/3545894513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb5f5d0 0x56051fb69ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86f993d640 1 -- 172.21.15.103:0/3545894513 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56051f8363c0 con 0x56051fb6a370 2024-09-19T06:55:28.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.258+0000 7f86f913c640 1 --2- 172.21.15.103:0/3545894513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb5f5d0 0x56051fb69ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:28.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.259+0000 7f86f993d640 1 --2- 172.21.15.103:0/3545894513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb6a370 0x56051fb674f0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f86ec01a6d0 tx=0x7f86ec004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be4a92552a8dcab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:28.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.259+0000 7f86dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:28.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.259+0000 7f86dcff9640 1 -- 172.21.15.103:0/3545894513 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec028060 con 0x56051fb6a370 2024-09-19T06:55:28.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.259+0000 7f86dcff9640 1 -- 172.21.15.103:0/3545894513 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f86ec014e20 con 0x56051fb6a370 2024-09-19T06:55:28.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.259+0000 7f86fd779740 1 -- 172.21.15.103:0/3545894513 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86e4005020 con 0x56051fb6a370 2024-09-19T06:55:28.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.259+0000 7f86dcff9640 1 -- 172.21.15.103:0/3545894513 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f86ec006720 con 0x56051fb6a370 2024-09-19T06:55:28.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.259+0000 7f86fd779740 10 client.17688 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:28.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.259+0000 7f86fd779740 1 -- 172.21.15.103:0/3545894513 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56051fb78a20 con 0x56051fb6a370 2024-09-19T06:55:28.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.259+0000 7f86dcff9640 1 -- 172.21.15.103:0/3545894513 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e4005020 con 0x56051fb6a370 2024-09-19T06:55:28.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.259+0000 7f86dcff9640 1 -- 172.21.15.103:0/3545894513 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f86ec0173c0 con 0x56051fb6a370 2024-09-19T06:55:28.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.259+0000 7f86fd779740 10 client.17688 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:28.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.259+0000 7f86fd779740 10 client.17688 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:55:28.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.259+0000 7f86fd779740 20 client.17688 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:28.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.259+0000 7f86fd779740 1 --2- 172.21.15.103:0/3545894513 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56051fb79ee0 0x56051fb7c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.260+0000 7f86fd779740 4 client.17688 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:28.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.260+0000 7f86fd779740 1 -- 172.21.15.103:0/3545894513 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56051fad2860 con 0x56051fb79ee0 2024-09-19T06:55:28.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.260+0000 7f86f913c640 1 --2- 172.21.15.103:0/3545894513 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56051fb79ee0 0x56051fb7c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:28.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.260+0000 7f86dcff9640 10 client.17688 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:55:28.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.260+0000 7f86f913c640 1 --2- 172.21.15.103:0/3545894513 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56051fb79ee0 0x56051fb7c380 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7045e91d3a80f4ca server_cookie=9de362687f185934 in_seq=0 out_seq=0 2024-09-19T06:55:28.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.261+0000 7f86dcff9640 1 -- 172.21.15.103:0/3545894513 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56051fad2860 con 0x56051fb79ee0 2024-09-19T06:55:28.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.261+0000 7f86dcff9640 10 client.17688 handle_command_reply: tid=0 2024-09-19T06:55:28.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.261+0000 7f86fd779740 1 client.17688 shutdown 2024-09-19T06:55:28.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.261+0000 7f86fd779740 1 -- 172.21.15.103:0/3545894513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb6a370 msgr2=0x56051fb674f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:28.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.261+0000 7f86fd779740 1 --2- 172.21.15.103:0/3545894513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb6a370 0x56051fb674f0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f86ec01a6d0 tx=0x7f86ec004b50 comp rx=0 tx=0).stop 2024-09-19T06:55:28.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.261+0000 7f86fd779740 1 -- 172.21.15.103:0/3545894513 shutdown_connections 2024-09-19T06:55:28.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.261+0000 7f86fd779740 1 --2- 172.21.15.103:0/3545894513 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56051fb79ee0 0x56051fb7c380 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.261+0000 7f86fd779740 1 --2- 172.21.15.103:0/3545894513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb6a370 0x56051fb674f0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.261+0000 7f86dcff9640 0 client.17688 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:55:28.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.261+0000 7f86fd779740 1 --2- 172.21.15.103:0/3545894513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb60b40 0x56051fb66f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.261+0000 7f86fd779740 1 --2- 172.21.15.103:0/3545894513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb5f5d0 0x56051fb69ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.261+0000 7f86fd779740 1 -- 172.21.15.103:0/3545894513 >> 172.21.15.103:0/3545894513 conn(0x56051fb28310 msgr2=0x56051fb61960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:28.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.261+0000 7f86fd779740 1 -- 172.21.15.103:0/3545894513 shutdown_connections 2024-09-19T06:55:28.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.261+0000 7f86fd779740 1 -- 172.21.15.103:0/3545894513 wait complete. 2024-09-19T06:55:28.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.262+0000 7f86fd779740 20 client.17688 trim_cache size 0 max 16384 2024-09-19T06:55:28.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.274+0000 7f86fd779740 1 Processor -- start 2024-09-19T06:55:28.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.274+0000 7f86fd779740 1 -- start start 2024-09-19T06:55:28.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.274+0000 7f86fd779740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb5f5d0 0x56051fb5be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.274+0000 7f86fd779740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb60b40 0x56051fb762b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.274+0000 7f86fd779740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb6a370 0x56051fb81c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.274+0000 7f86fd779740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86f41ad9f0 con 0x56051fb5f5d0 2024-09-19T06:55:28.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.274+0000 7f86fd779740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86f41af530 con 0x56051fb6a370 2024-09-19T06:55:28.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.274+0000 7f86fd779740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86f41ae0c0 con 0x56051fb60b40 2024-09-19T06:55:28.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.275+0000 7f86f913c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb5f5d0 0x56051fb5be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:28.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.275+0000 7f86f913c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb5f5d0 0x56051fb5be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34880/0 (socket says 172.21.15.103:34880) 2024-09-19T06:55:28.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.275+0000 7f86f913c640 1 -- 172.21.15.103:0/1975948799 learned_addr learned my addr 172.21.15.103:0/1975948799 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:28.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.275+0000 7f86f913c640 1 -- 172.21.15.103:0/1975948799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb60b40 msgr2=0x56051fb762b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:28.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.275+0000 7f86f913c640 1 --2- 172.21.15.103:0/1975948799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb60b40 0x56051fb762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.275+0000 7f86f913c640 1 -- 172.21.15.103:0/1975948799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb6a370 msgr2=0x56051fb81c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:28.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.275+0000 7f86f913c640 1 --2- 172.21.15.103:0/1975948799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb6a370 0x56051fb81c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.275+0000 7f86f913c640 1 -- 172.21.15.103:0/1975948799 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56051fb5b4f0 con 0x56051fb5f5d0 2024-09-19T06:55:28.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.275+0000 7f86f913c640 1 --2- 172.21.15.103:0/1975948799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb5f5d0 0x56051fb5be10 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f86f000bfd0 tx=0x7f86f00bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f4c73730b4083a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:28.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.275+0000 7f86deffd640 1 -- 172.21.15.103:0/1975948799 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f001e040 con 0x56051fb5f5d0 2024-09-19T06:55:28.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.275+0000 7f86deffd640 1 -- 172.21.15.103:0/1975948799 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f00a3f60 con 0x56051fb5f5d0 2024-09-19T06:55:28.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.275+0000 7f86deffd640 1 -- 172.21.15.103:0/1975948799 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0095e60 con 0x56051fb5f5d0 2024-09-19T06:55:28.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.276+0000 7f86fd779740 1 -- 172.21.15.103:0/1975948799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb5f5d0 msgr2=0x56051fb5be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:28.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.276+0000 7f86fd779740 1 --2- 172.21.15.103:0/1975948799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb5f5d0 0x56051fb5be10 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f86f000bfd0 tx=0x7f86f00bb040 comp rx=0 tx=0).stop 2024-09-19T06:55:28.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.276+0000 7f86fd779740 1 -- 172.21.15.103:0/1975948799 shutdown_connections 2024-09-19T06:55:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.276+0000 7f86fd779740 1 --2- 172.21.15.103:0/1975948799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb6a370 0x56051fb81c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.276+0000 7f86fd779740 1 --2- 172.21.15.103:0/1975948799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb60b40 0x56051fb762b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.276+0000 7f86fd779740 1 --2- 172.21.15.103:0/1975948799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb5f5d0 0x56051fb5be10 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.276+0000 7f86fd779740 1 -- 172.21.15.103:0/1975948799 >> 172.21.15.103:0/1975948799 conn(0x56051fb28310 msgr2=0x56051fb5b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.276+0000 7f86fd779740 1 -- 172.21.15.103:0/1975948799 shutdown_connections 2024-09-19T06:55:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.276+0000 7f86fd779740 1 -- 172.21.15.103:0/1975948799 wait complete. 2024-09-19T06:55:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.276+0000 7f86fd779740 1 -- start start 2024-09-19T06:55:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.276+0000 7f86fd779740 1 Processor -- start 2024-09-19T06:55:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86fd779740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb5f5d0 0x56051fb72420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86fd779740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb60b40 0x56051fb72960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86fd779740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb6a370 0x56051fb6f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86fd779740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86f41aee60 con 0x56051fb60b40 2024-09-19T06:55:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86fd779740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86f41ae790 con 0x56051fb6a370 2024-09-19T06:55:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86fd779740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86f41ab9b0 con 0x56051fb5f5d0 2024-09-19T06:55:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86fd779740 10 client.0 _ll_register_callbacks cb 0x7f86ec008070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86f893b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb60b40 0x56051fb72960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86f893b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb60b40 0x56051fb72960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34894/0 (socket says 172.21.15.103:34894) 2024-09-19T06:55:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86f893b640 1 -- 172.21.15.103:0/3479023794 learned_addr learned my addr 172.21.15.103:0/3479023794 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86f993d640 1 --2- 172.21.15.103:0/3479023794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb6a370 0x56051fb6f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86f913c640 1 --2- 172.21.15.103:0/3479023794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb5f5d0 0x56051fb72420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86f893b640 1 -- 172.21.15.103:0/3479023794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb5f5d0 msgr2=0x56051fb72420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86f893b640 1 --2- 172.21.15.103:0/3479023794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb5f5d0 0x56051fb72420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86f893b640 1 -- 172.21.15.103:0/3479023794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb6a370 msgr2=0x56051fb6f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86f893b640 1 --2- 172.21.15.103:0/3479023794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb6a370 0x56051fb6f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86f893b640 1 -- 172.21.15.103:0/3479023794 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56051fb72fd0 con 0x56051fb60b40 2024-09-19T06:55:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86f993d640 1 --2- 172.21.15.103:0/3479023794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb6a370 0x56051fb6f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:28.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86f913c640 1 --2- 172.21.15.103:0/3479023794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb5f5d0 0x56051fb72420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:28.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86f893b640 1 --2- 172.21.15.103:0/3479023794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb60b40 0x56051fb72960 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x56051fb60070 tx=0x7f86e4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372423f5eeda02f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:28.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:28.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86dcff9640 1 -- 172.21.15.103:0/3479023794 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e404b020 con 0x56051fb60b40 2024-09-19T06:55:28.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86fd779740 1 -- 172.21.15.103:0/3479023794 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f00a3f60 con 0x56051fb60b40 2024-09-19T06:55:28.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.277+0000 7f86dcff9640 1 -- 172.21.15.103:0/3479023794 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f86e4004450 con 0x56051fb60b40 2024-09-19T06:55:28.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.278+0000 7f86dcff9640 1 -- 172.21.15.103:0/3479023794 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f86e4056050 con 0x56051fb60b40 2024-09-19T06:55:28.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.278+0000 7f86fd779740 10 client.17694 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:28.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.278+0000 7f86fd779740 1 -- 172.21.15.103:0/3479023794 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56051fbe51a0 con 0x56051fb60b40 2024-09-19T06:55:28.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.278+0000 7f86dcff9640 1 -- 172.21.15.103:0/3479023794 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f00a3f60 con 0x56051fb60b40 2024-09-19T06:55:28.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.278+0000 7f86dcff9640 1 -- 172.21.15.103:0/3479023794 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f86e403c740 con 0x56051fb60b40 2024-09-19T06:55:28.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.278+0000 7f86fd779740 10 client.17694 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.278+0000 7f86fd779740 10 client.17694 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:55:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.278+0000 7f86fd779740 20 client.17694 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.278+0000 7f86fd779740 1 --2- 172.21.15.103:0/3479023794 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56051fbe5470 0x56051fbe7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.278+0000 7f86fd779740 4 client.17694 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:55:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.278+0000 7f86fd779740 1 -- 172.21.15.103:0/3479023794 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56051faccd70 con 0x56051fbe5470 2024-09-19T06:55:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.278+0000 7f86f913c640 1 --2- 172.21.15.103:0/3479023794 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56051fbe5470 0x56051fbe7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86f913c640 1 --2- 172.21.15.103:0/3479023794 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56051fbe5470 0x56051fbe7890 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2730b3cdc016cd66 server_cookie=2e6017a893e2c313 in_seq=0 out_seq=0 2024-09-19T06:55:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86dcff9640 10 client.17694 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:55:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86dcff9640 1 -- 172.21.15.103:0/3479023794 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56051faccd70 con 0x56051fbe5470 2024-09-19T06:55:28.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86dcff9640 10 client.17694 handle_command_reply: tid=0 2024-09-19T06:55:28.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86fd779740 1 client.17694 shutdown 2024-09-19T06:55:28.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86fd779740 1 -- 172.21.15.103:0/3479023794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb60b40 msgr2=0x56051fb72960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:28.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86fd779740 1 --2- 172.21.15.103:0/3479023794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb60b40 0x56051fb72960 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x56051fb60070 tx=0x7f86e4005850 comp rx=0 tx=0).stop 2024-09-19T06:55:28.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86fd779740 1 -- 172.21.15.103:0/3479023794 shutdown_connections 2024-09-19T06:55:28.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86fd779740 1 --2- 172.21.15.103:0/3479023794 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56051fbe5470 0x56051fbe7890 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86fd779740 1 --2- 172.21.15.103:0/3479023794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56051fb6a370 0x56051fb6f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86fd779740 1 --2- 172.21.15.103:0/3479023794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56051fb60b40 0x56051fb72960 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86fd779740 1 --2- 172.21.15.103:0/3479023794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56051fb5f5d0 0x56051fb72420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86fd779740 1 -- 172.21.15.103:0/3479023794 >> 172.21.15.103:0/3479023794 conn(0x56051fb28310 msgr2=0x56051fb687f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:28.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86dcff9640 0 client.17694 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:55:28.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86fd779740 1 -- 172.21.15.103:0/3479023794 shutdown_connections 2024-09-19T06:55:28.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86fd779740 1 -- 172.21.15.103:0/3479023794 wait complete. 2024-09-19T06:55:28.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.279+0000 7f86fd779740 20 client.17694 trim_cache size 0 max 16384 2024-09-19T06:55:28.289 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:55:28.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.280+0000 7f86fb3a0640 1 -- 172.21.15.103:0/3197192335 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f86e00a3430 msgr2=0x7f86e00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:28.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.280+0000 7f86fb3a0640 1 --2- 172.21.15.103:0/3197192335 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f86e00a3430 0x7f86e00a58f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f86ec014960 tx=0x7f86ec005df0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.280+0000 7f86fb3a0640 1 -- 172.21.15.103:0/3197192335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86f4101de0 msgr2=0x7f86f41985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:28.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.280+0000 7f86fb3a0640 1 --2- 172.21.15.103:0/3197192335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86f4101de0 0x7f86f41985b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f86f000b4a0 tx=0x7f86f000b960 comp rx=0 tx=0).stop 2024-09-19T06:55:28.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.280+0000 7f86fb3a0640 1 -- 172.21.15.103:0/3197192335 shutdown_connections 2024-09-19T06:55:28.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.280+0000 7f86fb3a0640 1 --2- 172.21.15.103:0/3197192335 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f86e00a3430 0x7f86e00a58f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.280+0000 7f86fb3a0640 1 --2- 172.21.15.103:0/3197192335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f4106f10 0x7f86f419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.280+0000 7f86fb3a0640 1 --2- 172.21.15.103:0/3197192335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86f4102790 0x7f86f4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.280+0000 7f86fb3a0640 1 --2- 172.21.15.103:0/3197192335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86f4101de0 0x7f86f41985b0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:28.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.280+0000 7f86fb3a0640 1 -- 172.21.15.103:0/3197192335 >> 172.21.15.103:0/3197192335 conn(0x7f86f40fd8c0 msgr2=0x7f86f40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:28.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.280+0000 7f86fb3a0640 1 -- 172.21.15.103:0/3197192335 shutdown_connections 2024-09-19T06:55:28.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:28.280+0000 7f86fb3a0640 1 -- 172.21.15.103:0/3197192335 wait complete. 2024-09-19T06:55:28.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:28 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:55:28.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:28 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:55:28.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:28 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:55:28.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:28 smithi134 ceph-mon[35356]: pgmap v190: 97 pgs: 97 active+clean; 2.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-09-19T06:55:28.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:28 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:55:28.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:28 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:55:28.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:28 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:55:28.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:28 smithi174 ceph-mon[35637]: pgmap v190: 97 pgs: 97 active+clean; 2.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-09-19T06:55:28.971 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:55:29.220 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:29.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.669+0000 7ff3ff41b640 1 -- 172.21.15.103:0/203032673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f8100310 msgr2=0x7ff3f81006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:29.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.669+0000 7ff3ff41b640 1 --2- 172.21.15.103:0/203032673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f8100310 0x7ff3f81006f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009900 tx=0x7ff3f402a410 comp rx=0 tx=0).stop 2024-09-19T06:55:29.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.669+0000 7ff3ff41b640 1 -- 172.21.15.103:0/203032673 shutdown_connections 2024-09-19T06:55:29.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.669+0000 7ff3ff41b640 1 --2- 172.21.15.103:0/203032673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f8106f30 0x7ff3f810eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.669+0000 7ff3ff41b640 1 --2- 172.21.15.103:0/203032673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f8100cc0 0x7ff3f8106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.669+0000 7ff3ff41b640 1 --2- 172.21.15.103:0/203032673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f8100310 0x7ff3f81006f0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.669+0000 7ff3ff41b640 1 -- 172.21.15.103:0/203032673 >> 172.21.15.103:0/203032673 conn(0x7ff3f8077210 msgr2=0x7ff3f80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:29.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.669+0000 7ff3ff41b640 1 -- 172.21.15.103:0/203032673 shutdown_connections 2024-09-19T06:55:29.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.669+0000 7ff3ff41b640 1 -- 172.21.15.103:0/203032673 wait complete. 2024-09-19T06:55:29.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.670+0000 7ff3ff41b640 1 Processor -- start 2024-09-19T06:55:29.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.670+0000 7ff3ff41b640 1 -- start start 2024-09-19T06:55:29.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.670+0000 7ff3ff41b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f8100310 0x7ff3f8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.670+0000 7ff3ff41b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f8100cc0 0x7ff3f8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.670+0000 7ff3ff41b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f8106f30 0x7ff3f819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.670+0000 7ff3ff41b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f8111f90 con 0x7ff3f8100310 2024-09-19T06:55:29.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.670+0000 7ff3ff41b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f8111e10 con 0x7ff3f8106f30 2024-09-19T06:55:29.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.670+0000 7ff3ff41b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f8112110 con 0x7ff3f8100cc0 2024-09-19T06:55:29.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.670+0000 7ff3fd1b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f8100310 0x7ff3f8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.670+0000 7ff3fd1b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f8100310 0x7ff3f8198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34908/0 (socket says 172.21.15.103:34908) 2024-09-19T06:55:29.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.670+0000 7ff3fd1b7640 1 -- 172.21.15.103:0/1117243865 learned_addr learned my addr 172.21.15.103:0/1117243865 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:29.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.671+0000 7ff3fc9b6640 1 --2- 172.21.15.103:0/1117243865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f8100cc0 0x7ff3f8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.671+0000 7ff3fd9b8640 1 --2- 172.21.15.103:0/1117243865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f8106f30 0x7ff3f819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.671+0000 7ff3fd1b7640 1 -- 172.21.15.103:0/1117243865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f8100cc0 msgr2=0x7ff3f8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:29.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.671+0000 7ff3fd1b7640 1 --2- 172.21.15.103:0/1117243865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f8100cc0 0x7ff3f8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.671+0000 7ff3fd1b7640 1 -- 172.21.15.103:0/1117243865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f8106f30 msgr2=0x7ff3f819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:29.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.671+0000 7ff3fd1b7640 1 --2- 172.21.15.103:0/1117243865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f8106f30 0x7ff3f819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.671+0000 7ff3fd1b7640 1 -- 172.21.15.103:0/1117243865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f819d3c0 con 0x7ff3f8100310 2024-09-19T06:55:29.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.671+0000 7ff3fc9b6640 1 --2- 172.21.15.103:0/1117243865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f8100cc0 0x7ff3f8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:29.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.671+0000 7ff3fd1b7640 1 --2- 172.21.15.103:0/1117243865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f8100310 0x7ff3f8198410 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4005ab0 tx=0x7ff3f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:29.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.671+0000 7ff3ee7fc640 1 -- 172.21.15.103:0/1117243865 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f403f030 con 0x7ff3f8100310 2024-09-19T06:55:29.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.671+0000 7ff3ee7fc640 1 -- 172.21.15.103:0/1117243865 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f4003ce0 con 0x7ff3f8100310 2024-09-19T06:55:29.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.671+0000 7ff3ff41b640 1 -- 172.21.15.103:0/1117243865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f819d650 con 0x7ff3f8100310 2024-09-19T06:55:29.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.671+0000 7ff3ee7fc640 1 -- 172.21.15.103:0/1117243865 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4035070 con 0x7ff3f8100310 2024-09-19T06:55:29.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.671+0000 7ff3ff41b640 1 -- 172.21.15.103:0/1117243865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f81011c0 con 0x7ff3f8100310 2024-09-19T06:55:29.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.673+0000 7ff3ee7fc640 1 -- 172.21.15.103:0/1117243865 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff3f4042030 con 0x7ff3f8100310 2024-09-19T06:55:29.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.674+0000 7ff4017f4740 1 Processor -- start 2024-09-19T06:55:29.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.674+0000 7ff4017f4740 1 -- start start 2024-09-19T06:55:29.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.674+0000 7ff3ee7fc640 1 --2- 172.21.15.103:0/1117243865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3e40a33b0 0x7ff3e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.674+0000 7ff4017f4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d36d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.674+0000 7ff4017f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d36d5d0 0x55864d36da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.674+0000 7ff4017f4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d378370 0x55864d37a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.674+0000 7ff4017f4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55864d2d32f0 con 0x55864d36eb40 2024-09-19T06:55:29.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.674+0000 7ff4017f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55864d27acf0 con 0x55864d36d5d0 2024-09-19T06:55:29.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.674+0000 7ff4017f4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55864d2cf2b0 con 0x55864d378370 2024-09-19T06:55:29.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.674+0000 7ff3fd1b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d36d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.674+0000 7ff3fd9b8640 1 --2- 172.21.15.103:0/1117243865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3e40a33b0 0x7ff3e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.675+0000 7ff3fd1b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d36d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34910/0 (socket says 172.21.15.103:34910) 2024-09-19T06:55:29.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.675+0000 7ff3fd1b7640 1 -- 172.21.15.103:0/3050461679 learned_addr learned my addr 172.21.15.103:0/3050461679 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:29.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.675+0000 7ff3ee7fc640 1 -- 172.21.15.103:0/1117243865 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff3f40f06d0 con 0x7ff3f8100310 2024-09-19T06:55:29.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.675+0000 7ff3fc9b6640 1 --2- 172.21.15.103:0/3050461679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d36d5d0 0x55864d36da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.675+0000 7ff3fd9b8640 1 --2- 172.21.15.103:0/3050461679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d378370 0x55864d37a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.675+0000 7ff3fd1b7640 1 -- 172.21.15.103:0/3050461679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d378370 msgr2=0x55864d37a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:29.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.675+0000 7ff3fd1b7640 1 --2- 172.21.15.103:0/3050461679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d378370 0x55864d37a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.675+0000 7ff3fd1b7640 1 -- 172.21.15.103:0/3050461679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d36d5d0 msgr2=0x55864d36da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:29.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.675+0000 7ff3fd1b7640 1 --2- 172.21.15.103:0/3050461679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d36d5d0 0x55864d36da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.675+0000 7ff3fd1b7640 1 -- 172.21.15.103:0/3050461679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55864d25f3a0 con 0x55864d36eb40 2024-09-19T06:55:29.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.675+0000 7ff3fd9b8640 1 --2- 172.21.15.103:0/1117243865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3e40a33b0 0x7ff3e40a5870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0005cc0 tx=0x7ff3f000a8d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:29.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.675+0000 7ff3fd1b7640 1 --2- 172.21.15.103:0/3050461679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d36d090 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40bc910 tx=0x7ff3f403b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fb96ec50f7bbaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:29.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.675+0000 7ff3e37fe640 1 -- 172.21.15.103:0/3050461679 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4049070 con 0x55864d36eb40 2024-09-19T06:55:29.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.675+0000 7ff3e37fe640 1 -- 172.21.15.103:0/3050461679 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f40bd080 con 0x55864d36eb40 2024-09-19T06:55:29.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.675+0000 7ff3e37fe640 1 -- 172.21.15.103:0/3050461679 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4040cb0 con 0x55864d36eb40 2024-09-19T06:55:29.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.676+0000 7ff4017f4740 1 -- 172.21.15.103:0/3050461679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 msgr2=0x55864d36d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:29.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.676+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3050461679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d36d090 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40bc910 tx=0x7ff3f403b800 comp rx=0 tx=0).stop 2024-09-19T06:55:29.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.676+0000 7ff4017f4740 1 -- 172.21.15.103:0/3050461679 shutdown_connections 2024-09-19T06:55:29.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.676+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3050461679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d378370 0x55864d37a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.676+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3050461679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d36d5d0 0x55864d36da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.676+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3050461679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d36d090 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.676+0000 7ff4017f4740 1 -- 172.21.15.103:0/3050461679 >> 172.21.15.103:0/3050461679 conn(0x55864d336310 msgr2=0x55864d369c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:29.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.676+0000 7ff4017f4740 1 -- 172.21.15.103:0/3050461679 shutdown_connections 2024-09-19T06:55:29.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.676+0000 7ff4017f4740 1 -- 172.21.15.103:0/3050461679 wait complete. 2024-09-19T06:55:29.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.676+0000 7ff4017f4740 1 -- start start 2024-09-19T06:55:29.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff4017f4740 1 Processor -- start 2024-09-19T06:55:29.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff4017f4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36d5d0 0x55864d377d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff4017f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d36eb40 0x55864d375010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff4017f4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d378370 0x55864d375600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff4017f4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55864d2d32f0 con 0x55864d36d5d0 2024-09-19T06:55:29.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff4017f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55864d28e060 con 0x55864d36eb40 2024-09-19T06:55:29.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff4017f4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55864d37ae30 con 0x55864d378370 2024-09-19T06:55:29.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff4017f4740 10 client.0 _ll_register_callbacks cb 0x55864cfe18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:29.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff3fd1b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36d5d0 0x55864d377d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff3fd1b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36d5d0 0x55864d377d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34924/0 (socket says 172.21.15.103:34924) 2024-09-19T06:55:29.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff3fd1b7640 1 -- 172.21.15.103:0/3730465064 learned_addr learned my addr 172.21.15.103:0/3730465064 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:29.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff3fd1b7640 1 -- 172.21.15.103:0/3730465064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d378370 msgr2=0x55864d375600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:29.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff3fd9b8640 1 --2- 172.21.15.103:0/3730465064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d378370 0x55864d375600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff3fc9b6640 1 --2- 172.21.15.103:0/3730465064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d36eb40 0x55864d375010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff3fd1b7640 1 --2- 172.21.15.103:0/3730465064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d378370 0x55864d375600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff3fd1b7640 1 -- 172.21.15.103:0/3730465064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d36eb40 msgr2=0x55864d375010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:29.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff3fd1b7640 1 --2- 172.21.15.103:0/3730465064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d36eb40 0x55864d375010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff3fd1b7640 1 -- 172.21.15.103:0/3730465064 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55864d0443c0 con 0x55864d36d5d0 2024-09-19T06:55:29.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.677+0000 7ff3fd9b8640 1 --2- 172.21.15.103:0/3730465064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d378370 0x55864d375600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:29.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.678+0000 7ff3fd1b7640 1 --2- 172.21.15.103:0/3730465064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36d5d0 0x55864d377d60 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40c50c0 tx=0x7ff3f403bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3360c52f612660 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:29.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.678+0000 7ff3e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:29.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.678+0000 7ff3e0ff9640 1 -- 172.21.15.103:0/3730465064 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4049040 con 0x55864d36d5d0 2024-09-19T06:55:29.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.678+0000 7ff3e0ff9640 1 -- 172.21.15.103:0/3730465064 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff3f40ee070 con 0x55864d36d5d0 2024-09-19T06:55:29.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.678+0000 7ff4017f4740 1 -- 172.21.15.103:0/3730465064 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3f40bd080 con 0x55864d36d5d0 2024-09-19T06:55:29.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.678+0000 7ff3e0ff9640 1 -- 172.21.15.103:0/3730465064 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff3f40c55f0 con 0x55864d36d5d0 2024-09-19T06:55:29.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.678+0000 7ff4017f4740 10 client.17706 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:29.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.678+0000 7ff4017f4740 1 -- 172.21.15.103:0/3730465064 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55864d386750 con 0x55864d36d5d0 2024-09-19T06:55:29.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.678+0000 7ff3e0ff9640 1 -- 172.21.15.103:0/3730465064 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f40bd080 con 0x55864d36d5d0 2024-09-19T06:55:29.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.678+0000 7ff3e0ff9640 1 -- 172.21.15.103:0/3730465064 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff3f40eb020 con 0x55864d36d5d0 2024-09-19T06:55:29.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.678+0000 7ff4017f4740 10 client.17706 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:29.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.678+0000 7ff4017f4740 10 client.17706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:29.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.678+0000 7ff4017f4740 20 client.17706 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:29.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.678+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3730465064 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55864d3878d0 0x55864d389cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.678+0000 7ff4017f4740 4 client.17706 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:29.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.678+0000 7ff4017f4740 1 -- 172.21.15.103:0/3730465064 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55864d2e0860 con 0x55864d3878d0 2024-09-19T06:55:29.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.679+0000 7ff3fc9b6640 1 --2- 172.21.15.103:0/3730465064 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55864d3878d0 0x55864d389cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.679+0000 7ff3fc9b6640 1 --2- 172.21.15.103:0/3730465064 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55864d3878d0 0x55864d389cc0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=617584e04ef8b4f6 server_cookie=3c1f3d55dd6b7dd4 in_seq=0 out_seq=0 2024-09-19T06:55:29.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.679+0000 7ff3e0ff9640 10 client.17706 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:29.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.680+0000 7ff3e0ff9640 1 -- 172.21.15.103:0/3730465064 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55864d2e0860 con 0x55864d3878d0 2024-09-19T06:55:29.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.680+0000 7ff3e0ff9640 10 client.17706 handle_command_reply: tid=0 2024-09-19T06:55:29.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.680+0000 7ff4017f4740 1 client.17706 shutdown 2024-09-19T06:55:29.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.680+0000 7ff4017f4740 1 -- 172.21.15.103:0/3730465064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36d5d0 msgr2=0x55864d377d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:29.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.680+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3730465064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36d5d0 0x55864d377d60 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40c50c0 tx=0x7ff3f403bf60 comp rx=0 tx=0).stop 2024-09-19T06:55:29.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.680+0000 7ff4017f4740 1 -- 172.21.15.103:0/3730465064 shutdown_connections 2024-09-19T06:55:29.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.680+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3730465064 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55864d3878d0 0x55864d389cc0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.680+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3730465064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d378370 0x55864d375600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.680+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3730465064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d36eb40 0x55864d375010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.680+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3730465064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36d5d0 0x55864d377d60 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.680+0000 7ff4017f4740 1 -- 172.21.15.103:0/3730465064 >> 172.21.15.103:0/3730465064 conn(0x55864d336310 msgr2=0x55864d36f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.680+0000 7ff3e0ff9640 0 client.17706 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.680+0000 7ff4017f4740 1 -- 172.21.15.103:0/3730465064 shutdown_connections 2024-09-19T06:55:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.680+0000 7ff4017f4740 1 -- 172.21.15.103:0/3730465064 wait complete. 2024-09-19T06:55:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.680+0000 7ff4017f4740 20 client.17706 trim_cache size 0 max 16384 2024-09-19T06:55:29.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.693+0000 7ff4017f4740 1 Processor -- start 2024-09-19T06:55:29.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.693+0000 7ff4017f4740 1 -- start start 2024-09-19T06:55:29.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.693+0000 7ff4017f4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d36d5d0 0x55864d383f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.693+0000 7ff4017f4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d370120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.693+0000 7ff4017f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d378370 0x55864d370660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.693+0000 7ff4017f4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f81afa70 con 0x55864d36eb40 2024-09-19T06:55:29.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.693+0000 7ff4017f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f81b15b0 con 0x55864d378370 2024-09-19T06:55:29.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.693+0000 7ff4017f4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f81b0140 con 0x55864d36d5d0 2024-09-19T06:55:29.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.693+0000 7ff3fc9b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d370120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.693+0000 7ff3fc9b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d370120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34938/0 (socket says 172.21.15.103:34938) 2024-09-19T06:55:29.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.693+0000 7ff3fc9b6640 1 -- 172.21.15.103:0/3288549293 learned_addr learned my addr 172.21.15.103:0/3288549293 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:29.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.694+0000 7ff3fc9b6640 1 -- 172.21.15.103:0/3288549293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d36d5d0 msgr2=0x55864d383f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:29.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.694+0000 7ff3fc9b6640 1 --2- 172.21.15.103:0/3288549293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d36d5d0 0x55864d383f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.694+0000 7ff3fd9b8640 1 --2- 172.21.15.103:0/3288549293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d378370 0x55864d370660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.694+0000 7ff3fc9b6640 1 -- 172.21.15.103:0/3288549293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d378370 msgr2=0x55864d370660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:29.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.694+0000 7ff3fc9b6640 1 --2- 172.21.15.103:0/3288549293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d378370 0x55864d370660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.694+0000 7ff3fc9b6640 1 -- 172.21.15.103:0/3288549293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55864d375970 con 0x55864d36eb40 2024-09-19T06:55:29.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.694+0000 7ff3fc9b6640 1 --2- 172.21.15.103:0/3288549293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d370120 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x55864d36aac0 tx=0x7ff3e800c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9e9f18ef4582f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:29.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.694+0000 7ff3e2ffd640 1 -- 172.21.15.103:0/3288549293 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8005650 con 0x55864d36eb40 2024-09-19T06:55:29.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.694+0000 7ff3e2ffd640 1 -- 172.21.15.103:0/3288549293 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e8004de0 con 0x55864d36eb40 2024-09-19T06:55:29.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.694+0000 7ff3e2ffd640 1 -- 172.21.15.103:0/3288549293 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e80119a0 con 0x55864d36eb40 2024-09-19T06:55:29.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 -- 172.21.15.103:0/3288549293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 msgr2=0x55864d370120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:29.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3288549293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d370120 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x55864d36aac0 tx=0x7ff3e800c4f0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 -- 172.21.15.103:0/3288549293 shutdown_connections 2024-09-19T06:55:29.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3288549293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d378370 0x55864d370660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3288549293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d370120 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3288549293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d36d5d0 0x55864d383f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 -- 172.21.15.103:0/3288549293 >> 172.21.15.103:0/3288549293 conn(0x55864d336310 msgr2=0x55864d369700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:29.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 -- 172.21.15.103:0/3288549293 shutdown_connections 2024-09-19T06:55:29.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 -- 172.21.15.103:0/3288549293 wait complete. 2024-09-19T06:55:29.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 -- start start 2024-09-19T06:55:29.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 Processor -- start 2024-09-19T06:55:29.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d36d5d0 0x55864d380500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d37b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d378370 0x55864d37bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f81b0ee0 con 0x55864d36eb40 2024-09-19T06:55:29.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f81b0810 con 0x55864d378370 2024-09-19T06:55:29.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f81ada30 con 0x55864d36d5d0 2024-09-19T06:55:29.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.695+0000 7ff4017f4740 10 client.0 _ll_register_callbacks cb 0x7ff3e800c390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:29.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff3fc9b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d37b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff3fc9b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d37b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34946/0 (socket says 172.21.15.103:34946) 2024-09-19T06:55:29.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff3fc9b6640 1 -- 172.21.15.103:0/3297376341 learned_addr learned my addr 172.21.15.103:0/3297376341 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:29.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff3fd9b8640 1 --2- 172.21.15.103:0/3297376341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d378370 0x55864d37bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff3fd1b7640 1 --2- 172.21.15.103:0/3297376341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d36d5d0 0x55864d380500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff3fc9b6640 1 -- 172.21.15.103:0/3297376341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d36d5d0 msgr2=0x55864d380500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:29.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff3fc9b6640 1 --2- 172.21.15.103:0/3297376341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d36d5d0 0x55864d380500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff3fc9b6640 1 -- 172.21.15.103:0/3297376341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d378370 msgr2=0x55864d37bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:29.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff3fc9b6640 1 --2- 172.21.15.103:0/3297376341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d378370 0x55864d37bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff3fc9b6640 1 -- 172.21.15.103:0/3297376341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55864d37c2b0 con 0x55864d36eb40 2024-09-19T06:55:29.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff3fc9b6640 1 --2- 172.21.15.103:0/3297376341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d37b830 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800d010 tx=0x7ff3e8011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7f657f6dc4fad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:29.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff3e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:29.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff3e0ff9640 1 -- 172.21.15.103:0/3297376341 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8004220 con 0x55864d36eb40 2024-09-19T06:55:29.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff3e0ff9640 1 -- 172.21.15.103:0/3297376341 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7ff3e80043c0 con 0x55864d36eb40 2024-09-19T06:55:29.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff4017f4740 1 -- 172.21.15.103:0/3297376341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3e8004de0 con 0x55864d36eb40 2024-09-19T06:55:29.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff3e0ff9640 1 -- 172.21.15.103:0/3297376341 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff3e8011830 con 0x55864d36eb40 2024-09-19T06:55:29.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff4017f4740 10 client.17712 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:29.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.696+0000 7ff4017f4740 1 -- 172.21.15.103:0/3297376341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55864d3f2210 con 0x55864d36eb40 2024-09-19T06:55:29.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.697+0000 7ff3e0ff9640 1 -- 172.21.15.103:0/3297376341 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e8004de0 con 0x55864d36eb40 2024-09-19T06:55:29.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.697+0000 7ff3e0ff9640 1 -- 172.21.15.103:0/3297376341 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff3e802b030 con 0x55864d36eb40 2024-09-19T06:55:29.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.697+0000 7ff4017f4740 10 client.17712 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:29.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.697+0000 7ff4017f4740 10 client.17712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:29.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.697+0000 7ff4017f4740 20 client.17712 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:29.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.697+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3297376341 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55864d3f28f0 0x55864d3f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.697+0000 7ff4017f4740 4 client.17712 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:55:29.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.697+0000 7ff4017f4740 1 -- 172.21.15.103:0/3297376341 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55864d2dad70 con 0x55864d3f28f0 2024-09-19T06:55:29.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.697+0000 7ff3fd1b7640 1 --2- 172.21.15.103:0/3297376341 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55864d3f28f0 0x55864d3f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff3e0ff9640 10 client.17712 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:29.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff3fd1b7640 1 --2- 172.21.15.103:0/3297376341 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55864d3f28f0 0x55864d3f4d10 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f1e3b3c970f529 server_cookie=da5026931eec1190 in_seq=0 out_seq=0 2024-09-19T06:55:29.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff3e0ff9640 1 -- 172.21.15.103:0/3297376341 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55864d2dad70 con 0x55864d3f28f0 2024-09-19T06:55:29.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff3e0ff9640 10 client.17712 handle_command_reply: tid=0 2024-09-19T06:55:29.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff4017f4740 1 client.17712 shutdown 2024-09-19T06:55:29.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff4017f4740 1 -- 172.21.15.103:0/3297376341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 msgr2=0x55864d37b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:29.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3297376341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d37b830 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800d010 tx=0x7ff3e8011440 comp rx=0 tx=0).stop 2024-09-19T06:55:29.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff4017f4740 1 -- 172.21.15.103:0/3297376341 shutdown_connections 2024-09-19T06:55:29.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3297376341 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55864d3f28f0 0x55864d3f4d10 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3297376341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55864d378370 0x55864d37bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3297376341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55864d36eb40 0x55864d37b830 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff4017f4740 1 --2- 172.21.15.103:0/3297376341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55864d36d5d0 0x55864d380500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff3e0ff9640 0 client.17712 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:29.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff4017f4740 1 -- 172.21.15.103:0/3297376341 >> 172.21.15.103:0/3297376341 conn(0x55864d336310 msgr2=0x55864d377090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:29.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff4017f4740 1 -- 172.21.15.103:0/3297376341 shutdown_connections 2024-09-19T06:55:29.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff4017f4740 1 -- 172.21.15.103:0/3297376341 wait complete. 2024-09-19T06:55:29.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.698+0000 7ff4017f4740 20 client.17712 trim_cache size 0 max 16384 2024-09-19T06:55:29.706 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:55:29.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.699+0000 7ff3ff41b640 1 -- 172.21.15.103:0/1117243865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3e40a33b0 msgr2=0x7ff3e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:29.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.699+0000 7ff3ff41b640 1 --2- 172.21.15.103:0/1117243865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3e40a33b0 0x7ff3e40a5870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0005cc0 tx=0x7ff3f000a8d0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.699+0000 7ff3ff41b640 1 -- 172.21.15.103:0/1117243865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f8100310 msgr2=0x7ff3f8198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:29.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.699+0000 7ff3ff41b640 1 --2- 172.21.15.103:0/1117243865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f8100310 0x7ff3f8198410 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4005ab0 tx=0x7ff3f40042f0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.699+0000 7ff3ff41b640 1 -- 172.21.15.103:0/1117243865 shutdown_connections 2024-09-19T06:55:29.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.699+0000 7ff3ff41b640 1 --2- 172.21.15.103:0/1117243865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3e40a33b0 0x7ff3e40a5870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.699+0000 7ff3ff41b640 1 --2- 172.21.15.103:0/1117243865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f8106f30 0x7ff3f819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.699+0000 7ff3ff41b640 1 --2- 172.21.15.103:0/1117243865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f8100cc0 0x7ff3f8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.699+0000 7ff3ff41b640 1 --2- 172.21.15.103:0/1117243865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f8100310 0x7ff3f8198410 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.699+0000 7ff3ff41b640 1 -- 172.21.15.103:0/1117243865 >> 172.21.15.103:0/1117243865 conn(0x7ff3f8077210 msgr2=0x7ff3f81057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:29.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.699+0000 7ff3ff41b640 1 -- 172.21.15.103:0/1117243865 shutdown_connections 2024-09-19T06:55:29.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:29.699+0000 7ff3ff41b640 1 -- 172.21.15.103:0/1117243865 wait complete. 2024-09-19T06:55:30.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:30 smithi103 ceph-mon[31158]: pgmap v191: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-09-19T06:55:30.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:30 smithi134 ceph-mon[35356]: pgmap v191: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-09-19T06:55:30.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:30 smithi174 ceph-mon[35637]: pgmap v191: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-09-19T06:55:31.473 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:55:31.474 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:55:31.735 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:32.122 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:32 smithi103 ceph-mon[31158]: pgmap v192: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:55:32.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.204+0000 7f66d6727640 1 -- 172.21.15.103:0/3008048258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66d0102d40 msgr2=0x7f66d0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:32.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.204+0000 7f66d6727640 1 --2- 172.21.15.103:0/3008048258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66d0102d40 0x7f66d0103120 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009900 tx=0x7f66c002a480 comp rx=0 tx=0).stop 2024-09-19T06:55:32.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.204+0000 7f66d6727640 1 -- 172.21.15.103:0/3008048258 shutdown_connections 2024-09-19T06:55:32.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.204+0000 7f66d6727640 1 --2- 172.21.15.103:0/3008048258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66d010c170 0x7f66d010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.204+0000 7f66d6727640 1 --2- 172.21.15.103:0/3008048258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66d0103660 0x7f66d010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.204+0000 7f66d6727640 1 --2- 172.21.15.103:0/3008048258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66d0102d40 0x7f66d0103120 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.204+0000 7f66d6727640 1 -- 172.21.15.103:0/3008048258 >> 172.21.15.103:0/3008048258 conn(0x7f66d00fc7a0 msgr2=0x7f66d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:32.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.204+0000 7f66d6727640 1 -- 172.21.15.103:0/3008048258 shutdown_connections 2024-09-19T06:55:32.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.204+0000 7f66d6727640 1 -- 172.21.15.103:0/3008048258 wait complete. 2024-09-19T06:55:32.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.204+0000 7f66d6727640 1 Processor -- start 2024-09-19T06:55:32.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.204+0000 7f66d6727640 1 -- start start 2024-09-19T06:55:32.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66d6727640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66d0102d40 0x7f66d0198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66d6727640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66d0103660 0x7f66d0198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66d6727640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66d010c170 0x7f66d019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66d6727640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66d0112030 con 0x7f66d0102d40 2024-09-19T06:55:32.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66d6727640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66d0111eb0 con 0x7f66d0103660 2024-09-19T06:55:32.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66d6727640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01121b0 con 0x7f66d010c170 2024-09-19T06:55:32.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66d0102d40 0x7f66d0198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:32.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66d0102d40 0x7f66d0198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34968/0 (socket says 172.21.15.103:34968) 2024-09-19T06:55:32.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66cffff640 1 -- 172.21.15.103:0/1402495353 learned_addr learned my addr 172.21.15.103:0/1402495353 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:32.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66d4cc4640 1 --2- 172.21.15.103:0/1402495353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66d010c170 0x7f66d019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:32.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66cf7fe640 1 --2- 172.21.15.103:0/1402495353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66d0103660 0x7f66d0198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:32.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66cffff640 1 -- 172.21.15.103:0/1402495353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66d010c170 msgr2=0x7f66d019cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:32.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66cffff640 1 --2- 172.21.15.103:0/1402495353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66d010c170 0x7f66d019cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66cffff640 1 -- 172.21.15.103:0/1402495353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66d0103660 msgr2=0x7f66d0198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:32.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66cffff640 1 --2- 172.21.15.103:0/1402495353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66d0103660 0x7f66d0198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66cffff640 1 -- 172.21.15.103:0/1402495353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d019d640 con 0x7f66d0102d40 2024-09-19T06:55:32.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66cf7fe640 1 --2- 172.21.15.103:0/1402495353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66d0103660 0x7f66d0198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:32.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66cffff640 1 --2- 172.21.15.103:0/1402495353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66d0102d40 0x7f66d0198690 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f66c0005ab0 tx=0x7f66c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:32.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66cd7fa640 1 -- 172.21.15.103:0/1402495353 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c003f030 con 0x7f66d0102d40 2024-09-19T06:55:32.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66cd7fa640 1 -- 172.21.15.103:0/1402495353 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66c0003ce0 con 0x7f66d0102d40 2024-09-19T06:55:32.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.205+0000 7f66d6727640 1 -- 172.21.15.103:0/1402495353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d019d8d0 con 0x7f66d0102d40 2024-09-19T06:55:32.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.206+0000 7f66cd7fa640 1 -- 172.21.15.103:0/1402495353 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0035070 con 0x7f66d0102d40 2024-09-19T06:55:32.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.206+0000 7f66d6727640 1 -- 172.21.15.103:0/1402495353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d01a5210 con 0x7f66d0102d40 2024-09-19T06:55:32.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.208+0000 7f66cd7fa640 1 -- 172.21.15.103:0/1402495353 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f66c0042030 con 0x7f66d0102d40 2024-09-19T06:55:32.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.208+0000 7f66d8b00740 1 Processor -- start 2024-09-19T06:55:32.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.208+0000 7f66d8b00740 1 -- start start 2024-09-19T06:55:32.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.208+0000 7f66d8b00740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546b97b60 0x565546b97f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.208+0000 7f66cd7fa640 1 --2- 172.21.15.103:0/1402495353 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66b80a33b0 0x7f66b80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.208+0000 7f66d8b00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546b98480 0x565546ba2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.208+0000 7f66d8b00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546ba2d40 0x565546ba5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.208+0000 7f66d8b00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565546afd280 con 0x565546ba2d40 2024-09-19T06:55:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.208+0000 7f66d8b00740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565546aa4b10 con 0x565546b97b60 2024-09-19T06:55:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.208+0000 7f66d8b00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565546af9240 con 0x565546b98480 2024-09-19T06:55:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66d4cc4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546ba2d40 0x565546ba5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66cffff640 1 --2- 172.21.15.103:0/1402495353 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66b80a33b0 0x7f66b80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66d4cc4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546ba2d40 0x565546ba5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34984/0 (socket says 172.21.15.103:34984) 2024-09-19T06:55:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66d4cc4640 1 -- 172.21.15.103:0/633428578 learned_addr learned my addr 172.21.15.103:0/633428578 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66d4cc4640 1 --2- 172.21.15.103:0/633428578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546b97b60 0x565546b97f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66cd7fa640 1 -- 172.21.15.103:0/1402495353 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f66c00f06d0 con 0x7f66d0102d40 2024-09-19T06:55:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66cf7fe640 1 --2- 172.21.15.103:0/633428578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546b98480 0x565546ba2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66d4cc4640 1 -- 172.21.15.103:0/633428578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546b98480 msgr2=0x565546ba2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66d4cc4640 1 --2- 172.21.15.103:0/633428578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546b98480 0x565546ba2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66d4cc4640 1 -- 172.21.15.103:0/633428578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546b97b60 msgr2=0x565546b97f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66d4cc4640 1 --2- 172.21.15.103:0/633428578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546b97b60 0x565546b97f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66d4cc4640 1 -- 172.21.15.103:0/633428578 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565546ab11f0 con 0x565546ba2d40 2024-09-19T06:55:32.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66cffff640 1 --2- 172.21.15.103:0/1402495353 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66b80a33b0 0x7f66b80a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f66c00c50c0 tx=0x7f66c00b1e10 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:32.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66d4cc4640 1 --2- 172.21.15.103:0/633428578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546ba2d40 0x565546ba5130 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f66c800a6f0 tx=0x7f66c80097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6265898a66bd0d10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:32.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66b67fc640 1 -- 172.21.15.103:0/633428578 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c801d070 con 0x565546ba2d40 2024-09-19T06:55:32.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66b67fc640 1 -- 172.21.15.103:0/633428578 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66c8011b40 con 0x565546ba2d40 2024-09-19T06:55:32.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.209+0000 7f66b67fc640 1 -- 172.21.15.103:0/633428578 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8018ad0 con 0x565546ba2d40 2024-09-19T06:55:32.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.210+0000 7f66d8b00740 1 -- 172.21.15.103:0/633428578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546ba2d40 msgr2=0x565546ba5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:32.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.210+0000 7f66d8b00740 1 --2- 172.21.15.103:0/633428578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546ba2d40 0x565546ba5130 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f66c800a6f0 tx=0x7f66c80097c0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.210+0000 7f66d8b00740 1 -- 172.21.15.103:0/633428578 shutdown_connections 2024-09-19T06:55:32.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.210+0000 7f66d8b00740 1 --2- 172.21.15.103:0/633428578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546ba2d40 0x565546ba5130 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.210+0000 7f66d8b00740 1 --2- 172.21.15.103:0/633428578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546b98480 0x565546ba2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.210+0000 7f66d8b00740 1 --2- 172.21.15.103:0/633428578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546b97b60 0x565546b97f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.210+0000 7f66d8b00740 1 -- 172.21.15.103:0/633428578 >> 172.21.15.103:0/633428578 conn(0x565546b940f0 msgr2=0x565546b944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:32.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.210+0000 7f66d8b00740 1 -- 172.21.15.103:0/633428578 shutdown_connections 2024-09-19T06:55:32.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.210+0000 7f66d8b00740 1 -- 172.21.15.103:0/633428578 wait complete. 2024-09-19T06:55:32.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.210+0000 7f66d8b00740 1 -- start start 2024-09-19T06:55:32.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66d8b00740 1 Processor -- start 2024-09-19T06:55:32.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66d8b00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546b97b60 0x565546b9fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66d8b00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546b98480 0x565546ba01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66d8b00740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546ba2d40 0x565546ba0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66d8b00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565546ba5910 con 0x565546b98480 2024-09-19T06:55:32.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66d8b00740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565546ab7f10 con 0x565546ba2d40 2024-09-19T06:55:32.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66d8b00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565546ba5a90 con 0x565546b97b60 2024-09-19T06:55:32.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66d8b00740 10 client.0 _ll_register_callbacks cb 0x565546a80a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:32.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66cf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546b98480 0x565546ba01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:32.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66cf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546b98480 0x565546ba01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34988/0 (socket says 172.21.15.103:34988) 2024-09-19T06:55:32.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66cf7fe640 1 -- 172.21.15.103:0/3125819642 learned_addr learned my addr 172.21.15.103:0/3125819642 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:32.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66cf7fe640 1 -- 172.21.15.103:0/3125819642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546b97b60 msgr2=0x565546b9fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:32.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66cf7fe640 1 --2- 172.21.15.103:0/3125819642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546b97b60 0x565546b9fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66cf7fe640 1 -- 172.21.15.103:0/3125819642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546ba2d40 msgr2=0x565546ba0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:32.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66d4cc4640 1 --2- 172.21.15.103:0/3125819642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546ba2d40 0x565546ba0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:32.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66cf7fe640 1 --2- 172.21.15.103:0/3125819642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546ba2d40 0x565546ba0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66cf7fe640 1 -- 172.21.15.103:0/3125819642 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554686e6a0 con 0x565546b98480 2024-09-19T06:55:32.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66cf7fe640 1 --2- 172.21.15.103:0/3125819642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546b98480 0x565546ba01e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f66c400e2d0 tx=0x7f66c400e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800ed9279d6c4ad1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:32.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:32.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.211+0000 7f66af7fe640 1 -- 172.21.15.103:0/3125819642 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c4017020 con 0x565546b98480 2024-09-19T06:55:32.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.212+0000 7f66af7fe640 1 -- 172.21.15.103:0/3125819642 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f66c4013660 con 0x565546b98480 2024-09-19T06:55:32.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.212+0000 7f66d8b00740 1 -- 172.21.15.103:0/3125819642 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66c8011b40 con 0x565546b98480 2024-09-19T06:55:32.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.212+0000 7f66af7fe640 1 -- 172.21.15.103:0/3125819642 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66c4022050 con 0x565546b98480 2024-09-19T06:55:32.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.212+0000 7f66d8b00740 10 client.17724 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:32.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.212+0000 7f66d8b00740 1 -- 172.21.15.103:0/3125819642 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565546bb15f0 con 0x565546b98480 2024-09-19T06:55:32.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.212+0000 7f66af7fe640 1 -- 172.21.15.103:0/3125819642 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66c8011b40 con 0x565546b98480 2024-09-19T06:55:32.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.212+0000 7f66af7fe640 1 -- 172.21.15.103:0/3125819642 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f66c4012af0 con 0x565546b98480 2024-09-19T06:55:32.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.212+0000 7f66d8b00740 10 client.17724 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:32.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.212+0000 7f66d8b00740 10 client.17724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:32.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.212+0000 7f66d8b00740 20 client.17724 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:32.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.212+0000 7f66d8b00740 1 --2- 172.21.15.103:0/3125819642 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x565546bb2b00 0x565546bb4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.212+0000 7f66d8b00740 4 client.17724 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:32.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.212+0000 7f66d8b00740 1 -- 172.21.15.103:0/3125819642 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565546b0a7f0 con 0x565546bb2b00 2024-09-19T06:55:32.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.213+0000 7f66d4cc4640 1 --2- 172.21.15.103:0/3125819642 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x565546bb2b00 0x565546bb4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:32.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.213+0000 7f66d4cc4640 1 --2- 172.21.15.103:0/3125819642 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x565546bb2b00 0x565546bb4f80 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee44d7451a124f7c server_cookie=53d80f9a4c71eec3 in_seq=0 out_seq=0 2024-09-19T06:55:32.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.213+0000 7f66af7fe640 10 client.17724 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:32.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.214+0000 7f66af7fe640 1 -- 172.21.15.103:0/3125819642 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565546b0a7f0 con 0x565546bb2b00 2024-09-19T06:55:32.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.214+0000 7f66af7fe640 10 client.17724 handle_command_reply: tid=0 2024-09-19T06:55:32.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.214+0000 7f66d8b00740 1 client.17724 shutdown 2024-09-19T06:55:32.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.214+0000 7f66d8b00740 1 -- 172.21.15.103:0/3125819642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546b98480 msgr2=0x565546ba01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:32.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.214+0000 7f66d8b00740 1 --2- 172.21.15.103:0/3125819642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546b98480 0x565546ba01e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f66c400e2d0 tx=0x7f66c400e790 comp rx=0 tx=0).stop 2024-09-19T06:55:32.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.214+0000 7f66d8b00740 1 -- 172.21.15.103:0/3125819642 shutdown_connections 2024-09-19T06:55:32.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.214+0000 7f66d8b00740 1 --2- 172.21.15.103:0/3125819642 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x565546bb2b00 0x565546bb4f80 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.214+0000 7f66d8b00740 1 --2- 172.21.15.103:0/3125819642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546ba2d40 0x565546ba0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.214+0000 7f66d8b00740 1 --2- 172.21.15.103:0/3125819642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546b98480 0x565546ba01e0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.214+0000 7f66d8b00740 1 --2- 172.21.15.103:0/3125819642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546b97b60 0x565546b9fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.214+0000 7f66d8b00740 1 -- 172.21.15.103:0/3125819642 >> 172.21.15.103:0/3125819642 conn(0x565546b940f0 msgr2=0x565546b99dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:32.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.214+0000 7f66af7fe640 0 client.17724 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:32.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.214+0000 7f66d8b00740 1 -- 172.21.15.103:0/3125819642 shutdown_connections 2024-09-19T06:55:32.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.214+0000 7f66d8b00740 1 -- 172.21.15.103:0/3125819642 wait complete. 2024-09-19T06:55:32.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.214+0000 7f66d8b00740 20 client.17724 trim_cache size 0 max 16384 2024-09-19T06:55:32.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.231+0000 7f66d8b00740 1 Processor -- start 2024-09-19T06:55:32.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.231+0000 7f66d8b00740 1 -- start start 2024-09-19T06:55:32.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.231+0000 7f66d8b00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546b97b60 0x565546ba1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.231+0000 7f66d8b00740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546b98480 0x565546bba850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.231+0000 7f66d8b00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546ba2d40 0x565546bbad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.231+0000 7f66d8b00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01ad990 con 0x565546b97b60 2024-09-19T06:55:32.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.231+0000 7f66d8b00740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01af4d0 con 0x565546b98480 2024-09-19T06:55:32.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.231+0000 7f66d8b00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01ae060 con 0x565546ba2d40 2024-09-19T06:55:32.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.232+0000 7f66d4cc4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546b97b60 0x565546ba1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:32.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.232+0000 7f66d4cc4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546b97b60 0x565546ba1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34994/0 (socket says 172.21.15.103:34994) 2024-09-19T06:55:32.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.232+0000 7f66d4cc4640 1 -- 172.21.15.103:0/3494574942 learned_addr learned my addr 172.21.15.103:0/3494574942 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:32.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.232+0000 7f66d4cc4640 1 --2- 172.21.15.103:0/3494574942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546ba2d40 0x565546bbad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:32.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.232+0000 7f66cf7fe640 1 --2- 172.21.15.103:0/3494574942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546b98480 0x565546bba850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:32.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.232+0000 7f66d4cc4640 1 -- 172.21.15.103:0/3494574942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546ba2d40 msgr2=0x565546bbad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:32.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.232+0000 7f66d4cc4640 1 --2- 172.21.15.103:0/3494574942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546ba2d40 0x565546bbad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.232+0000 7f66d4cc4640 1 -- 172.21.15.103:0/3494574942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546b98480 msgr2=0x565546bba850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:32.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.232+0000 7f66d4cc4640 1 --2- 172.21.15.103:0/3494574942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546b98480 0x565546bba850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.232+0000 7f66d4cc4640 1 -- 172.21.15.103:0/3494574942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565546ba2570 con 0x565546b97b60 2024-09-19T06:55:32.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.232+0000 7f66cf7fe640 1 --2- 172.21.15.103:0/3494574942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546b98480 0x565546bba850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:32.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.232+0000 7f66d4cc4640 1 --2- 172.21.15.103:0/3494574942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546b97b60 0x565546ba1b00 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f66c8018720 tx=0x7f66c80186b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0874c144344602 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:32.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.232+0000 7f66b5ffb640 1 -- 172.21.15.103:0/3494574942 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8027040 con 0x565546b97b60 2024-09-19T06:55:32.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.232+0000 7f66b5ffb640 1 -- 172.21.15.103:0/3494574942 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66c80187e0 con 0x565546b97b60 2024-09-19T06:55:32.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.232+0000 7f66b5ffb640 1 -- 172.21.15.103:0/3494574942 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c801d040 con 0x565546b97b60 2024-09-19T06:55:32.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.233+0000 7f66d8b00740 1 -- 172.21.15.103:0/3494574942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546b97b60 msgr2=0x565546ba1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:32.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.233+0000 7f66d8b00740 1 --2- 172.21.15.103:0/3494574942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546b97b60 0x565546ba1b00 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f66c8018720 tx=0x7f66c80186b0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.233+0000 7f66d8b00740 1 -- 172.21.15.103:0/3494574942 shutdown_connections 2024-09-19T06:55:32.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.233+0000 7f66d8b00740 1 --2- 172.21.15.103:0/3494574942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546ba2d40 0x565546bbad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.233+0000 7f66d8b00740 1 --2- 172.21.15.103:0/3494574942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546b98480 0x565546bba850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.233+0000 7f66d8b00740 1 --2- 172.21.15.103:0/3494574942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546b97b60 0x565546ba1b00 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.233+0000 7f66d8b00740 1 -- 172.21.15.103:0/3494574942 >> 172.21.15.103:0/3494574942 conn(0x565546b940f0 msgr2=0x565546ba13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:32.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.233+0000 7f66d8b00740 1 -- 172.21.15.103:0/3494574942 shutdown_connections 2024-09-19T06:55:32.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.233+0000 7f66d8b00740 1 -- 172.21.15.103:0/3494574942 wait complete. 2024-09-19T06:55:32.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.233+0000 7f66d8b00740 1 -- start start 2024-09-19T06:55:32.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d8b00740 1 Processor -- start 2024-09-19T06:55:32.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d8b00740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546b97b60 0x565546bab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d8b00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546b98480 0x565546ba7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d8b00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546ba2d40 0x565546ba81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d8b00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01aee00 con 0x565546ba2d40 2024-09-19T06:55:32.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d8b00740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01ae730 con 0x565546b97b60 2024-09-19T06:55:32.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d8b00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01ab950 con 0x565546b98480 2024-09-19T06:55:32.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d8b00740 10 client.0 _ll_register_callbacks cb 0x7f66d006b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:32.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d4cc4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546ba2d40 0x565546ba81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:32.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d4cc4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546ba2d40 0x565546ba81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34996/0 (socket says 172.21.15.103:34996) 2024-09-19T06:55:32.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d4cc4640 1 -- 172.21.15.103:0/1969963396 learned_addr learned my addr 172.21.15.103:0/1969963396 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:32.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d4cc4640 1 -- 172.21.15.103:0/1969963396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546b98480 msgr2=0x565546ba7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:32.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d4cc4640 1 --2- 172.21.15.103:0/1969963396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546b98480 0x565546ba7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d4cc4640 1 -- 172.21.15.103:0/1969963396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546b97b60 msgr2=0x565546bab2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:32.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d4cc4640 1 --2- 172.21.15.103:0/1969963396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546b97b60 0x565546bab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d4cc4640 1 -- 172.21.15.103:0/1969963396 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565546ba8860 con 0x565546ba2d40 2024-09-19T06:55:32.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66d4cc4640 1 --2- 172.21.15.103:0/1969963396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546ba2d40 0x565546ba81f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f66c80187b0 tx=0x7f66c8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3b0d443edc44fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:32.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.234+0000 7f66af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:32.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.235+0000 7f66af7fe640 1 -- 172.21.15.103:0/1969963396 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8027040 con 0x565546ba2d40 2024-09-19T06:55:32.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.235+0000 7f66d8b00740 1 -- 172.21.15.103:0/1969963396 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66c80187e0 con 0x565546ba2d40 2024-09-19T06:55:32.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.235+0000 7f66af7fe640 1 -- 172.21.15.103:0/1969963396 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f66c80116e0 con 0x565546ba2d40 2024-09-19T06:55:32.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.235+0000 7f66af7fe640 1 -- 172.21.15.103:0/1969963396 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66c801d040 con 0x565546ba2d40 2024-09-19T06:55:32.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.235+0000 7f66d8b00740 10 client.17730 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:32.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.235+0000 7f66d8b00740 1 -- 172.21.15.103:0/1969963396 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565546c1e650 con 0x565546ba2d40 2024-09-19T06:55:32.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.235+0000 7f66af7fe640 1 -- 172.21.15.103:0/1969963396 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66c80187e0 con 0x565546ba2d40 2024-09-19T06:55:32.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.235+0000 7f66af7fe640 1 -- 172.21.15.103:0/1969963396 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f66c8025740 con 0x565546ba2d40 2024-09-19T06:55:32.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.235+0000 7f66d8b00740 10 client.17730 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:32.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.235+0000 7f66d8b00740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:32.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.235+0000 7f66d8b00740 20 client.17730 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:32.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.235+0000 7f66d8b00740 1 --2- 172.21.15.103:0/1969963396 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x565546c1eac0 0x565546c20eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:32.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.235+0000 7f66d8b00740 4 client.17730 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:55:32.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.235+0000 7f66d8b00740 1 -- 172.21.15.103:0/1969963396 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x565546b04d00 con 0x565546c1eac0 2024-09-19T06:55:32.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.236+0000 7f66cf7fe640 1 --2- 172.21.15.103:0/1969963396 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x565546c1eac0 0x565546c20eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:32.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.236+0000 7f66cf7fe640 1 --2- 172.21.15.103:0/1969963396 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x565546c1eac0 0x565546c20eb0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eda57f0861ed480 server_cookie=14dc3d7c3b20d28a in_seq=0 out_seq=0 2024-09-19T06:55:32.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.236+0000 7f66af7fe640 10 client.17730 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:32.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.237+0000 7f66af7fe640 1 -- 172.21.15.103:0/1969963396 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x565546b04d00 con 0x565546c1eac0 2024-09-19T06:55:32.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.237+0000 7f66af7fe640 10 client.17730 handle_command_reply: tid=0 2024-09-19T06:55:32.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.237+0000 7f66d8b00740 1 client.17730 shutdown 2024-09-19T06:55:32.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.237+0000 7f66d8b00740 1 -- 172.21.15.103:0/1969963396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546ba2d40 msgr2=0x565546ba81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:32.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.237+0000 7f66d8b00740 1 --2- 172.21.15.103:0/1969963396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546ba2d40 0x565546ba81f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f66c80187b0 tx=0x7f66c8009f80 comp rx=0 tx=0).stop 2024-09-19T06:55:32.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.237+0000 7f66d8b00740 1 -- 172.21.15.103:0/1969963396 shutdown_connections 2024-09-19T06:55:32.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.237+0000 7f66d8b00740 1 --2- 172.21.15.103:0/1969963396 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x565546c1eac0 0x565546c20eb0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.237+0000 7f66d8b00740 1 --2- 172.21.15.103:0/1969963396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565546ba2d40 0x565546ba81f0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.237+0000 7f66af7fe640 0 client.17730 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:32.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.237+0000 7f66d8b00740 1 --2- 172.21.15.103:0/1969963396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565546b98480 0x565546ba7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.237+0000 7f66d8b00740 1 --2- 172.21.15.103:0/1969963396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565546b97b60 0x565546bab2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.237+0000 7f66d8b00740 1 -- 172.21.15.103:0/1969963396 >> 172.21.15.103:0/1969963396 conn(0x565546b940f0 msgr2=0x565546b99dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:32.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.238+0000 7f66d8b00740 1 -- 172.21.15.103:0/1969963396 shutdown_connections 2024-09-19T06:55:32.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.238+0000 7f66d8b00740 1 -- 172.21.15.103:0/1969963396 wait complete. 2024-09-19T06:55:32.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.238+0000 7f66d8b00740 20 client.17730 trim_cache size 0 max 16384 2024-09-19T06:55:32.246 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:55:32.246 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T06:55:32.246 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "35d0501c-c126-49f4-b1c5-6e050f552e2d", 2024-09-19T06:55:32.246 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T06:55:32.246 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:55:32.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.238+0000 7f66d6727640 1 -- 172.21.15.103:0/1402495353 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66b80a33b0 msgr2=0x7f66b80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:32.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.238+0000 7f66d6727640 1 --2- 172.21.15.103:0/1402495353 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66b80a33b0 0x7f66b80a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f66c00c50c0 tx=0x7f66c00b1e10 comp rx=0 tx=0).stop 2024-09-19T06:55:32.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.238+0000 7f66d6727640 1 -- 172.21.15.103:0/1402495353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66d0102d40 msgr2=0x7f66d0198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:32.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.238+0000 7f66d6727640 1 --2- 172.21.15.103:0/1402495353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66d0102d40 0x7f66d0198690 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f66c0005ab0 tx=0x7f66c00042f0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.238+0000 7f66d6727640 1 -- 172.21.15.103:0/1402495353 shutdown_connections 2024-09-19T06:55:32.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.238+0000 7f66d6727640 1 --2- 172.21.15.103:0/1402495353 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66b80a33b0 0x7f66b80a5870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.238+0000 7f66d6727640 1 --2- 172.21.15.103:0/1402495353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66d010c170 0x7f66d019cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.238+0000 7f66d6727640 1 --2- 172.21.15.103:0/1402495353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66d0103660 0x7f66d0198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.238+0000 7f66d6727640 1 --2- 172.21.15.103:0/1402495353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66d0102d40 0x7f66d0198690 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:32.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.238+0000 7f66d6727640 1 -- 172.21.15.103:0/1402495353 >> 172.21.15.103:0/1402495353 conn(0x7f66d00fc7a0 msgr2=0x7f66d00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:32.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.238+0000 7f66d6727640 1 -- 172.21.15.103:0/1402495353 shutdown_connections 2024-09-19T06:55:32.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:32.238+0000 7f66d6727640 1 -- 172.21.15.103:0/1402495353 wait complete. 2024-09-19T06:55:32.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:32 smithi134 ceph-mon[35356]: pgmap v192: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:55:32.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:32 smithi174 ceph-mon[35637]: pgmap v192: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:55:33.046 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:55:33.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:33 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T06:55:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:33 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:33 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T06:55:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:33 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.175+0000 7f197e9be640 1 -- 172.21.15.103:0/1494484402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19781069c0 msgr2=0x7f1978106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.175+0000 7f197e9be640 1 --2- 172.21.15.103:0/1494484402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19781069c0 0x7f1978106da0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1960009930 tx=0x7f196002a420 comp rx=0 tx=0).stop 2024-09-19T06:55:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.175+0000 7f197e9be640 1 -- 172.21.15.103:0/1494484402 shutdown_connections 2024-09-19T06:55:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.175+0000 7f197e9be640 1 --2- 172.21.15.103:0/1494484402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1978101350 0x7f197810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.175+0000 7f197e9be640 1 --2- 172.21.15.103:0/1494484402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19781009b0 0x7f1978100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.175+0000 7f197e9be640 1 --2- 172.21.15.103:0/1494484402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19781069c0 0x7f1978106da0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.175+0000 7f197e9be640 1 -- 172.21.15.103:0/1494484402 >> 172.21.15.103:0/1494484402 conn(0x7f1978077990 msgr2=0x7f19780fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.175+0000 7f197e9be640 1 -- 172.21.15.103:0/1494484402 shutdown_connections 2024-09-19T06:55:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.175+0000 7f197e9be640 1 -- 172.21.15.103:0/1494484402 wait complete. 2024-09-19T06:55:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.176+0000 7f197e9be640 1 Processor -- start 2024-09-19T06:55:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.176+0000 7f197e9be640 1 -- start start 2024-09-19T06:55:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.176+0000 7f197e9be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19781009b0 0x7f1978199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.176+0000 7f197e9be640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1978101350 0x7f1978199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.176+0000 7f197e9be640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19781069c0 0x7f19781a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.176+0000 7f197e9be640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1978110e00 con 0x7f19781009b0 2024-09-19T06:55:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.176+0000 7f197e9be640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1978110c80 con 0x7f19781069c0 2024-09-19T06:55:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.176+0000 7f197e9be640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1978110f80 con 0x7f1978101350 2024-09-19T06:55:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.176+0000 7f1977fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19781009b0 0x7f1978199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.176+0000 7f1977fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19781009b0 0x7f1978199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35022/0 (socket says 172.21.15.103:35022) 2024-09-19T06:55:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.176+0000 7f1977fff640 1 -- 172.21.15.103:0/1095618356 learned_addr learned my addr 172.21.15.103:0/1095618356 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.176+0000 7f197cf5b640 1 --2- 172.21.15.103:0/1095618356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19781069c0 0x7f19781a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.176+0000 7f1977fff640 1 -- 172.21.15.103:0/1095618356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1978101350 msgr2=0x7f1978199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.176+0000 7f19777fe640 1 --2- 172.21.15.103:0/1095618356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1978101350 0x7f1978199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.177+0000 7f1977fff640 1 --2- 172.21.15.103:0/1095618356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1978101350 0x7f1978199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.177+0000 7f1977fff640 1 -- 172.21.15.103:0/1095618356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19781069c0 msgr2=0x7f19781a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.177+0000 7f1977fff640 1 --2- 172.21.15.103:0/1095618356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19781069c0 0x7f19781a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.177+0000 7f1977fff640 1 -- 172.21.15.103:0/1095618356 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19781a47c0 con 0x7f19781009b0 2024-09-19T06:55:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.177+0000 7f197cf5b640 1 --2- 172.21.15.103:0/1095618356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19781069c0 0x7f19781a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.177+0000 7f1977fff640 1 --2- 172.21.15.103:0/1095618356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19781009b0 0x7f1978199200 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f1960009230 tx=0x7f1960004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.177+0000 7f19757fa640 1 -- 172.21.15.103:0/1095618356 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196003f030 con 0x7f19781009b0 2024-09-19T06:55:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.177+0000 7f197e9be640 1 -- 172.21.15.103:0/1095618356 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19781a4990 con 0x7f19781009b0 2024-09-19T06:55:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.177+0000 7f19757fa640 1 -- 172.21.15.103:0/1095618356 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1960003ad0 con 0x7f19781009b0 2024-09-19T06:55:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.177+0000 7f19757fa640 1 -- 172.21.15.103:0/1095618356 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960035070 con 0x7f19781009b0 2024-09-19T06:55:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.177+0000 7f197e9be640 1 -- 172.21.15.103:0/1095618356 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19781a4e50 con 0x7f19781009b0 2024-09-19T06:55:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f1981830740 1 Processor -- start 2024-09-19T06:55:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f1981830740 1 -- start start 2024-09-19T06:55:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f19757fa640 1 -- 172.21.15.103:0/1095618356 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1960042030 con 0x7f19781009b0 2024-09-19T06:55:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f1981830740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad1af0 0x563aa3ad1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f1981830740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3ad2410 0x563aa3adc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f1981830740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3adf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f1981830740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563aa3a362f0 con 0x563aa3adccb0 2024-09-19T06:55:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f1981830740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563aa39de270 con 0x563aa3ad2410 2024-09-19T06:55:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f1981830740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563aa3a322b0 con 0x563aa3ad1af0 2024-09-19T06:55:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f19777fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3adf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f19777fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3adf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35026/0 (socket says 172.21.15.103:35026) 2024-09-19T06:55:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f19777fe640 1 -- 172.21.15.103:0/1566221867 learned_addr learned my addr 172.21.15.103:0/1566221867 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f19777fe640 1 -- 172.21.15.103:0/1566221867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad1af0 msgr2=0x563aa3ad1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f1977fff640 1 --2- 172.21.15.103:0/1566221867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3ad2410 0x563aa3adc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f197cf5b640 1 --2- 172.21.15.103:0/1566221867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad1af0 0x563aa3ad1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f19777fe640 1 --2- 172.21.15.103:0/1566221867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad1af0 0x563aa3ad1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f19777fe640 1 -- 172.21.15.103:0/1566221867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3ad2410 msgr2=0x563aa3adc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f19777fe640 1 --2- 172.21.15.103:0/1566221867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3ad2410 0x563aa3adc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f19777fe640 1 -- 172.21.15.103:0/1566221867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aa39dd340 con 0x563aa3adccb0 2024-09-19T06:55:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f197cf5b640 1 --2- 172.21.15.103:0/1566221867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad1af0 0x563aa3ad1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f1977fff640 1 --2- 172.21.15.103:0/1566221867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3ad2410 0x563aa3adc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.180+0000 7f19757fa640 1 --2- 172.21.15.103:0/1095618356 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19640a3460 0x7f19640a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.181+0000 7f19777fe640 1 --2- 172.21.15.103:0/1566221867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3adf0a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f196800ba70 tx=0x7f196800bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fadab1046f23ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.181+0000 7f197cf5b640 1 --2- 172.21.15.103:0/1095618356 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19640a3460 0x7f19640a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.181+0000 7f196e7fc640 1 -- 172.21.15.103:0/1566221867 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968017420 con 0x563aa3adccb0 2024-09-19T06:55:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.181+0000 7f19757fa640 1 -- 172.21.15.103:0/1095618356 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f19600f02b0 con 0x7f19781009b0 2024-09-19T06:55:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.181+0000 7f196e7fc640 1 -- 172.21.15.103:0/1566221867 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1968017aa0 con 0x563aa3adccb0 2024-09-19T06:55:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.181+0000 7f196e7fc640 1 -- 172.21.15.103:0/1566221867 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196800fb70 con 0x563aa3adccb0 2024-09-19T06:55:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.181+0000 7f197cf5b640 1 --2- 172.21.15.103:0/1095618356 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19640a3460 0x7f19640a5920 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1970009370 tx=0x7f1970009510 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.181+0000 7f1981830740 1 -- 172.21.15.103:0/1566221867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 msgr2=0x563aa3adf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.181+0000 7f1981830740 1 --2- 172.21.15.103:0/1566221867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3adf0a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f196800ba70 tx=0x7f196800bf30 comp rx=0 tx=0).stop 2024-09-19T06:55:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.182+0000 7f1981830740 1 -- 172.21.15.103:0/1566221867 shutdown_connections 2024-09-19T06:55:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.182+0000 7f1981830740 1 --2- 172.21.15.103:0/1566221867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3adf0a0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.182+0000 7f1981830740 1 --2- 172.21.15.103:0/1566221867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3ad2410 0x563aa3adc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.182+0000 7f1981830740 1 --2- 172.21.15.103:0/1566221867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad1af0 0x563aa3ad1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.182+0000 7f1981830740 1 -- 172.21.15.103:0/1566221867 >> 172.21.15.103:0/1566221867 conn(0x563aa3ace060 msgr2=0x563aa3ace440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.182+0000 7f1981830740 1 -- 172.21.15.103:0/1566221867 shutdown_connections 2024-09-19T06:55:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.182+0000 7f1981830740 1 -- 172.21.15.103:0/1566221867 wait complete. 2024-09-19T06:55:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.182+0000 7f1981830740 1 -- start start 2024-09-19T06:55:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.182+0000 7f1981830740 1 Processor -- start 2024-09-19T06:55:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.182+0000 7f1981830740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3ad1af0 0x563aa3ad9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.182+0000 7f1981830740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad2410 0x563aa3ad9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.182+0000 7f1981830740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3adccb0 0x563aa3ada4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.182+0000 7f1981830740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563aa3adf880 con 0x563aa3ad1af0 2024-09-19T06:55:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.182+0000 7f1981830740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563aa39bf290 con 0x563aa3adccb0 2024-09-19T06:55:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.182+0000 7f1981830740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563aa3adfa00 con 0x563aa3ad2410 2024-09-19T06:55:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f1981830740 10 client.0 _ll_register_callbacks cb 0x563aa39f0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f1977fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3ad1af0 0x563aa3ad9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f1977fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3ad1af0 0x563aa3ad9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35034/0 (socket says 172.21.15.103:35034) 2024-09-19T06:55:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f1977fff640 1 -- 172.21.15.103:0/379300413 learned_addr learned my addr 172.21.15.103:0/379300413 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f197cf5b640 1 --2- 172.21.15.103:0/379300413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3adccb0 0x563aa3ada4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f19777fe640 1 --2- 172.21.15.103:0/379300413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad2410 0x563aa3ad9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f1977fff640 1 -- 172.21.15.103:0/379300413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad2410 msgr2=0x563aa3ad9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f1977fff640 1 --2- 172.21.15.103:0/379300413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad2410 0x563aa3ad9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f1977fff640 1 -- 172.21.15.103:0/379300413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3adccb0 msgr2=0x563aa3ada4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f1977fff640 1 --2- 172.21.15.103:0/379300413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3adccb0 0x563aa3ada4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f1977fff640 1 -- 172.21.15.103:0/379300413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aa3749210 con 0x563aa3ad1af0 2024-09-19T06:55:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f197cf5b640 1 --2- 172.21.15.103:0/379300413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3adccb0 0x563aa3ada4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f1977fff640 1 --2- 172.21.15.103:0/379300413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3ad1af0 0x563aa3ad9a30 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f1960009a50 tx=0x7f19600bbf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7393f5e29f99b9c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f19577fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f19577fe640 1 -- 172.21.15.103:0/379300413 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960049070 con 0x563aa3ad1af0 2024-09-19T06:55:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f19577fe640 1 -- 172.21.15.103:0/379300413 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1960003c80 con 0x563aa3ad1af0 2024-09-19T06:55:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f1981830740 1 -- 172.21.15.103:0/379300413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1968017aa0 con 0x563aa3ad1af0 2024-09-19T06:55:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f19577fe640 1 -- 172.21.15.103:0/379300413 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f19600c0950 con 0x563aa3ad1af0 2024-09-19T06:55:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f1981830740 10 client.17742 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f1981830740 1 -- 172.21.15.103:0/379300413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563aa3815620 con 0x563aa3ad1af0 2024-09-19T06:55:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.183+0000 7f19577fe640 1 -- 172.21.15.103:0/379300413 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1968017aa0 con 0x563aa3ad1af0 2024-09-19T06:55:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.184+0000 7f19577fe640 1 -- 172.21.15.103:0/379300413 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f19600bc800 con 0x563aa3ad1af0 2024-09-19T06:55:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.184+0000 7f1981830740 10 client.17742 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.184+0000 7f1981830740 10 client.17742 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.184+0000 7f1981830740 20 client.17742 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.184+0000 7f1981830740 1 --2- 172.21.15.103:0/379300413 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563aa3aeca20 0x563aa3aeee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.184+0000 7f1981830740 4 client.17742 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.184+0000 7f1981830740 1 -- 172.21.15.103:0/379300413 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563aa3a43a20 con 0x563aa3aeca20 2024-09-19T06:55:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.184+0000 7f19777fe640 1 --2- 172.21.15.103:0/379300413 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563aa3aeca20 0x563aa3aeee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.185+0000 7f19577fe640 10 client.17742 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.185+0000 7f19777fe640 1 --2- 172.21.15.103:0/379300413 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563aa3aeca20 0x563aa3aeee70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96c50994e8de6a44 server_cookie=c1e381d77739f1c in_seq=0 out_seq=0 2024-09-19T06:55:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.185+0000 7f19577fe640 1 -- 172.21.15.103:0/379300413 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563aa3a43a20 con 0x563aa3aeca20 2024-09-19T06:55:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.185+0000 7f19577fe640 10 client.17742 handle_command_reply: tid=0 2024-09-19T06:55:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.185+0000 7f1981830740 1 client.17742 shutdown 2024-09-19T06:55:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.185+0000 7f1981830740 1 -- 172.21.15.103:0/379300413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3ad1af0 msgr2=0x563aa3ad9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.185+0000 7f1981830740 1 --2- 172.21.15.103:0/379300413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3ad1af0 0x563aa3ad9a30 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f1960009a50 tx=0x7f19600bbf00 comp rx=0 tx=0).stop 2024-09-19T06:55:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.185+0000 7f1981830740 1 -- 172.21.15.103:0/379300413 shutdown_connections 2024-09-19T06:55:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.185+0000 7f1981830740 1 --2- 172.21.15.103:0/379300413 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563aa3aeca20 0x563aa3aeee70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.185+0000 7f1981830740 1 --2- 172.21.15.103:0/379300413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3adccb0 0x563aa3ada4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.185+0000 7f1981830740 1 --2- 172.21.15.103:0/379300413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad2410 0x563aa3ad9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.185+0000 7f1981830740 1 --2- 172.21.15.103:0/379300413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3ad1af0 0x563aa3ad9a30 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.185+0000 7f1981830740 1 -- 172.21.15.103:0/379300413 >> 172.21.15.103:0/379300413 conn(0x563aa3ace060 msgr2=0x563aa3ad3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.185+0000 7f19577fe640 0 client.17742 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.186+0000 7f1981830740 1 -- 172.21.15.103:0/379300413 shutdown_connections 2024-09-19T06:55:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.186+0000 7f1981830740 1 -- 172.21.15.103:0/379300413 wait complete. 2024-09-19T06:55:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.186+0000 7f1981830740 20 client.17742 trim_cache size 0 max 16384 2024-09-19T06:55:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f1981830740 1 Processor -- start 2024-09-19T06:55:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f1981830740 1 -- start start 2024-09-19T06:55:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f1981830740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3ad1af0 0x563aa3ad4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f1981830740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad2410 0x563aa3ada2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f1981830740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3af4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f1981830740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19781ad810 con 0x563aa3adccb0 2024-09-19T06:55:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f1981830740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19781af350 con 0x563aa3ad1af0 2024-09-19T06:55:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f1981830740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f19781adee0 con 0x563aa3ad2410 2024-09-19T06:55:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f197cf5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3af4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f197cf5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3af4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35044/0 (socket says 172.21.15.103:35044) 2024-09-19T06:55:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f197cf5b640 1 -- 172.21.15.103:0/4189107059 learned_addr learned my addr 172.21.15.103:0/4189107059 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f19777fe640 1 --2- 172.21.15.103:0/4189107059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad2410 0x563aa3ada2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f197cf5b640 1 -- 172.21.15.103:0/4189107059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad2410 msgr2=0x563aa3ada2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f197cf5b640 1 --2- 172.21.15.103:0/4189107059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad2410 0x563aa3ada2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f197cf5b640 1 -- 172.21.15.103:0/4189107059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3ad1af0 msgr2=0x563aa3ad4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f1977fff640 1 --2- 172.21.15.103:0/4189107059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3ad1af0 0x563aa3ad4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f197cf5b640 1 --2- 172.21.15.103:0/4189107059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3ad1af0 0x563aa3ad4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.200+0000 7f197cf5b640 1 -- 172.21.15.103:0/4189107059 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aa3acde90 con 0x563aa3adccb0 2024-09-19T06:55:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.201+0000 7f197cf5b640 1 --2- 172.21.15.103:0/4189107059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3af4720 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1970011810 tx=0x7f1970012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f754d761e8d7045 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.201+0000 7f196dffb640 1 -- 172.21.15.103:0/4189107059 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970010950 con 0x563aa3adccb0 2024-09-19T06:55:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.201+0000 7f196dffb640 1 -- 172.21.15.103:0/4189107059 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f197001f430 con 0x563aa3adccb0 2024-09-19T06:55:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.201+0000 7f196dffb640 1 -- 172.21.15.103:0/4189107059 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970027660 con 0x563aa3adccb0 2024-09-19T06:55:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 -- 172.21.15.103:0/4189107059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 msgr2=0x563aa3af4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 --2- 172.21.15.103:0/4189107059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3af4720 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1970011810 tx=0x7f1970012000 comp rx=0 tx=0).stop 2024-09-19T06:55:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 -- 172.21.15.103:0/4189107059 shutdown_connections 2024-09-19T06:55:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 --2- 172.21.15.103:0/4189107059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3af4720 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 --2- 172.21.15.103:0/4189107059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad2410 0x563aa3ada2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 --2- 172.21.15.103:0/4189107059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3ad1af0 0x563aa3ad4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 -- 172.21.15.103:0/4189107059 >> 172.21.15.103:0/4189107059 conn(0x563aa3ace060 msgr2=0x563aa3adbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 -- 172.21.15.103:0/4189107059 shutdown_connections 2024-09-19T06:55:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 -- 172.21.15.103:0/4189107059 wait complete. 2024-09-19T06:55:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 -- start start 2024-09-19T06:55:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 Processor -- start 2024-09-19T06:55:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad1af0 0x563aa3addfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3ad2410 0x563aa3ade4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3adea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19781aec80 con 0x563aa3adccb0 2024-09-19T06:55:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19781ae5b0 con 0x563aa3ad2410 2024-09-19T06:55:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f19781ab7d0 con 0x563aa3ad1af0 2024-09-19T06:55:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.202+0000 7f1981830740 10 client.0 _ll_register_callbacks cb 0x7f19600b8180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f197cf5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3adea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f197cf5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3adea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35046/0 (socket says 172.21.15.103:35046) 2024-09-19T06:55:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f197cf5b640 1 -- 172.21.15.103:0/4204599476 learned_addr learned my addr 172.21.15.103:0/4204599476 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f19777fe640 1 --2- 172.21.15.103:0/4204599476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3ad2410 0x563aa3ade4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f1977fff640 1 --2- 172.21.15.103:0/4204599476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad1af0 0x563aa3addfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f197cf5b640 1 -- 172.21.15.103:0/4204599476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad1af0 msgr2=0x563aa3addfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f197cf5b640 1 --2- 172.21.15.103:0/4204599476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad1af0 0x563aa3addfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f197cf5b640 1 -- 172.21.15.103:0/4204599476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3ad2410 msgr2=0x563aa3ade4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f197cf5b640 1 --2- 172.21.15.103:0/4204599476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3ad2410 0x563aa3ade4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f197cf5b640 1 -- 172.21.15.103:0/4204599476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aa3ae3650 con 0x563aa3adccb0 2024-09-19T06:55:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f197cf5b640 1 --2- 172.21.15.103:0/4204599476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3adea30 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1970011730 tx=0x7f1970005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b56e8bac0390fe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f19577fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f19577fe640 1 -- 172.21.15.103:0/4204599476 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970036060 con 0x563aa3adccb0 2024-09-19T06:55:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f19577fe640 1 -- 172.21.15.103:0/4204599476 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f1970010b60 con 0x563aa3adccb0 2024-09-19T06:55:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f1981830740 1 -- 172.21.15.103:0/4204599476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f197001f430 con 0x563aa3adccb0 2024-09-19T06:55:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f19577fe640 1 -- 172.21.15.103:0/4204599476 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f197001fe60 con 0x563aa3adccb0 2024-09-19T06:55:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f1981830740 10 client.17748 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.203+0000 7f1981830740 1 -- 172.21.15.103:0/4204599476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563aa3b5bf90 con 0x563aa3adccb0 2024-09-19T06:55:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.204+0000 7f19577fe640 1 -- 172.21.15.103:0/4204599476 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f197001f430 con 0x563aa3adccb0 2024-09-19T06:55:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.204+0000 7f19577fe640 1 -- 172.21.15.103:0/4204599476 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1970031860 con 0x563aa3adccb0 2024-09-19T06:55:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.204+0000 7f1981830740 10 client.17748 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.204+0000 7f1981830740 10 client.17748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.204+0000 7f1981830740 20 client.17748 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.204+0000 7f1981830740 1 --2- 172.21.15.103:0/4204599476 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563aa3b5c5a0 0x563aa3b5e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.204+0000 7f1981830740 4 client.17748 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:55:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.204+0000 7f1981830740 1 -- 172.21.15.103:0/4204599476 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563aa3a3dd70 con 0x563aa3b5c5a0 2024-09-19T06:55:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.204+0000 7f1977fff640 1 --2- 172.21.15.103:0/4204599476 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563aa3b5c5a0 0x563aa3b5e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.205+0000 7f1977fff640 1 --2- 172.21.15.103:0/4204599476 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563aa3b5c5a0 0x563aa3b5e9c0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73a9ab2ac1a43a51 server_cookie=39ebc7146507c823 in_seq=0 out_seq=0 2024-09-19T06:55:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.205+0000 7f19577fe640 10 client.17748 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.217+0000 7f19577fe640 1 -- 172.21.15.103:0/4204599476 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563aa3a3dd70 con 0x563aa3b5c5a0 2024-09-19T06:55:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.217+0000 7f19577fe640 10 client.17748 handle_command_reply: tid=0 2024-09-19T06:55:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.217+0000 7f1981830740 1 client.17748 shutdown 2024-09-19T06:55:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.217+0000 7f1981830740 1 -- 172.21.15.103:0/4204599476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 msgr2=0x563aa3adea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.217+0000 7f1981830740 1 --2- 172.21.15.103:0/4204599476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3adea30 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1970011730 tx=0x7f1970005ae0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.217+0000 7f1981830740 1 -- 172.21.15.103:0/4204599476 shutdown_connections 2024-09-19T06:55:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.217+0000 7f1981830740 1 --2- 172.21.15.103:0/4204599476 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563aa3b5c5a0 0x563aa3b5e9c0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.217+0000 7f1981830740 1 --2- 172.21.15.103:0/4204599476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa3adccb0 0x563aa3adea30 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.217+0000 7f19577fe640 0 client.17748 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.217+0000 7f1981830740 1 --2- 172.21.15.103:0/4204599476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563aa3ad2410 0x563aa3ade4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.217+0000 7f1981830740 1 --2- 172.21.15.103:0/4204599476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa3ad1af0 0x563aa3addfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.217+0000 7f1981830740 1 -- 172.21.15.103:0/4204599476 >> 172.21.15.103:0/4204599476 conn(0x563aa3ace060 msgr2=0x563aa3ad3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.217+0000 7f1981830740 1 -- 172.21.15.103:0/4204599476 shutdown_connections 2024-09-19T06:55:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.217+0000 7f1981830740 1 -- 172.21.15.103:0/4204599476 wait complete. 2024-09-19T06:55:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.218+0000 7f1981830740 20 client.17748 trim_cache size 0 max 16384 2024-09-19T06:55:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:55:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:55:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:55:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:55:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.218+0000 7f197e9be640 1 -- 172.21.15.103:0/1095618356 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19640a3460 msgr2=0x7f19640a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.218+0000 7f197e9be640 1 --2- 172.21.15.103:0/1095618356 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19640a3460 0x7f19640a5920 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1970009370 tx=0x7f1970009510 comp rx=0 tx=0).stop 2024-09-19T06:55:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.218+0000 7f197e9be640 1 -- 172.21.15.103:0/1095618356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19781009b0 msgr2=0x7f1978199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.218+0000 7f197e9be640 1 --2- 172.21.15.103:0/1095618356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19781009b0 0x7f1978199200 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f1960009230 tx=0x7f1960004270 comp rx=0 tx=0).stop 2024-09-19T06:55:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.218+0000 7f197e9be640 1 -- 172.21.15.103:0/1095618356 shutdown_connections 2024-09-19T06:55:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.218+0000 7f197e9be640 1 --2- 172.21.15.103:0/1095618356 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19640a3460 0x7f19640a5920 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.218+0000 7f197e9be640 1 --2- 172.21.15.103:0/1095618356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19781069c0 0x7f19781a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.218+0000 7f197e9be640 1 --2- 172.21.15.103:0/1095618356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1978101350 0x7f1978199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.218+0000 7f197e9be640 1 --2- 172.21.15.103:0/1095618356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19781009b0 0x7f1978199200 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.218+0000 7f197e9be640 1 -- 172.21.15.103:0/1095618356 >> 172.21.15.103:0/1095618356 conn(0x7f1978077990 msgr2=0x7f1978103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.218+0000 7f197e9be640 1 -- 172.21.15.103:0/1095618356 shutdown_connections 2024-09-19T06:55:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:33.218+0000 7f197e9be640 1 -- 172.21.15.103:0/1095618356 wait complete. 2024-09-19T06:55:33.304 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:33.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:33 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T06:55:33.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:33 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:33.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:33 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T06:55:33.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:33 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:33.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:33 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T06:55:33.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:33 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:33.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:33 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T06:55:33.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:33 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.728+0000 7fc3bcbed640 1 -- 172.21.15.103:0/1236381681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8106a80 msgr2=0x7fc3b8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.728+0000 7fc3bcbed640 1 --2- 172.21.15.103:0/1236381681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8106a80 0x7fc3b8106e60 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009930 tx=0x7fc3a802a420 comp rx=0 tx=0).stop 2024-09-19T06:55:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.728+0000 7fc3bcbed640 1 -- 172.21.15.103:0/1236381681 shutdown_connections 2024-09-19T06:55:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.728+0000 7fc3bcbed640 1 --2- 172.21.15.103:0/1236381681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3b8101420 0x7fc3b810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.728+0000 7fc3bcbed640 1 --2- 172.21.15.103:0/1236381681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3b8100a80 0x7fc3b8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.728+0000 7fc3bcbed640 1 --2- 172.21.15.103:0/1236381681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8106a80 0x7fc3b8106e60 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.728+0000 7fc3bcbed640 1 -- 172.21.15.103:0/1236381681 >> 172.21.15.103:0/1236381681 conn(0x7fc3b80fc7a0 msgr2=0x7fc3b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.728+0000 7fc3bcbed640 1 -- 172.21.15.103:0/1236381681 shutdown_connections 2024-09-19T06:55:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.728+0000 7fc3bcbed640 1 -- 172.21.15.103:0/1236381681 wait complete. 2024-09-19T06:55:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.729+0000 7fc3bcbed640 1 Processor -- start 2024-09-19T06:55:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.729+0000 7fc3bcbed640 1 -- start start 2024-09-19T06:55:33.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.729+0000 7fc3bcbed640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3b8100a80 0x7fc3b8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.729+0000 7fc3bcbed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8101420 0x7fc3b81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.729+0000 7fc3bcbed640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3b8106a80 0x7fc3b819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.729+0000 7fc3bcbed640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8111f60 con 0x7fc3b8101420 2024-09-19T06:55:33.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.729+0000 7fc3bcbed640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8111de0 con 0x7fc3b8100a80 2024-09-19T06:55:33.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.729+0000 7fc3bcbed640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81120e0 con 0x7fc3b8106a80 2024-09-19T06:55:33.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.729+0000 7fc3affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8101420 0x7fc3b81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.729+0000 7fc3affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8101420 0x7fc3b81989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35076/0 (socket says 172.21.15.103:35076) 2024-09-19T06:55:33.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.729+0000 7fc3affff640 1 -- 172.21.15.103:0/4116363549 learned_addr learned my addr 172.21.15.103:0/4116363549 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:33.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.730+0000 7fc3b659c640 1 --2- 172.21.15.103:0/4116363549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3b8100a80 0x7fc3b8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.730+0000 7fc3b6d9d640 1 --2- 172.21.15.103:0/4116363549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3b8106a80 0x7fc3b819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.730+0000 7fc3affff640 1 -- 172.21.15.103:0/4116363549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3b8106a80 msgr2=0x7fc3b819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.730+0000 7fc3affff640 1 --2- 172.21.15.103:0/4116363549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3b8106a80 0x7fc3b819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.730+0000 7fc3affff640 1 -- 172.21.15.103:0/4116363549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3b8100a80 msgr2=0x7fc3b8198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.730+0000 7fc3affff640 1 --2- 172.21.15.103:0/4116363549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3b8100a80 0x7fc3b8198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.730+0000 7fc3affff640 1 -- 172.21.15.103:0/4116363549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b819d410 con 0x7fc3b8101420 2024-09-19T06:55:33.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.730+0000 7fc3affff640 1 --2- 172.21.15.103:0/4116363549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8101420 0x7fc3b81989a0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000c890 tx=0x7fc3a000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:33.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.730+0000 7fc3af7fe640 1 -- 172.21.15.103:0/4116363549 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0004220 con 0x7fc3b8101420 2024-09-19T06:55:33.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.730+0000 7fc3af7fe640 1 -- 172.21.15.103:0/4116363549 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3a00043c0 con 0x7fc3b8101420 2024-09-19T06:55:33.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.730+0000 7fc3bcbed640 1 -- 172.21.15.103:0/4116363549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b819d700 con 0x7fc3b8101420 2024-09-19T06:55:33.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.730+0000 7fc3af7fe640 1 -- 172.21.15.103:0/4116363549 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a00049e0 con 0x7fc3b8101420 2024-09-19T06:55:33.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.730+0000 7fc3bcbed640 1 -- 172.21.15.103:0/4116363549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b81a4fe0 con 0x7fc3b8101420 2024-09-19T06:55:33.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.733+0000 7fc3af7fe640 1 -- 172.21.15.103:0/4116363549 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc3a00079b0 con 0x7fc3b8101420 2024-09-19T06:55:33.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.733+0000 7fc3befc6740 1 Processor -- start 2024-09-19T06:55:33.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3befc6740 1 -- start start 2024-09-19T06:55:33.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3af7fe640 1 --2- 172.21.15.103:0/4116363549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3a40a3430 0x7fc3a40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3befc6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1c4a0 0x55f683f1c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3befc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 0x55f683f27140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3befc6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f27680 0x55f683f29a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3b6d9d640 1 --2- 172.21.15.103:0/4116363549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3a40a3430 0x7fc3a40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3befc6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f683e822e0 con 0x55f683f1c4a0 2024-09-19T06:55:33.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3befc6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f683e29ce0 con 0x55f683f27680 2024-09-19T06:55:33.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3befc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f683e7e2a0 con 0x55f683f1cdc0 2024-09-19T06:55:33.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3af7fe640 1 -- 172.21.15.103:0/4116363549 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc3a00cfd70 con 0x7fc3b8101420 2024-09-19T06:55:33.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3b659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1c4a0 0x55f683f1c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3b659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1c4a0 0x55f683f1c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35082/0 (socket says 172.21.15.103:35082) 2024-09-19T06:55:33.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3b659c640 1 -- 172.21.15.103:0/4164321907 learned_addr learned my addr 172.21.15.103:0/4164321907 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:33.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3b6d9d640 1 --2- 172.21.15.103:0/4164321907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f27680 0x55f683f29a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3b659c640 1 -- 172.21.15.103:0/4164321907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 msgr2=0x55f683f27140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:33.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3affff640 1 --2- 172.21.15.103:0/4164321907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 0x55f683f27140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3b659c640 1 --2- 172.21.15.103:0/4164321907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 0x55f683f27140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3b659c640 1 -- 172.21.15.103:0/4164321907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f27680 msgr2=0x55f683f29a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:33.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3b659c640 1 --2- 172.21.15.103:0/4164321907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f27680 0x55f683f29a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3b659c640 1 -- 172.21.15.103:0/4164321907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f683e0e440 con 0x55f683f1c4a0 2024-09-19T06:55:33.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3b6d9d640 1 --2- 172.21.15.103:0/4164321907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f27680 0x55f683f29a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:33.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3affff640 1 --2- 172.21.15.103:0/4164321907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 0x55f683f27140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:33.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.734+0000 7fc3b6d9d640 1 --2- 172.21.15.103:0/4116363549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3a40a3430 0x7fc3a40a58f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0005fd0 tx=0x7fc3b0010740 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:33.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.735+0000 7fc3b659c640 1 --2- 172.21.15.103:0/4164321907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1c4a0 0x55f683f1c880 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009930 tx=0x7fc3a8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6977445884d6b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:33.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.735+0000 7fc3acff9640 1 -- 172.21.15.103:0/4164321907 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a803f070 con 0x55f683f1c4a0 2024-09-19T06:55:33.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.735+0000 7fc3acff9640 1 -- 172.21.15.103:0/4164321907 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3a80317b0 con 0x55f683f1c4a0 2024-09-19T06:55:33.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.735+0000 7fc3acff9640 1 -- 172.21.15.103:0/4164321907 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8035070 con 0x55f683f1c4a0 2024-09-19T06:55:33.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.735+0000 7fc3befc6740 1 -- 172.21.15.103:0/4164321907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1c4a0 msgr2=0x55f683f1c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.735+0000 7fc3befc6740 1 --2- 172.21.15.103:0/4164321907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1c4a0 0x55f683f1c880 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009930 tx=0x7fc3a8032ce0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.736+0000 7fc3befc6740 1 -- 172.21.15.103:0/4164321907 shutdown_connections 2024-09-19T06:55:33.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.736+0000 7fc3befc6740 1 --2- 172.21.15.103:0/4164321907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f27680 0x55f683f29a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.736+0000 7fc3befc6740 1 --2- 172.21.15.103:0/4164321907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 0x55f683f27140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.736+0000 7fc3befc6740 1 --2- 172.21.15.103:0/4164321907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1c4a0 0x55f683f1c880 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.736+0000 7fc3befc6740 1 -- 172.21.15.103:0/4164321907 >> 172.21.15.103:0/4164321907 conn(0x55f683f18a30 msgr2=0x55f683f18e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:33.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.736+0000 7fc3befc6740 1 -- 172.21.15.103:0/4164321907 shutdown_connections 2024-09-19T06:55:33.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.736+0000 7fc3befc6740 1 -- 172.21.15.103:0/4164321907 wait complete. 2024-09-19T06:55:33.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.736+0000 7fc3befc6740 1 -- start start 2024-09-19T06:55:33.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.736+0000 7fc3befc6740 1 Processor -- start 2024-09-19T06:55:33.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.736+0000 7fc3befc6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1c4a0 0x55f683f26f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.736+0000 7fc3befc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 0x55f683f24210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.736+0000 7fc3befc6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f27680 0x55f683f24750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.736+0000 7fc3befc6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f683f2a250 con 0x55f683f1c4a0 2024-09-19T06:55:33.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.736+0000 7fc3befc6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f683e3d050 con 0x55f683f27680 2024-09-19T06:55:33.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.736+0000 7fc3befc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f683f2a3d0 con 0x55f683f1cdc0 2024-09-19T06:55:33.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc3befc6740 10 client.0 _ll_register_callbacks cb 0x55f683e7d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:33.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc3b659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1c4a0 0x55f683f26f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc3b659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1c4a0 0x55f683f26f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35098/0 (socket says 172.21.15.103:35098) 2024-09-19T06:55:33.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc3b659c640 1 -- 172.21.15.103:0/419417159 learned_addr learned my addr 172.21.15.103:0/419417159 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:33.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc3affff640 1 --2- 172.21.15.103:0/419417159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 0x55f683f24210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc3b6d9d640 1 --2- 172.21.15.103:0/419417159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f27680 0x55f683f24750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc3b659c640 1 -- 172.21.15.103:0/419417159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 msgr2=0x55f683f24210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:33.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc3b659c640 1 --2- 172.21.15.103:0/419417159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 0x55f683f24210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc3b659c640 1 -- 172.21.15.103:0/419417159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f27680 msgr2=0x55f683f24750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:33.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc3b659c640 1 --2- 172.21.15.103:0/419417159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f27680 0x55f683f24750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc3b659c640 1 -- 172.21.15.103:0/419417159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f683bf33c0 con 0x55f683f1c4a0 2024-09-19T06:55:33.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc3affff640 1 --2- 172.21.15.103:0/419417159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 0x55f683f24210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:33.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc3b659c640 1 --2- 172.21.15.103:0/419417159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1c4a0 0x55f683f26f60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009900 tx=0x7fc3a8004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a483b9d7f62ff661 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:33.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc39dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:33.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc39dffb640 1 -- 172.21.15.103:0/419417159 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a803f030 con 0x55f683f1c4a0 2024-09-19T06:55:33.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc39dffb640 1 -- 172.21.15.103:0/419417159 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc3a80035f0 con 0x55f683f1c4a0 2024-09-19T06:55:33.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc3befc6740 1 -- 172.21.15.103:0/419417159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3a80317b0 con 0x55f683f1c4a0 2024-09-19T06:55:33.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc39dffb640 1 -- 172.21.15.103:0/419417159 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc3a8035040 con 0x55f683f1c4a0 2024-09-19T06:55:33.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc3befc6740 10 client.17760 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:33.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.737+0000 7fc3befc6740 1 -- 172.21.15.103:0/419417159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f683f35b30 con 0x55f683f1c4a0 2024-09-19T06:55:33.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.738+0000 7fc39dffb640 1 -- 172.21.15.103:0/419417159 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3a80317b0 con 0x55f683f1c4a0 2024-09-19T06:55:33.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.738+0000 7fc39dffb640 1 -- 172.21.15.103:0/419417159 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc3a8004ad0 con 0x55f683f1c4a0 2024-09-19T06:55:33.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.738+0000 7fc3befc6740 10 client.17760 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:33.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.738+0000 7fc3befc6740 10 client.17760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:33.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.738+0000 7fc3befc6740 20 client.17760 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:33.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.738+0000 7fc3befc6740 1 --2- 172.21.15.103:0/419417159 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f683f36e80 0x55f683f39270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.738+0000 7fc3befc6740 4 client.17760 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:33.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.738+0000 7fc3befc6740 1 -- 172.21.15.103:0/419417159 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f683e8f850 con 0x55f683f36e80 2024-09-19T06:55:33.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.738+0000 7fc3affff640 1 --2- 172.21.15.103:0/419417159 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f683f36e80 0x55f683f39270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.739+0000 7fc39dffb640 10 client.17760 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:33.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.739+0000 7fc3affff640 1 --2- 172.21.15.103:0/419417159 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f683f36e80 0x55f683f39270 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c07574cbbcbb3b3a server_cookie=2504b8799dc428e4 in_seq=0 out_seq=0 2024-09-19T06:55:33.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.739+0000 7fc39dffb640 1 -- 172.21.15.103:0/419417159 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f683e8f850 con 0x55f683f36e80 2024-09-19T06:55:33.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.739+0000 7fc39dffb640 10 client.17760 handle_command_reply: tid=0 2024-09-19T06:55:33.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.739+0000 7fc3befc6740 1 client.17760 shutdown 2024-09-19T06:55:33.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.739+0000 7fc3befc6740 1 -- 172.21.15.103:0/419417159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1c4a0 msgr2=0x55f683f26f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.739+0000 7fc3befc6740 1 --2- 172.21.15.103:0/419417159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1c4a0 0x55f683f26f60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009900 tx=0x7fc3a8004560 comp rx=0 tx=0).stop 2024-09-19T06:55:33.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.739+0000 7fc3befc6740 1 -- 172.21.15.103:0/419417159 shutdown_connections 2024-09-19T06:55:33.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.739+0000 7fc3befc6740 1 --2- 172.21.15.103:0/419417159 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f683f36e80 0x55f683f39270 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.739+0000 7fc3befc6740 1 --2- 172.21.15.103:0/419417159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f27680 0x55f683f24750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.739+0000 7fc3befc6740 1 --2- 172.21.15.103:0/419417159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 0x55f683f24210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.739+0000 7fc3befc6740 1 --2- 172.21.15.103:0/419417159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1c4a0 0x55f683f26f60 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.739+0000 7fc3befc6740 1 -- 172.21.15.103:0/419417159 >> 172.21.15.103:0/419417159 conn(0x55f683f18a30 msgr2=0x55f683f17b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:33.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.739+0000 7fc39dffb640 0 client.17760 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:33.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.740+0000 7fc3befc6740 1 -- 172.21.15.103:0/419417159 shutdown_connections 2024-09-19T06:55:33.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.740+0000 7fc3befc6740 1 -- 172.21.15.103:0/419417159 wait complete. 2024-09-19T06:55:33.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.740+0000 7fc3befc6740 20 client.17760 trim_cache size 0 max 16384 2024-09-19T06:55:33.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.756+0000 7fc3befc6740 1 Processor -- start 2024-09-19T06:55:33.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.756+0000 7fc3befc6740 1 -- start start 2024-09-19T06:55:33.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3befc6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f1c4a0 0x55f683f334b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3befc6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1cdc0 0x55f683f3e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3befc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f27680 0x55f683f3edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3befc6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81ad820 con 0x55f683f1cdc0 2024-09-19T06:55:33.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3befc6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81af360 con 0x55f683f1c4a0 2024-09-19T06:55:33.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3befc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81adef0 con 0x55f683f27680 2024-09-19T06:55:33.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1cdc0 0x55f683f3e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1cdc0 0x55f683f3e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35112/0 (socket says 172.21.15.103:35112) 2024-09-19T06:55:33.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3affff640 1 -- 172.21.15.103:0/1583664599 learned_addr learned my addr 172.21.15.103:0/1583664599 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:33.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3b659c640 1 --2- 172.21.15.103:0/1583664599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f1c4a0 0x55f683f334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3affff640 1 -- 172.21.15.103:0/1583664599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f27680 msgr2=0x55f683f3edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:33.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3affff640 1 --2- 172.21.15.103:0/1583664599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f27680 0x55f683f3edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3affff640 1 -- 172.21.15.103:0/1583664599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f1c4a0 msgr2=0x55f683f334b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:33.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3affff640 1 --2- 172.21.15.103:0/1583664599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f1c4a0 0x55f683f334b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3affff640 1 -- 172.21.15.103:0/1583664599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f683f1c220 con 0x55f683f1cdc0 2024-09-19T06:55:33.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3b659c640 1 --2- 172.21.15.103:0/1583664599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f1c4a0 0x55f683f334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:33.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc3affff640 1 --2- 172.21.15.103:0/1583664599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1cdc0 0x55f683f3e870 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc3a00cd4f0 tx=0x7fc3a00bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5643cfe5a2fde7ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:33.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc39ffff640 1 -- 172.21.15.103:0/1583664599 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a00b4070 con 0x55f683f1cdc0 2024-09-19T06:55:33.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc39ffff640 1 -- 172.21.15.103:0/1583664599 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3a00a4c90 con 0x55f683f1cdc0 2024-09-19T06:55:33.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.757+0000 7fc39ffff640 1 -- 172.21.15.103:0/1583664599 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a001d040 con 0x55f683f1cdc0 2024-09-19T06:55:33.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.758+0000 7fc3befc6740 1 -- 172.21.15.103:0/1583664599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1cdc0 msgr2=0x55f683f3e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.758+0000 7fc3befc6740 1 --2- 172.21.15.103:0/1583664599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1cdc0 0x55f683f3e870 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc3a00cd4f0 tx=0x7fc3a00bc040 comp rx=0 tx=0).stop 2024-09-19T06:55:33.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.758+0000 7fc3befc6740 1 -- 172.21.15.103:0/1583664599 shutdown_connections 2024-09-19T06:55:33.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.758+0000 7fc3befc6740 1 --2- 172.21.15.103:0/1583664599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f27680 0x55f683f3edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.758+0000 7fc3befc6740 1 --2- 172.21.15.103:0/1583664599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f1cdc0 0x55f683f3e870 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.758+0000 7fc3befc6740 1 --2- 172.21.15.103:0/1583664599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f1c4a0 0x55f683f334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.758+0000 7fc3befc6740 1 -- 172.21.15.103:0/1583664599 >> 172.21.15.103:0/1583664599 conn(0x55f683f18a30 msgr2=0x55f683f23300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:33.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.758+0000 7fc3befc6740 1 -- 172.21.15.103:0/1583664599 shutdown_connections 2024-09-19T06:55:33.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.758+0000 7fc3befc6740 1 -- 172.21.15.103:0/1583664599 wait complete. 2024-09-19T06:55:33.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.758+0000 7fc3befc6740 1 -- start start 2024-09-19T06:55:33.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.758+0000 7fc3befc6740 1 Processor -- start 2024-09-19T06:55:33.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3befc6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f1c4a0 0x55f683f99b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3befc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 0x55f683f9a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3befc6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f27680 0x55f683f2aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3befc6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81aec90 con 0x55f683f27680 2024-09-19T06:55:33.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3befc6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81ae5c0 con 0x55f683f1c4a0 2024-09-19T06:55:33.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3befc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81ab7e0 con 0x55f683f1cdc0 2024-09-19T06:55:33.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3befc6740 10 client.0 _ll_register_callbacks cb 0x7fc3a8032c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:33.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3b6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f27680 0x55f683f2aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3b6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f27680 0x55f683f2aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35118/0 (socket says 172.21.15.103:35118) 2024-09-19T06:55:33.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3b6d9d640 1 -- 172.21.15.103:0/43400379 learned_addr learned my addr 172.21.15.103:0/43400379 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:33.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3b659c640 1 --2- 172.21.15.103:0/43400379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f1c4a0 0x55f683f99b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3affff640 1 --2- 172.21.15.103:0/43400379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 0x55f683f9a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3b6d9d640 1 -- 172.21.15.103:0/43400379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 msgr2=0x55f683f9a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:33.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3b6d9d640 1 --2- 172.21.15.103:0/43400379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 0x55f683f9a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3b6d9d640 1 -- 172.21.15.103:0/43400379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f1c4a0 msgr2=0x55f683f99b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:33.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3b6d9d640 1 --2- 172.21.15.103:0/43400379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f1c4a0 0x55f683f99b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3b6d9d640 1 -- 172.21.15.103:0/43400379 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f683f2b290 con 0x55f683f27680 2024-09-19T06:55:33.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc3b6d9d640 1 --2- 172.21.15.103:0/43400379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f27680 0x55f683f2aba0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0013bd0 tx=0x7fc3b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9547b01955d8588a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:33.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc39dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:33.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.759+0000 7fc39dffb640 1 -- 172.21.15.103:0/43400379 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0011bf0 con 0x55f683f27680 2024-09-19T06:55:33.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.760+0000 7fc39dffb640 1 -- 172.21.15.103:0/43400379 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7fc3b0011d90 con 0x55f683f27680 2024-09-19T06:55:33.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.760+0000 7fc3befc6740 1 -- 172.21.15.103:0/43400379 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3a00a4c90 con 0x55f683f27680 2024-09-19T06:55:33.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.760+0000 7fc39dffb640 1 -- 172.21.15.103:0/43400379 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc3b0020440 con 0x55f683f27680 2024-09-19T06:55:33.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.760+0000 7fc3befc6740 10 client.17766 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:33.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.760+0000 7fc3befc6740 1 -- 172.21.15.103:0/43400379 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f683fa12e0 con 0x55f683f27680 2024-09-19T06:55:33.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.760+0000 7fc39dffb640 1 -- 172.21.15.103:0/43400379 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3a00a4c90 con 0x55f683f27680 2024-09-19T06:55:33.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.760+0000 7fc39dffb640 1 -- 172.21.15.103:0/43400379 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc3b000f040 con 0x55f683f27680 2024-09-19T06:55:33.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.760+0000 7fc3befc6740 10 client.17766 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:33.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.760+0000 7fc3befc6740 10 client.17766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:33.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.760+0000 7fc3befc6740 20 client.17766 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:33.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.760+0000 7fc3befc6740 1 --2- 172.21.15.103:0/43400379 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f683fa1a80 0x55f683fa3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:33.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.760+0000 7fc3befc6740 4 client.17766 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:55:33.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.760+0000 7fc3befc6740 1 -- 172.21.15.103:0/43400379 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f683e89d60 con 0x55f683fa1a80 2024-09-19T06:55:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.760+0000 7fc3b659c640 1 --2- 172.21.15.103:0/43400379 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f683fa1a80 0x55f683fa3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc3b659c640 1 --2- 172.21.15.103:0/43400379 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f683fa1a80 0x55f683fa3ea0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=425c2be4591ba692 server_cookie=787cd6ebb612a80e in_seq=0 out_seq=0 2024-09-19T06:55:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc39dffb640 10 client.17766 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc39dffb640 1 -- 172.21.15.103:0/43400379 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f683e89d60 con 0x55f683fa1a80 2024-09-19T06:55:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc39dffb640 10 client.17766 handle_command_reply: tid=0 2024-09-19T06:55:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc3befc6740 1 client.17766 shutdown 2024-09-19T06:55:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc3befc6740 1 -- 172.21.15.103:0/43400379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f27680 msgr2=0x55f683f2aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc3befc6740 1 --2- 172.21.15.103:0/43400379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f27680 0x55f683f2aba0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0013bd0 tx=0x7fc3b0004a40 comp rx=0 tx=0).stop 2024-09-19T06:55:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc3befc6740 1 -- 172.21.15.103:0/43400379 shutdown_connections 2024-09-19T06:55:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc3befc6740 1 --2- 172.21.15.103:0/43400379 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f683fa1a80 0x55f683fa3ea0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc3befc6740 1 --2- 172.21.15.103:0/43400379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f683f27680 0x55f683f2aba0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc39dffb640 0 client.17766 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc3befc6740 1 --2- 172.21.15.103:0/43400379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f683f1cdc0 0x55f683f9a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc3befc6740 1 --2- 172.21.15.103:0/43400379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f683f1c4a0 0x55f683f99b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc3befc6740 1 -- 172.21.15.103:0/43400379 >> 172.21.15.103:0/43400379 conn(0x55f683f18a30 msgr2=0x55f683f1ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:33.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc3befc6740 1 -- 172.21.15.103:0/43400379 shutdown_connections 2024-09-19T06:55:33.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc3befc6740 1 -- 172.21.15.103:0/43400379 wait complete. 2024-09-19T06:55:33.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.761+0000 7fc3befc6740 20 client.17766 trim_cache size 0 max 16384 2024-09-19T06:55:33.775 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:55:33.775 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:55:33.775 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T06:55:33.775 INFO:teuthology.orchestra.run.smithi103.stdout: "35d0501c-c126-49f4-b1c5-6e050f552e2d": { 2024-09-19T06:55:33.775 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T06:55:33.776 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "35d0501c-c126-49f4-b1c5-6e050f552e2d", 2024-09-19T06:55:33.776 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T06:55:33.776 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:55:33.776 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:55:33.776 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:55:33.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.762+0000 7fc3bcbed640 1 -- 172.21.15.103:0/4116363549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3a40a3430 msgr2=0x7fc3a40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.762+0000 7fc3bcbed640 1 --2- 172.21.15.103:0/4116363549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3a40a3430 0x7fc3a40a58f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0005fd0 tx=0x7fc3b0010740 comp rx=0 tx=0).stop 2024-09-19T06:55:33.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.762+0000 7fc3bcbed640 1 -- 172.21.15.103:0/4116363549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8101420 msgr2=0x7fc3b81989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:33.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.762+0000 7fc3bcbed640 1 --2- 172.21.15.103:0/4116363549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8101420 0x7fc3b81989a0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000c890 tx=0x7fc3a000cd50 comp rx=0 tx=0).stop 2024-09-19T06:55:33.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.762+0000 7fc3bcbed640 1 -- 172.21.15.103:0/4116363549 shutdown_connections 2024-09-19T06:55:33.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.762+0000 7fc3bcbed640 1 --2- 172.21.15.103:0/4116363549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3a40a3430 0x7fc3a40a58f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.762+0000 7fc3bcbed640 1 --2- 172.21.15.103:0/4116363549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3b8106a80 0x7fc3b819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.762+0000 7fc3bcbed640 1 --2- 172.21.15.103:0/4116363549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3b8101420 0x7fc3b81989a0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.762+0000 7fc3bcbed640 1 --2- 172.21.15.103:0/4116363549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3b8100a80 0x7fc3b8198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:33.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.762+0000 7fc3bcbed640 1 -- 172.21.15.103:0/4116363549 >> 172.21.15.103:0/4116363549 conn(0x7fc3b80fc7a0 msgr2=0x7fc3b80fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:33.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.762+0000 7fc3bcbed640 1 -- 172.21.15.103:0/4116363549 shutdown_connections 2024-09-19T06:55:33.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:55:33.762+0000 7fc3bcbed640 1 -- 172.21.15.103:0/4116363549 wait complete. 2024-09-19T06:55:34.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:34 smithi103 ceph-mon[31158]: pgmap v193: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:55:34.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:34 smithi134 ceph-mon[35356]: pgmap v193: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:55:34.556 INFO:tasks.cephfs.filesystem:scrub status for tag:35d0501c-c126-49f4-b1c5-6e050f552e2d - {'path': '/', 'tag': '35d0501c-c126-49f4-b1c5-6e050f552e2d', 'options': 'recursive,force'} 2024-09-19T06:55:34.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:34 smithi174 ceph-mon[35637]: pgmap v193: 97 pgs: 97 active+clean; 2.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:55:35.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:35 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T06:55:35.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:35 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T06:55:35.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:35 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T06:55:36.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:36 smithi103 ceph-mon[31158]: pgmap v194: 97 pgs: 97 active+clean; 2.8 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 63 KiB/s wr, 26 op/s 2024-09-19T06:55:36.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:36 smithi134 ceph-mon[35356]: pgmap v194: 97 pgs: 97 active+clean; 2.8 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 63 KiB/s wr, 26 op/s 2024-09-19T06:55:36.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:36 smithi174 ceph-mon[35637]: pgmap v194: 97 pgs: 97 active+clean; 2.8 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 63 KiB/s wr, 26 op/s 2024-09-19T06:55:38.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:38 smithi103 ceph-mon[31158]: pgmap v195: 97 pgs: 97 active+clean; 2.8 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-19T06:55:38.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:38 smithi134 ceph-mon[35356]: pgmap v195: 97 pgs: 97 active+clean; 2.8 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-19T06:55:38.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:38 smithi174 ceph-mon[35637]: pgmap v195: 97 pgs: 97 active+clean; 2.8 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-19T06:55:40.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:40 smithi103 ceph-mon[31158]: pgmap v196: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T06:55:40.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:40 smithi134 ceph-mon[35356]: pgmap v196: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T06:55:40.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:40 smithi174 ceph-mon[35637]: pgmap v196: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T06:55:42.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:42 smithi103 ceph-mon[31158]: pgmap v197: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-19T06:55:42.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:42 smithi134 ceph-mon[35356]: pgmap v197: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-19T06:55:42.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:42 smithi174 ceph-mon[35637]: pgmap v197: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-19T06:55:44.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:44 smithi103 ceph-mon[31158]: pgmap v198: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-19T06:55:44.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:44 smithi134 ceph-mon[35356]: pgmap v198: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-19T06:55:44.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:44 smithi174 ceph-mon[35637]: pgmap v198: 97 pgs: 97 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-19T06:55:45.957 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.957 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.957 INFO:tasks.workunit.client.0.smithi103.stdout:127/600: open fd = 3 2024-09-19T06:55:45.957 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.957 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.957 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.957 INFO:tasks.workunit.client.0.smithi103.stdout:128/600: open fd = 3 2024-09-19T06:55:45.957 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.958 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.958 INFO:tasks.workunit.client.0.smithi103.stdout:129/600: open fd = 3 2024-09-19T06:55:45.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.958 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.958 INFO:tasks.workunit.client.0.smithi103.stdout:130/600: open fd = 3 2024-09-19T06:55:45.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.958 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.958 INFO:tasks.workunit.client.0.smithi103.stdout:131/600: open fd = 3 2024-09-19T06:55:45.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.959 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.959 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.959 INFO:tasks.workunit.client.0.smithi103.stdout:132/600: open fd = 3 2024-09-19T06:55:45.959 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.959 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.959 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.959 INFO:tasks.workunit.client.0.smithi103.stdout:133/600: open fd = 3 2024-09-19T06:55:45.959 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.959 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.959 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.959 INFO:tasks.workunit.client.0.smithi103.stdout:134/600: open fd = 3 2024-09-19T06:55:45.959 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.959 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.959 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.959 INFO:tasks.workunit.client.0.smithi103.stdout:135/600: open fd = 3 2024-09-19T06:55:45.960 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.960 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.960 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.960 INFO:tasks.workunit.client.0.smithi103.stdout:136/600: open fd = 3 2024-09-19T06:55:45.960 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.960 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.960 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.960 INFO:tasks.workunit.client.0.smithi103.stdout:137/600: open fd = 3 2024-09-19T06:55:45.960 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.960 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.960 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.960 INFO:tasks.workunit.client.0.smithi103.stdout:138/600: open fd = 3 2024-09-19T06:55:45.960 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.961 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.961 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.961 INFO:tasks.workunit.client.0.smithi103.stdout:139/600: open fd = 3 2024-09-19T06:55:45.961 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.961 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.961 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.961 INFO:tasks.workunit.client.0.smithi103.stdout:140/600: open fd = 3 2024-09-19T06:55:45.961 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.961 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.961 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.961 INFO:tasks.workunit.client.0.smithi103.stdout:141/600: open fd = 3 2024-09-19T06:55:45.961 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.961 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.962 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.962 INFO:tasks.workunit.client.0.smithi103.stdout:142/600: open fd = 3 2024-09-19T06:55:45.962 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.962 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.962 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.962 INFO:tasks.workunit.client.0.smithi103.stdout:143/600: open fd = 3 2024-09-19T06:55:45.962 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.962 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.962 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.962 INFO:tasks.workunit.client.0.smithi103.stdout:144/600: open fd = 3 2024-09-19T06:55:45.962 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.962 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.962 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.962 INFO:tasks.workunit.client.0.smithi103.stdout:145/600: open fd = 3 2024-09-19T06:55:45.963 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.963 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.963 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.963 INFO:tasks.workunit.client.0.smithi103.stdout:146/600: open fd = 3 2024-09-19T06:55:45.963 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.963 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.963 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.963 INFO:tasks.workunit.client.0.smithi103.stdout:147/600: open fd = 3 2024-09-19T06:55:45.963 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.963 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.963 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.963 INFO:tasks.workunit.client.0.smithi103.stdout:148/600: open fd = 3 2024-09-19T06:55:45.963 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.964 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.964 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.964 INFO:tasks.workunit.client.0.smithi103.stdout:149/600: open fd = 3 2024-09-19T06:55:45.964 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.964 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.964 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.964 INFO:tasks.workunit.client.0.smithi103.stdout:150/600: open fd = 3 2024-09-19T06:55:45.964 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.964 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.964 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.964 INFO:tasks.workunit.client.0.smithi103.stdout:151/600: open fd = 3 2024-09-19T06:55:45.964 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.965 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.965 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.965 INFO:tasks.workunit.client.0.smithi103.stdout:152/600: open fd = 3 2024-09-19T06:55:45.965 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.965 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.965 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.965 INFO:tasks.workunit.client.0.smithi103.stdout:153/600: open fd = 3 2024-09-19T06:55:45.965 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.965 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.965 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.965 INFO:tasks.workunit.client.0.smithi103.stdout:154/600: open fd = 3 2024-09-19T06:55:45.965 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.965 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.966 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.966 INFO:tasks.workunit.client.0.smithi103.stdout:155/600: open fd = 3 2024-09-19T06:55:45.966 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.966 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.966 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.966 INFO:tasks.workunit.client.0.smithi103.stdout:156/600: open fd = 3 2024-09-19T06:55:45.966 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.966 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.966 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.966 INFO:tasks.workunit.client.0.smithi103.stdout:157/600: open fd = 3 2024-09-19T06:55:45.966 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.966 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.966 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.967 INFO:tasks.workunit.client.0.smithi103.stdout:158/600: open fd = 3 2024-09-19T06:55:45.967 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.967 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.967 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.967 INFO:tasks.workunit.client.0.smithi103.stdout:159/600: open fd = 3 2024-09-19T06:55:45.967 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.967 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.967 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.967 INFO:tasks.workunit.client.0.smithi103.stdout:160/600: open fd = 3 2024-09-19T06:55:45.967 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.967 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.967 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.967 INFO:tasks.workunit.client.0.smithi103.stdout:161/600: open fd = 3 2024-09-19T06:55:45.968 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.968 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.968 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.968 INFO:tasks.workunit.client.0.smithi103.stdout:162/600: open fd = 3 2024-09-19T06:55:45.968 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.968 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.968 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.968 INFO:tasks.workunit.client.0.smithi103.stdout:163/600: open fd = 3 2024-09-19T06:55:45.968 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.968 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.968 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.968 INFO:tasks.workunit.client.0.smithi103.stdout:164/600: open fd = 3 2024-09-19T06:55:45.968 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.969 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.969 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.969 INFO:tasks.workunit.client.0.smithi103.stdout:165/600: open fd = 3 2024-09-19T06:55:45.969 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.969 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.969 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.969 INFO:tasks.workunit.client.0.smithi103.stdout:166/600: open fd = 3 2024-09-19T06:55:45.969 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.969 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.969 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.969 INFO:tasks.workunit.client.0.smithi103.stdout:167/600: open fd = 3 2024-09-19T06:55:45.969 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.969 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.970 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.970 INFO:tasks.workunit.client.0.smithi103.stdout:168/600: open fd = 3 2024-09-19T06:55:45.970 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.970 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.970 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.970 INFO:tasks.workunit.client.0.smithi103.stdout:169/600: open fd = 3 2024-09-19T06:55:45.970 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.970 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.970 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.970 INFO:tasks.workunit.client.0.smithi103.stdout:170/600: open fd = 3 2024-09-19T06:55:45.970 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.970 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.970 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.970 INFO:tasks.workunit.client.0.smithi103.stdout:171/600: open fd = 3 2024-09-19T06:55:45.970 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.971 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.971 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.971 INFO:tasks.workunit.client.0.smithi103.stdout:172/600: open fd = 3 2024-09-19T06:55:45.971 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.971 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.971 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.971 INFO:tasks.workunit.client.0.smithi103.stdout:173/600: open fd = 3 2024-09-19T06:55:45.971 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.971 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.971 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.971 INFO:tasks.workunit.client.0.smithi103.stdout:174/600: open fd = 3 2024-09-19T06:55:45.971 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.971 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.971 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.972 INFO:tasks.workunit.client.0.smithi103.stdout:175/600: open fd = 3 2024-09-19T06:55:45.972 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.972 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.972 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.972 INFO:tasks.workunit.client.0.smithi103.stdout:176/600: open fd = 3 2024-09-19T06:55:45.972 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.972 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.972 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.972 INFO:tasks.workunit.client.0.smithi103.stdout:177/600: open fd = 3 2024-09-19T06:55:45.972 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.972 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.972 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.972 INFO:tasks.workunit.client.0.smithi103.stdout:178/600: open fd = 3 2024-09-19T06:55:45.972 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.973 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.973 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.973 INFO:tasks.workunit.client.0.smithi103.stdout:179/600: open fd = 3 2024-09-19T06:55:45.973 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.973 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.973 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.973 INFO:tasks.workunit.client.0.smithi103.stdout:180/600: open fd = 3 2024-09-19T06:55:45.973 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.973 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.973 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.973 INFO:tasks.workunit.client.0.smithi103.stdout:181/600: open fd = 3 2024-09-19T06:55:45.973 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.973 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.973 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.974 INFO:tasks.workunit.client.0.smithi103.stdout:182/600: open fd = 3 2024-09-19T06:55:45.974 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.974 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.974 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.974 INFO:tasks.workunit.client.0.smithi103.stdout:183/600: open fd = 3 2024-09-19T06:55:45.974 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.974 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.974 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.974 INFO:tasks.workunit.client.0.smithi103.stdout:184/600: open fd = 3 2024-09-19T06:55:45.974 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.974 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.974 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.974 INFO:tasks.workunit.client.0.smithi103.stdout:185/600: open fd = 3 2024-09-19T06:55:45.974 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.975 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.975 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.975 INFO:tasks.workunit.client.0.smithi103.stdout:186/600: open fd = 3 2024-09-19T06:55:45.975 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.975 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.975 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.975 INFO:tasks.workunit.client.0.smithi103.stdout:187/600: open fd = 3 2024-09-19T06:55:45.975 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.975 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.975 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:55:45.975 INFO:tasks.workunit.client.0.smithi103.stdout:188/600: open fd = 3 2024-09-19T06:55:45.975 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:45.975 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:55:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.315+0000 7f4470503640 1 -- 172.21.15.103:0/2778214646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4468101350 msgr2=0x7f446810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.315+0000 7f4470503640 1 --2- 172.21.15.103:0/2778214646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4468101350 0x7f446810d880 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f4460009930 tx=0x7f446002a420 comp rx=0 tx=0).stop 2024-09-19T06:55:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.315+0000 7f4470503640 1 -- 172.21.15.103:0/2778214646 shutdown_connections 2024-09-19T06:55:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.315+0000 7f4470503640 1 --2- 172.21.15.103:0/2778214646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4468101350 0x7f446810d880 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.315+0000 7f4470503640 1 --2- 172.21.15.103:0/2778214646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44681009b0 0x7f4468100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.315+0000 7f4470503640 1 --2- 172.21.15.103:0/2778214646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44681069c0 0x7f4468106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.315+0000 7f4470503640 1 -- 172.21.15.103:0/2778214646 >> 172.21.15.103:0/2778214646 conn(0x7f4468077990 msgr2=0x7f44680fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.315+0000 7f4470503640 1 -- 172.21.15.103:0/2778214646 shutdown_connections 2024-09-19T06:55:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.315+0000 7f4470503640 1 -- 172.21.15.103:0/2778214646 wait complete. 2024-09-19T06:55:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.316+0000 7f4470503640 1 Processor -- start 2024-09-19T06:55:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.316+0000 7f4470503640 1 -- start start 2024-09-19T06:55:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.316+0000 7f4470503640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44681009b0 0x7f4468199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.316+0000 7f4470503640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4468101350 0x7f44681996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.316+0000 7f4470503640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44681069c0 0x7f44681a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.316+0000 7f4470503640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4468110e00 con 0x7f4468101350 2024-09-19T06:55:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.316+0000 7f4470503640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4468110c80 con 0x7f44681069c0 2024-09-19T06:55:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.316+0000 7f4470503640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4468110f80 con 0x7f44681009b0 2024-09-19T06:55:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.316+0000 7f446da9e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4468101350 0x7f44681996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.316+0000 7f446da9e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4468101350 0x7f44681996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52406/0 (socket says 172.21.15.103:52406) 2024-09-19T06:55:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.316+0000 7f446da9e640 1 -- 172.21.15.103:0/3592627630 learned_addr learned my addr 172.21.15.103:0/3592627630 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.316+0000 7f446eaa0640 1 --2- 172.21.15.103:0/3592627630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44681069c0 0x7f44681a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.316+0000 7f446da9e640 1 -- 172.21.15.103:0/3592627630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44681009b0 msgr2=0x7f4468199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.317+0000 7f446da9e640 1 --2- 172.21.15.103:0/3592627630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44681009b0 0x7f4468199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.317+0000 7f446da9e640 1 -- 172.21.15.103:0/3592627630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44681069c0 msgr2=0x7f44681a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.317+0000 7f446da9e640 1 --2- 172.21.15.103:0/3592627630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44681069c0 0x7f44681a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.317+0000 7f446da9e640 1 -- 172.21.15.103:0/3592627630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44681a4730 con 0x7f4468101350 2024-09-19T06:55:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.317+0000 7f446eaa0640 1 --2- 172.21.15.103:0/3592627630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44681069c0 0x7f44681a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.317+0000 7f446da9e640 1 --2- 172.21.15.103:0/3592627630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4468101350 0x7f44681996b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f445800d770 tx=0x7f445800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.317+0000 7f445f7fe640 1 -- 172.21.15.103:0/3592627630 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458017070 con 0x7f4468101350 2024-09-19T06:55:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.317+0000 7f4470503640 1 -- 172.21.15.103:0/3592627630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44681a4900 con 0x7f4468101350 2024-09-19T06:55:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.317+0000 7f445f7fe640 1 -- 172.21.15.103:0/3592627630 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4458005500 con 0x7f4468101350 2024-09-19T06:55:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.317+0000 7f445f7fe640 1 -- 172.21.15.103:0/3592627630 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458004e00 con 0x7f4468101350 2024-09-19T06:55:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.317+0000 7f4470503640 1 -- 172.21.15.103:0/3592627630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44681a4dd0 con 0x7f4468101350 2024-09-19T06:55:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.319+0000 7f445f7fe640 1 -- 172.21.15.103:0/3592627630 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4458004fa0 con 0x7f4468101350 2024-09-19T06:55:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.320+0000 7f447329b740 1 Processor -- start 2024-09-19T06:55:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.320+0000 7f447329b740 1 -- start start 2024-09-19T06:55:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.320+0000 7f447329b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad8f00 0x56174aad92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.320+0000 7f447329b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aad9820 0x56174aae3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.320+0000 7f447329b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aae40c0 0x56174aae64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.320+0000 7f447329b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56174aa392f0 con 0x56174aad8f00 2024-09-19T06:55:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.320+0000 7f447329b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56174a9e1270 con 0x56174aad9820 2024-09-19T06:55:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.320+0000 7f447329b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56174aa352b0 con 0x56174aae40c0 2024-09-19T06:55:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f446eaa0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad8f00 0x56174aad92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f446eaa0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad8f00 0x56174aad92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52412/0 (socket says 172.21.15.103:52412) 2024-09-19T06:55:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f446eaa0640 1 -- 172.21.15.103:0/720600553 learned_addr learned my addr 172.21.15.103:0/720600553 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f446e29f640 1 --2- 172.21.15.103:0/720600553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aad9820 0x56174aae3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f445f7fe640 1 --2- 172.21.15.103:0/3592627630 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f44540a3460 0x7f44540a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f446da9e640 1 --2- 172.21.15.103:0/720600553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aae40c0 0x56174aae64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f446eaa0640 1 --2- 172.21.15.103:0/3592627630 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f44540a3460 0x7f44540a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f445f7fe640 1 -- 172.21.15.103:0/3592627630 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f44580cf840 con 0x7f4468101350 2024-09-19T06:55:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f446e29f640 1 -- 172.21.15.103:0/720600553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aae40c0 msgr2=0x56174aae64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f446e29f640 1 --2- 172.21.15.103:0/720600553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aae40c0 0x56174aae64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f446e29f640 1 -- 172.21.15.103:0/720600553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad8f00 msgr2=0x56174aad92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f446e29f640 1 --2- 172.21.15.103:0/720600553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad8f00 0x56174aad92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f446e29f640 1 -- 172.21.15.103:0/720600553 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56174a9e0340 con 0x56174aad9820 2024-09-19T06:55:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f446eaa0640 1 --2- 172.21.15.103:0/720600553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad8f00 0x56174aad92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f446da9e640 1 --2- 172.21.15.103:0/720600553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aae40c0 0x56174aae64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f446e29f640 1 --2- 172.21.15.103:0/720600553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aad9820 0x56174aae3b80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4464009cc0 tx=0x7f446400e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a1b2623f05dced6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f446eaa0640 1 --2- 172.21.15.103:0/3592627630 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f44540a3460 0x7f44540a5920 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4460031550 tx=0x7f446002a420 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.321+0000 7f445cbfe640 1 -- 172.21.15.103:0/720600553 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464016070 con 0x56174aad9820 2024-09-19T06:55:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.322+0000 7f445cbfe640 1 -- 172.21.15.103:0/720600553 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4464005020 con 0x56174aad9820 2024-09-19T06:55:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.322+0000 7f445cbfe640 1 -- 172.21.15.103:0/720600553 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44640116d0 con 0x56174aad9820 2024-09-19T06:55:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.322+0000 7f447329b740 1 -- 172.21.15.103:0/720600553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aad9820 msgr2=0x56174aae3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.322+0000 7f447329b740 1 --2- 172.21.15.103:0/720600553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aad9820 0x56174aae3b80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4464009cc0 tx=0x7f446400e6c0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.322+0000 7f447329b740 1 -- 172.21.15.103:0/720600553 shutdown_connections 2024-09-19T06:55:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.322+0000 7f447329b740 1 --2- 172.21.15.103:0/720600553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aae40c0 0x56174aae64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.322+0000 7f447329b740 1 --2- 172.21.15.103:0/720600553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aad9820 0x56174aae3b80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.322+0000 7f447329b740 1 --2- 172.21.15.103:0/720600553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad8f00 0x56174aad92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.322+0000 7f447329b740 1 -- 172.21.15.103:0/720600553 >> 172.21.15.103:0/720600553 conn(0x56174aad5470 msgr2=0x56174aad5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.322+0000 7f447329b740 1 -- 172.21.15.103:0/720600553 shutdown_connections 2024-09-19T06:55:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.322+0000 7f447329b740 1 -- 172.21.15.103:0/720600553 wait complete. 2024-09-19T06:55:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f447329b740 1 -- start start 2024-09-19T06:55:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f447329b740 1 Processor -- start 2024-09-19T06:55:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f447329b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aad8f00 0x56174aae0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f447329b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aad9820 0x56174aae1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f447329b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aae40c0 0x56174aae19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f447329b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56174aae6c90 con 0x56174aae40c0 2024-09-19T06:55:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f447329b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56174a9c2290 con 0x56174aad8f00 2024-09-19T06:55:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f447329b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56174aae6e10 con 0x56174aad9820 2024-09-19T06:55:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f447329b740 10 client.0 _ll_register_callbacks cb 0x56174a9f3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f446eaa0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aae40c0 0x56174aae19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f446eaa0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aae40c0 0x56174aae19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52418/0 (socket says 172.21.15.103:52418) 2024-09-19T06:55:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f446e29f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aad8f00 0x56174aae0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f446eaa0640 1 -- 172.21.15.103:0/575265333 learned_addr learned my addr 172.21.15.103:0/575265333 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f446da9e640 1 --2- 172.21.15.103:0/575265333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aad9820 0x56174aae1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f446eaa0640 1 -- 172.21.15.103:0/575265333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aad9820 msgr2=0x56174aae1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f446eaa0640 1 --2- 172.21.15.103:0/575265333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aad9820 0x56174aae1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f446eaa0640 1 -- 172.21.15.103:0/575265333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aad8f00 msgr2=0x56174aae0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f446eaa0640 1 --2- 172.21.15.103:0/575265333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aad8f00 0x56174aae0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f446eaa0640 1 -- 172.21.15.103:0/575265333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56174a74c210 con 0x56174aae40c0 2024-09-19T06:55:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f446da9e640 1 --2- 172.21.15.103:0/575265333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aad9820 0x56174aae1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.323+0000 7f446e29f640 1 --2- 172.21.15.103:0/575265333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aad8f00 0x56174aae0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.324+0000 7f446eaa0640 1 --2- 172.21.15.103:0/575265333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aae40c0 0x56174aae19b0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f4460005ab0 tx=0x7f4460009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4d9c76cf80d12b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.324+0000 7f4451ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.324+0000 7f4451ffb640 1 -- 172.21.15.103:0/575265333 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446004e020 con 0x56174aae40c0 2024-09-19T06:55:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.324+0000 7f447329b740 1 -- 172.21.15.103:0/575265333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4464005020 con 0x56174aae40c0 2024-09-19T06:55:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.324+0000 7f4451ffb640 1 -- 172.21.15.103:0/575265333 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f44600046c0 con 0x56174aae40c0 2024-09-19T06:55:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.324+0000 7f4451ffb640 1 -- 172.21.15.103:0/575265333 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4460059050 con 0x56174aae40c0 2024-09-19T06:55:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.324+0000 7f447329b740 10 client.17778 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.324+0000 7f447329b740 1 -- 172.21.15.103:0/575265333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56174a818620 con 0x56174aae40c0 2024-09-19T06:55:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.324+0000 7f4451ffb640 1 -- 172.21.15.103:0/575265333 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4464005020 con 0x56174aae40c0 2024-09-19T06:55:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.324+0000 7f4451ffb640 1 -- 172.21.15.103:0/575265333 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4460003ae0 con 0x56174aae40c0 2024-09-19T06:55:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.324+0000 7f447329b740 10 client.17778 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.324+0000 7f447329b740 10 client.17778 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.324+0000 7f447329b740 20 client.17778 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.324+0000 7f447329b740 1 --2- 172.21.15.103:0/575265333 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56174aaf3fe0 0x56174aaf6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.324+0000 7f447329b740 4 client.17778 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.324+0000 7f447329b740 1 -- 172.21.15.103:0/575265333 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56174aa46a20 con 0x56174aaf3fe0 2024-09-19T06:55:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.325+0000 7f446e29f640 1 --2- 172.21.15.103:0/575265333 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56174aaf3fe0 0x56174aaf6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.325+0000 7f4451ffb640 10 client.17778 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.325+0000 7f446e29f640 1 --2- 172.21.15.103:0/575265333 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56174aaf3fe0 0x56174aaf6410 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea694ecfa5fb7f8 server_cookie=71c27489e3da0ad3 in_seq=0 out_seq=0 2024-09-19T06:55:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.326+0000 7f4451ffb640 1 -- 172.21.15.103:0/575265333 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56174aa46a20 con 0x56174aaf3fe0 2024-09-19T06:55:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.326+0000 7f4451ffb640 10 client.17778 handle_command_reply: tid=0 2024-09-19T06:55:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.326+0000 7f447329b740 1 client.17778 shutdown 2024-09-19T06:55:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.326+0000 7f447329b740 1 -- 172.21.15.103:0/575265333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aae40c0 msgr2=0x56174aae19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.326+0000 7f447329b740 1 --2- 172.21.15.103:0/575265333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aae40c0 0x56174aae19b0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f4460005ab0 tx=0x7f4460009260 comp rx=0 tx=0).stop 2024-09-19T06:55:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.326+0000 7f447329b740 1 -- 172.21.15.103:0/575265333 shutdown_connections 2024-09-19T06:55:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.326+0000 7f447329b740 1 --2- 172.21.15.103:0/575265333 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56174aaf3fe0 0x56174aaf6410 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.326+0000 7f447329b740 1 --2- 172.21.15.103:0/575265333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aae40c0 0x56174aae19b0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.326+0000 7f447329b740 1 --2- 172.21.15.103:0/575265333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aad9820 0x56174aae1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.326+0000 7f4451ffb640 0 client.17778 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.326+0000 7f447329b740 1 --2- 172.21.15.103:0/575265333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aad8f00 0x56174aae0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.326+0000 7f447329b740 1 -- 172.21.15.103:0/575265333 >> 172.21.15.103:0/575265333 conn(0x56174aad5470 msgr2=0x56174aad01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.326+0000 7f447329b740 1 -- 172.21.15.103:0/575265333 shutdown_connections 2024-09-19T06:55:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.326+0000 7f447329b740 1 -- 172.21.15.103:0/575265333 wait complete. 2024-09-19T06:55:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.326+0000 7f447329b740 20 client.17778 trim_cache size 0 max 16384 2024-09-19T06:55:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f447329b740 1 Processor -- start 2024-09-19T06:55:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f447329b740 1 -- start start 2024-09-19T06:55:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f447329b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aad8f00 0x56174aad00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f447329b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad9820 0x56174aafbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f447329b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aae40c0 0x56174aafc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f447329b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44681ab590 con 0x56174aad9820 2024-09-19T06:55:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f447329b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f44681ad0d0 con 0x56174aae40c0 2024-09-19T06:55:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f447329b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44681abc60 con 0x56174aad8f00 2024-09-19T06:55:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f446da9e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad9820 0x56174aafbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f446da9e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad9820 0x56174aafbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52434/0 (socket says 172.21.15.103:52434) 2024-09-19T06:55:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f446da9e640 1 -- 172.21.15.103:0/3669264335 learned_addr learned my addr 172.21.15.103:0/3669264335 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f446e29f640 1 --2- 172.21.15.103:0/3669264335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aad8f00 0x56174aad00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f446da9e640 1 -- 172.21.15.103:0/3669264335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aad8f00 msgr2=0x56174aad00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f446eaa0640 1 --2- 172.21.15.103:0/3669264335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aae40c0 0x56174aafc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f446da9e640 1 --2- 172.21.15.103:0/3669264335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aad8f00 0x56174aad00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f446da9e640 1 -- 172.21.15.103:0/3669264335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aae40c0 msgr2=0x56174aafc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f446da9e640 1 --2- 172.21.15.103:0/3669264335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aae40c0 0x56174aafc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f446da9e640 1 -- 172.21.15.103:0/3669264335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56174aad52a0 con 0x56174aad9820 2024-09-19T06:55:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f446e29f640 1 --2- 172.21.15.103:0/3669264335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aad8f00 0x56174aad00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f446eaa0640 1 --2- 172.21.15.103:0/3669264335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aae40c0 0x56174aafc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.341+0000 7f446da9e640 1 --2- 172.21.15.103:0/3669264335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad9820 0x56174aafbd70 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f4458097f90 tx=0x7f4458086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b96b6dab9246143 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.342+0000 7f4453fff640 1 -- 172.21.15.103:0/3669264335 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458020070 con 0x56174aad9820 2024-09-19T06:55:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.342+0000 7f4453fff640 1 -- 172.21.15.103:0/3669264335 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f445809bb30 con 0x56174aad9820 2024-09-19T06:55:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.342+0000 7f4453fff640 1 -- 172.21.15.103:0/3669264335 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44580cd220 con 0x56174aad9820 2024-09-19T06:55:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.342+0000 7f447329b740 1 -- 172.21.15.103:0/3669264335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad9820 msgr2=0x56174aafbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.342+0000 7f447329b740 1 --2- 172.21.15.103:0/3669264335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad9820 0x56174aafbd70 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f4458097f90 tx=0x7f4458086040 comp rx=0 tx=0).stop 2024-09-19T06:55:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.342+0000 7f447329b740 1 -- 172.21.15.103:0/3669264335 shutdown_connections 2024-09-19T06:55:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.342+0000 7f447329b740 1 --2- 172.21.15.103:0/3669264335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aae40c0 0x56174aafc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.342+0000 7f447329b740 1 --2- 172.21.15.103:0/3669264335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad9820 0x56174aafbd70 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.342+0000 7f447329b740 1 --2- 172.21.15.103:0/3669264335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aad8f00 0x56174aad00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.342+0000 7f447329b740 1 -- 172.21.15.103:0/3669264335 >> 172.21.15.103:0/3669264335 conn(0x56174aad5470 msgr2=0x56174aacf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.342+0000 7f447329b740 1 -- 172.21.15.103:0/3669264335 shutdown_connections 2024-09-19T06:55:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.342+0000 7f447329b740 1 -- 172.21.15.103:0/3669264335 wait complete. 2024-09-19T06:55:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f447329b740 1 -- start start 2024-09-19T06:55:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f447329b740 1 Processor -- start 2024-09-19T06:55:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f447329b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad8f00 0x56174aaec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f447329b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aad9820 0x56174aaec890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f447329b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aae40c0 0x56174aadd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f447329b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44681aca00 con 0x56174aad8f00 2024-09-19T06:55:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f447329b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f44681ac330 con 0x56174aae40c0 2024-09-19T06:55:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f447329b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44681a9550 con 0x56174aad9820 2024-09-19T06:55:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f447329b740 10 client.0 _ll_register_callbacks cb 0x7f4460009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f446e29f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad8f00 0x56174aaec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f446e29f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad8f00 0x56174aaec350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52440/0 (socket says 172.21.15.103:52440) 2024-09-19T06:55:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f446e29f640 1 -- 172.21.15.103:0/217844301 learned_addr learned my addr 172.21.15.103:0/217844301 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f446e29f640 1 -- 172.21.15.103:0/217844301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aad9820 msgr2=0x56174aaec890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f446eaa0640 1 --2- 172.21.15.103:0/217844301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aae40c0 0x56174aadd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f446e29f640 1 --2- 172.21.15.103:0/217844301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aad9820 0x56174aaec890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f446e29f640 1 -- 172.21.15.103:0/217844301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aae40c0 msgr2=0x56174aadd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f446e29f640 1 --2- 172.21.15.103:0/217844301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aae40c0 0x56174aadd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f446e29f640 1 -- 172.21.15.103:0/217844301 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56174aad6100 con 0x56174aad8f00 2024-09-19T06:55:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f446eaa0640 1 --2- 172.21.15.103:0/217844301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aae40c0 0x56174aadd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f446e29f640 1 --2- 172.21.15.103:0/217844301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad8f00 0x56174aaec350 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f4464009dd0 tx=0x7f4464004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b947fc5264eff297 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.343+0000 7f4451ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.344+0000 7f4451ffb640 1 -- 172.21.15.103:0/217844301 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464016040 con 0x56174aad8f00 2024-09-19T06:55:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.344+0000 7f4451ffb640 1 -- 172.21.15.103:0/217844301 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f4464005030 con 0x56174aad8f00 2024-09-19T06:55:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.344+0000 7f447329b740 1 -- 172.21.15.103:0/217844301 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f445809bb30 con 0x56174aad8f00 2024-09-19T06:55:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.344+0000 7f4451ffb640 1 -- 172.21.15.103:0/217844301 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4464009e80 con 0x56174aad8f00 2024-09-19T06:55:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.344+0000 7f447329b740 10 client.17784 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.344+0000 7f447329b740 1 -- 172.21.15.103:0/217844301 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56174aaeba10 con 0x56174aad8f00 2024-09-19T06:55:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.344+0000 7f4451ffb640 1 -- 172.21.15.103:0/217844301 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f445809bb30 con 0x56174aad8f00 2024-09-19T06:55:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.344+0000 7f4451ffb640 1 -- 172.21.15.103:0/217844301 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f446400f040 con 0x56174aad8f00 2024-09-19T06:55:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.344+0000 7f447329b740 10 client.17784 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.344+0000 7f447329b740 10 client.17784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.344+0000 7f447329b740 20 client.17784 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.344+0000 7f447329b740 1 --2- 172.21.15.103:0/217844301 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56174ab5ed40 0x56174ab61130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.344+0000 7f447329b740 4 client.17784 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:55:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.344+0000 7f447329b740 1 -- 172.21.15.103:0/217844301 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56174aa40d70 con 0x56174ab5ed40 2024-09-19T06:55:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.344+0000 7f446da9e640 1 --2- 172.21.15.103:0/217844301 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56174ab5ed40 0x56174ab61130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.345+0000 7f446da9e640 1 --2- 172.21.15.103:0/217844301 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56174ab5ed40 0x56174ab61130 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dee3e9368e3c0405 server_cookie=ea87c61c45a35acc in_seq=0 out_seq=0 2024-09-19T06:55:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.345+0000 7f4451ffb640 10 client.17784 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.358+0000 7f4451ffb640 1 -- 172.21.15.103:0/217844301 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56174aa40d70 con 0x56174ab5ed40 2024-09-19T06:55:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.358+0000 7f4451ffb640 10 client.17784 handle_command_reply: tid=0 2024-09-19T06:55:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.358+0000 7f447329b740 1 client.17784 shutdown 2024-09-19T06:55:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.358+0000 7f447329b740 1 -- 172.21.15.103:0/217844301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad8f00 msgr2=0x56174aaec350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.358+0000 7f447329b740 1 --2- 172.21.15.103:0/217844301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad8f00 0x56174aaec350 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f4464009dd0 tx=0x7f4464004980 comp rx=0 tx=0).stop 2024-09-19T06:55:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.358+0000 7f447329b740 1 -- 172.21.15.103:0/217844301 shutdown_connections 2024-09-19T06:55:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.358+0000 7f447329b740 1 --2- 172.21.15.103:0/217844301 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56174ab5ed40 0x56174ab61130 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.358+0000 7f447329b740 1 --2- 172.21.15.103:0/217844301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56174aae40c0 0x56174aadd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.358+0000 7f447329b740 1 --2- 172.21.15.103:0/217844301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56174aad9820 0x56174aaec890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.358+0000 7f447329b740 1 --2- 172.21.15.103:0/217844301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174aad8f00 0x56174aaec350 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.358+0000 7f4451ffb640 0 client.17784 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.358+0000 7f447329b740 1 -- 172.21.15.103:0/217844301 >> 172.21.15.103:0/217844301 conn(0x56174aad5470 msgr2=0x56174aae0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.359+0000 7f447329b740 1 -- 172.21.15.103:0/217844301 shutdown_connections 2024-09-19T06:55:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.359+0000 7f447329b740 1 -- 172.21.15.103:0/217844301 wait complete. 2024-09-19T06:55:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.359+0000 7f447329b740 20 client.17784 trim_cache size 0 max 16384 2024-09-19T06:55:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:55:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:55:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:55:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:55:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.359+0000 7f4470503640 1 -- 172.21.15.103:0/3592627630 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f44540a3460 msgr2=0x7f44540a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.359+0000 7f4470503640 1 --2- 172.21.15.103:0/3592627630 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f44540a3460 0x7f44540a5920 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4460031550 tx=0x7f446002a420 comp rx=0 tx=0).stop 2024-09-19T06:55:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.359+0000 7f4470503640 1 -- 172.21.15.103:0/3592627630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4468101350 msgr2=0x7f44681996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.359+0000 7f4470503640 1 --2- 172.21.15.103:0/3592627630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4468101350 0x7f44681996b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f445800d770 tx=0x7f445800dc30 comp rx=0 tx=0).stop 2024-09-19T06:55:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.359+0000 7f4470503640 1 -- 172.21.15.103:0/3592627630 shutdown_connections 2024-09-19T06:55:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.359+0000 7f4470503640 1 --2- 172.21.15.103:0/3592627630 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f44540a3460 0x7f44540a5920 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.359+0000 7f4470503640 1 --2- 172.21.15.103:0/3592627630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44681069c0 0x7f44681a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.359+0000 7f4470503640 1 --2- 172.21.15.103:0/3592627630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4468101350 0x7f44681996b0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.359+0000 7f4470503640 1 --2- 172.21.15.103:0/3592627630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44681009b0 0x7f4468199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.359+0000 7f4470503640 1 -- 172.21.15.103:0/3592627630 >> 172.21.15.103:0/3592627630 conn(0x7f4468077990 msgr2=0x7f4468103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.359+0000 7f4470503640 1 -- 172.21.15.103:0/3592627630 shutdown_connections 2024-09-19T06:55:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:46.359+0000 7f4470503640 1 -- 172.21.15.103:0/3592627630 wait complete. 2024-09-19T06:55:46.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:46 smithi103 ceph-mon[31158]: pgmap v199: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 23 op/s 2024-09-19T06:55:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:46 smithi134 ceph-mon[35356]: pgmap v199: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 23 op/s 2024-09-19T06:55:46.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:46 smithi174 ceph-mon[35637]: pgmap v199: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 23 op/s 2024-09-19T06:55:48.385 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:48 smithi174 ceph-mon[35637]: pgmap v200: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-19T06:55:48.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:48 smithi103 ceph-mon[31158]: pgmap v200: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-19T06:55:48.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:48 smithi134 ceph-mon[35356]: pgmap v200: 97 pgs: 97 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-19T06:55:50.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:50 smithi103 ceph-mon[31158]: pgmap v201: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-09-19T06:55:50.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:50 smithi134 ceph-mon[35356]: pgmap v201: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-09-19T06:55:50.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:50 smithi174 ceph-mon[35637]: pgmap v201: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-09-19T06:55:52.836 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:52 smithi174 ceph-mon[35637]: pgmap v202: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-19T06:55:52.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:52 smithi103 ceph-mon[31158]: pgmap v202: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-19T06:55:52.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:52 smithi134 ceph-mon[35356]: pgmap v202: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-19T06:55:54.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:54 smithi103 ceph-mon[31158]: pgmap v203: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-19T06:55:54.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:54 smithi134 ceph-mon[35356]: pgmap v203: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-19T06:55:54.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:54 smithi174 ceph-mon[35637]: pgmap v203: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-19T06:55:56.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:56 smithi103 ceph-mon[31158]: pgmap v204: 97 pgs: 97 active+clean; 3.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-09-19T06:55:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:56 smithi134 ceph-mon[35356]: pgmap v204: 97 pgs: 97 active+clean; 3.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-09-19T06:55:56.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:56 smithi174 ceph-mon[35637]: pgmap v204: 97 pgs: 97 active+clean; 3.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-09-19T06:55:58.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:55:58 smithi103 ceph-mon[31158]: pgmap v205: 97 pgs: 97 active+clean; 3.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-09-19T06:55:58.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:55:58 smithi134 ceph-mon[35356]: pgmap v205: 97 pgs: 97 active+clean; 3.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-09-19T06:55:58.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:55:58 smithi174 ceph-mon[35637]: pgmap v205: 97 pgs: 97 active+clean; 3.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-09-19T06:55:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.450+0000 7f620aba6640 1 -- 172.21.15.103:0/1530345616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6204101350 msgr2=0x7f620410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.450+0000 7f620aba6640 1 --2- 172.21.15.103:0/1530345616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6204101350 0x7f620410d880 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009cd0 tx=0x7f61fc02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.450+0000 7f620aba6640 1 -- 172.21.15.103:0/1530345616 shutdown_connections 2024-09-19T06:55:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.450+0000 7f620aba6640 1 --2- 172.21.15.103:0/1530345616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6204101350 0x7f620410d880 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.450+0000 7f620aba6640 1 --2- 172.21.15.103:0/1530345616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62041009b0 0x7f6204100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.450+0000 7f620aba6640 1 --2- 172.21.15.103:0/1530345616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62041069c0 0x7f6204106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.450+0000 7f620aba6640 1 -- 172.21.15.103:0/1530345616 >> 172.21.15.103:0/1530345616 conn(0x7f6204077990 msgr2=0x7f62040fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.450+0000 7f620aba6640 1 -- 172.21.15.103:0/1530345616 shutdown_connections 2024-09-19T06:55:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.450+0000 7f620aba6640 1 -- 172.21.15.103:0/1530345616 wait complete. 2024-09-19T06:55:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f620aba6640 1 Processor -- start 2024-09-19T06:55:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f620aba6640 1 -- start start 2024-09-19T06:55:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f620aba6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62041009b0 0x7f6204199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f620aba6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6204101350 0x7f6204199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f620aba6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62041069c0 0x7f62041a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f620aba6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6204110e00 con 0x7f62041069c0 2024-09-19T06:55:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f620aba6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6204110c80 con 0x7f62041009b0 2024-09-19T06:55:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f620aba6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6204110f80 con 0x7f6204101350 2024-09-19T06:55:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f6209143640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62041069c0 0x7f62041a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f6209143640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62041069c0 0x7f62041a41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57172/0 (socket says 172.21.15.103:57172) 2024-09-19T06:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f6209143640 1 -- 172.21.15.103:0/4108902364 learned_addr learned my addr 172.21.15.103:0/4108902364 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f6203fff640 1 --2- 172.21.15.103:0/4108902364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6204101350 0x7f6204199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f6208942640 1 --2- 172.21.15.103:0/4108902364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62041009b0 0x7f6204199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f6209143640 1 -- 172.21.15.103:0/4108902364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6204101350 msgr2=0x7f6204199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f6209143640 1 --2- 172.21.15.103:0/4108902364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6204101350 0x7f6204199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f6209143640 1 -- 172.21.15.103:0/4108902364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62041009b0 msgr2=0x7f6204199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f6209143640 1 --2- 172.21.15.103:0/4108902364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62041009b0 0x7f6204199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f6209143640 1 -- 172.21.15.103:0/4108902364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62041a4710 con 0x7f62041069c0 2024-09-19T06:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.451+0000 7f6208942640 1 --2- 172.21.15.103:0/4108902364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62041009b0 0x7f6204199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.452+0000 7f6209143640 1 --2- 172.21.15.103:0/4108902364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62041069c0 0x7f62041a41d0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f61fc005cc0 tx=0x7f61fc032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.452+0000 7f6201ffb640 1 -- 172.21.15.103:0/4108902364 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc004040 con 0x7f62041069c0 2024-09-19T06:55:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.452+0000 7f6201ffb640 1 -- 172.21.15.103:0/4108902364 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61fc0046c0 con 0x7f62041069c0 2024-09-19T06:55:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.452+0000 7f620aba6640 1 -- 172.21.15.103:0/4108902364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62041a48e0 con 0x7f62041069c0 2024-09-19T06:55:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.452+0000 7f6201ffb640 1 -- 172.21.15.103:0/4108902364 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc030e00 con 0x7f62041069c0 2024-09-19T06:55:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.452+0000 7f620aba6640 1 -- 172.21.15.103:0/4108902364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62041a4da0 con 0x7f62041069c0 2024-09-19T06:55:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.454+0000 7f620da22740 1 Processor -- start 2024-09-19T06:55:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.454+0000 7f6201ffb640 1 -- 172.21.15.103:0/4108902364 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f61fc0305a0 con 0x7f62041069c0 2024-09-19T06:55:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.454+0000 7f620da22740 1 -- start start 2024-09-19T06:55:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f620da22740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bda9faf0 0x5579bda9fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f620da22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bdaa0410 0x5579bdaaa770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f620da22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bdaaacb0 0x5579bdaad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f620da22740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5579bda042f0 con 0x5579bda9faf0 2024-09-19T06:55:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f620da22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5579bd9ac270 con 0x5579bdaaacb0 2024-09-19T06:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f620da22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5579bda002b0 con 0x5579bdaa0410 2024-09-19T06:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f6203fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bda9faf0 0x5579bda9fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f6203fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bda9faf0 0x5579bda9fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57178/0 (socket says 172.21.15.103:57178) 2024-09-19T06:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f6203fff640 1 -- 172.21.15.103:0/2813864721 learned_addr learned my addr 172.21.15.103:0/2813864721 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f6208942640 1 --2- 172.21.15.103:0/2813864721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bdaa0410 0x5579bdaaa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f6201ffb640 1 --2- 172.21.15.103:0/4108902364 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f61f40a3460 0x7f61f40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f6203fff640 1 -- 172.21.15.103:0/2813864721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bdaa0410 msgr2=0x5579bdaaa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f6203fff640 1 --2- 172.21.15.103:0/2813864721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bdaa0410 0x5579bdaaa770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f6203fff640 1 -- 172.21.15.103:0/2813864721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bdaaacb0 msgr2=0x5579bdaad0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f6203fff640 1 --2- 172.21.15.103:0/2813864721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bdaaacb0 0x5579bdaad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f6203fff640 1 -- 172.21.15.103:0/2813864721 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579bd9ab340 con 0x5579bda9faf0 2024-09-19T06:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f6208942640 1 --2- 172.21.15.103:0/2813864721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bdaa0410 0x5579bdaaa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f6209143640 1 --2- 172.21.15.103:0/4108902364 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f61f40a3460 0x7f61f40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f6201ffb640 1 -- 172.21.15.103:0/4108902364 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f61fc02e030 con 0x7f62041069c0 2024-09-19T06:55:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.455+0000 7f6203fff640 1 --2- 172.21.15.103:0/2813864721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bda9faf0 0x5579bda9fed0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f61f000bd40 tx=0x7f61f00127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0392cc17292f28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.456+0000 7f61eeffd640 1 -- 172.21.15.103:0/2813864721 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f001b470 con 0x5579bda9faf0 2024-09-19T06:55:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.456+0000 7f61eeffd640 1 -- 172.21.15.103:0/2813864721 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61f0016070 con 0x5579bda9faf0 2024-09-19T06:55:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.456+0000 7f61eeffd640 1 -- 172.21.15.103:0/2813864721 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f000f620 con 0x5579bda9faf0 2024-09-19T06:55:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.456+0000 7f620da22740 1 -- 172.21.15.103:0/2813864721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bda9faf0 msgr2=0x5579bda9fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.456+0000 7f620da22740 1 --2- 172.21.15.103:0/2813864721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bda9faf0 0x5579bda9fed0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f61f000bd40 tx=0x7f61f00127c0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.456+0000 7f6209143640 1 --2- 172.21.15.103:0/4108902364 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f61f40a3460 0x7f61f40a5920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0bd520 tx=0x7f61fc088600 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.456+0000 7f620da22740 1 -- 172.21.15.103:0/2813864721 shutdown_connections 2024-09-19T06:55:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.456+0000 7f620da22740 1 --2- 172.21.15.103:0/2813864721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bdaaacb0 0x5579bdaad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.456+0000 7f620da22740 1 --2- 172.21.15.103:0/2813864721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bdaa0410 0x5579bdaaa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.456+0000 7f620da22740 1 --2- 172.21.15.103:0/2813864721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bda9faf0 0x5579bda9fed0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.456+0000 7f620da22740 1 -- 172.21.15.103:0/2813864721 >> 172.21.15.103:0/2813864721 conn(0x5579bda9c060 msgr2=0x5579bda9c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.456+0000 7f620da22740 1 -- 172.21.15.103:0/2813864721 shutdown_connections 2024-09-19T06:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.456+0000 7f620da22740 1 -- 172.21.15.103:0/2813864721 wait complete. 2024-09-19T06:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f620da22740 1 -- start start 2024-09-19T06:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f620da22740 1 Processor -- start 2024-09-19T06:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f620da22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bda9faf0 0x5579bdaa7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f620da22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bdaa0410 0x5579bdaa80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f620da22740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaaacb0 0x5579bdaa85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f620da22740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5579bdaad880 con 0x5579bdaaacb0 2024-09-19T06:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f620da22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5579bd98d290 con 0x5579bdaa0410 2024-09-19T06:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f620da22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5579bdaada00 con 0x5579bda9faf0 2024-09-19T06:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f620da22740 10 client.0 _ll_register_callbacks cb 0x5579bd9be140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f6203fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaaacb0 0x5579bdaa85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f6203fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaaacb0 0x5579bdaa85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57190/0 (socket says 172.21.15.103:57190) 2024-09-19T06:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f6203fff640 1 -- 172.21.15.103:0/1697290435 learned_addr learned my addr 172.21.15.103:0/1697290435 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f6208942640 1 --2- 172.21.15.103:0/1697290435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bdaa0410 0x5579bdaa80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f6203fff640 1 -- 172.21.15.103:0/1697290435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bda9faf0 msgr2=0x5579bdaa7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f6203fff640 1 --2- 172.21.15.103:0/1697290435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bda9faf0 0x5579bdaa7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f6203fff640 1 -- 172.21.15.103:0/1697290435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bdaa0410 msgr2=0x5579bdaa80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f6203fff640 1 --2- 172.21.15.103:0/1697290435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bdaa0410 0x5579bdaa80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f6203fff640 1 -- 172.21.15.103:0/1697290435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579bd717210 con 0x5579bdaaacb0 2024-09-19T06:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.457+0000 7f6208942640 1 --2- 172.21.15.103:0/1697290435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bdaa0410 0x5579bdaa80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.458+0000 7f6203fff640 1 --2- 172.21.15.103:0/1697290435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaaacb0 0x5579bdaa85e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f61f0012300 tx=0x7f61f0012330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78cba6392129b52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.458+0000 7f61e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.458+0000 7f61e3fff640 1 -- 172.21.15.103:0/1697290435 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0010ab0 con 0x5579bdaaacb0 2024-09-19T06:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.458+0000 7f61e3fff640 1 -- 172.21.15.103:0/1697290435 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f61f0005050 con 0x5579bdaaacb0 2024-09-19T06:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.458+0000 7f620da22740 1 -- 172.21.15.103:0/1697290435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61f0016070 con 0x5579bdaaacb0 2024-09-19T06:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.458+0000 7f61e3fff640 1 -- 172.21.15.103:0/1697290435 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f61f0005250 con 0x5579bdaaacb0 2024-09-19T06:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.458+0000 7f620da22740 10 client.17796 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.458+0000 7f620da22740 1 -- 172.21.15.103:0/1697290435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5579bd7e3620 con 0x5579bdaaacb0 2024-09-19T06:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.458+0000 7f61e3fff640 1 -- 172.21.15.103:0/1697290435 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61f0016070 con 0x5579bdaaacb0 2024-09-19T06:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.458+0000 7f61e3fff640 1 -- 172.21.15.103:0/1697290435 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f61f002f500 con 0x5579bdaaacb0 2024-09-19T06:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.458+0000 7f620da22740 10 client.17796 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.458+0000 7f620da22740 10 client.17796 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.458+0000 7f620da22740 20 client.17796 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.458+0000 7f620da22740 1 --2- 172.21.15.103:0/1697290435 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5579bdabaa80 0x5579bdabce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.458+0000 7f620da22740 4 client.17796 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.458+0000 7f620da22740 1 -- 172.21.15.103:0/1697290435 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579bda11a20 con 0x5579bdabaa80 2024-09-19T06:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.459+0000 7f6208942640 1 --2- 172.21.15.103:0/1697290435 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5579bdabaa80 0x5579bdabce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.459+0000 7f61e3fff640 10 client.17796 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.459+0000 7f6208942640 1 --2- 172.21.15.103:0/1697290435 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5579bdabaa80 0x5579bdabce70 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa3c102571a29726 server_cookie=a95e53cc3a2857e3 in_seq=0 out_seq=0 2024-09-19T06:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.460+0000 7f61e3fff640 1 -- 172.21.15.103:0/1697290435 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579bda11a20 con 0x5579bdabaa80 2024-09-19T06:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.460+0000 7f61e3fff640 10 client.17796 handle_command_reply: tid=0 2024-09-19T06:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.460+0000 7f620da22740 1 client.17796 shutdown 2024-09-19T06:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.460+0000 7f620da22740 1 -- 172.21.15.103:0/1697290435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaaacb0 msgr2=0x5579bdaa85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.460+0000 7f620da22740 1 --2- 172.21.15.103:0/1697290435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaaacb0 0x5579bdaa85e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f61f0012300 tx=0x7f61f0012330 comp rx=0 tx=0).stop 2024-09-19T06:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.460+0000 7f620da22740 1 -- 172.21.15.103:0/1697290435 shutdown_connections 2024-09-19T06:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.460+0000 7f620da22740 1 --2- 172.21.15.103:0/1697290435 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5579bdabaa80 0x5579bdabce70 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.460+0000 7f620da22740 1 --2- 172.21.15.103:0/1697290435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaaacb0 0x5579bdaa85e0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.460+0000 7f61e3fff640 0 client.17796 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.460+0000 7f620da22740 1 --2- 172.21.15.103:0/1697290435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bdaa0410 0x5579bdaa80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.460+0000 7f620da22740 1 --2- 172.21.15.103:0/1697290435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bda9faf0 0x5579bdaa7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.460+0000 7f620da22740 1 -- 172.21.15.103:0/1697290435 >> 172.21.15.103:0/1697290435 conn(0x5579bda9c060 msgr2=0x5579bdaa1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.460+0000 7f620da22740 1 -- 172.21.15.103:0/1697290435 shutdown_connections 2024-09-19T06:55:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.460+0000 7f620da22740 1 -- 172.21.15.103:0/1697290435 wait complete. 2024-09-19T06:55:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.460+0000 7f620da22740 20 client.17796 trim_cache size 0 max 16384 2024-09-19T06:55:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.474+0000 7f620da22740 1 Processor -- start 2024-09-19T06:55:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.474+0000 7f620da22740 1 -- start start 2024-09-19T06:55:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.474+0000 7f620da22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bda9faf0 0x5579bdaa8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.474+0000 7f620da22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bdaa0410 0x5579bdac2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.474+0000 7f620da22740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaaacb0 0x5579bdac29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.474+0000 7f620da22740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62041ad700 con 0x5579bdaaacb0 2024-09-19T06:55:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.474+0000 7f620da22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62041af240 con 0x5579bda9faf0 2024-09-19T06:55:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.474+0000 7f620da22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62041addd0 con 0x5579bdaa0410 2024-09-19T06:55:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.474+0000 7f6203fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaaacb0 0x5579bdac29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.474+0000 7f6203fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaaacb0 0x5579bdac29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57192/0 (socket says 172.21.15.103:57192) 2024-09-19T06:55:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.474+0000 7f6203fff640 1 -- 172.21.15.103:0/4175665016 learned_addr learned my addr 172.21.15.103:0/4175665016 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.475+0000 7f6203fff640 1 -- 172.21.15.103:0/4175665016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bdaa0410 msgr2=0x5579bdac2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.475+0000 7f6203fff640 1 --2- 172.21.15.103:0/4175665016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bdaa0410 0x5579bdac2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.475+0000 7f6203fff640 1 -- 172.21.15.103:0/4175665016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bda9faf0 msgr2=0x5579bdaa8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.475+0000 7f6203fff640 1 --2- 172.21.15.103:0/4175665016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bda9faf0 0x5579bdaa8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.475+0000 7f6203fff640 1 -- 172.21.15.103:0/4175665016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579bdaa0240 con 0x5579bdaaacb0 2024-09-19T06:55:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.475+0000 7f6203fff640 1 --2- 172.21.15.103:0/4175665016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaaacb0 0x5579bdac29b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f61f002fa70 tx=0x7f61f0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d591681f247641c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.475+0000 7f61ee7fc640 1 -- 172.21.15.103:0/4175665016 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f001f070 con 0x5579bdaaacb0 2024-09-19T06:55:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.475+0000 7f61ee7fc640 1 -- 172.21.15.103:0/4175665016 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61f0034070 con 0x5579bdaaacb0 2024-09-19T06:55:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.475+0000 7f61ee7fc640 1 -- 172.21.15.103:0/4175665016 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f002f6d0 con 0x5579bdaaacb0 2024-09-19T06:55:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.475+0000 7f620da22740 1 -- 172.21.15.103:0/4175665016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaaacb0 msgr2=0x5579bdac29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.475+0000 7f620da22740 1 --2- 172.21.15.103:0/4175665016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaaacb0 0x5579bdac29b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f61f002fa70 tx=0x7f61f0001e50 comp rx=0 tx=0).stop 2024-09-19T06:55:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f620da22740 1 -- 172.21.15.103:0/4175665016 shutdown_connections 2024-09-19T06:55:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f620da22740 1 --2- 172.21.15.103:0/4175665016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaaacb0 0x5579bdac29b0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f620da22740 1 --2- 172.21.15.103:0/4175665016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bdaa0410 0x5579bdac2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f620da22740 1 --2- 172.21.15.103:0/4175665016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bda9faf0 0x5579bdaa8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f620da22740 1 -- 172.21.15.103:0/4175665016 >> 172.21.15.103:0/4175665016 conn(0x5579bda9c060 msgr2=0x5579bdaa9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f620da22740 1 -- 172.21.15.103:0/4175665016 shutdown_connections 2024-09-19T06:55:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f620da22740 1 -- 172.21.15.103:0/4175665016 wait complete. 2024-09-19T06:55:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f620da22740 1 -- start start 2024-09-19T06:55:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f620da22740 1 Processor -- start 2024-09-19T06:55:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f620da22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bda9faf0 0x5579bdab34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f620da22740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaa0410 0x5579bdaae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f620da22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bdaaacb0 0x5579bdaaedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f620da22740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62041aeb70 con 0x5579bdaa0410 2024-09-19T06:55:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f620da22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62041ae4a0 con 0x5579bda9faf0 2024-09-19T06:55:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f620da22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62041ab6c0 con 0x5579bdaaacb0 2024-09-19T06:55:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f620da22740 10 client.0 _ll_register_callbacks cb 0x7f61f000bc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f6208942640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaa0410 0x5579bdaae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f6208942640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaa0410 0x5579bdaae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57200/0 (socket says 172.21.15.103:57200) 2024-09-19T06:55:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.476+0000 7f6208942640 1 -- 172.21.15.103:0/1130419534 learned_addr learned my addr 172.21.15.103:0/1130419534 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:55:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f6203fff640 1 --2- 172.21.15.103:0/1130419534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bdaaacb0 0x5579bdaaedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f6203fff640 1 --2- 172.21.15.103:0/1130419534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bda9faf0 0x5579bdab34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f6208942640 1 -- 172.21.15.103:0/1130419534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bdaaacb0 msgr2=0x5579bdaaedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f6208942640 1 --2- 172.21.15.103:0/1130419534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bdaaacb0 0x5579bdaaedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f6208942640 1 -- 172.21.15.103:0/1130419534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bda9faf0 msgr2=0x5579bdab34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f6208942640 1 --2- 172.21.15.103:0/1130419534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bda9faf0 0x5579bdab34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f6208942640 1 -- 172.21.15.103:0/1130419534 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579bdaaf600 con 0x5579bdaa0410 2024-09-19T06:55:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f6208942640 1 --2- 172.21.15.103:0/1130419534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaa0410 0x5579bdaae7e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x5579bdaa91a0 tx=0x7f61f800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ef38b8b72fd743 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f61e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:55:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f61e3fff640 1 -- 172.21.15.103:0/1130419534 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8004230 con 0x5579bdaa0410 2024-09-19T06:55:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f61e3fff640 1 -- 172.21.15.103:0/1130419534 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8512+0+0 (secure 0 0 0) 0x7f61f80043d0 con 0x5579bdaa0410 2024-09-19T06:55:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f620da22740 1 -- 172.21.15.103:0/1130419534 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61f0034070 con 0x5579bdaa0410 2024-09-19T06:55:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f61e3fff640 1 -- 172.21.15.103:0/1130419534 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f61f802b050 con 0x5579bdaa0410 2024-09-19T06:55:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f620da22740 10 client.17802 fetch_fsmap learned FSMap version 27 2024-09-19T06:55:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f620da22740 1 -- 172.21.15.103:0/1130419534 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5579bdb255c0 con 0x5579bdaa0410 2024-09-19T06:55:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f61e3fff640 1 -- 172.21.15.103:0/1130419534 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61f0034070 con 0x5579bdaa0410 2024-09-19T06:55:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f61e3fff640 1 -- 172.21.15.103:0/1130419534 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f61f8003b50 con 0x5579bdaa0410 2024-09-19T06:55:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f620da22740 10 client.17802 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:55:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f620da22740 10 client.17802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f620da22740 20 client.17802 populate_metadata read hostname 'smithi103' 2024-09-19T06:55:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f620da22740 1 --2- 172.21.15.103:0/1130419534 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5579bdb25d70 0x5579bdb28190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f620da22740 4 client.17802 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:55:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.477+0000 7f620da22740 1 -- 172.21.15.103:0/1130419534 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579bda0bd70 con 0x5579bdb25d70 2024-09-19T06:55:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.478+0000 7f6203fff640 1 --2- 172.21.15.103:0/1130419534 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5579bdb25d70 0x5579bdb28190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.478+0000 7f61e3fff640 10 client.17802 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.478+0000 7f6203fff640 1 --2- 172.21.15.103:0/1130419534 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5579bdb25d70 0x5579bdb28190 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=732990178ec47f8 server_cookie=94dc250f2e9c0938 in_seq=0 out_seq=0 2024-09-19T06:55:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.492+0000 7f61e3fff640 1 -- 172.21.15.103:0/1130419534 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579bda0bd70 con 0x5579bdb25d70 2024-09-19T06:55:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.492+0000 7f61e3fff640 10 client.17802 handle_command_reply: tid=0 2024-09-19T06:55:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.492+0000 7f620da22740 1 client.17802 shutdown 2024-09-19T06:55:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.492+0000 7f620da22740 1 -- 172.21.15.103:0/1130419534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaa0410 msgr2=0x5579bdaae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.492+0000 7f620da22740 1 --2- 172.21.15.103:0/1130419534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaa0410 0x5579bdaae7e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x5579bdaa91a0 tx=0x7f61f800f7f0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.492+0000 7f620da22740 1 -- 172.21.15.103:0/1130419534 shutdown_connections 2024-09-19T06:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.492+0000 7f620da22740 1 --2- 172.21.15.103:0/1130419534 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5579bdb25d70 0x5579bdb28190 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.492+0000 7f620da22740 1 --2- 172.21.15.103:0/1130419534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579bdaaacb0 0x5579bdaaedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.492+0000 7f620da22740 1 --2- 172.21.15.103:0/1130419534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5579bdaa0410 0x5579bdaae7e0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.492+0000 7f61e3fff640 0 client.17802 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.492+0000 7f620da22740 1 --2- 172.21.15.103:0/1130419534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5579bda9faf0 0x5579bdab34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.492+0000 7f620da22740 1 -- 172.21.15.103:0/1130419534 >> 172.21.15.103:0/1130419534 conn(0x5579bda9c060 msgr2=0x5579bdaa93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.492+0000 7f620da22740 1 -- 172.21.15.103:0/1130419534 shutdown_connections 2024-09-19T06:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.492+0000 7f620da22740 1 -- 172.21.15.103:0/1130419534 wait complete. 2024-09-19T06:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.492+0000 7f620da22740 20 client.17802 trim_cache size 0 max 16384 2024-09-19T06:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.493+0000 7f620aba6640 1 -- 172.21.15.103:0/4108902364 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f61f40a3460 msgr2=0x7f61f40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.493+0000 7f620aba6640 1 --2- 172.21.15.103:0/4108902364 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f61f40a3460 0x7f61f40a5920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0bd520 tx=0x7f61fc088600 comp rx=0 tx=0).stop 2024-09-19T06:55:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.493+0000 7f620aba6640 1 -- 172.21.15.103:0/4108902364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62041069c0 msgr2=0x7f62041a41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.493+0000 7f620aba6640 1 --2- 172.21.15.103:0/4108902364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62041069c0 0x7f62041a41d0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f61fc005cc0 tx=0x7f61fc032f40 comp rx=0 tx=0).stop 2024-09-19T06:55:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.493+0000 7f620aba6640 1 -- 172.21.15.103:0/4108902364 shutdown_connections 2024-09-19T06:55:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.493+0000 7f620aba6640 1 --2- 172.21.15.103:0/4108902364 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f61f40a3460 0x7f61f40a5920 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.493+0000 7f620aba6640 1 --2- 172.21.15.103:0/4108902364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62041069c0 0x7f62041a41d0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.493+0000 7f620aba6640 1 --2- 172.21.15.103:0/4108902364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6204101350 0x7f6204199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.493+0000 7f620aba6640 1 --2- 172.21.15.103:0/4108902364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62041009b0 0x7f6204199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.493+0000 7f620aba6640 1 -- 172.21.15.103:0/4108902364 >> 172.21.15.103:0/4108902364 conn(0x7f6204077990 msgr2=0x7f6204103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.493+0000 7f620aba6640 1 -- 172.21.15.103:0/4108902364 shutdown_connections 2024-09-19T06:55:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:55:59.493+0000 7f620aba6640 1 -- 172.21.15.103:0/4108902364 wait complete. 2024-09-19T06:56:00.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:00 smithi103 ceph-mon[31158]: pgmap v206: 97 pgs: 97 active+clean; 3.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-09-19T06:56:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:00 smithi134 ceph-mon[35356]: pgmap v206: 97 pgs: 97 active+clean; 3.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-09-19T06:56:00.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:00 smithi174 ceph-mon[35637]: pgmap v206: 97 pgs: 97 active+clean; 3.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 60 KiB/s wr, 24 op/s 2024-09-19T06:56:02.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:02 smithi103 ceph-mon[31158]: pgmap v207: 97 pgs: 97 active+clean; 3.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:56:02.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:02 smithi134 ceph-mon[35356]: pgmap v207: 97 pgs: 97 active+clean; 3.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:56:02.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:02 smithi174 ceph-mon[35637]: pgmap v207: 97 pgs: 97 active+clean; 3.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:56:04.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:04 smithi103 ceph-mon[31158]: pgmap v208: 97 pgs: 97 active+clean; 3.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:56:04.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:04 smithi134 ceph-mon[35356]: pgmap v208: 97 pgs: 97 active+clean; 3.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:56:04.557 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:56:04.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:04 smithi174 ceph-mon[35637]: pgmap v208: 97 pgs: 97 active+clean; 3.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:56:04.817 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:05.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.323+0000 7f9ba57c0640 1 -- 172.21.15.103:0/3025377870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ba0102d30 msgr2=0x7f9ba0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:05.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.323+0000 7f9ba57c0640 1 --2- 172.21.15.103:0/3025377870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ba0102d30 0x7f9ba0103110 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009930 tx=0x7f9b9002a420 comp rx=0 tx=0).stop 2024-09-19T06:56:05.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.323+0000 7f9ba57c0640 1 -- 172.21.15.103:0/3025377870 shutdown_connections 2024-09-19T06:56:05.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.323+0000 7f9ba57c0640 1 --2- 172.21.15.103:0/3025377870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ba01082a0 0x7f9ba010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.323+0000 7f9ba57c0640 1 --2- 172.21.15.103:0/3025377870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ba01036e0 0x7f9ba0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.323+0000 7f9ba57c0640 1 --2- 172.21.15.103:0/3025377870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ba0102d30 0x7f9ba0103110 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.323+0000 7f9ba57c0640 1 -- 172.21.15.103:0/3025377870 >> 172.21.15.103:0/3025377870 conn(0x7f9ba00fc7d0 msgr2=0x7f9ba00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:05.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.323+0000 7f9ba57c0640 1 -- 172.21.15.103:0/3025377870 shutdown_connections 2024-09-19T06:56:05.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.323+0000 7f9ba57c0640 1 -- 172.21.15.103:0/3025377870 wait complete. 2024-09-19T06:56:05.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.323+0000 7f9ba57c0640 1 Processor -- start 2024-09-19T06:56:05.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9ba57c0640 1 -- start start 2024-09-19T06:56:05.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9ba57c0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ba0102d30 0x7f9ba0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9ba57c0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ba01036e0 0x7f9ba0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9ba57c0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ba01082a0 0x7f9ba0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9ba57c0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba01120a0 con 0x7f9ba01036e0 2024-09-19T06:56:05.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9ba57c0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba0111f20 con 0x7f9ba01082a0 2024-09-19T06:56:05.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9ba57c0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba0112220 con 0x7f9ba0102d30 2024-09-19T06:56:05.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9b967fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ba01036e0 0x7f9ba0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9b967fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ba01036e0 0x7f9ba0100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57220/0 (socket says 172.21.15.103:57220) 2024-09-19T06:56:05.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9b967fc640 1 -- 172.21.15.103:0/4104732044 learned_addr learned my addr 172.21.15.103:0/4104732044 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:05.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9b9effd640 1 --2- 172.21.15.103:0/4104732044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ba0102d30 0x7f9ba0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9b9f7fe640 1 --2- 172.21.15.103:0/4104732044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ba01082a0 0x7f9ba0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9b967fc640 1 -- 172.21.15.103:0/4104732044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ba0102d30 msgr2=0x7f9ba0100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:05.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9b967fc640 1 --2- 172.21.15.103:0/4104732044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ba0102d30 0x7f9ba0100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9b967fc640 1 -- 172.21.15.103:0/4104732044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ba01082a0 msgr2=0x7f9ba0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:05.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9b967fc640 1 --2- 172.21.15.103:0/4104732044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ba01082a0 0x7f9ba0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9b967fc640 1 -- 172.21.15.103:0/4104732044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba01a6a60 con 0x7f9ba01036e0 2024-09-19T06:56:05.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9b9effd640 1 --2- 172.21.15.103:0/4104732044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ba0102d30 0x7f9ba0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:05.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9b9f7fe640 1 --2- 172.21.15.103:0/4104732044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ba01082a0 0x7f9ba0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:05.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.324+0000 7f9b967fc640 1 --2- 172.21.15.103:0/4104732044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ba01036e0 0x7f9ba0100ed0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800c8c0 tx=0x7f9b8800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:05.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.325+0000 7f9b9cff9640 1 -- 172.21.15.103:0/4104732044 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88007bb0 con 0x7f9ba01036e0 2024-09-19T06:56:05.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.325+0000 7f9ba57c0640 1 -- 172.21.15.103:0/4104732044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba01a6d50 con 0x7f9ba01036e0 2024-09-19T06:56:05.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.325+0000 7f9b9cff9640 1 -- 172.21.15.103:0/4104732044 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b88007d50 con 0x7f9ba01036e0 2024-09-19T06:56:05.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.325+0000 7f9b9cff9640 1 -- 172.21.15.103:0/4104732044 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88018830 con 0x7f9ba01036e0 2024-09-19T06:56:05.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.325+0000 7f9ba57c0640 1 -- 172.21.15.103:0/4104732044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba01a7250 con 0x7f9ba01036e0 2024-09-19T06:56:05.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.327+0000 7f9b9cff9640 1 -- 172.21.15.103:0/4104732044 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9b88018a10 con 0x7f9ba01036e0 2024-09-19T06:56:05.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.327+0000 7f9ba7b99740 1 Processor -- start 2024-09-19T06:56:05.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.327+0000 7f9ba7b99740 1 -- start start 2024-09-19T06:56:05.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9ba7b99740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e464a0 0x55dc64e46880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9ba7b99740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e46dc0 0x55dc64e51140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9ba7b99740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e51680 0x55dc64e53a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9ba7b99740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dc64dac2e0 con 0x55dc64e51680 2024-09-19T06:56:05.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9ba7b99740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dc64d53ce0 con 0x55dc64e46dc0 2024-09-19T06:56:05.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9ba7b99740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc64da82a0 con 0x55dc64e464a0 2024-09-19T06:56:05.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9b9cff9640 1 --2- 172.21.15.103:0/4104732044 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9b8c0a33b0 0x7f9b8c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9b967fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e51680 0x55dc64e53a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9b967fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e51680 0x55dc64e53a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57232/0 (socket says 172.21.15.103:57232) 2024-09-19T06:56:05.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9b967fc640 1 -- 172.21.15.103:0/931905255 learned_addr learned my addr 172.21.15.103:0/931905255 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:05.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9b9f7fe640 1 --2- 172.21.15.103:0/4104732044 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9b8c0a33b0 0x7f9b8c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9b9effd640 1 --2- 172.21.15.103:0/931905255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e46dc0 0x55dc64e51140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9b9f7fe640 1 --2- 172.21.15.103:0/931905255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e464a0 0x55dc64e46880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9b9cff9640 1 -- 172.21.15.103:0/4104732044 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8600+0+0 (secure 0 0 0) 0x7f9b880cff20 con 0x7f9ba01036e0 2024-09-19T06:56:05.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9b967fc640 1 -- 172.21.15.103:0/931905255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e464a0 msgr2=0x55dc64e46880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:05.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9b967fc640 1 --2- 172.21.15.103:0/931905255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e464a0 0x55dc64e46880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9b967fc640 1 -- 172.21.15.103:0/931905255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e46dc0 msgr2=0x55dc64e51140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:05.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9b967fc640 1 --2- 172.21.15.103:0/931905255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e46dc0 0x55dc64e51140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9b967fc640 1 -- 172.21.15.103:0/931905255 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc64d38440 con 0x55dc64e51680 2024-09-19T06:56:05.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.328+0000 7f9b9f7fe640 1 --2- 172.21.15.103:0/931905255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e464a0 0x55dc64e46880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:05.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.329+0000 7f9b967fc640 1 --2- 172.21.15.103:0/931905255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e51680 0x55dc64e53a70 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f9b88098610 tx=0x7f9b8806df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106a23620ec9b2d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:05.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.329+0000 7f9b9f7fe640 1 --2- 172.21.15.103:0/4104732044 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9b8c0a33b0 0x7f9b8c0a5870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b98005c40 tx=0x7f9b980094a0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:05.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.329+0000 7f9b957fa640 1 -- 172.21.15.103:0/931905255 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8800de20 con 0x55dc64e51680 2024-09-19T06:56:05.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.329+0000 7f9b957fa640 1 -- 172.21.15.103:0/931905255 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b8809c1b0 con 0x55dc64e51680 2024-09-19T06:56:05.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.329+0000 7f9b957fa640 1 -- 172.21.15.103:0/931905255 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b880cd8a0 con 0x55dc64e51680 2024-09-19T06:56:05.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 -- 172.21.15.103:0/931905255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e51680 msgr2=0x55dc64e53a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:05.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/931905255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e51680 0x55dc64e53a70 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f9b88098610 tx=0x7f9b8806df30 comp rx=0 tx=0).stop 2024-09-19T06:56:05.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 -- 172.21.15.103:0/931905255 shutdown_connections 2024-09-19T06:56:05.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/931905255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e51680 0x55dc64e53a70 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/931905255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e46dc0 0x55dc64e51140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/931905255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e464a0 0x55dc64e46880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 -- 172.21.15.103:0/931905255 >> 172.21.15.103:0/931905255 conn(0x55dc64e42a30 msgr2=0x55dc64e42e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:05.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 -- 172.21.15.103:0/931905255 shutdown_connections 2024-09-19T06:56:05.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 -- 172.21.15.103:0/931905255 wait complete. 2024-09-19T06:56:05.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 -- start start 2024-09-19T06:56:05.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 Processor -- start 2024-09-19T06:56:05.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e464a0 0x55dc64e51010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e46dc0 0x55dc64e4e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e51680 0x55dc64e4e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dc64e54250 con 0x55dc64e46dc0 2024-09-19T06:56:05.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dc64d67050 con 0x55dc64e51680 2024-09-19T06:56:05.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc64e543d0 con 0x55dc64e464a0 2024-09-19T06:56:05.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9ba7b99740 10 client.0 _ll_register_callbacks cb 0x55dc64da7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:05.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9b967fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e46dc0 0x55dc64e4e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.330+0000 7f9b967fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e46dc0 0x55dc64e4e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57246/0 (socket says 172.21.15.103:57246) 2024-09-19T06:56:05.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9b967fc640 1 -- 172.21.15.103:0/3514680004 learned_addr learned my addr 172.21.15.103:0/3514680004 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:05.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9b9f7fe640 1 --2- 172.21.15.103:0/3514680004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e51680 0x55dc64e4e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9b9effd640 1 --2- 172.21.15.103:0/3514680004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e464a0 0x55dc64e51010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9b967fc640 1 -- 172.21.15.103:0/3514680004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e464a0 msgr2=0x55dc64e51010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:05.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9b967fc640 1 --2- 172.21.15.103:0/3514680004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e464a0 0x55dc64e51010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9b967fc640 1 -- 172.21.15.103:0/3514680004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e51680 msgr2=0x55dc64e4e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:05.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9b967fc640 1 --2- 172.21.15.103:0/3514680004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e51680 0x55dc64e4e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9b967fc640 1 -- 172.21.15.103:0/3514680004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc64b1d3c0 con 0x55dc64e46dc0 2024-09-19T06:56:05.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9b9f7fe640 1 --2- 172.21.15.103:0/3514680004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e51680 0x55dc64e4e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:05.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9b967fc640 1 --2- 172.21.15.103:0/3514680004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e46dc0 0x55dc64e4e2c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f9b880cd6d0 tx=0x7f9b88067140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6be40fece47b6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:05.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9b867fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:05.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9b867fc640 1 -- 172.21.15.103:0/3514680004 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b880a4e20 con 0x55dc64e46dc0 2024-09-19T06:56:05.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9b867fc640 1 -- 172.21.15.103:0/3514680004 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8600+0+0 (secure 0 0 0) 0x7f9b88098820 con 0x55dc64e46dc0 2024-09-19T06:56:05.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3514680004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b8809c1b0 con 0x55dc64e46dc0 2024-09-19T06:56:05.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9b867fc640 1 -- 172.21.15.103:0/3514680004 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9b8800de20 con 0x55dc64e46dc0 2024-09-19T06:56:05.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9ba7b99740 10 client.17814 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:05.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3514680004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dc64e5fc40 con 0x55dc64e46dc0 2024-09-19T06:56:05.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.331+0000 7f9b867fc640 1 -- 172.21.15.103:0/3514680004 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b8809c1b0 con 0x55dc64e46dc0 2024-09-19T06:56:05.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.332+0000 7f9b867fc640 1 -- 172.21.15.103:0/3514680004 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9b8809c410 con 0x55dc64e46dc0 2024-09-19T06:56:05.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.332+0000 7f9ba7b99740 10 client.17814 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:05.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.332+0000 7f9ba7b99740 10 client.17814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:05.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.332+0000 7f9ba7b99740 20 client.17814 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:05.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.332+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/3514680004 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dc64e60f90 0x55dc64e63380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.332+0000 7f9ba7b99740 4 client.17814 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:05.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.332+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3514680004 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc64db9850 con 0x55dc64e60f90 2024-09-19T06:56:05.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.332+0000 7f9b9effd640 1 --2- 172.21.15.103:0/3514680004 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dc64e60f90 0x55dc64e63380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.332+0000 7f9b9effd640 1 --2- 172.21.15.103:0/3514680004 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dc64e60f90 0x55dc64e63380 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75456693e98f2351 server_cookie=2893e8a9ead0ec10 in_seq=0 out_seq=0 2024-09-19T06:56:05.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.332+0000 7f9b867fc640 10 client.17814 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:05.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.333+0000 7f9b867fc640 1 -- 172.21.15.103:0/3514680004 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc64db9850 con 0x55dc64e60f90 2024-09-19T06:56:05.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.333+0000 7f9b867fc640 10 client.17814 handle_command_reply: tid=0 2024-09-19T06:56:05.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.333+0000 7f9ba7b99740 1 client.17814 shutdown 2024-09-19T06:56:05.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.333+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3514680004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e46dc0 msgr2=0x55dc64e4e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:05.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.333+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/3514680004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e46dc0 0x55dc64e4e2c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f9b880cd6d0 tx=0x7f9b88067140 comp rx=0 tx=0).stop 2024-09-19T06:56:05.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.333+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3514680004 shutdown_connections 2024-09-19T06:56:05.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.333+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/3514680004 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dc64e60f90 0x55dc64e63380 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.333+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/3514680004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e51680 0x55dc64e4e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.333+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/3514680004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e46dc0 0x55dc64e4e2c0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.333+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/3514680004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e464a0 0x55dc64e51010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.333+0000 7f9b867fc640 0 client.17814 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:05.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.333+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3514680004 >> 172.21.15.103:0/3514680004 conn(0x55dc64e42a30 msgr2=0x55dc64e48760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:05.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.333+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3514680004 shutdown_connections 2024-09-19T06:56:05.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.333+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3514680004 wait complete. 2024-09-19T06:56:05.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.333+0000 7f9ba7b99740 20 client.17814 trim_cache size 0 max 16384 2024-09-19T06:56:05.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.351+0000 7f9ba7b99740 1 Processor -- start 2024-09-19T06:56:05.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.351+0000 7f9ba7b99740 1 -- start start 2024-09-19T06:56:05.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.351+0000 7f9ba7b99740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e42a30 0x55dc64e663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.351+0000 7f9ba7b99740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e464a0 0x55dc64e668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.351+0000 7f9ba7b99740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e46dc0 0x55dc64e4b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.351+0000 7f9ba7b99740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba01afbe0 con 0x55dc64e42a30 2024-09-19T06:56:05.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.351+0000 7f9ba7b99740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba01b1720 con 0x55dc64e464a0 2024-09-19T06:56:05.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.351+0000 7f9ba7b99740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba01b02b0 con 0x55dc64e46dc0 2024-09-19T06:56:05.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.351+0000 7f9b9effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e42a30 0x55dc64e663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.351+0000 7f9b9effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e42a30 0x55dc64e663a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57252/0 (socket says 172.21.15.103:57252) 2024-09-19T06:56:05.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.351+0000 7f9b9effd640 1 -- 172.21.15.103:0/3271051663 learned_addr learned my addr 172.21.15.103:0/3271051663 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:05.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.351+0000 7f9b967fc640 1 --2- 172.21.15.103:0/3271051663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e464a0 0x55dc64e668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.351+0000 7f9b9f7fe640 1 --2- 172.21.15.103:0/3271051663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e46dc0 0x55dc64e4b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.351+0000 7f9b9effd640 1 -- 172.21.15.103:0/3271051663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e46dc0 msgr2=0x55dc64e4b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:05.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.351+0000 7f9b9effd640 1 --2- 172.21.15.103:0/3271051663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e46dc0 0x55dc64e4b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.352+0000 7f9b9effd640 1 -- 172.21.15.103:0/3271051663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e464a0 msgr2=0x55dc64e668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:05.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.352+0000 7f9b9effd640 1 --2- 172.21.15.103:0/3271051663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e464a0 0x55dc64e668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.352+0000 7f9b9effd640 1 -- 172.21.15.103:0/3271051663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc64e51380 con 0x55dc64e42a30 2024-09-19T06:56:05.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.352+0000 7f9b9f7fe640 1 --2- 172.21.15.103:0/3271051663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e46dc0 0x55dc64e4b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:05.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.352+0000 7f9b967fc640 1 --2- 172.21.15.103:0/3271051663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e464a0 0x55dc64e668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:05.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.352+0000 7f9b9effd640 1 --2- 172.21.15.103:0/3271051663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e42a30 0x55dc64e663a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9b90031550 tx=0x7f9b900314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec8c4b642b2b1e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:05.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.352+0000 7f9b94ff9640 1 -- 172.21.15.103:0/3271051663 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b90035070 con 0x55dc64e42a30 2024-09-19T06:56:05.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.352+0000 7f9b94ff9640 1 -- 172.21.15.103:0/3271051663 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b90048070 con 0x55dc64e42a30 2024-09-19T06:56:05.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.352+0000 7f9b94ff9640 1 -- 172.21.15.103:0/3271051663 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b900046c0 con 0x55dc64e42a30 2024-09-19T06:56:05.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3271051663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e42a30 msgr2=0x55dc64e663a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:05.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/3271051663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e42a30 0x55dc64e663a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9b90031550 tx=0x7f9b900314e0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3271051663 shutdown_connections 2024-09-19T06:56:05.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/3271051663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e46dc0 0x55dc64e4b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/3271051663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e464a0 0x55dc64e668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/3271051663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e42a30 0x55dc64e663a0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3271051663 >> 172.21.15.103:0/3271051663 conn(0x55dc64e49cb0 msgr2=0x55dc64e50340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:05.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3271051663 shutdown_connections 2024-09-19T06:56:05.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3271051663 wait complete. 2024-09-19T06:56:05.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 -- start start 2024-09-19T06:56:05.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 Processor -- start 2024-09-19T06:56:05.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e42a30 0x55dc64e64470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e464a0 0x55dc64e649b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e46dc0 0x55dc64eca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba01b1050 con 0x55dc64e464a0 2024-09-19T06:56:05.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba01b0980 con 0x55dc64e42a30 2024-09-19T06:56:05.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba01adbe0 con 0x55dc64e46dc0 2024-09-19T06:56:05.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9ba7b99740 10 client.0 _ll_register_callbacks cb 0x7f9b8800a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:05.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9b967fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e464a0 0x55dc64e649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9b967fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e464a0 0x55dc64e649b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57254/0 (socket says 172.21.15.103:57254) 2024-09-19T06:56:05.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.353+0000 7f9b967fc640 1 -- 172.21.15.103:0/3158730017 learned_addr learned my addr 172.21.15.103:0/3158730017 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:05.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.354+0000 7f9b9effd640 1 --2- 172.21.15.103:0/3158730017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e42a30 0x55dc64e64470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.354+0000 7f9b9f7fe640 1 --2- 172.21.15.103:0/3158730017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e46dc0 0x55dc64eca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.354+0000 7f9b967fc640 1 -- 172.21.15.103:0/3158730017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e46dc0 msgr2=0x55dc64eca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:05.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.354+0000 7f9b967fc640 1 --2- 172.21.15.103:0/3158730017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e46dc0 0x55dc64eca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.354+0000 7f9b967fc640 1 -- 172.21.15.103:0/3158730017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e42a30 msgr2=0x55dc64e64470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:05.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.354+0000 7f9b967fc640 1 --2- 172.21.15.103:0/3158730017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e42a30 0x55dc64e64470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.354+0000 7f9b967fc640 1 -- 172.21.15.103:0/3158730017 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc64e65290 con 0x55dc64e464a0 2024-09-19T06:56:05.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.354+0000 7f9b9f7fe640 1 --2- 172.21.15.103:0/3158730017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e46dc0 0x55dc64eca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:05.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.354+0000 7f9b967fc640 1 --2- 172.21.15.103:0/3158730017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e464a0 0x55dc64e649b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9b8809a460 tx=0x7f9b88007f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11c596453f3597a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:05.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.354+0000 7f9b867fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:05.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.354+0000 7f9b867fc640 1 -- 172.21.15.103:0/3158730017 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b880a4e20 con 0x55dc64e464a0 2024-09-19T06:56:05.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.354+0000 7f9b867fc640 1 -- 172.21.15.103:0/3158730017 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8600+0+0 (secure 0 0 0) 0x7f9b88018d00 con 0x55dc64e464a0 2024-09-19T06:56:05.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.354+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3158730017 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b90048070 con 0x55dc64e464a0 2024-09-19T06:56:05.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.354+0000 7f9b867fc640 1 -- 172.21.15.103:0/3158730017 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9b8809c1b0 con 0x55dc64e464a0 2024-09-19T06:56:05.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.354+0000 7f9ba7b99740 10 client.17820 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:05.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.354+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3158730017 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dc64ecbe70 con 0x55dc64e464a0 2024-09-19T06:56:05.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.355+0000 7f9b867fc640 1 -- 172.21.15.103:0/3158730017 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b90048070 con 0x55dc64e464a0 2024-09-19T06:56:05.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.355+0000 7f9b867fc640 1 -- 172.21.15.103:0/3158730017 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9b880a4990 con 0x55dc64e464a0 2024-09-19T06:56:05.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.355+0000 7f9ba7b99740 10 client.17820 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:05.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.355+0000 7f9ba7b99740 10 client.17820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:05.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.355+0000 7f9ba7b99740 20 client.17820 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:05.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.355+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/3158730017 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dc64e51680 0x55dc64ece450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:05.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.355+0000 7f9ba7b99740 4 client.17820 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:56:05.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.355+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3158730017 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc64db3d60 con 0x55dc64e51680 2024-09-19T06:56:05.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.355+0000 7f9b9effd640 1 --2- 172.21.15.103:0/3158730017 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dc64e51680 0x55dc64ece450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:05.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.355+0000 7f9b9effd640 1 --2- 172.21.15.103:0/3158730017 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dc64e51680 0x55dc64ece450 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee5f15b38fc15e5 server_cookie=d69945aef6fa1e33 in_seq=0 out_seq=0 2024-09-19T06:56:05.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.355+0000 7f9b867fc640 10 client.17820 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:05.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.356+0000 7f9b867fc640 1 -- 172.21.15.103:0/3158730017 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dc64db3d60 con 0x55dc64e51680 2024-09-19T06:56:05.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.356+0000 7f9b867fc640 10 client.17820 handle_command_reply: tid=0 2024-09-19T06:56:05.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.356+0000 7f9ba7b99740 1 client.17820 shutdown 2024-09-19T06:56:05.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.356+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3158730017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e464a0 msgr2=0x55dc64e649b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:05.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.356+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/3158730017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e464a0 0x55dc64e649b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9b8809a460 tx=0x7f9b88007f80 comp rx=0 tx=0).stop 2024-09-19T06:56:05.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.356+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3158730017 shutdown_connections 2024-09-19T06:56:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.356+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/3158730017 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55dc64e51680 0x55dc64ece450 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.356+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/3158730017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc64e46dc0 0x55dc64eca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.356+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/3158730017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc64e464a0 0x55dc64e649b0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.356+0000 7f9ba7b99740 1 --2- 172.21.15.103:0/3158730017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc64e42a30 0x55dc64e64470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.356+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3158730017 >> 172.21.15.103:0/3158730017 conn(0x55dc64e49cb0 msgr2=0x55dc64e4ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.356+0000 7f9b867fc640 0 client.17820 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.356+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3158730017 shutdown_connections 2024-09-19T06:56:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.356+0000 7f9ba7b99740 1 -- 172.21.15.103:0/3158730017 wait complete. 2024-09-19T06:56:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.356+0000 7f9ba7b99740 20 client.17820 trim_cache size 0 max 16384 2024-09-19T06:56:05.364 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:56:05.365 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T06:56:05.365 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T06:56:05.365 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:56:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.357+0000 7f9ba57c0640 1 -- 172.21.15.103:0/4104732044 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9b8c0a33b0 msgr2=0x7f9b8c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.357+0000 7f9ba57c0640 1 --2- 172.21.15.103:0/4104732044 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9b8c0a33b0 0x7f9b8c0a5870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b98005c40 tx=0x7f9b980094a0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.357+0000 7f9ba57c0640 1 -- 172.21.15.103:0/4104732044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ba01036e0 msgr2=0x7f9ba0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.357+0000 7f9ba57c0640 1 --2- 172.21.15.103:0/4104732044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ba01036e0 0x7f9ba0100ed0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800c8c0 tx=0x7f9b8800cd80 comp rx=0 tx=0).stop 2024-09-19T06:56:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.357+0000 7f9ba57c0640 1 -- 172.21.15.103:0/4104732044 shutdown_connections 2024-09-19T06:56:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.357+0000 7f9ba57c0640 1 --2- 172.21.15.103:0/4104732044 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9b8c0a33b0 0x7f9b8c0a5870 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.357+0000 7f9ba57c0640 1 --2- 172.21.15.103:0/4104732044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ba01082a0 0x7f9ba0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.357+0000 7f9ba57c0640 1 --2- 172.21.15.103:0/4104732044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ba01036e0 0x7f9ba0100ed0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.357+0000 7f9ba57c0640 1 --2- 172.21.15.103:0/4104732044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ba0102d30 0x7f9ba0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.357+0000 7f9ba57c0640 1 -- 172.21.15.103:0/4104732044 >> 172.21.15.103:0/4104732044 conn(0x7f9ba00fc7d0 msgr2=0x7f9ba0105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.357+0000 7f9ba57c0640 1 -- 172.21.15.103:0/4104732044 shutdown_connections 2024-09-19T06:56:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:05.357+0000 7f9ba57c0640 1 -- 172.21.15.103:0/4104732044 wait complete. 2024-09-19T06:56:06.164 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:56:06.164 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:56:06.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:06 smithi103 ceph-mon[31158]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T06:56:06.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:06 smithi103 ceph-mon[31158]: pgmap v210: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-19T06:56:06.412 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:06.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:06 smithi134 ceph-mon[35356]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T06:56:06.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:06 smithi134 ceph-mon[35356]: pgmap v210: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-19T06:56:06.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:06 smithi174 ceph-mon[35637]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T06:56:06.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:06 smithi174 ceph-mon[35637]: pgmap v210: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-19T06:56:06.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.810+0000 7ff3da9a2640 1 -- 172.21.15.103:0/3833553658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3d4106f20 msgr2=0x7ff3d410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:06.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.810+0000 7ff3da9a2640 1 --2- 172.21.15.103:0/3833553658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3d4106f20 0x7ff3d410eaa0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009ca0 tx=0x7ff3cc02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.810+0000 7ff3da9a2640 1 -- 172.21.15.103:0/3833553658 shutdown_connections 2024-09-19T06:56:06.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.810+0000 7ff3da9a2640 1 --2- 172.21.15.103:0/3833553658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3d4106f20 0x7ff3d410eaa0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.810+0000 7ff3da9a2640 1 --2- 172.21.15.103:0/3833553658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3d41027a0 0x7ff3d41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.810+0000 7ff3da9a2640 1 --2- 172.21.15.103:0/3833553658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d4101df0 0x7ff3d41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.810+0000 7ff3da9a2640 1 -- 172.21.15.103:0/3833553658 >> 172.21.15.103:0/3833553658 conn(0x7ff3d40fd8b0 msgr2=0x7ff3d40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:06.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.810+0000 7ff3da9a2640 1 -- 172.21.15.103:0/3833553658 shutdown_connections 2024-09-19T06:56:06.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.810+0000 7ff3da9a2640 1 -- 172.21.15.103:0/3833553658 wait complete. 2024-09-19T06:56:06.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.811+0000 7ff3da9a2640 1 Processor -- start 2024-09-19T06:56:06.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.811+0000 7ff3da9a2640 1 -- start start 2024-09-19T06:56:06.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.811+0000 7ff3da9a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3d4101df0 0x7ff3d4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.811+0000 7ff3da9a2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d41027a0 0x7ff3d4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.811+0000 7ff3da9a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3d4106f20 0x7ff3d419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.811+0000 7ff3da9a2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d4112040 con 0x7ff3d4101df0 2024-09-19T06:56:06.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.811+0000 7ff3da9a2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d4111ec0 con 0x7ff3d41027a0 2024-09-19T06:56:06.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.811+0000 7ff3da9a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41121c0 con 0x7ff3d4106f20 2024-09-19T06:56:06.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.811+0000 7ff3d3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3d4101df0 0x7ff3d4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.811+0000 7ff3d3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3d4101df0 0x7ff3d4198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57268/0 (socket says 172.21.15.103:57268) 2024-09-19T06:56:06.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.812+0000 7ff3d3fff640 1 -- 172.21.15.103:0/3946590369 learned_addr learned my addr 172.21.15.103:0/3946590369 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:06.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.812+0000 7ff3d8f3f640 1 --2- 172.21.15.103:0/3946590369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3d4106f20 0x7ff3d419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.812+0000 7ff3cbfff640 1 --2- 172.21.15.103:0/3946590369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d41027a0 0x7ff3d4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.812+0000 7ff3d3fff640 1 -- 172.21.15.103:0/3946590369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3d4106f20 msgr2=0x7ff3d419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:06.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.812+0000 7ff3d3fff640 1 --2- 172.21.15.103:0/3946590369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3d4106f20 0x7ff3d419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.812+0000 7ff3d3fff640 1 -- 172.21.15.103:0/3946590369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d41027a0 msgr2=0x7ff3d4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:06.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.812+0000 7ff3d3fff640 1 --2- 172.21.15.103:0/3946590369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d41027a0 0x7ff3d4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.812+0000 7ff3d3fff640 1 -- 172.21.15.103:0/3946590369 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d419d4c0 con 0x7ff3d4101df0 2024-09-19T06:56:06.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.812+0000 7ff3d8f3f640 1 --2- 172.21.15.103:0/3946590369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3d4106f20 0x7ff3d419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:06.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.812+0000 7ff3cbfff640 1 --2- 172.21.15.103:0/3946590369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d41027a0 0x7ff3d4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:06.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.812+0000 7ff3d3fff640 1 --2- 172.21.15.103:0/3946590369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3d4101df0 0x7ff3d4198510 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400b4a0 tx=0x7ff3c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:06.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.812+0000 7ff3d1ffb640 1 -- 172.21.15.103:0/3946590369 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4014070 con 0x7ff3d4101df0 2024-09-19T06:56:06.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.812+0000 7ff3d1ffb640 1 -- 172.21.15.103:0/3946590369 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c4010af0 con 0x7ff3d4101df0 2024-09-19T06:56:06.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.812+0000 7ff3da9a2640 1 -- 172.21.15.103:0/3946590369 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d419d7b0 con 0x7ff3d4101df0 2024-09-19T06:56:06.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.812+0000 7ff3d1ffb640 1 -- 172.21.15.103:0/3946590369 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4017d10 con 0x7ff3d4101df0 2024-09-19T06:56:06.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.812+0000 7ff3da9a2640 1 -- 172.21.15.103:0/3946590369 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d41a5090 con 0x7ff3d4101df0 2024-09-19T06:56:06.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.813+0000 7ff3da9a2640 1 -- 172.21.15.103:0/3946590369 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d4102250 con 0x7ff3d4101df0 2024-09-19T06:56:06.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.814+0000 7ff3d1ffb640 1 -- 172.21.15.103:0/3946590369 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff3c4017070 con 0x7ff3d4101df0 2024-09-19T06:56:06.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.815+0000 7ff3d1ffb640 1 --2- 172.21.15.103:0/3946590369 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3c00a33b0 0x7ff3c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.815+0000 7ff3cbfff640 1 --2- 172.21.15.103:0/3946590369 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3c00a33b0 0x7ff3c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.815+0000 7ff3d1ffb640 1 -- 172.21.15.103:0/3946590369 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8600+0+0 (secure 0 0 0) 0x7ff3c40cf0e0 con 0x7ff3d4101df0 2024-09-19T06:56:06.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.816+0000 7ff3cbfff640 1 --2- 172.21.15.103:0/3946590369 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3c00a33b0 0x7ff3c00a5870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40ffdc0 tx=0x7ff3bc008040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:06.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:06.820+0000 7ff3d1ffb640 1 -- 172.21.15.103:0/3946590369 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff3c409bb80 con 0x7ff3d4101df0 2024-09-19T06:56:07.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:07.020+0000 7ff3da9a2640 1 -- 172.21.15.103:0/3946590369 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3d4199870 con 0x7ff3d4101df0 2024-09-19T06:56:07.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:07.021+0000 7ff3d1ffb640 1 -- 172.21.15.103:0/3946590369 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7ff3c409b480 con 0x7ff3d4101df0 2024-09-19T06:56:07.022 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:56:07.023 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:56:07.023 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:56:07.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:07.023+0000 7ff3da9a2640 1 -- 172.21.15.103:0/3946590369 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3c00a33b0 msgr2=0x7ff3c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:07.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:07.023+0000 7ff3da9a2640 1 --2- 172.21.15.103:0/3946590369 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3c00a33b0 0x7ff3c00a5870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40ffdc0 tx=0x7ff3bc008040 comp rx=0 tx=0).stop 2024-09-19T06:56:07.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:07.023+0000 7ff3da9a2640 1 -- 172.21.15.103:0/3946590369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3d4101df0 msgr2=0x7ff3d4198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:07.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:07.023+0000 7ff3da9a2640 1 --2- 172.21.15.103:0/3946590369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3d4101df0 0x7ff3d4198510 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400b4a0 tx=0x7ff3c400b960 comp rx=0 tx=0).stop 2024-09-19T06:56:07.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:07.023+0000 7ff3da9a2640 1 -- 172.21.15.103:0/3946590369 shutdown_connections 2024-09-19T06:56:07.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:07.023+0000 7ff3da9a2640 1 --2- 172.21.15.103:0/3946590369 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3c00a33b0 0x7ff3c00a5870 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:07.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:07.023+0000 7ff3da9a2640 1 --2- 172.21.15.103:0/3946590369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3d4106f20 0x7ff3d419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:07.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:07.023+0000 7ff3da9a2640 1 --2- 172.21.15.103:0/3946590369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d41027a0 0x7ff3d4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:07.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:07.023+0000 7ff3da9a2640 1 --2- 172.21.15.103:0/3946590369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3d4101df0 0x7ff3d4198510 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:07.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:07.023+0000 7ff3da9a2640 1 -- 172.21.15.103:0/3946590369 >> 172.21.15.103:0/3946590369 conn(0x7ff3d40fd8b0 msgr2=0x7ff3d40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:07.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:07.023+0000 7ff3da9a2640 1 -- 172.21.15.103:0/3946590369 shutdown_connections 2024-09-19T06:56:07.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:07.023+0000 7ff3da9a2640 1 -- 172.21.15.103:0/3946590369 wait complete. 2024-09-19T06:56:07.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:07 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3946590369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:07.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:07 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3946590369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:07.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:07 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3946590369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:07.639 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:56:07.885 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:08.279 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:08 smithi103 ceph-mon[31158]: pgmap v211: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-19T06:56:08.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.360+0000 7f47e7676640 1 -- 172.21.15.103:0/4264482473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e0106f10 msgr2=0x7f47e010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:08.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.360+0000 7f47e7676640 1 --2- 172.21.15.103:0/4264482473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e0106f10 0x7f47e010ea90 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009ca0 tx=0x7f47d802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.360+0000 7f47e7676640 1 -- 172.21.15.103:0/4264482473 shutdown_connections 2024-09-19T06:56:08.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.360+0000 7f47e7676640 1 --2- 172.21.15.103:0/4264482473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e0106f10 0x7f47e010ea90 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.360+0000 7f47e7676640 1 --2- 172.21.15.103:0/4264482473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e0102790 0x7f47e01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.360+0000 7f47e7676640 1 --2- 172.21.15.103:0/4264482473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47e0101de0 0x7f47e01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.360+0000 7f47e7676640 1 -- 172.21.15.103:0/4264482473 >> 172.21.15.103:0/4264482473 conn(0x7f47e00fd8c0 msgr2=0x7f47e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:08.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.360+0000 7f47e7676640 1 -- 172.21.15.103:0/4264482473 shutdown_connections 2024-09-19T06:56:08.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.360+0000 7f47e7676640 1 -- 172.21.15.103:0/4264482473 wait complete. 2024-09-19T06:56:08.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.361+0000 7f47e7676640 1 Processor -- start 2024-09-19T06:56:08.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.361+0000 7f47e7676640 1 -- start start 2024-09-19T06:56:08.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.361+0000 7f47e7676640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47e0101de0 0x7f47e0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.361+0000 7f47e7676640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e0102790 0x7f47e01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.361+0000 7f47e7676640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e0106f10 0x7f47e019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.361+0000 7f47e7676640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f47e0111f90 con 0x7f47e0102790 2024-09-19T06:56:08.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.361+0000 7f47e7676640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47e0111e10 con 0x7f47e0101de0 2024-09-19T06:56:08.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.361+0000 7f47e7676640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47e0112110 con 0x7f47e0106f10 2024-09-19T06:56:08.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.361+0000 7f47e4c11640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e0102790 0x7f47e01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:08.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.361+0000 7f47e4c11640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e0102790 0x7f47e01989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60728/0 (socket says 172.21.15.103:60728) 2024-09-19T06:56:08.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.361+0000 7f47e4c11640 1 -- 172.21.15.103:0/4260211985 learned_addr learned my addr 172.21.15.103:0/4260211985 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:08.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.361+0000 7f47e4c11640 1 -- 172.21.15.103:0/4260211985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e0106f10 msgr2=0x7f47e019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:08.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.361+0000 7f47e4c11640 1 --2- 172.21.15.103:0/4260211985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e0106f10 0x7f47e019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.361+0000 7f47e4c11640 1 -- 172.21.15.103:0/4260211985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47e0101de0 msgr2=0x7f47e0198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:08.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.361+0000 7f47e4c11640 1 --2- 172.21.15.103:0/4260211985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47e0101de0 0x7f47e0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.361+0000 7f47e4c11640 1 -- 172.21.15.103:0/4260211985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e019d420 con 0x7f47e0102790 2024-09-19T06:56:08.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.362+0000 7f47e4c11640 1 --2- 172.21.15.103:0/4260211985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e0102790 0x7f47e01989b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f47d000b4a0 tx=0x7f47d000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:08.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.362+0000 7f47d67fc640 1 -- 172.21.15.103:0/4260211985 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d0014070 con 0x7f47e0102790 2024-09-19T06:56:08.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.362+0000 7f47e7676640 1 -- 172.21.15.103:0/4260211985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e019d710 con 0x7f47e0102790 2024-09-19T06:56:08.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.362+0000 7f47d67fc640 1 -- 172.21.15.103:0/4260211985 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d0010af0 con 0x7f47e0102790 2024-09-19T06:56:08.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.362+0000 7f47d67fc640 1 -- 172.21.15.103:0/4260211985 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d0017d10 con 0x7f47e0102790 2024-09-19T06:56:08.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.362+0000 7f47e7676640 1 -- 172.21.15.103:0/4260211985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e01a4ff0 con 0x7f47e0102790 2024-09-19T06:56:08.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.364+0000 7f47d67fc640 1 -- 172.21.15.103:0/4260211985 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f47d0017070 con 0x7f47e0102790 2024-09-19T06:56:08.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.364+0000 7f47e9a4f740 1 Processor -- start 2024-09-19T06:56:08.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.364+0000 7f47e9a4f740 1 -- start start 2024-09-19T06:56:08.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e9a4f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8abb40 0x55ea0b8aa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e9a4f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e9a4f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8b5370 0x55ea0b8b7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e9a4f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ea0b8102f0 con 0x55ea0b8aa5d0 2024-09-19T06:56:08.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e9a4f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ea0b7b7cf0 con 0x55ea0b8abb40 2024-09-19T06:56:08.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e9a4f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ea0b80c2b0 con 0x55ea0b8b5370 2024-09-19T06:56:08.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47d67fc640 1 --2- 172.21.15.103:0/4260211985 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47cc0a33b0 0x7f47cc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e5412640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:08.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e5412640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8aaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60744/0 (socket says 172.21.15.103:60744) 2024-09-19T06:56:08.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e5412640 1 -- 172.21.15.103:0/3595857424 learned_addr learned my addr 172.21.15.103:0/3595857424 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:08.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e5c13640 1 --2- 172.21.15.103:0/3595857424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8abb40 0x55ea0b8aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:08.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e5c13640 1 --2- 172.21.15.103:0/4260211985 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47cc0a33b0 0x7f47cc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:08.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47d67fc640 1 -- 172.21.15.103:0/4260211985 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8600+0+0 (secure 0 0 0) 0x7f47d00cf150 con 0x7f47e0102790 2024-09-19T06:56:08.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e4c11640 1 --2- 172.21.15.103:0/3595857424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8b5370 0x55ea0b8b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:08.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e5412640 1 -- 172.21.15.103:0/3595857424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8b5370 msgr2=0x55ea0b8b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:08.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e5412640 1 --2- 172.21.15.103:0/3595857424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8b5370 0x55ea0b8b7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e5412640 1 -- 172.21.15.103:0/3595857424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8abb40 msgr2=0x55ea0b8aa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:08.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e5412640 1 --2- 172.21.15.103:0/3595857424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8abb40 0x55ea0b8aa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e5412640 1 -- 172.21.15.103:0/3595857424 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0b79c3a0 con 0x55ea0b8aa5d0 2024-09-19T06:56:08.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e5c13640 1 --2- 172.21.15.103:0/3595857424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8abb40 0x55ea0b8aa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:56:08.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e5412640 1 --2- 172.21.15.103:0/3595857424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8aaa10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00bb10 tx=0x7f47dc00d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbc7ab8f749892c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:08.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.365+0000 7f47e5c13640 1 --2- 172.21.15.103:0/4260211985 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47cc0a33b0 0x7f47cc0a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f47d802ade0 tx=0x7f47d80095c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:08.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.366+0000 7f47cb7fe640 1 -- 172.21.15.103:0/3595857424 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc016070 con 0x55ea0b8aa5d0 2024-09-19T06:56:08.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.366+0000 7f47cb7fe640 1 -- 172.21.15.103:0/3595857424 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47dc005500 con 0x55ea0b8aa5d0 2024-09-19T06:56:08.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.366+0000 7f47cb7fe640 1 -- 172.21.15.103:0/3595857424 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc010460 con 0x55ea0b8aa5d0 2024-09-19T06:56:08.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.366+0000 7f47e9a4f740 1 -- 172.21.15.103:0/3595857424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8aa5d0 msgr2=0x55ea0b8aaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:08.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.366+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/3595857424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8aaa10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00bb10 tx=0x7f47dc00d2e0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.366+0000 7f47e9a4f740 1 -- 172.21.15.103:0/3595857424 shutdown_connections 2024-09-19T06:56:08.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.366+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/3595857424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8b5370 0x55ea0b8b7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.366+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/3595857424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8aaa10 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.366+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/3595857424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8abb40 0x55ea0b8aa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.366+0000 7f47e9a4f740 1 -- 172.21.15.103:0/3595857424 >> 172.21.15.103:0/3595857424 conn(0x55ea0b873310 msgr2=0x55ea0b8a6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:08.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.366+0000 7f47e9a4f740 1 -- 172.21.15.103:0/3595857424 shutdown_connections 2024-09-19T06:56:08.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.366+0000 7f47e9a4f740 1 -- 172.21.15.103:0/3595857424 wait complete. 2024-09-19T06:56:08.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.367+0000 7f47e9a4f740 1 -- start start 2024-09-19T06:56:08.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.367+0000 7f47e9a4f740 1 Processor -- start 2024-09-19T06:56:08.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.367+0000 7f47e9a4f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.367+0000 7f47e9a4f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8abb40 0x55ea0b8b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.367+0000 7f47e9a4f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8b5370 0x55ea0b8b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.367+0000 7f47e9a4f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ea0b8b8070 con 0x55ea0b8abb40 2024-09-19T06:56:08.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.367+0000 7f47e9a4f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ea0b7cb060 con 0x55ea0b8b5370 2024-09-19T06:56:08.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.367+0000 7f47e9a4f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ea0b8b81f0 con 0x55ea0b8aa5d0 2024-09-19T06:56:08.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.367+0000 7f47e9a4f740 10 client.0 _ll_register_callbacks cb 0x55ea0b51e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:08.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.367+0000 7f47e4c11640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8abb40 0x55ea0b8b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:08.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47e4c11640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8abb40 0x55ea0b8b1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60752/0 (socket says 172.21.15.103:60752) 2024-09-19T06:56:08.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47e4c11640 1 -- 172.21.15.103:0/2924139031 learned_addr learned my addr 172.21.15.103:0/2924139031 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:08.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47e5412640 1 --2- 172.21.15.103:0/2924139031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:08.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47e5c13640 1 --2- 172.21.15.103:0/2924139031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8b5370 0x55ea0b8b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:08.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47e4c11640 1 -- 172.21.15.103:0/2924139031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8aa5d0 msgr2=0x55ea0b8b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:08.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47e4c11640 1 --2- 172.21.15.103:0/2924139031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47e4c11640 1 -- 172.21.15.103:0/2924139031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8b5370 msgr2=0x55ea0b8b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:08.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47e4c11640 1 --2- 172.21.15.103:0/2924139031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8b5370 0x55ea0b8b24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47e4c11640 1 -- 172.21.15.103:0/2924139031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0b5813c0 con 0x55ea0b8abb40 2024-09-19T06:56:08.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47e5412640 1 --2- 172.21.15.103:0/2924139031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:08.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47e4c11640 1 --2- 172.21.15.103:0/2924139031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8abb40 0x55ea0b8b1f70 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f47d00cce50 tx=0x7f47d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c033b93b86c05357 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:08.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:08.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47c8ff9640 1 -- 172.21.15.103:0/2924139031 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d00b4060 con 0x55ea0b8abb40 2024-09-19T06:56:08.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47c8ff9640 1 -- 172.21.15.103:0/2924139031 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8600+0+0 (secure 0 0 0) 0x7f47d00ccad0 con 0x55ea0b8abb40 2024-09-19T06:56:08.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47e9a4f740 1 -- 172.21.15.103:0/2924139031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47dc005500 con 0x55ea0b8abb40 2024-09-19T06:56:08.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47c8ff9640 1 -- 172.21.15.103:0/2924139031 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f47d001e040 con 0x55ea0b8abb40 2024-09-19T06:56:08.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47e9a4f740 10 client.17838 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:08.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47e9a4f740 1 -- 172.21.15.103:0/2924139031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea0b8c3a80 con 0x55ea0b8abb40 2024-09-19T06:56:08.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.368+0000 7f47c8ff9640 1 -- 172.21.15.103:0/2924139031 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47dc005500 con 0x55ea0b8abb40 2024-09-19T06:56:08.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.369+0000 7f47c8ff9640 1 -- 172.21.15.103:0/2924139031 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f47d00b4200 con 0x55ea0b8abb40 2024-09-19T06:56:08.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.369+0000 7f47e9a4f740 10 client.17838 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:08.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.369+0000 7f47e9a4f740 10 client.17838 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:56:08.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.369+0000 7f47e9a4f740 20 client.17838 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:08.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.369+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/2924139031 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55ea0b8c4f40 0x55ea0b8c7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.369+0000 7f47e9a4f740 4 client.17838 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:08.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.369+0000 7f47e9a4f740 1 -- 172.21.15.103:0/2924139031 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea0b81d860 con 0x55ea0b8c4f40 2024-09-19T06:56:08.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.369+0000 7f47e5412640 1 --2- 172.21.15.103:0/2924139031 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55ea0b8c4f40 0x55ea0b8c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:08.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.370+0000 7f47e5412640 1 --2- 172.21.15.103:0/2924139031 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55ea0b8c4f40 0x55ea0b8c7350 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8637abe3c90f6f7a server_cookie=3c25e858c8db75b9 in_seq=0 out_seq=0 2024-09-19T06:56:08.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.370+0000 7f47c8ff9640 10 client.17838 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:56:08.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.370+0000 7f47c8ff9640 1 -- 172.21.15.103:0/2924139031 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea0b81d860 con 0x55ea0b8c4f40 2024-09-19T06:56:08.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.370+0000 7f47c8ff9640 10 client.17838 handle_command_reply: tid=0 2024-09-19T06:56:08.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.370+0000 7f47e9a4f740 1 client.17838 shutdown 2024-09-19T06:56:08.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.370+0000 7f47e9a4f740 1 -- 172.21.15.103:0/2924139031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8abb40 msgr2=0x55ea0b8b1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:08.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.370+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/2924139031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8abb40 0x55ea0b8b1f70 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f47d00cce50 tx=0x7f47d00046c0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.371+0000 7f47e9a4f740 1 -- 172.21.15.103:0/2924139031 shutdown_connections 2024-09-19T06:56:08.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.371+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/2924139031 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55ea0b8c4f40 0x55ea0b8c7350 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.371+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/2924139031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8b5370 0x55ea0b8b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.371+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/2924139031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8abb40 0x55ea0b8b1f70 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.371+0000 7f47c8ff9640 0 client.17838 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:56:08.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.371+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/2924139031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.371+0000 7f47e9a4f740 1 -- 172.21.15.103:0/2924139031 >> 172.21.15.103:0/2924139031 conn(0x55ea0b873310 msgr2=0x55ea0b8ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:08.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.371+0000 7f47e9a4f740 1 -- 172.21.15.103:0/2924139031 shutdown_connections 2024-09-19T06:56:08.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.371+0000 7f47e9a4f740 1 -- 172.21.15.103:0/2924139031 wait complete. 2024-09-19T06:56:08.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.371+0000 7f47e9a4f740 20 client.17838 trim_cache size 0 max 16384 2024-09-19T06:56:08.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.383+0000 7f47e9a4f740 1 Processor -- start 2024-09-19T06:56:08.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.383+0000 7f47e9a4f740 1 -- start start 2024-09-19T06:56:08.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e9a4f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8c1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e9a4f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8abb40 0x55ea0b8b1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e9a4f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8b5370 0x55ea0b8bd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e9a4f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f47e01adab0 con 0x55ea0b8abb40 2024-09-19T06:56:08.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e9a4f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47e01af5f0 con 0x55ea0b8b5370 2024-09-19T06:56:08.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e9a4f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47e01ae180 con 0x55ea0b8aa5d0 2024-09-19T06:56:08.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e4c11640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8abb40 0x55ea0b8b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:08.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e4c11640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8abb40 0x55ea0b8b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60764/0 (socket says 172.21.15.103:60764) 2024-09-19T06:56:08.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e4c11640 1 -- 172.21.15.103:0/2326482306 learned_addr learned my addr 172.21.15.103:0/2326482306 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:08.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e5412640 1 --2- 172.21.15.103:0/2326482306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8c1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:08.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e4c11640 1 -- 172.21.15.103:0/2326482306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8aa5d0 msgr2=0x55ea0b8c1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:08.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e4c11640 1 --2- 172.21.15.103:0/2326482306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8c1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e5c13640 1 --2- 172.21.15.103:0/2326482306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8b5370 0x55ea0b8bd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:08.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e4c11640 1 -- 172.21.15.103:0/2326482306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8b5370 msgr2=0x55ea0b8bd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:08.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e4c11640 1 --2- 172.21.15.103:0/2326482306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8b5370 0x55ea0b8bd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e4c11640 1 -- 172.21.15.103:0/2326482306 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0b8a6de0 con 0x55ea0b8abb40 2024-09-19T06:56:08.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e5412640 1 --2- 172.21.15.103:0/2326482306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8c1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:08.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47e4c11640 1 --2- 172.21.15.103:0/2326482306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8abb40 0x55ea0b8b1830 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f47d009a960 tx=0x7f47d00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c242d8fe5da9d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:08.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47caffd640 1 -- 172.21.15.103:0/2326482306 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d00b4040 con 0x55ea0b8abb40 2024-09-19T06:56:08.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47caffd640 1 -- 172.21.15.103:0/2326482306 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d0010ca0 con 0x55ea0b8abb40 2024-09-19T06:56:08.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.384+0000 7f47caffd640 1 -- 172.21.15.103:0/2326482306 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d0097cf0 con 0x55ea0b8abb40 2024-09-19T06:56:08.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.385+0000 7f47e9a4f740 1 -- 172.21.15.103:0/2326482306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8abb40 msgr2=0x55ea0b8b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:08.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.385+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/2326482306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8abb40 0x55ea0b8b1830 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f47d009a960 tx=0x7f47d00047d0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.385+0000 7f47e9a4f740 1 -- 172.21.15.103:0/2326482306 shutdown_connections 2024-09-19T06:56:08.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.385+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/2326482306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8b5370 0x55ea0b8bd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.385+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/2326482306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8abb40 0x55ea0b8b1830 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.385+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/2326482306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8c1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.385+0000 7f47e9a4f740 1 -- 172.21.15.103:0/2326482306 >> 172.21.15.103:0/2326482306 conn(0x55ea0b873310 msgr2=0x55ea0b8b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:08.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.385+0000 7f47e9a4f740 1 -- 172.21.15.103:0/2326482306 shutdown_connections 2024-09-19T06:56:08.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.385+0000 7f47e9a4f740 1 -- 172.21.15.103:0/2326482306 wait complete. 2024-09-19T06:56:08.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e9a4f740 1 -- start start 2024-09-19T06:56:08.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e9a4f740 1 Processor -- start 2024-09-19T06:56:08.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e9a4f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8ae200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e9a4f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8abb40 0x55ea0b8ae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e9a4f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8b5370 0x55ea0b8aec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e9a4f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f47e01aef20 con 0x55ea0b8aa5d0 2024-09-19T06:56:08.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e9a4f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47e01ae850 con 0x55ea0b8b5370 2024-09-19T06:56:08.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e9a4f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47e01abab0 con 0x55ea0b8abb40 2024-09-19T06:56:08.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e9a4f740 10 client.0 _ll_register_callbacks cb 0x7f47d00970d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:08.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e5412640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:08.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e5412640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8ae200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60774/0 (socket says 172.21.15.103:60774) 2024-09-19T06:56:08.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e5412640 1 -- 172.21.15.103:0/4021253233 learned_addr learned my addr 172.21.15.103:0/4021253233 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:08.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e5412640 1 -- 172.21.15.103:0/4021253233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8abb40 msgr2=0x55ea0b8ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:08.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e5412640 1 --2- 172.21.15.103:0/4021253233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8abb40 0x55ea0b8ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e5412640 1 -- 172.21.15.103:0/4021253233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8b5370 msgr2=0x55ea0b8aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:08.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e5412640 1 --2- 172.21.15.103:0/4021253233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8b5370 0x55ea0b8aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.386+0000 7f47e5412640 1 -- 172.21.15.103:0/4021253233 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0b8af3c0 con 0x55ea0b8aa5d0 2024-09-19T06:56:08.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.387+0000 7f47e5412640 1 --2- 172.21.15.103:0/4021253233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8ae200 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00ba30 tx=0x7f47dc005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db083fbf09049b5f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:08.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.387+0000 7f47c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:08.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.387+0000 7f47c8ff9640 1 -- 172.21.15.103:0/4021253233 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc016040 con 0x55ea0b8aa5d0 2024-09-19T06:56:08.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.387+0000 7f47c8ff9640 1 -- 172.21.15.103:0/4021253233 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8600+0+0 (secure 0 0 0) 0x7f47dc0057a0 con 0x55ea0b8aa5d0 2024-09-19T06:56:08.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.387+0000 7f47e9a4f740 1 -- 172.21.15.103:0/4021253233 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47d0010ca0 con 0x55ea0b8aa5d0 2024-09-19T06:56:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.387+0000 7f47c8ff9640 1 -- 172.21.15.103:0/4021253233 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f47dc00dce0 con 0x55ea0b8aa5d0 2024-09-19T06:56:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.387+0000 7f47e9a4f740 10 client.17844 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.387+0000 7f47e9a4f740 1 -- 172.21.15.103:0/4021253233 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea0b934150 con 0x55ea0b8aa5d0 2024-09-19T06:56:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.387+0000 7f47c8ff9640 1 -- 172.21.15.103:0/4021253233 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d0010ca0 con 0x55ea0b8aa5d0 2024-09-19T06:56:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.387+0000 7f47c8ff9640 1 -- 172.21.15.103:0/4021253233 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f47dc005260 con 0x55ea0b8aa5d0 2024-09-19T06:56:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.387+0000 7f47e9a4f740 10 client.17844 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.387+0000 7f47e9a4f740 10 client.17844 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:56:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.387+0000 7f47e9a4f740 20 client.17844 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.387+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/4021253233 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55ea0b934550 0x55ea0b936940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.387+0000 7f47e9a4f740 4 client.17844 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:56:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.387+0000 7f47e9a4f740 1 -- 172.21.15.103:0/4021253233 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea0b817d70 con 0x55ea0b934550 2024-09-19T06:56:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.388+0000 7f47e4c11640 1 --2- 172.21.15.103:0/4021253233 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55ea0b934550 0x55ea0b936940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.388+0000 7f47e4c11640 1 --2- 172.21.15.103:0/4021253233 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55ea0b934550 0x55ea0b936940 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d255bcfcac813eba server_cookie=7a0729f5708c86e1 in_seq=0 out_seq=0 2024-09-19T06:56:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.388+0000 7f47c8ff9640 10 client.17844 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:56:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.389+0000 7f47c8ff9640 1 -- 172.21.15.103:0/4021253233 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea0b817d70 con 0x55ea0b934550 2024-09-19T06:56:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.389+0000 7f47c8ff9640 10 client.17844 handle_command_reply: tid=0 2024-09-19T06:56:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.389+0000 7f47e9a4f740 1 client.17844 shutdown 2024-09-19T06:56:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.389+0000 7f47e9a4f740 1 -- 172.21.15.103:0/4021253233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8aa5d0 msgr2=0x55ea0b8ae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.389+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/4021253233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8ae200 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00ba30 tx=0x7f47dc005730 comp rx=0 tx=0).stop 2024-09-19T06:56:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.389+0000 7f47e9a4f740 1 -- 172.21.15.103:0/4021253233 shutdown_connections 2024-09-19T06:56:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.389+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/4021253233 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55ea0b934550 0x55ea0b936940 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.389+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/4021253233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea0b8b5370 0x55ea0b8aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.389+0000 7f47c8ff9640 0 client.17844 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:56:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.389+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/4021253233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea0b8abb40 0x55ea0b8ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.389+0000 7f47e9a4f740 1 --2- 172.21.15.103:0/4021253233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea0b8aa5d0 0x55ea0b8ae200 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.389+0000 7f47e9a4f740 1 -- 172.21.15.103:0/4021253233 >> 172.21.15.103:0/4021253233 conn(0x55ea0b873310 msgr2=0x55ea0b8a54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.389+0000 7f47e9a4f740 1 -- 172.21.15.103:0/4021253233 shutdown_connections 2024-09-19T06:56:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.389+0000 7f47e9a4f740 1 -- 172.21.15.103:0/4021253233 wait complete. 2024-09-19T06:56:08.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.389+0000 7f47e9a4f740 20 client.17844 trim_cache size 0 max 16384 2024-09-19T06:56:08.395 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:56:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.390+0000 7f47e7676640 1 -- 172.21.15.103:0/4260211985 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47cc0a33b0 msgr2=0x7f47cc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.390+0000 7f47e7676640 1 --2- 172.21.15.103:0/4260211985 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47cc0a33b0 0x7f47cc0a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f47d802ade0 tx=0x7f47d80095c0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.390+0000 7f47e7676640 1 -- 172.21.15.103:0/4260211985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e0102790 msgr2=0x7f47e01989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.390+0000 7f47e7676640 1 --2- 172.21.15.103:0/4260211985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e0102790 0x7f47e01989b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f47d000b4a0 tx=0x7f47d000b960 comp rx=0 tx=0).stop 2024-09-19T06:56:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.390+0000 7f47e7676640 1 -- 172.21.15.103:0/4260211985 shutdown_connections 2024-09-19T06:56:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.390+0000 7f47e7676640 1 --2- 172.21.15.103:0/4260211985 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47cc0a33b0 0x7f47cc0a5870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.390+0000 7f47e7676640 1 --2- 172.21.15.103:0/4260211985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e0106f10 0x7f47e019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.390+0000 7f47e7676640 1 --2- 172.21.15.103:0/4260211985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e0102790 0x7f47e01989b0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.390+0000 7f47e7676640 1 --2- 172.21.15.103:0/4260211985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47e0101de0 0x7f47e0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.390+0000 7f47e7676640 1 -- 172.21.15.103:0/4260211985 >> 172.21.15.103:0/4260211985 conn(0x7f47e00fd8c0 msgr2=0x7f47e00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.390+0000 7f47e7676640 1 -- 172.21.15.103:0/4260211985 shutdown_connections 2024-09-19T06:56:08.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:08.390+0000 7f47e7676640 1 -- 172.21.15.103:0/4260211985 wait complete. 2024-09-19T06:56:08.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:08 smithi134 ceph-mon[35356]: pgmap v211: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-19T06:56:08.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:08 smithi174 ceph-mon[35637]: pgmap v211: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-19T06:56:09.106 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:56:09.351 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:09.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.763+0000 7f6416863640 1 -- 172.21.15.103:0/4284226027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6410106a80 msgr2=0x7f6410106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:09.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.763+0000 7f6416863640 1 --2- 172.21.15.103:0/4284226027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6410106a80 0x7f6410106e60 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6400009930 tx=0x7f640002a420 comp rx=0 tx=0).stop 2024-09-19T06:56:09.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.763+0000 7f6416863640 1 -- 172.21.15.103:0/4284226027 shutdown_connections 2024-09-19T06:56:09.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.763+0000 7f6416863640 1 --2- 172.21.15.103:0/4284226027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6410101420 0x7f641010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.763+0000 7f6416863640 1 --2- 172.21.15.103:0/4284226027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6410100a80 0x7f6410100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.763+0000 7f6416863640 1 --2- 172.21.15.103:0/4284226027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6410106a80 0x7f6410106e60 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.763+0000 7f6416863640 1 -- 172.21.15.103:0/4284226027 >> 172.21.15.103:0/4284226027 conn(0x7f64100fc7a0 msgr2=0x7f64100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:09.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.763+0000 7f6416863640 1 -- 172.21.15.103:0/4284226027 shutdown_connections 2024-09-19T06:56:09.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.764+0000 7f6416863640 1 -- 172.21.15.103:0/4284226027 wait complete. 2024-09-19T06:56:09.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.764+0000 7f6416863640 1 Processor -- start 2024-09-19T06:56:09.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.764+0000 7f6416863640 1 -- start start 2024-09-19T06:56:09.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f6416863640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6410100a80 0x7f641019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f6416863640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6410101420 0x7f641019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f6416863640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6410106a80 0x7f641019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f6416863640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6410111fa0 con 0x7f6410100a80 2024-09-19T06:56:09.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f6416863640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6410111e20 con 0x7f6410101420 2024-09-19T06:56:09.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f6416863640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6410112120 con 0x7f6410106a80 2024-09-19T06:56:09.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f6414e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6410106a80 0x7f641019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:09.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f640ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6410100a80 0x7f641019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:09.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f6414e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6410106a80 0x7f641019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:59836/0 (socket says 172.21.15.103:59836) 2024-09-19T06:56:09.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f6414e00640 1 -- 172.21.15.103:0/3002175640 learned_addr learned my addr 172.21.15.103:0/3002175640 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:09.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f640f7fe640 1 --2- 172.21.15.103:0/3002175640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6410101420 0x7f641019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:09.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f640ffff640 1 -- 172.21.15.103:0/3002175640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6410106a80 msgr2=0x7f641019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:09.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f640ffff640 1 --2- 172.21.15.103:0/3002175640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6410106a80 0x7f641019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f640ffff640 1 -- 172.21.15.103:0/3002175640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6410101420 msgr2=0x7f641019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:09.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f640ffff640 1 --2- 172.21.15.103:0/3002175640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6410101420 0x7f641019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f640ffff640 1 -- 172.21.15.103:0/3002175640 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641019f620 con 0x7f6410100a80 2024-09-19T06:56:09.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f6414e00640 1 --2- 172.21.15.103:0/3002175640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6410106a80 0x7f641019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:56:09.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f640f7fe640 1 --2- 172.21.15.103:0/3002175640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6410101420 0x7f641019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:56:09.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.765+0000 7f640ffff640 1 --2- 172.21.15.103:0/3002175640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6410100a80 0x7f641019a670 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6400005940 tx=0x7f64000040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:09.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.766+0000 7f640d7fa640 1 -- 172.21.15.103:0/3002175640 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640003f070 con 0x7f6410100a80 2024-09-19T06:56:09.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.766+0000 7f6416863640 1 -- 172.21.15.103:0/3002175640 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641019f8b0 con 0x7f6410100a80 2024-09-19T06:56:09.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.766+0000 7f640d7fa640 1 -- 172.21.15.103:0/3002175640 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64000044c0 con 0x7f6410100a80 2024-09-19T06:56:09.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.766+0000 7f640d7fa640 1 -- 172.21.15.103:0/3002175640 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400035070 con 0x7f6410100a80 2024-09-19T06:56:09.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.766+0000 7f6416863640 1 -- 172.21.15.103:0/3002175640 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64101a71f0 con 0x7f6410100a80 2024-09-19T06:56:09.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.768+0000 7f640d7fa640 1 -- 172.21.15.103:0/3002175640 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f640003b460 con 0x7f6410100a80 2024-09-19T06:56:09.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.768+0000 7f6418c3c740 1 Processor -- start 2024-09-19T06:56:09.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.768+0000 7f6418c3c740 1 -- start start 2024-09-19T06:56:09.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.768+0000 7f6418c3c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a54b40 0x563763a53090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.768+0000 7f6418c3c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a535d0 0x563763a53a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.768+0000 7f6418c3c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a5e370 0x563763a60760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.768+0000 7f6418c3c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5637639b92f0 con 0x563763a54b40 2024-09-19T06:56:09.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.768+0000 7f6418c3c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563763960cf0 con 0x563763a535d0 2024-09-19T06:56:09.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.768+0000 7f6418c3c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5637639b52b0 con 0x563763a5e370 2024-09-19T06:56:09.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.768+0000 7f6414e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a54b40 0x563763a53090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:09.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.769+0000 7f6414e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a54b40 0x563763a53090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60790/0 (socket says 172.21.15.103:60790) 2024-09-19T06:56:09.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.769+0000 7f6414e00640 1 -- 172.21.15.103:0/341551705 learned_addr learned my addr 172.21.15.103:0/341551705 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:09.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.769+0000 7f640d7fa640 1 --2- 172.21.15.103:0/3002175640 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f63f80a33b0 0x7f63f80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.769+0000 7f6414e00640 1 -- 172.21.15.103:0/341551705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a5e370 msgr2=0x563763a60760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:09.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.769+0000 7f6414e00640 1 --2- 172.21.15.103:0/341551705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a5e370 0x563763a60760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.769+0000 7f6414e00640 1 -- 172.21.15.103:0/341551705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a535d0 msgr2=0x563763a53a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:09.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.769+0000 7f6414e00640 1 --2- 172.21.15.103:0/341551705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a535d0 0x563763a53a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.769+0000 7f6414e00640 1 -- 172.21.15.103:0/341551705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637639453a0 con 0x563763a54b40 2024-09-19T06:56:09.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.769+0000 7f640d7fa640 1 -- 172.21.15.103:0/3002175640 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8600+0+0 (secure 0 0 0) 0x7f64000f0120 con 0x7f6410100a80 2024-09-19T06:56:09.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.769+0000 7f6414e00640 1 --2- 172.21.15.103:0/3002175640 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f63f80a33b0 0x7f63f80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:09.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.769+0000 7f6414e00640 1 --2- 172.21.15.103:0/341551705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a54b40 0x563763a53090 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f6408010990 tx=0x7f6408010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a78f6cded49773 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:09.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.769+0000 7f63f67fc640 1 -- 172.21.15.103:0/341551705 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408024070 con 0x563763a54b40 2024-09-19T06:56:09.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.769+0000 7f6414e00640 1 --2- 172.21.15.103:0/3002175640 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f63f80a33b0 0x7f63f80a5870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6408021000 tx=0x7f640801e000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:09.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.769+0000 7f63f67fc640 1 -- 172.21.15.103:0/341551705 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f640801eb60 con 0x563763a54b40 2024-09-19T06:56:09.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.769+0000 7f63f67fc640 1 -- 172.21.15.103:0/341551705 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408011a70 con 0x563763a54b40 2024-09-19T06:56:09.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.770+0000 7f6418c3c740 1 -- 172.21.15.103:0/341551705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a54b40 msgr2=0x563763a53090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:09.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.770+0000 7f6418c3c740 1 --2- 172.21.15.103:0/341551705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a54b40 0x563763a53090 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f6408010990 tx=0x7f6408010e50 comp rx=0 tx=0).stop 2024-09-19T06:56:09.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.770+0000 7f6418c3c740 1 -- 172.21.15.103:0/341551705 shutdown_connections 2024-09-19T06:56:09.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.770+0000 7f6418c3c740 1 --2- 172.21.15.103:0/341551705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a5e370 0x563763a60760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.770+0000 7f6418c3c740 1 --2- 172.21.15.103:0/341551705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a535d0 0x563763a53a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.770+0000 7f6418c3c740 1 --2- 172.21.15.103:0/341551705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a54b40 0x563763a53090 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.770+0000 7f6418c3c740 1 -- 172.21.15.103:0/341551705 >> 172.21.15.103:0/341551705 conn(0x563763a1c310 msgr2=0x563763a4fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:09.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.770+0000 7f6418c3c740 1 -- 172.21.15.103:0/341551705 shutdown_connections 2024-09-19T06:56:09.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.770+0000 7f6418c3c740 1 -- 172.21.15.103:0/341551705 wait complete. 2024-09-19T06:56:09.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.770+0000 7f6418c3c740 1 -- start start 2024-09-19T06:56:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f6418c3c740 1 Processor -- start 2024-09-19T06:56:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f6418c3c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a535d0 0x563763a5ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f6418c3c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a54b40 0x563763a5b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f6418c3c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a5e370 0x563763a5b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f6418c3c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563763a61070 con 0x563763a535d0 2024-09-19T06:56:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f6418c3c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563763974060 con 0x563763a54b40 2024-09-19T06:56:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f6418c3c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563763a611f0 con 0x563763a5e370 2024-09-19T06:56:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f6418c3c740 10 client.0 _ll_register_callbacks cb 0x5637636c78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f640ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a535d0 0x563763a5ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f640ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a535d0 0x563763a5ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60792/0 (socket says 172.21.15.103:60792) 2024-09-19T06:56:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f640ffff640 1 -- 172.21.15.103:0/2382982638 learned_addr learned my addr 172.21.15.103:0/2382982638 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f640f7fe640 1 --2- 172.21.15.103:0/2382982638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a54b40 0x563763a5b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f640ffff640 1 -- 172.21.15.103:0/2382982638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a5e370 msgr2=0x563763a5b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f6414e00640 1 --2- 172.21.15.103:0/2382982638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a5e370 0x563763a5b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f640ffff640 1 --2- 172.21.15.103:0/2382982638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a5e370 0x563763a5b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f640ffff640 1 -- 172.21.15.103:0/2382982638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a54b40 msgr2=0x563763a5b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f640ffff640 1 --2- 172.21.15.103:0/2382982638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a54b40 0x563763a5b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f640ffff640 1 -- 172.21.15.103:0/2382982638 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56376372a3c0 con 0x563763a535d0 2024-09-19T06:56:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.771+0000 7f640f7fe640 1 --2- 172.21.15.103:0/2382982638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a54b40 0x563763a5b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.772+0000 7f640ffff640 1 --2- 172.21.15.103:0/2382982638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a535d0 0x563763a5ddc0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f64000bc490 tx=0x7f64000bc100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278bdbceffe922b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.772+0000 7f63ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.772+0000 7f63ef7fe640 1 -- 172.21.15.103:0/2382982638 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400048040 con 0x563763a535d0 2024-09-19T06:56:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.772+0000 7f63ef7fe640 1 -- 172.21.15.103:0/2382982638 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8600+0+0 (secure 0 0 0) 0x7f64000c4c30 con 0x563763a535d0 2024-09-19T06:56:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.772+0000 7f6418c3c740 1 -- 172.21.15.103:0/2382982638 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f640801eb60 con 0x563763a535d0 2024-09-19T06:56:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.772+0000 7f63ef7fe640 1 -- 172.21.15.103:0/2382982638 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f64000c0800 con 0x563763a535d0 2024-09-19T06:56:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.772+0000 7f6418c3c740 10 client.17856 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.772+0000 7f6418c3c740 1 -- 172.21.15.103:0/2382982638 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563763a6cb80 con 0x563763a535d0 2024-09-19T06:56:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.772+0000 7f63ef7fe640 1 -- 172.21.15.103:0/2382982638 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f640801eb60 con 0x563763a535d0 2024-09-19T06:56:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.772+0000 7f63ef7fe640 1 -- 172.21.15.103:0/2382982638 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f640004b060 con 0x563763a535d0 2024-09-19T06:56:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.772+0000 7f6418c3c740 10 client.17856 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.772+0000 7f6418c3c740 10 client.17856 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:56:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.772+0000 7f6418c3c740 20 client.17856 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.772+0000 7f6418c3c740 1 --2- 172.21.15.103:0/2382982638 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x563763a6ded0 0x563763a702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.772+0000 7f6418c3c740 4 client.17856 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.772+0000 7f6418c3c740 1 -- 172.21.15.103:0/2382982638 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637639c6860 con 0x563763a6ded0 2024-09-19T06:56:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.773+0000 7f640f7fe640 1 --2- 172.21.15.103:0/2382982638 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x563763a6ded0 0x563763a702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.773+0000 7f63ef7fe640 10 client.17856 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:56:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.773+0000 7f640f7fe640 1 --2- 172.21.15.103:0/2382982638 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x563763a6ded0 0x563763a702c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ed2947522d71591b server_cookie=c9d98556a81f2e0a in_seq=0 out_seq=0 2024-09-19T06:56:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.774+0000 7f63ef7fe640 1 -- 172.21.15.103:0/2382982638 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5637639c6860 con 0x563763a6ded0 2024-09-19T06:56:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.774+0000 7f63ef7fe640 10 client.17856 handle_command_reply: tid=0 2024-09-19T06:56:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.774+0000 7f6418c3c740 1 client.17856 shutdown 2024-09-19T06:56:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.774+0000 7f6418c3c740 1 -- 172.21.15.103:0/2382982638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a535d0 msgr2=0x563763a5ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:09.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.774+0000 7f6418c3c740 1 --2- 172.21.15.103:0/2382982638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a535d0 0x563763a5ddc0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f64000bc490 tx=0x7f64000bc100 comp rx=0 tx=0).stop 2024-09-19T06:56:09.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.774+0000 7f6418c3c740 1 -- 172.21.15.103:0/2382982638 shutdown_connections 2024-09-19T06:56:09.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.774+0000 7f6418c3c740 1 --2- 172.21.15.103:0/2382982638 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x563763a6ded0 0x563763a702c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.774+0000 7f6418c3c740 1 --2- 172.21.15.103:0/2382982638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a5e370 0x563763a5b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.774+0000 7f6418c3c740 1 --2- 172.21.15.103:0/2382982638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a54b40 0x563763a5b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.774+0000 7f6418c3c740 1 --2- 172.21.15.103:0/2382982638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a535d0 0x563763a5ddc0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.774+0000 7f63ef7fe640 0 client.17856 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:56:09.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.774+0000 7f6418c3c740 1 -- 172.21.15.103:0/2382982638 >> 172.21.15.103:0/2382982638 conn(0x563763a1c310 msgr2=0x563763a55960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:09.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.774+0000 7f6418c3c740 1 -- 172.21.15.103:0/2382982638 shutdown_connections 2024-09-19T06:56:09.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.774+0000 7f6418c3c740 1 -- 172.21.15.103:0/2382982638 wait complete. 2024-09-19T06:56:09.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.774+0000 7f6418c3c740 20 client.17856 trim_cache size 0 max 16384 2024-09-19T06:56:09.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.787+0000 7f6418c3c740 1 Processor -- start 2024-09-19T06:56:09.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.787+0000 7f6418c3c740 1 -- start start 2024-09-19T06:56:09.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f6418c3c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a535d0 0x563763a56120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f6418c3c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a54b40 0x563763a56660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f6418c3c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a5e370 0x563763a75b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f6418c3c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f64101afb00 con 0x563763a5e370 2024-09-19T06:56:09.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f6418c3c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64101b1640 con 0x563763a54b40 2024-09-19T06:56:09.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f6418c3c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f64101b01d0 con 0x563763a535d0 2024-09-19T06:56:09.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f6414e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a5e370 0x563763a75b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:09.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f6414e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a5e370 0x563763a75b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60802/0 (socket says 172.21.15.103:60802) 2024-09-19T06:56:09.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f6414e00640 1 -- 172.21.15.103:0/939584878 learned_addr learned my addr 172.21.15.103:0/939584878 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:09.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f640f7fe640 1 --2- 172.21.15.103:0/939584878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a54b40 0x563763a56660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:09.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f640ffff640 1 --2- 172.21.15.103:0/939584878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a535d0 0x563763a56120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:09.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f6414e00640 1 -- 172.21.15.103:0/939584878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a535d0 msgr2=0x563763a56120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:09.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f6414e00640 1 --2- 172.21.15.103:0/939584878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a535d0 0x563763a56120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f6414e00640 1 -- 172.21.15.103:0/939584878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a54b40 msgr2=0x563763a56660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:09.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f6414e00640 1 --2- 172.21.15.103:0/939584878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a54b40 0x563763a56660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f6414e00640 1 -- 172.21.15.103:0/939584878 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563763a60ad0 con 0x563763a5e370 2024-09-19T06:56:09.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.788+0000 7f6414e00640 1 --2- 172.21.15.103:0/939584878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a5e370 0x563763a75b20 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f640800f010 tx=0x7f640800e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0245e6006f44b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:09.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.789+0000 7f63f5ffb640 1 -- 172.21.15.103:0/939584878 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408024040 con 0x563763a5e370 2024-09-19T06:56:09.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.789+0000 7f63f5ffb640 1 -- 172.21.15.103:0/939584878 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6408013270 con 0x563763a5e370 2024-09-19T06:56:09.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.789+0000 7f63f5ffb640 1 -- 172.21.15.103:0/939584878 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640801ed60 con 0x563763a5e370 2024-09-19T06:56:09.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.789+0000 7f6418c3c740 1 -- 172.21.15.103:0/939584878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a5e370 msgr2=0x563763a75b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:09.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.789+0000 7f6418c3c740 1 --2- 172.21.15.103:0/939584878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a5e370 0x563763a75b20 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f640800f010 tx=0x7f640800e9f0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.789+0000 7f6418c3c740 1 -- 172.21.15.103:0/939584878 shutdown_connections 2024-09-19T06:56:09.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.789+0000 7f6418c3c740 1 --2- 172.21.15.103:0/939584878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a5e370 0x563763a75b20 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.789+0000 7f6418c3c740 1 --2- 172.21.15.103:0/939584878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a54b40 0x563763a56660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.789+0000 7f6418c3c740 1 --2- 172.21.15.103:0/939584878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a535d0 0x563763a56120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.789+0000 7f6418c3c740 1 -- 172.21.15.103:0/939584878 >> 172.21.15.103:0/939584878 conn(0x563763a1c310 msgr2=0x563763a4ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:09.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.789+0000 7f6418c3c740 1 -- 172.21.15.103:0/939584878 shutdown_connections 2024-09-19T06:56:09.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.789+0000 7f6418c3c740 1 -- 172.21.15.103:0/939584878 wait complete. 2024-09-19T06:56:09.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.790+0000 7f6418c3c740 1 -- start start 2024-09-19T06:56:09.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.790+0000 7f6418c3c740 1 Processor -- start 2024-09-19T06:56:09.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.790+0000 7f6418c3c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a535d0 0x563763a66480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.790+0000 7f6418c3c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a54b40 0x563763a669c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.790+0000 7f6418c3c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a5e370 0x563763a63450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.790+0000 7f6418c3c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f64101b0f70 con 0x563763a54b40 2024-09-19T06:56:09.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.790+0000 7f6418c3c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64101b08a0 con 0x563763a5e370 2024-09-19T06:56:09.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.790+0000 7f6418c3c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f64101adb00 con 0x563763a535d0 2024-09-19T06:56:09.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.790+0000 7f6418c3c740 10 client.0 _ll_register_callbacks cb 0x7f6400005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:09.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.790+0000 7f640f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a54b40 0x563763a669c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:09.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.790+0000 7f640f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a54b40 0x563763a669c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60804/0 (socket says 172.21.15.103:60804) 2024-09-19T06:56:09.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.790+0000 7f640f7fe640 1 -- 172.21.15.103:0/2441025671 learned_addr learned my addr 172.21.15.103:0/2441025671 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:09.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.791+0000 7f640f7fe640 1 -- 172.21.15.103:0/2441025671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a535d0 msgr2=0x563763a66480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:09.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.791+0000 7f640f7fe640 1 --2- 172.21.15.103:0/2441025671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a535d0 0x563763a66480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.791+0000 7f640f7fe640 1 -- 172.21.15.103:0/2441025671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a5e370 msgr2=0x563763a63450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:09.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.791+0000 7f640f7fe640 1 --2- 172.21.15.103:0/2441025671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a5e370 0x563763a63450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.791+0000 7f640f7fe640 1 -- 172.21.15.103:0/2441025671 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563763a63990 con 0x563763a54b40 2024-09-19T06:56:09.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.791+0000 7f640f7fe640 1 --2- 172.21.15.103:0/2441025671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a54b40 0x563763a669c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x563763a5a900 tx=0x7f6404013910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40f8c406145fff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:09.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.791+0000 7f63ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:09.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.791+0000 7f63ef7fe640 1 -- 172.21.15.103:0/2441025671 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404023030 con 0x563763a54b40 2024-09-19T06:56:09.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.791+0000 7f63ef7fe640 1 -- 172.21.15.103:0/2441025671 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8600+0+0 (secure 0 0 0) 0x7f640401f5a0 con 0x563763a54b40 2024-09-19T06:56:09.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.791+0000 7f6418c3c740 1 -- 172.21.15.103:0/2441025671 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6408013270 con 0x563763a54b40 2024-09-19T06:56:09.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.791+0000 7f63ef7fe640 1 -- 172.21.15.103:0/2441025671 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f640401f7a0 con 0x563763a54b40 2024-09-19T06:56:09.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.791+0000 7f6418c3c740 10 client.17862 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:09.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.791+0000 7f6418c3c740 1 -- 172.21.15.103:0/2441025671 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563763ad9340 con 0x563763a54b40 2024-09-19T06:56:09.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.792+0000 7f63ef7fe640 1 -- 172.21.15.103:0/2441025671 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6408013270 con 0x563763a54b40 2024-09-19T06:56:09.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.792+0000 7f63ef7fe640 1 -- 172.21.15.103:0/2441025671 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6404010bb0 con 0x563763a54b40 2024-09-19T06:56:09.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.792+0000 7f6418c3c740 10 client.17862 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:09.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.792+0000 7f6418c3c740 10 client.17862 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:56:09.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.792+0000 7f6418c3c740 20 client.17862 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:09.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.792+0000 7f6418c3c740 1 --2- 172.21.15.103:0/2441025671 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x563763ad9af0 0x563763adbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:09.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.792+0000 7f6418c3c740 4 client.17862 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:56:09.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.792+0000 7f6418c3c740 1 -- 172.21.15.103:0/2441025671 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637639c0d70 con 0x563763ad9af0 2024-09-19T06:56:09.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.792+0000 7f640ffff640 1 --2- 172.21.15.103:0/2441025671 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x563763ad9af0 0x563763adbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:09.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f63ef7fe640 10 client.17862 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:56:09.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f640ffff640 1 --2- 172.21.15.103:0/2441025671 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x563763ad9af0 0x563763adbf70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fcceb77fdbab6074 server_cookie=bbe8ce1ef2c83fe6 in_seq=0 out_seq=0 2024-09-19T06:56:09.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f63ef7fe640 1 -- 172.21.15.103:0/2441025671 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637639c0d70 con 0x563763ad9af0 2024-09-19T06:56:09.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f63ef7fe640 10 client.17862 handle_command_reply: tid=0 2024-09-19T06:56:09.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f6418c3c740 1 client.17862 shutdown 2024-09-19T06:56:09.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f6418c3c740 1 -- 172.21.15.103:0/2441025671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a54b40 msgr2=0x563763a669c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:09.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f6418c3c740 1 --2- 172.21.15.103:0/2441025671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a54b40 0x563763a669c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x563763a5a900 tx=0x7f6404013910 comp rx=0 tx=0).stop 2024-09-19T06:56:09.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f6418c3c740 1 -- 172.21.15.103:0/2441025671 shutdown_connections 2024-09-19T06:56:09.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f6418c3c740 1 --2- 172.21.15.103:0/2441025671 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x563763ad9af0 0x563763adbf70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f63ef7fe640 0 client.17862 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:56:09.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f6418c3c740 1 --2- 172.21.15.103:0/2441025671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563763a5e370 0x563763a63450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f6418c3c740 1 --2- 172.21.15.103:0/2441025671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563763a54b40 0x563763a669c0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f6418c3c740 1 --2- 172.21.15.103:0/2441025671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563763a535d0 0x563763a66480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f6418c3c740 1 -- 172.21.15.103:0/2441025671 >> 172.21.15.103:0/2441025671 conn(0x563763a1c310 msgr2=0x563763a69300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:09.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f6418c3c740 1 -- 172.21.15.103:0/2441025671 shutdown_connections 2024-09-19T06:56:09.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f6418c3c740 1 -- 172.21.15.103:0/2441025671 wait complete. 2024-09-19T06:56:09.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.793+0000 7f6418c3c740 20 client.17862 trim_cache size 0 max 16384 2024-09-19T06:56:09.800 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:56:09.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.794+0000 7f6416863640 1 -- 172.21.15.103:0/3002175640 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f63f80a33b0 msgr2=0x7f63f80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.794+0000 7f6416863640 1 --2- 172.21.15.103:0/3002175640 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f63f80a33b0 0x7f63f80a5870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6408021000 tx=0x7f640801e000 comp rx=0 tx=0).stop 2024-09-19T06:56:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.794+0000 7f6416863640 1 -- 172.21.15.103:0/3002175640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6410100a80 msgr2=0x7f641019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.794+0000 7f6416863640 1 --2- 172.21.15.103:0/3002175640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6410100a80 0x7f641019a670 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6400005940 tx=0x7f64000040c0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.794+0000 7f6416863640 1 -- 172.21.15.103:0/3002175640 shutdown_connections 2024-09-19T06:56:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.794+0000 7f6416863640 1 --2- 172.21.15.103:0/3002175640 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f63f80a33b0 0x7f63f80a5870 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.794+0000 7f6416863640 1 --2- 172.21.15.103:0/3002175640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6410106a80 0x7f641019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.794+0000 7f6416863640 1 --2- 172.21.15.103:0/3002175640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6410101420 0x7f641019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.794+0000 7f6416863640 1 --2- 172.21.15.103:0/3002175640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6410100a80 0x7f641019a670 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.794+0000 7f6416863640 1 -- 172.21.15.103:0/3002175640 >> 172.21.15.103:0/3002175640 conn(0x7f64100fc7a0 msgr2=0x7f64100fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.794+0000 7f6416863640 1 -- 172.21.15.103:0/3002175640 shutdown_connections 2024-09-19T06:56:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:09.794+0000 7f6416863640 1 -- 172.21.15.103:0/3002175640 wait complete. 2024-09-19T06:56:10.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:10 smithi103 ceph-mon[31158]: pgmap v212: 97 pgs: 97 active+clean; 4.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 100 KiB/s wr, 36 op/s 2024-09-19T06:56:10.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:10 smithi134 ceph-mon[35356]: pgmap v212: 97 pgs: 97 active+clean; 4.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 100 KiB/s wr, 36 op/s 2024-09-19T06:56:10.523 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:56:10.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:10 smithi174 ceph-mon[35637]: pgmap v212: 97 pgs: 97 active+clean; 4.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 100 KiB/s wr, 36 op/s 2024-09-19T06:56:10.770 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:11.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.189+0000 7fa486e82640 1 -- 172.21.15.103:0/387327266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa480100a80 msgr2=0x7fa480100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:11.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.189+0000 7fa486e82640 1 --2- 172.21.15.103:0/387327266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa480100a80 0x7fa480100ee0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009930 tx=0x7fa46c02a420 comp rx=0 tx=0).stop 2024-09-19T06:56:11.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.189+0000 7fa486e82640 1 -- 172.21.15.103:0/387327266 shutdown_connections 2024-09-19T06:56:11.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.189+0000 7fa486e82640 1 --2- 172.21.15.103:0/387327266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa480101420 0x7fa48010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.189+0000 7fa486e82640 1 --2- 172.21.15.103:0/387327266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa480100a80 0x7fa480100ee0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.189+0000 7fa486e82640 1 --2- 172.21.15.103:0/387327266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa480106a80 0x7fa480106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.189+0000 7fa486e82640 1 -- 172.21.15.103:0/387327266 >> 172.21.15.103:0/387327266 conn(0x7fa4800fc7a0 msgr2=0x7fa4800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:11.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.189+0000 7fa486e82640 1 -- 172.21.15.103:0/387327266 shutdown_connections 2024-09-19T06:56:11.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.189+0000 7fa486e82640 1 -- 172.21.15.103:0/387327266 wait complete. 2024-09-19T06:56:11.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.189+0000 7fa486e82640 1 Processor -- start 2024-09-19T06:56:11.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.190+0000 7fa486e82640 1 -- start start 2024-09-19T06:56:11.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.190+0000 7fa486e82640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa480100a80 0x7fa480198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.190+0000 7fa486e82640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa480101420 0x7fa4801989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.190+0000 7fa486e82640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa480106a80 0x7fa48019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.190+0000 7fa486e82640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa480111fe0 con 0x7fa480106a80 2024-09-19T06:56:11.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.190+0000 7fa486e82640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa480111e60 con 0x7fa480100a80 2024-09-19T06:56:11.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.190+0000 7fa486e82640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa480112160 con 0x7fa480101420 2024-09-19T06:56:11.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.190+0000 7fa48541f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa480106a80 0x7fa48019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.190+0000 7fa48541f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa480106a80 0x7fa48019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60826/0 (socket says 172.21.15.103:60826) 2024-09-19T06:56:11.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.190+0000 7fa48541f640 1 -- 172.21.15.103:0/2382935248 learned_addr learned my addr 172.21.15.103:0/2382935248 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:11.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.190+0000 7fa48541f640 1 -- 172.21.15.103:0/2382935248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa480101420 msgr2=0x7fa4801989a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T06:56:11.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.190+0000 7fa48541f640 1 --2- 172.21.15.103:0/2382935248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa480101420 0x7fa4801989a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.190+0000 7fa48541f640 1 -- 172.21.15.103:0/2382935248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa480100a80 msgr2=0x7fa480198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:11.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.190+0000 7fa48541f640 1 --2- 172.21.15.103:0/2382935248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa480100a80 0x7fa480198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.190+0000 7fa48541f640 1 -- 172.21.15.103:0/2382935248 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48019d410 con 0x7fa480106a80 2024-09-19T06:56:11.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.190+0000 7fa48541f640 1 --2- 172.21.15.103:0/2382935248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa480106a80 0x7fa48019cd30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fa47800eaa0 tx=0x7fa47800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:11.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.191+0000 7fa47dffb640 1 -- 172.21.15.103:0/2382935248 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47800cdb0 con 0x7fa480106a80 2024-09-19T06:56:11.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.191+0000 7fa486e82640 1 -- 172.21.15.103:0/2382935248 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48019d700 con 0x7fa480106a80 2024-09-19T06:56:11.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.191+0000 7fa47dffb640 1 -- 172.21.15.103:0/2382935248 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa478005020 con 0x7fa480106a80 2024-09-19T06:56:11.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.191+0000 7fa47dffb640 1 -- 172.21.15.103:0/2382935248 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4780116b0 con 0x7fa480106a80 2024-09-19T06:56:11.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.191+0000 7fa486e82640 1 -- 172.21.15.103:0/2382935248 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4801a4fe0 con 0x7fa480106a80 2024-09-19T06:56:11.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.193+0000 7fa47dffb640 1 -- 172.21.15.103:0/2382935248 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa478005560 con 0x7fa480106a80 2024-09-19T06:56:11.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.193+0000 7fa48925b740 1 Processor -- start 2024-09-19T06:56:11.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.193+0000 7fa47dffb640 1 --2- 172.21.15.103:0/2382935248 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa4700a3400 0x7fa4700a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.193+0000 7fa48925b740 1 -- start start 2024-09-19T06:56:11.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa47ffff640 1 --2- 172.21.15.103:0/2382935248 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa4700a3400 0x7fa4700a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa47dffb640 1 -- 172.21.15.103:0/2382935248 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8600+0+0 (secure 0 0 0) 0x7fa4780d00f0 con 0x7fa480106a80 2024-09-19T06:56:11.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa48925b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b927b40 0x55585b926090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa48925b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b9265d0 0x55585b926a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa48925b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b931370 0x55585b933760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa48925b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55585b88c2f0 con 0x55585b931370 2024-09-19T06:56:11.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa48925b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55585b833cf0 con 0x55585b927b40 2024-09-19T06:56:11.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa48925b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55585b8882b0 con 0x55585b9265d0 2024-09-19T06:56:11.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa48541f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b931370 0x55585b933760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa48541f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b931370 0x55585b933760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60832/0 (socket says 172.21.15.103:60832) 2024-09-19T06:56:11.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa48541f640 1 -- 172.21.15.103:0/2087232497 learned_addr learned my addr 172.21.15.103:0/2087232497 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:11.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa47ffff640 1 --2- 172.21.15.103:0/2382935248 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa4700a3400 0x7fa4700a58c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa46c005c20 tx=0x7fa46c005fd0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:11.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa484c1e640 1 --2- 172.21.15.103:0/2087232497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b927b40 0x55585b926090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa48541f640 1 -- 172.21.15.103:0/2087232497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b9265d0 msgr2=0x55585b926a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:11.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa48541f640 1 --2- 172.21.15.103:0/2087232497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b9265d0 0x55585b926a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa48541f640 1 -- 172.21.15.103:0/2087232497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b927b40 msgr2=0x55585b926090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:11.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa48541f640 1 --2- 172.21.15.103:0/2087232497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b927b40 0x55585b926090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa48541f640 1 -- 172.21.15.103:0/2087232497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55585b8183a0 con 0x55585b931370 2024-09-19T06:56:11.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa48541f640 1 --2- 172.21.15.103:0/2087232497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b931370 0x55585b933760 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fa4780a4a60 tx=0x7fa47809bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89256427c74655c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa46affd640 1 -- 172.21.15.103:0/2087232497 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47801f030 con 0x55585b931370 2024-09-19T06:56:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa46affd640 1 -- 172.21.15.103:0/2087232497 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4780cda10 con 0x55585b931370 2024-09-19T06:56:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.194+0000 7fa46affd640 1 -- 172.21.15.103:0/2087232497 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4780bc420 con 0x55585b931370 2024-09-19T06:56:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.195+0000 7fa48925b740 1 -- 172.21.15.103:0/2087232497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b931370 msgr2=0x55585b933760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.195+0000 7fa48925b740 1 --2- 172.21.15.103:0/2087232497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b931370 0x55585b933760 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fa4780a4a60 tx=0x7fa47809bdb0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.195+0000 7fa48925b740 1 -- 172.21.15.103:0/2087232497 shutdown_connections 2024-09-19T06:56:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.195+0000 7fa48925b740 1 --2- 172.21.15.103:0/2087232497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b931370 0x55585b933760 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.195+0000 7fa48925b740 1 --2- 172.21.15.103:0/2087232497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b9265d0 0x55585b926a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.195+0000 7fa48925b740 1 --2- 172.21.15.103:0/2087232497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b927b40 0x55585b926090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.195+0000 7fa48925b740 1 -- 172.21.15.103:0/2087232497 >> 172.21.15.103:0/2087232497 conn(0x55585b8ef310 msgr2=0x55585b922c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.195+0000 7fa48925b740 1 -- 172.21.15.103:0/2087232497 shutdown_connections 2024-09-19T06:56:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.195+0000 7fa48925b740 1 -- 172.21.15.103:0/2087232497 wait complete. 2024-09-19T06:56:11.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.195+0000 7fa48925b740 1 -- start start 2024-09-19T06:56:11.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa48925b740 1 Processor -- start 2024-09-19T06:56:11.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa48925b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b9265d0 0x55585b930df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa48925b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b927b40 0x55585b92e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa48925b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b931370 0x55585b92e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa48925b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55585b934070 con 0x55585b927b40 2024-09-19T06:56:11.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa48925b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55585b847060 con 0x55585b931370 2024-09-19T06:56:11.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa48925b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55585b9341f0 con 0x55585b9265d0 2024-09-19T06:56:11.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa48925b740 10 client.0 _ll_register_callbacks cb 0x55585b59a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:11.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa47ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b927b40 0x55585b92e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa47ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b927b40 0x55585b92e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60834/0 (socket says 172.21.15.103:60834) 2024-09-19T06:56:11.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa47ffff640 1 -- 172.21.15.103:0/2319218311 learned_addr learned my addr 172.21.15.103:0/2319218311 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:11.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa484c1e640 1 --2- 172.21.15.103:0/2319218311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b9265d0 0x55585b930df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa47ffff640 1 -- 172.21.15.103:0/2319218311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b9265d0 msgr2=0x55585b930df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:11.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa47ffff640 1 --2- 172.21.15.103:0/2319218311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b9265d0 0x55585b930df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa47ffff640 1 -- 172.21.15.103:0/2319218311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b931370 msgr2=0x55585b92e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa47ffff640 1 --2- 172.21.15.103:0/2319218311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b931370 0x55585b92e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.196+0000 7fa47ffff640 1 -- 172.21.15.103:0/2319218311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55585b5fd3c0 con 0x55585b927b40 2024-09-19T06:56:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.197+0000 7fa47ffff640 1 --2- 172.21.15.103:0/2319218311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b927b40 0x55585b92e0a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009930 tx=0x7fa46c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12ccb70e31108b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.197+0000 7fa45ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.197+0000 7fa45ffff640 1 -- 172.21.15.103:0/2319218311 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c040070 con 0x55585b927b40 2024-09-19T06:56:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.197+0000 7fa48925b740 1 -- 172.21.15.103:0/2319218311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4780cda10 con 0x55585b927b40 2024-09-19T06:56:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.197+0000 7fa45ffff640 1 -- 172.21.15.103:0/2319218311 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8600+0+0 (secure 0 0 0) 0x7fa46c0315b0 con 0x55585b927b40 2024-09-19T06:56:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.197+0000 7fa45ffff640 1 -- 172.21.15.103:0/2319218311 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa46c055050 con 0x55585b927b40 2024-09-19T06:56:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.197+0000 7fa48925b740 10 client.17874 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.197+0000 7fa48925b740 1 -- 172.21.15.103:0/2319218311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55585b93fba0 con 0x55585b927b40 2024-09-19T06:56:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.197+0000 7fa45ffff640 1 -- 172.21.15.103:0/2319218311 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4780cda10 con 0x55585b927b40 2024-09-19T06:56:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.197+0000 7fa45ffff640 1 -- 172.21.15.103:0/2319218311 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa46c0478e0 con 0x55585b927b40 2024-09-19T06:56:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.197+0000 7fa48925b740 10 client.17874 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.197+0000 7fa48925b740 10 client.17874 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:56:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.197+0000 7fa48925b740 20 client.17874 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.197+0000 7fa48925b740 1 --2- 172.21.15.103:0/2319218311 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55585b941100 0x55585b943540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.197+0000 7fa48925b740 4 client.17874 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.197+0000 7fa48925b740 1 -- 172.21.15.103:0/2319218311 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55585b899860 con 0x55585b941100 2024-09-19T06:56:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.198+0000 7fa484c1e640 1 --2- 172.21.15.103:0/2319218311 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55585b941100 0x55585b943540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.198+0000 7fa45ffff640 10 client.17874 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:56:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.198+0000 7fa484c1e640 1 --2- 172.21.15.103:0/2319218311 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55585b941100 0x55585b943540 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90b168a99bd95d75 server_cookie=7217a5a8949cd97b in_seq=0 out_seq=0 2024-09-19T06:56:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.199+0000 7fa45ffff640 1 -- 172.21.15.103:0/2319218311 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55585b899860 con 0x55585b941100 2024-09-19T06:56:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.199+0000 7fa45ffff640 10 client.17874 handle_command_reply: tid=0 2024-09-19T06:56:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.199+0000 7fa48925b740 1 client.17874 shutdown 2024-09-19T06:56:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.199+0000 7fa48925b740 1 -- 172.21.15.103:0/2319218311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b927b40 msgr2=0x55585b92e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.199+0000 7fa48925b740 1 --2- 172.21.15.103:0/2319218311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b927b40 0x55585b92e0a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009930 tx=0x7fa46c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.199+0000 7fa48925b740 1 -- 172.21.15.103:0/2319218311 shutdown_connections 2024-09-19T06:56:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.199+0000 7fa48925b740 1 --2- 172.21.15.103:0/2319218311 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55585b941100 0x55585b943540 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.199+0000 7fa45ffff640 0 client.17874 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:56:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.199+0000 7fa48925b740 1 --2- 172.21.15.103:0/2319218311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b931370 0x55585b92e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.199+0000 7fa48925b740 1 --2- 172.21.15.103:0/2319218311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b927b40 0x55585b92e0a0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.199+0000 7fa48925b740 1 --2- 172.21.15.103:0/2319218311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b9265d0 0x55585b930df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.199+0000 7fa48925b740 1 -- 172.21.15.103:0/2319218311 >> 172.21.15.103:0/2319218311 conn(0x55585b8ef310 msgr2=0x55585b928990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.199+0000 7fa48925b740 1 -- 172.21.15.103:0/2319218311 shutdown_connections 2024-09-19T06:56:11.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.199+0000 7fa48925b740 1 -- 172.21.15.103:0/2319218311 wait complete. 2024-09-19T06:56:11.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.199+0000 7fa48925b740 20 client.17874 trim_cache size 0 max 16384 2024-09-19T06:56:11.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.212+0000 7fa48925b740 1 Processor -- start 2024-09-19T06:56:11.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.212+0000 7fa48925b740 1 -- start start 2024-09-19T06:56:11.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.212+0000 7fa48925b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b9265d0 0x55585b93d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.212+0000 7fa48925b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b927b40 0x55585b929180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.212+0000 7fa48925b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b931370 0x55585b9296c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.212+0000 7fa48925b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4801adab0 con 0x55585b9265d0 2024-09-19T06:56:11.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.212+0000 7fa48925b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa4801af5f0 con 0x55585b927b40 2024-09-19T06:56:11.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.212+0000 7fa48925b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4801ae180 con 0x55585b931370 2024-09-19T06:56:11.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.212+0000 7fa484c1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b9265d0 0x55585b93d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.212+0000 7fa484c1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b9265d0 0x55585b93d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60846/0 (socket says 172.21.15.103:60846) 2024-09-19T06:56:11.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.212+0000 7fa484c1e640 1 -- 172.21.15.103:0/3536363331 learned_addr learned my addr 172.21.15.103:0/3536363331 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:11.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.213+0000 7fa48541f640 1 --2- 172.21.15.103:0/3536363331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b931370 0x55585b9296c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.213+0000 7fa484c1e640 1 -- 172.21.15.103:0/3536363331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b931370 msgr2=0x55585b9296c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:11.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.213+0000 7fa484c1e640 1 --2- 172.21.15.103:0/3536363331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b931370 0x55585b9296c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.213+0000 7fa484c1e640 1 -- 172.21.15.103:0/3536363331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b927b40 msgr2=0x55585b929180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.213+0000 7fa484c1e640 1 --2- 172.21.15.103:0/3536363331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b927b40 0x55585b929180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.213+0000 7fa484c1e640 1 -- 172.21.15.103:0/3536363331 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55585b92eb50 con 0x55585b9265d0 2024-09-19T06:56:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.213+0000 7fa48541f640 1 --2- 172.21.15.103:0/3536363331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b931370 0x55585b9296c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.213+0000 7fa484c1e640 1 --2- 172.21.15.103:0/3536363331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b9265d0 0x55585b93d310 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fa474005fd0 tx=0x7fa474005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d79a7ac45f923d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.213+0000 7fa46a7fc640 1 -- 172.21.15.103:0/3536363331 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474004220 con 0x55585b9265d0 2024-09-19T06:56:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.213+0000 7fa46a7fc640 1 -- 172.21.15.103:0/3536363331 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4740043c0 con 0x55585b9265d0 2024-09-19T06:56:11.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.213+0000 7fa46a7fc640 1 -- 172.21.15.103:0/3536363331 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47401aa10 con 0x55585b9265d0 2024-09-19T06:56:11.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 -- 172.21.15.103:0/3536363331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b9265d0 msgr2=0x55585b93d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:11.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 --2- 172.21.15.103:0/3536363331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b9265d0 0x55585b93d310 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fa474005fd0 tx=0x7fa474005de0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 -- 172.21.15.103:0/3536363331 shutdown_connections 2024-09-19T06:56:11.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 --2- 172.21.15.103:0/3536363331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b931370 0x55585b9296c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 --2- 172.21.15.103:0/3536363331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b927b40 0x55585b929180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 --2- 172.21.15.103:0/3536363331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b9265d0 0x55585b93d310 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 -- 172.21.15.103:0/3536363331 >> 172.21.15.103:0/3536363331 conn(0x55585b8ef310 msgr2=0x55585b92d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:11.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 -- 172.21.15.103:0/3536363331 shutdown_connections 2024-09-19T06:56:11.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 -- 172.21.15.103:0/3536363331 wait complete. 2024-09-19T06:56:11.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 -- start start 2024-09-19T06:56:11.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 Processor -- start 2024-09-19T06:56:11.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b9265d0 0x55585b939d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b927b40 0x55585b92af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b931370 0x55585b92b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4801aef20 con 0x55585b931370 2024-09-19T06:56:11.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa4801ae850 con 0x55585b9265d0 2024-09-19T06:56:11.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4801abab0 con 0x55585b927b40 2024-09-19T06:56:11.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.214+0000 7fa48925b740 10 client.0 _ll_register_callbacks cb 0x7fa46c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:11.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa48541f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b931370 0x55585b92b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa48541f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b931370 0x55585b92b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60854/0 (socket says 172.21.15.103:60854) 2024-09-19T06:56:11.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa48541f640 1 -- 172.21.15.103:0/3404457376 learned_addr learned my addr 172.21.15.103:0/3404457376 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:11.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa484c1e640 1 --2- 172.21.15.103:0/3404457376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b9265d0 0x55585b939d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa47ffff640 1 --2- 172.21.15.103:0/3404457376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b927b40 0x55585b92af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa48541f640 1 -- 172.21.15.103:0/3404457376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b927b40 msgr2=0x55585b92af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:11.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa48541f640 1 --2- 172.21.15.103:0/3404457376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b927b40 0x55585b92af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa48541f640 1 -- 172.21.15.103:0/3404457376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b9265d0 msgr2=0x55585b939d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:11.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa48541f640 1 --2- 172.21.15.103:0/3404457376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b9265d0 0x55585b939d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa48541f640 1 -- 172.21.15.103:0/3404457376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55585b9aade0 con 0x55585b931370 2024-09-19T06:56:11.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa48541f640 1 --2- 172.21.15.103:0/3404457376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b931370 0x55585b92b670 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fa478009270 tx=0x7fa4780054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a18a07d653f5bab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:11.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa45ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:11.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa45ffff640 1 -- 172.21.15.103:0/3404457376 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47801f040 con 0x55585b931370 2024-09-19T06:56:11.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa45ffff640 1 -- 172.21.15.103:0/3404457376 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8600+0+0 (secure 0 0 0) 0x7fa4780cda10 con 0x55585b931370 2024-09-19T06:56:11.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa48925b740 1 -- 172.21.15.103:0/3404457376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4740043c0 con 0x55585b931370 2024-09-19T06:56:11.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa45ffff640 1 -- 172.21.15.103:0/3404457376 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa47809c0f0 con 0x55585b931370 2024-09-19T06:56:11.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa48925b740 10 client.17880 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:11.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa48925b740 1 -- 172.21.15.103:0/3404457376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55585b9abeb0 con 0x55585b931370 2024-09-19T06:56:11.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.215+0000 7fa45ffff640 1 -- 172.21.15.103:0/3404457376 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4740043c0 con 0x55585b931370 2024-09-19T06:56:11.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.216+0000 7fa45ffff640 1 -- 172.21.15.103:0/3404457376 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa47800f040 con 0x55585b931370 2024-09-19T06:56:11.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.216+0000 7fa48925b740 10 client.17880 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:11.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.216+0000 7fa48925b740 10 client.17880 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:56:11.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.216+0000 7fa48925b740 20 client.17880 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:11.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.216+0000 7fa48925b740 1 --2- 172.21.15.103:0/3404457376 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55585b941100 0x55585b9ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.216+0000 7fa48925b740 4 client.17880 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:56:11.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.216+0000 7fa48925b740 1 -- 172.21.15.103:0/3404457376 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55585b893d70 con 0x55585b941100 2024-09-19T06:56:11.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.216+0000 7fa484c1e640 1 --2- 172.21.15.103:0/3404457376 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55585b941100 0x55585b9ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.216+0000 7fa484c1e640 1 --2- 172.21.15.103:0/3404457376 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55585b941100 0x55585b9ae140 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd2740cee3860bc3 server_cookie=f5f33e711147ae15 in_seq=0 out_seq=0 2024-09-19T06:56:11.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.216+0000 7fa45ffff640 10 client.17880 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:56:11.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.217+0000 7fa45ffff640 1 -- 172.21.15.103:0/3404457376 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55585b893d70 con 0x55585b941100 2024-09-19T06:56:11.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.217+0000 7fa45ffff640 10 client.17880 handle_command_reply: tid=0 2024-09-19T06:56:11.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.217+0000 7fa48925b740 1 client.17880 shutdown 2024-09-19T06:56:11.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.217+0000 7fa48925b740 1 -- 172.21.15.103:0/3404457376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b931370 msgr2=0x55585b92b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:11.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.217+0000 7fa48925b740 1 --2- 172.21.15.103:0/3404457376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b931370 0x55585b92b670 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fa478009270 tx=0x7fa4780054d0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.217+0000 7fa48925b740 1 -- 172.21.15.103:0/3404457376 shutdown_connections 2024-09-19T06:56:11.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.217+0000 7fa48925b740 1 --2- 172.21.15.103:0/3404457376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55585b931370 0x55585b92b670 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.217+0000 7fa48925b740 1 --2- 172.21.15.103:0/3404457376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55585b927b40 0x55585b92af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.217+0000 7fa48925b740 1 --2- 172.21.15.103:0/3404457376 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55585b941100 0x55585b9ae140 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.217+0000 7fa48925b740 1 --2- 172.21.15.103:0/3404457376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55585b9265d0 0x55585b939d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.217+0000 7fa48925b740 1 -- 172.21.15.103:0/3404457376 >> 172.21.15.103:0/3404457376 conn(0x55585b8ef310 msgr2=0x55585b9214e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:11.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.217+0000 7fa45ffff640 0 client.17880 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:56:11.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.217+0000 7fa48925b740 1 -- 172.21.15.103:0/3404457376 shutdown_connections 2024-09-19T06:56:11.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.217+0000 7fa48925b740 1 -- 172.21.15.103:0/3404457376 wait complete. 2024-09-19T06:56:11.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.217+0000 7fa48925b740 20 client.17880 trim_cache size 0 max 16384 2024-09-19T06:56:11.230 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:56:11.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.218+0000 7fa486e82640 1 -- 172.21.15.103:0/2382935248 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa4700a3400 msgr2=0x7fa4700a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:11.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.218+0000 7fa486e82640 1 --2- 172.21.15.103:0/2382935248 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa4700a3400 0x7fa4700a58c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa46c005c20 tx=0x7fa46c005fd0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.218+0000 7fa486e82640 1 -- 172.21.15.103:0/2382935248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa480106a80 msgr2=0x7fa48019cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:11.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.218+0000 7fa486e82640 1 --2- 172.21.15.103:0/2382935248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa480106a80 0x7fa48019cd30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fa47800eaa0 tx=0x7fa47800ef60 comp rx=0 tx=0).stop 2024-09-19T06:56:11.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.218+0000 7fa486e82640 1 -- 172.21.15.103:0/2382935248 shutdown_connections 2024-09-19T06:56:11.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.218+0000 7fa486e82640 1 --2- 172.21.15.103:0/2382935248 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa4700a3400 0x7fa4700a58c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.218+0000 7fa486e82640 1 --2- 172.21.15.103:0/2382935248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa480106a80 0x7fa48019cd30 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.218+0000 7fa486e82640 1 --2- 172.21.15.103:0/2382935248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa480101420 0x7fa4801989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.218+0000 7fa486e82640 1 --2- 172.21.15.103:0/2382935248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa480100a80 0x7fa480198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.218+0000 7fa486e82640 1 -- 172.21.15.103:0/2382935248 >> 172.21.15.103:0/2382935248 conn(0x7fa4800fc7a0 msgr2=0x7fa4800fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:11.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.218+0000 7fa486e82640 1 -- 172.21.15.103:0/2382935248 shutdown_connections 2024-09-19T06:56:11.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:11.218+0000 7fa486e82640 1 -- 172.21.15.103:0/2382935248 wait complete. 2024-09-19T06:56:11.924 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:56:12.171 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.583+0000 7f101005c640 1 -- 172.21.15.103:0/772582489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1008101ae0 msgr2=0x7f1008101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.583+0000 7f101005c640 1 --2- 172.21.15.103:0/772582489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1008101ae0 0x7f1008101f40 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009930 tx=0x7f0ff802a420 comp rx=0 tx=0).stop 2024-09-19T06:56:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.583+0000 7f101005c640 1 -- 172.21.15.103:0/772582489 shutdown_connections 2024-09-19T06:56:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.583+0000 7f101005c640 1 --2- 172.21.15.103:0/772582489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1008102480 0x7f100810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.583+0000 7f101005c640 1 --2- 172.21.15.103:0/772582489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1008101ae0 0x7f1008101f40 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.583+0000 7f101005c640 1 --2- 172.21.15.103:0/772582489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1008107b80 0x7f1008107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.583+0000 7f101005c640 1 -- 172.21.15.103:0/772582489 >> 172.21.15.103:0/772582489 conn(0x7f10080fd7e0 msgr2=0x7f10080ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.583+0000 7f101005c640 1 -- 172.21.15.103:0/772582489 shutdown_connections 2024-09-19T06:56:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.583+0000 7f101005c640 1 -- 172.21.15.103:0/772582489 wait complete. 2024-09-19T06:56:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f101005c640 1 Processor -- start 2024-09-19T06:56:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f101005c640 1 -- start start 2024-09-19T06:56:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f101005c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1008101ae0 0x7f10080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f101005c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1008102480 0x7f1008074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f101005c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1008107b80 0x7f1008074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f101005c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1008111ef0 con 0x7f1008101ae0 2024-09-19T06:56:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f101005c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1008111d70 con 0x7f1008102480 2024-09-19T06:56:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f101005c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1008112070 con 0x7f1008107b80 2024-09-19T06:56:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f100ddf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1008101ae0 0x7f10080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f100ddf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1008101ae0 0x7f10080744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60884/0 (socket says 172.21.15.103:60884) 2024-09-19T06:56:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f100ddf8640 1 -- 172.21.15.103:0/2028537122 learned_addr learned my addr 172.21.15.103:0/2028537122 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f100e5f9640 1 --2- 172.21.15.103:0/2028537122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1008107b80 0x7f1008074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f100d5f7640 1 --2- 172.21.15.103:0/2028537122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1008102480 0x7f1008074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f100ddf8640 1 -- 172.21.15.103:0/2028537122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1008107b80 msgr2=0x7f1008074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f100ddf8640 1 --2- 172.21.15.103:0/2028537122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1008107b80 0x7f1008074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f100ddf8640 1 -- 172.21.15.103:0/2028537122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1008102480 msgr2=0x7f1008074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f100ddf8640 1 --2- 172.21.15.103:0/2028537122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1008102480 0x7f1008074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.584+0000 7f100ddf8640 1 -- 172.21.15.103:0/2028537122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10081a6880 con 0x7f1008101ae0 2024-09-19T06:56:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.585+0000 7f100ddf8640 1 --2- 172.21.15.103:0/2028537122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1008101ae0 0x7f10080744f0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f100400d950 tx=0x7f100400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.588+0000 7f0ffeffd640 1 -- 172.21.15.103:0/2028537122 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004017030 con 0x7f1008101ae0 2024-09-19T06:56:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.588+0000 7f0ffeffd640 1 -- 172.21.15.103:0/2028537122 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1004004020 con 0x7f1008101ae0 2024-09-19T06:56:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.588+0000 7f101005c640 1 -- 172.21.15.103:0/2028537122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10081a6b70 con 0x7f1008101ae0 2024-09-19T06:56:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.588+0000 7f0ffeffd640 1 -- 172.21.15.103:0/2028537122 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10040047a0 con 0x7f1008101ae0 2024-09-19T06:56:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.588+0000 7f101005c640 1 -- 172.21.15.103:0/2028537122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10081a70e0 con 0x7f1008101ae0 2024-09-19T06:56:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.590+0000 7f0ffeffd640 1 -- 172.21.15.103:0/2028537122 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f10040041d0 con 0x7f1008101ae0 2024-09-19T06:56:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.590+0000 7f1012ded740 1 Processor -- start 2024-09-19T06:56:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.590+0000 7f1012ded740 1 -- start start 2024-09-19T06:56:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f1012ded740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b835af0 0x55a24b835ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f1012ded740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b836410 0x55a24b840770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f1012ded740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b840cb0 0x55a24b8430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f1012ded740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a24b79a2f0 con 0x55a24b836410 2024-09-19T06:56:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f1012ded740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a24b742270 con 0x55a24b840cb0 2024-09-19T06:56:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f1012ded740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a24b7962b0 con 0x55a24b835af0 2024-09-19T06:56:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f100ddf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b836410 0x55a24b840770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f100ddf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b836410 0x55a24b840770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60900/0 (socket says 172.21.15.103:60900) 2024-09-19T06:56:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f100ddf8640 1 -- 172.21.15.103:0/2940439987 learned_addr learned my addr 172.21.15.103:0/2940439987 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f100e5f9640 1 --2- 172.21.15.103:0/2940439987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b835af0 0x55a24b835ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f100d5f7640 1 --2- 172.21.15.103:0/2940439987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b840cb0 0x55a24b8430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f0ffeffd640 1 --2- 172.21.15.103:0/2028537122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0ff40a34e0 0x7f0ff40a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f100e5f9640 1 --2- 172.21.15.103:0/2028537122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0ff40a34e0 0x7f0ff40a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f100ddf8640 1 -- 172.21.15.103:0/2940439987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b835af0 msgr2=0x55a24b835ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f100ddf8640 1 --2- 172.21.15.103:0/2940439987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b835af0 0x55a24b835ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f100ddf8640 1 -- 172.21.15.103:0/2940439987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b840cb0 msgr2=0x55a24b8430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f100ddf8640 1 --2- 172.21.15.103:0/2940439987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b840cb0 0x55a24b8430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f100ddf8640 1 -- 172.21.15.103:0/2940439987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a24b741340 con 0x55a24b836410 2024-09-19T06:56:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f100d5f7640 1 --2- 172.21.15.103:0/2940439987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b840cb0 0x55a24b8430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:56:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.591+0000 7f0ffeffd640 1 -- 172.21.15.103:0/2028537122 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f10040cf560 con 0x7f1008101ae0 2024-09-19T06:56:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.592+0000 7f100ddf8640 1 --2- 172.21.15.103:0/2940439987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b836410 0x55a24b840770 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f1004097c80 tx=0x7f1004005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586fed7d56e34017 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.592+0000 7f100e5f9640 1 --2- 172.21.15.103:0/2028537122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0ff40a34e0 0x7f0ff40a59a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1000005c40 tx=0x7f1000004ab0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.592+0000 7f0ff3fff640 1 -- 172.21.15.103:0/2940439987 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100409b360 con 0x55a24b836410 2024-09-19T06:56:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.592+0000 7f0ff3fff640 1 -- 172.21.15.103:0/2940439987 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f100409bf20 con 0x55a24b836410 2024-09-19T06:56:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.592+0000 7f0ff3fff640 1 -- 172.21.15.103:0/2940439987 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004048f70 con 0x55a24b836410 2024-09-19T06:56:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.593+0000 7f1012ded740 1 -- 172.21.15.103:0/2940439987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b836410 msgr2=0x55a24b840770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.593+0000 7f1012ded740 1 --2- 172.21.15.103:0/2940439987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b836410 0x55a24b840770 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f1004097c80 tx=0x7f1004005bf0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.593+0000 7f1012ded740 1 -- 172.21.15.103:0/2940439987 shutdown_connections 2024-09-19T06:56:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.593+0000 7f1012ded740 1 --2- 172.21.15.103:0/2940439987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b840cb0 0x55a24b8430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.593+0000 7f1012ded740 1 --2- 172.21.15.103:0/2940439987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b836410 0x55a24b840770 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.593+0000 7f1012ded740 1 --2- 172.21.15.103:0/2940439987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b835af0 0x55a24b835ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.593+0000 7f1012ded740 1 -- 172.21.15.103:0/2940439987 >> 172.21.15.103:0/2940439987 conn(0x55a24b832060 msgr2=0x55a24b832440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.593+0000 7f1012ded740 1 -- 172.21.15.103:0/2940439987 shutdown_connections 2024-09-19T06:56:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.593+0000 7f1012ded740 1 -- 172.21.15.103:0/2940439987 wait complete. 2024-09-19T06:56:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.593+0000 7f1012ded740 1 -- start start 2024-09-19T06:56:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.593+0000 7f1012ded740 1 Processor -- start 2024-09-19T06:56:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f1012ded740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b835af0 0x55a24b83db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f1012ded740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b836410 0x55a24b83e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f1012ded740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b840cb0 0x55a24b83e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f1012ded740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a24b843880 con 0x55a24b840cb0 2024-09-19T06:56:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f1012ded740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a24b723290 con 0x55a24b836410 2024-09-19T06:56:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f1012ded740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a24b843a00 con 0x55a24b835af0 2024-09-19T06:56:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f1012ded740 10 client.0 _ll_register_callbacks cb 0x55a24b754140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f100e5f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b840cb0 0x55a24b83e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f100e5f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b840cb0 0x55a24b83e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60914/0 (socket says 172.21.15.103:60914) 2024-09-19T06:56:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f100e5f9640 1 -- 172.21.15.103:0/1562190714 learned_addr learned my addr 172.21.15.103:0/1562190714 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f100d5f7640 1 --2- 172.21.15.103:0/1562190714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b836410 0x55a24b83e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f100ddf8640 1 --2- 172.21.15.103:0/1562190714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b835af0 0x55a24b83db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f100e5f9640 1 -- 172.21.15.103:0/1562190714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b835af0 msgr2=0x55a24b83db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f100e5f9640 1 --2- 172.21.15.103:0/1562190714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b835af0 0x55a24b83db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f100e5f9640 1 -- 172.21.15.103:0/1562190714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b836410 msgr2=0x55a24b83e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f100e5f9640 1 --2- 172.21.15.103:0/1562190714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b836410 0x55a24b83e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f100e5f9640 1 -- 172.21.15.103:0/1562190714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a24b4ad210 con 0x55a24b840cb0 2024-09-19T06:56:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f100d5f7640 1 --2- 172.21.15.103:0/1562190714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b836410 0x55a24b83e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.594+0000 7f100ddf8640 1 --2- 172.21.15.103:0/1562190714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b835af0 0x55a24b83db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.595+0000 7f100e5f9640 1 --2- 172.21.15.103:0/1562190714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b840cb0 0x55a24b83e610 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f1000003d70 tx=0x7f1000004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604ba9fc8ddd0382 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.595+0000 7f0ff17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.595+0000 7f0ff17fa640 1 -- 172.21.15.103:0/1562190714 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000025030 con 0x55a24b840cb0 2024-09-19T06:56:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.595+0000 7f0ff17fa640 1 -- 172.21.15.103:0/1562190714 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f10000045d0 con 0x55a24b840cb0 2024-09-19T06:56:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.595+0000 7f1012ded740 1 -- 172.21.15.103:0/1562190714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f100409bf20 con 0x55a24b840cb0 2024-09-19T06:56:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.595+0000 7f0ff17fa640 1 -- 172.21.15.103:0/1562190714 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f10000047d0 con 0x55a24b840cb0 2024-09-19T06:56:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.595+0000 7f1012ded740 10 client.17892 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.595+0000 7f1012ded740 1 -- 172.21.15.103:0/1562190714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a24b579620 con 0x55a24b840cb0 2024-09-19T06:56:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.595+0000 7f0ff17fa640 1 -- 172.21.15.103:0/1562190714 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f100409bf20 con 0x55a24b840cb0 2024-09-19T06:56:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.595+0000 7f0ff17fa640 1 -- 172.21.15.103:0/1562190714 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f100001f040 con 0x55a24b840cb0 2024-09-19T06:56:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.595+0000 7f1012ded740 10 client.17892 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.595+0000 7f1012ded740 10 client.17892 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.595+0000 7f1012ded740 20 client.17892 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.595+0000 7f1012ded740 1 --2- 172.21.15.103:0/1562190714 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a24b850c00 0x55a24b853030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.595+0000 7f1012ded740 4 client.17892 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.595+0000 7f1012ded740 1 -- 172.21.15.103:0/1562190714 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a24b7a7a20 con 0x55a24b850c00 2024-09-19T06:56:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.596+0000 7f100ddf8640 1 --2- 172.21.15.103:0/1562190714 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a24b850c00 0x55a24b853030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.596+0000 7f0ff17fa640 10 client.17892 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.596+0000 7f100ddf8640 1 --2- 172.21.15.103:0/1562190714 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a24b850c00 0x55a24b853030 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c369549f62c32be4 server_cookie=9893eac2cbe60d6b in_seq=0 out_seq=0 2024-09-19T06:56:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.597+0000 7f0ff17fa640 1 -- 172.21.15.103:0/1562190714 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a24b7a7a20 con 0x55a24b850c00 2024-09-19T06:56:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.597+0000 7f0ff17fa640 10 client.17892 handle_command_reply: tid=0 2024-09-19T06:56:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.597+0000 7f1012ded740 1 client.17892 shutdown 2024-09-19T06:56:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.597+0000 7f1012ded740 1 -- 172.21.15.103:0/1562190714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b840cb0 msgr2=0x55a24b83e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.597+0000 7f1012ded740 1 --2- 172.21.15.103:0/1562190714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b840cb0 0x55a24b83e610 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f1000003d70 tx=0x7f1000004a40 comp rx=0 tx=0).stop 2024-09-19T06:56:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.597+0000 7f1012ded740 1 -- 172.21.15.103:0/1562190714 shutdown_connections 2024-09-19T06:56:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.597+0000 7f1012ded740 1 --2- 172.21.15.103:0/1562190714 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a24b850c00 0x55a24b853030 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.597+0000 7f1012ded740 1 --2- 172.21.15.103:0/1562190714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b840cb0 0x55a24b83e610 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.597+0000 7f1012ded740 1 --2- 172.21.15.103:0/1562190714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b836410 0x55a24b83e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.597+0000 7f0ff17fa640 0 client.17892 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.597+0000 7f1012ded740 1 --2- 172.21.15.103:0/1562190714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b835af0 0x55a24b83db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.597+0000 7f1012ded740 1 -- 172.21.15.103:0/1562190714 >> 172.21.15.103:0/1562190714 conn(0x55a24b832060 msgr2=0x55a24b837dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.597+0000 7f1012ded740 1 -- 172.21.15.103:0/1562190714 shutdown_connections 2024-09-19T06:56:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.597+0000 7f1012ded740 1 -- 172.21.15.103:0/1562190714 wait complete. 2024-09-19T06:56:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.597+0000 7f1012ded740 20 client.17892 trim_cache size 0 max 16384 2024-09-19T06:56:12.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.612+0000 7f96bee47640 1 -- 172.21.15.103:0/593971618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b8102790 msgr2=0x7f96b81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.612+0000 7f96bee47640 1 --2- 172.21.15.103:0/593971618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b8102790 0x7f96b81067e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009ca0 tx=0x7f96a802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.612+0000 7f96bee47640 1 -- 172.21.15.103:0/593971618 shutdown_connections 2024-09-19T06:56:12.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.612+0000 7f96bee47640 1 --2- 172.21.15.103:0/593971618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b8106f10 0x7f96b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.612+0000 7f96bee47640 1 --2- 172.21.15.103:0/593971618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b8102790 0x7f96b81067e0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.612+0000 7f96bee47640 1 --2- 172.21.15.103:0/593971618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96b8101de0 0x7f96b81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.612+0000 7f96bee47640 1 -- 172.21.15.103:0/593971618 >> 172.21.15.103:0/593971618 conn(0x7f96b80fd8c0 msgr2=0x7f96b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:12.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.612+0000 7f96bee47640 1 -- 172.21.15.103:0/593971618 shutdown_connections 2024-09-19T06:56:12.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.612+0000 7f96bee47640 1 -- 172.21.15.103:0/593971618 wait complete. 2024-09-19T06:56:12.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.613+0000 7f96bee47640 1 Processor -- start 2024-09-19T06:56:12.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.613+0000 7f96bee47640 1 -- start start 2024-09-19T06:56:12.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.613+0000 7f96bee47640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b8101de0 0x7f96b8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.613+0000 7f96bee47640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96b8102790 0x7f96b8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.613+0000 7f96bee47640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b8106f10 0x7f96b819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.613+0000 7f96bee47640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96b8111f70 con 0x7f96b8106f10 2024-09-19T06:56:12.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.613+0000 7f96bee47640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96b8111df0 con 0x7f96b8102790 2024-09-19T06:56:12.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.613+0000 7f96bee47640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96b81120f0 con 0x7f96b8101de0 2024-09-19T06:56:12.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.613+0000 7f96bd3e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b8106f10 0x7f96b819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.613+0000 7f96b7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96b8102790 0x7f96b8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.613+0000 7f96bd3e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b8106f10 0x7f96b819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60920/0 (socket says 172.21.15.103:60920) 2024-09-19T06:56:12.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.613+0000 7f96bd3e4640 1 -- 172.21.15.103:0/2705750521 learned_addr learned my addr 172.21.15.103:0/2705750521 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:12.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.613+0000 7f96bcbe3640 1 --2- 172.21.15.103:0/2705750521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b8101de0 0x7f96b8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f96bd3e4640 1 -- 172.21.15.103:0/2705750521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b8101de0 msgr2=0x7f96b8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f96bd3e4640 1 --2- 172.21.15.103:0/2705750521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b8101de0 0x7f96b8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f96bd3e4640 1 -- 172.21.15.103:0/2705750521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96b8102790 msgr2=0x7f96b8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f96bd3e4640 1 --2- 172.21.15.103:0/2705750521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96b8102790 0x7f96b8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f96bd3e4640 1 -- 172.21.15.103:0/2705750521 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b819d3e0 con 0x7f96b8106f10 2024-09-19T06:56:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f1012ded740 1 Processor -- start 2024-09-19T06:56:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f1012ded740 1 -- start start 2024-09-19T06:56:12.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f96bd3e4640 1 --2- 172.21.15.103:0/2705750521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b8106f10 0x7f96b819cd00 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f96a000c970 tx=0x7f96a000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f1012ded740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b835af0 0x55a24b838560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f1012ded740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b836410 0x55a24b83e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f1012ded740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b840cb0 0x55a24b83fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f1012ded740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10081af9f0 con 0x55a24b835af0 2024-09-19T06:56:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f1012ded740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10081b1530 con 0x55a24b840cb0 2024-09-19T06:56:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f1012ded740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10081b00c0 con 0x55a24b836410 2024-09-19T06:56:12.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f96b5ffb640 1 -- 172.21.15.103:0/2705750521 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0007bb0 con 0x7f96b8106f10 2024-09-19T06:56:12.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f96b5ffb640 1 -- 172.21.15.103:0/2705750521 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a0007d50 con 0x7f96b8106f10 2024-09-19T06:56:12.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f96b5ffb640 1 -- 172.21.15.103:0/2705750521 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a000f450 con 0x7f96b8106f10 2024-09-19T06:56:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f100ddf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b835af0 0x55a24b838560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f100ddf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b835af0 0x55a24b838560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60930/0 (socket says 172.21.15.103:60930) 2024-09-19T06:56:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f100ddf8640 1 -- 172.21.15.103:0/2680976731 learned_addr learned my addr 172.21.15.103:0/2680976731 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:12.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f96bee47640 1 -- 172.21.15.103:0/2705750521 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b819d6d0 con 0x7f96b8106f10 2024-09-19T06:56:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f100d5f7640 1 --2- 172.21.15.103:0/2680976731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b836410 0x55a24b83e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f96bee47640 1 -- 172.21.15.103:0/2705750521 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b81a4fb0 con 0x7f96b8106f10 2024-09-19T06:56:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.614+0000 7f100e5f9640 1 --2- 172.21.15.103:0/2680976731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b840cb0 0x55a24b83fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.615+0000 7f100ddf8640 1 -- 172.21.15.103:0/2680976731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b836410 msgr2=0x55a24b83e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.615+0000 7f100ddf8640 1 --2- 172.21.15.103:0/2680976731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b836410 0x55a24b83e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.615+0000 7f100ddf8640 1 -- 172.21.15.103:0/2680976731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b840cb0 msgr2=0x55a24b83fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.615+0000 7f100ddf8640 1 --2- 172.21.15.103:0/2680976731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b840cb0 0x55a24b83fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.615+0000 7f100ddf8640 1 -- 172.21.15.103:0/2680976731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a24b831e90 con 0x55a24b835af0 2024-09-19T06:56:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.615+0000 7f100ddf8640 1 --2- 172.21.15.103:0/2680976731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b835af0 0x55a24b838560 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f10040cd290 tx=0x7f1004097ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ff498d607276cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.615+0000 7f0ff37fe640 1 -- 172.21.15.103:0/2680976731 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100409b180 con 0x55a24b835af0 2024-09-19T06:56:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.615+0000 7f0ff37fe640 1 -- 172.21.15.103:0/2680976731 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f100409bf20 con 0x55a24b835af0 2024-09-19T06:56:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.615+0000 7f0ff37fe640 1 -- 172.21.15.103:0/2680976731 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10040a4000 con 0x55a24b835af0 2024-09-19T06:56:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.616+0000 7f1012ded740 1 -- 172.21.15.103:0/2680976731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b835af0 msgr2=0x55a24b838560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.616+0000 7f1012ded740 1 --2- 172.21.15.103:0/2680976731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b835af0 0x55a24b838560 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f10040cd290 tx=0x7f1004097ac0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.616+0000 7f1012ded740 1 -- 172.21.15.103:0/2680976731 shutdown_connections 2024-09-19T06:56:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.616+0000 7f1012ded740 1 --2- 172.21.15.103:0/2680976731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b840cb0 0x55a24b83fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.616+0000 7f1012ded740 1 --2- 172.21.15.103:0/2680976731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b836410 0x55a24b83e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.616+0000 7f1012ded740 1 --2- 172.21.15.103:0/2680976731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b835af0 0x55a24b838560 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.616+0000 7f1012ded740 1 -- 172.21.15.103:0/2680976731 >> 172.21.15.103:0/2680976731 conn(0x55a24b832060 msgr2=0x55a24b83f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.616+0000 7f1012ded740 1 -- 172.21.15.103:0/2680976731 shutdown_connections 2024-09-19T06:56:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.616+0000 7f1012ded740 1 -- 172.21.15.103:0/2680976731 wait complete. 2024-09-19T06:56:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.616+0000 7f1012ded740 1 -- start start 2024-09-19T06:56:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.617+0000 7f1012ded740 1 Processor -- start 2024-09-19T06:56:12.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.617+0000 7f96b5ffb640 1 -- 172.21.15.103:0/2705750521 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f96a0004f20 con 0x7f96b8106f10 2024-09-19T06:56:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.617+0000 7f1012ded740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b835af0 0x55a24b848ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.617+0000 7f1012ded740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b836410 0x55a24b8491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.617+0000 7f1012ded740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b840cb0 0x55a24b844210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.617+0000 7f1012ded740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10081b0e60 con 0x55a24b840cb0 2024-09-19T06:56:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.617+0000 7f1012ded740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10081b0790 con 0x55a24b836410 2024-09-19T06:56:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.617+0000 7f1012ded740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10081ad9f0 con 0x55a24b835af0 2024-09-19T06:56:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.617+0000 7f1012ded740 10 client.0 _ll_register_callbacks cb 0x7f1000003a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.617+0000 7f100e5f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b840cb0 0x55a24b844210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.617+0000 7f100e5f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b840cb0 0x55a24b844210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60932/0 (socket says 172.21.15.103:60932) 2024-09-19T06:56:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.617+0000 7f100e5f9640 1 -- 172.21.15.103:0/672126697 learned_addr learned my addr 172.21.15.103:0/672126697 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:12.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.617+0000 7f96c1220740 1 Processor -- start 2024-09-19T06:56:12.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.617+0000 7f96c1220740 1 -- start start 2024-09-19T06:56:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.617+0000 7f100d5f7640 1 --2- 172.21.15.103:0/672126697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b836410 0x55a24b8491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.617+0000 7f100ddf8640 1 --2- 172.21.15.103:0/672126697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b835af0 0x55a24b848ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f100e5f9640 1 -- 172.21.15.103:0/672126697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b835af0 msgr2=0x55a24b848ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f100e5f9640 1 --2- 172.21.15.103:0/672126697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b835af0 0x55a24b848ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f100e5f9640 1 -- 172.21.15.103:0/672126697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b836410 msgr2=0x55a24b8491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f100e5f9640 1 --2- 172.21.15.103:0/672126697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b836410 0x55a24b8491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f100e5f9640 1 -- 172.21.15.103:0/672126697 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a24b844a40 con 0x55a24b840cb0 2024-09-19T06:56:12.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96c1220740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecf5b40 0x563f9ecf4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96c1220740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecf45d0 0x563f9ecf4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96b7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecf5b40 0x563f9ecf4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f100e5f9640 1 --2- 172.21.15.103:0/672126697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b840cb0 0x55a24b844210 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f1000005c10 tx=0x7f1000009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b44e42522e26ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:12.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96b7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecf5b40 0x563f9ecf4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:46934/0 (socket says 172.21.15.103:46934) 2024-09-19T06:56:12.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96bcbe3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecf45d0 0x563f9ecf4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96bcbe3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecf45d0 0x563f9ecf4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60946/0 (socket says 172.21.15.103:60946) 2024-09-19T06:56:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f0ff17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:12.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96b5ffb640 1 --2- 172.21.15.103:0/2705750521 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96a40a3430 0x7f96a40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96c1220740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecff370 0x563f9ed01760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96c1220740 1 -- 172.21.15.103:0/687734309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563f9ec5a2f0 con 0x563f9ecf45d0 2024-09-19T06:56:12.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96b7fff640 1 -- 172.21.15.103:0/687734309 learned_addr learned my addr 172.21.15.103:0/687734309 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:12.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96c1220740 1 -- 172.21.15.103:0/687734309 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563f9ec01cf0 con 0x563f9ecf5b40 2024-09-19T06:56:12.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96c1220740 1 -- 172.21.15.103:0/687734309 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563f9ec562b0 con 0x563f9ecff370 2024-09-19T06:56:12.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96bd3e4640 1 --2- 172.21.15.103:0/2705750521 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96a40a3430 0x7f96a40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f0ff17fa640 1 -- 172.21.15.103:0/672126697 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000025040 con 0x55a24b840cb0 2024-09-19T06:56:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f0ff17fa640 1 -- 172.21.15.103:0/672126697 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f1000032bc0 con 0x55a24b840cb0 2024-09-19T06:56:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f1012ded740 1 -- 172.21.15.103:0/672126697 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f100409bf20 con 0x55a24b840cb0 2024-09-19T06:56:12.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96b5ffb640 1 -- 172.21.15.103:0/2705750521 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f96a00cf820 con 0x7f96b8106f10 2024-09-19T06:56:12.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96b7fff640 1 --2- 172.21.15.103:0/687734309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecff370 0x563f9ed01760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96bcbe3640 1 -- 172.21.15.103:0/687734309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecff370 msgr2=0x563f9ed01760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:12.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96bcbe3640 1 --2- 172.21.15.103:0/687734309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecff370 0x563f9ed01760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96bcbe3640 1 -- 172.21.15.103:0/687734309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecf5b40 msgr2=0x563f9ecf4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.619+0000 7f0ff17fa640 1 -- 172.21.15.103:0/672126697 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1000004620 con 0x55a24b840cb0 2024-09-19T06:56:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.619+0000 7f1012ded740 10 client.17904 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.619+0000 7f1012ded740 1 -- 172.21.15.103:0/672126697 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a24b8bb410 con 0x55a24b840cb0 2024-09-19T06:56:12.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96bcbe3640 1 --2- 172.21.15.103:0/687734309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecf5b40 0x563f9ecf4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.618+0000 7f96bcbe3640 1 -- 172.21.15.103:0/687734309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f9ebe63a0 con 0x563f9ecf45d0 2024-09-19T06:56:12.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.619+0000 7f96b7fff640 1 --2- 172.21.15.103:0/687734309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecff370 0x563f9ed01760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:12.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.619+0000 7f96bcbe3640 1 --2- 172.21.15.103:0/687734309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecf45d0 0x563f9ecf4a10 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f96b000bbf0 tx=0x7f96b00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff58e8ac4efbafc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.619+0000 7f0ff17fa640 1 -- 172.21.15.103:0/672126697 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f100409bf20 con 0x55a24b840cb0 2024-09-19T06:56:12.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.619+0000 7f96bd3e4640 1 --2- 172.21.15.103:0/2705750521 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96a40a3430 0x7f96a40a58f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f96a009b890 tx=0x7f96a006a880 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:12.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.619+0000 7f96aeffd640 1 -- 172.21.15.103:0/687734309 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0004030 con 0x563f9ecf45d0 2024-09-19T06:56:12.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.619+0000 7f96aeffd640 1 -- 172.21.15.103:0/687734309 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b00041d0 con 0x563f9ecf45d0 2024-09-19T06:56:12.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.619+0000 7f96aeffd640 1 -- 172.21.15.103:0/687734309 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b00116c0 con 0x563f9ecf45d0 2024-09-19T06:56:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f0ff17fa640 1 -- 172.21.15.103:0/672126697 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1000005120 con 0x55a24b840cb0 2024-09-19T06:56:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f1012ded740 10 client.17904 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f1012ded740 10 client.17904 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f1012ded740 20 client.17904 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f1012ded740 1 --2- 172.21.15.103:0/672126697 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a24b8bba80 0x55a24b8bdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f1012ded740 4 client.17904 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:56:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f1012ded740 1 -- 172.21.15.103:0/672126697 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a24b7a1d70 con 0x55a24b8bba80 2024-09-19T06:56:12.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f96c1220740 1 -- 172.21.15.103:0/687734309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecf45d0 msgr2=0x563f9ecf4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f96c1220740 1 --2- 172.21.15.103:0/687734309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecf45d0 0x563f9ecf4a10 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f96b000bbf0 tx=0x7f96b00107a0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f100ddf8640 1 --2- 172.21.15.103:0/672126697 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a24b8bba80 0x55a24b8bdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f96c1220740 1 -- 172.21.15.103:0/687734309 shutdown_connections 2024-09-19T06:56:12.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f96c1220740 1 --2- 172.21.15.103:0/687734309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecff370 0x563f9ed01760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f96c1220740 1 --2- 172.21.15.103:0/687734309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecf45d0 0x563f9ecf4a10 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f96c1220740 1 --2- 172.21.15.103:0/687734309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecf5b40 0x563f9ecf4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f96c1220740 1 -- 172.21.15.103:0/687734309 >> 172.21.15.103:0/687734309 conn(0x563f9ecbd310 msgr2=0x563f9ecf0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:12.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f96c1220740 1 -- 172.21.15.103:0/687734309 shutdown_connections 2024-09-19T06:56:12.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f96c1220740 1 -- 172.21.15.103:0/687734309 wait complete. 2024-09-19T06:56:12.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.620+0000 7f96c1220740 1 -- start start 2024-09-19T06:56:12.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.621+0000 7f96c1220740 1 Processor -- start 2024-09-19T06:56:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.621+0000 7f100ddf8640 1 --2- 172.21.15.103:0/672126697 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a24b8bba80 0x55a24b8bdea0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef8549e171598782 server_cookie=ae4ec1d62ff749d8 in_seq=0 out_seq=0 2024-09-19T06:56:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.621+0000 7f0ff17fa640 10 client.17904 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:12.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.621+0000 7f96c1220740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecf45d0 0x563f9ecfed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.621+0000 7f96c1220740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecf5b40 0x563f9ecfbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.621+0000 7f96c1220740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecff370 0x563f9ecfc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.621+0000 7f96c1220740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563f9ed02070 con 0x563f9ecf45d0 2024-09-19T06:56:12.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.621+0000 7f96c1220740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563f9ec15060 con 0x563f9ecff370 2024-09-19T06:56:12.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.621+0000 7f96c1220740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563f9ed021f0 con 0x563f9ecf5b40 2024-09-19T06:56:12.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.621+0000 7f96c1220740 10 client.0 _ll_register_callbacks cb 0x563f9e9688c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:12.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.621+0000 7f96b7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecf45d0 0x563f9ecfed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.621+0000 7f96b7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecf45d0 0x563f9ecfed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60952/0 (socket says 172.21.15.103:60952) 2024-09-19T06:56:12.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.621+0000 7f96b7fff640 1 -- 172.21.15.103:0/3442183515 learned_addr learned my addr 172.21.15.103:0/3442183515 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:12.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.621+0000 7f96b7fff640 1 --2- 172.21.15.103:0/3442183515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecff370 0x563f9ecfc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.621+0000 7f96bcbe3640 1 --2- 172.21.15.103:0/3442183515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecf5b40 0x563f9ecfbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.622+0000 7f96b7fff640 1 -- 172.21.15.103:0/3442183515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecf5b40 msgr2=0x563f9ecfbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:12.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.622+0000 7f96b7fff640 1 --2- 172.21.15.103:0/3442183515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecf5b40 0x563f9ecfbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.622+0000 7f96b7fff640 1 -- 172.21.15.103:0/3442183515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecff370 msgr2=0x563f9ecfc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:12.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.622+0000 7f96b7fff640 1 --2- 172.21.15.103:0/3442183515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecff370 0x563f9ecfc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.622+0000 7f96b7fff640 1 -- 172.21.15.103:0/3442183515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f9e9cb3c0 con 0x563f9ecf45d0 2024-09-19T06:56:12.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.622+0000 7f96bcbe3640 1 --2- 172.21.15.103:0/3442183515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecf5b40 0x563f9ecfbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:12.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.622+0000 7f96b7fff640 1 --2- 172.21.15.103:0/3442183515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecf45d0 0x563f9ecfed40 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f96a8006010 tx=0x7f96a8005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9653c6cd52f505a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:12.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.622+0000 7f9697fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:12.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.622+0000 7f9697fff640 1 -- 172.21.15.103:0/3442183515 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a803f070 con 0x563f9ecf45d0 2024-09-19T06:56:12.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.622+0000 7f9697fff640 1 -- 172.21.15.103:0/3442183515 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f96a802faa0 con 0x563f9ecf45d0 2024-09-19T06:56:12.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.622+0000 7f96c1220740 1 -- 172.21.15.103:0/3442183515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96b00041d0 con 0x563f9ecf45d0 2024-09-19T06:56:12.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.622+0000 7f9697fff640 1 -- 172.21.15.103:0/3442183515 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f96a802fca0 con 0x563f9ecf45d0 2024-09-19T06:56:12.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.622+0000 7f96c1220740 10 client.17910 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:12.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.622+0000 7f96c1220740 1 -- 172.21.15.103:0/3442183515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f9ed16260 con 0x563f9ecf45d0 2024-09-19T06:56:12.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.622+0000 7f9697fff640 1 -- 172.21.15.103:0/3442183515 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96b00041d0 con 0x563f9ecf45d0 2024-09-19T06:56:12.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.623+0000 7f9697fff640 1 -- 172.21.15.103:0/3442183515 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f96a8054450 con 0x563f9ecf45d0 2024-09-19T06:56:12.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.623+0000 7f96c1220740 10 client.17910 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:12.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.623+0000 7f96c1220740 10 client.17910 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:56:12.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.623+0000 7f96c1220740 20 client.17910 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:12.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.623+0000 7f96c1220740 1 --2- 172.21.15.103:0/3442183515 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x563f9ed17650 0x563f9ed19aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.623+0000 7f96c1220740 4 client.17910 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:12.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.623+0000 7f96c1220740 1 -- 172.21.15.103:0/3442183515 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f9ec67860 con 0x563f9ed17650 2024-09-19T06:56:12.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.623+0000 7f96bcbe3640 1 --2- 172.21.15.103:0/3442183515 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x563f9ed17650 0x563f9ed19aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.623+0000 7f96bcbe3640 1 --2- 172.21.15.103:0/3442183515 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x563f9ed17650 0x563f9ed19aa0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9012cdb344e66f9 server_cookie=7c8f0e6ab5bcb3f1 in_seq=0 out_seq=0 2024-09-19T06:56:12.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.623+0000 7f9697fff640 10 client.17910 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:56:12.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.624+0000 7f9697fff640 1 -- 172.21.15.103:0/3442183515 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563f9ec67860 con 0x563f9ed17650 2024-09-19T06:56:12.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.624+0000 7f9697fff640 10 client.17910 handle_command_reply: tid=0 2024-09-19T06:56:12.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.624+0000 7f96c1220740 1 client.17910 shutdown 2024-09-19T06:56:12.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.624+0000 7f96c1220740 1 -- 172.21.15.103:0/3442183515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecf45d0 msgr2=0x563f9ecfed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.624+0000 7f96c1220740 1 --2- 172.21.15.103:0/3442183515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecf45d0 0x563f9ecfed40 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f96a8006010 tx=0x7f96a8005fa0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.625+0000 7f96c1220740 1 -- 172.21.15.103:0/3442183515 shutdown_connections 2024-09-19T06:56:12.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.625+0000 7f96c1220740 1 --2- 172.21.15.103:0/3442183515 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x563f9ed17650 0x563f9ed19aa0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.625+0000 7f96c1220740 1 --2- 172.21.15.103:0/3442183515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecff370 0x563f9ecfc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.625+0000 7f96c1220740 1 --2- 172.21.15.103:0/3442183515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecf5b40 0x563f9ecfbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.625+0000 7f9697fff640 0 client.17910 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:56:12.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.625+0000 7f96c1220740 1 --2- 172.21.15.103:0/3442183515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecf45d0 0x563f9ecfed40 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.625+0000 7f96c1220740 1 -- 172.21.15.103:0/3442183515 >> 172.21.15.103:0/3442183515 conn(0x563f9ecbd310 msgr2=0x563f9ecf6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:12.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.625+0000 7f96c1220740 1 -- 172.21.15.103:0/3442183515 shutdown_connections 2024-09-19T06:56:12.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.625+0000 7f96c1220740 1 -- 172.21.15.103:0/3442183515 wait complete. 2024-09-19T06:56:12.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.625+0000 7f96c1220740 20 client.17910 trim_cache size 0 max 16384 2024-09-19T06:56:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.634+0000 7f0ff17fa640 1 -- 172.21.15.103:0/672126697 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a24b7a1d70 con 0x55a24b8bba80 2024-09-19T06:56:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.634+0000 7f0ff17fa640 10 client.17904 handle_command_reply: tid=0 2024-09-19T06:56:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.634+0000 7f1012ded740 1 client.17904 shutdown 2024-09-19T06:56:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.634+0000 7f1012ded740 1 -- 172.21.15.103:0/672126697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b840cb0 msgr2=0x55a24b844210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.634+0000 7f1012ded740 1 --2- 172.21.15.103:0/672126697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b840cb0 0x55a24b844210 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f1000005c10 tx=0x7f1000009480 comp rx=0 tx=0).stop 2024-09-19T06:56:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f1012ded740 1 -- 172.21.15.103:0/672126697 shutdown_connections 2024-09-19T06:56:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f1012ded740 1 --2- 172.21.15.103:0/672126697 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a24b8bba80 0x55a24b8bdea0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f1012ded740 1 --2- 172.21.15.103:0/672126697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a24b840cb0 0x55a24b844210 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f1012ded740 1 --2- 172.21.15.103:0/672126697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a24b836410 0x55a24b8491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f0ff17fa640 0 client.17904 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f1012ded740 1 --2- 172.21.15.103:0/672126697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24b835af0 0x55a24b848ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f1012ded740 1 -- 172.21.15.103:0/672126697 >> 172.21.15.103:0/672126697 conn(0x55a24b832060 msgr2=0x55a24b8311c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f1012ded740 1 -- 172.21.15.103:0/672126697 shutdown_connections 2024-09-19T06:56:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f1012ded740 1 -- 172.21.15.103:0/672126697 wait complete. 2024-09-19T06:56:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f1012ded740 20 client.17904 trim_cache size 0 max 16384 2024-09-19T06:56:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:56:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:56:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:56:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:56:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f101005c640 1 -- 172.21.15.103:0/2028537122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0ff40a34e0 msgr2=0x7f0ff40a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f101005c640 1 --2- 172.21.15.103:0/2028537122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0ff40a34e0 0x7f0ff40a59a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1000005c40 tx=0x7f1000004ab0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f101005c640 1 -- 172.21.15.103:0/2028537122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1008101ae0 msgr2=0x7f10080744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f101005c640 1 --2- 172.21.15.103:0/2028537122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1008101ae0 0x7f10080744f0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f100400d950 tx=0x7f100400de10 comp rx=0 tx=0).stop 2024-09-19T06:56:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f101005c640 1 -- 172.21.15.103:0/2028537122 shutdown_connections 2024-09-19T06:56:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f101005c640 1 --2- 172.21.15.103:0/2028537122 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0ff40a34e0 0x7f0ff40a59a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f101005c640 1 --2- 172.21.15.103:0/2028537122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1008107b80 0x7f1008074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f101005c640 1 --2- 172.21.15.103:0/2028537122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1008102480 0x7f1008074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f101005c640 1 --2- 172.21.15.103:0/2028537122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1008101ae0 0x7f10080744f0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f101005c640 1 -- 172.21.15.103:0/2028537122 >> 172.21.15.103:0/2028537122 conn(0x7f10080fd7e0 msgr2=0x7f10080fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f101005c640 1 -- 172.21.15.103:0/2028537122 shutdown_connections 2024-09-19T06:56:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:12.635+0000 7f101005c640 1 -- 172.21.15.103:0/2028537122 wait complete. 2024-09-19T06:56:12.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96c1220740 1 Processor -- start 2024-09-19T06:56:12.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96c1220740 1 -- start start 2024-09-19T06:56:12.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96c1220740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecf45d0 0x563f9ecfdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96c1220740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecf5b40 0x563f9ecf7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96c1220740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecff370 0x563f9ecf7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96c1220740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96b81ab7e0 con 0x563f9ecff370 2024-09-19T06:56:12.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96c1220740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96b81ad320 con 0x563f9ecf45d0 2024-09-19T06:56:12.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96c1220740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96b81abeb0 con 0x563f9ecf5b40 2024-09-19T06:56:12.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96b7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecff370 0x563f9ecf7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96b7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecff370 0x563f9ecf7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60954/0 (socket says 172.21.15.103:60954) 2024-09-19T06:56:12.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96b7fff640 1 -- 172.21.15.103:0/2995302349 learned_addr learned my addr 172.21.15.103:0/2995302349 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:12.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96bcbe3640 1 --2- 172.21.15.103:0/2995302349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecf5b40 0x563f9ecf7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96b7fff640 1 -- 172.21.15.103:0/2995302349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecf5b40 msgr2=0x563f9ecf7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:12.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96b7fff640 1 --2- 172.21.15.103:0/2995302349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecf5b40 0x563f9ecf7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96b7fff640 1 -- 172.21.15.103:0/2995302349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecf45d0 msgr2=0x563f9ecfdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:12.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96b7fff640 1 --2- 172.21.15.103:0/2995302349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecf45d0 0x563f9ecfdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.639+0000 7f96b7fff640 1 -- 172.21.15.103:0/2995302349 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f9ecfe960 con 0x563f9ecff370 2024-09-19T06:56:12.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.640+0000 7f96b7fff640 1 --2- 172.21.15.103:0/2995302349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecff370 0x563f9ecf7660 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f96a802f370 tx=0x7f96a802f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae91ad9f81f78edc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:12.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.640+0000 7f96ae7fc640 1 -- 172.21.15.103:0/2995302349 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8059070 con 0x563f9ecff370 2024-09-19T06:56:12.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.640+0000 7f96ae7fc640 1 -- 172.21.15.103:0/2995302349 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a8036040 con 0x563f9ecff370 2024-09-19T06:56:12.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.640+0000 7f96ae7fc640 1 -- 172.21.15.103:0/2995302349 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8054e60 con 0x563f9ecff370 2024-09-19T06:56:12.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.641+0000 7f96c1220740 1 -- 172.21.15.103:0/2995302349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecff370 msgr2=0x563f9ecf7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.641+0000 7f96c1220740 1 --2- 172.21.15.103:0/2995302349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecff370 0x563f9ecf7660 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f96a802f370 tx=0x7f96a802f430 comp rx=0 tx=0).stop 2024-09-19T06:56:12.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.641+0000 7f96c1220740 1 -- 172.21.15.103:0/2995302349 shutdown_connections 2024-09-19T06:56:12.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.641+0000 7f96c1220740 1 --2- 172.21.15.103:0/2995302349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecff370 0x563f9ecf7660 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.641+0000 7f96c1220740 1 --2- 172.21.15.103:0/2995302349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecf5b40 0x563f9ecf7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.641+0000 7f96c1220740 1 --2- 172.21.15.103:0/2995302349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecf45d0 0x563f9ecfdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.641+0000 7f96c1220740 1 -- 172.21.15.103:0/2995302349 >> 172.21.15.103:0/2995302349 conn(0x563f9ecbd310 msgr2=0x563f9ecfd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:12.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.641+0000 7f96c1220740 1 -- 172.21.15.103:0/2995302349 shutdown_connections 2024-09-19T06:56:12.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.641+0000 7f96c1220740 1 -- 172.21.15.103:0/2995302349 wait complete. 2024-09-19T06:56:12.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.641+0000 7f96c1220740 1 -- start start 2024-09-19T06:56:12.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.641+0000 7f96c1220740 1 Processor -- start 2024-09-19T06:56:12.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96c1220740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecf45d0 0x563f9ed10320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96c1220740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecf5b40 0x563f9ecf8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96c1220740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecff370 0x563f9ecf91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96c1220740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96b81acc50 con 0x563f9ecff370 2024-09-19T06:56:12.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96c1220740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96b81ac580 con 0x563f9ecf45d0 2024-09-19T06:56:12.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96c1220740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96b81a97e0 con 0x563f9ecf5b40 2024-09-19T06:56:12.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96c1220740 10 client.0 _ll_register_callbacks cb 0x7f96a8009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:12.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96b7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecff370 0x563f9ecf91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96b7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecff370 0x563f9ecf91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60956/0 (socket says 172.21.15.103:60956) 2024-09-19T06:56:12.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96b7fff640 1 -- 172.21.15.103:0/426163003 learned_addr learned my addr 172.21.15.103:0/426163003 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:12.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96bcbe3640 1 --2- 172.21.15.103:0/426163003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecf5b40 0x563f9ecf8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96b7fff640 1 --2- 172.21.15.103:0/426163003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecf45d0 0x563f9ed10320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96b7fff640 1 -- 172.21.15.103:0/426163003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecf5b40 msgr2=0x563f9ecf8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:12.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96b7fff640 1 --2- 172.21.15.103:0/426163003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecf5b40 0x563f9ecf8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96b7fff640 1 -- 172.21.15.103:0/426163003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecf45d0 msgr2=0x563f9ed10320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:12.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96b7fff640 1 --2- 172.21.15.103:0/426163003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecf45d0 0x563f9ed10320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96b7fff640 1 -- 172.21.15.103:0/426163003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f9ecf98f0 con 0x563f9ecff370 2024-09-19T06:56:12.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96b7fff640 1 --2- 172.21.15.103:0/426163003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecff370 0x563f9ecf91e0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f96a8005d50 tx=0x7f96a80319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c97ff8bebbd05d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:12.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f9697fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:12.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f9697fff640 1 -- 172.21.15.103:0/426163003 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8059060 con 0x563f9ecff370 2024-09-19T06:56:12.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f9697fff640 1 -- 172.21.15.103:0/426163003 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f96a8054c90 con 0x563f9ecff370 2024-09-19T06:56:12.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.642+0000 7f96c1220740 1 -- 172.21.15.103:0/426163003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96a8036040 con 0x563f9ecff370 2024-09-19T06:56:12.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.643+0000 7f9697fff640 1 -- 172.21.15.103:0/426163003 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f96a803f040 con 0x563f9ecff370 2024-09-19T06:56:12.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.643+0000 7f96c1220740 10 client.17916 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:12.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.643+0000 7f96c1220740 1 -- 172.21.15.103:0/426163003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f9ed82280 con 0x563f9ecff370 2024-09-19T06:56:12.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.643+0000 7f9697fff640 1 -- 172.21.15.103:0/426163003 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a8036040 con 0x563f9ecff370 2024-09-19T06:56:12.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.643+0000 7f9697fff640 1 -- 172.21.15.103:0/426163003 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f96a804c430 con 0x563f9ecff370 2024-09-19T06:56:12.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.643+0000 7f96c1220740 10 client.17916 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:12.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.643+0000 7f96c1220740 10 client.17916 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:56:12.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.643+0000 7f96c1220740 20 client.17916 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:12.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.643+0000 7f96c1220740 1 --2- 172.21.15.103:0/426163003 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x563f9ed82940 0x563f9ed84d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.643+0000 7f96c1220740 4 client.17916 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:56:12.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.643+0000 7f96c1220740 1 -- 172.21.15.103:0/426163003 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f9ec61d70 con 0x563f9ed82940 2024-09-19T06:56:12.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.643+0000 7f96bcbe3640 1 --2- 172.21.15.103:0/426163003 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x563f9ed82940 0x563f9ed84d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.644+0000 7f96bcbe3640 1 --2- 172.21.15.103:0/426163003 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x563f9ed82940 0x563f9ed84d60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c264d46949da793d server_cookie=b1595458f9feafd5 in_seq=0 out_seq=0 2024-09-19T06:56:12.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.644+0000 7f9697fff640 10 client.17916 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:56:12.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.644+0000 7f9697fff640 1 -- 172.21.15.103:0/426163003 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f9ec61d70 con 0x563f9ed82940 2024-09-19T06:56:12.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.644+0000 7f9697fff640 10 client.17916 handle_command_reply: tid=0 2024-09-19T06:56:12.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.644+0000 7f96c1220740 1 client.17916 shutdown 2024-09-19T06:56:12.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.644+0000 7f96c1220740 1 -- 172.21.15.103:0/426163003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecff370 msgr2=0x563f9ecf91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.644+0000 7f96c1220740 1 --2- 172.21.15.103:0/426163003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecff370 0x563f9ecf91e0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f96a8005d50 tx=0x7f96a80319b0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.644+0000 7f96c1220740 1 -- 172.21.15.103:0/426163003 shutdown_connections 2024-09-19T06:56:12.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.644+0000 7f96c1220740 1 --2- 172.21.15.103:0/426163003 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x563f9ed82940 0x563f9ed84d60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.644+0000 7f96c1220740 1 --2- 172.21.15.103:0/426163003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f9ecff370 0x563f9ecf91e0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.644+0000 7f96c1220740 1 --2- 172.21.15.103:0/426163003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f9ecf5b40 0x563f9ecf8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.644+0000 7f96c1220740 1 --2- 172.21.15.103:0/426163003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f9ecf45d0 0x563f9ed10320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.644+0000 7f9697fff640 0 client.17916 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:56:12.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.644+0000 7f96c1220740 1 -- 172.21.15.103:0/426163003 >> 172.21.15.103:0/426163003 conn(0x563f9ecbd310 msgr2=0x563f9ecfe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:12.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.644+0000 7f96c1220740 1 -- 172.21.15.103:0/426163003 shutdown_connections 2024-09-19T06:56:12.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.644+0000 7f96c1220740 1 -- 172.21.15.103:0/426163003 wait complete. 2024-09-19T06:56:12.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.645+0000 7f96c1220740 20 client.17916 trim_cache size 0 max 16384 2024-09-19T06:56:12.656 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:56:12.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.645+0000 7f96bee47640 1 -- 172.21.15.103:0/2705750521 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96a40a3430 msgr2=0x7f96a40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.645+0000 7f96bee47640 1 --2- 172.21.15.103:0/2705750521 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96a40a3430 0x7f96a40a58f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f96a009b890 tx=0x7f96a006a880 comp rx=0 tx=0).stop 2024-09-19T06:56:12.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.645+0000 7f96bee47640 1 -- 172.21.15.103:0/2705750521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b8106f10 msgr2=0x7f96b819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.645+0000 7f96bee47640 1 --2- 172.21.15.103:0/2705750521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b8106f10 0x7f96b819cd00 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f96a000c970 tx=0x7f96a000ce30 comp rx=0 tx=0).stop 2024-09-19T06:56:12.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.645+0000 7f96bee47640 1 -- 172.21.15.103:0/2705750521 shutdown_connections 2024-09-19T06:56:12.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.645+0000 7f96bee47640 1 --2- 172.21.15.103:0/2705750521 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96a40a3430 0x7f96a40a58f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.645+0000 7f96bee47640 1 --2- 172.21.15.103:0/2705750521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b8106f10 0x7f96b819cd00 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.645+0000 7f96bee47640 1 --2- 172.21.15.103:0/2705750521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96b8102790 0x7f96b8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.645+0000 7f96bee47640 1 --2- 172.21.15.103:0/2705750521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b8101de0 0x7f96b8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.645+0000 7f96bee47640 1 -- 172.21.15.103:0/2705750521 >> 172.21.15.103:0/2705750521 conn(0x7f96b80fd8c0 msgr2=0x7f96b80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:12.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.645+0000 7f96bee47640 1 -- 172.21.15.103:0/2705750521 shutdown_connections 2024-09-19T06:56:12.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:12.645+0000 7f96bee47640 1 -- 172.21.15.103:0/2705750521 wait complete. 2024-09-19T06:56:12.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:12 smithi103 ceph-mon[31158]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T06:56:12.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:12 smithi103 ceph-mon[31158]: pgmap v214: 97 pgs: 97 active+clean; 4.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 125 KiB/s wr, 45 op/s 2024-09-19T06:56:12.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:12 smithi134 ceph-mon[35356]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T06:56:12.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:12 smithi134 ceph-mon[35356]: pgmap v214: 97 pgs: 97 active+clean; 4.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 125 KiB/s wr, 45 op/s 2024-09-19T06:56:13.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:12 smithi174 ceph-mon[35637]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T06:56:13.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:12 smithi174 ceph-mon[35637]: pgmap v214: 97 pgs: 97 active+clean; 4.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 125 KiB/s wr, 45 op/s 2024-09-19T06:56:13.325 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:56:13.571 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:14.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.030+0000 7f4233b5d640 1 -- 172.21.15.103:0/1557057575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422c101de0 msgr2=0x7f422c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:14.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.030+0000 7f4233b5d640 1 --2- 172.21.15.103:0/1557057575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422c101de0 0x7f422c1021c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f4228009900 tx=0x7f422802a410 comp rx=0 tx=0).stop 2024-09-19T06:56:14.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.030+0000 7f4233b5d640 1 -- 172.21.15.103:0/1557057575 shutdown_connections 2024-09-19T06:56:14.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.030+0000 7f4233b5d640 1 --2- 172.21.15.103:0/1557057575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f422c106f10 0x7f422c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.030+0000 7f4233b5d640 1 --2- 172.21.15.103:0/1557057575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f422c102790 0x7f422c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.030+0000 7f4233b5d640 1 --2- 172.21.15.103:0/1557057575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422c101de0 0x7f422c1021c0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.030+0000 7f4233b5d640 1 -- 172.21.15.103:0/1557057575 >> 172.21.15.103:0/1557057575 conn(0x7f422c0fd8c0 msgr2=0x7f422c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:14.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.030+0000 7f4233b5d640 1 -- 172.21.15.103:0/1557057575 shutdown_connections 2024-09-19T06:56:14.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.030+0000 7f4233b5d640 1 -- 172.21.15.103:0/1557057575 wait complete. 2024-09-19T06:56:14.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.031+0000 7f4233b5d640 1 Processor -- start 2024-09-19T06:56:14.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.031+0000 7f4233b5d640 1 -- start start 2024-09-19T06:56:14.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.031+0000 7f4233b5d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422c101de0 0x7f422c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.031+0000 7f4233b5d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f422c102790 0x7f422c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.031+0000 7f4233b5d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f422c106f10 0x7f422c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.031+0000 7f4233b5d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f422c111fb0 con 0x7f422c101de0 2024-09-19T06:56:14.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.031+0000 7f4233b5d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f422c111e30 con 0x7f422c102790 2024-09-19T06:56:14.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.031+0000 7f4233b5d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f422c112130 con 0x7f422c106f10 2024-09-19T06:56:14.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.031+0000 7f42318f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422c101de0 0x7f422c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.031+0000 7f42318f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422c101de0 0x7f422c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60982/0 (socket says 172.21.15.103:60982) 2024-09-19T06:56:14.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.031+0000 7f42318f9640 1 -- 172.21.15.103:0/2888053827 learned_addr learned my addr 172.21.15.103:0/2888053827 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:14.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.032+0000 7f42320fa640 1 --2- 172.21.15.103:0/2888053827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f422c106f10 0x7f422c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.032+0000 7f42318f9640 1 -- 172.21.15.103:0/2888053827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f422c106f10 msgr2=0x7f422c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:14.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.032+0000 7f42310f8640 1 --2- 172.21.15.103:0/2888053827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f422c102790 0x7f422c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.032+0000 7f42318f9640 1 --2- 172.21.15.103:0/2888053827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f422c106f10 0x7f422c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.032+0000 7f42318f9640 1 -- 172.21.15.103:0/2888053827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f422c102790 msgr2=0x7f422c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:14.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.032+0000 7f42318f9640 1 --2- 172.21.15.103:0/2888053827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f422c102790 0x7f422c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.032+0000 7f42318f9640 1 -- 172.21.15.103:0/2888053827 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422c1a6960 con 0x7f422c101de0 2024-09-19T06:56:14.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.032+0000 7f42320fa640 1 --2- 172.21.15.103:0/2888053827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f422c106f10 0x7f422c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:14.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.032+0000 7f42318f9640 1 --2- 172.21.15.103:0/2888053827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422c101de0 0x7f422c0744f0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f4228006010 tx=0x7f42280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:14.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.032+0000 7f4222ffd640 1 -- 172.21.15.103:0/2888053827 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422803f030 con 0x7f422c101de0 2024-09-19T06:56:14.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.032+0000 7f4222ffd640 1 -- 172.21.15.103:0/2888053827 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4228003ce0 con 0x7f422c101de0 2024-09-19T06:56:14.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.032+0000 7f4233b5d640 1 -- 172.21.15.103:0/2888053827 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422c1a6bf0 con 0x7f422c101de0 2024-09-19T06:56:14.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.032+0000 7f4222ffd640 1 -- 172.21.15.103:0/2888053827 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228035070 con 0x7f422c101de0 2024-09-19T06:56:14.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.032+0000 7f4233b5d640 1 -- 172.21.15.103:0/2888053827 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f422c1a70d0 con 0x7f422c101de0 2024-09-19T06:56:14.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.034+0000 7f4222ffd640 1 -- 172.21.15.103:0/2888053827 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4228042030 con 0x7f422c101de0 2024-09-19T06:56:14.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f4235f36740 1 Processor -- start 2024-09-19T06:56:14.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f4235f36740 1 -- start start 2024-09-19T06:56:14.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f4235f36740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fcb40 0x55f5484fb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f4222ffd640 1 --2- 172.21.15.103:0/2888053827 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f42180a33b0 0x7f42180a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f4235f36740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fb5d0 0x55f5484fba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f4235f36740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f548508760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f4235f36740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f5484612f0 con 0x55f5484fb5d0 2024-09-19T06:56:14.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f4235f36740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f548408cf0 con 0x55f5484fcb40 2024-09-19T06:56:14.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f4235f36740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f54845d2b0 con 0x55f548506370 2024-09-19T06:56:14.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f42310f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fb5d0 0x55f5484fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f42318f9640 1 --2- 172.21.15.103:0/2888053827 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f42180a33b0 0x7f42180a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f42310f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fb5d0 0x55f5484fba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60996/0 (socket says 172.21.15.103:60996) 2024-09-19T06:56:14.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f42310f8640 1 -- 172.21.15.103:0/1213436958 learned_addr learned my addr 172.21.15.103:0/1213436958 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:14.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f42320fa640 1 --2- 172.21.15.103:0/1213436958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fcb40 0x55f5484fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f42320fa640 1 --2- 172.21.15.103:0/1213436958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f548508760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f4222ffd640 1 -- 172.21.15.103:0/2888053827 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f42280f0300 con 0x7f422c101de0 2024-09-19T06:56:14.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.035+0000 7f42310f8640 1 -- 172.21.15.103:0/1213436958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 msgr2=0x55f548508760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:14.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.036+0000 7f42310f8640 1 --2- 172.21.15.103:0/1213436958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f548508760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.036+0000 7f42310f8640 1 -- 172.21.15.103:0/1213436958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fcb40 msgr2=0x55f5484fb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:14.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.036+0000 7f42310f8640 1 --2- 172.21.15.103:0/1213436958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fcb40 0x55f5484fb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.036+0000 7f42310f8640 1 -- 172.21.15.103:0/1213436958 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5483ed3a0 con 0x55f5484fb5d0 2024-09-19T06:56:14.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.036+0000 7f42320fa640 1 --2- 172.21.15.103:0/1213436958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f548508760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:14.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.036+0000 7f42310f8640 1 --2- 172.21.15.103:0/1213436958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fb5d0 0x55f5484fba10 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f421c009b60 tx=0x7f421c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a821f6edfd9860 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:14.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.036+0000 7f42318f9640 1 --2- 172.21.15.103:0/2888053827 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f42180a33b0 0x7f42180a5870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f42280edae0 tx=0x7f42280b8320 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:14.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.036+0000 7f4217fff640 1 -- 172.21.15.103:0/1213436958 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c016070 con 0x55f5484fb5d0 2024-09-19T06:56:14.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.036+0000 7f4217fff640 1 -- 172.21.15.103:0/1213436958 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f421c009e50 con 0x55f5484fb5d0 2024-09-19T06:56:14.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.036+0000 7f4217fff640 1 -- 172.21.15.103:0/1213436958 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c0107c0 con 0x55f5484fb5d0 2024-09-19T06:56:14.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.037+0000 7f4235f36740 1 -- 172.21.15.103:0/1213436958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fb5d0 msgr2=0x55f5484fba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:14.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.037+0000 7f4235f36740 1 --2- 172.21.15.103:0/1213436958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fb5d0 0x55f5484fba10 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f421c009b60 tx=0x7f421c00e6c0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.037+0000 7f4235f36740 1 -- 172.21.15.103:0/1213436958 shutdown_connections 2024-09-19T06:56:14.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.037+0000 7f4235f36740 1 --2- 172.21.15.103:0/1213436958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f548508760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.037+0000 7f4235f36740 1 --2- 172.21.15.103:0/1213436958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fb5d0 0x55f5484fba10 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.037+0000 7f4235f36740 1 --2- 172.21.15.103:0/1213436958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fcb40 0x55f5484fb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.037+0000 7f4235f36740 1 -- 172.21.15.103:0/1213436958 >> 172.21.15.103:0/1213436958 conn(0x55f5484c4310 msgr2=0x55f5484f7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:14.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.037+0000 7f4235f36740 1 -- 172.21.15.103:0/1213436958 shutdown_connections 2024-09-19T06:56:14.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.037+0000 7f4235f36740 1 -- 172.21.15.103:0/1213436958 wait complete. 2024-09-19T06:56:14.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.037+0000 7f4235f36740 1 -- start start 2024-09-19T06:56:14.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.037+0000 7f4235f36740 1 Processor -- start 2024-09-19T06:56:14.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f4235f36740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fb5d0 0x55f548505cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f4235f36740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fcb40 0x55f548502f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f4235f36740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f5485034d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f4235f36740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f548509070 con 0x55f5484fcb40 2024-09-19T06:56:14.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f4235f36740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f54841c060 con 0x55f5484fb5d0 2024-09-19T06:56:14.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f4235f36740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f5485091f0 con 0x55f548506370 2024-09-19T06:56:14.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f4235f36740 10 client.0 _ll_register_callbacks cb 0x55f54816f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:14.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f42310f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fcb40 0x55f548502f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f42310f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fcb40 0x55f548502f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32768/0 (socket says 172.21.15.103:32768) 2024-09-19T06:56:14.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f42310f8640 1 -- 172.21.15.103:0/122187061 learned_addr learned my addr 172.21.15.103:0/122187061 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:14.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f42320fa640 1 --2- 172.21.15.103:0/122187061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fb5d0 0x55f548505cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f42320fa640 1 --2- 172.21.15.103:0/122187061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f5485034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f42310f8640 1 -- 172.21.15.103:0/122187061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 msgr2=0x55f5485034d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:14.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f42310f8640 1 --2- 172.21.15.103:0/122187061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f5485034d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f42310f8640 1 -- 172.21.15.103:0/122187061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fb5d0 msgr2=0x55f548505cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:14.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f42310f8640 1 --2- 172.21.15.103:0/122187061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fb5d0 0x55f548505cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f42310f8640 1 -- 172.21.15.103:0/122187061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5481d23c0 con 0x55f5484fcb40 2024-09-19T06:56:14.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f42310f8640 1 --2- 172.21.15.103:0/122187061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fcb40 0x55f548502f70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f421c011f70 tx=0x7f421c0103f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12457eaf57eac1a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f42157fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f42157fa640 1 -- 172.21.15.103:0/122187061 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c016050 con 0x55f5484fcb40 2024-09-19T06:56:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f42157fa640 1 -- 172.21.15.103:0/122187061 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f421c011680 con 0x55f5484fcb40 2024-09-19T06:56:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.038+0000 7f4235f36740 1 -- 172.21.15.103:0/122187061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f421c009e50 con 0x55f5484fcb40 2024-09-19T06:56:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.039+0000 7f42157fa640 1 -- 172.21.15.103:0/122187061 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f421c030050 con 0x55f5484fcb40 2024-09-19T06:56:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.039+0000 7f4235f36740 10 client.17928 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.039+0000 7f4235f36740 1 -- 172.21.15.103:0/122187061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f548514ab0 con 0x55f5484fcb40 2024-09-19T06:56:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.039+0000 7f42157fa640 1 -- 172.21.15.103:0/122187061 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f421c009e50 con 0x55f5484fcb40 2024-09-19T06:56:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.039+0000 7f42157fa640 1 -- 172.21.15.103:0/122187061 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f421c02b030 con 0x55f5484fcb40 2024-09-19T06:56:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.039+0000 7f4235f36740 10 client.17928 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.039+0000 7f4235f36740 10 client.17928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.039+0000 7f4235f36740 20 client.17928 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.039+0000 7f4235f36740 1 --2- 172.21.15.103:0/122187061 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f548515eb0 0x55f5485182a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.039+0000 7f4235f36740 4 client.17928 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.039+0000 7f4235f36740 1 -- 172.21.15.103:0/122187061 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f54846e860 con 0x55f548515eb0 2024-09-19T06:56:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.039+0000 7f42320fa640 1 --2- 172.21.15.103:0/122187061 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f548515eb0 0x55f5485182a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.040+0000 7f42157fa640 10 client.17928 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.040+0000 7f42320fa640 1 --2- 172.21.15.103:0/122187061 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f548515eb0 0x55f5485182a0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6da772bce21f9a5b server_cookie=7028d1b8cf2005c5 in_seq=0 out_seq=0 2024-09-19T06:56:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.040+0000 7f42157fa640 1 -- 172.21.15.103:0/122187061 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f54846e860 con 0x55f548515eb0 2024-09-19T06:56:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.040+0000 7f42157fa640 10 client.17928 handle_command_reply: tid=0 2024-09-19T06:56:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.040+0000 7f4235f36740 1 client.17928 shutdown 2024-09-19T06:56:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.041+0000 7f4235f36740 1 -- 172.21.15.103:0/122187061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fcb40 msgr2=0x55f548502f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:14.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.041+0000 7f4235f36740 1 --2- 172.21.15.103:0/122187061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fcb40 0x55f548502f70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f421c011f70 tx=0x7f421c0103f0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.041+0000 7f4235f36740 1 -- 172.21.15.103:0/122187061 shutdown_connections 2024-09-19T06:56:14.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.041+0000 7f4235f36740 1 --2- 172.21.15.103:0/122187061 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f548515eb0 0x55f5485182a0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.041+0000 7f4235f36740 1 --2- 172.21.15.103:0/122187061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f5485034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.041+0000 7f4235f36740 1 --2- 172.21.15.103:0/122187061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fcb40 0x55f548502f70 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.041+0000 7f4235f36740 1 --2- 172.21.15.103:0/122187061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fb5d0 0x55f548505cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.041+0000 7f42157fa640 0 client.17928 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:14.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.041+0000 7f4235f36740 1 -- 172.21.15.103:0/122187061 >> 172.21.15.103:0/122187061 conn(0x55f5484c4310 msgr2=0x55f5484fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:14.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.041+0000 7f4235f36740 1 -- 172.21.15.103:0/122187061 shutdown_connections 2024-09-19T06:56:14.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.041+0000 7f4235f36740 1 -- 172.21.15.103:0/122187061 wait complete. 2024-09-19T06:56:14.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.041+0000 7f4235f36740 20 client.17928 trim_cache size 0 max 16384 2024-09-19T06:56:14.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f4235f36740 1 Processor -- start 2024-09-19T06:56:14.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f4235f36740 1 -- start start 2024-09-19T06:56:14.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f4235f36740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fb5d0 0x55f548512340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f4235f36740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fcb40 0x55f5484fe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f4235f36740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f5484fe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f4235f36740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1afac0 con 0x55f5484fcb40 2024-09-19T06:56:14.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f4235f36740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1b1600 con 0x55f5484fb5d0 2024-09-19T06:56:14.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f4235f36740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1b0190 con 0x55f548506370 2024-09-19T06:56:14.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f42310f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fcb40 0x55f5484fe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f42310f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fcb40 0x55f5484fe120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32776/0 (socket says 172.21.15.103:32776) 2024-09-19T06:56:14.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f42310f8640 1 -- 172.21.15.103:0/1706221465 learned_addr learned my addr 172.21.15.103:0/1706221465 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:14.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f42320fa640 1 --2- 172.21.15.103:0/1706221465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fb5d0 0x55f548512340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f42320fa640 1 --2- 172.21.15.103:0/1706221465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f5484fe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f42310f8640 1 -- 172.21.15.103:0/1706221465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 msgr2=0x55f5484fe660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:14.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f42310f8640 1 --2- 172.21.15.103:0/1706221465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f5484fe660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f42310f8640 1 -- 172.21.15.103:0/1706221465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fb5d0 msgr2=0x55f548512340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:14.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f42310f8640 1 --2- 172.21.15.103:0/1706221465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fb5d0 0x55f548512340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f42310f8640 1 -- 172.21.15.103:0/1706221465 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5484feba0 con 0x55f5484fcb40 2024-09-19T06:56:14.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f42320fa640 1 --2- 172.21.15.103:0/1706221465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fb5d0 0x55f548512340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:14.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.055+0000 7f42310f8640 1 --2- 172.21.15.103:0/1706221465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fcb40 0x55f5484fe120 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f421c0053d0 tx=0x7f421c0054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523f457da6daa9bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:14.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.056+0000 7f42177fe640 1 -- 172.21.15.103:0/1706221465 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c016070 con 0x55f5484fcb40 2024-09-19T06:56:14.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.056+0000 7f42177fe640 1 -- 172.21.15.103:0/1706221465 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f421c009c50 con 0x55f5484fcb40 2024-09-19T06:56:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.056+0000 7f42177fe640 1 -- 172.21.15.103:0/1706221465 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c0118c0 con 0x55f5484fcb40 2024-09-19T06:56:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.056+0000 7f4235f36740 1 -- 172.21.15.103:0/1706221465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fcb40 msgr2=0x55f5484fe120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.056+0000 7f4235f36740 1 --2- 172.21.15.103:0/1706221465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fcb40 0x55f5484fe120 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f421c0053d0 tx=0x7f421c0054f0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.056+0000 7f4235f36740 1 -- 172.21.15.103:0/1706221465 shutdown_connections 2024-09-19T06:56:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.056+0000 7f4235f36740 1 --2- 172.21.15.103:0/1706221465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f5484fe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.056+0000 7f4235f36740 1 --2- 172.21.15.103:0/1706221465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fcb40 0x55f5484fe120 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.056+0000 7f4235f36740 1 --2- 172.21.15.103:0/1706221465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fb5d0 0x55f548512340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.056+0000 7f4235f36740 1 -- 172.21.15.103:0/1706221465 >> 172.21.15.103:0/1706221465 conn(0x55f5484c4310 msgr2=0x55f548504ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.056+0000 7f4235f36740 1 -- 172.21.15.103:0/1706221465 shutdown_connections 2024-09-19T06:56:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.056+0000 7f4235f36740 1 -- 172.21.15.103:0/1706221465 wait complete. 2024-09-19T06:56:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.057+0000 7f4235f36740 1 -- start start 2024-09-19T06:56:14.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.057+0000 7f4235f36740 1 Processor -- start 2024-09-19T06:56:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.057+0000 7f4235f36740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fb5d0 0x55f54850e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.057+0000 7f4235f36740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fcb40 0x55f54850ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.057+0000 7f4235f36740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f5484ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.057+0000 7f4235f36740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1b0f30 con 0x55f5484fb5d0 2024-09-19T06:56:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.057+0000 7f4235f36740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1b0860 con 0x55f5484fcb40 2024-09-19T06:56:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.057+0000 7f4235f36740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f422c1adac0 con 0x55f548506370 2024-09-19T06:56:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.057+0000 7f4235f36740 10 client.0 _ll_register_callbacks cb 0x7f421c0099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.057+0000 7f42320fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fb5d0 0x55f54850e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.057+0000 7f42320fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fb5d0 0x55f54850e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32790/0 (socket says 172.21.15.103:32790) 2024-09-19T06:56:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.057+0000 7f42320fa640 1 -- 172.21.15.103:0/3137376044 learned_addr learned my addr 172.21.15.103:0/3137376044 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.058+0000 7f42310f8640 1 --2- 172.21.15.103:0/3137376044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fcb40 0x55f54850ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.058+0000 7f42320fa640 1 --2- 172.21.15.103:0/3137376044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f5484ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.058+0000 7f42320fa640 1 -- 172.21.15.103:0/3137376044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 msgr2=0x55f5484ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.058+0000 7f42320fa640 1 --2- 172.21.15.103:0/3137376044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f5484ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.058+0000 7f42320fa640 1 -- 172.21.15.103:0/3137376044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fcb40 msgr2=0x55f54850ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.058+0000 7f42320fa640 1 --2- 172.21.15.103:0/3137376044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fcb40 0x55f54850ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.058+0000 7f42320fa640 1 -- 172.21.15.103:0/3137376044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5484fffe0 con 0x55f5484fb5d0 2024-09-19T06:56:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.058+0000 7f42320fa640 1 --2- 172.21.15.103:0/3137376044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fb5d0 0x55f54850e700 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f42240145a0 tx=0x7f4224016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0838f4598969dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.058+0000 7f42157fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.058+0000 7f42157fa640 1 -- 172.21.15.103:0/3137376044 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422400fe50 con 0x55f5484fb5d0 2024-09-19T06:56:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.058+0000 7f4235f36740 1 -- 172.21.15.103:0/3137376044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f421c009c50 con 0x55f5484fb5d0 2024-09-19T06:56:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.058+0000 7f42157fa640 1 -- 172.21.15.103:0/3137376044 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f422401d390 con 0x55f5484fb5d0 2024-09-19T06:56:14.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.058+0000 7f42157fa640 1 -- 172.21.15.103:0/3137376044 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4224030050 con 0x55f5484fb5d0 2024-09-19T06:56:14.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.058+0000 7f4235f36740 10 client.17934 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:14.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.058+0000 7f4235f36740 1 -- 172.21.15.103:0/3137376044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5485803f0 con 0x55f5484fb5d0 2024-09-19T06:56:14.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.058+0000 7f42157fa640 1 -- 172.21.15.103:0/3137376044 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f421c009c50 con 0x55f5484fb5d0 2024-09-19T06:56:14.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.059+0000 7f42157fa640 1 -- 172.21.15.103:0/3137376044 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f422401d070 con 0x55f5484fb5d0 2024-09-19T06:56:14.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.059+0000 7f4235f36740 10 client.17934 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:14.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.059+0000 7f4235f36740 10 client.17934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:14.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.059+0000 7f4235f36740 20 client.17934 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:14.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.059+0000 7f4235f36740 1 --2- 172.21.15.103:0/3137376044 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f548580af0 0x55f548582f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:14.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.059+0000 7f4235f36740 4 client.17934 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:56:14.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.059+0000 7f4235f36740 1 -- 172.21.15.103:0/3137376044 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f548468d70 con 0x55f548580af0 2024-09-19T06:56:14.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.059+0000 7f42310f8640 1 --2- 172.21.15.103:0/3137376044 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f548580af0 0x55f548582f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:14.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.059+0000 7f42310f8640 1 --2- 172.21.15.103:0/3137376044 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f548580af0 0x55f548582f10 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=479c1d59162656c7 server_cookie=d14bd106cbe46833 in_seq=0 out_seq=0 2024-09-19T06:56:14.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.059+0000 7f42157fa640 10 client.17934 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:14.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.059+0000 7f42157fa640 1 -- 172.21.15.103:0/3137376044 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f548468d70 con 0x55f548580af0 2024-09-19T06:56:14.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.059+0000 7f42157fa640 10 client.17934 handle_command_reply: tid=0 2024-09-19T06:56:14.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4235f36740 1 client.17934 shutdown 2024-09-19T06:56:14.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4235f36740 1 -- 172.21.15.103:0/3137376044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fb5d0 msgr2=0x55f54850e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:14.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4235f36740 1 --2- 172.21.15.103:0/3137376044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fb5d0 0x55f54850e700 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f42240145a0 tx=0x7f4224016040 comp rx=0 tx=0).stop 2024-09-19T06:56:14.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4235f36740 1 -- 172.21.15.103:0/3137376044 shutdown_connections 2024-09-19T06:56:14.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4235f36740 1 --2- 172.21.15.103:0/3137376044 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f548580af0 0x55f548582f10 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4235f36740 1 --2- 172.21.15.103:0/3137376044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f548506370 0x55f5484ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f42157fa640 0 client.17934 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:14.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4235f36740 1 --2- 172.21.15.103:0/3137376044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5484fcb40 0x55f54850ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4235f36740 1 --2- 172.21.15.103:0/3137376044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5484fb5d0 0x55f54850e700 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4235f36740 1 -- 172.21.15.103:0/3137376044 >> 172.21.15.103:0/3137376044 conn(0x55f5484c4310 msgr2=0x55f5485047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:14.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4235f36740 1 -- 172.21.15.103:0/3137376044 shutdown_connections 2024-09-19T06:56:14.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4235f36740 1 -- 172.21.15.103:0/3137376044 wait complete. 2024-09-19T06:56:14.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4235f36740 20 client.17934 trim_cache size 0 max 16384 2024-09-19T06:56:14.074 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:56:14.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4233b5d640 1 -- 172.21.15.103:0/2888053827 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f42180a33b0 msgr2=0x7f42180a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:14.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4233b5d640 1 --2- 172.21.15.103:0/2888053827 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f42180a33b0 0x7f42180a5870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f42280edae0 tx=0x7f42280b8320 comp rx=0 tx=0).stop 2024-09-19T06:56:14.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4233b5d640 1 -- 172.21.15.103:0/2888053827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422c101de0 msgr2=0x7f422c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:14.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4233b5d640 1 --2- 172.21.15.103:0/2888053827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422c101de0 0x7f422c0744f0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f4228006010 tx=0x7f42280042f0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4233b5d640 1 -- 172.21.15.103:0/2888053827 shutdown_connections 2024-09-19T06:56:14.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.060+0000 7f4233b5d640 1 --2- 172.21.15.103:0/2888053827 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f42180a33b0 0x7f42180a5870 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.061+0000 7f4233b5d640 1 --2- 172.21.15.103:0/2888053827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f422c106f10 0x7f422c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.061+0000 7f4233b5d640 1 --2- 172.21.15.103:0/2888053827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f422c102790 0x7f422c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.061+0000 7f4233b5d640 1 --2- 172.21.15.103:0/2888053827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f422c101de0 0x7f422c0744f0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:14.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.061+0000 7f4233b5d640 1 -- 172.21.15.103:0/2888053827 >> 172.21.15.103:0/2888053827 conn(0x7f422c0fd8c0 msgr2=0x7f422c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:14.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.061+0000 7f4233b5d640 1 -- 172.21.15.103:0/2888053827 shutdown_connections 2024-09-19T06:56:14.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:14.061+0000 7f4233b5d640 1 -- 172.21.15.103:0/2888053827 wait complete. 2024-09-19T06:56:14.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:14 smithi103 ceph-mon[31158]: pgmap v215: 97 pgs: 97 active+clean; 4.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-09-19T06:56:14.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:14 smithi134 ceph-mon[35356]: pgmap v215: 97 pgs: 97 active+clean; 4.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-09-19T06:56:14.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:14 smithi174 ceph-mon[35637]: pgmap v215: 97 pgs: 97 active+clean; 4.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-09-19T06:56:15.758 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:56:15.758 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:56:16.009 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:16.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:16 smithi103 ceph-mon[31158]: pgmap v216: 97 pgs: 97 active+clean; 4.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T06:56:16.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:16 smithi134 ceph-mon[35356]: pgmap v216: 97 pgs: 97 active+clean; 4.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T06:56:16.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.476+0000 7f6dee24f640 1 -- 172.21.15.103:0/2251218867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de8102d40 msgr2=0x7f6de8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:16.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.476+0000 7f6dee24f640 1 --2- 172.21.15.103:0/2251218867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de8102d40 0x7f6de8103120 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009ca0 tx=0x7f6dd802a950 comp rx=0 tx=0).stop 2024-09-19T06:56:16.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.476+0000 7f6dee24f640 1 -- 172.21.15.103:0/2251218867 shutdown_connections 2024-09-19T06:56:16.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.476+0000 7f6dee24f640 1 --2- 172.21.15.103:0/2251218867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6de810c170 0x7f6de810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.476+0000 7f6dee24f640 1 --2- 172.21.15.103:0/2251218867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6de8103660 0x7f6de810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.476+0000 7f6dee24f640 1 --2- 172.21.15.103:0/2251218867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de8102d40 0x7f6de8103120 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.476+0000 7f6dee24f640 1 -- 172.21.15.103:0/2251218867 >> 172.21.15.103:0/2251218867 conn(0x7f6de80fc7a0 msgr2=0x7f6de80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:16.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.476+0000 7f6dee24f640 1 -- 172.21.15.103:0/2251218867 shutdown_connections 2024-09-19T06:56:16.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.476+0000 7f6dee24f640 1 -- 172.21.15.103:0/2251218867 wait complete. 2024-09-19T06:56:16.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.476+0000 7f6dee24f640 1 Processor -- start 2024-09-19T06:56:16.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6dee24f640 1 -- start start 2024-09-19T06:56:16.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6dee24f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6de8102d40 0x7f6de8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6dee24f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de8103660 0x7f6de8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6dee24f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6de810c170 0x7f6de819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6dee24f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6de8111f70 con 0x7f6de8103660 2024-09-19T06:56:16.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6dee24f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6de8111df0 con 0x7f6de810c170 2024-09-19T06:56:16.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6dee24f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81120f0 con 0x7f6de8102d40 2024-09-19T06:56:16.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6de6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de8103660 0x7f6de8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6de6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de8103660 0x7f6de8198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32800/0 (socket says 172.21.15.103:32800) 2024-09-19T06:56:16.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6de6ffd640 1 -- 172.21.15.103:0/2189175142 learned_addr learned my addr 172.21.15.103:0/2189175142 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:16.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6de77fe640 1 --2- 172.21.15.103:0/2189175142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6de8102d40 0x7f6de8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6de7fff640 1 --2- 172.21.15.103:0/2189175142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6de810c170 0x7f6de819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6de6ffd640 1 -- 172.21.15.103:0/2189175142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6de8102d40 msgr2=0x7f6de8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:16.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6de6ffd640 1 --2- 172.21.15.103:0/2189175142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6de8102d40 0x7f6de8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6de6ffd640 1 -- 172.21.15.103:0/2189175142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6de810c170 msgr2=0x7f6de819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:16.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6de6ffd640 1 --2- 172.21.15.103:0/2189175142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6de810c170 0x7f6de819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6de6ffd640 1 -- 172.21.15.103:0/2189175142 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de819d3e0 con 0x7f6de8103660 2024-09-19T06:56:16.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6de77fe640 1 --2- 172.21.15.103:0/2189175142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6de8102d40 0x7f6de8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:16.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6de7fff640 1 --2- 172.21.15.103:0/2189175142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6de810c170 0x7f6de819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:16.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6de6ffd640 1 --2- 172.21.15.103:0/2189175142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de8103660 0x7f6de8198970 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00c940 tx=0x7f6ddc00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:16.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6de4ff9640 1 -- 172.21.15.103:0/2189175142 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc004220 con 0x7f6de8103660 2024-09-19T06:56:16.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.477+0000 7f6de4ff9640 1 -- 172.21.15.103:0/2189175142 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ddc0043c0 con 0x7f6de8103660 2024-09-19T06:56:16.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.478+0000 7f6dee24f640 1 -- 172.21.15.103:0/2189175142 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de819d6d0 con 0x7f6de8103660 2024-09-19T06:56:16.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.478+0000 7f6de4ff9640 1 -- 172.21.15.103:0/2189175142 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc004a00 con 0x7f6de8103660 2024-09-19T06:56:16.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.478+0000 7f6dee24f640 1 -- 172.21.15.103:0/2189175142 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de81a4fb0 con 0x7f6de8103660 2024-09-19T06:56:16.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.480+0000 7f6de4ff9640 1 -- 172.21.15.103:0/2189175142 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f6ddc0079b0 con 0x7f6de8103660 2024-09-19T06:56:16.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.480+0000 7f6df0628740 1 Processor -- start 2024-09-19T06:56:16.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.480+0000 7f6df0628740 1 -- start start 2024-09-19T06:56:16.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.480+0000 7f6de4ff9640 1 --2- 172.21.15.103:0/2189175142 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6dd00a33b0 0x7f6dd00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.480+0000 7f6df0628740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c8b60 0x556e127c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.480+0000 7f6df0628740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127c9480 0x556e127d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.480+0000 7f6df0628740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127d3d40 0x556e127d6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.480+0000 7f6df0628740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556e1272e280 con 0x556e127c9480 2024-09-19T06:56:16.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.480+0000 7f6df0628740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556e126d5b10 con 0x556e127c8b60 2024-09-19T06:56:16.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.480+0000 7f6df0628740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556e1272a240 con 0x556e127d3d40 2024-09-19T06:56:16.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.480+0000 7f6de7fff640 1 --2- 172.21.15.103:0/2189175142 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6dd00a33b0 0x7f6dd00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.481+0000 7f6de6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127c9480 0x556e127d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.481+0000 7f6de4ff9640 1 -- 172.21.15.103:0/2189175142 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f6ddc0cfdf0 con 0x7f6de8103660 2024-09-19T06:56:16.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.481+0000 7f6de6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127c9480 0x556e127d3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32810/0 (socket says 172.21.15.103:32810) 2024-09-19T06:56:16.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.481+0000 7f6de6ffd640 1 -- 172.21.15.103:0/2624488461 learned_addr learned my addr 172.21.15.103:0/2624488461 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:16.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.481+0000 7f6de77fe640 1 --2- 172.21.15.103:0/2624488461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c8b60 0x556e127c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.481+0000 7f6de7fff640 1 --2- 172.21.15.103:0/2624488461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127d3d40 0x556e127d6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.481+0000 7f6de6ffd640 1 -- 172.21.15.103:0/2624488461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127d3d40 msgr2=0x556e127d6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:16.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.481+0000 7f6de6ffd640 1 --2- 172.21.15.103:0/2624488461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127d3d40 0x556e127d6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.481+0000 7f6de6ffd640 1 -- 172.21.15.103:0/2624488461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c8b60 msgr2=0x556e127c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:16.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.481+0000 7f6de6ffd640 1 --2- 172.21.15.103:0/2624488461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c8b60 0x556e127c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.481+0000 7f6de6ffd640 1 -- 172.21.15.103:0/2624488461 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e126e21f0 con 0x556e127c9480 2024-09-19T06:56:16.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.481+0000 7f6de7fff640 1 --2- 172.21.15.103:0/2189175142 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6dd00a33b0 0x7f6dd00a5870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6de000be20 tx=0x7f6de0014840 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:16.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.481+0000 7f6de6ffd640 1 --2- 172.21.15.103:0/2624488461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127c9480 0x556e127d3800 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0058c0 tx=0x7f6ddc00a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843e97b6c1bc9172 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:16.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.481+0000 7f6dcdffb640 1 -- 172.21.15.103:0/2624488461 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc01d040 con 0x556e127c9480 2024-09-19T06:56:16.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.481+0000 7f6dcdffb640 1 -- 172.21.15.103:0/2624488461 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ddc09c7b0 con 0x556e127c9480 2024-09-19T06:56:16.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.481+0000 7f6dcdffb640 1 -- 172.21.15.103:0/2624488461 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc064440 con 0x556e127c9480 2024-09-19T06:56:16.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.482+0000 7f6df0628740 1 -- 172.21.15.103:0/2624488461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127c9480 msgr2=0x556e127d3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:16.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.482+0000 7f6df0628740 1 --2- 172.21.15.103:0/2624488461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127c9480 0x556e127d3800 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0058c0 tx=0x7f6ddc00a540 comp rx=0 tx=0).stop 2024-09-19T06:56:16.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.482+0000 7f6df0628740 1 -- 172.21.15.103:0/2624488461 shutdown_connections 2024-09-19T06:56:16.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.482+0000 7f6df0628740 1 --2- 172.21.15.103:0/2624488461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127d3d40 0x556e127d6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.482+0000 7f6df0628740 1 --2- 172.21.15.103:0/2624488461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127c9480 0x556e127d3800 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.482+0000 7f6df0628740 1 --2- 172.21.15.103:0/2624488461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c8b60 0x556e127c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.482+0000 7f6df0628740 1 -- 172.21.15.103:0/2624488461 >> 172.21.15.103:0/2624488461 conn(0x556e127c50f0 msgr2=0x556e127c54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:16.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.482+0000 7f6df0628740 1 -- 172.21.15.103:0/2624488461 shutdown_connections 2024-09-19T06:56:16.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.482+0000 7f6df0628740 1 -- 172.21.15.103:0/2624488461 wait complete. 2024-09-19T06:56:16.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.482+0000 7f6df0628740 1 -- start start 2024-09-19T06:56:16.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6df0628740 1 Processor -- start 2024-09-19T06:56:16.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6df0628740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c8b60 0x556e127d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6df0628740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127c9480 0x556e127d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6df0628740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127d3d40 0x556e127d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6df0628740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556e127dad20 con 0x556e127d3d40 2024-09-19T06:56:16.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6df0628740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556e126e8f10 con 0x556e127c8b60 2024-09-19T06:56:16.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6df0628740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556e127daea0 con 0x556e127c9480 2024-09-19T06:56:16.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6df0628740 10 client.0 _ll_register_callbacks cb 0x556e126b1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:16.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6de7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127d3d40 0x556e127d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6de7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127d3d40 0x556e127d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32812/0 (socket says 172.21.15.103:32812) 2024-09-19T06:56:16.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6de7fff640 1 -- 172.21.15.103:0/3993016833 learned_addr learned my addr 172.21.15.103:0/3993016833 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:16.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6de77fe640 1 --2- 172.21.15.103:0/3993016833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c8b60 0x556e127d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6de6ffd640 1 --2- 172.21.15.103:0/3993016833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127c9480 0x556e127d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6de7fff640 1 -- 172.21.15.103:0/3993016833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127c9480 msgr2=0x556e127d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:16.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6de7fff640 1 --2- 172.21.15.103:0/3993016833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127c9480 0x556e127d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6de7fff640 1 -- 172.21.15.103:0/3993016833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c8b60 msgr2=0x556e127d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:16.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6de7fff640 1 --2- 172.21.15.103:0/3993016833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c8b60 0x556e127d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6de7fff640 1 -- 172.21.15.103:0/3993016833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e1249f6a0 con 0x556e127d3d40 2024-09-19T06:56:16.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.483+0000 7f6de6ffd640 1 --2- 172.21.15.103:0/3993016833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127c9480 0x556e127d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:16.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.484+0000 7f6de7fff640 1 --2- 172.21.15.103:0/3993016833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127d3d40 0x556e127d16a0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f6de00117a0 tx=0x7f6de0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac14fdfe7c17651 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:16.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.484+0000 7f6dc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:16.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.484+0000 7f6dc6ffd640 1 -- 172.21.15.103:0/3993016833 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0012bd0 con 0x556e127d3d40 2024-09-19T06:56:16.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.484+0000 7f6dc6ffd640 1 -- 172.21.15.103:0/3993016833 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f6de0012d70 con 0x556e127d3d40 2024-09-19T06:56:16.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.484+0000 7f6df0628740 1 -- 172.21.15.103:0/3993016833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ddc09c7b0 con 0x556e127d3d40 2024-09-19T06:56:16.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.484+0000 7f6dc6ffd640 1 -- 172.21.15.103:0/3993016833 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6de001e410 con 0x556e127d3d40 2024-09-19T06:56:16.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.484+0000 7f6df0628740 10 client.17946 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:16.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.484+0000 7f6df0628740 1 -- 172.21.15.103:0/3993016833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e127e6980 con 0x556e127d3d40 2024-09-19T06:56:16.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.484+0000 7f6dc6ffd640 1 -- 172.21.15.103:0/3993016833 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ddc09c7b0 con 0x556e127d3d40 2024-09-19T06:56:16.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.484+0000 7f6dc6ffd640 1 -- 172.21.15.103:0/3993016833 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6de001d070 con 0x556e127d3d40 2024-09-19T06:56:16.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.484+0000 7f6df0628740 10 client.17946 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:16.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.484+0000 7f6df0628740 10 client.17946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:16.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.484+0000 7f6df0628740 20 client.17946 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:16.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.484+0000 7f6df0628740 1 --2- 172.21.15.103:0/3993016833 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556e127e7e40 0x556e127ea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.484+0000 7f6df0628740 4 client.17946 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:16.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.484+0000 7f6df0628740 1 -- 172.21.15.103:0/3993016833 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e1273b7f0 con 0x556e127e7e40 2024-09-19T06:56:16.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.485+0000 7f6de77fe640 1 --2- 172.21.15.103:0/3993016833 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556e127e7e40 0x556e127ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.485+0000 7f6dc6ffd640 10 client.17946 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:16.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.485+0000 7f6de77fe640 1 --2- 172.21.15.103:0/3993016833 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556e127e7e40 0x556e127ea230 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13085307c2e56d7d server_cookie=677174f875a8abaf in_seq=0 out_seq=0 2024-09-19T06:56:16.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.486+0000 7f6dc6ffd640 1 -- 172.21.15.103:0/3993016833 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e1273b7f0 con 0x556e127e7e40 2024-09-19T06:56:16.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.486+0000 7f6dc6ffd640 10 client.17946 handle_command_reply: tid=0 2024-09-19T06:56:16.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.486+0000 7f6df0628740 1 client.17946 shutdown 2024-09-19T06:56:16.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.486+0000 7f6df0628740 1 -- 172.21.15.103:0/3993016833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127d3d40 msgr2=0x556e127d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:16.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.486+0000 7f6df0628740 1 --2- 172.21.15.103:0/3993016833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127d3d40 0x556e127d16a0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f6de00117a0 tx=0x7f6de0004730 comp rx=0 tx=0).stop 2024-09-19T06:56:16.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.486+0000 7f6df0628740 1 -- 172.21.15.103:0/3993016833 shutdown_connections 2024-09-19T06:56:16.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.486+0000 7f6df0628740 1 --2- 172.21.15.103:0/3993016833 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556e127e7e40 0x556e127ea230 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.486+0000 7f6df0628740 1 --2- 172.21.15.103:0/3993016833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127d3d40 0x556e127d16a0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.486+0000 7f6df0628740 1 --2- 172.21.15.103:0/3993016833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127c9480 0x556e127d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.486+0000 7f6dc6ffd640 0 client.17946 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:16.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.486+0000 7f6df0628740 1 --2- 172.21.15.103:0/3993016833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c8b60 0x556e127d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.486+0000 7f6df0628740 1 -- 172.21.15.103:0/3993016833 >> 172.21.15.103:0/3993016833 conn(0x556e127c50f0 msgr2=0x556e127cae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:16.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.486+0000 7f6df0628740 1 -- 172.21.15.103:0/3993016833 shutdown_connections 2024-09-19T06:56:16.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.486+0000 7f6df0628740 1 -- 172.21.15.103:0/3993016833 wait complete. 2024-09-19T06:56:16.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.486+0000 7f6df0628740 20 client.17946 trim_cache size 0 max 16384 2024-09-19T06:56:16.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.503+0000 7f6df0628740 1 Processor -- start 2024-09-19T06:56:16.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6df0628740 1 -- start start 2024-09-19T06:56:16.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6df0628740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127c8b60 0x556e127cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6df0628740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c9480 0x556e127d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6df0628740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127d3d40 0x556e127efb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6df0628740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81ab7a0 con 0x556e127d3d40 2024-09-19T06:56:16.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6df0628740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81ad2e0 con 0x556e127c9480 2024-09-19T06:56:16.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6df0628740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81abe70 con 0x556e127c8b60 2024-09-19T06:56:16.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6de7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127d3d40 0x556e127efb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6de7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127d3d40 0x556e127efb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32814/0 (socket says 172.21.15.103:32814) 2024-09-19T06:56:16.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6de7fff640 1 -- 172.21.15.103:0/3305988872 learned_addr learned my addr 172.21.15.103:0/3305988872 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:16.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6de77fe640 1 --2- 172.21.15.103:0/3305988872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127c8b60 0x556e127cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6de6ffd640 1 --2- 172.21.15.103:0/3305988872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c9480 0x556e127d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6de7fff640 1 -- 172.21.15.103:0/3305988872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127c8b60 msgr2=0x556e127cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:16.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6de7fff640 1 --2- 172.21.15.103:0/3305988872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127c8b60 0x556e127cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6de7fff640 1 -- 172.21.15.103:0/3305988872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c9480 msgr2=0x556e127d2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:16.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6de7fff640 1 --2- 172.21.15.103:0/3305988872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c9480 0x556e127d2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6de7fff640 1 -- 172.21.15.103:0/3305988872 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e127bf680 con 0x556e127d3d40 2024-09-19T06:56:16.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6de6ffd640 1 --2- 172.21.15.103:0/3305988872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c9480 0x556e127d2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:16.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.504+0000 7f6de7fff640 1 --2- 172.21.15.103:0/3305988872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127d3d40 0x556e127efb00 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f6de000b470 tx=0x7f6de0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c42e54366cb923 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:16.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.505+0000 7f6dcd7fa640 1 -- 172.21.15.103:0/3305988872 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0032070 con 0x556e127d3d40 2024-09-19T06:56:16.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.505+0000 7f6dcd7fa640 1 -- 172.21.15.103:0/3305988872 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de0012d00 con 0x556e127d3d40 2024-09-19T06:56:16.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.505+0000 7f6dcd7fa640 1 -- 172.21.15.103:0/3305988872 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de001eb60 con 0x556e127d3d40 2024-09-19T06:56:16.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.505+0000 7f6df0628740 1 -- 172.21.15.103:0/3305988872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127d3d40 msgr2=0x556e127efb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:16.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.505+0000 7f6df0628740 1 --2- 172.21.15.103:0/3305988872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127d3d40 0x556e127efb00 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f6de000b470 tx=0x7f6de0005c50 comp rx=0 tx=0).stop 2024-09-19T06:56:16.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.505+0000 7f6df0628740 1 -- 172.21.15.103:0/3305988872 shutdown_connections 2024-09-19T06:56:16.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.505+0000 7f6df0628740 1 --2- 172.21.15.103:0/3305988872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127d3d40 0x556e127efb00 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.505+0000 7f6df0628740 1 --2- 172.21.15.103:0/3305988872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c9480 0x556e127d2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.505+0000 7f6df0628740 1 --2- 172.21.15.103:0/3305988872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127c8b60 0x556e127cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.505+0000 7f6df0628740 1 -- 172.21.15.103:0/3305988872 >> 172.21.15.103:0/3305988872 conn(0x556e127c50f0 msgr2=0x556e127d2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:16.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.505+0000 7f6df0628740 1 -- 172.21.15.103:0/3305988872 shutdown_connections 2024-09-19T06:56:16.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.505+0000 7f6df0628740 1 -- 172.21.15.103:0/3305988872 wait complete. 2024-09-19T06:56:16.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6df0628740 1 -- start start 2024-09-19T06:56:16.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6df0628740 1 Processor -- start 2024-09-19T06:56:16.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6df0628740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127c8b60 0x556e127dc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6df0628740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c9480 0x556e127dcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6df0628740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127d3d40 0x556e1284bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6df0628740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81acc10 con 0x556e127c8b60 2024-09-19T06:56:16.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6df0628740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81ac540 con 0x556e127c9480 2024-09-19T06:56:16.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6df0628740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81a97a0 con 0x556e127d3d40 2024-09-19T06:56:16.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6df0628740 10 client.0 _ll_register_callbacks cb 0x7f6de0011470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:16.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6de77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127c8b60 0x556e127dc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6de77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127c8b60 0x556e127dc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32824/0 (socket says 172.21.15.103:32824) 2024-09-19T06:56:16.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6de77fe640 1 -- 172.21.15.103:0/525491915 learned_addr learned my addr 172.21.15.103:0/525491915 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:16.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6de7fff640 1 --2- 172.21.15.103:0/525491915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127d3d40 0x556e1284bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6de77fe640 1 -- 172.21.15.103:0/525491915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127d3d40 msgr2=0x556e1284bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:16.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6de77fe640 1 --2- 172.21.15.103:0/525491915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127d3d40 0x556e1284bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6de77fe640 1 -- 172.21.15.103:0/525491915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c9480 msgr2=0x556e127dcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:16.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.506+0000 7f6de77fe640 1 --2- 172.21.15.103:0/525491915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c9480 0x556e127dcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6de77fe640 1 -- 172.21.15.103:0/525491915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e1284c150 con 0x556e127c8b60 2024-09-19T06:56:16.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6de7fff640 1 --2- 172.21.15.103:0/525491915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127d3d40 0x556e1284bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:16.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6de77fe640 1 --2- 172.21.15.103:0/525491915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127c8b60 0x556e127dc700 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009ca0 tx=0x7f6dd802fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5993f2b699f93062 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:16.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6dc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:16.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6dc6ffd640 1 -- 172.21.15.103:0/525491915 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd804b070 con 0x556e127c8b60 2024-09-19T06:56:16.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6dc6ffd640 1 -- 172.21.15.103:0/525491915 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f6dd8003970 con 0x556e127c8b60 2024-09-19T06:56:16.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6df0628740 1 -- 172.21.15.103:0/525491915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de0012d00 con 0x556e127c8b60 2024-09-19T06:56:16.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6dc6ffd640 1 -- 172.21.15.103:0/525491915 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6dd8057050 con 0x556e127c8b60 2024-09-19T06:56:16.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6df0628740 10 client.17952 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:16.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6df0628740 1 -- 172.21.15.103:0/525491915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e12853510 con 0x556e127c8b60 2024-09-19T06:56:16.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6dc6ffd640 1 -- 172.21.15.103:0/525491915 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de0012d00 con 0x556e127c8b60 2024-09-19T06:56:16.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6dc6ffd640 1 -- 172.21.15.103:0/525491915 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6dd8046420 con 0x556e127c8b60 2024-09-19T06:56:16.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6df0628740 10 client.17952 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:16.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6df0628740 10 client.17952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:16.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6df0628740 20 client.17952 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:16.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6df0628740 1 --2- 172.21.15.103:0/525491915 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556e12853c10 0x556e12856070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:16.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6df0628740 4 client.17952 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:56:16.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.507+0000 7f6df0628740 1 -- 172.21.15.103:0/525491915 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556e12735d00 con 0x556e12853c10 2024-09-19T06:56:16.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.508+0000 7f6de6ffd640 1 --2- 172.21.15.103:0/525491915 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556e12853c10 0x556e12856070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:16.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.508+0000 7f6de6ffd640 1 --2- 172.21.15.103:0/525491915 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556e12853c10 0x556e12856070 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4da09c45a0e480 server_cookie=b75b545f3c960b60 in_seq=0 out_seq=0 2024-09-19T06:56:16.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.508+0000 7f6dc6ffd640 10 client.17952 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:16.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.509+0000 7f6dc6ffd640 1 -- 172.21.15.103:0/525491915 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556e12735d00 con 0x556e12853c10 2024-09-19T06:56:16.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.509+0000 7f6dc6ffd640 10 client.17952 handle_command_reply: tid=0 2024-09-19T06:56:16.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.509+0000 7f6df0628740 1 client.17952 shutdown 2024-09-19T06:56:16.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.509+0000 7f6df0628740 1 -- 172.21.15.103:0/525491915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127c8b60 msgr2=0x556e127dc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:16.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.509+0000 7f6df0628740 1 --2- 172.21.15.103:0/525491915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127c8b60 0x556e127dc700 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009ca0 tx=0x7f6dd802fbe0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.509+0000 7f6df0628740 1 -- 172.21.15.103:0/525491915 shutdown_connections 2024-09-19T06:56:16.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.509+0000 7f6df0628740 1 --2- 172.21.15.103:0/525491915 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556e12853c10 0x556e12856070 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.509+0000 7f6df0628740 1 --2- 172.21.15.103:0/525491915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e127d3d40 0x556e1284bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.509+0000 7f6df0628740 1 --2- 172.21.15.103:0/525491915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e127c9480 0x556e127dcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.509+0000 7f6dc6ffd640 0 client.17952 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:16.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.509+0000 7f6df0628740 1 --2- 172.21.15.103:0/525491915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e127c8b60 0x556e127dc700 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.509+0000 7f6df0628740 1 -- 172.21.15.103:0/525491915 >> 172.21.15.103:0/525491915 conn(0x556e127c50f0 msgr2=0x556e127d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:16.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.509+0000 7f6df0628740 1 -- 172.21.15.103:0/525491915 shutdown_connections 2024-09-19T06:56:16.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.509+0000 7f6df0628740 1 -- 172.21.15.103:0/525491915 wait complete. 2024-09-19T06:56:16.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.509+0000 7f6df0628740 20 client.17952 trim_cache size 0 max 16384 2024-09-19T06:56:16.526 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:56:16.526 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T06:56:16.526 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "b6e66b50-5561-43bb-a4ba-5320fa6d12f4", 2024-09-19T06:56:16.526 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T06:56:16.526 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:56:16.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.510+0000 7f6dee24f640 1 -- 172.21.15.103:0/2189175142 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6dd00a33b0 msgr2=0x7f6dd00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:16.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.510+0000 7f6dee24f640 1 --2- 172.21.15.103:0/2189175142 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6dd00a33b0 0x7f6dd00a5870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6de000be20 tx=0x7f6de0014840 comp rx=0 tx=0).stop 2024-09-19T06:56:16.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.510+0000 7f6dee24f640 1 -- 172.21.15.103:0/2189175142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de8103660 msgr2=0x7f6de8198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:16.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.510+0000 7f6dee24f640 1 --2- 172.21.15.103:0/2189175142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de8103660 0x7f6de8198970 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00c940 tx=0x7f6ddc00ce00 comp rx=0 tx=0).stop 2024-09-19T06:56:16.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.510+0000 7f6dee24f640 1 -- 172.21.15.103:0/2189175142 shutdown_connections 2024-09-19T06:56:16.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.510+0000 7f6dee24f640 1 --2- 172.21.15.103:0/2189175142 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6dd00a33b0 0x7f6dd00a5870 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.510+0000 7f6dee24f640 1 --2- 172.21.15.103:0/2189175142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6de810c170 0x7f6de819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.510+0000 7f6dee24f640 1 --2- 172.21.15.103:0/2189175142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de8103660 0x7f6de8198970 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.510+0000 7f6dee24f640 1 --2- 172.21.15.103:0/2189175142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6de8102d40 0x7f6de8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:16.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.510+0000 7f6dee24f640 1 -- 172.21.15.103:0/2189175142 >> 172.21.15.103:0/2189175142 conn(0x7f6de80fc7a0 msgr2=0x7f6de80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:16.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.510+0000 7f6dee24f640 1 -- 172.21.15.103:0/2189175142 shutdown_connections 2024-09-19T06:56:16.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:16.510+0000 7f6dee24f640 1 -- 172.21.15.103:0/2189175142 wait complete. 2024-09-19T06:56:16.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:16 smithi174 ceph-mon[35637]: pgmap v216: 97 pgs: 97 active+clean; 4.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T06:56:17.245 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:56:17.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:17 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T06:56:17.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:17 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:56:17.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:17 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T06:56:17.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:17 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:56:17.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:17 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T06:56:17.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:17 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:56:17.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:17 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T06:56:17.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:17 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:56:17.492 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:17.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:17 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T06:56:17.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:17 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:56:17.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:17 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T06:56:17.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:17 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:56:17.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.924+0000 7f0d2ad00640 1 -- 172.21.15.103:0/1004631500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d241036e0 msgr2=0x7f0d24107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:17.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.924+0000 7f0d2ad00640 1 --2- 172.21.15.103:0/1004631500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d241036e0 0x7f0d24107b90 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009930 tx=0x7f0d0c02a420 comp rx=0 tx=0).stop 2024-09-19T06:56:17.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.924+0000 7f0d2ad00640 1 -- 172.21.15.103:0/1004631500 shutdown_connections 2024-09-19T06:56:17.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.924+0000 7f0d2ad00640 1 --2- 172.21.15.103:0/1004631500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d241082a0 0x7f0d2410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.924+0000 7f0d2ad00640 1 --2- 172.21.15.103:0/1004631500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d241036e0 0x7f0d24107b90 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.924+0000 7f0d2ad00640 1 --2- 172.21.15.103:0/1004631500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d24102d30 0x7f0d24103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.924+0000 7f0d2ad00640 1 -- 172.21.15.103:0/1004631500 >> 172.21.15.103:0/1004631500 conn(0x7f0d240fc7d0 msgr2=0x7f0d240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:17.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.924+0000 7f0d2ad00640 1 -- 172.21.15.103:0/1004631500 shutdown_connections 2024-09-19T06:56:17.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.924+0000 7f0d2ad00640 1 -- 172.21.15.103:0/1004631500 wait complete. 2024-09-19T06:56:17.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.924+0000 7f0d2ad00640 1 Processor -- start 2024-09-19T06:56:17.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.924+0000 7f0d2ad00640 1 -- start start 2024-09-19T06:56:17.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d2ad00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d24102d30 0x7f0d241985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d2ad00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d241036e0 0x7f0d24198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d2ad00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d241082a0 0x7f0d2419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d2ad00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d241120c0 con 0x7f0d241036e0 2024-09-19T06:56:17.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d2ad00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d24111f40 con 0x7f0d241082a0 2024-09-19T06:56:17.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d2ad00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d24112240 con 0x7f0d24102d30 2024-09-19T06:56:17.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d23fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d241036e0 0x7f0d24198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d23fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d241036e0 0x7f0d24198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35220/0 (socket says 172.21.15.103:35220) 2024-09-19T06:56:17.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d23fff640 1 -- 172.21.15.103:0/1575327179 learned_addr learned my addr 172.21.15.103:0/1575327179 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:17.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d28a9c640 1 --2- 172.21.15.103:0/1575327179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d24102d30 0x7f0d241985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d2929d640 1 --2- 172.21.15.103:0/1575327179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d241082a0 0x7f0d2419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d23fff640 1 -- 172.21.15.103:0/1575327179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d24102d30 msgr2=0x7f0d241985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:17.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d23fff640 1 --2- 172.21.15.103:0/1575327179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d24102d30 0x7f0d241985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d23fff640 1 -- 172.21.15.103:0/1575327179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d241082a0 msgr2=0x7f0d2419cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:17.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d23fff640 1 --2- 172.21.15.103:0/1575327179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d241082a0 0x7f0d2419cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d23fff640 1 -- 172.21.15.103:0/1575327179 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2419d580 con 0x7f0d241036e0 2024-09-19T06:56:17.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d28a9c640 1 --2- 172.21.15.103:0/1575327179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d24102d30 0x7f0d241985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:17.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d2929d640 1 --2- 172.21.15.103:0/1575327179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d241082a0 0x7f0d2419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:17.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d23fff640 1 --2- 172.21.15.103:0/1575327179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d241036e0 0x7f0d24198b10 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c005940 tx=0x7f0d0c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:17.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d21ffb640 1 -- 172.21.15.103:0/1575327179 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c03f030 con 0x7f0d241036e0 2024-09-19T06:56:17.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d21ffb640 1 -- 172.21.15.103:0/1575327179 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d0c003ce0 con 0x7f0d241036e0 2024-09-19T06:56:17.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d2ad00640 1 -- 172.21.15.103:0/1575327179 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2419d810 con 0x7f0d241036e0 2024-09-19T06:56:17.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.925+0000 7f0d21ffb640 1 -- 172.21.15.103:0/1575327179 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c035070 con 0x7f0d241036e0 2024-09-19T06:56:17.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.926+0000 7f0d2ad00640 1 -- 172.21.15.103:0/1575327179 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d241a5150 con 0x7f0d241036e0 2024-09-19T06:56:17.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.928+0000 7f0d21ffb640 1 -- 172.21.15.103:0/1575327179 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0d0c042030 con 0x7f0d241036e0 2024-09-19T06:56:17.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.928+0000 7f0d2d0d9740 1 Processor -- start 2024-09-19T06:56:17.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.928+0000 7f0d2d0d9740 1 -- start start 2024-09-19T06:56:17.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.928+0000 7f0d2d0d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685e4a0 0x560e6685e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.928+0000 7f0d2d0d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e6685edc0 0x560e66869140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.928+0000 7f0d2d0d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e66869680 0x560e6686ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.928+0000 7f0d2d0d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560e667c42e0 con 0x560e66869680 2024-09-19T06:56:17.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.928+0000 7f0d2d0d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560e6676bce0 con 0x560e6685edc0 2024-09-19T06:56:17.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.928+0000 7f0d2d0d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560e667c02a0 con 0x560e6685e4a0 2024-09-19T06:56:17.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.928+0000 7f0d21ffb640 1 --2- 172.21.15.103:0/1575327179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0d100a33b0 0x7f0d100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.928+0000 7f0d23fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e66869680 0x560e6686ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.928+0000 7f0d23fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e66869680 0x560e6686ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35222/0 (socket says 172.21.15.103:35222) 2024-09-19T06:56:17.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.928+0000 7f0d23fff640 1 -- 172.21.15.103:0/37595141 learned_addr learned my addr 172.21.15.103:0/37595141 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:17.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.929+0000 7f0d28a9c640 1 --2- 172.21.15.103:0/37595141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e6685edc0 0x560e66869140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.929+0000 7f0d2929d640 1 --2- 172.21.15.103:0/1575327179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0d100a33b0 0x7f0d100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.929+0000 7f0d2929d640 1 --2- 172.21.15.103:0/37595141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685e4a0 0x560e6685e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.929+0000 7f0d21ffb640 1 -- 172.21.15.103:0/1575327179 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f0d0c0f03b0 con 0x7f0d241036e0 2024-09-19T06:56:17.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.929+0000 7f0d23fff640 1 -- 172.21.15.103:0/37595141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685e4a0 msgr2=0x560e6685e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:17.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.929+0000 7f0d23fff640 1 --2- 172.21.15.103:0/37595141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685e4a0 0x560e6685e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.929+0000 7f0d23fff640 1 -- 172.21.15.103:0/37595141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e6685edc0 msgr2=0x560e66869140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:17.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.929+0000 7f0d23fff640 1 --2- 172.21.15.103:0/37595141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e6685edc0 0x560e66869140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.929+0000 7f0d23fff640 1 -- 172.21.15.103:0/37595141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e66750440 con 0x560e66869680 2024-09-19T06:56:17.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.929+0000 7f0d2929d640 1 --2- 172.21.15.103:0/1575327179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0d100a33b0 0x7f0d100a5870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c013990 tx=0x7f0d1c00d770 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:17.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.929+0000 7f0d23fff640 1 --2- 172.21.15.103:0/37595141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e66869680 0x560e6686ba70 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c031550 tx=0x7f0d0c040fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883971ae45c93d92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:17.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.929+0000 7f0d1affd640 1 -- 172.21.15.103:0/37595141 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c049070 con 0x560e66869680 2024-09-19T06:56:17.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.929+0000 7f0d1affd640 1 -- 172.21.15.103:0/37595141 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d0c0bcd70 con 0x560e66869680 2024-09-19T06:56:17.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.929+0000 7f0d1affd640 1 -- 172.21.15.103:0/37595141 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c003800 con 0x560e66869680 2024-09-19T06:56:17.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.930+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/37595141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e66869680 msgr2=0x560e6686ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:17.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.930+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/37595141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e66869680 0x560e6686ba70 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c031550 tx=0x7f0d0c040fb0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.930+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/37595141 shutdown_connections 2024-09-19T06:56:17.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.930+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/37595141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e66869680 0x560e6686ba70 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.930+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/37595141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e6685edc0 0x560e66869140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.930+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/37595141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685e4a0 0x560e6685e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.930+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/37595141 >> 172.21.15.103:0/37595141 conn(0x560e6685aa30 msgr2=0x560e6685ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:17.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.930+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/37595141 shutdown_connections 2024-09-19T06:56:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.930+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/37595141 wait complete. 2024-09-19T06:56:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.930+0000 7f0d2d0d9740 1 -- start start 2024-09-19T06:56:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.930+0000 7f0d2d0d9740 1 Processor -- start 2024-09-19T06:56:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d2d0d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685e4a0 0x560e66868ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d2d0d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685edc0 0x560e668662a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d2d0d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e66869680 0x560e66866800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d2d0d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560e6686c010 con 0x560e6685edc0 2024-09-19T06:56:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d2d0d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560e6677f050 con 0x560e66869680 2024-09-19T06:56:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d2d0d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560e6686c190 con 0x560e6685e4a0 2024-09-19T06:56:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d2d0d9740 10 client.0 _ll_register_callbacks cb 0x560e667bf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d23fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685edc0 0x560e668662a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d23fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685edc0 0x560e668662a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35228/0 (socket says 172.21.15.103:35228) 2024-09-19T06:56:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d23fff640 1 -- 172.21.15.103:0/2254424459 learned_addr learned my addr 172.21.15.103:0/2254424459 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:17.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d2929d640 1 --2- 172.21.15.103:0/2254424459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e66869680 0x560e66866800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d28a9c640 1 --2- 172.21.15.103:0/2254424459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685e4a0 0x560e66868ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d23fff640 1 -- 172.21.15.103:0/2254424459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685e4a0 msgr2=0x560e66868ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:17.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d23fff640 1 --2- 172.21.15.103:0/2254424459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685e4a0 0x560e66868ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d23fff640 1 -- 172.21.15.103:0/2254424459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e66869680 msgr2=0x560e66866800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:17.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d23fff640 1 --2- 172.21.15.103:0/2254424459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e66869680 0x560e66866800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d23fff640 1 -- 172.21.15.103:0/2254424459 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e665353c0 con 0x560e6685edc0 2024-09-19T06:56:17.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d28a9c640 1 --2- 172.21.15.103:0/2254424459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685e4a0 0x560e66868ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:17.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d23fff640 1 --2- 172.21.15.103:0/2254424459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685edc0 0x560e668662a0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0edb90 tx=0x7f0d0c0bc4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b697ee3f86d82557 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:17.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.931+0000 7f0d03fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:17.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.932+0000 7f0d03fff640 1 -- 172.21.15.103:0/2254424459 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c049070 con 0x560e6685edc0 2024-09-19T06:56:17.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.932+0000 7f0d03fff640 1 -- 172.21.15.103:0/2254424459 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f0d0c003800 con 0x560e6685edc0 2024-09-19T06:56:17.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.932+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/2254424459 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d0c0bcd70 con 0x560e6685edc0 2024-09-19T06:56:17.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.932+0000 7f0d03fff640 1 -- 172.21.15.103:0/2254424459 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0d0c0c0a20 con 0x560e6685edc0 2024-09-19T06:56:17.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.932+0000 7f0d2d0d9740 10 client.17964 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:17.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.932+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/2254424459 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e6686c010 con 0x560e6685edc0 2024-09-19T06:56:17.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.932+0000 7f0d03fff640 1 -- 172.21.15.103:0/2254424459 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d0c0bcd70 con 0x560e6685edc0 2024-09-19T06:56:17.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.932+0000 7f0d03fff640 1 -- 172.21.15.103:0/2254424459 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0d0c03bc50 con 0x560e6685edc0 2024-09-19T06:56:17.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.932+0000 7f0d2d0d9740 10 client.17964 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:17.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.932+0000 7f0d2d0d9740 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:17.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.932+0000 7f0d2d0d9740 20 client.17964 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:17.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.932+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/2254424459 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560e66878a50 0x560e6687ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.932+0000 7f0d2d0d9740 4 client.17964 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:17.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.932+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/2254424459 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e667d1850 con 0x560e66878a50 2024-09-19T06:56:17.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.933+0000 7f0d28a9c640 1 --2- 172.21.15.103:0/2254424459 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560e66878a50 0x560e6687ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.933+0000 7f0d03fff640 10 client.17964 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:17.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.933+0000 7f0d28a9c640 1 --2- 172.21.15.103:0/2254424459 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560e66878a50 0x560e6687ae40 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2960563c4222d39 server_cookie=f6c97dc7f19f8317 in_seq=0 out_seq=0 2024-09-19T06:56:17.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.934+0000 7f0d03fff640 1 -- 172.21.15.103:0/2254424459 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e667d1850 con 0x560e66878a50 2024-09-19T06:56:17.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.934+0000 7f0d03fff640 10 client.17964 handle_command_reply: tid=0 2024-09-19T06:56:17.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.934+0000 7f0d2d0d9740 1 client.17964 shutdown 2024-09-19T06:56:17.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.934+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/2254424459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685edc0 msgr2=0x560e668662a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:17.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.934+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/2254424459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685edc0 0x560e668662a0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0edb90 tx=0x7f0d0c0bc4f0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.934+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/2254424459 shutdown_connections 2024-09-19T06:56:17.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.934+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/2254424459 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560e66878a50 0x560e6687ae40 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.934+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/2254424459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e66869680 0x560e66866800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.934+0000 7f0d03fff640 0 client.17964 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:17.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.934+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/2254424459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685edc0 0x560e668662a0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.934+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/2254424459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685e4a0 0x560e66868ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.934+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/2254424459 >> 172.21.15.103:0/2254424459 conn(0x560e6685aa30 msgr2=0x560e66860710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:17.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.934+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/2254424459 shutdown_connections 2024-09-19T06:56:17.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.934+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/2254424459 wait complete. 2024-09-19T06:56:17.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.934+0000 7f0d2d0d9740 20 client.17964 trim_cache size 0 max 16384 2024-09-19T06:56:17.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.951+0000 7f0d2d0d9740 1 Processor -- start 2024-09-19T06:56:17.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.951+0000 7f0d2d0d9740 1 -- start start 2024-09-19T06:56:17.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.951+0000 7f0d2d0d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685e4a0 0x560e668751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d2d0d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685edc0 0x560e668685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d2d0d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e66869680 0x560e668805b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d2d0d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d241ad940 con 0x560e6685edc0 2024-09-19T06:56:17.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d2d0d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d241af480 con 0x560e66869680 2024-09-19T06:56:17.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d2d0d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d241ae010 con 0x560e6685e4a0 2024-09-19T06:56:17.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d23fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685edc0 0x560e668685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d23fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685edc0 0x560e668685d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35234/0 (socket says 172.21.15.103:35234) 2024-09-19T06:56:17.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d23fff640 1 -- 172.21.15.103:0/3044747897 learned_addr learned my addr 172.21.15.103:0/3044747897 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:17.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d28a9c640 1 --2- 172.21.15.103:0/3044747897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685e4a0 0x560e668751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d2929d640 1 --2- 172.21.15.103:0/3044747897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e66869680 0x560e668805b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d23fff640 1 -- 172.21.15.103:0/3044747897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685e4a0 msgr2=0x560e668751e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:17.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d23fff640 1 --2- 172.21.15.103:0/3044747897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685e4a0 0x560e668751e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d23fff640 1 -- 172.21.15.103:0/3044747897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e66869680 msgr2=0x560e668805b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:17.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d23fff640 1 --2- 172.21.15.103:0/3044747897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e66869680 0x560e668805b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d23fff640 1 -- 172.21.15.103:0/3044747897 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e668647f0 con 0x560e6685edc0 2024-09-19T06:56:17.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d23fff640 1 --2- 172.21.15.103:0/3044747897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685edc0 0x560e668685d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009900 tx=0x7f0d0c03b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d75149bcd6625bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:17.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d1a7fc640 1 -- 172.21.15.103:0/3044747897 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c049070 con 0x560e6685edc0 2024-09-19T06:56:17.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d1a7fc640 1 -- 172.21.15.103:0/3044747897 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d0c0bcd70 con 0x560e6685edc0 2024-09-19T06:56:17.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.952+0000 7f0d1a7fc640 1 -- 172.21.15.103:0/3044747897 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0bc670 con 0x560e6685edc0 2024-09-19T06:56:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.953+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/3044747897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685edc0 msgr2=0x560e668685d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.953+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/3044747897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685edc0 0x560e668685d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009900 tx=0x7f0d0c03b720 comp rx=0 tx=0).stop 2024-09-19T06:56:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.953+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/3044747897 shutdown_connections 2024-09-19T06:56:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.953+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/3044747897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e66869680 0x560e668805b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.953+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/3044747897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685edc0 0x560e668685d0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.953+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/3044747897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685e4a0 0x560e668751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.953+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/3044747897 >> 172.21.15.103:0/3044747897 conn(0x560e6685aa30 msgr2=0x560e66860710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.953+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/3044747897 shutdown_connections 2024-09-19T06:56:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.953+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/3044747897 wait complete. 2024-09-19T06:56:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.953+0000 7f0d2d0d9740 1 -- start start 2024-09-19T06:56:17.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d2d0d9740 1 Processor -- start 2024-09-19T06:56:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d2d0d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685e4a0 0x560e6686d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d2d0d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685edc0 0x560e6686d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d2d0d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e66869680 0x560e6686dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d2d0d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d241aedb0 con 0x560e6685e4a0 2024-09-19T06:56:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d2d0d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d241ae6e0 con 0x560e66869680 2024-09-19T06:56:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d2d0d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d241ab940 con 0x560e6685edc0 2024-09-19T06:56:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d2d0d9740 10 client.0 _ll_register_callbacks cb 0x7f0d14009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d28a9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685e4a0 0x560e6686d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d28a9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685e4a0 0x560e6686d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35238/0 (socket says 172.21.15.103:35238) 2024-09-19T06:56:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d28a9c640 1 -- 172.21.15.103:0/661414471 learned_addr learned my addr 172.21.15.103:0/661414471 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d23fff640 1 --2- 172.21.15.103:0/661414471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685edc0 0x560e6686d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d28a9c640 1 -- 172.21.15.103:0/661414471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685edc0 msgr2=0x560e6686d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:17.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d28a9c640 1 --2- 172.21.15.103:0/661414471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685edc0 0x560e6686d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d28a9c640 1 -- 172.21.15.103:0/661414471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e66869680 msgr2=0x560e6686dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d28a9c640 1 --2- 172.21.15.103:0/661414471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e66869680 0x560e6686dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.954+0000 7f0d28a9c640 1 -- 172.21.15.103:0/661414471 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e668e26e0 con 0x560e6685e4a0 2024-09-19T06:56:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.955+0000 7f0d28a9c640 1 --2- 172.21.15.103:0/661414471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685e4a0 0x560e6686d2a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x560e6687b4f0 tx=0x7f0d1400cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf19b0a202fad7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.955+0000 7f0d03fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.955+0000 7f0d03fff640 1 -- 172.21.15.103:0/661414471 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d14026060 con 0x560e6685e4a0 2024-09-19T06:56:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.955+0000 7f0d03fff640 1 -- 172.21.15.103:0/661414471 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f0d14019930 con 0x560e6685e4a0 2024-09-19T06:56:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.955+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/661414471 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d0c0bcd70 con 0x560e6685e4a0 2024-09-19T06:56:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.955+0000 7f0d03fff640 1 -- 172.21.15.103:0/661414471 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0d14019b30 con 0x560e6685e4a0 2024-09-19T06:56:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.955+0000 7f0d2d0d9740 10 client.17970 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.955+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/661414471 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e668e3a10 con 0x560e6685e4a0 2024-09-19T06:56:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.955+0000 7f0d03fff640 1 -- 172.21.15.103:0/661414471 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d0c0bcd70 con 0x560e6685e4a0 2024-09-19T06:56:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.955+0000 7f0d03fff640 1 -- 172.21.15.103:0/661414471 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0d14017490 con 0x560e6685e4a0 2024-09-19T06:56:17.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.955+0000 7f0d2d0d9740 10 client.17970 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.955+0000 7f0d2d0d9740 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.955+0000 7f0d2d0d9740 20 client.17970 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.955+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/661414471 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560e668e4140 0x560e668e65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.955+0000 7f0d2d0d9740 4 client.17970 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:56:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.955+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/661414471 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e667cbd60 con 0x560e668e4140 2024-09-19T06:56:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.956+0000 7f0d23fff640 1 --2- 172.21.15.103:0/661414471 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560e668e4140 0x560e668e65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.956+0000 7f0d03fff640 10 client.17970 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.956+0000 7f0d23fff640 1 --2- 172.21.15.103:0/661414471 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560e668e4140 0x560e668e65a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e317220bfb1fb68b server_cookie=89b174db9c1a72bb in_seq=0 out_seq=0 2024-09-19T06:56:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.956+0000 7f0d03fff640 1 -- 172.21.15.103:0/661414471 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560e667cbd60 con 0x560e668e4140 2024-09-19T06:56:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.956+0000 7f0d03fff640 10 client.17970 handle_command_reply: tid=0 2024-09-19T06:56:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.956+0000 7f0d2d0d9740 1 client.17970 shutdown 2024-09-19T06:56:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.956+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/661414471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685e4a0 msgr2=0x560e6686d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:17.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.956+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/661414471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685e4a0 0x560e6686d2a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x560e6687b4f0 tx=0x7f0d1400cd40 comp rx=0 tx=0).stop 2024-09-19T06:56:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.956+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/661414471 shutdown_connections 2024-09-19T06:56:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.956+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/661414471 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560e668e4140 0x560e668e65a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.956+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/661414471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e66869680 0x560e6686dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.956+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/661414471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e6685edc0 0x560e6686d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.956+0000 7f0d03fff640 0 client.17970 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.956+0000 7f0d2d0d9740 1 --2- 172.21.15.103:0/661414471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e6685e4a0 0x560e6686d2a0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.956+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/661414471 >> 172.21.15.103:0/661414471 conn(0x560e6685aa30 msgr2=0x560e66864a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.957+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/661414471 shutdown_connections 2024-09-19T06:56:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.957+0000 7f0d2d0d9740 1 -- 172.21.15.103:0/661414471 wait complete. 2024-09-19T06:56:17.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.957+0000 7f0d2d0d9740 20 client.17970 trim_cache size 0 max 16384 2024-09-19T06:56:17.972 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:56:17.973 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:56:17.973 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T06:56:17.973 INFO:teuthology.orchestra.run.smithi103.stdout: "b6e66b50-5561-43bb-a4ba-5320fa6d12f4": { 2024-09-19T06:56:17.973 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T06:56:17.973 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "b6e66b50-5561-43bb-a4ba-5320fa6d12f4", 2024-09-19T06:56:17.973 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T06:56:17.973 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:56:17.973 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:56:17.973 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:56:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.957+0000 7f0d2ad00640 1 -- 172.21.15.103:0/1575327179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0d100a33b0 msgr2=0x7f0d100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.957+0000 7f0d2ad00640 1 --2- 172.21.15.103:0/1575327179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0d100a33b0 0x7f0d100a5870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c013990 tx=0x7f0d1c00d770 comp rx=0 tx=0).stop 2024-09-19T06:56:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.957+0000 7f0d2ad00640 1 -- 172.21.15.103:0/1575327179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d241036e0 msgr2=0x7f0d24198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.957+0000 7f0d2ad00640 1 --2- 172.21.15.103:0/1575327179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d241036e0 0x7f0d24198b10 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c005940 tx=0x7f0d0c0042f0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.957+0000 7f0d2ad00640 1 -- 172.21.15.103:0/1575327179 shutdown_connections 2024-09-19T06:56:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.957+0000 7f0d2ad00640 1 --2- 172.21.15.103:0/1575327179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0d100a33b0 0x7f0d100a5870 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.957+0000 7f0d2ad00640 1 --2- 172.21.15.103:0/1575327179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d241082a0 0x7f0d2419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.957+0000 7f0d2ad00640 1 --2- 172.21.15.103:0/1575327179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d241036e0 0x7f0d24198b10 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.957+0000 7f0d2ad00640 1 --2- 172.21.15.103:0/1575327179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d24102d30 0x7f0d241985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:17.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.957+0000 7f0d2ad00640 1 -- 172.21.15.103:0/1575327179 >> 172.21.15.103:0/1575327179 conn(0x7f0d240fc7d0 msgr2=0x7f0d24105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:17.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.957+0000 7f0d2ad00640 1 -- 172.21.15.103:0/1575327179 shutdown_connections 2024-09-19T06:56:17.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:17.957+0000 7f0d2ad00640 1 -- 172.21.15.103:0/1575327179 wait complete. 2024-09-19T06:56:18.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:18 smithi103 ceph-mon[31158]: pgmap v217: 97 pgs: 97 active+clean; 4.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T06:56:18.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:18 smithi134 ceph-mon[35356]: pgmap v217: 97 pgs: 97 active+clean; 4.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T06:56:18.484 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:18 smithi174 ceph-mon[35637]: pgmap v217: 97 pgs: 97 active+clean; 4.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T06:56:18.676 INFO:tasks.cephfs.filesystem:scrub status for tag:b6e66b50-5561-43bb-a4ba-5320fa6d12f4 - {'path': '/', 'tag': 'b6e66b50-5561-43bb-a4ba-5320fa6d12f4', 'options': 'recursive,force'} 2024-09-19T06:56:20.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:19 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T06:56:20.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:19 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T06:56:20.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:19 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T06:56:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:20 smithi103 ceph-mon[31158]: pgmap v218: 97 pgs: 97 active+clean; 4.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 KiB/s wr, 20 op/s 2024-09-19T06:56:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:20 smithi134 ceph-mon[35356]: pgmap v218: 97 pgs: 97 active+clean; 4.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 KiB/s wr, 20 op/s 2024-09-19T06:56:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:20 smithi174 ceph-mon[35637]: pgmap v218: 97 pgs: 97 active+clean; 4.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 52 KiB/s wr, 20 op/s 2024-09-19T06:56:22.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:22 smithi103 ceph-mon[31158]: pgmap v219: 97 pgs: 97 active+clean; 4.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-09-19T06:56:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:22 smithi134 ceph-mon[35356]: pgmap v219: 97 pgs: 97 active+clean; 4.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-09-19T06:56:22.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:22 smithi174 ceph-mon[35637]: pgmap v219: 97 pgs: 97 active+clean; 4.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-09-19T06:56:24.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:24 smithi103 ceph-mon[31158]: pgmap v220: 97 pgs: 97 active+clean; 4.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-09-19T06:56:24.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:24 smithi134 ceph-mon[35356]: pgmap v220: 97 pgs: 97 active+clean; 4.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-09-19T06:56:24.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:24 smithi174 ceph-mon[35637]: pgmap v220: 97 pgs: 97 active+clean; 4.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-09-19T06:56:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.733+0000 7f09a9b6d640 1 -- 172.21.15.103:0/4150631367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a41069e0 msgr2=0x7f09a4106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.733+0000 7f09a9b6d640 1 --2- 172.21.15.103:0/4150631367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a41069e0 0x7f09a4106dc0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f0994009930 tx=0x7f099402a420 comp rx=0 tx=0).stop 2024-09-19T06:56:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.733+0000 7f09a9b6d640 1 -- 172.21.15.103:0/4150631367 shutdown_connections 2024-09-19T06:56:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.733+0000 7f09a9b6d640 1 --2- 172.21.15.103:0/4150631367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a4101370 0x7f09a410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.733+0000 7f09a9b6d640 1 --2- 172.21.15.103:0/4150631367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a41009d0 0x7f09a4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.733+0000 7f09a9b6d640 1 --2- 172.21.15.103:0/4150631367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a41069e0 0x7f09a4106dc0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.733+0000 7f09a9b6d640 1 -- 172.21.15.103:0/4150631367 >> 172.21.15.103:0/4150631367 conn(0x7f09a40fc6f0 msgr2=0x7f09a40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.733+0000 7f09a9b6d640 1 -- 172.21.15.103:0/4150631367 shutdown_connections 2024-09-19T06:56:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.733+0000 7f09a9b6d640 1 -- 172.21.15.103:0/4150631367 wait complete. 2024-09-19T06:56:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.734+0000 7f09a9b6d640 1 Processor -- start 2024-09-19T06:56:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.734+0000 7f09a9b6d640 1 -- start start 2024-09-19T06:56:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.734+0000 7f09a9b6d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a41009d0 0x7f09a4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.734+0000 7f09a9b6d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a4101370 0x7f09a41988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.734+0000 7f09a9b6d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a41069e0 0x7f09a419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.734+0000 7f09a9b6d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4110de0 con 0x7f09a41009d0 2024-09-19T06:56:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.734+0000 7f09a9b6d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4110c60 con 0x7f09a41069e0 2024-09-19T06:56:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.734+0000 7f09a9b6d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4110f60 con 0x7f09a4101370 2024-09-19T06:56:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.734+0000 7f09a37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a41009d0 0x7f09a4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.735+0000 7f09a37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a41009d0 0x7f09a4198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35246/0 (socket says 172.21.15.103:35246) 2024-09-19T06:56:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.735+0000 7f09a37fe640 1 -- 172.21.15.103:0/135694477 learned_addr learned my addr 172.21.15.103:0/135694477 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.735+0000 7f09a3fff640 1 --2- 172.21.15.103:0/135694477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a41069e0 0x7f09a419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.735+0000 7f09a2ffd640 1 --2- 172.21.15.103:0/135694477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a4101370 0x7f09a41988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.735+0000 7f09a37fe640 1 -- 172.21.15.103:0/135694477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a4101370 msgr2=0x7f09a41988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.735+0000 7f09a37fe640 1 --2- 172.21.15.103:0/135694477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a4101370 0x7f09a41988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.735+0000 7f09a37fe640 1 -- 172.21.15.103:0/135694477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a41069e0 msgr2=0x7f09a419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.735+0000 7f09a37fe640 1 --2- 172.21.15.103:0/135694477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a41069e0 0x7f09a419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.735+0000 7f09a37fe640 1 -- 172.21.15.103:0/135694477 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a419d370 con 0x7f09a41009d0 2024-09-19T06:56:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.735+0000 7f09a37fe640 1 --2- 172.21.15.103:0/135694477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a41009d0 0x7f09a4198380 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f0994009230 tx=0x7f0994004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.735+0000 7f09a0ff9640 1 -- 172.21.15.103:0/135694477 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099403f030 con 0x7f09a41009d0 2024-09-19T06:56:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.735+0000 7f09a0ff9640 1 -- 172.21.15.103:0/135694477 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0994003ad0 con 0x7f09a41009d0 2024-09-19T06:56:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.735+0000 7f09a9b6d640 1 -- 172.21.15.103:0/135694477 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a419d600 con 0x7f09a41009d0 2024-09-19T06:56:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.735+0000 7f09a0ff9640 1 -- 172.21.15.103:0/135694477 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994035070 con 0x7f09a41009d0 2024-09-19T06:56:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.735+0000 7f09a9b6d640 1 -- 172.21.15.103:0/135694477 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a41a4ef0 con 0x7f09a41009d0 2024-09-19T06:56:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.738+0000 7f09a0ff9640 1 -- 172.21.15.103:0/135694477 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0994042030 con 0x7f09a41009d0 2024-09-19T06:56:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.738+0000 7f09ac9ea740 1 Processor -- start 2024-09-19T06:56:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.738+0000 7f09ac9ea740 1 -- start start 2024-09-19T06:56:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.738+0000 7f09ac9ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def888ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.738+0000 7f09ac9ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def893770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.738+0000 7f09ac9ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def8960a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.738+0000 7f09ac9ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556def7ed2f0 con 0x556def893cb0 2024-09-19T06:56:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.738+0000 7f09ac9ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556def795270 con 0x556def889410 2024-09-19T06:56:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.738+0000 7f09ac9ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556def7e92b0 con 0x556def888af0 2024-09-19T06:56:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.738+0000 7f09a0ff9640 1 --2- 172.21.15.103:0/135694477 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f098c0a3460 0x7f098c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.738+0000 7f09a2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def8960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.738+0000 7f09a2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def8960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35262/0 (socket says 172.21.15.103:35262) 2024-09-19T06:56:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.738+0000 7f09a2ffd640 1 -- 172.21.15.103:0/1203966457 learned_addr learned my addr 172.21.15.103:0/1203966457 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.738+0000 7f09a3fff640 1 --2- 172.21.15.103:0/135694477 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f098c0a3460 0x7f098c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.739+0000 7f09a0ff9640 1 -- 172.21.15.103:0/135694477 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f09940f00f0 con 0x7f09a41009d0 2024-09-19T06:56:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.739+0000 7f09a37fe640 1 --2- 172.21.15.103:0/1203966457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def893770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.739+0000 7f09a3fff640 1 --2- 172.21.15.103:0/1203966457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def888ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.739+0000 7f09a2ffd640 1 -- 172.21.15.103:0/1203966457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 msgr2=0x556def888ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.739+0000 7f09a2ffd640 1 --2- 172.21.15.103:0/1203966457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def888ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.739+0000 7f09a2ffd640 1 -- 172.21.15.103:0/1203966457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 msgr2=0x556def893770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.739+0000 7f09a2ffd640 1 --2- 172.21.15.103:0/1203966457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def893770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.739+0000 7f09a2ffd640 1 -- 172.21.15.103:0/1203966457 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556def794340 con 0x556def893cb0 2024-09-19T06:56:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.739+0000 7f09a37fe640 1 --2- 172.21.15.103:0/1203966457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def893770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.739+0000 7f09a3fff640 1 --2- 172.21.15.103:0/135694477 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f098c0a3460 0x7f098c0a5920 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f099c009fd0 tx=0x7f099c012a70 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.739+0000 7f09a2ffd640 1 --2- 172.21.15.103:0/1203966457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def8960a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f099800abf0 tx=0x7f099800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244c820d24aae74a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.739+0000 7f0989ffb640 1 -- 172.21.15.103:0/1203966457 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099801d070 con 0x556def893cb0 2024-09-19T06:56:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.739+0000 7f0989ffb640 1 -- 172.21.15.103:0/1203966457 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0998018b20 con 0x556def893cb0 2024-09-19T06:56:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.739+0000 7f0989ffb640 1 -- 172.21.15.103:0/1203966457 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998010ab0 con 0x556def893cb0 2024-09-19T06:56:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.740+0000 7f09ac9ea740 1 -- 172.21.15.103:0/1203966457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 msgr2=0x556def8960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.740+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/1203966457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def8960a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f099800abf0 tx=0x7f099800f790 comp rx=0 tx=0).stop 2024-09-19T06:56:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.740+0000 7f09ac9ea740 1 -- 172.21.15.103:0/1203966457 shutdown_connections 2024-09-19T06:56:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.740+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/1203966457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def8960a0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.740+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/1203966457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def893770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.740+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/1203966457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def888ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.740+0000 7f09ac9ea740 1 -- 172.21.15.103:0/1203966457 >> 172.21.15.103:0/1203966457 conn(0x556def885060 msgr2=0x556def885440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.740+0000 7f09ac9ea740 1 -- 172.21.15.103:0/1203966457 shutdown_connections 2024-09-19T06:56:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.740+0000 7f09ac9ea740 1 -- 172.21.15.103:0/1203966457 wait complete. 2024-09-19T06:56:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.740+0000 7f09ac9ea740 1 -- start start 2024-09-19T06:56:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09ac9ea740 1 Processor -- start 2024-09-19T06:56:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09ac9ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def890b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09ac9ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def8910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09ac9ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def891600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09ac9ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556def896880 con 0x556def893cb0 2024-09-19T06:56:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09ac9ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556def776290 con 0x556def889410 2024-09-19T06:56:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09ac9ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556def896a00 con 0x556def888af0 2024-09-19T06:56:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09ac9ea740 10 client.0 _ll_register_callbacks cb 0x556def7a7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09a3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def891600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09a3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def891600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35274/0 (socket says 172.21.15.103:35274) 2024-09-19T06:56:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09a3fff640 1 -- 172.21.15.103:0/734093834 learned_addr learned my addr 172.21.15.103:0/734093834 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09a37fe640 1 --2- 172.21.15.103:0/734093834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def890b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09a2ffd640 1 --2- 172.21.15.103:0/734093834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def8910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09a3fff640 1 -- 172.21.15.103:0/734093834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 msgr2=0x556def890b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09a3fff640 1 --2- 172.21.15.103:0/734093834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def890b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09a3fff640 1 -- 172.21.15.103:0/734093834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 msgr2=0x556def8910c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09a3fff640 1 --2- 172.21.15.103:0/734093834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def8910c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09a3fff640 1 -- 172.21.15.103:0/734093834 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556def500210 con 0x556def893cb0 2024-09-19T06:56:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09a37fe640 1 --2- 172.21.15.103:0/734093834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def890b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f09a3fff640 1 --2- 172.21.15.103:0/734093834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def891600 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f099c010b60 tx=0x7f099c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce300ba1afda652b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.741+0000 7f0982ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.742+0000 7f0982ffd640 1 -- 172.21.15.103:0/734093834 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c023030 con 0x556def893cb0 2024-09-19T06:56:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.742+0000 7f0982ffd640 1 -- 172.21.15.103:0/734093834 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f099c01ccf0 con 0x556def893cb0 2024-09-19T06:56:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.742+0000 7f09ac9ea740 1 -- 172.21.15.103:0/734093834 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0998018b20 con 0x556def893cb0 2024-09-19T06:56:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.742+0000 7f0982ffd640 1 -- 172.21.15.103:0/734093834 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f099c01f420 con 0x556def893cb0 2024-09-19T06:56:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.742+0000 7f09ac9ea740 10 client.17982 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.742+0000 7f09ac9ea740 1 -- 172.21.15.103:0/734093834 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556def5cc620 con 0x556def893cb0 2024-09-19T06:56:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.742+0000 7f0982ffd640 1 -- 172.21.15.103:0/734093834 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0998018b20 con 0x556def893cb0 2024-09-19T06:56:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.742+0000 7f0982ffd640 1 -- 172.21.15.103:0/734093834 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f099c027620 con 0x556def893cb0 2024-09-19T06:56:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.742+0000 7f09ac9ea740 10 client.17982 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.742+0000 7f09ac9ea740 10 client.17982 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.742+0000 7f09ac9ea740 20 client.17982 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.742+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/734093834 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556def8a3bf0 0x556def8a6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.742+0000 7f09ac9ea740 4 client.17982 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.742+0000 7f09ac9ea740 1 -- 172.21.15.103:0/734093834 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556def7faa20 con 0x556def8a3bf0 2024-09-19T06:56:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.743+0000 7f09a37fe640 1 --2- 172.21.15.103:0/734093834 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556def8a3bf0 0x556def8a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.743+0000 7f0982ffd640 10 client.17982 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.743+0000 7f09a37fe640 1 --2- 172.21.15.103:0/734093834 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556def8a3bf0 0x556def8a6080 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77a18973be09da7d server_cookie=5659253893d4dd3 in_seq=0 out_seq=0 2024-09-19T06:56:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.744+0000 7f0982ffd640 1 -- 172.21.15.103:0/734093834 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556def7faa20 con 0x556def8a3bf0 2024-09-19T06:56:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.744+0000 7f0982ffd640 10 client.17982 handle_command_reply: tid=0 2024-09-19T06:56:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.744+0000 7f09ac9ea740 1 client.17982 shutdown 2024-09-19T06:56:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.744+0000 7f09ac9ea740 1 -- 172.21.15.103:0/734093834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 msgr2=0x556def891600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.744+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/734093834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def891600 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f099c010b60 tx=0x7f099c0096a0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.744+0000 7f09ac9ea740 1 -- 172.21.15.103:0/734093834 shutdown_connections 2024-09-19T06:56:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.744+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/734093834 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556def8a3bf0 0x556def8a6080 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.744+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/734093834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def891600 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.744+0000 7f0982ffd640 0 client.17982 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.744+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/734093834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def8910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.744+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/734093834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def890b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.744+0000 7f09ac9ea740 1 -- 172.21.15.103:0/734093834 >> 172.21.15.103:0/734093834 conn(0x556def885060 msgr2=0x556def88ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.744+0000 7f09ac9ea740 1 -- 172.21.15.103:0/734093834 shutdown_connections 2024-09-19T06:56:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.744+0000 7f09ac9ea740 1 -- 172.21.15.103:0/734093834 wait complete. 2024-09-19T06:56:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.744+0000 7f09ac9ea740 20 client.17982 trim_cache size 0 max 16384 2024-09-19T06:56:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09ac9ea740 1 Processor -- start 2024-09-19T06:56:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09ac9ea740 1 -- start start 2024-09-19T06:56:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09ac9ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def891430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09ac9ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def8ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09ac9ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def8abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09ac9ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41ad810 con 0x556def893cb0 2024-09-19T06:56:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09ac9ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41af350 con 0x556def889410 2024-09-19T06:56:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09ac9ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41adee0 con 0x556def888af0 2024-09-19T06:56:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09a3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def8abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09a3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def8abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35288/0 (socket says 172.21.15.103:35288) 2024-09-19T06:56:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09a3fff640 1 -- 172.21.15.103:0/2968859051 learned_addr learned my addr 172.21.15.103:0/2968859051 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09a37fe640 1 --2- 172.21.15.103:0/2968859051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def891430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09a2ffd640 1 --2- 172.21.15.103:0/2968859051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def8ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09a3fff640 1 -- 172.21.15.103:0/2968859051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 msgr2=0x556def891430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09a3fff640 1 --2- 172.21.15.103:0/2968859051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def891430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09a3fff640 1 -- 172.21.15.103:0/2968859051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 msgr2=0x556def8ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09a3fff640 1 --2- 172.21.15.103:0/2968859051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def8ab9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09a3fff640 1 -- 172.21.15.103:0/2968859051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556def884e90 con 0x556def893cb0 2024-09-19T06:56:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.759+0000 7f09a3fff640 1 --2- 172.21.15.103:0/2968859051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def8abf20 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f099c0097c0 tx=0x7f099c009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afe2393f9abb42e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.760+0000 7f09897fa640 1 -- 172.21.15.103:0/2968859051 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c034020 con 0x556def893cb0 2024-09-19T06:56:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.760+0000 7f09897fa640 1 -- 172.21.15.103:0/2968859051 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099c027cd0 con 0x556def893cb0 2024-09-19T06:56:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.760+0000 7f09897fa640 1 -- 172.21.15.103:0/2968859051 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c023070 con 0x556def893cb0 2024-09-19T06:56:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.760+0000 7f09ac9ea740 1 -- 172.21.15.103:0/2968859051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 msgr2=0x556def8abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.760+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/2968859051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def8abf20 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f099c0097c0 tx=0x7f099c009c60 comp rx=0 tx=0).stop 2024-09-19T06:56:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.760+0000 7f09ac9ea740 1 -- 172.21.15.103:0/2968859051 shutdown_connections 2024-09-19T06:56:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.760+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/2968859051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def8abf20 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.760+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/2968859051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def8ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.760+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/2968859051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def891430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.760+0000 7f09ac9ea740 1 -- 172.21.15.103:0/2968859051 >> 172.21.15.103:0/2968859051 conn(0x556def885060 msgr2=0x556def892c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.760+0000 7f09ac9ea740 1 -- 172.21.15.103:0/2968859051 shutdown_connections 2024-09-19T06:56:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.760+0000 7f09ac9ea740 1 -- 172.21.15.103:0/2968859051 wait complete. 2024-09-19T06:56:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.760+0000 7f09ac9ea740 1 -- start start 2024-09-19T06:56:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09ac9ea740 1 Processor -- start 2024-09-19T06:56:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09ac9ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def89c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09ac9ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def8978e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09ac9ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def898070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09ac9ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41aec80 con 0x556def893cb0 2024-09-19T06:56:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09ac9ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41ae5b0 con 0x556def889410 2024-09-19T06:56:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09ac9ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41ab810 con 0x556def888af0 2024-09-19T06:56:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09ac9ea740 10 client.0 _ll_register_callbacks cb 0x7f099c009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09a3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def898070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09a3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def898070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35294/0 (socket says 172.21.15.103:35294) 2024-09-19T06:56:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09a3fff640 1 -- 172.21.15.103:0/1107413547 learned_addr learned my addr 172.21.15.103:0/1107413547 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09a37fe640 1 --2- 172.21.15.103:0/1107413547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def89c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09a2ffd640 1 --2- 172.21.15.103:0/1107413547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def8978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09a3fff640 1 -- 172.21.15.103:0/1107413547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 msgr2=0x556def89c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09a3fff640 1 --2- 172.21.15.103:0/1107413547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def89c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09a3fff640 1 -- 172.21.15.103:0/1107413547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 msgr2=0x556def8978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09a3fff640 1 --2- 172.21.15.103:0/1107413547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def8978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09a3fff640 1 -- 172.21.15.103:0/1107413547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556def88b940 con 0x556def893cb0 2024-09-19T06:56:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.761+0000 7f09a2ffd640 1 --2- 172.21.15.103:0/1107413547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def8978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.762+0000 7f09a3fff640 1 --2- 172.21.15.103:0/1107413547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def898070 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f099c011080 tx=0x7f099c004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e038be59dd4f77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.762+0000 7f0982ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.762+0000 7f0982ffd640 1 -- 172.21.15.103:0/1107413547 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c023070 con 0x556def893cb0 2024-09-19T06:56:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.762+0000 7f0982ffd640 1 -- 172.21.15.103:0/1107413547 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f099c0275c0 con 0x556def893cb0 2024-09-19T06:56:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.762+0000 7f09ac9ea740 1 -- 172.21.15.103:0/1107413547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f099c027cd0 con 0x556def893cb0 2024-09-19T06:56:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.762+0000 7f0982ffd640 1 -- 172.21.15.103:0/1107413547 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f099c034040 con 0x556def893cb0 2024-09-19T06:56:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.762+0000 7f09ac9ea740 10 client.17988 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.762+0000 7f09ac9ea740 1 -- 172.21.15.103:0/1107413547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556def912de0 con 0x556def893cb0 2024-09-19T06:56:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.762+0000 7f0982ffd640 1 -- 172.21.15.103:0/1107413547 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099c027cd0 con 0x556def893cb0 2024-09-19T06:56:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.762+0000 7f0982ffd640 1 -- 172.21.15.103:0/1107413547 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f099c01fbc0 con 0x556def893cb0 2024-09-19T06:56:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.762+0000 7f09ac9ea740 10 client.17988 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.762+0000 7f09ac9ea740 10 client.17988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.762+0000 7f09ac9ea740 20 client.17988 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.762+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/1107413547 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556def9133e0 0x556def9157d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.762+0000 7f09ac9ea740 4 client.17988 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:56:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.762+0000 7f09ac9ea740 1 -- 172.21.15.103:0/1107413547 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556def7f4d70 con 0x556def9133e0 2024-09-19T06:56:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.763+0000 7f09a37fe640 1 --2- 172.21.15.103:0/1107413547 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556def9133e0 0x556def9157d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.763+0000 7f0982ffd640 10 client.17988 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.763+0000 7f09a37fe640 1 --2- 172.21.15.103:0/1107413547 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556def9133e0 0x556def9157d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b0bb1d40dc7b4 server_cookie=ef89656c8ca19eb4 in_seq=0 out_seq=0 2024-09-19T06:56:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.777+0000 7f0982ffd640 1 -- 172.21.15.103:0/1107413547 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556def7f4d70 con 0x556def9133e0 2024-09-19T06:56:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.777+0000 7f0982ffd640 10 client.17988 handle_command_reply: tid=0 2024-09-19T06:56:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.777+0000 7f09ac9ea740 1 client.17988 shutdown 2024-09-19T06:56:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.777+0000 7f09ac9ea740 1 -- 172.21.15.103:0/1107413547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 msgr2=0x556def898070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.777+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/1107413547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def898070 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f099c011080 tx=0x7f099c004de0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.777+0000 7f09ac9ea740 1 -- 172.21.15.103:0/1107413547 shutdown_connections 2024-09-19T06:56:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.777+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/1107413547 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556def9133e0 0x556def9157d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.777+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/1107413547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556def893cb0 0x556def898070 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.777+0000 7f0982ffd640 0 client.17988 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.777+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/1107413547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556def889410 0x556def8978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.777+0000 7f09ac9ea740 1 --2- 172.21.15.103:0/1107413547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556def888af0 0x556def89c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.777+0000 7f09ac9ea740 1 -- 172.21.15.103:0/1107413547 >> 172.21.15.103:0/1107413547 conn(0x556def885060 msgr2=0x556def88a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.777+0000 7f09ac9ea740 1 -- 172.21.15.103:0/1107413547 shutdown_connections 2024-09-19T06:56:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.777+0000 7f09ac9ea740 1 -- 172.21.15.103:0/1107413547 wait complete. 2024-09-19T06:56:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.777+0000 7f09ac9ea740 20 client.17988 trim_cache size 0 max 16384 2024-09-19T06:56:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:56:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:56:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:56:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:56:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.778+0000 7f09a9b6d640 1 -- 172.21.15.103:0/135694477 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f098c0a3460 msgr2=0x7f098c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.778+0000 7f09a9b6d640 1 --2- 172.21.15.103:0/135694477 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f098c0a3460 0x7f098c0a5920 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f099c009fd0 tx=0x7f099c012a70 comp rx=0 tx=0).stop 2024-09-19T06:56:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.778+0000 7f09a9b6d640 1 -- 172.21.15.103:0/135694477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a41009d0 msgr2=0x7f09a4198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.778+0000 7f09a9b6d640 1 --2- 172.21.15.103:0/135694477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a41009d0 0x7f09a4198380 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f0994009230 tx=0x7f0994004270 comp rx=0 tx=0).stop 2024-09-19T06:56:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.778+0000 7f09a9b6d640 1 -- 172.21.15.103:0/135694477 shutdown_connections 2024-09-19T06:56:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.778+0000 7f09a9b6d640 1 --2- 172.21.15.103:0/135694477 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f098c0a3460 0x7f098c0a5920 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.778+0000 7f09a9b6d640 1 --2- 172.21.15.103:0/135694477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a41069e0 0x7f09a419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.778+0000 7f09a9b6d640 1 --2- 172.21.15.103:0/135694477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a4101370 0x7f09a41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.778+0000 7f09a9b6d640 1 --2- 172.21.15.103:0/135694477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a41009d0 0x7f09a4198380 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.778+0000 7f09a9b6d640 1 -- 172.21.15.103:0/135694477 >> 172.21.15.103:0/135694477 conn(0x7f09a40fc6f0 msgr2=0x7f09a41047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.778+0000 7f09a9b6d640 1 -- 172.21.15.103:0/135694477 shutdown_connections 2024-09-19T06:56:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:25.778+0000 7f09a9b6d640 1 -- 172.21.15.103:0/135694477 wait complete. 2024-09-19T06:56:26.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:26 smithi103 ceph-mon[31158]: pgmap v221: 97 pgs: 97 active+clean; 4.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T06:56:26.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:26 smithi134 ceph-mon[35356]: pgmap v221: 97 pgs: 97 active+clean; 4.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T06:56:26.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:26 smithi174 ceph-mon[35637]: pgmap v221: 97 pgs: 97 active+clean; 4.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T06:56:28.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:27 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:56:28.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:27 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:56:28.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:27 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:56:29.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:28 smithi174 ceph-mon[35637]: pgmap v222: 97 pgs: 97 active+clean; 4.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-09-19T06:56:29.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:28 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:56:29.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:28 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:56:29.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:28 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:56:29.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:28 smithi103 ceph-mon[31158]: pgmap v222: 97 pgs: 97 active+clean; 4.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-09-19T06:56:29.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:28 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:56:29.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:28 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:56:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:28 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:56:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:28 smithi134 ceph-mon[35356]: pgmap v222: 97 pgs: 97 active+clean; 4.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-09-19T06:56:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:28 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:56:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:28 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:56:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:28 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:56:30.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:30 smithi103 ceph-mon[31158]: pgmap v223: 97 pgs: 97 active+clean; 5.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-19T06:56:30.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:30 smithi134 ceph-mon[35356]: pgmap v223: 97 pgs: 97 active+clean; 5.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-19T06:56:30.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:30 smithi174 ceph-mon[35637]: pgmap v223: 97 pgs: 97 active+clean; 5.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-19T06:56:30.841 INFO:tasks.quiescer.fs.[cephfs]:Total duration (51.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T06:56:30.841 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.75 2024-09-19T06:56:30.841 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:56:31.103 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:31.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.583+0000 7f929a288640 1 -- 172.21.15.103:0/2686785460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9294102570 msgr2=0x7f92941029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:31.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.583+0000 7f929a288640 1 --2- 172.21.15.103:0/2686785460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9294102570 0x7f92941029b0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f927c009900 tx=0x7f927c02a480 comp rx=0 tx=0).stop 2024-09-19T06:56:31.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.583+0000 7f929a288640 1 -- 172.21.15.103:0/2686785460 shutdown_connections 2024-09-19T06:56:31.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.583+0000 7f929a288640 1 --2- 172.21.15.103:0/2686785460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9294102ef0 0x7f929410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.583+0000 7f929a288640 1 --2- 172.21.15.103:0/2686785460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9294102570 0x7f92941029b0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.583+0000 7f929a288640 1 --2- 172.21.15.103:0/2686785460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9294100e20 0x7f9294101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.583+0000 7f929a288640 1 -- 172.21.15.103:0/2686785460 >> 172.21.15.103:0/2686785460 conn(0x7f92940fc7d0 msgr2=0x7f92940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:31.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.583+0000 7f929a288640 1 -- 172.21.15.103:0/2686785460 shutdown_connections 2024-09-19T06:56:31.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.583+0000 7f929a288640 1 -- 172.21.15.103:0/2686785460 wait complete. 2024-09-19T06:56:31.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f929a288640 1 Processor -- start 2024-09-19T06:56:31.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f929a288640 1 -- start start 2024-09-19T06:56:31.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f929a288640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9294100e20 0x7f92941973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f929a288640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9294102570 0x7f9294197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f929a288640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9294102ef0 0x7f929419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f929a288640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9294110f80 con 0x7f9294100e20 2024-09-19T06:56:31.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f929a288640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9294110e00 con 0x7f9294102ef0 2024-09-19T06:56:31.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f929a288640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9294111100 con 0x7f9294102570 2024-09-19T06:56:31.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f9293fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9294100e20 0x7f92941973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f9293fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9294100e20 0x7f92941973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45320/0 (socket says 172.21.15.103:45320) 2024-09-19T06:56:31.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f9293fff640 1 -- 172.21.15.103:0/641820132 learned_addr learned my addr 172.21.15.103:0/641820132 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:31.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f92937fe640 1 --2- 172.21.15.103:0/641820132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9294102570 0x7f9294197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f9298825640 1 --2- 172.21.15.103:0/641820132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9294102ef0 0x7f929419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f9293fff640 1 -- 172.21.15.103:0/641820132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9294102570 msgr2=0x7f9294197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:31.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f9293fff640 1 --2- 172.21.15.103:0/641820132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9294102570 0x7f9294197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f9293fff640 1 -- 172.21.15.103:0/641820132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9294102ef0 msgr2=0x7f929419bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:31.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f9293fff640 1 --2- 172.21.15.103:0/641820132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9294102ef0 0x7f929419bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f9293fff640 1 -- 172.21.15.103:0/641820132 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f929419c370 con 0x7f9294100e20 2024-09-19T06:56:31.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.584+0000 7f9298825640 1 --2- 172.21.15.103:0/641820132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9294102ef0 0x7f929419bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:31.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.585+0000 7f9293fff640 1 --2- 172.21.15.103:0/641820132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9294100e20 0x7f92941973c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f928400d950 tx=0x7f928400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:31.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.585+0000 7f92917fa640 1 -- 172.21.15.103:0/641820132 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9284017030 con 0x7f9294100e20 2024-09-19T06:56:31.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.585+0000 7f92917fa640 1 -- 172.21.15.103:0/641820132 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9284004020 con 0x7f9294100e20 2024-09-19T06:56:31.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.585+0000 7f929a288640 1 -- 172.21.15.103:0/641820132 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f929419c660 con 0x7f9294100e20 2024-09-19T06:56:31.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.585+0000 7f92917fa640 1 -- 172.21.15.103:0/641820132 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92840047a0 con 0x7f9294100e20 2024-09-19T06:56:31.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.585+0000 7f929a288640 1 -- 172.21.15.103:0/641820132 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92941a3f40 con 0x7f9294100e20 2024-09-19T06:56:31.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.586+0000 7f929a288640 1 -- 172.21.15.103:0/641820132 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9294101200 con 0x7f9294100e20 2024-09-19T06:56:31.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.587+0000 7f92917fa640 1 -- 172.21.15.103:0/641820132 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f92840041d0 con 0x7f9294100e20 2024-09-19T06:56:31.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.588+0000 7f92917fa640 1 --2- 172.21.15.103:0/641820132 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f92800a3430 0x7f92800a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.588+0000 7f92937fe640 1 --2- 172.21.15.103:0/641820132 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f92800a3430 0x7f92800a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.588+0000 7f92917fa640 1 -- 172.21.15.103:0/641820132 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f92840cfdf0 con 0x7f9294100e20 2024-09-19T06:56:31.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.589+0000 7f92937fe640 1 --2- 172.21.15.103:0/641820132 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f92800a3430 0x7f92800a58f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f927c009230 tx=0x7f927c0058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:31.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.592+0000 7f92917fa640 1 -- 172.21.15.103:0/641820132 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f928409c7d0 con 0x7f9294100e20 2024-09-19T06:56:31.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.792+0000 7f929a288640 1 -- 172.21.15.103:0/641820132 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9294198720 con 0x7f9294100e20 2024-09-19T06:56:31.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.793+0000 7f92917fa640 1 -- 172.21.15.103:0/641820132 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f9294198720 con 0x7f9294100e20 2024-09-19T06:56:31.794 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:56:31.795 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:56:31.795 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:56:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.795+0000 7f929a288640 1 -- 172.21.15.103:0/641820132 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f92800a3430 msgr2=0x7f92800a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.795+0000 7f929a288640 1 --2- 172.21.15.103:0/641820132 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f92800a3430 0x7f92800a58f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f927c009230 tx=0x7f927c0058d0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.795+0000 7f929a288640 1 -- 172.21.15.103:0/641820132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9294100e20 msgr2=0x7f92941973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.795+0000 7f929a288640 1 --2- 172.21.15.103:0/641820132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9294100e20 0x7f92941973c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f928400d950 tx=0x7f928400de10 comp rx=0 tx=0).stop 2024-09-19T06:56:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.796+0000 7f929a288640 1 -- 172.21.15.103:0/641820132 shutdown_connections 2024-09-19T06:56:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.796+0000 7f929a288640 1 --2- 172.21.15.103:0/641820132 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f92800a3430 0x7f92800a58f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.796+0000 7f929a288640 1 --2- 172.21.15.103:0/641820132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9294102ef0 0x7f929419bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.796+0000 7f929a288640 1 --2- 172.21.15.103:0/641820132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9294102570 0x7f9294197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.796+0000 7f929a288640 1 --2- 172.21.15.103:0/641820132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9294100e20 0x7f92941973c0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.796+0000 7f929a288640 1 -- 172.21.15.103:0/641820132 >> 172.21.15.103:0/641820132 conn(0x7f92940fc7d0 msgr2=0x7f92940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.796+0000 7f929a288640 1 -- 172.21.15.103:0/641820132 shutdown_connections 2024-09-19T06:56:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:31.796+0000 7f929a288640 1 -- 172.21.15.103:0/641820132 wait complete. 2024-09-19T06:56:32.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:31 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/641820132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:31 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/641820132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:32.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:31 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/641820132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:32.574 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0' 2024-09-19T06:56:32.574 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0 2024-09-19T06:56:32.824 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:33.102 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:32 smithi103 ceph-mon[31158]: pgmap v224: 97 pgs: 97 active+clean; 5.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:56:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:32 smithi134 ceph-mon[35356]: pgmap v224: 97 pgs: 97 active+clean; 5.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:56:33.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.303+0000 7fa074640640 1 -- 172.21.15.103:0/1949019296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa06c102570 msgr2=0x7fa06c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.303+0000 7fa074640640 1 --2- 172.21.15.103:0/1949019296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa06c102570 0x7fa06c1029b0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009900 tx=0x7fa05c02a480 comp rx=0 tx=0).stop 2024-09-19T06:56:33.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.303+0000 7fa074640640 1 -- 172.21.15.103:0/1949019296 shutdown_connections 2024-09-19T06:56:33.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.303+0000 7fa074640640 1 --2- 172.21.15.103:0/1949019296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa06c102ef0 0x7fa06c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.303+0000 7fa074640640 1 --2- 172.21.15.103:0/1949019296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa06c102570 0x7fa06c1029b0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.303+0000 7fa074640640 1 --2- 172.21.15.103:0/1949019296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa06c100e20 0x7fa06c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.303+0000 7fa074640640 1 -- 172.21.15.103:0/1949019296 >> 172.21.15.103:0/1949019296 conn(0x7fa06c0fc7d0 msgr2=0x7fa06c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:33.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.303+0000 7fa074640640 1 -- 172.21.15.103:0/1949019296 shutdown_connections 2024-09-19T06:56:33.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.303+0000 7fa074640640 1 -- 172.21.15.103:0/1949019296 wait complete. 2024-09-19T06:56:33.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.303+0000 7fa074640640 1 Processor -- start 2024-09-19T06:56:33.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.303+0000 7fa074640640 1 -- start start 2024-09-19T06:56:33.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa074640640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa06c100e20 0x7fa06c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa074640640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa06c102570 0x7fa06c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa074640640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa06c102ef0 0x7fa06c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa074640640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c110f80 con 0x7fa06c102ef0 2024-09-19T06:56:33.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa074640640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c110e00 con 0x7fa06c102570 2024-09-19T06:56:33.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa074640640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c111100 con 0x7fa06c100e20 2024-09-19T06:56:33.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa072bdd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa06c102ef0 0x7fa06c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa072bdd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa06c102ef0 0x7fa06c0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45342/0 (socket says 172.21.15.103:45342) 2024-09-19T06:56:33.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa072bdd640 1 -- 172.21.15.103:0/1684350888 learned_addr learned my addr 172.21.15.103:0/1684350888 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:33.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa071bdb640 1 --2- 172.21.15.103:0/1684350888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa06c102570 0x7fa06c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa0723dc640 1 --2- 172.21.15.103:0/1684350888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa06c100e20 0x7fa06c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa072bdd640 1 -- 172.21.15.103:0/1684350888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa06c100e20 msgr2=0x7fa06c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa072bdd640 1 --2- 172.21.15.103:0/1684350888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa06c100e20 0x7fa06c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa072bdd640 1 -- 172.21.15.103:0/1684350888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa06c102570 msgr2=0x7fa06c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa072bdd640 1 --2- 172.21.15.103:0/1684350888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa06c102570 0x7fa06c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa072bdd640 1 -- 172.21.15.103:0/1684350888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06c1a6a30 con 0x7fa06c102ef0 2024-09-19T06:56:33.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa072bdd640 1 --2- 172.21.15.103:0/1684350888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa06c102ef0 0x7fa06c0fd700 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fa06400cad0 tx=0x7fa06400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:33.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa0637fe640 1 -- 172.21.15.103:0/1684350888 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06400ae20 con 0x7fa06c102ef0 2024-09-19T06:56:33.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa0637fe640 1 -- 172.21.15.103:0/1684350888 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa064005020 con 0x7fa06c102ef0 2024-09-19T06:56:33.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa074640640 1 -- 172.21.15.103:0/1684350888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06c1a6d20 con 0x7fa06c102ef0 2024-09-19T06:56:33.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.304+0000 7fa0637fe640 1 -- 172.21.15.103:0/1684350888 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06400f6f0 con 0x7fa06c102ef0 2024-09-19T06:56:33.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.305+0000 7fa074640640 1 -- 172.21.15.103:0/1684350888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06c1a71e0 con 0x7fa06c102ef0 2024-09-19T06:56:33.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.307+0000 7fa0637fe640 1 -- 172.21.15.103:0/1684350888 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa0640051d0 con 0x7fa06c102ef0 2024-09-19T06:56:33.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.307+0000 7fa076a19740 1 Processor -- start 2024-09-19T06:56:33.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.307+0000 7fa076a19740 1 -- start start 2024-09-19T06:56:33.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.307+0000 7fa076a19740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304750f850 0x56304750fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.307+0000 7fa076a19740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563047510170 0x56304751a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa076a19740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56304751aa30 0x56304751ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa076a19740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563047474160 con 0x56304751aa30 2024-09-19T06:56:33.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa076a19740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56304741b720 con 0x56304750f850 2024-09-19T06:56:33.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa076a19740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563047470120 con 0x563047510170 2024-09-19T06:56:33.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa0637fe640 1 --2- 172.21.15.103:0/1684350888 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa0580a33b0 0x7fa0580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa072bdd640 1 --2- 172.21.15.103:0/1684350888 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa0580a33b0 0x7fa0580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa071bdb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56304751aa30 0x56304751ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa071bdb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56304751aa30 0x56304751ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45354/0 (socket says 172.21.15.103:45354) 2024-09-19T06:56:33.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa071bdb640 1 -- 172.21.15.103:0/480180149 learned_addr learned my addr 172.21.15.103:0/480180149 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:33.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa0723dc640 1 --2- 172.21.15.103:0/480180149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563047510170 0x56304751a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa071bdb640 1 --2- 172.21.15.103:0/480180149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304750f850 0x56304750fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa0637fe640 1 -- 172.21.15.103:0/1684350888 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa0640cfd00 con 0x7fa06c102ef0 2024-09-19T06:56:33.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa071bdb640 1 -- 172.21.15.103:0/480180149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563047510170 msgr2=0x56304751a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:33.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa071bdb640 1 --2- 172.21.15.103:0/480180149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563047510170 0x56304751a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa071bdb640 1 -- 172.21.15.103:0/480180149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304750f850 msgr2=0x56304750fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:33.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa071bdb640 1 --2- 172.21.15.103:0/480180149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304750f850 0x56304750fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa071bdb640 1 -- 172.21.15.103:0/480180149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630471e5ba0 con 0x56304751aa30 2024-09-19T06:56:33.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa072bdd640 1 --2- 172.21.15.103:0/1684350888 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa0580a33b0 0x7fa0580a5870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa0640cd4e0 tx=0x7fa06406acb0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:33.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.308+0000 7fa071bdb640 1 --2- 172.21.15.103:0/480180149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56304751aa30 0x56304751ce20 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0314e0 tx=0x7fa05c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e759c1f9cfe5c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:33.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.309+0000 7fa060ff9640 1 -- 172.21.15.103:0/480180149 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c035040 con 0x56304751aa30 2024-09-19T06:56:33.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.309+0000 7fa060ff9640 1 -- 172.21.15.103:0/480180149 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa05c003b80 con 0x56304751aa30 2024-09-19T06:56:33.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.309+0000 7fa060ff9640 1 -- 172.21.15.103:0/480180149 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c031b10 con 0x56304751aa30 2024-09-19T06:56:33.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.309+0000 7fa076a19740 1 -- 172.21.15.103:0/480180149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56304751aa30 msgr2=0x56304751ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.309+0000 7fa076a19740 1 --2- 172.21.15.103:0/480180149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56304751aa30 0x56304751ce20 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0314e0 tx=0x7fa05c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.309+0000 7fa076a19740 1 -- 172.21.15.103:0/480180149 shutdown_connections 2024-09-19T06:56:33.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.309+0000 7fa076a19740 1 --2- 172.21.15.103:0/480180149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56304751aa30 0x56304751ce20 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.309+0000 7fa076a19740 1 --2- 172.21.15.103:0/480180149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563047510170 0x56304751a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.309+0000 7fa076a19740 1 --2- 172.21.15.103:0/480180149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304750f850 0x56304750fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.309+0000 7fa076a19740 1 -- 172.21.15.103:0/480180149 >> 172.21.15.103:0/480180149 conn(0x56304750bde0 msgr2=0x56304750c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:33.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.309+0000 7fa076a19740 1 -- 172.21.15.103:0/480180149 shutdown_connections 2024-09-19T06:56:33.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.309+0000 7fa076a19740 1 -- 172.21.15.103:0/480180149 wait complete. 2024-09-19T06:56:33.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa076a19740 1 -- start start 2024-09-19T06:56:33.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa076a19740 1 Processor -- start 2024-09-19T06:56:33.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa076a19740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304750f850 0x563047517960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa076a19740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563047510170 0x563047517ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa076a19740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304751aa30 0x5630475183e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa076a19740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56304751d600 con 0x563047510170 2024-09-19T06:56:33.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa076a19740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5630473fc900 con 0x56304750f850 2024-09-19T06:56:33.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa076a19740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56304751d780 con 0x56304751aa30 2024-09-19T06:56:33.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa076a19740 10 client.0 _ll_register_callbacks cb 0x5630473fa780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:33.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa0723dc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563047510170 0x563047517ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa0723dc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563047510170 0x563047517ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45356/0 (socket says 172.21.15.103:45356) 2024-09-19T06:56:33.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa0723dc640 1 -- 172.21.15.103:0/803019037 learned_addr learned my addr 172.21.15.103:0/803019037 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:33.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa071bdb640 1 --2- 172.21.15.103:0/803019037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304750f850 0x563047517960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa071bdb640 1 --2- 172.21.15.103:0/803019037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304751aa30 0x5630475183e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa0723dc640 1 -- 172.21.15.103:0/803019037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304751aa30 msgr2=0x5630475183e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:33.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa0723dc640 1 --2- 172.21.15.103:0/803019037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304751aa30 0x5630475183e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa0723dc640 1 -- 172.21.15.103:0/803019037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304750f850 msgr2=0x563047517960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:33.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa0723dc640 1 --2- 172.21.15.103:0/803019037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304750f850 0x563047517960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.310+0000 7fa0723dc640 1 -- 172.21.15.103:0/803019037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56304743cc50 con 0x563047510170 2024-09-19T06:56:33.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa071bdb640 1 --2- 172.21.15.103:0/803019037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304750f850 0x563047517960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:33.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa0723dc640 1 --2- 172.21.15.103:0/803019037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563047510170 0x563047517ea0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa06800bf10 tx=0x7fa068012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88f44ec29b1bc5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:33.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa055ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:33.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa055ffb640 1 -- 172.21.15.103:0/803019037 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068004230 con 0x563047510170 2024-09-19T06:56:33.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa055ffb640 1 -- 172.21.15.103:0/803019037 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa0680043d0 con 0x563047510170 2024-09-19T06:56:33.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa076a19740 1 -- 172.21.15.103:0/803019037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa05c003b80 con 0x563047510170 2024-09-19T06:56:33.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa055ffb640 1 -- 172.21.15.103:0/803019037 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa0680106a0 con 0x563047510170 2024-09-19T06:56:33.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa076a19740 10 client.18006 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:33.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa076a19740 1 -- 172.21.15.103:0/803019037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563047529390 con 0x563047510170 2024-09-19T06:56:33.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa055ffb640 1 -- 172.21.15.103:0/803019037 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa05c003b80 con 0x563047510170 2024-09-19T06:56:33.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa055ffb640 1 -- 172.21.15.103:0/803019037 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa068003be0 con 0x563047510170 2024-09-19T06:56:33.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa076a19740 10 client.18006 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:33.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa076a19740 10 client.18006 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:56:33.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa076a19740 20 client.18006 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:33.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa076a19740 1 --2- 172.21.15.103:0/803019037 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56304752a850 0x56304752cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa076a19740 4 client.18006 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:33.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.311+0000 7fa076a19740 1 -- 172.21.15.103:0/803019037 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563047481890 con 0x56304752a850 2024-09-19T06:56:33.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.312+0000 7fa071bdb640 1 --2- 172.21.15.103:0/803019037 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56304752a850 0x56304752cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.312+0000 7fa055ffb640 10 client.18006 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:33.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.312+0000 7fa071bdb640 1 --2- 172.21.15.103:0/803019037 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56304752a850 0x56304752cca0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac9e0d3046721e20 server_cookie=d51c5b981d14ad06 in_seq=0 out_seq=0 2024-09-19T06:56:33.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.313+0000 7fa055ffb640 1 -- 172.21.15.103:0/803019037 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563047481890 con 0x56304752a850 2024-09-19T06:56:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.313+0000 7fa055ffb640 10 client.18006 handle_command_reply: tid=0 2024-09-19T06:56:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.313+0000 7fa076a19740 1 client.18006 shutdown 2024-09-19T06:56:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.313+0000 7fa076a19740 1 -- 172.21.15.103:0/803019037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563047510170 msgr2=0x563047517ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.313+0000 7fa076a19740 1 --2- 172.21.15.103:0/803019037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563047510170 0x563047517ea0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa06800bf10 tx=0x7fa068012820 comp rx=0 tx=0).stop 2024-09-19T06:56:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.313+0000 7fa076a19740 1 -- 172.21.15.103:0/803019037 shutdown_connections 2024-09-19T06:56:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.313+0000 7fa076a19740 1 --2- 172.21.15.103:0/803019037 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56304752a850 0x56304752cca0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.313+0000 7fa076a19740 1 --2- 172.21.15.103:0/803019037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304751aa30 0x5630475183e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.313+0000 7fa076a19740 1 --2- 172.21.15.103:0/803019037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563047510170 0x563047517ea0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.313+0000 7fa076a19740 1 --2- 172.21.15.103:0/803019037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304750f850 0x563047517960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.313+0000 7fa076a19740 1 -- 172.21.15.103:0/803019037 >> 172.21.15.103:0/803019037 conn(0x56304750bde0 msgr2=0x563047511ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.313+0000 7fa055ffb640 0 client.18006 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.313+0000 7fa076a19740 1 -- 172.21.15.103:0/803019037 shutdown_connections 2024-09-19T06:56:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.313+0000 7fa076a19740 1 -- 172.21.15.103:0/803019037 wait complete. 2024-09-19T06:56:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.313+0000 7fa076a19740 20 client.18006 trim_cache size 0 max 16384 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.330+0000 7fa076a19740 1 Processor -- start 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.330+0000 7fa076a19740 1 -- start start 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.330+0000 7fa076a19740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304750f850 0x563047532570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.330+0000 7fa076a19740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563047510170 0x563047532ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.330+0000 7fa076a19740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304751aa30 0x5630475191c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.330+0000 7fa076a19740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1afb80 con 0x563047510170 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.330+0000 7fa076a19740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1b16c0 con 0x56304751aa30 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.330+0000 7fa076a19740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1b0250 con 0x56304750f850 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.331+0000 7fa0723dc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563047510170 0x563047532ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.331+0000 7fa0723dc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563047510170 0x563047532ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45370/0 (socket says 172.21.15.103:45370) 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.331+0000 7fa0723dc640 1 -- 172.21.15.103:0/1867176400 learned_addr learned my addr 172.21.15.103:0/1867176400 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.331+0000 7fa071bdb640 1 --2- 172.21.15.103:0/1867176400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304750f850 0x563047532570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.331+0000 7fa071bdb640 1 --2- 172.21.15.103:0/1867176400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304751aa30 0x5630475191c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.331+0000 7fa0723dc640 1 -- 172.21.15.103:0/1867176400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304750f850 msgr2=0x563047532570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.331+0000 7fa0723dc640 1 --2- 172.21.15.103:0/1867176400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304750f850 0x563047532570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.331+0000 7fa0723dc640 1 -- 172.21.15.103:0/1867176400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304751aa30 msgr2=0x5630475191c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.331+0000 7fa0723dc640 1 --2- 172.21.15.103:0/1867176400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304751aa30 0x5630475191c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.331+0000 7fa0723dc640 1 -- 172.21.15.103:0/1867176400 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56304750f680 con 0x563047510170 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.331+0000 7fa071bdb640 1 --2- 172.21.15.103:0/1867176400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304751aa30 0x5630475191c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.331+0000 7fa0723dc640 1 --2- 172.21.15.103:0/1867176400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563047510170 0x563047532ab0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fa068005880 tx=0x7fa06800bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623f2e303ae796e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.331+0000 7fa057fff640 1 -- 172.21.15.103:0/1867176400 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068028040 con 0x563047510170 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.331+0000 7fa057fff640 1 -- 172.21.15.103:0/1867176400 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa068005510 con 0x563047510170 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.331+0000 7fa057fff640 1 -- 172.21.15.103:0/1867176400 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06801b7f0 con 0x563047510170 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 -- 172.21.15.103:0/1867176400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563047510170 msgr2=0x563047532ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 --2- 172.21.15.103:0/1867176400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563047510170 0x563047532ab0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fa068005880 tx=0x7fa06800bad0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 -- 172.21.15.103:0/1867176400 shutdown_connections 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 --2- 172.21.15.103:0/1867176400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304751aa30 0x5630475191c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 --2- 172.21.15.103:0/1867176400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563047510170 0x563047532ab0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 --2- 172.21.15.103:0/1867176400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304750f850 0x563047532570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 -- 172.21.15.103:0/1867176400 >> 172.21.15.103:0/1867176400 conn(0x56304750bde0 msgr2=0x563047516230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 -- 172.21.15.103:0/1867176400 shutdown_connections 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 -- 172.21.15.103:0/1867176400 wait complete. 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 -- start start 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 Processor -- start 2024-09-19T06:56:33.336 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:32 smithi174 ceph-mon[35637]: pgmap v224: 97 pgs: 97 active+clean; 5.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:56:33.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56304750f850 0x563047523150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563047510170 0x56304751e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304751aa30 0x56304751e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1b0ff0 con 0x56304750f850 2024-09-19T06:56:33.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1b0920 con 0x56304751aa30 2024-09-19T06:56:33.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.332+0000 7fa076a19740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1adb80 con 0x563047510170 2024-09-19T06:56:33.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa076a19740 10 client.0 _ll_register_callbacks cb 0x7fa06800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:33.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa071bdb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56304750f850 0x563047523150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa071bdb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56304750f850 0x563047523150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45384/0 (socket says 172.21.15.103:45384) 2024-09-19T06:56:33.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa071bdb640 1 -- 172.21.15.103:0/3304839097 learned_addr learned my addr 172.21.15.103:0/3304839097 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:33.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa0723dc640 1 --2- 172.21.15.103:0/3304839097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563047510170 0x56304751e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa071bdb640 1 --2- 172.21.15.103:0/3304839097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304751aa30 0x56304751e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa071bdb640 1 -- 172.21.15.103:0/3304839097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563047510170 msgr2=0x56304751e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:33.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa071bdb640 1 --2- 172.21.15.103:0/3304839097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563047510170 0x56304751e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa071bdb640 1 -- 172.21.15.103:0/3304839097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304751aa30 msgr2=0x56304751e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:33.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa071bdb640 1 --2- 172.21.15.103:0/3304839097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304751aa30 0x56304751e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa071bdb640 1 -- 172.21.15.103:0/3304839097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56304751f1a0 con 0x56304750f850 2024-09-19T06:56:33.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa071bdb640 1 --2- 172.21.15.103:0/3304839097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56304750f850 0x563047523150 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa05c031700 tx=0x7fa05c04f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676825422f2d3d64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:33.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa055ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:33.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa055ffb640 1 -- 172.21.15.103:0/3304839097 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c035040 con 0x56304750f850 2024-09-19T06:56:33.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa055ffb640 1 -- 172.21.15.103:0/3304839097 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa05c0319b0 con 0x56304750f850 2024-09-19T06:56:33.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa076a19740 1 -- 172.21.15.103:0/3304839097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa068005510 con 0x56304750f850 2024-09-19T06:56:33.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa055ffb640 1 -- 172.21.15.103:0/3304839097 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa05c061050 con 0x56304750f850 2024-09-19T06:56:33.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa076a19740 10 client.18012 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:33.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.333+0000 7fa076a19740 1 -- 172.21.15.103:0/3304839097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563047595590 con 0x56304750f850 2024-09-19T06:56:33.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.334+0000 7fa055ffb640 1 -- 172.21.15.103:0/3304839097 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa068005510 con 0x56304750f850 2024-09-19T06:56:33.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.334+0000 7fa055ffb640 1 -- 172.21.15.103:0/3304839097 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa05c03d480 con 0x56304750f850 2024-09-19T06:56:33.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.334+0000 7fa076a19740 10 client.18012 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:33.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.334+0000 7fa076a19740 10 client.18012 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:56:33.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.334+0000 7fa076a19740 20 client.18012 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:33.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.334+0000 7fa076a19740 1 --2- 172.21.15.103:0/3304839097 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5630475958d0 0x563047597cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.334+0000 7fa076a19740 4 client.18012 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}] 2024-09-19T06:56:33.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.334+0000 7fa076a19740 1 -- 172.21.15.103:0/3304839097 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}) -- 0x563047507410 con 0x5630475958d0 2024-09-19T06:56:33.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.334+0000 7fa0723dc640 1 --2- 172.21.15.103:0/3304839097 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5630475958d0 0x563047597cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.334+0000 7fa055ffb640 10 client.18012 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:33.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.334+0000 7fa0723dc640 1 --2- 172.21.15.103:0/3304839097 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5630475958d0 0x563047597cc0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f652d709980b9fe server_cookie=70d9371ac5fe7ace in_seq=0 out_seq=0 2024-09-19T06:56:33.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.345+0000 7fa055ffb640 1 -- 172.21.15.103:0/3304839097 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563047507410 con 0x5630475958d0 2024-09-19T06:56:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.345+0000 7fa055ffb640 10 client.18012 handle_command_reply: tid=0 2024-09-19T06:56:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.345+0000 7fa076a19740 1 client.18012 shutdown 2024-09-19T06:56:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.345+0000 7fa076a19740 1 -- 172.21.15.103:0/3304839097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56304750f850 msgr2=0x563047523150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.345+0000 7fa076a19740 1 --2- 172.21.15.103:0/3304839097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56304750f850 0x563047523150 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa05c031700 tx=0x7fa05c04f040 comp rx=0 tx=0).stop 2024-09-19T06:56:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.345+0000 7fa076a19740 1 -- 172.21.15.103:0/3304839097 shutdown_connections 2024-09-19T06:56:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.345+0000 7fa076a19740 1 --2- 172.21.15.103:0/3304839097 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5630475958d0 0x563047597cc0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa076a19740 1 --2- 172.21.15.103:0/3304839097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56304751aa30 0x56304751e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa076a19740 1 --2- 172.21.15.103:0/3304839097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563047510170 0x56304751e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa076a19740 1 --2- 172.21.15.103:0/3304839097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56304750f850 0x563047523150 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa055ffb640 0 client.18012 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa076a19740 1 -- 172.21.15.103:0/3304839097 >> 172.21.15.103:0/3304839097 conn(0x56304750bde0 msgr2=0x563047517900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa076a19740 1 -- 172.21.15.103:0/3304839097 shutdown_connections 2024-09-19T06:56:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa076a19740 1 -- 172.21.15.103:0/3304839097 wait complete. 2024-09-19T06:56:33.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa076a19740 20 client.18012 trim_cache size 0 max 16384 2024-09-19T06:56:33.348 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:56:33.348 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:56:33.348 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:56:33.348 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 26, 2024-09-19T06:56:33.348 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:56:33.348 INFO:teuthology.orchestra.run.smithi103.stdout: "f2ce1f2b": { 2024-09-19T06:56:33.348 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 26, 2024-09-19T06:56:33.348 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:56:33.349 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:56:33.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:56:33.349 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:56:33.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:56:33.349 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:56:33.349 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 145.8, 2024-09-19T06:56:33.349 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:56:33.349 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:56:33.349 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:56:33.349 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:56:33.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:56:33.349 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:56:33.349 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:56:33.349 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:56:33.349 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:56:33.350 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:56:33.350 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:56:33.350 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:56:33.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa074640640 1 -- 172.21.15.103:0/1684350888 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa0580a33b0 msgr2=0x7fa0580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa074640640 1 --2- 172.21.15.103:0/1684350888 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa0580a33b0 0x7fa0580a5870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa0640cd4e0 tx=0x7fa06406acb0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa074640640 1 -- 172.21.15.103:0/1684350888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa06c102ef0 msgr2=0x7fa06c0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa074640640 1 --2- 172.21.15.103:0/1684350888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa06c102ef0 0x7fa06c0fd700 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fa06400cad0 tx=0x7fa06400cf90 comp rx=0 tx=0).stop 2024-09-19T06:56:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa074640640 1 -- 172.21.15.103:0/1684350888 shutdown_connections 2024-09-19T06:56:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa074640640 1 --2- 172.21.15.103:0/1684350888 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa0580a33b0 0x7fa0580a5870 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa074640640 1 --2- 172.21.15.103:0/1684350888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa06c102ef0 0x7fa06c0fd700 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa074640640 1 --2- 172.21.15.103:0/1684350888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa06c102570 0x7fa06c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa074640640 1 --2- 172.21.15.103:0/1684350888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa06c100e20 0x7fa06c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa074640640 1 -- 172.21.15.103:0/1684350888 >> 172.21.15.103:0/1684350888 conn(0x7fa06c0fc7d0 msgr2=0x7fa06c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa074640640 1 -- 172.21.15.103:0/1684350888 shutdown_connections 2024-09-19T06:56:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:33.346+0000 7fa074640640 1 -- 172.21.15.103:0/1684350888 wait complete. 2024-09-19T06:56:34.231 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f2ce1f2b', quiesce took 3.4 seconds. Will release after: 22.35 2024-09-19T06:56:34.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:34 smithi103 ceph-mon[31158]: pgmap v225: 97 pgs: 97 active+clean; 5.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:56:34.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:34 smithi134 ceph-mon[35356]: pgmap v225: 97 pgs: 97 active+clean; 5.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:56:34.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:34 smithi174 ceph-mon[35637]: pgmap v225: 97 pgs: 97 active+clean; 5.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:56:36.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:36 smithi103 ceph-mon[31158]: pgmap v226: 97 pgs: 97 active+clean; 5.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-09-19T06:56:36.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:36 smithi134 ceph-mon[35356]: pgmap v226: 97 pgs: 97 active+clean; 5.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-09-19T06:56:36.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:36 smithi174 ceph-mon[35637]: pgmap v226: 97 pgs: 97 active+clean; 5.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-09-19T06:56:38.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:38 smithi103 ceph-mon[31158]: pgmap v227: 97 pgs: 97 active+clean; 5.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T06:56:38.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:38 smithi134 ceph-mon[35356]: pgmap v227: 97 pgs: 97 active+clean; 5.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T06:56:38.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:38 smithi174 ceph-mon[35637]: pgmap v227: 97 pgs: 97 active+clean; 5.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T06:56:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.876+0000 7fb01b496640 1 -- 172.21.15.103:0/319348790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb014102460 msgr2=0x7fb01410e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.876+0000 7fb01b496640 1 --2- 172.21.15.103:0/319348790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb014102460 0x7fb01410e990 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009cd0 tx=0x7fb00c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.876+0000 7fb01b496640 1 -- 172.21.15.103:0/319348790 shutdown_connections 2024-09-19T06:56:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.876+0000 7fb01b496640 1 --2- 172.21.15.103:0/319348790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb014102460 0x7fb01410e990 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.876+0000 7fb01b496640 1 --2- 172.21.15.103:0/319348790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb014101ac0 0x7fb014101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.876+0000 7fb01b496640 1 --2- 172.21.15.103:0/319348790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb014107ad0 0x7fb014107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.876+0000 7fb01b496640 1 -- 172.21.15.103:0/319348790 >> 172.21.15.103:0/319348790 conn(0x7fb0140fd820 msgr2=0x7fb0140ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.876+0000 7fb01b496640 1 -- 172.21.15.103:0/319348790 shutdown_connections 2024-09-19T06:56:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.876+0000 7fb01b496640 1 -- 172.21.15.103:0/319348790 wait complete. 2024-09-19T06:56:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.877+0000 7fb01b496640 1 Processor -- start 2024-09-19T06:56:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.877+0000 7fb01b496640 1 -- start start 2024-09-19T06:56:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.877+0000 7fb01b496640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb014101ac0 0x7fb014198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.877+0000 7fb01b496640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb014102460 0x7fb0141988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.877+0000 7fb01b496640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb014107ad0 0x7fb01419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.877+0000 7fb01b496640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb014111f10 con 0x7fb014102460 2024-09-19T06:56:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.877+0000 7fb01b496640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb014111d90 con 0x7fb014101ac0 2024-09-19T06:56:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.877+0000 7fb01b496640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb014112090 con 0x7fb014107ad0 2024-09-19T06:56:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.877+0000 7fb018a31640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb014102460 0x7fb0141988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.878+0000 7fb018a31640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb014102460 0x7fb0141988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43976/0 (socket says 172.21.15.103:43976) 2024-09-19T06:56:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.878+0000 7fb018a31640 1 -- 172.21.15.103:0/1727630807 learned_addr learned my addr 172.21.15.103:0/1727630807 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.878+0000 7fb019232640 1 --2- 172.21.15.103:0/1727630807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb014101ac0 0x7fb014198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.878+0000 7fb019a33640 1 --2- 172.21.15.103:0/1727630807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb014107ad0 0x7fb01419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.878+0000 7fb018a31640 1 -- 172.21.15.103:0/1727630807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb014107ad0 msgr2=0x7fb01419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.878+0000 7fb018a31640 1 --2- 172.21.15.103:0/1727630807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb014107ad0 0x7fb01419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.878+0000 7fb018a31640 1 -- 172.21.15.103:0/1727630807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb014101ac0 msgr2=0x7fb014198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.878+0000 7fb018a31640 1 --2- 172.21.15.103:0/1727630807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb014101ac0 0x7fb014198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.878+0000 7fb018a31640 1 -- 172.21.15.103:0/1727630807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01419d370 con 0x7fb014102460 2024-09-19T06:56:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.878+0000 7fb019232640 1 --2- 172.21.15.103:0/1727630807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb014101ac0 0x7fb014198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.878+0000 7fb018a31640 1 --2- 172.21.15.103:0/1727630807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb014102460 0x7fb0141988c0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fb00400d770 tx=0x7fb00400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.878+0000 7fb00a7fc640 1 -- 172.21.15.103:0/1727630807 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004017070 con 0x7fb014102460 2024-09-19T06:56:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.878+0000 7fb00a7fc640 1 -- 172.21.15.103:0/1727630807 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb004005500 con 0x7fb014102460 2024-09-19T06:56:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.878+0000 7fb01b496640 1 -- 172.21.15.103:0/1727630807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01419d660 con 0x7fb014102460 2024-09-19T06:56:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.878+0000 7fb00a7fc640 1 -- 172.21.15.103:0/1727630807 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004004e00 con 0x7fb014102460 2024-09-19T06:56:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.878+0000 7fb01b496640 1 -- 172.21.15.103:0/1727630807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0141a4ef0 con 0x7fb014102460 2024-09-19T06:56:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.881+0000 7fb00a7fc640 1 -- 172.21.15.103:0/1727630807 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fb004004fa0 con 0x7fb014102460 2024-09-19T06:56:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.881+0000 7fb00a7fc640 1 --2- 172.21.15.103:0/1727630807 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb0000a3460 0x7fb0000a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.881+0000 7fb00a7fc640 1 -- 172.21.15.103:0/1727630807 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fb0040cf870 con 0x7fb014102460 2024-09-19T06:56:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.881+0000 7fb019232640 1 --2- 172.21.15.103:0/1727630807 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb0000a3460 0x7fb0000a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.882+0000 7fb019232640 1 --2- 172.21.15.103:0/1727630807 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb0000a3460 0x7fb0000a5920 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb010005e00 tx=0x7fb010005c70 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.882+0000 7fb01e22f740 1 Processor -- start 2024-09-19T06:56:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.882+0000 7fb01e22f740 1 -- start start 2024-09-19T06:56:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.882+0000 7fb01e22f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df8741af0 0x560df8741ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.882+0000 7fb01e22f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df874c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.882+0000 7fb01e22f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df874ccb0 0x560df874f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.882+0000 7fb01e22f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560df86a62f0 con 0x560df874ccb0 2024-09-19T06:56:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.882+0000 7fb01e22f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560df864e270 con 0x560df8741af0 2024-09-19T06:56:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.882+0000 7fb01e22f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560df86a22b0 con 0x560df8742410 2024-09-19T06:56:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.883+0000 7fb019a33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df874ccb0 0x560df874f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.883+0000 7fb019a33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df874ccb0 0x560df874f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43984/0 (socket says 172.21.15.103:43984) 2024-09-19T06:56:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.883+0000 7fb019a33640 1 -- 172.21.15.103:0/2439907311 learned_addr learned my addr 172.21.15.103:0/2439907311 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.883+0000 7fb018a31640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df874c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.883+0000 7fb019232640 1 --2- 172.21.15.103:0/2439907311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df8741af0 0x560df8741ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.883+0000 7fb019a33640 1 -- 172.21.15.103:0/2439907311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 msgr2=0x560df874c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.883+0000 7fb019a33640 1 --2- 172.21.15.103:0/2439907311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df874c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.883+0000 7fb019a33640 1 -- 172.21.15.103:0/2439907311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df8741af0 msgr2=0x560df8741ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.883+0000 7fb019a33640 1 --2- 172.21.15.103:0/2439907311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df8741af0 0x560df8741ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.883+0000 7fb019a33640 1 -- 172.21.15.103:0/2439907311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df864d340 con 0x560df874ccb0 2024-09-19T06:56:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.883+0000 7fb018a31640 1 --2- 172.21.15.103:0/2439907311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df874c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:56:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.883+0000 7fb019a33640 1 --2- 172.21.15.103:0/2439907311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df874ccb0 0x560df874f0a0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fb00c005cc0 tx=0x7fb00c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf78758bfd21641 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.883+0000 7fafff7fe640 1 -- 172.21.15.103:0/2439907311 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0318c0 con 0x560df874ccb0 2024-09-19T06:56:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.883+0000 7fafff7fe640 1 -- 172.21.15.103:0/2439907311 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb00c036040 con 0x560df874ccb0 2024-09-19T06:56:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.883+0000 7fafff7fe640 1 -- 172.21.15.103:0/2439907311 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c004400 con 0x560df874ccb0 2024-09-19T06:56:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.884+0000 7fb01e22f740 1 -- 172.21.15.103:0/2439907311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df874ccb0 msgr2=0x560df874f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.884+0000 7fb01e22f740 1 --2- 172.21.15.103:0/2439907311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df874ccb0 0x560df874f0a0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fb00c005cc0 tx=0x7fb00c009a20 comp rx=0 tx=0).stop 2024-09-19T06:56:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.884+0000 7fb01e22f740 1 -- 172.21.15.103:0/2439907311 shutdown_connections 2024-09-19T06:56:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.884+0000 7fb01e22f740 1 --2- 172.21.15.103:0/2439907311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df874ccb0 0x560df874f0a0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.884+0000 7fb01e22f740 1 --2- 172.21.15.103:0/2439907311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df874c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.884+0000 7fb01e22f740 1 --2- 172.21.15.103:0/2439907311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df8741af0 0x560df8741ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.884+0000 7fb01e22f740 1 -- 172.21.15.103:0/2439907311 >> 172.21.15.103:0/2439907311 conn(0x560df873e060 msgr2=0x560df873e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.884+0000 7fb01e22f740 1 -- 172.21.15.103:0/2439907311 shutdown_connections 2024-09-19T06:56:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.884+0000 7fb01e22f740 1 -- 172.21.15.103:0/2439907311 wait complete. 2024-09-19T06:56:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.885+0000 7fb01e22f740 1 -- start start 2024-09-19T06:56:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.885+0000 7fb01e22f740 1 Processor -- start 2024-09-19T06:56:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.885+0000 7fb01e22f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df8741af0 0x560df8749ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.885+0000 7fb01e22f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df874a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.885+0000 7fb01e22f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df874ccb0 0x560df874a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.885+0000 7fb01e22f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560df874f880 con 0x560df874ccb0 2024-09-19T06:56:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.885+0000 7fb01e22f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560df862f290 con 0x560df8741af0 2024-09-19T06:56:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.885+0000 7fb01e22f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560df874fa00 con 0x560df8742410 2024-09-19T06:56:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.885+0000 7fb01e22f740 10 client.0 _ll_register_callbacks cb 0x560df8660140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.885+0000 7fb019a33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df874ccb0 0x560df874a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.885+0000 7fb019232640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df8741af0 0x560df8749ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.885+0000 7fb019a33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df874ccb0 0x560df874a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43990/0 (socket says 172.21.15.103:43990) 2024-09-19T06:56:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.886+0000 7fb019a33640 1 -- 172.21.15.103:0/3280116060 learned_addr learned my addr 172.21.15.103:0/3280116060 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.886+0000 7fb018a31640 1 --2- 172.21.15.103:0/3280116060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df874a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.886+0000 7fb019a33640 1 -- 172.21.15.103:0/3280116060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 msgr2=0x560df874a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.886+0000 7fb019a33640 1 --2- 172.21.15.103:0/3280116060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df874a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.886+0000 7fb019a33640 1 -- 172.21.15.103:0/3280116060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df8741af0 msgr2=0x560df8749ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.886+0000 7fb019a33640 1 --2- 172.21.15.103:0/3280116060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df8741af0 0x560df8749ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.886+0000 7fb019a33640 1 -- 172.21.15.103:0/3280116060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df83b9210 con 0x560df874ccb0 2024-09-19T06:56:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.886+0000 7fb019a33640 1 --2- 172.21.15.103:0/3280116060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df874ccb0 0x560df874a620 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fb00c02a920 tx=0x7fb00c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87baa2cd3d444d06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.886+0000 7faffcff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.886+0000 7faffcff9640 1 -- 172.21.15.103:0/3280116060 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c056060 con 0x560df874ccb0 2024-09-19T06:56:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.886+0000 7faffcff9640 1 -- 172.21.15.103:0/3280116060 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fb00c031530 con 0x560df874ccb0 2024-09-19T06:56:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.886+0000 7fb01e22f740 1 -- 172.21.15.103:0/3280116060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb00c036040 con 0x560df874ccb0 2024-09-19T06:56:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.886+0000 7faffcff9640 1 -- 172.21.15.103:0/3280116060 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb00c0041c0 con 0x560df874ccb0 2024-09-19T06:56:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.886+0000 7fb01e22f740 10 client.18024 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.886+0000 7fb01e22f740 1 -- 172.21.15.103:0/3280116060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560df8485620 con 0x560df874ccb0 2024-09-19T06:56:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.887+0000 7faffcff9640 1 -- 172.21.15.103:0/3280116060 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb00c036040 con 0x560df874ccb0 2024-09-19T06:56:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.887+0000 7faffcff9640 1 -- 172.21.15.103:0/3280116060 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb00c030070 con 0x560df874ccb0 2024-09-19T06:56:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.887+0000 7fb01e22f740 10 client.18024 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.887+0000 7fb01e22f740 10 client.18024 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.887+0000 7fb01e22f740 20 client.18024 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.887+0000 7fb01e22f740 1 --2- 172.21.15.103:0/3280116060 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560df875ca90 0x560df875ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.887+0000 7fb01e22f740 4 client.18024 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.887+0000 7fb01e22f740 1 -- 172.21.15.103:0/3280116060 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560df86b3a20 con 0x560df875ca90 2024-09-19T06:56:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.887+0000 7fb019232640 1 --2- 172.21.15.103:0/3280116060 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560df875ca90 0x560df875ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.888+0000 7fb019232640 1 --2- 172.21.15.103:0/3280116060 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560df875ca90 0x560df875ee80 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca17b91f0e99bfbf server_cookie=e482e24107a06652 in_seq=0 out_seq=0 2024-09-19T06:56:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.888+0000 7faffcff9640 10 client.18024 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.888+0000 7faffcff9640 1 -- 172.21.15.103:0/3280116060 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560df86b3a20 con 0x560df875ca90 2024-09-19T06:56:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.888+0000 7faffcff9640 10 client.18024 handle_command_reply: tid=0 2024-09-19T06:56:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.888+0000 7fb01e22f740 1 client.18024 shutdown 2024-09-19T06:56:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.888+0000 7fb01e22f740 1 -- 172.21.15.103:0/3280116060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df874ccb0 msgr2=0x560df874a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.888+0000 7fb01e22f740 1 --2- 172.21.15.103:0/3280116060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df874ccb0 0x560df874a620 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fb00c02a920 tx=0x7fb00c004a40 comp rx=0 tx=0).stop 2024-09-19T06:56:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.889+0000 7fb01e22f740 1 -- 172.21.15.103:0/3280116060 shutdown_connections 2024-09-19T06:56:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.889+0000 7fb01e22f740 1 --2- 172.21.15.103:0/3280116060 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560df875ca90 0x560df875ee80 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.889+0000 7fb01e22f740 1 --2- 172.21.15.103:0/3280116060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df874ccb0 0x560df874a620 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.889+0000 7fb01e22f740 1 --2- 172.21.15.103:0/3280116060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df874a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.889+0000 7faffcff9640 0 client.18024 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.889+0000 7fb01e22f740 1 --2- 172.21.15.103:0/3280116060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df8741af0 0x560df8749ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.889+0000 7fb01e22f740 1 -- 172.21.15.103:0/3280116060 >> 172.21.15.103:0/3280116060 conn(0x560df873e060 msgr2=0x560df8743d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.889+0000 7fb01e22f740 1 -- 172.21.15.103:0/3280116060 shutdown_connections 2024-09-19T06:56:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.889+0000 7fb01e22f740 1 -- 172.21.15.103:0/3280116060 wait complete. 2024-09-19T06:56:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.889+0000 7fb01e22f740 20 client.18024 trim_cache size 0 max 16384 2024-09-19T06:56:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.903+0000 7fb01e22f740 1 Processor -- start 2024-09-19T06:56:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.903+0000 7fb01e22f740 1 -- start start 2024-09-19T06:56:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.903+0000 7fb01e22f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df8741af0 0x560df8744560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.903+0000 7fb01e22f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df874a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.903+0000 7fb01e22f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df874ccb0 0x560df8761ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.903+0000 7fb01e22f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0141ad7e0 con 0x560df8741af0 2024-09-19T06:56:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.903+0000 7fb01e22f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0141af320 con 0x560df874ccb0 2024-09-19T06:56:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.903+0000 7fb01e22f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0141adeb0 con 0x560df8742410 2024-09-19T06:56:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.903+0000 7fb019232640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df8741af0 0x560df8744560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.903+0000 7fb019232640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df8741af0 0x560df8744560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43992/0 (socket says 172.21.15.103:43992) 2024-09-19T06:56:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.903+0000 7fb019232640 1 -- 172.21.15.103:0/2285278410 learned_addr learned my addr 172.21.15.103:0/2285278410 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.904+0000 7fb019232640 1 -- 172.21.15.103:0/2285278410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 msgr2=0x560df874a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.904+0000 7fb019232640 1 --2- 172.21.15.103:0/2285278410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df874a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.904+0000 7fb019232640 1 -- 172.21.15.103:0/2285278410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df874ccb0 msgr2=0x560df8761ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.904+0000 7fb019232640 1 --2- 172.21.15.103:0/2285278410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df874ccb0 0x560df8761ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.904+0000 7fb019232640 1 -- 172.21.15.103:0/2285278410 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df8743000 con 0x560df8741af0 2024-09-19T06:56:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.904+0000 7fb019232640 1 --2- 172.21.15.103:0/2285278410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df8741af0 0x560df8744560 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x560df86a6400 tx=0x7fb01000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d771844e5d87d7ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.904+0000 7faffeffd640 1 -- 172.21.15.103:0/2285278410 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010021110 con 0x560df8741af0 2024-09-19T06:56:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.904+0000 7faffeffd640 1 -- 172.21.15.103:0/2285278410 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0100035f0 con 0x560df8741af0 2024-09-19T06:56:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.904+0000 7faffeffd640 1 -- 172.21.15.103:0/2285278410 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0100212b0 con 0x560df8741af0 2024-09-19T06:56:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.904+0000 7fb01e22f740 1 -- 172.21.15.103:0/2285278410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df8741af0 msgr2=0x560df8744560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.904+0000 7fb01e22f740 1 --2- 172.21.15.103:0/2285278410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df8741af0 0x560df8744560 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x560df86a6400 tx=0x7fb01000bf90 comp rx=0 tx=0).stop 2024-09-19T06:56:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.904+0000 7fb01e22f740 1 -- 172.21.15.103:0/2285278410 shutdown_connections 2024-09-19T06:56:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.904+0000 7fb01e22f740 1 --2- 172.21.15.103:0/2285278410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df874ccb0 0x560df8761ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.904+0000 7fb01e22f740 1 --2- 172.21.15.103:0/2285278410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df874a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.904+0000 7fb01e22f740 1 --2- 172.21.15.103:0/2285278410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df8741af0 0x560df8744560 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.905+0000 7fb01e22f740 1 -- 172.21.15.103:0/2285278410 >> 172.21.15.103:0/2285278410 conn(0x560df873e060 msgr2=0x560df874bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.905+0000 7fb01e22f740 1 -- 172.21.15.103:0/2285278410 shutdown_connections 2024-09-19T06:56:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.905+0000 7fb01e22f740 1 -- 172.21.15.103:0/2285278410 wait complete. 2024-09-19T06:56:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.905+0000 7fb01e22f740 1 -- start start 2024-09-19T06:56:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.905+0000 7fb01e22f740 1 Processor -- start 2024-09-19T06:56:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.905+0000 7fb01e22f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df8741af0 0x560df8762380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.905+0000 7fb01e22f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df875f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.905+0000 7fb01e22f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df874ccb0 0x560df875fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.905+0000 7fb01e22f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0141aec50 con 0x560df8741af0 2024-09-19T06:56:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.905+0000 7fb01e22f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0141ae580 con 0x560df874ccb0 2024-09-19T06:56:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.905+0000 7fb01e22f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0141ab7e0 con 0x560df8742410 2024-09-19T06:56:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.905+0000 7fb01e22f740 10 client.0 _ll_register_callbacks cb 0x7fb00400dea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.905+0000 7fb019232640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df8741af0 0x560df8762380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.905+0000 7fb019232640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df8741af0 0x560df8762380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44006/0 (socket says 172.21.15.103:44006) 2024-09-19T06:56:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.905+0000 7fb019232640 1 -- 172.21.15.103:0/1320858960 learned_addr learned my addr 172.21.15.103:0/1320858960 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7fb019a33640 1 --2- 172.21.15.103:0/1320858960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df874ccb0 0x560df875fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7fb018a31640 1 --2- 172.21.15.103:0/1320858960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df875f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7fb019232640 1 -- 172.21.15.103:0/1320858960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 msgr2=0x560df875f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7fb019232640 1 --2- 172.21.15.103:0/1320858960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df875f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7fb019232640 1 -- 172.21.15.103:0/1320858960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df874ccb0 msgr2=0x560df875fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7fb019232640 1 --2- 172.21.15.103:0/1320858960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df874ccb0 0x560df875fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7fb019232640 1 -- 172.21.15.103:0/1320858960 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df87601c0 con 0x560df8741af0 2024-09-19T06:56:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7fb018a31640 1 --2- 172.21.15.103:0/1320858960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df875f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7fb019232640 1 --2- 172.21.15.103:0/1320858960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df8741af0 0x560df8762380 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fb01002b040 tx=0x7fb0100212f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3b8126f04c1400 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7faffcff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7faffcff9640 1 -- 172.21.15.103:0/1320858960 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0100042c0 con 0x560df8741af0 2024-09-19T06:56:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7faffcff9640 1 -- 172.21.15.103:0/1320858960 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fb010004460 con 0x560df8741af0 2024-09-19T06:56:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7fb01e22f740 1 -- 172.21.15.103:0/1320858960 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0100035f0 con 0x560df8741af0 2024-09-19T06:56:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7faffcff9640 1 -- 172.21.15.103:0/1320858960 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0100037a0 con 0x560df8741af0 2024-09-19T06:56:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7fb01e22f740 10 client.18030 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7fb01e22f740 1 -- 172.21.15.103:0/1320858960 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560df87c7150 con 0x560df8741af0 2024-09-19T06:56:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.906+0000 7faffcff9640 1 -- 172.21.15.103:0/1320858960 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0100035f0 con 0x560df8741af0 2024-09-19T06:56:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.907+0000 7faffcff9640 1 -- 172.21.15.103:0/1320858960 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb01003b030 con 0x560df8741af0 2024-09-19T06:56:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.907+0000 7fb01e22f740 10 client.18030 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.907+0000 7fb01e22f740 10 client.18030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.907+0000 7fb01e22f740 20 client.18030 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.907+0000 7fb01e22f740 1 --2- 172.21.15.103:0/1320858960 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560df87c7800 0x560df87c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.907+0000 7fb01e22f740 4 client.18030 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:56:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.907+0000 7fb01e22f740 1 -- 172.21.15.103:0/1320858960 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560df86add70 con 0x560df87c7800 2024-09-19T06:56:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.907+0000 7fb018a31640 1 --2- 172.21.15.103:0/1320858960 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560df87c7800 0x560df87c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.907+0000 7faffcff9640 10 client.18030 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.907+0000 7fb018a31640 1 --2- 172.21.15.103:0/1320858960 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560df87c7800 0x560df87c9c20 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63feba2540e46c82 server_cookie=c512953079ba4c28 in_seq=0 out_seq=0 2024-09-19T06:56:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.920+0000 7faffcff9640 1 -- 172.21.15.103:0/1320858960 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560df86add70 con 0x560df87c7800 2024-09-19T06:56:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.920+0000 7faffcff9640 10 client.18030 handle_command_reply: tid=0 2024-09-19T06:56:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.920+0000 7fb01e22f740 1 client.18030 shutdown 2024-09-19T06:56:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.920+0000 7fb01e22f740 1 -- 172.21.15.103:0/1320858960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df8741af0 msgr2=0x560df8762380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.920+0000 7fb01e22f740 1 --2- 172.21.15.103:0/1320858960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df8741af0 0x560df8762380 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fb01002b040 tx=0x7fb0100212f0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.920+0000 7fb01e22f740 1 -- 172.21.15.103:0/1320858960 shutdown_connections 2024-09-19T06:56:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.920+0000 7fb01e22f740 1 --2- 172.21.15.103:0/1320858960 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560df87c7800 0x560df87c9c20 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.920+0000 7fb01e22f740 1 --2- 172.21.15.103:0/1320858960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560df874ccb0 0x560df875fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.920+0000 7fb01e22f740 1 --2- 172.21.15.103:0/1320858960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560df8742410 0x560df875f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01e22f740 1 --2- 172.21.15.103:0/1320858960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560df8741af0 0x560df8762380 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.920+0000 7faffcff9640 0 client.18030 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01e22f740 1 -- 172.21.15.103:0/1320858960 >> 172.21.15.103:0/1320858960 conn(0x560df873e060 msgr2=0x560df8757c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01e22f740 1 -- 172.21.15.103:0/1320858960 shutdown_connections 2024-09-19T06:56:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01e22f740 1 -- 172.21.15.103:0/1320858960 wait complete. 2024-09-19T06:56:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01e22f740 20 client.18030 trim_cache size 0 max 16384 2024-09-19T06:56:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:56:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:56:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:56:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:56:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01b496640 1 -- 172.21.15.103:0/1727630807 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb0000a3460 msgr2=0x7fb0000a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01b496640 1 --2- 172.21.15.103:0/1727630807 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb0000a3460 0x7fb0000a5920 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb010005e00 tx=0x7fb010005c70 comp rx=0 tx=0).stop 2024-09-19T06:56:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01b496640 1 -- 172.21.15.103:0/1727630807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb014102460 msgr2=0x7fb0141988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01b496640 1 --2- 172.21.15.103:0/1727630807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb014102460 0x7fb0141988c0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fb00400d770 tx=0x7fb00400dc30 comp rx=0 tx=0).stop 2024-09-19T06:56:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01b496640 1 -- 172.21.15.103:0/1727630807 shutdown_connections 2024-09-19T06:56:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01b496640 1 --2- 172.21.15.103:0/1727630807 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb0000a3460 0x7fb0000a5920 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01b496640 1 --2- 172.21.15.103:0/1727630807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb014107ad0 0x7fb01419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01b496640 1 --2- 172.21.15.103:0/1727630807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb014102460 0x7fb0141988c0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01b496640 1 --2- 172.21.15.103:0/1727630807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb014101ac0 0x7fb014198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01b496640 1 -- 172.21.15.103:0/1727630807 >> 172.21.15.103:0/1727630807 conn(0x7fb0140fd820 msgr2=0x7fb0140fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01b496640 1 -- 172.21.15.103:0/1727630807 shutdown_connections 2024-09-19T06:56:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:38.921+0000 7fb01b496640 1 -- 172.21.15.103:0/1727630807 wait complete. 2024-09-19T06:56:40.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:40 smithi103 ceph-mon[31158]: pgmap v228: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-09-19T06:56:40.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:40 smithi134 ceph-mon[35356]: pgmap v228: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-09-19T06:56:40.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:40 smithi174 ceph-mon[35637]: pgmap v228: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-09-19T06:56:42.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:42 smithi103 ceph-mon[31158]: pgmap v229: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-19T06:56:42.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:42 smithi134 ceph-mon[35356]: pgmap v229: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-19T06:56:42.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:42 smithi174 ceph-mon[35637]: pgmap v229: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-19T06:56:44.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:44 smithi103 ceph-mon[31158]: pgmap v230: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-19T06:56:44.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:44 smithi134 ceph-mon[35356]: pgmap v230: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-19T06:56:44.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:44 smithi174 ceph-mon[35637]: pgmap v230: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-19T06:56:46.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:46 smithi103 ceph-mon[31158]: pgmap v231: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-09-19T06:56:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:46 smithi134 ceph-mon[35356]: pgmap v231: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-09-19T06:56:46.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:46 smithi174 ceph-mon[35637]: pgmap v231: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-09-19T06:56:48.385 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:48 smithi174 ceph-mon[35637]: pgmap v232: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 11 op/s 2024-09-19T06:56:48.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:48 smithi103 ceph-mon[31158]: pgmap v232: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 11 op/s 2024-09-19T06:56:48.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:48 smithi134 ceph-mon[35356]: pgmap v232: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 11 op/s 2024-09-19T06:56:48.677 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:56:48.938 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:49.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.385+0000 7f2760b61640 1 -- 172.21.15.103:0/1962063711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f275c106f10 msgr2=0x7f275c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:49.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.385+0000 7f2760b61640 1 --2- 172.21.15.103:0/1962063711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f275c106f10 0x7f275c10ea90 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2754009ca0 tx=0x7f275402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.385+0000 7f2760b61640 1 -- 172.21.15.103:0/1962063711 shutdown_connections 2024-09-19T06:56:49.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.385+0000 7f2760b61640 1 --2- 172.21.15.103:0/1962063711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f275c106f10 0x7f275c10ea90 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.385+0000 7f2760b61640 1 --2- 172.21.15.103:0/1962063711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f275c102790 0x7f275c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.385+0000 7f2760b61640 1 --2- 172.21.15.103:0/1962063711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f275c101de0 0x7f275c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.385+0000 7f2760b61640 1 -- 172.21.15.103:0/1962063711 >> 172.21.15.103:0/1962063711 conn(0x7f275c0fd8c0 msgr2=0x7f275c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:49.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.386+0000 7f2760b61640 1 -- 172.21.15.103:0/1962063711 shutdown_connections 2024-09-19T06:56:49.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.386+0000 7f2760b61640 1 -- 172.21.15.103:0/1962063711 wait complete. 2024-09-19T06:56:49.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.386+0000 7f2760b61640 1 Processor -- start 2024-09-19T06:56:49.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.386+0000 7f2760b61640 1 -- start start 2024-09-19T06:56:49.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.386+0000 7f2760b61640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f275c101de0 0x7f275c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.386+0000 7f2760b61640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f275c102790 0x7f275c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.386+0000 7f2760b61640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f275c106f10 0x7f275c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.386+0000 7f2760b61640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f275c111f70 con 0x7f275c101de0 2024-09-19T06:56:49.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.386+0000 7f2760b61640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f275c111df0 con 0x7f275c102790 2024-09-19T06:56:49.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.386+0000 7f2760b61640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1120f0 con 0x7f275c106f10 2024-09-19T06:56:49.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.387+0000 7f275a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f275c101de0 0x7f275c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:49.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.387+0000 7f275a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f275c101de0 0x7f275c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59690/0 (socket says 172.21.15.103:59690) 2024-09-19T06:56:49.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.387+0000 7f275a59c640 1 -- 172.21.15.103:0/2160641100 learned_addr learned my addr 172.21.15.103:0/2160641100 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:49.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.387+0000 7f2759d9b640 1 --2- 172.21.15.103:0/2160641100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f275c102790 0x7f275c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:49.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.387+0000 7f275ad9d640 1 --2- 172.21.15.103:0/2160641100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f275c106f10 0x7f275c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:49.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.387+0000 7f275a59c640 1 -- 172.21.15.103:0/2160641100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f275c106f10 msgr2=0x7f275c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:49.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.387+0000 7f275a59c640 1 --2- 172.21.15.103:0/2160641100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f275c106f10 0x7f275c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.387+0000 7f275a59c640 1 -- 172.21.15.103:0/2160641100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f275c102790 msgr2=0x7f275c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:49.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.387+0000 7f275a59c640 1 --2- 172.21.15.103:0/2160641100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f275c102790 0x7f275c198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.387+0000 7f275a59c640 1 -- 172.21.15.103:0/2160641100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275c19d2e0 con 0x7f275c101de0 2024-09-19T06:56:49.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.387+0000 7f275a59c640 1 --2- 172.21.15.103:0/2160641100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f275c101de0 0x7f275c198330 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f274400c920 tx=0x7f274400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:49.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.387+0000 7f27537fe640 1 -- 172.21.15.103:0/2160641100 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744007c70 con 0x7f275c101de0 2024-09-19T06:56:49.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.387+0000 7f27537fe640 1 -- 172.21.15.103:0/2160641100 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2744007e10 con 0x7f275c101de0 2024-09-19T06:56:49.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.387+0000 7f27537fe640 1 -- 172.21.15.103:0/2160641100 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744018890 con 0x7f275c101de0 2024-09-19T06:56:49.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.387+0000 7f2760b61640 1 -- 172.21.15.103:0/2160641100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275c19d5d0 con 0x7f275c101de0 2024-09-19T06:56:49.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.387+0000 7f2760b61640 1 -- 172.21.15.103:0/2160641100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f275c1a4eb0 con 0x7f275c101de0 2024-09-19T06:56:49.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.390+0000 7f27537fe640 1 -- 172.21.15.103:0/2160641100 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f2744018a70 con 0x7f275c101de0 2024-09-19T06:56:49.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.390+0000 7f2762f3a740 1 Processor -- start 2024-09-19T06:56:49.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.390+0000 7f2762f3a740 1 -- start start 2024-09-19T06:56:49.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.390+0000 7f2762f3a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235bf4a0 0x560e235bf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.390+0000 7f2762f3a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235bfdc0 0x560e235ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.390+0000 7f2762f3a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235ca680 0x560e235cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.390+0000 7f2762f3a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560e235252e0 con 0x560e235ca680 2024-09-19T06:56:49.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.390+0000 7f2762f3a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560e234ccce0 con 0x560e235bfdc0 2024-09-19T06:56:49.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.390+0000 7f2762f3a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560e235212a0 con 0x560e235bf4a0 2024-09-19T06:56:49.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.390+0000 7f2759d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235ca680 0x560e235cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:49.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.390+0000 7f2759d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235ca680 0x560e235cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59696/0 (socket says 172.21.15.103:59696) 2024-09-19T06:56:49.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.390+0000 7f2759d9b640 1 -- 172.21.15.103:0/680526038 learned_addr learned my addr 172.21.15.103:0/680526038 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:49.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.391+0000 7f275a59c640 1 --2- 172.21.15.103:0/680526038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235bfdc0 0x560e235ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:49.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.391+0000 7f275ad9d640 1 --2- 172.21.15.103:0/680526038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235bf4a0 0x560e235bf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:49.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.391+0000 7f27537fe640 1 --2- 172.21.15.103:0/2160641100 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f27480a33b0 0x7f27480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.391+0000 7f2759d9b640 1 -- 172.21.15.103:0/680526038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235bf4a0 msgr2=0x560e235bf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:49.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.391+0000 7f2759d9b640 1 --2- 172.21.15.103:0/680526038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235bf4a0 0x560e235bf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.391+0000 7f2759d9b640 1 -- 172.21.15.103:0/680526038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235bfdc0 msgr2=0x560e235ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:49.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.391+0000 7f2759d9b640 1 --2- 172.21.15.103:0/680526038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235bfdc0 0x560e235ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.391+0000 7f2759d9b640 1 -- 172.21.15.103:0/680526038 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e234b1440 con 0x560e235ca680 2024-09-19T06:56:49.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.391+0000 7f275a59c640 1 --2- 172.21.15.103:0/680526038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235bfdc0 0x560e235ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:49.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.391+0000 7f275ad9d640 1 --2- 172.21.15.103:0/2160641100 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f27480a33b0 0x7f27480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:49.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.391+0000 7f27537fe640 1 -- 172.21.15.103:0/2160641100 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f27440cfe90 con 0x7f275c101de0 2024-09-19T06:56:49.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.391+0000 7f2759d9b640 1 --2- 172.21.15.103:0/680526038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235ca680 0x560e235cca70 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f274c00ab70 tx=0x7f274c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dac227829670704 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:49.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.391+0000 7f2750ff9640 1 -- 172.21.15.103:0/680526038 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c01d070 con 0x560e235ca680 2024-09-19T06:56:49.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.391+0000 7f2750ff9640 1 -- 172.21.15.103:0/680526038 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f274c018b20 con 0x560e235ca680 2024-09-19T06:56:49.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.391+0000 7f2750ff9640 1 -- 172.21.15.103:0/680526038 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c010a70 con 0x560e235ca680 2024-09-19T06:56:49.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.392+0000 7f2762f3a740 1 -- 172.21.15.103:0/680526038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235ca680 msgr2=0x560e235cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:49.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.392+0000 7f275ad9d640 1 --2- 172.21.15.103:0/2160641100 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f27480a33b0 0x7f27480a5870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f275402f540 tx=0x7f275403e040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:49.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.392+0000 7f2762f3a740 1 --2- 172.21.15.103:0/680526038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235ca680 0x560e235cca70 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f274c00ab70 tx=0x7f274c00f790 comp rx=0 tx=0).stop 2024-09-19T06:56:49.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.392+0000 7f2762f3a740 1 -- 172.21.15.103:0/680526038 shutdown_connections 2024-09-19T06:56:49.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.392+0000 7f2762f3a740 1 --2- 172.21.15.103:0/680526038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235ca680 0x560e235cca70 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.392+0000 7f2762f3a740 1 --2- 172.21.15.103:0/680526038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235bfdc0 0x560e235ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.392+0000 7f2762f3a740 1 --2- 172.21.15.103:0/680526038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235bf4a0 0x560e235bf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.392+0000 7f2762f3a740 1 -- 172.21.15.103:0/680526038 >> 172.21.15.103:0/680526038 conn(0x560e235bba30 msgr2=0x560e235bbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:49.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.392+0000 7f2762f3a740 1 -- 172.21.15.103:0/680526038 shutdown_connections 2024-09-19T06:56:49.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.392+0000 7f2762f3a740 1 -- 172.21.15.103:0/680526038 wait complete. 2024-09-19T06:56:49.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.392+0000 7f2762f3a740 1 -- start start 2024-09-19T06:56:49.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.392+0000 7f2762f3a740 1 Processor -- start 2024-09-19T06:56:49.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f2762f3a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f2762f3a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235bfdc0 0x560e235c7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f2762f3a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235ca680 0x560e235c7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f2762f3a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560e235cd250 con 0x560e235bf4a0 2024-09-19T06:56:49.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f2762f3a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560e234e0050 con 0x560e235bfdc0 2024-09-19T06:56:49.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f2762f3a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560e235cd3d0 con 0x560e235ca680 2024-09-19T06:56:49.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f2762f3a740 10 client.0 _ll_register_callbacks cb 0x560e23520160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:49.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f275a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:49.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f275a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235c9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59704/0 (socket says 172.21.15.103:59704) 2024-09-19T06:56:49.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f275a59c640 1 -- 172.21.15.103:0/4162432039 learned_addr learned my addr 172.21.15.103:0/4162432039 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:49.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f2759d9b640 1 --2- 172.21.15.103:0/4162432039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235bfdc0 0x560e235c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:49.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f275ad9d640 1 --2- 172.21.15.103:0/4162432039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235ca680 0x560e235c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:49.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f275a59c640 1 -- 172.21.15.103:0/4162432039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235ca680 msgr2=0x560e235c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:49.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f275a59c640 1 --2- 172.21.15.103:0/4162432039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235ca680 0x560e235c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f275a59c640 1 -- 172.21.15.103:0/4162432039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235bfdc0 msgr2=0x560e235c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:49.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f275a59c640 1 --2- 172.21.15.103:0/4162432039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235bfdc0 0x560e235c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f275a59c640 1 -- 172.21.15.103:0/4162432039 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e232963c0 con 0x560e235bf4a0 2024-09-19T06:56:49.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f275a59c640 1 --2- 172.21.15.103:0/4162432039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235c9f50 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f274409be50 tx=0x7f274409bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1a79d336024765 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:49.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f2741ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:49.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f2741ffb640 1 -- 172.21.15.103:0/4162432039 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274409c150 con 0x560e235bf4a0 2024-09-19T06:56:49.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f2741ffb640 1 -- 172.21.15.103:0/4162432039 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f27440987c0 con 0x560e235bf4a0 2024-09-19T06:56:49.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.393+0000 7f2762f3a740 1 -- 172.21.15.103:0/4162432039 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f274c018b20 con 0x560e235bf4a0 2024-09-19T06:56:49.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.394+0000 7f2741ffb640 1 -- 172.21.15.103:0/4162432039 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27440a0500 con 0x560e235bf4a0 2024-09-19T06:56:49.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.394+0000 7f2762f3a740 10 client.18042 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:49.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.394+0000 7f2762f3a740 1 -- 172.21.15.103:0/4162432039 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e235d8c00 con 0x560e235bf4a0 2024-09-19T06:56:49.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.394+0000 7f2741ffb640 1 -- 172.21.15.103:0/4162432039 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f274c018b20 con 0x560e235bf4a0 2024-09-19T06:56:49.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.394+0000 7f2741ffb640 1 -- 172.21.15.103:0/4162432039 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f27440a4930 con 0x560e235bf4a0 2024-09-19T06:56:49.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.394+0000 7f2762f3a740 10 client.18042 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:49.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.394+0000 7f2762f3a740 10 client.18042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:49.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.394+0000 7f2762f3a740 20 client.18042 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:49.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.394+0000 7f2762f3a740 1 --2- 172.21.15.103:0/4162432039 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560e235da010 0x560e235dc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.394+0000 7f2762f3a740 4 client.18042 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:49.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.394+0000 7f2762f3a740 1 -- 172.21.15.103:0/4162432039 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e23532850 con 0x560e235da010 2024-09-19T06:56:49.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.394+0000 7f2759d9b640 1 --2- 172.21.15.103:0/4162432039 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560e235da010 0x560e235dc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:49.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.395+0000 7f2741ffb640 10 client.18042 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:49.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.395+0000 7f2759d9b640 1 --2- 172.21.15.103:0/4162432039 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560e235da010 0x560e235dc440 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5429ad39a7b1dc0c server_cookie=d727a6ec7be4b30c in_seq=0 out_seq=0 2024-09-19T06:56:49.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.395+0000 7f2741ffb640 1 -- 172.21.15.103:0/4162432039 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e23532850 con 0x560e235da010 2024-09-19T06:56:49.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.395+0000 7f2741ffb640 10 client.18042 handle_command_reply: tid=0 2024-09-19T06:56:49.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.395+0000 7f2762f3a740 1 client.18042 shutdown 2024-09-19T06:56:49.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.395+0000 7f2762f3a740 1 -- 172.21.15.103:0/4162432039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 msgr2=0x560e235c9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:49.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.395+0000 7f2762f3a740 1 --2- 172.21.15.103:0/4162432039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235c9f50 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f274409be50 tx=0x7f274409bc90 comp rx=0 tx=0).stop 2024-09-19T06:56:49.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.395+0000 7f2762f3a740 1 -- 172.21.15.103:0/4162432039 shutdown_connections 2024-09-19T06:56:49.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.395+0000 7f2762f3a740 1 --2- 172.21.15.103:0/4162432039 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560e235da010 0x560e235dc440 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.395+0000 7f2762f3a740 1 --2- 172.21.15.103:0/4162432039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235ca680 0x560e235c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.396+0000 7f2762f3a740 1 --2- 172.21.15.103:0/4162432039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235bfdc0 0x560e235c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.396+0000 7f2741ffb640 0 client.18042 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:49.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.396+0000 7f2762f3a740 1 --2- 172.21.15.103:0/4162432039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235c9f50 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.396+0000 7f2762f3a740 1 -- 172.21.15.103:0/4162432039 >> 172.21.15.103:0/4162432039 conn(0x560e235bba30 msgr2=0x560e235c1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:49.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.396+0000 7f2762f3a740 1 -- 172.21.15.103:0/4162432039 shutdown_connections 2024-09-19T06:56:49.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.396+0000 7f2762f3a740 1 -- 172.21.15.103:0/4162432039 wait complete. 2024-09-19T06:56:49.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.396+0000 7f2762f3a740 20 client.18042 trim_cache size 0 max 16384 2024-09-19T06:56:49.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.412+0000 7f2762f3a740 1 Processor -- start 2024-09-19T06:56:49.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.412+0000 7f2762f3a740 1 -- start start 2024-09-19T06:56:49.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f2762f3a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235d64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f2762f3a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235bfdc0 0x560e235e1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f2762f3a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235ca680 0x560e235e2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f2762f3a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1ad820 con 0x560e235bf4a0 2024-09-19T06:56:49.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f2762f3a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1af360 con 0x560e235ca680 2024-09-19T06:56:49.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f2762f3a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1adef0 con 0x560e235bfdc0 2024-09-19T06:56:49.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f275a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235d64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:49.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f275a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235d64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59714/0 (socket says 172.21.15.103:59714) 2024-09-19T06:56:49.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f275a59c640 1 -- 172.21.15.103:0/1929903901 learned_addr learned my addr 172.21.15.103:0/1929903901 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:49.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f275a59c640 1 -- 172.21.15.103:0/1929903901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235bfdc0 msgr2=0x560e235e1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:49.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f275a59c640 1 --2- 172.21.15.103:0/1929903901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235bfdc0 0x560e235e1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f275a59c640 1 -- 172.21.15.103:0/1929903901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235ca680 msgr2=0x560e235e2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:49.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f275a59c640 1 --2- 172.21.15.103:0/1929903901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235ca680 0x560e235e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f275a59c640 1 -- 172.21.15.103:0/1929903901 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e235bfbf0 con 0x560e235bf4a0 2024-09-19T06:56:49.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f275a59c640 1 --2- 172.21.15.103:0/1929903901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235d64b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f274409b5c0 tx=0x7f27440d0220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f354032657807d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:49.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f2743fff640 1 -- 172.21.15.103:0/1929903901 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274409c150 con 0x560e235bf4a0 2024-09-19T06:56:49.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f2743fff640 1 -- 172.21.15.103:0/1929903901 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27440a4dc0 con 0x560e235bf4a0 2024-09-19T06:56:49.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.413+0000 7f2743fff640 1 -- 172.21.15.103:0/1929903901 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27440bc0b0 con 0x560e235bf4a0 2024-09-19T06:56:49.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.414+0000 7f2762f3a740 1 -- 172.21.15.103:0/1929903901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 msgr2=0x560e235d64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:49.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.414+0000 7f2762f3a740 1 --2- 172.21.15.103:0/1929903901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235d64b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f274409b5c0 tx=0x7f27440d0220 comp rx=0 tx=0).stop 2024-09-19T06:56:49.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.414+0000 7f2762f3a740 1 -- 172.21.15.103:0/1929903901 shutdown_connections 2024-09-19T06:56:49.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.414+0000 7f2762f3a740 1 --2- 172.21.15.103:0/1929903901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235ca680 0x560e235e2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.414+0000 7f2762f3a740 1 --2- 172.21.15.103:0/1929903901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235bfdc0 0x560e235e1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.414+0000 7f2762f3a740 1 --2- 172.21.15.103:0/1929903901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235d64b0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.414+0000 7f2762f3a740 1 -- 172.21.15.103:0/1929903901 >> 172.21.15.103:0/1929903901 conn(0x560e235bba30 msgr2=0x560e235c5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:49.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.414+0000 7f2762f3a740 1 -- 172.21.15.103:0/1929903901 shutdown_connections 2024-09-19T06:56:49.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.414+0000 7f2762f3a740 1 -- 172.21.15.103:0/1929903901 wait complete. 2024-09-19T06:56:49.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.414+0000 7f2762f3a740 1 -- start start 2024-09-19T06:56:49.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.414+0000 7f2762f3a740 1 Processor -- start 2024-09-19T06:56:49.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f2762f3a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f2762f3a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235bfdc0 0x560e235cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f2762f3a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235ca680 0x560e235cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f2762f3a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1aec90 con 0x560e235bf4a0 2024-09-19T06:56:49.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f2762f3a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1ae5c0 con 0x560e235bfdc0 2024-09-19T06:56:49.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f2762f3a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1ab820 con 0x560e235ca680 2024-09-19T06:56:49.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f2762f3a740 10 client.0 _ll_register_callbacks cb 0x7f274400c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:49.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f275a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:49.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f275a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59724/0 (socket says 172.21.15.103:59724) 2024-09-19T06:56:49.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f275a59c640 1 -- 172.21.15.103:0/3551380784 learned_addr learned my addr 172.21.15.103:0/3551380784 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:49.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f275ad9d640 1 --2- 172.21.15.103:0/3551380784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235ca680 0x560e235cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:49.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f275a59c640 1 -- 172.21.15.103:0/3551380784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235ca680 msgr2=0x560e235cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:49.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f275a59c640 1 --2- 172.21.15.103:0/3551380784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235ca680 0x560e235cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f275a59c640 1 -- 172.21.15.103:0/3551380784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235bfdc0 msgr2=0x560e235cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:49.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f275a59c640 1 --2- 172.21.15.103:0/3551380784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235bfdc0 0x560e235cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f275a59c640 1 -- 172.21.15.103:0/3551380784 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e23648390 con 0x560e235bf4a0 2024-09-19T06:56:49.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f275ad9d640 1 --2- 172.21.15.103:0/3551380784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235ca680 0x560e235cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:49.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f275a59c640 1 --2- 172.21.15.103:0/3551380784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235ce6a0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f27440cdbc0 tx=0x7f2744000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8eca0cf0ab333b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:49.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f2741ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:49.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f2741ffb640 1 -- 172.21.15.103:0/3551380784 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744066070 con 0x560e235bf4a0 2024-09-19T06:56:49.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f2741ffb640 1 -- 172.21.15.103:0/3551380784 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f27440987c0 con 0x560e235bf4a0 2024-09-19T06:56:49.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.415+0000 7f2762f3a740 1 -- 172.21.15.103:0/3551380784 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27440a4dc0 con 0x560e235bf4a0 2024-09-19T06:56:49.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.416+0000 7f2741ffb640 1 -- 172.21.15.103:0/3551380784 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f274409c150 con 0x560e235bf4a0 2024-09-19T06:56:49.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.416+0000 7f2762f3a740 10 client.18048 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:49.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.416+0000 7f2762f3a740 1 -- 172.21.15.103:0/3551380784 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e236496a0 con 0x560e235bf4a0 2024-09-19T06:56:49.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.416+0000 7f2741ffb640 1 -- 172.21.15.103:0/3551380784 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27440a4dc0 con 0x560e235bf4a0 2024-09-19T06:56:49.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.416+0000 7f2741ffb640 1 -- 172.21.15.103:0/3551380784 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f274409b010 con 0x560e235bf4a0 2024-09-19T06:56:49.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.416+0000 7f2762f3a740 10 client.18048 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:49.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.416+0000 7f2762f3a740 10 client.18048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:49.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.416+0000 7f2762f3a740 20 client.18048 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:49.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.416+0000 7f2762f3a740 1 --2- 172.21.15.103:0/3551380784 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560e236499e0 0x560e2364bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:49.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.416+0000 7f2762f3a740 4 client.18048 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:56:49.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.416+0000 7f2762f3a740 1 -- 172.21.15.103:0/3551380784 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e2352cd60 con 0x560e236499e0 2024-09-19T06:56:49.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.416+0000 7f2759d9b640 1 --2- 172.21.15.103:0/3551380784 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560e236499e0 0x560e2364bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:49.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2759d9b640 1 --2- 172.21.15.103:0/3551380784 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560e236499e0 0x560e2364bdd0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab39d55716e5ecf9 server_cookie=dfbbf25dd9602fb in_seq=0 out_seq=0 2024-09-19T06:56:49.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2741ffb640 10 client.18048 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:49.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2741ffb640 1 -- 172.21.15.103:0/3551380784 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560e2352cd60 con 0x560e236499e0 2024-09-19T06:56:49.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2741ffb640 10 client.18048 handle_command_reply: tid=0 2024-09-19T06:56:49.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2762f3a740 1 client.18048 shutdown 2024-09-19T06:56:49.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2762f3a740 1 -- 172.21.15.103:0/3551380784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 msgr2=0x560e235ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:49.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2762f3a740 1 --2- 172.21.15.103:0/3551380784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235ce6a0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f27440cdbc0 tx=0x7f2744000bc0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2762f3a740 1 -- 172.21.15.103:0/3551380784 shutdown_connections 2024-09-19T06:56:49.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2762f3a740 1 --2- 172.21.15.103:0/3551380784 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560e236499e0 0x560e2364bdd0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2762f3a740 1 --2- 172.21.15.103:0/3551380784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e235ca680 0x560e235cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2762f3a740 1 --2- 172.21.15.103:0/3551380784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e235bfdc0 0x560e235cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2741ffb640 0 client.18048 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:49.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2762f3a740 1 --2- 172.21.15.103:0/3551380784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e235bf4a0 0x560e235ce6a0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2762f3a740 1 -- 172.21.15.103:0/3551380784 >> 172.21.15.103:0/3551380784 conn(0x560e235bba30 msgr2=0x560e235bab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:49.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2762f3a740 1 -- 172.21.15.103:0/3551380784 shutdown_connections 2024-09-19T06:56:49.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2762f3a740 1 -- 172.21.15.103:0/3551380784 wait complete. 2024-09-19T06:56:49.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.417+0000 7f2762f3a740 20 client.18048 trim_cache size 0 max 16384 2024-09-19T06:56:49.434 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:56:49.434 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T06:56:49.434 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T06:56:49.434 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:56:49.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.418+0000 7f2760b61640 1 -- 172.21.15.103:0/2160641100 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f27480a33b0 msgr2=0x7f27480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:49.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.418+0000 7f2760b61640 1 --2- 172.21.15.103:0/2160641100 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f27480a33b0 0x7f27480a5870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f275402f540 tx=0x7f275403e040 comp rx=0 tx=0).stop 2024-09-19T06:56:49.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.418+0000 7f2760b61640 1 -- 172.21.15.103:0/2160641100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f275c101de0 msgr2=0x7f275c198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:49.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.418+0000 7f2760b61640 1 --2- 172.21.15.103:0/2160641100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f275c101de0 0x7f275c198330 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f274400c920 tx=0x7f274400cde0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.418+0000 7f2760b61640 1 -- 172.21.15.103:0/2160641100 shutdown_connections 2024-09-19T06:56:49.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.418+0000 7f2760b61640 1 --2- 172.21.15.103:0/2160641100 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f27480a33b0 0x7f27480a5870 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.418+0000 7f2760b61640 1 --2- 172.21.15.103:0/2160641100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f275c106f10 0x7f275c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.418+0000 7f2760b61640 1 --2- 172.21.15.103:0/2160641100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f275c102790 0x7f275c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.418+0000 7f2760b61640 1 --2- 172.21.15.103:0/2160641100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f275c101de0 0x7f275c198330 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:49.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.418+0000 7f2760b61640 1 -- 172.21.15.103:0/2160641100 >> 172.21.15.103:0/2160641100 conn(0x7f275c0fd8c0 msgr2=0x7f275c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:49.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.418+0000 7f2760b61640 1 -- 172.21.15.103:0/2160641100 shutdown_connections 2024-09-19T06:56:49.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:49.418+0000 7f2760b61640 1 -- 172.21.15.103:0/2160641100 wait complete. 2024-09-19T06:56:50.108 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:56:50.109 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:56:50.360 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:50.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:50 smithi103 ceph-mon[31158]: pgmap v233: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 16 op/s 2024-09-19T06:56:50.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:50 smithi134 ceph-mon[35356]: pgmap v233: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 16 op/s 2024-09-19T06:56:50.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:50 smithi174 ceph-mon[35637]: pgmap v233: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 16 op/s 2024-09-19T06:56:50.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.762+0000 7fcabce82640 1 -- 172.21.15.103:0/3723766688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8102790 msgr2=0x7fcab81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:50.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.762+0000 7fcabce82640 1 --2- 172.21.15.103:0/3723766688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8102790 0x7fcab81067e0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009900 tx=0x7fcaa802a410 comp rx=0 tx=0).stop 2024-09-19T06:56:50.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.762+0000 7fcabce82640 1 -- 172.21.15.103:0/3723766688 shutdown_connections 2024-09-19T06:56:50.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.762+0000 7fcabce82640 1 --2- 172.21.15.103:0/3723766688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab8106f10 0x7fcab810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:50.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.762+0000 7fcabce82640 1 --2- 172.21.15.103:0/3723766688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8102790 0x7fcab81067e0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:50.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.762+0000 7fcabce82640 1 --2- 172.21.15.103:0/3723766688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcab8101de0 0x7fcab81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:50.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.762+0000 7fcabce82640 1 -- 172.21.15.103:0/3723766688 >> 172.21.15.103:0/3723766688 conn(0x7fcab80fd8c0 msgr2=0x7fcab80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:50.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.762+0000 7fcabce82640 1 -- 172.21.15.103:0/3723766688 shutdown_connections 2024-09-19T06:56:50.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.762+0000 7fcabce82640 1 -- 172.21.15.103:0/3723766688 wait complete. 2024-09-19T06:56:50.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.763+0000 7fcabce82640 1 Processor -- start 2024-09-19T06:56:50.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.763+0000 7fcabce82640 1 -- start start 2024-09-19T06:56:50.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.763+0000 7fcabce82640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8101de0 0x7fcab8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:50.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.763+0000 7fcabce82640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab8102790 0x7fcab8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:50.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.763+0000 7fcabce82640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcab8106f10 0x7fcab819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:50.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.763+0000 7fcabce82640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcab8111f90 con 0x7fcab8101de0 2024-09-19T06:56:50.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.763+0000 7fcabce82640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcab8111e10 con 0x7fcab8102790 2024-09-19T06:56:50.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.763+0000 7fcabce82640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcab8112110 con 0x7fcab8106f10 2024-09-19T06:56:50.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.763+0000 7fcab659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8101de0 0x7fcab8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:50.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.763+0000 7fcab659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8101de0 0x7fcab8198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59748/0 (socket says 172.21.15.103:59748) 2024-09-19T06:56:50.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.763+0000 7fcab659c640 1 -- 172.21.15.103:0/3455187969 learned_addr learned my addr 172.21.15.103:0/3455187969 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:50.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.764+0000 7fcab5d9b640 1 --2- 172.21.15.103:0/3455187969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab8102790 0x7fcab8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:50.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.764+0000 7fcab6d9d640 1 --2- 172.21.15.103:0/3455187969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcab8106f10 0x7fcab819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:50.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.764+0000 7fcab659c640 1 -- 172.21.15.103:0/3455187969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcab8106f10 msgr2=0x7fcab819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:50.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.764+0000 7fcab659c640 1 --2- 172.21.15.103:0/3455187969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcab8106f10 0x7fcab819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:50.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.764+0000 7fcab659c640 1 -- 172.21.15.103:0/3455187969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab8102790 msgr2=0x7fcab8198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:50.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.764+0000 7fcab659c640 1 --2- 172.21.15.103:0/3455187969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab8102790 0x7fcab8198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:50.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.764+0000 7fcab659c640 1 -- 172.21.15.103:0/3455187969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab819d3d0 con 0x7fcab8101de0 2024-09-19T06:56:50.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.764+0000 7fcab6d9d640 1 --2- 172.21.15.103:0/3455187969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcab8106f10 0x7fcab819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:50.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.764+0000 7fcab659c640 1 --2- 172.21.15.103:0/3455187969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8101de0 0x7fcab8198420 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00c920 tx=0x7fcaac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:50.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.764+0000 7fcaa77fe640 1 -- 172.21.15.103:0/3455187969 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac007c70 con 0x7fcab8101de0 2024-09-19T06:56:50.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.764+0000 7fcaa77fe640 1 -- 172.21.15.103:0/3455187969 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaac007e10 con 0x7fcab8101de0 2024-09-19T06:56:50.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.764+0000 7fcabce82640 1 -- 172.21.15.103:0/3455187969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab819d6c0 con 0x7fcab8101de0 2024-09-19T06:56:50.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.764+0000 7fcaa77fe640 1 -- 172.21.15.103:0/3455187969 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac018890 con 0x7fcab8101de0 2024-09-19T06:56:50.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.764+0000 7fcabce82640 1 -- 172.21.15.103:0/3455187969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab81a4fa0 con 0x7fcab8101de0 2024-09-19T06:56:50.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.765+0000 7fcabce82640 1 -- 172.21.15.103:0/3455187969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab81021c0 con 0x7fcab8101de0 2024-09-19T06:56:50.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.766+0000 7fcaa77fe640 1 -- 172.21.15.103:0/3455187969 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fcaac018a50 con 0x7fcab8101de0 2024-09-19T06:56:50.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.767+0000 7fcaa77fe640 1 --2- 172.21.15.103:0/3455187969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcaa00a33b0 0x7fcaa00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:50.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.767+0000 7fcab5d9b640 1 --2- 172.21.15.103:0/3455187969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcaa00a33b0 0x7fcaa00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:50.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.767+0000 7fcaa77fe640 1 -- 172.21.15.103:0/3455187969 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fcaac0d0120 con 0x7fcab8101de0 2024-09-19T06:56:50.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.768+0000 7fcab5d9b640 1 --2- 172.21.15.103:0/3455187969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcaa00a33b0 0x7fcaa00a5870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8000c00 tx=0x7fcaa80058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:50.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.770+0000 7fcaa77fe640 1 -- 172.21.15.103:0/3455187969 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcaac014030 con 0x7fcab8101de0 2024-09-19T06:56:50.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.975+0000 7fcabce82640 1 -- 172.21.15.103:0/3455187969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcab8199780 con 0x7fcab8101de0 2024-09-19T06:56:50.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.975+0000 7fcaa77fe640 1 -- 172.21.15.103:0/3455187969 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fcab8199780 con 0x7fcab8101de0 2024-09-19T06:56:50.976 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:56:50.977 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:56:50.977 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:56:50.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.978+0000 7fcabce82640 1 -- 172.21.15.103:0/3455187969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcaa00a33b0 msgr2=0x7fcaa00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:50.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.978+0000 7fcabce82640 1 --2- 172.21.15.103:0/3455187969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcaa00a33b0 0x7fcaa00a5870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8000c00 tx=0x7fcaa80058d0 comp rx=0 tx=0).stop 2024-09-19T06:56:50.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.978+0000 7fcabce82640 1 -- 172.21.15.103:0/3455187969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8101de0 msgr2=0x7fcab8198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:50.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.978+0000 7fcabce82640 1 --2- 172.21.15.103:0/3455187969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8101de0 0x7fcab8198420 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00c920 tx=0x7fcaac00cde0 comp rx=0 tx=0).stop 2024-09-19T06:56:50.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.978+0000 7fcabce82640 1 -- 172.21.15.103:0/3455187969 shutdown_connections 2024-09-19T06:56:50.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.978+0000 7fcabce82640 1 --2- 172.21.15.103:0/3455187969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcaa00a33b0 0x7fcaa00a5870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:50.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.978+0000 7fcabce82640 1 --2- 172.21.15.103:0/3455187969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcab8106f10 0x7fcab819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:50.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.978+0000 7fcabce82640 1 --2- 172.21.15.103:0/3455187969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab8102790 0x7fcab8198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:50.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.978+0000 7fcabce82640 1 --2- 172.21.15.103:0/3455187969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8101de0 0x7fcab8198420 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:50.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.978+0000 7fcabce82640 1 -- 172.21.15.103:0/3455187969 >> 172.21.15.103:0/3455187969 conn(0x7fcab80fd8c0 msgr2=0x7fcab80fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:50.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.978+0000 7fcabce82640 1 -- 172.21.15.103:0/3455187969 shutdown_connections 2024-09-19T06:56:50.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:50.978+0000 7fcabce82640 1 -- 172.21.15.103:0/3455187969 wait complete. 2024-09-19T06:56:51.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:51 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3455187969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:51.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:51 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3455187969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:51.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:51 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3455187969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:51.588 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:56:51.835 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.011+0000 7fdccc981640 1 -- 172.21.15.103:0/1559980737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcc8100830 msgr2=0x7fdcc8100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.011+0000 7fdccc981640 1 --2- 172.21.15.103:0/1559980737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcc8100830 0x7fdcc8100c10 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009900 tx=0x7fdcb802a410 comp rx=0 tx=0).stop 2024-09-19T06:56:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.012+0000 7fdccc981640 1 -- 172.21.15.103:0/1559980737 shutdown_connections 2024-09-19T06:56:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.012+0000 7fdccc981640 1 --2- 172.21.15.103:0/1559980737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcc8105520 0x7fdcc810d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.012+0000 7fdccc981640 1 --2- 172.21.15.103:0/1559980737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcc81011e0 0x7fdcc8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.012+0000 7fdccc981640 1 --2- 172.21.15.103:0/1559980737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcc8100830 0x7fdcc8100c10 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.012+0000 7fdccc981640 1 -- 172.21.15.103:0/1559980737 >> 172.21.15.103:0/1559980737 conn(0x7fdcc8075f20 msgr2=0x7fdcc8076330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.012+0000 7fdccc981640 1 -- 172.21.15.103:0/1559980737 shutdown_connections 2024-09-19T06:56:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.012+0000 7fdccc981640 1 -- 172.21.15.103:0/1559980737 wait complete. 2024-09-19T06:56:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.012+0000 7fdccc981640 1 Processor -- start 2024-09-19T06:56:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.012+0000 7fdccc981640 1 -- start start 2024-09-19T06:56:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdccc981640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcc8100830 0x7fdcc81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdccc981640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcc81011e0 0x7fdcc81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdccc981640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcc8105520 0x7fdcc819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdccc981640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc8110e10 con 0x7fdcc8105520 2024-09-19T06:56:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdccc981640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc8110c90 con 0x7fdcc81011e0 2024-09-19T06:56:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdccc981640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc8110f90 con 0x7fdcc8100830 2024-09-19T06:56:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdcc6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcc8105520 0x7fdcc819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdcc6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcc8105520 0x7fdcc819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59770/0 (socket says 172.21.15.103:59770) 2024-09-19T06:56:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdcc6d9d640 1 -- 172.21.15.103:0/1452087973 learned_addr learned my addr 172.21.15.103:0/1452087973 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdcc659c640 1 --2- 172.21.15.103:0/1452087973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcc8100830 0x7fdcc81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdcc6d9d640 1 -- 172.21.15.103:0/1452087973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcc8100830 msgr2=0x7fdcc81983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdcc5d9b640 1 --2- 172.21.15.103:0/1452087973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcc81011e0 0x7fdcc81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdcc6d9d640 1 --2- 172.21.15.103:0/1452087973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcc8100830 0x7fdcc81983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdcc6d9d640 1 -- 172.21.15.103:0/1452087973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcc81011e0 msgr2=0x7fdcc81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdcc6d9d640 1 --2- 172.21.15.103:0/1452087973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcc81011e0 0x7fdcc81988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdcc6d9d640 1 -- 172.21.15.103:0/1452087973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc819d3a0 con 0x7fdcc8105520 2024-09-19T06:56:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdcc659c640 1 --2- 172.21.15.103:0/1452087973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcc8100830 0x7fdcc81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdcc6d9d640 1 --2- 172.21.15.103:0/1452087973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcc8105520 0x7fdcc819cc70 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009fd0 tx=0x7fdcc000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdcb77fe640 1 -- 172.21.15.103:0/1452087973 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0017070 con 0x7fdcc8105520 2024-09-19T06:56:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdccc981640 1 -- 172.21.15.103:0/1452087973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc819d690 con 0x7fdcc8105520 2024-09-19T06:56:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdcb77fe640 1 -- 172.21.15.103:0/1452087973 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc0003e50 con 0x7fdcc8105520 2024-09-19T06:56:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdcb77fe640 1 -- 172.21.15.103:0/1452087973 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc00106c0 con 0x7fdcc8105520 2024-09-19T06:56:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.013+0000 7fdccc981640 1 -- 172.21.15.103:0/1452087973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc81a4f20 con 0x7fdcc8105520 2024-09-19T06:56:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.016+0000 7fdcb77fe640 1 -- 172.21.15.103:0/1452087973 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fdcc0003970 con 0x7fdcc8105520 2024-09-19T06:56:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.016+0000 7fdccf7f3740 1 Processor -- start 2024-09-19T06:56:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.016+0000 7fdccf7f3740 1 -- start start 2024-09-19T06:56:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.016+0000 7fdccf7f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93c9b0 0x55b2be93cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.016+0000 7fdccf7f3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93d2d0 0x55b2be94bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.016+0000 7fdccf7f3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be94e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.016+0000 7fdccf7f3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b2be8a12f0 con 0x55b2be94c0c0 2024-09-19T06:56:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.016+0000 7fdccf7f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b2be849270 con 0x55b2be93c9b0 2024-09-19T06:56:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.016+0000 7fdccf7f3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b2be89d2b0 con 0x55b2be93d2d0 2024-09-19T06:56:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcb77fe640 1 --2- 172.21.15.103:0/1452087973 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdcb00a3460 0x7fdcb00a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcc5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be94e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcc5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be94e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59774/0 (socket says 172.21.15.103:59774) 2024-09-19T06:56:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcc5d9b640 1 -- 172.21.15.103:0/4189829484 learned_addr learned my addr 172.21.15.103:0/4189829484 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcc659c640 1 --2- 172.21.15.103:0/4189829484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93d2d0 0x55b2be94bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcc5d9b640 1 --2- 172.21.15.103:0/4189829484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93c9b0 0x55b2be93cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcc6d9d640 1 --2- 172.21.15.103:0/1452087973 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdcb00a3460 0x7fdcb00a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcb77fe640 1 -- 172.21.15.103:0/1452087973 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fdcc00cfb20 con 0x7fdcc8105520 2024-09-19T06:56:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcc5d9b640 1 -- 172.21.15.103:0/4189829484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93d2d0 msgr2=0x55b2be94bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcc5d9b640 1 --2- 172.21.15.103:0/4189829484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93d2d0 0x55b2be94bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcc5d9b640 1 -- 172.21.15.103:0/4189829484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93c9b0 msgr2=0x55b2be93cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcc5d9b640 1 --2- 172.21.15.103:0/4189829484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93c9b0 0x55b2be93cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcc5d9b640 1 -- 172.21.15.103:0/4189829484 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2be848340 con 0x55b2be94c0c0 2024-09-19T06:56:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcc659c640 1 --2- 172.21.15.103:0/4189829484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93d2d0 0x55b2be94bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:56:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcc5d9b640 1 --2- 172.21.15.103:0/4189829484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be94e4b0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc013800 tx=0x7fdcbc00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d6fdd21e082dc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcc6d9d640 1 --2- 172.21.15.103:0/1452087973 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdcb00a3460 0x7fdcb00a5920 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdcc009bd70 tx=0x7fdcc006ec10 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcb4ff9640 1 -- 172.21.15.103:0/4189829484 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc00b860 con 0x55b2be94c0c0 2024-09-19T06:56:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcb4ff9640 1 -- 172.21.15.103:0/4189829484 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcbc013e40 con 0x55b2be94c0c0 2024-09-19T06:56:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.017+0000 7fdcb4ff9640 1 -- 172.21.15.103:0/4189829484 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc01dd90 con 0x55b2be94c0c0 2024-09-19T06:56:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.018+0000 7fdccf7f3740 1 -- 172.21.15.103:0/4189829484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 msgr2=0x55b2be94e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.018+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/4189829484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be94e4b0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc013800 tx=0x7fdcbc00ef90 comp rx=0 tx=0).stop 2024-09-19T06:56:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.018+0000 7fdccf7f3740 1 -- 172.21.15.103:0/4189829484 shutdown_connections 2024-09-19T06:56:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.018+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/4189829484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be94e4b0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.018+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/4189829484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93d2d0 0x55b2be94bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.018+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/4189829484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93c9b0 0x55b2be93cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.018+0000 7fdccf7f3740 1 -- 172.21.15.103:0/4189829484 >> 172.21.15.103:0/4189829484 conn(0x55b2be939060 msgr2=0x55b2be939440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.018+0000 7fdccf7f3740 1 -- 172.21.15.103:0/4189829484 shutdown_connections 2024-09-19T06:56:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.018+0000 7fdccf7f3740 1 -- 172.21.15.103:0/4189829484 wait complete. 2024-09-19T06:56:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.018+0000 7fdccf7f3740 1 -- start start 2024-09-19T06:56:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdccf7f3740 1 Processor -- start 2024-09-19T06:56:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdccf7f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93c9b0 0x55b2be9490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdccf7f3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93d2d0 0x55b2be9495e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdccf7f3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be949b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdccf7f3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b2be94ec90 con 0x55b2be94c0c0 2024-09-19T06:56:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdccf7f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b2be82a290 con 0x55b2be93c9b0 2024-09-19T06:56:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdccf7f3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b2be94ee10 con 0x55b2be93d2d0 2024-09-19T06:56:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdccf7f3740 10 client.0 _ll_register_callbacks cb 0x55b2be85b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdcc5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be949b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdcc5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be949b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59778/0 (socket says 172.21.15.103:59778) 2024-09-19T06:56:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdcc5d9b640 1 -- 172.21.15.103:0/4055332324 learned_addr learned my addr 172.21.15.103:0/4055332324 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdcc659c640 1 --2- 172.21.15.103:0/4055332324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93d2d0 0x55b2be9495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdcc5d9b640 1 -- 172.21.15.103:0/4055332324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93d2d0 msgr2=0x55b2be9495e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdcc5d9b640 1 --2- 172.21.15.103:0/4055332324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93d2d0 0x55b2be9495e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdcc5d9b640 1 -- 172.21.15.103:0/4055332324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93c9b0 msgr2=0x55b2be9490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdcc5d9b640 1 --2- 172.21.15.103:0/4055332324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93c9b0 0x55b2be9490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdcc5d9b640 1 -- 172.21.15.103:0/4055332324 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2be5b4210 con 0x55b2be94c0c0 2024-09-19T06:56:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdcc659c640 1 --2- 172.21.15.103:0/4055332324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93d2d0 0x55b2be9495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdcc5d9b640 1 --2- 172.21.15.103:0/4055332324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be949b20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005fd0 tx=0x7fdcbc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0ace14ae95c471 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdcadffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.019+0000 7fdcadffb640 1 -- 172.21.15.103:0/4055332324 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc013070 con 0x55b2be94c0c0 2024-09-19T06:56:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.020+0000 7fdccf7f3740 1 -- 172.21.15.103:0/4055332324 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b2be8aea20 con 0x55b2be94c0c0 2024-09-19T06:56:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.020+0000 7fdcadffb640 1 -- 172.21.15.103:0/4055332324 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fdcbc013210 con 0x55b2be94c0c0 2024-09-19T06:56:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.020+0000 7fdcadffb640 1 -- 172.21.15.103:0/4055332324 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdcbc013410 con 0x55b2be94c0c0 2024-09-19T06:56:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.020+0000 7fdccf7f3740 10 client.18066 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.020+0000 7fdccf7f3740 1 -- 172.21.15.103:0/4055332324 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b2be680620 con 0x55b2be94c0c0 2024-09-19T06:56:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.020+0000 7fdcadffb640 1 -- 172.21.15.103:0/4055332324 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b2be8aea20 con 0x55b2be94c0c0 2024-09-19T06:56:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.020+0000 7fdcadffb640 1 -- 172.21.15.103:0/4055332324 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdcbc00f040 con 0x55b2be94c0c0 2024-09-19T06:56:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.020+0000 7fdccf7f3740 10 client.18066 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.020+0000 7fdccf7f3740 10 client.18066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.020+0000 7fdccf7f3740 20 client.18066 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.020+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/4055332324 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b2be95bff0 0x55b2be95e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.020+0000 7fdccf7f3740 4 client.18066 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.020+0000 7fdccf7f3740 1 -- 172.21.15.103:0/4055332324 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2be8a8d70 con 0x55b2be95bff0 2024-09-19T06:56:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.020+0000 7fdcc659c640 1 --2- 172.21.15.103:0/4055332324 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b2be95bff0 0x55b2be95e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.021+0000 7fdcadffb640 10 client.18066 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.021+0000 7fdcc659c640 1 --2- 172.21.15.103:0/4055332324 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b2be95bff0 0x55b2be95e3e0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14bad55e19311e16 server_cookie=dde945c60425f159 in_seq=0 out_seq=0 2024-09-19T06:56:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.021+0000 7fdcadffb640 1 -- 172.21.15.103:0/4055332324 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2be8a8d70 con 0x55b2be95bff0 2024-09-19T06:56:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.021+0000 7fdcadffb640 10 client.18066 handle_command_reply: tid=0 2024-09-19T06:56:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.022+0000 7fdccf7f3740 1 client.18066 shutdown 2024-09-19T06:56:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.022+0000 7fdccf7f3740 1 -- 172.21.15.103:0/4055332324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 msgr2=0x55b2be949b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.022+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/4055332324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be949b20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005fd0 tx=0x7fdcbc005850 comp rx=0 tx=0).stop 2024-09-19T06:56:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.022+0000 7fdccf7f3740 1 -- 172.21.15.103:0/4055332324 shutdown_connections 2024-09-19T06:56:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.022+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/4055332324 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b2be95bff0 0x55b2be95e3e0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.022+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/4055332324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be949b20 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.022+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/4055332324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93d2d0 0x55b2be9495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.022+0000 7fdcadffb640 0 client.18066 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.022+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/4055332324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93c9b0 0x55b2be9490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.022+0000 7fdccf7f3740 1 -- 172.21.15.103:0/4055332324 >> 172.21.15.103:0/4055332324 conn(0x55b2be939060 msgr2=0x55b2be93edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.022+0000 7fdccf7f3740 1 -- 172.21.15.103:0/4055332324 shutdown_connections 2024-09-19T06:56:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.022+0000 7fdccf7f3740 1 -- 172.21.15.103:0/4055332324 wait complete. 2024-09-19T06:56:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.022+0000 7fdccf7f3740 20 client.18066 trim_cache size 0 max 16384 2024-09-19T06:56:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.036+0000 7fdccf7f3740 1 Processor -- start 2024-09-19T06:56:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.036+0000 7fdccf7f3740 1 -- start start 2024-09-19T06:56:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.036+0000 7fdccf7f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93c9b0 0x55b2be94b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.036+0000 7fdccf7f3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93d2d0 0x55b2be9462e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.036+0000 7fdccf7f3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be946820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.036+0000 7fdccf7f3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81ab690 con 0x55b2be94c0c0 2024-09-19T06:56:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.036+0000 7fdccf7f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81ad1d0 con 0x55b2be93c9b0 2024-09-19T06:56:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.036+0000 7fdccf7f3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81abd60 con 0x55b2be93d2d0 2024-09-19T06:56:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.036+0000 7fdcc5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be946820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.036+0000 7fdcc5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be946820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59782/0 (socket says 172.21.15.103:59782) 2024-09-19T06:56:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.036+0000 7fdcc5d9b640 1 -- 172.21.15.103:0/2709118465 learned_addr learned my addr 172.21.15.103:0/2709118465 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.036+0000 7fdcc5d9b640 1 -- 172.21.15.103:0/2709118465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93d2d0 msgr2=0x55b2be9462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.036+0000 7fdcc5d9b640 1 --2- 172.21.15.103:0/2709118465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93d2d0 0x55b2be9462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.036+0000 7fdcc5d9b640 1 -- 172.21.15.103:0/2709118465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93c9b0 msgr2=0x55b2be94b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.036+0000 7fdcc5d9b640 1 --2- 172.21.15.103:0/2709118465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93c9b0 0x55b2be94b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.036+0000 7fdcc5d9b640 1 -- 172.21.15.103:0/2709118465 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2be93d100 con 0x55b2be94c0c0 2024-09-19T06:56:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.037+0000 7fdcc5d9b640 1 --2- 172.21.15.103:0/2709118465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be946820 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00eff0 tx=0x7fdcbc00e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb2a2c92af2f878 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.037+0000 7fdcaffff640 1 -- 172.21.15.103:0/2709118465 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc033070 con 0x55b2be94c0c0 2024-09-19T06:56:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.037+0000 7fdcaffff640 1 -- 172.21.15.103:0/2709118465 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcbc022030 con 0x55b2be94c0c0 2024-09-19T06:56:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.037+0000 7fdcaffff640 1 -- 172.21.15.103:0/2709118465 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc00bc90 con 0x55b2be94c0c0 2024-09-19T06:56:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.037+0000 7fdccf7f3740 1 -- 172.21.15.103:0/2709118465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 msgr2=0x55b2be946820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.037+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/2709118465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be946820 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00eff0 tx=0x7fdcbc00e910 comp rx=0 tx=0).stop 2024-09-19T06:56:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.037+0000 7fdccf7f3740 1 -- 172.21.15.103:0/2709118465 shutdown_connections 2024-09-19T06:56:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.037+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/2709118465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be946820 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.037+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/2709118465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93d2d0 0x55b2be9462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.037+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/2709118465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93c9b0 0x55b2be94b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.037+0000 7fdccf7f3740 1 -- 172.21.15.103:0/2709118465 >> 172.21.15.103:0/2709118465 conn(0x55b2be939060 msgr2=0x55b2be94a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.037+0000 7fdccf7f3740 1 -- 172.21.15.103:0/2709118465 shutdown_connections 2024-09-19T06:56:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.037+0000 7fdccf7f3740 1 -- 172.21.15.103:0/2709118465 wait complete. 2024-09-19T06:56:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdccf7f3740 1 -- start start 2024-09-19T06:56:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdccf7f3740 1 Processor -- start 2024-09-19T06:56:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdccf7f3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93c9b0 0x55b2be95f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdccf7f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93d2d0 0x55b2be95f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdccf7f3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be95fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdccf7f3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81acb00 con 0x55b2be94c0c0 2024-09-19T06:56:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdccf7f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81ac430 con 0x55b2be93d2d0 2024-09-19T06:56:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdccf7f3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81a9690 con 0x55b2be93c9b0 2024-09-19T06:56:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdccf7f3740 10 client.0 _ll_register_callbacks cb 0x7fdcbc00edd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdcc5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be95fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdcc5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be95fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59796/0 (socket says 172.21.15.103:59796) 2024-09-19T06:56:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdcc5d9b640 1 -- 172.21.15.103:0/165800586 learned_addr learned my addr 172.21.15.103:0/165800586 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdcc5d9b640 1 -- 172.21.15.103:0/165800586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93c9b0 msgr2=0x55b2be95f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdcc5d9b640 1 --2- 172.21.15.103:0/165800586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93c9b0 0x55b2be95f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdcc5d9b640 1 -- 172.21.15.103:0/165800586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93d2d0 msgr2=0x55b2be95f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdcc5d9b640 1 --2- 172.21.15.103:0/165800586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93d2d0 0x55b2be95f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.038+0000 7fdcc5d9b640 1 -- 172.21.15.103:0/165800586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2be9c5980 con 0x55b2be94c0c0 2024-09-19T06:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdcc5d9b640 1 --2- 172.21.15.103:0/165800586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be95fc30 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc000c00 tx=0x7fdcbc00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88081aef784c39e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdcadffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdcadffb640 1 -- 172.21.15.103:0/165800586 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc033040 con 0x55b2be94c0c0 2024-09-19T06:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdcadffb640 1 -- 172.21.15.103:0/165800586 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fdcbc00ba10 con 0x55b2be94c0c0 2024-09-19T06:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdccf7f3740 1 -- 172.21.15.103:0/165800586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcbc022030 con 0x55b2be94c0c0 2024-09-19T06:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdcadffb640 1 -- 172.21.15.103:0/165800586 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdcbc00bc10 con 0x55b2be94c0c0 2024-09-19T06:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdccf7f3740 10 client.18072 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdccf7f3740 1 -- 172.21.15.103:0/165800586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b2be9c6c50 con 0x55b2be94c0c0 2024-09-19T06:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdcadffb640 1 -- 172.21.15.103:0/165800586 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcbc022030 con 0x55b2be94c0c0 2024-09-19T06:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdcadffb640 1 -- 172.21.15.103:0/165800586 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdcbc005560 con 0x55b2be94c0c0 2024-09-19T06:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdccf7f3740 10 client.18072 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdccf7f3740 10 client.18072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdccf7f3740 20 client.18072 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/165800586 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b2be9c7250 0x55b2be9c9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdccf7f3740 4 client.18072 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdccf7f3740 1 -- 172.21.15.103:0/165800586 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2be825900 con 0x55b2be9c7250 2024-09-19T06:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.039+0000 7fdcc659c640 1 --2- 172.21.15.103:0/165800586 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b2be9c7250 0x55b2be9c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.040+0000 7fdcc659c640 1 --2- 172.21.15.103:0/165800586 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b2be9c7250 0x55b2be9c9640 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ff1ba37ee2cebcb server_cookie=c16af091a2075572 in_seq=0 out_seq=0 2024-09-19T06:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.040+0000 7fdcadffb640 10 client.18072 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.050+0000 7fdcadffb640 1 -- 172.21.15.103:0/165800586 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b2be825900 con 0x55b2be9c7250 2024-09-19T06:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.050+0000 7fdcadffb640 10 client.18072 handle_command_reply: tid=0 2024-09-19T06:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.050+0000 7fdccf7f3740 1 client.18072 shutdown 2024-09-19T06:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccf7f3740 1 -- 172.21.15.103:0/165800586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 msgr2=0x55b2be95fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/165800586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be95fc30 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc000c00 tx=0x7fdcbc00be70 comp rx=0 tx=0).stop 2024-09-19T06:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccf7f3740 1 -- 172.21.15.103:0/165800586 shutdown_connections 2024-09-19T06:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/165800586 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b2be9c7250 0x55b2be9c9640 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/165800586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2be94c0c0 0x55b2be95fc30 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/165800586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2be93d2d0 0x55b2be95f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccf7f3740 1 --2- 172.21.15.103:0/165800586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2be93c9b0 0x55b2be95f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdcadffb640 0 client.18072 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccf7f3740 1 -- 172.21.15.103:0/165800586 >> 172.21.15.103:0/165800586 conn(0x55b2be939060 msgr2=0x55b2be94b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccf7f3740 1 -- 172.21.15.103:0/165800586 shutdown_connections 2024-09-19T06:56:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccf7f3740 1 -- 172.21.15.103:0/165800586 wait complete. 2024-09-19T06:56:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccf7f3740 20 client.18072 trim_cache size 0 max 16384 2024-09-19T06:56:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:56:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:56:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:56:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:56:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccc981640 1 -- 172.21.15.103:0/1452087973 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdcb00a3460 msgr2=0x7fdcb00a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccc981640 1 --2- 172.21.15.103:0/1452087973 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdcb00a3460 0x7fdcb00a5920 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdcc009bd70 tx=0x7fdcc006ec10 comp rx=0 tx=0).stop 2024-09-19T06:56:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccc981640 1 -- 172.21.15.103:0/1452087973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcc8105520 msgr2=0x7fdcc819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccc981640 1 --2- 172.21.15.103:0/1452087973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcc8105520 0x7fdcc819cc70 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009fd0 tx=0x7fdcc000b260 comp rx=0 tx=0).stop 2024-09-19T06:56:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccc981640 1 -- 172.21.15.103:0/1452087973 shutdown_connections 2024-09-19T06:56:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccc981640 1 --2- 172.21.15.103:0/1452087973 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdcb00a3460 0x7fdcb00a5920 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccc981640 1 --2- 172.21.15.103:0/1452087973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcc8105520 0x7fdcc819cc70 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccc981640 1 --2- 172.21.15.103:0/1452087973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcc81011e0 0x7fdcc81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccc981640 1 --2- 172.21.15.103:0/1452087973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcc8100830 0x7fdcc81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccc981640 1 -- 172.21.15.103:0/1452087973 >> 172.21.15.103:0/1452087973 conn(0x7fdcc8075f20 msgr2=0x7fdcc81040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccc981640 1 -- 172.21.15.103:0/1452087973 shutdown_connections 2024-09-19T06:56:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:56:52.051+0000 7fdccc981640 1 -- 172.21.15.103:0/1452087973 wait complete. 2024-09-19T06:56:52.240 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:52 smithi103 ceph-mon[31158]: pgmap v234: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-19T06:56:52.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.320+0000 7fd72ab13640 1 -- 172.21.15.103:0/107153383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd724100e20 msgr2=0x7fd724101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.320+0000 7fd72ab13640 1 --2- 172.21.15.103:0/107153383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd724100e20 0x7fd724101200 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fd714009900 tx=0x7fd71402a480 comp rx=0 tx=0).stop 2024-09-19T06:56:52.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.321+0000 7fd72ab13640 1 -- 172.21.15.103:0/107153383 shutdown_connections 2024-09-19T06:56:52.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.321+0000 7fd72ab13640 1 --2- 172.21.15.103:0/107153383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd724102ef0 0x7fd72410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.321+0000 7fd72ab13640 1 --2- 172.21.15.103:0/107153383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd724102570 0x7fd7241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.321+0000 7fd72ab13640 1 --2- 172.21.15.103:0/107153383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd724100e20 0x7fd724101200 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.321+0000 7fd72ab13640 1 -- 172.21.15.103:0/107153383 >> 172.21.15.103:0/107153383 conn(0x7fd7240fc7d0 msgr2=0x7fd7240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:52.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.321+0000 7fd72ab13640 1 -- 172.21.15.103:0/107153383 shutdown_connections 2024-09-19T06:56:52.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.321+0000 7fd72ab13640 1 -- 172.21.15.103:0/107153383 wait complete. 2024-09-19T06:56:52.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.321+0000 7fd72ab13640 1 Processor -- start 2024-09-19T06:56:52.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.321+0000 7fd72ab13640 1 -- start start 2024-09-19T06:56:52.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.321+0000 7fd72ab13640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd724100e20 0x7fd724198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.321+0000 7fd72ab13640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd724102570 0x7fd724198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.321+0000 7fd72ab13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd724102ef0 0x7fd72419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.321+0000 7fd72ab13640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd724112120 con 0x7fd724102ef0 2024-09-19T06:56:52.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd72ab13640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd724111fa0 con 0x7fd724100e20 2024-09-19T06:56:52.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd72ab13640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd7241122a0 con 0x7fd724102570 2024-09-19T06:56:52.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd7290b0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd724102ef0 0x7fd72419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd7290b0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd724102ef0 0x7fd72419cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59826/0 (socket says 172.21.15.103:59826) 2024-09-19T06:56:52.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd7290b0640 1 -- 172.21.15.103:0/1472217935 learned_addr learned my addr 172.21.15.103:0/1472217935 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:52.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd723fff640 1 --2- 172.21.15.103:0/1472217935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd724102570 0x7fd724198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd7288af640 1 --2- 172.21.15.103:0/1472217935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd724100e20 0x7fd724198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd7290b0640 1 -- 172.21.15.103:0/1472217935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd724102570 msgr2=0x7fd724198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd7290b0640 1 --2- 172.21.15.103:0/1472217935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd724102570 0x7fd724198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd7290b0640 1 -- 172.21.15.103:0/1472217935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd724100e20 msgr2=0x7fd724198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd7290b0640 1 --2- 172.21.15.103:0/1472217935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd724100e20 0x7fd724198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd7290b0640 1 -- 172.21.15.103:0/1472217935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72419d630 con 0x7fd724102ef0 2024-09-19T06:56:52.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd723fff640 1 --2- 172.21.15.103:0/1472217935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd724102570 0x7fd724198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:52.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd7288af640 1 --2- 172.21.15.103:0/1472217935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd724100e20 0x7fd724198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:52.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd7290b0640 1 --2- 172.21.15.103:0/1472217935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd724102ef0 0x7fd72419cf50 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00b970 tx=0x7fd71c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:52.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd721ffb640 1 -- 172.21.15.103:0/1472217935 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c019070 con 0x7fd724102ef0 2024-09-19T06:56:52.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd72ab13640 1 -- 172.21.15.103:0/1472217935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72419d920 con 0x7fd724102ef0 2024-09-19T06:56:52.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd721ffb640 1 -- 172.21.15.103:0/1472217935 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd71c00cb40 con 0x7fd724102ef0 2024-09-19T06:56:52.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.322+0000 7fd721ffb640 1 -- 172.21.15.103:0/1472217935 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c007650 con 0x7fd724102ef0 2024-09-19T06:56:52.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.323+0000 7fd72ab13640 1 -- 172.21.15.103:0/1472217935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7241a5200 con 0x7fd724102ef0 2024-09-19T06:56:52.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.325+0000 7fd721ffb640 1 -- 172.21.15.103:0/1472217935 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd71c01c070 con 0x7fd724102ef0 2024-09-19T06:56:52.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.325+0000 7fd72ceec740 1 Processor -- start 2024-09-19T06:56:52.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.325+0000 7fd72ceec740 1 -- start start 2024-09-19T06:56:52.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.325+0000 7fd721ffb640 1 --2- 172.21.15.103:0/1472217935 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd70c0a33b0 0x7fd70c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.325+0000 7fd72ceec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10cb40 0x5566de10b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.325+0000 7fd72ceec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de10b5d0 0x5566de10ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.325+0000 7fd72ceec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de116370 0x5566de118760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.325+0000 7fd72ceec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5566de0712f0 con 0x5566de10cb40 2024-09-19T06:56:52.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.325+0000 7fd72ceec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5566de018cf0 con 0x5566de10b5d0 2024-09-19T06:56:52.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.325+0000 7fd72ceec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5566de06d2b0 con 0x5566de116370 2024-09-19T06:56:52.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.325+0000 7fd721ffb640 1 -- 172.21.15.103:0/1472217935 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fd71c0cf2d0 con 0x7fd724102ef0 2024-09-19T06:56:52.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.325+0000 7fd723fff640 1 --2- 172.21.15.103:0/1472217935 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd70c0a33b0 0x7fd70c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.325+0000 7fd7288af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10cb40 0x5566de10b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.325+0000 7fd7288af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10cb40 0x5566de10b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59834/0 (socket says 172.21.15.103:59834) 2024-09-19T06:56:52.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.325+0000 7fd7288af640 1 -- 172.21.15.103:0/2975136338 learned_addr learned my addr 172.21.15.103:0/2975136338 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:52.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.326+0000 7fd723fff640 1 --2- 172.21.15.103:0/2975136338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de10b5d0 0x5566de10ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.326+0000 7fd7290b0640 1 --2- 172.21.15.103:0/2975136338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de116370 0x5566de118760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.326+0000 7fd7288af640 1 -- 172.21.15.103:0/2975136338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de116370 msgr2=0x5566de118760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:52.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.326+0000 7fd7288af640 1 --2- 172.21.15.103:0/2975136338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de116370 0x5566de118760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.326+0000 7fd7288af640 1 -- 172.21.15.103:0/2975136338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de10b5d0 msgr2=0x5566de10ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:52.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.326+0000 7fd7288af640 1 --2- 172.21.15.103:0/2975136338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de10b5d0 0x5566de10ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.326+0000 7fd7288af640 1 -- 172.21.15.103:0/2975136338 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ddffd3a0 con 0x5566de10cb40 2024-09-19T06:56:52.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.326+0000 7fd723fff640 1 --2- 172.21.15.103:0/2975136338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de10b5d0 0x5566de10ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:52.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.326+0000 7fd723fff640 1 --2- 172.21.15.103:0/1472217935 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd70c0a33b0 0x7fd70c0a5870 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd7240ffea0 tx=0x7fd718014660 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:52.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.326+0000 7fd7288af640 1 --2- 172.21.15.103:0/2975136338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10cb40 0x5566de10b090 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fd7140314e0 tx=0x7fd714004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da87c515251c695d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:52.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.326+0000 7fd70affd640 1 -- 172.21.15.103:0/2975136338 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714035030 con 0x5566de10cb40 2024-09-19T06:56:52.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.326+0000 7fd70affd640 1 -- 172.21.15.103:0/2975136338 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd714003ad0 con 0x5566de10cb40 2024-09-19T06:56:52.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.326+0000 7fd70affd640 1 -- 172.21.15.103:0/2975136338 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714031c00 con 0x5566de10cb40 2024-09-19T06:56:52.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.327+0000 7fd72ceec740 1 -- 172.21.15.103:0/2975136338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10cb40 msgr2=0x5566de10b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.327+0000 7fd72ceec740 1 --2- 172.21.15.103:0/2975136338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10cb40 0x5566de10b090 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fd7140314e0 tx=0x7fd714004200 comp rx=0 tx=0).stop 2024-09-19T06:56:52.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.327+0000 7fd72ceec740 1 -- 172.21.15.103:0/2975136338 shutdown_connections 2024-09-19T06:56:52.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.327+0000 7fd72ceec740 1 --2- 172.21.15.103:0/2975136338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de116370 0x5566de118760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.327+0000 7fd72ceec740 1 --2- 172.21.15.103:0/2975136338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de10b5d0 0x5566de10ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.327+0000 7fd72ceec740 1 --2- 172.21.15.103:0/2975136338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10cb40 0x5566de10b090 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.327+0000 7fd72ceec740 1 -- 172.21.15.103:0/2975136338 >> 172.21.15.103:0/2975136338 conn(0x5566de0d4310 msgr2=0x5566de107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:52.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.327+0000 7fd72ceec740 1 -- 172.21.15.103:0/2975136338 shutdown_connections 2024-09-19T06:56:52.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.327+0000 7fd72ceec740 1 -- 172.21.15.103:0/2975136338 wait complete. 2024-09-19T06:56:52.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.327+0000 7fd72ceec740 1 -- start start 2024-09-19T06:56:52.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd72ceec740 1 Processor -- start 2024-09-19T06:56:52.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd72ceec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de10b5d0 0x5566de115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd72ceec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de10cb40 0x5566de112fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd72ceec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de116370 0x5566de113570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd72ceec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5566de11d420 con 0x5566de116370 2024-09-19T06:56:52.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd72ceec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5566de02c060 con 0x5566de10b5d0 2024-09-19T06:56:52.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd72ceec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5566de11d5a0 con 0x5566de10cb40 2024-09-19T06:56:52.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd72ceec740 10 client.0 _ll_register_callbacks cb 0x5566ddd7f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:52.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd7290b0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de116370 0x5566de113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd7290b0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de116370 0x5566de113570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59842/0 (socket says 172.21.15.103:59842) 2024-09-19T06:56:52.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd7290b0640 1 -- 172.21.15.103:0/2108453805 learned_addr learned my addr 172.21.15.103:0/2108453805 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:52.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd7288af640 1 --2- 172.21.15.103:0/2108453805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de10b5d0 0x5566de115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd723fff640 1 --2- 172.21.15.103:0/2108453805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de10cb40 0x5566de112fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd7290b0640 1 -- 172.21.15.103:0/2108453805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de10cb40 msgr2=0x5566de112fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:52.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd7290b0640 1 --2- 172.21.15.103:0/2108453805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de10cb40 0x5566de112fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd7290b0640 1 -- 172.21.15.103:0/2108453805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de10b5d0 msgr2=0x5566de115cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:52.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd7290b0640 1 --2- 172.21.15.103:0/2108453805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de10b5d0 0x5566de115cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd7290b0640 1 -- 172.21.15.103:0/2108453805 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566ddde23c0 con 0x5566de116370 2024-09-19T06:56:52.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.328+0000 7fd723fff640 1 --2- 172.21.15.103:0/2108453805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de10cb40 0x5566de112fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:52.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.329+0000 7fd7290b0640 1 --2- 172.21.15.103:0/2108453805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de116370 0x5566de113570 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fd71c09b630 tx=0x7fd71c09be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74ebbae66cfe8df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:52.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.329+0000 7fd703fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:52.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.329+0000 7fd703fff640 1 -- 172.21.15.103:0/2108453805 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c0a4310 con 0x5566de116370 2024-09-19T06:56:52.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.329+0000 7fd703fff640 1 -- 172.21.15.103:0/2108453805 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fd71c0ccc80 con 0x5566de116370 2024-09-19T06:56:52.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.329+0000 7fd72ceec740 1 -- 172.21.15.103:0/2108453805 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd714003ad0 con 0x5566de116370 2024-09-19T06:56:52.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.329+0000 7fd703fff640 1 -- 172.21.15.103:0/2108453805 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd71c097d10 con 0x5566de116370 2024-09-19T06:56:52.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.329+0000 7fd72ceec740 10 client.18084 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.329+0000 7fd72ceec740 1 -- 172.21.15.103:0/2108453805 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5566de128e70 con 0x5566de116370 2024-09-19T06:56:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.329+0000 7fd703fff640 1 -- 172.21.15.103:0/2108453805 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd714003ad0 con 0x5566de116370 2024-09-19T06:56:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.329+0000 7fd703fff640 1 -- 172.21.15.103:0/2108453805 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd71c0afbc0 con 0x5566de116370 2024-09-19T06:56:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.329+0000 7fd72ceec740 10 client.18084 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.329+0000 7fd72ceec740 10 client.18084 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:56:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.329+0000 7fd72ceec740 20 client.18084 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.329+0000 7fd72ceec740 1 --2- 172.21.15.103:0/2108453805 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5566de12a280 0x5566de12c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.329+0000 7fd72ceec740 4 client.18084 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.329+0000 7fd72ceec740 1 -- 172.21.15.103:0/2108453805 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566de07e860 con 0x5566de12a280 2024-09-19T06:56:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.330+0000 7fd7288af640 1 --2- 172.21.15.103:0/2108453805 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5566de12a280 0x5566de12c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.330+0000 7fd703fff640 10 client.18084 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:56:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.330+0000 7fd7288af640 1 --2- 172.21.15.103:0/2108453805 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5566de12a280 0x5566de12c6b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fca6f17d75d8dc12 server_cookie=9cc1adb46be20003 in_seq=0 out_seq=0 2024-09-19T06:56:52.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.331+0000 7fd703fff640 1 -- 172.21.15.103:0/2108453805 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566de07e860 con 0x5566de12a280 2024-09-19T06:56:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.331+0000 7fd703fff640 10 client.18084 handle_command_reply: tid=0 2024-09-19T06:56:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.331+0000 7fd72ceec740 1 client.18084 shutdown 2024-09-19T06:56:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.331+0000 7fd72ceec740 1 -- 172.21.15.103:0/2108453805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de116370 msgr2=0x5566de113570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.331+0000 7fd72ceec740 1 --2- 172.21.15.103:0/2108453805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de116370 0x5566de113570 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fd71c09b630 tx=0x7fd71c09be60 comp rx=0 tx=0).stop 2024-09-19T06:56:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.332+0000 7fd72ceec740 1 -- 172.21.15.103:0/2108453805 shutdown_connections 2024-09-19T06:56:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.332+0000 7fd72ceec740 1 --2- 172.21.15.103:0/2108453805 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5566de12a280 0x5566de12c6b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.332+0000 7fd72ceec740 1 --2- 172.21.15.103:0/2108453805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de116370 0x5566de113570 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.332+0000 7fd72ceec740 1 --2- 172.21.15.103:0/2108453805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de10cb40 0x5566de112fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.332+0000 7fd72ceec740 1 --2- 172.21.15.103:0/2108453805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de10b5d0 0x5566de115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.332+0000 7fd703fff640 0 client.18084 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:56:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.332+0000 7fd72ceec740 1 -- 172.21.15.103:0/2108453805 >> 172.21.15.103:0/2108453805 conn(0x5566de0d4310 msgr2=0x5566de10d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.332+0000 7fd72ceec740 1 -- 172.21.15.103:0/2108453805 shutdown_connections 2024-09-19T06:56:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.332+0000 7fd72ceec740 1 -- 172.21.15.103:0/2108453805 wait complete. 2024-09-19T06:56:52.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.332+0000 7fd72ceec740 20 client.18084 trim_cache size 0 max 16384 2024-09-19T06:56:52.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd72ceec740 1 Processor -- start 2024-09-19T06:56:52.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd72ceec740 1 -- start start 2024-09-19T06:56:52.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd72ceec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10b5d0 0x5566de1266c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd72ceec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de10cb40 0x5566de131f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd72ceec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de116370 0x5566de1324c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd72ceec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd7241ab7e0 con 0x5566de10b5d0 2024-09-19T06:56:52.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd72ceec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd7241ad320 con 0x5566de10cb40 2024-09-19T06:56:52.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd72ceec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd7241abeb0 con 0x5566de116370 2024-09-19T06:56:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd7288af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10b5d0 0x5566de1266c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd7288af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10b5d0 0x5566de1266c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59850/0 (socket says 172.21.15.103:59850) 2024-09-19T06:56:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd7288af640 1 -- 172.21.15.103:0/2419648912 learned_addr learned my addr 172.21.15.103:0/2419648912 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd7290b0640 1 --2- 172.21.15.103:0/2419648912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de116370 0x5566de1324c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd7288af640 1 -- 172.21.15.103:0/2419648912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de116370 msgr2=0x5566de1324c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd7288af640 1 --2- 172.21.15.103:0/2419648912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de116370 0x5566de1324c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd7288af640 1 -- 172.21.15.103:0/2419648912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de10cb40 msgr2=0x5566de131f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd7288af640 1 --2- 172.21.15.103:0/2419648912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de10cb40 0x5566de131f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd7288af640 1 -- 172.21.15.103:0/2419648912 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566de12cd30 con 0x5566de10b5d0 2024-09-19T06:56:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd7290b0640 1 --2- 172.21.15.103:0/2419648912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de116370 0x5566de1324c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.345+0000 7fd7288af640 1 --2- 172.21.15.103:0/2419648912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10b5d0 0x5566de1266c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fd714009900 tx=0x7fd7140318a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07a48d1369f5556 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.346+0000 7fd70a7fc640 1 -- 172.21.15.103:0/2419648912 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714035040 con 0x5566de10b5d0 2024-09-19T06:56:52.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.346+0000 7fd70a7fc640 1 -- 172.21.15.103:0/2419648912 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd714003cd0 con 0x5566de10b5d0 2024-09-19T06:56:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.346+0000 7fd70a7fc640 1 -- 172.21.15.103:0/2419648912 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71403c980 con 0x5566de10b5d0 2024-09-19T06:56:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.346+0000 7fd72ceec740 1 -- 172.21.15.103:0/2419648912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10b5d0 msgr2=0x5566de1266c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.346+0000 7fd72ceec740 1 --2- 172.21.15.103:0/2419648912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10b5d0 0x5566de1266c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fd714009900 tx=0x7fd7140318a0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.346+0000 7fd72ceec740 1 -- 172.21.15.103:0/2419648912 shutdown_connections 2024-09-19T06:56:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.346+0000 7fd72ceec740 1 --2- 172.21.15.103:0/2419648912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de116370 0x5566de1324c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.346+0000 7fd72ceec740 1 --2- 172.21.15.103:0/2419648912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de10cb40 0x5566de131f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.346+0000 7fd72ceec740 1 --2- 172.21.15.103:0/2419648912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10b5d0 0x5566de1266c0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.346+0000 7fd72ceec740 1 -- 172.21.15.103:0/2419648912 >> 172.21.15.103:0/2419648912 conn(0x5566de0d4310 msgr2=0x5566de112f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.346+0000 7fd72ceec740 1 -- 172.21.15.103:0/2419648912 shutdown_connections 2024-09-19T06:56:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.346+0000 7fd72ceec740 1 -- 172.21.15.103:0/2419648912 wait complete. 2024-09-19T06:56:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.347+0000 7fd72ceec740 1 -- start start 2024-09-19T06:56:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.347+0000 7fd72ceec740 1 Processor -- start 2024-09-19T06:56:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.347+0000 7fd72ceec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de10b5d0 0x5566de11e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.347+0000 7fd72ceec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10cb40 0x5566de11ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.347+0000 7fd72ceec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de116370 0x5566de11f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.347+0000 7fd72ceec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd7241acc50 con 0x5566de10cb40 2024-09-19T06:56:52.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.347+0000 7fd72ceec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd7241ac580 con 0x5566de116370 2024-09-19T06:56:52.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.347+0000 7fd72ceec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd7241a97c0 con 0x5566de10b5d0 2024-09-19T06:56:52.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.347+0000 7fd72ceec740 10 client.0 _ll_register_callbacks cb 0x7fd71c014b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:52.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.347+0000 7fd723fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10cb40 0x5566de11ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.347+0000 7fd723fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10cb40 0x5566de11ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59852/0 (socket says 172.21.15.103:59852) 2024-09-19T06:56:52.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.347+0000 7fd723fff640 1 -- 172.21.15.103:0/514131118 learned_addr learned my addr 172.21.15.103:0/514131118 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:52.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd7290b0640 1 --2- 172.21.15.103:0/514131118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de116370 0x5566de11f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd723fff640 1 -- 172.21.15.103:0/514131118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de10b5d0 msgr2=0x5566de11e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:52.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd7288af640 1 --2- 172.21.15.103:0/514131118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de10b5d0 0x5566de11e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd723fff640 1 --2- 172.21.15.103:0/514131118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de10b5d0 0x5566de11e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd723fff640 1 -- 172.21.15.103:0/514131118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de116370 msgr2=0x5566de11f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:52.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd723fff640 1 --2- 172.21.15.103:0/514131118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de116370 0x5566de11f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd723fff640 1 -- 172.21.15.103:0/514131118 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566de1941a0 con 0x5566de10cb40 2024-09-19T06:56:52.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd7290b0640 1 --2- 172.21.15.103:0/514131118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de116370 0x5566de11f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:52.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd7288af640 1 --2- 172.21.15.103:0/514131118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de10b5d0 0x5566de11e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:52.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd723fff640 1 --2- 172.21.15.103:0/514131118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10cb40 0x5566de11ed60 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x5566de108ac0 tx=0x7fd7180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4cace320a4cf1b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:52.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd703fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:52.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd703fff640 1 -- 172.21.15.103:0/514131118 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd718027020 con 0x5566de10cb40 2024-09-19T06:56:52.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd703fff640 1 -- 172.21.15.103:0/514131118 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fd718017d50 con 0x5566de10cb40 2024-09-19T06:56:52.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd72ceec740 1 -- 172.21.15.103:0/514131118 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd714003cd0 con 0x5566de10cb40 2024-09-19T06:56:52.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd703fff640 1 -- 172.21.15.103:0/514131118 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd718032050 con 0x5566de10cb40 2024-09-19T06:56:52.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd72ceec740 10 client.18090 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:52.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd72ceec740 1 -- 172.21.15.103:0/514131118 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5566de1953f0 con 0x5566de10cb40 2024-09-19T06:56:52.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.348+0000 7fd703fff640 1 -- 172.21.15.103:0/514131118 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd714003cd0 con 0x5566de10cb40 2024-09-19T06:56:52.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.349+0000 7fd703fff640 1 -- 172.21.15.103:0/514131118 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd718022030 con 0x5566de10cb40 2024-09-19T06:56:52.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.349+0000 7fd72ceec740 10 client.18090 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:52.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.349+0000 7fd72ceec740 10 client.18090 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:56:52.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.349+0000 7fd72ceec740 20 client.18090 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:52.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.349+0000 7fd72ceec740 1 --2- 172.21.15.103:0/514131118 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5566de195b90 0x5566de197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:52.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.349+0000 7fd72ceec740 4 client.18090 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:56:52.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.349+0000 7fd72ceec740 1 -- 172.21.15.103:0/514131118 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566de078d70 con 0x5566de195b90 2024-09-19T06:56:52.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.349+0000 7fd7288af640 1 --2- 172.21.15.103:0/514131118 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5566de195b90 0x5566de197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:52.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.349+0000 7fd7288af640 1 --2- 172.21.15.103:0/514131118 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5566de195b90 0x5566de197fb0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=33016ec4fa198b90 server_cookie=25de13d61658e800 in_seq=0 out_seq=0 2024-09-19T06:56:52.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.349+0000 7fd703fff640 10 client.18090 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:56:52.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.350+0000 7fd703fff640 1 -- 172.21.15.103:0/514131118 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566de078d70 con 0x5566de195b90 2024-09-19T06:56:52.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.350+0000 7fd703fff640 10 client.18090 handle_command_reply: tid=0 2024-09-19T06:56:52.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.350+0000 7fd72ceec740 1 client.18090 shutdown 2024-09-19T06:56:52.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.350+0000 7fd72ceec740 1 -- 172.21.15.103:0/514131118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10cb40 msgr2=0x5566de11ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.350+0000 7fd72ceec740 1 --2- 172.21.15.103:0/514131118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10cb40 0x5566de11ed60 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x5566de108ac0 tx=0x7fd7180046c0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.350+0000 7fd72ceec740 1 -- 172.21.15.103:0/514131118 shutdown_connections 2024-09-19T06:56:52.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.350+0000 7fd72ceec740 1 --2- 172.21.15.103:0/514131118 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5566de195b90 0x5566de197fb0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.350+0000 7fd72ceec740 1 --2- 172.21.15.103:0/514131118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566de116370 0x5566de11f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.350+0000 7fd72ceec740 1 --2- 172.21.15.103:0/514131118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566de10cb40 0x5566de11ed60 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.350+0000 7fd72ceec740 1 --2- 172.21.15.103:0/514131118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566de10b5d0 0x5566de11e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.350+0000 7fd72ceec740 1 -- 172.21.15.103:0/514131118 >> 172.21.15.103:0/514131118 conn(0x5566de0d4310 msgr2=0x5566de1064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:52.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.350+0000 7fd703fff640 0 client.18090 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:56:52.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.350+0000 7fd72ceec740 1 -- 172.21.15.103:0/514131118 shutdown_connections 2024-09-19T06:56:52.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.350+0000 7fd72ceec740 1 -- 172.21.15.103:0/514131118 wait complete. 2024-09-19T06:56:52.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.350+0000 7fd72ceec740 20 client.18090 trim_cache size 0 max 16384 2024-09-19T06:56:52.361 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:56:52.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.351+0000 7fd72ab13640 1 -- 172.21.15.103:0/1472217935 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd70c0a33b0 msgr2=0x7fd70c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.351+0000 7fd72ab13640 1 --2- 172.21.15.103:0/1472217935 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd70c0a33b0 0x7fd70c0a5870 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd7240ffea0 tx=0x7fd718014660 comp rx=0 tx=0).stop 2024-09-19T06:56:52.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.351+0000 7fd72ab13640 1 -- 172.21.15.103:0/1472217935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd724102ef0 msgr2=0x7fd72419cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:52.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.351+0000 7fd72ab13640 1 --2- 172.21.15.103:0/1472217935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd724102ef0 0x7fd72419cf50 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00b970 tx=0x7fd71c00be30 comp rx=0 tx=0).stop 2024-09-19T06:56:52.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.351+0000 7fd72ab13640 1 -- 172.21.15.103:0/1472217935 shutdown_connections 2024-09-19T06:56:52.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.351+0000 7fd72ab13640 1 --2- 172.21.15.103:0/1472217935 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd70c0a33b0 0x7fd70c0a5870 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.351+0000 7fd72ab13640 1 --2- 172.21.15.103:0/1472217935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd724102ef0 0x7fd72419cf50 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.351+0000 7fd72ab13640 1 --2- 172.21.15.103:0/1472217935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd724102570 0x7fd724198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.351+0000 7fd72ab13640 1 --2- 172.21.15.103:0/1472217935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd724100e20 0x7fd724198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:52.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.351+0000 7fd72ab13640 1 -- 172.21.15.103:0/1472217935 >> 172.21.15.103:0/1472217935 conn(0x7fd7240fc7d0 msgr2=0x7fd7240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:52.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.351+0000 7fd72ab13640 1 -- 172.21.15.103:0/1472217935 shutdown_connections 2024-09-19T06:56:52.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:52.351+0000 7fd72ab13640 1 -- 172.21.15.103:0/1472217935 wait complete. 2024-09-19T06:56:52.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:52 smithi134 ceph-mon[35356]: pgmap v234: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-19T06:56:52.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:52 smithi174 ceph-mon[35637]: pgmap v234: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-19T06:56:53.050 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:56:53.339 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:53.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.813+0000 7fd048268640 1 -- 172.21.15.103:0/192300120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd040101de0 msgr2=0x7fd0401021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:53.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.813+0000 7fd048268640 1 --2- 172.21.15.103:0/192300120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd040101de0 0x7fd0401021c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009900 tx=0x7fd03c02a410 comp rx=0 tx=0).stop 2024-09-19T06:56:53.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.813+0000 7fd048268640 1 -- 172.21.15.103:0/192300120 shutdown_connections 2024-09-19T06:56:53.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.813+0000 7fd048268640 1 --2- 172.21.15.103:0/192300120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd040106f10 0x7fd04010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.813+0000 7fd048268640 1 --2- 172.21.15.103:0/192300120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd040102790 0x7fd0401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.813+0000 7fd048268640 1 --2- 172.21.15.103:0/192300120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd040101de0 0x7fd0401021c0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.813+0000 7fd048268640 1 -- 172.21.15.103:0/192300120 >> 172.21.15.103:0/192300120 conn(0x7fd0400fd8c0 msgr2=0x7fd0400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:53.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.813+0000 7fd048268640 1 -- 172.21.15.103:0/192300120 shutdown_connections 2024-09-19T06:56:53.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.813+0000 7fd048268640 1 -- 172.21.15.103:0/192300120 wait complete. 2024-09-19T06:56:53.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd048268640 1 Processor -- start 2024-09-19T06:56:53.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd048268640 1 -- start start 2024-09-19T06:56:53.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd048268640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd040101de0 0x7fd0400744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd048268640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd040102790 0x7fd040074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd048268640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd040106f10 0x7fd040074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd048268640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd040111f30 con 0x7fd040101de0 2024-09-19T06:56:53.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd048268640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd040111db0 con 0x7fd040102790 2024-09-19T06:56:53.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd048268640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0401120b0 con 0x7fd040106f10 2024-09-19T06:56:53.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd046004640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd040101de0 0x7fd0400744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd046004640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd040101de0 0x7fd0400744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59874/0 (socket says 172.21.15.103:59874) 2024-09-19T06:56:53.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd046004640 1 -- 172.21.15.103:0/3199375278 learned_addr learned my addr 172.21.15.103:0/3199375278 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:53.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd045803640 1 --2- 172.21.15.103:0/3199375278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd040102790 0x7fd040074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd046805640 1 --2- 172.21.15.103:0/3199375278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd040106f10 0x7fd040074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd046004640 1 -- 172.21.15.103:0/3199375278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd040106f10 msgr2=0x7fd040074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:53.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd046004640 1 --2- 172.21.15.103:0/3199375278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd040106f10 0x7fd040074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd046004640 1 -- 172.21.15.103:0/3199375278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd040102790 msgr2=0x7fd040074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:53.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd046004640 1 --2- 172.21.15.103:0/3199375278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd040102790 0x7fd040074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd046004640 1 -- 172.21.15.103:0/3199375278 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0401a6980 con 0x7fd040101de0 2024-09-19T06:56:53.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd046805640 1 --2- 172.21.15.103:0/3199375278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd040106f10 0x7fd040074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:53.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.814+0000 7fd045803640 1 --2- 172.21.15.103:0/3199375278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd040102790 0x7fd040074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:53.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.815+0000 7fd046004640 1 --2- 172.21.15.103:0/3199375278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd040101de0 0x7fd0400744f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd03c006010 tx=0x7fd03c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:53.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.815+0000 7fd0377fe640 1 -- 172.21.15.103:0/3199375278 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c03f030 con 0x7fd040101de0 2024-09-19T06:56:53.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.815+0000 7fd0377fe640 1 -- 172.21.15.103:0/3199375278 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd03c003ce0 con 0x7fd040101de0 2024-09-19T06:56:53.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.815+0000 7fd048268640 1 -- 172.21.15.103:0/3199375278 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0401a6c10 con 0x7fd040101de0 2024-09-19T06:56:53.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.815+0000 7fd0377fe640 1 -- 172.21.15.103:0/3199375278 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c035070 con 0x7fd040101de0 2024-09-19T06:56:53.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.815+0000 7fd048268640 1 -- 172.21.15.103:0/3199375278 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0401a7140 con 0x7fd040101de0 2024-09-19T06:56:53.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.817+0000 7fd0377fe640 1 -- 172.21.15.103:0/3199375278 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd03c042030 con 0x7fd040101de0 2024-09-19T06:56:53.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.818+0000 7fd04a641740 1 Processor -- start 2024-09-19T06:56:53.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.818+0000 7fd04a641740 1 -- start start 2024-09-19T06:56:53.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.818+0000 7fd04a641740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563577632b40 0x563577631090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.818+0000 7fd0377fe640 1 --2- 172.21.15.103:0/3199375278 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd02c0a3430 0x7fd02c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.818+0000 7fd04a641740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635776315d0 0x563577631a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.818+0000 7fd04a641740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56357763c370 0x56357763e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.818+0000 7fd04a641740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5635775972f0 con 0x5635776315d0 2024-09-19T06:56:53.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.818+0000 7fd04a641740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56357753ecf0 con 0x56357763c370 2024-09-19T06:56:53.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.818+0000 7fd04a641740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5635775932b0 con 0x563577632b40 2024-09-19T06:56:53.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.818+0000 7fd045803640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635776315d0 0x563577631a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.818+0000 7fd046805640 1 --2- 172.21.15.103:0/3199375278 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd02c0a3430 0x7fd02c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.818+0000 7fd045803640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635776315d0 0x563577631a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59882/0 (socket says 172.21.15.103:59882) 2024-09-19T06:56:53.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.818+0000 7fd045803640 1 -- 172.21.15.103:0/4165548752 learned_addr learned my addr 172.21.15.103:0/4165548752 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:53.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.818+0000 7fd0377fe640 1 -- 172.21.15.103:0/3199375278 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fd03c0f0390 con 0x7fd040101de0 2024-09-19T06:56:53.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.818+0000 7fd046004640 1 --2- 172.21.15.103:0/4165548752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563577632b40 0x563577631090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.819+0000 7fd046805640 1 --2- 172.21.15.103:0/4165548752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56357763c370 0x56357763e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.819+0000 7fd046004640 1 -- 172.21.15.103:0/4165548752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56357763c370 msgr2=0x56357763e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:53.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.819+0000 7fd046004640 1 --2- 172.21.15.103:0/4165548752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56357763c370 0x56357763e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.819+0000 7fd046805640 1 --2- 172.21.15.103:0/3199375278 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd02c0a3430 0x7fd02c0a58f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd0380094d0 tx=0x7fd038009ab0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:53.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.819+0000 7fd046004640 1 -- 172.21.15.103:0/4165548752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635776315d0 msgr2=0x563577631a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:53.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.819+0000 7fd046004640 1 --2- 172.21.15.103:0/4165548752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635776315d0 0x563577631a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.819+0000 7fd046004640 1 -- 172.21.15.103:0/4165548752 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635775233a0 con 0x563577632b40 2024-09-19T06:56:53.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.819+0000 7fd045803640 1 --2- 172.21.15.103:0/4165548752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635776315d0 0x563577631a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:56:53.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.819+0000 7fd046004640 1 --2- 172.21.15.103:0/4165548752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563577632b40 0x563577631090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0eda60 tx=0x7fd03c02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a74e38516370b4dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:53.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.819+0000 7fd034dea640 1 -- 172.21.15.103:0/4165548752 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c049040 con 0x563577632b40 2024-09-19T06:56:53.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.819+0000 7fd034dea640 1 -- 172.21.15.103:0/4165548752 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd03c0bcdb0 con 0x563577632b40 2024-09-19T06:56:53.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.819+0000 7fd034dea640 1 -- 172.21.15.103:0/4165548752 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c0edce0 con 0x563577632b40 2024-09-19T06:56:53.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.820+0000 7fd04a641740 1 -- 172.21.15.103:0/4165548752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563577632b40 msgr2=0x563577631090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:53.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.820+0000 7fd04a641740 1 --2- 172.21.15.103:0/4165548752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563577632b40 0x563577631090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0eda60 tx=0x7fd03c02a410 comp rx=0 tx=0).stop 2024-09-19T06:56:53.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.820+0000 7fd04a641740 1 -- 172.21.15.103:0/4165548752 shutdown_connections 2024-09-19T06:56:53.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.820+0000 7fd04a641740 1 --2- 172.21.15.103:0/4165548752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56357763c370 0x56357763e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.820+0000 7fd04a641740 1 --2- 172.21.15.103:0/4165548752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635776315d0 0x563577631a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.820+0000 7fd04a641740 1 --2- 172.21.15.103:0/4165548752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563577632b40 0x563577631090 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.820+0000 7fd04a641740 1 -- 172.21.15.103:0/4165548752 >> 172.21.15.103:0/4165548752 conn(0x5635775fa310 msgr2=0x56357762dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:53.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.820+0000 7fd04a641740 1 -- 172.21.15.103:0/4165548752 shutdown_connections 2024-09-19T06:56:53.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.820+0000 7fd04a641740 1 -- 172.21.15.103:0/4165548752 wait complete. 2024-09-19T06:56:53.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.820+0000 7fd04a641740 1 -- start start 2024-09-19T06:56:53.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.820+0000 7fd04a641740 1 Processor -- start 2024-09-19T06:56:53.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd04a641740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635776315d0 0x563577639110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd04a641740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563577632b40 0x563577639650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd04a641740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357763c370 0x563577639b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd04a641740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5635775932b0 con 0x563577632b40 2024-09-19T06:56:53.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd04a641740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563577552060 con 0x5635776315d0 2024-09-19T06:56:53.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd04a641740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56357763ee30 con 0x56357763c370 2024-09-19T06:56:53.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd04a641740 10 client.0 _ll_register_callbacks cb 0x5635772a58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:53.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd045803640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563577632b40 0x563577639650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd045803640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563577632b40 0x563577639650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59892/0 (socket says 172.21.15.103:59892) 2024-09-19T06:56:53.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd045803640 1 -- 172.21.15.103:0/2618722740 learned_addr learned my addr 172.21.15.103:0/2618722740 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:53.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd046805640 1 --2- 172.21.15.103:0/2618722740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357763c370 0x563577639b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd046004640 1 --2- 172.21.15.103:0/2618722740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635776315d0 0x563577639110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd045803640 1 -- 172.21.15.103:0/2618722740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357763c370 msgr2=0x563577639b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:53.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd045803640 1 --2- 172.21.15.103:0/2618722740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357763c370 0x563577639b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd045803640 1 -- 172.21.15.103:0/2618722740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635776315d0 msgr2=0x563577639110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:53.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd045803640 1 --2- 172.21.15.103:0/2618722740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635776315d0 0x563577639110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd045803640 1 -- 172.21.15.103:0/2618722740 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635773083c0 con 0x563577632b40 2024-09-19T06:56:53.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd045803640 1 --2- 172.21.15.103:0/2618722740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563577632b40 0x563577639650 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fd03000bfc0 tx=0x7fd0300128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13342263baab7242 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:53.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.821+0000 7fd029ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:53.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.822+0000 7fd029ffb640 1 -- 172.21.15.103:0/2618722740 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03001f060 con 0x563577632b40 2024-09-19T06:56:53.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.822+0000 7fd029ffb640 1 -- 172.21.15.103:0/2618722740 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fd0300109c0 con 0x563577632b40 2024-09-19T06:56:53.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.822+0000 7fd04a641740 1 -- 172.21.15.103:0/2618722740 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd03c0bcdb0 con 0x563577632b40 2024-09-19T06:56:53.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.822+0000 7fd029ffb640 1 -- 172.21.15.103:0/2618722740 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd030010bc0 con 0x563577632b40 2024-09-19T06:56:53.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.822+0000 7fd04a641740 10 client.18102 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:53.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.822+0000 7fd04a641740 1 -- 172.21.15.103:0/2618722740 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56357764a8c0 con 0x563577632b40 2024-09-19T06:56:53.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.822+0000 7fd029ffb640 1 -- 172.21.15.103:0/2618722740 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd03c0bcdb0 con 0x563577632b40 2024-09-19T06:56:53.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.822+0000 7fd029ffb640 1 -- 172.21.15.103:0/2618722740 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd030019030 con 0x563577632b40 2024-09-19T06:56:53.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.822+0000 7fd04a641740 10 client.18102 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:53.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.822+0000 7fd04a641740 10 client.18102 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:56:53.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.822+0000 7fd04a641740 20 client.18102 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:53.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.822+0000 7fd04a641740 1 --2- 172.21.15.103:0/2618722740 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x56357764bd80 0x56357764e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.822+0000 7fd04a641740 4 client.18102 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:53.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.822+0000 7fd04a641740 1 -- 172.21.15.103:0/2618722740 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635775a4860 con 0x56357764bd80 2024-09-19T06:56:53.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.823+0000 7fd046004640 1 --2- 172.21.15.103:0/2618722740 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x56357764bd80 0x56357764e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.823+0000 7fd029ffb640 10 client.18102 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:56:53.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.823+0000 7fd046004640 1 --2- 172.21.15.103:0/2618722740 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x56357764bd80 0x56357764e1d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cea65328d8a56950 server_cookie=3c635ccc4d3639db in_seq=0 out_seq=0 2024-09-19T06:56:53.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.824+0000 7fd029ffb640 1 -- 172.21.15.103:0/2618722740 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635775a4860 con 0x56357764bd80 2024-09-19T06:56:53.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.824+0000 7fd029ffb640 10 client.18102 handle_command_reply: tid=0 2024-09-19T06:56:53.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.824+0000 7fd04a641740 1 client.18102 shutdown 2024-09-19T06:56:53.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.824+0000 7fd04a641740 1 -- 172.21.15.103:0/2618722740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563577632b40 msgr2=0x563577639650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:53.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.824+0000 7fd04a641740 1 --2- 172.21.15.103:0/2618722740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563577632b40 0x563577639650 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fd03000bfc0 tx=0x7fd0300128e0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.824+0000 7fd04a641740 1 -- 172.21.15.103:0/2618722740 shutdown_connections 2024-09-19T06:56:53.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.824+0000 7fd04a641740 1 --2- 172.21.15.103:0/2618722740 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x56357764bd80 0x56357764e1d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.824+0000 7fd04a641740 1 --2- 172.21.15.103:0/2618722740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357763c370 0x563577639b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.824+0000 7fd029ffb640 0 client.18102 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:56:53.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.824+0000 7fd04a641740 1 --2- 172.21.15.103:0/2618722740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563577632b40 0x563577639650 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.824+0000 7fd04a641740 1 --2- 172.21.15.103:0/2618722740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635776315d0 0x563577639110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.824+0000 7fd04a641740 1 -- 172.21.15.103:0/2618722740 >> 172.21.15.103:0/2618722740 conn(0x5635775fa310 msgr2=0x563577633960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:53.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.824+0000 7fd04a641740 1 -- 172.21.15.103:0/2618722740 shutdown_connections 2024-09-19T06:56:53.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.824+0000 7fd04a641740 1 -- 172.21.15.103:0/2618722740 wait complete. 2024-09-19T06:56:53.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.824+0000 7fd04a641740 20 client.18102 trim_cache size 0 max 16384 2024-09-19T06:56:53.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.837+0000 7fd04a641740 1 Processor -- start 2024-09-19T06:56:53.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.837+0000 7fd04a641740 1 -- start start 2024-09-19T06:56:53.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd04a641740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635776315d0 0x5635776379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd04a641740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563577632b40 0x563577634120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd04a641740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56357763c370 0x563577634660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd04a641740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd0401afa70 con 0x563577632b40 2024-09-19T06:56:53.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd04a641740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0401b15b0 con 0x56357763c370 2024-09-19T06:56:53.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd04a641740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0401b0140 con 0x5635776315d0 2024-09-19T06:56:53.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd045803640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563577632b40 0x563577634120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd045803640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563577632b40 0x563577634120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59904/0 (socket says 172.21.15.103:59904) 2024-09-19T06:56:53.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd045803640 1 -- 172.21.15.103:0/4169155333 learned_addr learned my addr 172.21.15.103:0/4169155333 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:53.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd046004640 1 --2- 172.21.15.103:0/4169155333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635776315d0 0x5635776379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd045803640 1 -- 172.21.15.103:0/4169155333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635776315d0 msgr2=0x5635776379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:53.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd045803640 1 --2- 172.21.15.103:0/4169155333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635776315d0 0x5635776379e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd046805640 1 --2- 172.21.15.103:0/4169155333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56357763c370 0x563577634660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd045803640 1 -- 172.21.15.103:0/4169155333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56357763c370 msgr2=0x563577634660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:53.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd045803640 1 --2- 172.21.15.103:0/4169155333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56357763c370 0x563577634660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd045803640 1 -- 172.21.15.103:0/4169155333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563577632760 con 0x563577632b40 2024-09-19T06:56:53.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd045803640 1 --2- 172.21.15.103:0/4169155333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563577632b40 0x563577634120 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd03000f680 tx=0x7fd030005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7693e24af38d8c8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:53.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd02bfff640 1 -- 172.21.15.103:0/4169155333 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0300107c0 con 0x563577632b40 2024-09-19T06:56:53.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd02bfff640 1 -- 172.21.15.103:0/4169155333 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd030010960 con 0x563577632b40 2024-09-19T06:56:53.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.838+0000 7fd02bfff640 1 -- 172.21.15.103:0/4169155333 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03000f700 con 0x563577632b40 2024-09-19T06:56:53.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.839+0000 7fd04a641740 1 -- 172.21.15.103:0/4169155333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563577632b40 msgr2=0x563577634120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:53.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.839+0000 7fd04a641740 1 --2- 172.21.15.103:0/4169155333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563577632b40 0x563577634120 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd03000f680 tx=0x7fd030005970 comp rx=0 tx=0).stop 2024-09-19T06:56:53.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.839+0000 7fd04a641740 1 -- 172.21.15.103:0/4169155333 shutdown_connections 2024-09-19T06:56:53.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.839+0000 7fd04a641740 1 --2- 172.21.15.103:0/4169155333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56357763c370 0x563577634660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.839+0000 7fd04a641740 1 --2- 172.21.15.103:0/4169155333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563577632b40 0x563577634120 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.839+0000 7fd04a641740 1 --2- 172.21.15.103:0/4169155333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635776315d0 0x5635776379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.839+0000 7fd04a641740 1 -- 172.21.15.103:0/4169155333 >> 172.21.15.103:0/4169155333 conn(0x5635775fa310 msgr2=0x56357763b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:53.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.839+0000 7fd04a641740 1 -- 172.21.15.103:0/4169155333 shutdown_connections 2024-09-19T06:56:53.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.839+0000 7fd04a641740 1 -- 172.21.15.103:0/4169155333 wait complete. 2024-09-19T06:56:53.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd04a641740 1 -- start start 2024-09-19T06:56:53.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd04a641740 1 Processor -- start 2024-09-19T06:56:53.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd04a641740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635776315d0 0x5635776360c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd04a641740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563577632b40 0x563577636600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd04a641740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357763c370 0x563577636b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd04a641740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd0401b0ee0 con 0x5635776315d0 2024-09-19T06:56:53.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd04a641740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0401b0810 con 0x563577632b40 2024-09-19T06:56:53.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd04a641740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0401ada70 con 0x56357763c370 2024-09-19T06:56:53.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd04a641740 10 client.0 _ll_register_callbacks cb 0x7fd03000b940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:53.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd046004640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635776315d0 0x5635776360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd046004640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635776315d0 0x5635776360c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59918/0 (socket says 172.21.15.103:59918) 2024-09-19T06:56:53.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd046004640 1 -- 172.21.15.103:0/3495595435 learned_addr learned my addr 172.21.15.103:0/3495595435 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:53.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd046805640 1 --2- 172.21.15.103:0/3495595435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357763c370 0x563577636b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd045803640 1 --2- 172.21.15.103:0/3495595435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563577632b40 0x563577636600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd046004640 1 -- 172.21.15.103:0/3495595435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357763c370 msgr2=0x563577636b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:53.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd046004640 1 --2- 172.21.15.103:0/3495595435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357763c370 0x563577636b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd046004640 1 -- 172.21.15.103:0/3495595435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563577632b40 msgr2=0x563577636600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:53.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd046004640 1 --2- 172.21.15.103:0/3495595435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563577632b40 0x563577636600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd046004640 1 -- 172.21.15.103:0/3495595435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635776b5ab0 con 0x5635776315d0 2024-09-19T06:56:53.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd046805640 1 --2- 172.21.15.103:0/3495595435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357763c370 0x563577636b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:53.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.840+0000 7fd045803640 1 --2- 172.21.15.103:0/3495595435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563577632b40 0x563577636600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:53.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.841+0000 7fd046004640 1 --2- 172.21.15.103:0/3495595435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635776315d0 0x5635776360c0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fd03c000c00 tx=0x7fd03c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a62ab35fe24791a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:53.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.841+0000 7fd029ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:53.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.841+0000 7fd029ffb640 1 -- 172.21.15.103:0/3495595435 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c049040 con 0x5635776315d0 2024-09-19T06:56:53.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.841+0000 7fd029ffb640 1 -- 172.21.15.103:0/3495595435 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fd03c0b8d20 con 0x5635776315d0 2024-09-19T06:56:53.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.841+0000 7fd04a641740 1 -- 172.21.15.103:0/3495595435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd030010960 con 0x5635776315d0 2024-09-19T06:56:53.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.841+0000 7fd029ffb640 1 -- 172.21.15.103:0/3495595435 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd03c0bcdb0 con 0x5635776315d0 2024-09-19T06:56:53.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.841+0000 7fd04a641740 10 client.18108 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:53.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.841+0000 7fd04a641740 1 -- 172.21.15.103:0/3495595435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635776b6d40 con 0x5635776315d0 2024-09-19T06:56:53.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.841+0000 7fd029ffb640 1 -- 172.21.15.103:0/3495595435 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd030010960 con 0x5635776315d0 2024-09-19T06:56:53.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.841+0000 7fd029ffb640 1 -- 172.21.15.103:0/3495595435 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd03c0c4e90 con 0x5635776315d0 2024-09-19T06:56:53.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.841+0000 7fd04a641740 10 client.18108 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:53.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.841+0000 7fd04a641740 10 client.18108 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:56:53.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.841+0000 7fd04a641740 20 client.18108 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.841+0000 7fd04a641740 1 --2- 172.21.15.103:0/3495595435 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5635776b72c0 0x5635776b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.841+0000 7fd04a641740 4 client.18108 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:56:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.841+0000 7fd04a641740 1 -- 172.21.15.103:0/3495595435 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56357759ed70 con 0x5635776b72c0 2024-09-19T06:56:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.842+0000 7fd045803640 1 --2- 172.21.15.103:0/3495595435 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5635776b72c0 0x5635776b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.842+0000 7fd029ffb640 10 client.18108 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:56:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.842+0000 7fd045803640 1 --2- 172.21.15.103:0/3495595435 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5635776b72c0 0x5635776b96b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d3e070aa988b3a9b server_cookie=471b696414dbf271 in_seq=0 out_seq=0 2024-09-19T06:56:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.842+0000 7fd029ffb640 1 -- 172.21.15.103:0/3495595435 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56357759ed70 con 0x5635776b72c0 2024-09-19T06:56:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.842+0000 7fd029ffb640 10 client.18108 handle_command_reply: tid=0 2024-09-19T06:56:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.842+0000 7fd04a641740 1 client.18108 shutdown 2024-09-19T06:56:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.842+0000 7fd04a641740 1 -- 172.21.15.103:0/3495595435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635776315d0 msgr2=0x5635776360c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.842+0000 7fd04a641740 1 --2- 172.21.15.103:0/3495595435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635776315d0 0x5635776360c0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fd03c000c00 tx=0x7fd03c005bd0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.842+0000 7fd04a641740 1 -- 172.21.15.103:0/3495595435 shutdown_connections 2024-09-19T06:56:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.842+0000 7fd04a641740 1 --2- 172.21.15.103:0/3495595435 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5635776b72c0 0x5635776b96b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.842+0000 7fd04a641740 1 --2- 172.21.15.103:0/3495595435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357763c370 0x563577636b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.842+0000 7fd04a641740 1 --2- 172.21.15.103:0/3495595435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563577632b40 0x563577636600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.842+0000 7fd029ffb640 0 client.18108 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:56:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.842+0000 7fd04a641740 1 --2- 172.21.15.103:0/3495595435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635776315d0 0x5635776360c0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.842+0000 7fd04a641740 1 -- 172.21.15.103:0/3495595435 >> 172.21.15.103:0/3495595435 conn(0x5635775fa310 msgr2=0x563577638fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.843+0000 7fd04a641740 1 -- 172.21.15.103:0/3495595435 shutdown_connections 2024-09-19T06:56:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.843+0000 7fd04a641740 1 -- 172.21.15.103:0/3495595435 wait complete. 2024-09-19T06:56:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.843+0000 7fd04a641740 20 client.18108 trim_cache size 0 max 16384 2024-09-19T06:56:53.859 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:56:53.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.843+0000 7fd048268640 1 -- 172.21.15.103:0/3199375278 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd02c0a3430 msgr2=0x7fd02c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.843+0000 7fd048268640 1 --2- 172.21.15.103:0/3199375278 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd02c0a3430 0x7fd02c0a58f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd0380094d0 tx=0x7fd038009ab0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.843+0000 7fd048268640 1 -- 172.21.15.103:0/3199375278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd040101de0 msgr2=0x7fd0400744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.843+0000 7fd048268640 1 --2- 172.21.15.103:0/3199375278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd040101de0 0x7fd0400744f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd03c006010 tx=0x7fd03c0042f0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.843+0000 7fd048268640 1 -- 172.21.15.103:0/3199375278 shutdown_connections 2024-09-19T06:56:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.843+0000 7fd048268640 1 --2- 172.21.15.103:0/3199375278 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd02c0a3430 0x7fd02c0a58f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.843+0000 7fd048268640 1 --2- 172.21.15.103:0/3199375278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd040106f10 0x7fd040074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.843+0000 7fd048268640 1 --2- 172.21.15.103:0/3199375278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd040102790 0x7fd040074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.843+0000 7fd048268640 1 --2- 172.21.15.103:0/3199375278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd040101de0 0x7fd0400744f0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.843+0000 7fd048268640 1 -- 172.21.15.103:0/3199375278 >> 172.21.15.103:0/3199375278 conn(0x7fd0400fd8c0 msgr2=0x7fd0400fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.843+0000 7fd048268640 1 -- 172.21.15.103:0/3199375278 shutdown_connections 2024-09-19T06:56:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:53.843+0000 7fd048268640 1 -- 172.21.15.103:0/3199375278 wait complete. 2024-09-19T06:56:54.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:54 smithi103 ceph-mon[31158]: pgmap v235: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-19T06:56:54.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:54 smithi134 ceph-mon[35356]: pgmap v235: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-19T06:56:54.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:54 smithi174 ceph-mon[35637]: pgmap v235: 97 pgs: 97 active+clean; 5.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-19T06:56:54.590 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:56:54.836 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:55.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.290+0000 7fabd333a640 1 -- 172.21.15.103:0/383230530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabcc100d00 msgr2=0x7fabcc106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:55.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.290+0000 7fabd333a640 1 --2- 172.21.15.103:0/383230530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabcc100d00 0x7fabcc106840 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009930 tx=0x7fabc002a420 comp rx=0 tx=0).stop 2024-09-19T06:56:55.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.290+0000 7fabd333a640 1 -- 172.21.15.103:0/383230530 shutdown_connections 2024-09-19T06:56:55.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.290+0000 7fabd333a640 1 --2- 172.21.15.103:0/383230530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabcc106f50 0x7fabcc10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.290+0000 7fabd333a640 1 --2- 172.21.15.103:0/383230530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabcc100d00 0x7fabcc106840 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.290+0000 7fabd333a640 1 --2- 172.21.15.103:0/383230530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabcc100350 0x7fabcc100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.290+0000 7fabd333a640 1 -- 172.21.15.103:0/383230530 >> 172.21.15.103:0/383230530 conn(0x7fabcc077210 msgr2=0x7fabcc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:55.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.290+0000 7fabd333a640 1 -- 172.21.15.103:0/383230530 shutdown_connections 2024-09-19T06:56:55.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.290+0000 7fabd333a640 1 -- 172.21.15.103:0/383230530 wait complete. 2024-09-19T06:56:55.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.290+0000 7fabd333a640 1 Processor -- start 2024-09-19T06:56:55.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.290+0000 7fabd333a640 1 -- start start 2024-09-19T06:56:55.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd333a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabcc100350 0x7fabcc1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd333a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabcc100d00 0x7fabcc198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd333a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabcc106f50 0x7fabcc19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd333a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc111fb0 con 0x7fabcc100d00 2024-09-19T06:56:55.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd333a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc111e30 con 0x7fabcc100350 2024-09-19T06:56:55.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd333a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc112130 con 0x7fabcc106f50 2024-09-19T06:56:55.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd08d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabcc100d00 0x7fabcc198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd08d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabcc100d00 0x7fabcc198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59938/0 (socket says 172.21.15.103:59938) 2024-09-19T06:56:55.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd08d5640 1 -- 172.21.15.103:0/3258052327 learned_addr learned my addr 172.21.15.103:0/3258052327 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:55.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd10d6640 1 --2- 172.21.15.103:0/3258052327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabcc100350 0x7fabcc1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd18d7640 1 --2- 172.21.15.103:0/3258052327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabcc106f50 0x7fabcc19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd08d5640 1 -- 172.21.15.103:0/3258052327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabcc106f50 msgr2=0x7fabcc19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:55.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd08d5640 1 --2- 172.21.15.103:0/3258052327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabcc106f50 0x7fabcc19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd08d5640 1 -- 172.21.15.103:0/3258052327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabcc100350 msgr2=0x7fabcc1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:55.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd08d5640 1 --2- 172.21.15.103:0/3258052327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabcc100350 0x7fabcc1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd08d5640 1 -- 172.21.15.103:0/3258052327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabcc19d4a0 con 0x7fabcc100d00 2024-09-19T06:56:55.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.291+0000 7fabd08d5640 1 --2- 172.21.15.103:0/3258052327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabcc100d00 0x7fabcc198a30 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fabc002a940 tx=0x7fabc0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:55.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.292+0000 7fabca7fc640 1 -- 172.21.15.103:0/3258052327 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc003f070 con 0x7fabcc100d00 2024-09-19T06:56:55.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.292+0000 7fabca7fc640 1 -- 172.21.15.103:0/3258052327 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc0031c90 con 0x7fabcc100d00 2024-09-19T06:56:55.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.292+0000 7fabca7fc640 1 -- 172.21.15.103:0/3258052327 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0035030 con 0x7fabcc100d00 2024-09-19T06:56:55.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.292+0000 7fabd333a640 1 -- 172.21.15.103:0/3258052327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabcc19d730 con 0x7fabcc100d00 2024-09-19T06:56:55.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.292+0000 7fabd333a640 1 -- 172.21.15.103:0/3258052327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabcc1a5070 con 0x7fabcc100d00 2024-09-19T06:56:55.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.294+0000 7fabca7fc640 1 -- 172.21.15.103:0/3258052327 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fabc0042070 con 0x7fabcc100d00 2024-09-19T06:56:55.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.294+0000 7fabd5713740 1 Processor -- start 2024-09-19T06:56:55.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.294+0000 7fabd5713740 1 -- start start 2024-09-19T06:56:55.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd5713740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47437b40 0x562d47436090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd5713740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d474365d0 0x562d47436a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd5713740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d47441370 0x562d47443760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd5713740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562d4739c2f0 con 0x562d47441370 2024-09-19T06:56:55.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd5713740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562d47343cf0 con 0x562d47437b40 2024-09-19T06:56:55.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd5713740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562d473982b0 con 0x562d474365d0 2024-09-19T06:56:55.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd08d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d47441370 0x562d47443760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabca7fc640 1 --2- 172.21.15.103:0/3258052327 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fabb40a33b0 0x7fabb40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd08d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d47441370 0x562d47443760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59946/0 (socket says 172.21.15.103:59946) 2024-09-19T06:56:55.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd08d5640 1 -- 172.21.15.103:0/773162247 learned_addr learned my addr 172.21.15.103:0/773162247 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:55.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd10d6640 1 --2- 172.21.15.103:0/773162247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d474365d0 0x562d47436a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd18d7640 1 --2- 172.21.15.103:0/773162247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47437b40 0x562d47436090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd18d7640 1 --2- 172.21.15.103:0/3258052327 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fabb40a33b0 0x7fabb40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabca7fc640 1 -- 172.21.15.103:0/3258052327 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fabc00f04b0 con 0x7fabcc100d00 2024-09-19T06:56:55.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd08d5640 1 -- 172.21.15.103:0/773162247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d474365d0 msgr2=0x562d47436a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:55.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd08d5640 1 --2- 172.21.15.103:0/773162247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d474365d0 0x562d47436a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd08d5640 1 -- 172.21.15.103:0/773162247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47437b40 msgr2=0x562d47436090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:55.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd08d5640 1 --2- 172.21.15.103:0/773162247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47437b40 0x562d47436090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd08d5640 1 -- 172.21.15.103:0/773162247 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d473283a0 con 0x562d47441370 2024-09-19T06:56:55.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd18d7640 1 --2- 172.21.15.103:0/773162247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47437b40 0x562d47436090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:55.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabd08d5640 1 --2- 172.21.15.103:0/773162247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d47441370 0x562d47443760 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fabc00c4eb0 tx=0x7fabc0034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c41bd5e6ee9d01b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:55.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.295+0000 7fabb37fe640 1 -- 172.21.15.103:0/773162247 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0049070 con 0x562d47441370 2024-09-19T06:56:55.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.296+0000 7fabb37fe640 1 -- 172.21.15.103:0/773162247 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc00c0b20 con 0x562d47441370 2024-09-19T06:56:55.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.296+0000 7fabb37fe640 1 -- 172.21.15.103:0/773162247 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc003b4f0 con 0x562d47441370 2024-09-19T06:56:55.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.296+0000 7fabd5713740 1 -- 172.21.15.103:0/773162247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d47441370 msgr2=0x562d47443760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:55.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.296+0000 7fabd18d7640 1 --2- 172.21.15.103:0/3258052327 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fabb40a33b0 0x7fabb40a5870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fabc4005fd0 tx=0x7fabc4009600 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:55.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.296+0000 7fabd5713740 1 --2- 172.21.15.103:0/773162247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d47441370 0x562d47443760 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fabc00c4eb0 tx=0x7fabc0034000 comp rx=0 tx=0).stop 2024-09-19T06:56:55.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.296+0000 7fabd5713740 1 -- 172.21.15.103:0/773162247 shutdown_connections 2024-09-19T06:56:55.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.296+0000 7fabd5713740 1 --2- 172.21.15.103:0/773162247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d47441370 0x562d47443760 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.296+0000 7fabd5713740 1 --2- 172.21.15.103:0/773162247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d474365d0 0x562d47436a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.296+0000 7fabd5713740 1 --2- 172.21.15.103:0/773162247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47437b40 0x562d47436090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.296+0000 7fabd5713740 1 -- 172.21.15.103:0/773162247 >> 172.21.15.103:0/773162247 conn(0x562d473ff310 msgr2=0x562d47432c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:55.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.296+0000 7fabd5713740 1 -- 172.21.15.103:0/773162247 shutdown_connections 2024-09-19T06:56:55.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.296+0000 7fabd5713740 1 -- 172.21.15.103:0/773162247 wait complete. 2024-09-19T06:56:55.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.297+0000 7fabd5713740 1 -- start start 2024-09-19T06:56:55.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.297+0000 7fabd5713740 1 Processor -- start 2024-09-19T06:56:55.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.297+0000 7fabd5713740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d474365d0 0x562d47440df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.297+0000 7fabd5713740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d47437b40 0x562d4743e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.297+0000 7fabd5713740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47441370 0x562d4743e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.297+0000 7fabd5713740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562d4739c2f0 con 0x562d474365d0 2024-09-19T06:56:55.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.297+0000 7fabd5713740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562d47357060 con 0x562d47441370 2024-09-19T06:56:55.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.297+0000 7fabd5713740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562d47443e30 con 0x562d47437b40 2024-09-19T06:56:55.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.297+0000 7fabd5713740 10 client.0 _ll_register_callbacks cb 0x562d470aa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:55.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.297+0000 7fabd10d6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d474365d0 0x562d47440df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.297+0000 7fabd10d6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d474365d0 0x562d47440df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59950/0 (socket says 172.21.15.103:59950) 2024-09-19T06:56:55.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.297+0000 7fabd10d6640 1 -- 172.21.15.103:0/472100514 learned_addr learned my addr 172.21.15.103:0/472100514 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:55.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.297+0000 7fabd18d7640 1 --2- 172.21.15.103:0/472100514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47441370 0x562d4743e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabd10d6640 1 -- 172.21.15.103:0/472100514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d47437b40 msgr2=0x562d4743e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:55.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabd10d6640 1 --2- 172.21.15.103:0/472100514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d47437b40 0x562d4743e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabd10d6640 1 -- 172.21.15.103:0/472100514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47441370 msgr2=0x562d4743e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:55.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabd10d6640 1 --2- 172.21.15.103:0/472100514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47441370 0x562d4743e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabd10d6640 1 -- 172.21.15.103:0/472100514 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d4710d3c0 con 0x562d474365d0 2024-09-19T06:56:55.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabd10d6640 1 --2- 172.21.15.103:0/472100514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d474365d0 0x562d47440df0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00bfc0 tx=0x7fabbc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62bb50be64af60e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:55.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:55.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabb0ff9640 1 -- 172.21.15.103:0/472100514 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc004020 con 0x562d474365d0 2024-09-19T06:56:55.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabb0ff9640 1 -- 172.21.15.103:0/472100514 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fabbc0041c0 con 0x562d474365d0 2024-09-19T06:56:55.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabd5713740 1 -- 172.21.15.103:0/472100514 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabc00c0b20 con 0x562d474365d0 2024-09-19T06:56:55.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabb0ff9640 1 -- 172.21.15.103:0/472100514 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fabbc0043c0 con 0x562d474365d0 2024-09-19T06:56:55.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabd5713740 10 client.18120 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:55.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabd5713740 1 -- 172.21.15.103:0/472100514 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d4744f7e0 con 0x562d474365d0 2024-09-19T06:56:55.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabb0ff9640 1 -- 172.21.15.103:0/472100514 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc00c0b20 con 0x562d474365d0 2024-09-19T06:56:55.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabb0ff9640 1 -- 172.21.15.103:0/472100514 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fabbc003b90 con 0x562d474365d0 2024-09-19T06:56:55.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabd5713740 10 client.18120 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:55.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabd5713740 10 client.18120 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:56:55.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.298+0000 7fabd5713740 20 client.18120 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:55.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.299+0000 7fabd5713740 1 --2- 172.21.15.103:0/472100514 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x562d47450d90 0x562d47453230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.299+0000 7fabd5713740 4 client.18120 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:55.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.299+0000 7fabd5713740 1 -- 172.21.15.103:0/472100514 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d473a9860 con 0x562d47450d90 2024-09-19T06:56:55.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.299+0000 7fabd08d5640 1 --2- 172.21.15.103:0/472100514 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x562d47450d90 0x562d47453230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.299+0000 7fabd08d5640 1 --2- 172.21.15.103:0/472100514 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x562d47450d90 0x562d47453230 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5007b35113389c4 server_cookie=b9b46d2a82071976 in_seq=0 out_seq=0 2024-09-19T06:56:55.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.299+0000 7fabb0ff9640 10 client.18120 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:56:55.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.300+0000 7fabb0ff9640 1 -- 172.21.15.103:0/472100514 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d473a9860 con 0x562d47450d90 2024-09-19T06:56:55.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.300+0000 7fabb0ff9640 10 client.18120 handle_command_reply: tid=0 2024-09-19T06:56:55.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.300+0000 7fabd5713740 1 client.18120 shutdown 2024-09-19T06:56:55.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.300+0000 7fabd5713740 1 -- 172.21.15.103:0/472100514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d474365d0 msgr2=0x562d47440df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:55.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.300+0000 7fabd5713740 1 --2- 172.21.15.103:0/472100514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d474365d0 0x562d47440df0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00bfc0 tx=0x7fabbc0118d0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.300+0000 7fabd5713740 1 -- 172.21.15.103:0/472100514 shutdown_connections 2024-09-19T06:56:55.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.300+0000 7fabd5713740 1 --2- 172.21.15.103:0/472100514 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x562d47450d90 0x562d47453230 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.300+0000 7fabd5713740 1 --2- 172.21.15.103:0/472100514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47441370 0x562d4743e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.300+0000 7fabd5713740 1 --2- 172.21.15.103:0/472100514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d47437b40 0x562d4743e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.300+0000 7fabb0ff9640 0 client.18120 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:56:55.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.300+0000 7fabd5713740 1 --2- 172.21.15.103:0/472100514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d474365d0 0x562d47440df0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.301+0000 7fabd5713740 1 -- 172.21.15.103:0/472100514 >> 172.21.15.103:0/472100514 conn(0x562d473ff310 msgr2=0x562d47438990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:55.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.301+0000 7fabd5713740 1 -- 172.21.15.103:0/472100514 shutdown_connections 2024-09-19T06:56:55.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.301+0000 7fabd5713740 1 -- 172.21.15.103:0/472100514 wait complete. 2024-09-19T06:56:55.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.301+0000 7fabd5713740 20 client.18120 trim_cache size 0 max 16384 2024-09-19T06:56:55.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.313+0000 7fabd5713740 1 Processor -- start 2024-09-19T06:56:55.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.313+0000 7fabd5713740 1 -- start start 2024-09-19T06:56:55.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.313+0000 7fabd5713740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d474365d0 0x562d4744cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.313+0000 7fabd5713740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d47437b40 0x562d47439180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.313+0000 7fabd5713740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d47441370 0x562d474396c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.313+0000 7fabd5713740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1ada50 con 0x562d47437b40 2024-09-19T06:56:55.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.313+0000 7fabd5713740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1af590 con 0x562d474365d0 2024-09-19T06:56:55.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabd5713740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1ae120 con 0x562d47441370 2024-09-19T06:56:55.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabd08d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d47437b40 0x562d47439180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabd08d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d47437b40 0x562d47439180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59966/0 (socket says 172.21.15.103:59966) 2024-09-19T06:56:55.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabd08d5640 1 -- 172.21.15.103:0/405794741 learned_addr learned my addr 172.21.15.103:0/405794741 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:55.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabd10d6640 1 --2- 172.21.15.103:0/405794741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d474365d0 0x562d4744cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabd18d7640 1 --2- 172.21.15.103:0/405794741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d47441370 0x562d474396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabd08d5640 1 -- 172.21.15.103:0/405794741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d47441370 msgr2=0x562d474396c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:55.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabd08d5640 1 --2- 172.21.15.103:0/405794741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d47441370 0x562d474396c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabd08d5640 1 -- 172.21.15.103:0/405794741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d474365d0 msgr2=0x562d4744cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:55.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabd08d5640 1 --2- 172.21.15.103:0/405794741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d474365d0 0x562d4744cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabd08d5640 1 -- 172.21.15.103:0/405794741 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d4744d490 con 0x562d47437b40 2024-09-19T06:56:55.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabd18d7640 1 --2- 172.21.15.103:0/405794741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d47441370 0x562d474396c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:55.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabd10d6640 1 --2- 172.21.15.103:0/405794741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d474365d0 0x562d4744cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:55.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabd08d5640 1 --2- 172.21.15.103:0/405794741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d47437b40 0x562d47439180 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fabc00edc90 tx=0x7fabc003b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9054506ddd89e963 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:55.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabb2ffd640 1 -- 172.21.15.103:0/405794741 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0049040 con 0x562d47437b40 2024-09-19T06:56:55.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabb2ffd640 1 -- 172.21.15.103:0/405794741 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc00c0b20 con 0x562d47437b40 2024-09-19T06:56:55.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.314+0000 7fabb2ffd640 1 -- 172.21.15.103:0/405794741 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc003b4f0 con 0x562d47437b40 2024-09-19T06:56:55.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.315+0000 7fabd5713740 1 -- 172.21.15.103:0/405794741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d47437b40 msgr2=0x562d47439180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:55.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.315+0000 7fabd5713740 1 --2- 172.21.15.103:0/405794741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d47437b40 0x562d47439180 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fabc00edc90 tx=0x7fabc003b890 comp rx=0 tx=0).stop 2024-09-19T06:56:55.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.315+0000 7fabd5713740 1 -- 172.21.15.103:0/405794741 shutdown_connections 2024-09-19T06:56:55.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.315+0000 7fabd5713740 1 --2- 172.21.15.103:0/405794741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d47441370 0x562d474396c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.315+0000 7fabd5713740 1 --2- 172.21.15.103:0/405794741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d47437b40 0x562d47439180 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.315+0000 7fabd5713740 1 --2- 172.21.15.103:0/405794741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d474365d0 0x562d4744cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.315+0000 7fabd5713740 1 -- 172.21.15.103:0/405794741 >> 172.21.15.103:0/405794741 conn(0x562d473ff310 msgr2=0x562d4743d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:55.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.315+0000 7fabd5713740 1 -- 172.21.15.103:0/405794741 shutdown_connections 2024-09-19T06:56:55.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.315+0000 7fabd5713740 1 -- 172.21.15.103:0/405794741 wait complete. 2024-09-19T06:56:55.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.315+0000 7fabd5713740 1 -- start start 2024-09-19T06:56:55.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.315+0000 7fabd5713740 1 Processor -- start 2024-09-19T06:56:55.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd5713740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d474365d0 0x562d474497a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd5713740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47437b40 0x562d47444b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd5713740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d47441370 0x562d47445060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd5713740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1aeec0 con 0x562d474365d0 2024-09-19T06:56:55.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd5713740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1ae7f0 con 0x562d47437b40 2024-09-19T06:56:55.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd5713740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1aba50 con 0x562d47441370 2024-09-19T06:56:55.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd5713740 10 client.0 _ll_register_callbacks cb 0x7fabbc00bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:55.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd10d6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d474365d0 0x562d474497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd10d6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d474365d0 0x562d474497a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59968/0 (socket says 172.21.15.103:59968) 2024-09-19T06:56:55.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd10d6640 1 -- 172.21.15.103:0/944636292 learned_addr learned my addr 172.21.15.103:0/944636292 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:55.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd18d7640 1 --2- 172.21.15.103:0/944636292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d47441370 0x562d47445060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd08d5640 1 --2- 172.21.15.103:0/944636292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47437b40 0x562d47444b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd10d6640 1 -- 172.21.15.103:0/944636292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d47441370 msgr2=0x562d47445060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:55.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd10d6640 1 --2- 172.21.15.103:0/944636292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d47441370 0x562d47445060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd10d6640 1 -- 172.21.15.103:0/944636292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47437b40 msgr2=0x562d47444b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:55.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd10d6640 1 --2- 172.21.15.103:0/944636292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47437b40 0x562d47444b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd10d6640 1 -- 172.21.15.103:0/944636292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d47445890 con 0x562d474365d0 2024-09-19T06:56:55.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd08d5640 1 --2- 172.21.15.103:0/944636292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47437b40 0x562d47444b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:55.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd10d6640 1 --2- 172.21.15.103:0/944636292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d474365d0 0x562d474497a0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fabbc000c00 tx=0x7fabbc005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1721d259fe6b3565 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:55.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:55.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabb0ff9640 1 -- 172.21.15.103:0/944636292 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc01ab40 con 0x562d474365d0 2024-09-19T06:56:55.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabb0ff9640 1 -- 172.21.15.103:0/944636292 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fabbc01ace0 con 0x562d474365d0 2024-09-19T06:56:55.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.316+0000 7fabd5713740 1 -- 172.21.15.103:0/944636292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabc00c0b20 con 0x562d474365d0 2024-09-19T06:56:55.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.317+0000 7fabb0ff9640 1 -- 172.21.15.103:0/944636292 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fabbc005070 con 0x562d474365d0 2024-09-19T06:56:55.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.317+0000 7fabd5713740 10 client.18126 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:55.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.317+0000 7fabd5713740 1 -- 172.21.15.103:0/944636292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d474bb9d0 con 0x562d474365d0 2024-09-19T06:56:55.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.317+0000 7fabb0ff9640 1 -- 172.21.15.103:0/944636292 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc00c0b20 con 0x562d474365d0 2024-09-19T06:56:55.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.317+0000 7fabb0ff9640 1 -- 172.21.15.103:0/944636292 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fabbc0035f0 con 0x562d474365d0 2024-09-19T06:56:55.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.317+0000 7fabd5713740 10 client.18126 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:55.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.317+0000 7fabd5713740 10 client.18126 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:56:55.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.317+0000 7fabd5713740 20 client.18126 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:55.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.317+0000 7fabd5713740 1 --2- 172.21.15.103:0/944636292 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x562d47450d90 0x562d474be160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:55.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.317+0000 7fabd5713740 4 client.18126 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:56:55.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.317+0000 7fabd5713740 1 -- 172.21.15.103:0/944636292 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d473a3d70 con 0x562d47450d90 2024-09-19T06:56:55.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.317+0000 7fabd08d5640 1 --2- 172.21.15.103:0/944636292 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x562d47450d90 0x562d474be160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:55.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabd08d5640 1 --2- 172.21.15.103:0/944636292 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x562d47450d90 0x562d474be160 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bbb549f707439421 server_cookie=670efa3a7463805 in_seq=0 out_seq=0 2024-09-19T06:56:55.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabb0ff9640 10 client.18126 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:56:55.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabb0ff9640 1 -- 172.21.15.103:0/944636292 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d473a3d70 con 0x562d47450d90 2024-09-19T06:56:55.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabb0ff9640 10 client.18126 handle_command_reply: tid=0 2024-09-19T06:56:55.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabd5713740 1 client.18126 shutdown 2024-09-19T06:56:55.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabd5713740 1 -- 172.21.15.103:0/944636292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d474365d0 msgr2=0x562d474497a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:55.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabd5713740 1 --2- 172.21.15.103:0/944636292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d474365d0 0x562d474497a0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fabbc000c00 tx=0x7fabbc005b90 comp rx=0 tx=0).stop 2024-09-19T06:56:55.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabd5713740 1 -- 172.21.15.103:0/944636292 shutdown_connections 2024-09-19T06:56:55.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabd5713740 1 --2- 172.21.15.103:0/944636292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d47441370 0x562d47445060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabd5713740 1 --2- 172.21.15.103:0/944636292 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x562d47450d90 0x562d474be160 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabd5713740 1 --2- 172.21.15.103:0/944636292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d47437b40 0x562d47444b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabd5713740 1 --2- 172.21.15.103:0/944636292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d474365d0 0x562d474497a0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabd5713740 1 -- 172.21.15.103:0/944636292 >> 172.21.15.103:0/944636292 conn(0x562d473ff310 msgr2=0x562d474314e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:55.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabb0ff9640 0 client.18126 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:56:55.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabd5713740 1 -- 172.21.15.103:0/944636292 shutdown_connections 2024-09-19T06:56:55.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabd5713740 1 -- 172.21.15.103:0/944636292 wait complete. 2024-09-19T06:56:55.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.318+0000 7fabd5713740 20 client.18126 trim_cache size 0 max 16384 2024-09-19T06:56:55.336 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:56:55.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.319+0000 7fabd333a640 1 -- 172.21.15.103:0/3258052327 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fabb40a33b0 msgr2=0x7fabb40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:55.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.319+0000 7fabd333a640 1 --2- 172.21.15.103:0/3258052327 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fabb40a33b0 0x7fabb40a5870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fabc4005fd0 tx=0x7fabc4009600 comp rx=0 tx=0).stop 2024-09-19T06:56:55.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.319+0000 7fabd333a640 1 -- 172.21.15.103:0/3258052327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabcc100d00 msgr2=0x7fabcc198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:55.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.319+0000 7fabd333a640 1 --2- 172.21.15.103:0/3258052327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabcc100d00 0x7fabcc198a30 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fabc002a940 tx=0x7fabc0003b90 comp rx=0 tx=0).stop 2024-09-19T06:56:55.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.319+0000 7fabd333a640 1 -- 172.21.15.103:0/3258052327 shutdown_connections 2024-09-19T06:56:55.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.319+0000 7fabd333a640 1 --2- 172.21.15.103:0/3258052327 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fabb40a33b0 0x7fabb40a5870 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.319+0000 7fabd333a640 1 --2- 172.21.15.103:0/3258052327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabcc106f50 0x7fabcc19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.319+0000 7fabd333a640 1 --2- 172.21.15.103:0/3258052327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabcc100d00 0x7fabcc198a30 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.319+0000 7fabd333a640 1 --2- 172.21.15.103:0/3258052327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabcc100350 0x7fabcc1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:55.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.319+0000 7fabd333a640 1 -- 172.21.15.103:0/3258052327 >> 172.21.15.103:0/3258052327 conn(0x7fabcc077210 msgr2=0x7fabcc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:55.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.319+0000 7fabd333a640 1 -- 172.21.15.103:0/3258052327 shutdown_connections 2024-09-19T06:56:55.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:55.319+0000 7fabd333a640 1 -- 172.21.15.103:0/3258052327 wait complete. 2024-09-19T06:56:56.038 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:56:56.288 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:56.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:56 smithi103 ceph-mon[31158]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-09-19T06:56:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:56 smithi134 ceph-mon[35356]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-09-19T06:56:56.582 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:56:56.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:56 smithi174 ceph-mon[35637]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-09-19T06:56:56.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.777+0000 7fa4a849e640 1 -- 172.21.15.103:0/885092189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4a0106a70 msgr2=0x7fa4a0106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:56.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.777+0000 7fa4a849e640 1 --2- 172.21.15.103:0/885092189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4a0106a70 0x7fa4a0106e50 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009930 tx=0x7fa49c02a420 comp rx=0 tx=0).stop 2024-09-19T06:56:56.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.777+0000 7fa4a849e640 1 -- 172.21.15.103:0/885092189 shutdown_connections 2024-09-19T06:56:56.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.777+0000 7fa4a849e640 1 --2- 172.21.15.103:0/885092189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4a0101410 0x7fa4a010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.777+0000 7fa4a849e640 1 --2- 172.21.15.103:0/885092189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4a0100a70 0x7fa4a0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.777+0000 7fa4a849e640 1 --2- 172.21.15.103:0/885092189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4a0106a70 0x7fa4a0106e50 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.777+0000 7fa4a849e640 1 -- 172.21.15.103:0/885092189 >> 172.21.15.103:0/885092189 conn(0x7fa4a00fc790 msgr2=0x7fa4a00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:56.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.777+0000 7fa4a849e640 1 -- 172.21.15.103:0/885092189 shutdown_connections 2024-09-19T06:56:56.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.777+0000 7fa4a849e640 1 -- 172.21.15.103:0/885092189 wait complete. 2024-09-19T06:56:56.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.777+0000 7fa4a849e640 1 Processor -- start 2024-09-19T06:56:56.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.777+0000 7fa4a849e640 1 -- start start 2024-09-19T06:56:56.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.778+0000 7fa4a849e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4a0100a70 0x7fa4a0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.778+0000 7fa4a849e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4a0101410 0x7fa4a01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.778+0000 7fa4a849e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4a0106a70 0x7fa4a019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.778+0000 7fa4a849e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a0110e70 con 0x7fa4a0106a70 2024-09-19T06:56:56.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.778+0000 7fa4a849e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a0110cf0 con 0x7fa4a0100a70 2024-09-19T06:56:56.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.778+0000 7fa4a849e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a0110ff0 con 0x7fa4a0101410 2024-09-19T06:56:56.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.778+0000 7fa4a6a3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4a0106a70 0x7fa4a019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.778+0000 7fa4a6a3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4a0106a70 0x7fa4a019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59980/0 (socket says 172.21.15.103:59980) 2024-09-19T06:56:56.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.778+0000 7fa4a6a3b640 1 -- 172.21.15.103:0/4050254929 learned_addr learned my addr 172.21.15.103:0/4050254929 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:56.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.778+0000 7fa4a5a39640 1 --2- 172.21.15.103:0/4050254929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4a0101410 0x7fa4a01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.778+0000 7fa4a623a640 1 --2- 172.21.15.103:0/4050254929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4a0100a70 0x7fa4a0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.778+0000 7fa4a6a3b640 1 -- 172.21.15.103:0/4050254929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4a0101410 msgr2=0x7fa4a01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:56.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.778+0000 7fa4a6a3b640 1 --2- 172.21.15.103:0/4050254929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4a0101410 0x7fa4a01989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.778+0000 7fa4a6a3b640 1 -- 172.21.15.103:0/4050254929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4a0100a70 msgr2=0x7fa4a0198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:56.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.778+0000 7fa4a6a3b640 1 --2- 172.21.15.103:0/4050254929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4a0100a70 0x7fa4a0198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.778+0000 7fa4a6a3b640 1 -- 172.21.15.103:0/4050254929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a019d430 con 0x7fa4a0106a70 2024-09-19T06:56:56.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.779+0000 7fa4a6a3b640 1 --2- 172.21.15.103:0/4050254929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4a0106a70 0x7fa4a019cd50 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fa49800cad0 tx=0x7fa49800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:56.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.779+0000 7fa4977fe640 1 -- 172.21.15.103:0/4050254929 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49800ae20 con 0x7fa4a0106a70 2024-09-19T06:56:56.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.779+0000 7fa4977fe640 1 -- 172.21.15.103:0/4050254929 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa498005020 con 0x7fa4a0106a70 2024-09-19T06:56:56.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.779+0000 7fa4977fe640 1 -- 172.21.15.103:0/4050254929 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49800f6f0 con 0x7fa4a0106a70 2024-09-19T06:56:56.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.779+0000 7fa4a849e640 1 -- 172.21.15.103:0/4050254929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a019d720 con 0x7fa4a0106a70 2024-09-19T06:56:56.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.779+0000 7fa4a849e640 1 -- 172.21.15.103:0/4050254929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a01a5000 con 0x7fa4a0106a70 2024-09-19T06:56:56.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.781+0000 7fa4977fe640 1 -- 172.21.15.103:0/4050254929 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa4980051d0 con 0x7fa4a0106a70 2024-09-19T06:56:56.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.782+0000 7fa4977fe640 1 --2- 172.21.15.103:0/4050254929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa48c0a3430 0x7fa48c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.782+0000 7fa4a623a640 1 --2- 172.21.15.103:0/4050254929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa48c0a3430 0x7fa48c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.782+0000 7fa4977fe640 1 -- 172.21.15.103:0/4050254929 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa4980cfc10 con 0x7fa4a0106a70 2024-09-19T06:56:56.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.782+0000 7fa4aa877740 1 Processor -- start 2024-09-19T06:56:56.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.782+0000 7fa4aa877740 1 -- start start 2024-09-19T06:56:56.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4a623a640 1 --2- 172.21.15.103:0/4050254929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa48c0a3430 0x7fa48c0a58f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009900 tx=0x7fa49c0058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:56.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4aa877740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c868b40 0x56373c867090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4aa877740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c8675d0 0x56373c867a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4aa877740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c872370 0x56373c874760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4aa877740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56373c7cd2f0 con 0x56373c872370 2024-09-19T06:56:56.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4aa877740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56373c774cf0 con 0x56373c8675d0 2024-09-19T06:56:56.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4aa877740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56373c7c92b0 con 0x56373c868b40 2024-09-19T06:56:56.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4a6a3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c872370 0x56373c874760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4a6a3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c872370 0x56373c874760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59994/0 (socket says 172.21.15.103:59994) 2024-09-19T06:56:56.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4a6a3b640 1 -- 172.21.15.103:0/2355623948 learned_addr learned my addr 172.21.15.103:0/2355623948 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:56.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4a623a640 1 --2- 172.21.15.103:0/2355623948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c868b40 0x56373c867090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4a5a39640 1 --2- 172.21.15.103:0/2355623948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c8675d0 0x56373c867a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4a6a3b640 1 -- 172.21.15.103:0/2355623948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c868b40 msgr2=0x56373c867090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:56.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4a6a3b640 1 --2- 172.21.15.103:0/2355623948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c868b40 0x56373c867090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4a6a3b640 1 -- 172.21.15.103:0/2355623948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c8675d0 msgr2=0x56373c867a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:56.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4a6a3b640 1 --2- 172.21.15.103:0/2355623948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c8675d0 0x56373c867a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4a6a3b640 1 -- 172.21.15.103:0/2355623948 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56373c7593a0 con 0x56373c872370 2024-09-19T06:56:56.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.783+0000 7fa4a6a3b640 1 --2- 172.21.15.103:0/2355623948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c872370 0x56373c874760 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fa4980983e0 tx=0x7fa49809bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155e78e5852dc11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:56.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.784+0000 7fa494ff9640 1 -- 172.21.15.103:0/2355623948 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49801d040 con 0x56373c872370 2024-09-19T06:56:56.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.784+0000 7fa494ff9640 1 -- 172.21.15.103:0/2355623948 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4980cd5c0 con 0x56373c872370 2024-09-19T06:56:56.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.784+0000 7fa494ff9640 1 -- 172.21.15.103:0/2355623948 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49809b3d0 con 0x56373c872370 2024-09-19T06:56:56.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.784+0000 7fa4aa877740 1 -- 172.21.15.103:0/2355623948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c872370 msgr2=0x56373c874760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:56.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.784+0000 7fa4aa877740 1 --2- 172.21.15.103:0/2355623948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c872370 0x56373c874760 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fa4980983e0 tx=0x7fa49809bbf0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.784+0000 7fa4aa877740 1 -- 172.21.15.103:0/2355623948 shutdown_connections 2024-09-19T06:56:56.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.784+0000 7fa4aa877740 1 --2- 172.21.15.103:0/2355623948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c872370 0x56373c874760 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.784+0000 7fa4aa877740 1 --2- 172.21.15.103:0/2355623948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c8675d0 0x56373c867a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.784+0000 7fa4aa877740 1 --2- 172.21.15.103:0/2355623948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c868b40 0x56373c867090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.784+0000 7fa4aa877740 1 -- 172.21.15.103:0/2355623948 >> 172.21.15.103:0/2355623948 conn(0x56373c830310 msgr2=0x56373c863c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:56.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.785+0000 7fa4aa877740 1 -- 172.21.15.103:0/2355623948 shutdown_connections 2024-09-19T06:56:56.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.785+0000 7fa4aa877740 1 -- 172.21.15.103:0/2355623948 wait complete. 2024-09-19T06:56:56.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.785+0000 7fa4aa877740 1 -- start start 2024-09-19T06:56:56.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.785+0000 7fa4aa877740 1 Processor -- start 2024-09-19T06:56:56.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.785+0000 7fa4aa877740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c8675d0 0x56373c871db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.785+0000 7fa4aa877740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c868b40 0x56373c86f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.785+0000 7fa4aa877740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c872370 0x56373c86f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.785+0000 7fa4aa877740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56373c875070 con 0x56373c8675d0 2024-09-19T06:56:56.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.785+0000 7fa4aa877740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56373c788060 con 0x56373c872370 2024-09-19T06:56:56.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.785+0000 7fa4aa877740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56373c8751f0 con 0x56373c868b40 2024-09-19T06:56:56.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.785+0000 7fa4aa877740 10 client.0 _ll_register_callbacks cb 0x56373c4db8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:56.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.785+0000 7fa4a623a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c8675d0 0x56373c871db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.786+0000 7fa4a623a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c8675d0 0x56373c871db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59998/0 (socket says 172.21.15.103:59998) 2024-09-19T06:56:56.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.786+0000 7fa4a623a640 1 -- 172.21.15.103:0/4100918413 learned_addr learned my addr 172.21.15.103:0/4100918413 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:56.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.786+0000 7fa4a5a39640 1 --2- 172.21.15.103:0/4100918413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c868b40 0x56373c86f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.786+0000 7fa4a6a3b640 1 --2- 172.21.15.103:0/4100918413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c872370 0x56373c86f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.786+0000 7fa4a623a640 1 -- 172.21.15.103:0/4100918413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c868b40 msgr2=0x56373c86f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:56.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.786+0000 7fa4a623a640 1 --2- 172.21.15.103:0/4100918413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c868b40 0x56373c86f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.786+0000 7fa4a623a640 1 -- 172.21.15.103:0/4100918413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c872370 msgr2=0x56373c86f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:56.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.786+0000 7fa4a623a640 1 --2- 172.21.15.103:0/4100918413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c872370 0x56373c86f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.786+0000 7fa4a623a640 1 -- 172.21.15.103:0/4100918413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56373c53e3c0 con 0x56373c8675d0 2024-09-19T06:56:56.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.786+0000 7fa4a5a39640 1 --2- 172.21.15.103:0/4100918413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c868b40 0x56373c86f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:56:56.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.786+0000 7fa4a623a640 1 --2- 172.21.15.103:0/4100918413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c8675d0 0x56373c871db0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009930 tx=0x7fa49c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e83d88569eaa0b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:56.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.786+0000 7fa489ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:56.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.786+0000 7fa489ffb640 1 -- 172.21.15.103:0/4100918413 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c035040 con 0x56373c8675d0 2024-09-19T06:56:56.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.786+0000 7fa489ffb640 1 -- 172.21.15.103:0/4100918413 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa49c04cb70 con 0x56373c8675d0 2024-09-19T06:56:56.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.786+0000 7fa4aa877740 1 -- 172.21.15.103:0/4100918413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4980cd5c0 con 0x56373c8675d0 2024-09-19T06:56:56.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.787+0000 7fa489ffb640 1 -- 172.21.15.103:0/4100918413 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa49c04cd70 con 0x56373c8675d0 2024-09-19T06:56:56.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.787+0000 7fa4aa877740 10 client.18138 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:56.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.787+0000 7fa4aa877740 1 -- 172.21.15.103:0/4100918413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56373c880b60 con 0x56373c8675d0 2024-09-19T06:56:56.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.787+0000 7fa489ffb640 1 -- 172.21.15.103:0/4100918413 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4980cd5c0 con 0x56373c8675d0 2024-09-19T06:56:56.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.787+0000 7fa489ffb640 1 -- 172.21.15.103:0/4100918413 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa49c0556d0 con 0x56373c8675d0 2024-09-19T06:56:56.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.787+0000 7fa4aa877740 10 client.18138 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:56.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.787+0000 7fa4aa877740 10 client.18138 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:56:56.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.787+0000 7fa4aa877740 20 client.18138 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:56.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.787+0000 7fa4aa877740 1 --2- 172.21.15.103:0/4100918413 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56373c882020 0x56373c8844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.787+0000 7fa4aa877740 4 client.18138 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:56.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.787+0000 7fa4aa877740 1 -- 172.21.15.103:0/4100918413 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56373c7da860 con 0x56373c882020 2024-09-19T06:56:56.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.787+0000 7fa4a5a39640 1 --2- 172.21.15.103:0/4100918413 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56373c882020 0x56373c8844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.788+0000 7fa489ffb640 10 client.18138 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:56:56.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.788+0000 7fa4a5a39640 1 --2- 172.21.15.103:0/4100918413 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56373c882020 0x56373c8844c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=68720c6a0d538805 server_cookie=8de37b4a6bdb201f in_seq=0 out_seq=0 2024-09-19T06:56:56.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.789+0000 7fa489ffb640 1 -- 172.21.15.103:0/4100918413 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56373c7da860 con 0x56373c882020 2024-09-19T06:56:56.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.789+0000 7fa489ffb640 10 client.18138 handle_command_reply: tid=0 2024-09-19T06:56:56.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.789+0000 7fa4aa877740 1 client.18138 shutdown 2024-09-19T06:56:56.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.789+0000 7fa4aa877740 1 -- 172.21.15.103:0/4100918413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c8675d0 msgr2=0x56373c871db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:56.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.789+0000 7fa4aa877740 1 --2- 172.21.15.103:0/4100918413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c8675d0 0x56373c871db0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009930 tx=0x7fa49c009260 comp rx=0 tx=0).stop 2024-09-19T06:56:56.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.789+0000 7fa4aa877740 1 -- 172.21.15.103:0/4100918413 shutdown_connections 2024-09-19T06:56:56.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.789+0000 7fa4aa877740 1 --2- 172.21.15.103:0/4100918413 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56373c882020 0x56373c8844c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.789+0000 7fa4aa877740 1 --2- 172.21.15.103:0/4100918413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c872370 0x56373c86f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.789+0000 7fa4aa877740 1 --2- 172.21.15.103:0/4100918413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c868b40 0x56373c86f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.789+0000 7fa4aa877740 1 --2- 172.21.15.103:0/4100918413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c8675d0 0x56373c871db0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.789+0000 7fa4aa877740 1 -- 172.21.15.103:0/4100918413 >> 172.21.15.103:0/4100918413 conn(0x56373c830310 msgr2=0x56373c869990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:56.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.789+0000 7fa489ffb640 0 client.18138 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:56:56.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.789+0000 7fa4aa877740 1 -- 172.21.15.103:0/4100918413 shutdown_connections 2024-09-19T06:56:56.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.789+0000 7fa4aa877740 1 -- 172.21.15.103:0/4100918413 wait complete. 2024-09-19T06:56:56.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.789+0000 7fa4aa877740 20 client.18138 trim_cache size 0 max 16384 2024-09-19T06:56:56.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.803+0000 7fa4aa877740 1 Processor -- start 2024-09-19T06:56:56.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.803+0000 7fa4aa877740 1 -- start start 2024-09-19T06:56:56.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.803+0000 7fa4aa877740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c8675d0 0x56373c863e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.803+0000 7fa4aa877740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c868b40 0x56373c889d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.803+0000 7fa4aa877740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c872370 0x56373c88a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.803+0000 7fa4aa877740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a01ab7b0 con 0x56373c872370 2024-09-19T06:56:56.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.803+0000 7fa4aa877740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a01ad2f0 con 0x56373c868b40 2024-09-19T06:56:56.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.803+0000 7fa4aa877740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a01abe80 con 0x56373c8675d0 2024-09-19T06:56:56.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.803+0000 7fa4a6a3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c872370 0x56373c88a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.803+0000 7fa4a6a3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c872370 0x56373c88a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60008/0 (socket says 172.21.15.103:60008) 2024-09-19T06:56:56.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.803+0000 7fa4a6a3b640 1 -- 172.21.15.103:0/894060815 learned_addr learned my addr 172.21.15.103:0/894060815 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:56.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.803+0000 7fa4a623a640 1 --2- 172.21.15.103:0/894060815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c8675d0 0x56373c863e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.803+0000 7fa4a6a3b640 1 -- 172.21.15.103:0/894060815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c8675d0 msgr2=0x56373c863e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:56.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.803+0000 7fa4a5a39640 1 --2- 172.21.15.103:0/894060815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c868b40 0x56373c889d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.803+0000 7fa4a6a3b640 1 --2- 172.21.15.103:0/894060815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c8675d0 0x56373c863e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.803+0000 7fa4a6a3b640 1 -- 172.21.15.103:0/894060815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c868b40 msgr2=0x56373c889d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:56.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.804+0000 7fa4a6a3b640 1 --2- 172.21.15.103:0/894060815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c868b40 0x56373c889d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.804+0000 7fa4a6a3b640 1 -- 172.21.15.103:0/894060815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56373c86fff0 con 0x56373c872370 2024-09-19T06:56:56.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.804+0000 7fa4a623a640 1 --2- 172.21.15.103:0/894060815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c8675d0 0x56373c863e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:56.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.804+0000 7fa4a5a39640 1 --2- 172.21.15.103:0/894060815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c868b40 0x56373c889d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:56.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.804+0000 7fa4a6a3b640 1 --2- 172.21.15.103:0/894060815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c872370 0x56373c88a2d0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fa49800a6e0 tx=0x7fa4980a0400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ee1d48efd18cb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:56.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.804+0000 7fa48bfff640 1 -- 172.21.15.103:0/894060815 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49807f070 con 0x56373c872370 2024-09-19T06:56:56.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.804+0000 7fa48bfff640 1 -- 172.21.15.103:0/894060815 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4980985f0 con 0x56373c872370 2024-09-19T06:56:56.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.804+0000 7fa48bfff640 1 -- 172.21.15.103:0/894060815 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4980b7070 con 0x56373c872370 2024-09-19T06:56:56.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.805+0000 7fa4aa877740 1 -- 172.21.15.103:0/894060815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c872370 msgr2=0x56373c88a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.805+0000 7fa4aa877740 1 --2- 172.21.15.103:0/894060815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c872370 0x56373c88a2d0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fa49800a6e0 tx=0x7fa4980a0400 comp rx=0 tx=0).stop 2024-09-19T06:56:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.805+0000 7fa4aa877740 1 -- 172.21.15.103:0/894060815 shutdown_connections 2024-09-19T06:56:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.805+0000 7fa4aa877740 1 --2- 172.21.15.103:0/894060815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c872370 0x56373c88a2d0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.805+0000 7fa4aa877740 1 --2- 172.21.15.103:0/894060815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c868b40 0x56373c889d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.805+0000 7fa4aa877740 1 --2- 172.21.15.103:0/894060815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c8675d0 0x56373c863e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.805+0000 7fa4aa877740 1 -- 172.21.15.103:0/894060815 >> 172.21.15.103:0/894060815 conn(0x56373c830310 msgr2=0x56373c863700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.805+0000 7fa4aa877740 1 -- 172.21.15.103:0/894060815 shutdown_connections 2024-09-19T06:56:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.805+0000 7fa4aa877740 1 -- 172.21.15.103:0/894060815 wait complete. 2024-09-19T06:56:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.805+0000 7fa4aa877740 1 -- start start 2024-09-19T06:56:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.805+0000 7fa4aa877740 1 Processor -- start 2024-09-19T06:56:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.806+0000 7fa4aa877740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c8675d0 0x56373c8759e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.806+0000 7fa4aa877740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c868b40 0x56373c875f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.806+0000 7fa4aa877740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c872370 0x56373c8f3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.806+0000 7fa4aa877740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a01acc20 con 0x56373c868b40 2024-09-19T06:56:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.806+0000 7fa4aa877740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a01ac550 con 0x56373c8675d0 2024-09-19T06:56:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.806+0000 7fa4a5a39640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c868b40 0x56373c875f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.806+0000 7fa4aa877740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a01a97b0 con 0x56373c872370 2024-09-19T06:56:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.806+0000 7fa4a5a39640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c868b40 0x56373c875f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60022/0 (socket says 172.21.15.103:60022) 2024-09-19T06:56:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.806+0000 7fa4a5a39640 1 -- 172.21.15.103:0/3487461430 learned_addr learned my addr 172.21.15.103:0/3487461430 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.806+0000 7fa4a623a640 1 --2- 172.21.15.103:0/3487461430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c8675d0 0x56373c8759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.806+0000 7fa4aa877740 10 client.0 _ll_register_callbacks cb 0x7fa49c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.806+0000 7fa4a6a3b640 1 --2- 172.21.15.103:0/3487461430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c872370 0x56373c8f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.807+0000 7fa4a5a39640 1 -- 172.21.15.103:0/3487461430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c872370 msgr2=0x56373c8f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.807+0000 7fa4a5a39640 1 --2- 172.21.15.103:0/3487461430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c872370 0x56373c8f3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.807+0000 7fa4a5a39640 1 -- 172.21.15.103:0/3487461430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c8675d0 msgr2=0x56373c8759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:56.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.807+0000 7fa4a5a39640 1 --2- 172.21.15.103:0/3487461430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c8675d0 0x56373c8759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.807+0000 7fa4a5a39640 1 -- 172.21.15.103:0/3487461430 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56373c8f3aa0 con 0x56373c868b40 2024-09-19T06:56:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.807+0000 7fa4a5a39640 1 --2- 172.21.15.103:0/3487461430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c868b40 0x56373c875f20 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x56373c864ac0 tx=0x7fa49000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8e28aa2bead3ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.807+0000 7fa489ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.807+0000 7fa489ffb640 1 -- 172.21.15.103:0/3487461430 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490005220 con 0x56373c868b40 2024-09-19T06:56:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.807+0000 7fa489ffb640 1 -- 172.21.15.103:0/3487461430 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa4900053c0 con 0x56373c868b40 2024-09-19T06:56:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.807+0000 7fa4aa877740 1 -- 172.21.15.103:0/3487461430 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4980985f0 con 0x56373c868b40 2024-09-19T06:56:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.807+0000 7fa489ffb640 1 -- 172.21.15.103:0/3487461430 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4900055c0 con 0x56373c868b40 2024-09-19T06:56:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.807+0000 7fa4aa877740 10 client.18144 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.807+0000 7fa4aa877740 1 -- 172.21.15.103:0/3487461430 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56373c8f4d20 con 0x56373c868b40 2024-09-19T06:56:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.808+0000 7fa489ffb640 1 -- 172.21.15.103:0/3487461430 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa490004c80 con 0x56373c868b40 2024-09-19T06:56:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.808+0000 7fa489ffb640 1 -- 172.21.15.103:0/3487461430 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa490011a40 con 0x56373c868b40 2024-09-19T06:56:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.808+0000 7fa4aa877740 10 client.18144 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.808+0000 7fa4aa877740 10 client.18144 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:56:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.808+0000 7fa4aa877740 20 client.18144 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:56.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.808+0000 7fa4aa877740 1 --2- 172.21.15.103:0/3487461430 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56373c8f5520 0x56373c8f7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.808+0000 7fa4aa877740 4 client.18144 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:56:56.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.808+0000 7fa4aa877740 1 -- 172.21.15.103:0/3487461430 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa4980985f0 con 0x56373c8f5520 2024-09-19T06:56:56.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.808+0000 7fa4a623a640 1 --2- 172.21.15.103:0/3487461430 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56373c8f5520 0x56373c8f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa4a623a640 1 --2- 172.21.15.103:0/3487461430 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56373c8f5520 0x56373c8f7940 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=761bf4ea726a8cae server_cookie=aca424686616bdfc in_seq=0 out_seq=0 2024-09-19T06:56:56.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa489ffb640 10 client.18144 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:56:56.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa489ffb640 1 -- 172.21.15.103:0/3487461430 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa4980985f0 con 0x56373c8f5520 2024-09-19T06:56:56.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa489ffb640 10 client.18144 handle_command_reply: tid=0 2024-09-19T06:56:56.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa4aa877740 1 client.18144 shutdown 2024-09-19T06:56:56.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa4aa877740 1 -- 172.21.15.103:0/3487461430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c868b40 msgr2=0x56373c875f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:56.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa4aa877740 1 --2- 172.21.15.103:0/3487461430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c868b40 0x56373c875f20 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x56373c864ac0 tx=0x7fa49000bf10 comp rx=0 tx=0).stop 2024-09-19T06:56:56.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa4aa877740 1 -- 172.21.15.103:0/3487461430 shutdown_connections 2024-09-19T06:56:56.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa4aa877740 1 --2- 172.21.15.103:0/3487461430 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x56373c8f5520 0x56373c8f7940 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa4aa877740 1 --2- 172.21.15.103:0/3487461430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373c872370 0x56373c8f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa4aa877740 1 --2- 172.21.15.103:0/3487461430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56373c868b40 0x56373c875f20 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa4aa877740 1 --2- 172.21.15.103:0/3487461430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56373c8675d0 0x56373c8759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa489ffb640 0 client.18144 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:56:56.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa4aa877740 1 -- 172.21.15.103:0/3487461430 >> 172.21.15.103:0/3487461430 conn(0x56373c830310 msgr2=0x56373c86e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:56.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa4aa877740 1 -- 172.21.15.103:0/3487461430 shutdown_connections 2024-09-19T06:56:56.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa4aa877740 1 -- 172.21.15.103:0/3487461430 wait complete. 2024-09-19T06:56:56.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.809+0000 7fa4aa877740 20 client.18144 trim_cache size 0 max 16384 2024-09-19T06:56:56.815 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:56:56.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.810+0000 7fa4a849e640 1 -- 172.21.15.103:0/4050254929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa48c0a3430 msgr2=0x7fa48c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:56.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.810+0000 7fa4a849e640 1 --2- 172.21.15.103:0/4050254929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa48c0a3430 0x7fa48c0a58f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009900 tx=0x7fa49c0058d0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.810+0000 7fa4a849e640 1 -- 172.21.15.103:0/4050254929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4a0106a70 msgr2=0x7fa4a019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:56.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.810+0000 7fa4a849e640 1 --2- 172.21.15.103:0/4050254929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4a0106a70 0x7fa4a019cd50 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fa49800cad0 tx=0x7fa49800cf90 comp rx=0 tx=0).stop 2024-09-19T06:56:56.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.810+0000 7fa4a849e640 1 -- 172.21.15.103:0/4050254929 shutdown_connections 2024-09-19T06:56:56.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.810+0000 7fa4a849e640 1 --2- 172.21.15.103:0/4050254929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa48c0a3430 0x7fa48c0a58f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.810+0000 7fa4a849e640 1 --2- 172.21.15.103:0/4050254929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4a0106a70 0x7fa4a019cd50 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.810+0000 7fa4a849e640 1 --2- 172.21.15.103:0/4050254929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4a0101410 0x7fa4a01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.810+0000 7fa4a849e640 1 --2- 172.21.15.103:0/4050254929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4a0100a70 0x7fa4a0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.810+0000 7fa4a849e640 1 -- 172.21.15.103:0/4050254929 >> 172.21.15.103:0/4050254929 conn(0x7fa4a00fc790 msgr2=0x7fa4a01048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:56.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.810+0000 7fa4a849e640 1 -- 172.21.15.103:0/4050254929 shutdown_connections 2024-09-19T06:56:56.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:56.810+0000 7fa4a849e640 1 -- 172.21.15.103:0/4050254929 wait complete. 2024-09-19T06:56:57.078 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:57.500 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:56:57.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.965+0000 7f7c4a3d1640 1 -- 172.21.15.103:0/2891287493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c44106a80 msgr2=0x7f7c44106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:57.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.965+0000 7f7c4a3d1640 1 --2- 172.21.15.103:0/2891287493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c44106a80 0x7f7c44106e60 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009930 tx=0x7f7c3402a420 comp rx=0 tx=0).stop 2024-09-19T06:56:57.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.965+0000 7f7c4a3d1640 1 -- 172.21.15.103:0/2891287493 shutdown_connections 2024-09-19T06:56:57.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.965+0000 7f7c4a3d1640 1 --2- 172.21.15.103:0/2891287493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c44101420 0x7f7c4410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:57.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.965+0000 7f7c4a3d1640 1 --2- 172.21.15.103:0/2891287493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c44100a80 0x7f7c44100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:57.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.965+0000 7f7c4a3d1640 1 --2- 172.21.15.103:0/2891287493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c44106a80 0x7f7c44106e60 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:57.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.965+0000 7f7c4a3d1640 1 -- 172.21.15.103:0/2891287493 >> 172.21.15.103:0/2891287493 conn(0x7f7c440fc7a0 msgr2=0x7f7c440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:57.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.965+0000 7f7c4a3d1640 1 -- 172.21.15.103:0/2891287493 shutdown_connections 2024-09-19T06:56:57.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.966+0000 7f7c4a3d1640 1 -- 172.21.15.103:0/2891287493 wait complete. 2024-09-19T06:56:57.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.966+0000 7f7c4a3d1640 1 Processor -- start 2024-09-19T06:56:57.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.966+0000 7f7c4a3d1640 1 -- start start 2024-09-19T06:56:57.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.966+0000 7f7c4a3d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c44100a80 0x7f7c441972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:57.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.966+0000 7f7c4a3d1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c44101420 0x7f7c44197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:57.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.966+0000 7f7c4a3d1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c44106a80 0x7f7c4419bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:57.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.966+0000 7f7c4a3d1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c44110e80 con 0x7f7c44106a80 2024-09-19T06:56:57.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.966+0000 7f7c4a3d1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c44110d00 con 0x7f7c44101420 2024-09-19T06:56:57.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.966+0000 7f7c4a3d1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c44111000 con 0x7f7c44100a80 2024-09-19T06:56:57.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.967+0000 7f7c4896e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c44106a80 0x7f7c4419bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:57.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.967+0000 7f7c4896e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c44106a80 0x7f7c4419bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45404/0 (socket says 172.21.15.103:45404) 2024-09-19T06:56:57.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.967+0000 7f7c4896e640 1 -- 172.21.15.103:0/800270622 learned_addr learned my addr 172.21.15.103:0/800270622 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:57.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.967+0000 7f7c43fff640 1 --2- 172.21.15.103:0/800270622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c44100a80 0x7f7c441972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:57.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.967+0000 7f7c3bfff640 1 --2- 172.21.15.103:0/800270622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c44101420 0x7f7c44197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:57.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.967+0000 7f7c4896e640 1 -- 172.21.15.103:0/800270622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c44100a80 msgr2=0x7f7c441972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:57.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.967+0000 7f7c4896e640 1 --2- 172.21.15.103:0/800270622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c44100a80 0x7f7c441972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:57.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.967+0000 7f7c4896e640 1 -- 172.21.15.103:0/800270622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c44101420 msgr2=0x7f7c44197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:57.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.967+0000 7f7c4896e640 1 --2- 172.21.15.103:0/800270622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c44101420 0x7f7c44197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:57.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.967+0000 7f7c4896e640 1 -- 172.21.15.103:0/800270622 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4419c290 con 0x7f7c44106a80 2024-09-19T06:56:57.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.967+0000 7f7c4896e640 1 --2- 172.21.15.103:0/800270622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c44106a80 0x7f7c4419bbb0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00eaa0 tx=0x7f7c3c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:57.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.967+0000 7f7c41ffb640 1 -- 172.21.15.103:0/800270622 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c004d20 con 0x7f7c44106a80 2024-09-19T06:56:57.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.967+0000 7f7c41ffb640 1 -- 172.21.15.103:0/800270622 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3c005020 con 0x7f7c44106a80 2024-09-19T06:56:57.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.967+0000 7f7c4a3d1640 1 -- 172.21.15.103:0/800270622 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4419c580 con 0x7f7c44106a80 2024-09-19T06:56:57.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.967+0000 7f7c41ffb640 1 -- 172.21.15.103:0/800270622 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c0116d0 con 0x7f7c44106a80 2024-09-19T06:56:57.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.967+0000 7f7c4a3d1640 1 -- 172.21.15.103:0/800270622 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c441a3e60 con 0x7f7c44106a80 2024-09-19T06:56:57.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.968+0000 7f7c4a3d1640 1 -- 172.21.15.103:0/800270622 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c44100ee0 con 0x7f7c44106a80 2024-09-19T06:56:57.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.969+0000 7f7c41ffb640 1 -- 172.21.15.103:0/800270622 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7c3c0051d0 con 0x7f7c44106a80 2024-09-19T06:56:57.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.970+0000 7f7c41ffb640 1 --2- 172.21.15.103:0/800270622 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c300a33b0 0x7f7c300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:57.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.970+0000 7f7c41ffb640 1 -- 172.21.15.103:0/800270622 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f7c3c0d0220 con 0x7f7c44106a80 2024-09-19T06:56:57.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.970+0000 7f7c43fff640 1 --2- 172.21.15.103:0/800270622 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c300a33b0 0x7f7c300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:57.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.970+0000 7f7c43fff640 1 --2- 172.21.15.103:0/800270622 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c300a33b0 0x7f7c300a5870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009900 tx=0x7f7c340058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:57.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:57.973+0000 7f7c41ffb640 1 -- 172.21.15.103:0/800270622 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7c3c016030 con 0x7f7c44106a80 2024-09-19T06:56:58.032 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:58.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:58.187+0000 7f7c4a3d1640 1 -- 172.21.15.103:0/800270622 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c441a4120 con 0x7f7c44106a80 2024-09-19T06:56:58.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:58.187+0000 7f7c41ffb640 1 -- 172.21.15.103:0/800270622 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f7c3c09cbe0 con 0x7f7c44106a80 2024-09-19T06:56:58.189 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:56:58.189 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:56:58.189 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:56:58.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:58.190+0000 7f7c4a3d1640 1 -- 172.21.15.103:0/800270622 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c300a33b0 msgr2=0x7f7c300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:58.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:58.190+0000 7f7c4a3d1640 1 --2- 172.21.15.103:0/800270622 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c300a33b0 0x7f7c300a5870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009900 tx=0x7f7c340058d0 comp rx=0 tx=0).stop 2024-09-19T06:56:58.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:58.190+0000 7f7c4a3d1640 1 -- 172.21.15.103:0/800270622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c44106a80 msgr2=0x7f7c4419bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:58.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:58.190+0000 7f7c4a3d1640 1 --2- 172.21.15.103:0/800270622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c44106a80 0x7f7c4419bbb0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00eaa0 tx=0x7f7c3c00eea0 comp rx=0 tx=0).stop 2024-09-19T06:56:58.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:58.190+0000 7f7c4a3d1640 1 -- 172.21.15.103:0/800270622 shutdown_connections 2024-09-19T06:56:58.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:58.190+0000 7f7c4a3d1640 1 --2- 172.21.15.103:0/800270622 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c300a33b0 0x7f7c300a5870 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:58.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:58.190+0000 7f7c4a3d1640 1 --2- 172.21.15.103:0/800270622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c44106a80 0x7f7c4419bbb0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:58.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:58.190+0000 7f7c4a3d1640 1 --2- 172.21.15.103:0/800270622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c44101420 0x7f7c44197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:58.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:58.190+0000 7f7c4a3d1640 1 --2- 172.21.15.103:0/800270622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c44100a80 0x7f7c441972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:58.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:58.190+0000 7f7c4a3d1640 1 -- 172.21.15.103:0/800270622 >> 172.21.15.103:0/800270622 conn(0x7f7c440fc7a0 msgr2=0x7f7c440fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:58.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:58.190+0000 7f7c4a3d1640 1 -- 172.21.15.103:0/800270622 shutdown_connections 2024-09-19T06:56:58.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:58.190+0000 7f7c4a3d1640 1 -- 172.21.15.103:0/800270622 wait complete. 2024-09-19T06:56:58.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:58 smithi103 ceph-mon[31158]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:56:58.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:58 smithi134 ceph-mon[35356]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:56:58.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:58 smithi174 ceph-mon[35637]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:56:58.930 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id f2ce1f2b --release --await' 2024-09-19T06:56:58.930 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id f2ce1f2b --release --await 2024-09-19T06:56:59.180 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:56:59 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/800270622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:59.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.178+0000 7f2d60c44640 1 -- 172.21.15.103:0/222126151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d5c10c170 msgr2=0x7f2d5c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:59.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.178+0000 7f2d60c44640 1 --2- 172.21.15.103:0/222126151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d5c10c170 0x7f2d5c10e610 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009ca0 tx=0x7f2d5402a950 comp rx=0 tx=0).stop 2024-09-19T06:56:59.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.179+0000 7f2d60c44640 1 -- 172.21.15.103:0/222126151 shutdown_connections 2024-09-19T06:56:59.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.179+0000 7f2d60c44640 1 --2- 172.21.15.103:0/222126151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d5c10c170 0x7f2d5c10e610 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.179+0000 7f2d60c44640 1 --2- 172.21.15.103:0/222126151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d5c103660 0x7f2d5c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.179+0000 7f2d60c44640 1 --2- 172.21.15.103:0/222126151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d5c102d40 0x7f2d5c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.179+0000 7f2d60c44640 1 -- 172.21.15.103:0/222126151 >> 172.21.15.103:0/222126151 conn(0x7f2d5c0fc7a0 msgr2=0x7f2d5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:59.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.179+0000 7f2d60c44640 1 -- 172.21.15.103:0/222126151 shutdown_connections 2024-09-19T06:56:59.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.179+0000 7f2d60c44640 1 -- 172.21.15.103:0/222126151 wait complete. 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.179+0000 7f2d60c44640 1 Processor -- start 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d60c44640 1 -- start start 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d60c44640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d5c102d40 0x7f2d5c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d60c44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d5c103660 0x7f2d5c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d60c44640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d5c10c170 0x7f2d5c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d60c44640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c111f70 con 0x7f2d5c103660 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d60c44640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c111df0 con 0x7f2d5c10c170 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d60c44640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c1120f0 con 0x7f2d5c102d40 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d59d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d5c103660 0x7f2d5c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d59d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d5c103660 0x7f2d5c198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45424/0 (socket says 172.21.15.103:45424) 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d59d9b640 1 -- 172.21.15.103:0/297839405 learned_addr learned my addr 172.21.15.103:0/297839405 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d5a59c640 1 --2- 172.21.15.103:0/297839405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d5c102d40 0x7f2d5c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d5ad9d640 1 --2- 172.21.15.103:0/297839405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d5c10c170 0x7f2d5c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d59d9b640 1 -- 172.21.15.103:0/297839405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d5c102d40 msgr2=0x7f2d5c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d59d9b640 1 --2- 172.21.15.103:0/297839405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d5c102d40 0x7f2d5c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d59d9b640 1 -- 172.21.15.103:0/297839405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d5c10c170 msgr2=0x7f2d5c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d59d9b640 1 --2- 172.21.15.103:0/297839405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d5c10c170 0x7f2d5c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d59d9b640 1 -- 172.21.15.103:0/297839405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5c19d3c0 con 0x7f2d5c103660 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.180+0000 7f2d5ad9d640 1 --2- 172.21.15.103:0/297839405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d5c10c170 0x7f2d5c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.181+0000 7f2d59d9b640 1 --2- 172.21.15.103:0/297839405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d5c103660 0x7f2d5c198950 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000e9f0 tx=0x7f2d5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.181+0000 7f2d4b7fe640 1 -- 172.21.15.103:0/297839405 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5000ccc0 con 0x7f2d5c103660 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.181+0000 7f2d4b7fe640 1 -- 172.21.15.103:0/297839405 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d50010910 con 0x7f2d5c103660 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.181+0000 7f2d60c44640 1 -- 172.21.15.103:0/297839405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d5c19d6b0 con 0x7f2d5c103660 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.181+0000 7f2d4b7fe640 1 -- 172.21.15.103:0/297839405 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50018920 con 0x7f2d5c103660 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.181+0000 7f2d60c44640 1 -- 172.21.15.103:0/297839405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d5c1a4f90 con 0x7f2d5c103660 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.183+0000 7f2d4b7fe640 1 -- 172.21.15.103:0/297839405 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f2d50010430 con 0x7f2d5c103660 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.183+0000 7f2d6301d740 1 Processor -- start 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d6301d740 1 -- start start 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d6301d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759eab40 0x564d759e9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d6301d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759e95d0 0x564d759e9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d6301d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759f4370 0x564d759f6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d6301d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564d7594f2f0 con 0x564d759e95d0 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d6301d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564d758f6cf0 con 0x564d759eab40 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d6301d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564d7594b2b0 con 0x564d759f4370 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d5a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759e95d0 0x564d759e9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d5a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759e95d0 0x564d759e9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45436/0 (socket says 172.21.15.103:45436) 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d5a59c640 1 -- 172.21.15.103:0/2540167206 learned_addr learned my addr 172.21.15.103:0/2540167206 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d5ad9d640 1 --2- 172.21.15.103:0/2540167206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759eab40 0x564d759e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d4b7fe640 1 --2- 172.21.15.103:0/297839405 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2d440a33b0 0x7f2d440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d59d9b640 1 --2- 172.21.15.103:0/2540167206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759f4370 0x564d759f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d5ad9d640 1 --2- 172.21.15.103:0/297839405 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2d440a33b0 0x7f2d440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d5a59c640 1 -- 172.21.15.103:0/2540167206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759f4370 msgr2=0x564d759f6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d5a59c640 1 --2- 172.21.15.103:0/2540167206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759f4370 0x564d759f6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d5a59c640 1 -- 172.21.15.103:0/2540167206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759eab40 msgr2=0x564d759e9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d5a59c640 1 --2- 172.21.15.103:0/2540167206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759eab40 0x564d759e9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d5a59c640 1 -- 172.21.15.103:0/2540167206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d758db3a0 con 0x564d759e95d0 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.184+0000 7f2d4b7fe640 1 -- 172.21.15.103:0/297839405 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f2d500d00d0 con 0x7f2d5c103660 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.185+0000 7f2d5a59c640 1 --2- 172.21.15.103:0/2540167206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759e95d0 0x564d759e9a10 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009c70 tx=0x7f2d4c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b4f3543ed3a061 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.185+0000 7f2d48ff9640 1 -- 172.21.15.103:0/2540167206 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c012450 con 0x564d759e95d0 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.185+0000 7f2d5ad9d640 1 --2- 172.21.15.103:0/297839405 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2d440a33b0 0x7f2d440a5870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009430 tx=0x7f2d5402a950 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.185+0000 7f2d48ff9640 1 -- 172.21.15.103:0/2540167206 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d4c012ad0 con 0x564d759e95d0 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.185+0000 7f2d48ff9640 1 -- 172.21.15.103:0/2540167206 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c00fbc0 con 0x564d759e95d0 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.186+0000 7f2d6301d740 1 -- 172.21.15.103:0/2540167206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759e95d0 msgr2=0x564d759e9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.186+0000 7f2d6301d740 1 --2- 172.21.15.103:0/2540167206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759e95d0 0x564d759e9a10 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009c70 tx=0x7f2d4c0117b0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.186+0000 7f2d6301d740 1 -- 172.21.15.103:0/2540167206 shutdown_connections 2024-09-19T06:56:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.186+0000 7f2d6301d740 1 --2- 172.21.15.103:0/2540167206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759f4370 0x564d759f6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.186+0000 7f2d6301d740 1 --2- 172.21.15.103:0/2540167206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759e95d0 0x564d759e9a10 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.186+0000 7f2d6301d740 1 --2- 172.21.15.103:0/2540167206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759eab40 0x564d759e9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.186+0000 7f2d6301d740 1 -- 172.21.15.103:0/2540167206 >> 172.21.15.103:0/2540167206 conn(0x564d759b2310 msgr2=0x564d759e5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.186+0000 7f2d6301d740 1 -- 172.21.15.103:0/2540167206 shutdown_connections 2024-09-19T06:56:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.186+0000 7f2d6301d740 1 -- 172.21.15.103:0/2540167206 wait complete. 2024-09-19T06:56:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.186+0000 7f2d6301d740 1 -- start start 2024-09-19T06:56:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.186+0000 7f2d6301d740 1 Processor -- start 2024-09-19T06:56:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.186+0000 7f2d6301d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759e95d0 0x564d759f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.186+0000 7f2d6301d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759eab40 0x564d759f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.186+0000 7f2d6301d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759f4370 0x564d759f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.186+0000 7f2d6301d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564d759f7070 con 0x564d759e95d0 2024-09-19T06:56:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d6301d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564d7590a060 con 0x564d759eab40 2024-09-19T06:56:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d6301d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564d759f71f0 con 0x564d759f4370 2024-09-19T06:56:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d6301d740 10 client.0 _ll_register_callbacks cb 0x564d7565d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d5a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759e95d0 0x564d759f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d5a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759e95d0 0x564d759f3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45442/0 (socket says 172.21.15.103:45442) 2024-09-19T06:56:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d5a59c640 1 -- 172.21.15.103:0/1829809342 learned_addr learned my addr 172.21.15.103:0/1829809342 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d5ad9d640 1 --2- 172.21.15.103:0/1829809342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759f4370 0x564d759f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d59d9b640 1 --2- 172.21.15.103:0/1829809342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759eab40 0x564d759f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d5a59c640 1 -- 172.21.15.103:0/1829809342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759f4370 msgr2=0x564d759f14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d5a59c640 1 --2- 172.21.15.103:0/1829809342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759f4370 0x564d759f14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d5a59c640 1 -- 172.21.15.103:0/1829809342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759eab40 msgr2=0x564d759f0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d5a59c640 1 --2- 172.21.15.103:0/1829809342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759eab40 0x564d759f0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d5a59c640 1 -- 172.21.15.103:0/1829809342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d756c03c0 con 0x564d759e95d0 2024-09-19T06:56:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d5a59c640 1 --2- 172.21.15.103:0/1829809342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759e95d0 0x564d759f3cb0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00f480 tx=0x7f2d4c004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dca8080fc66d81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d41ffb640 1 -- 172.21.15.103:0/1829809342 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c012450 con 0x564d759e95d0 2024-09-19T06:56:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d6301d740 1 -- 172.21.15.103:0/1829809342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d4c012ad0 con 0x564d759e95d0 2024-09-19T06:56:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d41ffb640 1 -- 172.21.15.103:0/1829809342 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f2d4c0125f0 con 0x564d759e95d0 2024-09-19T06:56:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.187+0000 7f2d41ffb640 1 -- 172.21.15.103:0/1829809342 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d4c031050 con 0x564d759e95d0 2024-09-19T06:56:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.188+0000 7f2d6301d740 10 client.18162 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.188+0000 7f2d6301d740 1 -- 172.21.15.103:0/1829809342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564d75a02a60 con 0x564d759e95d0 2024-09-19T06:56:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.188+0000 7f2d41ffb640 1 -- 172.21.15.103:0/1829809342 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d4c012ad0 con 0x564d759e95d0 2024-09-19T06:56:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.188+0000 7f2d41ffb640 1 -- 172.21.15.103:0/1829809342 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2d4c0052b0 con 0x564d759e95d0 2024-09-19T06:56:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.188+0000 7f2d6301d740 10 client.18162 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.188+0000 7f2d6301d740 10 client.18162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.188+0000 7f2d6301d740 20 client.18162 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.188+0000 7f2d6301d740 1 --2- 172.21.15.103:0/1829809342 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564d75a03ec0 0x564d75a062b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.188+0000 7f2d6301d740 4 client.18162 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.188+0000 7f2d6301d740 1 -- 172.21.15.103:0/1829809342 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d7595c860 con 0x564d75a03ec0 2024-09-19T06:56:59.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.188+0000 7f2d59d9b640 1 --2- 172.21.15.103:0/1829809342 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564d75a03ec0 0x564d75a062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.189+0000 7f2d41ffb640 10 client.18162 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:59.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.189+0000 7f2d59d9b640 1 --2- 172.21.15.103:0/1829809342 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564d75a03ec0 0x564d75a062b0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c155e0db54e6671b server_cookie=cb064e1169c8e64d in_seq=0 out_seq=0 2024-09-19T06:56:59.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.189+0000 7f2d41ffb640 1 -- 172.21.15.103:0/1829809342 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d7595c860 con 0x564d75a03ec0 2024-09-19T06:56:59.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.189+0000 7f2d41ffb640 10 client.18162 handle_command_reply: tid=0 2024-09-19T06:56:59.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.189+0000 7f2d6301d740 1 client.18162 shutdown 2024-09-19T06:56:59.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.190+0000 7f2d6301d740 1 -- 172.21.15.103:0/1829809342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759e95d0 msgr2=0x564d759f3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:59.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.190+0000 7f2d6301d740 1 --2- 172.21.15.103:0/1829809342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759e95d0 0x564d759f3cb0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00f480 tx=0x7f2d4c004590 comp rx=0 tx=0).stop 2024-09-19T06:56:59.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.190+0000 7f2d6301d740 1 -- 172.21.15.103:0/1829809342 shutdown_connections 2024-09-19T06:56:59.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.190+0000 7f2d6301d740 1 --2- 172.21.15.103:0/1829809342 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564d75a03ec0 0x564d75a062b0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.190+0000 7f2d6301d740 1 --2- 172.21.15.103:0/1829809342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759f4370 0x564d759f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.190+0000 7f2d6301d740 1 --2- 172.21.15.103:0/1829809342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759eab40 0x564d759f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.190+0000 7f2d41ffb640 0 client.18162 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:59.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.190+0000 7f2d6301d740 1 --2- 172.21.15.103:0/1829809342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759e95d0 0x564d759f3cb0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.190+0000 7f2d6301d740 1 -- 172.21.15.103:0/1829809342 >> 172.21.15.103:0/1829809342 conn(0x564d759b2310 msgr2=0x564d759eb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:59.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.190+0000 7f2d6301d740 1 -- 172.21.15.103:0/1829809342 shutdown_connections 2024-09-19T06:56:59.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.190+0000 7f2d6301d740 1 -- 172.21.15.103:0/1829809342 wait complete. 2024-09-19T06:56:59.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.190+0000 7f2d6301d740 20 client.18162 trim_cache size 0 max 16384 2024-09-19T06:56:59.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.202+0000 7f2d6301d740 1 Processor -- start 2024-09-19T06:56:59.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.202+0000 7f2d6301d740 1 -- start start 2024-09-19T06:56:59.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.202+0000 7f2d6301d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759e95d0 0x564d75a00310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.202+0000 7f2d6301d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759eab40 0x564d759f0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d6301d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759f4370 0x564d75a0b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d6301d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c1ad900 con 0x564d759eab40 2024-09-19T06:56:59.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d6301d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c1af440 con 0x564d759e95d0 2024-09-19T06:56:59.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d6301d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c1adfd0 con 0x564d759f4370 2024-09-19T06:56:59.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d59d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759eab40 0x564d759f0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d59d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759eab40 0x564d759f0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45458/0 (socket says 172.21.15.103:45458) 2024-09-19T06:56:59.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d59d9b640 1 -- 172.21.15.103:0/1946009098 learned_addr learned my addr 172.21.15.103:0/1946009098 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:59.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d5a59c640 1 --2- 172.21.15.103:0/1946009098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759e95d0 0x564d75a00310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d59d9b640 1 -- 172.21.15.103:0/1946009098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759f4370 msgr2=0x564d75a0b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:59.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d59d9b640 1 --2- 172.21.15.103:0/1946009098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759f4370 0x564d75a0b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d59d9b640 1 -- 172.21.15.103:0/1946009098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759e95d0 msgr2=0x564d75a00310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:59.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d59d9b640 1 --2- 172.21.15.103:0/1946009098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759e95d0 0x564d75a00310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d59d9b640 1 -- 172.21.15.103:0/1946009098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d759f22e0 con 0x564d759eab40 2024-09-19T06:56:59.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d59d9b640 1 --2- 172.21.15.103:0/1946009098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759eab40 0x564d759f0710 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000e9c0 tx=0x7f2d500bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2b92ab70a6869f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:59.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d43fff640 1 -- 172.21.15.103:0/1946009098 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d500a4fa0 con 0x564d759eab40 2024-09-19T06:56:59.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d43fff640 1 -- 172.21.15.103:0/1946009098 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d5009a630 con 0x564d759eab40 2024-09-19T06:56:59.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.203+0000 7f2d43fff640 1 -- 172.21.15.103:0/1946009098 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d500d20b0 con 0x564d759eab40 2024-09-19T06:56:59.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.204+0000 7f2d6301d740 1 -- 172.21.15.103:0/1946009098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759eab40 msgr2=0x564d759f0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:59.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.204+0000 7f2d6301d740 1 --2- 172.21.15.103:0/1946009098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759eab40 0x564d759f0710 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000e9c0 tx=0x7f2d500bc040 comp rx=0 tx=0).stop 2024-09-19T06:56:59.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.204+0000 7f2d6301d740 1 -- 172.21.15.103:0/1946009098 shutdown_connections 2024-09-19T06:56:59.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.204+0000 7f2d6301d740 1 --2- 172.21.15.103:0/1946009098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759f4370 0x564d75a0b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.204+0000 7f2d6301d740 1 --2- 172.21.15.103:0/1946009098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759eab40 0x564d759f0710 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.204+0000 7f2d6301d740 1 --2- 172.21.15.103:0/1946009098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759e95d0 0x564d75a00310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.204+0000 7f2d6301d740 1 -- 172.21.15.103:0/1946009098 >> 172.21.15.103:0/1946009098 conn(0x564d759b2310 msgr2=0x564d759e4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:59.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.204+0000 7f2d6301d740 1 -- 172.21.15.103:0/1946009098 shutdown_connections 2024-09-19T06:56:59.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.204+0000 7f2d6301d740 1 -- 172.21.15.103:0/1946009098 wait complete. 2024-09-19T06:56:59.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.205+0000 7f2d6301d740 1 -- start start 2024-09-19T06:56:59.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.205+0000 7f2d6301d740 1 Processor -- start 2024-09-19T06:56:59.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.205+0000 7f2d6301d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759e95d0 0x564d759fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.205+0000 7f2d6301d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759eab40 0x564d759f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.205+0000 7f2d6301d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759f4370 0x564d759f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.205+0000 7f2d6301d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c1aed70 con 0x564d759f4370 2024-09-19T06:56:59.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.205+0000 7f2d6301d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c1ae6a0 con 0x564d759e95d0 2024-09-19T06:56:59.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.205+0000 7f2d5ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759f4370 0x564d759f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.205+0000 7f2d5ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759f4370 0x564d759f83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45462/0 (socket says 172.21.15.103:45462) 2024-09-19T06:56:59.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.205+0000 7f2d5ad9d640 1 -- 172.21.15.103:0/2624096233 learned_addr learned my addr 172.21.15.103:0/2624096233 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:56:59.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.205+0000 7f2d6301d740 1 -- 172.21.15.103:0/2624096233 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c1ab900 con 0x564d759eab40 2024-09-19T06:56:59.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.205+0000 7f2d6301d740 10 client.0 _ll_register_callbacks cb 0x7f2d50005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:59.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.205+0000 7f2d5a59c640 1 --2- 172.21.15.103:0/2624096233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759e95d0 0x564d759fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.205+0000 7f2d59d9b640 1 --2- 172.21.15.103:0/2624096233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759eab40 0x564d759f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.206+0000 7f2d5ad9d640 1 -- 172.21.15.103:0/2624096233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759eab40 msgr2=0x564d759f7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:59.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.206+0000 7f2d5ad9d640 1 --2- 172.21.15.103:0/2624096233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759eab40 0x564d759f7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.206+0000 7f2d5ad9d640 1 -- 172.21.15.103:0/2624096233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759e95d0 msgr2=0x564d759fcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:59.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.206+0000 7f2d5ad9d640 1 --2- 172.21.15.103:0/2624096233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759e95d0 0x564d759fcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.206+0000 7f2d5ad9d640 1 -- 172.21.15.103:0/2624096233 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d759f8bf0 con 0x564d759f4370 2024-09-19T06:56:59.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.206+0000 7f2d5ad9d640 1 --2- 172.21.15.103:0/2624096233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759f4370 0x564d759f83c0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2d540099b0 tx=0x7f2d5402fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2b4a0555670552 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:59.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.206+0000 7f2d41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:56:59.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.206+0000 7f2d41ffb640 1 -- 172.21.15.103:0/2624096233 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5404f070 con 0x564d759f4370 2024-09-19T06:56:59.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.206+0000 7f2d41ffb640 1 -- 172.21.15.103:0/2624096233 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f2d54004450 con 0x564d759f4370 2024-09-19T06:56:59.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.206+0000 7f2d6301d740 1 -- 172.21.15.103:0/2624096233 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d5009a630 con 0x564d759f4370 2024-09-19T06:56:59.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.206+0000 7f2d41ffb640 1 -- 172.21.15.103:0/2624096233 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d54034040 con 0x564d759f4370 2024-09-19T06:56:59.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.206+0000 7f2d6301d740 10 client.18168 fetch_fsmap learned FSMap version 27 2024-09-19T06:56:59.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.206+0000 7f2d6301d740 1 -- 172.21.15.103:0/2624096233 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564d75a6e850 con 0x564d759f4370 2024-09-19T06:56:59.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.207+0000 7f2d41ffb640 1 -- 172.21.15.103:0/2624096233 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d5009a630 con 0x564d759f4370 2024-09-19T06:56:59.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.207+0000 7f2d41ffb640 1 -- 172.21.15.103:0/2624096233 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2d54048040 con 0x564d759f4370 2024-09-19T06:56:59.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.207+0000 7f2d6301d740 10 client.18168 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:56:59.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.207+0000 7f2d6301d740 10 client.18168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:59.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.207+0000 7f2d6301d740 20 client.18168 populate_metadata read hostname 'smithi103' 2024-09-19T06:56:59.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.207+0000 7f2d6301d740 1 --2- 172.21.15.103:0/2624096233 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564d75a6ef80 0x564d75a713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.207+0000 7f2d6301d740 4 client.18168 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:56:59.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.207+0000 7f2d6301d740 1 -- 172.21.15.103:0/2624096233 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d759f1830 con 0x564d75a6ef80 2024-09-19T06:56:59.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.207+0000 7f2d5a59c640 1 --2- 172.21.15.103:0/2624096233 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564d75a6ef80 0x564d75a713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d5a59c640 1 --2- 172.21.15.103:0/2624096233 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564d75a6ef80 0x564d75a713a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf9e929b88fab8e server_cookie=704cdecd4c3d4b57 in_seq=0 out_seq=0 2024-09-19T06:56:59.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d41ffb640 10 client.18168 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:59.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d41ffb640 1 -- 172.21.15.103:0/2624096233 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d759f1830 con 0x564d75a6ef80 2024-09-19T06:56:59.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d41ffb640 10 client.18168 handle_command_reply: tid=0 2024-09-19T06:56:59.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d6301d740 1 client.18168 shutdown 2024-09-19T06:56:59.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d6301d740 1 -- 172.21.15.103:0/2624096233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759f4370 msgr2=0x564d759f83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:59.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d6301d740 1 --2- 172.21.15.103:0/2624096233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759f4370 0x564d759f83c0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2d540099b0 tx=0x7f2d5402fec0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d6301d740 1 -- 172.21.15.103:0/2624096233 shutdown_connections 2024-09-19T06:56:59.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d6301d740 1 --2- 172.21.15.103:0/2624096233 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564d75a6ef80 0x564d75a713a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d6301d740 1 --2- 172.21.15.103:0/2624096233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d759f4370 0x564d759f83c0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d6301d740 1 --2- 172.21.15.103:0/2624096233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d759eab40 0x564d759f7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d6301d740 1 --2- 172.21.15.103:0/2624096233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d759e95d0 0x564d759fcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d6301d740 1 -- 172.21.15.103:0/2624096233 >> 172.21.15.103:0/2624096233 conn(0x564d759b2310 msgr2=0x564d759f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:59.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d41ffb640 0 client.18168 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:56:59.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d6301d740 1 -- 172.21.15.103:0/2624096233 shutdown_connections 2024-09-19T06:56:59.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d6301d740 1 -- 172.21.15.103:0/2624096233 wait complete. 2024-09-19T06:56:59.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.208+0000 7f2d6301d740 20 client.18168 trim_cache size 0 max 16384 2024-09-19T06:56:59.224 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:56:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.209+0000 7f2d60c44640 1 -- 172.21.15.103:0/297839405 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2d440a33b0 msgr2=0x7f2d440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.209+0000 7f2d60c44640 1 --2- 172.21.15.103:0/297839405 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2d440a33b0 0x7f2d440a5870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009430 tx=0x7f2d5402a950 comp rx=0 tx=0).stop 2024-09-19T06:56:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.209+0000 7f2d60c44640 1 -- 172.21.15.103:0/297839405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d5c103660 msgr2=0x7f2d5c198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.209+0000 7f2d60c44640 1 --2- 172.21.15.103:0/297839405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d5c103660 0x7f2d5c198950 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000e9f0 tx=0x7f2d5000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.209+0000 7f2d60c44640 1 -- 172.21.15.103:0/297839405 shutdown_connections 2024-09-19T06:56:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.209+0000 7f2d60c44640 1 --2- 172.21.15.103:0/297839405 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2d440a33b0 0x7f2d440a5870 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.209+0000 7f2d60c44640 1 --2- 172.21.15.103:0/297839405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d5c10c170 0x7f2d5c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.209+0000 7f2d60c44640 1 --2- 172.21.15.103:0/297839405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d5c103660 0x7f2d5c198950 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.209+0000 7f2d60c44640 1 --2- 172.21.15.103:0/297839405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d5c102d40 0x7f2d5c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.209+0000 7f2d60c44640 1 -- 172.21.15.103:0/297839405 >> 172.21.15.103:0/297839405 conn(0x7f2d5c0fc7a0 msgr2=0x7f2d5c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:59.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.209+0000 7f2d60c44640 1 -- 172.21.15.103:0/297839405 shutdown_connections 2024-09-19T06:56:59.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:56:59.209+0000 7f2d60c44640 1 -- 172.21.15.103:0/297839405 wait complete. 2024-09-19T06:56:59.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:56:59 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/800270622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:59.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:56:59 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/800270622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:59.912 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:00.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:00 smithi103 ceph-mon[31158]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 15 op/s 2024-09-19T06:57:00.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.414+0000 7f5314206640 1 -- 172.21.15.103:0/1124567204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f530c105ec0 msgr2=0x7f530c106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:00.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.414+0000 7f5314206640 1 --2- 172.21.15.103:0/1124567204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f530c105ec0 0x7f530c106340 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f5304009ca0 tx=0x7f530402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.414+0000 7f5314206640 1 -- 172.21.15.103:0/1124567204 shutdown_connections 2024-09-19T06:57:00.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.414+0000 7f5314206640 1 --2- 172.21.15.103:0/1124567204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f530c105ec0 0x7f530c106340 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.414+0000 7f5314206640 1 --2- 172.21.15.103:0/1124567204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f530c1011e0 0x7f530c105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.414+0000 7f5314206640 1 --2- 172.21.15.103:0/1124567204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f530c1008c0 0x7f530c100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.414+0000 7f5314206640 1 -- 172.21.15.103:0/1124567204 >> 172.21.15.103:0/1124567204 conn(0x7f530c0fc7d0 msgr2=0x7f530c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:00.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.415+0000 7f5314206640 1 -- 172.21.15.103:0/1124567204 shutdown_connections 2024-09-19T06:57:00.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.415+0000 7f5314206640 1 -- 172.21.15.103:0/1124567204 wait complete. 2024-09-19T06:57:00.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.415+0000 7f5314206640 1 Processor -- start 2024-09-19T06:57:00.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.415+0000 7f5314206640 1 -- start start 2024-09-19T06:57:00.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.415+0000 7f5314206640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f530c1008c0 0x7f530c1a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.415+0000 7f5314206640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f530c1011e0 0x7f530c1a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.415+0000 7f5314206640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f530c105ec0 0x7f530c19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.415+0000 7f5314206640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f530c118690 con 0x7f530c105ec0 2024-09-19T06:57:00.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.415+0000 7f5314206640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f530c118510 con 0x7f530c1011e0 2024-09-19T06:57:00.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.415+0000 7f5314206640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f530c118810 con 0x7f530c1008c0 2024-09-19T06:57:00.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.416+0000 7f53127a3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f530c105ec0 0x7f530c19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.416+0000 7f53127a3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f530c105ec0 0x7f530c19e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45490/0 (socket says 172.21.15.103:45490) 2024-09-19T06:57:00.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.416+0000 7f53117a1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f530c1011e0 0x7f530c1a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.416+0000 7f53127a3640 1 -- 172.21.15.103:0/4240755643 learned_addr learned my addr 172.21.15.103:0/4240755643 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:00.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.416+0000 7f5311fa2640 1 --2- 172.21.15.103:0/4240755643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f530c1008c0 0x7f530c1a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.416+0000 7f53127a3640 1 -- 172.21.15.103:0/4240755643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f530c1008c0 msgr2=0x7f530c1a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:00.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.416+0000 7f53127a3640 1 --2- 172.21.15.103:0/4240755643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f530c1008c0 0x7f530c1a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.416+0000 7f53127a3640 1 -- 172.21.15.103:0/4240755643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f530c1011e0 msgr2=0x7f530c1a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:00.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.416+0000 7f53127a3640 1 --2- 172.21.15.103:0/4240755643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f530c1011e0 0x7f530c1a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.416+0000 7f53127a3640 1 -- 172.21.15.103:0/4240755643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530c19f030 con 0x7f530c105ec0 2024-09-19T06:57:00.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.416+0000 7f53127a3640 1 --2- 172.21.15.103:0/4240755643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f530c105ec0 0x7f530c19e900 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f53040097b0 tx=0x7f5304003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.416+0000 7f5302ffd640 1 -- 172.21.15.103:0/4240755643 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5304031750 con 0x7f530c105ec0 2024-09-19T06:57:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.416+0000 7f5302ffd640 1 -- 172.21.15.103:0/4240755643 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53040318f0 con 0x7f530c105ec0 2024-09-19T06:57:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.416+0000 7f5314206640 1 -- 172.21.15.103:0/4240755643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f530c19f2c0 con 0x7f530c105ec0 2024-09-19T06:57:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.416+0000 7f5302ffd640 1 -- 172.21.15.103:0/4240755643 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5304030bd0 con 0x7f530c105ec0 2024-09-19T06:57:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.416+0000 7f5314206640 1 -- 172.21.15.103:0/4240755643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f530c1ab610 con 0x7f530c105ec0 2024-09-19T06:57:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.419+0000 7f5302ffd640 1 -- 172.21.15.103:0/4240755643 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f5304004040 con 0x7f530c105ec0 2024-09-19T06:57:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.419+0000 7f53165df740 1 Processor -- start 2024-09-19T06:57:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.419+0000 7f53165df740 1 -- start start 2024-09-19T06:57:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.419+0000 7f53165df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca39060 0x55d5aca39440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.419+0000 7f53165df740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca43d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.419+0000 7f53165df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca44240 0x55d5aca46630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.419+0000 7f53165df740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d5ac99e620 con 0x55d5aca39980 2024-09-19T06:57:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.419+0000 7f53165df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d5ac945e50 con 0x55d5aca44240 2024-09-19T06:57:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.419+0000 7f53165df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5ac99a5e0 con 0x55d5aca39060 2024-09-19T06:57:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.419+0000 7f5302ffd640 1 --2- 172.21.15.103:0/4240755643 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f52f80a33b0 0x7f52f80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.420+0000 7f5311fa2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca43d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.420+0000 7f5311fa2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca43d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45494/0 (socket says 172.21.15.103:45494) 2024-09-19T06:57:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.420+0000 7f5311fa2640 1 -- 172.21.15.103:0/1581862105 learned_addr learned my addr 172.21.15.103:0/1581862105 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.420+0000 7f53117a1640 1 --2- 172.21.15.103:0/1581862105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca44240 0x55d5aca46630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.420+0000 7f53127a3640 1 --2- 172.21.15.103:0/4240755643 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f52f80a33b0 0x7f52f80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.420+0000 7f53117a1640 1 --2- 172.21.15.103:0/1581862105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca39060 0x55d5aca39440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.420+0000 7f5302ffd640 1 -- 172.21.15.103:0/4240755643 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f53040f1120 con 0x7f530c105ec0 2024-09-19T06:57:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.420+0000 7f5311fa2640 1 -- 172.21.15.103:0/1581862105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca39060 msgr2=0x55d5aca39440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.420+0000 7f5311fa2640 1 --2- 172.21.15.103:0/1581862105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca39060 0x55d5aca39440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.420+0000 7f5311fa2640 1 -- 172.21.15.103:0/1581862105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca44240 msgr2=0x55d5aca46630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.420+0000 7f5311fa2640 1 --2- 172.21.15.103:0/1581862105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca44240 0x55d5aca46630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.420+0000 7f5311fa2640 1 -- 172.21.15.103:0/1581862105 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5ac936d40 con 0x55d5aca39980 2024-09-19T06:57:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.420+0000 7f53127a3640 1 --2- 172.21.15.103:0/4240755643 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f52f80a33b0 0x7f52f80a5870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f53040eeab0 tx=0x7f53040095c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.420+0000 7f5311fa2640 1 --2- 172.21.15.103:0/1581862105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca43d00 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f530800ac80 tx=0x7f530800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea2d77bcf5a84c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.420+0000 7f52f7fff640 1 -- 172.21.15.103:0/1581862105 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530801d070 con 0x55d5aca39980 2024-09-19T06:57:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.420+0000 7f52f7fff640 1 -- 172.21.15.103:0/1581862105 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5308018b20 con 0x55d5aca39980 2024-09-19T06:57:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.421+0000 7f52f7fff640 1 -- 172.21.15.103:0/1581862105 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308010bb0 con 0x55d5aca39980 2024-09-19T06:57:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.421+0000 7f53165df740 1 -- 172.21.15.103:0/1581862105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 msgr2=0x55d5aca43d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.421+0000 7f53165df740 1 --2- 172.21.15.103:0/1581862105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca43d00 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f530800ac80 tx=0x7f530800f790 comp rx=0 tx=0).stop 2024-09-19T06:57:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.421+0000 7f53165df740 1 -- 172.21.15.103:0/1581862105 shutdown_connections 2024-09-19T06:57:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.421+0000 7f53165df740 1 --2- 172.21.15.103:0/1581862105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca44240 0x55d5aca46630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.421+0000 7f53165df740 1 --2- 172.21.15.103:0/1581862105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca43d00 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.421+0000 7f53165df740 1 --2- 172.21.15.103:0/1581862105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca39060 0x55d5aca39440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.421+0000 7f53165df740 1 -- 172.21.15.103:0/1581862105 >> 172.21.15.103:0/1581862105 conn(0x55d5aca355f0 msgr2=0x55d5aca359d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.421+0000 7f53165df740 1 -- 172.21.15.103:0/1581862105 shutdown_connections 2024-09-19T06:57:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.421+0000 7f53165df740 1 -- 172.21.15.103:0/1581862105 wait complete. 2024-09-19T06:57:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.422+0000 7f53165df740 1 -- start start 2024-09-19T06:57:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.422+0000 7f53165df740 1 Processor -- start 2024-09-19T06:57:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.422+0000 7f53165df740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39060 0x55d5aca410d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.422+0000 7f53165df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca39980 0x55d5aca41610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.422+0000 7f53165df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca44240 0x55d5aca41b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.422+0000 7f53165df740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d5aca46e10 con 0x55d5aca39060 2024-09-19T06:57:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.422+0000 7f53165df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d5ac926f10 con 0x55d5aca39980 2024-09-19T06:57:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.422+0000 7f53165df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5aca46f90 con 0x55d5aca44240 2024-09-19T06:57:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.422+0000 7f53165df740 10 client.0 _ll_register_callbacks cb 0x55d5ac94adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.422+0000 7f53117a1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39060 0x55d5aca410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.422+0000 7f53117a1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39060 0x55d5aca410d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45502/0 (socket says 172.21.15.103:45502) 2024-09-19T06:57:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.422+0000 7f53117a1640 1 -- 172.21.15.103:0/1690948263 learned_addr learned my addr 172.21.15.103:0/1690948263 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f5311fa2640 1 --2- 172.21.15.103:0/1690948263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca39980 0x55d5aca41610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f53117a1640 1 -- 172.21.15.103:0/1690948263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca44240 msgr2=0x55d5aca41b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f53117a1640 1 --2- 172.21.15.103:0/1690948263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca44240 0x55d5aca41b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f53117a1640 1 -- 172.21.15.103:0/1690948263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca39980 msgr2=0x55d5aca41610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f53117a1640 1 --2- 172.21.15.103:0/1690948263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca39980 0x55d5aca41610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f53117a1640 1 -- 172.21.15.103:0/1690948263 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5ac6ad580 con 0x55d5aca39060 2024-09-19T06:57:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f5311fa2640 1 --2- 172.21.15.103:0/1690948263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca39980 0x55d5aca41610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f53117a1640 1 --2- 172.21.15.103:0/1690948263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39060 0x55d5aca410d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f52fc015640 tx=0x7f52fc015b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202b5217778823f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f52f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f52f57fa640 1 -- 172.21.15.103:0/1690948263 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc01b070 con 0x55d5aca39060 2024-09-19T06:57:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f52f57fa640 1 -- 172.21.15.103:0/1690948263 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f52fc016790 con 0x55d5aca39060 2024-09-19T06:57:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f53165df740 1 -- 172.21.15.103:0/1690948263 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5308018b20 con 0x55d5aca39060 2024-09-19T06:57:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f52f57fa640 1 -- 172.21.15.103:0/1690948263 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f52fc027050 con 0x55d5aca39060 2024-09-19T06:57:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f53165df740 10 client.18180 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f53165df740 1 -- 172.21.15.103:0/1690948263 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d5ac777960 con 0x55d5aca39060 2024-09-19T06:57:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f52f57fa640 1 -- 172.21.15.103:0/1690948263 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5308018b20 con 0x55d5aca39060 2024-09-19T06:57:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.423+0000 7f52f57fa640 1 -- 172.21.15.103:0/1690948263 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f52fc017ba0 con 0x55d5aca39060 2024-09-19T06:57:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.424+0000 7f53165df740 10 client.18180 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.424+0000 7f53165df740 10 client.18180 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:57:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.424+0000 7f53165df740 20 client.18180 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.424+0000 7f53165df740 1 --2- 172.21.15.103:0/1690948263 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d5aca53d40 0x55d5aca561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.424+0000 7f53165df740 4 client.18180 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.424+0000 7f53165df740 1 -- 172.21.15.103:0/1690948263 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5ac9abb90 con 0x55d5aca53d40 2024-09-19T06:57:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.424+0000 7f5311fa2640 1 --2- 172.21.15.103:0/1690948263 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d5aca53d40 0x55d5aca561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.424+0000 7f52f57fa640 10 client.18180 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.424+0000 7f5311fa2640 1 --2- 172.21.15.103:0/1690948263 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d5aca53d40 0x55d5aca561e0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b6bf714ccd81e1 server_cookie=9bf44a6716b2e3ef in_seq=0 out_seq=0 2024-09-19T06:57:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.425+0000 7f52f57fa640 1 -- 172.21.15.103:0/1690948263 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5ac9abb90 con 0x55d5aca53d40 2024-09-19T06:57:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.425+0000 7f52f57fa640 10 client.18180 handle_command_reply: tid=0 2024-09-19T06:57:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.425+0000 7f53165df740 1 client.18180 shutdown 2024-09-19T06:57:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.425+0000 7f53165df740 1 -- 172.21.15.103:0/1690948263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39060 msgr2=0x55d5aca410d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.425+0000 7f53165df740 1 --2- 172.21.15.103:0/1690948263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39060 0x55d5aca410d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f52fc015640 tx=0x7f52fc015b00 comp rx=0 tx=0).stop 2024-09-19T06:57:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.425+0000 7f53165df740 1 -- 172.21.15.103:0/1690948263 shutdown_connections 2024-09-19T06:57:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.425+0000 7f53165df740 1 --2- 172.21.15.103:0/1690948263 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d5aca53d40 0x55d5aca561e0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.425+0000 7f53165df740 1 --2- 172.21.15.103:0/1690948263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca44240 0x55d5aca41b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.425+0000 7f53165df740 1 --2- 172.21.15.103:0/1690948263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca39980 0x55d5aca41610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.425+0000 7f52f57fa640 0 client.18180 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.425+0000 7f53165df740 1 --2- 172.21.15.103:0/1690948263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39060 0x55d5aca410d0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.425+0000 7f53165df740 1 -- 172.21.15.103:0/1690948263 >> 172.21.15.103:0/1690948263 conn(0x55d5aca355f0 msgr2=0x55d5aca3b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.425+0000 7f53165df740 1 -- 172.21.15.103:0/1690948263 shutdown_connections 2024-09-19T06:57:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.425+0000 7f53165df740 1 -- 172.21.15.103:0/1690948263 wait complete. 2024-09-19T06:57:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.426+0000 7f53165df740 20 client.18180 trim_cache size 0 max 16384 2024-09-19T06:57:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:00 smithi134 ceph-mon[35356]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 15 op/s 2024-09-19T06:57:00.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.442+0000 7f53165df740 1 Processor -- start 2024-09-19T06:57:00.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.442+0000 7f53165df740 1 -- start start 2024-09-19T06:57:00.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.442+0000 7f53165df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca39060 0x55d5aca3bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.442+0000 7f53165df740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca3c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.442+0000 7f53165df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca44240 0x55d5aca34640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.442+0000 7f53165df740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f530c1b3f20 con 0x55d5aca39980 2024-09-19T06:57:00.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.442+0000 7f53165df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f530c1b5a60 con 0x55d5aca44240 2024-09-19T06:57:00.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.442+0000 7f53165df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f530c1b45f0 con 0x55d5aca39060 2024-09-19T06:57:00.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.442+0000 7f5311fa2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca3c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.442+0000 7f5311fa2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca3c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45506/0 (socket says 172.21.15.103:45506) 2024-09-19T06:57:00.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.442+0000 7f5311fa2640 1 -- 172.21.15.103:0/1219848332 learned_addr learned my addr 172.21.15.103:0/1219848332 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:00.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.443+0000 7f53117a1640 1 --2- 172.21.15.103:0/1219848332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca39060 0x55d5aca3bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.443+0000 7f5311fa2640 1 -- 172.21.15.103:0/1219848332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca39060 msgr2=0x55d5aca3bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:00.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.443+0000 7f5311fa2640 1 --2- 172.21.15.103:0/1219848332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca39060 0x55d5aca3bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.443+0000 7f5311fa2640 1 -- 172.21.15.103:0/1219848332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca44240 msgr2=0x55d5aca34640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:00.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.443+0000 7f5311fa2640 1 --2- 172.21.15.103:0/1219848332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca44240 0x55d5aca34640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.443+0000 7f5311fa2640 1 -- 172.21.15.103:0/1219848332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5aca43010 con 0x55d5aca39980 2024-09-19T06:57:00.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.443+0000 7f53117a1640 1 --2- 172.21.15.103:0/1219848332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca39060 0x55d5aca3bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:00.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.443+0000 7f5311fa2640 1 --2- 172.21.15.103:0/1219848332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca3c480 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f530801b040 tx=0x7f530800a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38cd52857ccd11b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:00.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.443+0000 7f52f77fe640 1 -- 172.21.15.103:0/1219848332 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530801d040 con 0x55d5aca39980 2024-09-19T06:57:00.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.443+0000 7f52f77fe640 1 -- 172.21.15.103:0/1219848332 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f530801e040 con 0x55d5aca39980 2024-09-19T06:57:00.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.443+0000 7f52f77fe640 1 -- 172.21.15.103:0/1219848332 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308018440 con 0x55d5aca39980 2024-09-19T06:57:00.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.444+0000 7f53165df740 1 -- 172.21.15.103:0/1219848332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 msgr2=0x55d5aca3c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:00.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.444+0000 7f53165df740 1 --2- 172.21.15.103:0/1219848332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca3c480 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f530801b040 tx=0x7f530800a510 comp rx=0 tx=0).stop 2024-09-19T06:57:00.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.444+0000 7f53165df740 1 -- 172.21.15.103:0/1219848332 shutdown_connections 2024-09-19T06:57:00.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.444+0000 7f53165df740 1 --2- 172.21.15.103:0/1219848332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca44240 0x55d5aca34640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.444+0000 7f53165df740 1 --2- 172.21.15.103:0/1219848332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca3c480 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.444+0000 7f53165df740 1 --2- 172.21.15.103:0/1219848332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca39060 0x55d5aca3bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.444+0000 7f53165df740 1 -- 172.21.15.103:0/1219848332 >> 172.21.15.103:0/1219848332 conn(0x55d5aca355f0 msgr2=0x55d5aca33f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:00.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.444+0000 7f53165df740 1 -- 172.21.15.103:0/1219848332 shutdown_connections 2024-09-19T06:57:00.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.444+0000 7f53165df740 1 -- 172.21.15.103:0/1219848332 wait complete. 2024-09-19T06:57:00.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.444+0000 7f53165df740 1 -- start start 2024-09-19T06:57:00.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.444+0000 7f53165df740 1 Processor -- start 2024-09-19T06:57:00.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f53165df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca39060 0x55d5aca48640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f53165df740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca48b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f53165df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca44240 0x55d5acabda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f53165df740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f530c1b5390 con 0x55d5aca39980 2024-09-19T06:57:00.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f53165df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f530c1b4cc0 con 0x55d5aca39060 2024-09-19T06:57:00.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f53165df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f530c1b1f20 con 0x55d5aca44240 2024-09-19T06:57:00.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f53165df740 10 client.0 _ll_register_callbacks cb 0x7f52fc009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:00.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f5311fa2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca48b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f5311fa2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca48b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45520/0 (socket says 172.21.15.103:45520) 2024-09-19T06:57:00.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f5311fa2640 1 -- 172.21.15.103:0/754753220 learned_addr learned my addr 172.21.15.103:0/754753220 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:00.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f53117a1640 1 --2- 172.21.15.103:0/754753220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca44240 0x55d5acabda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f53117a1640 1 --2- 172.21.15.103:0/754753220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca39060 0x55d5aca48640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f5311fa2640 1 -- 172.21.15.103:0/754753220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca44240 msgr2=0x55d5acabda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:00.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f5311fa2640 1 --2- 172.21.15.103:0/754753220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca44240 0x55d5acabda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f5311fa2640 1 -- 172.21.15.103:0/754753220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca39060 msgr2=0x55d5aca48640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:00.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f5311fa2640 1 --2- 172.21.15.103:0/754753220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca39060 0x55d5aca48640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f5311fa2640 1 -- 172.21.15.103:0/754753220 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5acabdfc0 con 0x55d5aca39980 2024-09-19T06:57:00.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f53117a1640 1 --2- 172.21.15.103:0/754753220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca39060 0x55d5aca48640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:00.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f5311fa2640 1 --2- 172.21.15.103:0/754753220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca48b80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f530800a5c0 tx=0x7f530800a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7348b4cb890acd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:00.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.445+0000 7f52f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:00.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.446+0000 7f52f57fa640 1 -- 172.21.15.103:0/754753220 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530803a020 con 0x55d5aca39980 2024-09-19T06:57:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.446+0000 7f52f57fa640 1 -- 172.21.15.103:0/754753220 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f5308010830 con 0x55d5aca39980 2024-09-19T06:57:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.446+0000 7f53165df740 1 -- 172.21.15.103:0/754753220 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f530801e040 con 0x55d5aca39980 2024-09-19T06:57:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.446+0000 7f52f57fa640 1 -- 172.21.15.103:0/754753220 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f530801d040 con 0x55d5aca39980 2024-09-19T06:57:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.446+0000 7f53165df740 10 client.18186 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.446+0000 7f53165df740 1 -- 172.21.15.103:0/754753220 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d5acabf240 con 0x55d5aca39980 2024-09-19T06:57:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.446+0000 7f52f57fa640 1 -- 172.21.15.103:0/754753220 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f530801e040 con 0x55d5aca39980 2024-09-19T06:57:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.446+0000 7f52f57fa640 1 -- 172.21.15.103:0/754753220 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5308004660 con 0x55d5aca39980 2024-09-19T06:57:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.446+0000 7f53165df740 10 client.18186 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.446+0000 7f53165df740 10 client.18186 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:57:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.446+0000 7f53165df740 20 client.18186 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.446+0000 7f53165df740 1 --2- 172.21.15.103:0/754753220 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d5acabf810 0x55d5acac1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.446+0000 7f53165df740 4 client.18186 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f2ce1f2b", "release": true, "await": true}] 2024-09-19T06:57:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.446+0000 7f53165df740 1 -- 172.21.15.103:0/754753220 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f2ce1f2b", "release": true, "await": true}) -- 0x55d5ac9a60a0 con 0x55d5acabf810 2024-09-19T06:57:00.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.447+0000 7f53117a1640 1 --2- 172.21.15.103:0/754753220 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d5acabf810 0x55d5acac1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.447+0000 7f52f57fa640 10 client.18186 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:00.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.447+0000 7f53117a1640 1 --2- 172.21.15.103:0/754753220 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d5acabf810 0x55d5acac1c30 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=756a5a7d5d0e2176 server_cookie=9c66fa5a67e319a2 in_seq=0 out_seq=0 2024-09-19T06:57:00.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.450+0000 7f52f57fa640 1 -- 172.21.15.103:0/754753220 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d5ac9a60a0 con 0x55d5acabf810 2024-09-19T06:57:00.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.450+0000 7f52f57fa640 10 client.18186 handle_command_reply: tid=0 2024-09-19T06:57:00.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.450+0000 7f53165df740 1 client.18186 shutdown 2024-09-19T06:57:00.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.450+0000 7f53165df740 1 -- 172.21.15.103:0/754753220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 msgr2=0x55d5aca48b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:00.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.450+0000 7f53165df740 1 --2- 172.21.15.103:0/754753220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca48b80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f530800a5c0 tx=0x7f530800a2d0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.450+0000 7f53165df740 1 -- 172.21.15.103:0/754753220 shutdown_connections 2024-09-19T06:57:00.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.450+0000 7f53165df740 1 --2- 172.21.15.103:0/754753220 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d5acabf810 0x55d5acac1c30 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.450+0000 7f53165df740 1 --2- 172.21.15.103:0/754753220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5aca44240 0x55d5acabda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.450+0000 7f53165df740 1 --2- 172.21.15.103:0/754753220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5aca39980 0x55d5aca48b80 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.450+0000 7f53165df740 1 --2- 172.21.15.103:0/754753220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5aca39060 0x55d5aca48640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.450+0000 7f52f57fa640 0 client.18186 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:00.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.450+0000 7f53165df740 1 -- 172.21.15.103:0/754753220 >> 172.21.15.103:0/754753220 conn(0x55d5aca355f0 msgr2=0x55d5aca34750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:00.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.450+0000 7f53165df740 1 -- 172.21.15.103:0/754753220 shutdown_connections 2024-09-19T06:57:00.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.450+0000 7f53165df740 1 -- 172.21.15.103:0/754753220 wait complete. 2024-09-19T06:57:00.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.451+0000 7f53165df740 20 client.18186 trim_cache size 0 max 16384 2024-09-19T06:57:00.458 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:57:00.458 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:57:00.458 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:57:00.458 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 28, 2024-09-19T06:57:00.458 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:57:00.458 INFO:teuthology.orchestra.run.smithi103.stdout: "f2ce1f2b": { 2024-09-19T06:57:00.458 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 28, 2024-09-19T06:57:00.458 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:57:00.458 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:57:00.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:57:00.459 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:57:00.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:57:00.459 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:57:00.459 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 145.8, 2024-09-19T06:57:00.459 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:57:00.459 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:57:00.459 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:57:00.459 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:57:00.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:57:00.459 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:57:00.460 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:00.460 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:00.460 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:00.460 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:00.460 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:00.460 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:57:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.451+0000 7f5314206640 1 -- 172.21.15.103:0/4240755643 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f52f80a33b0 msgr2=0x7f52f80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.451+0000 7f5314206640 1 --2- 172.21.15.103:0/4240755643 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f52f80a33b0 0x7f52f80a5870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f53040eeab0 tx=0x7f53040095c0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.451+0000 7f5314206640 1 -- 172.21.15.103:0/4240755643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f530c105ec0 msgr2=0x7f530c19e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.451+0000 7f5314206640 1 --2- 172.21.15.103:0/4240755643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f530c105ec0 0x7f530c19e900 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f53040097b0 tx=0x7f5304003f10 comp rx=0 tx=0).stop 2024-09-19T06:57:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.451+0000 7f5314206640 1 -- 172.21.15.103:0/4240755643 shutdown_connections 2024-09-19T06:57:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.451+0000 7f5314206640 1 --2- 172.21.15.103:0/4240755643 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f52f80a33b0 0x7f52f80a5870 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.451+0000 7f5314206640 1 --2- 172.21.15.103:0/4240755643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f530c105ec0 0x7f530c19e900 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.451+0000 7f5314206640 1 --2- 172.21.15.103:0/4240755643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f530c1011e0 0x7f530c1a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.451+0000 7f5314206640 1 --2- 172.21.15.103:0/4240755643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f530c1008c0 0x7f530c1a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.451+0000 7f5314206640 1 -- 172.21.15.103:0/4240755643 >> 172.21.15.103:0/4240755643 conn(0x7f530c0fc7d0 msgr2=0x7f530c103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:00.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.451+0000 7f5314206640 1 -- 172.21.15.103:0/4240755643 shutdown_connections 2024-09-19T06:57:00.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:00.451+0000 7f5314206640 1 -- 172.21.15.103:0/4240755643 wait complete. 2024-09-19T06:57:00.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:00 smithi174 ceph-mon[35637]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 15 op/s 2024-09-19T06:57:00.910 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:57:00.911 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:57:01.133 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f2ce1f2b', total seconds elapsed: 30.3 2024-09-19T06:57:01.133 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.75 2024-09-19T06:57:01.133 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:57:01.174 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.614+0000 7f4fe4f46640 1 -- 172.21.15.103:0/2173361646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe0102780 msgr2=0x7f4fe01067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.614+0000 7f4fe4f46640 1 --2- 172.21.15.103:0/2173361646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe0102780 0x7f4fe01067d0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009900 tx=0x7f4fc802a410 comp rx=0 tx=0).stop 2024-09-19T06:57:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.614+0000 7f4fe4f46640 1 -- 172.21.15.103:0/2173361646 shutdown_connections 2024-09-19T06:57:01.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.614+0000 7f4fe4f46640 1 --2- 172.21.15.103:0/2173361646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fe0106f00 0x7f4fe010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.614+0000 7f4fe4f46640 1 --2- 172.21.15.103:0/2173361646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe0102780 0x7f4fe01067d0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.614+0000 7f4fe4f46640 1 --2- 172.21.15.103:0/2173361646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fe0101dd0 0x7f4fe01021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.614+0000 7f4fe4f46640 1 -- 172.21.15.103:0/2173361646 >> 172.21.15.103:0/2173361646 conn(0x7f4fe00fd8b0 msgr2=0x7f4fe00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:01.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.614+0000 7f4fe4f46640 1 -- 172.21.15.103:0/2173361646 shutdown_connections 2024-09-19T06:57:01.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.614+0000 7f4fe4f46640 1 -- 172.21.15.103:0/2173361646 wait complete. 2024-09-19T06:57:01.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.614+0000 7f4fe4f46640 1 Processor -- start 2024-09-19T06:57:01.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fe4f46640 1 -- start start 2024-09-19T06:57:01.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fe4f46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fe0101dd0 0x7f4fe0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fe4f46640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fe0102780 0x7f4fe01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fe4f46640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe0106f00 0x7f4fe019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fe4f46640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe0111f60 con 0x7f4fe0106f00 2024-09-19T06:57:01.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fe4f46640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe0111de0 con 0x7f4fe0102780 2024-09-19T06:57:01.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fe4f46640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe01120e0 con 0x7f4fe0101dd0 2024-09-19T06:57:01.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fded9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe0106f00 0x7f4fe019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fd7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fe0102780 0x7f4fe01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fd7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fe0102780 0x7f4fe01989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:52360/0 (socket says 172.21.15.103:52360) 2024-09-19T06:57:01.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fded9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe0106f00 0x7f4fe019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45538/0 (socket says 172.21.15.103:45538) 2024-09-19T06:57:01.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fd7fff640 1 -- 172.21.15.103:0/1037063863 learned_addr learned my addr 172.21.15.103:0/1037063863 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:01.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fde59c640 1 --2- 172.21.15.103:0/1037063863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fe0101dd0 0x7f4fe0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fded9d640 1 -- 172.21.15.103:0/1037063863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fe0101dd0 msgr2=0x7f4fe0198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:01.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fded9d640 1 --2- 172.21.15.103:0/1037063863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fe0101dd0 0x7f4fe0198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fded9d640 1 -- 172.21.15.103:0/1037063863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fe0102780 msgr2=0x7f4fe01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:01.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fded9d640 1 --2- 172.21.15.103:0/1037063863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fe0102780 0x7f4fe01989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.615+0000 7f4fded9d640 1 -- 172.21.15.103:0/1037063863 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe019d430 con 0x7f4fe0106f00 2024-09-19T06:57:01.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.616+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/1037063863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fe0102780 0x7f4fe01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:57:01.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.616+0000 7f4fded9d640 1 --2- 172.21.15.103:0/1037063863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe0106f00 0x7f4fe019cd50 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800ca30 tx=0x7f4fd800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:01.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.616+0000 7f4fd77fe640 1 -- 172.21.15.103:0/1037063863 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8007d90 con 0x7f4fe0106f00 2024-09-19T06:57:01.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.616+0000 7f4fe4f46640 1 -- 172.21.15.103:0/1037063863 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe019d720 con 0x7f4fe0106f00 2024-09-19T06:57:01.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.616+0000 7f4fd77fe640 1 -- 172.21.15.103:0/1037063863 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd8005020 con 0x7f4fe0106f00 2024-09-19T06:57:01.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.616+0000 7f4fd77fe640 1 -- 172.21.15.103:0/1037063863 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd800f780 con 0x7f4fe0106f00 2024-09-19T06:57:01.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.616+0000 7f4fe4f46640 1 -- 172.21.15.103:0/1037063863 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe01a5000 con 0x7f4fe0106f00 2024-09-19T06:57:01.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.618+0000 7f4fd77fe640 1 -- 172.21.15.103:0/1037063863 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4fd80051d0 con 0x7f4fe0106f00 2024-09-19T06:57:01.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.619+0000 7f4fe731f740 1 Processor -- start 2024-09-19T06:57:01.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.619+0000 7f4fe731f740 1 -- start start 2024-09-19T06:57:01.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.619+0000 7f4fe731f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af960fb60 0x558af960ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.619+0000 7f4fe731f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af961a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.619+0000 7f4fe731f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af961ad40 0x558af961d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.619+0000 7f4fe731f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558af9575280 con 0x558af960fb60 2024-09-19T06:57:01.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.619+0000 7f4fe731f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558af951cb10 con 0x558af961ad40 2024-09-19T06:57:01.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.619+0000 7f4fe731f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558af9571240 con 0x558af9610480 2024-09-19T06:57:01.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.619+0000 7f4fd77fe640 1 --2- 172.21.15.103:0/1037063863 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fcc0a33b0 0x7f4fcc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.619+0000 7f4fd7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af960fb60 0x558af960ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.619+0000 7f4fd7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af960fb60 0x558af960ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45552/0 (socket says 172.21.15.103:45552) 2024-09-19T06:57:01.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.620+0000 7f4fd7fff640 1 -- 172.21.15.103:0/2008114639 learned_addr learned my addr 172.21.15.103:0/2008114639 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:01.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.620+0000 7f4fde59c640 1 --2- 172.21.15.103:0/2008114639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af961a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.620+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/2008114639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af961ad40 0x558af961d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.620+0000 7f4fded9d640 1 --2- 172.21.15.103:0/1037063863 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fcc0a33b0 0x7f4fcc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.620+0000 7f4fd7fff640 1 -- 172.21.15.103:0/2008114639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 msgr2=0x558af961a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:01.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.620+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/2008114639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af961a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.620+0000 7f4fd7fff640 1 -- 172.21.15.103:0/2008114639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af961ad40 msgr2=0x558af961d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:01.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.620+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/2008114639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af961ad40 0x558af961d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.620+0000 7f4fd7fff640 1 -- 172.21.15.103:0/2008114639 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af95291f0 con 0x558af960fb60 2024-09-19T06:57:01.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.620+0000 7f4fd77fe640 1 -- 172.21.15.103:0/1037063863 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f4fd80cfea0 con 0x7f4fe0106f00 2024-09-19T06:57:01.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.620+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/2008114639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af960fb60 0x558af960ff40 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80096c0 tx=0x7f4fc8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d043f212f89da9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:01.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.620+0000 7f4fd4ff9640 1 -- 172.21.15.103:0/2008114639 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8044020 con 0x558af960fb60 2024-09-19T06:57:01.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.620+0000 7f4fd4ff9640 1 -- 172.21.15.103:0/2008114639 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fc8005520 con 0x558af960fb60 2024-09-19T06:57:01.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.620+0000 7f4fd4ff9640 1 -- 172.21.15.103:0/2008114639 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8041030 con 0x558af960fb60 2024-09-19T06:57:01.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.621+0000 7f4fe731f740 1 -- 172.21.15.103:0/2008114639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af960fb60 msgr2=0x558af960ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:01.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.621+0000 7f4fe731f740 1 --2- 172.21.15.103:0/2008114639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af960fb60 0x558af960ff40 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80096c0 tx=0x7f4fc8005a20 comp rx=0 tx=0).stop 2024-09-19T06:57:01.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.621+0000 7f4fded9d640 1 --2- 172.21.15.103:0/1037063863 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fcc0a33b0 0x7f4fcc0a5870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80cd680 tx=0x7f4fd8066c00 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:01.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.621+0000 7f4fe731f740 1 -- 172.21.15.103:0/2008114639 shutdown_connections 2024-09-19T06:57:01.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.621+0000 7f4fe731f740 1 --2- 172.21.15.103:0/2008114639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af961ad40 0x558af961d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.621+0000 7f4fe731f740 1 --2- 172.21.15.103:0/2008114639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af961a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.621+0000 7f4fe731f740 1 --2- 172.21.15.103:0/2008114639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af960fb60 0x558af960ff40 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.621+0000 7f4fe731f740 1 -- 172.21.15.103:0/2008114639 >> 172.21.15.103:0/2008114639 conn(0x558af960c0f0 msgr2=0x558af960c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:01.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.621+0000 7f4fe731f740 1 -- 172.21.15.103:0/2008114639 shutdown_connections 2024-09-19T06:57:01.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.621+0000 7f4fe731f740 1 -- 172.21.15.103:0/2008114639 wait complete. 2024-09-19T06:57:01.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fe731f740 1 -- start start 2024-09-19T06:57:01.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fe731f740 1 Processor -- start 2024-09-19T06:57:01.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fe731f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af960fb60 0x558af9617c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fe731f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af96181b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fe731f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af96186f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fe731f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558af961d910 con 0x558af961ad40 2024-09-19T06:57:01.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fe731f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558af952ff10 con 0x558af960fb60 2024-09-19T06:57:01.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fe731f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558af961da90 con 0x558af9610480 2024-09-19T06:57:01.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fe731f740 10 client.0 _ll_register_callbacks cb 0x558af94f8a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:01.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fd7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af96186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fd7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af96186f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45564/0 (socket says 172.21.15.103:45564) 2024-09-19T06:57:01.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fd7fff640 1 -- 172.21.15.103:0/1061929149 learned_addr learned my addr 172.21.15.103:0/1061929149 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:01.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/1061929149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af960fb60 0x558af9617c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fde59c640 1 --2- 172.21.15.103:0/1061929149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af96181b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fd7fff640 1 -- 172.21.15.103:0/1061929149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 msgr2=0x558af96181b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:01.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/1061929149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af96181b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fd7fff640 1 -- 172.21.15.103:0/1061929149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af960fb60 msgr2=0x558af9617c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:01.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/1061929149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af960fb60 0x558af9617c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.622+0000 7f4fd7fff640 1 -- 172.21.15.103:0/1061929149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af92e66a0 con 0x558af961ad40 2024-09-19T06:57:01.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.623+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/1061929149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af96186f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80047f0 tx=0x7f4fc8031780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc9530ee1b57f33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:01.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.623+0000 7f4fc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:01.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.623+0000 7f4fc5ffb640 1 -- 172.21.15.103:0/1061929149 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc804f020 con 0x558af961ad40 2024-09-19T06:57:01.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.623+0000 7f4fc5ffb640 1 -- 172.21.15.103:0/1061929149 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f4fc8004100 con 0x558af961ad40 2024-09-19T06:57:01.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.623+0000 7f4fe731f740 1 -- 172.21.15.103:0/1061929149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fc8005520 con 0x558af961ad40 2024-09-19T06:57:01.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.623+0000 7f4fc5ffb640 1 -- 172.21.15.103:0/1061929149 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4fc805b050 con 0x558af961ad40 2024-09-19T06:57:01.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.623+0000 7f4fe731f740 10 client.18198 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:01.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.623+0000 7f4fe731f740 1 -- 172.21.15.103:0/1061929149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558af9629550 con 0x558af961ad40 2024-09-19T06:57:01.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.623+0000 7f4fc5ffb640 1 -- 172.21.15.103:0/1061929149 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fc8005520 con 0x558af961ad40 2024-09-19T06:57:01.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.624+0000 7f4fc5ffb640 1 -- 172.21.15.103:0/1061929149 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4fc80039f0 con 0x558af961ad40 2024-09-19T06:57:01.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.624+0000 7f4fe731f740 10 client.18198 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:01.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.624+0000 7f4fe731f740 10 client.18198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:01.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.624+0000 7f4fe731f740 20 client.18198 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:01.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.624+0000 7f4fe731f740 1 --2- 172.21.15.103:0/1061929149 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558af962a8a0 0x558af962cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.624+0000 7f4fe731f740 4 client.18198 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:01.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.624+0000 7f4fe731f740 1 -- 172.21.15.103:0/1061929149 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558af95827f0 con 0x558af962a8a0 2024-09-19T06:57:01.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.624+0000 7f4fde59c640 1 --2- 172.21.15.103:0/1061929149 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558af962a8a0 0x558af962cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.625+0000 7f4fde59c640 1 --2- 172.21.15.103:0/1061929149 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558af962a8a0 0x558af962cc90 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ed9cb86de31592 server_cookie=72e10bdf5e228b30 in_seq=0 out_seq=0 2024-09-19T06:57:01.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.625+0000 7f4fc5ffb640 10 client.18198 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:01.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.625+0000 7f4fc5ffb640 1 -- 172.21.15.103:0/1061929149 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558af95827f0 con 0x558af962a8a0 2024-09-19T06:57:01.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.625+0000 7f4fc5ffb640 10 client.18198 handle_command_reply: tid=0 2024-09-19T06:57:01.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.625+0000 7f4fe731f740 1 client.18198 shutdown 2024-09-19T06:57:01.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.625+0000 7f4fe731f740 1 -- 172.21.15.103:0/1061929149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 msgr2=0x558af96186f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:01.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.625+0000 7f4fe731f740 1 --2- 172.21.15.103:0/1061929149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af96186f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80047f0 tx=0x7f4fc8031780 comp rx=0 tx=0).stop 2024-09-19T06:57:01.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.625+0000 7f4fe731f740 1 -- 172.21.15.103:0/1061929149 shutdown_connections 2024-09-19T06:57:01.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.625+0000 7f4fe731f740 1 --2- 172.21.15.103:0/1061929149 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558af962a8a0 0x558af962cc90 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.625+0000 7f4fe731f740 1 --2- 172.21.15.103:0/1061929149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af96186f0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.625+0000 7f4fe731f740 1 --2- 172.21.15.103:0/1061929149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af96181b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.625+0000 7f4fe731f740 1 --2- 172.21.15.103:0/1061929149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af960fb60 0x558af9617c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.625+0000 7f4fe731f740 1 -- 172.21.15.103:0/1061929149 >> 172.21.15.103:0/1061929149 conn(0x558af960c0f0 msgr2=0x558af9611e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:01.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.626+0000 7f4fc5ffb640 0 client.18198 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:01.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.626+0000 7f4fe731f740 1 -- 172.21.15.103:0/1061929149 shutdown_connections 2024-09-19T06:57:01.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.626+0000 7f4fe731f740 1 -- 172.21.15.103:0/1061929149 wait complete. 2024-09-19T06:57:01.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.626+0000 7f4fe731f740 20 client.18198 trim_cache size 0 max 16384 2024-09-19T06:57:01.643 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:01.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.644+0000 7f4fe731f740 1 Processor -- start 2024-09-19T06:57:01.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.644+0000 7f4fe731f740 1 -- start start 2024-09-19T06:57:01.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.644+0000 7f4fe731f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af960fb60 0x558af96263f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.644+0000 7f4fe731f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af962f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.644+0000 7f4fe731f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af962fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.644+0000 7f4fe731f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe01ad760 con 0x558af961ad40 2024-09-19T06:57:01.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.644+0000 7f4fe731f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe01af2a0 con 0x558af960fb60 2024-09-19T06:57:01.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.644+0000 7f4fe731f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe01ade30 con 0x558af9610480 2024-09-19T06:57:01.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.645+0000 7f4fd7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af962fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.645+0000 7f4fd7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af962fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45568/0 (socket says 172.21.15.103:45568) 2024-09-19T06:57:01.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.645+0000 7f4fd7fff640 1 -- 172.21.15.103:0/893006 learned_addr learned my addr 172.21.15.103:0/893006 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:01.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.645+0000 7f4fde59c640 1 --2- 172.21.15.103:0/893006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af962f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.645+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/893006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af960fb60 0x558af96263f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.645+0000 7f4fd7fff640 1 -- 172.21.15.103:0/893006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 msgr2=0x558af962f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:01.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.645+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/893006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af962f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.645+0000 7f4fd7fff640 1 -- 172.21.15.103:0/893006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af960fb60 msgr2=0x558af96263f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:01.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.645+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/893006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af960fb60 0x558af96263f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.645+0000 7f4fd7fff640 1 -- 172.21.15.103:0/893006 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af96102b0 con 0x558af961ad40 2024-09-19T06:57:01.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.645+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/893006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af962fe10 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8005920 tx=0x7f4fc8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60748d2b8617bd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:01.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.645+0000 7f4fc7fff640 1 -- 172.21.15.103:0/893006 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc804f030 con 0x558af961ad40 2024-09-19T06:57:01.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.645+0000 7f4fc7fff640 1 -- 172.21.15.103:0/893006 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fc803d450 con 0x558af961ad40 2024-09-19T06:57:01.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.645+0000 7f4fc7fff640 1 -- 172.21.15.103:0/893006 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8041050 con 0x558af961ad40 2024-09-19T06:57:01.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.646+0000 7f4fe731f740 1 -- 172.21.15.103:0/893006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 msgr2=0x558af962fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:01.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.646+0000 7f4fe731f740 1 --2- 172.21.15.103:0/893006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af962fe10 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8005920 tx=0x7f4fc8005a20 comp rx=0 tx=0).stop 2024-09-19T06:57:01.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.646+0000 7f4fe731f740 1 -- 172.21.15.103:0/893006 shutdown_connections 2024-09-19T06:57:01.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.646+0000 7f4fe731f740 1 --2- 172.21.15.103:0/893006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af962fe10 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.646+0000 7f4fe731f740 1 --2- 172.21.15.103:0/893006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af962f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.646+0000 7f4fe731f740 1 --2- 172.21.15.103:0/893006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af960fb60 0x558af96263f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.646+0000 7f4fe731f740 1 -- 172.21.15.103:0/893006 >> 172.21.15.103:0/893006 conn(0x558af960c0f0 msgr2=0x558af9625ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:01.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.646+0000 7f4fe731f740 1 -- 172.21.15.103:0/893006 shutdown_connections 2024-09-19T06:57:01.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.646+0000 7f4fe731f740 1 -- 172.21.15.103:0/893006 wait complete. 2024-09-19T06:57:01.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.647+0000 7f4fe731f740 1 -- start start 2024-09-19T06:57:01.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.647+0000 7f4fe731f740 1 Processor -- start 2024-09-19T06:57:01.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.647+0000 7f4fe731f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af960fb60 0x558af962cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.647+0000 7f4fe731f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af962d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.647+0000 7f4fe731f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af968d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.647+0000 7f4fe731f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe01aebd0 con 0x558af961ad40 2024-09-19T06:57:01.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.647+0000 7f4fe731f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe01ae500 con 0x558af960fb60 2024-09-19T06:57:01.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.647+0000 7f4fe731f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe01ab760 con 0x558af9610480 2024-09-19T06:57:01.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.647+0000 7f4fe731f740 10 client.0 _ll_register_callbacks cb 0x7f4fd000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:01.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.647+0000 7f4fd7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af968d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.647+0000 7f4fde59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af962d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.647+0000 7f4fd7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af968d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45570/0 (socket says 172.21.15.103:45570) 2024-09-19T06:57:01.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.647+0000 7f4fd7fff640 1 -- 172.21.15.103:0/2809607603 learned_addr learned my addr 172.21.15.103:0/2809607603 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:01.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.647+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/2809607603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af960fb60 0x558af962cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.648+0000 7f4fd7fff640 1 -- 172.21.15.103:0/2809607603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 msgr2=0x558af962d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:01.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.648+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/2809607603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af962d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.648+0000 7f4fd7fff640 1 -- 172.21.15.103:0/2809607603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af960fb60 msgr2=0x558af962cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.648+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/2809607603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af960fb60 0x558af962cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.648+0000 7f4fd7fff640 1 -- 172.21.15.103:0/2809607603 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af968d850 con 0x558af961ad40 2024-09-19T06:57:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.648+0000 7f4fde59c640 1 --2- 172.21.15.103:0/2809607603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af962d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:57:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.648+0000 7f4fd7fff640 1 --2- 172.21.15.103:0/2809607603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af968d210 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8031550 tx=0x7f4fc80044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adab45b0d82e777f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.648+0000 7f4fc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.648+0000 7f4fc5ffb640 1 -- 172.21.15.103:0/2809607603 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8041030 con 0x558af961ad40 2024-09-19T06:57:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.648+0000 7f4fc5ffb640 1 -- 172.21.15.103:0/2809607603 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f4fc80049d0 con 0x558af961ad40 2024-09-19T06:57:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.648+0000 7f4fe731f740 1 -- 172.21.15.103:0/2809607603 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fc803d450 con 0x558af961ad40 2024-09-19T06:57:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.648+0000 7f4fc5ffb640 1 -- 172.21.15.103:0/2809607603 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4fc8066040 con 0x558af961ad40 2024-09-19T06:57:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.648+0000 7f4fe731f740 10 client.18204 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.648+0000 7f4fe731f740 1 -- 172.21.15.103:0/2809607603 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558af9694bb0 con 0x558af961ad40 2024-09-19T06:57:01.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.649+0000 7f4fc5ffb640 1 -- 172.21.15.103:0/2809607603 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fc803d450 con 0x558af961ad40 2024-09-19T06:57:01.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.649+0000 7f4fc5ffb640 1 -- 172.21.15.103:0/2809607603 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4fc806c070 con 0x558af961ad40 2024-09-19T06:57:01.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.649+0000 7f4fe731f740 10 client.18204 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:01.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.649+0000 7f4fe731f740 10 client.18204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:01.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.649+0000 7f4fe731f740 20 client.18204 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:01.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.649+0000 7f4fe731f740 1 --2- 172.21.15.103:0/2809607603 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558af9695240 0x558af9697660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:01.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.649+0000 7f4fe731f740 4 client.18204 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:57:01.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.649+0000 7f4fe731f740 1 -- 172.21.15.103:0/2809607603 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558af957cd00 con 0x558af9695240 2024-09-19T06:57:01.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.649+0000 7f4fde59c640 1 --2- 172.21.15.103:0/2809607603 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558af9695240 0x558af9697660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:01.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.650+0000 7f4fde59c640 1 --2- 172.21.15.103:0/2809607603 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558af9695240 0x558af9697660 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a31e93db4c8282d0 server_cookie=2a1a8c5b417fd1ca in_seq=0 out_seq=0 2024-09-19T06:57:01.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.650+0000 7f4fc5ffb640 10 client.18204 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:01.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.651+0000 7f4fc5ffb640 1 -- 172.21.15.103:0/2809607603 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558af957cd00 con 0x558af9695240 2024-09-19T06:57:01.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.651+0000 7f4fc5ffb640 10 client.18204 handle_command_reply: tid=0 2024-09-19T06:57:01.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.651+0000 7f4fe731f740 1 client.18204 shutdown 2024-09-19T06:57:01.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.651+0000 7f4fe731f740 1 -- 172.21.15.103:0/2809607603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 msgr2=0x558af968d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:01.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.651+0000 7f4fe731f740 1 --2- 172.21.15.103:0/2809607603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af968d210 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8031550 tx=0x7f4fc80044e0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.651+0000 7f4fe731f740 1 -- 172.21.15.103:0/2809607603 shutdown_connections 2024-09-19T06:57:01.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.651+0000 7f4fe731f740 1 --2- 172.21.15.103:0/2809607603 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558af9695240 0x558af9697660 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.651+0000 7f4fc5ffb640 0 client.18204 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:01.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.651+0000 7f4fe731f740 1 --2- 172.21.15.103:0/2809607603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af961ad40 0x558af968d210 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.651+0000 7f4fe731f740 1 --2- 172.21.15.103:0/2809607603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9610480 0x558af962d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.651+0000 7f4fe731f740 1 --2- 172.21.15.103:0/2809607603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af960fb60 0x558af962cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.651+0000 7f4fe731f740 1 -- 172.21.15.103:0/2809607603 >> 172.21.15.103:0/2809607603 conn(0x558af960c0f0 msgr2=0x558af9618450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:01.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.651+0000 7f4fe731f740 1 -- 172.21.15.103:0/2809607603 shutdown_connections 2024-09-19T06:57:01.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.651+0000 7f4fe731f740 1 -- 172.21.15.103:0/2809607603 wait complete. 2024-09-19T06:57:01.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.652+0000 7f4fe731f740 20 client.18204 trim_cache size 0 max 16384 2024-09-19T06:57:01.656 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:57:01.656 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T06:57:01.656 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "9871dc0a-2a04-4e9f-8b27-e2581cffb01f", 2024-09-19T06:57:01.656 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T06:57:01.656 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:57:01.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.652+0000 7f4fe4f46640 1 -- 172.21.15.103:0/1037063863 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fcc0a33b0 msgr2=0x7f4fcc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:01.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.652+0000 7f4fe4f46640 1 --2- 172.21.15.103:0/1037063863 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fcc0a33b0 0x7f4fcc0a5870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80cd680 tx=0x7f4fd8066c00 comp rx=0 tx=0).stop 2024-09-19T06:57:01.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.652+0000 7f4fe4f46640 1 -- 172.21.15.103:0/1037063863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe0106f00 msgr2=0x7f4fe019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:01.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.652+0000 7f4fe4f46640 1 --2- 172.21.15.103:0/1037063863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe0106f00 0x7f4fe019cd50 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800ca30 tx=0x7f4fd800cef0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.653+0000 7f4fe4f46640 1 -- 172.21.15.103:0/1037063863 shutdown_connections 2024-09-19T06:57:01.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.653+0000 7f4fe4f46640 1 --2- 172.21.15.103:0/1037063863 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fcc0a33b0 0x7f4fcc0a5870 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.653+0000 7f4fe4f46640 1 --2- 172.21.15.103:0/1037063863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe0106f00 0x7f4fe019cd50 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.653+0000 7f4fe4f46640 1 --2- 172.21.15.103:0/1037063863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fe0102780 0x7f4fe01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.653+0000 7f4fe4f46640 1 --2- 172.21.15.103:0/1037063863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fe0101dd0 0x7f4fe0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:01.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.653+0000 7f4fe4f46640 1 -- 172.21.15.103:0/1037063863 >> 172.21.15.103:0/1037063863 conn(0x7f4fe00fd8b0 msgr2=0x7f4fe00fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:01.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.653+0000 7f4fe4f46640 1 -- 172.21.15.103:0/1037063863 shutdown_connections 2024-09-19T06:57:01.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:01.653+0000 7f4fe4f46640 1 -- 172.21.15.103:0/1037063863 wait complete. 2024-09-19T06:57:01.999 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:01 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T06:57:01.999 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:01 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:57:01.999 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:01 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T06:57:01.999 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:01 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:57:02.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:01 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T06:57:02.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:01 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:57:02.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:01 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T06:57:02.279 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:01 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:57:02.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:01 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T06:57:02.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:01 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:57:02.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:01 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T06:57:02.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:01 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:57:02.506 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:57:02.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.677+0000 7fd42915d640 1 -- 172.21.15.103:0/1578441714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd424102ef0 msgr2=0x7fd42410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:02.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.677+0000 7fd42915d640 1 --2- 172.21.15.103:0/1578441714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd424102ef0 0x7fd42410da40 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009ca0 tx=0x7fd41c02a950 comp rx=0 tx=0).stop 2024-09-19T06:57:02.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.677+0000 7fd42915d640 1 -- 172.21.15.103:0/1578441714 shutdown_connections 2024-09-19T06:57:02.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.677+0000 7fd42915d640 1 --2- 172.21.15.103:0/1578441714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd424102ef0 0x7fd42410da40 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.677+0000 7fd42915d640 1 --2- 172.21.15.103:0/1578441714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd424102570 0x7fd4241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.677+0000 7fd42915d640 1 --2- 172.21.15.103:0/1578441714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd424100e20 0x7fd424101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.677+0000 7fd42915d640 1 -- 172.21.15.103:0/1578441714 >> 172.21.15.103:0/1578441714 conn(0x7fd4240fc7d0 msgr2=0x7fd4240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:02.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.677+0000 7fd42915d640 1 -- 172.21.15.103:0/1578441714 shutdown_connections 2024-09-19T06:57:02.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.677+0000 7fd42915d640 1 -- 172.21.15.103:0/1578441714 wait complete. 2024-09-19T06:57:02.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.678+0000 7fd42915d640 1 Processor -- start 2024-09-19T06:57:02.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.678+0000 7fd42915d640 1 -- start start 2024-09-19T06:57:02.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.678+0000 7fd42915d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd424100e20 0x7fd424197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:02.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.678+0000 7fd42915d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd424102570 0x7fd424197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:02.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.678+0000 7fd42915d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd424102ef0 0x7fd42419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:02.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.678+0000 7fd42915d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd424110fa0 con 0x7fd424102ef0 2024-09-19T06:57:02.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.678+0000 7fd42915d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd424110e20 con 0x7fd424102570 2024-09-19T06:57:02.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.678+0000 7fd42915d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd424111120 con 0x7fd424100e20 2024-09-19T06:57:02.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.678+0000 7fd42359e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd424102ef0 0x7fd42419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:02.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.679+0000 7fd42259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd424102570 0x7fd424197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:02.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.679+0000 7fd42359e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd424102ef0 0x7fd42419bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45584/0 (socket says 172.21.15.103:45584) 2024-09-19T06:57:02.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.679+0000 7fd42359e640 1 -- 172.21.15.103:0/2876379456 learned_addr learned my addr 172.21.15.103:0/2876379456 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:02.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.679+0000 7fd422d9d640 1 --2- 172.21.15.103:0/2876379456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd424100e20 0x7fd424197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:02.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.679+0000 7fd42359e640 1 -- 172.21.15.103:0/2876379456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd424100e20 msgr2=0x7fd424197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:02.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.679+0000 7fd42359e640 1 --2- 172.21.15.103:0/2876379456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd424100e20 0x7fd424197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.679+0000 7fd42359e640 1 -- 172.21.15.103:0/2876379456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd424102570 msgr2=0x7fd424197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:02.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.679+0000 7fd42359e640 1 --2- 172.21.15.103:0/2876379456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd424102570 0x7fd424197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.679+0000 7fd42359e640 1 -- 172.21.15.103:0/2876379456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42419c3f0 con 0x7fd424102ef0 2024-09-19T06:57:02.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.679+0000 7fd42359e640 1 --2- 172.21.15.103:0/2876379456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd424102ef0 0x7fd42419bd10 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fd41c02ae70 tx=0x7fd41c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:02.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.679+0000 7fd40bfff640 1 -- 172.21.15.103:0/2876379456 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c034040 con 0x7fd424102ef0 2024-09-19T06:57:02.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.679+0000 7fd40bfff640 1 -- 172.21.15.103:0/2876379456 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd41c030920 con 0x7fd424102ef0 2024-09-19T06:57:02.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.679+0000 7fd40bfff640 1 -- 172.21.15.103:0/2876379456 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c0047f0 con 0x7fd424102ef0 2024-09-19T06:57:02.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.679+0000 7fd42915d640 1 -- 172.21.15.103:0/2876379456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42419c680 con 0x7fd424102ef0 2024-09-19T06:57:02.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.679+0000 7fd42915d640 1 -- 172.21.15.103:0/2876379456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd424069f70 con 0x7fd424102ef0 2024-09-19T06:57:02.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.681+0000 7fd42915d640 1 -- 172.21.15.103:0/2876379456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4241021b0 con 0x7fd424102ef0 2024-09-19T06:57:02.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.681+0000 7fd40bfff640 1 -- 172.21.15.103:0/2876379456 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd41c037020 con 0x7fd424102ef0 2024-09-19T06:57:02.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.682+0000 7fd40bfff640 1 --2- 172.21.15.103:0/2876379456 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd40c0a33b0 0x7fd40c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:02.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.682+0000 7fd40bfff640 1 -- 172.21.15.103:0/2876379456 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fd41c0f0af0 con 0x7fd424102ef0 2024-09-19T06:57:02.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.682+0000 7fd422d9d640 1 --2- 172.21.15.103:0/2876379456 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd40c0a33b0 0x7fd40c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:02.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.683+0000 7fd422d9d640 1 --2- 172.21.15.103:0/2876379456 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd40c0a33b0 0x7fd40c0a5870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd414009990 tx=0x7fd414009110 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:02.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.687+0000 7fd40bfff640 1 -- 172.21.15.103:0/2876379456 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd41c0bd5c0 con 0x7fd424102ef0 2024-09-19T06:57:02.802 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:02.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.912+0000 7fd42915d640 1 -- 172.21.15.103:0/2876379456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4241987a0 con 0x7fd424102ef0 2024-09-19T06:57:02.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.912+0000 7fd40bfff640 1 -- 172.21.15.103:0/2876379456 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fd41c0bcec0 con 0x7fd424102ef0 2024-09-19T06:57:02.914 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:57:02.915 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:57:02.916 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:57:02.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.914+0000 7fd42915d640 1 -- 172.21.15.103:0/2876379456 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd40c0a33b0 msgr2=0x7fd40c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:02.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.914+0000 7fd42915d640 1 --2- 172.21.15.103:0/2876379456 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd40c0a33b0 0x7fd40c0a5870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd414009990 tx=0x7fd414009110 comp rx=0 tx=0).stop 2024-09-19T06:57:02.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.914+0000 7fd42915d640 1 -- 172.21.15.103:0/2876379456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd424102ef0 msgr2=0x7fd42419bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:02.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.914+0000 7fd42915d640 1 --2- 172.21.15.103:0/2876379456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd424102ef0 0x7fd42419bd10 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fd41c02ae70 tx=0x7fd41c02f5d0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.914+0000 7fd42915d640 1 -- 172.21.15.103:0/2876379456 shutdown_connections 2024-09-19T06:57:02.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.914+0000 7fd42915d640 1 --2- 172.21.15.103:0/2876379456 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd40c0a33b0 0x7fd40c0a5870 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.914+0000 7fd42915d640 1 --2- 172.21.15.103:0/2876379456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd424102ef0 0x7fd42419bd10 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.914+0000 7fd42915d640 1 --2- 172.21.15.103:0/2876379456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd424102570 0x7fd424197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.914+0000 7fd42915d640 1 --2- 172.21.15.103:0/2876379456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd424100e20 0x7fd424197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.914+0000 7fd42915d640 1 -- 172.21.15.103:0/2876379456 >> 172.21.15.103:0/2876379456 conn(0x7fd4240fc7d0 msgr2=0x7fd424106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:02.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.915+0000 7fd42915d640 1 -- 172.21.15.103:0/2876379456 shutdown_connections 2024-09-19T06:57:02.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:02.915+0000 7fd42915d640 1 -- 172.21.15.103:0/2876379456 wait complete. 2024-09-19T06:57:03.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:02 smithi103 ceph-mon[31158]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:57:03.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:02 smithi134 ceph-mon[35356]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:57:03.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:02 smithi174 ceph-mon[35637]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:57:03.748 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0' 2024-09-19T06:57:03.749 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0 2024-09-19T06:57:03.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.921+0000 7fee2e326640 1 -- 172.21.15.103:0/859890693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee281036e0 msgr2=0x7fee28107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:03.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.921+0000 7fee2e326640 1 --2- 172.21.15.103:0/859890693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee281036e0 0x7fee28107b90 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fee10009930 tx=0x7fee1002a420 comp rx=0 tx=0).stop 2024-09-19T06:57:03.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.921+0000 7fee2e326640 1 -- 172.21.15.103:0/859890693 shutdown_connections 2024-09-19T06:57:03.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.921+0000 7fee2e326640 1 --2- 172.21.15.103:0/859890693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee281082a0 0x7fee2810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.921+0000 7fee2e326640 1 --2- 172.21.15.103:0/859890693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee281036e0 0x7fee28107b90 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.921+0000 7fee2e326640 1 --2- 172.21.15.103:0/859890693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee28102d30 0x7fee28103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.921+0000 7fee2e326640 1 -- 172.21.15.103:0/859890693 >> 172.21.15.103:0/859890693 conn(0x7fee280fc7d0 msgr2=0x7fee280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:03.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.921+0000 7fee2e326640 1 -- 172.21.15.103:0/859890693 shutdown_connections 2024-09-19T06:57:03.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.921+0000 7fee2e326640 1 -- 172.21.15.103:0/859890693 wait complete. 2024-09-19T06:57:03.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.922+0000 7fee2e326640 1 Processor -- start 2024-09-19T06:57:03.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.922+0000 7fee2e326640 1 -- start start 2024-09-19T06:57:03.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.922+0000 7fee2e326640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee28102d30 0x7fee281985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.922+0000 7fee2e326640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee281036e0 0x7fee28198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.922+0000 7fee2e326640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee281082a0 0x7fee2819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.922+0000 7fee2e326640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee281120a0 con 0x7fee28102d30 2024-09-19T06:57:03.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.922+0000 7fee2e326640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fee28111f20 con 0x7fee281082a0 2024-09-19T06:57:03.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.922+0000 7fee2e326640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee28112220 con 0x7fee281036e0 2024-09-19T06:57:03.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.922+0000 7fee27fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee28102d30 0x7fee281985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.923+0000 7fee277fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee281036e0 0x7fee28198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.923+0000 7fee27fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee28102d30 0x7fee281985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45610/0 (socket says 172.21.15.103:45610) 2024-09-19T06:57:03.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.923+0000 7fee27fff640 1 -- 172.21.15.103:0/818556518 learned_addr learned my addr 172.21.15.103:0/818556518 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:03.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.923+0000 7fee2c8c3640 1 --2- 172.21.15.103:0/818556518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee281082a0 0x7fee2819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.923+0000 7fee27fff640 1 -- 172.21.15.103:0/818556518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee281036e0 msgr2=0x7fee28198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:03.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.923+0000 7fee27fff640 1 --2- 172.21.15.103:0/818556518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee281036e0 0x7fee28198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.923+0000 7fee27fff640 1 -- 172.21.15.103:0/818556518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee281082a0 msgr2=0x7fee2819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:03.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.923+0000 7fee27fff640 1 --2- 172.21.15.103:0/818556518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee281082a0 0x7fee2819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.923+0000 7fee27fff640 1 -- 172.21.15.103:0/818556518 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2819d570 con 0x7fee28102d30 2024-09-19T06:57:03.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.923+0000 7fee27fff640 1 --2- 172.21.15.103:0/818556518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee28102d30 0x7fee281985c0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fee1800d950 tx=0x7fee1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:03.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.923+0000 7fee257fa640 1 -- 172.21.15.103:0/818556518 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18017070 con 0x7fee28102d30 2024-09-19T06:57:03.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.923+0000 7fee257fa640 1 -- 172.21.15.103:0/818556518 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee18009c30 con 0x7fee28102d30 2024-09-19T06:57:03.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.923+0000 7fee257fa640 1 -- 172.21.15.103:0/818556518 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1800f6b0 con 0x7fee28102d30 2024-09-19T06:57:03.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.923+0000 7fee2e326640 1 -- 172.21.15.103:0/818556518 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2819d860 con 0x7fee28102d30 2024-09-19T06:57:03.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.924+0000 7fee2e326640 1 -- 172.21.15.103:0/818556518 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee281a5140 con 0x7fee28102d30 2024-09-19T06:57:03.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.926+0000 7fee257fa640 1 -- 172.21.15.103:0/818556518 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fee180109d0 con 0x7fee28102d30 2024-09-19T06:57:03.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.926+0000 7fee257fa640 1 --2- 172.21.15.103:0/818556518 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fee140a3430 0x7fee140a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.926+0000 7fee306ff740 1 Processor -- start 2024-09-19T06:57:03.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.926+0000 7fee306ff740 1 -- start start 2024-09-19T06:57:03.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.926+0000 7fee257fa640 1 -- 172.21.15.103:0/818556518 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fee180cf880 con 0x7fee28102d30 2024-09-19T06:57:03.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.926+0000 7fee277fe640 1 --2- 172.21.15.103:0/818556518 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fee140a3430 0x7fee140a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee306ff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e7524a0 0x560d1e752880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee306ff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e752dc0 0x560d1e75d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee306ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e75fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee306ff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560d1e6b82e0 con 0x560d1e752dc0 2024-09-19T06:57:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee306ff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560d1e65fce0 con 0x560d1e7524a0 2024-09-19T06:57:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee306ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560d1e6b42a0 con 0x560d1e75d680 2024-09-19T06:57:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee277fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e752dc0 0x560d1e75d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee27fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e7524a0 0x560d1e752880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee27fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e7524a0 0x560d1e752880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:52444/0 (socket says 172.21.15.103:52444) 2024-09-19T06:57:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee27fff640 1 -- 172.21.15.103:0/3405992230 learned_addr learned my addr 172.21.15.103:0/3405992230 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee2c8c3640 1 --2- 172.21.15.103:0/3405992230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e75fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee277fe640 1 --2- 172.21.15.103:0/818556518 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fee140a3430 0x7fee140a58f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fee10005940 tx=0x7fee10032c30 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:03.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee27fff640 1 -- 172.21.15.103:0/3405992230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 msgr2=0x560d1e75fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:03.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee27fff640 1 --2- 172.21.15.103:0/3405992230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e75fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee27fff640 1 -- 172.21.15.103:0/3405992230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e752dc0 msgr2=0x560d1e75d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:03.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee27fff640 1 --2- 172.21.15.103:0/3405992230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e752dc0 0x560d1e75d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee27fff640 1 -- 172.21.15.103:0/3405992230 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d1e644440 con 0x560d1e7524a0 2024-09-19T06:57:03.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.927+0000 7fee27fff640 1 --2- 172.21.15.103:0/3405992230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e7524a0 0x560d1e752880 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fee18098050 tx=0x7fee18000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=598f1e311fd37146 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:03.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.928+0000 7fee1e7fc640 1 -- 172.21.15.103:0/3405992230 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18020040 con 0x560d1e7524a0 2024-09-19T06:57:03.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.928+0000 7fee1e7fc640 1 -- 172.21.15.103:0/3405992230 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee1809fef0 con 0x560d1e7524a0 2024-09-19T06:57:03.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.928+0000 7fee1e7fc640 1 -- 172.21.15.103:0/3405992230 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1809b240 con 0x560d1e7524a0 2024-09-19T06:57:03.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.928+0000 7fee306ff740 1 -- 172.21.15.103:0/3405992230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e7524a0 msgr2=0x560d1e752880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:03.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.928+0000 7fee306ff740 1 --2- 172.21.15.103:0/3405992230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e7524a0 0x560d1e752880 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fee18098050 tx=0x7fee18000bc0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.928+0000 7fee306ff740 1 -- 172.21.15.103:0/3405992230 shutdown_connections 2024-09-19T06:57:03.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.928+0000 7fee306ff740 1 --2- 172.21.15.103:0/3405992230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e75fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.928+0000 7fee306ff740 1 --2- 172.21.15.103:0/3405992230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e752dc0 0x560d1e75d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.928+0000 7fee306ff740 1 --2- 172.21.15.103:0/3405992230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e7524a0 0x560d1e752880 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.928+0000 7fee306ff740 1 -- 172.21.15.103:0/3405992230 >> 172.21.15.103:0/3405992230 conn(0x560d1e74ea30 msgr2=0x560d1e74ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:03.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.929+0000 7fee306ff740 1 -- 172.21.15.103:0/3405992230 shutdown_connections 2024-09-19T06:57:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.929+0000 7fee306ff740 1 -- 172.21.15.103:0/3405992230 wait complete. 2024-09-19T06:57:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.929+0000 7fee306ff740 1 -- start start 2024-09-19T06:57:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.929+0000 7fee306ff740 1 Processor -- start 2024-09-19T06:57:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.929+0000 7fee306ff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e7524a0 0x560d1e75cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.929+0000 7fee306ff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e752dc0 0x560d1e75a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.929+0000 7fee306ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e75a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.929+0000 7fee306ff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560d1e760240 con 0x560d1e752dc0 2024-09-19T06:57:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.929+0000 7fee306ff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560d1e673050 con 0x560d1e7524a0 2024-09-19T06:57:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.929+0000 7fee306ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560d1e7603c0 con 0x560d1e75d680 2024-09-19T06:57:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee277fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e752dc0 0x560d1e75a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee306ff740 10 client.0 _ll_register_callbacks cb 0x560d1e6b3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee277fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e752dc0 0x560d1e75a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45632/0 (socket says 172.21.15.103:45632) 2024-09-19T06:57:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee2c8c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e75a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee277fe640 1 -- 172.21.15.103:0/3933586761 learned_addr learned my addr 172.21.15.103:0/3933586761 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee27fff640 1 --2- 172.21.15.103:0/3933586761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e7524a0 0x560d1e75cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee277fe640 1 -- 172.21.15.103:0/3933586761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 msgr2=0x560d1e75a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee277fe640 1 --2- 172.21.15.103:0/3933586761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e75a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee277fe640 1 -- 172.21.15.103:0/3933586761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e7524a0 msgr2=0x560d1e75cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee277fe640 1 --2- 172.21.15.103:0/3933586761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e7524a0 0x560d1e75cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee277fe640 1 -- 172.21.15.103:0/3933586761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1e4293c0 con 0x560d1e752dc0 2024-09-19T06:57:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee27fff640 1 --2- 172.21.15.103:0/3933586761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e7524a0 0x560d1e75cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:57:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee277fe640 1 --2- 172.21.15.103:0/3933586761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e752dc0 0x560d1e75a250 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fee10009900 tx=0x7fee1002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d044076903cc63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee077fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee077fe640 1 -- 172.21.15.103:0/3933586761 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee10035040 con 0x560d1e752dc0 2024-09-19T06:57:03.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee077fe640 1 -- 172.21.15.103:0/3933586761 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fee10005240 con 0x560d1e752dc0 2024-09-19T06:57:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.930+0000 7fee306ff740 1 -- 172.21.15.103:0/3933586761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee1809fef0 con 0x560d1e752dc0 2024-09-19T06:57:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.931+0000 7fee077fe640 1 -- 172.21.15.103:0/3933586761 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fee10059050 con 0x560d1e752dc0 2024-09-19T06:57:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.931+0000 7fee306ff740 10 client.18222 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.931+0000 7fee306ff740 1 -- 172.21.15.103:0/3933586761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560d1e76bc10 con 0x560d1e752dc0 2024-09-19T06:57:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.931+0000 7fee077fe640 1 -- 172.21.15.103:0/3933586761 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee1809fef0 con 0x560d1e752dc0 2024-09-19T06:57:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.931+0000 7fee077fe640 1 -- 172.21.15.103:0/3933586761 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fee10003850 con 0x560d1e752dc0 2024-09-19T06:57:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.931+0000 7fee306ff740 10 client.18222 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.931+0000 7fee306ff740 10 client.18222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.931+0000 7fee306ff740 20 client.18222 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.931+0000 7fee306ff740 1 --2- 172.21.15.103:0/3933586761 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560d1e76d070 0x560d1e76f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.931+0000 7fee306ff740 4 client.18222 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.931+0000 7fee306ff740 1 -- 172.21.15.103:0/3933586761 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d1e6c5850 con 0x560d1e76d070 2024-09-19T06:57:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.931+0000 7fee27fff640 1 --2- 172.21.15.103:0/3933586761 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560d1e76d070 0x560d1e76f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.932+0000 7fee27fff640 1 --2- 172.21.15.103:0/3933586761 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560d1e76d070 0x560d1e76f510 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31edafec8fbb9660 server_cookie=3c631e0f2d096039 in_seq=0 out_seq=0 2024-09-19T06:57:03.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.932+0000 7fee077fe640 10 client.18222 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:03.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.932+0000 7fee077fe640 1 -- 172.21.15.103:0/3933586761 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d1e6c5850 con 0x560d1e76d070 2024-09-19T06:57:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.933+0000 7fee077fe640 10 client.18222 handle_command_reply: tid=0 2024-09-19T06:57:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.933+0000 7fee306ff740 1 client.18222 shutdown 2024-09-19T06:57:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.933+0000 7fee306ff740 1 -- 172.21.15.103:0/3933586761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e752dc0 msgr2=0x560d1e75a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.933+0000 7fee306ff740 1 --2- 172.21.15.103:0/3933586761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e752dc0 0x560d1e75a250 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fee10009900 tx=0x7fee1002a420 comp rx=0 tx=0).stop 2024-09-19T06:57:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.933+0000 7fee306ff740 1 -- 172.21.15.103:0/3933586761 shutdown_connections 2024-09-19T06:57:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.933+0000 7fee306ff740 1 --2- 172.21.15.103:0/3933586761 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560d1e76d070 0x560d1e76f510 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.933+0000 7fee306ff740 1 --2- 172.21.15.103:0/3933586761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e75a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.933+0000 7fee306ff740 1 --2- 172.21.15.103:0/3933586761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e752dc0 0x560d1e75a250 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.933+0000 7fee306ff740 1 --2- 172.21.15.103:0/3933586761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e7524a0 0x560d1e75cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.933+0000 7fee306ff740 1 -- 172.21.15.103:0/3933586761 >> 172.21.15.103:0/3933586761 conn(0x560d1e74ea30 msgr2=0x560d1e74db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.933+0000 7fee077fe640 0 client.18222 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.933+0000 7fee306ff740 1 -- 172.21.15.103:0/3933586761 shutdown_connections 2024-09-19T06:57:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.933+0000 7fee306ff740 1 -- 172.21.15.103:0/3933586761 wait complete. 2024-09-19T06:57:03.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.933+0000 7fee306ff740 20 client.18222 trim_cache size 0 max 16384 2024-09-19T06:57:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.954+0000 7fee306ff740 1 Processor -- start 2024-09-19T06:57:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.954+0000 7fee306ff740 1 -- start start 2024-09-19T06:57:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee306ff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e7524a0 0x560d1e7694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee306ff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e752dc0 0x560d1e774de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee306ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e775320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee306ff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee281ad970 con 0x560d1e7524a0 2024-09-19T06:57:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee306ff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fee281af4b0 con 0x560d1e752dc0 2024-09-19T06:57:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee306ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee281ae040 con 0x560d1e75d680 2024-09-19T06:57:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee27fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e7524a0 0x560d1e7694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee27fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e7524a0 0x560d1e7694a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45638/0 (socket says 172.21.15.103:45638) 2024-09-19T06:57:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee27fff640 1 -- 172.21.15.103:0/1110066188 learned_addr learned my addr 172.21.15.103:0/1110066188 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee277fe640 1 --2- 172.21.15.103:0/1110066188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e752dc0 0x560d1e774de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee2c8c3640 1 --2- 172.21.15.103:0/1110066188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e775320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee27fff640 1 -- 172.21.15.103:0/1110066188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 msgr2=0x560d1e775320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee27fff640 1 --2- 172.21.15.103:0/1110066188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e775320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee27fff640 1 -- 172.21.15.103:0/1110066188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e752dc0 msgr2=0x560d1e774de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee27fff640 1 --2- 172.21.15.103:0/1110066188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e752dc0 0x560d1e774de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee27fff640 1 -- 172.21.15.103:0/1110066188 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d1e752bf0 con 0x560d1e7524a0 2024-09-19T06:57:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee277fe640 1 --2- 172.21.15.103:0/1110066188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e752dc0 0x560d1e774de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee27fff640 1 --2- 172.21.15.103:0/1110066188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e7524a0 0x560d1e7694a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fee180cd060 tx=0x7fee1809bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e015b7ebe388b45 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.955+0000 7fee1dffb640 1 -- 172.21.15.103:0/1110066188 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180b9070 con 0x560d1e7524a0 2024-09-19T06:57:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.956+0000 7fee1dffb640 1 -- 172.21.15.103:0/1110066188 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee18098260 con 0x560d1e7524a0 2024-09-19T06:57:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.956+0000 7fee1dffb640 1 -- 172.21.15.103:0/1110066188 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18020050 con 0x560d1e7524a0 2024-09-19T06:57:03.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.956+0000 7fee306ff740 1 -- 172.21.15.103:0/1110066188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e7524a0 msgr2=0x560d1e7694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:03.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.956+0000 7fee306ff740 1 --2- 172.21.15.103:0/1110066188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e7524a0 0x560d1e7694a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fee180cd060 tx=0x7fee1809bd30 comp rx=0 tx=0).stop 2024-09-19T06:57:03.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.956+0000 7fee306ff740 1 -- 172.21.15.103:0/1110066188 shutdown_connections 2024-09-19T06:57:03.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.956+0000 7fee306ff740 1 --2- 172.21.15.103:0/1110066188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e775320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.956+0000 7fee306ff740 1 --2- 172.21.15.103:0/1110066188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e752dc0 0x560d1e774de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.956+0000 7fee306ff740 1 --2- 172.21.15.103:0/1110066188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e7524a0 0x560d1e7694a0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.956+0000 7fee306ff740 1 -- 172.21.15.103:0/1110066188 >> 172.21.15.103:0/1110066188 conn(0x560d1e74ea30 msgr2=0x560d1e7557e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:03.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.956+0000 7fee306ff740 1 -- 172.21.15.103:0/1110066188 shutdown_connections 2024-09-19T06:57:03.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.956+0000 7fee306ff740 1 -- 172.21.15.103:0/1110066188 wait complete. 2024-09-19T06:57:03.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.957+0000 7fee306ff740 1 -- start start 2024-09-19T06:57:03.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.957+0000 7fee306ff740 1 Processor -- start 2024-09-19T06:57:03.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.957+0000 7fee306ff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e7524a0 0x560d1e765680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.957+0000 7fee306ff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e752dc0 0x560d1e765bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.957+0000 7fee306ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e7d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.957+0000 7fee306ff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee281aede0 con 0x560d1e7524a0 2024-09-19T06:57:03.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.957+0000 7fee306ff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fee281ae710 con 0x560d1e752dc0 2024-09-19T06:57:03.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.957+0000 7fee306ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee281ab970 con 0x560d1e75d680 2024-09-19T06:57:03.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.957+0000 7fee306ff740 10 client.0 _ll_register_callbacks cb 0x7fee10032f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:03.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.957+0000 7fee27fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e7524a0 0x560d1e765680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.957+0000 7fee27fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e7524a0 0x560d1e765680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45654/0 (socket says 172.21.15.103:45654) 2024-09-19T06:57:03.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.957+0000 7fee27fff640 1 -- 172.21.15.103:0/2904074505 learned_addr learned my addr 172.21.15.103:0/2904074505 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:03.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee277fe640 1 --2- 172.21.15.103:0/2904074505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e752dc0 0x560d1e765bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee2c8c3640 1 --2- 172.21.15.103:0/2904074505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e7d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee27fff640 1 -- 172.21.15.103:0/2904074505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 msgr2=0x560d1e7d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:03.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee27fff640 1 --2- 172.21.15.103:0/2904074505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e7d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee27fff640 1 -- 172.21.15.103:0/2904074505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e752dc0 msgr2=0x560d1e765bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:03.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee27fff640 1 --2- 172.21.15.103:0/2904074505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e752dc0 0x560d1e765bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee27fff640 1 -- 172.21.15.103:0/2904074505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1e7d0b30 con 0x560d1e7524a0 2024-09-19T06:57:03.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee277fe640 1 --2- 172.21.15.103:0/2904074505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e752dc0 0x560d1e765bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:03.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee2c8c3640 1 --2- 172.21.15.103:0/2904074505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e7d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:03.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee27fff640 1 --2- 172.21.15.103:0/2904074505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e7524a0 0x560d1e765680 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fee18098050 tx=0x7fee18005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d520185687beb79b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:03.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee077fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:03.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee077fe640 1 -- 172.21.15.103:0/2904074505 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1807f060 con 0x560d1e7524a0 2024-09-19T06:57:03.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee077fe640 1 -- 172.21.15.103:0/2904074505 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fee18099ef0 con 0x560d1e7524a0 2024-09-19T06:57:03.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee306ff740 1 -- 172.21.15.103:0/2904074505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee18098260 con 0x560d1e7524a0 2024-09-19T06:57:03.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee077fe640 1 -- 172.21.15.103:0/2904074505 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fee18020030 con 0x560d1e7524a0 2024-09-19T06:57:03.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee306ff740 10 client.18228 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:03.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.958+0000 7fee306ff740 1 -- 172.21.15.103:0/2904074505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560d1e7630e0 con 0x560d1e7524a0 2024-09-19T06:57:03.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.959+0000 7fee077fe640 1 -- 172.21.15.103:0/2904074505 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee18098260 con 0x560d1e7524a0 2024-09-19T06:57:03.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.959+0000 7fee077fe640 1 -- 172.21.15.103:0/2904074505 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fee1809b040 con 0x560d1e7524a0 2024-09-19T06:57:03.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.959+0000 7fee306ff740 10 client.18228 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:03.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.959+0000 7fee306ff740 10 client.18228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:03.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.959+0000 7fee306ff740 20 client.18228 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:03.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.959+0000 7fee306ff740 1 --2- 172.21.15.103:0/2904074505 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560d1e7dc1c0 0x560d1e7de5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:03.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.959+0000 7fee306ff740 4 client.18228 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:57:03.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.959+0000 7fee306ff740 1 -- 172.21.15.103:0/2904074505 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d1e6bfd60 con 0x560d1e7dc1c0 2024-09-19T06:57:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.959+0000 7fee277fe640 1 --2- 172.21.15.103:0/2904074505 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560d1e7dc1c0 0x560d1e7de5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee077fe640 10 client.18228 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee277fe640 1 --2- 172.21.15.103:0/2904074505 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560d1e7dc1c0 0x560d1e7de5b0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=286152ce798247e9 server_cookie=c827ac5828ac373c in_seq=0 out_seq=0 2024-09-19T06:57:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee077fe640 1 -- 172.21.15.103:0/2904074505 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560d1e6bfd60 con 0x560d1e7dc1c0 2024-09-19T06:57:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee077fe640 10 client.18228 handle_command_reply: tid=0 2024-09-19T06:57:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee306ff740 1 client.18228 shutdown 2024-09-19T06:57:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee306ff740 1 -- 172.21.15.103:0/2904074505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e7524a0 msgr2=0x560d1e765680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee306ff740 1 --2- 172.21.15.103:0/2904074505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e7524a0 0x560d1e765680 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fee18098050 tx=0x7fee18005f70 comp rx=0 tx=0).stop 2024-09-19T06:57:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee306ff740 1 -- 172.21.15.103:0/2904074505 shutdown_connections 2024-09-19T06:57:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee306ff740 1 --2- 172.21.15.103:0/2904074505 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560d1e7dc1c0 0x560d1e7de5b0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee077fe640 0 client.18228 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee306ff740 1 --2- 172.21.15.103:0/2904074505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d1e75d680 0x560d1e7d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee306ff740 1 --2- 172.21.15.103:0/2904074505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d1e752dc0 0x560d1e765bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee306ff740 1 --2- 172.21.15.103:0/2904074505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d1e7524a0 0x560d1e765680 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee306ff740 1 -- 172.21.15.103:0/2904074505 >> 172.21.15.103:0/2904074505 conn(0x560d1e74ea30 msgr2=0x560d1e74d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:03.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee306ff740 1 -- 172.21.15.103:0/2904074505 shutdown_connections 2024-09-19T06:57:03.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee306ff740 1 -- 172.21.15.103:0/2904074505 wait complete. 2024-09-19T06:57:03.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.960+0000 7fee306ff740 20 client.18228 trim_cache size 0 max 16384 2024-09-19T06:57:03.967 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:57:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:57:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T06:57:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "9871dc0a-2a04-4e9f-8b27-e2581cffb01f": { 2024-09-19T06:57:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T06:57:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "9871dc0a-2a04-4e9f-8b27-e2581cffb01f", 2024-09-19T06:57:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T06:57:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:03.968 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:57:03.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.961+0000 7fee2e326640 1 -- 172.21.15.103:0/818556518 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fee140a3430 msgr2=0x7fee140a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:03.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.961+0000 7fee2e326640 1 --2- 172.21.15.103:0/818556518 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fee140a3430 0x7fee140a58f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fee10005940 tx=0x7fee10032c30 comp rx=0 tx=0).stop 2024-09-19T06:57:03.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.961+0000 7fee2e326640 1 -- 172.21.15.103:0/818556518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee28102d30 msgr2=0x7fee281985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:03.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.961+0000 7fee2e326640 1 --2- 172.21.15.103:0/818556518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee28102d30 0x7fee281985c0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fee1800d950 tx=0x7fee1800de10 comp rx=0 tx=0).stop 2024-09-19T06:57:03.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.961+0000 7fee2e326640 1 -- 172.21.15.103:0/818556518 shutdown_connections 2024-09-19T06:57:03.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.961+0000 7fee2e326640 1 --2- 172.21.15.103:0/818556518 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fee140a3430 0x7fee140a58f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.961+0000 7fee2e326640 1 --2- 172.21.15.103:0/818556518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee281082a0 0x7fee2819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.961+0000 7fee2e326640 1 --2- 172.21.15.103:0/818556518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee281036e0 0x7fee28198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.961+0000 7fee2e326640 1 --2- 172.21.15.103:0/818556518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee28102d30 0x7fee281985c0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:03.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.961+0000 7fee2e326640 1 -- 172.21.15.103:0/818556518 >> 172.21.15.103:0/818556518 conn(0x7fee280fc7d0 msgr2=0x7fee28105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:03.969 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:03 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2876379456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:57:03.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.962+0000 7fee2e326640 1 -- 172.21.15.103:0/818556518 shutdown_connections 2024-09-19T06:57:03.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:03.962+0000 7fee2e326640 1 -- 172.21.15.103:0/818556518 wait complete. 2024-09-19T06:57:04.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:03 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2876379456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:57:04.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:03 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2876379456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:57:04.637 INFO:tasks.cephfs.filesystem:scrub status for tag:9871dc0a-2a04-4e9f-8b27-e2581cffb01f - {'path': '/', 'tag': '9871dc0a-2a04-4e9f-8b27-e2581cffb01f', 'options': 'recursive,force'} 2024-09-19T06:57:04.642 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:05.130 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:04 smithi103 ceph-mon[31158]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:57:05.131 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:04 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T06:57:05.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.129+0000 7fd29d345640 1 -- 172.21.15.103:0/2715852317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd29810c190 msgr2=0x7fd29810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.129+0000 7fd29d345640 1 --2- 172.21.15.103:0/2715852317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd29810c190 0x7fd29810e610 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fd288009900 tx=0x7fd28802a480 comp rx=0 tx=0).stop 2024-09-19T06:57:05.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.129+0000 7fd29d345640 1 -- 172.21.15.103:0/2715852317 shutdown_connections 2024-09-19T06:57:05.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.129+0000 7fd29d345640 1 --2- 172.21.15.103:0/2715852317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd29810c190 0x7fd29810e610 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.129+0000 7fd29d345640 1 --2- 172.21.15.103:0/2715852317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2981036d0 0x7fd29810bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.129+0000 7fd29d345640 1 --2- 172.21.15.103:0/2715852317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd298102db0 0x7fd298103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.129+0000 7fd29d345640 1 -- 172.21.15.103:0/2715852317 >> 172.21.15.103:0/2715852317 conn(0x7fd2980fc7d0 msgr2=0x7fd2980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:05.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.129+0000 7fd29d345640 1 -- 172.21.15.103:0/2715852317 shutdown_connections 2024-09-19T06:57:05.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.129+0000 7fd29d345640 1 -- 172.21.15.103:0/2715852317 wait complete. 2024-09-19T06:57:05.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.130+0000 7fd29d345640 1 Processor -- start 2024-09-19T06:57:05.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.130+0000 7fd29d345640 1 -- start start 2024-09-19T06:57:05.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.131+0000 7fd29d345640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd298102db0 0x7fd2981983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.131+0000 7fd29d345640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2981036d0 0x7fd2981988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.131+0000 7fd29d345640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd29810c190 0x7fd29819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.131+0000 7fd29d345640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd298111f70 con 0x7fd2981036d0 2024-09-19T06:57:05.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.131+0000 7fd29d345640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd298111df0 con 0x7fd29810c190 2024-09-19T06:57:05.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.131+0000 7fd29d345640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981120f0 con 0x7fd298102db0 2024-09-19T06:57:05.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.131+0000 7fd2967fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2981036d0 0x7fd2981988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.131+0000 7fd2967fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2981036d0 0x7fd2981988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45672/0 (socket says 172.21.15.103:45672) 2024-09-19T06:57:05.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.131+0000 7fd2967fc640 1 -- 172.21.15.103:0/3670594969 learned_addr learned my addr 172.21.15.103:0/3670594969 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:05.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.131+0000 7fd2977fe640 1 --2- 172.21.15.103:0/3670594969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd29810c190 0x7fd29819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.131+0000 7fd296ffd640 1 --2- 172.21.15.103:0/3670594969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd298102db0 0x7fd2981983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.131+0000 7fd2967fc640 1 -- 172.21.15.103:0/3670594969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd298102db0 msgr2=0x7fd2981983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.131+0000 7fd2967fc640 1 --2- 172.21.15.103:0/3670594969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd298102db0 0x7fd2981983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.131+0000 7fd2967fc640 1 -- 172.21.15.103:0/3670594969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd29810c190 msgr2=0x7fd29819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.131+0000 7fd2967fc640 1 --2- 172.21.15.103:0/3670594969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd29810c190 0x7fd29819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.131+0000 7fd2967fc640 1 -- 172.21.15.103:0/3670594969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29819d360 con 0x7fd2981036d0 2024-09-19T06:57:05.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.132+0000 7fd2967fc640 1 --2- 172.21.15.103:0/3670594969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2981036d0 0x7fd2981988f0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fd28000d950 tx=0x7fd28000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:05.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.132+0000 7fd28ffff640 1 -- 172.21.15.103:0/3670594969 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280017070 con 0x7fd2981036d0 2024-09-19T06:57:05.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.132+0000 7fd29d345640 1 -- 172.21.15.103:0/3670594969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29819d650 con 0x7fd2981036d0 2024-09-19T06:57:05.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.132+0000 7fd28ffff640 1 -- 172.21.15.103:0/3670594969 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd280009c30 con 0x7fd2981036d0 2024-09-19T06:57:05.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.132+0000 7fd28ffff640 1 -- 172.21.15.103:0/3670594969 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28000f6b0 con 0x7fd2981036d0 2024-09-19T06:57:05.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.132+0000 7fd29d345640 1 -- 172.21.15.103:0/3670594969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2981a4f30 con 0x7fd2981036d0 2024-09-19T06:57:05.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.134+0000 7fd28ffff640 1 -- 172.21.15.103:0/3670594969 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd2800109d0 con 0x7fd2981036d0 2024-09-19T06:57:05.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.135+0000 7fd28ffff640 1 --2- 172.21.15.103:0/3670594969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2840a3430 0x7fd2840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.135+0000 7fd29f71e740 1 Processor -- start 2024-09-19T06:57:05.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.135+0000 7fd296ffd640 1 --2- 172.21.15.103:0/3670594969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2840a3430 0x7fd2840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.135+0000 7fd28ffff640 1 -- 172.21.15.103:0/3670594969 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8600+0+0 (secure 0 0 0) 0x7fd2800cfd40 con 0x7fd2981036d0 2024-09-19T06:57:05.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.135+0000 7fd29f71e740 1 -- start start 2024-09-19T06:57:05.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd296ffd640 1 --2- 172.21.15.103:0/3670594969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2840a3430 0x7fd2840a58f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd290005c40 tx=0x7fd29000a780 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:05.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd29f71e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630ac850 0x562d630acc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd29f71e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 0x562d630b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd29f71e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630b7a30 0x562d630b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd29f71e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562d63011160 con 0x562d630ac850 2024-09-19T06:57:05.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd29f71e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562d62fb8720 con 0x562d630ad170 2024-09-19T06:57:05.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd29f71e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562d6300d120 con 0x562d630b7a30 2024-09-19T06:57:05.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd296ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630ac850 0x562d630acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd296ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630ac850 0x562d630acc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45684/0 (socket says 172.21.15.103:45684) 2024-09-19T06:57:05.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd296ffd640 1 -- 172.21.15.103:0/3595134763 learned_addr learned my addr 172.21.15.103:0/3595134763 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:05.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd2967fc640 1 --2- 172.21.15.103:0/3595134763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 0x562d630b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd2977fe640 1 --2- 172.21.15.103:0/3595134763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630b7a30 0x562d630b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd296ffd640 1 -- 172.21.15.103:0/3595134763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630b7a30 msgr2=0x562d630b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:05.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd296ffd640 1 --2- 172.21.15.103:0/3595134763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630b7a30 0x562d630b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd296ffd640 1 -- 172.21.15.103:0/3595134763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 msgr2=0x562d630b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:05.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd296ffd640 1 --2- 172.21.15.103:0/3595134763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 0x562d630b74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd296ffd640 1 -- 172.21.15.103:0/3595134763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d62d82ba0 con 0x562d630ac850 2024-09-19T06:57:05.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd2967fc640 1 --2- 172.21.15.103:0/3595134763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 0x562d630b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:05.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd296ffd640 1 --2- 172.21.15.103:0/3595134763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630ac850 0x562d630acc30 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fd290011570 tx=0x7fd290012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195464f7821031ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:05.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd28d7fa640 1 -- 172.21.15.103:0/3595134763 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290022070 con 0x562d630ac850 2024-09-19T06:57:05.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd28d7fa640 1 -- 172.21.15.103:0/3595134763 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd290010d80 con 0x562d630ac850 2024-09-19T06:57:05.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.136+0000 7fd28d7fa640 1 -- 172.21.15.103:0/3595134763 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29001d3b0 con 0x562d630ac850 2024-09-19T06:57:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.137+0000 7fd29f71e740 1 -- 172.21.15.103:0/3595134763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630ac850 msgr2=0x562d630acc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.137+0000 7fd29f71e740 1 --2- 172.21.15.103:0/3595134763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630ac850 0x562d630acc30 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fd290011570 tx=0x7fd290012000 comp rx=0 tx=0).stop 2024-09-19T06:57:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.137+0000 7fd29f71e740 1 -- 172.21.15.103:0/3595134763 shutdown_connections 2024-09-19T06:57:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.137+0000 7fd29f71e740 1 --2- 172.21.15.103:0/3595134763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630b7a30 0x562d630b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.137+0000 7fd29f71e740 1 --2- 172.21.15.103:0/3595134763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 0x562d630b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.137+0000 7fd29f71e740 1 --2- 172.21.15.103:0/3595134763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630ac850 0x562d630acc30 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.137+0000 7fd29f71e740 1 -- 172.21.15.103:0/3595134763 >> 172.21.15.103:0/3595134763 conn(0x562d630a8de0 msgr2=0x562d630a91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:05.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.137+0000 7fd29f71e740 1 -- 172.21.15.103:0/3595134763 shutdown_connections 2024-09-19T06:57:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.137+0000 7fd29f71e740 1 -- 172.21.15.103:0/3595134763 wait complete. 2024-09-19T06:57:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd29f71e740 1 -- start start 2024-09-19T06:57:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd29f71e740 1 Processor -- start 2024-09-19T06:57:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd29f71e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 0x562d630b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd29f71e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630ad170 0x562d630b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd29f71e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630b7a30 0x562d630b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd29f71e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562d630ba600 con 0x562d630ad170 2024-09-19T06:57:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd29f71e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562d62f99900 con 0x562d630b7a30 2024-09-19T06:57:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd29f71e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562d630ba780 con 0x562d630ac850 2024-09-19T06:57:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd2967fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630ad170 0x562d630b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd2967fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630ad170 0x562d630b4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45690/0 (socket says 172.21.15.103:45690) 2024-09-19T06:57:05.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd2967fc640 1 -- 172.21.15.103:0/4244531657 learned_addr learned my addr 172.21.15.103:0/4244531657 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd29f71e740 10 client.0 _ll_register_callbacks cb 0x562d62f97780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd296ffd640 1 --2- 172.21.15.103:0/4244531657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 0x562d630b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd2977fe640 1 --2- 172.21.15.103:0/4244531657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630b7a30 0x562d630b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd2967fc640 1 -- 172.21.15.103:0/4244531657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 msgr2=0x562d630b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd2967fc640 1 --2- 172.21.15.103:0/4244531657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 0x562d630b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd2967fc640 1 -- 172.21.15.103:0/4244531657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630b7a30 msgr2=0x562d630b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd2967fc640 1 --2- 172.21.15.103:0/4244531657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630b7a30 0x562d630b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd2967fc640 1 -- 172.21.15.103:0/4244531657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d62fd9c50 con 0x562d630ad170 2024-09-19T06:57:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.138+0000 7fd2977fe640 1 --2- 172.21.15.103:0/4244531657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630b7a30 0x562d630b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.139+0000 7fd2967fc640 1 --2- 172.21.15.103:0/4244531657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630ad170 0x562d630b4e70 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fd280098940 tx=0x7fd28009b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6171fb0f801e0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.139+0000 7fd27e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.139+0000 7fd27e7fc640 1 -- 172.21.15.103:0/4244531657 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2800b6060 con 0x562d630ad170 2024-09-19T06:57:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.139+0000 7fd29f71e740 1 -- 172.21.15.103:0/4244531657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd290010d80 con 0x562d630ad170 2024-09-19T06:57:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.139+0000 7fd27e7fc640 1 -- 172.21.15.103:0/4244531657 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8600+0+0 (secure 0 0 0) 0x7fd2800a4750 con 0x562d630ad170 2024-09-19T06:57:05.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.139+0000 7fd27e7fc640 1 -- 172.21.15.103:0/4244531657 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd280020040 con 0x562d630ad170 2024-09-19T06:57:05.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.139+0000 7fd29f71e740 10 client.18240 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:05.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.139+0000 7fd29f71e740 1 -- 172.21.15.103:0/4244531657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d630c6280 con 0x562d630ad170 2024-09-19T06:57:05.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.139+0000 7fd27e7fc640 1 -- 172.21.15.103:0/4244531657 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd290010d80 con 0x562d630ad170 2024-09-19T06:57:05.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.139+0000 7fd27e7fc640 1 -- 172.21.15.103:0/4244531657 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd2800babb0 con 0x562d630ad170 2024-09-19T06:57:05.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.139+0000 7fd29f71e740 10 client.18240 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:05.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.139+0000 7fd29f71e740 10 client.18240 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:57:05.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.139+0000 7fd29f71e740 20 client.18240 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:05.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.139+0000 7fd29f71e740 1 --2- 172.21.15.103:0/4244531657 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562d630c7700 0x562d630c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.139+0000 7fd29f71e740 4 client.18240 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:05.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.139+0000 7fd29f71e740 1 -- 172.21.15.103:0/4244531657 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d6301e890 con 0x562d630c7700 2024-09-19T06:57:05.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.140+0000 7fd296ffd640 1 --2- 172.21.15.103:0/4244531657 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562d630c7700 0x562d630c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.140+0000 7fd296ffd640 1 --2- 172.21.15.103:0/4244531657 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562d630c7700 0x562d630c9b60 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=accbdac9c3e5192e server_cookie=ff278f157fab3493 in_seq=0 out_seq=0 2024-09-19T06:57:05.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.140+0000 7fd27e7fc640 10 client.18240 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:05.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.141+0000 7fd27e7fc640 1 -- 172.21.15.103:0/4244531657 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d6301e890 con 0x562d630c7700 2024-09-19T06:57:05.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.141+0000 7fd27e7fc640 10 client.18240 handle_command_reply: tid=0 2024-09-19T06:57:05.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.141+0000 7fd29f71e740 1 client.18240 shutdown 2024-09-19T06:57:05.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.141+0000 7fd29f71e740 1 -- 172.21.15.103:0/4244531657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630ad170 msgr2=0x562d630b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.141+0000 7fd29f71e740 1 --2- 172.21.15.103:0/4244531657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630ad170 0x562d630b4e70 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fd280098940 tx=0x7fd28009b990 comp rx=0 tx=0).stop 2024-09-19T06:57:05.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.141+0000 7fd29f71e740 1 -- 172.21.15.103:0/4244531657 shutdown_connections 2024-09-19T06:57:05.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.141+0000 7fd29f71e740 1 --2- 172.21.15.103:0/4244531657 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562d630c7700 0x562d630c9b60 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.141+0000 7fd29f71e740 1 --2- 172.21.15.103:0/4244531657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630b7a30 0x562d630b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.141+0000 7fd29f71e740 1 --2- 172.21.15.103:0/4244531657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630ad170 0x562d630b4e70 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.141+0000 7fd29f71e740 1 --2- 172.21.15.103:0/4244531657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 0x562d630b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.141+0000 7fd27e7fc640 0 client.18240 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:05.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.141+0000 7fd29f71e740 1 -- 172.21.15.103:0/4244531657 >> 172.21.15.103:0/4244531657 conn(0x562d630a8de0 msgr2=0x562d630aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:05.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.141+0000 7fd29f71e740 1 -- 172.21.15.103:0/4244531657 shutdown_connections 2024-09-19T06:57:05.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.141+0000 7fd29f71e740 1 -- 172.21.15.103:0/4244531657 wait complete. 2024-09-19T06:57:05.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.141+0000 7fd29f71e740 20 client.18240 trim_cache size 0 max 16384 2024-09-19T06:57:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.144+0000 7ff4240d9640 1 -- 172.21.15.103:0/179770104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff41c107ad0 msgr2=0x7ff41c107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.144+0000 7ff4240d9640 1 --2- 172.21.15.103:0/179770104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff41c107ad0 0x7ff41c107eb0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ff418009930 tx=0x7ff41802a420 comp rx=0 tx=0).stop 2024-09-19T06:57:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.144+0000 7ff4240d9640 1 -- 172.21.15.103:0/179770104 shutdown_connections 2024-09-19T06:57:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.144+0000 7ff4240d9640 1 --2- 172.21.15.103:0/179770104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff41c102460 0x7ff41c10e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.144+0000 7ff4240d9640 1 --2- 172.21.15.103:0/179770104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff41c101ac0 0x7ff41c101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.144+0000 7ff4240d9640 1 --2- 172.21.15.103:0/179770104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff41c107ad0 0x7ff41c107eb0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.144+0000 7ff4240d9640 1 -- 172.21.15.103:0/179770104 >> 172.21.15.103:0/179770104 conn(0x7ff41c0fd820 msgr2=0x7ff41c0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.144+0000 7ff4240d9640 1 -- 172.21.15.103:0/179770104 shutdown_connections 2024-09-19T06:57:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.144+0000 7ff4240d9640 1 -- 172.21.15.103:0/179770104 wait complete. 2024-09-19T06:57:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff4240d9640 1 Processor -- start 2024-09-19T06:57:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff4240d9640 1 -- start start 2024-09-19T06:57:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff4240d9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff41c101ac0 0x7ff41c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff4240d9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff41c102460 0x7ff41c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff4240d9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff41c107ad0 0x7ff41c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff4240d9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c111ed0 con 0x7ff41c102460 2024-09-19T06:57:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff4240d9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c111d50 con 0x7ff41c107ad0 2024-09-19T06:57:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff4240d9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c112050 con 0x7ff41c101ac0 2024-09-19T06:57:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff421674640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff41c102460 0x7ff41c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff421674640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff41c102460 0x7ff41c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45696/0 (socket says 172.21.15.103:45696) 2024-09-19T06:57:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff421674640 1 -- 172.21.15.103:0/1140497314 learned_addr learned my addr 172.21.15.103:0/1140497314 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff421e75640 1 --2- 172.21.15.103:0/1140497314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff41c101ac0 0x7ff41c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff421674640 1 -- 172.21.15.103:0/1140497314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff41c101ac0 msgr2=0x7ff41c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff422676640 1 --2- 172.21.15.103:0/1140497314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff41c107ad0 0x7ff41c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff421674640 1 --2- 172.21.15.103:0/1140497314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff41c101ac0 0x7ff41c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff421674640 1 -- 172.21.15.103:0/1140497314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff41c107ad0 msgr2=0x7ff41c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff421674640 1 --2- 172.21.15.103:0/1140497314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff41c107ad0 0x7ff41c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff421674640 1 -- 172.21.15.103:0/1140497314 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c1a6880 con 0x7ff41c102460 2024-09-19T06:57:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.145+0000 7ff421674640 1 --2- 172.21.15.103:0/1140497314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff41c102460 0x7ff41c074a30 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00d950 tx=0x7ff40c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.146+0000 7ff412ffd640 1 -- 172.21.15.103:0/1140497314 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c017030 con 0x7ff41c102460 2024-09-19T06:57:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.146+0000 7ff4240d9640 1 -- 172.21.15.103:0/1140497314 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41c1a6b70 con 0x7ff41c102460 2024-09-19T06:57:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.146+0000 7ff412ffd640 1 -- 172.21.15.103:0/1140497314 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff40c004020 con 0x7ff41c102460 2024-09-19T06:57:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.146+0000 7ff4240d9640 1 -- 172.21.15.103:0/1140497314 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41c1a7110 con 0x7ff41c102460 2024-09-19T06:57:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.146+0000 7ff412ffd640 1 -- 172.21.15.103:0/1140497314 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c0047a0 con 0x7ff41c102460 2024-09-19T06:57:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.148+0000 7ff426e6e740 1 Processor -- start 2024-09-19T06:57:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.148+0000 7ff426e6e740 1 -- start start 2024-09-19T06:57:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.148+0000 7ff412ffd640 1 -- 172.21.15.103:0/1140497314 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff40c0041d0 con 0x7ff41c102460 2024-09-19T06:57:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff426e6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da75704300 0x55da757046e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff426e6e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704c20 0x55da7570ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff426e6e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da7570f4c0 0x55da757118b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff426e6e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55da756602f0 con 0x55da75704c20 2024-09-19T06:57:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff426e6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55da75608270 con 0x55da75704300 2024-09-19T06:57:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff426e6e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55da7565c2b0 con 0x55da7570f4c0 2024-09-19T06:57:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff421e75640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704c20 0x55da7570ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff421e75640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704c20 0x55da7570ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45700/0 (socket says 172.21.15.103:45700) 2024-09-19T06:57:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff421e75640 1 -- 172.21.15.103:0/1891529399 learned_addr learned my addr 172.21.15.103:0/1891529399 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff422676640 1 --2- 172.21.15.103:0/1891529399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da75704300 0x55da757046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff421674640 1 --2- 172.21.15.103:0/1891529399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da7570f4c0 0x55da757118b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff421e75640 1 -- 172.21.15.103:0/1891529399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da7570f4c0 msgr2=0x55da757118b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff421e75640 1 --2- 172.21.15.103:0/1891529399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da7570f4c0 0x55da757118b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff421e75640 1 -- 172.21.15.103:0/1891529399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da75704300 msgr2=0x55da757046e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff421e75640 1 --2- 172.21.15.103:0/1891529399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da75704300 0x55da757046e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff421e75640 1 -- 172.21.15.103:0/1891529399 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da75607340 con 0x55da75704c20 2024-09-19T06:57:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff422676640 1 --2- 172.21.15.103:0/1891529399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da75704300 0x55da757046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff421e75640 1 --2- 172.21.15.103:0/1891529399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704c20 0x55da7570ef80 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ff41802a940 tx=0x7ff418005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22e914295ce6cce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.149+0000 7ff412ffd640 1 --2- 172.21.15.103:0/1140497314 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4080a34e0 0x7ff4080a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.150+0000 7ff407fff640 1 -- 172.21.15.103:0/1891529399 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418040070 con 0x55da75704c20 2024-09-19T06:57:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.150+0000 7ff412ffd640 1 -- 172.21.15.103:0/1140497314 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8600+0+0 (secure 0 0 0) 0x7ff40c0cf6d0 con 0x7ff41c102460 2024-09-19T06:57:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.150+0000 7ff407fff640 1 -- 172.21.15.103:0/1891529399 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4180318f0 con 0x55da75704c20 2024-09-19T06:57:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.150+0000 7ff407fff640 1 -- 172.21.15.103:0/1891529399 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418035040 con 0x55da75704c20 2024-09-19T06:57:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.150+0000 7ff422676640 1 --2- 172.21.15.103:0/1140497314 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4080a34e0 0x7ff4080a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.150+0000 7ff426e6e740 1 -- 172.21.15.103:0/1891529399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704c20 msgr2=0x55da7570ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.150+0000 7ff422676640 1 --2- 172.21.15.103:0/1140497314 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4080a34e0 0x7ff4080a59a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff414005fd0 tx=0x7ff414009500 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.150+0000 7ff426e6e740 1 --2- 172.21.15.103:0/1891529399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704c20 0x55da7570ef80 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ff41802a940 tx=0x7ff418005c00 comp rx=0 tx=0).stop 2024-09-19T06:57:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.150+0000 7ff426e6e740 1 -- 172.21.15.103:0/1891529399 shutdown_connections 2024-09-19T06:57:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.150+0000 7ff426e6e740 1 --2- 172.21.15.103:0/1891529399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da7570f4c0 0x55da757118b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.150+0000 7ff426e6e740 1 --2- 172.21.15.103:0/1891529399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704c20 0x55da7570ef80 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.150+0000 7ff426e6e740 1 --2- 172.21.15.103:0/1891529399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da75704300 0x55da757046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.150+0000 7ff426e6e740 1 -- 172.21.15.103:0/1891529399 >> 172.21.15.103:0/1891529399 conn(0x55da75700870 msgr2=0x55da75700c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.151+0000 7ff426e6e740 1 -- 172.21.15.103:0/1891529399 shutdown_connections 2024-09-19T06:57:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.151+0000 7ff426e6e740 1 -- 172.21.15.103:0/1891529399 wait complete. 2024-09-19T06:57:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.151+0000 7ff426e6e740 1 -- start start 2024-09-19T06:57:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.151+0000 7ff426e6e740 1 Processor -- start 2024-09-19T06:57:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.151+0000 7ff426e6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da75704300 0x55da7570c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.151+0000 7ff426e6e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da75704c20 0x55da7570c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.151+0000 7ff426e6e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da7570f4c0 0x55da7570cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.151+0000 7ff426e6e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55da75712090 con 0x55da7570f4c0 2024-09-19T06:57:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.151+0000 7ff426e6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55da755e9290 con 0x55da75704300 2024-09-19T06:57:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.151+0000 7ff426e6e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55da75712210 con 0x55da75704c20 2024-09-19T06:57:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.151+0000 7ff426e6e740 10 client.0 _ll_register_callbacks cb 0x55da7561a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.151+0000 7ff422676640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da7570f4c0 0x55da7570cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff422676640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da7570f4c0 0x55da7570cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45714/0 (socket says 172.21.15.103:45714) 2024-09-19T06:57:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff422676640 1 -- 172.21.15.103:0/3090530970 learned_addr learned my addr 172.21.15.103:0/3090530970 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff421e75640 1 --2- 172.21.15.103:0/3090530970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da75704300 0x55da7570c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff421674640 1 --2- 172.21.15.103:0/3090530970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da75704c20 0x55da7570c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff422676640 1 -- 172.21.15.103:0/3090530970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da75704c20 msgr2=0x55da7570c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff422676640 1 --2- 172.21.15.103:0/3090530970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da75704c20 0x55da7570c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff422676640 1 -- 172.21.15.103:0/3090530970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da75704300 msgr2=0x55da7570c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff422676640 1 --2- 172.21.15.103:0/3090530970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da75704300 0x55da7570c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff422676640 1 -- 172.21.15.103:0/3090530970 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da75373210 con 0x55da7570f4c0 2024-09-19T06:57:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff422676640 1 --2- 172.21.15.103:0/3090530970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da7570f4c0 0x55da7570cdf0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ff414012880 tx=0x7ff414004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dd6c573e75e934 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff4057fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff4057fa640 1 -- 172.21.15.103:0/3090530970 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41400fb60 con 0x55da7570f4c0 2024-09-19T06:57:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff4057fa640 1 -- 172.21.15.103:0/3090530970 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8600+0+0 (secure 0 0 0) 0x7ff41400fd00 con 0x55da7570f4c0 2024-09-19T06:57:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff426e6e740 1 -- 172.21.15.103:0/3090530970 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4180318f0 con 0x55da7570f4c0 2024-09-19T06:57:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff4057fa640 1 -- 172.21.15.103:0/3090530970 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff41401f420 con 0x55da7570f4c0 2024-09-19T06:57:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff426e6e740 10 client.18252 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.152+0000 7ff426e6e740 1 -- 172.21.15.103:0/3090530970 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55da7543f620 con 0x55da7570f4c0 2024-09-19T06:57:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.153+0000 7ff4057fa640 1 -- 172.21.15.103:0/3090530970 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4180318f0 con 0x55da7570f4c0 2024-09-19T06:57:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.153+0000 7ff4057fa640 1 -- 172.21.15.103:0/3090530970 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff414027750 con 0x55da7570f4c0 2024-09-19T06:57:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.153+0000 7ff426e6e740 10 client.18252 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.153+0000 7ff426e6e740 10 client.18252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.153+0000 7ff426e6e740 20 client.18252 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.153+0000 7ff426e6e740 1 --2- 172.21.15.103:0/3090530970 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55da7571f550 0x55da75721980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.153+0000 7ff426e6e740 4 client.18252 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.153+0000 7ff426e6e740 1 -- 172.21.15.103:0/3090530970 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da7566da20 con 0x55da7571f550 2024-09-19T06:57:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.153+0000 7ff421e75640 1 --2- 172.21.15.103:0/3090530970 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55da7571f550 0x55da75721980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.153+0000 7ff4057fa640 10 client.18252 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.153+0000 7ff421e75640 1 --2- 172.21.15.103:0/3090530970 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55da7571f550 0x55da75721980 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cad4126326c8be server_cookie=89c74050c98421a3 in_seq=0 out_seq=0 2024-09-19T06:57:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.154+0000 7ff4057fa640 1 -- 172.21.15.103:0/3090530970 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da7566da20 con 0x55da7571f550 2024-09-19T06:57:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.154+0000 7ff4057fa640 10 client.18252 handle_command_reply: tid=0 2024-09-19T06:57:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.154+0000 7ff426e6e740 1 client.18252 shutdown 2024-09-19T06:57:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.154+0000 7ff426e6e740 1 -- 172.21.15.103:0/3090530970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da7570f4c0 msgr2=0x55da7570cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.154+0000 7ff426e6e740 1 --2- 172.21.15.103:0/3090530970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da7570f4c0 0x55da7570cdf0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ff414012880 tx=0x7ff414004a40 comp rx=0 tx=0).stop 2024-09-19T06:57:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.154+0000 7ff426e6e740 1 -- 172.21.15.103:0/3090530970 shutdown_connections 2024-09-19T06:57:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.154+0000 7ff426e6e740 1 --2- 172.21.15.103:0/3090530970 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55da7571f550 0x55da75721980 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.154+0000 7ff426e6e740 1 --2- 172.21.15.103:0/3090530970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da7570f4c0 0x55da7570cdf0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.154+0000 7ff4057fa640 0 client.18252 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.154+0000 7ff426e6e740 1 --2- 172.21.15.103:0/3090530970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da75704c20 0x55da7570c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.154+0000 7ff426e6e740 1 --2- 172.21.15.103:0/3090530970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da75704300 0x55da7570c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.154+0000 7ff426e6e740 1 -- 172.21.15.103:0/3090530970 >> 172.21.15.103:0/3090530970 conn(0x55da75700870 msgr2=0x55da757065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.154+0000 7ff426e6e740 1 -- 172.21.15.103:0/3090530970 shutdown_connections 2024-09-19T06:57:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.154+0000 7ff426e6e740 1 -- 172.21.15.103:0/3090530970 wait complete. 2024-09-19T06:57:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.155+0000 7ff426e6e740 20 client.18252 trim_cache size 0 max 16384 2024-09-19T06:57:05.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.160+0000 7fd29f71e740 1 Processor -- start 2024-09-19T06:57:05.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.160+0000 7fd29f71e740 1 -- start start 2024-09-19T06:57:05.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd29f71e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 0x562d630af300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd29f71e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 0x562d630cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd29f71e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630b7a30 0x562d630cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd29f71e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981afba0 con 0x562d630b7a30 2024-09-19T06:57:05.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd29f71e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981b16e0 con 0x562d630ad170 2024-09-19T06:57:05.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd29f71e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981b0270 con 0x562d630ac850 2024-09-19T06:57:05.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd296ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 0x562d630af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd2977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630b7a30 0x562d630cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd296ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 0x562d630af300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:51608/0 (socket says 172.21.15.103:51608) 2024-09-19T06:57:05.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd296ffd640 1 -- 172.21.15.103:0/3468572994 learned_addr learned my addr 172.21.15.103:0/3468572994 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:05.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd2977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630b7a30 0x562d630cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45726/0 (socket says 172.21.15.103:45726) 2024-09-19T06:57:05.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd2967fc640 1 --2- 172.21.15.103:0/3468572994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 0x562d630cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd296ffd640 1 -- 172.21.15.103:0/3468572994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 msgr2=0x562d630cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:05.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd296ffd640 1 --2- 172.21.15.103:0/3468572994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 0x562d630cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd296ffd640 1 -- 172.21.15.103:0/3468572994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630b7a30 msgr2=0x562d630cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:05.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd296ffd640 1 --2- 172.21.15.103:0/3468572994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630b7a30 0x562d630cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd296ffd640 1 -- 172.21.15.103:0/3468572994 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d630acfa0 con 0x562d630ac850 2024-09-19T06:57:05.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd2977fe640 1 --2- 172.21.15.103:0/3468572994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630b7a30 0x562d630cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:57:05.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.161+0000 7fd296ffd640 1 --2- 172.21.15.103:0/3468572994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 0x562d630af300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd290011e00 tx=0x7fd29001e830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=183cc8501c1331a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:05.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.162+0000 7fd28cff9640 1 -- 172.21.15.103:0/3468572994 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290022040 con 0x562d630ac850 2024-09-19T06:57:05.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.162+0000 7fd28cff9640 1 -- 172.21.15.103:0/3468572994 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29001e650 con 0x562d630ac850 2024-09-19T06:57:05.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.162+0000 7fd28cff9640 1 -- 172.21.15.103:0/3468572994 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2900276b0 con 0x562d630ac850 2024-09-19T06:57:05.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.162+0000 7fd29f71e740 1 -- 172.21.15.103:0/3468572994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 msgr2=0x562d630af300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.162+0000 7fd29f71e740 1 --2- 172.21.15.103:0/3468572994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 0x562d630af300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd290011e00 tx=0x7fd29001e830 comp rx=0 tx=0).stop 2024-09-19T06:57:05.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.163+0000 7fd29f71e740 1 -- 172.21.15.103:0/3468572994 shutdown_connections 2024-09-19T06:57:05.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.163+0000 7fd29f71e740 1 --2- 172.21.15.103:0/3468572994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630b7a30 0x562d630cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.163+0000 7fd29f71e740 1 --2- 172.21.15.103:0/3468572994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 0x562d630cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.163+0000 7fd29f71e740 1 --2- 172.21.15.103:0/3468572994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 0x562d630af300 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.163+0000 7fd29f71e740 1 -- 172.21.15.103:0/3468572994 >> 172.21.15.103:0/3468572994 conn(0x562d630a8de0 msgr2=0x562d630b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:05.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.163+0000 7fd29f71e740 1 -- 172.21.15.103:0/3468572994 shutdown_connections 2024-09-19T06:57:05.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.163+0000 7fd29f71e740 1 -- 172.21.15.103:0/3468572994 wait complete. 2024-09-19T06:57:05.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.163+0000 7fd29f71e740 1 -- start start 2024-09-19T06:57:05.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.163+0000 7fd29f71e740 1 Processor -- start 2024-09-19T06:57:05.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.163+0000 7fd29f71e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 0x562d630bbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.163+0000 7fd29f71e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 0x562d630bc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.163+0000 7fd29f71e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630b7a30 0x562d63131360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.163+0000 7fd29f71e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981b1010 con 0x562d630b7a30 2024-09-19T06:57:05.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.163+0000 7fd29f71e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981b0940 con 0x562d630ad170 2024-09-19T06:57:05.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.163+0000 7fd29f71e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981adba0 con 0x562d630ac850 2024-09-19T06:57:05.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.163+0000 7fd29f71e740 10 client.0 _ll_register_callbacks cb 0x7fd28000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:05.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd2977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630b7a30 0x562d63131360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd2977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630b7a30 0x562d63131360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45734/0 (socket says 172.21.15.103:45734) 2024-09-19T06:57:05.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd2977fe640 1 -- 172.21.15.103:0/3052744834 learned_addr learned my addr 172.21.15.103:0/3052744834 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:05.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd2967fc640 1 --2- 172.21.15.103:0/3052744834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 0x562d630bc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd296ffd640 1 --2- 172.21.15.103:0/3052744834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 0x562d630bbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd2977fe640 1 -- 172.21.15.103:0/3052744834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 msgr2=0x562d630bbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:05.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd2977fe640 1 --2- 172.21.15.103:0/3052744834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 0x562d630bbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd2977fe640 1 -- 172.21.15.103:0/3052744834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 msgr2=0x562d630bc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:05.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd2977fe640 1 --2- 172.21.15.103:0/3052744834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 0x562d630bc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd2977fe640 1 -- 172.21.15.103:0/3052744834 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d631318a0 con 0x562d630b7a30 2024-09-19T06:57:05.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd2967fc640 1 --2- 172.21.15.103:0/3052744834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 0x562d630bc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:57:05.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd2977fe640 1 --2- 172.21.15.103:0/3052744834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630b7a30 0x562d63131360 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd288009900 tx=0x7fd288032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61a189e7e2a152 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:05.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd27e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:05.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd27e7fc640 1 -- 172.21.15.103:0/3052744834 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288035040 con 0x562d630b7a30 2024-09-19T06:57:05.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd27e7fc640 1 -- 172.21.15.103:0/3052744834 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8600+0+0 (secure 0 0 0) 0x7fd288005030 con 0x562d630b7a30 2024-09-19T06:57:05.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd29f71e740 1 -- 172.21.15.103:0/3052744834 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd29001e650 con 0x562d630b7a30 2024-09-19T06:57:05.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd27e7fc640 1 -- 172.21.15.103:0/3052744834 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd288005230 con 0x562d630b7a30 2024-09-19T06:57:05.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd29f71e740 10 client.18258 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:05.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.164+0000 7fd29f71e740 1 -- 172.21.15.103:0/3052744834 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d63132a10 con 0x562d630b7a30 2024-09-19T06:57:05.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.165+0000 7fd27e7fc640 1 -- 172.21.15.103:0/3052744834 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29001e650 con 0x562d630b7a30 2024-09-19T06:57:05.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.165+0000 7fd27e7fc640 1 -- 172.21.15.103:0/3052744834 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd288003710 con 0x562d630b7a30 2024-09-19T06:57:05.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.165+0000 7fd29f71e740 10 client.18258 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:05.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.165+0000 7fd29f71e740 10 client.18258 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:57:05.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.165+0000 7fd29f71e740 20 client.18258 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:05.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.165+0000 7fd29f71e740 1 --2- 172.21.15.103:0/3052744834 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562d63133250 0x562d63135670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.165+0000 7fd29f71e740 4 client.18258 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}] 2024-09-19T06:57:05.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.165+0000 7fd29f71e740 1 -- 172.21.15.103:0/3052744834 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}) -- 0x562d63018be0 con 0x562d63133250 2024-09-19T06:57:05.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.165+0000 7fd296ffd640 1 --2- 172.21.15.103:0/3052744834 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562d63133250 0x562d63135670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.166+0000 7fd27e7fc640 10 client.18258 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:05.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.166+0000 7fd296ffd640 1 --2- 172.21.15.103:0/3052744834 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562d63133250 0x562d63135670 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3742a3fb132d9389 server_cookie=cd451c58b37ede20 in_seq=0 out_seq=0 2024-09-19T06:57:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.170+0000 7ff426e6e740 1 Processor -- start 2024-09-19T06:57:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.170+0000 7ff426e6e740 1 -- start start 2024-09-19T06:57:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.170+0000 7ff426e6e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da75704300 0x55da7570e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.170+0000 7ff426e6e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704c20 0x55da75727290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.170+0000 7ff426e6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da7570f4c0 0x55da757277d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.170+0000 7ff426e6e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff40000e840 con 0x55da75704c20 2024-09-19T06:57:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.170+0000 7ff426e6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff400010380 con 0x55da7570f4c0 2024-09-19T06:57:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.170+0000 7ff426e6e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff40000ef10 con 0x55da75704300 2024-09-19T06:57:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.170+0000 7ff421674640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704c20 0x55da75727290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.170+0000 7ff421674640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704c20 0x55da75727290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45750/0 (socket says 172.21.15.103:45750) 2024-09-19T06:57:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.170+0000 7ff421674640 1 -- 172.21.15.103:0/1732904714 learned_addr learned my addr 172.21.15.103:0/1732904714 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.170+0000 7ff422676640 1 --2- 172.21.15.103:0/1732904714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da7570f4c0 0x55da757277d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.171+0000 7ff421e75640 1 --2- 172.21.15.103:0/1732904714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da75704300 0x55da7570e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.171+0000 7ff421674640 1 -- 172.21.15.103:0/1732904714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da75704300 msgr2=0x55da7570e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.171+0000 7ff421674640 1 --2- 172.21.15.103:0/1732904714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da75704300 0x55da7570e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.171+0000 7ff421674640 1 -- 172.21.15.103:0/1732904714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da7570f4c0 msgr2=0x55da757277d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.171+0000 7ff421674640 1 --2- 172.21.15.103:0/1732904714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da7570f4c0 0x55da757277d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.171+0000 7ff421674640 1 -- 172.21.15.103:0/1732904714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da757006a0 con 0x55da75704c20 2024-09-19T06:57:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.171+0000 7ff421e75640 1 --2- 172.21.15.103:0/1732904714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da75704300 0x55da7570e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.171+0000 7ff422676640 1 --2- 172.21.15.103:0/1732904714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da7570f4c0 0x55da757277d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.171+0000 7ff421674640 1 --2- 172.21.15.103:0/1732904714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704c20 0x55da75727290 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff40c09b600 tx=0x7ff40c09b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9db163555c99f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.171+0000 7ff4077fe640 1 -- 172.21.15.103:0/1732904714 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c0860b0 con 0x55da75704c20 2024-09-19T06:57:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.171+0000 7ff4077fe640 1 -- 172.21.15.103:0/1732904714 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff40c09c000 con 0x55da75704c20 2024-09-19T06:57:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.171+0000 7ff4077fe640 1 -- 172.21.15.103:0/1732904714 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c0a40e0 con 0x55da75704c20 2024-09-19T06:57:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.172+0000 7ff426e6e740 1 -- 172.21.15.103:0/1732904714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704c20 msgr2=0x55da75727290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.172+0000 7ff426e6e740 1 --2- 172.21.15.103:0/1732904714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704c20 0x55da75727290 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff40c09b600 tx=0x7ff40c09b6d0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.172+0000 7ff426e6e740 1 -- 172.21.15.103:0/1732904714 shutdown_connections 2024-09-19T06:57:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.172+0000 7ff426e6e740 1 --2- 172.21.15.103:0/1732904714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da7570f4c0 0x55da757277d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.172+0000 7ff426e6e740 1 --2- 172.21.15.103:0/1732904714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704c20 0x55da75727290 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.172+0000 7ff426e6e740 1 --2- 172.21.15.103:0/1732904714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da75704300 0x55da7570e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.172+0000 7ff426e6e740 1 -- 172.21.15.103:0/1732904714 >> 172.21.15.103:0/1732904714 conn(0x55da75700870 msgr2=0x55da7570dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.172+0000 7ff426e6e740 1 -- 172.21.15.103:0/1732904714 shutdown_connections 2024-09-19T06:57:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.172+0000 7ff426e6e740 1 -- 172.21.15.103:0/1732904714 wait complete. 2024-09-19T06:57:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.172+0000 7ff426e6e740 1 -- start start 2024-09-19T06:57:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.172+0000 7ff426e6e740 1 Processor -- start 2024-09-19T06:57:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.172+0000 7ff426e6e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704300 0x55da757131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff426e6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da75704c20 0x55da75713700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff426e6e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da7570f4c0 0x55da75713c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff426e6e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff40000fcb0 con 0x55da75704300 2024-09-19T06:57:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff426e6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff40000f5e0 con 0x55da75704c20 2024-09-19T06:57:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff426e6e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff40000c840 con 0x55da7570f4c0 2024-09-19T06:57:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff426e6e740 10 client.0 _ll_register_callbacks cb 0x7ff414011f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff421e75640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704300 0x55da757131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff421e75640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704300 0x55da757131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45760/0 (socket says 172.21.15.103:45760) 2024-09-19T06:57:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff421e75640 1 -- 172.21.15.103:0/2575281046 learned_addr learned my addr 172.21.15.103:0/2575281046 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff421674640 1 --2- 172.21.15.103:0/2575281046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da75704c20 0x55da75713700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff422676640 1 --2- 172.21.15.103:0/2575281046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da7570f4c0 0x55da75713c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff421e75640 1 -- 172.21.15.103:0/2575281046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da7570f4c0 msgr2=0x55da75713c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff421e75640 1 --2- 172.21.15.103:0/2575281046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da7570f4c0 0x55da75713c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff421e75640 1 -- 172.21.15.103:0/2575281046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da75704c20 msgr2=0x55da75713700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff421e75640 1 --2- 172.21.15.103:0/2575281046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da75704c20 0x55da75713700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff421e75640 1 -- 172.21.15.103:0/2575281046 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da75789260 con 0x55da75704300 2024-09-19T06:57:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff421e75640 1 --2- 172.21.15.103:0/2575281046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704300 0x55da757131c0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7ff41802a940 tx=0x7ff41802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d2a081b905408a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff4057fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff4057fa640 1 -- 172.21.15.103:0/2575281046 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418040040 con 0x55da75704300 2024-09-19T06:57:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff4057fa640 1 -- 172.21.15.103:0/2575281046 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8600+0+0 (secure 0 0 0) 0x7ff418031900 con 0x55da75704300 2024-09-19T06:57:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff426e6e740 1 -- 172.21.15.103:0/2575281046 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff40c09c000 con 0x55da75704300 2024-09-19T06:57:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff4057fa640 1 -- 172.21.15.103:0/2575281046 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff418035040 con 0x55da75704300 2024-09-19T06:57:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff426e6e740 10 client.18264 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.173+0000 7ff426e6e740 1 -- 172.21.15.103:0/2575281046 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55da7578a440 con 0x55da75704300 2024-09-19T06:57:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.174+0000 7ff4057fa640 1 -- 172.21.15.103:0/2575281046 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff40c09c000 con 0x55da75704300 2024-09-19T06:57:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.174+0000 7ff4057fa640 1 -- 172.21.15.103:0/2575281046 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff418003aa0 con 0x55da75704300 2024-09-19T06:57:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.174+0000 7ff426e6e740 10 client.18264 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.174+0000 7ff426e6e740 10 client.18264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.174+0000 7ff426e6e740 20 client.18264 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.174+0000 7ff426e6e740 1 --2- 172.21.15.103:0/2575281046 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55da7578a840 0x55da7578cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.174+0000 7ff426e6e740 4 client.18264 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:57:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.174+0000 7ff426e6e740 1 -- 172.21.15.103:0/2575281046 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da75667d70 con 0x55da7578a840 2024-09-19T06:57:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.174+0000 7ff421674640 1 --2- 172.21.15.103:0/2575281046 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55da7578a840 0x55da7578cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.174+0000 7ff4057fa640 10 client.18264 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.174+0000 7ff421674640 1 --2- 172.21.15.103:0/2575281046 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55da7578a840 0x55da7578cc30 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9393a3e7b58f7bfc server_cookie=6094c085098560c2 in_seq=0 out_seq=0 2024-09-19T06:57:05.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.179+0000 7fd27e7fc640 1 -- 172.21.15.103:0/3052744834 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562d63018be0 con 0x562d63133250 2024-09-19T06:57:05.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.179+0000 7fd27e7fc640 10 client.18258 handle_command_reply: tid=0 2024-09-19T06:57:05.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.179+0000 7fd29f71e740 1 client.18258 shutdown 2024-09-19T06:57:05.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.180+0000 7fd29f71e740 1 -- 172.21.15.103:0/3052744834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630b7a30 msgr2=0x562d63131360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.180+0000 7fd29f71e740 1 --2- 172.21.15.103:0/3052744834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630b7a30 0x562d63131360 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd288009900 tx=0x7fd288032f40 comp rx=0 tx=0).stop 2024-09-19T06:57:05.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.180+0000 7fd29f71e740 1 -- 172.21.15.103:0/3052744834 shutdown_connections 2024-09-19T06:57:05.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.180+0000 7fd29f71e740 1 --2- 172.21.15.103:0/3052744834 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562d63133250 0x562d63135670 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.180+0000 7fd29f71e740 1 --2- 172.21.15.103:0/3052744834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d630b7a30 0x562d63131360 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.180+0000 7fd29f71e740 1 --2- 172.21.15.103:0/3052744834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d630ad170 0x562d630bc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.180+0000 7fd29f71e740 1 --2- 172.21.15.103:0/3052744834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d630ac850 0x562d630bbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.180+0000 7fd27e7fc640 0 client.18258 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:05.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.180+0000 7fd29f71e740 1 -- 172.21.15.103:0/3052744834 >> 172.21.15.103:0/3052744834 conn(0x562d630a8de0 msgr2=0x562d630b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:05.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.180+0000 7fd29f71e740 1 -- 172.21.15.103:0/3052744834 shutdown_connections 2024-09-19T06:57:05.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.180+0000 7fd29f71e740 1 -- 172.21.15.103:0/3052744834 wait complete. 2024-09-19T06:57:05.188 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:57:05.188 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:57:05.188 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:57:05.188 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 30, 2024-09-19T06:57:05.188 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:57:05.188 INFO:teuthology.orchestra.run.smithi103.stdout: "2c1f8d0d": { 2024-09-19T06:57:05.188 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 30, 2024-09-19T06:57:05.188 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:57:05.188 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:57:05.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:57:05.189 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:57:05.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:57:05.189 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:57:05.189 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 145.8, 2024-09-19T06:57:05.189 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:57:05.189 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:57:05.189 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:57:05.189 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:57:05.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:57:05.189 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:57:05.189 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:05.189 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:05.189 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:05.190 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:05.190 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:05.190 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:57:05.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.180+0000 7fd29f71e740 20 client.18258 trim_cache size 0 max 16384 2024-09-19T06:57:05.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.181+0000 7fd29d345640 1 -- 172.21.15.103:0/3670594969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2840a3430 msgr2=0x7fd2840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.181+0000 7fd29d345640 1 --2- 172.21.15.103:0/3670594969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2840a3430 0x7fd2840a58f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd290005c40 tx=0x7fd29000a780 comp rx=0 tx=0).stop 2024-09-19T06:57:05.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.181+0000 7fd29d345640 1 -- 172.21.15.103:0/3670594969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2981036d0 msgr2=0x7fd2981988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.181+0000 7fd29d345640 1 --2- 172.21.15.103:0/3670594969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2981036d0 0x7fd2981988f0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fd28000d950 tx=0x7fd28000de10 comp rx=0 tx=0).stop 2024-09-19T06:57:05.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.181+0000 7fd29d345640 1 -- 172.21.15.103:0/3670594969 shutdown_connections 2024-09-19T06:57:05.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.181+0000 7fd29d345640 1 --2- 172.21.15.103:0/3670594969 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2840a3430 0x7fd2840a58f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.181+0000 7fd29d345640 1 --2- 172.21.15.103:0/3670594969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd29810c190 0x7fd29819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.181+0000 7fd29d345640 1 --2- 172.21.15.103:0/3670594969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2981036d0 0x7fd2981988f0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.181+0000 7fd29d345640 1 --2- 172.21.15.103:0/3670594969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd298102db0 0x7fd2981983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.181+0000 7fd29d345640 1 -- 172.21.15.103:0/3670594969 >> 172.21.15.103:0/3670594969 conn(0x7fd2980fc7d0 msgr2=0x7fd2980fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:05.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.182+0000 7fd29d345640 1 -- 172.21.15.103:0/3670594969 shutdown_connections 2024-09-19T06:57:05.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:05.182+0000 7fd29d345640 1 -- 172.21.15.103:0/3670594969 wait complete. 2024-09-19T06:57:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.186+0000 7ff4057fa640 1 -- 172.21.15.103:0/2575281046 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55da75667d70 con 0x55da7578a840 2024-09-19T06:57:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.186+0000 7ff4057fa640 10 client.18264 handle_command_reply: tid=0 2024-09-19T06:57:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.186+0000 7ff426e6e740 1 client.18264 shutdown 2024-09-19T06:57:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.186+0000 7ff426e6e740 1 -- 172.21.15.103:0/2575281046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704300 msgr2=0x55da757131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.186+0000 7ff426e6e740 1 --2- 172.21.15.103:0/2575281046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704300 0x55da757131c0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7ff41802a940 tx=0x7ff41802a420 comp rx=0 tx=0).stop 2024-09-19T06:57:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.186+0000 7ff426e6e740 1 -- 172.21.15.103:0/2575281046 shutdown_connections 2024-09-19T06:57:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.186+0000 7ff426e6e740 1 --2- 172.21.15.103:0/2575281046 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55da7578a840 0x55da7578cc30 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.186+0000 7ff426e6e740 1 --2- 172.21.15.103:0/2575281046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da7570f4c0 0x55da75713c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.186+0000 7ff426e6e740 1 --2- 172.21.15.103:0/2575281046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da75704c20 0x55da75713700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.186+0000 7ff4057fa640 0 client.18264 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.186+0000 7ff426e6e740 1 --2- 172.21.15.103:0/2575281046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da75704300 0x55da757131c0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.186+0000 7ff426e6e740 1 -- 172.21.15.103:0/2575281046 >> 172.21.15.103:0/2575281046 conn(0x55da75700870 msgr2=0x55da7570c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.186+0000 7ff426e6e740 1 -- 172.21.15.103:0/2575281046 shutdown_connections 2024-09-19T06:57:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.186+0000 7ff426e6e740 1 -- 172.21.15.103:0/2575281046 wait complete. 2024-09-19T06:57:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.186+0000 7ff426e6e740 20 client.18264 trim_cache size 0 max 16384 2024-09-19T06:57:05.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:57:05.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:57:05.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:57:05.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:57:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.187+0000 7ff4240d9640 1 -- 172.21.15.103:0/1140497314 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4080a34e0 msgr2=0x7ff4080a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.187+0000 7ff4240d9640 1 --2- 172.21.15.103:0/1140497314 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4080a34e0 0x7ff4080a59a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff414005fd0 tx=0x7ff414009500 comp rx=0 tx=0).stop 2024-09-19T06:57:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.187+0000 7ff4240d9640 1 -- 172.21.15.103:0/1140497314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff41c102460 msgr2=0x7ff41c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.187+0000 7ff4240d9640 1 --2- 172.21.15.103:0/1140497314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff41c102460 0x7ff41c074a30 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00d950 tx=0x7ff40c00de10 comp rx=0 tx=0).stop 2024-09-19T06:57:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.187+0000 7ff4240d9640 1 -- 172.21.15.103:0/1140497314 shutdown_connections 2024-09-19T06:57:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.187+0000 7ff4240d9640 1 --2- 172.21.15.103:0/1140497314 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff4080a34e0 0x7ff4080a59a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.187+0000 7ff4240d9640 1 --2- 172.21.15.103:0/1140497314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff41c107ad0 0x7ff41c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.187+0000 7ff4240d9640 1 --2- 172.21.15.103:0/1140497314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff41c102460 0x7ff41c074a30 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.187+0000 7ff4240d9640 1 --2- 172.21.15.103:0/1140497314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff41c101ac0 0x7ff41c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.187+0000 7ff4240d9640 1 -- 172.21.15.103:0/1140497314 >> 172.21.15.103:0/1140497314 conn(0x7ff41c0fd820 msgr2=0x7ff41c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.187+0000 7ff4240d9640 1 -- 172.21.15.103:0/1140497314 shutdown_connections 2024-09-19T06:57:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:05.187+0000 7ff4240d9640 1 -- 172.21.15.103:0/1140497314 wait complete. 2024-09-19T06:57:05.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:04 smithi134 ceph-mon[35356]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:57:05.196 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:04 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T06:57:05.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:04 smithi174 ceph-mon[35637]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:57:05.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:04 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T06:57:05.929 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2c1f8d0d', quiesce took 4.8 seconds. Will release after: 20.95 2024-09-19T06:57:06.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:05 smithi103 ceph-mon[31158]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T06:57:06.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:05 smithi134 ceph-mon[35356]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T06:57:06.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:05 smithi174 ceph-mon[35637]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T06:57:07.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:06 smithi134 ceph-mon[35356]: pgmap v242: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 34 op/s 2024-09-19T06:57:07.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:06 smithi174 ceph-mon[35637]: pgmap v242: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 34 op/s 2024-09-19T06:57:07.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:06 smithi103 ceph-mon[31158]: pgmap v242: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 34 op/s 2024-09-19T06:57:08.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:08 smithi103 ceph-mon[31158]: pgmap v243: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 34 op/s 2024-09-19T06:57:08.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:08 smithi134 ceph-mon[35356]: pgmap v243: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 34 op/s 2024-09-19T06:57:08.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:08 smithi174 ceph-mon[35637]: pgmap v243: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 34 op/s 2024-09-19T06:57:10.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:10 smithi103 ceph-mon[31158]: pgmap v244: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 37 op/s 2024-09-19T06:57:10.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:10 smithi134 ceph-mon[35356]: pgmap v244: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 37 op/s 2024-09-19T06:57:10.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:10 smithi174 ceph-mon[35637]: pgmap v244: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 37 op/s 2024-09-19T06:57:12.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:12 smithi103 ceph-mon[31158]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T06:57:12.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:12 smithi103 ceph-mon[31158]: pgmap v246: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 104 KiB/s wr, 47 op/s 2024-09-19T06:57:12.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:12 smithi134 ceph-mon[35356]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T06:57:12.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:12 smithi134 ceph-mon[35356]: pgmap v246: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 104 KiB/s wr, 47 op/s 2024-09-19T06:57:13.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:12 smithi174 ceph-mon[35637]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T06:57:13.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:12 smithi174 ceph-mon[35637]: pgmap v246: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 104 KiB/s wr, 47 op/s 2024-09-19T06:57:14.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:14 smithi103 ceph-mon[31158]: pgmap v247: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 91 KiB/s wr, 41 op/s 2024-09-19T06:57:14.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:14 smithi134 ceph-mon[35356]: pgmap v247: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 91 KiB/s wr, 41 op/s 2024-09-19T06:57:14.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:14 smithi174 ceph-mon[35637]: pgmap v247: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 91 KiB/s wr, 41 op/s 2024-09-19T06:57:16.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:16 smithi103 ceph-mon[31158]: pgmap v248: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-09-19T06:57:16.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:16 smithi134 ceph-mon[35356]: pgmap v248: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-09-19T06:57:16.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:16 smithi174 ceph-mon[35637]: pgmap v248: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-09-19T06:57:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.283+0000 7f040fa27640 1 -- 172.21.15.103:0/4040093497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0408100a00 msgr2=0x7f0408100de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.283+0000 7f040fa27640 1 --2- 172.21.15.103:0/4040093497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0408100a00 0x7f0408100de0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009cd0 tx=0x7f03f802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.283+0000 7f040fa27640 1 -- 172.21.15.103:0/4040093497 shutdown_connections 2024-09-19T06:57:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.283+0000 7f040fa27640 1 --2- 172.21.15.103:0/4040093497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0408105f80 0x7f0408106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.283+0000 7f040fa27640 1 --2- 172.21.15.103:0/4040093497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04081013b0 0x7f0408105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.283+0000 7f040fa27640 1 --2- 172.21.15.103:0/4040093497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0408100a00 0x7f0408100de0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.283+0000 7f040fa27640 1 -- 172.21.15.103:0/4040093497 >> 172.21.15.103:0/4040093497 conn(0x7f04080fc6f0 msgr2=0x7f04080feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.283+0000 7f040fa27640 1 -- 172.21.15.103:0/4040093497 shutdown_connections 2024-09-19T06:57:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.283+0000 7f040fa27640 1 -- 172.21.15.103:0/4040093497 wait complete. 2024-09-19T06:57:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.284+0000 7f040fa27640 1 Processor -- start 2024-09-19T06:57:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.284+0000 7f040fa27640 1 -- start start 2024-09-19T06:57:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.284+0000 7f040fa27640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0408100a00 0x7f04081a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.284+0000 7f040fa27640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04081013b0 0x7f04081a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.284+0000 7f040fa27640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0408105f80 0x7f040819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.284+0000 7f040fa27640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0408118490 con 0x7f04081013b0 2024-09-19T06:57:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.284+0000 7f040fa27640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0408118310 con 0x7f0408105f80 2024-09-19T06:57:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.284+0000 7f040fa27640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0408118610 con 0x7f0408100a00 2024-09-19T06:57:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.284+0000 7f040cfc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04081013b0 0x7f04081a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.284+0000 7f040cfc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04081013b0 0x7f04081a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44282/0 (socket says 172.21.15.103:44282) 2024-09-19T06:57:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.284+0000 7f040cfc2640 1 -- 172.21.15.103:0/2024411973 learned_addr learned my addr 172.21.15.103:0/2024411973 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.285+0000 7f040d7c3640 1 --2- 172.21.15.103:0/2024411973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0408100a00 0x7f04081a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.285+0000 7f040dfc4640 1 --2- 172.21.15.103:0/2024411973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0408105f80 0x7f040819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.285+0000 7f040cfc2640 1 -- 172.21.15.103:0/2024411973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0408100a00 msgr2=0x7f04081a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.285+0000 7f040cfc2640 1 --2- 172.21.15.103:0/2024411973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0408100a00 0x7f04081a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.285+0000 7f040cfc2640 1 -- 172.21.15.103:0/2024411973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0408105f80 msgr2=0x7f040819e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.285+0000 7f040cfc2640 1 --2- 172.21.15.103:0/2024411973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0408105f80 0x7f040819e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.285+0000 7f040cfc2640 1 -- 172.21.15.103:0/2024411973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040819f020 con 0x7f04081013b0 2024-09-19T06:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.285+0000 7f040cfc2640 1 --2- 172.21.15.103:0/2024411973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04081013b0 0x7f04081a4be0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f03fc00d950 tx=0x7f03fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.285+0000 7f04067fc640 1 -- 172.21.15.103:0/2024411973 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc017070 con 0x7f04081013b0 2024-09-19T06:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.285+0000 7f04067fc640 1 -- 172.21.15.103:0/2024411973 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03fc009c30 con 0x7f04081013b0 2024-09-19T06:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.285+0000 7f040fa27640 1 -- 172.21.15.103:0/2024411973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040819f310 con 0x7f04081013b0 2024-09-19T06:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.285+0000 7f04067fc640 1 -- 172.21.15.103:0/2024411973 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc00f6b0 con 0x7f04081013b0 2024-09-19T06:57:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.285+0000 7f040fa27640 1 -- 172.21.15.103:0/2024411973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04081ab620 con 0x7f04081013b0 2024-09-19T06:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.287+0000 7f04067fc640 1 -- 172.21.15.103:0/2024411973 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f03fc0109d0 con 0x7f04081013b0 2024-09-19T06:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.288+0000 7f04127c7740 1 Processor -- start 2024-09-19T06:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.288+0000 7f04127c7740 1 -- start start 2024-09-19T06:57:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.288+0000 7f04127c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54aaf0 0x562cec54aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.288+0000 7f04127c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec555770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.288+0000 7f04127c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 0x562cec5580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.288+0000 7f04127c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562cec4af2f0 con 0x562cec54b410 2024-09-19T06:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.288+0000 7f04127c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562cec457270 con 0x562cec555cb0 2024-09-19T06:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.288+0000 7f04127c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562cec4ab2b0 con 0x562cec54aaf0 2024-09-19T06:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.288+0000 7f04067fc640 1 --2- 172.21.15.103:0/2024411973 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03f00a3460 0x7f03f00a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.288+0000 7f040d7c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec555770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.288+0000 7f040d7c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec555770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44288/0 (socket says 172.21.15.103:44288) 2024-09-19T06:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.288+0000 7f040d7c3640 1 -- 172.21.15.103:0/251973443 learned_addr learned my addr 172.21.15.103:0/251973443 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.289+0000 7f040dfc4640 1 --2- 172.21.15.103:0/251973443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54aaf0 0x562cec54aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.289+0000 7f040cfc2640 1 --2- 172.21.15.103:0/251973443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 0x562cec5580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.289+0000 7f040dfc4640 1 --2- 172.21.15.103:0/2024411973 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03f00a3460 0x7f03f00a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.289+0000 7f04067fc640 1 -- 172.21.15.103:0/2024411973 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f03fc0cf8e0 con 0x7f04081013b0 2024-09-19T06:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.289+0000 7f040d7c3640 1 -- 172.21.15.103:0/251973443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54aaf0 msgr2=0x562cec54aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.289+0000 7f040d7c3640 1 --2- 172.21.15.103:0/251973443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54aaf0 0x562cec54aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.289+0000 7f040d7c3640 1 -- 172.21.15.103:0/251973443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 msgr2=0x562cec5580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.289+0000 7f040d7c3640 1 --2- 172.21.15.103:0/251973443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 0x562cec5580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.289+0000 7f040d7c3640 1 -- 172.21.15.103:0/251973443 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cec456340 con 0x562cec54b410 2024-09-19T06:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.289+0000 7f040cfc2640 1 --2- 172.21.15.103:0/251973443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 0x562cec5580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.289+0000 7f040d7c3640 1 --2- 172.21.15.103:0/251973443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec555770 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009cd0 tx=0x7f03f8005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c688f78d7c93d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.289+0000 7f040dfc4640 1 --2- 172.21.15.103:0/2024411973 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03f00a3460 0x7f03f00a5920 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0400012870 tx=0x7f040000b2e0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.289+0000 7f03ef7fe640 1 -- 172.21.15.103:0/251973443 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8048070 con 0x562cec54b410 2024-09-19T06:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.289+0000 7f03ef7fe640 1 -- 172.21.15.103:0/251973443 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f8036070 con 0x562cec54b410 2024-09-19T06:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.289+0000 7f03ef7fe640 1 -- 172.21.15.103:0/251973443 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8004520 con 0x562cec54b410 2024-09-19T06:57:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.290+0000 7f04127c7740 1 -- 172.21.15.103:0/251973443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 msgr2=0x562cec555770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.290+0000 7f04127c7740 1 --2- 172.21.15.103:0/251973443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec555770 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009cd0 tx=0x7f03f8005fa0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.290+0000 7f04127c7740 1 -- 172.21.15.103:0/251973443 shutdown_connections 2024-09-19T06:57:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.290+0000 7f04127c7740 1 --2- 172.21.15.103:0/251973443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 0x562cec5580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.290+0000 7f04127c7740 1 --2- 172.21.15.103:0/251973443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec555770 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.290+0000 7f04127c7740 1 --2- 172.21.15.103:0/251973443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54aaf0 0x562cec54aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.290+0000 7f04127c7740 1 -- 172.21.15.103:0/251973443 >> 172.21.15.103:0/251973443 conn(0x562cec547060 msgr2=0x562cec547440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.290+0000 7f04127c7740 1 -- 172.21.15.103:0/251973443 shutdown_connections 2024-09-19T06:57:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.290+0000 7f04127c7740 1 -- 172.21.15.103:0/251973443 wait complete. 2024-09-19T06:57:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.290+0000 7f04127c7740 1 -- start start 2024-09-19T06:57:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f04127c7740 1 Processor -- start 2024-09-19T06:57:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f04127c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54aaf0 0x562cec552bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f04127c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec553130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f04127c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 0x562cec553670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f04127c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562cec558880 con 0x562cec54b410 2024-09-19T06:57:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f04127c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562cec438290 con 0x562cec555cb0 2024-09-19T06:57:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f04127c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562cec558a00 con 0x562cec54aaf0 2024-09-19T06:57:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f04127c7740 10 client.0 _ll_register_callbacks cb 0x562cec469140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f040cfc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec553130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f040cfc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec553130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44292/0 (socket says 172.21.15.103:44292) 2024-09-19T06:57:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f040cfc2640 1 -- 172.21.15.103:0/2204507117 learned_addr learned my addr 172.21.15.103:0/2204507117 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f040d7c3640 1 --2- 172.21.15.103:0/2204507117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54aaf0 0x562cec552bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f040dfc4640 1 --2- 172.21.15.103:0/2204507117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 0x562cec553670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f040cfc2640 1 -- 172.21.15.103:0/2204507117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54aaf0 msgr2=0x562cec552bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f040cfc2640 1 --2- 172.21.15.103:0/2204507117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54aaf0 0x562cec552bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f040cfc2640 1 -- 172.21.15.103:0/2204507117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 msgr2=0x562cec553670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f040cfc2640 1 --2- 172.21.15.103:0/2204507117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 0x562cec553670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f040cfc2640 1 -- 172.21.15.103:0/2204507117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cec1c2210 con 0x562cec54b410 2024-09-19T06:57:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.291+0000 7f040dfc4640 1 --2- 172.21.15.103:0/2204507117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 0x562cec553670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.292+0000 7f040cfc2640 1 --2- 172.21.15.103:0/2204507117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec553130 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f03fc09bbe0 tx=0x7f03fc01e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1117dc63a6acf272 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.292+0000 7f03ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.292+0000 7f03ecff9640 1 -- 172.21.15.103:0/2204507117 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc020040 con 0x562cec54b410 2024-09-19T06:57:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.292+0000 7f03ecff9640 1 -- 172.21.15.103:0/2204507117 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f03fc0666b0 con 0x562cec54b410 2024-09-19T06:57:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.292+0000 7f04127c7740 1 -- 172.21.15.103:0/2204507117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f8036070 con 0x562cec54b410 2024-09-19T06:57:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.292+0000 7f03ecff9640 1 -- 172.21.15.103:0/2204507117 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03fc009de0 con 0x562cec54b410 2024-09-19T06:57:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.292+0000 7f04127c7740 10 client.18276 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.292+0000 7f04127c7740 1 -- 172.21.15.103:0/2204507117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562cec28e620 con 0x562cec54b410 2024-09-19T06:57:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.292+0000 7f03ecff9640 1 -- 172.21.15.103:0/2204507117 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f8036070 con 0x562cec54b410 2024-09-19T06:57:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.292+0000 7f03ecff9640 1 -- 172.21.15.103:0/2204507117 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f03fc0a4380 con 0x562cec54b410 2024-09-19T06:57:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.292+0000 7f04127c7740 10 client.18276 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.292+0000 7f04127c7740 10 client.18276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.292+0000 7f04127c7740 20 client.18276 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.292+0000 7f04127c7740 1 --2- 172.21.15.103:0/2204507117 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562cec565ca0 0x562cec568090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.292+0000 7f04127c7740 4 client.18276 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.292+0000 7f04127c7740 1 -- 172.21.15.103:0/2204507117 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cec4bca20 con 0x562cec565ca0 2024-09-19T06:57:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.293+0000 7f040d7c3640 1 --2- 172.21.15.103:0/2204507117 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562cec565ca0 0x562cec568090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.293+0000 7f03ecff9640 10 client.18276 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.293+0000 7f040d7c3640 1 --2- 172.21.15.103:0/2204507117 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562cec565ca0 0x562cec568090 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=232567b94ce44dd3 server_cookie=4948e455fbeb54a1 in_seq=0 out_seq=0 2024-09-19T06:57:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.294+0000 7f03ecff9640 1 -- 172.21.15.103:0/2204507117 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cec4bca20 con 0x562cec565ca0 2024-09-19T06:57:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.294+0000 7f03ecff9640 10 client.18276 handle_command_reply: tid=0 2024-09-19T06:57:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.294+0000 7f04127c7740 1 client.18276 shutdown 2024-09-19T06:57:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.294+0000 7f04127c7740 1 -- 172.21.15.103:0/2204507117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 msgr2=0x562cec553130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.294+0000 7f04127c7740 1 --2- 172.21.15.103:0/2204507117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec553130 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f03fc09bbe0 tx=0x7f03fc01e770 comp rx=0 tx=0).stop 2024-09-19T06:57:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.294+0000 7f04127c7740 1 -- 172.21.15.103:0/2204507117 shutdown_connections 2024-09-19T06:57:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.294+0000 7f04127c7740 1 --2- 172.21.15.103:0/2204507117 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562cec565ca0 0x562cec568090 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.294+0000 7f04127c7740 1 --2- 172.21.15.103:0/2204507117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 0x562cec553670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.294+0000 7f04127c7740 1 --2- 172.21.15.103:0/2204507117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec553130 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.294+0000 7f03ecff9640 0 client.18276 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.294+0000 7f04127c7740 1 --2- 172.21.15.103:0/2204507117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54aaf0 0x562cec552bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.294+0000 7f04127c7740 1 -- 172.21.15.103:0/2204507117 >> 172.21.15.103:0/2204507117 conn(0x562cec547060 msgr2=0x562cec5461c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.294+0000 7f04127c7740 1 -- 172.21.15.103:0/2204507117 shutdown_connections 2024-09-19T06:57:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.294+0000 7f04127c7740 1 -- 172.21.15.103:0/2204507117 wait complete. 2024-09-19T06:57:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.294+0000 7f04127c7740 20 client.18276 trim_cache size 0 max 16384 2024-09-19T06:57:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.309+0000 7f04127c7740 1 Processor -- start 2024-09-19T06:57:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.309+0000 7f04127c7740 1 -- start start 2024-09-19T06:57:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.309+0000 7f04127c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec54aaf0 0x562cec5534a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.309+0000 7f04127c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54b410 0x562cec54cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.309+0000 7f04127c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec555cb0 0x562cec56d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.309+0000 7f04127c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f04081b3d40 con 0x562cec555cb0 2024-09-19T06:57:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.309+0000 7f04127c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04081b5880 con 0x562cec54aaf0 2024-09-19T06:57:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.309+0000 7f04127c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04081b4410 con 0x562cec54b410 2024-09-19T06:57:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.309+0000 7f040dfc4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec555cb0 0x562cec56d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.309+0000 7f040dfc4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec555cb0 0x562cec56d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44302/0 (socket says 172.21.15.103:44302) 2024-09-19T06:57:18.311 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.309+0000 7f040dfc4640 1 -- 172.21.15.103:0/2688398993 learned_addr learned my addr 172.21.15.103:0/2688398993 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.309+0000 7f040dfc4640 1 -- 172.21.15.103:0/2688398993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54b410 msgr2=0x562cec54cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.309+0000 7f040dfc4640 1 --2- 172.21.15.103:0/2688398993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54b410 0x562cec54cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.309+0000 7f040dfc4640 1 -- 172.21.15.103:0/2688398993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec54aaf0 msgr2=0x562cec5534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.309+0000 7f040dfc4640 1 --2- 172.21.15.103:0/2688398993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec54aaf0 0x562cec5534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.309+0000 7f040dfc4640 1 -- 172.21.15.103:0/2688398993 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cec54b240 con 0x562cec555cb0 2024-09-19T06:57:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.310+0000 7f040dfc4640 1 --2- 172.21.15.103:0/2688398993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec555cb0 0x562cec56d900 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f040000bbb0 tx=0x7f040000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c070168cfc42fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.310+0000 7f03eeffd640 1 -- 172.21.15.103:0/2688398993 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400025030 con 0x562cec555cb0 2024-09-19T06:57:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.310+0000 7f03eeffd640 1 -- 172.21.15.103:0/2688398993 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04000124b0 con 0x562cec555cb0 2024-09-19T06:57:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.310+0000 7f03eeffd640 1 -- 172.21.15.103:0/2688398993 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400021450 con 0x562cec555cb0 2024-09-19T06:57:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.310+0000 7f04127c7740 1 -- 172.21.15.103:0/2688398993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec555cb0 msgr2=0x562cec56d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.310+0000 7f04127c7740 1 --2- 172.21.15.103:0/2688398993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec555cb0 0x562cec56d900 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f040000bbb0 tx=0x7f040000d000 comp rx=0 tx=0).stop 2024-09-19T06:57:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.310+0000 7f04127c7740 1 -- 172.21.15.103:0/2688398993 shutdown_connections 2024-09-19T06:57:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.310+0000 7f04127c7740 1 --2- 172.21.15.103:0/2688398993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec555cb0 0x562cec56d900 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.310+0000 7f04127c7740 1 --2- 172.21.15.103:0/2688398993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54b410 0x562cec54cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.310+0000 7f04127c7740 1 --2- 172.21.15.103:0/2688398993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec54aaf0 0x562cec5534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.310+0000 7f04127c7740 1 -- 172.21.15.103:0/2688398993 >> 172.21.15.103:0/2688398993 conn(0x562cec547060 msgr2=0x562cec54c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.311+0000 7f04127c7740 1 -- 172.21.15.103:0/2688398993 shutdown_connections 2024-09-19T06:57:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.311+0000 7f04127c7740 1 -- 172.21.15.103:0/2688398993 wait complete. 2024-09-19T06:57:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.311+0000 7f04127c7740 1 -- start start 2024-09-19T06:57:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.311+0000 7f04127c7740 1 Processor -- start 2024-09-19T06:57:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.311+0000 7f04127c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54aaf0 0x562cec55e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.311+0000 7f04127c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec559480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.311+0000 7f04127c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 0x562cec5599c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.311+0000 7f04127c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f04081b51b0 con 0x562cec54b410 2024-09-19T06:57:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.311+0000 7f04127c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04081b4ae0 con 0x562cec555cb0 2024-09-19T06:57:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.311+0000 7f04127c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04081b1d40 con 0x562cec54aaf0 2024-09-19T06:57:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.311+0000 7f04127c7740 10 client.0 _ll_register_callbacks cb 0x7f03fc084f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.311+0000 7f040cfc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec559480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.311+0000 7f040cfc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec559480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44304/0 (socket says 172.21.15.103:44304) 2024-09-19T06:57:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.311+0000 7f040cfc2640 1 -- 172.21.15.103:0/59596780 learned_addr learned my addr 172.21.15.103:0/59596780 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f040dfc4640 1 --2- 172.21.15.103:0/59596780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 0x562cec5599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f040d7c3640 1 --2- 172.21.15.103:0/59596780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54aaf0 0x562cec55e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f040cfc2640 1 -- 172.21.15.103:0/59596780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54aaf0 msgr2=0x562cec55e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f040cfc2640 1 --2- 172.21.15.103:0/59596780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54aaf0 0x562cec55e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f040cfc2640 1 -- 172.21.15.103:0/59596780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 msgr2=0x562cec5599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f040cfc2640 1 --2- 172.21.15.103:0/59596780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 0x562cec5599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f040cfc2640 1 -- 172.21.15.103:0/59596780 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cec559f00 con 0x562cec54b410 2024-09-19T06:57:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f040d7c3640 1 --2- 172.21.15.103:0/59596780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54aaf0 0x562cec55e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f040dfc4640 1 --2- 172.21.15.103:0/59596780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 0x562cec5599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f040cfc2640 1 --2- 172.21.15.103:0/59596780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec559480 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f03fc010fd0 tx=0x7f03fc0cd530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557acd4e8c8fe278 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f03ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f03ecff9640 1 -- 172.21.15.103:0/59596780 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc020060 con 0x562cec54b410 2024-09-19T06:57:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f03ecff9640 1 -- 172.21.15.103:0/59596780 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f03fc096410 con 0x562cec54b410 2024-09-19T06:57:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f04127c7740 1 -- 172.21.15.103:0/59596780 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04000124b0 con 0x562cec54b410 2024-09-19T06:57:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f03ecff9640 1 -- 172.21.15.103:0/59596780 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03fc09bc50 con 0x562cec54b410 2024-09-19T06:57:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f04127c7740 10 client.18282 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f04127c7740 1 -- 172.21.15.103:0/59596780 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562cec5d0610 con 0x562cec54b410 2024-09-19T06:57:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f03ecff9640 1 -- 172.21.15.103:0/59596780 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04000124b0 con 0x562cec54b410 2024-09-19T06:57:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.312+0000 7f03ecff9640 1 -- 172.21.15.103:0/59596780 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f03fc086110 con 0x562cec54b410 2024-09-19T06:57:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.313+0000 7f04127c7740 10 client.18282 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.313+0000 7f04127c7740 10 client.18282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.313+0000 7f04127c7740 20 client.18282 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.313+0000 7f04127c7740 1 --2- 172.21.15.103:0/59596780 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562cec5d0d80 0x562cec5d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.313+0000 7f04127c7740 4 client.18282 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:57:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.313+0000 7f04127c7740 1 -- 172.21.15.103:0/59596780 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562cec4b6d70 con 0x562cec5d0d80 2024-09-19T06:57:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.313+0000 7f040d7c3640 1 --2- 172.21.15.103:0/59596780 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562cec5d0d80 0x562cec5d31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.313+0000 7f03ecff9640 10 client.18282 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.313+0000 7f040d7c3640 1 --2- 172.21.15.103:0/59596780 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562cec5d0d80 0x562cec5d31a0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecea615e6b9745f8 server_cookie=ca7c8300bf0a7fd5 in_seq=0 out_seq=0 2024-09-19T06:57:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.321+0000 7f03ecff9640 1 -- 172.21.15.103:0/59596780 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562cec4b6d70 con 0x562cec5d0d80 2024-09-19T06:57:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.321+0000 7f03ecff9640 10 client.18282 handle_command_reply: tid=0 2024-09-19T06:57:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.321+0000 7f04127c7740 1 client.18282 shutdown 2024-09-19T06:57:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f04127c7740 1 -- 172.21.15.103:0/59596780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 msgr2=0x562cec559480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f04127c7740 1 --2- 172.21.15.103:0/59596780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec559480 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f03fc010fd0 tx=0x7f03fc0cd530 comp rx=0 tx=0).stop 2024-09-19T06:57:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f04127c7740 1 -- 172.21.15.103:0/59596780 shutdown_connections 2024-09-19T06:57:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f04127c7740 1 --2- 172.21.15.103:0/59596780 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562cec5d0d80 0x562cec5d31a0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f04127c7740 1 --2- 172.21.15.103:0/59596780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cec555cb0 0x562cec5599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f04127c7740 1 --2- 172.21.15.103:0/59596780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cec54b410 0x562cec559480 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f03ecff9640 0 client.18282 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f04127c7740 1 --2- 172.21.15.103:0/59596780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cec54aaf0 0x562cec55e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f04127c7740 1 -- 172.21.15.103:0/59596780 >> 172.21.15.103:0/59596780 conn(0x562cec547060 msgr2=0x562cec5461c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f04127c7740 1 -- 172.21.15.103:0/59596780 shutdown_connections 2024-09-19T06:57:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f04127c7740 1 -- 172.21.15.103:0/59596780 wait complete. 2024-09-19T06:57:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f04127c7740 20 client.18282 trim_cache size 0 max 16384 2024-09-19T06:57:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:57:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:57:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:57:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:57:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f040fa27640 1 -- 172.21.15.103:0/2024411973 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03f00a3460 msgr2=0x7f03f00a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f040fa27640 1 --2- 172.21.15.103:0/2024411973 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03f00a3460 0x7f03f00a5920 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0400012870 tx=0x7f040000b2e0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f040fa27640 1 -- 172.21.15.103:0/2024411973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04081013b0 msgr2=0x7f04081a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f040fa27640 1 --2- 172.21.15.103:0/2024411973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04081013b0 0x7f04081a4be0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f03fc00d950 tx=0x7f03fc00de10 comp rx=0 tx=0).stop 2024-09-19T06:57:18.325 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f040fa27640 1 -- 172.21.15.103:0/2024411973 shutdown_connections 2024-09-19T06:57:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f040fa27640 1 --2- 172.21.15.103:0/2024411973 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03f00a3460 0x7f03f00a5920 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f040fa27640 1 --2- 172.21.15.103:0/2024411973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0408105f80 0x7f040819e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f040fa27640 1 --2- 172.21.15.103:0/2024411973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04081013b0 0x7f04081a4be0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f040fa27640 1 --2- 172.21.15.103:0/2024411973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0408100a00 0x7f04081a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.322+0000 7f040fa27640 1 -- 172.21.15.103:0/2024411973 >> 172.21.15.103:0/2024411973 conn(0x7f04080fc6f0 msgr2=0x7f0408103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.323+0000 7f040fa27640 1 -- 172.21.15.103:0/2024411973 shutdown_connections 2024-09-19T06:57:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:18.323+0000 7f040fa27640 1 -- 172.21.15.103:0/2024411973 wait complete. 2024-09-19T06:57:18.385 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:18 smithi174 ceph-mon[35637]: pgmap v249: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-09-19T06:57:18.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:18 smithi103 ceph-mon[31158]: pgmap v249: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-09-19T06:57:18.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:18 smithi134 ceph-mon[35356]: pgmap v249: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-09-19T06:57:20.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:20 smithi103 ceph-mon[31158]: pgmap v250: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-19T06:57:20.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:20 smithi134 ceph-mon[35356]: pgmap v250: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-19T06:57:20.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:20 smithi174 ceph-mon[35637]: pgmap v250: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-19T06:57:22.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:22 smithi103 ceph-mon[31158]: pgmap v251: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.1 KiB/s wr, 11 op/s 2024-09-19T06:57:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:22 smithi134 ceph-mon[35356]: pgmap v251: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.1 KiB/s wr, 11 op/s 2024-09-19T06:57:22.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:22 smithi174 ceph-mon[35637]: pgmap v251: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.1 KiB/s wr, 11 op/s 2024-09-19T06:57:24.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:24 smithi103 ceph-mon[31158]: pgmap v252: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-09-19T06:57:24.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:24 smithi134 ceph-mon[35356]: pgmap v252: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-09-19T06:57:24.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:24 smithi174 ceph-mon[35637]: pgmap v252: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-09-19T06:57:26.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:26 smithi103 ceph-mon[31158]: pgmap v253: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2024-09-19T06:57:26.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:26 smithi134 ceph-mon[35356]: pgmap v253: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2024-09-19T06:57:26.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:26 smithi174 ceph-mon[35637]: pgmap v253: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2024-09-19T06:57:26.880 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:57:27.143 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:27.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.609+0000 7fd35db93640 1 -- 172.21.15.103:0/270515785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd358102790 msgr2=0x7fd3581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:27.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.609+0000 7fd35db93640 1 --2- 172.21.15.103:0/270515785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd358102790 0x7fd3581067e0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fd340009900 tx=0x7fd34002a410 comp rx=0 tx=0).stop 2024-09-19T06:57:27.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.609+0000 7fd35db93640 1 -- 172.21.15.103:0/270515785 shutdown_connections 2024-09-19T06:57:27.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.609+0000 7fd35db93640 1 --2- 172.21.15.103:0/270515785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd358106f10 0x7fd35810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:27.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.609+0000 7fd35db93640 1 --2- 172.21.15.103:0/270515785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd358102790 0x7fd3581067e0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:27.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.609+0000 7fd35db93640 1 --2- 172.21.15.103:0/270515785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd358101de0 0x7fd3581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:27.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.609+0000 7fd35db93640 1 -- 172.21.15.103:0/270515785 >> 172.21.15.103:0/270515785 conn(0x7fd3580fd8c0 msgr2=0x7fd3580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:27.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.609+0000 7fd35db93640 1 -- 172.21.15.103:0/270515785 shutdown_connections 2024-09-19T06:57:27.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.609+0000 7fd35db93640 1 -- 172.21.15.103:0/270515785 wait complete. 2024-09-19T06:57:27.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.609+0000 7fd35db93640 1 Processor -- start 2024-09-19T06:57:27.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.609+0000 7fd35db93640 1 -- start start 2024-09-19T06:57:27.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd35db93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd358101de0 0x7fd3581984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:27.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd35db93640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd358102790 0x7fd3581989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:27.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd35db93640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd358106f10 0x7fd35819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:27.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd35db93640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd358111f70 con 0x7fd358101de0 2024-09-19T06:57:27.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd35db93640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd358111df0 con 0x7fd358102790 2024-09-19T06:57:27.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd35db93640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581120f0 con 0x7fd358106f10 2024-09-19T06:57:27.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd358101de0 0x7fd3581984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:27.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd358101de0 0x7fd3581984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47956/0 (socket says 172.21.15.103:47956) 2024-09-19T06:57:27.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd3577fe640 1 -- 172.21.15.103:0/130777783 learned_addr learned my addr 172.21.15.103:0/130777783 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:27.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd357fff640 1 --2- 172.21.15.103:0/130777783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd358106f10 0x7fd35819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:27.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd34ffff640 1 --2- 172.21.15.103:0/130777783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd358102790 0x7fd3581989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:27.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd3577fe640 1 -- 172.21.15.103:0/130777783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd358106f10 msgr2=0x7fd35819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:27.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd3577fe640 1 --2- 172.21.15.103:0/130777783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd358106f10 0x7fd35819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:27.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd3577fe640 1 -- 172.21.15.103:0/130777783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd358102790 msgr2=0x7fd3581989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:27.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd3577fe640 1 --2- 172.21.15.103:0/130777783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd358102790 0x7fd3581989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:27.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd3577fe640 1 -- 172.21.15.103:0/130777783 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35819d460 con 0x7fd358101de0 2024-09-19T06:57:27.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd357fff640 1 --2- 172.21.15.103:0/130777783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd358106f10 0x7fd35819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:27.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd3577fe640 1 --2- 172.21.15.103:0/130777783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd358101de0 0x7fd3581984b0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fd34800d770 tx=0x7fd34800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:27.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd3557fa640 1 -- 172.21.15.103:0/130777783 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348017070 con 0x7fd358101de0 2024-09-19T06:57:27.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd3557fa640 1 -- 172.21.15.103:0/130777783 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd348005500 con 0x7fd358101de0 2024-09-19T06:57:27.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd35db93640 1 -- 172.21.15.103:0/130777783 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35819d750 con 0x7fd358101de0 2024-09-19T06:57:27.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd3557fa640 1 -- 172.21.15.103:0/130777783 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348004e00 con 0x7fd358101de0 2024-09-19T06:57:27.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.610+0000 7fd35db93640 1 -- 172.21.15.103:0/130777783 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3581a5030 con 0x7fd358101de0 2024-09-19T06:57:27.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.612+0000 7fd35db93640 1 -- 172.21.15.103:0/130777783 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3581021c0 con 0x7fd358101de0 2024-09-19T06:57:27.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.613+0000 7fd3557fa640 1 -- 172.21.15.103:0/130777783 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd348004fa0 con 0x7fd358101de0 2024-09-19T06:57:27.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.613+0000 7fd3557fa640 1 --2- 172.21.15.103:0/130777783 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd3440a33b0 0x7fd3440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:27.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.613+0000 7fd3557fa640 1 -- 172.21.15.103:0/130777783 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7fd3480cf870 con 0x7fd358101de0 2024-09-19T06:57:27.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.613+0000 7fd34ffff640 1 --2- 172.21.15.103:0/130777783 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd3440a33b0 0x7fd3440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:27.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.614+0000 7fd34ffff640 1 --2- 172.21.15.103:0/130777783 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd3440a33b0 0x7fd3440a5870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd358108880 tx=0x7fd340005a20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:27.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.618+0000 7fd3557fa640 1 -- 172.21.15.103:0/130777783 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd34809c230 con 0x7fd358101de0 2024-09-19T06:57:27.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.817+0000 7fd35db93640 1 -- 172.21.15.103:0/130777783 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd35810a150 con 0x7fd358101de0 2024-09-19T06:57:27.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.818+0000 7fd3557fa640 1 -- 172.21.15.103:0/130777783 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fd34809bb30 con 0x7fd358101de0 2024-09-19T06:57:27.820 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:57:27.820 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:57:27.821 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:57:27.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.820+0000 7fd35db93640 1 -- 172.21.15.103:0/130777783 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd3440a33b0 msgr2=0x7fd3440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:27.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.820+0000 7fd35db93640 1 --2- 172.21.15.103:0/130777783 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd3440a33b0 0x7fd3440a5870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd358108880 tx=0x7fd340005a20 comp rx=0 tx=0).stop 2024-09-19T06:57:27.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.821+0000 7fd35db93640 1 -- 172.21.15.103:0/130777783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd358101de0 msgr2=0x7fd3581984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:27.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.821+0000 7fd35db93640 1 --2- 172.21.15.103:0/130777783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd358101de0 0x7fd3581984b0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fd34800d770 tx=0x7fd34800dc30 comp rx=0 tx=0).stop 2024-09-19T06:57:27.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.821+0000 7fd35db93640 1 -- 172.21.15.103:0/130777783 shutdown_connections 2024-09-19T06:57:27.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.821+0000 7fd35db93640 1 --2- 172.21.15.103:0/130777783 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd3440a33b0 0x7fd3440a5870 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:27.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.821+0000 7fd35db93640 1 --2- 172.21.15.103:0/130777783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd358106f10 0x7fd35819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:27.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.821+0000 7fd35db93640 1 --2- 172.21.15.103:0/130777783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd358102790 0x7fd3581989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:27.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.821+0000 7fd35db93640 1 --2- 172.21.15.103:0/130777783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd358101de0 0x7fd3581984b0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:27.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.821+0000 7fd35db93640 1 -- 172.21.15.103:0/130777783 >> 172.21.15.103:0/130777783 conn(0x7fd3580fd8c0 msgr2=0x7fd3580fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:27.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.821+0000 7fd35db93640 1 -- 172.21.15.103:0/130777783 shutdown_connections 2024-09-19T06:57:27.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:27.821+0000 7fd35db93640 1 -- 172.21.15.103:0/130777783 wait complete. 2024-09-19T06:57:28.098 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:27 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/130777783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:57:28.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:27 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/130777783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:57:28.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:27 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/130777783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:57:28.578 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 2c1f8d0d --release --await' 2024-09-19T06:57:28.578 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 2c1f8d0d --release --await 2024-09-19T06:57:28.860 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:29.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:28 smithi103 ceph-mon[31158]: pgmap v254: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T06:57:29.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:28 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:57:29.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:28 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:57:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:28 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:57:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:28 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:57:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:28 smithi134 ceph-mon[35356]: pgmap v254: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T06:57:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:28 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:57:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:28 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:57:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:28 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:57:29.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:28 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:57:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:28 smithi174 ceph-mon[35637]: pgmap v254: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T06:57:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:28 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:57:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:28 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:57:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:28 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:57:29.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:28 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:57:29.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.368+0000 7f6016555640 1 -- 172.21.15.103:0/3327400962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6010101610 msgr2=0x7f60101019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:29.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.368+0000 7f6016555640 1 --2- 172.21.15.103:0/3327400962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6010101610 0x7f60101019f0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f6000009900 tx=0x7f600002a410 comp rx=0 tx=0).stop 2024-09-19T06:57:29.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.368+0000 7f6016555640 1 -- 172.21.15.103:0/3327400962 shutdown_connections 2024-09-19T06:57:29.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.368+0000 7f6016555640 1 --2- 172.21.15.103:0/3327400962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6010105e40 0x7f601010d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.368+0000 7f6016555640 1 --2- 172.21.15.103:0/3327400962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6010101fc0 0x7f6010105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.368+0000 7f6016555640 1 --2- 172.21.15.103:0/3327400962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6010101610 0x7f60101019f0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.368+0000 7f6016555640 1 -- 172.21.15.103:0/3327400962 >> 172.21.15.103:0/3327400962 conn(0x7f60100fc5c0 msgr2=0x7f60100fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:29.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.368+0000 7f6016555640 1 -- 172.21.15.103:0/3327400962 shutdown_connections 2024-09-19T06:57:29.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.368+0000 7f6016555640 1 -- 172.21.15.103:0/3327400962 wait complete. 2024-09-19T06:57:29.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.369+0000 7f6016555640 1 Processor -- start 2024-09-19T06:57:29.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.369+0000 7f6016555640 1 -- start start 2024-09-19T06:57:29.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f6016555640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6010101610 0x7f60101992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f6016555640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6010101fc0 0x7f6010199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f6016555640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6010105e40 0x7f60101a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f6016555640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6010110ea0 con 0x7f6010105e40 2024-09-19T06:57:29.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f6016555640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6010110d20 con 0x7f6010101610 2024-09-19T06:57:29.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f6016555640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6010111020 con 0x7f6010101fc0 2024-09-19T06:57:29.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f6014af2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6010105e40 0x7f60101a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f6014af2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6010105e40 0x7f60101a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47976/0 (socket says 172.21.15.103:47976) 2024-09-19T06:57:29.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f6014af2640 1 -- 172.21.15.103:0/3596683476 learned_addr learned my addr 172.21.15.103:0/3596683476 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:29.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f600f7fe640 1 --2- 172.21.15.103:0/3596683476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6010101fc0 0x7f6010199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f600ffff640 1 --2- 172.21.15.103:0/3596683476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6010101610 0x7f60101992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f6014af2640 1 -- 172.21.15.103:0/3596683476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6010101fc0 msgr2=0x7f6010199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:29.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f6014af2640 1 --2- 172.21.15.103:0/3596683476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6010101fc0 0x7f6010199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f6014af2640 1 -- 172.21.15.103:0/3596683476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6010101610 msgr2=0x7f60101992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:29.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f6014af2640 1 --2- 172.21.15.103:0/3596683476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6010101610 0x7f60101992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f6014af2640 1 -- 172.21.15.103:0/3596683476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60101a4890 con 0x7f6010105e40 2024-09-19T06:57:29.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f600f7fe640 1 --2- 172.21.15.103:0/3596683476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6010101fc0 0x7f6010199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:29.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f600ffff640 1 --2- 172.21.15.103:0/3596683476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6010101610 0x7f60101992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:29.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f6014af2640 1 --2- 172.21.15.103:0/3596683476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6010105e40 0x7f60101a4350 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f6008009fd0 tx=0x7f600800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:29.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.370+0000 7f600d7fa640 1 -- 172.21.15.103:0/3596683476 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008017070 con 0x7f6010105e40 2024-09-19T06:57:29.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.371+0000 7f600d7fa640 1 -- 172.21.15.103:0/3596683476 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6008003e50 con 0x7f6010105e40 2024-09-19T06:57:29.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.371+0000 7f6016555640 1 -- 172.21.15.103:0/3596683476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60101a4a60 con 0x7f6010105e40 2024-09-19T06:57:29.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.371+0000 7f600d7fa640 1 -- 172.21.15.103:0/3596683476 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60080106c0 con 0x7f6010105e40 2024-09-19T06:57:29.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.371+0000 7f6016555640 1 -- 172.21.15.103:0/3596683476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60101a4f20 con 0x7f6010105e40 2024-09-19T06:57:29.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.373+0000 7f600d7fa640 1 -- 172.21.15.103:0/3596683476 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f6008003970 con 0x7f6010105e40 2024-09-19T06:57:29.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.373+0000 7f601892e740 1 Processor -- start 2024-09-19T06:57:29.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.373+0000 7f601892e740 1 -- start start 2024-09-19T06:57:29.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.373+0000 7f601892e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf06f20 0x55ec0bf07300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.373+0000 7f601892e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf07840 0x55ec0bf16110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.373+0000 7f601892e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf16650 0x55ec0bf18a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.373+0000 7f601892e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ec0be6c620 con 0x55ec0bf06f20 2024-09-19T06:57:29.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.373+0000 7f601892e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ec0be13e50 con 0x55ec0bf07840 2024-09-19T06:57:29.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.373+0000 7f601892e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ec0be685e0 con 0x55ec0bf16650 2024-09-19T06:57:29.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.373+0000 7f600d7fa640 1 --2- 172.21.15.103:0/3596683476 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5ff80a33b0 0x7f5ff80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.373+0000 7f600f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf06f20 0x55ec0bf07300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.373+0000 7f600f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf06f20 0x55ec0bf07300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47988/0 (socket says 172.21.15.103:47988) 2024-09-19T06:57:29.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.373+0000 7f600f7fe640 1 -- 172.21.15.103:0/2270068272 learned_addr learned my addr 172.21.15.103:0/2270068272 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:29.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.374+0000 7f6014af2640 1 --2- 172.21.15.103:0/3596683476 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5ff80a33b0 0x7f5ff80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.374+0000 7f600f7fe640 1 --2- 172.21.15.103:0/2270068272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf16650 0x55ec0bf18a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.374+0000 7f600ffff640 1 --2- 172.21.15.103:0/2270068272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf07840 0x55ec0bf16110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.374+0000 7f600d7fa640 1 -- 172.21.15.103:0/3596683476 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f60080cfb20 con 0x7f6010105e40 2024-09-19T06:57:29.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.374+0000 7f600f7fe640 1 -- 172.21.15.103:0/2270068272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf16650 msgr2=0x55ec0bf18a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:29.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.374+0000 7f600f7fe640 1 --2- 172.21.15.103:0/2270068272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf16650 0x55ec0bf18a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.374+0000 7f600f7fe640 1 -- 172.21.15.103:0/2270068272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf07840 msgr2=0x55ec0bf16110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:29.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.374+0000 7f600f7fe640 1 --2- 172.21.15.103:0/2270068272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf07840 0x55ec0bf16110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.374+0000 7f600f7fe640 1 -- 172.21.15.103:0/2270068272 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec0be04d40 con 0x55ec0bf06f20 2024-09-19T06:57:29.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.374+0000 7f600f7fe640 1 --2- 172.21.15.103:0/2270068272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf06f20 0x55ec0bf07300 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f6004015520 tx=0x7f60040095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847091bee320eff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:29.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.374+0000 7f5ff67fc640 1 -- 172.21.15.103:0/2270068272 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60040150f0 con 0x55ec0bf06f20 2024-09-19T06:57:29.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.374+0000 7f6014af2640 1 --2- 172.21.15.103:0/3596683476 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5ff80a33b0 0x7f5ff80a5870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f600809bae0 tx=0x7f6008066bc0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:29.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.374+0000 7f5ff67fc640 1 -- 172.21.15.103:0/2270068272 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6004005020 con 0x55ec0bf06f20 2024-09-19T06:57:29.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.374+0000 7f5ff67fc640 1 -- 172.21.15.103:0/2270068272 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60040049a0 con 0x55ec0bf06f20 2024-09-19T06:57:29.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.375+0000 7f601892e740 1 -- 172.21.15.103:0/2270068272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf06f20 msgr2=0x55ec0bf07300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:29.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.375+0000 7f601892e740 1 --2- 172.21.15.103:0/2270068272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf06f20 0x55ec0bf07300 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f6004015520 tx=0x7f60040095d0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.375+0000 7f601892e740 1 -- 172.21.15.103:0/2270068272 shutdown_connections 2024-09-19T06:57:29.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.375+0000 7f601892e740 1 --2- 172.21.15.103:0/2270068272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf16650 0x55ec0bf18a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.375+0000 7f601892e740 1 --2- 172.21.15.103:0/2270068272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf07840 0x55ec0bf16110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.375+0000 7f601892e740 1 --2- 172.21.15.103:0/2270068272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf06f20 0x55ec0bf07300 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.375+0000 7f601892e740 1 -- 172.21.15.103:0/2270068272 >> 172.21.15.103:0/2270068272 conn(0x55ec0bf035f0 msgr2=0x55ec0bf039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:29.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.375+0000 7f601892e740 1 -- 172.21.15.103:0/2270068272 shutdown_connections 2024-09-19T06:57:29.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.375+0000 7f601892e740 1 -- 172.21.15.103:0/2270068272 wait complete. 2024-09-19T06:57:29.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.375+0000 7f601892e740 1 -- start start 2024-09-19T06:57:29.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.375+0000 7f601892e740 1 Processor -- start 2024-09-19T06:57:29.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f601892e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf06f20 0x55ec0bf13560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f601892e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf07840 0x55ec0bf13aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f601892e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf16650 0x55ec0bf13fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f601892e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ec0bf19220 con 0x55ec0bf07840 2024-09-19T06:57:29.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f601892e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ec0bdf4f10 con 0x55ec0bf16650 2024-09-19T06:57:29.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f601892e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ec0bf193a0 con 0x55ec0bf06f20 2024-09-19T06:57:29.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f601892e740 10 client.0 _ll_register_callbacks cb 0x55ec0be18db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:29.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f600ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf07840 0x55ec0bf13aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f600ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf07840 0x55ec0bf13aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48000/0 (socket says 172.21.15.103:48000) 2024-09-19T06:57:29.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f600ffff640 1 -- 172.21.15.103:0/1723283391 learned_addr learned my addr 172.21.15.103:0/1723283391 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:29.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f600f7fe640 1 --2- 172.21.15.103:0/1723283391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf06f20 0x55ec0bf13560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f600f7fe640 1 --2- 172.21.15.103:0/1723283391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf16650 0x55ec0bf13fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f600ffff640 1 -- 172.21.15.103:0/1723283391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf06f20 msgr2=0x55ec0bf13560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:29.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f600ffff640 1 --2- 172.21.15.103:0/1723283391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf06f20 0x55ec0bf13560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f600ffff640 1 -- 172.21.15.103:0/1723283391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf16650 msgr2=0x55ec0bf13fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:29.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f600ffff640 1 --2- 172.21.15.103:0/1723283391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf16650 0x55ec0bf13fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f600ffff640 1 -- 172.21.15.103:0/1723283391 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec0bb7b580 con 0x55ec0bf07840 2024-09-19T06:57:29.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f600ffff640 1 --2- 172.21.15.103:0/1723283391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf07840 0x55ec0bf13aa0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f6000033000 tx=0x7f6000031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31dc084681d72e52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:29.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f5fef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:29.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f5fef7fe640 1 -- 172.21.15.103:0/1723283391 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600003f070 con 0x55ec0bf07840 2024-09-19T06:57:29.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f601892e740 1 -- 172.21.15.103:0/1723283391 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6004005020 con 0x55ec0bf07840 2024-09-19T06:57:29.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.376+0000 7f5fef7fe640 1 -- 172.21.15.103:0/1723283391 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f6000004450 con 0x55ec0bf07840 2024-09-19T06:57:29.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.377+0000 7f5fef7fe640 1 -- 172.21.15.103:0/1723283391 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6000035070 con 0x55ec0bf07840 2024-09-19T06:57:29.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.377+0000 7f601892e740 10 client.18300 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:29.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.377+0000 7f601892e740 1 -- 172.21.15.103:0/1723283391 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ec0bc45960 con 0x55ec0bf07840 2024-09-19T06:57:29.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.377+0000 7f5fef7fe640 1 -- 172.21.15.103:0/1723283391 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6004005020 con 0x55ec0bf07840 2024-09-19T06:57:29.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.377+0000 7f5fef7fe640 1 -- 172.21.15.103:0/1723283391 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6000004ad0 con 0x55ec0bf07840 2024-09-19T06:57:29.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.377+0000 7f601892e740 10 client.18300 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:29.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.377+0000 7f601892e740 10 client.18300 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:57:29.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.377+0000 7f601892e740 20 client.18300 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:29.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.377+0000 7f601892e740 1 --2- 172.21.15.103:0/1723283391 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ec0bf26250 0x55ec0bf28670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.377+0000 7f601892e740 4 client.18300 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:29.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.377+0000 7f601892e740 1 -- 172.21.15.103:0/1723283391 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec0be79b90 con 0x55ec0bf26250 2024-09-19T06:57:29.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.377+0000 7f600f7fe640 1 --2- 172.21.15.103:0/1723283391 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ec0bf26250 0x55ec0bf28670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.378+0000 7f5fef7fe640 10 client.18300 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:29.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.378+0000 7f600f7fe640 1 --2- 172.21.15.103:0/1723283391 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ec0bf26250 0x55ec0bf28670 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e0cf0e96f05d288 server_cookie=5d81335da4a359af in_seq=0 out_seq=0 2024-09-19T06:57:29.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.378+0000 7f5fef7fe640 1 -- 172.21.15.103:0/1723283391 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec0be79b90 con 0x55ec0bf26250 2024-09-19T06:57:29.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.378+0000 7f5fef7fe640 10 client.18300 handle_command_reply: tid=0 2024-09-19T06:57:29.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.378+0000 7f601892e740 1 client.18300 shutdown 2024-09-19T06:57:29.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.379+0000 7f601892e740 1 -- 172.21.15.103:0/1723283391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf07840 msgr2=0x55ec0bf13aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:29.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.379+0000 7f601892e740 1 --2- 172.21.15.103:0/1723283391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf07840 0x55ec0bf13aa0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f6000033000 tx=0x7f6000031860 comp rx=0 tx=0).stop 2024-09-19T06:57:29.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.379+0000 7f601892e740 1 -- 172.21.15.103:0/1723283391 shutdown_connections 2024-09-19T06:57:29.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.379+0000 7f601892e740 1 --2- 172.21.15.103:0/1723283391 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ec0bf26250 0x55ec0bf28670 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.379+0000 7f601892e740 1 --2- 172.21.15.103:0/1723283391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf16650 0x55ec0bf13fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.379+0000 7f601892e740 1 --2- 172.21.15.103:0/1723283391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf07840 0x55ec0bf13aa0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.379+0000 7f5fef7fe640 0 client.18300 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:29.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.379+0000 7f601892e740 1 --2- 172.21.15.103:0/1723283391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf06f20 0x55ec0bf13560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.379+0000 7f601892e740 1 -- 172.21.15.103:0/1723283391 >> 172.21.15.103:0/1723283391 conn(0x55ec0bf035f0 msgr2=0x55ec0bf02750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:29.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.379+0000 7f601892e740 1 -- 172.21.15.103:0/1723283391 shutdown_connections 2024-09-19T06:57:29.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.379+0000 7f601892e740 1 -- 172.21.15.103:0/1723283391 wait complete. 2024-09-19T06:57:29.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.379+0000 7f601892e740 20 client.18300 trim_cache size 0 max 16384 2024-09-19T06:57:29.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f601892e740 1 Processor -- start 2024-09-19T06:57:29.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f601892e740 1 -- start start 2024-09-19T06:57:29.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f601892e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf06f20 0x55ec0bf02640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f601892e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf07840 0x55ec0bf2df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f601892e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf16650 0x55ec0bf2e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f601892e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f60101ab6f0 con 0x55ec0bf07840 2024-09-19T06:57:29.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f601892e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60101ad230 con 0x55ec0bf06f20 2024-09-19T06:57:29.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f601892e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60101abdc0 con 0x55ec0bf16650 2024-09-19T06:57:29.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f600ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf07840 0x55ec0bf2df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f600ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf07840 0x55ec0bf2df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48014/0 (socket says 172.21.15.103:48014) 2024-09-19T06:57:29.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f600ffff640 1 -- 172.21.15.103:0/3664551792 learned_addr learned my addr 172.21.15.103:0/3664551792 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:29.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f600f7fe640 1 --2- 172.21.15.103:0/3664551792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf06f20 0x55ec0bf02640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f600f7fe640 1 --2- 172.21.15.103:0/3664551792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf16650 0x55ec0bf2e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f600ffff640 1 -- 172.21.15.103:0/3664551792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf16650 msgr2=0x55ec0bf2e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:29.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f600ffff640 1 --2- 172.21.15.103:0/3664551792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf16650 0x55ec0bf2e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f600ffff640 1 -- 172.21.15.103:0/3664551792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf06f20 msgr2=0x55ec0bf02640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:29.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f600ffff640 1 --2- 172.21.15.103:0/3664551792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf06f20 0x55ec0bf02640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f600ffff640 1 -- 172.21.15.103:0/3664551792 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec0bf07670 con 0x55ec0bf07840 2024-09-19T06:57:29.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.396+0000 7f600f7fe640 1 --2- 172.21.15.103:0/3664551792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf16650 0x55ec0bf2e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:29.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.397+0000 7f600ffff640 1 --2- 172.21.15.103:0/3664551792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf07840 0x55ec0bf2df20 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f6000003690 tx=0x7f60000314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d37c12df9f8c0eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:29.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.397+0000 7f5ff5ffb640 1 -- 172.21.15.103:0/3664551792 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000050040 con 0x55ec0bf07840 2024-09-19T06:57:29.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.397+0000 7f5ff5ffb640 1 -- 172.21.15.103:0/3664551792 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6000004ba0 con 0x55ec0bf07840 2024-09-19T06:57:29.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.397+0000 7f5ff5ffb640 1 -- 172.21.15.103:0/3664551792 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600003f040 con 0x55ec0bf07840 2024-09-19T06:57:29.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.397+0000 7f601892e740 1 -- 172.21.15.103:0/3664551792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf07840 msgr2=0x55ec0bf2df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:29.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.397+0000 7f601892e740 1 --2- 172.21.15.103:0/3664551792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf07840 0x55ec0bf2df20 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f6000003690 tx=0x7f60000314a0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.397+0000 7f601892e740 1 -- 172.21.15.103:0/3664551792 shutdown_connections 2024-09-19T06:57:29.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.397+0000 7f601892e740 1 --2- 172.21.15.103:0/3664551792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf16650 0x55ec0bf2e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.397+0000 7f601892e740 1 --2- 172.21.15.103:0/3664551792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf07840 0x55ec0bf2df20 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.397+0000 7f601892e740 1 --2- 172.21.15.103:0/3664551792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf06f20 0x55ec0bf02640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.398+0000 7f601892e740 1 -- 172.21.15.103:0/3664551792 >> 172.21.15.103:0/3664551792 conn(0x55ec0bf035f0 msgr2=0x55ec0bf01f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:29.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.398+0000 7f601892e740 1 -- 172.21.15.103:0/3664551792 shutdown_connections 2024-09-19T06:57:29.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.398+0000 7f601892e740 1 -- 172.21.15.103:0/3664551792 wait complete. 2024-09-19T06:57:29.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.398+0000 7f601892e740 1 -- start start 2024-09-19T06:57:29.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.398+0000 7f601892e740 1 Processor -- start 2024-09-19T06:57:29.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.398+0000 7f601892e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf06f20 0x55ec0bf1ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.398+0000 7f601892e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf07840 0x55ec0bf1ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.398+0000 7f601892e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf16650 0x55ec0bf1b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.398+0000 7f601892e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f60101acb60 con 0x55ec0bf06f20 2024-09-19T06:57:29.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.398+0000 7f601892e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60101ac490 con 0x55ec0bf07840 2024-09-19T06:57:29.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.398+0000 7f601892e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60101a96f0 con 0x55ec0bf16650 2024-09-19T06:57:29.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.398+0000 7f601892e740 10 client.0 _ll_register_callbacks cb 0x7f6000009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:29.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f600f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf06f20 0x55ec0bf1ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f600f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf06f20 0x55ec0bf1ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48018/0 (socket says 172.21.15.103:48018) 2024-09-19T06:57:29.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f600f7fe640 1 -- 172.21.15.103:0/2588381651 learned_addr learned my addr 172.21.15.103:0/2588381651 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:29.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f600ffff640 1 --2- 172.21.15.103:0/2588381651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf07840 0x55ec0bf1ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f600f7fe640 1 -- 172.21.15.103:0/2588381651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf16650 msgr2=0x55ec0bf1b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:29.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f600f7fe640 1 --2- 172.21.15.103:0/2588381651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf16650 0x55ec0bf1b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f600f7fe640 1 -- 172.21.15.103:0/2588381651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf07840 msgr2=0x55ec0bf1ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:29.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f600f7fe640 1 --2- 172.21.15.103:0/2588381651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf07840 0x55ec0bf1ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f600f7fe640 1 -- 172.21.15.103:0/2588381651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec0bf1b8c0 con 0x55ec0bf06f20 2024-09-19T06:57:29.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f600ffff640 1 --2- 172.21.15.103:0/2588381651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf07840 0x55ec0bf1ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:29.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f600f7fe640 1 --2- 172.21.15.103:0/2588381651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf06f20 0x55ec0bf1ea60 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f6004015e70 tx=0x7f6004009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d8343429381740 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:29.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f5fef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:29.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f5fef7fe640 1 -- 172.21.15.103:0/2588381651 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6004005250 con 0x55ec0bf06f20 2024-09-19T06:57:29.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f601892e740 1 -- 172.21.15.103:0/2588381651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6000004ba0 con 0x55ec0bf06f20 2024-09-19T06:57:29.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f5fef7fe640 1 -- 172.21.15.103:0/2588381651 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f60040053f0 con 0x55ec0bf06f20 2024-09-19T06:57:29.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f5fef7fe640 1 -- 172.21.15.103:0/2588381651 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f60040055f0 con 0x55ec0bf06f20 2024-09-19T06:57:29.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f601892e740 10 client.18306 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:29.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.399+0000 7f601892e740 1 -- 172.21.15.103:0/2588381651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ec0bf91b60 con 0x55ec0bf06f20 2024-09-19T06:57:29.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.400+0000 7f5fef7fe640 1 -- 172.21.15.103:0/2588381651 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6000004ba0 con 0x55ec0bf06f20 2024-09-19T06:57:29.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.400+0000 7f5fef7fe640 1 -- 172.21.15.103:0/2588381651 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6004031020 con 0x55ec0bf06f20 2024-09-19T06:57:29.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.400+0000 7f601892e740 10 client.18306 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:29.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.400+0000 7f601892e740 10 client.18306 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:57:29.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.400+0000 7f601892e740 20 client.18306 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:29.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.400+0000 7f601892e740 1 --2- 172.21.15.103:0/2588381651 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ec0bf92170 0x55ec0bf945b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:29.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.400+0000 7f601892e740 4 client.18306 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2c1f8d0d", "release": true, "await": true}] 2024-09-19T06:57:29.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.400+0000 7f601892e740 1 -- 172.21.15.103:0/2588381651 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2c1f8d0d", "release": true, "await": true}) -- 0x55ec0be740a0 con 0x55ec0bf92170 2024-09-19T06:57:29.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.400+0000 7f600ffff640 1 --2- 172.21.15.103:0/2588381651 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ec0bf92170 0x55ec0bf945b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:29.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.400+0000 7f5fef7fe640 10 client.18306 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:29.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.400+0000 7f600ffff640 1 --2- 172.21.15.103:0/2588381651 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ec0bf92170 0x55ec0bf945b0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=185f3c774778fbb2 server_cookie=11501fe3817af5ac in_seq=0 out_seq=0 2024-09-19T06:57:29.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.403+0000 7f5fef7fe640 1 -- 172.21.15.103:0/2588381651 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ec0be740a0 con 0x55ec0bf92170 2024-09-19T06:57:29.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.403+0000 7f5fef7fe640 10 client.18306 handle_command_reply: tid=0 2024-09-19T06:57:29.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.403+0000 7f601892e740 1 client.18306 shutdown 2024-09-19T06:57:29.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.403+0000 7f601892e740 1 -- 172.21.15.103:0/2588381651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf06f20 msgr2=0x55ec0bf1ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:29.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.403+0000 7f601892e740 1 --2- 172.21.15.103:0/2588381651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf06f20 0x55ec0bf1ea60 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f6004015e70 tx=0x7f6004009360 comp rx=0 tx=0).stop 2024-09-19T06:57:29.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.403+0000 7f601892e740 1 -- 172.21.15.103:0/2588381651 shutdown_connections 2024-09-19T06:57:29.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.403+0000 7f601892e740 1 --2- 172.21.15.103:0/2588381651 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ec0bf92170 0x55ec0bf945b0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.403+0000 7f601892e740 1 --2- 172.21.15.103:0/2588381651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec0bf16650 0x55ec0bf1b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.403+0000 7f601892e740 1 --2- 172.21.15.103:0/2588381651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec0bf07840 0x55ec0bf1ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.403+0000 7f601892e740 1 --2- 172.21.15.103:0/2588381651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec0bf06f20 0x55ec0bf1ea60 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.403+0000 7f601892e740 1 -- 172.21.15.103:0/2588381651 >> 172.21.15.103:0/2588381651 conn(0x55ec0bf035f0 msgr2=0x55ec0bf02750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:29.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.403+0000 7f5fef7fe640 0 client.18306 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:29.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.403+0000 7f601892e740 1 -- 172.21.15.103:0/2588381651 shutdown_connections 2024-09-19T06:57:29.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.403+0000 7f601892e740 1 -- 172.21.15.103:0/2588381651 wait complete. 2024-09-19T06:57:29.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.403+0000 7f601892e740 20 client.18306 trim_cache size 0 max 16384 2024-09-19T06:57:29.412 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:57:29.412 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:57:29.413 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:57:29.413 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 32, 2024-09-19T06:57:29.413 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:57:29.413 INFO:teuthology.orchestra.run.smithi103.stdout: "2c1f8d0d": { 2024-09-19T06:57:29.413 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 32, 2024-09-19T06:57:29.413 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:57:29.413 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:57:29.413 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:57:29.413 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:57:29.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:57:29.413 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:57:29.413 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 145.8, 2024-09-19T06:57:29.413 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:57:29.414 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:57:29.414 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:57:29.414 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:57:29.414 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:57:29.414 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:57:29.414 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:29.414 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:29.414 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:29.414 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:29.414 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:29.414 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:57:29.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.404+0000 7f6016555640 1 -- 172.21.15.103:0/3596683476 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5ff80a33b0 msgr2=0x7f5ff80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:29.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.404+0000 7f6016555640 1 --2- 172.21.15.103:0/3596683476 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5ff80a33b0 0x7f5ff80a5870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f600809bae0 tx=0x7f6008066bc0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.404+0000 7f6016555640 1 -- 172.21.15.103:0/3596683476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6010105e40 msgr2=0x7f60101a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:29.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.404+0000 7f6016555640 1 --2- 172.21.15.103:0/3596683476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6010105e40 0x7f60101a4350 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f6008009fd0 tx=0x7f600800b260 comp rx=0 tx=0).stop 2024-09-19T06:57:29.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.404+0000 7f6016555640 1 -- 172.21.15.103:0/3596683476 shutdown_connections 2024-09-19T06:57:29.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.404+0000 7f6016555640 1 --2- 172.21.15.103:0/3596683476 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5ff80a33b0 0x7f5ff80a5870 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.404+0000 7f6016555640 1 --2- 172.21.15.103:0/3596683476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6010105e40 0x7f60101a4350 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.404+0000 7f6016555640 1 --2- 172.21.15.103:0/3596683476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6010101fc0 0x7f6010199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.404+0000 7f6016555640 1 --2- 172.21.15.103:0/3596683476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6010101610 0x7f60101992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:29.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.404+0000 7f6016555640 1 -- 172.21.15.103:0/3596683476 >> 172.21.15.103:0/3596683476 conn(0x7f60100fc5c0 msgr2=0x7f60100fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:29.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.404+0000 7f6016555640 1 -- 172.21.15.103:0/3596683476 shutdown_connections 2024-09-19T06:57:29.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:29.404+0000 7f6016555640 1 -- 172.21.15.103:0/3596683476 wait complete. 2024-09-19T06:57:30.057 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2c1f8d0d', total seconds elapsed: 28.9 2024-09-19T06:57:30.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:30 smithi103 ceph-mon[31158]: pgmap v255: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 14 op/s 2024-09-19T06:57:30.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:30 smithi134 ceph-mon[35356]: pgmap v255: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 14 op/s 2024-09-19T06:57:30.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:30 smithi174 ceph-mon[35637]: pgmap v255: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 14 op/s 2024-09-19T06:57:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.421+0000 7fb6f7e77640 1 -- 172.21.15.103:0/3239147349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6f0106a00 msgr2=0x7fb6f0106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.421+0000 7fb6f7e77640 1 --2- 172.21.15.103:0/3239147349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6f0106a00 0x7fb6f0106de0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009cd0 tx=0x7fb6ec02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.421+0000 7fb6f7e77640 1 -- 172.21.15.103:0/3239147349 shutdown_connections 2024-09-19T06:57:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.422+0000 7fb6f7e77640 1 --2- 172.21.15.103:0/3239147349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6f0101390 0x7fb6f010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.422+0000 7fb6f7e77640 1 --2- 172.21.15.103:0/3239147349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f01009f0 0x7fb6f0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.422+0000 7fb6f7e77640 1 --2- 172.21.15.103:0/3239147349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6f0106a00 0x7fb6f0106de0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.422+0000 7fb6f7e77640 1 -- 172.21.15.103:0/3239147349 >> 172.21.15.103:0/3239147349 conn(0x7fb6f00fc6b0 msgr2=0x7fb6f00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.422+0000 7fb6f7e77640 1 -- 172.21.15.103:0/3239147349 shutdown_connections 2024-09-19T06:57:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.422+0000 7fb6f7e77640 1 -- 172.21.15.103:0/3239147349 wait complete. 2024-09-19T06:57:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.422+0000 7fb6f7e77640 1 Processor -- start 2024-09-19T06:57:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.422+0000 7fb6f7e77640 1 -- start start 2024-09-19T06:57:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.422+0000 7fb6f7e77640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f01009f0 0x7fb6f0196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.422+0000 7fb6f7e77640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6f0101390 0x7fb6f01966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.422+0000 7fb6f7e77640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6f0106a00 0x7fb6f019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.422+0000 7fb6f7e77640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f010fd40 con 0x7fb6f0106a00 2024-09-19T06:57:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.422+0000 7fb6f7e77640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f010fbc0 con 0x7fb6f01009f0 2024-09-19T06:57:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.422+0000 7fb6f7e77640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f010fec0 con 0x7fb6f0101390 2024-09-19T06:57:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.423+0000 7fb6f6414640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6f0106a00 0x7fb6f019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.423+0000 7fb6f6414640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6f0106a00 0x7fb6f019aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48030/0 (socket says 172.21.15.103:48030) 2024-09-19T06:57:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.423+0000 7fb6f6414640 1 -- 172.21.15.103:0/453266959 learned_addr learned my addr 172.21.15.103:0/453266959 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.423+0000 7fb6f5c13640 1 --2- 172.21.15.103:0/453266959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f01009f0 0x7fb6f0196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.423+0000 7fb6f5412640 1 --2- 172.21.15.103:0/453266959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6f0101390 0x7fb6f01966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.423+0000 7fb6f6414640 1 -- 172.21.15.103:0/453266959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6f0101390 msgr2=0x7fb6f01966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.423+0000 7fb6f6414640 1 --2- 172.21.15.103:0/453266959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6f0101390 0x7fb6f01966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.423+0000 7fb6f6414640 1 -- 172.21.15.103:0/453266959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f01009f0 msgr2=0x7fb6f0196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.423+0000 7fb6f6414640 1 --2- 172.21.15.103:0/453266959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f01009f0 0x7fb6f0196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.423+0000 7fb6f6414640 1 -- 172.21.15.103:0/453266959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f019b160 con 0x7fb6f0106a00 2024-09-19T06:57:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.423+0000 7fb6f6414640 1 --2- 172.21.15.103:0/453266959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6f0106a00 0x7fb6f019aa30 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800e970 tx=0x7fb6e800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.423+0000 7fb6e6ffd640 1 -- 172.21.15.103:0/453266959 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800ccc0 con 0x7fb6f0106a00 2024-09-19T06:57:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.423+0000 7fb6e6ffd640 1 -- 172.21.15.103:0/453266959 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e800ce60 con 0x7fb6f0106a00 2024-09-19T06:57:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.423+0000 7fb6f7e77640 1 -- 172.21.15.103:0/453266959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f019b450 con 0x7fb6f0106a00 2024-09-19T06:57:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.423+0000 7fb6e6ffd640 1 -- 172.21.15.103:0/453266959 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8010490 con 0x7fb6f0106a00 2024-09-19T06:57:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.423+0000 7fb6f7e77640 1 -- 172.21.15.103:0/453266959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f01a2ce0 con 0x7fb6f0106a00 2024-09-19T06:57:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.426+0000 7fb6e6ffd640 1 -- 172.21.15.103:0/453266959 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fb6e8005020 con 0x7fb6f0106a00 2024-09-19T06:57:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.426+0000 7fb6fac11740 1 Processor -- start 2024-09-19T06:57:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.426+0000 7fb6fac11740 1 -- start start 2024-09-19T06:57:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.426+0000 7fb6e6ffd640 1 --2- 172.21.15.103:0/453266959 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb6dc0a7910 0x7fb6dc0a9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.426+0000 7fb6fac11740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffccaf0 0x55835ffcced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6fac11740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffcd410 0x55835ffd7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6fac11740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffd7cb0 0x55835ffda0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6fac11740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55835ff312f0 con 0x55835ffd7cb0 2024-09-19T06:57:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6fac11740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55835fed9270 con 0x55835ffccaf0 2024-09-19T06:57:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6fac11740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55835ff2d2b0 con 0x55835ffcd410 2024-09-19T06:57:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6f6414640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffd7cb0 0x55835ffda0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6f5412640 1 --2- 172.21.15.103:0/453266959 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb6dc0a7910 0x7fb6dc0a9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6f6414640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffd7cb0 0x55835ffda0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48044/0 (socket says 172.21.15.103:48044) 2024-09-19T06:57:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6f6414640 1 -- 172.21.15.103:0/406673452 learned_addr learned my addr 172.21.15.103:0/406673452 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6e6ffd640 1 -- 172.21.15.103:0/453266959 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7fb6e80d0020 con 0x7fb6f0106a00 2024-09-19T06:57:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6f5c13640 1 --2- 172.21.15.103:0/406673452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffccaf0 0x55835ffcced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6f5412640 1 --2- 172.21.15.103:0/406673452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffcd410 0x55835ffd7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6f6414640 1 -- 172.21.15.103:0/406673452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffcd410 msgr2=0x55835ffd7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6f6414640 1 --2- 172.21.15.103:0/406673452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffcd410 0x55835ffd7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6f6414640 1 -- 172.21.15.103:0/406673452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffccaf0 msgr2=0x55835ffcced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6f6414640 1 --2- 172.21.15.103:0/406673452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffccaf0 0x55835ffcced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6f6414640 1 -- 172.21.15.103:0/406673452 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55835fed8340 con 0x55835ffd7cb0 2024-09-19T06:57:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6f5c13640 1 --2- 172.21.15.103:0/406673452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffccaf0 0x55835ffcced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:57:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6f5412640 1 --2- 172.21.15.103:0/453266959 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb6dc0a7910 0x7fb6dc0a9dd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00fec20 tx=0x7fb6e0009b60 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6f6414640 1 --2- 172.21.15.103:0/406673452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffd7cb0 0x55835ffda0a0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80cd7b0 tx=0x7fb6e809be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2518e9e1f8e2eaae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6dbfff640 1 -- 172.21.15.103:0/406673452 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e809c9e0 con 0x55835ffd7cb0 2024-09-19T06:57:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6dbfff640 1 -- 172.21.15.103:0/406673452 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e809c2e0 con 0x55835ffd7cb0 2024-09-19T06:57:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.427+0000 7fb6dbfff640 1 -- 172.21.15.103:0/406673452 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8098950 con 0x55835ffd7cb0 2024-09-19T06:57:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.428+0000 7fb6fac11740 1 -- 172.21.15.103:0/406673452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffd7cb0 msgr2=0x55835ffda0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.428+0000 7fb6fac11740 1 --2- 172.21.15.103:0/406673452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffd7cb0 0x55835ffda0a0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80cd7b0 tx=0x7fb6e809be20 comp rx=0 tx=0).stop 2024-09-19T06:57:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.428+0000 7fb6fac11740 1 -- 172.21.15.103:0/406673452 shutdown_connections 2024-09-19T06:57:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.428+0000 7fb6fac11740 1 --2- 172.21.15.103:0/406673452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffd7cb0 0x55835ffda0a0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.428+0000 7fb6fac11740 1 --2- 172.21.15.103:0/406673452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffcd410 0x55835ffd7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.428+0000 7fb6fac11740 1 --2- 172.21.15.103:0/406673452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffccaf0 0x55835ffcced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.428+0000 7fb6fac11740 1 -- 172.21.15.103:0/406673452 >> 172.21.15.103:0/406673452 conn(0x55835ffc9060 msgr2=0x55835ffc9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.428+0000 7fb6fac11740 1 -- 172.21.15.103:0/406673452 shutdown_connections 2024-09-19T06:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.428+0000 7fb6fac11740 1 -- 172.21.15.103:0/406673452 wait complete. 2024-09-19T06:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.429+0000 7fb6fac11740 1 -- start start 2024-09-19T06:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.429+0000 7fb6fac11740 1 Processor -- start 2024-09-19T06:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.429+0000 7fb6fac11740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffccaf0 0x55835ffd4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.429+0000 7fb6fac11740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffcd410 0x55835ffd5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.429+0000 7fb6fac11740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffd7cb0 0x55835ffd5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.429+0000 7fb6fac11740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55835ffda880 con 0x55835ffcd410 2024-09-19T06:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.429+0000 7fb6fac11740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55835feba290 con 0x55835ffccaf0 2024-09-19T06:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.429+0000 7fb6fac11740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55835ffdaa00 con 0x55835ffd7cb0 2024-09-19T06:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.429+0000 7fb6fac11740 10 client.0 _ll_register_callbacks cb 0x55835feeb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.429+0000 7fb6f5412640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffcd410 0x55835ffd5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.429+0000 7fb6f5412640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffcd410 0x55835ffd5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48050/0 (socket says 172.21.15.103:48050) 2024-09-19T06:57:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.429+0000 7fb6f5412640 1 -- 172.21.15.103:0/1764243175 learned_addr learned my addr 172.21.15.103:0/1764243175 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.429+0000 7fb6f6414640 1 --2- 172.21.15.103:0/1764243175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffd7cb0 0x55835ffd5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.430+0000 7fb6f5c13640 1 --2- 172.21.15.103:0/1764243175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffccaf0 0x55835ffd4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.430+0000 7fb6f5412640 1 -- 172.21.15.103:0/1764243175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffd7cb0 msgr2=0x55835ffd5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.430+0000 7fb6f5412640 1 --2- 172.21.15.103:0/1764243175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffd7cb0 0x55835ffd5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.430+0000 7fb6f5412640 1 -- 172.21.15.103:0/1764243175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffccaf0 msgr2=0x55835ffd4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.430+0000 7fb6f5412640 1 --2- 172.21.15.103:0/1764243175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffccaf0 0x55835ffd4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.430+0000 7fb6f5412640 1 -- 172.21.15.103:0/1764243175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55835fc44210 con 0x55835ffcd410 2024-09-19T06:57:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.430+0000 7fb6f6414640 1 --2- 172.21.15.103:0/1764243175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffd7cb0 0x55835ffd5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.430+0000 7fb6f5412640 1 --2- 172.21.15.103:0/1764243175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffcd410 0x55835ffd5110 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009e30 tx=0x7fb6e001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d5e8f7120e0f2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.430+0000 7fb6d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.430+0000 7fb6d97fa640 1 -- 172.21.15.103:0/1764243175 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0005020 con 0x55835ffcd410 2024-09-19T06:57:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.430+0000 7fb6d97fa640 1 -- 172.21.15.103:0/1764243175 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7fb6e00051c0 con 0x55835ffcd410 2024-09-19T06:57:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.430+0000 7fb6fac11740 1 -- 172.21.15.103:0/1764243175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6e809c2e0 con 0x55835ffcd410 2024-09-19T06:57:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.430+0000 7fb6d97fa640 1 -- 172.21.15.103:0/1764243175 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb6e00053c0 con 0x55835ffcd410 2024-09-19T06:57:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.430+0000 7fb6fac11740 10 client.18318 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.430+0000 7fb6fac11740 1 -- 172.21.15.103:0/1764243175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55835fd10620 con 0x55835ffcd410 2024-09-19T06:57:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.430+0000 7fb6d97fa640 1 -- 172.21.15.103:0/1764243175 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e809c2e0 con 0x55835ffcd410 2024-09-19T06:57:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.431+0000 7fb6d97fa640 1 -- 172.21.15.103:0/1764243175 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb6e0004b00 con 0x55835ffcd410 2024-09-19T06:57:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.431+0000 7fb6fac11740 10 client.18318 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.431+0000 7fb6fac11740 10 client.18318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.431+0000 7fb6fac11740 20 client.18318 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.431+0000 7fb6fac11740 1 --2- 172.21.15.103:0/1764243175 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55835ffe7c20 0x55835ffea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.431+0000 7fb6fac11740 4 client.18318 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.431+0000 7fb6fac11740 1 -- 172.21.15.103:0/1764243175 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55835ff3ea20 con 0x55835ffe7c20 2024-09-19T06:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.431+0000 7fb6f5c13640 1 --2- 172.21.15.103:0/1764243175 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55835ffe7c20 0x55835ffea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.431+0000 7fb6d97fa640 10 client.18318 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.431+0000 7fb6f5c13640 1 --2- 172.21.15.103:0/1764243175 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55835ffe7c20 0x55835ffea0a0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db45882bf6eba1e server_cookie=9b692818bbeccd2f in_seq=0 out_seq=0 2024-09-19T06:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.432+0000 7fb6d97fa640 1 -- 172.21.15.103:0/1764243175 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55835ff3ea20 con 0x55835ffe7c20 2024-09-19T06:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.432+0000 7fb6d97fa640 10 client.18318 handle_command_reply: tid=0 2024-09-19T06:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.432+0000 7fb6fac11740 1 client.18318 shutdown 2024-09-19T06:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.432+0000 7fb6fac11740 1 -- 172.21.15.103:0/1764243175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffcd410 msgr2=0x55835ffd5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.432+0000 7fb6fac11740 1 --2- 172.21.15.103:0/1764243175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffcd410 0x55835ffd5110 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009e30 tx=0x7fb6e001d000 comp rx=0 tx=0).stop 2024-09-19T06:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.433+0000 7fb6fac11740 1 -- 172.21.15.103:0/1764243175 shutdown_connections 2024-09-19T06:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.433+0000 7fb6fac11740 1 --2- 172.21.15.103:0/1764243175 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55835ffe7c20 0x55835ffea0a0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.433+0000 7fb6fac11740 1 --2- 172.21.15.103:0/1764243175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffd7cb0 0x55835ffd5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.433+0000 7fb6fac11740 1 --2- 172.21.15.103:0/1764243175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffcd410 0x55835ffd5110 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.433+0000 7fb6d97fa640 0 client.18318 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.433+0000 7fb6fac11740 1 --2- 172.21.15.103:0/1764243175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffccaf0 0x55835ffd4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.433+0000 7fb6fac11740 1 -- 172.21.15.103:0/1764243175 >> 172.21.15.103:0/1764243175 conn(0x55835ffc9060 msgr2=0x55835ffcedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.433+0000 7fb6fac11740 1 -- 172.21.15.103:0/1764243175 shutdown_connections 2024-09-19T06:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.433+0000 7fb6fac11740 1 -- 172.21.15.103:0/1764243175 wait complete. 2024-09-19T06:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.433+0000 7fb6fac11740 20 client.18318 trim_cache size 0 max 16384 2024-09-19T06:57:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.447+0000 7fb6fac11740 1 Processor -- start 2024-09-19T06:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.447+0000 7fb6fac11740 1 -- start start 2024-09-19T06:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.447+0000 7fb6fac11740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffccaf0 0x55835ffd5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.447+0000 7fb6fac11740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffcd410 0x55835ffd6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.447+0000 7fb6fac11740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffd7cb0 0x55835ffef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.447+0000 7fb6fac11740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01ab720 con 0x55835ffccaf0 2024-09-19T06:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.447+0000 7fb6fac11740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01ad260 con 0x55835ffd7cb0 2024-09-19T06:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.447+0000 7fb6fac11740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01abdf0 con 0x55835ffcd410 2024-09-19T06:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.447+0000 7fb6f5c13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffccaf0 0x55835ffd5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.448+0000 7fb6f5c13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffccaf0 0x55835ffd5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48060/0 (socket says 172.21.15.103:48060) 2024-09-19T06:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.448+0000 7fb6f5c13640 1 -- 172.21.15.103:0/2357483668 learned_addr learned my addr 172.21.15.103:0/2357483668 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.448+0000 7fb6f6414640 1 --2- 172.21.15.103:0/2357483668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffd7cb0 0x55835ffef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.448+0000 7fb6f5c13640 1 -- 172.21.15.103:0/2357483668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffcd410 msgr2=0x55835ffd6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.448+0000 7fb6f5c13640 1 --2- 172.21.15.103:0/2357483668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffcd410 0x55835ffd6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.448+0000 7fb6f5c13640 1 -- 172.21.15.103:0/2357483668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffd7cb0 msgr2=0x55835ffef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.448+0000 7fb6f5c13640 1 --2- 172.21.15.103:0/2357483668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffd7cb0 0x55835ffef9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.448+0000 7fb6f5c13640 1 -- 172.21.15.103:0/2357483668 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55835ffc8e90 con 0x55835ffccaf0 2024-09-19T06:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.448+0000 7fb6f6414640 1 --2- 172.21.15.103:0/2357483668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffd7cb0 0x55835ffef9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.448+0000 7fb6f5c13640 1 --2- 172.21.15.103:0/2357483668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffccaf0 0x55835ffd5480 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec02ae00 tx=0x7fb6ec009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ebf66a8b50810e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.448+0000 7fb6db7fe640 1 -- 172.21.15.103:0/2357483668 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0318c0 con 0x55835ffccaf0 2024-09-19T06:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.448+0000 7fb6db7fe640 1 -- 172.21.15.103:0/2357483668 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6ec036040 con 0x55835ffccaf0 2024-09-19T06:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.448+0000 7fb6db7fe640 1 -- 172.21.15.103:0/2357483668 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0045a0 con 0x55835ffccaf0 2024-09-19T06:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.449+0000 7fb6fac11740 1 -- 172.21.15.103:0/2357483668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffccaf0 msgr2=0x55835ffd5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.449+0000 7fb6fac11740 1 --2- 172.21.15.103:0/2357483668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffccaf0 0x55835ffd5480 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec02ae00 tx=0x7fb6ec009360 comp rx=0 tx=0).stop 2024-09-19T06:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.449+0000 7fb6fac11740 1 -- 172.21.15.103:0/2357483668 shutdown_connections 2024-09-19T06:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.449+0000 7fb6fac11740 1 --2- 172.21.15.103:0/2357483668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffd7cb0 0x55835ffef9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.449+0000 7fb6fac11740 1 --2- 172.21.15.103:0/2357483668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffcd410 0x55835ffd6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.449+0000 7fb6fac11740 1 --2- 172.21.15.103:0/2357483668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffccaf0 0x55835ffd5480 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.449+0000 7fb6fac11740 1 -- 172.21.15.103:0/2357483668 >> 172.21.15.103:0/2357483668 conn(0x55835ffc9060 msgr2=0x55835ffd6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.449+0000 7fb6fac11740 1 -- 172.21.15.103:0/2357483668 shutdown_connections 2024-09-19T06:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.449+0000 7fb6fac11740 1 -- 172.21.15.103:0/2357483668 wait complete. 2024-09-19T06:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.449+0000 7fb6fac11740 1 -- start start 2024-09-19T06:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.449+0000 7fb6fac11740 1 Processor -- start 2024-09-19T06:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6fac11740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffccaf0 0x55835ffe0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6fac11740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffcd410 0x55835ffdb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6fac11740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffd7cb0 0x55835ffdb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6fac11740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01acb90 con 0x55835ffcd410 2024-09-19T06:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6fac11740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01ac4c0 con 0x55835ffccaf0 2024-09-19T06:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6fac11740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01a9720 con 0x55835ffd7cb0 2024-09-19T06:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6fac11740 10 client.0 _ll_register_callbacks cb 0x7fb6e0011ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6f5412640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffcd410 0x55835ffdb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6f5412640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffcd410 0x55835ffdb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48066/0 (socket says 172.21.15.103:48066) 2024-09-19T06:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6f5412640 1 -- 172.21.15.103:0/2184974637 learned_addr learned my addr 172.21.15.103:0/2184974637 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6f5c13640 1 --2- 172.21.15.103:0/2184974637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffccaf0 0x55835ffe0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6f6414640 1 --2- 172.21.15.103:0/2184974637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffd7cb0 0x55835ffdb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6f5412640 1 -- 172.21.15.103:0/2184974637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffd7cb0 msgr2=0x55835ffdb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6f5412640 1 --2- 172.21.15.103:0/2184974637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffd7cb0 0x55835ffdb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6f5412640 1 -- 172.21.15.103:0/2184974637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffccaf0 msgr2=0x55835ffe0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6f5412640 1 --2- 172.21.15.103:0/2184974637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffccaf0 0x55835ffe0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6f5412640 1 -- 172.21.15.103:0/2184974637 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55835ffdbf40 con 0x55835ffcd410 2024-09-19T06:57:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6f6414640 1 --2- 172.21.15.103:0/2184974637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffd7cb0 0x55835ffdb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6f5412640 1 --2- 172.21.15.103:0/2184974637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffcd410 0x55835ffdb490 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000f790 tx=0x7fb6e0005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20402f072a9ed167 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6d97fa640 1 -- 172.21.15.103:0/2184974637 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0004400 con 0x55835ffcd410 2024-09-19T06:57:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6d97fa640 1 -- 172.21.15.103:0/2184974637 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7fb6e0005560 con 0x55835ffcd410 2024-09-19T06:57:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6fac11740 1 -- 172.21.15.103:0/2184974637 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6ec036040 con 0x55835ffcd410 2024-09-19T06:57:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6d97fa640 1 -- 172.21.15.103:0/2184974637 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb6e001fae0 con 0x55835ffcd410 2024-09-19T06:57:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6fac11740 10 client.18324 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.450+0000 7fb6fac11740 1 -- 172.21.15.103:0/2184974637 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558360052660 con 0x55835ffcd410 2024-09-19T06:57:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.451+0000 7fb6d97fa640 1 -- 172.21.15.103:0/2184974637 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6ec036040 con 0x55835ffcd410 2024-09-19T06:57:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.451+0000 7fb6d97fa640 1 -- 172.21.15.103:0/2184974637 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb6e0039030 con 0x55835ffcd410 2024-09-19T06:57:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.451+0000 7fb6fac11740 10 client.18324 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.451+0000 7fb6fac11740 10 client.18324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.451+0000 7fb6fac11740 20 client.18324 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.451+0000 7fb6fac11740 1 --2- 172.21.15.103:0/2184974637 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558360052d80 0x5583600551a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.451+0000 7fb6fac11740 4 client.18324 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.451+0000 7fb6fac11740 1 -- 172.21.15.103:0/2184974637 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55835ff38d70 con 0x558360052d80 2024-09-19T06:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.451+0000 7fb6f5c13640 1 --2- 172.21.15.103:0/2184974637 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558360052d80 0x5583600551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.452+0000 7fb6d97fa640 10 client.18324 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.452+0000 7fb6f5c13640 1 --2- 172.21.15.103:0/2184974637 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558360052d80 0x5583600551a0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc66ed82414d8fbb server_cookie=34e386713791aadc in_seq=0 out_seq=0 2024-09-19T06:57:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.460+0000 7fb6d97fa640 1 -- 172.21.15.103:0/2184974637 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55835ff38d70 con 0x558360052d80 2024-09-19T06:57:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.460+0000 7fb6d97fa640 10 client.18324 handle_command_reply: tid=0 2024-09-19T06:57:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.460+0000 7fb6fac11740 1 client.18324 shutdown 2024-09-19T06:57:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.460+0000 7fb6fac11740 1 -- 172.21.15.103:0/2184974637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffcd410 msgr2=0x55835ffdb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.460+0000 7fb6fac11740 1 --2- 172.21.15.103:0/2184974637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffcd410 0x55835ffdb490 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000f790 tx=0x7fb6e0005280 comp rx=0 tx=0).stop 2024-09-19T06:57:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.460+0000 7fb6fac11740 1 -- 172.21.15.103:0/2184974637 shutdown_connections 2024-09-19T06:57:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.460+0000 7fb6fac11740 1 --2- 172.21.15.103:0/2184974637 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558360052d80 0x5583600551a0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.460+0000 7fb6fac11740 1 --2- 172.21.15.103:0/2184974637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55835ffd7cb0 0x55835ffdb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.460+0000 7fb6fac11740 1 --2- 172.21.15.103:0/2184974637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55835ffcd410 0x55835ffdb490 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.460+0000 7fb6fac11740 1 --2- 172.21.15.103:0/2184974637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55835ffccaf0 0x55835ffe0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.460+0000 7fb6d97fa640 0 client.18324 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.460+0000 7fb6fac11740 1 -- 172.21.15.103:0/2184974637 >> 172.21.15.103:0/2184974637 conn(0x55835ffc9060 msgr2=0x55835ffc79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.460+0000 7fb6fac11740 1 -- 172.21.15.103:0/2184974637 shutdown_connections 2024-09-19T06:57:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.460+0000 7fb6fac11740 1 -- 172.21.15.103:0/2184974637 wait complete. 2024-09-19T06:57:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.460+0000 7fb6fac11740 20 client.18324 trim_cache size 0 max 16384 2024-09-19T06:57:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:57:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:57:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:57:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:57:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.461+0000 7fb6f7e77640 1 -- 172.21.15.103:0/453266959 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb6dc0a7910 msgr2=0x7fb6dc0a9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.461+0000 7fb6f7e77640 1 --2- 172.21.15.103:0/453266959 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb6dc0a7910 0x7fb6dc0a9dd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00fec20 tx=0x7fb6e0009b60 comp rx=0 tx=0).stop 2024-09-19T06:57:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.461+0000 7fb6f7e77640 1 -- 172.21.15.103:0/453266959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6f0106a00 msgr2=0x7fb6f019aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.461+0000 7fb6f7e77640 1 --2- 172.21.15.103:0/453266959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6f0106a00 0x7fb6f019aa30 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800e970 tx=0x7fb6e800ed70 comp rx=0 tx=0).stop 2024-09-19T06:57:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.461+0000 7fb6f7e77640 1 -- 172.21.15.103:0/453266959 shutdown_connections 2024-09-19T06:57:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.461+0000 7fb6f7e77640 1 --2- 172.21.15.103:0/453266959 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb6dc0a7910 0x7fb6dc0a9dd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.461+0000 7fb6f7e77640 1 --2- 172.21.15.103:0/453266959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6f0106a00 0x7fb6f019aa30 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.461+0000 7fb6f7e77640 1 --2- 172.21.15.103:0/453266959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6f0101390 0x7fb6f01966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.461+0000 7fb6f7e77640 1 --2- 172.21.15.103:0/453266959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f01009f0 0x7fb6f0196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.461+0000 7fb6f7e77640 1 -- 172.21.15.103:0/453266959 >> 172.21.15.103:0/453266959 conn(0x7fb6f00fc6b0 msgr2=0x7fb6f00fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.461+0000 7fb6f7e77640 1 -- 172.21.15.103:0/453266959 shutdown_connections 2024-09-19T06:57:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:31.461+0000 7fb6f7e77640 1 -- 172.21.15.103:0/453266959 wait complete. 2024-09-19T06:57:32.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:32 smithi103 ceph-mon[31158]: pgmap v256: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-09-19T06:57:32.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:32 smithi134 ceph-mon[35356]: pgmap v256: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-09-19T06:57:32.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:32 smithi174 ceph-mon[35637]: pgmap v256: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-09-19T06:57:34.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:34 smithi103 ceph-mon[31158]: pgmap v257: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-09-19T06:57:34.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:34 smithi134 ceph-mon[35356]: pgmap v257: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-09-19T06:57:34.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:34 smithi174 ceph-mon[35637]: pgmap v257: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-09-19T06:57:34.638 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:57:34.901 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:35.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.380+0000 7fc3efb2a640 1 -- 172.21.15.103:0/820039901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e8101420 msgr2=0x7fc3e810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:35.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.380+0000 7fc3efb2a640 1 --2- 172.21.15.103:0/820039901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e8101420 0x7fc3e810d950 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009cd0 tx=0x7fc3e002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.380+0000 7fc3efb2a640 1 -- 172.21.15.103:0/820039901 shutdown_connections 2024-09-19T06:57:35.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.380+0000 7fc3efb2a640 1 --2- 172.21.15.103:0/820039901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e8101420 0x7fc3e810d950 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.380+0000 7fc3efb2a640 1 --2- 172.21.15.103:0/820039901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3e8100a80 0x7fc3e8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.380+0000 7fc3efb2a640 1 --2- 172.21.15.103:0/820039901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3e8106a80 0x7fc3e8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.380+0000 7fc3efb2a640 1 -- 172.21.15.103:0/820039901 >> 172.21.15.103:0/820039901 conn(0x7fc3e80fc7a0 msgr2=0x7fc3e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:35.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.380+0000 7fc3efb2a640 1 -- 172.21.15.103:0/820039901 shutdown_connections 2024-09-19T06:57:35.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.380+0000 7fc3efb2a640 1 -- 172.21.15.103:0/820039901 wait complete. 2024-09-19T06:57:35.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3efb2a640 1 Processor -- start 2024-09-19T06:57:35.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3efb2a640 1 -- start start 2024-09-19T06:57:35.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3efb2a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3e8100a80 0x7fc3e8102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3efb2a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3e8101420 0x7fc3e8102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3efb2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e8106a80 0x7fc3e81030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3efb2a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8110e80 con 0x7fc3e8106a80 2024-09-19T06:57:35.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3efb2a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8110d00 con 0x7fc3e8100a80 2024-09-19T06:57:35.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3efb2a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8111000 con 0x7fc3e8101420 2024-09-19T06:57:35.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3ee0c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e8106a80 0x7fc3e81030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3ee0c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e8106a80 0x7fc3e81030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48080/0 (socket says 172.21.15.103:48080) 2024-09-19T06:57:35.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3ee0c7640 1 -- 172.21.15.103:0/2783631889 learned_addr learned my addr 172.21.15.103:0/2783631889 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:35.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3ed0c5640 1 --2- 172.21.15.103:0/2783631889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3e8101420 0x7fc3e8102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3ed8c6640 1 --2- 172.21.15.103:0/2783631889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3e8100a80 0x7fc3e8102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3ee0c7640 1 -- 172.21.15.103:0/2783631889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3e8101420 msgr2=0x7fc3e8102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:35.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3ee0c7640 1 --2- 172.21.15.103:0/2783631889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3e8101420 0x7fc3e8102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3ee0c7640 1 -- 172.21.15.103:0/2783631889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3e8100a80 msgr2=0x7fc3e8102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:35.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3ee0c7640 1 --2- 172.21.15.103:0/2783631889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3e8100a80 0x7fc3e8102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3ee0c7640 1 -- 172.21.15.103:0/2783631889 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e81a68c0 con 0x7fc3e8106a80 2024-09-19T06:57:35.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.381+0000 7fc3ed8c6640 1 --2- 172.21.15.103:0/2783631889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3e8100a80 0x7fc3e8102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:35.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.382+0000 7fc3ee0c7640 1 --2- 172.21.15.103:0/2783631889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e8106a80 0x7fc3e81030d0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0005cc0 tx=0x7fc3e0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:35.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.382+0000 7fc3deffd640 1 -- 172.21.15.103:0/2783631889 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0004040 con 0x7fc3e8106a80 2024-09-19T06:57:35.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.382+0000 7fc3efb2a640 1 -- 172.21.15.103:0/2783631889 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e81a6b50 con 0x7fc3e8106a80 2024-09-19T06:57:35.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.382+0000 7fc3deffd640 1 -- 172.21.15.103:0/2783631889 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e00046c0 con 0x7fc3e8106a80 2024-09-19T06:57:35.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.382+0000 7fc3deffd640 1 -- 172.21.15.103:0/2783631889 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0046480 con 0x7fc3e8106a80 2024-09-19T06:57:35.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.382+0000 7fc3efb2a640 1 -- 172.21.15.103:0/2783631889 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e81a70e0 con 0x7fc3e8106a80 2024-09-19T06:57:35.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.384+0000 7fc3deffd640 1 -- 172.21.15.103:0/2783631889 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc3e00305a0 con 0x7fc3e8106a80 2024-09-19T06:57:35.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.384+0000 7fc3f1f03740 1 Processor -- start 2024-09-19T06:57:35.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3f1f03740 1 -- start start 2024-09-19T06:57:35.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3f1f03740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c574a0 0x5639f5c57880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3deffd640 1 --2- 172.21.15.103:0/2783631889 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3d40a33b0 0x7fc3d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3f1f03740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c57dc0 0x5639f5c62140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3f1f03740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c62680 0x5639f5c64a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3f1f03740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5639f5bbd2e0 con 0x5639f5c574a0 2024-09-19T06:57:35.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3f1f03740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5639f5b64ce0 con 0x5639f5c62680 2024-09-19T06:57:35.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3f1f03740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5639f5bb92a0 con 0x5639f5c57dc0 2024-09-19T06:57:35.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3ed8c6640 1 --2- 172.21.15.103:0/2783631889 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3d40a33b0 0x7fc3d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3ed0c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c574a0 0x5639f5c57880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3ed0c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c574a0 0x5639f5c57880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48094/0 (socket says 172.21.15.103:48094) 2024-09-19T06:57:35.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3ed0c5640 1 -- 172.21.15.103:0/1855069734 learned_addr learned my addr 172.21.15.103:0/1855069734 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:35.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3ee0c7640 1 --2- 172.21.15.103:0/1855069734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c62680 0x5639f5c64a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3deffd640 1 -- 172.21.15.103:0/2783631889 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7fc3e002e030 con 0x7fc3e8106a80 2024-09-19T06:57:35.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3ed0c5640 1 --2- 172.21.15.103:0/1855069734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c57dc0 0x5639f5c62140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3ed0c5640 1 -- 172.21.15.103:0/1855069734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c57dc0 msgr2=0x5639f5c62140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:35.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3ed0c5640 1 --2- 172.21.15.103:0/1855069734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c57dc0 0x5639f5c62140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3ed0c5640 1 -- 172.21.15.103:0/1855069734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c62680 msgr2=0x5639f5c64a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:35.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3ed0c5640 1 --2- 172.21.15.103:0/1855069734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c62680 0x5639f5c64a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3ed0c5640 1 -- 172.21.15.103:0/1855069734 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f5b49440 con 0x5639f5c574a0 2024-09-19T06:57:35.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.385+0000 7fc3ed8c6640 1 --2- 172.21.15.103:0/2783631889 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3d40a33b0 0x7fc3d40a5870 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4005fd0 tx=0x7fc3e4005850 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:35.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.386+0000 7fc3ed0c5640 1 --2- 172.21.15.103:0/1855069734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c574a0 0x5639f5c57880 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80058a0 tx=0x7fc3d800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80badabe53f4c482 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:35.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.386+0000 7fc3d3fff640 1 -- 172.21.15.103:0/1855069734 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8004030 con 0x5639f5c574a0 2024-09-19T06:57:35.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.386+0000 7fc3d3fff640 1 -- 172.21.15.103:0/1855069734 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d80041d0 con 0x5639f5c574a0 2024-09-19T06:57:35.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.386+0000 7fc3d3fff640 1 -- 172.21.15.103:0/1855069734 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d80106c0 con 0x5639f5c574a0 2024-09-19T06:57:35.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.386+0000 7fc3f1f03740 1 -- 172.21.15.103:0/1855069734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c574a0 msgr2=0x5639f5c57880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:35.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.386+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/1855069734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c574a0 0x5639f5c57880 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80058a0 tx=0x7fc3d800b840 comp rx=0 tx=0).stop 2024-09-19T06:57:35.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.387+0000 7fc3f1f03740 1 -- 172.21.15.103:0/1855069734 shutdown_connections 2024-09-19T06:57:35.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.387+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/1855069734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c62680 0x5639f5c64a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.387+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/1855069734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c57dc0 0x5639f5c62140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.387+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/1855069734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c574a0 0x5639f5c57880 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.387+0000 7fc3f1f03740 1 -- 172.21.15.103:0/1855069734 >> 172.21.15.103:0/1855069734 conn(0x5639f5c53a30 msgr2=0x5639f5c53e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:35.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.387+0000 7fc3f1f03740 1 -- 172.21.15.103:0/1855069734 shutdown_connections 2024-09-19T06:57:35.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.387+0000 7fc3f1f03740 1 -- 172.21.15.103:0/1855069734 wait complete. 2024-09-19T06:57:35.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.387+0000 7fc3f1f03740 1 -- start start 2024-09-19T06:57:35.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.387+0000 7fc3f1f03740 1 Processor -- start 2024-09-19T06:57:35.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3f1f03740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c574a0 0x5639f5c61f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3f1f03740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c57dc0 0x5639f5c5f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3f1f03740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c62680 0x5639f5c5f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3f1f03740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5639f5c65250 con 0x5639f5c57dc0 2024-09-19T06:57:35.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3f1f03740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5639f5b78050 con 0x5639f5c574a0 2024-09-19T06:57:35.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3f1f03740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5639f5c653d0 con 0x5639f5c62680 2024-09-19T06:57:35.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3f1f03740 10 client.0 _ll_register_callbacks cb 0x5639f5bb8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:35.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3ed0c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c57dc0 0x5639f5c5f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3ed0c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c57dc0 0x5639f5c5f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48096/0 (socket says 172.21.15.103:48096) 2024-09-19T06:57:35.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3ed0c5640 1 -- 172.21.15.103:0/543048553 learned_addr learned my addr 172.21.15.103:0/543048553 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:35.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3ed8c6640 1 --2- 172.21.15.103:0/543048553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c574a0 0x5639f5c61f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3ee0c7640 1 --2- 172.21.15.103:0/543048553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c62680 0x5639f5c5f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3ed0c5640 1 -- 172.21.15.103:0/543048553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c62680 msgr2=0x5639f5c5f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:35.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3ed0c5640 1 --2- 172.21.15.103:0/543048553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c62680 0x5639f5c5f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3ed0c5640 1 -- 172.21.15.103:0/543048553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c574a0 msgr2=0x5639f5c61f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:35.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3ed0c5640 1 --2- 172.21.15.103:0/543048553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c574a0 0x5639f5c61f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3ed0c5640 1 -- 172.21.15.103:0/543048553 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f592e3c0 con 0x5639f5c57dc0 2024-09-19T06:57:35.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3ed0c5640 1 --2- 172.21.15.103:0/543048553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c57dc0 0x5639f5c5f1e0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8012300 tx=0x7fc3d8012330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52913ccd3019d58c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:35.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:35.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3d17fa640 1 -- 172.21.15.103:0/543048553 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8005600 con 0x5639f5c57dc0 2024-09-19T06:57:35.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3d17fa640 1 -- 172.21.15.103:0/543048553 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7fc3d80035f0 con 0x5639f5c57dc0 2024-09-19T06:57:35.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3f1f03740 1 -- 172.21.15.103:0/543048553 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d80041d0 con 0x5639f5c57dc0 2024-09-19T06:57:35.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3d17fa640 1 -- 172.21.15.103:0/543048553 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc3d80037f0 con 0x5639f5c57dc0 2024-09-19T06:57:35.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3f1f03740 10 client.18336 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:35.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.388+0000 7fc3f1f03740 1 -- 172.21.15.103:0/543048553 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5639f5c70b70 con 0x5639f5c57dc0 2024-09-19T06:57:35.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.389+0000 7fc3d17fa640 1 -- 172.21.15.103:0/543048553 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d80041d0 con 0x5639f5c57dc0 2024-09-19T06:57:35.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.389+0000 7fc3d17fa640 1 -- 172.21.15.103:0/543048553 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc3d800e490 con 0x5639f5c57dc0 2024-09-19T06:57:35.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.389+0000 7fc3f1f03740 10 client.18336 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:35.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.389+0000 7fc3f1f03740 10 client.18336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:35.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.389+0000 7fc3f1f03740 20 client.18336 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:35.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.389+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/543048553 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5639f5c72030 0x5639f5c74420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.389+0000 7fc3f1f03740 4 client.18336 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:35.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.389+0000 7fc3f1f03740 1 -- 172.21.15.103:0/543048553 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639f5bca850 con 0x5639f5c72030 2024-09-19T06:57:35.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.389+0000 7fc3ed8c6640 1 --2- 172.21.15.103:0/543048553 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5639f5c72030 0x5639f5c74420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.390+0000 7fc3ed8c6640 1 --2- 172.21.15.103:0/543048553 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5639f5c72030 0x5639f5c74420 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19aec5a3dba86967 server_cookie=e701ee9343a7ee95 in_seq=0 out_seq=0 2024-09-19T06:57:35.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.390+0000 7fc3d17fa640 10 client.18336 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:35.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.390+0000 7fc3d17fa640 1 -- 172.21.15.103:0/543048553 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5639f5bca850 con 0x5639f5c72030 2024-09-19T06:57:35.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.390+0000 7fc3d17fa640 10 client.18336 handle_command_reply: tid=0 2024-09-19T06:57:35.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.390+0000 7fc3f1f03740 1 client.18336 shutdown 2024-09-19T06:57:35.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.390+0000 7fc3f1f03740 1 -- 172.21.15.103:0/543048553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c57dc0 msgr2=0x5639f5c5f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:35.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.390+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/543048553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c57dc0 0x5639f5c5f1e0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8012300 tx=0x7fc3d8012330 comp rx=0 tx=0).stop 2024-09-19T06:57:35.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.390+0000 7fc3f1f03740 1 -- 172.21.15.103:0/543048553 shutdown_connections 2024-09-19T06:57:35.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.390+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/543048553 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5639f5c72030 0x5639f5c74420 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.391+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/543048553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c62680 0x5639f5c5f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.391+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/543048553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c57dc0 0x5639f5c5f1e0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.391+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/543048553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c574a0 0x5639f5c61f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.391+0000 7fc3d17fa640 0 client.18336 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:35.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.391+0000 7fc3f1f03740 1 -- 172.21.15.103:0/543048553 >> 172.21.15.103:0/543048553 conn(0x5639f5c53a30 msgr2=0x5639f5c59760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:35.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.391+0000 7fc3f1f03740 1 -- 172.21.15.103:0/543048553 shutdown_connections 2024-09-19T06:57:35.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.391+0000 7fc3f1f03740 1 -- 172.21.15.103:0/543048553 wait complete. 2024-09-19T06:57:35.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.391+0000 7fc3f1f03740 20 client.18336 trim_cache size 0 max 16384 2024-09-19T06:57:35.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.407+0000 7fc3f1f03740 1 Processor -- start 2024-09-19T06:57:35.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.407+0000 7fc3f1f03740 1 -- start start 2024-09-19T06:57:35.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3f1f03740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c574a0 0x5639f5c6e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3f1f03740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c57dc0 0x5639f5c5dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3f1f03740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c62680 0x5639f5c79b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3f1f03740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81afa00 con 0x5639f5c62680 2024-09-19T06:57:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3f1f03740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81b1540 con 0x5639f5c574a0 2024-09-19T06:57:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3f1f03740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81b00d0 con 0x5639f5c57dc0 2024-09-19T06:57:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3ee0c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c62680 0x5639f5c79b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3ee0c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c62680 0x5639f5c79b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48100/0 (socket says 172.21.15.103:48100) 2024-09-19T06:57:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3ee0c7640 1 -- 172.21.15.103:0/1219500826 learned_addr learned my addr 172.21.15.103:0/1219500826 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3ed0c5640 1 --2- 172.21.15.103:0/1219500826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c57dc0 0x5639f5c5dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3ed8c6640 1 --2- 172.21.15.103:0/1219500826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c574a0 0x5639f5c6e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3ee0c7640 1 -- 172.21.15.103:0/1219500826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c57dc0 msgr2=0x5639f5c5dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3ee0c7640 1 --2- 172.21.15.103:0/1219500826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c57dc0 0x5639f5c5dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3ee0c7640 1 -- 172.21.15.103:0/1219500826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c574a0 msgr2=0x5639f5c6e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3ee0c7640 1 --2- 172.21.15.103:0/1219500826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c574a0 0x5639f5c6e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3ee0c7640 1 -- 172.21.15.103:0/1219500826 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f5c54ae0 con 0x5639f5c62680 2024-09-19T06:57:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3ed8c6640 1 --2- 172.21.15.103:0/1219500826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c574a0 0x5639f5c6e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.408+0000 7fc3ee0c7640 1 --2- 172.21.15.103:0/1219500826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c62680 0x5639f5c79b90 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00eeac0 tx=0x7fc3e0037c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fc296702146a86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.409+0000 7fc3d37fe640 1 -- 172.21.15.103:0/1219500826 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e00bd130 con 0x5639f5c62680 2024-09-19T06:57:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.409+0000 7fc3d37fe640 1 -- 172.21.15.103:0/1219500826 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e0036020 con 0x5639f5c62680 2024-09-19T06:57:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.409+0000 7fc3d37fe640 1 -- 172.21.15.103:0/1219500826 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e00d0130 con 0x5639f5c62680 2024-09-19T06:57:35.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.409+0000 7fc3f1f03740 1 -- 172.21.15.103:0/1219500826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c62680 msgr2=0x5639f5c79b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:35.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.409+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/1219500826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c62680 0x5639f5c79b90 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00eeac0 tx=0x7fc3e0037c40 comp rx=0 tx=0).stop 2024-09-19T06:57:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.409+0000 7fc3f1f03740 1 -- 172.21.15.103:0/1219500826 shutdown_connections 2024-09-19T06:57:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.409+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/1219500826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c62680 0x5639f5c79b90 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.409+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/1219500826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c57dc0 0x5639f5c5dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.409+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/1219500826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c574a0 0x5639f5c6e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.409+0000 7fc3f1f03740 1 -- 172.21.15.103:0/1219500826 >> 172.21.15.103:0/1219500826 conn(0x5639f5c53a30 msgr2=0x5639f5c52b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.409+0000 7fc3f1f03740 1 -- 172.21.15.103:0/1219500826 shutdown_connections 2024-09-19T06:57:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.409+0000 7fc3f1f03740 1 -- 172.21.15.103:0/1219500826 wait complete. 2024-09-19T06:57:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.409+0000 7fc3f1f03740 1 -- start start 2024-09-19T06:57:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3f1f03740 1 Processor -- start 2024-09-19T06:57:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3f1f03740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c574a0 0x5639f5c66290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3f1f03740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c57dc0 0x5639f5c667d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3f1f03740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c62680 0x5639f5c66d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3f1f03740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81b0e70 con 0x5639f5c62680 2024-09-19T06:57:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3f1f03740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81b07a0 con 0x5639f5c574a0 2024-09-19T06:57:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3f1f03740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81ada00 con 0x5639f5c57dc0 2024-09-19T06:57:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3f1f03740 10 client.0 _ll_register_callbacks cb 0x7fc3e00e96a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3ee0c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c62680 0x5639f5c66d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3ee0c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c62680 0x5639f5c66d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48108/0 (socket says 172.21.15.103:48108) 2024-09-19T06:57:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3ee0c7640 1 -- 172.21.15.103:0/3798496025 learned_addr learned my addr 172.21.15.103:0/3798496025 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3ed0c5640 1 --2- 172.21.15.103:0/3798496025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c57dc0 0x5639f5c667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3ed8c6640 1 --2- 172.21.15.103:0/3798496025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c574a0 0x5639f5c66290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3ee0c7640 1 -- 172.21.15.103:0/3798496025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c57dc0 msgr2=0x5639f5c667d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3ee0c7640 1 --2- 172.21.15.103:0/3798496025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c57dc0 0x5639f5c667d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3ee0c7640 1 -- 172.21.15.103:0/3798496025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c574a0 msgr2=0x5639f5c66290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3ee0c7640 1 --2- 172.21.15.103:0/3798496025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c574a0 0x5639f5c66290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.410+0000 7fc3ee0c7640 1 -- 172.21.15.103:0/3798496025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f5cdbb60 con 0x5639f5c62680 2024-09-19T06:57:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.411+0000 7fc3ee0c7640 1 --2- 172.21.15.103:0/3798496025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c62680 0x5639f5c66d10 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00b9f70 tx=0x7fc3e00309b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52d1f82562d5832 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.411+0000 7fc3d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.411+0000 7fc3d17fa640 1 -- 172.21.15.103:0/3798496025 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e00d0310 con 0x5639f5c62680 2024-09-19T06:57:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.411+0000 7fc3d17fa640 1 -- 172.21.15.103:0/3798496025 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7fc3e00e9950 con 0x5639f5c62680 2024-09-19T06:57:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.411+0000 7fc3f1f03740 1 -- 172.21.15.103:0/3798496025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3e0036020 con 0x5639f5c62680 2024-09-19T06:57:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.411+0000 7fc3d17fa640 1 -- 172.21.15.103:0/3798496025 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc3e00bcf50 con 0x5639f5c62680 2024-09-19T06:57:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.411+0000 7fc3f1f03740 10 client.18342 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.411+0000 7fc3f1f03740 1 -- 172.21.15.103:0/3798496025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5639f5cdcce0 con 0x5639f5c62680 2024-09-19T06:57:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.411+0000 7fc3d17fa640 1 -- 172.21.15.103:0/3798496025 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e0036020 con 0x5639f5c62680 2024-09-19T06:57:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.411+0000 7fc3d17fa640 1 -- 172.21.15.103:0/3798496025 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc3e004e070 con 0x5639f5c62680 2024-09-19T06:57:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.411+0000 7fc3f1f03740 10 client.18342 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.411+0000 7fc3f1f03740 10 client.18342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.411+0000 7fc3f1f03740 20 client.18342 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.411+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/3798496025 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5639f5cdd3a0 0x5639f5cdf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.411+0000 7fc3f1f03740 4 client.18342 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:57:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.411+0000 7fc3f1f03740 1 -- 172.21.15.103:0/3798496025 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5639f5bc4d60 con 0x5639f5cdd3a0 2024-09-19T06:57:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3ed8c6640 1 --2- 172.21.15.103:0/3798496025 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5639f5cdd3a0 0x5639f5cdf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3ed8c6640 1 --2- 172.21.15.103:0/3798496025 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5639f5cdd3a0 0x5639f5cdf7c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3def7d37744e5b5f server_cookie=a8224ed22843ff0 in_seq=0 out_seq=0 2024-09-19T06:57:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3d17fa640 10 client.18342 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3d17fa640 1 -- 172.21.15.103:0/3798496025 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5639f5bc4d60 con 0x5639f5cdd3a0 2024-09-19T06:57:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3d17fa640 10 client.18342 handle_command_reply: tid=0 2024-09-19T06:57:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3f1f03740 1 client.18342 shutdown 2024-09-19T06:57:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3f1f03740 1 -- 172.21.15.103:0/3798496025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c62680 msgr2=0x5639f5c66d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/3798496025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c62680 0x5639f5c66d10 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00b9f70 tx=0x7fc3e00309b0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3f1f03740 1 -- 172.21.15.103:0/3798496025 shutdown_connections 2024-09-19T06:57:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/3798496025 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5639f5cdd3a0 0x5639f5cdf7c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/3798496025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f5c62680 0x5639f5c66d10 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3d17fa640 0 client.18342 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/3798496025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f5c57dc0 0x5639f5c667d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3f1f03740 1 --2- 172.21.15.103:0/3798496025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f5c574a0 0x5639f5c66290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3f1f03740 1 -- 172.21.15.103:0/3798496025 >> 172.21.15.103:0/3798496025 conn(0x5639f5c53a30 msgr2=0x5639f5c5eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3f1f03740 1 -- 172.21.15.103:0/3798496025 shutdown_connections 2024-09-19T06:57:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.412+0000 7fc3f1f03740 1 -- 172.21.15.103:0/3798496025 wait complete. 2024-09-19T06:57:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.413+0000 7fc3f1f03740 20 client.18342 trim_cache size 0 max 16384 2024-09-19T06:57:35.430 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:57:35.430 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T06:57:35.430 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T06:57:35.430 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:57:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.413+0000 7fc3efb2a640 1 -- 172.21.15.103:0/2783631889 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3d40a33b0 msgr2=0x7fc3d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.413+0000 7fc3efb2a640 1 --2- 172.21.15.103:0/2783631889 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3d40a33b0 0x7fc3d40a5870 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4005fd0 tx=0x7fc3e4005850 comp rx=0 tx=0).stop 2024-09-19T06:57:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.413+0000 7fc3efb2a640 1 -- 172.21.15.103:0/2783631889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e8106a80 msgr2=0x7fc3e81030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.413+0000 7fc3efb2a640 1 --2- 172.21.15.103:0/2783631889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e8106a80 0x7fc3e81030d0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0005cc0 tx=0x7fc3e0003970 comp rx=0 tx=0).stop 2024-09-19T06:57:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.413+0000 7fc3efb2a640 1 -- 172.21.15.103:0/2783631889 shutdown_connections 2024-09-19T06:57:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.413+0000 7fc3efb2a640 1 --2- 172.21.15.103:0/2783631889 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc3d40a33b0 0x7fc3d40a5870 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.413+0000 7fc3efb2a640 1 --2- 172.21.15.103:0/2783631889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3e8106a80 0x7fc3e81030d0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.413+0000 7fc3efb2a640 1 --2- 172.21.15.103:0/2783631889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3e8101420 0x7fc3e8102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.413+0000 7fc3efb2a640 1 --2- 172.21.15.103:0/2783631889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3e8100a80 0x7fc3e8102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.413+0000 7fc3efb2a640 1 -- 172.21.15.103:0/2783631889 >> 172.21.15.103:0/2783631889 conn(0x7fc3e80fc7a0 msgr2=0x7fc3e806a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.413+0000 7fc3efb2a640 1 -- 172.21.15.103:0/2783631889 shutdown_connections 2024-09-19T06:57:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:35.413+0000 7fc3efb2a640 1 -- 172.21.15.103:0/2783631889 wait complete. 2024-09-19T06:57:36.057 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:57:36.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:57:36.306 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:36.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:36 smithi103 ceph-mon[31158]: pgmap v258: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-19T06:57:36.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:36 smithi134 ceph-mon[35356]: pgmap v258: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-19T06:57:36.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:36 smithi174 ceph-mon[35637]: pgmap v258: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-19T06:57:36.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.728+0000 7f89bff70640 1 -- 172.21.15.103:0/1871707681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89b8108300 msgr2=0x7f89b810ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:36.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.728+0000 7f89bff70640 1 --2- 172.21.15.103:0/1871707681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89b8108300 0x7f89b810ec70 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009cd0 tx=0x7f89b002a8f0 comp rx=0 tx=0).stop 2024-09-19T06:57:36.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.728+0000 7f89bff70640 1 -- 172.21.15.103:0/1871707681 shutdown_connections 2024-09-19T06:57:36.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.728+0000 7f89bff70640 1 --2- 172.21.15.103:0/1871707681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89b8108300 0x7f89b810ec70 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:36.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.728+0000 7f89bff70640 1 --2- 172.21.15.103:0/1871707681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b8103780 0x7f89b8107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:36.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.728+0000 7f89bff70640 1 --2- 172.21.15.103:0/1871707681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b8102dd0 0x7f89b81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:36.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.728+0000 7f89bff70640 1 -- 172.21.15.103:0/1871707681 >> 172.21.15.103:0/1871707681 conn(0x7f89b80fc7d0 msgr2=0x7f89b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:36.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.728+0000 7f89bff70640 1 -- 172.21.15.103:0/1871707681 shutdown_connections 2024-09-19T06:57:36.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.728+0000 7f89bff70640 1 -- 172.21.15.103:0/1871707681 wait complete. 2024-09-19T06:57:36.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.728+0000 7f89bff70640 1 Processor -- start 2024-09-19T06:57:36.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89bff70640 1 -- start start 2024-09-19T06:57:36.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89bff70640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b8102dd0 0x7f89b8100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:36.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89bff70640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b8103780 0x7f89b8100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:36.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89bff70640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89b8108300 0x7f89b81014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:36.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89bff70640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f89b81120f0 con 0x7f89b8108300 2024-09-19T06:57:36.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89bff70640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89b8111f70 con 0x7f89b8102dd0 2024-09-19T06:57:36.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89bff70640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89b8112270 con 0x7f89b8103780 2024-09-19T06:57:36.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89be50d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89b8108300 0x7f89b81014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:36.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89be50d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89b8108300 0x7f89b81014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48114/0 (socket says 172.21.15.103:48114) 2024-09-19T06:57:36.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89be50d640 1 -- 172.21.15.103:0/2340988042 learned_addr learned my addr 172.21.15.103:0/2340988042 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:36.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89bd50b640 1 --2- 172.21.15.103:0/2340988042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b8103780 0x7f89b8100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:36.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89bdd0c640 1 --2- 172.21.15.103:0/2340988042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b8102dd0 0x7f89b8100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:36.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89be50d640 1 -- 172.21.15.103:0/2340988042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b8103780 msgr2=0x7f89b8100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:36.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89be50d640 1 --2- 172.21.15.103:0/2340988042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b8103780 0x7f89b8100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:36.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89be50d640 1 -- 172.21.15.103:0/2340988042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b8102dd0 msgr2=0x7f89b8100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:36.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89be50d640 1 --2- 172.21.15.103:0/2340988042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b8102dd0 0x7f89b8100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:36.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89be50d640 1 -- 172.21.15.103:0/2340988042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b81a6a10 con 0x7f89b8108300 2024-09-19T06:57:36.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.729+0000 7f89be50d640 1 --2- 172.21.15.103:0/2340988042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89b8108300 0x7f89b81014b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009ca0 tx=0x7f89b00039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:36.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.730+0000 7f89aeffd640 1 -- 172.21.15.103:0/2340988042 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0004040 con 0x7f89b8108300 2024-09-19T06:57:36.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.730+0000 7f89aeffd640 1 -- 172.21.15.103:0/2340988042 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b0040070 con 0x7f89b8108300 2024-09-19T06:57:36.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.730+0000 7f89bff70640 1 -- 172.21.15.103:0/2340988042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b81a6ca0 con 0x7f89b8108300 2024-09-19T06:57:36.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.730+0000 7f89aeffd640 1 -- 172.21.15.103:0/2340988042 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0045c90 con 0x7f89b8108300 2024-09-19T06:57:36.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.730+0000 7f89bff70640 1 -- 172.21.15.103:0/2340988042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b81a71e0 con 0x7f89b8108300 2024-09-19T06:57:36.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.731+0000 7f89bff70640 1 -- 172.21.15.103:0/2340988042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b81031b0 con 0x7f89b8108300 2024-09-19T06:57:36.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.732+0000 7f89aeffd640 1 -- 172.21.15.103:0/2340988042 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f89b0004b40 con 0x7f89b8108300 2024-09-19T06:57:36.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.732+0000 7f89aeffd640 1 --2- 172.21.15.103:0/2340988042 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f89a40a33b0 0x7f89a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:36.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.732+0000 7f89bdd0c640 1 --2- 172.21.15.103:0/2340988042 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f89a40a33b0 0x7f89a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:36.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.733+0000 7f89aeffd640 1 -- 172.21.15.103:0/2340988042 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f89b00f1280 con 0x7f89b8108300 2024-09-19T06:57:36.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.733+0000 7f89bdd0c640 1 --2- 172.21.15.103:0/2340988042 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f89a40a33b0 0x7f89a40a5870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f89a8005e10 tx=0x7f89a8005da0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:36.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.736+0000 7f89aeffd640 1 -- 172.21.15.103:0/2340988042 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f89b00bdde0 con 0x7f89b8108300 2024-09-19T06:57:36.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.936+0000 7f89bff70640 1 -- 172.21.15.103:0/2340988042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89b81a7410 con 0x7f89b8108300 2024-09-19T06:57:36.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.936+0000 7f89aeffd640 1 -- 172.21.15.103:0/2340988042 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f89b00bd6e0 con 0x7f89b8108300 2024-09-19T06:57:36.938 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:57:36.939 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:57:36.939 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:57:36.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.938+0000 7f89bff70640 1 -- 172.21.15.103:0/2340988042 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f89a40a33b0 msgr2=0x7f89a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:36.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.938+0000 7f89bff70640 1 --2- 172.21.15.103:0/2340988042 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f89a40a33b0 0x7f89a40a5870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f89a8005e10 tx=0x7f89a8005da0 comp rx=0 tx=0).stop 2024-09-19T06:57:36.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.938+0000 7f89bff70640 1 -- 172.21.15.103:0/2340988042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89b8108300 msgr2=0x7f89b81014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:36.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.938+0000 7f89bff70640 1 --2- 172.21.15.103:0/2340988042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89b8108300 0x7f89b81014b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009ca0 tx=0x7f89b00039f0 comp rx=0 tx=0).stop 2024-09-19T06:57:36.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.938+0000 7f89bff70640 1 -- 172.21.15.103:0/2340988042 shutdown_connections 2024-09-19T06:57:36.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.938+0000 7f89bff70640 1 --2- 172.21.15.103:0/2340988042 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f89a40a33b0 0x7f89a40a5870 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:36.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.938+0000 7f89bff70640 1 --2- 172.21.15.103:0/2340988042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89b8108300 0x7f89b81014b0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:36.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.938+0000 7f89bff70640 1 --2- 172.21.15.103:0/2340988042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b8103780 0x7f89b8100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:36.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.938+0000 7f89bff70640 1 --2- 172.21.15.103:0/2340988042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b8102dd0 0x7f89b8100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:36.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.938+0000 7f89bff70640 1 -- 172.21.15.103:0/2340988042 >> 172.21.15.103:0/2340988042 conn(0x7f89b80fc7d0 msgr2=0x7f89b8105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:36.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.938+0000 7f89bff70640 1 -- 172.21.15.103:0/2340988042 shutdown_connections 2024-09-19T06:57:36.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:36.939+0000 7f89bff70640 1 -- 172.21.15.103:0/2340988042 wait complete. 2024-09-19T06:57:37.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:37 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2340988042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:57:37.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:37 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2340988042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:57:37.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:37 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2340988042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:57:37.591 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:57:37.839 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:38.281 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:38 smithi103 ceph-mon[31158]: pgmap v259: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-19T06:57:38.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.278+0000 7f4762a8d640 1 -- 172.21.15.103:0/138754177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f475c101e10 msgr2=0x7f475c1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:38.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.278+0000 7f4762a8d640 1 --2- 172.21.15.103:0/138754177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f475c101e10 0x7f475c1021f0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f474c009ca0 tx=0x7f474c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.279+0000 7f4762a8d640 1 -- 172.21.15.103:0/138754177 shutdown_connections 2024-09-19T06:57:38.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.279+0000 7f4762a8d640 1 --2- 172.21.15.103:0/138754177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f475c106f40 0x7f475c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.279+0000 7f4762a8d640 1 --2- 172.21.15.103:0/138754177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f475c1027c0 0x7f475c106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.279+0000 7f4762a8d640 1 --2- 172.21.15.103:0/138754177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f475c101e10 0x7f475c1021f0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.279+0000 7f4762a8d640 1 -- 172.21.15.103:0/138754177 >> 172.21.15.103:0/138754177 conn(0x7f475c0fd8d0 msgr2=0x7f475c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:38.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.279+0000 7f4762a8d640 1 -- 172.21.15.103:0/138754177 shutdown_connections 2024-09-19T06:57:38.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.279+0000 7f4762a8d640 1 -- 172.21.15.103:0/138754177 wait complete. 2024-09-19T06:57:38.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.279+0000 7f4762a8d640 1 Processor -- start 2024-09-19T06:57:38.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.279+0000 7f4762a8d640 1 -- start start 2024-09-19T06:57:38.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f4762a8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f475c101e10 0x7f475c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f4762a8d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f475c1027c0 0x7f475c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f4762a8d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f475c106f40 0x7f475c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f4762a8d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f475c111f60 con 0x7f475c101e10 2024-09-19T06:57:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f4762a8d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f475c111de0 con 0x7f475c106f40 2024-09-19T06:57:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f4762a8d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f475c1120e0 con 0x7f475c1027c0 2024-09-19T06:57:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f4760829640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f475c101e10 0x7f475c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f4760829640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f475c101e10 0x7f475c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41424/0 (socket says 172.21.15.103:41424) 2024-09-19T06:57:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f4760829640 1 -- 172.21.15.103:0/229310513 learned_addr learned my addr 172.21.15.103:0/229310513 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f475bfff640 1 --2- 172.21.15.103:0/229310513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f475c1027c0 0x7f475c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f476102a640 1 --2- 172.21.15.103:0/229310513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f475c106f40 0x7f475c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f4760829640 1 -- 172.21.15.103:0/229310513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f475c1027c0 msgr2=0x7f475c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f4760829640 1 --2- 172.21.15.103:0/229310513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f475c1027c0 0x7f475c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f4760829640 1 -- 172.21.15.103:0/229310513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f475c106f40 msgr2=0x7f475c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f4760829640 1 --2- 172.21.15.103:0/229310513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f475c106f40 0x7f475c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f4760829640 1 -- 172.21.15.103:0/229310513 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475c19d3a0 con 0x7f475c101e10 2024-09-19T06:57:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f476102a640 1 --2- 172.21.15.103:0/229310513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f475c106f40 0x7f475c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f4760829640 1 --2- 172.21.15.103:0/229310513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f475c101e10 0x7f475c1983f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f474c0097b0 tx=0x7f474c031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.280+0000 7f4759ffb640 1 -- 172.21.15.103:0/229310513 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c02fba0 con 0x7f475c101e10 2024-09-19T06:57:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.281+0000 7f4759ffb640 1 -- 172.21.15.103:0/229310513 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f474c02fd40 con 0x7f475c101e10 2024-09-19T06:57:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.281+0000 7f4762a8d640 1 -- 172.21.15.103:0/229310513 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f475c19d630 con 0x7f475c101e10 2024-09-19T06:57:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.281+0000 7f4759ffb640 1 -- 172.21.15.103:0/229310513 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c03b860 con 0x7f475c101e10 2024-09-19T06:57:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.281+0000 7f4762a8d640 1 -- 172.21.15.103:0/229310513 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f475c1a4f70 con 0x7f475c101e10 2024-09-19T06:57:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.283+0000 7f4759ffb640 1 -- 172.21.15.103:0/229310513 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f474c030c30 con 0x7f475c101e10 2024-09-19T06:57:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.283+0000 7f4764e66740 1 Processor -- start 2024-09-19T06:57:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.283+0000 7f4764e66740 1 -- start start 2024-09-19T06:57:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f4764e66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e23b40 0x556882e22090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f4764e66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e225d0 0x556882e22a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f4764e66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e2d370 0x556882e2f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f4764e66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556882d882f0 con 0x556882e23b40 2024-09-19T06:57:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f4764e66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556882d2fcf0 con 0x556882e2d370 2024-09-19T06:57:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f4764e66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556882d842b0 con 0x556882e225d0 2024-09-19T06:57:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f4759ffb640 1 --2- 172.21.15.103:0/229310513 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47480a33b0 0x7f47480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f476102a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e23b40 0x556882e22090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f476102a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e23b40 0x556882e22090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41434/0 (socket says 172.21.15.103:41434) 2024-09-19T06:57:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f4760829640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e225d0 0x556882e22a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f476102a640 1 -- 172.21.15.103:0/2197308409 learned_addr learned my addr 172.21.15.103:0/2197308409 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f476102a640 1 --2- 172.21.15.103:0/229310513 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47480a33b0 0x7f47480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f475bfff640 1 --2- 172.21.15.103:0/2197308409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e2d370 0x556882e2f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f476102a640 1 -- 172.21.15.103:0/2197308409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e225d0 msgr2=0x556882e22a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f476102a640 1 --2- 172.21.15.103:0/2197308409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e225d0 0x556882e22a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f4759ffb640 1 -- 172.21.15.103:0/229310513 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f474c0f1330 con 0x7f475c101e10 2024-09-19T06:57:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f476102a640 1 -- 172.21.15.103:0/2197308409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e2d370 msgr2=0x556882e2f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f476102a640 1 --2- 172.21.15.103:0/2197308409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e2d370 0x556882e2f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f476102a640 1 -- 172.21.15.103:0/2197308409 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556882d143a0 con 0x556882e23b40 2024-09-19T06:57:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f476102a640 1 --2- 172.21.15.103:0/2197308409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e23b40 0x556882e22090 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f4754005fd0 tx=0x7f47540107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe14fae42e818c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f476102a640 1 --2- 172.21.15.103:0/229310513 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47480a33b0 0x7f47480a5870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4754005d50 tx=0x7f4754015000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.284+0000 7f4752ffd640 1 -- 172.21.15.103:0/2197308409 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475400bd40 con 0x556882e23b40 2024-09-19T06:57:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.285+0000 7f4752ffd640 1 -- 172.21.15.103:0/2197308409 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4754005500 con 0x556882e23b40 2024-09-19T06:57:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.285+0000 7f4752ffd640 1 -- 172.21.15.103:0/2197308409 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475401d3f0 con 0x556882e23b40 2024-09-19T06:57:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.285+0000 7f4764e66740 1 -- 172.21.15.103:0/2197308409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e23b40 msgr2=0x556882e22090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.285+0000 7f4764e66740 1 --2- 172.21.15.103:0/2197308409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e23b40 0x556882e22090 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f4754005fd0 tx=0x7f47540107a0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.285+0000 7f4764e66740 1 -- 172.21.15.103:0/2197308409 shutdown_connections 2024-09-19T06:57:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.285+0000 7f4764e66740 1 --2- 172.21.15.103:0/2197308409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e2d370 0x556882e2f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.285+0000 7f4764e66740 1 --2- 172.21.15.103:0/2197308409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e225d0 0x556882e22a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.285+0000 7f4764e66740 1 --2- 172.21.15.103:0/2197308409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e23b40 0x556882e22090 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.285+0000 7f4764e66740 1 -- 172.21.15.103:0/2197308409 >> 172.21.15.103:0/2197308409 conn(0x556882deb310 msgr2=0x556882e1ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.285+0000 7f4764e66740 1 -- 172.21.15.103:0/2197308409 shutdown_connections 2024-09-19T06:57:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.285+0000 7f4764e66740 1 -- 172.21.15.103:0/2197308409 wait complete. 2024-09-19T06:57:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.286+0000 7f4764e66740 1 -- start start 2024-09-19T06:57:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.286+0000 7f4764e66740 1 Processor -- start 2024-09-19T06:57:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.286+0000 7f4764e66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e225d0 0x556882e2cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.286+0000 7f4764e66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e23b40 0x556882e2a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.286+0000 7f4764e66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e2d370 0x556882e2a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.286+0000 7f4764e66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556882e30010 con 0x556882e225d0 2024-09-19T06:57:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.286+0000 7f4764e66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556882d43060 con 0x556882e23b40 2024-09-19T06:57:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.286+0000 7f4764e66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556882e30190 con 0x556882e2d370 2024-09-19T06:57:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.286+0000 7f4760829640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e225d0 0x556882e2cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.286+0000 7f4764e66740 10 client.0 _ll_register_callbacks cb 0x556882a968c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.286+0000 7f4760829640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e225d0 0x556882e2cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41448/0 (socket says 172.21.15.103:41448) 2024-09-19T06:57:38.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.286+0000 7f4760829640 1 -- 172.21.15.103:0/70098819 learned_addr learned my addr 172.21.15.103:0/70098819 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:38.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.286+0000 7f475bfff640 1 --2- 172.21.15.103:0/70098819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e23b40 0x556882e2a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.286+0000 7f4760829640 1 -- 172.21.15.103:0/70098819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e2d370 msgr2=0x556882e2a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:38.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.286+0000 7f4760829640 1 --2- 172.21.15.103:0/70098819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e2d370 0x556882e2a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.286+0000 7f4760829640 1 -- 172.21.15.103:0/70098819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e23b40 msgr2=0x556882e2a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:38.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f4760829640 1 --2- 172.21.15.103:0/70098819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e23b40 0x556882e2a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f4760829640 1 -- 172.21.15.103:0/70098819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556882af93c0 con 0x556882e225d0 2024-09-19T06:57:38.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f4760829640 1 --2- 172.21.15.103:0/70098819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e225d0 0x556882e2cda0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f474c0ba0d0 tx=0x7f474c0bd300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15aa23005342493 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:38.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f473bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:38.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f473bfff640 1 -- 172.21.15.103:0/70098819 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c0c60e0 con 0x556882e225d0 2024-09-19T06:57:38.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f473bfff640 1 -- 172.21.15.103:0/70098819 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f474c004b40 con 0x556882e225d0 2024-09-19T06:57:38.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f4764e66740 1 -- 172.21.15.103:0/70098819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4754005500 con 0x556882e225d0 2024-09-19T06:57:38.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f473bfff640 1 -- 172.21.15.103:0/70098819 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f474c0bde00 con 0x556882e225d0 2024-09-19T06:57:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f4764e66740 10 client.18360 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f4764e66740 1 -- 172.21.15.103:0/70098819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556882e30010 con 0x556882e225d0 2024-09-19T06:57:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f473bfff640 1 -- 172.21.15.103:0/70098819 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4754005500 con 0x556882e225d0 2024-09-19T06:57:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f473bfff640 1 -- 172.21.15.103:0/70098819 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f474c0bcb50 con 0x556882e225d0 2024-09-19T06:57:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f4764e66740 10 client.18360 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f4764e66740 10 client.18360 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:57:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f4764e66740 20 client.18360 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f4764e66740 1 --2- 172.21.15.103:0/70098819 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x556882e3ced0 0x556882e3f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f4764e66740 4 client.18360 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.287+0000 7f4764e66740 1 -- 172.21.15.103:0/70098819 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556882d95860 con 0x556882e3ced0 2024-09-19T06:57:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.288+0000 7f475bfff640 1 --2- 172.21.15.103:0/70098819 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x556882e3ced0 0x556882e3f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.288+0000 7f475bfff640 1 --2- 172.21.15.103:0/70098819 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x556882e3ced0 0x556882e3f360 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2620952f31c2d6fe server_cookie=39e12ac1f3f860bf in_seq=0 out_seq=0 2024-09-19T06:57:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.288+0000 7f473bfff640 10 client.18360 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:57:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.289+0000 7f473bfff640 1 -- 172.21.15.103:0/70098819 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556882d95860 con 0x556882e3ced0 2024-09-19T06:57:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.289+0000 7f473bfff640 10 client.18360 handle_command_reply: tid=0 2024-09-19T06:57:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.289+0000 7f4764e66740 1 client.18360 shutdown 2024-09-19T06:57:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.289+0000 7f4764e66740 1 -- 172.21.15.103:0/70098819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e225d0 msgr2=0x556882e2cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.289+0000 7f4764e66740 1 --2- 172.21.15.103:0/70098819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e225d0 0x556882e2cda0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f474c0ba0d0 tx=0x7f474c0bd300 comp rx=0 tx=0).stop 2024-09-19T06:57:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.289+0000 7f4764e66740 1 -- 172.21.15.103:0/70098819 shutdown_connections 2024-09-19T06:57:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.289+0000 7f4764e66740 1 --2- 172.21.15.103:0/70098819 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x556882e3ced0 0x556882e3f360 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.289+0000 7f4764e66740 1 --2- 172.21.15.103:0/70098819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e2d370 0x556882e2a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.289+0000 7f4764e66740 1 --2- 172.21.15.103:0/70098819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e23b40 0x556882e2a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.289+0000 7f4764e66740 1 --2- 172.21.15.103:0/70098819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e225d0 0x556882e2cda0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.289+0000 7f473bfff640 0 client.18360 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:57:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.289+0000 7f4764e66740 1 -- 172.21.15.103:0/70098819 >> 172.21.15.103:0/70098819 conn(0x556882deb310 msgr2=0x556882e24960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.289+0000 7f4764e66740 1 -- 172.21.15.103:0/70098819 shutdown_connections 2024-09-19T06:57:38.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.289+0000 7f4764e66740 1 -- 172.21.15.103:0/70098819 wait complete. 2024-09-19T06:57:38.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.289+0000 7f4764e66740 20 client.18360 trim_cache size 0 max 16384 2024-09-19T06:57:38.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f4764e66740 1 Processor -- start 2024-09-19T06:57:38.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f4764e66740 1 -- start start 2024-09-19T06:57:38.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f4764e66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e225d0 0x556882e1ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f4764e66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e23b40 0x556882e25120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f4764e66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e2d370 0x556882e25660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f4764e66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f475c1ad820 con 0x556882e225d0 2024-09-19T06:57:38.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f4764e66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f475c1af360 con 0x556882e23b40 2024-09-19T06:57:38.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f4764e66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f475c1adef0 con 0x556882e2d370 2024-09-19T06:57:38.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f4760829640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e225d0 0x556882e1ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f4760829640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e225d0 0x556882e1ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41464/0 (socket says 172.21.15.103:41464) 2024-09-19T06:57:38.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f4760829640 1 -- 172.21.15.103:0/2014253144 learned_addr learned my addr 172.21.15.103:0/2014253144 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:38.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f476102a640 1 --2- 172.21.15.103:0/2014253144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e2d370 0x556882e25660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f475bfff640 1 --2- 172.21.15.103:0/2014253144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e23b40 0x556882e25120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f4760829640 1 -- 172.21.15.103:0/2014253144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e2d370 msgr2=0x556882e25660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:38.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f4760829640 1 --2- 172.21.15.103:0/2014253144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e2d370 0x556882e25660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f4760829640 1 -- 172.21.15.103:0/2014253144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e23b40 msgr2=0x556882e25120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:38.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f4760829640 1 --2- 172.21.15.103:0/2014253144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e23b40 0x556882e25120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f4760829640 1 -- 172.21.15.103:0/2014253144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556882e3f9e0 con 0x556882e225d0 2024-09-19T06:57:38.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.302+0000 7f475bfff640 1 --2- 172.21.15.103:0/2014253144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e23b40 0x556882e25120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:38.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.303+0000 7f4760829640 1 --2- 172.21.15.103:0/2014253144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e225d0 0x556882e1ed70 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f474c0eec20 tx=0x7f474c030fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7b781eb42c09a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:38.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.303+0000 7f47527fc640 1 -- 172.21.15.103:0/2014253144 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c0bd700 con 0x556882e225d0 2024-09-19T06:57:38.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.303+0000 7f47527fc640 1 -- 172.21.15.103:0/2014253144 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f474c036040 con 0x556882e225d0 2024-09-19T06:57:38.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.303+0000 7f47527fc640 1 -- 172.21.15.103:0/2014253144 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c0c5ee0 con 0x556882e225d0 2024-09-19T06:57:38.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.303+0000 7f4764e66740 1 -- 172.21.15.103:0/2014253144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e225d0 msgr2=0x556882e1ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:38.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.303+0000 7f4764e66740 1 --2- 172.21.15.103:0/2014253144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e225d0 0x556882e1ed70 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f474c0eec20 tx=0x7f474c030fd0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.303+0000 7f4764e66740 1 -- 172.21.15.103:0/2014253144 shutdown_connections 2024-09-19T06:57:38.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.303+0000 7f4764e66740 1 --2- 172.21.15.103:0/2014253144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e2d370 0x556882e25660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.303+0000 7f4764e66740 1 --2- 172.21.15.103:0/2014253144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e23b40 0x556882e25120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.303+0000 7f4764e66740 1 --2- 172.21.15.103:0/2014253144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e225d0 0x556882e1ed70 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.303+0000 7f4764e66740 1 -- 172.21.15.103:0/2014253144 >> 172.21.15.103:0/2014253144 conn(0x556882deb310 msgr2=0x556882e1dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:38.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.303+0000 7f4764e66740 1 -- 172.21.15.103:0/2014253144 shutdown_connections 2024-09-19T06:57:38.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.303+0000 7f4764e66740 1 -- 172.21.15.103:0/2014253144 wait complete. 2024-09-19T06:57:38.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.304+0000 7f4764e66740 1 -- start start 2024-09-19T06:57:38.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.304+0000 7f4764e66740 1 Processor -- start 2024-09-19T06:57:38.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.304+0000 7f4764e66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e225d0 0x556882e35420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.304+0000 7f4764e66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e23b40 0x556882e35960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.304+0000 7f4764e66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e2d370 0x556882e323f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.304+0000 7f4764e66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f475c1aec90 con 0x556882e23b40 2024-09-19T06:57:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.304+0000 7f4764e66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f475c1ae5c0 con 0x556882e2d370 2024-09-19T06:57:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.304+0000 7f4764e66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f475c1ab820 con 0x556882e225d0 2024-09-19T06:57:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.304+0000 7f4764e66740 10 client.0 _ll_register_callbacks cb 0x7f474c0b9a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.304+0000 7f475bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e23b40 0x556882e35960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.304+0000 7f475bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e23b40 0x556882e35960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41470/0 (socket says 172.21.15.103:41470) 2024-09-19T06:57:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.304+0000 7f475bfff640 1 -- 172.21.15.103:0/1114361388 learned_addr learned my addr 172.21.15.103:0/1114361388 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.304+0000 7f476102a640 1 --2- 172.21.15.103:0/1114361388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e2d370 0x556882e323f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.304+0000 7f4760829640 1 --2- 172.21.15.103:0/1114361388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e225d0 0x556882e35420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f475bfff640 1 -- 172.21.15.103:0/1114361388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e225d0 msgr2=0x556882e35420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f475bfff640 1 --2- 172.21.15.103:0/1114361388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e225d0 0x556882e35420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f475bfff640 1 -- 172.21.15.103:0/1114361388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e2d370 msgr2=0x556882e323f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f475bfff640 1 --2- 172.21.15.103:0/1114361388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e2d370 0x556882e323f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f475bfff640 1 -- 172.21.15.103:0/1114361388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556882e1f960 con 0x556882e23b40 2024-09-19T06:57:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f476102a640 1 --2- 172.21.15.103:0/1114361388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e2d370 0x556882e323f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f475bfff640 1 --2- 172.21.15.103:0/1114361388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e23b40 0x556882e35960 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f474400aa70 tx=0x7f47440108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff898d3d410fb51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f473bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f473bfff640 1 -- 172.21.15.103:0/1114361388 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474401d880 con 0x556882e23b40 2024-09-19T06:57:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f473bfff640 1 -- 172.21.15.103:0/1114361388 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f474401da20 con 0x556882e23b40 2024-09-19T06:57:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f4764e66740 1 -- 172.21.15.103:0/1114361388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f474c036040 con 0x556882e23b40 2024-09-19T06:57:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f473bfff640 1 -- 172.21.15.103:0/1114361388 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f474402d050 con 0x556882e23b40 2024-09-19T06:57:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f4764e66740 10 client.18366 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f4764e66740 1 -- 172.21.15.103:0/1114361388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556882ea8330 con 0x556882e23b40 2024-09-19T06:57:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f473bfff640 1 -- 172.21.15.103:0/1114361388 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f474c036040 con 0x556882e23b40 2024-09-19T06:57:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f473bfff640 1 -- 172.21.15.103:0/1114361388 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f474401c460 con 0x556882e23b40 2024-09-19T06:57:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f4764e66740 10 client.18366 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f4764e66740 10 client.18366 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:57:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f4764e66740 20 client.18366 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f4764e66740 1 --2- 172.21.15.103:0/1114361388 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x556882ea8ae0 0x556882eaaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f4764e66740 4 client.18366 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:57:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.305+0000 7f4764e66740 1 -- 172.21.15.103:0/1114361388 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556882d8fd70 con 0x556882ea8ae0 2024-09-19T06:57:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.306+0000 7f4760829640 1 --2- 172.21.15.103:0/1114361388 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x556882ea8ae0 0x556882eaaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.306+0000 7f4760829640 1 --2- 172.21.15.103:0/1114361388 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x556882ea8ae0 0x556882eaaed0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3b777239cffb7310 server_cookie=741235e75edacab9 in_seq=0 out_seq=0 2024-09-19T06:57:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.306+0000 7f473bfff640 10 client.18366 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:57:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.306+0000 7f473bfff640 1 -- 172.21.15.103:0/1114361388 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556882d8fd70 con 0x556882ea8ae0 2024-09-19T06:57:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.306+0000 7f473bfff640 10 client.18366 handle_command_reply: tid=0 2024-09-19T06:57:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.306+0000 7f4764e66740 1 client.18366 shutdown 2024-09-19T06:57:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.307+0000 7f4764e66740 1 -- 172.21.15.103:0/1114361388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e23b40 msgr2=0x556882e35960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.307+0000 7f4764e66740 1 --2- 172.21.15.103:0/1114361388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e23b40 0x556882e35960 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f474400aa70 tx=0x7f47440108c0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.307+0000 7f4764e66740 1 -- 172.21.15.103:0/1114361388 shutdown_connections 2024-09-19T06:57:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.307+0000 7f4764e66740 1 --2- 172.21.15.103:0/1114361388 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x556882ea8ae0 0x556882eaaed0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.307+0000 7f4764e66740 1 --2- 172.21.15.103:0/1114361388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556882e2d370 0x556882e323f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.307+0000 7f4764e66740 1 --2- 172.21.15.103:0/1114361388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556882e23b40 0x556882e35960 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.307+0000 7f4764e66740 1 --2- 172.21.15.103:0/1114361388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556882e225d0 0x556882e35420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.307+0000 7f473bfff640 0 client.18366 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:57:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.307+0000 7f4764e66740 1 -- 172.21.15.103:0/1114361388 >> 172.21.15.103:0/1114361388 conn(0x556882deb310 msgr2=0x556882e382a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:38.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.307+0000 7f4764e66740 1 -- 172.21.15.103:0/1114361388 shutdown_connections 2024-09-19T06:57:38.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.307+0000 7f4764e66740 1 -- 172.21.15.103:0/1114361388 wait complete. 2024-09-19T06:57:38.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.307+0000 7f4764e66740 20 client.18366 trim_cache size 0 max 16384 2024-09-19T06:57:38.327 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:57:38.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.307+0000 7f4762a8d640 1 -- 172.21.15.103:0/229310513 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47480a33b0 msgr2=0x7f47480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:38.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.307+0000 7f4762a8d640 1 --2- 172.21.15.103:0/229310513 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47480a33b0 0x7f47480a5870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4754005d50 tx=0x7f4754015000 comp rx=0 tx=0).stop 2024-09-19T06:57:38.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.307+0000 7f4762a8d640 1 -- 172.21.15.103:0/229310513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f475c101e10 msgr2=0x7f475c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:38.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.307+0000 7f4762a8d640 1 --2- 172.21.15.103:0/229310513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f475c101e10 0x7f475c1983f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f474c0097b0 tx=0x7f474c031b10 comp rx=0 tx=0).stop 2024-09-19T06:57:38.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.308+0000 7f4762a8d640 1 -- 172.21.15.103:0/229310513 shutdown_connections 2024-09-19T06:57:38.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.308+0000 7f4762a8d640 1 --2- 172.21.15.103:0/229310513 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47480a33b0 0x7f47480a5870 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.308+0000 7f4762a8d640 1 --2- 172.21.15.103:0/229310513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f475c106f40 0x7f475c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.308+0000 7f4762a8d640 1 --2- 172.21.15.103:0/229310513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f475c1027c0 0x7f475c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.308+0000 7f4762a8d640 1 --2- 172.21.15.103:0/229310513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f475c101e10 0x7f475c1983f0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:38.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.308+0000 7f4762a8d640 1 -- 172.21.15.103:0/229310513 >> 172.21.15.103:0/229310513 conn(0x7f475c0fd8d0 msgr2=0x7f475c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:38.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.308+0000 7f4762a8d640 1 -- 172.21.15.103:0/229310513 shutdown_connections 2024-09-19T06:57:38.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:38.308+0000 7f4762a8d640 1 -- 172.21.15.103:0/229310513 wait complete. 2024-09-19T06:57:38.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:38 smithi134 ceph-mon[35356]: pgmap v259: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-19T06:57:38.469 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.470 INFO:tasks.workunit.client.0.smithi103.stdout:189/600: open fd = 3 2024-09-19T06:57:38.470 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.470 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.470 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.470 INFO:tasks.workunit.client.0.smithi103.stdout:190/600: open fd = 3 2024-09-19T06:57:38.470 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.470 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.470 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.470 INFO:tasks.workunit.client.0.smithi103.stdout:191/600: open fd = 3 2024-09-19T06:57:38.470 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.470 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.470 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.471 INFO:tasks.workunit.client.0.smithi103.stdout:192/600: open fd = 3 2024-09-19T06:57:38.471 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.471 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.471 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.471 INFO:tasks.workunit.client.0.smithi103.stdout:193/600: open fd = 3 2024-09-19T06:57:38.471 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.471 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.471 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.472 INFO:tasks.workunit.client.0.smithi103.stdout:194/600: open fd = 3 2024-09-19T06:57:38.472 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.472 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.472 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.472 INFO:tasks.workunit.client.0.smithi103.stdout:195/600: open fd = 3 2024-09-19T06:57:38.472 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.472 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.472 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.472 INFO:tasks.workunit.client.0.smithi103.stdout:196/600: open fd = 3 2024-09-19T06:57:38.472 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.472 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.472 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.473 INFO:tasks.workunit.client.0.smithi103.stdout:197/600: open fd = 3 2024-09-19T06:57:38.473 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.473 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.473 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.473 INFO:tasks.workunit.client.0.smithi103.stdout:198/600: open fd = 3 2024-09-19T06:57:38.473 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.473 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.473 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.473 INFO:tasks.workunit.client.0.smithi103.stdout:199/600: open fd = 3 2024-09-19T06:57:38.473 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.473 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.473 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.473 INFO:tasks.workunit.client.0.smithi103.stdout:200/600: open fd = 3 2024-09-19T06:57:38.473 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.474 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.474 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.474 INFO:tasks.workunit.client.0.smithi103.stdout:201/600: open fd = 3 2024-09-19T06:57:38.474 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.474 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.474 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.474 INFO:tasks.workunit.client.0.smithi103.stdout:202/600: open fd = 3 2024-09-19T06:57:38.474 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.474 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.474 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.474 INFO:tasks.workunit.client.0.smithi103.stdout:203/600: open fd = 3 2024-09-19T06:57:38.474 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.474 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.475 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.475 INFO:tasks.workunit.client.0.smithi103.stdout:204/600: open fd = 3 2024-09-19T06:57:38.475 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.475 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.475 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.475 INFO:tasks.workunit.client.0.smithi103.stdout:205/600: open fd = 3 2024-09-19T06:57:38.475 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.475 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.475 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.475 INFO:tasks.workunit.client.0.smithi103.stdout:206/600: open fd = 3 2024-09-19T06:57:38.475 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.475 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.475 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.475 INFO:tasks.workunit.client.0.smithi103.stdout:207/600: open fd = 3 2024-09-19T06:57:38.476 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.476 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.476 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.476 INFO:tasks.workunit.client.0.smithi103.stdout:208/600: open fd = 3 2024-09-19T06:57:38.476 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.476 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.476 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.476 INFO:tasks.workunit.client.0.smithi103.stdout:209/600: open fd = 3 2024-09-19T06:57:38.476 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.476 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.476 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.476 INFO:tasks.workunit.client.0.smithi103.stdout:210/600: open fd = 3 2024-09-19T06:57:38.476 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.477 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.477 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.477 INFO:tasks.workunit.client.0.smithi103.stdout:211/600: open fd = 3 2024-09-19T06:57:38.477 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.477 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.477 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.477 INFO:tasks.workunit.client.0.smithi103.stdout:212/600: open fd = 3 2024-09-19T06:57:38.477 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.477 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.477 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.477 INFO:tasks.workunit.client.0.smithi103.stdout:213/600: open fd = 3 2024-09-19T06:57:38.477 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.477 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.477 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.478 INFO:tasks.workunit.client.0.smithi103.stdout:214/600: open fd = 3 2024-09-19T06:57:38.478 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.478 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.478 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.478 INFO:tasks.workunit.client.0.smithi103.stdout:215/600: open fd = 3 2024-09-19T06:57:38.478 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.478 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.478 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.478 INFO:tasks.workunit.client.0.smithi103.stdout:216/600: open fd = 3 2024-09-19T06:57:38.478 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.478 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.478 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.478 INFO:tasks.workunit.client.0.smithi103.stdout:217/600: open fd = 3 2024-09-19T06:57:38.478 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.479 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.479 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.479 INFO:tasks.workunit.client.0.smithi103.stdout:218/600: open fd = 3 2024-09-19T06:57:38.479 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.479 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.479 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.479 INFO:tasks.workunit.client.0.smithi103.stdout:219/600: open fd = 3 2024-09-19T06:57:38.479 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.479 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.479 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.479 INFO:tasks.workunit.client.0.smithi103.stdout:220/600: open fd = 3 2024-09-19T06:57:38.479 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.479 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.480 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.480 INFO:tasks.workunit.client.0.smithi103.stdout:221/600: open fd = 3 2024-09-19T06:57:38.480 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.480 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.480 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.480 INFO:tasks.workunit.client.0.smithi103.stdout:222/600: open fd = 3 2024-09-19T06:57:38.480 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.480 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.480 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.480 INFO:tasks.workunit.client.0.smithi103.stdout:223/600: open fd = 3 2024-09-19T06:57:38.480 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.480 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.480 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.480 INFO:tasks.workunit.client.0.smithi103.stdout:224/600: open fd = 3 2024-09-19T06:57:38.481 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.481 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.481 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.481 INFO:tasks.workunit.client.0.smithi103.stdout:225/600: open fd = 3 2024-09-19T06:57:38.481 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.481 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.481 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.481 INFO:tasks.workunit.client.0.smithi103.stdout:226/600: open fd = 3 2024-09-19T06:57:38.481 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.481 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.481 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.481 INFO:tasks.workunit.client.0.smithi103.stdout:227/600: open fd = 3 2024-09-19T06:57:38.481 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.481 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.482 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.482 INFO:tasks.workunit.client.0.smithi103.stdout:228/600: open fd = 3 2024-09-19T06:57:38.482 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.482 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.482 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.482 INFO:tasks.workunit.client.0.smithi103.stdout:229/600: open fd = 3 2024-09-19T06:57:38.482 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.482 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.482 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.482 INFO:tasks.workunit.client.0.smithi103.stdout:230/600: open fd = 3 2024-09-19T06:57:38.482 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.482 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.482 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.482 INFO:tasks.workunit.client.0.smithi103.stdout:231/600: open fd = 3 2024-09-19T06:57:38.483 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.483 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.483 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.483 INFO:tasks.workunit.client.0.smithi103.stdout:232/600: open fd = 3 2024-09-19T06:57:38.483 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.483 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.483 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.483 INFO:tasks.workunit.client.0.smithi103.stdout:233/600: open fd = 3 2024-09-19T06:57:38.483 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.483 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.483 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.483 INFO:tasks.workunit.client.0.smithi103.stdout:234/600: open fd = 3 2024-09-19T06:57:38.483 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.484 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.484 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.484 INFO:tasks.workunit.client.0.smithi103.stdout:235/600: open fd = 3 2024-09-19T06:57:38.484 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.484 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.484 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.484 INFO:tasks.workunit.client.0.smithi103.stdout:236/600: open fd = 3 2024-09-19T06:57:38.484 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.484 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.484 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.484 INFO:tasks.workunit.client.0.smithi103.stdout:237/600: open fd = 3 2024-09-19T06:57:38.484 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.485 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.485 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.485 INFO:tasks.workunit.client.0.smithi103.stdout:238/600: open fd = 3 2024-09-19T06:57:38.485 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.485 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.485 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.485 INFO:tasks.workunit.client.0.smithi103.stdout:239/600: open fd = 3 2024-09-19T06:57:38.485 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.485 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.485 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.485 INFO:tasks.workunit.client.0.smithi103.stdout:240/600: open fd = 3 2024-09-19T06:57:38.485 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.485 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.486 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.486 INFO:tasks.workunit.client.0.smithi103.stdout:241/600: open fd = 3 2024-09-19T06:57:38.486 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.486 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.486 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.486 INFO:tasks.workunit.client.0.smithi103.stdout:242/600: open fd = 3 2024-09-19T06:57:38.486 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.486 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.486 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.486 INFO:tasks.workunit.client.0.smithi103.stdout:243/600: open fd = 3 2024-09-19T06:57:38.486 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.486 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.486 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.487 INFO:tasks.workunit.client.0.smithi103.stdout:244/600: open fd = 3 2024-09-19T06:57:38.487 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.487 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.487 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.487 INFO:tasks.workunit.client.0.smithi103.stdout:245/600: open fd = 3 2024-09-19T06:57:38.487 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.487 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.487 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.487 INFO:tasks.workunit.client.0.smithi103.stdout:246/600: open fd = 3 2024-09-19T06:57:38.487 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.487 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.487 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.487 INFO:tasks.workunit.client.0.smithi103.stdout:247/600: open fd = 3 2024-09-19T06:57:38.488 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.488 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.488 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.488 INFO:tasks.workunit.client.0.smithi103.stdout:248/600: open fd = 3 2024-09-19T06:57:38.488 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.488 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.488 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.488 INFO:tasks.workunit.client.0.smithi103.stdout:249/600: open fd = 3 2024-09-19T06:57:38.488 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.488 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.488 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:57:38.488 INFO:tasks.workunit.client.0.smithi103.stdout:250/600: open fd = 3 2024-09-19T06:57:38.488 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.488 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:57:38.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:38 smithi174 ceph-mon[35637]: pgmap v259: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-19T06:57:38.966 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:57:39.216 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:39.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.637+0000 7f06a06a2640 1 -- 172.21.15.103:0/4243000949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0698100a80 msgr2=0x7f0698100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:39.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.637+0000 7f06a06a2640 1 --2- 172.21.15.103:0/4243000949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0698100a80 0x7f0698100ee0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0688009930 tx=0x7f068802a420 comp rx=0 tx=0).stop 2024-09-19T06:57:39.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.637+0000 7f06a06a2640 1 -- 172.21.15.103:0/4243000949 shutdown_connections 2024-09-19T06:57:39.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.637+0000 7f06a06a2640 1 --2- 172.21.15.103:0/4243000949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0698101420 0x7f069810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.637+0000 7f06a06a2640 1 --2- 172.21.15.103:0/4243000949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0698100a80 0x7f0698100ee0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.637+0000 7f06a06a2640 1 --2- 172.21.15.103:0/4243000949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0698106a80 0x7f0698106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.637+0000 7f06a06a2640 1 -- 172.21.15.103:0/4243000949 >> 172.21.15.103:0/4243000949 conn(0x7f06980fc7a0 msgr2=0x7f06980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:39.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.637+0000 7f06a06a2640 1 -- 172.21.15.103:0/4243000949 shutdown_connections 2024-09-19T06:57:39.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.637+0000 7f06a06a2640 1 -- 172.21.15.103:0/4243000949 wait complete. 2024-09-19T06:57:39.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f06a06a2640 1 Processor -- start 2024-09-19T06:57:39.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f06a06a2640 1 -- start start 2024-09-19T06:57:39.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f06a06a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0698100a80 0x7f06981992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f06a06a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0698101420 0x7f0698199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f06a06a2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0698106a80 0x7f06981a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f06a06a2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0698110ea0 con 0x7f0698100a80 2024-09-19T06:57:39.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f06a06a2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0698110d20 con 0x7f0698106a80 2024-09-19T06:57:39.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f06a06a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0698111020 con 0x7f0698101420 2024-09-19T06:57:39.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f069e43e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0698100a80 0x7f06981992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:39.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f069e43e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0698100a80 0x7f06981992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41492/0 (socket says 172.21.15.103:41492) 2024-09-19T06:57:39.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f069e43e640 1 -- 172.21.15.103:0/2310075828 learned_addr learned my addr 172.21.15.103:0/2310075828 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:39.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f069dc3d640 1 --2- 172.21.15.103:0/2310075828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0698101420 0x7f0698199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:39.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f069e43e640 1 -- 172.21.15.103:0/2310075828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0698101420 msgr2=0x7f0698199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:39.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f069e43e640 1 --2- 172.21.15.103:0/2310075828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0698101420 0x7f0698199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f069e43e640 1 -- 172.21.15.103:0/2310075828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0698106a80 msgr2=0x7f06981a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:39.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f069e43e640 1 --2- 172.21.15.103:0/2310075828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0698106a80 0x7f06981a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.638+0000 7f069e43e640 1 -- 172.21.15.103:0/2310075828 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06981a48b0 con 0x7f0698100a80 2024-09-19T06:57:39.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.639+0000 7f069e43e640 1 --2- 172.21.15.103:0/2310075828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0698100a80 0x7f06981992f0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f069400b4a0 tx=0x7f069400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:39.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.639+0000 7f068f7fe640 1 -- 172.21.15.103:0/2310075828 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694014070 con 0x7f0698100a80 2024-09-19T06:57:39.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.639+0000 7f068f7fe640 1 -- 172.21.15.103:0/2310075828 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0694010af0 con 0x7f0698100a80 2024-09-19T06:57:39.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.639+0000 7f068f7fe640 1 -- 172.21.15.103:0/2310075828 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694017d10 con 0x7f0698100a80 2024-09-19T06:57:39.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.639+0000 7f06a06a2640 1 -- 172.21.15.103:0/2310075828 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06981a4a80 con 0x7f0698100a80 2024-09-19T06:57:39.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.639+0000 7f06a06a2640 1 -- 172.21.15.103:0/2310075828 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06981a4f40 con 0x7f0698100a80 2024-09-19T06:57:39.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.641+0000 7f068f7fe640 1 -- 172.21.15.103:0/2310075828 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0694017070 con 0x7f0698100a80 2024-09-19T06:57:39.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.641+0000 7f06a2a7b740 1 Processor -- start 2024-09-19T06:57:39.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.641+0000 7f06a2a7b740 1 -- start start 2024-09-19T06:57:39.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f06a2a7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8467b40 0x5587e8466090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f06a2a7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e84665d0 0x5587e8466a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f06a2a7b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e8475780 0x5587e8477b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f06a2a7b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5587e83cc2f0 con 0x5587e8475780 2024-09-19T06:57:39.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f06a2a7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5587e8373cf0 con 0x5587e8467b40 2024-09-19T06:57:39.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f06a2a7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5587e83c82b0 con 0x5587e84665d0 2024-09-19T06:57:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f069dc3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e8475780 0x5587e8477b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f069dc3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e8475780 0x5587e8477b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41500/0 (socket says 172.21.15.103:41500) 2024-09-19T06:57:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f069dc3d640 1 -- 172.21.15.103:0/1118557939 learned_addr learned my addr 172.21.15.103:0/1118557939 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f069ec3f640 1 --2- 172.21.15.103:0/1118557939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8467b40 0x5587e8466090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f069e43e640 1 --2- 172.21.15.103:0/1118557939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e84665d0 0x5587e8466a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f068f7fe640 1 --2- 172.21.15.103:0/2310075828 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f06840a33b0 0x7f06840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f069dc3d640 1 -- 172.21.15.103:0/1118557939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e84665d0 msgr2=0x5587e8466a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f069dc3d640 1 --2- 172.21.15.103:0/1118557939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e84665d0 0x5587e8466a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f069dc3d640 1 -- 172.21.15.103:0/1118557939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8467b40 msgr2=0x5587e8466090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f069dc3d640 1 --2- 172.21.15.103:0/1118557939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8467b40 0x5587e8466090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f069dc3d640 1 -- 172.21.15.103:0/1118557939 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e83583a0 con 0x5587e8475780 2024-09-19T06:57:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f069e43e640 1 --2- 172.21.15.103:0/1118557939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e84665d0 0x5587e8466a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:39.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f069ec3f640 1 --2- 172.21.15.103:0/2310075828 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f06840a33b0 0x7f06840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f068f7fe640 1 -- 172.21.15.103:0/2310075828 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f06940cf0b0 con 0x7f0698100a80 2024-09-19T06:57:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.642+0000 7f069dc3d640 1 --2- 172.21.15.103:0/1118557939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e8475780 0x5587e8477b70 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f0688005ab0 tx=0x7f0688031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f29a9aff3e467b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.643+0000 7f068cff9640 1 -- 172.21.15.103:0/1118557939 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068803f070 con 0x5587e8475780 2024-09-19T06:57:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.643+0000 7f068cff9640 1 -- 172.21.15.103:0/1118557939 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0688031a20 con 0x5587e8475780 2024-09-19T06:57:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.643+0000 7f068cff9640 1 -- 172.21.15.103:0/1118557939 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688035070 con 0x5587e8475780 2024-09-19T06:57:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.643+0000 7f06a2a7b740 1 -- 172.21.15.103:0/1118557939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e8475780 msgr2=0x5587e8477b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.643+0000 7f069ec3f640 1 --2- 172.21.15.103:0/2310075828 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f06840a33b0 0x7f06840a5870 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0690005fd0 tx=0x7f0690004a40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.643+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/1118557939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e8475780 0x5587e8477b70 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f0688005ab0 tx=0x7f0688031630 comp rx=0 tx=0).stop 2024-09-19T06:57:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.643+0000 7f06a2a7b740 1 -- 172.21.15.103:0/1118557939 shutdown_connections 2024-09-19T06:57:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.643+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/1118557939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e8475780 0x5587e8477b70 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.643+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/1118557939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e84665d0 0x5587e8466a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.643+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/1118557939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8467b40 0x5587e8466090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.643+0000 7f06a2a7b740 1 -- 172.21.15.103:0/1118557939 >> 172.21.15.103:0/1118557939 conn(0x5587e842f310 msgr2=0x5587e8462c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:39.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.644+0000 7f06a2a7b740 1 -- 172.21.15.103:0/1118557939 shutdown_connections 2024-09-19T06:57:39.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.644+0000 7f06a2a7b740 1 -- 172.21.15.103:0/1118557939 wait complete. 2024-09-19T06:57:39.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.644+0000 7f06a2a7b740 1 -- start start 2024-09-19T06:57:39.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.644+0000 7f06a2a7b740 1 Processor -- start 2024-09-19T06:57:39.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.644+0000 7f06a2a7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e84665d0 0x5587e84750c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.644+0000 7f06a2a7b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e8467b40 0x5587e8472370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.644+0000 7f06a2a7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8475780 0x5587e84728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.644+0000 7f06a2a7b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5587e8478420 con 0x5587e8467b40 2024-09-19T06:57:39.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.644+0000 7f06a2a7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5587e8387060 con 0x5587e8475780 2024-09-19T06:57:39.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.644+0000 7f06a2a7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5587e84785a0 con 0x5587e84665d0 2024-09-19T06:57:39.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.644+0000 7f06a2a7b740 10 client.0 _ll_register_callbacks cb 0x5587e80da8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:39.681 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:57:39.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.644+0000 7f069dc3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e8467b40 0x5587e8472370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:39.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.644+0000 7f069dc3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e8467b40 0x5587e8472370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41502/0 (socket says 172.21.15.103:41502) 2024-09-19T06:57:39.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.644+0000 7f069dc3d640 1 -- 172.21.15.103:0/148258274 learned_addr learned my addr 172.21.15.103:0/148258274 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:39.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.644+0000 7f069ec3f640 1 --2- 172.21.15.103:0/148258274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8475780 0x5587e84728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:39.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.644+0000 7f069e43e640 1 --2- 172.21.15.103:0/148258274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e84665d0 0x5587e84750c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:39.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f069dc3d640 1 -- 172.21.15.103:0/148258274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e84665d0 msgr2=0x5587e84750c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:39.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f069dc3d640 1 --2- 172.21.15.103:0/148258274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e84665d0 0x5587e84750c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f069dc3d640 1 -- 172.21.15.103:0/148258274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8475780 msgr2=0x5587e84728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:39.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f069dc3d640 1 --2- 172.21.15.103:0/148258274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8475780 0x5587e84728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f069dc3d640 1 -- 172.21.15.103:0/148258274 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587e813d3c0 con 0x5587e8467b40 2024-09-19T06:57:39.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f069dc3d640 1 --2- 172.21.15.103:0/148258274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e8467b40 0x5587e8472370 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f0688009900 tx=0x7f0688004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4442416f22a403b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:39.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f0681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:39.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f0681ffb640 1 -- 172.21.15.103:0/148258274 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688035070 con 0x5587e8467b40 2024-09-19T06:57:39.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f0681ffb640 1 -- 172.21.15.103:0/148258274 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f0688004030 con 0x5587e8467b40 2024-09-19T06:57:39.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f06a2a7b740 1 -- 172.21.15.103:0/148258274 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0688031a20 con 0x5587e8467b40 2024-09-19T06:57:39.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f0681ffb640 1 -- 172.21.15.103:0/148258274 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0688055050 con 0x5587e8467b40 2024-09-19T06:57:39.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f06a2a7b740 10 client.18378 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:39.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f06a2a7b740 1 -- 172.21.15.103:0/148258274 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5587e8483e10 con 0x5587e8467b40 2024-09-19T06:57:39.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f0681ffb640 1 -- 172.21.15.103:0/148258274 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0688031a20 con 0x5587e8467b40 2024-09-19T06:57:39.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f0681ffb640 1 -- 172.21.15.103:0/148258274 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f068803b430 con 0x5587e8467b40 2024-09-19T06:57:39.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f06a2a7b740 10 client.18378 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:39.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f06a2a7b740 10 client.18378 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:57:39.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f06a2a7b740 20 client.18378 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:39.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.645+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/148258274 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5587e8485160 0x5587e8487550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.646+0000 7f06a2a7b740 4 client.18378 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:39.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.646+0000 7f06a2a7b740 1 -- 172.21.15.103:0/148258274 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587e83d9860 con 0x5587e8485160 2024-09-19T06:57:39.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.646+0000 7f069e43e640 1 --2- 172.21.15.103:0/148258274 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5587e8485160 0x5587e8487550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:39.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.646+0000 7f069e43e640 1 --2- 172.21.15.103:0/148258274 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5587e8485160 0x5587e8487550 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f45d326053bbbe30 server_cookie=feb177b16f904532 in_seq=0 out_seq=0 2024-09-19T06:57:39.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.646+0000 7f0681ffb640 10 client.18378 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:57:39.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.647+0000 7f0681ffb640 1 -- 172.21.15.103:0/148258274 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587e83d9860 con 0x5587e8485160 2024-09-19T06:57:39.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.647+0000 7f0681ffb640 10 client.18378 handle_command_reply: tid=0 2024-09-19T06:57:39.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.647+0000 7f06a2a7b740 1 client.18378 shutdown 2024-09-19T06:57:39.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.647+0000 7f06a2a7b740 1 -- 172.21.15.103:0/148258274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e8467b40 msgr2=0x5587e8472370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:39.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.647+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/148258274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e8467b40 0x5587e8472370 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f0688009900 tx=0x7f0688004a50 comp rx=0 tx=0).stop 2024-09-19T06:57:39.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.647+0000 7f06a2a7b740 1 -- 172.21.15.103:0/148258274 shutdown_connections 2024-09-19T06:57:39.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.647+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/148258274 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5587e8485160 0x5587e8487550 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.647+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/148258274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8475780 0x5587e84728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.647+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/148258274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e8467b40 0x5587e8472370 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.647+0000 7f0681ffb640 0 client.18378 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:57:39.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.647+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/148258274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e84665d0 0x5587e84750c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.647+0000 7f06a2a7b740 1 -- 172.21.15.103:0/148258274 >> 172.21.15.103:0/148258274 conn(0x5587e842f310 msgr2=0x5587e8468960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:39.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.647+0000 7f06a2a7b740 1 -- 172.21.15.103:0/148258274 shutdown_connections 2024-09-19T06:57:39.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.647+0000 7f06a2a7b740 1 -- 172.21.15.103:0/148258274 wait complete. 2024-09-19T06:57:39.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.647+0000 7f06a2a7b740 20 client.18378 trim_cache size 0 max 16384 2024-09-19T06:57:39.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f06a2a7b740 1 Processor -- start 2024-09-19T06:57:39.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f06a2a7b740 1 -- start start 2024-09-19T06:57:39.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f06a2a7b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e84665d0 0x5587e84816c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f06a2a7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e8467b40 0x5587e8462700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f06a2a7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8475780 0x5587e848caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f06a2a7b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06981ab650 con 0x5587e84665d0 2024-09-19T06:57:39.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f06a2a7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06981ad190 con 0x5587e8475780 2024-09-19T06:57:39.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f06a2a7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06981abd20 con 0x5587e8467b40 2024-09-19T06:57:39.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f069e43e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e84665d0 0x5587e84816c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:39.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f069e43e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e84665d0 0x5587e84816c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41506/0 (socket says 172.21.15.103:41506) 2024-09-19T06:57:39.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f069e43e640 1 -- 172.21.15.103:0/3725463363 learned_addr learned my addr 172.21.15.103:0/3725463363 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:39.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f069dc3d640 1 --2- 172.21.15.103:0/3725463363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e8467b40 0x5587e8462700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:39.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f069e43e640 1 -- 172.21.15.103:0/3725463363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e8467b40 msgr2=0x5587e8462700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:39.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f069e43e640 1 --2- 172.21.15.103:0/3725463363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e8467b40 0x5587e8462700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f069e43e640 1 -- 172.21.15.103:0/3725463363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8475780 msgr2=0x5587e848caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:39.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f069e43e640 1 --2- 172.21.15.103:0/3725463363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8475780 0x5587e848caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.660+0000 7f069e43e640 1 -- 172.21.15.103:0/3725463363 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e8463960 con 0x5587e84665d0 2024-09-19T06:57:39.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.661+0000 7f069e43e640 1 --2- 172.21.15.103:0/3725463363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e84665d0 0x5587e84816c0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f069400b470 tx=0x7f06940047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5f76dfb96dbbed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:39.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.661+0000 7f0683fff640 1 -- 172.21.15.103:0/3725463363 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069401e040 con 0x5587e84665d0 2024-09-19T06:57:39.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.661+0000 7f0683fff640 1 -- 172.21.15.103:0/3725463363 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f069409f830 con 0x5587e84665d0 2024-09-19T06:57:39.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.661+0000 7f0683fff640 1 -- 172.21.15.103:0/3725463363 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694097af0 con 0x5587e84665d0 2024-09-19T06:57:39.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.661+0000 7f06a2a7b740 1 -- 172.21.15.103:0/3725463363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e84665d0 msgr2=0x5587e84816c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:39.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.661+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/3725463363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e84665d0 0x5587e84816c0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f069400b470 tx=0x7f06940047d0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.661+0000 7f06a2a7b740 1 -- 172.21.15.103:0/3725463363 shutdown_connections 2024-09-19T06:57:39.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.661+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/3725463363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8475780 0x5587e848caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.661+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/3725463363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e8467b40 0x5587e8462700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.661+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/3725463363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e84665d0 0x5587e84816c0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.661+0000 7f06a2a7b740 1 -- 172.21.15.103:0/3725463363 >> 172.21.15.103:0/3725463363 conn(0x5587e842f310 msgr2=0x5587e8470c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.661+0000 7f06a2a7b740 1 -- 172.21.15.103:0/3725463363 shutdown_connections 2024-09-19T06:57:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.661+0000 7f06a2a7b740 1 -- 172.21.15.103:0/3725463363 wait complete. 2024-09-19T06:57:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f06a2a7b740 1 -- start start 2024-09-19T06:57:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f06a2a7b740 1 Processor -- start 2024-09-19T06:57:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f06a2a7b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e84665d0 0x5587e8479870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f06a2a7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8467b40 0x5587e8479db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f06a2a7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e8475780 0x5587e847a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f06a2a7b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06981acac0 con 0x5587e84665d0 2024-09-19T06:57:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f06a2a7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06981ac3f0 con 0x5587e8467b40 2024-09-19T06:57:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f06a2a7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06981a9650 con 0x5587e8475780 2024-09-19T06:57:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f06a2a7b740 10 client.0 _ll_register_callbacks cb 0x7f06900073c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f069e43e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e84665d0 0x5587e8479870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f069e43e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e84665d0 0x5587e8479870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41514/0 (socket says 172.21.15.103:41514) 2024-09-19T06:57:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f069e43e640 1 -- 172.21.15.103:0/888518118 learned_addr learned my addr 172.21.15.103:0/888518118 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f069e43e640 1 -- 172.21.15.103:0/888518118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e8475780 msgr2=0x5587e847a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f069e43e640 1 --2- 172.21.15.103:0/888518118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e8475780 0x5587e847a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f069e43e640 1 -- 172.21.15.103:0/888518118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8467b40 msgr2=0x5587e8479db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f069e43e640 1 --2- 172.21.15.103:0/888518118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8467b40 0x5587e8479db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f069e43e640 1 -- 172.21.15.103:0/888518118 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587e84eed90 con 0x5587e84665d0 2024-09-19T06:57:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f069e43e640 1 --2- 172.21.15.103:0/888518118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e84665d0 0x5587e8479870 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f069409b130 tx=0x7f06940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17de7c07bfc84c73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.662+0000 7f0681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.663+0000 7f0681ffb640 1 -- 172.21.15.103:0/888518118 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069401e070 con 0x5587e84665d0 2024-09-19T06:57:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.663+0000 7f0681ffb640 1 -- 172.21.15.103:0/888518118 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f0694010ca0 con 0x5587e84665d0 2024-09-19T06:57:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.663+0000 7f06a2a7b740 1 -- 172.21.15.103:0/888518118 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f069409f830 con 0x5587e84665d0 2024-09-19T06:57:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.663+0000 7f0681ffb640 1 -- 172.21.15.103:0/888518118 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0694009230 con 0x5587e84665d0 2024-09-19T06:57:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.663+0000 7f06a2a7b740 10 client.18384 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.663+0000 7f06a2a7b740 1 -- 172.21.15.103:0/888518118 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5587e84f0090 con 0x5587e84665d0 2024-09-19T06:57:39.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.663+0000 7f0681ffb640 1 -- 172.21.15.103:0/888518118 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f069409f830 con 0x5587e84665d0 2024-09-19T06:57:39.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.663+0000 7f0681ffb640 1 -- 172.21.15.103:0/888518118 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f06940d10b0 con 0x5587e84665d0 2024-09-19T06:57:39.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.663+0000 7f06a2a7b740 10 client.18384 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:39.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.663+0000 7f06a2a7b740 10 client.18384 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:57:39.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.663+0000 7f06a2a7b740 20 client.18384 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:39.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.663+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/888518118 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5587e84f07d0 0x5587e84f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:39.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.663+0000 7f06a2a7b740 4 client.18384 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:57:39.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.663+0000 7f06a2a7b740 1 -- 172.21.15.103:0/888518118 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587e83d3d70 con 0x5587e84f07d0 2024-09-19T06:57:39.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.663+0000 7f069dc3d640 1 --2- 172.21.15.103:0/888518118 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5587e84f07d0 0x5587e84f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:39.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f0681ffb640 10 client.18384 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:57:39.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f069dc3d640 1 --2- 172.21.15.103:0/888518118 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5587e84f07d0 0x5587e84f2c40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b7c08c0aa5711b3 server_cookie=9521018c7f57bca0 in_seq=0 out_seq=0 2024-09-19T06:57:39.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f0681ffb640 1 -- 172.21.15.103:0/888518118 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587e83d3d70 con 0x5587e84f07d0 2024-09-19T06:57:39.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f0681ffb640 10 client.18384 handle_command_reply: tid=0 2024-09-19T06:57:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f06a2a7b740 1 client.18384 shutdown 2024-09-19T06:57:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f06a2a7b740 1 -- 172.21.15.103:0/888518118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e84665d0 msgr2=0x5587e8479870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/888518118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e84665d0 0x5587e8479870 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f069409b130 tx=0x7f06940046c0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f06a2a7b740 1 -- 172.21.15.103:0/888518118 shutdown_connections 2024-09-19T06:57:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/888518118 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5587e84f07d0 0x5587e84f2c40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/888518118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587e8475780 0x5587e847a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f0681ffb640 0 client.18384 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:57:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/888518118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587e8467b40 0x5587e8479db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f06a2a7b740 1 --2- 172.21.15.103:0/888518118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587e84665d0 0x5587e8479870 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f06a2a7b740 1 -- 172.21.15.103:0/888518118 >> 172.21.15.103:0/888518118 conn(0x5587e842f310 msgr2=0x5587e84743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f06a2a7b740 1 -- 172.21.15.103:0/888518118 shutdown_connections 2024-09-19T06:57:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f06a2a7b740 1 -- 172.21.15.103:0/888518118 wait complete. 2024-09-19T06:57:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.664+0000 7f06a2a7b740 20 client.18384 trim_cache size 0 max 16384 2024-09-19T06:57:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.665+0000 7f06a06a2640 1 -- 172.21.15.103:0/2310075828 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f06840a33b0 msgr2=0x7f06840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.665+0000 7f06a06a2640 1 --2- 172.21.15.103:0/2310075828 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f06840a33b0 0x7f06840a5870 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0690005fd0 tx=0x7f0690004a40 comp rx=0 tx=0).stop 2024-09-19T06:57:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.665+0000 7f06a06a2640 1 -- 172.21.15.103:0/2310075828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0698100a80 msgr2=0x7f06981992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.665+0000 7f06a06a2640 1 --2- 172.21.15.103:0/2310075828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0698100a80 0x7f06981992f0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f069400b4a0 tx=0x7f069400b960 comp rx=0 tx=0).stop 2024-09-19T06:57:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.665+0000 7f06a06a2640 1 -- 172.21.15.103:0/2310075828 shutdown_connections 2024-09-19T06:57:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.665+0000 7f06a06a2640 1 --2- 172.21.15.103:0/2310075828 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f06840a33b0 0x7f06840a5870 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.665+0000 7f06a06a2640 1 --2- 172.21.15.103:0/2310075828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0698106a80 0x7f06981a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.665+0000 7f06a06a2640 1 --2- 172.21.15.103:0/2310075828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0698101420 0x7f0698199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.665+0000 7f06a06a2640 1 --2- 172.21.15.103:0/2310075828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0698100a80 0x7f06981992f0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.665+0000 7f06a06a2640 1 -- 172.21.15.103:0/2310075828 >> 172.21.15.103:0/2310075828 conn(0x7f06980fc7a0 msgr2=0x7f06980fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.665+0000 7f06a06a2640 1 -- 172.21.15.103:0/2310075828 shutdown_connections 2024-09-19T06:57:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:39.665+0000 7f06a06a2640 1 -- 172.21.15.103:0/2310075828 wait complete. 2024-09-19T06:57:40.358 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:57:40.381 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:40 smithi103 ceph-mon[31158]: pgmap v260: 97 pgs: 97 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-09-19T06:57:40.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:40 smithi134 ceph-mon[35356]: pgmap v260: 97 pgs: 97 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-09-19T06:57:40.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:40 smithi174 ceph-mon[35637]: pgmap v260: 97 pgs: 97 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-09-19T06:57:40.604 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:41.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.065+0000 7fc04b3d2640 1 -- 172.21.15.103:0/1355450673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc044100a80 msgr2=0x7fc044100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:41.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.065+0000 7fc04b3d2640 1 --2- 172.21.15.103:0/1355450673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc044100a80 0x7fc044100ee0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc040009930 tx=0x7fc04002a420 comp rx=0 tx=0).stop 2024-09-19T06:57:41.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.065+0000 7fc04b3d2640 1 -- 172.21.15.103:0/1355450673 shutdown_connections 2024-09-19T06:57:41.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.065+0000 7fc04b3d2640 1 --2- 172.21.15.103:0/1355450673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc044101420 0x7fc04410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.065+0000 7fc04b3d2640 1 --2- 172.21.15.103:0/1355450673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc044100a80 0x7fc044100ee0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.065+0000 7fc04b3d2640 1 --2- 172.21.15.103:0/1355450673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc044106a80 0x7fc044106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.065+0000 7fc04b3d2640 1 -- 172.21.15.103:0/1355450673 >> 172.21.15.103:0/1355450673 conn(0x7fc0440fc7a0 msgr2=0x7fc0440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:41.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.065+0000 7fc04b3d2640 1 -- 172.21.15.103:0/1355450673 shutdown_connections 2024-09-19T06:57:41.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.065+0000 7fc04b3d2640 1 -- 172.21.15.103:0/1355450673 wait complete. 2024-09-19T06:57:41.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.065+0000 7fc04b3d2640 1 Processor -- start 2024-09-19T06:57:41.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.065+0000 7fc04b3d2640 1 -- start start 2024-09-19T06:57:41.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04b3d2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc044100a80 0x7fc04410e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04b3d2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc044101420 0x7fc04410e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04b3d2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc044106a80 0x7fc04410ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04b3d2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc044112020 con 0x7fc044106a80 2024-09-19T06:57:41.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04b3d2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc044111ea0 con 0x7fc044101420 2024-09-19T06:57:41.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04b3d2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441121a0 con 0x7fc044100a80 2024-09-19T06:57:41.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04996f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc044106a80 0x7fc04410ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:41.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04996f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc044106a80 0x7fc04410ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41538/0 (socket says 172.21.15.103:41538) 2024-09-19T06:57:41.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04996f640 1 -- 172.21.15.103:0/3467891251 learned_addr learned my addr 172.21.15.103:0/3467891251 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:41.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04916e640 1 --2- 172.21.15.103:0/3467891251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc044100a80 0x7fc04410e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:41.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04896d640 1 --2- 172.21.15.103:0/3467891251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc044101420 0x7fc04410e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:41.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04996f640 1 -- 172.21.15.103:0/3467891251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc044100a80 msgr2=0x7fc04410e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:41.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04996f640 1 --2- 172.21.15.103:0/3467891251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc044100a80 0x7fc04410e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04996f640 1 -- 172.21.15.103:0/3467891251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc044101420 msgr2=0x7fc04410e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:41.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04996f640 1 --2- 172.21.15.103:0/3467891251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc044101420 0x7fc04410e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04996f640 1 -- 172.21.15.103:0/3467891251 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0441a6a80 con 0x7fc044106a80 2024-09-19T06:57:41.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04996f640 1 --2- 172.21.15.103:0/3467891251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc044106a80 0x7fc04410ee60 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00b970 tx=0x7fc03c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:41.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc0367fc640 1 -- 172.21.15.103:0/3467891251 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c019070 con 0x7fc044106a80 2024-09-19T06:57:41.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc0367fc640 1 -- 172.21.15.103:0/3467891251 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc03c00cb40 con 0x7fc044106a80 2024-09-19T06:57:41.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc0367fc640 1 -- 172.21.15.103:0/3467891251 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c007650 con 0x7fc044106a80 2024-09-19T06:57:41.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04b3d2640 1 -- 172.21.15.103:0/3467891251 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0441a6d70 con 0x7fc044106a80 2024-09-19T06:57:41.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.066+0000 7fc04b3d2640 1 -- 172.21.15.103:0/3467891251 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0441a7230 con 0x7fc044106a80 2024-09-19T06:57:41.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.068+0000 7fc0367fc640 1 -- 172.21.15.103:0/3467891251 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc03c01c070 con 0x7fc044106a80 2024-09-19T06:57:41.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.069+0000 7fc04d7ab740 1 Processor -- start 2024-09-19T06:57:41.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.069+0000 7fc04d7ab740 1 -- start start 2024-09-19T06:57:41.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.069+0000 7fc04d7ab740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557030895b40 0x557030894090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.069+0000 7fc04d7ab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570308945d0 0x557030894a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.069+0000 7fc0367fc640 1 --2- 172.21.15.103:0/3467891251 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0300a33b0 0x7fc0300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.069+0000 7fc04d7ab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55703089f370 0x5570308a1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.069+0000 7fc04d7ab740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5570307fa2f0 con 0x557030895b40 2024-09-19T06:57:41.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.069+0000 7fc04d7ab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5570307a1cf0 con 0x55703089f370 2024-09-19T06:57:41.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.069+0000 7fc04d7ab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5570307f62b0 con 0x5570308945d0 2024-09-19T06:57:41.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.069+0000 7fc04896d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557030895b40 0x557030894090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:41.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.069+0000 7fc04896d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557030895b40 0x557030894090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41546/0 (socket says 172.21.15.103:41546) 2024-09-19T06:57:41.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.069+0000 7fc04896d640 1 -- 172.21.15.103:0/4271652473 learned_addr learned my addr 172.21.15.103:0/4271652473 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:41.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.069+0000 7fc04916e640 1 --2- 172.21.15.103:0/4271652473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570308945d0 0x557030894a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:41.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.070+0000 7fc0367fc640 1 -- 172.21.15.103:0/3467891251 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7fc03c0cf2c0 con 0x7fc044106a80 2024-09-19T06:57:41.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.070+0000 7fc04896d640 1 --2- 172.21.15.103:0/3467891251 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0300a33b0 0x7fc0300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:41.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.070+0000 7fc04996f640 1 --2- 172.21.15.103:0/4271652473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55703089f370 0x5570308a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:41.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.070+0000 7fc04896d640 1 -- 172.21.15.103:0/4271652473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570308945d0 msgr2=0x557030894a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:41.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.070+0000 7fc04896d640 1 --2- 172.21.15.103:0/4271652473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570308945d0 0x557030894a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.070+0000 7fc04896d640 1 -- 172.21.15.103:0/4271652473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55703089f370 msgr2=0x5570308a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:41.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.070+0000 7fc04896d640 1 --2- 172.21.15.103:0/4271652473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55703089f370 0x5570308a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.070+0000 7fc04896d640 1 -- 172.21.15.103:0/4271652473 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570307863a0 con 0x557030895b40 2024-09-19T06:57:41.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.070+0000 7fc04916e640 1 --2- 172.21.15.103:0/4271652473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570308945d0 0x557030894a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:57:41.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.070+0000 7fc04896d640 1 --2- 172.21.15.103:0/4271652473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557030895b40 0x557030894090 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc040031550 tx=0x7fc040004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa27b97f8393e28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:41.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.070+0000 7fc04896d640 1 --2- 172.21.15.103:0/3467891251 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0300a33b0 0x7fc0300a5870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc04002a940 tx=0x7fc040046000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:41.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.070+0000 7fc02f7fe640 1 -- 172.21.15.103:0/4271652473 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040035040 con 0x557030895b40 2024-09-19T06:57:41.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.070+0000 7fc02f7fe640 1 -- 172.21.15.103:0/4271652473 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0400044b0 con 0x557030895b40 2024-09-19T06:57:41.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.070+0000 7fc02f7fe640 1 -- 172.21.15.103:0/4271652473 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0400047a0 con 0x557030895b40 2024-09-19T06:57:41.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.071+0000 7fc04d7ab740 1 -- 172.21.15.103:0/4271652473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557030895b40 msgr2=0x557030894090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:41.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.071+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/4271652473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557030895b40 0x557030894090 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc040031550 tx=0x7fc040004140 comp rx=0 tx=0).stop 2024-09-19T06:57:41.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.071+0000 7fc04d7ab740 1 -- 172.21.15.103:0/4271652473 shutdown_connections 2024-09-19T06:57:41.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.071+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/4271652473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55703089f370 0x5570308a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.071+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/4271652473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570308945d0 0x557030894a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.071+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/4271652473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557030895b40 0x557030894090 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.071+0000 7fc04d7ab740 1 -- 172.21.15.103:0/4271652473 >> 172.21.15.103:0/4271652473 conn(0x55703085d310 msgr2=0x557030890c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:41.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.071+0000 7fc04d7ab740 1 -- 172.21.15.103:0/4271652473 shutdown_connections 2024-09-19T06:57:41.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.071+0000 7fc04d7ab740 1 -- 172.21.15.103:0/4271652473 wait complete. 2024-09-19T06:57:41.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.071+0000 7fc04d7ab740 1 -- start start 2024-09-19T06:57:41.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.071+0000 7fc04d7ab740 1 Processor -- start 2024-09-19T06:57:41.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04d7ab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570308945d0 0x55703089ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04d7ab740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557030895b40 0x55703089c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04d7ab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55703089f370 0x55703089c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04d7ab740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5570308a1fb0 con 0x557030895b40 2024-09-19T06:57:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04d7ab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5570307b5060 con 0x5570308945d0 2024-09-19T06:57:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04d7ab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5570308a2130 con 0x55703089f370 2024-09-19T06:57:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04d7ab740 10 client.0 _ll_register_callbacks cb 0x5570305088c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04896d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557030895b40 0x55703089c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04896d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557030895b40 0x55703089c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41558/0 (socket says 172.21.15.103:41558) 2024-09-19T06:57:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04896d640 1 -- 172.21.15.103:0/2456622919 learned_addr learned my addr 172.21.15.103:0/2456622919 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04916e640 1 --2- 172.21.15.103:0/2456622919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570308945d0 0x55703089ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04996f640 1 --2- 172.21.15.103:0/2456622919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55703089f370 0x55703089c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04896d640 1 -- 172.21.15.103:0/2456622919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55703089f370 msgr2=0x55703089c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04896d640 1 --2- 172.21.15.103:0/2456622919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55703089f370 0x55703089c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04896d640 1 -- 172.21.15.103:0/2456622919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570308945d0 msgr2=0x55703089ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04896d640 1 --2- 172.21.15.103:0/2456622919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570308945d0 0x55703089ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04896d640 1 -- 172.21.15.103:0/2456622919 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55703056b3c0 con 0x557030895b40 2024-09-19T06:57:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc04896d640 1 --2- 172.21.15.103:0/2456622919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557030895b40 0x55703089c010 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fc04002a460 tx=0x7fc040032ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738182f4a9753a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.072+0000 7fc02cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.073+0000 7fc02cff9640 1 -- 172.21.15.103:0/2456622919 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040055020 con 0x557030895b40 2024-09-19T06:57:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.073+0000 7fc02cff9640 1 -- 172.21.15.103:0/2456622919 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7fc040048840 con 0x557030895b40 2024-09-19T06:57:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.073+0000 7fc04d7ab740 1 -- 172.21.15.103:0/2456622919 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0400044b0 con 0x557030895b40 2024-09-19T06:57:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.073+0000 7fc02cff9640 1 -- 172.21.15.103:0/2456622919 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc040035030 con 0x557030895b40 2024-09-19T06:57:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.073+0000 7fc04d7ab740 10 client.18396 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.073+0000 7fc04d7ab740 1 -- 172.21.15.103:0/2456622919 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570308ada50 con 0x557030895b40 2024-09-19T06:57:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.073+0000 7fc02cff9640 1 -- 172.21.15.103:0/2456622919 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0400044b0 con 0x557030895b40 2024-09-19T06:57:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.073+0000 7fc02cff9640 1 -- 172.21.15.103:0/2456622919 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc040047070 con 0x557030895b40 2024-09-19T06:57:41.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.073+0000 7fc04d7ab740 10 client.18396 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:41.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.073+0000 7fc04d7ab740 10 client.18396 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:57:41.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.073+0000 7fc04d7ab740 20 client.18396 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:41.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.073+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/2456622919 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x5570308aeda0 0x5570308b1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.073+0000 7fc04d7ab740 4 client.18396 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:41.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.073+0000 7fc04d7ab740 1 -- 172.21.15.103:0/2456622919 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557030807860 con 0x5570308aeda0 2024-09-19T06:57:41.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.073+0000 7fc04916e640 1 --2- 172.21.15.103:0/2456622919 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x5570308aeda0 0x5570308b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:41.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.074+0000 7fc04916e640 1 --2- 172.21.15.103:0/2456622919 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x5570308aeda0 0x5570308b1190 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4aa727451d6afc1e server_cookie=1ab8bd344238ebbd in_seq=0 out_seq=0 2024-09-19T06:57:41.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.074+0000 7fc02cff9640 10 client.18396 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:57:41.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.074+0000 7fc02cff9640 1 -- 172.21.15.103:0/2456622919 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557030807860 con 0x5570308aeda0 2024-09-19T06:57:41.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.074+0000 7fc02cff9640 10 client.18396 handle_command_reply: tid=0 2024-09-19T06:57:41.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.075+0000 7fc04d7ab740 1 client.18396 shutdown 2024-09-19T06:57:41.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.075+0000 7fc04d7ab740 1 -- 172.21.15.103:0/2456622919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557030895b40 msgr2=0x55703089c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:41.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.075+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/2456622919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557030895b40 0x55703089c010 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fc04002a460 tx=0x7fc040032ef0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.075+0000 7fc04d7ab740 1 -- 172.21.15.103:0/2456622919 shutdown_connections 2024-09-19T06:57:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.075+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/2456622919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55703089f370 0x55703089c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.075+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/2456622919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557030895b40 0x55703089c010 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.075+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/2456622919 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x5570308aeda0 0x5570308b1190 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.075+0000 7fc02cff9640 0 client.18396 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:57:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.075+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/2456622919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570308945d0 0x55703089ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.075+0000 7fc04d7ab740 1 -- 172.21.15.103:0/2456622919 >> 172.21.15.103:0/2456622919 conn(0x55703085d310 msgr2=0x557030896960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.075+0000 7fc04d7ab740 1 -- 172.21.15.103:0/2456622919 shutdown_connections 2024-09-19T06:57:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.075+0000 7fc04d7ab740 1 -- 172.21.15.103:0/2456622919 wait complete. 2024-09-19T06:57:41.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.075+0000 7fc04d7ab740 20 client.18396 trim_cache size 0 max 16384 2024-09-19T06:57:41.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04d7ab740 1 Processor -- start 2024-09-19T06:57:41.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04d7ab740 1 -- start start 2024-09-19T06:57:41.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04d7ab740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570308945d0 0x5570308ab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04d7ab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557030895b40 0x55703088fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04d7ab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55703089f370 0x5570308b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04d7ab740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441afb10 con 0x5570308945d0 2024-09-19T06:57:41.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04d7ab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441b1650 con 0x55703089f370 2024-09-19T06:57:41.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04d7ab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441b01e0 con 0x557030895b40 2024-09-19T06:57:41.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04916e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570308945d0 0x5570308ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:41.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04916e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570308945d0 0x5570308ab250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41560/0 (socket says 172.21.15.103:41560) 2024-09-19T06:57:41.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04916e640 1 -- 172.21.15.103:0/1991878275 learned_addr learned my addr 172.21.15.103:0/1991878275 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:41.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04916e640 1 -- 172.21.15.103:0/1991878275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557030895b40 msgr2=0x55703088fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:41.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04916e640 1 --2- 172.21.15.103:0/1991878275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557030895b40 0x55703088fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04916e640 1 -- 172.21.15.103:0/1991878275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55703089f370 msgr2=0x5570308b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:41.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04916e640 1 --2- 172.21.15.103:0/1991878275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55703089f370 0x5570308b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04916e640 1 -- 172.21.15.103:0/1991878275 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55703089c970 con 0x5570308945d0 2024-09-19T06:57:41.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.088+0000 7fc04916e640 1 --2- 172.21.15.103:0/1991878275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570308945d0 0x5570308ab250 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fc03800ba80 tx=0x7fc038008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16b5aaf18737cf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:41.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.089+0000 7fc02effd640 1 -- 172.21.15.103:0/1991878275 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038027070 con 0x5570308945d0 2024-09-19T06:57:41.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.089+0000 7fc02effd640 1 -- 172.21.15.103:0/1991878275 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc038022900 con 0x5570308945d0 2024-09-19T06:57:41.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.089+0000 7fc02effd640 1 -- 172.21.15.103:0/1991878275 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03801a760 con 0x5570308945d0 2024-09-19T06:57:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.089+0000 7fc04d7ab740 1 -- 172.21.15.103:0/1991878275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570308945d0 msgr2=0x5570308ab250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.089+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/1991878275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570308945d0 0x5570308ab250 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fc03800ba80 tx=0x7fc038008040 comp rx=0 tx=0).stop 2024-09-19T06:57:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.089+0000 7fc04d7ab740 1 -- 172.21.15.103:0/1991878275 shutdown_connections 2024-09-19T06:57:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.089+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/1991878275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55703089f370 0x5570308b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.089+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/1991878275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557030895b40 0x55703088fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.089+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/1991878275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570308945d0 0x5570308ab250 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.089+0000 7fc04d7ab740 1 -- 172.21.15.103:0/1991878275 >> 172.21.15.103:0/1991878275 conn(0x55703085d310 msgr2=0x55703088f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.089+0000 7fc04d7ab740 1 -- 172.21.15.103:0/1991878275 shutdown_connections 2024-09-19T06:57:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.089+0000 7fc04d7ab740 1 -- 172.21.15.103:0/1991878275 wait complete. 2024-09-19T06:57:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.090+0000 7fc04d7ab740 1 -- start start 2024-09-19T06:57:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.090+0000 7fc04d7ab740 1 Processor -- start 2024-09-19T06:57:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.090+0000 7fc04d7ab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570308945d0 0x5570308a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.090+0000 7fc04d7ab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557030895b40 0x5570308a4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.090+0000 7fc04d7ab740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55703089f370 0x5570308a7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.090+0000 7fc04d7ab740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441b0f80 con 0x55703089f370 2024-09-19T06:57:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.090+0000 7fc04d7ab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441b08b0 con 0x557030895b40 2024-09-19T06:57:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.090+0000 7fc04d7ab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441adb10 con 0x5570308945d0 2024-09-19T06:57:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.090+0000 7fc04d7ab740 10 client.0 _ll_register_callbacks cb 0x7fc03c097270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.090+0000 7fc04996f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55703089f370 0x5570308a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.090+0000 7fc04996f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55703089f370 0x5570308a7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41572/0 (socket says 172.21.15.103:41572) 2024-09-19T06:57:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.090+0000 7fc04996f640 1 -- 172.21.15.103:0/378359761 learned_addr learned my addr 172.21.15.103:0/378359761 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.090+0000 7fc04896d640 1 --2- 172.21.15.103:0/378359761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557030895b40 0x5570308a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc04916e640 1 --2- 172.21.15.103:0/378359761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570308945d0 0x5570308a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc04996f640 1 -- 172.21.15.103:0/378359761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570308945d0 msgr2=0x5570308a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc04996f640 1 --2- 172.21.15.103:0/378359761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570308945d0 0x5570308a3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc04996f640 1 -- 172.21.15.103:0/378359761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557030895b40 msgr2=0x5570308a4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc04996f640 1 --2- 172.21.15.103:0/378359761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557030895b40 0x5570308a4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc04996f640 1 -- 172.21.15.103:0/378359761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570308a77d0 con 0x55703089f370 2024-09-19T06:57:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc04896d640 1 --2- 172.21.15.103:0/378359761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557030895b40 0x5570308a4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc04996f640 1 --2- 172.21.15.103:0/378359761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55703089f370 0x5570308a7160 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fc03c098060 tx=0x7fc03c0b4ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4aa03ef192efef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc02cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc02cff9640 1 -- 172.21.15.103:0/378359761 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c09bd90 con 0x55703089f370 2024-09-19T06:57:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc02cff9640 1 -- 172.21.15.103:0/378359761 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7fc03c0b4c50 con 0x55703089f370 2024-09-19T06:57:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc04d7ab740 1 -- 172.21.15.103:0/378359761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc038022900 con 0x55703089f370 2024-09-19T06:57:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc02cff9640 1 -- 172.21.15.103:0/378359761 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc03c06d8b0 con 0x55703089f370 2024-09-19T06:57:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc04d7ab740 10 client.18402 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc04d7ab740 1 -- 172.21.15.103:0/378359761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55703091a1f0 con 0x55703089f370 2024-09-19T06:57:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc02cff9640 1 -- 172.21.15.103:0/378359761 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc038022900 con 0x55703089f370 2024-09-19T06:57:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc02cff9640 1 -- 172.21.15.103:0/378359761 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc03c0afbb0 con 0x55703089f370 2024-09-19T06:57:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc04d7ab740 10 client.18402 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc04d7ab740 10 client.18402 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:57:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.091+0000 7fc04d7ab740 20 client.18402 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.092+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/378359761 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55703091a910 0x55703091cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.092+0000 7fc04d7ab740 4 client.18402 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:57:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.092+0000 7fc04d7ab740 1 -- 172.21.15.103:0/378359761 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557030801d70 con 0x55703091a910 2024-09-19T06:57:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.092+0000 7fc04916e640 1 --2- 172.21.15.103:0/378359761 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55703091a910 0x55703091cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.092+0000 7fc02cff9640 10 client.18402 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:57:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.092+0000 7fc04916e640 1 --2- 172.21.15.103:0/378359761 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55703091a910 0x55703091cd00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=264ca4ea4a8c1439 server_cookie=3acbe40754a08047 in_seq=0 out_seq=0 2024-09-19T06:57:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.092+0000 7fc02cff9640 1 -- 172.21.15.103:0/378359761 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557030801d70 con 0x55703091a910 2024-09-19T06:57:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.092+0000 7fc02cff9640 10 client.18402 handle_command_reply: tid=0 2024-09-19T06:57:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.092+0000 7fc04d7ab740 1 client.18402 shutdown 2024-09-19T06:57:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.092+0000 7fc04d7ab740 1 -- 172.21.15.103:0/378359761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55703089f370 msgr2=0x5570308a7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.092+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/378359761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55703089f370 0x5570308a7160 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fc03c098060 tx=0x7fc03c0b4ba0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.092+0000 7fc04d7ab740 1 -- 172.21.15.103:0/378359761 shutdown_connections 2024-09-19T06:57:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.092+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/378359761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55703089f370 0x5570308a7160 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.092+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/378359761 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55703091a910 0x55703091cd00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.092+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/378359761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557030895b40 0x5570308a4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.092+0000 7fc04d7ab740 1 --2- 172.21.15.103:0/378359761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570308945d0 0x5570308a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc04d7ab740 1 -- 172.21.15.103:0/378359761 >> 172.21.15.103:0/378359761 conn(0x55703085d310 msgr2=0x55703088fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc02cff9640 0 client.18402 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:57:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc04d7ab740 1 -- 172.21.15.103:0/378359761 shutdown_connections 2024-09-19T06:57:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc04d7ab740 1 -- 172.21.15.103:0/378359761 wait complete. 2024-09-19T06:57:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc04d7ab740 20 client.18402 trim_cache size 0 max 16384 2024-09-19T06:57:41.101 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:57:41.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc04b3d2640 1 -- 172.21.15.103:0/3467891251 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0300a33b0 msgr2=0x7fc0300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:41.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc04b3d2640 1 --2- 172.21.15.103:0/3467891251 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0300a33b0 0x7fc0300a5870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc04002a940 tx=0x7fc040046000 comp rx=0 tx=0).stop 2024-09-19T06:57:41.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc04b3d2640 1 -- 172.21.15.103:0/3467891251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc044106a80 msgr2=0x7fc04410ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:41.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc04b3d2640 1 --2- 172.21.15.103:0/3467891251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc044106a80 0x7fc04410ee60 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00b970 tx=0x7fc03c00be30 comp rx=0 tx=0).stop 2024-09-19T06:57:41.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc04b3d2640 1 -- 172.21.15.103:0/3467891251 shutdown_connections 2024-09-19T06:57:41.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc04b3d2640 1 --2- 172.21.15.103:0/3467891251 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0300a33b0 0x7fc0300a5870 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc04b3d2640 1 --2- 172.21.15.103:0/3467891251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc044106a80 0x7fc04410ee60 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc04b3d2640 1 --2- 172.21.15.103:0/3467891251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc044101420 0x7fc04410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc04b3d2640 1 --2- 172.21.15.103:0/3467891251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc044100a80 0x7fc04410e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:41.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc04b3d2640 1 -- 172.21.15.103:0/3467891251 >> 172.21.15.103:0/3467891251 conn(0x7fc0440fc7a0 msgr2=0x7fc0440fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:41.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc04b3d2640 1 -- 172.21.15.103:0/3467891251 shutdown_connections 2024-09-19T06:57:41.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:41.093+0000 7fc04b3d2640 1 -- 172.21.15.103:0/3467891251 wait complete. 2024-09-19T06:57:41.893 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:57:42.181 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:42.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:42 smithi103 ceph-mon[31158]: pgmap v261: 97 pgs: 97 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-19T06:57:42.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:42 smithi134 ceph-mon[35356]: pgmap v261: 97 pgs: 97 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-19T06:57:42.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:42 smithi174 ceph-mon[35637]: pgmap v261: 97 pgs: 97 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-19T06:57:42.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.640+0000 7fcf2c388640 1 -- 172.21.15.103:0/3576098552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf24103660 msgr2=0x7fcf2410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:42.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.640+0000 7fcf2c388640 1 --2- 172.21.15.103:0/3576098552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf24103660 0x7fcf2410bb50 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009900 tx=0x7fcf1402a480 comp rx=0 tx=0).stop 2024-09-19T06:57:42.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.640+0000 7fcf2c388640 1 -- 172.21.15.103:0/3576098552 shutdown_connections 2024-09-19T06:57:42.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.640+0000 7fcf2c388640 1 --2- 172.21.15.103:0/3576098552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf2410c170 0x7fcf2410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.640+0000 7fcf2c388640 1 --2- 172.21.15.103:0/3576098552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf24103660 0x7fcf2410bb50 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.640+0000 7fcf2c388640 1 --2- 172.21.15.103:0/3576098552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf24102d40 0x7fcf24103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.640+0000 7fcf2c388640 1 -- 172.21.15.103:0/3576098552 >> 172.21.15.103:0/3576098552 conn(0x7fcf240fc7a0 msgr2=0x7fcf240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:42.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.640+0000 7fcf2c388640 1 -- 172.21.15.103:0/3576098552 shutdown_connections 2024-09-19T06:57:42.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.640+0000 7fcf2c388640 1 -- 172.21.15.103:0/3576098552 wait complete. 2024-09-19T06:57:42.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.640+0000 7fcf2c388640 1 Processor -- start 2024-09-19T06:57:42.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.640+0000 7fcf2c388640 1 -- start start 2024-09-19T06:57:42.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.640+0000 7fcf2c388640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf24102d40 0x7fcf24198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2c388640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf24103660 0x7fcf24198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2c388640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf2410c170 0x7fcf2419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2c388640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf24111f70 con 0x7fcf24102d40 2024-09-19T06:57:42.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2c388640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf24111df0 con 0x7fcf24103660 2024-09-19T06:57:42.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2c388640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf241120f0 con 0x7fcf2410c170 2024-09-19T06:57:42.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2a124640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf24102d40 0x7fcf24198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2a124640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf24102d40 0x7fcf24198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41592/0 (socket says 172.21.15.103:41592) 2024-09-19T06:57:42.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2a124640 1 -- 172.21.15.103:0/539172914 learned_addr learned my addr 172.21.15.103:0/539172914 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:42.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2a925640 1 --2- 172.21.15.103:0/539172914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf2410c170 0x7fcf2419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf29923640 1 --2- 172.21.15.103:0/539172914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf24103660 0x7fcf24198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2a124640 1 -- 172.21.15.103:0/539172914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf2410c170 msgr2=0x7fcf2419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:42.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2a124640 1 --2- 172.21.15.103:0/539172914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf2410c170 0x7fcf2419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2a124640 1 -- 172.21.15.103:0/539172914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf24103660 msgr2=0x7fcf24198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:42.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2a124640 1 --2- 172.21.15.103:0/539172914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf24103660 0x7fcf24198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2a124640 1 -- 172.21.15.103:0/539172914 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2419d3e0 con 0x7fcf24102d40 2024-09-19T06:57:42.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf29923640 1 --2- 172.21.15.103:0/539172914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf24103660 0x7fcf24198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:42.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2a124640 1 --2- 172.21.15.103:0/539172914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf24102d40 0x7fcf24198430 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000c920 tx=0x7fcf2000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:42.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf1b7fe640 1 -- 172.21.15.103:0/539172914 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf20007c70 con 0x7fcf24102d40 2024-09-19T06:57:42.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf1b7fe640 1 -- 172.21.15.103:0/539172914 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf20007e10 con 0x7fcf24102d40 2024-09-19T06:57:42.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf1b7fe640 1 -- 172.21.15.103:0/539172914 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf20018890 con 0x7fcf24102d40 2024-09-19T06:57:42.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2c388640 1 -- 172.21.15.103:0/539172914 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2419d6d0 con 0x7fcf24102d40 2024-09-19T06:57:42.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.641+0000 7fcf2c388640 1 -- 172.21.15.103:0/539172914 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf241a4fb0 con 0x7fcf24102d40 2024-09-19T06:57:42.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.644+0000 7fcf1b7fe640 1 -- 172.21.15.103:0/539172914 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fcf20018a70 con 0x7fcf24102d40 2024-09-19T06:57:42.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.644+0000 7fcf2e761740 1 Processor -- start 2024-09-19T06:57:42.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.644+0000 7fcf2e761740 1 -- start start 2024-09-19T06:57:42.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf2e761740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170bb5d0 0x55b7170bb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf2e761740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170b6280 0x55b7170b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf2e761740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170c2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf2e761740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b7170122f0 con 0x55b7170b6c00 2024-09-19T06:57:42.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf2e761740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b716fb9cf0 con 0x55b7170bb5d0 2024-09-19T06:57:42.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf2e761740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b71700e2b0 con 0x55b7170b6280 2024-09-19T06:57:42.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf1b7fe640 1 --2- 172.21.15.103:0/539172914 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcf100a33b0 0x7fcf100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf2a124640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170b6280 0x55b7170b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf2a124640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170b6280 0x55b7170b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:60100/0 (socket says 172.21.15.103:60100) 2024-09-19T06:57:42.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf2a124640 1 -- 172.21.15.103:0/1159482180 learned_addr learned my addr 172.21.15.103:0/1159482180 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:42.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf29923640 1 --2- 172.21.15.103:0/1159482180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf2a925640 1 --2- 172.21.15.103:0/1159482180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170bb5d0 0x55b7170bb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf2a925640 1 --2- 172.21.15.103:0/539172914 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcf100a33b0 0x7fcf100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf1b7fe640 1 -- 172.21.15.103:0/539172914 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7fcf200cff40 con 0x7fcf24102d40 2024-09-19T06:57:42.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf2a124640 1 -- 172.21.15.103:0/1159482180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170bb5d0 msgr2=0x55b7170bb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:42.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf2a124640 1 --2- 172.21.15.103:0/1159482180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170bb5d0 0x55b7170bb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf2a124640 1 -- 172.21.15.103:0/1159482180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 msgr2=0x55b7170c2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:42.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf2a124640 1 --2- 172.21.15.103:0/1159482180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170c2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf2a124640 1 -- 172.21.15.103:0/1159482180 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b716f9e3a0 con 0x55b7170b6280 2024-09-19T06:57:42.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.645+0000 7fcf29923640 1 --2- 172.21.15.103:0/1159482180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170c2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:57:42.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.646+0000 7fcf2a124640 1 --2- 172.21.15.103:0/1159482180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170b6280 0x55b7170b66c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf200cd5e0 tx=0x7fcf200985d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=443af217e2ed4de0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:42.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.646+0000 7fcf2a925640 1 --2- 172.21.15.103:0/539172914 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcf100a33b0 0x7fcf100a5870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c013ac0 tx=0x7fcf1c00b860 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:42.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.646+0000 7fcf18dea640 1 -- 172.21.15.103:0/1159482180 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2009c200 con 0x55b7170b6280 2024-09-19T06:57:42.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.646+0000 7fcf18dea640 1 -- 172.21.15.103:0/1159482180 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf200a05b0 con 0x55b7170b6280 2024-09-19T06:57:42.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.646+0000 7fcf18dea640 1 -- 172.21.15.103:0/1159482180 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf200cd8f0 con 0x55b7170b6280 2024-09-19T06:57:42.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 -- 172.21.15.103:0/1159482180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170b6280 msgr2=0x55b7170b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:42.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 --2- 172.21.15.103:0/1159482180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170b6280 0x55b7170b66c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf200cd5e0 tx=0x7fcf200985d0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 -- 172.21.15.103:0/1159482180 shutdown_connections 2024-09-19T06:57:42.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 --2- 172.21.15.103:0/1159482180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170c2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 --2- 172.21.15.103:0/1159482180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170b6280 0x55b7170b66c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 --2- 172.21.15.103:0/1159482180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170bb5d0 0x55b7170bb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 -- 172.21.15.103:0/1159482180 >> 172.21.15.103:0/1159482180 conn(0x55b717075310 msgr2=0x55b7170a86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:42.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 -- 172.21.15.103:0/1159482180 shutdown_connections 2024-09-19T06:57:42.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 -- 172.21.15.103:0/1159482180 wait complete. 2024-09-19T06:57:42.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 -- start start 2024-09-19T06:57:42.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 Processor -- start 2024-09-19T06:57:42.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170b6280 0x55b7170c2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170b6c00 0x55b7170bf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170bb5d0 0x55b7170bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b7170c28d0 con 0x55b7170bb5d0 2024-09-19T06:57:42.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b716fcd060 con 0x55b7170b6280 2024-09-19T06:57:42.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b7170c2a50 con 0x55b7170b6c00 2024-09-19T06:57:42.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.647+0000 7fcf2e761740 10 client.0 _ll_register_callbacks cb 0x55b716d208c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:42.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf2a925640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170bb5d0 0x55b7170bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf29923640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170b6c00 0x55b7170bf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf2a925640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170bb5d0 0x55b7170bfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41612/0 (socket says 172.21.15.103:41612) 2024-09-19T06:57:42.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf29923640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170b6c00 0x55b7170bf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:60110/0 (socket says 172.21.15.103:60110) 2024-09-19T06:57:42.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf2a925640 1 -- 172.21.15.103:0/4028101582 learned_addr learned my addr 172.21.15.103:0/4028101582 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:42.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf2a124640 1 --2- 172.21.15.103:0/4028101582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170b6280 0x55b7170c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf2a925640 1 -- 172.21.15.103:0/4028101582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170b6c00 msgr2=0x55b7170bf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:42.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf2a925640 1 --2- 172.21.15.103:0/4028101582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170b6c00 0x55b7170bf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf2a925640 1 -- 172.21.15.103:0/4028101582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170b6280 msgr2=0x55b7170c2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:42.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf2a925640 1 --2- 172.21.15.103:0/4028101582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170b6280 0x55b7170c2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf2a925640 1 -- 172.21.15.103:0/4028101582 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b716d833c0 con 0x55b7170bb5d0 2024-09-19T06:57:42.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf29923640 1 --2- 172.21.15.103:0/4028101582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170b6c00 0x55b7170bf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:57:42.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf2a925640 1 --2- 172.21.15.103:0/4028101582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170bb5d0 0x55b7170bfad0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00bf60 tx=0x7fcf1c00b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d4d4c2e26656f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:42.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:42.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf0dffb640 1 -- 172.21.15.103:0/4028101582 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c026070 con 0x55b7170bb5d0 2024-09-19T06:57:42.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf0dffb640 1 -- 172.21.15.103:0/4028101582 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7fcf1c004d20 con 0x55b7170bb5d0 2024-09-19T06:57:42.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.648+0000 7fcf2e761740 1 -- 172.21.15.103:0/4028101582 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf200a05b0 con 0x55b7170bb5d0 2024-09-19T06:57:42.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.649+0000 7fcf0dffb640 1 -- 172.21.15.103:0/4028101582 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcf1c030050 con 0x55b7170bb5d0 2024-09-19T06:57:42.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.649+0000 7fcf2e761740 10 client.18414 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:42.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.649+0000 7fcf2e761740 1 -- 172.21.15.103:0/4028101582 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b7170ce310 con 0x55b7170bb5d0 2024-09-19T06:57:42.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.649+0000 7fcf0dffb640 1 -- 172.21.15.103:0/4028101582 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf200a05b0 con 0x55b7170bb5d0 2024-09-19T06:57:42.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.649+0000 7fcf0dffb640 1 -- 172.21.15.103:0/4028101582 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcf1c01f630 con 0x55b7170bb5d0 2024-09-19T06:57:42.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.649+0000 7fcf2e761740 10 client.18414 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:42.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.649+0000 7fcf2e761740 10 client.18414 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:57:42.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.649+0000 7fcf2e761740 20 client.18414 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:42.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.649+0000 7fcf2e761740 1 --2- 172.21.15.103:0/4028101582 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55b7170cf7d0 0x55b7170d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.649+0000 7fcf2e761740 4 client.18414 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:42.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.649+0000 7fcf2e761740 1 -- 172.21.15.103:0/4028101582 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b71701f860 con 0x55b7170cf7d0 2024-09-19T06:57:42.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.649+0000 7fcf2a124640 1 --2- 172.21.15.103:0/4028101582 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55b7170cf7d0 0x55b7170d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.650+0000 7fcf2a124640 1 --2- 172.21.15.103:0/4028101582 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55b7170cf7d0 0x55b7170d1bf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c067439e75ab90 server_cookie=1b9e1d176497cd46 in_seq=0 out_seq=0 2024-09-19T06:57:42.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.650+0000 7fcf0dffb640 10 client.18414 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:57:42.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.650+0000 7fcf0dffb640 1 -- 172.21.15.103:0/4028101582 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b71701f860 con 0x55b7170cf7d0 2024-09-19T06:57:42.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.650+0000 7fcf0dffb640 10 client.18414 handle_command_reply: tid=0 2024-09-19T06:57:42.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.650+0000 7fcf2e761740 1 client.18414 shutdown 2024-09-19T06:57:42.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.651+0000 7fcf2e761740 1 -- 172.21.15.103:0/4028101582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170bb5d0 msgr2=0x55b7170bfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:42.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.651+0000 7fcf2e761740 1 --2- 172.21.15.103:0/4028101582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170bb5d0 0x55b7170bfad0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00bf60 tx=0x7fcf1c00b540 comp rx=0 tx=0).stop 2024-09-19T06:57:42.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.651+0000 7fcf2e761740 1 -- 172.21.15.103:0/4028101582 shutdown_connections 2024-09-19T06:57:42.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.651+0000 7fcf2e761740 1 --2- 172.21.15.103:0/4028101582 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55b7170cf7d0 0x55b7170d1bf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.651+0000 7fcf2e761740 1 --2- 172.21.15.103:0/4028101582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170bb5d0 0x55b7170bfad0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.651+0000 7fcf2e761740 1 --2- 172.21.15.103:0/4028101582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170b6c00 0x55b7170bf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.651+0000 7fcf2e761740 1 --2- 172.21.15.103:0/4028101582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170b6280 0x55b7170c2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.651+0000 7fcf2e761740 1 -- 172.21.15.103:0/4028101582 >> 172.21.15.103:0/4028101582 conn(0x55b717075310 msgr2=0x55b7170b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:42.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.651+0000 7fcf0dffb640 0 client.18414 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:57:42.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.651+0000 7fcf2e761740 1 -- 172.21.15.103:0/4028101582 shutdown_connections 2024-09-19T06:57:42.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.651+0000 7fcf2e761740 1 -- 172.21.15.103:0/4028101582 wait complete. 2024-09-19T06:57:42.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.651+0000 7fcf2e761740 20 client.18414 trim_cache size 0 max 16384 2024-09-19T06:57:42.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf2e761740 1 Processor -- start 2024-09-19T06:57:42.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf2e761740 1 -- start start 2024-09-19T06:57:42.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf2e761740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170b6280 0x55b7170cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf2e761740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170a84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf2e761740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170bb5d0 0x55b7170b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf2e761740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf241ab710 con 0x55b7170b6c00 2024-09-19T06:57:42.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf2e761740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf241ad250 con 0x55b7170b6280 2024-09-19T06:57:42.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf2e761740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf241abde0 con 0x55b7170bb5d0 2024-09-19T06:57:42.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf29923640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170a84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf29923640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170a84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41620/0 (socket says 172.21.15.103:41620) 2024-09-19T06:57:42.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf29923640 1 -- 172.21.15.103:0/1410017323 learned_addr learned my addr 172.21.15.103:0/1410017323 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:42.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf2a124640 1 --2- 172.21.15.103:0/1410017323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170b6280 0x55b7170cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf2a925640 1 --2- 172.21.15.103:0/1410017323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170bb5d0 0x55b7170b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf29923640 1 -- 172.21.15.103:0/1410017323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170bb5d0 msgr2=0x55b7170b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:42.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf29923640 1 --2- 172.21.15.103:0/1410017323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170bb5d0 0x55b7170b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf29923640 1 -- 172.21.15.103:0/1410017323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170b6280 msgr2=0x55b7170cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:42.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf29923640 1 --2- 172.21.15.103:0/1410017323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170b6280 0x55b7170cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf29923640 1 -- 172.21.15.103:0/1410017323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7170c25c0 con 0x55b7170b6c00 2024-09-19T06:57:42.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf2a925640 1 --2- 172.21.15.103:0/1410017323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170bb5d0 0x55b7170b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:42.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf29923640 1 --2- 172.21.15.103:0/1410017323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170a84f0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009230 tx=0x7fcf14031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781e61aec4c68d0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:42.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf0ffff640 1 -- 172.21.15.103:0/1410017323 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14035040 con 0x55b7170b6c00 2024-09-19T06:57:42.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf0ffff640 1 -- 172.21.15.103:0/1410017323 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf14005040 con 0x55b7170b6c00 2024-09-19T06:57:42.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.664+0000 7fcf0ffff640 1 -- 172.21.15.103:0/1410017323 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14004a60 con 0x55b7170b6c00 2024-09-19T06:57:42.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.665+0000 7fcf2e761740 1 -- 172.21.15.103:0/1410017323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 msgr2=0x55b7170a84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:42.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.665+0000 7fcf2e761740 1 --2- 172.21.15.103:0/1410017323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170a84f0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009230 tx=0x7fcf14031740 comp rx=0 tx=0).stop 2024-09-19T06:57:42.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.665+0000 7fcf2e761740 1 -- 172.21.15.103:0/1410017323 shutdown_connections 2024-09-19T06:57:42.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.665+0000 7fcf2e761740 1 --2- 172.21.15.103:0/1410017323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170bb5d0 0x55b7170b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.665+0000 7fcf2e761740 1 --2- 172.21.15.103:0/1410017323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170a84f0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.665+0000 7fcf2e761740 1 --2- 172.21.15.103:0/1410017323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170b6280 0x55b7170cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.665+0000 7fcf2e761740 1 -- 172.21.15.103:0/1410017323 >> 172.21.15.103:0/1410017323 conn(0x55b717075310 msgr2=0x55b7170bf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:42.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.665+0000 7fcf2e761740 1 -- 172.21.15.103:0/1410017323 shutdown_connections 2024-09-19T06:57:42.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.665+0000 7fcf2e761740 1 -- 172.21.15.103:0/1410017323 wait complete. 2024-09-19T06:57:42.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.665+0000 7fcf2e761740 1 -- start start 2024-09-19T06:57:42.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf2e761740 1 Processor -- start 2024-09-19T06:57:42.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf2e761740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170b6280 0x55b7170c8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf2e761740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170d6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf2e761740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170bb5d0 0x55b7170d7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf2e761740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf241acb80 con 0x55b7170b6c00 2024-09-19T06:57:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf2e761740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf241ac4b0 con 0x55b7170b6280 2024-09-19T06:57:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf2e761740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf241a9710 con 0x55b7170bb5d0 2024-09-19T06:57:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf2e761740 10 client.0 _ll_register_callbacks cb 0x7fcf1c010cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf29923640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170d6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf29923640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170d6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41636/0 (socket says 172.21.15.103:41636) 2024-09-19T06:57:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf29923640 1 -- 172.21.15.103:0/193144720 learned_addr learned my addr 172.21.15.103:0/193144720 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf2a124640 1 --2- 172.21.15.103:0/193144720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170b6280 0x55b7170c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf29923640 1 -- 172.21.15.103:0/193144720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170bb5d0 msgr2=0x55b7170d7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf29923640 1 --2- 172.21.15.103:0/193144720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170bb5d0 0x55b7170d7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf29923640 1 -- 172.21.15.103:0/193144720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170b6280 msgr2=0x55b7170c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf29923640 1 --2- 172.21.15.103:0/193144720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170b6280 0x55b7170c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf29923640 1 -- 172.21.15.103:0/193144720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7170d7d90 con 0x55b7170b6c00 2024-09-19T06:57:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf29923640 1 --2- 172.21.15.103:0/193144720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170d6f50 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fcf14031550 tx=0x7fcf140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45638dc48053d613 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.666+0000 7fcf0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.667+0000 7fcf0dffb640 1 -- 172.21.15.103:0/193144720 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14054060 con 0x55b7170b6c00 2024-09-19T06:57:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.667+0000 7fcf0dffb640 1 -- 172.21.15.103:0/193144720 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7fcf14004840 con 0x55b7170b6c00 2024-09-19T06:57:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.667+0000 7fcf2e761740 1 -- 172.21.15.103:0/193144720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf14005040 con 0x55b7170b6c00 2024-09-19T06:57:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.667+0000 7fcf0dffb640 1 -- 172.21.15.103:0/193144720 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcf14035030 con 0x55b7170b6c00 2024-09-19T06:57:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.667+0000 7fcf2e761740 10 client.18420 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.667+0000 7fcf2e761740 1 -- 172.21.15.103:0/193144720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b71713a610 con 0x55b7170b6c00 2024-09-19T06:57:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.667+0000 7fcf0dffb640 1 -- 172.21.15.103:0/193144720 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf14005040 con 0x55b7170b6c00 2024-09-19T06:57:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.667+0000 7fcf0dffb640 1 -- 172.21.15.103:0/193144720 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcf1404e070 con 0x55b7170b6c00 2024-09-19T06:57:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.667+0000 7fcf2e761740 10 client.18420 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.667+0000 7fcf2e761740 10 client.18420 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:57:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.667+0000 7fcf2e761740 20 client.18420 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.667+0000 7fcf2e761740 1 --2- 172.21.15.103:0/193144720 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55b71713acd0 0x55b71713d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.667+0000 7fcf2e761740 4 client.18420 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:57:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.667+0000 7fcf2e761740 1 -- 172.21.15.103:0/193144720 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b717019d70 con 0x55b71713acd0 2024-09-19T06:57:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.667+0000 7fcf2a124640 1 --2- 172.21.15.103:0/193144720 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55b71713acd0 0x55b71713d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf2a124640 1 --2- 172.21.15.103:0/193144720 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55b71713acd0 0x55b71713d0c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9a5bd46b9c9888 server_cookie=c599007ce97d34a9 in_seq=0 out_seq=0 2024-09-19T06:57:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf0dffb640 10 client.18420 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:57:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf0dffb640 1 -- 172.21.15.103:0/193144720 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b717019d70 con 0x55b71713acd0 2024-09-19T06:57:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf0dffb640 10 client.18420 handle_command_reply: tid=0 2024-09-19T06:57:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf2e761740 1 client.18420 shutdown 2024-09-19T06:57:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf2e761740 1 -- 172.21.15.103:0/193144720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 msgr2=0x55b7170d6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf2e761740 1 --2- 172.21.15.103:0/193144720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170d6f50 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fcf14031550 tx=0x7fcf140046c0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf2e761740 1 -- 172.21.15.103:0/193144720 shutdown_connections 2024-09-19T06:57:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf2e761740 1 --2- 172.21.15.103:0/193144720 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55b71713acd0 0x55b71713d0c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf2e761740 1 --2- 172.21.15.103:0/193144720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7170bb5d0 0x55b7170d7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf2e761740 1 --2- 172.21.15.103:0/193144720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7170b6c00 0x55b7170d6f50 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf0dffb640 0 client.18420 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:57:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf2e761740 1 --2- 172.21.15.103:0/193144720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b7170b6280 0x55b7170c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf2e761740 1 -- 172.21.15.103:0/193144720 >> 172.21.15.103:0/193144720 conn(0x55b717075310 msgr2=0x55b7170c0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf2e761740 1 -- 172.21.15.103:0/193144720 shutdown_connections 2024-09-19T06:57:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf2e761740 1 -- 172.21.15.103:0/193144720 wait complete. 2024-09-19T06:57:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.668+0000 7fcf2e761740 20 client.18420 trim_cache size 0 max 16384 2024-09-19T06:57:42.677 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:57:42.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.669+0000 7fcf2c388640 1 -- 172.21.15.103:0/539172914 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcf100a33b0 msgr2=0x7fcf100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:42.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.669+0000 7fcf2c388640 1 --2- 172.21.15.103:0/539172914 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcf100a33b0 0x7fcf100a5870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c013ac0 tx=0x7fcf1c00b860 comp rx=0 tx=0).stop 2024-09-19T06:57:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.669+0000 7fcf2c388640 1 -- 172.21.15.103:0/539172914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf24102d40 msgr2=0x7fcf24198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.669+0000 7fcf2c388640 1 --2- 172.21.15.103:0/539172914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf24102d40 0x7fcf24198430 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000c920 tx=0x7fcf2000cde0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.669+0000 7fcf2c388640 1 -- 172.21.15.103:0/539172914 shutdown_connections 2024-09-19T06:57:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.669+0000 7fcf2c388640 1 --2- 172.21.15.103:0/539172914 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcf100a33b0 0x7fcf100a5870 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.669+0000 7fcf2c388640 1 --2- 172.21.15.103:0/539172914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf2410c170 0x7fcf2419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.669+0000 7fcf2c388640 1 --2- 172.21.15.103:0/539172914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf24103660 0x7fcf24198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.669+0000 7fcf2c388640 1 --2- 172.21.15.103:0/539172914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf24102d40 0x7fcf24198430 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.669+0000 7fcf2c388640 1 -- 172.21.15.103:0/539172914 >> 172.21.15.103:0/539172914 conn(0x7fcf240fc7a0 msgr2=0x7fcf240fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.669+0000 7fcf2c388640 1 -- 172.21.15.103:0/539172914 shutdown_connections 2024-09-19T06:57:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:42.669+0000 7fcf2c388640 1 -- 172.21.15.103:0/539172914 wait complete. 2024-09-19T06:57:43.545 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:57:43.792 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:44.211 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:44 smithi103 ceph-mon[31158]: pgmap v262: 97 pgs: 97 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-19T06:57:44.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.217+0000 7f621db8d640 1 -- 172.21.15.103:0/3875229120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6218100e20 msgr2=0x7f6218101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.217+0000 7f621db8d640 1 --2- 172.21.15.103:0/3875229120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6218100e20 0x7f6218101200 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f6208009900 tx=0x7f620802a480 comp rx=0 tx=0).stop 2024-09-19T06:57:44.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.217+0000 7f621db8d640 1 -- 172.21.15.103:0/3875229120 shutdown_connections 2024-09-19T06:57:44.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.217+0000 7f621db8d640 1 --2- 172.21.15.103:0/3875229120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218102ef0 0x7f621810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.217+0000 7f621db8d640 1 --2- 172.21.15.103:0/3875229120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6218102570 0x7f62181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.217+0000 7f621db8d640 1 --2- 172.21.15.103:0/3875229120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6218100e20 0x7f6218101200 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.217+0000 7f621db8d640 1 -- 172.21.15.103:0/3875229120 >> 172.21.15.103:0/3875229120 conn(0x7f62180fc7d0 msgr2=0x7f62180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:44.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.217+0000 7f621db8d640 1 -- 172.21.15.103:0/3875229120 shutdown_connections 2024-09-19T06:57:44.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.217+0000 7f621db8d640 1 -- 172.21.15.103:0/3875229120 wait complete. 2024-09-19T06:57:44.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.217+0000 7f621db8d640 1 Processor -- start 2024-09-19T06:57:44.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f621db8d640 1 -- start start 2024-09-19T06:57:44.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f621db8d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6218100e20 0x7f6218197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f621db8d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218102570 0x7f6218197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f621db8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6218102ef0 0x7f621819be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f621db8d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6218111020 con 0x7f6218102ef0 2024-09-19T06:57:44.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f621db8d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6218110ea0 con 0x7f6218102570 2024-09-19T06:57:44.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f621db8d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62181111a0 con 0x7f6218100e20 2024-09-19T06:57:44.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f6217fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6218102ef0 0x7f621819be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f6217fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6218102ef0 0x7f621819be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41646/0 (socket says 172.21.15.103:41646) 2024-09-19T06:57:44.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f6217fff640 1 -- 172.21.15.103:0/3763105408 learned_addr learned my addr 172.21.15.103:0/3763105408 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:44.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f62177fe640 1 --2- 172.21.15.103:0/3763105408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6218100e20 0x7f6218197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f620ffff640 1 --2- 172.21.15.103:0/3763105408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218102570 0x7f6218197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f6217fff640 1 -- 172.21.15.103:0/3763105408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6218100e20 msgr2=0x7f6218197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f6217fff640 1 --2- 172.21.15.103:0/3763105408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6218100e20 0x7f6218197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f6217fff640 1 -- 172.21.15.103:0/3763105408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218102570 msgr2=0x7f6218197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f6217fff640 1 --2- 172.21.15.103:0/3763105408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218102570 0x7f6218197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f6217fff640 1 -- 172.21.15.103:0/3763105408 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621819c530 con 0x7f6218102ef0 2024-09-19T06:57:44.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f620ffff640 1 --2- 172.21.15.103:0/3763105408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218102570 0x7f6218197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:44.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.218+0000 7f6217fff640 1 --2- 172.21.15.103:0/3763105408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6218102ef0 0x7f621819be50 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f621000b970 tx=0x7f621000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:44.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.219+0000 7f62157fa640 1 -- 172.21.15.103:0/3763105408 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210019070 con 0x7f6218102ef0 2024-09-19T06:57:44.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.219+0000 7f62157fa640 1 -- 172.21.15.103:0/3763105408 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621000cb40 con 0x7f6218102ef0 2024-09-19T06:57:44.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.219+0000 7f621db8d640 1 -- 172.21.15.103:0/3763105408 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621819c820 con 0x7f6218102ef0 2024-09-19T06:57:44.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.219+0000 7f62157fa640 1 -- 172.21.15.103:0/3763105408 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210007650 con 0x7f6218102ef0 2024-09-19T06:57:44.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.219+0000 7f621db8d640 1 -- 172.21.15.103:0/3763105408 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62181a4100 con 0x7f6218102ef0 2024-09-19T06:57:44.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.221+0000 7f62157fa640 1 -- 172.21.15.103:0/3763105408 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f621001c070 con 0x7f6218102ef0 2024-09-19T06:57:44.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.221+0000 7f621ff66740 1 Processor -- start 2024-09-19T06:57:44.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.221+0000 7f621ff66740 1 -- start start 2024-09-19T06:57:44.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f621ff66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a158b40 0x55e35a157090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f621ff66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a1575d0 0x55e35a157a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f621ff66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a164580 0x55e35a166970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f621ff66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e35a0bd2f0 con 0x55e35a164580 2024-09-19T06:57:44.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f621ff66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e35a064cf0 con 0x55e35a1575d0 2024-09-19T06:57:44.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f621ff66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e35a0b92b0 con 0x55e35a158b40 2024-09-19T06:57:44.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f62157fa640 1 --2- 172.21.15.103:0/3763105408 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62040a33b0 0x7f62040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f620ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a164580 0x55e35a166970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f620ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a164580 0x55e35a166970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41660/0 (socket says 172.21.15.103:41660) 2024-09-19T06:57:44.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f620ffff640 1 -- 172.21.15.103:0/2045357492 learned_addr learned my addr 172.21.15.103:0/2045357492 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:44.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f620ffff640 1 --2- 172.21.15.103:0/2045357492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a158b40 0x55e35a157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f62177fe640 1 --2- 172.21.15.103:0/2045357492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a1575d0 0x55e35a157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f62157fa640 1 -- 172.21.15.103:0/3763105408 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f62100cf200 con 0x7f6218102ef0 2024-09-19T06:57:44.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f6217fff640 1 --2- 172.21.15.103:0/3763105408 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62040a33b0 0x7f62040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f620ffff640 1 -- 172.21.15.103:0/2045357492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a158b40 msgr2=0x55e35a157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:44.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f620ffff640 1 --2- 172.21.15.103:0/2045357492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a158b40 0x55e35a157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f620ffff640 1 -- 172.21.15.103:0/2045357492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a1575d0 msgr2=0x55e35a157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:44.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f620ffff640 1 --2- 172.21.15.103:0/2045357492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a1575d0 0x55e35a157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f620ffff640 1 -- 172.21.15.103:0/2045357492 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e35a0493a0 con 0x55e35a164580 2024-09-19T06:57:44.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f620ffff640 1 --2- 172.21.15.103:0/2045357492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a164580 0x55e35a166970 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f6200016560 tx=0x7f620000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d327a9a657db65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:44.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.222+0000 7f6217fff640 1 --2- 172.21.15.103:0/3763105408 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62040a33b0 0x7f62040a5870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f62100ccaf0 tx=0x7f6210004b50 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:44.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.223+0000 7f620dffb640 1 -- 172.21.15.103:0/2045357492 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200018430 con 0x55e35a164580 2024-09-19T06:57:44.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.223+0000 7f620dffb640 1 -- 172.21.15.103:0/2045357492 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6200018ab0 con 0x55e35a164580 2024-09-19T06:57:44.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.223+0000 7f620dffb640 1 -- 172.21.15.103:0/2045357492 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200017870 con 0x55e35a164580 2024-09-19T06:57:44.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.223+0000 7f621ff66740 1 -- 172.21.15.103:0/2045357492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a164580 msgr2=0x55e35a166970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.223+0000 7f621ff66740 1 --2- 172.21.15.103:0/2045357492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a164580 0x55e35a166970 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f6200016560 tx=0x7f620000a340 comp rx=0 tx=0).stop 2024-09-19T06:57:44.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.223+0000 7f621ff66740 1 -- 172.21.15.103:0/2045357492 shutdown_connections 2024-09-19T06:57:44.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.223+0000 7f621ff66740 1 --2- 172.21.15.103:0/2045357492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a164580 0x55e35a166970 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.223+0000 7f621ff66740 1 --2- 172.21.15.103:0/2045357492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a1575d0 0x55e35a157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.223+0000 7f621ff66740 1 --2- 172.21.15.103:0/2045357492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a158b40 0x55e35a157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.223+0000 7f621ff66740 1 -- 172.21.15.103:0/2045357492 >> 172.21.15.103:0/2045357492 conn(0x55e35a120310 msgr2=0x55e35a153c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:44.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.223+0000 7f621ff66740 1 -- 172.21.15.103:0/2045357492 shutdown_connections 2024-09-19T06:57:44.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.223+0000 7f621ff66740 1 -- 172.21.15.103:0/2045357492 wait complete. 2024-09-19T06:57:44.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.224+0000 7f621ff66740 1 -- start start 2024-09-19T06:57:44.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.224+0000 7f621ff66740 1 Processor -- start 2024-09-19T06:57:44.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.224+0000 7f621ff66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a1575d0 0x55e35a163f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.224+0000 7f621ff66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a158b40 0x55e35a161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.224+0000 7f621ff66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a164580 0x55e35a1617f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.224+0000 7f621ff66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e35a167220 con 0x55e35a164580 2024-09-19T06:57:44.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.224+0000 7f621ff66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e35a078060 con 0x55e35a158b40 2024-09-19T06:57:44.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.224+0000 7f621ff66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e35a1673a0 con 0x55e35a1575d0 2024-09-19T06:57:44.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.224+0000 7f621ff66740 10 client.0 _ll_register_callbacks cb 0x55e359dcb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:44.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.224+0000 7f620ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a164580 0x55e35a1617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.224+0000 7f620ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a164580 0x55e35a1617f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41664/0 (socket says 172.21.15.103:41664) 2024-09-19T06:57:44.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.224+0000 7f620ffff640 1 -- 172.21.15.103:0/722670312 learned_addr learned my addr 172.21.15.103:0/722670312 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:44.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.225+0000 7f620ffff640 1 --2- 172.21.15.103:0/722670312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a1575d0 0x55e35a163f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.225+0000 7f62177fe640 1 --2- 172.21.15.103:0/722670312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a158b40 0x55e35a161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.225+0000 7f620ffff640 1 -- 172.21.15.103:0/722670312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a1575d0 msgr2=0x55e35a163f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:44.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.225+0000 7f620ffff640 1 --2- 172.21.15.103:0/722670312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a1575d0 0x55e35a163f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.225+0000 7f620ffff640 1 -- 172.21.15.103:0/722670312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a158b40 msgr2=0x55e35a161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:44.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.225+0000 7f620ffff640 1 --2- 172.21.15.103:0/722670312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a158b40 0x55e35a161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.225+0000 7f620ffff640 1 -- 172.21.15.103:0/722670312 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e359e2e3c0 con 0x55e35a164580 2024-09-19T06:57:44.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.225+0000 7f620ffff640 1 --2- 172.21.15.103:0/722670312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a164580 0x55e35a1617f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f620000afd0 tx=0x7f6200017590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd802e9c9ef47f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:44.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.225+0000 7f61f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:44.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.225+0000 7f61f6ffd640 1 -- 172.21.15.103:0/722670312 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620002b060 con 0x55e35a164580 2024-09-19T06:57:44.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.225+0000 7f61f6ffd640 1 -- 172.21.15.103:0/722670312 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f6200018640 con 0x55e35a164580 2024-09-19T06:57:44.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.225+0000 7f621ff66740 1 -- 172.21.15.103:0/722670312 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6200018ab0 con 0x55e35a164580 2024-09-19T06:57:44.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.225+0000 7f61f6ffd640 1 -- 172.21.15.103:0/722670312 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6200018840 con 0x55e35a164580 2024-09-19T06:57:44.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.225+0000 7f621ff66740 10 client.18432 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:44.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.225+0000 7f621ff66740 1 -- 172.21.15.103:0/722670312 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e35a172c90 con 0x55e35a164580 2024-09-19T06:57:44.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.225+0000 7f61f6ffd640 1 -- 172.21.15.103:0/722670312 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6200018ab0 con 0x55e35a164580 2024-09-19T06:57:44.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.226+0000 7f61f6ffd640 1 -- 172.21.15.103:0/722670312 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6200012050 con 0x55e35a164580 2024-09-19T06:57:44.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.226+0000 7f621ff66740 10 client.18432 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:44.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.226+0000 7f621ff66740 10 client.18432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:44.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.226+0000 7f621ff66740 20 client.18432 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:44.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.226+0000 7f621ff66740 1 --2- 172.21.15.103:0/722670312 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e35a174090 0x55e35a176480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.226+0000 7f621ff66740 4 client.18432 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:44.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.226+0000 7f621ff66740 1 -- 172.21.15.103:0/722670312 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e35a0ca860 con 0x55e35a174090 2024-09-19T06:57:44.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.226+0000 7f62177fe640 1 --2- 172.21.15.103:0/722670312 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e35a174090 0x55e35a176480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.226+0000 7f61f6ffd640 10 client.18432 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:44.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.226+0000 7f62177fe640 1 --2- 172.21.15.103:0/722670312 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e35a174090 0x55e35a176480 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ede22046f9e56fab server_cookie=56d80f422a44d067 in_seq=0 out_seq=0 2024-09-19T06:57:44.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.227+0000 7f61f6ffd640 1 -- 172.21.15.103:0/722670312 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e35a0ca860 con 0x55e35a174090 2024-09-19T06:57:44.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.227+0000 7f61f6ffd640 10 client.18432 handle_command_reply: tid=0 2024-09-19T06:57:44.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.227+0000 7f621ff66740 1 client.18432 shutdown 2024-09-19T06:57:44.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.227+0000 7f621ff66740 1 -- 172.21.15.103:0/722670312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a164580 msgr2=0x55e35a1617f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.227+0000 7f621ff66740 1 --2- 172.21.15.103:0/722670312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a164580 0x55e35a1617f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f620000afd0 tx=0x7f6200017590 comp rx=0 tx=0).stop 2024-09-19T06:57:44.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.227+0000 7f621ff66740 1 -- 172.21.15.103:0/722670312 shutdown_connections 2024-09-19T06:57:44.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.227+0000 7f621ff66740 1 --2- 172.21.15.103:0/722670312 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e35a174090 0x55e35a176480 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.227+0000 7f621ff66740 1 --2- 172.21.15.103:0/722670312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a164580 0x55e35a1617f0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.227+0000 7f621ff66740 1 --2- 172.21.15.103:0/722670312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a158b40 0x55e35a161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.227+0000 7f621ff66740 1 --2- 172.21.15.103:0/722670312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a1575d0 0x55e35a163f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.227+0000 7f621ff66740 1 -- 172.21.15.103:0/722670312 >> 172.21.15.103:0/722670312 conn(0x55e35a120310 msgr2=0x55e35a159960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:44.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.227+0000 7f61f6ffd640 0 client.18432 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:44.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.227+0000 7f621ff66740 1 -- 172.21.15.103:0/722670312 shutdown_connections 2024-09-19T06:57:44.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.227+0000 7f621ff66740 1 -- 172.21.15.103:0/722670312 wait complete. 2024-09-19T06:57:44.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.228+0000 7f621ff66740 20 client.18432 trim_cache size 0 max 16384 2024-09-19T06:57:44.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.240+0000 7f621ff66740 1 Processor -- start 2024-09-19T06:57:44.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.240+0000 7f621ff66740 1 -- start start 2024-09-19T06:57:44.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f621ff66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a1575d0 0x55e35a160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f621ff66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a158b40 0x55e35a163170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f621ff66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a164580 0x55e35a1790c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f621ff66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62181ab850 con 0x55e35a158b40 2024-09-19T06:57:44.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f621ff66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62181ad390 con 0x55e35a1575d0 2024-09-19T06:57:44.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f621ff66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62181abf20 con 0x55e35a164580 2024-09-19T06:57:44.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f62177fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a158b40 0x55e35a163170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f62177fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a158b40 0x55e35a163170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41668/0 (socket says 172.21.15.103:41668) 2024-09-19T06:57:44.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f62177fe640 1 -- 172.21.15.103:0/3495859947 learned_addr learned my addr 172.21.15.103:0/3495859947 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:44.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f620ffff640 1 --2- 172.21.15.103:0/3495859947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a164580 0x55e35a1790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f620ffff640 1 --2- 172.21.15.103:0/3495859947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a1575d0 0x55e35a160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f62177fe640 1 -- 172.21.15.103:0/3495859947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a164580 msgr2=0x55e35a1790c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:44.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f62177fe640 1 --2- 172.21.15.103:0/3495859947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a164580 0x55e35a1790c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f62177fe640 1 -- 172.21.15.103:0/3495859947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a1575d0 msgr2=0x55e35a160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:44.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f62177fe640 1 --2- 172.21.15.103:0/3495859947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a1575d0 0x55e35a160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f62177fe640 1 -- 172.21.15.103:0/3495859947 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e35a158300 con 0x55e35a158b40 2024-09-19T06:57:44.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f620ffff640 1 --2- 172.21.15.103:0/3495859947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a164580 0x55e35a1790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:44.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f62177fe640 1 --2- 172.21.15.103:0/3495859947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a158b40 0x55e35a163170 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f6208009900 tx=0x7f620802aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0b94ffc89be6ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:44.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f620d7fa640 1 -- 172.21.15.103:0/3495859947 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208035040 con 0x55e35a158b40 2024-09-19T06:57:44.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f620d7fa640 1 -- 172.21.15.103:0/3495859947 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6208005500 con 0x55e35a158b40 2024-09-19T06:57:44.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.241+0000 7f620d7fa640 1 -- 172.21.15.103:0/3495859947 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208003850 con 0x55e35a158b40 2024-09-19T06:57:44.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.242+0000 7f621ff66740 1 -- 172.21.15.103:0/3495859947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a158b40 msgr2=0x55e35a163170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.242+0000 7f621ff66740 1 --2- 172.21.15.103:0/3495859947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a158b40 0x55e35a163170 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f6208009900 tx=0x7f620802aa20 comp rx=0 tx=0).stop 2024-09-19T06:57:44.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.242+0000 7f621ff66740 1 -- 172.21.15.103:0/3495859947 shutdown_connections 2024-09-19T06:57:44.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.242+0000 7f621ff66740 1 --2- 172.21.15.103:0/3495859947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a164580 0x55e35a1790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.242+0000 7f621ff66740 1 --2- 172.21.15.103:0/3495859947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a158b40 0x55e35a163170 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.242+0000 7f621ff66740 1 --2- 172.21.15.103:0/3495859947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a1575d0 0x55e35a160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.242+0000 7f621ff66740 1 -- 172.21.15.103:0/3495859947 >> 172.21.15.103:0/3495859947 conn(0x55e35a120310 msgr2=0x55e35a162a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:44.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.242+0000 7f621ff66740 1 -- 172.21.15.103:0/3495859947 shutdown_connections 2024-09-19T06:57:44.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.242+0000 7f621ff66740 1 -- 172.21.15.103:0/3495859947 wait complete. 2024-09-19T06:57:44.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.242+0000 7f621ff66740 1 -- start start 2024-09-19T06:57:44.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f621ff66740 1 Processor -- start 2024-09-19T06:57:44.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f621ff66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a1575d0 0x55e35a1761c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f621ff66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a158b40 0x55e35a176700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f621ff66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a164580 0x55e35a1dc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f621ff66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62181accc0 con 0x55e35a1575d0 2024-09-19T06:57:44.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f621ff66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62181ac5f0 con 0x55e35a158b40 2024-09-19T06:57:44.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f621ff66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62181a9850 con 0x55e35a164580 2024-09-19T06:57:44.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f621ff66740 10 client.0 _ll_register_callbacks cb 0x7f6208005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:44.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f620ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a1575d0 0x55e35a1761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f620ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a1575d0 0x55e35a1761c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41672/0 (socket says 172.21.15.103:41672) 2024-09-19T06:57:44.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f620ffff640 1 -- 172.21.15.103:0/2341693778 learned_addr learned my addr 172.21.15.103:0/2341693778 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:44.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f62177fe640 1 --2- 172.21.15.103:0/2341693778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a158b40 0x55e35a176700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f620ffff640 1 -- 172.21.15.103:0/2341693778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a164580 msgr2=0x55e35a1dc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:44.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f620ffff640 1 --2- 172.21.15.103:0/2341693778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a164580 0x55e35a1dc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f620ffff640 1 -- 172.21.15.103:0/2341693778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a158b40 msgr2=0x55e35a176700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:44.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f620ffff640 1 --2- 172.21.15.103:0/2341693778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a158b40 0x55e35a176700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f620ffff640 1 -- 172.21.15.103:0/2341693778 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e35a176fe0 con 0x55e35a1575d0 2024-09-19T06:57:44.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.243+0000 7f62177fe640 1 --2- 172.21.15.103:0/2341693778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a158b40 0x55e35a176700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:44.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.244+0000 7f620ffff640 1 --2- 172.21.15.103:0/2341693778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a1575d0 0x55e35a1761c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f6200001eb0 tx=0x7f6200007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c352277580fe53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:44.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.244+0000 7f61f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:44.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.244+0000 7f61f6ffd640 1 -- 172.21.15.103:0/2341693778 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620002b040 con 0x55e35a1575d0 2024-09-19T06:57:44.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.244+0000 7f61f6ffd640 1 -- 172.21.15.103:0/2341693778 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f62000059c0 con 0x55e35a1575d0 2024-09-19T06:57:44.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.244+0000 7f621ff66740 1 -- 172.21.15.103:0/2341693778 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6208005500 con 0x55e35a1575d0 2024-09-19T06:57:44.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.244+0000 7f61f6ffd640 1 -- 172.21.15.103:0/2341693778 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f620000a730 con 0x55e35a1575d0 2024-09-19T06:57:44.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.244+0000 7f621ff66740 10 client.18438 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:44.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.244+0000 7f621ff66740 1 -- 172.21.15.103:0/2341693778 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e35a1de010 con 0x55e35a1575d0 2024-09-19T06:57:44.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.244+0000 7f61f6ffd640 1 -- 172.21.15.103:0/2341693778 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6208005500 con 0x55e35a1575d0 2024-09-19T06:57:44.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.244+0000 7f61f6ffd640 1 -- 172.21.15.103:0/2341693778 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6200012040 con 0x55e35a1575d0 2024-09-19T06:57:44.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.244+0000 7f621ff66740 10 client.18438 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:44.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.244+0000 7f621ff66740 10 client.18438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:44.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.244+0000 7f621ff66740 20 client.18438 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:44.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.244+0000 7f621ff66740 1 --2- 172.21.15.103:0/2341693778 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e35a1de6a0 0x55e35a1e0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.244+0000 7f621ff66740 4 client.18438 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:57:44.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.244+0000 7f621ff66740 1 -- 172.21.15.103:0/2341693778 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e35a0c4d70 con 0x55e35a1de6a0 2024-09-19T06:57:44.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.245+0000 7f62177fe640 1 --2- 172.21.15.103:0/2341693778 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e35a1de6a0 0x55e35a1e0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.245+0000 7f61f6ffd640 10 client.18438 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:44.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.245+0000 7f62177fe640 1 --2- 172.21.15.103:0/2341693778 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e35a1de6a0 0x55e35a1e0ae0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c351e052db78682f server_cookie=22fd20aea4e56521 in_seq=0 out_seq=0 2024-09-19T06:57:44.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.245+0000 7f61f6ffd640 1 -- 172.21.15.103:0/2341693778 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e35a0c4d70 con 0x55e35a1de6a0 2024-09-19T06:57:44.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.245+0000 7f61f6ffd640 10 client.18438 handle_command_reply: tid=0 2024-09-19T06:57:44.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.245+0000 7f621ff66740 1 client.18438 shutdown 2024-09-19T06:57:44.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.245+0000 7f621ff66740 1 -- 172.21.15.103:0/2341693778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a1575d0 msgr2=0x55e35a1761c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.245+0000 7f621ff66740 1 --2- 172.21.15.103:0/2341693778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a1575d0 0x55e35a1761c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f6200001eb0 tx=0x7f6200007f50 comp rx=0 tx=0).stop 2024-09-19T06:57:44.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.245+0000 7f621ff66740 1 -- 172.21.15.103:0/2341693778 shutdown_connections 2024-09-19T06:57:44.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.245+0000 7f621ff66740 1 --2- 172.21.15.103:0/2341693778 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e35a1de6a0 0x55e35a1e0ae0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.245+0000 7f621ff66740 1 --2- 172.21.15.103:0/2341693778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e35a164580 0x55e35a1dc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.245+0000 7f621ff66740 1 --2- 172.21.15.103:0/2341693778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e35a158b40 0x55e35a176700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.245+0000 7f61f6ffd640 0 client.18438 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:44.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.245+0000 7f621ff66740 1 --2- 172.21.15.103:0/2341693778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e35a1575d0 0x55e35a1761c0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.245+0000 7f621ff66740 1 -- 172.21.15.103:0/2341693778 >> 172.21.15.103:0/2341693778 conn(0x55e35a120310 msgr2=0x55e35a163280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:44.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.245+0000 7f621ff66740 1 -- 172.21.15.103:0/2341693778 shutdown_connections 2024-09-19T06:57:44.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.246+0000 7f621ff66740 1 -- 172.21.15.103:0/2341693778 wait complete. 2024-09-19T06:57:44.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.246+0000 7f621ff66740 20 client.18438 trim_cache size 0 max 16384 2024-09-19T06:57:44.255 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:57:44.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.246+0000 7f621db8d640 1 -- 172.21.15.103:0/3763105408 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62040a33b0 msgr2=0x7f62040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.246+0000 7f621db8d640 1 --2- 172.21.15.103:0/3763105408 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62040a33b0 0x7f62040a5870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f62100ccaf0 tx=0x7f6210004b50 comp rx=0 tx=0).stop 2024-09-19T06:57:44.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.246+0000 7f621db8d640 1 -- 172.21.15.103:0/3763105408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6218102ef0 msgr2=0x7f621819be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.246+0000 7f621db8d640 1 --2- 172.21.15.103:0/3763105408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6218102ef0 0x7f621819be50 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f621000b970 tx=0x7f621000be30 comp rx=0 tx=0).stop 2024-09-19T06:57:44.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.246+0000 7f621db8d640 1 -- 172.21.15.103:0/3763105408 shutdown_connections 2024-09-19T06:57:44.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.246+0000 7f621db8d640 1 --2- 172.21.15.103:0/3763105408 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62040a33b0 0x7f62040a5870 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.246+0000 7f621db8d640 1 --2- 172.21.15.103:0/3763105408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6218102ef0 0x7f621819be50 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.246+0000 7f621db8d640 1 --2- 172.21.15.103:0/3763105408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218102570 0x7f6218197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.246+0000 7f621db8d640 1 --2- 172.21.15.103:0/3763105408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6218100e20 0x7f6218197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.246+0000 7f621db8d640 1 -- 172.21.15.103:0/3763105408 >> 172.21.15.103:0/3763105408 conn(0x7f62180fc7d0 msgr2=0x7f62180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:44.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.246+0000 7f621db8d640 1 -- 172.21.15.103:0/3763105408 shutdown_connections 2024-09-19T06:57:44.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:44.246+0000 7f621db8d640 1 -- 172.21.15.103:0/3763105408 wait complete. 2024-09-19T06:57:44.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:44 smithi134 ceph-mon[35356]: pgmap v262: 97 pgs: 97 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-19T06:57:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.551+0000 7f09ac95b640 1 -- 172.21.15.103:0/3741926516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a8101350 msgr2=0x7f09a810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.551+0000 7f09ac95b640 1 --2- 172.21.15.103:0/3741926516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a8101350 0x7f09a810d880 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009cd0 tx=0x7f09a002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.551+0000 7f09ac95b640 1 -- 172.21.15.103:0/3741926516 shutdown_connections 2024-09-19T06:57:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.551+0000 7f09ac95b640 1 --2- 172.21.15.103:0/3741926516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a8101350 0x7f09a810d880 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.551+0000 7f09ac95b640 1 --2- 172.21.15.103:0/3741926516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a81009b0 0x7f09a8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.551+0000 7f09ac95b640 1 --2- 172.21.15.103:0/3741926516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a81069c0 0x7f09a8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.551+0000 7f09ac95b640 1 -- 172.21.15.103:0/3741926516 >> 172.21.15.103:0/3741926516 conn(0x7f09a8077990 msgr2=0x7f09a80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.551+0000 7f09ac95b640 1 -- 172.21.15.103:0/3741926516 shutdown_connections 2024-09-19T06:57:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.551+0000 7f09ac95b640 1 -- 172.21.15.103:0/3741926516 wait complete. 2024-09-19T06:57:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09ac95b640 1 Processor -- start 2024-09-19T06:57:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09ac95b640 1 -- start start 2024-09-19T06:57:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09ac95b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a81009b0 0x7f09a8199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09ac95b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a8101350 0x7f09a8199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09ac95b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a81069c0 0x7f09a81a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09ac95b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8110e00 con 0x7f09a81009b0 2024-09-19T06:57:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09ac95b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8110c80 con 0x7f09a81069c0 2024-09-19T06:57:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09ac95b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8110f80 con 0x7f09a8101350 2024-09-19T06:57:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09a659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a81009b0 0x7f09a8199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09a659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a81009b0 0x7f09a8199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41686/0 (socket says 172.21.15.103:41686) 2024-09-19T06:57:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09a659c640 1 -- 172.21.15.103:0/608798266 learned_addr learned my addr 172.21.15.103:0/608798266 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09a5d9b640 1 --2- 172.21.15.103:0/608798266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a8101350 0x7f09a8199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09a6d9d640 1 --2- 172.21.15.103:0/608798266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a81069c0 0x7f09a81a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09a659c640 1 -- 172.21.15.103:0/608798266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a8101350 msgr2=0x7f09a8199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09a659c640 1 --2- 172.21.15.103:0/608798266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a8101350 0x7f09a8199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09a659c640 1 -- 172.21.15.103:0/608798266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a81069c0 msgr2=0x7f09a81a4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09a659c640 1 --2- 172.21.15.103:0/608798266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a81069c0 0x7f09a81a4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09a659c640 1 -- 172.21.15.103:0/608798266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a81a46c0 con 0x7f09a81009b0 2024-09-19T06:57:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.552+0000 7f09a6d9d640 1 --2- 172.21.15.103:0/608798266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a81069c0 0x7f09a81a4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.553+0000 7f09a659c640 1 --2- 172.21.15.103:0/608798266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a81009b0 0x7f09a8199100 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f099800d950 tx=0x7f099800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.553+0000 7f09977fe640 1 -- 172.21.15.103:0/608798266 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998017070 con 0x7f09a81009b0 2024-09-19T06:57:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.553+0000 7f09977fe640 1 -- 172.21.15.103:0/608798266 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0998009c30 con 0x7f09a81009b0 2024-09-19T06:57:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.553+0000 7f09ac95b640 1 -- 172.21.15.103:0/608798266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a81a4890 con 0x7f09a81009b0 2024-09-19T06:57:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.553+0000 7f09977fe640 1 -- 172.21.15.103:0/608798266 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099800f6b0 con 0x7f09a81009b0 2024-09-19T06:57:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.553+0000 7f09ac95b640 1 -- 172.21.15.103:0/608798266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a81a4db0 con 0x7f09a81009b0 2024-09-19T06:57:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.555+0000 7f09af7ce740 1 Processor -- start 2024-09-19T06:57:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.555+0000 7f09977fe640 1 -- 172.21.15.103:0/608798266 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f09980109d0 con 0x7f09a81009b0 2024-09-19T06:57:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.555+0000 7f09af7ce740 1 -- start start 2024-09-19T06:57:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09af7ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fabaf0 0x564a61fabed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09af7ce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fac410 0x564a61fb6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09af7ce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fb6cb0 0x564a61fb90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09af7ce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a61f102f0 con 0x564a61fac410 2024-09-19T06:57:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09af7ce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564a61eb8270 con 0x564a61fb6cb0 2024-09-19T06:57:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09af7ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564a61f0c2b0 con 0x564a61fabaf0 2024-09-19T06:57:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09a659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fac410 0x564a61fb6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09a659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fac410 0x564a61fb6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41696/0 (socket says 172.21.15.103:41696) 2024-09-19T06:57:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09a6d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fabaf0 0x564a61fabed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09a659c640 1 -- 172.21.15.103:0/29501202 learned_addr learned my addr 172.21.15.103:0/29501202 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09a5d9b640 1 --2- 172.21.15.103:0/29501202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fb6cb0 0x564a61fb90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09a659c640 1 -- 172.21.15.103:0/29501202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fabaf0 msgr2=0x564a61fabed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09a659c640 1 --2- 172.21.15.103:0/29501202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fabaf0 0x564a61fabed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09a659c640 1 -- 172.21.15.103:0/29501202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fb6cb0 msgr2=0x564a61fb90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09a659c640 1 --2- 172.21.15.103:0/29501202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fb6cb0 0x564a61fb90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09a659c640 1 -- 172.21.15.103:0/29501202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a61eb7340 con 0x564a61fac410 2024-09-19T06:57:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09977fe640 1 --2- 172.21.15.103:0/608798266 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f09900a3460 0x7f09900a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09a6d9d640 1 --2- 172.21.15.103:0/608798266 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f09900a3460 0x7f09900a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09a659c640 1 --2- 172.21.15.103:0/29501202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fac410 0x564a61fb6770 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f09980cd030 tx=0x7f09980d3f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44813803ed054860 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f09977fe640 1 -- 172.21.15.103:0/608798266 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f09980cf990 con 0x7f09a81009b0 2024-09-19T06:57:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.556+0000 7f0994ff9640 1 -- 172.21.15.103:0/29501202 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998020070 con 0x564a61fac410 2024-09-19T06:57:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.557+0000 7f0994ff9640 1 -- 172.21.15.103:0/29501202 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099809c350 con 0x564a61fac410 2024-09-19T06:57:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.557+0000 7f0994ff9640 1 -- 172.21.15.103:0/29501202 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09980da040 con 0x564a61fac410 2024-09-19T06:57:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.557+0000 7f09af7ce740 1 -- 172.21.15.103:0/29501202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fac410 msgr2=0x564a61fb6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.557+0000 7f09a6d9d640 1 --2- 172.21.15.103:0/608798266 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f09900a3460 0x7f09900a5920 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009a60 tx=0x7f09a0005d80 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.557+0000 7f09af7ce740 1 --2- 172.21.15.103:0/29501202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fac410 0x564a61fb6770 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f09980cd030 tx=0x7f09980d3f80 comp rx=0 tx=0).stop 2024-09-19T06:57:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.557+0000 7f09af7ce740 1 -- 172.21.15.103:0/29501202 shutdown_connections 2024-09-19T06:57:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.557+0000 7f09af7ce740 1 --2- 172.21.15.103:0/29501202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fb6cb0 0x564a61fb90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.557+0000 7f09af7ce740 1 --2- 172.21.15.103:0/29501202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fac410 0x564a61fb6770 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.557+0000 7f09af7ce740 1 --2- 172.21.15.103:0/29501202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fabaf0 0x564a61fabed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.557+0000 7f09af7ce740 1 -- 172.21.15.103:0/29501202 >> 172.21.15.103:0/29501202 conn(0x564a61fa8060 msgr2=0x564a61fa8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.557+0000 7f09af7ce740 1 -- 172.21.15.103:0/29501202 shutdown_connections 2024-09-19T06:57:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.557+0000 7f09af7ce740 1 -- 172.21.15.103:0/29501202 wait complete. 2024-09-19T06:57:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09af7ce740 1 -- start start 2024-09-19T06:57:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09af7ce740 1 Processor -- start 2024-09-19T06:57:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09af7ce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fabaf0 0x564a61fb3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09af7ce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fac410 0x564a61fb40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09af7ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fb6cb0 0x564a61fb4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09af7ce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a61fb9880 con 0x564a61fac410 2024-09-19T06:57:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09af7ce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564a61e99290 con 0x564a61fabaf0 2024-09-19T06:57:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09af7ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564a61fb9a00 con 0x564a61fb6cb0 2024-09-19T06:57:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09af7ce740 10 client.0 _ll_register_callbacks cb 0x564a61eca140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09a5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fac410 0x564a61fb40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09a5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fac410 0x564a61fb40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41706/0 (socket says 172.21.15.103:41706) 2024-09-19T06:57:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09a5d9b640 1 -- 172.21.15.103:0/3096462824 learned_addr learned my addr 172.21.15.103:0/3096462824 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09a659c640 1 --2- 172.21.15.103:0/3096462824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fabaf0 0x564a61fb3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09a6d9d640 1 --2- 172.21.15.103:0/3096462824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fb6cb0 0x564a61fb4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09a5d9b640 1 -- 172.21.15.103:0/3096462824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fb6cb0 msgr2=0x564a61fb4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09a5d9b640 1 --2- 172.21.15.103:0/3096462824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fb6cb0 0x564a61fb4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09a5d9b640 1 -- 172.21.15.103:0/3096462824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fabaf0 msgr2=0x564a61fb3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09a5d9b640 1 --2- 172.21.15.103:0/3096462824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fabaf0 0x564a61fb3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.558+0000 7f09a5d9b640 1 -- 172.21.15.103:0/3096462824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a61c23210 con 0x564a61fac410 2024-09-19T06:57:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.559+0000 7f09a5d9b640 1 --2- 172.21.15.103:0/3096462824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fac410 0x564a61fb40d0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f099c0107f0 tx=0x7f099c010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6598869c6ca2d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.559+0000 7f098dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.559+0000 7f098dffb640 1 -- 172.21.15.103:0/3096462824 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c00cb10 con 0x564a61fac410 2024-09-19T06:57:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.559+0000 7f098dffb640 1 -- 172.21.15.103:0/3096462824 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f099c00ccb0 con 0x564a61fac410 2024-09-19T06:57:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.559+0000 7f09af7ce740 1 -- 172.21.15.103:0/3096462824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f099809c350 con 0x564a61fac410 2024-09-19T06:57:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.559+0000 7f098dffb640 1 -- 172.21.15.103:0/3096462824 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f099c011440 con 0x564a61fac410 2024-09-19T06:57:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.559+0000 7f09af7ce740 10 client.18450 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.559+0000 7f09af7ce740 1 -- 172.21.15.103:0/3096462824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564a61cef620 con 0x564a61fac410 2024-09-19T06:57:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.559+0000 7f098dffb640 1 -- 172.21.15.103:0/3096462824 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099809c350 con 0x564a61fac410 2024-09-19T06:57:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.559+0000 7f098dffb640 1 -- 172.21.15.103:0/3096462824 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f099c0226f0 con 0x564a61fac410 2024-09-19T06:57:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.559+0000 7f09af7ce740 10 client.18450 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.559+0000 7f09af7ce740 10 client.18450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.559+0000 7f09af7ce740 20 client.18450 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.559+0000 7f09af7ce740 1 --2- 172.21.15.103:0/3096462824 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a61fc6c80 0x564a61fc90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.559+0000 7f09af7ce740 4 client.18450 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.559+0000 7f09af7ce740 1 -- 172.21.15.103:0/3096462824 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a61f1da20 con 0x564a61fc6c80 2024-09-19T06:57:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.560+0000 7f09a659c640 1 --2- 172.21.15.103:0/3096462824 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a61fc6c80 0x564a61fc90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.560+0000 7f09a659c640 1 --2- 172.21.15.103:0/3096462824 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a61fc6c80 0x564a61fc90d0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c3722ba1eb63a7a server_cookie=426b2c690ab5dc14 in_seq=0 out_seq=0 2024-09-19T06:57:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.560+0000 7f098dffb640 10 client.18450 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.561+0000 7f098dffb640 1 -- 172.21.15.103:0/3096462824 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a61f1da20 con 0x564a61fc6c80 2024-09-19T06:57:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.561+0000 7f098dffb640 10 client.18450 handle_command_reply: tid=0 2024-09-19T06:57:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.561+0000 7f09af7ce740 1 client.18450 shutdown 2024-09-19T06:57:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.561+0000 7f09af7ce740 1 -- 172.21.15.103:0/3096462824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fac410 msgr2=0x564a61fb40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.561+0000 7f09af7ce740 1 --2- 172.21.15.103:0/3096462824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fac410 0x564a61fb40d0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f099c0107f0 tx=0x7f099c010cb0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.561+0000 7f09af7ce740 1 -- 172.21.15.103:0/3096462824 shutdown_connections 2024-09-19T06:57:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.561+0000 7f09af7ce740 1 --2- 172.21.15.103:0/3096462824 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a61fc6c80 0x564a61fc90d0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.561+0000 7f09af7ce740 1 --2- 172.21.15.103:0/3096462824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fb6cb0 0x564a61fb4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.561+0000 7f09af7ce740 1 --2- 172.21.15.103:0/3096462824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fac410 0x564a61fb40d0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.561+0000 7f09af7ce740 1 --2- 172.21.15.103:0/3096462824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fabaf0 0x564a61fb3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.561+0000 7f098dffb640 0 client.18450 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.561+0000 7f09af7ce740 1 -- 172.21.15.103:0/3096462824 >> 172.21.15.103:0/3096462824 conn(0x564a61fa8060 msgr2=0x564a61fadd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.561+0000 7f09af7ce740 1 -- 172.21.15.103:0/3096462824 shutdown_connections 2024-09-19T06:57:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.561+0000 7f09af7ce740 1 -- 172.21.15.103:0/3096462824 wait complete. 2024-09-19T06:57:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.561+0000 7f09af7ce740 20 client.18450 trim_cache size 0 max 16384 2024-09-19T06:57:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.575+0000 7f09af7ce740 1 Processor -- start 2024-09-19T06:57:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.575+0000 7f09af7ce740 1 -- start start 2024-09-19T06:57:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.575+0000 7f09af7ce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fabaf0 0x564a61fb4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.575+0000 7f09af7ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fac410 0x564a61fce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.575+0000 7f09af7ce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fb6cb0 0x564a61fcef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.575+0000 7f09af7ce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81ad8a0 con 0x564a61fabaf0 2024-09-19T06:57:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.575+0000 7f09af7ce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81af3e0 con 0x564a61fb6cb0 2024-09-19T06:57:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.575+0000 7f09af7ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81adf70 con 0x564a61fac410 2024-09-19T06:57:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.576+0000 7f09a659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fabaf0 0x564a61fb4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.576+0000 7f09a659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fabaf0 0x564a61fb4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41708/0 (socket says 172.21.15.103:41708) 2024-09-19T06:57:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.576+0000 7f09a659c640 1 -- 172.21.15.103:0/2609166771 learned_addr learned my addr 172.21.15.103:0/2609166771 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.576+0000 7f09a6d9d640 1 --2- 172.21.15.103:0/2609166771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fb6cb0 0x564a61fcef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.576+0000 7f09a5d9b640 1 --2- 172.21.15.103:0/2609166771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fac410 0x564a61fce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.576+0000 7f09a659c640 1 -- 172.21.15.103:0/2609166771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fac410 msgr2=0x564a61fce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.576+0000 7f09a659c640 1 --2- 172.21.15.103:0/2609166771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fac410 0x564a61fce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.576+0000 7f09a659c640 1 -- 172.21.15.103:0/2609166771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fb6cb0 msgr2=0x564a61fcef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.576+0000 7f09a659c640 1 --2- 172.21.15.103:0/2609166771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fb6cb0 0x564a61fcef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.576+0000 7f09a659c640 1 -- 172.21.15.103:0/2609166771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a61fb2b20 con 0x564a61fabaf0 2024-09-19T06:57:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.576+0000 7f09a659c640 1 --2- 172.21.15.103:0/2609166771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fabaf0 0x564a61fb4440 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f099809b950 tx=0x7f0998000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a91ff1183ba5e86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.576+0000 7f098ffff640 1 -- 172.21.15.103:0/2609166771 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09980b1070 con 0x564a61fabaf0 2024-09-19T06:57:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.576+0000 7f098ffff640 1 -- 172.21.15.103:0/2609166771 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099809c350 con 0x564a61fabaf0 2024-09-19T06:57:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.576+0000 7f098ffff640 1 -- 172.21.15.103:0/2609166771 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09980da040 con 0x564a61fabaf0 2024-09-19T06:57:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 -- 172.21.15.103:0/2609166771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fabaf0 msgr2=0x564a61fb4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 --2- 172.21.15.103:0/2609166771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fabaf0 0x564a61fb4440 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f099809b950 tx=0x7f0998000bc0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 -- 172.21.15.103:0/2609166771 shutdown_connections 2024-09-19T06:57:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 --2- 172.21.15.103:0/2609166771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fb6cb0 0x564a61fcef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 --2- 172.21.15.103:0/2609166771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fac410 0x564a61fce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 --2- 172.21.15.103:0/2609166771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fabaf0 0x564a61fb4440 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 -- 172.21.15.103:0/2609166771 >> 172.21.15.103:0/2609166771 conn(0x564a61fa8060 msgr2=0x564a61fb5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 -- 172.21.15.103:0/2609166771 shutdown_connections 2024-09-19T06:57:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 -- 172.21.15.103:0/2609166771 wait complete. 2024-09-19T06:57:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 -- start start 2024-09-19T06:57:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 Processor -- start 2024-09-19T06:57:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fabaf0 0x564a61fbf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fac410 0x564a61fba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fb6cb0 0x564a61fbb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81aed10 con 0x564a61fb6cb0 2024-09-19T06:57:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81ae640 con 0x564a61fabaf0 2024-09-19T06:57:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81ab8a0 con 0x564a61fac410 2024-09-19T06:57:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.577+0000 7f09af7ce740 10 client.0 _ll_register_callbacks cb 0x7f099c00e3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f09a6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fb6cb0 0x564a61fbb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f09a6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fb6cb0 0x564a61fbb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41722/0 (socket says 172.21.15.103:41722) 2024-09-19T06:57:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f09a659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fabaf0 0x564a61fbf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f09a6d9d640 1 -- 172.21.15.103:0/800565215 learned_addr learned my addr 172.21.15.103:0/800565215 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f09a5d9b640 1 --2- 172.21.15.103:0/800565215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fac410 0x564a61fba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f09a6d9d640 1 -- 172.21.15.103:0/800565215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fac410 msgr2=0x564a61fba8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f09a6d9d640 1 --2- 172.21.15.103:0/800565215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fac410 0x564a61fba8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f09a6d9d640 1 -- 172.21.15.103:0/800565215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fabaf0 msgr2=0x564a61fbf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f09a6d9d640 1 --2- 172.21.15.103:0/800565215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fabaf0 0x564a61fbf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f09a6d9d640 1 -- 172.21.15.103:0/800565215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a620309b0 con 0x564a61fb6cb0 2024-09-19T06:57:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f09a6d9d640 1 --2- 172.21.15.103:0/800565215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fb6cb0 0x564a61fbb000 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f09a0031eb0 tx=0x7f09a00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c348a1589d9245e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f098dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f098dffb640 1 -- 172.21.15.103:0/800565215 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0003e30 con 0x564a61fb6cb0 2024-09-19T06:57:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f098dffb640 1 -- 172.21.15.103:0/800565215 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f09a0004650 con 0x564a61fb6cb0 2024-09-19T06:57:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f09af7ce740 1 -- 172.21.15.103:0/800565215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f099809c350 con 0x564a61fb6cb0 2024-09-19T06:57:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f098dffb640 1 -- 172.21.15.103:0/800565215 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f09a0004850 con 0x564a61fb6cb0 2024-09-19T06:57:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f09af7ce740 10 client.18456 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f09af7ce740 1 -- 172.21.15.103:0/800565215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564a62031b60 con 0x564a61fb6cb0 2024-09-19T06:57:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.578+0000 7f098dffb640 1 -- 172.21.15.103:0/800565215 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099809c350 con 0x564a61fb6cb0 2024-09-19T06:57:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.579+0000 7f098dffb640 1 -- 172.21.15.103:0/800565215 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f09a0030070 con 0x564a61fb6cb0 2024-09-19T06:57:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.579+0000 7f09af7ce740 10 client.18456 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.579+0000 7f09af7ce740 10 client.18456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.579+0000 7f09af7ce740 20 client.18456 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.579+0000 7f09af7ce740 1 --2- 172.21.15.103:0/800565215 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a620322e0 0x564a62034700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.579+0000 7f09af7ce740 4 client.18456 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:57:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.579+0000 7f09af7ce740 1 -- 172.21.15.103:0/800565215 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a61f17d70 con 0x564a620322e0 2024-09-19T06:57:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.579+0000 7f09a659c640 1 --2- 172.21.15.103:0/800565215 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a620322e0 0x564a62034700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.579+0000 7f09a659c640 1 --2- 172.21.15.103:0/800565215 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a620322e0 0x564a62034700 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f063ed829f85c27 server_cookie=e15d3ae6da1ab36e in_seq=0 out_seq=0 2024-09-19T06:57:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.579+0000 7f098dffb640 10 client.18456 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:44.588 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:44 smithi174 ceph-mon[35637]: pgmap v262: 97 pgs: 97 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-19T06:57:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.591+0000 7f098dffb640 1 -- 172.21.15.103:0/800565215 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a61f17d70 con 0x564a620322e0 2024-09-19T06:57:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.591+0000 7f098dffb640 10 client.18456 handle_command_reply: tid=0 2024-09-19T06:57:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.591+0000 7f09af7ce740 1 client.18456 shutdown 2024-09-19T06:57:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.591+0000 7f09af7ce740 1 -- 172.21.15.103:0/800565215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fb6cb0 msgr2=0x564a61fbb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.591+0000 7f09af7ce740 1 --2- 172.21.15.103:0/800565215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fb6cb0 0x564a61fbb000 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f09a0031eb0 tx=0x7f09a00095c0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.591+0000 7f09af7ce740 1 -- 172.21.15.103:0/800565215 shutdown_connections 2024-09-19T06:57:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.591+0000 7f09af7ce740 1 --2- 172.21.15.103:0/800565215 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a620322e0 0x564a62034700 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.591+0000 7f09af7ce740 1 --2- 172.21.15.103:0/800565215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a61fb6cb0 0x564a61fbb000 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.591+0000 7f098dffb640 0 client.18456 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.591+0000 7f09af7ce740 1 --2- 172.21.15.103:0/800565215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a61fac410 0x564a61fba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.591+0000 7f09af7ce740 1 --2- 172.21.15.103:0/800565215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a61fabaf0 0x564a61fbf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.591+0000 7f09af7ce740 1 -- 172.21.15.103:0/800565215 >> 172.21.15.103:0/800565215 conn(0x564a61fa8060 msgr2=0x564a61fa69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.591+0000 7f09af7ce740 1 -- 172.21.15.103:0/800565215 shutdown_connections 2024-09-19T06:57:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.591+0000 7f09af7ce740 1 -- 172.21.15.103:0/800565215 wait complete. 2024-09-19T06:57:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.591+0000 7f09af7ce740 20 client.18456 trim_cache size 0 max 16384 2024-09-19T06:57:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:57:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:57:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:57:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:57:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.592+0000 7f09ac95b640 1 -- 172.21.15.103:0/608798266 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f09900a3460 msgr2=0x7f09900a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.592+0000 7f09ac95b640 1 --2- 172.21.15.103:0/608798266 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f09900a3460 0x7f09900a5920 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009a60 tx=0x7f09a0005d80 comp rx=0 tx=0).stop 2024-09-19T06:57:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.592+0000 7f09ac95b640 1 -- 172.21.15.103:0/608798266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a81009b0 msgr2=0x7f09a8199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.592+0000 7f09ac95b640 1 --2- 172.21.15.103:0/608798266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a81009b0 0x7f09a8199100 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f099800d950 tx=0x7f099800de10 comp rx=0 tx=0).stop 2024-09-19T06:57:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.592+0000 7f09ac95b640 1 -- 172.21.15.103:0/608798266 shutdown_connections 2024-09-19T06:57:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.592+0000 7f09ac95b640 1 --2- 172.21.15.103:0/608798266 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f09900a3460 0x7f09900a5920 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.592+0000 7f09ac95b640 1 --2- 172.21.15.103:0/608798266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a81069c0 0x7f09a81a4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.592+0000 7f09ac95b640 1 --2- 172.21.15.103:0/608798266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a8101350 0x7f09a8199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.592+0000 7f09ac95b640 1 --2- 172.21.15.103:0/608798266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a81009b0 0x7f09a8199100 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.592+0000 7f09ac95b640 1 -- 172.21.15.103:0/608798266 >> 172.21.15.103:0/608798266 conn(0x7f09a8077990 msgr2=0x7f09a81035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.592+0000 7f09ac95b640 1 -- 172.21.15.103:0/608798266 shutdown_connections 2024-09-19T06:57:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:44.592+0000 7f09ac95b640 1 -- 172.21.15.103:0/608798266 wait complete. 2024-09-19T06:57:45.970 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:57:45.970 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:57:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:46.352 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:46 smithi103 ceph-mon[31158]: pgmap v263: 97 pgs: 97 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 62 KiB/s wr, 24 op/s 2024-09-19T06:57:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:46 smithi134 ceph-mon[35356]: pgmap v263: 97 pgs: 97 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 62 KiB/s wr, 24 op/s 2024-09-19T06:57:46.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:46 smithi174 ceph-mon[35637]: pgmap v263: 97 pgs: 97 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 62 KiB/s wr, 24 op/s 2024-09-19T06:57:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.687+0000 7f3cae8f5640 1 -- 172.21.15.103:0/893738909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ca8101de0 msgr2=0x7f3ca81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.687+0000 7f3cae8f5640 1 --2- 172.21.15.103:0/893738909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ca8101de0 0x7f3ca81021c0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009900 tx=0x7f3c9802a410 comp rx=0 tx=0).stop 2024-09-19T06:57:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.687+0000 7f3cae8f5640 1 -- 172.21.15.103:0/893738909 shutdown_connections 2024-09-19T06:57:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.687+0000 7f3cae8f5640 1 --2- 172.21.15.103:0/893738909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ca8106f10 0x7f3ca810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.687+0000 7f3cae8f5640 1 --2- 172.21.15.103:0/893738909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ca8102790 0x7f3ca81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.687+0000 7f3cae8f5640 1 --2- 172.21.15.103:0/893738909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ca8101de0 0x7f3ca81021c0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.687+0000 7f3cae8f5640 1 -- 172.21.15.103:0/893738909 >> 172.21.15.103:0/893738909 conn(0x7f3ca80fd8c0 msgr2=0x7f3ca80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:46.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.687+0000 7f3cae8f5640 1 -- 172.21.15.103:0/893738909 shutdown_connections 2024-09-19T06:57:46.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.687+0000 7f3cae8f5640 1 -- 172.21.15.103:0/893738909 wait complete. 2024-09-19T06:57:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.688+0000 7f3cae8f5640 1 Processor -- start 2024-09-19T06:57:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.688+0000 7f3cae8f5640 1 -- start start 2024-09-19T06:57:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.688+0000 7f3cae8f5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ca8101de0 0x7f3ca8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.688+0000 7f3cae8f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ca8102790 0x7f3ca8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.688+0000 7f3cae8f5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ca8106f10 0x7f3ca819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.688+0000 7f3cae8f5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca8111f70 con 0x7f3ca8102790 2024-09-19T06:57:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.688+0000 7f3cae8f5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca8111df0 con 0x7f3ca8101de0 2024-09-19T06:57:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.688+0000 7f3cae8f5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca81120f0 con 0x7f3ca8106f10 2024-09-19T06:57:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.688+0000 7f3ca77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ca8102790 0x7f3ca8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.688+0000 7f3ca77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ca8102790 0x7f3ca8198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41752/0 (socket says 172.21.15.103:41752) 2024-09-19T06:57:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.688+0000 7f3ca77fe640 1 -- 172.21.15.103:0/1534823685 learned_addr learned my addr 172.21.15.103:0/1534823685 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.688+0000 7f3cace92640 1 --2- 172.21.15.103:0/1534823685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ca8106f10 0x7f3ca819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.689+0000 7f3ca7fff640 1 --2- 172.21.15.103:0/1534823685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ca8101de0 0x7f3ca8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.689+0000 7f3ca77fe640 1 -- 172.21.15.103:0/1534823685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ca8106f10 msgr2=0x7f3ca819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.689+0000 7f3ca77fe640 1 --2- 172.21.15.103:0/1534823685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ca8106f10 0x7f3ca819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.689+0000 7f3ca77fe640 1 -- 172.21.15.103:0/1534823685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ca8101de0 msgr2=0x7f3ca8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.689+0000 7f3ca77fe640 1 --2- 172.21.15.103:0/1534823685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ca8101de0 0x7f3ca8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.689+0000 7f3ca77fe640 1 -- 172.21.15.103:0/1534823685 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca819d3e0 con 0x7f3ca8102790 2024-09-19T06:57:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.689+0000 7f3ca77fe640 1 --2- 172.21.15.103:0/1534823685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ca8102790 0x7f3ca8198970 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00d950 tx=0x7f3c9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.689+0000 7f3ca57fa640 1 -- 172.21.15.103:0/1534823685 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c017030 con 0x7f3ca8102790 2024-09-19T06:57:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.689+0000 7f3ca57fa640 1 -- 172.21.15.103:0/1534823685 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c9c004020 con 0x7f3ca8102790 2024-09-19T06:57:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.689+0000 7f3cae8f5640 1 -- 172.21.15.103:0/1534823685 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca819d6d0 con 0x7f3ca8102790 2024-09-19T06:57:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.689+0000 7f3ca57fa640 1 -- 172.21.15.103:0/1534823685 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c0047a0 con 0x7f3ca8102790 2024-09-19T06:57:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.689+0000 7f3cae8f5640 1 -- 172.21.15.103:0/1534823685 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca81a4fb0 con 0x7f3ca8102790 2024-09-19T06:57:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.692+0000 7f3ca57fa640 1 -- 172.21.15.103:0/1534823685 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3c9c0041d0 con 0x7f3ca8102790 2024-09-19T06:57:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.692+0000 7f3cb0cce740 1 Processor -- start 2024-09-19T06:57:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.692+0000 7f3cb0cce740 1 -- start start 2024-09-19T06:57:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.692+0000 7f3cb0cce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56391397ab60 0x56391397af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.692+0000 7f3cb0cce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397b480 0x563913985800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.692+0000 7f3cb0cce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563913985d40 0x563913988130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.692+0000 7f3cb0cce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5639138e0280 con 0x56391397b480 2024-09-19T06:57:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.692+0000 7f3cb0cce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563913887b10 con 0x56391397ab60 2024-09-19T06:57:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.692+0000 7f3cb0cce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5639138dc240 con 0x563913985d40 2024-09-19T06:57:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.692+0000 7f3ca57fa640 1 --2- 172.21.15.103:0/1534823685 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c900a33b0 0x7f3c900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.692+0000 7f3ca7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397b480 0x563913985800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.692+0000 7f3ca7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397b480 0x563913985800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41766/0 (socket says 172.21.15.103:41766) 2024-09-19T06:57:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.692+0000 7f3ca7fff640 1 -- 172.21.15.103:0/1885764252 learned_addr learned my addr 172.21.15.103:0/1885764252 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.692+0000 7f3cace92640 1 --2- 172.21.15.103:0/1534823685 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c900a33b0 0x7f3c900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.693+0000 7f3cace92640 1 --2- 172.21.15.103:0/1885764252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56391397ab60 0x56391397af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.693+0000 7f3ca57fa640 1 -- 172.21.15.103:0/1534823685 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f3c9c0cf740 con 0x7f3ca8102790 2024-09-19T06:57:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.693+0000 7f3ca77fe640 1 --2- 172.21.15.103:0/1885764252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563913985d40 0x563913988130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.693+0000 7f3ca7fff640 1 -- 172.21.15.103:0/1885764252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563913985d40 msgr2=0x563913988130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.693+0000 7f3ca7fff640 1 --2- 172.21.15.103:0/1885764252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563913985d40 0x563913988130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.693+0000 7f3ca7fff640 1 -- 172.21.15.103:0/1885764252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56391397ab60 msgr2=0x56391397af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.693+0000 7f3ca7fff640 1 --2- 172.21.15.103:0/1885764252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56391397ab60 0x56391397af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.693+0000 7f3ca7fff640 1 -- 172.21.15.103:0/1885764252 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639138941f0 con 0x56391397b480 2024-09-19T06:57:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.693+0000 7f3ca7fff640 1 --2- 172.21.15.103:0/1885764252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397b480 0x563913985800 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009900 tx=0x7f3c98005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d00e90d99bd6c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.693+0000 7f3cace92640 1 --2- 172.21.15.103:0/1534823685 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c900a33b0 0x7f3c900a5870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009fd0 tx=0x7f3ca0004ab0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.693+0000 7f3c8e7fc640 1 -- 172.21.15.103:0/1885764252 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9803f070 con 0x56391397b480 2024-09-19T06:57:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.693+0000 7f3c8e7fc640 1 -- 172.21.15.103:0/1885764252 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c98004030 con 0x56391397b480 2024-09-19T06:57:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.693+0000 7f3c8e7fc640 1 -- 172.21.15.103:0/1885764252 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98035070 con 0x56391397b480 2024-09-19T06:57:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.694+0000 7f3cb0cce740 1 -- 172.21.15.103:0/1885764252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397b480 msgr2=0x563913985800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.694+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/1885764252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397b480 0x563913985800 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009900 tx=0x7f3c98005e80 comp rx=0 tx=0).stop 2024-09-19T06:57:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.694+0000 7f3cb0cce740 1 -- 172.21.15.103:0/1885764252 shutdown_connections 2024-09-19T06:57:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.694+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/1885764252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563913985d40 0x563913988130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.694+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/1885764252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397b480 0x563913985800 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.694+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/1885764252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56391397ab60 0x56391397af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.694+0000 7f3cb0cce740 1 -- 172.21.15.103:0/1885764252 >> 172.21.15.103:0/1885764252 conn(0x5639139770f0 msgr2=0x5639139774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.694+0000 7f3cb0cce740 1 -- 172.21.15.103:0/1885764252 shutdown_connections 2024-09-19T06:57:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.694+0000 7f3cb0cce740 1 -- 172.21.15.103:0/1885764252 wait complete. 2024-09-19T06:57:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.694+0000 7f3cb0cce740 1 -- start start 2024-09-19T06:57:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3cb0cce740 1 Processor -- start 2024-09-19T06:57:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3cb0cce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56391397ab60 0x563913982be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3cb0cce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56391397b480 0x563913983120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3cb0cce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563913985d40 0x563913983660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3cb0cce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563913988910 con 0x563913985d40 2024-09-19T06:57:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3cb0cce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56391389af10 con 0x56391397ab60 2024-09-19T06:57:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3cb0cce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563913988a90 con 0x56391397b480 2024-09-19T06:57:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3cb0cce740 10 client.0 _ll_register_callbacks cb 0x563913863a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3cace92640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563913985d40 0x563913983660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3cace92640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563913985d40 0x563913983660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41772/0 (socket says 172.21.15.103:41772) 2024-09-19T06:57:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3cace92640 1 -- 172.21.15.103:0/3613579117 learned_addr learned my addr 172.21.15.103:0/3613579117 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3ca77fe640 1 --2- 172.21.15.103:0/3613579117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56391397b480 0x563913983120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3ca7fff640 1 --2- 172.21.15.103:0/3613579117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56391397ab60 0x563913982be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3cace92640 1 -- 172.21.15.103:0/3613579117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56391397b480 msgr2=0x563913983120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3cace92640 1 --2- 172.21.15.103:0/3613579117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56391397b480 0x563913983120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3cace92640 1 -- 172.21.15.103:0/3613579117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56391397ab60 msgr2=0x563913982be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3cace92640 1 --2- 172.21.15.103:0/3613579117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56391397ab60 0x563913982be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3cace92640 1 -- 172.21.15.103:0/3613579117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639136516a0 con 0x563913985d40 2024-09-19T06:57:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.695+0000 7f3ca7fff640 1 --2- 172.21.15.103:0/3613579117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56391397ab60 0x563913982be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.696+0000 7f3cace92640 1 --2- 172.21.15.103:0/3613579117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563913985d40 0x563913983660 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0003d20 tx=0x7f3ca0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7201316390c7efa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.696+0000 7f3c877fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.696+0000 7f3c877fe640 1 -- 172.21.15.103:0/3613579117 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0003e10 con 0x563913985d40 2024-09-19T06:57:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.696+0000 7f3c877fe640 1 -- 172.21.15.103:0/3613579117 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f3ca00045b0 con 0x563913985d40 2024-09-19T06:57:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.696+0000 7f3cb0cce740 1 -- 172.21.15.103:0/3613579117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c98004030 con 0x563913985d40 2024-09-19T06:57:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.696+0000 7f3c877fe640 1 -- 172.21.15.103:0/3613579117 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3ca00047b0 con 0x563913985d40 2024-09-19T06:57:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.696+0000 7f3cb0cce740 10 client.18468 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.696+0000 7f3cb0cce740 1 -- 172.21.15.103:0/3613579117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563913994540 con 0x563913985d40 2024-09-19T06:57:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.696+0000 7f3c877fe640 1 -- 172.21.15.103:0/3613579117 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c98004030 con 0x563913985d40 2024-09-19T06:57:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.696+0000 7f3c877fe640 1 -- 172.21.15.103:0/3613579117 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3ca000f040 con 0x563913985d40 2024-09-19T06:57:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.696+0000 7f3cb0cce740 10 client.18468 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.696+0000 7f3cb0cce740 10 client.18468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.696+0000 7f3cb0cce740 20 client.18468 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.696+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/3613579117 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563913995950 0x563913997da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.696+0000 7f3cb0cce740 4 client.18468 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.696+0000 7f3cb0cce740 1 -- 172.21.15.103:0/3613579117 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639138ed7f0 con 0x563913995950 2024-09-19T06:57:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.697+0000 7f3ca7fff640 1 --2- 172.21.15.103:0/3613579117 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563913995950 0x563913997da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.697+0000 7f3ca7fff640 1 --2- 172.21.15.103:0/3613579117 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563913995950 0x563913997da0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67c4fcf5cc647226 server_cookie=f49ad1d46e2431f7 in_seq=0 out_seq=0 2024-09-19T06:57:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.697+0000 7f3c877fe640 10 client.18468 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.698+0000 7f3c877fe640 1 -- 172.21.15.103:0/3613579117 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5639138ed7f0 con 0x563913995950 2024-09-19T06:57:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.698+0000 7f3c877fe640 10 client.18468 handle_command_reply: tid=0 2024-09-19T06:57:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.698+0000 7f3cb0cce740 1 client.18468 shutdown 2024-09-19T06:57:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.698+0000 7f3cb0cce740 1 -- 172.21.15.103:0/3613579117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563913985d40 msgr2=0x563913983660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.698+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/3613579117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563913985d40 0x563913983660 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0003d20 tx=0x7f3ca0004a40 comp rx=0 tx=0).stop 2024-09-19T06:57:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.698+0000 7f3cb0cce740 1 -- 172.21.15.103:0/3613579117 shutdown_connections 2024-09-19T06:57:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.698+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/3613579117 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563913995950 0x563913997da0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.698+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/3613579117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563913985d40 0x563913983660 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.698+0000 7f3c877fe640 0 client.18468 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.698+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/3613579117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56391397b480 0x563913983120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.698+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/3613579117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56391397ab60 0x563913982be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.698+0000 7f3cb0cce740 1 -- 172.21.15.103:0/3613579117 >> 172.21.15.103:0/3613579117 conn(0x5639139770f0 msgr2=0x56391397ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.698+0000 7f3cb0cce740 1 -- 172.21.15.103:0/3613579117 shutdown_connections 2024-09-19T06:57:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.698+0000 7f3cb0cce740 1 -- 172.21.15.103:0/3613579117 wait complete. 2024-09-19T06:57:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.698+0000 7f3cb0cce740 20 client.18468 trim_cache size 0 max 16384 2024-09-19T06:57:46.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.715+0000 7f3cb0cce740 1 Processor -- start 2024-09-19T06:57:46.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.715+0000 7f3cb0cce740 1 -- start start 2024-09-19T06:57:46.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.715+0000 7f3cb0cce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397ab60 0x56391397d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.715+0000 7f3cb0cce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56391397b480 0x563913984a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.715+0000 7f3cb0cce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563913985d40 0x56391399d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.715+0000 7f3cb0cce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca81ad8c0 con 0x56391397ab60 2024-09-19T06:57:46.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.715+0000 7f3cb0cce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca81af400 con 0x563913985d40 2024-09-19T06:57:46.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.715+0000 7f3cb0cce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca81adf90 con 0x56391397b480 2024-09-19T06:57:46.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.715+0000 7f3ca7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397ab60 0x56391397d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.715+0000 7f3ca7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397ab60 0x56391397d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41786/0 (socket says 172.21.15.103:41786) 2024-09-19T06:57:46.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.715+0000 7f3ca7fff640 1 -- 172.21.15.103:0/4103336183 learned_addr learned my addr 172.21.15.103:0/4103336183 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:46.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.716+0000 7f3ca77fe640 1 --2- 172.21.15.103:0/4103336183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56391397b480 0x563913984a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.716+0000 7f3cace92640 1 --2- 172.21.15.103:0/4103336183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563913985d40 0x56391399d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.716+0000 7f3ca7fff640 1 -- 172.21.15.103:0/4103336183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56391397b480 msgr2=0x563913984a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:46.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.716+0000 7f3ca7fff640 1 --2- 172.21.15.103:0/4103336183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56391397b480 0x563913984a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.716+0000 7f3ca7fff640 1 -- 172.21.15.103:0/4103336183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563913985d40 msgr2=0x56391399d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:46.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.716+0000 7f3ca7fff640 1 --2- 172.21.15.103:0/4103336183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563913985d40 0x56391399d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.716+0000 7f3ca7fff640 1 -- 172.21.15.103:0/4103336183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56391397b2b0 con 0x56391397ab60 2024-09-19T06:57:46.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.716+0000 7f3ca77fe640 1 --2- 172.21.15.103:0/4103336183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56391397b480 0x563913984a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:46.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.716+0000 7f3ca7fff640 1 --2- 172.21.15.103:0/4103336183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397ab60 0x56391397d610 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f3c98000c00 tx=0x7f3c9802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d977ae591b9e7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:46.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.716+0000 7f3c8dffb640 1 -- 172.21.15.103:0/4103336183 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98035070 con 0x56391397ab60 2024-09-19T06:57:46.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.716+0000 7f3c8dffb640 1 -- 172.21.15.103:0/4103336183 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c980317b0 con 0x56391397ab60 2024-09-19T06:57:46.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.716+0000 7f3c8dffb640 1 -- 172.21.15.103:0/4103336183 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9803f040 con 0x56391397ab60 2024-09-19T06:57:46.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.717+0000 7f3cb0cce740 1 -- 172.21.15.103:0/4103336183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397ab60 msgr2=0x56391397d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:46.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.717+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/4103336183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397ab60 0x56391397d610 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f3c98000c00 tx=0x7f3c9802a410 comp rx=0 tx=0).stop 2024-09-19T06:57:46.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.717+0000 7f3cb0cce740 1 -- 172.21.15.103:0/4103336183 shutdown_connections 2024-09-19T06:57:46.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.717+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/4103336183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563913985d40 0x56391399d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.717+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/4103336183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56391397b480 0x563913984a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.717+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/4103336183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397ab60 0x56391397d610 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.717+0000 7f3cb0cce740 1 -- 172.21.15.103:0/4103336183 >> 172.21.15.103:0/4103336183 conn(0x5639139770f0 msgr2=0x563913984330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:46.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.717+0000 7f3cb0cce740 1 -- 172.21.15.103:0/4103336183 shutdown_connections 2024-09-19T06:57:46.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.717+0000 7f3cb0cce740 1 -- 172.21.15.103:0/4103336183 wait complete. 2024-09-19T06:57:46.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.717+0000 7f3cb0cce740 1 -- start start 2024-09-19T06:57:46.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.717+0000 7f3cb0cce740 1 Processor -- start 2024-09-19T06:57:46.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3cb0cce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56391397ab60 0x56391398e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3cb0cce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397b480 0x5639139898d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3cb0cce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563913985d40 0x56391398a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3cb0cce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca81aed30 con 0x56391397b480 2024-09-19T06:57:46.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3cb0cce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca81ae660 con 0x563913985d40 2024-09-19T06:57:46.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3cb0cce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca81ab8c0 con 0x56391397ab60 2024-09-19T06:57:46.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3cb0cce740 10 client.0 _ll_register_callbacks cb 0x7f3ca0003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:46.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3ca77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397b480 0x5639139898d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3ca77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397b480 0x5639139898d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41798/0 (socket says 172.21.15.103:41798) 2024-09-19T06:57:46.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3ca77fe640 1 -- 172.21.15.103:0/2141252520 learned_addr learned my addr 172.21.15.103:0/2141252520 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:46.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3ca7fff640 1 --2- 172.21.15.103:0/2141252520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56391397ab60 0x56391398e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3cace92640 1 --2- 172.21.15.103:0/2141252520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563913985d40 0x56391398a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3ca77fe640 1 -- 172.21.15.103:0/2141252520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56391397ab60 msgr2=0x56391398e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:46.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3ca77fe640 1 --2- 172.21.15.103:0/2141252520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56391397ab60 0x56391398e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3ca77fe640 1 -- 172.21.15.103:0/2141252520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563913985d40 msgr2=0x56391398a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:46.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3ca77fe640 1 --2- 172.21.15.103:0/2141252520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563913985d40 0x56391398a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3ca77fe640 1 -- 172.21.15.103:0/2141252520 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563913a07e50 con 0x56391397b480 2024-09-19T06:57:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3ca77fe640 1 --2- 172.21.15.103:0/2141252520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397b480 0x5639139898d0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0983d0 tx=0x7f3c9c00d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dda59279cbe004f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3c877fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3c877fe640 1 -- 172.21.15.103:0/2141252520 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c0a4670 con 0x56391397b480 2024-09-19T06:57:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3c877fe640 1 -- 172.21.15.103:0/2141252520 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f3c9c098070 con 0x56391397b480 2024-09-19T06:57:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3cb0cce740 1 -- 172.21.15.103:0/2141252520 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c980317b0 con 0x56391397b480 2024-09-19T06:57:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3c877fe640 1 -- 172.21.15.103:0/2141252520 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c9c0c09c0 con 0x56391397b480 2024-09-19T06:57:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.718+0000 7f3cb0cce740 10 client.18474 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.719+0000 7f3cb0cce740 1 -- 172.21.15.103:0/2141252520 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563913a08f80 con 0x56391397b480 2024-09-19T06:57:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.719+0000 7f3c877fe640 1 -- 172.21.15.103:0/2141252520 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c980317b0 con 0x56391397b480 2024-09-19T06:57:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.719+0000 7f3c877fe640 1 -- 172.21.15.103:0/2141252520 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3c9c09a8c0 con 0x56391397b480 2024-09-19T06:57:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.719+0000 7f3cb0cce740 10 client.18474 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:46.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.719+0000 7f3cb0cce740 10 client.18474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.719+0000 7f3cb0cce740 20 client.18474 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.719+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/2141252520 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563913995950 0x563913a0b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.719+0000 7f3cb0cce740 4 client.18474 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:57:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.719+0000 7f3cb0cce740 1 -- 172.21.15.103:0/2141252520 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5639138e7d00 con 0x563913995950 2024-09-19T06:57:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.719+0000 7f3ca7fff640 1 --2- 172.21.15.103:0/2141252520 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563913995950 0x563913a0b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.720+0000 7f3ca7fff640 1 --2- 172.21.15.103:0/2141252520 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563913995950 0x563913a0b740 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e53d27608f2eb2 server_cookie=17e376d4cd8fb2ba in_seq=0 out_seq=0 2024-09-19T06:57:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.720+0000 7f3c877fe640 10 client.18474 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.720+0000 7f3c877fe640 1 -- 172.21.15.103:0/2141252520 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5639138e7d00 con 0x563913995950 2024-09-19T06:57:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.720+0000 7f3c877fe640 10 client.18474 handle_command_reply: tid=0 2024-09-19T06:57:46.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.720+0000 7f3cb0cce740 1 client.18474 shutdown 2024-09-19T06:57:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.721+0000 7f3cb0cce740 1 -- 172.21.15.103:0/2141252520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397b480 msgr2=0x5639139898d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.721+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/2141252520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397b480 0x5639139898d0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0983d0 tx=0x7f3c9c00d8d0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.721+0000 7f3cb0cce740 1 -- 172.21.15.103:0/2141252520 shutdown_connections 2024-09-19T06:57:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.721+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/2141252520 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563913995950 0x563913a0b740 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.721+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/2141252520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563913985d40 0x56391398a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.721+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/2141252520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56391397b480 0x5639139898d0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.721+0000 7f3c877fe640 0 client.18474 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.721+0000 7f3cb0cce740 1 --2- 172.21.15.103:0/2141252520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56391397ab60 0x56391398e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.721+0000 7f3cb0cce740 1 -- 172.21.15.103:0/2141252520 >> 172.21.15.103:0/2141252520 conn(0x5639139770f0 msgr2=0x563913976270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.721+0000 7f3cb0cce740 1 -- 172.21.15.103:0/2141252520 shutdown_connections 2024-09-19T06:57:46.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.721+0000 7f3cb0cce740 1 -- 172.21.15.103:0/2141252520 wait complete. 2024-09-19T06:57:46.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.721+0000 7f3cb0cce740 20 client.18474 trim_cache size 0 max 16384 2024-09-19T06:57:46.730 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:57:46.730 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T06:57:46.730 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "90d83d10-45ec-4c12-b098-69e7f5e4c207", 2024-09-19T06:57:46.730 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T06:57:46.730 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:57:46.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.722+0000 7f3cae8f5640 1 -- 172.21.15.103:0/1534823685 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c900a33b0 msgr2=0x7f3c900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:46.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.722+0000 7f3cae8f5640 1 --2- 172.21.15.103:0/1534823685 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c900a33b0 0x7f3c900a5870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009fd0 tx=0x7f3ca0004ab0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.722+0000 7f3cae8f5640 1 -- 172.21.15.103:0/1534823685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ca8102790 msgr2=0x7f3ca8198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:46.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.722+0000 7f3cae8f5640 1 --2- 172.21.15.103:0/1534823685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ca8102790 0x7f3ca8198970 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00d950 tx=0x7f3c9c00de10 comp rx=0 tx=0).stop 2024-09-19T06:57:46.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.722+0000 7f3cae8f5640 1 -- 172.21.15.103:0/1534823685 shutdown_connections 2024-09-19T06:57:46.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.722+0000 7f3cae8f5640 1 --2- 172.21.15.103:0/1534823685 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c900a33b0 0x7f3c900a5870 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.722+0000 7f3cae8f5640 1 --2- 172.21.15.103:0/1534823685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ca8106f10 0x7f3ca819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.722+0000 7f3cae8f5640 1 --2- 172.21.15.103:0/1534823685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ca8102790 0x7f3ca8198970 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.722+0000 7f3cae8f5640 1 --2- 172.21.15.103:0/1534823685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ca8101de0 0x7f3ca8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:46.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.722+0000 7f3cae8f5640 1 -- 172.21.15.103:0/1534823685 >> 172.21.15.103:0/1534823685 conn(0x7f3ca80fd8c0 msgr2=0x7f3ca80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:46.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.722+0000 7f3cae8f5640 1 -- 172.21.15.103:0/1534823685 shutdown_connections 2024-09-19T06:57:46.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:46.722+0000 7f3cae8f5640 1 -- 172.21.15.103:0/1534823685 wait complete. 2024-09-19T06:57:47.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:47 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T06:57:47.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:47 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:57:47.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:47 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T06:57:47.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:47 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:57:47.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:47 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T06:57:47.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:47 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:57:47.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:47 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T06:57:47.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:47 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:57:47.538 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:57:47.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:47 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T06:57:47.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:47 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:57:47.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:47 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T06:57:47.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:47 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:57:47.784 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:48.195 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:48 smithi103 ceph-mon[31158]: pgmap v264: 97 pgs: 97 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T06:57:48.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.194+0000 7f85bdafa640 1 -- 172.21.15.103:0/609338095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85b8101bc0 msgr2=0x7f85b810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:48.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.194+0000 7f85bdafa640 1 --2- 172.21.15.103:0/609338095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85b8101bc0 0x7f85b810d910 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009ca0 tx=0x7f85b002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.194+0000 7f85bdafa640 1 -- 172.21.15.103:0/609338095 shutdown_connections 2024-09-19T06:57:48.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.194+0000 7f85bdafa640 1 --2- 172.21.15.103:0/609338095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85b8101bc0 0x7f85b810d910 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.194+0000 7f85bdafa640 1 --2- 172.21.15.103:0/609338095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85b8101240 0x7f85b8101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.194+0000 7f85bdafa640 1 --2- 172.21.15.103:0/609338095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85b8105a90 0x7f85b8105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.194+0000 7f85bdafa640 1 -- 172.21.15.103:0/609338095 >> 172.21.15.103:0/609338095 conn(0x7f85b8077210 msgr2=0x7f85b80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:48.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.194+0000 7f85bdafa640 1 -- 172.21.15.103:0/609338095 shutdown_connections 2024-09-19T06:57:48.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.194+0000 7f85bdafa640 1 -- 172.21.15.103:0/609338095 wait complete. 2024-09-19T06:57:48.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.195+0000 7f85bdafa640 1 Processor -- start 2024-09-19T06:57:48.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.195+0000 7f85bdafa640 1 -- start start 2024-09-19T06:57:48.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.195+0000 7f85bdafa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85b8101240 0x7f85b819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.195+0000 7f85bdafa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85b8101bc0 0x7f85b819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.195+0000 7f85bdafa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85b8105a90 0x7f85b819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.195+0000 7f85bdafa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8111030 con 0x7f85b8101bc0 2024-09-19T06:57:48.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.195+0000 7f85bdafa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8110eb0 con 0x7f85b8105a90 2024-09-19T06:57:48.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.195+0000 7f85bdafa640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81111b0 con 0x7f85b8101240 2024-09-19T06:57:48.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.195+0000 7f85b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85b8101bc0 0x7f85b819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:48.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.195+0000 7f85b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85b8101bc0 0x7f85b819ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58968/0 (socket says 172.21.15.103:58968) 2024-09-19T06:57:48.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.195+0000 7f85b6ffd640 1 -- 172.21.15.103:0/1282412468 learned_addr learned my addr 172.21.15.103:0/1282412468 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:48.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.195+0000 7f85b7fff640 1 --2- 172.21.15.103:0/1282412468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85b8105a90 0x7f85b819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:48.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.196+0000 7f85b77fe640 1 --2- 172.21.15.103:0/1282412468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85b8101240 0x7f85b819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:48.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.196+0000 7f85b6ffd640 1 -- 172.21.15.103:0/1282412468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85b8101240 msgr2=0x7f85b819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:48.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.196+0000 7f85b6ffd640 1 --2- 172.21.15.103:0/1282412468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85b8101240 0x7f85b819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.196+0000 7f85b6ffd640 1 -- 172.21.15.103:0/1282412468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85b8105a90 msgr2=0x7f85b819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:48.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.196+0000 7f85b6ffd640 1 --2- 172.21.15.103:0/1282412468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85b8105a90 0x7f85b819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.196+0000 7f85b6ffd640 1 -- 172.21.15.103:0/1282412468 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b819f670 con 0x7f85b8101bc0 2024-09-19T06:57:48.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.196+0000 7f85b6ffd640 1 --2- 172.21.15.103:0/1282412468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85b8101bc0 0x7f85b819ac00 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00b4a0 tx=0x7f85ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:48.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.196+0000 7f85b4ff9640 1 -- 172.21.15.103:0/1282412468 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac014070 con 0x7f85b8101bc0 2024-09-19T06:57:48.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.196+0000 7f85b4ff9640 1 -- 172.21.15.103:0/1282412468 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ac010af0 con 0x7f85b8101bc0 2024-09-19T06:57:48.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.196+0000 7f85bdafa640 1 -- 172.21.15.103:0/1282412468 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b819f960 con 0x7f85b8101bc0 2024-09-19T06:57:48.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.196+0000 7f85b4ff9640 1 -- 172.21.15.103:0/1282412468 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac017d10 con 0x7f85b8101bc0 2024-09-19T06:57:48.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.196+0000 7f85bdafa640 1 -- 172.21.15.103:0/1282412468 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b81a7240 con 0x7f85b8101bc0 2024-09-19T06:57:48.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.198+0000 7f85b4ff9640 1 -- 172.21.15.103:0/1282412468 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f85ac017070 con 0x7f85b8101bc0 2024-09-19T06:57:48.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.199+0000 7f85bfed3740 1 Processor -- start 2024-09-19T06:57:48.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.199+0000 7f85bfed3740 1 -- start start 2024-09-19T06:57:48.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.199+0000 7f85b4ff9640 1 --2- 172.21.15.103:0/1282412468 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f85a00a33b0 0x7f85a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.199+0000 7f85bfed3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca54a0 0x55e158ca5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.199+0000 7f85bfed3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158ca5dc0 0x55e158cb0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.199+0000 7f85bfed3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158cb0680 0x55e158cb2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.199+0000 7f85bfed3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e158c0b2e0 con 0x55e158ca54a0 2024-09-19T06:57:48.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.199+0000 7f85bfed3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e158bb2ce0 con 0x55e158ca5dc0 2024-09-19T06:57:48.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.199+0000 7f85bfed3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e158c072a0 con 0x55e158cb0680 2024-09-19T06:57:48.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.199+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca54a0 0x55e158ca5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:48.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.199+0000 7f85b7fff640 1 --2- 172.21.15.103:0/1282412468 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f85a00a33b0 0x7f85a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:48.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.199+0000 7f85b4ff9640 1 -- 172.21.15.103:0/1282412468 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f85ac0cf110 con 0x7f85b8101bc0 2024-09-19T06:57:48.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.199+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca54a0 0x55e158ca5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58978/0 (socket says 172.21.15.103:58978) 2024-09-19T06:57:48.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.199+0000 7f85b77fe640 1 -- 172.21.15.103:0/2527755133 learned_addr learned my addr 172.21.15.103:0/2527755133 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:48.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.200+0000 7f85b6ffd640 1 --2- 172.21.15.103:0/2527755133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158ca5dc0 0x55e158cb0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:48.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.200+0000 7f85b7fff640 1 --2- 172.21.15.103:0/2527755133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158cb0680 0x55e158cb2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:48.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.200+0000 7f85b77fe640 1 -- 172.21.15.103:0/2527755133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158cb0680 msgr2=0x55e158cb2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:48.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.200+0000 7f85b77fe640 1 --2- 172.21.15.103:0/2527755133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158cb0680 0x55e158cb2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.200+0000 7f85b77fe640 1 -- 172.21.15.103:0/2527755133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158ca5dc0 msgr2=0x55e158cb0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:48.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.200+0000 7f85b77fe640 1 --2- 172.21.15.103:0/2527755133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158ca5dc0 0x55e158cb0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.200+0000 7f85b77fe640 1 -- 172.21.15.103:0/2527755133 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e158b97440 con 0x55e158ca54a0 2024-09-19T06:57:48.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.200+0000 7f85b7fff640 1 --2- 172.21.15.103:0/1282412468 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f85a00a33b0 0x7f85a00a5870 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009430 tx=0x7f85b0031d40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:48.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.200+0000 7f85b77fe640 1 --2- 172.21.15.103:0/2527755133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca54a0 0x55e158ca5880 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f85a800a920 tx=0x7f85a800ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35697d19eb83319a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:48.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.200+0000 7f859dffb640 1 -- 172.21.15.103:0/2527755133 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8015070 con 0x55e158ca54a0 2024-09-19T06:57:48.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.200+0000 7f859dffb640 1 -- 172.21.15.103:0/2527755133 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a800cc00 con 0x55e158ca54a0 2024-09-19T06:57:48.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.200+0000 7f859dffb640 1 -- 172.21.15.103:0/2527755133 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a80185c0 con 0x55e158ca54a0 2024-09-19T06:57:48.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.201+0000 7f85bfed3740 1 -- 172.21.15.103:0/2527755133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca54a0 msgr2=0x55e158ca5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:48.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.201+0000 7f85bfed3740 1 --2- 172.21.15.103:0/2527755133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca54a0 0x55e158ca5880 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f85a800a920 tx=0x7f85a800ade0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.201+0000 7f85bfed3740 1 -- 172.21.15.103:0/2527755133 shutdown_connections 2024-09-19T06:57:48.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.201+0000 7f85bfed3740 1 --2- 172.21.15.103:0/2527755133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158cb0680 0x55e158cb2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.201+0000 7f85bfed3740 1 --2- 172.21.15.103:0/2527755133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158ca5dc0 0x55e158cb0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.201+0000 7f85bfed3740 1 --2- 172.21.15.103:0/2527755133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca54a0 0x55e158ca5880 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.201+0000 7f85bfed3740 1 -- 172.21.15.103:0/2527755133 >> 172.21.15.103:0/2527755133 conn(0x55e158ca1a30 msgr2=0x55e158ca1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:48.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.201+0000 7f85bfed3740 1 -- 172.21.15.103:0/2527755133 shutdown_connections 2024-09-19T06:57:48.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.201+0000 7f85bfed3740 1 -- 172.21.15.103:0/2527755133 wait complete. 2024-09-19T06:57:48.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.201+0000 7f85bfed3740 1 -- start start 2024-09-19T06:57:48.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.201+0000 7f85bfed3740 1 Processor -- start 2024-09-19T06:57:48.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85bfed3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca54a0 0x55e158caff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85bfed3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca5dc0 0x55e158cad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85bfed3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158cb0680 0x55e158cad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85bfed3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e158cb3250 con 0x55e158ca5dc0 2024-09-19T06:57:48.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85bfed3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e158bc6050 con 0x55e158cb0680 2024-09-19T06:57:48.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85bfed3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e158cb33d0 con 0x55e158ca54a0 2024-09-19T06:57:48.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85bfed3740 10 client.0 _ll_register_callbacks cb 0x55e158c06160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:48.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca5dc0 0x55e158cad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:48.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca5dc0 0x55e158cad1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58986/0 (socket says 172.21.15.103:58986) 2024-09-19T06:57:48.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85b6ffd640 1 -- 172.21.15.103:0/406740766 learned_addr learned my addr 172.21.15.103:0/406740766 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:48.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85b77fe640 1 --2- 172.21.15.103:0/406740766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca54a0 0x55e158caff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:48.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85b6ffd640 1 -- 172.21.15.103:0/406740766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca54a0 msgr2=0x55e158caff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:48.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85b6ffd640 1 --2- 172.21.15.103:0/406740766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca54a0 0x55e158caff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85b6ffd640 1 -- 172.21.15.103:0/406740766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158cb0680 msgr2=0x55e158cad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:48.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85b6ffd640 1 --2- 172.21.15.103:0/406740766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158cb0680 0x55e158cad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85b6ffd640 1 -- 172.21.15.103:0/406740766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e15897c3c0 con 0x55e158ca5dc0 2024-09-19T06:57:48.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85b77fe640 1 --2- 172.21.15.103:0/406740766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca54a0 0x55e158caff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:48.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85b6ffd640 1 --2- 172.21.15.103:0/406740766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca5dc0 0x55e158cad1f0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f85ac09b410 tx=0x7f85ac09aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939b27811012d13d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:48.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f8596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:48.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f8596ffd640 1 -- 172.21.15.103:0/406740766 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac01e030 con 0x55e158ca5dc0 2024-09-19T06:57:48.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f8596ffd640 1 -- 172.21.15.103:0/406740766 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f85ac010ca0 con 0x55e158ca5dc0 2024-09-19T06:57:48.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.202+0000 7f85bfed3740 1 -- 172.21.15.103:0/406740766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a800cc00 con 0x55e158ca5dc0 2024-09-19T06:57:48.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.203+0000 7f8596ffd640 1 -- 172.21.15.103:0/406740766 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f85ac0a40f0 con 0x55e158ca5dc0 2024-09-19T06:57:48.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.203+0000 7f85bfed3740 10 client.18486 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:48.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.203+0000 7f85bfed3740 1 -- 172.21.15.103:0/406740766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e158cbeba0 con 0x55e158ca5dc0 2024-09-19T06:57:48.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.203+0000 7f8596ffd640 1 -- 172.21.15.103:0/406740766 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a800cc00 con 0x55e158ca5dc0 2024-09-19T06:57:48.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.203+0000 7f8596ffd640 1 -- 172.21.15.103:0/406740766 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f85ac0cca60 con 0x55e158ca5dc0 2024-09-19T06:57:48.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.203+0000 7f85bfed3740 10 client.18486 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:48.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.203+0000 7f85bfed3740 10 client.18486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:48.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.203+0000 7f85bfed3740 20 client.18486 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:48.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.203+0000 7f85bfed3740 1 --2- 172.21.15.103:0/406740766 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e158cc0000 0x55e158cc2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.203+0000 7f85bfed3740 4 client.18486 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:48.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.203+0000 7f85bfed3740 1 -- 172.21.15.103:0/406740766 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e158c18850 con 0x55e158cc0000 2024-09-19T06:57:48.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.203+0000 7f85b77fe640 1 --2- 172.21.15.103:0/406740766 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e158cc0000 0x55e158cc2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:48.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.204+0000 7f8596ffd640 10 client.18486 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:48.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.204+0000 7f85b77fe640 1 --2- 172.21.15.103:0/406740766 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e158cc0000 0x55e158cc2450 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=829f9e1d700edce9 server_cookie=3d0c955d96130918 in_seq=0 out_seq=0 2024-09-19T06:57:48.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.204+0000 7f8596ffd640 1 -- 172.21.15.103:0/406740766 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e158c18850 con 0x55e158cc0000 2024-09-19T06:57:48.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.204+0000 7f8596ffd640 10 client.18486 handle_command_reply: tid=0 2024-09-19T06:57:48.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.204+0000 7f85bfed3740 1 client.18486 shutdown 2024-09-19T06:57:48.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.205+0000 7f85bfed3740 1 -- 172.21.15.103:0/406740766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca5dc0 msgr2=0x55e158cad1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:48.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.205+0000 7f85bfed3740 1 --2- 172.21.15.103:0/406740766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca5dc0 0x55e158cad1f0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f85ac09b410 tx=0x7f85ac09aed0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.205+0000 7f85bfed3740 1 -- 172.21.15.103:0/406740766 shutdown_connections 2024-09-19T06:57:48.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.205+0000 7f85bfed3740 1 --2- 172.21.15.103:0/406740766 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e158cc0000 0x55e158cc2450 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.205+0000 7f85bfed3740 1 --2- 172.21.15.103:0/406740766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158cb0680 0x55e158cad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.205+0000 7f85bfed3740 1 --2- 172.21.15.103:0/406740766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca5dc0 0x55e158cad1f0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.205+0000 7f85bfed3740 1 --2- 172.21.15.103:0/406740766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca54a0 0x55e158caff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.205+0000 7f85bfed3740 1 -- 172.21.15.103:0/406740766 >> 172.21.15.103:0/406740766 conn(0x55e158ca1a30 msgr2=0x55e158ca7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:48.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.205+0000 7f8596ffd640 0 client.18486 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:48.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.205+0000 7f85bfed3740 1 -- 172.21.15.103:0/406740766 shutdown_connections 2024-09-19T06:57:48.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.205+0000 7f85bfed3740 1 -- 172.21.15.103:0/406740766 wait complete. 2024-09-19T06:57:48.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.205+0000 7f85bfed3740 20 client.18486 trim_cache size 0 max 16384 2024-09-19T06:57:48.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.222+0000 7f85bfed3740 1 Processor -- start 2024-09-19T06:57:48.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.222+0000 7f85bfed3740 1 -- start start 2024-09-19T06:57:48.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.222+0000 7f85bfed3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca1a30 0x55e158caf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.222+0000 7f85bfed3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca54a0 0x55e158cc7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.222+0000 7f85bfed3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158ca5dc0 0x55e158cc8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.222+0000 7f85bfed3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81afaf0 con 0x55e158ca1a30 2024-09-19T06:57:48.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.222+0000 7f85bfed3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81b1630 con 0x55e158ca5dc0 2024-09-19T06:57:48.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.222+0000 7f85bfed3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81b01c0 con 0x55e158ca54a0 2024-09-19T06:57:48.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.222+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca1a30 0x55e158caf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:48.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.222+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca1a30 0x55e158caf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59000/0 (socket says 172.21.15.103:59000) 2024-09-19T06:57:48.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.222+0000 7f85b77fe640 1 -- 172.21.15.103:0/1339334220 learned_addr learned my addr 172.21.15.103:0/1339334220 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:48.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.222+0000 7f85b6ffd640 1 --2- 172.21.15.103:0/1339334220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca54a0 0x55e158cc7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:48.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.223+0000 7f85b7fff640 1 --2- 172.21.15.103:0/1339334220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158ca5dc0 0x55e158cc8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:48.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.223+0000 7f85b77fe640 1 -- 172.21.15.103:0/1339334220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca54a0 msgr2=0x55e158cc7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:48.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.223+0000 7f85b77fe640 1 --2- 172.21.15.103:0/1339334220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca54a0 0x55e158cc7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.223+0000 7f85b77fe640 1 -- 172.21.15.103:0/1339334220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158ca5dc0 msgr2=0x55e158cc8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:48.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.223+0000 7f85b77fe640 1 --2- 172.21.15.103:0/1339334220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158ca5dc0 0x55e158cc8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.223+0000 7f85b77fe640 1 -- 172.21.15.103:0/1339334220 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e158cb2db0 con 0x55e158ca1a30 2024-09-19T06:57:48.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.223+0000 7f85b7fff640 1 --2- 172.21.15.103:0/1339334220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158ca5dc0 0x55e158cc8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:48.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.223+0000 7f85b77fe640 1 --2- 172.21.15.103:0/1339334220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca1a30 0x55e158caf160 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f85a8000c00 tx=0x7f85a8011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b81fb5dd75b4ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:48.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.223+0000 7f859d7fa640 1 -- 172.21.15.103:0/1339334220 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a802e070 con 0x55e158ca1a30 2024-09-19T06:57:48.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.223+0000 7f859d7fa640 1 -- 172.21.15.103:0/1339334220 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a800cba0 con 0x55e158ca1a30 2024-09-19T06:57:48.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.223+0000 7f859d7fa640 1 -- 172.21.15.103:0/1339334220 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8015040 con 0x55e158ca1a30 2024-09-19T06:57:48.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.224+0000 7f85bfed3740 1 -- 172.21.15.103:0/1339334220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca1a30 msgr2=0x55e158caf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:48.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.224+0000 7f85bfed3740 1 --2- 172.21.15.103:0/1339334220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca1a30 0x55e158caf160 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f85a8000c00 tx=0x7f85a8011d80 comp rx=0 tx=0).stop 2024-09-19T06:57:48.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.224+0000 7f85bfed3740 1 -- 172.21.15.103:0/1339334220 shutdown_connections 2024-09-19T06:57:48.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.224+0000 7f85bfed3740 1 --2- 172.21.15.103:0/1339334220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158ca5dc0 0x55e158cc8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.224+0000 7f85bfed3740 1 --2- 172.21.15.103:0/1339334220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca54a0 0x55e158cc7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.224+0000 7f85bfed3740 1 --2- 172.21.15.103:0/1339334220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca1a30 0x55e158caf160 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.224+0000 7f85bfed3740 1 -- 172.21.15.103:0/1339334220 >> 172.21.15.103:0/1339334220 conn(0x55e158cad890 msgr2=0x55e158caea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:48.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.224+0000 7f85bfed3740 1 -- 172.21.15.103:0/1339334220 shutdown_connections 2024-09-19T06:57:48.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.224+0000 7f85bfed3740 1 -- 172.21.15.103:0/1339334220 wait complete. 2024-09-19T06:57:48.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.224+0000 7f85bfed3740 1 -- start start 2024-09-19T06:57:48.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.224+0000 7f85bfed3740 1 Processor -- start 2024-09-19T06:57:48.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85bfed3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca1a30 0x55e158cb6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85bfed3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca54a0 0x55e158cb6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85bfed3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158ca5dc0 0x55e158cb70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85bfed3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81b0f60 con 0x55e158ca54a0 2024-09-19T06:57:48.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85bfed3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81b0890 con 0x55e158ca5dc0 2024-09-19T06:57:48.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85bfed3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81adaf0 con 0x55e158ca1a30 2024-09-19T06:57:48.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85bfed3740 10 client.0 _ll_register_callbacks cb 0x7f85ac09b250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:48.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca54a0 0x55e158cb6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:48.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca54a0 0x55e158cb6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59014/0 (socket says 172.21.15.103:59014) 2024-09-19T06:57:48.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85b6ffd640 1 -- 172.21.15.103:0/684452155 learned_addr learned my addr 172.21.15.103:0/684452155 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:48.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85b77fe640 1 --2- 172.21.15.103:0/684452155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca1a30 0x55e158cb6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:48.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85b6ffd640 1 -- 172.21.15.103:0/684452155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca1a30 msgr2=0x55e158cb6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:48.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85b6ffd640 1 --2- 172.21.15.103:0/684452155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca1a30 0x55e158cb6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85b6ffd640 1 -- 172.21.15.103:0/684452155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158ca5dc0 msgr2=0x55e158cb70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:48.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85b6ffd640 1 --2- 172.21.15.103:0/684452155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158ca5dc0 0x55e158cb70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85b6ffd640 1 -- 172.21.15.103:0/684452155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e158d29fa0 con 0x55e158ca54a0 2024-09-19T06:57:48.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.225+0000 7f85b77fe640 1 --2- 172.21.15.103:0/684452155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca1a30 0x55e158cb6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:48.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.226+0000 7f85b6ffd640 1 --2- 172.21.15.103:0/684452155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca54a0 0x55e158cb6b70 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00b470 tx=0x7f85ac00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e6d8c1e9a94b9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.226+0000 7f8596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.226+0000 7f8596ffd640 1 -- 172.21.15.103:0/684452155 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac01e040 con 0x55e158ca54a0 2024-09-19T06:57:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.226+0000 7f8596ffd640 1 -- 172.21.15.103:0/684452155 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f85ac0c8360 con 0x55e158ca54a0 2024-09-19T06:57:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.226+0000 7f85bfed3740 1 -- 172.21.15.103:0/684452155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a800cba0 con 0x55e158ca54a0 2024-09-19T06:57:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.226+0000 7f8596ffd640 1 -- 172.21.15.103:0/684452155 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f85ac0b4250 con 0x55e158ca54a0 2024-09-19T06:57:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.226+0000 7f85bfed3740 10 client.18492 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.226+0000 7f85bfed3740 1 -- 172.21.15.103:0/684452155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e158d2b290 con 0x55e158ca54a0 2024-09-19T06:57:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.226+0000 7f8596ffd640 1 -- 172.21.15.103:0/684452155 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a800cba0 con 0x55e158ca54a0 2024-09-19T06:57:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.226+0000 7f8596ffd640 1 -- 172.21.15.103:0/684452155 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f85ac09a960 con 0x55e158ca54a0 2024-09-19T06:57:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.226+0000 7f85bfed3740 10 client.18492 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.226+0000 7f85bfed3740 10 client.18492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.226+0000 7f85bfed3740 20 client.18492 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.226+0000 7f85bfed3740 1 --2- 172.21.15.103:0/684452155 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e158cb0680 0x55e158d2da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.226+0000 7f85bfed3740 4 client.18492 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:57:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.226+0000 7f85bfed3740 1 -- 172.21.15.103:0/684452155 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e158c12d60 con 0x55e158cb0680 2024-09-19T06:57:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.227+0000 7f85b77fe640 1 --2- 172.21.15.103:0/684452155 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e158cb0680 0x55e158d2da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.227+0000 7f8596ffd640 10 client.18492 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.227+0000 7f85b77fe640 1 --2- 172.21.15.103:0/684452155 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e158cb0680 0x55e158d2da30 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da63437a6c3ca169 server_cookie=c9107bf60e3615d7 in_seq=0 out_seq=0 2024-09-19T06:57:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.227+0000 7f8596ffd640 1 -- 172.21.15.103:0/684452155 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e158c12d60 con 0x55e158cb0680 2024-09-19T06:57:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.227+0000 7f8596ffd640 10 client.18492 handle_command_reply: tid=0 2024-09-19T06:57:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.227+0000 7f85bfed3740 1 client.18492 shutdown 2024-09-19T06:57:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.227+0000 7f85bfed3740 1 -- 172.21.15.103:0/684452155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca54a0 msgr2=0x55e158cb6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.227+0000 7f85bfed3740 1 --2- 172.21.15.103:0/684452155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca54a0 0x55e158cb6b70 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00b470 tx=0x7f85ac00bf90 comp rx=0 tx=0).stop 2024-09-19T06:57:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.228+0000 7f85bfed3740 1 -- 172.21.15.103:0/684452155 shutdown_connections 2024-09-19T06:57:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.228+0000 7f85bfed3740 1 --2- 172.21.15.103:0/684452155 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e158cb0680 0x55e158d2da30 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.228+0000 7f8596ffd640 0 client.18492 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.228+0000 7f85bfed3740 1 --2- 172.21.15.103:0/684452155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e158ca5dc0 0x55e158cb70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.228+0000 7f85bfed3740 1 --2- 172.21.15.103:0/684452155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e158ca54a0 0x55e158cb6b70 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.228+0000 7f85bfed3740 1 --2- 172.21.15.103:0/684452155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e158ca1a30 0x55e158cb6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.228+0000 7f85bfed3740 1 -- 172.21.15.103:0/684452155 >> 172.21.15.103:0/684452155 conn(0x55e158cad890 msgr2=0x55e158caf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:48.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.228+0000 7f85bfed3740 1 -- 172.21.15.103:0/684452155 shutdown_connections 2024-09-19T06:57:48.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.228+0000 7f85bfed3740 1 -- 172.21.15.103:0/684452155 wait complete. 2024-09-19T06:57:48.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.228+0000 7f85bfed3740 20 client.18492 trim_cache size 0 max 16384 2024-09-19T06:57:48.243 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:57:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:57:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T06:57:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "90d83d10-45ec-4c12-b098-69e7f5e4c207": { 2024-09-19T06:57:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T06:57:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "90d83d10-45ec-4c12-b098-69e7f5e4c207", 2024-09-19T06:57:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T06:57:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:57:48.244 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:57:48.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.228+0000 7f85bdafa640 1 -- 172.21.15.103:0/1282412468 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f85a00a33b0 msgr2=0x7f85a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:48.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.228+0000 7f85bdafa640 1 --2- 172.21.15.103:0/1282412468 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f85a00a33b0 0x7f85a00a5870 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009430 tx=0x7f85b0031d40 comp rx=0 tx=0).stop 2024-09-19T06:57:48.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.228+0000 7f85bdafa640 1 -- 172.21.15.103:0/1282412468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85b8101bc0 msgr2=0x7f85b819ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:48.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.228+0000 7f85bdafa640 1 --2- 172.21.15.103:0/1282412468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85b8101bc0 0x7f85b819ac00 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00b4a0 tx=0x7f85ac00b960 comp rx=0 tx=0).stop 2024-09-19T06:57:48.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.229+0000 7f85bdafa640 1 -- 172.21.15.103:0/1282412468 shutdown_connections 2024-09-19T06:57:48.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.229+0000 7f85bdafa640 1 --2- 172.21.15.103:0/1282412468 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f85a00a33b0 0x7f85a00a5870 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.229+0000 7f85bdafa640 1 --2- 172.21.15.103:0/1282412468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85b8105a90 0x7f85b819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.229+0000 7f85bdafa640 1 --2- 172.21.15.103:0/1282412468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85b8101bc0 0x7f85b819ac00 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.229+0000 7f85bdafa640 1 --2- 172.21.15.103:0/1282412468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85b8101240 0x7f85b819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:48.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.229+0000 7f85bdafa640 1 -- 172.21.15.103:0/1282412468 >> 172.21.15.103:0/1282412468 conn(0x7f85b8077210 msgr2=0x7f85b8104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:48.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.229+0000 7f85bdafa640 1 -- 172.21.15.103:0/1282412468 shutdown_connections 2024-09-19T06:57:48.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:57:48.229+0000 7f85bdafa640 1 -- 172.21.15.103:0/1282412468 wait complete. 2024-09-19T06:57:48.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:48 smithi134 ceph-mon[35356]: pgmap v264: 97 pgs: 97 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T06:57:48.484 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:48 smithi174 ceph-mon[35637]: pgmap v264: 97 pgs: 97 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T06:57:48.853 INFO:tasks.cephfs.filesystem:scrub status for tag:90d83d10-45ec-4c12-b098-69e7f5e4c207 - {'path': '/', 'tag': '90d83d10-45ec-4c12-b098-69e7f5e4c207', 'options': 'recursive,force'} 2024-09-19T06:57:50.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:49 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T06:57:50.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:49 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T06:57:50.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:49 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T06:57:51.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:50 smithi134 ceph-mon[35356]: pgmap v265: 97 pgs: 97 active+clean; 2.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-19T06:57:51.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:50 smithi174 ceph-mon[35637]: pgmap v265: 97 pgs: 97 active+clean; 2.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-19T06:57:51.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:50 smithi103 ceph-mon[31158]: pgmap v265: 97 pgs: 97 active+clean; 2.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-19T06:57:52.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:52 smithi103 ceph-mon[31158]: pgmap v266: 97 pgs: 97 active+clean; 2.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T06:57:52.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:52 smithi134 ceph-mon[35356]: pgmap v266: 97 pgs: 97 active+clean; 2.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T06:57:52.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:52 smithi174 ceph-mon[35637]: pgmap v266: 97 pgs: 97 active+clean; 2.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T06:57:54.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:54 smithi103 ceph-mon[31158]: pgmap v267: 97 pgs: 97 active+clean; 2.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T06:57:54.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:54 smithi134 ceph-mon[35356]: pgmap v267: 97 pgs: 97 active+clean; 2.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T06:57:54.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:54 smithi174 ceph-mon[35637]: pgmap v267: 97 pgs: 97 active+clean; 2.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T06:57:56.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:56 smithi103 ceph-mon[31158]: pgmap v268: 97 pgs: 97 active+clean; 2.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T06:57:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:56 smithi134 ceph-mon[35356]: pgmap v268: 97 pgs: 97 active+clean; 2.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T06:57:56.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:56 smithi174 ceph-mon[35637]: pgmap v268: 97 pgs: 97 active+clean; 2.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T06:57:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.688+0000 7f7bbf2d2640 1 -- 172.21.15.103:0/3246796229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bb8103640 msgr2=0x7f7bb810bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.688+0000 7f7bbf2d2640 1 --2- 172.21.15.103:0/3246796229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bb8103640 0x7f7bb810bb20 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009900 tx=0x7f7bac02a480 comp rx=0 tx=0).stop 2024-09-19T06:57:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.688+0000 7f7bbf2d2640 1 -- 172.21.15.103:0/3246796229 shutdown_connections 2024-09-19T06:57:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.688+0000 7f7bbf2d2640 1 --2- 172.21.15.103:0/3246796229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb810c0f0 0x7f7bb810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.688+0000 7f7bbf2d2640 1 --2- 172.21.15.103:0/3246796229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bb8103640 0x7f7bb810bb20 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.688+0000 7f7bbf2d2640 1 --2- 172.21.15.103:0/3246796229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bb8102d20 0x7f7bb8103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.688+0000 7f7bbf2d2640 1 -- 172.21.15.103:0/3246796229 >> 172.21.15.103:0/3246796229 conn(0x7f7bb8077990 msgr2=0x7f7bb80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.688+0000 7f7bbf2d2640 1 -- 172.21.15.103:0/3246796229 shutdown_connections 2024-09-19T06:57:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.688+0000 7f7bbf2d2640 1 -- 172.21.15.103:0/3246796229 wait complete. 2024-09-19T06:57:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbf2d2640 1 Processor -- start 2024-09-19T06:57:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbf2d2640 1 -- start start 2024-09-19T06:57:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbf2d2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bb8102d20 0x7f7bb8198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbf2d2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bb8103640 0x7f7bb8198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbf2d2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb810c0f0 0x7f7bb819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbf2d2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb8111ec0 con 0x7f7bb8102d20 2024-09-19T06:57:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbf2d2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb8111d40 con 0x7f7bb810c0f0 2024-09-19T06:57:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbf2d2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb8112040 con 0x7f7bb8103640 2024-09-19T06:57:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbd06e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bb8102d20 0x7f7bb8198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbd06e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bb8102d20 0x7f7bb8198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38360/0 (socket says 172.21.15.103:38360) 2024-09-19T06:57:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbd06e640 1 -- 172.21.15.103:0/4092754705 learned_addr learned my addr 172.21.15.103:0/4092754705 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbc86d640 1 --2- 172.21.15.103:0/4092754705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bb8103640 0x7f7bb8198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbd06e640 1 -- 172.21.15.103:0/4092754705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bb8103640 msgr2=0x7f7bb8198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbd06e640 1 --2- 172.21.15.103:0/4092754705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bb8103640 0x7f7bb8198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbd86f640 1 --2- 172.21.15.103:0/4092754705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb810c0f0 0x7f7bb819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbd06e640 1 -- 172.21.15.103:0/4092754705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb810c0f0 msgr2=0x7f7bb819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbd06e640 1 --2- 172.21.15.103:0/4092754705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb810c0f0 0x7f7bb819cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbd06e640 1 -- 172.21.15.103:0/4092754705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb819d340 con 0x7f7bb8102d20 2024-09-19T06:57:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.689+0000 7f7bbc86d640 1 --2- 172.21.15.103:0/4092754705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bb8103640 0x7f7bb8198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.690+0000 7f7bbd06e640 1 --2- 172.21.15.103:0/4092754705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bb8102d20 0x7f7bb8198350 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800ea80 tx=0x7f7ba800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.690+0000 7f7bb67fc640 1 -- 172.21.15.103:0/4092754705 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800ce40 con 0x7f7bb8102d20 2024-09-19T06:57:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.690+0000 7f7bb67fc640 1 -- 172.21.15.103:0/4092754705 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba8005020 con 0x7f7bb8102d20 2024-09-19T06:57:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.690+0000 7f7bbf2d2640 1 -- 172.21.15.103:0/4092754705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb819d630 con 0x7f7bb8102d20 2024-09-19T06:57:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.690+0000 7f7bb67fc640 1 -- 172.21.15.103:0/4092754705 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8011760 con 0x7f7bb8102d20 2024-09-19T06:57:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.690+0000 7f7bbf2d2640 1 -- 172.21.15.103:0/4092754705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb81a4ec0 con 0x7f7bb8102d20 2024-09-19T06:57:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.692+0000 7f7bb67fc640 1 -- 172.21.15.103:0/4092754705 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7ba80051d0 con 0x7f7bb8102d20 2024-09-19T06:57:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bc2064740 1 Processor -- start 2024-09-19T06:57:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bc2064740 1 -- start start 2024-09-19T06:57:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bb67fc640 1 --2- 172.21.15.103:0/4092754705 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7ba00a3460 0x7f7ba00a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bc2064740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e5af0 0x5584884e5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bc2064740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884e6410 0x5584884f0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bc2064740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884f0cb0 0x5584884f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bc2064740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55848844a2f0 con 0x5584884e5af0 2024-09-19T06:57:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bc2064740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5584883f2270 con 0x5584884e6410 2024-09-19T06:57:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bc2064740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5584884462b0 con 0x5584884f0cb0 2024-09-19T06:57:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bbd86f640 1 --2- 172.21.15.103:0/4092754705 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7ba00a3460 0x7f7ba00a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bbd06e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e5af0 0x5584884e5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bbd06e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e5af0 0x5584884e5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38362/0 (socket says 172.21.15.103:38362) 2024-09-19T06:57:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bbd06e640 1 -- 172.21.15.103:0/2963391502 learned_addr learned my addr 172.21.15.103:0/2963391502 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bb67fc640 1 -- 172.21.15.103:0/4092754705 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f7ba80d0470 con 0x7f7bb8102d20 2024-09-19T06:57:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bbd86f640 1 --2- 172.21.15.103:0/2963391502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884f0cb0 0x5584884f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bbc86d640 1 --2- 172.21.15.103:0/2963391502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884e6410 0x5584884f0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bbd06e640 1 -- 172.21.15.103:0/2963391502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884f0cb0 msgr2=0x5584884f30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bbd06e640 1 --2- 172.21.15.103:0/2963391502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884f0cb0 0x5584884f30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bbd06e640 1 -- 172.21.15.103:0/2963391502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884e6410 msgr2=0x5584884f0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.693+0000 7f7bbd06e640 1 --2- 172.21.15.103:0/2963391502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884e6410 0x5584884f0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.694+0000 7f7bbd06e640 1 -- 172.21.15.103:0/2963391502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584883f1340 con 0x5584884e5af0 2024-09-19T06:57:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.694+0000 7f7bbd86f640 1 --2- 172.21.15.103:0/4092754705 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7ba00a3460 0x7f7ba00a5920 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000bd50 tx=0x7f7bb0014840 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.694+0000 7f7bbd06e640 1 --2- 172.21.15.103:0/2963391502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e5af0 0x5584884e5ed0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80a4e10 tx=0x7f7ba809cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daaefac840ec8843 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.694+0000 7f7b9f7fe640 1 -- 172.21.15.103:0/2963391502 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba801f070 con 0x5584884e5af0 2024-09-19T06:57:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.694+0000 7f7b9f7fe640 1 -- 172.21.15.103:0/2963391502 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba80a5340 con 0x5584884e5af0 2024-09-19T06:57:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.694+0000 7f7b9f7fe640 1 -- 172.21.15.103:0/2963391502 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8098ed0 con 0x5584884e5af0 2024-09-19T06:57:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.695+0000 7f7bc2064740 1 -- 172.21.15.103:0/2963391502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e5af0 msgr2=0x5584884e5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.695+0000 7f7bc2064740 1 --2- 172.21.15.103:0/2963391502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e5af0 0x5584884e5ed0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80a4e10 tx=0x7f7ba809cdd0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.695+0000 7f7bc2064740 1 -- 172.21.15.103:0/2963391502 shutdown_connections 2024-09-19T06:57:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.695+0000 7f7bc2064740 1 --2- 172.21.15.103:0/2963391502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884f0cb0 0x5584884f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.695+0000 7f7bc2064740 1 --2- 172.21.15.103:0/2963391502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884e6410 0x5584884f0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.695+0000 7f7bc2064740 1 --2- 172.21.15.103:0/2963391502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e5af0 0x5584884e5ed0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.695+0000 7f7bc2064740 1 -- 172.21.15.103:0/2963391502 >> 172.21.15.103:0/2963391502 conn(0x5584884e2060 msgr2=0x5584884e2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.695+0000 7f7bc2064740 1 -- 172.21.15.103:0/2963391502 shutdown_connections 2024-09-19T06:57:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.695+0000 7f7bc2064740 1 -- 172.21.15.103:0/2963391502 wait complete. 2024-09-19T06:57:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.695+0000 7f7bc2064740 1 -- start start 2024-09-19T06:57:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.695+0000 7f7bc2064740 1 Processor -- start 2024-09-19T06:57:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bc2064740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884e5af0 0x5584884edbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bc2064740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e6410 0x5584884ee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bc2064740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884f0cb0 0x5584884ee640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bc2064740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5584884f3880 con 0x5584884e6410 2024-09-19T06:57:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bc2064740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5584883d3290 con 0x5584884f0cb0 2024-09-19T06:57:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bc2064740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5584884f3a00 con 0x5584884e5af0 2024-09-19T06:57:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bc2064740 10 client.0 _ll_register_callbacks cb 0x558488404140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bbc86d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e6410 0x5584884ee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bbc86d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e6410 0x5584884ee100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38370/0 (socket says 172.21.15.103:38370) 2024-09-19T06:57:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bbc86d640 1 -- 172.21.15.103:0/2989120053 learned_addr learned my addr 172.21.15.103:0/2989120053 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bbd06e640 1 --2- 172.21.15.103:0/2989120053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884e5af0 0x5584884edbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bbd86f640 1 --2- 172.21.15.103:0/2989120053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884f0cb0 0x5584884ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bbc86d640 1 -- 172.21.15.103:0/2989120053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884e5af0 msgr2=0x5584884edbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bbc86d640 1 --2- 172.21.15.103:0/2989120053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884e5af0 0x5584884edbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bbc86d640 1 -- 172.21.15.103:0/2989120053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884f0cb0 msgr2=0x5584884ee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bbc86d640 1 --2- 172.21.15.103:0/2989120053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884f0cb0 0x5584884ee640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bbc86d640 1 -- 172.21.15.103:0/2989120053 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55848815d210 con 0x5584884e6410 2024-09-19T06:57:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bbc86d640 1 --2- 172.21.15.103:0/2989120053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e6410 0x5584884ee100 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009900 tx=0x7f7bac0318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcea8d1061ffab5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7b9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7b9cff9640 1 -- 172.21.15.103:0/2989120053 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac035070 con 0x5584884e6410 2024-09-19T06:57:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7b9cff9640 1 -- 172.21.15.103:0/2989120053 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f7bac004240 con 0x5584884e6410 2024-09-19T06:57:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.696+0000 7f7bc2064740 1 -- 172.21.15.103:0/2989120053 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ba80a5340 con 0x5584884e6410 2024-09-19T06:57:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.697+0000 7f7b9cff9640 1 -- 172.21.15.103:0/2989120053 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7bac004440 con 0x5584884e6410 2024-09-19T06:57:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.697+0000 7f7bc2064740 10 client.18504 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.697+0000 7f7bc2064740 1 -- 172.21.15.103:0/2989120053 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558488229620 con 0x5584884e6410 2024-09-19T06:57:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.697+0000 7f7b9cff9640 1 -- 172.21.15.103:0/2989120053 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba80a5340 con 0x5584884e6410 2024-09-19T06:57:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.697+0000 7f7b9cff9640 1 -- 172.21.15.103:0/2989120053 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7bac004c10 con 0x5584884e6410 2024-09-19T06:57:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.697+0000 7f7bc2064740 10 client.18504 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.697+0000 7f7bc2064740 10 client.18504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.697+0000 7f7bc2064740 20 client.18504 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.697+0000 7f7bc2064740 1 --2- 172.21.15.103:0/2989120053 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558488500c70 0x558488503060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.697+0000 7f7bc2064740 4 client.18504 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:57:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.697+0000 7f7bc2064740 1 -- 172.21.15.103:0/2989120053 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558488457a20 con 0x558488500c70 2024-09-19T06:57:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.697+0000 7f7bbd06e640 1 --2- 172.21.15.103:0/2989120053 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558488500c70 0x558488503060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.698+0000 7f7b9cff9640 10 client.18504 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.698+0000 7f7bbd06e640 1 --2- 172.21.15.103:0/2989120053 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558488500c70 0x558488503060 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4f9ec5d9435e889 server_cookie=da9f9dcddfe7784a in_seq=0 out_seq=0 2024-09-19T06:57:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.698+0000 7f7b9cff9640 1 -- 172.21.15.103:0/2989120053 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558488457a20 con 0x558488500c70 2024-09-19T06:57:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.698+0000 7f7b9cff9640 10 client.18504 handle_command_reply: tid=0 2024-09-19T06:57:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.698+0000 7f7bc2064740 1 client.18504 shutdown 2024-09-19T06:57:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.699+0000 7f7bc2064740 1 -- 172.21.15.103:0/2989120053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e6410 msgr2=0x5584884ee100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.699+0000 7f7bc2064740 1 --2- 172.21.15.103:0/2989120053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e6410 0x5584884ee100 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009900 tx=0x7f7bac0318d0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.699+0000 7f7bc2064740 1 -- 172.21.15.103:0/2989120053 shutdown_connections 2024-09-19T06:57:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.699+0000 7f7bc2064740 1 --2- 172.21.15.103:0/2989120053 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558488500c70 0x558488503060 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.699+0000 7f7bc2064740 1 --2- 172.21.15.103:0/2989120053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884f0cb0 0x5584884ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.699+0000 7f7bc2064740 1 --2- 172.21.15.103:0/2989120053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e6410 0x5584884ee100 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.699+0000 7f7bc2064740 1 --2- 172.21.15.103:0/2989120053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884e5af0 0x5584884edbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.699+0000 7f7b9cff9640 0 client.18504 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.699+0000 7f7bc2064740 1 -- 172.21.15.103:0/2989120053 >> 172.21.15.103:0/2989120053 conn(0x5584884e2060 msgr2=0x5584884e7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.699+0000 7f7bc2064740 1 -- 172.21.15.103:0/2989120053 shutdown_connections 2024-09-19T06:57:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.699+0000 7f7bc2064740 1 -- 172.21.15.103:0/2989120053 wait complete. 2024-09-19T06:57:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.699+0000 7f7bc2064740 20 client.18504 trim_cache size 0 max 16384 2024-09-19T06:57:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.713+0000 7f7bc2064740 1 Processor -- start 2024-09-19T06:57:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.713+0000 7f7bc2064740 1 -- start start 2024-09-19T06:57:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.713+0000 7f7bc2064740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e5af0 0x5584884ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.713+0000 7f7bc2064740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884e6410 0x5584884ee470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.713+0000 7f7bc2064740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884f0cb0 0x558488508950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.713+0000 7f7bc2064740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb81ad830 con 0x5584884e5af0 2024-09-19T06:57:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.713+0000 7f7bc2064740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb81af370 con 0x5584884e6410 2024-09-19T06:57:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.713+0000 7f7bc2064740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb81adf00 con 0x5584884f0cb0 2024-09-19T06:57:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.713+0000 7f7bbd06e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e5af0 0x5584884ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.714+0000 7f7bbd06e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e5af0 0x5584884ecae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38382/0 (socket says 172.21.15.103:38382) 2024-09-19T06:57:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.714+0000 7f7bbd06e640 1 -- 172.21.15.103:0/843144960 learned_addr learned my addr 172.21.15.103:0/843144960 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.714+0000 7f7bbd86f640 1 --2- 172.21.15.103:0/843144960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884f0cb0 0x558488508950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.714+0000 7f7bbc86d640 1 --2- 172.21.15.103:0/843144960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884e6410 0x5584884ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.714+0000 7f7bbd06e640 1 -- 172.21.15.103:0/843144960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884f0cb0 msgr2=0x558488508950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.714+0000 7f7bbd06e640 1 --2- 172.21.15.103:0/843144960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884f0cb0 0x558488508950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.714+0000 7f7bbd06e640 1 -- 172.21.15.103:0/843144960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884e6410 msgr2=0x5584884ee470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.714+0000 7f7bbd06e640 1 --2- 172.21.15.103:0/843144960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884e6410 0x5584884ee470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.714+0000 7f7bbd06e640 1 -- 172.21.15.103:0/843144960 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584884dc2f0 con 0x5584884e5af0 2024-09-19T06:57:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.714+0000 7f7bbd86f640 1 --2- 172.21.15.103:0/843144960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884f0cb0 0x558488508950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.714+0000 7f7bbc86d640 1 --2- 172.21.15.103:0/843144960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884e6410 0x5584884ee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.714+0000 7f7bbd06e640 1 --2- 172.21.15.103:0/843144960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e5af0 0x5584884ecae0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800efd0 tx=0x7f7ba809f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b48a90bf183554a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.714+0000 7f7b9effd640 1 -- 172.21.15.103:0/843144960 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba801f040 con 0x5584884e5af0 2024-09-19T06:57:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.714+0000 7f7b9effd640 1 -- 172.21.15.103:0/843144960 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba8005540 con 0x5584884e5af0 2024-09-19T06:57:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.714+0000 7f7b9effd640 1 -- 172.21.15.103:0/843144960 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80a5110 con 0x5584884e5af0 2024-09-19T06:57:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.715+0000 7f7bc2064740 1 -- 172.21.15.103:0/843144960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e5af0 msgr2=0x5584884ecae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.715+0000 7f7bc2064740 1 --2- 172.21.15.103:0/843144960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e5af0 0x5584884ecae0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800efd0 tx=0x7f7ba809f040 comp rx=0 tx=0).stop 2024-09-19T06:57:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.715+0000 7f7bc2064740 1 -- 172.21.15.103:0/843144960 shutdown_connections 2024-09-19T06:57:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.715+0000 7f7bc2064740 1 --2- 172.21.15.103:0/843144960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884f0cb0 0x558488508950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.715+0000 7f7bc2064740 1 --2- 172.21.15.103:0/843144960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884e6410 0x5584884ee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.715+0000 7f7bc2064740 1 --2- 172.21.15.103:0/843144960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e5af0 0x5584884ecae0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.715+0000 7f7bc2064740 1 -- 172.21.15.103:0/843144960 >> 172.21.15.103:0/843144960 conn(0x5584884e2060 msgr2=0x5584884efc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.715+0000 7f7bc2064740 1 -- 172.21.15.103:0/843144960 shutdown_connections 2024-09-19T06:57:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.715+0000 7f7bc2064740 1 -- 172.21.15.103:0/843144960 wait complete. 2024-09-19T06:57:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.715+0000 7f7bc2064740 1 -- start start 2024-09-19T06:57:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.715+0000 7f7bc2064740 1 Processor -- start 2024-09-19T06:57:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bc2064740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884e5af0 0x5584884f9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bc2064740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e6410 0x5584884f4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bc2064740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884f0cb0 0x5584884f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bc2064740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb81aeca0 con 0x5584884e6410 2024-09-19T06:57:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bc2064740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb81ae5d0 con 0x5584884f0cb0 2024-09-19T06:57:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bc2064740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb81ab830 con 0x5584884e5af0 2024-09-19T06:57:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bc2064740 10 client.0 _ll_register_callbacks cb 0x7f7bac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:57:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bbc86d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e6410 0x5584884f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bbc86d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e6410 0x5584884f4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38386/0 (socket says 172.21.15.103:38386) 2024-09-19T06:57:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bbc86d640 1 -- 172.21.15.103:0/767662073 learned_addr learned my addr 172.21.15.103:0/767662073 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:57:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bbd06e640 1 --2- 172.21.15.103:0/767662073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884e5af0 0x5584884f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bbd86f640 1 --2- 172.21.15.103:0/767662073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884f0cb0 0x5584884f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bbc86d640 1 -- 172.21.15.103:0/767662073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884e5af0 msgr2=0x5584884f9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bbc86d640 1 --2- 172.21.15.103:0/767662073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884e5af0 0x5584884f9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bbc86d640 1 -- 172.21.15.103:0/767662073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884f0cb0 msgr2=0x5584884f4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:57:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bbc86d640 1 --2- 172.21.15.103:0/767662073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884f0cb0 0x5584884f4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bbc86d640 1 -- 172.21.15.103:0/767662073 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584884f50f0 con 0x5584884e6410 2024-09-19T06:57:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bbd86f640 1 --2- 172.21.15.103:0/767662073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884f0cb0 0x5584884f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bbc86d640 1 --2- 172.21.15.103:0/767662073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e6410 0x5584884f4570 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f7bac003670 tx=0x7f7bac003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c42ab3ee8e76de9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7b9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:57:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7b9cff9640 1 -- 172.21.15.103:0/767662073 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac035070 con 0x5584884e6410 2024-09-19T06:57:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7b9cff9640 1 -- 172.21.15.103:0/767662073 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f7bac004fe0 con 0x5584884e6410 2024-09-19T06:57:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7bc2064740 1 -- 172.21.15.103:0/767662073 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ba8005540 con 0x5584884e6410 2024-09-19T06:57:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.716+0000 7f7b9cff9640 1 -- 172.21.15.103:0/767662073 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7bac050040 con 0x5584884e6410 2024-09-19T06:57:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.717+0000 7f7bc2064740 10 client.18510 fetch_fsmap learned FSMap version 27 2024-09-19T06:57:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.717+0000 7f7bc2064740 1 -- 172.21.15.103:0/767662073 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55848856b710 con 0x5584884e6410 2024-09-19T06:57:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.717+0000 7f7b9cff9640 1 -- 172.21.15.103:0/767662073 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba8005540 con 0x5584884e6410 2024-09-19T06:57:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.717+0000 7f7b9cff9640 1 -- 172.21.15.103:0/767662073 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7bac0048c0 con 0x5584884e6410 2024-09-19T06:57:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.717+0000 7f7bc2064740 10 client.18510 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:57:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.717+0000 7f7bc2064740 10 client.18510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:57:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.717+0000 7f7bc2064740 20 client.18510 populate_metadata read hostname 'smithi103' 2024-09-19T06:57:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.717+0000 7f7bc2064740 1 --2- 172.21.15.103:0/767662073 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55848856be90 0x55848856e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.717+0000 7f7bc2064740 4 client.18510 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:57:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.717+0000 7f7bc2064740 1 -- 172.21.15.103:0/767662073 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558488451d70 con 0x55848856be90 2024-09-19T06:57:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.717+0000 7f7bbd06e640 1 --2- 172.21.15.103:0/767662073 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55848856be90 0x55848856e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.718+0000 7f7bbd06e640 1 --2- 172.21.15.103:0/767662073 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55848856be90 0x55848856e2b0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74113b305e46acd2 server_cookie=94bcb0830ce072fa in_seq=0 out_seq=0 2024-09-19T06:57:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.718+0000 7f7b9cff9640 10 client.18510 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.730+0000 7f7b9cff9640 1 -- 172.21.15.103:0/767662073 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558488451d70 con 0x55848856be90 2024-09-19T06:57:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.730+0000 7f7b9cff9640 10 client.18510 handle_command_reply: tid=0 2024-09-19T06:57:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.730+0000 7f7bc2064740 1 client.18510 shutdown 2024-09-19T06:57:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.730+0000 7f7bc2064740 1 -- 172.21.15.103:0/767662073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e6410 msgr2=0x5584884f4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.730+0000 7f7bc2064740 1 --2- 172.21.15.103:0/767662073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e6410 0x5584884f4570 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f7bac003670 tx=0x7f7bac003f10 comp rx=0 tx=0).stop 2024-09-19T06:57:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.730+0000 7f7bc2064740 1 -- 172.21.15.103:0/767662073 shutdown_connections 2024-09-19T06:57:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.730+0000 7f7bc2064740 1 --2- 172.21.15.103:0/767662073 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55848856be90 0x55848856e2b0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.730+0000 7f7bc2064740 1 --2- 172.21.15.103:0/767662073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584884f0cb0 0x5584884f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.730+0000 7f7b9cff9640 0 client.18510 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:57:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.730+0000 7f7bc2064740 1 --2- 172.21.15.103:0/767662073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584884e6410 0x5584884f4570 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.730+0000 7f7bc2064740 1 --2- 172.21.15.103:0/767662073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584884e5af0 0x5584884f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.730+0000 7f7bc2064740 1 -- 172.21.15.103:0/767662073 >> 172.21.15.103:0/767662073 conn(0x5584884e2060 msgr2=0x5584884e7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.730+0000 7f7bc2064740 1 -- 172.21.15.103:0/767662073 shutdown_connections 2024-09-19T06:57:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.730+0000 7f7bc2064740 1 -- 172.21.15.103:0/767662073 wait complete. 2024-09-19T06:57:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.730+0000 7f7bc2064740 20 client.18510 trim_cache size 0 max 16384 2024-09-19T06:57:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:57:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:57:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:57:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.731+0000 7f7bbf2d2640 1 -- 172.21.15.103:0/4092754705 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7ba00a3460 msgr2=0x7f7ba00a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.731+0000 7f7bbf2d2640 1 --2- 172.21.15.103:0/4092754705 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7ba00a3460 0x7f7ba00a5920 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000bd50 tx=0x7f7bb0014840 comp rx=0 tx=0).stop 2024-09-19T06:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.731+0000 7f7bbf2d2640 1 -- 172.21.15.103:0/4092754705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bb8102d20 msgr2=0x7f7bb8198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.731+0000 7f7bbf2d2640 1 --2- 172.21.15.103:0/4092754705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bb8102d20 0x7f7bb8198350 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800ea80 tx=0x7f7ba800ef40 comp rx=0 tx=0).stop 2024-09-19T06:57:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.731+0000 7f7bbf2d2640 1 -- 172.21.15.103:0/4092754705 shutdown_connections 2024-09-19T06:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.731+0000 7f7bbf2d2640 1 --2- 172.21.15.103:0/4092754705 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7ba00a3460 0x7f7ba00a5920 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.731+0000 7f7bbf2d2640 1 --2- 172.21.15.103:0/4092754705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb810c0f0 0x7f7bb819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.731+0000 7f7bbf2d2640 1 --2- 172.21.15.103:0/4092754705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bb8103640 0x7f7bb8198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.731+0000 7f7bbf2d2640 1 --2- 172.21.15.103:0/4092754705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bb8102d20 0x7f7bb8198350 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.731+0000 7f7bbf2d2640 1 -- 172.21.15.103:0/4092754705 >> 172.21.15.103:0/4092754705 conn(0x7f7bb8077990 msgr2=0x7f7bb80fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.731+0000 7f7bbf2d2640 1 -- 172.21.15.103:0/4092754705 shutdown_connections 2024-09-19T06:57:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:57:57.731+0000 7f7bbf2d2640 1 -- 172.21.15.103:0/4092754705 wait complete. 2024-09-19T06:57:58.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:57:58 smithi103 ceph-mon[31158]: pgmap v269: 97 pgs: 97 active+clean; 2.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-09-19T06:57:58.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:57:58 smithi134 ceph-mon[35356]: pgmap v269: 97 pgs: 97 active+clean; 2.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-09-19T06:57:58.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:57:58 smithi174 ceph-mon[35637]: pgmap v269: 97 pgs: 97 active+clean; 2.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-09-19T06:58:00.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:00 smithi103 ceph-mon[31158]: pgmap v270: 97 pgs: 97 active+clean; 2.8 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T06:58:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:00 smithi134 ceph-mon[35356]: pgmap v270: 97 pgs: 97 active+clean; 2.8 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T06:58:00.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:00 smithi174 ceph-mon[35637]: pgmap v270: 97 pgs: 97 active+clean; 2.8 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T06:58:02.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:02 smithi103 ceph-mon[31158]: pgmap v271: 97 pgs: 97 active+clean; 2.8 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:58:02.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:02 smithi134 ceph-mon[35356]: pgmap v271: 97 pgs: 97 active+clean; 2.8 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:58:03.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:02 smithi174 ceph-mon[35637]: pgmap v271: 97 pgs: 97 active+clean; 2.8 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:58:04.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:04 smithi103 ceph-mon[31158]: pgmap v272: 97 pgs: 97 active+clean; 2.8 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:58:04.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:04 smithi134 ceph-mon[35356]: pgmap v272: 97 pgs: 97 active+clean; 2.8 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:58:04.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:04 smithi174 ceph-mon[35637]: pgmap v272: 97 pgs: 97 active+clean; 2.8 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T06:58:06.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:06 smithi103 ceph-mon[31158]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T06:58:06.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:06 smithi103 ceph-mon[31158]: pgmap v274: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-09-19T06:58:06.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:06 smithi134 ceph-mon[35356]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T06:58:06.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:06 smithi134 ceph-mon[35356]: pgmap v274: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-09-19T06:58:06.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:06 smithi174 ceph-mon[35637]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T06:58:06.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:06 smithi174 ceph-mon[35637]: pgmap v274: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-09-19T06:58:08.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:08 smithi103 ceph-mon[31158]: pgmap v275: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-09-19T06:58:08.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:08 smithi134 ceph-mon[35356]: pgmap v275: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-09-19T06:58:08.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:08 smithi174 ceph-mon[35637]: pgmap v275: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-09-19T06:58:10.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:10 smithi103 ceph-mon[31158]: pgmap v276: 97 pgs: 97 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 101 KiB/s wr, 36 op/s 2024-09-19T06:58:10.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:10 smithi134 ceph-mon[35356]: pgmap v276: 97 pgs: 97 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 101 KiB/s wr, 36 op/s 2024-09-19T06:58:10.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:10 smithi174 ceph-mon[35637]: pgmap v276: 97 pgs: 97 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 101 KiB/s wr, 36 op/s 2024-09-19T06:58:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.830+0000 7fa761f52640 1 -- 172.21.15.103:0/3045062420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa75c10c110 msgr2=0x7fa75c10e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.830+0000 7fa761f52640 1 --2- 172.21.15.103:0/3045062420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa75c10c110 0x7fa75c10e590 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa754009ca0 tx=0x7fa75402a950 comp rx=0 tx=0).stop 2024-09-19T06:58:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.830+0000 7fa761f52640 1 -- 172.21.15.103:0/3045062420 shutdown_connections 2024-09-19T06:58:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.830+0000 7fa761f52640 1 --2- 172.21.15.103:0/3045062420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa75c10c110 0x7fa75c10e590 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.830+0000 7fa761f52640 1 --2- 172.21.15.103:0/3045062420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa75c1035e0 0x7fa75c10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.830+0000 7fa761f52640 1 --2- 172.21.15.103:0/3045062420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa75c102cc0 0x7fa75c1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.830+0000 7fa761f52640 1 -- 172.21.15.103:0/3045062420 >> 172.21.15.103:0/3045062420 conn(0x7fa75c0fc6b0 msgr2=0x7fa75c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.830+0000 7fa761f52640 1 -- 172.21.15.103:0/3045062420 shutdown_connections 2024-09-19T06:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.830+0000 7fa761f52640 1 -- 172.21.15.103:0/3045062420 wait complete. 2024-09-19T06:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.830+0000 7fa761f52640 1 Processor -- start 2024-09-19T06:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa761f52640 1 -- start start 2024-09-19T06:58:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa761f52640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa75c102cc0 0x7fa75c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa761f52640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa75c1035e0 0x7fa75c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa761f52640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa75c10c110 0x7fa75c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa761f52640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c111ef0 con 0x7fa75c1035e0 2024-09-19T06:58:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa761f52640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c111d70 con 0x7fa75c10c110 2024-09-19T06:58:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa761f52640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c112070 con 0x7fa75c102cc0 2024-09-19T06:58:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa75affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa75c1035e0 0x7fa75c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa75affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa75c1035e0 0x7fa75c198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59216/0 (socket says 172.21.15.103:59216) 2024-09-19T06:58:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa75affd640 1 -- 172.21.15.103:0/884837071 learned_addr learned my addr 172.21.15.103:0/884837071 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa75b7fe640 1 --2- 172.21.15.103:0/884837071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa75c102cc0 0x7fa75c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa75bfff640 1 --2- 172.21.15.103:0/884837071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa75c10c110 0x7fa75c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa75affd640 1 -- 172.21.15.103:0/884837071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa75c102cc0 msgr2=0x7fa75c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa75affd640 1 --2- 172.21.15.103:0/884837071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa75c102cc0 0x7fa75c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa75affd640 1 -- 172.21.15.103:0/884837071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa75c10c110 msgr2=0x7fa75c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa75affd640 1 --2- 172.21.15.103:0/884837071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa75c10c110 0x7fa75c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa75affd640 1 -- 172.21.15.103:0/884837071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75c19d330 con 0x7fa75c1035e0 2024-09-19T06:58:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.831+0000 7fa75affd640 1 --2- 172.21.15.103:0/884837071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa75c1035e0 0x7fa75c198880 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa75000e9f0 tx=0x7fa75000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.832+0000 7fa758ff9640 1 -- 172.21.15.103:0/884837071 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75000ccc0 con 0x7fa75c1035e0 2024-09-19T06:58:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.832+0000 7fa758ff9640 1 -- 172.21.15.103:0/884837071 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa750010910 con 0x7fa75c1035e0 2024-09-19T06:58:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.832+0000 7fa758ff9640 1 -- 172.21.15.103:0/884837071 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750018920 con 0x7fa75c1035e0 2024-09-19T06:58:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.832+0000 7fa761f52640 1 -- 172.21.15.103:0/884837071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75c19d620 con 0x7fa75c1035e0 2024-09-19T06:58:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.832+0000 7fa761f52640 1 -- 172.21.15.103:0/884837071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa75c1a4eb0 con 0x7fa75c1035e0 2024-09-19T06:58:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.834+0000 7fa758ff9640 1 -- 172.21.15.103:0/884837071 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa750010430 con 0x7fa75c1035e0 2024-09-19T06:58:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.834+0000 7fa764dbf740 1 Processor -- start 2024-09-19T06:58:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.834+0000 7fa764dbf740 1 -- start start 2024-09-19T06:58:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.834+0000 7fa764dbf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5aaf0 0x55d363f5aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.834+0000 7fa764dbf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5b410 0x55d363f65770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.834+0000 7fa764dbf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f65cb0 0x55d363f680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.834+0000 7fa764dbf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d363ebf2f0 con 0x55d363f5b410 2024-09-19T06:58:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.834+0000 7fa764dbf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d363e67270 con 0x55d363f5aaf0 2024-09-19T06:58:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.834+0000 7fa764dbf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d363ebb2b0 con 0x55d363f65cb0 2024-09-19T06:58:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa75b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5b410 0x55d363f65770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa75bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5aaf0 0x55d363f5aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa75b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5b410 0x55d363f65770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59232/0 (socket says 172.21.15.103:59232) 2024-09-19T06:58:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa75b7fe640 1 -- 172.21.15.103:0/3255206408 learned_addr learned my addr 172.21.15.103:0/3255206408 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa75affd640 1 --2- 172.21.15.103:0/3255206408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f65cb0 0x55d363f680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa758ff9640 1 --2- 172.21.15.103:0/884837071 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7440a34e0 0x7fa7440a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa75bfff640 1 --2- 172.21.15.103:0/884837071 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7440a34e0 0x7fa7440a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa75b7fe640 1 -- 172.21.15.103:0/3255206408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f65cb0 msgr2=0x55d363f680a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa75b7fe640 1 --2- 172.21.15.103:0/3255206408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f65cb0 0x55d363f680a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa75b7fe640 1 -- 172.21.15.103:0/3255206408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5aaf0 msgr2=0x55d363f5aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa75b7fe640 1 --2- 172.21.15.103:0/3255206408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5aaf0 0x55d363f5aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa75b7fe640 1 -- 172.21.15.103:0/3255206408 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d363e66340 con 0x55d363f5b410 2024-09-19T06:58:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa75bfff640 1 --2- 172.21.15.103:0/3255206408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5aaf0 0x55d363f5aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:58:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa758ff9640 1 -- 172.21.15.103:0/884837071 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8600+0+0 (secure 0 0 0) 0x7fa7500cff60 con 0x7fa75c1035e0 2024-09-19T06:58:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa75b7fe640 1 --2- 172.21.15.103:0/3255206408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5b410 0x55d363f65770 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009d20 tx=0x7fa74c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc5d791648281f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa75bfff640 1 --2- 172.21.15.103:0/884837071 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7440a34e0 0x7fa7440a59a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa754009430 tx=0x7fa75402a950 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa741ffb640 1 -- 172.21.15.103:0/3255206408 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c01e070 con 0x55d363f5b410 2024-09-19T06:58:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa741ffb640 1 -- 172.21.15.103:0/3255206408 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74c019b10 con 0x55d363f5b410 2024-09-19T06:58:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.835+0000 7fa741ffb640 1 -- 172.21.15.103:0/3255206408 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c00ebb0 con 0x55d363f5b410 2024-09-19T06:58:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.836+0000 7fa764dbf740 1 -- 172.21.15.103:0/3255206408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5b410 msgr2=0x55d363f65770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.836+0000 7fa764dbf740 1 --2- 172.21.15.103:0/3255206408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5b410 0x55d363f65770 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009d20 tx=0x7fa74c011810 comp rx=0 tx=0).stop 2024-09-19T06:58:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.836+0000 7fa764dbf740 1 -- 172.21.15.103:0/3255206408 shutdown_connections 2024-09-19T06:58:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.836+0000 7fa764dbf740 1 --2- 172.21.15.103:0/3255206408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f65cb0 0x55d363f680a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.836+0000 7fa764dbf740 1 --2- 172.21.15.103:0/3255206408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5b410 0x55d363f65770 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.836+0000 7fa764dbf740 1 --2- 172.21.15.103:0/3255206408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5aaf0 0x55d363f5aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.836+0000 7fa764dbf740 1 -- 172.21.15.103:0/3255206408 >> 172.21.15.103:0/3255206408 conn(0x55d363f57060 msgr2=0x55d363f57440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.836+0000 7fa764dbf740 1 -- 172.21.15.103:0/3255206408 shutdown_connections 2024-09-19T06:58:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.836+0000 7fa764dbf740 1 -- 172.21.15.103:0/3255206408 wait complete. 2024-09-19T06:58:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa764dbf740 1 -- start start 2024-09-19T06:58:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa764dbf740 1 Processor -- start 2024-09-19T06:58:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa764dbf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5aaf0 0x55d363f62b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa764dbf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f5b410 0x55d363f630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa764dbf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f65cb0 0x55d363f635e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa764dbf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d363f68880 con 0x55d363f5aaf0 2024-09-19T06:58:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa764dbf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d363e48290 con 0x55d363f65cb0 2024-09-19T06:58:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa764dbf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d363f68a00 con 0x55d363f5b410 2024-09-19T06:58:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa764dbf740 10 client.0 _ll_register_callbacks cb 0x55d363e79140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa75b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5aaf0 0x55d363f62b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa75b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5aaf0 0x55d363f62b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59244/0 (socket says 172.21.15.103:59244) 2024-09-19T06:58:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa75b7fe640 1 -- 172.21.15.103:0/894466117 learned_addr learned my addr 172.21.15.103:0/894466117 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa75affd640 1 --2- 172.21.15.103:0/894466117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f5b410 0x55d363f630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa75bfff640 1 --2- 172.21.15.103:0/894466117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f65cb0 0x55d363f635e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa75b7fe640 1 -- 172.21.15.103:0/894466117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f5b410 msgr2=0x55d363f630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa75b7fe640 1 --2- 172.21.15.103:0/894466117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f5b410 0x55d363f630a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa75b7fe640 1 -- 172.21.15.103:0/894466117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f65cb0 msgr2=0x55d363f635e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa75b7fe640 1 --2- 172.21.15.103:0/894466117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f65cb0 0x55d363f635e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.837+0000 7fa75b7fe640 1 -- 172.21.15.103:0/894466117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d363bd2210 con 0x55d363f5aaf0 2024-09-19T06:58:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.838+0000 7fa75b7fe640 1 --2- 172.21.15.103:0/894466117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5aaf0 0x55d363f62b60 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa74c019690 tx=0x7fa74c00e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e937f2038b0780df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.838+0000 7fa73affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.838+0000 7fa73affd640 1 -- 172.21.15.103:0/894466117 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c027020 con 0x55d363f5aaf0 2024-09-19T06:58:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.838+0000 7fa73affd640 1 -- 172.21.15.103:0/894466117 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8600+0+0 (secure 0 0 0) 0x7fa74c00eaf0 con 0x55d363f5aaf0 2024-09-19T06:58:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.838+0000 7fa764dbf740 1 -- 172.21.15.103:0/894466117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa74c019b10 con 0x55d363f5aaf0 2024-09-19T06:58:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.838+0000 7fa73affd640 1 -- 172.21.15.103:0/894466117 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa74c032050 con 0x55d363f5aaf0 2024-09-19T06:58:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.838+0000 7fa764dbf740 10 client.18522 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.838+0000 7fa764dbf740 1 -- 172.21.15.103:0/894466117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d363c9e620 con 0x55d363f5aaf0 2024-09-19T06:58:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.838+0000 7fa73affd640 1 -- 172.21.15.103:0/894466117 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74c019b10 con 0x55d363f5aaf0 2024-09-19T06:58:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.838+0000 7fa73affd640 1 -- 172.21.15.103:0/894466117 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa74c0046c0 con 0x55d363f5aaf0 2024-09-19T06:58:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.838+0000 7fa764dbf740 10 client.18522 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.838+0000 7fa764dbf740 10 client.18522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:58:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.838+0000 7fa764dbf740 20 client.18522 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.838+0000 7fa764dbf740 1 --2- 172.21.15.103:0/894466117 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d363f75c20 0x55d363f78010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.838+0000 7fa764dbf740 4 client.18522 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:58:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.838+0000 7fa764dbf740 1 -- 172.21.15.103:0/894466117 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d363ecca20 con 0x55d363f75c20 2024-09-19T06:58:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.839+0000 7fa75affd640 1 --2- 172.21.15.103:0/894466117 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d363f75c20 0x55d363f78010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.839+0000 7fa73affd640 10 client.18522 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.839+0000 7fa75affd640 1 --2- 172.21.15.103:0/894466117 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d363f75c20 0x55d363f78010 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99a639747975a253 server_cookie=6b147302eac57df3 in_seq=0 out_seq=0 2024-09-19T06:58:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.840+0000 7fa73affd640 1 -- 172.21.15.103:0/894466117 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d363ecca20 con 0x55d363f75c20 2024-09-19T06:58:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.840+0000 7fa73affd640 10 client.18522 handle_command_reply: tid=0 2024-09-19T06:58:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.840+0000 7fa764dbf740 1 client.18522 shutdown 2024-09-19T06:58:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.840+0000 7fa764dbf740 1 -- 172.21.15.103:0/894466117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5aaf0 msgr2=0x55d363f62b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.840+0000 7fa764dbf740 1 --2- 172.21.15.103:0/894466117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5aaf0 0x55d363f62b60 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa74c019690 tx=0x7fa74c00e490 comp rx=0 tx=0).stop 2024-09-19T06:58:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.840+0000 7fa764dbf740 1 -- 172.21.15.103:0/894466117 shutdown_connections 2024-09-19T06:58:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.840+0000 7fa764dbf740 1 --2- 172.21.15.103:0/894466117 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d363f75c20 0x55d363f78010 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.840+0000 7fa764dbf740 1 --2- 172.21.15.103:0/894466117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f65cb0 0x55d363f635e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.840+0000 7fa764dbf740 1 --2- 172.21.15.103:0/894466117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f5b410 0x55d363f630a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.840+0000 7fa764dbf740 1 --2- 172.21.15.103:0/894466117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5aaf0 0x55d363f62b60 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.840+0000 7fa73affd640 0 client.18522 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.840+0000 7fa764dbf740 1 -- 172.21.15.103:0/894466117 >> 172.21.15.103:0/894466117 conn(0x55d363f57060 msgr2=0x55d363f5cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.840+0000 7fa764dbf740 1 -- 172.21.15.103:0/894466117 shutdown_connections 2024-09-19T06:58:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.840+0000 7fa764dbf740 1 -- 172.21.15.103:0/894466117 wait complete. 2024-09-19T06:58:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.840+0000 7fa764dbf740 20 client.18522 trim_cache size 0 max 16384 2024-09-19T06:58:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.855+0000 7fa764dbf740 1 Processor -- start 2024-09-19T06:58:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.855+0000 7fa764dbf740 1 -- start start 2024-09-19T06:58:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.855+0000 7fa764dbf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5aaf0 0x55d363f61a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.855+0000 7fa764dbf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5b410 0x55d363f5d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.855+0000 7fa764dbf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f65cb0 0x55d363f63410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.855+0000 7fa764dbf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1ad7f0 con 0x55d363f5b410 2024-09-19T06:58:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.855+0000 7fa764dbf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1af330 con 0x55d363f5aaf0 2024-09-19T06:58:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.855+0000 7fa764dbf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1adec0 con 0x55d363f65cb0 2024-09-19T06:58:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.855+0000 7fa75affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5b410 0x55d363f5d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.855+0000 7fa75affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5b410 0x55d363f5d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59260/0 (socket says 172.21.15.103:59260) 2024-09-19T06:58:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.855+0000 7fa75affd640 1 -- 172.21.15.103:0/101674033 learned_addr learned my addr 172.21.15.103:0/101674033 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.855+0000 7fa75b7fe640 1 --2- 172.21.15.103:0/101674033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5aaf0 0x55d363f61a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.855+0000 7fa75bfff640 1 --2- 172.21.15.103:0/101674033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f65cb0 0x55d363f63410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.855+0000 7fa75affd640 1 -- 172.21.15.103:0/101674033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f65cb0 msgr2=0x55d363f63410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.855+0000 7fa75affd640 1 --2- 172.21.15.103:0/101674033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f65cb0 0x55d363f63410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.855+0000 7fa75affd640 1 -- 172.21.15.103:0/101674033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5aaf0 msgr2=0x55d363f61a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.856+0000 7fa75affd640 1 --2- 172.21.15.103:0/101674033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5aaf0 0x55d363f61a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.856+0000 7fa75affd640 1 -- 172.21.15.103:0/101674033 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d363f581f0 con 0x55d363f5b410 2024-09-19T06:58:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.856+0000 7fa75affd640 1 --2- 172.21.15.103:0/101674033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5b410 0x55d363f5d4d0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fa75000c6a0 tx=0x7fa7500d2470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3141acbad86054e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.856+0000 7fa7417fa640 1 -- 172.21.15.103:0/101674033 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7500a05a0 con 0x55d363f5b410 2024-09-19T06:58:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.856+0000 7fa7417fa640 1 -- 172.21.15.103:0/101674033 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa750010720 con 0x55d363f5b410 2024-09-19T06:58:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.856+0000 7fa7417fa640 1 -- 172.21.15.103:0/101674033 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7500d20b0 con 0x55d363f5b410 2024-09-19T06:58:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 -- 172.21.15.103:0/101674033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5b410 msgr2=0x55d363f5d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 --2- 172.21.15.103:0/101674033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5b410 0x55d363f5d4d0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fa75000c6a0 tx=0x7fa7500d2470 comp rx=0 tx=0).stop 2024-09-19T06:58:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 -- 172.21.15.103:0/101674033 shutdown_connections 2024-09-19T06:58:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 --2- 172.21.15.103:0/101674033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f65cb0 0x55d363f63410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 --2- 172.21.15.103:0/101674033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5b410 0x55d363f5d4d0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 --2- 172.21.15.103:0/101674033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5aaf0 0x55d363f61a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 -- 172.21.15.103:0/101674033 >> 172.21.15.103:0/101674033 conn(0x55d363f57060 msgr2=0x55d363f64be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 -- 172.21.15.103:0/101674033 shutdown_connections 2024-09-19T06:58:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 -- 172.21.15.103:0/101674033 wait complete. 2024-09-19T06:58:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 -- start start 2024-09-19T06:58:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 Processor -- start 2024-09-19T06:58:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5aaf0 0x55d363f6e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5b410 0x55d363f6e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f65cb0 0x55d363f695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1aec60 con 0x55d363f5aaf0 2024-09-19T06:58:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1ae590 con 0x55d363f5b410 2024-09-19T06:58:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.857+0000 7fa764dbf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1ab7f0 con 0x55d363f65cb0 2024-09-19T06:58:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa764dbf740 10 client.0 _ll_register_callbacks cb 0x7fa74c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa75b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5aaf0 0x55d363f6e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa75b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5aaf0 0x55d363f6e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59266/0 (socket says 172.21.15.103:59266) 2024-09-19T06:58:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa75b7fe640 1 -- 172.21.15.103:0/2899728616 learned_addr learned my addr 172.21.15.103:0/2899728616 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa75bfff640 1 --2- 172.21.15.103:0/2899728616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f65cb0 0x55d363f695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa75affd640 1 --2- 172.21.15.103:0/2899728616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5b410 0x55d363f6e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa75b7fe640 1 -- 172.21.15.103:0/2899728616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f65cb0 msgr2=0x55d363f695c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa75b7fe640 1 --2- 172.21.15.103:0/2899728616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f65cb0 0x55d363f695c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa75b7fe640 1 -- 172.21.15.103:0/2899728616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5b410 msgr2=0x55d363f6e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa75b7fe640 1 --2- 172.21.15.103:0/2899728616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5b410 0x55d363f6e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa75b7fe640 1 -- 172.21.15.103:0/2899728616 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d363f69df0 con 0x55d363f5aaf0 2024-09-19T06:58:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa75affd640 1 --2- 172.21.15.103:0/2899728616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5b410 0x55d363f6e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:58:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa75b7fe640 1 --2- 172.21.15.103:0/2899728616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5aaf0 0x55d363f6e000 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fa74c011d10 tx=0x7fa74c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ade7ca83e34d80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa73affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa73affd640 1 -- 172.21.15.103:0/2899728616 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c027070 con 0x55d363f5aaf0 2024-09-19T06:58:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa73affd640 1 -- 172.21.15.103:0/2899728616 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8600+0+0 (secure 0 0 0) 0x7fa74c00e880 con 0x55d363f5aaf0 2024-09-19T06:58:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa764dbf740 1 -- 172.21.15.103:0/2899728616 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa750010720 con 0x55d363f5aaf0 2024-09-19T06:58:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa73affd640 1 -- 172.21.15.103:0/2899728616 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa74c01e070 con 0x55d363f5aaf0 2024-09-19T06:58:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa764dbf740 10 client.18528 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.858+0000 7fa764dbf740 1 -- 172.21.15.103:0/2899728616 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d363fe4730 con 0x55d363f5aaf0 2024-09-19T06:58:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.859+0000 7fa73affd640 1 -- 172.21.15.103:0/2899728616 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa750010720 con 0x55d363f5aaf0 2024-09-19T06:58:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.859+0000 7fa73affd640 1 -- 172.21.15.103:0/2899728616 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa74c0042e0 con 0x55d363f5aaf0 2024-09-19T06:58:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.859+0000 7fa764dbf740 10 client.18528 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.859+0000 7fa764dbf740 10 client.18528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:58:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.859+0000 7fa764dbf740 20 client.18528 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.859+0000 7fa764dbf740 1 --2- 172.21.15.103:0/2899728616 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d363fe4f00 0x55d363fe7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.859+0000 7fa764dbf740 4 client.18528 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:58:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.859+0000 7fa764dbf740 1 -- 172.21.15.103:0/2899728616 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d363ec6d70 con 0x55d363fe4f00 2024-09-19T06:58:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.859+0000 7fa75affd640 1 --2- 172.21.15.103:0/2899728616 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d363fe4f00 0x55d363fe7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.859+0000 7fa75affd640 1 --2- 172.21.15.103:0/2899728616 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d363fe4f00 0x55d363fe7320 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c055ab4bef6e7c19 server_cookie=31b54bbd242a51eb in_seq=0 out_seq=0 2024-09-19T06:58:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.859+0000 7fa73affd640 10 client.18528 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.873+0000 7fa73affd640 1 -- 172.21.15.103:0/2899728616 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d363ec6d70 con 0x55d363fe4f00 2024-09-19T06:58:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.873+0000 7fa73affd640 10 client.18528 handle_command_reply: tid=0 2024-09-19T06:58:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.873+0000 7fa764dbf740 1 client.18528 shutdown 2024-09-19T06:58:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa764dbf740 1 -- 172.21.15.103:0/2899728616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5aaf0 msgr2=0x55d363f6e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa764dbf740 1 --2- 172.21.15.103:0/2899728616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5aaf0 0x55d363f6e000 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fa74c011d10 tx=0x7fa74c011e90 comp rx=0 tx=0).stop 2024-09-19T06:58:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa764dbf740 1 -- 172.21.15.103:0/2899728616 shutdown_connections 2024-09-19T06:58:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa764dbf740 1 --2- 172.21.15.103:0/2899728616 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d363fe4f00 0x55d363fe7320 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa764dbf740 1 --2- 172.21.15.103:0/2899728616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d363f65cb0 0x55d363f695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa764dbf740 1 --2- 172.21.15.103:0/2899728616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d363f5b410 0x55d363f6e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa73affd640 0 client.18528 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa764dbf740 1 --2- 172.21.15.103:0/2899728616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d363f5aaf0 0x55d363f6e000 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa764dbf740 1 -- 172.21.15.103:0/2899728616 >> 172.21.15.103:0/2899728616 conn(0x55d363f57060 msgr2=0x55d363f643c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa764dbf740 1 -- 172.21.15.103:0/2899728616 shutdown_connections 2024-09-19T06:58:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa764dbf740 1 -- 172.21.15.103:0/2899728616 wait complete. 2024-09-19T06:58:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa764dbf740 20 client.18528 trim_cache size 0 max 16384 2024-09-19T06:58:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:58:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:58:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:58:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:58:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa761f52640 1 -- 172.21.15.103:0/884837071 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7440a34e0 msgr2=0x7fa7440a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa761f52640 1 --2- 172.21.15.103:0/884837071 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7440a34e0 0x7fa7440a59a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa754009430 tx=0x7fa75402a950 comp rx=0 tx=0).stop 2024-09-19T06:58:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa761f52640 1 -- 172.21.15.103:0/884837071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa75c1035e0 msgr2=0x7fa75c198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa761f52640 1 --2- 172.21.15.103:0/884837071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa75c1035e0 0x7fa75c198880 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa75000e9f0 tx=0x7fa75000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa761f52640 1 -- 172.21.15.103:0/884837071 shutdown_connections 2024-09-19T06:58:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa761f52640 1 --2- 172.21.15.103:0/884837071 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7440a34e0 0x7fa7440a59a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa761f52640 1 --2- 172.21.15.103:0/884837071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa75c10c110 0x7fa75c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa761f52640 1 --2- 172.21.15.103:0/884837071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa75c1035e0 0x7fa75c198880 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa761f52640 1 --2- 172.21.15.103:0/884837071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa75c102cc0 0x7fa75c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.874+0000 7fa761f52640 1 -- 172.21.15.103:0/884837071 >> 172.21.15.103:0/884837071 conn(0x7fa75c0fc6b0 msgr2=0x7fa75c0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.875+0000 7fa761f52640 1 -- 172.21.15.103:0/884837071 shutdown_connections 2024-09-19T06:58:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:10.875+0000 7fa761f52640 1 -- 172.21.15.103:0/884837071 wait complete. 2024-09-19T06:58:12.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:12 smithi103 ceph-mon[31158]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T06:58:12.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:12 smithi103 ceph-mon[31158]: pgmap v278: 97 pgs: 97 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 126 KiB/s wr, 45 op/s 2024-09-19T06:58:12.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:12 smithi134 ceph-mon[35356]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T06:58:12.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:12 smithi134 ceph-mon[35356]: pgmap v278: 97 pgs: 97 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 126 KiB/s wr, 45 op/s 2024-09-19T06:58:13.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:12 smithi174 ceph-mon[35637]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T06:58:13.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:12 smithi174 ceph-mon[35637]: pgmap v278: 97 pgs: 97 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 126 KiB/s wr, 45 op/s 2024-09-19T06:58:14.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:14 smithi103 ceph-mon[31158]: pgmap v279: 97 pgs: 97 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-09-19T06:58:14.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:14 smithi134 ceph-mon[35356]: pgmap v279: 97 pgs: 97 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-09-19T06:58:14.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:14 smithi174 ceph-mon[35637]: pgmap v279: 97 pgs: 97 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-09-19T06:58:16.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:16 smithi103 ceph-mon[31158]: pgmap v280: 97 pgs: 97 active+clean; 3.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T06:58:16.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:16 smithi134 ceph-mon[35356]: pgmap v280: 97 pgs: 97 active+clean; 3.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T06:58:16.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:16 smithi174 ceph-mon[35637]: pgmap v280: 97 pgs: 97 active+clean; 3.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T06:58:18.385 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:18 smithi174 ceph-mon[35637]: pgmap v281: 97 pgs: 97 active+clean; 3.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T06:58:18.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:18 smithi103 ceph-mon[31158]: pgmap v281: 97 pgs: 97 active+clean; 3.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T06:58:18.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:18 smithi134 ceph-mon[35356]: pgmap v281: 97 pgs: 97 active+clean; 3.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T06:58:18.855 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:58:19.119 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:58:19.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.553+0000 7f6db456e640 1 -- 172.21.15.103:0/4267331166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dac106a80 msgr2=0x7f6dac106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:19.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.553+0000 7f6db456e640 1 --2- 172.21.15.103:0/4267331166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dac106a80 0x7f6dac106e60 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009930 tx=0x7f6da802a420 comp rx=0 tx=0).stop 2024-09-19T06:58:19.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.553+0000 7f6db456e640 1 -- 172.21.15.103:0/4267331166 shutdown_connections 2024-09-19T06:58:19.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.553+0000 7f6db456e640 1 --2- 172.21.15.103:0/4267331166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dac101420 0x7f6dac10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.553+0000 7f6db456e640 1 --2- 172.21.15.103:0/4267331166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dac100a80 0x7f6dac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.553+0000 7f6db456e640 1 --2- 172.21.15.103:0/4267331166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dac106a80 0x7f6dac106e60 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.553+0000 7f6db456e640 1 -- 172.21.15.103:0/4267331166 >> 172.21.15.103:0/4267331166 conn(0x7f6dac0fc7a0 msgr2=0x7f6dac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:19.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.553+0000 7f6db456e640 1 -- 172.21.15.103:0/4267331166 shutdown_connections 2024-09-19T06:58:19.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.553+0000 7f6db456e640 1 -- 172.21.15.103:0/4267331166 wait complete. 2024-09-19T06:58:19.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.554+0000 7f6db456e640 1 Processor -- start 2024-09-19T06:58:19.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.554+0000 7f6db456e640 1 -- start start 2024-09-19T06:58:19.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.554+0000 7f6db456e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dac100a80 0x7f6dac102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.554+0000 7f6db456e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dac101420 0x7f6dac102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.554+0000 7f6db456e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dac106a80 0x7f6dac1030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.554+0000 7f6db456e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac110e80 con 0x7f6dac106a80 2024-09-19T06:58:19.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.554+0000 7f6db456e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac110d00 con 0x7f6dac101420 2024-09-19T06:58:19.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.554+0000 7f6db456e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac111000 con 0x7f6dac100a80 2024-09-19T06:58:19.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6db2b0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dac106a80 0x7f6dac1030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:19.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6db1b09640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dac101420 0x7f6dac102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:19.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6db2b0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dac106a80 0x7f6dac1030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38482/0 (socket says 172.21.15.103:38482) 2024-09-19T06:58:19.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6db2b0b640 1 -- 172.21.15.103:0/663786273 learned_addr learned my addr 172.21.15.103:0/663786273 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:19.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6db1b09640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dac101420 0x7f6dac102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:59020/0 (socket says 172.21.15.103:59020) 2024-09-19T06:58:19.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6db230a640 1 --2- 172.21.15.103:0/663786273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dac100a80 0x7f6dac102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:19.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6db2b0b640 1 -- 172.21.15.103:0/663786273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dac100a80 msgr2=0x7f6dac102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:19.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6db2b0b640 1 --2- 172.21.15.103:0/663786273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dac100a80 0x7f6dac102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6db2b0b640 1 -- 172.21.15.103:0/663786273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dac101420 msgr2=0x7f6dac102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:19.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6db2b0b640 1 --2- 172.21.15.103:0/663786273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dac101420 0x7f6dac102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6db2b0b640 1 -- 172.21.15.103:0/663786273 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dac1aae50 con 0x7f6dac106a80 2024-09-19T06:58:19.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6db2b0b640 1 --2- 172.21.15.103:0/663786273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dac106a80 0x7f6dac1030d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f6da400cad0 tx=0x7f6da400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:19.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6d9f7fe640 1 -- 172.21.15.103:0/663786273 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da400ae20 con 0x7f6dac106a80 2024-09-19T06:58:19.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6db456e640 1 -- 172.21.15.103:0/663786273 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dac1ab080 con 0x7f6dac106a80 2024-09-19T06:58:19.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6d9f7fe640 1 -- 172.21.15.103:0/663786273 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da4005020 con 0x7f6dac106a80 2024-09-19T06:58:19.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6db456e640 1 -- 172.21.15.103:0/663786273 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dac1ab5c0 con 0x7f6dac106a80 2024-09-19T06:58:19.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.555+0000 7f6d9f7fe640 1 -- 172.21.15.103:0/663786273 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da401d060 con 0x7f6dac106a80 2024-09-19T06:58:19.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.558+0000 7f6d9f7fe640 1 -- 172.21.15.103:0/663786273 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f6da4004a40 con 0x7f6dac106a80 2024-09-19T06:58:19.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.558+0000 7f6db6947740 1 Processor -- start 2024-09-19T06:58:19.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.558+0000 7f6db6947740 1 -- start start 2024-09-19T06:58:19.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db6947740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b94a0 0x55aa265b9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db6947740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265b9dc0 0x55aa265c4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db6947740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265c4680 0x55aa265c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db6947740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aa2651f2e0 con 0x55aa265b9dc0 2024-09-19T06:58:19.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db6947740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55aa264c6ce0 con 0x55aa265c4680 2024-09-19T06:58:19.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db6947740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aa2651b2a0 con 0x55aa265b94a0 2024-09-19T06:58:19.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db230a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265b9dc0 0x55aa265c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:19.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db230a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265b9dc0 0x55aa265c4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38486/0 (socket says 172.21.15.103:38486) 2024-09-19T06:58:19.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db230a640 1 -- 172.21.15.103:0/1104325114 learned_addr learned my addr 172.21.15.103:0/1104325114 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:19.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6d9f7fe640 1 --2- 172.21.15.103:0/663786273 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6d980a33b0 0x7f6d980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db1b09640 1 --2- 172.21.15.103:0/1104325114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b94a0 0x55aa265b9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:19.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db1b09640 1 --2- 172.21.15.103:0/1104325114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265c4680 0x55aa265c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:19.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db230a640 1 -- 172.21.15.103:0/1104325114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b94a0 msgr2=0x55aa265b9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:19.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db230a640 1 --2- 172.21.15.103:0/1104325114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b94a0 0x55aa265b9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db230a640 1 -- 172.21.15.103:0/1104325114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265c4680 msgr2=0x55aa265c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:19.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db230a640 1 --2- 172.21.15.103:0/1104325114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265c4680 0x55aa265c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db230a640 1 -- 172.21.15.103:0/1104325114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa264ab440 con 0x55aa265b9dc0 2024-09-19T06:58:19.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db1b09640 1 --2- 172.21.15.103:0/1104325114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b94a0 0x55aa265b9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:58:19.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db2b0b640 1 --2- 172.21.15.103:0/663786273 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6d980a33b0 0x7f6d980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:19.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6d9f7fe640 1 -- 172.21.15.103:0/663786273 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f6da40cfb90 con 0x7f6dac106a80 2024-09-19T06:58:19.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6db230a640 1 --2- 172.21.15.103:0/1104325114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265b9dc0 0x55aa265c4140 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009930 tx=0x7f6da8031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74154824f2138903 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:19.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.559+0000 7f6d9cff9640 1 -- 172.21.15.103:0/1104325114 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8040070 con 0x55aa265b9dc0 2024-09-19T06:58:19.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.560+0000 7f6d9cff9640 1 -- 172.21.15.103:0/1104325114 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da8005020 con 0x55aa265b9dc0 2024-09-19T06:58:19.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.560+0000 7f6d9cff9640 1 -- 172.21.15.103:0/1104325114 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8035040 con 0x55aa265b9dc0 2024-09-19T06:58:19.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.560+0000 7f6db6947740 1 -- 172.21.15.103:0/1104325114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265b9dc0 msgr2=0x55aa265c4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:19.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.560+0000 7f6db2b0b640 1 --2- 172.21.15.103:0/663786273 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6d980a33b0 0x7f6d980a5870 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6da409be90 tx=0x7f6da406dc80 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:19.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.560+0000 7f6db6947740 1 --2- 172.21.15.103:0/1104325114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265b9dc0 0x55aa265c4140 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009930 tx=0x7f6da8031630 comp rx=0 tx=0).stop 2024-09-19T06:58:19.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.560+0000 7f6db6947740 1 -- 172.21.15.103:0/1104325114 shutdown_connections 2024-09-19T06:58:19.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.560+0000 7f6db6947740 1 --2- 172.21.15.103:0/1104325114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265c4680 0x55aa265c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.560+0000 7f6db6947740 1 --2- 172.21.15.103:0/1104325114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265b9dc0 0x55aa265c4140 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.560+0000 7f6db6947740 1 --2- 172.21.15.103:0/1104325114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b94a0 0x55aa265b9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.560+0000 7f6db6947740 1 -- 172.21.15.103:0/1104325114 >> 172.21.15.103:0/1104325114 conn(0x55aa265b5a30 msgr2=0x55aa265b5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:19.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.560+0000 7f6db6947740 1 -- 172.21.15.103:0/1104325114 shutdown_connections 2024-09-19T06:58:19.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.560+0000 7f6db6947740 1 -- 172.21.15.103:0/1104325114 wait complete. 2024-09-19T06:58:19.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.561+0000 7f6db6947740 1 -- start start 2024-09-19T06:58:19.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.561+0000 7f6db6947740 1 Processor -- start 2024-09-19T06:58:19.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.561+0000 7f6db6947740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b94a0 0x55aa265c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.561+0000 7f6db6947740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265b9dc0 0x55aa265c1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.561+0000 7f6db6947740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265c4680 0x55aa265c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.561+0000 7f6db6947740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aa265c7250 con 0x55aa265c4680 2024-09-19T06:58:19.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.561+0000 7f6db6947740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55aa264da050 con 0x55aa265b9dc0 2024-09-19T06:58:19.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.561+0000 7f6db6947740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aa265c73d0 con 0x55aa265b94a0 2024-09-19T06:58:19.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.561+0000 7f6db6947740 10 client.0 _ll_register_callbacks cb 0x55aa2651a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:19.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.561+0000 7f6db1b09640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265c4680 0x55aa265c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:19.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.561+0000 7f6db1b09640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265c4680 0x55aa265c15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38494/0 (socket says 172.21.15.103:38494) 2024-09-19T06:58:19.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.561+0000 7f6db1b09640 1 -- 172.21.15.103:0/2181808120 learned_addr learned my addr 172.21.15.103:0/2181808120 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:19.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.561+0000 7f6db1b09640 1 --2- 172.21.15.103:0/2181808120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b94a0 0x55aa265c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:19.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.562+0000 7f6db230a640 1 --2- 172.21.15.103:0/2181808120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265b9dc0 0x55aa265c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:19.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.562+0000 7f6db1b09640 1 -- 172.21.15.103:0/2181808120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b94a0 msgr2=0x55aa265c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:19.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.562+0000 7f6db1b09640 1 --2- 172.21.15.103:0/2181808120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b94a0 0x55aa265c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.562+0000 7f6db1b09640 1 -- 172.21.15.103:0/2181808120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265b9dc0 msgr2=0x55aa265c1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:19.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.562+0000 7f6db1b09640 1 --2- 172.21.15.103:0/2181808120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265b9dc0 0x55aa265c1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.562+0000 7f6db1b09640 1 -- 172.21.15.103:0/2181808120 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa262903c0 con 0x55aa265c4680 2024-09-19T06:58:19.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.562+0000 7f6db230a640 1 --2- 172.21.15.103:0/2181808120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265b9dc0 0x55aa265c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:58:19.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.562+0000 7f6db1b09640 1 --2- 172.21.15.103:0/2181808120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265c4680 0x55aa265c15c0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f6da000bfe0 tx=0x7f6da000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998757e4231bbe94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:19.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.562+0000 7f6d95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:19.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.562+0000 7f6d95ffb640 1 -- 172.21.15.103:0/2181808120 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0005020 con 0x55aa265c4680 2024-09-19T06:58:19.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.562+0000 7f6d95ffb640 1 -- 172.21.15.103:0/2181808120 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f6da00051c0 con 0x55aa265c4680 2024-09-19T06:58:19.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.562+0000 7f6db6947740 1 -- 172.21.15.103:0/2181808120 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6da8005020 con 0x55aa265c4680 2024-09-19T06:58:19.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.562+0000 7f6d95ffb640 1 -- 172.21.15.103:0/2181808120 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6da00053c0 con 0x55aa265c4680 2024-09-19T06:58:19.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.562+0000 7f6db6947740 10 client.18540 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:19.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.562+0000 7f6db6947740 1 -- 172.21.15.103:0/2181808120 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa265d2a60 con 0x55aa265c4680 2024-09-19T06:58:19.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.563+0000 7f6d95ffb640 1 -- 172.21.15.103:0/2181808120 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da0005560 con 0x55aa265c4680 2024-09-19T06:58:19.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.563+0000 7f6d95ffb640 1 -- 172.21.15.103:0/2181808120 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6da0004c70 con 0x55aa265c4680 2024-09-19T06:58:19.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.563+0000 7f6db6947740 10 client.18540 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:19.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.563+0000 7f6db6947740 10 client.18540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:58:19.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.563+0000 7f6db6947740 20 client.18540 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:19.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.563+0000 7f6db6947740 1 --2- 172.21.15.103:0/2181808120 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aa265d3f20 0x55aa265d6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.563+0000 7f6db6947740 4 client.18540 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:58:19.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.563+0000 7f6db6947740 1 -- 172.21.15.103:0/2181808120 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6da8005020 con 0x55aa265d3f20 2024-09-19T06:58:19.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.563+0000 7f6db230a640 1 --2- 172.21.15.103:0/2181808120 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aa265d3f20 0x55aa265d6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:19.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.564+0000 7f6db230a640 1 --2- 172.21.15.103:0/2181808120 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aa265d3f20 0x55aa265d6370 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee77bcc134371988 server_cookie=7bf2292b3f12f075 in_seq=0 out_seq=0 2024-09-19T06:58:19.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.564+0000 7f6d95ffb640 10 client.18540 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:19.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.564+0000 7f6d95ffb640 1 -- 172.21.15.103:0/2181808120 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6da8005020 con 0x55aa265d3f20 2024-09-19T06:58:19.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.564+0000 7f6d95ffb640 10 client.18540 handle_command_reply: tid=0 2024-09-19T06:58:19.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.564+0000 7f6db6947740 1 client.18540 shutdown 2024-09-19T06:58:19.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.564+0000 7f6db6947740 1 -- 172.21.15.103:0/2181808120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265c4680 msgr2=0x55aa265c15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:19.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.564+0000 7f6db6947740 1 --2- 172.21.15.103:0/2181808120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265c4680 0x55aa265c15c0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f6da000bfe0 tx=0x7f6da000b7b0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.564+0000 7f6db6947740 1 -- 172.21.15.103:0/2181808120 shutdown_connections 2024-09-19T06:58:19.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.564+0000 7f6db6947740 1 --2- 172.21.15.103:0/2181808120 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aa265d3f20 0x55aa265d6370 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.564+0000 7f6db6947740 1 --2- 172.21.15.103:0/2181808120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265c4680 0x55aa265c15c0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.564+0000 7f6db6947740 1 --2- 172.21.15.103:0/2181808120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265b9dc0 0x55aa265c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.564+0000 7f6d95ffb640 0 client.18540 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:19.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.564+0000 7f6db6947740 1 --2- 172.21.15.103:0/2181808120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b94a0 0x55aa265c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.565+0000 7f6db6947740 1 -- 172.21.15.103:0/2181808120 >> 172.21.15.103:0/2181808120 conn(0x55aa265b5a30 msgr2=0x55aa265bb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:19.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.565+0000 7f6db6947740 1 -- 172.21.15.103:0/2181808120 shutdown_connections 2024-09-19T06:58:19.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.565+0000 7f6db6947740 1 -- 172.21.15.103:0/2181808120 wait complete. 2024-09-19T06:58:19.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.565+0000 7f6db6947740 20 client.18540 trim_cache size 0 max 16384 2024-09-19T06:58:19.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.582+0000 7f6db6947740 1 Processor -- start 2024-09-19T06:58:19.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.582+0000 7f6db6947740 1 -- start start 2024-09-19T06:58:19.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db6947740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265b94a0 0x55aa265c0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db6947740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b9dc0 0x55aa265dbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db6947740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265c4680 0x55aa265dc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db6947740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9000e7b0 con 0x55aa265c4680 2024-09-19T06:58:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db6947740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d900102f0 con 0x55aa265b94a0 2024-09-19T06:58:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db6947740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9000ee80 con 0x55aa265b9dc0 2024-09-19T06:58:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db1b09640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265c4680 0x55aa265dc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db1b09640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265c4680 0x55aa265dc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38502/0 (socket says 172.21.15.103:38502) 2024-09-19T06:58:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db1b09640 1 -- 172.21.15.103:0/2606022424 learned_addr learned my addr 172.21.15.103:0/2606022424 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db230a640 1 --2- 172.21.15.103:0/2606022424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b9dc0 0x55aa265dbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db1b09640 1 --2- 172.21.15.103:0/2606022424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265b94a0 0x55aa265c0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db1b09640 1 -- 172.21.15.103:0/2606022424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b9dc0 msgr2=0x55aa265dbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db1b09640 1 --2- 172.21.15.103:0/2606022424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b9dc0 0x55aa265dbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db1b09640 1 -- 172.21.15.103:0/2606022424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265b94a0 msgr2=0x55aa265c0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:19.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db1b09640 1 --2- 172.21.15.103:0/2606022424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265b94a0 0x55aa265c0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db1b09640 1 -- 172.21.15.103:0/2606022424 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa265b5860 con 0x55aa265c4680 2024-09-19T06:58:19.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.583+0000 7f6db1b09640 1 --2- 172.21.15.103:0/2606022424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265c4680 0x55aa265dc180 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f6da0005fd0 tx=0x7f6da0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1122075f5d9c5355 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:19.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.584+0000 7f6d97fff640 1 -- 172.21.15.103:0/2606022424 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da002c040 con 0x55aa265c4680 2024-09-19T06:58:19.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.584+0000 7f6d97fff640 1 -- 172.21.15.103:0/2606022424 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da0005560 con 0x55aa265c4680 2024-09-19T06:58:19.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.584+0000 7f6d97fff640 1 -- 172.21.15.103:0/2606022424 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da00278c0 con 0x55aa265c4680 2024-09-19T06:58:19.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.584+0000 7f6db6947740 1 -- 172.21.15.103:0/2606022424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265c4680 msgr2=0x55aa265dc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:19.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.584+0000 7f6db6947740 1 --2- 172.21.15.103:0/2606022424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265c4680 0x55aa265dc180 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f6da0005fd0 tx=0x7f6da0000bc0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.584+0000 7f6db6947740 1 -- 172.21.15.103:0/2606022424 shutdown_connections 2024-09-19T06:58:19.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.584+0000 7f6db6947740 1 --2- 172.21.15.103:0/2606022424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265c4680 0x55aa265dc180 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.584+0000 7f6db6947740 1 --2- 172.21.15.103:0/2606022424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b9dc0 0x55aa265dbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.584+0000 7f6db6947740 1 --2- 172.21.15.103:0/2606022424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265b94a0 0x55aa265c0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.584+0000 7f6db6947740 1 -- 172.21.15.103:0/2606022424 >> 172.21.15.103:0/2606022424 conn(0x55aa265b5a30 msgr2=0x55aa265b5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:19.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.584+0000 7f6db6947740 1 -- 172.21.15.103:0/2606022424 shutdown_connections 2024-09-19T06:58:19.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.584+0000 7f6db6947740 1 -- 172.21.15.103:0/2606022424 wait complete. 2024-09-19T06:58:19.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db6947740 1 -- start start 2024-09-19T06:58:19.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db6947740 1 Processor -- start 2024-09-19T06:58:19.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db6947740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b94a0 0x55aa265cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db6947740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265b9dc0 0x55aa265c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db6947740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265c4680 0x55aa265c5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db6947740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9000fc20 con 0x55aa265b9dc0 2024-09-19T06:58:19.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db6947740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9000f550 con 0x55aa265c4680 2024-09-19T06:58:19.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db6947740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9000c7b0 con 0x55aa265b94a0 2024-09-19T06:58:19.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db6947740 10 client.0 _ll_register_callbacks cb 0x7f6da00123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:19.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db230a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265b9dc0 0x55aa265c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:19.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db230a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265b9dc0 0x55aa265c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38506/0 (socket says 172.21.15.103:38506) 2024-09-19T06:58:19.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db230a640 1 -- 172.21.15.103:0/2195464768 learned_addr learned my addr 172.21.15.103:0/2195464768 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:19.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db230a640 1 -- 172.21.15.103:0/2195464768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b94a0 msgr2=0x55aa265cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:19.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db230a640 1 --2- 172.21.15.103:0/2195464768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b94a0 0x55aa265cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db230a640 1 -- 172.21.15.103:0/2195464768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265c4680 msgr2=0x55aa265c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:19.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db230a640 1 --2- 172.21.15.103:0/2195464768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265c4680 0x55aa265c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.585+0000 7f6db230a640 1 -- 172.21.15.103:0/2195464768 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa265c6060 con 0x55aa265b9dc0 2024-09-19T06:58:19.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.586+0000 7f6db230a640 1 --2- 172.21.15.103:0/2195464768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265b9dc0 0x55aa265c53e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f6da802a940 tx=0x7f6da8009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c230e0f82543475e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:19.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.586+0000 7f6d95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:19.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.586+0000 7f6d95ffb640 1 -- 172.21.15.103:0/2195464768 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8035040 con 0x55aa265b9dc0 2024-09-19T06:58:19.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.586+0000 7f6d95ffb640 1 -- 172.21.15.103:0/2195464768 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f6da804f7b0 con 0x55aa265b9dc0 2024-09-19T06:58:19.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.586+0000 7f6db6947740 1 -- 172.21.15.103:0/2195464768 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6da0005560 con 0x55aa265b9dc0 2024-09-19T06:58:19.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.586+0000 7f6d95ffb640 1 -- 172.21.15.103:0/2195464768 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6da8040040 con 0x55aa265b9dc0 2024-09-19T06:58:19.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.586+0000 7f6db6947740 10 client.18546 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:19.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.586+0000 7f6db6947740 1 -- 172.21.15.103:0/2195464768 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa2663ec10 con 0x55aa265b9dc0 2024-09-19T06:58:19.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.586+0000 7f6d95ffb640 1 -- 172.21.15.103:0/2195464768 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da0005560 con 0x55aa265b9dc0 2024-09-19T06:58:19.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.586+0000 7f6d95ffb640 1 -- 172.21.15.103:0/2195464768 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6da803cc50 con 0x55aa265b9dc0 2024-09-19T06:58:19.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.586+0000 7f6db6947740 10 client.18546 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:19.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.586+0000 7f6db6947740 10 client.18546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:58:19.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.586+0000 7f6db6947740 20 client.18546 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:19.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.586+0000 7f6db6947740 1 --2- 172.21.15.103:0/2195464768 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aa2663ef50 0x55aa26641370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:19.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.586+0000 7f6db6947740 4 client.18546 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:58:19.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.586+0000 7f6db6947740 1 -- 172.21.15.103:0/2195464768 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aa2652c850 con 0x55aa2663ef50 2024-09-19T06:58:19.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.587+0000 7f6db1b09640 1 --2- 172.21.15.103:0/2195464768 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aa2663ef50 0x55aa26641370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:19.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.587+0000 7f6d95ffb640 10 client.18546 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:19.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.587+0000 7f6db1b09640 1 --2- 172.21.15.103:0/2195464768 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aa2663ef50 0x55aa26641370 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=355edbaff7d74aeb server_cookie=d1794a711e2abf05 in_seq=0 out_seq=0 2024-09-19T06:58:19.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.587+0000 7f6d95ffb640 1 -- 172.21.15.103:0/2195464768 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55aa2652c850 con 0x55aa2663ef50 2024-09-19T06:58:19.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.587+0000 7f6d95ffb640 10 client.18546 handle_command_reply: tid=0 2024-09-19T06:58:19.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.587+0000 7f6db6947740 1 client.18546 shutdown 2024-09-19T06:58:19.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.587+0000 7f6db6947740 1 -- 172.21.15.103:0/2195464768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265b9dc0 msgr2=0x55aa265c53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:19.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.587+0000 7f6db6947740 1 --2- 172.21.15.103:0/2195464768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265b9dc0 0x55aa265c53e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f6da802a940 tx=0x7f6da8009610 comp rx=0 tx=0).stop 2024-09-19T06:58:19.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.587+0000 7f6db6947740 1 -- 172.21.15.103:0/2195464768 shutdown_connections 2024-09-19T06:58:19.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.587+0000 7f6db6947740 1 --2- 172.21.15.103:0/2195464768 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aa2663ef50 0x55aa26641370 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.587+0000 7f6db6947740 1 --2- 172.21.15.103:0/2195464768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aa265c4680 0x55aa265c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.587+0000 7f6db6947740 1 --2- 172.21.15.103:0/2195464768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa265b9dc0 0x55aa265c53e0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.587+0000 7f6db6947740 1 --2- 172.21.15.103:0/2195464768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa265b94a0 0x55aa265cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.587+0000 7f6db6947740 1 -- 172.21.15.103:0/2195464768 >> 172.21.15.103:0/2195464768 conn(0x55aa265b5a30 msgr2=0x55aa265c28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:19.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.587+0000 7f6d95ffb640 0 client.18546 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:19.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.588+0000 7f6db6947740 1 -- 172.21.15.103:0/2195464768 shutdown_connections 2024-09-19T06:58:19.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.588+0000 7f6db6947740 1 -- 172.21.15.103:0/2195464768 wait complete. 2024-09-19T06:58:19.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.588+0000 7f6db6947740 20 client.18546 trim_cache size 0 max 16384 2024-09-19T06:58:19.594 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:58:19.594 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T06:58:19.594 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T06:58:19.594 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:58:19.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.588+0000 7f6db456e640 1 -- 172.21.15.103:0/663786273 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6d980a33b0 msgr2=0x7f6d980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:19.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.588+0000 7f6db456e640 1 --2- 172.21.15.103:0/663786273 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6d980a33b0 0x7f6d980a5870 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6da409be90 tx=0x7f6da406dc80 comp rx=0 tx=0).stop 2024-09-19T06:58:19.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.588+0000 7f6db456e640 1 -- 172.21.15.103:0/663786273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dac106a80 msgr2=0x7f6dac1030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:19.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.588+0000 7f6db456e640 1 --2- 172.21.15.103:0/663786273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dac106a80 0x7f6dac1030d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f6da400cad0 tx=0x7f6da400cf90 comp rx=0 tx=0).stop 2024-09-19T06:58:19.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.588+0000 7f6db456e640 1 -- 172.21.15.103:0/663786273 shutdown_connections 2024-09-19T06:58:19.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.588+0000 7f6db456e640 1 --2- 172.21.15.103:0/663786273 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6d980a33b0 0x7f6d980a5870 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.588+0000 7f6db456e640 1 --2- 172.21.15.103:0/663786273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dac106a80 0x7f6dac1030d0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.588+0000 7f6db456e640 1 --2- 172.21.15.103:0/663786273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dac101420 0x7f6dac102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.588+0000 7f6db456e640 1 --2- 172.21.15.103:0/663786273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dac100a80 0x7f6dac102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:19.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.588+0000 7f6db456e640 1 -- 172.21.15.103:0/663786273 >> 172.21.15.103:0/663786273 conn(0x7f6dac0fc7a0 msgr2=0x7f6dac104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:19.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.588+0000 7f6db456e640 1 -- 172.21.15.103:0/663786273 shutdown_connections 2024-09-19T06:58:19.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:19.588+0000 7f6db456e640 1 -- 172.21.15.103:0/663786273 wait complete. 2024-09-19T06:58:20.347 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:58:20.347 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:58:20.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:20 smithi103 ceph-mon[31158]: pgmap v282: 97 pgs: 97 active+clean; 3.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-19T06:58:20.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:20 smithi134 ceph-mon[35356]: pgmap v282: 97 pgs: 97 active+clean; 3.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-19T06:58:20.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:20 smithi174 ceph-mon[35637]: pgmap v282: 97 pgs: 97 active+clean; 3.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-19T06:58:20.604 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:58:21.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.060+0000 7fe8674b2640 1 -- 172.21.15.103:0/2315680023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe860102ef0 msgr2=0x7fe86010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:21.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.060+0000 7fe8674b2640 1 --2- 172.21.15.103:0/2315680023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe860102ef0 0x7fe86010da40 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe858009900 tx=0x7fe85802a480 comp rx=0 tx=0).stop 2024-09-19T06:58:21.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.060+0000 7fe8674b2640 1 -- 172.21.15.103:0/2315680023 shutdown_connections 2024-09-19T06:58:21.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.060+0000 7fe8674b2640 1 --2- 172.21.15.103:0/2315680023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe860102ef0 0x7fe86010da40 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:21.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.060+0000 7fe8674b2640 1 --2- 172.21.15.103:0/2315680023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe860102570 0x7fe8601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:21.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.060+0000 7fe8674b2640 1 --2- 172.21.15.103:0/2315680023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe860100e20 0x7fe860101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:21.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.060+0000 7fe8674b2640 1 -- 172.21.15.103:0/2315680023 >> 172.21.15.103:0/2315680023 conn(0x7fe8600fc7d0 msgr2=0x7fe8600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:21.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.060+0000 7fe8674b2640 1 -- 172.21.15.103:0/2315680023 shutdown_connections 2024-09-19T06:58:21.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.060+0000 7fe8674b2640 1 -- 172.21.15.103:0/2315680023 wait complete. 2024-09-19T06:58:21.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.061+0000 7fe8674b2640 1 Processor -- start 2024-09-19T06:58:21.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.061+0000 7fe8674b2640 1 -- start start 2024-09-19T06:58:21.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.061+0000 7fe8674b2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe860100e20 0x7fe8601987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:21.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.061+0000 7fe8674b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe860102570 0x7fe860198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:21.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.061+0000 7fe8674b2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe860102ef0 0x7fe86019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:21.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.061+0000 7fe8674b2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe860110f80 con 0x7fe860102570 2024-09-19T06:58:21.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.061+0000 7fe8674b2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe860110e00 con 0x7fe860100e20 2024-09-19T06:58:21.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.061+0000 7fe8674b2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe860111100 con 0x7fe860102ef0 2024-09-19T06:58:21.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.061+0000 7fe864a4d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe860102570 0x7fe860198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:21.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.061+0000 7fe864a4d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe860102570 0x7fe860198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38528/0 (socket says 172.21.15.103:38528) 2024-09-19T06:58:21.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.061+0000 7fe864a4d640 1 -- 172.21.15.103:0/1219466660 learned_addr learned my addr 172.21.15.103:0/1219466660 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:21.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.061+0000 7fe86524e640 1 --2- 172.21.15.103:0/1219466660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe860100e20 0x7fe8601987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:21.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.061+0000 7fe865a4f640 1 --2- 172.21.15.103:0/1219466660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe860102ef0 0x7fe86019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:21.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.061+0000 7fe864a4d640 1 -- 172.21.15.103:0/1219466660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe860102ef0 msgr2=0x7fe86019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:21.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.062+0000 7fe864a4d640 1 --2- 172.21.15.103:0/1219466660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe860102ef0 0x7fe86019cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:21.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.062+0000 7fe864a4d640 1 -- 172.21.15.103:0/1219466660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe860100e20 msgr2=0x7fe8601987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:21.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.062+0000 7fe864a4d640 1 --2- 172.21.15.103:0/1219466660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe860100e20 0x7fe8601987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:21.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.062+0000 7fe864a4d640 1 -- 172.21.15.103:0/1219466660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86019d520 con 0x7fe860102570 2024-09-19T06:58:21.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.062+0000 7fe864a4d640 1 --2- 172.21.15.103:0/1219466660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe860102570 0x7fe860198d10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe85c00cce0 tx=0x7fe85c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:21.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.062+0000 7fe8567fc640 1 -- 172.21.15.103:0/1219466660 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c00f650 con 0x7fe860102570 2024-09-19T06:58:21.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.062+0000 7fe8674b2640 1 -- 172.21.15.103:0/1219466660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86019d810 con 0x7fe860102570 2024-09-19T06:58:21.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.062+0000 7fe8567fc640 1 -- 172.21.15.103:0/1219466660 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe85c00fcd0 con 0x7fe860102570 2024-09-19T06:58:21.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.062+0000 7fe8567fc640 1 -- 172.21.15.103:0/1219466660 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c018c70 con 0x7fe860102570 2024-09-19T06:58:21.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.062+0000 7fe8674b2640 1 -- 172.21.15.103:0/1219466660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8601a5060 con 0x7fe860102570 2024-09-19T06:58:21.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.063+0000 7fe8674b2640 1 -- 172.21.15.103:0/1219466660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe860101200 con 0x7fe860102570 2024-09-19T06:58:21.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.064+0000 7fe8567fc640 1 -- 172.21.15.103:0/1219466660 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fe85c00f7f0 con 0x7fe860102570 2024-09-19T06:58:21.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.065+0000 7fe8567fc640 1 --2- 172.21.15.103:0/1219466660 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe84c0a33b0 0x7fe84c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:21.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.065+0000 7fe86524e640 1 --2- 172.21.15.103:0/1219466660 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe84c0a33b0 0x7fe84c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:21.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.065+0000 7fe8567fc640 1 -- 172.21.15.103:0/1219466660 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7fe85c0d00b0 con 0x7fe860102570 2024-09-19T06:58:21.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.065+0000 7fe86524e640 1 --2- 172.21.15.103:0/1219466660 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe84c0a33b0 0x7fe84c0a5870 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe85000bbe0 tx=0x7fe85000b390 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:21.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.069+0000 7fe8567fc640 1 -- 172.21.15.103:0/1219466660 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe85c014030 con 0x7fe860102570 2024-09-19T06:58:21.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.271+0000 7fe8674b2640 1 -- 172.21.15.103:0/1219466660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe860199810 con 0x7fe860102570 2024-09-19T06:58:21.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.272+0000 7fe8567fc640 1 -- 172.21.15.103:0/1219466660 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fe85c09ca70 con 0x7fe860102570 2024-09-19T06:58:21.274 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:58:21.274 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:58:21.274 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:58:21.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.274+0000 7fe8674b2640 1 -- 172.21.15.103:0/1219466660 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe84c0a33b0 msgr2=0x7fe84c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:21.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.274+0000 7fe8674b2640 1 --2- 172.21.15.103:0/1219466660 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe84c0a33b0 0x7fe84c0a5870 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe85000bbe0 tx=0x7fe85000b390 comp rx=0 tx=0).stop 2024-09-19T06:58:21.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.274+0000 7fe8674b2640 1 -- 172.21.15.103:0/1219466660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe860102570 msgr2=0x7fe860198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:21.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.274+0000 7fe8674b2640 1 --2- 172.21.15.103:0/1219466660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe860102570 0x7fe860198d10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe85c00cce0 tx=0x7fe85c00a620 comp rx=0 tx=0).stop 2024-09-19T06:58:21.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.274+0000 7fe8674b2640 1 -- 172.21.15.103:0/1219466660 shutdown_connections 2024-09-19T06:58:21.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.274+0000 7fe8674b2640 1 --2- 172.21.15.103:0/1219466660 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe84c0a33b0 0x7fe84c0a5870 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:21.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.274+0000 7fe8674b2640 1 --2- 172.21.15.103:0/1219466660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe860102ef0 0x7fe86019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:21.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.274+0000 7fe8674b2640 1 --2- 172.21.15.103:0/1219466660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe860102570 0x7fe860198d10 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:21.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.274+0000 7fe8674b2640 1 --2- 172.21.15.103:0/1219466660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe860100e20 0x7fe8601987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:21.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.274+0000 7fe8674b2640 1 -- 172.21.15.103:0/1219466660 >> 172.21.15.103:0/1219466660 conn(0x7fe8600fc7d0 msgr2=0x7fe860103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:21.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.274+0000 7fe8674b2640 1 -- 172.21.15.103:0/1219466660 shutdown_connections 2024-09-19T06:58:21.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:21.274+0000 7fe8674b2640 1 -- 172.21.15.103:0/1219466660 wait complete. 2024-09-19T06:58:21.569 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:21 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1219466660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:58:21.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:21 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1219466660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:58:21.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:21 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1219466660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:58:21.910 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:58:22.203 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:58:22.623 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:22 smithi103 ceph-mon[31158]: pgmap v283: 97 pgs: 97 active+clean; 3.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-09-19T06:58:22.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.620+0000 7f83f6410640 1 -- 172.21.15.103:0/4147955328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f0101de0 msgr2=0x7f83f01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:22.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.620+0000 7f83f6410640 1 --2- 172.21.15.103:0/4147955328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f0101de0 0x7f83f01021c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009900 tx=0x7f83e002a410 comp rx=0 tx=0).stop 2024-09-19T06:58:22.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.620+0000 7f83f6410640 1 -- 172.21.15.103:0/4147955328 shutdown_connections 2024-09-19T06:58:22.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.620+0000 7f83f6410640 1 --2- 172.21.15.103:0/4147955328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f0106f10 0x7f83f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.620+0000 7f83f6410640 1 --2- 172.21.15.103:0/4147955328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f0102790 0x7f83f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.620+0000 7f83f6410640 1 --2- 172.21.15.103:0/4147955328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f0101de0 0x7f83f01021c0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.620+0000 7f83f6410640 1 -- 172.21.15.103:0/4147955328 >> 172.21.15.103:0/4147955328 conn(0x7f83f00fd8c0 msgr2=0x7f83f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:22.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.621+0000 7f83f6410640 1 -- 172.21.15.103:0/4147955328 shutdown_connections 2024-09-19T06:58:22.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.621+0000 7f83f6410640 1 -- 172.21.15.103:0/4147955328 wait complete. 2024-09-19T06:58:22.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.621+0000 7f83f6410640 1 Processor -- start 2024-09-19T06:58:22.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.621+0000 7f83f6410640 1 -- start start 2024-09-19T06:58:22.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.621+0000 7f83f6410640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f0101de0 0x7f83f019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.621+0000 7f83f6410640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f0102790 0x7f83f019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83f6410640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f0106f10 0x7f83f019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83f6410640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f83f0111f70 con 0x7f83f0101de0 2024-09-19T06:58:22.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83f6410640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83f0111df0 con 0x7f83f0102790 2024-09-19T06:58:22.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83f6410640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01120f0 con 0x7f83f0106f10 2024-09-19T06:58:22.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83effff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f0101de0 0x7f83f019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:22.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83effff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f0101de0 0x7f83f019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38550/0 (socket says 172.21.15.103:38550) 2024-09-19T06:58:22.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83effff640 1 -- 172.21.15.103:0/3763220684 learned_addr learned my addr 172.21.15.103:0/3763220684 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:22.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83f49ad640 1 --2- 172.21.15.103:0/3763220684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f0106f10 0x7f83f019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:22.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83e7fff640 1 --2- 172.21.15.103:0/3763220684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f0102790 0x7f83f019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:22.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83effff640 1 -- 172.21.15.103:0/3763220684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f0106f10 msgr2=0x7f83f019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:22.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83effff640 1 --2- 172.21.15.103:0/3763220684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f0106f10 0x7f83f019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83effff640 1 -- 172.21.15.103:0/3763220684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f0102790 msgr2=0x7f83f019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:22.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83effff640 1 --2- 172.21.15.103:0/3763220684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f0102790 0x7f83f019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83effff640 1 -- 172.21.15.103:0/3763220684 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f019f650 con 0x7f83f0101de0 2024-09-19T06:58:22.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83effff640 1 --2- 172.21.15.103:0/3763220684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f0101de0 0x7f83f019a6a0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f83e0006010 tx=0x7f83e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:22.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83edffb640 1 -- 172.21.15.103:0/3763220684 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e003f030 con 0x7f83f0101de0 2024-09-19T06:58:22.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83edffb640 1 -- 172.21.15.103:0/3763220684 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e0003ce0 con 0x7f83f0101de0 2024-09-19T06:58:22.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83f6410640 1 -- 172.21.15.103:0/3763220684 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f019f8e0 con 0x7f83f0101de0 2024-09-19T06:58:22.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.622+0000 7f83edffb640 1 -- 172.21.15.103:0/3763220684 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0035070 con 0x7f83f0101de0 2024-09-19T06:58:22.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.623+0000 7f83f6410640 1 -- 172.21.15.103:0/3763220684 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f01a7220 con 0x7f83f0101de0 2024-09-19T06:58:22.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.625+0000 7f83edffb640 1 -- 172.21.15.103:0/3763220684 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f83e0042030 con 0x7f83f0101de0 2024-09-19T06:58:22.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.625+0000 7f83f87e9740 1 Processor -- start 2024-09-19T06:58:22.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.625+0000 7f83f87e9740 1 -- start start 2024-09-19T06:58:22.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.625+0000 7f83edffb640 1 --2- 172.21.15.103:0/3763220684 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83dc0a33b0 0x7f83dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.625+0000 7f83f87e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cb4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.625+0000 7f83f87e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cb4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.625+0000 7f83f87e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cbf370 0x55e9a6cc1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.625+0000 7f83f87e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e9a6c1a2f0 con 0x55e9a6cb5b40 2024-09-19T06:58:22.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.625+0000 7f83f87e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e9a6bc1cf0 con 0x55e9a6cb45d0 2024-09-19T06:58:22.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.625+0000 7f83f87e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e9a6c162b0 con 0x55e9a6cbf370 2024-09-19T06:58:22.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.625+0000 7f83effff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cb4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:22.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.625+0000 7f83f49ad640 1 --2- 172.21.15.103:0/3763220684 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83dc0a33b0 0x7f83dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:22.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.626+0000 7f83effff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cb4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38552/0 (socket says 172.21.15.103:38552) 2024-09-19T06:58:22.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.626+0000 7f83effff640 1 -- 172.21.15.103:0/77314492 learned_addr learned my addr 172.21.15.103:0/77314492 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:22.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.626+0000 7f83edffb640 1 -- 172.21.15.103:0/3763220684 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f83e00f0350 con 0x7f83f0101de0 2024-09-19T06:58:22.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.626+0000 7f83e7fff640 1 --2- 172.21.15.103:0/77314492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cb4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:22.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.626+0000 7f83effff640 1 -- 172.21.15.103:0/77314492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cbf370 msgr2=0x55e9a6cc1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.626+0000 7f83effff640 1 --2- 172.21.15.103:0/77314492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cbf370 0x55e9a6cc1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.626+0000 7f83effff640 1 -- 172.21.15.103:0/77314492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb45d0 msgr2=0x55e9a6cb4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.626+0000 7f83effff640 1 --2- 172.21.15.103:0/77314492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cb4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.626+0000 7f83effff640 1 -- 172.21.15.103:0/77314492 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9a6ba63a0 con 0x55e9a6cb5b40 2024-09-19T06:58:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.626+0000 7f83f49ad640 1 --2- 172.21.15.103:0/3763220684 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83dc0a33b0 0x7f83dc0a5870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f83e8005fd0 tx=0x7f83e8010740 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.626+0000 7f83effff640 1 --2- 172.21.15.103:0/77314492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cb4090 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f83e00b9150 tx=0x7f83e00b8a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794ba026c2d204f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.626+0000 7f83e67fc640 1 -- 172.21.15.103:0/77314492 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0049070 con 0x55e9a6cb5b40 2024-09-19T06:58:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.626+0000 7f83e67fc640 1 -- 172.21.15.103:0/77314492 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e003b580 con 0x55e9a6cb5b40 2024-09-19T06:58:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.626+0000 7f83e67fc640 1 -- 172.21.15.103:0/77314492 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e004b300 con 0x55e9a6cb5b40 2024-09-19T06:58:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.627+0000 7f83f87e9740 1 -- 172.21.15.103:0/77314492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb5b40 msgr2=0x55e9a6cb4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.627+0000 7f83f87e9740 1 --2- 172.21.15.103:0/77314492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cb4090 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f83e00b9150 tx=0x7f83e00b8a90 comp rx=0 tx=0).stop 2024-09-19T06:58:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.627+0000 7f83f87e9740 1 -- 172.21.15.103:0/77314492 shutdown_connections 2024-09-19T06:58:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.627+0000 7f83f87e9740 1 --2- 172.21.15.103:0/77314492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cbf370 0x55e9a6cc1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.627+0000 7f83f87e9740 1 --2- 172.21.15.103:0/77314492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cb4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.627+0000 7f83f87e9740 1 --2- 172.21.15.103:0/77314492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cb4090 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.627+0000 7f83f87e9740 1 -- 172.21.15.103:0/77314492 >> 172.21.15.103:0/77314492 conn(0x55e9a6c7d310 msgr2=0x55e9a6cb0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.627+0000 7f83f87e9740 1 -- 172.21.15.103:0/77314492 shutdown_connections 2024-09-19T06:58:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.627+0000 7f83f87e9740 1 -- 172.21.15.103:0/77314492 wait complete. 2024-09-19T06:58:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.627+0000 7f83f87e9740 1 -- start start 2024-09-19T06:58:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83f87e9740 1 Processor -- start 2024-09-19T06:58:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83f87e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cbedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83f87e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cbc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83f87e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cbf370 0x55e9a6cbc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83f87e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e9a6cc1e30 con 0x55e9a6cb45d0 2024-09-19T06:58:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83f87e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e9a6bd5060 con 0x55e9a6cb5b40 2024-09-19T06:58:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83f87e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e9a6cc1fb0 con 0x55e9a6cbf370 2024-09-19T06:58:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83f87e9740 10 client.0 _ll_register_callbacks cb 0x55e9a69288c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83effff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cbedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83effff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cbedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38568/0 (socket says 172.21.15.103:38568) 2024-09-19T06:58:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83effff640 1 -- 172.21.15.103:0/3897975164 learned_addr learned my addr 172.21.15.103:0/3897975164 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83f49ad640 1 --2- 172.21.15.103:0/3897975164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cbf370 0x55e9a6cbc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83e7fff640 1 --2- 172.21.15.103:0/3897975164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cbc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83effff640 1 -- 172.21.15.103:0/3897975164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cbf370 msgr2=0x55e9a6cbc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83effff640 1 --2- 172.21.15.103:0/3897975164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cbf370 0x55e9a6cbc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83effff640 1 -- 172.21.15.103:0/3897975164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb5b40 msgr2=0x55e9a6cbc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83effff640 1 --2- 172.21.15.103:0/3897975164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cbc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83effff640 1 -- 172.21.15.103:0/3897975164 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9a698b3c0 con 0x55e9a6cb45d0 2024-09-19T06:58:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83effff640 1 --2- 172.21.15.103:0/3897975164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cbedb0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f83e003d040 tx=0x7f83e00433c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0081509852ea67 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.628+0000 7f83cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.629+0000 7f83cf7fe640 1 -- 172.21.15.103:0/3897975164 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0049070 con 0x55e9a6cb45d0 2024-09-19T06:58:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.629+0000 7f83cf7fe640 1 -- 172.21.15.103:0/3897975164 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f83e00b8df0 con 0x55e9a6cb45d0 2024-09-19T06:58:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.629+0000 7f83f87e9740 1 -- 172.21.15.103:0/3897975164 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e9a6c27860 con 0x55e9a6cb45d0 2024-09-19T06:58:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.629+0000 7f83cf7fe640 1 -- 172.21.15.103:0/3897975164 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f83e00044e0 con 0x55e9a6cb45d0 2024-09-19T06:58:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.629+0000 7f83f87e9740 10 client.18564 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.629+0000 7f83f87e9740 1 -- 172.21.15.103:0/3897975164 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e9a6cc1e30 con 0x55e9a6cb45d0 2024-09-19T06:58:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.629+0000 7f83cf7fe640 1 -- 172.21.15.103:0/3897975164 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e9a6c27860 con 0x55e9a6cb45d0 2024-09-19T06:58:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.629+0000 7f83cf7fe640 1 -- 172.21.15.103:0/3897975164 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f83e00b8040 con 0x55e9a6cb45d0 2024-09-19T06:58:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.629+0000 7f83f87e9740 10 client.18564 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.629+0000 7f83f87e9740 10 client.18564 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:58:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.629+0000 7f83f87e9740 20 client.18564 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.629+0000 7f83f87e9740 1 --2- 172.21.15.103:0/3897975164 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55e9a6cce990 0x55e9a6cd0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.629+0000 7f83f87e9740 4 client.18564 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:58:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.629+0000 7f83f87e9740 1 -- 172.21.15.103:0/3897975164 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9a6c21d70 con 0x55e9a6cce990 2024-09-19T06:58:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.630+0000 7f83e7fff640 1 --2- 172.21.15.103:0/3897975164 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55e9a6cce990 0x55e9a6cd0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.630+0000 7f83e7fff640 1 --2- 172.21.15.103:0/3897975164 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55e9a6cce990 0x55e9a6cd0d80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4e1802addc512700 server_cookie=81a3228c73d7f0fc in_seq=0 out_seq=0 2024-09-19T06:58:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.630+0000 7f83cf7fe640 10 client.18564 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:58:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.631+0000 7f83cf7fe640 1 -- 172.21.15.103:0/3897975164 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e9a6c21d70 con 0x55e9a6cce990 2024-09-19T06:58:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.631+0000 7f83cf7fe640 10 client.18564 handle_command_reply: tid=0 2024-09-19T06:58:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.631+0000 7f83f87e9740 1 client.18564 shutdown 2024-09-19T06:58:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.631+0000 7f83f87e9740 1 -- 172.21.15.103:0/3897975164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb45d0 msgr2=0x55e9a6cbedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.631+0000 7f83f87e9740 1 --2- 172.21.15.103:0/3897975164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cbedb0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f83e003d040 tx=0x7f83e00433c0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.631+0000 7f83f87e9740 1 -- 172.21.15.103:0/3897975164 shutdown_connections 2024-09-19T06:58:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.631+0000 7f83f87e9740 1 --2- 172.21.15.103:0/3897975164 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55e9a6cce990 0x55e9a6cd0d80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.631+0000 7f83cf7fe640 0 client.18564 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:58:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.631+0000 7f83f87e9740 1 --2- 172.21.15.103:0/3897975164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cbf370 0x55e9a6cbc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.631+0000 7f83f87e9740 1 --2- 172.21.15.103:0/3897975164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cbc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.631+0000 7f83f87e9740 1 --2- 172.21.15.103:0/3897975164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cbedb0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.631+0000 7f83f87e9740 1 -- 172.21.15.103:0/3897975164 >> 172.21.15.103:0/3897975164 conn(0x55e9a6c7d310 msgr2=0x55e9a6cb6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:22.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.631+0000 7f83f87e9740 1 -- 172.21.15.103:0/3897975164 shutdown_connections 2024-09-19T06:58:22.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.631+0000 7f83f87e9740 1 -- 172.21.15.103:0/3897975164 wait complete. 2024-09-19T06:58:22.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.631+0000 7f83f87e9740 20 client.18564 trim_cache size 0 max 16384 2024-09-19T06:58:22.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83f87e9740 1 Processor -- start 2024-09-19T06:58:22.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83f87e9740 1 -- start start 2024-09-19T06:58:22.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83f87e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cafc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83f87e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cb0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83f87e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cbf370 0x55e9a6cb7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83f87e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01afb10 con 0x55e9a6cb5b40 2024-09-19T06:58:22.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83f87e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01b1650 con 0x55e9a6cb45d0 2024-09-19T06:58:22.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83f87e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01b01e0 con 0x55e9a6cbf370 2024-09-19T06:58:22.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cb0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:22.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cb0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38582/0 (socket says 172.21.15.103:38582) 2024-09-19T06:58:22.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83e7fff640 1 -- 172.21.15.103:0/1924481628 learned_addr learned my addr 172.21.15.103:0/1924481628 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:22.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83e7fff640 1 -- 172.21.15.103:0/1924481628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cbf370 msgr2=0x55e9a6cb7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:22.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83e7fff640 1 --2- 172.21.15.103:0/1924481628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cbf370 0x55e9a6cb7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83e7fff640 1 -- 172.21.15.103:0/1924481628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb45d0 msgr2=0x55e9a6cafc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:22.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83e7fff640 1 --2- 172.21.15.103:0/1924481628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cafc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83e7fff640 1 -- 172.21.15.103:0/1924481628 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9a6cbcb20 con 0x55e9a6cb5b40 2024-09-19T06:58:22.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83e7fff640 1 --2- 172.21.15.103:0/1924481628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cb0150 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f83d8005fd0 tx=0x7f83d800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05eb975b44c22b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:22.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83e5ffb640 1 -- 172.21.15.103:0/1924481628 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d80139f0 con 0x55e9a6cb5b40 2024-09-19T06:58:22.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.644+0000 7f83e5ffb640 1 -- 172.21.15.103:0/1924481628 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d8013b90 con 0x55e9a6cb5b40 2024-09-19T06:58:22.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.645+0000 7f83e5ffb640 1 -- 172.21.15.103:0/1924481628 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d80117c0 con 0x55e9a6cb5b40 2024-09-19T06:58:22.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.645+0000 7f83f87e9740 1 -- 172.21.15.103:0/1924481628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb5b40 msgr2=0x55e9a6cb0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:22.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.645+0000 7f83f87e9740 1 --2- 172.21.15.103:0/1924481628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cb0150 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f83d8005fd0 tx=0x7f83d800b5b0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.645+0000 7f83f87e9740 1 -- 172.21.15.103:0/1924481628 shutdown_connections 2024-09-19T06:58:22.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.645+0000 7f83f87e9740 1 --2- 172.21.15.103:0/1924481628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cbf370 0x55e9a6cb7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.645+0000 7f83f87e9740 1 --2- 172.21.15.103:0/1924481628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cb0150 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.645+0000 7f83f87e9740 1 --2- 172.21.15.103:0/1924481628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cafc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.645+0000 7f83f87e9740 1 -- 172.21.15.103:0/1924481628 >> 172.21.15.103:0/1924481628 conn(0x55e9a6c7d310 msgr2=0x55e9a6caf500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:22.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.645+0000 7f83f87e9740 1 -- 172.21.15.103:0/1924481628 shutdown_connections 2024-09-19T06:58:22.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.645+0000 7f83f87e9740 1 -- 172.21.15.103:0/1924481628 wait complete. 2024-09-19T06:58:22.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.646+0000 7f83f87e9740 1 -- start start 2024-09-19T06:58:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.646+0000 7f83f87e9740 1 Processor -- start 2024-09-19T06:58:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.646+0000 7f83f87e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cc3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.646+0000 7f83f87e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cc36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.646+0000 7f83f87e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cbf370 0x55e9a6d384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.646+0000 7f83f87e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01b0f80 con 0x55e9a6cbf370 2024-09-19T06:58:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.646+0000 7f83f87e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01b08b0 con 0x55e9a6cb45d0 2024-09-19T06:58:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.646+0000 7f83f87e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01adb10 con 0x55e9a6cb5b40 2024-09-19T06:58:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.646+0000 7f83f87e9740 10 client.0 _ll_register_callbacks cb 0x7f83e00d5f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.646+0000 7f83f49ad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cbf370 0x55e9a6d384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.646+0000 7f83f49ad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cbf370 0x55e9a6d384c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38584/0 (socket says 172.21.15.103:38584) 2024-09-19T06:58:22.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.646+0000 7f83effff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cc3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:22.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.646+0000 7f83f49ad640 1 -- 172.21.15.103:0/2827261256 learned_addr learned my addr 172.21.15.103:0/2827261256 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:22.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.647+0000 7f83e7fff640 1 --2- 172.21.15.103:0/2827261256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cc36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:22.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.647+0000 7f83f49ad640 1 -- 172.21.15.103:0/2827261256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cb5b40 msgr2=0x55e9a6cc36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:22.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.647+0000 7f83f49ad640 1 --2- 172.21.15.103:0/2827261256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cc36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.647+0000 7f83f49ad640 1 -- 172.21.15.103:0/2827261256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb45d0 msgr2=0x55e9a6cc3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:22.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.647+0000 7f83f49ad640 1 --2- 172.21.15.103:0/2827261256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cc3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.647+0000 7f83f49ad640 1 -- 172.21.15.103:0/2827261256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9a6d38a00 con 0x55e9a6cbf370 2024-09-19T06:58:22.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.647+0000 7f83e7fff640 1 --2- 172.21.15.103:0/2827261256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cc36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:58:22.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.647+0000 7f83effff640 1 --2- 172.21.15.103:0/2827261256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cc3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:58:22.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.647+0000 7f83f49ad640 1 --2- 172.21.15.103:0/2827261256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cbf370 0x55e9a6d384c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f83e8012be0 tx=0x7f83e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4590a6c523b11d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:22.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.647+0000 7f83cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:22.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.647+0000 7f83cf7fe640 1 -- 172.21.15.103:0/2827261256 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8020420 con 0x55e9a6cbf370 2024-09-19T06:58:22.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.647+0000 7f83cf7fe640 1 -- 172.21.15.103:0/2827261256 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f83e80205c0 con 0x55e9a6cbf370 2024-09-19T06:58:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.647+0000 7f83f87e9740 1 -- 172.21.15.103:0/2827261256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d8013b90 con 0x55e9a6cbf370 2024-09-19T06:58:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.647+0000 7f83cf7fe640 1 -- 172.21.15.103:0/2827261256 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f83e802f050 con 0x55e9a6cbf370 2024-09-19T06:58:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.647+0000 7f83f87e9740 10 client.18570 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.647+0000 7f83f87e9740 1 -- 172.21.15.103:0/2827261256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e9a6d39bf0 con 0x55e9a6cbf370 2024-09-19T06:58:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.648+0000 7f83cf7fe640 1 -- 172.21.15.103:0/2827261256 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d8013b90 con 0x55e9a6cbf370 2024-09-19T06:58:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.648+0000 7f83cf7fe640 1 -- 172.21.15.103:0/2827261256 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f83e800f040 con 0x55e9a6cbf370 2024-09-19T06:58:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.648+0000 7f83f87e9740 10 client.18570 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.648+0000 7f83f87e9740 10 client.18570 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:58:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.648+0000 7f83f87e9740 20 client.18570 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.648+0000 7f83f87e9740 1 --2- 172.21.15.103:0/2827261256 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55e9a6cce990 0x55e9a6d3c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.648+0000 7f83f87e9740 4 client.18570 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:58:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.648+0000 7f83f87e9740 1 -- 172.21.15.103:0/2827261256 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9a6b9e170 con 0x55e9a6cce990 2024-09-19T06:58:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.648+0000 7f83effff640 1 --2- 172.21.15.103:0/2827261256 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55e9a6cce990 0x55e9a6d3c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.648+0000 7f83effff640 1 --2- 172.21.15.103:0/2827261256 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55e9a6cce990 0x55e9a6d3c400 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=45c0a0ca3bf72689 server_cookie=e50995d31c740510 in_seq=0 out_seq=0 2024-09-19T06:58:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.648+0000 7f83cf7fe640 10 client.18570 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:58:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.649+0000 7f83cf7fe640 1 -- 172.21.15.103:0/2827261256 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9a6b9e170 con 0x55e9a6cce990 2024-09-19T06:58:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.649+0000 7f83cf7fe640 10 client.18570 handle_command_reply: tid=0 2024-09-19T06:58:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.649+0000 7f83f87e9740 1 client.18570 shutdown 2024-09-19T06:58:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.649+0000 7f83f87e9740 1 -- 172.21.15.103:0/2827261256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cbf370 msgr2=0x55e9a6d384c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.649+0000 7f83f87e9740 1 --2- 172.21.15.103:0/2827261256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cbf370 0x55e9a6d384c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f83e8012be0 tx=0x7f83e8004a40 comp rx=0 tx=0).stop 2024-09-19T06:58:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.649+0000 7f83f87e9740 1 -- 172.21.15.103:0/2827261256 shutdown_connections 2024-09-19T06:58:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.649+0000 7f83f87e9740 1 --2- 172.21.15.103:0/2827261256 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55e9a6cce990 0x55e9a6d3c400 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.649+0000 7f83f87e9740 1 --2- 172.21.15.103:0/2827261256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9a6cbf370 0x55e9a6d384c0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.649+0000 7f83f87e9740 1 --2- 172.21.15.103:0/2827261256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9a6cb5b40 0x55e9a6cc36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.649+0000 7f83f87e9740 1 --2- 172.21.15.103:0/2827261256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9a6cb45d0 0x55e9a6cc3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.649+0000 7f83f87e9740 1 -- 172.21.15.103:0/2827261256 >> 172.21.15.103:0/2827261256 conn(0x55e9a6c7d310 msgr2=0x55e9a6cafd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:22.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.649+0000 7f83cf7fe640 0 client.18570 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:58:22.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.649+0000 7f83f87e9740 1 -- 172.21.15.103:0/2827261256 shutdown_connections 2024-09-19T06:58:22.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.649+0000 7f83f87e9740 1 -- 172.21.15.103:0/2827261256 wait complete. 2024-09-19T06:58:22.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.649+0000 7f83f87e9740 20 client.18570 trim_cache size 0 max 16384 2024-09-19T06:58:22.666 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:58:22.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.650+0000 7f83f6410640 1 -- 172.21.15.103:0/3763220684 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83dc0a33b0 msgr2=0x7f83dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:22.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.650+0000 7f83f6410640 1 --2- 172.21.15.103:0/3763220684 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83dc0a33b0 0x7f83dc0a5870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f83e8005fd0 tx=0x7f83e8010740 comp rx=0 tx=0).stop 2024-09-19T06:58:22.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.650+0000 7f83f6410640 1 -- 172.21.15.103:0/3763220684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f0101de0 msgr2=0x7f83f019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:22.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.650+0000 7f83f6410640 1 --2- 172.21.15.103:0/3763220684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f0101de0 0x7f83f019a6a0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f83e0006010 tx=0x7f83e00042f0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.650+0000 7f83f6410640 1 -- 172.21.15.103:0/3763220684 shutdown_connections 2024-09-19T06:58:22.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.650+0000 7f83f6410640 1 --2- 172.21.15.103:0/3763220684 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f83dc0a33b0 0x7f83dc0a5870 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.650+0000 7f83f6410640 1 --2- 172.21.15.103:0/3763220684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f0106f10 0x7f83f019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.650+0000 7f83f6410640 1 --2- 172.21.15.103:0/3763220684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83f0102790 0x7f83f019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.650+0000 7f83f6410640 1 --2- 172.21.15.103:0/3763220684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f0101de0 0x7f83f019a6a0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:22.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.650+0000 7f83f6410640 1 -- 172.21.15.103:0/3763220684 >> 172.21.15.103:0/3763220684 conn(0x7f83f00fd8c0 msgr2=0x7f83f00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:22.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.650+0000 7f83f6410640 1 -- 172.21.15.103:0/3763220684 shutdown_connections 2024-09-19T06:58:22.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:22.650+0000 7f83f6410640 1 -- 172.21.15.103:0/3763220684 wait complete. 2024-09-19T06:58:22.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:22 smithi134 ceph-mon[35356]: pgmap v283: 97 pgs: 97 active+clean; 3.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-09-19T06:58:22.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:22 smithi174 ceph-mon[35637]: pgmap v283: 97 pgs: 97 active+clean; 3.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-09-19T06:58:23.274 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:58:23.519 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:58:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.964+0000 7feed52e6640 1 -- 172.21.15.103:0/2130811750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed0102480 msgr2=0x7feed010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.964+0000 7feed52e6640 1 --2- 172.21.15.103:0/2130811750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed0102480 0x7feed010e9b0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7feec8009cd0 tx=0x7feec802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.964+0000 7feed52e6640 1 -- 172.21.15.103:0/2130811750 shutdown_connections 2024-09-19T06:58:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.964+0000 7feed52e6640 1 --2- 172.21.15.103:0/2130811750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed0102480 0x7feed010e9b0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.964+0000 7feed52e6640 1 --2- 172.21.15.103:0/2130811750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0101ae0 0x7feed0101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.964+0000 7feed52e6640 1 --2- 172.21.15.103:0/2130811750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed0107b80 0x7feed0107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.964+0000 7feed52e6640 1 -- 172.21.15.103:0/2130811750 >> 172.21.15.103:0/2130811750 conn(0x7feed00fd7e0 msgr2=0x7feed00ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.964+0000 7feed52e6640 1 -- 172.21.15.103:0/2130811750 shutdown_connections 2024-09-19T06:58:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.964+0000 7feed52e6640 1 -- 172.21.15.103:0/2130811750 wait complete. 2024-09-19T06:58:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.964+0000 7feed52e6640 1 Processor -- start 2024-09-19T06:58:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feed52e6640 1 -- start start 2024-09-19T06:58:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feed52e6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed0101ae0 0x7feed0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feed52e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0102480 0x7feed0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feed52e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed0107b80 0x7feed019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feed52e6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feed0111f30 con 0x7feed0107b80 2024-09-19T06:58:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feed52e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feed0111db0 con 0x7feed0102480 2024-09-19T06:58:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feed52e6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feed01120b0 con 0x7feed0101ae0 2024-09-19T06:58:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feecf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed0107b80 0x7feed019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feecf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed0107b80 0x7feed019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38608/0 (socket says 172.21.15.103:38608) 2024-09-19T06:58:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feecf7fe640 1 -- 172.21.15.103:0/2788289150 learned_addr learned my addr 172.21.15.103:0/2788289150 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feeceffd640 1 --2- 172.21.15.103:0/2788289150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed0101ae0 0x7feed0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feece7fc640 1 --2- 172.21.15.103:0/2788289150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0102480 0x7feed0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feecf7fe640 1 -- 172.21.15.103:0/2788289150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed0101ae0 msgr2=0x7feed0198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feecf7fe640 1 --2- 172.21.15.103:0/2788289150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed0101ae0 0x7feed0198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feecf7fe640 1 -- 172.21.15.103:0/2788289150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0102480 msgr2=0x7feed0198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feecf7fe640 1 --2- 172.21.15.103:0/2788289150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0102480 0x7feed0198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feecf7fe640 1 -- 172.21.15.103:0/2788289150 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed019d330 con 0x7feed0107b80 2024-09-19T06:58:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feeceffd640 1 --2- 172.21.15.103:0/2788289150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed0101ae0 0x7feed0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.965+0000 7feece7fc640 1 --2- 172.21.15.103:0/2788289150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0102480 0x7feed0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.966+0000 7feecf7fe640 1 --2- 172.21.15.103:0/2788289150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed0107b80 0x7feed019cc00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7feec8005cc0 tx=0x7feec8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.966+0000 7feeb7fff640 1 -- 172.21.15.103:0/2788289150 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8004040 con 0x7feed0107b80 2024-09-19T06:58:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.966+0000 7feed52e6640 1 -- 172.21.15.103:0/2788289150 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed019d5c0 con 0x7feed0107b80 2024-09-19T06:58:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.966+0000 7feeb7fff640 1 -- 172.21.15.103:0/2788289150 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feec80046c0 con 0x7feed0107b80 2024-09-19T06:58:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.966+0000 7feeb7fff640 1 -- 172.21.15.103:0/2788289150 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8046480 con 0x7feed0107b80 2024-09-19T06:58:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.966+0000 7feed52e6640 1 -- 172.21.15.103:0/2788289150 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed01a4eb0 con 0x7feed0107b80 2024-09-19T06:58:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.968+0000 7feeb7fff640 1 -- 172.21.15.103:0/2788289150 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7feec80305a0 con 0x7feed0107b80 2024-09-19T06:58:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.968+0000 7feed8080740 1 Processor -- start 2024-09-19T06:58:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.968+0000 7feed8080740 1 -- start start 2024-09-19T06:58:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.968+0000 7feed8080740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de37af0 0x558e2de37ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.968+0000 7feed8080740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de38410 0x558e2de42770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.968+0000 7feed8080740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de42cb0 0x558e2de450a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feed8080740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e2dd9c2f0 con 0x558e2de37af0 2024-09-19T06:58:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feed8080740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558e2dd44270 con 0x558e2de42cb0 2024-09-19T06:58:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feed8080740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e2dd982b0 con 0x558e2de38410 2024-09-19T06:58:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feece7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de37af0 0x558e2de37ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feece7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de37af0 0x558e2de37ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38614/0 (socket says 172.21.15.103:38614) 2024-09-19T06:58:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feece7fc640 1 -- 172.21.15.103:0/559236680 learned_addr learned my addr 172.21.15.103:0/559236680 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feeceffd640 1 --2- 172.21.15.103:0/559236680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de38410 0x558e2de42770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feece7fc640 1 -- 172.21.15.103:0/559236680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de38410 msgr2=0x558e2de42770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feece7fc640 1 --2- 172.21.15.103:0/559236680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de38410 0x558e2de42770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feeb7fff640 1 --2- 172.21.15.103:0/2788289150 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feeb80a3460 0x7feeb80a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feece7fc640 1 -- 172.21.15.103:0/559236680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de42cb0 msgr2=0x558e2de450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feece7fc640 1 --2- 172.21.15.103:0/559236680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de42cb0 0x558e2de450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feece7fc640 1 -- 172.21.15.103:0/559236680 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e2dd43340 con 0x558e2de37af0 2024-09-19T06:58:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feecf7fe640 1 --2- 172.21.15.103:0/2788289150 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feeb80a3460 0x7feeb80a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feeb7fff640 1 -- 172.21.15.103:0/2788289150 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7feec802e030 con 0x7feed0107b80 2024-09-19T06:58:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feece7fc640 1 --2- 172.21.15.103:0/559236680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de37af0 0x558e2de37ed0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7feec400ccf0 tx=0x7feec4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41b352adc15d9e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feeb57fa640 1 -- 172.21.15.103:0/559236680 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4016070 con 0x558e2de37af0 2024-09-19T06:58:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feeb57fa640 1 -- 172.21.15.103:0/559236680 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feec40047a0 con 0x558e2de37af0 2024-09-19T06:58:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.969+0000 7feeb57fa640 1 -- 172.21.15.103:0/559236680 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec40127e0 con 0x558e2de37af0 2024-09-19T06:58:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.970+0000 7feed8080740 1 -- 172.21.15.103:0/559236680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de37af0 msgr2=0x558e2de37ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.970+0000 7feecf7fe640 1 --2- 172.21.15.103:0/2788289150 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feeb80a3460 0x7feeb80a5920 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7feec80eee30 tx=0x7feec8088040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.970+0000 7feed8080740 1 --2- 172.21.15.103:0/559236680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de37af0 0x558e2de37ed0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7feec400ccf0 tx=0x7feec4010740 comp rx=0 tx=0).stop 2024-09-19T06:58:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.970+0000 7feed8080740 1 -- 172.21.15.103:0/559236680 shutdown_connections 2024-09-19T06:58:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.970+0000 7feed8080740 1 --2- 172.21.15.103:0/559236680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de42cb0 0x558e2de450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.970+0000 7feed8080740 1 --2- 172.21.15.103:0/559236680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de38410 0x558e2de42770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.970+0000 7feed8080740 1 --2- 172.21.15.103:0/559236680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de37af0 0x558e2de37ed0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.970+0000 7feed8080740 1 -- 172.21.15.103:0/559236680 >> 172.21.15.103:0/559236680 conn(0x558e2de34060 msgr2=0x558e2de34440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.970+0000 7feed8080740 1 -- 172.21.15.103:0/559236680 shutdown_connections 2024-09-19T06:58:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.970+0000 7feed8080740 1 -- 172.21.15.103:0/559236680 wait complete. 2024-09-19T06:58:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feed8080740 1 -- start start 2024-09-19T06:58:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feed8080740 1 Processor -- start 2024-09-19T06:58:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feed8080740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de37af0 0x558e2de3faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feed8080740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de40030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feed8080740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de42cb0 0x558e2de40570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feed8080740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e2de45880 con 0x558e2de38410 2024-09-19T06:58:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feed8080740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558e2dd25290 con 0x558e2de37af0 2024-09-19T06:58:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feed8080740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e2de45a00 con 0x558e2de42cb0 2024-09-19T06:58:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feed8080740 10 client.0 _ll_register_callbacks cb 0x558e2dd56140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feeceffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de40030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feeceffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de40030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38624/0 (socket says 172.21.15.103:38624) 2024-09-19T06:58:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feeceffd640 1 -- 172.21.15.103:0/2568833220 learned_addr learned my addr 172.21.15.103:0/2568833220 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feece7fc640 1 --2- 172.21.15.103:0/2568833220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de42cb0 0x558e2de40570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feece7fc640 1 --2- 172.21.15.103:0/2568833220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de37af0 0x558e2de3faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feeceffd640 1 -- 172.21.15.103:0/2568833220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de42cb0 msgr2=0x558e2de40570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feeceffd640 1 --2- 172.21.15.103:0/2568833220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de42cb0 0x558e2de40570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feeceffd640 1 -- 172.21.15.103:0/2568833220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de37af0 msgr2=0x558e2de3faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feeceffd640 1 --2- 172.21.15.103:0/2568833220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de37af0 0x558e2de3faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feeceffd640 1 -- 172.21.15.103:0/2568833220 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e2daaf210 con 0x558e2de38410 2024-09-19T06:58:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feeceffd640 1 --2- 172.21.15.103:0/2568833220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de40030 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7feec0012480 tx=0x7feec0012940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996859ffbe3a6ce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.971+0000 7feeb2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.972+0000 7feeb2ff5640 1 -- 172.21.15.103:0/2568833220 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec001f060 con 0x558e2de38410 2024-09-19T06:58:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.972+0000 7feeb2ff5640 1 -- 172.21.15.103:0/2568833220 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7feec00109e0 con 0x558e2de38410 2024-09-19T06:58:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.972+0000 7feed8080740 1 -- 172.21.15.103:0/2568833220 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec40047a0 con 0x558e2de38410 2024-09-19T06:58:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.972+0000 7feeb2ff5640 1 -- 172.21.15.103:0/2568833220 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feec0010be0 con 0x558e2de38410 2024-09-19T06:58:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.972+0000 7feed8080740 10 client.18582 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.972+0000 7feed8080740 1 -- 172.21.15.103:0/2568833220 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e2db7b620 con 0x558e2de38410 2024-09-19T06:58:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.972+0000 7feeb2ff5640 1 -- 172.21.15.103:0/2568833220 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feec40047a0 con 0x558e2de38410 2024-09-19T06:58:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.972+0000 7feeb2ff5640 1 -- 172.21.15.103:0/2568833220 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feec0019030 con 0x558e2de38410 2024-09-19T06:58:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.972+0000 7feed8080740 10 client.18582 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.972+0000 7feed8080740 10 client.18582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:58:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.972+0000 7feed8080740 20 client.18582 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.972+0000 7feed8080740 1 --2- 172.21.15.103:0/2568833220 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558e2de52c10 0x558e2de55000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.972+0000 7feed8080740 4 client.18582 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:58:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.972+0000 7feed8080740 1 -- 172.21.15.103:0/2568833220 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e2dda9a20 con 0x558e2de52c10 2024-09-19T06:58:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.973+0000 7feece7fc640 1 --2- 172.21.15.103:0/2568833220 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558e2de52c10 0x558e2de55000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.973+0000 7feeb2ff5640 10 client.18582 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.973+0000 7feece7fc640 1 --2- 172.21.15.103:0/2568833220 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558e2de52c10 0x558e2de55000 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16a7e7d8782d26f9 server_cookie=e678f41f03e62c81 in_seq=0 out_seq=0 2024-09-19T06:58:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.974+0000 7feeb2ff5640 1 -- 172.21.15.103:0/2568833220 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e2dda9a20 con 0x558e2de52c10 2024-09-19T06:58:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.974+0000 7feeb2ff5640 10 client.18582 handle_command_reply: tid=0 2024-09-19T06:58:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.974+0000 7feed8080740 1 client.18582 shutdown 2024-09-19T06:58:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.974+0000 7feed8080740 1 -- 172.21.15.103:0/2568833220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 msgr2=0x558e2de40030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.974+0000 7feed8080740 1 --2- 172.21.15.103:0/2568833220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de40030 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7feec0012480 tx=0x7feec0012940 comp rx=0 tx=0).stop 2024-09-19T06:58:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.974+0000 7feed8080740 1 -- 172.21.15.103:0/2568833220 shutdown_connections 2024-09-19T06:58:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.974+0000 7feed8080740 1 --2- 172.21.15.103:0/2568833220 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558e2de52c10 0x558e2de55000 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.974+0000 7feed8080740 1 --2- 172.21.15.103:0/2568833220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de42cb0 0x558e2de40570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.974+0000 7feed8080740 1 --2- 172.21.15.103:0/2568833220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de40030 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.974+0000 7feeb2ff5640 0 client.18582 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.974+0000 7feed8080740 1 --2- 172.21.15.103:0/2568833220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de37af0 0x558e2de3faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.974+0000 7feed8080740 1 -- 172.21.15.103:0/2568833220 >> 172.21.15.103:0/2568833220 conn(0x558e2de34060 msgr2=0x558e2de39dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.974+0000 7feed8080740 1 -- 172.21.15.103:0/2568833220 shutdown_connections 2024-09-19T06:58:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.974+0000 7feed8080740 1 -- 172.21.15.103:0/2568833220 wait complete. 2024-09-19T06:58:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.974+0000 7feed8080740 20 client.18582 trim_cache size 0 max 16384 2024-09-19T06:58:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.988+0000 7feed8080740 1 Processor -- start 2024-09-19T06:58:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.988+0000 7feed8080740 1 -- start start 2024-09-19T06:58:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feed8080740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de37af0 0x558e2de3a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feed8080740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de403a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feed8080740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de42cb0 0x558e2de41a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feed8080740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feed01ad7e0 con 0x558e2de38410 2024-09-19T06:58:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feed8080740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feed01af320 con 0x558e2de42cb0 2024-09-19T06:58:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feed8080740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feed01adeb0 con 0x558e2de37af0 2024-09-19T06:58:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feeceffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feeceffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de403a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38628/0 (socket says 172.21.15.103:38628) 2024-09-19T06:58:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feeceffd640 1 -- 172.21.15.103:0/2903814567 learned_addr learned my addr 172.21.15.103:0/2903814567 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feece7fc640 1 --2- 172.21.15.103:0/2903814567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de37af0 0x558e2de3a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feece7fc640 1 --2- 172.21.15.103:0/2903814567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de42cb0 0x558e2de41a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feeceffd640 1 -- 172.21.15.103:0/2903814567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de37af0 msgr2=0x558e2de3a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feeceffd640 1 --2- 172.21.15.103:0/2903814567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de37af0 0x558e2de3a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feeceffd640 1 -- 172.21.15.103:0/2903814567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de42cb0 msgr2=0x558e2de41a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feeceffd640 1 --2- 172.21.15.103:0/2903814567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de42cb0 0x558e2de41a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feeceffd640 1 -- 172.21.15.103:0/2903814567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e2de35110 con 0x558e2de38410 2024-09-19T06:58:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feeceffd640 1 --2- 172.21.15.103:0/2903814567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de403a0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7feec0015040 tx=0x7feec000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf98c173ed658de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feeb4ff9640 1 -- 172.21.15.103:0/2903814567 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00107e0 con 0x558e2de38410 2024-09-19T06:58:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feeb4ff9640 1 -- 172.21.15.103:0/2903814567 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feec0010980 con 0x558e2de38410 2024-09-19T06:58:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.989+0000 7feeb4ff9640 1 -- 172.21.15.103:0/2903814567 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00055d0 con 0x558e2de38410 2024-09-19T06:58:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.990+0000 7feed8080740 1 -- 172.21.15.103:0/2903814567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 msgr2=0x558e2de403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.990+0000 7feed8080740 1 --2- 172.21.15.103:0/2903814567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de403a0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7feec0015040 tx=0x7feec000bf90 comp rx=0 tx=0).stop 2024-09-19T06:58:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.990+0000 7feed8080740 1 -- 172.21.15.103:0/2903814567 shutdown_connections 2024-09-19T06:58:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.990+0000 7feed8080740 1 --2- 172.21.15.103:0/2903814567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de42cb0 0x558e2de41a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.990+0000 7feed8080740 1 --2- 172.21.15.103:0/2903814567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de403a0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.990+0000 7feed8080740 1 --2- 172.21.15.103:0/2903814567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de37af0 0x558e2de3a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.990+0000 7feed8080740 1 -- 172.21.15.103:0/2903814567 >> 172.21.15.103:0/2903814567 conn(0x558e2de34060 msgr2=0x558e2de41350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.990+0000 7feed8080740 1 -- 172.21.15.103:0/2903814567 shutdown_connections 2024-09-19T06:58:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.990+0000 7feed8080740 1 -- 172.21.15.103:0/2903814567 wait complete. 2024-09-19T06:58:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feed8080740 1 -- start start 2024-09-19T06:58:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feed8080740 1 Processor -- start 2024-09-19T06:58:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feed8080740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de37af0 0x558e2de464a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feed8080740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de469e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feed8080740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de42cb0 0x558e2de4ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feed8080740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feed01aec50 con 0x558e2de38410 2024-09-19T06:58:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feed8080740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feed01ae580 con 0x558e2de37af0 2024-09-19T06:58:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feed8080740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feed01ab7e0 con 0x558e2de42cb0 2024-09-19T06:58:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feed8080740 10 client.0 _ll_register_callbacks cb 0x7feec000b890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feeceffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feeceffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de469e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38644/0 (socket says 172.21.15.103:38644) 2024-09-19T06:58:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feeceffd640 1 -- 172.21.15.103:0/485009050 learned_addr learned my addr 172.21.15.103:0/485009050 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feece7fc640 1 --2- 172.21.15.103:0/485009050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de42cb0 0x558e2de4ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feece7fc640 1 --2- 172.21.15.103:0/485009050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de37af0 0x558e2de464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feeceffd640 1 -- 172.21.15.103:0/485009050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de42cb0 msgr2=0x558e2de4ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feeceffd640 1 --2- 172.21.15.103:0/485009050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de42cb0 0x558e2de4ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feeceffd640 1 -- 172.21.15.103:0/485009050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de37af0 msgr2=0x558e2de464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feeceffd640 1 --2- 172.21.15.103:0/485009050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de37af0 0x558e2de464a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feeceffd640 1 -- 172.21.15.103:0/485009050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e2de33e90 con 0x558e2de38410 2024-09-19T06:58:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feeceffd640 1 --2- 172.21.15.103:0/485009050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de469e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7feec000f440 tx=0x7feec000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe82ff4c9ab0ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.991+0000 7feeb2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.992+0000 7feeb2ff5640 1 -- 172.21.15.103:0/485009050 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec003a060 con 0x558e2de38410 2024-09-19T06:58:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.992+0000 7feeb2ff5640 1 -- 172.21.15.103:0/485009050 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7feec000f8e0 con 0x558e2de38410 2024-09-19T06:58:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.992+0000 7feed8080740 1 -- 172.21.15.103:0/485009050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec0010980 con 0x558e2de38410 2024-09-19T06:58:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.992+0000 7feeb2ff5640 1 -- 172.21.15.103:0/485009050 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feec001f030 con 0x558e2de38410 2024-09-19T06:58:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.992+0000 7feed8080740 10 client.18588 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.992+0000 7feed8080740 1 -- 172.21.15.103:0/485009050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e2dec1680 con 0x558e2de38410 2024-09-19T06:58:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.992+0000 7feeb2ff5640 1 -- 172.21.15.103:0/485009050 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feec0010980 con 0x558e2de38410 2024-09-19T06:58:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.992+0000 7feeb2ff5640 1 -- 172.21.15.103:0/485009050 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feec0019030 con 0x558e2de38410 2024-09-19T06:58:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.992+0000 7feed8080740 10 client.18588 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.992+0000 7feed8080740 10 client.18588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:58:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.992+0000 7feed8080740 20 client.18588 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.992+0000 7feed8080740 1 --2- 172.21.15.103:0/485009050 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558e2dec1d50 0x558e2dec4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.992+0000 7feed8080740 4 client.18588 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:58:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.992+0000 7feed8080740 1 -- 172.21.15.103:0/485009050 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e2dda3d70 con 0x558e2dec1d50 2024-09-19T06:58:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.993+0000 7feece7fc640 1 --2- 172.21.15.103:0/485009050 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558e2dec1d50 0x558e2dec4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.993+0000 7feeb2ff5640 10 client.18588 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:23.993+0000 7feece7fc640 1 --2- 172.21.15.103:0/485009050 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558e2dec1d50 0x558e2dec4170 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b0c770472bb210 server_cookie=84cc682efc687e59 in_seq=0 out_seq=0 2024-09-19T06:58:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.006+0000 7feeb2ff5640 1 -- 172.21.15.103:0/485009050 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558e2dda3d70 con 0x558e2dec1d50 2024-09-19T06:58:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.006+0000 7feeb2ff5640 10 client.18588 handle_command_reply: tid=0 2024-09-19T06:58:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.006+0000 7feed8080740 1 client.18588 shutdown 2024-09-19T06:58:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.006+0000 7feed8080740 1 -- 172.21.15.103:0/485009050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 msgr2=0x558e2de469e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.006+0000 7feed8080740 1 --2- 172.21.15.103:0/485009050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de469e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7feec000f440 tx=0x7feec000bc20 comp rx=0 tx=0).stop 2024-09-19T06:58:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.006+0000 7feed8080740 1 -- 172.21.15.103:0/485009050 shutdown_connections 2024-09-19T06:58:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.006+0000 7feed8080740 1 --2- 172.21.15.103:0/485009050 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558e2dec1d50 0x558e2dec4170 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.006+0000 7feed8080740 1 --2- 172.21.15.103:0/485009050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e2de42cb0 0x558e2de4ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.006+0000 7feeb2ff5640 0 client.18588 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.006+0000 7feed8080740 1 --2- 172.21.15.103:0/485009050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e2de38410 0x558e2de469e0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.006+0000 7feed8080740 1 --2- 172.21.15.103:0/485009050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e2de37af0 0x558e2de464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.006+0000 7feed8080740 1 -- 172.21.15.103:0/485009050 >> 172.21.15.103:0/485009050 conn(0x558e2de34060 msgr2=0x558e2de41b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.006+0000 7feed8080740 1 -- 172.21.15.103:0/485009050 shutdown_connections 2024-09-19T06:58:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.006+0000 7feed8080740 1 -- 172.21.15.103:0/485009050 wait complete. 2024-09-19T06:58:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.007+0000 7feed8080740 20 client.18588 trim_cache size 0 max 16384 2024-09-19T06:58:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:58:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:58:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:58:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:58:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.007+0000 7feed52e6640 1 -- 172.21.15.103:0/2788289150 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feeb80a3460 msgr2=0x7feeb80a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.007+0000 7feed52e6640 1 --2- 172.21.15.103:0/2788289150 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feeb80a3460 0x7feeb80a5920 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7feec80eee30 tx=0x7feec8088040 comp rx=0 tx=0).stop 2024-09-19T06:58:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.007+0000 7feed52e6640 1 -- 172.21.15.103:0/2788289150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed0107b80 msgr2=0x7feed019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.007+0000 7feed52e6640 1 --2- 172.21.15.103:0/2788289150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed0107b80 0x7feed019cc00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7feec8005cc0 tx=0x7feec8003970 comp rx=0 tx=0).stop 2024-09-19T06:58:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.007+0000 7feed52e6640 1 -- 172.21.15.103:0/2788289150 shutdown_connections 2024-09-19T06:58:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.007+0000 7feed52e6640 1 --2- 172.21.15.103:0/2788289150 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feeb80a3460 0x7feeb80a5920 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.007+0000 7feed52e6640 1 --2- 172.21.15.103:0/2788289150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed0107b80 0x7feed019cc00 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.007+0000 7feed52e6640 1 --2- 172.21.15.103:0/2788289150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feed0102480 0x7feed0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.007+0000 7feed52e6640 1 --2- 172.21.15.103:0/2788289150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed0101ae0 0x7feed0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.007+0000 7feed52e6640 1 -- 172.21.15.103:0/2788289150 >> 172.21.15.103:0/2788289150 conn(0x7feed00fd7e0 msgr2=0x7feed00fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.007+0000 7feed52e6640 1 -- 172.21.15.103:0/2788289150 shutdown_connections 2024-09-19T06:58:24.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:24.007+0000 7feed52e6640 1 -- 172.21.15.103:0/2788289150 wait complete. 2024-09-19T06:58:24.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.013+0000 7f3cf01f9640 1 -- 172.21.15.103:0/2714716650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ce810c180 msgr2=0x7f3ce810e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:24.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.013+0000 7f3cf01f9640 1 --2- 172.21.15.103:0/2714716650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ce810c180 0x7f3ce810e620 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009ca0 tx=0x7f3ce002a950 comp rx=0 tx=0).stop 2024-09-19T06:58:24.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.013+0000 7f3cf01f9640 1 -- 172.21.15.103:0/2714716650 shutdown_connections 2024-09-19T06:58:24.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.013+0000 7f3cf01f9640 1 --2- 172.21.15.103:0/2714716650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ce810c180 0x7f3ce810e620 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.013+0000 7f3cf01f9640 1 --2- 172.21.15.103:0/2714716650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce8103670 0x7f3ce810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.013+0000 7f3cf01f9640 1 --2- 172.21.15.103:0/2714716650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ce8102d50 0x7f3ce8103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.013+0000 7f3cf01f9640 1 -- 172.21.15.103:0/2714716650 >> 172.21.15.103:0/2714716650 conn(0x7f3ce80fc7d0 msgr2=0x7f3ce80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:24.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.013+0000 7f3cf01f9640 1 -- 172.21.15.103:0/2714716650 shutdown_connections 2024-09-19T06:58:24.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.013+0000 7f3cf01f9640 1 -- 172.21.15.103:0/2714716650 wait complete. 2024-09-19T06:58:24.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3cf01f9640 1 Processor -- start 2024-09-19T06:58:24.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3cf01f9640 1 -- start start 2024-09-19T06:58:24.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3cf01f9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ce8102d50 0x7f3ce819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3cf01f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ce8103670 0x7f3ce819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3cf01f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce810c180 0x7f3ce819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3cf01f9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce8111fc0 con 0x7f3ce8103670 2024-09-19T06:58:24.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3cf01f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce8111e40 con 0x7f3ce810c180 2024-09-19T06:58:24.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3cf01f9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce8112140 con 0x7f3ce8102d50 2024-09-19T06:58:24.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3ced794640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ce8103670 0x7f3ce819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3ced794640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ce8103670 0x7f3ce819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38656/0 (socket says 172.21.15.103:38656) 2024-09-19T06:58:24.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3ced794640 1 -- 172.21.15.103:0/1508036031 learned_addr learned my addr 172.21.15.103:0/1508036031 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:24.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3cedf95640 1 --2- 172.21.15.103:0/1508036031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ce8102d50 0x7f3ce819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3cee796640 1 --2- 172.21.15.103:0/1508036031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce810c180 0x7f3ce819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3ced794640 1 -- 172.21.15.103:0/1508036031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ce8102d50 msgr2=0x7f3ce819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:24.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3ced794640 1 --2- 172.21.15.103:0/1508036031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ce8102d50 0x7f3ce819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3ced794640 1 -- 172.21.15.103:0/1508036031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce810c180 msgr2=0x7f3ce819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:24.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3ced794640 1 --2- 172.21.15.103:0/1508036031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce810c180 0x7f3ce819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3ced794640 1 -- 172.21.15.103:0/1508036031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce819f640 con 0x7f3ce8103670 2024-09-19T06:58:24.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.014+0000 7f3cee796640 1 --2- 172.21.15.103:0/1508036031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce810c180 0x7f3ce819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:24.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.015+0000 7f3ced794640 1 --2- 172.21.15.103:0/1508036031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ce8103670 0x7f3ce819abd0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800d740 tx=0x7f3cd800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:24.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.015+0000 7f3cdeffd640 1 -- 172.21.15.103:0/1508036031 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8017070 con 0x7f3ce8103670 2024-09-19T06:58:24.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.015+0000 7f3cf01f9640 1 -- 172.21.15.103:0/1508036031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce819f930 con 0x7f3ce8103670 2024-09-19T06:58:24.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.015+0000 7f3cdeffd640 1 -- 172.21.15.103:0/1508036031 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd8010460 con 0x7f3ce8103670 2024-09-19T06:58:24.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.015+0000 7f3cdeffd640 1 -- 172.21.15.103:0/1508036031 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd800f620 con 0x7f3ce8103670 2024-09-19T06:58:24.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.015+0000 7f3cf01f9640 1 -- 172.21.15.103:0/1508036031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce81a7210 con 0x7f3ce8103670 2024-09-19T06:58:24.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.017+0000 7f3cdeffd640 1 -- 172.21.15.103:0/1508036031 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3cd8010610 con 0x7f3ce8103670 2024-09-19T06:58:24.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.018+0000 7f3cf25d2740 1 Processor -- start 2024-09-19T06:58:24.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.018+0000 7f3cf25d2740 1 -- start start 2024-09-19T06:58:24.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.018+0000 7f3cf25d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca966b40 0x55d6ca965090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.018+0000 7f3cf25d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca9655d0 0x55d6ca965a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.018+0000 7f3cf25d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca970370 0x55d6ca972760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3cdeffd640 1 --2- 172.21.15.103:0/1508036031 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cd40a33b0 0x7f3cd40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3cf25d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d6ca8cb2f0 con 0x55d6ca966b40 2024-09-19T06:58:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3cf25d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d6ca872cf0 con 0x55d6ca970370 2024-09-19T06:58:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3cf25d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d6ca8c72b0 con 0x55d6ca9655d0 2024-09-19T06:58:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3cee796640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca966b40 0x55d6ca965090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3cee796640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca966b40 0x55d6ca965090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38660/0 (socket says 172.21.15.103:38660) 2024-09-19T06:58:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3cee796640 1 -- 172.21.15.103:0/3521363610 learned_addr learned my addr 172.21.15.103:0/3521363610 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3cee796640 1 --2- 172.21.15.103:0/1508036031 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cd40a33b0 0x7f3cd40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3cedf95640 1 --2- 172.21.15.103:0/3521363610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca9655d0 0x55d6ca965a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3cdeffd640 1 -- 172.21.15.103:0/1508036031 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f3cd80cfca0 con 0x7f3ce8103670 2024-09-19T06:58:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3ced794640 1 --2- 172.21.15.103:0/3521363610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca970370 0x55d6ca972760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3cee796640 1 -- 172.21.15.103:0/3521363610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca9655d0 msgr2=0x55d6ca965a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3cee796640 1 --2- 172.21.15.103:0/3521363610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca9655d0 0x55d6ca965a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3cee796640 1 -- 172.21.15.103:0/3521363610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca970370 msgr2=0x55d6ca972760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3cee796640 1 --2- 172.21.15.103:0/3521363610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca970370 0x55d6ca972760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3cee796640 1 -- 172.21.15.103:0/3521363610 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6ca8573a0 con 0x55d6ca966b40 2024-09-19T06:58:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3ced794640 1 --2- 172.21.15.103:0/3521363610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca970370 0x55d6ca972760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.019+0000 7f3cee796640 1 --2- 172.21.15.103:0/3521363610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca966b40 0x55d6ca965090 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0030040 tx=0x7f3ce002f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828b37fef9e7fe29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.020+0000 7f3cee796640 1 --2- 172.21.15.103:0/1508036031 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cd40a33b0 0x7f3cd40a5870 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3ce002f7b0 tx=0x7f3ce0042000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.020+0000 7f3cd3fff640 1 -- 172.21.15.103:0/3521363610 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce004c070 con 0x55d6ca966b40 2024-09-19T06:58:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.020+0000 7f3cd3fff640 1 -- 172.21.15.103:0/3521363610 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce0004720 con 0x55d6ca966b40 2024-09-19T06:58:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.020+0000 7f3cd3fff640 1 -- 172.21.15.103:0/3521363610 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0034040 con 0x55d6ca966b40 2024-09-19T06:58:24.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.020+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3521363610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca966b40 msgr2=0x55d6ca965090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.020+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3521363610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca966b40 0x55d6ca965090 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0030040 tx=0x7f3ce002f740 comp rx=0 tx=0).stop 2024-09-19T06:58:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.020+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3521363610 shutdown_connections 2024-09-19T06:58:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.020+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3521363610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca970370 0x55d6ca972760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.020+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3521363610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca9655d0 0x55d6ca965a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.020+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3521363610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca966b40 0x55d6ca965090 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.020+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3521363610 >> 172.21.15.103:0/3521363610 conn(0x55d6ca92e310 msgr2=0x55d6ca961c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.021+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3521363610 shutdown_connections 2024-09-19T06:58:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.021+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3521363610 wait complete. 2024-09-19T06:58:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.021+0000 7f3cf25d2740 1 -- start start 2024-09-19T06:58:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.021+0000 7f3cf25d2740 1 Processor -- start 2024-09-19T06:58:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.021+0000 7f3cf25d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca9655d0 0x55d6ca96fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.021+0000 7f3cf25d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca966b40 0x55d6ca96d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.021+0000 7f3cf25d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca970370 0x55d6ca96d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.021+0000 7f3cf25d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d6ca973010 con 0x55d6ca9655d0 2024-09-19T06:58:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.021+0000 7f3cf25d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d6ca886060 con 0x55d6ca966b40 2024-09-19T06:58:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.021+0000 7f3cf25d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d6ca973190 con 0x55d6ca970370 2024-09-19T06:58:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.021+0000 7f3cf25d2740 10 client.0 _ll_register_callbacks cb 0x55d6ca5d98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.021+0000 7f3cedf95640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca9655d0 0x55d6ca96fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3cedf95640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca9655d0 0x55d6ca96fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38666/0 (socket says 172.21.15.103:38666) 2024-09-19T06:58:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3cedf95640 1 -- 172.21.15.103:0/3267269281 learned_addr learned my addr 172.21.15.103:0/3267269281 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3ced794640 1 --2- 172.21.15.103:0/3267269281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca966b40 0x55d6ca96d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3cee796640 1 --2- 172.21.15.103:0/3267269281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca970370 0x55d6ca96d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3cedf95640 1 -- 172.21.15.103:0/3267269281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca970370 msgr2=0x55d6ca96d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3cedf95640 1 --2- 172.21.15.103:0/3267269281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca970370 0x55d6ca96d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3cedf95640 1 -- 172.21.15.103:0/3267269281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca966b40 msgr2=0x55d6ca96d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3cedf95640 1 --2- 172.21.15.103:0/3267269281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca966b40 0x55d6ca96d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3cedf95640 1 -- 172.21.15.103:0/3267269281 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6ca63c3c0 con 0x55d6ca9655d0 2024-09-19T06:58:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3cedf95640 1 --2- 172.21.15.103:0/3267269281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca9655d0 0x55d6ca96fda0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4011540 tx=0x7f3ce4011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19ca372beee7fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3cd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3cd17fa640 1 -- 172.21.15.103:0/3267269281 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce400f6d0 con 0x55d6ca9655d0 2024-09-19T06:58:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3267269281 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ce0004720 con 0x55d6ca9655d0 2024-09-19T06:58:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3cd17fa640 1 -- 172.21.15.103:0/3267269281 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f3ce400f870 con 0x55d6ca9655d0 2024-09-19T06:58:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3cd17fa640 1 -- 172.21.15.103:0/3267269281 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3ce400fab0 con 0x55d6ca9655d0 2024-09-19T06:58:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3cf25d2740 10 client.18600 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.022+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3267269281 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d6ca97eba0 con 0x55d6ca9655d0 2024-09-19T06:58:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.023+0000 7f3cd17fa640 1 -- 172.21.15.103:0/3267269281 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce0004720 con 0x55d6ca9655d0 2024-09-19T06:58:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.023+0000 7f3cd17fa640 1 -- 172.21.15.103:0/3267269281 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3ce401d400 con 0x55d6ca9655d0 2024-09-19T06:58:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.023+0000 7f3cf25d2740 10 client.18600 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.023+0000 7f3cf25d2740 10 client.18600 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:58:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.023+0000 7f3cf25d2740 20 client.18600 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.023+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3267269281 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55d6ca980060 0x55d6ca9824b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.023+0000 7f3cf25d2740 4 client.18600 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:58:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.023+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3267269281 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6ca8d8860 con 0x55d6ca980060 2024-09-19T06:58:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.023+0000 7f3ced794640 1 --2- 172.21.15.103:0/3267269281 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55d6ca980060 0x55d6ca9824b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.023+0000 7f3ced794640 1 --2- 172.21.15.103:0/3267269281 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55d6ca980060 0x55d6ca9824b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6069be487af183f3 server_cookie=24a5d2ccf2ce7f26 in_seq=0 out_seq=0 2024-09-19T06:58:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.023+0000 7f3cd17fa640 10 client.18600 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:58:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.024+0000 7f3cd17fa640 1 -- 172.21.15.103:0/3267269281 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d6ca8d8860 con 0x55d6ca980060 2024-09-19T06:58:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.024+0000 7f3cd17fa640 10 client.18600 handle_command_reply: tid=0 2024-09-19T06:58:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.024+0000 7f3cf25d2740 1 client.18600 shutdown 2024-09-19T06:58:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.024+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3267269281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca9655d0 msgr2=0x55d6ca96fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.024+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3267269281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca9655d0 0x55d6ca96fda0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4011540 tx=0x7f3ce4011a00 comp rx=0 tx=0).stop 2024-09-19T06:58:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.024+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3267269281 shutdown_connections 2024-09-19T06:58:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.024+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3267269281 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55d6ca980060 0x55d6ca9824b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.024+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3267269281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca970370 0x55d6ca96d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.024+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3267269281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca966b40 0x55d6ca96d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.024+0000 7f3cd17fa640 0 client.18600 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:58:24.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.024+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3267269281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca9655d0 0x55d6ca96fda0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.024+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3267269281 >> 172.21.15.103:0/3267269281 conn(0x55d6ca92e310 msgr2=0x55d6ca967960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:24.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.024+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3267269281 shutdown_connections 2024-09-19T06:58:24.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.024+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3267269281 wait complete. 2024-09-19T06:58:24.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.025+0000 7f3cf25d2740 20 client.18600 trim_cache size 0 max 16384 2024-09-19T06:58:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.037+0000 7f3cf25d2740 1 Processor -- start 2024-09-19T06:58:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.037+0000 7f3cf25d2740 1 -- start start 2024-09-19T06:58:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.037+0000 7f3cf25d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca9655d0 0x55d6ca968120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.037+0000 7f3cf25d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca966b40 0x55d6ca968660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.037+0000 7f3cf25d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca970370 0x55d6ca987d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.037+0000 7f3cf25d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce81afb60 con 0x55d6ca970370 2024-09-19T06:58:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.037+0000 7f3cf25d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce81b16a0 con 0x55d6ca9655d0 2024-09-19T06:58:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.037+0000 7f3cf25d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce81b0230 con 0x55d6ca966b40 2024-09-19T06:58:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.038+0000 7f3cee796640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca970370 0x55d6ca987d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.038+0000 7f3cee796640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca970370 0x55d6ca987d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38682/0 (socket says 172.21.15.103:38682) 2024-09-19T06:58:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.038+0000 7f3cee796640 1 -- 172.21.15.103:0/3748471933 learned_addr learned my addr 172.21.15.103:0/3748471933 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.038+0000 7f3ced794640 1 --2- 172.21.15.103:0/3748471933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca966b40 0x55d6ca968660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.038+0000 7f3cedf95640 1 --2- 172.21.15.103:0/3748471933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca9655d0 0x55d6ca968120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.038+0000 7f3cee796640 1 -- 172.21.15.103:0/3748471933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca966b40 msgr2=0x55d6ca968660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.038+0000 7f3cee796640 1 --2- 172.21.15.103:0/3748471933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca966b40 0x55d6ca968660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.038+0000 7f3cee796640 1 -- 172.21.15.103:0/3748471933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca9655d0 msgr2=0x55d6ca968120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.038+0000 7f3cee796640 1 --2- 172.21.15.103:0/3748471933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca9655d0 0x55d6ca968120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.038+0000 7f3cee796640 1 -- 172.21.15.103:0/3748471933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6ca962960 con 0x55d6ca970370 2024-09-19T06:58:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.038+0000 7f3cedf95640 1 --2- 172.21.15.103:0/3748471933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca9655d0 0x55d6ca968120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:58:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.038+0000 7f3cee796640 1 --2- 172.21.15.103:0/3748471933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca970370 0x55d6ca987d80 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00304b0 tx=0x7f3ce0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7584d01cefd716 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.038+0000 7f3cd37fe640 1 -- 172.21.15.103:0/3748471933 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce005a070 con 0x55d6ca970370 2024-09-19T06:58:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.038+0000 7f3cd37fe640 1 -- 172.21.15.103:0/3748471933 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce0004520 con 0x55d6ca970370 2024-09-19T06:58:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.038+0000 7f3cd37fe640 1 -- 172.21.15.103:0/3748471933 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce004c040 con 0x55d6ca970370 2024-09-19T06:58:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.039+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3748471933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca970370 msgr2=0x55d6ca987d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.039+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3748471933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca970370 0x55d6ca987d80 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00304b0 tx=0x7f3ce0009800 comp rx=0 tx=0).stop 2024-09-19T06:58:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.039+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3748471933 shutdown_connections 2024-09-19T06:58:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.039+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3748471933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca970370 0x55d6ca987d80 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.039+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3748471933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca966b40 0x55d6ca968660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.039+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3748471933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca9655d0 0x55d6ca968120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.039+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3748471933 >> 172.21.15.103:0/3748471933 conn(0x55d6ca92e310 msgr2=0x55d6ca96c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.039+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3748471933 shutdown_connections 2024-09-19T06:58:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.039+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3748471933 wait complete. 2024-09-19T06:58:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.039+0000 7f3cf25d2740 1 -- start start 2024-09-19T06:58:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.039+0000 7f3cf25d2740 1 Processor -- start 2024-09-19T06:58:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cf25d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca9655d0 0x55d6ca978b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cf25d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca966b40 0x55d6ca969d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cf25d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca970370 0x55d6ca96a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cf25d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce81b0fd0 con 0x55d6ca970370 2024-09-19T06:58:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cf25d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce81b0900 con 0x55d6ca966b40 2024-09-19T06:58:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cf25d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce81adb60 con 0x55d6ca9655d0 2024-09-19T06:58:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cf25d2740 10 client.0 _ll_register_callbacks cb 0x7f3ce4009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cee796640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca970370 0x55d6ca96a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cee796640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca970370 0x55d6ca96a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38694/0 (socket says 172.21.15.103:38694) 2024-09-19T06:58:24.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cee796640 1 -- 172.21.15.103:0/3020839568 learned_addr learned my addr 172.21.15.103:0/3020839568 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:24.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cedf95640 1 --2- 172.21.15.103:0/3020839568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca9655d0 0x55d6ca978b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3ced794640 1 --2- 172.21.15.103:0/3020839568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca966b40 0x55d6ca969d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cee796640 1 -- 172.21.15.103:0/3020839568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca9655d0 msgr2=0x55d6ca978b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:24.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cee796640 1 --2- 172.21.15.103:0/3020839568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca9655d0 0x55d6ca978b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cee796640 1 -- 172.21.15.103:0/3020839568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca966b40 msgr2=0x55d6ca969d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:24.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cee796640 1 --2- 172.21.15.103:0/3020839568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca966b40 0x55d6ca969d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cee796640 1 -- 172.21.15.103:0/3020839568 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6ca96aaf0 con 0x55d6ca970370 2024-09-19T06:58:24.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3cedf95640 1 --2- 172.21.15.103:0/3020839568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca9655d0 0x55d6ca978b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:24.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.040+0000 7f3ced794640 1 --2- 172.21.15.103:0/3020839568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca966b40 0x55d6ca969d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:24.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.041+0000 7f3cee796640 1 --2- 172.21.15.103:0/3020839568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca970370 0x55d6ca96a2c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f3ce002ae70 tx=0x7f3ce0030440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3101c8be1475dde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:24.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.041+0000 7f3cd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.041+0000 7f3cd17fa640 1 -- 172.21.15.103:0/3020839568 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce005a070 con 0x55d6ca970370 2024-09-19T06:58:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.041+0000 7f3cd17fa640 1 -- 172.21.15.103:0/3020839568 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f3ce0030800 con 0x55d6ca970370 2024-09-19T06:58:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.041+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3020839568 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ce0004520 con 0x55d6ca970370 2024-09-19T06:58:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.041+0000 7f3cd17fa640 1 -- 172.21.15.103:0/3020839568 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3ce004c040 con 0x55d6ca970370 2024-09-19T06:58:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.041+0000 7f3cf25d2740 10 client.18606 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.041+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3020839568 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d6ca9eac30 con 0x55d6ca970370 2024-09-19T06:58:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.041+0000 7f3cd17fa640 1 -- 172.21.15.103:0/3020839568 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce0004520 con 0x55d6ca970370 2024-09-19T06:58:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.041+0000 7f3cd17fa640 1 -- 172.21.15.103:0/3020839568 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3ce00519a0 con 0x55d6ca970370 2024-09-19T06:58:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.041+0000 7f3cf25d2740 10 client.18606 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.041+0000 7f3cf25d2740 10 client.18606 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:58:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.041+0000 7f3cf25d2740 20 client.18606 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.041+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3020839568 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55d6ca9eaf00 0x55d6ca9ed2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.041+0000 7f3cf25d2740 4 client.18606 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:58:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.041+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3020839568 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6ca8d2d70 con 0x55d6ca9eaf00 2024-09-19T06:58:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.042+0000 7f3cedf95640 1 --2- 172.21.15.103:0/3020839568 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55d6ca9eaf00 0x55d6ca9ed2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.042+0000 7f3cedf95640 1 --2- 172.21.15.103:0/3020839568 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55d6ca9eaf00 0x55d6ca9ed2f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=64c5744fab99fd17 server_cookie=c7250d428682397c in_seq=0 out_seq=0 2024-09-19T06:58:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.042+0000 7f3cd17fa640 10 client.18606 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:58:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.042+0000 7f3cd17fa640 1 -- 172.21.15.103:0/3020839568 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6ca8d2d70 con 0x55d6ca9eaf00 2024-09-19T06:58:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.042+0000 7f3cd17fa640 10 client.18606 handle_command_reply: tid=0 2024-09-19T06:58:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.042+0000 7f3cf25d2740 1 client.18606 shutdown 2024-09-19T06:58:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.042+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3020839568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca970370 msgr2=0x55d6ca96a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.042+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3020839568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca970370 0x55d6ca96a2c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f3ce002ae70 tx=0x7f3ce0030440 comp rx=0 tx=0).stop 2024-09-19T06:58:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.042+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3020839568 shutdown_connections 2024-09-19T06:58:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.042+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3020839568 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55d6ca9eaf00 0x55d6ca9ed2f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3020839568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d6ca970370 0x55d6ca96a2c0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3020839568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d6ca966b40 0x55d6ca969d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf25d2740 1 --2- 172.21.15.103:0/3020839568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6ca9655d0 0x55d6ca978b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cd17fa640 0 client.18606 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:58:24.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3020839568 >> 172.21.15.103:0/3020839568 conn(0x55d6ca92e310 msgr2=0x55d6ca9604e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:24.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3020839568 shutdown_connections 2024-09-19T06:58:24.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf25d2740 1 -- 172.21.15.103:0/3020839568 wait complete. 2024-09-19T06:58:24.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf25d2740 20 client.18606 trim_cache size 0 max 16384 2024-09-19T06:58:24.051 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:58:24.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf01f9640 1 -- 172.21.15.103:0/1508036031 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cd40a33b0 msgr2=0x7f3cd40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:24.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf01f9640 1 --2- 172.21.15.103:0/1508036031 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cd40a33b0 0x7f3cd40a5870 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3ce002f7b0 tx=0x7f3ce0042000 comp rx=0 tx=0).stop 2024-09-19T06:58:24.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf01f9640 1 -- 172.21.15.103:0/1508036031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ce8103670 msgr2=0x7f3ce819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:24.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf01f9640 1 --2- 172.21.15.103:0/1508036031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ce8103670 0x7f3ce819abd0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800d740 tx=0x7f3cd800db40 comp rx=0 tx=0).stop 2024-09-19T06:58:24.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf01f9640 1 -- 172.21.15.103:0/1508036031 shutdown_connections 2024-09-19T06:58:24.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf01f9640 1 --2- 172.21.15.103:0/1508036031 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cd40a33b0 0x7f3cd40a5870 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf01f9640 1 --2- 172.21.15.103:0/1508036031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ce810c180 0x7f3ce819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf01f9640 1 --2- 172.21.15.103:0/1508036031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ce8103670 0x7f3ce819abd0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf01f9640 1 --2- 172.21.15.103:0/1508036031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ce8102d50 0x7f3ce819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:24.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.043+0000 7f3cf01f9640 1 -- 172.21.15.103:0/1508036031 >> 172.21.15.103:0/1508036031 conn(0x7f3ce80fc7d0 msgr2=0x7f3ce810d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:24.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.044+0000 7f3cf01f9640 1 -- 172.21.15.103:0/1508036031 shutdown_connections 2024-09-19T06:58:24.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:24.044+0000 7f3cf01f9640 1 -- 172.21.15.103:0/1508036031 wait complete. 2024-09-19T06:58:24.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:24 smithi103 ceph-mon[31158]: pgmap v284: 97 pgs: 97 active+clean; 3.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T06:58:24.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:24 smithi134 ceph-mon[35356]: pgmap v284: 97 pgs: 97 active+clean; 3.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T06:58:24.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:24 smithi174 ceph-mon[35637]: pgmap v284: 97 pgs: 97 active+clean; 3.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T06:58:24.773 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:58:25.020 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:58:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.455+0000 7fc67824f640 1 -- 172.21.15.103:0/1365421407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc670100e30 msgr2=0x7fc670104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.455+0000 7fc67824f640 1 --2- 172.21.15.103:0/1365421407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc670100e30 0x7fc670104e80 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc66000aa10 tx=0x7fc66002b720 comp rx=0 tx=0).stop 2024-09-19T06:58:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.455+0000 7fc67824f640 1 -- 172.21.15.103:0/1365421407 shutdown_connections 2024-09-19T06:58:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.455+0000 7fc67824f640 1 --2- 172.21.15.103:0/1365421407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6701055b0 0x7fc67010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.455+0000 7fc67824f640 1 --2- 172.21.15.103:0/1365421407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc670100e30 0x7fc670104e80 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.455+0000 7fc67824f640 1 --2- 172.21.15.103:0/1365421407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc670100480 0x7fc670100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.455+0000 7fc67824f640 1 -- 172.21.15.103:0/1365421407 >> 172.21.15.103:0/1365421407 conn(0x7fc670077210 msgr2=0x7fc6700fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.455+0000 7fc67824f640 1 -- 172.21.15.103:0/1365421407 shutdown_connections 2024-09-19T06:58:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.455+0000 7fc67824f640 1 -- 172.21.15.103:0/1365421407 wait complete. 2024-09-19T06:58:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.455+0000 7fc67824f640 1 Processor -- start 2024-09-19T06:58:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc67824f640 1 -- start start 2024-09-19T06:58:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc67824f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc670100480 0x7fc67019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc67824f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc670100e30 0x7fc67019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc67824f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6701055b0 0x7fc67019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc67824f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc67010fd90 con 0x7fc6701055b0 2024-09-19T06:58:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc67824f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc67010fc10 con 0x7fc670100e30 2024-09-19T06:58:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc67824f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc67010ff10 con 0x7fc670100480 2024-09-19T06:58:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc6767ec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6701055b0 0x7fc67019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc6767ec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6701055b0 0x7fc67019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38708/0 (socket says 172.21.15.103:38708) 2024-09-19T06:58:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc6767ec640 1 -- 172.21.15.103:0/3413975206 learned_addr learned my addr 172.21.15.103:0/3413975206 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc675feb640 1 --2- 172.21.15.103:0/3413975206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc670100480 0x7fc67019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc6757ea640 1 --2- 172.21.15.103:0/3413975206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc670100e30 0x7fc67019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc6767ec640 1 -- 172.21.15.103:0/3413975206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc670100480 msgr2=0x7fc67019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc6767ec640 1 --2- 172.21.15.103:0/3413975206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc670100480 0x7fc67019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc6767ec640 1 -- 172.21.15.103:0/3413975206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc670100e30 msgr2=0x7fc67019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc6767ec640 1 --2- 172.21.15.103:0/3413975206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc670100e30 0x7fc67019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc6767ec640 1 -- 172.21.15.103:0/3413975206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67019f600 con 0x7fc6701055b0 2024-09-19T06:58:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.456+0000 7fc6757ea640 1 --2- 172.21.15.103:0/3413975206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc670100e30 0x7fc67019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:58:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.457+0000 7fc6767ec640 1 --2- 172.21.15.103:0/3413975206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6701055b0 0x7fc67019ef20 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc66800eaa0 tx=0x7fc66800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.457+0000 7fc666ffd640 1 -- 172.21.15.103:0/3413975206 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66800cdb0 con 0x7fc6701055b0 2024-09-19T06:58:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.457+0000 7fc67824f640 1 -- 172.21.15.103:0/3413975206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67019f8f0 con 0x7fc6701055b0 2024-09-19T06:58:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.457+0000 7fc666ffd640 1 -- 172.21.15.103:0/3413975206 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc668005020 con 0x7fc6701055b0 2024-09-19T06:58:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.457+0000 7fc666ffd640 1 -- 172.21.15.103:0/3413975206 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6680116b0 con 0x7fc6701055b0 2024-09-19T06:58:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.457+0000 7fc67824f640 1 -- 172.21.15.103:0/3413975206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6701a71d0 con 0x7fc6701055b0 2024-09-19T06:58:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.459+0000 7fc666ffd640 1 -- 172.21.15.103:0/3413975206 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc668005560 con 0x7fc6701055b0 2024-09-19T06:58:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.459+0000 7fc67a628740 1 Processor -- start 2024-09-19T06:58:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.459+0000 7fc67a628740 1 -- start start 2024-09-19T06:58:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc67a628740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d25b40 0x55f785d24090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc67a628740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785d245d0 0x55f785d24a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc67a628740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d2f370 0x55f785d31760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc666ffd640 1 --2- 172.21.15.103:0/3413975206 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc65c0a33b0 0x7fc65c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc67a628740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f785c8a2f0 con 0x55f785d25b40 2024-09-19T06:58:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc67a628740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f785c31cf0 con 0x55f785d2f370 2024-09-19T06:58:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc67a628740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f785c862b0 con 0x55f785d245d0 2024-09-19T06:58:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc6757ea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d25b40 0x55f785d24090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc6757ea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d25b40 0x55f785d24090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38710/0 (socket says 172.21.15.103:38710) 2024-09-19T06:58:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc6757ea640 1 -- 172.21.15.103:0/2481852701 learned_addr learned my addr 172.21.15.103:0/2481852701 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc6767ec640 1 --2- 172.21.15.103:0/3413975206 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc65c0a33b0 0x7fc65c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc675feb640 1 --2- 172.21.15.103:0/2481852701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785d245d0 0x55f785d24a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc666ffd640 1 -- 172.21.15.103:0/3413975206 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7fc6680cffe0 con 0x7fc6701055b0 2024-09-19T06:58:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc6757ea640 1 -- 172.21.15.103:0/2481852701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785d245d0 msgr2=0x55f785d24a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc6757ea640 1 --2- 172.21.15.103:0/2481852701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785d245d0 0x55f785d24a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc6757ea640 1 -- 172.21.15.103:0/2481852701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d2f370 msgr2=0x55f785d31760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc6757ea640 1 --2- 172.21.15.103:0/2481852701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d2f370 0x55f785d31760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc6757ea640 1 -- 172.21.15.103:0/2481852701 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f785c163a0 con 0x55f785d25b40 2024-09-19T06:58:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc6757ea640 1 --2- 172.21.15.103:0/2481852701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d25b40 0x55f785d24090 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc66000a7d0 tx=0x7fc660030e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea07906f853ff47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc6767ec640 1 --2- 172.21.15.103:0/3413975206 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc65c0a33b0 0x7fc65c0a5870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc6680cdd10 tx=0x7fc66806f0d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.460+0000 7fc65bfff640 1 -- 172.21.15.103:0/2481852701 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660036040 con 0x55f785d25b40 2024-09-19T06:58:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.461+0000 7fc65bfff640 1 -- 172.21.15.103:0/2481852701 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc660030980 con 0x55f785d25b40 2024-09-19T06:58:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.461+0000 7fc65bfff640 1 -- 172.21.15.103:0/2481852701 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6600048c0 con 0x55f785d25b40 2024-09-19T06:58:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.461+0000 7fc67a628740 1 -- 172.21.15.103:0/2481852701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d25b40 msgr2=0x55f785d24090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.461+0000 7fc67a628740 1 --2- 172.21.15.103:0/2481852701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d25b40 0x55f785d24090 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc66000a7d0 tx=0x7fc660030e00 comp rx=0 tx=0).stop 2024-09-19T06:58:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.461+0000 7fc67a628740 1 -- 172.21.15.103:0/2481852701 shutdown_connections 2024-09-19T06:58:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.461+0000 7fc67a628740 1 --2- 172.21.15.103:0/2481852701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d2f370 0x55f785d31760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.461+0000 7fc67a628740 1 --2- 172.21.15.103:0/2481852701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785d245d0 0x55f785d24a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.461+0000 7fc67a628740 1 --2- 172.21.15.103:0/2481852701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d25b40 0x55f785d24090 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.461+0000 7fc67a628740 1 -- 172.21.15.103:0/2481852701 >> 172.21.15.103:0/2481852701 conn(0x55f785ced310 msgr2=0x55f785d20c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.461+0000 7fc67a628740 1 -- 172.21.15.103:0/2481852701 shutdown_connections 2024-09-19T06:58:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.461+0000 7fc67a628740 1 -- 172.21.15.103:0/2481852701 wait complete. 2024-09-19T06:58:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.462+0000 7fc67a628740 1 -- start start 2024-09-19T06:58:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.462+0000 7fc67a628740 1 Processor -- start 2024-09-19T06:58:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.462+0000 7fc67a628740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d245d0 0x55f785d2ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.462+0000 7fc67a628740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785d25b40 0x55f785d2c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.462+0000 7fc67a628740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d2f370 0x55f785d2c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.462+0000 7fc67a628740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f785d32010 con 0x55f785d245d0 2024-09-19T06:58:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.462+0000 7fc67a628740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f785c45060 con 0x55f785d2f370 2024-09-19T06:58:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.462+0000 7fc67a628740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f785d32190 con 0x55f785d25b40 2024-09-19T06:58:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.462+0000 7fc67a628740 10 client.0 _ll_register_callbacks cb 0x55f7859988c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.462+0000 7fc6757ea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d245d0 0x55f785d2ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.462+0000 7fc6757ea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d245d0 0x55f785d2ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38712/0 (socket says 172.21.15.103:38712) 2024-09-19T06:58:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.462+0000 7fc6757ea640 1 -- 172.21.15.103:0/3200005065 learned_addr learned my addr 172.21.15.103:0/3200005065 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.462+0000 7fc6757ea640 1 --2- 172.21.15.103:0/3200005065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d2f370 0x55f785d2c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.462+0000 7fc675feb640 1 --2- 172.21.15.103:0/3200005065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785d25b40 0x55f785d2c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc6757ea640 1 -- 172.21.15.103:0/3200005065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785d25b40 msgr2=0x55f785d2c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc6757ea640 1 --2- 172.21.15.103:0/3200005065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785d25b40 0x55f785d2c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc6757ea640 1 -- 172.21.15.103:0/3200005065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d2f370 msgr2=0x55f785d2c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc6757ea640 1 --2- 172.21.15.103:0/3200005065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d2f370 0x55f785d2c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc6757ea640 1 -- 172.21.15.103:0/3200005065 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7859fb3c0 con 0x55f785d245d0 2024-09-19T06:58:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc675feb640 1 --2- 172.21.15.103:0/3200005065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785d25b40 0x55f785d2c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:58:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc6757ea640 1 --2- 172.21.15.103:0/3200005065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d245d0 0x55f785d2ed50 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fc660005c40 tx=0x7fc660005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736c1a522eb95057 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc6597fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc6597fa640 1 -- 172.21.15.103:0/3200005065 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660050060 con 0x55f785d245d0 2024-09-19T06:58:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc67a628740 1 -- 172.21.15.103:0/3200005065 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc660030980 con 0x55f785d245d0 2024-09-19T06:58:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc6597fa640 1 -- 172.21.15.103:0/3200005065 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7fc660004480 con 0x55f785d245d0 2024-09-19T06:58:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc6597fa640 1 -- 172.21.15.103:0/3200005065 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc6600035f0 con 0x55f785d245d0 2024-09-19T06:58:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc67a628740 10 client.18618 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc67a628740 1 -- 172.21.15.103:0/3200005065 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f785d3da90 con 0x55f785d245d0 2024-09-19T06:58:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc6597fa640 1 -- 172.21.15.103:0/3200005065 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc660030980 con 0x55f785d245d0 2024-09-19T06:58:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc6597fa640 1 -- 172.21.15.103:0/3200005065 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc66000b040 con 0x55f785d245d0 2024-09-19T06:58:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc67a628740 10 client.18618 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.463+0000 7fc67a628740 10 client.18618 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:58:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.464+0000 7fc67a628740 20 client.18618 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.464+0000 7fc67a628740 1 --2- 172.21.15.103:0/3200005065 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55f785d3ede0 0x55f785d411d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.464+0000 7fc67a628740 4 client.18618 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:58:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.464+0000 7fc67a628740 1 -- 172.21.15.103:0/3200005065 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f785c97860 con 0x55f785d3ede0 2024-09-19T06:58:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.464+0000 7fc675feb640 1 --2- 172.21.15.103:0/3200005065 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55f785d3ede0 0x55f785d411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.464+0000 7fc675feb640 1 --2- 172.21.15.103:0/3200005065 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55f785d3ede0 0x55f785d411d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f14311278f2df587 server_cookie=9e73ba6c05db954b in_seq=0 out_seq=0 2024-09-19T06:58:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.464+0000 7fc6597fa640 10 client.18618 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:58:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.465+0000 7fc6597fa640 1 -- 172.21.15.103:0/3200005065 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f785c97860 con 0x55f785d3ede0 2024-09-19T06:58:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.465+0000 7fc6597fa640 10 client.18618 handle_command_reply: tid=0 2024-09-19T06:58:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.465+0000 7fc67a628740 1 client.18618 shutdown 2024-09-19T06:58:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.465+0000 7fc67a628740 1 -- 172.21.15.103:0/3200005065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d245d0 msgr2=0x55f785d2ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.465+0000 7fc67a628740 1 --2- 172.21.15.103:0/3200005065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d245d0 0x55f785d2ed50 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fc660005c40 tx=0x7fc660005b90 comp rx=0 tx=0).stop 2024-09-19T06:58:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.465+0000 7fc67a628740 1 -- 172.21.15.103:0/3200005065 shutdown_connections 2024-09-19T06:58:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.465+0000 7fc67a628740 1 --2- 172.21.15.103:0/3200005065 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55f785d3ede0 0x55f785d411d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.465+0000 7fc67a628740 1 --2- 172.21.15.103:0/3200005065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d2f370 0x55f785d2c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.465+0000 7fc67a628740 1 --2- 172.21.15.103:0/3200005065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785d25b40 0x55f785d2c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.465+0000 7fc67a628740 1 --2- 172.21.15.103:0/3200005065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d245d0 0x55f785d2ed50 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.465+0000 7fc6597fa640 0 client.18618 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:58:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.465+0000 7fc67a628740 1 -- 172.21.15.103:0/3200005065 >> 172.21.15.103:0/3200005065 conn(0x55f785ced310 msgr2=0x55f785d26960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.465+0000 7fc67a628740 1 -- 172.21.15.103:0/3200005065 shutdown_connections 2024-09-19T06:58:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.465+0000 7fc67a628740 1 -- 172.21.15.103:0/3200005065 wait complete. 2024-09-19T06:58:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.465+0000 7fc67a628740 20 client.18618 trim_cache size 0 max 16384 2024-09-19T06:58:25.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.479+0000 7fc67a628740 1 Processor -- start 2024-09-19T06:58:25.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.479+0000 7fc67a628740 1 -- start start 2024-09-19T06:58:25.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.479+0000 7fc67a628740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785d245d0 0x55f785d26fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.479+0000 7fc67a628740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d25b40 0x55f785d27500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.479+0000 7fc67a628740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d2f370 0x55f785d46800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.479+0000 7fc67a628740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc6701af9e0 con 0x55f785d25b40 2024-09-19T06:58:25.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.479+0000 7fc67a628740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc6701b1520 con 0x55f785d2f370 2024-09-19T06:58:25.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.479+0000 7fc67a628740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc6701b00b0 con 0x55f785d245d0 2024-09-19T06:58:25.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.479+0000 7fc675feb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d25b40 0x55f785d27500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.479+0000 7fc675feb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d25b40 0x55f785d27500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38724/0 (socket says 172.21.15.103:38724) 2024-09-19T06:58:25.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.479+0000 7fc675feb640 1 -- 172.21.15.103:0/1832672653 learned_addr learned my addr 172.21.15.103:0/1832672653 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.480+0000 7fc6757ea640 1 --2- 172.21.15.103:0/1832672653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785d245d0 0x55f785d26fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.480+0000 7fc6757ea640 1 --2- 172.21.15.103:0/1832672653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d2f370 0x55f785d46800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.480+0000 7fc675feb640 1 -- 172.21.15.103:0/1832672653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785d245d0 msgr2=0x55f785d26fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.480+0000 7fc675feb640 1 --2- 172.21.15.103:0/1832672653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785d245d0 0x55f785d26fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.480+0000 7fc675feb640 1 -- 172.21.15.103:0/1832672653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d2f370 msgr2=0x55f785d46800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.480+0000 7fc675feb640 1 --2- 172.21.15.103:0/1832672653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d2f370 0x55f785d46800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.480+0000 7fc675feb640 1 -- 172.21.15.103:0/1832672653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f785d2e970 con 0x55f785d25b40 2024-09-19T06:58:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.480+0000 7fc6757ea640 1 --2- 172.21.15.103:0/1832672653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d2f370 0x55f785d46800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:58:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.480+0000 7fc675feb640 1 --2- 172.21.15.103:0/1832672653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d25b40 0x55f785d27500 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x55f785d2b890 tx=0x7fc66c00b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac65e8a5fb8e2b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.480+0000 7fc65b7fe640 1 -- 172.21.15.103:0/1832672653 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c0142d0 con 0x55f785d25b40 2024-09-19T06:58:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.480+0000 7fc65b7fe640 1 -- 172.21.15.103:0/1832672653 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc66c01b810 con 0x55f785d25b40 2024-09-19T06:58:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.480+0000 7fc65b7fe640 1 -- 172.21.15.103:0/1832672653 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c0129f0 con 0x55f785d25b40 2024-09-19T06:58:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.481+0000 7fc67a628740 1 -- 172.21.15.103:0/1832672653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d25b40 msgr2=0x55f785d27500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.481+0000 7fc67a628740 1 --2- 172.21.15.103:0/1832672653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d25b40 0x55f785d27500 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x55f785d2b890 tx=0x7fc66c00b4d0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.481+0000 7fc67a628740 1 -- 172.21.15.103:0/1832672653 shutdown_connections 2024-09-19T06:58:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.481+0000 7fc67a628740 1 --2- 172.21.15.103:0/1832672653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d2f370 0x55f785d46800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.481+0000 7fc67a628740 1 --2- 172.21.15.103:0/1832672653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d25b40 0x55f785d27500 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.481+0000 7fc67a628740 1 --2- 172.21.15.103:0/1832672653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785d245d0 0x55f785d26fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.481+0000 7fc67a628740 1 -- 172.21.15.103:0/1832672653 >> 172.21.15.103:0/1832672653 conn(0x55f785ced310 msgr2=0x55f785d2e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.481+0000 7fc67a628740 1 -- 172.21.15.103:0/1832672653 shutdown_connections 2024-09-19T06:58:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.481+0000 7fc67a628740 1 -- 172.21.15.103:0/1832672653 wait complete. 2024-09-19T06:58:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.481+0000 7fc67a628740 1 -- start start 2024-09-19T06:58:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.481+0000 7fc67a628740 1 Processor -- start 2024-09-19T06:58:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc67a628740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d245d0 0x55f785d338d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc67a628740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d2f370 0x55f785d33e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc67a628740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785da67c0 0x55f785da8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc67a628740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc6701b0e50 con 0x55f785d2f370 2024-09-19T06:58:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc67a628740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc6701b0780 con 0x55f785d245d0 2024-09-19T06:58:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc67a628740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc6701ad9e0 con 0x55f785da67c0 2024-09-19T06:58:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc67a628740 10 client.0 _ll_register_callbacks cb 0x7fc66c00b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc675feb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d2f370 0x55f785d33e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc675feb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d2f370 0x55f785d33e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38726/0 (socket says 172.21.15.103:38726) 2024-09-19T06:58:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc675feb640 1 -- 172.21.15.103:0/4014971549 learned_addr learned my addr 172.21.15.103:0/4014971549 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc6757ea640 1 --2- 172.21.15.103:0/4014971549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d245d0 0x55f785d338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc6757ea640 1 --2- 172.21.15.103:0/4014971549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785da67c0 0x55f785da8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc675feb640 1 -- 172.21.15.103:0/4014971549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785da67c0 msgr2=0x55f785da8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc675feb640 1 --2- 172.21.15.103:0/4014971549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785da67c0 0x55f785da8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc675feb640 1 -- 172.21.15.103:0/4014971549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d245d0 msgr2=0x55f785d338d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc675feb640 1 --2- 172.21.15.103:0/4014971549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d245d0 0x55f785d338d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc675feb640 1 -- 172.21.15.103:0/4014971549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f785da91c0 con 0x55f785d2f370 2024-09-19T06:58:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc6757ea640 1 --2- 172.21.15.103:0/4014971549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785da67c0 0x55f785da8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:58:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc675feb640 1 --2- 172.21.15.103:0/4014971549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d2f370 0x55f785d33e10 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fc66c014040 tx=0x7fc66c0141c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fdd971ffa7826c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc6597fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.482+0000 7fc6597fa640 1 -- 172.21.15.103:0/4014971549 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c01b530 con 0x55f785d2f370 2024-09-19T06:58:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.483+0000 7fc6597fa640 1 -- 172.21.15.103:0/4014971549 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7fc66c01b9c0 con 0x55f785d2f370 2024-09-19T06:58:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.483+0000 7fc67a628740 1 -- 172.21.15.103:0/4014971549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc66c01b810 con 0x55f785d2f370 2024-09-19T06:58:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.483+0000 7fc6597fa640 1 -- 172.21.15.103:0/4014971549 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc66c01bbc0 con 0x55f785d2f370 2024-09-19T06:58:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.483+0000 7fc67a628740 10 client.18624 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.483+0000 7fc67a628740 1 -- 172.21.15.103:0/4014971549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f785daa430 con 0x55f785d2f370 2024-09-19T06:58:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.483+0000 7fc6597fa640 1 -- 172.21.15.103:0/4014971549 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc66c01b810 con 0x55f785d2f370 2024-09-19T06:58:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.483+0000 7fc6597fa640 1 -- 172.21.15.103:0/4014971549 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc66c0245c0 con 0x55f785d2f370 2024-09-19T06:58:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.483+0000 7fc67a628740 10 client.18624 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.483+0000 7fc67a628740 10 client.18624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:58:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.483+0000 7fc67a628740 20 client.18624 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.483+0000 7fc67a628740 1 --2- 172.21.15.103:0/4014971549 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55f785daab90 0x55f785dacfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.483+0000 7fc67a628740 4 client.18624 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:58:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.483+0000 7fc67a628740 1 -- 172.21.15.103:0/4014971549 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f785c91d70 con 0x55f785daab90 2024-09-19T06:58:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.483+0000 7fc6757ea640 1 --2- 172.21.15.103:0/4014971549 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55f785daab90 0x55f785dacfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc6757ea640 1 --2- 172.21.15.103:0/4014971549 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55f785daab90 0x55f785dacfb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0215b1afb45e8a9 server_cookie=1eca9dc3954d47e5 in_seq=0 out_seq=0 2024-09-19T06:58:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc6597fa640 10 client.18624 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:58:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc6597fa640 1 -- 172.21.15.103:0/4014971549 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f785c91d70 con 0x55f785daab90 2024-09-19T06:58:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc6597fa640 10 client.18624 handle_command_reply: tid=0 2024-09-19T06:58:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc67a628740 1 client.18624 shutdown 2024-09-19T06:58:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc67a628740 1 -- 172.21.15.103:0/4014971549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d2f370 msgr2=0x55f785d33e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc67a628740 1 --2- 172.21.15.103:0/4014971549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d2f370 0x55f785d33e10 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fc66c014040 tx=0x7fc66c0141c0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc67a628740 1 -- 172.21.15.103:0/4014971549 shutdown_connections 2024-09-19T06:58:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc67a628740 1 --2- 172.21.15.103:0/4014971549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f785da67c0 0x55f785da8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc67a628740 1 --2- 172.21.15.103:0/4014971549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f785d2f370 0x55f785d33e10 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc67a628740 1 --2- 172.21.15.103:0/4014971549 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55f785daab90 0x55f785dacfb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc67a628740 1 --2- 172.21.15.103:0/4014971549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f785d245d0 0x55f785d338d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc6597fa640 0 client.18624 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:58:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc67a628740 1 -- 172.21.15.103:0/4014971549 >> 172.21.15.103:0/4014971549 conn(0x55f785ced310 msgr2=0x55f785d1f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc67a628740 1 -- 172.21.15.103:0/4014971549 shutdown_connections 2024-09-19T06:58:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc67a628740 1 -- 172.21.15.103:0/4014971549 wait complete. 2024-09-19T06:58:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.484+0000 7fc67a628740 20 client.18624 trim_cache size 0 max 16384 2024-09-19T06:58:25.499 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:58:25.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.485+0000 7fc67824f640 1 -- 172.21.15.103:0/3413975206 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc65c0a33b0 msgr2=0x7fc65c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:25.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.485+0000 7fc67824f640 1 --2- 172.21.15.103:0/3413975206 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc65c0a33b0 0x7fc65c0a5870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc6680cdd10 tx=0x7fc66806f0d0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.485+0000 7fc67824f640 1 -- 172.21.15.103:0/3413975206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6701055b0 msgr2=0x7fc67019ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.485+0000 7fc67824f640 1 --2- 172.21.15.103:0/3413975206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6701055b0 0x7fc67019ef20 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc66800eaa0 tx=0x7fc66800ef60 comp rx=0 tx=0).stop 2024-09-19T06:58:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.485+0000 7fc67824f640 1 -- 172.21.15.103:0/3413975206 shutdown_connections 2024-09-19T06:58:25.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.485+0000 7fc67824f640 1 --2- 172.21.15.103:0/3413975206 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc65c0a33b0 0x7fc65c0a5870 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.485+0000 7fc67824f640 1 --2- 172.21.15.103:0/3413975206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6701055b0 0x7fc67019ef20 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.485+0000 7fc67824f640 1 --2- 172.21.15.103:0/3413975206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc670100e30 0x7fc67019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.485+0000 7fc67824f640 1 --2- 172.21.15.103:0/3413975206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc670100480 0x7fc67019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:25.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.485+0000 7fc67824f640 1 -- 172.21.15.103:0/3413975206 >> 172.21.15.103:0/3413975206 conn(0x7fc670077210 msgr2=0x7fc670102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:25.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.485+0000 7fc67824f640 1 -- 172.21.15.103:0/3413975206 shutdown_connections 2024-09-19T06:58:25.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:25.485+0000 7fc67824f640 1 -- 172.21.15.103:0/3413975206 wait complete. 2024-09-19T06:58:26.150 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:58:26.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:26 smithi103 ceph-mon[31158]: pgmap v285: 97 pgs: 97 active+clean; 4.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 67 KiB/s wr, 24 op/s 2024-09-19T06:58:26.399 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:58:26.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:26 smithi134 ceph-mon[35356]: pgmap v285: 97 pgs: 97 active+clean; 4.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 67 KiB/s wr, 24 op/s 2024-09-19T06:58:26.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:26 smithi174 ceph-mon[35637]: pgmap v285: 97 pgs: 97 active+clean; 4.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 67 KiB/s wr, 24 op/s 2024-09-19T06:58:26.843 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:58:26.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.805+0000 7f7b749b8640 1 -- 172.21.15.103:0/3337406242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b70106a80 msgr2=0x7f7b70106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:26.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.805+0000 7f7b749b8640 1 --2- 172.21.15.103:0/3337406242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b70106a80 0x7f7b70106e60 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009930 tx=0x7f7b6802a420 comp rx=0 tx=0).stop 2024-09-19T06:58:26.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.805+0000 7f7b749b8640 1 -- 172.21.15.103:0/3337406242 shutdown_connections 2024-09-19T06:58:26.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.805+0000 7f7b749b8640 1 --2- 172.21.15.103:0/3337406242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b70101420 0x7f7b7010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.805+0000 7f7b749b8640 1 --2- 172.21.15.103:0/3337406242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b70100a80 0x7f7b70100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.805+0000 7f7b749b8640 1 --2- 172.21.15.103:0/3337406242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b70106a80 0x7f7b70106e60 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.805+0000 7f7b749b8640 1 -- 172.21.15.103:0/3337406242 >> 172.21.15.103:0/3337406242 conn(0x7f7b700fc7a0 msgr2=0x7f7b700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:26.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.805+0000 7f7b749b8640 1 -- 172.21.15.103:0/3337406242 shutdown_connections 2024-09-19T06:58:26.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.805+0000 7f7b749b8640 1 -- 172.21.15.103:0/3337406242 wait complete. 2024-09-19T06:58:26.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.806+0000 7f7b749b8640 1 Processor -- start 2024-09-19T06:58:26.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.806+0000 7f7b749b8640 1 -- start start 2024-09-19T06:58:26.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.806+0000 7f7b749b8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b70100a80 0x7f7b7019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.806+0000 7f7b749b8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b70101420 0x7f7b7019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.806+0000 7f7b749b8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b70106a80 0x7f7b7019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.806+0000 7f7b749b8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b70110ea0 con 0x7f7b70106a80 2024-09-19T06:58:26.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.806+0000 7f7b749b8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b70110d20 con 0x7f7b70101420 2024-09-19T06:58:26.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.806+0000 7f7b749b8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b70111020 con 0x7f7b70100a80 2024-09-19T06:58:26.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.806+0000 7f7b6ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b70106a80 0x7f7b7019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:26.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.806+0000 7f7b6ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b70106a80 0x7f7b7019ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38744/0 (socket says 172.21.15.103:38744) 2024-09-19T06:58:26.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.806+0000 7f7b6ed9d640 1 -- 172.21.15.103:0/4172772956 learned_addr learned my addr 172.21.15.103:0/4172772956 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:26.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.806+0000 7f7b6e59c640 1 --2- 172.21.15.103:0/4172772956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b70100a80 0x7f7b7019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:26.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.806+0000 7f7b6dd9b640 1 --2- 172.21.15.103:0/4172772956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b70101420 0x7f7b7019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:26.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.807+0000 7f7b6ed9d640 1 -- 172.21.15.103:0/4172772956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b70100a80 msgr2=0x7f7b7019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:26.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.807+0000 7f7b6ed9d640 1 --2- 172.21.15.103:0/4172772956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b70100a80 0x7f7b7019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.807+0000 7f7b6ed9d640 1 -- 172.21.15.103:0/4172772956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b70101420 msgr2=0x7f7b7019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:26.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.807+0000 7f7b6ed9d640 1 --2- 172.21.15.103:0/4172772956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b70101420 0x7f7b7019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.807+0000 7f7b6ed9d640 1 -- 172.21.15.103:0/4172772956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7019f640 con 0x7f7b70106a80 2024-09-19T06:58:26.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.807+0000 7f7b6ed9d640 1 --2- 172.21.15.103:0/4172772956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b70106a80 0x7f7b7019ef60 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c970 tx=0x7f7b6000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:26.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.807+0000 7f7b5f7fe640 1 -- 172.21.15.103:0/4172772956 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60007bb0 con 0x7f7b70106a80 2024-09-19T06:58:26.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.807+0000 7f7b749b8640 1 -- 172.21.15.103:0/4172772956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7019f930 con 0x7f7b70106a80 2024-09-19T06:58:26.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.807+0000 7f7b5f7fe640 1 -- 172.21.15.103:0/4172772956 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b60007d50 con 0x7f7b70106a80 2024-09-19T06:58:26.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.807+0000 7f7b5f7fe640 1 -- 172.21.15.103:0/4172772956 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60018800 con 0x7f7b70106a80 2024-09-19T06:58:26.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.807+0000 7f7b749b8640 1 -- 172.21.15.103:0/4172772956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b701a7210 con 0x7f7b70106a80 2024-09-19T06:58:26.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.809+0000 7f7b5f7fe640 1 -- 172.21.15.103:0/4172772956 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7b6000fa50 con 0x7f7b70106a80 2024-09-19T06:58:26.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.809+0000 7f7b76d91740 1 Processor -- start 2024-09-19T06:58:26.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.810+0000 7f7b76d91740 1 -- start start 2024-09-19T06:58:26.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.810+0000 7f7b76d91740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123bb40 0x55f52123a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.810+0000 7f7b5f7fe640 1 --2- 172.21.15.103:0/4172772956 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7b580a33b0 0x7f7b580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.810+0000 7f7b76d91740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f52123a5d0 0x55f52123aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.810+0000 7f7b76d91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f521245370 0x55f521247760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.810+0000 7f7b76d91740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f5211a02f0 con 0x55f52123bb40 2024-09-19T06:58:26.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.810+0000 7f7b76d91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f521147cf0 con 0x55f521245370 2024-09-19T06:58:26.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.810+0000 7f7b76d91740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f52119c2b0 con 0x55f52123a5d0 2024-09-19T06:58:26.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.810+0000 7f7b6e59c640 1 --2- 172.21.15.103:0/4172772956 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7b580a33b0 0x7f7b580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:26.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.810+0000 7f7b6dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123bb40 0x55f52123a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:26.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.810+0000 7f7b6dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123bb40 0x55f52123a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38750/0 (socket says 172.21.15.103:38750) 2024-09-19T06:58:26.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.810+0000 7f7b6dd9b640 1 -- 172.21.15.103:0/1631304494 learned_addr learned my addr 172.21.15.103:0/1631304494 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:26.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.810+0000 7f7b6dd9b640 1 --2- 172.21.15.103:0/1631304494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f52123a5d0 0x55f52123aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:26.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.810+0000 7f7b5f7fe640 1 -- 172.21.15.103:0/4172772956 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f7b600cfe30 con 0x7f7b70106a80 2024-09-19T06:58:26.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.810+0000 7f7b6ed9d640 1 --2- 172.21.15.103:0/1631304494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f521245370 0x55f521247760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:26.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.811+0000 7f7b6dd9b640 1 -- 172.21.15.103:0/1631304494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f52123a5d0 msgr2=0x55f52123aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:26.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.811+0000 7f7b6dd9b640 1 --2- 172.21.15.103:0/1631304494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f52123a5d0 0x55f52123aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.811+0000 7f7b6dd9b640 1 -- 172.21.15.103:0/1631304494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f521245370 msgr2=0x55f521247760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:26.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.811+0000 7f7b6dd9b640 1 --2- 172.21.15.103:0/1631304494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f521245370 0x55f521247760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.811+0000 7f7b6dd9b640 1 -- 172.21.15.103:0/1631304494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f52112c3a0 con 0x55f52123bb40 2024-09-19T06:58:26.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.811+0000 7f7b6dd9b640 1 --2- 172.21.15.103:0/1631304494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123bb40 0x55f52123a090 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009440 tx=0x7f7b640097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4833d8006686024e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:26.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.811+0000 7f7b6e59c640 1 --2- 172.21.15.103:0/4172772956 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7b580a33b0 0x7f7b580a5870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7b68000c00 tx=0x7f7b68031490 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:26.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.811+0000 7f7b5cff9640 1 -- 172.21.15.103:0/1631304494 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400c0d0 con 0x55f52123bb40 2024-09-19T06:58:26.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.811+0000 7f7b5cff9640 1 -- 172.21.15.103:0/1631304494 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b64016040 con 0x55f52123bb40 2024-09-19T06:58:26.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.811+0000 7f7b5cff9640 1 -- 172.21.15.103:0/1631304494 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6401b3c0 con 0x55f52123bb40 2024-09-19T06:58:26.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.812+0000 7f7b76d91740 1 -- 172.21.15.103:0/1631304494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123bb40 msgr2=0x55f52123a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:26.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.812+0000 7f7b76d91740 1 --2- 172.21.15.103:0/1631304494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123bb40 0x55f52123a090 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009440 tx=0x7f7b640097c0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.812+0000 7f7b76d91740 1 -- 172.21.15.103:0/1631304494 shutdown_connections 2024-09-19T06:58:26.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.812+0000 7f7b76d91740 1 --2- 172.21.15.103:0/1631304494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f521245370 0x55f521247760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.812+0000 7f7b76d91740 1 --2- 172.21.15.103:0/1631304494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f52123a5d0 0x55f52123aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.812+0000 7f7b76d91740 1 --2- 172.21.15.103:0/1631304494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123bb40 0x55f52123a090 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.812+0000 7f7b76d91740 1 -- 172.21.15.103:0/1631304494 >> 172.21.15.103:0/1631304494 conn(0x55f521203310 msgr2=0x55f521236c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:26.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.812+0000 7f7b76d91740 1 -- 172.21.15.103:0/1631304494 shutdown_connections 2024-09-19T06:58:26.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.812+0000 7f7b76d91740 1 -- 172.21.15.103:0/1631304494 wait complete. 2024-09-19T06:58:26.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.812+0000 7f7b76d91740 1 -- start start 2024-09-19T06:58:26.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.812+0000 7f7b76d91740 1 Processor -- start 2024-09-19T06:58:26.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b76d91740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123a5d0 0x55f521244d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b76d91740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f52123bb40 0x55f521241fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b76d91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f521245370 0x55f521242570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b76d91740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f521248010 con 0x55f52123a5d0 2024-09-19T06:58:26.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b76d91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f52115b060 con 0x55f521245370 2024-09-19T06:58:26.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b76d91740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f521248190 con 0x55f52123bb40 2024-09-19T06:58:26.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b76d91740 10 client.0 _ll_register_callbacks cb 0x55f520eae8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:26.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b6e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123a5d0 0x55f521244d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:26.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b6e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123a5d0 0x55f521244d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38760/0 (socket says 172.21.15.103:38760) 2024-09-19T06:58:26.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b6e59c640 1 -- 172.21.15.103:0/2595451185 learned_addr learned my addr 172.21.15.103:0/2595451185 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:26.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b6dd9b640 1 --2- 172.21.15.103:0/2595451185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f52123bb40 0x55f521241fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:26.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b6ed9d640 1 --2- 172.21.15.103:0/2595451185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f521245370 0x55f521242570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:26.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b6e59c640 1 -- 172.21.15.103:0/2595451185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f52123bb40 msgr2=0x55f521241fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:26.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b6e59c640 1 --2- 172.21.15.103:0/2595451185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f52123bb40 0x55f521241fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b6e59c640 1 -- 172.21.15.103:0/2595451185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f521245370 msgr2=0x55f521242570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:26.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b6e59c640 1 --2- 172.21.15.103:0/2595451185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f521245370 0x55f521242570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b6e59c640 1 -- 172.21.15.103:0/2595451185 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f520f113c0 con 0x55f52123a5d0 2024-09-19T06:58:26.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b6ed9d640 1 --2- 172.21.15.103:0/2595451185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f521245370 0x55f521242570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:58:26.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b6e59c640 1 --2- 172.21.15.103:0/2595451185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123a5d0 0x55f521244d20 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f7b68033000 tx=0x7f7b68009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196c625e4fab268c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:26.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:26.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b55ffb640 1 -- 172.21.15.103:0/2595451185 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6804c020 con 0x55f52123a5d0 2024-09-19T06:58:26.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b55ffb640 1 -- 172.21.15.103:0/2595451185 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f7b68003b80 con 0x55f52123a5d0 2024-09-19T06:58:26.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.813+0000 7f7b76d91740 1 -- 172.21.15.103:0/2595451185 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b64016040 con 0x55f52123a5d0 2024-09-19T06:58:26.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.814+0000 7f7b55ffb640 1 -- 172.21.15.103:0/2595451185 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7b68057050 con 0x55f52123a5d0 2024-09-19T06:58:26.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.814+0000 7f7b76d91740 10 client.18636 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:26.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.814+0000 7f7b76d91740 1 -- 172.21.15.103:0/2595451185 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f521253a60 con 0x55f52123a5d0 2024-09-19T06:58:26.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.814+0000 7f7b55ffb640 1 -- 172.21.15.103:0/2595451185 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b64016040 con 0x55f52123a5d0 2024-09-19T06:58:26.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.814+0000 7f7b55ffb640 1 -- 172.21.15.103:0/2595451185 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7b6803b430 con 0x55f52123a5d0 2024-09-19T06:58:26.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.814+0000 7f7b76d91740 10 client.18636 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:26.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.814+0000 7f7b76d91740 10 client.18636 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:58:26.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.814+0000 7f7b76d91740 20 client.18636 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:26.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.814+0000 7f7b76d91740 1 --2- 172.21.15.103:0/2595451185 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55f521254f70 0x55f521257360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.814+0000 7f7b76d91740 4 client.18636 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:58:26.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.814+0000 7f7b76d91740 1 -- 172.21.15.103:0/2595451185 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5211ad860 con 0x55f521254f70 2024-09-19T06:58:26.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.814+0000 7f7b6dd9b640 1 --2- 172.21.15.103:0/2595451185 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55f521254f70 0x55f521257360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:26.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.815+0000 7f7b6dd9b640 1 --2- 172.21.15.103:0/2595451185 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55f521254f70 0x55f521257360 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b8e1e7510ac00eb server_cookie=4ce93bb894fc0252 in_seq=0 out_seq=0 2024-09-19T06:58:26.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.815+0000 7f7b55ffb640 10 client.18636 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:58:26.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.815+0000 7f7b55ffb640 1 -- 172.21.15.103:0/2595451185 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5211ad860 con 0x55f521254f70 2024-09-19T06:58:26.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.815+0000 7f7b55ffb640 10 client.18636 handle_command_reply: tid=0 2024-09-19T06:58:26.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.815+0000 7f7b76d91740 1 client.18636 shutdown 2024-09-19T06:58:26.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.815+0000 7f7b76d91740 1 -- 172.21.15.103:0/2595451185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123a5d0 msgr2=0x55f521244d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:26.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.815+0000 7f7b76d91740 1 --2- 172.21.15.103:0/2595451185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123a5d0 0x55f521244d20 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f7b68033000 tx=0x7f7b68009260 comp rx=0 tx=0).stop 2024-09-19T06:58:26.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.816+0000 7f7b76d91740 1 -- 172.21.15.103:0/2595451185 shutdown_connections 2024-09-19T06:58:26.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.816+0000 7f7b76d91740 1 --2- 172.21.15.103:0/2595451185 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55f521254f70 0x55f521257360 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.816+0000 7f7b76d91740 1 --2- 172.21.15.103:0/2595451185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f521245370 0x55f521242570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.816+0000 7f7b76d91740 1 --2- 172.21.15.103:0/2595451185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f52123bb40 0x55f521241fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.816+0000 7f7b55ffb640 0 client.18636 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:58:26.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.816+0000 7f7b76d91740 1 --2- 172.21.15.103:0/2595451185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123a5d0 0x55f521244d20 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.816+0000 7f7b76d91740 1 -- 172.21.15.103:0/2595451185 >> 172.21.15.103:0/2595451185 conn(0x55f521203310 msgr2=0x55f52123c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:26.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.816+0000 7f7b76d91740 1 -- 172.21.15.103:0/2595451185 shutdown_connections 2024-09-19T06:58:26.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.816+0000 7f7b76d91740 1 -- 172.21.15.103:0/2595451185 wait complete. 2024-09-19T06:58:26.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.816+0000 7f7b76d91740 20 client.18636 trim_cache size 0 max 16384 2024-09-19T06:58:26.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.828+0000 7f7b76d91740 1 Processor -- start 2024-09-19T06:58:26.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.828+0000 7f7b76d91740 1 -- start start 2024-09-19T06:58:26.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.828+0000 7f7b76d91740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123a5d0 0x55f52123d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.828+0000 7f7b76d91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f52123bb40 0x55f52123d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.829+0000 7f7b76d91740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f521245370 0x55f52125cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.829+0000 7f7b76d91740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701afaf0 con 0x55f52123a5d0 2024-09-19T06:58:26.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.829+0000 7f7b76d91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701b1630 con 0x55f52123bb40 2024-09-19T06:58:26.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.829+0000 7f7b76d91740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701b01c0 con 0x55f521245370 2024-09-19T06:58:26.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.829+0000 7f7b6e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123a5d0 0x55f52123d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:26.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.829+0000 7f7b6e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123a5d0 0x55f52123d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38764/0 (socket says 172.21.15.103:38764) 2024-09-19T06:58:26.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.829+0000 7f7b6e59c640 1 -- 172.21.15.103:0/3423450870 learned_addr learned my addr 172.21.15.103:0/3423450870 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:26.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.829+0000 7f7b6e59c640 1 -- 172.21.15.103:0/3423450870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f521245370 msgr2=0x55f52125cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:26.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.829+0000 7f7b6e59c640 1 --2- 172.21.15.103:0/3423450870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f521245370 0x55f52125cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.829+0000 7f7b6e59c640 1 -- 172.21.15.103:0/3423450870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f52123bb40 msgr2=0x55f52123d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:26.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.829+0000 7f7b6e59c640 1 --2- 172.21.15.103:0/3423450870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f52123bb40 0x55f52123d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.829+0000 7f7b6e59c640 1 -- 172.21.15.103:0/3423450870 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f52123dba0 con 0x55f52123a5d0 2024-09-19T06:58:26.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.829+0000 7f7b6e59c640 1 --2- 172.21.15.103:0/3423450870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123a5d0 0x55f52123d120 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f7b680096c0 tx=0x7f7b6802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203cd244f12e79aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:26.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.829+0000 7f7b57fff640 1 -- 172.21.15.103:0/3423450870 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6804c070 con 0x55f52123a5d0 2024-09-19T06:58:26.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.829+0000 7f7b57fff640 1 -- 172.21.15.103:0/3423450870 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b680315b0 con 0x55f52123a5d0 2024-09-19T06:58:26.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.829+0000 7f7b57fff640 1 -- 172.21.15.103:0/3423450870 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68035050 con 0x55f52123a5d0 2024-09-19T06:58:26.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.830+0000 7f7b76d91740 1 -- 172.21.15.103:0/3423450870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123a5d0 msgr2=0x55f52123d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:26.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.830+0000 7f7b76d91740 1 --2- 172.21.15.103:0/3423450870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123a5d0 0x55f52123d120 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f7b680096c0 tx=0x7f7b6802a420 comp rx=0 tx=0).stop 2024-09-19T06:58:26.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.830+0000 7f7b76d91740 1 -- 172.21.15.103:0/3423450870 shutdown_connections 2024-09-19T06:58:26.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.830+0000 7f7b76d91740 1 --2- 172.21.15.103:0/3423450870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f521245370 0x55f52125cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.830+0000 7f7b76d91740 1 --2- 172.21.15.103:0/3423450870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f52123bb40 0x55f52123d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.830+0000 7f7b76d91740 1 --2- 172.21.15.103:0/3423450870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f52123a5d0 0x55f52123d120 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.830+0000 7f7b76d91740 1 -- 172.21.15.103:0/3423450870 >> 172.21.15.103:0/3423450870 conn(0x55f521203310 msgr2=0x55f521235b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:26.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.830+0000 7f7b76d91740 1 -- 172.21.15.103:0/3423450870 shutdown_connections 2024-09-19T06:58:26.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.830+0000 7f7b76d91740 1 -- 172.21.15.103:0/3423450870 wait complete. 2024-09-19T06:58:26.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.830+0000 7f7b76d91740 1 -- start start 2024-09-19T06:58:26.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.830+0000 7f7b76d91740 1 Processor -- start 2024-09-19T06:58:26.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b76d91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f52123a5d0 0x55f521249d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b76d91740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f52123bb40 0x55f52124a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b76d91740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f521245370 0x55f5212b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b76d91740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701b0f60 con 0x55f521245370 2024-09-19T06:58:26.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b76d91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701b0890 con 0x55f52123a5d0 2024-09-19T06:58:26.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b76d91740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701adaf0 con 0x55f52123bb40 2024-09-19T06:58:26.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b76d91740 10 client.0 _ll_register_callbacks cb 0x7f7b6803f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:26.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b6ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f521245370 0x55f5212b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:26.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b6ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f521245370 0x55f5212b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38768/0 (socket says 172.21.15.103:38768) 2024-09-19T06:58:26.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b6ed9d640 1 -- 172.21.15.103:0/3946758134 learned_addr learned my addr 172.21.15.103:0/3946758134 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:26.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b6e59c640 1 --2- 172.21.15.103:0/3946758134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f52123a5d0 0x55f521249d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:26.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b6dd9b640 1 --2- 172.21.15.103:0/3946758134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f52123bb40 0x55f52124a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:26.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b6ed9d640 1 -- 172.21.15.103:0/3946758134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f52123bb40 msgr2=0x55f52124a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:26.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b6ed9d640 1 --2- 172.21.15.103:0/3946758134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f52123bb40 0x55f52124a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b6ed9d640 1 -- 172.21.15.103:0/3946758134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f52123a5d0 msgr2=0x55f521249d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:26.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b6ed9d640 1 --2- 172.21.15.103:0/3946758134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f52123a5d0 0x55f521249d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b6ed9d640 1 -- 172.21.15.103:0/3946758134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5212b83f0 con 0x55f521245370 2024-09-19T06:58:26.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b6e59c640 1 --2- 172.21.15.103:0/3946758134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f52123a5d0 0x55f521249d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:26.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b6ed9d640 1 --2- 172.21.15.103:0/3946758134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f521245370 0x55f5212b7be0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f7b6009bdf0 tx=0x7f7b6009bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0eaced2cbb884a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:26.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:26.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b55ffb640 1 -- 172.21.15.103:0/3946758134 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6009c0f0 con 0x55f521245370 2024-09-19T06:58:26.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b55ffb640 1 -- 172.21.15.103:0/3946758134 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f7b60098760 con 0x55f521245370 2024-09-19T06:58:26.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b76d91740 1 -- 172.21.15.103:0/3946758134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b680315b0 con 0x55f521245370 2024-09-19T06:58:26.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b55ffb640 1 -- 172.21.15.103:0/3946758134 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7b600a04a0 con 0x55f521245370 2024-09-19T06:58:26.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b76d91740 10 client.18642 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:26.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b76d91740 1 -- 172.21.15.103:0/3946758134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5212c0740 con 0x55f521245370 2024-09-19T06:58:26.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.831+0000 7f7b55ffb640 1 -- 172.21.15.103:0/3946758134 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b680315b0 con 0x55f521245370 2024-09-19T06:58:26.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.832+0000 7f7b55ffb640 1 -- 172.21.15.103:0/3946758134 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7b600b6050 con 0x55f521245370 2024-09-19T06:58:26.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.832+0000 7f7b76d91740 10 client.18642 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:26.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.832+0000 7f7b76d91740 10 client.18642 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:58:26.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.832+0000 7f7b76d91740 20 client.18642 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:26.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.832+0000 7f7b76d91740 1 --2- 172.21.15.103:0/3946758134 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55f5212c0e60 0x55f5212c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:26.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.832+0000 7f7b76d91740 4 client.18642 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:58:26.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.832+0000 7f7b76d91740 1 -- 172.21.15.103:0/3946758134 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5211a7d70 con 0x55f5212c0e60 2024-09-19T06:58:26.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.832+0000 7f7b6e59c640 1 --2- 172.21.15.103:0/3946758134 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55f5212c0e60 0x55f5212c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:26.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.832+0000 7f7b6e59c640 1 --2- 172.21.15.103:0/3946758134 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55f5212c0e60 0x55f5212c3250 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28df4f1996079718 server_cookie=8243b92dcf5820d0 in_seq=0 out_seq=0 2024-09-19T06:58:26.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.832+0000 7f7b55ffb640 10 client.18642 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:58:26.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b55ffb640 1 -- 172.21.15.103:0/3946758134 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5211a7d70 con 0x55f5212c0e60 2024-09-19T06:58:26.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b55ffb640 10 client.18642 handle_command_reply: tid=0 2024-09-19T06:58:26.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b76d91740 1 client.18642 shutdown 2024-09-19T06:58:26.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b76d91740 1 -- 172.21.15.103:0/3946758134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f521245370 msgr2=0x55f5212b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:26.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b76d91740 1 --2- 172.21.15.103:0/3946758134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f521245370 0x55f5212b7be0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f7b6009bdf0 tx=0x7f7b6009bd20 comp rx=0 tx=0).stop 2024-09-19T06:58:26.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b76d91740 1 -- 172.21.15.103:0/3946758134 shutdown_connections 2024-09-19T06:58:26.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b76d91740 1 --2- 172.21.15.103:0/3946758134 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55f5212c0e60 0x55f5212c3250 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b76d91740 1 --2- 172.21.15.103:0/3946758134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f521245370 0x55f5212b7be0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b76d91740 1 --2- 172.21.15.103:0/3946758134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f52123bb40 0x55f52124a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b55ffb640 0 client.18642 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:58:26.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b76d91740 1 --2- 172.21.15.103:0/3946758134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f52123a5d0 0x55f521249d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b76d91740 1 -- 172.21.15.103:0/3946758134 >> 172.21.15.103:0/3946758134 conn(0x55f521203310 msgr2=0x55f521244050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:26.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b76d91740 1 -- 172.21.15.103:0/3946758134 shutdown_connections 2024-09-19T06:58:26.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b76d91740 1 -- 172.21.15.103:0/3946758134 wait complete. 2024-09-19T06:58:26.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b76d91740 20 client.18642 trim_cache size 0 max 16384 2024-09-19T06:58:26.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b749b8640 1 -- 172.21.15.103:0/4172772956 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7b580a33b0 msgr2=0x7f7b580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:26.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b749b8640 1 --2- 172.21.15.103:0/4172772956 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7b580a33b0 0x7f7b580a5870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7b68000c00 tx=0x7f7b68031490 comp rx=0 tx=0).stop 2024-09-19T06:58:26.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b749b8640 1 -- 172.21.15.103:0/4172772956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b70106a80 msgr2=0x7f7b7019ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:26.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.833+0000 7f7b749b8640 1 --2- 172.21.15.103:0/4172772956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b70106a80 0x7f7b7019ef60 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c970 tx=0x7f7b6000ce30 comp rx=0 tx=0).stop 2024-09-19T06:58:26.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.834+0000 7f7b749b8640 1 -- 172.21.15.103:0/4172772956 shutdown_connections 2024-09-19T06:58:26.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.834+0000 7f7b749b8640 1 --2- 172.21.15.103:0/4172772956 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7b580a33b0 0x7f7b580a5870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.834+0000 7f7b749b8640 1 --2- 172.21.15.103:0/4172772956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b70106a80 0x7f7b7019ef60 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.834+0000 7f7b749b8640 1 --2- 172.21.15.103:0/4172772956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b70101420 0x7f7b7019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.834+0000 7f7b749b8640 1 --2- 172.21.15.103:0/4172772956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b70100a80 0x7f7b7019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:26.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.834+0000 7f7b749b8640 1 -- 172.21.15.103:0/4172772956 >> 172.21.15.103:0/4172772956 conn(0x7f7b700fc7a0 msgr2=0x7f7b7006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:26.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.834+0000 7f7b749b8640 1 -- 172.21.15.103:0/4172772956 shutdown_connections 2024-09-19T06:58:26.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:26.834+0000 7f7b749b8640 1 -- 172.21.15.103:0/4172772956 wait complete. 2024-09-19T06:58:27.484 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:58:27.731 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:58:28.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.248+0000 7fc8f44ad640 1 -- 172.21.15.103:0/3152120138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8ec102790 msgr2=0x7fc8ec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:28.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.248+0000 7fc8f44ad640 1 --2- 172.21.15.103:0/3152120138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8ec102790 0x7fc8ec1067e0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009900 tx=0x7fc8e002a410 comp rx=0 tx=0).stop 2024-09-19T06:58:28.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.248+0000 7fc8f44ad640 1 -- 172.21.15.103:0/3152120138 shutdown_connections 2024-09-19T06:58:28.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.248+0000 7fc8f44ad640 1 --2- 172.21.15.103:0/3152120138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8ec106f10 0x7fc8ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.248+0000 7fc8f44ad640 1 --2- 172.21.15.103:0/3152120138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8ec102790 0x7fc8ec1067e0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.248+0000 7fc8f44ad640 1 --2- 172.21.15.103:0/3152120138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec101de0 0x7fc8ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.248+0000 7fc8f44ad640 1 -- 172.21.15.103:0/3152120138 >> 172.21.15.103:0/3152120138 conn(0x7fc8ec0fd8c0 msgr2=0x7fc8ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:28.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.248+0000 7fc8f44ad640 1 -- 172.21.15.103:0/3152120138 shutdown_connections 2024-09-19T06:58:28.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.248+0000 7fc8f44ad640 1 -- 172.21.15.103:0/3152120138 wait complete. 2024-09-19T06:58:28.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.248+0000 7fc8f44ad640 1 Processor -- start 2024-09-19T06:58:28.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.248+0000 7fc8f44ad640 1 -- start start 2024-09-19T06:58:28.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.248+0000 7fc8f44ad640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec101de0 0x7fc8ec0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.248+0000 7fc8f44ad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8ec102790 0x7fc8ec074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f44ad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8ec106f10 0x7fc8ec074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f44ad640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec111f90 con 0x7fc8ec102790 2024-09-19T06:58:28.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f44ad640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec111e10 con 0x7fc8ec106f10 2024-09-19T06:58:28.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f44ad640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec112110 con 0x7fc8ec101de0 2024-09-19T06:58:28.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f1a48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8ec102790 0x7fc8ec074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:28.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f1a48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8ec102790 0x7fc8ec074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53588/0 (socket says 172.21.15.103:53588) 2024-09-19T06:58:28.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f1a48640 1 -- 172.21.15.103:0/2061008689 learned_addr learned my addr 172.21.15.103:0/2061008689 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:28.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f2249640 1 --2- 172.21.15.103:0/2061008689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec101de0 0x7fc8ec0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:28.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f2a4a640 1 --2- 172.21.15.103:0/2061008689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8ec106f10 0x7fc8ec074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:28.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f1a48640 1 -- 172.21.15.103:0/2061008689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec101de0 msgr2=0x7fc8ec0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:28.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f1a48640 1 --2- 172.21.15.103:0/2061008689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec101de0 0x7fc8ec0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f1a48640 1 -- 172.21.15.103:0/2061008689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8ec106f10 msgr2=0x7fc8ec074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:28.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f1a48640 1 --2- 172.21.15.103:0/2061008689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8ec106f10 0x7fc8ec074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f1a48640 1 -- 172.21.15.103:0/2061008689 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec1a6940 con 0x7fc8ec102790 2024-09-19T06:58:28.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f2a4a640 1 --2- 172.21.15.103:0/2061008689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8ec106f10 0x7fc8ec074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:28.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f1a48640 1 --2- 172.21.15.103:0/2061008689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8ec102790 0x7fc8ec074a30 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0005ab0 tx=0x7fc8e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:28.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8df7fe640 1 -- 172.21.15.103:0/2061008689 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e003f030 con 0x7fc8ec102790 2024-09-19T06:58:28.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8df7fe640 1 -- 172.21.15.103:0/2061008689 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e0003ce0 con 0x7fc8ec102790 2024-09-19T06:58:28.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f44ad640 1 -- 172.21.15.103:0/2061008689 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ec1a6bd0 con 0x7fc8ec102790 2024-09-19T06:58:28.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8df7fe640 1 -- 172.21.15.103:0/2061008689 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0035070 con 0x7fc8ec102790 2024-09-19T06:58:28.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.249+0000 7fc8f44ad640 1 -- 172.21.15.103:0/2061008689 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ec1a70b0 con 0x7fc8ec102790 2024-09-19T06:58:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.252+0000 7fc8df7fe640 1 -- 172.21.15.103:0/2061008689 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc8e0042030 con 0x7fc8ec102790 2024-09-19T06:58:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.252+0000 7fc8f6886740 1 Processor -- start 2024-09-19T06:58:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.252+0000 7fc8f6886740 1 -- start start 2024-09-19T06:58:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.252+0000 7fc8df7fe640 1 --2- 172.21.15.103:0/2061008689 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc8d80a33b0 0x7fc8d80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.252+0000 7fc8f6886740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf631fb40 0x55aaf631e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.252+0000 7fc8f6886740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631e5d0 0x55aaf631ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.252+0000 7fc8f6886740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf6329370 0x55aaf632b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.252+0000 7fc8f6886740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aaf62842f0 con 0x55aaf631e5d0 2024-09-19T06:58:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8f6886740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55aaf622bcf0 con 0x55aaf631fb40 2024-09-19T06:58:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8f6886740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aaf62802b0 con 0x55aaf6329370 2024-09-19T06:58:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8f2a4a640 1 --2- 172.21.15.103:0/2061008689 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc8d80a33b0 0x7fc8d80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8f1a48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631e5d0 0x55aaf631ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8f1a48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631e5d0 0x55aaf631ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53596/0 (socket says 172.21.15.103:53596) 2024-09-19T06:58:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8df7fe640 1 -- 172.21.15.103:0/2061008689 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7fc8e00f02c0 con 0x7fc8ec102790 2024-09-19T06:58:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8f1a48640 1 -- 172.21.15.103:0/1717214277 learned_addr learned my addr 172.21.15.103:0/1717214277 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8f2249640 1 --2- 172.21.15.103:0/1717214277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf631fb40 0x55aaf631e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8f1a48640 1 -- 172.21.15.103:0/1717214277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf6329370 msgr2=0x55aaf632b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8f1a48640 1 --2- 172.21.15.103:0/1717214277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf6329370 0x55aaf632b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8f1a48640 1 -- 172.21.15.103:0/1717214277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf631fb40 msgr2=0x55aaf631e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:28.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8f1a48640 1 --2- 172.21.15.103:0/1717214277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf631fb40 0x55aaf631e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8f1a48640 1 -- 172.21.15.103:0/1717214277 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf62103a0 con 0x55aaf631e5d0 2024-09-19T06:58:28.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8f2249640 1 --2- 172.21.15.103:0/1717214277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf631fb40 0x55aaf631e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:58:28.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8f2a4a640 1 --2- 172.21.15.103:0/2061008689 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc8d80a33b0 0x7fc8d80a5870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4005fd0 tx=0x7fc8e400a5c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:28.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8f1a48640 1 --2- 172.21.15.103:0/1717214277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631e5d0 0x55aaf631ea10 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00bc510 tx=0x7fc8e004b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c6493ab0cd70b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:28.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8dcff9640 1 -- 172.21.15.103:0/1717214277 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0049070 con 0x55aaf631e5d0 2024-09-19T06:58:28.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8dcff9640 1 -- 172.21.15.103:0/1717214277 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e00bcc80 con 0x55aaf631e5d0 2024-09-19T06:58:28.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.253+0000 7fc8dcff9640 1 -- 172.21.15.103:0/1717214277 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00c51f0 con 0x55aaf631e5d0 2024-09-19T06:58:28.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.254+0000 7fc8f6886740 1 -- 172.21.15.103:0/1717214277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631e5d0 msgr2=0x55aaf631ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:28.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.254+0000 7fc8f6886740 1 --2- 172.21.15.103:0/1717214277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631e5d0 0x55aaf631ea10 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00bc510 tx=0x7fc8e004b000 comp rx=0 tx=0).stop 2024-09-19T06:58:28.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.254+0000 7fc8f6886740 1 -- 172.21.15.103:0/1717214277 shutdown_connections 2024-09-19T06:58:28.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.254+0000 7fc8f6886740 1 --2- 172.21.15.103:0/1717214277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf6329370 0x55aaf632b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.254+0000 7fc8f6886740 1 --2- 172.21.15.103:0/1717214277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631e5d0 0x55aaf631ea10 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.254+0000 7fc8f6886740 1 --2- 172.21.15.103:0/1717214277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf631fb40 0x55aaf631e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.254+0000 7fc8f6886740 1 -- 172.21.15.103:0/1717214277 >> 172.21.15.103:0/1717214277 conn(0x55aaf62e7310 msgr2=0x55aaf631ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:28.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.254+0000 7fc8f6886740 1 -- 172.21.15.103:0/1717214277 shutdown_connections 2024-09-19T06:58:28.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.254+0000 7fc8f6886740 1 -- 172.21.15.103:0/1717214277 wait complete. 2024-09-19T06:58:28.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f6886740 1 -- start start 2024-09-19T06:58:28.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f6886740 1 Processor -- start 2024-09-19T06:58:28.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f6886740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf631e5d0 0x55aaf6328dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f6886740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6326070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f6886740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf6329370 0x55aaf6326800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f6886740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aaf632be30 con 0x55aaf631fb40 2024-09-19T06:58:28.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f6886740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55aaf623f060 con 0x55aaf6329370 2024-09-19T06:58:28.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f6886740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aaf632bfb0 con 0x55aaf631e5d0 2024-09-19T06:58:28.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f6886740 10 client.0 _ll_register_callbacks cb 0x55aaf5f928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:28.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f1a48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6326070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:28.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f1a48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6326070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53602/0 (socket says 172.21.15.103:53602) 2024-09-19T06:58:28.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f1a48640 1 -- 172.21.15.103:0/1725983742 learned_addr learned my addr 172.21.15.103:0/1725983742 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:28.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f2a4a640 1 --2- 172.21.15.103:0/1725983742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf6329370 0x55aaf6326800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:28.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f2249640 1 --2- 172.21.15.103:0/1725983742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf631e5d0 0x55aaf6328dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:28.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f1a48640 1 -- 172.21.15.103:0/1725983742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf631e5d0 msgr2=0x55aaf6328dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:28.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f1a48640 1 --2- 172.21.15.103:0/1725983742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf631e5d0 0x55aaf6328dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f1a48640 1 -- 172.21.15.103:0/1725983742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf6329370 msgr2=0x55aaf6326800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:28.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f1a48640 1 --2- 172.21.15.103:0/1725983742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf6329370 0x55aaf6326800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f1a48640 1 -- 172.21.15.103:0/1725983742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf5ff53c0 con 0x55aaf631fb40 2024-09-19T06:58:28.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.255+0000 7fc8f2a4a640 1 --2- 172.21.15.103:0/1725983742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf6329370 0x55aaf6326800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:28.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.256+0000 7fc8f1a48640 1 --2- 172.21.15.103:0/1725983742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6326070 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0006010 tx=0x7fc8e00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61afb935289354c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:28.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.256+0000 7fc8d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:28.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.256+0000 7fc8d5ffb640 1 -- 172.21.15.103:0/1725983742 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0049070 con 0x55aaf631fb40 2024-09-19T06:58:28.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.256+0000 7fc8d5ffb640 1 -- 172.21.15.103:0/1725983742 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7fc8e00b8bf0 con 0x55aaf631fb40 2024-09-19T06:58:28.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.256+0000 7fc8f6886740 1 -- 172.21.15.103:0/1725983742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8e00bcc80 con 0x55aaf631fb40 2024-09-19T06:58:28.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.256+0000 7fc8d5ffb640 1 -- 172.21.15.103:0/1725983742 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8e00d5050 con 0x55aaf631fb40 2024-09-19T06:58:28.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.256+0000 7fc8f6886740 10 client.18654 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:28.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.256+0000 7fc8f6886740 1 -- 172.21.15.103:0/1725983742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aaf632be30 con 0x55aaf631fb40 2024-09-19T06:58:28.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.256+0000 7fc8d5ffb640 1 -- 172.21.15.103:0/1725983742 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e00bcc80 con 0x55aaf631fb40 2024-09-19T06:58:28.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.256+0000 7fc8d5ffb640 1 -- 172.21.15.103:0/1725983742 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc8e00bc7e0 con 0x55aaf631fb40 2024-09-19T06:58:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.256+0000 7fc8f6886740 10 client.18654 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.256+0000 7fc8f6886740 10 client.18654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:58:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.256+0000 7fc8f6886740 20 client.18654 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.256+0000 7fc8f6886740 1 --2- 172.21.15.103:0/1725983742 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aaf6338a00 0x55aaf633adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.256+0000 7fc8f6886740 4 client.18654 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:58:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.256+0000 7fc8f6886740 1 -- 172.21.15.103:0/1725983742 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaf6291860 con 0x55aaf6338a00 2024-09-19T06:58:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.257+0000 7fc8f2249640 1 --2- 172.21.15.103:0/1725983742 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aaf6338a00 0x55aaf633adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.257+0000 7fc8f2249640 1 --2- 172.21.15.103:0/1725983742 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aaf6338a00 0x55aaf633adf0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=441a3ed41dbdf1bc server_cookie=f7b79e7e91325c4c in_seq=0 out_seq=0 2024-09-19T06:58:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.257+0000 7fc8d5ffb640 10 client.18654 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.258+0000 7fc8d5ffb640 1 -- 172.21.15.103:0/1725983742 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aaf6291860 con 0x55aaf6338a00 2024-09-19T06:58:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.258+0000 7fc8d5ffb640 10 client.18654 handle_command_reply: tid=0 2024-09-19T06:58:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.258+0000 7fc8f6886740 1 client.18654 shutdown 2024-09-19T06:58:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.258+0000 7fc8f6886740 1 -- 172.21.15.103:0/1725983742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 msgr2=0x55aaf6326070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:28.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.258+0000 7fc8f6886740 1 --2- 172.21.15.103:0/1725983742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6326070 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0006010 tx=0x7fc8e00058b0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.258+0000 7fc8f6886740 1 -- 172.21.15.103:0/1725983742 shutdown_connections 2024-09-19T06:58:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.258+0000 7fc8f6886740 1 --2- 172.21.15.103:0/1725983742 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aaf6338a00 0x55aaf633adf0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.258+0000 7fc8f6886740 1 --2- 172.21.15.103:0/1725983742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf6329370 0x55aaf6326800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.258+0000 7fc8d5ffb640 0 client.18654 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.258+0000 7fc8f6886740 1 --2- 172.21.15.103:0/1725983742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6326070 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.258+0000 7fc8f6886740 1 --2- 172.21.15.103:0/1725983742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf631e5d0 0x55aaf6328dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.258+0000 7fc8f6886740 1 -- 172.21.15.103:0/1725983742 >> 172.21.15.103:0/1725983742 conn(0x55aaf62e7310 msgr2=0x55aaf6320930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.258+0000 7fc8f6886740 1 -- 172.21.15.103:0/1725983742 shutdown_connections 2024-09-19T06:58:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.258+0000 7fc8f6886740 1 -- 172.21.15.103:0/1725983742 wait complete. 2024-09-19T06:58:28.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.258+0000 7fc8f6886740 20 client.18654 trim_cache size 0 max 16384 2024-09-19T06:58:28.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f6886740 1 Processor -- start 2024-09-19T06:58:28.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f6886740 1 -- start start 2024-09-19T06:58:28.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f6886740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf631e5d0 0x55aaf6321120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f6886740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6321660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f6886740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf6329370 0x55aaf6340580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f6886740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1afa30 con 0x55aaf631fb40 2024-09-19T06:58:28.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f6886740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1b1570 con 0x55aaf6329370 2024-09-19T06:58:28.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f6886740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1b0100 con 0x55aaf631e5d0 2024-09-19T06:58:28.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f1a48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6321660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:28.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f1a48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6321660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53614/0 (socket says 172.21.15.103:53614) 2024-09-19T06:58:28.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f1a48640 1 -- 172.21.15.103:0/4125844123 learned_addr learned my addr 172.21.15.103:0/4125844123 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:28.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f1a48640 1 -- 172.21.15.103:0/4125844123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf631e5d0 msgr2=0x55aaf6321120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:28.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f1a48640 1 --2- 172.21.15.103:0/4125844123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf631e5d0 0x55aaf6321120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f1a48640 1 -- 172.21.15.103:0/4125844123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf6329370 msgr2=0x55aaf6340580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:28.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f1a48640 1 --2- 172.21.15.103:0/4125844123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf6329370 0x55aaf6340580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f1a48640 1 -- 172.21.15.103:0/4125844123 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf6326b70 con 0x55aaf631fb40 2024-09-19T06:58:28.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.271+0000 7fc8f1a48640 1 --2- 172.21.15.103:0/4125844123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6321660 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00ba830 tx=0x7fc8e00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b2a20f5a938c7f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:28.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.272+0000 7fc8d7fff640 1 -- 172.21.15.103:0/4125844123 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00d9030 con 0x55aaf631fb40 2024-09-19T06:58:28.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.272+0000 7fc8d7fff640 1 -- 172.21.15.103:0/4125844123 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e00bcc80 con 0x55aaf631fb40 2024-09-19T06:58:28.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.272+0000 7fc8d7fff640 1 -- 172.21.15.103:0/4125844123 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0049070 con 0x55aaf631fb40 2024-09-19T06:58:28.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.272+0000 7fc8f6886740 1 -- 172.21.15.103:0/4125844123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 msgr2=0x55aaf6321660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:28.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.272+0000 7fc8f6886740 1 --2- 172.21.15.103:0/4125844123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6321660 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00ba830 tx=0x7fc8e00045e0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.272+0000 7fc8f6886740 1 -- 172.21.15.103:0/4125844123 shutdown_connections 2024-09-19T06:58:28.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.272+0000 7fc8f6886740 1 --2- 172.21.15.103:0/4125844123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf6329370 0x55aaf6340580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.272+0000 7fc8f6886740 1 --2- 172.21.15.103:0/4125844123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6321660 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.272+0000 7fc8f6886740 1 --2- 172.21.15.103:0/4125844123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf631e5d0 0x55aaf6321120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.272+0000 7fc8f6886740 1 -- 172.21.15.103:0/4125844123 >> 172.21.15.103:0/4125844123 conn(0x55aaf62e7310 msgr2=0x55aaf6325030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:28.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.272+0000 7fc8f6886740 1 -- 172.21.15.103:0/4125844123 shutdown_connections 2024-09-19T06:58:28.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.272+0000 7fc8f6886740 1 -- 172.21.15.103:0/4125844123 wait complete. 2024-09-19T06:58:28.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f6886740 1 -- start start 2024-09-19T06:58:28.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f6886740 1 Processor -- start 2024-09-19T06:58:28.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f6886740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf631e5d0 0x55aaf6331200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f6886740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6331740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f6886740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf6329370 0x55aaf63226a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f6886740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1b0ea0 con 0x55aaf631fb40 2024-09-19T06:58:28.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f6886740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1b07d0 con 0x55aaf6329370 2024-09-19T06:58:28.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f6886740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1ada30 con 0x55aaf631e5d0 2024-09-19T06:58:28.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f6886740 10 client.0 _ll_register_callbacks cb 0x7fc8e8009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:28.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f1a48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6331740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:28.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f1a48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6331740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53622/0 (socket says 172.21.15.103:53622) 2024-09-19T06:58:28.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f1a48640 1 -- 172.21.15.103:0/3380656287 learned_addr learned my addr 172.21.15.103:0/3380656287 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:28.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f2a4a640 1 --2- 172.21.15.103:0/3380656287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf6329370 0x55aaf63226a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:28.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f2249640 1 --2- 172.21.15.103:0/3380656287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf631e5d0 0x55aaf6331200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:28.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f1a48640 1 -- 172.21.15.103:0/3380656287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf631e5d0 msgr2=0x55aaf6331200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:28.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f1a48640 1 --2- 172.21.15.103:0/3380656287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf631e5d0 0x55aaf6331200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f1a48640 1 -- 172.21.15.103:0/3380656287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf6329370 msgr2=0x55aaf63226a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:28.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f1a48640 1 --2- 172.21.15.103:0/3380656287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf6329370 0x55aaf63226a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f1a48640 1 -- 172.21.15.103:0/3380656287 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf6322be0 con 0x55aaf631fb40 2024-09-19T06:58:28.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f2249640 1 --2- 172.21.15.103:0/3380656287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf631e5d0 0x55aaf6331200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:58:28.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f2a4a640 1 --2- 172.21.15.103:0/3380656287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf6329370 0x55aaf63226a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:28.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8f1a48640 1 --2- 172.21.15.103:0/3380656287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6331740 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0003fd0 tx=0x7fc8e002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1760ed345b66c84f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:28.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.273+0000 7fc8d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:28.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.274+0000 7fc8d5ffb640 1 -- 172.21.15.103:0/3380656287 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00d9070 con 0x55aaf631fb40 2024-09-19T06:58:28.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.274+0000 7fc8d5ffb640 1 -- 172.21.15.103:0/3380656287 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7fc8e00c4d60 con 0x55aaf631fb40 2024-09-19T06:58:28.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.274+0000 7fc8f6886740 1 -- 172.21.15.103:0/3380656287 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8e00bcc80 con 0x55aaf631fb40 2024-09-19T06:58:28.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.274+0000 7fc8d5ffb640 1 -- 172.21.15.103:0/3380656287 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8e0049050 con 0x55aaf631fb40 2024-09-19T06:58:28.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.274+0000 7fc8f6886740 10 client.18660 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:28.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.274+0000 7fc8f6886740 1 -- 172.21.15.103:0/3380656287 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aaf6322be0 con 0x55aaf631fb40 2024-09-19T06:58:28.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.274+0000 7fc8d5ffb640 1 -- 172.21.15.103:0/3380656287 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e00bcc80 con 0x55aaf631fb40 2024-09-19T06:58:28.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.274+0000 7fc8d5ffb640 1 -- 172.21.15.103:0/3380656287 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc8e0087200 con 0x55aaf631fb40 2024-09-19T06:58:28.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.274+0000 7fc8f6886740 10 client.18660 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.274+0000 7fc8f6886740 10 client.18660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:58:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.274+0000 7fc8f6886740 20 client.18660 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.274+0000 7fc8f6886740 1 --2- 172.21.15.103:0/3380656287 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aaf63a2f60 0x55aaf63a5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.274+0000 7fc8f6886740 4 client.18660 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:58:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.274+0000 7fc8f6886740 1 -- 172.21.15.103:0/3380656287 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aaf628bd70 con 0x55aaf63a2f60 2024-09-19T06:58:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.274+0000 7fc8f2249640 1 --2- 172.21.15.103:0/3380656287 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aaf63a2f60 0x55aaf63a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8d5ffb640 10 client.18660 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8f2249640 1 --2- 172.21.15.103:0/3380656287 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aaf63a2f60 0x55aaf63a5380 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db7d47b60dec4e3d server_cookie=e4af3bc8c6429e2a in_seq=0 out_seq=0 2024-09-19T06:58:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8d5ffb640 1 -- 172.21.15.103:0/3380656287 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aaf628bd70 con 0x55aaf63a2f60 2024-09-19T06:58:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8d5ffb640 10 client.18660 handle_command_reply: tid=0 2024-09-19T06:58:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8f6886740 1 client.18660 shutdown 2024-09-19T06:58:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8f6886740 1 -- 172.21.15.103:0/3380656287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 msgr2=0x55aaf6331740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8f6886740 1 --2- 172.21.15.103:0/3380656287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6331740 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0003fd0 tx=0x7fc8e002a410 comp rx=0 tx=0).stop 2024-09-19T06:58:28.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8f6886740 1 -- 172.21.15.103:0/3380656287 shutdown_connections 2024-09-19T06:58:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8f6886740 1 --2- 172.21.15.103:0/3380656287 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55aaf63a2f60 0x55aaf63a5380 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8f6886740 1 --2- 172.21.15.103:0/3380656287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aaf6329370 0x55aaf63226a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8f6886740 1 --2- 172.21.15.103:0/3380656287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaf631fb40 0x55aaf6331740 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8f6886740 1 --2- 172.21.15.103:0/3380656287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf631e5d0 0x55aaf6331200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8f6886740 1 -- 172.21.15.103:0/3380656287 >> 172.21.15.103:0/3380656287 conn(0x55aaf62e7310 msgr2=0x55aaf6324810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8d5ffb640 0 client.18660 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8f6886740 1 -- 172.21.15.103:0/3380656287 shutdown_connections 2024-09-19T06:58:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8f6886740 1 -- 172.21.15.103:0/3380656287 wait complete. 2024-09-19T06:58:28.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.275+0000 7fc8f6886740 20 client.18660 trim_cache size 0 max 16384 2024-09-19T06:58:28.283 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:58:28.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.276+0000 7fc8f44ad640 1 -- 172.21.15.103:0/2061008689 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc8d80a33b0 msgr2=0x7fc8d80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:28.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.276+0000 7fc8f44ad640 1 --2- 172.21.15.103:0/2061008689 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc8d80a33b0 0x7fc8d80a5870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4005fd0 tx=0x7fc8e400a5c0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.276+0000 7fc8f44ad640 1 -- 172.21.15.103:0/2061008689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8ec102790 msgr2=0x7fc8ec074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.276+0000 7fc8f44ad640 1 --2- 172.21.15.103:0/2061008689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8ec102790 0x7fc8ec074a30 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0005ab0 tx=0x7fc8e00042f0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.276+0000 7fc8f44ad640 1 -- 172.21.15.103:0/2061008689 shutdown_connections 2024-09-19T06:58:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.276+0000 7fc8f44ad640 1 --2- 172.21.15.103:0/2061008689 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc8d80a33b0 0x7fc8d80a5870 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.276+0000 7fc8f44ad640 1 --2- 172.21.15.103:0/2061008689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8ec106f10 0x7fc8ec074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.276+0000 7fc8f44ad640 1 --2- 172.21.15.103:0/2061008689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8ec102790 0x7fc8ec074a30 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.276+0000 7fc8f44ad640 1 --2- 172.21.15.103:0/2061008689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec101de0 0x7fc8ec0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.276+0000 7fc8f44ad640 1 -- 172.21.15.103:0/2061008689 >> 172.21.15.103:0/2061008689 conn(0x7fc8ec0fd8c0 msgr2=0x7fc8ec0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.276+0000 7fc8f44ad640 1 -- 172.21.15.103:0/2061008689 shutdown_connections 2024-09-19T06:58:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:28.276+0000 7fc8f44ad640 1 -- 172.21.15.103:0/2061008689 wait complete. 2024-09-19T06:58:29.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:29 smithi103 ceph-mon[31158]: pgmap v286: 97 pgs: 97 active+clean; 4.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T06:58:29.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:29 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:58:29.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:29 smithi134 ceph-mon[35356]: pgmap v286: 97 pgs: 97 active+clean; 4.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T06:58:29.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:29 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:58:29.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:29 smithi174 ceph-mon[35637]: pgmap v286: 97 pgs: 97 active+clean; 4.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T06:58:29.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:29 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:58:29.966 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:58:29.967 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:58:30.219 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:58:30.340 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:30 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:58:30.340 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:30 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:58:30.340 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:30 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:58:30.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:30 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:58:30.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:30 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:58:30.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:30 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:58:30.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:30 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:58:30.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:30 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:58:30.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:30 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:58:30.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.660+0000 7f8aaee95640 1 -- 172.21.15.103:0/893891271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8aa8103660 msgr2=0x7f8aa810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:30.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.660+0000 7f8aaee95640 1 --2- 172.21.15.103:0/893891271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8aa8103660 0x7f8aa810bb50 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009900 tx=0x7f8a9c02a480 comp rx=0 tx=0).stop 2024-09-19T06:58:30.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.660+0000 7f8aaee95640 1 -- 172.21.15.103:0/893891271 shutdown_connections 2024-09-19T06:58:30.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.660+0000 7f8aaee95640 1 --2- 172.21.15.103:0/893891271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8aa810c170 0x7f8aa810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.660+0000 7f8aaee95640 1 --2- 172.21.15.103:0/893891271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8aa8103660 0x7f8aa810bb50 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.660+0000 7f8aaee95640 1 --2- 172.21.15.103:0/893891271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa8102d40 0x7f8aa8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.660+0000 7f8aaee95640 1 -- 172.21.15.103:0/893891271 >> 172.21.15.103:0/893891271 conn(0x7f8aa80fc7a0 msgr2=0x7f8aa80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:30.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.660+0000 7f8aaee95640 1 -- 172.21.15.103:0/893891271 shutdown_connections 2024-09-19T06:58:30.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.660+0000 7f8aaee95640 1 -- 172.21.15.103:0/893891271 wait complete. 2024-09-19T06:58:30.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.660+0000 7f8aaee95640 1 Processor -- start 2024-09-19T06:58:30.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aaee95640 1 -- start start 2024-09-19T06:58:30.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aaee95640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa8102d40 0x7f8aa8198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aaee95640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8aa8103660 0x7f8aa81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aaee95640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8aa810c170 0x7f8aa819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aaee95640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa8111f70 con 0x7f8aa810c170 2024-09-19T06:58:30.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aaee95640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa8111df0 con 0x7f8aa8102d40 2024-09-19T06:58:30.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aaee95640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81120f0 con 0x7f8aa8103660 2024-09-19T06:58:30.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aad432640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8aa810c170 0x7f8aa819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aad432640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8aa810c170 0x7f8aa819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53642/0 (socket says 172.21.15.103:53642) 2024-09-19T06:58:30.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aad432640 1 -- 172.21.15.103:0/2834833300 learned_addr learned my addr 172.21.15.103:0/2834833300 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:30.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aacc31640 1 --2- 172.21.15.103:0/2834833300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa8102d40 0x7f8aa8198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aa7fff640 1 --2- 172.21.15.103:0/2834833300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8aa8103660 0x7f8aa81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aad432640 1 -- 172.21.15.103:0/2834833300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8aa8103660 msgr2=0x7f8aa81989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:30.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aad432640 1 --2- 172.21.15.103:0/2834833300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8aa8103660 0x7f8aa81989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aad432640 1 -- 172.21.15.103:0/2834833300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa8102d40 msgr2=0x7f8aa8198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:30.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aad432640 1 --2- 172.21.15.103:0/2834833300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa8102d40 0x7f8aa8198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aad432640 1 -- 172.21.15.103:0/2834833300 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa819d420 con 0x7f8aa810c170 2024-09-19T06:58:30.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aad432640 1 --2- 172.21.15.103:0/2834833300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8aa810c170 0x7f8aa819cd40 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000eaa0 tx=0x7f8aa000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:30.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aa5ffb640 1 -- 172.21.15.103:0/2834833300 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa000cdb0 con 0x7f8aa810c170 2024-09-19T06:58:30.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.661+0000 7f8aa5ffb640 1 -- 172.21.15.103:0/2834833300 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aa0005020 con 0x7f8aa810c170 2024-09-19T06:58:30.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.662+0000 7f8aaee95640 1 -- 172.21.15.103:0/2834833300 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa819d710 con 0x7f8aa810c170 2024-09-19T06:58:30.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.662+0000 7f8aa5ffb640 1 -- 172.21.15.103:0/2834833300 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa00116b0 con 0x7f8aa810c170 2024-09-19T06:58:30.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.662+0000 7f8aaee95640 1 -- 172.21.15.103:0/2834833300 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa81a4ff0 con 0x7f8aa810c170 2024-09-19T06:58:30.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.664+0000 7f8aa5ffb640 1 -- 172.21.15.103:0/2834833300 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8aa0005560 con 0x7f8aa810c170 2024-09-19T06:58:30.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.664+0000 7f8ab126e740 1 Processor -- start 2024-09-19T06:58:30.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.664+0000 7f8ab126e740 1 -- start start 2024-09-19T06:58:30.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.664+0000 7f8aa5ffb640 1 --2- 172.21.15.103:0/2834833300 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8a900a33b0 0x7f8a900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.664+0000 7f8ab126e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb8b60 0x564efeeb8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.664+0000 7f8ab126e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeeb9480 0x564efeec3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.664+0000 7f8ab126e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 0x564efeec6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.664+0000 7f8ab126e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564efee1e280 con 0x564efeeb8b60 2024-09-19T06:58:30.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.664+0000 7f8ab126e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564efedc5b10 con 0x564efeec3d40 2024-09-19T06:58:30.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.664+0000 7f8ab126e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564efee1a240 con 0x564efeeb9480 2024-09-19T06:58:30.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8aa7fff640 1 --2- 172.21.15.103:0/2834833300 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8a900a33b0 0x7f8a900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8aa5ffb640 1 -- 172.21.15.103:0/2834833300 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f8aa00cfed0 con 0x7f8aa810c170 2024-09-19T06:58:30.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8aacc31640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb8b60 0x564efeeb8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8aacc31640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb8b60 0x564efeeb8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53648/0 (socket says 172.21.15.103:53648) 2024-09-19T06:58:30.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8aacc31640 1 -- 172.21.15.103:0/1820874077 learned_addr learned my addr 172.21.15.103:0/1820874077 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:30.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8aa7fff640 1 --2- 172.21.15.103:0/1820874077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeeb9480 0x564efeec3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8aacc31640 1 -- 172.21.15.103:0/1820874077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeeb9480 msgr2=0x564efeec3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:30.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8aacc31640 1 --2- 172.21.15.103:0/1820874077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeeb9480 0x564efeec3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8aacc31640 1 -- 172.21.15.103:0/1820874077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 msgr2=0x564efeec6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:30.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8aad432640 1 --2- 172.21.15.103:0/1820874077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 0x564efeec6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8aacc31640 1 --2- 172.21.15.103:0/1820874077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 0x564efeec6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8aacc31640 1 -- 172.21.15.103:0/1820874077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564efedd21f0 con 0x564efeeb8b60 2024-09-19T06:58:30.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8aa7fff640 1 --2- 172.21.15.103:0/2834833300 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8a900a33b0 0x7f8a900a5870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009230 tx=0x7f8a9c005d50 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:30.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8aacc31640 1 --2- 172.21.15.103:0/1820874077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb8b60 0x564efeeb8f40 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800bcd0 tx=0x7f8a980107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9bd5b34166bcb56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:30.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8a8effd640 1 -- 172.21.15.103:0/1820874077 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9801c680 con 0x564efeeb8b60 2024-09-19T06:58:30.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8a8effd640 1 -- 172.21.15.103:0/1820874077 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9801cd00 con 0x564efeeb8b60 2024-09-19T06:58:30.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.665+0000 7f8a8effd640 1 -- 172.21.15.103:0/1820874077 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98011b60 con 0x564efeeb8b60 2024-09-19T06:58:30.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.666+0000 7f8ab126e740 1 -- 172.21.15.103:0/1820874077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb8b60 msgr2=0x564efeeb8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:30.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.666+0000 7f8ab126e740 1 --2- 172.21.15.103:0/1820874077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb8b60 0x564efeeb8f40 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800bcd0 tx=0x7f8a980107a0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.666+0000 7f8ab126e740 1 -- 172.21.15.103:0/1820874077 shutdown_connections 2024-09-19T06:58:30.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.666+0000 7f8ab126e740 1 --2- 172.21.15.103:0/1820874077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 0x564efeec6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.666+0000 7f8ab126e740 1 --2- 172.21.15.103:0/1820874077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeeb9480 0x564efeec3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.666+0000 7f8ab126e740 1 --2- 172.21.15.103:0/1820874077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb8b60 0x564efeeb8f40 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.666+0000 7f8ab126e740 1 -- 172.21.15.103:0/1820874077 >> 172.21.15.103:0/1820874077 conn(0x564efeeb50f0 msgr2=0x564efeeb54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:30.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.666+0000 7f8ab126e740 1 -- 172.21.15.103:0/1820874077 shutdown_connections 2024-09-19T06:58:30.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.666+0000 7f8ab126e740 1 -- 172.21.15.103:0/1820874077 wait complete. 2024-09-19T06:58:30.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.666+0000 7f8ab126e740 1 -- start start 2024-09-19T06:58:30.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.666+0000 7f8ab126e740 1 Processor -- start 2024-09-19T06:58:30.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8ab126e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeeb8b60 0x564efeec3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8ab126e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb9480 0x564efeec0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8ab126e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 0x564efeec12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8ab126e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564efeecad20 con 0x564efeeb9480 2024-09-19T06:58:30.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8ab126e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564efedd8f10 con 0x564efeec3d40 2024-09-19T06:58:30.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8ab126e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564efeecaea0 con 0x564efeeb8b60 2024-09-19T06:58:30.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8ab126e740 10 client.0 _ll_register_callbacks cb 0x564efeda1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:30.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8aa7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb9480 0x564efeec0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8aa7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb9480 0x564efeec0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53652/0 (socket says 172.21.15.103:53652) 2024-09-19T06:58:30.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8aa7fff640 1 -- 172.21.15.103:0/203016255 learned_addr learned my addr 172.21.15.103:0/203016255 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:30.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8aacc31640 1 --2- 172.21.15.103:0/203016255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeeb8b60 0x564efeec3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8aad432640 1 --2- 172.21.15.103:0/203016255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 0x564efeec12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8aa7fff640 1 -- 172.21.15.103:0/203016255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeeb8b60 msgr2=0x564efeec3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:30.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8aa7fff640 1 --2- 172.21.15.103:0/203016255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeeb8b60 0x564efeec3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8aa7fff640 1 -- 172.21.15.103:0/203016255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 msgr2=0x564efeec12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:30.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8aa7fff640 1 --2- 172.21.15.103:0/203016255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 0x564efeec12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8aa7fff640 1 -- 172.21.15.103:0/203016255 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564efeb8f6a0 con 0x564efeeb9480 2024-09-19T06:58:30.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.667+0000 7f8aad432640 1 --2- 172.21.15.103:0/203016255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 0x564efeec12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:30.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.668+0000 7f8aa7fff640 1 --2- 172.21.15.103:0/203016255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb9480 0x564efeec0b50 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009900 tx=0x7f8a9c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ff3ed418f26356 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:30.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.668+0000 7f8a87fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:30.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.668+0000 7f8a87fff640 1 -- 172.21.15.103:0/203016255 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c050030 con 0x564efeeb9480 2024-09-19T06:58:30.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.668+0000 7f8a87fff640 1 -- 172.21.15.103:0/203016255 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f8a9c003d50 con 0x564efeeb9480 2024-09-19T06:58:30.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.668+0000 7f8ab126e740 1 -- 172.21.15.103:0/203016255 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a9801cd00 con 0x564efeeb9480 2024-09-19T06:58:30.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.668+0000 7f8a87fff640 1 -- 172.21.15.103:0/203016255 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8a9c04c480 con 0x564efeeb9480 2024-09-19T06:58:30.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.668+0000 7f8ab126e740 10 client.18672 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:30.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.668+0000 7f8ab126e740 1 -- 172.21.15.103:0/203016255 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564efeed6850 con 0x564efeeb9480 2024-09-19T06:58:30.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.668+0000 7f8a87fff640 1 -- 172.21.15.103:0/203016255 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9801cd00 con 0x564efeeb9480 2024-09-19T06:58:30.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.668+0000 7f8a87fff640 1 -- 172.21.15.103:0/203016255 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8a9c03d4e0 con 0x564efeeb9480 2024-09-19T06:58:30.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.668+0000 7f8ab126e740 10 client.18672 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:30.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.668+0000 7f8ab126e740 10 client.18672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:58:30.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.668+0000 7f8ab126e740 20 client.18672 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:30.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.668+0000 7f8ab126e740 1 --2- 172.21.15.103:0/203016255 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564efeed7d20 0x564efeeda160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.668+0000 7f8ab126e740 4 client.18672 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:58:30.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.668+0000 7f8ab126e740 1 -- 172.21.15.103:0/203016255 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564efee2b7f0 con 0x564efeed7d20 2024-09-19T06:58:30.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.669+0000 7f8aacc31640 1 --2- 172.21.15.103:0/203016255 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564efeed7d20 0x564efeeda160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.669+0000 7f8a87fff640 10 client.18672 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:30.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.669+0000 7f8aacc31640 1 --2- 172.21.15.103:0/203016255 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564efeed7d20 0x564efeeda160 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c70f2216c8d4baad server_cookie=49008b4f617344f6 in_seq=0 out_seq=0 2024-09-19T06:58:30.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.670+0000 7f8a87fff640 1 -- 172.21.15.103:0/203016255 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564efee2b7f0 con 0x564efeed7d20 2024-09-19T06:58:30.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.670+0000 7f8a87fff640 10 client.18672 handle_command_reply: tid=0 2024-09-19T06:58:30.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.670+0000 7f8ab126e740 1 client.18672 shutdown 2024-09-19T06:58:30.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.670+0000 7f8ab126e740 1 -- 172.21.15.103:0/203016255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb9480 msgr2=0x564efeec0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:30.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.670+0000 7f8ab126e740 1 --2- 172.21.15.103:0/203016255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb9480 0x564efeec0b50 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009900 tx=0x7f8a9c02a480 comp rx=0 tx=0).stop 2024-09-19T06:58:30.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.670+0000 7f8ab126e740 1 -- 172.21.15.103:0/203016255 shutdown_connections 2024-09-19T06:58:30.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.670+0000 7f8ab126e740 1 --2- 172.21.15.103:0/203016255 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564efeed7d20 0x564efeeda160 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.670+0000 7f8ab126e740 1 --2- 172.21.15.103:0/203016255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 0x564efeec12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.670+0000 7f8ab126e740 1 --2- 172.21.15.103:0/203016255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb9480 0x564efeec0b50 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.670+0000 7f8ab126e740 1 --2- 172.21.15.103:0/203016255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeeb8b60 0x564efeec3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.670+0000 7f8ab126e740 1 -- 172.21.15.103:0/203016255 >> 172.21.15.103:0/203016255 conn(0x564efeeb50f0 msgr2=0x564efeeb4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:30.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.670+0000 7f8a87fff640 0 client.18672 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:30.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.670+0000 7f8ab126e740 1 -- 172.21.15.103:0/203016255 shutdown_connections 2024-09-19T06:58:30.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.670+0000 7f8ab126e740 1 -- 172.21.15.103:0/203016255 wait complete. 2024-09-19T06:58:30.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.670+0000 7f8ab126e740 20 client.18672 trim_cache size 0 max 16384 2024-09-19T06:58:30.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.687+0000 7f8ab126e740 1 Processor -- start 2024-09-19T06:58:30.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.687+0000 7f8ab126e740 1 -- start start 2024-09-19T06:58:30.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.687+0000 7f8ab126e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb8b60 0x564efeebfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.687+0000 7f8ab126e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 0x564efeedfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.687+0000 7f8ab126e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeedff70 0x564efeebddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.687+0000 7f8ab126e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81ab760 con 0x564efeeb8b60 2024-09-19T06:58:30.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.687+0000 7f8ab126e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81ad2a0 con 0x564efeec3d40 2024-09-19T06:58:30.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.687+0000 7f8ab126e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81abe30 con 0x564efeedff70 2024-09-19T06:58:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.687+0000 7f8aacc31640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb8b60 0x564efeebfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.687+0000 7f8aacc31640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb8b60 0x564efeebfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53668/0 (socket says 172.21.15.103:53668) 2024-09-19T06:58:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.687+0000 7f8aacc31640 1 -- 172.21.15.103:0/2130470038 learned_addr learned my addr 172.21.15.103:0/2130470038 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.688+0000 7f8aa7fff640 1 --2- 172.21.15.103:0/2130470038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 0x564efeedfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.688+0000 7f8aad432640 1 --2- 172.21.15.103:0/2130470038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeedff70 0x564efeebddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.688+0000 7f8aacc31640 1 -- 172.21.15.103:0/2130470038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeedff70 msgr2=0x564efeebddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.688+0000 7f8aacc31640 1 --2- 172.21.15.103:0/2130470038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeedff70 0x564efeebddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.688+0000 7f8aacc31640 1 -- 172.21.15.103:0/2130470038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 msgr2=0x564efeedfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.688+0000 7f8aacc31640 1 --2- 172.21.15.103:0/2130470038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 0x564efeedfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.688+0000 7f8aacc31640 1 -- 172.21.15.103:0/2130470038 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564efeeb0830 con 0x564efeeb8b60 2024-09-19T06:58:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.688+0000 7f8aa7fff640 1 --2- 172.21.15.103:0/2130470038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 0x564efeedfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.688+0000 7f8aacc31640 1 --2- 172.21.15.103:0/2130470038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb8b60 0x564efeebfb30 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8a98006010 tx=0x7f8a98011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11908dfe0c9a18a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.688+0000 7f8a8e7fc640 1 -- 172.21.15.103:0/2130470038 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9802c070 con 0x564efeeb8b60 2024-09-19T06:58:30.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.688+0000 7f8a8e7fc640 1 -- 172.21.15.103:0/2130470038 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a98005520 con 0x564efeeb8b60 2024-09-19T06:58:30.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.688+0000 7f8a8e7fc640 1 -- 172.21.15.103:0/2130470038 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98011ba0 con 0x564efeeb8b60 2024-09-19T06:58:30.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.689+0000 7f8ab126e740 1 -- 172.21.15.103:0/2130470038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb8b60 msgr2=0x564efeebfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:30.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.689+0000 7f8ab126e740 1 --2- 172.21.15.103:0/2130470038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb8b60 0x564efeebfb30 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8a98006010 tx=0x7f8a98011740 comp rx=0 tx=0).stop 2024-09-19T06:58:30.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.689+0000 7f8ab126e740 1 -- 172.21.15.103:0/2130470038 shutdown_connections 2024-09-19T06:58:30.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.689+0000 7f8ab126e740 1 --2- 172.21.15.103:0/2130470038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeedff70 0x564efeebddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.689+0000 7f8ab126e740 1 --2- 172.21.15.103:0/2130470038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeec3d40 0x564efeedfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.689+0000 7f8ab126e740 1 --2- 172.21.15.103:0/2130470038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeeb8b60 0x564efeebfb30 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.689+0000 7f8ab126e740 1 -- 172.21.15.103:0/2130470038 >> 172.21.15.103:0/2130470038 conn(0x564efeeb50f0 msgr2=0x564efeebf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:30.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.689+0000 7f8ab126e740 1 -- 172.21.15.103:0/2130470038 shutdown_connections 2024-09-19T06:58:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.689+0000 7f8ab126e740 1 -- 172.21.15.103:0/2130470038 wait complete. 2024-09-19T06:58:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.689+0000 7f8ab126e740 1 -- start start 2024-09-19T06:58:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.689+0000 7f8ab126e740 1 Processor -- start 2024-09-19T06:58:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8ab126e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeeb8b60 0x564efeecc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8ab126e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeec3d40 0x564efeeccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8ab126e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeedff70 0x564efef3ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8ab126e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81acbd0 con 0x564efeedff70 2024-09-19T06:58:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8ab126e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81ac500 con 0x564efeeb8b60 2024-09-19T06:58:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8ab126e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81a9760 con 0x564efeec3d40 2024-09-19T06:58:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8ab126e740 10 client.0 _ll_register_callbacks cb 0x7f8a9c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8aad432640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeedff70 0x564efef3ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8aad432640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeedff70 0x564efef3ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53674/0 (socket says 172.21.15.103:53674) 2024-09-19T06:58:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8aad432640 1 -- 172.21.15.103:0/163032371 learned_addr learned my addr 172.21.15.103:0/163032371 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8aacc31640 1 --2- 172.21.15.103:0/163032371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeeb8b60 0x564efeecc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8aa7fff640 1 --2- 172.21.15.103:0/163032371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeec3d40 0x564efeeccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8aad432640 1 -- 172.21.15.103:0/163032371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeec3d40 msgr2=0x564efeeccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8aad432640 1 --2- 172.21.15.103:0/163032371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeec3d40 0x564efeeccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8aad432640 1 -- 172.21.15.103:0/163032371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeeb8b60 msgr2=0x564efeecc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8aad432640 1 --2- 172.21.15.103:0/163032371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeeb8b60 0x564efeecc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8aad432640 1 -- 172.21.15.103:0/163032371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564efef3c080 con 0x564efeedff70 2024-09-19T06:58:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8aa7fff640 1 --2- 172.21.15.103:0/163032371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeec3d40 0x564efeeccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:58:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8aad432640 1 --2- 172.21.15.103:0/163032371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeedff70 0x564efef3ba40 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f8aa009c170 tx=0x7f8aa009c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b94b3c390fb6c24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8a87fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8a87fff640 1 -- 172.21.15.103:0/163032371 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa001f050 con 0x564efeedff70 2024-09-19T06:58:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8a87fff640 1 -- 172.21.15.103:0/163032371 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f8aa00051d0 con 0x564efeedff70 2024-09-19T06:58:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.690+0000 7f8ab126e740 1 -- 172.21.15.103:0/163032371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a98005520 con 0x564efeedff70 2024-09-19T06:58:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.691+0000 7f8a87fff640 1 -- 172.21.15.103:0/163032371 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8aa00bc0b0 con 0x564efeedff70 2024-09-19T06:58:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.691+0000 7f8ab126e740 10 client.18678 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.691+0000 7f8ab126e740 1 -- 172.21.15.103:0/163032371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564efef43490 con 0x564efeedff70 2024-09-19T06:58:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.691+0000 7f8a87fff640 1 -- 172.21.15.103:0/163032371 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a98005520 con 0x564efeedff70 2024-09-19T06:58:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.691+0000 7f8a87fff640 1 -- 172.21.15.103:0/163032371 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8aa000f040 con 0x564efeedff70 2024-09-19T06:58:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.691+0000 7f8ab126e740 10 client.18678 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.691+0000 7f8ab126e740 10 client.18678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:58:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.691+0000 7f8ab126e740 20 client.18678 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.691+0000 7f8ab126e740 1 --2- 172.21.15.103:0/163032371 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564efef43c00 0x564efef45ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.691+0000 7f8ab126e740 4 client.18678 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:58:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.691+0000 7f8ab126e740 1 -- 172.21.15.103:0/163032371 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564efee25d00 con 0x564efef43c00 2024-09-19T06:58:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.691+0000 7f8aacc31640 1 --2- 172.21.15.103:0/163032371 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564efef43c00 0x564efef45ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.691+0000 7f8aacc31640 1 --2- 172.21.15.103:0/163032371 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564efef43c00 0x564efef45ff0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7bbc72d93379b7d server_cookie=dd7c83e7bd814cbe in_seq=0 out_seq=0 2024-09-19T06:58:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.691+0000 7f8a87fff640 10 client.18678 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.692+0000 7f8a87fff640 1 -- 172.21.15.103:0/163032371 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564efee25d00 con 0x564efef43c00 2024-09-19T06:58:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.692+0000 7f8a87fff640 10 client.18678 handle_command_reply: tid=0 2024-09-19T06:58:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.692+0000 7f8ab126e740 1 client.18678 shutdown 2024-09-19T06:58:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.692+0000 7f8ab126e740 1 -- 172.21.15.103:0/163032371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeedff70 msgr2=0x564efef3ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.692+0000 7f8ab126e740 1 --2- 172.21.15.103:0/163032371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeedff70 0x564efef3ba40 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f8aa009c170 tx=0x7f8aa009c2a0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.692+0000 7f8ab126e740 1 -- 172.21.15.103:0/163032371 shutdown_connections 2024-09-19T06:58:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.692+0000 7f8ab126e740 1 --2- 172.21.15.103:0/163032371 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564efef43c00 0x564efef45ff0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.692+0000 7f8ab126e740 1 --2- 172.21.15.103:0/163032371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564efeedff70 0x564efef3ba40 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.693+0000 7f8ab126e740 1 --2- 172.21.15.103:0/163032371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564efeec3d40 0x564efeeccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.693+0000 7f8ab126e740 1 --2- 172.21.15.103:0/163032371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564efeeb8b60 0x564efeecc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.693+0000 7f8ab126e740 1 -- 172.21.15.103:0/163032371 >> 172.21.15.103:0/163032371 conn(0x564efeeb50f0 msgr2=0x564efeebfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.693+0000 7f8a87fff640 0 client.18678 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.693+0000 7f8ab126e740 1 -- 172.21.15.103:0/163032371 shutdown_connections 2024-09-19T06:58:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.693+0000 7f8ab126e740 1 -- 172.21.15.103:0/163032371 wait complete. 2024-09-19T06:58:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.693+0000 7f8ab126e740 20 client.18678 trim_cache size 0 max 16384 2024-09-19T06:58:30.700 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:58:30.700 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T06:58:30.700 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "9d110c86-7c69-46c0-85c5-b0c85c024129", 2024-09-19T06:58:30.700 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T06:58:30.700 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:58:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.693+0000 7f8aaee95640 1 -- 172.21.15.103:0/2834833300 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8a900a33b0 msgr2=0x7f8a900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.693+0000 7f8aaee95640 1 --2- 172.21.15.103:0/2834833300 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8a900a33b0 0x7f8a900a5870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009230 tx=0x7f8a9c005d50 comp rx=0 tx=0).stop 2024-09-19T06:58:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.693+0000 7f8aaee95640 1 -- 172.21.15.103:0/2834833300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8aa810c170 msgr2=0x7f8aa819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.693+0000 7f8aaee95640 1 --2- 172.21.15.103:0/2834833300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8aa810c170 0x7f8aa819cd40 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000eaa0 tx=0x7f8aa000ef60 comp rx=0 tx=0).stop 2024-09-19T06:58:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.693+0000 7f8aaee95640 1 -- 172.21.15.103:0/2834833300 shutdown_connections 2024-09-19T06:58:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.694+0000 7f8aaee95640 1 --2- 172.21.15.103:0/2834833300 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8a900a33b0 0x7f8a900a5870 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.694+0000 7f8aaee95640 1 --2- 172.21.15.103:0/2834833300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8aa810c170 0x7f8aa819cd40 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.694+0000 7f8aaee95640 1 --2- 172.21.15.103:0/2834833300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8aa8103660 0x7f8aa81989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.694+0000 7f8aaee95640 1 --2- 172.21.15.103:0/2834833300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa8102d40 0x7f8aa8198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:30.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.694+0000 7f8aaee95640 1 -- 172.21.15.103:0/2834833300 >> 172.21.15.103:0/2834833300 conn(0x7f8aa80fc7a0 msgr2=0x7f8aa80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:30.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.694+0000 7f8aaee95640 1 -- 172.21.15.103:0/2834833300 shutdown_connections 2024-09-19T06:58:30.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:30.694+0000 7f8aaee95640 1 -- 172.21.15.103:0/2834833300 wait complete. 2024-09-19T06:58:31.403 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:58:31.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:31 smithi174 ceph-mon[35637]: pgmap v287: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-09-19T06:58:31.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:31 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T06:58:31.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:31 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:58:31.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:31 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T06:58:31.584 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:31 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:58:31.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:31 smithi103 ceph-mon[31158]: pgmap v287: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-09-19T06:58:31.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:31 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T06:58:31.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:31 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:58:31.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:31 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T06:58:31.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:31 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:58:31.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:31 smithi134 ceph-mon[35356]: pgmap v287: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-09-19T06:58:31.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:31 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T06:58:31.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:31 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:58:31.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:31 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T06:58:31.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:31 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:58:31.691 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:58:32.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.150+0000 7fae71870640 1 -- 172.21.15.103:0/848172787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae6c102790 msgr2=0x7fae6c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:32.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.150+0000 7fae71870640 1 --2- 172.21.15.103:0/848172787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae6c102790 0x7fae6c1067e0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fae60009900 tx=0x7fae6002a3f0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.150+0000 7fae71870640 1 -- 172.21.15.103:0/848172787 shutdown_connections 2024-09-19T06:58:32.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.150+0000 7fae71870640 1 --2- 172.21.15.103:0/848172787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae6c106f10 0x7fae6c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.150+0000 7fae71870640 1 --2- 172.21.15.103:0/848172787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae6c102790 0x7fae6c1067e0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.150+0000 7fae71870640 1 --2- 172.21.15.103:0/848172787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae6c101de0 0x7fae6c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.150+0000 7fae71870640 1 -- 172.21.15.103:0/848172787 >> 172.21.15.103:0/848172787 conn(0x7fae6c0fd8c0 msgr2=0x7fae6c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:32.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.150+0000 7fae71870640 1 -- 172.21.15.103:0/848172787 shutdown_connections 2024-09-19T06:58:32.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.150+0000 7fae71870640 1 -- 172.21.15.103:0/848172787 wait complete. 2024-09-19T06:58:32.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.151+0000 7fae71870640 1 Processor -- start 2024-09-19T06:58:32.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.151+0000 7fae71870640 1 -- start start 2024-09-19T06:58:32.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.151+0000 7fae71870640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae6c101de0 0x7fae6c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.151+0000 7fae71870640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae6c102790 0x7fae6c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.151+0000 7fae71870640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae6c106f10 0x7fae6c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.151+0000 7fae71870640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c111f90 con 0x7fae6c106f10 2024-09-19T06:58:32.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.151+0000 7fae71870640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c111e10 con 0x7fae6c101de0 2024-09-19T06:58:32.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.151+0000 7fae71870640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c112110 con 0x7fae6c102790 2024-09-19T06:58:32.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.151+0000 7fae6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae6c106f10 0x7fae6c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.151+0000 7fae6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae6c106f10 0x7fae6c19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53688/0 (socket says 172.21.15.103:53688) 2024-09-19T06:58:32.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.151+0000 7fae6b7fe640 1 -- 172.21.15.103:0/1426174941 learned_addr learned my addr 172.21.15.103:0/1426174941 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:32.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.151+0000 7fae6affd640 1 --2- 172.21.15.103:0/1426174941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae6c101de0 0x7fae6c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.152+0000 7fae6a7fc640 1 --2- 172.21.15.103:0/1426174941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae6c102790 0x7fae6c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.152+0000 7fae6b7fe640 1 -- 172.21.15.103:0/1426174941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae6c102790 msgr2=0x7fae6c19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:32.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.152+0000 7fae6b7fe640 1 --2- 172.21.15.103:0/1426174941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae6c102790 0x7fae6c19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.152+0000 7fae6b7fe640 1 -- 172.21.15.103:0/1426174941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae6c101de0 msgr2=0x7fae6c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:32.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.152+0000 7fae6b7fe640 1 --2- 172.21.15.103:0/1426174941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae6c101de0 0x7fae6c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.152+0000 7fae6b7fe640 1 -- 172.21.15.103:0/1426174941 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c19f640 con 0x7fae6c106f10 2024-09-19T06:58:32.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.152+0000 7fae6a7fc640 1 --2- 172.21.15.103:0/1426174941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae6c102790 0x7fae6c19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:58:32.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.152+0000 7fae6b7fe640 1 --2- 172.21.15.103:0/1426174941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae6c106f10 0x7fae6c19ef60 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fae6400b970 tx=0x7fae6400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:32.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.152+0000 7fae7086e640 1 -- 172.21.15.103:0/1426174941 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64019070 con 0x7fae6c106f10 2024-09-19T06:58:32.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.152+0000 7fae7086e640 1 -- 172.21.15.103:0/1426174941 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae64014440 con 0x7fae6c106f10 2024-09-19T06:58:32.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.152+0000 7fae71870640 1 -- 172.21.15.103:0/1426174941 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6c19f930 con 0x7fae6c106f10 2024-09-19T06:58:32.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.152+0000 7fae7086e640 1 -- 172.21.15.103:0/1426174941 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640075d0 con 0x7fae6c106f10 2024-09-19T06:58:32.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.152+0000 7fae71870640 1 -- 172.21.15.103:0/1426174941 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6c1a7210 con 0x7fae6c106f10 2024-09-19T06:58:32.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.154+0000 7fae7086e640 1 -- 172.21.15.103:0/1426174941 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fae6401c070 con 0x7fae6c106f10 2024-09-19T06:58:32.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae73c49740 1 Processor -- start 2024-09-19T06:58:32.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae73c49740 1 -- start start 2024-09-19T06:58:32.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae73c49740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091b14a0 0x5652091b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae73c49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b1dc0 0x5652091bc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae73c49740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091bc680 0x5652091bea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae73c49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5652091172e0 con 0x5652091b1dc0 2024-09-19T06:58:32.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae73c49740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5652090bece0 con 0x5652091b14a0 2024-09-19T06:58:32.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae73c49740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5652091132a0 con 0x5652091bc680 2024-09-19T06:58:32.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae7086e640 1 --2- 172.21.15.103:0/1426174941 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fae540a33b0 0x7fae540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae6affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b1dc0 0x5652091bc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae6affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b1dc0 0x5652091bc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53702/0 (socket says 172.21.15.103:53702) 2024-09-19T06:58:32.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae6affd640 1 -- 172.21.15.103:0/2566478960 learned_addr learned my addr 172.21.15.103:0/2566478960 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:32.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae6b7fe640 1 --2- 172.21.15.103:0/1426174941 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fae540a33b0 0x7fae540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae6a7fc640 1 --2- 172.21.15.103:0/2566478960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091b14a0 0x5652091b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae6a7fc640 1 --2- 172.21.15.103:0/2566478960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091bc680 0x5652091bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae7086e640 1 -- 172.21.15.103:0/1426174941 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7fae640cf2e0 con 0x7fae6c106f10 2024-09-19T06:58:32.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae6affd640 1 -- 172.21.15.103:0/2566478960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091bc680 msgr2=0x5652091bea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:32.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae6affd640 1 --2- 172.21.15.103:0/2566478960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091bc680 0x5652091bea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae6affd640 1 -- 172.21.15.103:0/2566478960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091b14a0 msgr2=0x5652091b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:32.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae6affd640 1 --2- 172.21.15.103:0/2566478960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091b14a0 0x5652091b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae6affd640 1 -- 172.21.15.103:0/2566478960 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652090a3440 con 0x5652091b1dc0 2024-09-19T06:58:32.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.155+0000 7fae6a7fc640 1 --2- 172.21.15.103:0/2566478960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091b14a0 0x5652091b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:32.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.156+0000 7fae6affd640 1 --2- 172.21.15.103:0/2566478960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b1dc0 0x5652091bc140 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00ad00 tx=0x7fae5c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5ceaa1eaed77fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:32.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.156+0000 7fae6b7fe640 1 --2- 172.21.15.103:0/1426174941 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fae540a33b0 0x7fae540a5870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fae640613a0 tx=0x7fae6406d3d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:32.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.156+0000 7fae51ffb640 1 -- 172.21.15.103:0/2566478960 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c01d070 con 0x5652091b1dc0 2024-09-19T06:58:32.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.156+0000 7fae51ffb640 1 -- 172.21.15.103:0/2566478960 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae5c018b20 con 0x5652091b1dc0 2024-09-19T06:58:32.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.156+0000 7fae51ffb640 1 -- 172.21.15.103:0/2566478960 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c010bc0 con 0x5652091b1dc0 2024-09-19T06:58:32.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.156+0000 7fae73c49740 1 -- 172.21.15.103:0/2566478960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b1dc0 msgr2=0x5652091bc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:32.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.156+0000 7fae73c49740 1 --2- 172.21.15.103:0/2566478960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b1dc0 0x5652091bc140 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00ad00 tx=0x7fae5c00f790 comp rx=0 tx=0).stop 2024-09-19T06:58:32.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.156+0000 7fae73c49740 1 -- 172.21.15.103:0/2566478960 shutdown_connections 2024-09-19T06:58:32.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.156+0000 7fae73c49740 1 --2- 172.21.15.103:0/2566478960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091bc680 0x5652091bea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.156+0000 7fae73c49740 1 --2- 172.21.15.103:0/2566478960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b1dc0 0x5652091bc140 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.156+0000 7fae73c49740 1 --2- 172.21.15.103:0/2566478960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091b14a0 0x5652091b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.156+0000 7fae73c49740 1 -- 172.21.15.103:0/2566478960 >> 172.21.15.103:0/2566478960 conn(0x5652091ada30 msgr2=0x5652091ade10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:32.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.157+0000 7fae73c49740 1 -- 172.21.15.103:0/2566478960 shutdown_connections 2024-09-19T06:58:32.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.157+0000 7fae73c49740 1 -- 172.21.15.103:0/2566478960 wait complete. 2024-09-19T06:58:32.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.157+0000 7fae73c49740 1 -- start start 2024-09-19T06:58:32.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.157+0000 7fae73c49740 1 Processor -- start 2024-09-19T06:58:32.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.157+0000 7fae73c49740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091b14a0 0x5652091bbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.157+0000 7fae73c49740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091b1dc0 0x5652091b9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.157+0000 7fae73c49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091bc680 0x5652091b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.157+0000 7fae73c49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5652091bf250 con 0x5652091bc680 2024-09-19T06:58:32.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.157+0000 7fae73c49740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5652090d2050 con 0x5652091b1dc0 2024-09-19T06:58:32.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.157+0000 7fae73c49740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5652091bf3d0 con 0x5652091b14a0 2024-09-19T06:58:32.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.157+0000 7fae73c49740 10 client.0 _ll_register_callbacks cb 0x565209112160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:32.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.157+0000 7fae6a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091bc680 0x5652091b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae6a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091bc680 0x5652091b9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53716/0 (socket says 172.21.15.103:53716) 2024-09-19T06:58:32.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae6a7fc640 1 -- 172.21.15.103:0/867902506 learned_addr learned my addr 172.21.15.103:0/867902506 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:32.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae6affd640 1 --2- 172.21.15.103:0/867902506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091b1dc0 0x5652091b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae6a7fc640 1 --2- 172.21.15.103:0/867902506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091b14a0 0x5652091bbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae6a7fc640 1 -- 172.21.15.103:0/867902506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091b14a0 msgr2=0x5652091bbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:32.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae6a7fc640 1 --2- 172.21.15.103:0/867902506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091b14a0 0x5652091bbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae6a7fc640 1 -- 172.21.15.103:0/867902506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091b1dc0 msgr2=0x5652091b9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:32.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae6a7fc640 1 --2- 172.21.15.103:0/867902506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091b1dc0 0x5652091b9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae6a7fc640 1 -- 172.21.15.103:0/867902506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565208e883c0 con 0x5652091bc680 2024-09-19T06:58:32.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae6a7fc640 1 --2- 172.21.15.103:0/867902506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091bc680 0x5652091b9760 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fae60031ba0 tx=0x7fae6002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2509095454635791 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:32.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae4affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:32.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae4affd640 1 -- 172.21.15.103:0/867902506 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60045030 con 0x5652091bc680 2024-09-19T06:58:32.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae73c49740 1 -- 172.21.15.103:0/867902506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae5c018b20 con 0x5652091bc680 2024-09-19T06:58:32.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae4affd640 1 -- 172.21.15.103:0/867902506 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7fae60004480 con 0x5652091bc680 2024-09-19T06:58:32.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae4affd640 1 -- 172.21.15.103:0/867902506 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae60035040 con 0x5652091bc680 2024-09-19T06:58:32.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae73c49740 10 client.18690 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:32.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.158+0000 7fae73c49740 1 -- 172.21.15.103:0/867902506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5652091cabc0 con 0x5652091bc680 2024-09-19T06:58:32.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.159+0000 7fae4affd640 1 -- 172.21.15.103:0/867902506 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae5c018b20 con 0x5652091bc680 2024-09-19T06:58:32.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.159+0000 7fae4affd640 1 -- 172.21.15.103:0/867902506 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fae6003a040 con 0x5652091bc680 2024-09-19T06:58:32.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.159+0000 7fae73c49740 10 client.18690 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:32.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.159+0000 7fae73c49740 10 client.18690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:58:32.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.159+0000 7fae73c49740 20 client.18690 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:32.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.159+0000 7fae73c49740 1 --2- 172.21.15.103:0/867902506 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5652091cbfd0 0x5652091ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.159+0000 7fae73c49740 4 client.18690 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:58:32.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.159+0000 7fae73c49740 1 -- 172.21.15.103:0/867902506 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565209124850 con 0x5652091cbfd0 2024-09-19T06:58:32.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.159+0000 7fae6affd640 1 --2- 172.21.15.103:0/867902506 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5652091cbfd0 0x5652091ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.159+0000 7fae4affd640 10 client.18690 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:32.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.159+0000 7fae6affd640 1 --2- 172.21.15.103:0/867902506 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5652091cbfd0 0x5652091ce3e0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98ff90c380f64b7e server_cookie=fcf545eba1b1cfd4 in_seq=0 out_seq=0 2024-09-19T06:58:32.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.160+0000 7fae4affd640 1 -- 172.21.15.103:0/867902506 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565209124850 con 0x5652091cbfd0 2024-09-19T06:58:32.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.160+0000 7fae4affd640 10 client.18690 handle_command_reply: tid=0 2024-09-19T06:58:32.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.160+0000 7fae73c49740 1 client.18690 shutdown 2024-09-19T06:58:32.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.160+0000 7fae73c49740 1 -- 172.21.15.103:0/867902506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091bc680 msgr2=0x5652091b9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:32.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.160+0000 7fae73c49740 1 --2- 172.21.15.103:0/867902506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091bc680 0x5652091b9760 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fae60031ba0 tx=0x7fae6002a3f0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.160+0000 7fae73c49740 1 -- 172.21.15.103:0/867902506 shutdown_connections 2024-09-19T06:58:32.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.160+0000 7fae73c49740 1 --2- 172.21.15.103:0/867902506 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5652091cbfd0 0x5652091ce3e0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.160+0000 7fae73c49740 1 --2- 172.21.15.103:0/867902506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091bc680 0x5652091b9760 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.160+0000 7fae73c49740 1 --2- 172.21.15.103:0/867902506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091b1dc0 0x5652091b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.160+0000 7fae73c49740 1 --2- 172.21.15.103:0/867902506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091b14a0 0x5652091bbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.160+0000 7fae4affd640 0 client.18690 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:32.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.160+0000 7fae73c49740 1 -- 172.21.15.103:0/867902506 >> 172.21.15.103:0/867902506 conn(0x5652091ada30 msgr2=0x5652091acb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:32.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.160+0000 7fae73c49740 1 -- 172.21.15.103:0/867902506 shutdown_connections 2024-09-19T06:58:32.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.160+0000 7fae73c49740 1 -- 172.21.15.103:0/867902506 wait complete. 2024-09-19T06:58:32.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.161+0000 7fae73c49740 20 client.18690 trim_cache size 0 max 16384 2024-09-19T06:58:32.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.177+0000 7fae73c49740 1 Processor -- start 2024-09-19T06:58:32.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.177+0000 7fae73c49740 1 -- start start 2024-09-19T06:58:32.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae73c49740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091ada30 0x5652091c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae73c49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b14a0 0x5652091d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae73c49740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091b1dc0 0x5652091d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae73c49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1afb60 con 0x5652091b14a0 2024-09-19T06:58:32.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae73c49740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b16a0 con 0x5652091ada30 2024-09-19T06:58:32.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae73c49740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b0230 con 0x5652091b1dc0 2024-09-19T06:58:32.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae6affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b14a0 0x5652091d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae6affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b14a0 0x5652091d3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53720/0 (socket says 172.21.15.103:53720) 2024-09-19T06:58:32.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae6affd640 1 -- 172.21.15.103:0/742872417 learned_addr learned my addr 172.21.15.103:0/742872417 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:32.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae6a7fc640 1 --2- 172.21.15.103:0/742872417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091ada30 0x5652091c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae6a7fc640 1 --2- 172.21.15.103:0/742872417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091b1dc0 0x5652091d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae6affd640 1 -- 172.21.15.103:0/742872417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091b1dc0 msgr2=0x5652091d41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:32.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae6affd640 1 --2- 172.21.15.103:0/742872417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091b1dc0 0x5652091d41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae6affd640 1 -- 172.21.15.103:0/742872417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091ada30 msgr2=0x5652091c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:32.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae6affd640 1 --2- 172.21.15.103:0/742872417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091ada30 0x5652091c84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae6affd640 1 -- 172.21.15.103:0/742872417 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652091bc2e0 con 0x5652091b14a0 2024-09-19T06:58:32.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae6a7fc640 1 --2- 172.21.15.103:0/742872417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091ada30 0x5652091c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:32.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae6affd640 1 --2- 172.21.15.103:0/742872417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b14a0 0x5652091d3cb0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00ad00 tx=0x7fae5c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d891ede9471248a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:32.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae517fa640 1 -- 172.21.15.103:0/742872417 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c031070 con 0x5652091b14a0 2024-09-19T06:58:32.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae517fa640 1 -- 172.21.15.103:0/742872417 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae5c00a730 con 0x5652091b14a0 2024-09-19T06:58:32.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.178+0000 7fae517fa640 1 -- 172.21.15.103:0/742872417 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c01d040 con 0x5652091b14a0 2024-09-19T06:58:32.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.179+0000 7fae73c49740 1 -- 172.21.15.103:0/742872417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b14a0 msgr2=0x5652091d3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:32.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.179+0000 7fae73c49740 1 --2- 172.21.15.103:0/742872417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b14a0 0x5652091d3cb0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00ad00 tx=0x7fae5c010800 comp rx=0 tx=0).stop 2024-09-19T06:58:32.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.179+0000 7fae73c49740 1 -- 172.21.15.103:0/742872417 shutdown_connections 2024-09-19T06:58:32.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.179+0000 7fae73c49740 1 --2- 172.21.15.103:0/742872417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091b1dc0 0x5652091d41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.179+0000 7fae73c49740 1 --2- 172.21.15.103:0/742872417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b14a0 0x5652091d3cb0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.179+0000 7fae73c49740 1 --2- 172.21.15.103:0/742872417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091ada30 0x5652091c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.179+0000 7fae73c49740 1 -- 172.21.15.103:0/742872417 >> 172.21.15.103:0/742872417 conn(0x5652091b98c0 msgr2=0x5652091b7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:32.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.179+0000 7fae73c49740 1 -- 172.21.15.103:0/742872417 shutdown_connections 2024-09-19T06:58:32.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.179+0000 7fae73c49740 1 -- 172.21.15.103:0/742872417 wait complete. 2024-09-19T06:58:32.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.179+0000 7fae73c49740 1 -- start start 2024-09-19T06:58:32.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae73c49740 1 Processor -- start 2024-09-19T06:58:32.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae73c49740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091ada30 0x5652091c0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae73c49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b14a0 0x5652091c0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae73c49740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091b1dc0 0x5652091c0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae73c49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b0fd0 con 0x5652091b14a0 2024-09-19T06:58:32.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae73c49740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b0900 con 0x5652091b1dc0 2024-09-19T06:58:32.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae73c49740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1adb60 con 0x5652091ada30 2024-09-19T06:58:32.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae73c49740 10 client.0 _ll_register_callbacks cb 0x7fae60009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:32.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae6affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b14a0 0x5652091c0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae6affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b14a0 0x5652091c0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53724/0 (socket says 172.21.15.103:53724) 2024-09-19T06:58:32.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae6affd640 1 -- 172.21.15.103:0/518580273 learned_addr learned my addr 172.21.15.103:0/518580273 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:32.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae6a7fc640 1 --2- 172.21.15.103:0/518580273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091ada30 0x5652091c0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae6affd640 1 -- 172.21.15.103:0/518580273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091ada30 msgr2=0x5652091c0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:32.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae6affd640 1 --2- 172.21.15.103:0/518580273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091ada30 0x5652091c0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae6affd640 1 -- 172.21.15.103:0/518580273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091b1dc0 msgr2=0x5652091c0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:32.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae6affd640 1 --2- 172.21.15.103:0/518580273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091b1dc0 0x5652091c0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae6affd640 1 -- 172.21.15.103:0/518580273 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565209235d50 con 0x5652091b14a0 2024-09-19T06:58:32.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.180+0000 7fae6a7fc640 1 --2- 172.21.15.103:0/518580273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091ada30 0x5652091c0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:58:32.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.181+0000 7fae6affd640 1 --2- 172.21.15.103:0/518580273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b14a0 0x5652091c0980 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00a5c0 tx=0x7fae5c010870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b627a4b0f35c45 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:32.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.181+0000 7fae4affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:32.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.181+0000 7fae4affd640 1 -- 172.21.15.103:0/518580273 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c01d030 con 0x5652091b14a0 2024-09-19T06:58:32.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.181+0000 7fae4affd640 1 -- 172.21.15.103:0/518580273 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7fae5c010b20 con 0x5652091b14a0 2024-09-19T06:58:32.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.181+0000 7fae73c49740 1 -- 172.21.15.103:0/518580273 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae5c00a730 con 0x5652091b14a0 2024-09-19T06:58:32.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.181+0000 7fae4affd640 1 -- 172.21.15.103:0/518580273 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae5c031070 con 0x5652091b14a0 2024-09-19T06:58:32.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.181+0000 7fae73c49740 10 client.18696 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:32.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.181+0000 7fae73c49740 1 -- 172.21.15.103:0/518580273 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565209237040 con 0x5652091b14a0 2024-09-19T06:58:32.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.181+0000 7fae4affd640 1 -- 172.21.15.103:0/518580273 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae5c00a730 con 0x5652091b14a0 2024-09-19T06:58:32.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.181+0000 7fae4affd640 1 -- 172.21.15.103:0/518580273 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fae5c005480 con 0x5652091b14a0 2024-09-19T06:58:32.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.181+0000 7fae73c49740 10 client.18696 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:32.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.181+0000 7fae73c49740 10 client.18696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:58:32.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.181+0000 7fae73c49740 20 client.18696 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:32.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.181+0000 7fae73c49740 1 --2- 172.21.15.103:0/518580273 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5652091bc680 0x5652092395a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:32.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.181+0000 7fae73c49740 4 client.18696 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:58:32.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.181+0000 7fae73c49740 1 -- 172.21.15.103:0/518580273 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56520911ed60 con 0x5652091bc680 2024-09-19T06:58:32.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.182+0000 7fae6a7fc640 1 --2- 172.21.15.103:0/518580273 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5652091bc680 0x5652092395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:32.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.182+0000 7fae4affd640 10 client.18696 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:32.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.182+0000 7fae6a7fc640 1 --2- 172.21.15.103:0/518580273 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5652091bc680 0x5652092395a0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eacf0fa5d4e3907 server_cookie=85abfc6825e282b8 in_seq=0 out_seq=0 2024-09-19T06:58:32.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.182+0000 7fae4affd640 1 -- 172.21.15.103:0/518580273 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56520911ed60 con 0x5652091bc680 2024-09-19T06:58:32.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.182+0000 7fae4affd640 10 client.18696 handle_command_reply: tid=0 2024-09-19T06:58:32.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.182+0000 7fae73c49740 1 client.18696 shutdown 2024-09-19T06:58:32.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.182+0000 7fae73c49740 1 -- 172.21.15.103:0/518580273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b14a0 msgr2=0x5652091c0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:32.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.182+0000 7fae73c49740 1 --2- 172.21.15.103:0/518580273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b14a0 0x5652091c0980 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00a5c0 tx=0x7fae5c010870 comp rx=0 tx=0).stop 2024-09-19T06:58:32.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.182+0000 7fae73c49740 1 -- 172.21.15.103:0/518580273 shutdown_connections 2024-09-19T06:58:32.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.182+0000 7fae73c49740 1 --2- 172.21.15.103:0/518580273 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5652091bc680 0x5652092395a0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.182+0000 7fae73c49740 1 --2- 172.21.15.103:0/518580273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652091b1dc0 0x5652091c0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.182+0000 7fae73c49740 1 --2- 172.21.15.103:0/518580273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652091b14a0 0x5652091c0980 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.182+0000 7fae73c49740 1 --2- 172.21.15.103:0/518580273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652091ada30 0x5652091c0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.182+0000 7fae73c49740 1 -- 172.21.15.103:0/518580273 >> 172.21.15.103:0/518580273 conn(0x5652091b98c0 msgr2=0x5652091ac350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:32.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.182+0000 7fae4affd640 0 client.18696 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:32.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.183+0000 7fae73c49740 1 -- 172.21.15.103:0/518580273 shutdown_connections 2024-09-19T06:58:32.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.183+0000 7fae73c49740 1 -- 172.21.15.103:0/518580273 wait complete. 2024-09-19T06:58:32.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.183+0000 7fae73c49740 20 client.18696 trim_cache size 0 max 16384 2024-09-19T06:58:32.190 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:58:32.190 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:58:32.191 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T06:58:32.191 INFO:teuthology.orchestra.run.smithi103.stdout: "9d110c86-7c69-46c0-85c5-b0c85c024129": { 2024-09-19T06:58:32.191 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T06:58:32.191 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "9d110c86-7c69-46c0-85c5-b0c85c024129", 2024-09-19T06:58:32.191 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T06:58:32.191 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:58:32.191 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:58:32.191 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:58:32.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.183+0000 7fae71870640 1 -- 172.21.15.103:0/1426174941 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fae540a33b0 msgr2=0x7fae540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:32.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.183+0000 7fae71870640 1 --2- 172.21.15.103:0/1426174941 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fae540a33b0 0x7fae540a5870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fae640613a0 tx=0x7fae6406d3d0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.183+0000 7fae71870640 1 -- 172.21.15.103:0/1426174941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae6c106f10 msgr2=0x7fae6c19ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:32.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.183+0000 7fae71870640 1 --2- 172.21.15.103:0/1426174941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae6c106f10 0x7fae6c19ef60 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fae6400b970 tx=0x7fae6400bd70 comp rx=0 tx=0).stop 2024-09-19T06:58:32.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.183+0000 7fae71870640 1 -- 172.21.15.103:0/1426174941 shutdown_connections 2024-09-19T06:58:32.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.183+0000 7fae71870640 1 --2- 172.21.15.103:0/1426174941 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fae540a33b0 0x7fae540a5870 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.183+0000 7fae71870640 1 --2- 172.21.15.103:0/1426174941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae6c106f10 0x7fae6c19ef60 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.183+0000 7fae71870640 1 --2- 172.21.15.103:0/1426174941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae6c102790 0x7fae6c19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.183+0000 7fae71870640 1 --2- 172.21.15.103:0/1426174941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae6c101de0 0x7fae6c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:32.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.183+0000 7fae71870640 1 -- 172.21.15.103:0/1426174941 >> 172.21.15.103:0/1426174941 conn(0x7fae6c0fd8c0 msgr2=0x7fae6c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:32.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.183+0000 7fae71870640 1 -- 172.21.15.103:0/1426174941 shutdown_connections 2024-09-19T06:58:32.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:58:32.183+0000 7fae71870640 1 -- 172.21.15.103:0/1426174941 wait complete. 2024-09-19T06:58:32.777 INFO:tasks.cephfs.filesystem:scrub status for tag:9d110c86-7c69-46c0-85c5-b0c85c024129 - {'path': '/', 'tag': '9d110c86-7c69-46c0-85c5-b0c85c024129', 'options': 'recursive,force'} 2024-09-19T06:58:33.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:33 smithi174 ceph-mon[35637]: pgmap v288: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T06:58:33.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:33 smithi103 ceph-mon[31158]: pgmap v288: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T06:58:33.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:33 smithi134 ceph-mon[35356]: pgmap v288: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T06:58:35.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:35 smithi174 ceph-mon[35637]: pgmap v289: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T06:58:35.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:35 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T06:58:35.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:35 smithi103 ceph-mon[31158]: pgmap v289: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T06:58:35.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:35 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T06:58:35.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:35 smithi134 ceph-mon[35356]: pgmap v289: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T06:58:35.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:35 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T06:58:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.104+0000 7ff5b7e7b640 1 -- 172.21.15.103:0/3104158544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b01069e0 msgr2=0x7ff5b0106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.104+0000 7ff5b7e7b640 1 --2- 172.21.15.103:0/3104158544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b01069e0 0x7ff5b0106dc0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009930 tx=0x7ff5ac02a420 comp rx=0 tx=0).stop 2024-09-19T06:58:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.104+0000 7ff5b7e7b640 1 -- 172.21.15.103:0/3104158544 shutdown_connections 2024-09-19T06:58:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.104+0000 7ff5b7e7b640 1 --2- 172.21.15.103:0/3104158544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5b0101370 0x7ff5b010d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.104+0000 7ff5b7e7b640 1 --2- 172.21.15.103:0/3104158544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5b01009d0 0x7ff5b0100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.104+0000 7ff5b7e7b640 1 --2- 172.21.15.103:0/3104158544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b01069e0 0x7ff5b0106dc0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.104+0000 7ff5b7e7b640 1 -- 172.21.15.103:0/3104158544 >> 172.21.15.103:0/3104158544 conn(0x7ff5b00fc6b0 msgr2=0x7ff5b00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.105+0000 7ff5b7e7b640 1 -- 172.21.15.103:0/3104158544 shutdown_connections 2024-09-19T06:58:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.105+0000 7ff5b7e7b640 1 -- 172.21.15.103:0/3104158544 wait complete. 2024-09-19T06:58:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.105+0000 7ff5b7e7b640 1 Processor -- start 2024-09-19T06:58:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.105+0000 7ff5b7e7b640 1 -- start start 2024-09-19T06:58:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.105+0000 7ff5b7e7b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5b01009d0 0x7ff5b01991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.105+0000 7ff5b7e7b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5b0101370 0x7ff5b0199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.105+0000 7ff5b7e7b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b01069e0 0x7ff5b01a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.105+0000 7ff5b7e7b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0110e20 con 0x7ff5b01069e0 2024-09-19T06:58:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.105+0000 7ff5b7e7b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0110ca0 con 0x7ff5b01009d0 2024-09-19T06:58:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.105+0000 7ff5b7e7b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0110fa0 con 0x7ff5b0101370 2024-09-19T06:58:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.106+0000 7ff5b6418640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b01069e0 0x7ff5b01a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.106+0000 7ff5b6418640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b01069e0 0x7ff5b01a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53740/0 (socket says 172.21.15.103:53740) 2024-09-19T06:58:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.106+0000 7ff5b6418640 1 -- 172.21.15.103:0/1934779448 learned_addr learned my addr 172.21.15.103:0/1934779448 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.106+0000 7ff5b5c17640 1 --2- 172.21.15.103:0/1934779448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5b01009d0 0x7ff5b01991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.106+0000 7ff5b5416640 1 --2- 172.21.15.103:0/1934779448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5b0101370 0x7ff5b0199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.106+0000 7ff5b6418640 1 -- 172.21.15.103:0/1934779448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5b0101370 msgr2=0x7ff5b0199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.106+0000 7ff5b6418640 1 --2- 172.21.15.103:0/1934779448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5b0101370 0x7ff5b0199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.106+0000 7ff5b6418640 1 -- 172.21.15.103:0/1934779448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5b01009d0 msgr2=0x7ff5b01991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.106+0000 7ff5b6418640 1 --2- 172.21.15.103:0/1934779448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5b01009d0 0x7ff5b01991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.106+0000 7ff5b6418640 1 -- 172.21.15.103:0/1934779448 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b01a47b0 con 0x7ff5b01069e0 2024-09-19T06:58:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.106+0000 7ff5b6418640 1 --2- 172.21.15.103:0/1934779448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b01069e0 0x7ff5b01a4270 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009fd0 tx=0x7ff5a800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.106+0000 7ff5a6ffd640 1 -- 172.21.15.103:0/1934779448 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8017070 con 0x7ff5b01069e0 2024-09-19T06:58:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.106+0000 7ff5a6ffd640 1 -- 172.21.15.103:0/1934779448 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a8003e50 con 0x7ff5b01069e0 2024-09-19T06:58:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.106+0000 7ff5b7e7b640 1 -- 172.21.15.103:0/1934779448 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b01a4980 con 0x7ff5b01069e0 2024-09-19T06:58:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.106+0000 7ff5a6ffd640 1 -- 172.21.15.103:0/1934779448 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80106c0 con 0x7ff5b01069e0 2024-09-19T06:58:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.106+0000 7ff5b7e7b640 1 -- 172.21.15.103:0/1934779448 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b01a4ea0 con 0x7ff5b01069e0 2024-09-19T06:58:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.109+0000 7ff5a6ffd640 1 -- 172.21.15.103:0/1934779448 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff5a8003970 con 0x7ff5b01069e0 2024-09-19T06:58:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.109+0000 7ff5bac18740 1 Processor -- start 2024-09-19T06:58:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.109+0000 7ff5bac18740 1 -- start start 2024-09-19T06:58:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5bac18740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5bac18740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8a5410 0x55ff7b8af770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5bac18740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5bac18740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ff7b8092f0 con 0x55ff7b8a5410 2024-09-19T06:58:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5bac18740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ff7b7b1270 con 0x55ff7b8afcb0 2024-09-19T06:58:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5bac18740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ff7b8052b0 con 0x55ff7b8a4af0 2024-09-19T06:58:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5a6ffd640 1 --2- 172.21.15.103:0/1934779448 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff59c0a34e0 0x7ff59c0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5b5c17640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8a5410 0x55ff7b8af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5b5c17640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8a5410 0x55ff7b8af770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53754/0 (socket says 172.21.15.103:53754) 2024-09-19T06:58:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5b5c17640 1 -- 172.21.15.103:0/3516403440 learned_addr learned my addr 172.21.15.103:0/3516403440 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5b6418640 1 --2- 172.21.15.103:0/1934779448 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff59c0a34e0 0x7ff59c0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5b5416640 1 --2- 172.21.15.103:0/3516403440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5a6ffd640 1 -- 172.21.15.103:0/1934779448 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7ff5a80cf920 con 0x7ff5b01069e0 2024-09-19T06:58:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5b5416640 1 --2- 172.21.15.103:0/3516403440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5b5c17640 1 -- 172.21.15.103:0/3516403440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a4af0 msgr2=0x55ff7b8a4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5b5c17640 1 --2- 172.21.15.103:0/3516403440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8a4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5b5c17640 1 -- 172.21.15.103:0/3516403440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8afcb0 msgr2=0x55ff7b8b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5b5c17640 1 --2- 172.21.15.103:0/3516403440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5b5c17640 1 -- 172.21.15.103:0/3516403440 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff7b7b0340 con 0x55ff7b8a5410 2024-09-19T06:58:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5b6418640 1 --2- 172.21.15.103:0/1934779448 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff59c0a34e0 0x7ff59c0a59a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80cd650 tx=0x7ff5a8091120 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.110+0000 7ff5b5c17640 1 --2- 172.21.15.103:0/3516403440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8a5410 0x55ff7b8af770 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac02a940 tx=0x7ff5ac005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3abc735569abb74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.111+0000 7ff59bfff640 1 -- 172.21.15.103:0/3516403440 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac040070 con 0x55ff7b8a5410 2024-09-19T06:58:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.111+0000 7ff59bfff640 1 -- 172.21.15.103:0/3516403440 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ac0318f0 con 0x55ff7b8a5410 2024-09-19T06:58:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.111+0000 7ff59bfff640 1 -- 172.21.15.103:0/3516403440 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac035040 con 0x55ff7b8a5410 2024-09-19T06:58:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.111+0000 7ff5bac18740 1 -- 172.21.15.103:0/3516403440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8a5410 msgr2=0x55ff7b8af770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.111+0000 7ff5bac18740 1 --2- 172.21.15.103:0/3516403440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8a5410 0x55ff7b8af770 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac02a940 tx=0x7ff5ac005c00 comp rx=0 tx=0).stop 2024-09-19T06:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.111+0000 7ff5bac18740 1 -- 172.21.15.103:0/3516403440 shutdown_connections 2024-09-19T06:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.111+0000 7ff5bac18740 1 --2- 172.21.15.103:0/3516403440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.111+0000 7ff5bac18740 1 --2- 172.21.15.103:0/3516403440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8a5410 0x55ff7b8af770 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.111+0000 7ff5bac18740 1 --2- 172.21.15.103:0/3516403440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8a4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.111+0000 7ff5bac18740 1 -- 172.21.15.103:0/3516403440 >> 172.21.15.103:0/3516403440 conn(0x55ff7b8a1060 msgr2=0x55ff7b8a1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.111+0000 7ff5bac18740 1 -- 172.21.15.103:0/3516403440 shutdown_connections 2024-09-19T06:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.111+0000 7ff5bac18740 1 -- 172.21.15.103:0/3516403440 wait complete. 2024-09-19T06:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.112+0000 7ff5bac18740 1 -- start start 2024-09-19T06:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.112+0000 7ff5bac18740 1 Processor -- start 2024-09-19T06:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.112+0000 7ff5bac18740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8aca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.112+0000 7ff5bac18740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a5410 0x55ff7b8acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.112+0000 7ff5bac18740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.112+0000 7ff5bac18740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ff7b8b2880 con 0x55ff7b8a4af0 2024-09-19T06:58:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.112+0000 7ff5bac18740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ff7b792290 con 0x55ff7b8afcb0 2024-09-19T06:58:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.112+0000 7ff5bac18740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ff7b8b2a00 con 0x55ff7b8a5410 2024-09-19T06:58:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.112+0000 7ff5bac18740 10 client.0 _ll_register_callbacks cb 0x55ff7b7c3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.112+0000 7ff5b5416640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.112+0000 7ff5b5416640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8aca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53768/0 (socket says 172.21.15.103:53768) 2024-09-19T06:58:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.112+0000 7ff5b5416640 1 -- 172.21.15.103:0/622585736 learned_addr learned my addr 172.21.15.103:0/622585736 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.113+0000 7ff5b5416640 1 --2- 172.21.15.103:0/622585736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.113+0000 7ff5b5c17640 1 --2- 172.21.15.103:0/622585736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a5410 0x55ff7b8acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.113+0000 7ff5b5416640 1 -- 172.21.15.103:0/622585736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a5410 msgr2=0x55ff7b8acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.113+0000 7ff5b5416640 1 --2- 172.21.15.103:0/622585736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a5410 0x55ff7b8acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.113+0000 7ff5b5416640 1 -- 172.21.15.103:0/622585736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8afcb0 msgr2=0x55ff7b8ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.113+0000 7ff5b5416640 1 --2- 172.21.15.103:0/622585736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8ad4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.113+0000 7ff5b5416640 1 -- 172.21.15.103:0/622585736 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff7b51c210 con 0x55ff7b8a4af0 2024-09-19T06:58:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.113+0000 7ff5b5416640 1 --2- 172.21.15.103:0/622585736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8aca60 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0005d00 tx=0x7ff5a000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf4e41efb892ea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.113+0000 7ff5997fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.113+0000 7ff5997fa640 1 -- 172.21.15.103:0/622585736 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0004020 con 0x55ff7b8a4af0 2024-09-19T06:58:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.113+0000 7ff5bac18740 1 -- 172.21.15.103:0/622585736 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5ac0318f0 con 0x55ff7b8a4af0 2024-09-19T06:58:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.113+0000 7ff5997fa640 1 -- 172.21.15.103:0/622585736 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7ff5a00041c0 con 0x55ff7b8a4af0 2024-09-19T06:58:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.113+0000 7ff5997fa640 1 -- 172.21.15.103:0/622585736 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5a00043c0 con 0x55ff7b8a4af0 2024-09-19T06:58:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.113+0000 7ff5bac18740 10 client.18708 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.113+0000 7ff5bac18740 1 -- 172.21.15.103:0/622585736 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ff7b5e8620 con 0x55ff7b8a4af0 2024-09-19T06:58:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.113+0000 7ff5997fa640 1 -- 172.21.15.103:0/622585736 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ac0318f0 con 0x55ff7b8a4af0 2024-09-19T06:58:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.114+0000 7ff5997fa640 1 -- 172.21.15.103:0/622585736 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff5a0004b70 con 0x55ff7b8a4af0 2024-09-19T06:58:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.114+0000 7ff5bac18740 10 client.18708 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.114+0000 7ff5bac18740 10 client.18708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:58:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.114+0000 7ff5bac18740 20 client.18708 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.114+0000 7ff5bac18740 1 --2- 172.21.15.103:0/622585736 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ff7b8bfb00 0x55ff7b8c1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.114+0000 7ff5bac18740 4 client.18708 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:58:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.114+0000 7ff5bac18740 1 -- 172.21.15.103:0/622585736 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff7b816a20 con 0x55ff7b8bfb00 2024-09-19T06:58:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.114+0000 7ff5b5c17640 1 --2- 172.21.15.103:0/622585736 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ff7b8bfb00 0x55ff7b8c1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.114+0000 7ff5997fa640 10 client.18708 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.114+0000 7ff5b5c17640 1 --2- 172.21.15.103:0/622585736 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ff7b8bfb00 0x55ff7b8c1ef0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3853a95e97db0de4 server_cookie=423a7ed313cfec86 in_seq=0 out_seq=0 2024-09-19T06:58:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.115+0000 7ff5997fa640 1 -- 172.21.15.103:0/622585736 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff7b816a20 con 0x55ff7b8bfb00 2024-09-19T06:58:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.115+0000 7ff5997fa640 10 client.18708 handle_command_reply: tid=0 2024-09-19T06:58:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.115+0000 7ff5bac18740 1 client.18708 shutdown 2024-09-19T06:58:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.115+0000 7ff5bac18740 1 -- 172.21.15.103:0/622585736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8a4af0 msgr2=0x55ff7b8aca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.115+0000 7ff5bac18740 1 --2- 172.21.15.103:0/622585736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8aca60 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0005d00 tx=0x7ff5a000bbb0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.115+0000 7ff5bac18740 1 -- 172.21.15.103:0/622585736 shutdown_connections 2024-09-19T06:58:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.115+0000 7ff5bac18740 1 --2- 172.21.15.103:0/622585736 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ff7b8bfb00 0x55ff7b8c1ef0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.115+0000 7ff5bac18740 1 --2- 172.21.15.103:0/622585736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.115+0000 7ff5bac18740 1 --2- 172.21.15.103:0/622585736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a5410 0x55ff7b8acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.115+0000 7ff5bac18740 1 --2- 172.21.15.103:0/622585736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8aca60 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.115+0000 7ff5997fa640 0 client.18708 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.115+0000 7ff5bac18740 1 -- 172.21.15.103:0/622585736 >> 172.21.15.103:0/622585736 conn(0x55ff7b8a1060 msgr2=0x55ff7b8a6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.115+0000 7ff5bac18740 1 -- 172.21.15.103:0/622585736 shutdown_connections 2024-09-19T06:58:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.116+0000 7ff5bac18740 1 -- 172.21.15.103:0/622585736 wait complete. 2024-09-19T06:58:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.116+0000 7ff5bac18740 20 client.18708 trim_cache size 0 max 16384 2024-09-19T06:58:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5bac18740 1 Processor -- start 2024-09-19T06:58:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5bac18740 1 -- start start 2024-09-19T06:58:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5bac18740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5bac18740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8a5410 0x55ff7b8c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5bac18740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5bac18740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01ad850 con 0x55ff7b8afcb0 2024-09-19T06:58:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5bac18740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01af390 con 0x55ff7b8a5410 2024-09-19T06:58:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5bac18740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01adf20 con 0x55ff7b8a4af0 2024-09-19T06:58:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5b5416640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5b5416640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53778/0 (socket says 172.21.15.103:53778) 2024-09-19T06:58:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5b5416640 1 -- 172.21.15.103:0/2861851298 learned_addr learned my addr 172.21.15.103:0/2861851298 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5b5c17640 1 --2- 172.21.15.103:0/2861851298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8a5410 0x55ff7b8c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5b5416640 1 -- 172.21.15.103:0/2861851298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a4af0 msgr2=0x55ff7b8aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5b5416640 1 --2- 172.21.15.103:0/2861851298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5b5416640 1 -- 172.21.15.103:0/2861851298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8a5410 msgr2=0x55ff7b8c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5b5416640 1 --2- 172.21.15.103:0/2861851298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8a5410 0x55ff7b8c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.130+0000 7ff5b5416640 1 -- 172.21.15.103:0/2861851298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff7b8a0e90 con 0x55ff7b8afcb0 2024-09-19T06:58:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.131+0000 7ff5b5416640 1 --2- 172.21.15.103:0/2861851298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8c7d20 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000ba40 tx=0x7ff5a000b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee154689b54309f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.131+0000 7ff59b7fe640 1 -- 172.21.15.103:0/2861851298 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a00036c0 con 0x55ff7b8afcb0 2024-09-19T06:58:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.131+0000 7ff59b7fe640 1 -- 172.21.15.103:0/2861851298 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a0003860 con 0x55ff7b8afcb0 2024-09-19T06:58:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.131+0000 7ff59b7fe640 1 -- 172.21.15.103:0/2861851298 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0003da0 con 0x55ff7b8afcb0 2024-09-19T06:58:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.131+0000 7ff5bac18740 1 -- 172.21.15.103:0/2861851298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8afcb0 msgr2=0x55ff7b8c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.131+0000 7ff5bac18740 1 --2- 172.21.15.103:0/2861851298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8c7d20 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000ba40 tx=0x7ff5a000b710 comp rx=0 tx=0).stop 2024-09-19T06:58:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.131+0000 7ff5bac18740 1 -- 172.21.15.103:0/2861851298 shutdown_connections 2024-09-19T06:58:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.131+0000 7ff5bac18740 1 --2- 172.21.15.103:0/2861851298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8c7d20 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.131+0000 7ff5bac18740 1 --2- 172.21.15.103:0/2861851298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8a5410 0x55ff7b8c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.131+0000 7ff5bac18740 1 --2- 172.21.15.103:0/2861851298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.131+0000 7ff5bac18740 1 -- 172.21.15.103:0/2861851298 >> 172.21.15.103:0/2861851298 conn(0x55ff7b8a1060 msgr2=0x55ff7b8bb360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.131+0000 7ff5bac18740 1 -- 172.21.15.103:0/2861851298 shutdown_connections 2024-09-19T06:58:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.131+0000 7ff5bac18740 1 -- 172.21.15.103:0/2861851298 wait complete. 2024-09-19T06:58:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.132+0000 7ff5bac18740 1 -- start start 2024-09-19T06:58:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.132+0000 7ff5bac18740 1 Processor -- start 2024-09-19T06:58:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.132+0000 7ff5bac18740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.132+0000 7ff5bac18740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a5410 0x55ff7b8b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.132+0000 7ff5bac18740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.132+0000 7ff5bac18740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01aecc0 con 0x55ff7b8afcb0 2024-09-19T06:58:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.132+0000 7ff5bac18740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01ae5f0 con 0x55ff7b8a4af0 2024-09-19T06:58:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.132+0000 7ff5bac18740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01ab850 con 0x55ff7b8a5410 2024-09-19T06:58:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.132+0000 7ff5bac18740 10 client.0 _ll_register_callbacks cb 0x7ff5a00127e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.132+0000 7ff5b5416640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.132+0000 7ff5b5416640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53782/0 (socket says 172.21.15.103:53782) 2024-09-19T06:58:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.132+0000 7ff5b5416640 1 -- 172.21.15.103:0/3791348497 learned_addr learned my addr 172.21.15.103:0/3791348497 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.133+0000 7ff5b5416640 1 --2- 172.21.15.103:0/3791348497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.133+0000 7ff5b5c17640 1 --2- 172.21.15.103:0/3791348497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a5410 0x55ff7b8b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.133+0000 7ff5b5416640 1 -- 172.21.15.103:0/3791348497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a5410 msgr2=0x55ff7b8b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.133+0000 7ff5b5416640 1 --2- 172.21.15.103:0/3791348497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a5410 0x55ff7b8b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.133+0000 7ff5b5416640 1 -- 172.21.15.103:0/3791348497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8a4af0 msgr2=0x55ff7b8b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.133+0000 7ff5b5416640 1 --2- 172.21.15.103:0/3791348497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.133+0000 7ff5b5416640 1 -- 172.21.15.103:0/3791348497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff7b8b4500 con 0x55ff7b8afcb0 2024-09-19T06:58:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.133+0000 7ff5b5c17640 1 --2- 172.21.15.103:0/3791348497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a5410 0x55ff7b8b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.133+0000 7ff5b5416640 1 --2- 172.21.15.103:0/3791348497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8b3cd0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00124f0 tx=0x7ff5a00122d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa359187bd322f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.133+0000 7ff5997fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.133+0000 7ff5997fa640 1 -- 172.21.15.103:0/3791348497 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0003d50 con 0x55ff7b8afcb0 2024-09-19T06:58:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.133+0000 7ff5997fa640 1 -- 172.21.15.103:0/3791348497 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7ff5a0039040 con 0x55ff7b8afcb0 2024-09-19T06:58:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.133+0000 7ff5bac18740 1 -- 172.21.15.103:0/3791348497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a0003860 con 0x55ff7b8afcb0 2024-09-19T06:58:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.133+0000 7ff5997fa640 1 -- 172.21.15.103:0/3791348497 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5a0003f80 con 0x55ff7b8afcb0 2024-09-19T06:58:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.133+0000 7ff5bac18740 10 client.18714 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.133+0000 7ff5bac18740 1 -- 172.21.15.103:0/3791348497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ff7b92a820 con 0x55ff7b8afcb0 2024-09-19T06:58:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.134+0000 7ff5997fa640 1 -- 172.21.15.103:0/3791348497 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a0003860 con 0x55ff7b8afcb0 2024-09-19T06:58:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.134+0000 7ff5997fa640 1 -- 172.21.15.103:0/3791348497 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff5a003c020 con 0x55ff7b8afcb0 2024-09-19T06:58:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.134+0000 7ff5bac18740 10 client.18714 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.134+0000 7ff5bac18740 10 client.18714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:58:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.134+0000 7ff5bac18740 20 client.18714 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.134+0000 7ff5bac18740 1 --2- 172.21.15.103:0/3791348497 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ff7b92ae80 0x55ff7b92d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.134+0000 7ff5bac18740 4 client.18714 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:58:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.134+0000 7ff5bac18740 1 -- 172.21.15.103:0/3791348497 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ff7b810d70 con 0x55ff7b92ae80 2024-09-19T06:58:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.134+0000 7ff5b5c17640 1 --2- 172.21.15.103:0/3791348497 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ff7b92ae80 0x55ff7b92d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.134+0000 7ff5997fa640 10 client.18714 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.134+0000 7ff5b5c17640 1 --2- 172.21.15.103:0/3791348497 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ff7b92ae80 0x55ff7b92d270 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428acea543df758c server_cookie=f06c77db80b3c0b3 in_seq=0 out_seq=0 2024-09-19T06:58:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.147+0000 7ff5997fa640 1 -- 172.21.15.103:0/3791348497 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ff7b810d70 con 0x55ff7b92ae80 2024-09-19T06:58:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.147+0000 7ff5997fa640 10 client.18714 handle_command_reply: tid=0 2024-09-19T06:58:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.147+0000 7ff5bac18740 1 client.18714 shutdown 2024-09-19T06:58:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.147+0000 7ff5bac18740 1 -- 172.21.15.103:0/3791348497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8afcb0 msgr2=0x55ff7b8b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.147+0000 7ff5bac18740 1 --2- 172.21.15.103:0/3791348497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8b3cd0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00124f0 tx=0x7ff5a00122d0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.147+0000 7ff5bac18740 1 -- 172.21.15.103:0/3791348497 shutdown_connections 2024-09-19T06:58:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.147+0000 7ff5bac18740 1 --2- 172.21.15.103:0/3791348497 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ff7b92ae80 0x55ff7b92d270 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.147+0000 7ff5bac18740 1 --2- 172.21.15.103:0/3791348497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff7b8afcb0 0x55ff7b8b3cd0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.147+0000 7ff5bac18740 1 --2- 172.21.15.103:0/3791348497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff7b8a5410 0x55ff7b8b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.147+0000 7ff5997fa640 0 client.18714 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.147+0000 7ff5bac18740 1 --2- 172.21.15.103:0/3791348497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff7b8a4af0 0x55ff7b8b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.147+0000 7ff5bac18740 1 -- 172.21.15.103:0/3791348497 >> 172.21.15.103:0/3791348497 conn(0x55ff7b8a1060 msgr2=0x55ff7b89f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.147+0000 7ff5bac18740 1 -- 172.21.15.103:0/3791348497 shutdown_connections 2024-09-19T06:58:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.147+0000 7ff5bac18740 1 -- 172.21.15.103:0/3791348497 wait complete. 2024-09-19T06:58:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.147+0000 7ff5bac18740 20 client.18714 trim_cache size 0 max 16384 2024-09-19T06:58:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:58:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:58:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:58:37.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:58:37.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.148+0000 7ff5b7e7b640 1 -- 172.21.15.103:0/1934779448 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff59c0a34e0 msgr2=0x7ff59c0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:37.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.148+0000 7ff5b7e7b640 1 --2- 172.21.15.103:0/1934779448 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff59c0a34e0 0x7ff59c0a59a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80cd650 tx=0x7ff5a8091120 comp rx=0 tx=0).stop 2024-09-19T06:58:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.148+0000 7ff5b7e7b640 1 -- 172.21.15.103:0/1934779448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b01069e0 msgr2=0x7ff5b01a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.148+0000 7ff5b7e7b640 1 --2- 172.21.15.103:0/1934779448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b01069e0 0x7ff5b01a4270 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009fd0 tx=0x7ff5a800b260 comp rx=0 tx=0).stop 2024-09-19T06:58:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.148+0000 7ff5b7e7b640 1 -- 172.21.15.103:0/1934779448 shutdown_connections 2024-09-19T06:58:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.148+0000 7ff5b7e7b640 1 --2- 172.21.15.103:0/1934779448 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff59c0a34e0 0x7ff59c0a59a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.148+0000 7ff5b7e7b640 1 --2- 172.21.15.103:0/1934779448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b01069e0 0x7ff5b01a4270 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.148+0000 7ff5b7e7b640 1 --2- 172.21.15.103:0/1934779448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5b0101370 0x7ff5b0199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.148+0000 7ff5b7e7b640 1 --2- 172.21.15.103:0/1934779448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5b01009d0 0x7ff5b01991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.148+0000 7ff5b7e7b640 1 -- 172.21.15.103:0/1934779448 >> 172.21.15.103:0/1934779448 conn(0x7ff5b00fc6b0 msgr2=0x7ff5b00fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.148+0000 7ff5b7e7b640 1 -- 172.21.15.103:0/1934779448 shutdown_connections 2024-09-19T06:58:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:37.148+0000 7ff5b7e7b640 1 -- 172.21.15.103:0/1934779448 wait complete. 2024-09-19T06:58:37.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:37 smithi174 ceph-mon[35637]: pgmap v290: 97 pgs: 97 active+clean; 4.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-19T06:58:37.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:37 smithi103 ceph-mon[31158]: pgmap v290: 97 pgs: 97 active+clean; 4.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-19T06:58:37.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:37 smithi134 ceph-mon[35356]: pgmap v290: 97 pgs: 97 active+clean; 4.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-19T06:58:39.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:39 smithi174 ceph-mon[35637]: pgmap v291: 97 pgs: 97 active+clean; 4.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-09-19T06:58:39.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:39 smithi103 ceph-mon[31158]: pgmap v291: 97 pgs: 97 active+clean; 4.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-09-19T06:58:39.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:39 smithi134 ceph-mon[35356]: pgmap v291: 97 pgs: 97 active+clean; 4.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-09-19T06:58:40.950 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.950 INFO:tasks.workunit.client.0.smithi103.stdout:251/600: open fd = 3 2024-09-19T06:58:40.950 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.950 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.950 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.950 INFO:tasks.workunit.client.0.smithi103.stdout:252/600: open fd = 3 2024-09-19T06:58:40.950 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.950 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.950 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.950 INFO:tasks.workunit.client.0.smithi103.stdout:253/600: open fd = 3 2024-09-19T06:58:40.950 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.951 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.951 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.951 INFO:tasks.workunit.client.0.smithi103.stdout:254/600: open fd = 3 2024-09-19T06:58:40.951 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.951 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.951 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.951 INFO:tasks.workunit.client.0.smithi103.stdout:255/600: open fd = 3 2024-09-19T06:58:40.951 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.951 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.951 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.951 INFO:tasks.workunit.client.0.smithi103.stdout:256/600: open fd = 3 2024-09-19T06:58:40.951 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.951 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.951 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.952 INFO:tasks.workunit.client.0.smithi103.stdout:257/600: open fd = 3 2024-09-19T06:58:40.952 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.952 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.952 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.952 INFO:tasks.workunit.client.0.smithi103.stdout:258/600: open fd = 3 2024-09-19T06:58:40.952 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.952 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.952 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.952 INFO:tasks.workunit.client.0.smithi103.stdout:259/600: open fd = 3 2024-09-19T06:58:40.952 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.952 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.952 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.952 INFO:tasks.workunit.client.0.smithi103.stdout:260/600: open fd = 3 2024-09-19T06:58:40.952 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.953 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.953 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.953 INFO:tasks.workunit.client.0.smithi103.stdout:261/600: open fd = 3 2024-09-19T06:58:40.953 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.953 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.953 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.953 INFO:tasks.workunit.client.0.smithi103.stdout:262/600: open fd = 3 2024-09-19T06:58:40.953 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.953 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.953 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.953 INFO:tasks.workunit.client.0.smithi103.stdout:263/600: open fd = 3 2024-09-19T06:58:40.953 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.953 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.954 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.954 INFO:tasks.workunit.client.0.smithi103.stdout:264/600: open fd = 3 2024-09-19T06:58:40.954 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.954 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.954 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.954 INFO:tasks.workunit.client.0.smithi103.stdout:265/600: open fd = 3 2024-09-19T06:58:40.954 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.954 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.954 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.954 INFO:tasks.workunit.client.0.smithi103.stdout:266/600: open fd = 3 2024-09-19T06:58:40.954 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.955 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.955 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.955 INFO:tasks.workunit.client.0.smithi103.stdout:267/600: open fd = 3 2024-09-19T06:58:40.955 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.955 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.955 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.955 INFO:tasks.workunit.client.0.smithi103.stdout:268/600: open fd = 3 2024-09-19T06:58:40.955 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.955 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.955 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.955 INFO:tasks.workunit.client.0.smithi103.stdout:269/600: open fd = 3 2024-09-19T06:58:40.955 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.955 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.955 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.955 INFO:tasks.workunit.client.0.smithi103.stdout:270/600: open fd = 3 2024-09-19T06:58:40.956 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.956 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.956 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.956 INFO:tasks.workunit.client.0.smithi103.stdout:271/600: open fd = 3 2024-09-19T06:58:40.956 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.956 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.956 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.956 INFO:tasks.workunit.client.0.smithi103.stdout:272/600: open fd = 3 2024-09-19T06:58:40.956 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.956 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.956 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.956 INFO:tasks.workunit.client.0.smithi103.stdout:273/600: open fd = 3 2024-09-19T06:58:40.956 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.956 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.957 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.957 INFO:tasks.workunit.client.0.smithi103.stdout:274/600: open fd = 3 2024-09-19T06:58:40.957 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.957 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.957 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.957 INFO:tasks.workunit.client.0.smithi103.stdout:275/600: open fd = 3 2024-09-19T06:58:40.957 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.957 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.957 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.957 INFO:tasks.workunit.client.0.smithi103.stdout:276/600: open fd = 3 2024-09-19T06:58:40.957 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.957 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.957 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.957 INFO:tasks.workunit.client.0.smithi103.stdout:277/600: open fd = 3 2024-09-19T06:58:40.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.958 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.958 INFO:tasks.workunit.client.0.smithi103.stdout:278/600: open fd = 3 2024-09-19T06:58:40.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.958 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.958 INFO:tasks.workunit.client.0.smithi103.stdout:279/600: open fd = 3 2024-09-19T06:58:40.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.958 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.958 INFO:tasks.workunit.client.0.smithi103.stdout:280/600: open fd = 3 2024-09-19T06:58:40.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.958 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.959 INFO:tasks.workunit.client.0.smithi103.stdout:281/600: open fd = 3 2024-09-19T06:58:40.959 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.959 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.959 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.959 INFO:tasks.workunit.client.0.smithi103.stdout:282/600: open fd = 3 2024-09-19T06:58:40.959 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.959 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.959 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.959 INFO:tasks.workunit.client.0.smithi103.stdout:283/600: open fd = 3 2024-09-19T06:58:40.959 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.959 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.959 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.959 INFO:tasks.workunit.client.0.smithi103.stdout:284/600: open fd = 3 2024-09-19T06:58:40.959 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.960 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.960 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.960 INFO:tasks.workunit.client.0.smithi103.stdout:285/600: open fd = 3 2024-09-19T06:58:40.960 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.960 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.960 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.960 INFO:tasks.workunit.client.0.smithi103.stdout:286/600: open fd = 3 2024-09-19T06:58:40.960 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.960 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.960 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.960 INFO:tasks.workunit.client.0.smithi103.stdout:287/600: open fd = 3 2024-09-19T06:58:40.960 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.960 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.960 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.961 INFO:tasks.workunit.client.0.smithi103.stdout:288/600: open fd = 3 2024-09-19T06:58:40.961 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.961 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.961 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.961 INFO:tasks.workunit.client.0.smithi103.stdout:289/600: open fd = 3 2024-09-19T06:58:40.961 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.961 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.961 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.961 INFO:tasks.workunit.client.0.smithi103.stdout:290/600: open fd = 3 2024-09-19T06:58:40.961 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.961 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.961 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.961 INFO:tasks.workunit.client.0.smithi103.stdout:291/600: open fd = 3 2024-09-19T06:58:40.961 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.962 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.962 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.962 INFO:tasks.workunit.client.0.smithi103.stdout:292/600: open fd = 3 2024-09-19T06:58:40.962 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.962 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.962 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.962 INFO:tasks.workunit.client.0.smithi103.stdout:293/600: open fd = 3 2024-09-19T06:58:40.962 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.962 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.962 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.962 INFO:tasks.workunit.client.0.smithi103.stdout:294/600: open fd = 3 2024-09-19T06:58:40.962 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.962 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.962 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.962 INFO:tasks.workunit.client.0.smithi103.stdout:295/600: open fd = 3 2024-09-19T06:58:40.963 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.963 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.963 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.963 INFO:tasks.workunit.client.0.smithi103.stdout:296/600: open fd = 3 2024-09-19T06:58:40.963 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.963 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.963 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.963 INFO:tasks.workunit.client.0.smithi103.stdout:297/600: open fd = 3 2024-09-19T06:58:40.963 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.963 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.963 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.963 INFO:tasks.workunit.client.0.smithi103.stdout:298/600: open fd = 3 2024-09-19T06:58:40.963 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.963 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.964 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.964 INFO:tasks.workunit.client.0.smithi103.stdout:299/600: open fd = 3 2024-09-19T06:58:40.964 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.964 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.964 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.964 INFO:tasks.workunit.client.0.smithi103.stdout:300/600: open fd = 3 2024-09-19T06:58:40.964 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.964 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.964 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.964 INFO:tasks.workunit.client.0.smithi103.stdout:301/600: open fd = 3 2024-09-19T06:58:40.964 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.964 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.964 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.964 INFO:tasks.workunit.client.0.smithi103.stdout:302/600: open fd = 3 2024-09-19T06:58:40.965 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.965 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.965 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.965 INFO:tasks.workunit.client.0.smithi103.stdout:303/600: open fd = 3 2024-09-19T06:58:40.965 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.965 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.965 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.965 INFO:tasks.workunit.client.0.smithi103.stdout:304/600: open fd = 3 2024-09-19T06:58:40.965 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.965 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.965 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.965 INFO:tasks.workunit.client.0.smithi103.stdout:305/600: open fd = 3 2024-09-19T06:58:40.965 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.965 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.965 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.966 INFO:tasks.workunit.client.0.smithi103.stdout:306/600: open fd = 3 2024-09-19T06:58:40.966 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.966 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.966 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.966 INFO:tasks.workunit.client.0.smithi103.stdout:307/600: open fd = 3 2024-09-19T06:58:40.966 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.966 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.966 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.966 INFO:tasks.workunit.client.0.smithi103.stdout:308/600: open fd = 3 2024-09-19T06:58:40.966 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.966 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.966 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.966 INFO:tasks.workunit.client.0.smithi103.stdout:309/600: open fd = 3 2024-09-19T06:58:40.967 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.967 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.967 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.967 INFO:tasks.workunit.client.0.smithi103.stdout:310/600: open fd = 3 2024-09-19T06:58:40.967 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.967 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.967 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.967 INFO:tasks.workunit.client.0.smithi103.stdout:311/600: open fd = 3 2024-09-19T06:58:40.967 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.967 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.967 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T06:58:40.967 INFO:tasks.workunit.client.0.smithi103.stdout:312/600: open fd = 3 2024-09-19T06:58:40.967 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:40.968 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T06:58:41.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:41 smithi103 ceph-mon[31158]: pgmap v292: 97 pgs: 97 active+clean; 4.6 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-19T06:58:41.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:41 smithi134 ceph-mon[35356]: pgmap v292: 97 pgs: 97 active+clean; 4.6 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-19T06:58:41.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:41 smithi174 ceph-mon[35637]: pgmap v292: 97 pgs: 97 active+clean; 4.6 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-19T06:58:43.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:43 smithi103 ceph-mon[31158]: pgmap v293: 97 pgs: 97 active+clean; 4.6 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T06:58:43.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:43 smithi134 ceph-mon[35356]: pgmap v293: 97 pgs: 97 active+clean; 4.6 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T06:58:44.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:43 smithi174 ceph-mon[35637]: pgmap v293: 97 pgs: 97 active+clean; 4.6 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T06:58:45.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:45 smithi103 ceph-mon[31158]: pgmap v294: 97 pgs: 97 active+clean; 4.6 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T06:58:45.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:45 smithi134 ceph-mon[35356]: pgmap v294: 97 pgs: 97 active+clean; 4.6 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T06:58:46.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:45 smithi174 ceph-mon[35637]: pgmap v294: 97 pgs: 97 active+clean; 4.6 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T06:58:47.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:47 smithi103 ceph-mon[31158]: pgmap v295: 97 pgs: 97 active+clean; 4.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-19T06:58:47.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:47 smithi134 ceph-mon[35356]: pgmap v295: 97 pgs: 97 active+clean; 4.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-19T06:58:48.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:47 smithi174 ceph-mon[35637]: pgmap v295: 97 pgs: 97 active+clean; 4.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-19T06:58:49.885 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:49 smithi174 ceph-mon[35637]: pgmap v296: 97 pgs: 97 active+clean; 4.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T06:58:49.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:49 smithi103 ceph-mon[31158]: pgmap v296: 97 pgs: 97 active+clean; 4.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T06:58:49.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:49 smithi134 ceph-mon[35356]: pgmap v296: 97 pgs: 97 active+clean; 4.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T06:58:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.242+0000 7ff13136d640 1 -- 172.21.15.103:0/238065253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff12c101370 msgr2=0x7ff12c10d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.242+0000 7ff13136d640 1 --2- 172.21.15.103:0/238065253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff12c101370 0x7ff12c10d8a0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff124009cd0 tx=0x7ff12402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.242+0000 7ff13136d640 1 -- 172.21.15.103:0/238065253 shutdown_connections 2024-09-19T06:58:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.242+0000 7ff13136d640 1 --2- 172.21.15.103:0/238065253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff12c101370 0x7ff12c10d8a0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.242+0000 7ff13136d640 1 --2- 172.21.15.103:0/238065253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff12c1009d0 0x7ff12c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.242+0000 7ff13136d640 1 --2- 172.21.15.103:0/238065253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff12c1069e0 0x7ff12c106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.243+0000 7ff13136d640 1 -- 172.21.15.103:0/238065253 >> 172.21.15.103:0/238065253 conn(0x7ff12c0fc6f0 msgr2=0x7ff12c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.243+0000 7ff13136d640 1 -- 172.21.15.103:0/238065253 shutdown_connections 2024-09-19T06:58:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.243+0000 7ff13136d640 1 -- 172.21.15.103:0/238065253 wait complete. 2024-09-19T06:58:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.243+0000 7ff13136d640 1 Processor -- start 2024-09-19T06:58:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.243+0000 7ff13136d640 1 -- start start 2024-09-19T06:58:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff13136d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff12c1009d0 0x7ff12c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff13136d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff12c101370 0x7ff12c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff13136d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff12c1069e0 0x7ff12c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff13136d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c111f60 con 0x7ff12c101370 2024-09-19T06:58:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff13136d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c111de0 con 0x7ff12c1069e0 2024-09-19T06:58:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff13136d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c1120e0 con 0x7ff12c1009d0 2024-09-19T06:58:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff1227fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff12c101370 0x7ff12c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff1227fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff12c101370 0x7ff12c1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33786/0 (socket says 172.21.15.103:33786) 2024-09-19T06:58:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff1227fc640 1 -- 172.21.15.103:0/3570997232 learned_addr learned my addr 172.21.15.103:0/3570997232 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff12b7fe640 1 --2- 172.21.15.103:0/3570997232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff12c1069e0 0x7ff12c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff12affd640 1 --2- 172.21.15.103:0/3570997232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff12c1009d0 0x7ff12c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff1227fc640 1 -- 172.21.15.103:0/3570997232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff12c1009d0 msgr2=0x7ff12c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff1227fc640 1 --2- 172.21.15.103:0/3570997232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff12c1009d0 0x7ff12c1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff1227fc640 1 -- 172.21.15.103:0/3570997232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff12c1069e0 msgr2=0x7ff12c19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff1227fc640 1 --2- 172.21.15.103:0/3570997232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff12c1069e0 0x7ff12c19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff1227fc640 1 -- 172.21.15.103:0/3570997232 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c19d2a0 con 0x7ff12c101370 2024-09-19T06:58:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff1227fc640 1 --2- 172.21.15.103:0/3570997232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff12c101370 0x7ff12c1987f0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff11400c920 tx=0x7ff11400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.244+0000 7ff128ff9640 1 -- 172.21.15.103:0/3570997232 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114007c70 con 0x7ff12c101370 2024-09-19T06:58:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.245+0000 7ff128ff9640 1 -- 172.21.15.103:0/3570997232 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff114007e10 con 0x7ff12c101370 2024-09-19T06:58:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.245+0000 7ff13136d640 1 -- 172.21.15.103:0/3570997232 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12c19d590 con 0x7ff12c101370 2024-09-19T06:58:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.245+0000 7ff128ff9640 1 -- 172.21.15.103:0/3570997232 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114018890 con 0x7ff12c101370 2024-09-19T06:58:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.245+0000 7ff13136d640 1 -- 172.21.15.103:0/3570997232 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12c1a4e20 con 0x7ff12c101370 2024-09-19T06:58:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.247+0000 7ff128ff9640 1 -- 172.21.15.103:0/3570997232 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff114018a70 con 0x7ff12c101370 2024-09-19T06:58:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.248+0000 7ff1341ea740 1 Processor -- start 2024-09-19T06:58:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.248+0000 7ff1341ea740 1 -- start start 2024-09-19T06:58:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.248+0000 7ff1341ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6e99d0 0x5602cd6e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.248+0000 7ff1341ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6ea2f0 0x5602cd6f8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.248+0000 7ff1341ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6f90c0 0x5602cd6fb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.248+0000 7ff1341ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5602cd64e2f0 con 0x5602cd6f90c0 2024-09-19T06:58:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.248+0000 7ff1341ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5602cd5f6270 con 0x5602cd6ea2f0 2024-09-19T06:58:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.248+0000 7ff1341ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5602cd64a2b0 con 0x5602cd6e99d0 2024-09-19T06:58:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.248+0000 7ff1227fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6f90c0 0x5602cd6fb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.248+0000 7ff1227fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6f90c0 0x5602cd6fb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33796/0 (socket says 172.21.15.103:33796) 2024-09-19T06:58:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.248+0000 7ff1227fc640 1 -- 172.21.15.103:0/2058142370 learned_addr learned my addr 172.21.15.103:0/2058142370 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.248+0000 7ff12b7fe640 1 --2- 172.21.15.103:0/2058142370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6e99d0 0x5602cd6e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.248+0000 7ff12affd640 1 --2- 172.21.15.103:0/2058142370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6ea2f0 0x5602cd6f8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.249+0000 7ff128ff9640 1 --2- 172.21.15.103:0/3570997232 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1180a3460 0x7ff1180a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.249+0000 7ff1227fc640 1 -- 172.21.15.103:0/2058142370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6e99d0 msgr2=0x5602cd6e9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.249+0000 7ff1227fc640 1 --2- 172.21.15.103:0/2058142370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6e99d0 0x5602cd6e9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.249+0000 7ff1227fc640 1 -- 172.21.15.103:0/2058142370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6ea2f0 msgr2=0x5602cd6f8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.249+0000 7ff1227fc640 1 --2- 172.21.15.103:0/2058142370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6ea2f0 0x5602cd6f8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.249+0000 7ff1227fc640 1 -- 172.21.15.103:0/2058142370 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602cd5f5340 con 0x5602cd6f90c0 2024-09-19T06:58:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.249+0000 7ff12b7fe640 1 --2- 172.21.15.103:0/3570997232 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1180a3460 0x7ff1180a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.249+0000 7ff128ff9640 1 -- 172.21.15.103:0/3570997232 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7ff1140cfde0 con 0x7ff12c101370 2024-09-19T06:58:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.249+0000 7ff1227fc640 1 --2- 172.21.15.103:0/2058142370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6f90c0 0x5602cd6fb4b0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7ff1140cd450 tx=0x7ff114066a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7dc47abde5d8c19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.249+0000 7ff1217fa640 1 -- 172.21.15.103:0/2058142370 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11409ba00 con 0x5602cd6f90c0 2024-09-19T06:58:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.249+0000 7ff12b7fe640 1 --2- 172.21.15.103:0/3570997232 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1180a3460 0x7ff1180a5920 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff124005e30 tx=0x7ff1240095c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.249+0000 7ff1217fa640 1 -- 172.21.15.103:0/2058142370 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1140893d0 con 0x5602cd6f90c0 2024-09-19T06:58:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.249+0000 7ff1217fa640 1 -- 172.21.15.103:0/2058142370 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1140891d0 con 0x5602cd6f90c0 2024-09-19T06:58:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.250+0000 7ff1341ea740 1 -- 172.21.15.103:0/2058142370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6f90c0 msgr2=0x5602cd6fb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.250+0000 7ff1341ea740 1 --2- 172.21.15.103:0/2058142370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6f90c0 0x5602cd6fb4b0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7ff1140cd450 tx=0x7ff114066a90 comp rx=0 tx=0).stop 2024-09-19T06:58:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.250+0000 7ff1341ea740 1 -- 172.21.15.103:0/2058142370 shutdown_connections 2024-09-19T06:58:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.250+0000 7ff1341ea740 1 --2- 172.21.15.103:0/2058142370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6f90c0 0x5602cd6fb4b0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.250+0000 7ff1341ea740 1 --2- 172.21.15.103:0/2058142370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6ea2f0 0x5602cd6f8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.250+0000 7ff1341ea740 1 --2- 172.21.15.103:0/2058142370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6e99d0 0x5602cd6e9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.250+0000 7ff1341ea740 1 -- 172.21.15.103:0/2058142370 >> 172.21.15.103:0/2058142370 conn(0x5602cd6e6060 msgr2=0x5602cd6e6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.250+0000 7ff1341ea740 1 -- 172.21.15.103:0/2058142370 shutdown_connections 2024-09-19T06:58:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.250+0000 7ff1341ea740 1 -- 172.21.15.103:0/2058142370 wait complete. 2024-09-19T06:58:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff1341ea740 1 -- start start 2024-09-19T06:58:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff1341ea740 1 Processor -- start 2024-09-19T06:58:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff1341ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 0x5602cd6f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff1341ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6ea2f0 0x5602cd6f6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff1341ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6f90c0 0x5602cd6f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff1341ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5602cd6fbc90 con 0x5602cd6f90c0 2024-09-19T06:58:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff1341ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5602cd5d7290 con 0x5602cd6e99d0 2024-09-19T06:58:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff1341ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5602cd6fbe10 con 0x5602cd6ea2f0 2024-09-19T06:58:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff1341ea740 10 client.0 _ll_register_callbacks cb 0x5602cd608140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff12b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6f90c0 0x5602cd6f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff12b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6f90c0 0x5602cd6f6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33800/0 (socket says 172.21.15.103:33800) 2024-09-19T06:58:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff12b7fe640 1 -- 172.21.15.103:0/2257104246 learned_addr learned my addr 172.21.15.103:0/2257104246 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff1227fc640 1 --2- 172.21.15.103:0/2257104246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6ea2f0 0x5602cd6f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff12affd640 1 --2- 172.21.15.103:0/2257104246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 0x5602cd6f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff12b7fe640 1 -- 172.21.15.103:0/2257104246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6ea2f0 msgr2=0x5602cd6f6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff12b7fe640 1 --2- 172.21.15.103:0/2257104246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6ea2f0 0x5602cd6f6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff12b7fe640 1 -- 172.21.15.103:0/2257104246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 msgr2=0x5602cd6f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff12b7fe640 1 --2- 172.21.15.103:0/2257104246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 0x5602cd6f5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff12b7fe640 1 -- 172.21.15.103:0/2257104246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602cd361210 con 0x5602cd6f90c0 2024-09-19T06:58:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff1227fc640 1 --2- 172.21.15.103:0/2257104246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6ea2f0 0x5602cd6f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff12b7fe640 1 --2- 172.21.15.103:0/2257104246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6f90c0 0x5602cd6f6a40 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7ff124035000 tx=0x7ff12402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ba0a382f04556c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.251+0000 7ff1127fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.252+0000 7ff1127fc640 1 -- 172.21.15.103:0/2257104246 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240305a0 con 0x5602cd6f90c0 2024-09-19T06:58:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.252+0000 7ff1127fc640 1 -- 172.21.15.103:0/2257104246 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7ff124030740 con 0x5602cd6f90c0 2024-09-19T06:58:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.252+0000 7ff1341ea740 1 -- 172.21.15.103:0/2257104246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5602cd65ba20 con 0x5602cd6f90c0 2024-09-19T06:58:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.252+0000 7ff1127fc640 1 -- 172.21.15.103:0/2257104246 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff124030940 con 0x5602cd6f90c0 2024-09-19T06:58:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.252+0000 7ff1341ea740 10 client.18726 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.252+0000 7ff1341ea740 1 -- 172.21.15.103:0/2257104246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5602cd42d620 con 0x5602cd6f90c0 2024-09-19T06:58:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.252+0000 7ff1127fc640 1 -- 172.21.15.103:0/2257104246 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5602cd65ba20 con 0x5602cd6f90c0 2024-09-19T06:58:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.252+0000 7ff1127fc640 1 -- 172.21.15.103:0/2257104246 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff124050070 con 0x5602cd6f90c0 2024-09-19T06:58:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.252+0000 7ff1341ea740 10 client.18726 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.252+0000 7ff1341ea740 10 client.18726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:58:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.252+0000 7ff1341ea740 20 client.18726 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.252+0000 7ff1341ea740 1 --2- 172.21.15.103:0/2257104246 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5602cd709000 0x5602cd70b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.252+0000 7ff1341ea740 4 client.18726 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:58:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.252+0000 7ff1341ea740 1 -- 172.21.15.103:0/2257104246 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602cd655d70 con 0x5602cd709000 2024-09-19T06:58:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.253+0000 7ff12affd640 1 --2- 172.21.15.103:0/2257104246 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5602cd709000 0x5602cd70b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.253+0000 7ff1127fc640 10 client.18726 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.253+0000 7ff12affd640 1 --2- 172.21.15.103:0/2257104246 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5602cd709000 0x5602cd70b3f0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2605f98c8c6db26a server_cookie=73fe235c93904d76 in_seq=0 out_seq=0 2024-09-19T06:58:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.254+0000 7ff1127fc640 1 -- 172.21.15.103:0/2257104246 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602cd655d70 con 0x5602cd709000 2024-09-19T06:58:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.254+0000 7ff1127fc640 10 client.18726 handle_command_reply: tid=0 2024-09-19T06:58:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.254+0000 7ff1341ea740 1 client.18726 shutdown 2024-09-19T06:58:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.254+0000 7ff1341ea740 1 -- 172.21.15.103:0/2257104246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6f90c0 msgr2=0x5602cd6f6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.254+0000 7ff1341ea740 1 --2- 172.21.15.103:0/2257104246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6f90c0 0x5602cd6f6a40 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7ff124035000 tx=0x7ff12402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.254+0000 7ff1341ea740 1 -- 172.21.15.103:0/2257104246 shutdown_connections 2024-09-19T06:58:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.254+0000 7ff1341ea740 1 --2- 172.21.15.103:0/2257104246 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5602cd709000 0x5602cd70b3f0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.254+0000 7ff1341ea740 1 --2- 172.21.15.103:0/2257104246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6f90c0 0x5602cd6f6a40 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.254+0000 7ff1341ea740 1 --2- 172.21.15.103:0/2257104246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6ea2f0 0x5602cd6f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.254+0000 7ff1127fc640 0 client.18726 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.254+0000 7ff1341ea740 1 --2- 172.21.15.103:0/2257104246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 0x5602cd6f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.254+0000 7ff1341ea740 1 -- 172.21.15.103:0/2257104246 >> 172.21.15.103:0/2257104246 conn(0x5602cd6e6060 msgr2=0x5602cd6ebdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.254+0000 7ff1341ea740 1 -- 172.21.15.103:0/2257104246 shutdown_connections 2024-09-19T06:58:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.254+0000 7ff1341ea740 1 -- 172.21.15.103:0/2257104246 wait complete. 2024-09-19T06:58:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.254+0000 7ff1341ea740 20 client.18726 trim_cache size 0 max 16384 2024-09-19T06:58:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.268+0000 7ff1341ea740 1 Processor -- start 2024-09-19T06:58:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.268+0000 7ff1341ea740 1 -- start start 2024-09-19T06:58:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.268+0000 7ff1341ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 0x5602cd6f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff1341ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6ea2f0 0x5602cd710ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff1341ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6f90c0 0x5602cd711220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff1341ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c1ab5d0 con 0x5602cd6ea2f0 2024-09-19T06:58:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff1341ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c1ad110 con 0x5602cd6e99d0 2024-09-19T06:58:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff1341ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c1abca0 con 0x5602cd6f90c0 2024-09-19T06:58:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff1227fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6ea2f0 0x5602cd710ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff1227fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6ea2f0 0x5602cd710ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33804/0 (socket says 172.21.15.103:33804) 2024-09-19T06:58:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff1227fc640 1 -- 172.21.15.103:0/1065353280 learned_addr learned my addr 172.21.15.103:0/1065353280 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff12affd640 1 --2- 172.21.15.103:0/1065353280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 0x5602cd6f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff1227fc640 1 -- 172.21.15.103:0/1065353280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6f90c0 msgr2=0x5602cd711220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff1227fc640 1 --2- 172.21.15.103:0/1065353280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6f90c0 0x5602cd711220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff1227fc640 1 -- 172.21.15.103:0/1065353280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 msgr2=0x5602cd6f7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff1227fc640 1 --2- 172.21.15.103:0/1065353280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 0x5602cd6f7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff1227fc640 1 -- 172.21.15.103:0/1065353280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602cd70ba50 con 0x5602cd6ea2f0 2024-09-19T06:58:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff12affd640 1 --2- 172.21.15.103:0/1065353280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 0x5602cd6f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff1227fc640 1 --2- 172.21.15.103:0/1065353280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6ea2f0 0x5602cd710ce0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7ff114007590 tx=0x7ff1140cd4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61f633474d9f6cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff120ff9640 1 -- 172.21.15.103:0/1065353280 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11409c7a0 con 0x5602cd6ea2f0 2024-09-19T06:58:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff120ff9640 1 -- 172.21.15.103:0/1065353280 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff11406f3d0 con 0x5602cd6ea2f0 2024-09-19T06:58:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.269+0000 7ff120ff9640 1 -- 172.21.15.103:0/1065353280 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11406d0b0 con 0x5602cd6ea2f0 2024-09-19T06:58:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.270+0000 7ff1341ea740 1 -- 172.21.15.103:0/1065353280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6ea2f0 msgr2=0x5602cd710ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.270+0000 7ff1341ea740 1 --2- 172.21.15.103:0/1065353280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6ea2f0 0x5602cd710ce0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7ff114007590 tx=0x7ff1140cd4b0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.270+0000 7ff1341ea740 1 -- 172.21.15.103:0/1065353280 shutdown_connections 2024-09-19T06:58:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.270+0000 7ff1341ea740 1 --2- 172.21.15.103:0/1065353280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6f90c0 0x5602cd711220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.270+0000 7ff1341ea740 1 --2- 172.21.15.103:0/1065353280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6ea2f0 0x5602cd710ce0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.270+0000 7ff1341ea740 1 --2- 172.21.15.103:0/1065353280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 0x5602cd6f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.270+0000 7ff1341ea740 1 -- 172.21.15.103:0/1065353280 >> 172.21.15.103:0/1065353280 conn(0x5602cd6e6060 msgr2=0x5602cd6f7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.270+0000 7ff1341ea740 1 -- 172.21.15.103:0/1065353280 shutdown_connections 2024-09-19T06:58:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.270+0000 7ff1341ea740 1 -- 172.21.15.103:0/1065353280 wait complete. 2024-09-19T06:58:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1341ea740 1 -- start start 2024-09-19T06:58:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1341ea740 1 Processor -- start 2024-09-19T06:58:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1341ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 0x5602cd7015f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1341ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6ea2f0 0x5602cd6f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1341ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6f90c0 0x5602cd6f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1341ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c1aca40 con 0x5602cd6ea2f0 2024-09-19T06:58:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1341ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c1ac370 con 0x5602cd6e99d0 2024-09-19T06:58:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1341ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c1a95f0 con 0x5602cd6f90c0 2024-09-19T06:58:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1341ea740 10 client.0 _ll_register_callbacks cb 0x7ff124009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:58:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1227fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6ea2f0 0x5602cd6f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1227fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6ea2f0 0x5602cd6f2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33818/0 (socket says 172.21.15.103:33818) 2024-09-19T06:58:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1227fc640 1 -- 172.21.15.103:0/1782158524 learned_addr learned my addr 172.21.15.103:0/1782158524 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:58:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff12affd640 1 --2- 172.21.15.103:0/1782158524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 0x5602cd7015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff12b7fe640 1 --2- 172.21.15.103:0/1782158524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6f90c0 0x5602cd6f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1227fc640 1 -- 172.21.15.103:0/1782158524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6f90c0 msgr2=0x5602cd6f2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1227fc640 1 --2- 172.21.15.103:0/1782158524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6f90c0 0x5602cd6f2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1227fc640 1 -- 172.21.15.103:0/1782158524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 msgr2=0x5602cd7015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:58:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1227fc640 1 --2- 172.21.15.103:0/1782158524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 0x5602cd7015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1227fc640 1 -- 172.21.15.103:0/1782158524 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602cd6e5e90 con 0x5602cd6ea2f0 2024-09-19T06:58:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff12affd640 1 --2- 172.21.15.103:0/1782158524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 0x5602cd7015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:58:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1227fc640 1 --2- 172.21.15.103:0/1782158524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6ea2f0 0x5602cd6f2ab0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7ff11409a350 tx=0x7ff11400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3fd11110708065 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:58:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1127fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:58:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.271+0000 7ff1127fc640 1 -- 172.21.15.103:0/1782158524 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11409c7a0 con 0x5602cd6ea2f0 2024-09-19T06:58:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.272+0000 7ff1127fc640 1 -- 172.21.15.103:0/1782158524 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7ff114018d60 con 0x5602cd6ea2f0 2024-09-19T06:58:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.272+0000 7ff1341ea740 1 -- 172.21.15.103:0/1782158524 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff11406f3d0 con 0x5602cd6ea2f0 2024-09-19T06:58:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.272+0000 7ff1127fc640 1 -- 172.21.15.103:0/1782158524 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff1140d39e0 con 0x5602cd6ea2f0 2024-09-19T06:58:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.272+0000 7ff1341ea740 10 client.18732 fetch_fsmap learned FSMap version 27 2024-09-19T06:58:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.272+0000 7ff1341ea740 1 -- 172.21.15.103:0/1782158524 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5602cd7738c0 con 0x5602cd6ea2f0 2024-09-19T06:58:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.272+0000 7ff1127fc640 1 -- 172.21.15.103:0/1782158524 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff11406f3d0 con 0x5602cd6ea2f0 2024-09-19T06:58:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.272+0000 7ff1127fc640 1 -- 172.21.15.103:0/1782158524 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff1140d2070 con 0x5602cd6ea2f0 2024-09-19T06:58:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.272+0000 7ff1341ea740 10 client.18732 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:58:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.272+0000 7ff1341ea740 10 client.18732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:58:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.272+0000 7ff1341ea740 20 client.18732 populate_metadata read hostname 'smithi103' 2024-09-19T06:58:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.272+0000 7ff1341ea740 1 --2- 172.21.15.103:0/1782158524 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5602cd773a90 0x5602cd775e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:58:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.272+0000 7ff1341ea740 4 client.18732 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:58:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.272+0000 7ff1341ea740 1 -- 172.21.15.103:0/1782158524 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5602cd5d2900 con 0x5602cd773a90 2024-09-19T06:58:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.272+0000 7ff12affd640 1 --2- 172.21.15.103:0/1782158524 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5602cd773a90 0x5602cd775e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:58:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.273+0000 7ff1127fc640 10 client.18732 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.273+0000 7ff12affd640 1 --2- 172.21.15.103:0/1782158524 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5602cd773a90 0x5602cd775e80 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f9d25fa6dde88c4 server_cookie=52bddc05127983ea in_seq=0 out_seq=0 2024-09-19T06:58:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.286+0000 7ff1127fc640 1 -- 172.21.15.103:0/1782158524 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5602cd5d2900 con 0x5602cd773a90 2024-09-19T06:58:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.286+0000 7ff1127fc640 10 client.18732 handle_command_reply: tid=0 2024-09-19T06:58:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.286+0000 7ff1341ea740 1 client.18732 shutdown 2024-09-19T06:58:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.286+0000 7ff1341ea740 1 -- 172.21.15.103:0/1782158524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6ea2f0 msgr2=0x5602cd6f2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.286+0000 7ff1341ea740 1 --2- 172.21.15.103:0/1782158524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6ea2f0 0x5602cd6f2ab0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7ff11409a350 tx=0x7ff11400a560 comp rx=0 tx=0).stop 2024-09-19T06:58:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.286+0000 7ff1341ea740 1 -- 172.21.15.103:0/1782158524 shutdown_connections 2024-09-19T06:58:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.286+0000 7ff1341ea740 1 --2- 172.21.15.103:0/1782158524 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5602cd773a90 0x5602cd775e80 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.286+0000 7ff1341ea740 1 --2- 172.21.15.103:0/1782158524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602cd6f90c0 0x5602cd6f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.286+0000 7ff1341ea740 1 --2- 172.21.15.103:0/1782158524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602cd6ea2f0 0x5602cd6f2ab0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.286+0000 7ff1341ea740 1 --2- 172.21.15.103:0/1782158524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602cd6e99d0 0x5602cd7015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.286+0000 7ff1127fc640 0 client.18732 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:58:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.286+0000 7ff1341ea740 1 -- 172.21.15.103:0/1782158524 >> 172.21.15.103:0/1782158524 conn(0x5602cd6e6060 msgr2=0x5602cd6e51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.286+0000 7ff1341ea740 1 -- 172.21.15.103:0/1782158524 shutdown_connections 2024-09-19T06:58:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.286+0000 7ff1341ea740 1 -- 172.21.15.103:0/1782158524 wait complete. 2024-09-19T06:58:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.286+0000 7ff1341ea740 20 client.18732 trim_cache size 0 max 16384 2024-09-19T06:58:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:58:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:58:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:58:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:58:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.287+0000 7ff13136d640 1 -- 172.21.15.103:0/3570997232 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1180a3460 msgr2=0x7ff1180a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.287+0000 7ff13136d640 1 --2- 172.21.15.103:0/3570997232 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1180a3460 0x7ff1180a5920 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff124005e30 tx=0x7ff1240095c0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.287+0000 7ff13136d640 1 -- 172.21.15.103:0/3570997232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff12c101370 msgr2=0x7ff12c1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:58:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.287+0000 7ff13136d640 1 --2- 172.21.15.103:0/3570997232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff12c101370 0x7ff12c1987f0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff11400c920 tx=0x7ff11400cde0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.287+0000 7ff13136d640 1 -- 172.21.15.103:0/3570997232 shutdown_connections 2024-09-19T06:58:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.287+0000 7ff13136d640 1 --2- 172.21.15.103:0/3570997232 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1180a3460 0x7ff1180a5920 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.287+0000 7ff13136d640 1 --2- 172.21.15.103:0/3570997232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff12c1069e0 0x7ff12c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.287+0000 7ff13136d640 1 --2- 172.21.15.103:0/3570997232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff12c101370 0x7ff12c1987f0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.287+0000 7ff13136d640 1 --2- 172.21.15.103:0/3570997232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff12c1009d0 0x7ff12c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:58:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.287+0000 7ff13136d640 1 -- 172.21.15.103:0/3570997232 >> 172.21.15.103:0/3570997232 conn(0x7ff12c0fc6f0 msgr2=0x7ff12c0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:58:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.287+0000 7ff13136d640 1 -- 172.21.15.103:0/3570997232 shutdown_connections 2024-09-19T06:58:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:58:50.287+0000 7ff13136d640 1 -- 172.21.15.103:0/3570997232 wait complete. 2024-09-19T06:58:51.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:51 smithi103 ceph-mon[31158]: pgmap v297: 97 pgs: 97 active+clean; 4.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-09-19T06:58:51.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:51 smithi134 ceph-mon[35356]: pgmap v297: 97 pgs: 97 active+clean; 4.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-09-19T06:58:52.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:51 smithi174 ceph-mon[35637]: pgmap v297: 97 pgs: 97 active+clean; 4.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-09-19T06:58:53.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:53 smithi134 ceph-mon[35356]: pgmap v298: 97 pgs: 97 active+clean; 4.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-19T06:58:54.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:53 smithi174 ceph-mon[35637]: pgmap v298: 97 pgs: 97 active+clean; 4.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-19T06:58:54.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:53 smithi103 ceph-mon[31158]: pgmap v298: 97 pgs: 97 active+clean; 4.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-19T06:58:55.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:55 smithi134 ceph-mon[35356]: pgmap v299: 97 pgs: 97 active+clean; 4.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-19T06:58:56.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:55 smithi174 ceph-mon[35637]: pgmap v299: 97 pgs: 97 active+clean; 4.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-19T06:58:56.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:55 smithi103 ceph-mon[31158]: pgmap v299: 97 pgs: 97 active+clean; 4.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-19T06:58:57.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:57 smithi134 ceph-mon[35356]: pgmap v300: 97 pgs: 97 active+clean; 5.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-09-19T06:58:58.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:57 smithi174 ceph-mon[35637]: pgmap v300: 97 pgs: 97 active+clean; 5.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-09-19T06:58:58.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:57 smithi103 ceph-mon[31158]: pgmap v300: 97 pgs: 97 active+clean; 5.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-09-19T06:58:59.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:58:59 smithi134 ceph-mon[35356]: pgmap v301: 97 pgs: 97 active+clean; 5.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-09-19T06:59:00.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:58:59 smithi174 ceph-mon[35637]: pgmap v301: 97 pgs: 97 active+clean; 5.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-09-19T06:59:00.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:58:59 smithi103 ceph-mon[31158]: pgmap v301: 97 pgs: 97 active+clean; 5.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-09-19T06:59:01.938 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:01 smithi134 ceph-mon[35356]: pgmap v302: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-09-19T06:59:02.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:01 smithi174 ceph-mon[35637]: pgmap v302: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-09-19T06:59:02.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:01 smithi103 ceph-mon[31158]: pgmap v302: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-09-19T06:59:02.778 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:59:03.038 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.375+0000 7f82b739f640 1 -- 172.21.15.103:0/176731052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b01078e0 msgr2=0x7f82b0107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.375+0000 7f82b739f640 1 --2- 172.21.15.103:0/176731052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b01078e0 0x7f82b0107cc0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009930 tx=0x7f82a802a420 comp rx=0 tx=0).stop 2024-09-19T06:59:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.375+0000 7f82b739f640 1 -- 172.21.15.103:0/176731052 shutdown_connections 2024-09-19T06:59:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.375+0000 7f82b739f640 1 --2- 172.21.15.103:0/176731052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82b0102270 0x7f82b010e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.375+0000 7f82b739f640 1 --2- 172.21.15.103:0/176731052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82b01018d0 0x7f82b0101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.375+0000 7f82b739f640 1 --2- 172.21.15.103:0/176731052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b01078e0 0x7f82b0107cc0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.375+0000 7f82b739f640 1 -- 172.21.15.103:0/176731052 >> 172.21.15.103:0/176731052 conn(0x7f82b00fd5f0 msgr2=0x7f82b00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.376+0000 7f82b739f640 1 -- 172.21.15.103:0/176731052 shutdown_connections 2024-09-19T06:59:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.376+0000 7f82b739f640 1 -- 172.21.15.103:0/176731052 wait complete. 2024-09-19T06:59:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.376+0000 7f82b739f640 1 Processor -- start 2024-09-19T06:59:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.376+0000 7f82b739f640 1 -- start start 2024-09-19T06:59:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.376+0000 7f82b739f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82b01018d0 0x7f82b01982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.376+0000 7f82b739f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82b0102270 0x7f82b01987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.376+0000 7f82b739f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b01078e0 0x7f82b019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.376+0000 7f82b739f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0111d20 con 0x7f82b01078e0 2024-09-19T06:59:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.376+0000 7f82b739f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0111ba0 con 0x7f82b0102270 2024-09-19T06:59:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.376+0000 7f82b739f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0111ea0 con 0x7f82b01018d0 2024-09-19T06:59:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.376+0000 7f82b6b9e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b01078e0 0x7f82b019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.377+0000 7f82b6b9e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b01078e0 0x7f82b019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52350/0 (socket says 172.21.15.103:52350) 2024-09-19T06:59:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.377+0000 7f82b6b9e640 1 -- 172.21.15.103:0/2098341773 learned_addr learned my addr 172.21.15.103:0/2098341773 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.377+0000 7f82b5b9c640 1 --2- 172.21.15.103:0/2098341773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82b0102270 0x7f82b01987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.377+0000 7f82b639d640 1 --2- 172.21.15.103:0/2098341773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82b01018d0 0x7f82b01982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.377+0000 7f82b6b9e640 1 -- 172.21.15.103:0/2098341773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82b01018d0 msgr2=0x7f82b01982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.377+0000 7f82b6b9e640 1 --2- 172.21.15.103:0/2098341773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82b01018d0 0x7f82b01982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.377+0000 7f82b6b9e640 1 -- 172.21.15.103:0/2098341773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82b0102270 msgr2=0x7f82b01987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.377+0000 7f82b6b9e640 1 --2- 172.21.15.103:0/2098341773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82b0102270 0x7f82b01987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.377+0000 7f82b6b9e640 1 -- 172.21.15.103:0/2098341773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b019d2a0 con 0x7f82b01078e0 2024-09-19T06:59:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.377+0000 7f82b639d640 1 --2- 172.21.15.103:0/2098341773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82b01018d0 0x7f82b01982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.377+0000 7f82b6b9e640 1 --2- 172.21.15.103:0/2098341773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b01078e0 0x7f82b019cb70 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f82a000cce0 tx=0x7f82a000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.377+0000 7f82af7fe640 1 -- 172.21.15.103:0/2098341773 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a000f450 con 0x7f82b01078e0 2024-09-19T06:59:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.377+0000 7f82af7fe640 1 -- 172.21.15.103:0/2098341773 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82a000fad0 con 0x7f82b01078e0 2024-09-19T06:59:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.377+0000 7f82af7fe640 1 -- 172.21.15.103:0/2098341773 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0018c10 con 0x7f82b01078e0 2024-09-19T06:59:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.377+0000 7f82b739f640 1 -- 172.21.15.103:0/2098341773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b019d590 con 0x7f82b01078e0 2024-09-19T06:59:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.377+0000 7f82b739f640 1 -- 172.21.15.103:0/2098341773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b01a4e20 con 0x7f82b01078e0 2024-09-19T06:59:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.379+0000 7f82af7fe640 1 -- 172.21.15.103:0/2098341773 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f82a000f5f0 con 0x7f82b01078e0 2024-09-19T06:59:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.380+0000 7f82bb475740 1 Processor -- start 2024-09-19T06:59:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.380+0000 7f82bb475740 1 -- start start 2024-09-19T06:59:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.380+0000 7f82bb475740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7abbcc0 0x559be7abc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.380+0000 7f82bb475740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7abc5e0 0x559be7ac6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.380+0000 7f82bb475740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7ac6e80 0x559be7ac9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.380+0000 7f82bb475740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559be7a202f0 con 0x559be7ac6e80 2024-09-19T06:59:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.380+0000 7f82bb475740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559be79c8270 con 0x559be7abc5e0 2024-09-19T06:59:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.380+0000 7f82bb475740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559be7a1c2b0 con 0x559be7abbcc0 2024-09-19T06:59:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82af7fe640 1 --2- 172.21.15.103:0/2098341773 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f82980a3430 0x7f82980a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82b5b9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7ac6e80 0x559be7ac9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82b5b9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7ac6e80 0x559be7ac9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52362/0 (socket says 172.21.15.103:52362) 2024-09-19T06:59:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82b639d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7abc5e0 0x559be7ac6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82b5b9c640 1 -- 172.21.15.103:0/283213590 learned_addr learned my addr 172.21.15.103:0/283213590 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82b5b9c640 1 --2- 172.21.15.103:0/283213590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7abbcc0 0x559be7abc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82b6b9e640 1 --2- 172.21.15.103:0/2098341773 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f82980a3430 0x7f82980a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82af7fe640 1 -- 172.21.15.103:0/2098341773 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f82a00d02f0 con 0x7f82b01078e0 2024-09-19T06:59:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82b5b9c640 1 -- 172.21.15.103:0/283213590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7abbcc0 msgr2=0x559be7abc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82b5b9c640 1 --2- 172.21.15.103:0/283213590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7abbcc0 0x559be7abc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82b5b9c640 1 -- 172.21.15.103:0/283213590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7abc5e0 msgr2=0x559be7ac6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82b5b9c640 1 --2- 172.21.15.103:0/283213590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7abc5e0 0x559be7ac6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82b5b9c640 1 -- 172.21.15.103:0/283213590 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559be79c7340 con 0x559be7ac6e80 2024-09-19T06:59:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82b639d640 1 --2- 172.21.15.103:0/283213590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7abc5e0 0x559be7ac6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:59:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82b6b9e640 1 --2- 172.21.15.103:0/2098341773 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f82980a3430 0x7f82980a58f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f82a009c2b0 tx=0x7f82a006e330 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82b5b9c640 1 --2- 172.21.15.103:0/283213590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7ac6e80 0x559be7ac9270 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f82a4011730 tx=0x7f82a400a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c394b4afa1889e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82acbfe640 1 -- 172.21.15.103:0/283213590 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4004020 con 0x559be7ac6e80 2024-09-19T06:59:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82acbfe640 1 -- 172.21.15.103:0/283213590 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82a4005500 con 0x559be7ac6e80 2024-09-19T06:59:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.381+0000 7f82acbfe640 1 -- 172.21.15.103:0/283213590 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4004bd0 con 0x559be7ac6e80 2024-09-19T06:59:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.382+0000 7f82bb475740 1 -- 172.21.15.103:0/283213590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7ac6e80 msgr2=0x559be7ac9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.382+0000 7f82bb475740 1 --2- 172.21.15.103:0/283213590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7ac6e80 0x559be7ac9270 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f82a4011730 tx=0x7f82a400a6a0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.382+0000 7f82bb475740 1 -- 172.21.15.103:0/283213590 shutdown_connections 2024-09-19T06:59:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.382+0000 7f82bb475740 1 --2- 172.21.15.103:0/283213590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7ac6e80 0x559be7ac9270 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.382+0000 7f82bb475740 1 --2- 172.21.15.103:0/283213590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7abc5e0 0x559be7ac6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.382+0000 7f82bb475740 1 --2- 172.21.15.103:0/283213590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7abbcc0 0x559be7abc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.382+0000 7f82bb475740 1 -- 172.21.15.103:0/283213590 >> 172.21.15.103:0/283213590 conn(0x559be7ab8230 msgr2=0x559be7ab8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.382+0000 7f82bb475740 1 -- 172.21.15.103:0/283213590 shutdown_connections 2024-09-19T06:59:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.382+0000 7f82bb475740 1 -- 172.21.15.103:0/283213590 wait complete. 2024-09-19T06:59:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82bb475740 1 -- start start 2024-09-19T06:59:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82bb475740 1 Processor -- start 2024-09-19T06:59:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82bb475740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7abbcc0 0x559be7ac3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82bb475740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7abc5e0 0x559be7ac42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82bb475740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7ac6e80 0x559be7ac4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82bb475740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559be7ac9a50 con 0x559be7abbcc0 2024-09-19T06:59:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82bb475740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559be79a9290 con 0x559be7abc5e0 2024-09-19T06:59:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82bb475740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559be7ac9bd0 con 0x559be7ac6e80 2024-09-19T06:59:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82bb475740 10 client.0 _ll_register_callbacks cb 0x559be79da140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82b5b9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7abbcc0 0x559be7ac3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82b5b9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7abbcc0 0x559be7ac3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52372/0 (socket says 172.21.15.103:52372) 2024-09-19T06:59:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82b5b9c640 1 -- 172.21.15.103:0/2456582638 learned_addr learned my addr 172.21.15.103:0/2456582638 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82b5b9c640 1 -- 172.21.15.103:0/2456582638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7ac6e80 msgr2=0x559be7ac4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82b5b9c640 1 --2- 172.21.15.103:0/2456582638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7ac6e80 0x559be7ac4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82b5b9c640 1 -- 172.21.15.103:0/2456582638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7abc5e0 msgr2=0x559be7ac42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82b5b9c640 1 --2- 172.21.15.103:0/2456582638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7abc5e0 0x559be7ac42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82b5b9c640 1 -- 172.21.15.103:0/2456582638 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559be7733210 con 0x559be7abbcc0 2024-09-19T06:59:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f82b5b9c640 1 --2- 172.21.15.103:0/2456582638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7abbcc0 0x559be7ac3d80 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f82a4011700 tx=0x7f82a4004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae1deef5a60638d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f8295ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.383+0000 7f8295ffb640 1 -- 172.21.15.103:0/2456582638 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a400f990 con 0x559be7abbcc0 2024-09-19T06:59:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.384+0000 7f8295ffb640 1 -- 172.21.15.103:0/2456582638 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f82a400fb30 con 0x559be7abbcc0 2024-09-19T06:59:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.384+0000 7f82bb475740 1 -- 172.21.15.103:0/2456582638 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82a4005500 con 0x559be7abbcc0 2024-09-19T06:59:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.384+0000 7f8295ffb640 1 -- 172.21.15.103:0/2456582638 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82a40046c0 con 0x559be7abbcc0 2024-09-19T06:59:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.384+0000 7f82bb475740 10 client.18744 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.384+0000 7f82bb475740 1 -- 172.21.15.103:0/2456582638 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559be77ff620 con 0x559be7abbcc0 2024-09-19T06:59:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.384+0000 7f8295ffb640 1 -- 172.21.15.103:0/2456582638 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82a4005500 con 0x559be7abbcc0 2024-09-19T06:59:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.384+0000 7f8295ffb640 1 -- 172.21.15.103:0/2456582638 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f82a4025020 con 0x559be7abbcc0 2024-09-19T06:59:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.384+0000 7f82bb475740 10 client.18744 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.384+0000 7f82bb475740 10 client.18744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.384+0000 7f82bb475740 20 client.18744 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.384+0000 7f82bb475740 1 --2- 172.21.15.103:0/2456582638 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559be7ad6c80 0x559be7ad9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.384+0000 7f82bb475740 4 client.18744 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.384+0000 7f82bb475740 1 -- 172.21.15.103:0/2456582638 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559be7a2da20 con 0x559be7ad6c80 2024-09-19T06:59:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.384+0000 7f82b639d640 1 --2- 172.21.15.103:0/2456582638 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559be7ad6c80 0x559be7ad9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.385+0000 7f82b639d640 1 --2- 172.21.15.103:0/2456582638 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559be7ad6c80 0x559be7ad9070 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=211f53f89b07149b server_cookie=5a4a561da38fa1e3 in_seq=0 out_seq=0 2024-09-19T06:59:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.385+0000 7f8295ffb640 10 client.18744 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.385+0000 7f8295ffb640 1 -- 172.21.15.103:0/2456582638 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559be7a2da20 con 0x559be7ad6c80 2024-09-19T06:59:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.385+0000 7f8295ffb640 10 client.18744 handle_command_reply: tid=0 2024-09-19T06:59:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.385+0000 7f82bb475740 1 client.18744 shutdown 2024-09-19T06:59:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.386+0000 7f82bb475740 1 -- 172.21.15.103:0/2456582638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7abbcc0 msgr2=0x559be7ac3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.386+0000 7f82bb475740 1 --2- 172.21.15.103:0/2456582638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7abbcc0 0x559be7ac3d80 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f82a4011700 tx=0x7f82a4004bc0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.386+0000 7f82bb475740 1 -- 172.21.15.103:0/2456582638 shutdown_connections 2024-09-19T06:59:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.386+0000 7f82bb475740 1 --2- 172.21.15.103:0/2456582638 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559be7ad6c80 0x559be7ad9070 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.386+0000 7f82bb475740 1 --2- 172.21.15.103:0/2456582638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7ac6e80 0x559be7ac4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.386+0000 7f82bb475740 1 --2- 172.21.15.103:0/2456582638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7abc5e0 0x559be7ac42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.386+0000 7f8295ffb640 0 client.18744 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.386+0000 7f82bb475740 1 --2- 172.21.15.103:0/2456582638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7abbcc0 0x559be7ac3d80 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.386+0000 7f82bb475740 1 -- 172.21.15.103:0/2456582638 >> 172.21.15.103:0/2456582638 conn(0x559be7ab8230 msgr2=0x559be7abdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.386+0000 7f82bb475740 1 -- 172.21.15.103:0/2456582638 shutdown_connections 2024-09-19T06:59:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.386+0000 7f82bb475740 1 -- 172.21.15.103:0/2456582638 wait complete. 2024-09-19T06:59:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.386+0000 7f82bb475740 20 client.18744 trim_cache size 0 max 16384 2024-09-19T06:59:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82bb475740 1 Processor -- start 2024-09-19T06:59:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82bb475740 1 -- start start 2024-09-19T06:59:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82bb475740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7abbcc0 0x559be7abe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82bb475740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7abc5e0 0x559be7ac5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82bb475740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7ac6e80 0x559be7ac42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82bb475740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01ad680 con 0x559be7abc5e0 2024-09-19T06:59:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82bb475740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01af1c0 con 0x559be7ac6e80 2024-09-19T06:59:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82bb475740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01add50 con 0x559be7abbcc0 2024-09-19T06:59:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82b639d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7abc5e0 0x559be7ac5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82b639d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7abc5e0 0x559be7ac5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52376/0 (socket says 172.21.15.103:52376) 2024-09-19T06:59:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82b639d640 1 -- 172.21.15.103:0/276298369 learned_addr learned my addr 172.21.15.103:0/276298369 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82b5b9c640 1 --2- 172.21.15.103:0/276298369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7ac6e80 0x559be7ac42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82b639d640 1 -- 172.21.15.103:0/276298369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7abbcc0 msgr2=0x559be7abe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82b639d640 1 --2- 172.21.15.103:0/276298369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7abbcc0 0x559be7abe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82b639d640 1 -- 172.21.15.103:0/276298369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7ac6e80 msgr2=0x559be7ac42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82b639d640 1 --2- 172.21.15.103:0/276298369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7ac6e80 0x559be7ac42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82b639d640 1 -- 172.21.15.103:0/276298369 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559be7ab8060 con 0x559be7abc5e0 2024-09-19T06:59:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.400+0000 7f82b5b9c640 1 --2- 172.21.15.103:0/276298369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7ac6e80 0x559be7ac42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.401+0000 7f82b639d640 1 --2- 172.21.15.103:0/276298369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7abc5e0 0x559be7ac5cf0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f82a8005ab0 tx=0x7f82a8004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9715b25ac68c85d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.401+0000 7f8297fff640 1 -- 172.21.15.103:0/276298369 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8035040 con 0x559be7abc5e0 2024-09-19T06:59:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.401+0000 7f8297fff640 1 -- 172.21.15.103:0/276298369 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82a8004400 con 0x559be7abc5e0 2024-09-19T06:59:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.401+0000 7f8297fff640 1 -- 172.21.15.103:0/276298369 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a80035f0 con 0x559be7abc5e0 2024-09-19T06:59:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.401+0000 7f82bb475740 1 -- 172.21.15.103:0/276298369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7abc5e0 msgr2=0x559be7ac5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.401+0000 7f82bb475740 1 --2- 172.21.15.103:0/276298369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7abc5e0 0x559be7ac5cf0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f82a8005ab0 tx=0x7f82a8004310 comp rx=0 tx=0).stop 2024-09-19T06:59:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.401+0000 7f82bb475740 1 -- 172.21.15.103:0/276298369 shutdown_connections 2024-09-19T06:59:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.401+0000 7f82bb475740 1 --2- 172.21.15.103:0/276298369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7ac6e80 0x559be7ac42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.401+0000 7f82bb475740 1 --2- 172.21.15.103:0/276298369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7abc5e0 0x559be7ac5cf0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.401+0000 7f82bb475740 1 --2- 172.21.15.103:0/276298369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7abbcc0 0x559be7abe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.401+0000 7f82bb475740 1 -- 172.21.15.103:0/276298369 >> 172.21.15.103:0/276298369 conn(0x559be7ab8230 msgr2=0x559be7ac55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.401+0000 7f82bb475740 1 -- 172.21.15.103:0/276298369 shutdown_connections 2024-09-19T06:59:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.401+0000 7f82bb475740 1 -- 172.21.15.103:0/276298369 wait complete. 2024-09-19T06:59:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82bb475740 1 -- start start 2024-09-19T06:59:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82bb475740 1 Processor -- start 2024-09-19T06:59:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82bb475740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7abbcc0 0x559be7acf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82bb475740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7abc5e0 0x559be7aca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82bb475740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7ac6e80 0x559be7acab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82bb475740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01aeaf0 con 0x559be7ac6e80 2024-09-19T06:59:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82bb475740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01ae420 con 0x559be7abc5e0 2024-09-19T06:59:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82bb475740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01ab680 con 0x559be7abbcc0 2024-09-19T06:59:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82bb475740 10 client.0 _ll_register_callbacks cb 0x7f82a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82b5b9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7ac6e80 0x559be7acab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82b5b9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7ac6e80 0x559be7acab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52378/0 (socket says 172.21.15.103:52378) 2024-09-19T06:59:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82b5b9c640 1 -- 172.21.15.103:0/804451367 learned_addr learned my addr 172.21.15.103:0/804451367 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82b639d640 1 --2- 172.21.15.103:0/804451367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7abc5e0 0x559be7aca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82b5b9c640 1 --2- 172.21.15.103:0/804451367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7abbcc0 0x559be7acf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82b5b9c640 1 -- 172.21.15.103:0/804451367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7abbcc0 msgr2=0x559be7acf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82b5b9c640 1 --2- 172.21.15.103:0/804451367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7abbcc0 0x559be7acf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82b5b9c640 1 -- 172.21.15.103:0/804451367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7abc5e0 msgr2=0x559be7aca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82b5b9c640 1 --2- 172.21.15.103:0/804451367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7abc5e0 0x559be7aca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82b5b9c640 1 -- 172.21.15.103:0/804451367 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559be7ac6910 con 0x559be7ac6e80 2024-09-19T06:59:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f82b5b9c640 1 --2- 172.21.15.103:0/804451367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7ac6e80 0x559be7acab70 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f82a401eeb0 tx=0x7f82a4027440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ab9217d75aef55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f8295ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.402+0000 7f8295ffb640 1 -- 172.21.15.103:0/804451367 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4033070 con 0x559be7ac6e80 2024-09-19T06:59:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.403+0000 7f82bb475740 1 -- 172.21.15.103:0/804451367 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82a8004400 con 0x559be7ac6e80 2024-09-19T06:59:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.403+0000 7f8295ffb640 1 -- 172.21.15.103:0/804451367 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f82a4004020 con 0x559be7ac6e80 2024-09-19T06:59:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.403+0000 7f8295ffb640 1 -- 172.21.15.103:0/804451367 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82a403e050 con 0x559be7ac6e80 2024-09-19T06:59:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.403+0000 7f82bb475740 10 client.18750 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.403+0000 7f82bb475740 1 -- 172.21.15.103:0/804451367 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559be7ab8ec0 con 0x559be7ac6e80 2024-09-19T06:59:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.403+0000 7f8295ffb640 1 -- 172.21.15.103:0/804451367 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82a8004400 con 0x559be7ac6e80 2024-09-19T06:59:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.403+0000 7f8295ffb640 1 -- 172.21.15.103:0/804451367 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f82a4025070 con 0x559be7ac6e80 2024-09-19T06:59:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.403+0000 7f82bb475740 10 client.18750 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.403+0000 7f82bb475740 10 client.18750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.403+0000 7f82bb475740 20 client.18750 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.403+0000 7f82bb475740 1 --2- 172.21.15.103:0/804451367 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559be7b41730 0x559be7b43b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.403+0000 7f82bb475740 4 client.18750 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:59:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.403+0000 7f82bb475740 1 -- 172.21.15.103:0/804451367 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559be7a27d70 con 0x559be7b41730 2024-09-19T06:59:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.403+0000 7f82b639d640 1 --2- 172.21.15.103:0/804451367 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559be7b41730 0x559be7b43b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.404+0000 7f82b639d640 1 --2- 172.21.15.103:0/804451367 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559be7b41730 0x559be7b43b50 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a1ad72627dbf598 server_cookie=8ff9d1e7f4a2c44a in_seq=0 out_seq=0 2024-09-19T06:59:03.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.404+0000 7f8295ffb640 10 client.18750 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.417+0000 7f8295ffb640 1 -- 172.21.15.103:0/804451367 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559be7a27d70 con 0x559be7b41730 2024-09-19T06:59:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.417+0000 7f8295ffb640 10 client.18750 handle_command_reply: tid=0 2024-09-19T06:59:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.417+0000 7f82bb475740 1 client.18750 shutdown 2024-09-19T06:59:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.417+0000 7f82bb475740 1 -- 172.21.15.103:0/804451367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7ac6e80 msgr2=0x559be7acab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.417+0000 7f82bb475740 1 --2- 172.21.15.103:0/804451367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7ac6e80 0x559be7acab70 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f82a401eeb0 tx=0x7f82a4027440 comp rx=0 tx=0).stop 2024-09-19T06:59:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.417+0000 7f82bb475740 1 -- 172.21.15.103:0/804451367 shutdown_connections 2024-09-19T06:59:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.417+0000 7f82bb475740 1 --2- 172.21.15.103:0/804451367 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559be7b41730 0x559be7b43b50 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.417+0000 7f82bb475740 1 --2- 172.21.15.103:0/804451367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559be7ac6e80 0x559be7acab70 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.417+0000 7f8295ffb640 0 client.18750 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.417+0000 7f82bb475740 1 --2- 172.21.15.103:0/804451367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be7abc5e0 0x559be7aca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.417+0000 7f82bb475740 1 --2- 172.21.15.103:0/804451367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559be7abbcc0 0x559be7acf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.417+0000 7f82bb475740 1 -- 172.21.15.103:0/804451367 >> 172.21.15.103:0/804451367 conn(0x559be7ab8230 msgr2=0x559be7ab6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.417+0000 7f82bb475740 1 -- 172.21.15.103:0/804451367 shutdown_connections 2024-09-19T06:59:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.417+0000 7f82bb475740 1 -- 172.21.15.103:0/804451367 wait complete. 2024-09-19T06:59:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.417+0000 7f82bb475740 20 client.18750 trim_cache size 0 max 16384 2024-09-19T06:59:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:59:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:59:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:59:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:59:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.418+0000 7f82b739f640 1 -- 172.21.15.103:0/2098341773 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f82980a3430 msgr2=0x7f82980a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.418+0000 7f82b739f640 1 --2- 172.21.15.103:0/2098341773 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f82980a3430 0x7f82980a58f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f82a009c2b0 tx=0x7f82a006e330 comp rx=0 tx=0).stop 2024-09-19T06:59:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.418+0000 7f82b739f640 1 -- 172.21.15.103:0/2098341773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b01078e0 msgr2=0x7f82b019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.418+0000 7f82b739f640 1 --2- 172.21.15.103:0/2098341773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b01078e0 0x7f82b019cb70 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f82a000cce0 tx=0x7f82a000a620 comp rx=0 tx=0).stop 2024-09-19T06:59:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.418+0000 7f82b739f640 1 -- 172.21.15.103:0/2098341773 shutdown_connections 2024-09-19T06:59:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.418+0000 7f82b739f640 1 --2- 172.21.15.103:0/2098341773 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f82980a3430 0x7f82980a58f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.418+0000 7f82b739f640 1 --2- 172.21.15.103:0/2098341773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b01078e0 0x7f82b019cb70 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.418+0000 7f82b739f640 1 --2- 172.21.15.103:0/2098341773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82b0102270 0x7f82b01987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.418+0000 7f82b739f640 1 --2- 172.21.15.103:0/2098341773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82b01018d0 0x7f82b01982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.418+0000 7f82b739f640 1 -- 172.21.15.103:0/2098341773 >> 172.21.15.103:0/2098341773 conn(0x7f82b00fd5f0 msgr2=0x7f82b00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.418+0000 7f82b739f640 1 -- 172.21.15.103:0/2098341773 shutdown_connections 2024-09-19T06:59:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:03.418+0000 7f82b739f640 1 -- 172.21.15.103:0/2098341773 wait complete. 2024-09-19T06:59:03.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.518+0000 7f261121a640 1 -- 172.21.15.103:0/2086677113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f260c100e20 msgr2=0x7f260c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.518+0000 7f261121a640 1 --2- 172.21.15.103:0/2086677113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f260c100e20 0x7f260c101200 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009900 tx=0x7f25fc02a480 comp rx=0 tx=0).stop 2024-09-19T06:59:03.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.518+0000 7f261121a640 1 -- 172.21.15.103:0/2086677113 shutdown_connections 2024-09-19T06:59:03.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.518+0000 7f261121a640 1 --2- 172.21.15.103:0/2086677113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f260c102ef0 0x7f260c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.518+0000 7f261121a640 1 --2- 172.21.15.103:0/2086677113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f260c102570 0x7f260c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.518+0000 7f261121a640 1 --2- 172.21.15.103:0/2086677113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f260c100e20 0x7f260c101200 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.518+0000 7f261121a640 1 -- 172.21.15.103:0/2086677113 >> 172.21.15.103:0/2086677113 conn(0x7f260c0fc7d0 msgr2=0x7f260c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:03.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.518+0000 7f261121a640 1 -- 172.21.15.103:0/2086677113 shutdown_connections 2024-09-19T06:59:03.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.518+0000 7f261121a640 1 -- 172.21.15.103:0/2086677113 wait complete. 2024-09-19T06:59:03.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.518+0000 7f261121a640 1 Processor -- start 2024-09-19T06:59:03.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f261121a640 1 -- start start 2024-09-19T06:59:03.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f261121a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f260c100e20 0x7f260c1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f261121a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f260c102570 0x7f260c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f261121a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f260c102ef0 0x7f260c1a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f261121a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f260c110fa0 con 0x7f260c102570 2024-09-19T06:59:03.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f261121a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f260c110e20 con 0x7f260c100e20 2024-09-19T06:59:03.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f261121a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f260c111120 con 0x7f260c102ef0 2024-09-19T06:59:03.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f260a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f260c102570 0x7f260c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f260a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f260c102570 0x7f260c199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52382/0 (socket says 172.21.15.103:52382) 2024-09-19T06:59:03.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f260a59c640 1 -- 172.21.15.103:0/1614338257 learned_addr learned my addr 172.21.15.103:0/1614338257 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:03.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f260ad9d640 1 --2- 172.21.15.103:0/1614338257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f260c100e20 0x7f260c1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f260b59e640 1 --2- 172.21.15.103:0/1614338257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f260c102ef0 0x7f260c1a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f260a59c640 1 -- 172.21.15.103:0/1614338257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f260c102ef0 msgr2=0x7f260c1a4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f260a59c640 1 --2- 172.21.15.103:0/1614338257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f260c102ef0 0x7f260c1a4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f260a59c640 1 -- 172.21.15.103:0/1614338257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f260c100e20 msgr2=0x7f260c1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f260a59c640 1 --2- 172.21.15.103:0/1614338257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f260c100e20 0x7f260c1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.519+0000 7f260a59c640 1 -- 172.21.15.103:0/1614338257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c1a49b0 con 0x7f260c102570 2024-09-19T06:59:03.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.520+0000 7f260a59c640 1 --2- 172.21.15.103:0/1614338257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f260c102570 0x7f260c199930 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f25f400b4a0 tx=0x7f25f400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:03.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.520+0000 7f2603fff640 1 -- 172.21.15.103:0/1614338257 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4014070 con 0x7f260c102570 2024-09-19T06:59:03.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.520+0000 7f261121a640 1 -- 172.21.15.103:0/1614338257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260c1a4b80 con 0x7f260c102570 2024-09-19T06:59:03.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.520+0000 7f2603fff640 1 -- 172.21.15.103:0/1614338257 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f4010af0 con 0x7f260c102570 2024-09-19T06:59:03.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.520+0000 7f2603fff640 1 -- 172.21.15.103:0/1614338257 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4017d10 con 0x7f260c102570 2024-09-19T06:59:03.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.520+0000 7f261121a640 1 -- 172.21.15.103:0/1614338257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f260c1a50c0 con 0x7f260c102570 2024-09-19T06:59:03.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.522+0000 7f2603fff640 1 -- 172.21.15.103:0/1614338257 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f25f4017070 con 0x7f260c102570 2024-09-19T06:59:03.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.523+0000 7f26135f3740 1 Processor -- start 2024-09-19T06:59:03.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.523+0000 7f26135f3740 1 -- start start 2024-09-19T06:59:03.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.523+0000 7f26135f3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df796f64a0 0x55df796f6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.523+0000 7f26135f3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f6dc0 0x55df79701140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.523+0000 7f26135f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df79701680 0x55df79703a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.523+0000 7f26135f3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55df7965c2e0 con 0x55df796f64a0 2024-09-19T06:59:03.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.523+0000 7f26135f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55df79603ce0 con 0x55df79701680 2024-09-19T06:59:03.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.523+0000 7f26135f3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55df796582a0 con 0x55df796f6dc0 2024-09-19T06:59:03.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.523+0000 7f260b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df796f64a0 0x55df796f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.523+0000 7f260b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df796f64a0 0x55df796f6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52394/0 (socket says 172.21.15.103:52394) 2024-09-19T06:59:03.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.523+0000 7f260b59e640 1 -- 172.21.15.103:0/3274314904 learned_addr learned my addr 172.21.15.103:0/3274314904 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:03.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.523+0000 7f260a59c640 1 --2- 172.21.15.103:0/3274314904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df79701680 0x55df79703a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.523+0000 7f260ad9d640 1 --2- 172.21.15.103:0/3274314904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f6dc0 0x55df79701140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.524+0000 7f2603fff640 1 --2- 172.21.15.103:0/1614338257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f25f80a33b0 0x7f25f80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.524+0000 7f260b59e640 1 -- 172.21.15.103:0/3274314904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f6dc0 msgr2=0x55df79701140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:03.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.524+0000 7f260b59e640 1 --2- 172.21.15.103:0/3274314904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f6dc0 0x55df79701140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.524+0000 7f260b59e640 1 -- 172.21.15.103:0/3274314904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df79701680 msgr2=0x55df79703a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.524+0000 7f260b59e640 1 --2- 172.21.15.103:0/3274314904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df79701680 0x55df79703a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.524+0000 7f260b59e640 1 -- 172.21.15.103:0/3274314904 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df795e8440 con 0x55df796f64a0 2024-09-19T06:59:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.524+0000 7f2603fff640 1 -- 172.21.15.103:0/1614338257 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f25f40cf0b0 con 0x7f260c102570 2024-09-19T06:59:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.524+0000 7f260b59e640 1 --2- 172.21.15.103:0/3274314904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df796f64a0 0x55df796f6880 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f2604007e50 tx=0x7f260400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cd2624a5d56492 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.524+0000 7f260b59e640 1 --2- 172.21.15.103:0/1614338257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f25f80a33b0 0x7f25f80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.524+0000 7f26017fa640 1 -- 172.21.15.103:0/3274314904 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604020070 con 0x55df796f64a0 2024-09-19T06:59:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.524+0000 7f26017fa640 1 -- 172.21.15.103:0/3274314904 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f260401ae30 con 0x55df796f64a0 2024-09-19T06:59:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.524+0000 7f26017fa640 1 -- 172.21.15.103:0/3274314904 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604014070 con 0x55df796f64a0 2024-09-19T06:59:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.525+0000 7f26135f3740 1 -- 172.21.15.103:0/3274314904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df796f64a0 msgr2=0x55df796f6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.525+0000 7f26135f3740 1 --2- 172.21.15.103:0/3274314904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df796f64a0 0x55df796f6880 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f2604007e50 tx=0x7f260400e720 comp rx=0 tx=0).stop 2024-09-19T06:59:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.525+0000 7f260b59e640 1 --2- 172.21.15.103:0/1614338257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f25f80a33b0 0x7f25f80a5870 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f260401d000 tx=0x7f2604020000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.525+0000 7f26135f3740 1 -- 172.21.15.103:0/3274314904 shutdown_connections 2024-09-19T06:59:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.525+0000 7f26135f3740 1 --2- 172.21.15.103:0/3274314904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df79701680 0x55df79703a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.525+0000 7f26135f3740 1 --2- 172.21.15.103:0/3274314904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f6dc0 0x55df79701140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.525+0000 7f26135f3740 1 --2- 172.21.15.103:0/3274314904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df796f64a0 0x55df796f6880 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.525+0000 7f26135f3740 1 -- 172.21.15.103:0/3274314904 >> 172.21.15.103:0/3274314904 conn(0x55df796f2a30 msgr2=0x55df796f2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.525+0000 7f26135f3740 1 -- 172.21.15.103:0/3274314904 shutdown_connections 2024-09-19T06:59:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.525+0000 7f26135f3740 1 -- 172.21.15.103:0/3274314904 wait complete. 2024-09-19T06:59:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.525+0000 7f26135f3740 1 -- start start 2024-09-19T06:59:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f26135f3740 1 Processor -- start 2024-09-19T06:59:03.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f26135f3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f64a0 0x55df79700fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f26135f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df796f6dc0 0x55df796fe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f26135f3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df796fe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f26135f3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55df79704250 con 0x55df79701680 2024-09-19T06:59:03.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f26135f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55df79617050 con 0x55df796f6dc0 2024-09-19T06:59:03.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f26135f3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55df797043d0 con 0x55df796f64a0 2024-09-19T06:59:03.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f26135f3740 10 client.0 _ll_register_callbacks cb 0x55df79657160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f260b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df796fe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f260b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df796fe7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52398/0 (socket says 172.21.15.103:52398) 2024-09-19T06:59:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f260b59e640 1 -- 172.21.15.103:0/3063266846 learned_addr learned my addr 172.21.15.103:0/3063266846 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f260a59c640 1 --2- 172.21.15.103:0/3063266846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df796f6dc0 0x55df796fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f260ad9d640 1 --2- 172.21.15.103:0/3063266846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f64a0 0x55df79700fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f260b59e640 1 -- 172.21.15.103:0/3063266846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f64a0 msgr2=0x55df79700fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f260b59e640 1 --2- 172.21.15.103:0/3063266846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f64a0 0x55df79700fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f260b59e640 1 -- 172.21.15.103:0/3063266846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df796f6dc0 msgr2=0x55df796fe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f260b59e640 1 --2- 172.21.15.103:0/3063266846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df796f6dc0 0x55df796fe280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.526+0000 7f260b59e640 1 -- 172.21.15.103:0/3063266846 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df793cd3c0 con 0x55df79701680 2024-09-19T06:59:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.527+0000 7f260b59e640 1 --2- 172.21.15.103:0/3063266846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df796fe7e0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f260400fba0 tx=0x7f2604010b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd9aa3d230a258e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.527+0000 7f25f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.527+0000 7f25f27fc640 1 -- 172.21.15.103:0/3063266846 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604014070 con 0x55df79701680 2024-09-19T06:59:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.527+0000 7f25f27fc640 1 -- 172.21.15.103:0/3063266846 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f2604003b80 con 0x55df79701680 2024-09-19T06:59:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.527+0000 7f26135f3740 1 -- 172.21.15.103:0/3063266846 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f260401ae30 con 0x55df79701680 2024-09-19T06:59:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.527+0000 7f25f27fc640 1 -- 172.21.15.103:0/3063266846 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f260400f830 con 0x55df79701680 2024-09-19T06:59:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.527+0000 7f26135f3740 10 client.18762 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.527+0000 7f26135f3740 1 -- 172.21.15.103:0/3063266846 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df7970fc20 con 0x55df79701680 2024-09-19T06:59:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.527+0000 7f25f27fc640 1 -- 172.21.15.103:0/3063266846 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f260401ae30 con 0x55df79701680 2024-09-19T06:59:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.527+0000 7f25f27fc640 1 -- 172.21.15.103:0/3063266846 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f260402c070 con 0x55df79701680 2024-09-19T06:59:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.527+0000 7f26135f3740 10 client.18762 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.527+0000 7f26135f3740 10 client.18762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.527+0000 7f26135f3740 20 client.18762 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.527+0000 7f26135f3740 1 --2- 172.21.15.103:0/3063266846 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55df79710fd0 0x55df797133c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.527+0000 7f26135f3740 4 client.18762 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.527+0000 7f26135f3740 1 -- 172.21.15.103:0/3063266846 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df79669850 con 0x55df79710fd0 2024-09-19T06:59:03.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.528+0000 7f260ad9d640 1 --2- 172.21.15.103:0/3063266846 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55df79710fd0 0x55df797133c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.528+0000 7f260ad9d640 1 --2- 172.21.15.103:0/3063266846 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55df79710fd0 0x55df797133c0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf422bbe458685a7 server_cookie=efefa19f7afd1edb in_seq=0 out_seq=0 2024-09-19T06:59:03.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.528+0000 7f25f27fc640 10 client.18762 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:03.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.529+0000 7f25f27fc640 1 -- 172.21.15.103:0/3063266846 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df79669850 con 0x55df79710fd0 2024-09-19T06:59:03.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.529+0000 7f25f27fc640 10 client.18762 handle_command_reply: tid=0 2024-09-19T06:59:03.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.529+0000 7f26135f3740 1 client.18762 shutdown 2024-09-19T06:59:03.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.529+0000 7f26135f3740 1 -- 172.21.15.103:0/3063266846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 msgr2=0x55df796fe7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.529+0000 7f26135f3740 1 --2- 172.21.15.103:0/3063266846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df796fe7e0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f260400fba0 tx=0x7f2604010b10 comp rx=0 tx=0).stop 2024-09-19T06:59:03.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.529+0000 7f26135f3740 1 -- 172.21.15.103:0/3063266846 shutdown_connections 2024-09-19T06:59:03.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.529+0000 7f26135f3740 1 --2- 172.21.15.103:0/3063266846 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55df79710fd0 0x55df797133c0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.529+0000 7f26135f3740 1 --2- 172.21.15.103:0/3063266846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df796fe7e0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.529+0000 7f26135f3740 1 --2- 172.21.15.103:0/3063266846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df796f6dc0 0x55df796fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.529+0000 7f26135f3740 1 --2- 172.21.15.103:0/3063266846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f64a0 0x55df79700fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.529+0000 7f25f27fc640 0 client.18762 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:03.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.529+0000 7f26135f3740 1 -- 172.21.15.103:0/3063266846 >> 172.21.15.103:0/3063266846 conn(0x55df796f2a30 msgr2=0x55df796f8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:03.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.529+0000 7f26135f3740 1 -- 172.21.15.103:0/3063266846 shutdown_connections 2024-09-19T06:59:03.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.529+0000 7f26135f3740 1 -- 172.21.15.103:0/3063266846 wait complete. 2024-09-19T06:59:03.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.529+0000 7f26135f3740 20 client.18762 trim_cache size 0 max 16384 2024-09-19T06:59:03.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.546+0000 7f26135f3740 1 Processor -- start 2024-09-19T06:59:03.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.546+0000 7f26135f3740 1 -- start start 2024-09-19T06:59:03.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f26135f3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f64a0 0x55df7970d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f26135f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df796f6dc0 0x55df79718b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f26135f3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df797190c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f26135f3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1ad9a0 con 0x55df79701680 2024-09-19T06:59:03.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f26135f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1af4e0 con 0x55df796f6dc0 2024-09-19T06:59:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f26135f3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1ae070 con 0x55df796f64a0 2024-09-19T06:59:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f260b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df797190c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f260b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df797190c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52400/0 (socket says 172.21.15.103:52400) 2024-09-19T06:59:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f260b59e640 1 -- 172.21.15.103:0/2354395110 learned_addr learned my addr 172.21.15.103:0/2354395110 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f260a59c640 1 --2- 172.21.15.103:0/2354395110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df796f6dc0 0x55df79718b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f260ad9d640 1 --2- 172.21.15.103:0/2354395110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f64a0 0x55df7970d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f260b59e640 1 -- 172.21.15.103:0/2354395110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f64a0 msgr2=0x55df7970d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f260b59e640 1 --2- 172.21.15.103:0/2354395110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f64a0 0x55df7970d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f260b59e640 1 -- 172.21.15.103:0/2354395110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df796f6dc0 msgr2=0x55df79718b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f260b59e640 1 --2- 172.21.15.103:0/2354395110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df796f6dc0 0x55df79718b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f260b59e640 1 -- 172.21.15.103:0/2354395110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df796feb50 con 0x55df79701680 2024-09-19T06:59:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f260a59c640 1 --2- 172.21.15.103:0/2354395110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df796f6dc0 0x55df79718b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f260b59e640 1 --2- 172.21.15.103:0/2354395110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df797190c0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f2604004930 tx=0x7f26040072a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749193ccb0b86013 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f2600ff9640 1 -- 172.21.15.103:0/2354395110 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604014070 con 0x55df79701680 2024-09-19T06:59:03.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f2600ff9640 1 -- 172.21.15.103:0/2354395110 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f260400eb60 con 0x55df79701680 2024-09-19T06:59:03.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.547+0000 7f2600ff9640 1 -- 172.21.15.103:0/2354395110 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604010c00 con 0x55df79701680 2024-09-19T06:59:03.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.548+0000 7f26135f3740 1 -- 172.21.15.103:0/2354395110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 msgr2=0x55df797190c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.548+0000 7f26135f3740 1 --2- 172.21.15.103:0/2354395110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df797190c0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f2604004930 tx=0x7f26040072a0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.548+0000 7f26135f3740 1 -- 172.21.15.103:0/2354395110 shutdown_connections 2024-09-19T06:59:03.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.548+0000 7f26135f3740 1 --2- 172.21.15.103:0/2354395110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df797190c0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.548+0000 7f26135f3740 1 --2- 172.21.15.103:0/2354395110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df796f6dc0 0x55df79718b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.548+0000 7f26135f3740 1 --2- 172.21.15.103:0/2354395110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f64a0 0x55df7970d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.548+0000 7f26135f3740 1 -- 172.21.15.103:0/2354395110 >> 172.21.15.103:0/2354395110 conn(0x55df796f2a30 msgr2=0x55df796fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:03.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.548+0000 7f26135f3740 1 -- 172.21.15.103:0/2354395110 shutdown_connections 2024-09-19T06:59:03.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.548+0000 7f26135f3740 1 -- 172.21.15.103:0/2354395110 wait complete. 2024-09-19T06:59:03.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f26135f3740 1 -- start start 2024-09-19T06:59:03.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f26135f3740 1 Processor -- start 2024-09-19T06:59:03.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f26135f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df796f64a0 0x55df79706b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f26135f3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f6dc0 0x55df797070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f26135f3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df79779b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f26135f3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1aee10 con 0x55df79701680 2024-09-19T06:59:03.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f26135f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1ae740 con 0x55df796f64a0 2024-09-19T06:59:03.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f26135f3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1ab9a0 con 0x55df796f6dc0 2024-09-19T06:59:03.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f26135f3740 10 client.0 _ll_register_callbacks cb 0x7f260401e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:03.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f260b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df79779b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f260b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df79779b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52416/0 (socket says 172.21.15.103:52416) 2024-09-19T06:59:03.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f260b59e640 1 -- 172.21.15.103:0/760858093 learned_addr learned my addr 172.21.15.103:0/760858093 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:03.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f260ad9d640 1 --2- 172.21.15.103:0/760858093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df796f64a0 0x55df79706b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f260a59c640 1 --2- 172.21.15.103:0/760858093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f6dc0 0x55df797070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f260b59e640 1 -- 172.21.15.103:0/760858093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f6dc0 msgr2=0x55df797070a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:03.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f260b59e640 1 --2- 172.21.15.103:0/760858093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f6dc0 0x55df797070a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f260b59e640 1 -- 172.21.15.103:0/760858093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df796f64a0 msgr2=0x55df79706b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:03.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f260b59e640 1 --2- 172.21.15.103:0/760858093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df796f64a0 0x55df79706b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f260b59e640 1 -- 172.21.15.103:0/760858093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df79701340 con 0x55df79701680 2024-09-19T06:59:03.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.549+0000 7f260a59c640 1 --2- 172.21.15.103:0/760858093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f6dc0 0x55df797070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:03.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.550+0000 7f260b59e640 1 --2- 172.21.15.103:0/760858093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df79779b50 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f260400e760 tx=0x7f2604003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfd437c10c15793 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:03.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.550+0000 7f25f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:03.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.550+0000 7f25f27fc640 1 -- 172.21.15.103:0/760858093 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604014070 con 0x55df79701680 2024-09-19T06:59:03.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.550+0000 7f25f27fc640 1 -- 172.21.15.103:0/760858093 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f2604003fa0 con 0x55df79701680 2024-09-19T06:59:03.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.550+0000 7f26135f3740 1 -- 172.21.15.103:0/760858093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f260400eb60 con 0x55df79701680 2024-09-19T06:59:03.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.550+0000 7f25f27fc640 1 -- 172.21.15.103:0/760858093 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f260404c050 con 0x55df79701680 2024-09-19T06:59:03.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.550+0000 7f26135f3740 10 client.18768 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:03.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.550+0000 7f26135f3740 1 -- 172.21.15.103:0/760858093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df7977b180 con 0x55df79701680 2024-09-19T06:59:03.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.550+0000 7f25f27fc640 1 -- 172.21.15.103:0/760858093 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f260400eb60 con 0x55df79701680 2024-09-19T06:59:03.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.550+0000 7f25f27fc640 1 -- 172.21.15.103:0/760858093 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f260402c040 con 0x55df79701680 2024-09-19T06:59:03.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.550+0000 7f26135f3740 10 client.18768 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:03.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.550+0000 7f26135f3740 10 client.18768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:03.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.550+0000 7f26135f3740 20 client.18768 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:03.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.550+0000 7f26135f3740 1 --2- 172.21.15.103:0/760858093 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55df7977b7d0 0x55df7977dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:03.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.550+0000 7f26135f3740 4 client.18768 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:59:03.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.550+0000 7f26135f3740 1 -- 172.21.15.103:0/760858093 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df79663d60 con 0x55df7977b7d0 2024-09-19T06:59:03.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f260ad9d640 1 --2- 172.21.15.103:0/760858093 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55df7977b7d0 0x55df7977dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:03.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f25f27fc640 10 client.18768 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:03.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f260ad9d640 1 --2- 172.21.15.103:0/760858093 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55df7977b7d0 0x55df7977dbc0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aed3ea8afa4f520 server_cookie=e8a6885b77edfdba in_seq=0 out_seq=0 2024-09-19T06:59:03.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f25f27fc640 1 -- 172.21.15.103:0/760858093 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55df79663d60 con 0x55df7977b7d0 2024-09-19T06:59:03.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f25f27fc640 10 client.18768 handle_command_reply: tid=0 2024-09-19T06:59:03.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f26135f3740 1 client.18768 shutdown 2024-09-19T06:59:03.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f26135f3740 1 -- 172.21.15.103:0/760858093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 msgr2=0x55df79779b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f26135f3740 1 --2- 172.21.15.103:0/760858093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df79779b50 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f260400e760 tx=0x7f2604003e30 comp rx=0 tx=0).stop 2024-09-19T06:59:03.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f26135f3740 1 -- 172.21.15.103:0/760858093 shutdown_connections 2024-09-19T06:59:03.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f26135f3740 1 --2- 172.21.15.103:0/760858093 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55df7977b7d0 0x55df7977dbc0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f26135f3740 1 --2- 172.21.15.103:0/760858093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df79701680 0x55df79779b50 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f25f27fc640 0 client.18768 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:03.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f26135f3740 1 --2- 172.21.15.103:0/760858093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df796f6dc0 0x55df797070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f26135f3740 1 --2- 172.21.15.103:0/760858093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df796f64a0 0x55df79706b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f26135f3740 1 -- 172.21.15.103:0/760858093 >> 172.21.15.103:0/760858093 conn(0x55df796f2a30 msgr2=0x55df796f1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:03.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f26135f3740 1 -- 172.21.15.103:0/760858093 shutdown_connections 2024-09-19T06:59:03.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.551+0000 7f26135f3740 1 -- 172.21.15.103:0/760858093 wait complete. 2024-09-19T06:59:03.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.552+0000 7f26135f3740 20 client.18768 trim_cache size 0 max 16384 2024-09-19T06:59:03.559 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:59:03.559 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T06:59:03.559 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T06:59:03.559 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:59:03.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.552+0000 7f261121a640 1 -- 172.21.15.103:0/1614338257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f25f80a33b0 msgr2=0x7f25f80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.552+0000 7f261121a640 1 --2- 172.21.15.103:0/1614338257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f25f80a33b0 0x7f25f80a5870 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f260401d000 tx=0x7f2604020000 comp rx=0 tx=0).stop 2024-09-19T06:59:03.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.552+0000 7f261121a640 1 -- 172.21.15.103:0/1614338257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f260c102570 msgr2=0x7f260c199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:03.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.552+0000 7f261121a640 1 --2- 172.21.15.103:0/1614338257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f260c102570 0x7f260c199930 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f25f400b4a0 tx=0x7f25f400b960 comp rx=0 tx=0).stop 2024-09-19T06:59:03.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.552+0000 7f261121a640 1 -- 172.21.15.103:0/1614338257 shutdown_connections 2024-09-19T06:59:03.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.552+0000 7f261121a640 1 --2- 172.21.15.103:0/1614338257 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f25f80a33b0 0x7f25f80a5870 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.552+0000 7f261121a640 1 --2- 172.21.15.103:0/1614338257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f260c102ef0 0x7f260c1a4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.552+0000 7f261121a640 1 --2- 172.21.15.103:0/1614338257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f260c102570 0x7f260c199930 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.552+0000 7f261121a640 1 --2- 172.21.15.103:0/1614338257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f260c100e20 0x7f260c1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:03.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.552+0000 7f261121a640 1 -- 172.21.15.103:0/1614338257 >> 172.21.15.103:0/1614338257 conn(0x7f260c0fc7d0 msgr2=0x7f260c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:03.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.552+0000 7f261121a640 1 -- 172.21.15.103:0/1614338257 shutdown_connections 2024-09-19T06:59:03.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:03.552+0000 7f261121a640 1 -- 172.21.15.103:0/1614338257 wait complete. 2024-09-19T06:59:03.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:03 smithi103 ceph-mon[31158]: pgmap v303: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-09-19T06:59:03.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:03 smithi134 ceph-mon[35356]: pgmap v303: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-09-19T06:59:04.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:03 smithi174 ceph-mon[35637]: pgmap v303: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-09-19T06:59:04.320 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:59:04.321 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:59:04.571 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:05.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.051+0000 7eff4ed18640 1 -- 172.21.15.103:0/3771157857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff48102d30 msgr2=0x7eff48103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:05.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.051+0000 7eff4ed18640 1 --2- 172.21.15.103:0/3771157857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff48102d30 0x7eff48103110 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7eff40009930 tx=0x7eff4002a440 comp rx=0 tx=0).stop 2024-09-19T06:59:05.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.051+0000 7eff4ed18640 1 -- 172.21.15.103:0/3771157857 shutdown_connections 2024-09-19T06:59:05.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.051+0000 7eff4ed18640 1 --2- 172.21.15.103:0/3771157857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff481082a0 0x7eff4810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:05.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.051+0000 7eff4ed18640 1 --2- 172.21.15.103:0/3771157857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff481036e0 0x7eff48107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:05.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.051+0000 7eff4ed18640 1 --2- 172.21.15.103:0/3771157857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff48102d30 0x7eff48103110 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:05.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.051+0000 7eff4ed18640 1 -- 172.21.15.103:0/3771157857 >> 172.21.15.103:0/3771157857 conn(0x7eff480fc7d0 msgr2=0x7eff480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:05.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.051+0000 7eff4ed18640 1 -- 172.21.15.103:0/3771157857 shutdown_connections 2024-09-19T06:59:05.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.051+0000 7eff4ed18640 1 -- 172.21.15.103:0/3771157857 wait complete. 2024-09-19T06:59:05.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.051+0000 7eff4ed18640 1 Processor -- start 2024-09-19T06:59:05.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.051+0000 7eff4ed18640 1 -- start start 2024-09-19T06:59:05.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff4ed18640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff48102d30 0x7eff48100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:05.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff4ed18640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff481036e0 0x7eff48100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:05.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff4ed18640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff481082a0 0x7eff48101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:05.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff4ed18640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7eff48112160 con 0x7eff481036e0 2024-09-19T06:59:05.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff4ed18640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff48111fe0 con 0x7eff481082a0 2024-09-19T06:59:05.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff4ed18640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7eff481122e0 con 0x7eff48102d30 2024-09-19T06:59:05.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff481036e0 0x7eff48100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:05.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff481036e0 0x7eff48100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52436/0 (socket says 172.21.15.103:52436) 2024-09-19T06:59:05.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff47fff640 1 -- 172.21.15.103:0/590896238 learned_addr learned my addr 172.21.15.103:0/590896238 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:05.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff4cab4640 1 --2- 172.21.15.103:0/590896238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff48102d30 0x7eff48100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:05.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff47fff640 1 -- 172.21.15.103:0/590896238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff48102d30 msgr2=0x7eff48100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:05.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff47fff640 1 --2- 172.21.15.103:0/590896238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff48102d30 0x7eff48100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:05.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff47fff640 1 -- 172.21.15.103:0/590896238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff481082a0 msgr2=0x7eff48101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:05.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff47fff640 1 --2- 172.21.15.103:0/590896238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff481082a0 0x7eff48101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:05.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff47fff640 1 -- 172.21.15.103:0/590896238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff481a6be0 con 0x7eff481036e0 2024-09-19T06:59:05.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff47fff640 1 --2- 172.21.15.103:0/590896238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff481036e0 0x7eff48100ed0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7eff3800b990 tx=0x7eff3800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:05.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff45ffb640 1 -- 172.21.15.103:0/590896238 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38019070 con 0x7eff481036e0 2024-09-19T06:59:05.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff45ffb640 1 -- 172.21.15.103:0/590896238 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff3800cba0 con 0x7eff481036e0 2024-09-19T06:59:05.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff4ed18640 1 -- 172.21.15.103:0/590896238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff481a6ed0 con 0x7eff481036e0 2024-09-19T06:59:05.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.052+0000 7eff45ffb640 1 -- 172.21.15.103:0/590896238 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff380126b0 con 0x7eff481036e0 2024-09-19T06:59:05.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.053+0000 7eff4ed18640 1 -- 172.21.15.103:0/590896238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff481a73d0 con 0x7eff481036e0 2024-09-19T06:59:05.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.054+0000 7eff4ed18640 1 -- 172.21.15.103:0/590896238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff4806a850 con 0x7eff481036e0 2024-09-19T06:59:05.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.055+0000 7eff45ffb640 1 -- 172.21.15.103:0/590896238 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7eff3801c070 con 0x7eff481036e0 2024-09-19T06:59:05.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.056+0000 7eff45ffb640 1 --2- 172.21.15.103:0/590896238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff340a3430 0x7eff340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:05.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.056+0000 7eff4cab4640 1 --2- 172.21.15.103:0/590896238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff340a3430 0x7eff340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:05.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.056+0000 7eff45ffb640 1 -- 172.21.15.103:0/590896238 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7eff380cf0e0 con 0x7eff481036e0 2024-09-19T06:59:05.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.056+0000 7eff4cab4640 1 --2- 172.21.15.103:0/590896238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff340a3430 0x7eff340a58f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7eff40000c00 tx=0x7eff40031490 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:05.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.060+0000 7eff45ffb640 1 -- 172.21.15.103:0/590896238 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7eff3809bc50 con 0x7eff481036e0 2024-09-19T06:59:05.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.265+0000 7eff4ed18640 1 -- 172.21.15.103:0/590896238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff4802d7c0 con 0x7eff481036e0 2024-09-19T06:59:05.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.266+0000 7eff45ffb640 1 -- 172.21.15.103:0/590896238 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7eff3809b550 con 0x7eff481036e0 2024-09-19T06:59:05.267 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:59:05.267 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:59:05.268 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:59:05.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.268+0000 7eff4ed18640 1 -- 172.21.15.103:0/590896238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff340a3430 msgr2=0x7eff340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:05.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.268+0000 7eff4ed18640 1 --2- 172.21.15.103:0/590896238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff340a3430 0x7eff340a58f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7eff40000c00 tx=0x7eff40031490 comp rx=0 tx=0).stop 2024-09-19T06:59:05.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.268+0000 7eff4ed18640 1 -- 172.21.15.103:0/590896238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff481036e0 msgr2=0x7eff48100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:05.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.268+0000 7eff4ed18640 1 --2- 172.21.15.103:0/590896238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff481036e0 0x7eff48100ed0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7eff3800b990 tx=0x7eff3800be50 comp rx=0 tx=0).stop 2024-09-19T06:59:05.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.268+0000 7eff4ed18640 1 -- 172.21.15.103:0/590896238 shutdown_connections 2024-09-19T06:59:05.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.268+0000 7eff4ed18640 1 --2- 172.21.15.103:0/590896238 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff340a3430 0x7eff340a58f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:05.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.268+0000 7eff4ed18640 1 --2- 172.21.15.103:0/590896238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff481082a0 0x7eff48101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:05.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.268+0000 7eff4ed18640 1 --2- 172.21.15.103:0/590896238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff481036e0 0x7eff48100ed0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:05.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.268+0000 7eff4ed18640 1 --2- 172.21.15.103:0/590896238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff48102d30 0x7eff48100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:05.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.268+0000 7eff4ed18640 1 -- 172.21.15.103:0/590896238 >> 172.21.15.103:0/590896238 conn(0x7eff480fc7d0 msgr2=0x7eff48105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:05.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.269+0000 7eff4ed18640 1 -- 172.21.15.103:0/590896238 shutdown_connections 2024-09-19T06:59:05.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:05.269+0000 7eff4ed18640 1 -- 172.21.15.103:0/590896238 wait complete. 2024-09-19T06:59:05.701 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T06:59:05.701 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-19T06:59:05.702 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:59:05.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:05 smithi103 ceph-mon[31158]: pgmap v304: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-09-19T06:59:05.888 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:05 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/590896238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:59:05.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:05 smithi134 ceph-mon[35356]: pgmap v304: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-09-19T06:59:05.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:05 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/590896238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:59:06.041 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:59:06.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:05 smithi174 ceph-mon[35637]: pgmap v304: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-09-19T06:59:06.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:05 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/590896238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:59:06.087 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:06.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.559+0000 7f1a290ca640 1 -- 172.21.15.103:0/546033636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a24101440 msgr2=0x7f1a24101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:06.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.559+0000 7f1a290ca640 1 --2- 172.21.15.103:0/546033636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a24101440 0x7f1a24101820 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009900 tx=0x7f1a1402a410 comp rx=0 tx=0).stop 2024-09-19T06:59:06.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.559+0000 7f1a290ca640 1 -- 172.21.15.103:0/546033636 shutdown_connections 2024-09-19T06:59:06.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.559+0000 7f1a290ca640 1 --2- 172.21.15.103:0/546033636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a24106050 0x7f1a2410dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:06.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.559+0000 7f1a290ca640 1 --2- 172.21.15.103:0/546033636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a24101df0 0x7f1a24105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:06.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.559+0000 7f1a290ca640 1 --2- 172.21.15.103:0/546033636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a24101440 0x7f1a24101820 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:06.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.559+0000 7f1a290ca640 1 -- 172.21.15.103:0/546033636 >> 172.21.15.103:0/546033636 conn(0x7f1a24077210 msgr2=0x7f1a240fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:06.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.559+0000 7f1a290ca640 1 -- 172.21.15.103:0/546033636 shutdown_connections 2024-09-19T06:59:06.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.559+0000 7f1a290ca640 1 -- 172.21.15.103:0/546033636 wait complete. 2024-09-19T06:59:06.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.560+0000 7f1a290ca640 1 Processor -- start 2024-09-19T06:59:06.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.560+0000 7f1a290ca640 1 -- start start 2024-09-19T06:59:06.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.560+0000 7f1a290ca640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a24101440 0x7f1a241996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:06.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.560+0000 7f1a290ca640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a24101df0 0x7f1a24199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:06.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.560+0000 7f1a290ca640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a24106050 0x7f1a2419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:06.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.560+0000 7f1a290ca640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1a24111150 con 0x7f1a24101df0 2024-09-19T06:59:06.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.560+0000 7f1a290ca640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1a24110fd0 con 0x7f1a24101440 2024-09-19T06:59:06.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.560+0000 7f1a290ca640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1a241112d0 con 0x7f1a24106050 2024-09-19T06:59:06.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.560+0000 7f1a2259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a24101df0 0x7f1a24199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:06.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.560+0000 7f1a2259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a24101df0 0x7f1a24199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52454/0 (socket says 172.21.15.103:52454) 2024-09-19T06:59:06.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.560+0000 7f1a2259c640 1 -- 172.21.15.103:0/1987439344 learned_addr learned my addr 172.21.15.103:0/1987439344 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:06.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.560+0000 7f1a22d9d640 1 --2- 172.21.15.103:0/1987439344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a24101440 0x7f1a241996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:06.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.561+0000 7f1a2359e640 1 --2- 172.21.15.103:0/1987439344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a24106050 0x7f1a2419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:06.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.561+0000 7f1a2259c640 1 -- 172.21.15.103:0/1987439344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a24106050 msgr2=0x7f1a2419a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:06.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.561+0000 7f1a2259c640 1 --2- 172.21.15.103:0/1987439344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a24106050 0x7f1a2419a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:06.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.561+0000 7f1a2259c640 1 -- 172.21.15.103:0/1987439344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a24101440 msgr2=0x7f1a241996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:06.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.561+0000 7f1a2259c640 1 --2- 172.21.15.103:0/1987439344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a24101440 0x7f1a241996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:06.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.561+0000 7f1a2259c640 1 -- 172.21.15.103:0/1987439344 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a241a4a90 con 0x7f1a24101df0 2024-09-19T06:59:06.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.561+0000 7f1a2359e640 1 --2- 172.21.15.103:0/1987439344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a24106050 0x7f1a2419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:06.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.561+0000 7f1a2259c640 1 --2- 172.21.15.103:0/1987439344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a24101df0 0x7f1a24199c20 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00b4a0 tx=0x7f1a0c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:06.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.561+0000 7f1a1bfff640 1 -- 172.21.15.103:0/1987439344 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c014070 con 0x7f1a24101df0 2024-09-19T06:59:06.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.561+0000 7f1a1bfff640 1 -- 172.21.15.103:0/1987439344 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a0c010af0 con 0x7f1a24101df0 2024-09-19T06:59:06.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.561+0000 7f1a290ca640 1 -- 172.21.15.103:0/1987439344 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a241a4d80 con 0x7f1a24101df0 2024-09-19T06:59:06.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.561+0000 7f1a1bfff640 1 -- 172.21.15.103:0/1987439344 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c017d10 con 0x7f1a24101df0 2024-09-19T06:59:06.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.561+0000 7f1a290ca640 1 -- 172.21.15.103:0/1987439344 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a241a5260 con 0x7f1a24101df0 2024-09-19T06:59:06.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.562+0000 7f1a290ca640 1 -- 172.21.15.103:0/1987439344 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a241018a0 con 0x7f1a24101df0 2024-09-19T06:59:06.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.563+0000 7f1a1bfff640 1 -- 172.21.15.103:0/1987439344 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1a0c017070 con 0x7f1a24101df0 2024-09-19T06:59:06.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.564+0000 7f1a1bfff640 1 --2- 172.21.15.103:0/1987439344 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1a100a33b0 0x7f1a100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:06.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.564+0000 7f1a22d9d640 1 --2- 172.21.15.103:0/1987439344 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1a100a33b0 0x7f1a100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:06.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.564+0000 7f1a1bfff640 1 -- 172.21.15.103:0/1987439344 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7f1a0c0cefd0 con 0x7f1a24101df0 2024-09-19T06:59:06.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.565+0000 7f1a22d9d640 1 --2- 172.21.15.103:0/1987439344 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1a100a33b0 0x7f1a100a5870 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1a14000c00 tx=0x7f1a140058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:06.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.568+0000 7f1a1bfff640 1 -- 172.21.15.103:0/1987439344 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1a0c09ba70 con 0x7f1a24101df0 2024-09-19T06:59:06.609 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:06.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.779+0000 7f1a290ca640 1 -- 172.21.15.103:0/1987439344 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a241006c0 con 0x7f1a24101df0 2024-09-19T06:59:06.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.779+0000 7f1a1bfff640 1 -- 172.21.15.103:0/1987439344 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f1a0c09b370 con 0x7f1a24101df0 2024-09-19T06:59:06.780 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:59:06.781 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:59:06.781 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:59:06.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.781+0000 7f1a290ca640 1 -- 172.21.15.103:0/1987439344 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1a100a33b0 msgr2=0x7f1a100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:06.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.781+0000 7f1a290ca640 1 --2- 172.21.15.103:0/1987439344 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1a100a33b0 0x7f1a100a5870 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1a14000c00 tx=0x7f1a140058d0 comp rx=0 tx=0).stop 2024-09-19T06:59:06.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.781+0000 7f1a290ca640 1 -- 172.21.15.103:0/1987439344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a24101df0 msgr2=0x7f1a24199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:06.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.781+0000 7f1a290ca640 1 --2- 172.21.15.103:0/1987439344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a24101df0 0x7f1a24199c20 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00b4a0 tx=0x7f1a0c00b960 comp rx=0 tx=0).stop 2024-09-19T06:59:06.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.782+0000 7f1a290ca640 1 -- 172.21.15.103:0/1987439344 shutdown_connections 2024-09-19T06:59:06.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.782+0000 7f1a290ca640 1 --2- 172.21.15.103:0/1987439344 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1a100a33b0 0x7f1a100a5870 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:06.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.782+0000 7f1a290ca640 1 --2- 172.21.15.103:0/1987439344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a24106050 0x7f1a2419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:06.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.782+0000 7f1a290ca640 1 --2- 172.21.15.103:0/1987439344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a24101df0 0x7f1a24199c20 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:06.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.782+0000 7f1a290ca640 1 --2- 172.21.15.103:0/1987439344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a24101440 0x7f1a241996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:06.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.782+0000 7f1a290ca640 1 -- 172.21.15.103:0/1987439344 >> 172.21.15.103:0/1987439344 conn(0x7f1a24077210 msgr2=0x7f1a24104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:06.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.782+0000 7f1a290ca640 1 -- 172.21.15.103:0/1987439344 shutdown_connections 2024-09-19T06:59:06.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:06.782+0000 7f1a290ca640 1 -- 172.21.15.103:0/1987439344 wait complete. 2024-09-19T06:59:06.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:06 smithi134 ceph-mon[35356]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T06:59:07.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:06 smithi174 ceph-mon[35637]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T06:59:07.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:06 smithi103 ceph-mon[31158]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T06:59:07.604 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-19T06:59:07.604 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-19T06:59:07.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.808+0000 7fed2d6e0640 1 -- 172.21.15.103:0/287206641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed28102d30 msgr2=0x7fed28103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:07.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.808+0000 7fed2d6e0640 1 --2- 172.21.15.103:0/287206641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed28102d30 0x7fed28103110 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fed10009900 tx=0x7fed1002a480 comp rx=0 tx=0).stop 2024-09-19T06:59:07.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.808+0000 7fed2d6e0640 1 -- 172.21.15.103:0/287206641 shutdown_connections 2024-09-19T06:59:07.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.808+0000 7fed2d6e0640 1 --2- 172.21.15.103:0/287206641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed2810c160 0x7fed2810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.808+0000 7fed2d6e0640 1 --2- 172.21.15.103:0/287206641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed28103650 0x7fed2810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.808+0000 7fed2d6e0640 1 --2- 172.21.15.103:0/287206641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed28102d30 0x7fed28103110 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.808+0000 7fed2d6e0640 1 -- 172.21.15.103:0/287206641 >> 172.21.15.103:0/287206641 conn(0x7fed280fc790 msgr2=0x7fed280febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:07.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.809+0000 7fed2d6e0640 1 -- 172.21.15.103:0/287206641 shutdown_connections 2024-09-19T06:59:07.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.809+0000 7fed2d6e0640 1 -- 172.21.15.103:0/287206641 wait complete. 2024-09-19T06:59:07.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.809+0000 7fed2d6e0640 1 Processor -- start 2024-09-19T06:59:07.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.809+0000 7fed2d6e0640 1 -- start start 2024-09-19T06:59:07.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.809+0000 7fed2d6e0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed28102d30 0x7fed2819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.809+0000 7fed2d6e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed28103650 0x7fed2819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed2d6e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed2810c160 0x7fed2819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed2d6e0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed28111f60 con 0x7fed2810c160 2024-09-19T06:59:07.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed2d6e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed28111de0 con 0x7fed28103650 2024-09-19T06:59:07.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed2d6e0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed281120e0 con 0x7fed28102d30 2024-09-19T06:59:07.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed277fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed2810c160 0x7fed2819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed277fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed2810c160 0x7fed2819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60468/0 (socket says 172.21.15.103:60468) 2024-09-19T06:59:07.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed277fe640 1 -- 172.21.15.103:0/1345321592 learned_addr learned my addr 172.21.15.103:0/1345321592 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:07.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed267fc640 1 --2- 172.21.15.103:0/1345321592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed28103650 0x7fed2819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed26ffd640 1 --2- 172.21.15.103:0/1345321592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed28102d30 0x7fed2819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed277fe640 1 -- 172.21.15.103:0/1345321592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed28102d30 msgr2=0x7fed2819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:07.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed277fe640 1 --2- 172.21.15.103:0/1345321592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed28102d30 0x7fed2819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed277fe640 1 -- 172.21.15.103:0/1345321592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed28103650 msgr2=0x7fed2819ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:07.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed277fe640 1 --2- 172.21.15.103:0/1345321592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed28103650 0x7fed2819ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed277fe640 1 -- 172.21.15.103:0/1345321592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2819f6a0 con 0x7fed2810c160 2024-09-19T06:59:07.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed277fe640 1 --2- 172.21.15.103:0/1345321592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed2810c160 0x7fed2819efc0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fed2000eb10 tx=0x7fed2000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:07.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed1ffff640 1 -- 172.21.15.103:0/1345321592 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20004b20 con 0x7fed2810c160 2024-09-19T06:59:07.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed2d6e0640 1 -- 172.21.15.103:0/1345321592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2819f990 con 0x7fed2810c160 2024-09-19T06:59:07.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed1ffff640 1 -- 172.21.15.103:0/1345321592 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed20004cc0 con 0x7fed2810c160 2024-09-19T06:59:07.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.810+0000 7fed1ffff640 1 -- 172.21.15.103:0/1345321592 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20010760 con 0x7fed2810c160 2024-09-19T06:59:07.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.811+0000 7fed2d6e0640 1 -- 172.21.15.103:0/1345321592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed281a7270 con 0x7fed2810c160 2024-09-19T06:59:07.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.813+0000 7fed1ffff640 1 -- 172.21.15.103:0/1345321592 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fed20005020 con 0x7fed2810c160 2024-09-19T06:59:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.813+0000 7fed2fab9740 1 Processor -- start 2024-09-19T06:59:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.813+0000 7fed2fab9740 1 -- start start 2024-09-19T06:59:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed2fab9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555945581b40 0x555945580090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed2fab9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5559455805d0 0x555945580a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed2fab9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55594558b370 0x55594558d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed2fab9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5559454e62f0 con 0x5559455805d0 2024-09-19T06:59:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed2fab9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55594548dcf0 con 0x55594558b370 2024-09-19T06:59:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed2fab9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5559454e22b0 con 0x555945581b40 2024-09-19T06:59:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed26ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5559455805d0 0x555945580a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed26ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5559455805d0 0x555945580a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60480/0 (socket says 172.21.15.103:60480) 2024-09-19T06:59:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed26ffd640 1 -- 172.21.15.103:0/1439713096 learned_addr learned my addr 172.21.15.103:0/1439713096 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed1ffff640 1 --2- 172.21.15.103:0/1345321592 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed140a33b0 0x7fed140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed267fc640 1 --2- 172.21.15.103:0/1439713096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555945581b40 0x555945580090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed277fe640 1 --2- 172.21.15.103:0/1345321592 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed140a33b0 0x7fed140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed267fc640 1 --2- 172.21.15.103:0/1439713096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55594558b370 0x55594558d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed26ffd640 1 -- 172.21.15.103:0/1439713096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555945581b40 msgr2=0x555945580090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed26ffd640 1 --2- 172.21.15.103:0/1439713096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555945581b40 0x555945580090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed26ffd640 1 -- 172.21.15.103:0/1439713096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55594558b370 msgr2=0x55594558d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed26ffd640 1 --2- 172.21.15.103:0/1439713096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55594558b370 0x55594558d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed26ffd640 1 -- 172.21.15.103:0/1439713096 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559454723a0 con 0x5559455805d0 2024-09-19T06:59:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed267fc640 1 --2- 172.21.15.103:0/1439713096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55594558b370 0x55594558d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed1ffff640 1 -- 172.21.15.103:0/1345321592 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7fed200d03b0 con 0x7fed2810c160 2024-09-19T06:59:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.814+0000 7fed26ffd640 1 --2- 172.21.15.103:0/1439713096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5559455805d0 0x555945580a10 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fed10009900 tx=0x7fed10005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b8e7f19e25f0ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.815+0000 7fed1d7fa640 1 -- 172.21.15.103:0/1439713096 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10040070 con 0x5559455805d0 2024-09-19T06:59:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.815+0000 7fed277fe640 1 --2- 172.21.15.103:0/1345321592 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed140a33b0 0x7fed140a5870 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fed20098aa0 tx=0x7fed2006e3c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:07.829 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:07 smithi103 ceph-mon[31158]: pgmap v306: 97 pgs: 97 active+clean; 1.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-09-19T06:59:07.829 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:07 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1987439344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:59:07.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.815+0000 7fed1d7fa640 1 -- 172.21.15.103:0/1439713096 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed100318f0 con 0x5559455805d0 2024-09-19T06:59:07.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.815+0000 7fed1d7fa640 1 -- 172.21.15.103:0/1439713096 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10035040 con 0x5559455805d0 2024-09-19T06:59:07.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.815+0000 7fed2fab9740 1 -- 172.21.15.103:0/1439713096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5559455805d0 msgr2=0x555945580a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:07.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.815+0000 7fed2fab9740 1 --2- 172.21.15.103:0/1439713096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5559455805d0 0x555945580a10 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fed10009900 tx=0x7fed10005e00 comp rx=0 tx=0).stop 2024-09-19T06:59:07.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed2fab9740 1 -- 172.21.15.103:0/1439713096 shutdown_connections 2024-09-19T06:59:07.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed2fab9740 1 --2- 172.21.15.103:0/1439713096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55594558b370 0x55594558d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed2fab9740 1 --2- 172.21.15.103:0/1439713096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5559455805d0 0x555945580a10 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed2fab9740 1 --2- 172.21.15.103:0/1439713096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555945581b40 0x555945580090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed2fab9740 1 -- 172.21.15.103:0/1439713096 >> 172.21.15.103:0/1439713096 conn(0x555945549310 msgr2=0x55594557cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed2fab9740 1 -- 172.21.15.103:0/1439713096 shutdown_connections 2024-09-19T06:59:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed2fab9740 1 -- 172.21.15.103:0/1439713096 wait complete. 2024-09-19T06:59:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed2fab9740 1 -- start start 2024-09-19T06:59:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed2fab9740 1 Processor -- start 2024-09-19T06:59:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed2fab9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559455805d0 0x55594558ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed2fab9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555945581b40 0x555945587fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed2fab9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55594558b370 0x555945588570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed2fab9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55594558e010 con 0x555945581b40 2024-09-19T06:59:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed2fab9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5559454a1060 con 0x55594558b370 2024-09-19T06:59:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed2fab9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55594558e190 con 0x5559455805d0 2024-09-19T06:59:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed2fab9740 10 client.0 _ll_register_callbacks cb 0x5559451f48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed26ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555945581b40 0x555945587fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed26ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555945581b40 0x555945587fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60490/0 (socket says 172.21.15.103:60490) 2024-09-19T06:59:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed26ffd640 1 -- 172.21.15.103:0/2706426749 learned_addr learned my addr 172.21.15.103:0/2706426749 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.816+0000 7fed267fc640 1 --2- 172.21.15.103:0/2706426749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55594558b370 0x555945588570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed267fc640 1 --2- 172.21.15.103:0/2706426749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559455805d0 0x55594558ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed26ffd640 1 -- 172.21.15.103:0/2706426749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559455805d0 msgr2=0x55594558ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed26ffd640 1 --2- 172.21.15.103:0/2706426749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559455805d0 0x55594558ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed26ffd640 1 -- 172.21.15.103:0/2706426749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55594558b370 msgr2=0x555945588570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed26ffd640 1 --2- 172.21.15.103:0/2706426749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55594558b370 0x555945588570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed26ffd640 1 -- 172.21.15.103:0/2706426749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559452573c0 con 0x555945581b40 2024-09-19T06:59:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed267fc640 1 --2- 172.21.15.103:0/2706426749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559455805d0 0x55594558ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed26ffd640 1 --2- 172.21.15.103:0/2706426749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555945581b40 0x555945587fd0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fed10005940 tx=0x7fed10005470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26213e89089cd903 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed0e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed0e7fc640 1 -- 172.21.15.103:0/2706426749 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1004c070 con 0x555945581b40 2024-09-19T06:59:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed0e7fc640 1 -- 172.21.15.103:0/2706426749 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7fed100048d0 con 0x555945581b40 2024-09-19T06:59:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed2fab9740 1 -- 172.21.15.103:0/2706426749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed100318f0 con 0x555945581b40 2024-09-19T06:59:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed0e7fc640 1 -- 172.21.15.103:0/2706426749 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed10058050 con 0x555945581b40 2024-09-19T06:59:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed2fab9740 10 client.18792 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed2fab9740 1 -- 172.21.15.103:0/2706426749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5559455999d0 con 0x555945581b40 2024-09-19T06:59:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed0e7fc640 1 -- 172.21.15.103:0/2706426749 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed100318f0 con 0x555945581b40 2024-09-19T06:59:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.817+0000 7fed0e7fc640 1 -- 172.21.15.103:0/2706426749 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fed1003cb00 con 0x555945581b40 2024-09-19T06:59:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.818+0000 7fed2fab9740 10 client.18792 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.818+0000 7fed2fab9740 10 client.18792 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:59:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.818+0000 7fed2fab9740 20 client.18792 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.818+0000 7fed2fab9740 1 --2- 172.21.15.103:0/2706426749 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55594559ad20 0x55594559d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.818+0000 7fed2fab9740 4 client.18792 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.818+0000 7fed2fab9740 1 -- 172.21.15.103:0/2706426749 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559454f3860 con 0x55594559ad20 2024-09-19T06:59:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.818+0000 7fed267fc640 1 --2- 172.21.15.103:0/2706426749 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55594559ad20 0x55594559d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.818+0000 7fed267fc640 1 --2- 172.21.15.103:0/2706426749 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55594559ad20 0x55594559d110 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c976b9d1fd6b8bd8 server_cookie=d0a0d3df1aebc1f3 in_seq=0 out_seq=0 2024-09-19T06:59:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.818+0000 7fed0e7fc640 10 client.18792 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:59:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.819+0000 7fed0e7fc640 1 -- 172.21.15.103:0/2706426749 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559454f3860 con 0x55594559ad20 2024-09-19T06:59:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.819+0000 7fed0e7fc640 10 client.18792 handle_command_reply: tid=0 2024-09-19T06:59:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.819+0000 7fed2fab9740 1 client.18792 shutdown 2024-09-19T06:59:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.819+0000 7fed2fab9740 1 -- 172.21.15.103:0/2706426749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555945581b40 msgr2=0x555945587fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.819+0000 7fed2fab9740 1 --2- 172.21.15.103:0/2706426749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555945581b40 0x555945587fd0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fed10005940 tx=0x7fed10005470 comp rx=0 tx=0).stop 2024-09-19T06:59:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.819+0000 7fed2fab9740 1 -- 172.21.15.103:0/2706426749 shutdown_connections 2024-09-19T06:59:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.819+0000 7fed2fab9740 1 --2- 172.21.15.103:0/2706426749 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55594559ad20 0x55594559d110 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.819+0000 7fed2fab9740 1 --2- 172.21.15.103:0/2706426749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55594558b370 0x555945588570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.819+0000 7fed2fab9740 1 --2- 172.21.15.103:0/2706426749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555945581b40 0x555945587fd0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.819+0000 7fed2fab9740 1 --2- 172.21.15.103:0/2706426749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559455805d0 0x55594558ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.819+0000 7fed0e7fc640 0 client.18792 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:59:07.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.819+0000 7fed2fab9740 1 -- 172.21.15.103:0/2706426749 >> 172.21.15.103:0/2706426749 conn(0x555945549310 msgr2=0x555945582960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:07.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.819+0000 7fed2fab9740 1 -- 172.21.15.103:0/2706426749 shutdown_connections 2024-09-19T06:59:07.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.819+0000 7fed2fab9740 1 -- 172.21.15.103:0/2706426749 wait complete. 2024-09-19T06:59:07.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.819+0000 7fed2fab9740 20 client.18792 trim_cache size 0 max 16384 2024-09-19T06:59:07.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.834+0000 7fed2fab9740 1 Processor -- start 2024-09-19T06:59:07.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.834+0000 7fed2fab9740 1 -- start start 2024-09-19T06:59:07.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.834+0000 7fed2fab9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559455805d0 0x5559455972b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.834+0000 7fed2fab9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555945581b40 0x55594558c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.834+0000 7fed2fab9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55594558b370 0x55594558cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.834+0000 7fed2fab9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed281afc00 con 0x55594558b370 2024-09-19T06:59:07.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.834+0000 7fed2fab9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed281b1740 con 0x555945581b40 2024-09-19T06:59:07.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.834+0000 7fed2fab9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed281b02d0 con 0x5559455805d0 2024-09-19T06:59:07.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.834+0000 7fed267fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55594558b370 0x55594558cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.835+0000 7fed267fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55594558b370 0x55594558cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60504/0 (socket says 172.21.15.103:60504) 2024-09-19T06:59:07.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.835+0000 7fed267fc640 1 -- 172.21.15.103:0/3453448898 learned_addr learned my addr 172.21.15.103:0/3453448898 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:07.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.835+0000 7fed267fc640 1 --2- 172.21.15.103:0/3453448898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559455805d0 0x5559455972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.835+0000 7fed26ffd640 1 --2- 172.21.15.103:0/3453448898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555945581b40 0x55594558c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.835+0000 7fed267fc640 1 -- 172.21.15.103:0/3453448898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559455805d0 msgr2=0x5559455972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:07.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.835+0000 7fed267fc640 1 --2- 172.21.15.103:0/3453448898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559455805d0 0x5559455972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.835+0000 7fed267fc640 1 -- 172.21.15.103:0/3453448898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555945581b40 msgr2=0x55594558c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:07.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.835+0000 7fed267fc640 1 --2- 172.21.15.103:0/3453448898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555945581b40 0x55594558c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.835+0000 7fed267fc640 1 -- 172.21.15.103:0/3453448898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559455888e0 con 0x55594558b370 2024-09-19T06:59:07.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.835+0000 7fed26ffd640 1 --2- 172.21.15.103:0/3453448898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555945581b40 0x55594558c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:07.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.835+0000 7fed267fc640 1 --2- 172.21.15.103:0/3453448898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55594558b370 0x55594558cca0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fed1800be70 tx=0x7fed18005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7020a2e71ccd0708 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:07.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.835+0000 7fed1cff9640 1 -- 172.21.15.103:0/3453448898 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18013050 con 0x55594558b370 2024-09-19T06:59:07.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.835+0000 7fed1cff9640 1 -- 172.21.15.103:0/3453448898 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed18010da0 con 0x55594558b370 2024-09-19T06:59:07.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.835+0000 7fed1cff9640 1 -- 172.21.15.103:0/3453448898 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed180054f0 con 0x55594558b370 2024-09-19T06:59:07.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.836+0000 7fed2fab9740 1 -- 172.21.15.103:0/3453448898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55594558b370 msgr2=0x55594558cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:07.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.836+0000 7fed2fab9740 1 --2- 172.21.15.103:0/3453448898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55594558b370 0x55594558cca0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fed1800be70 tx=0x7fed18005c90 comp rx=0 tx=0).stop 2024-09-19T06:59:07.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.836+0000 7fed2fab9740 1 -- 172.21.15.103:0/3453448898 shutdown_connections 2024-09-19T06:59:07.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.836+0000 7fed2fab9740 1 --2- 172.21.15.103:0/3453448898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55594558b370 0x55594558cca0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.836+0000 7fed2fab9740 1 --2- 172.21.15.103:0/3453448898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555945581b40 0x55594558c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.836+0000 7fed2fab9740 1 --2- 172.21.15.103:0/3453448898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559455805d0 0x5559455972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.836+0000 7fed2fab9740 1 -- 172.21.15.103:0/3453448898 >> 172.21.15.103:0/3453448898 conn(0x555945549310 msgr2=0x5559455868a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:07.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.836+0000 7fed2fab9740 1 -- 172.21.15.103:0/3453448898 shutdown_connections 2024-09-19T06:59:07.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.836+0000 7fed2fab9740 1 -- 172.21.15.103:0/3453448898 wait complete. 2024-09-19T06:59:07.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.837+0000 7fed2fab9740 1 -- start start 2024-09-19T06:59:07.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.837+0000 7fed2fab9740 1 Processor -- start 2024-09-19T06:59:07.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.837+0000 7fed2fab9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559455805d0 0x555945593700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.837+0000 7fed2fab9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555945581b40 0x555945593c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.837+0000 7fed2fab9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55594558b370 0x555945585340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.837+0000 7fed2fab9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed281b1070 con 0x55594558b370 2024-09-19T06:59:07.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.837+0000 7fed2fab9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed281b09a0 con 0x5559455805d0 2024-09-19T06:59:07.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.837+0000 7fed2fab9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed281adc00 con 0x555945581b40 2024-09-19T06:59:07.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.837+0000 7fed2fab9740 10 client.0 _ll_register_callbacks cb 0x7fed10005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:07.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.837+0000 7fed267fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55594558b370 0x555945585340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.837+0000 7fed267fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55594558b370 0x555945585340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60516/0 (socket says 172.21.15.103:60516) 2024-09-19T06:59:07.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.837+0000 7fed267fc640 1 -- 172.21.15.103:0/408255058 learned_addr learned my addr 172.21.15.103:0/408255058 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:07.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.837+0000 7fed26ffd640 1 --2- 172.21.15.103:0/408255058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555945581b40 0x555945593c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.838+0000 7fed267fc640 1 --2- 172.21.15.103:0/408255058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559455805d0 0x555945593700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.838+0000 7fed267fc640 1 -- 172.21.15.103:0/408255058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555945581b40 msgr2=0x555945593c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:07.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.838+0000 7fed267fc640 1 --2- 172.21.15.103:0/408255058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555945581b40 0x555945593c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.838+0000 7fed267fc640 1 -- 172.21.15.103:0/408255058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559455805d0 msgr2=0x555945593700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:07.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.838+0000 7fed267fc640 1 --2- 172.21.15.103:0/408255058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559455805d0 0x555945593700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.838+0000 7fed267fc640 1 -- 172.21.15.103:0/408255058 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555945585b40 con 0x55594558b370 2024-09-19T06:59:07.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.838+0000 7fed267fc640 1 --2- 172.21.15.103:0/408255058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55594558b370 0x555945585340 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fed1800bd80 tx=0x7fed1801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e19e9c1f61e45cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:07.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.838+0000 7fed0e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:07.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.838+0000 7fed0e7fc640 1 -- 172.21.15.103:0/408255058 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1802c020 con 0x55594558b370 2024-09-19T06:59:07.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.838+0000 7fed0e7fc640 1 -- 172.21.15.103:0/408255058 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7fed18018070 con 0x55594558b370 2024-09-19T06:59:07.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.838+0000 7fed2fab9740 1 -- 172.21.15.103:0/408255058 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed18010da0 con 0x55594558b370 2024-09-19T06:59:07.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.839+0000 7fed0e7fc640 1 -- 172.21.15.103:0/408255058 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed1803d050 con 0x55594558b370 2024-09-19T06:59:07.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.839+0000 7fed2fab9740 10 client.18798 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:07.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.839+0000 7fed2fab9740 1 -- 172.21.15.103:0/408255058 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5559455a28f0 con 0x55594558b370 2024-09-19T06:59:07.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.839+0000 7fed0e7fc640 1 -- 172.21.15.103:0/408255058 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed18010da0 con 0x55594558b370 2024-09-19T06:59:07.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.839+0000 7fed0e7fc640 1 -- 172.21.15.103:0/408255058 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fed180049c0 con 0x55594558b370 2024-09-19T06:59:07.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.839+0000 7fed2fab9740 10 client.18798 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:07.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.839+0000 7fed2fab9740 10 client.18798 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:59:07.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.839+0000 7fed2fab9740 20 client.18798 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:07.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.839+0000 7fed2fab9740 1 --2- 172.21.15.103:0/408255058 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x555945605990 0x555945607db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:07.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.839+0000 7fed2fab9740 4 client.18798 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:59:07.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.839+0000 7fed2fab9740 1 -- 172.21.15.103:0/408255058 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559454edd70 con 0x555945605990 2024-09-19T06:59:07.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.839+0000 7fed26ffd640 1 --2- 172.21.15.103:0/408255058 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x555945605990 0x555945607db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:07.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.840+0000 7fed0e7fc640 10 client.18798 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:59:07.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.840+0000 7fed26ffd640 1 --2- 172.21.15.103:0/408255058 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x555945605990 0x555945607db0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=70cfb035e3364d92 server_cookie=7c50cad54727a9fa in_seq=0 out_seq=0 2024-09-19T06:59:07.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.840+0000 7fed0e7fc640 1 -- 172.21.15.103:0/408255058 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559454edd70 con 0x555945605990 2024-09-19T06:59:07.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.840+0000 7fed0e7fc640 10 client.18798 handle_command_reply: tid=0 2024-09-19T06:59:07.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.840+0000 7fed2fab9740 1 client.18798 shutdown 2024-09-19T06:59:07.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.840+0000 7fed2fab9740 1 -- 172.21.15.103:0/408255058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55594558b370 msgr2=0x555945585340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:07.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.840+0000 7fed2fab9740 1 --2- 172.21.15.103:0/408255058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55594558b370 0x555945585340 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fed1800bd80 tx=0x7fed1801e040 comp rx=0 tx=0).stop 2024-09-19T06:59:07.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.840+0000 7fed2fab9740 1 -- 172.21.15.103:0/408255058 shutdown_connections 2024-09-19T06:59:07.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.840+0000 7fed2fab9740 1 --2- 172.21.15.103:0/408255058 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x555945605990 0x555945607db0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.840+0000 7fed2fab9740 1 --2- 172.21.15.103:0/408255058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55594558b370 0x555945585340 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.840+0000 7fed2fab9740 1 --2- 172.21.15.103:0/408255058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555945581b40 0x555945593c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.840+0000 7fed0e7fc640 0 client.18798 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:59:07.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.840+0000 7fed2fab9740 1 --2- 172.21.15.103:0/408255058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559455805d0 0x555945593700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.840+0000 7fed2fab9740 1 -- 172.21.15.103:0/408255058 >> 172.21.15.103:0/408255058 conn(0x555945549310 msgr2=0x5559455864a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:07.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.840+0000 7fed2fab9740 1 -- 172.21.15.103:0/408255058 shutdown_connections 2024-09-19T06:59:07.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.841+0000 7fed2fab9740 1 -- 172.21.15.103:0/408255058 wait complete. 2024-09-19T06:59:07.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.841+0000 7fed2fab9740 20 client.18798 trim_cache size 0 max 16384 2024-09-19T06:59:07.846 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:59:07.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.841+0000 7fed2d6e0640 1 -- 172.21.15.103:0/1345321592 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed140a33b0 msgr2=0x7fed140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:07.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.841+0000 7fed2d6e0640 1 --2- 172.21.15.103:0/1345321592 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed140a33b0 0x7fed140a5870 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fed20098aa0 tx=0x7fed2006e3c0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.841+0000 7fed2d6e0640 1 -- 172.21.15.103:0/1345321592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed2810c160 msgr2=0x7fed2819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:07.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.841+0000 7fed2d6e0640 1 --2- 172.21.15.103:0/1345321592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed2810c160 0x7fed2819efc0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fed2000eb10 tx=0x7fed2000c2d0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.841+0000 7fed2d6e0640 1 -- 172.21.15.103:0/1345321592 shutdown_connections 2024-09-19T06:59:07.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.841+0000 7fed2d6e0640 1 --2- 172.21.15.103:0/1345321592 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed140a33b0 0x7fed140a5870 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.841+0000 7fed2d6e0640 1 --2- 172.21.15.103:0/1345321592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed2810c160 0x7fed2819efc0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.841+0000 7fed2d6e0640 1 --2- 172.21.15.103:0/1345321592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed28103650 0x7fed2819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.841+0000 7fed2d6e0640 1 --2- 172.21.15.103:0/1345321592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed28102d30 0x7fed2819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:07.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.841+0000 7fed2d6e0640 1 -- 172.21.15.103:0/1345321592 >> 172.21.15.103:0/1345321592 conn(0x7fed280fc790 msgr2=0x7fed280fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:07.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.841+0000 7fed2d6e0640 1 -- 172.21.15.103:0/1345321592 shutdown_connections 2024-09-19T06:59:07.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:07.842+0000 7fed2d6e0640 1 -- 172.21.15.103:0/1345321592 wait complete. 2024-09-19T06:59:07.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:07 smithi134 ceph-mon[35356]: pgmap v306: 97 pgs: 97 active+clean; 1.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-09-19T06:59:07.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:07 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1987439344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:59:08.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:07 smithi174 ceph-mon[35637]: pgmap v306: 97 pgs: 97 active+clean; 1.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-09-19T06:59:08.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:07 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1987439344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:59:08.591 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:59:08.631 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.115+0000 7fb5971cb640 1 -- 172.21.15.103:0/201458829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590106f10 msgr2=0x7fb59010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.115+0000 7fb5971cb640 1 --2- 172.21.15.103:0/201458829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590106f10 0x7fb59010ea90 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fb588009ca0 tx=0x7fb58802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.115+0000 7fb5971cb640 1 -- 172.21.15.103:0/201458829 shutdown_connections 2024-09-19T06:59:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.115+0000 7fb5971cb640 1 --2- 172.21.15.103:0/201458829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590106f10 0x7fb59010ea90 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.115+0000 7fb5971cb640 1 --2- 172.21.15.103:0/201458829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590102790 0x7fb5901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.115+0000 7fb5971cb640 1 --2- 172.21.15.103:0/201458829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590101de0 0x7fb5901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.115+0000 7fb5971cb640 1 -- 172.21.15.103:0/201458829 >> 172.21.15.103:0/201458829 conn(0x7fb5900fd8c0 msgr2=0x7fb5900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.115+0000 7fb5971cb640 1 -- 172.21.15.103:0/201458829 shutdown_connections 2024-09-19T06:59:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.115+0000 7fb5971cb640 1 -- 172.21.15.103:0/201458829 wait complete. 2024-09-19T06:59:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.115+0000 7fb5971cb640 1 Processor -- start 2024-09-19T06:59:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.115+0000 7fb5971cb640 1 -- start start 2024-09-19T06:59:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb5971cb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590101de0 0x7fb5900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb5971cb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590102790 0x7fb590074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb5971cb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590106f10 0x7fb590074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb5971cb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb590111f90 con 0x7fb590101de0 2024-09-19T06:59:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb5971cb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb590111e10 con 0x7fb590102790 2024-09-19T06:59:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb5971cb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb590112110 con 0x7fb590106f10 2024-09-19T06:59:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb594f67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590101de0 0x7fb5900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb594f67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590101de0 0x7fb5900744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60528/0 (socket says 172.21.15.103:60528) 2024-09-19T06:59:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb594f67640 1 -- 172.21.15.103:0/1851017467 learned_addr learned my addr 172.21.15.103:0/1851017467 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb58ffff640 1 --2- 172.21.15.103:0/1851017467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590102790 0x7fb590074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb595768640 1 --2- 172.21.15.103:0/1851017467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590106f10 0x7fb590074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb594f67640 1 -- 172.21.15.103:0/1851017467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590106f10 msgr2=0x7fb590074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb594f67640 1 --2- 172.21.15.103:0/1851017467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590106f10 0x7fb590074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb594f67640 1 -- 172.21.15.103:0/1851017467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590102790 msgr2=0x7fb590074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb594f67640 1 --2- 172.21.15.103:0/1851017467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590102790 0x7fb590074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb594f67640 1 -- 172.21.15.103:0/1851017467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5901a6920 con 0x7fb590101de0 2024-09-19T06:59:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.116+0000 7fb594f67640 1 --2- 172.21.15.103:0/1851017467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590101de0 0x7fb5900744f0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fb58000b4a0 tx=0x7fb58000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.117+0000 7fb58dffb640 1 -- 172.21.15.103:0/1851017467 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580014070 con 0x7fb590101de0 2024-09-19T06:59:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.117+0000 7fb58dffb640 1 -- 172.21.15.103:0/1851017467 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb580010af0 con 0x7fb590101de0 2024-09-19T06:59:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.117+0000 7fb5971cb640 1 -- 172.21.15.103:0/1851017467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5901a6c10 con 0x7fb590101de0 2024-09-19T06:59:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.117+0000 7fb58dffb640 1 -- 172.21.15.103:0/1851017467 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580017d10 con 0x7fb590101de0 2024-09-19T06:59:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.117+0000 7fb5971cb640 1 -- 172.21.15.103:0/1851017467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5901a7150 con 0x7fb590101de0 2024-09-19T06:59:09.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.119+0000 7fb58dffb640 1 -- 172.21.15.103:0/1851017467 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fb580017070 con 0x7fb590101de0 2024-09-19T06:59:09.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.120+0000 7fb5995a4740 1 Processor -- start 2024-09-19T06:59:09.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.120+0000 7fb5995a4740 1 -- start start 2024-09-19T06:59:09.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.120+0000 7fb58dffb640 1 --2- 172.21.15.103:0/1851017467 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb5780a3430 0x7fb5780a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.120+0000 7fb5995a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2187760 0x55f4d2187b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.120+0000 7fb5995a4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2188080 0x55f4d2192400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.120+0000 7fb5995a4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2192940 0x55f4d2194d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.120+0000 7fb58dffb640 1 -- 172.21.15.103:0/1851017467 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7fb5800cef10 con 0x7fb590101de0 2024-09-19T06:59:09.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.120+0000 7fb5995a4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f4d20ed2a0 con 0x55f4d2192940 2024-09-19T06:59:09.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.120+0000 7fb5995a4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f4d2094b10 con 0x55f4d2188080 2024-09-19T06:59:09.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.120+0000 7fb5995a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f4d20e9260 con 0x55f4d2187760 2024-09-19T06:59:09.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.120+0000 7fb595768640 1 --2- 172.21.15.103:0/1851017467 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb5780a3430 0x7fb5780a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.120+0000 7fb594f67640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2187760 0x55f4d2187b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.120+0000 7fb595768640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2192940 0x55f4d2194d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.120+0000 7fb58ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2188080 0x55f4d2192400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.120+0000 7fb58ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2188080 0x55f4d2192400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:51960/0 (socket says 172.21.15.103:51960) 2024-09-19T06:59:09.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.120+0000 7fb58ffff640 1 -- 172.21.15.103:0/132917668 learned_addr learned my addr 172.21.15.103:0/132917668 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:09.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.121+0000 7fb58ffff640 1 -- 172.21.15.103:0/132917668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2187760 msgr2=0x55f4d2187b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:09.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.121+0000 7fb58ffff640 1 --2- 172.21.15.103:0/132917668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2187760 0x55f4d2187b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.121+0000 7fb58ffff640 1 -- 172.21.15.103:0/132917668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2192940 msgr2=0x55f4d2194d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:09.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.121+0000 7fb58ffff640 1 --2- 172.21.15.103:0/132917668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2192940 0x55f4d2194d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.121+0000 7fb58ffff640 1 -- 172.21.15.103:0/132917668 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4d1e5eb90 con 0x55f4d2188080 2024-09-19T06:59:09.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.121+0000 7fb595768640 1 --2- 172.21.15.103:0/132917668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2192940 0x55f4d2194d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:59:09.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.121+0000 7fb594f67640 1 --2- 172.21.15.103:0/132917668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2187760 0x55f4d2187b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:59:09.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.121+0000 7fb595768640 1 --2- 172.21.15.103:0/1851017467 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb5780a3430 0x7fb5780a58f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb588009430 tx=0x7fb588031e30 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:09.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.121+0000 7fb58ffff640 1 --2- 172.21.15.103:0/132917668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2188080 0x55f4d2192400 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb58400abf0 tx=0x7fb58400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3321a3cafebf41c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.121+0000 7fb576ffd640 1 -- 172.21.15.103:0/132917668 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58401d070 con 0x55f4d2188080 2024-09-19T06:59:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.121+0000 7fb576ffd640 1 -- 172.21.15.103:0/132917668 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb584018b20 con 0x55f4d2188080 2024-09-19T06:59:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.121+0000 7fb576ffd640 1 -- 172.21.15.103:0/132917668 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584010a50 con 0x55f4d2188080 2024-09-19T06:59:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.122+0000 7fb5995a4740 1 -- 172.21.15.103:0/132917668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2188080 msgr2=0x55f4d2192400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.122+0000 7fb5995a4740 1 --2- 172.21.15.103:0/132917668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2188080 0x55f4d2192400 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb58400abf0 tx=0x7fb58400f790 comp rx=0 tx=0).stop 2024-09-19T06:59:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.122+0000 7fb5995a4740 1 -- 172.21.15.103:0/132917668 shutdown_connections 2024-09-19T06:59:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.122+0000 7fb5995a4740 1 --2- 172.21.15.103:0/132917668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2192940 0x55f4d2194d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.122+0000 7fb5995a4740 1 --2- 172.21.15.103:0/132917668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2188080 0x55f4d2192400 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.122+0000 7fb5995a4740 1 --2- 172.21.15.103:0/132917668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2187760 0x55f4d2187b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.122+0000 7fb5995a4740 1 -- 172.21.15.103:0/132917668 >> 172.21.15.103:0/132917668 conn(0x55f4d2183cf0 msgr2=0x55f4d21840d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.122+0000 7fb5995a4740 1 -- 172.21.15.103:0/132917668 shutdown_connections 2024-09-19T06:59:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.122+0000 7fb5995a4740 1 -- 172.21.15.103:0/132917668 wait complete. 2024-09-19T06:59:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.122+0000 7fb5995a4740 1 -- start start 2024-09-19T06:59:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb5995a4740 1 Processor -- start 2024-09-19T06:59:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb5995a4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2187760 0x55f4d218f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb5995a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2188080 0x55f4d218fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb5995a4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2192940 0x55f4d2190260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb5995a4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f4d2195510 con 0x55f4d2187760 2024-09-19T06:59:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb5995a4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f4d20756e0 con 0x55f4d2192940 2024-09-19T06:59:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb5995a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f4d2195690 con 0x55f4d2188080 2024-09-19T06:59:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb5995a4740 10 client.0 _ll_register_callbacks cb 0x55f4d2099a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb594f67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2187760 0x55f4d218f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb594f67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2187760 0x55f4d218f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60538/0 (socket says 172.21.15.103:60538) 2024-09-19T06:59:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb594f67640 1 -- 172.21.15.103:0/3289264783 learned_addr learned my addr 172.21.15.103:0/3289264783 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb595768640 1 --2- 172.21.15.103:0/3289264783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2192940 0x55f4d2190260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb58ffff640 1 --2- 172.21.15.103:0/3289264783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2188080 0x55f4d218fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb594f67640 1 -- 172.21.15.103:0/3289264783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2188080 msgr2=0x55f4d218fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:09.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb594f67640 1 --2- 172.21.15.103:0/3289264783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2188080 0x55f4d218fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb594f67640 1 -- 172.21.15.103:0/3289264783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2192940 msgr2=0x55f4d2190260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:09.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb594f67640 1 --2- 172.21.15.103:0/3289264783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2192940 0x55f4d2190260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb594f67640 1 -- 172.21.15.103:0/3289264783 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4d219ea00 con 0x55f4d2187760 2024-09-19T06:59:09.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.123+0000 7fb595768640 1 --2- 172.21.15.103:0/3289264783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2192940 0x55f4d2190260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:09.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.124+0000 7fb594f67640 1 --2- 172.21.15.103:0/3289264783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2187760 0x55f4d218f7e0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fb5800cc780 tx=0x7fb58009b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfa90e0829488a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:09.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.124+0000 7fb56ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:09.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.124+0000 7fb56ffff640 1 -- 172.21.15.103:0/3289264783 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58001e070 con 0x55f4d2187760 2024-09-19T06:59:09.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.124+0000 7fb56ffff640 1 -- 172.21.15.103:0/3289264783 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7fb5800979d0 con 0x55f4d2187760 2024-09-19T06:59:09.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.124+0000 7fb5995a4740 1 -- 172.21.15.103:0/3289264783 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb584018b20 con 0x55f4d2187760 2024-09-19T06:59:09.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.124+0000 7fb56ffff640 1 -- 172.21.15.103:0/3289264783 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb5800a3f20 con 0x55f4d2187760 2024-09-19T06:59:09.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.124+0000 7fb5995a4740 10 client.18810 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:09.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.124+0000 7fb5995a4740 1 -- 172.21.15.103:0/3289264783 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f4d21a1370 con 0x55f4d2187760 2024-09-19T06:59:09.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.124+0000 7fb56ffff640 1 -- 172.21.15.103:0/3289264783 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb584018b20 con 0x55f4d2187760 2024-09-19T06:59:09.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.124+0000 7fb56ffff640 1 -- 172.21.15.103:0/3289264783 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb580050be0 con 0x55f4d2187760 2024-09-19T06:59:09.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.125+0000 7fb5995a4740 10 client.18810 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:09.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.125+0000 7fb5995a4740 10 client.18810 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:59:09.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.125+0000 7fb5995a4740 20 client.18810 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:09.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.125+0000 7fb5995a4740 1 --2- 172.21.15.103:0/3289264783 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f4d21a28e0 0x55f4d21a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.125+0000 7fb5995a4740 4 client.18810 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:09.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.125+0000 7fb5995a4740 1 -- 172.21.15.103:0/3289264783 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4d20fa9d0 con 0x55f4d21a28e0 2024-09-19T06:59:09.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.125+0000 7fb58ffff640 1 --2- 172.21.15.103:0/3289264783 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f4d21a28e0 0x55f4d21a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.125+0000 7fb56ffff640 10 client.18810 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:09.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.125+0000 7fb58ffff640 1 --2- 172.21.15.103:0/3289264783 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f4d21a28e0 0x55f4d21a4da0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63836607813302fd server_cookie=e5ec84bcfe4ed39d in_seq=0 out_seq=0 2024-09-19T06:59:09.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.126+0000 7fb56ffff640 1 -- 172.21.15.103:0/3289264783 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4d20fa9d0 con 0x55f4d21a28e0 2024-09-19T06:59:09.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.126+0000 7fb56ffff640 10 client.18810 handle_command_reply: tid=0 2024-09-19T06:59:09.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.126+0000 7fb5995a4740 1 client.18810 shutdown 2024-09-19T06:59:09.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.126+0000 7fb5995a4740 1 -- 172.21.15.103:0/3289264783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2187760 msgr2=0x55f4d218f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:09.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.126+0000 7fb5995a4740 1 --2- 172.21.15.103:0/3289264783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2187760 0x55f4d218f7e0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fb5800cc780 tx=0x7fb58009b360 comp rx=0 tx=0).stop 2024-09-19T06:59:09.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.126+0000 7fb5995a4740 1 -- 172.21.15.103:0/3289264783 shutdown_connections 2024-09-19T06:59:09.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.126+0000 7fb5995a4740 1 --2- 172.21.15.103:0/3289264783 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f4d21a28e0 0x55f4d21a4da0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.126+0000 7fb56ffff640 0 client.18810 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:09.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.126+0000 7fb5995a4740 1 --2- 172.21.15.103:0/3289264783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2192940 0x55f4d2190260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.126+0000 7fb5995a4740 1 --2- 172.21.15.103:0/3289264783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2188080 0x55f4d218fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.126+0000 7fb5995a4740 1 --2- 172.21.15.103:0/3289264783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2187760 0x55f4d218f7e0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.126+0000 7fb5995a4740 1 -- 172.21.15.103:0/3289264783 >> 172.21.15.103:0/3289264783 conn(0x55f4d2183cf0 msgr2=0x55f4d21899d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:09.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.127+0000 7fb5995a4740 1 -- 172.21.15.103:0/3289264783 shutdown_connections 2024-09-19T06:59:09.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.127+0000 7fb5995a4740 1 -- 172.21.15.103:0/3289264783 wait complete. 2024-09-19T06:59:09.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.127+0000 7fb5995a4740 20 client.18810 trim_cache size 0 max 16384 2024-09-19T06:59:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.146+0000 7fb5995a4740 1 Processor -- start 2024-09-19T06:59:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.146+0000 7fb5995a4740 1 -- start start 2024-09-19T06:59:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.146+0000 7fb5995a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2187760 0x55f4d218a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.146+0000 7fb5995a4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2188080 0x55f4d21aa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.146+0000 7fb5995a4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2192940 0x55f4d21aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.146+0000 7fb5995a4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901afb10 con 0x55f4d2188080 2024-09-19T06:59:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.146+0000 7fb5995a4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901b1650 con 0x55f4d2192940 2024-09-19T06:59:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.146+0000 7fb5995a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901b01e0 con 0x55f4d2187760 2024-09-19T06:59:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.146+0000 7fb58ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2188080 0x55f4d21aa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.146+0000 7fb58ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2188080 0x55f4d21aa540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60552/0 (socket says 172.21.15.103:60552) 2024-09-19T06:59:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.146+0000 7fb58ffff640 1 -- 172.21.15.103:0/2456463654 learned_addr learned my addr 172.21.15.103:0/2456463654 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.147+0000 7fb594f67640 1 --2- 172.21.15.103:0/2456463654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2187760 0x55f4d218a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.147+0000 7fb595768640 1 --2- 172.21.15.103:0/2456463654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2192940 0x55f4d21aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.147+0000 7fb58ffff640 1 -- 172.21.15.103:0/2456463654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2187760 msgr2=0x55f4d218a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.147+0000 7fb58ffff640 1 --2- 172.21.15.103:0/2456463654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2187760 0x55f4d218a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.147+0000 7fb58ffff640 1 -- 172.21.15.103:0/2456463654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2192940 msgr2=0x55f4d21aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.147+0000 7fb58ffff640 1 --2- 172.21.15.103:0/2456463654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2192940 0x55f4d21aaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.147+0000 7fb58ffff640 1 -- 172.21.15.103:0/2456463654 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4d217f260 con 0x55f4d2188080 2024-09-19T06:59:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.147+0000 7fb594f67640 1 --2- 172.21.15.103:0/2456463654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2187760 0x55f4d218a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.147+0000 7fb595768640 1 --2- 172.21.15.103:0/2456463654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2192940 0x55f4d21aaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.147+0000 7fb58ffff640 1 --2- 172.21.15.103:0/2456463654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2188080 0x55f4d21aa540 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fb58401b040 tx=0x7fb5840106f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5924e5cda670552 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.147+0000 7fb5767fc640 1 -- 172.21.15.103:0/2456463654 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58401d040 con 0x55f4d2188080 2024-09-19T06:59:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.147+0000 7fb5767fc640 1 -- 172.21.15.103:0/2456463654 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb58400a7f0 con 0x55f4d2188080 2024-09-19T06:59:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.147+0000 7fb5767fc640 1 -- 172.21.15.103:0/2456463654 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584010ae0 con 0x55f4d2188080 2024-09-19T06:59:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.148+0000 7fb5995a4740 1 -- 172.21.15.103:0/2456463654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2188080 msgr2=0x55f4d21aa540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.148+0000 7fb5995a4740 1 --2- 172.21.15.103:0/2456463654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2188080 0x55f4d21aa540 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fb58401b040 tx=0x7fb5840106f0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.148+0000 7fb5995a4740 1 -- 172.21.15.103:0/2456463654 shutdown_connections 2024-09-19T06:59:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.148+0000 7fb5995a4740 1 --2- 172.21.15.103:0/2456463654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2192940 0x55f4d21aaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.148+0000 7fb5995a4740 1 --2- 172.21.15.103:0/2456463654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2188080 0x55f4d21aa540 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.148+0000 7fb5995a4740 1 --2- 172.21.15.103:0/2456463654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2187760 0x55f4d218a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.148+0000 7fb5995a4740 1 -- 172.21.15.103:0/2456463654 >> 172.21.15.103:0/2456463654 conn(0x55f4d2183cf0 msgr2=0x55f4d2191860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.148+0000 7fb5995a4740 1 -- 172.21.15.103:0/2456463654 shutdown_connections 2024-09-19T06:59:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.148+0000 7fb5995a4740 1 -- 172.21.15.103:0/2456463654 wait complete. 2024-09-19T06:59:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.148+0000 7fb5995a4740 1 -- start start 2024-09-19T06:59:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.148+0000 7fb5995a4740 1 Processor -- start 2024-09-19T06:59:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.149+0000 7fb5995a4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2187760 0x55f4d2196f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.149+0000 7fb5995a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2188080 0x55f4d2197440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.149+0000 7fb5995a4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2192940 0x55f4d22064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.149+0000 7fb5995a4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901b0f80 con 0x55f4d2192940 2024-09-19T06:59:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.149+0000 7fb5995a4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901b08b0 con 0x55f4d2187760 2024-09-19T06:59:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.149+0000 7fb5995a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901adb10 con 0x55f4d2188080 2024-09-19T06:59:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.149+0000 7fb5995a4740 10 client.0 _ll_register_callbacks cb 0x7fb58009adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.149+0000 7fb595768640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2192940 0x55f4d22064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.149+0000 7fb595768640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2192940 0x55f4d22064e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60554/0 (socket says 172.21.15.103:60554) 2024-09-19T06:59:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.149+0000 7fb595768640 1 -- 172.21.15.103:0/2496630369 learned_addr learned my addr 172.21.15.103:0/2496630369 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.149+0000 7fb594f67640 1 --2- 172.21.15.103:0/2496630369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2187760 0x55f4d2196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.150+0000 7fb594f67640 1 -- 172.21.15.103:0/2496630369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2188080 msgr2=0x55f4d2197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.150+0000 7fb58ffff640 1 --2- 172.21.15.103:0/2496630369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2188080 0x55f4d2197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.150+0000 7fb594f67640 1 --2- 172.21.15.103:0/2496630369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2188080 0x55f4d2197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.150+0000 7fb594f67640 1 -- 172.21.15.103:0/2496630369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2192940 msgr2=0x55f4d22064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.150+0000 7fb594f67640 1 --2- 172.21.15.103:0/2496630369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2192940 0x55f4d22064e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.150+0000 7fb594f67640 1 -- 172.21.15.103:0/2496630369 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4d2206b20 con 0x55f4d2187760 2024-09-19T06:59:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.150+0000 7fb595768640 1 --2- 172.21.15.103:0/2496630369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2192940 0x55f4d22064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.150+0000 7fb58ffff640 1 --2- 172.21.15.103:0/2496630369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2188080 0x55f4d2197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.150+0000 7fb594f67640 1 --2- 172.21.15.103:0/2496630369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2187760 0x55f4d2196f00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb58009b060 tx=0x7fb5800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca49de364615ce2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.150+0000 7fb56ffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:59:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.150+0000 7fb56ffff640 1 -- 172.21.15.103:0/2496630369 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5800b9060 con 0x55f4d2187760 2024-09-19T06:59:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.150+0000 7fb56ffff640 1 -- 172.21.15.103:0/2496630369 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7fb5800cc7d0 con 0x55f4d2187760 2024-09-19T06:59:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.150+0000 7fb5995a4740 1 -- 172.21.15.103:0/2496630369 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f4d20f4d20 con 0x55f4d2187760 2024-09-19T06:59:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.151+0000 7fb56ffff640 1 -- 172.21.15.103:0/2496630369 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb58001e050 con 0x55f4d2187760 2024-09-19T06:59:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.151+0000 7fb5995a4740 10 client.27152 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.151+0000 7fb5995a4740 1 -- 172.21.15.103:0/2496630369 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f4d220de00 con 0x55f4d2187760 2024-09-19T06:59:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.151+0000 7fb56ffff640 1 -- 172.21.15.103:0/2496630369 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f4d20f4d20 con 0x55f4d2187760 2024-09-19T06:59:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.151+0000 7fb56ffff640 1 -- 172.21.15.103:0/2496630369 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb58009f150 con 0x55f4d2187760 2024-09-19T06:59:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.151+0000 7fb5995a4740 10 client.27152 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.151+0000 7fb5995a4740 10 client.27152 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:59:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.151+0000 7fb5995a4740 20 client.27152 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.151+0000 7fb5995a4740 1 --2- 172.21.15.103:0/2496630369 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f4d21a28e0 0x55f4d2210360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.151+0000 7fb5995a4740 4 client.27152 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-19T06:59:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.151+0000 7fb5995a4740 1 -- 172.21.15.103:0/2496630369 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55f4d2070ec0 con 0x55f4d21a28e0 2024-09-19T06:59:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.152+0000 7fb58ffff640 1 --2- 172.21.15.103:0/2496630369 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f4d21a28e0 0x55f4d2210360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.152+0000 7fb56ffff640 10 client.27152 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.152+0000 7fb58ffff640 1 --2- 172.21.15.103:0/2496630369 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f4d21a28e0 0x55f4d2210360 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2636e35e22377dab server_cookie=13740912802f6c74 in_seq=0 out_seq=0 2024-09-19T06:59:09.158 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:09.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.163+0000 7fb56ffff640 1 -- 172.21.15.103:0/2496630369 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f4d2070ec0 con 0x55f4d21a28e0 2024-09-19T06:59:09.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.163+0000 7fb56ffff640 10 client.27152 handle_command_reply: tid=0 2024-09-19T06:59:09.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.163+0000 7fb5995a4740 1 client.27152 shutdown 2024-09-19T06:59:09.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.163+0000 7fb5995a4740 1 -- 172.21.15.103:0/2496630369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2187760 msgr2=0x55f4d2196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.163+0000 7fb5995a4740 1 --2- 172.21.15.103:0/2496630369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2187760 0x55f4d2196f00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb58009b060 tx=0x7fb5800046c0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.163+0000 7fb5995a4740 1 -- 172.21.15.103:0/2496630369 shutdown_connections 2024-09-19T06:59:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.163+0000 7fb5995a4740 1 --2- 172.21.15.103:0/2496630369 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f4d21a28e0 0x55f4d2210360 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.163+0000 7fb5995a4740 1 --2- 172.21.15.103:0/2496630369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d2192940 0x55f4d22064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.163+0000 7fb5995a4740 1 --2- 172.21.15.103:0/2496630369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d2188080 0x55f4d2197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.163+0000 7fb56ffff640 0 client.27152 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.163+0000 7fb5995a4740 1 --2- 172.21.15.103:0/2496630369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4d2187760 0x55f4d2196f00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.163+0000 7fb5995a4740 1 -- 172.21.15.103:0/2496630369 >> 172.21.15.103:0/2496630369 conn(0x55f4d2183cf0 msgr2=0x55f4d2182650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.163+0000 7fb5995a4740 1 -- 172.21.15.103:0/2496630369 shutdown_connections 2024-09-19T06:59:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.163+0000 7fb5995a4740 1 -- 172.21.15.103:0/2496630369 wait complete. 2024-09-19T06:59:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.163+0000 7fb5995a4740 20 client.27152 trim_cache size 0 max 16384 2024-09-19T06:59:09.166 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:59:09.166 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:59:09.166 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:59:09.166 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 34, 2024-09-19T06:59:09.166 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:59:09.166 INFO:teuthology.orchestra.run.smithi103.stdout: "c9039c34": { 2024-09-19T06:59:09.166 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 34, 2024-09-19T06:59:09.166 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:59:09.166 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:59:09.166 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:59:09.166 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:59:09.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:59:09.167 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:59:09.167 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 150.0, 2024-09-19T06:59:09.167 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:59:09.167 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:59:09.167 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:59:09.167 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:59:09.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:59:09.167 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:59:09.167 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:09.167 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:09.167 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:09.167 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:09.167 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:09.168 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:59:09.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.164+0000 7fb5971cb640 1 -- 172.21.15.103:0/1851017467 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb5780a3430 msgr2=0x7fb5780a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:09.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.164+0000 7fb5971cb640 1 --2- 172.21.15.103:0/1851017467 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb5780a3430 0x7fb5780a58f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb588009430 tx=0x7fb588031e30 comp rx=0 tx=0).stop 2024-09-19T06:59:09.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.164+0000 7fb5971cb640 1 -- 172.21.15.103:0/1851017467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590101de0 msgr2=0x7fb5900744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:09.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.164+0000 7fb5971cb640 1 --2- 172.21.15.103:0/1851017467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590101de0 0x7fb5900744f0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fb58000b4a0 tx=0x7fb58000b960 comp rx=0 tx=0).stop 2024-09-19T06:59:09.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.164+0000 7fb5971cb640 1 -- 172.21.15.103:0/1851017467 shutdown_connections 2024-09-19T06:59:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.164+0000 7fb5971cb640 1 --2- 172.21.15.103:0/1851017467 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb5780a3430 0x7fb5780a58f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.164+0000 7fb5971cb640 1 --2- 172.21.15.103:0/1851017467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590106f10 0x7fb590074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.164+0000 7fb5971cb640 1 --2- 172.21.15.103:0/1851017467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590102790 0x7fb590074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.164+0000 7fb5971cb640 1 --2- 172.21.15.103:0/1851017467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590101de0 0x7fb5900744f0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.164+0000 7fb5971cb640 1 -- 172.21.15.103:0/1851017467 >> 172.21.15.103:0/1851017467 conn(0x7fb5900fd8c0 msgr2=0x7fb5900fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.164+0000 7fb5971cb640 1 -- 172.21.15.103:0/1851017467 shutdown_connections 2024-09-19T06:59:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:09.164+0000 7fb5971cb640 1 -- 172.21.15.103:0/1851017467 wait complete. 2024-09-19T06:59:09.775 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:09 smithi103 ceph-mon[31158]: pgmap v307: 97 pgs: 97 active+clean; 1.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-09-19T06:59:09.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:09 smithi134 ceph-mon[35356]: pgmap v307: 97 pgs: 97 active+clean; 1.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-09-19T06:59:10.043 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c9039c34', quiesce took 4.3 seconds. Will release after: 25.7 2024-09-19T06:59:10.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:09 smithi174 ceph-mon[35637]: pgmap v307: 97 pgs: 97 active+clean; 1.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-09-19T06:59:10.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.247+0000 7f6e8f66e640 1 -- 172.21.15.103:0/2741539887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e88106f20 msgr2=0x7f6e8810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:10.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.247+0000 7f6e8f66e640 1 --2- 172.21.15.103:0/2741539887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e88106f20 0x7f6e8810eaa0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009ca0 tx=0x7f6e8002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.247+0000 7f6e8f66e640 1 -- 172.21.15.103:0/2741539887 shutdown_connections 2024-09-19T06:59:10.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.247+0000 7f6e8f66e640 1 --2- 172.21.15.103:0/2741539887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e88106f20 0x7f6e8810eaa0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.247+0000 7f6e8f66e640 1 --2- 172.21.15.103:0/2741539887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e881027a0 0x7f6e881067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.247+0000 7f6e8f66e640 1 --2- 172.21.15.103:0/2741539887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e88101df0 0x7f6e881021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.247+0000 7f6e8f66e640 1 -- 172.21.15.103:0/2741539887 >> 172.21.15.103:0/2741539887 conn(0x7f6e880fd8b0 msgr2=0x7f6e880ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:10.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.247+0000 7f6e8f66e640 1 -- 172.21.15.103:0/2741539887 shutdown_connections 2024-09-19T06:59:10.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.247+0000 7f6e8f66e640 1 -- 172.21.15.103:0/2741539887 wait complete. 2024-09-19T06:59:10.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8f66e640 1 Processor -- start 2024-09-19T06:59:10.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8f66e640 1 -- start start 2024-09-19T06:59:10.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8f66e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e88101df0 0x7f6e88198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8f66e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e881027a0 0x7f6e88198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8f66e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e88106f20 0x7f6e8819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8f66e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e88112020 con 0x7f6e88106f20 2024-09-19T06:59:10.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8f66e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6e88111ea0 con 0x7f6e881027a0 2024-09-19T06:59:10.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8f66e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e881121a0 con 0x7f6e88101df0 2024-09-19T06:59:10.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8dc0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e88106f20 0x7f6e8819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8dc0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e88106f20 0x7f6e8819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60572/0 (socket says 172.21.15.103:60572) 2024-09-19T06:59:10.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8dc0b640 1 -- 172.21.15.103:0/2129919311 learned_addr learned my addr 172.21.15.103:0/2129919311 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:10.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8cc09640 1 --2- 172.21.15.103:0/2129919311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e881027a0 0x7f6e88198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8d40a640 1 --2- 172.21.15.103:0/2129919311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e88101df0 0x7f6e88198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8dc0b640 1 -- 172.21.15.103:0/2129919311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e88101df0 msgr2=0x7f6e88198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:10.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8dc0b640 1 --2- 172.21.15.103:0/2129919311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e88101df0 0x7f6e88198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8dc0b640 1 -- 172.21.15.103:0/2129919311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e881027a0 msgr2=0x7f6e88198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:10.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8dc0b640 1 --2- 172.21.15.103:0/2129919311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e881027a0 0x7f6e88198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8dc0b640 1 -- 172.21.15.103:0/2129919311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8819d4b0 con 0x7f6e88106f20 2024-09-19T06:59:10.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.248+0000 7f6e8d40a640 1 --2- 172.21.15.103:0/2129919311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e88101df0 0x7f6e88198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:10.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.249+0000 7f6e8dc0b640 1 --2- 172.21.15.103:0/2129919311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e88106f20 0x7f6e8819cdd0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6e80031a60 tx=0x7f6e80031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:10.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.249+0000 7f6e7e7fc640 1 -- 172.21.15.103:0/2129919311 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80003c30 con 0x7f6e88106f20 2024-09-19T06:59:10.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.249+0000 7f6e7e7fc640 1 -- 172.21.15.103:0/2129919311 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e80003dd0 con 0x7f6e88106f20 2024-09-19T06:59:10.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.249+0000 7f6e8f66e640 1 -- 172.21.15.103:0/2129919311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e8819d740 con 0x7f6e88106f20 2024-09-19T06:59:10.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.249+0000 7f6e7e7fc640 1 -- 172.21.15.103:0/2129919311 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800047e0 con 0x7f6e88106f20 2024-09-19T06:59:10.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.249+0000 7f6e8f66e640 1 -- 172.21.15.103:0/2129919311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e881a5080 con 0x7f6e88106f20 2024-09-19T06:59:10.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.251+0000 7f6e7e7fc640 1 -- 172.21.15.103:0/2129919311 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f6e8002f790 con 0x7f6e88106f20 2024-09-19T06:59:10.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.251+0000 7f6e91a47740 1 Processor -- start 2024-09-19T06:59:10.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e91a47740 1 -- start start 2024-09-19T06:59:10.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e7e7fc640 1 --2- 172.21.15.103:0/2129919311 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6e740a33b0 0x7f6e740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e91a47740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca01872b40 0x55ca01871090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e91a47740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca018715d0 0x55ca01871a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e91a47740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca0187c370 0x55ca0187e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e91a47740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca017d72f0 con 0x55ca01872b40 2024-09-19T06:59:10.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e91a47740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ca0177ecf0 con 0x55ca018715d0 2024-09-19T06:59:10.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e91a47740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ca017d32b0 con 0x55ca0187c370 2024-09-19T06:59:10.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e8d40a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca01872b40 0x55ca01871090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e8cc09640 1 --2- 172.21.15.103:0/2129919311 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6e740a33b0 0x7f6e740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e8d40a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca01872b40 0x55ca01871090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60578/0 (socket says 172.21.15.103:60578) 2024-09-19T06:59:10.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e8d40a640 1 -- 172.21.15.103:0/2235502643 learned_addr learned my addr 172.21.15.103:0/2235502643 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:10.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e7e7fc640 1 -- 172.21.15.103:0/2129919311 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7f6e800f1140 con 0x7f6e88106f20 2024-09-19T06:59:10.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e8cc09640 1 --2- 172.21.15.103:0/2235502643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca018715d0 0x55ca01871a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e8d40a640 1 -- 172.21.15.103:0/2235502643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca0187c370 msgr2=0x55ca0187e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:10.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e8dc0b640 1 --2- 172.21.15.103:0/2235502643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca0187c370 0x55ca0187e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e8d40a640 1 --2- 172.21.15.103:0/2235502643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca0187c370 0x55ca0187e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e8d40a640 1 -- 172.21.15.103:0/2235502643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca018715d0 msgr2=0x55ca01871a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:10.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e8d40a640 1 --2- 172.21.15.103:0/2235502643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca018715d0 0x55ca01871a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e8d40a640 1 -- 172.21.15.103:0/2235502643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca017633a0 con 0x55ca01872b40 2024-09-19T06:59:10.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e8cc09640 1 --2- 172.21.15.103:0/2235502643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca018715d0 0x55ca01871a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:10.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e8cc09640 1 --2- 172.21.15.103:0/2129919311 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6e740a33b0 0x7f6e740a5870 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55ca0186eac0 tx=0x7f6e780092c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:10.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.252+0000 7f6e8d40a640 1 --2- 172.21.15.103:0/2235502643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca01872b40 0x55ca01871090 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400aa10 tx=0x7f6e8400aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b31faaf6ff07d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:10.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.253+0000 7f6e737fe640 1 -- 172.21.15.103:0/2235502643 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84015070 con 0x55ca01872b40 2024-09-19T06:59:10.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.253+0000 7f6e737fe640 1 -- 172.21.15.103:0/2235502643 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e8400cc60 con 0x55ca01872b40 2024-09-19T06:59:10.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.253+0000 7f6e737fe640 1 -- 172.21.15.103:0/2235502643 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84018720 con 0x55ca01872b40 2024-09-19T06:59:10.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.253+0000 7f6e91a47740 1 -- 172.21.15.103:0/2235502643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca01872b40 msgr2=0x55ca01871090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:10.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.253+0000 7f6e91a47740 1 --2- 172.21.15.103:0/2235502643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca01872b40 0x55ca01871090 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400aa10 tx=0x7f6e8400aed0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.253+0000 7f6e91a47740 1 -- 172.21.15.103:0/2235502643 shutdown_connections 2024-09-19T06:59:10.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.253+0000 7f6e91a47740 1 --2- 172.21.15.103:0/2235502643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca0187c370 0x55ca0187e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.253+0000 7f6e91a47740 1 --2- 172.21.15.103:0/2235502643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca018715d0 0x55ca01871a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.253+0000 7f6e91a47740 1 --2- 172.21.15.103:0/2235502643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca01872b40 0x55ca01871090 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.253+0000 7f6e91a47740 1 -- 172.21.15.103:0/2235502643 >> 172.21.15.103:0/2235502643 conn(0x55ca0183a310 msgr2=0x55ca0186dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:10.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.253+0000 7f6e91a47740 1 -- 172.21.15.103:0/2235502643 shutdown_connections 2024-09-19T06:59:10.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.253+0000 7f6e91a47740 1 -- 172.21.15.103:0/2235502643 wait complete. 2024-09-19T06:59:10.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.254+0000 7f6e91a47740 1 -- start start 2024-09-19T06:59:10.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.254+0000 7f6e91a47740 1 Processor -- start 2024-09-19T06:59:10.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.254+0000 7f6e91a47740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca018715d0 0x55ca0187bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.254+0000 7f6e91a47740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 0x55ca01878f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.254+0000 7f6e91a47740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01879490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.254+0000 7f6e91a47740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca0187f070 con 0x55ca0187c370 2024-09-19T06:59:10.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.254+0000 7f6e91a47740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ca01792060 con 0x55ca01872b40 2024-09-19T06:59:10.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.254+0000 7f6e91a47740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ca0187f1f0 con 0x55ca018715d0 2024-09-19T06:59:10.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.254+0000 7f6e91a47740 10 client.0 _ll_register_callbacks cb 0x55ca014e58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:10.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.254+0000 7f6e8dc0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01879490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.254+0000 7f6e8dc0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01879490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60594/0 (socket says 172.21.15.103:60594) 2024-09-19T06:59:10.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.254+0000 7f6e8dc0b640 1 -- 172.21.15.103:0/3896473738 learned_addr learned my addr 172.21.15.103:0/3896473738 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:10.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.254+0000 7f6e8cc09640 1 --2- 172.21.15.103:0/3896473738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 0x55ca01878f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e8d40a640 1 --2- 172.21.15.103:0/3896473738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca018715d0 0x55ca0187bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e8dc0b640 1 -- 172.21.15.103:0/3896473738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca018715d0 msgr2=0x55ca0187bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:10.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e8dc0b640 1 --2- 172.21.15.103:0/3896473738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca018715d0 0x55ca0187bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e8dc0b640 1 -- 172.21.15.103:0/3896473738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 msgr2=0x55ca01878f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:10.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e8dc0b640 1 --2- 172.21.15.103:0/3896473738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 0x55ca01878f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e8dc0b640 1 -- 172.21.15.103:0/3896473738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca015483c0 con 0x55ca0187c370 2024-09-19T06:59:10.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e8cc09640 1 --2- 172.21.15.103:0/3896473738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 0x55ca01878f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:10.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e8dc0b640 1 --2- 172.21.15.103:0/3896473738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01879490 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f6e80031f80 tx=0x7f6e800bd150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9ea18fbc88ca41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:10.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:10.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e70ff9640 1 -- 172.21.15.103:0/3896473738 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800bdbe0 con 0x55ca0187c370 2024-09-19T06:59:10.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e70ff9640 1 -- 172.21.15.103:0/3896473738 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7f6e800dd420 con 0x55ca0187c370 2024-09-19T06:59:10.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e91a47740 1 -- 172.21.15.103:0/3896473738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e8400cc60 con 0x55ca0187c370 2024-09-19T06:59:10.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e70ff9640 1 -- 172.21.15.103:0/3896473738 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6e800bd4e0 con 0x55ca0187c370 2024-09-19T06:59:10.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e91a47740 10 client.18828 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:10.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e91a47740 1 -- 172.21.15.103:0/3896473738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ca0187f070 con 0x55ca0187c370 2024-09-19T06:59:10.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e70ff9640 1 -- 172.21.15.103:0/3896473738 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e8400cc60 con 0x55ca0187c370 2024-09-19T06:59:10.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e70ff9640 1 -- 172.21.15.103:0/3896473738 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6e800c5cc0 con 0x55ca0187c370 2024-09-19T06:59:10.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e91a47740 10 client.18828 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:10.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e91a47740 10 client.18828 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:59:10.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.255+0000 7f6e91a47740 20 client.18828 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:10.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.256+0000 7f6e91a47740 1 --2- 172.21.15.103:0/3896473738 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55ca0188bf60 0x55ca0188e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.256+0000 7f6e91a47740 4 client.18828 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:10.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.256+0000 7f6e91a47740 1 -- 172.21.15.103:0/3896473738 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca017e4860 con 0x55ca0188bf60 2024-09-19T06:59:10.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.256+0000 7f6e8d40a640 1 --2- 172.21.15.103:0/3896473738 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55ca0188bf60 0x55ca0188e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.256+0000 7f6e70ff9640 10 client.18828 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:59:10.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.256+0000 7f6e8d40a640 1 --2- 172.21.15.103:0/3896473738 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55ca0188bf60 0x55ca0188e390 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5157e2687c4445be server_cookie=17ea5975d585984b in_seq=0 out_seq=0 2024-09-19T06:59:10.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.257+0000 7f6e70ff9640 1 -- 172.21.15.103:0/3896473738 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca017e4860 con 0x55ca0188bf60 2024-09-19T06:59:10.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.257+0000 7f6e70ff9640 10 client.18828 handle_command_reply: tid=0 2024-09-19T06:59:10.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.257+0000 7f6e91a47740 1 client.18828 shutdown 2024-09-19T06:59:10.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.257+0000 7f6e91a47740 1 -- 172.21.15.103:0/3896473738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 msgr2=0x55ca01879490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:10.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.257+0000 7f6e91a47740 1 --2- 172.21.15.103:0/3896473738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01879490 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f6e80031f80 tx=0x7f6e800bd150 comp rx=0 tx=0).stop 2024-09-19T06:59:10.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.257+0000 7f6e91a47740 1 -- 172.21.15.103:0/3896473738 shutdown_connections 2024-09-19T06:59:10.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.257+0000 7f6e91a47740 1 --2- 172.21.15.103:0/3896473738 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55ca0188bf60 0x55ca0188e390 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.257+0000 7f6e91a47740 1 --2- 172.21.15.103:0/3896473738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01879490 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.257+0000 7f6e91a47740 1 --2- 172.21.15.103:0/3896473738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 0x55ca01878f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.257+0000 7f6e91a47740 1 --2- 172.21.15.103:0/3896473738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca018715d0 0x55ca0187bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.257+0000 7f6e70ff9640 0 client.18828 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:59:10.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.257+0000 7f6e91a47740 1 -- 172.21.15.103:0/3896473738 >> 172.21.15.103:0/3896473738 conn(0x55ca0183a310 msgr2=0x55ca01873930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:10.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.257+0000 7f6e91a47740 1 -- 172.21.15.103:0/3896473738 shutdown_connections 2024-09-19T06:59:10.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.257+0000 7f6e91a47740 1 -- 172.21.15.103:0/3896473738 wait complete. 2024-09-19T06:59:10.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.258+0000 7f6e91a47740 20 client.18828 trim_cache size 0 max 16384 2024-09-19T06:59:10.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.270+0000 7f6e91a47740 1 Processor -- start 2024-09-19T06:59:10.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e91a47740 1 -- start start 2024-09-19T06:59:10.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e91a47740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca018715d0 0x55ca01888310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e91a47740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 0x55ca01870bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e91a47740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01893c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e91a47740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e881ad8f0 con 0x55ca0187c370 2024-09-19T06:59:10.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e91a47740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6e881af430 con 0x55ca01872b40 2024-09-19T06:59:10.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e91a47740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e881adfc0 con 0x55ca018715d0 2024-09-19T06:59:10.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e8dc0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01893c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e8dc0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01893c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60604/0 (socket says 172.21.15.103:60604) 2024-09-19T06:59:10.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e8dc0b640 1 -- 172.21.15.103:0/2011630037 learned_addr learned my addr 172.21.15.103:0/2011630037 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:10.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e8cc09640 1 --2- 172.21.15.103:0/2011630037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 0x55ca01870bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e8d40a640 1 --2- 172.21.15.103:0/2011630037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca018715d0 0x55ca01888310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e8dc0b640 1 -- 172.21.15.103:0/2011630037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca018715d0 msgr2=0x55ca01888310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:10.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e8dc0b640 1 --2- 172.21.15.103:0/2011630037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca018715d0 0x55ca01888310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e8dc0b640 1 -- 172.21.15.103:0/2011630037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 msgr2=0x55ca01870bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:10.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e8dc0b640 1 --2- 172.21.15.103:0/2011630037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 0x55ca01870bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e8dc0b640 1 -- 172.21.15.103:0/2011630037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca01871d50 con 0x55ca0187c370 2024-09-19T06:59:10.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e8cc09640 1 --2- 172.21.15.103:0/2011630037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 0x55ca01870bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:10.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e8d40a640 1 --2- 172.21.15.103:0/2011630037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca018715d0 0x55ca01888310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:10.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e8dc0b640 1 --2- 172.21.15.103:0/2011630037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01893c70 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f6e800c5c20 tx=0x7f6e800bd020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7044d969b08b74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:10.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.271+0000 7f6e72ffd640 1 -- 172.21.15.103:0/2011630037 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800bdbe0 con 0x55ca0187c370 2024-09-19T06:59:10.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.272+0000 7f6e72ffd640 1 -- 172.21.15.103:0/2011630037 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e80036070 con 0x55ca0187c370 2024-09-19T06:59:10.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.272+0000 7f6e72ffd640 1 -- 172.21.15.103:0/2011630037 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800eea30 con 0x55ca0187c370 2024-09-19T06:59:10.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.272+0000 7f6e91a47740 1 -- 172.21.15.103:0/2011630037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 msgr2=0x55ca01893c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:10.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.272+0000 7f6e91a47740 1 --2- 172.21.15.103:0/2011630037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01893c70 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f6e800c5c20 tx=0x7f6e800bd020 comp rx=0 tx=0).stop 2024-09-19T06:59:10.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.272+0000 7f6e91a47740 1 -- 172.21.15.103:0/2011630037 shutdown_connections 2024-09-19T06:59:10.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.272+0000 7f6e91a47740 1 --2- 172.21.15.103:0/2011630037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01893c70 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.272+0000 7f6e91a47740 1 --2- 172.21.15.103:0/2011630037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 0x55ca01870bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.272+0000 7f6e91a47740 1 --2- 172.21.15.103:0/2011630037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca018715d0 0x55ca01888310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.272+0000 7f6e91a47740 1 -- 172.21.15.103:0/2011630037 >> 172.21.15.103:0/2011630037 conn(0x55ca0183a310 msgr2=0x55ca01877820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:10.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.272+0000 7f6e91a47740 1 -- 172.21.15.103:0/2011630037 shutdown_connections 2024-09-19T06:59:10.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.272+0000 7f6e91a47740 1 -- 172.21.15.103:0/2011630037 wait complete. 2024-09-19T06:59:10.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e91a47740 1 -- start start 2024-09-19T06:59:10.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e91a47740 1 Processor -- start 2024-09-19T06:59:10.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e91a47740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca018715d0 0x55ca018760d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e91a47740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 0x55ca01876610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e91a47740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01876b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e91a47740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e881aed60 con 0x55ca0187c370 2024-09-19T06:59:10.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e91a47740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6e881ae690 con 0x55ca01872b40 2024-09-19T06:59:10.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e91a47740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e881ab8f0 con 0x55ca018715d0 2024-09-19T06:59:10.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e91a47740 10 client.0 _ll_register_callbacks cb 0x7f6e800b90c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:10.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e8dc0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01876b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e8dc0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01876b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60614/0 (socket says 172.21.15.103:60614) 2024-09-19T06:59:10.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e8dc0b640 1 -- 172.21.15.103:0/3405375136 learned_addr learned my addr 172.21.15.103:0/3405375136 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:10.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e8d40a640 1 --2- 172.21.15.103:0/3405375136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca018715d0 0x55ca018760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e8cc09640 1 --2- 172.21.15.103:0/3405375136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 0x55ca01876610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e8dc0b640 1 -- 172.21.15.103:0/3405375136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca018715d0 msgr2=0x55ca018760d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:10.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e8dc0b640 1 --2- 172.21.15.103:0/3405375136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca018715d0 0x55ca018760d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e8dc0b640 1 -- 172.21.15.103:0/3405375136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 msgr2=0x55ca01876610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:10.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e8dc0b640 1 --2- 172.21.15.103:0/3405375136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 0x55ca01876610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.273+0000 7f6e8dc0b640 1 -- 172.21.15.103:0/3405375136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca018fe5b0 con 0x55ca0187c370 2024-09-19T06:59:10.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.274+0000 7f6e8dc0b640 1 --2- 172.21.15.103:0/3405375136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01876b50 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f6e80031b50 tx=0x7f6e8002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f171c993051e6f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:10.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.274+0000 7f6e70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:10.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.274+0000 7f6e70ff9640 1 -- 172.21.15.103:0/3405375136 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800c6150 con 0x55ca0187c370 2024-09-19T06:59:10.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.274+0000 7f6e70ff9640 1 -- 172.21.15.103:0/3405375136 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7f6e800d62a0 con 0x55ca0187c370 2024-09-19T06:59:10.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.274+0000 7f6e91a47740 1 -- 172.21.15.103:0/3405375136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e80036070 con 0x55ca0187c370 2024-09-19T06:59:10.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.274+0000 7f6e70ff9640 1 -- 172.21.15.103:0/3405375136 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6e800c1890 con 0x55ca0187c370 2024-09-19T06:59:10.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.274+0000 7f6e91a47740 10 client.18834 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:10.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.274+0000 7f6e91a47740 1 -- 172.21.15.103:0/3405375136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ca018fe5b0 con 0x55ca0187c370 2024-09-19T06:59:10.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.274+0000 7f6e70ff9640 1 -- 172.21.15.103:0/3405375136 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e80036070 con 0x55ca0187c370 2024-09-19T06:59:10.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.274+0000 7f6e70ff9640 1 -- 172.21.15.103:0/3405375136 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6e80004b40 con 0x55ca0187c370 2024-09-19T06:59:10.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.274+0000 7f6e91a47740 10 client.18834 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:10.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.274+0000 7f6e91a47740 10 client.18834 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:59:10.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.274+0000 7f6e91a47740 20 client.18834 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:10.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.274+0000 7f6e91a47740 1 --2- 172.21.15.103:0/3405375136 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55ca018ffb00 0x55ca01901ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:10.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.274+0000 7f6e91a47740 4 client.18834 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:59:10.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.274+0000 7f6e91a47740 1 -- 172.21.15.103:0/3405375136 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca017ded70 con 0x55ca018ffb00 2024-09-19T06:59:10.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.275+0000 7f6e8d40a640 1 --2- 172.21.15.103:0/3405375136 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55ca018ffb00 0x55ca01901ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:10.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.275+0000 7f6e70ff9640 10 client.18834 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:59:10.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.275+0000 7f6e8d40a640 1 --2- 172.21.15.103:0/3405375136 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55ca018ffb00 0x55ca01901ef0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=53a57ddcdd3fd548 server_cookie=6f8907df5ca55fc2 in_seq=0 out_seq=0 2024-09-19T06:59:10.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.275+0000 7f6e70ff9640 1 -- 172.21.15.103:0/3405375136 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca017ded70 con 0x55ca018ffb00 2024-09-19T06:59:10.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.275+0000 7f6e70ff9640 10 client.18834 handle_command_reply: tid=0 2024-09-19T06:59:10.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.275+0000 7f6e91a47740 1 client.18834 shutdown 2024-09-19T06:59:10.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.275+0000 7f6e91a47740 1 -- 172.21.15.103:0/3405375136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 msgr2=0x55ca01876b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:10.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.275+0000 7f6e91a47740 1 --2- 172.21.15.103:0/3405375136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01876b50 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f6e80031b50 tx=0x7f6e8002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.275+0000 7f6e91a47740 1 -- 172.21.15.103:0/3405375136 shutdown_connections 2024-09-19T06:59:10.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.275+0000 7f6e91a47740 1 --2- 172.21.15.103:0/3405375136 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55ca018ffb00 0x55ca01901ef0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.275+0000 7f6e91a47740 1 --2- 172.21.15.103:0/3405375136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca0187c370 0x55ca01876b50 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.275+0000 7f6e91a47740 1 --2- 172.21.15.103:0/3405375136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca01872b40 0x55ca01876610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.275+0000 7f6e91a47740 1 --2- 172.21.15.103:0/3405375136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca018715d0 0x55ca018760d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.275+0000 7f6e70ff9640 0 client.18834 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:59:10.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.275+0000 7f6e91a47740 1 -- 172.21.15.103:0/3405375136 >> 172.21.15.103:0/3405375136 conn(0x55ca0183a310 msgr2=0x55ca0187a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:10.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.276+0000 7f6e91a47740 1 -- 172.21.15.103:0/3405375136 shutdown_connections 2024-09-19T06:59:10.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.276+0000 7f6e91a47740 1 -- 172.21.15.103:0/3405375136 wait complete. 2024-09-19T06:59:10.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.276+0000 7f6e91a47740 20 client.18834 trim_cache size 0 max 16384 2024-09-19T06:59:10.289 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:59:10.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.276+0000 7f6e8f66e640 1 -- 172.21.15.103:0/2129919311 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6e740a33b0 msgr2=0x7f6e740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.276+0000 7f6e8f66e640 1 --2- 172.21.15.103:0/2129919311 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6e740a33b0 0x7f6e740a5870 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55ca0186eac0 tx=0x7f6e780092c0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.276+0000 7f6e8f66e640 1 -- 172.21.15.103:0/2129919311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e88106f20 msgr2=0x7f6e8819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.276+0000 7f6e8f66e640 1 --2- 172.21.15.103:0/2129919311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e88106f20 0x7f6e8819cdd0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6e80031a60 tx=0x7f6e80031a90 comp rx=0 tx=0).stop 2024-09-19T06:59:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.276+0000 7f6e8f66e640 1 -- 172.21.15.103:0/2129919311 shutdown_connections 2024-09-19T06:59:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.276+0000 7f6e8f66e640 1 --2- 172.21.15.103:0/2129919311 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f6e740a33b0 0x7f6e740a5870 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.276+0000 7f6e8f66e640 1 --2- 172.21.15.103:0/2129919311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e88106f20 0x7f6e8819cdd0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.276+0000 7f6e8f66e640 1 --2- 172.21.15.103:0/2129919311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e881027a0 0x7f6e88198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.276+0000 7f6e8f66e640 1 --2- 172.21.15.103:0/2129919311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e88101df0 0x7f6e88198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.276+0000 7f6e8f66e640 1 -- 172.21.15.103:0/2129919311 >> 172.21.15.103:0/2129919311 conn(0x7f6e880fd8b0 msgr2=0x7f6e880fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.276+0000 7f6e8f66e640 1 -- 172.21.15.103:0/2129919311 shutdown_connections 2024-09-19T06:59:10.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:10.276+0000 7f6e8f66e640 1 -- 172.21.15.103:0/2129919311 wait complete. 2024-09-19T06:59:11.062 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:59:11.312 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:11.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.731+0000 7f98dca53640 1 -- 172.21.15.103:0/1398218581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d8106f10 msgr2=0x7f98d810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:11.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.731+0000 7f98dca53640 1 --2- 172.21.15.103:0/1398218581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d8106f10 0x7f98d810ea90 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009ca0 tx=0x7f98d002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.731+0000 7f98dca53640 1 -- 172.21.15.103:0/1398218581 shutdown_connections 2024-09-19T06:59:11.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.731+0000 7f98dca53640 1 --2- 172.21.15.103:0/1398218581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d8106f10 0x7f98d810ea90 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.731+0000 7f98dca53640 1 --2- 172.21.15.103:0/1398218581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98d8102790 0x7f98d81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.731+0000 7f98dca53640 1 --2- 172.21.15.103:0/1398218581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98d8101de0 0x7f98d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.731+0000 7f98dca53640 1 -- 172.21.15.103:0/1398218581 >> 172.21.15.103:0/1398218581 conn(0x7f98d80fd8c0 msgr2=0x7f98d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:11.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.731+0000 7f98dca53640 1 -- 172.21.15.103:0/1398218581 shutdown_connections 2024-09-19T06:59:11.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.731+0000 7f98dca53640 1 -- 172.21.15.103:0/1398218581 wait complete. 2024-09-19T06:59:11.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.732+0000 7f98dca53640 1 Processor -- start 2024-09-19T06:59:11.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.732+0000 7f98dca53640 1 -- start start 2024-09-19T06:59:11.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.732+0000 7f98dca53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d8101de0 0x7f98d80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.732+0000 7f98dca53640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98d8102790 0x7f98d8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.732+0000 7f98dca53640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98d8106f10 0x7f98d8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.732+0000 7f98dca53640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98d8111f70 con 0x7f98d8101de0 2024-09-19T06:59:11.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.732+0000 7f98dca53640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98d8111df0 con 0x7f98d8102790 2024-09-19T06:59:11.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.732+0000 7f98dca53640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98d81120f0 con 0x7f98d8106f10 2024-09-19T06:59:11.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.732+0000 7f98d659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d8101de0 0x7f98d80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.732+0000 7f98d659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d8101de0 0x7f98d80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60634/0 (socket says 172.21.15.103:60634) 2024-09-19T06:59:11.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.732+0000 7f98d659c640 1 -- 172.21.15.103:0/2279569641 learned_addr learned my addr 172.21.15.103:0/2279569641 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:11.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.732+0000 7f98d6d9d640 1 --2- 172.21.15.103:0/2279569641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98d8106f10 0x7f98d8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.732+0000 7f98d5d9b640 1 --2- 172.21.15.103:0/2279569641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98d8102790 0x7f98d8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.733+0000 7f98d659c640 1 -- 172.21.15.103:0/2279569641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98d8106f10 msgr2=0x7f98d8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:11.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.733+0000 7f98d659c640 1 --2- 172.21.15.103:0/2279569641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98d8106f10 0x7f98d8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.733+0000 7f98d659c640 1 -- 172.21.15.103:0/2279569641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98d8102790 msgr2=0x7f98d8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:11.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.733+0000 7f98d659c640 1 --2- 172.21.15.103:0/2279569641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98d8102790 0x7f98d8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.733+0000 7f98d659c640 1 -- 172.21.15.103:0/2279569641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d81a6870 con 0x7f98d8101de0 2024-09-19T06:59:11.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.733+0000 7f98d5d9b640 1 --2- 172.21.15.103:0/2279569641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98d8102790 0x7f98d8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:11.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.733+0000 7f98d659c640 1 --2- 172.21.15.103:0/2279569641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d8101de0 0x7f98d80744f0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f98c000c890 tx=0x7f98c000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:11.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.733+0000 7f98cf7fe640 1 -- 172.21.15.103:0/2279569641 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0004220 con 0x7f98d8101de0 2024-09-19T06:59:11.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.733+0000 7f98dca53640 1 -- 172.21.15.103:0/2279569641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d81a6b60 con 0x7f98d8101de0 2024-09-19T06:59:11.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.733+0000 7f98cf7fe640 1 -- 172.21.15.103:0/2279569641 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98c00043c0 con 0x7f98d8101de0 2024-09-19T06:59:11.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.733+0000 7f98cf7fe640 1 -- 172.21.15.103:0/2279569641 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0004a00 con 0x7f98d8101de0 2024-09-19T06:59:11.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.733+0000 7f98dca53640 1 -- 172.21.15.103:0/2279569641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d81a70a0 con 0x7f98d8101de0 2024-09-19T06:59:11.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.736+0000 7f98cf7fe640 1 -- 172.21.15.103:0/2279569641 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f98c00079b0 con 0x7f98d8101de0 2024-09-19T06:59:11.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.736+0000 7f98dee2c740 1 Processor -- start 2024-09-19T06:59:11.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.736+0000 7f98dee2c740 1 -- start start 2024-09-19T06:59:11.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.736+0000 7f98dee2c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b92b40 0x556962b91090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.736+0000 7f98dee2c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b91a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.736+0000 7f98dee2c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b9c370 0x556962b9e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.736+0000 7f98dee2c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556962af72f0 con 0x556962b915d0 2024-09-19T06:59:11.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.736+0000 7f98dee2c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556962a9ecf0 con 0x556962b9c370 2024-09-19T06:59:11.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.736+0000 7f98dee2c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556962af32b0 con 0x556962b92b40 2024-09-19T06:59:11.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.736+0000 7f98cf7fe640 1 --2- 172.21.15.103:0/2279569641 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f98c40a33b0 0x7f98c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.736+0000 7f98d659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b91a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.736+0000 7f98d659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b91a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60648/0 (socket says 172.21.15.103:60648) 2024-09-19T06:59:11.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.736+0000 7f98d659c640 1 -- 172.21.15.103:0/3693667769 learned_addr learned my addr 172.21.15.103:0/3693667769 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:11.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.737+0000 7f98d6d9d640 1 --2- 172.21.15.103:0/3693667769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b92b40 0x556962b91090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.737+0000 7f98d5d9b640 1 --2- 172.21.15.103:0/3693667769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b9c370 0x556962b9e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.737+0000 7f98d6d9d640 1 --2- 172.21.15.103:0/2279569641 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f98c40a33b0 0x7f98c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.737+0000 7f98cf7fe640 1 -- 172.21.15.103:0/2279569641 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f98c00cfaa0 con 0x7f98d8101de0 2024-09-19T06:59:11.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.737+0000 7f98d659c640 1 -- 172.21.15.103:0/3693667769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b92b40 msgr2=0x556962b91090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:11.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.737+0000 7f98d659c640 1 --2- 172.21.15.103:0/3693667769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b92b40 0x556962b91090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.737+0000 7f98d659c640 1 -- 172.21.15.103:0/3693667769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b9c370 msgr2=0x556962b9e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:11.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.737+0000 7f98d659c640 1 --2- 172.21.15.103:0/3693667769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b9c370 0x556962b9e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.737+0000 7f98d659c640 1 -- 172.21.15.103:0/3693667769 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556962a833a0 con 0x556962b915d0 2024-09-19T06:59:11.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.737+0000 7f98d6d9d640 1 --2- 172.21.15.103:0/2279569641 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f98c40a33b0 0x7f98c40a5870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f98d0005fe0 tx=0x7f98d0005f50 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:11.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.737+0000 7f98d659c640 1 --2- 172.21.15.103:0/3693667769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b91a10 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f98c009bc90 tx=0x7f98c00054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f1eba38d85804 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:11.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.737+0000 7f98ccff9640 1 -- 172.21.15.103:0/3693667769 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c001d030 con 0x556962b915d0 2024-09-19T06:59:11.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.737+0000 7f98ccff9640 1 -- 172.21.15.103:0/3693667769 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98c009c400 con 0x556962b915d0 2024-09-19T06:59:11.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.737+0000 7f98ccff9640 1 -- 172.21.15.103:0/3693667769 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c00cd3f0 con 0x556962b915d0 2024-09-19T06:59:11.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.738+0000 7f98dee2c740 1 -- 172.21.15.103:0/3693667769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 msgr2=0x556962b91a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:11.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.738+0000 7f98dee2c740 1 --2- 172.21.15.103:0/3693667769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b91a10 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f98c009bc90 tx=0x7f98c00054a0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.738+0000 7f98dee2c740 1 -- 172.21.15.103:0/3693667769 shutdown_connections 2024-09-19T06:59:11.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.738+0000 7f98dee2c740 1 --2- 172.21.15.103:0/3693667769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b9c370 0x556962b9e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.738+0000 7f98dee2c740 1 --2- 172.21.15.103:0/3693667769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b91a10 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.738+0000 7f98dee2c740 1 --2- 172.21.15.103:0/3693667769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b92b40 0x556962b91090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.738+0000 7f98dee2c740 1 -- 172.21.15.103:0/3693667769 >> 172.21.15.103:0/3693667769 conn(0x556962b5a310 msgr2=0x556962b8dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:11.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.738+0000 7f98dee2c740 1 -- 172.21.15.103:0/3693667769 shutdown_connections 2024-09-19T06:59:11.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.738+0000 7f98dee2c740 1 -- 172.21.15.103:0/3693667769 wait complete. 2024-09-19T06:59:11.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98dee2c740 1 -- start start 2024-09-19T06:59:11.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98dee2c740 1 Processor -- start 2024-09-19T06:59:11.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98dee2c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b915d0 0x556962b9bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98dee2c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b92b40 0x556962b99060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98dee2c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b9c370 0x556962b997f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98dee2c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556962b9f010 con 0x556962b9c370 2024-09-19T06:59:11.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98dee2c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556962ab2060 con 0x556962b92b40 2024-09-19T06:59:11.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98dee2c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556962b9f190 con 0x556962b915d0 2024-09-19T06:59:11.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98dee2c740 10 client.0 _ll_register_callbacks cb 0x5569628058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:11.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98d6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b9c370 0x556962b997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98d6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b9c370 0x556962b997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60650/0 (socket says 172.21.15.103:60650) 2024-09-19T06:59:11.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98d6d9d640 1 -- 172.21.15.103:0/1260507215 learned_addr learned my addr 172.21.15.103:0/1260507215 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:11.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98d5d9b640 1 --2- 172.21.15.103:0/1260507215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b92b40 0x556962b99060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98d659c640 1 --2- 172.21.15.103:0/1260507215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b915d0 0x556962b9bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98d6d9d640 1 -- 172.21.15.103:0/1260507215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b915d0 msgr2=0x556962b9bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:11.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98d6d9d640 1 --2- 172.21.15.103:0/1260507215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b915d0 0x556962b9bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98d6d9d640 1 -- 172.21.15.103:0/1260507215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b92b40 msgr2=0x556962b99060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:11.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98d6d9d640 1 --2- 172.21.15.103:0/1260507215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b92b40 0x556962b99060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98d6d9d640 1 -- 172.21.15.103:0/1260507215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569628683c0 con 0x556962b9c370 2024-09-19T06:59:11.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.739+0000 7f98d5d9b640 1 --2- 172.21.15.103:0/1260507215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b92b40 0x556962b99060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:11.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.740+0000 7f98d6d9d640 1 --2- 172.21.15.103:0/1260507215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b9c370 0x556962b997f0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009ca0 tx=0x7f98d00093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d2a3ad6ca6dbce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:11.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.740+0000 7f98bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:11.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.740+0000 7f98bdffb640 1 -- 172.21.15.103:0/1260507215 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0030890 con 0x556962b9c370 2024-09-19T06:59:11.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.740+0000 7f98bdffb640 1 -- 172.21.15.103:0/1260507215 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f98d0030a30 con 0x556962b9c370 2024-09-19T06:59:11.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.740+0000 7f98dee2c740 1 -- 172.21.15.103:0/1260507215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98c009c400 con 0x556962b9c370 2024-09-19T06:59:11.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.740+0000 7f98bdffb640 1 -- 172.21.15.103:0/1260507215 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f98d0030c30 con 0x556962b9c370 2024-09-19T06:59:11.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.740+0000 7f98dee2c740 10 client.18846 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:11.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.740+0000 7f98dee2c740 1 -- 172.21.15.103:0/1260507215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556962baab00 con 0x556962b9c370 2024-09-19T06:59:11.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.740+0000 7f98bdffb640 1 -- 172.21.15.103:0/1260507215 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98c009c400 con 0x556962b9c370 2024-09-19T06:59:11.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.741+0000 7f98bdffb640 1 -- 172.21.15.103:0/1260507215 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f98d004a440 con 0x556962b9c370 2024-09-19T06:59:11.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.741+0000 7f98dee2c740 10 client.18846 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:11.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.741+0000 7f98dee2c740 10 client.18846 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:59:11.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.741+0000 7f98dee2c740 20 client.18846 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:11.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.741+0000 7f98dee2c740 1 --2- 172.21.15.103:0/1260507215 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x556962babf60 0x556962bae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.741+0000 7f98dee2c740 4 client.18846 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:11.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.741+0000 7f98dee2c740 1 -- 172.21.15.103:0/1260507215 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556962b04860 con 0x556962babf60 2024-09-19T06:59:11.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.741+0000 7f98d659c640 1 --2- 172.21.15.103:0/1260507215 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x556962babf60 0x556962bae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.741+0000 7f98bdffb640 10 client.18846 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:59:11.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.741+0000 7f98d659c640 1 --2- 172.21.15.103:0/1260507215 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x556962babf60 0x556962bae3c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=86c2799cf874d428 server_cookie=4a7b6723ad744dc2 in_seq=0 out_seq=0 2024-09-19T06:59:11.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.742+0000 7f98bdffb640 1 -- 172.21.15.103:0/1260507215 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556962b04860 con 0x556962babf60 2024-09-19T06:59:11.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.742+0000 7f98bdffb640 10 client.18846 handle_command_reply: tid=0 2024-09-19T06:59:11.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.742+0000 7f98dee2c740 1 client.18846 shutdown 2024-09-19T06:59:11.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.742+0000 7f98dee2c740 1 -- 172.21.15.103:0/1260507215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b9c370 msgr2=0x556962b997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:11.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.742+0000 7f98dee2c740 1 --2- 172.21.15.103:0/1260507215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b9c370 0x556962b997f0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009ca0 tx=0x7f98d00093f0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.742+0000 7f98dee2c740 1 -- 172.21.15.103:0/1260507215 shutdown_connections 2024-09-19T06:59:11.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.742+0000 7f98dee2c740 1 --2- 172.21.15.103:0/1260507215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b9c370 0x556962b997f0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.742+0000 7f98dee2c740 1 --2- 172.21.15.103:0/1260507215 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x556962babf60 0x556962bae3c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.742+0000 7f98dee2c740 1 --2- 172.21.15.103:0/1260507215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b92b40 0x556962b99060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.742+0000 7f98dee2c740 1 --2- 172.21.15.103:0/1260507215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b915d0 0x556962b9bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.742+0000 7f98dee2c740 1 -- 172.21.15.103:0/1260507215 >> 172.21.15.103:0/1260507215 conn(0x556962b5a310 msgr2=0x556962b93960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:11.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.742+0000 7f98bdffb640 0 client.18846 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:59:11.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.743+0000 7f98dee2c740 1 -- 172.21.15.103:0/1260507215 shutdown_connections 2024-09-19T06:59:11.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.743+0000 7f98dee2c740 1 -- 172.21.15.103:0/1260507215 wait complete. 2024-09-19T06:59:11.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.743+0000 7f98dee2c740 20 client.18846 trim_cache size 0 max 16384 2024-09-19T06:59:11.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.755+0000 7f98dee2c740 1 Processor -- start 2024-09-19T06:59:11.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.755+0000 7f98dee2c740 1 -- start start 2024-09-19T06:59:11.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98dee2c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b99770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98dee2c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b92b40 0x556962b8d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98dee2c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b9c370 0x556962b8dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98dee2c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98d81afb40 con 0x556962b915d0 2024-09-19T06:59:11.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98dee2c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98d81b1680 con 0x556962b92b40 2024-09-19T06:59:11.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98dee2c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98d81b0210 con 0x556962b9c370 2024-09-19T06:59:11.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98d659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b99770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98d659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b99770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60666/0 (socket says 172.21.15.103:60666) 2024-09-19T06:59:11.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98d659c640 1 -- 172.21.15.103:0/3786412675 learned_addr learned my addr 172.21.15.103:0/3786412675 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:11.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98d6d9d640 1 --2- 172.21.15.103:0/3786412675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b9c370 0x556962b8dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98d5d9b640 1 --2- 172.21.15.103:0/3786412675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b92b40 0x556962b8d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98d659c640 1 -- 172.21.15.103:0/3786412675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b9c370 msgr2=0x556962b8dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:11.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98d659c640 1 --2- 172.21.15.103:0/3786412675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b9c370 0x556962b8dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98d659c640 1 -- 172.21.15.103:0/3786412675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b92b40 msgr2=0x556962b8d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:11.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98d659c640 1 --2- 172.21.15.103:0/3786412675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b92b40 0x556962b8d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98d659c640 1 -- 172.21.15.103:0/3786412675 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556962b8e180 con 0x556962b915d0 2024-09-19T06:59:11.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98d659c640 1 --2- 172.21.15.103:0/3786412675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b99770 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f98c0098160 tx=0x7f98c00cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c6746dd6f46a76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:11.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98bffff640 1 -- 172.21.15.103:0/3786412675 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c00b9070 con 0x556962b915d0 2024-09-19T06:59:11.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98bffff640 1 -- 172.21.15.103:0/3786412675 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98c009bd00 con 0x556962b915d0 2024-09-19T06:59:11.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.756+0000 7f98bffff640 1 -- 172.21.15.103:0/3786412675 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c001d040 con 0x556962b915d0 2024-09-19T06:59:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.757+0000 7f98dee2c740 1 -- 172.21.15.103:0/3786412675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 msgr2=0x556962b99770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.757+0000 7f98dee2c740 1 --2- 172.21.15.103:0/3786412675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b99770 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f98c0098160 tx=0x7f98c00cf040 comp rx=0 tx=0).stop 2024-09-19T06:59:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.757+0000 7f98dee2c740 1 -- 172.21.15.103:0/3786412675 shutdown_connections 2024-09-19T06:59:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.757+0000 7f98dee2c740 1 --2- 172.21.15.103:0/3786412675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b9c370 0x556962b8dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.757+0000 7f98dee2c740 1 --2- 172.21.15.103:0/3786412675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b92b40 0x556962b8d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.757+0000 7f98dee2c740 1 --2- 172.21.15.103:0/3786412675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b99770 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.757+0000 7f98dee2c740 1 -- 172.21.15.103:0/3786412675 >> 172.21.15.103:0/3786412675 conn(0x556962b5a310 msgr2=0x556962b88a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.757+0000 7f98dee2c740 1 -- 172.21.15.103:0/3786412675 shutdown_connections 2024-09-19T06:59:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.757+0000 7f98dee2c740 1 -- 172.21.15.103:0/3786412675 wait complete. 2024-09-19T06:59:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.757+0000 7f98dee2c740 1 -- start start 2024-09-19T06:59:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98dee2c740 1 Processor -- start 2024-09-19T06:59:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98dee2c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b96130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98dee2c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b92b40 0x556962b96670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98dee2c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b9c370 0x556962b96bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98dee2c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98d81b0fb0 con 0x556962b915d0 2024-09-19T06:59:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98dee2c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98d81b08e0 con 0x556962b92b40 2024-09-19T06:59:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98dee2c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98d81adb40 con 0x556962b9c370 2024-09-19T06:59:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98d659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b96130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98d659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b96130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60676/0 (socket says 172.21.15.103:60676) 2024-09-19T06:59:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98d659c640 1 -- 172.21.15.103:0/1788426694 learned_addr learned my addr 172.21.15.103:0/1788426694 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98dee2c740 10 client.0 _ll_register_callbacks cb 0x7f98d0031f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98d6d9d640 1 --2- 172.21.15.103:0/1788426694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b9c370 0x556962b96bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98d5d9b640 1 --2- 172.21.15.103:0/1788426694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b92b40 0x556962b96670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98d659c640 1 -- 172.21.15.103:0/1788426694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b9c370 msgr2=0x556962b96bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98d659c640 1 --2- 172.21.15.103:0/1788426694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b9c370 0x556962b96bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98d659c640 1 -- 172.21.15.103:0/1788426694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b92b40 msgr2=0x556962b96670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98d659c640 1 --2- 172.21.15.103:0/1788426694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b92b40 0x556962b96670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98d659c640 1 -- 172.21.15.103:0/1788426694 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556962c15d10 con 0x556962b915d0 2024-09-19T06:59:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98d659c640 1 --2- 172.21.15.103:0/1788426694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b96130 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f98c00a4440 tx=0x7f98c00cd250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d112e7571e60ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.758+0000 7f98bdffb640 1 -- 172.21.15.103:0/1788426694 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c001d040 con 0x556962b915d0 2024-09-19T06:59:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.759+0000 7f98bdffb640 1 -- 172.21.15.103:0/1788426694 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f98c00cd3f0 con 0x556962b915d0 2024-09-19T06:59:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.759+0000 7f98dee2c740 1 -- 172.21.15.103:0/1788426694 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556962afed70 con 0x556962b915d0 2024-09-19T06:59:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.759+0000 7f98bdffb640 1 -- 172.21.15.103:0/1788426694 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f98c00b9040 con 0x556962b915d0 2024-09-19T06:59:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.759+0000 7f98dee2c740 10 client.18852 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.759+0000 7f98dee2c740 1 -- 172.21.15.103:0/1788426694 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556962c16ff0 con 0x556962b915d0 2024-09-19T06:59:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.759+0000 7f98bdffb640 1 -- 172.21.15.103:0/1788426694 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556962afed70 con 0x556962b915d0 2024-09-19T06:59:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.759+0000 7f98bdffb640 1 -- 172.21.15.103:0/1788426694 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f98c009abc0 con 0x556962b915d0 2024-09-19T06:59:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.759+0000 7f98dee2c740 10 client.18852 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.759+0000 7f98dee2c740 10 client.18852 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:59:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.759+0000 7f98dee2c740 20 client.18852 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.759+0000 7f98dee2c740 1 --2- 172.21.15.103:0/1788426694 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x556962c17210 0x556962c19600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.759+0000 7f98dee2c740 4 client.18852 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:59:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.759+0000 7f98dee2c740 1 -- 172.21.15.103:0/1788426694 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556962a7b170 con 0x556962c17210 2024-09-19T06:59:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.759+0000 7f98d5d9b640 1 --2- 172.21.15.103:0/1788426694 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x556962c17210 0x556962c19600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98bdffb640 10 client.18852 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:59:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98d5d9b640 1 --2- 172.21.15.103:0/1788426694 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x556962c17210 0x556962c19600 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e194a0b64340cd2 server_cookie=cffd8f20df8ca860 in_seq=0 out_seq=0 2024-09-19T06:59:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98bdffb640 1 -- 172.21.15.103:0/1788426694 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556962a7b170 con 0x556962c17210 2024-09-19T06:59:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98bdffb640 10 client.18852 handle_command_reply: tid=0 2024-09-19T06:59:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98dee2c740 1 client.18852 shutdown 2024-09-19T06:59:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98dee2c740 1 -- 172.21.15.103:0/1788426694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 msgr2=0x556962b96130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98dee2c740 1 --2- 172.21.15.103:0/1788426694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b96130 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f98c00a4440 tx=0x7f98c00cd250 comp rx=0 tx=0).stop 2024-09-19T06:59:11.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98dee2c740 1 -- 172.21.15.103:0/1788426694 shutdown_connections 2024-09-19T06:59:11.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98dee2c740 1 --2- 172.21.15.103:0/1788426694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556962b9c370 0x556962b96bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98dee2c740 1 --2- 172.21.15.103:0/1788426694 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x556962c17210 0x556962c19600 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98dee2c740 1 --2- 172.21.15.103:0/1788426694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556962b92b40 0x556962b96670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98dee2c740 1 --2- 172.21.15.103:0/1788426694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556962b915d0 0x556962b96130 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98bdffb640 0 client.18852 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:59:11.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98dee2c740 1 -- 172.21.15.103:0/1788426694 >> 172.21.15.103:0/1788426694 conn(0x556962b5a310 msgr2=0x556962b974f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:11.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98dee2c740 1 -- 172.21.15.103:0/1788426694 shutdown_connections 2024-09-19T06:59:11.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98dee2c740 1 -- 172.21.15.103:0/1788426694 wait complete. 2024-09-19T06:59:11.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.760+0000 7f98dee2c740 20 client.18852 trim_cache size 0 max 16384 2024-09-19T06:59:11.767 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:59:11.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.761+0000 7f98dca53640 1 -- 172.21.15.103:0/2279569641 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f98c40a33b0 msgr2=0x7f98c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.761+0000 7f98dca53640 1 --2- 172.21.15.103:0/2279569641 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f98c40a33b0 0x7f98c40a5870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f98d0005fe0 tx=0x7f98d0005f50 comp rx=0 tx=0).stop 2024-09-19T06:59:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.761+0000 7f98dca53640 1 -- 172.21.15.103:0/2279569641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d8101de0 msgr2=0x7f98d80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.761+0000 7f98dca53640 1 --2- 172.21.15.103:0/2279569641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d8101de0 0x7f98d80744f0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f98c000c890 tx=0x7f98c000cd50 comp rx=0 tx=0).stop 2024-09-19T06:59:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.761+0000 7f98dca53640 1 -- 172.21.15.103:0/2279569641 shutdown_connections 2024-09-19T06:59:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.761+0000 7f98dca53640 1 --2- 172.21.15.103:0/2279569641 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f98c40a33b0 0x7f98c40a5870 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.761+0000 7f98dca53640 1 --2- 172.21.15.103:0/2279569641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98d8106f10 0x7f98d8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.761+0000 7f98dca53640 1 --2- 172.21.15.103:0/2279569641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98d8102790 0x7f98d8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.761+0000 7f98dca53640 1 --2- 172.21.15.103:0/2279569641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d8101de0 0x7f98d80744f0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.761+0000 7f98dca53640 1 -- 172.21.15.103:0/2279569641 >> 172.21.15.103:0/2279569641 conn(0x7f98d80fd8c0 msgr2=0x7f98d80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.761+0000 7f98dca53640 1 -- 172.21.15.103:0/2279569641 shutdown_connections 2024-09-19T06:59:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:11.761+0000 7f98dca53640 1 -- 172.21.15.103:0/2279569641 wait complete. 2024-09-19T06:59:11.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:11 smithi134 ceph-mon[35356]: pgmap v308: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-19T06:59:11.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:11 smithi134 ceph-mon[35356]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T06:59:12.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:11 smithi174 ceph-mon[35637]: pgmap v308: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-19T06:59:12.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:11 smithi174 ceph-mon[35637]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T06:59:12.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:11 smithi103 ceph-mon[31158]: pgmap v308: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-19T06:59:12.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:11 smithi103 ceph-mon[31158]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T06:59:12.491 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:59:12.742 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:13.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.199+0000 7fa8a375c640 1 -- 172.21.15.103:0/2248005702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa89c100e20 msgr2=0x7fa89c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:13.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.199+0000 7fa8a375c640 1 --2- 172.21.15.103:0/2248005702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa89c100e20 0x7fa89c101200 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa898009900 tx=0x7fa89802a480 comp rx=0 tx=0).stop 2024-09-19T06:59:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.199+0000 7fa8a375c640 1 -- 172.21.15.103:0/2248005702 shutdown_connections 2024-09-19T06:59:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.199+0000 7fa8a375c640 1 --2- 172.21.15.103:0/2248005702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa89c102ef0 0x7fa89c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.199+0000 7fa8a375c640 1 --2- 172.21.15.103:0/2248005702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa89c102570 0x7fa89c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.199+0000 7fa8a375c640 1 --2- 172.21.15.103:0/2248005702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa89c100e20 0x7fa89c101200 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.199+0000 7fa8a375c640 1 -- 172.21.15.103:0/2248005702 >> 172.21.15.103:0/2248005702 conn(0x7fa89c0fc7d0 msgr2=0x7fa89c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.199+0000 7fa8a375c640 1 -- 172.21.15.103:0/2248005702 shutdown_connections 2024-09-19T06:59:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.199+0000 7fa8a375c640 1 -- 172.21.15.103:0/2248005702 wait complete. 2024-09-19T06:59:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.199+0000 7fa8a375c640 1 Processor -- start 2024-09-19T06:59:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.199+0000 7fa8a375c640 1 -- start start 2024-09-19T06:59:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a375c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa89c100e20 0x7fa89c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a375c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa89c102570 0x7fa89c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a375c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa89c102ef0 0x7fa89c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a375c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1120a0 con 0x7fa89c102570 2024-09-19T06:59:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a375c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c111f20 con 0x7fa89c100e20 2024-09-19T06:59:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a375c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c112220 con 0x7fa89c102ef0 2024-09-19T06:59:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a0cf7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa89c102570 0x7fa89c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a0cf7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa89c102570 0x7fa89c198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60692/0 (socket says 172.21.15.103:60692) 2024-09-19T06:59:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a0cf7640 1 -- 172.21.15.103:0/429343463 learned_addr learned my addr 172.21.15.103:0/429343463 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a14f8640 1 --2- 172.21.15.103:0/429343463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa89c100e20 0x7fa89c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a1cf9640 1 --2- 172.21.15.103:0/429343463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa89c102ef0 0x7fa89c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a0cf7640 1 -- 172.21.15.103:0/429343463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa89c102ef0 msgr2=0x7fa89c19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a0cf7640 1 --2- 172.21.15.103:0/429343463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa89c102ef0 0x7fa89c19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a0cf7640 1 -- 172.21.15.103:0/429343463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa89c100e20 msgr2=0x7fa89c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a0cf7640 1 --2- 172.21.15.103:0/429343463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa89c100e20 0x7fa89c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a0cf7640 1 -- 172.21.15.103:0/429343463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89c19d470 con 0x7fa89c102570 2024-09-19T06:59:13.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a0cf7640 1 --2- 172.21.15.103:0/429343463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa89c102570 0x7fa89c198a00 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00d770 tx=0x7fa88c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:13.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8927fc640 1 -- 172.21.15.103:0/429343463 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c017070 con 0x7fa89c102570 2024-09-19T06:59:13.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8927fc640 1 -- 172.21.15.103:0/429343463 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa88c005500 con 0x7fa89c102570 2024-09-19T06:59:13.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.200+0000 7fa8a375c640 1 -- 172.21.15.103:0/429343463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89c19d760 con 0x7fa89c102570 2024-09-19T06:59:13.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.201+0000 7fa8927fc640 1 -- 172.21.15.103:0/429343463 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c004e00 con 0x7fa89c102570 2024-09-19T06:59:13.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.201+0000 7fa8a375c640 1 -- 172.21.15.103:0/429343463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa89c1a5040 con 0x7fa89c102570 2024-09-19T06:59:13.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.203+0000 7fa8927fc640 1 -- 172.21.15.103:0/429343463 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa88c004fa0 con 0x7fa89c102570 2024-09-19T06:59:13.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.203+0000 7fa8a5b35740 1 Processor -- start 2024-09-19T06:59:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.203+0000 7fa8a5b35740 1 -- start start 2024-09-19T06:59:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.203+0000 7fa8a5b35740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deaeab40 0x5638deae9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a5b35740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deae95d0 0x5638deae9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8927fc640 1 --2- 172.21.15.103:0/429343463 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa8880a33b0 0x7fa8880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a5b35740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaf4370 0x5638deaf6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a5b35740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5638dea4f2f0 con 0x5638deaf4370 2024-09-19T06:59:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a5b35740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5638de9f6cf0 con 0x5638deaeab40 2024-09-19T06:59:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a5b35740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5638dea4b2b0 con 0x5638deae95d0 2024-09-19T06:59:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a1cf9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaf4370 0x5638deaf6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a1cf9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaf4370 0x5638deaf6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60700/0 (socket says 172.21.15.103:60700) 2024-09-19T06:59:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a1cf9640 1 -- 172.21.15.103:0/2578566540 learned_addr learned my addr 172.21.15.103:0/2578566540 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a14f8640 1 --2- 172.21.15.103:0/2578566540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deae95d0 0x5638deae9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8927fc640 1 -- 172.21.15.103:0/429343463 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7fa88c0cf820 con 0x7fa89c102570 2024-09-19T06:59:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a1cf9640 1 --2- 172.21.15.103:0/2578566540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deaeab40 0x5638deae9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a0cf7640 1 --2- 172.21.15.103:0/429343463 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa8880a33b0 0x7fa8880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a14f8640 1 -- 172.21.15.103:0/2578566540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deaeab40 msgr2=0x5638deae9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a14f8640 1 --2- 172.21.15.103:0/2578566540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deaeab40 0x5638deae9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a14f8640 1 -- 172.21.15.103:0/2578566540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaf4370 msgr2=0x5638deaf6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a14f8640 1 --2- 172.21.15.103:0/2578566540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaf4370 0x5638deaf6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a14f8640 1 -- 172.21.15.103:0/2578566540 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638de9db3a0 con 0x5638deae95d0 2024-09-19T06:59:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a0cf7640 1 --2- 172.21.15.103:0/429343463 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa8880a33b0 0x7fa8880a5870 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa88c0ccf50 tx=0x7fa88c086040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.204+0000 7fa8a14f8640 1 --2- 172.21.15.103:0/2578566540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deae95d0 0x5638deae9a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa898005940 tx=0x7fa898004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6017a90962582269 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.205+0000 7fa8877fe640 1 -- 172.21.15.103:0/2578566540 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898035030 con 0x5638deae95d0 2024-09-19T06:59:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.205+0000 7fa8877fe640 1 -- 172.21.15.103:0/2578566540 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa898040070 con 0x5638deae95d0 2024-09-19T06:59:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.205+0000 7fa8877fe640 1 -- 172.21.15.103:0/2578566540 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898031a40 con 0x5638deae95d0 2024-09-19T06:59:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.205+0000 7fa8a5b35740 1 -- 172.21.15.103:0/2578566540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deae95d0 msgr2=0x5638deae9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:13.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.205+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/2578566540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deae95d0 0x5638deae9a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa898005940 tx=0x7fa898004170 comp rx=0 tx=0).stop 2024-09-19T06:59:13.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.205+0000 7fa8a5b35740 1 -- 172.21.15.103:0/2578566540 shutdown_connections 2024-09-19T06:59:13.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.205+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/2578566540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaf4370 0x5638deaf6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.205+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/2578566540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deae95d0 0x5638deae9a10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.205+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/2578566540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deaeab40 0x5638deae9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.205+0000 7fa8a5b35740 1 -- 172.21.15.103:0/2578566540 >> 172.21.15.103:0/2578566540 conn(0x5638deab2310 msgr2=0x5638deae5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:13.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.205+0000 7fa8a5b35740 1 -- 172.21.15.103:0/2578566540 shutdown_connections 2024-09-19T06:59:13.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.205+0000 7fa8a5b35740 1 -- 172.21.15.103:0/2578566540 wait complete. 2024-09-19T06:59:13.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.206+0000 7fa8a5b35740 1 -- start start 2024-09-19T06:59:13.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.206+0000 7fa8a5b35740 1 Processor -- start 2024-09-19T06:59:13.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.206+0000 7fa8a5b35740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deae95d0 0x5638deaf3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.206+0000 7fa8a5b35740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaeab40 0x5638deaf1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.206+0000 7fa8a5b35740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deaf4370 0x5638deaf1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.206+0000 7fa8a5b35740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5638deafb420 con 0x5638deaeab40 2024-09-19T06:59:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.206+0000 7fa8a5b35740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5638dea0a060 con 0x5638deae95d0 2024-09-19T06:59:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.206+0000 7fa8a5b35740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5638deafb5a0 con 0x5638deaf4370 2024-09-19T06:59:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.206+0000 7fa8a5b35740 10 client.0 _ll_register_callbacks cb 0x5638de75d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.206+0000 7fa8a14f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaeab40 0x5638deaf1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.206+0000 7fa8a14f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaeab40 0x5638deaf1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60712/0 (socket says 172.21.15.103:60712) 2024-09-19T06:59:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.206+0000 7fa8a14f8640 1 -- 172.21.15.103:0/1603786735 learned_addr learned my addr 172.21.15.103:0/1603786735 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.207+0000 7fa8a1cf9640 1 --2- 172.21.15.103:0/1603786735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deaf4370 0x5638deaf1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.207+0000 7fa8a1cf9640 1 --2- 172.21.15.103:0/1603786735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deae95d0 0x5638deaf3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.207+0000 7fa8a14f8640 1 -- 172.21.15.103:0/1603786735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deaf4370 msgr2=0x5638deaf1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.207+0000 7fa8a14f8640 1 --2- 172.21.15.103:0/1603786735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deaf4370 0x5638deaf1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.207+0000 7fa8a14f8640 1 -- 172.21.15.103:0/1603786735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deae95d0 msgr2=0x5638deaf3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.207+0000 7fa8a14f8640 1 --2- 172.21.15.103:0/1603786735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deae95d0 0x5638deaf3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.207+0000 7fa8a14f8640 1 -- 172.21.15.103:0/1603786735 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638de7c03c0 con 0x5638deaeab40 2024-09-19T06:59:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.207+0000 7fa8a14f8640 1 --2- 172.21.15.103:0/1603786735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaeab40 0x5638deaf1020 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fa898006010 tx=0x7fa89802ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bf2e4e7df6dc55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.207+0000 7fa884ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.207+0000 7fa884ff9640 1 -- 172.21.15.103:0/1603786735 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89804d070 con 0x5638deaeab40 2024-09-19T06:59:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.207+0000 7fa884ff9640 1 -- 172.21.15.103:0/1603786735 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7fa898032e00 con 0x5638deaeab40 2024-09-19T06:59:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.207+0000 7fa8a5b35740 1 -- 172.21.15.103:0/1603786735 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa898040070 con 0x5638deaeab40 2024-09-19T06:59:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.207+0000 7fa884ff9640 1 -- 172.21.15.103:0/1603786735 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa898058050 con 0x5638deaeab40 2024-09-19T06:59:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.207+0000 7fa8a5b35740 10 client.18864 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.207+0000 7fa8a5b35740 1 -- 172.21.15.103:0/1603786735 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5638deb06ec0 con 0x5638deaeab40 2024-09-19T06:59:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.208+0000 7fa884ff9640 1 -- 172.21.15.103:0/1603786735 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa898040070 con 0x5638deaeab40 2024-09-19T06:59:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.208+0000 7fa884ff9640 1 -- 172.21.15.103:0/1603786735 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa8980035f0 con 0x5638deaeab40 2024-09-19T06:59:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.208+0000 7fa8a5b35740 10 client.18864 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.208+0000 7fa8a5b35740 10 client.18864 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:59:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.208+0000 7fa8a5b35740 20 client.18864 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.208+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/1603786735 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5638deb08210 0x5638deb0a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.208+0000 7fa8a5b35740 4 client.18864 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.208+0000 7fa8a5b35740 1 -- 172.21.15.103:0/1603786735 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638dea5c860 con 0x5638deb08210 2024-09-19T06:59:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.208+0000 7fa8a1cf9640 1 --2- 172.21.15.103:0/1603786735 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5638deb08210 0x5638deb0a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.209+0000 7fa8a1cf9640 1 --2- 172.21.15.103:0/1603786735 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5638deb08210 0x5638deb0a600 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bfbcc55c0592341b server_cookie=d0499e226136c39d in_seq=0 out_seq=0 2024-09-19T06:59:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.209+0000 7fa884ff9640 10 client.18864 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:59:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.209+0000 7fa884ff9640 1 -- 172.21.15.103:0/1603786735 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638dea5c860 con 0x5638deb08210 2024-09-19T06:59:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.209+0000 7fa884ff9640 10 client.18864 handle_command_reply: tid=0 2024-09-19T06:59:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.209+0000 7fa8a5b35740 1 client.18864 shutdown 2024-09-19T06:59:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.209+0000 7fa8a5b35740 1 -- 172.21.15.103:0/1603786735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaeab40 msgr2=0x5638deaf1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.209+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/1603786735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaeab40 0x5638deaf1020 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fa898006010 tx=0x7fa89802ab80 comp rx=0 tx=0).stop 2024-09-19T06:59:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.209+0000 7fa8a5b35740 1 -- 172.21.15.103:0/1603786735 shutdown_connections 2024-09-19T06:59:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.209+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/1603786735 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5638deb08210 0x5638deb0a600 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.209+0000 7fa884ff9640 0 client.18864 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:59:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.209+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/1603786735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deaf4370 0x5638deaf1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.209+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/1603786735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaeab40 0x5638deaf1020 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.209+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/1603786735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deae95d0 0x5638deaf3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.209+0000 7fa8a5b35740 1 -- 172.21.15.103:0/1603786735 >> 172.21.15.103:0/1603786735 conn(0x5638deab2310 msgr2=0x5638deaeb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.210+0000 7fa8a5b35740 1 -- 172.21.15.103:0/1603786735 shutdown_connections 2024-09-19T06:59:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.210+0000 7fa8a5b35740 1 -- 172.21.15.103:0/1603786735 wait complete. 2024-09-19T06:59:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.210+0000 7fa8a5b35740 20 client.18864 trim_cache size 0 max 16384 2024-09-19T06:59:13.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.222+0000 7fa8a5b35740 1 Processor -- start 2024-09-19T06:59:13.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.222+0000 7fa8a5b35740 1 -- start start 2024-09-19T06:59:13.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a5b35740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deae95d0 0x5638deaf08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a5b35740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaeab40 0x5638deaec120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a5b35740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deaf4370 0x5638deaec660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a5b35740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1ab5a0 con 0x5638deaeab40 2024-09-19T06:59:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a5b35740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1ad0e0 con 0x5638deae95d0 2024-09-19T06:59:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a5b35740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1abc70 con 0x5638deaf4370 2024-09-19T06:59:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a14f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaeab40 0x5638deaec120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a14f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaeab40 0x5638deaec120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60720/0 (socket says 172.21.15.103:60720) 2024-09-19T06:59:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a14f8640 1 -- 172.21.15.103:0/2261989333 learned_addr learned my addr 172.21.15.103:0/2261989333 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a1cf9640 1 --2- 172.21.15.103:0/2261989333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deae95d0 0x5638deaf08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a14f8640 1 -- 172.21.15.103:0/2261989333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deaf4370 msgr2=0x5638deaec660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a14f8640 1 --2- 172.21.15.103:0/2261989333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deaf4370 0x5638deaec660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a14f8640 1 -- 172.21.15.103:0/2261989333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deae95d0 msgr2=0x5638deaf08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a14f8640 1 --2- 172.21.15.103:0/2261989333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deae95d0 0x5638deaf08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a14f8640 1 -- 172.21.15.103:0/2261989333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638deae92d0 con 0x5638deaeab40 2024-09-19T06:59:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a1cf9640 1 --2- 172.21.15.103:0/2261989333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deae95d0 0x5638deaf08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa8a14f8640 1 --2- 172.21.15.103:0/2261989333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaeab40 0x5638deaec120 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fa898003a90 tx=0x7fa8980095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ac88f143f694f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa886ffd640 1 -- 172.21.15.103:0/2261989333 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89804d030 con 0x5638deaeab40 2024-09-19T06:59:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa886ffd640 1 -- 172.21.15.103:0/2261989333 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa898040040 con 0x5638deaeab40 2024-09-19T06:59:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.223+0000 7fa886ffd640 1 -- 172.21.15.103:0/2261989333 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898035050 con 0x5638deaeab40 2024-09-19T06:59:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.224+0000 7fa8a5b35740 1 -- 172.21.15.103:0/2261989333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaeab40 msgr2=0x5638deaec120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.224+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/2261989333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaeab40 0x5638deaec120 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fa898003a90 tx=0x7fa8980095a0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.224+0000 7fa8a5b35740 1 -- 172.21.15.103:0/2261989333 shutdown_connections 2024-09-19T06:59:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.224+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/2261989333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deaf4370 0x5638deaec660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.224+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/2261989333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaeab40 0x5638deaec120 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.224+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/2261989333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deae95d0 0x5638deaf08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.224+0000 7fa8a5b35740 1 -- 172.21.15.103:0/2261989333 >> 172.21.15.103:0/2261989333 conn(0x5638deab2310 msgr2=0x5638deaf30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.224+0000 7fa8a5b35740 1 -- 172.21.15.103:0/2261989333 shutdown_connections 2024-09-19T06:59:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.224+0000 7fa8a5b35740 1 -- 172.21.15.103:0/2261989333 wait complete. 2024-09-19T06:59:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.224+0000 7fa8a5b35740 1 -- start start 2024-09-19T06:59:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.224+0000 7fa8a5b35740 1 Processor -- start 2024-09-19T06:59:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a5b35740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deae95d0 0x5638deb0aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a5b35740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deaeab40 0x5638deb0b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a5b35740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaf4370 0x5638deb70b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a5b35740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1aca10 con 0x5638deaf4370 2024-09-19T06:59:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a5b35740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1ac340 con 0x5638deae95d0 2024-09-19T06:59:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a5b35740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1a95a0 con 0x5638deaeab40 2024-09-19T06:59:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a5b35740 10 client.0 _ll_register_callbacks cb 0x7fa894007c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a1cf9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaf4370 0x5638deb70b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a1cf9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaf4370 0x5638deb70b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60736/0 (socket says 172.21.15.103:60736) 2024-09-19T06:59:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a1cf9640 1 -- 172.21.15.103:0/1017812878 learned_addr learned my addr 172.21.15.103:0/1017812878 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a14f8640 1 --2- 172.21.15.103:0/1017812878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deaeab40 0x5638deb0b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a1cf9640 1 --2- 172.21.15.103:0/1017812878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deae95d0 0x5638deb0aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a1cf9640 1 -- 172.21.15.103:0/1017812878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deaeab40 msgr2=0x5638deb0b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a1cf9640 1 --2- 172.21.15.103:0/1017812878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deaeab40 0x5638deb0b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a1cf9640 1 -- 172.21.15.103:0/1017812878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deae95d0 msgr2=0x5638deb0aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a1cf9640 1 --2- 172.21.15.103:0/1017812878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deae95d0 0x5638deb0aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a1cf9640 1 -- 172.21.15.103:0/1017812878 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638deb71130 con 0x5638deaf4370 2024-09-19T06:59:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa8a1cf9640 1 --2- 172.21.15.103:0/1017812878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaf4370 0x5638deb70b40 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x5638deafb690 tx=0x7fa894007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ccf01eb81fb986 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa884ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa884ff9640 1 -- 172.21.15.103:0/1017812878 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894026070 con 0x5638deaf4370 2024-09-19T06:59:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.225+0000 7fa884ff9640 1 -- 172.21.15.103:0/1017812878 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7fa894004b20 con 0x5638deaf4370 2024-09-19T06:59:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.226+0000 7fa8a5b35740 1 -- 172.21.15.103:0/1017812878 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa898040040 con 0x5638deaf4370 2024-09-19T06:59:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.226+0000 7fa884ff9640 1 -- 172.21.15.103:0/1017812878 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa894004d20 con 0x5638deaf4370 2024-09-19T06:59:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.226+0000 7fa8a5b35740 10 client.18870 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.226+0000 7fa8a5b35740 1 -- 172.21.15.103:0/1017812878 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5638deb72450 con 0x5638deaf4370 2024-09-19T06:59:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.226+0000 7fa884ff9640 1 -- 172.21.15.103:0/1017812878 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa898040040 con 0x5638deaf4370 2024-09-19T06:59:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.226+0000 7fa884ff9640 1 -- 172.21.15.103:0/1017812878 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa894011440 con 0x5638deaf4370 2024-09-19T06:59:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.226+0000 7fa8a5b35740 10 client.18870 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.226+0000 7fa8a5b35740 10 client.18870 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:59:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.226+0000 7fa8a5b35740 20 client.18870 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.226+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/1017812878 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5638deb08210 0x5638deb74b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.226+0000 7fa8a5b35740 4 client.18870 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:59:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.226+0000 7fa8a5b35740 1 -- 172.21.15.103:0/1017812878 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638dea56d70 con 0x5638deb08210 2024-09-19T06:59:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.226+0000 7fa8a14f8640 1 --2- 172.21.15.103:0/1017812878 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5638deb08210 0x5638deb74b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.227+0000 7fa884ff9640 10 client.18870 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:59:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.227+0000 7fa8a14f8640 1 --2- 172.21.15.103:0/1017812878 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5638deb08210 0x5638deb74b30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b7e1885b8ffb06c1 server_cookie=48329e06c5bd9d in_seq=0 out_seq=0 2024-09-19T06:59:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.227+0000 7fa884ff9640 1 -- 172.21.15.103:0/1017812878 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638dea56d70 con 0x5638deb08210 2024-09-19T06:59:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.227+0000 7fa884ff9640 10 client.18870 handle_command_reply: tid=0 2024-09-19T06:59:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.227+0000 7fa8a5b35740 1 client.18870 shutdown 2024-09-19T06:59:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.227+0000 7fa8a5b35740 1 -- 172.21.15.103:0/1017812878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaf4370 msgr2=0x5638deb70b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.227+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/1017812878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaf4370 0x5638deb70b40 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x5638deafb690 tx=0x7fa894007db0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.227+0000 7fa8a5b35740 1 -- 172.21.15.103:0/1017812878 shutdown_connections 2024-09-19T06:59:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.227+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/1017812878 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5638deb08210 0x5638deb74b30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.227+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/1017812878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638deaf4370 0x5638deb70b40 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.227+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/1017812878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638deaeab40 0x5638deb0b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.227+0000 7fa8a5b35740 1 --2- 172.21.15.103:0/1017812878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638deae95d0 0x5638deb0aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.227+0000 7fa8a5b35740 1 -- 172.21.15.103:0/1017812878 >> 172.21.15.103:0/1017812878 conn(0x5638deab2310 msgr2=0x5638deaf30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.227+0000 7fa884ff9640 0 client.18870 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:59:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.227+0000 7fa8a5b35740 1 -- 172.21.15.103:0/1017812878 shutdown_connections 2024-09-19T06:59:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.227+0000 7fa8a5b35740 1 -- 172.21.15.103:0/1017812878 wait complete. 2024-09-19T06:59:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.228+0000 7fa8a5b35740 20 client.18870 trim_cache size 0 max 16384 2024-09-19T06:59:13.235 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:59:13.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.228+0000 7fa8a375c640 1 -- 172.21.15.103:0/429343463 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa8880a33b0 msgr2=0x7fa8880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:13.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.228+0000 7fa8a375c640 1 --2- 172.21.15.103:0/429343463 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa8880a33b0 0x7fa8880a5870 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa88c0ccf50 tx=0x7fa88c086040 comp rx=0 tx=0).stop 2024-09-19T06:59:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.228+0000 7fa8a375c640 1 -- 172.21.15.103:0/429343463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa89c102570 msgr2=0x7fa89c198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.228+0000 7fa8a375c640 1 --2- 172.21.15.103:0/429343463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa89c102570 0x7fa89c198a00 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00d770 tx=0x7fa88c00dc30 comp rx=0 tx=0).stop 2024-09-19T06:59:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.228+0000 7fa8a375c640 1 -- 172.21.15.103:0/429343463 shutdown_connections 2024-09-19T06:59:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.228+0000 7fa8a375c640 1 --2- 172.21.15.103:0/429343463 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa8880a33b0 0x7fa8880a5870 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.228+0000 7fa8a375c640 1 --2- 172.21.15.103:0/429343463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa89c102ef0 0x7fa89c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.228+0000 7fa8a375c640 1 --2- 172.21.15.103:0/429343463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa89c102570 0x7fa89c198a00 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.228+0000 7fa8a375c640 1 --2- 172.21.15.103:0/429343463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa89c100e20 0x7fa89c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.228+0000 7fa8a375c640 1 -- 172.21.15.103:0/429343463 >> 172.21.15.103:0/429343463 conn(0x7fa89c0fc7d0 msgr2=0x7fa89c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.228+0000 7fa8a375c640 1 -- 172.21.15.103:0/429343463 shutdown_connections 2024-09-19T06:59:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:13.228+0000 7fa8a375c640 1 -- 172.21.15.103:0/429343463 wait complete. 2024-09-19T06:59:13.881 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:13 smithi103 ceph-mon[31158]: pgmap v310: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 130 KiB/s wr, 49 op/s 2024-09-19T06:59:13.911 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:59:13.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:13 smithi134 ceph-mon[35356]: pgmap v310: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 130 KiB/s wr, 49 op/s 2024-09-19T06:59:14.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:13 smithi174 ceph-mon[35637]: pgmap v310: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 130 KiB/s wr, 49 op/s 2024-09-19T06:59:14.162 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:14.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.547+0000 7f3cce5cd640 1 -- 172.21.15.103:0/2294000396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc8103660 msgr2=0x7f3cc810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:14.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.547+0000 7f3cce5cd640 1 --2- 172.21.15.103:0/2294000396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc8103660 0x7f3cc810bb50 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009900 tx=0x7f3cbc02a480 comp rx=0 tx=0).stop 2024-09-19T06:59:14.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.548+0000 7f3cce5cd640 1 -- 172.21.15.103:0/2294000396 shutdown_connections 2024-09-19T06:59:14.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.548+0000 7f3cce5cd640 1 --2- 172.21.15.103:0/2294000396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cc810c170 0x7f3cc810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.548+0000 7f3cce5cd640 1 --2- 172.21.15.103:0/2294000396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc8103660 0x7f3cc810bb50 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.548+0000 7f3cce5cd640 1 --2- 172.21.15.103:0/2294000396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc8102d40 0x7f3cc8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.548+0000 7f3cce5cd640 1 -- 172.21.15.103:0/2294000396 >> 172.21.15.103:0/2294000396 conn(0x7f3cc80fc7a0 msgr2=0x7f3cc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:14.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.548+0000 7f3cce5cd640 1 -- 172.21.15.103:0/2294000396 shutdown_connections 2024-09-19T06:59:14.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.548+0000 7f3cce5cd640 1 -- 172.21.15.103:0/2294000396 wait complete. 2024-09-19T06:59:14.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.548+0000 7f3cce5cd640 1 Processor -- start 2024-09-19T06:59:14.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.548+0000 7f3cce5cd640 1 -- start start 2024-09-19T06:59:14.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cce5cd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cc8102d40 0x7f3cc81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cce5cd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc8103660 0x7f3cc8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cce5cd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc810c170 0x7f3cc8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cce5cd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc8111f70 con 0x7f3cc8103660 2024-09-19T06:59:14.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cce5cd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc8111df0 con 0x7f3cc8102d40 2024-09-19T06:59:14.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cce5cd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc81120f0 con 0x7f3cc810c170 2024-09-19T06:59:14.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cc77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc8103660 0x7f3cc8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cc77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc8103660 0x7f3cc8100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60758/0 (socket says 172.21.15.103:60758) 2024-09-19T06:59:14.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cc77fe640 1 -- 172.21.15.103:0/3868652879 learned_addr learned my addr 172.21.15.103:0/3868652879 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:14.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cc7fff640 1 --2- 172.21.15.103:0/3868652879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cc8102d40 0x7f3cc81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cccb6a640 1 --2- 172.21.15.103:0/3868652879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc810c170 0x7f3cc8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cc77fe640 1 -- 172.21.15.103:0/3868652879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc810c170 msgr2=0x7f3cc8101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:14.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cc77fe640 1 --2- 172.21.15.103:0/3868652879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc810c170 0x7f3cc8101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cc77fe640 1 -- 172.21.15.103:0/3868652879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cc8102d40 msgr2=0x7f3cc81009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:14.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cc77fe640 1 --2- 172.21.15.103:0/3868652879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cc8102d40 0x7f3cc81009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cc77fe640 1 -- 172.21.15.103:0/3868652879 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc81a69c0 con 0x7f3cc8103660 2024-09-19T06:59:14.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cc77fe640 1 --2- 172.21.15.103:0/3868652879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc8103660 0x7f3cc8100ee0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc006010 tx=0x7f3cbc031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:14.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.549+0000 7f3cc57fa640 1 -- 172.21.15.103:0/3868652879 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc03f030 con 0x7f3cc8103660 2024-09-19T06:59:14.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.550+0000 7f3cce5cd640 1 -- 172.21.15.103:0/3868652879 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc81a6c50 con 0x7f3cc8103660 2024-09-19T06:59:14.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.550+0000 7f3cc57fa640 1 -- 172.21.15.103:0/3868652879 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cbc0043d0 con 0x7f3cc8103660 2024-09-19T06:59:14.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.550+0000 7f3cc57fa640 1 -- 172.21.15.103:0/3868652879 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc035070 con 0x7f3cc8103660 2024-09-19T06:59:14.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.550+0000 7f3cce5cd640 1 -- 172.21.15.103:0/3868652879 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc81a7180 con 0x7f3cc8103660 2024-09-19T06:59:14.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.552+0000 7f3cc57fa640 1 -- 172.21.15.103:0/3868652879 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3cbc042030 con 0x7f3cc8103660 2024-09-19T06:59:14.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.552+0000 7f3cd09a6740 1 Processor -- start 2024-09-19T06:59:14.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.552+0000 7f3cd09a6740 1 -- start start 2024-09-19T06:59:14.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cd09a6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96d6b40 0x5595e96d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cd09a6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d55d0 0x5595e96d5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cd09a6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96e0370 0x5595e96e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cd09a6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5595e963b2f0 con 0x5595e96d55d0 2024-09-19T06:59:14.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cd09a6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5595e95e2cf0 con 0x5595e96d6b40 2024-09-19T06:59:14.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cd09a6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5595e96372b0 con 0x5595e96e0370 2024-09-19T06:59:14.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cc57fa640 1 --2- 172.21.15.103:0/3868652879 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cb00a3430 0x7f3cb00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cc7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d55d0 0x5595e96d5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cc7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d55d0 0x5595e96d5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60770/0 (socket says 172.21.15.103:60770) 2024-09-19T06:59:14.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cc7fff640 1 -- 172.21.15.103:0/227529090 learned_addr learned my addr 172.21.15.103:0/227529090 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:14.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cccb6a640 1 --2- 172.21.15.103:0/227529090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96d6b40 0x5595e96d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cccb6a640 1 --2- 172.21.15.103:0/3868652879 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cb00a3430 0x7f3cb00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cc77fe640 1 --2- 172.21.15.103:0/227529090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96e0370 0x5595e96e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cc57fa640 1 -- 172.21.15.103:0/3868652879 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f3cbc0f0040 con 0x7f3cc8103660 2024-09-19T06:59:14.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cc7fff640 1 -- 172.21.15.103:0/227529090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96e0370 msgr2=0x5595e96e2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:14.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cc7fff640 1 --2- 172.21.15.103:0/227529090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96e0370 0x5595e96e2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cc7fff640 1 -- 172.21.15.103:0/227529090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96d6b40 msgr2=0x5595e96d5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:14.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cc7fff640 1 --2- 172.21.15.103:0/227529090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96d6b40 0x5595e96d5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cc7fff640 1 -- 172.21.15.103:0/227529090 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e95c73a0 con 0x5595e96d55d0 2024-09-19T06:59:14.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cc7fff640 1 --2- 172.21.15.103:0/227529090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d55d0 0x5595e96d5a10 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009d20 tx=0x7f3cb8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337e37f9792192bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:14.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cccb6a640 1 --2- 172.21.15.103:0/3868652879 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cb00a3430 0x7f3cb00a58f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0005fd0 tx=0x7f3cc0009500 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:14.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.553+0000 7f3cae7fc640 1 -- 172.21.15.103:0/227529090 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb801e070 con 0x5595e96d55d0 2024-09-19T06:59:14.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.554+0000 7f3cae7fc640 1 -- 172.21.15.103:0/227529090 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb8019b10 con 0x5595e96d55d0 2024-09-19T06:59:14.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.554+0000 7f3cae7fc640 1 -- 172.21.15.103:0/227529090 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb800ebb0 con 0x5595e96d55d0 2024-09-19T06:59:14.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.554+0000 7f3cd09a6740 1 -- 172.21.15.103:0/227529090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d55d0 msgr2=0x5595e96d5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:14.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.554+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/227529090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d55d0 0x5595e96d5a10 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009d20 tx=0x7f3cb8011810 comp rx=0 tx=0).stop 2024-09-19T06:59:14.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.554+0000 7f3cd09a6740 1 -- 172.21.15.103:0/227529090 shutdown_connections 2024-09-19T06:59:14.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.554+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/227529090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96e0370 0x5595e96e2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.554+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/227529090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d55d0 0x5595e96d5a10 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.554+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/227529090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96d6b40 0x5595e96d5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.554+0000 7f3cd09a6740 1 -- 172.21.15.103:0/227529090 >> 172.21.15.103:0/227529090 conn(0x5595e969e310 msgr2=0x5595e96d1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:14.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.554+0000 7f3cd09a6740 1 -- 172.21.15.103:0/227529090 shutdown_connections 2024-09-19T06:59:14.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.554+0000 7f3cd09a6740 1 -- 172.21.15.103:0/227529090 wait complete. 2024-09-19T06:59:14.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.555+0000 7f3cd09a6740 1 -- start start 2024-09-19T06:59:14.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.555+0000 7f3cd09a6740 1 Processor -- start 2024-09-19T06:59:14.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.555+0000 7f3cd09a6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96d55d0 0x5595e96dfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.555+0000 7f3cd09a6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96dcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.555+0000 7f3cd09a6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96e0370 0x5595e96dd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.555+0000 7f3cd09a6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5595e96e3070 con 0x5595e96d6b40 2024-09-19T06:59:14.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.555+0000 7f3cd09a6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5595e95f6060 con 0x5595e96d55d0 2024-09-19T06:59:14.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.555+0000 7f3cd09a6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5595e96e31f0 con 0x5595e96e0370 2024-09-19T06:59:14.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.555+0000 7f3cd09a6740 10 client.0 _ll_register_callbacks cb 0x5595e93498c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:14.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.555+0000 7f3cc77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96dcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.555+0000 7f3cc77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96dcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60778/0 (socket says 172.21.15.103:60778) 2024-09-19T06:59:14.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.555+0000 7f3cc77fe640 1 -- 172.21.15.103:0/1197747876 learned_addr learned my addr 172.21.15.103:0/1197747876 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:14.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.555+0000 7f3cccb6a640 1 --2- 172.21.15.103:0/1197747876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96e0370 0x5595e96dd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3cc77fe640 1 -- 172.21.15.103:0/1197747876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96e0370 msgr2=0x5595e96dd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:14.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3cc7fff640 1 --2- 172.21.15.103:0/1197747876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96d55d0 0x5595e96dfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3cc77fe640 1 --2- 172.21.15.103:0/1197747876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96e0370 0x5595e96dd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3cc77fe640 1 -- 172.21.15.103:0/1197747876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96d55d0 msgr2=0x5595e96dfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:14.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3cc77fe640 1 --2- 172.21.15.103:0/1197747876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96d55d0 0x5595e96dfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3cc77fe640 1 -- 172.21.15.103:0/1197747876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e93ac3c0 con 0x5595e96d6b40 2024-09-19T06:59:14.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3cccb6a640 1 --2- 172.21.15.103:0/1197747876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96e0370 0x5595e96dd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:14.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3cc77fe640 1 --2- 172.21.15.103:0/1197747876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96dcff0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0ed820 tx=0x7f3cbc0bc190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc7ea2e9ca23e70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:14.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3ca77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:14.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3ca77fe640 1 -- 172.21.15.103:0/1197747876 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc049040 con 0x5595e96d6b40 2024-09-19T06:59:14.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3ca77fe640 1 -- 172.21.15.103:0/1197747876 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f3cbc003c60 con 0x5595e96d6b40 2024-09-19T06:59:14.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3cd09a6740 1 -- 172.21.15.103:0/1197747876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cb8019b10 con 0x5595e96d6b40 2024-09-19T06:59:14.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3ca77fe640 1 -- 172.21.15.103:0/1197747876 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3cbc0bc300 con 0x5595e96d6b40 2024-09-19T06:59:14.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3cd09a6740 10 client.18882 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:14.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3cd09a6740 1 -- 172.21.15.103:0/1197747876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5595e96eeae0 con 0x5595e96d6b40 2024-09-19T06:59:14.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3ca77fe640 1 -- 172.21.15.103:0/1197747876 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb8019b10 con 0x5595e96d6b40 2024-09-19T06:59:14.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3ca77fe640 1 -- 172.21.15.103:0/1197747876 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3cbc0d5120 con 0x5595e96d6b40 2024-09-19T06:59:14.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3cd09a6740 10 client.18882 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:14.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3cd09a6740 10 client.18882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:14.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.556+0000 7f3cd09a6740 20 client.18882 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:14.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.557+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/1197747876 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5595e96efef0 0x5595e96f2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.557+0000 7f3cd09a6740 4 client.18882 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:14.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.557+0000 7f3cd09a6740 1 -- 172.21.15.103:0/1197747876 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595e9648860 con 0x5595e96efef0 2024-09-19T06:59:14.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.557+0000 7f3cc7fff640 1 --2- 172.21.15.103:0/1197747876 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5595e96efef0 0x5595e96f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.557+0000 7f3ca77fe640 10 client.18882 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:14.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.557+0000 7f3cc7fff640 1 --2- 172.21.15.103:0/1197747876 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5595e96efef0 0x5595e96f2300 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e93cf4ed9cfaa71 server_cookie=9ebc660e4ab15a0f in_seq=0 out_seq=0 2024-09-19T06:59:14.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.558+0000 7f3ca77fe640 1 -- 172.21.15.103:0/1197747876 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595e9648860 con 0x5595e96efef0 2024-09-19T06:59:14.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.558+0000 7f3ca77fe640 10 client.18882 handle_command_reply: tid=0 2024-09-19T06:59:14.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.558+0000 7f3cd09a6740 1 client.18882 shutdown 2024-09-19T06:59:14.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.558+0000 7f3cd09a6740 1 -- 172.21.15.103:0/1197747876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 msgr2=0x5595e96dcff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:14.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.558+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/1197747876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96dcff0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0ed820 tx=0x7f3cbc0bc190 comp rx=0 tx=0).stop 2024-09-19T06:59:14.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.558+0000 7f3cd09a6740 1 -- 172.21.15.103:0/1197747876 shutdown_connections 2024-09-19T06:59:14.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.558+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/1197747876 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5595e96efef0 0x5595e96f2300 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.558+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/1197747876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96e0370 0x5595e96dd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.558+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/1197747876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96dcff0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.558+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/1197747876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96d55d0 0x5595e96dfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.558+0000 7f3cd09a6740 1 -- 172.21.15.103:0/1197747876 >> 172.21.15.103:0/1197747876 conn(0x5595e969e310 msgr2=0x5595e96d7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:14.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.558+0000 7f3ca77fe640 0 client.18882 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:14.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.558+0000 7f3cd09a6740 1 -- 172.21.15.103:0/1197747876 shutdown_connections 2024-09-19T06:59:14.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.558+0000 7f3cd09a6740 1 -- 172.21.15.103:0/1197747876 wait complete. 2024-09-19T06:59:14.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.558+0000 7f3cd09a6740 20 client.18882 trim_cache size 0 max 16384 2024-09-19T06:59:14.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.571+0000 7f3cd09a6740 1 Processor -- start 2024-09-19T06:59:14.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.571+0000 7f3cd09a6740 1 -- start start 2024-09-19T06:59:14.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.571+0000 7f3cd09a6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96d55d0 0x5595e96d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.571+0000 7f3cd09a6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96ec310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.571+0000 7f3cd09a6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96e0370 0x5595e96dc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.571+0000 7f3cd09a6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc81af9e0 con 0x5595e96d6b40 2024-09-19T06:59:14.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.571+0000 7f3cd09a6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc81b1520 con 0x5595e96e0370 2024-09-19T06:59:14.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.571+0000 7f3cd09a6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc81b00b0 con 0x5595e96d55d0 2024-09-19T06:59:14.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.571+0000 7f3cc77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96ec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.571+0000 7f3cc77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96ec310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60782/0 (socket says 172.21.15.103:60782) 2024-09-19T06:59:14.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.571+0000 7f3cc77fe640 1 -- 172.21.15.103:0/3096109140 learned_addr learned my addr 172.21.15.103:0/3096109140 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:14.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.571+0000 7f3cc7fff640 1 --2- 172.21.15.103:0/3096109140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96d55d0 0x5595e96d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.571+0000 7f3cc77fe640 1 -- 172.21.15.103:0/3096109140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96d55d0 msgr2=0x5595e96d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:14.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.571+0000 7f3cc77fe640 1 --2- 172.21.15.103:0/3096109140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96d55d0 0x5595e96d1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.572+0000 7f3cc77fe640 1 -- 172.21.15.103:0/3096109140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96e0370 msgr2=0x5595e96dc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:14.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.572+0000 7f3cc77fe640 1 --2- 172.21.15.103:0/3096109140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96e0370 0x5595e96dc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.572+0000 7f3cc77fe640 1 -- 172.21.15.103:0/3096109140 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e96d6760 con 0x5595e96d6b40 2024-09-19T06:59:14.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.572+0000 7f3cc7fff640 1 --2- 172.21.15.103:0/3096109140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96d55d0 0x5595e96d1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:14.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.572+0000 7f3cc77fe640 1 --2- 172.21.15.103:0/3096109140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96ec310 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0edd70 tx=0x7f3cbc0bc200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4a1d801085c71d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:14.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.572+0000 7f3cadffb640 1 -- 172.21.15.103:0/3096109140 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc0c4f70 con 0x5595e96d6b40 2024-09-19T06:59:14.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.572+0000 7f3cadffb640 1 -- 172.21.15.103:0/3096109140 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cbc0c06b0 con 0x5595e96d6b40 2024-09-19T06:59:14.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.572+0000 7f3cadffb640 1 -- 172.21.15.103:0/3096109140 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc0bb750 con 0x5595e96d6b40 2024-09-19T06:59:14.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.572+0000 7f3cd09a6740 1 -- 172.21.15.103:0/3096109140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 msgr2=0x5595e96ec310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:14.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.572+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/3096109140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96ec310 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0edd70 tx=0x7f3cbc0bc200 comp rx=0 tx=0).stop 2024-09-19T06:59:14.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.572+0000 7f3cd09a6740 1 -- 172.21.15.103:0/3096109140 shutdown_connections 2024-09-19T06:59:14.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.572+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/3096109140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96e0370 0x5595e96dc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.572+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/3096109140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96ec310 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.572+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/3096109140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96d55d0 0x5595e96d1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.573+0000 7f3cd09a6740 1 -- 172.21.15.103:0/3096109140 >> 172.21.15.103:0/3096109140 conn(0x5595e969e310 msgr2=0x5595e96d1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:14.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.573+0000 7f3cd09a6740 1 -- 172.21.15.103:0/3096109140 shutdown_connections 2024-09-19T06:59:14.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.573+0000 7f3cd09a6740 1 -- 172.21.15.103:0/3096109140 wait complete. 2024-09-19T06:59:14.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.573+0000 7f3cd09a6740 1 -- start start 2024-09-19T06:59:14.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.573+0000 7f3cd09a6740 1 Processor -- start 2024-09-19T06:59:14.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.573+0000 7f3cd09a6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96d55d0 0x5595e96e8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.573+0000 7f3cd09a6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96e89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.573+0000 7f3cd09a6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96e0370 0x5595e96d93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.573+0000 7f3cd09a6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc81b0e50 con 0x5595e96d6b40 2024-09-19T06:59:14.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.573+0000 7f3cd09a6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc81b0780 con 0x5595e96e0370 2024-09-19T06:59:14.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.573+0000 7f3cd09a6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc81ad9e0 con 0x5595e96d55d0 2024-09-19T06:59:14.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3cd09a6740 10 client.0 _ll_register_callbacks cb 0x7f3cbc005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:14.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3cc77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96e89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3cc77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96e89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60794/0 (socket says 172.21.15.103:60794) 2024-09-19T06:59:14.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3cc77fe640 1 -- 172.21.15.103:0/2032823928 learned_addr learned my addr 172.21.15.103:0/2032823928 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:14.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3cc7fff640 1 --2- 172.21.15.103:0/2032823928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96d55d0 0x5595e96e8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3cccb6a640 1 --2- 172.21.15.103:0/2032823928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96e0370 0x5595e96d93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3cc77fe640 1 -- 172.21.15.103:0/2032823928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96d55d0 msgr2=0x5595e96e8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:14.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3cc77fe640 1 --2- 172.21.15.103:0/2032823928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96d55d0 0x5595e96e8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3cc77fe640 1 -- 172.21.15.103:0/2032823928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96e0370 msgr2=0x5595e96d93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:14.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3cc77fe640 1 --2- 172.21.15.103:0/2032823928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96e0370 0x5595e96d93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3cc77fe640 1 -- 172.21.15.103:0/2032823928 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e96d98e0 con 0x5595e96d6b40 2024-09-19T06:59:14.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3cccb6a640 1 --2- 172.21.15.103:0/2032823928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96e0370 0x5595e96d93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:14.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3cc77fe640 1 --2- 172.21.15.103:0/2032823928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96e89c0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0bbfb0 tx=0x7f3cbc02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89084a085a49d66a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:14.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3ca77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:14.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3ca77fe640 1 -- 172.21.15.103:0/2032823928 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc049070 con 0x5595e96d6b40 2024-09-19T06:59:14.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3ca77fe640 1 -- 172.21.15.103:0/2032823928 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f3cbc0dc0b0 con 0x5595e96d6b40 2024-09-19T06:59:14.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3cd09a6740 1 -- 172.21.15.103:0/2032823928 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cbc0c06b0 con 0x5595e96d6b40 2024-09-19T06:59:14.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3ca77fe640 1 -- 172.21.15.103:0/2032823928 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3cbc0b8970 con 0x5595e96d6b40 2024-09-19T06:59:14.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3cd09a6740 10 client.18888 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:14.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.574+0000 7f3cd09a6740 1 -- 172.21.15.103:0/2032823928 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5595e96d98e0 con 0x5595e96d6b40 2024-09-19T06:59:14.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.575+0000 7f3ca77fe640 1 -- 172.21.15.103:0/2032823928 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cbc0c06b0 con 0x5595e96d6b40 2024-09-19T06:59:14.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.575+0000 7f3ca77fe640 1 -- 172.21.15.103:0/2032823928 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3cbc0d0210 con 0x5595e96d6b40 2024-09-19T06:59:14.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.575+0000 7f3cd09a6740 10 client.18888 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:14.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.575+0000 7f3cd09a6740 10 client.18888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:14.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.575+0000 7f3cd09a6740 20 client.18888 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:14.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.575+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/2032823928 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5595e975a540 0x5595e975c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:14.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.575+0000 7f3cd09a6740 4 client.18888 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:59:14.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.575+0000 7f3cd09a6740 1 -- 172.21.15.103:0/2032823928 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595e9642d70 con 0x5595e975a540 2024-09-19T06:59:14.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.575+0000 7f3cc7fff640 1 --2- 172.21.15.103:0/2032823928 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5595e975a540 0x5595e975c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:14.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.575+0000 7f3ca77fe640 10 client.18888 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:14.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.575+0000 7f3cc7fff640 1 --2- 172.21.15.103:0/2032823928 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5595e975a540 0x5595e975c930 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e43cca75953062b7 server_cookie=d115b17e0a1c2f61 in_seq=0 out_seq=0 2024-09-19T06:59:14.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.576+0000 7f3ca77fe640 1 -- 172.21.15.103:0/2032823928 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595e9642d70 con 0x5595e975a540 2024-09-19T06:59:14.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.576+0000 7f3ca77fe640 10 client.18888 handle_command_reply: tid=0 2024-09-19T06:59:14.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.576+0000 7f3cd09a6740 1 client.18888 shutdown 2024-09-19T06:59:14.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.576+0000 7f3cd09a6740 1 -- 172.21.15.103:0/2032823928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 msgr2=0x5595e96e89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:14.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.576+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/2032823928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96e89c0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0bbfb0 tx=0x7f3cbc02a480 comp rx=0 tx=0).stop 2024-09-19T06:59:14.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.576+0000 7f3cd09a6740 1 -- 172.21.15.103:0/2032823928 shutdown_connections 2024-09-19T06:59:14.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.576+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/2032823928 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5595e975a540 0x5595e975c930 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.576+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/2032823928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595e96e0370 0x5595e96d93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.576+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/2032823928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5595e96d6b40 0x5595e96e89c0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.576+0000 7f3ca77fe640 0 client.18888 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:14.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.576+0000 7f3cd09a6740 1 --2- 172.21.15.103:0/2032823928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e96d55d0 0x5595e96e8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.576+0000 7f3cd09a6740 1 -- 172.21.15.103:0/2032823928 >> 172.21.15.103:0/2032823928 conn(0x5595e969e310 msgr2=0x5595e96d1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:14.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.576+0000 7f3cd09a6740 1 -- 172.21.15.103:0/2032823928 shutdown_connections 2024-09-19T06:59:14.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.576+0000 7f3cd09a6740 1 -- 172.21.15.103:0/2032823928 wait complete. 2024-09-19T06:59:14.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.576+0000 7f3cd09a6740 20 client.18888 trim_cache size 0 max 16384 2024-09-19T06:59:14.584 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:59:14.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.577+0000 7f3cce5cd640 1 -- 172.21.15.103:0/3868652879 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cb00a3430 msgr2=0x7f3cb00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:14.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.577+0000 7f3cce5cd640 1 --2- 172.21.15.103:0/3868652879 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cb00a3430 0x7f3cb00a58f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0005fd0 tx=0x7f3cc0009500 comp rx=0 tx=0).stop 2024-09-19T06:59:14.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.577+0000 7f3cce5cd640 1 -- 172.21.15.103:0/3868652879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc8103660 msgr2=0x7f3cc8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:14.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.577+0000 7f3cce5cd640 1 --2- 172.21.15.103:0/3868652879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc8103660 0x7f3cc8100ee0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc006010 tx=0x7f3cbc031b20 comp rx=0 tx=0).stop 2024-09-19T06:59:14.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.577+0000 7f3cce5cd640 1 -- 172.21.15.103:0/3868652879 shutdown_connections 2024-09-19T06:59:14.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.577+0000 7f3cce5cd640 1 --2- 172.21.15.103:0/3868652879 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3cb00a3430 0x7f3cb00a58f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.577+0000 7f3cce5cd640 1 --2- 172.21.15.103:0/3868652879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc810c170 0x7f3cc8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.577+0000 7f3cce5cd640 1 --2- 172.21.15.103:0/3868652879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc8103660 0x7f3cc8100ee0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.577+0000 7f3cce5cd640 1 --2- 172.21.15.103:0/3868652879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cc8102d40 0x7f3cc81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:14.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.577+0000 7f3cce5cd640 1 -- 172.21.15.103:0/3868652879 >> 172.21.15.103:0/3868652879 conn(0x7f3cc80fc7a0 msgr2=0x7f3cc80fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:14.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.577+0000 7f3cce5cd640 1 -- 172.21.15.103:0/3868652879 shutdown_connections 2024-09-19T06:59:14.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:14.577+0000 7f3cce5cd640 1 -- 172.21.15.103:0/3868652879 wait complete. 2024-09-19T06:59:16.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:15 smithi174 ceph-mon[35637]: pgmap v311: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-09-19T06:59:16.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:15 smithi103 ceph-mon[31158]: pgmap v311: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-09-19T06:59:16.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:15 smithi134 ceph-mon[35356]: pgmap v311: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-09-19T06:59:16.271 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:59:16.272 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:59:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.508+0000 7f0def647640 1 -- 172.21.15.103:0/4200914049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0de81069e0 msgr2=0x7f0de8106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.508+0000 7f0def647640 1 --2- 172.21.15.103:0/4200914049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0de81069e0 0x7f0de8106dc0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009930 tx=0x7f0de402a420 comp rx=0 tx=0).stop 2024-09-19T06:59:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.509+0000 7f0def647640 1 -- 172.21.15.103:0/4200914049 shutdown_connections 2024-09-19T06:59:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.509+0000 7f0def647640 1 --2- 172.21.15.103:0/4200914049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0de8101370 0x7f0de810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.509+0000 7f0def647640 1 --2- 172.21.15.103:0/4200914049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0de81009d0 0x7f0de8100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.509+0000 7f0def647640 1 --2- 172.21.15.103:0/4200914049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0de81069e0 0x7f0de8106dc0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.509+0000 7f0def647640 1 -- 172.21.15.103:0/4200914049 >> 172.21.15.103:0/4200914049 conn(0x7f0de80fc6f0 msgr2=0x7f0de80feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.509+0000 7f0def647640 1 -- 172.21.15.103:0/4200914049 shutdown_connections 2024-09-19T06:59:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.509+0000 7f0def647640 1 -- 172.21.15.103:0/4200914049 wait complete. 2024-09-19T06:59:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.509+0000 7f0def647640 1 Processor -- start 2024-09-19T06:59:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.509+0000 7f0def647640 1 -- start start 2024-09-19T06:59:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.509+0000 7f0def647640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0de81009d0 0x7f0de8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.509+0000 7f0def647640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0de8101370 0x7f0de8198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0def647640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0de81069e0 0x7f0de819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0def647640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0de8111ef0 con 0x7f0de8101370 2024-09-19T06:59:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0def647640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0de8111d70 con 0x7f0de81069e0 2024-09-19T06:59:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0def647640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0de8112070 con 0x7f0de81009d0 2024-09-19T06:59:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0decbe2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0de8101370 0x7f0de8198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0decbe2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0de8101370 0x7f0de8198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60810/0 (socket says 172.21.15.103:60810) 2024-09-19T06:59:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0decbe2640 1 -- 172.21.15.103:0/2569739197 learned_addr learned my addr 172.21.15.103:0/2569739197 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0ded3e3640 1 --2- 172.21.15.103:0/2569739197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0de81009d0 0x7f0de8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0dedbe4640 1 --2- 172.21.15.103:0/2569739197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0de81069e0 0x7f0de819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0decbe2640 1 -- 172.21.15.103:0/2569739197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0de81009d0 msgr2=0x7f0de8198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0decbe2640 1 --2- 172.21.15.103:0/2569739197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0de81009d0 0x7f0de8198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0decbe2640 1 -- 172.21.15.103:0/2569739197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0de81069e0 msgr2=0x7f0de819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0decbe2640 1 --2- 172.21.15.103:0/2569739197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0de81069e0 0x7f0de819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0decbe2640 1 -- 172.21.15.103:0/2569739197 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de819d330 con 0x7f0de8101370 2024-09-19T06:59:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0dedbe4640 1 --2- 172.21.15.103:0/2569739197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0de81069e0 0x7f0de819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0decbe2640 1 --2- 172.21.15.103:0/2569739197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0de8101370 0x7f0de8198880 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800d950 tx=0x7f0dd800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.510+0000 7f0dde7fc640 1 -- 172.21.15.103:0/2569739197 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8017030 con 0x7f0de8101370 2024-09-19T06:59:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.511+0000 7f0dde7fc640 1 -- 172.21.15.103:0/2569739197 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dd8004020 con 0x7f0de8101370 2024-09-19T06:59:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.511+0000 7f0dde7fc640 1 -- 172.21.15.103:0/2569739197 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd80047a0 con 0x7f0de8101370 2024-09-19T06:59:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.511+0000 7f0def647640 1 -- 172.21.15.103:0/2569739197 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de819d620 con 0x7f0de8101370 2024-09-19T06:59:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.511+0000 7f0def647640 1 -- 172.21.15.103:0/2569739197 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de81a4eb0 con 0x7f0de8101370 2024-09-19T06:59:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.513+0000 7f0dde7fc640 1 -- 172.21.15.103:0/2569739197 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0dd80041d0 con 0x7f0de8101370 2024-09-19T06:59:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.513+0000 7f0df23df740 1 Processor -- start 2024-09-19T06:59:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.513+0000 7f0df23df740 1 -- start start 2024-09-19T06:59:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0df23df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b6fcaf0 0x563f5b6fced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0df23df740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b707770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0df23df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 0x563f5b70a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0df23df740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563f5b6612f0 con 0x563f5b6fd410 2024-09-19T06:59:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0df23df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563f5b609270 con 0x563f5b707cb0 2024-09-19T06:59:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0df23df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563f5b65d2b0 con 0x563f5b6fcaf0 2024-09-19T06:59:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0dde7fc640 1 --2- 172.21.15.103:0/2569739197 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0dd40a3460 0x7f0dd40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0ded3e3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b707770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0ded3e3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b707770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60822/0 (socket says 172.21.15.103:60822) 2024-09-19T06:59:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0ded3e3640 1 -- 172.21.15.103:0/2484288177 learned_addr learned my addr 172.21.15.103:0/2484288177 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0dedbe4640 1 --2- 172.21.15.103:0/2569739197 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0dd40a3460 0x7f0dd40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0dde7fc640 1 -- 172.21.15.103:0/2569739197 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f0dd80cf760 con 0x7f0de8101370 2024-09-19T06:59:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0decbe2640 1 --2- 172.21.15.103:0/2484288177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 0x563f5b70a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0dedbe4640 1 --2- 172.21.15.103:0/2484288177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b6fcaf0 0x563f5b6fced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0ded3e3640 1 -- 172.21.15.103:0/2484288177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b6fcaf0 msgr2=0x563f5b6fced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0ded3e3640 1 --2- 172.21.15.103:0/2484288177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b6fcaf0 0x563f5b6fced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0ded3e3640 1 -- 172.21.15.103:0/2484288177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 msgr2=0x563f5b70a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0ded3e3640 1 --2- 172.21.15.103:0/2484288177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 0x563f5b70a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0ded3e3640 1 -- 172.21.15.103:0/2484288177 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f5b608340 con 0x563f5b6fd410 2024-09-19T06:59:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.514+0000 7f0decbe2640 1 --2- 172.21.15.103:0/2484288177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 0x563f5b70a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:59:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.515+0000 7f0ded3e3640 1 --2- 172.21.15.103:0/2484288177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b707770 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009930 tx=0x7f0de4004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccd01191ad3719a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.515+0000 7f0dedbe4640 1 --2- 172.21.15.103:0/2569739197 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0dd40a3460 0x7f0dd40a5920 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0de0013ab0 tx=0x7f0de000ac90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.515+0000 7f0dd37fe640 1 -- 172.21.15.103:0/2484288177 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4035040 con 0x563f5b6fd410 2024-09-19T06:59:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.515+0000 7f0dd37fe640 1 -- 172.21.15.103:0/2484288177 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0de4005520 con 0x563f5b6fd410 2024-09-19T06:59:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.515+0000 7f0dd37fe640 1 -- 172.21.15.103:0/2484288177 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4003850 con 0x563f5b6fd410 2024-09-19T06:59:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.516+0000 7f0df23df740 1 -- 172.21.15.103:0/2484288177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 msgr2=0x563f5b707770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.516+0000 7f0df23df740 1 --2- 172.21.15.103:0/2484288177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b707770 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009930 tx=0x7f0de4004180 comp rx=0 tx=0).stop 2024-09-19T06:59:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.516+0000 7f0df23df740 1 -- 172.21.15.103:0/2484288177 shutdown_connections 2024-09-19T06:59:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.516+0000 7f0df23df740 1 --2- 172.21.15.103:0/2484288177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 0x563f5b70a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.516+0000 7f0df23df740 1 --2- 172.21.15.103:0/2484288177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b707770 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.516+0000 7f0df23df740 1 --2- 172.21.15.103:0/2484288177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b6fcaf0 0x563f5b6fced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.516+0000 7f0df23df740 1 -- 172.21.15.103:0/2484288177 >> 172.21.15.103:0/2484288177 conn(0x563f5b6f9060 msgr2=0x563f5b6f9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.516+0000 7f0df23df740 1 -- 172.21.15.103:0/2484288177 shutdown_connections 2024-09-19T06:59:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.516+0000 7f0df23df740 1 -- 172.21.15.103:0/2484288177 wait complete. 2024-09-19T06:59:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.516+0000 7f0df23df740 1 -- start start 2024-09-19T06:59:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.516+0000 7f0df23df740 1 Processor -- start 2024-09-19T06:59:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0df23df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b6fcaf0 0x563f5b704be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0df23df740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b705120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0df23df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 0x563f5b705660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0df23df740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563f5b70a880 con 0x563f5b6fd410 2024-09-19T06:59:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0df23df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563f5b5ea290 con 0x563f5b707cb0 2024-09-19T06:59:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0df23df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563f5b70aa00 con 0x563f5b6fcaf0 2024-09-19T06:59:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0df23df740 10 client.0 _ll_register_callbacks cb 0x563f5b61b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0decbe2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b705120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0decbe2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b705120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60836/0 (socket says 172.21.15.103:60836) 2024-09-19T06:59:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0decbe2640 1 -- 172.21.15.103:0/1780096934 learned_addr learned my addr 172.21.15.103:0/1780096934 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0dedbe4640 1 --2- 172.21.15.103:0/1780096934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 0x563f5b705660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0decbe2640 1 -- 172.21.15.103:0/1780096934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b6fcaf0 msgr2=0x563f5b704be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0decbe2640 1 --2- 172.21.15.103:0/1780096934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b6fcaf0 0x563f5b704be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0decbe2640 1 -- 172.21.15.103:0/1780096934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 msgr2=0x563f5b705660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0decbe2640 1 --2- 172.21.15.103:0/1780096934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 0x563f5b705660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0decbe2640 1 -- 172.21.15.103:0/1780096934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f5b374210 con 0x563f5b6fd410 2024-09-19T06:59:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0dedbe4640 1 --2- 172.21.15.103:0/1780096934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 0x563f5b705660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0decbe2640 1 --2- 172.21.15.103:0/1780096934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b705120 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8004770 tx=0x7f0dd809b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0ebd1f850c79f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0dd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0dd0ff9640 1 -- 172.21.15.103:0/1780096934 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd809fd70 con 0x563f5b6fd410 2024-09-19T06:59:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0dd0ff9640 1 -- 172.21.15.103:0/1780096934 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f0dd8018d30 con 0x563f5b6fd410 2024-09-19T06:59:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.517+0000 7f0df23df740 1 -- 172.21.15.103:0/1780096934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0de4005520 con 0x563f5b6fd410 2024-09-19T06:59:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.518+0000 7f0dd0ff9640 1 -- 172.21.15.103:0/1780096934 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0dd80a4630 con 0x563f5b6fd410 2024-09-19T06:59:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.518+0000 7f0df23df740 10 client.18900 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.518+0000 7f0df23df740 1 -- 172.21.15.103:0/1780096934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f5b440620 con 0x563f5b6fd410 2024-09-19T06:59:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.518+0000 7f0dd0ff9640 1 -- 172.21.15.103:0/1780096934 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0de4005520 con 0x563f5b6fd410 2024-09-19T06:59:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.518+0000 7f0dd0ff9640 1 -- 172.21.15.103:0/1780096934 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0dd80cd140 con 0x563f5b6fd410 2024-09-19T06:59:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.518+0000 7f0df23df740 10 client.18900 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.518+0000 7f0df23df740 10 client.18900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.518+0000 7f0df23df740 20 client.18900 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.518+0000 7f0df23df740 1 --2- 172.21.15.103:0/1780096934 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563f5b717c40 0x563f5b71a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.518+0000 7f0df23df740 4 client.18900 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.518+0000 7f0df23df740 1 -- 172.21.15.103:0/1780096934 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f5b66ea20 con 0x563f5b717c40 2024-09-19T06:59:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.518+0000 7f0ded3e3640 1 --2- 172.21.15.103:0/1780096934 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563f5b717c40 0x563f5b71a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.519+0000 7f0dd0ff9640 10 client.18900 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.519+0000 7f0ded3e3640 1 --2- 172.21.15.103:0/1780096934 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563f5b717c40 0x563f5b71a030 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bad9097f929bca34 server_cookie=f148c8827da886db in_seq=0 out_seq=0 2024-09-19T06:59:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.519+0000 7f0dd0ff9640 1 -- 172.21.15.103:0/1780096934 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563f5b66ea20 con 0x563f5b717c40 2024-09-19T06:59:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.519+0000 7f0dd0ff9640 10 client.18900 handle_command_reply: tid=0 2024-09-19T06:59:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.520+0000 7f0df23df740 1 client.18900 shutdown 2024-09-19T06:59:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.520+0000 7f0df23df740 1 -- 172.21.15.103:0/1780096934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 msgr2=0x563f5b705120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.520+0000 7f0df23df740 1 --2- 172.21.15.103:0/1780096934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b705120 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8004770 tx=0x7f0dd809b5c0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.520+0000 7f0df23df740 1 -- 172.21.15.103:0/1780096934 shutdown_connections 2024-09-19T06:59:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.520+0000 7f0df23df740 1 --2- 172.21.15.103:0/1780096934 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563f5b717c40 0x563f5b71a030 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.520+0000 7f0df23df740 1 --2- 172.21.15.103:0/1780096934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 0x563f5b705660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.520+0000 7f0dd0ff9640 0 client.18900 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.520+0000 7f0df23df740 1 --2- 172.21.15.103:0/1780096934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b705120 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.520+0000 7f0df23df740 1 --2- 172.21.15.103:0/1780096934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b6fcaf0 0x563f5b704be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.520+0000 7f0df23df740 1 -- 172.21.15.103:0/1780096934 >> 172.21.15.103:0/1780096934 conn(0x563f5b6f9060 msgr2=0x563f5b6fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.520+0000 7f0df23df740 1 -- 172.21.15.103:0/1780096934 shutdown_connections 2024-09-19T06:59:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.520+0000 7f0df23df740 1 -- 172.21.15.103:0/1780096934 wait complete. 2024-09-19T06:59:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.520+0000 7f0df23df740 20 client.18900 trim_cache size 0 max 16384 2024-09-19T06:59:16.535 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.535+0000 7f0df23df740 1 Processor -- start 2024-09-19T06:59:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.535+0000 7f0df23df740 1 -- start start 2024-09-19T06:59:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.535+0000 7f0df23df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b6fcaf0 0x563f5b6ff560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.535+0000 7f0df23df740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b706b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.535+0000 7f0df23df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b707cb0 0x563f5b705490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.535+0000 7f0df23df740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0de81ad870 con 0x563f5b6fd410 2024-09-19T06:59:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.535+0000 7f0df23df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0de81af3b0 con 0x563f5b6fcaf0 2024-09-19T06:59:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.535+0000 7f0df23df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0de81adf40 con 0x563f5b707cb0 2024-09-19T06:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.535+0000 7f0decbe2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b706b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.535+0000 7f0decbe2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b706b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60852/0 (socket says 172.21.15.103:60852) 2024-09-19T06:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.535+0000 7f0decbe2640 1 -- 172.21.15.103:0/1664270388 learned_addr learned my addr 172.21.15.103:0/1664270388 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.535+0000 7f0ded3e3640 1 --2- 172.21.15.103:0/1664270388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b6fcaf0 0x563f5b6ff560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.536+0000 7f0dedbe4640 1 --2- 172.21.15.103:0/1664270388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b707cb0 0x563f5b705490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.536+0000 7f0decbe2640 1 -- 172.21.15.103:0/1664270388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b707cb0 msgr2=0x563f5b705490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.536+0000 7f0decbe2640 1 --2- 172.21.15.103:0/1664270388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b707cb0 0x563f5b705490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.536+0000 7f0decbe2640 1 -- 172.21.15.103:0/1664270388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b6fcaf0 msgr2=0x563f5b6ff560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.536+0000 7f0decbe2640 1 --2- 172.21.15.103:0/1664270388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b6fcaf0 0x563f5b6ff560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.536+0000 7f0decbe2640 1 -- 172.21.15.103:0/1664270388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f5b6fc870 con 0x563f5b6fd410 2024-09-19T06:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.536+0000 7f0ded3e3640 1 --2- 172.21.15.103:0/1664270388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b6fcaf0 0x563f5b6ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.536+0000 7f0dedbe4640 1 --2- 172.21.15.103:0/1664270388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b707cb0 0x563f5b705490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.536+0000 7f0decbe2640 1 --2- 172.21.15.103:0/1664270388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b706b50 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8086010 tx=0x7f0dd8099ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df01ec8ff29f86dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.536+0000 7f0dd2ffd640 1 -- 172.21.15.103:0/1664270388 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd809fd70 con 0x563f5b6fd410 2024-09-19T06:59:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.536+0000 7f0dd2ffd640 1 -- 172.21.15.103:0/1664270388 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dd80b4070 con 0x563f5b6fd410 2024-09-19T06:59:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.536+0000 7f0dd2ffd640 1 -- 172.21.15.103:0/1664270388 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd80b4e20 con 0x563f5b6fd410 2024-09-19T06:59:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.537+0000 7f0df23df740 1 -- 172.21.15.103:0/1664270388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 msgr2=0x563f5b706b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.537+0000 7f0df23df740 1 --2- 172.21.15.103:0/1664270388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b706b50 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8086010 tx=0x7f0dd8099ed0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.537+0000 7f0df23df740 1 -- 172.21.15.103:0/1664270388 shutdown_connections 2024-09-19T06:59:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.537+0000 7f0df23df740 1 --2- 172.21.15.103:0/1664270388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b707cb0 0x563f5b705490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.537+0000 7f0df23df740 1 --2- 172.21.15.103:0/1664270388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b706b50 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.537+0000 7f0df23df740 1 --2- 172.21.15.103:0/1664270388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b6fcaf0 0x563f5b6ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.537+0000 7f0df23df740 1 -- 172.21.15.103:0/1664270388 >> 172.21.15.103:0/1664270388 conn(0x563f5b6f9060 msgr2=0x563f5b706440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.537+0000 7f0df23df740 1 -- 172.21.15.103:0/1664270388 shutdown_connections 2024-09-19T06:59:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.537+0000 7f0df23df740 1 -- 172.21.15.103:0/1664270388 wait complete. 2024-09-19T06:59:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.537+0000 7f0df23df740 1 -- start start 2024-09-19T06:59:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.537+0000 7f0df23df740 1 Processor -- start 2024-09-19T06:59:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0df23df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b6fcaf0 0x563f5b70ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0df23df740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b710470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0df23df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 0x563f5b701540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0df23df740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0de81aece0 con 0x563f5b6fd410 2024-09-19T06:59:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0df23df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0de81ae610 con 0x563f5b707cb0 2024-09-19T06:59:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0df23df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0de81ab870 con 0x563f5b6fcaf0 2024-09-19T06:59:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0df23df740 10 client.0 _ll_register_callbacks cb 0x7f0dd8097c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0decbe2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b710470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0decbe2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b710470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60858/0 (socket says 172.21.15.103:60858) 2024-09-19T06:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0decbe2640 1 -- 172.21.15.103:0/404456511 learned_addr learned my addr 172.21.15.103:0/404456511 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0ded3e3640 1 --2- 172.21.15.103:0/404456511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b6fcaf0 0x563f5b70ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0dedbe4640 1 --2- 172.21.15.103:0/404456511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 0x563f5b701540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0decbe2640 1 -- 172.21.15.103:0/404456511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b6fcaf0 msgr2=0x563f5b70ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0decbe2640 1 --2- 172.21.15.103:0/404456511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b6fcaf0 0x563f5b70ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0decbe2640 1 -- 172.21.15.103:0/404456511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 msgr2=0x563f5b701540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0decbe2640 1 --2- 172.21.15.103:0/404456511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 0x563f5b701540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0decbe2640 1 -- 172.21.15.103:0/404456511 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f5b701d40 con 0x563f5b6fd410 2024-09-19T06:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0decbe2640 1 --2- 172.21.15.103:0/404456511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b710470 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8005fd0 tx=0x7f0dd809b190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54a1364af59c5e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0dd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0dd0ff9640 1 -- 172.21.15.103:0/404456511 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd807e070 con 0x563f5b6fd410 2024-09-19T06:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0dd0ff9640 1 -- 172.21.15.103:0/404456511 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f0dd80b4bc0 con 0x563f5b6fd410 2024-09-19T06:59:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.538+0000 7f0df23df740 1 -- 172.21.15.103:0/404456511 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dd80b4070 con 0x563f5b6fd410 2024-09-19T06:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.539+0000 7f0dd0ff9640 1 -- 172.21.15.103:0/404456511 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0dd8020160 con 0x563f5b6fd410 2024-09-19T06:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.539+0000 7f0df23df740 10 client.18906 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.539+0000 7f0df23df740 1 -- 172.21.15.103:0/404456511 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f5b7867d0 con 0x563f5b6fd410 2024-09-19T06:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.539+0000 7f0dd0ff9640 1 -- 172.21.15.103:0/404456511 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dd80b4070 con 0x563f5b6fd410 2024-09-19T06:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.539+0000 7f0dd0ff9640 1 -- 172.21.15.103:0/404456511 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0dd80a0fe0 con 0x563f5b6fd410 2024-09-19T06:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.539+0000 7f0df23df740 10 client.18906 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.539+0000 7f0df23df740 10 client.18906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.539+0000 7f0df23df740 20 client.18906 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.539+0000 7f0df23df740 1 --2- 172.21.15.103:0/404456511 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563f5b786c80 0x563f5b7890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.539+0000 7f0df23df740 4 client.18906 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.539+0000 7f0df23df740 1 -- 172.21.15.103:0/404456511 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f5b668d70 con 0x563f5b786c80 2024-09-19T06:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.539+0000 7f0ded3e3640 1 --2- 172.21.15.103:0/404456511 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563f5b786c80 0x563f5b7890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.540+0000 7f0dd0ff9640 10 client.18906 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.540+0000 7f0ded3e3640 1 --2- 172.21.15.103:0/404456511 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563f5b786c80 0x563f5b7890a0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfa7e55470e5ed6b server_cookie=bcf62d8b187a5bd8 in_seq=0 out_seq=0 2024-09-19T06:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.552+0000 7f0dd0ff9640 1 -- 172.21.15.103:0/404456511 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f5b668d70 con 0x563f5b786c80 2024-09-19T06:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.552+0000 7f0dd0ff9640 10 client.18906 handle_command_reply: tid=0 2024-09-19T06:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.552+0000 7f0df23df740 1 client.18906 shutdown 2024-09-19T06:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.552+0000 7f0df23df740 1 -- 172.21.15.103:0/404456511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 msgr2=0x563f5b710470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.552+0000 7f0df23df740 1 --2- 172.21.15.103:0/404456511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b710470 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8005fd0 tx=0x7f0dd809b190 comp rx=0 tx=0).stop 2024-09-19T06:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.552+0000 7f0df23df740 1 -- 172.21.15.103:0/404456511 shutdown_connections 2024-09-19T06:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.552+0000 7f0df23df740 1 --2- 172.21.15.103:0/404456511 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563f5b786c80 0x563f5b7890a0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.552+0000 7f0dd0ff9640 0 client.18906 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.552+0000 7f0df23df740 1 --2- 172.21.15.103:0/404456511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f5b707cb0 0x563f5b701540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.552+0000 7f0df23df740 1 --2- 172.21.15.103:0/404456511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f5b6fd410 0x563f5b710470 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.552+0000 7f0df23df740 1 --2- 172.21.15.103:0/404456511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5b6fcaf0 0x563f5b70ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.552+0000 7f0df23df740 1 -- 172.21.15.103:0/404456511 >> 172.21.15.103:0/404456511 conn(0x563f5b6f9060 msgr2=0x563f5b703050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.553+0000 7f0df23df740 1 -- 172.21.15.103:0/404456511 shutdown_connections 2024-09-19T06:59:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.553+0000 7f0df23df740 1 -- 172.21.15.103:0/404456511 wait complete. 2024-09-19T06:59:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.553+0000 7f0df23df740 20 client.18906 trim_cache size 0 max 16384 2024-09-19T06:59:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:59:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:59:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:59:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:59:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.553+0000 7f0def647640 1 -- 172.21.15.103:0/2569739197 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0dd40a3460 msgr2=0x7f0dd40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.553+0000 7f0def647640 1 --2- 172.21.15.103:0/2569739197 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0dd40a3460 0x7f0dd40a5920 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0de0013ab0 tx=0x7f0de000ac90 comp rx=0 tx=0).stop 2024-09-19T06:59:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.553+0000 7f0def647640 1 -- 172.21.15.103:0/2569739197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0de8101370 msgr2=0x7f0de8198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.553+0000 7f0def647640 1 --2- 172.21.15.103:0/2569739197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0de8101370 0x7f0de8198880 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800d950 tx=0x7f0dd800de10 comp rx=0 tx=0).stop 2024-09-19T06:59:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.554+0000 7f0def647640 1 -- 172.21.15.103:0/2569739197 shutdown_connections 2024-09-19T06:59:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.554+0000 7f0def647640 1 --2- 172.21.15.103:0/2569739197 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0dd40a3460 0x7f0dd40a5920 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.554+0000 7f0def647640 1 --2- 172.21.15.103:0/2569739197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0de81069e0 0x7f0de819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.554+0000 7f0def647640 1 --2- 172.21.15.103:0/2569739197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0de8101370 0x7f0de8198880 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.554+0000 7f0def647640 1 --2- 172.21.15.103:0/2569739197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0de81009d0 0x7f0de8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.554+0000 7f0def647640 1 -- 172.21.15.103:0/2569739197 >> 172.21.15.103:0/2569739197 conn(0x7f0de80fc6f0 msgr2=0x7f0de80fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.554+0000 7f0def647640 1 -- 172.21.15.103:0/2569739197 shutdown_connections 2024-09-19T06:59:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:16.554+0000 7f0def647640 1 -- 172.21.15.103:0/2569739197 wait complete. 2024-09-19T06:59:16.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.967+0000 7fcdb4d9e640 1 -- 172.21.15.103:0/2984030217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0102ef0 msgr2=0x7fcdb010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:16.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.967+0000 7fcdb4d9e640 1 --2- 172.21.15.103:0/2984030217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0102ef0 0x7fcdb010da40 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009ca0 tx=0x7fcda802a950 comp rx=0 tx=0).stop 2024-09-19T06:59:16.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.967+0000 7fcdb4d9e640 1 -- 172.21.15.103:0/2984030217 shutdown_connections 2024-09-19T06:59:16.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.967+0000 7fcdb4d9e640 1 --2- 172.21.15.103:0/2984030217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0102ef0 0x7fcdb010da40 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.967+0000 7fcdb4d9e640 1 --2- 172.21.15.103:0/2984030217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdb0102570 0x7fcdb01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.967+0000 7fcdb4d9e640 1 --2- 172.21.15.103:0/2984030217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb0100e20 0x7fcdb0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.967+0000 7fcdb4d9e640 1 -- 172.21.15.103:0/2984030217 >> 172.21.15.103:0/2984030217 conn(0x7fcdb00fc7d0 msgr2=0x7fcdb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:16.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.967+0000 7fcdb4d9e640 1 -- 172.21.15.103:0/2984030217 shutdown_connections 2024-09-19T06:59:16.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.967+0000 7fcdb4d9e640 1 -- 172.21.15.103:0/2984030217 wait complete. 2024-09-19T06:59:16.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdb4d9e640 1 Processor -- start 2024-09-19T06:59:16.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdb4d9e640 1 -- start start 2024-09-19T06:59:16.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdb4d9e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0100e20 0x7fcdb01994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdb4d9e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb0102570 0x7fcdb01999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdb4d9e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdb0102ef0 0x7fcdb01a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdb4d9e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb0111020 con 0x7fcdb0100e20 2024-09-19T06:59:16.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdb4d9e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb0110ea0 con 0x7fcdb0102ef0 2024-09-19T06:59:16.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdb4d9e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb01111a0 con 0x7fcdb0102570 2024-09-19T06:59:16.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0100e20 0x7fcdb01994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0100e20 0x7fcdb01994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60874/0 (socket says 172.21.15.103:60874) 2024-09-19T06:59:16.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdae59c640 1 -- 172.21.15.103:0/621017814 learned_addr learned my addr 172.21.15.103:0/621017814 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:16.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdaed9d640 1 --2- 172.21.15.103:0/621017814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdb0102ef0 0x7fcdb01a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdadd9b640 1 --2- 172.21.15.103:0/621017814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb0102570 0x7fcdb01999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdae59c640 1 -- 172.21.15.103:0/621017814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb0102570 msgr2=0x7fcdb01999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:16.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdae59c640 1 --2- 172.21.15.103:0/621017814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb0102570 0x7fcdb01999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdae59c640 1 -- 172.21.15.103:0/621017814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdb0102ef0 msgr2=0x7fcdb01a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:16.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdae59c640 1 --2- 172.21.15.103:0/621017814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdb0102ef0 0x7fcdb01a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdae59c640 1 -- 172.21.15.103:0/621017814 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb01a4a70 con 0x7fcdb0100e20 2024-09-19T06:59:16.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.968+0000 7fcdaed9d640 1 --2- 172.21.15.103:0/621017814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdb0102ef0 0x7fcdb01a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:16.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.969+0000 7fcdae59c640 1 --2- 172.21.15.103:0/621017814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0100e20 0x7fcdb01994b0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fcda000b4a0 tx=0x7fcda000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:16.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.969+0000 7fcda77fe640 1 -- 172.21.15.103:0/621017814 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0014070 con 0x7fcdb0100e20 2024-09-19T06:59:16.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.969+0000 7fcdb4d9e640 1 -- 172.21.15.103:0/621017814 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb01a4c40 con 0x7fcdb0100e20 2024-09-19T06:59:16.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.969+0000 7fcda77fe640 1 -- 172.21.15.103:0/621017814 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda0010af0 con 0x7fcdb0100e20 2024-09-19T06:59:16.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.969+0000 7fcda77fe640 1 -- 172.21.15.103:0/621017814 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0017d10 con 0x7fcdb0100e20 2024-09-19T06:59:16.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.969+0000 7fcdb4d9e640 1 -- 172.21.15.103:0/621017814 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb01a5150 con 0x7fcdb0100e20 2024-09-19T06:59:16.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.972+0000 7fcdb7177740 1 Processor -- start 2024-09-19T06:59:16.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.972+0000 7fcdb7177740 1 -- start start 2024-09-19T06:59:16.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.972+0000 7fcda77fe640 1 -- 172.21.15.103:0/621017814 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fcda0017070 con 0x7fcdb0100e20 2024-09-19T06:59:16.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.972+0000 7fcdb7177740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a394ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.972+0000 7fcdb7177740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a3950480 0x5620a395a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.972+0000 7fcdb7177740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a395ad40 0x5620a395d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.972+0000 7fcdb7177740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5620a38b5280 con 0x5620a394fb60 2024-09-19T06:59:16.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.972+0000 7fcdb7177740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5620a385cb10 con 0x5620a395ad40 2024-09-19T06:59:16.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.972+0000 7fcdb7177740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5620a38b1240 con 0x5620a3950480 2024-09-19T06:59:16.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.972+0000 7fcdaed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a394ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.972+0000 7fcdaed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a394ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60882/0 (socket says 172.21.15.103:60882) 2024-09-19T06:59:16.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.972+0000 7fcdaed9d640 1 -- 172.21.15.103:0/2040380250 learned_addr learned my addr 172.21.15.103:0/2040380250 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:16.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.973+0000 7fcdae59c640 1 --2- 172.21.15.103:0/2040380250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a3950480 0x5620a395a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.973+0000 7fcdadd9b640 1 --2- 172.21.15.103:0/2040380250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a395ad40 0x5620a395d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.973+0000 7fcdaed9d640 1 -- 172.21.15.103:0/2040380250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a3950480 msgr2=0x5620a395a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:16.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.973+0000 7fcdaed9d640 1 --2- 172.21.15.103:0/2040380250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a3950480 0x5620a395a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.973+0000 7fcdaed9d640 1 -- 172.21.15.103:0/2040380250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a395ad40 msgr2=0x5620a395d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:16.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.973+0000 7fcdaed9d640 1 --2- 172.21.15.103:0/2040380250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a395ad40 0x5620a395d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.973+0000 7fcdaed9d640 1 -- 172.21.15.103:0/2040380250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620a38691f0 con 0x5620a394fb60 2024-09-19T06:59:16.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.973+0000 7fcdadd9b640 1 --2- 172.21.15.103:0/2040380250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a395ad40 0x5620a395d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:16.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.973+0000 7fcdaed9d640 1 --2- 172.21.15.103:0/2040380250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a394ff40 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fcda8030030 tx=0x7fcda8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b23278bf59eeb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:16.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.973+0000 7fcda77fe640 1 --2- 172.21.15.103:0/621017814 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd9c0a33b0 0x7fcd9c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.973+0000 7fcda4ff9640 1 -- 172.21.15.103:0/2040380250 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8034050 con 0x5620a394fb60 2024-09-19T06:59:16.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.973+0000 7fcda4ff9640 1 -- 172.21.15.103:0/2040380250 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda8003970 con 0x5620a394fb60 2024-09-19T06:59:16.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.973+0000 7fcda4ff9640 1 -- 172.21.15.103:0/2040380250 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8004590 con 0x5620a394fb60 2024-09-19T06:59:16.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.973+0000 7fcdae59c640 1 --2- 172.21.15.103:0/621017814 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd9c0a33b0 0x7fcd9c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.973+0000 7fcda77fe640 1 -- 172.21.15.103:0/621017814 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7fcda00cf010 con 0x7fcdb0100e20 2024-09-19T06:59:16.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.974+0000 7fcdb7177740 1 -- 172.21.15.103:0/2040380250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 msgr2=0x5620a394ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:16.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.974+0000 7fcdae59c640 1 --2- 172.21.15.103:0/621017814 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd9c0a33b0 0x7fcd9c0a5870 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcda00cc900 tx=0x7fcda00d2c40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:16.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.974+0000 7fcdb7177740 1 --2- 172.21.15.103:0/2040380250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a394ff40 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fcda8030030 tx=0x7fcda8009a20 comp rx=0 tx=0).stop 2024-09-19T06:59:16.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.974+0000 7fcdb7177740 1 -- 172.21.15.103:0/2040380250 shutdown_connections 2024-09-19T06:59:16.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.974+0000 7fcdb7177740 1 --2- 172.21.15.103:0/2040380250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a395ad40 0x5620a395d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.974+0000 7fcdb7177740 1 --2- 172.21.15.103:0/2040380250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a3950480 0x5620a395a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.974+0000 7fcdb7177740 1 --2- 172.21.15.103:0/2040380250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a394ff40 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.974+0000 7fcdb7177740 1 -- 172.21.15.103:0/2040380250 >> 172.21.15.103:0/2040380250 conn(0x5620a394c0f0 msgr2=0x5620a394c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:16.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.974+0000 7fcdb7177740 1 -- 172.21.15.103:0/2040380250 shutdown_connections 2024-09-19T06:59:16.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.974+0000 7fcdb7177740 1 -- 172.21.15.103:0/2040380250 wait complete. 2024-09-19T06:59:16.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdb7177740 1 -- start start 2024-09-19T06:59:16.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdb7177740 1 Processor -- start 2024-09-19T06:59:16.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdb7177740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a395a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdb7177740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a3950480 0x5620a3957b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdb7177740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a395ad40 0x5620a39582c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdb7177740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5620a395d910 con 0x5620a394fb60 2024-09-19T06:59:16.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdb7177740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5620a386ff10 con 0x5620a3950480 2024-09-19T06:59:16.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdb7177740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5620a395da90 con 0x5620a395ad40 2024-09-19T06:59:16.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdb7177740 10 client.0 _ll_register_callbacks cb 0x5620a3838a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:16.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdaed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a395a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdaed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a395a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60888/0 (socket says 172.21.15.103:60888) 2024-09-19T06:59:16.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdaed9d640 1 -- 172.21.15.103:0/3485705136 learned_addr learned my addr 172.21.15.103:0/3485705136 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:16.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdadd9b640 1 --2- 172.21.15.103:0/3485705136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a3950480 0x5620a3957b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdaed9d640 1 --2- 172.21.15.103:0/3485705136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a395ad40 0x5620a39582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdaed9d640 1 -- 172.21.15.103:0/3485705136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a395ad40 msgr2=0x5620a39582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:16.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdaed9d640 1 --2- 172.21.15.103:0/3485705136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a395ad40 0x5620a39582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdaed9d640 1 -- 172.21.15.103:0/3485705136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a3950480 msgr2=0x5620a3957b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:16.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdaed9d640 1 --2- 172.21.15.103:0/3485705136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a3950480 0x5620a3957b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdaed9d640 1 -- 172.21.15.103:0/3485705136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620a36266a0 con 0x5620a394fb60 2024-09-19T06:59:16.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.975+0000 7fcdadd9b640 1 --2- 172.21.15.103:0/3485705136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a3950480 0x5620a3957b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:16.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.976+0000 7fcdaed9d640 1 --2- 172.21.15.103:0/3485705136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a395a7e0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fcda802a990 tx=0x7fcda802fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8c8faf982a0fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:16.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.976+0000 7fcd95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:16.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.976+0000 7fcd95ffb640 1 -- 172.21.15.103:0/3485705136 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8054060 con 0x5620a394fb60 2024-09-19T06:59:16.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.976+0000 7fcd95ffb640 1 -- 172.21.15.103:0/3485705136 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7fcda80044d0 con 0x5620a394fb60 2024-09-19T06:59:16.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.976+0000 7fcdb7177740 1 -- 172.21.15.103:0/3485705136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcda8003970 con 0x5620a394fb60 2024-09-19T06:59:16.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.976+0000 7fcd95ffb640 1 -- 172.21.15.103:0/3485705136 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcda8048040 con 0x5620a394fb60 2024-09-19T06:59:16.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.976+0000 7fcdb7177740 10 client.18918 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:16.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.976+0000 7fcdb7177740 1 -- 172.21.15.103:0/3485705136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5620a395d910 con 0x5620a394fb60 2024-09-19T06:59:16.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.976+0000 7fcd95ffb640 1 -- 172.21.15.103:0/3485705136 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda8003970 con 0x5620a394fb60 2024-09-19T06:59:16.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.976+0000 7fcd95ffb640 1 -- 172.21.15.103:0/3485705136 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcda804e030 con 0x5620a394fb60 2024-09-19T06:59:16.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.976+0000 7fcdb7177740 10 client.18918 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:16.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.976+0000 7fcdb7177740 10 client.18918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:16.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.976+0000 7fcdb7177740 20 client.18918 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:16.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.976+0000 7fcdb7177740 1 --2- 172.21.15.103:0/3485705136 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5620a396a5d0 0x5620a396c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:16.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.976+0000 7fcdb7177740 4 client.18918 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:16.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.976+0000 7fcdb7177740 1 -- 172.21.15.103:0/3485705136 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620a38c27f0 con 0x5620a396a5d0 2024-09-19T06:59:16.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.977+0000 7fcdadd9b640 1 --2- 172.21.15.103:0/3485705136 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5620a396a5d0 0x5620a396c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:16.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.977+0000 7fcdadd9b640 1 --2- 172.21.15.103:0/3485705136 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5620a396a5d0 0x5620a396c9c0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97943622a88df625 server_cookie=43a551ee26b7c4c8 in_seq=0 out_seq=0 2024-09-19T06:59:16.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.977+0000 7fcd95ffb640 10 client.18918 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:16.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.978+0000 7fcd95ffb640 1 -- 172.21.15.103:0/3485705136 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620a38c27f0 con 0x5620a396a5d0 2024-09-19T06:59:16.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.978+0000 7fcd95ffb640 10 client.18918 handle_command_reply: tid=0 2024-09-19T06:59:16.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.978+0000 7fcdb7177740 1 client.18918 shutdown 2024-09-19T06:59:16.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.978+0000 7fcdb7177740 1 -- 172.21.15.103:0/3485705136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 msgr2=0x5620a395a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:16.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.978+0000 7fcdb7177740 1 --2- 172.21.15.103:0/3485705136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a395a7e0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fcda802a990 tx=0x7fcda802fe00 comp rx=0 tx=0).stop 2024-09-19T06:59:16.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.978+0000 7fcdb7177740 1 -- 172.21.15.103:0/3485705136 shutdown_connections 2024-09-19T06:59:16.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.978+0000 7fcdb7177740 1 --2- 172.21.15.103:0/3485705136 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5620a396a5d0 0x5620a396c9c0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.978+0000 7fcdb7177740 1 --2- 172.21.15.103:0/3485705136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a395ad40 0x5620a39582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.978+0000 7fcdb7177740 1 --2- 172.21.15.103:0/3485705136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a3950480 0x5620a3957b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.978+0000 7fcdb7177740 1 --2- 172.21.15.103:0/3485705136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a395a7e0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:16.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.978+0000 7fcdb7177740 1 -- 172.21.15.103:0/3485705136 >> 172.21.15.103:0/3485705136 conn(0x5620a394c0f0 msgr2=0x5620a394b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:16.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.978+0000 7fcd95ffb640 0 client.18918 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:16.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.978+0000 7fcdb7177740 1 -- 172.21.15.103:0/3485705136 shutdown_connections 2024-09-19T06:59:16.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.978+0000 7fcdb7177740 1 -- 172.21.15.103:0/3485705136 wait complete. 2024-09-19T06:59:16.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.978+0000 7fcdb7177740 20 client.18918 trim_cache size 0 max 16384 2024-09-19T06:59:17.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.995+0000 7fcdb7177740 1 Processor -- start 2024-09-19T06:59:17.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.995+0000 7fcdb7177740 1 -- start start 2024-09-19T06:59:17.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.995+0000 7fcdb7177740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a394fb60 0x5620a3959990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:17.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.995+0000 7fcdb7177740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a3950480 0x5620a3972100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:17.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.995+0000 7fcdb7177740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a395ad40 0x5620a3972640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.995+0000 7fcdb7177740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb01ad9e0 con 0x5620a395ad40 2024-09-19T06:59:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.995+0000 7fcdb7177740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb01af520 con 0x5620a394fb60 2024-09-19T06:59:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.995+0000 7fcdb7177740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb01ae0b0 con 0x5620a3950480 2024-09-19T06:59:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.995+0000 7fcdaed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a395ad40 0x5620a3972640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.995+0000 7fcdaed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a395ad40 0x5620a3972640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60900/0 (socket says 172.21.15.103:60900) 2024-09-19T06:59:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.995+0000 7fcdaed9d640 1 -- 172.21.15.103:0/463542525 learned_addr learned my addr 172.21.15.103:0/463542525 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.995+0000 7fcdaed9d640 1 -- 172.21.15.103:0/463542525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a3950480 msgr2=0x5620a3972100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.995+0000 7fcdaed9d640 1 --2- 172.21.15.103:0/463542525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a3950480 0x5620a3972100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.995+0000 7fcdaed9d640 1 -- 172.21.15.103:0/463542525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a394fb60 msgr2=0x5620a3959990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.995+0000 7fcdaed9d640 1 --2- 172.21.15.103:0/463542525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a394fb60 0x5620a3959990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.995+0000 7fcdaed9d640 1 -- 172.21.15.103:0/463542525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620a39502b0 con 0x5620a395ad40 2024-09-19T06:59:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.996+0000 7fcdaed9d640 1 --2- 172.21.15.103:0/463542525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a395ad40 0x5620a3972640 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fcda8002230 tx=0x7fcda8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b656744ef284d40f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.996+0000 7fcd97fff640 1 -- 172.21.15.103:0/463542525 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8054070 con 0x5620a395ad40 2024-09-19T06:59:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.996+0000 7fcd97fff640 1 -- 172.21.15.103:0/463542525 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda8003970 con 0x5620a395ad40 2024-09-19T06:59:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.996+0000 7fcd97fff640 1 -- 172.21.15.103:0/463542525 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8034040 con 0x5620a395ad40 2024-09-19T06:59:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.996+0000 7fcdb7177740 1 -- 172.21.15.103:0/463542525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a395ad40 msgr2=0x5620a3972640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.996+0000 7fcdb7177740 1 --2- 172.21.15.103:0/463542525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a395ad40 0x5620a3972640 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fcda8002230 tx=0x7fcda8005ef0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.996+0000 7fcdb7177740 1 -- 172.21.15.103:0/463542525 shutdown_connections 2024-09-19T06:59:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.996+0000 7fcdb7177740 1 --2- 172.21.15.103:0/463542525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a395ad40 0x5620a3972640 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.996+0000 7fcdb7177740 1 --2- 172.21.15.103:0/463542525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a3950480 0x5620a3972100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.996+0000 7fcdb7177740 1 --2- 172.21.15.103:0/463542525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a394fb60 0x5620a3959990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.996+0000 7fcdb7177740 1 -- 172.21.15.103:0/463542525 >> 172.21.15.103:0/463542525 conn(0x5620a394c0f0 msgr2=0x5620a3959280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.996+0000 7fcdb7177740 1 -- 172.21.15.103:0/463542525 shutdown_connections 2024-09-19T06:59:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.996+0000 7fcdb7177740 1 -- 172.21.15.103:0/463542525 wait complete. 2024-09-19T06:59:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.997+0000 7fcdb7177740 1 -- start start 2024-09-19T06:59:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.997+0000 7fcdb7177740 1 Processor -- start 2024-09-19T06:59:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.997+0000 7fcdb7177740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a3962d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.997+0000 7fcdb7177740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a3950480 0x5620a3963250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.997+0000 7fcdb7177740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a395ad40 0x5620a395fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.997+0000 7fcdb7177740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb01aee50 con 0x5620a394fb60 2024-09-19T06:59:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.997+0000 7fcdb7177740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb01ae780 con 0x5620a3950480 2024-09-19T06:59:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.997+0000 7fcdb7177740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb01ab9e0 con 0x5620a395ad40 2024-09-19T06:59:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.997+0000 7fcdb7177740 10 client.0 _ll_register_callbacks cb 0x7fcda00d2b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.997+0000 7fcdaed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a3962d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.997+0000 7fcdaed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a3962d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60912/0 (socket says 172.21.15.103:60912) 2024-09-19T06:59:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.997+0000 7fcdaed9d640 1 -- 172.21.15.103:0/2953326330 learned_addr learned my addr 172.21.15.103:0/2953326330 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.997+0000 7fcdadd9b640 1 --2- 172.21.15.103:0/2953326330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a3950480 0x5620a3963250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.997+0000 7fcdaed9d640 1 --2- 172.21.15.103:0/2953326330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a395ad40 0x5620a395fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.998+0000 7fcdaed9d640 1 -- 172.21.15.103:0/2953326330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a395ad40 msgr2=0x5620a395fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.998+0000 7fcdaed9d640 1 --2- 172.21.15.103:0/2953326330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a395ad40 0x5620a395fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.998+0000 7fcdaed9d640 1 -- 172.21.15.103:0/2953326330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a3950480 msgr2=0x5620a3963250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.998+0000 7fcdaed9d640 1 --2- 172.21.15.103:0/2953326330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a3950480 0x5620a3963250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.998+0000 7fcdaed9d640 1 -- 172.21.15.103:0/2953326330 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620a3963910 con 0x5620a394fb60 2024-09-19T06:59:17.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.998+0000 7fcdaed9d640 1 --2- 172.21.15.103:0/2953326330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a3962d10 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fcda8003ed0 tx=0x7fcda80098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48783bfd797560b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:17.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.998+0000 7fcd95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.998+0000 7fcd95ffb640 1 -- 172.21.15.103:0/2953326330 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8034030 con 0x5620a394fb60 2024-09-19T06:59:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.998+0000 7fcd95ffb640 1 -- 172.21.15.103:0/2953326330 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7fcda80045a0 con 0x5620a394fb60 2024-09-19T06:59:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.998+0000 7fcdb7177740 1 -- 172.21.15.103:0/2953326330 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcda8003970 con 0x5620a394fb60 2024-09-19T06:59:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.998+0000 7fcd95ffb640 1 -- 172.21.15.103:0/2953326330 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcda8054070 con 0x5620a394fb60 2024-09-19T06:59:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.998+0000 7fcdb7177740 10 client.18924 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.998+0000 7fcdb7177740 1 -- 172.21.15.103:0/2953326330 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5620a39d56e0 con 0x5620a394fb60 2024-09-19T06:59:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.998+0000 7fcd95ffb640 1 -- 172.21.15.103:0/2953326330 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda8003970 con 0x5620a394fb60 2024-09-19T06:59:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.999+0000 7fcd95ffb640 1 -- 172.21.15.103:0/2953326330 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcda804e040 con 0x5620a394fb60 2024-09-19T06:59:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.999+0000 7fcdb7177740 10 client.18924 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:17.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.999+0000 7fcdb7177740 10 client.18924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:17.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.999+0000 7fcdb7177740 20 client.18924 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:17.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.999+0000 7fcdb7177740 1 --2- 172.21.15.103:0/2953326330 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5620a39d5ac0 0x5620a39d7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:17.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.999+0000 7fcdb7177740 4 client.18924 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:59:17.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.999+0000 7fcdb7177740 1 -- 172.21.15.103:0/2953326330 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5620a38bcd00 con 0x5620a39d5ac0 2024-09-19T06:59:17.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.999+0000 7fcdadd9b640 1 --2- 172.21.15.103:0/2953326330 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5620a39d5ac0 0x5620a39d7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:17.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.999+0000 7fcd95ffb640 10 client.18924 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:17.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:16.999+0000 7fcdadd9b640 1 --2- 172.21.15.103:0/2953326330 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5620a39d5ac0 0x5620a39d7eb0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dfea0f0b7f7a226 server_cookie=efdcc390cde78317 in_seq=0 out_seq=0 2024-09-19T06:59:17.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.000+0000 7fcd95ffb640 1 -- 172.21.15.103:0/2953326330 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5620a38bcd00 con 0x5620a39d5ac0 2024-09-19T06:59:17.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.000+0000 7fcd95ffb640 10 client.18924 handle_command_reply: tid=0 2024-09-19T06:59:17.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.000+0000 7fcdb7177740 1 client.18924 shutdown 2024-09-19T06:59:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.000+0000 7fcdb7177740 1 -- 172.21.15.103:0/2953326330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 msgr2=0x5620a3962d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.000+0000 7fcdb7177740 1 --2- 172.21.15.103:0/2953326330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a3962d10 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fcda8003ed0 tx=0x7fcda80098c0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.001+0000 7fcdb7177740 1 -- 172.21.15.103:0/2953326330 shutdown_connections 2024-09-19T06:59:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.001+0000 7fcdb7177740 1 --2- 172.21.15.103:0/2953326330 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5620a39d5ac0 0x5620a39d7eb0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.001+0000 7fcdb7177740 1 --2- 172.21.15.103:0/2953326330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620a395ad40 0x5620a395fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.001+0000 7fcdb7177740 1 --2- 172.21.15.103:0/2953326330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620a3950480 0x5620a3963250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.001+0000 7fcd95ffb640 0 client.18924 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.001+0000 7fcdb7177740 1 --2- 172.21.15.103:0/2953326330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620a394fb60 0x5620a3962d10 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.001+0000 7fcdb7177740 1 -- 172.21.15.103:0/2953326330 >> 172.21.15.103:0/2953326330 conn(0x5620a394c0f0 msgr2=0x5620a3956af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.001+0000 7fcdb7177740 1 -- 172.21.15.103:0/2953326330 shutdown_connections 2024-09-19T06:59:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.001+0000 7fcdb7177740 1 -- 172.21.15.103:0/2953326330 wait complete. 2024-09-19T06:59:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.001+0000 7fcdb7177740 20 client.18924 trim_cache size 0 max 16384 2024-09-19T06:59:17.011 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:59:17.011 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T06:59:17.011 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "72bdeff4-2204-4c05-a1cb-97e8da6f7bc9", 2024-09-19T06:59:17.011 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T06:59:17.011 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:59:17.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.001+0000 7fcdb4d9e640 1 -- 172.21.15.103:0/621017814 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd9c0a33b0 msgr2=0x7fcd9c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:17.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.001+0000 7fcdb4d9e640 1 --2- 172.21.15.103:0/621017814 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd9c0a33b0 0x7fcd9c0a5870 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcda00cc900 tx=0x7fcda00d2c40 comp rx=0 tx=0).stop 2024-09-19T06:59:17.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.001+0000 7fcdb4d9e640 1 -- 172.21.15.103:0/621017814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0100e20 msgr2=0x7fcdb01994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:17.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.001+0000 7fcdb4d9e640 1 --2- 172.21.15.103:0/621017814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0100e20 0x7fcdb01994b0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fcda000b4a0 tx=0x7fcda000b960 comp rx=0 tx=0).stop 2024-09-19T06:59:17.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.002+0000 7fcdb4d9e640 1 -- 172.21.15.103:0/621017814 shutdown_connections 2024-09-19T06:59:17.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.002+0000 7fcdb4d9e640 1 --2- 172.21.15.103:0/621017814 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd9c0a33b0 0x7fcd9c0a5870 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.002+0000 7fcdb4d9e640 1 --2- 172.21.15.103:0/621017814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdb0102ef0 0x7fcdb01a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.002+0000 7fcdb4d9e640 1 --2- 172.21.15.103:0/621017814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb0102570 0x7fcdb01999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.002+0000 7fcdb4d9e640 1 --2- 172.21.15.103:0/621017814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0100e20 0x7fcdb01994b0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:17.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.002+0000 7fcdb4d9e640 1 -- 172.21.15.103:0/621017814 >> 172.21.15.103:0/621017814 conn(0x7fcdb00fc7d0 msgr2=0x7fcdb00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:17.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.002+0000 7fcdb4d9e640 1 -- 172.21.15.103:0/621017814 shutdown_connections 2024-09-19T06:59:17.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:17.002+0000 7fcdb4d9e640 1 -- 172.21.15.103:0/621017814 wait complete. 2024-09-19T06:59:17.643 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:59:17.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:17 smithi103 ceph-mon[31158]: pgmap v312: 97 pgs: 97 active+clean; 1.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-19T06:59:17.891 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:18.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:17 smithi174 ceph-mon[35637]: pgmap v312: 97 pgs: 97 active+clean; 1.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-19T06:59:18.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:17 smithi134 ceph-mon[35356]: pgmap v312: 97 pgs: 97 active+clean; 1.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-19T06:59:18.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.365+0000 7f962224b640 1 -- 172.21.15.103:0/1540345925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f961c106f10 msgr2=0x7f961c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:18.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.365+0000 7f962224b640 1 --2- 172.21.15.103:0/1540345925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f961c106f10 0x7f961c10ea90 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f9614009ca0 tx=0x7f961402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.365+0000 7f962224b640 1 -- 172.21.15.103:0/1540345925 shutdown_connections 2024-09-19T06:59:18.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.365+0000 7f962224b640 1 --2- 172.21.15.103:0/1540345925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f961c106f10 0x7f961c10ea90 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.365+0000 7f962224b640 1 --2- 172.21.15.103:0/1540345925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f961c102790 0x7f961c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.365+0000 7f962224b640 1 --2- 172.21.15.103:0/1540345925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f961c101de0 0x7f961c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.365+0000 7f962224b640 1 -- 172.21.15.103:0/1540345925 >> 172.21.15.103:0/1540345925 conn(0x7f961c0fd8c0 msgr2=0x7f961c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:18.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.365+0000 7f962224b640 1 -- 172.21.15.103:0/1540345925 shutdown_connections 2024-09-19T06:59:18.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.365+0000 7f962224b640 1 -- 172.21.15.103:0/1540345925 wait complete. 2024-09-19T06:59:18.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.365+0000 7f962224b640 1 Processor -- start 2024-09-19T06:59:18.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.365+0000 7f962224b640 1 -- start start 2024-09-19T06:59:18.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f962224b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f961c101de0 0x7f961c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f962224b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f961c102790 0x7f961c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f962224b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f961c106f10 0x7f961c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f962224b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f961c111f70 con 0x7f961c102790 2024-09-19T06:59:18.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f962224b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f961c111df0 con 0x7f961c101de0 2024-09-19T06:59:18.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f962224b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f961c1120f0 con 0x7f961c106f10 2024-09-19T06:59:18.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f961affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f961c102790 0x7f961c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:18.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f961affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f961c102790 0x7f961c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60970/0 (socket says 172.21.15.103:60970) 2024-09-19T06:59:18.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f961affd640 1 -- 172.21.15.103:0/1422051293 learned_addr learned my addr 172.21.15.103:0/1422051293 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:18.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f961b7fe640 1 --2- 172.21.15.103:0/1422051293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f961c101de0 0x7f961c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:18.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f961bfff640 1 --2- 172.21.15.103:0/1422051293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f961c106f10 0x7f961c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:18.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f961affd640 1 -- 172.21.15.103:0/1422051293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f961c106f10 msgr2=0x7f961c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:18.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f961affd640 1 --2- 172.21.15.103:0/1422051293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f961c106f10 0x7f961c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f961affd640 1 -- 172.21.15.103:0/1422051293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f961c101de0 msgr2=0x7f961c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:18.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f961affd640 1 --2- 172.21.15.103:0/1422051293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f961c101de0 0x7f961c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f961affd640 1 -- 172.21.15.103:0/1422051293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c1a6800 con 0x7f961c102790 2024-09-19T06:59:18.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f961affd640 1 --2- 172.21.15.103:0/1422051293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f961c102790 0x7f961c074a30 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f961000c920 tx=0x7f961000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:18.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f9618ff9640 1 -- 172.21.15.103:0/1422051293 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610007c70 con 0x7f961c102790 2024-09-19T06:59:18.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f962224b640 1 -- 172.21.15.103:0/1422051293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961c1a6af0 con 0x7f961c102790 2024-09-19T06:59:18.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f9618ff9640 1 -- 172.21.15.103:0/1422051293 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9610007e10 con 0x7f961c102790 2024-09-19T06:59:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f9618ff9640 1 -- 172.21.15.103:0/1422051293 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610018890 con 0x7f961c102790 2024-09-19T06:59:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.366+0000 7f962224b640 1 -- 172.21.15.103:0/1422051293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961c1a7080 con 0x7f961c102790 2024-09-19T06:59:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.368+0000 7f9618ff9640 1 -- 172.21.15.103:0/1422051293 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9610018a70 con 0x7f961c102790 2024-09-19T06:59:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.369+0000 7f9624624740 1 Processor -- start 2024-09-19T06:59:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.369+0000 7f9624624740 1 -- start start 2024-09-19T06:59:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.369+0000 7f9618ff9640 1 --2- 172.21.15.103:0/1422051293 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96040a33b0 0x7f96040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.369+0000 7f9624624740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382d4a0 0x564f2382d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.369+0000 7f9624624740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f2382ddc0 0x564f23838140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.369+0000 7f9624624740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f23838680 0x564f2383aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.369+0000 7f9624624740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564f237932e0 con 0x564f2382d4a0 2024-09-19T06:59:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.369+0000 7f961bfff640 1 --2- 172.21.15.103:0/1422051293 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96040a33b0 0x7f96040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.369+0000 7f9624624740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564f2373ace0 con 0x564f23838680 2024-09-19T06:59:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.369+0000 7f9624624740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564f2378f2a0 con 0x564f2382ddc0 2024-09-19T06:59:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.369+0000 7f9618ff9640 1 -- 172.21.15.103:0/1422051293 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f96100cff20 con 0x7f961c102790 2024-09-19T06:59:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.369+0000 7f961b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382d4a0 0x564f2382d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.369+0000 7f961b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382d4a0 0x564f2382d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60984/0 (socket says 172.21.15.103:60984) 2024-09-19T06:59:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.369+0000 7f961b7fe640 1 -- 172.21.15.103:0/4155055448 learned_addr learned my addr 172.21.15.103:0/4155055448 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.369+0000 7f961bfff640 1 --2- 172.21.15.103:0/4155055448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f23838680 0x564f2383aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.370+0000 7f961b7fe640 1 -- 172.21.15.103:0/4155055448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f2382ddc0 msgr2=0x564f23838140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.370+0000 7f961affd640 1 --2- 172.21.15.103:0/4155055448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f2382ddc0 0x564f23838140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.370+0000 7f961b7fe640 1 --2- 172.21.15.103:0/4155055448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f2382ddc0 0x564f23838140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.370+0000 7f961b7fe640 1 -- 172.21.15.103:0/4155055448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f23838680 msgr2=0x564f2383aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.370+0000 7f961b7fe640 1 --2- 172.21.15.103:0/4155055448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f23838680 0x564f2383aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.370+0000 7f961b7fe640 1 -- 172.21.15.103:0/4155055448 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2371f440 con 0x564f2382d4a0 2024-09-19T06:59:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.370+0000 7f961bfff640 1 --2- 172.21.15.103:0/4155055448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f23838680 0x564f2383aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.370+0000 7f961affd640 1 --2- 172.21.15.103:0/4155055448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f2382ddc0 0x564f23838140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:18.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.370+0000 7f961bfff640 1 --2- 172.21.15.103:0/1422051293 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96040a33b0 0x7f96040a5870 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9614006010 tx=0x7f9614031bb0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:18.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.370+0000 7f961b7fe640 1 --2- 172.21.15.103:0/4155055448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382d4a0 0x564f2382d880 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f960c00bcb0 tx=0x7f960c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df98384186e9b44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:18.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.370+0000 7f9601ffb640 1 -- 172.21.15.103:0/4155055448 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c015070 con 0x564f2382d4a0 2024-09-19T06:59:18.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.370+0000 7f9601ffb640 1 -- 172.21.15.103:0/4155055448 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f960c01a940 con 0x564f2382d4a0 2024-09-19T06:59:18.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.370+0000 7f9601ffb640 1 -- 172.21.15.103:0/4155055448 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c011a00 con 0x564f2382d4a0 2024-09-19T06:59:18.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.371+0000 7f9624624740 1 -- 172.21.15.103:0/4155055448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382d4a0 msgr2=0x564f2382d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:18.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.371+0000 7f9624624740 1 --2- 172.21.15.103:0/4155055448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382d4a0 0x564f2382d880 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f960c00bcb0 tx=0x7f960c00f730 comp rx=0 tx=0).stop 2024-09-19T06:59:18.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.371+0000 7f9624624740 1 -- 172.21.15.103:0/4155055448 shutdown_connections 2024-09-19T06:59:18.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.371+0000 7f9624624740 1 --2- 172.21.15.103:0/4155055448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f23838680 0x564f2383aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.371+0000 7f9624624740 1 --2- 172.21.15.103:0/4155055448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f2382ddc0 0x564f23838140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.371+0000 7f9624624740 1 --2- 172.21.15.103:0/4155055448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382d4a0 0x564f2382d880 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.371+0000 7f9624624740 1 -- 172.21.15.103:0/4155055448 >> 172.21.15.103:0/4155055448 conn(0x564f23829a30 msgr2=0x564f23829e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:18.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.371+0000 7f9624624740 1 -- 172.21.15.103:0/4155055448 shutdown_connections 2024-09-19T06:59:18.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.371+0000 7f9624624740 1 -- 172.21.15.103:0/4155055448 wait complete. 2024-09-19T06:59:18.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.371+0000 7f9624624740 1 -- start start 2024-09-19T06:59:18.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.371+0000 7f9624624740 1 Processor -- start 2024-09-19T06:59:18.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f9624624740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f2382d4a0 0x564f23837f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f9624624740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382ddc0 0x564f238351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f9624624740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f23838680 0x564f23835730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f9624624740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564f2383b250 con 0x564f2382ddc0 2024-09-19T06:59:18.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f9624624740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564f2374e050 con 0x564f23838680 2024-09-19T06:59:18.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f9624624740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564f2383b3d0 con 0x564f2382d4a0 2024-09-19T06:59:18.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f9624624740 10 client.0 _ll_register_callbacks cb 0x564f2378e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:18.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f961affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382ddc0 0x564f238351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:18.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f961affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382ddc0 0x564f238351f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60998/0 (socket says 172.21.15.103:60998) 2024-09-19T06:59:18.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f961affd640 1 -- 172.21.15.103:0/2423950098 learned_addr learned my addr 172.21.15.103:0/2423950098 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:18.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f961bfff640 1 --2- 172.21.15.103:0/2423950098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f23838680 0x564f23835730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:18.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f961b7fe640 1 --2- 172.21.15.103:0/2423950098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f2382d4a0 0x564f23837f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:18.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f961affd640 1 -- 172.21.15.103:0/2423950098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f2382d4a0 msgr2=0x564f23837f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:18.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f961affd640 1 --2- 172.21.15.103:0/2423950098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f2382d4a0 0x564f23837f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f961affd640 1 -- 172.21.15.103:0/2423950098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f23838680 msgr2=0x564f23835730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:18.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f961affd640 1 --2- 172.21.15.103:0/2423950098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f23838680 0x564f23835730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f961affd640 1 -- 172.21.15.103:0/2423950098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f235043c0 con 0x564f2382ddc0 2024-09-19T06:59:18.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f961bfff640 1 --2- 172.21.15.103:0/2423950098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f23838680 0x564f23835730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:18.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f961affd640 1 --2- 172.21.15.103:0/2423950098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382ddc0 0x564f238351f0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f961009be80 tx=0x7f9610018e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab176d25f1a2bbac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:18.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f95faffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:18.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f95faffd640 1 -- 172.21.15.103:0/2423950098 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961009c180 con 0x564f2382ddc0 2024-09-19T06:59:18.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f95faffd640 1 -- 172.21.15.103:0/2423950098 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f96100a0170 con 0x564f2382ddc0 2024-09-19T06:59:18.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.372+0000 7f9624624740 1 -- 172.21.15.103:0/2423950098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f960c01a940 con 0x564f2382ddc0 2024-09-19T06:59:18.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.373+0000 7f95faffd640 1 -- 172.21.15.103:0/2423950098 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f96100a0530 con 0x564f2382ddc0 2024-09-19T06:59:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.373+0000 7f9624624740 10 client.18936 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.373+0000 7f9624624740 1 -- 172.21.15.103:0/2423950098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564f23846b10 con 0x564f2382ddc0 2024-09-19T06:59:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.373+0000 7f95faffd640 1 -- 172.21.15.103:0/2423950098 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f960c01a940 con 0x564f2382ddc0 2024-09-19T06:59:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.373+0000 7f95faffd640 1 -- 172.21.15.103:0/2423950098 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f96100cd870 con 0x564f2382ddc0 2024-09-19T06:59:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.373+0000 7f9624624740 10 client.18936 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.373+0000 7f9624624740 10 client.18936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.373+0000 7f9624624740 20 client.18936 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.373+0000 7f9624624740 1 --2- 172.21.15.103:0/2423950098 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564f23847f20 0x564f2384a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.373+0000 7f9624624740 4 client.18936 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.373+0000 7f9624624740 1 -- 172.21.15.103:0/2423950098 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f237a0850 con 0x564f23847f20 2024-09-19T06:59:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.373+0000 7f961b7fe640 1 --2- 172.21.15.103:0/2423950098 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564f23847f20 0x564f2384a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.374+0000 7f95faffd640 10 client.18936 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.374+0000 7f961b7fe640 1 --2- 172.21.15.103:0/2423950098 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564f23847f20 0x564f2384a350 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=719018e89067e970 server_cookie=9b5e2cc69a253170 in_seq=0 out_seq=0 2024-09-19T06:59:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.374+0000 7f95faffd640 1 -- 172.21.15.103:0/2423950098 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f237a0850 con 0x564f23847f20 2024-09-19T06:59:18.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.374+0000 7f95faffd640 10 client.18936 handle_command_reply: tid=0 2024-09-19T06:59:18.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.375+0000 7f9624624740 1 client.18936 shutdown 2024-09-19T06:59:18.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.375+0000 7f9624624740 1 -- 172.21.15.103:0/2423950098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382ddc0 msgr2=0x564f238351f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:18.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.375+0000 7f9624624740 1 --2- 172.21.15.103:0/2423950098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382ddc0 0x564f238351f0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f961009be80 tx=0x7f9610018e20 comp rx=0 tx=0).stop 2024-09-19T06:59:18.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.375+0000 7f9624624740 1 -- 172.21.15.103:0/2423950098 shutdown_connections 2024-09-19T06:59:18.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.375+0000 7f9624624740 1 --2- 172.21.15.103:0/2423950098 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564f23847f20 0x564f2384a350 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.375+0000 7f9624624740 1 --2- 172.21.15.103:0/2423950098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f23838680 0x564f23835730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.375+0000 7f9624624740 1 --2- 172.21.15.103:0/2423950098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382ddc0 0x564f238351f0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.375+0000 7f95faffd640 0 client.18936 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:18.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.375+0000 7f9624624740 1 --2- 172.21.15.103:0/2423950098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f2382d4a0 0x564f23837f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.375+0000 7f9624624740 1 -- 172.21.15.103:0/2423950098 >> 172.21.15.103:0/2423950098 conn(0x564f23829a30 msgr2=0x564f23828b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:18.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.375+0000 7f9624624740 1 -- 172.21.15.103:0/2423950098 shutdown_connections 2024-09-19T06:59:18.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.375+0000 7f9624624740 1 -- 172.21.15.103:0/2423950098 wait complete. 2024-09-19T06:59:18.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.375+0000 7f9624624740 20 client.18936 trim_cache size 0 max 16384 2024-09-19T06:59:18.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f9624624740 1 Processor -- start 2024-09-19T06:59:18.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f9624624740 1 -- start start 2024-09-19T06:59:18.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f9624624740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382d4a0 0x564f238444b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f9624624740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f2382ddc0 0x564f23837160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f9624624740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f23838680 0x564f2384fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f9624624740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f961c1afa00 con 0x564f2382d4a0 2024-09-19T06:59:18.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f9624624740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f961c1b1540 con 0x564f23838680 2024-09-19T06:59:18.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f9624624740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f961c1b00d0 con 0x564f2382ddc0 2024-09-19T06:59:18.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f961b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382d4a0 0x564f238444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:18.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f961b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382d4a0 0x564f238444b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32782/0 (socket says 172.21.15.103:32782) 2024-09-19T06:59:18.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f961b7fe640 1 -- 172.21.15.103:0/441303286 learned_addr learned my addr 172.21.15.103:0/441303286 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:18.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f961b7fe640 1 -- 172.21.15.103:0/441303286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f2382ddc0 msgr2=0x564f23837160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:18.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f961b7fe640 1 --2- 172.21.15.103:0/441303286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f2382ddc0 0x564f23837160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f961b7fe640 1 -- 172.21.15.103:0/441303286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f23838680 msgr2=0x564f2384fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:18.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f961b7fe640 1 --2- 172.21.15.103:0/441303286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f23838680 0x564f2384fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f961b7fe640 1 -- 172.21.15.103:0/441303286 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2382dbf0 con 0x564f2382d4a0 2024-09-19T06:59:18.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.392+0000 7f961b7fe640 1 --2- 172.21.15.103:0/441303286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382d4a0 0x564f238444b0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f960c0115b0 tx=0x7f960c0115e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902b2fe2c5ff600b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:18.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.393+0000 7f96017fa640 1 -- 172.21.15.103:0/441303286 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c015050 con 0x564f2382d4a0 2024-09-19T06:59:18.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.393+0000 7f96017fa640 1 -- 172.21.15.103:0/441303286 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f960c005520 con 0x564f2382d4a0 2024-09-19T06:59:18.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.393+0000 7f96017fa640 1 -- 172.21.15.103:0/441303286 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c011b30 con 0x564f2382d4a0 2024-09-19T06:59:18.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.393+0000 7f9624624740 1 -- 172.21.15.103:0/441303286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382d4a0 msgr2=0x564f238444b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:18.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.393+0000 7f9624624740 1 --2- 172.21.15.103:0/441303286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382d4a0 0x564f238444b0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f960c0115b0 tx=0x7f960c0115e0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.393+0000 7f9624624740 1 -- 172.21.15.103:0/441303286 shutdown_connections 2024-09-19T06:59:18.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.393+0000 7f9624624740 1 --2- 172.21.15.103:0/441303286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f23838680 0x564f2384fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.393+0000 7f9624624740 1 --2- 172.21.15.103:0/441303286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f2382ddc0 0x564f23837160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.393+0000 7f9624624740 1 --2- 172.21.15.103:0/441303286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382d4a0 0x564f238444b0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.393+0000 7f9624624740 1 -- 172.21.15.103:0/441303286 >> 172.21.15.103:0/441303286 conn(0x564f23829a30 msgr2=0x564f23836a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:18.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.393+0000 7f9624624740 1 -- 172.21.15.103:0/441303286 shutdown_connections 2024-09-19T06:59:18.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.393+0000 7f9624624740 1 -- 172.21.15.103:0/441303286 wait complete. 2024-09-19T06:59:18.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.394+0000 7f9624624740 1 -- start start 2024-09-19T06:59:18.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.394+0000 7f9624624740 1 Processor -- start 2024-09-19T06:59:18.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.394+0000 7f9624624740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f2382d4a0 0x564f2383c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.394+0000 7f9624624740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382ddc0 0x564f2383cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.394+0000 7f9624624740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f23838680 0x564f238b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.394+0000 7f9624624740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f961c1b0e70 con 0x564f2382ddc0 2024-09-19T06:59:18.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.394+0000 7f9624624740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f961c1b07a0 con 0x564f2382d4a0 2024-09-19T06:59:18.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.394+0000 7f9624624740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f961c1ada00 con 0x564f23838680 2024-09-19T06:59:18.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.394+0000 7f9624624740 10 client.0 _ll_register_callbacks cb 0x7f961000cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:18.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.394+0000 7f961affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382ddc0 0x564f2383cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:18.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.394+0000 7f961affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382ddc0 0x564f2383cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32788/0 (socket says 172.21.15.103:32788) 2024-09-19T06:59:18.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.394+0000 7f961affd640 1 -- 172.21.15.103:0/198741575 learned_addr learned my addr 172.21.15.103:0/198741575 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:18.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f961b7fe640 1 --2- 172.21.15.103:0/198741575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f2382d4a0 0x564f2383c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:18.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f961bfff640 1 --2- 172.21.15.103:0/198741575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f23838680 0x564f238b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:18.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f961affd640 1 -- 172.21.15.103:0/198741575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f23838680 msgr2=0x564f238b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:18.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f961affd640 1 --2- 172.21.15.103:0/198741575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f23838680 0x564f238b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f961affd640 1 -- 172.21.15.103:0/198741575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f2382d4a0 msgr2=0x564f2383c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:18.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f961affd640 1 --2- 172.21.15.103:0/198741575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f2382d4a0 0x564f2383c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f961affd640 1 -- 172.21.15.103:0/198741575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f238b2130 con 0x564f2382ddc0 2024-09-19T06:59:18.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f961bfff640 1 --2- 172.21.15.103:0/198741575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f23838680 0x564f238b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:18.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f961affd640 1 --2- 172.21.15.103:0/198741575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382ddc0 0x564f2383cf00 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f9610007590 tx=0x7f961009bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b55638598a83ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f95faffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f95faffd640 1 -- 172.21.15.103:0/198741575 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610098420 con 0x564f2382ddc0 2024-09-19T06:59:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f9624624740 1 -- 172.21.15.103:0/198741575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f960c005520 con 0x564f2382ddc0 2024-09-19T06:59:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f95faffd640 1 -- 172.21.15.103:0/198741575 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f96100bc220 con 0x564f2382ddc0 2024-09-19T06:59:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f95faffd640 1 -- 172.21.15.103:0/198741575 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f96100cda80 con 0x564f2382ddc0 2024-09-19T06:59:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f9624624740 10 client.18942 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f9624624740 1 -- 172.21.15.103:0/198741575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564f238b3340 con 0x564f2382ddc0 2024-09-19T06:59:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.395+0000 7f95faffd640 1 -- 172.21.15.103:0/198741575 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f960c005520 con 0x564f2382ddc0 2024-09-19T06:59:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.396+0000 7f95faffd640 1 -- 172.21.15.103:0/198741575 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9610010000 con 0x564f2382ddc0 2024-09-19T06:59:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.396+0000 7f9624624740 10 client.18942 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.396+0000 7f9624624740 10 client.18942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.396+0000 7f9624624740 20 client.18942 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.396+0000 7f9624624740 1 --2- 172.21.15.103:0/198741575 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564f238b3a50 0x564f238b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.396+0000 7f9624624740 4 client.18942 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:59:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.396+0000 7f9624624740 1 -- 172.21.15.103:0/198741575 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564f2379ad60 con 0x564f238b3a50 2024-09-19T06:59:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.396+0000 7f961b7fe640 1 --2- 172.21.15.103:0/198741575 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564f238b3a50 0x564f238b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.396+0000 7f95faffd640 10 client.18942 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.396+0000 7f961b7fe640 1 --2- 172.21.15.103:0/198741575 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564f238b3a50 0x564f238b5e40 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=137f3cbc3edc59a4 server_cookie=231319b069ab2426 in_seq=0 out_seq=0 2024-09-19T06:59:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.396+0000 7f95faffd640 1 -- 172.21.15.103:0/198741575 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564f2379ad60 con 0x564f238b3a50 2024-09-19T06:59:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.396+0000 7f95faffd640 10 client.18942 handle_command_reply: tid=0 2024-09-19T06:59:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.396+0000 7f9624624740 1 client.18942 shutdown 2024-09-19T06:59:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f9624624740 1 -- 172.21.15.103:0/198741575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382ddc0 msgr2=0x564f2383cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f9624624740 1 --2- 172.21.15.103:0/198741575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382ddc0 0x564f2383cf00 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f9610007590 tx=0x7f961009bbb0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f9624624740 1 -- 172.21.15.103:0/198741575 shutdown_connections 2024-09-19T06:59:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f9624624740 1 --2- 172.21.15.103:0/198741575 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564f238b3a50 0x564f238b5e40 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f9624624740 1 --2- 172.21.15.103:0/198741575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f23838680 0x564f238b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f9624624740 1 --2- 172.21.15.103:0/198741575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2382ddc0 0x564f2383cf00 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f95faffd640 0 client.18942 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:18.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f9624624740 1 --2- 172.21.15.103:0/198741575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f2382d4a0 0x564f2383c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f9624624740 1 -- 172.21.15.103:0/198741575 >> 172.21.15.103:0/198741575 conn(0x564f23829a30 msgr2=0x564f2382eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:18.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f9624624740 1 -- 172.21.15.103:0/198741575 shutdown_connections 2024-09-19T06:59:18.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f9624624740 1 -- 172.21.15.103:0/198741575 wait complete. 2024-09-19T06:59:18.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f9624624740 20 client.18942 trim_cache size 0 max 16384 2024-09-19T06:59:18.408 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:59:18.408 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:59:18.408 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T06:59:18.408 INFO:teuthology.orchestra.run.smithi103.stdout: "72bdeff4-2204-4c05-a1cb-97e8da6f7bc9": { 2024-09-19T06:59:18.408 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T06:59:18.409 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "72bdeff4-2204-4c05-a1cb-97e8da6f7bc9", 2024-09-19T06:59:18.409 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T06:59:18.409 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:18.409 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:18.409 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:59:18.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f962224b640 1 -- 172.21.15.103:0/1422051293 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96040a33b0 msgr2=0x7f96040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:18.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f962224b640 1 --2- 172.21.15.103:0/1422051293 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96040a33b0 0x7f96040a5870 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9614006010 tx=0x7f9614031bb0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f962224b640 1 -- 172.21.15.103:0/1422051293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f961c102790 msgr2=0x7f961c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:18.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f962224b640 1 --2- 172.21.15.103:0/1422051293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f961c102790 0x7f961c074a30 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f961000c920 tx=0x7f961000cde0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f962224b640 1 -- 172.21.15.103:0/1422051293 shutdown_connections 2024-09-19T06:59:18.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f962224b640 1 --2- 172.21.15.103:0/1422051293 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96040a33b0 0x7f96040a5870 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f962224b640 1 --2- 172.21.15.103:0/1422051293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f961c106f10 0x7f961c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f962224b640 1 --2- 172.21.15.103:0/1422051293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f961c102790 0x7f961c074a30 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f962224b640 1 --2- 172.21.15.103:0/1422051293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f961c101de0 0x7f961c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:18.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.397+0000 7f962224b640 1 -- 172.21.15.103:0/1422051293 >> 172.21.15.103:0/1422051293 conn(0x7f961c0fd8c0 msgr2=0x7f961c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:18.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.398+0000 7f962224b640 1 -- 172.21.15.103:0/1422051293 shutdown_connections 2024-09-19T06:59:18.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:18.398+0000 7f962224b640 1 -- 172.21.15.103:0/1422051293 wait complete. 2024-09-19T06:59:19.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:18 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T06:59:19.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:18 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:59:19.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:18 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T06:59:19.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:18 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T06:59:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:18 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T06:59:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:18 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:59:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:18 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T06:59:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:18 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T06:59:19.114 INFO:tasks.cephfs.filesystem:scrub status for tag:72bdeff4-2204-4c05-a1cb-97e8da6f7bc9 - {'path': '/', 'tag': '72bdeff4-2204-4c05-a1cb-97e8da6f7bc9', 'options': 'recursive,force'} 2024-09-19T06:59:19.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:18 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T06:59:19.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:18 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:59:19.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:18 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T06:59:19.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:18 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T06:59:19.984 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:19 smithi174 ceph-mon[35637]: pgmap v313: 97 pgs: 97 active+clean; 1.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-19T06:59:20.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:19 smithi103 ceph-mon[31158]: pgmap v313: 97 pgs: 97 active+clean; 1.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-19T06:59:20.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:19 smithi134 ceph-mon[35356]: pgmap v313: 97 pgs: 97 active+clean; 1.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-19T06:59:21.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:20 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T06:59:21.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:20 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T06:59:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:20 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T06:59:22.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:21 smithi174 ceph-mon[35637]: pgmap v314: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 KiB/s wr, 15 op/s 2024-09-19T06:59:22.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:21 smithi103 ceph-mon[31158]: pgmap v314: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 KiB/s wr, 15 op/s 2024-09-19T06:59:22.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:21 smithi134 ceph-mon[35356]: pgmap v314: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 KiB/s wr, 15 op/s 2024-09-19T06:59:24.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:23 smithi174 ceph-mon[35637]: pgmap v315: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 14 op/s 2024-09-19T06:59:24.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:23 smithi103 ceph-mon[31158]: pgmap v315: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 14 op/s 2024-09-19T06:59:24.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:23 smithi134 ceph-mon[35356]: pgmap v315: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 14 op/s 2024-09-19T06:59:26.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:25 smithi174 ceph-mon[35637]: pgmap v316: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.0 KiB/s wr, 12 op/s 2024-09-19T06:59:26.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:25 smithi103 ceph-mon[31158]: pgmap v316: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.0 KiB/s wr, 12 op/s 2024-09-19T06:59:26.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:25 smithi134 ceph-mon[35356]: pgmap v316: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.0 KiB/s wr, 12 op/s 2024-09-19T06:59:28.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:27 smithi174 ceph-mon[35637]: pgmap v317: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 16 op/s 2024-09-19T06:59:28.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:27 smithi103 ceph-mon[31158]: pgmap v317: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 16 op/s 2024-09-19T06:59:28.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:27 smithi134 ceph-mon[35356]: pgmap v317: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 16 op/s 2024-09-19T06:59:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.644+0000 7ff3f9cba640 1 -- 172.21.15.103:0/527861070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f4103f40 msgr2=0x7ff3f4104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.644+0000 7ff3f9cba640 1 --2- 172.21.15.103:0/527861070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f4103f40 0x7ff3f4104320 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009900 tx=0x7ff3e402a410 comp rx=0 tx=0).stop 2024-09-19T06:59:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.644+0000 7ff3f9cba640 1 -- 172.21.15.103:0/527861070 shutdown_connections 2024-09-19T06:59:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.644+0000 7ff3f9cba640 1 --2- 172.21.15.103:0/527861070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f4109090 0x7ff3f4110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.644+0000 7ff3f9cba640 1 --2- 172.21.15.103:0/527861070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f41048f0 0x7ff3f4108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.644+0000 7ff3f9cba640 1 --2- 172.21.15.103:0/527861070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f4103f40 0x7ff3f4104320 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.644+0000 7ff3f9cba640 1 -- 172.21.15.103:0/527861070 >> 172.21.15.103:0/527861070 conn(0x7ff3f40ff9b0 msgr2=0x7ff3f4101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.644+0000 7ff3f9cba640 1 -- 172.21.15.103:0/527861070 shutdown_connections 2024-09-19T06:59:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.644+0000 7ff3f9cba640 1 -- 172.21.15.103:0/527861070 wait complete. 2024-09-19T06:59:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.645+0000 7ff3f9cba640 1 Processor -- start 2024-09-19T06:59:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.645+0000 7ff3f9cba640 1 -- start start 2024-09-19T06:59:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.645+0000 7ff3f9cba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f4103f40 0x7ff3f419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.645+0000 7ff3f9cba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f4109090 0x7ff3f419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.645+0000 7ff3f9cba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f419ee50 0x7ff3f419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.645+0000 7ff3f9cba640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41140e0 con 0x7ff3f4109090 2024-09-19T06:59:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.645+0000 7ff3f9cba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f4113f60 con 0x7ff3f419ee50 2024-09-19T06:59:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.645+0000 7ff3f9cba640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f4114260 con 0x7ff3f4103f40 2024-09-19T06:59:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.645+0000 7ff3f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f4109090 0x7ff3f419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.645+0000 7ff3f37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f4103f40 0x7ff3f419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.645+0000 7ff3f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f4109090 0x7ff3f419aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60560/0 (socket says 172.21.15.103:60560) 2024-09-19T06:59:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.645+0000 7ff3f2ffd640 1 -- 172.21.15.103:0/3312687562 learned_addr learned my addr 172.21.15.103:0/3312687562 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.645+0000 7ff3f3fff640 1 --2- 172.21.15.103:0/3312687562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f419ee50 0x7ff3f419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.646+0000 7ff3f2ffd640 1 -- 172.21.15.103:0/3312687562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f4103f40 msgr2=0x7ff3f419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.646+0000 7ff3f2ffd640 1 --2- 172.21.15.103:0/3312687562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f4103f40 0x7ff3f419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.646+0000 7ff3f2ffd640 1 -- 172.21.15.103:0/3312687562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f419ee50 msgr2=0x7ff3f419f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.646+0000 7ff3f2ffd640 1 --2- 172.21.15.103:0/3312687562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f419ee50 0x7ff3f419f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.646+0000 7ff3f2ffd640 1 -- 172.21.15.103:0/3312687562 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f419f940 con 0x7ff3f4109090 2024-09-19T06:59:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.646+0000 7ff3f2ffd640 1 --2- 172.21.15.103:0/3312687562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f4109090 0x7ff3f419aad0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800e9f0 tx=0x7ff3e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.646+0000 7ff3f0ff9640 1 -- 172.21.15.103:0/3312687562 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e800ccc0 con 0x7ff3f4109090 2024-09-19T06:59:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.646+0000 7ff3f9cba640 1 -- 172.21.15.103:0/3312687562 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f419fc30 con 0x7ff3f4109090 2024-09-19T06:59:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.646+0000 7ff3f0ff9640 1 -- 172.21.15.103:0/3312687562 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e8005500 con 0x7ff3f4109090 2024-09-19T06:59:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.646+0000 7ff3f0ff9640 1 -- 172.21.15.103:0/3312687562 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8010690 con 0x7ff3f4109090 2024-09-19T06:59:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.646+0000 7ff3f9cba640 1 -- 172.21.15.103:0/3312687562 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f41a73d0 con 0x7ff3f4109090 2024-09-19T06:59:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.648+0000 7ff3f0ff9640 1 -- 172.21.15.103:0/3312687562 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff3e80046c0 con 0x7ff3f4109090 2024-09-19T06:59:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.648+0000 7ff3fca59740 1 Processor -- start 2024-09-19T06:59:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.648+0000 7ff3fca59740 1 -- start start 2024-09-19T06:59:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3fca59740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e2af0 0x56349b7e2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3fca59740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e3410 0x56349b7ed770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3fca59740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7edcb0 0x56349b7f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3fca59740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56349b7472f0 con 0x56349b7e3410 2024-09-19T06:59:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3fca59740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56349b6ef270 con 0x56349b7e2af0 2024-09-19T06:59:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3fca59740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56349b7432b0 con 0x56349b7edcb0 2024-09-19T06:59:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3f37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e3410 0x56349b7ed770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3f37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e3410 0x56349b7ed770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60574/0 (socket says 172.21.15.103:60574) 2024-09-19T06:59:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3f37fe640 1 -- 172.21.15.103:0/3851672250 learned_addr learned my addr 172.21.15.103:0/3851672250 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3f2ffd640 1 --2- 172.21.15.103:0/3851672250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7edcb0 0x56349b7f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3f3fff640 1 --2- 172.21.15.103:0/3851672250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e2af0 0x56349b7e2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3f37fe640 1 -- 172.21.15.103:0/3851672250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7edcb0 msgr2=0x56349b7f00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3f37fe640 1 --2- 172.21.15.103:0/3851672250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7edcb0 0x56349b7f00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3f37fe640 1 -- 172.21.15.103:0/3851672250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e2af0 msgr2=0x56349b7e2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3f37fe640 1 --2- 172.21.15.103:0/3851672250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e2af0 0x56349b7e2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3f37fe640 1 -- 172.21.15.103:0/3851672250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56349b6ee340 con 0x56349b7e3410 2024-09-19T06:59:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3f3fff640 1 --2- 172.21.15.103:0/3851672250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e2af0 0x56349b7e2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:59:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3f37fe640 1 --2- 172.21.15.103:0/3851672250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e3410 0x56349b7ed770 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009900 tx=0x7ff3e4005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c965a7393d7a97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.649+0000 7ff3f0ff9640 1 --2- 172.21.15.103:0/3312687562 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3dc0a3460 0x7ff3dc0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.650+0000 7ff3f3fff640 1 --2- 172.21.15.103:0/3312687562 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3dc0a3460 0x7ff3dc0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.650+0000 7ff3d9ffb640 1 -- 172.21.15.103:0/3851672250 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4040070 con 0x56349b7e3410 2024-09-19T06:59:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.650+0000 7ff3f0ff9640 1 -- 172.21.15.103:0/3312687562 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7ff3e80d0130 con 0x7ff3f4109090 2024-09-19T06:59:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.650+0000 7ff3d9ffb640 1 -- 172.21.15.103:0/3851672250 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e4031870 con 0x56349b7e3410 2024-09-19T06:59:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.650+0000 7ff3d9ffb640 1 -- 172.21.15.103:0/3851672250 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4035040 con 0x56349b7e3410 2024-09-19T06:59:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.650+0000 7ff3fca59740 1 -- 172.21.15.103:0/3851672250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e3410 msgr2=0x56349b7ed770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.650+0000 7ff3f3fff640 1 --2- 172.21.15.103:0/3312687562 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3dc0a3460 0x7ff3dc0a5920 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009fd0 tx=0x7ff3ec009500 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.650+0000 7ff3fca59740 1 --2- 172.21.15.103:0/3851672250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e3410 0x56349b7ed770 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009900 tx=0x7ff3e4005be0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.650+0000 7ff3fca59740 1 -- 172.21.15.103:0/3851672250 shutdown_connections 2024-09-19T06:59:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.650+0000 7ff3fca59740 1 --2- 172.21.15.103:0/3851672250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7edcb0 0x56349b7f00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.650+0000 7ff3fca59740 1 --2- 172.21.15.103:0/3851672250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e3410 0x56349b7ed770 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.650+0000 7ff3fca59740 1 --2- 172.21.15.103:0/3851672250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e2af0 0x56349b7e2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.650+0000 7ff3fca59740 1 -- 172.21.15.103:0/3851672250 >> 172.21.15.103:0/3851672250 conn(0x56349b7df060 msgr2=0x56349b7df440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.650+0000 7ff3fca59740 1 -- 172.21.15.103:0/3851672250 shutdown_connections 2024-09-19T06:59:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.650+0000 7ff3fca59740 1 -- 172.21.15.103:0/3851672250 wait complete. 2024-09-19T06:59:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.651+0000 7ff3fca59740 1 -- start start 2024-09-19T06:59:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.651+0000 7ff3fca59740 1 Processor -- start 2024-09-19T06:59:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.651+0000 7ff3fca59740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e2af0 0x56349b7eab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.651+0000 7ff3fca59740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e3410 0x56349b7eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.651+0000 7ff3fca59740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7edcb0 0x56349b7eb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.651+0000 7ff3fca59740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56349b7f0880 con 0x56349b7e2af0 2024-09-19T06:59:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.651+0000 7ff3fca59740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56349b6d0290 con 0x56349b7e3410 2024-09-19T06:59:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.651+0000 7ff3fca59740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56349b7f0a00 con 0x56349b7edcb0 2024-09-19T06:59:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.651+0000 7ff3fca59740 10 client.0 _ll_register_callbacks cb 0x56349b701140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.651+0000 7ff3f37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e2af0 0x56349b7eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.651+0000 7ff3f37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e2af0 0x56349b7eab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60582/0 (socket says 172.21.15.103:60582) 2024-09-19T06:59:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.651+0000 7ff3f37fe640 1 -- 172.21.15.103:0/299058632 learned_addr learned my addr 172.21.15.103:0/299058632 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.651+0000 7ff3f3fff640 1 --2- 172.21.15.103:0/299058632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7edcb0 0x56349b7eb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.652+0000 7ff3f2ffd640 1 --2- 172.21.15.103:0/299058632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e3410 0x56349b7eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.652+0000 7ff3f37fe640 1 -- 172.21.15.103:0/299058632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7edcb0 msgr2=0x56349b7eb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.652+0000 7ff3f37fe640 1 --2- 172.21.15.103:0/299058632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7edcb0 0x56349b7eb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.652+0000 7ff3f37fe640 1 -- 172.21.15.103:0/299058632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e3410 msgr2=0x56349b7eb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.652+0000 7ff3f37fe640 1 --2- 172.21.15.103:0/299058632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e3410 0x56349b7eb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.652+0000 7ff3f37fe640 1 -- 172.21.15.103:0/299058632 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56349b45a210 con 0x56349b7e2af0 2024-09-19T06:59:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.652+0000 7ff3f37fe640 1 --2- 172.21.15.103:0/299058632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e2af0 0x56349b7eab60 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4000c00 tx=0x7ff3e402ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4b83351dfa65b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.652+0000 7ff3d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.652+0000 7ff3d2ffd640 1 -- 172.21.15.103:0/299058632 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e404d060 con 0x56349b7e2af0 2024-09-19T06:59:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.652+0000 7ff3d2ffd640 1 -- 172.21.15.103:0/299058632 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7ff3e4004240 con 0x56349b7e2af0 2024-09-19T06:59:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.652+0000 7ff3fca59740 1 -- 172.21.15.103:0/299058632 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3e4031870 con 0x56349b7e2af0 2024-09-19T06:59:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.652+0000 7ff3d2ffd640 1 -- 172.21.15.103:0/299058632 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff3e4040070 con 0x56349b7e2af0 2024-09-19T06:59:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.652+0000 7ff3fca59740 10 client.18954 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.652+0000 7ff3fca59740 1 -- 172.21.15.103:0/299058632 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56349b526620 con 0x56349b7e2af0 2024-09-19T06:59:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.652+0000 7ff3d2ffd640 1 -- 172.21.15.103:0/299058632 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e4031870 con 0x56349b7e2af0 2024-09-19T06:59:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.653+0000 7ff3d2ffd640 1 -- 172.21.15.103:0/299058632 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff3e40035f0 con 0x56349b7e2af0 2024-09-19T06:59:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.653+0000 7ff3fca59740 10 client.18954 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.653+0000 7ff3fca59740 10 client.18954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.653+0000 7ff3fca59740 20 client.18954 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.653+0000 7ff3fca59740 1 --2- 172.21.15.103:0/299058632 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56349b7fda50 0x56349b7ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.653+0000 7ff3fca59740 4 client.18954 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.653+0000 7ff3fca59740 1 -- 172.21.15.103:0/299058632 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56349b754a20 con 0x56349b7fda50 2024-09-19T06:59:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.653+0000 7ff3f2ffd640 1 --2- 172.21.15.103:0/299058632 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56349b7fda50 0x56349b7ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.653+0000 7ff3f2ffd640 1 --2- 172.21.15.103:0/299058632 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56349b7fda50 0x56349b7ffe40 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fbf5394f515c710 server_cookie=e87b5af2d3362233 in_seq=0 out_seq=0 2024-09-19T06:59:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.653+0000 7ff3d2ffd640 10 client.18954 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.654+0000 7ff3d2ffd640 1 -- 172.21.15.103:0/299058632 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56349b754a20 con 0x56349b7fda50 2024-09-19T06:59:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.654+0000 7ff3d2ffd640 10 client.18954 handle_command_reply: tid=0 2024-09-19T06:59:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.654+0000 7ff3fca59740 1 client.18954 shutdown 2024-09-19T06:59:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.654+0000 7ff3fca59740 1 -- 172.21.15.103:0/299058632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e2af0 msgr2=0x56349b7eab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.654+0000 7ff3fca59740 1 --2- 172.21.15.103:0/299058632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e2af0 0x56349b7eab60 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4000c00 tx=0x7ff3e402ab10 comp rx=0 tx=0).stop 2024-09-19T06:59:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.654+0000 7ff3fca59740 1 -- 172.21.15.103:0/299058632 shutdown_connections 2024-09-19T06:59:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.654+0000 7ff3fca59740 1 --2- 172.21.15.103:0/299058632 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56349b7fda50 0x56349b7ffe40 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.654+0000 7ff3fca59740 1 --2- 172.21.15.103:0/299058632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7edcb0 0x56349b7eb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.654+0000 7ff3fca59740 1 --2- 172.21.15.103:0/299058632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e3410 0x56349b7eb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.654+0000 7ff3d2ffd640 0 client.18954 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.654+0000 7ff3fca59740 1 --2- 172.21.15.103:0/299058632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e2af0 0x56349b7eab60 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.654+0000 7ff3fca59740 1 -- 172.21.15.103:0/299058632 >> 172.21.15.103:0/299058632 conn(0x56349b7df060 msgr2=0x56349b7e4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.654+0000 7ff3fca59740 1 -- 172.21.15.103:0/299058632 shutdown_connections 2024-09-19T06:59:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.654+0000 7ff3fca59740 1 -- 172.21.15.103:0/299058632 wait complete. 2024-09-19T06:59:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.654+0000 7ff3fca59740 20 client.18954 trim_cache size 0 max 16384 2024-09-19T06:59:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3fca59740 1 Processor -- start 2024-09-19T06:59:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3fca59740 1 -- start start 2024-09-19T06:59:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3fca59740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e2af0 0x56349b7eb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3fca59740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7e3410 0x56349b802e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3fca59740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7edcb0 0x56349b8033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3fca59740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41af9f0 con 0x56349b7e2af0 2024-09-19T06:59:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3fca59740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41b1530 con 0x56349b7edcb0 2024-09-19T06:59:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3fca59740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41b00c0 con 0x56349b7e3410 2024-09-19T06:59:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3f37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e2af0 0x56349b7eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3f37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e2af0 0x56349b7eb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60586/0 (socket says 172.21.15.103:60586) 2024-09-19T06:59:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3f37fe640 1 -- 172.21.15.103:0/3122299377 learned_addr learned my addr 172.21.15.103:0/3122299377 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3f3fff640 1 --2- 172.21.15.103:0/3122299377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7edcb0 0x56349b8033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3f2ffd640 1 --2- 172.21.15.103:0/3122299377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7e3410 0x56349b802e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3f37fe640 1 -- 172.21.15.103:0/3122299377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7e3410 msgr2=0x56349b802e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3f37fe640 1 --2- 172.21.15.103:0/3122299377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7e3410 0x56349b802e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3f37fe640 1 -- 172.21.15.103:0/3122299377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7edcb0 msgr2=0x56349b8033a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3f37fe640 1 --2- 172.21.15.103:0/3122299377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7edcb0 0x56349b8033a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.669+0000 7ff3f37fe640 1 -- 172.21.15.103:0/3122299377 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56349b7e5940 con 0x56349b7e2af0 2024-09-19T06:59:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.670+0000 7ff3f37fe640 1 --2- 172.21.15.103:0/3122299377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e2af0 0x56349b7eb410 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4004000 tx=0x7ff3e4005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8578dbe16d9fd33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.670+0000 7ff3d97fa640 1 -- 172.21.15.103:0/3122299377 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e404d070 con 0x56349b7e2af0 2024-09-19T06:59:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.670+0000 7ff3d97fa640 1 -- 172.21.15.103:0/3122299377 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e40035f0 con 0x56349b7e2af0 2024-09-19T06:59:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.670+0000 7ff3d97fa640 1 -- 172.21.15.103:0/3122299377 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4035040 con 0x56349b7e2af0 2024-09-19T06:59:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.671+0000 7ff3fca59740 1 -- 172.21.15.103:0/3122299377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e2af0 msgr2=0x56349b7eb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.671+0000 7ff3fca59740 1 --2- 172.21.15.103:0/3122299377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e2af0 0x56349b7eb410 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4004000 tx=0x7ff3e4005230 comp rx=0 tx=0).stop 2024-09-19T06:59:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.671+0000 7ff3fca59740 1 -- 172.21.15.103:0/3122299377 shutdown_connections 2024-09-19T06:59:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.671+0000 7ff3fca59740 1 --2- 172.21.15.103:0/3122299377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7edcb0 0x56349b8033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.671+0000 7ff3fca59740 1 --2- 172.21.15.103:0/3122299377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7e3410 0x56349b802e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.671+0000 7ff3fca59740 1 --2- 172.21.15.103:0/3122299377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7e2af0 0x56349b7eb410 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.671+0000 7ff3fca59740 1 -- 172.21.15.103:0/3122299377 >> 172.21.15.103:0/3122299377 conn(0x56349b7df060 msgr2=0x56349b7ecbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.671+0000 7ff3fca59740 1 -- 172.21.15.103:0/3122299377 shutdown_connections 2024-09-19T06:59:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.671+0000 7ff3fca59740 1 -- 172.21.15.103:0/3122299377 wait complete. 2024-09-19T06:59:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.671+0000 7ff3fca59740 1 -- start start 2024-09-19T06:59:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.671+0000 7ff3fca59740 1 Processor -- start 2024-09-19T06:59:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3fca59740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e2af0 0x56349b8032f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3fca59740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7e3410 0x56349b800630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3fca59740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7edcb0 0x56349b800b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3fca59740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41b0e60 con 0x56349b7edcb0 2024-09-19T06:59:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3fca59740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41b0790 con 0x56349b7e2af0 2024-09-19T06:59:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3fca59740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41ad9f0 con 0x56349b7e3410 2024-09-19T06:59:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3fca59740 10 client.0 _ll_register_callbacks cb 0x7ff3ec009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3f3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7edcb0 0x56349b800b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3f37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e2af0 0x56349b8032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3f2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7e3410 0x56349b800630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3f37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e2af0 0x56349b8032f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:47966/0 (socket says 172.21.15.103:47966) 2024-09-19T06:59:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3f2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7e3410 0x56349b800630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40826/0 (socket says 172.21.15.103:40826) 2024-09-19T06:59:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3f3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7edcb0 0x56349b800b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60592/0 (socket says 172.21.15.103:60592) 2024-09-19T06:59:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3f37fe640 1 -- 172.21.15.103:0/887676730 learned_addr learned my addr 172.21.15.103:0/887676730 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3f37fe640 1 -- 172.21.15.103:0/887676730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7e3410 msgr2=0x56349b800630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3f37fe640 1 --2- 172.21.15.103:0/887676730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7e3410 0x56349b800630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3f37fe640 1 -- 172.21.15.103:0/887676730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7edcb0 msgr2=0x56349b800b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3f37fe640 1 --2- 172.21.15.103:0/887676730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7edcb0 0x56349b800b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3f37fe640 1 -- 172.21.15.103:0/887676730 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56349b8010b0 con 0x56349b7e2af0 2024-09-19T06:59:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.672+0000 7ff3f3fff640 1 --2- 172.21.15.103:0/887676730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7edcb0 0x56349b800b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:59:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.673+0000 7ff3f37fe640 1 --2- 172.21.15.103:0/887676730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e2af0 0x56349b8032f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff3e402a930 tx=0x7ff3e4003de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b2247832281b678 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.673+0000 7ff3d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T06:59:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.673+0000 7ff3d2ffd640 1 -- 172.21.15.103:0/887676730 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e404d070 con 0x56349b7e2af0 2024-09-19T06:59:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.673+0000 7ff3fca59740 1 -- 172.21.15.103:0/887676730 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3e40035f0 con 0x56349b7e2af0 2024-09-19T06:59:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.673+0000 7ff3d2ffd640 1 -- 172.21.15.103:0/887676730 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7ff3e4031880 con 0x56349b7e2af0 2024-09-19T06:59:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.673+0000 7ff3d2ffd640 1 -- 172.21.15.103:0/887676730 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff3e4035040 con 0x56349b7e2af0 2024-09-19T06:59:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.673+0000 7ff3fca59740 10 client.27257 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.673+0000 7ff3fca59740 1 -- 172.21.15.103:0/887676730 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56349b8680d0 con 0x56349b7e2af0 2024-09-19T06:59:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.673+0000 7ff3d2ffd640 1 -- 172.21.15.103:0/887676730 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e40035f0 con 0x56349b7e2af0 2024-09-19T06:59:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.673+0000 7ff3d2ffd640 1 -- 172.21.15.103:0/887676730 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff3e406b480 con 0x56349b7e2af0 2024-09-19T06:59:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.673+0000 7ff3fca59740 10 client.27257 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.673+0000 7ff3fca59740 10 client.27257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.673+0000 7ff3fca59740 20 client.27257 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.673+0000 7ff3fca59740 1 --2- 172.21.15.103:0/887676730 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56349b868730 0x56349b86ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.673+0000 7ff3fca59740 4 client.27257 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:59:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.673+0000 7ff3fca59740 1 -- 172.21.15.103:0/887676730 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56349b74ed70 con 0x56349b868730 2024-09-19T06:59:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.674+0000 7ff3f2ffd640 1 --2- 172.21.15.103:0/887676730 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56349b868730 0x56349b86ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.674+0000 7ff3d2ffd640 10 client.27257 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.674+0000 7ff3f2ffd640 1 --2- 172.21.15.103:0/887676730 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56349b868730 0x56349b86ab50 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9f7ccc6af6e2489 server_cookie=41ff4ec7b91e2676 in_seq=0 out_seq=0 2024-09-19T06:59:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.683+0000 7ff3d2ffd640 1 -- 172.21.15.103:0/887676730 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56349b74ed70 con 0x56349b868730 2024-09-19T06:59:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.683+0000 7ff3d2ffd640 10 client.27257 handle_command_reply: tid=0 2024-09-19T06:59:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.683+0000 7ff3fca59740 1 client.27257 shutdown 2024-09-19T06:59:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.683+0000 7ff3fca59740 1 -- 172.21.15.103:0/887676730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e2af0 msgr2=0x56349b8032f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.683+0000 7ff3fca59740 1 --2- 172.21.15.103:0/887676730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e2af0 0x56349b8032f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff3e402a930 tx=0x7ff3e4003de0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.683+0000 7ff3fca59740 1 -- 172.21.15.103:0/887676730 shutdown_connections 2024-09-19T06:59:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.683+0000 7ff3fca59740 1 --2- 172.21.15.103:0/887676730 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56349b868730 0x56349b86ab50 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.683+0000 7ff3fca59740 1 --2- 172.21.15.103:0/887676730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56349b7edcb0 0x56349b800b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.683+0000 7ff3d2ffd640 0 client.27257 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.683+0000 7ff3fca59740 1 --2- 172.21.15.103:0/887676730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56349b7e3410 0x56349b800630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.683+0000 7ff3fca59740 1 --2- 172.21.15.103:0/887676730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56349b7e2af0 0x56349b8032f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.683+0000 7ff3fca59740 1 -- 172.21.15.103:0/887676730 >> 172.21.15.103:0/887676730 conn(0x56349b7df060 msgr2=0x56349b7f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.683+0000 7ff3fca59740 1 -- 172.21.15.103:0/887676730 shutdown_connections 2024-09-19T06:59:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.683+0000 7ff3fca59740 1 -- 172.21.15.103:0/887676730 wait complete. 2024-09-19T06:59:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.683+0000 7ff3fca59740 20 client.27257 trim_cache size 0 max 16384 2024-09-19T06:59:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:59:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:59:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:59:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:59:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.684+0000 7ff3f9cba640 1 -- 172.21.15.103:0/3312687562 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3dc0a3460 msgr2=0x7ff3dc0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.684+0000 7ff3f9cba640 1 --2- 172.21.15.103:0/3312687562 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3dc0a3460 0x7ff3dc0a5920 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009fd0 tx=0x7ff3ec009500 comp rx=0 tx=0).stop 2024-09-19T06:59:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.684+0000 7ff3f9cba640 1 -- 172.21.15.103:0/3312687562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f4109090 msgr2=0x7ff3f419aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.684+0000 7ff3f9cba640 1 --2- 172.21.15.103:0/3312687562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f4109090 0x7ff3f419aad0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800e9f0 tx=0x7ff3e800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.684+0000 7ff3f9cba640 1 -- 172.21.15.103:0/3312687562 shutdown_connections 2024-09-19T06:59:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.684+0000 7ff3f9cba640 1 --2- 172.21.15.103:0/3312687562 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff3dc0a3460 0x7ff3dc0a5920 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.684+0000 7ff3f9cba640 1 --2- 172.21.15.103:0/3312687562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3f419ee50 0x7ff3f419f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.684+0000 7ff3f9cba640 1 --2- 172.21.15.103:0/3312687562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3f4109090 0x7ff3f419aad0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.684+0000 7ff3f9cba640 1 --2- 172.21.15.103:0/3312687562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f4103f40 0x7ff3f419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.684+0000 7ff3f9cba640 1 -- 172.21.15.103:0/3312687562 >> 172.21.15.103:0/3312687562 conn(0x7ff3f40ff9b0 msgr2=0x7ff3f4100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.684+0000 7ff3f9cba640 1 -- 172.21.15.103:0/3312687562 shutdown_connections 2024-09-19T06:59:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:29.684+0000 7ff3f9cba640 1 -- 172.21.15.103:0/3312687562 wait complete. 2024-09-19T06:59:30.008 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:29 smithi134 ceph-mon[35356]: pgmap v318: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-09-19T06:59:30.008 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:29 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:59:30.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:29 smithi103 ceph-mon[31158]: pgmap v318: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-09-19T06:59:30.037 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:29 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:59:30.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:29 smithi174 ceph-mon[35637]: pgmap v318: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-09-19T06:59:30.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:29 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:59:32.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: pgmap v319: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-09-19T06:59:32.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:59:32.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:59:32.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: pgmap v319: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-09-19T06:59:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:59:32.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:59:32.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: pgmap v319: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-09-19T06:59:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:59:32.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:59:32.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:59:32.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T06:59:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:32 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-19T06:59:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:32 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T06:59:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:32 smithi103 ceph-mon[31158]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T06:59:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:32 smithi103 ceph-mon[31158]: Unable to set osd_memory_target on smithi134 to 184595353: error parsing value: Value '184595353' is below minimum 939524096 2024-09-19T06:59:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:32 smithi103 ceph-mon[31158]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T06:59:33.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:32 smithi103 ceph-mon[31158]: Unable to set osd_memory_target on smithi174 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T06:59:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:32 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-19T06:59:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:32 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T06:59:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:32 smithi134 ceph-mon[35356]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T06:59:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:32 smithi134 ceph-mon[35356]: Unable to set osd_memory_target on smithi134 to 184595353: error parsing value: Value '184595353' is below minimum 939524096 2024-09-19T06:59:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:32 smithi134 ceph-mon[35356]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T06:59:33.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:32 smithi134 ceph-mon[35356]: Unable to set osd_memory_target on smithi174 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T06:59:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:32 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-19T06:59:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:32 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T06:59:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:32 smithi174 ceph-mon[35637]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T06:59:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:32 smithi174 ceph-mon[35637]: Unable to set osd_memory_target on smithi134 to 184595353: error parsing value: Value '184595353' is below minimum 939524096 2024-09-19T06:59:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:32 smithi174 ceph-mon[35637]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T06:59:33.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:32 smithi174 ceph-mon[35637]: Unable to set osd_memory_target on smithi174 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T06:59:34.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:33 smithi103 ceph-mon[31158]: pgmap v320: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-09-19T06:59:34.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:33 smithi134 ceph-mon[35356]: pgmap v320: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-09-19T06:59:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:33 smithi174 ceph-mon[35637]: pgmap v320: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-09-19T06:59:35.745 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:59:36.010 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:36.130 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:35 smithi103 ceph-mon[31158]: pgmap v321: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-09-19T06:59:36.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:35 smithi134 ceph-mon[35356]: pgmap v321: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-09-19T06:59:36.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:35 smithi174 ceph-mon[35637]: pgmap v321: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-09-19T06:59:36.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.505+0000 7fe29c09f640 1 -- 172.21.15.103:0/2941449246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe294102ef0 msgr2=0x7fe29410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:36.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.505+0000 7fe29c09f640 1 --2- 172.21.15.103:0/2941449246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe294102ef0 0x7fe29410da40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009ca0 tx=0x7fe28c02a950 comp rx=0 tx=0).stop 2024-09-19T06:59:36.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.505+0000 7fe29c09f640 1 -- 172.21.15.103:0/2941449246 shutdown_connections 2024-09-19T06:59:36.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.505+0000 7fe29c09f640 1 --2- 172.21.15.103:0/2941449246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe294102ef0 0x7fe29410da40 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:36.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.505+0000 7fe29c09f640 1 --2- 172.21.15.103:0/2941449246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe294102570 0x7fe2941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:36.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.505+0000 7fe29c09f640 1 --2- 172.21.15.103:0/2941449246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe294100e20 0x7fe294101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:36.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.505+0000 7fe29c09f640 1 -- 172.21.15.103:0/2941449246 >> 172.21.15.103:0/2941449246 conn(0x7fe2940fc7d0 msgr2=0x7fe2940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:36.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.505+0000 7fe29c09f640 1 -- 172.21.15.103:0/2941449246 shutdown_connections 2024-09-19T06:59:36.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.505+0000 7fe29c09f640 1 -- 172.21.15.103:0/2941449246 wait complete. 2024-09-19T06:59:36.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.505+0000 7fe29c09f640 1 Processor -- start 2024-09-19T06:59:36.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.505+0000 7fe29c09f640 1 -- start start 2024-09-19T06:59:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe29c09f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe294100e20 0x7fe294198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe29c09f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe294102570 0x7fe294198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe29c09f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe294102ef0 0x7fe29419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe29c09f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe294111020 con 0x7fe294102570 2024-09-19T06:59:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe29c09f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe294110ea0 con 0x7fe294102ef0 2024-09-19T06:59:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe29c09f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2941111a0 con 0x7fe294100e20 2024-09-19T06:59:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe29963a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe294102570 0x7fe294198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe29963a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe294102570 0x7fe294198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60612/0 (socket says 172.21.15.103:60612) 2024-09-19T06:59:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe29963a640 1 -- 172.21.15.103:0/3258570749 learned_addr learned my addr 172.21.15.103:0/3258570749 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe29a63c640 1 --2- 172.21.15.103:0/3258570749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe294102ef0 0x7fe29419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe299e3b640 1 --2- 172.21.15.103:0/3258570749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe294100e20 0x7fe294198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:36.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe29963a640 1 -- 172.21.15.103:0/3258570749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe294100e20 msgr2=0x7fe294198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:36.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe29963a640 1 --2- 172.21.15.103:0/3258570749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe294100e20 0x7fe294198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:36.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe29963a640 1 -- 172.21.15.103:0/3258570749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe294102ef0 msgr2=0x7fe29419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:36.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe29963a640 1 --2- 172.21.15.103:0/3258570749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe294102ef0 0x7fe29419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:36.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe29963a640 1 -- 172.21.15.103:0/3258570749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29419d5b0 con 0x7fe294102570 2024-09-19T06:59:36.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe29963a640 1 --2- 172.21.15.103:0/3258570749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe294102570 0x7fe294198b40 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fe28400b4a0 tx=0x7fe28400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:36.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe28affd640 1 -- 172.21.15.103:0/3258570749 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284014070 con 0x7fe294102570 2024-09-19T06:59:36.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.506+0000 7fe28affd640 1 -- 172.21.15.103:0/3258570749 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe284010af0 con 0x7fe294102570 2024-09-19T06:59:36.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.507+0000 7fe29c09f640 1 -- 172.21.15.103:0/3258570749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29419d8a0 con 0x7fe294102570 2024-09-19T06:59:36.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.507+0000 7fe28affd640 1 -- 172.21.15.103:0/3258570749 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284017d10 con 0x7fe294102570 2024-09-19T06:59:36.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.507+0000 7fe29c09f640 1 -- 172.21.15.103:0/3258570749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2941a5180 con 0x7fe294102570 2024-09-19T06:59:36.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.508+0000 7fe29c09f640 1 -- 172.21.15.103:0/3258570749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe294101280 con 0x7fe294102570 2024-09-19T06:59:36.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.509+0000 7fe28affd640 1 -- 172.21.15.103:0/3258570749 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fe284017070 con 0x7fe294102570 2024-09-19T06:59:36.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.510+0000 7fe28affd640 1 --2- 172.21.15.103:0/3258570749 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe2800a3430 0x7fe2800a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:36.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.510+0000 7fe299e3b640 1 --2- 172.21.15.103:0/3258570749 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe2800a3430 0x7fe2800a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:36.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.510+0000 7fe28affd640 1 -- 172.21.15.103:0/3258570749 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7fe2840cf3a0 con 0x7fe294102570 2024-09-19T06:59:36.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.510+0000 7fe299e3b640 1 --2- 172.21.15.103:0/3258570749 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe2800a3430 0x7fe2800a58f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe290009890 tx=0x7fe290008040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:36.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.513+0000 7fe28affd640 1 -- 172.21.15.103:0/3258570749 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe28409be10 con 0x7fe294102570 2024-09-19T06:59:36.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.713+0000 7fe29c09f640 1 -- 172.21.15.103:0/3258570749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe294199960 con 0x7fe294102570 2024-09-19T06:59:36.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.714+0000 7fe28affd640 1 -- 172.21.15.103:0/3258570749 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fe294199960 con 0x7fe294102570 2024-09-19T06:59:36.716 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:59:36.716 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:59:36.716 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:59:36.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.716+0000 7fe29c09f640 1 -- 172.21.15.103:0/3258570749 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe2800a3430 msgr2=0x7fe2800a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.716+0000 7fe29c09f640 1 --2- 172.21.15.103:0/3258570749 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe2800a3430 0x7fe2800a58f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe290009890 tx=0x7fe290008040 comp rx=0 tx=0).stop 2024-09-19T06:59:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.716+0000 7fe29c09f640 1 -- 172.21.15.103:0/3258570749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe294102570 msgr2=0x7fe294198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.716+0000 7fe29c09f640 1 --2- 172.21.15.103:0/3258570749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe294102570 0x7fe294198b40 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fe28400b4a0 tx=0x7fe28400b960 comp rx=0 tx=0).stop 2024-09-19T06:59:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.716+0000 7fe29c09f640 1 -- 172.21.15.103:0/3258570749 shutdown_connections 2024-09-19T06:59:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.716+0000 7fe29c09f640 1 --2- 172.21.15.103:0/3258570749 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe2800a3430 0x7fe2800a58f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.716+0000 7fe29c09f640 1 --2- 172.21.15.103:0/3258570749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe294102ef0 0x7fe29419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.716+0000 7fe29c09f640 1 --2- 172.21.15.103:0/3258570749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe294102570 0x7fe294198b40 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.716+0000 7fe29c09f640 1 --2- 172.21.15.103:0/3258570749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe294100e20 0x7fe294198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.716+0000 7fe29c09f640 1 -- 172.21.15.103:0/3258570749 >> 172.21.15.103:0/3258570749 conn(0x7fe2940fc7d0 msgr2=0x7fe294069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.717+0000 7fe29c09f640 1 -- 172.21.15.103:0/3258570749 shutdown_connections 2024-09-19T06:59:36.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:36.717+0000 7fe29c09f640 1 -- 172.21.15.103:0/3258570749 wait complete. 2024-09-19T06:59:37.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:36 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3258570749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:59:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:36 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3258570749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:59:37.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:36 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3258570749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:59:37.459 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id c9039c34 --release --await' 2024-09-19T06:59:37.459 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id c9039c34 --release --await 2024-09-19T06:59:37.706 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:38.068 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:37 smithi103 ceph-mon[31158]: pgmap v322: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-19T06:59:38.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.153+0000 7f4eadee2640 1 -- 172.21.15.103:0/2140735791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ea8103660 msgr2=0x7f4ea810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:38.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.153+0000 7f4eadee2640 1 --2- 172.21.15.103:0/2140735791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ea8103660 0x7f4ea810bb50 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009900 tx=0x7f4e9c02a480 comp rx=0 tx=0).stop 2024-09-19T06:59:38.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.153+0000 7f4eadee2640 1 -- 172.21.15.103:0/2140735791 shutdown_connections 2024-09-19T06:59:38.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.153+0000 7f4eadee2640 1 --2- 172.21.15.103:0/2140735791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ea810c170 0x7f4ea810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.153+0000 7f4eadee2640 1 --2- 172.21.15.103:0/2140735791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ea8103660 0x7f4ea810bb50 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.153+0000 7f4eadee2640 1 --2- 172.21.15.103:0/2140735791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ea8102d40 0x7f4ea8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.153+0000 7f4eadee2640 1 -- 172.21.15.103:0/2140735791 >> 172.21.15.103:0/2140735791 conn(0x7f4ea80fc7a0 msgr2=0x7f4ea80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:38.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.154+0000 7f4eadee2640 1 -- 172.21.15.103:0/2140735791 shutdown_connections 2024-09-19T06:59:38.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.154+0000 7f4eadee2640 1 -- 172.21.15.103:0/2140735791 wait complete. 2024-09-19T06:59:38.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.154+0000 7f4eadee2640 1 Processor -- start 2024-09-19T06:59:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.154+0000 7f4eadee2640 1 -- start start 2024-09-19T06:59:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.154+0000 7f4eadee2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ea8102d40 0x7f4ea8198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.154+0000 7f4eadee2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ea8103660 0x7f4ea8198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.154+0000 7f4eadee2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ea810c170 0x7f4ea819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.154+0000 7f4eadee2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea8112030 con 0x7f4ea810c170 2024-09-19T06:59:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.154+0000 7f4eadee2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea8111eb0 con 0x7f4ea8102d40 2024-09-19T06:59:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.154+0000 7f4eadee2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81121b0 con 0x7f4ea8103660 2024-09-19T06:59:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4ea7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ea810c170 0x7f4ea819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4ea7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ea810c170 0x7f4ea819cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58802/0 (socket says 172.21.15.103:58802) 2024-09-19T06:59:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4ea7fff640 1 -- 172.21.15.103:0/1201221571 learned_addr learned my addr 172.21.15.103:0/1201221571 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4ea6ffd640 1 --2- 172.21.15.103:0/1201221571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ea8103660 0x7f4ea8198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4ea77fe640 1 --2- 172.21.15.103:0/1201221571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ea8102d40 0x7f4ea8198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4ea7fff640 1 -- 172.21.15.103:0/1201221571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ea8103660 msgr2=0x7f4ea8198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4ea7fff640 1 --2- 172.21.15.103:0/1201221571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ea8103660 0x7f4ea8198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4ea7fff640 1 -- 172.21.15.103:0/1201221571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ea8102d40 msgr2=0x7f4ea8198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4ea7fff640 1 --2- 172.21.15.103:0/1201221571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ea8102d40 0x7f4ea8198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4ea7fff640 1 -- 172.21.15.103:0/1201221571 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea819d600 con 0x7f4ea810c170 2024-09-19T06:59:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4ea77fe640 1 --2- 172.21.15.103:0/1201221571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ea8102d40 0x7f4ea8198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4ea7fff640 1 --2- 172.21.15.103:0/1201221571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ea810c170 0x7f4ea819cf20 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000b970 tx=0x7f4ea000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4ea4ff9640 1 -- 172.21.15.103:0/1201221571 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0019070 con 0x7f4ea810c170 2024-09-19T06:59:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4eadee2640 1 -- 172.21.15.103:0/1201221571 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea819d8f0 con 0x7f4ea810c170 2024-09-19T06:59:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4ea4ff9640 1 -- 172.21.15.103:0/1201221571 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea000cb40 con 0x7f4ea810c170 2024-09-19T06:59:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4ea4ff9640 1 -- 172.21.15.103:0/1201221571 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0007610 con 0x7f4ea810c170 2024-09-19T06:59:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.155+0000 7f4eadee2640 1 -- 172.21.15.103:0/1201221571 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea81a51d0 con 0x7f4ea810c170 2024-09-19T06:59:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.158+0000 7f4eb02bb740 1 Processor -- start 2024-09-19T06:59:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.158+0000 7f4ea4ff9640 1 -- 172.21.15.103:0/1201221571 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4ea001c070 con 0x7f4ea810c170 2024-09-19T06:59:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.158+0000 7f4eb02bb740 1 -- start start 2024-09-19T06:59:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.158+0000 7f4eb02bb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84060 0x558348c84440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.158+0000 7f4eb02bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c84980 0x558348c8ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.158+0000 7f4eb02bb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c8f240 0x558348c91630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.158+0000 7f4eb02bb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558348be9620 con 0x558348c84060 2024-09-19T06:59:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.158+0000 7f4eb02bb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558348b90e50 con 0x558348c8f240 2024-09-19T06:59:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.158+0000 7f4eb02bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558348be55e0 con 0x558348c84980 2024-09-19T06:59:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4ea6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84060 0x558348c84440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4ea6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84060 0x558348c84440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58814/0 (socket says 172.21.15.103:58814) 2024-09-19T06:59:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4ea6ffd640 1 -- 172.21.15.103:0/4013624893 learned_addr learned my addr 172.21.15.103:0/4013624893 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4ea77fe640 1 --2- 172.21.15.103:0/4013624893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c84980 0x558348c8ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4ea6ffd640 1 --2- 172.21.15.103:0/4013624893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c8f240 0x558348c91630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4ea6ffd640 1 -- 172.21.15.103:0/4013624893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c84980 msgr2=0x558348c8ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:38.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4ea6ffd640 1 --2- 172.21.15.103:0/4013624893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c84980 0x558348c8ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4ea6ffd640 1 -- 172.21.15.103:0/4013624893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c8f240 msgr2=0x558348c91630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:38.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4ea6ffd640 1 --2- 172.21.15.103:0/4013624893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c8f240 0x558348c91630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4ea4ff9640 1 --2- 172.21.15.103:0/1201221571 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4e900a33b0 0x7f4e900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4ea6ffd640 1 -- 172.21.15.103:0/4013624893 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558348b81d40 con 0x558348c84060 2024-09-19T06:59:38.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4ea77fe640 1 --2- 172.21.15.103:0/1201221571 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4e900a33b0 0x7f4e900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4ea6ffd640 1 --2- 172.21.15.103:0/4013624893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84060 0x558348c84440 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x558348c81970 tx=0x7f4e9c0314d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee8d0a8921640fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:38.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4ea4ff9640 1 -- 172.21.15.103:0/1201221571 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f4ea00cf120 con 0x7f4ea810c170 2024-09-19T06:59:38.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4e8dffb640 1 -- 172.21.15.103:0/4013624893 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c035040 con 0x558348c84060 2024-09-19T06:59:38.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4e8dffb640 1 -- 172.21.15.103:0/4013624893 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c005020 con 0x558348c84060 2024-09-19T06:59:38.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.159+0000 7f4e8dffb640 1 -- 172.21.15.103:0/4013624893 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c004b00 con 0x558348c84060 2024-09-19T06:59:38.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.160+0000 7f4eb02bb740 1 -- 172.21.15.103:0/4013624893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84060 msgr2=0x558348c84440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:38.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.160+0000 7f4ea77fe640 1 --2- 172.21.15.103:0/1201221571 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4e900a33b0 0x7f4e900a5870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x558348c81450 tx=0x7f4e98008040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:38.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.160+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/4013624893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84060 0x558348c84440 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x558348c81970 tx=0x7f4e9c0314d0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.160+0000 7f4eb02bb740 1 -- 172.21.15.103:0/4013624893 shutdown_connections 2024-09-19T06:59:38.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.160+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/4013624893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c8f240 0x558348c91630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.160+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/4013624893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c84980 0x558348c8ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.160+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/4013624893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84060 0x558348c84440 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.160+0000 7f4eb02bb740 1 -- 172.21.15.103:0/4013624893 >> 172.21.15.103:0/4013624893 conn(0x558348c805f0 msgr2=0x558348c809d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:38.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.160+0000 7f4eb02bb740 1 -- 172.21.15.103:0/4013624893 shutdown_connections 2024-09-19T06:59:38.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.160+0000 7f4eb02bb740 1 -- 172.21.15.103:0/4013624893 wait complete. 2024-09-19T06:59:38.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4eb02bb740 1 -- start start 2024-09-19T06:59:38.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4eb02bb740 1 Processor -- start 2024-09-19T06:59:38.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4eb02bb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84060 0x558348c8c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4eb02bb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c84980 0x558348c8c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4eb02bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c8f240 0x558348c8cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4eb02bb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558348c91e60 con 0x558348c84060 2024-09-19T06:59:38.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4eb02bb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558348b71f10 con 0x558348c84980 2024-09-19T06:59:38.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4eb02bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558348c91fe0 con 0x558348c8f240 2024-09-19T06:59:38.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4eb02bb740 10 client.0 _ll_register_callbacks cb 0x558348b95db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:38.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4ea77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84060 0x558348c8c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4ea77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84060 0x558348c8c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58826/0 (socket says 172.21.15.103:58826) 2024-09-19T06:59:38.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4ea77fe640 1 -- 172.21.15.103:0/763614288 learned_addr learned my addr 172.21.15.103:0/763614288 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:38.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4ea6ffd640 1 --2- 172.21.15.103:0/763614288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c84980 0x558348c8c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4ea77fe640 1 -- 172.21.15.103:0/763614288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c8f240 msgr2=0x558348c8cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:38.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4ea77fe640 1 --2- 172.21.15.103:0/763614288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c8f240 0x558348c8cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4ea77fe640 1 -- 172.21.15.103:0/763614288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c84980 msgr2=0x558348c8c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:38.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4ea77fe640 1 --2- 172.21.15.103:0/763614288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c84980 0x558348c8c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.161+0000 7f4ea77fe640 1 -- 172.21.15.103:0/763614288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583488f8580 con 0x558348c84060 2024-09-19T06:59:38.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.162+0000 7f4ea77fe640 1 --2- 172.21.15.103:0/763614288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84060 0x558348c8c0e0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f4e98016a00 tx=0x7f4e980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6033407d88d287b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:38.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.162+0000 7f4e86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:38.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.162+0000 7f4e86ffd640 1 -- 172.21.15.103:0/763614288 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9801c050 con 0x558348c84060 2024-09-19T06:59:38.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.162+0000 7f4eb02bb740 1 -- 172.21.15.103:0/763614288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e9c005020 con 0x558348c84060 2024-09-19T06:59:38.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.162+0000 7f4e86ffd640 1 -- 172.21.15.103:0/763614288 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f4e98014b40 con 0x558348c84060 2024-09-19T06:59:38.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.162+0000 7f4e86ffd640 1 -- 172.21.15.103:0/763614288 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4e98014d40 con 0x558348c84060 2024-09-19T06:59:38.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.162+0000 7f4eb02bb740 10 client.18978 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:38.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.162+0000 7f4eb02bb740 1 -- 172.21.15.103:0/763614288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5583489c2960 con 0x558348c84060 2024-09-19T06:59:38.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.162+0000 7f4e86ffd640 1 -- 172.21.15.103:0/763614288 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c005020 con 0x558348c84060 2024-09-19T06:59:38.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.162+0000 7f4e86ffd640 1 -- 172.21.15.103:0/763614288 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4e980065e0 con 0x558348c84060 2024-09-19T06:59:38.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.162+0000 7f4eb02bb740 10 client.18978 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:38.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.162+0000 7f4eb02bb740 10 client.18978 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:59:38.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.162+0000 7f4eb02bb740 20 client.18978 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:38.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.162+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/763614288 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558348c9eeb0 0x558348ca1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.162+0000 7f4eb02bb740 4 client.18978 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:38.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.162+0000 7f4eb02bb740 1 -- 172.21.15.103:0/763614288 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558348bf6b90 con 0x558348c9eeb0 2024-09-19T06:59:38.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.163+0000 7f4ea6ffd640 1 --2- 172.21.15.103:0/763614288 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558348c9eeb0 0x558348ca1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.163+0000 7f4e86ffd640 10 client.18978 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:38.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.163+0000 7f4ea6ffd640 1 --2- 172.21.15.103:0/763614288 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558348c9eeb0 0x558348ca1370 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=345e45dd6101438b server_cookie=2b2d10848a7c8bb in_seq=0 out_seq=0 2024-09-19T06:59:38.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.164+0000 7f4e86ffd640 1 -- 172.21.15.103:0/763614288 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558348bf6b90 con 0x558348c9eeb0 2024-09-19T06:59:38.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.164+0000 7f4e86ffd640 10 client.18978 handle_command_reply: tid=0 2024-09-19T06:59:38.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.164+0000 7f4eb02bb740 1 client.18978 shutdown 2024-09-19T06:59:38.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.164+0000 7f4eb02bb740 1 -- 172.21.15.103:0/763614288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84060 msgr2=0x558348c8c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:38.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.164+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/763614288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84060 0x558348c8c0e0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f4e98016a00 tx=0x7f4e980046c0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.164+0000 7f4eb02bb740 1 -- 172.21.15.103:0/763614288 shutdown_connections 2024-09-19T06:59:38.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.164+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/763614288 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558348c9eeb0 0x558348ca1370 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.164+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/763614288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c8f240 0x558348c8cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.164+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/763614288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c84980 0x558348c8c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.164+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/763614288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84060 0x558348c8c0e0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.164+0000 7f4eb02bb740 1 -- 172.21.15.103:0/763614288 >> 172.21.15.103:0/763614288 conn(0x558348c805f0 msgr2=0x558348c86320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:38.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.164+0000 7f4e86ffd640 0 client.18978 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:38.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.164+0000 7f4eb02bb740 1 -- 172.21.15.103:0/763614288 shutdown_connections 2024-09-19T06:59:38.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.164+0000 7f4eb02bb740 1 -- 172.21.15.103:0/763614288 wait complete. 2024-09-19T06:59:38.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.164+0000 7f4eb02bb740 20 client.18978 trim_cache size 0 max 16384 2024-09-19T06:59:38.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4eb02bb740 1 Processor -- start 2024-09-19T06:59:38.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4eb02bb740 1 -- start start 2024-09-19T06:59:38.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4eb02bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c84060 0x558348c875b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4eb02bb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84980 0x558348c7f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4eb02bb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c8f240 0x558348ca6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4eb02bb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81ab7c0 con 0x558348c84980 2024-09-19T06:59:38.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4eb02bb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81ad300 con 0x558348c8f240 2024-09-19T06:59:38.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4eb02bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81abe90 con 0x558348c84060 2024-09-19T06:59:38.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4ea6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84980 0x558348c7f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4ea6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84980 0x558348c7f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58828/0 (socket says 172.21.15.103:58828) 2024-09-19T06:59:38.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4ea6ffd640 1 -- 172.21.15.103:0/2779062255 learned_addr learned my addr 172.21.15.103:0/2779062255 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:38.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4ea77fe640 1 --2- 172.21.15.103:0/2779062255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c84060 0x558348c875b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4ea7fff640 1 --2- 172.21.15.103:0/2779062255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c8f240 0x558348ca6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4ea6ffd640 1 -- 172.21.15.103:0/2779062255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c84060 msgr2=0x558348c875b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:38.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4ea6ffd640 1 --2- 172.21.15.103:0/2779062255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c84060 0x558348c875b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4ea6ffd640 1 -- 172.21.15.103:0/2779062255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c8f240 msgr2=0x558348ca6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:38.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4ea6ffd640 1 --2- 172.21.15.103:0/2779062255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c8f240 0x558348ca6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.181+0000 7f4ea6ffd640 1 -- 172.21.15.103:0/2779062255 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558348c8e020 con 0x558348c84980 2024-09-19T06:59:38.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.182+0000 7f4ea6ffd640 1 --2- 172.21.15.103:0/2779062255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84980 0x558348c7f640 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c02a9a0 tx=0x7f4e9c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6893b0f14e3e2c2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:38.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.182+0000 7f4e8d7fa640 1 -- 172.21.15.103:0/2779062255 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c035040 con 0x558348c84980 2024-09-19T06:59:38.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.182+0000 7f4e8d7fa640 1 -- 172.21.15.103:0/2779062255 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c057070 con 0x558348c84980 2024-09-19T06:59:38.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.182+0000 7f4e8d7fa640 1 -- 172.21.15.103:0/2779062255 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c003de0 con 0x558348c84980 2024-09-19T06:59:38.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.182+0000 7f4eb02bb740 1 -- 172.21.15.103:0/2779062255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84980 msgr2=0x558348c7f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:38.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.182+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/2779062255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84980 0x558348c7f640 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c02a9a0 tx=0x7f4e9c005fd0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.182+0000 7f4eb02bb740 1 -- 172.21.15.103:0/2779062255 shutdown_connections 2024-09-19T06:59:38.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.182+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/2779062255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c8f240 0x558348ca6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.182+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/2779062255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84980 0x558348c7f640 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.182+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/2779062255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c84060 0x558348c875b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.182+0000 7f4eb02bb740 1 -- 172.21.15.103:0/2779062255 >> 172.21.15.103:0/2779062255 conn(0x558348c805f0 msgr2=0x558348c7ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:38.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.182+0000 7f4eb02bb740 1 -- 172.21.15.103:0/2779062255 shutdown_connections 2024-09-19T06:59:38.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.183+0000 7f4eb02bb740 1 -- 172.21.15.103:0/2779062255 wait complete. 2024-09-19T06:59:38.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.183+0000 7f4eb02bb740 1 -- start start 2024-09-19T06:59:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.183+0000 7f4eb02bb740 1 Processor -- start 2024-09-19T06:59:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.183+0000 7f4eb02bb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c84060 0x558348c93600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.183+0000 7f4eb02bb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84980 0x558348c93b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.183+0000 7f4eb02bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c8f240 0x558348d08c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.183+0000 7f4eb02bb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81acc30 con 0x558348c84980 2024-09-19T06:59:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.183+0000 7f4eb02bb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81ac560 con 0x558348c84060 2024-09-19T06:59:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.183+0000 7f4eb02bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81a97c0 con 0x558348c8f240 2024-09-19T06:59:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.183+0000 7f4ea6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84980 0x558348c93b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.183+0000 7f4eb02bb740 10 client.0 _ll_register_callbacks cb 0x7f4e980166b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.183+0000 7f4ea6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84980 0x558348c93b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58838/0 (socket says 172.21.15.103:58838) 2024-09-19T06:59:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.183+0000 7f4ea6ffd640 1 -- 172.21.15.103:0/3424123269 learned_addr learned my addr 172.21.15.103:0/3424123269 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.183+0000 7f4ea7fff640 1 --2- 172.21.15.103:0/3424123269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c8f240 0x558348d08c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.184+0000 7f4ea77fe640 1 --2- 172.21.15.103:0/3424123269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c84060 0x558348c93600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.184+0000 7f4ea6ffd640 1 -- 172.21.15.103:0/3424123269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c8f240 msgr2=0x558348d08c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.184+0000 7f4ea6ffd640 1 --2- 172.21.15.103:0/3424123269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c8f240 0x558348d08c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.184+0000 7f4ea6ffd640 1 -- 172.21.15.103:0/3424123269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c84060 msgr2=0x558348c93600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.184+0000 7f4ea6ffd640 1 --2- 172.21.15.103:0/3424123269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c84060 0x558348c93600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.184+0000 7f4ea6ffd640 1 -- 172.21.15.103:0/3424123269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558348d09150 con 0x558348c84980 2024-09-19T06:59:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.184+0000 7f4ea6ffd640 1 --2- 172.21.15.103:0/3424123269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84980 0x558348c93b40 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c005ec0 tx=0x7f4e9c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ac8195d430a40d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.184+0000 7f4e86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.184+0000 7f4e86ffd640 1 -- 172.21.15.103:0/3424123269 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c060060 con 0x558348c84980 2024-09-19T06:59:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.184+0000 7f4e86ffd640 1 -- 172.21.15.103:0/3424123269 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f4e9c004100 con 0x558348c84980 2024-09-19T06:59:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.184+0000 7f4eb02bb740 1 -- 172.21.15.103:0/3424123269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e9c057070 con 0x558348c84980 2024-09-19T06:59:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.184+0000 7f4e86ffd640 1 -- 172.21.15.103:0/3424123269 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4e9c035030 con 0x558348c84980 2024-09-19T06:59:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.184+0000 7f4eb02bb740 10 client.18984 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.184+0000 7f4eb02bb740 1 -- 172.21.15.103:0/3424123269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558348d0a3d0 con 0x558348c84980 2024-09-19T06:59:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.184+0000 7f4e86ffd640 1 -- 172.21.15.103:0/3424123269 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c057070 con 0x558348c84980 2024-09-19T06:59:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.185+0000 7f4e86ffd640 1 -- 172.21.15.103:0/3424123269 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4e9c04a820 con 0x558348c84980 2024-09-19T06:59:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.185+0000 7f4eb02bb740 10 client.18984 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.185+0000 7f4eb02bb740 10 client.18984 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:59:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.185+0000 7f4eb02bb740 20 client.18984 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.185+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/3424123269 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558348d0a600 0x558348d0c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.185+0000 7f4eb02bb740 4 client.18984 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c9039c34", "release": true, "await": true}] 2024-09-19T06:59:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.185+0000 7f4eb02bb740 1 -- 172.21.15.103:0/3424123269 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c9039c34", "release": true, "await": true}) -- 0x558348bf10a0 con 0x558348d0a600 2024-09-19T06:59:38.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.185+0000 7f4ea77fe640 1 --2- 172.21.15.103:0/3424123269 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558348d0a600 0x558348d0c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:38.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.185+0000 7f4e86ffd640 10 client.18984 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:38.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.185+0000 7f4ea77fe640 1 --2- 172.21.15.103:0/3424123269 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558348d0a600 0x558348d0c9f0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2507c0efabf8fd03 server_cookie=4d226243e43a8960 in_seq=0 out_seq=0 2024-09-19T06:59:38.192 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:37 smithi134 ceph-mon[35356]: pgmap v322: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-19T06:59:38.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.188+0000 7f4e86ffd640 1 -- 172.21.15.103:0/3424123269 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558348bf10a0 con 0x558348d0a600 2024-09-19T06:59:38.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.188+0000 7f4e86ffd640 10 client.18984 handle_command_reply: tid=0 2024-09-19T06:59:38.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.188+0000 7f4eb02bb740 1 client.18984 shutdown 2024-09-19T06:59:38.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eb02bb740 1 -- 172.21.15.103:0/3424123269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84980 msgr2=0x558348c93b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:38.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/3424123269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84980 0x558348c93b40 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c005ec0 tx=0x7f4e9c009780 comp rx=0 tx=0).stop 2024-09-19T06:59:38.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eb02bb740 1 -- 172.21.15.103:0/3424123269 shutdown_connections 2024-09-19T06:59:38.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/3424123269 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558348d0a600 0x558348d0c9f0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/3424123269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558348c8f240 0x558348d08c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/3424123269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558348c84980 0x558348c93b40 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4e86ffd640 0 client.18984 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:38.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eb02bb740 1 --2- 172.21.15.103:0/3424123269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558348c84060 0x558348c93600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eb02bb740 1 -- 172.21.15.103:0/3424123269 >> 172.21.15.103:0/3424123269 conn(0x558348c805f0 msgr2=0x558348c7f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:38.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eb02bb740 1 -- 172.21.15.103:0/3424123269 shutdown_connections 2024-09-19T06:59:38.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eb02bb740 1 -- 172.21.15.103:0/3424123269 wait complete. 2024-09-19T06:59:38.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eb02bb740 20 client.18984 trim_cache size 0 max 16384 2024-09-19T06:59:38.194 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:59:38.194 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:59:38.195 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:59:38.195 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 36, 2024-09-19T06:59:38.195 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:59:38.195 INFO:teuthology.orchestra.run.smithi103.stdout: "c9039c34": { 2024-09-19T06:59:38.195 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 36, 2024-09-19T06:59:38.195 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:59:38.195 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:59:38.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:59:38.195 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:59:38.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:59:38.195 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:59:38.195 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 150.0, 2024-09-19T06:59:38.196 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:59:38.196 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:59:38.196 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:59:38.196 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:59:38.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T06:59:38.196 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:59:38.196 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:38.196 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:38.196 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:38.196 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:38.197 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:38.197 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:59:38.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eadee2640 1 -- 172.21.15.103:0/1201221571 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4e900a33b0 msgr2=0x7f4e900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:38.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eadee2640 1 --2- 172.21.15.103:0/1201221571 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4e900a33b0 0x7f4e900a5870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x558348c81450 tx=0x7f4e98008040 comp rx=0 tx=0).stop 2024-09-19T06:59:38.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eadee2640 1 -- 172.21.15.103:0/1201221571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ea810c170 msgr2=0x7f4ea819cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:38.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eadee2640 1 --2- 172.21.15.103:0/1201221571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ea810c170 0x7f4ea819cf20 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000b970 tx=0x7f4ea000be30 comp rx=0 tx=0).stop 2024-09-19T06:59:38.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eadee2640 1 -- 172.21.15.103:0/1201221571 shutdown_connections 2024-09-19T06:59:38.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eadee2640 1 --2- 172.21.15.103:0/1201221571 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4e900a33b0 0x7f4e900a5870 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eadee2640 1 --2- 172.21.15.103:0/1201221571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ea810c170 0x7f4ea819cf20 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eadee2640 1 --2- 172.21.15.103:0/1201221571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ea8103660 0x7f4ea8198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eadee2640 1 --2- 172.21.15.103:0/1201221571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ea8102d40 0x7f4ea8198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:38.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.189+0000 7f4eadee2640 1 -- 172.21.15.103:0/1201221571 >> 172.21.15.103:0/1201221571 conn(0x7f4ea80fc7a0 msgr2=0x7f4ea80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:38.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.190+0000 7f4eadee2640 1 -- 172.21.15.103:0/1201221571 shutdown_connections 2024-09-19T06:59:38.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:38.190+0000 7f4eadee2640 1 -- 172.21.15.103:0/1201221571 wait complete. 2024-09-19T06:59:38.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:37 smithi174 ceph-mon[35637]: pgmap v322: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-19T06:59:38.817 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c9039c34', total seconds elapsed: 33.1 2024-09-19T06:59:38.817 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-19T06:59:38.817 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:59:39.065 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:39.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.546+0000 7fa5ce4ae640 1 -- 172.21.15.103:0/3310021698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c8102d40 msgr2=0x7fa5c8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:39.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.546+0000 7fa5ce4ae640 1 --2- 172.21.15.103:0/3310021698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c8102d40 0x7fa5c8103120 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009900 tx=0x7fa5b802a480 comp rx=0 tx=0).stop 2024-09-19T06:59:39.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.546+0000 7fa5ce4ae640 1 -- 172.21.15.103:0/3310021698 shutdown_connections 2024-09-19T06:59:39.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.546+0000 7fa5ce4ae640 1 --2- 172.21.15.103:0/3310021698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5c810c170 0x7fa5c810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:39.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.546+0000 7fa5ce4ae640 1 --2- 172.21.15.103:0/3310021698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c8103660 0x7fa5c810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:39.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.546+0000 7fa5ce4ae640 1 --2- 172.21.15.103:0/3310021698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c8102d40 0x7fa5c8103120 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:39.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.546+0000 7fa5ce4ae640 1 -- 172.21.15.103:0/3310021698 >> 172.21.15.103:0/3310021698 conn(0x7fa5c80fc7a0 msgr2=0x7fa5c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:39.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.546+0000 7fa5ce4ae640 1 -- 172.21.15.103:0/3310021698 shutdown_connections 2024-09-19T06:59:39.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.546+0000 7fa5ce4ae640 1 -- 172.21.15.103:0/3310021698 wait complete. 2024-09-19T06:59:39.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.547+0000 7fa5ce4ae640 1 Processor -- start 2024-09-19T06:59:39.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.547+0000 7fa5ce4ae640 1 -- start start 2024-09-19T06:59:39.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5ce4ae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5c8102d40 0x7fa5c8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:39.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5ce4ae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c8103660 0x7fa5c81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:39.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5ce4ae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c810c170 0x7fa5c819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:39.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5ce4ae640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8111fd0 con 0x7fa5c810c170 2024-09-19T06:59:39.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5ce4ae640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8111e50 con 0x7fa5c8102d40 2024-09-19T06:59:39.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5ce4ae640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8112150 con 0x7fa5c8103660 2024-09-19T06:59:39.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5cca4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c810c170 0x7fa5c819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:39.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5cca4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c810c170 0x7fa5c819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58862/0 (socket says 172.21.15.103:58862) 2024-09-19T06:59:39.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5cca4b640 1 -- 172.21.15.103:0/549242865 learned_addr learned my addr 172.21.15.103:0/549242865 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:39.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5c77fe640 1 --2- 172.21.15.103:0/549242865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c8103660 0x7fa5c81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:39.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5c7fff640 1 --2- 172.21.15.103:0/549242865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5c8102d40 0x7fa5c8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:39.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5cca4b640 1 -- 172.21.15.103:0/549242865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c8103660 msgr2=0x7fa5c81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:39.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5cca4b640 1 --2- 172.21.15.103:0/549242865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c8103660 0x7fa5c81989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:39.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5cca4b640 1 -- 172.21.15.103:0/549242865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5c8102d40 msgr2=0x7fa5c8198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:39.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5cca4b640 1 --2- 172.21.15.103:0/549242865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5c8102d40 0x7fa5c8198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:39.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5cca4b640 1 -- 172.21.15.103:0/549242865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c819d440 con 0x7fa5c810c170 2024-09-19T06:59:39.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5cca4b640 1 --2- 172.21.15.103:0/549242865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c810c170 0x7fa5c819cd60 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000b970 tx=0x7fa5c000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:39.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.548+0000 7fa5c57fa640 1 -- 172.21.15.103:0/549242865 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0019070 con 0x7fa5c810c170 2024-09-19T06:59:39.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.549+0000 7fa5ce4ae640 1 -- 172.21.15.103:0/549242865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c819d730 con 0x7fa5c810c170 2024-09-19T06:59:39.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.549+0000 7fa5c57fa640 1 -- 172.21.15.103:0/549242865 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c000cb40 con 0x7fa5c810c170 2024-09-19T06:59:39.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.549+0000 7fa5c57fa640 1 -- 172.21.15.103:0/549242865 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0007650 con 0x7fa5c810c170 2024-09-19T06:59:39.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.549+0000 7fa5ce4ae640 1 -- 172.21.15.103:0/549242865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c81a5010 con 0x7fa5c810c170 2024-09-19T06:59:39.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.550+0000 7fa5ce4ae640 1 -- 172.21.15.103:0/549242865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c8103120 con 0x7fa5c810c170 2024-09-19T06:59:39.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.551+0000 7fa5c57fa640 1 -- 172.21.15.103:0/549242865 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa5c001c070 con 0x7fa5c810c170 2024-09-19T06:59:39.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.551+0000 7fa5c57fa640 1 --2- 172.21.15.103:0/549242865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5b00a33b0 0x7fa5b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:39.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.552+0000 7fa5c7fff640 1 --2- 172.21.15.103:0/549242865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5b00a33b0 0x7fa5b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:39.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.552+0000 7fa5c57fa640 1 -- 172.21.15.103:0/549242865 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7fa5c00cf6d0 con 0x7fa5c810c170 2024-09-19T06:59:39.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.552+0000 7fa5c7fff640 1 --2- 172.21.15.103:0/549242865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5b00a33b0 0x7fa5b00a5870 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8005ab0 tx=0x7fa5b8005a20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:39.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.555+0000 7fa5c57fa640 1 -- 172.21.15.103:0/549242865 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa5c009bb50 con 0x7fa5c810c170 2024-09-19T06:59:39.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.753+0000 7fa5ce4ae640 1 -- 172.21.15.103:0/549242865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5c81997f0 con 0x7fa5c810c170 2024-09-19T06:59:39.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.754+0000 7fa5c57fa640 1 -- 172.21.15.103:0/549242865 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fa5c81997f0 con 0x7fa5c810c170 2024-09-19T06:59:39.755 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:59:39.755 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:59:39.756 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:59:39.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.756+0000 7fa5ce4ae640 1 -- 172.21.15.103:0/549242865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5b00a33b0 msgr2=0x7fa5b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:39.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.756+0000 7fa5ce4ae640 1 --2- 172.21.15.103:0/549242865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5b00a33b0 0x7fa5b00a5870 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8005ab0 tx=0x7fa5b8005a20 comp rx=0 tx=0).stop 2024-09-19T06:59:39.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.756+0000 7fa5ce4ae640 1 -- 172.21.15.103:0/549242865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c810c170 msgr2=0x7fa5c819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:39.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.756+0000 7fa5ce4ae640 1 --2- 172.21.15.103:0/549242865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c810c170 0x7fa5c819cd60 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000b970 tx=0x7fa5c000be30 comp rx=0 tx=0).stop 2024-09-19T06:59:39.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.756+0000 7fa5ce4ae640 1 -- 172.21.15.103:0/549242865 shutdown_connections 2024-09-19T06:59:39.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.756+0000 7fa5ce4ae640 1 --2- 172.21.15.103:0/549242865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5b00a33b0 0x7fa5b00a5870 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:39.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.756+0000 7fa5ce4ae640 1 --2- 172.21.15.103:0/549242865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c810c170 0x7fa5c819cd60 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:39.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.756+0000 7fa5ce4ae640 1 --2- 172.21.15.103:0/549242865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c8103660 0x7fa5c81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:39.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.756+0000 7fa5ce4ae640 1 --2- 172.21.15.103:0/549242865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5c8102d40 0x7fa5c8198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:39.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.756+0000 7fa5ce4ae640 1 -- 172.21.15.103:0/549242865 >> 172.21.15.103:0/549242865 conn(0x7fa5c80fc7a0 msgr2=0x7fa5c80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:39.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.756+0000 7fa5ce4ae640 1 -- 172.21.15.103:0/549242865 shutdown_connections 2024-09-19T06:59:39.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:39.756+0000 7fa5ce4ae640 1 -- 172.21.15.103:0/549242865 wait complete. 2024-09-19T06:59:40.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:39 smithi103 ceph-mon[31158]: pgmap v323: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:59:40.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:39 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/549242865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:59:40.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:39 smithi134 ceph-mon[35356]: pgmap v323: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:59:40.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:39 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/549242865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:59:40.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:39 smithi174 ceph-mon[35637]: pgmap v323: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T06:59:40.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:39 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/549242865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:59:40.451 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-19T06:59:40.451 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-19T06:59:40.701 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:41.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.107+0000 7ff9a438b640 1 -- 172.21.15.103:0/1845156801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff99c100e20 msgr2=0x7ff99c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:41.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.107+0000 7ff9a438b640 1 --2- 172.21.15.103:0/1845156801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff99c100e20 0x7ff99c101200 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7ff998009900 tx=0x7ff99802a480 comp rx=0 tx=0).stop 2024-09-19T06:59:41.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.107+0000 7ff9a438b640 1 -- 172.21.15.103:0/1845156801 shutdown_connections 2024-09-19T06:59:41.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.107+0000 7ff9a438b640 1 --2- 172.21.15.103:0/1845156801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99c102ef0 0x7ff99c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.107+0000 7ff9a438b640 1 --2- 172.21.15.103:0/1845156801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff99c102570 0x7ff99c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.107+0000 7ff9a438b640 1 --2- 172.21.15.103:0/1845156801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff99c100e20 0x7ff99c101200 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.107+0000 7ff9a438b640 1 -- 172.21.15.103:0/1845156801 >> 172.21.15.103:0/1845156801 conn(0x7ff99c0fc7d0 msgr2=0x7ff99c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:41.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.107+0000 7ff9a438b640 1 -- 172.21.15.103:0/1845156801 shutdown_connections 2024-09-19T06:59:41.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.107+0000 7ff9a438b640 1 -- 172.21.15.103:0/1845156801 wait complete. 2024-09-19T06:59:41.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.108+0000 7ff9a438b640 1 Processor -- start 2024-09-19T06:59:41.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.108+0000 7ff9a438b640 1 -- start start 2024-09-19T06:59:41.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.108+0000 7ff9a438b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99c100e20 0x7ff99c197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.108+0000 7ff9a438b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff99c102570 0x7ff99c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.108+0000 7ff9a438b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff99c102ef0 0x7ff99c19bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.108+0000 7ff9a438b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c110fa0 con 0x7ff99c102ef0 2024-09-19T06:59:41.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.108+0000 7ff9a438b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c110e20 con 0x7ff99c100e20 2024-09-19T06:59:41.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.108+0000 7ff9a438b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c111120 con 0x7ff99c102570 2024-09-19T06:59:41.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.108+0000 7ff9a2928640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff99c102ef0 0x7ff99c19bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.108+0000 7ff9a2928640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff99c102ef0 0x7ff99c19bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58874/0 (socket says 172.21.15.103:58874) 2024-09-19T06:59:41.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.108+0000 7ff9a2928640 1 -- 172.21.15.103:0/1571260145 learned_addr learned my addr 172.21.15.103:0/1571260145 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:41.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.108+0000 7ff9a2127640 1 --2- 172.21.15.103:0/1571260145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99c100e20 0x7ff99c197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.108+0000 7ff9a1926640 1 --2- 172.21.15.103:0/1571260145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff99c102570 0x7ff99c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.109+0000 7ff9a2928640 1 -- 172.21.15.103:0/1571260145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff99c102570 msgr2=0x7ff99c197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:41.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.109+0000 7ff9a2928640 1 --2- 172.21.15.103:0/1571260145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff99c102570 0x7ff99c197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.109+0000 7ff9a2928640 1 -- 172.21.15.103:0/1571260145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99c100e20 msgr2=0x7ff99c197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:41.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.109+0000 7ff9a2928640 1 --2- 172.21.15.103:0/1571260145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99c100e20 0x7ff99c197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.109+0000 7ff9a2928640 1 -- 172.21.15.103:0/1571260145 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99c19c3b0 con 0x7ff99c102ef0 2024-09-19T06:59:41.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.109+0000 7ff9a2127640 1 --2- 172.21.15.103:0/1571260145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99c100e20 0x7ff99c197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:41.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.109+0000 7ff9a2928640 1 --2- 172.21.15.103:0/1571260145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff99c102ef0 0x7ff99c19bcd0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7ff99400cce0 tx=0x7ff99400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:41.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.109+0000 7ff9937fe640 1 -- 172.21.15.103:0/1571260145 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99400f450 con 0x7ff99c102ef0 2024-09-19T06:59:41.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.109+0000 7ff9937fe640 1 -- 172.21.15.103:0/1571260145 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff99400fad0 con 0x7ff99c102ef0 2024-09-19T06:59:41.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.109+0000 7ff9a438b640 1 -- 172.21.15.103:0/1571260145 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99c19c6a0 con 0x7ff99c102ef0 2024-09-19T06:59:41.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.109+0000 7ff9937fe640 1 -- 172.21.15.103:0/1571260145 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994018c10 con 0x7ff99c102ef0 2024-09-19T06:59:41.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.109+0000 7ff9a438b640 1 -- 172.21.15.103:0/1571260145 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99c1a3f80 con 0x7ff99c102ef0 2024-09-19T06:59:41.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.111+0000 7ff9937fe640 1 -- 172.21.15.103:0/1571260145 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff99400f5f0 con 0x7ff99c102ef0 2024-09-19T06:59:41.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.112+0000 7ff9937fe640 1 --2- 172.21.15.103:0/1571260145 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9880a33b0 0x7ff9880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.112+0000 7ff9a2127640 1 --2- 172.21.15.103:0/1571260145 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9880a33b0 0x7ff9880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.112+0000 7ff9937fe640 1 -- 172.21.15.103:0/1571260145 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7ff9940d0490 con 0x7ff99c102ef0 2024-09-19T06:59:41.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.112+0000 7ff9a2127640 1 --2- 172.21.15.103:0/1571260145 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9880a33b0 0x7ff9880a5870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff998005ab0 tx=0x7ff998005a20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:41.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.112+0000 7ff9a6764740 1 Processor -- start 2024-09-19T06:59:41.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a6764740 1 -- start start 2024-09-19T06:59:41.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a6764740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0ccfe5e0 0x564f0ccfe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a6764740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfef00 0x564f0cd0b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a6764740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0cd0bb50 0x564f0cd0df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a6764740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564f0cc642a0 con 0x564f0cd0bb50 2024-09-19T06:59:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a6764740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564f0cc0bb10 con 0x564f0ccfe5e0 2024-09-19T06:59:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a6764740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564f0cc60260 con 0x564f0ccfef00 2024-09-19T06:59:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a2928640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0cd0bb50 0x564f0cd0df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a2928640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0cd0bb50 0x564f0cd0df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58888/0 (socket says 172.21.15.103:58888) 2024-09-19T06:59:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a2928640 1 -- 172.21.15.103:0/548553227 learned_addr learned my addr 172.21.15.103:0/548553227 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a1926640 1 --2- 172.21.15.103:0/548553227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfef00 0x564f0cd0b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a2127640 1 --2- 172.21.15.103:0/548553227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0ccfe5e0 0x564f0ccfe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a2928640 1 -- 172.21.15.103:0/548553227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfef00 msgr2=0x564f0cd0b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a2928640 1 --2- 172.21.15.103:0/548553227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfef00 0x564f0cd0b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a2928640 1 -- 172.21.15.103:0/548553227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0ccfe5e0 msgr2=0x564f0ccfe9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a2928640 1 --2- 172.21.15.103:0/548553227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0ccfe5e0 0x564f0ccfe9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a2928640 1 -- 172.21.15.103:0/548553227 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f0c9d5b90 con 0x564f0cd0bb50 2024-09-19T06:59:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a1926640 1 --2- 172.21.15.103:0/548553227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfef00 0x564f0cd0b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.113+0000 7ff9a2127640 1 --2- 172.21.15.103:0/548553227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0ccfe5e0 0x564f0ccfe9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.114+0000 7ff9a2928640 1 --2- 172.21.15.103:0/548553227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0cd0bb50 0x564f0cd0df40 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7ff994098bb0 tx=0x7ff99409ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f896fb831e77fe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.114+0000 7ff990dea640 1 -- 172.21.15.103:0/548553227 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940b5070 con 0x564f0cd0bb50 2024-09-19T06:59:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.114+0000 7ff990dea640 1 -- 172.21.15.103:0/548553227 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9940a53c0 con 0x564f0cd0bb50 2024-09-19T06:59:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.114+0000 7ff990dea640 1 -- 172.21.15.103:0/548553227 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994098dc0 con 0x564f0cd0bb50 2024-09-19T06:59:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.114+0000 7ff9a6764740 1 -- 172.21.15.103:0/548553227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0cd0bb50 msgr2=0x564f0cd0df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.114+0000 7ff9a6764740 1 --2- 172.21.15.103:0/548553227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0cd0bb50 0x564f0cd0df40 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7ff994098bb0 tx=0x7ff99409ce50 comp rx=0 tx=0).stop 2024-09-19T06:59:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.114+0000 7ff9a6764740 1 -- 172.21.15.103:0/548553227 shutdown_connections 2024-09-19T06:59:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.114+0000 7ff9a6764740 1 --2- 172.21.15.103:0/548553227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0cd0bb50 0x564f0cd0df40 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a6764740 1 --2- 172.21.15.103:0/548553227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfef00 0x564f0cd0b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a6764740 1 --2- 172.21.15.103:0/548553227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0ccfe5e0 0x564f0ccfe9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a6764740 1 -- 172.21.15.103:0/548553227 >> 172.21.15.103:0/548553227 conn(0x564f0ccfacf0 msgr2=0x564f0ccfb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a6764740 1 -- 172.21.15.103:0/548553227 shutdown_connections 2024-09-19T06:59:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a6764740 1 -- 172.21.15.103:0/548553227 wait complete. 2024-09-19T06:59:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a6764740 1 -- start start 2024-09-19T06:59:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a6764740 1 Processor -- start 2024-09-19T06:59:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a6764740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfe5e0 0x564f0cd08b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a6764740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0ccfef00 0x564f0cd090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a6764740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0cd0bb50 0x564f0cd095e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a6764740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564f0cd0e780 con 0x564f0cd0bb50 2024-09-19T06:59:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a6764740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564f0cbec6e0 con 0x564f0ccfef00 2024-09-19T06:59:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a6764740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564f0cd0e900 con 0x564f0ccfe5e0 2024-09-19T06:59:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a6764740 10 client.0 _ll_register_callbacks cb 0x564f0cc10a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a2928640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0cd0bb50 0x564f0cd095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a1926640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0ccfef00 0x564f0cd090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a2928640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0cd0bb50 0x564f0cd095e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58902/0 (socket says 172.21.15.103:58902) 2024-09-19T06:59:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.115+0000 7ff9a2928640 1 -- 172.21.15.103:0/633138887 learned_addr learned my addr 172.21.15.103:0/633138887 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.116+0000 7ff9a2127640 1 --2- 172.21.15.103:0/633138887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfe5e0 0x564f0cd08b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.116+0000 7ff9a2928640 1 -- 172.21.15.103:0/633138887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfe5e0 msgr2=0x564f0cd08b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.116+0000 7ff9a2928640 1 --2- 172.21.15.103:0/633138887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfe5e0 0x564f0cd08b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.116+0000 7ff9a2928640 1 -- 172.21.15.103:0/633138887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0ccfef00 msgr2=0x564f0cd090a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.116+0000 7ff9a2928640 1 --2- 172.21.15.103:0/633138887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0ccfef00 0x564f0cd090a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.116+0000 7ff9a2928640 1 -- 172.21.15.103:0/633138887 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f0cd17dd0 con 0x564f0cd0bb50 2024-09-19T06:59:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.116+0000 7ff9a2928640 1 --2- 172.21.15.103:0/633138887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0cd0bb50 0x564f0cd095e0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7ff99409d000 tx=0x7ff9940cde60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad36cf4f1e04195 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.116+0000 7ff985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.116+0000 7ff985ffb640 1 -- 172.21.15.103:0/633138887 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940ba060 con 0x564f0cd0bb50 2024-09-19T06:59:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.116+0000 7ff985ffb640 1 -- 172.21.15.103:0/633138887 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7ff99400fc80 con 0x564f0cd0bb50 2024-09-19T06:59:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.116+0000 7ff9a6764740 1 -- 172.21.15.103:0/633138887 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9940a53c0 con 0x564f0cd0bb50 2024-09-19T06:59:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.116+0000 7ff985ffb640 1 -- 172.21.15.103:0/633138887 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff9940b5040 con 0x564f0cd0bb50 2024-09-19T06:59:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.116+0000 7ff9a6764740 10 client.19002 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.116+0000 7ff9a6764740 1 -- 172.21.15.103:0/633138887 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564f0cd1a6d0 con 0x564f0cd0bb50 2024-09-19T06:59:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.116+0000 7ff985ffb640 1 -- 172.21.15.103:0/633138887 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9940a53c0 con 0x564f0cd0bb50 2024-09-19T06:59:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.117+0000 7ff985ffb640 1 -- 172.21.15.103:0/633138887 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff99409b610 con 0x564f0cd0bb50 2024-09-19T06:59:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.117+0000 7ff9a6764740 10 client.19002 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.117+0000 7ff9a6764740 10 client.19002 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:59:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.117+0000 7ff9a6764740 20 client.19002 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.117+0000 7ff9a6764740 1 --2- 172.21.15.103:0/633138887 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564f0cd1ba20 0x564f0cd1de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.117+0000 7ff9a6764740 4 client.19002 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.117+0000 7ff9a6764740 1 -- 172.21.15.103:0/633138887 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f0cc719d0 con 0x564f0cd1ba20 2024-09-19T06:59:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.117+0000 7ff9a2127640 1 --2- 172.21.15.103:0/633138887 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564f0cd1ba20 0x564f0cd1de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.117+0000 7ff985ffb640 10 client.19002 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.117+0000 7ff9a2127640 1 --2- 172.21.15.103:0/633138887 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564f0cd1ba20 0x564f0cd1de10 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf69ebaa883971f7 server_cookie=7a84b513d30759b3 in_seq=0 out_seq=0 2024-09-19T06:59:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.118+0000 7ff985ffb640 1 -- 172.21.15.103:0/633138887 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f0cc719d0 con 0x564f0cd1ba20 2024-09-19T06:59:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.118+0000 7ff985ffb640 10 client.19002 handle_command_reply: tid=0 2024-09-19T06:59:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.118+0000 7ff9a6764740 1 client.19002 shutdown 2024-09-19T06:59:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.118+0000 7ff9a6764740 1 -- 172.21.15.103:0/633138887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0cd0bb50 msgr2=0x564f0cd095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.118+0000 7ff9a6764740 1 --2- 172.21.15.103:0/633138887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0cd0bb50 0x564f0cd095e0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7ff99409d000 tx=0x7ff9940cde60 comp rx=0 tx=0).stop 2024-09-19T06:59:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.118+0000 7ff9a6764740 1 -- 172.21.15.103:0/633138887 shutdown_connections 2024-09-19T06:59:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.118+0000 7ff9a6764740 1 --2- 172.21.15.103:0/633138887 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564f0cd1ba20 0x564f0cd1de10 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.118+0000 7ff9a6764740 1 --2- 172.21.15.103:0/633138887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0cd0bb50 0x564f0cd095e0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.118+0000 7ff9a6764740 1 --2- 172.21.15.103:0/633138887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0ccfef00 0x564f0cd090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.118+0000 7ff9a6764740 1 --2- 172.21.15.103:0/633138887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfe5e0 0x564f0cd08b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.118+0000 7ff9a6764740 1 -- 172.21.15.103:0/633138887 >> 172.21.15.103:0/633138887 conn(0x564f0ccfacf0 msgr2=0x564f0cd009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.118+0000 7ff985ffb640 0 client.19002 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:41.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.119+0000 7ff9a6764740 1 -- 172.21.15.103:0/633138887 shutdown_connections 2024-09-19T06:59:41.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.119+0000 7ff9a6764740 1 -- 172.21.15.103:0/633138887 wait complete. 2024-09-19T06:59:41.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.119+0000 7ff9a6764740 20 client.19002 trim_cache size 0 max 16384 2024-09-19T06:59:41.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.135+0000 7ff9a6764740 1 Processor -- start 2024-09-19T06:59:41.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.135+0000 7ff9a6764740 1 -- start start 2024-09-19T06:59:41.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.135+0000 7ff9a6764740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfe5e0 0x564f0cd17260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.135+0000 7ff9a6764740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0ccfef00 0x564f0cd20e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.135+0000 7ff9a6764740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0cd0bb50 0x564f0cd21380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.135+0000 7ff9a6764740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1ab6c0 con 0x564f0ccfef00 2024-09-19T06:59:41.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.135+0000 7ff9a6764740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1ad200 con 0x564f0cd0bb50 2024-09-19T06:59:41.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.135+0000 7ff9a6764740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1abd90 con 0x564f0ccfe5e0 2024-09-19T06:59:41.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.136+0000 7ff9a1926640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0ccfef00 0x564f0cd20e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.136+0000 7ff9a1926640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0ccfef00 0x564f0cd20e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58904/0 (socket says 172.21.15.103:58904) 2024-09-19T06:59:41.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.136+0000 7ff9a1926640 1 -- 172.21.15.103:0/2279273419 learned_addr learned my addr 172.21.15.103:0/2279273419 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:41.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.136+0000 7ff9a2127640 1 --2- 172.21.15.103:0/2279273419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfe5e0 0x564f0cd17260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.136+0000 7ff9a1926640 1 -- 172.21.15.103:0/2279273419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfe5e0 msgr2=0x564f0cd17260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:41.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.136+0000 7ff9a1926640 1 --2- 172.21.15.103:0/2279273419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfe5e0 0x564f0cd17260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.136+0000 7ff9a1926640 1 -- 172.21.15.103:0/2279273419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0cd0bb50 msgr2=0x564f0cd21380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:41.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.136+0000 7ff9a1926640 1 --2- 172.21.15.103:0/2279273419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0cd0bb50 0x564f0cd21380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.136+0000 7ff9a1926640 1 -- 172.21.15.103:0/2279273419 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f0cd04510 con 0x564f0ccfef00 2024-09-19T06:59:41.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.136+0000 7ff9a1926640 1 --2- 172.21.15.103:0/2279273419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0ccfef00 0x564f0cd20e40 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00b6d0 tx=0x7ff98c00bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57b9106d0f19a6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:41.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.136+0000 7ff987fff640 1 -- 172.21.15.103:0/2279273419 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c019030 con 0x564f0ccfef00 2024-09-19T06:59:41.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.136+0000 7ff987fff640 1 -- 172.21.15.103:0/2279273419 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff98c00bde0 con 0x564f0ccfef00 2024-09-19T06:59:41.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.136+0000 7ff987fff640 1 -- 172.21.15.103:0/2279273419 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c015c40 con 0x564f0ccfef00 2024-09-19T06:59:41.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.137+0000 7ff9a6764740 1 -- 172.21.15.103:0/2279273419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0ccfef00 msgr2=0x564f0cd20e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:41.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.137+0000 7ff9a6764740 1 --2- 172.21.15.103:0/2279273419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0ccfef00 0x564f0cd20e40 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00b6d0 tx=0x7ff98c00bb90 comp rx=0 tx=0).stop 2024-09-19T06:59:41.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.137+0000 7ff9a6764740 1 -- 172.21.15.103:0/2279273419 shutdown_connections 2024-09-19T06:59:41.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.137+0000 7ff9a6764740 1 --2- 172.21.15.103:0/2279273419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0cd0bb50 0x564f0cd21380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.137+0000 7ff9a6764740 1 --2- 172.21.15.103:0/2279273419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0ccfef00 0x564f0cd20e40 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.137+0000 7ff9a6764740 1 --2- 172.21.15.103:0/2279273419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfe5e0 0x564f0cd17260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.137+0000 7ff9a6764740 1 -- 172.21.15.103:0/2279273419 >> 172.21.15.103:0/2279273419 conn(0x564f0ccfacf0 msgr2=0x564f0cd16b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:41.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.137+0000 7ff9a6764740 1 -- 172.21.15.103:0/2279273419 shutdown_connections 2024-09-19T06:59:41.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.137+0000 7ff9a6764740 1 -- 172.21.15.103:0/2279273419 wait complete. 2024-09-19T06:59:41.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.137+0000 7ff9a6764740 1 -- start start 2024-09-19T06:59:41.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.137+0000 7ff9a6764740 1 Processor -- start 2024-09-19T06:59:41.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a6764740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfe5e0 0x564f0cd80d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a6764740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0ccfef00 0x564f0cd81240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a6764740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0cd0bb50 0x564f0cd02420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a6764740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1acb30 con 0x564f0ccfef00 2024-09-19T06:59:41.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a6764740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1ac460 con 0x564f0cd0bb50 2024-09-19T06:59:41.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a6764740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1a96c0 con 0x564f0ccfe5e0 2024-09-19T06:59:41.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a6764740 10 client.0 _ll_register_callbacks cb 0x7ff99409c360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:41.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a1926640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0ccfef00 0x564f0cd81240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a1926640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0ccfef00 0x564f0cd81240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58908/0 (socket says 172.21.15.103:58908) 2024-09-19T06:59:41.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a1926640 1 -- 172.21.15.103:0/2468141494 learned_addr learned my addr 172.21.15.103:0/2468141494 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:41.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a2127640 1 --2- 172.21.15.103:0/2468141494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfe5e0 0x564f0cd80d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a2928640 1 --2- 172.21.15.103:0/2468141494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0cd0bb50 0x564f0cd02420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a1926640 1 -- 172.21.15.103:0/2468141494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfe5e0 msgr2=0x564f0cd80d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:41.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a1926640 1 --2- 172.21.15.103:0/2468141494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfe5e0 0x564f0cd80d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a1926640 1 -- 172.21.15.103:0/2468141494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0cd0bb50 msgr2=0x564f0cd02420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:41.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a1926640 1 --2- 172.21.15.103:0/2468141494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0cd0bb50 0x564f0cd02420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a1926640 1 -- 172.21.15.103:0/2468141494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f0cd02b60 con 0x564f0ccfef00 2024-09-19T06:59:41.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a2928640 1 --2- 172.21.15.103:0/2468141494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0cd0bb50 0x564f0cd02420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:41.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a1926640 1 --2- 172.21.15.103:0/2468141494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0ccfef00 0x564f0cd81240 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7ff98c015b20 tx=0x7ff98c0158d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e2bc3d67e14908 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:41.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:41.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff985ffb640 1 -- 172.21.15.103:0/2468141494 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c019040 con 0x564f0ccfef00 2024-09-19T06:59:41.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff985ffb640 1 -- 172.21.15.103:0/2468141494 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7ff98c012cf0 con 0x564f0ccfef00 2024-09-19T06:59:41.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.138+0000 7ff9a6764740 1 -- 172.21.15.103:0/2468141494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff98c00bde0 con 0x564f0ccfef00 2024-09-19T06:59:41.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.139+0000 7ff985ffb640 1 -- 172.21.15.103:0/2468141494 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff98c0059c0 con 0x564f0ccfef00 2024-09-19T06:59:41.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.139+0000 7ff9a6764740 10 client.19008 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:41.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.139+0000 7ff9a6764740 1 -- 172.21.15.103:0/2468141494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564f0cd88530 con 0x564f0ccfef00 2024-09-19T06:59:41.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.139+0000 7ff985ffb640 1 -- 172.21.15.103:0/2468141494 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff98c00bde0 con 0x564f0ccfef00 2024-09-19T06:59:41.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.139+0000 7ff985ffb640 1 -- 172.21.15.103:0/2468141494 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff98c025070 con 0x564f0ccfef00 2024-09-19T06:59:41.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.139+0000 7ff9a6764740 10 client.19008 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:41.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.139+0000 7ff9a6764740 10 client.19008 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T06:59:41.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.139+0000 7ff9a6764740 20 client.19008 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:41.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.139+0000 7ff9a6764740 1 --2- 172.21.15.103:0/2468141494 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564f0cd88c70 0x564f0cd8b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:41.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.139+0000 7ff9a6764740 4 client.19008 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-19T06:59:41.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.139+0000 7ff9a6764740 1 -- 172.21.15.103:0/2468141494 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x564f0cc6bd20 con 0x564f0cd88c70 2024-09-19T06:59:41.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.139+0000 7ff9a2127640 1 --2- 172.21.15.103:0/2468141494 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564f0cd88c70 0x564f0cd8b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:41.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.140+0000 7ff985ffb640 10 client.19008 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:41.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.140+0000 7ff9a2127640 1 --2- 172.21.15.103:0/2468141494 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564f0cd88c70 0x564f0cd8b090 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42905ee31e4ae17e server_cookie=3d17510c9738dd6 in_seq=0 out_seq=0 2024-09-19T06:59:41.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.150+0000 7ff985ffb640 1 -- 172.21.15.103:0/2468141494 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564f0cc6bd20 con 0x564f0cd88c70 2024-09-19T06:59:41.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.150+0000 7ff985ffb640 10 client.19008 handle_command_reply: tid=0 2024-09-19T06:59:41.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.150+0000 7ff9a6764740 1 client.19008 shutdown 2024-09-19T06:59:41.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.150+0000 7ff9a6764740 1 -- 172.21.15.103:0/2468141494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0ccfef00 msgr2=0x564f0cd81240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:41.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.150+0000 7ff9a6764740 1 --2- 172.21.15.103:0/2468141494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0ccfef00 0x564f0cd81240 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7ff98c015b20 tx=0x7ff98c0158d0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.150+0000 7ff9a6764740 1 -- 172.21.15.103:0/2468141494 shutdown_connections 2024-09-19T06:59:41.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.150+0000 7ff9a6764740 1 --2- 172.21.15.103:0/2468141494 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564f0cd88c70 0x564f0cd8b090 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.150+0000 7ff9a6764740 1 --2- 172.21.15.103:0/2468141494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f0cd0bb50 0x564f0cd02420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.150+0000 7ff9a6764740 1 --2- 172.21.15.103:0/2468141494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f0ccfef00 0x564f0cd81240 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.150+0000 7ff9a6764740 1 --2- 172.21.15.103:0/2468141494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f0ccfe5e0 0x564f0cd80d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.150+0000 7ff9a6764740 1 -- 172.21.15.103:0/2468141494 >> 172.21.15.103:0/2468141494 conn(0x564f0ccfacf0 msgr2=0x564f0cd16b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:41.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.150+0000 7ff985ffb640 0 client.19008 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:41.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.150+0000 7ff9a6764740 1 -- 172.21.15.103:0/2468141494 shutdown_connections 2024-09-19T06:59:41.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.150+0000 7ff9a6764740 1 -- 172.21.15.103:0/2468141494 wait complete. 2024-09-19T06:59:41.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.150+0000 7ff9a6764740 20 client.19008 trim_cache size 0 max 16384 2024-09-19T06:59:41.153 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:59:41.153 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T06:59:41.153 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T06:59:41.153 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 38, 2024-09-19T06:59:41.153 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T06:59:41.153 INFO:teuthology.orchestra.run.smithi103.stdout: "5003d0af": { 2024-09-19T06:59:41.153 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 38, 2024-09-19T06:59:41.154 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T06:59:41.154 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:59:41.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:59:41.154 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:59:41.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T06:59:41.154 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T06:59:41.154 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 150.0, 2024-09-19T06:59:41.154 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T06:59:41.154 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T06:59:41.154 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T06:59:41.154 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T06:59:41.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T06:59:41.154 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T06:59:41.154 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:41.154 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:41.155 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:41.155 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:41.155 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T06:59:41.155 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:59:41.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.151+0000 7ff9a438b640 1 -- 172.21.15.103:0/1571260145 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9880a33b0 msgr2=0x7ff9880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:41.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.151+0000 7ff9a438b640 1 --2- 172.21.15.103:0/1571260145 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9880a33b0 0x7ff9880a5870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff998005ab0 tx=0x7ff998005a20 comp rx=0 tx=0).stop 2024-09-19T06:59:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.151+0000 7ff9a438b640 1 -- 172.21.15.103:0/1571260145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff99c102ef0 msgr2=0x7ff99c19bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.151+0000 7ff9a438b640 1 --2- 172.21.15.103:0/1571260145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff99c102ef0 0x7ff99c19bcd0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7ff99400cce0 tx=0x7ff99400a620 comp rx=0 tx=0).stop 2024-09-19T06:59:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.151+0000 7ff9a438b640 1 -- 172.21.15.103:0/1571260145 shutdown_connections 2024-09-19T06:59:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.151+0000 7ff9a438b640 1 --2- 172.21.15.103:0/1571260145 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9880a33b0 0x7ff9880a5870 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.151+0000 7ff9a438b640 1 --2- 172.21.15.103:0/1571260145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff99c102ef0 0x7ff99c19bcd0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.151+0000 7ff9a438b640 1 --2- 172.21.15.103:0/1571260145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff99c102570 0x7ff99c197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.151+0000 7ff9a438b640 1 --2- 172.21.15.103:0/1571260145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff99c100e20 0x7ff99c197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.151+0000 7ff9a438b640 1 -- 172.21.15.103:0/1571260145 >> 172.21.15.103:0/1571260145 conn(0x7ff99c0fc7d0 msgr2=0x7ff99c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.151+0000 7ff9a438b640 1 -- 172.21.15.103:0/1571260145 shutdown_connections 2024-09-19T06:59:41.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:41.151+0000 7ff9a438b640 1 -- 172.21.15.103:0/1571260145 wait complete. 2024-09-19T06:59:41.831 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5003d0af', quiesce took 3.0 seconds. Will release after: 27.0 2024-09-19T06:59:42.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:41 smithi103 ceph-mon[31158]: pgmap v324: 97 pgs: 97 active+clean; 2.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T06:59:42.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:41 smithi134 ceph-mon[35356]: pgmap v324: 97 pgs: 97 active+clean; 2.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T06:59:42.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:41 smithi174 ceph-mon[35637]: pgmap v324: 97 pgs: 97 active+clean; 2.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T06:59:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.781+0000 7f8c4071f640 1 -- 172.21.15.103:0/3789352572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c38101390 msgr2=0x7f8c3810d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.781+0000 7f8c4071f640 1 --2- 172.21.15.103:0/3789352572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c38101390 0x7f8c3810d8c0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009cd0 tx=0x7f8c3002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.781+0000 7f8c4071f640 1 -- 172.21.15.103:0/3789352572 shutdown_connections 2024-09-19T06:59:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.781+0000 7f8c4071f640 1 --2- 172.21.15.103:0/3789352572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c38101390 0x7f8c3810d8c0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.781+0000 7f8c4071f640 1 --2- 172.21.15.103:0/3789352572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c381009f0 0x7f8c38100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.781+0000 7f8c4071f640 1 --2- 172.21.15.103:0/3789352572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c38106a00 0x7f8c38106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.781+0000 7f8c4071f640 1 -- 172.21.15.103:0/3789352572 >> 172.21.15.103:0/3789352572 conn(0x7f8c380fc6b0 msgr2=0x7f8c380fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.781+0000 7f8c4071f640 1 -- 172.21.15.103:0/3789352572 shutdown_connections 2024-09-19T06:59:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.781+0000 7f8c4071f640 1 -- 172.21.15.103:0/3789352572 wait complete. 2024-09-19T06:59:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c4071f640 1 Processor -- start 2024-09-19T06:59:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c4071f640 1 -- start start 2024-09-19T06:59:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c4071f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c381009f0 0x7f8c38197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c4071f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c38101390 0x7f8c381977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c4071f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c38106a00 0x7f8c3819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c4071f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8c38110e40 con 0x7f8c38101390 2024-09-19T06:59:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c4071f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c38110cc0 con 0x7f8c381009f0 2024-09-19T06:59:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c4071f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c38110fc0 con 0x7f8c38106a00 2024-09-19T06:59:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c3dcba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c38101390 0x7f8c381977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c3dcba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c38101390 0x7f8c381977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58912/0 (socket says 172.21.15.103:58912) 2024-09-19T06:59:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c3dcba640 1 -- 172.21.15.103:0/1642780549 learned_addr learned my addr 172.21.15.103:0/1642780549 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c3ecbc640 1 --2- 172.21.15.103:0/1642780549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c38106a00 0x7f8c3819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c3e4bb640 1 --2- 172.21.15.103:0/1642780549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c381009f0 0x7f8c38197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c3dcba640 1 -- 172.21.15.103:0/1642780549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c38106a00 msgr2=0x7f8c3819bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c3dcba640 1 --2- 172.21.15.103:0/1642780549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c38106a00 0x7f8c3819bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c3dcba640 1 -- 172.21.15.103:0/1642780549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c381009f0 msgr2=0x7f8c38197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c3dcba640 1 --2- 172.21.15.103:0/1642780549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c381009f0 0x7f8c38197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c3dcba640 1 -- 172.21.15.103:0/1642780549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3819c260 con 0x7f8c38101390 2024-09-19T06:59:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.782+0000 7f8c3e4bb640 1 --2- 172.21.15.103:0/1642780549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c381009f0 0x7f8c38197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.783+0000 7f8c3dcba640 1 --2- 172.21.15.103:0/1642780549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c38101390 0x7f8c381977b0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800d950 tx=0x7f8c2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.783+0000 7f8c2f7fe640 1 -- 172.21.15.103:0/1642780549 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28017070 con 0x7f8c38101390 2024-09-19T06:59:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.783+0000 7f8c2f7fe640 1 -- 172.21.15.103:0/1642780549 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c28009c30 con 0x7f8c38101390 2024-09-19T06:59:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.783+0000 7f8c4071f640 1 -- 172.21.15.103:0/1642780549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3819c550 con 0x7f8c38101390 2024-09-19T06:59:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.783+0000 7f8c2f7fe640 1 -- 172.21.15.103:0/1642780549 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2800f6b0 con 0x7f8c38101390 2024-09-19T06:59:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.783+0000 7f8c4071f640 1 -- 172.21.15.103:0/1642780549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c381a3de0 con 0x7f8c38101390 2024-09-19T06:59:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.785+0000 7f8c2f7fe640 1 -- 172.21.15.103:0/1642780549 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8c280109d0 con 0x7f8c38101390 2024-09-19T06:59:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.785+0000 7f8c434b4740 1 Processor -- start 2024-09-19T06:59:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c434b4740 1 -- start start 2024-09-19T06:59:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c434b4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55573375c970 0x55573375cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c434b4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55573375d290 0x555733769980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c434b4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555733769ec0 0x55573376c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c434b4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5557336c12f0 con 0x555733769ec0 2024-09-19T06:59:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c434b4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555733669270 con 0x55573375d290 2024-09-19T06:59:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c434b4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5557336bd2b0 con 0x55573375c970 2024-09-19T06:59:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c3dcba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555733769ec0 0x55573376c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c3dcba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555733769ec0 0x55573376c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58926/0 (socket says 172.21.15.103:58926) 2024-09-19T06:59:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c3dcba640 1 -- 172.21.15.103:0/18991440 learned_addr learned my addr 172.21.15.103:0/18991440 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c3ecbc640 1 --2- 172.21.15.103:0/18991440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55573375c970 0x55573375cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c2f7fe640 1 --2- 172.21.15.103:0/1642780549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8c240a34e0 0x7f8c240a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c3e4bb640 1 --2- 172.21.15.103:0/18991440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55573375d290 0x555733769980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c3ecbc640 1 --2- 172.21.15.103:0/1642780549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8c240a34e0 0x7f8c240a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c3dcba640 1 -- 172.21.15.103:0/18991440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55573375c970 msgr2=0x55573375cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c3dcba640 1 --2- 172.21.15.103:0/18991440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55573375c970 0x55573375cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c3dcba640 1 -- 172.21.15.103:0/18991440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55573375d290 msgr2=0x555733769980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c3dcba640 1 --2- 172.21.15.103:0/18991440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55573375d290 0x555733769980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.786+0000 7f8c3dcba640 1 -- 172.21.15.103:0/18991440 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555733668340 con 0x555733769ec0 2024-09-19T06:59:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.787+0000 7f8c2f7fe640 1 -- 172.21.15.103:0/1642780549 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f8c280cf8d0 con 0x7f8c38101390 2024-09-19T06:59:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.787+0000 7f8c3dcba640 1 --2- 172.21.15.103:0/18991440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555733769ec0 0x55573376c2b0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f8c280ccf70 tx=0x7f8c28022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd0bb5a0235d9f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.787+0000 7f8c3ecbc640 1 --2- 172.21.15.103:0/1642780549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8c240a34e0 0x7f8c240a59a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8c30004ab0 tx=0x7f8c3002a8e0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.787+0000 7f8c2cff9640 1 -- 172.21.15.103:0/18991440 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2809b6d0 con 0x555733769ec0 2024-09-19T06:59:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.787+0000 7f8c2cff9640 1 -- 172.21.15.103:0/18991440 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c28009de0 con 0x555733769ec0 2024-09-19T06:59:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.787+0000 7f8c2cff9640 1 -- 172.21.15.103:0/18991440 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2809b4f0 con 0x555733769ec0 2024-09-19T06:59:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 -- 172.21.15.103:0/18991440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555733769ec0 msgr2=0x55573376c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 --2- 172.21.15.103:0/18991440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555733769ec0 0x55573376c2b0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f8c280ccf70 tx=0x7f8c28022000 comp rx=0 tx=0).stop 2024-09-19T06:59:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 -- 172.21.15.103:0/18991440 shutdown_connections 2024-09-19T06:59:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 --2- 172.21.15.103:0/18991440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555733769ec0 0x55573376c2b0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 --2- 172.21.15.103:0/18991440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55573375d290 0x555733769980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 --2- 172.21.15.103:0/18991440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55573375c970 0x55573375cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 -- 172.21.15.103:0/18991440 >> 172.21.15.103:0/18991440 conn(0x555733759060 msgr2=0x555733759440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 -- 172.21.15.103:0/18991440 shutdown_connections 2024-09-19T06:59:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 -- 172.21.15.103:0/18991440 wait complete. 2024-09-19T06:59:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 -- start start 2024-09-19T06:59:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 Processor -- start 2024-09-19T06:59:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375c970 0x555733766de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55573375d290 0x555733767320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555733769ec0 0x555733767860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55573376ca30 con 0x55573375c970 2024-09-19T06:59:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55573364a290 con 0x55573375d290 2024-09-19T06:59:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55573376cbb0 con 0x555733769ec0 2024-09-19T06:59:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.788+0000 7f8c434b4740 10 client.0 _ll_register_callbacks cb 0x55573367b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c3e4bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375c970 0x555733766de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c3dcba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55573375d290 0x555733767320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c3e4bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375c970 0x555733766de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58942/0 (socket says 172.21.15.103:58942) 2024-09-19T06:59:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c3e4bb640 1 -- 172.21.15.103:0/1153584508 learned_addr learned my addr 172.21.15.103:0/1153584508 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c3ecbc640 1 --2- 172.21.15.103:0/1153584508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555733769ec0 0x555733767860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c3e4bb640 1 -- 172.21.15.103:0/1153584508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555733769ec0 msgr2=0x555733767860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c3e4bb640 1 --2- 172.21.15.103:0/1153584508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555733769ec0 0x555733767860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c3e4bb640 1 -- 172.21.15.103:0/1153584508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55573375d290 msgr2=0x555733767320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c3e4bb640 1 --2- 172.21.15.103:0/1153584508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55573375d290 0x555733767320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c3e4bb640 1 -- 172.21.15.103:0/1153584508 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557333d4210 con 0x55573375c970 2024-09-19T06:59:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c3e4bb640 1 --2- 172.21.15.103:0/1153584508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375c970 0x555733766de0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f8c340106d0 tx=0x7f8c34010b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3ed3bc4685773f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c21ffb640 1 -- 172.21.15.103:0/1153584508 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3400ead0 con 0x55573375c970 2024-09-19T06:59:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c21ffb640 1 -- 172.21.15.103:0/1153584508 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f8c3400ec70 con 0x55573375c970 2024-09-19T06:59:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c434b4740 1 -- 172.21.15.103:0/1153584508 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c28009de0 con 0x55573375c970 2024-09-19T06:59:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c21ffb640 1 -- 172.21.15.103:0/1153584508 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8c3400ee70 con 0x55573375c970 2024-09-19T06:59:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c434b4740 10 client.19020 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.789+0000 7f8c434b4740 1 -- 172.21.15.103:0/1153584508 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5557334a0620 con 0x55573375c970 2024-09-19T06:59:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.790+0000 7f8c21ffb640 1 -- 172.21.15.103:0/1153584508 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c28009de0 con 0x55573375c970 2024-09-19T06:59:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.790+0000 7f8c21ffb640 1 -- 172.21.15.103:0/1153584508 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8c3400f040 con 0x55573375c970 2024-09-19T06:59:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.790+0000 7f8c434b4740 10 client.19020 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.790+0000 7f8c434b4740 10 client.19020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.790+0000 7f8c434b4740 20 client.19020 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.790+0000 7f8c434b4740 1 --2- 172.21.15.103:0/1153584508 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555733779e90 0x55573377c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.790+0000 7f8c434b4740 4 client.19020 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.790+0000 7f8c434b4740 1 -- 172.21.15.103:0/1153584508 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557336cea20 con 0x555733779e90 2024-09-19T06:59:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.790+0000 7f8c3dcba640 1 --2- 172.21.15.103:0/1153584508 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555733779e90 0x55573377c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.791+0000 7f8c3dcba640 1 --2- 172.21.15.103:0/1153584508 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555733779e90 0x55573377c2a0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7b0430f2b47cf20 server_cookie=59565b7883c67c91 in_seq=0 out_seq=0 2024-09-19T06:59:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.791+0000 7f8c21ffb640 10 client.19020 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.791+0000 7f8c21ffb640 1 -- 172.21.15.103:0/1153584508 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557336cea20 con 0x555733779e90 2024-09-19T06:59:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.791+0000 7f8c21ffb640 10 client.19020 handle_command_reply: tid=0 2024-09-19T06:59:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.791+0000 7f8c434b4740 1 client.19020 shutdown 2024-09-19T06:59:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.791+0000 7f8c434b4740 1 -- 172.21.15.103:0/1153584508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375c970 msgr2=0x555733766de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.791+0000 7f8c434b4740 1 --2- 172.21.15.103:0/1153584508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375c970 0x555733766de0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f8c340106d0 tx=0x7f8c34010b90 comp rx=0 tx=0).stop 2024-09-19T06:59:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.791+0000 7f8c434b4740 1 -- 172.21.15.103:0/1153584508 shutdown_connections 2024-09-19T06:59:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.792+0000 7f8c434b4740 1 --2- 172.21.15.103:0/1153584508 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555733779e90 0x55573377c2a0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.792+0000 7f8c434b4740 1 --2- 172.21.15.103:0/1153584508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555733769ec0 0x555733767860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.792+0000 7f8c21ffb640 0 client.19020 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.792+0000 7f8c434b4740 1 --2- 172.21.15.103:0/1153584508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55573375d290 0x555733767320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.792+0000 7f8c434b4740 1 --2- 172.21.15.103:0/1153584508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375c970 0x555733766de0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.792+0000 7f8c434b4740 1 -- 172.21.15.103:0/1153584508 >> 172.21.15.103:0/1153584508 conn(0x555733759060 msgr2=0x55573375edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.792+0000 7f8c434b4740 1 -- 172.21.15.103:0/1153584508 shutdown_connections 2024-09-19T06:59:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.792+0000 7f8c434b4740 1 -- 172.21.15.103:0/1153584508 wait complete. 2024-09-19T06:59:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.792+0000 7f8c434b4740 20 client.19020 trim_cache size 0 max 16384 2024-09-19T06:59:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.806+0000 7f8c434b4740 1 Processor -- start 2024-09-19T06:59:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.806+0000 7f8c434b4740 1 -- start start 2024-09-19T06:59:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.806+0000 7f8c434b4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375c970 0x555733768d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.806+0000 7f8c434b4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55573375d290 0x555733772210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.806+0000 7f8c434b4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555733769ec0 0x555733772750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.806+0000 7f8c434b4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381ab640 con 0x55573375c970 2024-09-19T06:59:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.806+0000 7f8c434b4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381ad180 con 0x555733769ec0 2024-09-19T06:59:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.806+0000 7f8c434b4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381abd10 con 0x55573375d290 2024-09-19T06:59:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.806+0000 7f8c3e4bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375c970 0x555733768d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.806+0000 7f8c3e4bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375c970 0x555733768d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58944/0 (socket says 172.21.15.103:58944) 2024-09-19T06:59:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.806+0000 7f8c3e4bb640 1 -- 172.21.15.103:0/2723147515 learned_addr learned my addr 172.21.15.103:0/2723147515 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.807+0000 7f8c3ecbc640 1 --2- 172.21.15.103:0/2723147515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555733769ec0 0x555733772750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.807+0000 7f8c3e4bb640 1 -- 172.21.15.103:0/2723147515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55573375d290 msgr2=0x555733772210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.807+0000 7f8c3e4bb640 1 --2- 172.21.15.103:0/2723147515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55573375d290 0x555733772210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.807+0000 7f8c3e4bb640 1 -- 172.21.15.103:0/2723147515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555733769ec0 msgr2=0x555733772750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.807+0000 7f8c3e4bb640 1 --2- 172.21.15.103:0/2723147515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555733769ec0 0x555733772750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.807+0000 7f8c3e4bb640 1 -- 172.21.15.103:0/2723147515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555733778a30 con 0x55573375c970 2024-09-19T06:59:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.807+0000 7f8c3e4bb640 1 --2- 172.21.15.103:0/2723147515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375c970 0x555733768d50 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f8c3401c620 tx=0x7f8c34009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ea20b02e6986be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.807+0000 7f8c23fff640 1 -- 172.21.15.103:0/2723147515 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34029040 con 0x55573375c970 2024-09-19T06:59:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.807+0000 7f8c23fff640 1 -- 172.21.15.103:0/2723147515 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c3401c3f0 con 0x55573375c970 2024-09-19T06:59:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.807+0000 7f8c23fff640 1 -- 172.21.15.103:0/2723147515 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34005380 con 0x55573375c970 2024-09-19T06:59:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 -- 172.21.15.103:0/2723147515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375c970 msgr2=0x555733768d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 --2- 172.21.15.103:0/2723147515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375c970 0x555733768d50 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f8c3401c620 tx=0x7f8c34009320 comp rx=0 tx=0).stop 2024-09-19T06:59:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 -- 172.21.15.103:0/2723147515 shutdown_connections 2024-09-19T06:59:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 --2- 172.21.15.103:0/2723147515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555733769ec0 0x555733772750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 --2- 172.21.15.103:0/2723147515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55573375d290 0x555733772210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 --2- 172.21.15.103:0/2723147515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375c970 0x555733768d50 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 -- 172.21.15.103:0/2723147515 >> 172.21.15.103:0/2723147515 conn(0x555733759060 msgr2=0x555733768640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 -- 172.21.15.103:0/2723147515 shutdown_connections 2024-09-19T06:59:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 -- 172.21.15.103:0/2723147515 wait complete. 2024-09-19T06:59:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 -- start start 2024-09-19T06:59:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 Processor -- start 2024-09-19T06:59:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55573375c970 0x555733761120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375d290 0x55573375e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555733769ec0 0x55573375ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381acab0 con 0x55573375d290 2024-09-19T06:59:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381ac3e0 con 0x55573375c970 2024-09-19T06:59:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381a9640 con 0x555733769ec0 2024-09-19T06:59:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c434b4740 10 client.0 _ll_register_callbacks cb 0x7f8c340093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c3dcba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375d290 0x55573375e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c3dcba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375d290 0x55573375e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58948/0 (socket says 172.21.15.103:58948) 2024-09-19T06:59:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.808+0000 7f8c3dcba640 1 -- 172.21.15.103:0/358504847 learned_addr learned my addr 172.21.15.103:0/358504847 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c3e4bb640 1 --2- 172.21.15.103:0/358504847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55573375c970 0x555733761120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c3ecbc640 1 --2- 172.21.15.103:0/358504847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555733769ec0 0x55573375ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c3dcba640 1 -- 172.21.15.103:0/358504847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555733769ec0 msgr2=0x55573375ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c3dcba640 1 --2- 172.21.15.103:0/358504847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555733769ec0 0x55573375ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c3dcba640 1 -- 172.21.15.103:0/358504847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55573375c970 msgr2=0x555733761120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c3dcba640 1 --2- 172.21.15.103:0/358504847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55573375c970 0x555733761120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c3dcba640 1 -- 172.21.15.103:0/358504847 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55573375f1b0 con 0x55573375d290 2024-09-19T06:59:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c3e4bb640 1 --2- 172.21.15.103:0/358504847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55573375c970 0x555733761120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c3dcba640 1 --2- 172.21.15.103:0/358504847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375d290 0x55573375e500 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f8c280cd0b0 tx=0x7f8c28010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de09d3ef364e45fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c21ffb640 1 -- 172.21.15.103:0/358504847 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28020040 con 0x55573375d290 2024-09-19T06:59:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c21ffb640 1 -- 172.21.15.103:0/358504847 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f8c28097990 con 0x55573375d290 2024-09-19T06:59:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c434b4740 1 -- 172.21.15.103:0/358504847 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c3401c3f0 con 0x55573375d290 2024-09-19T06:59:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c21ffb640 1 -- 172.21.15.103:0/358504847 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8c28009de0 con 0x55573375d290 2024-09-19T06:59:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c434b4740 10 client.19026 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c434b4740 1 -- 172.21.15.103:0/358504847 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5557337e6b80 con 0x55573375d290 2024-09-19T06:59:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c21ffb640 1 -- 172.21.15.103:0/358504847 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c3401c3f0 con 0x55573375d290 2024-09-19T06:59:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c21ffb640 1 -- 172.21.15.103:0/358504847 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8c280cd280 con 0x55573375d290 2024-09-19T06:59:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.809+0000 7f8c434b4740 10 client.19026 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.810+0000 7f8c434b4740 10 client.19026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.810+0000 7f8c434b4740 20 client.19026 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.810+0000 7f8c434b4740 1 --2- 172.21.15.103:0/358504847 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5557337e6ec0 0x5557337e92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.810+0000 7f8c434b4740 4 client.19026 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:59:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.810+0000 7f8c434b4740 1 -- 172.21.15.103:0/358504847 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557336c8d70 con 0x5557337e6ec0 2024-09-19T06:59:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.810+0000 7f8c3e4bb640 1 --2- 172.21.15.103:0/358504847 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5557337e6ec0 0x5557337e92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.810+0000 7f8c3e4bb640 1 --2- 172.21.15.103:0/358504847 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5557337e6ec0 0x5557337e92b0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=743d378d6d2136e1 server_cookie=57bc4f8a39243e21 in_seq=0 out_seq=0 2024-09-19T06:59:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.810+0000 7f8c21ffb640 10 client.19026 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c21ffb640 1 -- 172.21.15.103:0/358504847 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557336c8d70 con 0x5557337e6ec0 2024-09-19T06:59:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c21ffb640 10 client.19026 handle_command_reply: tid=0 2024-09-19T06:59:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c434b4740 1 client.19026 shutdown 2024-09-19T06:59:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c434b4740 1 -- 172.21.15.103:0/358504847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375d290 msgr2=0x55573375e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c434b4740 1 --2- 172.21.15.103:0/358504847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375d290 0x55573375e500 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f8c280cd0b0 tx=0x7f8c28010d70 comp rx=0 tx=0).stop 2024-09-19T06:59:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c434b4740 1 -- 172.21.15.103:0/358504847 shutdown_connections 2024-09-19T06:59:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c434b4740 1 --2- 172.21.15.103:0/358504847 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5557337e6ec0 0x5557337e92b0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c434b4740 1 --2- 172.21.15.103:0/358504847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555733769ec0 0x55573375ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c21ffb640 0 client.19026 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c434b4740 1 --2- 172.21.15.103:0/358504847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55573375d290 0x55573375e500 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c434b4740 1 --2- 172.21.15.103:0/358504847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55573375c970 0x555733761120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c434b4740 1 -- 172.21.15.103:0/358504847 >> 172.21.15.103:0/358504847 conn(0x555733759060 msgr2=0x5557337581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c434b4740 1 -- 172.21.15.103:0/358504847 shutdown_connections 2024-09-19T06:59:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c434b4740 1 -- 172.21.15.103:0/358504847 wait complete. 2024-09-19T06:59:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c434b4740 20 client.19026 trim_cache size 0 max 16384 2024-09-19T06:59:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:59:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:59:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:59:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:59:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c4071f640 1 -- 172.21.15.103:0/1642780549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8c240a34e0 msgr2=0x7f8c240a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.821+0000 7f8c4071f640 1 --2- 172.21.15.103:0/1642780549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8c240a34e0 0x7f8c240a59a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8c30004ab0 tx=0x7f8c3002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.822+0000 7f8c4071f640 1 -- 172.21.15.103:0/1642780549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c38101390 msgr2=0x7f8c381977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.822+0000 7f8c4071f640 1 --2- 172.21.15.103:0/1642780549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c38101390 0x7f8c381977b0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800d950 tx=0x7f8c2800de10 comp rx=0 tx=0).stop 2024-09-19T06:59:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.822+0000 7f8c4071f640 1 -- 172.21.15.103:0/1642780549 shutdown_connections 2024-09-19T06:59:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.822+0000 7f8c4071f640 1 --2- 172.21.15.103:0/1642780549 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8c240a34e0 0x7f8c240a59a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.822+0000 7f8c4071f640 1 --2- 172.21.15.103:0/1642780549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c38106a00 0x7f8c3819bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.822+0000 7f8c4071f640 1 --2- 172.21.15.103:0/1642780549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c38101390 0x7f8c381977b0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.822+0000 7f8c4071f640 1 --2- 172.21.15.103:0/1642780549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c381009f0 0x7f8c38197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.822+0000 7f8c4071f640 1 -- 172.21.15.103:0/1642780549 >> 172.21.15.103:0/1642780549 conn(0x7f8c380fc6b0 msgr2=0x7f8c380fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.822+0000 7f8c4071f640 1 -- 172.21.15.103:0/1642780549 shutdown_connections 2024-09-19T06:59:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:42.822+0000 7f8c4071f640 1 -- 172.21.15.103:0/1642780549 wait complete. 2024-09-19T06:59:44.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:43 smithi103 ceph-mon[31158]: pgmap v325: 97 pgs: 97 active+clean; 2.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2024-09-19T06:59:44.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:43 smithi134 ceph-mon[35356]: pgmap v325: 97 pgs: 97 active+clean; 2.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2024-09-19T06:59:44.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:43 smithi174 ceph-mon[35637]: pgmap v325: 97 pgs: 97 active+clean; 2.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2024-09-19T06:59:46.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:45 smithi103 ceph-mon[31158]: pgmap v326: 97 pgs: 97 active+clean; 2.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2024-09-19T06:59:46.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:45 smithi134 ceph-mon[35356]: pgmap v326: 97 pgs: 97 active+clean; 2.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2024-09-19T06:59:46.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:45 smithi174 ceph-mon[35637]: pgmap v326: 97 pgs: 97 active+clean; 2.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2024-09-19T06:59:48.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:47 smithi103 ceph-mon[31158]: pgmap v327: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-19T06:59:48.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:47 smithi134 ceph-mon[35356]: pgmap v327: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-19T06:59:48.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:47 smithi174 ceph-mon[35637]: pgmap v327: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-19T06:59:49.116 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:59:49.375 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.790+0000 7fd8f04bd640 1 -- 172.21.15.103:0/1854039318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e8102d40 msgr2=0x7fd8e8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.790+0000 7fd8f04bd640 1 --2- 172.21.15.103:0/1854039318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e8102d40 0x7fd8e8103120 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009900 tx=0x7fd8e402a480 comp rx=0 tx=0).stop 2024-09-19T06:59:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.790+0000 7fd8f04bd640 1 -- 172.21.15.103:0/1854039318 shutdown_connections 2024-09-19T06:59:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.790+0000 7fd8f04bd640 1 --2- 172.21.15.103:0/1854039318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8e810c170 0x7fd8e810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.790+0000 7fd8f04bd640 1 --2- 172.21.15.103:0/1854039318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8e8103660 0x7fd8e810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.790+0000 7fd8f04bd640 1 --2- 172.21.15.103:0/1854039318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e8102d40 0x7fd8e8103120 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.790+0000 7fd8f04bd640 1 -- 172.21.15.103:0/1854039318 >> 172.21.15.103:0/1854039318 conn(0x7fd8e80fc7a0 msgr2=0x7fd8e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.791+0000 7fd8f04bd640 1 -- 172.21.15.103:0/1854039318 shutdown_connections 2024-09-19T06:59:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.791+0000 7fd8f04bd640 1 -- 172.21.15.103:0/1854039318 wait complete. 2024-09-19T06:59:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.791+0000 7fd8f04bd640 1 Processor -- start 2024-09-19T06:59:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.791+0000 7fd8f04bd640 1 -- start start 2024-09-19T06:59:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.791+0000 7fd8f04bd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8e8102d40 0x7fd8e8198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.791+0000 7fd8f04bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8e8103660 0x7fd8e81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.791+0000 7fd8f04bd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e810c170 0x7fd8e819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.791+0000 7fd8f04bd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e8111f70 con 0x7fd8e810c170 2024-09-19T06:59:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.791+0000 7fd8f04bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e8111df0 con 0x7fd8e8103660 2024-09-19T06:59:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.791+0000 7fd8f04bd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e81120f0 con 0x7fd8e8102d40 2024-09-19T06:59:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.791+0000 7fd8eea5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e810c170 0x7fd8e819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.791+0000 7fd8eea5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e810c170 0x7fd8e819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35212/0 (socket says 172.21.15.103:35212) 2024-09-19T06:59:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.792+0000 7fd8ee259640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8e8102d40 0x7fd8e8198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.792+0000 7fd8eda58640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8e8103660 0x7fd8e81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.792+0000 7fd8eea5a640 1 -- 172.21.15.103:0/3615495850 learned_addr learned my addr 172.21.15.103:0/3615495850 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.792+0000 7fd8eea5a640 1 -- 172.21.15.103:0/3615495850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8e8102d40 msgr2=0x7fd8e8198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.792+0000 7fd8eea5a640 1 --2- 172.21.15.103:0/3615495850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8e8102d40 0x7fd8e8198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.792+0000 7fd8eea5a640 1 -- 172.21.15.103:0/3615495850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8e8103660 msgr2=0x7fd8e81989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.792+0000 7fd8eea5a640 1 --2- 172.21.15.103:0/3615495850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8e8103660 0x7fd8e81989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.792+0000 7fd8eea5a640 1 -- 172.21.15.103:0/3615495850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e819d420 con 0x7fd8e810c170 2024-09-19T06:59:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.792+0000 7fd8eea5a640 1 --2- 172.21.15.103:0/3615495850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e810c170 0x7fd8e819cd40 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fd8e000eaa0 tx=0x7fd8e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.792+0000 7fd8df7fe640 1 -- 172.21.15.103:0/3615495850 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e000cdb0 con 0x7fd8e810c170 2024-09-19T06:59:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.792+0000 7fd8f04bd640 1 -- 172.21.15.103:0/3615495850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e819d710 con 0x7fd8e810c170 2024-09-19T06:59:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.792+0000 7fd8df7fe640 1 -- 172.21.15.103:0/3615495850 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8e0005020 con 0x7fd8e810c170 2024-09-19T06:59:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.792+0000 7fd8df7fe640 1 -- 172.21.15.103:0/3615495850 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00116b0 con 0x7fd8e810c170 2024-09-19T06:59:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.792+0000 7fd8f04bd640 1 -- 172.21.15.103:0/3615495850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e81a4ff0 con 0x7fd8e810c170 2024-09-19T06:59:49.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.794+0000 7fd8df7fe640 1 -- 172.21.15.103:0/3615495850 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd8e0005560 con 0x7fd8e810c170 2024-09-19T06:59:49.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.795+0000 7fd8f2896740 1 Processor -- start 2024-09-19T06:59:49.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.795+0000 7fd8f2896740 1 -- start start 2024-09-19T06:59:49.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.795+0000 7fd8df7fe640 1 --2- 172.21.15.103:0/3615495850 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd8d40a33b0 0x7fd8d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.795+0000 7fd8f2896740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17e4a0 0x55f44c17e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.795+0000 7fd8f2896740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c17edc0 0x55f44c189140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.795+0000 7fd8f2896740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c18ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.795+0000 7fd8f2896740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f44c0e42e0 con 0x55f44c189680 2024-09-19T06:59:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.795+0000 7fd8f2896740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f44c08bce0 con 0x55f44c17edc0 2024-09-19T06:59:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.795+0000 7fd8f2896740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f44c0e02a0 con 0x55f44c17e4a0 2024-09-19T06:59:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.795+0000 7fd8eda58640 1 --2- 172.21.15.103:0/3615495850 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd8d40a33b0 0x7fd8d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.795+0000 7fd8df7fe640 1 -- 172.21.15.103:0/3615495850 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7fd8e00cfed0 con 0x7fd8e810c170 2024-09-19T06:59:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.795+0000 7fd8eea5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c18ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.795+0000 7fd8eea5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c18ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35224/0 (socket says 172.21.15.103:35224) 2024-09-19T06:59:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.795+0000 7fd8eea5a640 1 -- 172.21.15.103:0/514372040 learned_addr learned my addr 172.21.15.103:0/514372040 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.796+0000 7fd8ee259640 1 --2- 172.21.15.103:0/514372040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17e4a0 0x55f44c17e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.796+0000 7fd8eda58640 1 --2- 172.21.15.103:0/514372040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c17edc0 0x55f44c189140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.796+0000 7fd8eea5a640 1 -- 172.21.15.103:0/514372040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17e4a0 msgr2=0x55f44c17e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.796+0000 7fd8eea5a640 1 --2- 172.21.15.103:0/514372040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17e4a0 0x55f44c17e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.796+0000 7fd8eea5a640 1 -- 172.21.15.103:0/514372040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c17edc0 msgr2=0x55f44c189140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:49.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.796+0000 7fd8eea5a640 1 --2- 172.21.15.103:0/514372040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c17edc0 0x55f44c189140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.796+0000 7fd8eea5a640 1 -- 172.21.15.103:0/514372040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f44c070440 con 0x55f44c189680 2024-09-19T06:59:49.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.796+0000 7fd8eda58640 1 --2- 172.21.15.103:0/3615495850 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd8d40a33b0 0x7fd8d40a5870 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd8e810d5b0 tx=0x7fd8d8013880 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:49.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.796+0000 7fd8eea5a640 1 --2- 172.21.15.103:0/514372040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c18ba70 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0098bb0 tx=0x7fd8e009be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a79753cce32d41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:49.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.796+0000 7fd8dcff9640 1 -- 172.21.15.103:0/514372040 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e001f040 con 0x55f44c189680 2024-09-19T06:59:49.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.796+0000 7fd8dcff9640 1 -- 172.21.15.103:0/514372040 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8e00791b0 con 0x55f44c189680 2024-09-19T06:59:49.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.796+0000 7fd8dcff9640 1 -- 172.21.15.103:0/514372040 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00bc0d0 con 0x55f44c189680 2024-09-19T06:59:49.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.797+0000 7fd8f2896740 1 -- 172.21.15.103:0/514372040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 msgr2=0x55f44c18ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:49.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.797+0000 7fd8f2896740 1 --2- 172.21.15.103:0/514372040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c18ba70 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0098bb0 tx=0x7fd8e009be30 comp rx=0 tx=0).stop 2024-09-19T06:59:49.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.797+0000 7fd8f2896740 1 -- 172.21.15.103:0/514372040 shutdown_connections 2024-09-19T06:59:49.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.797+0000 7fd8f2896740 1 --2- 172.21.15.103:0/514372040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c18ba70 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.797+0000 7fd8f2896740 1 --2- 172.21.15.103:0/514372040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c17edc0 0x55f44c189140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.797+0000 7fd8f2896740 1 --2- 172.21.15.103:0/514372040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17e4a0 0x55f44c17e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.797+0000 7fd8f2896740 1 -- 172.21.15.103:0/514372040 >> 172.21.15.103:0/514372040 conn(0x55f44c17aa30 msgr2=0x55f44c17ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:49.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.797+0000 7fd8f2896740 1 -- 172.21.15.103:0/514372040 shutdown_connections 2024-09-19T06:59:49.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.797+0000 7fd8f2896740 1 -- 172.21.15.103:0/514372040 wait complete. 2024-09-19T06:59:49.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.797+0000 7fd8f2896740 1 -- start start 2024-09-19T06:59:49.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.797+0000 7fd8f2896740 1 Processor -- start 2024-09-19T06:59:49.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8f2896740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c17e4a0 0x55f44c188f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8f2896740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17edc0 0x55f44c186220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8f2896740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c186760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8f2896740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f44c190660 con 0x55f44c189680 2024-09-19T06:59:49.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8f2896740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f44c09f050 con 0x55f44c17e4a0 2024-09-19T06:59:49.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8f2896740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f44c1907e0 con 0x55f44c17edc0 2024-09-19T06:59:49.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8f2896740 10 client.0 _ll_register_callbacks cb 0x55f44c0df160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:49.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8eea5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c186760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8eea5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c186760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35226/0 (socket says 172.21.15.103:35226) 2024-09-19T06:59:49.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8eea5a640 1 -- 172.21.15.103:0/3079152276 learned_addr learned my addr 172.21.15.103:0/3079152276 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:49.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8ee259640 1 --2- 172.21.15.103:0/3079152276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c17e4a0 0x55f44c188f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8eda58640 1 --2- 172.21.15.103:0/3079152276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17edc0 0x55f44c186220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8eea5a640 1 -- 172.21.15.103:0/3079152276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17edc0 msgr2=0x55f44c186220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:49.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8eea5a640 1 --2- 172.21.15.103:0/3079152276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17edc0 0x55f44c186220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8eea5a640 1 -- 172.21.15.103:0/3079152276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c17e4a0 msgr2=0x55f44c188f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:49.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8eea5a640 1 --2- 172.21.15.103:0/3079152276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c17e4a0 0x55f44c188f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8eea5a640 1 -- 172.21.15.103:0/3079152276 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f44be553c0 con 0x55f44c189680 2024-09-19T06:59:49.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8eda58640 1 --2- 172.21.15.103:0/3079152276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17edc0 0x55f44c186220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:49.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8ee259640 1 --2- 172.21.15.103:0/3079152276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c17e4a0 0x55f44c188f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:49.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8eea5a640 1 --2- 172.21.15.103:0/3079152276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c186760 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00a4870 tx=0x7fd8e00985b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4441d5146b122064 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:49.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:49.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.798+0000 7fd8d1ffb640 1 -- 172.21.15.103:0/3079152276 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00b4070 con 0x55f44c189680 2024-09-19T06:59:49.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.799+0000 7fd8d1ffb640 1 -- 172.21.15.103:0/3079152276 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7fd8e0098850 con 0x55f44c189680 2024-09-19T06:59:49.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.799+0000 7fd8f2896740 1 -- 172.21.15.103:0/3079152276 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8e00791b0 con 0x55f44c189680 2024-09-19T06:59:49.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.799+0000 7fd8d1ffb640 1 -- 172.21.15.103:0/3079152276 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd8e001f050 con 0x55f44c189680 2024-09-19T06:59:49.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.799+0000 7fd8f2896740 10 client.19038 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:49.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.799+0000 7fd8f2896740 1 -- 172.21.15.103:0/3079152276 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f44c19bfd0 con 0x55f44c189680 2024-09-19T06:59:49.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.799+0000 7fd8d1ffb640 1 -- 172.21.15.103:0/3079152276 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8e00791b0 con 0x55f44c189680 2024-09-19T06:59:49.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.799+0000 7fd8d1ffb640 1 -- 172.21.15.103:0/3079152276 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd8e000f040 con 0x55f44c189680 2024-09-19T06:59:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.799+0000 7fd8f2896740 10 client.19038 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.799+0000 7fd8f2896740 10 client.19038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.799+0000 7fd8f2896740 20 client.19038 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.799+0000 7fd8f2896740 1 --2- 172.21.15.103:0/3079152276 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f44c19d320 0x55f44c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.799+0000 7fd8f2896740 4 client.19038 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.799+0000 7fd8f2896740 1 -- 172.21.15.103:0/3079152276 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f44c0f1850 con 0x55f44c19d320 2024-09-19T06:59:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.799+0000 7fd8ee259640 1 --2- 172.21.15.103:0/3079152276 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f44c19d320 0x55f44c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.800+0000 7fd8ee259640 1 --2- 172.21.15.103:0/3079152276 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f44c19d320 0x55f44c19f710 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2e10c563e614db7 server_cookie=73cc75729935499f in_seq=0 out_seq=0 2024-09-19T06:59:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.800+0000 7fd8d1ffb640 10 client.19038 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.800+0000 7fd8d1ffb640 1 -- 172.21.15.103:0/3079152276 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f44c0f1850 con 0x55f44c19d320 2024-09-19T06:59:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.800+0000 7fd8d1ffb640 10 client.19038 handle_command_reply: tid=0 2024-09-19T06:59:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.801+0000 7fd8f2896740 1 client.19038 shutdown 2024-09-19T06:59:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.801+0000 7fd8f2896740 1 -- 172.21.15.103:0/3079152276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 msgr2=0x55f44c186760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.801+0000 7fd8f2896740 1 --2- 172.21.15.103:0/3079152276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c186760 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00a4870 tx=0x7fd8e00985b0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.801+0000 7fd8f2896740 1 -- 172.21.15.103:0/3079152276 shutdown_connections 2024-09-19T06:59:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.801+0000 7fd8f2896740 1 --2- 172.21.15.103:0/3079152276 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f44c19d320 0x55f44c19f710 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.801+0000 7fd8f2896740 1 --2- 172.21.15.103:0/3079152276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c186760 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.801+0000 7fd8f2896740 1 --2- 172.21.15.103:0/3079152276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17edc0 0x55f44c186220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.801+0000 7fd8f2896740 1 --2- 172.21.15.103:0/3079152276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c17e4a0 0x55f44c188f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.801+0000 7fd8f2896740 1 -- 172.21.15.103:0/3079152276 >> 172.21.15.103:0/3079152276 conn(0x55f44c17aa30 msgr2=0x55f44c180760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.801+0000 7fd8d1ffb640 0 client.19038 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.801+0000 7fd8f2896740 1 -- 172.21.15.103:0/3079152276 shutdown_connections 2024-09-19T06:59:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.801+0000 7fd8f2896740 1 -- 172.21.15.103:0/3079152276 wait complete. 2024-09-19T06:59:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.801+0000 7fd8f2896740 20 client.19038 trim_cache size 0 max 16384 2024-09-19T06:59:49.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8f2896740 1 Processor -- start 2024-09-19T06:59:49.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8f2896740 1 -- start start 2024-09-19T06:59:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8f2896740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c17e4a0 0x55f44c1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8f2896740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17edc0 0x55f44c187a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8f2896740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c1a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8f2896740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e81ab6a0 con 0x55f44c189680 2024-09-19T06:59:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8f2896740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e81ad1e0 con 0x55f44c17e4a0 2024-09-19T06:59:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8f2896740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e81abd70 con 0x55f44c17edc0 2024-09-19T06:59:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8eea5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c1a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8eea5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c1a4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35228/0 (socket says 172.21.15.103:35228) 2024-09-19T06:59:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8eea5a640 1 -- 172.21.15.103:0/2754988169 learned_addr learned my addr 172.21.15.103:0/2754988169 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8eda58640 1 --2- 172.21.15.103:0/2754988169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17edc0 0x55f44c187a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8ee259640 1 --2- 172.21.15.103:0/2754988169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c17e4a0 0x55f44c1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8eea5a640 1 -- 172.21.15.103:0/2754988169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17edc0 msgr2=0x55f44c187a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:49.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8eea5a640 1 --2- 172.21.15.103:0/2754988169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17edc0 0x55f44c187a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8eea5a640 1 -- 172.21.15.103:0/2754988169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c17e4a0 msgr2=0x55f44c1998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:49.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.818+0000 7fd8eea5a640 1 --2- 172.21.15.103:0/2754988169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c17e4a0 0x55f44c1998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.819+0000 7fd8eea5a640 1 -- 172.21.15.103:0/2754988169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f44c188930 con 0x55f44c189680 2024-09-19T06:59:49.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.819+0000 7fd8eea5a640 1 --2- 172.21.15.103:0/2754988169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c1a4d40 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00a3190 tx=0x7fd8e009be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d39018afe52fd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:49.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.819+0000 7fd8d3fff640 1 -- 172.21.15.103:0/2754988169 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00b4070 con 0x55f44c189680 2024-09-19T06:59:49.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.819+0000 7fd8d3fff640 1 -- 172.21.15.103:0/2754988169 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8e004b020 con 0x55f44c189680 2024-09-19T06:59:49.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.819+0000 7fd8d3fff640 1 -- 172.21.15.103:0/2754988169 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0098850 con 0x55f44c189680 2024-09-19T06:59:49.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.819+0000 7fd8f2896740 1 -- 172.21.15.103:0/2754988169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 msgr2=0x55f44c1a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:49.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.819+0000 7fd8f2896740 1 --2- 172.21.15.103:0/2754988169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c1a4d40 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00a3190 tx=0x7fd8e009be50 comp rx=0 tx=0).stop 2024-09-19T06:59:49.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.819+0000 7fd8f2896740 1 -- 172.21.15.103:0/2754988169 shutdown_connections 2024-09-19T06:59:49.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.819+0000 7fd8f2896740 1 --2- 172.21.15.103:0/2754988169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c189680 0x55f44c1a4d40 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.819+0000 7fd8f2896740 1 --2- 172.21.15.103:0/2754988169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17edc0 0x55f44c187a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.819+0000 7fd8f2896740 1 --2- 172.21.15.103:0/2754988169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c17e4a0 0x55f44c1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.819+0000 7fd8f2896740 1 -- 172.21.15.103:0/2754988169 >> 172.21.15.103:0/2754988169 conn(0x55f44c17aa30 msgr2=0x55f44c1851e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:49.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.820+0000 7fd8f2896740 1 -- 172.21.15.103:0/2754988169 shutdown_connections 2024-09-19T06:59:49.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.820+0000 7fd8f2896740 1 -- 172.21.15.103:0/2754988169 wait complete. 2024-09-19T06:59:49.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.820+0000 7fd8f2896740 1 -- start start 2024-09-19T06:59:49.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.820+0000 7fd8f2896740 1 Processor -- start 2024-09-19T06:59:49.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.820+0000 7fd8f2896740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17e4a0 0x55f44c195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.820+0000 7fd8f2896740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c17edc0 0x55f44c1911f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.820+0000 7fd8f2896740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c189680 0x55f44c191730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.820+0000 7fd8f2896740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e81acb10 con 0x55f44c17edc0 2024-09-19T06:59:49.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.820+0000 7fd8f2896740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e81ac440 con 0x55f44c189680 2024-09-19T06:59:49.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.820+0000 7fd8f2896740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e81a96a0 con 0x55f44c17e4a0 2024-09-19T06:59:49.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.820+0000 7fd8f2896740 10 client.0 _ll_register_callbacks cb 0x7fd8e4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:49.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.820+0000 7fd8eda58640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c17edc0 0x55f44c1911f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.820+0000 7fd8eda58640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c17edc0 0x55f44c1911f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35236/0 (socket says 172.21.15.103:35236) 2024-09-19T06:59:49.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.820+0000 7fd8eda58640 1 -- 172.21.15.103:0/3274260661 learned_addr learned my addr 172.21.15.103:0/3274260661 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:49.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.820+0000 7fd8eea5a640 1 --2- 172.21.15.103:0/3274260661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c189680 0x55f44c191730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.821+0000 7fd8ee259640 1 --2- 172.21.15.103:0/3274260661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17e4a0 0x55f44c195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.821+0000 7fd8eda58640 1 -- 172.21.15.103:0/3274260661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17e4a0 msgr2=0x55f44c195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:49.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.821+0000 7fd8eda58640 1 --2- 172.21.15.103:0/3274260661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17e4a0 0x55f44c195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.821+0000 7fd8eda58640 1 -- 172.21.15.103:0/3274260661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c189680 msgr2=0x55f44c191730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:49.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.821+0000 7fd8eda58640 1 --2- 172.21.15.103:0/3274260661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c189680 0x55f44c191730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.821+0000 7fd8eda58640 1 -- 172.21.15.103:0/3274260661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f44c191c70 con 0x55f44c17edc0 2024-09-19T06:59:49.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.821+0000 7fd8eda58640 1 --2- 172.21.15.103:0/3274260661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c17edc0 0x55f44c1911f0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x55f44c17b890 tx=0x7fd8d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f8f7f0d4db6cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:49.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.821+0000 7fd8d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:49.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.821+0000 7fd8d1ffb640 1 -- 172.21.15.103:0/3274260661 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8021d70 con 0x55f44c17edc0 2024-09-19T06:59:49.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.821+0000 7fd8d1ffb640 1 -- 172.21.15.103:0/3274260661 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7fd8d8005020 con 0x55f44c17edc0 2024-09-19T06:59:49.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.821+0000 7fd8f2896740 1 -- 172.21.15.103:0/3274260661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8e004b020 con 0x55f44c17edc0 2024-09-19T06:59:49.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.821+0000 7fd8d1ffb640 1 -- 172.21.15.103:0/3274260661 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd8d800a950 con 0x55f44c17edc0 2024-09-19T06:59:49.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.821+0000 7fd8f2896740 10 client.19044 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:49.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.821+0000 7fd8f2896740 1 -- 172.21.15.103:0/3274260661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f44c207aa0 con 0x55f44c17edc0 2024-09-19T06:59:49.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.821+0000 7fd8d1ffb640 1 -- 172.21.15.103:0/3274260661 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8e004b020 con 0x55f44c17edc0 2024-09-19T06:59:49.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.822+0000 7fd8d1ffb640 1 -- 172.21.15.103:0/3274260661 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd8d8033420 con 0x55f44c17edc0 2024-09-19T06:59:49.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.822+0000 7fd8f2896740 10 client.19044 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:49.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.822+0000 7fd8f2896740 10 client.19044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:49.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.822+0000 7fd8f2896740 20 client.19044 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:49.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.822+0000 7fd8f2896740 1 --2- 172.21.15.103:0/3274260661 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f44c208320 0x55f44c20a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:49.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.822+0000 7fd8f2896740 4 client.19044 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:59:49.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.822+0000 7fd8f2896740 1 -- 172.21.15.103:0/3274260661 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f44c0ebd60 con 0x55f44c208320 2024-09-19T06:59:49.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.822+0000 7fd8ee259640 1 --2- 172.21.15.103:0/3274260661 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f44c208320 0x55f44c20a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:49.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.822+0000 7fd8ee259640 1 --2- 172.21.15.103:0/3274260661 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f44c208320 0x55f44c20a740 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba80dcedda50734b server_cookie=b55caeeeda51a54f in_seq=0 out_seq=0 2024-09-19T06:59:49.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.822+0000 7fd8d1ffb640 10 client.19044 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:49.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.822+0000 7fd8d1ffb640 1 -- 172.21.15.103:0/3274260661 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f44c0ebd60 con 0x55f44c208320 2024-09-19T06:59:49.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.822+0000 7fd8d1ffb640 10 client.19044 handle_command_reply: tid=0 2024-09-19T06:59:49.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.822+0000 7fd8f2896740 1 client.19044 shutdown 2024-09-19T06:59:49.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f2896740 1 -- 172.21.15.103:0/3274260661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c17edc0 msgr2=0x55f44c1911f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:49.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f2896740 1 --2- 172.21.15.103:0/3274260661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c17edc0 0x55f44c1911f0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x55f44c17b890 tx=0x7fd8d8004730 comp rx=0 tx=0).stop 2024-09-19T06:59:49.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f2896740 1 -- 172.21.15.103:0/3274260661 shutdown_connections 2024-09-19T06:59:49.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f2896740 1 --2- 172.21.15.103:0/3274260661 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f44c208320 0x55f44c20a740 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f2896740 1 --2- 172.21.15.103:0/3274260661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f44c189680 0x55f44c191730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f2896740 1 --2- 172.21.15.103:0/3274260661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44c17edc0 0x55f44c1911f0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f2896740 1 --2- 172.21.15.103:0/3274260661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f44c17e4a0 0x55f44c195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f2896740 1 -- 172.21.15.103:0/3274260661 >> 172.21.15.103:0/3274260661 conn(0x55f44c17aa30 msgr2=0x55f44c1849c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:49.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8d1ffb640 0 client.19044 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:49.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f2896740 1 -- 172.21.15.103:0/3274260661 shutdown_connections 2024-09-19T06:59:49.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f2896740 1 -- 172.21.15.103:0/3274260661 wait complete. 2024-09-19T06:59:49.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f2896740 20 client.19044 trim_cache size 0 max 16384 2024-09-19T06:59:49.830 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T06:59:49.831 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T06:59:49.831 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T06:59:49.831 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T06:59:49.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f04bd640 1 -- 172.21.15.103:0/3615495850 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd8d40a33b0 msgr2=0x7fd8d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:49.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f04bd640 1 --2- 172.21.15.103:0/3615495850 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd8d40a33b0 0x7fd8d40a5870 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd8e810d5b0 tx=0x7fd8d8013880 comp rx=0 tx=0).stop 2024-09-19T06:59:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f04bd640 1 -- 172.21.15.103:0/3615495850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e810c170 msgr2=0x7fd8e819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f04bd640 1 --2- 172.21.15.103:0/3615495850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e810c170 0x7fd8e819cd40 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fd8e000eaa0 tx=0x7fd8e000ef60 comp rx=0 tx=0).stop 2024-09-19T06:59:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f04bd640 1 -- 172.21.15.103:0/3615495850 shutdown_connections 2024-09-19T06:59:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f04bd640 1 --2- 172.21.15.103:0/3615495850 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd8d40a33b0 0x7fd8d40a5870 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f04bd640 1 --2- 172.21.15.103:0/3615495850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8e810c170 0x7fd8e819cd40 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f04bd640 1 --2- 172.21.15.103:0/3615495850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8e8103660 0x7fd8e81989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f04bd640 1 --2- 172.21.15.103:0/3615495850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8e8102d40 0x7fd8e8198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f04bd640 1 -- 172.21.15.103:0/3615495850 >> 172.21.15.103:0/3615495850 conn(0x7fd8e80fc7a0 msgr2=0x7fd8e80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.823+0000 7fd8f04bd640 1 -- 172.21.15.103:0/3615495850 shutdown_connections 2024-09-19T06:59:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:49.824+0000 7fd8f04bd640 1 -- 172.21.15.103:0/3615495850 wait complete. 2024-09-19T06:59:50.135 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:49 smithi174 ceph-mon[35637]: pgmap v328: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-19T06:59:50.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:49 smithi103 ceph-mon[31158]: pgmap v328: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-19T06:59:50.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:49 smithi134 ceph-mon[35356]: pgmap v328: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-19T06:59:50.630 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:59:50.630 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:59:50.880 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:51.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.273+0000 7ff66149e640 1 -- 172.21.15.103:0/3237697163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff65c106a80 msgr2=0x7ff65c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:51.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.273+0000 7ff66149e640 1 --2- 172.21.15.103:0/3237697163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff65c106a80 0x7ff65c106e60 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7ff654009930 tx=0x7ff65402a420 comp rx=0 tx=0).stop 2024-09-19T06:59:51.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.273+0000 7ff66149e640 1 -- 172.21.15.103:0/3237697163 shutdown_connections 2024-09-19T06:59:51.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.273+0000 7ff66149e640 1 --2- 172.21.15.103:0/3237697163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff65c101420 0x7ff65c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:51.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.273+0000 7ff66149e640 1 --2- 172.21.15.103:0/3237697163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65c100a80 0x7ff65c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:51.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.273+0000 7ff66149e640 1 --2- 172.21.15.103:0/3237697163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff65c106a80 0x7ff65c106e60 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:51.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.273+0000 7ff66149e640 1 -- 172.21.15.103:0/3237697163 >> 172.21.15.103:0/3237697163 conn(0x7ff65c0fc7a0 msgr2=0x7ff65c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:51.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.273+0000 7ff66149e640 1 -- 172.21.15.103:0/3237697163 shutdown_connections 2024-09-19T06:59:51.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.273+0000 7ff66149e640 1 -- 172.21.15.103:0/3237697163 wait complete. 2024-09-19T06:59:51.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.273+0000 7ff66149e640 1 Processor -- start 2024-09-19T06:59:51.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.273+0000 7ff66149e640 1 -- start start 2024-09-19T06:59:51.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff66149e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65c100a80 0x7ff65c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:51.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff66149e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff65c101420 0x7ff65c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:51.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff66149e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff65c106a80 0x7ff65c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:51.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff66149e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c111f60 con 0x7ff65c101420 2024-09-19T06:59:51.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff66149e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c111de0 con 0x7ff65c100a80 2024-09-19T06:59:51.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff66149e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c1120e0 con 0x7ff65c106a80 2024-09-19T06:59:51.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff65a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff65c101420 0x7ff65c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:51.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff65a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff65c101420 0x7ff65c198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35256/0 (socket says 172.21.15.103:35256) 2024-09-19T06:59:51.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff65a7fc640 1 -- 172.21.15.103:0/2042892928 learned_addr learned my addr 172.21.15.103:0/2042892928 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:51.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff65affd640 1 --2- 172.21.15.103:0/2042892928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65c100a80 0x7ff65c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:51.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff65b7fe640 1 --2- 172.21.15.103:0/2042892928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff65c106a80 0x7ff65c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:51.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff65a7fc640 1 -- 172.21.15.103:0/2042892928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff65c106a80 msgr2=0x7ff65c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:51.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff65a7fc640 1 --2- 172.21.15.103:0/2042892928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff65c106a80 0x7ff65c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:51.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff65a7fc640 1 -- 172.21.15.103:0/2042892928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65c100a80 msgr2=0x7ff65c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:51.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff65a7fc640 1 --2- 172.21.15.103:0/2042892928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65c100a80 0x7ff65c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:51.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff65a7fc640 1 -- 172.21.15.103:0/2042892928 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65c19d3c0 con 0x7ff65c101420 2024-09-19T06:59:51.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff65b7fe640 1 --2- 172.21.15.103:0/2042892928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff65c106a80 0x7ff65c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:51.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff65a7fc640 1 --2- 172.21.15.103:0/2042892928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff65c101420 0x7ff65c198950 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7ff64400c920 tx=0x7ff64400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:51.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff653fff640 1 -- 172.21.15.103:0/2042892928 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644007c70 con 0x7ff65c101420 2024-09-19T06:59:51.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.274+0000 7ff66149e640 1 -- 172.21.15.103:0/2042892928 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff65c19d6b0 con 0x7ff65c101420 2024-09-19T06:59:51.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.275+0000 7ff653fff640 1 -- 172.21.15.103:0/2042892928 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff644007e10 con 0x7ff65c101420 2024-09-19T06:59:51.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.275+0000 7ff653fff640 1 -- 172.21.15.103:0/2042892928 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644018890 con 0x7ff65c101420 2024-09-19T06:59:51.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.275+0000 7ff66149e640 1 -- 172.21.15.103:0/2042892928 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff65c1a4f90 con 0x7ff65c101420 2024-09-19T06:59:51.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.276+0000 7ff66149e640 1 -- 172.21.15.103:0/2042892928 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff65c100ee0 con 0x7ff65c101420 2024-09-19T06:59:51.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.277+0000 7ff653fff640 1 -- 172.21.15.103:0/2042892928 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff644018a70 con 0x7ff65c101420 2024-09-19T06:59:51.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.277+0000 7ff653fff640 1 --2- 172.21.15.103:0/2042892928 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff6480a33b0 0x7ff6480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:51.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.277+0000 7ff65affd640 1 --2- 172.21.15.103:0/2042892928 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff6480a33b0 0x7ff6480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:51.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.277+0000 7ff653fff640 1 -- 172.21.15.103:0/2042892928 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7ff6440cfe90 con 0x7ff65c101420 2024-09-19T06:59:51.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.278+0000 7ff65affd640 1 --2- 172.21.15.103:0/2042892928 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff6480a33b0 0x7ff6480a5870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff654009900 tx=0x7ff6540058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:51.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.282+0000 7ff653fff640 1 -- 172.21.15.103:0/2042892928 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff644014030 con 0x7ff65c101420 2024-09-19T06:59:51.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.486+0000 7ff66149e640 1 -- 172.21.15.103:0/2042892928 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff65c108a60 con 0x7ff65c101420 2024-09-19T06:59:51.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.486+0000 7ff653fff640 1 -- 172.21.15.103:0/2042892928 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7ff64409c850 con 0x7ff65c101420 2024-09-19T06:59:51.488 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T06:59:51.488 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T06:59:51.488 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T06:59:51.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.488+0000 7ff66149e640 1 -- 172.21.15.103:0/2042892928 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff6480a33b0 msgr2=0x7ff6480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:51.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.488+0000 7ff66149e640 1 --2- 172.21.15.103:0/2042892928 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff6480a33b0 0x7ff6480a5870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff654009900 tx=0x7ff6540058d0 comp rx=0 tx=0).stop 2024-09-19T06:59:51.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.488+0000 7ff66149e640 1 -- 172.21.15.103:0/2042892928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff65c101420 msgr2=0x7ff65c198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:51.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.488+0000 7ff66149e640 1 --2- 172.21.15.103:0/2042892928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff65c101420 0x7ff65c198950 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7ff64400c920 tx=0x7ff64400cde0 comp rx=0 tx=0).stop 2024-09-19T06:59:51.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.489+0000 7ff66149e640 1 -- 172.21.15.103:0/2042892928 shutdown_connections 2024-09-19T06:59:51.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.489+0000 7ff66149e640 1 --2- 172.21.15.103:0/2042892928 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff6480a33b0 0x7ff6480a5870 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:51.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.489+0000 7ff66149e640 1 --2- 172.21.15.103:0/2042892928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff65c106a80 0x7ff65c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:51.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.489+0000 7ff66149e640 1 --2- 172.21.15.103:0/2042892928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff65c101420 0x7ff65c198950 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:51.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.489+0000 7ff66149e640 1 --2- 172.21.15.103:0/2042892928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65c100a80 0x7ff65c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:51.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.489+0000 7ff66149e640 1 -- 172.21.15.103:0/2042892928 >> 172.21.15.103:0/2042892928 conn(0x7ff65c0fc7a0 msgr2=0x7ff65c0fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:51.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.489+0000 7ff66149e640 1 -- 172.21.15.103:0/2042892928 shutdown_connections 2024-09-19T06:59:51.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:51.489+0000 7ff66149e640 1 -- 172.21.15.103:0/2042892928 wait complete. 2024-09-19T06:59:52.099 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:51 smithi103 ceph-mon[31158]: pgmap v329: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-19T06:59:52.100 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:51 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2042892928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:59:52.127 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T06:59:52.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:51 smithi134 ceph-mon[35356]: pgmap v329: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-19T06:59:52.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:51 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2042892928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:59:52.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:51 smithi174 ceph-mon[35637]: pgmap v329: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-19T06:59:52.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:51 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2042892928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:59:52.374 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:52.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.792+0000 7fccfd683640 1 -- 172.21.15.103:0/2163684725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccf8102d30 msgr2=0x7fccf8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:52.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.792+0000 7fccfd683640 1 --2- 172.21.15.103:0/2163684725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccf8102d30 0x7fccf8103110 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009cd0 tx=0x7fcce802a910 comp rx=0 tx=0).stop 2024-09-19T06:59:52.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.792+0000 7fccfd683640 1 -- 172.21.15.103:0/2163684725 shutdown_connections 2024-09-19T06:59:52.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.792+0000 7fccfd683640 1 --2- 172.21.15.103:0/2163684725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccf81082a0 0x7fccf810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.792+0000 7fccfd683640 1 --2- 172.21.15.103:0/2163684725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccf81036e0 0x7fccf8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.792+0000 7fccfd683640 1 --2- 172.21.15.103:0/2163684725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccf8102d30 0x7fccf8103110 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.792+0000 7fccfd683640 1 -- 172.21.15.103:0/2163684725 >> 172.21.15.103:0/2163684725 conn(0x7fccf80fc7d0 msgr2=0x7fccf80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:52.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.793+0000 7fccfd683640 1 -- 172.21.15.103:0/2163684725 shutdown_connections 2024-09-19T06:59:52.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.793+0000 7fccfd683640 1 -- 172.21.15.103:0/2163684725 wait complete. 2024-09-19T06:59:52.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.793+0000 7fccfd683640 1 Processor -- start 2024-09-19T06:59:52.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.793+0000 7fccfd683640 1 -- start start 2024-09-19T06:59:52.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.793+0000 7fccfd683640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccf8102d30 0x7fccf8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.793+0000 7fccfd683640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccf81036e0 0x7fccf8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccfd683640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccf81082a0 0x7fccf8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccfd683640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81120e0 con 0x7fccf81082a0 2024-09-19T06:59:52.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccfd683640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fccf8111f60 con 0x7fccf81036e0 2024-09-19T06:59:52.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccfd683640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fccf8112260 con 0x7fccf8102d30 2024-09-19T06:59:52.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccf77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccf81082a0 0x7fccf8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccf77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccf81082a0 0x7fccf8101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35284/0 (socket says 172.21.15.103:35284) 2024-09-19T06:59:52.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccf77fe640 1 -- 172.21.15.103:0/1063607631 learned_addr learned my addr 172.21.15.103:0/1063607631 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:52.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccf67fc640 1 --2- 172.21.15.103:0/1063607631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccf81036e0 0x7fccf8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccf6ffd640 1 --2- 172.21.15.103:0/1063607631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccf8102d30 0x7fccf8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccf77fe640 1 -- 172.21.15.103:0/1063607631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccf8102d30 msgr2=0x7fccf8100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:52.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccf77fe640 1 --2- 172.21.15.103:0/1063607631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccf8102d30 0x7fccf8100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccf77fe640 1 -- 172.21.15.103:0/1063607631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccf81036e0 msgr2=0x7fccf8100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:52.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccf77fe640 1 --2- 172.21.15.103:0/1063607631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccf81036e0 0x7fccf8100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccf77fe640 1 -- 172.21.15.103:0/1063607631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf81a6ae0 con 0x7fccf81082a0 2024-09-19T06:59:52.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccf77fe640 1 --2- 172.21.15.103:0/1063607631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccf81082a0 0x7fccf8101410 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fccf000d750 tx=0x7fccf000db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:52.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccdffff640 1 -- 172.21.15.103:0/1063607631 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0017070 con 0x7fccf81082a0 2024-09-19T06:59:52.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccfd683640 1 -- 172.21.15.103:0/1063607631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf81a6dd0 con 0x7fccf81082a0 2024-09-19T06:59:52.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccdffff640 1 -- 172.21.15.103:0/1063607631 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccf00097c0 con 0x7fccf81082a0 2024-09-19T06:59:52.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccdffff640 1 -- 172.21.15.103:0/1063607631 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0004d00 con 0x7fccf81082a0 2024-09-19T06:59:52.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.794+0000 7fccfd683640 1 -- 172.21.15.103:0/1063607631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf81a7290 con 0x7fccf81082a0 2024-09-19T06:59:52.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.797+0000 7fccdffff640 1 -- 172.21.15.103:0/1063607631 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fccf0009970 con 0x7fccf81082a0 2024-09-19T06:59:52.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.797+0000 7fccffa5c740 1 Processor -- start 2024-09-19T06:59:52.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.797+0000 7fccffa5c740 1 -- start start 2024-09-19T06:59:52.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccffa5c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d12596b40 0x559d12595090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccffa5c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d125955d0 0x559d12595a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccffa5c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d125a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccffa5c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559d124fb2f0 con 0x559d125955d0 2024-09-19T06:59:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccffa5c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559d124a2cf0 con 0x559d125a0370 2024-09-19T06:59:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccffa5c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559d124f72b0 con 0x559d12596b40 2024-09-19T06:59:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccf6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d125955d0 0x559d12595a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccf6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d125955d0 0x559d12595a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35290/0 (socket says 172.21.15.103:35290) 2024-09-19T06:59:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccf6ffd640 1 -- 172.21.15.103:0/919208957 learned_addr learned my addr 172.21.15.103:0/919208957 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccdffff640 1 --2- 172.21.15.103:0/1063607631 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcce00a33b0 0x7fcce00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccf67fc640 1 --2- 172.21.15.103:0/919208957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d12596b40 0x559d12595090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccf67fc640 1 --2- 172.21.15.103:0/919208957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d125a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccf6ffd640 1 -- 172.21.15.103:0/919208957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d12596b40 msgr2=0x559d12595090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccf6ffd640 1 --2- 172.21.15.103:0/919208957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d12596b40 0x559d12595090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccf6ffd640 1 -- 172.21.15.103:0/919208957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 msgr2=0x559d125a2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccf6ffd640 1 --2- 172.21.15.103:0/919208957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d125a2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccf6ffd640 1 -- 172.21.15.103:0/919208957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d124873a0 con 0x559d125955d0 2024-09-19T06:59:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccf67fc640 1 --2- 172.21.15.103:0/919208957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d125a2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccdffff640 1 -- 172.21.15.103:0/1063607631 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7fccf00cf740 con 0x7fccf81082a0 2024-09-19T06:59:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccf77fe640 1 --2- 172.21.15.103:0/1063607631 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcce00a33b0 0x7fcce00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccf6ffd640 1 --2- 172.21.15.103:0/919208957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d125955d0 0x559d12595a10 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009cd0 tx=0x7fcce8009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ca833641b09460 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.798+0000 7fccdd7fa640 1 -- 172.21.15.103:0/919208957 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8031d80 con 0x559d125955d0 2024-09-19T06:59:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.799+0000 7fccdd7fa640 1 -- 172.21.15.103:0/919208957 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce8036040 con 0x559d125955d0 2024-09-19T06:59:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.799+0000 7fccdd7fa640 1 -- 172.21.15.103:0/919208957 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8004330 con 0x559d125955d0 2024-09-19T06:59:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.799+0000 7fccffa5c740 1 -- 172.21.15.103:0/919208957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d125955d0 msgr2=0x559d12595a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.799+0000 7fccf77fe640 1 --2- 172.21.15.103:0/1063607631 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcce00a33b0 0x7fcce00a5870 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fccf009b990 tx=0x7fccf00046c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.799+0000 7fccffa5c740 1 --2- 172.21.15.103:0/919208957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d125955d0 0x559d12595a10 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009cd0 tx=0x7fcce8009870 comp rx=0 tx=0).stop 2024-09-19T06:59:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.799+0000 7fccffa5c740 1 -- 172.21.15.103:0/919208957 shutdown_connections 2024-09-19T06:59:52.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.799+0000 7fccffa5c740 1 --2- 172.21.15.103:0/919208957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d125a2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.799+0000 7fccffa5c740 1 --2- 172.21.15.103:0/919208957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d125955d0 0x559d12595a10 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.799+0000 7fccffa5c740 1 --2- 172.21.15.103:0/919208957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d12596b40 0x559d12595090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.799+0000 7fccffa5c740 1 -- 172.21.15.103:0/919208957 >> 172.21.15.103:0/919208957 conn(0x559d1255e310 msgr2=0x559d12591c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:52.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.799+0000 7fccffa5c740 1 -- 172.21.15.103:0/919208957 shutdown_connections 2024-09-19T06:59:52.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.799+0000 7fccffa5c740 1 -- 172.21.15.103:0/919208957 wait complete. 2024-09-19T06:59:52.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.800+0000 7fccffa5c740 1 -- start start 2024-09-19T06:59:52.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.800+0000 7fccffa5c740 1 Processor -- start 2024-09-19T06:59:52.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.800+0000 7fccffa5c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d125955d0 0x559d1259fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.800+0000 7fccffa5c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d12596b40 0x559d1259cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.800+0000 7fccffa5c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d1259d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.800+0000 7fccffa5c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559d125a3010 con 0x559d12596b40 2024-09-19T06:59:52.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.800+0000 7fccffa5c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559d124b6060 con 0x559d125a0370 2024-09-19T06:59:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.800+0000 7fccffa5c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559d125a3190 con 0x559d125955d0 2024-09-19T06:59:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.800+0000 7fccffa5c740 10 client.0 _ll_register_callbacks cb 0x559d122098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.800+0000 7fccf6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d12596b40 0x559d1259cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.800+0000 7fccf6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d12596b40 0x559d1259cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35296/0 (socket says 172.21.15.103:35296) 2024-09-19T06:59:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.800+0000 7fccf6ffd640 1 -- 172.21.15.103:0/2830592935 learned_addr learned my addr 172.21.15.103:0/2830592935 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.800+0000 7fccf67fc640 1 --2- 172.21.15.103:0/2830592935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d125955d0 0x559d1259fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.800+0000 7fccf67fc640 1 --2- 172.21.15.103:0/2830592935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d1259d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.800+0000 7fccf6ffd640 1 -- 172.21.15.103:0/2830592935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d125955d0 msgr2=0x559d1259fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.800+0000 7fccf6ffd640 1 --2- 172.21.15.103:0/2830592935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d125955d0 0x559d1259fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.801+0000 7fccf6ffd640 1 -- 172.21.15.103:0/2830592935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 msgr2=0x559d1259d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.801+0000 7fccf6ffd640 1 --2- 172.21.15.103:0/2830592935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d1259d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.801+0000 7fccf6ffd640 1 -- 172.21.15.103:0/2830592935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d1226c3c0 con 0x559d12596b40 2024-09-19T06:59:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.801+0000 7fccf67fc640 1 --2- 172.21.15.103:0/2830592935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d1259d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.801+0000 7fccf6ffd640 1 --2- 172.21.15.103:0/2830592935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d12596b40 0x559d1259cfd0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fcce802a950 tx=0x7fcce80314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52da9d5ed1ba760c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.801+0000 7fccdaff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.801+0000 7fccdaff5640 1 -- 172.21.15.103:0/2830592935 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8031d90 con 0x559d12596b40 2024-09-19T06:59:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.801+0000 7fccdaff5640 1 -- 172.21.15.103:0/2830592935 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7fcce8003970 con 0x559d12596b40 2024-09-19T06:59:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.801+0000 7fccffa5c740 1 -- 172.21.15.103:0/2830592935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce8036040 con 0x559d12596b40 2024-09-19T06:59:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.801+0000 7fccdaff5640 1 -- 172.21.15.103:0/2830592935 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcce805c050 con 0x559d12596b40 2024-09-19T06:59:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.801+0000 7fccffa5c740 10 client.19062 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.801+0000 7fccffa5c740 1 -- 172.21.15.103:0/2830592935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559d125ae9d0 con 0x559d12596b40 2024-09-19T06:59:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.801+0000 7fccdaff5640 1 -- 172.21.15.103:0/2830592935 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce8036040 con 0x559d12596b40 2024-09-19T06:59:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.802+0000 7fccdaff5640 1 -- 172.21.15.103:0/2830592935 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcce802f670 con 0x559d12596b40 2024-09-19T06:59:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.802+0000 7fccffa5c740 10 client.19062 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.802+0000 7fccffa5c740 10 client.19062 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:59:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.802+0000 7fccffa5c740 20 client.19062 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.802+0000 7fccffa5c740 1 --2- 172.21.15.103:0/2830592935 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x559d125afd20 0x559d125b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.802+0000 7fccffa5c740 4 client.19062 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.802+0000 7fccffa5c740 1 -- 172.21.15.103:0/2830592935 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d12508860 con 0x559d125afd20 2024-09-19T06:59:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.802+0000 7fccf67fc640 1 --2- 172.21.15.103:0/2830592935 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x559d125afd20 0x559d125b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.802+0000 7fccdaff5640 10 client.19062 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:59:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.802+0000 7fccf67fc640 1 --2- 172.21.15.103:0/2830592935 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x559d125afd20 0x559d125b2110 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=96695f14e61fa0e6 server_cookie=179a9e99f1db0061 in_seq=0 out_seq=0 2024-09-19T06:59:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.803+0000 7fccdaff5640 1 -- 172.21.15.103:0/2830592935 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d12508860 con 0x559d125afd20 2024-09-19T06:59:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.803+0000 7fccdaff5640 10 client.19062 handle_command_reply: tid=0 2024-09-19T06:59:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.803+0000 7fccffa5c740 1 client.19062 shutdown 2024-09-19T06:59:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.803+0000 7fccffa5c740 1 -- 172.21.15.103:0/2830592935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d12596b40 msgr2=0x559d1259cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.803+0000 7fccffa5c740 1 --2- 172.21.15.103:0/2830592935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d12596b40 0x559d1259cfd0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fcce802a950 tx=0x7fcce80314e0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.803+0000 7fccffa5c740 1 -- 172.21.15.103:0/2830592935 shutdown_connections 2024-09-19T06:59:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.803+0000 7fccffa5c740 1 --2- 172.21.15.103:0/2830592935 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x559d125afd20 0x559d125b2110 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.803+0000 7fccffa5c740 1 --2- 172.21.15.103:0/2830592935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d1259d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.803+0000 7fccffa5c740 1 --2- 172.21.15.103:0/2830592935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d12596b40 0x559d1259cfd0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.803+0000 7fccffa5c740 1 --2- 172.21.15.103:0/2830592935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d125955d0 0x559d1259fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.803+0000 7fccffa5c740 1 -- 172.21.15.103:0/2830592935 >> 172.21.15.103:0/2830592935 conn(0x559d1255e310 msgr2=0x559d12597960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:52.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.803+0000 7fccdaff5640 0 client.19062 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:59:52.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.803+0000 7fccffa5c740 1 -- 172.21.15.103:0/2830592935 shutdown_connections 2024-09-19T06:59:52.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.803+0000 7fccffa5c740 1 -- 172.21.15.103:0/2830592935 wait complete. 2024-09-19T06:59:52.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.803+0000 7fccffa5c740 20 client.19062 trim_cache size 0 max 16384 2024-09-19T06:59:52.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.816+0000 7fccffa5c740 1 Processor -- start 2024-09-19T06:59:52.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.816+0000 7fccffa5c740 1 -- start start 2024-09-19T06:59:52.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccffa5c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d125955d0 0x559d1259cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccffa5c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d12596b40 0x559d125ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccffa5c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d125a1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccffa5c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81afa70 con 0x559d125955d0 2024-09-19T06:59:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccffa5c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81b15b0 con 0x559d125a0370 2024-09-19T06:59:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccffa5c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81b0140 con 0x559d12596b40 2024-09-19T06:59:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccf67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d125955d0 0x559d1259cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccf67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d125955d0 0x559d1259cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35298/0 (socket says 172.21.15.103:35298) 2024-09-19T06:59:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccf67fc640 1 -- 172.21.15.103:0/2833776294 learned_addr learned my addr 172.21.15.103:0/2833776294 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccf6ffd640 1 --2- 172.21.15.103:0/2833776294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d12596b40 0x559d125ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccf67fc640 1 -- 172.21.15.103:0/2833776294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d12596b40 msgr2=0x559d125ac2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccf67fc640 1 --2- 172.21.15.103:0/2833776294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d12596b40 0x559d125ac2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccf67fc640 1 -- 172.21.15.103:0/2833776294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 msgr2=0x559d125a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccf67fc640 1 --2- 172.21.15.103:0/2833776294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d125a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccf67fc640 1 -- 172.21.15.103:0/2833776294 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d12591ed0 con 0x559d125955d0 2024-09-19T06:59:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccf67fc640 1 --2- 172.21.15.103:0/2833776294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d125955d0 0x559d1259cfa0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fccf806a370 tx=0x7fccec009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62eaa6f32b4448f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccdcff9640 1 -- 172.21.15.103:0/2833776294 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec015420 con 0x559d125955d0 2024-09-19T06:59:52.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccdcff9640 1 -- 172.21.15.103:0/2833776294 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccec023070 con 0x559d125955d0 2024-09-19T06:59:52.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.817+0000 7fccdcff9640 1 -- 172.21.15.103:0/2833776294 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec01e7b0 con 0x559d125955d0 2024-09-19T06:59:52.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.818+0000 7fccffa5c740 1 -- 172.21.15.103:0/2833776294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d125955d0 msgr2=0x559d1259cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:52.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.818+0000 7fccffa5c740 1 --2- 172.21.15.103:0/2833776294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d125955d0 0x559d1259cfa0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fccf806a370 tx=0x7fccec009110 comp rx=0 tx=0).stop 2024-09-19T06:59:52.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.818+0000 7fccffa5c740 1 -- 172.21.15.103:0/2833776294 shutdown_connections 2024-09-19T06:59:52.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.818+0000 7fccffa5c740 1 --2- 172.21.15.103:0/2833776294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d125a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.818+0000 7fccffa5c740 1 --2- 172.21.15.103:0/2833776294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d12596b40 0x559d125ac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.818+0000 7fccffa5c740 1 --2- 172.21.15.103:0/2833776294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d125955d0 0x559d1259cfa0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.818+0000 7fccffa5c740 1 -- 172.21.15.103:0/2833776294 >> 172.21.15.103:0/2833776294 conn(0x559d1255e310 msgr2=0x559d1259c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:52.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.818+0000 7fccffa5c740 1 -- 172.21.15.103:0/2833776294 shutdown_connections 2024-09-19T06:59:52.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.818+0000 7fccffa5c740 1 -- 172.21.15.103:0/2833776294 wait complete. 2024-09-19T06:59:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.818+0000 7fccffa5c740 1 -- start start 2024-09-19T06:59:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.818+0000 7fccffa5c740 1 Processor -- start 2024-09-19T06:59:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccffa5c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d125955d0 0x559d125a8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccffa5c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d12596b40 0x559d125a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccffa5c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d12599a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccffa5c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81b0ee0 con 0x559d12596b40 2024-09-19T06:59:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccffa5c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81b0810 con 0x559d125a0370 2024-09-19T06:59:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccffa5c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81ada70 con 0x559d125955d0 2024-09-19T06:59:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccffa5c740 10 client.0 _ll_register_callbacks cb 0x7fcce80097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccf6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d12596b40 0x559d125a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccf6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d12596b40 0x559d125a8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35312/0 (socket says 172.21.15.103:35312) 2024-09-19T06:59:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccf6ffd640 1 -- 172.21.15.103:0/3412895561 learned_addr learned my addr 172.21.15.103:0/3412895561 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccf67fc640 1 --2- 172.21.15.103:0/3412895561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d12599a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccf67fc640 1 --2- 172.21.15.103:0/3412895561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d125955d0 0x559d125a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccf6ffd640 1 -- 172.21.15.103:0/3412895561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d125955d0 msgr2=0x559d125a8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccf6ffd640 1 --2- 172.21.15.103:0/3412895561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d125955d0 0x559d125a8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccf6ffd640 1 -- 172.21.15.103:0/3412895561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 msgr2=0x559d12599a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccf6ffd640 1 --2- 172.21.15.103:0/3412895561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d12599a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccf6ffd640 1 -- 172.21.15.103:0/3412895561 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d125a0090 con 0x559d12596b40 2024-09-19T06:59:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccf67fc640 1 --2- 172.21.15.103:0/3412895561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d125955d0 0x559d125a8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccf6ffd640 1 --2- 172.21.15.103:0/3412895561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d12596b40 0x559d125a8c50 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fcce8002230 tx=0x7fcce8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c339d00c042fa036 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccdaff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccdaff5640 1 -- 172.21.15.103:0/3412895561 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8065060 con 0x559d12596b40 2024-09-19T06:59:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccdaff5640 1 -- 172.21.15.103:0/3412895561 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7fcce802fcb0 con 0x559d12596b40 2024-09-19T06:59:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.819+0000 7fccffa5c740 1 -- 172.21.15.103:0/3412895561 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccec023070 con 0x559d12596b40 2024-09-19T06:59:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.820+0000 7fccdaff5640 1 -- 172.21.15.103:0/3412895561 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcce802feb0 con 0x559d12596b40 2024-09-19T06:59:52.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.820+0000 7fccffa5c740 10 client.19068 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.820+0000 7fccffa5c740 1 -- 172.21.15.103:0/3412895561 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559d1259d8e0 con 0x559d12596b40 2024-09-19T06:59:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.820+0000 7fccdaff5640 1 -- 172.21.15.103:0/3412895561 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccec023070 con 0x559d12596b40 2024-09-19T06:59:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.820+0000 7fccdaff5640 1 -- 172.21.15.103:0/3412895561 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcce8004200 con 0x559d12596b40 2024-09-19T06:59:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.820+0000 7fccffa5c740 10 client.19068 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.820+0000 7fccffa5c740 10 client.19068 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T06:59:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.820+0000 7fccffa5c740 20 client.19068 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.820+0000 7fccffa5c740 1 --2- 172.21.15.103:0/3412895561 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x559d1261a710 0x559d1261cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.820+0000 7fccffa5c740 4 client.19068 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:59:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.820+0000 7fccffa5c740 1 -- 172.21.15.103:0/3412895561 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d12502d70 con 0x559d1261a710 2024-09-19T06:59:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.820+0000 7fccf67fc640 1 --2- 172.21.15.103:0/3412895561 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x559d1261a710 0x559d1261cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccf67fc640 1 --2- 172.21.15.103:0/3412895561 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x559d1261a710 0x559d1261cb00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5832db7f0506d0c7 server_cookie=6e794d8801c18d1 in_seq=0 out_seq=0 2024-09-19T06:59:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccdaff5640 10 client.19068 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T06:59:52.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccdaff5640 1 -- 172.21.15.103:0/3412895561 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d12502d70 con 0x559d1261a710 2024-09-19T06:59:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccdaff5640 10 client.19068 handle_command_reply: tid=0 2024-09-19T06:59:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccffa5c740 1 client.19068 shutdown 2024-09-19T06:59:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccffa5c740 1 -- 172.21.15.103:0/3412895561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d12596b40 msgr2=0x559d125a8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccffa5c740 1 --2- 172.21.15.103:0/3412895561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d12596b40 0x559d125a8c50 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fcce8002230 tx=0x7fcce8005bd0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccffa5c740 1 -- 172.21.15.103:0/3412895561 shutdown_connections 2024-09-19T06:59:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccffa5c740 1 --2- 172.21.15.103:0/3412895561 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x559d1261a710 0x559d1261cb00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccffa5c740 1 --2- 172.21.15.103:0/3412895561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d125a0370 0x559d12599a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccffa5c740 1 --2- 172.21.15.103:0/3412895561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d12596b40 0x559d125a8c50 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccffa5c740 1 --2- 172.21.15.103:0/3412895561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d125955d0 0x559d125a8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccdaff5640 0 client.19068 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T06:59:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccffa5c740 1 -- 172.21.15.103:0/3412895561 >> 172.21.15.103:0/3412895561 conn(0x559d1255e310 msgr2=0x559d1259bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccffa5c740 1 -- 172.21.15.103:0/3412895561 shutdown_connections 2024-09-19T06:59:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccffa5c740 1 -- 172.21.15.103:0/3412895561 wait complete. 2024-09-19T06:59:52.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.821+0000 7fccffa5c740 20 client.19068 trim_cache size 0 max 16384 2024-09-19T06:59:52.828 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:59:52.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.822+0000 7fccfd683640 1 -- 172.21.15.103:0/1063607631 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcce00a33b0 msgr2=0x7fcce00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:52.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.822+0000 7fccfd683640 1 --2- 172.21.15.103:0/1063607631 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcce00a33b0 0x7fcce00a5870 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fccf009b990 tx=0x7fccf00046c0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.822+0000 7fccfd683640 1 -- 172.21.15.103:0/1063607631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccf81082a0 msgr2=0x7fccf8101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:52.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.822+0000 7fccfd683640 1 --2- 172.21.15.103:0/1063607631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccf81082a0 0x7fccf8101410 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fccf000d750 tx=0x7fccf000db50 comp rx=0 tx=0).stop 2024-09-19T06:59:52.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.822+0000 7fccfd683640 1 -- 172.21.15.103:0/1063607631 shutdown_connections 2024-09-19T06:59:52.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.822+0000 7fccfd683640 1 --2- 172.21.15.103:0/1063607631 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcce00a33b0 0x7fcce00a5870 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.822+0000 7fccfd683640 1 --2- 172.21.15.103:0/1063607631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccf81082a0 0x7fccf8101410 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.822+0000 7fccfd683640 1 --2- 172.21.15.103:0/1063607631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccf81036e0 0x7fccf8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.822+0000 7fccfd683640 1 --2- 172.21.15.103:0/1063607631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccf8102d30 0x7fccf8100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:52.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.822+0000 7fccfd683640 1 -- 172.21.15.103:0/1063607631 >> 172.21.15.103:0/1063607631 conn(0x7fccf80fc7d0 msgr2=0x7fccf8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:52.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.822+0000 7fccfd683640 1 -- 172.21.15.103:0/1063607631 shutdown_connections 2024-09-19T06:59:52.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:52.822+0000 7fccfd683640 1 -- 172.21.15.103:0/1063607631 wait complete. 2024-09-19T06:59:53.546 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T06:59:53.793 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:54.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:53 smithi103 ceph-mon[31158]: pgmap v330: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2024-09-19T06:59:54.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:53 smithi134 ceph-mon[35356]: pgmap v330: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2024-09-19T06:59:54.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.251+0000 7f370f548640 1 -- 172.21.15.103:0/2342565142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3708102790 msgr2=0x7f37081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:54.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.251+0000 7f370f548640 1 --2- 172.21.15.103:0/2342565142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3708102790 0x7f37081067e0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009900 tx=0x7f36f802a410 comp rx=0 tx=0).stop 2024-09-19T06:59:54.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.252+0000 7f370f548640 1 -- 172.21.15.103:0/2342565142 shutdown_connections 2024-09-19T06:59:54.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.252+0000 7f370f548640 1 --2- 172.21.15.103:0/2342565142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3708106f10 0x7f370810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.252+0000 7f370f548640 1 --2- 172.21.15.103:0/2342565142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3708102790 0x7f37081067e0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.252+0000 7f370f548640 1 --2- 172.21.15.103:0/2342565142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3708101de0 0x7f37081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.252+0000 7f370f548640 1 -- 172.21.15.103:0/2342565142 >> 172.21.15.103:0/2342565142 conn(0x7f37080fd8c0 msgr2=0x7f37080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:54.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.252+0000 7f370f548640 1 -- 172.21.15.103:0/2342565142 shutdown_connections 2024-09-19T06:59:54.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.252+0000 7f370f548640 1 -- 172.21.15.103:0/2342565142 wait complete. 2024-09-19T06:59:54.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.252+0000 7f370f548640 1 Processor -- start 2024-09-19T06:59:54.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.252+0000 7f370f548640 1 -- start start 2024-09-19T06:59:54.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.252+0000 7f370f548640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3708101de0 0x7f3708198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.252+0000 7f370f548640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3708102790 0x7f37081988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.252+0000 7f370f548640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3708106f10 0x7f370819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.252+0000 7f370f548640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3708111f30 con 0x7f3708102790 2024-09-19T06:59:54.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.252+0000 7f370f548640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3708111db0 con 0x7f3708106f10 2024-09-19T06:59:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.252+0000 7f370f548640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37081120b0 con 0x7f3708101de0 2024-09-19T06:59:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.253+0000 7f370cae3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3708102790 0x7f37081988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.253+0000 7f370cae3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3708102790 0x7f37081988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35338/0 (socket says 172.21.15.103:35338) 2024-09-19T06:59:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.253+0000 7f370cae3640 1 -- 172.21.15.103:0/3424219607 learned_addr learned my addr 172.21.15.103:0/3424219607 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.253+0000 7f370dae5640 1 --2- 172.21.15.103:0/3424219607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3708106f10 0x7f370819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.253+0000 7f370d2e4640 1 --2- 172.21.15.103:0/3424219607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3708101de0 0x7f3708198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.253+0000 7f370cae3640 1 -- 172.21.15.103:0/3424219607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3708101de0 msgr2=0x7f3708198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.253+0000 7f370cae3640 1 --2- 172.21.15.103:0/3424219607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3708101de0 0x7f3708198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.253+0000 7f370cae3640 1 -- 172.21.15.103:0/3424219607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3708106f10 msgr2=0x7f370819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.253+0000 7f370cae3640 1 --2- 172.21.15.103:0/3424219607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3708106f10 0x7f370819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.253+0000 7f370cae3640 1 -- 172.21.15.103:0/3424219607 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370819d330 con 0x7f3708102790 2024-09-19T06:59:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.253+0000 7f370cae3640 1 --2- 172.21.15.103:0/3424219607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3708102790 0x7f37081988c0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f36f8006010 tx=0x7f36f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.253+0000 7f36fe7fc640 1 -- 172.21.15.103:0/3424219607 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f803f030 con 0x7f3708102790 2024-09-19T06:59:54.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.253+0000 7f36fe7fc640 1 -- 172.21.15.103:0/3424219607 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f8003ce0 con 0x7f3708102790 2024-09-19T06:59:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.253+0000 7f370f548640 1 -- 172.21.15.103:0/3424219607 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370819d5c0 con 0x7f3708102790 2024-09-19T06:59:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.253+0000 7f36fe7fc640 1 -- 172.21.15.103:0/3424219607 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8035070 con 0x7f3708102790 2024-09-19T06:59:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.253+0000 7f370f548640 1 -- 172.21.15.103:0/3424219607 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37081a4f00 con 0x7f3708102790 2024-09-19T06:59:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.256+0000 7f36fe7fc640 1 -- 172.21.15.103:0/3424219607 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f36f8042030 con 0x7f3708102790 2024-09-19T06:59:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.256+0000 7f3711921740 1 Processor -- start 2024-09-19T06:59:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.256+0000 7f3711921740 1 -- start start 2024-09-19T06:59:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.256+0000 7f3711921740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da6943a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.256+0000 7f3711921740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da6943a5d0 0x55da6943aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.256+0000 7f36fe7fc640 1 --2- 172.21.15.103:0/3424219607 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f36f40a33b0 0x7f36f40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.256+0000 7f3711921740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da69445370 0x55da69447760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f3711921740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55da693a02f0 con 0x55da6943bb40 2024-09-19T06:59:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f3711921740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55da69347cf0 con 0x55da6943a5d0 2024-09-19T06:59:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f3711921740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55da6939c2b0 con 0x55da69445370 2024-09-19T06:59:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f370dae5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da6943a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:54.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f370dae5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da6943a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35340/0 (socket says 172.21.15.103:35340) 2024-09-19T06:59:54.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f370dae5640 1 -- 172.21.15.103:0/1681866305 learned_addr learned my addr 172.21.15.103:0/1681866305 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:54.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f370dae5640 1 --2- 172.21.15.103:0/3424219607 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f36f40a33b0 0x7f36f40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:54.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f36fe7fc640 1 -- 172.21.15.103:0/3424219607 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f36f80f04b0 con 0x7f3708102790 2024-09-19T06:59:54.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f370d2e4640 1 --2- 172.21.15.103:0/1681866305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da6943a5d0 0x55da6943aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:54.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f370cae3640 1 --2- 172.21.15.103:0/1681866305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da69445370 0x55da69447760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:54.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f370dae5640 1 -- 172.21.15.103:0/1681866305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da69445370 msgr2=0x55da69447760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:54.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f370dae5640 1 --2- 172.21.15.103:0/1681866305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da69445370 0x55da69447760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f370dae5640 1 -- 172.21.15.103:0/1681866305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da6943a5d0 msgr2=0x55da6943aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:54.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f370dae5640 1 --2- 172.21.15.103:0/1681866305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da6943a5d0 0x55da6943aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f370dae5640 1 -- 172.21.15.103:0/1681866305 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da6932c3a0 con 0x55da6943bb40 2024-09-19T06:59:54.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f370cae3640 1 --2- 172.21.15.103:0/1681866305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da69445370 0x55da69447760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:54.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f370d2e4640 1 --2- 172.21.15.103:0/1681866305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da6943a5d0 0x55da6943aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f370dae5640 1 --2- 172.21.15.103:0/1681866305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da6943a090 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f3700012910 tx=0x7f3700012dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4f604cb73cc12b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f36f37fe640 1 -- 172.21.15.103:0/1681866305 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370001d9d0 con 0x55da6943bb40 2024-09-19T06:59:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f36f37fe640 1 -- 172.21.15.103:0/1681866305 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3700022070 con 0x55da6943bb40 2024-09-19T06:59:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f36f37fe640 1 -- 172.21.15.103:0/1681866305 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700011c70 con 0x55da6943bb40 2024-09-19T06:59:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.257+0000 7f370dae5640 1 --2- 172.21.15.103:0/3424219607 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f36f40a33b0 0x7f36f40a5870 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3700021000 tx=0x7f370001d000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.258+0000 7f3711921740 1 -- 172.21.15.103:0/1681866305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 msgr2=0x55da6943a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.258+0000 7f3711921740 1 --2- 172.21.15.103:0/1681866305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da6943a090 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f3700012910 tx=0x7f3700012dd0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.258+0000 7f3711921740 1 -- 172.21.15.103:0/1681866305 shutdown_connections 2024-09-19T06:59:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.258+0000 7f3711921740 1 --2- 172.21.15.103:0/1681866305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da69445370 0x55da69447760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.258+0000 7f3711921740 1 --2- 172.21.15.103:0/1681866305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da6943a5d0 0x55da6943aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.258+0000 7f3711921740 1 --2- 172.21.15.103:0/1681866305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da6943a090 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.258+0000 7f3711921740 1 -- 172.21.15.103:0/1681866305 >> 172.21.15.103:0/1681866305 conn(0x55da69403310 msgr2=0x55da69436c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:54.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.258+0000 7f3711921740 1 -- 172.21.15.103:0/1681866305 shutdown_connections 2024-09-19T06:59:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.258+0000 7f3711921740 1 -- 172.21.15.103:0/1681866305 wait complete. 2024-09-19T06:59:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.258+0000 7f3711921740 1 -- start start 2024-09-19T06:59:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f3711921740 1 Processor -- start 2024-09-19T06:59:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f3711921740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da6943a5d0 0x55da69444db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f3711921740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da69442060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f3711921740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da69445370 0x55da694427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f3711921740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55da69448070 con 0x55da6943bb40 2024-09-19T06:59:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f3711921740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55da6935b060 con 0x55da69445370 2024-09-19T06:59:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f3711921740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55da694481f0 con 0x55da6943a5d0 2024-09-19T06:59:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f3711921740 10 client.0 _ll_register_callbacks cb 0x55da690ae8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f370cae3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da69442060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:54.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f370cae3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da69442060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35344/0 (socket says 172.21.15.103:35344) 2024-09-19T06:59:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f370cae3640 1 -- 172.21.15.103:0/3747064967 learned_addr learned my addr 172.21.15.103:0/3747064967 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f370dae5640 1 --2- 172.21.15.103:0/3747064967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da69445370 0x55da694427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f370d2e4640 1 --2- 172.21.15.103:0/3747064967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da6943a5d0 0x55da69444db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f370cae3640 1 -- 172.21.15.103:0/3747064967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da6943a5d0 msgr2=0x55da69444db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f370cae3640 1 --2- 172.21.15.103:0/3747064967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da6943a5d0 0x55da69444db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f370cae3640 1 -- 172.21.15.103:0/3747064967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da69445370 msgr2=0x55da694427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f370cae3640 1 --2- 172.21.15.103:0/3747064967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da69445370 0x55da694427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f370cae3640 1 -- 172.21.15.103:0/3747064967 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da691113c0 con 0x55da6943bb40 2024-09-19T06:59:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f370dae5640 1 --2- 172.21.15.103:0/3747064967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da69445370 0x55da694427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.259+0000 7f370d2e4640 1 --2- 172.21.15.103:0/3747064967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da6943a5d0 0x55da69444db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.260+0000 7f370cae3640 1 --2- 172.21.15.103:0/3747064967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da69442060 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f36f80b8c90 tx=0x7f36f80b8ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49460fc6088a908 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.260+0000 7f36f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.260+0000 7f36f0ff9640 1 -- 172.21.15.103:0/3747064967 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8049040 con 0x55da6943bb40 2024-09-19T06:59:54.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.260+0000 7f36f0ff9640 1 -- 172.21.15.103:0/3747064967 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f36f80b8ea0 con 0x55da6943bb40 2024-09-19T06:59:54.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.260+0000 7f3711921740 1 -- 172.21.15.103:0/3747064967 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3700022070 con 0x55da6943bb40 2024-09-19T06:59:54.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.260+0000 7f36f0ff9640 1 -- 172.21.15.103:0/3747064967 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36f80bc190 con 0x55da6943bb40 2024-09-19T06:59:54.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.260+0000 7f3711921740 10 client.19080 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:54.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.260+0000 7f3711921740 1 -- 172.21.15.103:0/3747064967 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55da69453b60 con 0x55da6943bb40 2024-09-19T06:59:54.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.260+0000 7f36f0ff9640 1 -- 172.21.15.103:0/3747064967 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3700022070 con 0x55da6943bb40 2024-09-19T06:59:54.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.260+0000 7f36f0ff9640 1 -- 172.21.15.103:0/3747064967 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f36f80c5010 con 0x55da6943bb40 2024-09-19T06:59:54.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.260+0000 7f3711921740 10 client.19080 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:54.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.260+0000 7f3711921740 10 client.19080 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:59:54.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.260+0000 7f3711921740 20 client.19080 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:54.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.260+0000 7f3711921740 1 --2- 172.21.15.103:0/3747064967 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55da69454fa0 0x55da69457430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.260+0000 7f3711921740 4 client.19080 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:54.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.260+0000 7f3711921740 1 -- 172.21.15.103:0/3747064967 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da693ad860 con 0x55da69454fa0 2024-09-19T06:59:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.261+0000 7f370d2e4640 1 --2- 172.21.15.103:0/3747064967 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55da69454fa0 0x55da69457430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.261+0000 7f36f0ff9640 10 client.19080 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:59:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.261+0000 7f370d2e4640 1 --2- 172.21.15.103:0/3747064967 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55da69454fa0 0x55da69457430 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d01be70def505a01 server_cookie=5348590018761606 in_seq=0 out_seq=0 2024-09-19T06:59:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.262+0000 7f36f0ff9640 1 -- 172.21.15.103:0/3747064967 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da693ad860 con 0x55da69454fa0 2024-09-19T06:59:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.262+0000 7f36f0ff9640 10 client.19080 handle_command_reply: tid=0 2024-09-19T06:59:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.262+0000 7f3711921740 1 client.19080 shutdown 2024-09-19T06:59:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.262+0000 7f3711921740 1 -- 172.21.15.103:0/3747064967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 msgr2=0x55da69442060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.262+0000 7f3711921740 1 --2- 172.21.15.103:0/3747064967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da69442060 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f36f80b8c90 tx=0x7f36f80b8ad0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.262+0000 7f3711921740 1 -- 172.21.15.103:0/3747064967 shutdown_connections 2024-09-19T06:59:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.262+0000 7f3711921740 1 --2- 172.21.15.103:0/3747064967 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55da69454fa0 0x55da69457430 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.262+0000 7f3711921740 1 --2- 172.21.15.103:0/3747064967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da69445370 0x55da694427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.262+0000 7f36f0ff9640 0 client.19080 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:59:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.262+0000 7f3711921740 1 --2- 172.21.15.103:0/3747064967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da69442060 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.262+0000 7f3711921740 1 --2- 172.21.15.103:0/3747064967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da6943a5d0 0x55da69444db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.262+0000 7f3711921740 1 -- 172.21.15.103:0/3747064967 >> 172.21.15.103:0/3747064967 conn(0x55da69403310 msgr2=0x55da6943c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:54.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.262+0000 7f3711921740 1 -- 172.21.15.103:0/3747064967 shutdown_connections 2024-09-19T06:59:54.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.262+0000 7f3711921740 1 -- 172.21.15.103:0/3747064967 wait complete. 2024-09-19T06:59:54.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.262+0000 7f3711921740 20 client.19080 trim_cache size 0 max 16384 2024-09-19T06:59:54.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f3711921740 1 Processor -- start 2024-09-19T06:59:54.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f3711921740 1 -- start start 2024-09-19T06:59:54.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f3711921740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943a5d0 0x55da69441920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f3711921740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da6943bb40 0x55da69443fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f3711921740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da69445370 0x55da6943d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f3711921740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37081ad6f0 con 0x55da6943a5d0 2024-09-19T06:59:54.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f3711921740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37081af230 con 0x55da6943bb40 2024-09-19T06:59:54.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f3711921740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37081addc0 con 0x55da69445370 2024-09-19T06:59:54.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f370d2e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943a5d0 0x55da69441920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f370d2e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943a5d0 0x55da69441920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35358/0 (socket says 172.21.15.103:35358) 2024-09-19T06:59:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f370d2e4640 1 -- 172.21.15.103:0/733745166 learned_addr learned my addr 172.21.15.103:0/733745166 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f370d2e4640 1 -- 172.21.15.103:0/733745166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da69445370 msgr2=0x55da6943d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f370d2e4640 1 --2- 172.21.15.103:0/733745166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da69445370 0x55da6943d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f370d2e4640 1 -- 172.21.15.103:0/733745166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da6943bb40 msgr2=0x55da69443fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f370d2e4640 1 --2- 172.21.15.103:0/733745166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da6943bb40 0x55da69443fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f370d2e4640 1 -- 172.21.15.103:0/733745166 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da69457ab0 con 0x55da6943a5d0 2024-09-19T06:59:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.275+0000 7f370d2e4640 1 --2- 172.21.15.103:0/733745166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943a5d0 0x55da69441920 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x55da69457c80 tx=0x7f370400a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4495f4658f69e2ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.276+0000 7f36f2ffd640 1 -- 172.21.15.103:0/733745166 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3704004250 con 0x55da6943a5d0 2024-09-19T06:59:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.276+0000 7f36f2ffd640 1 -- 172.21.15.103:0/733745166 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37040043f0 con 0x55da6943a5d0 2024-09-19T06:59:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.276+0000 7f36f2ffd640 1 -- 172.21.15.103:0/733745166 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370401b920 con 0x55da6943a5d0 2024-09-19T06:59:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.276+0000 7f3711921740 1 -- 172.21.15.103:0/733745166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943a5d0 msgr2=0x55da69441920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.276+0000 7f3711921740 1 --2- 172.21.15.103:0/733745166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943a5d0 0x55da69441920 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x55da69457c80 tx=0x7f370400a310 comp rx=0 tx=0).stop 2024-09-19T06:59:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.276+0000 7f3711921740 1 -- 172.21.15.103:0/733745166 shutdown_connections 2024-09-19T06:59:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.276+0000 7f3711921740 1 --2- 172.21.15.103:0/733745166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da69445370 0x55da6943d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.276+0000 7f3711921740 1 --2- 172.21.15.103:0/733745166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da6943bb40 0x55da69443fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.276+0000 7f3711921740 1 --2- 172.21.15.103:0/733745166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943a5d0 0x55da69441920 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.276+0000 7f3711921740 1 -- 172.21.15.103:0/733745166 >> 172.21.15.103:0/733745166 conn(0x55da69403310 msgr2=0x55da694438c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.276+0000 7f3711921740 1 -- 172.21.15.103:0/733745166 shutdown_connections 2024-09-19T06:59:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.276+0000 7f3711921740 1 -- 172.21.15.103:0/733745166 wait complete. 2024-09-19T06:59:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.277+0000 7f3711921740 1 -- start start 2024-09-19T06:59:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.277+0000 7f3711921740 1 Processor -- start 2024-09-19T06:59:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.277+0000 7f3711921740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da6943a5d0 0x55da694495b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.277+0000 7f3711921740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da69449af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.277+0000 7f3711921740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da69445370 0x55da694bed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.277+0000 7f3711921740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37081aeb60 con 0x55da6943bb40 2024-09-19T06:59:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.277+0000 7f3711921740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37081ae490 con 0x55da69445370 2024-09-19T06:59:54.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.277+0000 7f3711921740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37081ab6f0 con 0x55da6943a5d0 2024-09-19T06:59:54.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.277+0000 7f3711921740 10 client.0 _ll_register_callbacks cb 0x7f36f803b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:54.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.277+0000 7f370cae3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da69449af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:54.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.277+0000 7f370cae3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da69449af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35366/0 (socket says 172.21.15.103:35366) 2024-09-19T06:59:54.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.277+0000 7f370cae3640 1 -- 172.21.15.103:0/2165576031 learned_addr learned my addr 172.21.15.103:0/2165576031 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:54.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.277+0000 7f370dae5640 1 --2- 172.21.15.103:0/2165576031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da69445370 0x55da694bed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:54.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.277+0000 7f370d2e4640 1 --2- 172.21.15.103:0/2165576031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da6943a5d0 0x55da694495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:54.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.277+0000 7f370cae3640 1 -- 172.21.15.103:0/2165576031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da6943a5d0 msgr2=0x55da694495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:54.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f370cae3640 1 --2- 172.21.15.103:0/2165576031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da6943a5d0 0x55da694495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f370cae3640 1 -- 172.21.15.103:0/2165576031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da69445370 msgr2=0x55da694bed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:54.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f370cae3640 1 --2- 172.21.15.103:0/2165576031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da69445370 0x55da694bed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f370cae3640 1 -- 172.21.15.103:0/2165576031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da6944a3d0 con 0x55da6943bb40 2024-09-19T06:59:54.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f370dae5640 1 --2- 172.21.15.103:0/2165576031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da69445370 0x55da694bed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:54.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f370d2e4640 1 --2- 172.21.15.103:0/2165576031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da6943a5d0 0x55da694495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:54.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f370cae3640 1 --2- 172.21.15.103:0/2165576031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da69449af0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f36f8000c00 tx=0x7f36f80c5240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1be7ab91a11cf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:54.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f36f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:54.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f36f0ff9640 1 -- 172.21.15.103:0/2165576031 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f80044e0 con 0x55da6943bb40 2024-09-19T06:59:54.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f36f0ff9640 1 -- 172.21.15.103:0/2165576031 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f36f8081bb0 con 0x55da6943bb40 2024-09-19T06:59:54.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f3711921740 1 -- 172.21.15.103:0/2165576031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37040043f0 con 0x55da6943bb40 2024-09-19T06:59:54.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f36f0ff9640 1 -- 172.21.15.103:0/2165576031 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36f803bcb0 con 0x55da6943bb40 2024-09-19T06:59:54.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f3711921740 10 client.19086 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:54.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f3711921740 1 -- 172.21.15.103:0/2165576031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55da694c0360 con 0x55da6943bb40 2024-09-19T06:59:54.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f36f0ff9640 1 -- 172.21.15.103:0/2165576031 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37040043f0 con 0x55da6943bb40 2024-09-19T06:59:54.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f36f0ff9640 1 -- 172.21.15.103:0/2165576031 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f36f80bdb60 con 0x55da6943bb40 2024-09-19T06:59:54.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f3711921740 10 client.19086 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:54.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f3711921740 10 client.19086 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T06:59:54.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f3711921740 20 client.19086 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:54.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.278+0000 7f3711921740 1 --2- 172.21.15.103:0/2165576031 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55da694c0910 0x55da694c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:54.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f3711921740 4 client.19086 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:59:54.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f3711921740 1 -- 172.21.15.103:0/2165576031 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da693a7d70 con 0x55da694c0910 2024-09-19T06:59:54.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f370d2e4640 1 --2- 172.21.15.103:0/2165576031 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55da694c0910 0x55da694c2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:54.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f370d2e4640 1 --2- 172.21.15.103:0/2165576031 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55da694c0910 0x55da694c2d30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9d211805dade7074 server_cookie=937c8f54df5c5066 in_seq=0 out_seq=0 2024-09-19T06:59:54.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f36f0ff9640 10 client.19086 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T06:59:54.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f36f0ff9640 1 -- 172.21.15.103:0/2165576031 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da693a7d70 con 0x55da694c0910 2024-09-19T06:59:54.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f36f0ff9640 10 client.19086 handle_command_reply: tid=0 2024-09-19T06:59:54.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f3711921740 1 client.19086 shutdown 2024-09-19T06:59:54.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f3711921740 1 -- 172.21.15.103:0/2165576031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 msgr2=0x55da69449af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:54.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f3711921740 1 --2- 172.21.15.103:0/2165576031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da69449af0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f36f8000c00 tx=0x7f36f80c5240 comp rx=0 tx=0).stop 2024-09-19T06:59:54.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f3711921740 1 -- 172.21.15.103:0/2165576031 shutdown_connections 2024-09-19T06:59:54.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f3711921740 1 --2- 172.21.15.103:0/2165576031 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55da694c0910 0x55da694c2d30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f3711921740 1 --2- 172.21.15.103:0/2165576031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da69445370 0x55da694bed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f3711921740 1 --2- 172.21.15.103:0/2165576031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da6943bb40 0x55da69449af0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f36f0ff9640 0 client.19086 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T06:59:54.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f3711921740 1 --2- 172.21.15.103:0/2165576031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da6943a5d0 0x55da694495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.279+0000 7f3711921740 1 -- 172.21.15.103:0/2165576031 >> 172.21.15.103:0/2165576031 conn(0x55da69403310 msgr2=0x55da69450300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:54.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.280+0000 7f3711921740 1 -- 172.21.15.103:0/2165576031 shutdown_connections 2024-09-19T06:59:54.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.280+0000 7f3711921740 1 -- 172.21.15.103:0/2165576031 wait complete. 2024-09-19T06:59:54.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.280+0000 7f3711921740 20 client.19086 trim_cache size 0 max 16384 2024-09-19T06:59:54.297 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:59:54.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.280+0000 7f370f548640 1 -- 172.21.15.103:0/3424219607 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f36f40a33b0 msgr2=0x7f36f40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:54.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.280+0000 7f370f548640 1 --2- 172.21.15.103:0/3424219607 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f36f40a33b0 0x7f36f40a5870 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3700021000 tx=0x7f370001d000 comp rx=0 tx=0).stop 2024-09-19T06:59:54.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.280+0000 7f370f548640 1 -- 172.21.15.103:0/3424219607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3708102790 msgr2=0x7f37081988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:54.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.280+0000 7f370f548640 1 --2- 172.21.15.103:0/3424219607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3708102790 0x7f37081988c0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f36f8006010 tx=0x7f36f80042f0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.280+0000 7f370f548640 1 -- 172.21.15.103:0/3424219607 shutdown_connections 2024-09-19T06:59:54.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.280+0000 7f370f548640 1 --2- 172.21.15.103:0/3424219607 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f36f40a33b0 0x7f36f40a5870 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.280+0000 7f370f548640 1 --2- 172.21.15.103:0/3424219607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3708106f10 0x7f370819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.280+0000 7f370f548640 1 --2- 172.21.15.103:0/3424219607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3708102790 0x7f37081988c0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.280+0000 7f370f548640 1 --2- 172.21.15.103:0/3424219607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3708101de0 0x7f3708198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:54.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.280+0000 7f370f548640 1 -- 172.21.15.103:0/3424219607 >> 172.21.15.103:0/3424219607 conn(0x7f37080fd8c0 msgr2=0x7f37080fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:54.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.280+0000 7f370f548640 1 -- 172.21.15.103:0/3424219607 shutdown_connections 2024-09-19T06:59:54.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:54.280+0000 7f370f548640 1 -- 172.21.15.103:0/3424219607 wait complete. 2024-09-19T06:59:54.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:53 smithi174 ceph-mon[35637]: pgmap v330: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2024-09-19T06:59:54.971 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:59:55.219 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:55.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.644+0000 7f54c0fee640 1 -- 172.21.15.103:0/2079594616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc100e20 msgr2=0x7f54bc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.644+0000 7f54c0fee640 1 --2- 172.21.15.103:0/2079594616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc100e20 0x7f54bc101200 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009900 tx=0x7f54ac02a480 comp rx=0 tx=0).stop 2024-09-19T06:59:55.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.644+0000 7f54c0fee640 1 -- 172.21.15.103:0/2079594616 shutdown_connections 2024-09-19T06:59:55.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.644+0000 7f54c0fee640 1 --2- 172.21.15.103:0/2079594616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54bc102ef0 0x7f54bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.644+0000 7f54c0fee640 1 --2- 172.21.15.103:0/2079594616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc102570 0x7f54bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.644+0000 7f54c0fee640 1 --2- 172.21.15.103:0/2079594616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc100e20 0x7f54bc101200 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.644+0000 7f54c0fee640 1 -- 172.21.15.103:0/2079594616 >> 172.21.15.103:0/2079594616 conn(0x7f54bc0fc7d0 msgr2=0x7f54bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:55.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.645+0000 7f54c0fee640 1 -- 172.21.15.103:0/2079594616 shutdown_connections 2024-09-19T06:59:55.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.645+0000 7f54c0fee640 1 -- 172.21.15.103:0/2079594616 wait complete. 2024-09-19T06:59:55.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.645+0000 7f54c0fee640 1 Processor -- start 2024-09-19T06:59:55.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.645+0000 7f54c0fee640 1 -- start start 2024-09-19T06:59:55.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.645+0000 7f54c0fee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc100e20 0x7f54bc197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.646+0000 7f54c0fee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc102570 0x7f54bc1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.646+0000 7f54c0fee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54bc102ef0 0x7f54bc19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.646+0000 7f54c0fee640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc110f80 con 0x7f54bc102570 2024-09-19T06:59:55.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.646+0000 7f54c0fee640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc110e00 con 0x7f54bc102ef0 2024-09-19T06:59:55.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.646+0000 7f54c0fee640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc111100 con 0x7f54bc100e20 2024-09-19T06:59:55.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.646+0000 7f54b9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc102570 0x7f54bc1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.646+0000 7f54b9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc102570 0x7f54bc1979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35388/0 (socket says 172.21.15.103:35388) 2024-09-19T06:59:55.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.646+0000 7f54b9d9b640 1 -- 172.21.15.103:0/3320176931 learned_addr learned my addr 172.21.15.103:0/3320176931 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:55.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.646+0000 7f54bad9d640 1 --2- 172.21.15.103:0/3320176931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54bc102ef0 0x7f54bc19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.646+0000 7f54b9d9b640 1 -- 172.21.15.103:0/3320176931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc100e20 msgr2=0x7f54bc197460 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T06:59:55.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.646+0000 7f54b9d9b640 1 --2- 172.21.15.103:0/3320176931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc100e20 0x7f54bc197460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.646+0000 7f54b9d9b640 1 -- 172.21.15.103:0/3320176931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54bc102ef0 msgr2=0x7f54bc19bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.646+0000 7f54b9d9b640 1 --2- 172.21.15.103:0/3320176931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54bc102ef0 0x7f54bc19bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.646+0000 7f54b9d9b640 1 -- 172.21.15.103:0/3320176931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54bc19c410 con 0x7f54bc102570 2024-09-19T06:59:55.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.647+0000 7f54b9d9b640 1 --2- 172.21.15.103:0/3320176931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc102570 0x7f54bc1979a0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f54b000afb0 tx=0x7f54b000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:55.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.647+0000 7f54ab7fe640 1 -- 172.21.15.103:0/3320176931 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b001e070 con 0x7f54bc102570 2024-09-19T06:59:55.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.647+0000 7f54c0fee640 1 -- 172.21.15.103:0/3320176931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54bc19c700 con 0x7f54bc102570 2024-09-19T06:59:55.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.647+0000 7f54ab7fe640 1 -- 172.21.15.103:0/3320176931 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b0011c30 con 0x7f54bc102570 2024-09-19T06:59:55.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.647+0000 7f54ab7fe640 1 -- 172.21.15.103:0/3320176931 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b00199e0 con 0x7f54bc102570 2024-09-19T06:59:55.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.647+0000 7f54c0fee640 1 -- 172.21.15.103:0/3320176931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54bc069f70 con 0x7f54bc102570 2024-09-19T06:59:55.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.649+0000 7f54ab7fe640 1 -- 172.21.15.103:0/3320176931 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f54b0021070 con 0x7f54bc102570 2024-09-19T06:59:55.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.650+0000 7f54c33c7740 1 Processor -- start 2024-09-19T06:59:55.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.650+0000 7f54c33c7740 1 -- start start 2024-09-19T06:59:55.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.650+0000 7f54c33c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e918b40 0x55ee6e917090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.650+0000 7f54c33c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e9175d0 0x55ee6e917a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.650+0000 7f54c33c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e922370 0x55ee6e924760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.650+0000 7f54c33c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ee6e87d2f0 con 0x55ee6e922370 2024-09-19T06:59:55.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.650+0000 7f54c33c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ee6e824cf0 con 0x55ee6e9175d0 2024-09-19T06:59:55.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.650+0000 7f54c33c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ee6e8792b0 con 0x55ee6e918b40 2024-09-19T06:59:55.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.650+0000 7f54ab7fe640 1 --2- 172.21.15.103:0/3320176931 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f54a40a33b0 0x7f54a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.650+0000 7f54b9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e922370 0x55ee6e924760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.650+0000 7f54b9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e922370 0x55ee6e924760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35404/0 (socket says 172.21.15.103:35404) 2024-09-19T06:59:55.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.650+0000 7f54b9d9b640 1 -- 172.21.15.103:0/290089269 learned_addr learned my addr 172.21.15.103:0/290089269 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:55.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.650+0000 7f54bad9d640 1 --2- 172.21.15.103:0/290089269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e918b40 0x55ee6e917090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.650+0000 7f54ba59c640 1 --2- 172.21.15.103:0/290089269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e9175d0 0x55ee6e917a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.650+0000 7f54bad9d640 1 --2- 172.21.15.103:0/3320176931 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f54a40a33b0 0x7f54a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.650+0000 7f54ab7fe640 1 -- 172.21.15.103:0/3320176931 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f54b00d3970 con 0x7f54bc102570 2024-09-19T06:59:55.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.651+0000 7f54bad9d640 1 -- 172.21.15.103:0/290089269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e9175d0 msgr2=0x55ee6e917a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:55.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.651+0000 7f54bad9d640 1 --2- 172.21.15.103:0/290089269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e9175d0 0x55ee6e917a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.651+0000 7f54bad9d640 1 -- 172.21.15.103:0/290089269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e922370 msgr2=0x55ee6e924760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:55.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.651+0000 7f54bad9d640 1 --2- 172.21.15.103:0/290089269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e922370 0x55ee6e924760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.651+0000 7f54bad9d640 1 -- 172.21.15.103:0/290089269 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee6e8093a0 con 0x55ee6e918b40 2024-09-19T06:59:55.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.651+0000 7f54b9d9b640 1 --2- 172.21.15.103:0/290089269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e922370 0x55ee6e924760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:59:55.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.651+0000 7f54bad9d640 1 --2- 172.21.15.103:0/290089269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e918b40 0x55ee6e917090 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f54b400c990 tx=0x7f54b400ce50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8368486459326b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:55.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.651+0000 7f54bad9d640 1 --2- 172.21.15.103:0/3320176931 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f54a40a33b0 0x7f54a40a5870 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f54b4017630 tx=0x7f54b400e000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:55.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.651+0000 7f54a8ff9640 1 -- 172.21.15.103:0/290089269 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b401f070 con 0x55ee6e918b40 2024-09-19T06:59:55.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.651+0000 7f54a8ff9640 1 -- 172.21.15.103:0/290089269 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b4003e50 con 0x55ee6e918b40 2024-09-19T06:59:55.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.651+0000 7f54a8ff9640 1 -- 172.21.15.103:0/290089269 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4019c40 con 0x55ee6e918b40 2024-09-19T06:59:55.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.652+0000 7f54c33c7740 1 -- 172.21.15.103:0/290089269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e918b40 msgr2=0x55ee6e917090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.652+0000 7f54c33c7740 1 --2- 172.21.15.103:0/290089269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e918b40 0x55ee6e917090 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f54b400c990 tx=0x7f54b400ce50 comp rx=0 tx=0).stop 2024-09-19T06:59:55.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.652+0000 7f54c33c7740 1 -- 172.21.15.103:0/290089269 shutdown_connections 2024-09-19T06:59:55.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.652+0000 7f54c33c7740 1 --2- 172.21.15.103:0/290089269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e922370 0x55ee6e924760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.652+0000 7f54c33c7740 1 --2- 172.21.15.103:0/290089269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e9175d0 0x55ee6e917a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.652+0000 7f54c33c7740 1 --2- 172.21.15.103:0/290089269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e918b40 0x55ee6e917090 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.652+0000 7f54c33c7740 1 -- 172.21.15.103:0/290089269 >> 172.21.15.103:0/290089269 conn(0x55ee6e8e0310 msgr2=0x55ee6e913c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:55.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.652+0000 7f54c33c7740 1 -- 172.21.15.103:0/290089269 shutdown_connections 2024-09-19T06:59:55.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.652+0000 7f54c33c7740 1 -- 172.21.15.103:0/290089269 wait complete. 2024-09-19T06:59:55.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.652+0000 7f54c33c7740 1 -- start start 2024-09-19T06:59:55.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54c33c7740 1 Processor -- start 2024-09-19T06:59:55.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54c33c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e9175d0 0x55ee6e921db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54c33c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e918b40 0x55ee6e91f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54c33c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e922370 0x55ee6e91f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54c33c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ee6e925010 con 0x55ee6e9175d0 2024-09-19T06:59:55.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54c33c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ee6e838060 con 0x55ee6e922370 2024-09-19T06:59:55.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54c33c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ee6e925190 con 0x55ee6e918b40 2024-09-19T06:59:55.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54c33c7740 10 client.0 _ll_register_callbacks cb 0x55ee6e58b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:55.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54ba59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e9175d0 0x55ee6e921db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54ba59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e9175d0 0x55ee6e921db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35410/0 (socket says 172.21.15.103:35410) 2024-09-19T06:59:55.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54ba59c640 1 -- 172.21.15.103:0/2173282329 learned_addr learned my addr 172.21.15.103:0/2173282329 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:55.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54bad9d640 1 --2- 172.21.15.103:0/2173282329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e922370 0x55ee6e91f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54b9d9b640 1 --2- 172.21.15.103:0/2173282329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e918b40 0x55ee6e91f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54ba59c640 1 -- 172.21.15.103:0/2173282329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e918b40 msgr2=0x55ee6e91f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:55.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54ba59c640 1 --2- 172.21.15.103:0/2173282329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e918b40 0x55ee6e91f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54ba59c640 1 -- 172.21.15.103:0/2173282329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e922370 msgr2=0x55ee6e91f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:55.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54ba59c640 1 --2- 172.21.15.103:0/2173282329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e922370 0x55ee6e91f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54ba59c640 1 -- 172.21.15.103:0/2173282329 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee6e5ee3c0 con 0x55ee6e9175d0 2024-09-19T06:59:55.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54bad9d640 1 --2- 172.21.15.103:0/2173282329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e922370 0x55ee6e91f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:55.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54ba59c640 1 --2- 172.21.15.103:0/2173282329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e9175d0 0x55ee6e921db0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f54ac006010 tx=0x7f54ac0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3d19340d92a97d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:55.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.653+0000 7f54a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:55.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.654+0000 7f54a1ffb640 1 -- 172.21.15.103:0/2173282329 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac040070 con 0x55ee6e9175d0 2024-09-19T06:59:55.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.654+0000 7f54a1ffb640 1 -- 172.21.15.103:0/2173282329 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f54ac003a00 con 0x55ee6e9175d0 2024-09-19T06:59:55.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.654+0000 7f54c33c7740 1 -- 172.21.15.103:0/2173282329 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54b4003e50 con 0x55ee6e9175d0 2024-09-19T06:59:55.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.654+0000 7f54a1ffb640 1 -- 172.21.15.103:0/2173282329 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54ac035040 con 0x55ee6e9175d0 2024-09-19T06:59:55.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.654+0000 7f54c33c7740 10 client.19098 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:55.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.654+0000 7f54c33c7740 1 -- 172.21.15.103:0/2173282329 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee6e930b00 con 0x55ee6e9175d0 2024-09-19T06:59:55.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.654+0000 7f54a1ffb640 1 -- 172.21.15.103:0/2173282329 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b4003e50 con 0x55ee6e9175d0 2024-09-19T06:59:55.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.654+0000 7f54a1ffb640 1 -- 172.21.15.103:0/2173282329 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f54ac045460 con 0x55ee6e9175d0 2024-09-19T06:59:55.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.654+0000 7f54c33c7740 10 client.19098 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:55.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.654+0000 7f54c33c7740 10 client.19098 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:59:55.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.654+0000 7f54c33c7740 20 client.19098 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:55.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.654+0000 7f54c33c7740 1 --2- 172.21.15.103:0/2173282329 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55ee6e931fc0 0x55ee6e9343b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.654+0000 7f54c33c7740 4 client.19098 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:55.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.654+0000 7f54c33c7740 1 -- 172.21.15.103:0/2173282329 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee6e88a860 con 0x55ee6e931fc0 2024-09-19T06:59:55.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.655+0000 7f54b9d9b640 1 --2- 172.21.15.103:0/2173282329 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55ee6e931fc0 0x55ee6e9343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.655+0000 7f54b9d9b640 1 --2- 172.21.15.103:0/2173282329 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55ee6e931fc0 0x55ee6e9343b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a21a714a63b1c10 server_cookie=9b6e7feef96ab310 in_seq=0 out_seq=0 2024-09-19T06:59:55.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.655+0000 7f54a1ffb640 10 client.19098 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:59:55.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.656+0000 7f54a1ffb640 1 -- 172.21.15.103:0/2173282329 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee6e88a860 con 0x55ee6e931fc0 2024-09-19T06:59:55.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.656+0000 7f54a1ffb640 10 client.19098 handle_command_reply: tid=0 2024-09-19T06:59:55.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.656+0000 7f54c33c7740 1 client.19098 shutdown 2024-09-19T06:59:55.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.656+0000 7f54c33c7740 1 -- 172.21.15.103:0/2173282329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e9175d0 msgr2=0x55ee6e921db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.656+0000 7f54c33c7740 1 --2- 172.21.15.103:0/2173282329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e9175d0 0x55ee6e921db0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f54ac006010 tx=0x7f54ac0043b0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.656+0000 7f54c33c7740 1 -- 172.21.15.103:0/2173282329 shutdown_connections 2024-09-19T06:59:55.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.656+0000 7f54c33c7740 1 --2- 172.21.15.103:0/2173282329 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55ee6e931fc0 0x55ee6e9343b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.656+0000 7f54c33c7740 1 --2- 172.21.15.103:0/2173282329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e922370 0x55ee6e91f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.656+0000 7f54c33c7740 1 --2- 172.21.15.103:0/2173282329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e918b40 0x55ee6e91f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.656+0000 7f54c33c7740 1 --2- 172.21.15.103:0/2173282329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e9175d0 0x55ee6e921db0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.656+0000 7f54c33c7740 1 -- 172.21.15.103:0/2173282329 >> 172.21.15.103:0/2173282329 conn(0x55ee6e8e0310 msgr2=0x55ee6e919960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:55.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.656+0000 7f54a1ffb640 0 client.19098 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:59:55.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.656+0000 7f54c33c7740 1 -- 172.21.15.103:0/2173282329 shutdown_connections 2024-09-19T06:59:55.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.656+0000 7f54c33c7740 1 -- 172.21.15.103:0/2173282329 wait complete. 2024-09-19T06:59:55.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.656+0000 7f54c33c7740 20 client.19098 trim_cache size 0 max 16384 2024-09-19T06:59:55.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54c33c7740 1 Processor -- start 2024-09-19T06:59:55.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54c33c7740 1 -- start start 2024-09-19T06:59:55.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54c33c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e9175d0 0x55ee6e91e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54c33c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e918b40 0x55ee6e913700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54c33c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e922370 0x55ee6e913c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54c33c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc1ab730 con 0x55ee6e918b40 2024-09-19T06:59:55.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54c33c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc1ad270 con 0x55ee6e9175d0 2024-09-19T06:59:55.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54c33c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc1abe00 con 0x55ee6e922370 2024-09-19T06:59:55.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54b9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e918b40 0x55ee6e913700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54b9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e918b40 0x55ee6e913700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35414/0 (socket says 172.21.15.103:35414) 2024-09-19T06:59:55.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54b9d9b640 1 -- 172.21.15.103:0/2390490029 learned_addr learned my addr 172.21.15.103:0/2390490029 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:55.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54b9d9b640 1 -- 172.21.15.103:0/2390490029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e922370 msgr2=0x55ee6e913c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:55.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54b9d9b640 1 --2- 172.21.15.103:0/2390490029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e922370 0x55ee6e913c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54b9d9b640 1 -- 172.21.15.103:0/2390490029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e9175d0 msgr2=0x55ee6e91e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:55.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54b9d9b640 1 --2- 172.21.15.103:0/2390490029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e9175d0 0x55ee6e91e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54b9d9b640 1 -- 172.21.15.103:0/2390490029 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee6e914960 con 0x55ee6e918b40 2024-09-19T06:59:55.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.669+0000 7f54b9d9b640 1 --2- 172.21.15.103:0/2390490029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e918b40 0x55ee6e913700 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f54b009fcd0 tx=0x7f54b009bfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435607677ca96616 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:55.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.670+0000 7f54a3fff640 1 -- 172.21.15.103:0/2390490029 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b00a40f0 con 0x55ee6e918b40 2024-09-19T06:59:55.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.670+0000 7f54a3fff640 1 -- 172.21.15.103:0/2390490029 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b00a89b0 con 0x55ee6e918b40 2024-09-19T06:59:55.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.670+0000 7f54a3fff640 1 -- 172.21.15.103:0/2390490029 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b00a0200 con 0x55ee6e918b40 2024-09-19T06:59:55.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.670+0000 7f54c33c7740 1 -- 172.21.15.103:0/2390490029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e918b40 msgr2=0x55ee6e913700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.670+0000 7f54c33c7740 1 --2- 172.21.15.103:0/2390490029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e918b40 0x55ee6e913700 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f54b009fcd0 tx=0x7f54b009bfe0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.670+0000 7f54c33c7740 1 -- 172.21.15.103:0/2390490029 shutdown_connections 2024-09-19T06:59:55.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.670+0000 7f54c33c7740 1 --2- 172.21.15.103:0/2390490029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e922370 0x55ee6e913c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.670+0000 7f54c33c7740 1 --2- 172.21.15.103:0/2390490029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e918b40 0x55ee6e913700 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.670+0000 7f54c33c7740 1 --2- 172.21.15.103:0/2390490029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e9175d0 0x55ee6e91e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.670+0000 7f54c33c7740 1 -- 172.21.15.103:0/2390490029 >> 172.21.15.103:0/2390490029 conn(0x55ee6e8e0310 msgr2=0x55ee6e91a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:55.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.670+0000 7f54c33c7740 1 -- 172.21.15.103:0/2390490029 shutdown_connections 2024-09-19T06:59:55.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.670+0000 7f54c33c7740 1 -- 172.21.15.103:0/2390490029 wait complete. 2024-09-19T06:59:55.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.670+0000 7f54c33c7740 1 -- start start 2024-09-19T06:59:55.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54c33c7740 1 Processor -- start 2024-09-19T06:59:55.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54c33c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e9175d0 0x55ee6e92a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54c33c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e918b40 0x55ee6e925bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54c33c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e922370 0x55ee6e926130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54c33c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc1acba0 con 0x55ee6e918b40 2024-09-19T06:59:55.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54c33c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc1ac4d0 con 0x55ee6e9175d0 2024-09-19T06:59:55.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54c33c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc1a9730 con 0x55ee6e922370 2024-09-19T06:59:55.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54c33c7740 10 client.0 _ll_register_callbacks cb 0x7f54ac004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:55.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54b9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e918b40 0x55ee6e925bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54b9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e918b40 0x55ee6e925bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35426/0 (socket says 172.21.15.103:35426) 2024-09-19T06:59:55.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54b9d9b640 1 -- 172.21.15.103:0/2170045341 learned_addr learned my addr 172.21.15.103:0/2170045341 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:55.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54ba59c640 1 --2- 172.21.15.103:0/2170045341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e9175d0 0x55ee6e92a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54bad9d640 1 --2- 172.21.15.103:0/2170045341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e922370 0x55ee6e926130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54b9d9b640 1 -- 172.21.15.103:0/2170045341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e922370 msgr2=0x55ee6e926130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:55.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54b9d9b640 1 --2- 172.21.15.103:0/2170045341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e922370 0x55ee6e926130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54b9d9b640 1 -- 172.21.15.103:0/2170045341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e9175d0 msgr2=0x55ee6e92a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:55.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54b9d9b640 1 --2- 172.21.15.103:0/2170045341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e9175d0 0x55ee6e92a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54b9d9b640 1 -- 172.21.15.103:0/2170045341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee6e926670 con 0x55ee6e918b40 2024-09-19T06:59:55.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54b9d9b640 1 --2- 172.21.15.103:0/2170045341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e918b40 0x55ee6e925bf0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f54b00b7040 tx=0x7f54b00d3d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b3da4c4eb5ab4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:55.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.671+0000 7f54a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:55.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.672+0000 7f54a1ffb640 1 -- 172.21.15.103:0/2170045341 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b009ffa0 con 0x55ee6e918b40 2024-09-19T06:59:55.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.672+0000 7f54a1ffb640 1 -- 172.21.15.103:0/2170045341 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f54b00a0140 con 0x55ee6e918b40 2024-09-19T06:59:55.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.672+0000 7f54c33c7740 1 -- 172.21.15.103:0/2170045341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54b00a89b0 con 0x55ee6e918b40 2024-09-19T06:59:55.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.672+0000 7f54a1ffb640 1 -- 172.21.15.103:0/2170045341 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54b00a8520 con 0x55ee6e918b40 2024-09-19T06:59:55.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.672+0000 7f54c33c7740 10 client.19104 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:55.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.672+0000 7f54c33c7740 1 -- 172.21.15.103:0/2170045341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee6e99c9b0 con 0x55ee6e918b40 2024-09-19T06:59:55.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.672+0000 7f54a1ffb640 1 -- 172.21.15.103:0/2170045341 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b00a89b0 con 0x55ee6e918b40 2024-09-19T06:59:55.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.672+0000 7f54a1ffb640 1 -- 172.21.15.103:0/2170045341 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f54b009f200 con 0x55ee6e918b40 2024-09-19T06:59:55.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.672+0000 7f54c33c7740 10 client.19104 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:55.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.672+0000 7f54c33c7740 10 client.19104 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T06:59:55.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.672+0000 7f54c33c7740 20 client.19104 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:55.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.672+0000 7f54c33c7740 1 --2- 172.21.15.103:0/2170045341 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55ee6e99d180 0x55ee6e99f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.672+0000 7f54c33c7740 4 client.19104 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:59:55.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.672+0000 7f54c33c7740 1 -- 172.21.15.103:0/2170045341 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee6e884d70 con 0x55ee6e99d180 2024-09-19T06:59:55.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54ba59c640 1 --2- 172.21.15.103:0/2170045341 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55ee6e99d180 0x55ee6e99f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54a1ffb640 10 client.19104 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T06:59:55.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54ba59c640 1 --2- 172.21.15.103:0/2170045341 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55ee6e99d180 0x55ee6e99f5a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c51513cac15cc76 server_cookie=5a6ccc61500a5c09 in_seq=0 out_seq=0 2024-09-19T06:59:55.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54a1ffb640 1 -- 172.21.15.103:0/2170045341 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee6e884d70 con 0x55ee6e99d180 2024-09-19T06:59:55.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54a1ffb640 10 client.19104 handle_command_reply: tid=0 2024-09-19T06:59:55.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54c33c7740 1 client.19104 shutdown 2024-09-19T06:59:55.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54c33c7740 1 -- 172.21.15.103:0/2170045341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e918b40 msgr2=0x55ee6e925bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54c33c7740 1 --2- 172.21.15.103:0/2170045341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e918b40 0x55ee6e925bf0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f54b00b7040 tx=0x7f54b00d3d40 comp rx=0 tx=0).stop 2024-09-19T06:59:55.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54c33c7740 1 -- 172.21.15.103:0/2170045341 shutdown_connections 2024-09-19T06:59:55.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54c33c7740 1 --2- 172.21.15.103:0/2170045341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee6e922370 0x55ee6e926130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54c33c7740 1 --2- 172.21.15.103:0/2170045341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee6e918b40 0x55ee6e925bf0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54c33c7740 1 --2- 172.21.15.103:0/2170045341 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55ee6e99d180 0x55ee6e99f5a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54c33c7740 1 --2- 172.21.15.103:0/2170045341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee6e9175d0 0x55ee6e92a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54c33c7740 1 -- 172.21.15.103:0/2170045341 >> 172.21.15.103:0/2170045341 conn(0x55ee6e8e0310 msgr2=0x55ee6e91d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:55.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54a1ffb640 0 client.19104 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T06:59:55.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54c33c7740 1 -- 172.21.15.103:0/2170045341 shutdown_connections 2024-09-19T06:59:55.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54c33c7740 1 -- 172.21.15.103:0/2170045341 wait complete. 2024-09-19T06:59:55.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.673+0000 7f54c33c7740 20 client.19104 trim_cache size 0 max 16384 2024-09-19T06:59:55.680 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:59:55.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.674+0000 7f54c0fee640 1 -- 172.21.15.103:0/3320176931 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f54a40a33b0 msgr2=0x7f54a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.674+0000 7f54c0fee640 1 --2- 172.21.15.103:0/3320176931 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f54a40a33b0 0x7f54a40a5870 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f54b4017630 tx=0x7f54b400e000 comp rx=0 tx=0).stop 2024-09-19T06:59:55.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.674+0000 7f54c0fee640 1 -- 172.21.15.103:0/3320176931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc102570 msgr2=0x7f54bc1979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.674+0000 7f54c0fee640 1 --2- 172.21.15.103:0/3320176931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc102570 0x7f54bc1979a0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f54b000afb0 tx=0x7f54b000cd50 comp rx=0 tx=0).stop 2024-09-19T06:59:55.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.674+0000 7f54c0fee640 1 -- 172.21.15.103:0/3320176931 shutdown_connections 2024-09-19T06:59:55.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.674+0000 7f54c0fee640 1 --2- 172.21.15.103:0/3320176931 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f54a40a33b0 0x7f54a40a5870 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.674+0000 7f54c0fee640 1 --2- 172.21.15.103:0/3320176931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54bc102ef0 0x7f54bc19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.674+0000 7f54c0fee640 1 --2- 172.21.15.103:0/3320176931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54bc102570 0x7f54bc1979a0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.674+0000 7f54c0fee640 1 --2- 172.21.15.103:0/3320176931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc100e20 0x7f54bc197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.674+0000 7f54c0fee640 1 -- 172.21.15.103:0/3320176931 >> 172.21.15.103:0/3320176931 conn(0x7f54bc0fc7d0 msgr2=0x7f54bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:55.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.674+0000 7f54c0fee640 1 -- 172.21.15.103:0/3320176931 shutdown_connections 2024-09-19T06:59:55.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:55.674+0000 7f54c0fee640 1 -- 172.21.15.103:0/3320176931 wait complete. 2024-09-19T06:59:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.913+0000 7f11c9d9f640 1 -- 172.21.15.103:0/1716291313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11c4100260 msgr2=0x7f11c4100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.913+0000 7f11c9d9f640 1 --2- 172.21.15.103:0/1716291313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11c4100260 0x7f11c4100640 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009900 tx=0x7f11c002a410 comp rx=0 tx=0).stop 2024-09-19T06:59:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.913+0000 7f11c9d9f640 1 -- 172.21.15.103:0/1716291313 shutdown_connections 2024-09-19T06:59:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.913+0000 7f11c9d9f640 1 --2- 172.21.15.103:0/1716291313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c4106c70 0x7f11c410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.913+0000 7f11c9d9f640 1 --2- 172.21.15.103:0/1716291313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11c4100c10 0x7f11c4106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.913+0000 7f11c9d9f640 1 --2- 172.21.15.103:0/1716291313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11c4100260 0x7f11c4100640 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.913+0000 7f11c9d9f640 1 -- 172.21.15.103:0/1716291313 >> 172.21.15.103:0/1716291313 conn(0x7f11c40fc300 msgr2=0x7f11c40fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.913+0000 7f11c9d9f640 1 -- 172.21.15.103:0/1716291313 shutdown_connections 2024-09-19T06:59:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.913+0000 7f11c9d9f640 1 -- 172.21.15.103:0/1716291313 wait complete. 2024-09-19T06:59:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.913+0000 7f11c9d9f640 1 Processor -- start 2024-09-19T06:59:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.913+0000 7f11c9d9f640 1 -- start start 2024-09-19T06:59:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.913+0000 7f11c9d9f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c4100260 0x7f11c4198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11c9d9f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11c4106c70 0x7f11c41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11c9d9f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11c419ca50 0x7f11c419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11c9d9f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f11c4111d00 con 0x7f11c4106c70 2024-09-19T06:59:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11c9d9f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f11c4111b80 con 0x7f11c4100260 2024-09-19T06:59:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11c9d9f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f11c4111e80 con 0x7f11c419ca50 2024-09-19T06:59:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11c4106c70 0x7f11c41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11c4106c70 0x7f11c41986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35440/0 (socket says 172.21.15.103:35440) 2024-09-19T06:59:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11bbfff640 1 -- 172.21.15.103:0/490148282 learned_addr learned my addr 172.21.15.103:0/490148282 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11c8d9d640 1 --2- 172.21.15.103:0/490148282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c4100260 0x7f11c4198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11c959e640 1 --2- 172.21.15.103:0/490148282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11c419ca50 0x7f11c419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11bbfff640 1 -- 172.21.15.103:0/490148282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11c419ca50 msgr2=0x7f11c419cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11bbfff640 1 --2- 172.21.15.103:0/490148282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11c419ca50 0x7f11c419cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11bbfff640 1 -- 172.21.15.103:0/490148282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c4100260 msgr2=0x7f11c4198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11bbfff640 1 --2- 172.21.15.103:0/490148282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c4100260 0x7f11c4198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11bbfff640 1 -- 172.21.15.103:0/490148282 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c419d540 con 0x7f11c4106c70 2024-09-19T06:59:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11c8d9d640 1 --2- 172.21.15.103:0/490148282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c4100260 0x7f11c4198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11c959e640 1 --2- 172.21.15.103:0/490148282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11c419ca50 0x7f11c419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11bbfff640 1 --2- 172.21.15.103:0/490148282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11c4106c70 0x7f11c41986d0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f11b400c8c0 tx=0x7f11b400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11b9ffb640 1 -- 172.21.15.103:0/490148282 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4007bb0 con 0x7f11c4106c70 2024-09-19T06:59:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11b9ffb640 1 -- 172.21.15.103:0/490148282 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b4007d50 con 0x7f11c4106c70 2024-09-19T06:59:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11c9d9f640 1 -- 172.21.15.103:0/490148282 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c419d830 con 0x7f11c4106c70 2024-09-19T06:59:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.914+0000 7f11b9ffb640 1 -- 172.21.15.103:0/490148282 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4018830 con 0x7f11c4106c70 2024-09-19T06:59:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.915+0000 7f11c9d9f640 1 -- 172.21.15.103:0/490148282 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c41a5020 con 0x7f11c4106c70 2024-09-19T06:59:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.917+0000 7f11b9ffb640 1 -- 172.21.15.103:0/490148282 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f11b4018a10 con 0x7f11c4106c70 2024-09-19T06:59:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.917+0000 7f11cde70740 1 Processor -- start 2024-09-19T06:59:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.917+0000 7f11cde70740 1 -- start start 2024-09-19T06:59:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.917+0000 7f11b9ffb640 1 --2- 172.21.15.103:0/490148282 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f11ac0a33b0 0x7f11ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.917+0000 7f11cde70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1651f8b80 0x55a1651f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.917+0000 7f11cde70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1651f94a0 0x55a165207d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.917+0000 7f11cde70740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a165208290 0x55a16520a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.917+0000 7f11cde70740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a16515d2f0 con 0x55a165208290 2024-09-19T06:59:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.917+0000 7f11cde70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a165105270 con 0x55a1651f94a0 2024-09-19T06:59:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.917+0000 7f11cde70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a1651592b0 con 0x55a1651f8b80 2024-09-19T06:59:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.917+0000 7f11c959e640 1 --2- 172.21.15.103:0/490148282 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f11ac0a33b0 0x7f11ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.917+0000 7f11b9ffb640 1 -- 172.21.15.103:0/490148282 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f11b40cff20 con 0x7f11c4106c70 2024-09-19T06:59:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.917+0000 7f11c959e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a165208290 0x55a16520a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.917+0000 7f11c959e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a165208290 0x55a16520a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35450/0 (socket says 172.21.15.103:35450) 2024-09-19T06:59:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.917+0000 7f11c959e640 1 -- 172.21.15.103:0/3265877756 learned_addr learned my addr 172.21.15.103:0/3265877756 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.918+0000 7f11c8d9d640 1 --2- 172.21.15.103:0/3265877756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1651f8b80 0x55a1651f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.918+0000 7f11bbfff640 1 --2- 172.21.15.103:0/3265877756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1651f94a0 0x55a165207d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.918+0000 7f11c959e640 1 -- 172.21.15.103:0/3265877756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1651f8b80 msgr2=0x55a1651f8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.918+0000 7f11c959e640 1 --2- 172.21.15.103:0/3265877756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1651f8b80 0x55a1651f8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.918+0000 7f11c959e640 1 -- 172.21.15.103:0/3265877756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1651f94a0 msgr2=0x55a165207d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.918+0000 7f11c959e640 1 --2- 172.21.15.103:0/3265877756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1651f94a0 0x55a165207d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.918+0000 7f11c959e640 1 -- 172.21.15.103:0/3265877756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a165104340 con 0x55a165208290 2024-09-19T06:59:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.918+0000 7f11c959e640 1 --2- 172.21.15.103:0/490148282 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f11ac0a33b0 0x7f11ac0a5870 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f11bc012a60 tx=0x7f11bc00acb0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.918+0000 7f11c959e640 1 --2- 172.21.15.103:0/3265877756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a165208290 0x55a16520a680 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00fba0 tx=0x7f11bc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b6368c2e97be31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.918+0000 7f11aaffd640 1 -- 172.21.15.103:0/3265877756 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc022070 con 0x55a165208290 2024-09-19T06:59:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.918+0000 7f11aaffd640 1 -- 172.21.15.103:0/3265877756 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11bc00dde0 con 0x55a165208290 2024-09-19T06:59:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.918+0000 7f11aaffd640 1 -- 172.21.15.103:0/3265877756 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc010630 con 0x55a165208290 2024-09-19T06:59:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.919+0000 7f11cde70740 1 -- 172.21.15.103:0/3265877756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a165208290 msgr2=0x55a16520a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.919+0000 7f11cde70740 1 --2- 172.21.15.103:0/3265877756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a165208290 0x55a16520a680 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00fba0 tx=0x7f11bc019000 comp rx=0 tx=0).stop 2024-09-19T06:59:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.919+0000 7f11cde70740 1 -- 172.21.15.103:0/3265877756 shutdown_connections 2024-09-19T06:59:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.919+0000 7f11cde70740 1 --2- 172.21.15.103:0/3265877756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a165208290 0x55a16520a680 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.919+0000 7f11cde70740 1 --2- 172.21.15.103:0/3265877756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1651f94a0 0x55a165207d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.919+0000 7f11cde70740 1 --2- 172.21.15.103:0/3265877756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1651f8b80 0x55a1651f8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.919+0000 7f11cde70740 1 -- 172.21.15.103:0/3265877756 >> 172.21.15.103:0/3265877756 conn(0x55a1651f5230 msgr2=0x55a1651f5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.919+0000 7f11cde70740 1 -- 172.21.15.103:0/3265877756 shutdown_connections 2024-09-19T06:59:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.919+0000 7f11cde70740 1 -- 172.21.15.103:0/3265877756 wait complete. 2024-09-19T06:59:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.919+0000 7f11cde70740 1 -- start start 2024-09-19T06:59:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.919+0000 7f11cde70740 1 Processor -- start 2024-09-19T06:59:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11cde70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1651f8b80 0x55a165205210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11cde70740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1651f94a0 0x55a165205750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11cde70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a165208290 0x55a165205c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11cde70740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a16520ae00 con 0x55a1651f94a0 2024-09-19T06:59:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11cde70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a1650e6290 con 0x55a165208290 2024-09-19T06:59:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11cde70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a16520af80 con 0x55a1651f8b80 2024-09-19T06:59:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11cde70740 10 client.0 _ll_register_callbacks cb 0x55a165117140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1651f94a0 0x55a165205750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1651f94a0 0x55a165205750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35452/0 (socket says 172.21.15.103:35452) 2024-09-19T06:59:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11bbfff640 1 -- 172.21.15.103:0/2157594539 learned_addr learned my addr 172.21.15.103:0/2157594539 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11c8d9d640 1 --2- 172.21.15.103:0/2157594539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1651f8b80 0x55a165205210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11c959e640 1 --2- 172.21.15.103:0/2157594539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a165208290 0x55a165205c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11bbfff640 1 -- 172.21.15.103:0/2157594539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1651f8b80 msgr2=0x55a165205210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11bbfff640 1 --2- 172.21.15.103:0/2157594539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1651f8b80 0x55a165205210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11bbfff640 1 -- 172.21.15.103:0/2157594539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a165208290 msgr2=0x55a165205c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11bbfff640 1 --2- 172.21.15.103:0/2157594539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a165208290 0x55a165205c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11bbfff640 1 -- 172.21.15.103:0/2157594539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a164e70210 con 0x55a1651f94a0 2024-09-19T06:59:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11c959e640 1 --2- 172.21.15.103:0/2157594539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a165208290 0x55a165205c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11bbfff640 1 --2- 172.21.15.103:0/2157594539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1651f94a0 0x55a165205750 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f11b4098640 tx=0x7f11b40aeff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91c83199956bcf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.920+0000 7f11a3fff640 1 -- 172.21.15.103:0/2157594539 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b40aeaf0 con 0x55a1651f94a0 2024-09-19T06:59:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.921+0000 7f11a3fff640 1 -- 172.21.15.103:0/2157594539 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f11b4018d00 con 0x55a1651f94a0 2024-09-19T06:59:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.921+0000 7f11cde70740 1 -- 172.21.15.103:0/2157594539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11bc00dde0 con 0x55a1651f94a0 2024-09-19T06:59:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.921+0000 7f11a3fff640 1 -- 172.21.15.103:0/2157594539 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f11b40ae050 con 0x55a1651f94a0 2024-09-19T06:59:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.921+0000 7f11cde70740 10 client.19116 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.921+0000 7f11cde70740 1 -- 172.21.15.103:0/2157594539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a164f3c620 con 0x55a1651f94a0 2024-09-19T06:59:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.921+0000 7f11a3fff640 1 -- 172.21.15.103:0/2157594539 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b40a8020 con 0x55a1651f94a0 2024-09-19T06:59:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.921+0000 7f11a3fff640 1 -- 172.21.15.103:0/2157594539 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f11b4098850 con 0x55a1651f94a0 2024-09-19T06:59:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.921+0000 7f11cde70740 10 client.19116 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.921+0000 7f11cde70740 10 client.19116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.921+0000 7f11cde70740 20 client.19116 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.921+0000 7f11cde70740 1 --2- 172.21.15.103:0/2157594539 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a1652182c0 0x55a16521a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.921+0000 7f11cde70740 4 client.19116 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.921+0000 7f11cde70740 1 -- 172.21.15.103:0/2157594539 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a16516aa20 con 0x55a1652182c0 2024-09-19T06:59:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.921+0000 7f11c8d9d640 1 --2- 172.21.15.103:0/2157594539 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a1652182c0 0x55a16521a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.922+0000 7f11a3fff640 10 client.19116 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.922+0000 7f11c8d9d640 1 --2- 172.21.15.103:0/2157594539 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a1652182c0 0x55a16521a6b0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ad27ba4f61f970c server_cookie=39da529af161f072 in_seq=0 out_seq=0 2024-09-19T06:59:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.922+0000 7f11a3fff640 1 -- 172.21.15.103:0/2157594539 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a16516aa20 con 0x55a1652182c0 2024-09-19T06:59:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.922+0000 7f11a3fff640 10 client.19116 handle_command_reply: tid=0 2024-09-19T06:59:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.922+0000 7f11cde70740 1 client.19116 shutdown 2024-09-19T06:59:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.923+0000 7f11cde70740 1 -- 172.21.15.103:0/2157594539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1651f94a0 msgr2=0x55a165205750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.923+0000 7f11cde70740 1 --2- 172.21.15.103:0/2157594539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1651f94a0 0x55a165205750 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f11b4098640 tx=0x7f11b40aeff0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.923+0000 7f11cde70740 1 -- 172.21.15.103:0/2157594539 shutdown_connections 2024-09-19T06:59:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.923+0000 7f11cde70740 1 --2- 172.21.15.103:0/2157594539 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a1652182c0 0x55a16521a6b0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.923+0000 7f11cde70740 1 --2- 172.21.15.103:0/2157594539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a165208290 0x55a165205c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.923+0000 7f11cde70740 1 --2- 172.21.15.103:0/2157594539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1651f94a0 0x55a165205750 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.923+0000 7f11a3fff640 0 client.19116 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.923+0000 7f11cde70740 1 --2- 172.21.15.103:0/2157594539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1651f8b80 0x55a165205210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.923+0000 7f11cde70740 1 -- 172.21.15.103:0/2157594539 >> 172.21.15.103:0/2157594539 conn(0x55a1651f5230 msgr2=0x55a1651faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.923+0000 7f11cde70740 1 -- 172.21.15.103:0/2157594539 shutdown_connections 2024-09-19T06:59:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.923+0000 7f11cde70740 1 -- 172.21.15.103:0/2157594539 wait complete. 2024-09-19T06:59:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.923+0000 7f11cde70740 20 client.19116 trim_cache size 0 max 16384 2024-09-19T06:59:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.937+0000 7f11cde70740 1 Processor -- start 2024-09-19T06:59:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.937+0000 7f11cde70740 1 -- start start 2024-09-19T06:59:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.937+0000 7f11cde70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1651f8b80 0x55a1651fadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.937+0000 7f11cde70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1651f94a0 0x55a16521ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.937+0000 7f11cde70740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a165208290 0x55a165220490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.937+0000 7f11cde70740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f11c41ab3f0 con 0x55a165208290 2024-09-19T06:59:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.937+0000 7f11cde70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f11c41acf30 con 0x55a1651f94a0 2024-09-19T06:59:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.937+0000 7f11cde70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f11c41abac0 con 0x55a1651f8b80 2024-09-19T06:59:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.937+0000 7f11c959e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a165208290 0x55a165220490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.937+0000 7f11c959e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a165208290 0x55a165220490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35462/0 (socket says 172.21.15.103:35462) 2024-09-19T06:59:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.937+0000 7f11c959e640 1 -- 172.21.15.103:0/3163800367 learned_addr learned my addr 172.21.15.103:0/3163800367 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.937+0000 7f11c8d9d640 1 --2- 172.21.15.103:0/3163800367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1651f8b80 0x55a1651fadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.937+0000 7f11bbfff640 1 --2- 172.21.15.103:0/3163800367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1651f94a0 0x55a16521ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.938+0000 7f11c959e640 1 -- 172.21.15.103:0/3163800367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1651f8b80 msgr2=0x55a1651fadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.938+0000 7f11c959e640 1 --2- 172.21.15.103:0/3163800367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1651f8b80 0x55a1651fadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.938+0000 7f11c959e640 1 -- 172.21.15.103:0/3163800367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1651f94a0 msgr2=0x55a16521ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.938+0000 7f11c959e640 1 --2- 172.21.15.103:0/3163800367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1651f94a0 0x55a16521ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.938+0000 7f11c959e640 1 -- 172.21.15.103:0/3163800367 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1651f92d0 con 0x55a165208290 2024-09-19T06:59:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.938+0000 7f11c959e640 1 --2- 172.21.15.103:0/3163800367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a165208290 0x55a165220490 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00e010 tx=0x7f11bc004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c473caca0abdf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.938+0000 7f11aa7fc640 1 -- 172.21.15.103:0/3163800367 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc035070 con 0x55a165208290 2024-09-19T06:59:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.938+0000 7f11aa7fc640 1 -- 172.21.15.103:0/3163800367 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11bc00a9c0 con 0x55a165208290 2024-09-19T06:59:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.938+0000 7f11aa7fc640 1 -- 172.21.15.103:0/3163800367 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc022040 con 0x55a165208290 2024-09-19T06:59:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.938+0000 7f11cde70740 1 -- 172.21.15.103:0/3163800367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a165208290 msgr2=0x55a165220490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.938+0000 7f11cde70740 1 --2- 172.21.15.103:0/3163800367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a165208290 0x55a165220490 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00e010 tx=0x7f11bc004a00 comp rx=0 tx=0).stop 2024-09-19T06:59:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.938+0000 7f11cde70740 1 -- 172.21.15.103:0/3163800367 shutdown_connections 2024-09-19T06:59:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.938+0000 7f11cde70740 1 --2- 172.21.15.103:0/3163800367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a165208290 0x55a165220490 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.938+0000 7f11cde70740 1 --2- 172.21.15.103:0/3163800367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1651f94a0 0x55a16521ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.938+0000 7f11cde70740 1 --2- 172.21.15.103:0/3163800367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1651f8b80 0x55a1651fadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11cde70740 1 -- 172.21.15.103:0/3163800367 >> 172.21.15.103:0/3163800367 conn(0x55a1651f5230 msgr2=0x55a1651fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11cde70740 1 -- 172.21.15.103:0/3163800367 shutdown_connections 2024-09-19T06:59:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11cde70740 1 -- 172.21.15.103:0/3163800367 wait complete. 2024-09-19T06:59:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11cde70740 1 -- start start 2024-09-19T06:59:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11cde70740 1 Processor -- start 2024-09-19T06:59:55.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11cde70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1651f8b80 0x55a16520c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11cde70740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1651f94a0 0x55a16520c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11cde70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a165208290 0x55a16520ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11cde70740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f11c41ac860 con 0x55a1651f94a0 2024-09-19T06:59:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11cde70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f11c41ac190 con 0x55a1651f8b80 2024-09-19T06:59:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11cde70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f11c41a93f0 con 0x55a165208290 2024-09-19T06:59:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11cde70740 10 client.0 _ll_register_callbacks cb 0x7f11b409bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1651f94a0 0x55a16520c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1651f94a0 0x55a16520c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35472/0 (socket says 172.21.15.103:35472) 2024-09-19T06:59:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11bbfff640 1 -- 172.21.15.103:0/861105310 learned_addr learned my addr 172.21.15.103:0/861105310 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11c8d9d640 1 --2- 172.21.15.103:0/861105310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1651f8b80 0x55a16520c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.939+0000 7f11c959e640 1 --2- 172.21.15.103:0/861105310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a165208290 0x55a16520ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11bbfff640 1 -- 172.21.15.103:0/861105310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a165208290 msgr2=0x55a16520ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11bbfff640 1 --2- 172.21.15.103:0/861105310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a165208290 0x55a16520ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11bbfff640 1 -- 172.21.15.103:0/861105310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1651f8b80 msgr2=0x55a16520c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11bbfff640 1 --2- 172.21.15.103:0/861105310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1651f8b80 0x55a16520c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11bbfff640 1 -- 172.21.15.103:0/861105310 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1652821c0 con 0x55a1651f94a0 2024-09-19T06:59:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11bbfff640 1 --2- 172.21.15.103:0/861105310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1651f94a0 0x55a16520c790 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f11b4098bb0 tx=0x7f11b409c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af40452a8c0164b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11a3fff640 1 -- 172.21.15.103:0/861105310 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b409c1e0 con 0x55a1651f94a0 2024-09-19T06:59:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11a3fff640 1 -- 172.21.15.103:0/861105310 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f11b4018d00 con 0x55a1651f94a0 2024-09-19T06:59:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11cde70740 1 -- 172.21.15.103:0/861105310 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11bc00a9c0 con 0x55a1651f94a0 2024-09-19T06:59:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11a3fff640 1 -- 172.21.15.103:0/861105310 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f11b40a4e50 con 0x55a1651f94a0 2024-09-19T06:59:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11cde70740 10 client.19122 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11cde70740 1 -- 172.21.15.103:0/861105310 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a1652834b0 con 0x55a1651f94a0 2024-09-19T06:59:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11a3fff640 1 -- 172.21.15.103:0/861105310 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11bc00a9c0 con 0x55a1651f94a0 2024-09-19T06:59:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11a3fff640 1 -- 172.21.15.103:0/861105310 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f11b40a49c0 con 0x55a1651f94a0 2024-09-19T06:59:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11cde70740 10 client.19122 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11cde70740 10 client.19122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11cde70740 20 client.19122 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11cde70740 1 --2- 172.21.15.103:0/861105310 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a165283b40 0x55a165285f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11cde70740 4 client.19122 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:59:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.940+0000 7f11cde70740 1 -- 172.21.15.103:0/861105310 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a165164d70 con 0x55a165283b40 2024-09-19T06:59:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.941+0000 7f11c8d9d640 1 --2- 172.21.15.103:0/861105310 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a165283b40 0x55a165285f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.941+0000 7f11a3fff640 10 client.19122 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.941+0000 7f11c8d9d640 1 --2- 172.21.15.103:0/861105310 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a165283b40 0x55a165285f30 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ee75c6c332e5958 server_cookie=32cb98715ddcdcb6 in_seq=0 out_seq=0 2024-09-19T06:59:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.950+0000 7f11a3fff640 1 -- 172.21.15.103:0/861105310 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a165164d70 con 0x55a165283b40 2024-09-19T06:59:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.950+0000 7f11a3fff640 10 client.19122 handle_command_reply: tid=0 2024-09-19T06:59:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.950+0000 7f11cde70740 1 client.19122 shutdown 2024-09-19T06:59:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.950+0000 7f11cde70740 1 -- 172.21.15.103:0/861105310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1651f94a0 msgr2=0x55a16520c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.950+0000 7f11cde70740 1 --2- 172.21.15.103:0/861105310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1651f94a0 0x55a16520c790 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f11b4098bb0 tx=0x7f11b409c040 comp rx=0 tx=0).stop 2024-09-19T06:59:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.950+0000 7f11cde70740 1 -- 172.21.15.103:0/861105310 shutdown_connections 2024-09-19T06:59:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.950+0000 7f11cde70740 1 --2- 172.21.15.103:0/861105310 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a165283b40 0x55a165285f30 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.950+0000 7f11cde70740 1 --2- 172.21.15.103:0/861105310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a165208290 0x55a16520ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.950+0000 7f11a3fff640 0 client.19122 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.950+0000 7f11cde70740 1 --2- 172.21.15.103:0/861105310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1651f94a0 0x55a16520c790 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.950+0000 7f11cde70740 1 --2- 172.21.15.103:0/861105310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1651f8b80 0x55a16520c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.950+0000 7f11cde70740 1 -- 172.21.15.103:0/861105310 >> 172.21.15.103:0/861105310 conn(0x55a1651f5230 msgr2=0x55a1651f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.950+0000 7f11cde70740 1 -- 172.21.15.103:0/861105310 shutdown_connections 2024-09-19T06:59:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.950+0000 7f11cde70740 1 -- 172.21.15.103:0/861105310 wait complete. 2024-09-19T06:59:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.950+0000 7f11cde70740 20 client.19122 trim_cache size 0 max 16384 2024-09-19T06:59:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T06:59:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T06:59:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T06:59:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T06:59:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.951+0000 7f11c9d9f640 1 -- 172.21.15.103:0/490148282 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f11ac0a33b0 msgr2=0x7f11ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.951+0000 7f11c9d9f640 1 --2- 172.21.15.103:0/490148282 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f11ac0a33b0 0x7f11ac0a5870 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f11bc012a60 tx=0x7f11bc00acb0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.951+0000 7f11c9d9f640 1 -- 172.21.15.103:0/490148282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11c4106c70 msgr2=0x7f11c41986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.951+0000 7f11c9d9f640 1 --2- 172.21.15.103:0/490148282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11c4106c70 0x7f11c41986d0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f11b400c8c0 tx=0x7f11b400cd80 comp rx=0 tx=0).stop 2024-09-19T06:59:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.951+0000 7f11c9d9f640 1 -- 172.21.15.103:0/490148282 shutdown_connections 2024-09-19T06:59:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.951+0000 7f11c9d9f640 1 --2- 172.21.15.103:0/490148282 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f11ac0a33b0 0x7f11ac0a5870 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.951+0000 7f11c9d9f640 1 --2- 172.21.15.103:0/490148282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11c419ca50 0x7f11c419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.951+0000 7f11c9d9f640 1 --2- 172.21.15.103:0/490148282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11c4106c70 0x7f11c41986d0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.951+0000 7f11c9d9f640 1 --2- 172.21.15.103:0/490148282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11c4100260 0x7f11c4198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.951+0000 7f11c9d9f640 1 -- 172.21.15.103:0/490148282 >> 172.21.15.103:0/490148282 conn(0x7f11c40fc300 msgr2=0x7f11c4105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.951+0000 7f11c9d9f640 1 -- 172.21.15.103:0/490148282 shutdown_connections 2024-09-19T06:59:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T06:59:55.951+0000 7f11c9d9f640 1 -- 172.21.15.103:0/490148282 wait complete. 2024-09-19T06:59:55.992 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:55 smithi103 ceph-mon[31158]: pgmap v331: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2024-09-19T06:59:56.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:55 smithi134 ceph-mon[35356]: pgmap v331: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2024-09-19T06:59:56.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:55 smithi174 ceph-mon[35637]: pgmap v331: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2024-09-19T06:59:56.397 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:59:56.683 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:57.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.120+0000 7f4677536640 1 -- 172.21.15.103:0/4260883362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4670102d40 msgr2=0x7f4670103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:57.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.120+0000 7f4677536640 1 --2- 172.21.15.103:0/4260883362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4670102d40 0x7f4670103120 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f4660009900 tx=0x7f466002a480 comp rx=0 tx=0).stop 2024-09-19T06:59:57.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.120+0000 7f4677536640 1 -- 172.21.15.103:0/4260883362 shutdown_connections 2024-09-19T06:59:57.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.120+0000 7f4677536640 1 --2- 172.21.15.103:0/4260883362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f467010c170 0x7f467010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.120+0000 7f4677536640 1 --2- 172.21.15.103:0/4260883362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4670103660 0x7f467010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.120+0000 7f4677536640 1 --2- 172.21.15.103:0/4260883362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4670102d40 0x7f4670103120 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.120+0000 7f4677536640 1 -- 172.21.15.103:0/4260883362 >> 172.21.15.103:0/4260883362 conn(0x7f46700fc7a0 msgr2=0x7f46700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:57.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.120+0000 7f4677536640 1 -- 172.21.15.103:0/4260883362 shutdown_connections 2024-09-19T06:59:57.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.120+0000 7f4677536640 1 -- 172.21.15.103:0/4260883362 wait complete. 2024-09-19T06:59:57.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.121+0000 7f4677536640 1 Processor -- start 2024-09-19T06:59:57.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.121+0000 7f4677536640 1 -- start start 2024-09-19T06:59:57.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.121+0000 7f4677536640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4670102d40 0x7f46701009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.121+0000 7f4677536640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4670103660 0x7f4670100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.121+0000 7f4677536640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f467010c170 0x7f4670101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.121+0000 7f4677536640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4670111f70 con 0x7f4670103660 2024-09-19T06:59:57.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.121+0000 7f4677536640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4670111df0 con 0x7f4670102d40 2024-09-19T06:59:57.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.121+0000 7f4677536640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46701120f0 con 0x7f467010c170 2024-09-19T06:59:57.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.121+0000 7f4674ad1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4670103660 0x7f4670100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.121+0000 7f4674ad1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4670103660 0x7f4670100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35498/0 (socket says 172.21.15.103:35498) 2024-09-19T06:59:57.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.121+0000 7f4674ad1640 1 -- 172.21.15.103:0/2770745989 learned_addr learned my addr 172.21.15.103:0/2770745989 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:57.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.121+0000 7f4675ad3640 1 --2- 172.21.15.103:0/2770745989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f467010c170 0x7f4670101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.121+0000 7f46752d2640 1 --2- 172.21.15.103:0/2770745989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4670102d40 0x7f46701009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.122+0000 7f4674ad1640 1 -- 172.21.15.103:0/2770745989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f467010c170 msgr2=0x7f4670101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:57.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.122+0000 7f4674ad1640 1 --2- 172.21.15.103:0/2770745989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f467010c170 0x7f4670101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.122+0000 7f4674ad1640 1 -- 172.21.15.103:0/2770745989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4670102d40 msgr2=0x7f46701009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:57.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.122+0000 7f4674ad1640 1 --2- 172.21.15.103:0/2770745989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4670102d40 0x7f46701009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.122+0000 7f4674ad1640 1 -- 172.21.15.103:0/2770745989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46701a6930 con 0x7f4670103660 2024-09-19T06:59:57.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.122+0000 7f4674ad1640 1 --2- 172.21.15.103:0/2770745989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4670103660 0x7f4670100ee0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f466c00d750 tx=0x7f466c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:57.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.122+0000 7f46667fc640 1 -- 172.21.15.103:0/2770745989 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c017070 con 0x7f4670103660 2024-09-19T06:59:57.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.122+0000 7f46667fc640 1 -- 172.21.15.103:0/2770745989 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f466c005020 con 0x7f4670103660 2024-09-19T06:59:57.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.122+0000 7f4677536640 1 -- 172.21.15.103:0/2770745989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46701a6c20 con 0x7f4670103660 2024-09-19T06:59:57.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.122+0000 7f46667fc640 1 -- 172.21.15.103:0/2770745989 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c004d00 con 0x7f4670103660 2024-09-19T06:59:57.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.122+0000 7f4677536640 1 -- 172.21.15.103:0/2770745989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46701a7160 con 0x7f4670103660 2024-09-19T06:59:57.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.125+0000 7f46667fc640 1 -- 172.21.15.103:0/2770745989 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f466c009a30 con 0x7f4670103660 2024-09-19T06:59:57.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.125+0000 7f467990f740 1 Processor -- start 2024-09-19T06:59:57.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.125+0000 7f467990f740 1 -- start start 2024-09-19T06:59:57.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.125+0000 7f467990f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d955eb40 0x5627d955d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.125+0000 7f467990f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d955d5d0 0x5627d955da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.125+0000 7f467990f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d9568370 0x5627d956a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.125+0000 7f467990f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5627d94c32f0 con 0x5627d955d5d0 2024-09-19T06:59:57.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.125+0000 7f467990f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5627d946acf0 con 0x5627d9568370 2024-09-19T06:59:57.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.125+0000 7f467990f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5627d94bf2b0 con 0x5627d955eb40 2024-09-19T06:59:57.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.125+0000 7f46667fc640 1 --2- 172.21.15.103:0/2770745989 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f465c0a3430 0x7f465c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.125+0000 7f46752d2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d955d5d0 0x5627d955da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.125+0000 7f46752d2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d955d5d0 0x5627d955da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35506/0 (socket says 172.21.15.103:35506) 2024-09-19T06:59:57.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.125+0000 7f46752d2640 1 -- 172.21.15.103:0/2526072337 learned_addr learned my addr 172.21.15.103:0/2526072337 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:57.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.125+0000 7f4675ad3640 1 --2- 172.21.15.103:0/2770745989 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f465c0a3430 0x7f465c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.126+0000 7f4674ad1640 1 --2- 172.21.15.103:0/2526072337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d9568370 0x5627d956a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.126+0000 7f4675ad3640 1 --2- 172.21.15.103:0/2526072337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d955eb40 0x5627d955d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.126+0000 7f46752d2640 1 -- 172.21.15.103:0/2526072337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d955eb40 msgr2=0x5627d955d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:57.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.126+0000 7f46752d2640 1 --2- 172.21.15.103:0/2526072337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d955eb40 0x5627d955d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.126+0000 7f46752d2640 1 -- 172.21.15.103:0/2526072337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d9568370 msgr2=0x5627d956a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:57.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.126+0000 7f46752d2640 1 --2- 172.21.15.103:0/2526072337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d9568370 0x5627d956a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.126+0000 7f46752d2640 1 -- 172.21.15.103:0/2526072337 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627d944f3a0 con 0x5627d955d5d0 2024-09-19T06:59:57.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.126+0000 7f46667fc640 1 -- 172.21.15.103:0/2770745989 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f466c0cf640 con 0x7f4670103660 2024-09-19T06:59:57.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.126+0000 7f4674ad1640 1 --2- 172.21.15.103:0/2526072337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d9568370 0x5627d956a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:57.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.126+0000 7f4675ad3640 1 --2- 172.21.15.103:0/2770745989 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f465c0a3430 0x7f465c0a58f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f466800afd0 tx=0x7f4668005bd0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:57.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.126+0000 7f46752d2640 1 --2- 172.21.15.103:0/2526072337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d955d5d0 0x5627d955da10 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f4660009900 tx=0x7f4660031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a274777d0ed234f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:57.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.126+0000 7f465b7fe640 1 -- 172.21.15.103:0/2526072337 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660040070 con 0x5627d955d5d0 2024-09-19T06:59:57.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.126+0000 7f465b7fe640 1 -- 172.21.15.103:0/2526072337 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4660005020 con 0x5627d955d5d0 2024-09-19T06:59:57.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.126+0000 7f465b7fe640 1 -- 172.21.15.103:0/2526072337 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660035040 con 0x5627d955d5d0 2024-09-19T06:59:57.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.127+0000 7f467990f740 1 -- 172.21.15.103:0/2526072337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d955d5d0 msgr2=0x5627d955da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:57.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.127+0000 7f467990f740 1 --2- 172.21.15.103:0/2526072337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d955d5d0 0x5627d955da10 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f4660009900 tx=0x7f4660031630 comp rx=0 tx=0).stop 2024-09-19T06:59:57.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.127+0000 7f467990f740 1 -- 172.21.15.103:0/2526072337 shutdown_connections 2024-09-19T06:59:57.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.127+0000 7f467990f740 1 --2- 172.21.15.103:0/2526072337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d9568370 0x5627d956a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.127+0000 7f467990f740 1 --2- 172.21.15.103:0/2526072337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d955d5d0 0x5627d955da10 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.127+0000 7f467990f740 1 --2- 172.21.15.103:0/2526072337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d955eb40 0x5627d955d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.127+0000 7f467990f740 1 -- 172.21.15.103:0/2526072337 >> 172.21.15.103:0/2526072337 conn(0x5627d9526310 msgr2=0x5627d9559c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:57.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.127+0000 7f467990f740 1 -- 172.21.15.103:0/2526072337 shutdown_connections 2024-09-19T06:59:57.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.127+0000 7f467990f740 1 -- 172.21.15.103:0/2526072337 wait complete. 2024-09-19T06:59:57.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.127+0000 7f467990f740 1 -- start start 2024-09-19T06:59:57.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.127+0000 7f467990f740 1 Processor -- start 2024-09-19T06:59:57.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f467990f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d955d5d0 0x5627d9567ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f467990f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d955eb40 0x5627d9564f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f467990f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d9568370 0x5627d9565490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f467990f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5627d956b010 con 0x5627d955eb40 2024-09-19T06:59:57.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f467990f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5627d947e060 con 0x5627d955d5d0 2024-09-19T06:59:57.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f467990f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5627d956b190 con 0x5627d9568370 2024-09-19T06:59:57.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f467990f740 10 client.0 _ll_register_callbacks cb 0x5627d91d18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:57.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f4674ad1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d955eb40 0x5627d9564f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f4674ad1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d955eb40 0x5627d9564f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35520/0 (socket says 172.21.15.103:35520) 2024-09-19T06:59:57.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f4674ad1640 1 -- 172.21.15.103:0/1712850789 learned_addr learned my addr 172.21.15.103:0/1712850789 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:57.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f4675ad3640 1 --2- 172.21.15.103:0/1712850789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d9568370 0x5627d9565490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f46752d2640 1 --2- 172.21.15.103:0/1712850789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d955d5d0 0x5627d9567ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f4674ad1640 1 -- 172.21.15.103:0/1712850789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d9568370 msgr2=0x5627d9565490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:57.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f4674ad1640 1 --2- 172.21.15.103:0/1712850789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d9568370 0x5627d9565490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f4674ad1640 1 -- 172.21.15.103:0/1712850789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d955d5d0 msgr2=0x5627d9567ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:57.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f4674ad1640 1 --2- 172.21.15.103:0/1712850789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d955d5d0 0x5627d9567ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f4674ad1640 1 -- 172.21.15.103:0/1712850789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627d92343c0 con 0x5627d955eb40 2024-09-19T06:59:57.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.128+0000 7f46752d2640 1 --2- 172.21.15.103:0/1712850789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d955d5d0 0x5627d9567ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:59:57.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.129+0000 7f4674ad1640 1 --2- 172.21.15.103:0/1712850789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d955eb40 0x5627d9564f50 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f466c09b6b0 tx=0x7f466c018a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2864c1cd3c4c98fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:57.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.129+0000 7f4658ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:57.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.129+0000 7f4658ff9640 1 -- 172.21.15.103:0/1712850789 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c020040 con 0x5627d955eb40 2024-09-19T06:59:57.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.129+0000 7f4658ff9640 1 -- 172.21.15.103:0/1712850789 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f466c086030 con 0x5627d955eb40 2024-09-19T06:59:57.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.129+0000 7f467990f740 1 -- 172.21.15.103:0/1712850789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4660005020 con 0x5627d955eb40 2024-09-19T06:59:57.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.129+0000 7f4658ff9640 1 -- 172.21.15.103:0/1712850789 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f466c0a4570 con 0x5627d955eb40 2024-09-19T06:59:57.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.129+0000 7f467990f740 10 client.19134 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:57.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.129+0000 7f467990f740 1 -- 172.21.15.103:0/1712850789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5627d95769f0 con 0x5627d955eb40 2024-09-19T06:59:57.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.129+0000 7f4658ff9640 1 -- 172.21.15.103:0/1712850789 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4660005020 con 0x5627d955eb40 2024-09-19T06:59:57.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.129+0000 7f4658ff9640 1 -- 172.21.15.103:0/1712850789 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f466c0ccff0 con 0x5627d955eb40 2024-09-19T06:59:57.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.129+0000 7f467990f740 10 client.19134 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:57.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.129+0000 7f467990f740 10 client.19134 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:59:57.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.129+0000 7f467990f740 20 client.19134 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:57.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.129+0000 7f467990f740 1 --2- 172.21.15.103:0/1712850789 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5627d9577d40 0x5627d957a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.129+0000 7f467990f740 4 client.19134 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:57.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.129+0000 7f467990f740 1 -- 172.21.15.103:0/1712850789 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627d94d0860 con 0x5627d9577d40 2024-09-19T06:59:57.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.130+0000 7f46752d2640 1 --2- 172.21.15.103:0/1712850789 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5627d9577d40 0x5627d957a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.130+0000 7f46752d2640 1 --2- 172.21.15.103:0/1712850789 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5627d9577d40 0x5627d957a190 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bcfb6fd7caecab7a server_cookie=5eb973e847ece278 in_seq=0 out_seq=0 2024-09-19T06:59:57.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.130+0000 7f4658ff9640 10 client.19134 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:59:57.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.131+0000 7f4658ff9640 1 -- 172.21.15.103:0/1712850789 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627d94d0860 con 0x5627d9577d40 2024-09-19T06:59:57.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.131+0000 7f4658ff9640 10 client.19134 handle_command_reply: tid=0 2024-09-19T06:59:57.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.131+0000 7f467990f740 1 client.19134 shutdown 2024-09-19T06:59:57.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.131+0000 7f467990f740 1 -- 172.21.15.103:0/1712850789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d955eb40 msgr2=0x5627d9564f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:57.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.131+0000 7f467990f740 1 --2- 172.21.15.103:0/1712850789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d955eb40 0x5627d9564f50 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f466c09b6b0 tx=0x7f466c018a40 comp rx=0 tx=0).stop 2024-09-19T06:59:57.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.131+0000 7f467990f740 1 -- 172.21.15.103:0/1712850789 shutdown_connections 2024-09-19T06:59:57.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.131+0000 7f467990f740 1 --2- 172.21.15.103:0/1712850789 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5627d9577d40 0x5627d957a190 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.131+0000 7f467990f740 1 --2- 172.21.15.103:0/1712850789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d9568370 0x5627d9565490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.131+0000 7f467990f740 1 --2- 172.21.15.103:0/1712850789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d955eb40 0x5627d9564f50 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.131+0000 7f467990f740 1 --2- 172.21.15.103:0/1712850789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d955d5d0 0x5627d9567ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.131+0000 7f467990f740 1 -- 172.21.15.103:0/1712850789 >> 172.21.15.103:0/1712850789 conn(0x5627d9526310 msgr2=0x5627d955f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:57.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.131+0000 7f4658ff9640 0 client.19134 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:59:57.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.131+0000 7f467990f740 1 -- 172.21.15.103:0/1712850789 shutdown_connections 2024-09-19T06:59:57.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.131+0000 7f467990f740 1 -- 172.21.15.103:0/1712850789 wait complete. 2024-09-19T06:59:57.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.131+0000 7f467990f740 20 client.19134 trim_cache size 0 max 16384 2024-09-19T06:59:57.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.144+0000 7f467990f740 1 Processor -- start 2024-09-19T06:59:57.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.144+0000 7f467990f740 1 -- start start 2024-09-19T06:59:57.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.144+0000 7f467990f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d955d5d0 0x5627d95742e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.144+0000 7f467990f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d9568370 0x5627d9569760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.144+0000 7f467990f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d9569ca0 0x5627d9560120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.144+0000 7f467990f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f46701afa20 con 0x5627d9568370 2024-09-19T06:59:57.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.144+0000 7f467990f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f46701b1560 con 0x5627d9569ca0 2024-09-19T06:59:57.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.144+0000 7f467990f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46701b00f0 con 0x5627d955d5d0 2024-09-19T06:59:57.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.144+0000 7f4674ad1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d9568370 0x5627d9569760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.145+0000 7f4674ad1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d9568370 0x5627d9569760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35526/0 (socket says 172.21.15.103:35526) 2024-09-19T06:59:57.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.145+0000 7f4674ad1640 1 -- 172.21.15.103:0/4257822983 learned_addr learned my addr 172.21.15.103:0/4257822983 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:57.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.145+0000 7f4675ad3640 1 --2- 172.21.15.103:0/4257822983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d9569ca0 0x5627d9560120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.145+0000 7f46752d2640 1 --2- 172.21.15.103:0/4257822983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d955d5d0 0x5627d95742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.145+0000 7f4674ad1640 1 -- 172.21.15.103:0/4257822983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d955d5d0 msgr2=0x5627d95742e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:57.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.145+0000 7f4674ad1640 1 --2- 172.21.15.103:0/4257822983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d955d5d0 0x5627d95742e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.145+0000 7f4674ad1640 1 -- 172.21.15.103:0/4257822983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d9569ca0 msgr2=0x5627d9560120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:57.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.145+0000 7f4674ad1640 1 --2- 172.21.15.103:0/4257822983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d9569ca0 0x5627d9560120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.145+0000 7f4674ad1640 1 -- 172.21.15.103:0/4257822983 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627d956a100 con 0x5627d9568370 2024-09-19T06:59:57.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.145+0000 7f4674ad1640 1 --2- 172.21.15.103:0/4257822983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d9568370 0x5627d9569760 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f466c099c60 tx=0x7f466c099ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76d0911512c637e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:57.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.145+0000 7f465affd640 1 -- 172.21.15.103:0/4257822983 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c020040 con 0x5627d9568370 2024-09-19T06:59:57.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.145+0000 7f465affd640 1 -- 172.21.15.103:0/4257822983 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f466c094020 con 0x5627d9568370 2024-09-19T06:59:57.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.145+0000 7f465affd640 1 -- 172.21.15.103:0/4257822983 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c09fcb0 con 0x5627d9568370 2024-09-19T06:59:57.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.146+0000 7f467990f740 1 -- 172.21.15.103:0/4257822983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d9568370 msgr2=0x5627d9569760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:57.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.146+0000 7f467990f740 1 --2- 172.21.15.103:0/4257822983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d9568370 0x5627d9569760 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f466c099c60 tx=0x7f466c099ec0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.146+0000 7f467990f740 1 -- 172.21.15.103:0/4257822983 shutdown_connections 2024-09-19T06:59:57.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.146+0000 7f467990f740 1 --2- 172.21.15.103:0/4257822983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d9569ca0 0x5627d9560120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.146+0000 7f467990f740 1 --2- 172.21.15.103:0/4257822983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d9568370 0x5627d9569760 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.146+0000 7f467990f740 1 --2- 172.21.15.103:0/4257822983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d955d5d0 0x5627d95742e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.146+0000 7f467990f740 1 -- 172.21.15.103:0/4257822983 >> 172.21.15.103:0/4257822983 conn(0x5627d9526310 msgr2=0x5627d9564f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:57.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.146+0000 7f467990f740 1 -- 172.21.15.103:0/4257822983 shutdown_connections 2024-09-19T06:59:57.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.146+0000 7f467990f740 1 -- 172.21.15.103:0/4257822983 wait complete. 2024-09-19T06:59:57.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.146+0000 7f467990f740 1 -- start start 2024-09-19T06:59:57.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.146+0000 7f467990f740 1 Processor -- start 2024-09-19T06:59:57.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f467990f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d955d5d0 0x5627d9570960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f467990f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d9568370 0x5627d9562240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f467990f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d9569ca0 0x5627d9562780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f467990f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f46701b0e90 con 0x5627d9568370 2024-09-19T06:59:57.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f467990f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f46701b07c0 con 0x5627d955d5d0 2024-09-19T06:59:57.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f467990f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46701ada20 con 0x5627d9569ca0 2024-09-19T06:59:57.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f467990f740 10 client.0 _ll_register_callbacks cb 0x7f466c0044e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:57.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f4674ad1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d9568370 0x5627d9562240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f4674ad1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d9568370 0x5627d9562240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35536/0 (socket says 172.21.15.103:35536) 2024-09-19T06:59:57.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f4674ad1640 1 -- 172.21.15.103:0/1400839066 learned_addr learned my addr 172.21.15.103:0/1400839066 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:57.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f4675ad3640 1 --2- 172.21.15.103:0/1400839066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d9569ca0 0x5627d9562780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f46752d2640 1 --2- 172.21.15.103:0/1400839066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d955d5d0 0x5627d9570960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f4674ad1640 1 -- 172.21.15.103:0/1400839066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d9569ca0 msgr2=0x5627d9562780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:57.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f4674ad1640 1 --2- 172.21.15.103:0/1400839066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d9569ca0 0x5627d9562780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f4674ad1640 1 -- 172.21.15.103:0/1400839066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d955d5d0 msgr2=0x5627d9570960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:57.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f4674ad1640 1 --2- 172.21.15.103:0/1400839066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d955d5d0 0x5627d9570960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f4674ad1640 1 -- 172.21.15.103:0/1400839066 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627d9562cc0 con 0x5627d9568370 2024-09-19T06:59:57.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f4674ad1640 1 --2- 172.21.15.103:0/1400839066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d9568370 0x5627d9562240 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f466c098380 tx=0x7f466c005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170bd250b60a97b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:57.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f4658ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:57.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f4658ff9640 1 -- 172.21.15.103:0/1400839066 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c0b9060 con 0x5627d9568370 2024-09-19T06:59:57.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f4658ff9640 1 -- 172.21.15.103:0/1400839066 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f466c098020 con 0x5627d9568370 2024-09-19T06:59:57.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.147+0000 7f467990f740 1 -- 172.21.15.103:0/1400839066 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f466c094020 con 0x5627d9568370 2024-09-19T06:59:57.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.148+0000 7f4658ff9640 1 -- 172.21.15.103:0/1400839066 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f466c020030 con 0x5627d9568370 2024-09-19T06:59:57.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.148+0000 7f467990f740 10 client.19140 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:57.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.148+0000 7f467990f740 1 -- 172.21.15.103:0/1400839066 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5627d9561ff0 con 0x5627d9568370 2024-09-19T06:59:57.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.148+0000 7f4658ff9640 1 -- 172.21.15.103:0/1400839066 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f466c094020 con 0x5627d9568370 2024-09-19T06:59:57.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.148+0000 7f4658ff9640 1 -- 172.21.15.103:0/1400839066 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f466c0040e0 con 0x5627d9568370 2024-09-19T06:59:57.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.148+0000 7f467990f740 10 client.19140 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:57.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.148+0000 7f467990f740 10 client.19140 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:59:57.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.148+0000 7f467990f740 20 client.19140 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:57.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.148+0000 7f467990f740 1 --2- 172.21.15.103:0/1400839066 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5627d95e3dd0 0x5627d95e61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.148+0000 7f467990f740 4 client.19140 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:59:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.148+0000 7f467990f740 1 -- 172.21.15.103:0/1400839066 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5627d94cad70 con 0x5627d95e3dd0 2024-09-19T06:59:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.148+0000 7f46752d2640 1 --2- 172.21.15.103:0/1400839066 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5627d95e3dd0 0x5627d95e61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f4658ff9640 10 client.19140 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T06:59:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f46752d2640 1 --2- 172.21.15.103:0/1400839066 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5627d95e3dd0 0x5627d95e61f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72700c526ab384fa server_cookie=394953c3150fbd6d in_seq=0 out_seq=0 2024-09-19T06:59:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f4658ff9640 1 -- 172.21.15.103:0/1400839066 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5627d94cad70 con 0x5627d95e3dd0 2024-09-19T06:59:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f4658ff9640 10 client.19140 handle_command_reply: tid=0 2024-09-19T06:59:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f467990f740 1 client.19140 shutdown 2024-09-19T06:59:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f467990f740 1 -- 172.21.15.103:0/1400839066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d9568370 msgr2=0x5627d9562240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f467990f740 1 --2- 172.21.15.103:0/1400839066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d9568370 0x5627d9562240 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f466c098380 tx=0x7f466c005400 comp rx=0 tx=0).stop 2024-09-19T06:59:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f467990f740 1 -- 172.21.15.103:0/1400839066 shutdown_connections 2024-09-19T06:59:57.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f467990f740 1 --2- 172.21.15.103:0/1400839066 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5627d95e3dd0 0x5627d95e61f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f467990f740 1 --2- 172.21.15.103:0/1400839066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627d9569ca0 0x5627d9562780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f467990f740 1 --2- 172.21.15.103:0/1400839066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627d9568370 0x5627d9562240 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f467990f740 1 --2- 172.21.15.103:0/1400839066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627d955d5d0 0x5627d9570960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f467990f740 1 -- 172.21.15.103:0/1400839066 >> 172.21.15.103:0/1400839066 conn(0x5627d9526310 msgr2=0x5627d95667b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:57.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f4658ff9640 0 client.19140 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T06:59:57.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f467990f740 1 -- 172.21.15.103:0/1400839066 shutdown_connections 2024-09-19T06:59:57.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f467990f740 1 -- 172.21.15.103:0/1400839066 wait complete. 2024-09-19T06:59:57.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.149+0000 7f467990f740 20 client.19140 trim_cache size 0 max 16384 2024-09-19T06:59:57.157 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:59:57.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.150+0000 7f4677536640 1 -- 172.21.15.103:0/2770745989 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f465c0a3430 msgr2=0x7f465c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.150+0000 7f4677536640 1 --2- 172.21.15.103:0/2770745989 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f465c0a3430 0x7f465c0a58f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f466800afd0 tx=0x7f4668005bd0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.150+0000 7f4677536640 1 -- 172.21.15.103:0/2770745989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4670103660 msgr2=0x7f4670100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.150+0000 7f4677536640 1 --2- 172.21.15.103:0/2770745989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4670103660 0x7f4670100ee0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f466c00d750 tx=0x7f466c00dc10 comp rx=0 tx=0).stop 2024-09-19T06:59:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.150+0000 7f4677536640 1 -- 172.21.15.103:0/2770745989 shutdown_connections 2024-09-19T06:59:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.150+0000 7f4677536640 1 --2- 172.21.15.103:0/2770745989 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f465c0a3430 0x7f465c0a58f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.150+0000 7f4677536640 1 --2- 172.21.15.103:0/2770745989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f467010c170 0x7f4670101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.150+0000 7f4677536640 1 --2- 172.21.15.103:0/2770745989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4670103660 0x7f4670100ee0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.150+0000 7f4677536640 1 --2- 172.21.15.103:0/2770745989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4670102d40 0x7f46701009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.150+0000 7f4677536640 1 -- 172.21.15.103:0/2770745989 >> 172.21.15.103:0/2770745989 conn(0x7f46700fc7a0 msgr2=0x7f46700fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.150+0000 7f4677536640 1 -- 172.21.15.103:0/2770745989 shutdown_connections 2024-09-19T06:59:57.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:57.150+0000 7f4677536640 1 -- 172.21.15.103:0/2770745989 wait complete. 2024-09-19T06:59:57.860 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:59:58.112 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:59:58.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:57 smithi103 ceph-mon[31158]: pgmap v332: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.6 KiB/s wr, 15 op/s 2024-09-19T06:59:58.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:57 smithi134 ceph-mon[35356]: pgmap v332: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.6 KiB/s wr, 15 op/s 2024-09-19T06:59:58.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:57 smithi174 ceph-mon[35637]: pgmap v332: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.6 KiB/s wr, 15 op/s 2024-09-19T06:59:58.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.538+0000 7f70c68cd640 1 -- 172.21.15.103:0/563506871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0100e20 msgr2=0x7f70c0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:58.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.538+0000 7f70c68cd640 1 --2- 172.21.15.103:0/563506871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0100e20 0x7f70c0101200 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009900 tx=0x7f70b002a480 comp rx=0 tx=0).stop 2024-09-19T06:59:58.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.538+0000 7f70c68cd640 1 -- 172.21.15.103:0/563506871 shutdown_connections 2024-09-19T06:59:58.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.538+0000 7f70c68cd640 1 --2- 172.21.15.103:0/563506871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0102ef0 0x7f70c010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.538+0000 7f70c68cd640 1 --2- 172.21.15.103:0/563506871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c0102570 0x7f70c01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.538+0000 7f70c68cd640 1 --2- 172.21.15.103:0/563506871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0100e20 0x7f70c0101200 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.538+0000 7f70c68cd640 1 -- 172.21.15.103:0/563506871 >> 172.21.15.103:0/563506871 conn(0x7f70c00fc7d0 msgr2=0x7f70c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:58.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.538+0000 7f70c68cd640 1 -- 172.21.15.103:0/563506871 shutdown_connections 2024-09-19T06:59:58.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.538+0000 7f70c68cd640 1 -- 172.21.15.103:0/563506871 wait complete. 2024-09-19T06:59:58.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.539+0000 7f70c68cd640 1 Processor -- start 2024-09-19T06:59:58.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.539+0000 7f70c68cd640 1 -- start start 2024-09-19T06:59:58.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.539+0000 7f70c68cd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0100e20 0x7f70c0197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.539+0000 7f70c68cd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0102570 0x7f70c0197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.539+0000 7f70c68cd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c0102ef0 0x7f70c019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.539+0000 7f70c68cd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70c0110f50 con 0x7f70c0100e20 2024-09-19T06:59:58.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.539+0000 7f70c68cd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70c0110dd0 con 0x7f70c0102570 2024-09-19T06:59:58.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.539+0000 7f70c68cd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01110d0 con 0x7f70c0102ef0 2024-09-19T06:59:58.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.539+0000 7f70bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0100e20 0x7f70c0197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.539+0000 7f70bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0100e20 0x7f70c0197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33572/0 (socket says 172.21.15.103:33572) 2024-09-19T06:59:58.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.539+0000 7f70bffff640 1 -- 172.21.15.103:0/665417956 learned_addr learned my addr 172.21.15.103:0/665417956 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:58.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.539+0000 7f70c4e6a640 1 --2- 172.21.15.103:0/665417956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c0102ef0 0x7f70c019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.540+0000 7f70bf7fe640 1 --2- 172.21.15.103:0/665417956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0102570 0x7f70c0197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.540+0000 7f70bffff640 1 -- 172.21.15.103:0/665417956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c0102ef0 msgr2=0x7f70c019bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:58.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.540+0000 7f70bffff640 1 --2- 172.21.15.103:0/665417956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c0102ef0 0x7f70c019bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.540+0000 7f70bffff640 1 -- 172.21.15.103:0/665417956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0102570 msgr2=0x7f70c0197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:58.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.540+0000 7f70bffff640 1 --2- 172.21.15.103:0/665417956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0102570 0x7f70c0197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.540+0000 7f70bffff640 1 -- 172.21.15.103:0/665417956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c019c400 con 0x7f70c0100e20 2024-09-19T06:59:58.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.540+0000 7f70bf7fe640 1 --2- 172.21.15.103:0/665417956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0102570 0x7f70c0197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:58.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.540+0000 7f70bffff640 1 --2- 172.21.15.103:0/665417956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0100e20 0x7f70c0197450 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f70b0006010 tx=0x7f70b0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:58.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.540+0000 7f70bd7fa640 1 -- 172.21.15.103:0/665417956 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b003f070 con 0x7f70c0100e20 2024-09-19T06:59:58.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.540+0000 7f70c68cd640 1 -- 172.21.15.103:0/665417956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c019c690 con 0x7f70c0100e20 2024-09-19T06:59:58.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.540+0000 7f70bd7fa640 1 -- 172.21.15.103:0/665417956 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b0031c30 con 0x7f70c0100e20 2024-09-19T06:59:58.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.540+0000 7f70bd7fa640 1 -- 172.21.15.103:0/665417956 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0035030 con 0x7f70c0100e20 2024-09-19T06:59:58.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.540+0000 7f70c68cd640 1 -- 172.21.15.103:0/665417956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c01a3fd0 con 0x7f70c0100e20 2024-09-19T06:59:58.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.542+0000 7f70bd7fa640 1 -- 172.21.15.103:0/665417956 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f70b0042070 con 0x7f70c0100e20 2024-09-19T06:59:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.543+0000 7f70bd7fa640 1 --2- 172.21.15.103:0/665417956 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70a80a33b0 0x7f70a80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.543+0000 7f70c8ca6740 1 Processor -- start 2024-09-19T06:59:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.543+0000 7f70c8ca6740 1 -- start start 2024-09-19T06:59:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.543+0000 7f70bf7fe640 1 --2- 172.21.15.103:0/665417956 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70a80a33b0 0x7f70a80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.543+0000 7f70bd7fa640 1 -- 172.21.15.103:0/665417956 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f70b00f0610 con 0x7f70c0100e20 2024-09-19T06:59:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.543+0000 7f70c8ca6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4ff3b40 0x556da4ff2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.543+0000 7f70c8ca6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff25d0 0x556da4ff2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.543+0000 7f70c8ca6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4fff580 0x556da5001970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.543+0000 7f70c8ca6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556da4f582f0 con 0x556da4ff25d0 2024-09-19T06:59:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.543+0000 7f70c8ca6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556da4effcf0 con 0x556da4fff580 2024-09-19T06:59:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.543+0000 7f70c8ca6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556da4f542b0 con 0x556da4ff3b40 2024-09-19T06:59:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.543+0000 7f70bf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff25d0 0x556da4ff2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.543+0000 7f70bf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff25d0 0x556da4ff2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33580/0 (socket says 172.21.15.103:33580) 2024-09-19T06:59:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.543+0000 7f70bf7fe640 1 -- 172.21.15.103:0/2876243871 learned_addr learned my addr 172.21.15.103:0/2876243871 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.544+0000 7f70c4e6a640 1 --2- 172.21.15.103:0/2876243871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4fff580 0x556da5001970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.544+0000 7f70bffff640 1 --2- 172.21.15.103:0/2876243871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4ff3b40 0x556da4ff2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.544+0000 7f70bf7fe640 1 --2- 172.21.15.103:0/665417956 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70a80a33b0 0x7f70a80a5870 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f70c00ffee0 tx=0x7f70b400f730 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.544+0000 7f70bf7fe640 1 -- 172.21.15.103:0/2876243871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4ff3b40 msgr2=0x556da4ff2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.544+0000 7f70bf7fe640 1 --2- 172.21.15.103:0/2876243871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4ff3b40 0x556da4ff2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.544+0000 7f70bf7fe640 1 -- 172.21.15.103:0/2876243871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4fff580 msgr2=0x556da5001970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.544+0000 7f70bf7fe640 1 --2- 172.21.15.103:0/2876243871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4fff580 0x556da5001970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.544+0000 7f70bf7fe640 1 -- 172.21.15.103:0/2876243871 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556da4ee43a0 con 0x556da4ff25d0 2024-09-19T06:59:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.544+0000 7f70bffff640 1 --2- 172.21.15.103:0/2876243871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4ff3b40 0x556da4ff2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:59:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.544+0000 7f70bf7fe640 1 --2- 172.21.15.103:0/2876243871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff25d0 0x556da4ff2a10 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f70b4019000 tx=0x7f70b401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468a5addb07eeb7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.544+0000 7f70a67fc640 1 -- 172.21.15.103:0/2876243871 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b401d620 con 0x556da4ff25d0 2024-09-19T06:59:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.544+0000 7f70a67fc640 1 -- 172.21.15.103:0/2876243871 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b401dca0 con 0x556da4ff25d0 2024-09-19T06:59:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.544+0000 7f70a67fc640 1 -- 172.21.15.103:0/2876243871 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4010d30 con 0x556da4ff25d0 2024-09-19T06:59:58.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.545+0000 7f70c8ca6740 1 -- 172.21.15.103:0/2876243871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff25d0 msgr2=0x556da4ff2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.545+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/2876243871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff25d0 0x556da4ff2a10 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f70b4019000 tx=0x7f70b401a000 comp rx=0 tx=0).stop 2024-09-19T06:59:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.545+0000 7f70c8ca6740 1 -- 172.21.15.103:0/2876243871 shutdown_connections 2024-09-19T06:59:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.545+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/2876243871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4fff580 0x556da5001970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.545+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/2876243871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff25d0 0x556da4ff2a10 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.545+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/2876243871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4ff3b40 0x556da4ff2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.545+0000 7f70c8ca6740 1 -- 172.21.15.103:0/2876243871 >> 172.21.15.103:0/2876243871 conn(0x556da4fbb310 msgr2=0x556da4feec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.545+0000 7f70c8ca6740 1 -- 172.21.15.103:0/2876243871 shutdown_connections 2024-09-19T06:59:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.545+0000 7f70c8ca6740 1 -- 172.21.15.103:0/2876243871 wait complete. 2024-09-19T06:59:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.545+0000 7f70c8ca6740 1 -- start start 2024-09-19T06:59:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c8ca6740 1 Processor -- start 2024-09-19T06:59:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c8ca6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4ff25d0 0x556da4ffa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c8ca6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4ff3b40 0x556da4ffabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c8ca6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4fff580 0x556da4ffb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c8ca6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556da4ff5550 con 0x556da4fff580 2024-09-19T06:59:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c8ca6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556da4f13060 con 0x556da4ff3b40 2024-09-19T06:59:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c8ca6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556da4ff56d0 con 0x556da4ff25d0 2024-09-19T06:59:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c8ca6740 10 client.0 _ll_register_callbacks cb 0x556da4c668c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c4e6a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4fff580 0x556da4ffb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c4e6a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4fff580 0x556da4ffb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33588/0 (socket says 172.21.15.103:33588) 2024-09-19T06:59:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c4e6a640 1 -- 172.21.15.103:0/3417996441 learned_addr learned my addr 172.21.15.103:0/3417996441 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70bffff640 1 --2- 172.21.15.103:0/3417996441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4ff25d0 0x556da4ffa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70bf7fe640 1 --2- 172.21.15.103:0/3417996441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4ff3b40 0x556da4ffabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c4e6a640 1 -- 172.21.15.103:0/3417996441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4ff25d0 msgr2=0x556da4ffa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c4e6a640 1 --2- 172.21.15.103:0/3417996441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4ff25d0 0x556da4ffa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c4e6a640 1 -- 172.21.15.103:0/3417996441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4ff3b40 msgr2=0x556da4ffabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c4e6a640 1 --2- 172.21.15.103:0/3417996441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4ff3b40 0x556da4ffabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c4e6a640 1 -- 172.21.15.103:0/3417996441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556da4cc93c0 con 0x556da4fff580 2024-09-19T06:59:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f70c4e6a640 1 --2- 172.21.15.103:0/3417996441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4fff580 0x556da4ffb150 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f70b8010a50 tx=0x7f70b8010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1767fdad7adf7650 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.546+0000 7f709f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.547+0000 7f709f7fe640 1 -- 172.21.15.103:0/3417996441 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b801e060 con 0x556da4fff580 2024-09-19T06:59:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.547+0000 7f709f7fe640 1 -- 172.21.15.103:0/3417996441 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f70b800fcf0 con 0x556da4fff580 2024-09-19T06:59:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.547+0000 7f70c8ca6740 1 -- 172.21.15.103:0/3417996441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b401dca0 con 0x556da4fff580 2024-09-19T06:59:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.547+0000 7f709f7fe640 1 -- 172.21.15.103:0/3417996441 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f70b8022430 con 0x556da4fff580 2024-09-19T06:59:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.547+0000 7f70c8ca6740 10 client.19152 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.547+0000 7f70c8ca6740 1 -- 172.21.15.103:0/3417996441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556da500bb90 con 0x556da4fff580 2024-09-19T06:59:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.547+0000 7f709f7fe640 1 -- 172.21.15.103:0/3417996441 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b401dca0 con 0x556da4fff580 2024-09-19T06:59:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.547+0000 7f709f7fe640 1 -- 172.21.15.103:0/3417996441 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f70b800e570 con 0x556da4fff580 2024-09-19T06:59:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.547+0000 7f70c8ca6740 10 client.19152 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:58.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.547+0000 7f70c8ca6740 10 client.19152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:58.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.547+0000 7f70c8ca6740 20 client.19152 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:58.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.547+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/3417996441 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556da500d050 0x556da500f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.547+0000 7f70c8ca6740 4 client.19152 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:59:58.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.547+0000 7f70c8ca6740 1 -- 172.21.15.103:0/3417996441 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556da4f65860 con 0x556da500d050 2024-09-19T06:59:58.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.548+0000 7f70bffff640 1 --2- 172.21.15.103:0/3417996441 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556da500d050 0x556da500f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.548+0000 7f709f7fe640 10 client.19152 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:58.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.548+0000 7f70bffff640 1 --2- 172.21.15.103:0/3417996441 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556da500d050 0x556da500f460 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd94dd36fa3de68d server_cookie=693c3d0d3ea900f2 in_seq=0 out_seq=0 2024-09-19T06:59:58.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.549+0000 7f709f7fe640 1 -- 172.21.15.103:0/3417996441 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556da4f65860 con 0x556da500d050 2024-09-19T06:59:58.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.549+0000 7f709f7fe640 10 client.19152 handle_command_reply: tid=0 2024-09-19T06:59:58.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.549+0000 7f70c8ca6740 1 client.19152 shutdown 2024-09-19T06:59:58.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.549+0000 7f70c8ca6740 1 -- 172.21.15.103:0/3417996441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4fff580 msgr2=0x556da4ffb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:58.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.549+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/3417996441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4fff580 0x556da4ffb150 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f70b8010a50 tx=0x7f70b8010f10 comp rx=0 tx=0).stop 2024-09-19T06:59:58.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.549+0000 7f70c8ca6740 1 -- 172.21.15.103:0/3417996441 shutdown_connections 2024-09-19T06:59:58.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.549+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/3417996441 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556da500d050 0x556da500f460 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.549+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/3417996441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4fff580 0x556da4ffb150 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.549+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/3417996441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4ff3b40 0x556da4ffabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.549+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/3417996441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4ff25d0 0x556da4ffa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.549+0000 7f709f7fe640 0 client.19152 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:58.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.549+0000 7f70c8ca6740 1 -- 172.21.15.103:0/3417996441 >> 172.21.15.103:0/3417996441 conn(0x556da4fbb310 msgr2=0x556da4ff4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:58.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.549+0000 7f70c8ca6740 1 -- 172.21.15.103:0/3417996441 shutdown_connections 2024-09-19T06:59:58.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.549+0000 7f70c8ca6740 1 -- 172.21.15.103:0/3417996441 wait complete. 2024-09-19T06:59:58.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.549+0000 7f70c8ca6740 20 client.19152 trim_cache size 0 max 16384 2024-09-19T06:59:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70c8ca6740 1 Processor -- start 2024-09-19T06:59:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70c8ca6740 1 -- start start 2024-09-19T06:59:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70c8ca6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4ff25d0 0x556da4ffaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70c8ca6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff3b40 0x556da5014d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70c8ca6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4fff580 0x556da5015270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70c8ca6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01ab740 con 0x556da4ff3b40 2024-09-19T06:59:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70c8ca6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01ad280 con 0x556da4ff25d0 2024-09-19T06:59:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70c8ca6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01abe10 con 0x556da4fff580 2024-09-19T06:59:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70bf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff3b40 0x556da5014d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70bf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff3b40 0x556da5014d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33598/0 (socket says 172.21.15.103:33598) 2024-09-19T06:59:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70bf7fe640 1 -- 172.21.15.103:0/39598107 learned_addr learned my addr 172.21.15.103:0/39598107 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70bffff640 1 --2- 172.21.15.103:0/39598107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4ff25d0 0x556da4ffaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70c4e6a640 1 --2- 172.21.15.103:0/39598107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4fff580 0x556da5015270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70bf7fe640 1 -- 172.21.15.103:0/39598107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4fff580 msgr2=0x556da5015270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:58.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70bf7fe640 1 --2- 172.21.15.103:0/39598107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4fff580 0x556da5015270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70bf7fe640 1 -- 172.21.15.103:0/39598107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4ff25d0 msgr2=0x556da4ffaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:58.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70bf7fe640 1 --2- 172.21.15.103:0/39598107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4ff25d0 0x556da4ffaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70bf7fe640 1 -- 172.21.15.103:0/39598107 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556da4ff5120 con 0x556da4ff3b40 2024-09-19T06:59:58.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.562+0000 7f70bf7fe640 1 --2- 172.21.15.103:0/39598107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff3b40 0x556da5014d30 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f70b4019000 tx=0x7f70b4012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2948b356a2c9c826 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:58.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.563+0000 7f70a5ffb640 1 -- 172.21.15.103:0/39598107 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4031030 con 0x556da4ff3b40 2024-09-19T06:59:58.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.563+0000 7f70a5ffb640 1 -- 172.21.15.103:0/39598107 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b401de50 con 0x556da4ff3b40 2024-09-19T06:59:58.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.563+0000 7f70a5ffb640 1 -- 172.21.15.103:0/39598107 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4003b40 con 0x556da4ff3b40 2024-09-19T06:59:58.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.563+0000 7f70c8ca6740 1 -- 172.21.15.103:0/39598107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff3b40 msgr2=0x556da5014d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:58.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.563+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/39598107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff3b40 0x556da5014d30 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f70b4019000 tx=0x7f70b4012f90 comp rx=0 tx=0).stop 2024-09-19T06:59:58.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.563+0000 7f70c8ca6740 1 -- 172.21.15.103:0/39598107 shutdown_connections 2024-09-19T06:59:58.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.563+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/39598107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4fff580 0x556da5015270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.563+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/39598107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff3b40 0x556da5014d30 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.563+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/39598107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4ff25d0 0x556da4ffaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.563+0000 7f70c8ca6740 1 -- 172.21.15.103:0/39598107 >> 172.21.15.103:0/39598107 conn(0x556da4fbb310 msgr2=0x556da4ffe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:58.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.563+0000 7f70c8ca6740 1 -- 172.21.15.103:0/39598107 shutdown_connections 2024-09-19T06:59:58.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.563+0000 7f70c8ca6740 1 -- 172.21.15.103:0/39598107 wait complete. 2024-09-19T06:59:58.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.564+0000 7f70c8ca6740 1 -- start start 2024-09-19T06:59:58.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.564+0000 7f70c8ca6740 1 Processor -- start 2024-09-19T06:59:58.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.564+0000 7f70c8ca6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff25d0 0x556da50045b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.564+0000 7f70c8ca6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4ff3b40 0x556da5004af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.564+0000 7f70c8ca6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4fff580 0x556da5076d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.564+0000 7f70c8ca6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01acbb0 con 0x556da4ff25d0 2024-09-19T06:59:58.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.564+0000 7f70c8ca6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01ac4e0 con 0x556da4fff580 2024-09-19T06:59:58.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.564+0000 7f70c8ca6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01a9740 con 0x556da4ff3b40 2024-09-19T06:59:58.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.564+0000 7f70c8ca6740 10 client.0 _ll_register_callbacks cb 0x7f70b800a980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:59:58.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.564+0000 7f70bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff25d0 0x556da50045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.564+0000 7f70bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff25d0 0x556da50045b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33606/0 (socket says 172.21.15.103:33606) 2024-09-19T06:59:58.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.564+0000 7f70bffff640 1 -- 172.21.15.103:0/1808170884 learned_addr learned my addr 172.21.15.103:0/1808170884 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T06:59:58.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.564+0000 7f70c4e6a640 1 --2- 172.21.15.103:0/1808170884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4fff580 0x556da5076d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.564+0000 7f70bf7fe640 1 --2- 172.21.15.103:0/1808170884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4ff3b40 0x556da5004af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.565+0000 7f70bffff640 1 -- 172.21.15.103:0/1808170884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4ff3b40 msgr2=0x556da5004af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:58.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.565+0000 7f70bffff640 1 --2- 172.21.15.103:0/1808170884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4ff3b40 0x556da5004af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.565+0000 7f70bffff640 1 -- 172.21.15.103:0/1808170884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4fff580 msgr2=0x556da5076d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:59:58.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.565+0000 7f70bffff640 1 --2- 172.21.15.103:0/1808170884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4fff580 0x556da5076d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.565+0000 7f70bffff640 1 -- 172.21.15.103:0/1808170884 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556da4ffb4c0 con 0x556da4ff25d0 2024-09-19T06:59:58.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.565+0000 7f70bffff640 1 --2- 172.21.15.103:0/1808170884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff25d0 0x556da50045b0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f70b00b8cd0 tx=0x7f70b0031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d95cb7dd4700513 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:59:58.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.565+0000 7f709f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T06:59:58.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.565+0000 7f709f7fe640 1 -- 172.21.15.103:0/1808170884 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0049040 con 0x556da4ff25d0 2024-09-19T06:59:58.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.565+0000 7f709f7fe640 1 -- 172.21.15.103:0/1808170884 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f70b00edf60 con 0x556da4ff25d0 2024-09-19T06:59:58.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.565+0000 7f70c8ca6740 1 -- 172.21.15.103:0/1808170884 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b401de50 con 0x556da4ff25d0 2024-09-19T06:59:58.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.565+0000 7f709f7fe640 1 -- 172.21.15.103:0/1808170884 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f70b00c0c20 con 0x556da4ff25d0 2024-09-19T06:59:58.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.565+0000 7f70c8ca6740 10 client.19158 fetch_fsmap learned FSMap version 27 2024-09-19T06:59:58.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.565+0000 7f70c8ca6740 1 -- 172.21.15.103:0/1808170884 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556da50782d0 con 0x556da4ff25d0 2024-09-19T06:59:58.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.565+0000 7f709f7fe640 1 -- 172.21.15.103:0/1808170884 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b401de50 con 0x556da4ff25d0 2024-09-19T06:59:58.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.565+0000 7f709f7fe640 1 -- 172.21.15.103:0/1808170884 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f70b00c5050 con 0x556da4ff25d0 2024-09-19T06:59:58.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.566+0000 7f70c8ca6740 10 client.19158 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:59:58.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.566+0000 7f70c8ca6740 10 client.19158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:59:58.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.566+0000 7f70c8ca6740 20 client.19158 populate_metadata read hostname 'smithi103' 2024-09-19T06:59:58.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.566+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/1808170884 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556da5078a40 0x556da507ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:59:58.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.566+0000 7f70c8ca6740 4 client.19158 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:59:58.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.566+0000 7f70c8ca6740 1 -- 172.21.15.103:0/1808170884 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556da4f5fd70 con 0x556da5078a40 2024-09-19T06:59:58.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.566+0000 7f70bf7fe640 1 --2- 172.21.15.103:0/1808170884 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556da5078a40 0x556da507ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:59:58.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.566+0000 7f709f7fe640 10 client.19158 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:58.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.566+0000 7f70bf7fe640 1 --2- 172.21.15.103:0/1808170884 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556da5078a40 0x556da507ae30 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96dcc73595c45c41 server_cookie=32aedc2921355623 in_seq=0 out_seq=0 2024-09-19T06:59:58.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.566+0000 7f709f7fe640 1 -- 172.21.15.103:0/1808170884 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556da4f5fd70 con 0x556da5078a40 2024-09-19T06:59:58.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.566+0000 7f709f7fe640 10 client.19158 handle_command_reply: tid=0 2024-09-19T06:59:58.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.566+0000 7f70c8ca6740 1 client.19158 shutdown 2024-09-19T06:59:58.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.567+0000 7f70c8ca6740 1 -- 172.21.15.103:0/1808170884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff25d0 msgr2=0x556da50045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:58.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.567+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/1808170884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff25d0 0x556da50045b0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f70b00b8cd0 tx=0x7f70b0031ea0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.567+0000 7f70c8ca6740 1 -- 172.21.15.103:0/1808170884 shutdown_connections 2024-09-19T06:59:58.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.567+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/1808170884 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556da5078a40 0x556da507ae30 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.567+0000 7f709f7fe640 0 client.19158 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T06:59:58.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.567+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/1808170884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556da4fff580 0x556da5076d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.567+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/1808170884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556da4ff3b40 0x556da5004af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.567+0000 7f70c8ca6740 1 --2- 172.21.15.103:0/1808170884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556da4ff25d0 0x556da50045b0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.567+0000 7f70c8ca6740 1 -- 172.21.15.103:0/1808170884 >> 172.21.15.103:0/1808170884 conn(0x556da4fbb310 msgr2=0x556da4ffdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:58.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.567+0000 7f70c8ca6740 1 -- 172.21.15.103:0/1808170884 shutdown_connections 2024-09-19T06:59:58.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.567+0000 7f70c8ca6740 1 -- 172.21.15.103:0/1808170884 wait complete. 2024-09-19T06:59:58.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.567+0000 7f70c8ca6740 20 client.19158 trim_cache size 0 max 16384 2024-09-19T06:59:58.573 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T06:59:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.567+0000 7f70c68cd640 1 -- 172.21.15.103:0/665417956 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70a80a33b0 msgr2=0x7f70a80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.567+0000 7f70c68cd640 1 --2- 172.21.15.103:0/665417956 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70a80a33b0 0x7f70a80a5870 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f70c00ffee0 tx=0x7f70b400f730 comp rx=0 tx=0).stop 2024-09-19T06:59:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.567+0000 7f70c68cd640 1 -- 172.21.15.103:0/665417956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0100e20 msgr2=0x7f70c0197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:59:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.567+0000 7f70c68cd640 1 --2- 172.21.15.103:0/665417956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0100e20 0x7f70c0197450 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f70b0006010 tx=0x7f70b0003c10 comp rx=0 tx=0).stop 2024-09-19T06:59:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.568+0000 7f70c68cd640 1 -- 172.21.15.103:0/665417956 shutdown_connections 2024-09-19T06:59:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.568+0000 7f70c68cd640 1 --2- 172.21.15.103:0/665417956 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70a80a33b0 0x7f70a80a5870 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.568+0000 7f70c68cd640 1 --2- 172.21.15.103:0/665417956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c0102ef0 0x7f70c019bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.568+0000 7f70c68cd640 1 --2- 172.21.15.103:0/665417956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70c0102570 0x7f70c0197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.568+0000 7f70c68cd640 1 --2- 172.21.15.103:0/665417956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c0100e20 0x7f70c0197450 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:59:58.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.568+0000 7f70c68cd640 1 -- 172.21.15.103:0/665417956 >> 172.21.15.103:0/665417956 conn(0x7f70c00fc7d0 msgr2=0x7f70c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:59:58.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.568+0000 7f70c68cd640 1 -- 172.21.15.103:0/665417956 shutdown_connections 2024-09-19T06:59:58.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T06:59:58.568+0000 7f70c68cd640 1 -- 172.21.15.103:0/665417956 wait complete. 2024-09-19T07:00:00.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 06:59:59 smithi134 ceph-mon[35356]: pgmap v333: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:00:00.243 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:00:00.244 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:00:00.268 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 06:59:59 smithi103 ceph-mon[31158]: pgmap v333: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:00:00.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 06:59:59 smithi174 ceph-mon[35637]: pgmap v333: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:00:00.497 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:00:01.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.008+0000 7fad10615640 1 -- 172.21.15.103:0/529130691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad08100e20 msgr2=0x7fad08101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:01.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.008+0000 7fad10615640 1 --2- 172.21.15.103:0/529130691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad08100e20 0x7fad08101200 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fad04009900 tx=0x7fad0402a480 comp rx=0 tx=0).stop 2024-09-19T07:00:01.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.008+0000 7fad10615640 1 -- 172.21.15.103:0/529130691 shutdown_connections 2024-09-19T07:00:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.008+0000 7fad10615640 1 --2- 172.21.15.103:0/529130691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad08102ef0 0x7fad0810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.008+0000 7fad10615640 1 --2- 172.21.15.103:0/529130691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad08102570 0x7fad081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.008+0000 7fad10615640 1 --2- 172.21.15.103:0/529130691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad08100e20 0x7fad08101200 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.008+0000 7fad10615640 1 -- 172.21.15.103:0/529130691 >> 172.21.15.103:0/529130691 conn(0x7fad080fc7d0 msgr2=0x7fad080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.008+0000 7fad10615640 1 -- 172.21.15.103:0/529130691 shutdown_connections 2024-09-19T07:00:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.008+0000 7fad10615640 1 -- 172.21.15.103:0/529130691 wait complete. 2024-09-19T07:00:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad10615640 1 Processor -- start 2024-09-19T07:00:01.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad10615640 1 -- start start 2024-09-19T07:00:01.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad10615640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad08100e20 0x7fad080fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad10615640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad08102570 0x7fad080fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad10615640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad08102ef0 0x7fad080fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad10615640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad08110f80 con 0x7fad08100e20 2024-09-19T07:00:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad10615640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad08110e00 con 0x7fad08102ef0 2024-09-19T07:00:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad10615640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fad08111100 con 0x7fad08102570 2024-09-19T07:00:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad0e3b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad08100e20 0x7fad080fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad0e3b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad08100e20 0x7fad080fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33634/0 (socket says 172.21.15.103:33634) 2024-09-19T07:00:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad0e3b1640 1 -- 172.21.15.103:0/930861976 learned_addr learned my addr 172.21.15.103:0/930861976 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad0dbb0640 1 --2- 172.21.15.103:0/930861976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad08102570 0x7fad080fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad0ebb2640 1 --2- 172.21.15.103:0/930861976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad08102ef0 0x7fad080fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad0e3b1640 1 -- 172.21.15.103:0/930861976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad08102570 msgr2=0x7fad080fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad0e3b1640 1 --2- 172.21.15.103:0/930861976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad08102570 0x7fad080fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad0e3b1640 1 -- 172.21.15.103:0/930861976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad08102ef0 msgr2=0x7fad080fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad0e3b1640 1 --2- 172.21.15.103:0/930861976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad08102ef0 0x7fad080fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.009+0000 7fad0e3b1640 1 -- 172.21.15.103:0/930861976 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad081a6a30 con 0x7fad08100e20 2024-09-19T07:00:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.010+0000 7fad0e3b1640 1 --2- 172.21.15.103:0/930861976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad08100e20 0x7fad080fcc80 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fad04006010 tx=0x7fad04003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.010+0000 7facff7fe640 1 -- 172.21.15.103:0/930861976 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0403f070 con 0x7fad08100e20 2024-09-19T07:00:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.010+0000 7facff7fe640 1 -- 172.21.15.103:0/930861976 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad04031e30 con 0x7fad08100e20 2024-09-19T07:00:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.010+0000 7fad10615640 1 -- 172.21.15.103:0/930861976 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad081a6cc0 con 0x7fad08100e20 2024-09-19T07:00:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.010+0000 7facff7fe640 1 -- 172.21.15.103:0/930861976 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04035030 con 0x7fad08100e20 2024-09-19T07:00:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.010+0000 7fad10615640 1 -- 172.21.15.103:0/930861976 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad081a71a0 con 0x7fad08100e20 2024-09-19T07:00:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.012+0000 7facff7fe640 1 -- 172.21.15.103:0/930861976 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fad04042070 con 0x7fad08100e20 2024-09-19T07:00:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.013+0000 7fad129ee740 1 Processor -- start 2024-09-19T07:00:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.013+0000 7fad129ee740 1 -- start start 2024-09-19T07:00:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.013+0000 7fad129ee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae40b60 0x555baae40f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.013+0000 7facff7fe640 1 --2- 172.21.15.103:0/930861976 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7facf40a3430 0x7facf40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.013+0000 7fad129ee740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae41480 0x555baae4b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.013+0000 7fad129ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae4bd40 0x555baae4e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.013+0000 7fad129ee740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555baada6280 con 0x555baae41480 2024-09-19T07:00:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.013+0000 7fad129ee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555baad4db10 con 0x555baae40b60 2024-09-19T07:00:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.013+0000 7fad129ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555baada2240 con 0x555baae4bd40 2024-09-19T07:00:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.013+0000 7fad0ebb2640 1 --2- 172.21.15.103:0/930861976 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7facf40a3430 0x7facf40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.013+0000 7fad0dbb0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae41480 0x555baae4b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.013+0000 7fad0dbb0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae41480 0x555baae4b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33638/0 (socket says 172.21.15.103:33638) 2024-09-19T07:00:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.013+0000 7fad0dbb0640 1 -- 172.21.15.103:0/1763473842 learned_addr learned my addr 172.21.15.103:0/1763473842 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.013+0000 7facff7fe640 1 -- 172.21.15.103:0/930861976 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7fad040f05c0 con 0x7fad08100e20 2024-09-19T07:00:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.013+0000 7fad0e3b1640 1 --2- 172.21.15.103:0/1763473842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae40b60 0x555baae40f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.014+0000 7fad0ebb2640 1 --2- 172.21.15.103:0/1763473842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae4bd40 0x555baae4e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.014+0000 7fad0dbb0640 1 -- 172.21.15.103:0/1763473842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae4bd40 msgr2=0x555baae4e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.014+0000 7fad0dbb0640 1 --2- 172.21.15.103:0/1763473842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae4bd40 0x555baae4e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.014+0000 7fad0dbb0640 1 -- 172.21.15.103:0/1763473842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae40b60 msgr2=0x555baae40f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.014+0000 7fad0dbb0640 1 --2- 172.21.15.103:0/1763473842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae40b60 0x555baae40f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.014+0000 7fad0dbb0640 1 -- 172.21.15.103:0/1763473842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555baad5a1f0 con 0x555baae41480 2024-09-19T07:00:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.014+0000 7fad0ebb2640 1 --2- 172.21.15.103:0/1763473842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae4bd40 0x555baae4e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:01.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.014+0000 7fad0ebb2640 1 --2- 172.21.15.103:0/930861976 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7facf40a3430 0x7facf40a58f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fad0000a780 tx=0x7fad0000a8d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:01.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.014+0000 7fad0dbb0640 1 --2- 172.21.15.103:0/1763473842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae41480 0x555baae4b800 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7facf800abe0 tx=0x7facf800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ea6e77b84db9f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:01.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.014+0000 7facfcff9640 1 -- 172.21.15.103:0/1763473842 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf801d070 con 0x555baae41480 2024-09-19T07:00:01.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.014+0000 7facfcff9640 1 -- 172.21.15.103:0/1763473842 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf8018b20 con 0x555baae41480 2024-09-19T07:00:01.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.014+0000 7facfcff9640 1 -- 172.21.15.103:0/1763473842 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8010b60 con 0x555baae41480 2024-09-19T07:00:01.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.015+0000 7fad129ee740 1 -- 172.21.15.103:0/1763473842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae41480 msgr2=0x555baae4b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:01.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.015+0000 7fad129ee740 1 --2- 172.21.15.103:0/1763473842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae41480 0x555baae4b800 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7facf800abe0 tx=0x7facf800f790 comp rx=0 tx=0).stop 2024-09-19T07:00:01.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.015+0000 7fad129ee740 1 -- 172.21.15.103:0/1763473842 shutdown_connections 2024-09-19T07:00:01.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.015+0000 7fad129ee740 1 --2- 172.21.15.103:0/1763473842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae4bd40 0x555baae4e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.015+0000 7fad129ee740 1 --2- 172.21.15.103:0/1763473842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae41480 0x555baae4b800 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.015+0000 7fad129ee740 1 --2- 172.21.15.103:0/1763473842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae40b60 0x555baae40f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.015+0000 7fad129ee740 1 -- 172.21.15.103:0/1763473842 >> 172.21.15.103:0/1763473842 conn(0x555baae3d0f0 msgr2=0x555baae3d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:01.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.015+0000 7fad129ee740 1 -- 172.21.15.103:0/1763473842 shutdown_connections 2024-09-19T07:00:01.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.015+0000 7fad129ee740 1 -- 172.21.15.103:0/1763473842 wait complete. 2024-09-19T07:00:01.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.015+0000 7fad129ee740 1 -- start start 2024-09-19T07:00:01.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.015+0000 7fad129ee740 1 Processor -- start 2024-09-19T07:00:01.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad129ee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae40b60 0x555baae48ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad129ee740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae41480 0x555baae490e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad129ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae4bd40 0x555baae49620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad129ee740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555baae4e910 con 0x555baae41480 2024-09-19T07:00:01.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad129ee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555baad60f10 con 0x555baae40b60 2024-09-19T07:00:01.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad129ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555baae4ea90 con 0x555baae4bd40 2024-09-19T07:00:01.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad129ee740 10 client.0 _ll_register_callbacks cb 0x555baad29a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:01.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad0dbb0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae41480 0x555baae490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad0dbb0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae41480 0x555baae490e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33640/0 (socket says 172.21.15.103:33640) 2024-09-19T07:00:01.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad0dbb0640 1 -- 172.21.15.103:0/2122677235 learned_addr learned my addr 172.21.15.103:0/2122677235 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:01.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad0ebb2640 1 --2- 172.21.15.103:0/2122677235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae4bd40 0x555baae49620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad0e3b1640 1 --2- 172.21.15.103:0/2122677235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae40b60 0x555baae48ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad0dbb0640 1 -- 172.21.15.103:0/2122677235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae4bd40 msgr2=0x555baae49620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:01.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad0dbb0640 1 --2- 172.21.15.103:0/2122677235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae4bd40 0x555baae49620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad0dbb0640 1 -- 172.21.15.103:0/2122677235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae40b60 msgr2=0x555baae48ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:01.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad0dbb0640 1 --2- 172.21.15.103:0/2122677235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae40b60 0x555baae48ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad0dbb0640 1 -- 172.21.15.103:0/2122677235 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555baab176a0 con 0x555baae41480 2024-09-19T07:00:01.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad0e3b1640 1 --2- 172.21.15.103:0/2122677235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae40b60 0x555baae48ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:01.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7fad0dbb0640 1 --2- 172.21.15.103:0/2122677235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae41480 0x555baae490e0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7facf800a5b0 tx=0x7facf8010b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65fa532a932a899 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:01.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.016+0000 7facf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:01.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.017+0000 7facf1ffb640 1 -- 172.21.15.103:0/2122677235 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf801d050 con 0x555baae41480 2024-09-19T07:00:01.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.017+0000 7facf1ffb640 1 -- 172.21.15.103:0/2122677235 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7facf8005230 con 0x555baae41480 2024-09-19T07:00:01.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.017+0000 7fad129ee740 1 -- 172.21.15.103:0/2122677235 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf8018b20 con 0x555baae41480 2024-09-19T07:00:01.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.017+0000 7facf1ffb640 1 -- 172.21.15.103:0/2122677235 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7facf8031050 con 0x555baae41480 2024-09-19T07:00:01.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.017+0000 7fad129ee740 10 client.19170 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:01.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.017+0000 7fad129ee740 1 -- 172.21.15.103:0/2122677235 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555baae5a440 con 0x555baae41480 2024-09-19T07:00:01.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.017+0000 7facf1ffb640 1 -- 172.21.15.103:0/2122677235 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf8018b20 con 0x555baae41480 2024-09-19T07:00:01.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.017+0000 7facf1ffb640 1 -- 172.21.15.103:0/2122677235 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7facf802c030 con 0x555baae41480 2024-09-19T07:00:01.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.017+0000 7fad129ee740 10 client.19170 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:01.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.017+0000 7fad129ee740 10 client.19170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:01.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.017+0000 7fad129ee740 20 client.19170 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:01.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.017+0000 7fad129ee740 1 --2- 172.21.15.103:0/2122677235 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555baae5b900 0x555baae5dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.017+0000 7fad129ee740 4 client.19170 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:00:01.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.017+0000 7fad129ee740 1 -- 172.21.15.103:0/2122677235 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555baadb37f0 con 0x555baae5b900 2024-09-19T07:00:01.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.018+0000 7fad0e3b1640 1 --2- 172.21.15.103:0/2122677235 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555baae5b900 0x555baae5dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.018+0000 7fad0e3b1640 1 --2- 172.21.15.103:0/2122677235 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555baae5b900 0x555baae5dcf0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e4211b44e2a456e server_cookie=3a76b727a60b28d5 in_seq=0 out_seq=0 2024-09-19T07:00:01.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.018+0000 7facf1ffb640 10 client.19170 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:01.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.019+0000 7facf1ffb640 1 -- 172.21.15.103:0/2122677235 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555baadb37f0 con 0x555baae5b900 2024-09-19T07:00:01.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.019+0000 7facf1ffb640 10 client.19170 handle_command_reply: tid=0 2024-09-19T07:00:01.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.019+0000 7fad129ee740 1 client.19170 shutdown 2024-09-19T07:00:01.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.019+0000 7fad129ee740 1 -- 172.21.15.103:0/2122677235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae41480 msgr2=0x555baae490e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:01.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.019+0000 7fad129ee740 1 --2- 172.21.15.103:0/2122677235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae41480 0x555baae490e0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7facf800a5b0 tx=0x7facf8010b10 comp rx=0 tx=0).stop 2024-09-19T07:00:01.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.019+0000 7fad129ee740 1 -- 172.21.15.103:0/2122677235 shutdown_connections 2024-09-19T07:00:01.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.019+0000 7fad129ee740 1 --2- 172.21.15.103:0/2122677235 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555baae5b900 0x555baae5dcf0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.019+0000 7fad129ee740 1 --2- 172.21.15.103:0/2122677235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae4bd40 0x555baae49620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.019+0000 7fad129ee740 1 --2- 172.21.15.103:0/2122677235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae41480 0x555baae490e0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.019+0000 7fad129ee740 1 --2- 172.21.15.103:0/2122677235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae40b60 0x555baae48ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.019+0000 7facf1ffb640 0 client.19170 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:01.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.019+0000 7fad129ee740 1 -- 172.21.15.103:0/2122677235 >> 172.21.15.103:0/2122677235 conn(0x555baae3d0f0 msgr2=0x555baae42e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:01.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.019+0000 7fad129ee740 1 -- 172.21.15.103:0/2122677235 shutdown_connections 2024-09-19T07:00:01.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.019+0000 7fad129ee740 1 -- 172.21.15.103:0/2122677235 wait complete. 2024-09-19T07:00:01.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.019+0000 7fad129ee740 20 client.19170 trim_cache size 0 max 16384 2024-09-19T07:00:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.036+0000 7fad129ee740 1 Processor -- start 2024-09-19T07:00:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.036+0000 7fad129ee740 1 -- start start 2024-09-19T07:00:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.036+0000 7fad129ee740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae40b60 0x555baae4aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.036+0000 7fad129ee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae41480 0x555baae632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.036+0000 7fad129ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae4bd40 0x555baae63830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.036+0000 7fad129ee740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad081afb10 con 0x555baae40b60 2024-09-19T07:00:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.036+0000 7fad129ee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad081b1650 con 0x555baae41480 2024-09-19T07:00:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.036+0000 7fad129ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fad081b01e0 con 0x555baae4bd40 2024-09-19T07:00:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.036+0000 7fad0e3b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae40b60 0x555baae4aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.036+0000 7fad0e3b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae40b60 0x555baae4aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33654/0 (socket says 172.21.15.103:33654) 2024-09-19T07:00:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.036+0000 7fad0e3b1640 1 -- 172.21.15.103:0/1051757501 learned_addr learned my addr 172.21.15.103:0/1051757501 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.037+0000 7fad0ebb2640 1 --2- 172.21.15.103:0/1051757501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae4bd40 0x555baae63830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.037+0000 7fad0dbb0640 1 --2- 172.21.15.103:0/1051757501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae41480 0x555baae632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.037+0000 7fad0e3b1640 1 -- 172.21.15.103:0/1051757501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae4bd40 msgr2=0x555baae63830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.037+0000 7fad0e3b1640 1 --2- 172.21.15.103:0/1051757501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae4bd40 0x555baae63830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.037+0000 7fad0e3b1640 1 -- 172.21.15.103:0/1051757501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae41480 msgr2=0x555baae632f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.037+0000 7fad0e3b1640 1 --2- 172.21.15.103:0/1051757501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae41480 0x555baae632f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.037+0000 7fad0e3b1640 1 -- 172.21.15.103:0/1051757501 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555baae4ea70 con 0x555baae40b60 2024-09-19T07:00:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.037+0000 7fad0e3b1640 1 --2- 172.21.15.103:0/1051757501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae40b60 0x555baae4aa00 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fad040edcf0 tx=0x7fad04034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a2e3b969a0827f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.037+0000 7facf3fff640 1 -- 172.21.15.103:0/1051757501 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04049040 con 0x555baae40b60 2024-09-19T07:00:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.037+0000 7facf3fff640 1 -- 172.21.15.103:0/1051757501 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad040bc8d0 con 0x555baae40b60 2024-09-19T07:00:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.037+0000 7facf3fff640 1 -- 172.21.15.103:0/1051757501 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad040bac60 con 0x555baae40b60 2024-09-19T07:00:01.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 -- 172.21.15.103:0/1051757501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae40b60 msgr2=0x555baae4aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:01.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 --2- 172.21.15.103:0/1051757501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae40b60 0x555baae4aa00 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fad040edcf0 tx=0x7fad04034000 comp rx=0 tx=0).stop 2024-09-19T07:00:01.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 -- 172.21.15.103:0/1051757501 shutdown_connections 2024-09-19T07:00:01.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 --2- 172.21.15.103:0/1051757501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae4bd40 0x555baae63830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 --2- 172.21.15.103:0/1051757501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae41480 0x555baae632f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 --2- 172.21.15.103:0/1051757501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae40b60 0x555baae4aa00 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 -- 172.21.15.103:0/1051757501 >> 172.21.15.103:0/1051757501 conn(0x555baae3d0f0 msgr2=0x555baae4a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:01.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 -- 172.21.15.103:0/1051757501 shutdown_connections 2024-09-19T07:00:01.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 -- 172.21.15.103:0/1051757501 wait complete. 2024-09-19T07:00:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 -- start start 2024-09-19T07:00:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 Processor -- start 2024-09-19T07:00:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae40b60 0x555baae54640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae41480 0x555baae4f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae4bd40 0x555baae50100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad081b0f80 con 0x555baae40b60 2024-09-19T07:00:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad081b08b0 con 0x555baae4bd40 2024-09-19T07:00:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fad081adb10 con 0x555baae41480 2024-09-19T07:00:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad129ee740 10 client.0 _ll_register_callbacks cb 0x7facf801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad0e3b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae40b60 0x555baae54640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad0e3b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae40b60 0x555baae54640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33666/0 (socket says 172.21.15.103:33666) 2024-09-19T07:00:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad0e3b1640 1 -- 172.21.15.103:0/3515420205 learned_addr learned my addr 172.21.15.103:0/3515420205 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.038+0000 7fad0dbb0640 1 --2- 172.21.15.103:0/3515420205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae41480 0x555baae4f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.039+0000 7fad0ebb2640 1 --2- 172.21.15.103:0/3515420205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae4bd40 0x555baae50100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.039+0000 7fad0e3b1640 1 -- 172.21.15.103:0/3515420205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae41480 msgr2=0x555baae4f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.039+0000 7fad0e3b1640 1 --2- 172.21.15.103:0/3515420205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae41480 0x555baae4f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.039+0000 7fad0e3b1640 1 -- 172.21.15.103:0/3515420205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae4bd40 msgr2=0x555baae50100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.039+0000 7fad0e3b1640 1 --2- 172.21.15.103:0/3515420205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae4bd40 0x555baae50100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.039+0000 7fad0e3b1640 1 -- 172.21.15.103:0/3515420205 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555baae3cf20 con 0x555baae40b60 2024-09-19T07:00:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.039+0000 7fad0e3b1640 1 --2- 172.21.15.103:0/3515420205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae40b60 0x555baae54640 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fad040c4f60 tx=0x7fad040c5520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4f62626e4e9c81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.039+0000 7facf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.039+0000 7facf1ffb640 1 -- 172.21.15.103:0/3515420205 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04049040 con 0x555baae40b60 2024-09-19T07:00:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.039+0000 7facf1ffb640 1 -- 172.21.15.103:0/3515420205 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7fad0403b5f0 con 0x555baae40b60 2024-09-19T07:00:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.039+0000 7fad129ee740 1 -- 172.21.15.103:0/3515420205 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad040bc8d0 con 0x555baae40b60 2024-09-19T07:00:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.039+0000 7facf1ffb640 1 -- 172.21.15.103:0/3515420205 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fad040b8620 con 0x555baae40b60 2024-09-19T07:00:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.039+0000 7fad129ee740 10 client.19176 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.039+0000 7fad129ee740 1 -- 172.21.15.103:0/3515420205 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555baae419f0 con 0x555baae40b60 2024-09-19T07:00:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.039+0000 7facf1ffb640 1 -- 172.21.15.103:0/3515420205 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad040bc8d0 con 0x555baae40b60 2024-09-19T07:00:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.040+0000 7facf1ffb640 1 -- 172.21.15.103:0/3515420205 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fad040a5c60 con 0x555baae40b60 2024-09-19T07:00:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.040+0000 7fad129ee740 10 client.19176 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.040+0000 7fad129ee740 10 client.19176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.040+0000 7fad129ee740 20 client.19176 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.040+0000 7fad129ee740 1 --2- 172.21.15.103:0/3515420205 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555baaec6760 0x555baaec8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.040+0000 7fad129ee740 4 client.19176 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:00:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.040+0000 7fad129ee740 1 -- 172.21.15.103:0/3515420205 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555baadadd00 con 0x555baaec6760 2024-09-19T07:00:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.040+0000 7fad0dbb0640 1 --2- 172.21.15.103:0/3515420205 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555baaec6760 0x555baaec8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.040+0000 7facf1ffb640 10 client.19176 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.040+0000 7fad0dbb0640 1 --2- 172.21.15.103:0/3515420205 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555baaec6760 0x555baaec8b50 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0b04edacb539ebc server_cookie=8c2cbbc7e907fb24 in_seq=0 out_seq=0 2024-09-19T07:00:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.042+0000 7facf1ffb640 1 -- 172.21.15.103:0/3515420205 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555baadadd00 con 0x555baaec6760 2024-09-19T07:00:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.042+0000 7facf1ffb640 10 client.19176 handle_command_reply: tid=0 2024-09-19T07:00:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.042+0000 7fad129ee740 1 client.19176 shutdown 2024-09-19T07:00:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.042+0000 7fad129ee740 1 -- 172.21.15.103:0/3515420205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae40b60 msgr2=0x555baae54640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.042+0000 7fad129ee740 1 --2- 172.21.15.103:0/3515420205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae40b60 0x555baae54640 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fad040c4f60 tx=0x7fad040c5520 comp rx=0 tx=0).stop 2024-09-19T07:00:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.042+0000 7fad129ee740 1 -- 172.21.15.103:0/3515420205 shutdown_connections 2024-09-19T07:00:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.042+0000 7fad129ee740 1 --2- 172.21.15.103:0/3515420205 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x555baaec6760 0x555baaec8b50 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.042+0000 7fad129ee740 1 --2- 172.21.15.103:0/3515420205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555baae4bd40 0x555baae50100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.042+0000 7fad129ee740 1 --2- 172.21.15.103:0/3515420205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555baae41480 0x555baae4f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.042+0000 7fad129ee740 1 --2- 172.21.15.103:0/3515420205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555baae40b60 0x555baae54640 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.042+0000 7fad129ee740 1 -- 172.21.15.103:0/3515420205 >> 172.21.15.103:0/3515420205 conn(0x555baae3d0f0 msgr2=0x555baae3c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.042+0000 7facf1ffb640 0 client.19176 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.042+0000 7fad129ee740 1 -- 172.21.15.103:0/3515420205 shutdown_connections 2024-09-19T07:00:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.042+0000 7fad129ee740 1 -- 172.21.15.103:0/3515420205 wait complete. 2024-09-19T07:00:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.042+0000 7fad129ee740 20 client.19176 trim_cache size 0 max 16384 2024-09-19T07:00:01.048 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:00:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:00:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "5d50ac73-4b34-4cbf-a202-c9de5f7842e0", 2024-09-19T07:00:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:00:01.048 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:00:01.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.043+0000 7fad10615640 1 -- 172.21.15.103:0/930861976 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7facf40a3430 msgr2=0x7facf40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:01.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.043+0000 7fad10615640 1 --2- 172.21.15.103:0/930861976 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7facf40a3430 0x7facf40a58f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fad0000a780 tx=0x7fad0000a8d0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.043+0000 7fad10615640 1 -- 172.21.15.103:0/930861976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad08100e20 msgr2=0x7fad080fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:01.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.043+0000 7fad10615640 1 --2- 172.21.15.103:0/930861976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad08100e20 0x7fad080fcc80 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fad04006010 tx=0x7fad04003c80 comp rx=0 tx=0).stop 2024-09-19T07:00:01.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.043+0000 7fad10615640 1 -- 172.21.15.103:0/930861976 shutdown_connections 2024-09-19T07:00:01.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.043+0000 7fad10615640 1 --2- 172.21.15.103:0/930861976 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7facf40a3430 0x7facf40a58f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.043+0000 7fad10615640 1 --2- 172.21.15.103:0/930861976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad08102ef0 0x7fad080fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.043+0000 7fad10615640 1 --2- 172.21.15.103:0/930861976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad08102570 0x7fad080fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.043+0000 7fad10615640 1 --2- 172.21.15.103:0/930861976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad08100e20 0x7fad080fcc80 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:01.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.043+0000 7fad10615640 1 -- 172.21.15.103:0/930861976 >> 172.21.15.103:0/930861976 conn(0x7fad080fc7d0 msgr2=0x7fad0806a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:01.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.043+0000 7fad10615640 1 -- 172.21.15.103:0/930861976 shutdown_connections 2024-09-19T07:00:01.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:01.043+0000 7fad10615640 1 -- 172.21.15.103:0/930861976 wait complete. 2024-09-19T07:00:01.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:00 smithi134 ceph-mon[35356]: overall HEALTH_OK 2024-09-19T07:00:01.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:00 smithi174 ceph-mon[35637]: overall HEALTH_OK 2024-09-19T07:00:01.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:00 smithi103 ceph-mon[31158]: overall HEALTH_OK 2024-09-19T07:00:01.727 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:00:02.019 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:00:02.157 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:01 smithi103 ceph-mon[31158]: pgmap v334: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-09-19T07:00:02.158 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:01 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T07:00:02.158 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:01 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:00:02.158 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:01 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T07:00:02.158 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:01 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:00:02.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:01 smithi134 ceph-mon[35356]: pgmap v334: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-09-19T07:00:02.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:01 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T07:00:02.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:01 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:00:02.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:01 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T07:00:02.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:01 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:00:02.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:01 smithi174 ceph-mon[35637]: pgmap v334: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-09-19T07:00:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:01 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T07:00:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:01 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:00:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:01 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T07:00:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:01 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:00:02.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.498+0000 7f2ce3f93640 1 -- 172.21.15.103:0/887554512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cdc102d40 msgr2=0x7f2cdc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:02.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.498+0000 7f2ce3f93640 1 --2- 172.21.15.103:0/887554512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cdc102d40 0x7f2cdc103120 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009900 tx=0x7f2cd802a480 comp rx=0 tx=0).stop 2024-09-19T07:00:02.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.498+0000 7f2ce3f93640 1 -- 172.21.15.103:0/887554512 shutdown_connections 2024-09-19T07:00:02.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.498+0000 7f2ce3f93640 1 --2- 172.21.15.103:0/887554512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cdc10c170 0x7f2cdc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.498+0000 7f2ce3f93640 1 --2- 172.21.15.103:0/887554512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cdc103660 0x7f2cdc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.498+0000 7f2ce3f93640 1 --2- 172.21.15.103:0/887554512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cdc102d40 0x7f2cdc103120 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.498+0000 7f2ce3f93640 1 -- 172.21.15.103:0/887554512 >> 172.21.15.103:0/887554512 conn(0x7f2cdc0fc7a0 msgr2=0x7f2cdc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:02.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.499+0000 7f2ce3f93640 1 -- 172.21.15.103:0/887554512 shutdown_connections 2024-09-19T07:00:02.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.499+0000 7f2ce3f93640 1 -- 172.21.15.103:0/887554512 wait complete. 2024-09-19T07:00:02.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.499+0000 7f2ce3f93640 1 Processor -- start 2024-09-19T07:00:02.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.499+0000 7f2ce3f93640 1 -- start start 2024-09-19T07:00:02.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.499+0000 7f2ce3f93640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cdc102d40 0x7f2cdc1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.499+0000 7f2ce3f93640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cdc103660 0x7f2cdc1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.499+0000 7f2ce3f93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cdc10c170 0x7f2cdc19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.499+0000 7f2ce3f93640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc111fb0 con 0x7f2cdc10c170 2024-09-19T07:00:02.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.499+0000 7f2ce3f93640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc111e30 con 0x7f2cdc103660 2024-09-19T07:00:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.499+0000 7f2ce3f93640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc112130 con 0x7f2cdc102d40 2024-09-19T07:00:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.499+0000 7f2ce2530640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cdc10c170 0x7f2cdc19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.499+0000 7f2ce2530640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cdc10c170 0x7f2cdc19cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33678/0 (socket says 172.21.15.103:33678) 2024-09-19T07:00:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.500+0000 7f2ce2530640 1 -- 172.21.15.103:0/4215669274 learned_addr learned my addr 172.21.15.103:0/4215669274 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.500+0000 7f2ce1d2f640 1 --2- 172.21.15.103:0/4215669274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cdc102d40 0x7f2cdc1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.500+0000 7f2ce152e640 1 --2- 172.21.15.103:0/4215669274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cdc103660 0x7f2cdc1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.500+0000 7f2ce2530640 1 -- 172.21.15.103:0/4215669274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cdc102d40 msgr2=0x7f2cdc1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.500+0000 7f2ce2530640 1 --2- 172.21.15.103:0/4215669274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cdc102d40 0x7f2cdc1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.500+0000 7f2ce2530640 1 -- 172.21.15.103:0/4215669274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cdc103660 msgr2=0x7f2cdc1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.500+0000 7f2ce2530640 1 --2- 172.21.15.103:0/4215669274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cdc103660 0x7f2cdc1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.500+0000 7f2ce2530640 1 -- 172.21.15.103:0/4215669274 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cdc19d460 con 0x7f2cdc10c170 2024-09-19T07:00:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.500+0000 7f2ce1d2f640 1 --2- 172.21.15.103:0/4215669274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cdc102d40 0x7f2cdc1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:02.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.500+0000 7f2ce2530640 1 --2- 172.21.15.103:0/4215669274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cdc10c170 0x7f2cdc19cd80 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400eaa0 tx=0x7f2cd400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:02.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.500+0000 7f2cd2ffd640 1 -- 172.21.15.103:0/4215669274 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd400cdb0 con 0x7f2cdc10c170 2024-09-19T07:00:02.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.500+0000 7f2cd2ffd640 1 -- 172.21.15.103:0/4215669274 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd4005020 con 0x7f2cdc10c170 2024-09-19T07:00:02.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.500+0000 7f2ce3f93640 1 -- 172.21.15.103:0/4215669274 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cdc19d750 con 0x7f2cdc10c170 2024-09-19T07:00:02.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.500+0000 7f2cd2ffd640 1 -- 172.21.15.103:0/4215669274 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd40116b0 con 0x7f2cdc10c170 2024-09-19T07:00:02.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.500+0000 7f2ce3f93640 1 -- 172.21.15.103:0/4215669274 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cdc1a5030 con 0x7f2cdc10c170 2024-09-19T07:00:02.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.502+0000 7f2cd2ffd640 1 -- 172.21.15.103:0/4215669274 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f2cd4005560 con 0x7f2cdc10c170 2024-09-19T07:00:02.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.503+0000 7f2ce636c740 1 Processor -- start 2024-09-19T07:00:02.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.503+0000 7f2ce636c740 1 -- start start 2024-09-19T07:00:02.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.503+0000 7f2ce636c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf73f2380 0x561cf73f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.503+0000 7f2ce636c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2ca0 0x561cf7401550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.503+0000 7f2ce636c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf7401a90 0x561cf7403e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.503+0000 7f2ce636c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561cf73582e0 con 0x561cf73f2ca0 2024-09-19T07:00:02.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.503+0000 7f2ce636c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561cf72ffce0 con 0x561cf73f2380 2024-09-19T07:00:02.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.503+0000 7f2ce636c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561cf73542a0 con 0x561cf7401a90 2024-09-19T07:00:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.503+0000 7f2ce1d2f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2ca0 0x561cf7401550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.503+0000 7f2ce1d2f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2ca0 0x561cf7401550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33694/0 (socket says 172.21.15.103:33694) 2024-09-19T07:00:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.503+0000 7f2cd2ffd640 1 --2- 172.21.15.103:0/4215669274 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2cc80a33b0 0x7f2cc80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.503+0000 7f2ce1d2f640 1 -- 172.21.15.103:0/2711515076 learned_addr learned my addr 172.21.15.103:0/2711515076 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.503+0000 7f2ce152e640 1 --2- 172.21.15.103:0/2711515076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf73f2380 0x561cf73f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.503+0000 7f2ce2530640 1 --2- 172.21.15.103:0/4215669274 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2cc80a33b0 0x7f2cc80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.503+0000 7f2ce152e640 1 --2- 172.21.15.103:0/2711515076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf7401a90 0x561cf7403e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.504+0000 7f2ce1d2f640 1 -- 172.21.15.103:0/2711515076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf7401a90 msgr2=0x561cf7403e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.504+0000 7f2ce1d2f640 1 --2- 172.21.15.103:0/2711515076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf7401a90 0x561cf7403e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.504+0000 7f2cd2ffd640 1 -- 172.21.15.103:0/4215669274 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f2cd40d00d0 con 0x7f2cdc10c170 2024-09-19T07:00:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.504+0000 7f2ce1d2f640 1 -- 172.21.15.103:0/2711515076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf73f2380 msgr2=0x561cf73f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.504+0000 7f2ce1d2f640 1 --2- 172.21.15.103:0/2711515076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf73f2380 0x561cf73f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.504+0000 7f2ce1d2f640 1 -- 172.21.15.103:0/2711515076 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf72e4440 con 0x561cf73f2ca0 2024-09-19T07:00:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.504+0000 7f2ce152e640 1 --2- 172.21.15.103:0/2711515076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf7401a90 0x561cf7403e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.504+0000 7f2ce1d2f640 1 --2- 172.21.15.103:0/2711515076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2ca0 0x561cf7401550 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8033000 tx=0x7f2cd8032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed940fd370ef37c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.504+0000 7f2ce2530640 1 --2- 172.21.15.103:0/4215669274 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2cc80a33b0 0x7f2cc80a5870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4098790 tx=0x7f2cd4086c20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.504+0000 7f2cc7fff640 1 -- 172.21.15.103:0/2711515076 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd803f070 con 0x561cf73f2ca0 2024-09-19T07:00:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.504+0000 7f2cc7fff640 1 -- 172.21.15.103:0/2711515076 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd80317b0 con 0x561cf73f2ca0 2024-09-19T07:00:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.504+0000 7f2cc7fff640 1 -- 172.21.15.103:0/2711515076 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8035070 con 0x561cf73f2ca0 2024-09-19T07:00:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.505+0000 7f2ce636c740 1 -- 172.21.15.103:0/2711515076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2ca0 msgr2=0x561cf7401550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.505+0000 7f2ce636c740 1 --2- 172.21.15.103:0/2711515076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2ca0 0x561cf7401550 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8033000 tx=0x7f2cd8032d60 comp rx=0 tx=0).stop 2024-09-19T07:00:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.505+0000 7f2ce636c740 1 -- 172.21.15.103:0/2711515076 shutdown_connections 2024-09-19T07:00:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.505+0000 7f2ce636c740 1 --2- 172.21.15.103:0/2711515076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf7401a90 0x561cf7403e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.505+0000 7f2ce636c740 1 --2- 172.21.15.103:0/2711515076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2ca0 0x561cf7401550 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.505+0000 7f2ce636c740 1 --2- 172.21.15.103:0/2711515076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf73f2380 0x561cf73f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.505+0000 7f2ce636c740 1 -- 172.21.15.103:0/2711515076 >> 172.21.15.103:0/2711515076 conn(0x561cf73eea30 msgr2=0x561cf73eee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.505+0000 7f2ce636c740 1 -- 172.21.15.103:0/2711515076 shutdown_connections 2024-09-19T07:00:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.505+0000 7f2ce636c740 1 -- 172.21.15.103:0/2711515076 wait complete. 2024-09-19T07:00:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.505+0000 7f2ce636c740 1 -- start start 2024-09-19T07:00:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.505+0000 7f2ce636c740 1 Processor -- start 2024-09-19T07:00:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce636c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2380 0x561cf7401420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce636c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf73f2ca0 0x561cf73fe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce636c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf7401a90 0x561cf73fec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce636c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561cf7404660 con 0x561cf73f2380 2024-09-19T07:00:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce636c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561cf7313050 con 0x561cf73f2ca0 2024-09-19T07:00:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce636c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561cf74047e0 con 0x561cf7401a90 2024-09-19T07:00:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce636c740 10 client.0 _ll_register_callbacks cb 0x561cf7353160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:02.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce152e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2380 0x561cf7401420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:02.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce152e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2380 0x561cf7401420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33710/0 (socket says 172.21.15.103:33710) 2024-09-19T07:00:02.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce152e640 1 -- 172.21.15.103:0/3829450274 learned_addr learned my addr 172.21.15.103:0/3829450274 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:02.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce152e640 1 --2- 172.21.15.103:0/3829450274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf7401a90 0x561cf73fec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:02.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce1d2f640 1 --2- 172.21.15.103:0/3829450274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf73f2ca0 0x561cf73fe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:02.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce152e640 1 -- 172.21.15.103:0/3829450274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf7401a90 msgr2=0x561cf73fec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:02.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce152e640 1 --2- 172.21.15.103:0/3829450274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf7401a90 0x561cf73fec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce152e640 1 -- 172.21.15.103:0/3829450274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf73f2ca0 msgr2=0x561cf73fe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:02.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce152e640 1 --2- 172.21.15.103:0/3829450274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf73f2ca0 0x561cf73fe6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce152e640 1 -- 172.21.15.103:0/3829450274 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf70c93c0 con 0x561cf73f2380 2024-09-19T07:00:02.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce1d2f640 1 --2- 172.21.15.103:0/3829450274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf73f2ca0 0x561cf73fe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:02.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2ce152e640 1 --2- 172.21.15.103:0/3829450274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2380 0x561cf7401420 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc10d640 tx=0x7f2ccc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65753dcd52857492 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:02.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.506+0000 7f2cc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:02.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.507+0000 7f2cc57fa640 1 -- 172.21.15.103:0/3829450274 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc01f060 con 0x561cf73f2380 2024-09-19T07:00:02.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.507+0000 7f2ce636c740 1 -- 172.21.15.103:0/3829450274 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cd80317b0 con 0x561cf73f2380 2024-09-19T07:00:02.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.507+0000 7f2cc57fa640 1 -- 172.21.15.103:0/3829450274 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f2ccc014820 con 0x561cf73f2380 2024-09-19T07:00:02.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.507+0000 7f2cc57fa640 1 -- 172.21.15.103:0/3829450274 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ccc014a20 con 0x561cf73f2380 2024-09-19T07:00:02.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.507+0000 7f2ce636c740 10 client.19188 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:02.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.507+0000 7f2ce636c740 1 -- 172.21.15.103:0/3829450274 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561cf7410140 con 0x561cf73f2380 2024-09-19T07:00:02.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.507+0000 7f2cc57fa640 1 -- 172.21.15.103:0/3829450274 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd80317b0 con 0x561cf73f2380 2024-09-19T07:00:02.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.507+0000 7f2cc57fa640 1 -- 172.21.15.103:0/3829450274 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2ccc015040 con 0x561cf73f2380 2024-09-19T07:00:02.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.507+0000 7f2ce636c740 10 client.19188 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:02.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.507+0000 7f2ce636c740 10 client.19188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:02.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.507+0000 7f2ce636c740 20 client.19188 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:02.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.507+0000 7f2ce636c740 1 --2- 172.21.15.103:0/3829450274 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561cf7411600 0x561cf7413a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.507+0000 7f2ce636c740 4 client.19188 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:00:02.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.507+0000 7f2ce636c740 1 -- 172.21.15.103:0/3829450274 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cf7365850 con 0x561cf7411600 2024-09-19T07:00:02.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.507+0000 7f2ce1d2f640 1 --2- 172.21.15.103:0/3829450274 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561cf7411600 0x561cf7413a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:02.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.508+0000 7f2ce1d2f640 1 --2- 172.21.15.103:0/3829450274 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561cf7411600 0x561cf7413a60 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46cae9e8cc5ae5eb server_cookie=987b1b9292b457cc in_seq=0 out_seq=0 2024-09-19T07:00:02.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.508+0000 7f2cc57fa640 10 client.19188 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.509+0000 7f2cc57fa640 1 -- 172.21.15.103:0/3829450274 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cf7365850 con 0x561cf7411600 2024-09-19T07:00:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.509+0000 7f2cc57fa640 10 client.19188 handle_command_reply: tid=0 2024-09-19T07:00:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.509+0000 7f2ce636c740 1 client.19188 shutdown 2024-09-19T07:00:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.509+0000 7f2ce636c740 1 -- 172.21.15.103:0/3829450274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2380 msgr2=0x561cf7401420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.509+0000 7f2ce636c740 1 --2- 172.21.15.103:0/3829450274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2380 0x561cf7401420 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc10d640 tx=0x7f2ccc0046c0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.509+0000 7f2ce636c740 1 -- 172.21.15.103:0/3829450274 shutdown_connections 2024-09-19T07:00:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.509+0000 7f2ce636c740 1 --2- 172.21.15.103:0/3829450274 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561cf7411600 0x561cf7413a60 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.509+0000 7f2ce636c740 1 --2- 172.21.15.103:0/3829450274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf7401a90 0x561cf73fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.509+0000 7f2ce636c740 1 --2- 172.21.15.103:0/3829450274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf73f2ca0 0x561cf73fe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.509+0000 7f2ce636c740 1 --2- 172.21.15.103:0/3829450274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2380 0x561cf7401420 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.509+0000 7f2cc57fa640 0 client.19188 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.509+0000 7f2ce636c740 1 -- 172.21.15.103:0/3829450274 >> 172.21.15.103:0/3829450274 conn(0x561cf73eea30 msgr2=0x561cf73f4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.509+0000 7f2ce636c740 1 -- 172.21.15.103:0/3829450274 shutdown_connections 2024-09-19T07:00:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.509+0000 7f2ce636c740 1 -- 172.21.15.103:0/3829450274 wait complete. 2024-09-19T07:00:02.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.509+0000 7f2ce636c740 20 client.19188 trim_cache size 0 max 16384 2024-09-19T07:00:02.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce636c740 1 Processor -- start 2024-09-19T07:00:02.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce636c740 1 -- start start 2024-09-19T07:00:02.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce636c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf73f2380 0x561cf740d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce636c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf73f2ca0 0x561cf73fcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce636c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf7401a90 0x561cf7419330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce636c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc1ab640 con 0x561cf7401a90 2024-09-19T07:00:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce636c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc1ad180 con 0x561cf73f2ca0 2024-09-19T07:00:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce636c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc1abd10 con 0x561cf73f2380 2024-09-19T07:00:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce152e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf7401a90 0x561cf7419330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce152e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf7401a90 0x561cf7419330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33724/0 (socket says 172.21.15.103:33724) 2024-09-19T07:00:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce152e640 1 -- 172.21.15.103:0/1822810554 learned_addr learned my addr 172.21.15.103:0/1822810554 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce152e640 1 -- 172.21.15.103:0/1822810554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf73f2380 msgr2=0x561cf740d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce152e640 1 --2- 172.21.15.103:0/1822810554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf73f2380 0x561cf740d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce152e640 1 -- 172.21.15.103:0/1822810554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf73f2ca0 msgr2=0x561cf73fcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce152e640 1 --2- 172.21.15.103:0/1822810554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf73f2ca0 0x561cf73fcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce152e640 1 -- 172.21.15.103:0/1822810554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf73fd680 con 0x561cf7401a90 2024-09-19T07:00:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2ce152e640 1 --2- 172.21.15.103:0/1822810554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf7401a90 0x561cf7419330 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc004810 tx=0x7f2ccc00aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f152aa1d12e6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:02.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.526+0000 7f2cc77fe640 1 -- 172.21.15.103:0/1822810554 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc006ca0 con 0x561cf7401a90 2024-09-19T07:00:02.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.527+0000 7f2cc77fe640 1 -- 172.21.15.103:0/1822810554 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ccc0223d0 con 0x561cf7401a90 2024-09-19T07:00:02.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.527+0000 7f2cc77fe640 1 -- 172.21.15.103:0/1822810554 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc014e20 con 0x561cf7401a90 2024-09-19T07:00:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.527+0000 7f2ce636c740 1 -- 172.21.15.103:0/1822810554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf7401a90 msgr2=0x561cf7419330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.527+0000 7f2ce636c740 1 --2- 172.21.15.103:0/1822810554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf7401a90 0x561cf7419330 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc004810 tx=0x7f2ccc00aee0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.527+0000 7f2ce636c740 1 -- 172.21.15.103:0/1822810554 shutdown_connections 2024-09-19T07:00:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.527+0000 7f2ce636c740 1 --2- 172.21.15.103:0/1822810554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf7401a90 0x561cf7419330 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.527+0000 7f2ce636c740 1 --2- 172.21.15.103:0/1822810554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf73f2ca0 0x561cf73fcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.527+0000 7f2ce636c740 1 --2- 172.21.15.103:0/1822810554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf73f2380 0x561cf740d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.527+0000 7f2ce636c740 1 -- 172.21.15.103:0/1822810554 >> 172.21.15.103:0/1822810554 conn(0x561cf73eea30 msgr2=0x561cf73f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:02.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.527+0000 7f2ce636c740 1 -- 172.21.15.103:0/1822810554 shutdown_connections 2024-09-19T07:00:02.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.527+0000 7f2ce636c740 1 -- 172.21.15.103:0/1822810554 wait complete. 2024-09-19T07:00:02.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.528+0000 7f2ce636c740 1 -- start start 2024-09-19T07:00:02.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.528+0000 7f2ce636c740 1 Processor -- start 2024-09-19T07:00:02.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.528+0000 7f2ce636c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf73f2380 0x561cf740a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.528+0000 7f2ce636c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2ca0 0x561cf7405660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.528+0000 7f2ce636c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7401a90 0x561cf7405c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.528+0000 7f2ce636c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc1acab0 con 0x561cf73f2ca0 2024-09-19T07:00:02.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.528+0000 7f2ce636c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc1ac3e0 con 0x561cf7401a90 2024-09-19T07:00:02.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.528+0000 7f2ce636c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc1a9640 con 0x561cf73f2380 2024-09-19T07:00:02.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.528+0000 7f2ce636c740 10 client.0 _ll_register_callbacks cb 0x7f2ccc00a620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:02.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.528+0000 7f2ce1d2f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2ca0 0x561cf7405660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:02.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.528+0000 7f2ce1d2f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2ca0 0x561cf7405660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33730/0 (socket says 172.21.15.103:33730) 2024-09-19T07:00:02.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.528+0000 7f2ce1d2f640 1 -- 172.21.15.103:0/3053818557 learned_addr learned my addr 172.21.15.103:0/3053818557 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:02.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.528+0000 7f2ce152e640 1 --2- 172.21.15.103:0/3053818557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7401a90 0x561cf7405c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:02.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.528+0000 7f2ce152e640 1 --2- 172.21.15.103:0/3053818557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf73f2380 0x561cf740a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:02.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2ce1d2f640 1 -- 172.21.15.103:0/3053818557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf73f2380 msgr2=0x561cf740a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:02.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2ce1d2f640 1 --2- 172.21.15.103:0/3053818557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf73f2380 0x561cf740a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2ce1d2f640 1 -- 172.21.15.103:0/3053818557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7401a90 msgr2=0x561cf7405c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:02.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2ce1d2f640 1 --2- 172.21.15.103:0/3053818557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7401a90 0x561cf7405c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2ce1d2f640 1 -- 172.21.15.103:0/3053818557 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf73fefc0 con 0x561cf73f2ca0 2024-09-19T07:00:02.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2ce152e640 1 --2- 172.21.15.103:0/3053818557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf73f2380 0x561cf740a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:02.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2ce1d2f640 1 --2- 172.21.15.103:0/3053818557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2ca0 0x561cf7405660 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8006010 tx=0x7f2cd8005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9560d6dd3bf3d94c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:02.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2cc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:02.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2cc57fa640 1 -- 172.21.15.103:0/3053818557 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8050060 con 0x561cf73f2ca0 2024-09-19T07:00:02.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2cc57fa640 1 -- 172.21.15.103:0/3053818557 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f2cd80319c0 con 0x561cf73f2ca0 2024-09-19T07:00:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2ce636c740 1 -- 172.21.15.103:0/3053818557 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ccc0223d0 con 0x561cf73f2ca0 2024-09-19T07:00:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2cc57fa640 1 -- 172.21.15.103:0/3053818557 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2cd803f050 con 0x561cf73f2ca0 2024-09-19T07:00:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2ce636c740 10 client.19194 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2ce636c740 1 -- 172.21.15.103:0/3053818557 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561cf747c2b0 con 0x561cf73f2ca0 2024-09-19T07:00:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2cc57fa640 1 -- 172.21.15.103:0/3053818557 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ccc0223d0 con 0x561cf73f2ca0 2024-09-19T07:00:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2cc57fa640 1 -- 172.21.15.103:0/3053818557 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2cd8003c60 con 0x561cf73f2ca0 2024-09-19T07:00:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2ce636c740 10 client.19194 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2ce636c740 10 client.19194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2ce636c740 20 client.19194 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2ce636c740 1 --2- 172.21.15.103:0/3053818557 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561cf747c610 0x561cf747ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2ce636c740 4 client.19194 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:00:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.529+0000 7f2ce636c740 1 -- 172.21.15.103:0/3053818557 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561cf735fd60 con 0x561cf747c610 2024-09-19T07:00:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.530+0000 7f2ce152e640 1 --2- 172.21.15.103:0/3053818557 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561cf747c610 0x561cf747ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.530+0000 7f2cc57fa640 10 client.19194 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.530+0000 7f2ce152e640 1 --2- 172.21.15.103:0/3053818557 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561cf747c610 0x561cf747ea00 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3db207b5efa0000 server_cookie=202edc8ec8695100 in_seq=0 out_seq=0 2024-09-19T07:00:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.530+0000 7f2cc57fa640 1 -- 172.21.15.103:0/3053818557 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561cf735fd60 con 0x561cf747c610 2024-09-19T07:00:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.530+0000 7f2cc57fa640 10 client.19194 handle_command_reply: tid=0 2024-09-19T07:00:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce636c740 1 client.19194 shutdown 2024-09-19T07:00:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce636c740 1 -- 172.21.15.103:0/3053818557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2ca0 msgr2=0x561cf7405660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce636c740 1 --2- 172.21.15.103:0/3053818557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2ca0 0x561cf7405660 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8006010 tx=0x7f2cd8005750 comp rx=0 tx=0).stop 2024-09-19T07:00:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce636c740 1 -- 172.21.15.103:0/3053818557 shutdown_connections 2024-09-19T07:00:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce636c740 1 --2- 172.21.15.103:0/3053818557 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561cf747c610 0x561cf747ea00 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce636c740 1 --2- 172.21.15.103:0/3053818557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7401a90 0x561cf7405c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce636c740 1 --2- 172.21.15.103:0/3053818557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561cf73f2ca0 0x561cf7405660 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2cc57fa640 0 client.19194 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce636c740 1 --2- 172.21.15.103:0/3053818557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561cf73f2380 0x561cf740a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce636c740 1 -- 172.21.15.103:0/3053818557 >> 172.21.15.103:0/3053818557 conn(0x561cf73eea30 msgr2=0x561cf73f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce636c740 1 -- 172.21.15.103:0/3053818557 shutdown_connections 2024-09-19T07:00:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce636c740 1 -- 172.21.15.103:0/3053818557 wait complete. 2024-09-19T07:00:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce636c740 20 client.19194 trim_cache size 0 max 16384 2024-09-19T07:00:02.538 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:00:02.538 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:00:02.538 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:00:02.538 INFO:teuthology.orchestra.run.smithi103.stdout: "5d50ac73-4b34-4cbf-a202-c9de5f7842e0": { 2024-09-19T07:00:02.538 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:00:02.538 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "5d50ac73-4b34-4cbf-a202-c9de5f7842e0", 2024-09-19T07:00:02.539 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:00:02.539 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:00:02.539 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:00:02.539 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:00:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce3f93640 1 -- 172.21.15.103:0/4215669274 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2cc80a33b0 msgr2=0x7f2cc80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce3f93640 1 --2- 172.21.15.103:0/4215669274 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2cc80a33b0 0x7f2cc80a5870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4098790 tx=0x7f2cd4086c20 comp rx=0 tx=0).stop 2024-09-19T07:00:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce3f93640 1 -- 172.21.15.103:0/4215669274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cdc10c170 msgr2=0x7f2cdc19cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce3f93640 1 --2- 172.21.15.103:0/4215669274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cdc10c170 0x7f2cdc19cd80 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400eaa0 tx=0x7f2cd400ef60 comp rx=0 tx=0).stop 2024-09-19T07:00:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce3f93640 1 -- 172.21.15.103:0/4215669274 shutdown_connections 2024-09-19T07:00:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce3f93640 1 --2- 172.21.15.103:0/4215669274 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2cc80a33b0 0x7f2cc80a5870 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce3f93640 1 --2- 172.21.15.103:0/4215669274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cdc10c170 0x7f2cdc19cd80 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce3f93640 1 --2- 172.21.15.103:0/4215669274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cdc103660 0x7f2cdc1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.531+0000 7f2ce3f93640 1 --2- 172.21.15.103:0/4215669274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cdc102d40 0x7f2cdc1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.532+0000 7f2ce3f93640 1 -- 172.21.15.103:0/4215669274 >> 172.21.15.103:0/4215669274 conn(0x7f2cdc0fc7a0 msgr2=0x7f2cdc0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.532+0000 7f2ce3f93640 1 -- 172.21.15.103:0/4215669274 shutdown_connections 2024-09-19T07:00:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:02.532+0000 7f2ce3f93640 1 -- 172.21.15.103:0/4215669274 wait complete. 2024-09-19T07:00:03.253 INFO:tasks.cephfs.filesystem:scrub status for tag:5d50ac73-4b34-4cbf-a202-c9de5f7842e0 - {'path': '/', 'tag': '5d50ac73-4b34-4cbf-a202-c9de5f7842e0', 'options': 'recursive,force'} 2024-09-19T07:00:04.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:03 smithi134 ceph-mon[35356]: pgmap v335: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-19T07:00:04.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:03 smithi174 ceph-mon[35637]: pgmap v335: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-19T07:00:04.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:03 smithi103 ceph-mon[31158]: pgmap v335: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-19T07:00:05.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:04 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T07:00:05.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:04 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T07:00:05.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:04 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T07:00:06.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:05 smithi134 ceph-mon[35356]: pgmap v336: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-19T07:00:06.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:05 smithi174 ceph-mon[35637]: pgmap v336: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-19T07:00:06.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:05 smithi103 ceph-mon[31158]: pgmap v336: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-19T07:00:08.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:07 smithi134 ceph-mon[35356]: pgmap v337: 97 pgs: 97 active+clean; 2.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 48 KiB/s wr, 32 op/s 2024-09-19T07:00:08.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:07 smithi174 ceph-mon[35637]: pgmap v337: 97 pgs: 97 active+clean; 2.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 48 KiB/s wr, 32 op/s 2024-09-19T07:00:08.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:07 smithi103 ceph-mon[31158]: pgmap v337: 97 pgs: 97 active+clean; 2.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 48 KiB/s wr, 32 op/s 2024-09-19T07:00:08.832 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:00:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.044+0000 7f15de05e640 1 -- 172.21.15.103:0/208291880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15d8107b70 msgr2=0x7f15d8107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.044+0000 7f15de05e640 1 --2- 172.21.15.103:0/208291880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15d8107b70 0x7f15d8107f50 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009930 tx=0x7f15c802a420 comp rx=0 tx=0).stop 2024-09-19T07:00:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.045+0000 7f15de05e640 1 -- 172.21.15.103:0/208291880 shutdown_connections 2024-09-19T07:00:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.045+0000 7f15de05e640 1 --2- 172.21.15.103:0/208291880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15d8102470 0x7f15d810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.045+0000 7f15de05e640 1 --2- 172.21.15.103:0/208291880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d8101ad0 0x7f15d8101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.045+0000 7f15de05e640 1 --2- 172.21.15.103:0/208291880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15d8107b70 0x7f15d8107f50 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.045+0000 7f15de05e640 1 -- 172.21.15.103:0/208291880 >> 172.21.15.103:0/208291880 conn(0x7f15d80fd7d0 msgr2=0x7f15d80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.045+0000 7f15de05e640 1 -- 172.21.15.103:0/208291880 shutdown_connections 2024-09-19T07:00:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.045+0000 7f15de05e640 1 -- 172.21.15.103:0/208291880 wait complete. 2024-09-19T07:00:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.045+0000 7f15de05e640 1 Processor -- start 2024-09-19T07:00:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.045+0000 7f15de05e640 1 -- start start 2024-09-19T07:00:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15de05e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d8101ad0 0x7f15d8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15de05e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15d8102470 0x7f15d81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15de05e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15d8107b70 0x7f15d819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15de05e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f15d8111f70 con 0x7f15d8107b70 2024-09-19T07:00:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15de05e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15d8111df0 con 0x7f15d8101ad0 2024-09-19T07:00:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15de05e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15d81120f0 con 0x7f15d8102470 2024-09-19T07:00:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15d7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15d8107b70 0x7f15d819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15d7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15d8107b70 0x7f15d819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44074/0 (socket says 172.21.15.103:44074) 2024-09-19T07:00:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15d7fff640 1 -- 172.21.15.103:0/1736824527 learned_addr learned my addr 172.21.15.103:0/1736824527 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15d6ffd640 1 --2- 172.21.15.103:0/1736824527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15d8102470 0x7f15d81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15d77fe640 1 --2- 172.21.15.103:0/1736824527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d8101ad0 0x7f15d8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15d7fff640 1 -- 172.21.15.103:0/1736824527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15d8102470 msgr2=0x7f15d81988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15d7fff640 1 --2- 172.21.15.103:0/1736824527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15d8102470 0x7f15d81988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15d7fff640 1 -- 172.21.15.103:0/1736824527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d8101ad0 msgr2=0x7f15d8198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15d7fff640 1 --2- 172.21.15.103:0/1736824527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d8101ad0 0x7f15d8198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15d7fff640 1 -- 172.21.15.103:0/1736824527 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d819d350 con 0x7f15d8107b70 2024-09-19T07:00:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.046+0000 7f15d7fff640 1 --2- 172.21.15.103:0/1736824527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15d8107b70 0x7f15d819cc20 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f15d000eaa0 tx=0x7f15d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.047+0000 7f15d4ff9640 1 -- 172.21.15.103:0/1736824527 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d000cdb0 con 0x7f15d8107b70 2024-09-19T07:00:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.047+0000 7f15de05e640 1 -- 172.21.15.103:0/1736824527 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d819d640 con 0x7f15d8107b70 2024-09-19T07:00:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.047+0000 7f15d4ff9640 1 -- 172.21.15.103:0/1736824527 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15d0005020 con 0x7f15d8107b70 2024-09-19T07:00:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.047+0000 7f15d4ff9640 1 -- 172.21.15.103:0/1736824527 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d00116d0 con 0x7f15d8107b70 2024-09-19T07:00:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.047+0000 7f15de05e640 1 -- 172.21.15.103:0/1736824527 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d81a4ed0 con 0x7f15d8107b70 2024-09-19T07:00:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.049+0000 7f15d4ff9640 1 -- 172.21.15.103:0/1736824527 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f15d0004b20 con 0x7f15d8107b70 2024-09-19T07:00:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.049+0000 7f15e0df9740 1 Processor -- start 2024-09-19T07:00:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.049+0000 7f15e0df9740 1 -- start start 2024-09-19T07:00:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.049+0000 7f15d4ff9640 1 --2- 172.21.15.103:0/1736824527 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f15c00a3460 0x7f15c00a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15e0df9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dbe9b0 0x55ac53dbed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15e0df9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbf2d0 0x55ac53dcdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15e0df9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dce0c0 0x55ac53dd04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15e0df9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ac53d232f0 con 0x55ac53dbf2d0 2024-09-19T07:00:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15e0df9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ac53ccb270 con 0x55ac53dbe9b0 2024-09-19T07:00:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15e0df9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ac53d1f2b0 con 0x55ac53dce0c0 2024-09-19T07:00:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15d6ffd640 1 --2- 172.21.15.103:0/1736824527 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f15c00a3460 0x7f15c00a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15d4ff9640 1 -- 172.21.15.103:0/1736824527 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f15d00cfef0 con 0x7f15d8107b70 2024-09-19T07:00:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbf2d0 0x55ac53dcdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbf2d0 0x55ac53dcdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44076/0 (socket says 172.21.15.103:44076) 2024-09-19T07:00:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15d6ffd640 1 -- 172.21.15.103:0/1871884435 learned_addr learned my addr 172.21.15.103:0/1871884435 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15d77fe640 1 --2- 172.21.15.103:0/1871884435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dbe9b0 0x55ac53dbed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15d7fff640 1 --2- 172.21.15.103:0/1871884435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dce0c0 0x55ac53dd04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15d6ffd640 1 -- 172.21.15.103:0/1871884435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dce0c0 msgr2=0x55ac53dd04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15d6ffd640 1 --2- 172.21.15.103:0/1871884435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dce0c0 0x55ac53dd04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15d6ffd640 1 -- 172.21.15.103:0/1871884435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dbe9b0 msgr2=0x55ac53dbed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15d6ffd640 1 --2- 172.21.15.103:0/1871884435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dbe9b0 0x55ac53dbed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15d6ffd640 1 -- 172.21.15.103:0/1871884435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac53cca340 con 0x55ac53dbf2d0 2024-09-19T07:00:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15d6ffd640 1 --2- 172.21.15.103:0/1736824527 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f15c00a3460 0x7f15c00a5920 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f15cc012960 tx=0x7f15cc00bb70 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15d6ffd640 1 --2- 172.21.15.103:0/1871884435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbf2d0 0x55ac53dcdb80 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f15cc010740 tx=0x7f15cc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61db26656e0074d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.050+0000 7f15bdffb640 1 -- 172.21.15.103:0/1871884435 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc024070 con 0x55ac53dbf2d0 2024-09-19T07:00:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.051+0000 7f15bdffb640 1 -- 172.21.15.103:0/1871884435 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15cc010e00 con 0x55ac53dbf2d0 2024-09-19T07:00:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.051+0000 7f15bdffb640 1 -- 172.21.15.103:0/1871884435 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc020970 con 0x55ac53dbf2d0 2024-09-19T07:00:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.051+0000 7f15e0df9740 1 -- 172.21.15.103:0/1871884435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbf2d0 msgr2=0x55ac53dcdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.051+0000 7f15e0df9740 1 --2- 172.21.15.103:0/1871884435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbf2d0 0x55ac53dcdb80 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f15cc010740 tx=0x7f15cc01c000 comp rx=0 tx=0).stop 2024-09-19T07:00:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.051+0000 7f15e0df9740 1 -- 172.21.15.103:0/1871884435 shutdown_connections 2024-09-19T07:00:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.051+0000 7f15e0df9740 1 --2- 172.21.15.103:0/1871884435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dce0c0 0x55ac53dd04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.051+0000 7f15e0df9740 1 --2- 172.21.15.103:0/1871884435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbf2d0 0x55ac53dcdb80 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.051+0000 7f15e0df9740 1 --2- 172.21.15.103:0/1871884435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dbe9b0 0x55ac53dbed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.051+0000 7f15e0df9740 1 -- 172.21.15.103:0/1871884435 >> 172.21.15.103:0/1871884435 conn(0x55ac53dbb060 msgr2=0x55ac53dbb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.051+0000 7f15e0df9740 1 -- 172.21.15.103:0/1871884435 shutdown_connections 2024-09-19T07:00:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.051+0000 7f15e0df9740 1 -- 172.21.15.103:0/1871884435 wait complete. 2024-09-19T07:00:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.052+0000 7f15e0df9740 1 -- start start 2024-09-19T07:00:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.052+0000 7f15e0df9740 1 Processor -- start 2024-09-19T07:00:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.052+0000 7f15e0df9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dbe9b0 0x55ac53dcb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.052+0000 7f15e0df9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbf2d0 0x55ac53dcb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.052+0000 7f15e0df9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dce0c0 0x55ac53dcba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.052+0000 7f15e0df9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ac53dd0c30 con 0x55ac53dbf2d0 2024-09-19T07:00:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.052+0000 7f15e0df9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ac53cac290 con 0x55ac53dbe9b0 2024-09-19T07:00:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.052+0000 7f15e0df9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ac53dd0db0 con 0x55ac53dce0c0 2024-09-19T07:00:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.052+0000 7f15e0df9740 10 client.0 _ll_register_callbacks cb 0x55ac53cdd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.052+0000 7f15d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbf2d0 0x55ac53dcb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.052+0000 7f15d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbf2d0 0x55ac53dcb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44090/0 (socket says 172.21.15.103:44090) 2024-09-19T07:00:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.052+0000 7f15d6ffd640 1 -- 172.21.15.103:0/2284900080 learned_addr learned my addr 172.21.15.103:0/2284900080 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.052+0000 7f15d7fff640 1 --2- 172.21.15.103:0/2284900080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dce0c0 0x55ac53dcba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.052+0000 7f15d77fe640 1 --2- 172.21.15.103:0/2284900080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dbe9b0 0x55ac53dcb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.053+0000 7f15d6ffd640 1 -- 172.21.15.103:0/2284900080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dce0c0 msgr2=0x55ac53dcba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.053+0000 7f15d6ffd640 1 --2- 172.21.15.103:0/2284900080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dce0c0 0x55ac53dcba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.053+0000 7f15d6ffd640 1 -- 172.21.15.103:0/2284900080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dbe9b0 msgr2=0x55ac53dcb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.053+0000 7f15d6ffd640 1 --2- 172.21.15.103:0/2284900080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dbe9b0 0x55ac53dcb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.053+0000 7f15d6ffd640 1 -- 172.21.15.103:0/2284900080 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac53a36210 con 0x55ac53dbf2d0 2024-09-19T07:00:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.053+0000 7f15d6ffd640 1 --2- 172.21.15.103:0/2284900080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbf2d0 0x55ac53dcb550 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0058a0 tx=0x7f15cc020f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc451c5a49a64a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.053+0000 7f15b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.053+0000 7f15b6ffd640 1 -- 172.21.15.103:0/2284900080 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc024050 con 0x55ac53dbf2d0 2024-09-19T07:00:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.053+0000 7f15b6ffd640 1 -- 172.21.15.103:0/2284900080 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f15cc020900 con 0x55ac53dbf2d0 2024-09-19T07:00:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.053+0000 7f15e0df9740 1 -- 172.21.15.103:0/2284900080 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15cc010e00 con 0x55ac53dbf2d0 2024-09-19T07:00:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.053+0000 7f15b6ffd640 1 -- 172.21.15.103:0/2284900080 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f15cc005040 con 0x55ac53dbf2d0 2024-09-19T07:00:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.053+0000 7f15e0df9740 10 client.19206 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.053+0000 7f15e0df9740 1 -- 172.21.15.103:0/2284900080 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ac53b02620 con 0x55ac53dbf2d0 2024-09-19T07:00:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.053+0000 7f15b6ffd640 1 -- 172.21.15.103:0/2284900080 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15cc010e00 con 0x55ac53dbf2d0 2024-09-19T07:00:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.054+0000 7f15b6ffd640 1 -- 172.21.15.103:0/2284900080 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f15cc01f070 con 0x55ac53dbf2d0 2024-09-19T07:00:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.054+0000 7f15e0df9740 10 client.19206 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.054+0000 7f15e0df9740 10 client.19206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.054+0000 7f15e0df9740 20 client.19206 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.054+0000 7f15e0df9740 1 --2- 172.21.15.103:0/2284900080 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ac53dddf90 0x55ac53de0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.054+0000 7f15e0df9740 4 client.19206 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:00:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.054+0000 7f15e0df9740 1 -- 172.21.15.103:0/2284900080 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac53d30a20 con 0x55ac53dddf90 2024-09-19T07:00:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.054+0000 7f15d77fe640 1 --2- 172.21.15.103:0/2284900080 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ac53dddf90 0x55ac53de0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.055+0000 7f15b6ffd640 10 client.19206 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.055+0000 7f15d77fe640 1 --2- 172.21.15.103:0/2284900080 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ac53dddf90 0x55ac53de0380 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f32cc3f5b33b1109 server_cookie=16cf33839f41878f in_seq=0 out_seq=0 2024-09-19T07:00:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.055+0000 7f15b6ffd640 1 -- 172.21.15.103:0/2284900080 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac53d30a20 con 0x55ac53dddf90 2024-09-19T07:00:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.055+0000 7f15b6ffd640 10 client.19206 handle_command_reply: tid=0 2024-09-19T07:00:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.055+0000 7f15e0df9740 1 client.19206 shutdown 2024-09-19T07:00:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.055+0000 7f15e0df9740 1 -- 172.21.15.103:0/2284900080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbf2d0 msgr2=0x55ac53dcb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.055+0000 7f15e0df9740 1 --2- 172.21.15.103:0/2284900080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbf2d0 0x55ac53dcb550 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0058a0 tx=0x7f15cc020f20 comp rx=0 tx=0).stop 2024-09-19T07:00:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.055+0000 7f15e0df9740 1 -- 172.21.15.103:0/2284900080 shutdown_connections 2024-09-19T07:00:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.056+0000 7f15e0df9740 1 --2- 172.21.15.103:0/2284900080 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ac53dddf90 0x55ac53de0380 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.056+0000 7f15e0df9740 1 --2- 172.21.15.103:0/2284900080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dce0c0 0x55ac53dcba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.056+0000 7f15e0df9740 1 --2- 172.21.15.103:0/2284900080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbf2d0 0x55ac53dcb550 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.056+0000 7f15b6ffd640 0 client.19206 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.056+0000 7f15e0df9740 1 --2- 172.21.15.103:0/2284900080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dbe9b0 0x55ac53dcb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.056+0000 7f15e0df9740 1 -- 172.21.15.103:0/2284900080 >> 172.21.15.103:0/2284900080 conn(0x55ac53dbb060 msgr2=0x55ac53dc0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.056+0000 7f15e0df9740 1 -- 172.21.15.103:0/2284900080 shutdown_connections 2024-09-19T07:00:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.056+0000 7f15e0df9740 1 -- 172.21.15.103:0/2284900080 wait complete. 2024-09-19T07:00:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.056+0000 7f15e0df9740 20 client.19206 trim_cache size 0 max 16384 2024-09-19T07:00:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.071+0000 7f15e0df9740 1 Processor -- start 2024-09-19T07:00:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.071+0000 7f15e0df9740 1 -- start start 2024-09-19T07:00:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.071+0000 7f15e0df9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbe9b0 0x55ac53dba0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.071+0000 7f15e0df9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dbf2d0 0x55ac53de5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.071+0000 7f15e0df9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dce0c0 0x55ac53de6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.071+0000 7f15e0df9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f15d81ab680 con 0x55ac53dbe9b0 2024-09-19T07:00:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.071+0000 7f15e0df9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15d81ad1c0 con 0x55ac53dce0c0 2024-09-19T07:00:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.071+0000 7f15e0df9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15d81abd50 con 0x55ac53dbf2d0 2024-09-19T07:00:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.071+0000 7f15d77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbe9b0 0x55ac53dba0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.071+0000 7f15d77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbe9b0 0x55ac53dba0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44096/0 (socket says 172.21.15.103:44096) 2024-09-19T07:00:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.071+0000 7f15d77fe640 1 -- 172.21.15.103:0/3432162619 learned_addr learned my addr 172.21.15.103:0/3432162619 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.071+0000 7f15d6ffd640 1 --2- 172.21.15.103:0/3432162619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dbf2d0 0x55ac53de5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.072+0000 7f15d7fff640 1 --2- 172.21.15.103:0/3432162619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dce0c0 0x55ac53de6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.072+0000 7f15d77fe640 1 -- 172.21.15.103:0/3432162619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dbf2d0 msgr2=0x55ac53de5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.072+0000 7f15d77fe640 1 --2- 172.21.15.103:0/3432162619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dbf2d0 0x55ac53de5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.072+0000 7f15d77fe640 1 -- 172.21.15.103:0/3432162619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dce0c0 msgr2=0x55ac53de6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.072+0000 7f15d77fe640 1 --2- 172.21.15.103:0/3432162619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dce0c0 0x55ac53de6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.072+0000 7f15d77fe640 1 -- 172.21.15.103:0/3432162619 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac53dbf8b0 con 0x55ac53dbe9b0 2024-09-19T07:00:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.072+0000 7f15d6ffd640 1 --2- 172.21.15.103:0/3432162619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dbf2d0 0x55ac53de5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.072+0000 7f15d77fe640 1 --2- 172.21.15.103:0/3432162619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbe9b0 0x55ac53dba0b0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f15c8000c00 tx=0x7f15c8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fea7ac9369cfd23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.072+0000 7f15bd7fa640 1 -- 172.21.15.103:0/3432162619 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8035040 con 0x55ac53dbe9b0 2024-09-19T07:00:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.072+0000 7f15bd7fa640 1 -- 172.21.15.103:0/3432162619 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c804a070 con 0x55ac53dbe9b0 2024-09-19T07:00:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.072+0000 7f15bd7fa640 1 -- 172.21.15.103:0/3432162619 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8031810 con 0x55ac53dbe9b0 2024-09-19T07:00:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.073+0000 7f15e0df9740 1 -- 172.21.15.103:0/3432162619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbe9b0 msgr2=0x55ac53dba0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.073+0000 7f15e0df9740 1 --2- 172.21.15.103:0/3432162619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbe9b0 0x55ac53dba0b0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f15c8000c00 tx=0x7f15c8005b50 comp rx=0 tx=0).stop 2024-09-19T07:00:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.073+0000 7f15e0df9740 1 -- 172.21.15.103:0/3432162619 shutdown_connections 2024-09-19T07:00:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.073+0000 7f15e0df9740 1 --2- 172.21.15.103:0/3432162619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dce0c0 0x55ac53de6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.073+0000 7f15e0df9740 1 --2- 172.21.15.103:0/3432162619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dbf2d0 0x55ac53de5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.073+0000 7f15e0df9740 1 --2- 172.21.15.103:0/3432162619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dbe9b0 0x55ac53dba0b0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.073+0000 7f15e0df9740 1 -- 172.21.15.103:0/3432162619 >> 172.21.15.103:0/3432162619 conn(0x55ac53dbb060 msgr2=0x55ac53db99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.073+0000 7f15e0df9740 1 -- 172.21.15.103:0/3432162619 shutdown_connections 2024-09-19T07:00:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.073+0000 7f15e0df9740 1 -- 172.21.15.103:0/3432162619 wait complete. 2024-09-19T07:00:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.073+0000 7f15e0df9740 1 -- start start 2024-09-19T07:00:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.073+0000 7f15e0df9740 1 Processor -- start 2024-09-19T07:00:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15e0df9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dbe9b0 0x55ac53dd29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15e0df9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dbf2d0 0x55ac53dd2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15e0df9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dce0c0 0x55ac53e40aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15e0df9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f15d81acaf0 con 0x55ac53dce0c0 2024-09-19T07:00:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15e0df9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15d81ac420 con 0x55ac53dbf2d0 2024-09-19T07:00:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15e0df9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15d81a96a0 con 0x55ac53dbe9b0 2024-09-19T07:00:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15e0df9740 10 client.0 _ll_register_callbacks cb 0x7f15c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15d7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dce0c0 0x55ac53e40aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15d7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dce0c0 0x55ac53e40aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44112/0 (socket says 172.21.15.103:44112) 2024-09-19T07:00:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15d7fff640 1 -- 172.21.15.103:0/94561067 learned_addr learned my addr 172.21.15.103:0/94561067 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15d77fe640 1 --2- 172.21.15.103:0/94561067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dbe9b0 0x55ac53dd29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15d6ffd640 1 --2- 172.21.15.103:0/94561067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dbf2d0 0x55ac53dd2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15d7fff640 1 -- 172.21.15.103:0/94561067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dbe9b0 msgr2=0x55ac53dd29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15d7fff640 1 --2- 172.21.15.103:0/94561067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dbe9b0 0x55ac53dd29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15d7fff640 1 -- 172.21.15.103:0/94561067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dbf2d0 msgr2=0x55ac53dd2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15d7fff640 1 --2- 172.21.15.103:0/94561067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dbf2d0 0x55ac53dd2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15d7fff640 1 -- 172.21.15.103:0/94561067 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac53dc5d50 con 0x55ac53dce0c0 2024-09-19T07:00:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15d77fe640 1 --2- 172.21.15.103:0/94561067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dbe9b0 0x55ac53dd29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15d6ffd640 1 --2- 172.21.15.103:0/94561067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dbf2d0 0x55ac53dd2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15d7fff640 1 --2- 172.21.15.103:0/94561067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dce0c0 0x55ac53e40aa0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f15d009c140 tx=0x7f15d0013930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44e7304e38b21c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15b6ffd640 1 -- 172.21.15.103:0/94561067 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d001f040 con 0x55ac53dce0c0 2024-09-19T07:00:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15b6ffd640 1 -- 172.21.15.103:0/94561067 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f15d00051d0 con 0x55ac53dce0c0 2024-09-19T07:00:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.074+0000 7f15e0df9740 1 -- 172.21.15.103:0/94561067 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15c804a070 con 0x55ac53dce0c0 2024-09-19T07:00:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.075+0000 7f15b6ffd640 1 -- 172.21.15.103:0/94561067 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f15d00a0560 con 0x55ac53dce0c0 2024-09-19T07:00:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.075+0000 7f15e0df9740 10 client.19212 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.075+0000 7f15e0df9740 1 -- 172.21.15.103:0/94561067 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ac53e49510 con 0x55ac53dce0c0 2024-09-19T07:00:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.075+0000 7f15b6ffd640 1 -- 172.21.15.103:0/94561067 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c804a070 con 0x55ac53dce0c0 2024-09-19T07:00:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.075+0000 7f15b6ffd640 1 -- 172.21.15.103:0/94561067 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f15d000f040 con 0x55ac53dce0c0 2024-09-19T07:00:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.075+0000 7f15e0df9740 10 client.19212 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.075+0000 7f15e0df9740 10 client.19212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.075+0000 7f15e0df9740 20 client.19212 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.075+0000 7f15e0df9740 1 --2- 172.21.15.103:0/94561067 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ac53e49be0 0x55ac53e4bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.075+0000 7f15e0df9740 4 client.19212 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:00:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.075+0000 7f15e0df9740 1 -- 172.21.15.103:0/94561067 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac53d2ad70 con 0x55ac53e49be0 2024-09-19T07:00:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.075+0000 7f15d77fe640 1 --2- 172.21.15.103:0/94561067 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ac53e49be0 0x55ac53e4bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.076+0000 7f15b6ffd640 10 client.19212 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.076+0000 7f15d77fe640 1 --2- 172.21.15.103:0/94561067 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ac53e49be0 0x55ac53e4bff0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c83b74f84e30728 server_cookie=6b7674682dd09489 in_seq=0 out_seq=0 2024-09-19T07:00:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.083+0000 7f15b6ffd640 1 -- 172.21.15.103:0/94561067 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac53d2ad70 con 0x55ac53e49be0 2024-09-19T07:00:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.083+0000 7f15b6ffd640 10 client.19212 handle_command_reply: tid=0 2024-09-19T07:00:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.083+0000 7f15e0df9740 1 client.19212 shutdown 2024-09-19T07:00:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.083+0000 7f15e0df9740 1 -- 172.21.15.103:0/94561067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dce0c0 msgr2=0x55ac53e40aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.083+0000 7f15e0df9740 1 --2- 172.21.15.103:0/94561067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dce0c0 0x55ac53e40aa0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f15d009c140 tx=0x7f15d0013930 comp rx=0 tx=0).stop 2024-09-19T07:00:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.083+0000 7f15e0df9740 1 -- 172.21.15.103:0/94561067 shutdown_connections 2024-09-19T07:00:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.083+0000 7f15e0df9740 1 --2- 172.21.15.103:0/94561067 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ac53e49be0 0x55ac53e4bff0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.083+0000 7f15e0df9740 1 --2- 172.21.15.103:0/94561067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac53dce0c0 0x55ac53e40aa0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.083+0000 7f15b6ffd640 0 client.19212 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.083+0000 7f15e0df9740 1 --2- 172.21.15.103:0/94561067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac53dbf2d0 0x55ac53dd2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.083+0000 7f15e0df9740 1 --2- 172.21.15.103:0/94561067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac53dbe9b0 0x55ac53dd29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.083+0000 7f15e0df9740 1 -- 172.21.15.103:0/94561067 >> 172.21.15.103:0/94561067 conn(0x55ac53dbb060 msgr2=0x55ac53dba1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.083+0000 7f15e0df9740 1 -- 172.21.15.103:0/94561067 shutdown_connections 2024-09-19T07:00:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.083+0000 7f15e0df9740 1 -- 172.21.15.103:0/94561067 wait complete. 2024-09-19T07:00:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.083+0000 7f15e0df9740 20 client.19212 trim_cache size 0 max 16384 2024-09-19T07:00:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:00:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:00:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:00:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:00:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.084+0000 7f15de05e640 1 -- 172.21.15.103:0/1736824527 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f15c00a3460 msgr2=0x7f15c00a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.084+0000 7f15de05e640 1 --2- 172.21.15.103:0/1736824527 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f15c00a3460 0x7f15c00a5920 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f15cc012960 tx=0x7f15cc00bb70 comp rx=0 tx=0).stop 2024-09-19T07:00:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.084+0000 7f15de05e640 1 -- 172.21.15.103:0/1736824527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15d8107b70 msgr2=0x7f15d819cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.084+0000 7f15de05e640 1 --2- 172.21.15.103:0/1736824527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15d8107b70 0x7f15d819cc20 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f15d000eaa0 tx=0x7f15d000ef60 comp rx=0 tx=0).stop 2024-09-19T07:00:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.084+0000 7f15de05e640 1 -- 172.21.15.103:0/1736824527 shutdown_connections 2024-09-19T07:00:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.084+0000 7f15de05e640 1 --2- 172.21.15.103:0/1736824527 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f15c00a3460 0x7f15c00a5920 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.084+0000 7f15de05e640 1 --2- 172.21.15.103:0/1736824527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15d8107b70 0x7f15d819cc20 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.084+0000 7f15de05e640 1 --2- 172.21.15.103:0/1736824527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15d8102470 0x7f15d81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.084+0000 7f15de05e640 1 --2- 172.21.15.103:0/1736824527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d8101ad0 0x7f15d8198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.084+0000 7f15de05e640 1 -- 172.21.15.103:0/1736824527 >> 172.21.15.103:0/1736824527 conn(0x7f15d80fd7d0 msgr2=0x7f15d80fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.084+0000 7f15de05e640 1 -- 172.21.15.103:0/1736824527 shutdown_connections 2024-09-19T07:00:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:09.084+0000 7f15de05e640 1 -- 172.21.15.103:0/1736824527 wait complete. 2024-09-19T07:00:09.106 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:00:09.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.539+0000 7f4aa9c81640 1 -- 172.21.15.103:0/1122546147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4aa4102ef0 msgr2=0x7f4aa410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:09.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.539+0000 7f4aa9c81640 1 --2- 172.21.15.103:0/1122546147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4aa4102ef0 0x7f4aa410da40 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009ca0 tx=0x7f4a9c02a950 comp rx=0 tx=0).stop 2024-09-19T07:00:09.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.540+0000 7f4aa9c81640 1 -- 172.21.15.103:0/1122546147 shutdown_connections 2024-09-19T07:00:09.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.540+0000 7f4aa9c81640 1 --2- 172.21.15.103:0/1122546147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4aa4102ef0 0x7f4aa410da40 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.540+0000 7f4aa9c81640 1 --2- 172.21.15.103:0/1122546147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4aa4102570 0x7f4aa41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.540+0000 7f4aa9c81640 1 --2- 172.21.15.103:0/1122546147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4aa4100e20 0x7f4aa4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.540+0000 7f4aa9c81640 1 -- 172.21.15.103:0/1122546147 >> 172.21.15.103:0/1122546147 conn(0x7f4aa40fc7d0 msgr2=0x7f4aa40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:09.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.540+0000 7f4aa9c81640 1 -- 172.21.15.103:0/1122546147 shutdown_connections 2024-09-19T07:00:09.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.540+0000 7f4aa9c81640 1 -- 172.21.15.103:0/1122546147 wait complete. 2024-09-19T07:00:09.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.540+0000 7f4aa9c81640 1 Processor -- start 2024-09-19T07:00:09.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.540+0000 7f4aa9c81640 1 -- start start 2024-09-19T07:00:09.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4aa9c81640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4aa4100e20 0x7f4aa40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4aa9c81640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4aa4102570 0x7f4aa40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4aa9c81640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4aa4102ef0 0x7f4aa40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4aa9c81640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa4110f80 con 0x7f4aa4102ef0 2024-09-19T07:00:09.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4aa9c81640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa4110e00 con 0x7f4aa4102570 2024-09-19T07:00:09.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4aa9c81640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa4111100 con 0x7f4aa4100e20 2024-09-19T07:00:09.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4aa3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4aa4102ef0 0x7f4aa40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4aa3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4aa4102ef0 0x7f4aa40fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44134/0 (socket says 172.21.15.103:44134) 2024-09-19T07:00:09.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4aa3fff640 1 -- 172.21.15.103:0/1158203077 learned_addr learned my addr 172.21.15.103:0/1158203077 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:09.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4a9affd640 1 --2- 172.21.15.103:0/1158203077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4aa4102570 0x7f4aa40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4aa37fe640 1 --2- 172.21.15.103:0/1158203077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4aa4100e20 0x7f4aa40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4aa3fff640 1 -- 172.21.15.103:0/1158203077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4aa4100e20 msgr2=0x7f4aa40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:09.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4aa3fff640 1 --2- 172.21.15.103:0/1158203077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4aa4100e20 0x7f4aa40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4aa3fff640 1 -- 172.21.15.103:0/1158203077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4aa4102570 msgr2=0x7f4aa40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:09.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4aa3fff640 1 --2- 172.21.15.103:0/1158203077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4aa4102570 0x7f4aa40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4aa3fff640 1 -- 172.21.15.103:0/1158203077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa41a68a0 con 0x7f4aa4102ef0 2024-09-19T07:00:09.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.541+0000 7f4aa3fff640 1 --2- 172.21.15.103:0/1158203077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4aa4102ef0 0x7f4aa40fd700 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c02ae70 tx=0x7f4a9c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:09.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.542+0000 7f4aa17fa640 1 -- 172.21.15.103:0/1158203077 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c034040 con 0x7f4aa4102ef0 2024-09-19T07:00:09.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.542+0000 7f4aa17fa640 1 -- 172.21.15.103:0/1158203077 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a9c030920 con 0x7f4aa4102ef0 2024-09-19T07:00:09.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.542+0000 7f4aa9c81640 1 -- 172.21.15.103:0/1158203077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa41a6b30 con 0x7f4aa4102ef0 2024-09-19T07:00:09.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.542+0000 7f4aa17fa640 1 -- 172.21.15.103:0/1158203077 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c0047f0 con 0x7f4aa4102ef0 2024-09-19T07:00:09.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.542+0000 7f4aa9c81640 1 -- 172.21.15.103:0/1158203077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa41a7010 con 0x7f4aa4102ef0 2024-09-19T07:00:09.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.543+0000 7f4aa9c81640 1 -- 172.21.15.103:0/1158203077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aa4101200 con 0x7f4aa4102ef0 2024-09-19T07:00:09.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.544+0000 7f4aa17fa640 1 -- 172.21.15.103:0/1158203077 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4a9c037020 con 0x7f4aa4102ef0 2024-09-19T07:00:09.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.545+0000 7f4aa17fa640 1 --2- 172.21.15.103:0/1158203077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4a900a33b0 0x7f4a900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:09.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.545+0000 7f4aa37fe640 1 --2- 172.21.15.103:0/1158203077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4a900a33b0 0x7f4a900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:09.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.545+0000 7f4aa17fa640 1 -- 172.21.15.103:0/1158203077 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f4a9c0f0b70 con 0x7f4aa4102ef0 2024-09-19T07:00:09.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.545+0000 7f4aa37fe640 1 --2- 172.21.15.103:0/1158203077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4a900a33b0 0x7f4a900a5870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400bbe0 tx=0x7f4a9400b390 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:09.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.548+0000 7f4aa17fa640 1 -- 172.21.15.103:0/1158203077 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4a9c0bd5e0 con 0x7f4aa4102ef0 2024-09-19T07:00:09.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.747+0000 7f4aa9c81640 1 -- 172.21.15.103:0/1158203077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4aa410a0a0 con 0x7f4aa4102ef0 2024-09-19T07:00:09.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.748+0000 7f4aa17fa640 1 -- 172.21.15.103:0/1158203077 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f4a9c0bcee0 con 0x7f4aa4102ef0 2024-09-19T07:00:09.751 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:00:09.751 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:00:09.751 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:00:09.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.750+0000 7f4aa9c81640 1 -- 172.21.15.103:0/1158203077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4a900a33b0 msgr2=0x7f4a900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:09.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.750+0000 7f4aa9c81640 1 --2- 172.21.15.103:0/1158203077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4a900a33b0 0x7f4a900a5870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400bbe0 tx=0x7f4a9400b390 comp rx=0 tx=0).stop 2024-09-19T07:00:09.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.750+0000 7f4aa9c81640 1 -- 172.21.15.103:0/1158203077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4aa4102ef0 msgr2=0x7f4aa40fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:09.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.750+0000 7f4aa9c81640 1 --2- 172.21.15.103:0/1158203077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4aa4102ef0 0x7f4aa40fd700 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c02ae70 tx=0x7f4a9c02f5d0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.750+0000 7f4aa9c81640 1 -- 172.21.15.103:0/1158203077 shutdown_connections 2024-09-19T07:00:09.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.750+0000 7f4aa9c81640 1 --2- 172.21.15.103:0/1158203077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4a900a33b0 0x7f4a900a5870 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.750+0000 7f4aa9c81640 1 --2- 172.21.15.103:0/1158203077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4aa4102ef0 0x7f4aa40fd700 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.750+0000 7f4aa9c81640 1 --2- 172.21.15.103:0/1158203077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4aa4102570 0x7f4aa40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.750+0000 7f4aa9c81640 1 --2- 172.21.15.103:0/1158203077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4aa4100e20 0x7f4aa40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:09.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.750+0000 7f4aa9c81640 1 -- 172.21.15.103:0/1158203077 >> 172.21.15.103:0/1158203077 conn(0x7f4aa40fc7d0 msgr2=0x7f4aa406a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:09.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.750+0000 7f4aa9c81640 1 -- 172.21.15.103:0/1158203077 shutdown_connections 2024-09-19T07:00:09.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:09.750+0000 7f4aa9c81640 1 -- 172.21.15.103:0/1158203077 wait complete. 2024-09-19T07:00:10.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:09 smithi134 ceph-mon[35356]: pgmap v338: 97 pgs: 97 active+clean; 2.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 28 op/s 2024-09-19T07:00:10.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:09 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1158203077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:00:10.245 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:09 smithi103 ceph-mon[31158]: pgmap v338: 97 pgs: 97 active+clean; 2.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 28 op/s 2024-09-19T07:00:10.245 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:09 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1158203077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:00:10.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:09 smithi174 ceph-mon[35637]: pgmap v338: 97 pgs: 97 active+clean; 2.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 28 op/s 2024-09-19T07:00:10.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:09 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1158203077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:00:10.472 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 5003d0af --release --await' 2024-09-19T07:00:10.473 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 5003d0af --release --await 2024-09-19T07:00:10.727 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:00:11.095 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:10 smithi103 ceph-mon[31158]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T07:00:11.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.180+0000 7f2918fd8640 1 -- 172.21.15.103:0/3340949626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2914101420 msgr2=0x7f291410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:11.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.180+0000 7f2918fd8640 1 --2- 172.21.15.103:0/3340949626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2914101420 0x7f291410d950 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f2908009930 tx=0x7f290802a420 comp rx=0 tx=0).stop 2024-09-19T07:00:11.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.180+0000 7f2918fd8640 1 -- 172.21.15.103:0/3340949626 shutdown_connections 2024-09-19T07:00:11.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.180+0000 7f2918fd8640 1 --2- 172.21.15.103:0/3340949626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2914101420 0x7f291410d950 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.180+0000 7f2918fd8640 1 --2- 172.21.15.103:0/3340949626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2914100a80 0x7f2914100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.180+0000 7f2918fd8640 1 --2- 172.21.15.103:0/3340949626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2914106a80 0x7f2914106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.180+0000 7f2918fd8640 1 -- 172.21.15.103:0/3340949626 >> 172.21.15.103:0/3340949626 conn(0x7f29140fc7a0 msgr2=0x7f29140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:11.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.180+0000 7f2918fd8640 1 -- 172.21.15.103:0/3340949626 shutdown_connections 2024-09-19T07:00:11.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.180+0000 7f2918fd8640 1 -- 172.21.15.103:0/3340949626 wait complete. 2024-09-19T07:00:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2918fd8640 1 Processor -- start 2024-09-19T07:00:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2918fd8640 1 -- start start 2024-09-19T07:00:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2918fd8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2914100a80 0x7f29141973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2918fd8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2914101420 0x7f2914197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2918fd8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2914106a80 0x7f291419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2918fd8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2914110ec0 con 0x7f2914106a80 2024-09-19T07:00:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2918fd8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2914110d40 con 0x7f2914101420 2024-09-19T07:00:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2918fd8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2914111040 con 0x7f2914100a80 2024-09-19T07:00:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2912d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2914106a80 0x7f291419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2912d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2914106a80 0x7f291419bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44148/0 (socket says 172.21.15.103:44148) 2024-09-19T07:00:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2912d9d640 1 -- 172.21.15.103:0/27125406 learned_addr learned my addr 172.21.15.103:0/27125406 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f291259c640 1 --2- 172.21.15.103:0/27125406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2914100a80 0x7f29141973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2911d9b640 1 --2- 172.21.15.103:0/27125406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2914101420 0x7f2914197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2912d9d640 1 -- 172.21.15.103:0/27125406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2914100a80 msgr2=0x7f29141973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2912d9d640 1 --2- 172.21.15.103:0/27125406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2914100a80 0x7f29141973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2912d9d640 1 -- 172.21.15.103:0/27125406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2914101420 msgr2=0x7f2914197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:11.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2912d9d640 1 --2- 172.21.15.103:0/27125406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2914101420 0x7f2914197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.181+0000 7f2912d9d640 1 -- 172.21.15.103:0/27125406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291419c370 con 0x7f2914106a80 2024-09-19T07:00:11.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.182+0000 7f2912d9d640 1 --2- 172.21.15.103:0/27125406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2914106a80 0x7f291419bc90 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f2908005ab0 tx=0x7f29080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:11.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.182+0000 7f29037fe640 1 -- 172.21.15.103:0/27125406 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290803f030 con 0x7f2914106a80 2024-09-19T07:00:11.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.182+0000 7f29037fe640 1 -- 172.21.15.103:0/27125406 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2908003ce0 con 0x7f2914106a80 2024-09-19T07:00:11.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.182+0000 7f2918fd8640 1 -- 172.21.15.103:0/27125406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291419c600 con 0x7f2914106a80 2024-09-19T07:00:11.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.182+0000 7f29037fe640 1 -- 172.21.15.103:0/27125406 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908035070 con 0x7f2914106a80 2024-09-19T07:00:11.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.182+0000 7f2918fd8640 1 -- 172.21.15.103:0/27125406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291406a790 con 0x7f2914106a80 2024-09-19T07:00:11.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.185+0000 7f291b3b1740 1 Processor -- start 2024-09-19T07:00:11.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.185+0000 7f29037fe640 1 -- 172.21.15.103:0/27125406 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f2908042030 con 0x7f2914106a80 2024-09-19T07:00:11.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.185+0000 7f291b3b1740 1 -- start start 2024-09-19T07:00:11.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:10 smithi134 ceph-mon[35356]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T07:00:11.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.185+0000 7f291b3b1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f10302a060 0x55f10302a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.185+0000 7f291b3b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a980 0x55f103034d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.185+0000 7f291b3b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f103035240 0x55f103037630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.185+0000 7f291b3b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f102f8f620 con 0x55f10302a980 2024-09-19T07:00:11.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.185+0000 7f291b3b1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f102f36e50 con 0x55f10302a060 2024-09-19T07:00:11.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.185+0000 7f291b3b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f102f8b5e0 con 0x55f103035240 2024-09-19T07:00:11.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f291259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a980 0x55f103034d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:11.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f291259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a980 0x55f103034d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44152/0 (socket says 172.21.15.103:44152) 2024-09-19T07:00:11.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f291259c640 1 -- 172.21.15.103:0/3565824064 learned_addr learned my addr 172.21.15.103:0/3565824064 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:11.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f2911d9b640 1 --2- 172.21.15.103:0/3565824064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f10302a060 0x55f10302a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:11.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f2911d9b640 1 --2- 172.21.15.103:0/3565824064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f103035240 0x55f103037630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:11.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f29037fe640 1 --2- 172.21.15.103:0/27125406 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28fc0a3430 0x7f28fc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f291259c640 1 -- 172.21.15.103:0/3565824064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f103035240 msgr2=0x55f103037630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:11.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f291259c640 1 --2- 172.21.15.103:0/3565824064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f103035240 0x55f103037630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f2912d9d640 1 --2- 172.21.15.103:0/27125406 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28fc0a3430 0x7f28fc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:11.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f291259c640 1 -- 172.21.15.103:0/3565824064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f10302a060 msgr2=0x55f10302a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f291259c640 1 --2- 172.21.15.103:0/3565824064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f10302a060 0x55f10302a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f291259c640 1 -- 172.21.15.103:0/3565824064 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f102f27d40 con 0x55f10302a980 2024-09-19T07:00:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f29037fe640 1 -- 172.21.15.103:0/27125406 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8600+0+0 (secure 0 0 0) 0x7f29080f0150 con 0x7f2914106a80 2024-09-19T07:00:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f291259c640 1 --2- 172.21.15.103:0/3565824064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a980 0x55f103034d00 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f2904009a70 tx=0x7f2904009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab77c268591c68c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f2900ff9640 1 -- 172.21.15.103:0/3565824064 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904017070 con 0x55f10302a980 2024-09-19T07:00:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f2900ff9640 1 -- 172.21.15.103:0/3565824064 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2904013470 con 0x55f10302a980 2024-09-19T07:00:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f2900ff9640 1 -- 172.21.15.103:0/3565824064 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904012610 con 0x55f10302a980 2024-09-19T07:00:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.186+0000 7f2912d9d640 1 --2- 172.21.15.103:0/27125406 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28fc0a3430 0x7f28fc0a58f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f29080ed900 tx=0x7f29080871c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.187+0000 7f291b3b1740 1 -- 172.21.15.103:0/3565824064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a980 msgr2=0x55f103034d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.187+0000 7f291b3b1740 1 --2- 172.21.15.103:0/3565824064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a980 0x55f103034d00 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f2904009a70 tx=0x7f2904009f30 comp rx=0 tx=0).stop 2024-09-19T07:00:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.187+0000 7f291b3b1740 1 -- 172.21.15.103:0/3565824064 shutdown_connections 2024-09-19T07:00:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.187+0000 7f291b3b1740 1 --2- 172.21.15.103:0/3565824064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f103035240 0x55f103037630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.187+0000 7f291b3b1740 1 --2- 172.21.15.103:0/3565824064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a980 0x55f103034d00 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.187+0000 7f291b3b1740 1 --2- 172.21.15.103:0/3565824064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f10302a060 0x55f10302a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.187+0000 7f291b3b1740 1 -- 172.21.15.103:0/3565824064 >> 172.21.15.103:0/3565824064 conn(0x55f1030265f0 msgr2=0x55f1030269d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:11.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.187+0000 7f291b3b1740 1 -- 172.21.15.103:0/3565824064 shutdown_connections 2024-09-19T07:00:11.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.187+0000 7f291b3b1740 1 -- 172.21.15.103:0/3565824064 wait complete. 2024-09-19T07:00:11.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f291b3b1740 1 -- start start 2024-09-19T07:00:11.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f291b3b1740 1 Processor -- start 2024-09-19T07:00:11.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f291b3b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a060 0x55f103032110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f291b3b1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f10302a980 0x55f103032650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f291b3b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f103035240 0x55f103032b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f291b3b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f103037e10 con 0x55f10302a060 2024-09-19T07:00:11.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f291b3b1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f102f17f10 con 0x55f10302a980 2024-09-19T07:00:11.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f291b3b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f103037f90 con 0x55f103035240 2024-09-19T07:00:11.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f291b3b1740 10 client.0 _ll_register_callbacks cb 0x55f102f3bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:11.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f2911d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a060 0x55f103032110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:11.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f2911d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a060 0x55f103032110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44166/0 (socket says 172.21.15.103:44166) 2024-09-19T07:00:11.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f2911d9b640 1 -- 172.21.15.103:0/3082590264 learned_addr learned my addr 172.21.15.103:0/3082590264 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:11.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f291259c640 1 --2- 172.21.15.103:0/3082590264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f10302a980 0x55f103032650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:11.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f2911d9b640 1 -- 172.21.15.103:0/3082590264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f103035240 msgr2=0x55f103032b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:11.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f2911d9b640 1 --2- 172.21.15.103:0/3082590264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f103035240 0x55f103032b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f2911d9b640 1 -- 172.21.15.103:0/3082590264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f10302a980 msgr2=0x55f103032650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:11.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f2911d9b640 1 --2- 172.21.15.103:0/3082590264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f10302a980 0x55f103032650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f2911d9b640 1 -- 172.21.15.103:0/3082590264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f102c9e580 con 0x55f10302a060 2024-09-19T07:00:11.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.188+0000 7f291259c640 1 --2- 172.21.15.103:0/3082590264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f10302a980 0x55f103032650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:11.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.189+0000 7f2911d9b640 1 --2- 172.21.15.103:0/3082590264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a060 0x55f103032110 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f290c014a30 tx=0x7f290c014ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321ab4f326515def server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:11.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.189+0000 7f28f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:11.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.189+0000 7f28f9ffb640 1 -- 172.21.15.103:0/3082590264 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c011bf0 con 0x55f10302a060 2024-09-19T07:00:11.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.189+0000 7f28f9ffb640 1 -- 172.21.15.103:0/3082590264 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8600+0+0 (secure 0 0 0) 0x7f290c011d90 con 0x55f10302a060 2024-09-19T07:00:11.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.189+0000 7f291b3b1740 1 -- 172.21.15.103:0/3082590264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2904013470 con 0x55f10302a060 2024-09-19T07:00:11.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.189+0000 7f28f9ffb640 1 -- 172.21.15.103:0/3082590264 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f290c025050 con 0x55f10302a060 2024-09-19T07:00:11.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.189+0000 7f291b3b1740 10 client.19230 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:11.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.189+0000 7f291b3b1740 1 -- 172.21.15.103:0/3082590264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f102d68960 con 0x55f10302a060 2024-09-19T07:00:11.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.189+0000 7f28f9ffb640 1 -- 172.21.15.103:0/3082590264 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2904013470 con 0x55f10302a060 2024-09-19T07:00:11.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.189+0000 7f28f9ffb640 1 -- 172.21.15.103:0/3082590264 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f290c00c070 con 0x55f10302a060 2024-09-19T07:00:11.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.189+0000 7f291b3b1740 10 client.19230 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:11.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.189+0000 7f291b3b1740 10 client.19230 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:00:11.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.189+0000 7f291b3b1740 20 client.19230 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:11.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.189+0000 7f291b3b1740 1 --2- 172.21.15.103:0/3082590264 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f103044ed0 0x55f103047300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.189+0000 7f291b3b1740 4 client.19230 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:00:11.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.189+0000 7f291b3b1740 1 -- 172.21.15.103:0/3082590264 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f102f9cb90 con 0x55f103044ed0 2024-09-19T07:00:11.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.190+0000 7f291259c640 1 --2- 172.21.15.103:0/3082590264 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f103044ed0 0x55f103047300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:11.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.190+0000 7f291259c640 1 --2- 172.21.15.103:0/3082590264 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f103044ed0 0x55f103047300 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc441286c12c44ec server_cookie=26f4c80d852167a9 in_seq=0 out_seq=0 2024-09-19T07:00:11.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.190+0000 7f28f9ffb640 10 client.19230 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:11.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.191+0000 7f28f9ffb640 1 -- 172.21.15.103:0/3082590264 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f102f9cb90 con 0x55f103044ed0 2024-09-19T07:00:11.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.191+0000 7f28f9ffb640 10 client.19230 handle_command_reply: tid=0 2024-09-19T07:00:11.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.191+0000 7f291b3b1740 1 client.19230 shutdown 2024-09-19T07:00:11.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.191+0000 7f291b3b1740 1 -- 172.21.15.103:0/3082590264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a060 msgr2=0x55f103032110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:11.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.191+0000 7f291b3b1740 1 --2- 172.21.15.103:0/3082590264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a060 0x55f103032110 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f290c014a30 tx=0x7f290c014ef0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.191+0000 7f291b3b1740 1 -- 172.21.15.103:0/3082590264 shutdown_connections 2024-09-19T07:00:11.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.191+0000 7f291b3b1740 1 --2- 172.21.15.103:0/3082590264 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f103044ed0 0x55f103047300 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.191+0000 7f291b3b1740 1 --2- 172.21.15.103:0/3082590264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f103035240 0x55f103032b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.191+0000 7f291b3b1740 1 --2- 172.21.15.103:0/3082590264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f10302a980 0x55f103032650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.191+0000 7f28f9ffb640 0 client.19230 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:11.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.191+0000 7f291b3b1740 1 --2- 172.21.15.103:0/3082590264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a060 0x55f103032110 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.191+0000 7f291b3b1740 1 -- 172.21.15.103:0/3082590264 >> 172.21.15.103:0/3082590264 conn(0x55f1030265f0 msgr2=0x55f103025750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:11.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.191+0000 7f291b3b1740 1 -- 172.21.15.103:0/3082590264 shutdown_connections 2024-09-19T07:00:11.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.191+0000 7f291b3b1740 1 -- 172.21.15.103:0/3082590264 wait complete. 2024-09-19T07:00:11.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.191+0000 7f291b3b1740 20 client.19230 trim_cache size 0 max 16384 2024-09-19T07:00:11.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.207+0000 7f291b3b1740 1 Processor -- start 2024-09-19T07:00:11.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.207+0000 7f291b3b1740 1 -- start start 2024-09-19T07:00:11.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f291b3b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a060 0x55f103031070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f291b3b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f10302a980 0x55f10304cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f291b3b1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f103035240 0x55f10304d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f291b3b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f29141afbb0 con 0x55f10302a060 2024-09-19T07:00:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f291b3b1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29141b16f0 con 0x55f103035240 2024-09-19T07:00:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f291b3b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f29141b0280 con 0x55f10302a980 2024-09-19T07:00:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f2911d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a060 0x55f103031070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f2911d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a060 0x55f103031070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44176/0 (socket says 172.21.15.103:44176) 2024-09-19T07:00:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f2911d9b640 1 -- 172.21.15.103:0/769178330 learned_addr learned my addr 172.21.15.103:0/769178330 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f2911d9b640 1 -- 172.21.15.103:0/769178330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f10302a980 msgr2=0x55f10304cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f2911d9b640 1 --2- 172.21.15.103:0/769178330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f10302a980 0x55f10304cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f2911d9b640 1 -- 172.21.15.103:0/769178330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f103035240 msgr2=0x55f10304d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f2911d9b640 1 --2- 172.21.15.103:0/769178330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f103035240 0x55f10304d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f2911d9b640 1 -- 172.21.15.103:0/769178330 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f10302a7b0 con 0x55f10302a060 2024-09-19T07:00:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f2911d9b640 1 --2- 172.21.15.103:0/769178330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a060 0x55f103031070 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f290c009e00 tx=0x7f290c009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba5ad16c5783241 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f28fbfff640 1 -- 172.21.15.103:0/769178330 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c02d020 con 0x55f10302a060 2024-09-19T07:00:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f28fbfff640 1 -- 172.21.15.103:0/769178330 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f290c011040 con 0x55f10302a060 2024-09-19T07:00:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.208+0000 7f28fbfff640 1 -- 172.21.15.103:0/769178330 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c029430 con 0x55f10302a060 2024-09-19T07:00:11.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.209+0000 7f291b3b1740 1 -- 172.21.15.103:0/769178330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a060 msgr2=0x55f103031070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:11.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.209+0000 7f291b3b1740 1 --2- 172.21.15.103:0/769178330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a060 0x55f103031070 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f290c009e00 tx=0x7f290c009aa0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.209+0000 7f291b3b1740 1 -- 172.21.15.103:0/769178330 shutdown_connections 2024-09-19T07:00:11.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.209+0000 7f291b3b1740 1 --2- 172.21.15.103:0/769178330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f103035240 0x55f10304d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.209+0000 7f291b3b1740 1 --2- 172.21.15.103:0/769178330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f10302a980 0x55f10304cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.209+0000 7f291b3b1740 1 --2- 172.21.15.103:0/769178330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f10302a060 0x55f103031070 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.209+0000 7f291b3b1740 1 -- 172.21.15.103:0/769178330 >> 172.21.15.103:0/769178330 conn(0x55f1030265f0 msgr2=0x55f103034190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:11.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.209+0000 7f291b3b1740 1 -- 172.21.15.103:0/769178330 shutdown_connections 2024-09-19T07:00:11.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.209+0000 7f291b3b1740 1 -- 172.21.15.103:0/769178330 wait complete. 2024-09-19T07:00:11.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.209+0000 7f291b3b1740 1 -- start start 2024-09-19T07:00:11.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.209+0000 7f291b3b1740 1 Processor -- start 2024-09-19T07:00:11.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f291b3b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f10302a060 0x55f10303d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f291b3b1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f10302a980 0x55f10303d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f291b3b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f103035240 0x55f10303a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f291b3b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f29141b1020 con 0x55f103035240 2024-09-19T07:00:11.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f291b3b1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29141b0950 con 0x55f10302a980 2024-09-19T07:00:11.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f291b3b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f29141adbb0 con 0x55f10302a060 2024-09-19T07:00:11.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f291b3b1740 10 client.0 _ll_register_callbacks cb 0x7f290c014670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:11.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f2911d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f103035240 0x55f10303a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:11.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f2911d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f103035240 0x55f10303a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44186/0 (socket says 172.21.15.103:44186) 2024-09-19T07:00:11.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f2911d9b640 1 -- 172.21.15.103:0/2859279260 learned_addr learned my addr 172.21.15.103:0/2859279260 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:11.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f2911d9b640 1 --2- 172.21.15.103:0/2859279260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f10302a060 0x55f10303d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:11.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f2911d9b640 1 -- 172.21.15.103:0/2859279260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f10302a060 msgr2=0x55f10303d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:11.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f2911d9b640 1 --2- 172.21.15.103:0/2859279260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f10302a060 0x55f10303d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f2911d9b640 1 -- 172.21.15.103:0/2859279260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f10302a980 msgr2=0x55f10303d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:11.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f291259c640 1 --2- 172.21.15.103:0/2859279260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f10302a980 0x55f10303d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:11.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f2911d9b640 1 --2- 172.21.15.103:0/2859279260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f10302a980 0x55f10303d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f2911d9b640 1 -- 172.21.15.103:0/2859279260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f10303a6f0 con 0x55f103035240 2024-09-19T07:00:11.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f2911d9b640 1 --2- 172.21.15.103:0/2859279260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f103035240 0x55f10303a1b0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f290c009e90 tx=0x7f290c005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7dabc8b53ea4ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:11.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.210+0000 7f28f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:11.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.211+0000 7f28f9ffb640 1 -- 172.21.15.103:0/2859279260 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c02d030 con 0x55f103035240 2024-09-19T07:00:11.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.211+0000 7f28f9ffb640 1 -- 172.21.15.103:0/2859279260 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8600+0+0 (secure 0 0 0) 0x7f290c011bf0 con 0x55f103035240 2024-09-19T07:00:11.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.211+0000 7f291b3b1740 1 -- 172.21.15.103:0/2859279260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290c011040 con 0x55f103035240 2024-09-19T07:00:11.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.211+0000 7f28f9ffb640 1 -- 172.21.15.103:0/2859279260 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f290c011df0 con 0x55f103035240 2024-09-19T07:00:11.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.211+0000 7f291b3b1740 10 client.19236 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:11.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.211+0000 7f291b3b1740 1 -- 172.21.15.103:0/2859279260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f1030b0450 con 0x55f103035240 2024-09-19T07:00:11.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.211+0000 7f28f9ffb640 1 -- 172.21.15.103:0/2859279260 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f290c011040 con 0x55f103035240 2024-09-19T07:00:11.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.211+0000 7f28f9ffb640 1 -- 172.21.15.103:0/2859279260 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f290c00c040 con 0x55f103035240 2024-09-19T07:00:11.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.211+0000 7f291b3b1740 10 client.19236 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:11.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.211+0000 7f291b3b1740 10 client.19236 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:00:11.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.211+0000 7f291b3b1740 20 client.19236 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:11.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.211+0000 7f291b3b1740 1 --2- 172.21.15.103:0/2859279260 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f1030b0b40 0x55f1030b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:11.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.211+0000 7f291b3b1740 4 client.19236 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5003d0af", "release": true, "await": true}] 2024-09-19T07:00:11.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.211+0000 7f291b3b1740 1 -- 172.21.15.103:0/2859279260 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5003d0af", "release": true, "await": true}) -- 0x55f102f970a0 con 0x55f1030b0b40 2024-09-19T07:00:11.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.211+0000 7f291259c640 1 --2- 172.21.15.103:0/2859279260 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f1030b0b40 0x55f1030b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:11.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.212+0000 7f28f9ffb640 10 client.19236 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:11.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.212+0000 7f291259c640 1 --2- 172.21.15.103:0/2859279260 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f1030b0b40 0x55f1030b2f60 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4336aafae74ce773 server_cookie=bb44ee6267ece24 in_seq=0 out_seq=0 2024-09-19T07:00:11.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.215+0000 7f28f9ffb640 1 -- 172.21.15.103:0/2859279260 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f102f970a0 con 0x55f1030b0b40 2024-09-19T07:00:11.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.215+0000 7f28f9ffb640 10 client.19236 handle_command_reply: tid=0 2024-09-19T07:00:11.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.215+0000 7f291b3b1740 1 client.19236 shutdown 2024-09-19T07:00:11.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.215+0000 7f291b3b1740 1 -- 172.21.15.103:0/2859279260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f103035240 msgr2=0x55f10303a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:11.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.215+0000 7f291b3b1740 1 --2- 172.21.15.103:0/2859279260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f103035240 0x55f10303a1b0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f290c009e90 tx=0x7f290c005ac0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.215+0000 7f291b3b1740 1 -- 172.21.15.103:0/2859279260 shutdown_connections 2024-09-19T07:00:11.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.215+0000 7f291b3b1740 1 --2- 172.21.15.103:0/2859279260 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f1030b0b40 0x55f1030b2f60 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.215+0000 7f291b3b1740 1 --2- 172.21.15.103:0/2859279260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f103035240 0x55f10303a1b0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.215+0000 7f291b3b1740 1 --2- 172.21.15.103:0/2859279260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f10302a980 0x55f10303d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.215+0000 7f291b3b1740 1 --2- 172.21.15.103:0/2859279260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f10302a060 0x55f10303d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.215+0000 7f291b3b1740 1 -- 172.21.15.103:0/2859279260 >> 172.21.15.103:0/2859279260 conn(0x55f1030265f0 msgr2=0x55f10302bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:11.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.215+0000 7f28f9ffb640 0 client.19236 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:11.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.215+0000 7f291b3b1740 1 -- 172.21.15.103:0/2859279260 shutdown_connections 2024-09-19T07:00:11.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.215+0000 7f291b3b1740 1 -- 172.21.15.103:0/2859279260 wait complete. 2024-09-19T07:00:11.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.215+0000 7f291b3b1740 20 client.19236 trim_cache size 0 max 16384 2024-09-19T07:00:11.220 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:00:11.221 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T07:00:11.221 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T07:00:11.221 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 40, 2024-09-19T07:00:11.221 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:00:11.221 INFO:teuthology.orchestra.run.smithi103.stdout: "5003d0af": { 2024-09-19T07:00:11.221 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 40, 2024-09-19T07:00:11.221 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:00:11.221 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:00:11.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:00:11.221 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:00:11.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:00:11.221 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:00:11.221 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 150.0, 2024-09-19T07:00:11.222 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:00:11.222 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:00:11.222 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:00:11.222 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:00:11.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:00:11.222 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:00:11.222 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:00:11.222 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:00:11.222 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:00:11.222 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:00:11.222 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:00:11.222 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:00:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.216+0000 7f2918fd8640 1 -- 172.21.15.103:0/27125406 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28fc0a3430 msgr2=0x7f28fc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.216+0000 7f2918fd8640 1 --2- 172.21.15.103:0/27125406 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28fc0a3430 0x7f28fc0a58f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f29080ed900 tx=0x7f29080871c0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.216+0000 7f2918fd8640 1 -- 172.21.15.103:0/27125406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2914106a80 msgr2=0x7f291419bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.216+0000 7f2918fd8640 1 --2- 172.21.15.103:0/27125406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2914106a80 0x7f291419bc90 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f2908005ab0 tx=0x7f29080042f0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.216+0000 7f2918fd8640 1 -- 172.21.15.103:0/27125406 shutdown_connections 2024-09-19T07:00:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.216+0000 7f2918fd8640 1 --2- 172.21.15.103:0/27125406 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28fc0a3430 0x7f28fc0a58f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.216+0000 7f2918fd8640 1 --2- 172.21.15.103:0/27125406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2914106a80 0x7f291419bc90 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.216+0000 7f2918fd8640 1 --2- 172.21.15.103:0/27125406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2914101420 0x7f2914197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.216+0000 7f2918fd8640 1 --2- 172.21.15.103:0/27125406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2914100a80 0x7f29141973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.216+0000 7f2918fd8640 1 -- 172.21.15.103:0/27125406 >> 172.21.15.103:0/27125406 conn(0x7f29140fc7a0 msgr2=0x7f29140fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:11.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.216+0000 7f2918fd8640 1 -- 172.21.15.103:0/27125406 shutdown_connections 2024-09-19T07:00:11.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:11.216+0000 7f2918fd8640 1 -- 172.21.15.103:0/27125406 wait complete. 2024-09-19T07:00:11.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:10 smithi174 ceph-mon[35637]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T07:00:11.886 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5003d0af', total seconds elapsed: 33.1 2024-09-19T07:00:12.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:11 smithi103 ceph-mon[31158]: pgmap v340: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-19T07:00:12.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:11 smithi134 ceph-mon[35356]: pgmap v340: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-19T07:00:12.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:11 smithi174 ceph-mon[35637]: pgmap v340: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-19T07:00:14.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:13 smithi134 ceph-mon[35356]: pgmap v341: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-19T07:00:14.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:13 smithi174 ceph-mon[35637]: pgmap v341: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-19T07:00:14.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:13 smithi103 ceph-mon[31158]: pgmap v341: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-19T07:00:16.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:15 smithi134 ceph-mon[35356]: pgmap v342: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-19T07:00:16.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:15 smithi174 ceph-mon[35637]: pgmap v342: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-19T07:00:16.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:15 smithi103 ceph-mon[31158]: pgmap v342: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-09-19T07:00:18.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:17 smithi134 ceph-mon[35356]: pgmap v343: 97 pgs: 97 active+clean; 2.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-19T07:00:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:17 smithi174 ceph-mon[35637]: pgmap v343: 97 pgs: 97 active+clean; 2.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-19T07:00:18.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:17 smithi103 ceph-mon[31158]: pgmap v343: 97 pgs: 97 active+clean; 2.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-19T07:00:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:19 smithi174 ceph-mon[35637]: pgmap v344: 97 pgs: 97 active+clean; 2.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-19T07:00:20.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:19 smithi103 ceph-mon[31158]: pgmap v344: 97 pgs: 97 active+clean; 2.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-19T07:00:20.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:19 smithi134 ceph-mon[35356]: pgmap v344: 97 pgs: 97 active+clean; 2.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-19T07:00:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.185+0000 7fc01ac80640 1 -- 172.21.15.103:0/1197740065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0141080f0 msgr2=0x7fc01410ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.185+0000 7fc01ac80640 1 --2- 172.21.15.103:0/1197740065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0141080f0 0x7fc01410ea40 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009cd0 tx=0x7fc00c02a970 comp rx=0 tx=0).stop 2024-09-19T07:00:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.185+0000 7fc01ac80640 1 -- 172.21.15.103:0/1197740065 shutdown_connections 2024-09-19T07:00:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.185+0000 7fc01ac80640 1 --2- 172.21.15.103:0/1197740065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0141080f0 0x7fc01410ea40 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.185+0000 7fc01ac80640 1 --2- 172.21.15.103:0/1197740065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc014101150 0x7fc014107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.185+0000 7fc01ac80640 1 --2- 172.21.15.103:0/1197740065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc014100830 0x7fc014100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.185+0000 7fc01ac80640 1 -- 172.21.15.103:0/1197740065 >> 172.21.15.103:0/1197740065 conn(0x7fc014075f20 msgr2=0x7fc014076330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.185+0000 7fc01ac80640 1 -- 172.21.15.103:0/1197740065 shutdown_connections 2024-09-19T07:00:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.185+0000 7fc01ac80640 1 -- 172.21.15.103:0/1197740065 wait complete. 2024-09-19T07:00:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.185+0000 7fc01ac80640 1 Processor -- start 2024-09-19T07:00:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.185+0000 7fc01ac80640 1 -- start start 2024-09-19T07:00:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc01ac80640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc014100830 0x7fc014198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc01ac80640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc014101150 0x7fc014198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc01ac80640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0141080f0 0x7fc01419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc01ac80640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc014111f20 con 0x7fc014100830 2024-09-19T07:00:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc01ac80640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc014111da0 con 0x7fc0141080f0 2024-09-19T07:00:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc01ac80640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141120a0 con 0x7fc014101150 2024-09-19T07:00:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc018a1c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc014100830 0x7fc014198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc018a1c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc014100830 0x7fc014198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46842/0 (socket says 172.21.15.103:46842) 2024-09-19T07:00:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc018a1c640 1 -- 172.21.15.103:0/304406950 learned_addr learned my addr 172.21.15.103:0/304406950 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc013fff640 1 --2- 172.21.15.103:0/304406950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc014101150 0x7fc014198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc01921d640 1 --2- 172.21.15.103:0/304406950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0141080f0 0x7fc01419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc018a1c640 1 -- 172.21.15.103:0/304406950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc014101150 msgr2=0x7fc014198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc018a1c640 1 --2- 172.21.15.103:0/304406950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc014101150 0x7fc014198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc018a1c640 1 -- 172.21.15.103:0/304406950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0141080f0 msgr2=0x7fc01419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc018a1c640 1 --2- 172.21.15.103:0/304406950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0141080f0 0x7fc01419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc018a1c640 1 -- 172.21.15.103:0/304406950 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01419d320 con 0x7fc014100830 2024-09-19T07:00:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc013fff640 1 --2- 172.21.15.103:0/304406950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc014101150 0x7fc014198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc01921d640 1 --2- 172.21.15.103:0/304406950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0141080f0 0x7fc01419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc018a1c640 1 --2- 172.21.15.103:0/304406950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc014100830 0x7fc014198330 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fc00400d950 tx=0x7fc00400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc011ffb640 1 -- 172.21.15.103:0/304406950 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004017070 con 0x7fc014100830 2024-09-19T07:00:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc011ffb640 1 -- 172.21.15.103:0/304406950 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc004009c30 con 0x7fc014100830 2024-09-19T07:00:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc01ac80640 1 -- 172.21.15.103:0/304406950 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01419d610 con 0x7fc014100830 2024-09-19T07:00:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.186+0000 7fc011ffb640 1 -- 172.21.15.103:0/304406950 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00400f6b0 con 0x7fc014100830 2024-09-19T07:00:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.187+0000 7fc01ac80640 1 -- 172.21.15.103:0/304406950 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0141a4ea0 con 0x7fc014100830 2024-09-19T07:00:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.189+0000 7fc011ffb640 1 -- 172.21.15.103:0/304406950 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc0040109d0 con 0x7fc014100830 2024-09-19T07:00:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc01da0e740 1 Processor -- start 2024-09-19T07:00:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc01da0e740 1 -- start start 2024-09-19T07:00:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc01da0e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804ffefaf0 0x55804ffefed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc01da0e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fff0410 0x55804fffa770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc01da0e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fffacb0 0x55804fffd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc011ffb640 1 --2- 172.21.15.103:0/304406950 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbffc0a3460 0x7fbffc0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc01da0e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55804ff542f0 con 0x55804ffefaf0 2024-09-19T07:00:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc01da0e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55804fefc270 con 0x55804fff0410 2024-09-19T07:00:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc01da0e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55804ff502b0 con 0x55804fffacb0 2024-09-19T07:00:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc01921d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804ffefaf0 0x55804ffefed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc018a1c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fff0410 0x55804fffa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc01921d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804ffefaf0 0x55804ffefed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46854/0 (socket says 172.21.15.103:46854) 2024-09-19T07:00:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc01921d640 1 -- 172.21.15.103:0/2776908887 learned_addr learned my addr 172.21.15.103:0/2776908887 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc011ffb640 1 -- 172.21.15.103:0/304406950 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc0040cfa40 con 0x7fc014100830 2024-09-19T07:00:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc01921d640 1 --2- 172.21.15.103:0/304406950 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbffc0a3460 0x7fbffc0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc013fff640 1 --2- 172.21.15.103:0/2776908887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fffacb0 0x55804fffd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc018a1c640 1 -- 172.21.15.103:0/2776908887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fffacb0 msgr2=0x55804fffd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc018a1c640 1 --2- 172.21.15.103:0/2776908887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fffacb0 0x55804fffd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc018a1c640 1 -- 172.21.15.103:0/2776908887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804ffefaf0 msgr2=0x55804ffefed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc018a1c640 1 --2- 172.21.15.103:0/2776908887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804ffefaf0 0x55804ffefed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.190+0000 7fc018a1c640 1 -- 172.21.15.103:0/2776908887 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55804fefb340 con 0x55804fff0410 2024-09-19T07:00:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.191+0000 7fc01921d640 1 --2- 172.21.15.103:0/304406950 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbffc0a3460 0x7fbffc0a5920 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc00c005e30 tx=0x7fc00c0095c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.191+0000 7fc018a1c640 1 --2- 172.21.15.103:0/2776908887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fff0410 0x55804fffa770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc0040cd170 tx=0x7fc004010d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f33b6b6ab7289990 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.191+0000 7fbffaffd640 1 -- 172.21.15.103:0/2776908887 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004020070 con 0x55804fff0410 2024-09-19T07:00:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.191+0000 7fbffaffd640 1 -- 172.21.15.103:0/2776908887 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc00409c400 con 0x55804fff0410 2024-09-19T07:00:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.191+0000 7fbffaffd640 1 -- 172.21.15.103:0/2776908887 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004098370 con 0x55804fff0410 2024-09-19T07:00:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.192+0000 7fc01da0e740 1 -- 172.21.15.103:0/2776908887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fff0410 msgr2=0x55804fffa770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.192+0000 7fc01da0e740 1 --2- 172.21.15.103:0/2776908887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fff0410 0x55804fffa770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc0040cd170 tx=0x7fc004010d70 comp rx=0 tx=0).stop 2024-09-19T07:00:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.192+0000 7fc01da0e740 1 -- 172.21.15.103:0/2776908887 shutdown_connections 2024-09-19T07:00:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.192+0000 7fc01da0e740 1 --2- 172.21.15.103:0/2776908887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fffacb0 0x55804fffd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.192+0000 7fc01da0e740 1 --2- 172.21.15.103:0/2776908887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fff0410 0x55804fffa770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.192+0000 7fc01da0e740 1 --2- 172.21.15.103:0/2776908887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804ffefaf0 0x55804ffefed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.192+0000 7fc01da0e740 1 -- 172.21.15.103:0/2776908887 >> 172.21.15.103:0/2776908887 conn(0x55804ffec060 msgr2=0x55804ffec440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.192+0000 7fc01da0e740 1 -- 172.21.15.103:0/2776908887 shutdown_connections 2024-09-19T07:00:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.192+0000 7fc01da0e740 1 -- 172.21.15.103:0/2776908887 wait complete. 2024-09-19T07:00:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.192+0000 7fc01da0e740 1 -- start start 2024-09-19T07:00:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.192+0000 7fc01da0e740 1 Processor -- start 2024-09-19T07:00:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc01da0e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804ffefaf0 0x55804fff7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc01da0e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fff0410 0x55804fff80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc01da0e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fffacb0 0x55804fff8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc01da0e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55804fffd820 con 0x55804ffefaf0 2024-09-19T07:00:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc01da0e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55804fedd290 con 0x55804fffacb0 2024-09-19T07:00:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc01da0e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55804fffd9a0 con 0x55804fff0410 2024-09-19T07:00:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc01da0e740 10 client.0 _ll_register_callbacks cb 0x55804ff0e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc018a1c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804ffefaf0 0x55804fff7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc018a1c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804ffefaf0 0x55804fff7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46866/0 (socket says 172.21.15.103:46866) 2024-09-19T07:00:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc018a1c640 1 -- 172.21.15.103:0/2053483079 learned_addr learned my addr 172.21.15.103:0/2053483079 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc01921d640 1 --2- 172.21.15.103:0/2053483079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fffacb0 0x55804fff8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc013fff640 1 --2- 172.21.15.103:0/2053483079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fff0410 0x55804fff80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc018a1c640 1 -- 172.21.15.103:0/2053483079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fff0410 msgr2=0x55804fff80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc018a1c640 1 --2- 172.21.15.103:0/2053483079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fff0410 0x55804fff80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc018a1c640 1 -- 172.21.15.103:0/2053483079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fffacb0 msgr2=0x55804fff8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc018a1c640 1 --2- 172.21.15.103:0/2053483079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fffacb0 0x55804fff8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc018a1c640 1 -- 172.21.15.103:0/2053483079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55804fc67210 con 0x55804ffefaf0 2024-09-19T07:00:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc01921d640 1 --2- 172.21.15.103:0/2053483079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fffacb0 0x55804fff8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc013fff640 1 --2- 172.21.15.103:0/2053483079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fff0410 0x55804fff80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc018a1c640 1 --2- 172.21.15.103:0/2053483079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804ffefaf0 0x55804fff7bb0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fc0040cd220 tx=0x7fc004000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cec2e44f82c1d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fbff3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fbff3fff640 1 -- 172.21.15.103:0/2053483079 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004020070 con 0x55804ffefaf0 2024-09-19T07:00:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fbff3fff640 1 -- 172.21.15.103:0/2053483079 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc0040a28a0 con 0x55804ffefaf0 2024-09-19T07:00:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc01da0e740 1 -- 172.21.15.103:0/2053483079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc00409c400 con 0x55804ffefaf0 2024-09-19T07:00:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fbff3fff640 1 -- 172.21.15.103:0/2053483079 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc0040a2aa0 con 0x55804ffefaf0 2024-09-19T07:00:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc01da0e740 10 client.19248 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.193+0000 7fc01da0e740 1 -- 172.21.15.103:0/2053483079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55804fd33620 con 0x55804ffefaf0 2024-09-19T07:00:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.194+0000 7fbff3fff640 1 -- 172.21.15.103:0/2053483079 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc00409c400 con 0x55804ffefaf0 2024-09-19T07:00:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.194+0000 7fbff3fff640 1 -- 172.21.15.103:0/2053483079 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc00409b150 con 0x55804ffefaf0 2024-09-19T07:00:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.194+0000 7fc01da0e740 10 client.19248 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.194+0000 7fc01da0e740 10 client.19248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.194+0000 7fc01da0e740 20 client.19248 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.194+0000 7fc01da0e740 1 --2- 172.21.15.103:0/2053483079 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55805000aa50 0x55805000ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.194+0000 7fc01da0e740 4 client.19248 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:00:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.194+0000 7fc01da0e740 1 -- 172.21.15.103:0/2053483079 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55804ff61a20 con 0x55805000aa50 2024-09-19T07:00:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.194+0000 7fc013fff640 1 --2- 172.21.15.103:0/2053483079 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55805000aa50 0x55805000ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.194+0000 7fbff3fff640 10 client.19248 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.194+0000 7fc013fff640 1 --2- 172.21.15.103:0/2053483079 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55805000aa50 0x55805000ce40 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de77334dcdb9333b server_cookie=f051ac3dc2c34c91 in_seq=0 out_seq=0 2024-09-19T07:00:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.195+0000 7fbff3fff640 1 -- 172.21.15.103:0/2053483079 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55804ff61a20 con 0x55805000aa50 2024-09-19T07:00:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.195+0000 7fbff3fff640 10 client.19248 handle_command_reply: tid=0 2024-09-19T07:00:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.195+0000 7fc01da0e740 1 client.19248 shutdown 2024-09-19T07:00:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.195+0000 7fc01da0e740 1 -- 172.21.15.103:0/2053483079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804ffefaf0 msgr2=0x55804fff7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.195+0000 7fc01da0e740 1 --2- 172.21.15.103:0/2053483079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804ffefaf0 0x55804fff7bb0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fc0040cd220 tx=0x7fc004000bc0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.196+0000 7fc01da0e740 1 -- 172.21.15.103:0/2053483079 shutdown_connections 2024-09-19T07:00:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.196+0000 7fc01da0e740 1 --2- 172.21.15.103:0/2053483079 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55805000aa50 0x55805000ce40 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.196+0000 7fc01da0e740 1 --2- 172.21.15.103:0/2053483079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fffacb0 0x55804fff8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.196+0000 7fc01da0e740 1 --2- 172.21.15.103:0/2053483079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fff0410 0x55804fff80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.196+0000 7fbff3fff640 0 client.19248 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.196+0000 7fc01da0e740 1 --2- 172.21.15.103:0/2053483079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804ffefaf0 0x55804fff7bb0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.196+0000 7fc01da0e740 1 -- 172.21.15.103:0/2053483079 >> 172.21.15.103:0/2053483079 conn(0x55804ffec060 msgr2=0x55804fff1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.196+0000 7fc01da0e740 1 -- 172.21.15.103:0/2053483079 shutdown_connections 2024-09-19T07:00:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.196+0000 7fc01da0e740 1 -- 172.21.15.103:0/2053483079 wait complete. 2024-09-19T07:00:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.196+0000 7fc01da0e740 20 client.19248 trim_cache size 0 max 16384 2024-09-19T07:00:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.210+0000 7fc01da0e740 1 Processor -- start 2024-09-19T07:00:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.210+0000 7fc01da0e740 1 -- start start 2024-09-19T07:00:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.210+0000 7fc01da0e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804ffefaf0 0x55804fff8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.210+0000 7fc01da0e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804fff0410 0x558050012440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.210+0000 7fc01da0e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fffacb0 0x558050012980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.210+0000 7fc01da0e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141ad8c0 con 0x55804fff0410 2024-09-19T07:00:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.210+0000 7fc01da0e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141af400 con 0x55804fffacb0 2024-09-19T07:00:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.210+0000 7fc01da0e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141adf90 con 0x55804ffefaf0 2024-09-19T07:00:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.211+0000 7fc013fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804fff0410 0x558050012440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.211+0000 7fc013fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804fff0410 0x558050012440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46878/0 (socket says 172.21.15.103:46878) 2024-09-19T07:00:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.211+0000 7fc013fff640 1 -- 172.21.15.103:0/2139348858 learned_addr learned my addr 172.21.15.103:0/2139348858 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.211+0000 7fc018a1c640 1 --2- 172.21.15.103:0/2139348858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804ffefaf0 0x55804fff8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.211+0000 7fc01921d640 1 --2- 172.21.15.103:0/2139348858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fffacb0 0x558050012980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.211+0000 7fc013fff640 1 -- 172.21.15.103:0/2139348858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804ffefaf0 msgr2=0x55804fff8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.211+0000 7fc013fff640 1 --2- 172.21.15.103:0/2139348858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804ffefaf0 0x55804fff8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.211+0000 7fc013fff640 1 -- 172.21.15.103:0/2139348858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fffacb0 msgr2=0x558050012980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.211+0000 7fc013fff640 1 --2- 172.21.15.103:0/2139348858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fffacb0 0x558050012980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.211+0000 7fc013fff640 1 -- 172.21.15.103:0/2139348858 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55804ffebe90 con 0x55804fff0410 2024-09-19T07:00:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.211+0000 7fc018a1c640 1 --2- 172.21.15.103:0/2139348858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804ffefaf0 0x55804fff8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.211+0000 7fc01921d640 1 --2- 172.21.15.103:0/2139348858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fffacb0 0x558050012980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.211+0000 7fc013fff640 1 --2- 172.21.15.103:0/2139348858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804fff0410 0x558050012440 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fc0080058c0 tx=0x7fc008005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77cc3e702501f1b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.211+0000 7fbffa7fc640 1 -- 172.21.15.103:0/2139348858 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00801d070 con 0x55804fff0410 2024-09-19T07:00:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.211+0000 7fbffa7fc640 1 -- 172.21.15.103:0/2139348858 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc008019a60 con 0x55804fff0410 2024-09-19T07:00:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.211+0000 7fbffa7fc640 1 -- 172.21.15.103:0/2139348858 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00800fc20 con 0x55804fff0410 2024-09-19T07:00:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.212+0000 7fc01da0e740 1 -- 172.21.15.103:0/2139348858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804fff0410 msgr2=0x558050012440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.212+0000 7fc01da0e740 1 --2- 172.21.15.103:0/2139348858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804fff0410 0x558050012440 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fc0080058c0 tx=0x7fc008005e90 comp rx=0 tx=0).stop 2024-09-19T07:00:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.212+0000 7fc01da0e740 1 -- 172.21.15.103:0/2139348858 shutdown_connections 2024-09-19T07:00:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.212+0000 7fc01da0e740 1 --2- 172.21.15.103:0/2139348858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804fffacb0 0x558050012980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.212+0000 7fc01da0e740 1 --2- 172.21.15.103:0/2139348858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804fff0410 0x558050012440 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.212+0000 7fc01da0e740 1 --2- 172.21.15.103:0/2139348858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804ffefaf0 0x55804fff8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.212+0000 7fc01da0e740 1 -- 172.21.15.103:0/2139348858 >> 172.21.15.103:0/2139348858 conn(0x55804ffec060 msgr2=0x55804fff9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.212+0000 7fc01da0e740 1 -- 172.21.15.103:0/2139348858 shutdown_connections 2024-09-19T07:00:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.212+0000 7fc01da0e740 1 -- 172.21.15.103:0/2139348858 wait complete. 2024-09-19T07:00:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.212+0000 7fc01da0e740 1 -- start start 2024-09-19T07:00:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.212+0000 7fc01da0e740 1 Processor -- start 2024-09-19T07:00:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01da0e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804ffefaf0 0x55804fffdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01da0e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fff0410 0x55804fffe130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01da0e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804fffacb0 0x55804fffe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01da0e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141aed30 con 0x55804fffacb0 2024-09-19T07:00:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01da0e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141ae660 con 0x55804ffefaf0 2024-09-19T07:00:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01da0e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141ab8c0 con 0x55804fff0410 2024-09-19T07:00:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01da0e740 10 client.0 _ll_register_callbacks cb 0x7fc00c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01921d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804fffacb0 0x55804fffe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01921d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804fffacb0 0x55804fffe670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46880/0 (socket says 172.21.15.103:46880) 2024-09-19T07:00:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01921d640 1 -- 172.21.15.103:0/1723541515 learned_addr learned my addr 172.21.15.103:0/1723541515 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc013fff640 1 --2- 172.21.15.103:0/1723541515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fff0410 0x55804fffe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc018a1c640 1 --2- 172.21.15.103:0/1723541515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804ffefaf0 0x55804fffdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01921d640 1 -- 172.21.15.103:0/1723541515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fff0410 msgr2=0x55804fffe130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01921d640 1 --2- 172.21.15.103:0/1723541515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fff0410 0x55804fffe130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01921d640 1 -- 172.21.15.103:0/1723541515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804ffefaf0 msgr2=0x55804fffdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01921d640 1 --2- 172.21.15.103:0/1723541515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804ffefaf0 0x55804fffdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01921d640 1 -- 172.21.15.103:0/1723541515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558050073650 con 0x55804fffacb0 2024-09-19T07:00:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc013fff640 1 --2- 172.21.15.103:0/1723541515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fff0410 0x55804fffe130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01921d640 1 --2- 172.21.15.103:0/1723541515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804fffacb0 0x55804fffe670 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fc00c030000 tx=0x7fc00c02a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1155c83868411d44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fbff3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fbff3fff640 1 -- 172.21.15.103:0/1723541515 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c034040 con 0x55804fffacb0 2024-09-19T07:00:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fbff3fff640 1 -- 172.21.15.103:0/1723541515 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc00c004450 con 0x55804fffacb0 2024-09-19T07:00:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01da0e740 1 -- 172.21.15.103:0/1723541515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc008019a60 con 0x55804fffacb0 2024-09-19T07:00:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fbff3fff640 1 -- 172.21.15.103:0/1723541515 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc00c02f900 con 0x55804fffacb0 2024-09-19T07:00:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.213+0000 7fc01da0e740 10 client.19254 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.214+0000 7fc01da0e740 1 -- 172.21.15.103:0/1723541515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5580500748a0 con 0x55804fffacb0 2024-09-19T07:00:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.214+0000 7fbff3fff640 1 -- 172.21.15.103:0/1723541515 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc008019a60 con 0x55804fffacb0 2024-09-19T07:00:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.214+0000 7fbff3fff640 1 -- 172.21.15.103:0/1723541515 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc00c02eb50 con 0x55804fffacb0 2024-09-19T07:00:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.214+0000 7fc01da0e740 10 client.19254 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.214+0000 7fc01da0e740 10 client.19254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.214+0000 7fc01da0e740 20 client.19254 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.214+0000 7fc01da0e740 1 --2- 172.21.15.103:0/1723541515 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5580500750d0 0x5580500774f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.214+0000 7fc01da0e740 4 client.19254 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:00:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.214+0000 7fc01da0e740 1 -- 172.21.15.103:0/1723541515 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55804ff5bd70 con 0x5580500750d0 2024-09-19T07:00:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.214+0000 7fc018a1c640 1 --2- 172.21.15.103:0/1723541515 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5580500750d0 0x5580500774f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.215+0000 7fbff3fff640 10 client.19254 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.215+0000 7fc018a1c640 1 --2- 172.21.15.103:0/1723541515 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5580500750d0 0x5580500774f0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5485461921666b85 server_cookie=7ecef9b1119bcc21 in_seq=0 out_seq=0 2024-09-19T07:00:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.225+0000 7fbff3fff640 1 -- 172.21.15.103:0/1723541515 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55804ff5bd70 con 0x5580500750d0 2024-09-19T07:00:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.225+0000 7fbff3fff640 10 client.19254 handle_command_reply: tid=0 2024-09-19T07:00:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.225+0000 7fc01da0e740 1 client.19254 shutdown 2024-09-19T07:00:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.226+0000 7fc01da0e740 1 -- 172.21.15.103:0/1723541515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804fffacb0 msgr2=0x55804fffe670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.226+0000 7fc01da0e740 1 --2- 172.21.15.103:0/1723541515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804fffacb0 0x55804fffe670 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fc00c030000 tx=0x7fc00c02a970 comp rx=0 tx=0).stop 2024-09-19T07:00:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.226+0000 7fc01da0e740 1 -- 172.21.15.103:0/1723541515 shutdown_connections 2024-09-19T07:00:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.226+0000 7fc01da0e740 1 --2- 172.21.15.103:0/1723541515 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5580500750d0 0x5580500774f0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.226+0000 7fc01da0e740 1 --2- 172.21.15.103:0/1723541515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55804fffacb0 0x55804fffe670 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.226+0000 7fc01da0e740 1 --2- 172.21.15.103:0/1723541515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55804fff0410 0x55804fffe130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.226+0000 7fbff3fff640 0 client.19254 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.226+0000 7fc01da0e740 1 --2- 172.21.15.103:0/1723541515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55804ffefaf0 0x55804fffdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.226+0000 7fc01da0e740 1 -- 172.21.15.103:0/1723541515 >> 172.21.15.103:0/1723541515 conn(0x55804ffec060 msgr2=0x55804fff6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.226+0000 7fc01da0e740 1 -- 172.21.15.103:0/1723541515 shutdown_connections 2024-09-19T07:00:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.226+0000 7fc01da0e740 1 -- 172.21.15.103:0/1723541515 wait complete. 2024-09-19T07:00:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.226+0000 7fc01da0e740 20 client.19254 trim_cache size 0 max 16384 2024-09-19T07:00:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:00:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:00:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:00:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:00:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.226+0000 7fc01ac80640 1 -- 172.21.15.103:0/304406950 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbffc0a3460 msgr2=0x7fbffc0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.226+0000 7fc01ac80640 1 --2- 172.21.15.103:0/304406950 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbffc0a3460 0x7fbffc0a5920 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc00c005e30 tx=0x7fc00c0095c0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.226+0000 7fc01ac80640 1 -- 172.21.15.103:0/304406950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc014100830 msgr2=0x7fc014198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.226+0000 7fc01ac80640 1 --2- 172.21.15.103:0/304406950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc014100830 0x7fc014198330 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fc00400d950 tx=0x7fc00400de10 comp rx=0 tx=0).stop 2024-09-19T07:00:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.227+0000 7fc01ac80640 1 -- 172.21.15.103:0/304406950 shutdown_connections 2024-09-19T07:00:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.227+0000 7fc01ac80640 1 --2- 172.21.15.103:0/304406950 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbffc0a3460 0x7fbffc0a5920 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.227+0000 7fc01ac80640 1 --2- 172.21.15.103:0/304406950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0141080f0 0x7fc01419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.227+0000 7fc01ac80640 1 --2- 172.21.15.103:0/304406950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc014101150 0x7fc014198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.227+0000 7fc01ac80640 1 --2- 172.21.15.103:0/304406950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc014100830 0x7fc014198330 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.227+0000 7fc01ac80640 1 -- 172.21.15.103:0/304406950 >> 172.21.15.103:0/304406950 conn(0x7fc014075f20 msgr2=0x7fc0140ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.227+0000 7fc01ac80640 1 -- 172.21.15.103:0/304406950 shutdown_connections 2024-09-19T07:00:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:22.227+0000 7fc01ac80640 1 -- 172.21.15.103:0/304406950 wait complete. 2024-09-19T07:00:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:21 smithi174 ceph-mon[35637]: pgmap v345: 97 pgs: 97 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 20 op/s 2024-09-19T07:00:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:21 smithi174 ceph-mon[35637]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T07:00:22.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:21 smithi103 ceph-mon[31158]: pgmap v345: 97 pgs: 97 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 20 op/s 2024-09-19T07:00:22.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:21 smithi103 ceph-mon[31158]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T07:00:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:21 smithi134 ceph-mon[35356]: pgmap v345: 97 pgs: 97 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 20 op/s 2024-09-19T07:00:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:21 smithi134 ceph-mon[35356]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T07:00:24.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:23 smithi174 ceph-mon[35637]: pgmap v347: 97 pgs: 97 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-19T07:00:24.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:23 smithi103 ceph-mon[31158]: pgmap v347: 97 pgs: 97 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-19T07:00:24.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:23 smithi134 ceph-mon[35356]: pgmap v347: 97 pgs: 97 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-19T07:00:26.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:25 smithi174 ceph-mon[35637]: pgmap v348: 97 pgs: 97 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-19T07:00:26.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:25 smithi103 ceph-mon[31158]: pgmap v348: 97 pgs: 97 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-19T07:00:26.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:25 smithi134 ceph-mon[35356]: pgmap v348: 97 pgs: 97 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-09-19T07:00:28.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:27 smithi174 ceph-mon[35637]: pgmap v349: 97 pgs: 97 active+clean; 2.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-19T07:00:28.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:27 smithi103 ceph-mon[31158]: pgmap v349: 97 pgs: 97 active+clean; 2.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-19T07:00:28.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:27 smithi134 ceph-mon[35356]: pgmap v349: 97 pgs: 97 active+clean; 2.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-19T07:00:30.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:29 smithi174 ceph-mon[35637]: pgmap v350: 97 pgs: 97 active+clean; 2.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-19T07:00:30.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:29 smithi103 ceph-mon[31158]: pgmap v350: 97 pgs: 97 active+clean; 2.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-19T07:00:30.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:29 smithi134 ceph-mon[35356]: pgmap v350: 97 pgs: 97 active+clean; 2.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-19T07:00:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:31 smithi174 ceph-mon[35637]: pgmap v351: 97 pgs: 97 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-09-19T07:00:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:00:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:00:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:00:32.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:31 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:00:32.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:31 smithi103 ceph-mon[31158]: pgmap v351: 97 pgs: 97 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-09-19T07:00:32.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:00:32.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:00:32.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:00:32.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:31 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:00:32.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:31 smithi134 ceph-mon[35356]: pgmap v351: 97 pgs: 97 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-09-19T07:00:32.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:00:32.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:00:32.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:00:32.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:31 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:00:33.255 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:00:33.516 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:00:34.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.957+0000 7f7e8f60a640 1 -- 172.21.15.103:0/692978645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e88106f10 msgr2=0x7f7e8810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:34.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.957+0000 7f7e8f60a640 1 --2- 172.21.15.103:0/692978645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e88106f10 0x7f7e8810ea90 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009900 tx=0x7f7e8402a410 comp rx=0 tx=0).stop 2024-09-19T07:00:34.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.957+0000 7f7e8f60a640 1 -- 172.21.15.103:0/692978645 shutdown_connections 2024-09-19T07:00:34.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.957+0000 7f7e8f60a640 1 --2- 172.21.15.103:0/692978645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e88106f10 0x7f7e8810ea90 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.957+0000 7f7e8f60a640 1 --2- 172.21.15.103:0/692978645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e88102790 0x7f7e881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.957+0000 7f7e8f60a640 1 --2- 172.21.15.103:0/692978645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e88101de0 0x7f7e881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.957+0000 7f7e8f60a640 1 -- 172.21.15.103:0/692978645 >> 172.21.15.103:0/692978645 conn(0x7f7e880fd8c0 msgr2=0x7f7e880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:34.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.957+0000 7f7e8f60a640 1 -- 172.21.15.103:0/692978645 shutdown_connections 2024-09-19T07:00:34.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.957+0000 7f7e8f60a640 1 -- 172.21.15.103:0/692978645 wait complete. 2024-09-19T07:00:34.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.958+0000 7f7e8f60a640 1 Processor -- start 2024-09-19T07:00:34.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.958+0000 7f7e8f60a640 1 -- start start 2024-09-19T07:00:34.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.958+0000 7f7e8f60a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e88101de0 0x7f7e880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.958+0000 7f7e8f60a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e88102790 0x7f7e88074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.958+0000 7f7e8f60a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e88106f10 0x7f7e88074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.958+0000 7f7e8f60a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88111f70 con 0x7f7e88101de0 2024-09-19T07:00:34.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.958+0000 7f7e8f60a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88111df0 con 0x7f7e88106f10 2024-09-19T07:00:34.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.958+0000 7f7e8f60a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7e881120f0 con 0x7f7e88102790 2024-09-19T07:00:34.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.958+0000 7f7e8d3a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e88101de0 0x7f7e880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.958+0000 7f7e8d3a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e88101de0 0x7f7e880744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53184/0 (socket says 172.21.15.103:53184) 2024-09-19T07:00:34.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.958+0000 7f7e8d3a6640 1 -- 172.21.15.103:0/2233952726 learned_addr learned my addr 172.21.15.103:0/2233952726 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:34.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.959+0000 7f7e8d3a6640 1 -- 172.21.15.103:0/2233952726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e88102790 msgr2=0x7f7e88074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.959+0000 7f7e8d3a6640 1 --2- 172.21.15.103:0/2233952726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e88102790 0x7f7e88074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.959+0000 7f7e8dba7640 1 --2- 172.21.15.103:0/2233952726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e88106f10 0x7f7e88074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.959+0000 7f7e8d3a6640 1 -- 172.21.15.103:0/2233952726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e88106f10 msgr2=0x7f7e88074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.959+0000 7f7e8d3a6640 1 --2- 172.21.15.103:0/2233952726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e88106f10 0x7f7e88074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.959+0000 7f7e8d3a6640 1 -- 172.21.15.103:0/2233952726 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e881a6810 con 0x7f7e88101de0 2024-09-19T07:00:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.959+0000 7f7e8d3a6640 1 --2- 172.21.15.103:0/2233952726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e88101de0 0x7f7e880744f0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000eab0 tx=0x7f7e8000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.959+0000 7f7e7e7fc640 1 -- 172.21.15.103:0/2233952726 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8000ce40 con 0x7f7e88101de0 2024-09-19T07:00:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.959+0000 7f7e7e7fc640 1 -- 172.21.15.103:0/2233952726 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e80011950 con 0x7f7e88101de0 2024-09-19T07:00:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.959+0000 7f7e8f60a640 1 -- 172.21.15.103:0/2233952726 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e881a6b00 con 0x7f7e88101de0 2024-09-19T07:00:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.959+0000 7f7e7e7fc640 1 -- 172.21.15.103:0/2233952726 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80010ae0 con 0x7f7e88101de0 2024-09-19T07:00:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.959+0000 7f7e8f60a640 1 -- 172.21.15.103:0/2233952726 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e881a7040 con 0x7f7e88101de0 2024-09-19T07:00:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.962+0000 7f7e7e7fc640 1 -- 172.21.15.103:0/2233952726 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7e80011b00 con 0x7f7e88101de0 2024-09-19T07:00:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.962+0000 7f7e919e3740 1 Processor -- start 2024-09-19T07:00:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.962+0000 7f7e919e3740 1 -- start start 2024-09-19T07:00:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.962+0000 7f7e919e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea5a4a0 0x55a5fea5a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.962+0000 7f7e919e3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea5adc0 0x55a5fea65140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.962+0000 7f7e919e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea65680 0x55a5fea67a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.962+0000 7f7e919e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a5fe9c02e0 con 0x55a5fea65680 2024-09-19T07:00:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.962+0000 7f7e919e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a5fe967ce0 con 0x55a5fea5a4a0 2024-09-19T07:00:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.962+0000 7f7e919e3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a5fe9bc2a0 con 0x55a5fea5adc0 2024-09-19T07:00:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.962+0000 7f7e8cba5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea65680 0x55a5fea67a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.963+0000 7f7e8cba5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea65680 0x55a5fea67a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53186/0 (socket says 172.21.15.103:53186) 2024-09-19T07:00:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.963+0000 7f7e8cba5640 1 -- 172.21.15.103:0/3169295376 learned_addr learned my addr 172.21.15.103:0/3169295376 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:34.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.963+0000 7f7e8d3a6640 1 --2- 172.21.15.103:0/3169295376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea5adc0 0x55a5fea65140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.963+0000 7f7e8dba7640 1 --2- 172.21.15.103:0/3169295376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea5a4a0 0x55a5fea5a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.963+0000 7f7e8cba5640 1 -- 172.21.15.103:0/3169295376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea5adc0 msgr2=0x55a5fea65140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:34.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.963+0000 7f7e8cba5640 1 --2- 172.21.15.103:0/3169295376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea5adc0 0x55a5fea65140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.963+0000 7f7e7e7fc640 1 --2- 172.21.15.103:0/2233952726 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7e740a33b0 0x7f7e740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.963+0000 7f7e8cba5640 1 -- 172.21.15.103:0/3169295376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea5a4a0 msgr2=0x55a5fea5a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:34.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.963+0000 7f7e8cba5640 1 --2- 172.21.15.103:0/3169295376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea5a4a0 0x55a5fea5a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.963+0000 7f7e8cba5640 1 -- 172.21.15.103:0/3169295376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5fe94c440 con 0x55a5fea65680 2024-09-19T07:00:34.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.963+0000 7f7e8dba7640 1 --2- 172.21.15.103:0/2233952726 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7e740a33b0 0x7f7e740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.963+0000 7f7e7e7fc640 1 -- 172.21.15.103:0/2233952726 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f7e800d0520 con 0x7f7e88101de0 2024-09-19T07:00:34.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.963+0000 7f7e8cba5640 1 --2- 172.21.15.103:0/3169295376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea65680 0x55a5fea67a70 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800a9e0 tx=0x7f7e7800aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edef0086145e7eee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:34.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.964+0000 7f7e8dba7640 1 --2- 172.21.15.103:0/2233952726 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7e740a33b0 0x7f7e740a5870 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7e84005c40 tx=0x7f7e84009260 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:34.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.964+0000 7f7e737fe640 1 -- 172.21.15.103:0/3169295376 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800cdb0 con 0x55a5fea65680 2024-09-19T07:00:34.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.964+0000 7f7e737fe640 1 -- 172.21.15.103:0/3169295376 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e780035f0 con 0x55a5fea65680 2024-09-19T07:00:34.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.964+0000 7f7e737fe640 1 -- 172.21.15.103:0/3169295376 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800f690 con 0x55a5fea65680 2024-09-19T07:00:34.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.964+0000 7f7e919e3740 1 -- 172.21.15.103:0/3169295376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea65680 msgr2=0x55a5fea67a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:34.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.964+0000 7f7e919e3740 1 --2- 172.21.15.103:0/3169295376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea65680 0x55a5fea67a70 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800a9e0 tx=0x7f7e7800aea0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.964+0000 7f7e919e3740 1 -- 172.21.15.103:0/3169295376 shutdown_connections 2024-09-19T07:00:34.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.964+0000 7f7e919e3740 1 --2- 172.21.15.103:0/3169295376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea65680 0x55a5fea67a70 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.964+0000 7f7e919e3740 1 --2- 172.21.15.103:0/3169295376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea5adc0 0x55a5fea65140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.964+0000 7f7e919e3740 1 --2- 172.21.15.103:0/3169295376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea5a4a0 0x55a5fea5a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.964+0000 7f7e919e3740 1 -- 172.21.15.103:0/3169295376 >> 172.21.15.103:0/3169295376 conn(0x55a5fea56a30 msgr2=0x55a5fea56e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:34.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.965+0000 7f7e919e3740 1 -- 172.21.15.103:0/3169295376 shutdown_connections 2024-09-19T07:00:34.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.965+0000 7f7e919e3740 1 -- 172.21.15.103:0/3169295376 wait complete. 2024-09-19T07:00:34.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.965+0000 7f7e919e3740 1 -- start start 2024-09-19T07:00:34.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.965+0000 7f7e919e3740 1 Processor -- start 2024-09-19T07:00:34.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.965+0000 7f7e919e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea5a4a0 0x55a5fea64f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.965+0000 7f7e919e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea5adc0 0x55a5fea621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.965+0000 7f7e919e3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea65680 0x55a5fea626f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.965+0000 7f7e919e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a5fea68250 con 0x55a5fea5a4a0 2024-09-19T07:00:34.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.965+0000 7f7e919e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a5fe97b050 con 0x55a5fea5adc0 2024-09-19T07:00:34.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.965+0000 7f7e919e3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a5fea683d0 con 0x55a5fea65680 2024-09-19T07:00:34.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.965+0000 7f7e919e3740 10 client.0 _ll_register_callbacks cb 0x55a5fe9bb160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:34.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.965+0000 7f7e8d3a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea5a4a0 0x55a5fea64f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.032 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:00:34.032 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:00:34.032 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:00:34.032 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:00:34.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e8d3a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea5a4a0 0x55a5fea64f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53202/0 (socket says 172.21.15.103:53202) 2024-09-19T07:00:34.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e8d3a6640 1 -- 172.21.15.103:0/2534196168 learned_addr learned my addr 172.21.15.103:0/2534196168 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:34.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e8dba7640 1 --2- 172.21.15.103:0/2534196168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea65680 0x55a5fea626f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e8cba5640 1 --2- 172.21.15.103:0/2534196168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea5adc0 0x55a5fea621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e8d3a6640 1 -- 172.21.15.103:0/2534196168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea65680 msgr2=0x55a5fea626f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:34.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e8d3a6640 1 --2- 172.21.15.103:0/2534196168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea65680 0x55a5fea626f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e8d3a6640 1 -- 172.21.15.103:0/2534196168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea5adc0 msgr2=0x55a5fea621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:34.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e8d3a6640 1 --2- 172.21.15.103:0/2534196168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea5adc0 0x55a5fea621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e8d3a6640 1 -- 172.21.15.103:0/2534196168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5fe7313c0 con 0x55a5fea5a4a0 2024-09-19T07:00:34.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e8cba5640 1 --2- 172.21.15.103:0/2534196168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea5adc0 0x55a5fea621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:34.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e8d3a6640 1 --2- 172.21.15.103:0/2534196168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea5a4a0 0x55a5fea64f00 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f7e800cdcb0 tx=0x7f7e80098420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe9492711e0d7cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:34.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:34.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e70ff9640 1 -- 172.21.15.103:0/2534196168 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8009c5b0 con 0x55a5fea5a4a0 2024-09-19T07:00:34.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e70ff9640 1 -- 172.21.15.103:0/2534196168 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f7e8009ca40 con 0x55a5fea5a4a0 2024-09-19T07:00:34.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e919e3740 1 -- 172.21.15.103:0/2534196168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e780035f0 con 0x55a5fea5a4a0 2024-09-19T07:00:34.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e70ff9640 1 -- 172.21.15.103:0/2534196168 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e8009c140 con 0x55a5fea5a4a0 2024-09-19T07:00:34.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e919e3740 10 client.19266 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:34.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e919e3740 1 -- 172.21.15.103:0/2534196168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a5fea73bc0 con 0x55a5fea5a4a0 2024-09-19T07:00:34.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.966+0000 7f7e70ff9640 1 -- 172.21.15.103:0/2534196168 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e780035f0 con 0x55a5fea5a4a0 2024-09-19T07:00:34.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.967+0000 7f7e70ff9640 1 -- 172.21.15.103:0/2534196168 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7e8000f040 con 0x55a5fea5a4a0 2024-09-19T07:00:34.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.967+0000 7f7e919e3740 10 client.19266 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:34.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.967+0000 7f7e919e3740 10 client.19266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:34.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.967+0000 7f7e919e3740 20 client.19266 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:34.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.967+0000 7f7e919e3740 1 --2- 172.21.15.103:0/2534196168 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a5fea74fd0 0x55a5fea773c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.967+0000 7f7e919e3740 4 client.19266 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:00:34.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.967+0000 7f7e919e3740 1 -- 172.21.15.103:0/2534196168 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5fe9cd850 con 0x55a5fea74fd0 2024-09-19T07:00:34.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.967+0000 7f7e8cba5640 1 --2- 172.21.15.103:0/2534196168 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a5fea74fd0 0x55a5fea773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.968+0000 7f7e70ff9640 10 client.19266 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:34.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.968+0000 7f7e8cba5640 1 --2- 172.21.15.103:0/2534196168 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a5fea74fd0 0x55a5fea773c0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb765da1a22478d8 server_cookie=6cb3cbfe6d5dead0 in_seq=0 out_seq=0 2024-09-19T07:00:34.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.968+0000 7f7e70ff9640 1 -- 172.21.15.103:0/2534196168 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5fe9cd850 con 0x55a5fea74fd0 2024-09-19T07:00:34.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.968+0000 7f7e70ff9640 10 client.19266 handle_command_reply: tid=0 2024-09-19T07:00:34.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.968+0000 7f7e919e3740 1 client.19266 shutdown 2024-09-19T07:00:34.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.969+0000 7f7e919e3740 1 -- 172.21.15.103:0/2534196168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea5a4a0 msgr2=0x55a5fea64f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:34.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.969+0000 7f7e919e3740 1 --2- 172.21.15.103:0/2534196168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea5a4a0 0x55a5fea64f00 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f7e800cdcb0 tx=0x7f7e80098420 comp rx=0 tx=0).stop 2024-09-19T07:00:34.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.969+0000 7f7e919e3740 1 -- 172.21.15.103:0/2534196168 shutdown_connections 2024-09-19T07:00:34.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.969+0000 7f7e919e3740 1 --2- 172.21.15.103:0/2534196168 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a5fea74fd0 0x55a5fea773c0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.969+0000 7f7e919e3740 1 --2- 172.21.15.103:0/2534196168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea65680 0x55a5fea626f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.969+0000 7f7e919e3740 1 --2- 172.21.15.103:0/2534196168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea5adc0 0x55a5fea621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.969+0000 7f7e70ff9640 0 client.19266 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:34.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.969+0000 7f7e919e3740 1 --2- 172.21.15.103:0/2534196168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea5a4a0 0x55a5fea64f00 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.969+0000 7f7e919e3740 1 -- 172.21.15.103:0/2534196168 >> 172.21.15.103:0/2534196168 conn(0x55a5fea56a30 msgr2=0x55a5fea5c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:34.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.969+0000 7f7e919e3740 1 -- 172.21.15.103:0/2534196168 shutdown_connections 2024-09-19T07:00:34.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.969+0000 7f7e919e3740 1 -- 172.21.15.103:0/2534196168 wait complete. 2024-09-19T07:00:34.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.969+0000 7f7e919e3740 20 client.19266 trim_cache size 0 max 16384 2024-09-19T07:00:34.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.986+0000 7f7e919e3740 1 Processor -- start 2024-09-19T07:00:34.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.986+0000 7f7e919e3740 1 -- start start 2024-09-19T07:00:34.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.986+0000 7f7e919e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea5a4a0 0x55a5fea714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.986+0000 7f7e919e3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea5adc0 0x55a5fea7cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.986+0000 7f7e919e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea65680 0x55a5fea7d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.986+0000 7f7e919e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7e881afa10 con 0x55a5fea65680 2024-09-19T07:00:34.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.986+0000 7f7e919e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7e881b1550 con 0x55a5fea5a4a0 2024-09-19T07:00:34.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.986+0000 7f7e919e3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7e881b00e0 con 0x55a5fea5adc0 2024-09-19T07:00:34.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.987+0000 7f7e8dba7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea65680 0x55a5fea7d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.987+0000 7f7e8dba7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea65680 0x55a5fea7d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53206/0 (socket says 172.21.15.103:53206) 2024-09-19T07:00:34.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.987+0000 7f7e8dba7640 1 -- 172.21.15.103:0/963326896 learned_addr learned my addr 172.21.15.103:0/963326896 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:34.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.987+0000 7f7e8d3a6640 1 --2- 172.21.15.103:0/963326896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea5a4a0 0x55a5fea714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.987+0000 7f7e8cba5640 1 --2- 172.21.15.103:0/963326896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea5adc0 0x55a5fea7cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.987+0000 7f7e8dba7640 1 -- 172.21.15.103:0/963326896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea5adc0 msgr2=0x55a5fea7cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:34.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.987+0000 7f7e8dba7640 1 --2- 172.21.15.103:0/963326896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea5adc0 0x55a5fea7cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.987+0000 7f7e8dba7640 1 -- 172.21.15.103:0/963326896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea5a4a0 msgr2=0x55a5fea714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:34.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.987+0000 7f7e8dba7640 1 --2- 172.21.15.103:0/963326896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea5a4a0 0x55a5fea714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.987+0000 7f7e8dba7640 1 -- 172.21.15.103:0/963326896 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5fea65270 con 0x55a5fea65680 2024-09-19T07:00:34.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.987+0000 7f7e8dba7640 1 --2- 172.21.15.103:0/963326896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea65680 0x55a5fea7d1d0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009900 tx=0x7f7e8403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c0b6aac731681e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:34.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.987+0000 7f7e72ffd640 1 -- 172.21.15.103:0/963326896 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8404e070 con 0x55a5fea65680 2024-09-19T07:00:34.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.987+0000 7f7e72ffd640 1 -- 172.21.15.103:0/963326896 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e84003cd0 con 0x55a5fea65680 2024-09-19T07:00:34.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.987+0000 7f7e72ffd640 1 -- 172.21.15.103:0/963326896 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84035040 con 0x55a5fea65680 2024-09-19T07:00:34.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.988+0000 7f7e919e3740 1 -- 172.21.15.103:0/963326896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea65680 msgr2=0x55a5fea7d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:34.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.988+0000 7f7e919e3740 1 --2- 172.21.15.103:0/963326896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea65680 0x55a5fea7d1d0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009900 tx=0x7f7e8403d000 comp rx=0 tx=0).stop 2024-09-19T07:00:34.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.988+0000 7f7e919e3740 1 -- 172.21.15.103:0/963326896 shutdown_connections 2024-09-19T07:00:34.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.988+0000 7f7e919e3740 1 --2- 172.21.15.103:0/963326896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea65680 0x55a5fea7d1d0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.988+0000 7f7e919e3740 1 --2- 172.21.15.103:0/963326896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea5adc0 0x55a5fea7cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.988+0000 7f7e919e3740 1 --2- 172.21.15.103:0/963326896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea5a4a0 0x55a5fea714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.988+0000 7f7e919e3740 1 -- 172.21.15.103:0/963326896 >> 172.21.15.103:0/963326896 conn(0x55a5fea56a30 msgr2=0x55a5fea60a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:34.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.988+0000 7f7e919e3740 1 -- 172.21.15.103:0/963326896 shutdown_connections 2024-09-19T07:00:34.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.988+0000 7f7e919e3740 1 -- 172.21.15.103:0/963326896 wait complete. 2024-09-19T07:00:34.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.988+0000 7f7e919e3740 1 -- start start 2024-09-19T07:00:34.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e919e3740 1 Processor -- start 2024-09-19T07:00:34.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e919e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea5a4a0 0x55a5fea693a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e919e3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea5adc0 0x55a5fea698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e919e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea65680 0x55a5fea69e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e919e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7e881b0e80 con 0x55a5fea5a4a0 2024-09-19T07:00:34.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e919e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7e881b07b0 con 0x55a5fea65680 2024-09-19T07:00:34.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e919e3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7e881ada10 con 0x55a5fea5adc0 2024-09-19T07:00:34.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e919e3740 10 client.0 _ll_register_callbacks cb 0x7f7e800103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:34.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e8d3a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea5a4a0 0x55a5fea693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e8d3a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea5a4a0 0x55a5fea693a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53222/0 (socket says 172.21.15.103:53222) 2024-09-19T07:00:34.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e8d3a6640 1 -- 172.21.15.103:0/138228326 learned_addr learned my addr 172.21.15.103:0/138228326 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:34.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e8cba5640 1 --2- 172.21.15.103:0/138228326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea5adc0 0x55a5fea698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e8dba7640 1 --2- 172.21.15.103:0/138228326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea65680 0x55a5fea69e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e8d3a6640 1 -- 172.21.15.103:0/138228326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea5adc0 msgr2=0x55a5fea698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:34.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e8d3a6640 1 --2- 172.21.15.103:0/138228326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea5adc0 0x55a5fea698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e8d3a6640 1 -- 172.21.15.103:0/138228326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea65680 msgr2=0x55a5fea69e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:34.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e8d3a6640 1 --2- 172.21.15.103:0/138228326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea65680 0x55a5fea69e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e8d3a6640 1 -- 172.21.15.103:0/138228326 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5feadec90 con 0x55a5fea5a4a0 2024-09-19T07:00:34.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.989+0000 7f7e8dba7640 1 --2- 172.21.15.103:0/138228326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea65680 0x55a5fea69e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:34.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.990+0000 7f7e8d3a6640 1 --2- 172.21.15.103:0/138228326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea5a4a0 0x55a5fea693a0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f7e800991b0 tx=0x7f7e8009be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1564bffac01e3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:34.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.990+0000 7f7e70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:34.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.990+0000 7f7e70ff9640 1 -- 172.21.15.103:0/138228326 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8009c320 con 0x55a5fea5a4a0 2024-09-19T07:00:34.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.990+0000 7f7e70ff9640 1 -- 172.21.15.103:0/138228326 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f7e800d0250 con 0x55a5fea5a4a0 2024-09-19T07:00:34.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.990+0000 7f7e919e3740 1 -- 172.21.15.103:0/138228326 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e84003cd0 con 0x55a5fea5a4a0 2024-09-19T07:00:34.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.990+0000 7f7e70ff9640 1 -- 172.21.15.103:0/138228326 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e8009c5b0 con 0x55a5fea5a4a0 2024-09-19T07:00:34.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.990+0000 7f7e919e3740 10 client.19272 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:34.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.990+0000 7f7e919e3740 1 -- 172.21.15.103:0/138228326 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a5feadff60 con 0x55a5fea5a4a0 2024-09-19T07:00:34.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.990+0000 7f7e70ff9640 1 -- 172.21.15.103:0/138228326 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e84003cd0 con 0x55a5fea5a4a0 2024-09-19T07:00:34.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.990+0000 7f7e70ff9640 1 -- 172.21.15.103:0/138228326 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7e8009db10 con 0x55a5fea5a4a0 2024-09-19T07:00:34.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.990+0000 7f7e919e3740 10 client.19272 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:34.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.990+0000 7f7e919e3740 10 client.19272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:34.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.990+0000 7f7e919e3740 20 client.19272 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:34.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.990+0000 7f7e919e3740 1 --2- 172.21.15.103:0/138228326 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a5feae0540 0x55a5feae2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:34.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.990+0000 7f7e919e3740 4 client.19272 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:00:34.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.990+0000 7f7e919e3740 1 -- 172.21.15.103:0/138228326 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a5fe9c7d60 con 0x55a5feae0540 2024-09-19T07:00:34.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.991+0000 7f7e8cba5640 1 --2- 172.21.15.103:0/138228326 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a5feae0540 0x55a5feae2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:34.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.991+0000 7f7e8cba5640 1 --2- 172.21.15.103:0/138228326 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a5feae0540 0x55a5feae2930 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f389a40754bd0c9d server_cookie=c43d0c87f7129118 in_seq=0 out_seq=0 2024-09-19T07:00:34.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.991+0000 7f7e70ff9640 10 client.19272 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:34.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.991+0000 7f7e70ff9640 1 -- 172.21.15.103:0/138228326 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a5fe9c7d60 con 0x55a5feae0540 2024-09-19T07:00:34.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.991+0000 7f7e70ff9640 10 client.19272 handle_command_reply: tid=0 2024-09-19T07:00:34.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.991+0000 7f7e919e3740 1 client.19272 shutdown 2024-09-19T07:00:34.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.991+0000 7f7e919e3740 1 -- 172.21.15.103:0/138228326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea5a4a0 msgr2=0x55a5fea693a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:34.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.991+0000 7f7e919e3740 1 --2- 172.21.15.103:0/138228326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea5a4a0 0x55a5fea693a0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f7e800991b0 tx=0x7f7e8009be40 comp rx=0 tx=0).stop 2024-09-19T07:00:34.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.991+0000 7f7e919e3740 1 -- 172.21.15.103:0/138228326 shutdown_connections 2024-09-19T07:00:34.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.991+0000 7f7e919e3740 1 --2- 172.21.15.103:0/138228326 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55a5feae0540 0x55a5feae2930 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.991+0000 7f7e919e3740 1 --2- 172.21.15.103:0/138228326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5fea65680 0x55a5fea69e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.991+0000 7f7e70ff9640 0 client.19272 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:34.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.991+0000 7f7e919e3740 1 --2- 172.21.15.103:0/138228326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5fea5adc0 0x55a5fea698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.991+0000 7f7e919e3740 1 --2- 172.21.15.103:0/138228326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5fea5a4a0 0x55a5fea693a0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.991+0000 7f7e919e3740 1 -- 172.21.15.103:0/138228326 >> 172.21.15.103:0/138228326 conn(0x55a5fea56a30 msgr2=0x55a5fea55350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:34.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.991+0000 7f7e919e3740 1 -- 172.21.15.103:0/138228326 shutdown_connections 2024-09-19T07:00:34.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.992+0000 7f7e919e3740 1 -- 172.21.15.103:0/138228326 wait complete. 2024-09-19T07:00:34.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.992+0000 7f7e919e3740 20 client.19272 trim_cache size 0 max 16384 2024-09-19T07:00:34.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.992+0000 7f7e8f60a640 1 -- 172.21.15.103:0/2233952726 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7e740a33b0 msgr2=0x7f7e740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:34.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.992+0000 7f7e8f60a640 1 --2- 172.21.15.103:0/2233952726 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7e740a33b0 0x7f7e740a5870 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7e84005c40 tx=0x7f7e84009260 comp rx=0 tx=0).stop 2024-09-19T07:00:34.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.992+0000 7f7e8f60a640 1 -- 172.21.15.103:0/2233952726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e88101de0 msgr2=0x7f7e880744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:34.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.992+0000 7f7e8f60a640 1 --2- 172.21.15.103:0/2233952726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e88101de0 0x7f7e880744f0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000eab0 tx=0x7f7e8000ef70 comp rx=0 tx=0).stop 2024-09-19T07:00:34.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.992+0000 7f7e8f60a640 1 -- 172.21.15.103:0/2233952726 shutdown_connections 2024-09-19T07:00:34.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.992+0000 7f7e8f60a640 1 --2- 172.21.15.103:0/2233952726 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7e740a33b0 0x7f7e740a5870 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.992+0000 7f7e8f60a640 1 --2- 172.21.15.103:0/2233952726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e88106f10 0x7f7e88074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.992+0000 7f7e8f60a640 1 --2- 172.21.15.103:0/2233952726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e88102790 0x7f7e88074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.992+0000 7f7e8f60a640 1 --2- 172.21.15.103:0/2233952726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e88101de0 0x7f7e880744f0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:34.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.992+0000 7f7e8f60a640 1 -- 172.21.15.103:0/2233952726 >> 172.21.15.103:0/2233952726 conn(0x7f7e880fd8c0 msgr2=0x7f7e880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:34.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.992+0000 7f7e8f60a640 1 -- 172.21.15.103:0/2233952726 shutdown_connections 2024-09-19T07:00:34.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:33.992+0000 7f7e8f60a640 1 -- 172.21.15.103:0/2233952726 wait complete. 2024-09-19T07:00:34.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:33 smithi174 ceph-mon[35637]: pgmap v352: 97 pgs: 97 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-19T07:00:34.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:33 smithi103 ceph-mon[31158]: pgmap v352: 97 pgs: 97 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-19T07:00:34.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:33 smithi134 ceph-mon[35356]: pgmap v352: 97 pgs: 97 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-19T07:00:34.862 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:00:34.862 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:00:35.111 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:00:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.316+0000 7f5b550af640 1 -- 172.21.15.103:0/2145900897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b501009b0 msgr2=0x7f5b50100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.316+0000 7f5b550af640 1 --2- 172.21.15.103:0/2145900897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b501009b0 0x7f5b50100e10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009900 tx=0x7f5b4402a410 comp rx=0 tx=0).stop 2024-09-19T07:00:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.316+0000 7f5b550af640 1 -- 172.21.15.103:0/2145900897 shutdown_connections 2024-09-19T07:00:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.316+0000 7f5b550af640 1 --2- 172.21.15.103:0/2145900897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b50101350 0x7f5b5010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.316+0000 7f5b550af640 1 --2- 172.21.15.103:0/2145900897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b501009b0 0x7f5b50100e10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.316+0000 7f5b550af640 1 --2- 172.21.15.103:0/2145900897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b501069c0 0x7f5b50106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.316+0000 7f5b550af640 1 -- 172.21.15.103:0/2145900897 >> 172.21.15.103:0/2145900897 conn(0x7f5b50077990 msgr2=0x7f5b500fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.316+0000 7f5b550af640 1 -- 172.21.15.103:0/2145900897 shutdown_connections 2024-09-19T07:00:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.316+0000 7f5b550af640 1 -- 172.21.15.103:0/2145900897 wait complete. 2024-09-19T07:00:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.317+0000 7f5b550af640 1 Processor -- start 2024-09-19T07:00:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.317+0000 7f5b550af640 1 -- start start 2024-09-19T07:00:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.317+0000 7f5b550af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b501009b0 0x7f5b500fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.317+0000 7f5b550af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b50101350 0x7f5b500fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.317+0000 7f5b550af640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b501069c0 0x7f5b500fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.317+0000 7f5b550af640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b50110e00 con 0x7f5b50101350 2024-09-19T07:00:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.317+0000 7f5b550af640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5b50110c80 con 0x7f5b501009b0 2024-09-19T07:00:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.317+0000 7f5b550af640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b50110f80 con 0x7f5b501069c0 2024-09-19T07:00:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.317+0000 7f5b4e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b50101350 0x7f5b500fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.317+0000 7f5b4e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b50101350 0x7f5b500fd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53226/0 (socket says 172.21.15.103:53226) 2024-09-19T07:00:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.318+0000 7f5b4e59c640 1 -- 172.21.15.103:0/2357553294 learned_addr learned my addr 172.21.15.103:0/2357553294 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.318+0000 7f5b4ed9d640 1 --2- 172.21.15.103:0/2357553294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b501009b0 0x7f5b500fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.318+0000 7f5b4f59e640 1 --2- 172.21.15.103:0/2357553294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b501069c0 0x7f5b500fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.318+0000 7f5b4ed9d640 1 -- 172.21.15.103:0/2357553294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b501069c0 msgr2=0x7f5b500fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.318+0000 7f5b4ed9d640 1 --2- 172.21.15.103:0/2357553294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b501069c0 0x7f5b500fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.318+0000 7f5b4ed9d640 1 -- 172.21.15.103:0/2357553294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b50101350 msgr2=0x7f5b500fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.318+0000 7f5b4ed9d640 1 --2- 172.21.15.103:0/2357553294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b50101350 0x7f5b500fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.318+0000 7f5b4ed9d640 1 -- 172.21.15.103:0/2357553294 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b501a6820 con 0x7f5b501009b0 2024-09-19T07:00:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.318+0000 7f5b4e59c640 1 --2- 172.21.15.103:0/2357553294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b50101350 0x7f5b500fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:00:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.318+0000 7f5b4ed9d640 1 --2- 172.21.15.103:0/2357553294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b501009b0 0x7f5b500fcde0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000e9f0 tx=0x7f5b4000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.318+0000 7f5b37fff640 1 -- 172.21.15.103:0/2357553294 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4000ccc0 con 0x7f5b501009b0 2024-09-19T07:00:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.318+0000 7f5b550af640 1 -- 172.21.15.103:0/2357553294 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b501a6b10 con 0x7f5b501009b0 2024-09-19T07:00:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.318+0000 7f5b37fff640 1 -- 172.21.15.103:0/2357553294 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b40005020 con 0x7f5b501009b0 2024-09-19T07:00:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.318+0000 7f5b37fff640 1 -- 172.21.15.103:0/2357553294 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40004a00 con 0x7f5b501009b0 2024-09-19T07:00:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.318+0000 7f5b550af640 1 -- 172.21.15.103:0/2357553294 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b501a7010 con 0x7f5b501009b0 2024-09-19T07:00:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.321+0000 7f5b37fff640 1 -- 172.21.15.103:0/2357553294 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f5b400051d0 con 0x7f5b501009b0 2024-09-19T07:00:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.321+0000 7f5b57e42740 1 Processor -- start 2024-09-19T07:00:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.321+0000 7f5b57e42740 1 -- start start 2024-09-19T07:00:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.321+0000 7f5b57e42740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd870af0 0x556efd870ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.321+0000 7f5b57e42740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd871410 0x556efd87b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.321+0000 7f5b57e42740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd87bcb0 0x556efd87e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.321+0000 7f5b57e42740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556efd7d52f0 con 0x556efd87bcb0 2024-09-19T07:00:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.321+0000 7f5b57e42740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556efd77d270 con 0x556efd871410 2024-09-19T07:00:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.321+0000 7f5b57e42740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556efd7d12b0 con 0x556efd870af0 2024-09-19T07:00:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.321+0000 7f5b4e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd87bcb0 0x556efd87e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.321+0000 7f5b4e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd87bcb0 0x556efd87e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53240/0 (socket says 172.21.15.103:53240) 2024-09-19T07:00:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.321+0000 7f5b4e59c640 1 -- 172.21.15.103:0/1152341205 learned_addr learned my addr 172.21.15.103:0/1152341205 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.321+0000 7f5b37fff640 1 --2- 172.21.15.103:0/2357553294 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5b380a3460 0x7f5b380a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.321+0000 7f5b4ed9d640 1 --2- 172.21.15.103:0/1152341205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd871410 0x556efd87b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.322+0000 7f5b4f59e640 1 --2- 172.21.15.103:0/1152341205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd870af0 0x556efd870ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.322+0000 7f5b4f59e640 1 --2- 172.21.15.103:0/2357553294 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5b380a3460 0x7f5b380a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.322+0000 7f5b4e59c640 1 -- 172.21.15.103:0/1152341205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd870af0 msgr2=0x556efd870ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.322+0000 7f5b4e59c640 1 --2- 172.21.15.103:0/1152341205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd870af0 0x556efd870ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.322+0000 7f5b4e59c640 1 -- 172.21.15.103:0/1152341205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd871410 msgr2=0x556efd87b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.322+0000 7f5b4e59c640 1 --2- 172.21.15.103:0/1152341205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd871410 0x556efd87b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.322+0000 7f5b4e59c640 1 -- 172.21.15.103:0/1152341205 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556efd77c340 con 0x556efd87bcb0 2024-09-19T07:00:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.322+0000 7f5b37fff640 1 -- 172.21.15.103:0/2357553294 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f5b400cfc10 con 0x7f5b501009b0 2024-09-19T07:00:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.322+0000 7f5b4e59c640 1 --2- 172.21.15.103:0/1152341205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd87bcb0 0x556efd87e0a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f5b44000c00 tx=0x7f5b44005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b84e897f7ae288 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.322+0000 7f5b4f59e640 1 --2- 172.21.15.103:0/2357553294 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5b380a3460 0x7f5b380a5920 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5b480099a0 tx=0x7f5b48009500 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.322+0000 7f5b357fa640 1 -- 172.21.15.103:0/1152341205 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44040070 con 0x556efd87bcb0 2024-09-19T07:00:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.322+0000 7f5b357fa640 1 -- 172.21.15.103:0/1152341205 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b44003cd0 con 0x556efd87bcb0 2024-09-19T07:00:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.322+0000 7f5b357fa640 1 -- 172.21.15.103:0/1152341205 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44035040 con 0x556efd87bcb0 2024-09-19T07:00:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.323+0000 7f5b57e42740 1 -- 172.21.15.103:0/1152341205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd87bcb0 msgr2=0x556efd87e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.323+0000 7f5b57e42740 1 --2- 172.21.15.103:0/1152341205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd87bcb0 0x556efd87e0a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f5b44000c00 tx=0x7f5b44005e80 comp rx=0 tx=0).stop 2024-09-19T07:00:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.323+0000 7f5b57e42740 1 -- 172.21.15.103:0/1152341205 shutdown_connections 2024-09-19T07:00:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.323+0000 7f5b57e42740 1 --2- 172.21.15.103:0/1152341205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd87bcb0 0x556efd87e0a0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.323+0000 7f5b57e42740 1 --2- 172.21.15.103:0/1152341205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd871410 0x556efd87b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.323+0000 7f5b57e42740 1 --2- 172.21.15.103:0/1152341205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd870af0 0x556efd870ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.323+0000 7f5b57e42740 1 -- 172.21.15.103:0/1152341205 >> 172.21.15.103:0/1152341205 conn(0x556efd86d060 msgr2=0x556efd86d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.323+0000 7f5b57e42740 1 -- 172.21.15.103:0/1152341205 shutdown_connections 2024-09-19T07:00:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.323+0000 7f5b57e42740 1 -- 172.21.15.103:0/1152341205 wait complete. 2024-09-19T07:00:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.323+0000 7f5b57e42740 1 -- start start 2024-09-19T07:00:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.323+0000 7f5b57e42740 1 Processor -- start 2024-09-19T07:00:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b57e42740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd870af0 0x556efd878b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b57e42740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd871410 0x556efd8790b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b57e42740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd87bcb0 0x556efd8795f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b57e42740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556efd87e880 con 0x556efd87bcb0 2024-09-19T07:00:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b57e42740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556efd75e290 con 0x556efd870af0 2024-09-19T07:00:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b57e42740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556efd87ea00 con 0x556efd871410 2024-09-19T07:00:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b57e42740 10 client.0 _ll_register_callbacks cb 0x556efd78f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b4f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd87bcb0 0x556efd8795f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b4f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd87bcb0 0x556efd8795f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53252/0 (socket says 172.21.15.103:53252) 2024-09-19T07:00:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b4f59e640 1 -- 172.21.15.103:0/2466509535 learned_addr learned my addr 172.21.15.103:0/2466509535 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b4ed9d640 1 --2- 172.21.15.103:0/2466509535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd870af0 0x556efd878b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b4e59c640 1 --2- 172.21.15.103:0/2466509535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd871410 0x556efd8790b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b4f59e640 1 -- 172.21.15.103:0/2466509535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd871410 msgr2=0x556efd8790b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b4f59e640 1 --2- 172.21.15.103:0/2466509535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd871410 0x556efd8790b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b4f59e640 1 -- 172.21.15.103:0/2466509535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd870af0 msgr2=0x556efd878b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b4f59e640 1 --2- 172.21.15.103:0/2466509535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd870af0 0x556efd878b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b4f59e640 1 -- 172.21.15.103:0/2466509535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556efd4e8210 con 0x556efd87bcb0 2024-09-19T07:00:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b4f59e640 1 --2- 172.21.15.103:0/2466509535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd87bcb0 0x556efd8795f0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f5b48011da0 tx=0x7f5b48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4381c7d4efad31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b32ff5640 1 -- 172.21.15.103:0/2466509535 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4802c060 con 0x556efd87bcb0 2024-09-19T07:00:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b32ff5640 1 -- 172.21.15.103:0/2466509535 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f5b4801e930 con 0x556efd87bcb0 2024-09-19T07:00:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.324+0000 7f5b57e42740 1 -- 172.21.15.103:0/2466509535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b44003cd0 con 0x556efd87bcb0 2024-09-19T07:00:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.325+0000 7f5b32ff5640 1 -- 172.21.15.103:0/2466509535 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5b4801eb30 con 0x556efd87bcb0 2024-09-19T07:00:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.325+0000 7f5b57e42740 10 client.19284 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.325+0000 7f5b57e42740 1 -- 172.21.15.103:0/2466509535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556efd5b4620 con 0x556efd87bcb0 2024-09-19T07:00:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.325+0000 7f5b32ff5640 1 -- 172.21.15.103:0/2466509535 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b44003cd0 con 0x556efd87bcb0 2024-09-19T07:00:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.325+0000 7f5b32ff5640 1 -- 172.21.15.103:0/2466509535 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5b48027cd0 con 0x556efd87bcb0 2024-09-19T07:00:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.325+0000 7f5b57e42740 10 client.19284 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.325+0000 7f5b57e42740 10 client.19284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.325+0000 7f5b57e42740 20 client.19284 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.325+0000 7f5b57e42740 1 --2- 172.21.15.103:0/2466509535 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556efd88bc30 0x556efd88e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.325+0000 7f5b57e42740 4 client.19284 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:00:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.325+0000 7f5b57e42740 1 -- 172.21.15.103:0/2466509535 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556efd7e2a20 con 0x556efd88bc30 2024-09-19T07:00:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.325+0000 7f5b4ed9d640 1 --2- 172.21.15.103:0/2466509535 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556efd88bc30 0x556efd88e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.326+0000 7f5b4ed9d640 1 --2- 172.21.15.103:0/2466509535 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556efd88bc30 0x556efd88e040 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbe56cc73700e6a server_cookie=c6752ee9d4e4680d in_seq=0 out_seq=0 2024-09-19T07:00:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.326+0000 7f5b32ff5640 10 client.19284 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.326+0000 7f5b32ff5640 1 -- 172.21.15.103:0/2466509535 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556efd7e2a20 con 0x556efd88bc30 2024-09-19T07:00:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.326+0000 7f5b32ff5640 10 client.19284 handle_command_reply: tid=0 2024-09-19T07:00:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.326+0000 7f5b57e42740 1 client.19284 shutdown 2024-09-19T07:00:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.326+0000 7f5b57e42740 1 -- 172.21.15.103:0/2466509535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd87bcb0 msgr2=0x556efd8795f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.326+0000 7f5b57e42740 1 --2- 172.21.15.103:0/2466509535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd87bcb0 0x556efd8795f0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f5b48011da0 tx=0x7f5b48004a40 comp rx=0 tx=0).stop 2024-09-19T07:00:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.326+0000 7f5b57e42740 1 -- 172.21.15.103:0/2466509535 shutdown_connections 2024-09-19T07:00:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.326+0000 7f5b57e42740 1 --2- 172.21.15.103:0/2466509535 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556efd88bc30 0x556efd88e040 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.326+0000 7f5b57e42740 1 --2- 172.21.15.103:0/2466509535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd87bcb0 0x556efd8795f0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.326+0000 7f5b32ff5640 0 client.19284 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.326+0000 7f5b57e42740 1 --2- 172.21.15.103:0/2466509535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd871410 0x556efd8790b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.326+0000 7f5b57e42740 1 --2- 172.21.15.103:0/2466509535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd870af0 0x556efd878b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.326+0000 7f5b57e42740 1 -- 172.21.15.103:0/2466509535 >> 172.21.15.103:0/2466509535 conn(0x556efd86d060 msgr2=0x556efd872d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.327+0000 7f5b57e42740 1 -- 172.21.15.103:0/2466509535 shutdown_connections 2024-09-19T07:00:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.327+0000 7f5b57e42740 1 -- 172.21.15.103:0/2466509535 wait complete. 2024-09-19T07:00:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.327+0000 7f5b57e42740 20 client.19284 trim_cache size 0 max 16384 2024-09-19T07:00:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.341+0000 7f5b57e42740 1 Processor -- start 2024-09-19T07:00:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.341+0000 7f5b57e42740 1 -- start start 2024-09-19T07:00:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.341+0000 7f5b57e42740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd870af0 0x556efd879420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.341+0000 7f5b57e42740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd871410 0x556efd893950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.341+0000 7f5b57e42740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd87bcb0 0x556efd893e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.341+0000 7f5b57e42740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b501af9a0 con 0x556efd871410 2024-09-19T07:00:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.341+0000 7f5b57e42740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5b501b14e0 con 0x556efd870af0 2024-09-19T07:00:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.341+0000 7f5b57e42740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b501b0070 con 0x556efd87bcb0 2024-09-19T07:00:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.341+0000 7f5b4e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd871410 0x556efd893950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.341+0000 7f5b4e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd871410 0x556efd893950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53268/0 (socket says 172.21.15.103:53268) 2024-09-19T07:00:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.341+0000 7f5b4e59c640 1 -- 172.21.15.103:0/718855899 learned_addr learned my addr 172.21.15.103:0/718855899 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.342+0000 7f5b4f59e640 1 --2- 172.21.15.103:0/718855899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd87bcb0 0x556efd893e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.342+0000 7f5b4e59c640 1 -- 172.21.15.103:0/718855899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd87bcb0 msgr2=0x556efd893e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.342+0000 7f5b4e59c640 1 --2- 172.21.15.103:0/718855899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd87bcb0 0x556efd893e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.342+0000 7f5b4e59c640 1 -- 172.21.15.103:0/718855899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd870af0 msgr2=0x556efd879420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.342+0000 7f5b4ed9d640 1 --2- 172.21.15.103:0/718855899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd870af0 0x556efd879420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.342+0000 7f5b4e59c640 1 --2- 172.21.15.103:0/718855899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd870af0 0x556efd879420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.342+0000 7f5b4e59c640 1 -- 172.21.15.103:0/718855899 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556efd871240 con 0x556efd871410 2024-09-19T07:00:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.342+0000 7f5b4f59e640 1 --2- 172.21.15.103:0/718855899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd87bcb0 0x556efd893e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.342+0000 7f5b4e59c640 1 --2- 172.21.15.103:0/718855899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd871410 0x556efd893950 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5b44005ab0 tx=0x7f5b44005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c6395b3aa6bef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.342+0000 7f5b34ff9640 1 -- 172.21.15.103:0/718855899 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44035050 con 0x556efd871410 2024-09-19T07:00:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.342+0000 7f5b34ff9640 1 -- 172.21.15.103:0/718855899 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b440317a0 con 0x556efd871410 2024-09-19T07:00:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.342+0000 7f5b34ff9640 1 -- 172.21.15.103:0/718855899 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44040040 con 0x556efd871410 2024-09-19T07:00:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 -- 172.21.15.103:0/718855899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd871410 msgr2=0x556efd893950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 --2- 172.21.15.103:0/718855899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd871410 0x556efd893950 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5b44005ab0 tx=0x7f5b44005fd0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 -- 172.21.15.103:0/718855899 shutdown_connections 2024-09-19T07:00:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 --2- 172.21.15.103:0/718855899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd87bcb0 0x556efd893e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 --2- 172.21.15.103:0/718855899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd871410 0x556efd893950 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 --2- 172.21.15.103:0/718855899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd870af0 0x556efd879420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 -- 172.21.15.103:0/718855899 >> 172.21.15.103:0/718855899 conn(0x556efd86d060 msgr2=0x556efd87abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 -- 172.21.15.103:0/718855899 shutdown_connections 2024-09-19T07:00:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 -- 172.21.15.103:0/718855899 wait complete. 2024-09-19T07:00:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 -- start start 2024-09-19T07:00:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 Processor -- start 2024-09-19T07:00:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd870af0 0x556efd884200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd871410 0x556efd8756e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd87bcb0 0x556efd875c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b501b0e10 con 0x556efd870af0 2024-09-19T07:00:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5b501b0740 con 0x556efd87bcb0 2024-09-19T07:00:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b501ad9a0 con 0x556efd871410 2024-09-19T07:00:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b57e42740 10 client.0 _ll_register_callbacks cb 0x7f5b500fee70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.343+0000 7f5b4ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd870af0 0x556efd884200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b4ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd870af0 0x556efd884200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53278/0 (socket says 172.21.15.103:53278) 2024-09-19T07:00:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b4ed9d640 1 -- 172.21.15.103:0/734733333 learned_addr learned my addr 172.21.15.103:0/734733333 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b4f59e640 1 --2- 172.21.15.103:0/734733333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd87bcb0 0x556efd875c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b4e59c640 1 --2- 172.21.15.103:0/734733333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd871410 0x556efd8756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b4ed9d640 1 -- 172.21.15.103:0/734733333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd871410 msgr2=0x556efd8756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b4ed9d640 1 --2- 172.21.15.103:0/734733333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd871410 0x556efd8756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b4ed9d640 1 -- 172.21.15.103:0/734733333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd87bcb0 msgr2=0x556efd875c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b4ed9d640 1 --2- 172.21.15.103:0/734733333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd87bcb0 0x556efd875c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b4ed9d640 1 -- 172.21.15.103:0/734733333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556efd876260 con 0x556efd870af0 2024-09-19T07:00:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b4e59c640 1 --2- 172.21.15.103:0/734733333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd871410 0x556efd8756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b4ed9d640 1 --2- 172.21.15.103:0/734733333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd870af0 0x556efd884200 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5b4009bbd0 tx=0x7f5b40098780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadedcf2cf7fa83d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b32ff5640 1 -- 172.21.15.103:0/734733333 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4001c040 con 0x556efd870af0 2024-09-19T07:00:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b32ff5640 1 -- 172.21.15.103:0/734733333 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f5b40086040 con 0x556efd870af0 2024-09-19T07:00:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b57e42740 1 -- 172.21.15.103:0/734733333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b440317a0 con 0x556efd870af0 2024-09-19T07:00:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b32ff5640 1 -- 172.21.15.103:0/734733333 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5b4009c5d0 con 0x556efd870af0 2024-09-19T07:00:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b57e42740 10 client.19290 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b57e42740 1 -- 172.21.15.103:0/734733333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556efd8f6770 con 0x556efd870af0 2024-09-19T07:00:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.344+0000 7f5b32ff5640 1 -- 172.21.15.103:0/734733333 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b440317a0 con 0x556efd870af0 2024-09-19T07:00:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.345+0000 7f5b32ff5640 1 -- 172.21.15.103:0/734733333 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5b400cd650 con 0x556efd870af0 2024-09-19T07:00:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.345+0000 7f5b57e42740 10 client.19290 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.345+0000 7f5b57e42740 10 client.19290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.345+0000 7f5b57e42740 20 client.19290 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.345+0000 7f5b57e42740 1 --2- 172.21.15.103:0/734733333 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556efd8f6e40 0x556efd8f9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.345+0000 7f5b57e42740 4 client.19290 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:00:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.345+0000 7f5b57e42740 1 -- 172.21.15.103:0/734733333 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556efd7dcd70 con 0x556efd8f6e40 2024-09-19T07:00:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.345+0000 7f5b4e59c640 1 --2- 172.21.15.103:0/734733333 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556efd8f6e40 0x556efd8f9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.345+0000 7f5b4e59c640 1 --2- 172.21.15.103:0/734733333 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556efd8f6e40 0x556efd8f9260 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4c858c8c75b4bbb server_cookie=874f30e122e99a60 in_seq=0 out_seq=0 2024-09-19T07:00:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.345+0000 7f5b32ff5640 10 client.19290 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.358+0000 7f5b32ff5640 1 -- 172.21.15.103:0/734733333 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556efd7dcd70 con 0x556efd8f6e40 2024-09-19T07:00:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.358+0000 7f5b32ff5640 10 client.19290 handle_command_reply: tid=0 2024-09-19T07:00:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.358+0000 7f5b57e42740 1 client.19290 shutdown 2024-09-19T07:00:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.358+0000 7f5b57e42740 1 -- 172.21.15.103:0/734733333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd870af0 msgr2=0x556efd884200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.358+0000 7f5b57e42740 1 --2- 172.21.15.103:0/734733333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd870af0 0x556efd884200 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5b4009bbd0 tx=0x7f5b40098780 comp rx=0 tx=0).stop 2024-09-19T07:00:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b57e42740 1 -- 172.21.15.103:0/734733333 shutdown_connections 2024-09-19T07:00:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b57e42740 1 --2- 172.21.15.103:0/734733333 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556efd8f6e40 0x556efd8f9260 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b57e42740 1 --2- 172.21.15.103:0/734733333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efd87bcb0 0x556efd875c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b32ff5640 0 client.19290 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b57e42740 1 --2- 172.21.15.103:0/734733333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556efd871410 0x556efd8756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b57e42740 1 --2- 172.21.15.103:0/734733333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556efd870af0 0x556efd884200 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b57e42740 1 -- 172.21.15.103:0/734733333 >> 172.21.15.103:0/734733333 conn(0x556efd86d060 msgr2=0x556efd86b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b57e42740 1 -- 172.21.15.103:0/734733333 shutdown_connections 2024-09-19T07:00:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b57e42740 1 -- 172.21.15.103:0/734733333 wait complete. 2024-09-19T07:00:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b57e42740 20 client.19290 trim_cache size 0 max 16384 2024-09-19T07:00:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:00:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:00:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:00:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:00:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b550af640 1 -- 172.21.15.103:0/2357553294 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5b380a3460 msgr2=0x7f5b380a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b550af640 1 --2- 172.21.15.103:0/2357553294 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5b380a3460 0x7f5b380a5920 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5b480099a0 tx=0x7f5b48009500 comp rx=0 tx=0).stop 2024-09-19T07:00:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b550af640 1 -- 172.21.15.103:0/2357553294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b501009b0 msgr2=0x7f5b500fcde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b550af640 1 --2- 172.21.15.103:0/2357553294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b501009b0 0x7f5b500fcde0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000e9f0 tx=0x7f5b4000edf0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b550af640 1 -- 172.21.15.103:0/2357553294 shutdown_connections 2024-09-19T07:00:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b550af640 1 --2- 172.21.15.103:0/2357553294 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5b380a3460 0x7f5b380a5920 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b550af640 1 --2- 172.21.15.103:0/2357553294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b501069c0 0x7f5b500fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b550af640 1 --2- 172.21.15.103:0/2357553294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b50101350 0x7f5b500fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b550af640 1 --2- 172.21.15.103:0/2357553294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b501009b0 0x7f5b500fcde0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b550af640 1 -- 172.21.15.103:0/2357553294 >> 172.21.15.103:0/2357553294 conn(0x7f5b50077990 msgr2=0x7f5b50069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b550af640 1 -- 172.21.15.103:0/2357553294 shutdown_connections 2024-09-19T07:00:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:35.359+0000 7f5b550af640 1 -- 172.21.15.103:0/2357553294 wait complete. 2024-09-19T07:00:35.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.541+0000 7fb64b42b640 1 -- 172.21.15.103:0/297808431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb644106f10 msgr2=0x7fb64410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:35.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.541+0000 7fb64b42b640 1 --2- 172.21.15.103:0/297808431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb644106f10 0x7fb64410ea90 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009ca0 tx=0x7fb63c02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.541+0000 7fb64b42b640 1 -- 172.21.15.103:0/297808431 shutdown_connections 2024-09-19T07:00:35.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.541+0000 7fb64b42b640 1 --2- 172.21.15.103:0/297808431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb644106f10 0x7fb64410ea90 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.541+0000 7fb64b42b640 1 --2- 172.21.15.103:0/297808431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb644102790 0x7fb6441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.541+0000 7fb64b42b640 1 --2- 172.21.15.103:0/297808431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb644101de0 0x7fb6441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.541+0000 7fb64b42b640 1 -- 172.21.15.103:0/297808431 >> 172.21.15.103:0/297808431 conn(0x7fb6440fd8c0 msgr2=0x7fb6440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:35.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.541+0000 7fb64b42b640 1 -- 172.21.15.103:0/297808431 shutdown_connections 2024-09-19T07:00:35.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.541+0000 7fb64b42b640 1 -- 172.21.15.103:0/297808431 wait complete. 2024-09-19T07:00:35.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.542+0000 7fb64b42b640 1 Processor -- start 2024-09-19T07:00:35.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.542+0000 7fb64b42b640 1 -- start start 2024-09-19T07:00:35.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.542+0000 7fb64b42b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb644101de0 0x7fb6440744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.542+0000 7fb64b42b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb644102790 0x7fb644074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.542+0000 7fb64b42b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb644106f10 0x7fb644074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.542+0000 7fb64b42b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb644112010 con 0x7fb644106f10 2024-09-19T07:00:35.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.542+0000 7fb64b42b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb644111e90 con 0x7fb644101de0 2024-09-19T07:00:35.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.542+0000 7fb64b42b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb644112190 con 0x7fb644102790 2024-09-19T07:00:35.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.542+0000 7fb6499c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb644106f10 0x7fb644074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.542+0000 7fb6499c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb644106f10 0x7fb644074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53282/0 (socket says 172.21.15.103:53282) 2024-09-19T07:00:35.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.542+0000 7fb6499c8640 1 -- 172.21.15.103:0/3511743790 learned_addr learned my addr 172.21.15.103:0/3511743790 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:35.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.542+0000 7fb6491c7640 1 --2- 172.21.15.103:0/3511743790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb644101de0 0x7fb6440744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.542+0000 7fb6489c6640 1 --2- 172.21.15.103:0/3511743790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb644102790 0x7fb644074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.543+0000 7fb6499c8640 1 -- 172.21.15.103:0/3511743790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb644102790 msgr2=0x7fb644074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:35.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.543+0000 7fb6499c8640 1 --2- 172.21.15.103:0/3511743790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb644102790 0x7fb644074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.543+0000 7fb6499c8640 1 -- 172.21.15.103:0/3511743790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb644101de0 msgr2=0x7fb6440744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:35.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.543+0000 7fb6499c8640 1 --2- 172.21.15.103:0/3511743790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb644101de0 0x7fb6440744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.543+0000 7fb6499c8640 1 -- 172.21.15.103:0/3511743790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6441a6ab0 con 0x7fb644106f10 2024-09-19T07:00:35.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.543+0000 7fb6499c8640 1 --2- 172.21.15.103:0/3511743790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb644106f10 0x7fb644074f70 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fb63c031a60 tx=0x7fb63c031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:35.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.543+0000 7fb63a7fc640 1 -- 172.21.15.103:0/3511743790 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c003b70 con 0x7fb644106f10 2024-09-19T07:00:35.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.543+0000 7fb63a7fc640 1 -- 172.21.15.103:0/3511743790 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63c003d10 con 0x7fb644106f10 2024-09-19T07:00:35.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.543+0000 7fb64b42b640 1 -- 172.21.15.103:0/3511743790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6441a6d40 con 0x7fb644106f10 2024-09-19T07:00:35.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.543+0000 7fb63a7fc640 1 -- 172.21.15.103:0/3511743790 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c004770 con 0x7fb644106f10 2024-09-19T07:00:35.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.543+0000 7fb64b42b640 1 -- 172.21.15.103:0/3511743790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6441a7280 con 0x7fb644106f10 2024-09-19T07:00:35.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.544+0000 7fb64b42b640 1 -- 172.21.15.103:0/3511743790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb644102240 con 0x7fb644106f10 2024-09-19T07:00:35.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.545+0000 7fb63a7fc640 1 -- 172.21.15.103:0/3511743790 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fb63c02f790 con 0x7fb644106f10 2024-09-19T07:00:35.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.546+0000 7fb63a7fc640 1 --2- 172.21.15.103:0/3511743790 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb6300a33b0 0x7fb6300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:35.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.546+0000 7fb6491c7640 1 --2- 172.21.15.103:0/3511743790 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb6300a33b0 0x7fb6300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:35.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.546+0000 7fb63a7fc640 1 -- 172.21.15.103:0/3511743790 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fb63c0f0f00 con 0x7fb644106f10 2024-09-19T07:00:35.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.546+0000 7fb6491c7640 1 --2- 172.21.15.103:0/3511743790 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb6300a33b0 0x7fb6300a5870 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb640009990 tx=0x7fb640009110 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:35.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.550+0000 7fb63a7fc640 1 -- 172.21.15.103:0/3511743790 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb63c0bd970 con 0x7fb644106f10 2024-09-19T07:00:35.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.750+0000 7fb64b42b640 1 -- 172.21.15.103:0/3511743790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb64410a0f0 con 0x7fb644106f10 2024-09-19T07:00:35.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.751+0000 7fb63a7fc640 1 -- 172.21.15.103:0/3511743790 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fb63c0bd270 con 0x7fb644106f10 2024-09-19T07:00:35.752 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:00:35.752 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:00:35.753 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:00:35.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.753+0000 7fb64b42b640 1 -- 172.21.15.103:0/3511743790 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb6300a33b0 msgr2=0x7fb6300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:35.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.753+0000 7fb64b42b640 1 --2- 172.21.15.103:0/3511743790 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb6300a33b0 0x7fb6300a5870 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb640009990 tx=0x7fb640009110 comp rx=0 tx=0).stop 2024-09-19T07:00:35.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.753+0000 7fb64b42b640 1 -- 172.21.15.103:0/3511743790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb644106f10 msgr2=0x7fb644074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:35.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.753+0000 7fb64b42b640 1 --2- 172.21.15.103:0/3511743790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb644106f10 0x7fb644074f70 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fb63c031a60 tx=0x7fb63c031a90 comp rx=0 tx=0).stop 2024-09-19T07:00:35.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.753+0000 7fb64b42b640 1 -- 172.21.15.103:0/3511743790 shutdown_connections 2024-09-19T07:00:35.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.753+0000 7fb64b42b640 1 --2- 172.21.15.103:0/3511743790 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb6300a33b0 0x7fb6300a5870 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.753+0000 7fb64b42b640 1 --2- 172.21.15.103:0/3511743790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb644106f10 0x7fb644074f70 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.753+0000 7fb64b42b640 1 --2- 172.21.15.103:0/3511743790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb644102790 0x7fb644074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.753+0000 7fb64b42b640 1 --2- 172.21.15.103:0/3511743790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb644101de0 0x7fb6440744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:35.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.753+0000 7fb64b42b640 1 -- 172.21.15.103:0/3511743790 >> 172.21.15.103:0/3511743790 conn(0x7fb6440fd8c0 msgr2=0x7fb6440fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:35.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.753+0000 7fb64b42b640 1 -- 172.21.15.103:0/3511743790 shutdown_connections 2024-09-19T07:00:35.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:35.753+0000 7fb64b42b640 1 -- 172.21.15.103:0/3511743790 wait complete. 2024-09-19T07:00:36.229 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:35 smithi103 ceph-mon[31158]: pgmap v353: 97 pgs: 97 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T07:00:36.229 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:35 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3511743790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:00:36.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:35 smithi174 ceph-mon[35637]: pgmap v353: 97 pgs: 97 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T07:00:36.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:35 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3511743790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:00:36.388 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:00:36.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:35 smithi134 ceph-mon[35356]: pgmap v353: 97 pgs: 97 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T07:00:36.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:35 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3511743790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:00:36.637 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:00:37.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.143+0000 7fe7ddd86640 1 -- 172.21.15.103:0/3058149784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7d8101200 msgr2=0x7fe7d8101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:37.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.143+0000 7fe7ddd86640 1 --2- 172.21.15.103:0/3058149784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7d8101200 0x7fe7d8101660 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0009900 tx=0x7fe7c002a410 comp rx=0 tx=0).stop 2024-09-19T07:00:37.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.143+0000 7fe7ddd86640 1 -- 172.21.15.103:0/3058149784 shutdown_connections 2024-09-19T07:00:37.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.143+0000 7fe7ddd86640 1 --2- 172.21.15.103:0/3058149784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7d8101ba0 0x7fe7d810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.143+0000 7fe7ddd86640 1 --2- 172.21.15.103:0/3058149784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7d8101200 0x7fe7d8101660 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.143+0000 7fe7ddd86640 1 --2- 172.21.15.103:0/3058149784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7d8105ad0 0x7fe7d8105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.143+0000 7fe7ddd86640 1 -- 172.21.15.103:0/3058149784 >> 172.21.15.103:0/3058149784 conn(0x7fe7d8077210 msgr2=0x7fe7d80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:37.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.143+0000 7fe7ddd86640 1 -- 172.21.15.103:0/3058149784 shutdown_connections 2024-09-19T07:00:37.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.143+0000 7fe7ddd86640 1 -- 172.21.15.103:0/3058149784 wait complete. 2024-09-19T07:00:37.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.144+0000 7fe7ddd86640 1 Processor -- start 2024-09-19T07:00:37.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.144+0000 7fe7ddd86640 1 -- start start 2024-09-19T07:00:37.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.144+0000 7fe7ddd86640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7d8101200 0x7fe7d819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.144+0000 7fe7ddd86640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7d8101ba0 0x7fe7d819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.144+0000 7fe7ddd86640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7d8105ad0 0x7fe7d819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.144+0000 7fe7ddd86640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8110ef0 con 0x7fe7d8101200 2024-09-19T07:00:37.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.144+0000 7fe7ddd86640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8110d70 con 0x7fe7d8105ad0 2024-09-19T07:00:37.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.144+0000 7fe7ddd86640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8111070 con 0x7fe7d8101ba0 2024-09-19T07:00:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7d77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7d8101200 0x7fe7d819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7d77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7d8101200 0x7fe7d819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53306/0 (socket says 172.21.15.103:53306) 2024-09-19T07:00:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7d77fe640 1 -- 172.21.15.103:0/652702001 learned_addr learned my addr 172.21.15.103:0/652702001 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7d7fff640 1 --2- 172.21.15.103:0/652702001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7d8105ad0 0x7fe7d819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7d6ffd640 1 --2- 172.21.15.103:0/652702001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7d8101ba0 0x7fe7d819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7d77fe640 1 -- 172.21.15.103:0/652702001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7d8101ba0 msgr2=0x7fe7d819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7d77fe640 1 --2- 172.21.15.103:0/652702001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7d8101ba0 0x7fe7d819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7d77fe640 1 -- 172.21.15.103:0/652702001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7d8105ad0 msgr2=0x7fe7d819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7d77fe640 1 --2- 172.21.15.103:0/652702001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7d8105ad0 0x7fe7d819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7d77fe640 1 -- 172.21.15.103:0/652702001 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d819f570 con 0x7fe7d8101200 2024-09-19T07:00:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7d7fff640 1 --2- 172.21.15.103:0/652702001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7d8105ad0 0x7fe7d819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7d77fe640 1 --2- 172.21.15.103:0/652702001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7d8101200 0x7fe7d819a5c0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800e9f0 tx=0x7fe7c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7d4ff9640 1 -- 172.21.15.103:0/652702001 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c800cd80 con 0x7fe7d8101200 2024-09-19T07:00:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7d4ff9640 1 -- 172.21.15.103:0/652702001 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c8010910 con 0x7fe7d8101200 2024-09-19T07:00:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7ddd86640 1 -- 172.21.15.103:0/652702001 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d819f860 con 0x7fe7d8101200 2024-09-19T07:00:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7d4ff9640 1 -- 172.21.15.103:0/652702001 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8018990 con 0x7fe7d8101200 2024-09-19T07:00:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.145+0000 7fe7ddd86640 1 -- 172.21.15.103:0/652702001 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d81a7140 con 0x7fe7d8101200 2024-09-19T07:00:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.147+0000 7fe7d4ff9640 1 -- 172.21.15.103:0/652702001 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fe7c8010ac0 con 0x7fe7d8101200 2024-09-19T07:00:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.148+0000 7fe7e015f740 1 Processor -- start 2024-09-19T07:00:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.148+0000 7fe7e015f740 1 -- start start 2024-09-19T07:00:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.148+0000 7fe7e015f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a12b40 0x561f05a11090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.148+0000 7fe7e015f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a115d0 0x561f05a11a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.148+0000 7fe7e015f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a1e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.148+0000 7fe7e015f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561f059772f0 con 0x561f05a1c370 2024-09-19T07:00:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.148+0000 7fe7e015f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561f0591ecf0 con 0x561f05a115d0 2024-09-19T07:00:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.148+0000 7fe7e015f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561f059732b0 con 0x561f05a12b40 2024-09-19T07:00:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.148+0000 7fe7d4ff9640 1 --2- 172.21.15.103:0/652702001 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe7c40a33b0 0x7fe7c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.148+0000 7fe7d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a1e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.148+0000 7fe7d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a1e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53308/0 (socket says 172.21.15.103:53308) 2024-09-19T07:00:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.148+0000 7fe7d6ffd640 1 -- 172.21.15.103:0/2919871420 learned_addr learned my addr 172.21.15.103:0/2919871420 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.148+0000 7fe7d7fff640 1 --2- 172.21.15.103:0/652702001 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe7c40a33b0 0x7fe7c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.148+0000 7fe7d7fff640 1 --2- 172.21.15.103:0/2919871420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a12b40 0x561f05a11090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.148+0000 7fe7d4ff9640 1 -- 172.21.15.103:0/652702001 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fe7c80d0110 con 0x7fe7d8101200 2024-09-19T07:00:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.148+0000 7fe7d77fe640 1 --2- 172.21.15.103:0/2919871420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a115d0 0x561f05a11a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.149+0000 7fe7d6ffd640 1 -- 172.21.15.103:0/2919871420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a12b40 msgr2=0x561f05a11090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.149+0000 7fe7d6ffd640 1 --2- 172.21.15.103:0/2919871420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a12b40 0x561f05a11090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.149+0000 7fe7d6ffd640 1 -- 172.21.15.103:0/2919871420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a115d0 msgr2=0x561f05a11a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.149+0000 7fe7d6ffd640 1 --2- 172.21.15.103:0/2919871420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a115d0 0x561f05a11a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.149+0000 7fe7d6ffd640 1 -- 172.21.15.103:0/2919871420 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f059033a0 con 0x561f05a1c370 2024-09-19T07:00:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.149+0000 7fe7d77fe640 1 --2- 172.21.15.103:0/2919871420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a115d0 0x561f05a11a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.149+0000 7fe7d6ffd640 1 --2- 172.21.15.103:0/2919871420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a1e760 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fe7c002a930 tx=0x7fe7c0005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2181d5649fbac238 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.149+0000 7fe7d7fff640 1 --2- 172.21.15.103:0/652702001 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe7c40a33b0 0x7fe7c40a5870 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8106710 tx=0x7fe7d00098e0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.149+0000 7fe7cdffb640 1 -- 172.21.15.103:0/2919871420 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0035030 con 0x561f05a1c370 2024-09-19T07:00:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.149+0000 7fe7cdffb640 1 -- 172.21.15.103:0/2919871420 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c0005520 con 0x561f05a1c370 2024-09-19T07:00:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.149+0000 7fe7cdffb640 1 -- 172.21.15.103:0/2919871420 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00035f0 con 0x561f05a1c370 2024-09-19T07:00:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.150+0000 7fe7e015f740 1 -- 172.21.15.103:0/2919871420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 msgr2=0x561f05a1e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.150+0000 7fe7e015f740 1 --2- 172.21.15.103:0/2919871420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a1e760 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fe7c002a930 tx=0x7fe7c0005e60 comp rx=0 tx=0).stop 2024-09-19T07:00:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.150+0000 7fe7e015f740 1 -- 172.21.15.103:0/2919871420 shutdown_connections 2024-09-19T07:00:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.150+0000 7fe7e015f740 1 --2- 172.21.15.103:0/2919871420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a1e760 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.150+0000 7fe7e015f740 1 --2- 172.21.15.103:0/2919871420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a115d0 0x561f05a11a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.150+0000 7fe7e015f740 1 --2- 172.21.15.103:0/2919871420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a12b40 0x561f05a11090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.150+0000 7fe7e015f740 1 -- 172.21.15.103:0/2919871420 >> 172.21.15.103:0/2919871420 conn(0x561f059da310 msgr2=0x561f05a0dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.150+0000 7fe7e015f740 1 -- 172.21.15.103:0/2919871420 shutdown_connections 2024-09-19T07:00:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.150+0000 7fe7e015f740 1 -- 172.21.15.103:0/2919871420 wait complete. 2024-09-19T07:00:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.150+0000 7fe7e015f740 1 -- start start 2024-09-19T07:00:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.150+0000 7fe7e015f740 1 Processor -- start 2024-09-19T07:00:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7e015f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a115d0 0x561f05a1bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7e015f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a12b40 0x561f05a18fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7e015f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a19570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7e015f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561f05a1f010 con 0x561f05a1c370 2024-09-19T07:00:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7e015f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561f05932060 con 0x561f05a12b40 2024-09-19T07:00:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7e015f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561f05a1f190 con 0x561f05a115d0 2024-09-19T07:00:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7e015f740 10 client.0 _ll_register_callbacks cb 0x561f056858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7d7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a19570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7d7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a19570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53324/0 (socket says 172.21.15.103:53324) 2024-09-19T07:00:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7d7fff640 1 -- 172.21.15.103:0/3743255884 learned_addr learned my addr 172.21.15.103:0/3743255884 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7d77fe640 1 --2- 172.21.15.103:0/3743255884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a115d0 0x561f05a1bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7d6ffd640 1 --2- 172.21.15.103:0/3743255884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a12b40 0x561f05a18fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7d7fff640 1 -- 172.21.15.103:0/3743255884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a115d0 msgr2=0x561f05a1bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:37.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7d7fff640 1 --2- 172.21.15.103:0/3743255884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a115d0 0x561f05a1bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7d7fff640 1 -- 172.21.15.103:0/3743255884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a12b40 msgr2=0x561f05a18fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:37.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7d7fff640 1 --2- 172.21.15.103:0/3743255884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a12b40 0x561f05a18fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7d7fff640 1 -- 172.21.15.103:0/3743255884 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f056e83c0 con 0x561f05a1c370 2024-09-19T07:00:37.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7d6ffd640 1 --2- 172.21.15.103:0/3743255884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a12b40 0x561f05a18fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:37.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7d7fff640 1 --2- 172.21.15.103:0/3743255884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a19570 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fe7d001e6a0 tx=0x7fe7d0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70618e28681e1e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:37.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:37.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.151+0000 7fe7b6ffd640 1 -- 172.21.15.103:0/3743255884 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d002e060 con 0x561f05a1c370 2024-09-19T07:00:37.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.152+0000 7fe7b6ffd640 1 -- 172.21.15.103:0/3743255884 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fe7d001b070 con 0x561f05a1c370 2024-09-19T07:00:37.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.152+0000 7fe7e015f740 1 -- 172.21.15.103:0/3743255884 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7c0005520 con 0x561f05a1c370 2024-09-19T07:00:37.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.152+0000 7fe7b6ffd640 1 -- 172.21.15.103:0/3743255884 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7d0032430 con 0x561f05a1c370 2024-09-19T07:00:37.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.152+0000 7fe7e015f740 10 client.19308 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:37.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.152+0000 7fe7e015f740 1 -- 172.21.15.103:0/3743255884 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f05a2aa80 con 0x561f05a1c370 2024-09-19T07:00:37.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.152+0000 7fe7b6ffd640 1 -- 172.21.15.103:0/3743255884 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c0005520 con 0x561f05a1c370 2024-09-19T07:00:37.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.152+0000 7fe7b6ffd640 1 -- 172.21.15.103:0/3743255884 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe7d0031590 con 0x561f05a1c370 2024-09-19T07:00:37.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.152+0000 7fe7e015f740 10 client.19308 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:37.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.152+0000 7fe7e015f740 10 client.19308 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:00:37.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.152+0000 7fe7e015f740 20 client.19308 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:37.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.152+0000 7fe7e015f740 1 --2- 172.21.15.103:0/3743255884 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x561f05a2bf90 0x561f05a2e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.152+0000 7fe7e015f740 4 client.19308 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:00:37.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.152+0000 7fe7e015f740 1 -- 172.21.15.103:0/3743255884 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f05984860 con 0x561f05a2bf90 2024-09-19T07:00:37.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.152+0000 7fe7d77fe640 1 --2- 172.21.15.103:0/3743255884 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x561f05a2bf90 0x561f05a2e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.153+0000 7fe7d77fe640 1 --2- 172.21.15.103:0/3743255884 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x561f05a2bf90 0x561f05a2e430 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4083ce08988930bb server_cookie=f2376acadfae8a45 in_seq=0 out_seq=0 2024-09-19T07:00:37.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.153+0000 7fe7b6ffd640 10 client.19308 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:00:37.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.154+0000 7fe7b6ffd640 1 -- 172.21.15.103:0/3743255884 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f05984860 con 0x561f05a2bf90 2024-09-19T07:00:37.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.154+0000 7fe7b6ffd640 10 client.19308 handle_command_reply: tid=0 2024-09-19T07:00:37.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.154+0000 7fe7e015f740 1 client.19308 shutdown 2024-09-19T07:00:37.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.154+0000 7fe7e015f740 1 -- 172.21.15.103:0/3743255884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 msgr2=0x561f05a19570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:37.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.154+0000 7fe7e015f740 1 --2- 172.21.15.103:0/3743255884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a19570 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fe7d001e6a0 tx=0x7fe7d0004ab0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.154+0000 7fe7e015f740 1 -- 172.21.15.103:0/3743255884 shutdown_connections 2024-09-19T07:00:37.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.154+0000 7fe7e015f740 1 --2- 172.21.15.103:0/3743255884 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x561f05a2bf90 0x561f05a2e430 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.154+0000 7fe7e015f740 1 --2- 172.21.15.103:0/3743255884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a19570 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.154+0000 7fe7e015f740 1 --2- 172.21.15.103:0/3743255884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a12b40 0x561f05a18fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.154+0000 7fe7e015f740 1 --2- 172.21.15.103:0/3743255884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a115d0 0x561f05a1bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.154+0000 7fe7e015f740 1 -- 172.21.15.103:0/3743255884 >> 172.21.15.103:0/3743255884 conn(0x561f059da310 msgr2=0x561f05a13960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:37.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.154+0000 7fe7b6ffd640 0 client.19308 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:00:37.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.154+0000 7fe7e015f740 1 -- 172.21.15.103:0/3743255884 shutdown_connections 2024-09-19T07:00:37.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.154+0000 7fe7e015f740 1 -- 172.21.15.103:0/3743255884 wait complete. 2024-09-19T07:00:37.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.154+0000 7fe7e015f740 20 client.19308 trim_cache size 0 max 16384 2024-09-19T07:00:37.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.167+0000 7fe7e015f740 1 Processor -- start 2024-09-19T07:00:37.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.167+0000 7fe7e015f740 1 -- start start 2024-09-19T07:00:37.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.167+0000 7fe7e015f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a115d0 0x561f05a1af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.167+0000 7fe7e015f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a12b40 0x561f05a14120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.167+0000 7fe7e015f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a1c370 0x561f05a14660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.167+0000 7fe7e015f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81af9f0 con 0x561f05a12b40 2024-09-19T07:00:37.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.167+0000 7fe7e015f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81b1530 con 0x561f05a1c370 2024-09-19T07:00:37.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.167+0000 7fe7e015f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81b00c0 con 0x561f05a115d0 2024-09-19T07:00:37.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.168+0000 7fe7d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a12b40 0x561f05a14120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.168+0000 7fe7d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a12b40 0x561f05a14120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53332/0 (socket says 172.21.15.103:53332) 2024-09-19T07:00:37.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.168+0000 7fe7d6ffd640 1 -- 172.21.15.103:0/3537390175 learned_addr learned my addr 172.21.15.103:0/3537390175 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:37.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.168+0000 7fe7d7fff640 1 --2- 172.21.15.103:0/3537390175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a1c370 0x561f05a14660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.168+0000 7fe7d77fe640 1 --2- 172.21.15.103:0/3537390175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a115d0 0x561f05a1af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.168+0000 7fe7d6ffd640 1 -- 172.21.15.103:0/3537390175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a115d0 msgr2=0x561f05a1af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:37.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.168+0000 7fe7d6ffd640 1 --2- 172.21.15.103:0/3537390175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a115d0 0x561f05a1af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.168+0000 7fe7d6ffd640 1 -- 172.21.15.103:0/3537390175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a1c370 msgr2=0x561f05a14660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:37.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.168+0000 7fe7d6ffd640 1 --2- 172.21.15.103:0/3537390175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a1c370 0x561f05a14660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.168+0000 7fe7d6ffd640 1 -- 172.21.15.103:0/3537390175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f05a0e960 con 0x561f05a12b40 2024-09-19T07:00:37.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.168+0000 7fe7d6ffd640 1 --2- 172.21.15.103:0/3537390175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a12b40 0x561f05a14120 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x561f05a12070 tx=0x7fe7c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62dcfc77a913ceb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:37.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.168+0000 7fe7cd7fa640 1 -- 172.21.15.103:0/3537390175 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0035070 con 0x561f05a12b40 2024-09-19T07:00:37.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.168+0000 7fe7cd7fa640 1 -- 172.21.15.103:0/3537390175 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c0005550 con 0x561f05a12b40 2024-09-19T07:00:37.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.168+0000 7fe7cd7fa640 1 -- 172.21.15.103:0/3537390175 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0004e60 con 0x561f05a12b40 2024-09-19T07:00:37.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.169+0000 7fe7e015f740 1 -- 172.21.15.103:0/3537390175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a12b40 msgr2=0x561f05a14120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:37.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.169+0000 7fe7e015f740 1 --2- 172.21.15.103:0/3537390175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a12b40 0x561f05a14120 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x561f05a12070 tx=0x7fe7c0000bc0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.169+0000 7fe7e015f740 1 -- 172.21.15.103:0/3537390175 shutdown_connections 2024-09-19T07:00:37.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.169+0000 7fe7e015f740 1 --2- 172.21.15.103:0/3537390175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a1c370 0x561f05a14660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.169+0000 7fe7e015f740 1 --2- 172.21.15.103:0/3537390175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a12b40 0x561f05a14120 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.169+0000 7fe7e015f740 1 --2- 172.21.15.103:0/3537390175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a115d0 0x561f05a1af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.169+0000 7fe7e015f740 1 -- 172.21.15.103:0/3537390175 >> 172.21.15.103:0/3537390175 conn(0x561f059da310 msgr2=0x561f05a1a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:37.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.169+0000 7fe7e015f740 1 -- 172.21.15.103:0/3537390175 shutdown_connections 2024-09-19T07:00:37.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.169+0000 7fe7e015f740 1 -- 172.21.15.103:0/3537390175 wait complete. 2024-09-19T07:00:37.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.169+0000 7fe7e015f740 1 -- start start 2024-09-19T07:00:37.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7e015f740 1 Processor -- start 2024-09-19T07:00:37.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7e015f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a115d0 0x561f05a15f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7e015f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a12b40 0x561f05a164a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7e015f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7e015f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81b0e60 con 0x561f05a1c370 2024-09-19T07:00:37.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7e015f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81b0790 con 0x561f05a12b40 2024-09-19T07:00:37.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7e015f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81ad9f0 con 0x561f05a115d0 2024-09-19T07:00:37.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7e015f740 10 client.0 _ll_register_callbacks cb 0x7fe7d0009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:37.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7d7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7d7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a169e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53346/0 (socket says 172.21.15.103:53346) 2024-09-19T07:00:37.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7d6ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a12b40 0x561f05a164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7d7fff640 1 -- 172.21.15.103:0/1416102327 learned_addr learned my addr 172.21.15.103:0/1416102327 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:37.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7d77fe640 1 --2- 172.21.15.103:0/1416102327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a115d0 0x561f05a15f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7d7fff640 1 -- 172.21.15.103:0/1416102327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a115d0 msgr2=0x561f05a15f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:37.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7d7fff640 1 --2- 172.21.15.103:0/1416102327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a115d0 0x561f05a15f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7d7fff640 1 -- 172.21.15.103:0/1416102327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a12b40 msgr2=0x561f05a164a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:37.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7d7fff640 1 --2- 172.21.15.103:0/1416102327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a12b40 0x561f05a164a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7d7fff640 1 -- 172.21.15.103:0/1416102327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f05a95cd0 con 0x561f05a1c370 2024-09-19T07:00:37.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7d7fff640 1 --2- 172.21.15.103:0/1416102327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a169e0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0031ba0 tx=0x7fe7d0009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e8749e70e03423 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:37.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.170+0000 7fe7b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:37.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.171+0000 7fe7b6ffd640 1 -- 172.21.15.103:0/1416102327 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0040060 con 0x561f05a1c370 2024-09-19T07:00:37.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.171+0000 7fe7b6ffd640 1 -- 172.21.15.103:0/1416102327 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fe7d001b040 con 0x561f05a1c370 2024-09-19T07:00:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.171+0000 7fe7e015f740 1 -- 172.21.15.103:0/1416102327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7c0005550 con 0x561f05a1c370 2024-09-19T07:00:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.171+0000 7fe7b6ffd640 1 -- 172.21.15.103:0/1416102327 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7d00315e0 con 0x561f05a1c370 2024-09-19T07:00:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.171+0000 7fe7e015f740 10 client.19314 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.171+0000 7fe7e015f740 1 -- 172.21.15.103:0/1416102327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f05a96f10 con 0x561f05a1c370 2024-09-19T07:00:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.171+0000 7fe7b6ffd640 1 -- 172.21.15.103:0/1416102327 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c0005550 con 0x561f05a1c370 2024-09-19T07:00:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.171+0000 7fe7b6ffd640 1 -- 172.21.15.103:0/1416102327 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe7d003b4f0 con 0x561f05a1c370 2024-09-19T07:00:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.171+0000 7fe7e015f740 10 client.19314 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.171+0000 7fe7e015f740 10 client.19314 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:00:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.171+0000 7fe7e015f740 20 client.19314 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.171+0000 7fe7e015f740 1 --2- 172.21.15.103:0/1416102327 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x561f05a97570 0x561f05a99960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.171+0000 7fe7e015f740 4 client.19314 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:00:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.171+0000 7fe7e015f740 1 -- 172.21.15.103:0/1416102327 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561f0597ed70 con 0x561f05a97570 2024-09-19T07:00:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.171+0000 7fe7d77fe640 1 --2- 172.21.15.103:0/1416102327 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x561f05a97570 0x561f05a99960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7b6ffd640 10 client.19314 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:00:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7d77fe640 1 --2- 172.21.15.103:0/1416102327 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x561f05a97570 0x561f05a99960 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6ccd32b5f4458275 server_cookie=ece432953fdf756 in_seq=0 out_seq=0 2024-09-19T07:00:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7b6ffd640 1 -- 172.21.15.103:0/1416102327 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561f0597ed70 con 0x561f05a97570 2024-09-19T07:00:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7b6ffd640 10 client.19314 handle_command_reply: tid=0 2024-09-19T07:00:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7e015f740 1 client.19314 shutdown 2024-09-19T07:00:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7e015f740 1 -- 172.21.15.103:0/1416102327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 msgr2=0x561f05a169e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7e015f740 1 --2- 172.21.15.103:0/1416102327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a169e0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0031ba0 tx=0x7fe7d0009570 comp rx=0 tx=0).stop 2024-09-19T07:00:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7e015f740 1 -- 172.21.15.103:0/1416102327 shutdown_connections 2024-09-19T07:00:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7e015f740 1 --2- 172.21.15.103:0/1416102327 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x561f05a97570 0x561f05a99960 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7e015f740 1 --2- 172.21.15.103:0/1416102327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f05a1c370 0x561f05a169e0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7e015f740 1 --2- 172.21.15.103:0/1416102327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05a12b40 0x561f05a164a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7b6ffd640 0 client.19314 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:00:37.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7e015f740 1 --2- 172.21.15.103:0/1416102327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f05a115d0 0x561f05a15f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7e015f740 1 -- 172.21.15.103:0/1416102327 >> 172.21.15.103:0/1416102327 conn(0x561f059da310 msgr2=0x561f05a1a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:37.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7e015f740 1 -- 172.21.15.103:0/1416102327 shutdown_connections 2024-09-19T07:00:37.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7e015f740 1 -- 172.21.15.103:0/1416102327 wait complete. 2024-09-19T07:00:37.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.172+0000 7fe7e015f740 20 client.19314 trim_cache size 0 max 16384 2024-09-19T07:00:37.186 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:00:37.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.173+0000 7fe7ddd86640 1 -- 172.21.15.103:0/652702001 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe7c40a33b0 msgr2=0x7fe7c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:37.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.173+0000 7fe7ddd86640 1 --2- 172.21.15.103:0/652702001 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe7c40a33b0 0x7fe7c40a5870 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8106710 tx=0x7fe7d00098e0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.173+0000 7fe7ddd86640 1 -- 172.21.15.103:0/652702001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7d8101200 msgr2=0x7fe7d819a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:37.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.173+0000 7fe7ddd86640 1 --2- 172.21.15.103:0/652702001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7d8101200 0x7fe7d819a5c0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800e9f0 tx=0x7fe7c800eeb0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.173+0000 7fe7ddd86640 1 -- 172.21.15.103:0/652702001 shutdown_connections 2024-09-19T07:00:37.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.173+0000 7fe7ddd86640 1 --2- 172.21.15.103:0/652702001 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe7c40a33b0 0x7fe7c40a5870 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.173+0000 7fe7ddd86640 1 --2- 172.21.15.103:0/652702001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7d8105ad0 0x7fe7d819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.173+0000 7fe7ddd86640 1 --2- 172.21.15.103:0/652702001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7d8101ba0 0x7fe7d819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.173+0000 7fe7ddd86640 1 --2- 172.21.15.103:0/652702001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7d8101200 0x7fe7d819a5c0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:37.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.173+0000 7fe7ddd86640 1 -- 172.21.15.103:0/652702001 >> 172.21.15.103:0/652702001 conn(0x7fe7d8077210 msgr2=0x7fe7d8104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:37.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.173+0000 7fe7ddd86640 1 -- 172.21.15.103:0/652702001 shutdown_connections 2024-09-19T07:00:37.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:37.173+0000 7fe7ddd86640 1 -- 172.21.15.103:0/652702001 wait complete. 2024-09-19T07:00:37.914 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:00:38.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:37 smithi103 ceph-mon[31158]: pgmap v354: 97 pgs: 97 active+clean; 3.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-09-19T07:00:38.175 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:00:38.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:37 smithi174 ceph-mon[35637]: pgmap v354: 97 pgs: 97 active+clean; 3.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-09-19T07:00:38.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:37 smithi134 ceph-mon[35356]: pgmap v354: 97 pgs: 97 active+clean; 3.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-09-19T07:00:38.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.626+0000 7f4889f80640 1 -- 172.21.15.103:0/637695807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4884106a80 msgr2=0x7f4884106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:38.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.626+0000 7f4889f80640 1 --2- 172.21.15.103:0/637695807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4884106a80 0x7f4884106e60 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f4874009930 tx=0x7f487402a420 comp rx=0 tx=0).stop 2024-09-19T07:00:38.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.626+0000 7f4889f80640 1 -- 172.21.15.103:0/637695807 shutdown_connections 2024-09-19T07:00:38.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.626+0000 7f4889f80640 1 --2- 172.21.15.103:0/637695807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4884101420 0x7f488410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.626+0000 7f4889f80640 1 --2- 172.21.15.103:0/637695807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4884100a80 0x7f4884100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.626+0000 7f4889f80640 1 --2- 172.21.15.103:0/637695807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4884106a80 0x7f4884106e60 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.626+0000 7f4889f80640 1 -- 172.21.15.103:0/637695807 >> 172.21.15.103:0/637695807 conn(0x7f48840fc7a0 msgr2=0x7f48840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:38.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.626+0000 7f4889f80640 1 -- 172.21.15.103:0/637695807 shutdown_connections 2024-09-19T07:00:38.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.626+0000 7f4889f80640 1 -- 172.21.15.103:0/637695807 wait complete. 2024-09-19T07:00:38.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f4889f80640 1 Processor -- start 2024-09-19T07:00:38.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f4889f80640 1 -- start start 2024-09-19T07:00:38.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f4889f80640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4884100a80 0x7f48841984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f4889f80640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4884101420 0x7f4884198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f4889f80640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4884106a80 0x7f488419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f4889f80640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4884111fa0 con 0x7f4884106a80 2024-09-19T07:00:38.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f4889f80640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4884111e20 con 0x7f4884100a80 2024-09-19T07:00:38.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f4889f80640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4884112120 con 0x7f4884101420 2024-09-19T07:00:38.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f4883fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4884106a80 0x7f488419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f4883fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4884106a80 0x7f488419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57044/0 (socket says 172.21.15.103:57044) 2024-09-19T07:00:38.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f4883fff640 1 -- 172.21.15.103:0/904991431 learned_addr learned my addr 172.21.15.103:0/904991431 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:38.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f48837fe640 1 --2- 172.21.15.103:0/904991431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4884100a80 0x7f48841984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f487bfff640 1 --2- 172.21.15.103:0/904991431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4884101420 0x7f4884198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f4883fff640 1 -- 172.21.15.103:0/904991431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4884101420 msgr2=0x7f4884198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:38.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f4883fff640 1 --2- 172.21.15.103:0/904991431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4884101420 0x7f4884198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f4883fff640 1 -- 172.21.15.103:0/904991431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4884100a80 msgr2=0x7f48841984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:38.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f4883fff640 1 --2- 172.21.15.103:0/904991431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4884100a80 0x7f48841984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.627+0000 7f4883fff640 1 -- 172.21.15.103:0/904991431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488419d470 con 0x7f4884106a80 2024-09-19T07:00:38.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.628+0000 7f4883fff640 1 --2- 172.21.15.103:0/904991431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4884106a80 0x7f488419cd90 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f487c00cce0 tx=0x7f487c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:38.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.628+0000 7f48817fa640 1 -- 172.21.15.103:0/904991431 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c00f450 con 0x7f4884106a80 2024-09-19T07:00:38.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.628+0000 7f48817fa640 1 -- 172.21.15.103:0/904991431 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f487c00fad0 con 0x7f4884106a80 2024-09-19T07:00:38.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.628+0000 7f48817fa640 1 -- 172.21.15.103:0/904991431 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c018b60 con 0x7f4884106a80 2024-09-19T07:00:38.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.628+0000 7f4889f80640 1 -- 172.21.15.103:0/904991431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488419d760 con 0x7f4884106a80 2024-09-19T07:00:38.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.628+0000 7f4889f80640 1 -- 172.21.15.103:0/904991431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48841a5040 con 0x7f4884106a80 2024-09-19T07:00:38.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.630+0000 7f48817fa640 1 -- 172.21.15.103:0/904991431 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f487c00fc80 con 0x7f4884106a80 2024-09-19T07:00:38.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.630+0000 7f488c359740 1 Processor -- start 2024-09-19T07:00:38.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f488c359740 1 -- start start 2024-09-19T07:00:38.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f48817fa640 1 --2- 172.21.15.103:0/904991431 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48700a33b0 0x7f48700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f488c359740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da6ab40 0x557e9da69090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f488c359740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da695d0 0x557e9da69a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f488c359740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da74370 0x557e9da76760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f488c359740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557e9d9cf2f0 con 0x557e9da74370 2024-09-19T07:00:38.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f488c359740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557e9d976cf0 con 0x557e9da695d0 2024-09-19T07:00:38.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f488c359740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557e9d9cb2b0 con 0x557e9da6ab40 2024-09-19T07:00:38.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f487bfff640 1 --2- 172.21.15.103:0/904991431 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48700a33b0 0x7f48700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f4883fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da74370 0x557e9da76760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f4883fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da74370 0x557e9da76760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57048/0 (socket says 172.21.15.103:57048) 2024-09-19T07:00:38.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f48817fa640 1 -- 172.21.15.103:0/904991431 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f487c0d01f0 con 0x7f4884106a80 2024-09-19T07:00:38.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f4883fff640 1 -- 172.21.15.103:0/3438704315 learned_addr learned my addr 172.21.15.103:0/3438704315 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:38.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f487bfff640 1 --2- 172.21.15.103:0/3438704315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da695d0 0x557e9da69a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f48837fe640 1 --2- 172.21.15.103:0/3438704315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da6ab40 0x557e9da69090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f4883fff640 1 -- 172.21.15.103:0/3438704315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da6ab40 msgr2=0x557e9da69090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:38.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f4883fff640 1 --2- 172.21.15.103:0/3438704315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da6ab40 0x557e9da69090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f4883fff640 1 -- 172.21.15.103:0/3438704315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da695d0 msgr2=0x557e9da69a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:38.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f4883fff640 1 --2- 172.21.15.103:0/3438704315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da695d0 0x557e9da69a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f4883fff640 1 -- 172.21.15.103:0/3438704315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e9d95b3a0 con 0x557e9da74370 2024-09-19T07:00:38.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f487bfff640 1 --2- 172.21.15.103:0/904991431 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48700a33b0 0x7f48700a5870 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f48840feda0 tx=0x7f486c0156d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:38.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.631+0000 7f4883fff640 1 --2- 172.21.15.103:0/3438704315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da74370 0x557e9da76760 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f487c098e80 tx=0x7f487c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7704706bad39e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:38.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.632+0000 7f4879ffb640 1 -- 172.21.15.103:0/3438704315 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c00d530 con 0x557e9da74370 2024-09-19T07:00:38.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.632+0000 7f4879ffb640 1 -- 172.21.15.103:0/3438704315 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f487c09cbb0 con 0x557e9da74370 2024-09-19T07:00:38.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.632+0000 7f4879ffb640 1 -- 172.21.15.103:0/3438704315 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c013af0 con 0x557e9da74370 2024-09-19T07:00:38.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.632+0000 7f488c359740 1 -- 172.21.15.103:0/3438704315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da74370 msgr2=0x557e9da76760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:38.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.632+0000 7f488c359740 1 --2- 172.21.15.103:0/3438704315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da74370 0x557e9da76760 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f487c098e80 tx=0x7f487c01c000 comp rx=0 tx=0).stop 2024-09-19T07:00:38.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.632+0000 7f488c359740 1 -- 172.21.15.103:0/3438704315 shutdown_connections 2024-09-19T07:00:38.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.632+0000 7f488c359740 1 --2- 172.21.15.103:0/3438704315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da74370 0x557e9da76760 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.632+0000 7f488c359740 1 --2- 172.21.15.103:0/3438704315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da695d0 0x557e9da69a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.632+0000 7f488c359740 1 --2- 172.21.15.103:0/3438704315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da6ab40 0x557e9da69090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.632+0000 7f488c359740 1 -- 172.21.15.103:0/3438704315 >> 172.21.15.103:0/3438704315 conn(0x557e9da32310 msgr2=0x557e9da65c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:38.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.632+0000 7f488c359740 1 -- 172.21.15.103:0/3438704315 shutdown_connections 2024-09-19T07:00:38.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.632+0000 7f488c359740 1 -- 172.21.15.103:0/3438704315 wait complete. 2024-09-19T07:00:38.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.633+0000 7f488c359740 1 -- start start 2024-09-19T07:00:38.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.633+0000 7f488c359740 1 Processor -- start 2024-09-19T07:00:38.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.633+0000 7f488c359740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da695d0 0x557e9da73dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.633+0000 7f488c359740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da6ab40 0x557e9da71080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.633+0000 7f488c359740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da74370 0x557e9da71810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.633+0000 7f488c359740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557e9da77070 con 0x557e9da695d0 2024-09-19T07:00:38.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.633+0000 7f488c359740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557e9d98a060 con 0x557e9da6ab40 2024-09-19T07:00:38.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.633+0000 7f488c359740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557e9da771f0 con 0x557e9da74370 2024-09-19T07:00:38.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.633+0000 7f488c359740 10 client.0 _ll_register_callbacks cb 0x557e9d6dd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:38.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.633+0000 7f48837fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da695d0 0x557e9da73dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.633+0000 7f48837fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da695d0 0x557e9da73dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57058/0 (socket says 172.21.15.103:57058) 2024-09-19T07:00:38.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.633+0000 7f48837fe640 1 -- 172.21.15.103:0/3440719913 learned_addr learned my addr 172.21.15.103:0/3440719913 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:38.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f4883fff640 1 --2- 172.21.15.103:0/3440719913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da74370 0x557e9da71810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f487bfff640 1 --2- 172.21.15.103:0/3440719913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da6ab40 0x557e9da71080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f48837fe640 1 -- 172.21.15.103:0/3440719913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da74370 msgr2=0x557e9da71810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:38.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f48837fe640 1 --2- 172.21.15.103:0/3440719913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da74370 0x557e9da71810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f48837fe640 1 -- 172.21.15.103:0/3440719913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da6ab40 msgr2=0x557e9da71080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:38.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f48837fe640 1 --2- 172.21.15.103:0/3440719913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da6ab40 0x557e9da71080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f48837fe640 1 -- 172.21.15.103:0/3440719913 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9d7403c0 con 0x557e9da695d0 2024-09-19T07:00:38.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f487bfff640 1 --2- 172.21.15.103:0/3440719913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da6ab40 0x557e9da71080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:38.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f48837fe640 1 --2- 172.21.15.103:0/3440719913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da695d0 0x557e9da73dd0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f4874006010 tx=0x7f4874031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f705e7de6d5fb156 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:38.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f4862ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:38.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f4862ffd640 1 -- 172.21.15.103:0/3440719913 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874035040 con 0x557e9da695d0 2024-09-19T07:00:38.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f4862ffd640 1 -- 172.21.15.103:0/3440719913 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f4874004380 con 0x557e9da695d0 2024-09-19T07:00:38.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f488c359740 1 -- 172.21.15.103:0/3440719913 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f487c09cbb0 con 0x557e9da695d0 2024-09-19T07:00:38.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f4862ffd640 1 -- 172.21.15.103:0/3440719913 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f48740035f0 con 0x557e9da695d0 2024-09-19T07:00:38.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f488c359740 10 client.19326 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:38.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f488c359740 1 -- 172.21.15.103:0/3440719913 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557e9da82b90 con 0x557e9da695d0 2024-09-19T07:00:38.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f4862ffd640 1 -- 172.21.15.103:0/3440719913 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f487c09cbb0 con 0x557e9da695d0 2024-09-19T07:00:38.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.634+0000 7f4862ffd640 1 -- 172.21.15.103:0/3440719913 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f487403c960 con 0x557e9da695d0 2024-09-19T07:00:38.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.635+0000 7f488c359740 10 client.19326 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:38.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.635+0000 7f488c359740 10 client.19326 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:00:38.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.635+0000 7f488c359740 20 client.19326 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:38.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.635+0000 7f488c359740 1 --2- 172.21.15.103:0/3440719913 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x557e9da840a0 0x557e9da86540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.635+0000 7f488c359740 4 client.19326 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:00:38.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.635+0000 7f488c359740 1 -- 172.21.15.103:0/3440719913 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e9d9dc860 con 0x557e9da840a0 2024-09-19T07:00:38.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.635+0000 7f487bfff640 1 --2- 172.21.15.103:0/3440719913 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x557e9da840a0 0x557e9da86540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.635+0000 7f487bfff640 1 --2- 172.21.15.103:0/3440719913 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x557e9da840a0 0x557e9da86540 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=66de9136a0a68f04 server_cookie=c9986c5b458542a in_seq=0 out_seq=0 2024-09-19T07:00:38.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.635+0000 7f4862ffd640 10 client.19326 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:00:38.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.636+0000 7f4862ffd640 1 -- 172.21.15.103:0/3440719913 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e9d9dc860 con 0x557e9da840a0 2024-09-19T07:00:38.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.636+0000 7f4862ffd640 10 client.19326 handle_command_reply: tid=0 2024-09-19T07:00:38.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.636+0000 7f488c359740 1 client.19326 shutdown 2024-09-19T07:00:38.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.636+0000 7f488c359740 1 -- 172.21.15.103:0/3440719913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da695d0 msgr2=0x557e9da73dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:38.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.636+0000 7f488c359740 1 --2- 172.21.15.103:0/3440719913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da695d0 0x557e9da73dd0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f4874006010 tx=0x7f4874031a60 comp rx=0 tx=0).stop 2024-09-19T07:00:38.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.636+0000 7f488c359740 1 -- 172.21.15.103:0/3440719913 shutdown_connections 2024-09-19T07:00:38.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.636+0000 7f488c359740 1 --2- 172.21.15.103:0/3440719913 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x557e9da840a0 0x557e9da86540 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.636+0000 7f488c359740 1 --2- 172.21.15.103:0/3440719913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da74370 0x557e9da71810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.636+0000 7f4862ffd640 0 client.19326 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:00:38.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.636+0000 7f488c359740 1 --2- 172.21.15.103:0/3440719913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da6ab40 0x557e9da71080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.636+0000 7f488c359740 1 --2- 172.21.15.103:0/3440719913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da695d0 0x557e9da73dd0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.636+0000 7f488c359740 1 -- 172.21.15.103:0/3440719913 >> 172.21.15.103:0/3440719913 conn(0x557e9da32310 msgr2=0x557e9da6b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:38.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.636+0000 7f488c359740 1 -- 172.21.15.103:0/3440719913 shutdown_connections 2024-09-19T07:00:38.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.636+0000 7f488c359740 1 -- 172.21.15.103:0/3440719913 wait complete. 2024-09-19T07:00:38.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.636+0000 7f488c359740 20 client.19326 trim_cache size 0 max 16384 2024-09-19T07:00:38.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.649+0000 7f488c359740 1 Processor -- start 2024-09-19T07:00:38.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.649+0000 7f488c359740 1 -- start start 2024-09-19T07:00:38.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.649+0000 7f488c359740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da695d0 0x557e9da6c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.649+0000 7f488c359740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da6ab40 0x557e9da6c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.649+0000 7f488c359740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da74370 0x557e9da8be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.649+0000 7f488c359740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f48841ad9b0 con 0x557e9da6ab40 2024-09-19T07:00:38.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.649+0000 7f488c359740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f48841af4f0 con 0x557e9da74370 2024-09-19T07:00:38.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.649+0000 7f488c359740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f48841ae080 con 0x557e9da695d0 2024-09-19T07:00:38.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.649+0000 7f487bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da6ab40 0x557e9da6c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.649+0000 7f487bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da6ab40 0x557e9da6c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57066/0 (socket says 172.21.15.103:57066) 2024-09-19T07:00:38.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.649+0000 7f487bfff640 1 -- 172.21.15.103:0/2530628516 learned_addr learned my addr 172.21.15.103:0/2530628516 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:38.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.650+0000 7f4883fff640 1 --2- 172.21.15.103:0/2530628516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da74370 0x557e9da8be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.650+0000 7f48837fe640 1 --2- 172.21.15.103:0/2530628516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da695d0 0x557e9da6c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.650+0000 7f487bfff640 1 -- 172.21.15.103:0/2530628516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da695d0 msgr2=0x557e9da6c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:38.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.650+0000 7f487bfff640 1 --2- 172.21.15.103:0/2530628516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da695d0 0x557e9da6c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.650+0000 7f487bfff640 1 -- 172.21.15.103:0/2530628516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da74370 msgr2=0x557e9da8be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:38.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.650+0000 7f487bfff640 1 --2- 172.21.15.103:0/2530628516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da74370 0x557e9da8be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.650+0000 7f487bfff640 1 -- 172.21.15.103:0/2530628516 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e9da6f5d0 con 0x557e9da6ab40 2024-09-19T07:00:38.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.650+0000 7f48837fe640 1 --2- 172.21.15.103:0/2530628516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da695d0 0x557e9da6c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:38.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.650+0000 7f4883fff640 1 --2- 172.21.15.103:0/2530628516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da74370 0x557e9da8be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:38.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.650+0000 7f487bfff640 1 --2- 172.21.15.103:0/2530628516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da6ab40 0x557e9da6c6c0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f486c00a010 tx=0x7f486c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c1b350bcc418b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:38.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.650+0000 7f48797fa640 1 -- 172.21.15.103:0/2530628516 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c034070 con 0x557e9da6ab40 2024-09-19T07:00:38.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.650+0000 7f48797fa640 1 -- 172.21.15.103:0/2530628516 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f486c02fb00 con 0x557e9da6ab40 2024-09-19T07:00:38.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.650+0000 7f48797fa640 1 -- 172.21.15.103:0/2530628516 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c02b040 con 0x557e9da6ab40 2024-09-19T07:00:38.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.651+0000 7f488c359740 1 -- 172.21.15.103:0/2530628516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da6ab40 msgr2=0x557e9da6c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:38.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.651+0000 7f488c359740 1 --2- 172.21.15.103:0/2530628516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da6ab40 0x557e9da6c6c0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f486c00a010 tx=0x7f486c01c000 comp rx=0 tx=0).stop 2024-09-19T07:00:38.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.651+0000 7f488c359740 1 -- 172.21.15.103:0/2530628516 shutdown_connections 2024-09-19T07:00:38.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.651+0000 7f488c359740 1 --2- 172.21.15.103:0/2530628516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da74370 0x557e9da8be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.651+0000 7f488c359740 1 --2- 172.21.15.103:0/2530628516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da6ab40 0x557e9da6c6c0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.651+0000 7f488c359740 1 --2- 172.21.15.103:0/2530628516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da695d0 0x557e9da6c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.651+0000 7f488c359740 1 -- 172.21.15.103:0/2530628516 >> 172.21.15.103:0/2530628516 conn(0x557e9da32310 msgr2=0x557e9da7fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:38.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.651+0000 7f488c359740 1 -- 172.21.15.103:0/2530628516 shutdown_connections 2024-09-19T07:00:38.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.651+0000 7f488c359740 1 -- 172.21.15.103:0/2530628516 wait complete. 2024-09-19T07:00:38.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.651+0000 7f488c359740 1 -- start start 2024-09-19T07:00:38.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.651+0000 7f488c359740 1 Processor -- start 2024-09-19T07:00:38.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f488c359740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da695d0 0x557e9da783a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f488c359740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da6ab40 0x557e9da788e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f488c359740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da74370 0x557e9da78e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f488c359740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f48841aee20 con 0x557e9da74370 2024-09-19T07:00:38.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f488c359740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f48841ae750 con 0x557e9da6ab40 2024-09-19T07:00:38.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f488c359740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f48841ab9b0 con 0x557e9da695d0 2024-09-19T07:00:38.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f488c359740 10 client.0 _ll_register_callbacks cb 0x7f4874009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:38.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f4883fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da74370 0x557e9da78e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f4883fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da74370 0x557e9da78e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57074/0 (socket says 172.21.15.103:57074) 2024-09-19T07:00:38.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f4883fff640 1 -- 172.21.15.103:0/1882184077 learned_addr learned my addr 172.21.15.103:0/1882184077 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:38.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f48837fe640 1 --2- 172.21.15.103:0/1882184077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da695d0 0x557e9da783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f487bfff640 1 --2- 172.21.15.103:0/1882184077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da6ab40 0x557e9da788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f4883fff640 1 -- 172.21.15.103:0/1882184077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da695d0 msgr2=0x557e9da783a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:38.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f4883fff640 1 --2- 172.21.15.103:0/1882184077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da695d0 0x557e9da783a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f4883fff640 1 -- 172.21.15.103:0/1882184077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da6ab40 msgr2=0x557e9da788e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:38.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f4883fff640 1 --2- 172.21.15.103:0/1882184077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da6ab40 0x557e9da788e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f4883fff640 1 -- 172.21.15.103:0/1882184077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9daedfe0 con 0x557e9da74370 2024-09-19T07:00:38.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f487bfff640 1 --2- 172.21.15.103:0/1882184077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da6ab40 0x557e9da788e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:38.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f4883fff640 1 --2- 172.21.15.103:0/1882184077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da74370 0x557e9da78e20 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f487c098910 tx=0x7f487c013af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cc81961f6ef662 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:38.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f4862ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:38.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f4862ffd640 1 -- 172.21.15.103:0/1882184077 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c0b9020 con 0x557e9da74370 2024-09-19T07:00:38.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f4862ffd640 1 -- 172.21.15.103:0/1882184077 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f487c09c710 con 0x557e9da74370 2024-09-19T07:00:38.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.652+0000 7f488c359740 1 -- 172.21.15.103:0/1882184077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f486c02fb00 con 0x557e9da74370 2024-09-19T07:00:38.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.653+0000 7f4862ffd640 1 -- 172.21.15.103:0/1882184077 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f487c00d340 con 0x557e9da74370 2024-09-19T07:00:38.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.653+0000 7f488c359740 10 client.19332 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:38.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.653+0000 7f488c359740 1 -- 172.21.15.103:0/1882184077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557e9daef1f0 con 0x557e9da74370 2024-09-19T07:00:38.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.653+0000 7f4862ffd640 1 -- 172.21.15.103:0/1882184077 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f486c02fb00 con 0x557e9da74370 2024-09-19T07:00:38.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.653+0000 7f4862ffd640 1 -- 172.21.15.103:0/1882184077 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f487c020030 con 0x557e9da74370 2024-09-19T07:00:38.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.653+0000 7f488c359740 10 client.19332 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:38.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.653+0000 7f488c359740 10 client.19332 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:00:38.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.653+0000 7f488c359740 20 client.19332 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:38.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.653+0000 7f488c359740 1 --2- 172.21.15.103:0/1882184077 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x557e9daef530 0x557e9daf1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:38.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.653+0000 7f488c359740 4 client.19332 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:00:38.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.653+0000 7f488c359740 1 -- 172.21.15.103:0/1882184077 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e9d9d6d70 con 0x557e9daef530 2024-09-19T07:00:38.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.653+0000 7f48837fe640 1 --2- 172.21.15.103:0/1882184077 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x557e9daef530 0x557e9daf1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:38.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f4862ffd640 10 client.19332 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:00:38.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f48837fe640 1 --2- 172.21.15.103:0/1882184077 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x557e9daef530 0x557e9daf1920 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9983d754574ee4f0 server_cookie=f5bf7f65878f920d in_seq=0 out_seq=0 2024-09-19T07:00:38.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f4862ffd640 1 -- 172.21.15.103:0/1882184077 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e9d9d6d70 con 0x557e9daef530 2024-09-19T07:00:38.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f4862ffd640 10 client.19332 handle_command_reply: tid=0 2024-09-19T07:00:38.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f488c359740 1 client.19332 shutdown 2024-09-19T07:00:38.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f488c359740 1 -- 172.21.15.103:0/1882184077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da74370 msgr2=0x557e9da78e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:38.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f488c359740 1 --2- 172.21.15.103:0/1882184077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da74370 0x557e9da78e20 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f487c098910 tx=0x7f487c013af0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f488c359740 1 -- 172.21.15.103:0/1882184077 shutdown_connections 2024-09-19T07:00:38.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f488c359740 1 --2- 172.21.15.103:0/1882184077 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x557e9daef530 0x557e9daf1920 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f488c359740 1 --2- 172.21.15.103:0/1882184077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e9da74370 0x557e9da78e20 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f488c359740 1 --2- 172.21.15.103:0/1882184077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e9da6ab40 0x557e9da788e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f488c359740 1 --2- 172.21.15.103:0/1882184077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e9da695d0 0x557e9da783a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f488c359740 1 -- 172.21.15.103:0/1882184077 >> 172.21.15.103:0/1882184077 conn(0x557e9da32310 msgr2=0x557e9da728e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:38.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f4862ffd640 0 client.19332 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:00:38.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f488c359740 1 -- 172.21.15.103:0/1882184077 shutdown_connections 2024-09-19T07:00:38.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f488c359740 1 -- 172.21.15.103:0/1882184077 wait complete. 2024-09-19T07:00:38.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.654+0000 7f488c359740 20 client.19332 trim_cache size 0 max 16384 2024-09-19T07:00:38.668 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:00:38.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.655+0000 7f4889f80640 1 -- 172.21.15.103:0/904991431 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48700a33b0 msgr2=0x7f48700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:38.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.655+0000 7f4889f80640 1 --2- 172.21.15.103:0/904991431 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48700a33b0 0x7f48700a5870 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f48840feda0 tx=0x7f486c0156d0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.655+0000 7f4889f80640 1 -- 172.21.15.103:0/904991431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4884106a80 msgr2=0x7f488419cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:38.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.655+0000 7f4889f80640 1 --2- 172.21.15.103:0/904991431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4884106a80 0x7f488419cd90 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f487c00cce0 tx=0x7f487c00a620 comp rx=0 tx=0).stop 2024-09-19T07:00:38.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.655+0000 7f4889f80640 1 -- 172.21.15.103:0/904991431 shutdown_connections 2024-09-19T07:00:38.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.655+0000 7f4889f80640 1 --2- 172.21.15.103:0/904991431 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48700a33b0 0x7f48700a5870 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.655+0000 7f4889f80640 1 --2- 172.21.15.103:0/904991431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4884106a80 0x7f488419cd90 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.655+0000 7f4889f80640 1 --2- 172.21.15.103:0/904991431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4884101420 0x7f4884198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.655+0000 7f4889f80640 1 --2- 172.21.15.103:0/904991431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4884100a80 0x7f48841984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:38.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.655+0000 7f4889f80640 1 -- 172.21.15.103:0/904991431 >> 172.21.15.103:0/904991431 conn(0x7f48840fc7a0 msgr2=0x7f48840fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:38.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.655+0000 7f4889f80640 1 -- 172.21.15.103:0/904991431 shutdown_connections 2024-09-19T07:00:38.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:38.655+0000 7f4889f80640 1 -- 172.21.15.103:0/904991431 wait complete. 2024-09-19T07:00:39.356 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:00:39.603 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:00:40.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.053+0000 7fd49b52c640 1 -- 172.21.15.103:0/437676187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd494103660 msgr2=0x7fd49410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:40.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.053+0000 7fd49b52c640 1 --2- 172.21.15.103:0/437676187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd494103660 0x7fd49410bb50 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fd484009900 tx=0x7fd48402a480 comp rx=0 tx=0).stop 2024-09-19T07:00:40.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.053+0000 7fd49b52c640 1 -- 172.21.15.103:0/437676187 shutdown_connections 2024-09-19T07:00:40.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.053+0000 7fd49b52c640 1 --2- 172.21.15.103:0/437676187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd49410c170 0x7fd49410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.053+0000 7fd49b52c640 1 --2- 172.21.15.103:0/437676187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd494103660 0x7fd49410bb50 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.053+0000 7fd49b52c640 1 --2- 172.21.15.103:0/437676187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd494102d40 0x7fd494103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.053+0000 7fd49b52c640 1 -- 172.21.15.103:0/437676187 >> 172.21.15.103:0/437676187 conn(0x7fd4940fc7a0 msgr2=0x7fd4940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:40.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.053+0000 7fd49b52c640 1 -- 172.21.15.103:0/437676187 shutdown_connections 2024-09-19T07:00:40.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.053+0000 7fd49b52c640 1 -- 172.21.15.103:0/437676187 wait complete. 2024-09-19T07:00:40.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.054+0000 7fd49b52c640 1 Processor -- start 2024-09-19T07:00:40.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.054+0000 7fd49b52c640 1 -- start start 2024-09-19T07:00:40.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.054+0000 7fd49b52c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd494102d40 0x7fd494198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.054+0000 7fd49b52c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd494103660 0x7fd494198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.054+0000 7fd49b52c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd49410c170 0x7fd49419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.054+0000 7fd49b52c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd494111f70 con 0x7fd494102d40 2024-09-19T07:00:40.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.054+0000 7fd49b52c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd494111df0 con 0x7fd494103660 2024-09-19T07:00:40.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.054+0000 7fd49b52c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd4941120f0 con 0x7fd49410c170 2024-09-19T07:00:40.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.054+0000 7fd4992c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd494102d40 0x7fd494198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:40.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.054+0000 7fd4992c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd494102d40 0x7fd494198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57092/0 (socket says 172.21.15.103:57092) 2024-09-19T07:00:40.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.054+0000 7fd4992c8640 1 -- 172.21.15.103:0/1587367075 learned_addr learned my addr 172.21.15.103:0/1587367075 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:40.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.055+0000 7fd498ac7640 1 --2- 172.21.15.103:0/1587367075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd494103660 0x7fd494198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:40.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.055+0000 7fd499ac9640 1 --2- 172.21.15.103:0/1587367075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd49410c170 0x7fd49419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:40.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.055+0000 7fd4992c8640 1 -- 172.21.15.103:0/1587367075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd49410c170 msgr2=0x7fd49419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:40.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.055+0000 7fd4992c8640 1 --2- 172.21.15.103:0/1587367075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd49410c170 0x7fd49419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.055+0000 7fd4992c8640 1 -- 172.21.15.103:0/1587367075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd494103660 msgr2=0x7fd494198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:40.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.055+0000 7fd4992c8640 1 --2- 172.21.15.103:0/1587367075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd494103660 0x7fd494198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.055+0000 7fd4992c8640 1 -- 172.21.15.103:0/1587367075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49419d3c0 con 0x7fd494102d40 2024-09-19T07:00:40.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.055+0000 7fd498ac7640 1 --2- 172.21.15.103:0/1587367075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd494103660 0x7fd494198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:40.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.055+0000 7fd4992c8640 1 --2- 172.21.15.103:0/1587367075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd494102d40 0x7fd494198410 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fd49000c920 tx=0x7fd49000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:40.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.055+0000 7fd48a7fc640 1 -- 172.21.15.103:0/1587367075 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490007c70 con 0x7fd494102d40 2024-09-19T07:00:40.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.055+0000 7fd49b52c640 1 -- 172.21.15.103:0/1587367075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49419d6b0 con 0x7fd494102d40 2024-09-19T07:00:40.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.055+0000 7fd48a7fc640 1 -- 172.21.15.103:0/1587367075 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd490007e10 con 0x7fd494102d40 2024-09-19T07:00:40.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.055+0000 7fd48a7fc640 1 -- 172.21.15.103:0/1587367075 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490018890 con 0x7fd494102d40 2024-09-19T07:00:40.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.055+0000 7fd49b52c640 1 -- 172.21.15.103:0/1587367075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4941a4f90 con 0x7fd494102d40 2024-09-19T07:00:40.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.057+0000 7fd48a7fc640 1 -- 172.21.15.103:0/1587367075 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd490018a70 con 0x7fd494102d40 2024-09-19T07:00:40.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.058+0000 7fd49d905740 1 Processor -- start 2024-09-19T07:00:40.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.058+0000 7fd49d905740 1 -- start start 2024-09-19T07:00:40.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.058+0000 7fd48a7fc640 1 --2- 172.21.15.103:0/1587367075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd4800a33b0 0x7fd4800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.058+0000 7fd49d905740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e09b40 0x559df3e08090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.058+0000 7fd49d905740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e085d0 0x559df3e08a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.058+0000 7fd49d905740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e17780 0x559df3e19b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.058+0000 7fd49d905740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559df3d6e2f0 con 0x559df3e09b40 2024-09-19T07:00:40.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.058+0000 7fd49d905740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559df3d15cf0 con 0x559df3e17780 2024-09-19T07:00:40.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.058+0000 7fd49d905740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559df3d6a2b0 con 0x559df3e085d0 2024-09-19T07:00:40.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.058+0000 7fd499ac9640 1 --2- 172.21.15.103:0/1587367075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd4800a33b0 0x7fd4800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:40.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.058+0000 7fd4992c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e09b40 0x559df3e08090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:40.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.058+0000 7fd4992c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e09b40 0x559df3e08090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57102/0 (socket says 172.21.15.103:57102) 2024-09-19T07:00:40.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.058+0000 7fd4992c8640 1 -- 172.21.15.103:0/1503599004 learned_addr learned my addr 172.21.15.103:0/1503599004 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:40.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.058+0000 7fd48a7fc640 1 -- 172.21.15.103:0/1587367075 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fd4900cff60 con 0x7fd494102d40 2024-09-19T07:00:40.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.058+0000 7fd498ac7640 1 --2- 172.21.15.103:0/1503599004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e085d0 0x559df3e08a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:40.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.059+0000 7fd499ac9640 1 --2- 172.21.15.103:0/1503599004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e17780 0x559df3e19b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:40.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.059+0000 7fd4992c8640 1 -- 172.21.15.103:0/1503599004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e085d0 msgr2=0x559df3e08a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:40.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.059+0000 7fd4992c8640 1 --2- 172.21.15.103:0/1503599004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e085d0 0x559df3e08a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.059+0000 7fd4992c8640 1 -- 172.21.15.103:0/1503599004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e17780 msgr2=0x559df3e19b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:40.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.059+0000 7fd4992c8640 1 --2- 172.21.15.103:0/1503599004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e17780 0x559df3e19b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.059+0000 7fd4992c8640 1 -- 172.21.15.103:0/1503599004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559df3cfa3a0 con 0x559df3e09b40 2024-09-19T07:00:40.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.059+0000 7fd499ac9640 1 --2- 172.21.15.103:0/1503599004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e17780 0x559df3e19b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:40.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.059+0000 7fd4992c8640 1 --2- 172.21.15.103:0/1503599004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e09b40 0x559df3e08090 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fd490007590 tx=0x7fd49001dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42a932e5da32388 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:40.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.059+0000 7fd499ac9640 1 --2- 172.21.15.103:0/1587367075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd4800a33b0 0x7fd4800a5870 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x559df3e09070 tx=0x7fd48c0089a0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:40.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.059+0000 7fd47f7fe640 1 -- 172.21.15.103:0/1503599004 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49009c900 con 0x559df3e09b40 2024-09-19T07:00:40.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.059+0000 7fd47f7fe640 1 -- 172.21.15.103:0/1503599004 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd49009c200 con 0x559df3e09b40 2024-09-19T07:00:40.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.059+0000 7fd47f7fe640 1 -- 172.21.15.103:0/1503599004 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49000fb30 con 0x559df3e09b40 2024-09-19T07:00:40.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.060+0000 7fd49d905740 1 -- 172.21.15.103:0/1503599004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e09b40 msgr2=0x559df3e08090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:40.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.060+0000 7fd49d905740 1 --2- 172.21.15.103:0/1503599004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e09b40 0x559df3e08090 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fd490007590 tx=0x7fd49001dcd0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.060+0000 7fd49d905740 1 -- 172.21.15.103:0/1503599004 shutdown_connections 2024-09-19T07:00:40.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.060+0000 7fd49d905740 1 --2- 172.21.15.103:0/1503599004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e17780 0x559df3e19b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.060+0000 7fd49d905740 1 --2- 172.21.15.103:0/1503599004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e085d0 0x559df3e08a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.060+0000 7fd49d905740 1 --2- 172.21.15.103:0/1503599004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e09b40 0x559df3e08090 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.060+0000 7fd49d905740 1 -- 172.21.15.103:0/1503599004 >> 172.21.15.103:0/1503599004 conn(0x559df3dd1310 msgr2=0x559df3e04c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:40.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.060+0000 7fd49d905740 1 -- 172.21.15.103:0/1503599004 shutdown_connections 2024-09-19T07:00:40.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.060+0000 7fd49d905740 1 -- 172.21.15.103:0/1503599004 wait complete. 2024-09-19T07:00:40.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.060+0000 7fd49d905740 1 -- start start 2024-09-19T07:00:40.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.060+0000 7fd49d905740 1 Processor -- start 2024-09-19T07:00:40.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd49d905740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e085d0 0x559df3e17180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd49d905740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e09b40 0x559df3e14430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd49d905740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e17780 0x559df3e14a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd49d905740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559df3e1a480 con 0x559df3e085d0 2024-09-19T07:00:40.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd49d905740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559df3d29060 con 0x559df3e09b40 2024-09-19T07:00:40.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd49d905740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559df3e1a600 con 0x559df3e17780 2024-09-19T07:00:40.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd49d905740 10 client.0 _ll_register_callbacks cb 0x559df3a7c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:40.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd4992c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e085d0 0x559df3e17180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:40.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd4992c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e085d0 0x559df3e17180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57108/0 (socket says 172.21.15.103:57108) 2024-09-19T07:00:40.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd4992c8640 1 -- 172.21.15.103:0/3345548693 learned_addr learned my addr 172.21.15.103:0/3345548693 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:40.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd4992c8640 1 -- 172.21.15.103:0/3345548693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e17780 msgr2=0x559df3e14a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:40.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd499ac9640 1 --2- 172.21.15.103:0/3345548693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e17780 0x559df3e14a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:40.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd498ac7640 1 --2- 172.21.15.103:0/3345548693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e09b40 0x559df3e14430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:40.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd4992c8640 1 --2- 172.21.15.103:0/3345548693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e17780 0x559df3e14a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd4992c8640 1 -- 172.21.15.103:0/3345548693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e09b40 msgr2=0x559df3e14430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:40.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd4992c8640 1 --2- 172.21.15.103:0/3345548693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e09b40 0x559df3e14430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd4992c8640 1 -- 172.21.15.103:0/3345548693 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559df3adf3c0 con 0x559df3e085d0 2024-09-19T07:00:40.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd499ac9640 1 --2- 172.21.15.103:0/3345548693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e17780 0x559df3e14a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:40.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd498ac7640 1 --2- 172.21.15.103:0/3345548693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e09b40 0x559df3e14430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:40.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd4992c8640 1 --2- 172.21.15.103:0/3345548693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e085d0 0x559df3e17180 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fd49009c140 tx=0x7fd4900a0700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4bac219cf89943 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:40.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd47cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:40.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.061+0000 7fd47cff9640 1 -- 172.21.15.103:0/3345548693 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490014210 con 0x559df3e085d0 2024-09-19T07:00:40.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.062+0000 7fd47cff9640 1 -- 172.21.15.103:0/3345548693 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fd4900610b0 con 0x559df3e085d0 2024-09-19T07:00:40.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.062+0000 7fd49d905740 1 -- 172.21.15.103:0/3345548693 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd49009c200 con 0x559df3e085d0 2024-09-19T07:00:40.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.062+0000 7fd47cff9640 1 -- 172.21.15.103:0/3345548693 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd49009c900 con 0x559df3e085d0 2024-09-19T07:00:40.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.062+0000 7fd49d905740 10 client.19344 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:40.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.062+0000 7fd49d905740 1 -- 172.21.15.103:0/3345548693 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559df3e25e90 con 0x559df3e085d0 2024-09-19T07:00:40.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.062+0000 7fd47cff9640 1 -- 172.21.15.103:0/3345548693 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd49009c200 con 0x559df3e085d0 2024-09-19T07:00:40.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.062+0000 7fd47cff9640 1 -- 172.21.15.103:0/3345548693 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd49005fce0 con 0x559df3e085d0 2024-09-19T07:00:40.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.062+0000 7fd49d905740 10 client.19344 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:40.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.062+0000 7fd49d905740 10 client.19344 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:00:40.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.062+0000 7fd49d905740 20 client.19344 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:40.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.062+0000 7fd49d905740 1 --2- 172.21.15.103:0/3345548693 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x559df3e271e0 0x559df3e295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.062+0000 7fd49d905740 4 client.19344 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:00:40.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.062+0000 7fd49d905740 1 -- 172.21.15.103:0/3345548693 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559df3d7b860 con 0x559df3e271e0 2024-09-19T07:00:40.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.063+0000 7fd498ac7640 1 --2- 172.21.15.103:0/3345548693 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x559df3e271e0 0x559df3e295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:40.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.063+0000 7fd498ac7640 1 --2- 172.21.15.103:0/3345548693 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x559df3e271e0 0x559df3e295d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c91d6f77b28fbdd3 server_cookie=aefff5451081f5a4 in_seq=0 out_seq=0 2024-09-19T07:00:40.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.063+0000 7fd47cff9640 10 client.19344 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:00:40.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.064+0000 7fd47cff9640 1 -- 172.21.15.103:0/3345548693 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559df3d7b860 con 0x559df3e271e0 2024-09-19T07:00:40.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.064+0000 7fd47cff9640 10 client.19344 handle_command_reply: tid=0 2024-09-19T07:00:40.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.064+0000 7fd49d905740 1 client.19344 shutdown 2024-09-19T07:00:40.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.064+0000 7fd49d905740 1 -- 172.21.15.103:0/3345548693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e085d0 msgr2=0x559df3e17180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:40.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.064+0000 7fd49d905740 1 --2- 172.21.15.103:0/3345548693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e085d0 0x559df3e17180 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fd49009c140 tx=0x7fd4900a0700 comp rx=0 tx=0).stop 2024-09-19T07:00:40.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.064+0000 7fd49d905740 1 -- 172.21.15.103:0/3345548693 shutdown_connections 2024-09-19T07:00:40.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.064+0000 7fd49d905740 1 --2- 172.21.15.103:0/3345548693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e17780 0x559df3e14a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.064+0000 7fd49d905740 1 --2- 172.21.15.103:0/3345548693 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x559df3e271e0 0x559df3e295d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.064+0000 7fd49d905740 1 --2- 172.21.15.103:0/3345548693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e09b40 0x559df3e14430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.064+0000 7fd49d905740 1 --2- 172.21.15.103:0/3345548693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e085d0 0x559df3e17180 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.064+0000 7fd47cff9640 0 client.19344 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:00:40.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.064+0000 7fd49d905740 1 -- 172.21.15.103:0/3345548693 >> 172.21.15.103:0/3345548693 conn(0x559df3dd1310 msgr2=0x559df3e0a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:40.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.064+0000 7fd49d905740 1 -- 172.21.15.103:0/3345548693 shutdown_connections 2024-09-19T07:00:40.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.064+0000 7fd49d905740 1 -- 172.21.15.103:0/3345548693 wait complete. 2024-09-19T07:00:40.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.064+0000 7fd49d905740 20 client.19344 trim_cache size 0 max 16384 2024-09-19T07:00:40.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.077+0000 7fd49d905740 1 Processor -- start 2024-09-19T07:00:40.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.077+0000 7fd49d905740 1 -- start start 2024-09-19T07:00:40.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.077+0000 7fd49d905740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e085d0 0x559df3e13cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.077+0000 7fd49d905740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e09b40 0x559df3e2c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.077+0000 7fd49d905740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e17780 0x559df3e2cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.077+0000 7fd49d905740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd4941ab720 con 0x559df3e09b40 2024-09-19T07:00:40.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.077+0000 7fd49d905740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd4941ad260 con 0x559df3e17780 2024-09-19T07:00:40.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.077+0000 7fd49d905740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd4941abdf0 con 0x559df3e085d0 2024-09-19T07:00:40.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.077+0000 7fd498ac7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e09b40 0x559df3e2c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:40.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.077+0000 7fd498ac7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e09b40 0x559df3e2c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57118/0 (socket says 172.21.15.103:57118) 2024-09-19T07:00:40.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.077+0000 7fd498ac7640 1 -- 172.21.15.103:0/152505745 learned_addr learned my addr 172.21.15.103:0/152505745 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.077+0000 7fd498ac7640 1 -- 172.21.15.103:0/152505745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e085d0 msgr2=0x559df3e13cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.077+0000 7fd498ac7640 1 --2- 172.21.15.103:0/152505745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e085d0 0x559df3e13cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.077+0000 7fd498ac7640 1 -- 172.21.15.103:0/152505745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e17780 msgr2=0x559df3e2cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.077+0000 7fd498ac7640 1 --2- 172.21.15.103:0/152505745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e17780 0x559df3e2cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.077+0000 7fd498ac7640 1 -- 172.21.15.103:0/152505745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559df3e14d90 con 0x559df3e09b40 2024-09-19T07:00:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.078+0000 7fd498ac7640 1 --2- 172.21.15.103:0/152505745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e09b40 0x559df3e2c5f0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fd48402a9a0 tx=0x7fd4840058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea06234549d64321 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:40.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.078+0000 7fd47effd640 1 -- 172.21.15.103:0/152505745 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484048070 con 0x559df3e09b40 2024-09-19T07:00:40.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.078+0000 7fd47effd640 1 -- 172.21.15.103:0/152505745 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd484031a50 con 0x559df3e09b40 2024-09-19T07:00:40.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.078+0000 7fd47effd640 1 -- 172.21.15.103:0/152505745 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484035040 con 0x559df3e09b40 2024-09-19T07:00:40.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.078+0000 7fd49d905740 1 -- 172.21.15.103:0/152505745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e09b40 msgr2=0x559df3e2c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:40.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.078+0000 7fd49d905740 1 --2- 172.21.15.103:0/152505745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e09b40 0x559df3e2c5f0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fd48402a9a0 tx=0x7fd4840058b0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.078+0000 7fd49d905740 1 -- 172.21.15.103:0/152505745 shutdown_connections 2024-09-19T07:00:40.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.078+0000 7fd49d905740 1 --2- 172.21.15.103:0/152505745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e17780 0x559df3e2cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.078+0000 7fd49d905740 1 --2- 172.21.15.103:0/152505745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e09b40 0x559df3e2c5f0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.078+0000 7fd49d905740 1 --2- 172.21.15.103:0/152505745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e085d0 0x559df3e13cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.078+0000 7fd49d905740 1 -- 172.21.15.103:0/152505745 >> 172.21.15.103:0/152505745 conn(0x559df3dd1310 msgr2=0x559df3e12d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:40.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.079+0000 7fd49d905740 1 -- 172.21.15.103:0/152505745 shutdown_connections 2024-09-19T07:00:40.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.079+0000 7fd49d905740 1 -- 172.21.15.103:0/152505745 wait complete. 2024-09-19T07:00:40.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.079+0000 7fd49d905740 1 -- start start 2024-09-19T07:00:40.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.079+0000 7fd49d905740 1 Processor -- start 2024-09-19T07:00:40.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.079+0000 7fd49d905740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e085d0 0x559df3e89f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.079+0000 7fd49d905740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e09b40 0x559df3e8a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.079+0000 7fd49d905740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e17780 0x559df3e29bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.079+0000 7fd49d905740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd4941acb90 con 0x559df3e085d0 2024-09-19T07:00:40.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.079+0000 7fd49d905740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd4941ac4c0 con 0x559df3e17780 2024-09-19T07:00:40.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.079+0000 7fd49d905740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd4941a9720 con 0x559df3e09b40 2024-09-19T07:00:40.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.079+0000 7fd49d905740 10 client.0 _ll_register_callbacks cb 0x7fd49406b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:40.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.079+0000 7fd4992c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e085d0 0x559df3e89f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:40.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.079+0000 7fd4992c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e085d0 0x559df3e89f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57124/0 (socket says 172.21.15.103:57124) 2024-09-19T07:00:40.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.079+0000 7fd4992c8640 1 -- 172.21.15.103:0/1641642576 learned_addr learned my addr 172.21.15.103:0/1641642576 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:40.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.080+0000 7fd499ac9640 1 --2- 172.21.15.103:0/1641642576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e17780 0x559df3e29bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:40.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.080+0000 7fd498ac7640 1 --2- 172.21.15.103:0/1641642576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e09b40 0x559df3e8a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:40.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.080+0000 7fd4992c8640 1 -- 172.21.15.103:0/1641642576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e09b40 msgr2=0x559df3e8a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:40.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.080+0000 7fd4992c8640 1 --2- 172.21.15.103:0/1641642576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e09b40 0x559df3e8a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.080+0000 7fd4992c8640 1 -- 172.21.15.103:0/1641642576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e17780 msgr2=0x559df3e29bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:40.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.080+0000 7fd4992c8640 1 --2- 172.21.15.103:0/1641642576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e17780 0x559df3e29bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.080+0000 7fd4992c8640 1 -- 172.21.15.103:0/1641642576 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559df3e2a3f0 con 0x559df3e085d0 2024-09-19T07:00:40.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.080+0000 7fd4992c8640 1 --2- 172.21.15.103:0/1641642576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e085d0 0x559df3e89f30 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fd49009bf00 tx=0x7fd49000ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be46bda9339b393 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:40.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.080+0000 7fd47cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:40.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.080+0000 7fd47cff9640 1 -- 172.21.15.103:0/1641642576 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4900a0170 con 0x559df3e085d0 2024-09-19T07:00:40.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.080+0000 7fd47cff9640 1 -- 172.21.15.103:0/1641642576 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fd4900610b0 con 0x559df3e085d0 2024-09-19T07:00:40.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.080+0000 7fd49d905740 1 -- 172.21.15.103:0/1641642576 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd484031a50 con 0x559df3e085d0 2024-09-19T07:00:40.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.080+0000 7fd47cff9640 1 -- 172.21.15.103:0/1641642576 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd49009c200 con 0x559df3e085d0 2024-09-19T07:00:40.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.080+0000 7fd49d905740 10 client.19350 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:40.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.080+0000 7fd49d905740 1 -- 172.21.15.103:0/1641642576 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559df3e91690 con 0x559df3e085d0 2024-09-19T07:00:40.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.080+0000 7fd47cff9640 1 -- 172.21.15.103:0/1641642576 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd484031a50 con 0x559df3e085d0 2024-09-19T07:00:40.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.081+0000 7fd47cff9640 1 -- 172.21.15.103:0/1641642576 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd4900bc0d0 con 0x559df3e085d0 2024-09-19T07:00:40.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.081+0000 7fd49d905740 10 client.19350 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:40.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.081+0000 7fd49d905740 10 client.19350 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:00:40.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.081+0000 7fd49d905740 20 client.19350 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:40.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.081+0000 7fd49d905740 1 --2- 172.21.15.103:0/1641642576 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x559df3e91bc0 0x559df3e93fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:40.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.081+0000 7fd49d905740 4 client.19350 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:00:40.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.081+0000 7fd49d905740 1 -- 172.21.15.103:0/1641642576 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559df3d75d70 con 0x559df3e91bc0 2024-09-19T07:00:40.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.081+0000 7fd498ac7640 1 --2- 172.21.15.103:0/1641642576 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x559df3e91bc0 0x559df3e93fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:40.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.081+0000 7fd47cff9640 10 client.19350 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:00:40.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.081+0000 7fd498ac7640 1 --2- 172.21.15.103:0/1641642576 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x559df3e91bc0 0x559df3e93fb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60d8035f082c420b server_cookie=cb2e0d13cc042309 in_seq=0 out_seq=0 2024-09-19T07:00:40.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd47cff9640 1 -- 172.21.15.103:0/1641642576 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559df3d75d70 con 0x559df3e91bc0 2024-09-19T07:00:40.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd47cff9640 10 client.19350 handle_command_reply: tid=0 2024-09-19T07:00:40.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd49d905740 1 client.19350 shutdown 2024-09-19T07:00:40.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd49d905740 1 -- 172.21.15.103:0/1641642576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e085d0 msgr2=0x559df3e89f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:40.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd49d905740 1 --2- 172.21.15.103:0/1641642576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e085d0 0x559df3e89f30 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fd49009bf00 tx=0x7fd49000ff30 comp rx=0 tx=0).stop 2024-09-19T07:00:40.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd49d905740 1 -- 172.21.15.103:0/1641642576 shutdown_connections 2024-09-19T07:00:40.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd49d905740 1 --2- 172.21.15.103:0/1641642576 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x559df3e91bc0 0x559df3e93fb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd47cff9640 0 client.19350 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:00:40.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd49d905740 1 --2- 172.21.15.103:0/1641642576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559df3e17780 0x559df3e29bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd49d905740 1 --2- 172.21.15.103:0/1641642576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559df3e09b40 0x559df3e8a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd49d905740 1 --2- 172.21.15.103:0/1641642576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559df3e085d0 0x559df3e89f30 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd49d905740 1 -- 172.21.15.103:0/1641642576 >> 172.21.15.103:0/1641642576 conn(0x559df3dd1310 msgr2=0x559df3e0a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:40.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd49d905740 1 -- 172.21.15.103:0/1641642576 shutdown_connections 2024-09-19T07:00:40.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd49d905740 1 -- 172.21.15.103:0/1641642576 wait complete. 2024-09-19T07:00:40.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd49d905740 20 client.19350 trim_cache size 0 max 16384 2024-09-19T07:00:40.089 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:00:40.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd49b52c640 1 -- 172.21.15.103:0/1587367075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd4800a33b0 msgr2=0x7fd4800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:40.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd49b52c640 1 --2- 172.21.15.103:0/1587367075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd4800a33b0 0x7fd4800a5870 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x559df3e09070 tx=0x7fd48c0089a0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd49b52c640 1 -- 172.21.15.103:0/1587367075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd494102d40 msgr2=0x7fd494198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:40.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.082+0000 7fd49b52c640 1 --2- 172.21.15.103:0/1587367075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd494102d40 0x7fd494198410 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fd49000c920 tx=0x7fd49000cde0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.083+0000 7fd49b52c640 1 -- 172.21.15.103:0/1587367075 shutdown_connections 2024-09-19T07:00:40.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.083+0000 7fd49b52c640 1 --2- 172.21.15.103:0/1587367075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd4800a33b0 0x7fd4800a5870 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.083+0000 7fd49b52c640 1 --2- 172.21.15.103:0/1587367075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd49410c170 0x7fd49419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.083+0000 7fd49b52c640 1 --2- 172.21.15.103:0/1587367075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd494103660 0x7fd494198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.083+0000 7fd49b52c640 1 --2- 172.21.15.103:0/1587367075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd494102d40 0x7fd494198410 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:40.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.083+0000 7fd49b52c640 1 -- 172.21.15.103:0/1587367075 >> 172.21.15.103:0/1587367075 conn(0x7fd4940fc7a0 msgr2=0x7fd4940fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:40.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.083+0000 7fd49b52c640 1 -- 172.21.15.103:0/1587367075 shutdown_connections 2024-09-19T07:00:40.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:40.083+0000 7fd49b52c640 1 -- 172.21.15.103:0/1587367075 wait complete. 2024-09-19T07:00:40.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:39 smithi174 ceph-mon[35637]: pgmap v355: 97 pgs: 97 active+clean; 3.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-19T07:00:40.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:39 smithi103 ceph-mon[31158]: pgmap v355: 97 pgs: 97 active+clean; 3.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-19T07:00:40.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:39 smithi134 ceph-mon[35356]: pgmap v355: 97 pgs: 97 active+clean; 3.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-19T07:00:40.741 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:00:40.998 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:00:41.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.445+0000 7fcda83c6640 1 -- 172.21.15.103:0/2895076837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda0106f10 msgr2=0x7fcda010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:41.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.445+0000 7fcda83c6640 1 --2- 172.21.15.103:0/2895076837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda0106f10 0x7fcda010ea90 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fcd98009ca0 tx=0x7fcd9802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.445+0000 7fcda83c6640 1 -- 172.21.15.103:0/2895076837 shutdown_connections 2024-09-19T07:00:41.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.445+0000 7fcda83c6640 1 --2- 172.21.15.103:0/2895076837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda0106f10 0x7fcda010ea90 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.445+0000 7fcda83c6640 1 --2- 172.21.15.103:0/2895076837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda0102790 0x7fcda01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.445+0000 7fcda83c6640 1 --2- 172.21.15.103:0/2895076837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcda0101de0 0x7fcda01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.445+0000 7fcda83c6640 1 -- 172.21.15.103:0/2895076837 >> 172.21.15.103:0/2895076837 conn(0x7fcda00fd8c0 msgr2=0x7fcda00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:41.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.445+0000 7fcda83c6640 1 -- 172.21.15.103:0/2895076837 shutdown_connections 2024-09-19T07:00:41.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.445+0000 7fcda83c6640 1 -- 172.21.15.103:0/2895076837 wait complete. 2024-09-19T07:00:41.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda83c6640 1 Processor -- start 2024-09-19T07:00:41.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda83c6640 1 -- start start 2024-09-19T07:00:41.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda83c6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcda0101de0 0x7fcda00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda83c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda0102790 0x7fcda0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda83c6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda0106f10 0x7fcda0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda83c6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcda0112010 con 0x7fcda0106f10 2024-09-19T07:00:41.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda83c6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcda0111e90 con 0x7fcda0102790 2024-09-19T07:00:41.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda83c6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcda0112190 con 0x7fcda0101de0 2024-09-19T07:00:41.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda6963640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda0106f10 0x7fcda0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:41.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda6963640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda0106f10 0x7fcda0074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57138/0 (socket says 172.21.15.103:57138) 2024-09-19T07:00:41.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda6963640 1 -- 172.21.15.103:0/2482319547 learned_addr learned my addr 172.21.15.103:0/2482319547 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:41.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda6162640 1 --2- 172.21.15.103:0/2482319547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcda0101de0 0x7fcda00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:41.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda6963640 1 -- 172.21.15.103:0/2482319547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcda0101de0 msgr2=0x7fcda00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:41.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda6963640 1 --2- 172.21.15.103:0/2482319547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcda0101de0 0x7fcda00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda6963640 1 -- 172.21.15.103:0/2482319547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda0102790 msgr2=0x7fcda0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:41.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda6963640 1 --2- 172.21.15.103:0/2482319547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda0102790 0x7fcda0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda6963640 1 -- 172.21.15.103:0/2482319547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda01a6ab0 con 0x7fcda0106f10 2024-09-19T07:00:41.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda6162640 1 --2- 172.21.15.103:0/2482319547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcda0101de0 0x7fcda00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:41.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.446+0000 7fcda6963640 1 --2- 172.21.15.103:0/2482319547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda0106f10 0x7fcda0074f70 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fcd98031a60 tx=0x7fcd98031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:41.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.447+0000 7fcd977fe640 1 -- 172.21.15.103:0/2482319547 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98003c30 con 0x7fcda0106f10 2024-09-19T07:00:41.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.447+0000 7fcd977fe640 1 -- 172.21.15.103:0/2482319547 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd98003dd0 con 0x7fcda0106f10 2024-09-19T07:00:41.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.447+0000 7fcda83c6640 1 -- 172.21.15.103:0/2482319547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda01a6d40 con 0x7fcda0106f10 2024-09-19T07:00:41.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.447+0000 7fcd977fe640 1 -- 172.21.15.103:0/2482319547 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd980047e0 con 0x7fcda0106f10 2024-09-19T07:00:41.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.447+0000 7fcda83c6640 1 -- 172.21.15.103:0/2482319547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda01a7280 con 0x7fcda0106f10 2024-09-19T07:00:41.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.449+0000 7fcd977fe640 1 -- 172.21.15.103:0/2482319547 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fcd9802f790 con 0x7fcda0106f10 2024-09-19T07:00:41.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.449+0000 7fcdaa79f740 1 Processor -- start 2024-09-19T07:00:41.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.449+0000 7fcdaa79f740 1 -- start start 2024-09-19T07:00:41.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcdaa79f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4db40 0x558af9c4c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcdaa79f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4c5d0 0x558af9c4ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcdaa79f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c57370 0x558af9c59760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcdaa79f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558af9bb22f0 con 0x558af9c4c5d0 2024-09-19T07:00:41.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcdaa79f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558af9b59cf0 con 0x558af9c4db40 2024-09-19T07:00:41.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcdaa79f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558af9bae2b0 con 0x558af9c57370 2024-09-19T07:00:41.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcd977fe640 1 --2- 172.21.15.103:0/2482319547 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd8c0a33b0 0x7fcd8c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcda6162640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4c5d0 0x558af9c4ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:41.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcda6162640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4c5d0 0x558af9c4ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57140/0 (socket says 172.21.15.103:57140) 2024-09-19T07:00:41.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcda6162640 1 -- 172.21.15.103:0/4151830479 learned_addr learned my addr 172.21.15.103:0/4151830479 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:41.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcda6963640 1 --2- 172.21.15.103:0/2482319547 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd8c0a33b0 0x7fcd8c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:41.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcda5961640 1 --2- 172.21.15.103:0/4151830479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c57370 0x558af9c59760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:41.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcd977fe640 1 -- 172.21.15.103:0/2482319547 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fcd980f10c0 con 0x7fcda0106f10 2024-09-19T07:00:41.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcda5961640 1 --2- 172.21.15.103:0/4151830479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4db40 0x558af9c4c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:41.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcda6162640 1 -- 172.21.15.103:0/4151830479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c57370 msgr2=0x558af9c59760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:41.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcda6162640 1 --2- 172.21.15.103:0/4151830479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c57370 0x558af9c59760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcda6162640 1 -- 172.21.15.103:0/4151830479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4db40 msgr2=0x558af9c4c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:41.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcda6162640 1 --2- 172.21.15.103:0/4151830479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4db40 0x558af9c4c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcda6162640 1 -- 172.21.15.103:0/4151830479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af9b3e3a0 con 0x558af9c4c5d0 2024-09-19T07:00:41.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcda5961640 1 --2- 172.21.15.103:0/4151830479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4db40 0x558af9c4c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:41.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcda6162640 1 --2- 172.21.15.103:0/4151830479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4c5d0 0x558af9c4ca10 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000aa10 tx=0x7fcd9000aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e226868b16a39d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:41.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcda6963640 1 --2- 172.21.15.103:0/2482319547 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd8c0a33b0 0x7fcd8c0a5870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcd980b9e60 tx=0x7fcd980b9040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:41.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.450+0000 7fcd94dea640 1 -- 172.21.15.103:0/4151830479 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd90015070 con 0x558af9c4c5d0 2024-09-19T07:00:41.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.451+0000 7fcd94dea640 1 -- 172.21.15.103:0/4151830479 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd9000cc60 con 0x558af9c4c5d0 2024-09-19T07:00:41.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.451+0000 7fcd94dea640 1 -- 172.21.15.103:0/4151830479 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd90018760 con 0x558af9c4c5d0 2024-09-19T07:00:41.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.451+0000 7fcdaa79f740 1 -- 172.21.15.103:0/4151830479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4c5d0 msgr2=0x558af9c4ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:41.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.451+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/4151830479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4c5d0 0x558af9c4ca10 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000aa10 tx=0x7fcd9000aed0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.451+0000 7fcdaa79f740 1 -- 172.21.15.103:0/4151830479 shutdown_connections 2024-09-19T07:00:41.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.451+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/4151830479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c57370 0x558af9c59760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.451+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/4151830479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4c5d0 0x558af9c4ca10 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.451+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/4151830479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4db40 0x558af9c4c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.451+0000 7fcdaa79f740 1 -- 172.21.15.103:0/4151830479 >> 172.21.15.103:0/4151830479 conn(0x558af9c15310 msgr2=0x558af9c48c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:41.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.451+0000 7fcdaa79f740 1 -- 172.21.15.103:0/4151830479 shutdown_connections 2024-09-19T07:00:41.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.451+0000 7fcdaa79f740 1 -- 172.21.15.103:0/4151830479 wait complete. 2024-09-19T07:00:41.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcdaa79f740 1 -- start start 2024-09-19T07:00:41.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcdaa79f740 1 Processor -- start 2024-09-19T07:00:41.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcdaa79f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4c5d0 0x558af9c56d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcdaa79f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4db40 0x558af9c53fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcdaa79f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c57370 0x558af9c54560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcdaa79f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558af9c5a070 con 0x558af9c4db40 2024-09-19T07:00:41.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcdaa79f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558af9b6d060 con 0x558af9c4c5d0 2024-09-19T07:00:41.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcdaa79f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558af9c5a1f0 con 0x558af9c57370 2024-09-19T07:00:41.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcdaa79f740 10 client.0 _ll_register_callbacks cb 0x558af98c08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:41.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcda6162640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4db40 0x558af9c53fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:41.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcda6162640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4db40 0x558af9c53fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57148/0 (socket says 172.21.15.103:57148) 2024-09-19T07:00:41.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcda6162640 1 -- 172.21.15.103:0/3638414935 learned_addr learned my addr 172.21.15.103:0/3638414935 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:41.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcda5961640 1 --2- 172.21.15.103:0/3638414935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c57370 0x558af9c54560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:41.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcda6162640 1 -- 172.21.15.103:0/3638414935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c57370 msgr2=0x558af9c54560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:41.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcda6162640 1 --2- 172.21.15.103:0/3638414935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c57370 0x558af9c54560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcda6162640 1 -- 172.21.15.103:0/3638414935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4c5d0 msgr2=0x558af9c56d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:41.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcda6162640 1 --2- 172.21.15.103:0/3638414935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4c5d0 0x558af9c56d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcda6162640 1 -- 172.21.15.103:0/3638414935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af99233c0 con 0x558af9c4db40 2024-09-19T07:00:41.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.452+0000 7fcda5961640 1 --2- 172.21.15.103:0/3638414935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c57370 0x558af9c54560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:41.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.453+0000 7fcda6162640 1 --2- 172.21.15.103:0/3638414935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4db40 0x558af9c53fc0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fcd90004700 tx=0x7fcd90011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879e25c0210a8027 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:41.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.453+0000 7fcd89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:41.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.453+0000 7fcd89ffb640 1 -- 172.21.15.103:0/3638414935 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd90015050 con 0x558af9c4db40 2024-09-19T07:00:41.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.453+0000 7fcd89ffb640 1 -- 172.21.15.103:0/3638414935 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fcd9001a450 con 0x558af9c4db40 2024-09-19T07:00:41.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.453+0000 7fcdaa79f740 1 -- 172.21.15.103:0/3638414935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd9000cc60 con 0x558af9c4db40 2024-09-19T07:00:41.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.453+0000 7fcd89ffb640 1 -- 172.21.15.103:0/3638414935 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcd9000a5e0 con 0x558af9c4db40 2024-09-19T07:00:41.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.453+0000 7fcdaa79f740 10 client.19362 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:41.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.453+0000 7fcdaa79f740 1 -- 172.21.15.103:0/3638414935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558af9c65a60 con 0x558af9c4db40 2024-09-19T07:00:41.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.453+0000 7fcd89ffb640 1 -- 172.21.15.103:0/3638414935 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd9000cc60 con 0x558af9c4db40 2024-09-19T07:00:41.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.453+0000 7fcd89ffb640 1 -- 172.21.15.103:0/3638414935 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcd90018830 con 0x558af9c4db40 2024-09-19T07:00:41.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.453+0000 7fcdaa79f740 10 client.19362 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:41.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.453+0000 7fcdaa79f740 10 client.19362 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:00:41.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.453+0000 7fcdaa79f740 20 client.19362 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:41.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.453+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/3638414935 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x558af9c66db0 0x558af9c691a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.453+0000 7fcdaa79f740 4 client.19362 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:00:41.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.453+0000 7fcdaa79f740 1 -- 172.21.15.103:0/3638414935 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558af9bbf860 con 0x558af9c66db0 2024-09-19T07:00:41.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.454+0000 7fcda5961640 1 --2- 172.21.15.103:0/3638414935 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x558af9c66db0 0x558af9c691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:41.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.454+0000 7fcd89ffb640 10 client.19362 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:00:41.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.454+0000 7fcda5961640 1 --2- 172.21.15.103:0/3638414935 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x558af9c66db0 0x558af9c691a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91b358fd6463d2c6 server_cookie=5fd4e425a44acde2 in_seq=0 out_seq=0 2024-09-19T07:00:41.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.455+0000 7fcd89ffb640 1 -- 172.21.15.103:0/3638414935 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558af9bbf860 con 0x558af9c66db0 2024-09-19T07:00:41.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.455+0000 7fcd89ffb640 10 client.19362 handle_command_reply: tid=0 2024-09-19T07:00:41.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.455+0000 7fcdaa79f740 1 client.19362 shutdown 2024-09-19T07:00:41.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.455+0000 7fcdaa79f740 1 -- 172.21.15.103:0/3638414935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4db40 msgr2=0x558af9c53fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:41.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.455+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/3638414935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4db40 0x558af9c53fc0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fcd90004700 tx=0x7fcd90011bb0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.455+0000 7fcdaa79f740 1 -- 172.21.15.103:0/3638414935 shutdown_connections 2024-09-19T07:00:41.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.455+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/3638414935 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x558af9c66db0 0x558af9c691a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.455+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/3638414935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c57370 0x558af9c54560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.455+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/3638414935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4db40 0x558af9c53fc0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.455+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/3638414935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4c5d0 0x558af9c56d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.455+0000 7fcdaa79f740 1 -- 172.21.15.103:0/3638414935 >> 172.21.15.103:0/3638414935 conn(0x558af9c15310 msgr2=0x558af9c4e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:41.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.455+0000 7fcd89ffb640 0 client.19362 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:00:41.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.455+0000 7fcdaa79f740 1 -- 172.21.15.103:0/3638414935 shutdown_connections 2024-09-19T07:00:41.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.455+0000 7fcdaa79f740 1 -- 172.21.15.103:0/3638414935 wait complete. 2024-09-19T07:00:41.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.455+0000 7fcdaa79f740 20 client.19362 trim_cache size 0 max 16384 2024-09-19T07:00:41.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcdaa79f740 1 Processor -- start 2024-09-19T07:00:41.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcdaa79f740 1 -- start start 2024-09-19T07:00:41.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcdaa79f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4c5d0 0x558af9c63340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcdaa79f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4db40 0x558af9c55f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcdaa79f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c57370 0x558af9c6e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcdaa79f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcda01afbd0 con 0x558af9c4c5d0 2024-09-19T07:00:41.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcdaa79f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcda01b1710 con 0x558af9c4db40 2024-09-19T07:00:41.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcdaa79f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcda01b02a0 con 0x558af9c57370 2024-09-19T07:00:41.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcda5961640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4c5d0 0x558af9c63340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:41.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcda5961640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4c5d0 0x558af9c63340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57162/0 (socket says 172.21.15.103:57162) 2024-09-19T07:00:41.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcda5961640 1 -- 172.21.15.103:0/4189007498 learned_addr learned my addr 172.21.15.103:0/4189007498 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:41.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcda5961640 1 -- 172.21.15.103:0/4189007498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c57370 msgr2=0x558af9c6e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:41.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcda5961640 1 --2- 172.21.15.103:0/4189007498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c57370 0x558af9c6e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcda5961640 1 -- 172.21.15.103:0/4189007498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4db40 msgr2=0x558af9c55f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:41.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcda5961640 1 --2- 172.21.15.103:0/4189007498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4db40 0x558af9c55f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcda5961640 1 -- 172.21.15.103:0/4189007498 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af9c5a210 con 0x558af9c4c5d0 2024-09-19T07:00:41.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcda5961640 1 --2- 172.21.15.103:0/4189007498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4c5d0 0x558af9c63340 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009fd0 tx=0x7fcd9c000f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd56bb0e9cd1678 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:41.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.468+0000 7fcd8bfff640 1 -- 172.21.15.103:0/4189007498 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c024070 con 0x558af9c4c5d0 2024-09-19T07:00:41.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.469+0000 7fcd8bfff640 1 -- 172.21.15.103:0/4189007498 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd9c017d60 con 0x558af9c4c5d0 2024-09-19T07:00:41.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.469+0000 7fcd8bfff640 1 -- 172.21.15.103:0/4189007498 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c019bc0 con 0x558af9c4c5d0 2024-09-19T07:00:41.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.469+0000 7fcdaa79f740 1 -- 172.21.15.103:0/4189007498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4c5d0 msgr2=0x558af9c63340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:41.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.469+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/4189007498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4c5d0 0x558af9c63340 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009fd0 tx=0x7fcd9c000f60 comp rx=0 tx=0).stop 2024-09-19T07:00:41.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.469+0000 7fcdaa79f740 1 -- 172.21.15.103:0/4189007498 shutdown_connections 2024-09-19T07:00:41.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.469+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/4189007498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c57370 0x558af9c6e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.469+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/4189007498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4db40 0x558af9c55f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.469+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/4189007498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c4c5d0 0x558af9c63340 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.469+0000 7fcdaa79f740 1 -- 172.21.15.103:0/4189007498 >> 172.21.15.103:0/4189007498 conn(0x558af9c15310 msgr2=0x558af9c55820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:41.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.469+0000 7fcdaa79f740 1 -- 172.21.15.103:0/4189007498 shutdown_connections 2024-09-19T07:00:41.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.469+0000 7fcdaa79f740 1 -- 172.21.15.103:0/4189007498 wait complete. 2024-09-19T07:00:41.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.469+0000 7fcdaa79f740 1 -- start start 2024-09-19T07:00:41.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcdaa79f740 1 Processor -- start 2024-09-19T07:00:41.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcdaa79f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4c5d0 0x558af9c50ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcdaa79f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c4db40 0x558af9c51400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcdaa79f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c57370 0x558af9c51940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcdaa79f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcda01b1040 con 0x558af9c57370 2024-09-19T07:00:41.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcdaa79f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcda01b0970 con 0x558af9c4c5d0 2024-09-19T07:00:41.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcdaa79f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcda01adbd0 con 0x558af9c4db40 2024-09-19T07:00:41.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcdaa79f740 10 client.0 _ll_register_callbacks cb 0x7fcd9000a860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:41.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcda5961640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c57370 0x558af9c51940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:41.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcda5961640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c57370 0x558af9c51940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57176/0 (socket says 172.21.15.103:57176) 2024-09-19T07:00:41.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcda5961640 1 -- 172.21.15.103:0/2488627381 learned_addr learned my addr 172.21.15.103:0/2488627381 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:41.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcda6162640 1 --2- 172.21.15.103:0/2488627381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c4db40 0x558af9c51400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:41.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcda5961640 1 --2- 172.21.15.103:0/2488627381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4c5d0 0x558af9c50ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:41.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcda5961640 1 -- 172.21.15.103:0/2488627381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c4db40 msgr2=0x558af9c51400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:41.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcda5961640 1 --2- 172.21.15.103:0/2488627381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c4db40 0x558af9c51400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcda5961640 1 -- 172.21.15.103:0/2488627381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4c5d0 msgr2=0x558af9c50ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:41.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcda5961640 1 --2- 172.21.15.103:0/2488627381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4c5d0 0x558af9c50ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcda5961640 1 -- 172.21.15.103:0/2488627381 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af9c49960 con 0x558af9c57370 2024-09-19T07:00:41.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcda5961640 1 --2- 172.21.15.103:0/2488627381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c57370 0x558af9c51940 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009c30 tx=0x7fcd9c01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22edf2465bc2815c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:41.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.470+0000 7fcd89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:41.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.471+0000 7fcd89ffb640 1 -- 172.21.15.103:0/2488627381 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c024040 con 0x558af9c57370 2024-09-19T07:00:41.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.471+0000 7fcd89ffb640 1 -- 172.21.15.103:0/2488627381 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fcd9c017880 con 0x558af9c57370 2024-09-19T07:00:41.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.471+0000 7fcdaa79f740 1 -- 172.21.15.103:0/2488627381 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd9c017d60 con 0x558af9c57370 2024-09-19T07:00:41.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.471+0000 7fcd89ffb640 1 -- 172.21.15.103:0/2488627381 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcd9c017a80 con 0x558af9c57370 2024-09-19T07:00:41.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.471+0000 7fcdaa79f740 10 client.19368 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:41.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.471+0000 7fcdaa79f740 1 -- 172.21.15.103:0/2488627381 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558af9c47310 con 0x558af9c57370 2024-09-19T07:00:41.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.471+0000 7fcd89ffb640 1 -- 172.21.15.103:0/2488627381 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd9c017d60 con 0x558af9c57370 2024-09-19T07:00:41.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.471+0000 7fcd89ffb640 1 -- 172.21.15.103:0/2488627381 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcd9c0343f0 con 0x558af9c57370 2024-09-19T07:00:41.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.471+0000 7fcdaa79f740 10 client.19368 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:41.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.471+0000 7fcdaa79f740 10 client.19368 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:00:41.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.471+0000 7fcdaa79f740 20 client.19368 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:41.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.471+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/2488627381 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x558af9cd1c60 0x558af9cd4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:41.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.471+0000 7fcdaa79f740 4 client.19368 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:00:41.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.471+0000 7fcdaa79f740 1 -- 172.21.15.103:0/2488627381 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558af9bb9d70 con 0x558af9cd1c60 2024-09-19T07:00:41.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.471+0000 7fcda6162640 1 --2- 172.21.15.103:0/2488627381 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x558af9cd1c60 0x558af9cd4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:41.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcd89ffb640 10 client.19368 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:00:41.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcda6162640 1 --2- 172.21.15.103:0/2488627381 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x558af9cd1c60 0x558af9cd4080 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81fcd731f536d9af server_cookie=adac29494ce74b15 in_seq=0 out_seq=0 2024-09-19T07:00:41.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcd89ffb640 1 -- 172.21.15.103:0/2488627381 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558af9bb9d70 con 0x558af9cd1c60 2024-09-19T07:00:41.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcd89ffb640 10 client.19368 handle_command_reply: tid=0 2024-09-19T07:00:41.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcdaa79f740 1 client.19368 shutdown 2024-09-19T07:00:41.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcdaa79f740 1 -- 172.21.15.103:0/2488627381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c57370 msgr2=0x558af9c51940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:41.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/2488627381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c57370 0x558af9c51940 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009c30 tx=0x7fcd9c01c040 comp rx=0 tx=0).stop 2024-09-19T07:00:41.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcdaa79f740 1 -- 172.21.15.103:0/2488627381 shutdown_connections 2024-09-19T07:00:41.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/2488627381 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x558af9cd1c60 0x558af9cd4080 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/2488627381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558af9c57370 0x558af9c51940 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/2488627381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558af9c4db40 0x558af9c51400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcd89ffb640 0 client.19368 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:00:41.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcdaa79f740 1 --2- 172.21.15.103:0/2488627381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558af9c4c5d0 0x558af9c50ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcdaa79f740 1 -- 172.21.15.103:0/2488627381 >> 172.21.15.103:0/2488627381 conn(0x558af9c15310 msgr2=0x558af9c56040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:41.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcdaa79f740 1 -- 172.21.15.103:0/2488627381 shutdown_connections 2024-09-19T07:00:41.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcdaa79f740 1 -- 172.21.15.103:0/2488627381 wait complete. 2024-09-19T07:00:41.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.472+0000 7fcdaa79f740 20 client.19368 trim_cache size 0 max 16384 2024-09-19T07:00:41.489 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:00:41.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.473+0000 7fcda83c6640 1 -- 172.21.15.103:0/2482319547 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd8c0a33b0 msgr2=0x7fcd8c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:41.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.473+0000 7fcda83c6640 1 --2- 172.21.15.103:0/2482319547 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd8c0a33b0 0x7fcd8c0a5870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcd980b9e60 tx=0x7fcd980b9040 comp rx=0 tx=0).stop 2024-09-19T07:00:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.473+0000 7fcda83c6640 1 -- 172.21.15.103:0/2482319547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda0106f10 msgr2=0x7fcda0074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.473+0000 7fcda83c6640 1 --2- 172.21.15.103:0/2482319547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda0106f10 0x7fcda0074f70 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fcd98031a60 tx=0x7fcd98031a90 comp rx=0 tx=0).stop 2024-09-19T07:00:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.473+0000 7fcda83c6640 1 -- 172.21.15.103:0/2482319547 shutdown_connections 2024-09-19T07:00:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.473+0000 7fcda83c6640 1 --2- 172.21.15.103:0/2482319547 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd8c0a33b0 0x7fcd8c0a5870 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.473+0000 7fcda83c6640 1 --2- 172.21.15.103:0/2482319547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda0106f10 0x7fcda0074f70 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.473+0000 7fcda83c6640 1 --2- 172.21.15.103:0/2482319547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda0102790 0x7fcda0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.473+0000 7fcda83c6640 1 --2- 172.21.15.103:0/2482319547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcda0101de0 0x7fcda00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.473+0000 7fcda83c6640 1 -- 172.21.15.103:0/2482319547 >> 172.21.15.103:0/2482319547 conn(0x7fcda00fd8c0 msgr2=0x7fcda00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.473+0000 7fcda83c6640 1 -- 172.21.15.103:0/2482319547 shutdown_connections 2024-09-19T07:00:41.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:41.473+0000 7fcda83c6640 1 -- 172.21.15.103:0/2482319547 wait complete. 2024-09-19T07:00:42.119 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:41 smithi103 ceph-mon[31158]: pgmap v356: 97 pgs: 97 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-09-19T07:00:42.148 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:00:42.239 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:41 smithi134 ceph-mon[35356]: pgmap v356: 97 pgs: 97 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-09-19T07:00:42.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:41 smithi174 ceph-mon[35637]: pgmap v356: 97 pgs: 97 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-09-19T07:00:42.398 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:00:42.519 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.519 INFO:tasks.workunit.client.0.smithi103.stdout:313/600: open fd = 3 2024-09-19T07:00:42.520 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.520 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.520 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.520 INFO:tasks.workunit.client.0.smithi103.stdout:314/600: open fd = 3 2024-09-19T07:00:42.520 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.520 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.520 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.520 INFO:tasks.workunit.client.0.smithi103.stdout:315/600: open fd = 3 2024-09-19T07:00:42.520 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.520 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.520 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.520 INFO:tasks.workunit.client.0.smithi103.stdout:316/600: open fd = 3 2024-09-19T07:00:42.520 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.521 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.521 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.521 INFO:tasks.workunit.client.0.smithi103.stdout:317/600: open fd = 3 2024-09-19T07:00:42.521 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.521 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.521 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.521 INFO:tasks.workunit.client.0.smithi103.stdout:318/600: open fd = 3 2024-09-19T07:00:42.521 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.521 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.521 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.521 INFO:tasks.workunit.client.0.smithi103.stdout:319/600: open fd = 3 2024-09-19T07:00:42.521 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.521 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.522 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.522 INFO:tasks.workunit.client.0.smithi103.stdout:320/600: open fd = 3 2024-09-19T07:00:42.522 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.522 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.522 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.522 INFO:tasks.workunit.client.0.smithi103.stdout:321/600: open fd = 3 2024-09-19T07:00:42.522 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.522 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.522 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.522 INFO:tasks.workunit.client.0.smithi103.stdout:322/600: open fd = 3 2024-09-19T07:00:42.522 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.523 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.523 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.523 INFO:tasks.workunit.client.0.smithi103.stdout:323/600: open fd = 3 2024-09-19T07:00:42.523 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.523 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.523 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.523 INFO:tasks.workunit.client.0.smithi103.stdout:324/600: open fd = 3 2024-09-19T07:00:42.523 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.523 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.523 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.523 INFO:tasks.workunit.client.0.smithi103.stdout:325/600: open fd = 3 2024-09-19T07:00:42.524 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.524 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.524 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.524 INFO:tasks.workunit.client.0.smithi103.stdout:326/600: open fd = 3 2024-09-19T07:00:42.524 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.524 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.524 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.524 INFO:tasks.workunit.client.0.smithi103.stdout:327/600: open fd = 3 2024-09-19T07:00:42.524 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.524 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.524 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.524 INFO:tasks.workunit.client.0.smithi103.stdout:328/600: open fd = 3 2024-09-19T07:00:42.524 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.525 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.525 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.525 INFO:tasks.workunit.client.0.smithi103.stdout:329/600: open fd = 3 2024-09-19T07:00:42.525 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.525 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.525 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.525 INFO:tasks.workunit.client.0.smithi103.stdout:330/600: open fd = 3 2024-09-19T07:00:42.525 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.525 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.525 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.525 INFO:tasks.workunit.client.0.smithi103.stdout:331/600: open fd = 3 2024-09-19T07:00:42.525 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.525 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.526 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.526 INFO:tasks.workunit.client.0.smithi103.stdout:332/600: open fd = 3 2024-09-19T07:00:42.526 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.526 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.526 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.526 INFO:tasks.workunit.client.0.smithi103.stdout:333/600: open fd = 3 2024-09-19T07:00:42.526 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.526 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.526 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.526 INFO:tasks.workunit.client.0.smithi103.stdout:334/600: open fd = 3 2024-09-19T07:00:42.526 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.527 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.527 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.527 INFO:tasks.workunit.client.0.smithi103.stdout:335/600: open fd = 3 2024-09-19T07:00:42.527 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.527 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.527 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.527 INFO:tasks.workunit.client.0.smithi103.stdout:336/600: open fd = 3 2024-09-19T07:00:42.527 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.527 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.527 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.527 INFO:tasks.workunit.client.0.smithi103.stdout:337/600: open fd = 3 2024-09-19T07:00:42.528 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.528 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.528 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.528 INFO:tasks.workunit.client.0.smithi103.stdout:338/600: open fd = 3 2024-09-19T07:00:42.528 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.528 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.528 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.528 INFO:tasks.workunit.client.0.smithi103.stdout:339/600: open fd = 3 2024-09-19T07:00:42.528 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.529 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.529 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.529 INFO:tasks.workunit.client.0.smithi103.stdout:340/600: open fd = 3 2024-09-19T07:00:42.529 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.529 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.529 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.529 INFO:tasks.workunit.client.0.smithi103.stdout:341/600: open fd = 3 2024-09-19T07:00:42.529 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.529 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.529 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.529 INFO:tasks.workunit.client.0.smithi103.stdout:342/600: open fd = 3 2024-09-19T07:00:42.529 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.529 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.530 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.530 INFO:tasks.workunit.client.0.smithi103.stdout:343/600: open fd = 3 2024-09-19T07:00:42.530 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.530 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.530 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.530 INFO:tasks.workunit.client.0.smithi103.stdout:344/600: open fd = 3 2024-09-19T07:00:42.530 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.530 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.530 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.530 INFO:tasks.workunit.client.0.smithi103.stdout:345/600: open fd = 3 2024-09-19T07:00:42.530 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.530 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.531 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.531 INFO:tasks.workunit.client.0.smithi103.stdout:346/600: open fd = 3 2024-09-19T07:00:42.531 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.531 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.531 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.531 INFO:tasks.workunit.client.0.smithi103.stdout:347/600: open fd = 3 2024-09-19T07:00:42.531 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.531 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.531 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.531 INFO:tasks.workunit.client.0.smithi103.stdout:348/600: open fd = 3 2024-09-19T07:00:42.531 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.531 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.531 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.532 INFO:tasks.workunit.client.0.smithi103.stdout:349/600: open fd = 3 2024-09-19T07:00:42.532 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.532 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.532 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.532 INFO:tasks.workunit.client.0.smithi103.stdout:350/600: open fd = 3 2024-09-19T07:00:42.532 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.532 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.532 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.532 INFO:tasks.workunit.client.0.smithi103.stdout:351/600: open fd = 3 2024-09-19T07:00:42.532 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.532 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.532 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.533 INFO:tasks.workunit.client.0.smithi103.stdout:352/600: open fd = 3 2024-09-19T07:00:42.533 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.533 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.533 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.533 INFO:tasks.workunit.client.0.smithi103.stdout:353/600: open fd = 3 2024-09-19T07:00:42.533 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.533 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.533 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.533 INFO:tasks.workunit.client.0.smithi103.stdout:354/600: open fd = 3 2024-09-19T07:00:42.533 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.533 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.533 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.534 INFO:tasks.workunit.client.0.smithi103.stdout:355/600: open fd = 3 2024-09-19T07:00:42.534 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.534 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.534 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.534 INFO:tasks.workunit.client.0.smithi103.stdout:356/600: open fd = 3 2024-09-19T07:00:42.534 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.534 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.534 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.534 INFO:tasks.workunit.client.0.smithi103.stdout:357/600: open fd = 3 2024-09-19T07:00:42.534 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.534 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.535 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.535 INFO:tasks.workunit.client.0.smithi103.stdout:358/600: open fd = 3 2024-09-19T07:00:42.535 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.535 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.535 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.535 INFO:tasks.workunit.client.0.smithi103.stdout:359/600: open fd = 3 2024-09-19T07:00:42.535 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.535 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.535 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.535 INFO:tasks.workunit.client.0.smithi103.stdout:360/600: open fd = 3 2024-09-19T07:00:42.535 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.536 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.536 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.536 INFO:tasks.workunit.client.0.smithi103.stdout:361/600: open fd = 3 2024-09-19T07:00:42.536 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.536 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.536 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.536 INFO:tasks.workunit.client.0.smithi103.stdout:362/600: open fd = 3 2024-09-19T07:00:42.536 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.536 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.536 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.536 INFO:tasks.workunit.client.0.smithi103.stdout:363/600: open fd = 3 2024-09-19T07:00:42.536 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.536 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.537 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.537 INFO:tasks.workunit.client.0.smithi103.stdout:364/600: open fd = 3 2024-09-19T07:00:42.537 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.537 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.537 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.537 INFO:tasks.workunit.client.0.smithi103.stdout:365/600: open fd = 3 2024-09-19T07:00:42.537 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.537 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.537 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.537 INFO:tasks.workunit.client.0.smithi103.stdout:366/600: open fd = 3 2024-09-19T07:00:42.537 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.538 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.538 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.538 INFO:tasks.workunit.client.0.smithi103.stdout:367/600: open fd = 3 2024-09-19T07:00:42.538 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.538 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.538 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.538 INFO:tasks.workunit.client.0.smithi103.stdout:368/600: open fd = 3 2024-09-19T07:00:42.538 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.538 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.538 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.539 INFO:tasks.workunit.client.0.smithi103.stdout:369/600: open fd = 3 2024-09-19T07:00:42.539 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.539 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.539 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.539 INFO:tasks.workunit.client.0.smithi103.stdout:370/600: open fd = 3 2024-09-19T07:00:42.539 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.539 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.539 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.539 INFO:tasks.workunit.client.0.smithi103.stdout:371/600: open fd = 3 2024-09-19T07:00:42.539 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.539 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.539 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.540 INFO:tasks.workunit.client.0.smithi103.stdout:372/600: open fd = 3 2024-09-19T07:00:42.540 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.540 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.540 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.540 INFO:tasks.workunit.client.0.smithi103.stdout:373/600: open fd = 3 2024-09-19T07:00:42.540 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.540 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.540 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.540 INFO:tasks.workunit.client.0.smithi103.stdout:374/600: open fd = 3 2024-09-19T07:00:42.540 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.540 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:00:42.540 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:00:42.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.811+0000 7f555d690640 1 -- 172.21.15.103:0/2303800530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558100e20 msgr2=0x7f5558101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:42.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.811+0000 7f555d690640 1 --2- 172.21.15.103:0/2303800530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558100e20 0x7f5558101200 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f5548009ca0 tx=0x7f554802a950 comp rx=0 tx=0).stop 2024-09-19T07:00:42.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.811+0000 7f555d690640 1 -- 172.21.15.103:0/2303800530 shutdown_connections 2024-09-19T07:00:42.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.811+0000 7f555d690640 1 --2- 172.21.15.103:0/2303800530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558102ef0 0x7f555810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.811+0000 7f555d690640 1 --2- 172.21.15.103:0/2303800530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5558102570 0x7f55581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.811+0000 7f555d690640 1 --2- 172.21.15.103:0/2303800530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558100e20 0x7f5558101200 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.811+0000 7f555d690640 1 -- 172.21.15.103:0/2303800530 >> 172.21.15.103:0/2303800530 conn(0x7f55580fc7d0 msgr2=0x7f55580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:42.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.811+0000 7f555d690640 1 -- 172.21.15.103:0/2303800530 shutdown_connections 2024-09-19T07:00:42.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.811+0000 7f555d690640 1 -- 172.21.15.103:0/2303800530 wait complete. 2024-09-19T07:00:42.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.811+0000 7f555d690640 1 Processor -- start 2024-09-19T07:00:42.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f555d690640 1 -- start start 2024-09-19T07:00:42.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f555d690640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558100e20 0x7f555819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f555d690640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5558102570 0x7f555819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f555d690640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558102ef0 0x7f555819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f555d690640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5558110f80 con 0x7f5558102ef0 2024-09-19T07:00:42.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f555d690640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5558110e00 con 0x7f5558102570 2024-09-19T07:00:42.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f555d690640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5558111100 con 0x7f5558100e20 2024-09-19T07:00:42.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f55577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558102ef0 0x7f555819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:42.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f55577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558102ef0 0x7f555819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57194/0 (socket says 172.21.15.103:57194) 2024-09-19T07:00:42.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f55577fe640 1 -- 172.21.15.103:0/2598683663 learned_addr learned my addr 172.21.15.103:0/2598683663 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:42.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f5556ffd640 1 --2- 172.21.15.103:0/2598683663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558100e20 0x7f555819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:42.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f55567fc640 1 --2- 172.21.15.103:0/2598683663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5558102570 0x7f555819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:42.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f55577fe640 1 -- 172.21.15.103:0/2598683663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558100e20 msgr2=0x7f555819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:42.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f55577fe640 1 --2- 172.21.15.103:0/2598683663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558100e20 0x7f555819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f55577fe640 1 -- 172.21.15.103:0/2598683663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5558102570 msgr2=0x7f555819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:42.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f55577fe640 1 --2- 172.21.15.103:0/2598683663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5558102570 0x7f555819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f55577fe640 1 -- 172.21.15.103:0/2598683663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555819f600 con 0x7f5558102ef0 2024-09-19T07:00:42.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f5556ffd640 1 --2- 172.21.15.103:0/2598683663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558100e20 0x7f555819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:42.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.812+0000 7f55577fe640 1 --2- 172.21.15.103:0/2598683663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558102ef0 0x7f555819ef20 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f555000d750 tx=0x7f555000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:42.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.813+0000 7f553ffff640 1 -- 172.21.15.103:0/2598683663 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550017070 con 0x7f5558102ef0 2024-09-19T07:00:42.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.813+0000 7f553ffff640 1 -- 172.21.15.103:0/2598683663 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5550005500 con 0x7f5558102ef0 2024-09-19T07:00:42.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.813+0000 7f555d690640 1 -- 172.21.15.103:0/2598683663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555819f8f0 con 0x7f5558102ef0 2024-09-19T07:00:42.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.813+0000 7f553ffff640 1 -- 172.21.15.103:0/2598683663 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550004d90 con 0x7f5558102ef0 2024-09-19T07:00:42.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.813+0000 7f555d690640 1 -- 172.21.15.103:0/2598683663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55581a71d0 con 0x7f5558102ef0 2024-09-19T07:00:42.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.815+0000 7f553ffff640 1 -- 172.21.15.103:0/2598683663 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f5550004f30 con 0x7f5558102ef0 2024-09-19T07:00:42.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.815+0000 7f555fa69740 1 Processor -- start 2024-09-19T07:00:42.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.815+0000 7f555fa69740 1 -- start start 2024-09-19T07:00:42.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f555fa69740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a08a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f555fa69740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a08a5d0 0x55fa1a08aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f555fa69740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a095370 0x55fa1a097760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f555fa69740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fa19ff02f0 con 0x55fa1a08bb40 2024-09-19T07:00:42.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f555fa69740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fa19f97cf0 con 0x55fa1a08a5d0 2024-09-19T07:00:42.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f555fa69740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fa19fec2b0 con 0x55fa1a095370 2024-09-19T07:00:42.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f553ffff640 1 --2- 172.21.15.103:0/2598683663 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55400a33b0 0x7f55400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f55567fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a08a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:42.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f55567fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a08a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57204/0 (socket says 172.21.15.103:57204) 2024-09-19T07:00:42.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f55567fc640 1 -- 172.21.15.103:0/3458112527 learned_addr learned my addr 172.21.15.103:0/3458112527 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:42.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f55577fe640 1 --2- 172.21.15.103:0/2598683663 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55400a33b0 0x7f55400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:42.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f5556ffd640 1 --2- 172.21.15.103:0/3458112527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a08a5d0 0x55fa1a08aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:42.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f553ffff640 1 -- 172.21.15.103:0/2598683663 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f55500cf7c0 con 0x7f5558102ef0 2024-09-19T07:00:42.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f55567fc640 1 --2- 172.21.15.103:0/3458112527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a095370 0x55fa1a097760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:42.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f55567fc640 1 -- 172.21.15.103:0/3458112527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a095370 msgr2=0x55fa1a097760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:42.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f55567fc640 1 --2- 172.21.15.103:0/3458112527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a095370 0x55fa1a097760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f55567fc640 1 -- 172.21.15.103:0/3458112527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a08a5d0 msgr2=0x55fa1a08aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:42.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f55567fc640 1 --2- 172.21.15.103:0/3458112527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a08a5d0 0x55fa1a08aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f55567fc640 1 -- 172.21.15.103:0/3458112527 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa19f7c3a0 con 0x55fa1a08bb40 2024-09-19T07:00:42.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f55577fe640 1 --2- 172.21.15.103:0/2598683663 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55400a33b0 0x7f55400a5870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f55500cd4f0 tx=0x7f5550090f10 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:42.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f55567fc640 1 --2- 172.21.15.103:0/3458112527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a08a090 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f554c00f360 tx=0x7f554c00b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf2bb74d419ddd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:42.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.816+0000 7f553d7fa640 1 -- 172.21.15.103:0/3458112527 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c005220 con 0x55fa1a08bb40 2024-09-19T07:00:42.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.817+0000 7f553d7fa640 1 -- 172.21.15.103:0/3458112527 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f554c004c80 con 0x55fa1a08bb40 2024-09-19T07:00:42.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.817+0000 7f553d7fa640 1 -- 172.21.15.103:0/3458112527 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c0129e0 con 0x55fa1a08bb40 2024-09-19T07:00:42.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.817+0000 7f555fa69740 1 -- 172.21.15.103:0/3458112527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 msgr2=0x55fa1a08a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:42.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.817+0000 7f555fa69740 1 --2- 172.21.15.103:0/3458112527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a08a090 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f554c00f360 tx=0x7f554c00b500 comp rx=0 tx=0).stop 2024-09-19T07:00:42.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.817+0000 7f555fa69740 1 -- 172.21.15.103:0/3458112527 shutdown_connections 2024-09-19T07:00:42.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.817+0000 7f555fa69740 1 --2- 172.21.15.103:0/3458112527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a095370 0x55fa1a097760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.817+0000 7f555fa69740 1 --2- 172.21.15.103:0/3458112527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a08a5d0 0x55fa1a08aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.817+0000 7f555fa69740 1 --2- 172.21.15.103:0/3458112527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a08a090 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.817+0000 7f555fa69740 1 -- 172.21.15.103:0/3458112527 >> 172.21.15.103:0/3458112527 conn(0x55fa1a053310 msgr2=0x55fa1a086c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:42.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.817+0000 7f555fa69740 1 -- 172.21.15.103:0/3458112527 shutdown_connections 2024-09-19T07:00:42.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.817+0000 7f555fa69740 1 -- 172.21.15.103:0/3458112527 wait complete. 2024-09-19T07:00:42.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f555fa69740 1 -- start start 2024-09-19T07:00:42.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f555fa69740 1 Processor -- start 2024-09-19T07:00:42.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f555fa69740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08a5d0 0x55fa1a094ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f555fa69740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a08bb40 0x55fa1a091f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f555fa69740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a095370 0x55fa1a0924f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f555fa69740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fa1a098070 con 0x55fa1a08a5d0 2024-09-19T07:00:42.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f555fa69740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fa19fab060 con 0x55fa1a095370 2024-09-19T07:00:42.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f555fa69740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fa1a0981f0 con 0x55fa1a08bb40 2024-09-19T07:00:42.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f555fa69740 10 client.0 _ll_register_callbacks cb 0x55fa19cfe8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:42.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f55567fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08a5d0 0x55fa1a094ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:42.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f55567fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08a5d0 0x55fa1a094ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57214/0 (socket says 172.21.15.103:57214) 2024-09-19T07:00:42.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f55567fc640 1 -- 172.21.15.103:0/2326265270 learned_addr learned my addr 172.21.15.103:0/2326265270 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:42.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f55567fc640 1 -- 172.21.15.103:0/2326265270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a08bb40 msgr2=0x55fa1a091f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:42.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f55567fc640 1 --2- 172.21.15.103:0/2326265270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a08bb40 0x55fa1a091f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f55567fc640 1 -- 172.21.15.103:0/2326265270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a095370 msgr2=0x55fa1a0924f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:42.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f55567fc640 1 --2- 172.21.15.103:0/2326265270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a095370 0x55fa1a0924f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.818+0000 7f55567fc640 1 -- 172.21.15.103:0/2326265270 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa19d613c0 con 0x55fa1a08a5d0 2024-09-19T07:00:42.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.819+0000 7f55567fc640 1 --2- 172.21.15.103:0/2326265270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08a5d0 0x55fa1a094ce0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f554c0056b0 tx=0x7f554c00bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aef7204d090e852 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:42.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.819+0000 7f553aff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:42.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.819+0000 7f553aff5640 1 -- 172.21.15.103:0/2326265270 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c0052c0 con 0x55fa1a08a5d0 2024-09-19T07:00:42.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.819+0000 7f553aff5640 1 -- 172.21.15.103:0/2326265270 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f554c004640 con 0x55fa1a08a5d0 2024-09-19T07:00:42.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.819+0000 7f555fa69740 1 -- 172.21.15.103:0/2326265270 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f554c004c80 con 0x55fa1a08a5d0 2024-09-19T07:00:42.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.819+0000 7f553aff5640 1 -- 172.21.15.103:0/2326265270 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f554c004840 con 0x55fa1a08a5d0 2024-09-19T07:00:42.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.819+0000 7f555fa69740 10 client.19380 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:42.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.819+0000 7f555fa69740 1 -- 172.21.15.103:0/2326265270 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa1a0a3a80 con 0x55fa1a08a5d0 2024-09-19T07:00:42.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.819+0000 7f553aff5640 1 -- 172.21.15.103:0/2326265270 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f554c004c80 con 0x55fa1a08a5d0 2024-09-19T07:00:42.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.819+0000 7f553aff5640 1 -- 172.21.15.103:0/2326265270 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f554c01d560 con 0x55fa1a08a5d0 2024-09-19T07:00:42.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.819+0000 7f555fa69740 10 client.19380 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:42.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.819+0000 7f555fa69740 10 client.19380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:42.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.819+0000 7f555fa69740 20 client.19380 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:42.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.819+0000 7f555fa69740 1 --2- 172.21.15.103:0/2326265270 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fa1a0a4e90 0x55fa1a0a7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.819+0000 7f555fa69740 4 client.19380 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:00:42.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.819+0000 7f555fa69740 1 -- 172.21.15.103:0/2326265270 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa19ffd860 con 0x55fa1a0a4e90 2024-09-19T07:00:42.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.820+0000 7f5556ffd640 1 --2- 172.21.15.103:0/2326265270 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fa1a0a4e90 0x55fa1a0a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:42.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.820+0000 7f5556ffd640 1 --2- 172.21.15.103:0/2326265270 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fa1a0a4e90 0x55fa1a0a7280 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e4f8d90d87175d7 server_cookie=308f064b3990e7c9 in_seq=0 out_seq=0 2024-09-19T07:00:42.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.820+0000 7f553aff5640 10 client.19380 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:42.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.821+0000 7f553aff5640 1 -- 172.21.15.103:0/2326265270 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa19ffd860 con 0x55fa1a0a4e90 2024-09-19T07:00:42.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.821+0000 7f553aff5640 10 client.19380 handle_command_reply: tid=0 2024-09-19T07:00:42.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.821+0000 7f555fa69740 1 client.19380 shutdown 2024-09-19T07:00:42.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.821+0000 7f555fa69740 1 -- 172.21.15.103:0/2326265270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08a5d0 msgr2=0x55fa1a094ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:42.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.821+0000 7f555fa69740 1 --2- 172.21.15.103:0/2326265270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08a5d0 0x55fa1a094ce0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f554c0056b0 tx=0x7f554c00bc30 comp rx=0 tx=0).stop 2024-09-19T07:00:42.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.821+0000 7f555fa69740 1 -- 172.21.15.103:0/2326265270 shutdown_connections 2024-09-19T07:00:42.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.821+0000 7f555fa69740 1 --2- 172.21.15.103:0/2326265270 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fa1a0a4e90 0x55fa1a0a7280 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.821+0000 7f555fa69740 1 --2- 172.21.15.103:0/2326265270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a095370 0x55fa1a0924f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.821+0000 7f555fa69740 1 --2- 172.21.15.103:0/2326265270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a08bb40 0x55fa1a091f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.821+0000 7f555fa69740 1 --2- 172.21.15.103:0/2326265270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08a5d0 0x55fa1a094ce0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.821+0000 7f555fa69740 1 -- 172.21.15.103:0/2326265270 >> 172.21.15.103:0/2326265270 conn(0x55fa1a053310 msgr2=0x55fa1a08c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:42.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.821+0000 7f553aff5640 0 client.19380 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:42.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.821+0000 7f555fa69740 1 -- 172.21.15.103:0/2326265270 shutdown_connections 2024-09-19T07:00:42.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.821+0000 7f555fa69740 1 -- 172.21.15.103:0/2326265270 wait complete. 2024-09-19T07:00:42.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.821+0000 7f555fa69740 20 client.19380 trim_cache size 0 max 16384 2024-09-19T07:00:42.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.834+0000 7f555fa69740 1 Processor -- start 2024-09-19T07:00:42.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.834+0000 7f555fa69740 1 -- start start 2024-09-19T07:00:42.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.834+0000 7f555fa69740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a08a5d0 0x55fa1a085fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.834+0000 7f555fa69740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a0a1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.834+0000 7f555fa69740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a095370 0x55fa1a08d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.834+0000 7f555fa69740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55581afa40 con 0x55fa1a08bb40 2024-09-19T07:00:42.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.834+0000 7f555fa69740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55581b1580 con 0x55fa1a095370 2024-09-19T07:00:42.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.834+0000 7f555fa69740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55581b0110 con 0x55fa1a08a5d0 2024-09-19T07:00:42.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.834+0000 7f5556ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a0a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:42.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.834+0000 7f5556ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a0a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57218/0 (socket says 172.21.15.103:57218) 2024-09-19T07:00:42.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.834+0000 7f5556ffd640 1 -- 172.21.15.103:0/1106783293 learned_addr learned my addr 172.21.15.103:0/1106783293 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:42.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.835+0000 7f55567fc640 1 --2- 172.21.15.103:0/1106783293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a08a5d0 0x55fa1a085fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:42.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.835+0000 7f55567fc640 1 --2- 172.21.15.103:0/1106783293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a095370 0x55fa1a08d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:42.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.835+0000 7f5556ffd640 1 -- 172.21.15.103:0/1106783293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a08a5d0 msgr2=0x55fa1a085fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:42.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.835+0000 7f5556ffd640 1 --2- 172.21.15.103:0/1106783293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a08a5d0 0x55fa1a085fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.835+0000 7f5556ffd640 1 -- 172.21.15.103:0/1106783293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a095370 msgr2=0x55fa1a08d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:42.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.835+0000 7f5556ffd640 1 --2- 172.21.15.103:0/1106783293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a095370 0x55fa1a08d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.835+0000 7f5556ffd640 1 -- 172.21.15.103:0/1106783293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa1a08b300 con 0x55fa1a08bb40 2024-09-19T07:00:42.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.836+0000 7f5556ffd640 1 --2- 172.21.15.103:0/1106783293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a0a1310 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f5548006010 tx=0x7f554802fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34d615dcd34b8d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:42.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.836+0000 7f553cff9640 1 -- 172.21.15.103:0/1106783293 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548034040 con 0x55fa1a08bb40 2024-09-19T07:00:42.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.836+0000 7f553cff9640 1 -- 172.21.15.103:0/1106783293 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5548003e50 con 0x55fa1a08bb40 2024-09-19T07:00:42.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.836+0000 7f553cff9640 1 -- 172.21.15.103:0/1106783293 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55480045a0 con 0x55fa1a08bb40 2024-09-19T07:00:42.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 -- 172.21.15.103:0/1106783293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 msgr2=0x55fa1a0a1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:42.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 --2- 172.21.15.103:0/1106783293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a0a1310 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f5548006010 tx=0x7f554802fc00 comp rx=0 tx=0).stop 2024-09-19T07:00:42.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 -- 172.21.15.103:0/1106783293 shutdown_connections 2024-09-19T07:00:42.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 --2- 172.21.15.103:0/1106783293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a095370 0x55fa1a08d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 --2- 172.21.15.103:0/1106783293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a0a1310 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 --2- 172.21.15.103:0/1106783293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a08a5d0 0x55fa1a085fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 -- 172.21.15.103:0/1106783293 >> 172.21.15.103:0/1106783293 conn(0x55fa1a053310 msgr2=0x55fa1a094010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:42.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 -- 172.21.15.103:0/1106783293 shutdown_connections 2024-09-19T07:00:42.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 -- 172.21.15.103:0/1106783293 wait complete. 2024-09-19T07:00:42.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 -- start start 2024-09-19T07:00:42.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 Processor -- start 2024-09-19T07:00:42.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a08a5d0 0x55fa1a09d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a09dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a095370 0x55fa1a08eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55581b0eb0 con 0x55fa1a08bb40 2024-09-19T07:00:42.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55581b07e0 con 0x55fa1a08a5d0 2024-09-19T07:00:42.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55581ada40 con 0x55fa1a095370 2024-09-19T07:00:42.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.837+0000 7f555fa69740 10 client.0 _ll_register_callbacks cb 0x7f5548009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:42.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.838+0000 7f5556ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a09dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:42.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.838+0000 7f55567fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a08a5d0 0x55fa1a09d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:42.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.838+0000 7f5556ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a09dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57234/0 (socket says 172.21.15.103:57234) 2024-09-19T07:00:42.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.838+0000 7f55567fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a08a5d0 0x55fa1a09d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:38302/0 (socket says 172.21.15.103:38302) 2024-09-19T07:00:42.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.838+0000 7f5556ffd640 1 -- 172.21.15.103:0/3642524807 learned_addr learned my addr 172.21.15.103:0/3642524807 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:42.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.838+0000 7f55567fc640 1 --2- 172.21.15.103:0/3642524807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a095370 0x55fa1a08eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:42.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.838+0000 7f55567fc640 1 -- 172.21.15.103:0/3642524807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a095370 msgr2=0x55fa1a08eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:42.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.838+0000 7f55567fc640 1 --2- 172.21.15.103:0/3642524807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a095370 0x55fa1a08eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.838+0000 7f55567fc640 1 -- 172.21.15.103:0/3642524807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 msgr2=0x55fa1a09dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:42.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.838+0000 7f55567fc640 1 --2- 172.21.15.103:0/3642524807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a09dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.838+0000 7f55567fc640 1 -- 172.21.15.103:0/3642524807 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa1a08f3c0 con 0x55fa1a08a5d0 2024-09-19T07:00:42.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.838+0000 7f55567fc640 1 --2- 172.21.15.103:0/3642524807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a08a5d0 0x55fa1a09d810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f554c01d900 tx=0x7f554c00bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f124366f3c5e45b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:42.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.838+0000 7f553aff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:00:42.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.838+0000 7f553aff5640 1 -- 172.21.15.103:0/3642524807 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c035020 con 0x55fa1a08a5d0 2024-09-19T07:00:42.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.838+0000 7f555fa69740 1 -- 172.21.15.103:0/3642524807 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5548003e50 con 0x55fa1a08a5d0 2024-09-19T07:00:42.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.838+0000 7f553aff5640 1 -- 172.21.15.103:0/3642524807 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f554c004ca0 con 0x55fa1a08a5d0 2024-09-19T07:00:42.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.839+0000 7f553aff5640 1 -- 172.21.15.103:0/3642524807 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f554c0050c0 con 0x55fa1a08a5d0 2024-09-19T07:00:42.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.839+0000 7f555fa69740 10 client.27515 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:42.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.839+0000 7f555fa69740 1 -- 172.21.15.103:0/3642524807 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa1a10f480 con 0x55fa1a08a5d0 2024-09-19T07:00:42.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.839+0000 7f553aff5640 1 -- 172.21.15.103:0/3642524807 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5548003e50 con 0x55fa1a08a5d0 2024-09-19T07:00:42.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.839+0000 7f553aff5640 1 -- 172.21.15.103:0/3642524807 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f554c03b070 con 0x55fa1a08a5d0 2024-09-19T07:00:42.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.839+0000 7f555fa69740 10 client.27515 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:42.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.839+0000 7f555fa69740 10 client.27515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:42.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.839+0000 7f555fa69740 20 client.27515 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:42.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.839+0000 7f555fa69740 1 --2- 172.21.15.103:0/3642524807 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fa1a10fa80 0x55fa1a111e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:42.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.839+0000 7f555fa69740 4 client.27515 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:00:42.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.839+0000 7f555fa69740 1 -- 172.21.15.103:0/3642524807 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa19ff7d70 con 0x55fa1a10fa80 2024-09-19T07:00:42.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.839+0000 7f5556ffd640 1 --2- 172.21.15.103:0/3642524807 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fa1a10fa80 0x55fa1a111e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:42.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f553aff5640 10 client.27515 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:42.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f5556ffd640 1 --2- 172.21.15.103:0/3642524807 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fa1a10fa80 0x55fa1a111e70 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43890e2e27e0ef2b server_cookie=e67c592bc196c7a1 in_seq=0 out_seq=0 2024-09-19T07:00:42.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f553aff5640 1 -- 172.21.15.103:0/3642524807 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa19ff7d70 con 0x55fa1a10fa80 2024-09-19T07:00:42.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f553aff5640 10 client.27515 handle_command_reply: tid=0 2024-09-19T07:00:42.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f555fa69740 1 client.27515 shutdown 2024-09-19T07:00:42.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f555fa69740 1 -- 172.21.15.103:0/3642524807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a08a5d0 msgr2=0x55fa1a09d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:42.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f555fa69740 1 --2- 172.21.15.103:0/3642524807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a08a5d0 0x55fa1a09d810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f554c01d900 tx=0x7f554c00bc30 comp rx=0 tx=0).stop 2024-09-19T07:00:42.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f555fa69740 1 -- 172.21.15.103:0/3642524807 shutdown_connections 2024-09-19T07:00:42.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f555fa69740 1 --2- 172.21.15.103:0/3642524807 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fa1a10fa80 0x55fa1a111e70 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f555fa69740 1 --2- 172.21.15.103:0/3642524807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa1a095370 0x55fa1a08eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f553aff5640 0 client.27515 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:42.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f555fa69740 1 --2- 172.21.15.103:0/3642524807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa1a08bb40 0x55fa1a09dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f555fa69740 1 --2- 172.21.15.103:0/3642524807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa1a08a5d0 0x55fa1a09d810 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f555fa69740 1 -- 172.21.15.103:0/3642524807 >> 172.21.15.103:0/3642524807 conn(0x55fa1a053310 msgr2=0x55fa1a090730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:42.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f555fa69740 1 -- 172.21.15.103:0/3642524807 shutdown_connections 2024-09-19T07:00:42.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f555fa69740 1 -- 172.21.15.103:0/3642524807 wait complete. 2024-09-19T07:00:42.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.840+0000 7f555fa69740 20 client.27515 trim_cache size 0 max 16384 2024-09-19T07:00:42.857 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:00:42.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.841+0000 7f555d690640 1 -- 172.21.15.103:0/2598683663 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55400a33b0 msgr2=0x7f55400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:42.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.841+0000 7f555d690640 1 --2- 172.21.15.103:0/2598683663 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55400a33b0 0x7f55400a5870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f55500cd4f0 tx=0x7f5550090f10 comp rx=0 tx=0).stop 2024-09-19T07:00:42.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.841+0000 7f555d690640 1 -- 172.21.15.103:0/2598683663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558102ef0 msgr2=0x7f555819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:42.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.841+0000 7f555d690640 1 --2- 172.21.15.103:0/2598683663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558102ef0 0x7f555819ef20 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f555000d750 tx=0x7f555000dc10 comp rx=0 tx=0).stop 2024-09-19T07:00:42.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.841+0000 7f555d690640 1 -- 172.21.15.103:0/2598683663 shutdown_connections 2024-09-19T07:00:42.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.841+0000 7f555d690640 1 --2- 172.21.15.103:0/2598683663 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55400a33b0 0x7f55400a5870 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.841+0000 7f555d690640 1 --2- 172.21.15.103:0/2598683663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558102ef0 0x7f555819ef20 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.841+0000 7f555d690640 1 --2- 172.21.15.103:0/2598683663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5558102570 0x7f555819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.841+0000 7f555d690640 1 --2- 172.21.15.103:0/2598683663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558100e20 0x7f555819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:42.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.841+0000 7f555d690640 1 -- 172.21.15.103:0/2598683663 >> 172.21.15.103:0/2598683663 conn(0x7f55580fc7d0 msgr2=0x7f5558069fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:42.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.841+0000 7f555d690640 1 -- 172.21.15.103:0/2598683663 shutdown_connections 2024-09-19T07:00:42.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:42.841+0000 7f555d690640 1 -- 172.21.15.103:0/2598683663 wait complete. 2024-09-19T07:00:44.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:43 smithi174 ceph-mon[35637]: pgmap v357: 97 pgs: 97 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T07:00:44.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:43 smithi103 ceph-mon[31158]: pgmap v357: 97 pgs: 97 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T07:00:44.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:43 smithi134 ceph-mon[35356]: pgmap v357: 97 pgs: 97 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T07:00:44.517 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:00:44.517 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:00:44.777 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:00:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.255+0000 7f804ec2f640 1 -- 172.21.15.103:0/3583865634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8048106f10 msgr2=0x7f804810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.255+0000 7f804ec2f640 1 --2- 172.21.15.103:0/3583865634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8048106f10 0x7f804810ea90 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f8040009900 tx=0x7f804002a410 comp rx=0 tx=0).stop 2024-09-19T07:00:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.255+0000 7f804ec2f640 1 -- 172.21.15.103:0/3583865634 shutdown_connections 2024-09-19T07:00:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.255+0000 7f804ec2f640 1 --2- 172.21.15.103:0/3583865634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8048106f10 0x7f804810ea90 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.255+0000 7f804ec2f640 1 --2- 172.21.15.103:0/3583865634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8048102790 0x7f80481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.255+0000 7f804ec2f640 1 --2- 172.21.15.103:0/3583865634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8048101de0 0x7f80481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.255+0000 7f804ec2f640 1 -- 172.21.15.103:0/3583865634 >> 172.21.15.103:0/3583865634 conn(0x7f80480fd8c0 msgr2=0x7f80480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.255+0000 7f804ec2f640 1 -- 172.21.15.103:0/3583865634 shutdown_connections 2024-09-19T07:00:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.255+0000 7f804ec2f640 1 -- 172.21.15.103:0/3583865634 wait complete. 2024-09-19T07:00:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804ec2f640 1 Processor -- start 2024-09-19T07:00:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804ec2f640 1 -- start start 2024-09-19T07:00:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804ec2f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8048101de0 0x7f8048198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804ec2f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8048102790 0x7f80481989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804ec2f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8048106f10 0x7f804819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804ec2f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8048111fb0 con 0x7f8048106f10 2024-09-19T07:00:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804ec2f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8048111e30 con 0x7f8048101de0 2024-09-19T07:00:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804ec2f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8048112130 con 0x7f8048102790 2024-09-19T07:00:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804d1cc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8048106f10 0x7f804819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804d1cc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8048106f10 0x7f804819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57260/0 (socket says 172.21.15.103:57260) 2024-09-19T07:00:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804d1cc640 1 -- 172.21.15.103:0/1751189302 learned_addr learned my addr 172.21.15.103:0/1751189302 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f803ffff640 1 --2- 172.21.15.103:0/1751189302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8048102790 0x7f80481989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804c9cb640 1 --2- 172.21.15.103:0/1751189302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8048101de0 0x7f8048198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804d1cc640 1 -- 172.21.15.103:0/1751189302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8048102790 msgr2=0x7f80481989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:45.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804d1cc640 1 --2- 172.21.15.103:0/1751189302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8048102790 0x7f80481989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804d1cc640 1 -- 172.21.15.103:0/1751189302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8048101de0 msgr2=0x7f8048198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:45.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804d1cc640 1 --2- 172.21.15.103:0/1751189302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8048101de0 0x7f8048198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.256+0000 7f804d1cc640 1 -- 172.21.15.103:0/1751189302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804819d440 con 0x7f8048106f10 2024-09-19T07:00:45.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.257+0000 7f804d1cc640 1 --2- 172.21.15.103:0/1751189302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8048106f10 0x7f804819cd60 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f8040005ab0 tx=0x7f80400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:45.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.257+0000 7f803dffb640 1 -- 172.21.15.103:0/1751189302 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804003f030 con 0x7f8048106f10 2024-09-19T07:00:45.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.257+0000 7f803dffb640 1 -- 172.21.15.103:0/1751189302 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8040003ce0 con 0x7f8048106f10 2024-09-19T07:00:45.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.257+0000 7f804ec2f640 1 -- 172.21.15.103:0/1751189302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804819d6d0 con 0x7f8048106f10 2024-09-19T07:00:45.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.257+0000 7f803dffb640 1 -- 172.21.15.103:0/1751189302 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040035070 con 0x7f8048106f10 2024-09-19T07:00:45.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.257+0000 7f804ec2f640 1 -- 172.21.15.103:0/1751189302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80481a5010 con 0x7f8048106f10 2024-09-19T07:00:45.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.259+0000 7f803dffb640 1 -- 172.21.15.103:0/1751189302 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8040042030 con 0x7f8048106f10 2024-09-19T07:00:45.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.259+0000 7f8051008740 1 Processor -- start 2024-09-19T07:00:45.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f8051008740 1 -- start start 2024-09-19T07:00:45.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f8051008740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564794491b60 0x564794491f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f8051008740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794492480 0x56479449c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f8051008740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479449cd40 0x56479449f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f8051008740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5647943f7280 con 0x564794491b60 2024-09-19T07:00:45.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f8051008740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56479439eb10 con 0x56479449cd40 2024-09-19T07:00:45.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f8051008740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5647943f3240 con 0x564794492480 2024-09-19T07:00:45.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f803dffb640 1 --2- 172.21.15.103:0/1751189302 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80300a33b0 0x7f80300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f803ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564794491b60 0x564794491f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f803ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564794491b60 0x564794491f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57262/0 (socket says 172.21.15.103:57262) 2024-09-19T07:00:45.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f803ffff640 1 -- 172.21.15.103:0/1252267175 learned_addr learned my addr 172.21.15.103:0/1252267175 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:45.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f803ffff640 1 --2- 172.21.15.103:0/1252267175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479449cd40 0x56479449f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f804d1cc640 1 --2- 172.21.15.103:0/1751189302 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80300a33b0 0x7f80300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f804c9cb640 1 --2- 172.21.15.103:0/1252267175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794492480 0x56479449c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f803dffb640 1 -- 172.21.15.103:0/1751189302 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f80400f0510 con 0x7f8048106f10 2024-09-19T07:00:45.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f803ffff640 1 -- 172.21.15.103:0/1252267175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794492480 msgr2=0x56479449c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:45.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f803ffff640 1 --2- 172.21.15.103:0/1252267175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794492480 0x56479449c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f803ffff640 1 -- 172.21.15.103:0/1252267175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479449cd40 msgr2=0x56479449f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:45.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f803ffff640 1 --2- 172.21.15.103:0/1252267175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479449cd40 0x56479449f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f803ffff640 1 -- 172.21.15.103:0/1252267175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647943ab1f0 con 0x564794491b60 2024-09-19T07:00:45.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.260+0000 7f804c9cb640 1 --2- 172.21.15.103:0/1252267175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794492480 0x56479449c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:45.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.261+0000 7f804d1cc640 1 --2- 172.21.15.103:0/1751189302 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80300a33b0 0x7f80300a5870 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f80400edcf0 tx=0x7f8040087110 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:45.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.261+0000 7f803ffff640 1 --2- 172.21.15.103:0/1252267175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564794491b60 0x564794491f40 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f80380176c0 tx=0x7f8038009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6a4dbf10848d3e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:45.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.261+0000 7f802effd640 1 -- 172.21.15.103:0/1252267175 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803800f040 con 0x564794491b60 2024-09-19T07:00:45.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.261+0000 7f802effd640 1 -- 172.21.15.103:0/1252267175 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f803801d070 con 0x564794491b60 2024-09-19T07:00:45.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.261+0000 7f802effd640 1 -- 172.21.15.103:0/1252267175 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80380185f0 con 0x564794491b60 2024-09-19T07:00:45.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.261+0000 7f8051008740 1 -- 172.21.15.103:0/1252267175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564794491b60 msgr2=0x564794491f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:45.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.261+0000 7f8051008740 1 --2- 172.21.15.103:0/1252267175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564794491b60 0x564794491f40 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f80380176c0 tx=0x7f8038009520 comp rx=0 tx=0).stop 2024-09-19T07:00:45.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.262+0000 7f8051008740 1 -- 172.21.15.103:0/1252267175 shutdown_connections 2024-09-19T07:00:45.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.262+0000 7f8051008740 1 --2- 172.21.15.103:0/1252267175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479449cd40 0x56479449f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.262+0000 7f8051008740 1 --2- 172.21.15.103:0/1252267175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794492480 0x56479449c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.262+0000 7f8051008740 1 --2- 172.21.15.103:0/1252267175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564794491b60 0x564794491f40 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.262+0000 7f8051008740 1 -- 172.21.15.103:0/1252267175 >> 172.21.15.103:0/1252267175 conn(0x56479448e0f0 msgr2=0x56479448e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:45.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.262+0000 7f8051008740 1 -- 172.21.15.103:0/1252267175 shutdown_connections 2024-09-19T07:00:45.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.262+0000 7f8051008740 1 -- 172.21.15.103:0/1252267175 wait complete. 2024-09-19T07:00:45.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.262+0000 7f8051008740 1 -- start start 2024-09-19T07:00:45.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.262+0000 7f8051008740 1 Processor -- start 2024-09-19T07:00:45.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.262+0000 7f8051008740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794491b60 0x564794499c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.262+0000 7f8051008740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564794492480 0x56479449a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.262+0000 7f8051008740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56479449cd40 0x56479449a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.262+0000 7f8051008740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56479449f910 con 0x56479449cd40 2024-09-19T07:00:45.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.262+0000 7f8051008740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5647943b1f10 con 0x564794492480 2024-09-19T07:00:45.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.262+0000 7f8051008740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56479449fa90 con 0x564794491b60 2024-09-19T07:00:45.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.262+0000 7f8051008740 10 client.0 _ll_register_callbacks cb 0x56479437aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:45.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f803ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56479449cd40 0x56479449a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f803ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56479449cd40 0x56479449a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57276/0 (socket says 172.21.15.103:57276) 2024-09-19T07:00:45.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f803ffff640 1 -- 172.21.15.103:0/3289486091 learned_addr learned my addr 172.21.15.103:0/3289486091 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:45.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f803ffff640 1 --2- 172.21.15.103:0/3289486091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794491b60 0x564794499c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f804c9cb640 1 --2- 172.21.15.103:0/3289486091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564794492480 0x56479449a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f803ffff640 1 -- 172.21.15.103:0/3289486091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794491b60 msgr2=0x564794499c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:45.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f803ffff640 1 --2- 172.21.15.103:0/3289486091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794491b60 0x564794499c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f803ffff640 1 -- 172.21.15.103:0/3289486091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564794492480 msgr2=0x56479449a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:45.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f803ffff640 1 --2- 172.21.15.103:0/3289486091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564794492480 0x56479449a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f803ffff640 1 -- 172.21.15.103:0/3289486091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647941686a0 con 0x56479449cd40 2024-09-19T07:00:45.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f804c9cb640 1 --2- 172.21.15.103:0/3289486091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564794492480 0x56479449a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:45.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f803ffff640 1 --2- 172.21.15.103:0/3289486091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56479449cd40 0x56479449a690 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f80380186e0 tx=0x7f8038015e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3902f7f8ffeae52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:45.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f8027fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:45.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f8027fff640 1 -- 172.21.15.103:0/3289486091 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038028070 con 0x56479449cd40 2024-09-19T07:00:45.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f8027fff640 1 -- 172.21.15.103:0/3289486091 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f80380158e0 con 0x56479449cd40 2024-09-19T07:00:45.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f8051008740 1 -- 172.21.15.103:0/3289486091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f803801d070 con 0x56479449cd40 2024-09-19T07:00:45.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f8027fff640 1 -- 172.21.15.103:0/3289486091 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8038015ae0 con 0x56479449cd40 2024-09-19T07:00:45.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f8051008740 10 client.19398 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:45.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.263+0000 7f8051008740 1 -- 172.21.15.103:0/3289486091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5647944ab510 con 0x56479449cd40 2024-09-19T07:00:45.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.264+0000 7f8027fff640 1 -- 172.21.15.103:0/3289486091 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f803801d070 con 0x56479449cd40 2024-09-19T07:00:45.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.264+0000 7f8027fff640 1 -- 172.21.15.103:0/3289486091 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8038024820 con 0x56479449cd40 2024-09-19T07:00:45.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.264+0000 7f8051008740 10 client.19398 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:45.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.264+0000 7f8051008740 10 client.19398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:45.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.264+0000 7f8051008740 20 client.19398 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:45.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.264+0000 7f8051008740 1 --2- 172.21.15.103:0/3289486091 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5647944ac8c0 0x5647944aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.264+0000 7f8051008740 4 client.19398 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:00:45.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.264+0000 7f8051008740 1 -- 172.21.15.103:0/3289486091 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647944047f0 con 0x5647944ac8c0 2024-09-19T07:00:45.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.264+0000 7f804c9cb640 1 --2- 172.21.15.103:0/3289486091 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5647944ac8c0 0x5647944aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.265+0000 7f804c9cb640 1 --2- 172.21.15.103:0/3289486091 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5647944ac8c0 0x5647944aecb0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14b641103ffa972b server_cookie=bb83866f01d2769e in_seq=0 out_seq=0 2024-09-19T07:00:45.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.265+0000 7f8027fff640 10 client.19398 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:45.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.265+0000 7f8027fff640 1 -- 172.21.15.103:0/3289486091 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647944047f0 con 0x5647944ac8c0 2024-09-19T07:00:45.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.265+0000 7f8027fff640 10 client.19398 handle_command_reply: tid=0 2024-09-19T07:00:45.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.265+0000 7f8051008740 1 client.19398 shutdown 2024-09-19T07:00:45.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.265+0000 7f8051008740 1 -- 172.21.15.103:0/3289486091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56479449cd40 msgr2=0x56479449a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:45.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.265+0000 7f8051008740 1 --2- 172.21.15.103:0/3289486091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56479449cd40 0x56479449a690 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f80380186e0 tx=0x7f8038015e20 comp rx=0 tx=0).stop 2024-09-19T07:00:45.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.265+0000 7f8051008740 1 -- 172.21.15.103:0/3289486091 shutdown_connections 2024-09-19T07:00:45.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.266+0000 7f8051008740 1 --2- 172.21.15.103:0/3289486091 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5647944ac8c0 0x5647944aecb0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.266+0000 7f8051008740 1 --2- 172.21.15.103:0/3289486091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56479449cd40 0x56479449a690 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.266+0000 7f8051008740 1 --2- 172.21.15.103:0/3289486091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564794492480 0x56479449a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.266+0000 7f8051008740 1 --2- 172.21.15.103:0/3289486091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794491b60 0x564794499c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.266+0000 7f8051008740 1 -- 172.21.15.103:0/3289486091 >> 172.21.15.103:0/3289486091 conn(0x56479448e0f0 msgr2=0x564794493dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:45.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.266+0000 7f8027fff640 0 client.19398 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:45.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.266+0000 7f8051008740 1 -- 172.21.15.103:0/3289486091 shutdown_connections 2024-09-19T07:00:45.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.266+0000 7f8051008740 1 -- 172.21.15.103:0/3289486091 wait complete. 2024-09-19T07:00:45.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.266+0000 7f8051008740 20 client.19398 trim_cache size 0 max 16384 2024-09-19T07:00:45.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.282+0000 7f8051008740 1 Processor -- start 2024-09-19T07:00:45.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.282+0000 7f8051008740 1 -- start start 2024-09-19T07:00:45.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.282+0000 7f8051008740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794491b60 0x5647944945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f8051008740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564794492480 0x564794498ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f8051008740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479449cd40 0x5647944b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f8051008740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80481ad980 con 0x564794492480 2024-09-19T07:00:45.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f8051008740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f80481af4c0 con 0x56479449cd40 2024-09-19T07:00:45.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f8051008740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80481ae050 con 0x564794491b60 2024-09-19T07:00:45.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f804c9cb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564794492480 0x564794498ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f804c9cb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564794492480 0x564794498ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57282/0 (socket says 172.21.15.103:57282) 2024-09-19T07:00:45.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f804c9cb640 1 -- 172.21.15.103:0/4037598063 learned_addr learned my addr 172.21.15.103:0/4037598063 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:45.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f803ffff640 1 --2- 172.21.15.103:0/4037598063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794491b60 0x5647944945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f803ffff640 1 --2- 172.21.15.103:0/4037598063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479449cd40 0x5647944b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f804c9cb640 1 -- 172.21.15.103:0/4037598063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794491b60 msgr2=0x5647944945c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:45.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f804c9cb640 1 --2- 172.21.15.103:0/4037598063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794491b60 0x5647944945c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f804c9cb640 1 -- 172.21.15.103:0/4037598063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479449cd40 msgr2=0x5647944b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:45.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f804c9cb640 1 --2- 172.21.15.103:0/4037598063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479449cd40 0x5647944b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f804c9cb640 1 -- 172.21.15.103:0/4037598063 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564794488640 con 0x564794492480 2024-09-19T07:00:45.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f803ffff640 1 --2- 172.21.15.103:0/4037598063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479449cd40 0x5647944b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:45.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f804c9cb640 1 --2- 172.21.15.103:0/4037598063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564794492480 0x564794498ac0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x56479448ef50 tx=0x7f8044009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ba497359706570 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:45.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f802e7fc640 1 -- 172.21.15.103:0/4037598063 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044013250 con 0x564794492480 2024-09-19T07:00:45.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f802e7fc640 1 -- 172.21.15.103:0/4037598063 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f804401f030 con 0x564794492480 2024-09-19T07:00:45.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.283+0000 7f802e7fc640 1 -- 172.21.15.103:0/4037598063 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804401b770 con 0x564794492480 2024-09-19T07:00:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.284+0000 7f8051008740 1 -- 172.21.15.103:0/4037598063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564794492480 msgr2=0x564794498ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.284+0000 7f8051008740 1 --2- 172.21.15.103:0/4037598063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564794492480 0x564794498ac0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x56479448ef50 tx=0x7f8044009660 comp rx=0 tx=0).stop 2024-09-19T07:00:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.284+0000 7f8051008740 1 -- 172.21.15.103:0/4037598063 shutdown_connections 2024-09-19T07:00:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.284+0000 7f8051008740 1 --2- 172.21.15.103:0/4037598063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479449cd40 0x5647944b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.284+0000 7f8051008740 1 --2- 172.21.15.103:0/4037598063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564794492480 0x564794498ac0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.284+0000 7f8051008740 1 --2- 172.21.15.103:0/4037598063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794491b60 0x5647944945c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.284+0000 7f8051008740 1 -- 172.21.15.103:0/4037598063 >> 172.21.15.103:0/4037598063 conn(0x56479448e0f0 msgr2=0x5647944983b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.284+0000 7f8051008740 1 -- 172.21.15.103:0/4037598063 shutdown_connections 2024-09-19T07:00:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.284+0000 7f8051008740 1 -- 172.21.15.103:0/4037598063 wait complete. 2024-09-19T07:00:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.284+0000 7f8051008740 1 -- start start 2024-09-19T07:00:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.284+0000 7f8051008740 1 Processor -- start 2024-09-19T07:00:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f8051008740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794491b60 0x5647944a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f8051008740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564794492480 0x5647944a0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f8051008740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56479449cd40 0x5647944a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f8051008740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80481aedf0 con 0x56479449cd40 2024-09-19T07:00:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f8051008740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f80481ae720 con 0x564794492480 2024-09-19T07:00:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f8051008740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80481ab980 con 0x564794491b60 2024-09-19T07:00:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f8051008740 10 client.0 _ll_register_callbacks cb 0x7f8038009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f803ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56479449cd40 0x5647944a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f803ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56479449cd40 0x5647944a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57286/0 (socket says 172.21.15.103:57286) 2024-09-19T07:00:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f803ffff640 1 -- 172.21.15.103:0/2191165841 learned_addr learned my addr 172.21.15.103:0/2191165841 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f803ffff640 1 --2- 172.21.15.103:0/2191165841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794491b60 0x5647944a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f804c9cb640 1 --2- 172.21.15.103:0/2191165841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564794492480 0x5647944a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f803ffff640 1 -- 172.21.15.103:0/2191165841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794491b60 msgr2=0x5647944a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f803ffff640 1 --2- 172.21.15.103:0/2191165841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794491b60 0x5647944a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f803ffff640 1 -- 172.21.15.103:0/2191165841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564794492480 msgr2=0x5647944a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f803ffff640 1 --2- 172.21.15.103:0/2191165841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564794492480 0x5647944a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f803ffff640 1 -- 172.21.15.103:0/2191165841 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564794516410 con 0x56479449cd40 2024-09-19T07:00:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f803ffff640 1 --2- 172.21.15.103:0/2191165841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56479449cd40 0x5647944a10a0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f8038000c00 tx=0x7f8038015e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3765d09e9d302570 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.285+0000 7f8027fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.286+0000 7f8027fff640 1 -- 172.21.15.103:0/2191165841 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038036070 con 0x56479449cd40 2024-09-19T07:00:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.286+0000 7f8027fff640 1 -- 172.21.15.103:0/2191165841 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f8038009480 con 0x56479449cd40 2024-09-19T07:00:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.286+0000 7f8051008740 1 -- 172.21.15.103:0/2191165841 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f804401f030 con 0x56479449cd40 2024-09-19T07:00:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.286+0000 7f8027fff640 1 -- 172.21.15.103:0/2191165841 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8038042050 con 0x56479449cd40 2024-09-19T07:00:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.286+0000 7f8051008740 10 client.19404 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.286+0000 7f8051008740 1 -- 172.21.15.103:0/2191165841 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564794517580 con 0x56479449cd40 2024-09-19T07:00:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.286+0000 7f8027fff640 1 -- 172.21.15.103:0/2191165841 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f804401f030 con 0x56479449cd40 2024-09-19T07:00:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.286+0000 7f8027fff640 1 -- 172.21.15.103:0/2191165841 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f80380234c0 con 0x56479449cd40 2024-09-19T07:00:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.286+0000 7f8051008740 10 client.19404 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.286+0000 7f8051008740 10 client.19404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.286+0000 7f8051008740 20 client.19404 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.286+0000 7f8051008740 1 --2- 172.21.15.103:0/2191165841 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564794517bf0 0x564794519fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.286+0000 7f8051008740 4 client.19404 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:00:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.286+0000 7f8051008740 1 -- 172.21.15.103:0/2191165841 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5647943fed00 con 0x564794517bf0 2024-09-19T07:00:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.286+0000 7f804c9cb640 1 --2- 172.21.15.103:0/2191165841 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564794517bf0 0x564794519fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.287+0000 7f8027fff640 10 client.19404 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.287+0000 7f804c9cb640 1 --2- 172.21.15.103:0/2191165841 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564794517bf0 0x564794519fe0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8d9f3c8d6c3d270 server_cookie=1ab6f3925786bce7 in_seq=0 out_seq=0 2024-09-19T07:00:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.288+0000 7f8027fff640 1 -- 172.21.15.103:0/2191165841 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5647943fed00 con 0x564794517bf0 2024-09-19T07:00:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.288+0000 7f8027fff640 10 client.19404 handle_command_reply: tid=0 2024-09-19T07:00:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.288+0000 7f8051008740 1 client.19404 shutdown 2024-09-19T07:00:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.288+0000 7f8051008740 1 -- 172.21.15.103:0/2191165841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56479449cd40 msgr2=0x5647944a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.288+0000 7f8051008740 1 --2- 172.21.15.103:0/2191165841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56479449cd40 0x5647944a10a0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f8038000c00 tx=0x7f8038015e20 comp rx=0 tx=0).stop 2024-09-19T07:00:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.288+0000 7f8051008740 1 -- 172.21.15.103:0/2191165841 shutdown_connections 2024-09-19T07:00:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.288+0000 7f8051008740 1 --2- 172.21.15.103:0/2191165841 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564794517bf0 0x564794519fe0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f8051008740 1 --2- 172.21.15.103:0/2191165841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56479449cd40 0x5647944a10a0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f8051008740 1 --2- 172.21.15.103:0/2191165841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564794492480 0x5647944a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f8027fff640 0 client.19404 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f8051008740 1 --2- 172.21.15.103:0/2191165841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564794491b60 0x5647944a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f8051008740 1 -- 172.21.15.103:0/2191165841 >> 172.21.15.103:0/2191165841 conn(0x56479448e0f0 msgr2=0x56479448d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f8051008740 1 -- 172.21.15.103:0/2191165841 shutdown_connections 2024-09-19T07:00:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f8051008740 1 -- 172.21.15.103:0/2191165841 wait complete. 2024-09-19T07:00:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f8051008740 20 client.19404 trim_cache size 0 max 16384 2024-09-19T07:00:45.295 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:00:45.295 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:00:45.295 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "a2cecf98-c2f3-4beb-9d96-dec1409c2bcc", 2024-09-19T07:00:45.295 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:00:45.295 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:00:45.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f804ec2f640 1 -- 172.21.15.103:0/1751189302 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80300a33b0 msgr2=0x7f80300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:45.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f804ec2f640 1 --2- 172.21.15.103:0/1751189302 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80300a33b0 0x7f80300a5870 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f80400edcf0 tx=0x7f8040087110 comp rx=0 tx=0).stop 2024-09-19T07:00:45.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f804ec2f640 1 -- 172.21.15.103:0/1751189302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8048106f10 msgr2=0x7f804819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:45.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f804ec2f640 1 --2- 172.21.15.103:0/1751189302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8048106f10 0x7f804819cd60 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f8040005ab0 tx=0x7f80400042f0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f804ec2f640 1 -- 172.21.15.103:0/1751189302 shutdown_connections 2024-09-19T07:00:45.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f804ec2f640 1 --2- 172.21.15.103:0/1751189302 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80300a33b0 0x7f80300a5870 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f804ec2f640 1 --2- 172.21.15.103:0/1751189302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8048106f10 0x7f804819cd60 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f804ec2f640 1 --2- 172.21.15.103:0/1751189302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8048102790 0x7f80481989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f804ec2f640 1 --2- 172.21.15.103:0/1751189302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8048101de0 0x7f8048198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:45.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f804ec2f640 1 -- 172.21.15.103:0/1751189302 >> 172.21.15.103:0/1751189302 conn(0x7f80480fd8c0 msgr2=0x7f80480fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:45.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.289+0000 7f804ec2f640 1 -- 172.21.15.103:0/1751189302 shutdown_connections 2024-09-19T07:00:45.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:45.290+0000 7f804ec2f640 1 -- 172.21.15.103:0/1751189302 wait complete. 2024-09-19T07:00:45.960 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:00:46.210 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:00:46.332 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:45 smithi103 ceph-mon[31158]: pgmap v358: 97 pgs: 97 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T07:00:46.332 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:45 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T07:00:46.332 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:45 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:00:46.332 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:45 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T07:00:46.332 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:45 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:00:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:45 smithi174 ceph-mon[35637]: pgmap v358: 97 pgs: 97 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T07:00:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:45 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T07:00:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:45 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:00:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:45 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T07:00:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:45 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:00:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:45 smithi134 ceph-mon[35356]: pgmap v358: 97 pgs: 97 active+clean; 3.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-09-19T07:00:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:45 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T07:00:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:45 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:00:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:45 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T07:00:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:45 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:00:46.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.662+0000 7f9771d1c640 1 -- 172.21.15.103:0/2021320813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f976c100a80 msgr2=0x7f976c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:46.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.662+0000 7f9771d1c640 1 --2- 172.21.15.103:0/2021320813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f976c100a80 0x7f976c100ee0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f9754009930 tx=0x7f975402a420 comp rx=0 tx=0).stop 2024-09-19T07:00:46.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.662+0000 7f9771d1c640 1 -- 172.21.15.103:0/2021320813 shutdown_connections 2024-09-19T07:00:46.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.662+0000 7f9771d1c640 1 --2- 172.21.15.103:0/2021320813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f976c101420 0x7f976c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.662+0000 7f9771d1c640 1 --2- 172.21.15.103:0/2021320813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f976c100a80 0x7f976c100ee0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.662+0000 7f9771d1c640 1 --2- 172.21.15.103:0/2021320813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f976c106a80 0x7f976c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.662+0000 7f9771d1c640 1 -- 172.21.15.103:0/2021320813 >> 172.21.15.103:0/2021320813 conn(0x7f976c0fc7a0 msgr2=0x7f976c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:46.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.662+0000 7f9771d1c640 1 -- 172.21.15.103:0/2021320813 shutdown_connections 2024-09-19T07:00:46.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.662+0000 7f9771d1c640 1 -- 172.21.15.103:0/2021320813 wait complete. 2024-09-19T07:00:46.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.663+0000 7f9771d1c640 1 Processor -- start 2024-09-19T07:00:46.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.663+0000 7f9771d1c640 1 -- start start 2024-09-19T07:00:46.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.663+0000 7f9771d1c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f976c100a80 0x7f976c0fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.663+0000 7f9771d1c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f976c101420 0x7f976c0fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.663+0000 7f9771d1c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f976c106a80 0x7f976c0fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.663+0000 7f9771d1c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f976c110ec0 con 0x7f976c100a80 2024-09-19T07:00:46.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.663+0000 7f9771d1c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f976c110d40 con 0x7f976c106a80 2024-09-19T07:00:46.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.663+0000 7f9771d1c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f976c111040 con 0x7f976c101420 2024-09-19T07:00:46.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.663+0000 7f976b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f976c100a80 0x7f976c0fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.664+0000 7f976b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f976c100a80 0x7f976c0fcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57296/0 (socket says 172.21.15.103:57296) 2024-09-19T07:00:46.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.664+0000 7f976b7fe640 1 -- 172.21.15.103:0/3410025798 learned_addr learned my addr 172.21.15.103:0/3410025798 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:46.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.664+0000 7f976bfff640 1 --2- 172.21.15.103:0/3410025798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f976c106a80 0x7f976c0fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.664+0000 7f9762ffd640 1 --2- 172.21.15.103:0/3410025798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f976c101420 0x7f976c0fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.664+0000 7f976b7fe640 1 -- 172.21.15.103:0/3410025798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f976c101420 msgr2=0x7f976c0fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:46.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.664+0000 7f976b7fe640 1 --2- 172.21.15.103:0/3410025798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f976c101420 0x7f976c0fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.664+0000 7f976b7fe640 1 -- 172.21.15.103:0/3410025798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f976c106a80 msgr2=0x7f976c0fd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:46.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.664+0000 7f976b7fe640 1 --2- 172.21.15.103:0/3410025798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f976c106a80 0x7f976c0fd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.664+0000 7f976b7fe640 1 -- 172.21.15.103:0/3410025798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976c1a6950 con 0x7f976c100a80 2024-09-19T07:00:46.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.664+0000 7f976b7fe640 1 --2- 172.21.15.103:0/3410025798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f976c100a80 0x7f976c0fcd40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f975c00d770 tx=0x7f975c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.665+0000 7f97697fa640 1 -- 172.21.15.103:0/3410025798 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c017070 con 0x7f976c100a80 2024-09-19T07:00:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.665+0000 7f97697fa640 1 -- 172.21.15.103:0/3410025798 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f975c005500 con 0x7f976c100a80 2024-09-19T07:00:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.665+0000 7f97697fa640 1 -- 172.21.15.103:0/3410025798 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c004e00 con 0x7f976c100a80 2024-09-19T07:00:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.665+0000 7f9771d1c640 1 -- 172.21.15.103:0/3410025798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976c1a6c40 con 0x7f976c100a80 2024-09-19T07:00:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.665+0000 7f9771d1c640 1 -- 172.21.15.103:0/3410025798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f976c1a7100 con 0x7f976c100a80 2024-09-19T07:00:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.667+0000 7f97697fa640 1 -- 172.21.15.103:0/3410025798 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f975c004fa0 con 0x7f976c100a80 2024-09-19T07:00:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.667+0000 7f97740f5740 1 Processor -- start 2024-09-19T07:00:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.667+0000 7f97740f5740 1 -- start start 2024-09-19T07:00:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f97740f5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b774a0 0x55c2f2b77880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f97740f5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b82140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f97740f5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b82680 0x55c2f2b84a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f97740f5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2f2add2e0 con 0x55c2f2b774a0 2024-09-19T07:00:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f97740f5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c2f2a84ce0 con 0x55c2f2b82680 2024-09-19T07:00:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f97740f5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2f2ad92a0 con 0x55c2f2b77dc0 2024-09-19T07:00:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f976bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b774a0 0x55c2f2b77880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f976bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b774a0 0x55c2f2b77880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57306/0 (socket says 172.21.15.103:57306) 2024-09-19T07:00:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f976bfff640 1 -- 172.21.15.103:0/1296676749 learned_addr learned my addr 172.21.15.103:0/1296676749 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f976b7fe640 1 --2- 172.21.15.103:0/1296676749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b82140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f9762ffd640 1 --2- 172.21.15.103:0/1296676749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b82680 0x55c2f2b84a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f97697fa640 1 --2- 172.21.15.103:0/3410025798 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f97580a3500 0x7f97580a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f976bfff640 1 -- 172.21.15.103:0/1296676749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b77dc0 msgr2=0x55c2f2b82140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f976bfff640 1 --2- 172.21.15.103:0/1296676749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b82140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f976bfff640 1 -- 172.21.15.103:0/1296676749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b82680 msgr2=0x55c2f2b84a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f976bfff640 1 --2- 172.21.15.103:0/1296676749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b82680 0x55c2f2b84a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f976bfff640 1 -- 172.21.15.103:0/1296676749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f2a69440 con 0x55c2f2b774a0 2024-09-19T07:00:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f976bfff640 1 --2- 172.21.15.103:0/3410025798 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f97580a3500 0x7f97580a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.668+0000 7f97697fa640 1 -- 172.21.15.103:0/3410025798 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f975c0cf6c0 con 0x7f976c100a80 2024-09-19T07:00:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.669+0000 7f976bfff640 1 --2- 172.21.15.103:0/1296676749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b774a0 0x55c2f2b77880 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f976400a5d0 tx=0x7f976400aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1deecbd8d9575815 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.669+0000 7f9761ffb640 1 -- 172.21.15.103:0/1296676749 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976401a040 con 0x55c2f2b774a0 2024-09-19T07:00:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.669+0000 7f9761ffb640 1 -- 172.21.15.103:0/1296676749 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9764021ba0 con 0x55c2f2b774a0 2024-09-19T07:00:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.669+0000 7f9761ffb640 1 -- 172.21.15.103:0/1296676749 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976401f350 con 0x55c2f2b774a0 2024-09-19T07:00:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.669+0000 7f97740f5740 1 -- 172.21.15.103:0/1296676749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b774a0 msgr2=0x55c2f2b77880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.669+0000 7f976bfff640 1 --2- 172.21.15.103:0/3410025798 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f97580a3500 0x7f97580a59c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9764020000 tx=0x7f9764021000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.669+0000 7f97740f5740 1 --2- 172.21.15.103:0/1296676749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b774a0 0x55c2f2b77880 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f976400a5d0 tx=0x7f976400aa90 comp rx=0 tx=0).stop 2024-09-19T07:00:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.669+0000 7f97740f5740 1 -- 172.21.15.103:0/1296676749 shutdown_connections 2024-09-19T07:00:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.669+0000 7f97740f5740 1 --2- 172.21.15.103:0/1296676749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b82680 0x55c2f2b84a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.669+0000 7f97740f5740 1 --2- 172.21.15.103:0/1296676749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b82140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.669+0000 7f97740f5740 1 --2- 172.21.15.103:0/1296676749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b774a0 0x55c2f2b77880 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.669+0000 7f97740f5740 1 -- 172.21.15.103:0/1296676749 >> 172.21.15.103:0/1296676749 conn(0x55c2f2b73a30 msgr2=0x55c2f2b73e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.670+0000 7f97740f5740 1 -- 172.21.15.103:0/1296676749 shutdown_connections 2024-09-19T07:00:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.670+0000 7f97740f5740 1 -- 172.21.15.103:0/1296676749 wait complete. 2024-09-19T07:00:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.670+0000 7f97740f5740 1 -- start start 2024-09-19T07:00:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.670+0000 7f97740f5740 1 Processor -- start 2024-09-19T07:00:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.670+0000 7f97740f5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b774a0 0x55c2f2b81ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.670+0000 7f97740f5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b7f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.670+0000 7f97740f5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b82680 0x55c2f2b7f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.670+0000 7f97740f5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2f2b85250 con 0x55c2f2b77dc0 2024-09-19T07:00:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.670+0000 7f97740f5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c2f2a98050 con 0x55c2f2b82680 2024-09-19T07:00:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.670+0000 7f97740f5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2f2b853d0 con 0x55c2f2b774a0 2024-09-19T07:00:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.670+0000 7f97740f5740 10 client.0 _ll_register_callbacks cb 0x55c2f2ad8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.671+0000 7f9762ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b7f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.671+0000 7f976bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b82680 0x55c2f2b7f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.671+0000 7f9762ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b7f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57312/0 (socket says 172.21.15.103:57312) 2024-09-19T07:00:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.671+0000 7f9762ffd640 1 -- 172.21.15.103:0/2984603267 learned_addr learned my addr 172.21.15.103:0/2984603267 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.671+0000 7f976b7fe640 1 --2- 172.21.15.103:0/2984603267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b774a0 0x55c2f2b81ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.671+0000 7f9762ffd640 1 -- 172.21.15.103:0/2984603267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b774a0 msgr2=0x55c2f2b81ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.671+0000 7f9762ffd640 1 --2- 172.21.15.103:0/2984603267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b774a0 0x55c2f2b81ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.671+0000 7f9762ffd640 1 -- 172.21.15.103:0/2984603267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b82680 msgr2=0x55c2f2b7f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.671+0000 7f9762ffd640 1 --2- 172.21.15.103:0/2984603267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b82680 0x55c2f2b7f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.671+0000 7f9762ffd640 1 -- 172.21.15.103:0/2984603267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f284e3c0 con 0x55c2f2b77dc0 2024-09-19T07:00:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.671+0000 7f976bfff640 1 --2- 172.21.15.103:0/2984603267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b82680 0x55c2f2b7f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:00:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.671+0000 7f9762ffd640 1 --2- 172.21.15.103:0/2984603267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b7f1a0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f9754005940 tx=0x7f97540318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5ecd5fca82f256 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.671+0000 7f974affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.671+0000 7f974affd640 1 -- 172.21.15.103:0/2984603267 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754035070 con 0x55c2f2b77dc0 2024-09-19T07:00:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.671+0000 7f974affd640 1 -- 172.21.15.103:0/2984603267 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f9754004240 con 0x55c2f2b77dc0 2024-09-19T07:00:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.671+0000 7f97740f5740 1 -- 172.21.15.103:0/2984603267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9764021ba0 con 0x55c2f2b77dc0 2024-09-19T07:00:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.672+0000 7f974affd640 1 -- 172.21.15.103:0/2984603267 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9754004440 con 0x55c2f2b77dc0 2024-09-19T07:00:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.672+0000 7f97740f5740 10 client.19416 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.672+0000 7f97740f5740 1 -- 172.21.15.103:0/2984603267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2f2b90b90 con 0x55c2f2b77dc0 2024-09-19T07:00:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.672+0000 7f974affd640 1 -- 172.21.15.103:0/2984603267 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9764021ba0 con 0x55c2f2b77dc0 2024-09-19T07:00:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.672+0000 7f974affd640 1 -- 172.21.15.103:0/2984603267 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9754004b80 con 0x55c2f2b77dc0 2024-09-19T07:00:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.672+0000 7f97740f5740 10 client.19416 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.672+0000 7f97740f5740 10 client.19416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.672+0000 7f97740f5740 20 client.19416 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.672+0000 7f97740f5740 1 --2- 172.21.15.103:0/2984603267 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c2f2b920a0 0x55c2f2b944c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.672+0000 7f97740f5740 4 client.19416 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:00:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.672+0000 7f97740f5740 1 -- 172.21.15.103:0/2984603267 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2f2aea850 con 0x55c2f2b920a0 2024-09-19T07:00:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.672+0000 7f976b7fe640 1 --2- 172.21.15.103:0/2984603267 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c2f2b920a0 0x55c2f2b944c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.673+0000 7f974affd640 10 client.19416 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.673+0000 7f976b7fe640 1 --2- 172.21.15.103:0/2984603267 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c2f2b920a0 0x55c2f2b944c0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67e3fa4930fb66e9 server_cookie=e2ffc7e5813df3d2 in_seq=0 out_seq=0 2024-09-19T07:00:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.674+0000 7f974affd640 1 -- 172.21.15.103:0/2984603267 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2f2aea850 con 0x55c2f2b920a0 2024-09-19T07:00:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.674+0000 7f974affd640 10 client.19416 handle_command_reply: tid=0 2024-09-19T07:00:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.674+0000 7f97740f5740 1 client.19416 shutdown 2024-09-19T07:00:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.674+0000 7f97740f5740 1 -- 172.21.15.103:0/2984603267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b77dc0 msgr2=0x55c2f2b7f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.674+0000 7f97740f5740 1 --2- 172.21.15.103:0/2984603267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b7f1a0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f9754005940 tx=0x7f97540318d0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.674+0000 7f97740f5740 1 -- 172.21.15.103:0/2984603267 shutdown_connections 2024-09-19T07:00:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.674+0000 7f97740f5740 1 --2- 172.21.15.103:0/2984603267 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c2f2b920a0 0x55c2f2b944c0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.674+0000 7f97740f5740 1 --2- 172.21.15.103:0/2984603267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b82680 0x55c2f2b7f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.674+0000 7f97740f5740 1 --2- 172.21.15.103:0/2984603267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b7f1a0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.674+0000 7f97740f5740 1 --2- 172.21.15.103:0/2984603267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b774a0 0x55c2f2b81ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.674+0000 7f974affd640 0 client.19416 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.674+0000 7f97740f5740 1 -- 172.21.15.103:0/2984603267 >> 172.21.15.103:0/2984603267 conn(0x55c2f2b73a30 msgr2=0x55c2f2b79710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.674+0000 7f97740f5740 1 -- 172.21.15.103:0/2984603267 shutdown_connections 2024-09-19T07:00:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.674+0000 7f97740f5740 1 -- 172.21.15.103:0/2984603267 wait complete. 2024-09-19T07:00:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.674+0000 7f97740f5740 20 client.19416 trim_cache size 0 max 16384 2024-09-19T07:00:46.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.691+0000 7f97740f5740 1 Processor -- start 2024-09-19T07:00:46.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.691+0000 7f97740f5740 1 -- start start 2024-09-19T07:00:46.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.691+0000 7f97740f5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b774a0 0x55c2f2b814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.691+0000 7f97740f5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b99d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.691+0000 7f97740f5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b82680 0x55c2f2b9a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.691+0000 7f97740f5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1af9c0 con 0x55c2f2b774a0 2024-09-19T07:00:46.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.691+0000 7f97740f5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1b1500 con 0x55c2f2b77dc0 2024-09-19T07:00:46.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.691+0000 7f97740f5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1b0090 con 0x55c2f2b82680 2024-09-19T07:00:46.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.691+0000 7f976b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b774a0 0x55c2f2b814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.691+0000 7f976b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b774a0 0x55c2f2b814d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57328/0 (socket says 172.21.15.103:57328) 2024-09-19T07:00:46.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.691+0000 7f976b7fe640 1 -- 172.21.15.103:0/4073017414 learned_addr learned my addr 172.21.15.103:0/4073017414 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:46.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.692+0000 7f976bfff640 1 --2- 172.21.15.103:0/4073017414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b82680 0x55c2f2b9a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.692+0000 7f9762ffd640 1 --2- 172.21.15.103:0/4073017414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b99d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.692+0000 7f976b7fe640 1 -- 172.21.15.103:0/4073017414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b82680 msgr2=0x55c2f2b9a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:46.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.692+0000 7f976b7fe640 1 --2- 172.21.15.103:0/4073017414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b82680 0x55c2f2b9a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.692+0000 7f976b7fe640 1 -- 172.21.15.103:0/4073017414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b77dc0 msgr2=0x55c2f2b99d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.692+0000 7f976b7fe640 1 --2- 172.21.15.103:0/4073017414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b99d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.692+0000 7f976b7fe640 1 -- 172.21.15.103:0/4073017414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f2b734d0 con 0x55c2f2b774a0 2024-09-19T07:00:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.692+0000 7f9762ffd640 1 --2- 172.21.15.103:0/4073017414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b99d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.692+0000 7f976b7fe640 1 --2- 172.21.15.103:0/4073017414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b774a0 0x55c2f2b814d0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f975c009c70 tx=0x7f975c0d1bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217488fdcdc9f764 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.692+0000 7f97617fa640 1 -- 172.21.15.103:0/4073017414 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c020040 con 0x55c2f2b774a0 2024-09-19T07:00:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.692+0000 7f97617fa640 1 -- 172.21.15.103:0/4073017414 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f975c09c080 con 0x55c2f2b774a0 2024-09-19T07:00:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.692+0000 7f97617fa640 1 -- 172.21.15.103:0/4073017414 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c009cc0 con 0x55c2f2b774a0 2024-09-19T07:00:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.693+0000 7f97740f5740 1 -- 172.21.15.103:0/4073017414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b774a0 msgr2=0x55c2f2b814d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.693+0000 7f97740f5740 1 --2- 172.21.15.103:0/4073017414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b774a0 0x55c2f2b814d0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f975c009c70 tx=0x7f975c0d1bd0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.693+0000 7f97740f5740 1 -- 172.21.15.103:0/4073017414 shutdown_connections 2024-09-19T07:00:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.693+0000 7f97740f5740 1 --2- 172.21.15.103:0/4073017414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b82680 0x55c2f2b9a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.693+0000 7f97740f5740 1 --2- 172.21.15.103:0/4073017414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b99d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.693+0000 7f97740f5740 1 --2- 172.21.15.103:0/4073017414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b774a0 0x55c2f2b814d0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.693+0000 7f97740f5740 1 -- 172.21.15.103:0/4073017414 >> 172.21.15.103:0/4073017414 conn(0x55c2f2b73a30 msgr2=0x55c2f2b72b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.693+0000 7f97740f5740 1 -- 172.21.15.103:0/4073017414 shutdown_connections 2024-09-19T07:00:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.693+0000 7f97740f5740 1 -- 172.21.15.103:0/4073017414 wait complete. 2024-09-19T07:00:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.693+0000 7f97740f5740 1 -- start start 2024-09-19T07:00:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.693+0000 7f97740f5740 1 Processor -- start 2024-09-19T07:00:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f97740f5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b774a0 0x55c2f2b88370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f97740f5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b888b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f97740f5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b82680 0x55c2f2b88df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f97740f5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1b0e30 con 0x55c2f2b82680 2024-09-19T07:00:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f97740f5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1b0760 con 0x55c2f2b774a0 2024-09-19T07:00:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f97740f5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1ad9c0 con 0x55c2f2b77dc0 2024-09-19T07:00:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f97740f5740 10 client.0 _ll_register_callbacks cb 0x7f9754009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f976bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b82680 0x55c2f2b88df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f976bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b82680 0x55c2f2b88df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57344/0 (socket says 172.21.15.103:57344) 2024-09-19T07:00:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f976bfff640 1 -- 172.21.15.103:0/2285269925 learned_addr learned my addr 172.21.15.103:0/2285269925 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f9762ffd640 1 --2- 172.21.15.103:0/2285269925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b888b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f976b7fe640 1 --2- 172.21.15.103:0/2285269925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b774a0 0x55c2f2b88370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f976bfff640 1 -- 172.21.15.103:0/2285269925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b77dc0 msgr2=0x55c2f2b888b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f976bfff640 1 --2- 172.21.15.103:0/2285269925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b888b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f976bfff640 1 -- 172.21.15.103:0/2285269925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b774a0 msgr2=0x55c2f2b88370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f976bfff640 1 --2- 172.21.15.103:0/2285269925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b774a0 0x55c2f2b88370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f976bfff640 1 -- 172.21.15.103:0/2285269925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f2bfbd70 con 0x55c2f2b82680 2024-09-19T07:00:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f976bfff640 1 --2- 172.21.15.103:0/2285269925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b82680 0x55c2f2b88df0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f976400a5d0 tx=0x7f9764004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b617c44ccdc6b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f974affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f974affd640 1 -- 172.21.15.103:0/2285269925 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764037060 con 0x55c2f2b82680 2024-09-19T07:00:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f974affd640 1 -- 172.21.15.103:0/2285269925 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f97640205b0 con 0x55c2f2b82680 2024-09-19T07:00:46.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.694+0000 7f97740f5740 1 -- 172.21.15.103:0/2285269925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f975c09c080 con 0x55c2f2b82680 2024-09-19T07:00:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.695+0000 7f974affd640 1 -- 172.21.15.103:0/2285269925 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f976401a040 con 0x55c2f2b82680 2024-09-19T07:00:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.695+0000 7f97740f5740 10 client.19422 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.695+0000 7f97740f5740 1 -- 172.21.15.103:0/2285269925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2f2bfcfa0 con 0x55c2f2b82680 2024-09-19T07:00:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.695+0000 7f974affd640 1 -- 172.21.15.103:0/2285269925 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f975c09c080 con 0x55c2f2b82680 2024-09-19T07:00:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.695+0000 7f974affd640 1 -- 172.21.15.103:0/2285269925 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9764028040 con 0x55c2f2b82680 2024-09-19T07:00:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.695+0000 7f97740f5740 10 client.19422 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.695+0000 7f97740f5740 10 client.19422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.695+0000 7f97740f5740 20 client.19422 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.695+0000 7f97740f5740 1 --2- 172.21.15.103:0/2285269925 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c2f2bfd590 0x55c2f2bff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:46.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.695+0000 7f97740f5740 4 client.19422 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:00:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.695+0000 7f97740f5740 1 -- 172.21.15.103:0/2285269925 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c2f2ae4d60 con 0x55c2f2bfd590 2024-09-19T07:00:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.695+0000 7f976b7fe640 1 --2- 172.21.15.103:0/2285269925 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c2f2bfd590 0x55c2f2bff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f976b7fe640 1 --2- 172.21.15.103:0/2285269925 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c2f2bfd590 0x55c2f2bff9b0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2fe08693bc03662 server_cookie=79a4ec598e9fe364 in_seq=0 out_seq=0 2024-09-19T07:00:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f974affd640 10 client.19422 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f974affd640 1 -- 172.21.15.103:0/2285269925 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c2f2ae4d60 con 0x55c2f2bfd590 2024-09-19T07:00:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f974affd640 10 client.19422 handle_command_reply: tid=0 2024-09-19T07:00:46.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f97740f5740 1 client.19422 shutdown 2024-09-19T07:00:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f97740f5740 1 -- 172.21.15.103:0/2285269925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b82680 msgr2=0x55c2f2b88df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f97740f5740 1 --2- 172.21.15.103:0/2285269925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b82680 0x55c2f2b88df0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f976400a5d0 tx=0x7f9764004a40 comp rx=0 tx=0).stop 2024-09-19T07:00:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f97740f5740 1 -- 172.21.15.103:0/2285269925 shutdown_connections 2024-09-19T07:00:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f97740f5740 1 --2- 172.21.15.103:0/2285269925 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c2f2bfd590 0x55c2f2bff9b0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f97740f5740 1 --2- 172.21.15.103:0/2285269925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f2b82680 0x55c2f2b88df0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f97740f5740 1 --2- 172.21.15.103:0/2285269925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f2b77dc0 0x55c2f2b888b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f97740f5740 1 --2- 172.21.15.103:0/2285269925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c2f2b774a0 0x55c2f2b88370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f97740f5740 1 -- 172.21.15.103:0/2285269925 >> 172.21.15.103:0/2285269925 conn(0x55c2f2b73a30 msgr2=0x55c2f2b7ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:46.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f974affd640 0 client.19422 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:46.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f97740f5740 1 -- 172.21.15.103:0/2285269925 shutdown_connections 2024-09-19T07:00:46.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f97740f5740 1 -- 172.21.15.103:0/2285269925 wait complete. 2024-09-19T07:00:46.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.696+0000 7f97740f5740 20 client.19422 trim_cache size 0 max 16384 2024-09-19T07:00:46.706 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:00:46.706 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:00:46.707 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:00:46.707 INFO:teuthology.orchestra.run.smithi103.stdout: "a2cecf98-c2f3-4beb-9d96-dec1409c2bcc": { 2024-09-19T07:00:46.707 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:00:46.707 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "a2cecf98-c2f3-4beb-9d96-dec1409c2bcc", 2024-09-19T07:00:46.707 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:00:46.707 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:00:46.707 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:00:46.707 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:00:46.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.697+0000 7f9771d1c640 1 -- 172.21.15.103:0/3410025798 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f97580a3500 msgr2=0x7f97580a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:46.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.697+0000 7f9771d1c640 1 --2- 172.21.15.103:0/3410025798 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f97580a3500 0x7f97580a59c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9764020000 tx=0x7f9764021000 comp rx=0 tx=0).stop 2024-09-19T07:00:46.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.697+0000 7f9771d1c640 1 -- 172.21.15.103:0/3410025798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f976c100a80 msgr2=0x7f976c0fcd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:46.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.697+0000 7f9771d1c640 1 --2- 172.21.15.103:0/3410025798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f976c100a80 0x7f976c0fcd40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f975c00d770 tx=0x7f975c00dc30 comp rx=0 tx=0).stop 2024-09-19T07:00:46.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.697+0000 7f9771d1c640 1 -- 172.21.15.103:0/3410025798 shutdown_connections 2024-09-19T07:00:46.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.697+0000 7f9771d1c640 1 --2- 172.21.15.103:0/3410025798 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f97580a3500 0x7f97580a59c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.697+0000 7f9771d1c640 1 --2- 172.21.15.103:0/3410025798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f976c106a80 0x7f976c0fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.697+0000 7f9771d1c640 1 --2- 172.21.15.103:0/3410025798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f976c101420 0x7f976c0fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.697+0000 7f9771d1c640 1 --2- 172.21.15.103:0/3410025798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f976c100a80 0x7f976c0fcd40 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:46.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.697+0000 7f9771d1c640 1 -- 172.21.15.103:0/3410025798 >> 172.21.15.103:0/3410025798 conn(0x7f976c0fc7a0 msgr2=0x7f976c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:46.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.697+0000 7f9771d1c640 1 -- 172.21.15.103:0/3410025798 shutdown_connections 2024-09-19T07:00:46.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:00:46.697+0000 7f9771d1c640 1 -- 172.21.15.103:0/3410025798 wait complete. 2024-09-19T07:00:47.331 INFO:tasks.cephfs.filesystem:scrub status for tag:a2cecf98-c2f3-4beb-9d96-dec1409c2bcc - {'path': '/', 'tag': 'a2cecf98-c2f3-4beb-9d96-dec1409c2bcc', 'options': 'recursive,force'} 2024-09-19T07:00:48.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:47 smithi174 ceph-mon[35637]: pgmap v359: 97 pgs: 97 active+clean; 3.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-09-19T07:00:48.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:47 smithi103 ceph-mon[31158]: pgmap v359: 97 pgs: 97 active+clean; 3.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-09-19T07:00:48.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:47 smithi134 ceph-mon[35356]: pgmap v359: 97 pgs: 97 active+clean; 3.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-09-19T07:00:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.448+0000 7fc5ea3d6640 1 -- 172.21.15.103:0/1493854108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5e41069c0 msgr2=0x7fc5e4106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.448+0000 7fc5ea3d6640 1 --2- 172.21.15.103:0/1493854108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5e41069c0 0x7fc5e4106da0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009930 tx=0x7fc5d402a420 comp rx=0 tx=0).stop 2024-09-19T07:00:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.448+0000 7fc5ea3d6640 1 -- 172.21.15.103:0/1493854108 shutdown_connections 2024-09-19T07:00:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.448+0000 7fc5ea3d6640 1 --2- 172.21.15.103:0/1493854108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5e4101350 0x7fc5e410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.448+0000 7fc5ea3d6640 1 --2- 172.21.15.103:0/1493854108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e41009b0 0x7fc5e4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.448+0000 7fc5ea3d6640 1 --2- 172.21.15.103:0/1493854108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5e41069c0 0x7fc5e4106da0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.448+0000 7fc5ea3d6640 1 -- 172.21.15.103:0/1493854108 >> 172.21.15.103:0/1493854108 conn(0x7fc5e4077990 msgr2=0x7fc5e40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.448+0000 7fc5ea3d6640 1 -- 172.21.15.103:0/1493854108 shutdown_connections 2024-09-19T07:00:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.448+0000 7fc5ea3d6640 1 -- 172.21.15.103:0/1493854108 wait complete. 2024-09-19T07:00:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.448+0000 7fc5ea3d6640 1 Processor -- start 2024-09-19T07:00:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5ea3d6640 1 -- start start 2024-09-19T07:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5ea3d6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5e41009b0 0x7fc5e4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5ea3d6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5e4101350 0x7fc5e41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5ea3d6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e41069c0 0x7fc5e41a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5ea3d6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e4110e00 con 0x7fc5e4101350 2024-09-19T07:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5ea3d6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e4110c80 con 0x7fc5e41069c0 2024-09-19T07:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5ea3d6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e4110f80 con 0x7fc5e41009b0 2024-09-19T07:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5e37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5e4101350 0x7fc5e41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5e37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5e4101350 0x7fc5e41996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47976/0 (socket says 172.21.15.103:47976) 2024-09-19T07:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5e37fe640 1 -- 172.21.15.103:0/67990938 learned_addr learned my addr 172.21.15.103:0/67990938 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5e3fff640 1 --2- 172.21.15.103:0/67990938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5e41009b0 0x7fc5e4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5e8973640 1 --2- 172.21.15.103:0/67990938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e41069c0 0x7fc5e41a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5e37fe640 1 -- 172.21.15.103:0/67990938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5e41009b0 msgr2=0x7fc5e4199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5e37fe640 1 --2- 172.21.15.103:0/67990938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5e41009b0 0x7fc5e4199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5e37fe640 1 -- 172.21.15.103:0/67990938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e41069c0 msgr2=0x7fc5e41a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5e37fe640 1 --2- 172.21.15.103:0/67990938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e41069c0 0x7fc5e41a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5e37fe640 1 -- 172.21.15.103:0/67990938 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e41a4750 con 0x7fc5e4101350 2024-09-19T07:00:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5e8973640 1 --2- 172.21.15.103:0/67990938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e41069c0 0x7fc5e41a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:00:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.449+0000 7fc5e37fe640 1 --2- 172.21.15.103:0/67990938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5e4101350 0x7fc5e41996d0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800d950 tx=0x7fc5d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.450+0000 7fc5e17fa640 1 -- 172.21.15.103:0/67990938 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8017070 con 0x7fc5e4101350 2024-09-19T07:00:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.450+0000 7fc5e17fa640 1 -- 172.21.15.103:0/67990938 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d8009c30 con 0x7fc5e4101350 2024-09-19T07:00:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.450+0000 7fc5ea3d6640 1 -- 172.21.15.103:0/67990938 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e41a4920 con 0x7fc5e4101350 2024-09-19T07:00:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.450+0000 7fc5e17fa640 1 -- 172.21.15.103:0/67990938 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d800f6b0 con 0x7fc5e4101350 2024-09-19T07:00:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.450+0000 7fc5ea3d6640 1 -- 172.21.15.103:0/67990938 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e41a4e40 con 0x7fc5e4101350 2024-09-19T07:00:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.452+0000 7fc5e17fa640 1 -- 172.21.15.103:0/67990938 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc5d80109d0 con 0x7fc5e4101350 2024-09-19T07:00:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.452+0000 7fc5ed24f740 1 Processor -- start 2024-09-19T07:00:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.452+0000 7fc5ed24f740 1 -- start start 2024-09-19T07:00:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.452+0000 7fc5ed24f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc6852af0 0x55cdc6852ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.452+0000 7fc5ed24f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6853410 0x55cdc685d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.452+0000 7fc5ed24f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc685dcb0 0x55cdc68600a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.452+0000 7fc5ed24f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cdc67b72f0 con 0x55cdc685dcb0 2024-09-19T07:00:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5ed24f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cdc675f270 con 0x55cdc6852af0 2024-09-19T07:00:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5ed24f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cdc67b32b0 con 0x55cdc6853410 2024-09-19T07:00:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5e17fa640 1 --2- 172.21.15.103:0/67990938 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc5cc0a3460 0x7fc5cc0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5e37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc685dcb0 0x55cdc68600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5e37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc685dcb0 0x55cdc68600a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47990/0 (socket says 172.21.15.103:47990) 2024-09-19T07:00:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5e37fe640 1 -- 172.21.15.103:0/26488934 learned_addr learned my addr 172.21.15.103:0/26488934 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5e8973640 1 --2- 172.21.15.103:0/67990938 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc5cc0a3460 0x7fc5cc0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5e3fff640 1 --2- 172.21.15.103:0/26488934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6853410 0x55cdc685d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5e17fa640 1 -- 172.21.15.103:0/67990938 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc5d80cf990 con 0x7fc5e4101350 2024-09-19T07:00:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5e8973640 1 --2- 172.21.15.103:0/26488934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc6852af0 0x55cdc6852ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5e37fe640 1 -- 172.21.15.103:0/26488934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6853410 msgr2=0x55cdc685d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5e37fe640 1 --2- 172.21.15.103:0/26488934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6853410 0x55cdc685d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5e37fe640 1 -- 172.21.15.103:0/26488934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc6852af0 msgr2=0x55cdc6852ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5e37fe640 1 --2- 172.21.15.103:0/26488934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc6852af0 0x55cdc6852ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5e37fe640 1 -- 172.21.15.103:0/26488934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc675e340 con 0x55cdc685dcb0 2024-09-19T07:00:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5e3fff640 1 --2- 172.21.15.103:0/26488934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6853410 0x55cdc685d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5e37fe640 1 --2- 172.21.15.103:0/26488934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc685dcb0 0x55cdc68600a0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80a01c0 tx=0x7fc5d800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8ded50b77b11e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5e8973640 1 --2- 172.21.15.103:0/67990938 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc5cc0a3460 0x7fc5cc0a5920 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc005c40 tx=0x7fc5dc0094a0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5ca7fc640 1 -- 172.21.15.103:0/26488934 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8020070 con 0x55cdc685dcb0 2024-09-19T07:00:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5ca7fc640 1 -- 172.21.15.103:0/26488934 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d80b4e50 con 0x55cdc685dcb0 2024-09-19T07:00:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.453+0000 7fc5ca7fc640 1 -- 172.21.15.103:0/26488934 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80a48c0 con 0x55cdc685dcb0 2024-09-19T07:00:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.454+0000 7fc5ed24f740 1 -- 172.21.15.103:0/26488934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc685dcb0 msgr2=0x55cdc68600a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.454+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/26488934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc685dcb0 0x55cdc68600a0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80a01c0 tx=0x7fc5d800bf60 comp rx=0 tx=0).stop 2024-09-19T07:00:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.454+0000 7fc5ed24f740 1 -- 172.21.15.103:0/26488934 shutdown_connections 2024-09-19T07:00:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.454+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/26488934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc685dcb0 0x55cdc68600a0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.454+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/26488934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6853410 0x55cdc685d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.454+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/26488934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc6852af0 0x55cdc6852ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.454+0000 7fc5ed24f740 1 -- 172.21.15.103:0/26488934 >> 172.21.15.103:0/26488934 conn(0x55cdc684f060 msgr2=0x55cdc684f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.454+0000 7fc5ed24f740 1 -- 172.21.15.103:0/26488934 shutdown_connections 2024-09-19T07:00:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.454+0000 7fc5ed24f740 1 -- 172.21.15.103:0/26488934 wait complete. 2024-09-19T07:00:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.454+0000 7fc5ed24f740 1 -- start start 2024-09-19T07:00:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5ed24f740 1 Processor -- start 2024-09-19T07:00:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5ed24f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc6852af0 0x55cdc685abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5ed24f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6853410 0x55cdc685b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5ed24f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc685dcb0 0x55cdc685b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5ed24f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cdc6860820 con 0x55cdc6852af0 2024-09-19T07:00:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5ed24f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cdc6740290 con 0x55cdc685dcb0 2024-09-19T07:00:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5ed24f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cdc68609a0 con 0x55cdc6853410 2024-09-19T07:00:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5e3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc6852af0 0x55cdc685abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5ed24f740 10 client.0 _ll_register_callbacks cb 0x55cdc6771140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5e3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc6852af0 0x55cdc685abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48004/0 (socket says 172.21.15.103:48004) 2024-09-19T07:00:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5e3fff640 1 -- 172.21.15.103:0/4243515820 learned_addr learned my addr 172.21.15.103:0/4243515820 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5e8973640 1 --2- 172.21.15.103:0/4243515820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc685dcb0 0x55cdc685b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5e3fff640 1 -- 172.21.15.103:0/4243515820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6853410 msgr2=0x55cdc685b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5e3fff640 1 --2- 172.21.15.103:0/4243515820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6853410 0x55cdc685b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5e3fff640 1 -- 172.21.15.103:0/4243515820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc685dcb0 msgr2=0x55cdc685b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5e3fff640 1 --2- 172.21.15.103:0/4243515820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc685dcb0 0x55cdc685b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5e3fff640 1 -- 172.21.15.103:0/4243515820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc64ca210 con 0x55cdc6852af0 2024-09-19T07:00:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.455+0000 7fc5e8973640 1 --2- 172.21.15.103:0/4243515820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc685dcb0 0x55cdc685b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:00:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5e3fff640 1 --2- 172.21.15.103:0/4243515820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc6852af0 0x55cdc685abb0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fc5d402a940 tx=0x7fc5d4031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d727f195a3df60 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5c37fe640 1 -- 172.21.15.103:0/4243515820 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4035040 con 0x55cdc6852af0 2024-09-19T07:00:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5c37fe640 1 -- 172.21.15.103:0/4243515820 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc5d4004240 con 0x55cdc6852af0 2024-09-19T07:00:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5ed24f740 1 -- 172.21.15.103:0/4243515820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5d80b4e50 con 0x55cdc6852af0 2024-09-19T07:00:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5c37fe640 1 -- 172.21.15.103:0/4243515820 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc5d4004440 con 0x55cdc6852af0 2024-09-19T07:00:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5ed24f740 10 client.19434 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5ed24f740 1 -- 172.21.15.103:0/4243515820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cdc6596620 con 0x55cdc6852af0 2024-09-19T07:00:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5c37fe640 1 -- 172.21.15.103:0/4243515820 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d80b4e50 con 0x55cdc6852af0 2024-09-19T07:00:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5c37fe640 1 -- 172.21.15.103:0/4243515820 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc5d404e6c0 con 0x55cdc6852af0 2024-09-19T07:00:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5ed24f740 10 client.19434 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5ed24f740 10 client.19434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5ed24f740 20 client.19434 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/4243515820 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cdc686db70 0x55cdc686ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5ed24f740 4 client.19434 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:00:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5ed24f740 1 -- 172.21.15.103:0/4243515820 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdc67c4a20 con 0x55cdc686db70 2024-09-19T07:00:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.456+0000 7fc5e37fe640 1 --2- 172.21.15.103:0/4243515820 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cdc686db70 0x55cdc686ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.457+0000 7fc5c37fe640 10 client.19434 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.457+0000 7fc5e37fe640 1 --2- 172.21.15.103:0/4243515820 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cdc686db70 0x55cdc686ff60 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=481f2179c7222a32 server_cookie=3d48694dc4d21cb5 in_seq=0 out_seq=0 2024-09-19T07:00:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.458+0000 7fc5c37fe640 1 -- 172.21.15.103:0/4243515820 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cdc67c4a20 con 0x55cdc686db70 2024-09-19T07:00:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.458+0000 7fc5c37fe640 10 client.19434 handle_command_reply: tid=0 2024-09-19T07:00:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.458+0000 7fc5ed24f740 1 client.19434 shutdown 2024-09-19T07:00:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.458+0000 7fc5ed24f740 1 -- 172.21.15.103:0/4243515820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc6852af0 msgr2=0x55cdc685abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.458+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/4243515820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc6852af0 0x55cdc685abb0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fc5d402a940 tx=0x7fc5d4031860 comp rx=0 tx=0).stop 2024-09-19T07:00:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.458+0000 7fc5ed24f740 1 -- 172.21.15.103:0/4243515820 shutdown_connections 2024-09-19T07:00:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.458+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/4243515820 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cdc686db70 0x55cdc686ff60 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.458+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/4243515820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc685dcb0 0x55cdc685b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.458+0000 7fc5c37fe640 0 client.19434 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.458+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/4243515820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6853410 0x55cdc685b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.458+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/4243515820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc6852af0 0x55cdc685abb0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.458+0000 7fc5ed24f740 1 -- 172.21.15.103:0/4243515820 >> 172.21.15.103:0/4243515820 conn(0x55cdc684f060 msgr2=0x55cdc6854d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.458+0000 7fc5ed24f740 1 -- 172.21.15.103:0/4243515820 shutdown_connections 2024-09-19T07:00:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.458+0000 7fc5ed24f740 1 -- 172.21.15.103:0/4243515820 wait complete. 2024-09-19T07:00:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.458+0000 7fc5ed24f740 20 client.19434 trim_cache size 0 max 16384 2024-09-19T07:00:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.472+0000 7fc5ed24f740 1 Processor -- start 2024-09-19T07:00:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.472+0000 7fc5ed24f740 1 -- start start 2024-09-19T07:00:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.472+0000 7fc5ed24f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6852af0 0x55cdc685b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.472+0000 7fc5ed24f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc6853410 0x55cdc6875870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.472+0000 7fc5ed24f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc685dcb0 0x55cdc6875db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.472+0000 7fc5ed24f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e41ad8e0 con 0x55cdc6853410 2024-09-19T07:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.472+0000 7fc5ed24f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e41af420 con 0x55cdc685dcb0 2024-09-19T07:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.472+0000 7fc5ed24f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e41adfb0 con 0x55cdc6852af0 2024-09-19T07:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.472+0000 7fc5e37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc6853410 0x55cdc6875870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.472+0000 7fc5e37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc6853410 0x55cdc6875870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48010/0 (socket says 172.21.15.103:48010) 2024-09-19T07:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.472+0000 7fc5e37fe640 1 -- 172.21.15.103:0/261098299 learned_addr learned my addr 172.21.15.103:0/261098299 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.472+0000 7fc5e3fff640 1 --2- 172.21.15.103:0/261098299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6852af0 0x55cdc685b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.473+0000 7fc5e37fe640 1 -- 172.21.15.103:0/261098299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6852af0 msgr2=0x55cdc685b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.473+0000 7fc5e37fe640 1 --2- 172.21.15.103:0/261098299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6852af0 0x55cdc685b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.473+0000 7fc5e37fe640 1 -- 172.21.15.103:0/261098299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc685dcb0 msgr2=0x55cdc6875db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.473+0000 7fc5e37fe640 1 --2- 172.21.15.103:0/261098299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc685dcb0 0x55cdc6875db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.473+0000 7fc5e37fe640 1 -- 172.21.15.103:0/261098299 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc68501f0 con 0x55cdc6853410 2024-09-19T07:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.473+0000 7fc5e37fe640 1 --2- 172.21.15.103:0/261098299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc6853410 0x55cdc6875870 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80cd170 tx=0x7fc5d8097860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88007d4f907a0b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.473+0000 7fc5c9ffb640 1 -- 172.21.15.103:0/261098299 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80b1070 con 0x55cdc6853410 2024-09-19T07:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.473+0000 7fc5c9ffb640 1 -- 172.21.15.103:0/261098299 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d80b4e50 con 0x55cdc6853410 2024-09-19T07:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.473+0000 7fc5c9ffb640 1 -- 172.21.15.103:0/261098299 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8020040 con 0x55cdc6853410 2024-09-19T07:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.473+0000 7fc5ed24f740 1 -- 172.21.15.103:0/261098299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc6853410 msgr2=0x55cdc6875870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.473+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/261098299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc6853410 0x55cdc6875870 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80cd170 tx=0x7fc5d8097860 comp rx=0 tx=0).stop 2024-09-19T07:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.473+0000 7fc5ed24f740 1 -- 172.21.15.103:0/261098299 shutdown_connections 2024-09-19T07:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.473+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/261098299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc685dcb0 0x55cdc6875db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.473+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/261098299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc6853410 0x55cdc6875870 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.473+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/261098299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6852af0 0x55cdc685b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.473+0000 7fc5ed24f740 1 -- 172.21.15.103:0/261098299 >> 172.21.15.103:0/261098299 conn(0x55cdc684f060 msgr2=0x55cdc685cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.474+0000 7fc5ed24f740 1 -- 172.21.15.103:0/261098299 shutdown_connections 2024-09-19T07:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.474+0000 7fc5ed24f740 1 -- 172.21.15.103:0/261098299 wait complete. 2024-09-19T07:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.474+0000 7fc5ed24f740 1 -- start start 2024-09-19T07:00:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.474+0000 7fc5ed24f740 1 Processor -- start 2024-09-19T07:00:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.474+0000 7fc5ed24f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6852af0 0x55cdc6866500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.474+0000 7fc5ed24f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc6853410 0x55cdc68579d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.474+0000 7fc5ed24f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc685dcb0 0x55cdc6858110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.474+0000 7fc5ed24f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e41aed50 con 0x55cdc685dcb0 2024-09-19T07:00:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.474+0000 7fc5ed24f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e41ae680 con 0x55cdc6853410 2024-09-19T07:00:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.474+0000 7fc5ed24f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e41ab8e0 con 0x55cdc6852af0 2024-09-19T07:00:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.474+0000 7fc5ed24f740 10 client.0 _ll_register_callbacks cb 0x7fc5d4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:00:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.474+0000 7fc5e8973640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc685dcb0 0x55cdc6858110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.474+0000 7fc5e8973640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc685dcb0 0x55cdc6858110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48014/0 (socket says 172.21.15.103:48014) 2024-09-19T07:00:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.474+0000 7fc5e8973640 1 -- 172.21.15.103:0/1255987595 learned_addr learned my addr 172.21.15.103:0/1255987595 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:00:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5e37fe640 1 --2- 172.21.15.103:0/1255987595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc6853410 0x55cdc68579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5e3fff640 1 --2- 172.21.15.103:0/1255987595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6852af0 0x55cdc6866500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5e8973640 1 -- 172.21.15.103:0/1255987595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6852af0 msgr2=0x55cdc6866500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5e8973640 1 --2- 172.21.15.103:0/1255987595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6852af0 0x55cdc6866500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5e8973640 1 -- 172.21.15.103:0/1255987595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc6853410 msgr2=0x55cdc68579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:00:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5e8973640 1 --2- 172.21.15.103:0/1255987595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc6853410 0x55cdc68579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5e8973640 1 -- 172.21.15.103:0/1255987595 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc68d7840 con 0x55cdc685dcb0 2024-09-19T07:00:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5e8973640 1 --2- 172.21.15.103:0/1255987595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc685dcb0 0x55cdc6858110 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc014fa0 tx=0x7fc5dc009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3db165fb9f47c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:00:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:00:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5c37fe640 1 -- 172.21.15.103:0/1255987595 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc0127b0 con 0x55cdc685dcb0 2024-09-19T07:00:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5ed24f740 1 -- 172.21.15.103:0/1255987595 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5d80b4e50 con 0x55cdc685dcb0 2024-09-19T07:00:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5c37fe640 1 -- 172.21.15.103:0/1255987595 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fc5dc012950 con 0x55cdc685dcb0 2024-09-19T07:00:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5c37fe640 1 -- 172.21.15.103:0/1255987595 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc5dc012b50 con 0x55cdc685dcb0 2024-09-19T07:00:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5ed24f740 10 client.19440 fetch_fsmap learned FSMap version 27 2024-09-19T07:00:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5ed24f740 1 -- 172.21.15.103:0/1255987595 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cdc68d89f0 con 0x55cdc685dcb0 2024-09-19T07:00:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5c37fe640 1 -- 172.21.15.103:0/1255987595 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d80b4e50 con 0x55cdc685dcb0 2024-09-19T07:00:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.475+0000 7fc5c37fe640 1 -- 172.21.15.103:0/1255987595 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc5dc01d070 con 0x55cdc685dcb0 2024-09-19T07:00:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.476+0000 7fc5ed24f740 10 client.19440 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:00:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.476+0000 7fc5ed24f740 10 client.19440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:00:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.476+0000 7fc5ed24f740 20 client.19440 populate_metadata read hostname 'smithi103' 2024-09-19T07:00:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.476+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/1255987595 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cdc68d91c0 0x55cdc68db5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:00:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.476+0000 7fc5ed24f740 4 client.19440 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:00:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.476+0000 7fc5ed24f740 1 -- 172.21.15.103:0/1255987595 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cdc67bed70 con 0x55cdc68d91c0 2024-09-19T07:00:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.476+0000 7fc5e3fff640 1 --2- 172.21.15.103:0/1255987595 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cdc68d91c0 0x55cdc68db5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:00:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.476+0000 7fc5c37fe640 10 client.19440 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.476+0000 7fc5e3fff640 1 --2- 172.21.15.103:0/1255987595 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cdc68d91c0 0x55cdc68db5e0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=272c65d30af11379 server_cookie=6ee9973717e894e9 in_seq=0 out_seq=0 2024-09-19T07:00:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.489+0000 7fc5c37fe640 1 -- 172.21.15.103:0/1255987595 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cdc67bed70 con 0x55cdc68d91c0 2024-09-19T07:00:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.489+0000 7fc5c37fe640 10 client.19440 handle_command_reply: tid=0 2024-09-19T07:00:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.489+0000 7fc5ed24f740 1 client.19440 shutdown 2024-09-19T07:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.489+0000 7fc5ed24f740 1 -- 172.21.15.103:0/1255987595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc685dcb0 msgr2=0x55cdc6858110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.489+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/1255987595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc685dcb0 0x55cdc6858110 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc014fa0 tx=0x7fc5dc009cd0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.489+0000 7fc5ed24f740 1 -- 172.21.15.103:0/1255987595 shutdown_connections 2024-09-19T07:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.489+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/1255987595 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cdc68d91c0 0x55cdc68db5e0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.489+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/1255987595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cdc685dcb0 0x55cdc6858110 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.489+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/1255987595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cdc6853410 0x55cdc68579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.489+0000 7fc5ed24f740 1 --2- 172.21.15.103:0/1255987595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdc6852af0 0x55cdc6866500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.489+0000 7fc5c37fe640 0 client.19440 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.489+0000 7fc5ed24f740 1 -- 172.21.15.103:0/1255987595 >> 172.21.15.103:0/1255987595 conn(0x55cdc684f060 msgr2=0x55cdc684d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.489+0000 7fc5ed24f740 1 -- 172.21.15.103:0/1255987595 shutdown_connections 2024-09-19T07:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.489+0000 7fc5ed24f740 1 -- 172.21.15.103:0/1255987595 wait complete. 2024-09-19T07:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.489+0000 7fc5ed24f740 20 client.19440 trim_cache size 0 max 16384 2024-09-19T07:00:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:00:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:00:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:00:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:00:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.490+0000 7fc5ea3d6640 1 -- 172.21.15.103:0/67990938 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc5cc0a3460 msgr2=0x7fc5cc0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.490+0000 7fc5ea3d6640 1 --2- 172.21.15.103:0/67990938 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc5cc0a3460 0x7fc5cc0a5920 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc005c40 tx=0x7fc5dc0094a0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.490+0000 7fc5ea3d6640 1 -- 172.21.15.103:0/67990938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5e4101350 msgr2=0x7fc5e41996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:00:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.490+0000 7fc5ea3d6640 1 --2- 172.21.15.103:0/67990938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5e4101350 0x7fc5e41996d0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800d950 tx=0x7fc5d800de10 comp rx=0 tx=0).stop 2024-09-19T07:00:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.490+0000 7fc5ea3d6640 1 -- 172.21.15.103:0/67990938 shutdown_connections 2024-09-19T07:00:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.490+0000 7fc5ea3d6640 1 --2- 172.21.15.103:0/67990938 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc5cc0a3460 0x7fc5cc0a5920 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.490+0000 7fc5ea3d6640 1 --2- 172.21.15.103:0/67990938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e41069c0 0x7fc5e41a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.490+0000 7fc5ea3d6640 1 --2- 172.21.15.103:0/67990938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5e4101350 0x7fc5e41996d0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.490+0000 7fc5ea3d6640 1 --2- 172.21.15.103:0/67990938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5e41009b0 0x7fc5e4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:00:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.490+0000 7fc5ea3d6640 1 -- 172.21.15.103:0/67990938 >> 172.21.15.103:0/67990938 conn(0x7fc5e4077990 msgr2=0x7fc5e41037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:00:48.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.490+0000 7fc5ea3d6640 1 -- 172.21.15.103:0/67990938 shutdown_connections 2024-09-19T07:00:48.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:00:48.490+0000 7fc5ea3d6640 1 -- 172.21.15.103:0/67990938 wait complete. 2024-09-19T07:00:50.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:49 smithi174 ceph-mon[35637]: pgmap v360: 97 pgs: 97 active+clean; 3.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T07:00:50.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:49 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T07:00:50.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:49 smithi103 ceph-mon[31158]: pgmap v360: 97 pgs: 97 active+clean; 3.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T07:00:50.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:49 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T07:00:50.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:49 smithi134 ceph-mon[35356]: pgmap v360: 97 pgs: 97 active+clean; 3.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T07:00:50.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:49 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T07:00:52.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:51 smithi174 ceph-mon[35637]: pgmap v361: 97 pgs: 97 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 65 KiB/s wr, 25 op/s 2024-09-19T07:00:52.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:51 smithi103 ceph-mon[31158]: pgmap v361: 97 pgs: 97 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 65 KiB/s wr, 25 op/s 2024-09-19T07:00:52.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:51 smithi134 ceph-mon[35356]: pgmap v361: 97 pgs: 97 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 65 KiB/s wr, 25 op/s 2024-09-19T07:00:54.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:53 smithi174 ceph-mon[35637]: pgmap v362: 97 pgs: 97 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T07:00:54.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:53 smithi103 ceph-mon[31158]: pgmap v362: 97 pgs: 97 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T07:00:54.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:53 smithi134 ceph-mon[35356]: pgmap v362: 97 pgs: 97 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T07:00:56.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:56 smithi174 ceph-mon[35637]: pgmap v363: 97 pgs: 97 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T07:00:56.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:56 smithi103 ceph-mon[31158]: pgmap v363: 97 pgs: 97 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T07:00:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:56 smithi134 ceph-mon[35356]: pgmap v363: 97 pgs: 97 active+clean; 3.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T07:00:58.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:00:58 smithi174 ceph-mon[35637]: pgmap v364: 97 pgs: 97 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 23 op/s 2024-09-19T07:00:58.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:00:58 smithi103 ceph-mon[31158]: pgmap v364: 97 pgs: 97 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 23 op/s 2024-09-19T07:00:58.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:00:58 smithi134 ceph-mon[35356]: pgmap v364: 97 pgs: 97 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 23 op/s 2024-09-19T07:01:00.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:00 smithi174 ceph-mon[35637]: pgmap v365: 97 pgs: 97 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-09-19T07:01:00.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:00 smithi103 ceph-mon[31158]: pgmap v365: 97 pgs: 97 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-09-19T07:01:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:00 smithi134 ceph-mon[35356]: pgmap v365: 97 pgs: 97 active+clean; 3.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-09-19T07:01:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.584+0000 7f3c811cd640 1 -- 172.21.15.103:0/4053182721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c7c102470 msgr2=0x7f3c7c10e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.584+0000 7f3c811cd640 1 --2- 172.21.15.103:0/4053182721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c7c102470 0x7f3c7c10e9a0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009cd0 tx=0x7f3c7402a8e0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.584+0000 7f3c811cd640 1 -- 172.21.15.103:0/4053182721 shutdown_connections 2024-09-19T07:01:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.584+0000 7f3c811cd640 1 --2- 172.21.15.103:0/4053182721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c7c102470 0x7f3c7c10e9a0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.584+0000 7f3c811cd640 1 --2- 172.21.15.103:0/4053182721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c7c101ad0 0x7f3c7c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.584+0000 7f3c811cd640 1 --2- 172.21.15.103:0/4053182721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c7c107b70 0x7f3c7c107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.584+0000 7f3c811cd640 1 -- 172.21.15.103:0/4053182721 >> 172.21.15.103:0/4053182721 conn(0x7f3c7c0fd7d0 msgr2=0x7f3c7c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.584+0000 7f3c811cd640 1 -- 172.21.15.103:0/4053182721 shutdown_connections 2024-09-19T07:01:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.584+0000 7f3c811cd640 1 -- 172.21.15.103:0/4053182721 wait complete. 2024-09-19T07:01:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.585+0000 7f3c811cd640 1 Processor -- start 2024-09-19T07:01:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.585+0000 7f3c811cd640 1 -- start start 2024-09-19T07:01:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.585+0000 7f3c811cd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c7c101ad0 0x7f3c7c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.585+0000 7f3c811cd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c7c102470 0x7f3c7c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.585+0000 7f3c811cd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c7c107b70 0x7f3c7c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.585+0000 7f3c811cd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c111f20 con 0x7f3c7c107b70 2024-09-19T07:01:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.585+0000 7f3c811cd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c111da0 con 0x7f3c7c101ad0 2024-09-19T07:01:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.585+0000 7f3c811cd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1120a0 con 0x7f3c7c102470 2024-09-19T07:01:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.585+0000 7f3c7b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c7c107b70 0x7f3c7c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.585+0000 7f3c7b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c7c107b70 0x7f3c7c19cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59004/0 (socket says 172.21.15.103:59004) 2024-09-19T07:01:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.585+0000 7f3c7b59e640 1 -- 172.21.15.103:0/3279871562 learned_addr learned my addr 172.21.15.103:0/3279871562 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.586+0000 7f3c7ad9d640 1 --2- 172.21.15.103:0/3279871562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c7c101ad0 0x7f3c7c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.586+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/3279871562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c7c102470 0x7f3c7c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.586+0000 7f3c7b59e640 1 -- 172.21.15.103:0/3279871562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c7c102470 msgr2=0x7f3c7c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.586+0000 7f3c7b59e640 1 --2- 172.21.15.103:0/3279871562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c7c102470 0x7f3c7c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.586+0000 7f3c7b59e640 1 -- 172.21.15.103:0/3279871562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c7c101ad0 msgr2=0x7f3c7c1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.586+0000 7f3c7b59e640 1 --2- 172.21.15.103:0/3279871562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c7c101ad0 0x7f3c7c1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.586+0000 7f3c7b59e640 1 -- 172.21.15.103:0/3279871562 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7c19d2c0 con 0x7f3c7c107b70 2024-09-19T07:01:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.586+0000 7f3c7ad9d640 1 --2- 172.21.15.103:0/3279871562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c7c101ad0 0x7f3c7c1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:01:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.586+0000 7f3c7b59e640 1 --2- 172.21.15.103:0/3279871562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c7c107b70 0x7f3c7c19cb90 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f3c74005cc0 tx=0x7f3c74032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.586+0000 7f3c63fff640 1 -- 172.21.15.103:0/3279871562 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74004040 con 0x7f3c7c107b70 2024-09-19T07:01:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.586+0000 7f3c63fff640 1 -- 172.21.15.103:0/3279871562 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c740046c0 con 0x7f3c7c107b70 2024-09-19T07:01:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.586+0000 7f3c811cd640 1 -- 172.21.15.103:0/3279871562 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7c19d550 con 0x7f3c7c107b70 2024-09-19T07:01:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.586+0000 7f3c63fff640 1 -- 172.21.15.103:0/3279871562 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74030e00 con 0x7f3c7c107b70 2024-09-19T07:01:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.586+0000 7f3c811cd640 1 -- 172.21.15.103:0/3279871562 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c7c1a4e40 con 0x7f3c7c107b70 2024-09-19T07:01:01.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.589+0000 7f3c63fff640 1 -- 172.21.15.103:0/3279871562 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3c740305a0 con 0x7f3c7c107b70 2024-09-19T07:01:01.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.589+0000 7f3c84042740 1 Processor -- start 2024-09-19T07:01:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.589+0000 7f3c84042740 1 -- start start 2024-09-19T07:01:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c84042740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b4242af0 0x55e5b4242ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c84042740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b4243410 0x55e5b424d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c84042740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b424dcb0 0x55e5b42500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c84042740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e5b41a72f0 con 0x55e5b424dcb0 2024-09-19T07:01:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c84042740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e5b414f270 con 0x55e5b4243410 2024-09-19T07:01:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c84042740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e5b41a32b0 con 0x55e5b4242af0 2024-09-19T07:01:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c63fff640 1 --2- 172.21.15.103:0/3279871562 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c640a3460 0x7f3c640a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c7a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b424dcb0 0x55e5b42500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c7a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b424dcb0 0x55e5b42500a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59016/0 (socket says 172.21.15.103:59016) 2024-09-19T07:01:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c7a59c640 1 -- 172.21.15.103:0/3087251629 learned_addr learned my addr 172.21.15.103:0/3087251629 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c7ad9d640 1 --2- 172.21.15.103:0/3087251629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b4243410 0x55e5b424d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c7b59e640 1 --2- 172.21.15.103:0/3279871562 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c640a3460 0x7f3c640a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/3087251629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b4242af0 0x55e5b4242ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c63fff640 1 -- 172.21.15.103:0/3279871562 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f3c7402e030 con 0x7f3c7c107b70 2024-09-19T07:01:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c7a59c640 1 -- 172.21.15.103:0/3087251629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b4242af0 msgr2=0x55e5b4242ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/3087251629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b4242af0 0x55e5b4242ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c7a59c640 1 -- 172.21.15.103:0/3087251629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b4243410 msgr2=0x55e5b424d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/3087251629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b4243410 0x55e5b424d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c7a59c640 1 -- 172.21.15.103:0/3087251629 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5b414e340 con 0x55e5b424dcb0 2024-09-19T07:01:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/3087251629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b424dcb0 0x55e5b42500a0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009f90 tx=0x7f3c6c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf5f8ebdd7d76e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c7b59e640 1 --2- 172.21.15.103:0/3279871562 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c640a3460 0x7f3c640a5920 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3c740bd470 tx=0x7f3c740a8040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.590+0000 7f3c617fa640 1 -- 172.21.15.103:0/3087251629 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c004050 con 0x55e5b424dcb0 2024-09-19T07:01:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.591+0000 7f3c617fa640 1 -- 172.21.15.103:0/3087251629 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c6c005500 con 0x55e5b424dcb0 2024-09-19T07:01:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.591+0000 7f3c617fa640 1 -- 172.21.15.103:0/3087251629 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c004cd0 con 0x55e5b424dcb0 2024-09-19T07:01:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.591+0000 7f3c84042740 1 -- 172.21.15.103:0/3087251629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b424dcb0 msgr2=0x55e5b42500a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.591+0000 7f3c84042740 1 --2- 172.21.15.103:0/3087251629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b424dcb0 0x55e5b42500a0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009f90 tx=0x7f3c6c005850 comp rx=0 tx=0).stop 2024-09-19T07:01:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.591+0000 7f3c84042740 1 -- 172.21.15.103:0/3087251629 shutdown_connections 2024-09-19T07:01:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.591+0000 7f3c84042740 1 --2- 172.21.15.103:0/3087251629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b424dcb0 0x55e5b42500a0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.591+0000 7f3c84042740 1 --2- 172.21.15.103:0/3087251629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b4243410 0x55e5b424d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.591+0000 7f3c84042740 1 --2- 172.21.15.103:0/3087251629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b4242af0 0x55e5b4242ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.591+0000 7f3c84042740 1 -- 172.21.15.103:0/3087251629 >> 172.21.15.103:0/3087251629 conn(0x55e5b423f060 msgr2=0x55e5b423f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.591+0000 7f3c84042740 1 -- 172.21.15.103:0/3087251629 shutdown_connections 2024-09-19T07:01:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.591+0000 7f3c84042740 1 -- 172.21.15.103:0/3087251629 wait complete. 2024-09-19T07:01:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c84042740 1 -- start start 2024-09-19T07:01:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c84042740 1 Processor -- start 2024-09-19T07:01:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c84042740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b4242af0 0x55e5b424abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c84042740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b4243410 0x55e5b424b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c84042740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b424dcb0 0x55e5b424b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c84042740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e5b4250880 con 0x55e5b4242af0 2024-09-19T07:01:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c84042740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e5b4130290 con 0x55e5b424dcb0 2024-09-19T07:01:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c84042740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e5b4250a00 con 0x55e5b4243410 2024-09-19T07:01:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c84042740 10 client.0 _ll_register_callbacks cb 0x55e5b4161140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c7a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b4242af0 0x55e5b424abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c7a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b4242af0 0x55e5b424abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59032/0 (socket says 172.21.15.103:59032) 2024-09-19T07:01:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c7a59c640 1 -- 172.21.15.103:0/2435962073 learned_addr learned my addr 172.21.15.103:0/2435962073 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c7ad9d640 1 --2- 172.21.15.103:0/2435962073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b4243410 0x55e5b424b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c7a59c640 1 -- 172.21.15.103:0/2435962073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b4243410 msgr2=0x55e5b424b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/2435962073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b4243410 0x55e5b424b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c7a59c640 1 -- 172.21.15.103:0/2435962073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b424dcb0 msgr2=0x55e5b424b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/2435962073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b424dcb0 0x55e5b424b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c7a59c640 1 -- 172.21.15.103:0/2435962073 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5b3eba210 con 0x55e5b4242af0 2024-09-19T07:01:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.592+0000 7f3c7ad9d640 1 --2- 172.21.15.103:0/2435962073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b4243410 0x55e5b424b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.593+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/2435962073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b4242af0 0x55e5b424abe0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c013000 tx=0x7f3c6c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8949ee1e8476472 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.593+0000 7f3c5eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.593+0000 7f3c5eff5640 1 -- 172.21.15.103:0/2435962073 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0046c0 con 0x55e5b4242af0 2024-09-19T07:01:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.593+0000 7f3c5eff5640 1 -- 172.21.15.103:0/2435962073 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f3c6c02a470 con 0x55e5b4242af0 2024-09-19T07:01:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.593+0000 7f3c84042740 1 -- 172.21.15.103:0/2435962073 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c6c005500 con 0x55e5b4242af0 2024-09-19T07:01:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.593+0000 7f3c5eff5640 1 -- 172.21.15.103:0/2435962073 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c6c004e60 con 0x55e5b4242af0 2024-09-19T07:01:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.593+0000 7f3c84042740 10 client.19452 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.593+0000 7f3c84042740 1 -- 172.21.15.103:0/2435962073 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e5b3f86620 con 0x55e5b4242af0 2024-09-19T07:01:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.593+0000 7f3c5eff5640 1 -- 172.21.15.103:0/2435962073 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c6c005500 con 0x55e5b4242af0 2024-09-19T07:01:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.593+0000 7f3c5eff5640 1 -- 172.21.15.103:0/2435962073 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3c6c01e6c0 con 0x55e5b4242af0 2024-09-19T07:01:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.593+0000 7f3c84042740 10 client.19452 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.593+0000 7f3c84042740 10 client.19452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.593+0000 7f3c84042740 20 client.19452 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.593+0000 7f3c84042740 1 --2- 172.21.15.103:0/2435962073 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e5b425dc20 0x55e5b4260010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.593+0000 7f3c84042740 4 client.19452 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.593+0000 7f3c84042740 1 -- 172.21.15.103:0/2435962073 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5b41b4a20 con 0x55e5b425dc20 2024-09-19T07:01:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.594+0000 7f3c7ad9d640 1 --2- 172.21.15.103:0/2435962073 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e5b425dc20 0x55e5b4260010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.594+0000 7f3c7ad9d640 1 --2- 172.21.15.103:0/2435962073 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e5b425dc20 0x55e5b4260010 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59282fd1d617aac1 server_cookie=f384aaddc7ccca19 in_seq=0 out_seq=0 2024-09-19T07:01:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.594+0000 7f3c5eff5640 10 client.19452 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.595+0000 7f3c5eff5640 1 -- 172.21.15.103:0/2435962073 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5b41b4a20 con 0x55e5b425dc20 2024-09-19T07:01:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.595+0000 7f3c5eff5640 10 client.19452 handle_command_reply: tid=0 2024-09-19T07:01:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.595+0000 7f3c84042740 1 client.19452 shutdown 2024-09-19T07:01:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.595+0000 7f3c84042740 1 -- 172.21.15.103:0/2435962073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b4242af0 msgr2=0x55e5b424abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.595+0000 7f3c84042740 1 --2- 172.21.15.103:0/2435962073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b4242af0 0x55e5b424abe0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c013000 tx=0x7f3c6c0096c0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.595+0000 7f3c84042740 1 -- 172.21.15.103:0/2435962073 shutdown_connections 2024-09-19T07:01:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.595+0000 7f3c84042740 1 --2- 172.21.15.103:0/2435962073 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e5b425dc20 0x55e5b4260010 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.595+0000 7f3c84042740 1 --2- 172.21.15.103:0/2435962073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b424dcb0 0x55e5b424b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.595+0000 7f3c84042740 1 --2- 172.21.15.103:0/2435962073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b4243410 0x55e5b424b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.595+0000 7f3c84042740 1 --2- 172.21.15.103:0/2435962073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b4242af0 0x55e5b424abe0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.595+0000 7f3c5eff5640 0 client.19452 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.595+0000 7f3c84042740 1 -- 172.21.15.103:0/2435962073 >> 172.21.15.103:0/2435962073 conn(0x55e5b423f060 msgr2=0x55e5b4244d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.595+0000 7f3c84042740 1 -- 172.21.15.103:0/2435962073 shutdown_connections 2024-09-19T07:01:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.595+0000 7f3c84042740 1 -- 172.21.15.103:0/2435962073 wait complete. 2024-09-19T07:01:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.595+0000 7f3c84042740 20 client.19452 trim_cache size 0 max 16384 2024-09-19T07:01:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.609+0000 7f3c84042740 1 Processor -- start 2024-09-19T07:01:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.609+0000 7f3c84042740 1 -- start start 2024-09-19T07:01:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.609+0000 7f3c84042740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b4242af0 0x55e5b424b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.609+0000 7f3c84042740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b4243410 0x55e5b42655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c84042740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b424dcb0 0x55e5b4265b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c84042740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1ad730 con 0x55e5b424dcb0 2024-09-19T07:01:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c84042740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1af270 con 0x55e5b4243410 2024-09-19T07:01:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c84042740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1ade00 con 0x55e5b4242af0 2024-09-19T07:01:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c7a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b424dcb0 0x55e5b4265b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c7a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b424dcb0 0x55e5b4265b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59034/0 (socket says 172.21.15.103:59034) 2024-09-19T07:01:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c7a59c640 1 -- 172.21.15.103:0/1495067794 learned_addr learned my addr 172.21.15.103:0/1495067794 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/1495067794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b4242af0 0x55e5b424b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c7ad9d640 1 --2- 172.21.15.103:0/1495067794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b4243410 0x55e5b42655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c7a59c640 1 -- 172.21.15.103:0/1495067794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b4242af0 msgr2=0x55e5b424b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/1495067794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b4242af0 0x55e5b424b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c7a59c640 1 -- 172.21.15.103:0/1495067794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b4243410 msgr2=0x55e5b42655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/1495067794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b4243410 0x55e5b42655e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c7a59c640 1 -- 172.21.15.103:0/1495067794 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5b4243240 con 0x55e5b424dcb0 2024-09-19T07:01:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/1495067794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b424dcb0 0x55e5b4265b20 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009f90 tx=0x7f3c6c005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85481a736c631425 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c60ff9640 1 -- 172.21.15.103:0/1495067794 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c036040 con 0x55e5b424dcb0 2024-09-19T07:01:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c60ff9640 1 -- 172.21.15.103:0/1495067794 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c6c025070 con 0x55e5b424dcb0 2024-09-19T07:01:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.610+0000 7f3c60ff9640 1 -- 172.21.15.103:0/1495067794 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c03f910 con 0x55e5b424dcb0 2024-09-19T07:01:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.611+0000 7f3c84042740 1 -- 172.21.15.103:0/1495067794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b424dcb0 msgr2=0x55e5b4265b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.611+0000 7f3c84042740 1 --2- 172.21.15.103:0/1495067794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b424dcb0 0x55e5b4265b20 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009f90 tx=0x7f3c6c005b90 comp rx=0 tx=0).stop 2024-09-19T07:01:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.611+0000 7f3c84042740 1 -- 172.21.15.103:0/1495067794 shutdown_connections 2024-09-19T07:01:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.611+0000 7f3c84042740 1 --2- 172.21.15.103:0/1495067794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b424dcb0 0x55e5b4265b20 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.611+0000 7f3c84042740 1 --2- 172.21.15.103:0/1495067794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b4243410 0x55e5b42655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.611+0000 7f3c84042740 1 --2- 172.21.15.103:0/1495067794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b4242af0 0x55e5b424b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.611+0000 7f3c84042740 1 -- 172.21.15.103:0/1495067794 >> 172.21.15.103:0/1495067794 conn(0x55e5b423f060 msgr2=0x55e5b424cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.611+0000 7f3c84042740 1 -- 172.21.15.103:0/1495067794 shutdown_connections 2024-09-19T07:01:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.611+0000 7f3c84042740 1 -- 172.21.15.103:0/1495067794 wait complete. 2024-09-19T07:01:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.611+0000 7f3c84042740 1 -- start start 2024-09-19T07:01:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.611+0000 7f3c84042740 1 Processor -- start 2024-09-19T07:01:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.611+0000 7f3c84042740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b4242af0 0x55e5b4256280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c84042740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b4243410 0x55e5b4247760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c84042740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b424dcb0 0x55e5b4247ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c84042740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1aeba0 con 0x55e5b4243410 2024-09-19T07:01:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c84042740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1ae4d0 con 0x55e5b4242af0 2024-09-19T07:01:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c84042740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1ab730 con 0x55e5b424dcb0 2024-09-19T07:01:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c84042740 10 client.0 _ll_register_callbacks cb 0x7f3c7000ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c7ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b4243410 0x55e5b4247760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c7ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b4243410 0x55e5b4247760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59042/0 (socket says 172.21.15.103:59042) 2024-09-19T07:01:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c7ad9d640 1 -- 172.21.15.103:0/1310637317 learned_addr learned my addr 172.21.15.103:0/1310637317 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/1310637317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b424dcb0 0x55e5b4247ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/1310637317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b4242af0 0x55e5b4256280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c7ad9d640 1 -- 172.21.15.103:0/1310637317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b424dcb0 msgr2=0x55e5b4247ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c7ad9d640 1 --2- 172.21.15.103:0/1310637317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b424dcb0 0x55e5b4247ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c7ad9d640 1 -- 172.21.15.103:0/1310637317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b4242af0 msgr2=0x55e5b4256280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c7ad9d640 1 --2- 172.21.15.103:0/1310637317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b4242af0 0x55e5b4256280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c7ad9d640 1 -- 172.21.15.103:0/1310637317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5b4248310 con 0x55e5b4243410 2024-09-19T07:01:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/1310637317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b4242af0 0x55e5b4256280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c7ad9d640 1 --2- 172.21.15.103:0/1310637317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b4243410 0x55e5b4247760 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f3c70004190 tx=0x7f3c700039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29d765655158ef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c5eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c5eff5640 1 -- 172.21.15.103:0/1310637317 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7001d670 con 0x55e5b4243410 2024-09-19T07:01:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c5eff5640 1 -- 172.21.15.103:0/1310637317 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f3c7001d810 con 0x55e5b4243410 2024-09-19T07:01:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c84042740 1 -- 172.21.15.103:0/1310637317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c6c025070 con 0x55e5b4243410 2024-09-19T07:01:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c5eff5640 1 -- 172.21.15.103:0/1310637317 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c7002d050 con 0x55e5b4243410 2024-09-19T07:01:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c84042740 10 client.19458 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.612+0000 7f3c84042740 1 -- 172.21.15.103:0/1310637317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e5b42c84c0 con 0x55e5b4243410 2024-09-19T07:01:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.613+0000 7f3c5eff5640 1 -- 172.21.15.103:0/1310637317 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c6c025070 con 0x55e5b4243410 2024-09-19T07:01:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.613+0000 7f3c5eff5640 1 -- 172.21.15.103:0/1310637317 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3c70010430 con 0x55e5b4243410 2024-09-19T07:01:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.613+0000 7f3c84042740 10 client.19458 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.613+0000 7f3c84042740 10 client.19458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.613+0000 7f3c84042740 20 client.19458 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.613+0000 7f3c84042740 1 --2- 172.21.15.103:0/1310637317 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e5b42c8b90 0x55e5b42cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.613+0000 7f3c84042740 4 client.19458 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:01:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.613+0000 7f3c84042740 1 -- 172.21.15.103:0/1310637317 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e5b4245a00 con 0x55e5b42c8b90 2024-09-19T07:01:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.613+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/1310637317 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e5b42c8b90 0x55e5b42cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.614+0000 7f3c7a59c640 1 --2- 172.21.15.103:0/1310637317 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e5b42c8b90 0x55e5b42cafb0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6d6ebebf4dd52ab server_cookie=3ee725857704ece9 in_seq=0 out_seq=0 2024-09-19T07:01:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.614+0000 7f3c5eff5640 10 client.19458 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.628+0000 7f3c5eff5640 1 -- 172.21.15.103:0/1310637317 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e5b4245a00 con 0x55e5b42c8b90 2024-09-19T07:01:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.628+0000 7f3c5eff5640 10 client.19458 handle_command_reply: tid=0 2024-09-19T07:01:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.628+0000 7f3c84042740 1 client.19458 shutdown 2024-09-19T07:01:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.628+0000 7f3c84042740 1 -- 172.21.15.103:0/1310637317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b4243410 msgr2=0x55e5b4247760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.628+0000 7f3c84042740 1 --2- 172.21.15.103:0/1310637317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b4243410 0x55e5b4247760 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f3c70004190 tx=0x7f3c700039c0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.628+0000 7f3c84042740 1 -- 172.21.15.103:0/1310637317 shutdown_connections 2024-09-19T07:01:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.628+0000 7f3c84042740 1 --2- 172.21.15.103:0/1310637317 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e5b42c8b90 0x55e5b42cafb0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.628+0000 7f3c84042740 1 --2- 172.21.15.103:0/1310637317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5b424dcb0 0x55e5b4247ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.628+0000 7f3c84042740 1 --2- 172.21.15.103:0/1310637317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5b4243410 0x55e5b4247760 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.628+0000 7f3c5eff5640 0 client.19458 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.628+0000 7f3c84042740 1 --2- 172.21.15.103:0/1310637317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5b4242af0 0x55e5b4256280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.628+0000 7f3c84042740 1 -- 172.21.15.103:0/1310637317 >> 172.21.15.103:0/1310637317 conn(0x55e5b423f060 msgr2=0x55e5b4244550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.628+0000 7f3c84042740 1 -- 172.21.15.103:0/1310637317 shutdown_connections 2024-09-19T07:01:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.628+0000 7f3c84042740 1 -- 172.21.15.103:0/1310637317 wait complete. 2024-09-19T07:01:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.628+0000 7f3c84042740 20 client.19458 trim_cache size 0 max 16384 2024-09-19T07:01:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:01:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:01:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:01:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.629+0000 7f3c811cd640 1 -- 172.21.15.103:0/3279871562 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c640a3460 msgr2=0x7f3c640a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.629+0000 7f3c811cd640 1 --2- 172.21.15.103:0/3279871562 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c640a3460 0x7f3c640a5920 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3c740bd470 tx=0x7f3c740a8040 comp rx=0 tx=0).stop 2024-09-19T07:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.629+0000 7f3c811cd640 1 -- 172.21.15.103:0/3279871562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c7c107b70 msgr2=0x7f3c7c19cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.629+0000 7f3c811cd640 1 --2- 172.21.15.103:0/3279871562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c7c107b70 0x7f3c7c19cb90 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f3c74005cc0 tx=0x7f3c74032f40 comp rx=0 tx=0).stop 2024-09-19T07:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.629+0000 7f3c811cd640 1 -- 172.21.15.103:0/3279871562 shutdown_connections 2024-09-19T07:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.629+0000 7f3c811cd640 1 --2- 172.21.15.103:0/3279871562 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c640a3460 0x7f3c640a5920 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.629+0000 7f3c811cd640 1 --2- 172.21.15.103:0/3279871562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c7c107b70 0x7f3c7c19cb90 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.629+0000 7f3c811cd640 1 --2- 172.21.15.103:0/3279871562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c7c102470 0x7f3c7c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.629+0000 7f3c811cd640 1 --2- 172.21.15.103:0/3279871562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c7c101ad0 0x7f3c7c1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.629+0000 7f3c811cd640 1 -- 172.21.15.103:0/3279871562 >> 172.21.15.103:0/3279871562 conn(0x7f3c7c0fd7d0 msgr2=0x7f3c7c0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.629+0000 7f3c811cd640 1 -- 172.21.15.103:0/3279871562 shutdown_connections 2024-09-19T07:01:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:01.629+0000 7f3c811cd640 1 -- 172.21.15.103:0/3279871562 wait complete. 2024-09-19T07:01:02.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:02 smithi174 ceph-mon[35637]: pgmap v366: 97 pgs: 97 active+clean; 4.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T07:01:02.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:02 smithi103 ceph-mon[31158]: pgmap v366: 97 pgs: 97 active+clean; 4.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T07:01:02.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:02 smithi134 ceph-mon[35356]: pgmap v366: 97 pgs: 97 active+clean; 4.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T07:01:04.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:04 smithi174 ceph-mon[35637]: pgmap v367: 97 pgs: 97 active+clean; 4.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-19T07:01:04.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:04 smithi103 ceph-mon[31158]: pgmap v367: 97 pgs: 97 active+clean; 4.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-19T07:01:04.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:04 smithi134 ceph-mon[35356]: pgmap v367: 97 pgs: 97 active+clean; 4.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-19T07:01:06.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:06 smithi174 ceph-mon[35637]: pgmap v368: 97 pgs: 97 active+clean; 4.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-19T07:01:06.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:06 smithi174 ceph-mon[35637]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T07:01:06.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:06 smithi103 ceph-mon[31158]: pgmap v368: 97 pgs: 97 active+clean; 4.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-19T07:01:06.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:06 smithi103 ceph-mon[31158]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T07:01:06.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:06 smithi134 ceph-mon[35356]: pgmap v368: 97 pgs: 97 active+clean; 4.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-19T07:01:06.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:06 smithi134 ceph-mon[35356]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T07:01:08.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:08 smithi174 ceph-mon[35637]: pgmap v370: 97 pgs: 97 active+clean; 4.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-19T07:01:08.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:08 smithi103 ceph-mon[31158]: pgmap v370: 97 pgs: 97 active+clean; 4.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-19T07:01:08.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:08 smithi134 ceph-mon[35356]: pgmap v370: 97 pgs: 97 active+clean; 4.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-19T07:01:10.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:10 smithi174 ceph-mon[35637]: pgmap v371: 97 pgs: 97 active+clean; 4.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-19T07:01:10.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:10 smithi103 ceph-mon[31158]: pgmap v371: 97 pgs: 97 active+clean; 4.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-19T07:01:10.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:10 smithi134 ceph-mon[35356]: pgmap v371: 97 pgs: 97 active+clean; 4.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-09-19T07:01:12.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:12 smithi174 ceph-mon[35637]: pgmap v372: 97 pgs: 97 active+clean; 4.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-09-19T07:01:12.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:12 smithi174 ceph-mon[35637]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T07:01:12.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:12 smithi103 ceph-mon[31158]: pgmap v372: 97 pgs: 97 active+clean; 4.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-09-19T07:01:12.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:12 smithi103 ceph-mon[31158]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T07:01:12.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:12 smithi134 ceph-mon[35356]: pgmap v372: 97 pgs: 97 active+clean; 4.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-09-19T07:01:12.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:12 smithi134 ceph-mon[35356]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T07:01:14.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:14 smithi174 ceph-mon[35637]: pgmap v374: 97 pgs: 97 active+clean; 4.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 135 KiB/s wr, 49 op/s 2024-09-19T07:01:14.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:14 smithi103 ceph-mon[31158]: pgmap v374: 97 pgs: 97 active+clean; 4.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 135 KiB/s wr, 49 op/s 2024-09-19T07:01:14.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:14 smithi134 ceph-mon[35356]: pgmap v374: 97 pgs: 97 active+clean; 4.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 135 KiB/s wr, 49 op/s 2024-09-19T07:01:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.728+0000 7f9eb0f3b640 1 -- 172.21.15.103:0/2284260624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9eac100850 msgr2=0x7f9eac100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.728+0000 7f9eb0f3b640 1 --2- 172.21.15.103:0/2284260624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9eac100850 0x7f9eac100c30 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c009cd0 tx=0x7f9e9c02a8e0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.728+0000 7f9eb0f3b640 1 -- 172.21.15.103:0/2284260624 shutdown_connections 2024-09-19T07:01:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.728+0000 7f9eb0f3b640 1 --2- 172.21.15.103:0/2284260624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eac108110 0x7f9eac10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.728+0000 7f9eb0f3b640 1 --2- 172.21.15.103:0/2284260624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac101200 0x7f9eac1079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.728+0000 7f9eb0f3b640 1 --2- 172.21.15.103:0/2284260624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9eac100850 0x7f9eac100c30 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.728+0000 7f9eb0f3b640 1 -- 172.21.15.103:0/2284260624 >> 172.21.15.103:0/2284260624 conn(0x7f9eac075db0 msgr2=0x7f9eac0761c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.728+0000 7f9eb0f3b640 1 -- 172.21.15.103:0/2284260624 shutdown_connections 2024-09-19T07:01:14.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.728+0000 7f9eb0f3b640 1 -- 172.21.15.103:0/2284260624 wait complete. 2024-09-19T07:01:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eb0f3b640 1 Processor -- start 2024-09-19T07:01:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eb0f3b640 1 -- start start 2024-09-19T07:01:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eb0f3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9eac100850 0x7f9eac1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eb0f3b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac108110 0x7f9eac1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eb0f3b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eac19cc60 0x7f9eac19d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eb0f3b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac111ed0 con 0x7f9eac100850 2024-09-19T07:01:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eb0f3b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac111d50 con 0x7f9eac19cc60 2024-09-19T07:01:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eb0f3b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac112050 con 0x7f9eac108110 2024-09-19T07:01:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eaa59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9eac100850 0x7f9eac1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eaa59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9eac100850 0x7f9eac1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42712/0 (socket says 172.21.15.103:42712) 2024-09-19T07:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eaa59c640 1 -- 172.21.15.103:0/946756440 learned_addr learned my addr 172.21.15.103:0/946756440 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9ea9d9b640 1 --2- 172.21.15.103:0/946756440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac108110 0x7f9eac1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eaad9d640 1 --2- 172.21.15.103:0/946756440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eac19cc60 0x7f9eac19d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eaa59c640 1 -- 172.21.15.103:0/946756440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac108110 msgr2=0x7f9eac1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eaa59c640 1 --2- 172.21.15.103:0/946756440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac108110 0x7f9eac1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eaa59c640 1 -- 172.21.15.103:0/946756440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eac19cc60 msgr2=0x7f9eac19d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eaa59c640 1 --2- 172.21.15.103:0/946756440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eac19cc60 0x7f9eac19d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eaa59c640 1 -- 172.21.15.103:0/946756440 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac19d750 con 0x7f9eac100850 2024-09-19T07:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.729+0000 7f9eaad9d640 1 --2- 172.21.15.103:0/946756440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eac19cc60 0x7f9eac19d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.730+0000 7f9eaa59c640 1 --2- 172.21.15.103:0/946756440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9eac100850 0x7f9eac1983a0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c02ae00 tx=0x7f9e9c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.730+0000 7f9e9b7fe640 1 -- 172.21.15.103:0/946756440 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c004040 con 0x7f9eac100850 2024-09-19T07:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.730+0000 7f9eb0f3b640 1 -- 172.21.15.103:0/946756440 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac19d9e0 con 0x7f9eac100850 2024-09-19T07:01:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.730+0000 7f9e9b7fe640 1 -- 172.21.15.103:0/946756440 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9c0046c0 con 0x7f9eac100850 2024-09-19T07:01:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.730+0000 7f9e9b7fe640 1 -- 172.21.15.103:0/946756440 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c046480 con 0x7f9eac100850 2024-09-19T07:01:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.730+0000 7f9eb0f3b640 1 -- 172.21.15.103:0/946756440 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac1a5180 con 0x7f9eac100850 2024-09-19T07:01:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.732+0000 7f9e9b7fe640 1 -- 172.21.15.103:0/946756440 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9e9c0305a0 con 0x7f9eac100850 2024-09-19T07:01:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.732+0000 7f9eb3cd7740 1 Processor -- start 2024-09-19T07:01:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.733+0000 7f9eb3cd7740 1 -- start start 2024-09-19T07:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.733+0000 7f9eb3cd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea8fedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.733+0000 7f9eb3cd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea90db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.733+0000 7f9eb3cd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea90e0c0 0x55c0ea9104b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.733+0000 7f9eb3cd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c0ea8632f0 con 0x55c0ea8fe9d0 2024-09-19T07:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.733+0000 7f9eb3cd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c0ea80b270 con 0x55c0ea90e0c0 2024-09-19T07:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.733+0000 7f9eb3cd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c0ea85f2b0 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.733+0000 7f9eaad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea8fedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.733+0000 7f9eaad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea8fedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42726/0 (socket says 172.21.15.103:42726) 2024-09-19T07:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.733+0000 7f9eaad9d640 1 -- 172.21.15.103:0/159098217 learned_addr learned my addr 172.21.15.103:0/159098217 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.733+0000 7f9e9b7fe640 1 --2- 172.21.15.103:0/946756440 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9e940a3460 0x7f9e940a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.733+0000 7f9ea9d9b640 1 --2- 172.21.15.103:0/159098217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea90e0c0 0x55c0ea9104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.733+0000 7f9eaad9d640 1 --2- 172.21.15.103:0/946756440 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9e940a3460 0x7f9e940a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.733+0000 7f9eaa59c640 1 --2- 172.21.15.103:0/159098217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea90db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.734+0000 7f9eaad9d640 1 -- 172.21.15.103:0/159098217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea8ff2f0 msgr2=0x55c0ea90db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.734+0000 7f9e9b7fe640 1 -- 172.21.15.103:0/946756440 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f9e9c02e030 con 0x7f9eac100850 2024-09-19T07:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.734+0000 7f9eaad9d640 1 --2- 172.21.15.103:0/159098217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea90db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.734+0000 7f9eaad9d640 1 -- 172.21.15.103:0/159098217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea90e0c0 msgr2=0x55c0ea9104b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.734+0000 7f9eaad9d640 1 --2- 172.21.15.103:0/159098217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea90e0c0 0x55c0ea9104b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.734+0000 7f9eaad9d640 1 -- 172.21.15.103:0/159098217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0ea80a340 con 0x55c0ea8fe9d0 2024-09-19T07:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.734+0000 7f9eaad9d640 1 --2- 172.21.15.103:0/159098217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea8fedb0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40146d0 tx=0x7f9ea400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7142097987f1a60a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.734+0000 7f9e98ff9640 1 -- 172.21.15.103:0/159098217 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4021070 con 0x55c0ea8fe9d0 2024-09-19T07:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.734+0000 7f9e98ff9640 1 -- 172.21.15.103:0/159098217 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea400bb90 con 0x55c0ea8fe9d0 2024-09-19T07:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.734+0000 7f9e98ff9640 1 -- 172.21.15.103:0/159098217 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea400f630 con 0x55c0ea8fe9d0 2024-09-19T07:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.734+0000 7f9eaad9d640 1 --2- 172.21.15.103:0/946756440 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9e940a3460 0x7f9e940a5920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9ea401e000 tx=0x7f9ea401b000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.735+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/159098217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8fe9d0 msgr2=0x55c0ea8fedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.735+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/159098217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea8fedb0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40146d0 tx=0x7f9ea400bf90 comp rx=0 tx=0).stop 2024-09-19T07:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.735+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/159098217 shutdown_connections 2024-09-19T07:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.735+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/159098217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea90e0c0 0x55c0ea9104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.735+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/159098217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea90db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.735+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/159098217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea8fedb0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.735+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/159098217 >> 172.21.15.103:0/159098217 conn(0x55c0ea8fb060 msgr2=0x55c0ea8fb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.735+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/159098217 shutdown_connections 2024-09-19T07:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.735+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/159098217 wait complete. 2024-09-19T07:01:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.735+0000 7f9eb3cd7740 1 -- start start 2024-09-19T07:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.735+0000 7f9eb3cd7740 1 Processor -- start 2024-09-19T07:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.735+0000 7f9eb3cd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea90b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9eb3cd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea90b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9eb3cd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea90e0c0 0x55c0ea90bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9eb3cd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c0ea910cf0 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9eb3cd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c0ea7ec290 con 0x55c0ea90e0c0 2024-09-19T07:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9eb3cd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c0ea910e70 con 0x55c0ea8fe9d0 2024-09-19T07:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9eb3cd7740 10 client.0 _ll_register_callbacks cb 0x55c0ea81d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9ea9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea90b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9ea9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea90b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42736/0 (socket says 172.21.15.103:42736) 2024-09-19T07:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9ea9d9b640 1 -- 172.21.15.103:0/3184197148 learned_addr learned my addr 172.21.15.103:0/3184197148 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9eaad9d640 1 --2- 172.21.15.103:0/3184197148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea90e0c0 0x55c0ea90bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9eaa59c640 1 --2- 172.21.15.103:0/3184197148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea90b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9ea9d9b640 1 -- 172.21.15.103:0/3184197148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea8fe9d0 msgr2=0x55c0ea90b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9ea9d9b640 1 --2- 172.21.15.103:0/3184197148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea90b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9ea9d9b640 1 -- 172.21.15.103:0/3184197148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea90e0c0 msgr2=0x55c0ea90bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9ea9d9b640 1 --2- 172.21.15.103:0/3184197148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea90e0c0 0x55c0ea90bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9ea9d9b640 1 -- 172.21.15.103:0/3184197148 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0ea576210 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9e91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9ea9d9b640 1 --2- 172.21.15.103:0/3184197148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea90b580 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000bf80 tx=0x7f9ea00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82877f3e914a1500 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9e91ffb640 1 -- 172.21.15.103:0/3184197148 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0004020 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9e91ffb640 1 -- 172.21.15.103:0/3184197148 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f9ea00041c0 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/3184197148 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea400bb90 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9e91ffb640 1 -- 172.21.15.103:0/3184197148 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9ea00043c0 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9eb3cd7740 10 client.19470 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.736+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/3184197148 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c0ea642620 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.737+0000 7f9e91ffb640 1 -- 172.21.15.103:0/3184197148 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea400bb90 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.737+0000 7f9e91ffb640 1 -- 172.21.15.103:0/3184197148 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9ea0003a60 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.737+0000 7f9eb3cd7740 10 client.19470 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.737+0000 7f9eb3cd7740 10 client.19470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.737+0000 7f9eb3cd7740 20 client.19470 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.737+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/3184197148 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c0ea91e1c0 0x55c0ea9205b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.737+0000 7f9eb3cd7740 4 client.19470 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.737+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/3184197148 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0ea870a20 con 0x55c0ea91e1c0 2024-09-19T07:01:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.737+0000 7f9eaa59c640 1 --2- 172.21.15.103:0/3184197148 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c0ea91e1c0 0x55c0ea9205b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.738+0000 7f9eaa59c640 1 --2- 172.21.15.103:0/3184197148 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c0ea91e1c0 0x55c0ea9205b0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745e0d0ac3de25ca server_cookie=91d24c8bc3f82597 in_seq=0 out_seq=0 2024-09-19T07:01:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.738+0000 7f9e91ffb640 10 client.19470 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.738+0000 7f9e91ffb640 1 -- 172.21.15.103:0/3184197148 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0ea870a20 con 0x55c0ea91e1c0 2024-09-19T07:01:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.738+0000 7f9e91ffb640 10 client.19470 handle_command_reply: tid=0 2024-09-19T07:01:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.738+0000 7f9eb3cd7740 1 client.19470 shutdown 2024-09-19T07:01:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.738+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/3184197148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 msgr2=0x55c0ea90b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.738+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/3184197148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea90b580 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000bf80 tx=0x7f9ea00118d0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.738+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/3184197148 shutdown_connections 2024-09-19T07:01:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.738+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/3184197148 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c0ea91e1c0 0x55c0ea9205b0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.738+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/3184197148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea90e0c0 0x55c0ea90bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.738+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/3184197148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea90b580 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.738+0000 7f9e91ffb640 0 client.19470 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.738+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/3184197148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea90b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.738+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/3184197148 >> 172.21.15.103:0/3184197148 conn(0x55c0ea8fb060 msgr2=0x55c0ea900dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.738+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/3184197148 shutdown_connections 2024-09-19T07:01:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.738+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/3184197148 wait complete. 2024-09-19T07:01:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.739+0000 7f9eb3cd7740 20 client.19470 trim_cache size 0 max 16384 2024-09-19T07:01:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9eb3cd7740 1 Processor -- start 2024-09-19T07:01:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9eb3cd7740 1 -- start start 2024-09-19T07:01:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9eb3cd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea90b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9eb3cd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea925e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9eb3cd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea90e0c0 0x55c0ea9263b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9eb3cd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1ab5c0 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9eb3cd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1ad100 con 0x55c0ea8fe9d0 2024-09-19T07:01:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9eb3cd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1abc90 con 0x55c0ea90e0c0 2024-09-19T07:01:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9ea9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea925e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9ea9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea925e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42748/0 (socket says 172.21.15.103:42748) 2024-09-19T07:01:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9ea9d9b640 1 -- 172.21.15.103:0/3608266320 learned_addr learned my addr 172.21.15.103:0/3608266320 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9eaa59c640 1 --2- 172.21.15.103:0/3608266320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea90b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9ea9d9b640 1 -- 172.21.15.103:0/3608266320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea90e0c0 msgr2=0x55c0ea9263b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9eaad9d640 1 --2- 172.21.15.103:0/3608266320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea90e0c0 0x55c0ea9263b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9ea9d9b640 1 --2- 172.21.15.103:0/3608266320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea90e0c0 0x55c0ea9263b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9ea9d9b640 1 -- 172.21.15.103:0/3608266320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea8fe9d0 msgr2=0x55c0ea90b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9ea9d9b640 1 --2- 172.21.15.103:0/3608266320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea90b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.753+0000 7f9ea9d9b640 1 -- 172.21.15.103:0/3608266320 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0ea8ff120 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.754+0000 7f9ea9d9b640 1 --2- 172.21.15.103:0/3608266320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea925e70 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0003e80 tx=0x7f9ea0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5a40d80146fdd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.754+0000 7f9e93fff640 1 -- 172.21.15.103:0/3608266320 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0030070 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.754+0000 7f9e93fff640 1 -- 172.21.15.103:0/3608266320 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea000b480 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.754+0000 7f9e93fff640 1 -- 172.21.15.103:0/3608266320 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00052a0 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.754+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/3608266320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 msgr2=0x55c0ea925e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.754+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/3608266320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea925e70 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0003e80 tx=0x7f9ea0005850 comp rx=0 tx=0).stop 2024-09-19T07:01:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.754+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/3608266320 shutdown_connections 2024-09-19T07:01:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.754+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/3608266320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea90e0c0 0x55c0ea9263b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.754+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/3608266320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea925e70 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.754+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/3608266320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea90b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.754+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/3608266320 >> 172.21.15.103:0/3608266320 conn(0x55c0ea8fb060 msgr2=0x55c0ea90d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.754+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/3608266320 shutdown_connections 2024-09-19T07:01:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.754+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/3608266320 wait complete. 2024-09-19T07:01:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9eb3cd7740 1 -- start start 2024-09-19T07:01:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9eb3cd7740 1 Processor -- start 2024-09-19T07:01:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9eb3cd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea916870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9eb3cd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea911b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9eb3cd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea90e0c0 0x55c0ea9120d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9eb3cd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1aca30 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9eb3cd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1ac360 con 0x55c0ea90e0c0 2024-09-19T07:01:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9eb3cd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1a95c0 con 0x55c0ea8fe9d0 2024-09-19T07:01:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9eb3cd7740 10 client.0 _ll_register_callbacks cb 0x7f9ea000bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9ea9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea911b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9ea9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea911b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42756/0 (socket says 172.21.15.103:42756) 2024-09-19T07:01:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9ea9d9b640 1 -- 172.21.15.103:0/629931141 learned_addr learned my addr 172.21.15.103:0/629931141 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9eaad9d640 1 --2- 172.21.15.103:0/629931141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea90e0c0 0x55c0ea9120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9eaa59c640 1 --2- 172.21.15.103:0/629931141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea916870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9ea9d9b640 1 -- 172.21.15.103:0/629931141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea8fe9d0 msgr2=0x55c0ea916870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9ea9d9b640 1 --2- 172.21.15.103:0/629931141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea916870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9ea9d9b640 1 -- 172.21.15.103:0/629931141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea90e0c0 msgr2=0x55c0ea9120d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9ea9d9b640 1 --2- 172.21.15.103:0/629931141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea90e0c0 0x55c0ea9120d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9ea9d9b640 1 -- 172.21.15.103:0/629931141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0ea912920 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.755+0000 7f9eaad9d640 1 --2- 172.21.15.103:0/629931141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea90e0c0 0x55c0ea9120d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.756+0000 7f9ea9d9b640 1 --2- 172.21.15.103:0/629931141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea911b90 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f9ea002e040 tx=0x7f9ea0004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d70cc6d3b5f813 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.756+0000 7f9e91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.756+0000 7f9e91ffb640 1 -- 172.21.15.103:0/629931141 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0030070 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.756+0000 7f9e91ffb640 1 -- 172.21.15.103:0/629931141 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f9ea0003800 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.756+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/629931141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea000b480 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.756+0000 7f9e91ffb640 1 -- 172.21.15.103:0/629931141 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9ea0003a00 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.756+0000 7f9eb3cd7740 10 client.19476 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.756+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/629931141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c0ea988e80 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.756+0000 7f9e91ffb640 1 -- 172.21.15.103:0/629931141 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea000b480 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.756+0000 7f9e91ffb640 1 -- 172.21.15.103:0/629931141 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9ea0022950 con 0x55c0ea8ff2f0 2024-09-19T07:01:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.756+0000 7f9eb3cd7740 10 client.19476 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.756+0000 7f9eb3cd7740 10 client.19476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.756+0000 7f9eb3cd7740 20 client.19476 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.756+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/629931141 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c0ea9894e0 0x55c0ea98b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.756+0000 7f9eb3cd7740 4 client.19476 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:01:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.756+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/629931141 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0ea86ad70 con 0x55c0ea9894e0 2024-09-19T07:01:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.757+0000 7f9eaa59c640 1 --2- 172.21.15.103:0/629931141 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c0ea9894e0 0x55c0ea98b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.757+0000 7f9eaa59c640 1 --2- 172.21.15.103:0/629931141 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c0ea9894e0 0x55c0ea98b900 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d11227d039a25911 server_cookie=f4ad51946adc5f8a in_seq=0 out_seq=0 2024-09-19T07:01:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.757+0000 7f9e91ffb640 10 client.19476 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.771+0000 7f9e91ffb640 1 -- 172.21.15.103:0/629931141 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0ea86ad70 con 0x55c0ea9894e0 2024-09-19T07:01:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.771+0000 7f9e91ffb640 10 client.19476 handle_command_reply: tid=0 2024-09-19T07:01:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.771+0000 7f9eb3cd7740 1 client.19476 shutdown 2024-09-19T07:01:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.771+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/629931141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 msgr2=0x55c0ea911b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.771+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/629931141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea911b90 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f9ea002e040 tx=0x7f9ea0004200 comp rx=0 tx=0).stop 2024-09-19T07:01:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.771+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/629931141 shutdown_connections 2024-09-19T07:01:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.771+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/629931141 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c0ea9894e0 0x55c0ea98b900 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.771+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/629931141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0ea90e0c0 0x55c0ea9120d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.771+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/629931141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0ea8ff2f0 0x55c0ea911b90 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.771+0000 7f9e91ffb640 0 client.19476 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.771+0000 7f9eb3cd7740 1 --2- 172.21.15.103:0/629931141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0ea8fe9d0 0x55c0ea916870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.771+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/629931141 >> 172.21.15.103:0/629931141 conn(0x55c0ea8fb060 msgr2=0x55c0ea8f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.772+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/629931141 shutdown_connections 2024-09-19T07:01:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.772+0000 7f9eb3cd7740 1 -- 172.21.15.103:0/629931141 wait complete. 2024-09-19T07:01:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.772+0000 7f9eb3cd7740 20 client.19476 trim_cache size 0 max 16384 2024-09-19T07:01:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:01:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:01:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:01:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:01:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.772+0000 7f9eb0f3b640 1 -- 172.21.15.103:0/946756440 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9e940a3460 msgr2=0x7f9e940a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.772+0000 7f9eb0f3b640 1 --2- 172.21.15.103:0/946756440 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9e940a3460 0x7f9e940a5920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9ea401e000 tx=0x7f9ea401b000 comp rx=0 tx=0).stop 2024-09-19T07:01:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.772+0000 7f9eb0f3b640 1 -- 172.21.15.103:0/946756440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9eac100850 msgr2=0x7f9eac1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.772+0000 7f9eb0f3b640 1 --2- 172.21.15.103:0/946756440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9eac100850 0x7f9eac1983a0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c02ae00 tx=0x7f9e9c003970 comp rx=0 tx=0).stop 2024-09-19T07:01:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.772+0000 7f9eb0f3b640 1 -- 172.21.15.103:0/946756440 shutdown_connections 2024-09-19T07:01:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.772+0000 7f9eb0f3b640 1 --2- 172.21.15.103:0/946756440 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9e940a3460 0x7f9e940a5920 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.772+0000 7f9eb0f3b640 1 --2- 172.21.15.103:0/946756440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9eac19cc60 0x7f9eac19d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.772+0000 7f9eb0f3b640 1 --2- 172.21.15.103:0/946756440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac108110 0x7f9eac1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.772+0000 7f9eb0f3b640 1 --2- 172.21.15.103:0/946756440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9eac100850 0x7f9eac1983a0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.772+0000 7f9eb0f3b640 1 -- 172.21.15.103:0/946756440 >> 172.21.15.103:0/946756440 conn(0x7f9eac075db0 msgr2=0x7f9eac105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.772+0000 7f9eb0f3b640 1 -- 172.21.15.103:0/946756440 shutdown_connections 2024-09-19T07:01:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:14.772+0000 7f9eb0f3b640 1 -- 172.21.15.103:0/946756440 wait complete. 2024-09-19T07:01:16.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:16 smithi174 ceph-mon[35637]: pgmap v375: 97 pgs: 97 active+clean; 4.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 KiB/s wr, 8 op/s 2024-09-19T07:01:16.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:16 smithi103 ceph-mon[31158]: pgmap v375: 97 pgs: 97 active+clean; 4.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 KiB/s wr, 8 op/s 2024-09-19T07:01:16.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:16 smithi134 ceph-mon[35356]: pgmap v375: 97 pgs: 97 active+clean; 4.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 KiB/s wr, 8 op/s 2024-09-19T07:01:17.333 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:01:17.595 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:18.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.003+0000 7f4b3d0b6640 1 -- 172.21.15.103:0/2263377365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b38102790 msgr2=0x7f4b381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:18.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.003+0000 7f4b3d0b6640 1 --2- 172.21.15.103:0/2263377365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b38102790 0x7f4b381067e0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009900 tx=0x7f4b2c02a3f0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.003+0000 7f4b3d0b6640 1 -- 172.21.15.103:0/2263377365 shutdown_connections 2024-09-19T07:01:18.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.003+0000 7f4b3d0b6640 1 --2- 172.21.15.103:0/2263377365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b38106f10 0x7f4b3810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.003+0000 7f4b3d0b6640 1 --2- 172.21.15.103:0/2263377365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b38102790 0x7f4b381067e0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.003+0000 7f4b3d0b6640 1 --2- 172.21.15.103:0/2263377365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b38101de0 0x7f4b381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.003+0000 7f4b3d0b6640 1 -- 172.21.15.103:0/2263377365 >> 172.21.15.103:0/2263377365 conn(0x7f4b380fd8c0 msgr2=0x7f4b380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:18.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.004+0000 7f4b3d0b6640 1 -- 172.21.15.103:0/2263377365 shutdown_connections 2024-09-19T07:01:18.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.004+0000 7f4b3d0b6640 1 -- 172.21.15.103:0/2263377365 wait complete. 2024-09-19T07:01:18.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.005+0000 7f4b3d0b6640 1 Processor -- start 2024-09-19T07:01:18.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.005+0000 7f4b3d0b6640 1 -- start start 2024-09-19T07:01:18.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.005+0000 7f4b3d0b6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b38101de0 0x7f4b38198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.005+0000 7f4b3d0b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b38102790 0x7f4b38198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.005+0000 7f4b3d0b6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b38106f10 0x7f4b3819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.005+0000 7f4b3d0b6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b38111f30 con 0x7f4b38102790 2024-09-19T07:01:18.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.005+0000 7f4b3d0b6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b38111db0 con 0x7f4b38106f10 2024-09-19T07:01:18.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.005+0000 7f4b3d0b6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b381120b0 con 0x7f4b38101de0 2024-09-19T07:01:18.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.005+0000 7f4b3659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b38102790 0x7f4b38198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.005+0000 7f4b3659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b38102790 0x7f4b38198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46362/0 (socket says 172.21.15.103:46362) 2024-09-19T07:01:18.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.005+0000 7f4b3659c640 1 -- 172.21.15.103:0/1280197456 learned_addr learned my addr 172.21.15.103:0/1280197456 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:18.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.005+0000 7f4b36d9d640 1 --2- 172.21.15.103:0/1280197456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b38101de0 0x7f4b38198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.005+0000 7f4b3759e640 1 --2- 172.21.15.103:0/1280197456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b38106f10 0x7f4b3819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.006+0000 7f4b3659c640 1 -- 172.21.15.103:0/1280197456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b38101de0 msgr2=0x7f4b38198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:18.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.006+0000 7f4b3659c640 1 --2- 172.21.15.103:0/1280197456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b38101de0 0x7f4b38198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.006+0000 7f4b3659c640 1 -- 172.21.15.103:0/1280197456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b38106f10 msgr2=0x7f4b3819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:18.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.006+0000 7f4b3659c640 1 --2- 172.21.15.103:0/1280197456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b38106f10 0x7f4b3819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.006+0000 7f4b3659c640 1 -- 172.21.15.103:0/1280197456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3819d3c0 con 0x7f4b38102790 2024-09-19T07:01:18.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.006+0000 7f4b3759e640 1 --2- 172.21.15.103:0/1280197456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b38106f10 0x7f4b3819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:18.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.006+0000 7f4b3659c640 1 --2- 172.21.15.103:0/1280197456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b38102790 0x7f4b38198950 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c005ab0 tx=0x7f4b2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:18.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.006+0000 7f4b1ffff640 1 -- 172.21.15.103:0/1280197456 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c03f030 con 0x7f4b38102790 2024-09-19T07:01:18.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.006+0000 7f4b3d0b6640 1 -- 172.21.15.103:0/1280197456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3819d650 con 0x7f4b38102790 2024-09-19T07:01:18.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.006+0000 7f4b1ffff640 1 -- 172.21.15.103:0/1280197456 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c003ce0 con 0x7f4b38102790 2024-09-19T07:01:18.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.006+0000 7f4b1ffff640 1 -- 172.21.15.103:0/1280197456 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c035070 con 0x7f4b38102790 2024-09-19T07:01:18.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.006+0000 7f4b3d0b6640 1 -- 172.21.15.103:0/1280197456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b381a4f90 con 0x7f4b38102790 2024-09-19T07:01:18.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.008+0000 7f4b1ffff640 1 -- 172.21.15.103:0/1280197456 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4b2c042030 con 0x7f4b38102790 2024-09-19T07:01:18.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3f48f740 1 Processor -- start 2024-09-19T07:01:18.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3f48f740 1 -- start start 2024-09-19T07:01:18.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3f48f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a624a0 0x55f071a62880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3f48f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a62dc0 0x55f071a6d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3f48f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a6d680 0x55f071a6fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3f48f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f0719c82e0 con 0x55f071a6d680 2024-09-19T07:01:18.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3f48f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f07196fce0 con 0x55f071a624a0 2024-09-19T07:01:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3f48f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f0719c42a0 con 0x55f071a62dc0 2024-09-19T07:01:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a6d680 0x55f071a6fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a6d680 0x55f071a6fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46366/0 (socket says 172.21.15.103:46366) 2024-09-19T07:01:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3659c640 1 -- 172.21.15.103:0/2391579129 learned_addr learned my addr 172.21.15.103:0/2391579129 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b1ffff640 1 --2- 172.21.15.103:0/1280197456 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4b200a33b0 0x7f4b200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b36d9d640 1 --2- 172.21.15.103:0/2391579129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a62dc0 0x55f071a6d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3759e640 1 --2- 172.21.15.103:0/2391579129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a624a0 0x55f071a62880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3759e640 1 --2- 172.21.15.103:0/1280197456 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4b200a33b0 0x7f4b200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3659c640 1 -- 172.21.15.103:0/2391579129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a62dc0 msgr2=0x55f071a6d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3659c640 1 --2- 172.21.15.103:0/2391579129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a62dc0 0x55f071a6d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3659c640 1 -- 172.21.15.103:0/2391579129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a624a0 msgr2=0x55f071a62880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.009+0000 7f4b3659c640 1 --2- 172.21.15.103:0/2391579129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a624a0 0x55f071a62880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.010+0000 7f4b1ffff640 1 -- 172.21.15.103:0/1280197456 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f4b2c0f02e0 con 0x7f4b38102790 2024-09-19T07:01:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.010+0000 7f4b3659c640 1 -- 172.21.15.103:0/2391579129 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f071954440 con 0x55f071a6d680 2024-09-19T07:01:18.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.010+0000 7f4b36d9d640 1 --2- 172.21.15.103:0/2391579129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a62dc0 0x55f071a6d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:18.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.010+0000 7f4b3659c640 1 --2- 172.21.15.103:0/2391579129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a6d680 0x55f071a6fa70 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c0ed9b0 tx=0x7f4b2c02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9fb13e29dcf700 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:18.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.010+0000 7f4b1d7fa640 1 -- 172.21.15.103:0/2391579129 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c049070 con 0x55f071a6d680 2024-09-19T07:01:18.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.010+0000 7f4b3759e640 1 --2- 172.21.15.103:0/1280197456 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4b200a33b0 0x7f4b200a5870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4b30005d50 tx=0x7f4b30005890 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:18.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.010+0000 7f4b1d7fa640 1 -- 172.21.15.103:0/2391579129 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c0bcca0 con 0x55f071a6d680 2024-09-19T07:01:18.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.010+0000 7f4b1d7fa640 1 -- 172.21.15.103:0/2391579129 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c0c0950 con 0x55f071a6d680 2024-09-19T07:01:18.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 -- 172.21.15.103:0/2391579129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a6d680 msgr2=0x55f071a6fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:18.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/2391579129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a6d680 0x55f071a6fa70 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c0ed9b0 tx=0x7f4b2c02a3f0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 -- 172.21.15.103:0/2391579129 shutdown_connections 2024-09-19T07:01:18.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/2391579129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a6d680 0x55f071a6fa70 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/2391579129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a62dc0 0x55f071a6d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/2391579129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a624a0 0x55f071a62880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 -- 172.21.15.103:0/2391579129 >> 172.21.15.103:0/2391579129 conn(0x55f071a5ea30 msgr2=0x55f071a5ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:18.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 -- 172.21.15.103:0/2391579129 shutdown_connections 2024-09-19T07:01:18.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 -- 172.21.15.103:0/2391579129 wait complete. 2024-09-19T07:01:18.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 -- start start 2024-09-19T07:01:18.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 Processor -- start 2024-09-19T07:01:18.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a624a0 0x55f071a6cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a62dc0 0x55f071a6a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a6d680 0x55f071a6a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f071a70010 con 0x55f071a624a0 2024-09-19T07:01:18.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f071983050 con 0x55f071a62dc0 2024-09-19T07:01:18.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f071a70190 con 0x55f071a6d680 2024-09-19T07:01:18.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b3f48f740 10 client.0 _ll_register_callbacks cb 0x55f0719c3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:18.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.011+0000 7f4b36d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a624a0 0x55f071a6cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b36d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a624a0 0x55f071a6cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46374/0 (socket says 172.21.15.103:46374) 2024-09-19T07:01:18.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b36d9d640 1 -- 172.21.15.103:0/606147672 learned_addr learned my addr 172.21.15.103:0/606147672 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:18.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b3659c640 1 --2- 172.21.15.103:0/606147672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a62dc0 0x55f071a6a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b36d9d640 1 -- 172.21.15.103:0/606147672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a6d680 msgr2=0x55f071a6a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:18.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b36d9d640 1 --2- 172.21.15.103:0/606147672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a6d680 0x55f071a6a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b36d9d640 1 -- 172.21.15.103:0/606147672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a62dc0 msgr2=0x55f071a6a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:18.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b36d9d640 1 --2- 172.21.15.103:0/606147672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a62dc0 0x55f071a6a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b36d9d640 1 -- 172.21.15.103:0/606147672 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0717393c0 con 0x55f071a624a0 2024-09-19T07:01:18.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b36d9d640 1 --2- 172.21.15.103:0/606147672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a624a0 0x55f071a6cf50 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f4b28011ae0 tx=0x7f4b28011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56ce0cda885f1c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:18.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:18.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b1aff5640 1 -- 172.21.15.103:0/606147672 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2801b400 con 0x55f071a624a0 2024-09-19T07:01:18.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b1aff5640 1 -- 172.21.15.103:0/606147672 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f4b2801a3a0 con 0x55f071a624a0 2024-09-19T07:01:18.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b3f48f740 1 -- 172.21.15.103:0/606147672 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b2c0bcca0 con 0x55f071a624a0 2024-09-19T07:01:18.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b1aff5640 1 -- 172.21.15.103:0/606147672 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b2801b5a0 con 0x55f071a624a0 2024-09-19T07:01:18.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b3f48f740 10 client.19488 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:18.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b3f48f740 1 -- 172.21.15.103:0/606147672 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f071a7b8b0 con 0x55f071a624a0 2024-09-19T07:01:18.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.012+0000 7f4b1aff5640 1 -- 172.21.15.103:0/606147672 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c0bcca0 con 0x55f071a624a0 2024-09-19T07:01:18.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.013+0000 7f4b1aff5640 1 -- 172.21.15.103:0/606147672 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4b2801ab40 con 0x55f071a624a0 2024-09-19T07:01:18.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.013+0000 7f4b3f48f740 10 client.19488 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:18.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.013+0000 7f4b3f48f740 10 client.19488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:18.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.013+0000 7f4b3f48f740 20 client.19488 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:18.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.013+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/606147672 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f071a7cd70 0x55f071a7f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.013+0000 7f4b3f48f740 4 client.19488 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:18.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.013+0000 7f4b3f48f740 1 -- 172.21.15.103:0/606147672 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0719d5850 con 0x55f071a7cd70 2024-09-19T07:01:18.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.013+0000 7f4b3659c640 1 --2- 172.21.15.103:0/606147672 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f071a7cd70 0x55f071a7f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.013+0000 7f4b1aff5640 10 client.19488 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:18.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.013+0000 7f4b3659c640 1 --2- 172.21.15.103:0/606147672 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f071a7cd70 0x55f071a7f200 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d244d3ba004774d server_cookie=67b4baedf5e7856b in_seq=0 out_seq=0 2024-09-19T07:01:18.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.014+0000 7f4b1aff5640 1 -- 172.21.15.103:0/606147672 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0719d5850 con 0x55f071a7cd70 2024-09-19T07:01:18.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.014+0000 7f4b1aff5640 10 client.19488 handle_command_reply: tid=0 2024-09-19T07:01:18.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.014+0000 7f4b3f48f740 1 client.19488 shutdown 2024-09-19T07:01:18.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.014+0000 7f4b3f48f740 1 -- 172.21.15.103:0/606147672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a624a0 msgr2=0x55f071a6cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:18.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.014+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/606147672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a624a0 0x55f071a6cf50 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f4b28011ae0 tx=0x7f4b28011ee0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.014+0000 7f4b3f48f740 1 -- 172.21.15.103:0/606147672 shutdown_connections 2024-09-19T07:01:18.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.014+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/606147672 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f071a7cd70 0x55f071a7f200 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.014+0000 7f4b1aff5640 0 client.19488 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:18.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.014+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/606147672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a6d680 0x55f071a6a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.014+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/606147672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a62dc0 0x55f071a6a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.014+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/606147672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a624a0 0x55f071a6cf50 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.014+0000 7f4b3f48f740 1 -- 172.21.15.103:0/606147672 >> 172.21.15.103:0/606147672 conn(0x55f071a5ea30 msgr2=0x55f071a64760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:18.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.014+0000 7f4b3f48f740 1 -- 172.21.15.103:0/606147672 shutdown_connections 2024-09-19T07:01:18.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.014+0000 7f4b3f48f740 1 -- 172.21.15.103:0/606147672 wait complete. 2024-09-19T07:01:18.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.014+0000 7f4b3f48f740 20 client.19488 trim_cache size 0 max 16384 2024-09-19T07:01:18.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.031+0000 7f4b3f48f740 1 Processor -- start 2024-09-19T07:01:18.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.031+0000 7f4b3f48f740 1 -- start start 2024-09-19T07:01:18.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.031+0000 7f4b3f48f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a5ea30 0x55f071a791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.031+0000 7f4b3f48f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a624a0 0x55f071a657f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.031+0000 7f4b3f48f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a62dc0 0x55f071a84ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.031+0000 7f4b3f48f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b381ad8b0 con 0x55f071a62dc0 2024-09-19T07:01:18.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.031+0000 7f4b3f48f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b381af3f0 con 0x55f071a624a0 2024-09-19T07:01:18.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.031+0000 7f4b3f48f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b381adf80 con 0x55f071a5ea30 2024-09-19T07:01:18.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.031+0000 7f4b3759e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a62dc0 0x55f071a84ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.031+0000 7f4b3759e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a62dc0 0x55f071a84ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46384/0 (socket says 172.21.15.103:46384) 2024-09-19T07:01:18.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.032+0000 7f4b3759e640 1 -- 172.21.15.103:0/3456854265 learned_addr learned my addr 172.21.15.103:0/3456854265 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:18.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.032+0000 7f4b3659c640 1 --2- 172.21.15.103:0/3456854265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a624a0 0x55f071a657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.032+0000 7f4b36d9d640 1 --2- 172.21.15.103:0/3456854265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a5ea30 0x55f071a791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.032+0000 7f4b3759e640 1 -- 172.21.15.103:0/3456854265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a5ea30 msgr2=0x55f071a791a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:18.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.032+0000 7f4b3759e640 1 --2- 172.21.15.103:0/3456854265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a5ea30 0x55f071a791a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.032+0000 7f4b3759e640 1 -- 172.21.15.103:0/3456854265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a624a0 msgr2=0x55f071a657f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:18.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.032+0000 7f4b3759e640 1 --2- 172.21.15.103:0/3456854265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a624a0 0x55f071a657f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.032+0000 7f4b3759e640 1 -- 172.21.15.103:0/3456854265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f071a5e860 con 0x55f071a62dc0 2024-09-19T07:01:18.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.032+0000 7f4b36d9d640 1 --2- 172.21.15.103:0/3456854265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a5ea30 0x55f071a791a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:18.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.032+0000 7f4b3759e640 1 --2- 172.21.15.103:0/3456854265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a62dc0 0x55f071a84ad0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f4b30010780 tx=0x7f4b30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b66a0eb301b5e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:18.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.032+0000 7f4b1cff9640 1 -- 172.21.15.103:0/3456854265 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30021070 con 0x55f071a62dc0 2024-09-19T07:01:18.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.032+0000 7f4b1cff9640 1 -- 172.21.15.103:0/3456854265 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b30017070 con 0x55f071a62dc0 2024-09-19T07:01:18.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.032+0000 7f4b1cff9640 1 -- 172.21.15.103:0/3456854265 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3000fa30 con 0x55f071a62dc0 2024-09-19T07:01:18.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 -- 172.21.15.103:0/3456854265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a62dc0 msgr2=0x55f071a84ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:18.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/3456854265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a62dc0 0x55f071a84ad0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f4b30010780 tx=0x7f4b30004730 comp rx=0 tx=0).stop 2024-09-19T07:01:18.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 -- 172.21.15.103:0/3456854265 shutdown_connections 2024-09-19T07:01:18.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/3456854265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a62dc0 0x55f071a84ad0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/3456854265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a624a0 0x55f071a657f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/3456854265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a5ea30 0x55f071a791a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 -- 172.21.15.103:0/3456854265 >> 172.21.15.103:0/3456854265 conn(0x55f071a6a8a0 msgr2=0x55f071a650e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:18.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 -- 172.21.15.103:0/3456854265 shutdown_connections 2024-09-19T07:01:18.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 -- 172.21.15.103:0/3456854265 wait complete. 2024-09-19T07:01:18.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 -- start start 2024-09-19T07:01:18.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 Processor -- start 2024-09-19T07:01:18.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a5ea30 0x55f071a75c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a624a0 0x55f071a70f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a62dc0 0x55f071a716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b381aed20 con 0x55f071a5ea30 2024-09-19T07:01:18.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b381ae650 con 0x55f071a62dc0 2024-09-19T07:01:18.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b381ab8b0 con 0x55f071a624a0 2024-09-19T07:01:18.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.033+0000 7f4b3f48f740 10 client.0 _ll_register_callbacks cb 0x7f4b28007910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:18.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b36d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a5ea30 0x55f071a75c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b36d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a5ea30 0x55f071a75c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46394/0 (socket says 172.21.15.103:46394) 2024-09-19T07:01:18.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b36d9d640 1 -- 172.21.15.103:0/805180163 learned_addr learned my addr 172.21.15.103:0/805180163 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:18.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b3759e640 1 --2- 172.21.15.103:0/805180163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a62dc0 0x55f071a716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b3659c640 1 --2- 172.21.15.103:0/805180163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a624a0 0x55f071a70f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b36d9d640 1 -- 172.21.15.103:0/805180163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a624a0 msgr2=0x55f071a70f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:18.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b36d9d640 1 --2- 172.21.15.103:0/805180163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a624a0 0x55f071a70f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b36d9d640 1 -- 172.21.15.103:0/805180163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a62dc0 msgr2=0x55f071a716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:18.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b36d9d640 1 --2- 172.21.15.103:0/805180163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a62dc0 0x55f071a716d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b36d9d640 1 -- 172.21.15.103:0/805180163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f071ae6aa0 con 0x55f071a5ea30 2024-09-19T07:01:18.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b3659c640 1 --2- 172.21.15.103:0/805180163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a624a0 0x55f071a70f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:18.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b3759e640 1 --2- 172.21.15.103:0/805180163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a62dc0 0x55f071a716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:18.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b36d9d640 1 --2- 172.21.15.103:0/805180163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a5ea30 0x55f071a75c10 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f4b2801af60 tx=0x7f4b28023890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb49cb566664d7ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:18.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:18.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b1aff5640 1 -- 172.21.15.103:0/805180163 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28030060 con 0x55f071a5ea30 2024-09-19T07:01:18.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b1aff5640 1 -- 172.21.15.103:0/805180163 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f4b28023b00 con 0x55f071a5ea30 2024-09-19T07:01:18.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b3f48f740 1 -- 172.21.15.103:0/805180163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b30017070 con 0x55f071a5ea30 2024-09-19T07:01:18.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b1aff5640 1 -- 172.21.15.103:0/805180163 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b28023d00 con 0x55f071a5ea30 2024-09-19T07:01:18.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b3f48f740 10 client.19494 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:18.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.034+0000 7f4b3f48f740 1 -- 172.21.15.103:0/805180163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f071ae7cd0 con 0x55f071a5ea30 2024-09-19T07:01:18.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.035+0000 7f4b1aff5640 1 -- 172.21.15.103:0/805180163 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b30017070 con 0x55f071a5ea30 2024-09-19T07:01:18.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.035+0000 7f4b1aff5640 1 -- 172.21.15.103:0/805180163 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4b2801a7c0 con 0x55f071a5ea30 2024-09-19T07:01:18.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.035+0000 7f4b3f48f740 10 client.19494 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:18.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.035+0000 7f4b3f48f740 10 client.19494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:18.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.035+0000 7f4b3f48f740 20 client.19494 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:18.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.035+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/805180163 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f071a6d680 0x55f071aea460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:18.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.035+0000 7f4b3f48f740 4 client.19494 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:01:18.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.035+0000 7f4b3f48f740 1 -- 172.21.15.103:0/805180163 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f0719cfd60 con 0x55f071a6d680 2024-09-19T07:01:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.035+0000 7f4b3659c640 1 --2- 172.21.15.103:0/805180163 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f071a6d680 0x55f071aea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.035+0000 7f4b1aff5640 10 client.19494 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.035+0000 7f4b3659c640 1 --2- 172.21.15.103:0/805180163 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f071a6d680 0x55f071aea460 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=656f38023bd0e6de server_cookie=f613e877d1ba693d in_seq=0 out_seq=0 2024-09-19T07:01:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b1aff5640 1 -- 172.21.15.103:0/805180163 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f0719cfd60 con 0x55f071a6d680 2024-09-19T07:01:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b1aff5640 10 client.19494 handle_command_reply: tid=0 2024-09-19T07:01:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b3f48f740 1 client.19494 shutdown 2024-09-19T07:01:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b3f48f740 1 -- 172.21.15.103:0/805180163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a5ea30 msgr2=0x55f071a75c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/805180163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a5ea30 0x55f071a75c10 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f4b2801af60 tx=0x7f4b28023890 comp rx=0 tx=0).stop 2024-09-19T07:01:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b3f48f740 1 -- 172.21.15.103:0/805180163 shutdown_connections 2024-09-19T07:01:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/805180163 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f071a6d680 0x55f071aea460 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/805180163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f071a62dc0 0x55f071a716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/805180163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f071a624a0 0x55f071a70f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b3f48f740 1 --2- 172.21.15.103:0/805180163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f071a5ea30 0x55f071a75c10 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b3f48f740 1 -- 172.21.15.103:0/805180163 >> 172.21.15.103:0/805180163 conn(0x55f071a6a8a0 msgr2=0x55f071a65900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b1aff5640 0 client.19494 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b3f48f740 1 -- 172.21.15.103:0/805180163 shutdown_connections 2024-09-19T07:01:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b3f48f740 1 -- 172.21.15.103:0/805180163 wait complete. 2024-09-19T07:01:18.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b3f48f740 20 client.19494 trim_cache size 0 max 16384 2024-09-19T07:01:18.043 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:01:18.043 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:01:18.043 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:01:18.043 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:01:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b3d0b6640 1 -- 172.21.15.103:0/1280197456 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4b200a33b0 msgr2=0x7f4b200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b3d0b6640 1 --2- 172.21.15.103:0/1280197456 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4b200a33b0 0x7f4b200a5870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4b30005d50 tx=0x7f4b30005890 comp rx=0 tx=0).stop 2024-09-19T07:01:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b3d0b6640 1 -- 172.21.15.103:0/1280197456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b38102790 msgr2=0x7f4b38198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.036+0000 7f4b3d0b6640 1 --2- 172.21.15.103:0/1280197456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b38102790 0x7f4b38198950 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c005ab0 tx=0x7f4b2c0042f0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.037+0000 7f4b3d0b6640 1 -- 172.21.15.103:0/1280197456 shutdown_connections 2024-09-19T07:01:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.037+0000 7f4b3d0b6640 1 --2- 172.21.15.103:0/1280197456 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4b200a33b0 0x7f4b200a5870 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.037+0000 7f4b3d0b6640 1 --2- 172.21.15.103:0/1280197456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b38106f10 0x7f4b3819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.037+0000 7f4b3d0b6640 1 --2- 172.21.15.103:0/1280197456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b38102790 0x7f4b38198950 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.037+0000 7f4b3d0b6640 1 --2- 172.21.15.103:0/1280197456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b38101de0 0x7f4b38198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:18.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.037+0000 7f4b3d0b6640 1 -- 172.21.15.103:0/1280197456 >> 172.21.15.103:0/1280197456 conn(0x7f4b380fd8c0 msgr2=0x7f4b380fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:18.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.037+0000 7f4b3d0b6640 1 -- 172.21.15.103:0/1280197456 shutdown_connections 2024-09-19T07:01:18.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:18.037+0000 7f4b3d0b6640 1 -- 172.21.15.103:0/1280197456 wait complete. 2024-09-19T07:01:18.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:18 smithi174 ceph-mon[35637]: pgmap v376: 97 pgs: 97 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T07:01:18.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:18 smithi103 ceph-mon[31158]: pgmap v376: 97 pgs: 97 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T07:01:18.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:18 smithi134 ceph-mon[35356]: pgmap v376: 97 pgs: 97 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T07:01:18.831 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:01:18.832 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:01:19.079 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:19.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.500+0000 7f7437fb7640 1 -- 172.21.15.103:0/1006604572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7430106f50 msgr2=0x7f743010eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:19.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.500+0000 7f7437fb7640 1 --2- 172.21.15.103:0/1006604572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7430106f50 0x7f743010eaf0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f7428009cd0 tx=0x7f742802a8e0 comp rx=0 tx=0).stop 2024-09-19T07:01:19.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.500+0000 7f7437fb7640 1 -- 172.21.15.103:0/1006604572 shutdown_connections 2024-09-19T07:01:19.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.500+0000 7f7437fb7640 1 --2- 172.21.15.103:0/1006604572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7430106f50 0x7f743010eaf0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:19.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.500+0000 7f7437fb7640 1 --2- 172.21.15.103:0/1006604572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7430100ca0 0x7f7430106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:19.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.500+0000 7f7437fb7640 1 --2- 172.21.15.103:0/1006604572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74301002f0 0x7f74301006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:19.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.500+0000 7f7437fb7640 1 -- 172.21.15.103:0/1006604572 >> 172.21.15.103:0/1006604572 conn(0x7f7430077210 msgr2=0x7f74300fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:19.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.500+0000 7f7437fb7640 1 -- 172.21.15.103:0/1006604572 shutdown_connections 2024-09-19T07:01:19.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.500+0000 7f7437fb7640 1 -- 172.21.15.103:0/1006604572 wait complete. 2024-09-19T07:01:19.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.501+0000 7f7437fb7640 1 Processor -- start 2024-09-19T07:01:19.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.501+0000 7f7437fb7640 1 -- start start 2024-09-19T07:01:19.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.501+0000 7f7437fb7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74301002f0 0x7f74301983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:19.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.501+0000 7f7437fb7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7430100ca0 0x7f7430198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:19.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.501+0000 7f7437fb7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7430106f50 0x7f743019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:19.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.501+0000 7f7437fb7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7430111fa0 con 0x7f74301002f0 2024-09-19T07:01:19.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.501+0000 7f7437fb7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7430111e20 con 0x7f7430100ca0 2024-09-19T07:01:19.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.501+0000 7f7437fb7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7430112120 con 0x7f7430106f50 2024-09-19T07:01:19.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.501+0000 7f7435d53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74301002f0 0x7f74301983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:19.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.501+0000 7f7435d53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74301002f0 0x7f74301983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46400/0 (socket says 172.21.15.103:46400) 2024-09-19T07:01:19.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.501+0000 7f7435d53640 1 -- 172.21.15.103:0/892111075 learned_addr learned my addr 172.21.15.103:0/892111075 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:19.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.501+0000 7f7436554640 1 --2- 172.21.15.103:0/892111075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7430106f50 0x7f743019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:19.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.502+0000 7f7435552640 1 --2- 172.21.15.103:0/892111075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7430100ca0 0x7f7430198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:19.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.502+0000 7f7435d53640 1 -- 172.21.15.103:0/892111075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7430106f50 msgr2=0x7f743019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:19.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.502+0000 7f7435d53640 1 --2- 172.21.15.103:0/892111075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7430106f50 0x7f743019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:19.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.502+0000 7f7435d53640 1 -- 172.21.15.103:0/892111075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7430100ca0 msgr2=0x7f7430198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:19.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.502+0000 7f7435d53640 1 --2- 172.21.15.103:0/892111075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7430100ca0 0x7f7430198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:19.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.502+0000 7f7435d53640 1 -- 172.21.15.103:0/892111075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743019d370 con 0x7f74301002f0 2024-09-19T07:01:19.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.502+0000 7f7435552640 1 --2- 172.21.15.103:0/892111075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7430100ca0 0x7f7430198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:19.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.502+0000 7f7435d53640 1 --2- 172.21.15.103:0/892111075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74301002f0 0x7f74301983c0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f742c00c920 tx=0x7f742c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:19.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.502+0000 7f7426ffd640 1 -- 172.21.15.103:0/892111075 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c007c70 con 0x7f74301002f0 2024-09-19T07:01:19.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.502+0000 7f7426ffd640 1 -- 172.21.15.103:0/892111075 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f742c007e10 con 0x7f74301002f0 2024-09-19T07:01:19.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.502+0000 7f7437fb7640 1 -- 172.21.15.103:0/892111075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743019d660 con 0x7f74301002f0 2024-09-19T07:01:19.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.502+0000 7f7426ffd640 1 -- 172.21.15.103:0/892111075 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c018890 con 0x7f74301002f0 2024-09-19T07:01:19.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.502+0000 7f7437fb7640 1 -- 172.21.15.103:0/892111075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74301a4f40 con 0x7f74301002f0 2024-09-19T07:01:19.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.503+0000 7f7437fb7640 1 -- 172.21.15.103:0/892111075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74301006d0 con 0x7f74301002f0 2024-09-19T07:01:19.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.505+0000 7f7426ffd640 1 -- 172.21.15.103:0/892111075 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f742c018a50 con 0x7f74301002f0 2024-09-19T07:01:19.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.505+0000 7f7426ffd640 1 --2- 172.21.15.103:0/892111075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f741c0a33b0 0x7f741c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:19.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.506+0000 7f7435552640 1 --2- 172.21.15.103:0/892111075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f741c0a33b0 0x7f741c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:19.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.506+0000 7f7426ffd640 1 -- 172.21.15.103:0/892111075 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f742c0d0120 con 0x7f74301002f0 2024-09-19T07:01:19.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.506+0000 7f7435552640 1 --2- 172.21.15.103:0/892111075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f741c0a33b0 0x7f741c0a5870 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f74301046e0 tx=0x7f7420005d20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:19.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.509+0000 7f7426ffd640 1 -- 172.21.15.103:0/892111075 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f742c014030 con 0x7f74301002f0 2024-09-19T07:01:19.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.709+0000 7f7437fb7640 1 -- 172.21.15.103:0/892111075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f743010a170 con 0x7f74301002f0 2024-09-19T07:01:19.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.709+0000 7f7426ffd640 1 -- 172.21.15.103:0/892111075 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f742c09cb70 con 0x7f74301002f0 2024-09-19T07:01:19.710 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:01:19.710 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:01:19.711 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:01:19.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.711+0000 7f7437fb7640 1 -- 172.21.15.103:0/892111075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f741c0a33b0 msgr2=0x7f741c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:19.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.711+0000 7f7437fb7640 1 --2- 172.21.15.103:0/892111075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f741c0a33b0 0x7f741c0a5870 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f74301046e0 tx=0x7f7420005d20 comp rx=0 tx=0).stop 2024-09-19T07:01:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.711+0000 7f7437fb7640 1 -- 172.21.15.103:0/892111075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74301002f0 msgr2=0x7f74301983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.711+0000 7f7437fb7640 1 --2- 172.21.15.103:0/892111075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74301002f0 0x7f74301983c0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f742c00c920 tx=0x7f742c00cde0 comp rx=0 tx=0).stop 2024-09-19T07:01:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.711+0000 7f7437fb7640 1 -- 172.21.15.103:0/892111075 shutdown_connections 2024-09-19T07:01:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.711+0000 7f7437fb7640 1 --2- 172.21.15.103:0/892111075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f741c0a33b0 0x7f741c0a5870 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.711+0000 7f7437fb7640 1 --2- 172.21.15.103:0/892111075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7430106f50 0x7f743019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.711+0000 7f7437fb7640 1 --2- 172.21.15.103:0/892111075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7430100ca0 0x7f7430198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.711+0000 7f7437fb7640 1 --2- 172.21.15.103:0/892111075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74301002f0 0x7f74301983c0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.711+0000 7f7437fb7640 1 -- 172.21.15.103:0/892111075 >> 172.21.15.103:0/892111075 conn(0x7f7430077210 msgr2=0x7f74301056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.711+0000 7f7437fb7640 1 -- 172.21.15.103:0/892111075 shutdown_connections 2024-09-19T07:01:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:19.712+0000 7f7437fb7640 1 -- 172.21.15.103:0/892111075 wait complete. 2024-09-19T07:01:20.285 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:20 smithi103 ceph-mon[31158]: pgmap v377: 97 pgs: 97 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T07:01:20.285 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:20 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/892111075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:01:20.313 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:01:20.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:20 smithi174 ceph-mon[35637]: pgmap v377: 97 pgs: 97 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T07:01:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:20 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/892111075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:01:20.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:20 smithi134 ceph-mon[35356]: pgmap v377: 97 pgs: 97 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T07:01:20.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:20 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/892111075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:01:20.560 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:21.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.005+0000 7f5c2ccd8640 1 -- 172.21.15.103:0/921883495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c281082a0 msgr2=0x7f5c2810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:21.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.005+0000 7f5c2ccd8640 1 --2- 172.21.15.103:0/921883495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c281082a0 0x7f5c2810ec40 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009930 tx=0x7f5c1802a420 comp rx=0 tx=0).stop 2024-09-19T07:01:21.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.005+0000 7f5c2ccd8640 1 -- 172.21.15.103:0/921883495 shutdown_connections 2024-09-19T07:01:21.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.005+0000 7f5c2ccd8640 1 --2- 172.21.15.103:0/921883495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c281082a0 0x7f5c2810ec40 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.005+0000 7f5c2ccd8640 1 --2- 172.21.15.103:0/921883495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c281036e0 0x7f5c28107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.005+0000 7f5c2ccd8640 1 --2- 172.21.15.103:0/921883495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c28102d30 0x7f5c28103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.005+0000 7f5c2ccd8640 1 -- 172.21.15.103:0/921883495 >> 172.21.15.103:0/921883495 conn(0x7f5c280fc7d0 msgr2=0x7f5c280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:21.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.005+0000 7f5c2ccd8640 1 -- 172.21.15.103:0/921883495 shutdown_connections 2024-09-19T07:01:21.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.005+0000 7f5c2ccd8640 1 -- 172.21.15.103:0/921883495 wait complete. 2024-09-19T07:01:21.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.005+0000 7f5c2ccd8640 1 Processor -- start 2024-09-19T07:01:21.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c2ccd8640 1 -- start start 2024-09-19T07:01:21.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c2ccd8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c28102d30 0x7f5c28198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c2ccd8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c281036e0 0x7f5c28198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c2ccd8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c281082a0 0x7f5c2819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c2ccd8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5c281120a0 con 0x7f5c281036e0 2024-09-19T07:01:21.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c2ccd8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c28111f20 con 0x7f5c281082a0 2024-09-19T07:01:21.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c2ccd8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c28112220 con 0x7f5c28102d30 2024-09-19T07:01:21.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c25d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c281036e0 0x7f5c28198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c25d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c281036e0 0x7f5c28198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46412/0 (socket says 172.21.15.103:46412) 2024-09-19T07:01:21.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c25d9b640 1 -- 172.21.15.103:0/1620378769 learned_addr learned my addr 172.21.15.103:0/1620378769 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:21.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c2659c640 1 --2- 172.21.15.103:0/1620378769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c28102d30 0x7f5c28198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c26d9d640 1 --2- 172.21.15.103:0/1620378769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c281082a0 0x7f5c2819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c25d9b640 1 -- 172.21.15.103:0/1620378769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c28102d30 msgr2=0x7f5c28198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:21.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c25d9b640 1 --2- 172.21.15.103:0/1620378769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c28102d30 0x7f5c28198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c25d9b640 1 -- 172.21.15.103:0/1620378769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c281082a0 msgr2=0x7f5c2819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:21.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c25d9b640 1 --2- 172.21.15.103:0/1620378769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c281082a0 0x7f5c2819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c25d9b640 1 -- 172.21.15.103:0/1620378769 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2819d4c0 con 0x7f5c281036e0 2024-09-19T07:01:21.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c25d9b640 1 --2- 172.21.15.103:0/1620378769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c281036e0 0x7f5c28198a50 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00ea50 tx=0x7f5c1c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:21.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.006+0000 7f5c177fe640 1 -- 172.21.15.103:0/1620378769 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c00cdc0 con 0x7f5c281036e0 2024-09-19T07:01:21.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.007+0000 7f5c2ccd8640 1 -- 172.21.15.103:0/1620378769 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2819d7b0 con 0x7f5c281036e0 2024-09-19T07:01:21.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.007+0000 7f5c177fe640 1 -- 172.21.15.103:0/1620378769 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c1c011950 con 0x7f5c281036e0 2024-09-19T07:01:21.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.007+0000 7f5c177fe640 1 -- 172.21.15.103:0/1620378769 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c01aaf0 con 0x7f5c281036e0 2024-09-19T07:01:21.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.007+0000 7f5c2ccd8640 1 -- 172.21.15.103:0/1620378769 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c281a5090 con 0x7f5c281036e0 2024-09-19T07:01:21.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.009+0000 7f5c177fe640 1 -- 172.21.15.103:0/1620378769 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f5c1c01ac90 con 0x7f5c281036e0 2024-09-19T07:01:21.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.009+0000 7f5c2f0b1740 1 Processor -- start 2024-09-19T07:01:21.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.009+0000 7f5c2f0b1740 1 -- start start 2024-09-19T07:01:21.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c2f0b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b6b40 0x55c8444b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c2f0b1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444b55d0 0x55c8444b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c2f0b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444c0370 0x55c8444c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c2f0b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c84441b2f0 con 0x55c8444c0370 2024-09-19T07:01:21.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c2f0b1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c8443c2cf0 con 0x55c8444b55d0 2024-09-19T07:01:21.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c2f0b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c8444172b0 con 0x55c8444b6b40 2024-09-19T07:01:21.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c177fe640 1 --2- 172.21.15.103:0/1620378769 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5c100a33b0 0x7f5c100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c25d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444c0370 0x55c8444c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c26d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b6b40 0x55c8444b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c25d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444c0370 0x55c8444c2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46418/0 (socket says 172.21.15.103:46418) 2024-09-19T07:01:21.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c25d9b640 1 -- 172.21.15.103:0/2851209 learned_addr learned my addr 172.21.15.103:0/2851209 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:21.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c26d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b6b40 0x55c8444b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:52596/0 (socket says 172.21.15.103:52596) 2024-09-19T07:01:21.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c2659c640 1 --2- 172.21.15.103:0/2851209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444b55d0 0x55c8444b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c26d9d640 1 --2- 172.21.15.103:0/1620378769 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5c100a33b0 0x7f5c100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c177fe640 1 -- 172.21.15.103:0/1620378769 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f5c1c0d0260 con 0x7f5c281036e0 2024-09-19T07:01:21.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c25d9b640 1 -- 172.21.15.103:0/2851209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b6b40 msgr2=0x55c8444b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:21.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c25d9b640 1 --2- 172.21.15.103:0/2851209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b6b40 0x55c8444b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c25d9b640 1 -- 172.21.15.103:0/2851209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444b55d0 msgr2=0x55c8444b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:21.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c25d9b640 1 --2- 172.21.15.103:0/2851209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444b55d0 0x55c8444b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c25d9b640 1 -- 172.21.15.103:0/2851209 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8443a73a0 con 0x55c8444c0370 2024-09-19T07:01:21.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.010+0000 7f5c25d9b640 1 --2- 172.21.15.103:0/2851209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444c0370 0x55c8444c2760 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c098ef0 tx=0x7f5c1c011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ffeb53af686ac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:21.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.011+0000 7f5c26d9d640 1 --2- 172.21.15.103:0/1620378769 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5c100a33b0 0x7f5c100a5870 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5c18005ec0 tx=0x7f5c18005fd0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:21.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.011+0000 7f5c14ff9640 1 -- 172.21.15.103:0/2851209 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c01a400 con 0x55c8444c0370 2024-09-19T07:01:21.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.011+0000 7f5c14ff9640 1 -- 172.21.15.103:0/2851209 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c1c09cc20 con 0x55c8444c0370 2024-09-19T07:01:21.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.011+0000 7f5c14ff9640 1 -- 172.21.15.103:0/2851209 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c098b90 con 0x55c8444c0370 2024-09-19T07:01:21.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.011+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2851209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444c0370 msgr2=0x55c8444c2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:21.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.011+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/2851209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444c0370 0x55c8444c2760 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c098ef0 tx=0x7f5c1c011f60 comp rx=0 tx=0).stop 2024-09-19T07:01:21.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.011+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2851209 shutdown_connections 2024-09-19T07:01:21.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.011+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/2851209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444c0370 0x55c8444c2760 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.011+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/2851209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444b55d0 0x55c8444b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.011+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/2851209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b6b40 0x55c8444b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.011+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2851209 >> 172.21.15.103:0/2851209 conn(0x55c84447e310 msgr2=0x55c8444b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:21.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.012+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2851209 shutdown_connections 2024-09-19T07:01:21.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.012+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2851209 wait complete. 2024-09-19T07:01:21.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.012+0000 7f5c2f0b1740 1 -- start start 2024-09-19T07:01:21.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.012+0000 7f5c2f0b1740 1 Processor -- start 2024-09-19T07:01:21.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.012+0000 7f5c2f0b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b55d0 0x55c8444bfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.012+0000 7f5c2f0b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b6b40 0x55c8444bd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.012+0000 7f5c2f0b1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444c0370 0x55c8444bd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.012+0000 7f5c2f0b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c8444c3070 con 0x55c8444b6b40 2024-09-19T07:01:21.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.012+0000 7f5c2f0b1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c8443d6060 con 0x55c8444c0370 2024-09-19T07:01:21.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.012+0000 7f5c2f0b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c8444c31f0 con 0x55c8444b55d0 2024-09-19T07:01:21.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.012+0000 7f5c2f0b1740 10 client.0 _ll_register_callbacks cb 0x55c8441298c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:21.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.012+0000 7f5c25d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b6b40 0x55c8444bd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.012+0000 7f5c25d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b6b40 0x55c8444bd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46430/0 (socket says 172.21.15.103:46430) 2024-09-19T07:01:21.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.012+0000 7f5c25d9b640 1 -- 172.21.15.103:0/4165012454 learned_addr learned my addr 172.21.15.103:0/4165012454 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:21.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c2659c640 1 --2- 172.21.15.103:0/4165012454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b55d0 0x55c8444bfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c26d9d640 1 --2- 172.21.15.103:0/4165012454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444c0370 0x55c8444bd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c25d9b640 1 -- 172.21.15.103:0/4165012454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b55d0 msgr2=0x55c8444bfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:21.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c25d9b640 1 --2- 172.21.15.103:0/4165012454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b55d0 0x55c8444bfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c25d9b640 1 -- 172.21.15.103:0/4165012454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444c0370 msgr2=0x55c8444bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:21.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c25d9b640 1 --2- 172.21.15.103:0/4165012454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444c0370 0x55c8444bd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c25d9b640 1 -- 172.21.15.103:0/4165012454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c84418c3c0 con 0x55c8444b6b40 2024-09-19T07:01:21.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c2659c640 1 --2- 172.21.15.103:0/4165012454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b55d0 0x55c8444bfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:01:21.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c25d9b640 1 --2- 172.21.15.103:0/4165012454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b6b40 0x55c8444bd080 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c09c4b0 tx=0x7f5c1c098dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac75bbd7192a0f3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:21.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:21.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c0dffb640 1 -- 172.21.15.103:0/4165012454 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c01a400 con 0x55c8444b6b40 2024-09-19T07:01:21.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c0dffb640 1 -- 172.21.15.103:0/4165012454 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f5c1c0b51f0 con 0x55c8444b6b40 2024-09-19T07:01:21.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/4165012454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c1c09cc20 con 0x55c8444b6b40 2024-09-19T07:01:21.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c0dffb640 1 -- 172.21.15.103:0/4165012454 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5c1c09c520 con 0x55c8444b6b40 2024-09-19T07:01:21.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c2f0b1740 10 client.19512 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:21.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/4165012454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c8444ceb90 con 0x55c8444b6b40 2024-09-19T07:01:21.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.013+0000 7f5c0dffb640 1 -- 172.21.15.103:0/4165012454 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c1c09cc20 con 0x55c8444b6b40 2024-09-19T07:01:21.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.014+0000 7f5c0dffb640 1 -- 172.21.15.103:0/4165012454 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5c1c022030 con 0x55c8444b6b40 2024-09-19T07:01:21.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.014+0000 7f5c2f0b1740 10 client.19512 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:21.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.014+0000 7f5c2f0b1740 10 client.19512 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:01:21.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.014+0000 7f5c2f0b1740 20 client.19512 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:21.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.014+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/4165012454 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55c8444cfee0 0x55c8444d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.014+0000 7f5c2f0b1740 4 client.19512 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:21.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.014+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/4165012454 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c844428860 con 0x55c8444cfee0 2024-09-19T07:01:21.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.014+0000 7f5c2659c640 1 --2- 172.21.15.103:0/4165012454 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55c8444cfee0 0x55c8444d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.014+0000 7f5c0dffb640 10 client.19512 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:01:21.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.014+0000 7f5c2659c640 1 --2- 172.21.15.103:0/4165012454 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55c8444cfee0 0x55c8444d22d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=acb2236d93a2047c server_cookie=f02d17d4803e696c in_seq=0 out_seq=0 2024-09-19T07:01:21.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.015+0000 7f5c0dffb640 1 -- 172.21.15.103:0/4165012454 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c844428860 con 0x55c8444cfee0 2024-09-19T07:01:21.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.015+0000 7f5c0dffb640 10 client.19512 handle_command_reply: tid=0 2024-09-19T07:01:21.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.015+0000 7f5c2f0b1740 1 client.19512 shutdown 2024-09-19T07:01:21.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.015+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/4165012454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b6b40 msgr2=0x55c8444bd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:21.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.015+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/4165012454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b6b40 0x55c8444bd080 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c09c4b0 tx=0x7f5c1c098dc0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.015+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/4165012454 shutdown_connections 2024-09-19T07:01:21.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.015+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/4165012454 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55c8444cfee0 0x55c8444d22d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.015+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/4165012454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444c0370 0x55c8444bd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.015+0000 7f5c0dffb640 0 client.19512 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:01:21.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.015+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/4165012454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b6b40 0x55c8444bd080 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.015+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/4165012454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b55d0 0x55c8444bfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.015+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/4165012454 >> 172.21.15.103:0/4165012454 conn(0x55c84447e310 msgr2=0x55c8444b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:21.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.015+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/4165012454 shutdown_connections 2024-09-19T07:01:21.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.015+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/4165012454 wait complete. 2024-09-19T07:01:21.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.016+0000 7f5c2f0b1740 20 client.19512 trim_cache size 0 max 16384 2024-09-19T07:01:21.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.028+0000 7f5c2f0b1740 1 Processor -- start 2024-09-19T07:01:21.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.028+0000 7f5c2f0b1740 1 -- start start 2024-09-19T07:01:21.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.028+0000 7f5c2f0b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b55d0 0x55c8444b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.028+0000 7f5c2f0b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b6b40 0x55c8444bc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.028+0000 7f5c2f0b1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444c0370 0x55c8444cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.028+0000 7f5c2f0b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5c281ad970 con 0x55c8444b55d0 2024-09-19T07:01:21.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.028+0000 7f5c2f0b1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c281af4b0 con 0x55c8444c0370 2024-09-19T07:01:21.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.028+0000 7f5c2f0b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c281ae040 con 0x55c8444b6b40 2024-09-19T07:01:21.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.028+0000 7f5c2659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b55d0 0x55c8444b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.028+0000 7f5c2659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b55d0 0x55c8444b6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46446/0 (socket says 172.21.15.103:46446) 2024-09-19T07:01:21.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.028+0000 7f5c2659c640 1 -- 172.21.15.103:0/2542045419 learned_addr learned my addr 172.21.15.103:0/2542045419 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:21.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.029+0000 7f5c26d9d640 1 --2- 172.21.15.103:0/2542045419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444c0370 0x55c8444cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.029+0000 7f5c25d9b640 1 --2- 172.21.15.103:0/2542045419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b6b40 0x55c8444bc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.029+0000 7f5c2659c640 1 -- 172.21.15.103:0/2542045419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b6b40 msgr2=0x55c8444bc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:21.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.029+0000 7f5c2659c640 1 --2- 172.21.15.103:0/2542045419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b6b40 0x55c8444bc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.029+0000 7f5c2659c640 1 -- 172.21.15.103:0/2542045419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444c0370 msgr2=0x55c8444cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:21.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.029+0000 7f5c2659c640 1 --2- 172.21.15.103:0/2542045419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444c0370 0x55c8444cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.029+0000 7f5c2659c640 1 -- 172.21.15.103:0/2542045419 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8444bb5d0 con 0x55c8444b55d0 2024-09-19T07:01:21.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.029+0000 7f5c25d9b640 1 --2- 172.21.15.103:0/2542045419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b6b40 0x55c8444bc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:21.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.029+0000 7f5c2659c640 1 --2- 172.21.15.103:0/2542045419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b55d0 0x55c8444b6300 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f5c20000c00 tx=0x7f5c20005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c25c15a9515d8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:21.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.029+0000 7f5c0ffff640 1 -- 172.21.15.103:0/2542045419 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c20005220 con 0x55c8444b55d0 2024-09-19T07:01:21.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.029+0000 7f5c0ffff640 1 -- 172.21.15.103:0/2542045419 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c20004c80 con 0x55c8444b55d0 2024-09-19T07:01:21.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.029+0000 7f5c0ffff640 1 -- 172.21.15.103:0/2542045419 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c20010830 con 0x55c8444b55d0 2024-09-19T07:01:21.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2542045419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b55d0 msgr2=0x55c8444b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:21.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/2542045419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b55d0 0x55c8444b6300 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f5c20000c00 tx=0x7f5c20005f90 comp rx=0 tx=0).stop 2024-09-19T07:01:21.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2542045419 shutdown_connections 2024-09-19T07:01:21.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/2542045419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444c0370 0x55c8444cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/2542045419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444b6b40 0x55c8444bc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/2542045419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b55d0 0x55c8444b6300 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2542045419 >> 172.21.15.103:0/2542045419 conn(0x55c84447e310 msgr2=0x55c8444cb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2542045419 shutdown_connections 2024-09-19T07:01:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2542045419 wait complete. 2024-09-19T07:01:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 -- start start 2024-09-19T07:01:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 Processor -- start 2024-09-19T07:01:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444b55d0 0x55c8444d57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b6b40 0x55c8444d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444c0370 0x55c8444d3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5c281aede0 con 0x55c8444b6b40 2024-09-19T07:01:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c281ae710 con 0x55c8444b55d0 2024-09-19T07:01:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c281ab970 con 0x55c8444c0370 2024-09-19T07:01:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c2f0b1740 10 client.0 _ll_register_callbacks cb 0x7f5c20009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.030+0000 7f5c25d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b6b40 0x55c8444d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c25d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b6b40 0x55c8444d2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46448/0 (socket says 172.21.15.103:46448) 2024-09-19T07:01:21.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c25d9b640 1 -- 172.21.15.103:0/2892000337 learned_addr learned my addr 172.21.15.103:0/2892000337 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:21.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c26d9d640 1 --2- 172.21.15.103:0/2892000337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444c0370 0x55c8444d3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c2659c640 1 --2- 172.21.15.103:0/2892000337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444b55d0 0x55c8444d57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c25d9b640 1 -- 172.21.15.103:0/2892000337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444c0370 msgr2=0x55c8444d3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:21.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c25d9b640 1 --2- 172.21.15.103:0/2892000337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444c0370 0x55c8444d3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c25d9b640 1 -- 172.21.15.103:0/2892000337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444b55d0 msgr2=0x55c8444d57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:21.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c25d9b640 1 --2- 172.21.15.103:0/2892000337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444b55d0 0x55c8444d57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c25d9b640 1 -- 172.21.15.103:0/2892000337 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8444d35b0 con 0x55c8444b6b40 2024-09-19T07:01:21.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c25d9b640 1 --2- 172.21.15.103:0/2892000337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b6b40 0x55c8444d2b00 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c09b970 tx=0x7f5c1c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d265c893e7131f16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:21.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:21.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c0dffb640 1 -- 172.21.15.103:0/2892000337 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c0987c0 con 0x55c8444b6b40 2024-09-19T07:01:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c0dffb640 1 -- 172.21.15.103:0/2892000337 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f5c1c0b53f0 con 0x55c8444b6b40 2024-09-19T07:01:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2892000337 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c20004c80 con 0x55c8444b6b40 2024-09-19T07:01:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c0dffb640 1 -- 172.21.15.103:0/2892000337 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5c1c01a400 con 0x55c8444b6b40 2024-09-19T07:01:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c2f0b1740 10 client.19518 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2892000337 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c84453a580 con 0x55c8444b6b40 2024-09-19T07:01:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.031+0000 7f5c0dffb640 1 -- 172.21.15.103:0/2892000337 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c1c07d070 con 0x55c8444b6b40 2024-09-19T07:01:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.032+0000 7f5c0dffb640 1 -- 172.21.15.103:0/2892000337 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5c1c022070 con 0x55c8444b6b40 2024-09-19T07:01:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.032+0000 7f5c2f0b1740 10 client.19518 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.032+0000 7f5c2f0b1740 10 client.19518 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:01:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.032+0000 7f5c2f0b1740 20 client.19518 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.032+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/2892000337 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55c8444cfee0 0x55c84453cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.032+0000 7f5c2f0b1740 4 client.19518 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:01:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.032+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2892000337 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c844422d70 con 0x55c8444cfee0 2024-09-19T07:01:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.032+0000 7f5c2659c640 1 --2- 172.21.15.103:0/2892000337 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55c8444cfee0 0x55c84453cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.032+0000 7f5c2659c640 1 --2- 172.21.15.103:0/2892000337 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55c8444cfee0 0x55c84453cac0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d96fd2bf932e5961 server_cookie=24902679d55bf46c in_seq=0 out_seq=0 2024-09-19T07:01:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.032+0000 7f5c0dffb640 10 client.19518 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:01:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.032+0000 7f5c0dffb640 1 -- 172.21.15.103:0/2892000337 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c844422d70 con 0x55c8444cfee0 2024-09-19T07:01:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.032+0000 7f5c0dffb640 10 client.19518 handle_command_reply: tid=0 2024-09-19T07:01:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2f0b1740 1 client.19518 shutdown 2024-09-19T07:01:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2892000337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b6b40 msgr2=0x55c8444d2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/2892000337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b6b40 0x55c8444d2b00 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c09b970 tx=0x7f5c1c009220 comp rx=0 tx=0).stop 2024-09-19T07:01:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2892000337 shutdown_connections 2024-09-19T07:01:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/2892000337 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55c8444cfee0 0x55c84453cac0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/2892000337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8444c0370 0x55c8444d3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/2892000337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8444b6b40 0x55c8444d2b00 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c0dffb640 0 client.19518 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:01:21.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2f0b1740 1 --2- 172.21.15.103:0/2892000337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8444b55d0 0x55c8444d57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2892000337 >> 172.21.15.103:0/2892000337 conn(0x55c84447e310 msgr2=0x55c8444bf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:21.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2892000337 shutdown_connections 2024-09-19T07:01:21.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2f0b1740 1 -- 172.21.15.103:0/2892000337 wait complete. 2024-09-19T07:01:21.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2f0b1740 20 client.19518 trim_cache size 0 max 16384 2024-09-19T07:01:21.042 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:01:21.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2ccd8640 1 -- 172.21.15.103:0/1620378769 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5c100a33b0 msgr2=0x7f5c100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:21.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2ccd8640 1 --2- 172.21.15.103:0/1620378769 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5c100a33b0 0x7f5c100a5870 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5c18005ec0 tx=0x7f5c18005fd0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2ccd8640 1 -- 172.21.15.103:0/1620378769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c281036e0 msgr2=0x7f5c28198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:21.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2ccd8640 1 --2- 172.21.15.103:0/1620378769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c281036e0 0x7f5c28198a50 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00ea50 tx=0x7f5c1c00ef10 comp rx=0 tx=0).stop 2024-09-19T07:01:21.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2ccd8640 1 -- 172.21.15.103:0/1620378769 shutdown_connections 2024-09-19T07:01:21.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2ccd8640 1 --2- 172.21.15.103:0/1620378769 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f5c100a33b0 0x7f5c100a5870 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2ccd8640 1 --2- 172.21.15.103:0/1620378769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c281082a0 0x7f5c2819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2ccd8640 1 --2- 172.21.15.103:0/1620378769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c281036e0 0x7f5c28198a50 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2ccd8640 1 --2- 172.21.15.103:0/1620378769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c28102d30 0x7f5c28198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:21.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2ccd8640 1 -- 172.21.15.103:0/1620378769 >> 172.21.15.103:0/1620378769 conn(0x7f5c280fc7d0 msgr2=0x7f5c28105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:21.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.033+0000 7f5c2ccd8640 1 -- 172.21.15.103:0/1620378769 shutdown_connections 2024-09-19T07:01:21.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:21.034+0000 7f5c2ccd8640 1 -- 172.21.15.103:0/1620378769 wait complete. 2024-09-19T07:01:21.799 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:01:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:22 smithi174 ceph-mon[35637]: pgmap v378: 97 pgs: 97 active+clean; 5.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-09-19T07:01:22.373 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:22 smithi103 ceph-mon[31158]: pgmap v378: 97 pgs: 97 active+clean; 5.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-09-19T07:01:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:22 smithi134 ceph-mon[35356]: pgmap v378: 97 pgs: 97 active+clean; 5.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-09-19T07:01:22.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.453+0000 7f96088da640 1 -- 172.21.15.103:0/3812220798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9604106a80 msgr2=0x7f9604106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:22.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.453+0000 7f96088da640 1 --2- 172.21.15.103:0/3812220798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9604106a80 0x7f9604106e60 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009930 tx=0x7f95f402a420 comp rx=0 tx=0).stop 2024-09-19T07:01:22.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.453+0000 7f96088da640 1 -- 172.21.15.103:0/3812220798 shutdown_connections 2024-09-19T07:01:22.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.453+0000 7f96088da640 1 --2- 172.21.15.103:0/3812220798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9604101420 0x7f960410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.453+0000 7f96088da640 1 --2- 172.21.15.103:0/3812220798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9604100a80 0x7f9604100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.453+0000 7f96088da640 1 --2- 172.21.15.103:0/3812220798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9604106a80 0x7f9604106e60 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.453+0000 7f96088da640 1 -- 172.21.15.103:0/3812220798 >> 172.21.15.103:0/3812220798 conn(0x7f96040fc7a0 msgr2=0x7f96040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:22.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.453+0000 7f96088da640 1 -- 172.21.15.103:0/3812220798 shutdown_connections 2024-09-19T07:01:22.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.453+0000 7f96088da640 1 -- 172.21.15.103:0/3812220798 wait complete. 2024-09-19T07:01:22.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.454+0000 7f96088da640 1 Processor -- start 2024-09-19T07:01:22.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.454+0000 7f96088da640 1 -- start start 2024-09-19T07:01:22.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.454+0000 7f96088da640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9604100a80 0x7f9604199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.454+0000 7f96088da640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9604101420 0x7f9604199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.454+0000 7f96088da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9604106a80 0x7f96041a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.454+0000 7f96088da640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9604110ea0 con 0x7f9604101420 2024-09-19T07:01:22.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.454+0000 7f96088da640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9604110d20 con 0x7f9604106a80 2024-09-19T07:01:22.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.454+0000 7f96088da640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9604111020 con 0x7f9604100a80 2024-09-19T07:01:22.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.454+0000 7f9601d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9604101420 0x7f9604199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.454+0000 7f960259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9604100a80 0x7f9604199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.454+0000 7f960259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9604100a80 0x7f9604199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:52648/0 (socket says 172.21.15.103:52648) 2024-09-19T07:01:22.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.454+0000 7f960259c640 1 -- 172.21.15.103:0/2853952925 learned_addr learned my addr 172.21.15.103:0/2853952925 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:22.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.454+0000 7f9601d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9604101420 0x7f9604199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46472/0 (socket says 172.21.15.103:46472) 2024-09-19T07:01:22.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.455+0000 7f9602d9d640 1 --2- 172.21.15.103:0/2853952925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9604106a80 0x7f96041a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.455+0000 7f9601d9b640 1 -- 172.21.15.103:0/2853952925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9604100a80 msgr2=0x7f9604199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:22.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.455+0000 7f9601d9b640 1 --2- 172.21.15.103:0/2853952925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9604100a80 0x7f9604199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.455+0000 7f9601d9b640 1 -- 172.21.15.103:0/2853952925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9604106a80 msgr2=0x7f96041a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:22.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.455+0000 7f9601d9b640 1 --2- 172.21.15.103:0/2853952925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9604106a80 0x7f96041a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.455+0000 7f9601d9b640 1 -- 172.21.15.103:0/2853952925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96041a48d0 con 0x7f9604101420 2024-09-19T07:01:22.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.455+0000 7f9601d9b640 1 --2- 172.21.15.103:0/2853952925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9604101420 0x7f9604199850 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00c970 tx=0x7f95ec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:22.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.455+0000 7f95fb7fe640 1 -- 172.21.15.103:0/2853952925 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec007bb0 con 0x7f9604101420 2024-09-19T07:01:22.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.455+0000 7f95fb7fe640 1 -- 172.21.15.103:0/2853952925 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ec007d50 con 0x7f9604101420 2024-09-19T07:01:22.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.455+0000 7f96088da640 1 -- 172.21.15.103:0/2853952925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96041a4aa0 con 0x7f9604101420 2024-09-19T07:01:22.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.455+0000 7f95fb7fe640 1 -- 172.21.15.103:0/2853952925 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec00f450 con 0x7f9604101420 2024-09-19T07:01:22.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.455+0000 7f96088da640 1 -- 172.21.15.103:0/2853952925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96041a4fb0 con 0x7f9604101420 2024-09-19T07:01:22.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.457+0000 7f95fb7fe640 1 -- 172.21.15.103:0/2853952925 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f95ec004ef0 con 0x7f9604101420 2024-09-19T07:01:22.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.458+0000 7f960acb3740 1 Processor -- start 2024-09-19T07:01:22.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.458+0000 7f960acb3740 1 -- start start 2024-09-19T07:01:22.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.458+0000 7f960acb3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090bb40 0x55d95090a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.458+0000 7f960acb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090a5d0 0x55d95090aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.458+0000 7f960acb3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d950915370 0x55d950917760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.458+0000 7f960acb3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d9508702f0 con 0x55d95090bb40 2024-09-19T07:01:22.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.458+0000 7f960acb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d950817cf0 con 0x55d95090a5d0 2024-09-19T07:01:22.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.458+0000 7f960acb3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d95086c2b0 con 0x55d950915370 2024-09-19T07:01:22.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.458+0000 7f9602d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090bb40 0x55d95090a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.458+0000 7f9602d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090bb40 0x55d95090a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46482/0 (socket says 172.21.15.103:46482) 2024-09-19T07:01:22.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.458+0000 7f9602d9d640 1 -- 172.21.15.103:0/2272755204 learned_addr learned my addr 172.21.15.103:0/2272755204 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:22.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.458+0000 7f95fb7fe640 1 --2- 172.21.15.103:0/2853952925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f95f00a33b0 0x7f95f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.458+0000 7f960259c640 1 --2- 172.21.15.103:0/2272755204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090a5d0 0x55d95090aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.459+0000 7f95fb7fe640 1 -- 172.21.15.103:0/2853952925 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f95ec0cf930 con 0x7f9604101420 2024-09-19T07:01:22.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.459+0000 7f9601d9b640 1 --2- 172.21.15.103:0/2272755204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d950915370 0x55d950917760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.459+0000 7f9602d9d640 1 -- 172.21.15.103:0/2272755204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d950915370 msgr2=0x55d950917760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:22.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.459+0000 7f9602d9d640 1 --2- 172.21.15.103:0/2272755204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d950915370 0x55d950917760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.459+0000 7f9602d9d640 1 -- 172.21.15.103:0/2272755204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090a5d0 msgr2=0x55d95090aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:22.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.459+0000 7f9602d9d640 1 --2- 172.21.15.103:0/2272755204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090a5d0 0x55d95090aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.459+0000 7f9602d9d640 1 -- 172.21.15.103:0/2272755204 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9507fc3a0 con 0x55d95090bb40 2024-09-19T07:01:22.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.459+0000 7f9602d9d640 1 --2- 172.21.15.103:0/2853952925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f95f00a33b0 0x7f95f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.459+0000 7f9602d9d640 1 --2- 172.21.15.103:0/2272755204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090bb40 0x55d95090a090 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00a500 tx=0x7f95fc00a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f3ff125210fdc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:22.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.459+0000 7f95f8ff9640 1 -- 172.21.15.103:0/2272755204 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc01a040 con 0x55d95090bb40 2024-09-19T07:01:22.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.459+0000 7f95f8ff9640 1 -- 172.21.15.103:0/2272755204 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95fc021ba0 con 0x55d95090bb40 2024-09-19T07:01:22.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.459+0000 7f95f8ff9640 1 -- 172.21.15.103:0/2272755204 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0203b0 con 0x55d95090bb40 2024-09-19T07:01:22.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.460+0000 7f960acb3740 1 -- 172.21.15.103:0/2272755204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090bb40 msgr2=0x55d95090a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:22.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.460+0000 7f9602d9d640 1 --2- 172.21.15.103:0/2853952925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f95f00a33b0 0x7f95f00a5870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f95fc020000 tx=0x7f95fc021000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:22.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.460+0000 7f960acb3740 1 --2- 172.21.15.103:0/2272755204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090bb40 0x55d95090a090 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00a500 tx=0x7f95fc00a9c0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.460+0000 7f960acb3740 1 -- 172.21.15.103:0/2272755204 shutdown_connections 2024-09-19T07:01:22.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.460+0000 7f960acb3740 1 --2- 172.21.15.103:0/2272755204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d950915370 0x55d950917760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.460+0000 7f960acb3740 1 --2- 172.21.15.103:0/2272755204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090a5d0 0x55d95090aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.460+0000 7f960acb3740 1 --2- 172.21.15.103:0/2272755204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090bb40 0x55d95090a090 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.460+0000 7f960acb3740 1 -- 172.21.15.103:0/2272755204 >> 172.21.15.103:0/2272755204 conn(0x55d9508d3310 msgr2=0x55d950906c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:22.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.460+0000 7f960acb3740 1 -- 172.21.15.103:0/2272755204 shutdown_connections 2024-09-19T07:01:22.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.460+0000 7f960acb3740 1 -- 172.21.15.103:0/2272755204 wait complete. 2024-09-19T07:01:22.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.460+0000 7f960acb3740 1 -- start start 2024-09-19T07:01:22.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.460+0000 7f960acb3740 1 Processor -- start 2024-09-19T07:01:22.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f960acb3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d95090a5d0 0x55d950914d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f960acb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090bb40 0x55d950912020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f960acb3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d950915370 0x55d950912610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f960acb3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d950918070 con 0x55d950915370 2024-09-19T07:01:22.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f960acb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d95082b060 con 0x55d95090bb40 2024-09-19T07:01:22.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f960acb3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d9509181f0 con 0x55d95090a5d0 2024-09-19T07:01:22.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f960acb3740 10 client.0 _ll_register_callbacks cb 0x55d95057e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:22.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f9602d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d950915370 0x55d950912610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f9602d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d950915370 0x55d950912610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46498/0 (socket says 172.21.15.103:46498) 2024-09-19T07:01:22.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f9602d9d640 1 -- 172.21.15.103:0/1407088279 learned_addr learned my addr 172.21.15.103:0/1407088279 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:22.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f960259c640 1 --2- 172.21.15.103:0/1407088279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d95090a5d0 0x55d950914d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f9601d9b640 1 --2- 172.21.15.103:0/1407088279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090bb40 0x55d950912020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f9602d9d640 1 -- 172.21.15.103:0/1407088279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d95090a5d0 msgr2=0x55d950914d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:22.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f9602d9d640 1 --2- 172.21.15.103:0/1407088279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d95090a5d0 0x55d950914d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f9602d9d640 1 -- 172.21.15.103:0/1407088279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090bb40 msgr2=0x55d950912020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:22.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f9602d9d640 1 --2- 172.21.15.103:0/1407088279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090bb40 0x55d950912020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f9602d9d640 1 -- 172.21.15.103:0/1407088279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9505e13c0 con 0x55d950915370 2024-09-19T07:01:22.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f9601d9b640 1 --2- 172.21.15.103:0/1407088279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090bb40 0x55d950912020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:22.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f960259c640 1 --2- 172.21.15.103:0/1407088279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d95090a5d0 0x55d950914d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:22.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f9602d9d640 1 --2- 172.21.15.103:0/1407088279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d950915370 0x55d950912610 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009900 tx=0x7f95fc009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edeb9f0e58873fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:22.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.461+0000 7f95e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:22.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.462+0000 7f95e9ffb640 1 -- 172.21.15.103:0/1407088279 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc02d070 con 0x55d950915370 2024-09-19T07:01:22.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.462+0000 7f95e9ffb640 1 -- 172.21.15.103:0/1407088279 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f95fc021920 con 0x55d950915370 2024-09-19T07:01:22.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.462+0000 7f960acb3740 1 -- 172.21.15.103:0/1407088279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95fc021ba0 con 0x55d950915370 2024-09-19T07:01:22.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.462+0000 7f95e9ffb640 1 -- 172.21.15.103:0/1407088279 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f95fc039050 con 0x55d950915370 2024-09-19T07:01:22.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.462+0000 7f960acb3740 10 client.19530 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:22.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.462+0000 7f960acb3740 1 -- 172.21.15.103:0/1407088279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d950923b10 con 0x55d950915370 2024-09-19T07:01:22.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.462+0000 7f95e9ffb640 1 -- 172.21.15.103:0/1407088279 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95fc021ba0 con 0x55d950915370 2024-09-19T07:01:22.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.462+0000 7f95e9ffb640 1 -- 172.21.15.103:0/1407088279 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f95fc028040 con 0x55d950915370 2024-09-19T07:01:22.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.462+0000 7f960acb3740 10 client.19530 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:22.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.462+0000 7f960acb3740 10 client.19530 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:01:22.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.462+0000 7f960acb3740 20 client.19530 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:22.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.462+0000 7f960acb3740 1 --2- 172.21.15.103:0/1407088279 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55d950924ec0 0x55d9509272b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.462+0000 7f960acb3740 4 client.19530 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:22.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.462+0000 7f960acb3740 1 -- 172.21.15.103:0/1407088279 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d95087d860 con 0x55d950924ec0 2024-09-19T07:01:22.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.463+0000 7f960259c640 1 --2- 172.21.15.103:0/1407088279 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55d950924ec0 0x55d9509272b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.463+0000 7f95e9ffb640 10 client.19530 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:01:22.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.463+0000 7f960259c640 1 --2- 172.21.15.103:0/1407088279 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55d950924ec0 0x55d9509272b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=584d7158a764afe7 server_cookie=681c28e38111ef4c in_seq=0 out_seq=0 2024-09-19T07:01:22.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.464+0000 7f95e9ffb640 1 -- 172.21.15.103:0/1407088279 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d95087d860 con 0x55d950924ec0 2024-09-19T07:01:22.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.464+0000 7f95e9ffb640 10 client.19530 handle_command_reply: tid=0 2024-09-19T07:01:22.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.464+0000 7f960acb3740 1 client.19530 shutdown 2024-09-19T07:01:22.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.464+0000 7f960acb3740 1 -- 172.21.15.103:0/1407088279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d950915370 msgr2=0x55d950912610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:22.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.464+0000 7f960acb3740 1 --2- 172.21.15.103:0/1407088279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d950915370 0x55d950912610 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009900 tx=0x7f95fc009db0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.464+0000 7f960acb3740 1 -- 172.21.15.103:0/1407088279 shutdown_connections 2024-09-19T07:01:22.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.464+0000 7f960acb3740 1 --2- 172.21.15.103:0/1407088279 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55d950924ec0 0x55d9509272b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.464+0000 7f960acb3740 1 --2- 172.21.15.103:0/1407088279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d950915370 0x55d950912610 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.464+0000 7f95e9ffb640 0 client.19530 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:01:22.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.464+0000 7f960acb3740 1 --2- 172.21.15.103:0/1407088279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090bb40 0x55d950912020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.464+0000 7f960acb3740 1 --2- 172.21.15.103:0/1407088279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d95090a5d0 0x55d950914d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.464+0000 7f960acb3740 1 -- 172.21.15.103:0/1407088279 >> 172.21.15.103:0/1407088279 conn(0x55d9508d3310 msgr2=0x55d95090c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:22.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.464+0000 7f960acb3740 1 -- 172.21.15.103:0/1407088279 shutdown_connections 2024-09-19T07:01:22.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.464+0000 7f960acb3740 1 -- 172.21.15.103:0/1407088279 wait complete. 2024-09-19T07:01:22.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.464+0000 7f960acb3740 20 client.19530 trim_cache size 0 max 16384 2024-09-19T07:01:22.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.477+0000 7f960acb3740 1 Processor -- start 2024-09-19T07:01:22.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.477+0000 7f960acb3740 1 -- start start 2024-09-19T07:01:22.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.477+0000 7f960acb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090a5d0 0x55d950921310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.477+0000 7f960acb3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090bb40 0x55d95090d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.477+0000 7f960acb3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d950915370 0x55d95090d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.477+0000 7f960acb3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96041ad8f0 con 0x55d95090bb40 2024-09-19T07:01:22.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.477+0000 7f960acb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96041af430 con 0x55d95090a5d0 2024-09-19T07:01:22.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.477+0000 7f960acb3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96041adfc0 con 0x55d950915370 2024-09-19T07:01:22.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.477+0000 7f9601d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090bb40 0x55d95090d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.477+0000 7f9601d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090bb40 0x55d95090d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46502/0 (socket says 172.21.15.103:46502) 2024-09-19T07:01:22.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.477+0000 7f9601d9b640 1 -- 172.21.15.103:0/1681885750 learned_addr learned my addr 172.21.15.103:0/1681885750 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:22.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.477+0000 7f9602d9d640 1 --2- 172.21.15.103:0/1681885750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d950915370 0x55d95090d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.477+0000 7f960259c640 1 --2- 172.21.15.103:0/1681885750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090a5d0 0x55d950921310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.477+0000 7f9601d9b640 1 -- 172.21.15.103:0/1681885750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d950915370 msgr2=0x55d95090d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:22.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.478+0000 7f9601d9b640 1 --2- 172.21.15.103:0/1681885750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d950915370 0x55d95090d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.478+0000 7f9601d9b640 1 -- 172.21.15.103:0/1681885750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090a5d0 msgr2=0x55d950921310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:22.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.478+0000 7f9601d9b640 1 --2- 172.21.15.103:0/1681885750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090a5d0 0x55d950921310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.478+0000 7f9601d9b640 1 -- 172.21.15.103:0/1681885750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d950927840 con 0x55d95090bb40 2024-09-19T07:01:22.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.478+0000 7f960259c640 1 --2- 172.21.15.103:0/1681885750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090a5d0 0x55d950921310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:22.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.478+0000 7f9601d9b640 1 --2- 172.21.15.103:0/1681885750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090bb40 0x55d95090d120 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0cd660 tx=0x7f95ec086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b80fd6294f45c7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:22.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.478+0000 7f95ebfff640 1 -- 172.21.15.103:0/1681885750 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec01d050 con 0x55d95090bb40 2024-09-19T07:01:22.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.478+0000 7f95ebfff640 1 -- 172.21.15.103:0/1681885750 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ec0a4860 con 0x55d95090bb40 2024-09-19T07:01:22.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.478+0000 7f95ebfff640 1 -- 172.21.15.103:0/1681885750 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec0051e0 con 0x55d95090bb40 2024-09-19T07:01:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.478+0000 7f960acb3740 1 -- 172.21.15.103:0/1681885750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090bb40 msgr2=0x55d95090d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.478+0000 7f960acb3740 1 --2- 172.21.15.103:0/1681885750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090bb40 0x55d95090d120 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0cd660 tx=0x7f95ec086040 comp rx=0 tx=0).stop 2024-09-19T07:01:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.478+0000 7f960acb3740 1 -- 172.21.15.103:0/1681885750 shutdown_connections 2024-09-19T07:01:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.478+0000 7f960acb3740 1 --2- 172.21.15.103:0/1681885750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d950915370 0x55d95090d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.478+0000 7f960acb3740 1 --2- 172.21.15.103:0/1681885750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090bb40 0x55d95090d120 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.478+0000 7f960acb3740 1 --2- 172.21.15.103:0/1681885750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d95090a5d0 0x55d950921310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.479+0000 7f960acb3740 1 -- 172.21.15.103:0/1681885750 >> 172.21.15.103:0/1681885750 conn(0x55d9508d3310 msgr2=0x55d950911110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.479+0000 7f960acb3740 1 -- 172.21.15.103:0/1681885750 shutdown_connections 2024-09-19T07:01:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.479+0000 7f960acb3740 1 -- 172.21.15.103:0/1681885750 wait complete. 2024-09-19T07:01:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.479+0000 7f960acb3740 1 -- start start 2024-09-19T07:01:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.479+0000 7f960acb3740 1 Processor -- start 2024-09-19T07:01:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.479+0000 7f960acb3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090a5d0 0x55d95091dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.479+0000 7f960acb3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d95090bb40 0x55d950918f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.479+0000 7f960acb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d950915370 0x55d9509194d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.479+0000 7f960acb3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96041aed60 con 0x55d95090a5d0 2024-09-19T07:01:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.479+0000 7f960acb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96041ae690 con 0x55d950915370 2024-09-19T07:01:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.479+0000 7f960acb3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96041ab8f0 con 0x55d95090bb40 2024-09-19T07:01:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.479+0000 7f960acb3740 10 client.0 _ll_register_callbacks cb 0x7f95f4005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.479+0000 7f960259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090a5d0 0x55d95091dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.479+0000 7f960259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090a5d0 0x55d95091dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46506/0 (socket says 172.21.15.103:46506) 2024-09-19T07:01:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.479+0000 7f960259c640 1 -- 172.21.15.103:0/1577858999 learned_addr learned my addr 172.21.15.103:0/1577858999 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.479+0000 7f9601d9b640 1 --2- 172.21.15.103:0/1577858999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d95090bb40 0x55d950918f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f9602d9d640 1 --2- 172.21.15.103:0/1577858999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d950915370 0x55d9509194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f960259c640 1 -- 172.21.15.103:0/1577858999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d95090bb40 msgr2=0x55d950918f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:22.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f960259c640 1 --2- 172.21.15.103:0/1577858999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d95090bb40 0x55d950918f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f960259c640 1 -- 172.21.15.103:0/1577858999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d950915370 msgr2=0x55d9509194d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:22.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f960259c640 1 --2- 172.21.15.103:0/1577858999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d950915370 0x55d9509194d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f960259c640 1 -- 172.21.15.103:0/1577858999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d950919d00 con 0x55d95090a5d0 2024-09-19T07:01:22.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f960259c640 1 --2- 172.21.15.103:0/1577858999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090a5d0 0x55d95091dc40 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f95f402a460 tx=0x7f95f4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7fb8722c4e4989 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:22.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f95e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:22.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f95e9ffb640 1 -- 172.21.15.103:0/1577858999 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4049070 con 0x55d95090a5d0 2024-09-19T07:01:22.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f95e9ffb640 1 -- 172.21.15.103:0/1577858999 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f95f4005040 con 0x55d95090a5d0 2024-09-19T07:01:22.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f960acb3740 1 -- 172.21.15.103:0/1577858999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95ec0a4860 con 0x55d95090a5d0 2024-09-19T07:01:22.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f95e9ffb640 1 -- 172.21.15.103:0/1577858999 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f95f4055050 con 0x55d95090a5d0 2024-09-19T07:01:22.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f960acb3740 10 client.19536 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:22.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f960acb3740 1 -- 172.21.15.103:0/1577858999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d95098faa0 con 0x55d95090a5d0 2024-09-19T07:01:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f95e9ffb640 1 -- 172.21.15.103:0/1577858999 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ec0a4860 con 0x55d95090a5d0 2024-09-19T07:01:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f95e9ffb640 1 -- 172.21.15.103:0/1577858999 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f95f40035f0 con 0x55d95090a5d0 2024-09-19T07:01:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f960acb3740 10 client.19536 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f960acb3740 10 client.19536 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:01:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f960acb3740 20 client.19536 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f960acb3740 1 --2- 172.21.15.103:0/1577858999 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55d9509901a0 0x55d9509925c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f960acb3740 4 client.19536 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:01:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.480+0000 7f960acb3740 1 -- 172.21.15.103:0/1577858999 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d950877d70 con 0x55d9509901a0 2024-09-19T07:01:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.481+0000 7f9601d9b640 1 --2- 172.21.15.103:0/1577858999 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55d9509901a0 0x55d9509925c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.481+0000 7f95e9ffb640 10 client.19536 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:01:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.481+0000 7f9601d9b640 1 --2- 172.21.15.103:0/1577858999 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55d9509901a0 0x55d9509925c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9f04a83a5970ff11 server_cookie=366db67a60182a0a in_seq=0 out_seq=0 2024-09-19T07:01:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.481+0000 7f95e9ffb640 1 -- 172.21.15.103:0/1577858999 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d950877d70 con 0x55d9509901a0 2024-09-19T07:01:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.481+0000 7f95e9ffb640 10 client.19536 handle_command_reply: tid=0 2024-09-19T07:01:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.481+0000 7f960acb3740 1 client.19536 shutdown 2024-09-19T07:01:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f960acb3740 1 -- 172.21.15.103:0/1577858999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090a5d0 msgr2=0x55d95091dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f960acb3740 1 --2- 172.21.15.103:0/1577858999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090a5d0 0x55d95091dc40 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f95f402a460 tx=0x7f95f4004040 comp rx=0 tx=0).stop 2024-09-19T07:01:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f960acb3740 1 -- 172.21.15.103:0/1577858999 shutdown_connections 2024-09-19T07:01:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f960acb3740 1 --2- 172.21.15.103:0/1577858999 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55d9509901a0 0x55d9509925c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f960acb3740 1 --2- 172.21.15.103:0/1577858999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d950915370 0x55d9509194d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f960acb3740 1 --2- 172.21.15.103:0/1577858999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d95090bb40 0x55d950918f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f95e9ffb640 0 client.19536 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:01:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f960acb3740 1 --2- 172.21.15.103:0/1577858999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d95090a5d0 0x55d95091dc40 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f960acb3740 1 -- 172.21.15.103:0/1577858999 >> 172.21.15.103:0/1577858999 conn(0x55d9508d3310 msgr2=0x55d950913880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f960acb3740 1 -- 172.21.15.103:0/1577858999 shutdown_connections 2024-09-19T07:01:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f960acb3740 1 -- 172.21.15.103:0/1577858999 wait complete. 2024-09-19T07:01:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f960acb3740 20 client.19536 trim_cache size 0 max 16384 2024-09-19T07:01:22.489 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:01:22.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f96088da640 1 -- 172.21.15.103:0/2853952925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f95f00a33b0 msgr2=0x7f95f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:22.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f96088da640 1 --2- 172.21.15.103:0/2853952925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f95f00a33b0 0x7f95f00a5870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f95fc020000 tx=0x7f95fc021000 comp rx=0 tx=0).stop 2024-09-19T07:01:22.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f96088da640 1 -- 172.21.15.103:0/2853952925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9604101420 msgr2=0x7f9604199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:22.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f96088da640 1 --2- 172.21.15.103:0/2853952925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9604101420 0x7f9604199850 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00c970 tx=0x7f95ec00ce30 comp rx=0 tx=0).stop 2024-09-19T07:01:22.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f96088da640 1 -- 172.21.15.103:0/2853952925 shutdown_connections 2024-09-19T07:01:22.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f96088da640 1 --2- 172.21.15.103:0/2853952925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f95f00a33b0 0x7f95f00a5870 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f96088da640 1 --2- 172.21.15.103:0/2853952925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9604106a80 0x7f96041a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f96088da640 1 --2- 172.21.15.103:0/2853952925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9604101420 0x7f9604199850 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f96088da640 1 --2- 172.21.15.103:0/2853952925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9604100a80 0x7f9604199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:22.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f96088da640 1 -- 172.21.15.103:0/2853952925 >> 172.21.15.103:0/2853952925 conn(0x7f96040fc7a0 msgr2=0x7f96040fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:22.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.482+0000 7f96088da640 1 -- 172.21.15.103:0/2853952925 shutdown_connections 2024-09-19T07:01:22.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:22.483+0000 7f96088da640 1 -- 172.21.15.103:0/2853952925 wait complete. 2024-09-19T07:01:23.097 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:01:23.343 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:23.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.719+0000 7f0b88bfe640 1 -- 172.21.15.103:0/391312236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b84106f50 msgr2=0x7f0b8410eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:23.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.719+0000 7f0b88bfe640 1 --2- 172.21.15.103:0/391312236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b84106f50 0x7f0b8410eaf0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009cd0 tx=0x7f0b7c02a8e0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.719+0000 7f0b88bfe640 1 -- 172.21.15.103:0/391312236 shutdown_connections 2024-09-19T07:01:23.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.719+0000 7f0b88bfe640 1 --2- 172.21.15.103:0/391312236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b84106f50 0x7f0b8410eaf0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.719+0000 7f0b88bfe640 1 --2- 172.21.15.103:0/391312236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b84100ca0 0x7f0b84106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.719+0000 7f0b88bfe640 1 --2- 172.21.15.103:0/391312236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b841002f0 0x7f0b841006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.719+0000 7f0b88bfe640 1 -- 172.21.15.103:0/391312236 >> 172.21.15.103:0/391312236 conn(0x7f0b84077210 msgr2=0x7f0b840fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:23.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.719+0000 7f0b88bfe640 1 -- 172.21.15.103:0/391312236 shutdown_connections 2024-09-19T07:01:23.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.719+0000 7f0b88bfe640 1 -- 172.21.15.103:0/391312236 wait complete. 2024-09-19T07:01:23.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b88bfe640 1 Processor -- start 2024-09-19T07:01:23.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b88bfe640 1 -- start start 2024-09-19T07:01:23.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b88bfe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b841002f0 0x7f0b841984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b88bfe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b84100ca0 0x7f0b84198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b88bfe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b84106f50 0x7f0b8419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b88bfe640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b84112000 con 0x7f0b84106f50 2024-09-19T07:01:23.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b88bfe640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b84111e80 con 0x7f0b84100ca0 2024-09-19T07:01:23.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b88bfe640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b84112180 con 0x7f0b841002f0 2024-09-19T07:01:23.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b82d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b84106f50 0x7f0b8419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b82d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b84106f50 0x7f0b8419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46526/0 (socket says 172.21.15.103:46526) 2024-09-19T07:01:23.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b82d9d640 1 -- 172.21.15.103:0/1252335953 learned_addr learned my addr 172.21.15.103:0/1252335953 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:23.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/1252335953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b84100ca0 0x7f0b84198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b8259c640 1 --2- 172.21.15.103:0/1252335953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b841002f0 0x7f0b841984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b82d9d640 1 -- 172.21.15.103:0/1252335953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b841002f0 msgr2=0x7f0b841984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:23.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b82d9d640 1 --2- 172.21.15.103:0/1252335953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b841002f0 0x7f0b841984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b82d9d640 1 -- 172.21.15.103:0/1252335953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b84100ca0 msgr2=0x7f0b84198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:23.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b82d9d640 1 --2- 172.21.15.103:0/1252335953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b84100ca0 0x7f0b84198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.720+0000 7f0b82d9d640 1 -- 172.21.15.103:0/1252335953 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8419d470 con 0x7f0b84106f50 2024-09-19T07:01:23.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.721+0000 7f0b82d9d640 1 --2- 172.21.15.103:0/1252335953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b84106f50 0x7f0b8419cd90 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c006010 tx=0x7f0b7c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:23.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.721+0000 7f0b7b7fe640 1 -- 172.21.15.103:0/1252335953 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c004450 con 0x7f0b84106f50 2024-09-19T07:01:23.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.721+0000 7f0b7b7fe640 1 -- 172.21.15.103:0/1252335953 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b7c003e10 con 0x7f0b84106f50 2024-09-19T07:01:23.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.721+0000 7f0b7b7fe640 1 -- 172.21.15.103:0/1252335953 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c046480 con 0x7f0b84106f50 2024-09-19T07:01:23.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.721+0000 7f0b88bfe640 1 -- 172.21.15.103:0/1252335953 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8419d700 con 0x7f0b84106f50 2024-09-19T07:01:23.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.721+0000 7f0b88bfe640 1 -- 172.21.15.103:0/1252335953 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b841a5040 con 0x7f0b84106f50 2024-09-19T07:01:23.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.723+0000 7f0b7b7fe640 1 -- 172.21.15.103:0/1252335953 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0b7c0045f0 con 0x7f0b84106f50 2024-09-19T07:01:23.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.723+0000 7f0b8afd7740 1 Processor -- start 2024-09-19T07:01:23.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b8afd7740 1 -- start start 2024-09-19T07:01:23.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b8afd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93db40 0x56191a93c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b8afd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a93c5d0 0x56191a93ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b8afd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a947370 0x56191a949760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b8afd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56191a8a22f0 con 0x56191a93db40 2024-09-19T07:01:23.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b8afd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56191a849cf0 con 0x56191a947370 2024-09-19T07:01:23.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b8afd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56191a89e2b0 con 0x56191a93c5d0 2024-09-19T07:01:23.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93db40 0x56191a93c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93db40 0x56191a93c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46532/0 (socket says 172.21.15.103:46532) 2024-09-19T07:01:23.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b7bfff640 1 -- 172.21.15.103:0/2775716166 learned_addr learned my addr 172.21.15.103:0/2775716166 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:23.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/1252335953 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b700a33b0 0x7f0b700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b8259c640 1 --2- 172.21.15.103:0/2775716166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a93c5d0 0x56191a93ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/2775716166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a947370 0x56191a949760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b82d9d640 1 --2- 172.21.15.103:0/1252335953 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b700a33b0 0x7f0b700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b7bfff640 1 -- 172.21.15.103:0/2775716166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a93c5d0 msgr2=0x56191a93ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:23.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/2775716166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a93c5d0 0x56191a93ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b7bfff640 1 -- 172.21.15.103:0/2775716166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a947370 msgr2=0x56191a949760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:23.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/2775716166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a947370 0x56191a949760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b7bfff640 1 -- 172.21.15.103:0/2775716166 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56191a82e3a0 con 0x56191a93db40 2024-09-19T07:01:23.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.724+0000 7f0b7b7fe640 1 -- 172.21.15.103:0/1252335953 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f0b7c02e030 con 0x7f0b84106f50 2024-09-19T07:01:23.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.725+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/2775716166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93db40 0x56191a93c090 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00afd0 tx=0x7f0b6c00a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f8eb7862741d38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:23.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.725+0000 7f0b82d9d640 1 --2- 172.21.15.103:0/1252335953 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b700a33b0 0x7f0b700a5870 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0bd610 tx=0x7f0b7c0b99a0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:23.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.725+0000 7f0b78ff9640 1 -- 172.21.15.103:0/2775716166 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c019030 con 0x56191a93db40 2024-09-19T07:01:23.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.725+0000 7f0b78ff9640 1 -- 172.21.15.103:0/2775716166 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b6c005500 con 0x56191a93db40 2024-09-19T07:01:23.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.725+0000 7f0b78ff9640 1 -- 172.21.15.103:0/2775716166 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c004d90 con 0x56191a93db40 2024-09-19T07:01:23.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 -- 172.21.15.103:0/2775716166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93db40 msgr2=0x56191a93c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:23.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/2775716166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93db40 0x56191a93c090 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00afd0 tx=0x7f0b6c00a720 comp rx=0 tx=0).stop 2024-09-19T07:01:23.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 -- 172.21.15.103:0/2775716166 shutdown_connections 2024-09-19T07:01:23.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/2775716166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a947370 0x56191a949760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/2775716166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a93c5d0 0x56191a93ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/2775716166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93db40 0x56191a93c090 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 -- 172.21.15.103:0/2775716166 >> 172.21.15.103:0/2775716166 conn(0x56191a905310 msgr2=0x56191a938c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:23.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 -- 172.21.15.103:0/2775716166 shutdown_connections 2024-09-19T07:01:23.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 -- 172.21.15.103:0/2775716166 wait complete. 2024-09-19T07:01:23.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 -- start start 2024-09-19T07:01:23.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 Processor -- start 2024-09-19T07:01:23.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a93c5d0 0x56191a946cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93db40 0x56191a943f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a947370 0x56191a9444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56191a94a070 con 0x56191a93db40 2024-09-19T07:01:23.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56191a85d060 con 0x56191a93c5d0 2024-09-19T07:01:23.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.726+0000 7f0b8afd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56191a94a1f0 con 0x56191a947370 2024-09-19T07:01:23.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b8afd7740 10 client.0 _ll_register_callbacks cb 0x56191a5b08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:23.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b8259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93db40 0x56191a943f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b8259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93db40 0x56191a943f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46540/0 (socket says 172.21.15.103:46540) 2024-09-19T07:01:23.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b8259c640 1 -- 172.21.15.103:0/2907516387 learned_addr learned my addr 172.21.15.103:0/2907516387 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:23.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/2907516387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a93c5d0 0x56191a946cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/2907516387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a947370 0x56191a9444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b8259c640 1 -- 172.21.15.103:0/2907516387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a947370 msgr2=0x56191a9444d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:23.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b8259c640 1 --2- 172.21.15.103:0/2907516387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a947370 0x56191a9444d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b8259c640 1 -- 172.21.15.103:0/2907516387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a93c5d0 msgr2=0x56191a946cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:23.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b8259c640 1 --2- 172.21.15.103:0/2907516387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a93c5d0 0x56191a946cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b8259c640 1 -- 172.21.15.103:0/2907516387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56191a6133c0 con 0x56191a93db40 2024-09-19T07:01:23.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/2907516387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a947370 0x56191a9444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:23.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b8259c640 1 --2- 172.21.15.103:0/2907516387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93db40 0x56191a943f70 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009ee0 tx=0x7f0b74011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64aa3a1c352af8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:23.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:23.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b69ffb640 1 -- 172.21.15.103:0/2907516387 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74019070 con 0x56191a93db40 2024-09-19T07:01:23.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b69ffb640 1 -- 172.21.15.103:0/2907516387 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f0b74015660 con 0x56191a93db40 2024-09-19T07:01:23.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b8afd7740 1 -- 172.21.15.103:0/2907516387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b6c005500 con 0x56191a93db40 2024-09-19T07:01:23.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b69ffb640 1 -- 172.21.15.103:0/2907516387 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0b74025050 con 0x56191a93db40 2024-09-19T07:01:23.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b8afd7740 10 client.19548 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:23.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.727+0000 7f0b8afd7740 1 -- 172.21.15.103:0/2907516387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56191a955ac0 con 0x56191a93db40 2024-09-19T07:01:23.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.728+0000 7f0b69ffb640 1 -- 172.21.15.103:0/2907516387 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b6c005500 con 0x56191a93db40 2024-09-19T07:01:23.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.728+0000 7f0b69ffb640 1 -- 172.21.15.103:0/2907516387 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0b74014b40 con 0x56191a93db40 2024-09-19T07:01:23.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.728+0000 7f0b8afd7740 10 client.19548 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:23.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.728+0000 7f0b8afd7740 10 client.19548 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:01:23.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.728+0000 7f0b8afd7740 20 client.19548 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:23.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.728+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/2907516387 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56191a956f70 0x56191a9593b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.728+0000 7f0b8afd7740 4 client.19548 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:23.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.728+0000 7f0b8afd7740 1 -- 172.21.15.103:0/2907516387 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56191a8af860 con 0x56191a956f70 2024-09-19T07:01:23.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.728+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/2907516387 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56191a956f70 0x56191a9593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.729+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/2907516387 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56191a956f70 0x56191a9593b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1908da46c473471 server_cookie=10df27cc91df7992 in_seq=0 out_seq=0 2024-09-19T07:01:23.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.729+0000 7f0b69ffb640 10 client.19548 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:01:23.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.729+0000 7f0b69ffb640 1 -- 172.21.15.103:0/2907516387 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56191a8af860 con 0x56191a956f70 2024-09-19T07:01:23.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.729+0000 7f0b69ffb640 10 client.19548 handle_command_reply: tid=0 2024-09-19T07:01:23.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.729+0000 7f0b8afd7740 1 client.19548 shutdown 2024-09-19T07:01:23.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.729+0000 7f0b8afd7740 1 -- 172.21.15.103:0/2907516387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93db40 msgr2=0x56191a943f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:23.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.729+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/2907516387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93db40 0x56191a943f70 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009ee0 tx=0x7f0b74011630 comp rx=0 tx=0).stop 2024-09-19T07:01:23.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.729+0000 7f0b8afd7740 1 -- 172.21.15.103:0/2907516387 shutdown_connections 2024-09-19T07:01:23.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.730+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/2907516387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a947370 0x56191a9444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.730+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/2907516387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93db40 0x56191a943f70 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.730+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/2907516387 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56191a956f70 0x56191a9593b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.730+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/2907516387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a93c5d0 0x56191a946cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.730+0000 7f0b8afd7740 1 -- 172.21.15.103:0/2907516387 >> 172.21.15.103:0/2907516387 conn(0x56191a905310 msgr2=0x56191a93e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:23.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.730+0000 7f0b69ffb640 0 client.19548 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:01:23.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.730+0000 7f0b8afd7740 1 -- 172.21.15.103:0/2907516387 shutdown_connections 2024-09-19T07:01:23.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.730+0000 7f0b8afd7740 1 -- 172.21.15.103:0/2907516387 wait complete. 2024-09-19T07:01:23.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.730+0000 7f0b8afd7740 20 client.19548 trim_cache size 0 max 16384 2024-09-19T07:01:23.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.742+0000 7f0b8afd7740 1 Processor -- start 2024-09-19T07:01:23.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.742+0000 7f0b8afd7740 1 -- start start 2024-09-19T07:01:23.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b8afd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a93c5d0 0x56191a953310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b8afd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a93db40 0x56191a93f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b8afd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a947370 0x56191a93f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b8afd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841ad890 con 0x56191a947370 2024-09-19T07:01:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b8afd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841af3d0 con 0x56191a93db40 2024-09-19T07:01:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b8afd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841adf60 con 0x56191a93c5d0 2024-09-19T07:01:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a947370 0x56191a93f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a947370 0x56191a93f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46548/0 (socket says 172.21.15.103:46548) 2024-09-19T07:01:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b7bfff640 1 -- 172.21.15.103:0/256963503 learned_addr learned my addr 172.21.15.103:0/256963503 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b8259c640 1 --2- 172.21.15.103:0/256963503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a93db40 0x56191a93f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/256963503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a93c5d0 0x56191a953310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b7bfff640 1 -- 172.21.15.103:0/256963503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a93c5d0 msgr2=0x56191a953310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/256963503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a93c5d0 0x56191a953310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b7bfff640 1 -- 172.21.15.103:0/256963503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a93db40 msgr2=0x56191a93f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/256963503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a93db40 0x56191a93f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b7bfff640 1 -- 172.21.15.103:0/256963503 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56191a939960 con 0x56191a947370 2024-09-19T07:01:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/256963503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a947370 0x56191a93f660 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0048a0 tx=0x7f0b6c004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e892a3e0449b39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:23.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b6bfff640 1 -- 172.21.15.103:0/256963503 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c019040 con 0x56191a947370 2024-09-19T07:01:23.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b6bfff640 1 -- 172.21.15.103:0/256963503 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b6c0110d0 con 0x56191a947370 2024-09-19T07:01:23.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.743+0000 7f0b6bfff640 1 -- 172.21.15.103:0/256963503 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c02d670 con 0x56191a947370 2024-09-19T07:01:23.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.744+0000 7f0b8afd7740 1 -- 172.21.15.103:0/256963503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a947370 msgr2=0x56191a93f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:23.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.744+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/256963503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a947370 0x56191a93f660 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0048a0 tx=0x7f0b6c004970 comp rx=0 tx=0).stop 2024-09-19T07:01:23.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.744+0000 7f0b8afd7740 1 -- 172.21.15.103:0/256963503 shutdown_connections 2024-09-19T07:01:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.744+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/256963503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a947370 0x56191a93f660 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.744+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/256963503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a93db40 0x56191a93f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.744+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/256963503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a93c5d0 0x56191a953310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.744+0000 7f0b8afd7740 1 -- 172.21.15.103:0/256963503 >> 172.21.15.103:0/256963503 conn(0x56191a905310 msgr2=0x56191a943060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.744+0000 7f0b8afd7740 1 -- 172.21.15.103:0/256963503 shutdown_connections 2024-09-19T07:01:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.744+0000 7f0b8afd7740 1 -- 172.21.15.103:0/256963503 wait complete. 2024-09-19T07:01:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.744+0000 7f0b8afd7740 1 -- start start 2024-09-19T07:01:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.744+0000 7f0b8afd7740 1 Processor -- start 2024-09-19T07:01:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b8afd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93c5d0 0x56191a94fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b8afd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a93db40 0x56191a94af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b8afd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a947370 0x56191a94b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b8afd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841aed00 con 0x56191a93c5d0 2024-09-19T07:01:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b8afd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841ae630 con 0x56191a93db40 2024-09-19T07:01:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b8afd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841ab890 con 0x56191a947370 2024-09-19T07:01:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b8afd7740 10 client.0 _ll_register_callbacks cb 0x7f0b740094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93c5d0 0x56191a94fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93c5d0 0x56191a94fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46552/0 (socket says 172.21.15.103:46552) 2024-09-19T07:01:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b7bfff640 1 -- 172.21.15.103:0/3895875768 learned_addr learned my addr 172.21.15.103:0/3895875768 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b8259c640 1 --2- 172.21.15.103:0/3895875768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a93db40 0x56191a94af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b7bfff640 1 -- 172.21.15.103:0/3895875768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a947370 msgr2=0x56191a94b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/3895875768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a947370 0x56191a94b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b7bfff640 1 -- 172.21.15.103:0/3895875768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a93db40 msgr2=0x56191a94af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/3895875768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a93db40 0x56191a94af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b7bfff640 1 -- 172.21.15.103:0/3895875768 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56191a94bc60 con 0x56191a93c5d0 2024-09-19T07:01:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/3895875768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93c5d0 0x56191a94fba0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0113b0 tx=0x7f0b6c00a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a990d6ac21d67ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b69ffb640 1 -- 172.21.15.103:0/3895875768 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c019030 con 0x56191a93c5d0 2024-09-19T07:01:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b69ffb640 1 -- 172.21.15.103:0/3895875768 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f0b6c0116b0 con 0x56191a93c5d0 2024-09-19T07:01:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.745+0000 7f0b8afd7740 1 -- 172.21.15.103:0/3895875768 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b6c0110d0 con 0x56191a93c5d0 2024-09-19T07:01:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.746+0000 7f0b69ffb640 1 -- 172.21.15.103:0/3895875768 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0b6c0118b0 con 0x56191a93c5d0 2024-09-19T07:01:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.746+0000 7f0b8afd7740 10 client.19554 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.746+0000 7f0b8afd7740 1 -- 172.21.15.103:0/3895875768 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56191a9c1c20 con 0x56191a93c5d0 2024-09-19T07:01:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.746+0000 7f0b69ffb640 1 -- 172.21.15.103:0/3895875768 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b6c0110d0 con 0x56191a93c5d0 2024-09-19T07:01:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.746+0000 7f0b69ffb640 1 -- 172.21.15.103:0/3895875768 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0b6c036470 con 0x56191a93c5d0 2024-09-19T07:01:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.746+0000 7f0b8afd7740 10 client.19554 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.746+0000 7f0b8afd7740 10 client.19554 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:01:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.746+0000 7f0b8afd7740 20 client.19554 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.746+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/3895875768 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56191a9c20f0 0x56191a9c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.746+0000 7f0b8afd7740 4 client.19554 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:01:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.746+0000 7f0b8afd7740 1 -- 172.21.15.103:0/3895875768 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56191a8a9d70 con 0x56191a9c20f0 2024-09-19T07:01:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.746+0000 7f0b8259c640 1 --2- 172.21.15.103:0/3895875768 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56191a9c20f0 0x56191a9c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b8259c640 1 --2- 172.21.15.103:0/3895875768 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56191a9c20f0 0x56191a9c44e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d478979d208f404 server_cookie=1c310c713f8c0be1 in_seq=0 out_seq=0 2024-09-19T07:01:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b69ffb640 10 client.19554 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:01:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b69ffb640 1 -- 172.21.15.103:0/3895875768 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56191a8a9d70 con 0x56191a9c20f0 2024-09-19T07:01:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b69ffb640 10 client.19554 handle_command_reply: tid=0 2024-09-19T07:01:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b8afd7740 1 client.19554 shutdown 2024-09-19T07:01:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b8afd7740 1 -- 172.21.15.103:0/3895875768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93c5d0 msgr2=0x56191a94fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/3895875768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93c5d0 0x56191a94fba0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0113b0 tx=0x7f0b6c00a740 comp rx=0 tx=0).stop 2024-09-19T07:01:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b8afd7740 1 -- 172.21.15.103:0/3895875768 shutdown_connections 2024-09-19T07:01:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/3895875768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56191a947370 0x56191a94b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/3895875768 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56191a9c20f0 0x56191a9c44e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/3895875768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56191a93db40 0x56191a94af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b69ffb640 0 client.19554 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:01:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b8afd7740 1 --2- 172.21.15.103:0/3895875768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56191a93c5d0 0x56191a94fba0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b8afd7740 1 -- 172.21.15.103:0/3895875768 >> 172.21.15.103:0/3895875768 conn(0x56191a905310 msgr2=0x56191a9374e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:23.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b8afd7740 1 -- 172.21.15.103:0/3895875768 shutdown_connections 2024-09-19T07:01:23.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b8afd7740 1 -- 172.21.15.103:0/3895875768 wait complete. 2024-09-19T07:01:23.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.747+0000 7f0b8afd7740 20 client.19554 trim_cache size 0 max 16384 2024-09-19T07:01:23.754 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:01:23.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.748+0000 7f0b88bfe640 1 -- 172.21.15.103:0/1252335953 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b700a33b0 msgr2=0x7f0b700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:23.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.748+0000 7f0b88bfe640 1 --2- 172.21.15.103:0/1252335953 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b700a33b0 0x7f0b700a5870 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0bd610 tx=0x7f0b7c0b99a0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.748+0000 7f0b88bfe640 1 -- 172.21.15.103:0/1252335953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b84106f50 msgr2=0x7f0b8419cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:23.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.748+0000 7f0b88bfe640 1 --2- 172.21.15.103:0/1252335953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b84106f50 0x7f0b8419cd90 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c006010 tx=0x7f0b7c003c20 comp rx=0 tx=0).stop 2024-09-19T07:01:23.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.748+0000 7f0b88bfe640 1 -- 172.21.15.103:0/1252335953 shutdown_connections 2024-09-19T07:01:23.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.748+0000 7f0b88bfe640 1 --2- 172.21.15.103:0/1252335953 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b700a33b0 0x7f0b700a5870 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.748+0000 7f0b88bfe640 1 --2- 172.21.15.103:0/1252335953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b84106f50 0x7f0b8419cd90 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.748+0000 7f0b88bfe640 1 --2- 172.21.15.103:0/1252335953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b84100ca0 0x7f0b84198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.748+0000 7f0b88bfe640 1 --2- 172.21.15.103:0/1252335953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b841002f0 0x7f0b841984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:23.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.748+0000 7f0b88bfe640 1 -- 172.21.15.103:0/1252335953 >> 172.21.15.103:0/1252335953 conn(0x7f0b84077210 msgr2=0x7f0b840fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:23.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.748+0000 7f0b88bfe640 1 -- 172.21.15.103:0/1252335953 shutdown_connections 2024-09-19T07:01:23.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:23.748+0000 7f0b88bfe640 1 -- 172.21.15.103:0/1252335953 wait complete. 2024-09-19T07:01:24.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:24 smithi174 ceph-mon[35637]: pgmap v379: 97 pgs: 97 active+clean; 5.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-19T07:01:24.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:24 smithi103 ceph-mon[31158]: pgmap v379: 97 pgs: 97 active+clean; 5.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-19T07:01:24.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:24 smithi134 ceph-mon[35356]: pgmap v379: 97 pgs: 97 active+clean; 5.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-19T07:01:24.455 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:01:24.700 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.146+0000 7fd036e7d640 1 -- 172.21.15.103:0/1173749066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd030106ba0 msgr2=0x7fd030106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.146+0000 7fd036e7d640 1 --2- 172.21.15.103:0/1173749066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd030106ba0 0x7fd030106f80 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fd028009900 tx=0x7fd02802a410 comp rx=0 tx=0).stop 2024-09-19T07:01:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.146+0000 7fd036e7d640 1 -- 172.21.15.103:0/1173749066 shutdown_connections 2024-09-19T07:01:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.146+0000 7fd036e7d640 1 --2- 172.21.15.103:0/1173749066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd030100a70 0x7fd03010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.146+0000 7fd036e7d640 1 --2- 172.21.15.103:0/1173749066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0301000d0 0x7fd030100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.146+0000 7fd036e7d640 1 --2- 172.21.15.103:0/1173749066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd030106ba0 0x7fd030106f80 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.146+0000 7fd036e7d640 1 -- 172.21.15.103:0/1173749066 >> 172.21.15.103:0/1173749066 conn(0x7fd030077210 msgr2=0x7fd0300fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:25.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.146+0000 7fd036e7d640 1 -- 172.21.15.103:0/1173749066 shutdown_connections 2024-09-19T07:01:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.146+0000 7fd036e7d640 1 -- 172.21.15.103:0/1173749066 wait complete. 2024-09-19T07:01:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd036e7d640 1 Processor -- start 2024-09-19T07:01:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd036e7d640 1 -- start start 2024-09-19T07:01:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd036e7d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0301000d0 0x7fd0301985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd036e7d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd030100a70 0x7fd030198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd036e7d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd030106ba0 0x7fd03019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd036e7d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd030112130 con 0x7fd0301000d0 2024-09-19T07:01:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd036e7d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd030111fb0 con 0x7fd030100a70 2024-09-19T07:01:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd036e7d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0301122b0 con 0x7fd030106ba0 2024-09-19T07:01:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd034c19640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0301000d0 0x7fd0301985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd034c19640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0301000d0 0x7fd0301985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46578/0 (socket says 172.21.15.103:46578) 2024-09-19T07:01:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd034c19640 1 -- 172.21.15.103:0/4132259668 learned_addr learned my addr 172.21.15.103:0/4132259668 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd03541a640 1 --2- 172.21.15.103:0/4132259668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd030106ba0 0x7fd03019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd034c19640 1 -- 172.21.15.103:0/4132259668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd030106ba0 msgr2=0x7fd03019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:25.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd034c19640 1 --2- 172.21.15.103:0/4132259668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd030106ba0 0x7fd03019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd034c19640 1 -- 172.21.15.103:0/4132259668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd030100a70 msgr2=0x7fd030198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:25.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd034c19640 1 --2- 172.21.15.103:0/4132259668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd030100a70 0x7fd030198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd034c19640 1 -- 172.21.15.103:0/4132259668 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03019d550 con 0x7fd0301000d0 2024-09-19T07:01:25.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.147+0000 7fd034c19640 1 --2- 172.21.15.103:0/4132259668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0301000d0 0x7fd0301985a0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fd028006010 tx=0x7fd028003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:25.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.148+0000 7fd02dffb640 1 -- 172.21.15.103:0/4132259668 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02803f070 con 0x7fd0301000d0 2024-09-19T07:01:25.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.148+0000 7fd036e7d640 1 -- 172.21.15.103:0/4132259668 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03019d7e0 con 0x7fd0301000d0 2024-09-19T07:01:25.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.148+0000 7fd02dffb640 1 -- 172.21.15.103:0/4132259668 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd028031e30 con 0x7fd0301000d0 2024-09-19T07:01:25.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.148+0000 7fd02dffb640 1 -- 172.21.15.103:0/4132259668 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028035030 con 0x7fd0301000d0 2024-09-19T07:01:25.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.148+0000 7fd036e7d640 1 -- 172.21.15.103:0/4132259668 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0301a5120 con 0x7fd0301000d0 2024-09-19T07:01:25.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.150+0000 7fd02dffb640 1 -- 172.21.15.103:0/4132259668 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd028042070 con 0x7fd0301000d0 2024-09-19T07:01:25.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.150+0000 7fd039256740 1 Processor -- start 2024-09-19T07:01:25.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.150+0000 7fd039256740 1 -- start start 2024-09-19T07:01:25.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd039256740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b72b40 0x55d868b71090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd039256740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b71a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd039256740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b7c370 0x55d868b7e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd02dffb640 1 --2- 172.21.15.103:0/4132259668 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd01c0a33b0 0x7fd01c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd039256740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d868ad72f0 con 0x55d868b715d0 2024-09-19T07:01:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd039256740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d868a7ecf0 con 0x55d868b7c370 2024-09-19T07:01:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd039256740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d868ad32b0 con 0x55d868b72b40 2024-09-19T07:01:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd034c19640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b71a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd03541a640 1 --2- 172.21.15.103:0/4132259668 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd01c0a33b0 0x7fd01c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd034c19640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b71a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46582/0 (socket says 172.21.15.103:46582) 2024-09-19T07:01:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd034c19640 1 -- 172.21.15.103:0/346330040 learned_addr learned my addr 172.21.15.103:0/346330040 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd03541a640 1 --2- 172.21.15.103:0/346330040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b72b40 0x55d868b71090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd02ffff640 1 --2- 172.21.15.103:0/346330040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b7c370 0x55d868b7e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd02dffb640 1 -- 172.21.15.103:0/4132259668 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7fd0280f0740 con 0x7fd0301000d0 2024-09-19T07:01:25.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd034c19640 1 -- 172.21.15.103:0/346330040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b72b40 msgr2=0x55d868b71090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd034c19640 1 --2- 172.21.15.103:0/346330040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b72b40 0x55d868b71090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd034c19640 1 -- 172.21.15.103:0/346330040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b7c370 msgr2=0x55d868b7e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd034c19640 1 --2- 172.21.15.103:0/346330040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b7c370 0x55d868b7e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd034c19640 1 -- 172.21.15.103:0/346330040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d868a633a0 con 0x55d868b715d0 2024-09-19T07:01:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd034c19640 1 --2- 172.21.15.103:0/346330040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b71a10 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fd0280b8e60 tx=0x7fd02808f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4079f3140aa4c884 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.151+0000 7fd03541a640 1 --2- 172.21.15.103:0/4132259668 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd01c0a33b0 0x7fd01c0a5870 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd01800afd0 tx=0x7fd01800a820 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.152+0000 7fd026ffd640 1 -- 172.21.15.103:0/346330040 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028049070 con 0x55d868b715d0 2024-09-19T07:01:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.152+0000 7fd026ffd640 1 -- 172.21.15.103:0/346330040 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0280bd100 con 0x55d868b715d0 2024-09-19T07:01:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.152+0000 7fd026ffd640 1 -- 172.21.15.103:0/346330040 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0280bca00 con 0x55d868b715d0 2024-09-19T07:01:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.152+0000 7fd039256740 1 -- 172.21.15.103:0/346330040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 msgr2=0x55d868b71a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.152+0000 7fd039256740 1 --2- 172.21.15.103:0/346330040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b71a10 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fd0280b8e60 tx=0x7fd02808f830 comp rx=0 tx=0).stop 2024-09-19T07:01:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.152+0000 7fd039256740 1 -- 172.21.15.103:0/346330040 shutdown_connections 2024-09-19T07:01:25.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.152+0000 7fd039256740 1 --2- 172.21.15.103:0/346330040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b7c370 0x55d868b7e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.152+0000 7fd039256740 1 --2- 172.21.15.103:0/346330040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b71a10 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.152+0000 7fd039256740 1 --2- 172.21.15.103:0/346330040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b72b40 0x55d868b71090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.152+0000 7fd039256740 1 -- 172.21.15.103:0/346330040 >> 172.21.15.103:0/346330040 conn(0x55d868b3a310 msgr2=0x55d868b6dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:25.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.152+0000 7fd039256740 1 -- 172.21.15.103:0/346330040 shutdown_connections 2024-09-19T07:01:25.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.152+0000 7fd039256740 1 -- 172.21.15.103:0/346330040 wait complete. 2024-09-19T07:01:25.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.153+0000 7fd039256740 1 -- start start 2024-09-19T07:01:25.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.153+0000 7fd039256740 1 Processor -- start 2024-09-19T07:01:25.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.153+0000 7fd039256740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b715d0 0x55d868b7bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.153+0000 7fd039256740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b72b40 0x55d868b79030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.153+0000 7fd039256740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b7c370 0x55d868b79640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.153+0000 7fd039256740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d868b7ee30 con 0x55d868b72b40 2024-09-19T07:01:25.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.153+0000 7fd039256740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d868a92060 con 0x55d868b7c370 2024-09-19T07:01:25.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.153+0000 7fd039256740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d868b7efb0 con 0x55d868b715d0 2024-09-19T07:01:25.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.153+0000 7fd039256740 10 client.0 _ll_register_callbacks cb 0x55d8687e58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:25.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.153+0000 7fd02ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b72b40 0x55d868b79030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd02ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b72b40 0x55d868b79030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46584/0 (socket says 172.21.15.103:46584) 2024-09-19T07:01:25.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd02ffff640 1 -- 172.21.15.103:0/318601120 learned_addr learned my addr 172.21.15.103:0/318601120 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:25.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd03541a640 1 --2- 172.21.15.103:0/318601120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b7c370 0x55d868b79640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd034c19640 1 --2- 172.21.15.103:0/318601120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b715d0 0x55d868b7bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd02ffff640 1 -- 172.21.15.103:0/318601120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b715d0 msgr2=0x55d868b7bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:25.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd02ffff640 1 --2- 172.21.15.103:0/318601120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b715d0 0x55d868b7bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd02ffff640 1 -- 172.21.15.103:0/318601120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b7c370 msgr2=0x55d868b79640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:25.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd02ffff640 1 --2- 172.21.15.103:0/318601120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b7c370 0x55d868b79640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd02ffff640 1 -- 172.21.15.103:0/318601120 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8688483c0 con 0x55d868b72b40 2024-09-19T07:01:25.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd034c19640 1 --2- 172.21.15.103:0/318601120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b715d0 0x55d868b7bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:25.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd03541a640 1 --2- 172.21.15.103:0/318601120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b7c370 0x55d868b79640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:25.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd02ffff640 1 --2- 172.21.15.103:0/318601120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b72b40 0x55d868b79030 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fd02000aad0 tx=0x7fd02000af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b9dbedc146c500 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:25.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd00ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:25.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd00ffff640 1 -- 172.21.15.103:0/318601120 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020018020 con 0x55d868b72b40 2024-09-19T07:01:25.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd00ffff640 1 -- 172.21.15.103:0/318601120 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7fd02000ec30 con 0x55d868b72b40 2024-09-19T07:01:25.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd039256740 1 -- 172.21.15.103:0/318601120 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0280bd100 con 0x55d868b72b40 2024-09-19T07:01:25.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd00ffff640 1 -- 172.21.15.103:0/318601120 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd020023050 con 0x55d868b72b40 2024-09-19T07:01:25.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd039256740 10 client.19566 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:25.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd039256740 1 -- 172.21.15.103:0/318601120 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d868b8a850 con 0x55d868b72b40 2024-09-19T07:01:25.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.154+0000 7fd00ffff640 1 -- 172.21.15.103:0/318601120 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0280bd100 con 0x55d868b72b40 2024-09-19T07:01:25.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.155+0000 7fd00ffff640 1 -- 172.21.15.103:0/318601120 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd02001c430 con 0x55d868b72b40 2024-09-19T07:01:25.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.155+0000 7fd039256740 10 client.19566 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:25.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.155+0000 7fd039256740 10 client.19566 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:01:25.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.155+0000 7fd039256740 20 client.19566 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:25.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.155+0000 7fd039256740 1 --2- 172.21.15.103:0/318601120 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d868b8bdb0 0x55d868b8e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.155+0000 7fd039256740 4 client.19566 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:25.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.155+0000 7fd039256740 1 -- 172.21.15.103:0/318601120 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d868ae4860 con 0x55d868b8bdb0 2024-09-19T07:01:25.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.155+0000 7fd034c19640 1 --2- 172.21.15.103:0/318601120 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d868b8bdb0 0x55d868b8e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.155+0000 7fd034c19640 1 --2- 172.21.15.103:0/318601120 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d868b8bdb0 0x55d868b8e1f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc23d15ad67f7731 server_cookie=9d4fcaac4dc94845 in_seq=0 out_seq=0 2024-09-19T07:01:25.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.155+0000 7fd00ffff640 10 client.19566 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:01:25.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.156+0000 7fd00ffff640 1 -- 172.21.15.103:0/318601120 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d868ae4860 con 0x55d868b8bdb0 2024-09-19T07:01:25.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.156+0000 7fd00ffff640 10 client.19566 handle_command_reply: tid=0 2024-09-19T07:01:25.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.156+0000 7fd039256740 1 client.19566 shutdown 2024-09-19T07:01:25.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.156+0000 7fd039256740 1 -- 172.21.15.103:0/318601120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b72b40 msgr2=0x55d868b79030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:25.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.156+0000 7fd039256740 1 --2- 172.21.15.103:0/318601120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b72b40 0x55d868b79030 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fd02000aad0 tx=0x7fd02000af90 comp rx=0 tx=0).stop 2024-09-19T07:01:25.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.156+0000 7fd039256740 1 -- 172.21.15.103:0/318601120 shutdown_connections 2024-09-19T07:01:25.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.156+0000 7fd039256740 1 --2- 172.21.15.103:0/318601120 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d868b8bdb0 0x55d868b8e1f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.156+0000 7fd039256740 1 --2- 172.21.15.103:0/318601120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b7c370 0x55d868b79640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.156+0000 7fd039256740 1 --2- 172.21.15.103:0/318601120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b72b40 0x55d868b79030 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.156+0000 7fd00ffff640 0 client.19566 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:01:25.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.156+0000 7fd039256740 1 --2- 172.21.15.103:0/318601120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b715d0 0x55d868b7bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.156+0000 7fd039256740 1 -- 172.21.15.103:0/318601120 >> 172.21.15.103:0/318601120 conn(0x55d868b3a310 msgr2=0x55d868b73990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:25.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.156+0000 7fd039256740 1 -- 172.21.15.103:0/318601120 shutdown_connections 2024-09-19T07:01:25.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.156+0000 7fd039256740 1 -- 172.21.15.103:0/318601120 wait complete. 2024-09-19T07:01:25.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.156+0000 7fd039256740 20 client.19566 trim_cache size 0 max 16384 2024-09-19T07:01:25.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.169+0000 7fd039256740 1 Processor -- start 2024-09-19T07:01:25.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.169+0000 7fd039256740 1 -- start start 2024-09-19T07:01:25.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.169+0000 7fd039256740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b87f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.169+0000 7fd039256740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b72b40 0x55d868b78010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.169+0000 7fd039256740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b7c370 0x55d868b93ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.169+0000 7fd039256740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd0301ada80 con 0x55d868b715d0 2024-09-19T07:01:25.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.169+0000 7fd039256740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0301af5c0 con 0x55d868b72b40 2024-09-19T07:01:25.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.169+0000 7fd039256740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0301ae150 con 0x55d868b7c370 2024-09-19T07:01:25.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.169+0000 7fd034c19640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b87f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.169+0000 7fd034c19640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b87f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46598/0 (socket says 172.21.15.103:46598) 2024-09-19T07:01:25.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.169+0000 7fd034c19640 1 -- 172.21.15.103:0/753388672 learned_addr learned my addr 172.21.15.103:0/753388672 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:25.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.169+0000 7fd02ffff640 1 --2- 172.21.15.103:0/753388672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b72b40 0x55d868b78010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.170+0000 7fd03541a640 1 --2- 172.21.15.103:0/753388672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b7c370 0x55d868b93ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.170+0000 7fd034c19640 1 -- 172.21.15.103:0/753388672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b7c370 msgr2=0x55d868b93ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.170+0000 7fd034c19640 1 --2- 172.21.15.103:0/753388672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b7c370 0x55d868b93ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.170+0000 7fd034c19640 1 -- 172.21.15.103:0/753388672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b72b40 msgr2=0x55d868b78010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.170+0000 7fd034c19640 1 --2- 172.21.15.103:0/753388672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b72b40 0x55d868b78010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.170+0000 7fd034c19640 1 -- 172.21.15.103:0/753388672 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d868b77580 con 0x55d868b715d0 2024-09-19T07:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.170+0000 7fd03541a640 1 --2- 172.21.15.103:0/753388672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b7c370 0x55d868b93ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.170+0000 7fd034c19640 1 --2- 172.21.15.103:0/753388672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b87f20 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fd0280bc6b0 tx=0x7fd0280e77d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113a1766b4c48a73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.170+0000 7fd0267fc640 1 -- 172.21.15.103:0/753388672 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028049040 con 0x55d868b715d0 2024-09-19T07:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.170+0000 7fd0267fc640 1 -- 172.21.15.103:0/753388672 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0280bcc60 con 0x55d868b715d0 2024-09-19T07:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.170+0000 7fd0267fc640 1 -- 172.21.15.103:0/753388672 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0280bca00 con 0x55d868b715d0 2024-09-19T07:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.170+0000 7fd039256740 1 -- 172.21.15.103:0/753388672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 msgr2=0x55d868b87f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.170+0000 7fd039256740 1 --2- 172.21.15.103:0/753388672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b87f20 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fd0280bc6b0 tx=0x7fd0280e77d0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.171+0000 7fd039256740 1 -- 172.21.15.103:0/753388672 shutdown_connections 2024-09-19T07:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.171+0000 7fd039256740 1 --2- 172.21.15.103:0/753388672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b7c370 0x55d868b93ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.171+0000 7fd039256740 1 --2- 172.21.15.103:0/753388672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b72b40 0x55d868b78010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.171+0000 7fd039256740 1 --2- 172.21.15.103:0/753388672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b87f20 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.171+0000 7fd039256740 1 -- 172.21.15.103:0/753388672 >> 172.21.15.103:0/753388672 conn(0x55d868b3a310 msgr2=0x55d868b77900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.171+0000 7fd039256740 1 -- 172.21.15.103:0/753388672 shutdown_connections 2024-09-19T07:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.171+0000 7fd039256740 1 -- 172.21.15.103:0/753388672 wait complete. 2024-09-19T07:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.171+0000 7fd039256740 1 -- start start 2024-09-19T07:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.171+0000 7fd039256740 1 Processor -- start 2024-09-19T07:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.171+0000 7fd039256740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b76100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.171+0000 7fd039256740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b72b40 0x55d868b76640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.171+0000 7fd039256740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b7c370 0x55d868b76b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.171+0000 7fd039256740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd0301aeef0 con 0x55d868b715d0 2024-09-19T07:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.171+0000 7fd039256740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0301ae820 con 0x55d868b7c370 2024-09-19T07:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.171+0000 7fd039256740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0301aba80 con 0x55d868b72b40 2024-09-19T07:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.171+0000 7fd039256740 10 client.0 _ll_register_callbacks cb 0x7fd0200099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd034c19640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b76100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd034c19640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b76100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46612/0 (socket says 172.21.15.103:46612) 2024-09-19T07:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd034c19640 1 -- 172.21.15.103:0/4151063709 learned_addr learned my addr 172.21.15.103:0/4151063709 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd03541a640 1 --2- 172.21.15.103:0/4151063709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b7c370 0x55d868b76b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd02ffff640 1 --2- 172.21.15.103:0/4151063709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b72b40 0x55d868b76640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd034c19640 1 -- 172.21.15.103:0/4151063709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b72b40 msgr2=0x55d868b76640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd034c19640 1 --2- 172.21.15.103:0/4151063709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b72b40 0x55d868b76640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd034c19640 1 -- 172.21.15.103:0/4151063709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b7c370 msgr2=0x55d868b76b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd034c19640 1 --2- 172.21.15.103:0/4151063709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b7c370 0x55d868b76b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd034c19640 1 -- 172.21.15.103:0/4151063709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d868bf5b40 con 0x55d868b715d0 2024-09-19T07:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd034c19640 1 --2- 172.21.15.103:0/4151063709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b76100 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fd0280eded0 tx=0x7fd02803b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817f9fbd3da43402 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd00ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd00ffff640 1 -- 172.21.15.103:0/4151063709 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0280d5060 con 0x55d868b715d0 2024-09-19T07:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd00ffff640 1 -- 172.21.15.103:0/4151063709 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7fd0280bad00 con 0x55d868b715d0 2024-09-19T07:01:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd039256740 1 -- 172.21.15.103:0/4151063709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0280bcc60 con 0x55d868b715d0 2024-09-19T07:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd00ffff640 1 -- 172.21.15.103:0/4151063709 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd028049070 con 0x55d868b715d0 2024-09-19T07:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd039256740 10 client.19572 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd039256740 1 -- 172.21.15.103:0/4151063709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d868bf6d90 con 0x55d868b715d0 2024-09-19T07:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.172+0000 7fd00ffff640 1 -- 172.21.15.103:0/4151063709 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0280bcc60 con 0x55d868b715d0 2024-09-19T07:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.173+0000 7fd00ffff640 1 -- 172.21.15.103:0/4151063709 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd0280c2180 con 0x55d868b715d0 2024-09-19T07:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.173+0000 7fd039256740 10 client.19572 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.173+0000 7fd039256740 10 client.19572 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.173+0000 7fd039256740 20 client.19572 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.173+0000 7fd039256740 1 --2- 172.21.15.103:0/4151063709 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d868bf7030 0x55d868bf9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.173+0000 7fd039256740 4 client.19572 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.173+0000 7fd039256740 1 -- 172.21.15.103:0/4151063709 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d868aded70 con 0x55d868bf7030 2024-09-19T07:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.173+0000 7fd02ffff640 1 --2- 172.21.15.103:0/4151063709 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d868bf7030 0x55d868bf9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.173+0000 7fd02ffff640 1 --2- 172.21.15.103:0/4151063709 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d868bf7030 0x55d868bf9420 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d25abe99bdc74ec7 server_cookie=cf7ea3afc861c634 in_seq=0 out_seq=0 2024-09-19T07:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.173+0000 7fd00ffff640 10 client.19572 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:01:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd00ffff640 1 -- 172.21.15.103:0/4151063709 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d868aded70 con 0x55d868bf7030 2024-09-19T07:01:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd00ffff640 10 client.19572 handle_command_reply: tid=0 2024-09-19T07:01:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd039256740 1 client.19572 shutdown 2024-09-19T07:01:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd039256740 1 -- 172.21.15.103:0/4151063709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 msgr2=0x55d868b76100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd039256740 1 --2- 172.21.15.103:0/4151063709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b76100 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fd0280eded0 tx=0x7fd02803b990 comp rx=0 tx=0).stop 2024-09-19T07:01:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd039256740 1 -- 172.21.15.103:0/4151063709 shutdown_connections 2024-09-19T07:01:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd039256740 1 --2- 172.21.15.103:0/4151063709 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d868bf7030 0x55d868bf9420 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd039256740 1 --2- 172.21.15.103:0/4151063709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d868b7c370 0x55d868b76b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd00ffff640 0 client.19572 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:01:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd039256740 1 --2- 172.21.15.103:0/4151063709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d868b72b40 0x55d868b76640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd039256740 1 --2- 172.21.15.103:0/4151063709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868b715d0 0x55d868b76100 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd039256740 1 -- 172.21.15.103:0/4151063709 >> 172.21.15.103:0/4151063709 conn(0x55d868b3a310 msgr2=0x55d868b78120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd039256740 1 -- 172.21.15.103:0/4151063709 shutdown_connections 2024-09-19T07:01:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd039256740 1 -- 172.21.15.103:0/4151063709 wait complete. 2024-09-19T07:01:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd039256740 20 client.19572 trim_cache size 0 max 16384 2024-09-19T07:01:25.185 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:01:25.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd036e7d640 1 -- 172.21.15.103:0/4132259668 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd01c0a33b0 msgr2=0x7fd01c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd036e7d640 1 --2- 172.21.15.103:0/4132259668 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd01c0a33b0 0x7fd01c0a5870 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd01800afd0 tx=0x7fd01800a820 comp rx=0 tx=0).stop 2024-09-19T07:01:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd036e7d640 1 -- 172.21.15.103:0/4132259668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0301000d0 msgr2=0x7fd0301985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.174+0000 7fd036e7d640 1 --2- 172.21.15.103:0/4132259668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0301000d0 0x7fd0301985a0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fd028006010 tx=0x7fd028003c80 comp rx=0 tx=0).stop 2024-09-19T07:01:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.175+0000 7fd036e7d640 1 -- 172.21.15.103:0/4132259668 shutdown_connections 2024-09-19T07:01:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.175+0000 7fd036e7d640 1 --2- 172.21.15.103:0/4132259668 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd01c0a33b0 0x7fd01c0a5870 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.175+0000 7fd036e7d640 1 --2- 172.21.15.103:0/4132259668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd030106ba0 0x7fd03019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.175+0000 7fd036e7d640 1 --2- 172.21.15.103:0/4132259668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd030100a70 0x7fd030198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.175+0000 7fd036e7d640 1 --2- 172.21.15.103:0/4132259668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0301000d0 0x7fd0301985a0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.175+0000 7fd036e7d640 1 -- 172.21.15.103:0/4132259668 >> 172.21.15.103:0/4132259668 conn(0x7fd030077210 msgr2=0x7fd030104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.175+0000 7fd036e7d640 1 -- 172.21.15.103:0/4132259668 shutdown_connections 2024-09-19T07:01:25.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:25.175+0000 7fd036e7d640 1 -- 172.21.15.103:0/4132259668 wait complete. 2024-09-19T07:01:25.824 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:01:26.076 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:26.302 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:26 smithi103 ceph-mon[31158]: pgmap v380: 97 pgs: 97 active+clean; 5.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T07:01:26.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:26 smithi174 ceph-mon[35637]: pgmap v380: 97 pgs: 97 active+clean; 5.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T07:01:26.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:26 smithi134 ceph-mon[35356]: pgmap v380: 97 pgs: 97 active+clean; 5.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T07:01:26.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.503+0000 7f2908eeb640 1 -- 172.21.15.103:0/3773830977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2904106a80 msgr2=0x7f2904106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:26.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.503+0000 7f2908eeb640 1 --2- 172.21.15.103:0/3773830977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2904106a80 0x7f2904106e60 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009930 tx=0x7f28f402a420 comp rx=0 tx=0).stop 2024-09-19T07:01:26.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.503+0000 7f2908eeb640 1 -- 172.21.15.103:0/3773830977 shutdown_connections 2024-09-19T07:01:26.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.503+0000 7f2908eeb640 1 --2- 172.21.15.103:0/3773830977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2904101420 0x7f290410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.503+0000 7f2908eeb640 1 --2- 172.21.15.103:0/3773830977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2904100a80 0x7f2904100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.503+0000 7f2908eeb640 1 --2- 172.21.15.103:0/3773830977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2904106a80 0x7f2904106e60 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.503+0000 7f2908eeb640 1 -- 172.21.15.103:0/3773830977 >> 172.21.15.103:0/3773830977 conn(0x7f29040fc7a0 msgr2=0x7f29040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:26.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.503+0000 7f2908eeb640 1 -- 172.21.15.103:0/3773830977 shutdown_connections 2024-09-19T07:01:26.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.503+0000 7f2908eeb640 1 -- 172.21.15.103:0/3773830977 wait complete. 2024-09-19T07:01:26.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.504+0000 7f2908eeb640 1 Processor -- start 2024-09-19T07:01:26.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.504+0000 7f2908eeb640 1 -- start start 2024-09-19T07:01:26.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.504+0000 7f2908eeb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2904100a80 0x7f2904199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.504+0000 7f2908eeb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2904101420 0x7f2904199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.504+0000 7f2908eeb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2904106a80 0x7f29041a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.504+0000 7f2908eeb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2904110e80 con 0x7f2904106a80 2024-09-19T07:01:26.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.504+0000 7f2908eeb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2904110d00 con 0x7f2904101420 2024-09-19T07:01:26.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.504+0000 7f2908eeb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2904111000 con 0x7f2904100a80 2024-09-19T07:01:26.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.504+0000 7f2902d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2904106a80 0x7f29041a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:26.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.504+0000 7f2902d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2904106a80 0x7f29041a43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46638/0 (socket says 172.21.15.103:46638) 2024-09-19T07:01:26.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.504+0000 7f2902d9d640 1 -- 172.21.15.103:0/420900095 learned_addr learned my addr 172.21.15.103:0/420900095 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:26.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.504+0000 7f28fbfff640 1 --2- 172.21.15.103:0/420900095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2904101420 0x7f2904199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:26.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.504+0000 7f290259c640 1 --2- 172.21.15.103:0/420900095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2904100a80 0x7f2904199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:26.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.505+0000 7f2902d9d640 1 -- 172.21.15.103:0/420900095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2904100a80 msgr2=0x7f2904199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:26.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.505+0000 7f2902d9d640 1 --2- 172.21.15.103:0/420900095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2904100a80 0x7f2904199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.505+0000 7f2902d9d640 1 -- 172.21.15.103:0/420900095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2904101420 msgr2=0x7f2904199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:26.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.505+0000 7f2902d9d640 1 --2- 172.21.15.103:0/420900095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2904101420 0x7f2904199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.505+0000 7f2902d9d640 1 -- 172.21.15.103:0/420900095 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29041a48e0 con 0x7f2904106a80 2024-09-19T07:01:26.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.505+0000 7f2902d9d640 1 --2- 172.21.15.103:0/420900095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2904106a80 0x7f29041a43a0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00cad0 tx=0x7f28fc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:26.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.505+0000 7f28fb7fe640 1 -- 172.21.15.103:0/420900095 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc00ae20 con 0x7f2904106a80 2024-09-19T07:01:26.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.505+0000 7f28fb7fe640 1 -- 172.21.15.103:0/420900095 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28fc005020 con 0x7f2904106a80 2024-09-19T07:01:26.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.505+0000 7f2908eeb640 1 -- 172.21.15.103:0/420900095 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29041a4ab0 con 0x7f2904106a80 2024-09-19T07:01:26.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.505+0000 7f28fb7fe640 1 -- 172.21.15.103:0/420900095 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc00f6f0 con 0x7f2904106a80 2024-09-19T07:01:26.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.505+0000 7f2908eeb640 1 -- 172.21.15.103:0/420900095 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29041a4f70 con 0x7f2904106a80 2024-09-19T07:01:26.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.507+0000 7f28fb7fe640 1 -- 172.21.15.103:0/420900095 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f28fc0051d0 con 0x7f2904106a80 2024-09-19T07:01:26.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.508+0000 7f290b2c4740 1 Processor -- start 2024-09-19T07:01:26.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.508+0000 7f290b2c4740 1 -- start start 2024-09-19T07:01:26.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.508+0000 7f290b2c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a60b40 0x562113a5f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.508+0000 7f290b2c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a5fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.508+0000 7f290b2c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a6e780 0x562113a70b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.508+0000 7f290b2c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5621139c52f0 con 0x562113a5f5d0 2024-09-19T07:01:26.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.508+0000 7f290b2c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56211396ccf0 con 0x562113a6e780 2024-09-19T07:01:26.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.508+0000 7f290b2c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5621139c12b0 con 0x562113a60b40 2024-09-19T07:01:26.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.508+0000 7f28fb7fe640 1 --2- 172.21.15.103:0/420900095 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28f00a33b0 0x7f28f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.508+0000 7f290259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a5fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:26.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.508+0000 7f290259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a5fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46642/0 (socket says 172.21.15.103:46642) 2024-09-19T07:01:26.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.508+0000 7f290259c640 1 -- 172.21.15.103:0/2517417647 learned_addr learned my addr 172.21.15.103:0/2517417647 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:26.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.508+0000 7f2902d9d640 1 --2- 172.21.15.103:0/420900095 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28f00a33b0 0x7f28f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:26.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.508+0000 7f28fbfff640 1 --2- 172.21.15.103:0/2517417647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a60b40 0x562113a5f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:26.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.509+0000 7f28fbfff640 1 --2- 172.21.15.103:0/2517417647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a6e780 0x562113a70b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:26.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.509+0000 7f290259c640 1 -- 172.21.15.103:0/2517417647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a60b40 msgr2=0x562113a5f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:26.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.509+0000 7f28fb7fe640 1 -- 172.21.15.103:0/420900095 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f28fc0cfd70 con 0x7f2904106a80 2024-09-19T07:01:26.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.509+0000 7f290259c640 1 --2- 172.21.15.103:0/2517417647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a60b40 0x562113a5f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.509+0000 7f290259c640 1 -- 172.21.15.103:0/2517417647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a6e780 msgr2=0x562113a70b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:26.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.509+0000 7f290259c640 1 --2- 172.21.15.103:0/2517417647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a6e780 0x562113a70b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.509+0000 7f290259c640 1 -- 172.21.15.103:0/2517417647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621139513a0 con 0x562113a5f5d0 2024-09-19T07:01:26.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.509+0000 7f28fbfff640 1 --2- 172.21.15.103:0/2517417647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a60b40 0x562113a5f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:26.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.509+0000 7f290259c640 1 --2- 172.21.15.103:0/2517417647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a5fa10 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009930 tx=0x7f28f4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe66586468d4d1a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:26.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.509+0000 7f2902d9d640 1 --2- 172.21.15.103:0/420900095 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28f00a33b0 0x7f28f00a5870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0a4770 tx=0x7f28fc005960 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:26.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.509+0000 7f28f8ff9640 1 -- 172.21.15.103:0/2517417647 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f403f070 con 0x562113a5f5d0 2024-09-19T07:01:26.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.509+0000 7f28f8ff9640 1 -- 172.21.15.103:0/2517417647 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28f40317b0 con 0x562113a5f5d0 2024-09-19T07:01:26.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.509+0000 7f28f8ff9640 1 -- 172.21.15.103:0/2517417647 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f4035070 con 0x562113a5f5d0 2024-09-19T07:01:26.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.510+0000 7f290b2c4740 1 -- 172.21.15.103:0/2517417647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 msgr2=0x562113a5fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:26.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.510+0000 7f290b2c4740 1 --2- 172.21.15.103:0/2517417647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a5fa10 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009930 tx=0x7f28f4032ce0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.510+0000 7f290b2c4740 1 -- 172.21.15.103:0/2517417647 shutdown_connections 2024-09-19T07:01:26.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.510+0000 7f290b2c4740 1 --2- 172.21.15.103:0/2517417647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a6e780 0x562113a70b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.510+0000 7f290b2c4740 1 --2- 172.21.15.103:0/2517417647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a5fa10 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.510+0000 7f290b2c4740 1 --2- 172.21.15.103:0/2517417647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a60b40 0x562113a5f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.510+0000 7f290b2c4740 1 -- 172.21.15.103:0/2517417647 >> 172.21.15.103:0/2517417647 conn(0x562113a28310 msgr2=0x562113a5bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:26.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.510+0000 7f290b2c4740 1 -- 172.21.15.103:0/2517417647 shutdown_connections 2024-09-19T07:01:26.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.510+0000 7f290b2c4740 1 -- 172.21.15.103:0/2517417647 wait complete. 2024-09-19T07:01:26.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.510+0000 7f290b2c4740 1 -- start start 2024-09-19T07:01:26.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.510+0000 7f290b2c4740 1 Processor -- start 2024-09-19T07:01:26.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f290b2c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a6e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f290b2c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a60b40 0x562113a6b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f290b2c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a6e780 0x562113a6ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f290b2c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562113a71420 con 0x562113a5f5d0 2024-09-19T07:01:26.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f290b2c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562113980060 con 0x562113a60b40 2024-09-19T07:01:26.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f290b2c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562113a715a0 con 0x562113a6e780 2024-09-19T07:01:26.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f290b2c4740 10 client.0 _ll_register_callbacks cb 0x5621136d38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:26.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f28fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a6e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:26.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f28fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a6e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46652/0 (socket says 172.21.15.103:46652) 2024-09-19T07:01:26.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f28fbfff640 1 -- 172.21.15.103:0/4185958630 learned_addr learned my addr 172.21.15.103:0/4185958630 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:26.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f290259c640 1 --2- 172.21.15.103:0/4185958630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a60b40 0x562113a6b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:26.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f28fbfff640 1 -- 172.21.15.103:0/4185958630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a6e780 msgr2=0x562113a6ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:26.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f28fbfff640 1 --2- 172.21.15.103:0/4185958630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a6e780 0x562113a6ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f28fbfff640 1 -- 172.21.15.103:0/4185958630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a60b40 msgr2=0x562113a6b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:26.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f28fbfff640 1 --2- 172.21.15.103:0/4185958630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a60b40 0x562113a6b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f28fbfff640 1 -- 172.21.15.103:0/4185958630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621137363c0 con 0x562113a5f5d0 2024-09-19T07:01:26.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f290259c640 1 --2- 172.21.15.103:0/4185958630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a60b40 0x562113a6b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:26.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f28fbfff640 1 --2- 172.21.15.103:0/4185958630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a6e180 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f28ec004500 tx=0x7f28ec0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d047ed2403275e8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:26.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f28e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:26.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f28e9ffb640 1 -- 172.21.15.103:0/4185958630 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec018070 con 0x562113a5f5d0 2024-09-19T07:01:26.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f28e9ffb640 1 -- 172.21.15.103:0/4185958630 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f28ec005220 con 0x562113a5f5d0 2024-09-19T07:01:26.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.511+0000 7f290b2c4740 1 -- 172.21.15.103:0/4185958630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28f40317b0 con 0x562113a5f5d0 2024-09-19T07:01:26.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.512+0000 7f28e9ffb640 1 -- 172.21.15.103:0/4185958630 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f28ec005420 con 0x562113a5f5d0 2024-09-19T07:01:26.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.512+0000 7f290b2c4740 10 client.19584 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:26.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.512+0000 7f290b2c4740 1 -- 172.21.15.103:0/4185958630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562113a7cf50 con 0x562113a5f5d0 2024-09-19T07:01:26.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.512+0000 7f28e9ffb640 1 -- 172.21.15.103:0/4185958630 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28f40317b0 con 0x562113a5f5d0 2024-09-19T07:01:26.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.512+0000 7f28e9ffb640 1 -- 172.21.15.103:0/4185958630 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f28ec012750 con 0x562113a5f5d0 2024-09-19T07:01:26.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.512+0000 7f290b2c4740 10 client.19584 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:26.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.512+0000 7f290b2c4740 10 client.19584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:26.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.512+0000 7f290b2c4740 20 client.19584 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:26.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.512+0000 7f290b2c4740 1 --2- 172.21.15.103:0/4185958630 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562113a7e410 0x562113a80800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.512+0000 7f290b2c4740 4 client.19584 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:26.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.512+0000 7f290b2c4740 1 -- 172.21.15.103:0/4185958630 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621139d2860 con 0x562113a7e410 2024-09-19T07:01:26.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.512+0000 7f290259c640 1 --2- 172.21.15.103:0/4185958630 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562113a7e410 0x562113a80800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:26.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.513+0000 7f290259c640 1 --2- 172.21.15.103:0/4185958630 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562113a7e410 0x562113a80800 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c27fa03187f74a4 server_cookie=4a6d6e2583005a48 in_seq=0 out_seq=0 2024-09-19T07:01:26.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.513+0000 7f28e9ffb640 10 client.19584 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:26.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.513+0000 7f28e9ffb640 1 -- 172.21.15.103:0/4185958630 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621139d2860 con 0x562113a7e410 2024-09-19T07:01:26.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.513+0000 7f28e9ffb640 10 client.19584 handle_command_reply: tid=0 2024-09-19T07:01:26.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.514+0000 7f290b2c4740 1 client.19584 shutdown 2024-09-19T07:01:26.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.514+0000 7f290b2c4740 1 -- 172.21.15.103:0/4185958630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 msgr2=0x562113a6e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:26.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.514+0000 7f290b2c4740 1 --2- 172.21.15.103:0/4185958630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a6e180 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f28ec004500 tx=0x7f28ec0045f0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.514+0000 7f290b2c4740 1 -- 172.21.15.103:0/4185958630 shutdown_connections 2024-09-19T07:01:26.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.514+0000 7f290b2c4740 1 --2- 172.21.15.103:0/4185958630 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562113a7e410 0x562113a80800 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.514+0000 7f28e9ffb640 0 client.19584 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:26.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.514+0000 7f290b2c4740 1 --2- 172.21.15.103:0/4185958630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a6e780 0x562113a6ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.514+0000 7f290b2c4740 1 --2- 172.21.15.103:0/4185958630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a60b40 0x562113a6b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.514+0000 7f290b2c4740 1 --2- 172.21.15.103:0/4185958630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a6e180 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.514+0000 7f290b2c4740 1 -- 172.21.15.103:0/4185958630 >> 172.21.15.103:0/4185958630 conn(0x562113a28310 msgr2=0x562113a61930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:26.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.514+0000 7f290b2c4740 1 -- 172.21.15.103:0/4185958630 shutdown_connections 2024-09-19T07:01:26.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.514+0000 7f290b2c4740 1 -- 172.21.15.103:0/4185958630 wait complete. 2024-09-19T07:01:26.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.514+0000 7f290b2c4740 20 client.19584 trim_cache size 0 max 16384 2024-09-19T07:01:26.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f290b2c4740 1 Processor -- start 2024-09-19T07:01:26.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f290b2c4740 1 -- start start 2024-09-19T07:01:26.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f290b2c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a5be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f290b2c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a60b40 0x562113a7a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f290b2c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a6e780 0x562113a6acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f290b2c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f29041ab700 con 0x562113a5f5d0 2024-09-19T07:01:26.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f290b2c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29041ad240 con 0x562113a6e780 2024-09-19T07:01:26.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f290b2c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f29041abdd0 con 0x562113a60b40 2024-09-19T07:01:26.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f28fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a5be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:26.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f28fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a5be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46660/0 (socket says 172.21.15.103:46660) 2024-09-19T07:01:26.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f28fbfff640 1 -- 172.21.15.103:0/486501107 learned_addr learned my addr 172.21.15.103:0/486501107 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:26.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f290259c640 1 --2- 172.21.15.103:0/486501107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a60b40 0x562113a7a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:26.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f28fbfff640 1 -- 172.21.15.103:0/486501107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a60b40 msgr2=0x562113a7a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:26.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f28fbfff640 1 --2- 172.21.15.103:0/486501107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a60b40 0x562113a7a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f28fbfff640 1 -- 172.21.15.103:0/486501107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a6e780 msgr2=0x562113a6acf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:01:26.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f28fbfff640 1 --2- 172.21.15.103:0/486501107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a6e780 0x562113a6acf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f28fbfff640 1 -- 172.21.15.103:0/486501107 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562113a6bd90 con 0x562113a5f5d0 2024-09-19T07:01:26.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f290259c640 1 --2- 172.21.15.103:0/486501107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a60b40 0x562113a7a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:26.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.527+0000 7f28fbfff640 1 --2- 172.21.15.103:0/486501107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a5be10 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f28ec012b20 tx=0x7f28ec01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b71540c4d030d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:26.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.528+0000 7f28ebfff640 1 -- 172.21.15.103:0/486501107 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec018070 con 0x562113a5f5d0 2024-09-19T07:01:26.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.528+0000 7f28ebfff640 1 -- 172.21.15.103:0/486501107 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28ec01c4d0 con 0x562113a5f5d0 2024-09-19T07:01:26.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.528+0000 7f28ebfff640 1 -- 172.21.15.103:0/486501107 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec029040 con 0x562113a5f5d0 2024-09-19T07:01:26.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.528+0000 7f290b2c4740 1 -- 172.21.15.103:0/486501107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 msgr2=0x562113a5be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:26.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.528+0000 7f290b2c4740 1 --2- 172.21.15.103:0/486501107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a5be10 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f28ec012b20 tx=0x7f28ec01c380 comp rx=0 tx=0).stop 2024-09-19T07:01:26.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.528+0000 7f290b2c4740 1 -- 172.21.15.103:0/486501107 shutdown_connections 2024-09-19T07:01:26.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.528+0000 7f290b2c4740 1 --2- 172.21.15.103:0/486501107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a6e780 0x562113a6acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.528+0000 7f290b2c4740 1 --2- 172.21.15.103:0/486501107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a60b40 0x562113a7a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.528+0000 7f290b2c4740 1 --2- 172.21.15.103:0/486501107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a5f5d0 0x562113a5be10 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.528+0000 7f290b2c4740 1 -- 172.21.15.103:0/486501107 >> 172.21.15.103:0/486501107 conn(0x562113a28310 msgr2=0x562113a5b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:26.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.528+0000 7f290b2c4740 1 -- 172.21.15.103:0/486501107 shutdown_connections 2024-09-19T07:01:26.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.528+0000 7f290b2c4740 1 -- 172.21.15.103:0/486501107 wait complete. 2024-09-19T07:01:26.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f290b2c4740 1 -- start start 2024-09-19T07:01:26.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f290b2c4740 1 Processor -- start 2024-09-19T07:01:26.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f290b2c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a5f5d0 0x562113a76b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f290b2c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a60b40 0x562113a77060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f290b2c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a6e780 0x562113a67e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f290b2c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f29041acb70 con 0x562113a6e780 2024-09-19T07:01:26.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f290b2c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29041ac4a0 con 0x562113a60b40 2024-09-19T07:01:26.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f290b2c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f29041a9700 con 0x562113a5f5d0 2024-09-19T07:01:26.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f290b2c4740 10 client.0 _ll_register_callbacks cb 0x7f28ec004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:26.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f28fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a6e780 0x562113a67e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:26.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f28fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a6e780 0x562113a67e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46662/0 (socket says 172.21.15.103:46662) 2024-09-19T07:01:26.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f28fbfff640 1 -- 172.21.15.103:0/267363489 learned_addr learned my addr 172.21.15.103:0/267363489 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:26.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f290259c640 1 --2- 172.21.15.103:0/267363489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a60b40 0x562113a77060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:26.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f28fbfff640 1 --2- 172.21.15.103:0/267363489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a5f5d0 0x562113a76b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:26.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f28fbfff640 1 -- 172.21.15.103:0/267363489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a5f5d0 msgr2=0x562113a76b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:26.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f28fbfff640 1 --2- 172.21.15.103:0/267363489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a5f5d0 0x562113a76b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f28fbfff640 1 -- 172.21.15.103:0/267363489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a60b40 msgr2=0x562113a77060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:26.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f28fbfff640 1 --2- 172.21.15.103:0/267363489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a60b40 0x562113a77060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.529+0000 7f28fbfff640 1 -- 172.21.15.103:0/267363489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562113a68640 con 0x562113a6e780 2024-09-19T07:01:26.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.530+0000 7f28fbfff640 1 --2- 172.21.15.103:0/267363489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a6e780 0x562113a67e10 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f28ec01c4a0 tx=0x7f28ec01ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18298b983eacb93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:26.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.530+0000 7f28e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:26.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.530+0000 7f28e9ffb640 1 -- 172.21.15.103:0/267363489 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec018030 con 0x562113a6e780 2024-09-19T07:01:26.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.530+0000 7f28e9ffb640 1 -- 172.21.15.103:0/267363489 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f28ec012860 con 0x562113a6e780 2024-09-19T07:01:26.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.530+0000 7f290b2c4740 1 -- 172.21.15.103:0/267363489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28ec01c4d0 con 0x562113a6e780 2024-09-19T07:01:26.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.530+0000 7f28e9ffb640 1 -- 172.21.15.103:0/267363489 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f28ec01c680 con 0x562113a6e780 2024-09-19T07:01:26.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.530+0000 7f290b2c4740 10 client.19590 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:26.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.530+0000 7f290b2c4740 1 -- 172.21.15.103:0/267363489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562113ae8bb0 con 0x562113a6e780 2024-09-19T07:01:26.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.530+0000 7f28e9ffb640 1 -- 172.21.15.103:0/267363489 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28ec01c4d0 con 0x562113a6e780 2024-09-19T07:01:26.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.530+0000 7f28e9ffb640 1 -- 172.21.15.103:0/267363489 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f28ec02d420 con 0x562113a6e780 2024-09-19T07:01:26.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.530+0000 7f290b2c4740 10 client.19590 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:26.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.530+0000 7f290b2c4740 10 client.19590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:26.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.530+0000 7f290b2c4740 20 client.19590 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:26.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.530+0000 7f290b2c4740 1 --2- 172.21.15.103:0/267363489 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562113a7e410 0x562113aeb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:26.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.530+0000 7f290b2c4740 4 client.19590 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:01:26.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.530+0000 7f290b2c4740 1 -- 172.21.15.103:0/267363489 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621139ccd70 con 0x562113a7e410 2024-09-19T07:01:26.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.531+0000 7f290259c640 1 --2- 172.21.15.103:0/267363489 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562113a7e410 0x562113aeb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:26.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.531+0000 7f290259c640 1 --2- 172.21.15.103:0/267363489 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562113a7e410 0x562113aeb210 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8178a6522a269ce server_cookie=e33e49070f636c41 in_seq=0 out_seq=0 2024-09-19T07:01:26.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.531+0000 7f28e9ffb640 10 client.19590 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:26.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.531+0000 7f28e9ffb640 1 -- 172.21.15.103:0/267363489 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621139ccd70 con 0x562113a7e410 2024-09-19T07:01:26.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.531+0000 7f28e9ffb640 10 client.19590 handle_command_reply: tid=0 2024-09-19T07:01:26.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.531+0000 7f290b2c4740 1 client.19590 shutdown 2024-09-19T07:01:26.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.531+0000 7f290b2c4740 1 -- 172.21.15.103:0/267363489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a6e780 msgr2=0x562113a67e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:26.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.531+0000 7f290b2c4740 1 --2- 172.21.15.103:0/267363489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a6e780 0x562113a67e10 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f28ec01c4a0 tx=0x7f28ec01ce60 comp rx=0 tx=0).stop 2024-09-19T07:01:26.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.531+0000 7f290b2c4740 1 -- 172.21.15.103:0/267363489 shutdown_connections 2024-09-19T07:01:26.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.531+0000 7f290b2c4740 1 --2- 172.21.15.103:0/267363489 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562113a7e410 0x562113aeb210 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.531+0000 7f290b2c4740 1 --2- 172.21.15.103:0/267363489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562113a6e780 0x562113a67e10 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.531+0000 7f290b2c4740 1 --2- 172.21.15.103:0/267363489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562113a60b40 0x562113a77060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.531+0000 7f290b2c4740 1 --2- 172.21.15.103:0/267363489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562113a5f5d0 0x562113a76b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.531+0000 7f28e9ffb640 0 client.19590 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:26.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.531+0000 7f290b2c4740 1 -- 172.21.15.103:0/267363489 >> 172.21.15.103:0/267363489 conn(0x562113a28310 msgr2=0x562113a5a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:26.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.532+0000 7f290b2c4740 1 -- 172.21.15.103:0/267363489 shutdown_connections 2024-09-19T07:01:26.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.532+0000 7f290b2c4740 1 -- 172.21.15.103:0/267363489 wait complete. 2024-09-19T07:01:26.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.532+0000 7f290b2c4740 20 client.19590 trim_cache size 0 max 16384 2024-09-19T07:01:26.538 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:01:26.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.532+0000 7f2908eeb640 1 -- 172.21.15.103:0/420900095 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28f00a33b0 msgr2=0x7f28f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:26.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.532+0000 7f2908eeb640 1 --2- 172.21.15.103:0/420900095 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28f00a33b0 0x7f28f00a5870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0a4770 tx=0x7f28fc005960 comp rx=0 tx=0).stop 2024-09-19T07:01:26.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.532+0000 7f2908eeb640 1 -- 172.21.15.103:0/420900095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2904106a80 msgr2=0x7f29041a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:26.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.532+0000 7f2908eeb640 1 --2- 172.21.15.103:0/420900095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2904106a80 0x7f29041a43a0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00cad0 tx=0x7f28fc00cf90 comp rx=0 tx=0).stop 2024-09-19T07:01:26.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.532+0000 7f2908eeb640 1 -- 172.21.15.103:0/420900095 shutdown_connections 2024-09-19T07:01:26.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.532+0000 7f2908eeb640 1 --2- 172.21.15.103:0/420900095 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f28f00a33b0 0x7f28f00a5870 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.532+0000 7f2908eeb640 1 --2- 172.21.15.103:0/420900095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2904106a80 0x7f29041a43a0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.532+0000 7f2908eeb640 1 --2- 172.21.15.103:0/420900095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2904101420 0x7f2904199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.532+0000 7f2908eeb640 1 --2- 172.21.15.103:0/420900095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2904100a80 0x7f2904199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:26.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.532+0000 7f2908eeb640 1 -- 172.21.15.103:0/420900095 >> 172.21.15.103:0/420900095 conn(0x7f29040fc7a0 msgr2=0x7f29040fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:26.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.532+0000 7f2908eeb640 1 -- 172.21.15.103:0/420900095 shutdown_connections 2024-09-19T07:01:26.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:26.532+0000 7f2908eeb640 1 -- 172.21.15.103:0/420900095 wait complete. 2024-09-19T07:01:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.867+0000 7f1283f78640 1 -- 172.21.15.103:0/1729806195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f127c10c0d0 msgr2=0x7f127c10e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.867+0000 7f1283f78640 1 --2- 172.21.15.103:0/1729806195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f127c10c0d0 0x7f127c10e590 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f1274009ca0 tx=0x7f127402a950 comp rx=0 tx=0).stop 2024-09-19T07:01:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.868+0000 7f1283f78640 1 -- 172.21.15.103:0/1729806195 shutdown_connections 2024-09-19T07:01:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.868+0000 7f1283f78640 1 --2- 172.21.15.103:0/1729806195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f127c10c0d0 0x7f127c10e590 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.868+0000 7f1283f78640 1 --2- 172.21.15.103:0/1729806195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f127c1035d0 0x7f127c10bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.868+0000 7f1283f78640 1 --2- 172.21.15.103:0/1729806195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f127c102cb0 0x7f127c103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.868+0000 7f1283f78640 1 -- 172.21.15.103:0/1729806195 >> 172.21.15.103:0/1729806195 conn(0x7f127c0fc6c0 msgr2=0x7f127c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.868+0000 7f1283f78640 1 -- 172.21.15.103:0/1729806195 shutdown_connections 2024-09-19T07:01:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.868+0000 7f1283f78640 1 -- 172.21.15.103:0/1729806195 wait complete. 2024-09-19T07:01:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.868+0000 7f1283f78640 1 Processor -- start 2024-09-19T07:01:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.868+0000 7f1283f78640 1 -- start start 2024-09-19T07:01:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.868+0000 7f1283f78640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f127c102cb0 0x7f127c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.868+0000 7f1283f78640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f127c1035d0 0x7f127c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.868+0000 7f1283f78640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f127c10c0d0 0x7f127c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.868+0000 7f1283f78640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f127c111f20 con 0x7f127c102cb0 2024-09-19T07:01:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.868+0000 7f1283f78640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f127c111da0 con 0x7f127c10c0d0 2024-09-19T07:01:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.868+0000 7f1283f78640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f127c1120a0 con 0x7f127c1035d0 2024-09-19T07:01:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.869+0000 7f1281d14640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f127c102cb0 0x7f127c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.869+0000 7f1281d14640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f127c102cb0 0x7f127c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52502/0 (socket says 172.21.15.103:52502) 2024-09-19T07:01:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.869+0000 7f1281d14640 1 -- 172.21.15.103:0/4238831309 learned_addr learned my addr 172.21.15.103:0/4238831309 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.869+0000 7f1281513640 1 --2- 172.21.15.103:0/4238831309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f127c1035d0 0x7f127c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.869+0000 7f1282515640 1 --2- 172.21.15.103:0/4238831309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f127c10c0d0 0x7f127c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.869+0000 7f1281d14640 1 -- 172.21.15.103:0/4238831309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f127c1035d0 msgr2=0x7f127c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.869+0000 7f1281d14640 1 --2- 172.21.15.103:0/4238831309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f127c1035d0 0x7f127c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.869+0000 7f1281d14640 1 -- 172.21.15.103:0/4238831309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f127c10c0d0 msgr2=0x7f127c19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.869+0000 7f1281d14640 1 --2- 172.21.15.103:0/4238831309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f127c10c0d0 0x7f127c19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.869+0000 7f1281d14640 1 -- 172.21.15.103:0/4238831309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127c19d360 con 0x7f127c102cb0 2024-09-19T07:01:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.869+0000 7f1281d14640 1 --2- 172.21.15.103:0/4238831309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f127c102cb0 0x7f127c198370 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f127800d950 tx=0x7f127800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.869+0000 7f1272ffd640 1 -- 172.21.15.103:0/4238831309 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278017070 con 0x7f127c102cb0 2024-09-19T07:01:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.869+0000 7f1272ffd640 1 -- 172.21.15.103:0/4238831309 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1278009c30 con 0x7f127c102cb0 2024-09-19T07:01:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.869+0000 7f1283f78640 1 -- 172.21.15.103:0/4238831309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f127c19d650 con 0x7f127c102cb0 2024-09-19T07:01:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.869+0000 7f1272ffd640 1 -- 172.21.15.103:0/4238831309 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127800f6b0 con 0x7f127c102cb0 2024-09-19T07:01:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.869+0000 7f1283f78640 1 -- 172.21.15.103:0/4238831309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f127c1a4ee0 con 0x7f127c102cb0 2024-09-19T07:01:27.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.872+0000 7f1272ffd640 1 -- 172.21.15.103:0/4238831309 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f12780109d0 con 0x7f127c102cb0 2024-09-19T07:01:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.872+0000 7f1286def740 1 Processor -- start 2024-09-19T07:01:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.872+0000 7f1286def740 1 -- start start 2024-09-19T07:01:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.872+0000 7f1286def740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c7fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.872+0000 7f1286def740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c80410 0x55c8a6c8a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.872+0000 7f1286def740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c8acb0 0x55c8a6c8d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.872+0000 7f1286def740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c8a6be42f0 con 0x55c8a6c7faf0 2024-09-19T07:01:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.872+0000 7f1286def740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c8a6b8c270 con 0x55c8a6c8acb0 2024-09-19T07:01:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.872+0000 7f1286def740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c8a6be02b0 con 0x55c8a6c80410 2024-09-19T07:01:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.872+0000 7f1282515640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c7fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.872+0000 7f1282515640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c7fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52514/0 (socket says 172.21.15.103:52514) 2024-09-19T07:01:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.872+0000 7f1282515640 1 -- 172.21.15.103:0/4025322175 learned_addr learned my addr 172.21.15.103:0/4025322175 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.873+0000 7f1281513640 1 --2- 172.21.15.103:0/4025322175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c8acb0 0x55c8a6c8d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.873+0000 7f1272ffd640 1 --2- 172.21.15.103:0/4238831309 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f12680a3460 0x7f12680a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.873+0000 7f1281d14640 1 --2- 172.21.15.103:0/4025322175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c80410 0x55c8a6c8a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.873+0000 7f1282515640 1 -- 172.21.15.103:0/4025322175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c80410 msgr2=0x55c8a6c8a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.873+0000 7f1282515640 1 --2- 172.21.15.103:0/4025322175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c80410 0x55c8a6c8a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.873+0000 7f1282515640 1 -- 172.21.15.103:0/4025322175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c8acb0 msgr2=0x55c8a6c8d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.873+0000 7f1282515640 1 --2- 172.21.15.103:0/4025322175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c8acb0 0x55c8a6c8d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.873+0000 7f1282515640 1 -- 172.21.15.103:0/4025322175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8a6b8b340 con 0x55c8a6c7faf0 2024-09-19T07:01:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.873+0000 7f1272ffd640 1 -- 172.21.15.103:0/4238831309 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f12780cf940 con 0x7f127c102cb0 2024-09-19T07:01:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.873+0000 7f1282515640 1 --2- 172.21.15.103:0/4238831309 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f12680a3460 0x7f12680a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.873+0000 7f1282515640 1 --2- 172.21.15.103:0/4025322175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c7fed0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f1274009ca0 tx=0x7f127402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9e2337c7bce8b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.873+0000 7f1267fff640 1 -- 172.21.15.103:0/4025322175 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274034040 con 0x55c8a6c7faf0 2024-09-19T07:01:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.873+0000 7f1267fff640 1 -- 172.21.15.103:0/4025322175 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f127402fc00 con 0x55c8a6c7faf0 2024-09-19T07:01:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.873+0000 7f1267fff640 1 -- 172.21.15.103:0/4025322175 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274046670 con 0x55c8a6c7faf0 2024-09-19T07:01:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.874+0000 7f1282515640 1 --2- 172.21.15.103:0/4238831309 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f12680a3460 0x7f12680a5920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f127404d000 tx=0x7f127404a000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.874+0000 7f1286def740 1 -- 172.21.15.103:0/4025322175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c7faf0 msgr2=0x55c8a6c7fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.874+0000 7f1286def740 1 --2- 172.21.15.103:0/4025322175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c7fed0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f1274009ca0 tx=0x7f127402f470 comp rx=0 tx=0).stop 2024-09-19T07:01:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.874+0000 7f1286def740 1 -- 172.21.15.103:0/4025322175 shutdown_connections 2024-09-19T07:01:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.874+0000 7f1286def740 1 --2- 172.21.15.103:0/4025322175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c8acb0 0x55c8a6c8d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.874+0000 7f1286def740 1 --2- 172.21.15.103:0/4025322175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c80410 0x55c8a6c8a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.874+0000 7f1286def740 1 --2- 172.21.15.103:0/4025322175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c7fed0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.874+0000 7f1286def740 1 -- 172.21.15.103:0/4025322175 >> 172.21.15.103:0/4025322175 conn(0x55c8a6c7c060 msgr2=0x55c8a6c7c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.874+0000 7f1286def740 1 -- 172.21.15.103:0/4025322175 shutdown_connections 2024-09-19T07:01:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.874+0000 7f1286def740 1 -- 172.21.15.103:0/4025322175 wait complete. 2024-09-19T07:01:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1286def740 1 -- start start 2024-09-19T07:01:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1286def740 1 Processor -- start 2024-09-19T07:01:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1286def740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c87b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1286def740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c80410 0x55c8a6c88070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1286def740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c8acb0 0x55c8a6c885b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1286def740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c8a6c8d880 con 0x55c8a6c80410 2024-09-19T07:01:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1286def740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c8a6b6d290 con 0x55c8a6c8acb0 2024-09-19T07:01:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1286def740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c8a6c8da00 con 0x55c8a6c7faf0 2024-09-19T07:01:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1286def740 10 client.0 _ll_register_callbacks cb 0x55c8a6b9e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1281513640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c80410 0x55c8a6c88070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1281513640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c80410 0x55c8a6c88070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52518/0 (socket says 172.21.15.103:52518) 2024-09-19T07:01:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1281513640 1 -- 172.21.15.103:0/1883742200 learned_addr learned my addr 172.21.15.103:0/1883742200 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1281d14640 1 --2- 172.21.15.103:0/1883742200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c87b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1281513640 1 -- 172.21.15.103:0/1883742200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c7faf0 msgr2=0x55c8a6c87b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1281513640 1 --2- 172.21.15.103:0/1883742200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c87b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1281513640 1 -- 172.21.15.103:0/1883742200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c8acb0 msgr2=0x55c8a6c885b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1281513640 1 --2- 172.21.15.103:0/1883742200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c8acb0 0x55c8a6c885b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.875+0000 7f1281513640 1 -- 172.21.15.103:0/1883742200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8a68f7210 con 0x55c8a6c80410 2024-09-19T07:01:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.876+0000 7f1281513640 1 --2- 172.21.15.103:0/1883742200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c80410 0x55c8a6c88070 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f126c012700 tx=0x7f126c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8652d66337b60c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.876+0000 7f12657fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.876+0000 7f12657fa640 1 -- 172.21.15.103:0/1883742200 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c00f920 con 0x55c8a6c80410 2024-09-19T07:01:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.876+0000 7f12657fa640 1 -- 172.21.15.103:0/1883742200 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f126c00fac0 con 0x55c8a6c80410 2024-09-19T07:01:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.876+0000 7f1286def740 1 -- 172.21.15.103:0/1883742200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f127402fc00 con 0x55c8a6c80410 2024-09-19T07:01:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.876+0000 7f12657fa640 1 -- 172.21.15.103:0/1883742200 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f126c00fcc0 con 0x55c8a6c80410 2024-09-19T07:01:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.876+0000 7f1286def740 10 client.19602 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.876+0000 7f1286def740 1 -- 172.21.15.103:0/1883742200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c8a69c3620 con 0x55c8a6c80410 2024-09-19T07:01:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.876+0000 7f12657fa640 1 -- 172.21.15.103:0/1883742200 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f127402fc00 con 0x55c8a6c80410 2024-09-19T07:01:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.876+0000 7f12657fa640 1 -- 172.21.15.103:0/1883742200 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f126c0105f0 con 0x55c8a6c80410 2024-09-19T07:01:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.876+0000 7f1286def740 10 client.19602 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.876+0000 7f1286def740 10 client.19602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.876+0000 7f1286def740 20 client.19602 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.876+0000 7f1286def740 1 --2- 172.21.15.103:0/1883742200 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c8a6c9ac60 0x55c8a6c9d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.876+0000 7f1286def740 4 client.19602 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.876+0000 7f1286def740 1 -- 172.21.15.103:0/1883742200 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8a6bf1a20 con 0x55c8a6c9ac60 2024-09-19T07:01:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.877+0000 7f1281d14640 1 --2- 172.21.15.103:0/1883742200 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c8a6c9ac60 0x55c8a6c9d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.877+0000 7f1281d14640 1 --2- 172.21.15.103:0/1883742200 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c8a6c9ac60 0x55c8a6c9d070 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8edf0d91f06e004c server_cookie=c849e4bfb56f476 in_seq=0 out_seq=0 2024-09-19T07:01:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.877+0000 7f12657fa640 10 client.19602 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.878+0000 7f12657fa640 1 -- 172.21.15.103:0/1883742200 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8a6bf1a20 con 0x55c8a6c9ac60 2024-09-19T07:01:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.878+0000 7f12657fa640 10 client.19602 handle_command_reply: tid=0 2024-09-19T07:01:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.878+0000 7f1286def740 1 client.19602 shutdown 2024-09-19T07:01:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.878+0000 7f1286def740 1 -- 172.21.15.103:0/1883742200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c80410 msgr2=0x55c8a6c88070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.878+0000 7f1286def740 1 --2- 172.21.15.103:0/1883742200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c80410 0x55c8a6c88070 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f126c012700 tx=0x7f126c012bc0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.878+0000 7f1286def740 1 -- 172.21.15.103:0/1883742200 shutdown_connections 2024-09-19T07:01:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.878+0000 7f1286def740 1 --2- 172.21.15.103:0/1883742200 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c8a6c9ac60 0x55c8a6c9d070 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.878+0000 7f1286def740 1 --2- 172.21.15.103:0/1883742200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c8acb0 0x55c8a6c885b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.878+0000 7f1286def740 1 --2- 172.21.15.103:0/1883742200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c80410 0x55c8a6c88070 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.878+0000 7f12657fa640 0 client.19602 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.878+0000 7f1286def740 1 --2- 172.21.15.103:0/1883742200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c87b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.878+0000 7f1286def740 1 -- 172.21.15.103:0/1883742200 >> 172.21.15.103:0/1883742200 conn(0x55c8a6c7c060 msgr2=0x55c8a6c81d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.878+0000 7f1286def740 1 -- 172.21.15.103:0/1883742200 shutdown_connections 2024-09-19T07:01:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.878+0000 7f1286def740 1 -- 172.21.15.103:0/1883742200 wait complete. 2024-09-19T07:01:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.878+0000 7f1286def740 20 client.19602 trim_cache size 0 max 16384 2024-09-19T07:01:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1286def740 1 Processor -- start 2024-09-19T07:01:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1286def740 1 -- start start 2024-09-19T07:01:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1286def740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c82560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1286def740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c80410 0x55c8a6c883e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1286def740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c8acb0 0x55c8a6ca2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1286def740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f127c1ad8a0 con 0x55c8a6c8acb0 2024-09-19T07:01:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1286def740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f127c1af3e0 con 0x55c8a6c80410 2024-09-19T07:01:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1286def740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f127c1adf70 con 0x55c8a6c7faf0 2024-09-19T07:01:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1282515640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c8acb0 0x55c8a6ca2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1282515640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c8acb0 0x55c8a6ca2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52534/0 (socket says 172.21.15.103:52534) 2024-09-19T07:01:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1282515640 1 -- 172.21.15.103:0/971027846 learned_addr learned my addr 172.21.15.103:0/971027846 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1282515640 1 -- 172.21.15.103:0/971027846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c7faf0 msgr2=0x55c8a6c82560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1282515640 1 --2- 172.21.15.103:0/971027846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c82560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1281513640 1 --2- 172.21.15.103:0/971027846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c80410 0x55c8a6c883e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1282515640 1 -- 172.21.15.103:0/971027846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c80410 msgr2=0x55c8a6c883e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1282515640 1 --2- 172.21.15.103:0/971027846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c80410 0x55c8a6c883e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1282515640 1 -- 172.21.15.103:0/971027846 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8a6c7be90 con 0x55c8a6c8acb0 2024-09-19T07:01:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.893+0000 7f1282515640 1 --2- 172.21.15.103:0/971027846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c8acb0 0x55c8a6ca2980 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f127402af10 tx=0x7f1274005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab72204475831e5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.894+0000 7f12677fe640 1 -- 172.21.15.103:0/971027846 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274046670 con 0x55c8a6c8acb0 2024-09-19T07:01:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.894+0000 7f12677fe640 1 -- 172.21.15.103:0/971027846 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f127405a070 con 0x55c8a6c8acb0 2024-09-19T07:01:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.894+0000 7f12677fe640 1 -- 172.21.15.103:0/971027846 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274004040 con 0x55c8a6c8acb0 2024-09-19T07:01:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.894+0000 7f1286def740 1 -- 172.21.15.103:0/971027846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c8acb0 msgr2=0x55c8a6ca2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.894+0000 7f1286def740 1 --2- 172.21.15.103:0/971027846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c8acb0 0x55c8a6ca2980 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f127402af10 tx=0x7f1274005d80 comp rx=0 tx=0).stop 2024-09-19T07:01:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.894+0000 7f1286def740 1 -- 172.21.15.103:0/971027846 shutdown_connections 2024-09-19T07:01:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.894+0000 7f1286def740 1 --2- 172.21.15.103:0/971027846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c8acb0 0x55c8a6ca2980 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.894+0000 7f1286def740 1 --2- 172.21.15.103:0/971027846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c80410 0x55c8a6c883e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.894+0000 7f1286def740 1 --2- 172.21.15.103:0/971027846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c82560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.894+0000 7f1286def740 1 -- 172.21.15.103:0/971027846 >> 172.21.15.103:0/971027846 conn(0x55c8a6c7c060 msgr2=0x55c8a6c89bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.894+0000 7f1286def740 1 -- 172.21.15.103:0/971027846 shutdown_connections 2024-09-19T07:01:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.894+0000 7f1286def740 1 -- 172.21.15.103:0/971027846 wait complete. 2024-09-19T07:01:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1286def740 1 -- start start 2024-09-19T07:01:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1286def740 1 Processor -- start 2024-09-19T07:01:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1286def740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c933a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1286def740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c80410 0x55c8a6c8e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1286def740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c8acb0 0x55c8a6c8eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1286def740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f127c1aed10 con 0x55c8a6c7faf0 2024-09-19T07:01:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1286def740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f127c1ae640 con 0x55c8a6c8acb0 2024-09-19T07:01:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1286def740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f127c1ab8a0 con 0x55c8a6c80410 2024-09-19T07:01:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1286def740 10 client.0 _ll_register_callbacks cb 0x7f126c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1281d14640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1281d14640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c933a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52544/0 (socket says 172.21.15.103:52544) 2024-09-19T07:01:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1281d14640 1 -- 172.21.15.103:0/546663234 learned_addr learned my addr 172.21.15.103:0/546663234 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1281513640 1 --2- 172.21.15.103:0/546663234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c80410 0x55c8a6c8e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1281d14640 1 -- 172.21.15.103:0/546663234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c80410 msgr2=0x55c8a6c8e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1281d14640 1 --2- 172.21.15.103:0/546663234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c80410 0x55c8a6c8e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1281d14640 1 -- 172.21.15.103:0/546663234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c8acb0 msgr2=0x55c8a6c8eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1281d14640 1 --2- 172.21.15.103:0/546663234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c8acb0 0x55c8a6c8eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.895+0000 7f1281d14640 1 -- 172.21.15.103:0/546663234 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8a6c8f4d0 con 0x55c8a6c7faf0 2024-09-19T07:01:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.896+0000 7f1281d14640 1 --2- 172.21.15.103:0/546663234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c933a0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f12780cd070 tx=0x7f127800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9d5d67b84ba234 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.896+0000 7f12657fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.896+0000 7f12657fa640 1 -- 172.21.15.103:0/546663234 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278020040 con 0x55c8a6c7faf0 2024-09-19T07:01:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.896+0000 7f12657fa640 1 -- 172.21.15.103:0/546663234 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f12780cd290 con 0x55c8a6c7faf0 2024-09-19T07:01:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.896+0000 7f1286def740 1 -- 172.21.15.103:0/546663234 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f127405a070 con 0x55c8a6c7faf0 2024-09-19T07:01:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.896+0000 7f12657fa640 1 -- 172.21.15.103:0/546663234 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f12780cd490 con 0x55c8a6c7faf0 2024-09-19T07:01:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.896+0000 7f1286def740 10 client.19608 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.896+0000 7f1286def740 1 -- 172.21.15.103:0/546663234 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c8a6d058f0 con 0x55c8a6c7faf0 2024-09-19T07:01:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.896+0000 7f12657fa640 1 -- 172.21.15.103:0/546663234 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f127405a070 con 0x55c8a6c7faf0 2024-09-19T07:01:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.896+0000 7f12657fa640 1 -- 172.21.15.103:0/546663234 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f12780979a0 con 0x55c8a6c7faf0 2024-09-19T07:01:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.896+0000 7f1286def740 10 client.19608 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.896+0000 7f1286def740 10 client.19608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.896+0000 7f1286def740 20 client.19608 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.896+0000 7f1286def740 1 --2- 172.21.15.103:0/546663234 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c8a6d05fc0 0x55c8a6d083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.896+0000 7f1286def740 4 client.19608 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:01:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.896+0000 7f1286def740 1 -- 172.21.15.103:0/546663234 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8a6bebd70 con 0x55c8a6d05fc0 2024-09-19T07:01:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.897+0000 7f1281513640 1 --2- 172.21.15.103:0/546663234 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c8a6d05fc0 0x55c8a6d083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.897+0000 7f12657fa640 10 client.19608 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.897+0000 7f1281513640 1 --2- 172.21.15.103:0/546663234 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c8a6d05fc0 0x55c8a6d083e0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=593f54b034a62eaa server_cookie=d8ea820cb459adc0 in_seq=0 out_seq=0 2024-09-19T07:01:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.911+0000 7f12657fa640 1 -- 172.21.15.103:0/546663234 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8a6bebd70 con 0x55c8a6d05fc0 2024-09-19T07:01:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.911+0000 7f12657fa640 10 client.19608 handle_command_reply: tid=0 2024-09-19T07:01:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.911+0000 7f1286def740 1 client.19608 shutdown 2024-09-19T07:01:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.911+0000 7f1286def740 1 -- 172.21.15.103:0/546663234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c7faf0 msgr2=0x55c8a6c933a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.911+0000 7f1286def740 1 --2- 172.21.15.103:0/546663234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c933a0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f12780cd070 tx=0x7f127800bf60 comp rx=0 tx=0).stop 2024-09-19T07:01:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.911+0000 7f1286def740 1 -- 172.21.15.103:0/546663234 shutdown_connections 2024-09-19T07:01:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.911+0000 7f1286def740 1 --2- 172.21.15.103:0/546663234 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c8a6d05fc0 0x55c8a6d083e0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.911+0000 7f1286def740 1 --2- 172.21.15.103:0/546663234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8a6c8acb0 0x55c8a6c8eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.911+0000 7f1286def740 1 --2- 172.21.15.103:0/546663234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8a6c80410 0x55c8a6c8e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.911+0000 7f1286def740 1 --2- 172.21.15.103:0/546663234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8a6c7faf0 0x55c8a6c933a0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.911+0000 7f12657fa640 0 client.19608 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.911+0000 7f1286def740 1 -- 172.21.15.103:0/546663234 >> 172.21.15.103:0/546663234 conn(0x55c8a6c7c060 msgr2=0x55c8a6c7a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.912+0000 7f1286def740 1 -- 172.21.15.103:0/546663234 shutdown_connections 2024-09-19T07:01:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.912+0000 7f1286def740 1 -- 172.21.15.103:0/546663234 wait complete. 2024-09-19T07:01:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.912+0000 7f1286def740 20 client.19608 trim_cache size 0 max 16384 2024-09-19T07:01:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:01:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:01:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:01:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:01:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.912+0000 7f1283f78640 1 -- 172.21.15.103:0/4238831309 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f12680a3460 msgr2=0x7f12680a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.912+0000 7f1283f78640 1 --2- 172.21.15.103:0/4238831309 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f12680a3460 0x7f12680a5920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f127404d000 tx=0x7f127404a000 comp rx=0 tx=0).stop 2024-09-19T07:01:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.912+0000 7f1283f78640 1 -- 172.21.15.103:0/4238831309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f127c102cb0 msgr2=0x7f127c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.912+0000 7f1283f78640 1 --2- 172.21.15.103:0/4238831309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f127c102cb0 0x7f127c198370 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f127800d950 tx=0x7f127800de10 comp rx=0 tx=0).stop 2024-09-19T07:01:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.912+0000 7f1283f78640 1 -- 172.21.15.103:0/4238831309 shutdown_connections 2024-09-19T07:01:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.912+0000 7f1283f78640 1 --2- 172.21.15.103:0/4238831309 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f12680a3460 0x7f12680a5920 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.912+0000 7f1283f78640 1 --2- 172.21.15.103:0/4238831309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f127c10c0d0 0x7f127c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.912+0000 7f1283f78640 1 --2- 172.21.15.103:0/4238831309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f127c1035d0 0x7f127c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.912+0000 7f1283f78640 1 --2- 172.21.15.103:0/4238831309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f127c102cb0 0x7f127c198370 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.912+0000 7f1283f78640 1 -- 172.21.15.103:0/4238831309 >> 172.21.15.103:0/4238831309 conn(0x7f127c0fc6c0 msgr2=0x7f127c0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.912+0000 7f1283f78640 1 -- 172.21.15.103:0/4238831309 shutdown_connections 2024-09-19T07:01:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:27.912+0000 7f1283f78640 1 -- 172.21.15.103:0/4238831309 wait complete. 2024-09-19T07:01:28.308 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:01:28.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:01:28.331 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:28 smithi103 ceph-mon[31158]: pgmap v381: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-09-19T07:01:28.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:28 smithi174 ceph-mon[35637]: pgmap v381: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-09-19T07:01:28.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:28 smithi134 ceph-mon[35356]: pgmap v381: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-09-19T07:01:28.556 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:29.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.998+0000 7f80516f5640 1 -- 172.21.15.103:0/1149649891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804c100e20 msgr2=0x7f804c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:29.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.998+0000 7f80516f5640 1 --2- 172.21.15.103:0/1149649891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804c100e20 0x7f804c101200 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f803c009900 tx=0x7f803c02a480 comp rx=0 tx=0).stop 2024-09-19T07:01:29.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.998+0000 7f80516f5640 1 -- 172.21.15.103:0/1149649891 shutdown_connections 2024-09-19T07:01:29.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.998+0000 7f80516f5640 1 --2- 172.21.15.103:0/1149649891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f804c102ef0 0x7f804c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.998+0000 7f80516f5640 1 --2- 172.21.15.103:0/1149649891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f804c102570 0x7f804c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.998+0000 7f80516f5640 1 --2- 172.21.15.103:0/1149649891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804c100e20 0x7f804c101200 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.998+0000 7f80516f5640 1 -- 172.21.15.103:0/1149649891 >> 172.21.15.103:0/1149649891 conn(0x7f804c0fc7d0 msgr2=0x7f804c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:29.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.998+0000 7f80516f5640 1 -- 172.21.15.103:0/1149649891 shutdown_connections 2024-09-19T07:01:29.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.998+0000 7f80516f5640 1 -- 172.21.15.103:0/1149649891 wait complete. 2024-09-19T07:01:29.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.999+0000 7f80516f5640 1 Processor -- start 2024-09-19T07:01:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.999+0000 7f80516f5640 1 -- start start 2024-09-19T07:01:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.999+0000 7f80516f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804c100e20 0x7f804c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.999+0000 7f80516f5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f804c102570 0x7f804c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.999+0000 7f80516f5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f804c102ef0 0x7f804c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.999+0000 7f80516f5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f804c110f50 con 0x7f804c100e20 2024-09-19T07:01:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.999+0000 7f80516f5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f804c110dd0 con 0x7f804c102570 2024-09-19T07:01:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.999+0000 7f80516f5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f804c1110d0 con 0x7f804c102ef0 2024-09-19T07:01:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:28.999+0000 7f804affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804c100e20 0x7f804c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.000+0000 7f804affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804c100e20 0x7f804c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52550/0 (socket says 172.21.15.103:52550) 2024-09-19T07:01:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.000+0000 7f804affd640 1 -- 172.21.15.103:0/3698107123 learned_addr learned my addr 172.21.15.103:0/3698107123 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.000+0000 7f8043fff640 1 --2- 172.21.15.103:0/3698107123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f804c102570 0x7f804c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.000+0000 7f804b7fe640 1 --2- 172.21.15.103:0/3698107123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f804c102ef0 0x7f804c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.000+0000 7f804affd640 1 -- 172.21.15.103:0/3698107123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f804c102ef0 msgr2=0x7f804c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.000+0000 7f804affd640 1 --2- 172.21.15.103:0/3698107123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f804c102ef0 0x7f804c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.000+0000 7f804affd640 1 -- 172.21.15.103:0/3698107123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f804c102570 msgr2=0x7f804c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.000+0000 7f804affd640 1 --2- 172.21.15.103:0/3698107123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f804c102570 0x7f804c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.000+0000 7f804affd640 1 -- 172.21.15.103:0/3698107123 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c1a69e0 con 0x7f804c100e20 2024-09-19T07:01:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.000+0000 7f804b7fe640 1 --2- 172.21.15.103:0/3698107123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f804c102ef0 0x7f804c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.000+0000 7f804affd640 1 --2- 172.21.15.103:0/3698107123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804c100e20 0x7f804c0fcc80 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f803c006010 tx=0x7f803c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.000+0000 7f8048ff9640 1 -- 172.21.15.103:0/3698107123 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c03f070 con 0x7f804c100e20 2024-09-19T07:01:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.000+0000 7f8048ff9640 1 -- 172.21.15.103:0/3698107123 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f803c031e30 con 0x7f804c100e20 2024-09-19T07:01:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.000+0000 7f80516f5640 1 -- 172.21.15.103:0/3698107123 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804c1a6c70 con 0x7f804c100e20 2024-09-19T07:01:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.000+0000 7f8048ff9640 1 -- 172.21.15.103:0/3698107123 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c035030 con 0x7f804c100e20 2024-09-19T07:01:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.000+0000 7f80516f5640 1 -- 172.21.15.103:0/3698107123 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f804c1a7150 con 0x7f804c100e20 2024-09-19T07:01:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.003+0000 7f8048ff9640 1 -- 172.21.15.103:0/3698107123 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f803c042070 con 0x7f804c100e20 2024-09-19T07:01:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.003+0000 7f8053ace740 1 Processor -- start 2024-09-19T07:01:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.003+0000 7f8053ace740 1 -- start start 2024-09-19T07:01:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.003+0000 7f8053ace740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 0x55b25797ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.003+0000 7f8053ace740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b257980480 0x55b25798a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.003+0000 7f8053ace740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25798ad40 0x55b25798d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.003+0000 7f8053ace740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b2578e5280 con 0x55b257980480 2024-09-19T07:01:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.003+0000 7f8053ace740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b25788cb10 con 0x55b25797fb60 2024-09-19T07:01:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.003+0000 7f8053ace740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b2578e1240 con 0x55b25798ad40 2024-09-19T07:01:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.003+0000 7f804affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b257980480 0x55b25798a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.003+0000 7f8048ff9640 1 --2- 172.21.15.103:0/3698107123 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80380a33b0 0x7f80380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f804affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b257980480 0x55b25798a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52552/0 (socket says 172.21.15.103:52552) 2024-09-19T07:01:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f804affd640 1 -- 172.21.15.103:0/690059332 learned_addr learned my addr 172.21.15.103:0/690059332 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f804b7fe640 1 --2- 172.21.15.103:0/690059332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 0x55b25797ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f8043fff640 1 --2- 172.21.15.103:0/690059332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25798ad40 0x55b25798d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f8048ff9640 1 -- 172.21.15.103:0/3698107123 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f803c0f0740 con 0x7f804c100e20 2024-09-19T07:01:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f804b7fe640 1 --2- 172.21.15.103:0/3698107123 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80380a33b0 0x7f80380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f804affd640 1 -- 172.21.15.103:0/690059332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25798ad40 msgr2=0x55b25798d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:29.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f804affd640 1 --2- 172.21.15.103:0/690059332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25798ad40 0x55b25798d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f804affd640 1 -- 172.21.15.103:0/690059332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 msgr2=0x55b25797ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:29.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f804affd640 1 --2- 172.21.15.103:0/690059332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 0x55b25797ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f804affd640 1 -- 172.21.15.103:0/690059332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2578991f0 con 0x55b257980480 2024-09-19T07:01:29.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f804b7fe640 1 --2- 172.21.15.103:0/690059332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 0x55b25797ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:01:29.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f804affd640 1 --2- 172.21.15.103:0/690059332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b257980480 0x55b25798a800 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f803c0ede10 tx=0x7f803c03b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cac5526599bc85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:29.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f80417fa640 1 -- 172.21.15.103:0/690059332 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c049070 con 0x55b257980480 2024-09-19T07:01:29.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f804b7fe640 1 --2- 172.21.15.103:0/3698107123 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80380a33b0 0x7f80380a5870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8044008fd0 tx=0x7f8044004ab0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:29.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f80417fa640 1 -- 172.21.15.103:0/690059332 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f803c0bd100 con 0x55b257980480 2024-09-19T07:01:29.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.004+0000 7f80417fa640 1 -- 172.21.15.103:0/690059332 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c0ee0f0 con 0x55b257980480 2024-09-19T07:01:29.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.005+0000 7f8053ace740 1 -- 172.21.15.103:0/690059332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b257980480 msgr2=0x55b25798a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:29.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.005+0000 7f8053ace740 1 --2- 172.21.15.103:0/690059332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b257980480 0x55b25798a800 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f803c0ede10 tx=0x7f803c03b990 comp rx=0 tx=0).stop 2024-09-19T07:01:29.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.005+0000 7f8053ace740 1 -- 172.21.15.103:0/690059332 shutdown_connections 2024-09-19T07:01:29.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.005+0000 7f8053ace740 1 --2- 172.21.15.103:0/690059332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25798ad40 0x55b25798d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.005+0000 7f8053ace740 1 --2- 172.21.15.103:0/690059332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b257980480 0x55b25798a800 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.005+0000 7f8053ace740 1 --2- 172.21.15.103:0/690059332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 0x55b25797ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.005+0000 7f8053ace740 1 -- 172.21.15.103:0/690059332 >> 172.21.15.103:0/690059332 conn(0x55b25797c0f0 msgr2=0x55b25797c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:29.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.005+0000 7f8053ace740 1 -- 172.21.15.103:0/690059332 shutdown_connections 2024-09-19T07:01:29.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.005+0000 7f8053ace740 1 -- 172.21.15.103:0/690059332 wait complete. 2024-09-19T07:01:29.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.005+0000 7f8053ace740 1 -- start start 2024-09-19T07:01:29.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f8053ace740 1 Processor -- start 2024-09-19T07:01:29.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f8053ace740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 0x55b257987ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f8053ace740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b257980480 0x55b257988220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f8053ace740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b25798ad40 0x55b257988760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f8053ace740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b25798d6d0 con 0x55b25798ad40 2024-09-19T07:01:29.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f8053ace740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b25789ff10 con 0x55b25797fb60 2024-09-19T07:01:29.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f8053ace740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b25798d850 con 0x55b257980480 2024-09-19T07:01:29.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f8053ace740 10 client.0 _ll_register_callbacks cb 0x55b257868a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f804b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b25798ad40 0x55b257988760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f804b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b25798ad40 0x55b257988760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52562/0 (socket says 172.21.15.103:52562) 2024-09-19T07:01:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f804b7fe640 1 -- 172.21.15.103:0/668028245 learned_addr learned my addr 172.21.15.103:0/668028245 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f804affd640 1 --2- 172.21.15.103:0/668028245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 0x55b257987ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f804b7fe640 1 -- 172.21.15.103:0/668028245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b257980480 msgr2=0x55b257988220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f804b7fe640 1 --2- 172.21.15.103:0/668028245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b257980480 0x55b257988220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f804b7fe640 1 -- 172.21.15.103:0/668028245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 msgr2=0x55b257987ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f804b7fe640 1 --2- 172.21.15.103:0/668028245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 0x55b257987ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.006+0000 7f804b7fe640 1 -- 172.21.15.103:0/668028245 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2576566a0 con 0x55b25798ad40 2024-09-19T07:01:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.007+0000 7f804b7fe640 1 --2- 172.21.15.103:0/668028245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b25798ad40 0x55b257988760 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f8044011e60 tx=0x7f8044004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d37e3b2abe16cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.007+0000 7f80327fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.007+0000 7f80327fc640 1 -- 172.21.15.103:0/668028245 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044019070 con 0x55b25798ad40 2024-09-19T07:01:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.007+0000 7f80327fc640 1 -- 172.21.15.103:0/668028245 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f804400e640 con 0x55b25798ad40 2024-09-19T07:01:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.007+0000 7f8053ace740 1 -- 172.21.15.103:0/668028245 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f803c0bd100 con 0x55b25798ad40 2024-09-19T07:01:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.007+0000 7f80327fc640 1 -- 172.21.15.103:0/668028245 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f804402e050 con 0x55b25798ad40 2024-09-19T07:01:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.007+0000 7f8053ace740 10 client.19620 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.007+0000 7f8053ace740 1 -- 172.21.15.103:0/668028245 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b2579993b0 con 0x55b25798ad40 2024-09-19T07:01:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.007+0000 7f80327fc640 1 -- 172.21.15.103:0/668028245 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f803c0bd100 con 0x55b25798ad40 2024-09-19T07:01:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.007+0000 7f80327fc640 1 -- 172.21.15.103:0/668028245 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8044020680 con 0x55b25798ad40 2024-09-19T07:01:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.007+0000 7f8053ace740 10 client.19620 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.007+0000 7f8053ace740 10 client.19620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.008+0000 7f8053ace740 20 client.19620 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.008+0000 7f8053ace740 1 --2- 172.21.15.103:0/668028245 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b25799a910 0x55b25799cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.008+0000 7f8053ace740 4 client.19620 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.008+0000 7f8053ace740 1 -- 172.21.15.103:0/668028245 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2578f27f0 con 0x55b25799a910 2024-09-19T07:01:29.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.008+0000 7f804affd640 1 --2- 172.21.15.103:0/668028245 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b25799a910 0x55b25799cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:29.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.008+0000 7f80327fc640 10 client.19620 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:29.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.008+0000 7f804affd640 1 --2- 172.21.15.103:0/668028245 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b25799a910 0x55b25799cd70 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e9c4b13339aca51 server_cookie=4cf17efed1cc883c in_seq=0 out_seq=0 2024-09-19T07:01:29.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.009+0000 7f80327fc640 1 -- 172.21.15.103:0/668028245 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2578f27f0 con 0x55b25799a910 2024-09-19T07:01:29.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.009+0000 7f80327fc640 10 client.19620 handle_command_reply: tid=0 2024-09-19T07:01:29.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.009+0000 7f8053ace740 1 client.19620 shutdown 2024-09-19T07:01:29.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.009+0000 7f8053ace740 1 -- 172.21.15.103:0/668028245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b25798ad40 msgr2=0x55b257988760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:29.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.009+0000 7f8053ace740 1 --2- 172.21.15.103:0/668028245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b25798ad40 0x55b257988760 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f8044011e60 tx=0x7f8044004a40 comp rx=0 tx=0).stop 2024-09-19T07:01:29.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.009+0000 7f8053ace740 1 -- 172.21.15.103:0/668028245 shutdown_connections 2024-09-19T07:01:29.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.009+0000 7f8053ace740 1 --2- 172.21.15.103:0/668028245 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b25799a910 0x55b25799cd70 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.009+0000 7f8053ace740 1 --2- 172.21.15.103:0/668028245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b25798ad40 0x55b257988760 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.009+0000 7f8053ace740 1 --2- 172.21.15.103:0/668028245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b257980480 0x55b257988220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.009+0000 7f8053ace740 1 --2- 172.21.15.103:0/668028245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 0x55b257987ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.009+0000 7f8053ace740 1 -- 172.21.15.103:0/668028245 >> 172.21.15.103:0/668028245 conn(0x55b25797c0f0 msgr2=0x55b25797b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:29.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.009+0000 7f80327fc640 0 client.19620 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:29.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.009+0000 7f8053ace740 1 -- 172.21.15.103:0/668028245 shutdown_connections 2024-09-19T07:01:29.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.009+0000 7f8053ace740 1 -- 172.21.15.103:0/668028245 wait complete. 2024-09-19T07:01:29.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.009+0000 7f8053ace740 20 client.19620 trim_cache size 0 max 16384 2024-09-19T07:01:29.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.026+0000 7f8053ace740 1 Processor -- start 2024-09-19T07:01:29.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.026+0000 7f8053ace740 1 -- start start 2024-09-19T07:01:29.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f8053ace740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 0x55b257982610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f8053ace740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b257980480 0x55b257989b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f8053ace740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25798ad40 0x55b2579a2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f8053ace740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f804c1afa80 con 0x55b257980480 2024-09-19T07:01:29.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f8053ace740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f804c1b15c0 con 0x55b25797fb60 2024-09-19T07:01:29.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f8053ace740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f804c1b0150 con 0x55b25798ad40 2024-09-19T07:01:29.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f8043fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b257980480 0x55b257989b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:29.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f8043fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b257980480 0x55b257989b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52574/0 (socket says 172.21.15.103:52574) 2024-09-19T07:01:29.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f8043fff640 1 -- 172.21.15.103:0/1151943033 learned_addr learned my addr 172.21.15.103:0/1151943033 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:29.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f804affd640 1 --2- 172.21.15.103:0/1151943033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 0x55b257982610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:29.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f8043fff640 1 -- 172.21.15.103:0/1151943033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25798ad40 msgr2=0x55b2579a2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:29.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f8043fff640 1 --2- 172.21.15.103:0/1151943033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25798ad40 0x55b2579a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f8043fff640 1 -- 172.21.15.103:0/1151943033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 msgr2=0x55b257982610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:29.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f8043fff640 1 --2- 172.21.15.103:0/1151943033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 0x55b257982610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f8043fff640 1 -- 172.21.15.103:0/1151943033 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b25797bb90 con 0x55b257980480 2024-09-19T07:01:29.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f804affd640 1 --2- 172.21.15.103:0/1151943033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 0x55b257982610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:29.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.027+0000 7f8043fff640 1 --2- 172.21.15.103:0/1151943033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b257980480 0x55b257989b40 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f80340045b0 tx=0x7f803400cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbb9d26edd50f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:29.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.028+0000 7f8040ff9640 1 -- 172.21.15.103:0/1151943033 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034003d50 con 0x55b257980480 2024-09-19T07:01:29.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.028+0000 7f8040ff9640 1 -- 172.21.15.103:0/1151943033 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8034010910 con 0x55b257980480 2024-09-19T07:01:29.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.028+0000 7f8040ff9640 1 -- 172.21.15.103:0/1151943033 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803400e8c0 con 0x55b257980480 2024-09-19T07:01:29.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.028+0000 7f8053ace740 1 -- 172.21.15.103:0/1151943033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b257980480 msgr2=0x55b257989b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:29.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.028+0000 7f8053ace740 1 --2- 172.21.15.103:0/1151943033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b257980480 0x55b257989b40 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f80340045b0 tx=0x7f803400cb60 comp rx=0 tx=0).stop 2024-09-19T07:01:29.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.028+0000 7f8053ace740 1 -- 172.21.15.103:0/1151943033 shutdown_connections 2024-09-19T07:01:29.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.028+0000 7f8053ace740 1 --2- 172.21.15.103:0/1151943033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25798ad40 0x55b2579a2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.028+0000 7f8053ace740 1 --2- 172.21.15.103:0/1151943033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b257980480 0x55b257989b40 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.028+0000 7f8053ace740 1 --2- 172.21.15.103:0/1151943033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b25797fb60 0x55b257982610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.028+0000 7f8053ace740 1 -- 172.21.15.103:0/1151943033 >> 172.21.15.103:0/1151943033 conn(0x55b25797c0f0 msgr2=0x55b257989430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:29.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.028+0000 7f8053ace740 1 -- 172.21.15.103:0/1151943033 shutdown_connections 2024-09-19T07:01:29.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.028+0000 7f8053ace740 1 -- 172.21.15.103:0/1151943033 wait complete. 2024-09-19T07:01:29.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.029+0000 7f8053ace740 1 -- start start 2024-09-19T07:01:29.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.029+0000 7f8053ace740 1 Processor -- start 2024-09-19T07:01:29.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.029+0000 7f8053ace740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b25797fb60 0x55b257992e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.029+0000 7f8053ace740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b257980480 0x55b25798e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.029+0000 7f8053ace740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25798ad40 0x55b25798e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.029+0000 7f8053ace740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f804c1b0ef0 con 0x55b25797fb60 2024-09-19T07:01:29.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.029+0000 7f8053ace740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f804c1b0820 con 0x55b257980480 2024-09-19T07:01:29.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.029+0000 7f8053ace740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f804c1ada80 con 0x55b25798ad40 2024-09-19T07:01:29.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.029+0000 7f804affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b25797fb60 0x55b257992e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:29.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.029+0000 7f804affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b25797fb60 0x55b257992e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52590/0 (socket says 172.21.15.103:52590) 2024-09-19T07:01:29.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.029+0000 7f804affd640 1 -- 172.21.15.103:0/2190289253 learned_addr learned my addr 172.21.15.103:0/2190289253 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:29.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.029+0000 7f8053ace740 10 client.0 _ll_register_callbacks cb 0x7f8044014f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:29.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.029+0000 7f804b7fe640 1 --2- 172.21.15.103:0/2190289253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25798ad40 0x55b25798e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:29.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.029+0000 7f8043fff640 1 --2- 172.21.15.103:0/2190289253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b257980480 0x55b25798e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:29.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f804affd640 1 -- 172.21.15.103:0/2190289253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25798ad40 msgr2=0x55b25798e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:29.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f804affd640 1 --2- 172.21.15.103:0/2190289253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25798ad40 0x55b25798e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f804affd640 1 -- 172.21.15.103:0/2190289253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b257980480 msgr2=0x55b25798e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:29.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f804affd640 1 --2- 172.21.15.103:0/2190289253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b257980480 0x55b25798e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f804affd640 1 -- 172.21.15.103:0/2190289253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b25798ecc0 con 0x55b25797fb60 2024-09-19T07:01:29.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f8043fff640 1 --2- 172.21.15.103:0/2190289253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b257980480 0x55b25798e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:29.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f804b7fe640 1 --2- 172.21.15.103:0/2190289253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25798ad40 0x55b25798e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:29.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f804affd640 1 --2- 172.21.15.103:0/2190289253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b25797fb60 0x55b257992e90 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f803c005940 tx=0x7f803c0bc900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19cca614ae251d30 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:29.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f80327fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:29.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f80327fc640 1 -- 172.21.15.103:0/2190289253 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c049040 con 0x55b25797fb60 2024-09-19T07:01:29.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f80327fc640 1 -- 172.21.15.103:0/2190289253 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f803c0ee0f0 con 0x55b25797fb60 2024-09-19T07:01:29.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f8053ace740 1 -- 172.21.15.103:0/2190289253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8034010910 con 0x55b25797fb60 2024-09-19T07:01:29.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f80327fc640 1 -- 172.21.15.103:0/2190289253 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f803c0bd100 con 0x55b25797fb60 2024-09-19T07:01:29.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f8053ace740 10 client.19626 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:29.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f8053ace740 1 -- 172.21.15.103:0/2190289253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b257a05390 con 0x55b25797fb60 2024-09-19T07:01:29.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f80327fc640 1 -- 172.21.15.103:0/2190289253 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8034010910 con 0x55b25797fb60 2024-09-19T07:01:29.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f80327fc640 1 -- 172.21.15.103:0/2190289253 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f803c0b9070 con 0x55b25797fb60 2024-09-19T07:01:29.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f8053ace740 10 client.19626 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:29.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f8053ace740 10 client.19626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:29.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f8053ace740 20 client.19626 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:29.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f8053ace740 1 --2- 172.21.15.103:0/2190289253 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b257a05920 0x55b257a07d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:29.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f8053ace740 4 client.19626 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:01:29.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.030+0000 7f8053ace740 1 -- 172.21.15.103:0/2190289253 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b2578ecd00 con 0x55b257a05920 2024-09-19T07:01:29.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.031+0000 7f8043fff640 1 --2- 172.21.15.103:0/2190289253 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b257a05920 0x55b257a07d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:29.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.031+0000 7f80327fc640 10 client.19626 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:29.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.031+0000 7f8043fff640 1 --2- 172.21.15.103:0/2190289253 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b257a05920 0x55b257a07d10 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=592b962c9473906d server_cookie=dde3dc3dbd94d804 in_seq=0 out_seq=0 2024-09-19T07:01:29.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.032+0000 7f80327fc640 1 -- 172.21.15.103:0/2190289253 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b2578ecd00 con 0x55b257a05920 2024-09-19T07:01:29.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.032+0000 7f80327fc640 10 client.19626 handle_command_reply: tid=0 2024-09-19T07:01:29.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.032+0000 7f8053ace740 1 client.19626 shutdown 2024-09-19T07:01:29.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.032+0000 7f8053ace740 1 -- 172.21.15.103:0/2190289253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b25797fb60 msgr2=0x55b257992e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:29.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.032+0000 7f8053ace740 1 --2- 172.21.15.103:0/2190289253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b25797fb60 0x55b257992e90 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f803c005940 tx=0x7f803c0bc900 comp rx=0 tx=0).stop 2024-09-19T07:01:29.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.032+0000 7f8053ace740 1 -- 172.21.15.103:0/2190289253 shutdown_connections 2024-09-19T07:01:29.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.032+0000 7f8053ace740 1 --2- 172.21.15.103:0/2190289253 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b257a05920 0x55b257a07d10 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.032+0000 7f8053ace740 1 --2- 172.21.15.103:0/2190289253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25798ad40 0x55b25798e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.032+0000 7f8053ace740 1 --2- 172.21.15.103:0/2190289253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b257980480 0x55b25798e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.032+0000 7f8053ace740 1 --2- 172.21.15.103:0/2190289253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b25797fb60 0x55b257992e90 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.032+0000 7f8053ace740 1 -- 172.21.15.103:0/2190289253 >> 172.21.15.103:0/2190289253 conn(0x55b25797c0f0 msgr2=0x55b25797aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:29.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.032+0000 7f80327fc640 0 client.19626 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:29.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.032+0000 7f8053ace740 1 -- 172.21.15.103:0/2190289253 shutdown_connections 2024-09-19T07:01:29.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.032+0000 7f8053ace740 1 -- 172.21.15.103:0/2190289253 wait complete. 2024-09-19T07:01:29.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.032+0000 7f8053ace740 20 client.19626 trim_cache size 0 max 16384 2024-09-19T07:01:29.039 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:01:29.039 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:01:29.039 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "26ccc08c-bd5c-470d-b4d0-b1c57a1d0cdf", 2024-09-19T07:01:29.039 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:01:29.040 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:01:29.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.033+0000 7f80516f5640 1 -- 172.21.15.103:0/3698107123 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80380a33b0 msgr2=0x7f80380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:29.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.033+0000 7f80516f5640 1 --2- 172.21.15.103:0/3698107123 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80380a33b0 0x7f80380a5870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8044008fd0 tx=0x7f8044004ab0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.033+0000 7f80516f5640 1 -- 172.21.15.103:0/3698107123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804c100e20 msgr2=0x7f804c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:29.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.033+0000 7f80516f5640 1 --2- 172.21.15.103:0/3698107123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804c100e20 0x7f804c0fcc80 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f803c006010 tx=0x7f803c003c80 comp rx=0 tx=0).stop 2024-09-19T07:01:29.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.033+0000 7f80516f5640 1 -- 172.21.15.103:0/3698107123 shutdown_connections 2024-09-19T07:01:29.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.033+0000 7f80516f5640 1 --2- 172.21.15.103:0/3698107123 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f80380a33b0 0x7f80380a5870 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.033+0000 7f80516f5640 1 --2- 172.21.15.103:0/3698107123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f804c102ef0 0x7f804c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.033+0000 7f80516f5640 1 --2- 172.21.15.103:0/3698107123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f804c102570 0x7f804c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.033+0000 7f80516f5640 1 --2- 172.21.15.103:0/3698107123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f804c100e20 0x7f804c0fcc80 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:29.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.033+0000 7f80516f5640 1 -- 172.21.15.103:0/3698107123 >> 172.21.15.103:0/3698107123 conn(0x7f804c0fc7d0 msgr2=0x7f804c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:29.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.033+0000 7f80516f5640 1 -- 172.21.15.103:0/3698107123 shutdown_connections 2024-09-19T07:01:29.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:29.033+0000 7f80516f5640 1 -- 172.21.15.103:0/3698107123 wait complete. 2024-09-19T07:01:29.720 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:01:30.019 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:30.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:30 smithi174 ceph-mon[35637]: pgmap v382: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-09-19T07:01:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:30 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T07:01:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:30 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:01:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:30 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T07:01:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:30 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:01:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:30 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T07:01:30.340 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:30 smithi103 ceph-mon[31158]: pgmap v382: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-09-19T07:01:30.340 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:30 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T07:01:30.340 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:30 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:01:30.340 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:30 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T07:01:30.340 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:30 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:01:30.341 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:30 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T07:01:30.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.420+0000 7fbed0761640 1 -- 172.21.15.103:0/3338778344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbec810c170 msgr2=0x7fbec810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:30.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.420+0000 7fbed0761640 1 --2- 172.21.15.103:0/3338778344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbec810c170 0x7fbec810e610 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009900 tx=0x7fbeb802a480 comp rx=0 tx=0).stop 2024-09-19T07:01:30.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.420+0000 7fbed0761640 1 -- 172.21.15.103:0/3338778344 shutdown_connections 2024-09-19T07:01:30.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.420+0000 7fbed0761640 1 --2- 172.21.15.103:0/3338778344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbec810c170 0x7fbec810e610 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.420+0000 7fbed0761640 1 --2- 172.21.15.103:0/3338778344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbec8103660 0x7fbec810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.420+0000 7fbed0761640 1 --2- 172.21.15.103:0/3338778344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbec8102d40 0x7fbec8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.420+0000 7fbed0761640 1 -- 172.21.15.103:0/3338778344 >> 172.21.15.103:0/3338778344 conn(0x7fbec80fc7a0 msgr2=0x7fbec80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.420+0000 7fbed0761640 1 -- 172.21.15.103:0/3338778344 shutdown_connections 2024-09-19T07:01:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.420+0000 7fbed0761640 1 -- 172.21.15.103:0/3338778344 wait complete. 2024-09-19T07:01:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbed0761640 1 Processor -- start 2024-09-19T07:01:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbed0761640 1 -- start start 2024-09-19T07:01:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbed0761640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbec8102d40 0x7fbec8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbed0761640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbec8103660 0x7fbec8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbed0761640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbec810c170 0x7fbec819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbed0761640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbec8111fb0 con 0x7fbec8102d40 2024-09-19T07:01:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbed0761640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbec8111e30 con 0x7fbec8103660 2024-09-19T07:01:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbed0761640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbec8112130 con 0x7fbec810c170 2024-09-19T07:01:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbece4fd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbec8102d40 0x7fbec8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbece4fd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbec8102d40 0x7fbec8198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52602/0 (socket says 172.21.15.103:52602) 2024-09-19T07:01:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbece4fd640 1 -- 172.21.15.103:0/716181932 learned_addr learned my addr 172.21.15.103:0/716181932 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbecdcfc640 1 --2- 172.21.15.103:0/716181932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbec8103660 0x7fbec8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:30.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbece4fd640 1 -- 172.21.15.103:0/716181932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbec810c170 msgr2=0x7fbec819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:30.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbece4fd640 1 --2- 172.21.15.103:0/716181932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbec810c170 0x7fbec819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbece4fd640 1 -- 172.21.15.103:0/716181932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbec8103660 msgr2=0x7fbec8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:30.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbece4fd640 1 --2- 172.21.15.103:0/716181932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbec8103660 0x7fbec8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbece4fd640 1 -- 172.21.15.103:0/716181932 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec819d3e0 con 0x7fbec8102d40 2024-09-19T07:01:30.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbecdcfc640 1 --2- 172.21.15.103:0/716181932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbec8103660 0x7fbec8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:30.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.421+0000 7fbece4fd640 1 --2- 172.21.15.103:0/716181932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbec8102d40 0x7fbec8198430 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fbec000d950 tx=0x7fbec000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:30.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.422+0000 7fbec77fe640 1 -- 172.21.15.103:0/716181932 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0017070 con 0x7fbec8102d40 2024-09-19T07:01:30.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.422+0000 7fbec77fe640 1 -- 172.21.15.103:0/716181932 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbec0009c30 con 0x7fbec8102d40 2024-09-19T07:01:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.422+0000 7fbed0761640 1 -- 172.21.15.103:0/716181932 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec819d6d0 con 0x7fbec8102d40 2024-09-19T07:01:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.422+0000 7fbec77fe640 1 -- 172.21.15.103:0/716181932 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec000f6b0 con 0x7fbec8102d40 2024-09-19T07:01:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.422+0000 7fbed0761640 1 -- 172.21.15.103:0/716181932 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec81a4fb0 con 0x7fbec8102d40 2024-09-19T07:01:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.424+0000 7fbec77fe640 1 -- 172.21.15.103:0/716181932 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fbec00109d0 con 0x7fbec8102d40 2024-09-19T07:01:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.424+0000 7fbed2b3a740 1 Processor -- start 2024-09-19T07:01:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.424+0000 7fbed2b3a740 1 -- start start 2024-09-19T07:01:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.424+0000 7fbed2b3a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eee24a0 0x55ca7eee2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.424+0000 7fbed2b3a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee2dc0 0x55ca7eeed140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbed2b3a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eeed680 0x55ca7eeefa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbed2b3a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca7ee482e0 con 0x55ca7eee2dc0 2024-09-19T07:01:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbed2b3a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ca7edefce0 con 0x55ca7eee24a0 2024-09-19T07:01:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbed2b3a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ca7ee442a0 con 0x55ca7eeed680 2024-09-19T07:01:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbec77fe640 1 --2- 172.21.15.103:0/716181932 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbeb00a33b0 0x7fbeb00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbece4fd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee2dc0 0x55ca7eeed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:30.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbece4fd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee2dc0 0x55ca7eeed140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52618/0 (socket says 172.21.15.103:52618) 2024-09-19T07:01:30.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbece4fd640 1 -- 172.21.15.103:0/688085789 learned_addr learned my addr 172.21.15.103:0/688085789 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:30.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbececfe640 1 --2- 172.21.15.103:0/688085789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eee24a0 0x55ca7eee2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:30.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbececfe640 1 --2- 172.21.15.103:0/716181932 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbeb00a33b0 0x7fbeb00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:30.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbecdcfc640 1 --2- 172.21.15.103:0/688085789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eeed680 0x55ca7eeefa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:30.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbec77fe640 1 -- 172.21.15.103:0/716181932 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7fbec00cfa40 con 0x7fbec8102d40 2024-09-19T07:01:30.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbece4fd640 1 -- 172.21.15.103:0/688085789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eeed680 msgr2=0x55ca7eeefa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:30.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbece4fd640 1 --2- 172.21.15.103:0/688085789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eeed680 0x55ca7eeefa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbece4fd640 1 -- 172.21.15.103:0/688085789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eee24a0 msgr2=0x55ca7eee2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:30.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbece4fd640 1 --2- 172.21.15.103:0/688085789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eee24a0 0x55ca7eee2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbece4fd640 1 -- 172.21.15.103:0/688085789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca7edd4440 con 0x55ca7eee2dc0 2024-09-19T07:01:30.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbecdcfc640 1 --2- 172.21.15.103:0/688085789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eeed680 0x55ca7eeefa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:30.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbececfe640 1 --2- 172.21.15.103:0/688085789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eee24a0 0x55ca7eee2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:01:30.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbece4fd640 1 --2- 172.21.15.103:0/688085789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee2dc0 0x55ca7eeed140 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fbec00a4440 tx=0x7fbec0010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ac64be43640d8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:30.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.425+0000 7fbececfe640 1 --2- 172.21.15.103:0/716181932 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbeb00a33b0 0x7fbeb00a5870 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbeb802a9a0 tx=0x7fbeb8009260 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:30.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.426+0000 7fbec4ff9640 1 -- 172.21.15.103:0/688085789 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0020070 con 0x55ca7eee2dc0 2024-09-19T07:01:30.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.426+0000 7fbec4ff9640 1 -- 172.21.15.103:0/688085789 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbec0009de0 con 0x55ca7eee2dc0 2024-09-19T07:01:30.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.426+0000 7fbec4ff9640 1 -- 172.21.15.103:0/688085789 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0098370 con 0x55ca7eee2dc0 2024-09-19T07:01:30.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.426+0000 7fbed2b3a740 1 -- 172.21.15.103:0/688085789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee2dc0 msgr2=0x55ca7eeed140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.426+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/688085789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee2dc0 0x55ca7eeed140 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fbec00a4440 tx=0x7fbec0010d70 comp rx=0 tx=0).stop 2024-09-19T07:01:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.426+0000 7fbed2b3a740 1 -- 172.21.15.103:0/688085789 shutdown_connections 2024-09-19T07:01:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.426+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/688085789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eeed680 0x55ca7eeefa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.426+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/688085789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee2dc0 0x55ca7eeed140 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.426+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/688085789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eee24a0 0x55ca7eee2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.426+0000 7fbed2b3a740 1 -- 172.21.15.103:0/688085789 >> 172.21.15.103:0/688085789 conn(0x55ca7eedea30 msgr2=0x55ca7eedee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.426+0000 7fbed2b3a740 1 -- 172.21.15.103:0/688085789 shutdown_connections 2024-09-19T07:01:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.426+0000 7fbed2b3a740 1 -- 172.21.15.103:0/688085789 wait complete. 2024-09-19T07:01:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.427+0000 7fbed2b3a740 1 -- start start 2024-09-19T07:01:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.427+0000 7fbed2b3a740 1 Processor -- start 2024-09-19T07:01:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.427+0000 7fbed2b3a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee24a0 0x55ca7eeed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.427+0000 7fbed2b3a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eee2dc0 0x55ca7eeea2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.427+0000 7fbed2b3a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eeed680 0x55ca7eeea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.427+0000 7fbed2b3a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca7eef01f0 con 0x55ca7eee24a0 2024-09-19T07:01:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.427+0000 7fbed2b3a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ca7ee03050 con 0x55ca7eee2dc0 2024-09-19T07:01:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.427+0000 7fbed2b3a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ca7eef0370 con 0x55ca7eeed680 2024-09-19T07:01:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.427+0000 7fbed2b3a740 10 client.0 _ll_register_callbacks cb 0x55ca7ee43160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.427+0000 7fbece4fd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee24a0 0x55ca7eeed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.427+0000 7fbece4fd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee24a0 0x55ca7eeed030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52630/0 (socket says 172.21.15.103:52630) 2024-09-19T07:01:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.427+0000 7fbece4fd640 1 -- 172.21.15.103:0/3306187081 learned_addr learned my addr 172.21.15.103:0/3306187081 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbececfe640 1 --2- 172.21.15.103:0/3306187081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eeed680 0x55ca7eeea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbece4fd640 1 -- 172.21.15.103:0/3306187081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eeed680 msgr2=0x55ca7eeea880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbece4fd640 1 --2- 172.21.15.103:0/3306187081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eeed680 0x55ca7eeea880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbece4fd640 1 -- 172.21.15.103:0/3306187081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eee2dc0 msgr2=0x55ca7eeea2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbece4fd640 1 --2- 172.21.15.103:0/3306187081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eee2dc0 0x55ca7eeea2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbece4fd640 1 -- 172.21.15.103:0/3306187081 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca7ebb93c0 con 0x55ca7eee24a0 2024-09-19T07:01:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbececfe640 1 --2- 172.21.15.103:0/3306187081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eeed680 0x55ca7eeea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbece4fd640 1 --2- 172.21.15.103:0/3306187081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee24a0 0x55ca7eeed030 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fbec00cd770 tx=0x7fbec00cd620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e21022099a1e119 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbeadffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbeadffb640 1 -- 172.21.15.103:0/3306187081 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0020070 con 0x55ca7eee24a0 2024-09-19T07:01:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbeadffb640 1 -- 172.21.15.103:0/3306187081 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7fbec009bf60 con 0x55ca7eee24a0 2024-09-19T07:01:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbed2b3a740 1 -- 172.21.15.103:0/3306187081 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbec0009de0 con 0x55ca7eee24a0 2024-09-19T07:01:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbeadffb640 1 -- 172.21.15.103:0/3306187081 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbec00a4970 con 0x55ca7eee24a0 2024-09-19T07:01:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbed2b3a740 10 client.19638 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbed2b3a740 1 -- 172.21.15.103:0/3306187081 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ca7eefbc00 con 0x55ca7eee24a0 2024-09-19T07:01:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbeadffb640 1 -- 172.21.15.103:0/3306187081 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbec0009de0 con 0x55ca7eee24a0 2024-09-19T07:01:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbeadffb640 1 -- 172.21.15.103:0/3306187081 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbec00a4bc0 con 0x55ca7eee24a0 2024-09-19T07:01:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbed2b3a740 10 client.19638 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbed2b3a740 10 client.19638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbed2b3a740 20 client.19638 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.428+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/3306187081 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ca7eefcf50 0x55ca7eeff340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.429+0000 7fbed2b3a740 4 client.19638 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.429+0000 7fbed2b3a740 1 -- 172.21.15.103:0/3306187081 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca7ee55850 con 0x55ca7eefcf50 2024-09-19T07:01:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.429+0000 7fbecdcfc640 1 --2- 172.21.15.103:0/3306187081 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ca7eefcf50 0x55ca7eeff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.429+0000 7fbeadffb640 10 client.19638 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.429+0000 7fbecdcfc640 1 --2- 172.21.15.103:0/3306187081 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ca7eefcf50 0x55ca7eeff340 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=503b2e29f9bc2f09 server_cookie=e9b1055ea2eeaaa9 in_seq=0 out_seq=0 2024-09-19T07:01:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.430+0000 7fbeadffb640 1 -- 172.21.15.103:0/3306187081 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca7ee55850 con 0x55ca7eefcf50 2024-09-19T07:01:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.430+0000 7fbeadffb640 10 client.19638 handle_command_reply: tid=0 2024-09-19T07:01:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.430+0000 7fbed2b3a740 1 client.19638 shutdown 2024-09-19T07:01:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.430+0000 7fbed2b3a740 1 -- 172.21.15.103:0/3306187081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee24a0 msgr2=0x55ca7eeed030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.430+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/3306187081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee24a0 0x55ca7eeed030 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fbec00cd770 tx=0x7fbec00cd620 comp rx=0 tx=0).stop 2024-09-19T07:01:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.430+0000 7fbed2b3a740 1 -- 172.21.15.103:0/3306187081 shutdown_connections 2024-09-19T07:01:30.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.430+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/3306187081 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ca7eefcf50 0x55ca7eeff340 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.430+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/3306187081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eeed680 0x55ca7eeea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.430+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/3306187081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eee2dc0 0x55ca7eeea2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.430+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/3306187081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee24a0 0x55ca7eeed030 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.430+0000 7fbeadffb640 0 client.19638 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:30.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.430+0000 7fbed2b3a740 1 -- 172.21.15.103:0/3306187081 >> 172.21.15.103:0/3306187081 conn(0x55ca7eedea30 msgr2=0x55ca7eee4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:30.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.430+0000 7fbed2b3a740 1 -- 172.21.15.103:0/3306187081 shutdown_connections 2024-09-19T07:01:30.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.430+0000 7fbed2b3a740 1 -- 172.21.15.103:0/3306187081 wait complete. 2024-09-19T07:01:30.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.430+0000 7fbed2b3a740 20 client.19638 trim_cache size 0 max 16384 2024-09-19T07:01:30.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:30 smithi134 ceph-mon[35356]: pgmap v382: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-09-19T07:01:30.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:30 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T07:01:30.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:30 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:01:30.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:30 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T07:01:30.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:30 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:01:30.439 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:30 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T07:01:30.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.447+0000 7fbed2b3a740 1 Processor -- start 2024-09-19T07:01:30.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.447+0000 7fbed2b3a740 1 -- start start 2024-09-19T07:01:30.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.447+0000 7fbed2b3a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eee24a0 0x55ca7ef02370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.447+0000 7fbed2b3a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eee2dc0 0x55ca7ef028b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.447+0000 7fbed2b3a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eeed680 0x55ca7eee7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.447+0000 7fbed2b3a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81ada60 con 0x55ca7eeed680 2024-09-19T07:01:30.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.447+0000 7fbed2b3a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81af5a0 con 0x55ca7eee24a0 2024-09-19T07:01:30.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.447+0000 7fbed2b3a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81ae130 con 0x55ca7eee2dc0 2024-09-19T07:01:30.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.447+0000 7fbececfe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eeed680 0x55ca7eee7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:30.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.447+0000 7fbececfe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eeed680 0x55ca7eee7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52642/0 (socket says 172.21.15.103:52642) 2024-09-19T07:01:30.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.447+0000 7fbececfe640 1 -- 172.21.15.103:0/3547072065 learned_addr learned my addr 172.21.15.103:0/3547072065 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:30.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.447+0000 7fbecdcfc640 1 --2- 172.21.15.103:0/3547072065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eee2dc0 0x55ca7ef028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:30.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.447+0000 7fbece4fd640 1 --2- 172.21.15.103:0/3547072065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eee24a0 0x55ca7ef02370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:30.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.448+0000 7fbececfe640 1 -- 172.21.15.103:0/3547072065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eee2dc0 msgr2=0x55ca7ef028b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:30.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.448+0000 7fbececfe640 1 --2- 172.21.15.103:0/3547072065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eee2dc0 0x55ca7ef028b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.448+0000 7fbececfe640 1 -- 172.21.15.103:0/3547072065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eee24a0 msgr2=0x55ca7ef02370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:30.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.448+0000 7fbececfe640 1 --2- 172.21.15.103:0/3547072065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eee24a0 0x55ca7ef02370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.448+0000 7fbececfe640 1 -- 172.21.15.103:0/3547072065 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca7eeeabf0 con 0x55ca7eeed680 2024-09-19T07:01:30.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.448+0000 7fbece4fd640 1 --2- 172.21.15.103:0/3547072065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eee24a0 0x55ca7ef02370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:30.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.448+0000 7fbececfe640 1 --2- 172.21.15.103:0/3547072065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eeed680 0x55ca7eee7540 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8031b20 tx=0x7fbeb8037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9f697eb9139c72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:30.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.448+0000 7fbeaffff640 1 -- 172.21.15.103:0/3547072065 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8035070 con 0x55ca7eeed680 2024-09-19T07:01:30.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.448+0000 7fbeaffff640 1 -- 172.21.15.103:0/3547072065 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeb8050070 con 0x55ca7eeed680 2024-09-19T07:01:30.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.448+0000 7fbeaffff640 1 -- 172.21.15.103:0/3547072065 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb803ca10 con 0x55ca7eeed680 2024-09-19T07:01:30.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 -- 172.21.15.103:0/3547072065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eeed680 msgr2=0x55ca7eee7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:30.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/3547072065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eeed680 0x55ca7eee7540 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8031b20 tx=0x7fbeb8037000 comp rx=0 tx=0).stop 2024-09-19T07:01:30.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 -- 172.21.15.103:0/3547072065 shutdown_connections 2024-09-19T07:01:30.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/3547072065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eeed680 0x55ca7eee7540 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/3547072065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eee2dc0 0x55ca7ef028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/3547072065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eee24a0 0x55ca7ef02370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 -- 172.21.15.103:0/3547072065 >> 172.21.15.103:0/3547072065 conn(0x55ca7eedea30 msgr2=0x55ca7eedda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:30.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 -- 172.21.15.103:0/3547072065 shutdown_connections 2024-09-19T07:01:30.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 -- 172.21.15.103:0/3547072065 wait complete. 2024-09-19T07:01:30.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 -- start start 2024-09-19T07:01:30.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 Processor -- start 2024-09-19T07:01:30.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee24a0 0x55ca7ef01c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eee2dc0 0x55ca7ef02140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eeed680 0x55ca7ef5e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81aeed0 con 0x55ca7eee24a0 2024-09-19T07:01:30.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81ae800 con 0x55ca7eeed680 2024-09-19T07:01:30.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81aba60 con 0x55ca7eee2dc0 2024-09-19T07:01:30.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbed2b3a740 10 client.0 _ll_register_callbacks cb 0x7fbeb8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:30.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbece4fd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee24a0 0x55ca7ef01c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:30.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbece4fd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee24a0 0x55ca7ef01c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52652/0 (socket says 172.21.15.103:52652) 2024-09-19T07:01:30.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.449+0000 7fbece4fd640 1 -- 172.21.15.103:0/4252919091 learned_addr learned my addr 172.21.15.103:0/4252919091 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:30.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.450+0000 7fbececfe640 1 --2- 172.21.15.103:0/4252919091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eeed680 0x55ca7ef5e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:30.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.450+0000 7fbecdcfc640 1 --2- 172.21.15.103:0/4252919091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eee2dc0 0x55ca7ef02140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:30.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.450+0000 7fbece4fd640 1 -- 172.21.15.103:0/4252919091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eee2dc0 msgr2=0x55ca7ef02140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:30.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.450+0000 7fbece4fd640 1 --2- 172.21.15.103:0/4252919091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eee2dc0 0x55ca7ef02140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.450+0000 7fbece4fd640 1 -- 172.21.15.103:0/4252919091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eeed680 msgr2=0x55ca7ef5e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:30.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.450+0000 7fbece4fd640 1 --2- 172.21.15.103:0/4252919091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eeed680 0x55ca7ef5e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.450+0000 7fbece4fd640 1 -- 172.21.15.103:0/4252919091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca7ef5eef0 con 0x55ca7eee24a0 2024-09-19T07:01:30.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.450+0000 7fbece4fd640 1 --2- 172.21.15.103:0/4252919091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee24a0 0x55ca7ef01c00 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fbec0010fd0 tx=0x7fbec000b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3ee190ed9bbab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:30.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.450+0000 7fbeadffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:30.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.450+0000 7fbeadffb640 1 -- 172.21.15.103:0/4252919091 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0020070 con 0x55ca7eee24a0 2024-09-19T07:01:30.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.450+0000 7fbeadffb640 1 -- 172.21.15.103:0/4252919091 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7fbec009c160 con 0x55ca7eee24a0 2024-09-19T07:01:30.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.450+0000 7fbed2b3a740 1 -- 172.21.15.103:0/4252919091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeb8050070 con 0x55ca7eee24a0 2024-09-19T07:01:30.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.450+0000 7fbeadffb640 1 -- 172.21.15.103:0/4252919091 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbec009bad0 con 0x55ca7eee24a0 2024-09-19T07:01:30.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.450+0000 7fbed2b3a740 10 client.19644 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:30.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.450+0000 7fbed2b3a740 1 -- 172.21.15.103:0/4252919091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ca7ef67490 con 0x55ca7eee24a0 2024-09-19T07:01:30.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.450+0000 7fbeadffb640 1 -- 172.21.15.103:0/4252919091 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeb8050070 con 0x55ca7eee24a0 2024-09-19T07:01:30.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.451+0000 7fbeadffb640 1 -- 172.21.15.103:0/4252919091 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbec00ba380 con 0x55ca7eee24a0 2024-09-19T07:01:30.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.451+0000 7fbed2b3a740 10 client.19644 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:30.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.451+0000 7fbed2b3a740 10 client.19644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:30.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.451+0000 7fbed2b3a740 20 client.19644 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:30.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.451+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/4252919091 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ca7ef679c0 0x55ca7ef69db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:30.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.451+0000 7fbed2b3a740 4 client.19644 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:01:30.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.451+0000 7fbed2b3a740 1 -- 172.21.15.103:0/4252919091 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ca7ee4fd60 con 0x55ca7ef679c0 2024-09-19T07:01:30.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.451+0000 7fbecdcfc640 1 --2- 172.21.15.103:0/4252919091 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ca7ef679c0 0x55ca7ef69db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:30.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.451+0000 7fbecdcfc640 1 --2- 172.21.15.103:0/4252919091 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ca7ef679c0 0x55ca7ef69db0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=434336ded6830368 server_cookie=b640871fb3d144a in_seq=0 out_seq=0 2024-09-19T07:01:30.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.451+0000 7fbeadffb640 10 client.19644 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:30.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.451+0000 7fbeadffb640 1 -- 172.21.15.103:0/4252919091 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ca7ee4fd60 con 0x55ca7ef679c0 2024-09-19T07:01:30.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.451+0000 7fbeadffb640 10 client.19644 handle_command_reply: tid=0 2024-09-19T07:01:30.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.451+0000 7fbed2b3a740 1 client.19644 shutdown 2024-09-19T07:01:30.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed2b3a740 1 -- 172.21.15.103:0/4252919091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee24a0 msgr2=0x55ca7ef01c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:30.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/4252919091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee24a0 0x55ca7ef01c00 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fbec0010fd0 tx=0x7fbec000b530 comp rx=0 tx=0).stop 2024-09-19T07:01:30.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed2b3a740 1 -- 172.21.15.103:0/4252919091 shutdown_connections 2024-09-19T07:01:30.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/4252919091 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ca7ef679c0 0x55ca7ef69db0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/4252919091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7eeed680 0x55ca7ef5e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/4252919091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca7eee2dc0 0x55ca7ef02140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed2b3a740 1 --2- 172.21.15.103:0/4252919091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7eee24a0 0x55ca7ef01c00 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed2b3a740 1 -- 172.21.15.103:0/4252919091 >> 172.21.15.103:0/4252919091 conn(0x55ca7eedea30 msgr2=0x55ca7eeebb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:30.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbeadffb640 0 client.19644 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:30.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed2b3a740 1 -- 172.21.15.103:0/4252919091 shutdown_connections 2024-09-19T07:01:30.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed2b3a740 1 -- 172.21.15.103:0/4252919091 wait complete. 2024-09-19T07:01:30.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed2b3a740 20 client.19644 trim_cache size 0 max 16384 2024-09-19T07:01:30.459 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:01:30.459 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:01:30.459 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:01:30.459 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:01:30.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed0761640 1 -- 172.21.15.103:0/716181932 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbeb00a33b0 msgr2=0x7fbeb00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:30.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed0761640 1 --2- 172.21.15.103:0/716181932 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbeb00a33b0 0x7fbeb00a5870 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbeb802a9a0 tx=0x7fbeb8009260 comp rx=0 tx=0).stop 2024-09-19T07:01:30.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed0761640 1 -- 172.21.15.103:0/716181932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbec8102d40 msgr2=0x7fbec8198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:30.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed0761640 1 --2- 172.21.15.103:0/716181932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbec8102d40 0x7fbec8198430 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fbec000d950 tx=0x7fbec000de10 comp rx=0 tx=0).stop 2024-09-19T07:01:30.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed0761640 1 -- 172.21.15.103:0/716181932 shutdown_connections 2024-09-19T07:01:30.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed0761640 1 --2- 172.21.15.103:0/716181932 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbeb00a33b0 0x7fbeb00a5870 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed0761640 1 --2- 172.21.15.103:0/716181932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbec810c170 0x7fbec819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed0761640 1 --2- 172.21.15.103:0/716181932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbec8103660 0x7fbec8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed0761640 1 --2- 172.21.15.103:0/716181932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbec8102d40 0x7fbec8198430 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:30.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed0761640 1 -- 172.21.15.103:0/716181932 >> 172.21.15.103:0/716181932 conn(0x7fbec80fc7a0 msgr2=0x7fbec80fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:30.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed0761640 1 -- 172.21.15.103:0/716181932 shutdown_connections 2024-09-19T07:01:30.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:30.452+0000 7fbed0761640 1 -- 172.21.15.103:0/716181932 wait complete. 2024-09-19T07:01:31.117 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:01:31.117 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:01:31.365 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:31.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 -- 172.21.15.103:0/248007847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62181082a0 msgr2=0x7f621810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 --2- 172.21.15.103:0/248007847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62181082a0 0x7f621810ec40 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f6208009930 tx=0x7f620802a440 comp rx=0 tx=0).stop 2024-09-19T07:01:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 -- 172.21.15.103:0/248007847 shutdown_connections 2024-09-19T07:01:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 --2- 172.21.15.103:0/248007847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62181082a0 0x7f621810ec40 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 --2- 172.21.15.103:0/248007847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62181036e0 0x7f6218107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 --2- 172.21.15.103:0/248007847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6218102d30 0x7f6218103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 -- 172.21.15.103:0/248007847 >> 172.21.15.103:0/248007847 conn(0x7f62180fc7d0 msgr2=0x7f62180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 -- 172.21.15.103:0/248007847 shutdown_connections 2024-09-19T07:01:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 -- 172.21.15.103:0/248007847 wait complete. 2024-09-19T07:01:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 Processor -- start 2024-09-19T07:01:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 -- start start 2024-09-19T07:01:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218102d30 0x7f62181985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:31.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62181036e0 0x7f6218198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:31.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62181082a0 0x7f621819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:31.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6218112100 con 0x7f62181036e0 2024-09-19T07:01:31.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6218111f80 con 0x7f6218102d30 2024-09-19T07:01:31.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.795+0000 7f62207ad640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6218112280 con 0x7f62181082a0 2024-09-19T07:01:31.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.796+0000 7f621dd48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62181036e0 0x7f6218198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:31.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.796+0000 7f621dd48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62181036e0 0x7f6218198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52666/0 (socket says 172.21.15.103:52666) 2024-09-19T07:01:31.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.796+0000 7f621dd48640 1 -- 172.21.15.103:0/1348879278 learned_addr learned my addr 172.21.15.103:0/1348879278 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:31.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.796+0000 7f621e549640 1 --2- 172.21.15.103:0/1348879278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218102d30 0x7f62181985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:31.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.796+0000 7f621ed4a640 1 --2- 172.21.15.103:0/1348879278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62181082a0 0x7f621819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:31.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.796+0000 7f621dd48640 1 -- 172.21.15.103:0/1348879278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62181082a0 msgr2=0x7f621819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:31.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.796+0000 7f621dd48640 1 --2- 172.21.15.103:0/1348879278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62181082a0 0x7f621819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:31.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.796+0000 7f621dd48640 1 -- 172.21.15.103:0/1348879278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218102d30 msgr2=0x7f62181985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:31.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.796+0000 7f621dd48640 1 --2- 172.21.15.103:0/1348879278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218102d30 0x7f62181985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:31.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.796+0000 7f621dd48640 1 -- 172.21.15.103:0/1348879278 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621819d550 con 0x7f62181036e0 2024-09-19T07:01:31.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.796+0000 7f621dd48640 1 --2- 172.21.15.103:0/1348879278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62181036e0 0x7f6218198ae0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f620c006fd0 tx=0x7f620c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:31.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.796+0000 7f62177fe640 1 -- 172.21.15.103:0/1348879278 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c011830 con 0x7f62181036e0 2024-09-19T07:01:31.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.796+0000 7f62207ad640 1 -- 172.21.15.103:0/1348879278 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621819d840 con 0x7f62181036e0 2024-09-19T07:01:31.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.796+0000 7f62177fe640 1 -- 172.21.15.103:0/1348879278 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f620c016070 con 0x7f62181036e0 2024-09-19T07:01:31.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.796+0000 7f62177fe640 1 -- 172.21.15.103:0/1348879278 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c00f2e0 con 0x7f62181036e0 2024-09-19T07:01:31.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.796+0000 7f62207ad640 1 -- 172.21.15.103:0/1348879278 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62181a5120 con 0x7f62181036e0 2024-09-19T07:01:31.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.797+0000 7f62207ad640 1 -- 172.21.15.103:0/1348879278 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6218103110 con 0x7f62181036e0 2024-09-19T07:01:31.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.799+0000 7f62177fe640 1 -- 172.21.15.103:0/1348879278 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f620c01b460 con 0x7f62181036e0 2024-09-19T07:01:31.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.799+0000 7f62177fe640 1 --2- 172.21.15.103:0/1348879278 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62000a33b0 0x7f62000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:31.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.799+0000 7f621e549640 1 --2- 172.21.15.103:0/1348879278 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62000a33b0 0x7f62000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:31.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.799+0000 7f62177fe640 1 -- 172.21.15.103:0/1348879278 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f620c0cf8d0 con 0x7f62181036e0 2024-09-19T07:01:31.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.800+0000 7f621e549640 1 --2- 172.21.15.103:0/1348879278 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62000a33b0 0x7f62000a5870 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6210005fd0 tx=0x7f6210009680 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:31.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:31.803+0000 7f62177fe640 1 -- 172.21.15.103:0/1348879278 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f620c09c3a0 con 0x7f62181036e0 2024-09-19T07:01:32.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:32.010+0000 7f62207ad640 1 -- 172.21.15.103:0/1348879278 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6218199900 con 0x7f62181036e0 2024-09-19T07:01:32.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:32.011+0000 7f62177fe640 1 -- 172.21.15.103:0/1348879278 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f6218199900 con 0x7f62181036e0 2024-09-19T07:01:32.012 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:01:32.012 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:01:32.013 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:01:32.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:32.013+0000 7f62207ad640 1 -- 172.21.15.103:0/1348879278 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62000a33b0 msgr2=0x7f62000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:32.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:32.013+0000 7f62207ad640 1 --2- 172.21.15.103:0/1348879278 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62000a33b0 0x7f62000a5870 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6210005fd0 tx=0x7f6210009680 comp rx=0 tx=0).stop 2024-09-19T07:01:32.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:32.013+0000 7f62207ad640 1 -- 172.21.15.103:0/1348879278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62181036e0 msgr2=0x7f6218198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:32.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:32.013+0000 7f62207ad640 1 --2- 172.21.15.103:0/1348879278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62181036e0 0x7f6218198ae0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f620c006fd0 tx=0x7f620c00b8e0 comp rx=0 tx=0).stop 2024-09-19T07:01:32.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:32.013+0000 7f62207ad640 1 -- 172.21.15.103:0/1348879278 shutdown_connections 2024-09-19T07:01:32.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:32.013+0000 7f62207ad640 1 --2- 172.21.15.103:0/1348879278 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62000a33b0 0x7f62000a5870 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:32.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:32.013+0000 7f62207ad640 1 --2- 172.21.15.103:0/1348879278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62181082a0 0x7f621819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:32.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:32.013+0000 7f62207ad640 1 --2- 172.21.15.103:0/1348879278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62181036e0 0x7f6218198ae0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:32.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:32.013+0000 7f62207ad640 1 --2- 172.21.15.103:0/1348879278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218102d30 0x7f62181985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:32.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:32.013+0000 7f62207ad640 1 -- 172.21.15.103:0/1348879278 >> 172.21.15.103:0/1348879278 conn(0x7f62180fc7d0 msgr2=0x7f6218105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:32.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:32.013+0000 7f62207ad640 1 -- 172.21.15.103:0/1348879278 shutdown_connections 2024-09-19T07:01:32.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:32.014+0000 7f62207ad640 1 -- 172.21.15.103:0/1348879278 wait complete. 2024-09-19T07:01:32.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:32 smithi174 ceph-mon[35637]: pgmap v383: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-19T07:01:32.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:32 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:01:32.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:32 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1348879278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:01:32.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:32 smithi103 ceph-mon[31158]: pgmap v383: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-19T07:01:32.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:32 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:01:32.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:32 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1348879278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:01:32.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:32 smithi134 ceph-mon[35356]: pgmap v383: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-19T07:01:32.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:32 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:01:32.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:32 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1348879278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:01:32.763 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:01:33.023 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:33.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:33 smithi103 ceph-mon[31158]: pgmap v384: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-19T07:01:33.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:33 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:33 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:33 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:33 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:33 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:33 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:33 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:01:33.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:33 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:01:33.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:33 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.472+0000 7f1c29542640 1 -- 172.21.15.103:0/946759191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c24101420 msgr2=0x7f1c2410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:33.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.472+0000 7f1c29542640 1 --2- 172.21.15.103:0/946759191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c24101420 0x7f1c2410d950 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009cd0 tx=0x7f1c1c02a8e0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.472+0000 7f1c29542640 1 -- 172.21.15.103:0/946759191 shutdown_connections 2024-09-19T07:01:33.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.472+0000 7f1c29542640 1 --2- 172.21.15.103:0/946759191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c24101420 0x7f1c2410d950 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.472+0000 7f1c29542640 1 --2- 172.21.15.103:0/946759191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c24100a80 0x7f1c24100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.472+0000 7f1c29542640 1 --2- 172.21.15.103:0/946759191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c24106a80 0x7f1c24106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.472+0000 7f1c29542640 1 -- 172.21.15.103:0/946759191 >> 172.21.15.103:0/946759191 conn(0x7f1c240fc7a0 msgr2=0x7f1c240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:33.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.472+0000 7f1c29542640 1 -- 172.21.15.103:0/946759191 shutdown_connections 2024-09-19T07:01:33.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.472+0000 7f1c29542640 1 -- 172.21.15.103:0/946759191 wait complete. 2024-09-19T07:01:33.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.473+0000 7f1c29542640 1 Processor -- start 2024-09-19T07:01:33.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.473+0000 7f1c29542640 1 -- start start 2024-09-19T07:01:33.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.473+0000 7f1c29542640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c24100a80 0x7f1c24198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.473+0000 7f1c29542640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c24101420 0x7f1c24198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.473+0000 7f1c29542640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c24106a80 0x7f1c2419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.473+0000 7f1c29542640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c24111fa0 con 0x7f1c24100a80 2024-09-19T07:01:33.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.473+0000 7f1c29542640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1c24111e20 con 0x7f1c24101420 2024-09-19T07:01:33.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.473+0000 7f1c29542640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c24112120 con 0x7f1c24106a80 2024-09-19T07:01:33.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.473+0000 7f1c22ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c24100a80 0x7f1c24198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.473+0000 7f1c22ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c24100a80 0x7f1c24198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52684/0 (socket says 172.21.15.103:52684) 2024-09-19T07:01:33.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.473+0000 7f1c22ffd640 1 -- 172.21.15.103:0/1720260419 learned_addr learned my addr 172.21.15.103:0/1720260419 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:33.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.473+0000 7f1c227fc640 1 --2- 172.21.15.103:0/1720260419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c24101420 0x7f1c24198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.474+0000 7f1c237fe640 1 --2- 172.21.15.103:0/1720260419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c24106a80 0x7f1c2419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.474+0000 7f1c22ffd640 1 -- 172.21.15.103:0/1720260419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c24106a80 msgr2=0x7f1c2419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:33.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.474+0000 7f1c22ffd640 1 --2- 172.21.15.103:0/1720260419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c24106a80 0x7f1c2419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.474+0000 7f1c22ffd640 1 -- 172.21.15.103:0/1720260419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c24101420 msgr2=0x7f1c24198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:33.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.474+0000 7f1c22ffd640 1 --2- 172.21.15.103:0/1720260419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c24101420 0x7f1c24198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.474+0000 7f1c22ffd640 1 -- 172.21.15.103:0/1720260419 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2419d3e0 con 0x7f1c24100a80 2024-09-19T07:01:33.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.474+0000 7f1c22ffd640 1 --2- 172.21.15.103:0/1720260419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c24100a80 0x7f1c24198430 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400e9f0 tx=0x7f1c1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:33.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.474+0000 7f1c0bfff640 1 -- 172.21.15.103:0/1720260419 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1400cd80 con 0x7f1c24100a80 2024-09-19T07:01:33.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.474+0000 7f1c29542640 1 -- 172.21.15.103:0/1720260419 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2419d6d0 con 0x7f1c24100a80 2024-09-19T07:01:33.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.474+0000 7f1c0bfff640 1 -- 172.21.15.103:0/1720260419 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c14005500 con 0x7f1c24100a80 2024-09-19T07:01:33.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.474+0000 7f1c0bfff640 1 -- 172.21.15.103:0/1720260419 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14010690 con 0x7f1c24100a80 2024-09-19T07:01:33.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.474+0000 7f1c29542640 1 -- 172.21.15.103:0/1720260419 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2410e310 con 0x7f1c24100a80 2024-09-19T07:01:33.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.476+0000 7f1c0bfff640 1 -- 172.21.15.103:0/1720260419 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1c140046c0 con 0x7f1c24100a80 2024-09-19T07:01:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.477+0000 7f1c2b91b740 1 Processor -- start 2024-09-19T07:01:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.477+0000 7f1c2b91b740 1 -- start start 2024-09-19T07:01:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.477+0000 7f1c2b91b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea61b40 0x55d7dea60090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.477+0000 7f1c2b91b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea605d0 0x55d7dea60a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.477+0000 7f1c2b91b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea6b370 0x55d7dea6d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.477+0000 7f1c2b91b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d7de9c62f0 con 0x55d7dea605d0 2024-09-19T07:01:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.477+0000 7f1c2b91b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d7de96dcf0 con 0x55d7dea61b40 2024-09-19T07:01:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.477+0000 7f1c2b91b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d7de9c22b0 con 0x55d7dea6b370 2024-09-19T07:01:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.477+0000 7f1c0bfff640 1 --2- 172.21.15.103:0/1720260419 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1c0c0a33b0 0x7f1c0c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.477+0000 7f1c22ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea605d0 0x55d7dea60a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.477+0000 7f1c22ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea605d0 0x55d7dea60a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52700/0 (socket says 172.21.15.103:52700) 2024-09-19T07:01:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.477+0000 7f1c22ffd640 1 -- 172.21.15.103:0/3263202161 learned_addr learned my addr 172.21.15.103:0/3263202161 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.477+0000 7f1c237fe640 1 --2- 172.21.15.103:0/3263202161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea61b40 0x55d7dea60090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.477+0000 7f1c237fe640 1 --2- 172.21.15.103:0/1720260419 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1c0c0a33b0 0x7f1c0c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.477+0000 7f1c227fc640 1 --2- 172.21.15.103:0/3263202161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea6b370 0x55d7dea6d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.477+0000 7f1c0bfff640 1 -- 172.21.15.103:0/1720260419 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f1c140d0170 con 0x7f1c24100a80 2024-09-19T07:01:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.478+0000 7f1c237fe640 1 -- 172.21.15.103:0/3263202161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea6b370 msgr2=0x55d7dea6d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.478+0000 7f1c237fe640 1 --2- 172.21.15.103:0/3263202161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea6b370 0x55d7dea6d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.478+0000 7f1c237fe640 1 -- 172.21.15.103:0/3263202161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea605d0 msgr2=0x55d7dea60a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.478+0000 7f1c237fe640 1 --2- 172.21.15.103:0/3263202161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea605d0 0x55d7dea60a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.478+0000 7f1c237fe640 1 -- 172.21.15.103:0/3263202161 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7de9523a0 con 0x55d7dea61b40 2024-09-19T07:01:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.478+0000 7f1c22ffd640 1 --2- 172.21.15.103:0/3263202161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea605d0 0x55d7dea60a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:01:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.478+0000 7f1c237fe640 1 --2- 172.21.15.103:0/3263202161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea61b40 0x55d7dea60090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c006010 tx=0x7f1c1c031be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9065668fa0cd73a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.478+0000 7f1c237fe640 1 --2- 172.21.15.103:0/1720260419 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1c0c0a33b0 0x7f1c0c0a5870 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c033040 tx=0x7f1c1c04c000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.478+0000 7f1c097fa640 1 -- 172.21.15.103:0/3263202161 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c058070 con 0x55d7dea61b40 2024-09-19T07:01:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.478+0000 7f1c097fa640 1 -- 172.21.15.103:0/3263202161 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c036040 con 0x55d7dea61b40 2024-09-19T07:01:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.478+0000 7f1c097fa640 1 -- 172.21.15.103:0/3263202161 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c045040 con 0x55d7dea61b40 2024-09-19T07:01:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.479+0000 7f1c2b91b740 1 -- 172.21.15.103:0/3263202161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea61b40 msgr2=0x55d7dea60090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.479+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/3263202161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea61b40 0x55d7dea60090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c006010 tx=0x7f1c1c031be0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.479+0000 7f1c2b91b740 1 -- 172.21.15.103:0/3263202161 shutdown_connections 2024-09-19T07:01:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.479+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/3263202161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea6b370 0x55d7dea6d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.479+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/3263202161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea605d0 0x55d7dea60a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.479+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/3263202161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea61b40 0x55d7dea60090 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.479+0000 7f1c2b91b740 1 -- 172.21.15.103:0/3263202161 >> 172.21.15.103:0/3263202161 conn(0x55d7dea29310 msgr2=0x55d7dea5cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.479+0000 7f1c2b91b740 1 -- 172.21.15.103:0/3263202161 shutdown_connections 2024-09-19T07:01:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.479+0000 7f1c2b91b740 1 -- 172.21.15.103:0/3263202161 wait complete. 2024-09-19T07:01:33.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.479+0000 7f1c2b91b740 1 -- start start 2024-09-19T07:01:33.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.479+0000 7f1c2b91b740 1 Processor -- start 2024-09-19T07:01:33.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c2b91b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea605d0 0x55d7dea6ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c2b91b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea61b40 0x55d7dea68040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c2b91b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea6b370 0x55d7dea68650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c2b91b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d7dea6e010 con 0x55d7dea61b40 2024-09-19T07:01:33.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c2b91b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d7de981060 con 0x55d7dea6b370 2024-09-19T07:01:33.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c2b91b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d7dea6e190 con 0x55d7dea605d0 2024-09-19T07:01:33.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c2b91b740 10 client.0 _ll_register_callbacks cb 0x55d7de6d48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:33.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c227fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea61b40 0x55d7dea68040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c227fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea61b40 0x55d7dea68040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52702/0 (socket says 172.21.15.103:52702) 2024-09-19T07:01:33.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c227fc640 1 -- 172.21.15.103:0/1793339836 learned_addr learned my addr 172.21.15.103:0/1793339836 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:33.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c22ffd640 1 --2- 172.21.15.103:0/1793339836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea605d0 0x55d7dea6ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c237fe640 1 --2- 172.21.15.103:0/1793339836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea6b370 0x55d7dea68650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c227fc640 1 -- 172.21.15.103:0/1793339836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea605d0 msgr2=0x55d7dea6ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:33.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c227fc640 1 --2- 172.21.15.103:0/1793339836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea605d0 0x55d7dea6ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c227fc640 1 -- 172.21.15.103:0/1793339836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea6b370 msgr2=0x55d7dea68650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:33.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c227fc640 1 --2- 172.21.15.103:0/1793339836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea6b370 0x55d7dea68650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c227fc640 1 -- 172.21.15.103:0/1793339836 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7de7373c0 con 0x55d7dea61b40 2024-09-19T07:01:33.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c237fe640 1 --2- 172.21.15.103:0/1793339836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea6b370 0x55d7dea68650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:33.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c227fc640 1 --2- 172.21.15.103:0/1793339836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea61b40 0x55d7dea68040 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800bad0 tx=0x7f1c1800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9927be0a9bbef8d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:33.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c06ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:33.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.480+0000 7f1c06ff5640 1 -- 172.21.15.103:0/1793339836 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18003e20 con 0x55d7dea61b40 2024-09-19T07:01:33.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.481+0000 7f1c06ff5640 1 -- 172.21.15.103:0/1793339836 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f1c18011440 con 0x55d7dea61b40 2024-09-19T07:01:33.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.481+0000 7f1c2b91b740 1 -- 172.21.15.103:0/1793339836 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c1c036040 con 0x55d7dea61b40 2024-09-19T07:01:33.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.481+0000 7f1c06ff5640 1 -- 172.21.15.103:0/1793339836 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c18005020 con 0x55d7dea61b40 2024-09-19T07:01:33.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.481+0000 7f1c2b91b740 10 client.19662 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:33.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.481+0000 7f1c2b91b740 1 -- 172.21.15.103:0/1793339836 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d7dea79b40 con 0x55d7dea61b40 2024-09-19T07:01:33.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.481+0000 7f1c06ff5640 1 -- 172.21.15.103:0/1793339836 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c036040 con 0x55d7dea61b40 2024-09-19T07:01:33.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.481+0000 7f1c06ff5640 1 -- 172.21.15.103:0/1793339836 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1c18011640 con 0x55d7dea61b40 2024-09-19T07:01:33.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.481+0000 7f1c2b91b740 10 client.19662 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:33.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.481+0000 7f1c2b91b740 10 client.19662 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:01:33.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.481+0000 7f1c2b91b740 20 client.19662 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:33.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.481+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/1793339836 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55d7dea7b0a0 0x55d7dea7d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.481+0000 7f1c2b91b740 4 client.19662 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:33.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.481+0000 7f1c2b91b740 1 -- 172.21.15.103:0/1793339836 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7de9d3860 con 0x55d7dea7b0a0 2024-09-19T07:01:33.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.481+0000 7f1c22ffd640 1 --2- 172.21.15.103:0/1793339836 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55d7dea7b0a0 0x55d7dea7d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.482+0000 7f1c22ffd640 1 --2- 172.21.15.103:0/1793339836 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55d7dea7b0a0 0x55d7dea7d4c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c5f160e2bc5d4480 server_cookie=e53d0021fab72d1b in_seq=0 out_seq=0 2024-09-19T07:01:33.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.482+0000 7f1c06ff5640 10 client.19662 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:01:33.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.482+0000 7f1c06ff5640 1 -- 172.21.15.103:0/1793339836 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7de9d3860 con 0x55d7dea7b0a0 2024-09-19T07:01:33.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.482+0000 7f1c06ff5640 10 client.19662 handle_command_reply: tid=0 2024-09-19T07:01:33.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.482+0000 7f1c2b91b740 1 client.19662 shutdown 2024-09-19T07:01:33.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.482+0000 7f1c2b91b740 1 -- 172.21.15.103:0/1793339836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea61b40 msgr2=0x55d7dea68040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:33.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.482+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/1793339836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea61b40 0x55d7dea68040 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800bad0 tx=0x7f1c1800bf90 comp rx=0 tx=0).stop 2024-09-19T07:01:33.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.483+0000 7f1c2b91b740 1 -- 172.21.15.103:0/1793339836 shutdown_connections 2024-09-19T07:01:33.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.483+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/1793339836 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55d7dea7b0a0 0x55d7dea7d4c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.483+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/1793339836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea6b370 0x55d7dea68650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.483+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/1793339836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea61b40 0x55d7dea68040 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.483+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/1793339836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea605d0 0x55d7dea6ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.483+0000 7f1c2b91b740 1 -- 172.21.15.103:0/1793339836 >> 172.21.15.103:0/1793339836 conn(0x55d7dea29310 msgr2=0x55d7dea629c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:33.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.483+0000 7f1c06ff5640 0 client.19662 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:01:33.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.483+0000 7f1c2b91b740 1 -- 172.21.15.103:0/1793339836 shutdown_connections 2024-09-19T07:01:33.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.483+0000 7f1c2b91b740 1 -- 172.21.15.103:0/1793339836 wait complete. 2024-09-19T07:01:33.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.483+0000 7f1c2b91b740 20 client.19662 trim_cache size 0 max 16384 2024-09-19T07:01:33.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.495+0000 7f1c2b91b740 1 Processor -- start 2024-09-19T07:01:33.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.495+0000 7f1c2b91b740 1 -- start start 2024-09-19T07:01:33.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.495+0000 7f1c2b91b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea605d0 0x55d7dea772b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.495+0000 7f1c2b91b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea61b40 0x55d7dea67020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.495+0000 7f1c2b91b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea6b370 0x55d7dea82d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.495+0000 7f1c2b91b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241ab780 con 0x55d7dea61b40 2024-09-19T07:01:33.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.495+0000 7f1c2b91b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241ad2c0 con 0x55d7dea6b370 2024-09-19T07:01:33.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.495+0000 7f1c2b91b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241abe50 con 0x55d7dea605d0 2024-09-19T07:01:33.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.496+0000 7f1c227fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea61b40 0x55d7dea67020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.496+0000 7f1c227fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea61b40 0x55d7dea67020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52706/0 (socket says 172.21.15.103:52706) 2024-09-19T07:01:33.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.496+0000 7f1c227fc640 1 -- 172.21.15.103:0/160540302 learned_addr learned my addr 172.21.15.103:0/160540302 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:33.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.496+0000 7f1c22ffd640 1 --2- 172.21.15.103:0/160540302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea605d0 0x55d7dea772b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.496+0000 7f1c227fc640 1 -- 172.21.15.103:0/160540302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea605d0 msgr2=0x55d7dea772b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:33.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.496+0000 7f1c227fc640 1 --2- 172.21.15.103:0/160540302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea605d0 0x55d7dea772b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.496+0000 7f1c227fc640 1 -- 172.21.15.103:0/160540302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea6b370 msgr2=0x55d7dea82d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:33.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.496+0000 7f1c237fe640 1 --2- 172.21.15.103:0/160540302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea6b370 0x55d7dea82d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.496+0000 7f1c227fc640 1 --2- 172.21.15.103:0/160540302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea6b370 0x55d7dea82d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.496+0000 7f1c227fc640 1 -- 172.21.15.103:0/160540302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7dea5c510 con 0x55d7dea61b40 2024-09-19T07:01:33.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.496+0000 7f1c227fc640 1 --2- 172.21.15.103:0/160540302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea61b40 0x55d7dea67020 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f1c18011ab0 tx=0x7f1c18005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650e55a10bf7d371 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:33.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.496+0000 7f1c08ff9640 1 -- 172.21.15.103:0/160540302 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18030070 con 0x55d7dea61b40 2024-09-19T07:01:33.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.496+0000 7f1c08ff9640 1 -- 172.21.15.103:0/160540302 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c18005850 con 0x55d7dea61b40 2024-09-19T07:01:33.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.496+0000 7f1c08ff9640 1 -- 172.21.15.103:0/160540302 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18027040 con 0x55d7dea61b40 2024-09-19T07:01:33.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.497+0000 7f1c2b91b740 1 -- 172.21.15.103:0/160540302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea61b40 msgr2=0x55d7dea67020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:33.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.497+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/160540302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea61b40 0x55d7dea67020 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f1c18011ab0 tx=0x7f1c18005c50 comp rx=0 tx=0).stop 2024-09-19T07:01:33.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.497+0000 7f1c2b91b740 1 -- 172.21.15.103:0/160540302 shutdown_connections 2024-09-19T07:01:33.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.497+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/160540302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea6b370 0x55d7dea82d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.497+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/160540302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea61b40 0x55d7dea67020 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.497+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/160540302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea605d0 0x55d7dea772b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.497+0000 7f1c2b91b740 1 -- 172.21.15.103:0/160540302 >> 172.21.15.103:0/160540302 conn(0x55d7dea29310 msgr2=0x55d7dea66910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:33.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.497+0000 7f1c2b91b740 1 -- 172.21.15.103:0/160540302 shutdown_connections 2024-09-19T07:01:33.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.497+0000 7f1c2b91b740 1 -- 172.21.15.103:0/160540302 wait complete. 2024-09-19T07:01:33.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.497+0000 7f1c2b91b740 1 -- start start 2024-09-19T07:01:33.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.497+0000 7f1c2b91b740 1 Processor -- start 2024-09-19T07:01:33.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.497+0000 7f1c2b91b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea605d0 0x55d7dea739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c2b91b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea61b40 0x55d7dea6ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c2b91b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea6b370 0x55d7dea6f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c2b91b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241acbf0 con 0x55d7dea605d0 2024-09-19T07:01:33.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c2b91b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241ac520 con 0x55d7dea61b40 2024-09-19T07:01:33.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c2b91b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241a9780 con 0x55d7dea6b370 2024-09-19T07:01:33.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c2b91b740 10 client.0 _ll_register_callbacks cb 0x7f1c18004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:33.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c22ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea605d0 0x55d7dea739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c22ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea605d0 0x55d7dea739e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52720/0 (socket says 172.21.15.103:52720) 2024-09-19T07:01:33.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c22ffd640 1 -- 172.21.15.103:0/3999457695 learned_addr learned my addr 172.21.15.103:0/3999457695 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:33.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c237fe640 1 --2- 172.21.15.103:0/3999457695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea6b370 0x55d7dea6f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c227fc640 1 --2- 172.21.15.103:0/3999457695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea61b40 0x55d7dea6ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c22ffd640 1 -- 172.21.15.103:0/3999457695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea6b370 msgr2=0x55d7dea6f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:33.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c22ffd640 1 --2- 172.21.15.103:0/3999457695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea6b370 0x55d7dea6f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c22ffd640 1 -- 172.21.15.103:0/3999457695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea61b40 msgr2=0x55d7dea6ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:33.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c22ffd640 1 --2- 172.21.15.103:0/3999457695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea61b40 0x55d7dea6ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c22ffd640 1 -- 172.21.15.103:0/3999457695 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7dea6f8c0 con 0x55d7dea605d0 2024-09-19T07:01:33.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c237fe640 1 --2- 172.21.15.103:0/3999457695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea6b370 0x55d7dea6f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:33.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c227fc640 1 --2- 172.21.15.103:0/3999457695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea61b40 0x55d7dea6ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:33.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c22ffd640 1 --2- 172.21.15.103:0/3999457695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea605d0 0x55d7dea739e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f1c1409c130 tx=0x7f1c1409f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33dd3eb2350a2f99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:33.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c06ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:33.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c06ff5640 1 -- 172.21.15.103:0/3999457695 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c140a1970 con 0x55d7dea605d0 2024-09-19T07:01:33.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c06ff5640 1 -- 172.21.15.103:0/3999457695 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f1c140a1bb0 con 0x55d7dea605d0 2024-09-19T07:01:33.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c2b91b740 1 -- 172.21.15.103:0/3999457695 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c18005850 con 0x55d7dea605d0 2024-09-19T07:01:33.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c06ff5640 1 -- 172.21.15.103:0/3999457695 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c1409cb30 con 0x55d7dea605d0 2024-09-19T07:01:33.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c2b91b740 10 client.19668 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:33.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.498+0000 7f1c2b91b740 1 -- 172.21.15.103:0/3999457695 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d7deae5c30 con 0x55d7dea605d0 2024-09-19T07:01:33.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.499+0000 7f1c06ff5640 1 -- 172.21.15.103:0/3999457695 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c18005850 con 0x55d7dea605d0 2024-09-19T07:01:33.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.499+0000 7f1c06ff5640 1 -- 172.21.15.103:0/3999457695 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1c140abe80 con 0x55d7dea605d0 2024-09-19T07:01:33.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.499+0000 7f1c2b91b740 10 client.19668 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:33.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.499+0000 7f1c2b91b740 10 client.19668 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:01:33.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.499+0000 7f1c2b91b740 20 client.19668 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:33.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.499+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/3999457695 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55d7deae6350 0x55d7deae8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:33.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.499+0000 7f1c2b91b740 4 client.19668 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:01:33.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.499+0000 7f1c2b91b740 1 -- 172.21.15.103:0/3999457695 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7de9cdd70 con 0x55d7deae6350 2024-09-19T07:01:33.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.499+0000 7f1c227fc640 1 --2- 172.21.15.103:0/3999457695 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55d7deae6350 0x55d7deae8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:33.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.499+0000 7f1c06ff5640 10 client.19668 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:01:33.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.499+0000 7f1c227fc640 1 --2- 172.21.15.103:0/3999457695 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55d7deae6350 0x55d7deae8740 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=18a1bb6361a4702e server_cookie=6642de5c471f0bbb in_seq=0 out_seq=0 2024-09-19T07:01:33.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c06ff5640 1 -- 172.21.15.103:0/3999457695 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7de9cdd70 con 0x55d7deae6350 2024-09-19T07:01:33.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c06ff5640 10 client.19668 handle_command_reply: tid=0 2024-09-19T07:01:33.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c2b91b740 1 client.19668 shutdown 2024-09-19T07:01:33.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c2b91b740 1 -- 172.21.15.103:0/3999457695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea605d0 msgr2=0x55d7dea739e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:33.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/3999457695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea605d0 0x55d7dea739e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f1c1409c130 tx=0x7f1c1409f040 comp rx=0 tx=0).stop 2024-09-19T07:01:33.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c2b91b740 1 -- 172.21.15.103:0/3999457695 shutdown_connections 2024-09-19T07:01:33.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/3999457695 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55d7deae6350 0x55d7deae8740 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/3999457695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7dea6b370 0x55d7dea6f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/3999457695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7dea61b40 0x55d7dea6ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c2b91b740 1 --2- 172.21.15.103:0/3999457695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7dea605d0 0x55d7dea739e0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c06ff5640 0 client.19668 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:01:33.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c2b91b740 1 -- 172.21.15.103:0/3999457695 >> 172.21.15.103:0/3999457695 conn(0x55d7dea29310 msgr2=0x55d7dea698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:33.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c2b91b740 1 -- 172.21.15.103:0/3999457695 shutdown_connections 2024-09-19T07:01:33.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c2b91b740 1 -- 172.21.15.103:0/3999457695 wait complete. 2024-09-19T07:01:33.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c2b91b740 20 client.19668 trim_cache size 0 max 16384 2024-09-19T07:01:33.507 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:01:33.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c29542640 1 -- 172.21.15.103:0/1720260419 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1c0c0a33b0 msgr2=0x7f1c0c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:33.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c29542640 1 --2- 172.21.15.103:0/1720260419 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1c0c0a33b0 0x7f1c0c0a5870 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c033040 tx=0x7f1c1c04c000 comp rx=0 tx=0).stop 2024-09-19T07:01:33.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c29542640 1 -- 172.21.15.103:0/1720260419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c24100a80 msgr2=0x7f1c24198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:33.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.500+0000 7f1c29542640 1 --2- 172.21.15.103:0/1720260419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c24100a80 0x7f1c24198430 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400e9f0 tx=0x7f1c1400edf0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.501+0000 7f1c29542640 1 -- 172.21.15.103:0/1720260419 shutdown_connections 2024-09-19T07:01:33.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.501+0000 7f1c29542640 1 --2- 172.21.15.103:0/1720260419 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1c0c0a33b0 0x7f1c0c0a5870 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.501+0000 7f1c29542640 1 --2- 172.21.15.103:0/1720260419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c24106a80 0x7f1c2419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.501+0000 7f1c29542640 1 --2- 172.21.15.103:0/1720260419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c24101420 0x7f1c24198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.501+0000 7f1c29542640 1 --2- 172.21.15.103:0/1720260419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c24100a80 0x7f1c24198430 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:33.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.501+0000 7f1c29542640 1 -- 172.21.15.103:0/1720260419 >> 172.21.15.103:0/1720260419 conn(0x7f1c240fc7a0 msgr2=0x7f1c240fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:33.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.501+0000 7f1c29542640 1 -- 172.21.15.103:0/1720260419 shutdown_connections 2024-09-19T07:01:33.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:33.501+0000 7f1c29542640 1 -- 172.21.15.103:0/1720260419 wait complete. 2024-09-19T07:01:33.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:33 smithi174 ceph-mon[35637]: pgmap v384: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-19T07:01:33.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:33 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:33 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:33 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:33 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:33 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:33 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:33 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:01:33.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:33 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:01:33.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:33 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:33 smithi134 ceph-mon[35356]: pgmap v384: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-19T07:01:33.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:33 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:33 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:33 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:33 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:33 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:33 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:33.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:33 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:01:33.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:33 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:01:33.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:33 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:01:34.323 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:01:34.575 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:35.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.008+0000 7ff1fbffb640 1 -- 172.21.15.103:0/661970277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1f4074eb0 msgr2=0x7ff1f4112640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:35.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.008+0000 7ff1fbffb640 1 --2- 172.21.15.103:0/661970277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1f4074eb0 0x7ff1f4112640 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009900 tx=0x7ff1e402a480 comp rx=0 tx=0).stop 2024-09-19T07:01:35.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.008+0000 7ff1fbffb640 1 -- 172.21.15.103:0/661970277 shutdown_connections 2024-09-19T07:01:35.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.008+0000 7ff1fbffb640 1 --2- 172.21.15.103:0/661970277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1f4112b80 0x7ff1f4114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.008+0000 7ff1fbffb640 1 --2- 172.21.15.103:0/661970277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1f4074eb0 0x7ff1f4112640 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.008+0000 7ff1fbffb640 1 --2- 172.21.15.103:0/661970277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1f4074590 0x7ff1f4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.008+0000 7ff1fbffb640 1 -- 172.21.15.103:0/661970277 >> 172.21.15.103:0/661970277 conn(0x7ff1f40fd8b0 msgr2=0x7ff1f40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:35.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.008+0000 7ff1fbffb640 1 -- 172.21.15.103:0/661970277 shutdown_connections 2024-09-19T07:01:35.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.008+0000 7ff1fbffb640 1 -- 172.21.15.103:0/661970277 wait complete. 2024-09-19T07:01:35.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.009+0000 7ff1fbffb640 1 Processor -- start 2024-09-19T07:01:35.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.009+0000 7ff1fbffb640 1 -- start start 2024-09-19T07:01:35.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.009+0000 7ff1fbffb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1f4074590 0x7ff1f41a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.009+0000 7ff1fbffb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1f4074eb0 0x7ff1f41a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.009+0000 7ff1fbffb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1f4112b80 0x7ff1f419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.009+0000 7ff1fbffb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f4118530 con 0x7ff1f4112b80 2024-09-19T07:01:35.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.009+0000 7ff1fbffb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41183b0 con 0x7ff1f4074590 2024-09-19T07:01:35.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.009+0000 7ff1fbffb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41186b0 con 0x7ff1f4074eb0 2024-09-19T07:01:35.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.009+0000 7ff1fa598640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1f4112b80 0x7ff1f419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.009+0000 7ff1fa598640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1f4112b80 0x7ff1f419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52734/0 (socket says 172.21.15.103:52734) 2024-09-19T07:01:35.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.009+0000 7ff1fa598640 1 -- 172.21.15.103:0/415519157 learned_addr learned my addr 172.21.15.103:0/415519157 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:35.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.009+0000 7ff1f9596640 1 --2- 172.21.15.103:0/415519157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1f4074eb0 0x7ff1f41a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.009+0000 7ff1f9d97640 1 --2- 172.21.15.103:0/415519157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1f4074590 0x7ff1f41a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.010+0000 7ff1fa598640 1 -- 172.21.15.103:0/415519157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1f4074eb0 msgr2=0x7ff1f41a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:35.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.010+0000 7ff1fa598640 1 --2- 172.21.15.103:0/415519157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1f4074eb0 0x7ff1f41a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.010+0000 7ff1fa598640 1 -- 172.21.15.103:0/415519157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1f4074590 msgr2=0x7ff1f41a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:35.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.010+0000 7ff1fa598640 1 --2- 172.21.15.103:0/415519157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1f4074590 0x7ff1f41a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.010+0000 7ff1fa598640 1 -- 172.21.15.103:0/415519157 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f419efe0 con 0x7ff1f4112b80 2024-09-19T07:01:35.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.010+0000 7ff1fa598640 1 --2- 172.21.15.103:0/415519157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1f4112b80 0x7ff1f419e840 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00cad0 tx=0x7ff1ec00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:35.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.010+0000 7ff1eaffd640 1 -- 172.21.15.103:0/415519157 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec00ad60 con 0x7ff1f4112b80 2024-09-19T07:01:35.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.010+0000 7ff1eaffd640 1 -- 172.21.15.103:0/415519157 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1ec004030 con 0x7ff1f4112b80 2024-09-19T07:01:35.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.010+0000 7ff1fbffb640 1 -- 172.21.15.103:0/415519157 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f419f2d0 con 0x7ff1f4112b80 2024-09-19T07:01:35.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.010+0000 7ff1eaffd640 1 -- 172.21.15.103:0/415519157 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec00f6b0 con 0x7ff1f4112b80 2024-09-19T07:01:35.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.010+0000 7ff1fbffb640 1 -- 172.21.15.103:0/415519157 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f41ab520 con 0x7ff1f4112b80 2024-09-19T07:01:35.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.012+0000 7ff1eaffd640 1 -- 172.21.15.103:0/415519157 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff1ec0041e0 con 0x7ff1f4112b80 2024-09-19T07:01:35.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.013+0000 7ff1fe3d4740 1 Processor -- start 2024-09-19T07:01:35.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.013+0000 7ff1fe3d4740 1 -- start start 2024-09-19T07:01:35.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.013+0000 7ff1fe3d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b5b40 0x55899e5b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.013+0000 7ff1fe3d4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b45d0 0x55899e5b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.013+0000 7ff1fe3d4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5bf370 0x55899e5c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.013+0000 7ff1fe3d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55899e51a2f0 con 0x55899e5b5b40 2024-09-19T07:01:35.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.013+0000 7ff1fe3d4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55899e4c1cf0 con 0x55899e5b45d0 2024-09-19T07:01:35.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.013+0000 7ff1fe3d4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55899e5162b0 con 0x55899e5bf370 2024-09-19T07:01:35.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.013+0000 7ff1f9596640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b5b40 0x55899e5b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.013+0000 7ff1f9596640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b5b40 0x55899e5b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52736/0 (socket says 172.21.15.103:52736) 2024-09-19T07:01:35.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.013+0000 7ff1f9596640 1 -- 172.21.15.103:0/1924077781 learned_addr learned my addr 172.21.15.103:0/1924077781 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:35.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.013+0000 7ff1f9d97640 1 --2- 172.21.15.103:0/1924077781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b45d0 0x55899e5b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.013+0000 7ff1eaffd640 1 --2- 172.21.15.103:0/415519157 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1e00a33b0 0x7ff1e00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.014+0000 7ff1fa598640 1 --2- 172.21.15.103:0/415519157 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1e00a33b0 0x7ff1e00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.014+0000 7ff1f9596640 1 --2- 172.21.15.103:0/1924077781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5bf370 0x55899e5c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.014+0000 7ff1f9d97640 1 -- 172.21.15.103:0/1924077781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5bf370 msgr2=0x55899e5c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:35.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.014+0000 7ff1f9d97640 1 --2- 172.21.15.103:0/1924077781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5bf370 0x55899e5c1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.014+0000 7ff1f9d97640 1 -- 172.21.15.103:0/1924077781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b5b40 msgr2=0x55899e5b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:35.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.014+0000 7ff1f9d97640 1 --2- 172.21.15.103:0/1924077781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b5b40 0x55899e5b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.014+0000 7ff1eaffd640 1 -- 172.21.15.103:0/415519157 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7ff1ec0cfc50 con 0x7ff1f4112b80 2024-09-19T07:01:35.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.014+0000 7ff1f9d97640 1 -- 172.21.15.103:0/1924077781 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55899e4a63a0 con 0x55899e5b45d0 2024-09-19T07:01:35.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.014+0000 7ff1f9596640 1 --2- 172.21.15.103:0/1924077781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b5b40 0x55899e5b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:01:35.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.014+0000 7ff1f9d97640 1 --2- 172.21.15.103:0/1924077781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b45d0 0x55899e5b4a10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000abf0 tx=0x7ff1f000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b65d24496080e1ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:35.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.014+0000 7ff1fa598640 1 --2- 172.21.15.103:0/415519157 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1e00a33b0 0x7ff1e00a5870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec09bc10 tx=0x7ff1ec0c6ce0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:35.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.014+0000 7ff1dffff640 1 -- 172.21.15.103:0/1924077781 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0005220 con 0x55899e5b45d0 2024-09-19T07:01:35.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.014+0000 7ff1dffff640 1 -- 172.21.15.103:0/1924077781 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1f0004c80 con 0x55899e5b45d0 2024-09-19T07:01:35.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.014+0000 7ff1dffff640 1 -- 172.21.15.103:0/1924077781 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0018840 con 0x55899e5b45d0 2024-09-19T07:01:35.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.015+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/1924077781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b45d0 msgr2=0x55899e5b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:35.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.015+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/1924077781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b45d0 0x55899e5b4a10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000abf0 tx=0x7ff1f000f790 comp rx=0 tx=0).stop 2024-09-19T07:01:35.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.015+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/1924077781 shutdown_connections 2024-09-19T07:01:35.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.015+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/1924077781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5bf370 0x55899e5c1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.015+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/1924077781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b45d0 0x55899e5b4a10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.015+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/1924077781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b5b40 0x55899e5b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.015+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/1924077781 >> 172.21.15.103:0/1924077781 conn(0x55899e57d310 msgr2=0x55899e5b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:35.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.015+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/1924077781 shutdown_connections 2024-09-19T07:01:35.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.015+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/1924077781 wait complete. 2024-09-19T07:01:35.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.015+0000 7ff1fe3d4740 1 -- start start 2024-09-19T07:01:35.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1fe3d4740 1 Processor -- start 2024-09-19T07:01:35.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1fe3d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b45d0 0x55899e5becc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1fe3d4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b5b40 0x55899e5bbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1fe3d4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5bf370 0x55899e5bc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1fe3d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55899e5c2070 con 0x55899e5b45d0 2024-09-19T07:01:35.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1fe3d4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55899e4d5060 con 0x55899e5b5b40 2024-09-19T07:01:35.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1fe3d4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55899e5c21f0 con 0x55899e5bf370 2024-09-19T07:01:35.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1fe3d4740 10 client.0 _ll_register_callbacks cb 0x55899e2288c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:35.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1f9596640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b45d0 0x55899e5becc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1f9596640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b45d0 0x55899e5becc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52744/0 (socket says 172.21.15.103:52744) 2024-09-19T07:01:35.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1f9596640 1 -- 172.21.15.103:0/1804662973 learned_addr learned my addr 172.21.15.103:0/1804662973 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:35.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1f9596640 1 --2- 172.21.15.103:0/1804662973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5bf370 0x55899e5bc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1f9d97640 1 --2- 172.21.15.103:0/1804662973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b5b40 0x55899e5bbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1f9596640 1 -- 172.21.15.103:0/1804662973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5bf370 msgr2=0x55899e5bc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:35.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1f9596640 1 --2- 172.21.15.103:0/1804662973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5bf370 0x55899e5bc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1f9596640 1 -- 172.21.15.103:0/1804662973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b5b40 msgr2=0x55899e5bbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:35.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1f9596640 1 --2- 172.21.15.103:0/1804662973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b5b40 0x55899e5bbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1f9596640 1 -- 172.21.15.103:0/1804662973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899e28b3c0 con 0x55899e5b45d0 2024-09-19T07:01:35.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1f9596640 1 --2- 172.21.15.103:0/1804662973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b45d0 0x55899e5becc0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4005940 tx=0x7ff1e40058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce59b9d2100990db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:35.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.016+0000 7ff1dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:35.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.017+0000 7ff1dd7fa640 1 -- 172.21.15.103:0/1804662973 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4035040 con 0x55899e5b45d0 2024-09-19T07:01:35.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.017+0000 7ff1dd7fa640 1 -- 172.21.15.103:0/1804662973 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7ff1e4005020 con 0x55899e5b45d0 2024-09-19T07:01:35.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.017+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/1804662973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1f0004c80 con 0x55899e5b45d0 2024-09-19T07:01:35.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.017+0000 7ff1dd7fa640 1 -- 172.21.15.103:0/1804662973 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff1e4052050 con 0x55899e5b45d0 2024-09-19T07:01:35.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.017+0000 7ff1fe3d4740 10 client.19680 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:35.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.017+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/1804662973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55899e5cda80 con 0x55899e5b45d0 2024-09-19T07:01:35.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.017+0000 7ff1dd7fa640 1 -- 172.21.15.103:0/1804662973 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1f0004c80 con 0x55899e5b45d0 2024-09-19T07:01:35.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.017+0000 7ff1dd7fa640 1 -- 172.21.15.103:0/1804662973 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff1e40047a0 con 0x55899e5b45d0 2024-09-19T07:01:35.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.017+0000 7ff1fe3d4740 10 client.19680 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:35.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.017+0000 7ff1fe3d4740 10 client.19680 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:01:35.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.017+0000 7ff1fe3d4740 20 client.19680 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:35.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.017+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/1804662973 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55899e5cee90 0x55899e5d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.017+0000 7ff1fe3d4740 4 client.19680 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:35.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.017+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/1804662973 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55899e527860 con 0x55899e5cee90 2024-09-19T07:01:35.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.018+0000 7ff1f9d97640 1 --2- 172.21.15.103:0/1804662973 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55899e5cee90 0x55899e5d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.018+0000 7ff1f9d97640 1 --2- 172.21.15.103:0/1804662973 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55899e5cee90 0x55899e5d1280 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7689e25552aeef0d server_cookie=85038be2c39daaf4 in_seq=0 out_seq=0 2024-09-19T07:01:35.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.018+0000 7ff1dd7fa640 10 client.19680 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:01:35.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.019+0000 7ff1dd7fa640 1 -- 172.21.15.103:0/1804662973 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55899e527860 con 0x55899e5cee90 2024-09-19T07:01:35.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.019+0000 7ff1dd7fa640 10 client.19680 handle_command_reply: tid=0 2024-09-19T07:01:35.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.019+0000 7ff1fe3d4740 1 client.19680 shutdown 2024-09-19T07:01:35.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.019+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/1804662973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b45d0 msgr2=0x55899e5becc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:35.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.019+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/1804662973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b45d0 0x55899e5becc0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4005940 tx=0x7ff1e40058d0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.019+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/1804662973 shutdown_connections 2024-09-19T07:01:35.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.019+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/1804662973 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55899e5cee90 0x55899e5d1280 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.019+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/1804662973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5bf370 0x55899e5bc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.019+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/1804662973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b5b40 0x55899e5bbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.019+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/1804662973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b45d0 0x55899e5becc0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.019+0000 7ff1dd7fa640 0 client.19680 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:01:35.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.019+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/1804662973 >> 172.21.15.103:0/1804662973 conn(0x55899e57d310 msgr2=0x55899e5b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:35.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.019+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/1804662973 shutdown_connections 2024-09-19T07:01:35.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.019+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/1804662973 wait complete. 2024-09-19T07:01:35.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.019+0000 7ff1fe3d4740 20 client.19680 trim_cache size 0 max 16384 2024-09-19T07:01:35.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1fe3d4740 1 Processor -- start 2024-09-19T07:01:35.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1fe3d4740 1 -- start start 2024-09-19T07:01:35.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1fe3d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b45d0 0x55899e5b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1fe3d4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b5b40 0x55899e5cb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1fe3d4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5bf370 0x55899e5bb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1fe3d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41b3e40 con 0x55899e5b45d0 2024-09-19T07:01:35.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1fe3d4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41b5980 con 0x55899e5b5b40 2024-09-19T07:01:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1fe3d4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41b4510 con 0x55899e5bf370 2024-09-19T07:01:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1f9596640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b45d0 0x55899e5b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1f9596640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b45d0 0x55899e5b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52746/0 (socket says 172.21.15.103:52746) 2024-09-19T07:01:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1f9596640 1 -- 172.21.15.103:0/835553557 learned_addr learned my addr 172.21.15.103:0/835553557 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:35.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1f9596640 1 --2- 172.21.15.103:0/835553557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5bf370 0x55899e5bb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1f9d97640 1 --2- 172.21.15.103:0/835553557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b5b40 0x55899e5cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1f9596640 1 -- 172.21.15.103:0/835553557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5bf370 msgr2=0x55899e5bb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1f9596640 1 --2- 172.21.15.103:0/835553557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5bf370 0x55899e5bb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1f9596640 1 -- 172.21.15.103:0/835553557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b5b40 msgr2=0x55899e5cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:35.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1f9596640 1 --2- 172.21.15.103:0/835553557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b5b40 0x55899e5cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.032+0000 7ff1f9596640 1 -- 172.21.15.103:0/835553557 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55899e5b7b70 con 0x55899e5b45d0 2024-09-19T07:01:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.033+0000 7ff1f9596640 1 --2- 172.21.15.103:0/835553557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b45d0 0x55899e5b0e10 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4001eb0 tx=0x7ff1e402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f777ceafd90014 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.033+0000 7ff1df7fe640 1 -- 172.21.15.103:0/835553557 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4035040 con 0x55899e5b45d0 2024-09-19T07:01:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.033+0000 7ff1df7fe640 1 -- 172.21.15.103:0/835553557 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1e4003ce0 con 0x55899e5b45d0 2024-09-19T07:01:35.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.033+0000 7ff1df7fe640 1 -- 172.21.15.103:0/835553557 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4004fa0 con 0x55899e5b45d0 2024-09-19T07:01:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.033+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/835553557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b45d0 msgr2=0x55899e5b0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.033+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/835553557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b45d0 0x55899e5b0e10 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4001eb0 tx=0x7ff1e402a480 comp rx=0 tx=0).stop 2024-09-19T07:01:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.033+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/835553557 shutdown_connections 2024-09-19T07:01:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.033+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/835553557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5bf370 0x55899e5bb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.033+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/835553557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b5b40 0x55899e5cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.033+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/835553557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5b45d0 0x55899e5b0e10 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.033+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/835553557 >> 172.21.15.103:0/835553557 conn(0x55899e57d310 msgr2=0x55899e5b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.033+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/835553557 shutdown_connections 2024-09-19T07:01:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.033+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/835553557 wait complete. 2024-09-19T07:01:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.034+0000 7ff1fe3d4740 1 -- start start 2024-09-19T07:01:35.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.034+0000 7ff1fe3d4740 1 Processor -- start 2024-09-19T07:01:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.034+0000 7ff1fe3d4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b45d0 0x55899e5c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.034+0000 7ff1fe3d4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5b5b40 0x55899e5c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.034+0000 7ff1fe3d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5bf370 0x55899e5c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.034+0000 7ff1fe3d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41b52b0 con 0x55899e5bf370 2024-09-19T07:01:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.034+0000 7ff1fe3d4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41b4be0 con 0x55899e5b45d0 2024-09-19T07:01:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.034+0000 7ff1fe3d4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41b1e40 con 0x55899e5b5b40 2024-09-19T07:01:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.034+0000 7ff1fe3d4740 10 client.0 _ll_register_callbacks cb 0x7ff1e4031770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.034+0000 7ff1f9596640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5bf370 0x55899e5c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.034+0000 7ff1f9596640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5bf370 0x55899e5c32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52760/0 (socket says 172.21.15.103:52760) 2024-09-19T07:01:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.034+0000 7ff1f9596640 1 -- 172.21.15.103:0/3895664507 learned_addr learned my addr 172.21.15.103:0/3895664507 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:35.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.034+0000 7ff1f9596640 1 --2- 172.21.15.103:0/3895664507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b45d0 0x55899e5c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1f9d97640 1 --2- 172.21.15.103:0/3895664507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5b5b40 0x55899e5c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1f9596640 1 -- 172.21.15.103:0/3895664507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5b5b40 msgr2=0x55899e5c2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:35.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1f9596640 1 --2- 172.21.15.103:0/3895664507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5b5b40 0x55899e5c2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1f9596640 1 -- 172.21.15.103:0/3895664507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b45d0 msgr2=0x55899e5c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:35.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1f9596640 1 --2- 172.21.15.103:0/3895664507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b45d0 0x55899e5c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1f9596640 1 -- 172.21.15.103:0/3895664507 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899e5c39e0 con 0x55899e5bf370 2024-09-19T07:01:35.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1f9596640 1 --2- 172.21.15.103:0/3895664507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5bf370 0x55899e5c32f0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7ff1e402a9a0 tx=0x7ff1e40049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5f97cc885245a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:35.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:35.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1dd7fa640 1 -- 172.21.15.103:0/3895664507 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4035030 con 0x55899e5bf370 2024-09-19T07:01:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1dd7fa640 1 -- 172.21.15.103:0/3895664507 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7ff1e4003a00 con 0x55899e5bf370 2024-09-19T07:01:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/3895664507 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e4003ce0 con 0x55899e5bf370 2024-09-19T07:01:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1dd7fa640 1 -- 172.21.15.103:0/3895664507 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff1e406a050 con 0x55899e5bf370 2024-09-19T07:01:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1fe3d4740 10 client.19686 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/3895664507 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55899e6399c0 con 0x55899e5bf370 2024-09-19T07:01:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1dd7fa640 1 -- 172.21.15.103:0/3895664507 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1e4003ce0 con 0x55899e5bf370 2024-09-19T07:01:35.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1dd7fa640 1 -- 172.21.15.103:0/3895664507 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff1e404b450 con 0x55899e5bf370 2024-09-19T07:01:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1fe3d4740 10 client.19686 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1fe3d4740 10 client.19686 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:01:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1fe3d4740 20 client.19686 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/3895664507 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55899e639fd0 0x55899e63c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1fe3d4740 4 client.19686 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:01:35.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.035+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/3895664507 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55899e521d70 con 0x55899e639fd0 2024-09-19T07:01:35.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.036+0000 7ff1f9d97640 1 --2- 172.21.15.103:0/3895664507 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55899e639fd0 0x55899e63c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:35.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.036+0000 7ff1f9d97640 1 --2- 172.21.15.103:0/3895664507 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55899e639fd0 0x55899e63c3c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=856d7f8a19ecd04c server_cookie=c7eba58d5fd85f50 in_seq=0 out_seq=0 2024-09-19T07:01:35.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.036+0000 7ff1dd7fa640 10 client.19686 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:01:35.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.036+0000 7ff1dd7fa640 1 -- 172.21.15.103:0/3895664507 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55899e521d70 con 0x55899e639fd0 2024-09-19T07:01:35.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.036+0000 7ff1dd7fa640 10 client.19686 handle_command_reply: tid=0 2024-09-19T07:01:35.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.036+0000 7ff1fe3d4740 1 client.19686 shutdown 2024-09-19T07:01:35.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.036+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/3895664507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5bf370 msgr2=0x55899e5c32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:35.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.036+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/3895664507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5bf370 0x55899e5c32f0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7ff1e402a9a0 tx=0x7ff1e40049c0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/3895664507 shutdown_connections 2024-09-19T07:01:35.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/3895664507 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55899e639fd0 0x55899e63c3c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/3895664507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55899e5bf370 0x55899e5c32f0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1dd7fa640 0 client.19686 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:01:35.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/3895664507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55899e5b5b40 0x55899e5c2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fe3d4740 1 --2- 172.21.15.103:0/3895664507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55899e5b45d0 0x55899e5c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/3895664507 >> 172.21.15.103:0/3895664507 conn(0x55899e57d310 msgr2=0x55899e5af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:35.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/3895664507 shutdown_connections 2024-09-19T07:01:35.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fe3d4740 1 -- 172.21.15.103:0/3895664507 wait complete. 2024-09-19T07:01:35.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fe3d4740 20 client.19686 trim_cache size 0 max 16384 2024-09-19T07:01:35.071 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:01:35.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fbffb640 1 -- 172.21.15.103:0/415519157 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1e00a33b0 msgr2=0x7ff1e00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fbffb640 1 --2- 172.21.15.103:0/415519157 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1e00a33b0 0x7ff1e00a5870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec09bc10 tx=0x7ff1ec0c6ce0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fbffb640 1 -- 172.21.15.103:0/415519157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1f4112b80 msgr2=0x7ff1f419e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fbffb640 1 --2- 172.21.15.103:0/415519157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1f4112b80 0x7ff1f419e840 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00cad0 tx=0x7ff1ec00ced0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fbffb640 1 -- 172.21.15.103:0/415519157 shutdown_connections 2024-09-19T07:01:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fbffb640 1 --2- 172.21.15.103:0/415519157 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff1e00a33b0 0x7ff1e00a5870 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fbffb640 1 --2- 172.21.15.103:0/415519157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1f4112b80 0x7ff1f419e840 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fbffb640 1 --2- 172.21.15.103:0/415519157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1f4074eb0 0x7ff1f41a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fbffb640 1 --2- 172.21.15.103:0/415519157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1f4074590 0x7ff1f41a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:35.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.037+0000 7ff1fbffb640 1 -- 172.21.15.103:0/415519157 >> 172.21.15.103:0/415519157 conn(0x7ff1f40fd8b0 msgr2=0x7ff1f40ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:35.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.038+0000 7ff1fbffb640 1 -- 172.21.15.103:0/415519157 shutdown_connections 2024-09-19T07:01:35.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:35.038+0000 7ff1fbffb640 1 -- 172.21.15.103:0/415519157 wait complete. 2024-09-19T07:01:35.338 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:35 smithi103 ceph-mon[31158]: pgmap v385: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-19T07:01:35.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:35 smithi174 ceph-mon[35637]: pgmap v385: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-19T07:01:35.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:35 smithi134 ceph-mon[35356]: pgmap v385: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-19T07:01:35.735 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:01:35.984 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:36.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.473+0000 7f5a04168640 1 -- 172.21.15.103:0/277501394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59fc102790 msgr2=0x7f59fc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:36.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.473+0000 7f5a04168640 1 --2- 172.21.15.103:0/277501394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59fc102790 0x7f59fc1067e0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009900 tx=0x7f59ec02a410 comp rx=0 tx=0).stop 2024-09-19T07:01:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.473+0000 7f5a04168640 1 -- 172.21.15.103:0/277501394 shutdown_connections 2024-09-19T07:01:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.473+0000 7f5a04168640 1 --2- 172.21.15.103:0/277501394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59fc106f10 0x7f59fc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.473+0000 7f5a04168640 1 --2- 172.21.15.103:0/277501394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59fc102790 0x7f59fc1067e0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.473+0000 7f5a04168640 1 --2- 172.21.15.103:0/277501394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59fc101de0 0x7f59fc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.473+0000 7f5a04168640 1 -- 172.21.15.103:0/277501394 >> 172.21.15.103:0/277501394 conn(0x7f59fc0fd8c0 msgr2=0x7f59fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.473+0000 7f5a04168640 1 -- 172.21.15.103:0/277501394 shutdown_connections 2024-09-19T07:01:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.473+0000 7f5a04168640 1 -- 172.21.15.103:0/277501394 wait complete. 2024-09-19T07:01:36.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.474+0000 7f5a04168640 1 Processor -- start 2024-09-19T07:01:36.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.474+0000 7f5a04168640 1 -- start start 2024-09-19T07:01:36.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.474+0000 7f5a04168640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59fc101de0 0x7f59fc19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.474+0000 7f5a04168640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59fc102790 0x7f59fc19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.474+0000 7f5a04168640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59fc106f10 0x7f59fc19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.474+0000 7f5a04168640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc111fb0 con 0x7f59fc102790 2024-09-19T07:01:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.474+0000 7f5a04168640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc111e30 con 0x7f59fc101de0 2024-09-19T07:01:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.474+0000 7f5a04168640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc112130 con 0x7f59fc106f10 2024-09-19T07:01:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.474+0000 7f5a01703640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59fc102790 0x7f59fc19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.474+0000 7f5a01703640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59fc102790 0x7f59fc19ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52766/0 (socket says 172.21.15.103:52766) 2024-09-19T07:01:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.474+0000 7f5a01703640 1 -- 172.21.15.103:0/990213730 learned_addr learned my addr 172.21.15.103:0/990213730 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.474+0000 7f5a02705640 1 --2- 172.21.15.103:0/990213730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59fc106f10 0x7f59fc19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.475+0000 7f5a01f04640 1 --2- 172.21.15.103:0/990213730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59fc101de0 0x7f59fc19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.475+0000 7f5a01703640 1 -- 172.21.15.103:0/990213730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59fc106f10 msgr2=0x7f59fc19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.475+0000 7f5a01703640 1 --2- 172.21.15.103:0/990213730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59fc106f10 0x7f59fc19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.475+0000 7f5a01703640 1 -- 172.21.15.103:0/990213730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59fc101de0 msgr2=0x7f59fc19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.475+0000 7f5a01703640 1 --2- 172.21.15.103:0/990213730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59fc101de0 0x7f59fc19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.475+0000 7f5a01703640 1 -- 172.21.15.103:0/990213730 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc19f6c0 con 0x7f59fc102790 2024-09-19T07:01:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.475+0000 7f5a02705640 1 --2- 172.21.15.103:0/990213730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59fc106f10 0x7f59fc19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:36.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.475+0000 7f5a01f04640 1 --2- 172.21.15.103:0/990213730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59fc101de0 0x7f59fc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:36.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.475+0000 7f5a01703640 1 --2- 172.21.15.103:0/990213730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59fc102790 0x7f59fc19ac50 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f59ec006010 tx=0x7f59ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:36.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.475+0000 7f59f2ffd640 1 -- 172.21.15.103:0/990213730 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec03f030 con 0x7f59fc102790 2024-09-19T07:01:36.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.475+0000 7f5a04168640 1 -- 172.21.15.103:0/990213730 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59fc19f950 con 0x7f59fc102790 2024-09-19T07:01:36.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.475+0000 7f59f2ffd640 1 -- 172.21.15.103:0/990213730 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59ec003ce0 con 0x7f59fc102790 2024-09-19T07:01:36.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.475+0000 7f59f2ffd640 1 -- 172.21.15.103:0/990213730 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec035070 con 0x7f59fc102790 2024-09-19T07:01:36.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.475+0000 7f5a04168640 1 -- 172.21.15.103:0/990213730 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59fc1a7290 con 0x7f59fc102790 2024-09-19T07:01:36.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.477+0000 7f59f2ffd640 1 -- 172.21.15.103:0/990213730 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f59ec042030 con 0x7f59fc102790 2024-09-19T07:01:36.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.477+0000 7f5a06541740 1 Processor -- start 2024-09-19T07:01:36.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.478+0000 7f5a06541740 1 -- start start 2024-09-19T07:01:36.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.478+0000 7f5a06541740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565117196b40 0x565117195090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.478+0000 7f5a06541740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651171955d0 0x565117195a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.478+0000 7f5a06541740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171a0370 0x5651171a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.478+0000 7f5a06541740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5651170fb2f0 con 0x5651171a0370 2024-09-19T07:01:36.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.478+0000 7f5a06541740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5651170a2cf0 con 0x565117196b40 2024-09-19T07:01:36.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.478+0000 7f5a06541740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5651170f72b0 con 0x5651171955d0 2024-09-19T07:01:36.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.478+0000 7f59f2ffd640 1 --2- 172.21.15.103:0/990213730 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59e80a3430 0x7f59e80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.478+0000 7f5a01703640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171a0370 0x5651171a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:36.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.478+0000 7f5a01703640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171a0370 0x5651171a2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52776/0 (socket says 172.21.15.103:52776) 2024-09-19T07:01:36.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.478+0000 7f5a02705640 1 --2- 172.21.15.103:0/990213730 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59e80a3430 0x7f59e80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:36.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.478+0000 7f5a01703640 1 -- 172.21.15.103:0/3977076247 learned_addr learned my addr 172.21.15.103:0/3977076247 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:36.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.478+0000 7f5a01f04640 1 --2- 172.21.15.103:0/3977076247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651171955d0 0x565117195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:36.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.478+0000 7f5a02705640 1 --2- 172.21.15.103:0/3977076247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565117196b40 0x565117195090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:36.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.478+0000 7f59f2ffd640 1 -- 172.21.15.103:0/990213730 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f59ec0f0380 con 0x7f59fc102790 2024-09-19T07:01:36.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.479+0000 7f5a01703640 1 -- 172.21.15.103:0/3977076247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651171955d0 msgr2=0x565117195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.479+0000 7f5a01703640 1 --2- 172.21.15.103:0/3977076247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651171955d0 0x565117195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.479+0000 7f5a01703640 1 -- 172.21.15.103:0/3977076247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565117196b40 msgr2=0x565117195090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.479+0000 7f5a01703640 1 --2- 172.21.15.103:0/3977076247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565117196b40 0x565117195090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.479+0000 7f5a01703640 1 -- 172.21.15.103:0/3977076247 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651170873a0 con 0x5651171a0370 2024-09-19T07:01:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.479+0000 7f5a02705640 1 --2- 172.21.15.103:0/990213730 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59e80a3430 0x7f59e80a58f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009440 tx=0x7f59f4009e20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.479+0000 7f5a01703640 1 --2- 172.21.15.103:0/3977076247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171a0370 0x5651171a2760 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0b8b60 tx=0x7f59ec02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739d6166745759f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.479+0000 7f59e7fff640 1 -- 172.21.15.103:0/3977076247 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec049070 con 0x5651171a0370 2024-09-19T07:01:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.479+0000 7f59e7fff640 1 -- 172.21.15.103:0/3977076247 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59ec03b580 con 0x5651171a0370 2024-09-19T07:01:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.479+0000 7f59e7fff640 1 -- 172.21.15.103:0/3977076247 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec0c0ab0 con 0x5651171a0370 2024-09-19T07:01:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.480+0000 7f5a06541740 1 -- 172.21.15.103:0/3977076247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171a0370 msgr2=0x5651171a2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.480+0000 7f5a06541740 1 --2- 172.21.15.103:0/3977076247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171a0370 0x5651171a2760 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0b8b60 tx=0x7f59ec02a410 comp rx=0 tx=0).stop 2024-09-19T07:01:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.480+0000 7f5a06541740 1 -- 172.21.15.103:0/3977076247 shutdown_connections 2024-09-19T07:01:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.480+0000 7f5a06541740 1 --2- 172.21.15.103:0/3977076247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171a0370 0x5651171a2760 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.480+0000 7f5a06541740 1 --2- 172.21.15.103:0/3977076247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651171955d0 0x565117195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.480+0000 7f5a06541740 1 --2- 172.21.15.103:0/3977076247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565117196b40 0x565117195090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.480+0000 7f5a06541740 1 -- 172.21.15.103:0/3977076247 >> 172.21.15.103:0/3977076247 conn(0x56511715e310 msgr2=0x565117191c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.480+0000 7f5a06541740 1 -- 172.21.15.103:0/3977076247 shutdown_connections 2024-09-19T07:01:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.480+0000 7f5a06541740 1 -- 172.21.15.103:0/3977076247 wait complete. 2024-09-19T07:01:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.480+0000 7f5a06541740 1 -- start start 2024-09-19T07:01:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.480+0000 7f5a06541740 1 Processor -- start 2024-09-19T07:01:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a06541740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651171955d0 0x56511719fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a06541740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565117196b40 0x56511719d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a06541740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171a0370 0x56511719d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a06541740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5651170fb2f0 con 0x5651171a0370 2024-09-19T07:01:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a06541740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5651170b6060 con 0x565117196b40 2024-09-19T07:01:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a06541740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5651171a2e30 con 0x5651171955d0 2024-09-19T07:01:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a06541740 10 client.0 _ll_register_callbacks cb 0x565116e098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:36.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a02705640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171a0370 0x56511719d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:36.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a02705640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171a0370 0x56511719d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52792/0 (socket says 172.21.15.103:52792) 2024-09-19T07:01:36.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a02705640 1 -- 172.21.15.103:0/2117906917 learned_addr learned my addr 172.21.15.103:0/2117906917 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:36.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a01703640 1 --2- 172.21.15.103:0/2117906917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565117196b40 0x56511719d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:36.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a01f04640 1 --2- 172.21.15.103:0/2117906917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651171955d0 0x56511719fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:36.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a02705640 1 -- 172.21.15.103:0/2117906917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651171955d0 msgr2=0x56511719fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:36.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a02705640 1 --2- 172.21.15.103:0/2117906917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651171955d0 0x56511719fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a02705640 1 -- 172.21.15.103:0/2117906917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565117196b40 msgr2=0x56511719d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:36.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a02705640 1 --2- 172.21.15.103:0/2117906917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565117196b40 0x56511719d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a02705640 1 -- 172.21.15.103:0/2117906917 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565116e6c3c0 con 0x5651171a0370 2024-09-19T07:01:36.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f5a02705640 1 --2- 172.21.15.103:0/2117906917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171a0370 0x56511719d7f0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f59f400aed0 tx=0x7f59f40093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994e8154fce24a5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:36.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.481+0000 7f59e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:36.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.482+0000 7f59e57fa640 1 -- 172.21.15.103:0/2117906917 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f4028060 con 0x5651171a0370 2024-09-19T07:01:36.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.482+0000 7f5a06541740 1 -- 172.21.15.103:0/2117906917 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565117108860 con 0x5651171a0370 2024-09-19T07:01:36.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.482+0000 7f59e57fa640 1 -- 172.21.15.103:0/2117906917 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f59f4018560 con 0x5651171a0370 2024-09-19T07:01:36.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.482+0000 7f59e57fa640 1 -- 172.21.15.103:0/2117906917 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f59f4013040 con 0x5651171a0370 2024-09-19T07:01:36.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.482+0000 7f5a06541740 10 client.19698 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:36.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.482+0000 7f5a06541740 1 -- 172.21.15.103:0/2117906917 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5651171ae7a0 con 0x5651171a0370 2024-09-19T07:01:36.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.482+0000 7f59e57fa640 1 -- 172.21.15.103:0/2117906917 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x565117108860 con 0x5651171a0370 2024-09-19T07:01:36.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.482+0000 7f59e57fa640 1 -- 172.21.15.103:0/2117906917 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f59f4022070 con 0x5651171a0370 2024-09-19T07:01:36.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.482+0000 7f5a06541740 10 client.19698 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:36.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.482+0000 7f5a06541740 10 client.19698 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:01:36.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.482+0000 7f5a06541740 20 client.19698 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:36.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.482+0000 7f5a06541740 1 --2- 172.21.15.103:0/2117906917 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x5651171afc60 0x5651171b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.482+0000 7f5a06541740 4 client.19698 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:36.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.482+0000 7f5a06541740 1 -- 172.21.15.103:0/2117906917 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565117102d70 con 0x5651171afc60 2024-09-19T07:01:36.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.482+0000 7f5a01f04640 1 --2- 172.21.15.103:0/2117906917 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x5651171afc60 0x5651171b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:36.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.483+0000 7f59e57fa640 10 client.19698 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:01:36.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.483+0000 7f5a01f04640 1 --2- 172.21.15.103:0/2117906917 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x5651171afc60 0x5651171b20c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d415cdde76a9c7da server_cookie=d4bb62226165702c in_seq=0 out_seq=0 2024-09-19T07:01:36.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.484+0000 7f59e57fa640 1 -- 172.21.15.103:0/2117906917 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565117102d70 con 0x5651171afc60 2024-09-19T07:01:36.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.484+0000 7f59e57fa640 10 client.19698 handle_command_reply: tid=0 2024-09-19T07:01:36.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.484+0000 7f5a06541740 1 client.19698 shutdown 2024-09-19T07:01:36.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.484+0000 7f5a06541740 1 -- 172.21.15.103:0/2117906917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171a0370 msgr2=0x56511719d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:36.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.484+0000 7f5a06541740 1 --2- 172.21.15.103:0/2117906917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171a0370 0x56511719d7f0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f59f400aed0 tx=0x7f59f40093d0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.484+0000 7f5a06541740 1 -- 172.21.15.103:0/2117906917 shutdown_connections 2024-09-19T07:01:36.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.484+0000 7f5a06541740 1 --2- 172.21.15.103:0/2117906917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171a0370 0x56511719d7f0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.484+0000 7f5a06541740 1 --2- 172.21.15.103:0/2117906917 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x5651171afc60 0x5651171b20c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.484+0000 7f5a06541740 1 --2- 172.21.15.103:0/2117906917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565117196b40 0x56511719d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.484+0000 7f5a06541740 1 --2- 172.21.15.103:0/2117906917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651171955d0 0x56511719fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.484+0000 7f59e57fa640 0 client.19698 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:01:36.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.484+0000 7f5a06541740 1 -- 172.21.15.103:0/2117906917 >> 172.21.15.103:0/2117906917 conn(0x56511715e310 msgr2=0x565117197990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:36.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.484+0000 7f5a06541740 1 -- 172.21.15.103:0/2117906917 shutdown_connections 2024-09-19T07:01:36.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.484+0000 7f5a06541740 1 -- 172.21.15.103:0/2117906917 wait complete. 2024-09-19T07:01:36.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.484+0000 7f5a06541740 20 client.19698 trim_cache size 0 max 16384 2024-09-19T07:01:36.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.496+0000 7f5a06541740 1 Processor -- start 2024-09-19T07:01:36.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a06541740 1 -- start start 2024-09-19T07:01:36.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a06541740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171955d0 0x565117196300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a06541740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565117196b40 0x56511719c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a06541740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5651171a0370 0x565117198180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a06541740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc1afaf0 con 0x5651171955d0 2024-09-19T07:01:36.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a06541740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc1b1630 con 0x5651171a0370 2024-09-19T07:01:36.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a06541740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc1b01c0 con 0x565117196b40 2024-09-19T07:01:36.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a01f04640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171955d0 0x565117196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:36.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a01f04640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171955d0 0x565117196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52802/0 (socket says 172.21.15.103:52802) 2024-09-19T07:01:36.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a01f04640 1 -- 172.21.15.103:0/1434331521 learned_addr learned my addr 172.21.15.103:0/1434331521 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:36.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a01703640 1 --2- 172.21.15.103:0/1434331521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565117196b40 0x56511719c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:36.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a01f04640 1 -- 172.21.15.103:0/1434331521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565117196b40 msgr2=0x56511719c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:36.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a01f04640 1 --2- 172.21.15.103:0/1434331521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565117196b40 0x56511719c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a01f04640 1 -- 172.21.15.103:0/1434331521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5651171a0370 msgr2=0x565117198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:36.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a01f04640 1 --2- 172.21.15.103:0/1434331521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5651171a0370 0x565117198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a01f04640 1 -- 172.21.15.103:0/1434331521 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651171abf50 con 0x5651171955d0 2024-09-19T07:01:36.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a01703640 1 --2- 172.21.15.103:0/1434331521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565117196b40 0x56511719c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:36.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f5a01f04640 1 --2- 172.21.15.103:0/1434331521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171955d0 0x565117196300 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x5651171b2930 tx=0x7f59f80094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6621b5494584731b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:36.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f59e77fe640 1 -- 172.21.15.103:0/1434331521 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8005030 con 0x5651171955d0 2024-09-19T07:01:36.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f59e77fe640 1 -- 172.21.15.103:0/1434331521 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59f8004730 con 0x5651171955d0 2024-09-19T07:01:36.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.497+0000 7f59e77fe640 1 -- 172.21.15.103:0/1434331521 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f801b620 con 0x5651171955d0 2024-09-19T07:01:36.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.498+0000 7f5a06541740 1 -- 172.21.15.103:0/1434331521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171955d0 msgr2=0x565117196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:36.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.498+0000 7f5a06541740 1 --2- 172.21.15.103:0/1434331521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171955d0 0x565117196300 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x5651171b2930 tx=0x7f59f80094a0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.498+0000 7f5a06541740 1 -- 172.21.15.103:0/1434331521 shutdown_connections 2024-09-19T07:01:36.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.498+0000 7f5a06541740 1 --2- 172.21.15.103:0/1434331521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5651171a0370 0x565117198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.498+0000 7f5a06541740 1 --2- 172.21.15.103:0/1434331521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565117196b40 0x56511719c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.498+0000 7f5a06541740 1 --2- 172.21.15.103:0/1434331521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171955d0 0x565117196300 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.498+0000 7f5a06541740 1 -- 172.21.15.103:0/1434331521 >> 172.21.15.103:0/1434331521 conn(0x56511715e310 msgr2=0x56511718ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:36.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.498+0000 7f5a06541740 1 -- 172.21.15.103:0/1434331521 shutdown_connections 2024-09-19T07:01:36.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.498+0000 7f5a06541740 1 -- 172.21.15.103:0/1434331521 wait complete. 2024-09-19T07:01:36.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.498+0000 7f5a06541740 1 -- start start 2024-09-19T07:01:36.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.499+0000 7f5a06541740 1 Processor -- start 2024-09-19T07:01:36.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.499+0000 7f5a06541740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171955d0 0x5651171a84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.499+0000 7f5a06541740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565117196b40 0x5651171a3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.499+0000 7f5a06541740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5651171a0370 0x5651171a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.499+0000 7f5a06541740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc1b0f60 con 0x5651171955d0 2024-09-19T07:01:36.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.499+0000 7f5a06541740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc1b0890 con 0x5651171a0370 2024-09-19T07:01:36.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.499+0000 7f5a06541740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc1adaf0 con 0x565117196b40 2024-09-19T07:01:36.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.499+0000 7f5a06541740 10 client.0 _ll_register_callbacks cb 0x7f59f400a660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:36.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.499+0000 7f5a01f04640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171955d0 0x5651171a84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:36.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.499+0000 7f5a01f04640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171955d0 0x5651171a84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52814/0 (socket says 172.21.15.103:52814) 2024-09-19T07:01:36.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.499+0000 7f5a01f04640 1 -- 172.21.15.103:0/3277199922 learned_addr learned my addr 172.21.15.103:0/3277199922 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:36.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.499+0000 7f5a01f04640 1 -- 172.21.15.103:0/3277199922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565117196b40 msgr2=0x5651171a3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:36.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.499+0000 7f5a01f04640 1 --2- 172.21.15.103:0/3277199922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565117196b40 0x5651171a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.499+0000 7f5a01f04640 1 -- 172.21.15.103:0/3277199922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5651171a0370 msgr2=0x5651171a3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:36.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.499+0000 7f5a01f04640 1 --2- 172.21.15.103:0/3277199922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5651171a0370 0x5651171a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.499+0000 7f5a01f04640 1 -- 172.21.15.103:0/3277199922 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651171a42a0 con 0x5651171955d0 2024-09-19T07:01:36.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.500+0000 7f5a01f04640 1 --2- 172.21.15.103:0/3277199922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171955d0 0x5651171a84f0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f59f8004650 tx=0x7f59f8004b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d9c5c38b29a1e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:36.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.500+0000 7f59e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:36.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.500+0000 7f59e57fa640 1 -- 172.21.15.103:0/3277199922 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8005200 con 0x5651171955d0 2024-09-19T07:01:36.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.500+0000 7f59e57fa640 1 -- 172.21.15.103:0/3277199922 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f59f80053a0 con 0x5651171955d0 2024-09-19T07:01:36.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.500+0000 7f5a06541740 1 -- 172.21.15.103:0/3277199922 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59f8004730 con 0x5651171955d0 2024-09-19T07:01:36.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.500+0000 7f59e57fa640 1 -- 172.21.15.103:0/3277199922 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f59f8012450 con 0x5651171955d0 2024-09-19T07:01:36.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.500+0000 7f5a06541740 10 client.19704 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:36.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.500+0000 7f5a06541740 1 -- 172.21.15.103:0/3277199922 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56511721a650 con 0x5651171955d0 2024-09-19T07:01:36.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.500+0000 7f59e57fa640 1 -- 172.21.15.103:0/3277199922 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59f8004730 con 0x5651171955d0 2024-09-19T07:01:36.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.500+0000 7f59e57fa640 1 -- 172.21.15.103:0/3277199922 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f59f800f040 con 0x5651171955d0 2024-09-19T07:01:36.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.500+0000 7f5a06541740 10 client.19704 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:36.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.500+0000 7f5a06541740 10 client.19704 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:01:36.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.500+0000 7f5a06541740 20 client.19704 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:36.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.500+0000 7f5a06541740 1 --2- 172.21.15.103:0/3277199922 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56511721ace0 0x56511721d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:36.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.500+0000 7f5a06541740 4 client.19704 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:01:36.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.500+0000 7f5a06541740 1 -- 172.21.15.103:0/3277199922 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56511707f170 con 0x56511721ace0 2024-09-19T07:01:36.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f5a01703640 1 --2- 172.21.15.103:0/3277199922 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56511721ace0 0x56511721d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:36.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f59e57fa640 10 client.19704 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:01:36.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f5a01703640 1 --2- 172.21.15.103:0/3277199922 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56511721ace0 0x56511721d160 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de3066a5fe881262 server_cookie=93c30a4084a25b41 in_seq=0 out_seq=0 2024-09-19T07:01:36.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f59e57fa640 1 -- 172.21.15.103:0/3277199922 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56511707f170 con 0x56511721ace0 2024-09-19T07:01:36.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f59e57fa640 10 client.19704 handle_command_reply: tid=0 2024-09-19T07:01:36.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f5a06541740 1 client.19704 shutdown 2024-09-19T07:01:36.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f5a06541740 1 -- 172.21.15.103:0/3277199922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171955d0 msgr2=0x5651171a84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:36.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f5a06541740 1 --2- 172.21.15.103:0/3277199922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171955d0 0x5651171a84f0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f59f8004650 tx=0x7f59f8004b10 comp rx=0 tx=0).stop 2024-09-19T07:01:36.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f5a06541740 1 -- 172.21.15.103:0/3277199922 shutdown_connections 2024-09-19T07:01:36.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f5a06541740 1 --2- 172.21.15.103:0/3277199922 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56511721ace0 0x56511721d160 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f5a06541740 1 --2- 172.21.15.103:0/3277199922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5651171a0370 0x5651171a3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f5a06541740 1 --2- 172.21.15.103:0/3277199922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565117196b40 0x5651171a3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f59e57fa640 0 client.19704 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:01:36.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f5a06541740 1 --2- 172.21.15.103:0/3277199922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651171955d0 0x5651171a84f0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f5a06541740 1 -- 172.21.15.103:0/3277199922 >> 172.21.15.103:0/3277199922 conn(0x56511715e310 msgr2=0x56511719c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:36.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f5a06541740 1 -- 172.21.15.103:0/3277199922 shutdown_connections 2024-09-19T07:01:36.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.501+0000 7f5a06541740 1 -- 172.21.15.103:0/3277199922 wait complete. 2024-09-19T07:01:36.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.502+0000 7f5a06541740 20 client.19704 trim_cache size 0 max 16384 2024-09-19T07:01:36.508 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:01:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.502+0000 7f5a04168640 1 -- 172.21.15.103:0/990213730 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59e80a3430 msgr2=0x7f59e80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.502+0000 7f5a04168640 1 --2- 172.21.15.103:0/990213730 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59e80a3430 0x7f59e80a58f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009440 tx=0x7f59f4009e20 comp rx=0 tx=0).stop 2024-09-19T07:01:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.502+0000 7f5a04168640 1 -- 172.21.15.103:0/990213730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59fc102790 msgr2=0x7f59fc19ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.502+0000 7f5a04168640 1 --2- 172.21.15.103:0/990213730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59fc102790 0x7f59fc19ac50 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f59ec006010 tx=0x7f59ec0042f0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.502+0000 7f5a04168640 1 -- 172.21.15.103:0/990213730 shutdown_connections 2024-09-19T07:01:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.502+0000 7f5a04168640 1 --2- 172.21.15.103:0/990213730 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59e80a3430 0x7f59e80a58f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.502+0000 7f5a04168640 1 --2- 172.21.15.103:0/990213730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59fc106f10 0x7f59fc19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.502+0000 7f5a04168640 1 --2- 172.21.15.103:0/990213730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59fc102790 0x7f59fc19ac50 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.502+0000 7f5a04168640 1 --2- 172.21.15.103:0/990213730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59fc101de0 0x7f59fc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:36.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.502+0000 7f5a04168640 1 -- 172.21.15.103:0/990213730 >> 172.21.15.103:0/990213730 conn(0x7f59fc0fd8c0 msgr2=0x7f59fc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:36.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.502+0000 7f5a04168640 1 -- 172.21.15.103:0/990213730 shutdown_connections 2024-09-19T07:01:36.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:36.502+0000 7f5a04168640 1 -- 172.21.15.103:0/990213730 wait complete. 2024-09-19T07:01:37.233 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:01:37.481 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:37.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:37 smithi103 ceph-mon[31158]: pgmap v386: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-19T07:01:37.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:37 smithi174 ceph-mon[35637]: pgmap v386: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-19T07:01:37.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:37 smithi134 ceph-mon[35356]: pgmap v386: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-19T07:01:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.898+0000 7f51df43c640 1 -- 172.21.15.103:0/3132820519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51d8103660 msgr2=0x7f51d810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.898+0000 7f51df43c640 1 --2- 172.21.15.103:0/3132820519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51d8103660 0x7f51d810bb50 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009900 tx=0x7f51c802a480 comp rx=0 tx=0).stop 2024-09-19T07:01:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.899+0000 7f51df43c640 1 -- 172.21.15.103:0/3132820519 shutdown_connections 2024-09-19T07:01:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.899+0000 7f51df43c640 1 --2- 172.21.15.103:0/3132820519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d810c170 0x7f51d810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.899+0000 7f51df43c640 1 --2- 172.21.15.103:0/3132820519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51d8103660 0x7f51d810bb50 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.899+0000 7f51df43c640 1 --2- 172.21.15.103:0/3132820519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51d8102d40 0x7f51d8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.899+0000 7f51df43c640 1 -- 172.21.15.103:0/3132820519 >> 172.21.15.103:0/3132820519 conn(0x7f51d80fc7a0 msgr2=0x7f51d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.899+0000 7f51df43c640 1 -- 172.21.15.103:0/3132820519 shutdown_connections 2024-09-19T07:01:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.899+0000 7f51df43c640 1 -- 172.21.15.103:0/3132820519 wait complete. 2024-09-19T07:01:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.899+0000 7f51df43c640 1 Processor -- start 2024-09-19T07:01:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.899+0000 7f51df43c640 1 -- start start 2024-09-19T07:01:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51df43c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d8102d40 0x7f51d8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51df43c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51d8103660 0x7f51d8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51df43c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51d810c170 0x7f51d819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51df43c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f51d8111f70 con 0x7f51d8103660 2024-09-19T07:01:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51df43c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51d8111df0 con 0x7f51d8102d40 2024-09-19T07:01:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51df43c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f51d81120f0 con 0x7f51d810c170 2024-09-19T07:01:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51dc9d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51d8103660 0x7f51d8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51dc9d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51d8103660 0x7f51d8198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57780/0 (socket says 172.21.15.103:57780) 2024-09-19T07:01:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51dc9d7640 1 -- 172.21.15.103:0/653106440 learned_addr learned my addr 172.21.15.103:0/653106440 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51dd1d8640 1 --2- 172.21.15.103:0/653106440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d8102d40 0x7f51d8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51dd9d9640 1 --2- 172.21.15.103:0/653106440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51d810c170 0x7f51d819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51dc9d7640 1 -- 172.21.15.103:0/653106440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51d810c170 msgr2=0x7f51d819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51dc9d7640 1 --2- 172.21.15.103:0/653106440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51d810c170 0x7f51d819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51dc9d7640 1 -- 172.21.15.103:0/653106440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d8102d40 msgr2=0x7f51d8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51dc9d7640 1 --2- 172.21.15.103:0/653106440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d8102d40 0x7f51d8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51dc9d7640 1 -- 172.21.15.103:0/653106440 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d819d400 con 0x7f51d8103660 2024-09-19T07:01:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51dc9d7640 1 --2- 172.21.15.103:0/653106440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51d8103660 0x7f51d8198990 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f51c8006010 tx=0x7f51c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51ce7fc640 1 -- 172.21.15.103:0/653106440 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c803f030 con 0x7f51d8103660 2024-09-19T07:01:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51df43c640 1 -- 172.21.15.103:0/653106440 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d819d690 con 0x7f51d8103660 2024-09-19T07:01:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51ce7fc640 1 -- 172.21.15.103:0/653106440 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51c8003ce0 con 0x7f51d8103660 2024-09-19T07:01:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51ce7fc640 1 -- 172.21.15.103:0/653106440 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c8035070 con 0x7f51d8103660 2024-09-19T07:01:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.900+0000 7f51df43c640 1 -- 172.21.15.103:0/653106440 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d81a4fd0 con 0x7f51d8103660 2024-09-19T07:01:37.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.903+0000 7f51e1815740 1 Processor -- start 2024-09-19T07:01:37.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51e1815740 1 -- start start 2024-09-19T07:01:37.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51ce7fc640 1 -- 172.21.15.103:0/653106440 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f51c8042030 con 0x7f51d8103660 2024-09-19T07:01:37.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51e1815740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf61b40 0x55d74cf60090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51e1815740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf605d0 0x55d74cf60a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51e1815740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf6b370 0x55d74cf6d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51e1815740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d74cec62f0 con 0x55d74cf6b370 2024-09-19T07:01:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51e1815740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d74ce6dcf0 con 0x55d74cf605d0 2024-09-19T07:01:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51e1815740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d74cec22b0 con 0x55d74cf61b40 2024-09-19T07:01:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51dc9d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf6b370 0x55d74cf6d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51dc9d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf6b370 0x55d74cf6d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57794/0 (socket says 172.21.15.103:57794) 2024-09-19T07:01:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51dc9d7640 1 -- 172.21.15.103:0/1758615245 learned_addr learned my addr 172.21.15.103:0/1758615245 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51dd9d9640 1 --2- 172.21.15.103:0/1758615245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf61b40 0x55d74cf60090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51dd1d8640 1 --2- 172.21.15.103:0/1758615245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf605d0 0x55d74cf60a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51dc9d7640 1 -- 172.21.15.103:0/1758615245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf61b40 msgr2=0x55d74cf60090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51dc9d7640 1 --2- 172.21.15.103:0/1758615245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf61b40 0x55d74cf60090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51dc9d7640 1 -- 172.21.15.103:0/1758615245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf605d0 msgr2=0x55d74cf60a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51dc9d7640 1 --2- 172.21.15.103:0/1758615245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf605d0 0x55d74cf60a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.904+0000 7f51dc9d7640 1 -- 172.21.15.103:0/1758615245 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d74ce523a0 con 0x55d74cf6b370 2024-09-19T07:01:37.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.905+0000 7f51dc9d7640 1 --2- 172.21.15.103:0/1758615245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf6b370 0x55d74cf6d760 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f51c80ed940 tx=0x7f51c80f8500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566c6164f12c3571 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:37.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.905+0000 7f51ce7fc640 1 --2- 172.21.15.103:0/653106440 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f51c40a33b0 0x7f51c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.905+0000 7f51dd9d9640 1 --2- 172.21.15.103:0/653106440 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f51c40a33b0 0x7f51c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.905+0000 7f51c37fe640 1 -- 172.21.15.103:0/1758615245 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c80b88c0 con 0x55d74cf6b370 2024-09-19T07:01:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.905+0000 7f51ce7fc640 1 -- 172.21.15.103:0/653106440 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f51c80f02a0 con 0x7f51d8103660 2024-09-19T07:01:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.905+0000 7f51c37fe640 1 -- 172.21.15.103:0/1758615245 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51c8039b50 con 0x55d74cf6b370 2024-09-19T07:01:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.905+0000 7f51c37fe640 1 -- 172.21.15.103:0/1758615245 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c80babe0 con 0x55d74cf6b370 2024-09-19T07:01:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.906+0000 7f51e1815740 1 -- 172.21.15.103:0/1758615245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf6b370 msgr2=0x55d74cf6d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.906+0000 7f51dd9d9640 1 --2- 172.21.15.103:0/653106440 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f51c40a33b0 0x7f51c40a5870 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f51d0005fd0 tx=0x7f51d00093c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.906+0000 7f51e1815740 1 --2- 172.21.15.103:0/1758615245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf6b370 0x55d74cf6d760 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f51c80ed940 tx=0x7f51c80f8500 comp rx=0 tx=0).stop 2024-09-19T07:01:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.906+0000 7f51e1815740 1 -- 172.21.15.103:0/1758615245 shutdown_connections 2024-09-19T07:01:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.906+0000 7f51e1815740 1 --2- 172.21.15.103:0/1758615245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf6b370 0x55d74cf6d760 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.906+0000 7f51e1815740 1 --2- 172.21.15.103:0/1758615245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf605d0 0x55d74cf60a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.906+0000 7f51e1815740 1 --2- 172.21.15.103:0/1758615245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf61b40 0x55d74cf60090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.906+0000 7f51e1815740 1 -- 172.21.15.103:0/1758615245 >> 172.21.15.103:0/1758615245 conn(0x55d74cf29310 msgr2=0x55d74cf5cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.906+0000 7f51e1815740 1 -- 172.21.15.103:0/1758615245 shutdown_connections 2024-09-19T07:01:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.906+0000 7f51e1815740 1 -- 172.21.15.103:0/1758615245 wait complete. 2024-09-19T07:01:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.906+0000 7f51e1815740 1 -- start start 2024-09-19T07:01:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.906+0000 7f51e1815740 1 Processor -- start 2024-09-19T07:01:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51e1815740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf605d0 0x55d74cf6ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51e1815740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf61b40 0x55d74cf68000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51e1815740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf6b370 0x55d74cf685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51e1815740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d74cf6de30 con 0x55d74cf6b370 2024-09-19T07:01:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51e1815740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d74ce81060 con 0x55d74cf61b40 2024-09-19T07:01:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51e1815740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d74cf6dfb0 con 0x55d74cf605d0 2024-09-19T07:01:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51e1815740 10 client.0 _ll_register_callbacks cb 0x55d74cbd48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51dd9d9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf6b370 0x55d74cf685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51dd9d9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf6b370 0x55d74cf685d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57808/0 (socket says 172.21.15.103:57808) 2024-09-19T07:01:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51dd9d9640 1 -- 172.21.15.103:0/1033460327 learned_addr learned my addr 172.21.15.103:0/1033460327 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51dc9d7640 1 --2- 172.21.15.103:0/1033460327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf61b40 0x55d74cf68000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51dd1d8640 1 --2- 172.21.15.103:0/1033460327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf605d0 0x55d74cf6ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51dd9d9640 1 -- 172.21.15.103:0/1033460327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf605d0 msgr2=0x55d74cf6ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51dd9d9640 1 --2- 172.21.15.103:0/1033460327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf605d0 0x55d74cf6ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51dd9d9640 1 -- 172.21.15.103:0/1033460327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf61b40 msgr2=0x55d74cf68000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51dd9d9640 1 --2- 172.21.15.103:0/1033460327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf61b40 0x55d74cf68000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51dd9d9640 1 -- 172.21.15.103:0/1033460327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d74cc373c0 con 0x55d74cf6b370 2024-09-19T07:01:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51dd9d9640 1 --2- 172.21.15.103:0/1033460327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf6b370 0x55d74cf685d0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f51d00127a0 tx=0x7f51d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2a6bd939748c73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51c0ff9640 1 -- 172.21.15.103:0/1033460327 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d000fb50 con 0x55d74cf6b370 2024-09-19T07:01:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51c0ff9640 1 -- 172.21.15.103:0/1033460327 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f51d000fcf0 con 0x55d74cf6b370 2024-09-19T07:01:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.907+0000 7f51e1815740 1 -- 172.21.15.103:0/1033460327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51c8039b50 con 0x55d74cf6b370 2024-09-19T07:01:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.908+0000 7f51c0ff9640 1 -- 172.21.15.103:0/1033460327 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f51d001f420 con 0x55d74cf6b370 2024-09-19T07:01:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.908+0000 7f51e1815740 10 client.19716 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.908+0000 7f51e1815740 1 -- 172.21.15.103:0/1033460327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d74cf79850 con 0x55d74cf6b370 2024-09-19T07:01:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.908+0000 7f51c0ff9640 1 -- 172.21.15.103:0/1033460327 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51c8039b50 con 0x55d74cf6b370 2024-09-19T07:01:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.908+0000 7f51c0ff9640 1 -- 172.21.15.103:0/1033460327 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f51d00276d0 con 0x55d74cf6b370 2024-09-19T07:01:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.908+0000 7f51e1815740 10 client.19716 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.908+0000 7f51e1815740 10 client.19716 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:01:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.908+0000 7f51e1815740 20 client.19716 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.908+0000 7f51e1815740 1 --2- 172.21.15.103:0/1033460327 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d74cf7ad10 0x55d74cf7d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.908+0000 7f51e1815740 4 client.19716 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.908+0000 7f51e1815740 1 -- 172.21.15.103:0/1033460327 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d74ced3860 con 0x55d74cf7ad10 2024-09-19T07:01:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.908+0000 7f51dd1d8640 1 --2- 172.21.15.103:0/1033460327 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d74cf7ad10 0x55d74cf7d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.909+0000 7f51c0ff9640 10 client.19716 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:01:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.909+0000 7f51dd1d8640 1 --2- 172.21.15.103:0/1033460327 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d74cf7ad10 0x55d74cf7d100 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e25677f12c62b8f7 server_cookie=3b821a7447260384 in_seq=0 out_seq=0 2024-09-19T07:01:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.910+0000 7f51c0ff9640 1 -- 172.21.15.103:0/1033460327 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d74ced3860 con 0x55d74cf7ad10 2024-09-19T07:01:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.910+0000 7f51c0ff9640 10 client.19716 handle_command_reply: tid=0 2024-09-19T07:01:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.910+0000 7f51e1815740 1 client.19716 shutdown 2024-09-19T07:01:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.910+0000 7f51e1815740 1 -- 172.21.15.103:0/1033460327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf6b370 msgr2=0x55d74cf685d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.910+0000 7f51e1815740 1 --2- 172.21.15.103:0/1033460327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf6b370 0x55d74cf685d0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f51d00127a0 tx=0x7f51d0004a40 comp rx=0 tx=0).stop 2024-09-19T07:01:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.910+0000 7f51e1815740 1 -- 172.21.15.103:0/1033460327 shutdown_connections 2024-09-19T07:01:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.910+0000 7f51e1815740 1 --2- 172.21.15.103:0/1033460327 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d74cf7ad10 0x55d74cf7d100 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.910+0000 7f51e1815740 1 --2- 172.21.15.103:0/1033460327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf6b370 0x55d74cf685d0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.910+0000 7f51e1815740 1 --2- 172.21.15.103:0/1033460327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf61b40 0x55d74cf68000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.910+0000 7f51e1815740 1 --2- 172.21.15.103:0/1033460327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf605d0 0x55d74cf6ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.910+0000 7f51c0ff9640 0 client.19716 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:01:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.910+0000 7f51e1815740 1 -- 172.21.15.103:0/1033460327 >> 172.21.15.103:0/1033460327 conn(0x55d74cf29310 msgr2=0x55d74cf62960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.910+0000 7f51e1815740 1 -- 172.21.15.103:0/1033460327 shutdown_connections 2024-09-19T07:01:37.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.910+0000 7f51e1815740 1 -- 172.21.15.103:0/1033460327 wait complete. 2024-09-19T07:01:37.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.910+0000 7f51e1815740 20 client.19716 trim_cache size 0 max 16384 2024-09-19T07:01:37.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51e1815740 1 Processor -- start 2024-09-19T07:01:37.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51e1815740 1 -- start start 2024-09-19T07:01:37.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51e1815740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf605d0 0x55d74cf5ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51e1815740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf61b40 0x55d74cf77090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51e1815740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf6b370 0x55d74cf63120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51e1815740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f51d81ad9a0 con 0x55d74cf61b40 2024-09-19T07:01:37.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51e1815740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51d81af4e0 con 0x55d74cf605d0 2024-09-19T07:01:37.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51e1815740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f51d81ae070 con 0x55d74cf6b370 2024-09-19T07:01:37.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51dc9d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf61b40 0x55d74cf77090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:37.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51dc9d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf61b40 0x55d74cf77090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57820/0 (socket says 172.21.15.103:57820) 2024-09-19T07:01:37.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51dc9d7640 1 -- 172.21.15.103:0/1771647093 learned_addr learned my addr 172.21.15.103:0/1771647093 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:37.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51dd1d8640 1 --2- 172.21.15.103:0/1771647093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf605d0 0x55d74cf5ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:37.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51dc9d7640 1 -- 172.21.15.103:0/1771647093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf6b370 msgr2=0x55d74cf63120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:37.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51dc9d7640 1 --2- 172.21.15.103:0/1771647093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf6b370 0x55d74cf63120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51dc9d7640 1 -- 172.21.15.103:0/1771647093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf605d0 msgr2=0x55d74cf5ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:37.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51dc9d7640 1 --2- 172.21.15.103:0/1771647093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf605d0 0x55d74cf5ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51dc9d7640 1 -- 172.21.15.103:0/1771647093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d74cf7d6e0 con 0x55d74cf61b40 2024-09-19T07:01:37.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51dd1d8640 1 --2- 172.21.15.103:0/1771647093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf605d0 0x55d74cf5ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:37.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.923+0000 7f51dc9d7640 1 --2- 172.21.15.103:0/1771647093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf61b40 0x55d74cf77090 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f51c80bb2f0 tx=0x7f51c8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db96a0fd8df285b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:37.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.924+0000 7f51c2ffd640 1 -- 172.21.15.103:0/1771647093 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c8049030 con 0x55d74cf61b40 2024-09-19T07:01:37.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.924+0000 7f51c2ffd640 1 -- 172.21.15.103:0/1771647093 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51c80f3bc0 con 0x55d74cf61b40 2024-09-19T07:01:37.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.924+0000 7f51c2ffd640 1 -- 172.21.15.103:0/1771647093 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c80badd0 con 0x55d74cf61b40 2024-09-19T07:01:37.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.924+0000 7f51e1815740 1 -- 172.21.15.103:0/1771647093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf61b40 msgr2=0x55d74cf77090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:37.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.924+0000 7f51e1815740 1 --2- 172.21.15.103:0/1771647093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf61b40 0x55d74cf77090 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f51c80bb2f0 tx=0x7f51c8005c90 comp rx=0 tx=0).stop 2024-09-19T07:01:37.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.924+0000 7f51e1815740 1 -- 172.21.15.103:0/1771647093 shutdown_connections 2024-09-19T07:01:37.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.924+0000 7f51e1815740 1 --2- 172.21.15.103:0/1771647093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf6b370 0x55d74cf63120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.924+0000 7f51e1815740 1 --2- 172.21.15.103:0/1771647093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf61b40 0x55d74cf77090 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.924+0000 7f51e1815740 1 --2- 172.21.15.103:0/1771647093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf605d0 0x55d74cf5ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.924+0000 7f51e1815740 1 -- 172.21.15.103:0/1771647093 >> 172.21.15.103:0/1771647093 conn(0x55d74cf29310 msgr2=0x55d74cf5c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:37.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.924+0000 7f51e1815740 1 -- 172.21.15.103:0/1771647093 shutdown_connections 2024-09-19T07:01:37.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.924+0000 7f51e1815740 1 -- 172.21.15.103:0/1771647093 wait complete. 2024-09-19T07:01:37.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51e1815740 1 -- start start 2024-09-19T07:01:37.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51e1815740 1 Processor -- start 2024-09-19T07:01:37.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51e1815740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf605d0 0x55d74cf73830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51e1815740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf61b40 0x55d74cf6eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51e1815740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf6b370 0x55d74cf6f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51e1815740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f51d81aee10 con 0x55d74cf61b40 2024-09-19T07:01:37.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51e1815740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51d81ae740 con 0x55d74cf6b370 2024-09-19T07:01:37.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51e1815740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f51d81ab9a0 con 0x55d74cf605d0 2024-09-19T07:01:37.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51e1815740 10 client.0 _ll_register_callbacks cb 0x7f51d0009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:37.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51dc9d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf61b40 0x55d74cf6eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:37.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51dc9d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf61b40 0x55d74cf6eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57830/0 (socket says 172.21.15.103:57830) 2024-09-19T07:01:37.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51dc9d7640 1 -- 172.21.15.103:0/2599114544 learned_addr learned my addr 172.21.15.103:0/2599114544 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:37.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51dc9d7640 1 -- 172.21.15.103:0/2599114544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf605d0 msgr2=0x55d74cf73830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:37.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51dd9d9640 1 --2- 172.21.15.103:0/2599114544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf6b370 0x55d74cf6f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:37.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51dc9d7640 1 --2- 172.21.15.103:0/2599114544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf605d0 0x55d74cf73830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51dc9d7640 1 -- 172.21.15.103:0/2599114544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf6b370 msgr2=0x55d74cf6f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:37.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.925+0000 7f51dc9d7640 1 --2- 172.21.15.103:0/2599114544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf6b370 0x55d74cf6f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51dc9d7640 1 -- 172.21.15.103:0/2599114544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d74cf6f790 con 0x55d74cf61b40 2024-09-19T07:01:37.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51dd9d9640 1 --2- 172.21.15.103:0/2599114544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf6b370 0x55d74cf6f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:37.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51dc9d7640 1 --2- 172.21.15.103:0/2599114544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf61b40 0x55d74cf6eb60 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f51c8005ab0 tx=0x7f51c803b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c572e768e214a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:37.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:37.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51c0ff9640 1 -- 172.21.15.103:0/2599114544 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c8049040 con 0x55d74cf61b40 2024-09-19T07:01:37.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51c0ff9640 1 -- 172.21.15.103:0/2599114544 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f51c80edc50 con 0x55d74cf61b40 2024-09-19T07:01:37.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51e1815740 1 -- 172.21.15.103:0/2599114544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51c80f3bc0 con 0x55d74cf61b40 2024-09-19T07:01:37.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51c0ff9640 1 -- 172.21.15.103:0/2599114544 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f51c80bb150 con 0x55d74cf61b40 2024-09-19T07:01:37.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51e1815740 10 client.19722 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:37.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51e1815740 1 -- 172.21.15.103:0/2599114544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d74cf6f790 con 0x55d74cf61b40 2024-09-19T07:01:37.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51c0ff9640 1 -- 172.21.15.103:0/2599114544 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51c80f3bc0 con 0x55d74cf61b40 2024-09-19T07:01:37.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51c0ff9640 1 -- 172.21.15.103:0/2599114544 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f51c80b8c90 con 0x55d74cf61b40 2024-09-19T07:01:37.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51e1815740 10 client.19722 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:37.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51e1815740 10 client.19722 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:01:37.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51e1815740 20 client.19722 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:37.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51e1815740 1 --2- 172.21.15.103:0/2599114544 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d74cfe5960 0x55d74cfe7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:37.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51e1815740 4 client.19722 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:01:37.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.926+0000 7f51e1815740 1 -- 172.21.15.103:0/2599114544 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d74cecdd70 con 0x55d74cfe5960 2024-09-19T07:01:37.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.927+0000 7f51dd1d8640 1 --2- 172.21.15.103:0/2599114544 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d74cfe5960 0x55d74cfe7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:37.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.927+0000 7f51c0ff9640 10 client.19722 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:01:37.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.927+0000 7f51dd1d8640 1 --2- 172.21.15.103:0/2599114544 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d74cfe5960 0x55d74cfe7d50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=286e79ff385d6d7b server_cookie=1f2d29571f361b6d in_seq=0 out_seq=0 2024-09-19T07:01:37.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.927+0000 7f51c0ff9640 1 -- 172.21.15.103:0/2599114544 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d74cecdd70 con 0x55d74cfe5960 2024-09-19T07:01:37.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.927+0000 7f51c0ff9640 10 client.19722 handle_command_reply: tid=0 2024-09-19T07:01:37.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.927+0000 7f51e1815740 1 client.19722 shutdown 2024-09-19T07:01:37.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.927+0000 7f51e1815740 1 -- 172.21.15.103:0/2599114544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf61b40 msgr2=0x55d74cf6eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:37.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.927+0000 7f51e1815740 1 --2- 172.21.15.103:0/2599114544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf61b40 0x55d74cf6eb60 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f51c8005ab0 tx=0x7f51c803b5d0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.927+0000 7f51e1815740 1 -- 172.21.15.103:0/2599114544 shutdown_connections 2024-09-19T07:01:37.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.927+0000 7f51e1815740 1 --2- 172.21.15.103:0/2599114544 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d74cfe5960 0x55d74cfe7d50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.927+0000 7f51e1815740 1 --2- 172.21.15.103:0/2599114544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d74cf6b370 0x55d74cf6f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.927+0000 7f51e1815740 1 --2- 172.21.15.103:0/2599114544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d74cf61b40 0x55d74cf6eb60 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.927+0000 7f51c0ff9640 0 client.19722 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:01:37.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.927+0000 7f51e1815740 1 --2- 172.21.15.103:0/2599114544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d74cf605d0 0x55d74cf73830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.927+0000 7f51e1815740 1 -- 172.21.15.103:0/2599114544 >> 172.21.15.103:0/2599114544 conn(0x55d74cf29310 msgr2=0x55d74cf69860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:37.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.928+0000 7f51e1815740 1 -- 172.21.15.103:0/2599114544 shutdown_connections 2024-09-19T07:01:37.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.928+0000 7f51e1815740 1 -- 172.21.15.103:0/2599114544 wait complete. 2024-09-19T07:01:37.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.928+0000 7f51e1815740 20 client.19722 trim_cache size 0 max 16384 2024-09-19T07:01:37.934 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:01:37.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.928+0000 7f51df43c640 1 -- 172.21.15.103:0/653106440 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f51c40a33b0 msgr2=0x7f51c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:37.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.928+0000 7f51df43c640 1 --2- 172.21.15.103:0/653106440 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f51c40a33b0 0x7f51c40a5870 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f51d0005fd0 tx=0x7f51d00093c0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.928+0000 7f51df43c640 1 -- 172.21.15.103:0/653106440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51d8103660 msgr2=0x7f51d8198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:37.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.928+0000 7f51df43c640 1 --2- 172.21.15.103:0/653106440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51d8103660 0x7f51d8198990 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f51c8006010 tx=0x7f51c80042f0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.928+0000 7f51df43c640 1 -- 172.21.15.103:0/653106440 shutdown_connections 2024-09-19T07:01:37.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.928+0000 7f51df43c640 1 --2- 172.21.15.103:0/653106440 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f51c40a33b0 0x7f51c40a5870 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.928+0000 7f51df43c640 1 --2- 172.21.15.103:0/653106440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51d810c170 0x7f51d819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.928+0000 7f51df43c640 1 --2- 172.21.15.103:0/653106440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51d8103660 0x7f51d8198990 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.928+0000 7f51df43c640 1 --2- 172.21.15.103:0/653106440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d8102d40 0x7f51d8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:37.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.928+0000 7f51df43c640 1 -- 172.21.15.103:0/653106440 >> 172.21.15.103:0/653106440 conn(0x7f51d80fc7a0 msgr2=0x7f51d80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:37.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.928+0000 7f51df43c640 1 -- 172.21.15.103:0/653106440 shutdown_connections 2024-09-19T07:01:37.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:37.928+0000 7f51df43c640 1 -- 172.21.15.103:0/653106440 wait complete. 2024-09-19T07:01:38.605 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:01:38.853 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:39.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.333+0000 7f8768304640 1 -- 172.21.15.103:0/1265494356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87601055b0 msgr2=0x7f876010d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:39.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.333+0000 7f8768304640 1 --2- 172.21.15.103:0/1265494356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87601055b0 0x7f876010d130 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f8758009ca0 tx=0x7f875802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.334+0000 7f8768304640 1 -- 172.21.15.103:0/1265494356 shutdown_connections 2024-09-19T07:01:39.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.334+0000 7f8768304640 1 --2- 172.21.15.103:0/1265494356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87601055b0 0x7f876010d130 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.334+0000 7f8768304640 1 --2- 172.21.15.103:0/1265494356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8760100e30 0x7f8760104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.334+0000 7f8768304640 1 --2- 172.21.15.103:0/1265494356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8760100480 0x7f8760100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.334+0000 7f8768304640 1 -- 172.21.15.103:0/1265494356 >> 172.21.15.103:0/1265494356 conn(0x7f8760077210 msgr2=0x7f87600fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:39.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.334+0000 7f8768304640 1 -- 172.21.15.103:0/1265494356 shutdown_connections 2024-09-19T07:01:39.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.334+0000 7f8768304640 1 -- 172.21.15.103:0/1265494356 wait complete. 2024-09-19T07:01:39.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.334+0000 7f8768304640 1 Processor -- start 2024-09-19T07:01:39.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.334+0000 7f8768304640 1 -- start start 2024-09-19T07:01:39.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.334+0000 7f8768304640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8760100480 0x7f8760198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.334+0000 7f8768304640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8760100e30 0x7f8760198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.334+0000 7f8768304640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87601055b0 0x7f876019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.334+0000 7f8768304640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f87601106b0 con 0x7f8760100480 2024-09-19T07:01:39.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.334+0000 7f8768304640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8760110530 con 0x7f8760100e30 2024-09-19T07:01:39.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.334+0000 7f8768304640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8760110830 con 0x7f87601055b0 2024-09-19T07:01:39.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.334+0000 7f87660a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8760100480 0x7f8760198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.335+0000 7f87660a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8760100480 0x7f8760198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57848/0 (socket says 172.21.15.103:57848) 2024-09-19T07:01:39.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.335+0000 7f87660a0640 1 -- 172.21.15.103:0/2196664408 learned_addr learned my addr 172.21.15.103:0/2196664408 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:39.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.335+0000 7f876589f640 1 --2- 172.21.15.103:0/2196664408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8760100e30 0x7f8760198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.335+0000 7f87668a1640 1 --2- 172.21.15.103:0/2196664408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87601055b0 0x7f876019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.335+0000 7f87660a0640 1 -- 172.21.15.103:0/2196664408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87601055b0 msgr2=0x7f876019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:39.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.335+0000 7f87660a0640 1 --2- 172.21.15.103:0/2196664408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87601055b0 0x7f876019ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.335+0000 7f87660a0640 1 -- 172.21.15.103:0/2196664408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8760100e30 msgr2=0x7f8760198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:39.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.335+0000 7f87660a0640 1 --2- 172.21.15.103:0/2196664408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8760100e30 0x7f8760198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.335+0000 7f87660a0640 1 -- 172.21.15.103:0/2196664408 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876019d530 con 0x7f8760100480 2024-09-19T07:01:39.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.335+0000 7f87660a0640 1 --2- 172.21.15.103:0/2196664408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8760100480 0x7f8760198580 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f875c00b4a0 tx=0x7f875c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:39.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.335+0000 7f87537fe640 1 -- 172.21.15.103:0/2196664408 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c015020 con 0x7f8760100480 2024-09-19T07:01:39.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.335+0000 7f87537fe640 1 -- 172.21.15.103:0/2196664408 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f875c011d10 con 0x7f8760100480 2024-09-19T07:01:39.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.335+0000 7f8768304640 1 -- 172.21.15.103:0/2196664408 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876019d820 con 0x7f8760100480 2024-09-19T07:01:39.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.335+0000 7f87537fe640 1 -- 172.21.15.103:0/2196664408 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c00f2e0 con 0x7f8760100480 2024-09-19T07:01:39.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.335+0000 7f8768304640 1 -- 172.21.15.103:0/2196664408 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87601a5100 con 0x7f8760100480 2024-09-19T07:01:39.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.337+0000 7f87537fe640 1 -- 172.21.15.103:0/2196664408 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f875c018020 con 0x7f8760100480 2024-09-19T07:01:39.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.338+0000 7f876a6dd740 1 Processor -- start 2024-09-19T07:01:39.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.338+0000 7f876a6dd740 1 -- start start 2024-09-19T07:01:39.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.338+0000 7f876a6dd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b40b40 0x5565f8b3f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.338+0000 7f876a6dd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b3f5d0 0x5565f8b3fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.338+0000 7f876a6dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b4a370 0x5565f8b4c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.338+0000 7f876a6dd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5565f8aa52f0 con 0x5565f8b40b40 2024-09-19T07:01:39.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.338+0000 7f876a6dd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565f8a4ccf0 con 0x5565f8b3f5d0 2024-09-19T07:01:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.338+0000 7f876a6dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5565f8aa12b0 con 0x5565f8b4a370 2024-09-19T07:01:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.338+0000 7f87668a1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b40b40 0x5565f8b3f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f87668a1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b40b40 0x5565f8b3f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57856/0 (socket says 172.21.15.103:57856) 2024-09-19T07:01:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f87668a1640 1 -- 172.21.15.103:0/1244728959 learned_addr learned my addr 172.21.15.103:0/1244728959 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f87537fe640 1 --2- 172.21.15.103:0/2196664408 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f874c0a33b0 0x7f874c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f87660a0640 1 --2- 172.21.15.103:0/1244728959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b3f5d0 0x5565f8b3fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f876589f640 1 --2- 172.21.15.103:0/1244728959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b4a370 0x5565f8b4c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f87668a1640 1 --2- 172.21.15.103:0/2196664408 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f874c0a33b0 0x7f874c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f87668a1640 1 -- 172.21.15.103:0/1244728959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b4a370 msgr2=0x5565f8b4c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f87668a1640 1 --2- 172.21.15.103:0/1244728959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b4a370 0x5565f8b4c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f87668a1640 1 -- 172.21.15.103:0/1244728959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b3f5d0 msgr2=0x5565f8b3fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f87668a1640 1 --2- 172.21.15.103:0/1244728959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b3f5d0 0x5565f8b3fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f87668a1640 1 -- 172.21.15.103:0/1244728959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565f8a313a0 con 0x5565f8b40b40 2024-09-19T07:01:39.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f876589f640 1 --2- 172.21.15.103:0/1244728959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b4a370 0x5565f8b4c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:01:39.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f87537fe640 1 -- 172.21.15.103:0/2196664408 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f875c0cf480 con 0x7f8760100480 2024-09-19T07:01:39.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f87668a1640 1 --2- 172.21.15.103:0/1244728959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b40b40 0x5565f8b3f090 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f8758009ca0 tx=0x7f8758003ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa5d5d6f0603581 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:39.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f8750dea640 1 -- 172.21.15.103:0/1244728959 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758031820 con 0x5565f8b40b40 2024-09-19T07:01:39.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f8750dea640 1 -- 172.21.15.103:0/1244728959 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8758053070 con 0x5565f8b40b40 2024-09-19T07:01:39.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f8750dea640 1 -- 172.21.15.103:0/1244728959 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758004840 con 0x5565f8b40b40 2024-09-19T07:01:39.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.339+0000 7f87668a1640 1 --2- 172.21.15.103:0/2196664408 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f874c0a33b0 0x7f874c0a5870 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8758052000 tx=0x7f8758050000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:39.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.340+0000 7f876a6dd740 1 -- 172.21.15.103:0/1244728959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b40b40 msgr2=0x5565f8b3f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:39.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.340+0000 7f876a6dd740 1 --2- 172.21.15.103:0/1244728959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b40b40 0x5565f8b3f090 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f8758009ca0 tx=0x7f8758003ac0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.340+0000 7f876a6dd740 1 -- 172.21.15.103:0/1244728959 shutdown_connections 2024-09-19T07:01:39.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.340+0000 7f876a6dd740 1 --2- 172.21.15.103:0/1244728959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b4a370 0x5565f8b4c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.340+0000 7f876a6dd740 1 --2- 172.21.15.103:0/1244728959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b3f5d0 0x5565f8b3fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.340+0000 7f876a6dd740 1 --2- 172.21.15.103:0/1244728959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b40b40 0x5565f8b3f090 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.340+0000 7f876a6dd740 1 -- 172.21.15.103:0/1244728959 >> 172.21.15.103:0/1244728959 conn(0x5565f8b08310 msgr2=0x5565f8b3bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.340+0000 7f876a6dd740 1 -- 172.21.15.103:0/1244728959 shutdown_connections 2024-09-19T07:01:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.340+0000 7f876a6dd740 1 -- 172.21.15.103:0/1244728959 wait complete. 2024-09-19T07:01:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f876a6dd740 1 -- start start 2024-09-19T07:01:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f876a6dd740 1 Processor -- start 2024-09-19T07:01:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f876a6dd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b3f5d0 0x5565f8b49d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f876a6dd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b40b40 0x5565f8b46fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f876a6dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b4a370 0x5565f8b47570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f876a6dd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5565f8b4cef0 con 0x5565f8b3f5d0 2024-09-19T07:01:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f876a6dd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565f8a60060 con 0x5565f8b40b40 2024-09-19T07:01:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f876a6dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5565f8b4d070 con 0x5565f8b4a370 2024-09-19T07:01:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f876a6dd740 10 client.0 _ll_register_callbacks cb 0x5565f87b38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f87660a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b3f5d0 0x5565f8b49d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f87660a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b3f5d0 0x5565f8b49d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57868/0 (socket says 172.21.15.103:57868) 2024-09-19T07:01:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f87660a0640 1 -- 172.21.15.103:0/3815226909 learned_addr learned my addr 172.21.15.103:0/3815226909 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f876589f640 1 --2- 172.21.15.103:0/3815226909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b40b40 0x5565f8b46fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f87660a0640 1 -- 172.21.15.103:0/3815226909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b4a370 msgr2=0x5565f8b47570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f87668a1640 1 --2- 172.21.15.103:0/3815226909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b4a370 0x5565f8b47570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f87660a0640 1 --2- 172.21.15.103:0/3815226909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b4a370 0x5565f8b47570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f87660a0640 1 -- 172.21.15.103:0/3815226909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b40b40 msgr2=0x5565f8b46fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f87660a0640 1 --2- 172.21.15.103:0/3815226909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b40b40 0x5565f8b46fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f87660a0640 1 -- 172.21.15.103:0/3815226909 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565f88163c0 con 0x5565f8b3f5d0 2024-09-19T07:01:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f876589f640 1 --2- 172.21.15.103:0/3815226909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b40b40 0x5565f8b46fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.341+0000 7f87668a1640 1 --2- 172.21.15.103:0/3815226909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b4a370 0x5565f8b47570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.342+0000 7f87660a0640 1 --2- 172.21.15.103:0/3815226909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b3f5d0 0x5565f8b49d20 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f875c0cd1b0 tx=0x7f875c0105e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468090c72b40fd70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:39.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.342+0000 7f8749ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.342+0000 7f8749ffb640 1 -- 172.21.15.103:0/3815226909 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c09b1b0 con 0x5565f8b3f5d0 2024-09-19T07:01:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.342+0000 7f8749ffb640 1 -- 172.21.15.103:0/3815226909 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f875c097ec0 con 0x5565f8b3f5d0 2024-09-19T07:01:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.342+0000 7f876a6dd740 1 -- 172.21.15.103:0/3815226909 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8758053070 con 0x5565f8b3f5d0 2024-09-19T07:01:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.342+0000 7f8749ffb640 1 -- 172.21.15.103:0/3815226909 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f875c0092b0 con 0x5565f8b3f5d0 2024-09-19T07:01:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.342+0000 7f876a6dd740 10 client.19734 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.342+0000 7f876a6dd740 1 -- 172.21.15.103:0/3815226909 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5565f8b589a0 con 0x5565f8b3f5d0 2024-09-19T07:01:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.342+0000 7f8749ffb640 1 -- 172.21.15.103:0/3815226909 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8758053070 con 0x5565f8b3f5d0 2024-09-19T07:01:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.342+0000 7f8749ffb640 1 -- 172.21.15.103:0/3815226909 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f875c09bab0 con 0x5565f8b3f5d0 2024-09-19T07:01:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.342+0000 7f876a6dd740 10 client.19734 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.342+0000 7f876a6dd740 10 client.19734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.342+0000 7f876a6dd740 20 client.19734 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.342+0000 7f876a6dd740 1 --2- 172.21.15.103:0/3815226909 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565f8b59e60 0x5565f8b5c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.342+0000 7f876a6dd740 4 client.19734 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:39.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.342+0000 7f876a6dd740 1 -- 172.21.15.103:0/3815226909 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565f8ab2860 con 0x5565f8b59e60 2024-09-19T07:01:39.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.343+0000 7f876589f640 1 --2- 172.21.15.103:0/3815226909 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565f8b59e60 0x5565f8b5c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.343+0000 7f876589f640 1 --2- 172.21.15.103:0/3815226909 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565f8b59e60 0x5565f8b5c2d0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b47f7549e5df8bba server_cookie=9fecf5c57c458b2d in_seq=0 out_seq=0 2024-09-19T07:01:39.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.343+0000 7f8749ffb640 10 client.19734 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:39.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.344+0000 7f8749ffb640 1 -- 172.21.15.103:0/3815226909 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565f8ab2860 con 0x5565f8b59e60 2024-09-19T07:01:39.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.344+0000 7f8749ffb640 10 client.19734 handle_command_reply: tid=0 2024-09-19T07:01:39.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.344+0000 7f876a6dd740 1 client.19734 shutdown 2024-09-19T07:01:39.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.344+0000 7f876a6dd740 1 -- 172.21.15.103:0/3815226909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b3f5d0 msgr2=0x5565f8b49d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:39.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.344+0000 7f876a6dd740 1 --2- 172.21.15.103:0/3815226909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b3f5d0 0x5565f8b49d20 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f875c0cd1b0 tx=0x7f875c0105e0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.344+0000 7f876a6dd740 1 -- 172.21.15.103:0/3815226909 shutdown_connections 2024-09-19T07:01:39.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.344+0000 7f876a6dd740 1 --2- 172.21.15.103:0/3815226909 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565f8b59e60 0x5565f8b5c2d0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.344+0000 7f876a6dd740 1 --2- 172.21.15.103:0/3815226909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b4a370 0x5565f8b47570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.344+0000 7f876a6dd740 1 --2- 172.21.15.103:0/3815226909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b40b40 0x5565f8b46fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.344+0000 7f876a6dd740 1 --2- 172.21.15.103:0/3815226909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b3f5d0 0x5565f8b49d20 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.344+0000 7f8749ffb640 0 client.19734 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:39.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.344+0000 7f876a6dd740 1 -- 172.21.15.103:0/3815226909 >> 172.21.15.103:0/3815226909 conn(0x5565f8b08310 msgr2=0x5565f8b41960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:39.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.344+0000 7f876a6dd740 1 -- 172.21.15.103:0/3815226909 shutdown_connections 2024-09-19T07:01:39.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.344+0000 7f876a6dd740 1 -- 172.21.15.103:0/3815226909 wait complete. 2024-09-19T07:01:39.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.345+0000 7f876a6dd740 20 client.19734 trim_cache size 0 max 16384 2024-09-19T07:01:39.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.357+0000 7f876a6dd740 1 Processor -- start 2024-09-19T07:01:39.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.357+0000 7f876a6dd740 1 -- start start 2024-09-19T07:01:39.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.357+0000 7f876a6dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b3f5d0 0x5565f8b46890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.357+0000 7f876a6dd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b40b40 0x5565f8b3ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.357+0000 7f876a6dd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b4a370 0x5565f8b42120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.358+0000 7f876a6dd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f87601ab7b0 con 0x5565f8b40b40 2024-09-19T07:01:39.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.358+0000 7f876a6dd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87601ad2f0 con 0x5565f8b4a370 2024-09-19T07:01:39.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.358+0000 7f876a6dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87601abe80 con 0x5565f8b3f5d0 2024-09-19T07:01:39.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.358+0000 7f876589f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b40b40 0x5565f8b3ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.358+0000 7f876589f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b40b40 0x5565f8b3ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57880/0 (socket says 172.21.15.103:57880) 2024-09-19T07:01:39.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.358+0000 7f876589f640 1 -- 172.21.15.103:0/2461792868 learned_addr learned my addr 172.21.15.103:0/2461792868 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:39.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.358+0000 7f87668a1640 1 --2- 172.21.15.103:0/2461792868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b4a370 0x5565f8b42120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.358+0000 7f876589f640 1 -- 172.21.15.103:0/2461792868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b3f5d0 msgr2=0x5565f8b46890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:39.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.358+0000 7f876589f640 1 --2- 172.21.15.103:0/2461792868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b3f5d0 0x5565f8b46890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.358+0000 7f876589f640 1 -- 172.21.15.103:0/2461792868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b4a370 msgr2=0x5565f8b42120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:39.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.358+0000 7f876589f640 1 --2- 172.21.15.103:0/2461792868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b4a370 0x5565f8b42120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.358+0000 7f876589f640 1 -- 172.21.15.103:0/2461792868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565f8b3a310 con 0x5565f8b40b40 2024-09-19T07:01:39.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.358+0000 7f876589f640 1 --2- 172.21.15.103:0/2461792868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b40b40 0x5565f8b3ac10 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f8754009330 tx=0x7f8754008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd19d1257f9a720 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:39.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.358+0000 7f874bfff640 1 -- 172.21.15.103:0/2461792868 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754017e40 con 0x5565f8b40b40 2024-09-19T07:01:39.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.358+0000 7f874bfff640 1 -- 172.21.15.103:0/2461792868 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8754021030 con 0x5565f8b40b40 2024-09-19T07:01:39.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.358+0000 7f874bfff640 1 -- 172.21.15.103:0/2461792868 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875401d3d0 con 0x5565f8b40b40 2024-09-19T07:01:39.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.359+0000 7f876a6dd740 1 -- 172.21.15.103:0/2461792868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b40b40 msgr2=0x5565f8b3ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:39.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.359+0000 7f876a6dd740 1 --2- 172.21.15.103:0/2461792868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b40b40 0x5565f8b3ac10 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f8754009330 tx=0x7f8754008040 comp rx=0 tx=0).stop 2024-09-19T07:01:39.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.359+0000 7f876a6dd740 1 -- 172.21.15.103:0/2461792868 shutdown_connections 2024-09-19T07:01:39.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.359+0000 7f876a6dd740 1 --2- 172.21.15.103:0/2461792868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b4a370 0x5565f8b42120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.359+0000 7f876a6dd740 1 --2- 172.21.15.103:0/2461792868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b40b40 0x5565f8b3ac10 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.359+0000 7f876a6dd740 1 --2- 172.21.15.103:0/2461792868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b3f5d0 0x5565f8b46890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.359+0000 7f876a6dd740 1 -- 172.21.15.103:0/2461792868 >> 172.21.15.103:0/2461792868 conn(0x5565f8b08310 msgr2=0x5565f8b3a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:39.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.359+0000 7f876a6dd740 1 -- 172.21.15.103:0/2461792868 shutdown_connections 2024-09-19T07:01:39.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.359+0000 7f876a6dd740 1 -- 172.21.15.103:0/2461792868 wait complete. 2024-09-19T07:01:39.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.359+0000 7f876a6dd740 1 -- start start 2024-09-19T07:01:39.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f876a6dd740 1 Processor -- start 2024-09-19T07:01:39.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f876a6dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b3f5d0 0x5565f8b52c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f876a6dd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b40b40 0x5565f8b4df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f876a6dd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b4a370 0x5565f8b4e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f876a6dd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f87601acc20 con 0x5565f8b4a370 2024-09-19T07:01:39.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f876a6dd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87601ac550 con 0x5565f8b40b40 2024-09-19T07:01:39.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f876a6dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87601a97b0 con 0x5565f8b3f5d0 2024-09-19T07:01:39.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f876a6dd740 10 client.0 _ll_register_callbacks cb 0x7f875c00bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:39.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f87668a1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b4a370 0x5565f8b4e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f87668a1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b4a370 0x5565f8b4e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57892/0 (socket says 172.21.15.103:57892) 2024-09-19T07:01:39.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f87668a1640 1 -- 172.21.15.103:0/2009527021 learned_addr learned my addr 172.21.15.103:0/2009527021 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:39.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f876589f640 1 --2- 172.21.15.103:0/2009527021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b40b40 0x5565f8b4df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f87660a0640 1 --2- 172.21.15.103:0/2009527021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b3f5d0 0x5565f8b52c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f87668a1640 1 -- 172.21.15.103:0/2009527021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b3f5d0 msgr2=0x5565f8b52c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:39.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f87668a1640 1 --2- 172.21.15.103:0/2009527021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b3f5d0 0x5565f8b52c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f87668a1640 1 -- 172.21.15.103:0/2009527021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b40b40 msgr2=0x5565f8b4df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:39.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f87668a1640 1 --2- 172.21.15.103:0/2009527021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b40b40 0x5565f8b4df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f87668a1640 1 -- 172.21.15.103:0/2009527021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565f8bc3b40 con 0x5565f8b4a370 2024-09-19T07:01:39.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f876589f640 1 --2- 172.21.15.103:0/2009527021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b40b40 0x5565f8b4df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:39.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f87660a0640 1 --2- 172.21.15.103:0/2009527021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b3f5d0 0x5565f8b52c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:39.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f87668a1640 1 --2- 172.21.15.103:0/2009527021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b4a370 0x5565f8b4e6e0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f8758009bf0 tx=0x7f8758031de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646cf4784899c1f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:39.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.360+0000 7f8749ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:39.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.361+0000 7f8749ffb640 1 -- 172.21.15.103:0/2009527021 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875803d260 con 0x5565f8b4a370 2024-09-19T07:01:39.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.361+0000 7f8749ffb640 1 -- 172.21.15.103:0/2009527021 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f8758041c00 con 0x5565f8b4a370 2024-09-19T07:01:39.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.361+0000 7f876a6dd740 1 -- 172.21.15.103:0/2009527021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8754021030 con 0x5565f8b4a370 2024-09-19T07:01:39.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.361+0000 7f8749ffb640 1 -- 172.21.15.103:0/2009527021 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f875802f720 con 0x5565f8b4a370 2024-09-19T07:01:39.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.361+0000 7f876a6dd740 10 client.19740 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:39.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.361+0000 7f876a6dd740 1 -- 172.21.15.103:0/2009527021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5565f8bc4d20 con 0x5565f8b4a370 2024-09-19T07:01:39.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.361+0000 7f8749ffb640 1 -- 172.21.15.103:0/2009527021 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8754021030 con 0x5565f8b4a370 2024-09-19T07:01:39.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.361+0000 7f8749ffb640 1 -- 172.21.15.103:0/2009527021 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8758030b60 con 0x5565f8b4a370 2024-09-19T07:01:39.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.361+0000 7f876a6dd740 10 client.19740 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:39.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.361+0000 7f876a6dd740 10 client.19740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:39.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.361+0000 7f876a6dd740 20 client.19740 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:39.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.361+0000 7f876a6dd740 1 --2- 172.21.15.103:0/2009527021 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565f8bc5100 0x5565f8bc74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:39.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.361+0000 7f876a6dd740 4 client.19740 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:01:39.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.361+0000 7f876a6dd740 1 -- 172.21.15.103:0/2009527021 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565f8aacd70 con 0x5565f8bc5100 2024-09-19T07:01:39.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f87660a0640 1 --2- 172.21.15.103:0/2009527021 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565f8bc5100 0x5565f8bc74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:39.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f8749ffb640 10 client.19740 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:39.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f87660a0640 1 --2- 172.21.15.103:0/2009527021 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565f8bc5100 0x5565f8bc74f0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9365df5dbced648 server_cookie=5085987323a2436f in_seq=0 out_seq=0 2024-09-19T07:01:39.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f8749ffb640 1 -- 172.21.15.103:0/2009527021 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565f8aacd70 con 0x5565f8bc5100 2024-09-19T07:01:39.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f8749ffb640 10 client.19740 handle_command_reply: tid=0 2024-09-19T07:01:39.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f876a6dd740 1 client.19740 shutdown 2024-09-19T07:01:39.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f876a6dd740 1 -- 172.21.15.103:0/2009527021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b4a370 msgr2=0x5565f8b4e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:39.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f876a6dd740 1 --2- 172.21.15.103:0/2009527021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b4a370 0x5565f8b4e6e0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f8758009bf0 tx=0x7f8758031de0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f876a6dd740 1 -- 172.21.15.103:0/2009527021 shutdown_connections 2024-09-19T07:01:39.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f876a6dd740 1 --2- 172.21.15.103:0/2009527021 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565f8bc5100 0x5565f8bc74f0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f876a6dd740 1 --2- 172.21.15.103:0/2009527021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565f8b4a370 0x5565f8b4e6e0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f876a6dd740 1 --2- 172.21.15.103:0/2009527021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8b40b40 0x5565f8b4df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f876a6dd740 1 --2- 172.21.15.103:0/2009527021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f8b3f5d0 0x5565f8b52c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f8749ffb640 0 client.19740 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:39.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f876a6dd740 1 -- 172.21.15.103:0/2009527021 >> 172.21.15.103:0/2009527021 conn(0x5565f8b08310 msgr2=0x5565f8b48830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:39.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f876a6dd740 1 -- 172.21.15.103:0/2009527021 shutdown_connections 2024-09-19T07:01:39.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.362+0000 7f876a6dd740 1 -- 172.21.15.103:0/2009527021 wait complete. 2024-09-19T07:01:39.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.363+0000 7f876a6dd740 20 client.19740 trim_cache size 0 max 16384 2024-09-19T07:01:39.369 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:01:39.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.363+0000 7f8768304640 1 -- 172.21.15.103:0/2196664408 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f874c0a33b0 msgr2=0x7f874c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:39.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.363+0000 7f8768304640 1 --2- 172.21.15.103:0/2196664408 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f874c0a33b0 0x7f874c0a5870 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8758052000 tx=0x7f8758050000 comp rx=0 tx=0).stop 2024-09-19T07:01:39.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.363+0000 7f8768304640 1 -- 172.21.15.103:0/2196664408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8760100480 msgr2=0x7f8760198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:39.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.363+0000 7f8768304640 1 --2- 172.21.15.103:0/2196664408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8760100480 0x7f8760198580 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f875c00b4a0 tx=0x7f875c00b8a0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.363+0000 7f8768304640 1 -- 172.21.15.103:0/2196664408 shutdown_connections 2024-09-19T07:01:39.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.363+0000 7f8768304640 1 --2- 172.21.15.103:0/2196664408 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f874c0a33b0 0x7f874c0a5870 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.363+0000 7f8768304640 1 --2- 172.21.15.103:0/2196664408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87601055b0 0x7f876019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.363+0000 7f8768304640 1 --2- 172.21.15.103:0/2196664408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8760100e30 0x7f8760198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.363+0000 7f8768304640 1 --2- 172.21.15.103:0/2196664408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8760100480 0x7f8760198580 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:39.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.363+0000 7f8768304640 1 -- 172.21.15.103:0/2196664408 >> 172.21.15.103:0/2196664408 conn(0x7f8760077210 msgr2=0x7f87601035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:39.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.363+0000 7f8768304640 1 -- 172.21.15.103:0/2196664408 shutdown_connections 2024-09-19T07:01:39.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:39.363+0000 7f8768304640 1 -- 172.21.15.103:0/2196664408 wait complete. 2024-09-19T07:01:39.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:39 smithi174 ceph-mon[35637]: pgmap v387: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-09-19T07:01:39.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:39 smithi103 ceph-mon[31158]: pgmap v387: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-09-19T07:01:39.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:39 smithi134 ceph-mon[35356]: pgmap v387: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-09-19T07:01:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.009+0000 7febd5a8c640 1 -- 172.21.15.103:0/2823841504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febd0101350 msgr2=0x7febd010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.009+0000 7febd5a8c640 1 --2- 172.21.15.103:0/2823841504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febd0101350 0x7febd010d880 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7febc8009cd0 tx=0x7febc802a8e0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.009+0000 7febd5a8c640 1 -- 172.21.15.103:0/2823841504 shutdown_connections 2024-09-19T07:01:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.009+0000 7febd5a8c640 1 --2- 172.21.15.103:0/2823841504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febd0101350 0x7febd010d880 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.009+0000 7febd5a8c640 1 --2- 172.21.15.103:0/2823841504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd01009b0 0x7febd0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.009+0000 7febd5a8c640 1 --2- 172.21.15.103:0/2823841504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd01069c0 0x7febd0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.009+0000 7febd5a8c640 1 -- 172.21.15.103:0/2823841504 >> 172.21.15.103:0/2823841504 conn(0x7febd0077990 msgr2=0x7febd00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.009+0000 7febd5a8c640 1 -- 172.21.15.103:0/2823841504 shutdown_connections 2024-09-19T07:01:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.009+0000 7febd5a8c640 1 -- 172.21.15.103:0/2823841504 wait complete. 2024-09-19T07:01:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.010+0000 7febd5a8c640 1 Processor -- start 2024-09-19T07:01:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.010+0000 7febd5a8c640 1 -- start start 2024-09-19T07:01:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.010+0000 7febd5a8c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd01009b0 0x7febd0197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.010+0000 7febd5a8c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd0101350 0x7febd0197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.010+0000 7febd5a8c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febd01069c0 0x7febd019bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.010+0000 7febd5a8c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7febd0110e50 con 0x7febd01069c0 2024-09-19T07:01:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.010+0000 7febd5a8c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7febd0110cd0 con 0x7febd0101350 2024-09-19T07:01:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.010+0000 7febd5a8c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7febd0110fd0 con 0x7febd01009b0 2024-09-19T07:01:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.010+0000 7febcffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febd01069c0 0x7febd019bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.010+0000 7febcffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febd01069c0 0x7febd019bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57910/0 (socket says 172.21.15.103:57910) 2024-09-19T07:01:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.010+0000 7febcffff640 1 -- 172.21.15.103:0/1110600539 learned_addr learned my addr 172.21.15.103:0/1110600539 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.011+0000 7febceffd640 1 --2- 172.21.15.103:0/1110600539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd0101350 0x7febd0197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.011+0000 7febcf7fe640 1 --2- 172.21.15.103:0/1110600539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd01009b0 0x7febd0197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.011+0000 7febcffff640 1 -- 172.21.15.103:0/1110600539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd01009b0 msgr2=0x7febd0197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.011+0000 7febcffff640 1 --2- 172.21.15.103:0/1110600539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd01009b0 0x7febd0197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.011+0000 7febcffff640 1 -- 172.21.15.103:0/1110600539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd0101350 msgr2=0x7febd0197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.011+0000 7febcffff640 1 --2- 172.21.15.103:0/1110600539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd0101350 0x7febd0197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.011+0000 7febcffff640 1 -- 172.21.15.103:0/1110600539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd019c210 con 0x7febd01069c0 2024-09-19T07:01:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.011+0000 7febceffd640 1 --2- 172.21.15.103:0/1110600539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd0101350 0x7febd0197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:01:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.011+0000 7febcffff640 1 --2- 172.21.15.103:0/1110600539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febd01069c0 0x7febd019bae0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7febc8033040 tx=0x7febc8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.011+0000 7febccff9640 1 -- 172.21.15.103:0/1110600539 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8004040 con 0x7febd01069c0 2024-09-19T07:01:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.011+0000 7febccff9640 1 -- 172.21.15.103:0/1110600539 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febc80046c0 con 0x7febd01069c0 2024-09-19T07:01:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.011+0000 7febd5a8c640 1 -- 172.21.15.103:0/1110600539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd019c4a0 con 0x7febd01069c0 2024-09-19T07:01:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.011+0000 7febccff9640 1 -- 172.21.15.103:0/1110600539 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8046480 con 0x7febd01069c0 2024-09-19T07:01:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.011+0000 7febd5a8c640 1 -- 172.21.15.103:0/1110600539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd01a3d90 con 0x7febd01069c0 2024-09-19T07:01:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.014+0000 7febccff9640 1 -- 172.21.15.103:0/1110600539 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7febc80305a0 con 0x7febd01069c0 2024-09-19T07:01:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.014+0000 7febd8825740 1 Processor -- start 2024-09-19T07:01:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.014+0000 7febd8825740 1 -- start start 2024-09-19T07:01:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febd8825740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750ca10 0x55ae5750cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febd8825740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750d330 0x55ae57519980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febd8825740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae57519ec0 0x55ae5751c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febd8825740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ae574712f0 con 0x55ae57519ec0 2024-09-19T07:01:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febd8825740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ae57419270 con 0x55ae5750ca10 2024-09-19T07:01:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febd8825740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ae5746d2b0 con 0x55ae5750d330 2024-09-19T07:01:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febccff9640 1 --2- 172.21.15.103:0/1110600539 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7febb80a3460 0x7febb80a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febceffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae57519ec0 0x55ae5751c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febcf7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750d330 0x55ae57519980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febcf7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750d330 0x55ae57519980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:50584/0 (socket says 172.21.15.103:50584) 2024-09-19T07:01:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febcf7fe640 1 -- 172.21.15.103:0/540251835 learned_addr learned my addr 172.21.15.103:0/540251835 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febcffff640 1 --2- 172.21.15.103:0/1110600539 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7febb80a3460 0x7febb80a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febceffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae57519ec0 0x55ae5751c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57912/0 (socket says 172.21.15.103:57912) 2024-09-19T07:01:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febceffd640 1 --2- 172.21.15.103:0/540251835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750ca10 0x55ae5750cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febccff9640 1 -- 172.21.15.103:0/1110600539 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7febc802e030 con 0x7febd01069c0 2024-09-19T07:01:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febcf7fe640 1 -- 172.21.15.103:0/540251835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750ca10 msgr2=0x55ae5750cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febcf7fe640 1 --2- 172.21.15.103:0/540251835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750ca10 0x55ae5750cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febcf7fe640 1 -- 172.21.15.103:0/540251835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae57519ec0 msgr2=0x55ae5751c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febcf7fe640 1 --2- 172.21.15.103:0/540251835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae57519ec0 0x55ae5751c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febcf7fe640 1 -- 172.21.15.103:0/540251835 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae57418340 con 0x55ae5750d330 2024-09-19T07:01:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febceffd640 1 --2- 172.21.15.103:0/540251835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae57519ec0 0x55ae5751c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:01:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.015+0000 7febcffff640 1 --2- 172.21.15.103:0/1110600539 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7febb80a3460 0x7febb80a5920 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7febc80eec20 tx=0x7febc80af040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.016+0000 7febcf7fe640 1 --2- 172.21.15.103:0/540251835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750d330 0x55ae57519980 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7febc000ab90 tx=0x7febc000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39508fddcae8ee43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.016+0000 7febb5ffb640 1 -- 172.21.15.103:0/540251835 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc001d070 con 0x55ae5750d330 2024-09-19T07:01:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.016+0000 7febb5ffb640 1 -- 172.21.15.103:0/540251835 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febc0018b20 con 0x55ae5750d330 2024-09-19T07:01:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.016+0000 7febb5ffb640 1 -- 172.21.15.103:0/540251835 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0010a70 con 0x55ae5750d330 2024-09-19T07:01:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.017+0000 7febd8825740 1 -- 172.21.15.103:0/540251835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750d330 msgr2=0x55ae57519980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.017+0000 7febd8825740 1 --2- 172.21.15.103:0/540251835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750d330 0x55ae57519980 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7febc000ab90 tx=0x7febc000f790 comp rx=0 tx=0).stop 2024-09-19T07:01:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.017+0000 7febd8825740 1 -- 172.21.15.103:0/540251835 shutdown_connections 2024-09-19T07:01:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.017+0000 7febd8825740 1 --2- 172.21.15.103:0/540251835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae57519ec0 0x55ae5751c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.017+0000 7febd8825740 1 --2- 172.21.15.103:0/540251835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750d330 0x55ae57519980 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.017+0000 7febd8825740 1 --2- 172.21.15.103:0/540251835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750ca10 0x55ae5750cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.017+0000 7febd8825740 1 -- 172.21.15.103:0/540251835 >> 172.21.15.103:0/540251835 conn(0x55ae57509060 msgr2=0x55ae57509440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.017+0000 7febd8825740 1 -- 172.21.15.103:0/540251835 shutdown_connections 2024-09-19T07:01:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.017+0000 7febd8825740 1 -- 172.21.15.103:0/540251835 wait complete. 2024-09-19T07:01:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.017+0000 7febd8825740 1 -- start start 2024-09-19T07:01:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.017+0000 7febd8825740 1 Processor -- start 2024-09-19T07:01:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febd8825740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750ca10 0x55ae57516dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febd8825740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750d330 0x55ae57517300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febd8825740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae57519ec0 0x55ae57517840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febd8825740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ae5751ca90 con 0x55ae57519ec0 2024-09-19T07:01:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febd8825740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ae573fa290 con 0x55ae5750d330 2024-09-19T07:01:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febd8825740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ae5751cc10 con 0x55ae5750ca10 2024-09-19T07:01:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febd8825740 10 client.0 _ll_register_callbacks cb 0x55ae5742b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febcf7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750d330 0x55ae57517300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febceffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae57519ec0 0x55ae57517840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febcf7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750d330 0x55ae57517300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:51944/0 (socket says 172.21.15.103:51944) 2024-09-19T07:01:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febcf7fe640 1 -- 172.21.15.103:0/2353701368 learned_addr learned my addr 172.21.15.103:0/2353701368 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febceffd640 1 --2- 172.21.15.103:0/2353701368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750ca10 0x55ae57516dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febceffd640 1 -- 172.21.15.103:0/2353701368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750ca10 msgr2=0x55ae57516dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febceffd640 1 --2- 172.21.15.103:0/2353701368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750ca10 0x55ae57516dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febceffd640 1 -- 172.21.15.103:0/2353701368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750d330 msgr2=0x55ae57517300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febceffd640 1 --2- 172.21.15.103:0/2353701368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750d330 0x55ae57517300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febceffd640 1 -- 172.21.15.103:0/2353701368 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae57184210 con 0x55ae57519ec0 2024-09-19T07:01:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febcf7fe640 1 --2- 172.21.15.103:0/2353701368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750d330 0x55ae57517300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:01:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febceffd640 1 --2- 172.21.15.103:0/2353701368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae57519ec0 0x55ae57517840 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7febc4005d00 tx=0x7febc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d18f90a30a9c2a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.018+0000 7febaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.019+0000 7febaeffd640 1 -- 172.21.15.103:0/2353701368 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4004020 con 0x55ae57519ec0 2024-09-19T07:01:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.019+0000 7febaeffd640 1 -- 172.21.15.103:0/2353701368 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7febc40041c0 con 0x55ae57519ec0 2024-09-19T07:01:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.019+0000 7febd8825740 1 -- 172.21.15.103:0/2353701368 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febc0018b20 con 0x55ae57519ec0 2024-09-19T07:01:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.019+0000 7febaeffd640 1 -- 172.21.15.103:0/2353701368 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7febc402c050 con 0x55ae57519ec0 2024-09-19T07:01:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.019+0000 7febd8825740 10 client.19752 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.019+0000 7febd8825740 1 -- 172.21.15.103:0/2353701368 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ae57250620 con 0x55ae57519ec0 2024-09-19T07:01:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.019+0000 7febaeffd640 1 -- 172.21.15.103:0/2353701368 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febc0018b20 con 0x55ae57519ec0 2024-09-19T07:01:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.019+0000 7febaeffd640 1 -- 172.21.15.103:0/2353701368 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7febc400e560 con 0x55ae57519ec0 2024-09-19T07:01:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.019+0000 7febd8825740 10 client.19752 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.019+0000 7febd8825740 10 client.19752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.019+0000 7febd8825740 20 client.19752 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.019+0000 7febd8825740 1 --2- 172.21.15.103:0/2353701368 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ae57529e70 0x55ae5752c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.019+0000 7febd8825740 4 client.19752 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.019+0000 7febd8825740 1 -- 172.21.15.103:0/2353701368 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae5747ea20 con 0x55ae57529e70 2024-09-19T07:01:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.020+0000 7febcf7fe640 1 --2- 172.21.15.103:0/2353701368 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ae57529e70 0x55ae5752c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.020+0000 7febcf7fe640 1 --2- 172.21.15.103:0/2353701368 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ae57529e70 0x55ae5752c280 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=713afd6aeabcbe52 server_cookie=9c27953c64d184b5 in_seq=0 out_seq=0 2024-09-19T07:01:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.020+0000 7febaeffd640 10 client.19752 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.021+0000 7febaeffd640 1 -- 172.21.15.103:0/2353701368 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae5747ea20 con 0x55ae57529e70 2024-09-19T07:01:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.021+0000 7febaeffd640 10 client.19752 handle_command_reply: tid=0 2024-09-19T07:01:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.021+0000 7febd8825740 1 client.19752 shutdown 2024-09-19T07:01:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.021+0000 7febd8825740 1 -- 172.21.15.103:0/2353701368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae57519ec0 msgr2=0x55ae57517840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.021+0000 7febd8825740 1 --2- 172.21.15.103:0/2353701368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae57519ec0 0x55ae57517840 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7febc4005d00 tx=0x7febc4004730 comp rx=0 tx=0).stop 2024-09-19T07:01:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.021+0000 7febd8825740 1 -- 172.21.15.103:0/2353701368 shutdown_connections 2024-09-19T07:01:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.021+0000 7febd8825740 1 --2- 172.21.15.103:0/2353701368 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ae57529e70 0x55ae5752c280 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.021+0000 7febd8825740 1 --2- 172.21.15.103:0/2353701368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae57519ec0 0x55ae57517840 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.021+0000 7febd8825740 1 --2- 172.21.15.103:0/2353701368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750d330 0x55ae57517300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.021+0000 7febaeffd640 0 client.19752 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.021+0000 7febd8825740 1 --2- 172.21.15.103:0/2353701368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750ca10 0x55ae57516dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.021+0000 7febd8825740 1 -- 172.21.15.103:0/2353701368 >> 172.21.15.103:0/2353701368 conn(0x55ae57509060 msgr2=0x55ae5750ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.021+0000 7febd8825740 1 -- 172.21.15.103:0/2353701368 shutdown_connections 2024-09-19T07:01:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.021+0000 7febd8825740 1 -- 172.21.15.103:0/2353701368 wait complete. 2024-09-19T07:01:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.021+0000 7febd8825740 20 client.19752 trim_cache size 0 max 16384 2024-09-19T07:01:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.035+0000 7febd8825740 1 Processor -- start 2024-09-19T07:01:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.035+0000 7febd8825740 1 -- start start 2024-09-19T07:01:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febd8825740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750ca10 0x55ae57531b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febd8825740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5750d330 0x55ae575320d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febd8825740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae57519ec0 0x55ae57514040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febd8825740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7febd01ab5c0 con 0x55ae5750d330 2024-09-19T07:01:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febd8825740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7febd01ad100 con 0x55ae5750ca10 2024-09-19T07:01:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febd8825740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7febd01abc90 con 0x55ae57519ec0 2024-09-19T07:01:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febcf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5750d330 0x55ae575320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febcf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5750d330 0x55ae575320d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57922/0 (socket says 172.21.15.103:57922) 2024-09-19T07:01:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febcf7fe640 1 -- 172.21.15.103:0/2115053508 learned_addr learned my addr 172.21.15.103:0/2115053508 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febceffd640 1 --2- 172.21.15.103:0/2115053508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750ca10 0x55ae57531b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febcf7fe640 1 -- 172.21.15.103:0/2115053508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae57519ec0 msgr2=0x55ae57514040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febcf7fe640 1 --2- 172.21.15.103:0/2115053508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae57519ec0 0x55ae57514040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febcf7fe640 1 -- 172.21.15.103:0/2115053508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750ca10 msgr2=0x55ae57531b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febcf7fe640 1 --2- 172.21.15.103:0/2115053508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750ca10 0x55ae57531b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febcf7fe640 1 -- 172.21.15.103:0/2115053508 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae57511c90 con 0x55ae5750d330 2024-09-19T07:01:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febceffd640 1 --2- 172.21.15.103:0/2115053508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750ca10 0x55ae57531b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febcf7fe640 1 --2- 172.21.15.103:0/2115053508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5750d330 0x55ae575320d0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7febc00106c0 tx=0x7febc000fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34420ac9e69d354b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febb57fa640 1 -- 172.21.15.103:0/2115053508 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc002a070 con 0x55ae5750d330 2024-09-19T07:01:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febb57fa640 1 -- 172.21.15.103:0/2115053508 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febc001e040 con 0x55ae5750d330 2024-09-19T07:01:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.036+0000 7febb57fa640 1 -- 172.21.15.103:0/2115053508 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc001d040 con 0x55ae5750d330 2024-09-19T07:01:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.037+0000 7febd8825740 1 -- 172.21.15.103:0/2115053508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5750d330 msgr2=0x55ae575320d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.037+0000 7febd8825740 1 --2- 172.21.15.103:0/2115053508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5750d330 0x55ae575320d0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7febc00106c0 tx=0x7febc000fdc0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.037+0000 7febd8825740 1 -- 172.21.15.103:0/2115053508 shutdown_connections 2024-09-19T07:01:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.037+0000 7febd8825740 1 --2- 172.21.15.103:0/2115053508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae57519ec0 0x55ae57514040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.037+0000 7febd8825740 1 --2- 172.21.15.103:0/2115053508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5750d330 0x55ae575320d0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.037+0000 7febd8825740 1 --2- 172.21.15.103:0/2115053508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae5750ca10 0x55ae57531b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.037+0000 7febd8825740 1 -- 172.21.15.103:0/2115053508 >> 172.21.15.103:0/2115053508 conn(0x55ae57509060 msgr2=0x55ae57518e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.037+0000 7febd8825740 1 -- 172.21.15.103:0/2115053508 shutdown_connections 2024-09-19T07:01:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.037+0000 7febd8825740 1 -- 172.21.15.103:0/2115053508 wait complete. 2024-09-19T07:01:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.037+0000 7febd8825740 1 -- start start 2024-09-19T07:01:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febd8825740 1 Processor -- start 2024-09-19T07:01:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febd8825740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750ca10 0x55ae57513e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febd8825740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5750d330 0x55ae575143a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febd8825740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae57519ec0 0x55ae575148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febd8825740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7febd01aca30 con 0x55ae5750d330 2024-09-19T07:01:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febd8825740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7febd01ac360 con 0x55ae57519ec0 2024-09-19T07:01:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febd8825740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7febd01a95c0 con 0x55ae5750ca10 2024-09-19T07:01:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febd8825740 10 client.0 _ll_register_callbacks cb 0x7febc4012cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febcf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5750d330 0x55ae575143a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febcf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5750d330 0x55ae575143a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57938/0 (socket says 172.21.15.103:57938) 2024-09-19T07:01:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febcf7fe640 1 -- 172.21.15.103:0/2325376981 learned_addr learned my addr 172.21.15.103:0/2325376981 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febceffd640 1 --2- 172.21.15.103:0/2325376981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750ca10 0x55ae57513e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febceffd640 1 --2- 172.21.15.103:0/2325376981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae57519ec0 0x55ae575148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febcf7fe640 1 -- 172.21.15.103:0/2325376981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750ca10 msgr2=0x55ae57513e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febcf7fe640 1 --2- 172.21.15.103:0/2325376981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750ca10 0x55ae57513e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febcf7fe640 1 -- 172.21.15.103:0/2325376981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae57519ec0 msgr2=0x55ae575148e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febcf7fe640 1 --2- 172.21.15.103:0/2325376981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae57519ec0 0x55ae575148e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febcf7fe640 1 -- 172.21.15.103:0/2325376981 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae57593be0 con 0x55ae5750d330 2024-09-19T07:01:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febceffd640 1 --2- 172.21.15.103:0/2325376981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae57519ec0 0x55ae575148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febcf7fe640 1 --2- 172.21.15.103:0/2325376981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5750d330 0x55ae575143a0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7febc000ab60 tx=0x7febc000af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fc8b5ca451f7e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.038+0000 7febaeffd640 1 -- 172.21.15.103:0/2325376981 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0033030 con 0x55ae5750d330 2024-09-19T07:01:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.039+0000 7febaeffd640 1 -- 172.21.15.103:0/2325376981 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7febc0010b80 con 0x55ae5750d330 2024-09-19T07:01:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.039+0000 7febd8825740 1 -- 172.21.15.103:0/2325376981 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febc001e040 con 0x55ae5750d330 2024-09-19T07:01:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.039+0000 7febaeffd640 1 -- 172.21.15.103:0/2325376981 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7febc001d070 con 0x55ae5750d330 2024-09-19T07:01:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.039+0000 7febd8825740 10 client.19758 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.039+0000 7febd8825740 1 -- 172.21.15.103:0/2325376981 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ae57594e20 con 0x55ae5750d330 2024-09-19T07:01:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.039+0000 7febaeffd640 1 -- 172.21.15.103:0/2325376981 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febc001e040 con 0x55ae5750d330 2024-09-19T07:01:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.039+0000 7febaeffd640 1 -- 172.21.15.103:0/2325376981 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7febc0024040 con 0x55ae5750d330 2024-09-19T07:01:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.039+0000 7febd8825740 10 client.19758 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.039+0000 7febd8825740 10 client.19758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.039+0000 7febd8825740 20 client.19758 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.039+0000 7febd8825740 1 --2- 172.21.15.103:0/2325376981 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ae57595370 0x55ae57597790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.039+0000 7febd8825740 4 client.19758 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:01:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.039+0000 7febd8825740 1 -- 172.21.15.103:0/2325376981 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae57478d70 con 0x55ae57595370 2024-09-19T07:01:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.039+0000 7febceffd640 1 --2- 172.21.15.103:0/2325376981 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ae57595370 0x55ae57597790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.040+0000 7febceffd640 1 --2- 172.21.15.103:0/2325376981 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ae57595370 0x55ae57597790 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fc753ca4f8092f0 server_cookie=5c9246a31c6f7b51 in_seq=0 out_seq=0 2024-09-19T07:01:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.040+0000 7febaeffd640 10 client.19758 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:41.047 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:01:41.047 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:01:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.053+0000 7febaeffd640 1 -- 172.21.15.103:0/2325376981 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae57478d70 con 0x55ae57595370 2024-09-19T07:01:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.053+0000 7febaeffd640 10 client.19758 handle_command_reply: tid=0 2024-09-19T07:01:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.053+0000 7febd8825740 1 client.19758 shutdown 2024-09-19T07:01:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.054+0000 7febd8825740 1 -- 172.21.15.103:0/2325376981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5750d330 msgr2=0x55ae575143a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.054+0000 7febd8825740 1 --2- 172.21.15.103:0/2325376981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5750d330 0x55ae575143a0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7febc000ab60 tx=0x7febc000af20 comp rx=0 tx=0).stop 2024-09-19T07:01:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.054+0000 7febd8825740 1 -- 172.21.15.103:0/2325376981 shutdown_connections 2024-09-19T07:01:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.054+0000 7febd8825740 1 --2- 172.21.15.103:0/2325376981 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ae57595370 0x55ae57597790 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.054+0000 7febd8825740 1 --2- 172.21.15.103:0/2325376981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae57519ec0 0x55ae575148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.054+0000 7febd8825740 1 --2- 172.21.15.103:0/2325376981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5750d330 0x55ae575143a0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.054+0000 7febaeffd640 0 client.19758 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.054+0000 7febd8825740 1 --2- 172.21.15.103:0/2325376981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae5750ca10 0x55ae57513e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.054+0000 7febd8825740 1 -- 172.21.15.103:0/2325376981 >> 172.21.15.103:0/2325376981 conn(0x55ae57509060 msgr2=0x55ae57516d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.054+0000 7febd8825740 1 -- 172.21.15.103:0/2325376981 shutdown_connections 2024-09-19T07:01:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.054+0000 7febd8825740 1 -- 172.21.15.103:0/2325376981 wait complete. 2024-09-19T07:01:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.054+0000 7febd8825740 20 client.19758 trim_cache size 0 max 16384 2024-09-19T07:01:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:01:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:01:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:01:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:01:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.055+0000 7febd5a8c640 1 -- 172.21.15.103:0/1110600539 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7febb80a3460 msgr2=0x7febb80a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.055+0000 7febd5a8c640 1 --2- 172.21.15.103:0/1110600539 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7febb80a3460 0x7febb80a5920 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7febc80eec20 tx=0x7febc80af040 comp rx=0 tx=0).stop 2024-09-19T07:01:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.055+0000 7febd5a8c640 1 -- 172.21.15.103:0/1110600539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febd01069c0 msgr2=0x7febd019bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.055+0000 7febd5a8c640 1 --2- 172.21.15.103:0/1110600539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febd01069c0 0x7febd019bae0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7febc8033040 tx=0x7febc8003970 comp rx=0 tx=0).stop 2024-09-19T07:01:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.055+0000 7febd5a8c640 1 -- 172.21.15.103:0/1110600539 shutdown_connections 2024-09-19T07:01:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.055+0000 7febd5a8c640 1 --2- 172.21.15.103:0/1110600539 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7febb80a3460 0x7febb80a5920 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.055+0000 7febd5a8c640 1 --2- 172.21.15.103:0/1110600539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febd01069c0 0x7febd019bae0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.055+0000 7febd5a8c640 1 --2- 172.21.15.103:0/1110600539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd0101350 0x7febd0197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.055+0000 7febd5a8c640 1 --2- 172.21.15.103:0/1110600539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd01009b0 0x7febd0197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.055+0000 7febd5a8c640 1 -- 172.21.15.103:0/1110600539 >> 172.21.15.103:0/1110600539 conn(0x7febd0077990 msgr2=0x7febd00fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.055+0000 7febd5a8c640 1 -- 172.21.15.103:0/1110600539 shutdown_connections 2024-09-19T07:01:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:41.055+0000 7febd5a8c640 1 -- 172.21.15.103:0/1110600539 wait complete. 2024-09-19T07:01:41.296 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:41.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:41 smithi103 ceph-mon[31158]: pgmap v388: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-19T07:01:41.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:41 smithi174 ceph-mon[35637]: pgmap v388: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-19T07:01:41.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:41 smithi134 ceph-mon[35356]: pgmap v388: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-19T07:01:41.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.773+0000 7fed24bb6640 1 -- 172.21.15.103:0/1972250954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed20106f10 msgr2=0x7fed2010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.773+0000 7fed24bb6640 1 --2- 172.21.15.103:0/1972250954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed20106f10 0x7fed2010ea90 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fed18009900 tx=0x7fed1802a410 comp rx=0 tx=0).stop 2024-09-19T07:01:41.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.773+0000 7fed24bb6640 1 -- 172.21.15.103:0/1972250954 shutdown_connections 2024-09-19T07:01:41.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.773+0000 7fed24bb6640 1 --2- 172.21.15.103:0/1972250954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed20106f10 0x7fed2010ea90 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.773+0000 7fed24bb6640 1 --2- 172.21.15.103:0/1972250954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed20102790 0x7fed201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.773+0000 7fed24bb6640 1 --2- 172.21.15.103:0/1972250954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed20101de0 0x7fed201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.773+0000 7fed24bb6640 1 -- 172.21.15.103:0/1972250954 >> 172.21.15.103:0/1972250954 conn(0x7fed200fd8c0 msgr2=0x7fed200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:41.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.773+0000 7fed24bb6640 1 -- 172.21.15.103:0/1972250954 shutdown_connections 2024-09-19T07:01:41.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.773+0000 7fed24bb6640 1 -- 172.21.15.103:0/1972250954 wait complete. 2024-09-19T07:01:41.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.774+0000 7fed24bb6640 1 Processor -- start 2024-09-19T07:01:41.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.774+0000 7fed24bb6640 1 -- start start 2024-09-19T07:01:41.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.774+0000 7fed24bb6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed20101de0 0x7fed201983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.774+0000 7fed24bb6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed20102790 0x7fed20198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.774+0000 7fed24bb6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed20106f10 0x7fed2019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.774+0000 7fed24bb6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed20111f70 con 0x7fed20102790 2024-09-19T07:01:41.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.774+0000 7fed24bb6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed20111df0 con 0x7fed20101de0 2024-09-19T07:01:41.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.774+0000 7fed24bb6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed201120f0 con 0x7fed20106f10 2024-09-19T07:01:41.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.774+0000 7fed1dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed20102790 0x7fed20198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.774+0000 7fed1dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed20102790 0x7fed20198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57956/0 (socket says 172.21.15.103:57956) 2024-09-19T07:01:41.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.774+0000 7fed1dd9b640 1 -- 172.21.15.103:0/2175400343 learned_addr learned my addr 172.21.15.103:0/2175400343 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:41.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.774+0000 7fed1e59c640 1 --2- 172.21.15.103:0/2175400343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed20101de0 0x7fed201983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.775+0000 7fed1ed9d640 1 --2- 172.21.15.103:0/2175400343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed20106f10 0x7fed2019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.775+0000 7fed1dd9b640 1 -- 172.21.15.103:0/2175400343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed20106f10 msgr2=0x7fed2019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.775+0000 7fed1dd9b640 1 --2- 172.21.15.103:0/2175400343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed20106f10 0x7fed2019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.775+0000 7fed1dd9b640 1 -- 172.21.15.103:0/2175400343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed20101de0 msgr2=0x7fed201983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.775+0000 7fed1dd9b640 1 --2- 172.21.15.103:0/2175400343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed20101de0 0x7fed201983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.775+0000 7fed1dd9b640 1 -- 172.21.15.103:0/2175400343 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2019d380 con 0x7fed20102790 2024-09-19T07:01:41.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.775+0000 7fed1ed9d640 1 --2- 172.21.15.103:0/2175400343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed20106f10 0x7fed2019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:41.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.775+0000 7fed1dd9b640 1 --2- 172.21.15.103:0/2175400343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed20102790 0x7fed20198910 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fed1400c890 tx=0x7fed1400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:41.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.775+0000 7fed0f7fe640 1 -- 172.21.15.103:0/2175400343 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed14004220 con 0x7fed20102790 2024-09-19T07:01:41.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.775+0000 7fed0f7fe640 1 -- 172.21.15.103:0/2175400343 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed140043c0 con 0x7fed20102790 2024-09-19T07:01:41.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.775+0000 7fed24bb6640 1 -- 172.21.15.103:0/2175400343 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2019d670 con 0x7fed20102790 2024-09-19T07:01:41.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.775+0000 7fed0f7fe640 1 -- 172.21.15.103:0/2175400343 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed140049e0 con 0x7fed20102790 2024-09-19T07:01:41.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.775+0000 7fed24bb6640 1 -- 172.21.15.103:0/2175400343 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed201a4f50 con 0x7fed20102790 2024-09-19T07:01:41.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.777+0000 7fed0f7fe640 1 -- 172.21.15.103:0/2175400343 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fed140079b0 con 0x7fed20102790 2024-09-19T07:01:41.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.778+0000 7fed26f8f740 1 Processor -- start 2024-09-19T07:01:41.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.778+0000 7fed26f8f740 1 -- start start 2024-09-19T07:01:41.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.778+0000 7fed26f8f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 0x55558b16bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.778+0000 7fed26f8f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b16c480 0x55558b176800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.778+0000 7fed26f8f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b176d40 0x55558b179130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.778+0000 7fed26f8f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55558b0d1280 con 0x55558b176d40 2024-09-19T07:01:41.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.778+0000 7fed26f8f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55558b078b10 con 0x55558b16c480 2024-09-19T07:01:41.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.778+0000 7fed26f8f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55558b0cd240 con 0x55558b16bb60 2024-09-19T07:01:41.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.778+0000 7fed0f7fe640 1 --2- 172.21.15.103:0/2175400343 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed080a3430 0x7fed080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.778+0000 7fed1dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b176d40 0x55558b179130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.779+0000 7fed1ed9d640 1 --2- 172.21.15.103:0/2175400343 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed080a3430 0x7fed080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.779+0000 7fed1dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b176d40 0x55558b179130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57958/0 (socket says 172.21.15.103:57958) 2024-09-19T07:01:41.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.779+0000 7fed1dd9b640 1 -- 172.21.15.103:0/4027756152 learned_addr learned my addr 172.21.15.103:0/4027756152 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:41.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.779+0000 7fed1e59c640 1 --2- 172.21.15.103:0/4027756152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b16c480 0x55558b176800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.779+0000 7fed1ed9d640 1 --2- 172.21.15.103:0/4027756152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 0x55558b16bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.779+0000 7fed0f7fe640 1 -- 172.21.15.103:0/2175400343 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7fed140cfcc0 con 0x7fed20102790 2024-09-19T07:01:41.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.779+0000 7fed1e59c640 1 -- 172.21.15.103:0/4027756152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 msgr2=0x55558b16bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:41.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.779+0000 7fed1e59c640 1 --2- 172.21.15.103:0/4027756152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 0x55558b16bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.779+0000 7fed1e59c640 1 -- 172.21.15.103:0/4027756152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b176d40 msgr2=0x55558b179130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:41.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.779+0000 7fed1e59c640 1 --2- 172.21.15.103:0/4027756152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b176d40 0x55558b179130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.779+0000 7fed1e59c640 1 -- 172.21.15.103:0/4027756152 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55558b0851f0 con 0x55558b16c480 2024-09-19T07:01:41.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.779+0000 7fed1dd9b640 1 --2- 172.21.15.103:0/4027756152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b176d40 0x55558b179130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:01:41.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.779+0000 7fed1ed9d640 1 --2- 172.21.15.103:0/2175400343 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed080a3430 0x7fed080a58f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fed18005ab0 tx=0x7fed180095e0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:41.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.779+0000 7fed1e59c640 1 --2- 172.21.15.103:0/4027756152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b16c480 0x55558b176800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fed1000ae70 tx=0x7fed1000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88ab08bb06ea8ec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:41.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.779+0000 7fed0cff9640 1 -- 172.21.15.103:0/4027756152 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10003b70 con 0x55558b16c480 2024-09-19T07:01:41.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.780+0000 7fed0cff9640 1 -- 172.21.15.103:0/4027756152 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed10003d10 con 0x55558b16c480 2024-09-19T07:01:41.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.780+0000 7fed0cff9640 1 -- 172.21.15.103:0/4027756152 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10010b40 con 0x55558b16c480 2024-09-19T07:01:41.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.780+0000 7fed26f8f740 1 -- 172.21.15.103:0/4027756152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b16c480 msgr2=0x55558b176800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.780+0000 7fed26f8f740 1 --2- 172.21.15.103:0/4027756152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b16c480 0x55558b176800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fed1000ae70 tx=0x7fed1000f790 comp rx=0 tx=0).stop 2024-09-19T07:01:41.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.780+0000 7fed26f8f740 1 -- 172.21.15.103:0/4027756152 shutdown_connections 2024-09-19T07:01:41.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.780+0000 7fed26f8f740 1 --2- 172.21.15.103:0/4027756152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b176d40 0x55558b179130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.780+0000 7fed26f8f740 1 --2- 172.21.15.103:0/4027756152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b16c480 0x55558b176800 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.780+0000 7fed26f8f740 1 --2- 172.21.15.103:0/4027756152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 0x55558b16bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.780+0000 7fed26f8f740 1 -- 172.21.15.103:0/4027756152 >> 172.21.15.103:0/4027756152 conn(0x55558b1680f0 msgr2=0x55558b1684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:41.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.780+0000 7fed26f8f740 1 -- 172.21.15.103:0/4027756152 shutdown_connections 2024-09-19T07:01:41.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.780+0000 7fed26f8f740 1 -- 172.21.15.103:0/4027756152 wait complete. 2024-09-19T07:01:41.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.781+0000 7fed26f8f740 1 -- start start 2024-09-19T07:01:41.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.781+0000 7fed26f8f740 1 Processor -- start 2024-09-19T07:01:41.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.781+0000 7fed26f8f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 0x55558b176760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.781+0000 7fed26f8f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b173ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.781+0000 7fed26f8f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b176d40 0x55558b1740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.781+0000 7fed26f8f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55558b17dd20 con 0x55558b16c480 2024-09-19T07:01:41.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.781+0000 7fed26f8f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55558b08bf10 con 0x55558b176d40 2024-09-19T07:01:41.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.781+0000 7fed26f8f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55558b17dea0 con 0x55558b16bb60 2024-09-19T07:01:41.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.781+0000 7fed26f8f740 10 client.0 _ll_register_callbacks cb 0x55558b054a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:41.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.781+0000 7fed1dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b173ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.781+0000 7fed1dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b173ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57962/0 (socket says 172.21.15.103:57962) 2024-09-19T07:01:41.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.781+0000 7fed1dd9b640 1 -- 172.21.15.103:0/4102004404 learned_addr learned my addr 172.21.15.103:0/4102004404 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:41.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.781+0000 7fed1ed9d640 1 --2- 172.21.15.103:0/4102004404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b176d40 0x55558b1740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.781+0000 7fed1e59c640 1 --2- 172.21.15.103:0/4102004404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 0x55558b176760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.781+0000 7fed1dd9b640 1 -- 172.21.15.103:0/4102004404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 msgr2=0x55558b176760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:41.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.781+0000 7fed1dd9b640 1 --2- 172.21.15.103:0/4102004404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 0x55558b176760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.782+0000 7fed1dd9b640 1 -- 172.21.15.103:0/4102004404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b176d40 msgr2=0x55558b1740c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:41.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.782+0000 7fed1dd9b640 1 --2- 172.21.15.103:0/4102004404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b176d40 0x55558b1740c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.782+0000 7fed1dd9b640 1 -- 172.21.15.103:0/4102004404 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55558ae426a0 con 0x55558b16c480 2024-09-19T07:01:41.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.782+0000 7fed1dd9b640 1 --2- 172.21.15.103:0/4102004404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b173ab0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fed14098390 tx=0x7fed1406ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b6c96c6de97c56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:41.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.782+0000 7fed05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:41.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.782+0000 7fed05ffb640 1 -- 172.21.15.103:0/4102004404 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1401d040 con 0x55558b16c480 2024-09-19T07:01:41.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.782+0000 7fed05ffb640 1 -- 172.21.15.103:0/4102004404 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7fed14071160 con 0x55558b16c480 2024-09-19T07:01:41.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.782+0000 7fed26f8f740 1 -- 172.21.15.103:0/4102004404 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed10003d10 con 0x55558b16c480 2024-09-19T07:01:41.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.782+0000 7fed05ffb640 1 -- 172.21.15.103:0/4102004404 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed14071360 con 0x55558b16c480 2024-09-19T07:01:41.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.782+0000 7fed26f8f740 10 client.19770 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:41.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.782+0000 7fed26f8f740 1 -- 172.21.15.103:0/4102004404 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55558b189810 con 0x55558b16c480 2024-09-19T07:01:41.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.782+0000 7fed05ffb640 1 -- 172.21.15.103:0/4102004404 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed10003d10 con 0x55558b16c480 2024-09-19T07:01:41.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.782+0000 7fed05ffb640 1 -- 172.21.15.103:0/4102004404 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fed1409c1e0 con 0x55558b16c480 2024-09-19T07:01:41.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.782+0000 7fed26f8f740 10 client.19770 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:41.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.782+0000 7fed26f8f740 10 client.19770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:41.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.783+0000 7fed26f8f740 20 client.19770 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:41.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.783+0000 7fed26f8f740 1 --2- 172.21.15.103:0/4102004404 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55558b18ac70 0x55558b18d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.783+0000 7fed26f8f740 4 client.19770 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:41.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.783+0000 7fed26f8f740 1 -- 172.21.15.103:0/4102004404 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55558b0de7f0 con 0x55558b18ac70 2024-09-19T07:01:41.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.783+0000 7fed1e59c640 1 --2- 172.21.15.103:0/4102004404 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55558b18ac70 0x55558b18d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.783+0000 7fed05ffb640 10 client.19770 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:41.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.783+0000 7fed1e59c640 1 --2- 172.21.15.103:0/4102004404 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55558b18ac70 0x55558b18d060 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b315609fb4b509e server_cookie=ffc758c1f20ec694 in_seq=0 out_seq=0 2024-09-19T07:01:41.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.784+0000 7fed05ffb640 1 -- 172.21.15.103:0/4102004404 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55558b0de7f0 con 0x55558b18ac70 2024-09-19T07:01:41.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.784+0000 7fed05ffb640 10 client.19770 handle_command_reply: tid=0 2024-09-19T07:01:41.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.784+0000 7fed26f8f740 1 client.19770 shutdown 2024-09-19T07:01:41.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.784+0000 7fed26f8f740 1 -- 172.21.15.103:0/4102004404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 msgr2=0x55558b173ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.784+0000 7fed26f8f740 1 --2- 172.21.15.103:0/4102004404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b173ab0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fed14098390 tx=0x7fed1406ac70 comp rx=0 tx=0).stop 2024-09-19T07:01:41.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.784+0000 7fed26f8f740 1 -- 172.21.15.103:0/4102004404 shutdown_connections 2024-09-19T07:01:41.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.784+0000 7fed26f8f740 1 --2- 172.21.15.103:0/4102004404 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55558b18ac70 0x55558b18d060 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.784+0000 7fed26f8f740 1 --2- 172.21.15.103:0/4102004404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b176d40 0x55558b1740c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.784+0000 7fed26f8f740 1 --2- 172.21.15.103:0/4102004404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b173ab0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.784+0000 7fed26f8f740 1 --2- 172.21.15.103:0/4102004404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 0x55558b176760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.784+0000 7fed26f8f740 1 -- 172.21.15.103:0/4102004404 >> 172.21.15.103:0/4102004404 conn(0x55558b1680f0 msgr2=0x55558b16ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:41.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.784+0000 7fed05ffb640 0 client.19770 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:41.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.784+0000 7fed26f8f740 1 -- 172.21.15.103:0/4102004404 shutdown_connections 2024-09-19T07:01:41.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.784+0000 7fed26f8f740 1 -- 172.21.15.103:0/4102004404 wait complete. 2024-09-19T07:01:41.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.784+0000 7fed26f8f740 20 client.19770 trim_cache size 0 max 16384 2024-09-19T07:01:41.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.801+0000 7fed26f8f740 1 Processor -- start 2024-09-19T07:01:41.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.801+0000 7fed26f8f740 1 -- start start 2024-09-19T07:01:41.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.801+0000 7fed26f8f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 0x55558b186f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.801+0000 7fed26f8f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b16dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.801+0000 7fed26f8f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b176d40 0x55558b192930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.801+0000 7fed26f8f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed201ab6d0 con 0x55558b16c480 2024-09-19T07:01:41.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.801+0000 7fed26f8f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed201ad210 con 0x55558b176d40 2024-09-19T07:01:41.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.801+0000 7fed26f8f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed201abda0 con 0x55558b16bb60 2024-09-19T07:01:41.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.801+0000 7fed1dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b16dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.801+0000 7fed1dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b16dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57964/0 (socket says 172.21.15.103:57964) 2024-09-19T07:01:41.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.801+0000 7fed1dd9b640 1 -- 172.21.15.103:0/39098598 learned_addr learned my addr 172.21.15.103:0/39098598 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:41.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.802+0000 7fed1e59c640 1 --2- 172.21.15.103:0/39098598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 0x55558b186f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.802+0000 7fed1dd9b640 1 -- 172.21.15.103:0/39098598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 msgr2=0x55558b186f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:41.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.802+0000 7fed1dd9b640 1 --2- 172.21.15.103:0/39098598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 0x55558b186f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.802+0000 7fed1dd9b640 1 -- 172.21.15.103:0/39098598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b176d40 msgr2=0x55558b192930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:41.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.802+0000 7fed1dd9b640 1 --2- 172.21.15.103:0/39098598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b176d40 0x55558b192930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.802+0000 7fed1dd9b640 1 -- 172.21.15.103:0/39098598 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55558b168ca0 con 0x55558b16c480 2024-09-19T07:01:41.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.802+0000 7fed1dd9b640 1 --2- 172.21.15.103:0/39098598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b16dcc0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fed140cd9f0 tx=0x7fed140054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e47eebceffc733 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:41.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.802+0000 7fed07fff640 1 -- 172.21.15.103:0/39098598 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1409c680 con 0x55558b16c480 2024-09-19T07:01:41.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.802+0000 7fed07fff640 1 -- 172.21.15.103:0/39098598 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed140b6070 con 0x55558b16c480 2024-09-19T07:01:41.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.802+0000 7fed07fff640 1 -- 172.21.15.103:0/39098598 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed140a4760 con 0x55558b16c480 2024-09-19T07:01:41.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 -- 172.21.15.103:0/39098598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 msgr2=0x55558b16dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 --2- 172.21.15.103:0/39098598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b16dcc0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fed140cd9f0 tx=0x7fed140054a0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 -- 172.21.15.103:0/39098598 shutdown_connections 2024-09-19T07:01:41.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 --2- 172.21.15.103:0/39098598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b176d40 0x55558b192930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 --2- 172.21.15.103:0/39098598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b16dcc0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 --2- 172.21.15.103:0/39098598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 0x55558b186f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 -- 172.21.15.103:0/39098598 >> 172.21.15.103:0/39098598 conn(0x55558b1680f0 msgr2=0x55558b16d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:41.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 -- 172.21.15.103:0/39098598 shutdown_connections 2024-09-19T07:01:41.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 -- 172.21.15.103:0/39098598 wait complete. 2024-09-19T07:01:41.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 -- start start 2024-09-19T07:01:41.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 Processor -- start 2024-09-19T07:01:41.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 0x55558b183100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b183640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b176d40 0x55558b17e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed201acb40 con 0x55558b16c480 2024-09-19T07:01:41.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed201ac470 con 0x55558b176d40 2024-09-19T07:01:41.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed201a96d0 con 0x55558b16bb60 2024-09-19T07:01:41.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed26f8f740 10 client.0 _ll_register_callbacks cb 0x7fed140046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:41.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.803+0000 7fed1dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b183640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed1dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b183640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57980/0 (socket says 172.21.15.103:57980) 2024-09-19T07:01:41.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed1dd9b640 1 -- 172.21.15.103:0/441376930 learned_addr learned my addr 172.21.15.103:0/441376930 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:41.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed1e59c640 1 --2- 172.21.15.103:0/441376930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 0x55558b183100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed1dd9b640 1 -- 172.21.15.103:0/441376930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 msgr2=0x55558b183100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed1dd9b640 1 --2- 172.21.15.103:0/441376930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 0x55558b183100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed1dd9b640 1 -- 172.21.15.103:0/441376930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b176d40 msgr2=0x55558b17e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed1dd9b640 1 --2- 172.21.15.103:0/441376930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b176d40 0x55558b17e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed1dd9b640 1 -- 172.21.15.103:0/441376930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55558b176ad0 con 0x55558b16c480 2024-09-19T07:01:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed1dd9b640 1 --2- 172.21.15.103:0/441376930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b183640 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fed14004770 tx=0x7fed1409c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d6711a03deac1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed05ffb640 1 -- 172.21.15.103:0/441376930 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed14076060 con 0x55558b16c480 2024-09-19T07:01:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed05ffb640 1 -- 172.21.15.103:0/441376930 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7fed140babc0 con 0x55558b16c480 2024-09-19T07:01:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed26f8f740 1 -- 172.21.15.103:0/441376930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed140b6070 con 0x55558b16c480 2024-09-19T07:01:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed05ffb640 1 -- 172.21.15.103:0/441376930 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed1409c680 con 0x55558b16c480 2024-09-19T07:01:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed26f8f740 10 client.19776 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed26f8f740 1 -- 172.21.15.103:0/441376930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55558b1f60a0 con 0x55558b16c480 2024-09-19T07:01:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.804+0000 7fed05ffb640 1 -- 172.21.15.103:0/441376930 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed140b6070 con 0x55558b16c480 2024-09-19T07:01:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.805+0000 7fed05ffb640 1 -- 172.21.15.103:0/441376930 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fed140cdb40 con 0x55558b16c480 2024-09-19T07:01:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.805+0000 7fed26f8f740 10 client.19776 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.805+0000 7fed26f8f740 10 client.19776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.805+0000 7fed26f8f740 20 client.19776 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.805+0000 7fed26f8f740 1 --2- 172.21.15.103:0/441376930 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55558b1f6550 0x55558b1f8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.805+0000 7fed26f8f740 4 client.19776 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:01:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.805+0000 7fed26f8f740 1 -- 172.21.15.103:0/441376930 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55558b0d8d00 con 0x55558b1f6550 2024-09-19T07:01:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.805+0000 7fed1e59c640 1 --2- 172.21.15.103:0/441376930 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55558b1f6550 0x55558b1f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.805+0000 7fed05ffb640 10 client.19776 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.805+0000 7fed1e59c640 1 --2- 172.21.15.103:0/441376930 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55558b1f6550 0x55558b1f8970 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de06e6c136172cab server_cookie=7815618da71b5ee in_seq=0 out_seq=0 2024-09-19T07:01:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.806+0000 7fed05ffb640 1 -- 172.21.15.103:0/441376930 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55558b0d8d00 con 0x55558b1f6550 2024-09-19T07:01:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.806+0000 7fed05ffb640 10 client.19776 handle_command_reply: tid=0 2024-09-19T07:01:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.806+0000 7fed26f8f740 1 client.19776 shutdown 2024-09-19T07:01:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.806+0000 7fed26f8f740 1 -- 172.21.15.103:0/441376930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 msgr2=0x55558b183640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed26f8f740 1 --2- 172.21.15.103:0/441376930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b183640 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fed14004770 tx=0x7fed1409c410 comp rx=0 tx=0).stop 2024-09-19T07:01:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed26f8f740 1 -- 172.21.15.103:0/441376930 shutdown_connections 2024-09-19T07:01:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed26f8f740 1 --2- 172.21.15.103:0/441376930 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55558b1f6550 0x55558b1f8970 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed26f8f740 1 --2- 172.21.15.103:0/441376930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55558b176d40 0x55558b17e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed05ffb640 0 client.19776 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed26f8f740 1 --2- 172.21.15.103:0/441376930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558b16c480 0x55558b183640 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed26f8f740 1 --2- 172.21.15.103:0/441376930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558b16bb60 0x55558b183100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed26f8f740 1 -- 172.21.15.103:0/441376930 >> 172.21.15.103:0/441376930 conn(0x55558b1680f0 msgr2=0x55558b166a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed26f8f740 1 -- 172.21.15.103:0/441376930 shutdown_connections 2024-09-19T07:01:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed26f8f740 1 -- 172.21.15.103:0/441376930 wait complete. 2024-09-19T07:01:41.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed26f8f740 20 client.19776 trim_cache size 0 max 16384 2024-09-19T07:01:41.815 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:01:41.815 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:01:41.815 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "17d61dfb-6e9c-4bff-9e3b-2a71bbdce090", 2024-09-19T07:01:41.815 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:01:41.815 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:01:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed24bb6640 1 -- 172.21.15.103:0/2175400343 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed080a3430 msgr2=0x7fed080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed24bb6640 1 --2- 172.21.15.103:0/2175400343 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed080a3430 0x7fed080a58f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fed18005ab0 tx=0x7fed180095e0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed24bb6640 1 -- 172.21.15.103:0/2175400343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed20102790 msgr2=0x7fed20198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed24bb6640 1 --2- 172.21.15.103:0/2175400343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed20102790 0x7fed20198910 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fed1400c890 tx=0x7fed1400cd50 comp rx=0 tx=0).stop 2024-09-19T07:01:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed24bb6640 1 -- 172.21.15.103:0/2175400343 shutdown_connections 2024-09-19T07:01:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed24bb6640 1 --2- 172.21.15.103:0/2175400343 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fed080a3430 0x7fed080a58f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed24bb6640 1 --2- 172.21.15.103:0/2175400343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed20106f10 0x7fed2019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed24bb6640 1 --2- 172.21.15.103:0/2175400343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed20102790 0x7fed20198910 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed24bb6640 1 --2- 172.21.15.103:0/2175400343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed20101de0 0x7fed201983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed24bb6640 1 -- 172.21.15.103:0/2175400343 >> 172.21.15.103:0/2175400343 conn(0x7fed200fd8c0 msgr2=0x7fed200feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed24bb6640 1 -- 172.21.15.103:0/2175400343 shutdown_connections 2024-09-19T07:01:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:41.807+0000 7fed24bb6640 1 -- 172.21.15.103:0/2175400343 wait complete. 2024-09-19T07:01:42.517 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:42 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T07:01:42.517 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:42 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:01:42.517 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:42 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T07:01:42.517 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:42 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:01:42.540 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:01:42.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:42 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T07:01:42.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:42 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:01:42.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:42 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T07:01:42.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:42 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:01:42.588 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:42 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T07:01:42.588 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:42 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:01:42.588 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:42 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T07:01:42.588 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:42 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:01:42.830 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:01:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.256+0000 7f75636aa640 1 -- 172.21.15.103:0/3265927327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f755c106f20 msgr2=0x7f755c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.256+0000 7f75636aa640 1 --2- 172.21.15.103:0/3265927327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f755c106f20 0x7f755c10eaa0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f754c009900 tx=0x7f754c02a410 comp rx=0 tx=0).stop 2024-09-19T07:01:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.256+0000 7f75636aa640 1 -- 172.21.15.103:0/3265927327 shutdown_connections 2024-09-19T07:01:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.256+0000 7f75636aa640 1 --2- 172.21.15.103:0/3265927327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f755c106f20 0x7f755c10eaa0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.256+0000 7f75636aa640 1 --2- 172.21.15.103:0/3265927327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f755c1027a0 0x7f755c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.256+0000 7f75636aa640 1 --2- 172.21.15.103:0/3265927327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f755c101df0 0x7f755c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.256+0000 7f75636aa640 1 -- 172.21.15.103:0/3265927327 >> 172.21.15.103:0/3265927327 conn(0x7f755c0fd8f0 msgr2=0x7f755c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.256+0000 7f75636aa640 1 -- 172.21.15.103:0/3265927327 shutdown_connections 2024-09-19T07:01:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.256+0000 7f75636aa640 1 -- 172.21.15.103:0/3265927327 wait complete. 2024-09-19T07:01:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.256+0000 7f75636aa640 1 Processor -- start 2024-09-19T07:01:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.256+0000 7f75636aa640 1 -- start start 2024-09-19T07:01:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f75636aa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f755c101df0 0x7f755c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f75636aa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f755c1027a0 0x7f755c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f75636aa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f755c106f20 0x7f755c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f75636aa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f755c111f80 con 0x7f755c101df0 2024-09-19T07:01:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f75636aa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f755c111e00 con 0x7f755c106f20 2024-09-19T07:01:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f75636aa640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f755c112100 con 0x7f755c1027a0 2024-09-19T07:01:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f7561446640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f755c101df0 0x7f755c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f7561446640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f755c101df0 0x7f755c198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58010/0 (socket says 172.21.15.103:58010) 2024-09-19T07:01:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f7561446640 1 -- 172.21.15.103:0/2493179880 learned_addr learned my addr 172.21.15.103:0/2493179880 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f7560c45640 1 --2- 172.21.15.103:0/2493179880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f755c1027a0 0x7f755c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f7561c47640 1 --2- 172.21.15.103:0/2493179880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f755c106f20 0x7f755c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f7561446640 1 -- 172.21.15.103:0/2493179880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f755c1027a0 msgr2=0x7f755c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f7561446640 1 --2- 172.21.15.103:0/2493179880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f755c1027a0 0x7f755c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f7561446640 1 -- 172.21.15.103:0/2493179880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f755c106f20 msgr2=0x7f755c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f7561446640 1 --2- 172.21.15.103:0/2493179880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f755c106f20 0x7f755c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f7561446640 1 -- 172.21.15.103:0/2493179880 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755c19d3f0 con 0x7f755c101df0 2024-09-19T07:01:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f7561c47640 1 --2- 172.21.15.103:0/2493179880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f755c106f20 0x7f755c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f7561446640 1 --2- 172.21.15.103:0/2493179880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f755c101df0 0x7f755c198440 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f755400d950 tx=0x7f755400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:43.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.257+0000 7f755a7fc640 1 -- 172.21.15.103:0/2493179880 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554017070 con 0x7f755c101df0 2024-09-19T07:01:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.258+0000 7f75636aa640 1 -- 172.21.15.103:0/2493179880 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755c19d6e0 con 0x7f755c101df0 2024-09-19T07:01:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.258+0000 7f755a7fc640 1 -- 172.21.15.103:0/2493179880 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7554009c30 con 0x7f755c101df0 2024-09-19T07:01:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.258+0000 7f755a7fc640 1 -- 172.21.15.103:0/2493179880 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755400f6b0 con 0x7f755c101df0 2024-09-19T07:01:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.258+0000 7f75636aa640 1 -- 172.21.15.103:0/2493179880 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f755c1a4fc0 con 0x7f755c101df0 2024-09-19T07:01:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.260+0000 7f755a7fc640 1 -- 172.21.15.103:0/2493179880 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f75540109d0 con 0x7f755c101df0 2024-09-19T07:01:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.260+0000 7f7565a83740 1 Processor -- start 2024-09-19T07:01:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.260+0000 7f7565a83740 1 -- start start 2024-09-19T07:01:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.260+0000 7f7565a83740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c8444a0 0x556c0c844880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.260+0000 7f7565a83740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c844dc0 0x556c0c84f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.260+0000 7f7565a83740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c84f680 0x556c0c851a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.260+0000 7f7565a83740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c0c7aa2e0 con 0x556c0c84f680 2024-09-19T07:01:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f7565a83740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c0c751ce0 con 0x556c0c844dc0 2024-09-19T07:01:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f7565a83740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c0c7a62a0 con 0x556c0c8444a0 2024-09-19T07:01:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f755a7fc640 1 --2- 172.21.15.103:0/2493179880 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75440a33b0 0x7f75440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f7560c45640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c84f680 0x556c0c851a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f7560c45640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c84f680 0x556c0c851a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58018/0 (socket says 172.21.15.103:58018) 2024-09-19T07:01:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f7560c45640 1 -- 172.21.15.103:0/873481609 learned_addr learned my addr 172.21.15.103:0/873481609 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f7561c47640 1 --2- 172.21.15.103:0/873481609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c8444a0 0x556c0c844880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f7561446640 1 --2- 172.21.15.103:0/873481609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c844dc0 0x556c0c84f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f7561c47640 1 --2- 172.21.15.103:0/2493179880 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75440a33b0 0x7f75440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f7560c45640 1 -- 172.21.15.103:0/873481609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c8444a0 msgr2=0x556c0c844880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:43.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f7560c45640 1 --2- 172.21.15.103:0/873481609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c8444a0 0x556c0c844880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f7560c45640 1 -- 172.21.15.103:0/873481609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c844dc0 msgr2=0x556c0c84f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:43.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f7560c45640 1 --2- 172.21.15.103:0/873481609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c844dc0 0x556c0c84f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f7560c45640 1 -- 172.21.15.103:0/873481609 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c0c736440 con 0x556c0c84f680 2024-09-19T07:01:43.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f755a7fc640 1 -- 172.21.15.103:0/2493179880 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f75540cfa00 con 0x7f755c101df0 2024-09-19T07:01:43.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f7560c45640 1 --2- 172.21.15.103:0/873481609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c84f680 0x556c0c851a70 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f7550009ed0 tx=0x7f7550010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df5f5fe05fa75ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:43.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f7561c47640 1 --2- 172.21.15.103:0/2493179880 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75440a33b0 0x7f75440a5870 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f754c0057c0 tx=0x7f754c009260 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:43.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.261+0000 7f75437fe640 1 -- 172.21.15.103:0/873481609 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550016070 con 0x556c0c84f680 2024-09-19T07:01:43.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.262+0000 7f75437fe640 1 -- 172.21.15.103:0/873481609 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7550003b70 con 0x556c0c84f680 2024-09-19T07:01:43.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.262+0000 7f75437fe640 1 -- 172.21.15.103:0/873481609 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75500127b0 con 0x556c0c84f680 2024-09-19T07:01:43.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.262+0000 7f7565a83740 1 -- 172.21.15.103:0/873481609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c84f680 msgr2=0x556c0c851a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.262+0000 7f7565a83740 1 --2- 172.21.15.103:0/873481609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c84f680 0x556c0c851a70 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f7550009ed0 tx=0x7f7550010740 comp rx=0 tx=0).stop 2024-09-19T07:01:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.263+0000 7f7565a83740 1 -- 172.21.15.103:0/873481609 shutdown_connections 2024-09-19T07:01:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.263+0000 7f7565a83740 1 --2- 172.21.15.103:0/873481609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c84f680 0x556c0c851a70 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.263+0000 7f7565a83740 1 --2- 172.21.15.103:0/873481609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c844dc0 0x556c0c84f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.263+0000 7f7565a83740 1 --2- 172.21.15.103:0/873481609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c8444a0 0x556c0c844880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.263+0000 7f7565a83740 1 -- 172.21.15.103:0/873481609 >> 172.21.15.103:0/873481609 conn(0x556c0c840a30 msgr2=0x556c0c840e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.263+0000 7f7565a83740 1 -- 172.21.15.103:0/873481609 shutdown_connections 2024-09-19T07:01:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.263+0000 7f7565a83740 1 -- 172.21.15.103:0/873481609 wait complete. 2024-09-19T07:01:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.263+0000 7f7565a83740 1 -- start start 2024-09-19T07:01:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.263+0000 7f7565a83740 1 Processor -- start 2024-09-19T07:01:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7565a83740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c8444a0 0x556c0c84ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7565a83740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c844dc0 0x556c0c84c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7565a83740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c84f680 0x556c0c84c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7565a83740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c0c856660 con 0x556c0c8444a0 2024-09-19T07:01:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7565a83740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c0c765050 con 0x556c0c844dc0 2024-09-19T07:01:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7565a83740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c0c8567e0 con 0x556c0c84f680 2024-09-19T07:01:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7561446640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c8444a0 0x556c0c84ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7565a83740 10 client.0 _ll_register_callbacks cb 0x556c0c7a5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7561446640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c8444a0 0x556c0c84ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58022/0 (socket says 172.21.15.103:58022) 2024-09-19T07:01:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7561446640 1 -- 172.21.15.103:0/2241603174 learned_addr learned my addr 172.21.15.103:0/2241603174 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7560c45640 1 --2- 172.21.15.103:0/2241603174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c844dc0 0x556c0c84c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7561c47640 1 --2- 172.21.15.103:0/2241603174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c84f680 0x556c0c84c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7561446640 1 -- 172.21.15.103:0/2241603174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c84f680 msgr2=0x556c0c84c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7561446640 1 --2- 172.21.15.103:0/2241603174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c84f680 0x556c0c84c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7561446640 1 -- 172.21.15.103:0/2241603174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c844dc0 msgr2=0x556c0c84c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7561446640 1 --2- 172.21.15.103:0/2241603174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c844dc0 0x556c0c84c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7561446640 1 -- 172.21.15.103:0/2241603174 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c0c51b3c0 con 0x556c0c8444a0 2024-09-19T07:01:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7561446640 1 --2- 172.21.15.103:0/2241603174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c8444a0 0x556c0c84ef60 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f755409bc50 tx=0x7f755406b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675d7a6036a63205 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.264+0000 7f7540ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.265+0000 7f7540ff9640 1 -- 172.21.15.103:0/2241603174 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554020050 con 0x556c0c8444a0 2024-09-19T07:01:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.265+0000 7f7540ff9640 1 -- 172.21.15.103:0/2241603174 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f7554098330 con 0x556c0c8444a0 2024-09-19T07:01:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.265+0000 7f7565a83740 1 -- 172.21.15.103:0/2241603174 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7550003b70 con 0x556c0c8444a0 2024-09-19T07:01:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.265+0000 7f7540ff9640 1 -- 172.21.15.103:0/2241603174 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7554009de0 con 0x556c0c8444a0 2024-09-19T07:01:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.265+0000 7f7565a83740 10 client.19788 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.265+0000 7f7565a83740 1 -- 172.21.15.103:0/2241603174 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556c0c862020 con 0x556c0c8444a0 2024-09-19T07:01:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.266+0000 7f7540ff9640 1 -- 172.21.15.103:0/2241603174 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7550003b70 con 0x556c0c8444a0 2024-09-19T07:01:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.266+0000 7f7540ff9640 1 -- 172.21.15.103:0/2241603174 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f755409b110 con 0x556c0c8444a0 2024-09-19T07:01:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.266+0000 7f7565a83740 10 client.19788 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.266+0000 7f7565a83740 10 client.19788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.266+0000 7f7565a83740 20 client.19788 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.266+0000 7f7565a83740 1 --2- 172.21.15.103:0/2241603174 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556c0c8634e0 0x556c0c8658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.266+0000 7f7565a83740 4 client.19788 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.266+0000 7f7565a83740 1 -- 172.21.15.103:0/2241603174 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c0c7b7850 con 0x556c0c8634e0 2024-09-19T07:01:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.266+0000 7f7560c45640 1 --2- 172.21.15.103:0/2241603174 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556c0c8634e0 0x556c0c8658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.267+0000 7f7540ff9640 10 client.19788 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.267+0000 7f7560c45640 1 --2- 172.21.15.103:0/2241603174 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556c0c8634e0 0x556c0c8658d0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b5e066186732100 server_cookie=f4058cc7e0a8d641 in_seq=0 out_seq=0 2024-09-19T07:01:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.267+0000 7f7540ff9640 1 -- 172.21.15.103:0/2241603174 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c0c7b7850 con 0x556c0c8634e0 2024-09-19T07:01:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.267+0000 7f7540ff9640 10 client.19788 handle_command_reply: tid=0 2024-09-19T07:01:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.267+0000 7f7565a83740 1 client.19788 shutdown 2024-09-19T07:01:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.267+0000 7f7565a83740 1 -- 172.21.15.103:0/2241603174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c8444a0 msgr2=0x556c0c84ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.267+0000 7f7565a83740 1 --2- 172.21.15.103:0/2241603174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c8444a0 0x556c0c84ef60 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f755409bc50 tx=0x7f755406b8e0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.267+0000 7f7565a83740 1 -- 172.21.15.103:0/2241603174 shutdown_connections 2024-09-19T07:01:43.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.267+0000 7f7565a83740 1 --2- 172.21.15.103:0/2241603174 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556c0c8634e0 0x556c0c8658d0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.267+0000 7f7565a83740 1 --2- 172.21.15.103:0/2241603174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c84f680 0x556c0c84c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.267+0000 7f7565a83740 1 --2- 172.21.15.103:0/2241603174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c844dc0 0x556c0c84c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.267+0000 7f7565a83740 1 --2- 172.21.15.103:0/2241603174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c8444a0 0x556c0c84ef60 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.267+0000 7f7565a83740 1 -- 172.21.15.103:0/2241603174 >> 172.21.15.103:0/2241603174 conn(0x556c0c840a30 msgr2=0x556c0c846710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:43.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.267+0000 7f7540ff9640 0 client.19788 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:43.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.268+0000 7f7565a83740 1 -- 172.21.15.103:0/2241603174 shutdown_connections 2024-09-19T07:01:43.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.268+0000 7f7565a83740 1 -- 172.21.15.103:0/2241603174 wait complete. 2024-09-19T07:01:43.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.268+0000 7f7565a83740 20 client.19788 trim_cache size 0 max 16384 2024-09-19T07:01:43.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7565a83740 1 Processor -- start 2024-09-19T07:01:43.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7565a83740 1 -- start start 2024-09-19T07:01:43.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7565a83740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c840a30 0x556c0c85f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7565a83740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c8444a0 0x556c0c86b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7565a83740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c844dc0 0x556c0c86b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7565a83740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f755c1ab6b0 con 0x556c0c844dc0 2024-09-19T07:01:43.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7565a83740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f755c1ad1f0 con 0x556c0c840a30 2024-09-19T07:01:43.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7565a83740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f755c1abd80 con 0x556c0c8444a0 2024-09-19T07:01:43.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7561c47640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c844dc0 0x556c0c86b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7561c47640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c844dc0 0x556c0c86b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58032/0 (socket says 172.21.15.103:58032) 2024-09-19T07:01:43.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7561c47640 1 -- 172.21.15.103:0/687913582 learned_addr learned my addr 172.21.15.103:0/687913582 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:43.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7561446640 1 --2- 172.21.15.103:0/687913582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c840a30 0x556c0c85f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7560c45640 1 --2- 172.21.15.103:0/687913582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c8444a0 0x556c0c86b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7561c47640 1 -- 172.21.15.103:0/687913582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c8444a0 msgr2=0x556c0c86b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:43.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7561c47640 1 --2- 172.21.15.103:0/687913582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c8444a0 0x556c0c86b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7561c47640 1 -- 172.21.15.103:0/687913582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c840a30 msgr2=0x556c0c85f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:43.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7561c47640 1 --2- 172.21.15.103:0/687913582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c840a30 0x556c0c85f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7561c47640 1 -- 172.21.15.103:0/687913582 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c0c865eb0 con 0x556c0c844dc0 2024-09-19T07:01:43.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.285+0000 7f7561446640 1 --2- 172.21.15.103:0/687913582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c840a30 0x556c0c85f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:43.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.286+0000 7f7561c47640 1 --2- 172.21.15.103:0/687913582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c844dc0 0x556c0c86b620 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f754c009900 tx=0x7f754c040070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7429b426eb3fc7ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:43.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.286+0000 7f7542ffd640 1 -- 172.21.15.103:0/687913582 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c035040 con 0x556c0c844dc0 2024-09-19T07:01:43.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.286+0000 7f7542ffd640 1 -- 172.21.15.103:0/687913582 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f754c0043c0 con 0x556c0c844dc0 2024-09-19T07:01:43.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.286+0000 7f7542ffd640 1 -- 172.21.15.103:0/687913582 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c04b820 con 0x556c0c844dc0 2024-09-19T07:01:43.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.286+0000 7f7565a83740 1 -- 172.21.15.103:0/687913582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c844dc0 msgr2=0x556c0c86b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:43.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.286+0000 7f7565a83740 1 --2- 172.21.15.103:0/687913582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c844dc0 0x556c0c86b620 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f754c009900 tx=0x7f754c040070 comp rx=0 tx=0).stop 2024-09-19T07:01:43.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.286+0000 7f7565a83740 1 -- 172.21.15.103:0/687913582 shutdown_connections 2024-09-19T07:01:43.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.286+0000 7f7565a83740 1 --2- 172.21.15.103:0/687913582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c844dc0 0x556c0c86b620 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.286+0000 7f7565a83740 1 --2- 172.21.15.103:0/687913582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c8444a0 0x556c0c86b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.286+0000 7f7565a83740 1 --2- 172.21.15.103:0/687913582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c840a30 0x556c0c85f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.286+0000 7f7565a83740 1 -- 172.21.15.103:0/687913582 >> 172.21.15.103:0/687913582 conn(0x556c0c84c8b0 msgr2=0x556c0c84aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:43.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.286+0000 7f7565a83740 1 -- 172.21.15.103:0/687913582 shutdown_connections 2024-09-19T07:01:43.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.286+0000 7f7565a83740 1 -- 172.21.15.103:0/687913582 wait complete. 2024-09-19T07:01:43.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.287+0000 7f7565a83740 1 -- start start 2024-09-19T07:01:43.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.287+0000 7f7565a83740 1 Processor -- start 2024-09-19T07:01:43.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.287+0000 7f7565a83740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c840a30 0x556c0c857fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.287+0000 7f7565a83740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c8444a0 0x556c0c8584f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.287+0000 7f7565a83740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c844dc0 0x556c0c8cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.287+0000 7f7565a83740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f755c1acb20 con 0x556c0c840a30 2024-09-19T07:01:43.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.287+0000 7f7565a83740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f755c1ac450 con 0x556c0c8444a0 2024-09-19T07:01:43.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.287+0000 7f7565a83740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f755c1a96b0 con 0x556c0c844dc0 2024-09-19T07:01:43.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.287+0000 7f7561446640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c840a30 0x556c0c857fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.287+0000 7f7565a83740 10 client.0 _ll_register_callbacks cb 0x7f755400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:43.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.287+0000 7f7561446640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c840a30 0x556c0c857fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58040/0 (socket says 172.21.15.103:58040) 2024-09-19T07:01:43.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7561446640 1 -- 172.21.15.103:0/506414181 learned_addr learned my addr 172.21.15.103:0/506414181 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:43.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7561c47640 1 --2- 172.21.15.103:0/506414181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c844dc0 0x556c0c8cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7560c45640 1 --2- 172.21.15.103:0/506414181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c8444a0 0x556c0c8584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7561446640 1 -- 172.21.15.103:0/506414181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c844dc0 msgr2=0x556c0c8cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:43.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7561446640 1 --2- 172.21.15.103:0/506414181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c844dc0 0x556c0c8cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7561446640 1 -- 172.21.15.103:0/506414181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c8444a0 msgr2=0x556c0c8584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:43.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7561446640 1 --2- 172.21.15.103:0/506414181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c8444a0 0x556c0c8584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7561446640 1 -- 172.21.15.103:0/506414181 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c0c8442d0 con 0x556c0c840a30 2024-09-19T07:01:43.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7561446640 1 --2- 172.21.15.103:0/506414181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c840a30 0x556c0c857fb0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f75540058c0 tx=0x7f7554010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963079c33854a29d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:43.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7540ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:43.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7540ff9640 1 -- 172.21.15.103:0/506414181 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554020070 con 0x556c0c840a30 2024-09-19T07:01:43.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7540ff9640 1 -- 172.21.15.103:0/506414181 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f7554098330 con 0x556c0c840a30 2024-09-19T07:01:43.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7565a83740 1 -- 172.21.15.103:0/506414181 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f754c0043c0 con 0x556c0c840a30 2024-09-19T07:01:43.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7540ff9640 1 -- 172.21.15.103:0/506414181 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7554010460 con 0x556c0c840a30 2024-09-19T07:01:43.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7565a83740 10 client.19794 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:43.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7565a83740 1 -- 172.21.15.103:0/506414181 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556c0c8ce7c0 con 0x556c0c840a30 2024-09-19T07:01:43.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.288+0000 7f7540ff9640 1 -- 172.21.15.103:0/506414181 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f754c0043c0 con 0x556c0c840a30 2024-09-19T07:01:43.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.289+0000 7f7540ff9640 1 -- 172.21.15.103:0/506414181 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f75540b3070 con 0x556c0c840a30 2024-09-19T07:01:43.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.289+0000 7f7565a83740 10 client.19794 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:43.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.289+0000 7f7565a83740 10 client.19794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:43.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.289+0000 7f7565a83740 20 client.19794 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:43.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.289+0000 7f7565a83740 1 --2- 172.21.15.103:0/506414181 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556c0c84f680 0x556c0c8d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:43.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.289+0000 7f7565a83740 4 client.19794 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:01:43.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.289+0000 7f7565a83740 1 -- 172.21.15.103:0/506414181 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c0c7b1d60 con 0x556c0c84f680 2024-09-19T07:01:43.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.289+0000 7f7560c45640 1 --2- 172.21.15.103:0/506414181 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556c0c84f680 0x556c0c8d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:43.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.289+0000 7f7560c45640 1 --2- 172.21.15.103:0/506414181 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556c0c84f680 0x556c0c8d0f00 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c52864eaa9548602 server_cookie=c2973f3b2516d025 in_seq=0 out_seq=0 2024-09-19T07:01:43.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.289+0000 7f7540ff9640 10 client.19794 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:43.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.289+0000 7f7540ff9640 1 -- 172.21.15.103:0/506414181 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556c0c7b1d60 con 0x556c0c84f680 2024-09-19T07:01:43.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.289+0000 7f7540ff9640 10 client.19794 handle_command_reply: tid=0 2024-09-19T07:01:43.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.289+0000 7f7565a83740 1 client.19794 shutdown 2024-09-19T07:01:43.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.290+0000 7f7565a83740 1 -- 172.21.15.103:0/506414181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c840a30 msgr2=0x556c0c857fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:43.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.290+0000 7f7565a83740 1 --2- 172.21.15.103:0/506414181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c840a30 0x556c0c857fb0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f75540058c0 tx=0x7f7554010d70 comp rx=0 tx=0).stop 2024-09-19T07:01:43.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.290+0000 7f7565a83740 1 -- 172.21.15.103:0/506414181 shutdown_connections 2024-09-19T07:01:43.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.290+0000 7f7565a83740 1 --2- 172.21.15.103:0/506414181 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556c0c84f680 0x556c0c8d0f00 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.290+0000 7f7565a83740 1 --2- 172.21.15.103:0/506414181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0c844dc0 0x556c0c8cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.290+0000 7f7565a83740 1 --2- 172.21.15.103:0/506414181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0c8444a0 0x556c0c8584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.290+0000 7f7565a83740 1 --2- 172.21.15.103:0/506414181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0c840a30 0x556c0c857fb0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.290+0000 7f7565a83740 1 -- 172.21.15.103:0/506414181 >> 172.21.15.103:0/506414181 conn(0x556c0c84c8b0 msgr2=0x556c0c83f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:43.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.290+0000 7f7540ff9640 0 client.19794 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:43.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.290+0000 7f7565a83740 1 -- 172.21.15.103:0/506414181 shutdown_connections 2024-09-19T07:01:43.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.290+0000 7f7565a83740 1 -- 172.21.15.103:0/506414181 wait complete. 2024-09-19T07:01:43.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.290+0000 7f7565a83740 20 client.19794 trim_cache size 0 max 16384 2024-09-19T07:01:43.297 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:01:43.297 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:01:43.297 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:01:43.297 INFO:teuthology.orchestra.run.smithi103.stdout: "17d61dfb-6e9c-4bff-9e3b-2a71bbdce090": { 2024-09-19T07:01:43.297 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:01:43.297 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "17d61dfb-6e9c-4bff-9e3b-2a71bbdce090", 2024-09-19T07:01:43.297 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:01:43.298 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:01:43.298 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:01:43.298 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:01:43.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.290+0000 7f75636aa640 1 -- 172.21.15.103:0/2493179880 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75440a33b0 msgr2=0x7f75440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:43.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.290+0000 7f75636aa640 1 --2- 172.21.15.103:0/2493179880 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75440a33b0 0x7f75440a5870 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f754c0057c0 tx=0x7f754c009260 comp rx=0 tx=0).stop 2024-09-19T07:01:43.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.291+0000 7f75636aa640 1 -- 172.21.15.103:0/2493179880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f755c101df0 msgr2=0x7f755c198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:43.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.291+0000 7f75636aa640 1 --2- 172.21.15.103:0/2493179880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f755c101df0 0x7f755c198440 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f755400d950 tx=0x7f755400de10 comp rx=0 tx=0).stop 2024-09-19T07:01:43.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.291+0000 7f75636aa640 1 -- 172.21.15.103:0/2493179880 shutdown_connections 2024-09-19T07:01:43.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.291+0000 7f75636aa640 1 --2- 172.21.15.103:0/2493179880 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75440a33b0 0x7f75440a5870 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.291+0000 7f75636aa640 1 --2- 172.21.15.103:0/2493179880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f755c106f20 0x7f755c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.291+0000 7f75636aa640 1 --2- 172.21.15.103:0/2493179880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f755c1027a0 0x7f755c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.291+0000 7f75636aa640 1 --2- 172.21.15.103:0/2493179880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f755c101df0 0x7f755c198440 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:43.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.291+0000 7f75636aa640 1 -- 172.21.15.103:0/2493179880 >> 172.21.15.103:0/2493179880 conn(0x7f755c0fd8f0 msgr2=0x7f755c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:43.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.291+0000 7f75636aa640 1 -- 172.21.15.103:0/2493179880 shutdown_connections 2024-09-19T07:01:43.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:01:43.291+0000 7f75636aa640 1 -- 172.21.15.103:0/2493179880 wait complete. 2024-09-19T07:01:43.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:43 smithi174 ceph-mon[35637]: pgmap v389: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-19T07:01:43.602 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:43 smithi103 ceph-mon[31158]: pgmap v389: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-19T07:01:43.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:43 smithi134 ceph-mon[35356]: pgmap v389: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-19T07:01:43.999 INFO:tasks.cephfs.filesystem:scrub status for tag:17d61dfb-6e9c-4bff-9e3b-2a71bbdce090 - {'path': '/', 'tag': '17d61dfb-6e9c-4bff-9e3b-2a71bbdce090', 'options': 'recursive,force'} 2024-09-19T07:01:45.045 INFO:tasks.workunit.client.0.smithi103.stdout:375/600: open fd = 3 2024-09-19T07:01:45.045 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.045 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.045 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.045 INFO:tasks.workunit.client.0.smithi103.stdout:376/600: open fd = 3 2024-09-19T07:01:45.045 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.045 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.045 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.045 INFO:tasks.workunit.client.0.smithi103.stdout:377/600: open fd = 3 2024-09-19T07:01:45.045 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.045 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.046 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.046 INFO:tasks.workunit.client.0.smithi103.stdout:378/600: open fd = 3 2024-09-19T07:01:45.046 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.046 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.046 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.046 INFO:tasks.workunit.client.0.smithi103.stdout:379/600: open fd = 3 2024-09-19T07:01:45.046 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.046 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.046 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.046 INFO:tasks.workunit.client.0.smithi103.stdout:380/600: open fd = 3 2024-09-19T07:01:45.046 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.046 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.046 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.046 INFO:tasks.workunit.client.0.smithi103.stdout:381/600: open fd = 3 2024-09-19T07:01:45.046 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.047 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.047 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.047 INFO:tasks.workunit.client.0.smithi103.stdout:382/600: open fd = 3 2024-09-19T07:01:45.047 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.047 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.047 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.047 INFO:tasks.workunit.client.0.smithi103.stdout:383/600: open fd = 3 2024-09-19T07:01:45.047 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.047 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.047 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.047 INFO:tasks.workunit.client.0.smithi103.stdout:384/600: open fd = 3 2024-09-19T07:01:45.047 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.047 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.047 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.047 INFO:tasks.workunit.client.0.smithi103.stdout:385/600: open fd = 3 2024-09-19T07:01:45.047 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.048 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.048 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.048 INFO:tasks.workunit.client.0.smithi103.stdout:386/600: open fd = 3 2024-09-19T07:01:45.048 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.048 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.048 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.048 INFO:tasks.workunit.client.0.smithi103.stdout:387/600: open fd = 3 2024-09-19T07:01:45.048 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.048 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.048 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.048 INFO:tasks.workunit.client.0.smithi103.stdout:388/600: open fd = 3 2024-09-19T07:01:45.048 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.048 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.048 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.048 INFO:tasks.workunit.client.0.smithi103.stdout:389/600: open fd = 3 2024-09-19T07:01:45.049 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.049 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.049 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.049 INFO:tasks.workunit.client.0.smithi103.stdout:390/600: open fd = 3 2024-09-19T07:01:45.049 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.049 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.049 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.049 INFO:tasks.workunit.client.0.smithi103.stdout:391/600: open fd = 3 2024-09-19T07:01:45.049 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.049 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.049 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.049 INFO:tasks.workunit.client.0.smithi103.stdout:392/600: open fd = 3 2024-09-19T07:01:45.049 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.049 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.049 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.049 INFO:tasks.workunit.client.0.smithi103.stdout:393/600: open fd = 3 2024-09-19T07:01:45.050 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.050 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.050 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.050 INFO:tasks.workunit.client.0.smithi103.stdout:394/600: open fd = 3 2024-09-19T07:01:45.050 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.050 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.050 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.050 INFO:tasks.workunit.client.0.smithi103.stdout:395/600: open fd = 3 2024-09-19T07:01:45.050 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.050 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.050 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.050 INFO:tasks.workunit.client.0.smithi103.stdout:396/600: open fd = 3 2024-09-19T07:01:45.050 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.051 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.051 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.051 INFO:tasks.workunit.client.0.smithi103.stdout:397/600: open fd = 3 2024-09-19T07:01:45.051 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.051 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.051 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.051 INFO:tasks.workunit.client.0.smithi103.stdout:398/600: open fd = 3 2024-09-19T07:01:45.051 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.051 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.051 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.052 INFO:tasks.workunit.client.0.smithi103.stdout:399/600: open fd = 3 2024-09-19T07:01:45.052 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.052 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.052 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.052 INFO:tasks.workunit.client.0.smithi103.stdout:400/600: open fd = 3 2024-09-19T07:01:45.052 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.052 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.052 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.052 INFO:tasks.workunit.client.0.smithi103.stdout:401/600: open fd = 3 2024-09-19T07:01:45.052 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.052 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.052 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.052 INFO:tasks.workunit.client.0.smithi103.stdout:402/600: open fd = 3 2024-09-19T07:01:45.052 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.053 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.053 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.053 INFO:tasks.workunit.client.0.smithi103.stdout:403/600: open fd = 3 2024-09-19T07:01:45.053 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.053 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.053 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.053 INFO:tasks.workunit.client.0.smithi103.stdout:404/600: open fd = 3 2024-09-19T07:01:45.053 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.053 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.053 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.053 INFO:tasks.workunit.client.0.smithi103.stdout:405/600: open fd = 3 2024-09-19T07:01:45.053 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.053 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.053 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.054 INFO:tasks.workunit.client.0.smithi103.stdout:406/600: open fd = 3 2024-09-19T07:01:45.054 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.054 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.054 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.054 INFO:tasks.workunit.client.0.smithi103.stdout:407/600: open fd = 3 2024-09-19T07:01:45.054 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.054 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.054 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.054 INFO:tasks.workunit.client.0.smithi103.stdout:408/600: open fd = 3 2024-09-19T07:01:45.054 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.054 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.054 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.054 INFO:tasks.workunit.client.0.smithi103.stdout:409/600: open fd = 3 2024-09-19T07:01:45.054 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.054 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.055 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.055 INFO:tasks.workunit.client.0.smithi103.stdout:410/600: open fd = 3 2024-09-19T07:01:45.055 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.055 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.055 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.055 INFO:tasks.workunit.client.0.smithi103.stdout:411/600: open fd = 3 2024-09-19T07:01:45.055 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.055 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.055 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.055 INFO:tasks.workunit.client.0.smithi103.stdout:412/600: open fd = 3 2024-09-19T07:01:45.055 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.055 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.055 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.055 INFO:tasks.workunit.client.0.smithi103.stdout:413/600: open fd = 3 2024-09-19T07:01:45.056 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.056 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.056 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.056 INFO:tasks.workunit.client.0.smithi103.stdout:414/600: open fd = 3 2024-09-19T07:01:45.056 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.056 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.056 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.056 INFO:tasks.workunit.client.0.smithi103.stdout:415/600: open fd = 3 2024-09-19T07:01:45.056 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.056 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.056 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.056 INFO:tasks.workunit.client.0.smithi103.stdout:416/600: open fd = 3 2024-09-19T07:01:45.056 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.056 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.057 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.057 INFO:tasks.workunit.client.0.smithi103.stdout:417/600: open fd = 3 2024-09-19T07:01:45.057 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.057 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.057 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.057 INFO:tasks.workunit.client.0.smithi103.stdout:418/600: open fd = 3 2024-09-19T07:01:45.057 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.057 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.057 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.057 INFO:tasks.workunit.client.0.smithi103.stdout:419/600: open fd = 3 2024-09-19T07:01:45.057 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.057 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.057 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.057 INFO:tasks.workunit.client.0.smithi103.stdout:420/600: open fd = 3 2024-09-19T07:01:45.058 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.058 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.058 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.058 INFO:tasks.workunit.client.0.smithi103.stdout:421/600: open fd = 3 2024-09-19T07:01:45.058 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.058 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.058 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.058 INFO:tasks.workunit.client.0.smithi103.stdout:422/600: open fd = 3 2024-09-19T07:01:45.058 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.058 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.058 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.058 INFO:tasks.workunit.client.0.smithi103.stdout:423/600: open fd = 3 2024-09-19T07:01:45.058 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.058 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.058 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.059 INFO:tasks.workunit.client.0.smithi103.stdout:424/600: open fd = 3 2024-09-19T07:01:45.059 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.059 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.059 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.059 INFO:tasks.workunit.client.0.smithi103.stdout:425/600: open fd = 3 2024-09-19T07:01:45.059 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.059 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.059 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.059 INFO:tasks.workunit.client.0.smithi103.stdout:426/600: open fd = 3 2024-09-19T07:01:45.059 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.059 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.059 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.059 INFO:tasks.workunit.client.0.smithi103.stdout:427/600: open fd = 3 2024-09-19T07:01:45.059 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.060 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.060 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.060 INFO:tasks.workunit.client.0.smithi103.stdout:428/600: open fd = 3 2024-09-19T07:01:45.060 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.060 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.060 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.060 INFO:tasks.workunit.client.0.smithi103.stdout:429/600: open fd = 3 2024-09-19T07:01:45.060 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.060 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.060 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.060 INFO:tasks.workunit.client.0.smithi103.stdout:430/600: open fd = 3 2024-09-19T07:01:45.060 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.060 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.060 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.061 INFO:tasks.workunit.client.0.smithi103.stdout:431/600: open fd = 3 2024-09-19T07:01:45.061 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.061 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.061 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.061 INFO:tasks.workunit.client.0.smithi103.stdout:432/600: open fd = 3 2024-09-19T07:01:45.061 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.061 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.061 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.061 INFO:tasks.workunit.client.0.smithi103.stdout:433/600: open fd = 3 2024-09-19T07:01:45.061 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.061 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.061 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.061 INFO:tasks.workunit.client.0.smithi103.stdout:434/600: open fd = 3 2024-09-19T07:01:45.061 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.062 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.062 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.062 INFO:tasks.workunit.client.0.smithi103.stdout:435/600: open fd = 3 2024-09-19T07:01:45.062 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.062 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.062 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.062 INFO:tasks.workunit.client.0.smithi103.stdout:436/600: open fd = 3 2024-09-19T07:01:45.062 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.062 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:01:45.062 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:01:45.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:45 smithi174 ceph-mon[35637]: pgmap v390: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-19T07:01:45.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:45 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T07:01:45.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:45 smithi103 ceph-mon[31158]: pgmap v390: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-19T07:01:45.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:45 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T07:01:45.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:45 smithi134 ceph-mon[35356]: pgmap v390: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-19T07:01:45.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:45 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T07:01:47.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:47 smithi174 ceph-mon[35637]: pgmap v391: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 23 op/s 2024-09-19T07:01:47.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:47 smithi103 ceph-mon[31158]: pgmap v391: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 23 op/s 2024-09-19T07:01:47.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:47 smithi134 ceph-mon[35356]: pgmap v391: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 23 op/s 2024-09-19T07:01:49.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:49 smithi174 ceph-mon[35637]: pgmap v392: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-09-19T07:01:49.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:49 smithi103 ceph-mon[31158]: pgmap v392: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-09-19T07:01:49.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:49 smithi134 ceph-mon[35356]: pgmap v392: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-09-19T07:01:51.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:51 smithi174 ceph-mon[35637]: pgmap v393: 97 pgs: 97 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-09-19T07:01:51.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:51 smithi103 ceph-mon[31158]: pgmap v393: 97 pgs: 97 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-09-19T07:01:51.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:51 smithi134 ceph-mon[35356]: pgmap v393: 97 pgs: 97 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-09-19T07:01:53.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:53 smithi134 ceph-mon[35356]: pgmap v394: 97 pgs: 97 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T07:01:54.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:53 smithi174 ceph-mon[35637]: pgmap v394: 97 pgs: 97 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T07:01:54.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:53 smithi103 ceph-mon[31158]: pgmap v394: 97 pgs: 97 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T07:01:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.153+0000 7f7d5a39f640 1 -- 172.21.15.103:0/4061900293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d54102270 msgr2=0x7f7d5410e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.153+0000 7f7d5a39f640 1 --2- 172.21.15.103:0/4061900293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d54102270 0x7f7d5410e7a0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009cd0 tx=0x7f7d4402a8e0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.153+0000 7f7d5a39f640 1 -- 172.21.15.103:0/4061900293 shutdown_connections 2024-09-19T07:01:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.153+0000 7f7d5a39f640 1 --2- 172.21.15.103:0/4061900293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d54102270 0x7f7d5410e7a0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.153+0000 7f7d5a39f640 1 --2- 172.21.15.103:0/4061900293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d541018d0 0x7f7d54101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.153+0000 7f7d5a39f640 1 --2- 172.21.15.103:0/4061900293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d541078e0 0x7f7d54107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.153+0000 7f7d5a39f640 1 -- 172.21.15.103:0/4061900293 >> 172.21.15.103:0/4061900293 conn(0x7f7d540fd5f0 msgr2=0x7f7d540ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.154+0000 7f7d5a39f640 1 -- 172.21.15.103:0/4061900293 shutdown_connections 2024-09-19T07:01:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.154+0000 7f7d5a39f640 1 -- 172.21.15.103:0/4061900293 wait complete. 2024-09-19T07:01:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.154+0000 7f7d5a39f640 1 Processor -- start 2024-09-19T07:01:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.154+0000 7f7d5a39f640 1 -- start start 2024-09-19T07:01:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.154+0000 7f7d5a39f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d541018d0 0x7f7d54198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d5a39f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d54102270 0x7f7d54198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d5a39f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d541078e0 0x7f7d5419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d5a39f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d54111d20 con 0x7f7d541018d0 2024-09-19T07:01:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d5a39f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d54111ba0 con 0x7f7d541078e0 2024-09-19T07:01:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d5a39f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d54111ea0 con 0x7f7d54102270 2024-09-19T07:01:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d5939d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d541018d0 0x7f7d54198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d5939d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d541018d0 0x7f7d54198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39462/0 (socket says 172.21.15.103:39462) 2024-09-19T07:01:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d5939d640 1 -- 172.21.15.103:0/750650561 learned_addr learned my addr 172.21.15.103:0/750650561 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d58b9c640 1 --2- 172.21.15.103:0/750650561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d54102270 0x7f7d54198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d59b9e640 1 --2- 172.21.15.103:0/750650561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d541078e0 0x7f7d5419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d5939d640 1 -- 172.21.15.103:0/750650561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d54102270 msgr2=0x7f7d54198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d5939d640 1 --2- 172.21.15.103:0/750650561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d54102270 0x7f7d54198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d5939d640 1 -- 172.21.15.103:0/750650561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d541078e0 msgr2=0x7f7d5419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d5939d640 1 --2- 172.21.15.103:0/750650561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d541078e0 0x7f7d5419cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d5939d640 1 -- 172.21.15.103:0/750650561 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5419d200 con 0x7f7d541018d0 2024-09-19T07:01:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d5939d640 1 --2- 172.21.15.103:0/750650561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d541018d0 0x7f7d54198210 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00d950 tx=0x7f7d4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d527fc640 1 -- 172.21.15.103:0/750650561 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c017070 con 0x7f7d541018d0 2024-09-19T07:01:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d527fc640 1 -- 172.21.15.103:0/750650561 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4c009c30 con 0x7f7d541018d0 2024-09-19T07:01:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d5a39f640 1 -- 172.21.15.103:0/750650561 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5419d4f0 con 0x7f7d541018d0 2024-09-19T07:01:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d527fc640 1 -- 172.21.15.103:0/750650561 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c00f6b0 con 0x7f7d541018d0 2024-09-19T07:01:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.155+0000 7f7d5a39f640 1 -- 172.21.15.103:0/750650561 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d541a4d80 con 0x7f7d541018d0 2024-09-19T07:01:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.158+0000 7f7d527fc640 1 -- 172.21.15.103:0/750650561 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7d4c0109d0 con 0x7f7d541018d0 2024-09-19T07:01:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.158+0000 7f7d5e470740 1 Processor -- start 2024-09-19T07:01:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.158+0000 7f7d5e470740 1 -- start start 2024-09-19T07:01:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.158+0000 7f7d5e470740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101df0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.158+0000 7f7d5e470740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101e9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.158+0000 7f7d5e470740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c101ec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.158+0000 7f7d5e470740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c101432f0 con 0x556c101df5e0 2024-09-19T07:01:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.158+0000 7f7d5e470740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c100eb270 con 0x556c101e9e80 2024-09-19T07:01:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.158+0000 7f7d5e470740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c1013f2b0 con 0x556c101decc0 2024-09-19T07:01:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.158+0000 7f7d527fc640 1 --2- 172.21.15.103:0/750650561 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d3c0a33b0 0x7f7d3c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d5939d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101e9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d5939d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101e9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39478/0 (socket says 172.21.15.103:39478) 2024-09-19T07:01:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d59b9e640 1 --2- 172.21.15.103:0/750650561 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d3c0a33b0 0x7f7d3c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d5939d640 1 -- 172.21.15.103:0/708678854 learned_addr learned my addr 172.21.15.103:0/708678854 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d58b9c640 1 --2- 172.21.15.103:0/708678854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c101ec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d59b9e640 1 --2- 172.21.15.103:0/708678854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101df0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d527fc640 1 -- 172.21.15.103:0/750650561 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f7d4c0cfa20 con 0x7f7d541018d0 2024-09-19T07:01:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d5939d640 1 -- 172.21.15.103:0/708678854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 msgr2=0x556c101df0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d5939d640 1 --2- 172.21.15.103:0/708678854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101df0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d5939d640 1 -- 172.21.15.103:0/708678854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 msgr2=0x556c101ec270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d5939d640 1 --2- 172.21.15.103:0/708678854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c101ec270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d5939d640 1 -- 172.21.15.103:0/708678854 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c100ea340 con 0x556c101df5e0 2024-09-19T07:01:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d59b9e640 1 --2- 172.21.15.103:0/750650561 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d3c0a33b0 0x7f7d3c0a5870 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009a60 tx=0x7f7d44005ef0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d5939d640 1 --2- 172.21.15.103:0/708678854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101e9940 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c098650 tx=0x7f7d4c06b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8acbb946613c3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d3b7fe640 1 -- 172.21.15.103:0/708678854 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c020070 con 0x556c101df5e0 2024-09-19T07:01:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d3b7fe640 1 -- 172.21.15.103:0/708678854 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4c066ad0 con 0x556c101df5e0 2024-09-19T07:01:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.159+0000 7f7d3b7fe640 1 -- 172.21.15.103:0/708678854 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c0a0030 con 0x556c101df5e0 2024-09-19T07:01:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.160+0000 7f7d5e470740 1 -- 172.21.15.103:0/708678854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 msgr2=0x556c101e9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.160+0000 7f7d5e470740 1 --2- 172.21.15.103:0/708678854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101e9940 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c098650 tx=0x7f7d4c06b8a0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.160+0000 7f7d5e470740 1 -- 172.21.15.103:0/708678854 shutdown_connections 2024-09-19T07:01:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.160+0000 7f7d5e470740 1 --2- 172.21.15.103:0/708678854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c101ec270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.160+0000 7f7d5e470740 1 --2- 172.21.15.103:0/708678854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101e9940 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.160+0000 7f7d5e470740 1 --2- 172.21.15.103:0/708678854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101df0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.160+0000 7f7d5e470740 1 -- 172.21.15.103:0/708678854 >> 172.21.15.103:0/708678854 conn(0x556c101db230 msgr2=0x556c101db610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.160+0000 7f7d5e470740 1 -- 172.21.15.103:0/708678854 shutdown_connections 2024-09-19T07:01:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.160+0000 7f7d5e470740 1 -- 172.21.15.103:0/708678854 wait complete. 2024-09-19T07:01:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.161+0000 7f7d5e470740 1 -- start start 2024-09-19T07:01:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.161+0000 7f7d5e470740 1 Processor -- start 2024-09-19T07:01:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.161+0000 7f7d5e470740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101e6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.161+0000 7f7d5e470740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101e72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.161+0000 7f7d5e470740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c101e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.161+0000 7f7d5e470740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c101ec9f0 con 0x556c101df5e0 2024-09-19T07:01:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.161+0000 7f7d5e470740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c100cc290 con 0x556c101e9e80 2024-09-19T07:01:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.161+0000 7f7d5e470740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c101ecb70 con 0x556c101decc0 2024-09-19T07:01:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.161+0000 7f7d5e470740 10 client.0 _ll_register_callbacks cb 0x556c100fd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.161+0000 7f7d58b9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101e72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.161+0000 7f7d58b9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101e72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39488/0 (socket says 172.21.15.103:39488) 2024-09-19T07:01:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.161+0000 7f7d58b9c640 1 -- 172.21.15.103:0/2263172008 learned_addr learned my addr 172.21.15.103:0/2263172008 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.161+0000 7f7d5939d640 1 --2- 172.21.15.103:0/2263172008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101e6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.161+0000 7f7d59b9e640 1 --2- 172.21.15.103:0/2263172008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c101e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.162+0000 7f7d58b9c640 1 -- 172.21.15.103:0/2263172008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 msgr2=0x556c101e6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.162+0000 7f7d58b9c640 1 --2- 172.21.15.103:0/2263172008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101e6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.162+0000 7f7d58b9c640 1 -- 172.21.15.103:0/2263172008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 msgr2=0x556c101e77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.162+0000 7f7d58b9c640 1 --2- 172.21.15.103:0/2263172008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c101e77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.162+0000 7f7d58b9c640 1 -- 172.21.15.103:0/2263172008 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c0fe56210 con 0x556c101df5e0 2024-09-19T07:01:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.162+0000 7f7d5939d640 1 --2- 172.21.15.103:0/2263172008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101e6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.162+0000 7f7d58b9c640 1 --2- 172.21.15.103:0/2263172008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101e72a0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800da40 tx=0x7f7d4800df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c70ccfb118df9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.162+0000 7f7d38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.162+0000 7f7d38ff9640 1 -- 172.21.15.103:0/2263172008 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48003ea0 con 0x556c101df5e0 2024-09-19T07:01:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.162+0000 7f7d38ff9640 1 -- 172.21.15.103:0/2263172008 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f7d48005220 con 0x556c101df5e0 2024-09-19T07:01:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.162+0000 7f7d5e470740 1 -- 172.21.15.103:0/2263172008 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d4c066ad0 con 0x556c101df5e0 2024-09-19T07:01:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.162+0000 7f7d38ff9640 1 -- 172.21.15.103:0/2263172008 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d48005420 con 0x556c101df5e0 2024-09-19T07:01:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.162+0000 7f7d5e470740 10 client.19806 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.162+0000 7f7d5e470740 1 -- 172.21.15.103:0/2263172008 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556c0ff22620 con 0x556c101df5e0 2024-09-19T07:01:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.162+0000 7f7d38ff9640 1 -- 172.21.15.103:0/2263172008 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4c066ad0 con 0x556c101df5e0 2024-09-19T07:01:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.162+0000 7f7d38ff9640 1 -- 172.21.15.103:0/2263172008 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7d48019030 con 0x556c101df5e0 2024-09-19T07:01:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.163+0000 7f7d5e470740 10 client.19806 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.163+0000 7f7d5e470740 10 client.19806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.163+0000 7f7d5e470740 20 client.19806 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.163+0000 7f7d5e470740 1 --2- 172.21.15.103:0/2263172008 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556c101f9df0 0x556c101fc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.163+0000 7f7d5e470740 4 client.19806 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:01:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.163+0000 7f7d5e470740 1 -- 172.21.15.103:0/2263172008 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c10150a20 con 0x556c101f9df0 2024-09-19T07:01:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.163+0000 7f7d5939d640 1 --2- 172.21.15.103:0/2263172008 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556c101f9df0 0x556c101fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.164+0000 7f7d38ff9640 10 client.19806 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.164+0000 7f7d5939d640 1 --2- 172.21.15.103:0/2263172008 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556c101f9df0 0x556c101fc220 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ced69d579c98999 server_cookie=da7c47abc09d2c52 in_seq=0 out_seq=0 2024-09-19T07:01:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.164+0000 7f7d38ff9640 1 -- 172.21.15.103:0/2263172008 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c10150a20 con 0x556c101f9df0 2024-09-19T07:01:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.164+0000 7f7d38ff9640 10 client.19806 handle_command_reply: tid=0 2024-09-19T07:01:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.164+0000 7f7d5e470740 1 client.19806 shutdown 2024-09-19T07:01:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.164+0000 7f7d5e470740 1 -- 172.21.15.103:0/2263172008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 msgr2=0x556c101e72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.164+0000 7f7d5e470740 1 --2- 172.21.15.103:0/2263172008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101e72a0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800da40 tx=0x7f7d4800df00 comp rx=0 tx=0).stop 2024-09-19T07:01:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.165+0000 7f7d5e470740 1 -- 172.21.15.103:0/2263172008 shutdown_connections 2024-09-19T07:01:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.165+0000 7f7d5e470740 1 --2- 172.21.15.103:0/2263172008 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556c101f9df0 0x556c101fc220 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.165+0000 7f7d5e470740 1 --2- 172.21.15.103:0/2263172008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c101e77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.165+0000 7f7d38ff9640 0 client.19806 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.165+0000 7f7d5e470740 1 --2- 172.21.15.103:0/2263172008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101e72a0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.165+0000 7f7d5e470740 1 --2- 172.21.15.103:0/2263172008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101e6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.165+0000 7f7d5e470740 1 -- 172.21.15.103:0/2263172008 >> 172.21.15.103:0/2263172008 conn(0x556c101db230 msgr2=0x556c101e0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.165+0000 7f7d5e470740 1 -- 172.21.15.103:0/2263172008 shutdown_connections 2024-09-19T07:01:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.165+0000 7f7d5e470740 1 -- 172.21.15.103:0/2263172008 wait complete. 2024-09-19T07:01:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.165+0000 7f7d5e470740 20 client.19806 trim_cache size 0 max 16384 2024-09-19T07:01:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.179+0000 7f7d5e470740 1 Processor -- start 2024-09-19T07:01:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.179+0000 7f7d5e470740 1 -- start start 2024-09-19T07:01:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.179+0000 7f7d5e470740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101e7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.179+0000 7f7d5e470740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101da150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.179+0000 7f7d5e470740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c10201b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.179+0000 7f7d5e470740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d541ad800 con 0x556c101df5e0 2024-09-19T07:01:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.179+0000 7f7d5e470740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d541af340 con 0x556c101e9e80 2024-09-19T07:01:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.179+0000 7f7d5e470740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d541aded0 con 0x556c101decc0 2024-09-19T07:01:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.179+0000 7f7d58b9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101da150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.179+0000 7f7d58b9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101da150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39492/0 (socket says 172.21.15.103:39492) 2024-09-19T07:01:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.179+0000 7f7d58b9c640 1 -- 172.21.15.103:0/44677006 learned_addr learned my addr 172.21.15.103:0/44677006 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.180+0000 7f7d58b9c640 1 -- 172.21.15.103:0/44677006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 msgr2=0x556c101e7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.180+0000 7f7d59b9e640 1 --2- 172.21.15.103:0/44677006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c10201b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.180+0000 7f7d5939d640 1 --2- 172.21.15.103:0/44677006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101e7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.180+0000 7f7d58b9c640 1 --2- 172.21.15.103:0/44677006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101e7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.180+0000 7f7d58b9c640 1 -- 172.21.15.103:0/44677006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 msgr2=0x556c10201b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.180+0000 7f7d58b9c640 1 --2- 172.21.15.103:0/44677006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c10201b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.180+0000 7f7d58b9c640 1 -- 172.21.15.103:0/44677006 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c101dbec0 con 0x556c101df5e0 2024-09-19T07:01:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.180+0000 7f7d59b9e640 1 --2- 172.21.15.103:0/44677006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c10201b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.180+0000 7f7d5939d640 1 --2- 172.21.15.103:0/44677006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101e7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.180+0000 7f7d58b9c640 1 --2- 172.21.15.103:0/44677006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101da150 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f7d48000c00 tx=0x7f7d48005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b16d15d21bbcb61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.180+0000 7f7d3affd640 1 -- 172.21.15.103:0/44677006 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48027070 con 0x556c101df5e0 2024-09-19T07:01:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.180+0000 7f7d3affd640 1 -- 172.21.15.103:0/44677006 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d48028040 con 0x556c101df5e0 2024-09-19T07:01:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.180+0000 7f7d3affd640 1 -- 172.21.15.103:0/44677006 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48005270 con 0x556c101df5e0 2024-09-19T07:01:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.181+0000 7f7d5e470740 1 -- 172.21.15.103:0/44677006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 msgr2=0x556c101da150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.181+0000 7f7d5e470740 1 --2- 172.21.15.103:0/44677006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101da150 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f7d48000c00 tx=0x7f7d48005db0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.181+0000 7f7d5e470740 1 -- 172.21.15.103:0/44677006 shutdown_connections 2024-09-19T07:01:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.181+0000 7f7d5e470740 1 --2- 172.21.15.103:0/44677006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c10201b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.181+0000 7f7d5e470740 1 --2- 172.21.15.103:0/44677006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101da150 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.181+0000 7f7d5e470740 1 --2- 172.21.15.103:0/44677006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101e7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.181+0000 7f7d5e470740 1 -- 172.21.15.103:0/44677006 >> 172.21.15.103:0/44677006 conn(0x556c101db230 msgr2=0x556c101d9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.181+0000 7f7d5e470740 1 -- 172.21.15.103:0/44677006 shutdown_connections 2024-09-19T07:01:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.181+0000 7f7d5e470740 1 -- 172.21.15.103:0/44677006 wait complete. 2024-09-19T07:01:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.181+0000 7f7d5e470740 1 -- start start 2024-09-19T07:01:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.181+0000 7f7d5e470740 1 Processor -- start 2024-09-19T07:01:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.181+0000 7f7d5e470740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101edad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d5e470740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101ee010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d5e470740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c101ee550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d5e470740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d541aec70 con 0x556c101df5e0 2024-09-19T07:01:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d5e470740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d541ae5a0 con 0x556c101e9e80 2024-09-19T07:01:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d5e470740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d541ab800 con 0x556c101decc0 2024-09-19T07:01:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d5e470740 10 client.0 _ll_register_callbacks cb 0x7f7d48004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:01:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d58b9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d58b9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101ee010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39494/0 (socket says 172.21.15.103:39494) 2024-09-19T07:01:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d58b9c640 1 -- 172.21.15.103:0/3726555914 learned_addr learned my addr 172.21.15.103:0/3726555914 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:01:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d5939d640 1 --2- 172.21.15.103:0/3726555914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101edad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d59b9e640 1 --2- 172.21.15.103:0/3726555914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c101ee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d58b9c640 1 -- 172.21.15.103:0/3726555914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 msgr2=0x556c101edad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d58b9c640 1 --2- 172.21.15.103:0/3726555914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101edad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d58b9c640 1 -- 172.21.15.103:0/3726555914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 msgr2=0x556c101ee550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:01:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d58b9c640 1 --2- 172.21.15.103:0/3726555914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c101ee550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d58b9c640 1 -- 172.21.15.103:0/3726555914 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c101e1b10 con 0x556c101df5e0 2024-09-19T07:01:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d5939d640 1 --2- 172.21.15.103:0/3726555914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101edad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:01:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d59b9e640 1 --2- 172.21.15.103:0/3726555914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c101ee550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:01:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d58b9c640 1 --2- 172.21.15.103:0/3726555914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101ee010 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800d9d0 tx=0x7f7d4800d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c0e9f1d97bd069 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:01:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:01:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d38ff9640 1 -- 172.21.15.103:0/3726555914 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48027030 con 0x556c101df5e0 2024-09-19T07:01:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d38ff9640 1 -- 172.21.15.103:0/3726555914 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f7d48011440 con 0x556c101df5e0 2024-09-19T07:01:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d5e470740 1 -- 172.21.15.103:0/3726555914 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d48028040 con 0x556c101df5e0 2024-09-19T07:01:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d38ff9640 1 -- 172.21.15.103:0/3726555914 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d480119e0 con 0x556c101df5e0 2024-09-19T07:01:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d5e470740 10 client.19812 fetch_fsmap learned FSMap version 27 2024-09-19T07:01:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.182+0000 7f7d5e470740 1 -- 172.21.15.103:0/3726555914 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556c10264af0 con 0x556c101df5e0 2024-09-19T07:01:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.183+0000 7f7d38ff9640 1 -- 172.21.15.103:0/3726555914 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d48028040 con 0x556c101df5e0 2024-09-19T07:01:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.183+0000 7f7d38ff9640 1 -- 172.21.15.103:0/3726555914 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7d48019070 con 0x556c101df5e0 2024-09-19T07:01:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.183+0000 7f7d5e470740 10 client.19812 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:01:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.183+0000 7f7d5e470740 10 client.19812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:01:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.183+0000 7f7d5e470740 20 client.19812 populate_metadata read hostname 'smithi103' 2024-09-19T07:01:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.183+0000 7f7d5e470740 1 --2- 172.21.15.103:0/3726555914 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556c10265180 0x556c102675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:01:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.183+0000 7f7d5e470740 4 client.19812 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:01:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.183+0000 7f7d5e470740 1 -- 172.21.15.103:0/3726555914 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c1014ad70 con 0x556c10265180 2024-09-19T07:01:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.183+0000 7f7d5939d640 1 --2- 172.21.15.103:0/3726555914 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556c10265180 0x556c102675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:01:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.183+0000 7f7d5939d640 1 --2- 172.21.15.103:0/3726555914 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556c10265180 0x556c102675a0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b3627f19418bf85 server_cookie=f8216b7f53af2203 in_seq=0 out_seq=0 2024-09-19T07:01:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.183+0000 7f7d38ff9640 10 client.19812 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.196+0000 7f7d38ff9640 1 -- 172.21.15.103:0/3726555914 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c1014ad70 con 0x556c10265180 2024-09-19T07:01:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.196+0000 7f7d38ff9640 10 client.19812 handle_command_reply: tid=0 2024-09-19T07:01:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.196+0000 7f7d5e470740 1 client.19812 shutdown 2024-09-19T07:01:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.196+0000 7f7d5e470740 1 -- 172.21.15.103:0/3726555914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 msgr2=0x556c101ee010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.196+0000 7f7d5e470740 1 --2- 172.21.15.103:0/3726555914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101ee010 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800d9d0 tx=0x7f7d4800d630 comp rx=0 tx=0).stop 2024-09-19T07:01:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.196+0000 7f7d5e470740 1 -- 172.21.15.103:0/3726555914 shutdown_connections 2024-09-19T07:01:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.196+0000 7f7d5e470740 1 --2- 172.21.15.103:0/3726555914 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556c10265180 0x556c102675a0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.196+0000 7f7d5e470740 1 --2- 172.21.15.103:0/3726555914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c101e9e80 0x556c101ee550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.196+0000 7f7d5e470740 1 --2- 172.21.15.103:0/3726555914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c101df5e0 0x556c101ee010 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.196+0000 7f7d38ff9640 0 client.19812 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:01:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.196+0000 7f7d5e470740 1 --2- 172.21.15.103:0/3726555914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c101decc0 0x556c101edad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.196+0000 7f7d5e470740 1 -- 172.21.15.103:0/3726555914 >> 172.21.15.103:0/3726555914 conn(0x556c101db230 msgr2=0x556c101da260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.196+0000 7f7d5e470740 1 -- 172.21.15.103:0/3726555914 shutdown_connections 2024-09-19T07:01:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.196+0000 7f7d5e470740 1 -- 172.21.15.103:0/3726555914 wait complete. 2024-09-19T07:01:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.196+0000 7f7d5e470740 20 client.19812 trim_cache size 0 max 16384 2024-09-19T07:01:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:01:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:01:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:01:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:01:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.197+0000 7f7d5a39f640 1 -- 172.21.15.103:0/750650561 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d3c0a33b0 msgr2=0x7f7d3c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.197+0000 7f7d5a39f640 1 --2- 172.21.15.103:0/750650561 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d3c0a33b0 0x7f7d3c0a5870 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009a60 tx=0x7f7d44005ef0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.197+0000 7f7d5a39f640 1 -- 172.21.15.103:0/750650561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d541018d0 msgr2=0x7f7d54198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:01:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.197+0000 7f7d5a39f640 1 --2- 172.21.15.103:0/750650561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d541018d0 0x7f7d54198210 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00d950 tx=0x7f7d4c00de10 comp rx=0 tx=0).stop 2024-09-19T07:01:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.197+0000 7f7d5a39f640 1 -- 172.21.15.103:0/750650561 shutdown_connections 2024-09-19T07:01:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.197+0000 7f7d5a39f640 1 --2- 172.21.15.103:0/750650561 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7d3c0a33b0 0x7f7d3c0a5870 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.197+0000 7f7d5a39f640 1 --2- 172.21.15.103:0/750650561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d541078e0 0x7f7d5419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.197+0000 7f7d5a39f640 1 --2- 172.21.15.103:0/750650561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d54102270 0x7f7d54198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.197+0000 7f7d5a39f640 1 --2- 172.21.15.103:0/750650561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d541018d0 0x7f7d54198210 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:01:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.197+0000 7f7d5a39f640 1 -- 172.21.15.103:0/750650561 >> 172.21.15.103:0/750650561 conn(0x7f7d540fd5f0 msgr2=0x7f7d540fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:01:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.197+0000 7f7d5a39f640 1 -- 172.21.15.103:0/750650561 shutdown_connections 2024-09-19T07:01:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:01:54.197+0000 7f7d5a39f640 1 -- 172.21.15.103:0/750650561 wait complete. 2024-09-19T07:01:55.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:55 smithi134 ceph-mon[35356]: pgmap v395: 97 pgs: 97 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T07:01:56.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:55 smithi174 ceph-mon[35637]: pgmap v395: 97 pgs: 97 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T07:01:56.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:55 smithi103 ceph-mon[31158]: pgmap v395: 97 pgs: 97 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-09-19T07:01:57.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:57 smithi134 ceph-mon[35356]: pgmap v396: 97 pgs: 97 active+clean; 2.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 25 op/s 2024-09-19T07:01:58.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:57 smithi174 ceph-mon[35637]: pgmap v396: 97 pgs: 97 active+clean; 2.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 25 op/s 2024-09-19T07:01:58.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:57 smithi103 ceph-mon[31158]: pgmap v396: 97 pgs: 97 active+clean; 2.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 25 op/s 2024-09-19T07:01:59.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:01:59 smithi134 ceph-mon[35356]: pgmap v397: 97 pgs: 97 active+clean; 2.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T07:02:00.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:01:59 smithi174 ceph-mon[35637]: pgmap v397: 97 pgs: 97 active+clean; 2.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T07:02:00.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:01:59 smithi103 ceph-mon[31158]: pgmap v397: 97 pgs: 97 active+clean; 2.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T07:02:02.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:01 smithi174 ceph-mon[35637]: pgmap v398: 97 pgs: 97 active+clean; 2.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-09-19T07:02:02.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:01 smithi103 ceph-mon[31158]: pgmap v398: 97 pgs: 97 active+clean; 2.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-09-19T07:02:02.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:01 smithi134 ceph-mon[35356]: pgmap v398: 97 pgs: 97 active+clean; 2.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-09-19T07:02:03.315 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T07:02:03.315 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.2 2024-09-19T07:02:03.316 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:02:03.578 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:03.957 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:03 smithi103 ceph-mon[31158]: pgmap v399: 97 pgs: 97 active+clean; 2.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T07:02:04.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.034+0000 7f2b471da640 1 -- 172.21.15.103:0/2398618218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b4010c170 msgr2=0x7f2b4010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:04.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.034+0000 7f2b471da640 1 --2- 172.21.15.103:0/2398618218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b4010c170 0x7f2b4010e610 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009ca0 tx=0x7f2b3802a950 comp rx=0 tx=0).stop 2024-09-19T07:02:04.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.034+0000 7f2b471da640 1 -- 172.21.15.103:0/2398618218 shutdown_connections 2024-09-19T07:02:04.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.034+0000 7f2b471da640 1 --2- 172.21.15.103:0/2398618218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b4010c170 0x7f2b4010e610 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:04.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.034+0000 7f2b471da640 1 --2- 172.21.15.103:0/2398618218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b40103660 0x7f2b4010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:04.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.034+0000 7f2b471da640 1 --2- 172.21.15.103:0/2398618218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b40102d40 0x7f2b40103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:04.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.034+0000 7f2b471da640 1 -- 172.21.15.103:0/2398618218 >> 172.21.15.103:0/2398618218 conn(0x7f2b400fc7a0 msgr2=0x7f2b400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:04.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.034+0000 7f2b471da640 1 -- 172.21.15.103:0/2398618218 shutdown_connections 2024-09-19T07:02:04.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.034+0000 7f2b471da640 1 -- 172.21.15.103:0/2398618218 wait complete. 2024-09-19T07:02:04.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.035+0000 7f2b471da640 1 Processor -- start 2024-09-19T07:02:04.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.035+0000 7f2b471da640 1 -- start start 2024-09-19T07:02:04.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.035+0000 7f2b471da640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b40102d40 0x7f2b40198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:04.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.035+0000 7f2b471da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b40103660 0x7f2b40198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:04.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.035+0000 7f2b471da640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b4010c170 0x7f2b4019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:04.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.035+0000 7f2b471da640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2b40111fb0 con 0x7f2b4010c170 2024-09-19T07:02:04.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.035+0000 7f2b471da640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b40111e30 con 0x7f2b40103660 2024-09-19T07:02:04.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.035+0000 7f2b471da640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b40112130 con 0x7f2b40102d40 2024-09-19T07:02:04.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.035+0000 7f2b45777640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b4010c170 0x7f2b4019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:04.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.035+0000 7f2b45777640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b4010c170 0x7f2b4019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34528/0 (socket says 172.21.15.103:34528) 2024-09-19T07:02:04.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.035+0000 7f2b45777640 1 -- 172.21.15.103:0/360670925 learned_addr learned my addr 172.21.15.103:0/360670925 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:04.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.036+0000 7f2b44f76640 1 --2- 172.21.15.103:0/360670925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b40102d40 0x7f2b40198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:04.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.036+0000 7f2b3ffff640 1 --2- 172.21.15.103:0/360670925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b40103660 0x7f2b40198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:04.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.036+0000 7f2b45777640 1 -- 172.21.15.103:0/360670925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b40102d40 msgr2=0x7f2b40198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:04.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.036+0000 7f2b45777640 1 --2- 172.21.15.103:0/360670925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b40102d40 0x7f2b40198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:04.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.036+0000 7f2b45777640 1 -- 172.21.15.103:0/360670925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b40103660 msgr2=0x7f2b40198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.036+0000 7f2b45777640 1 --2- 172.21.15.103:0/360670925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b40103660 0x7f2b40198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.036+0000 7f2b45777640 1 -- 172.21.15.103:0/360670925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4019d400 con 0x7f2b4010c170 2024-09-19T07:02:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.036+0000 7f2b45777640 1 --2- 172.21.15.103:0/360670925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b4010c170 0x7f2b4019cd20 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f2b38006010 tx=0x7f2b3802f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.036+0000 7f2b3dffb640 1 -- 172.21.15.103:0/360670925 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38034040 con 0x7f2b4010c170 2024-09-19T07:02:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.036+0000 7f2b3dffb640 1 -- 172.21.15.103:0/360670925 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b38003e00 con 0x7f2b4010c170 2024-09-19T07:02:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.036+0000 7f2b3dffb640 1 -- 172.21.15.103:0/360670925 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38045480 con 0x7f2b4010c170 2024-09-19T07:02:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.036+0000 7f2b471da640 1 -- 172.21.15.103:0/360670925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4019d690 con 0x7f2b4010c170 2024-09-19T07:02:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.036+0000 7f2b471da640 1 -- 172.21.15.103:0/360670925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4019db20 con 0x7f2b4010c170 2024-09-19T07:02:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.037+0000 7f2b471da640 1 -- 172.21.15.103:0/360670925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b40103120 con 0x7f2b4010c170 2024-09-19T07:02:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.038+0000 7f2b3dffb640 1 -- 172.21.15.103:0/360670925 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f2b38037020 con 0x7f2b4010c170 2024-09-19T07:02:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.039+0000 7f2b3dffb640 1 --2- 172.21.15.103:0/360670925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2b300a33b0 0x7f2b300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.039+0000 7f2b44f76640 1 --2- 172.21.15.103:0/360670925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2b300a33b0 0x7f2b300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:04.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.039+0000 7f2b3dffb640 1 -- 172.21.15.103:0/360670925 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f2b380f0d70 con 0x7f2b4010c170 2024-09-19T07:02:04.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.039+0000 7f2b44f76640 1 --2- 172.21.15.103:0/360670925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2b300a33b0 0x7f2b300a5870 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009990 tx=0x7f2b34009110 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:04.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.043+0000 7f2b3dffb640 1 -- 172.21.15.103:0/360670925 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2b380bd840 con 0x7f2b4010c170 2024-09-19T07:02:04.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:03 smithi174 ceph-mon[35637]: pgmap v399: 97 pgs: 97 active+clean; 2.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T07:02:04.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:03 smithi134 ceph-mon[35356]: pgmap v399: 97 pgs: 97 active+clean; 2.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T07:02:04.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.248+0000 7f2b471da640 1 -- 172.21.15.103:0/360670925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b401a5190 con 0x7f2b4010c170 2024-09-19T07:02:04.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.248+0000 7f2b3dffb640 1 -- 172.21.15.103:0/360670925 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f2b380bd140 con 0x7f2b4010c170 2024-09-19T07:02:04.250 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:02:04.251 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:02:04.251 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:02:04.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.251+0000 7f2b471da640 1 -- 172.21.15.103:0/360670925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2b300a33b0 msgr2=0x7f2b300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:04.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.251+0000 7f2b471da640 1 --2- 172.21.15.103:0/360670925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2b300a33b0 0x7f2b300a5870 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009990 tx=0x7f2b34009110 comp rx=0 tx=0).stop 2024-09-19T07:02:04.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.251+0000 7f2b471da640 1 -- 172.21.15.103:0/360670925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b4010c170 msgr2=0x7f2b4019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:04.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.251+0000 7f2b471da640 1 --2- 172.21.15.103:0/360670925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b4010c170 0x7f2b4019cd20 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f2b38006010 tx=0x7f2b3802f710 comp rx=0 tx=0).stop 2024-09-19T07:02:04.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.251+0000 7f2b471da640 1 -- 172.21.15.103:0/360670925 shutdown_connections 2024-09-19T07:02:04.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.251+0000 7f2b471da640 1 --2- 172.21.15.103:0/360670925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2b300a33b0 0x7f2b300a5870 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:04.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.251+0000 7f2b471da640 1 --2- 172.21.15.103:0/360670925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b4010c170 0x7f2b4019cd20 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:04.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.251+0000 7f2b471da640 1 --2- 172.21.15.103:0/360670925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b40103660 0x7f2b40198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:04.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.251+0000 7f2b471da640 1 --2- 172.21.15.103:0/360670925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b40102d40 0x7f2b40198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:04.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.251+0000 7f2b471da640 1 -- 172.21.15.103:0/360670925 >> 172.21.15.103:0/360670925 conn(0x7f2b400fc7a0 msgr2=0x7f2b400fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:04.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.251+0000 7f2b471da640 1 -- 172.21.15.103:0/360670925 shutdown_connections 2024-09-19T07:02:04.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:04.251+0000 7f2b471da640 1 -- 172.21.15.103:0/360670925 wait complete. 2024-09-19T07:02:05.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:04 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/360670925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:02:05.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:04 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/360670925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:02:05.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:04 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/360670925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:02:05.246 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0' 2024-09-19T07:02:05.247 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0 2024-09-19T07:02:05.496 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:05.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:05 smithi103 ceph-mon[31158]: pgmap v400: 97 pgs: 97 active+clean; 2.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T07:02:05.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.915+0000 7feb1c0d8640 1 -- 172.21.15.103:0/3685418020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb14103670 msgr2=0x7feb1410bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:05.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.915+0000 7feb1c0d8640 1 --2- 172.21.15.103:0/3685418020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb14103670 0x7feb1410bb60 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7feb04009900 tx=0x7feb0402a480 comp rx=0 tx=0).stop 2024-09-19T07:02:05.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.915+0000 7feb1c0d8640 1 -- 172.21.15.103:0/3685418020 shutdown_connections 2024-09-19T07:02:05.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.915+0000 7feb1c0d8640 1 --2- 172.21.15.103:0/3685418020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb1410c180 0x7feb1410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.916+0000 7feb1c0d8640 1 --2- 172.21.15.103:0/3685418020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb14103670 0x7feb1410bb60 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.916+0000 7feb1c0d8640 1 --2- 172.21.15.103:0/3685418020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb14102d50 0x7feb14103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.916+0000 7feb1c0d8640 1 -- 172.21.15.103:0/3685418020 >> 172.21.15.103:0/3685418020 conn(0x7feb140fc7d0 msgr2=0x7feb140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:05.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.916+0000 7feb1c0d8640 1 -- 172.21.15.103:0/3685418020 shutdown_connections 2024-09-19T07:02:05.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.916+0000 7feb1c0d8640 1 -- 172.21.15.103:0/3685418020 wait complete. 2024-09-19T07:02:05.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.916+0000 7feb1c0d8640 1 Processor -- start 2024-09-19T07:02:05.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.916+0000 7feb1c0d8640 1 -- start start 2024-09-19T07:02:05.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.916+0000 7feb1c0d8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb14102d50 0x7feb14198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.916+0000 7feb1c0d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb14103670 0x7feb14198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.916+0000 7feb1c0d8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb1410c180 0x7feb1419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.916+0000 7feb1c0d8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb14111f80 con 0x7feb14103670 2024-09-19T07:02:05.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.916+0000 7feb1c0d8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feb14111e00 con 0x7feb14102d50 2024-09-19T07:02:05.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.916+0000 7feb1c0d8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb14112100 con 0x7feb1410c180 2024-09-19T07:02:05.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.917+0000 7feb19673640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb14103670 0x7feb14198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.917+0000 7feb19673640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb14103670 0x7feb14198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34558/0 (socket says 172.21.15.103:34558) 2024-09-19T07:02:05.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.917+0000 7feb19673640 1 -- 172.21.15.103:0/196283398 learned_addr learned my addr 172.21.15.103:0/196283398 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:05.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.917+0000 7feb19e74640 1 --2- 172.21.15.103:0/196283398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb14102d50 0x7feb14198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.917+0000 7feb1a675640 1 --2- 172.21.15.103:0/196283398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb1410c180 0x7feb1419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.917+0000 7feb19673640 1 -- 172.21.15.103:0/196283398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb1410c180 msgr2=0x7feb1419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:05.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.917+0000 7feb19673640 1 --2- 172.21.15.103:0/196283398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb1410c180 0x7feb1419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.917+0000 7feb19673640 1 -- 172.21.15.103:0/196283398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb14102d50 msgr2=0x7feb14198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:05.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.917+0000 7feb19673640 1 --2- 172.21.15.103:0/196283398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb14102d50 0x7feb14198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.917+0000 7feb19673640 1 -- 172.21.15.103:0/196283398 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1419d4b0 con 0x7feb14103670 2024-09-19T07:02:05.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.917+0000 7feb19673640 1 --2- 172.21.15.103:0/196283398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb14103670 0x7feb14198a40 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7feb04005ab0 tx=0x7feb040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:05.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.917+0000 7feb0affd640 1 -- 172.21.15.103:0/196283398 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0403f030 con 0x7feb14103670 2024-09-19T07:02:05.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.917+0000 7feb1c0d8640 1 -- 172.21.15.103:0/196283398 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1419d740 con 0x7feb14103670 2024-09-19T07:02:05.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.917+0000 7feb0affd640 1 -- 172.21.15.103:0/196283398 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb04003ce0 con 0x7feb14103670 2024-09-19T07:02:05.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.917+0000 7feb0affd640 1 -- 172.21.15.103:0/196283398 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04035070 con 0x7feb14103670 2024-09-19T07:02:05.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.917+0000 7feb1c0d8640 1 -- 172.21.15.103:0/196283398 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb141a5080 con 0x7feb14103670 2024-09-19T07:02:05.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.920+0000 7feb0affd640 1 -- 172.21.15.103:0/196283398 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7feb04042030 con 0x7feb14103670 2024-09-19T07:02:05.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.920+0000 7feb1e4b1740 1 Processor -- start 2024-09-19T07:02:05.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.920+0000 7feb1e4b1740 1 -- start start 2024-09-19T07:02:05.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.920+0000 7feb0affd640 1 --2- 172.21.15.103:0/196283398 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feb000a33b0 0x7feb000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.920+0000 7feb1e4b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed3b760 0x560d7ed3bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.920+0000 7feb1e4b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed3c080 0x560d7ed46400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.920+0000 7feb1e4b1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed46940 0x560d7ed48d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.920+0000 7feb1e4b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560d7eca12a0 con 0x560d7ed3b760 2024-09-19T07:02:05.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.920+0000 7feb1e4b1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560d7ec48b10 con 0x560d7ed46940 2024-09-19T07:02:05.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.920+0000 7feb1e4b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560d7ec9d260 con 0x560d7ed3c080 2024-09-19T07:02:05.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.921+0000 7feb1a675640 1 --2- 172.21.15.103:0/196283398 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feb000a33b0 0x7feb000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.921+0000 7feb0affd640 1 -- 172.21.15.103:0/196283398 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7feb040f0520 con 0x7feb14103670 2024-09-19T07:02:05.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.921+0000 7feb19673640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed3c080 0x560d7ed46400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.921+0000 7feb19673640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed3c080 0x560d7ed46400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:57074/0 (socket says 172.21.15.103:57074) 2024-09-19T07:02:05.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.921+0000 7feb19673640 1 -- 172.21.15.103:0/1294344257 learned_addr learned my addr 172.21.15.103:0/1294344257 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:05.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.921+0000 7feb19e74640 1 --2- 172.21.15.103:0/1294344257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed3b760 0x560d7ed3bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.921+0000 7feb1a675640 1 --2- 172.21.15.103:0/1294344257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed46940 0x560d7ed48d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.921+0000 7feb19e74640 1 -- 172.21.15.103:0/1294344257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed3c080 msgr2=0x560d7ed46400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:05.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.921+0000 7feb19e74640 1 --2- 172.21.15.103:0/1294344257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed3c080 0x560d7ed46400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.921+0000 7feb19e74640 1 -- 172.21.15.103:0/1294344257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed46940 msgr2=0x560d7ed48d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:05.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.921+0000 7feb19e74640 1 --2- 172.21.15.103:0/1294344257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed46940 0x560d7ed48d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.921+0000 7feb19e74640 1 -- 172.21.15.103:0/1294344257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7ea12b90 con 0x560d7ed3b760 2024-09-19T07:02:05.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.921+0000 7feb1a675640 1 --2- 172.21.15.103:0/196283398 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feb000a33b0 0x7feb000a5870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00a780 tx=0x7feb0c013a50 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:05.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.921+0000 7feb19e74640 1 --2- 172.21.15.103:0/1294344257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed3b760 0x560d7ed3bb40 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7feb1000ab40 tx=0x7feb1000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49382262e3d3eb52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:05.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.921+0000 7feafffff640 1 -- 172.21.15.103:0/1294344257 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10004020 con 0x560d7ed3b760 2024-09-19T07:02:05.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.922+0000 7feafffff640 1 -- 172.21.15.103:0/1294344257 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb100041c0 con 0x560d7ed3b760 2024-09-19T07:02:05.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.922+0000 7feafffff640 1 -- 172.21.15.103:0/1294344257 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10018770 con 0x560d7ed3b760 2024-09-19T07:02:05.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.922+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1294344257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed3b760 msgr2=0x560d7ed3bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:05.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.922+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/1294344257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed3b760 0x560d7ed3bb40 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7feb1000ab40 tx=0x7feb1000f3c0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.922+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1294344257 shutdown_connections 2024-09-19T07:02:05.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.922+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/1294344257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed46940 0x560d7ed48d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.922+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/1294344257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed3c080 0x560d7ed46400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.922+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/1294344257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed3b760 0x560d7ed3bb40 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.922+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1294344257 >> 172.21.15.103:0/1294344257 conn(0x560d7ed37cf0 msgr2=0x560d7ed380d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:05.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.922+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1294344257 shutdown_connections 2024-09-19T07:02:05.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.922+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1294344257 wait complete. 2024-09-19T07:02:05.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.923+0000 7feb1e4b1740 1 -- start start 2024-09-19T07:02:05.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.923+0000 7feb1e4b1740 1 Processor -- start 2024-09-19T07:02:05.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.923+0000 7feb1e4b1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 0x560d7ed43780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.923+0000 7feb1e4b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed3c080 0x560d7ed43cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.923+0000 7feb1e4b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed46940 0x560d7ed44200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.923+0000 7feb1e4b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560d7ed4d920 con 0x560d7ed46940 2024-09-19T07:02:05.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.923+0000 7feb1e4b1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560d7ec296e0 con 0x560d7ed3b760 2024-09-19T07:02:05.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.923+0000 7feb1e4b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560d7ed4daa0 con 0x560d7ed3c080 2024-09-19T07:02:05.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.923+0000 7feb1e4b1740 10 client.0 _ll_register_callbacks cb 0x560d7ec4da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:05.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.923+0000 7feb1a675640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed46940 0x560d7ed44200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.923+0000 7feb1a675640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed46940 0x560d7ed44200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34576/0 (socket says 172.21.15.103:34576) 2024-09-19T07:02:05.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.923+0000 7feb1a675640 1 -- 172.21.15.103:0/1687326599 learned_addr learned my addr 172.21.15.103:0/1687326599 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:05.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feb19673640 1 --2- 172.21.15.103:0/1687326599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed3c080 0x560d7ed43cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feb19e74640 1 --2- 172.21.15.103:0/1687326599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 0x560d7ed43780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feb1a675640 1 -- 172.21.15.103:0/1687326599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed3c080 msgr2=0x560d7ed43cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:05.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feb1a675640 1 --2- 172.21.15.103:0/1687326599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed3c080 0x560d7ed43cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feb1a675640 1 -- 172.21.15.103:0/1687326599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 msgr2=0x560d7ed43780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:05.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feb1a675640 1 --2- 172.21.15.103:0/1687326599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 0x560d7ed43780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feb1a675640 1 -- 172.21.15.103:0/1687326599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7ed56d90 con 0x560d7ed46940 2024-09-19T07:02:05.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feb19e74640 1 --2- 172.21.15.103:0/1687326599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 0x560d7ed43780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:05.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feb1a675640 1 --2- 172.21.15.103:0/1687326599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed46940 0x560d7ed44200 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7feb0c011ed0 tx=0x7feb0c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f79a8779a96b95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:05.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feafd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:05.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feafd7fa640 1 -- 172.21.15.103:0/1687326599 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c004d20 con 0x560d7ed46940 2024-09-19T07:02:05.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feafd7fa640 1 -- 172.21.15.103:0/1687326599 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7feb0c005020 con 0x560d7ed46940 2024-09-19T07:02:05.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1687326599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb100041c0 con 0x560d7ed46940 2024-09-19T07:02:05.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feafd7fa640 1 -- 172.21.15.103:0/1687326599 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feb0c003970 con 0x560d7ed46940 2024-09-19T07:02:05.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feb1e4b1740 10 client.19830 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:05.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1687326599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560d7ed596f0 con 0x560d7ed46940 2024-09-19T07:02:05.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feafd7fa640 1 -- 172.21.15.103:0/1687326599 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb100041c0 con 0x560d7ed46940 2024-09-19T07:02:05.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.924+0000 7feafd7fa640 1 -- 172.21.15.103:0/1687326599 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feb0c004020 con 0x560d7ed46940 2024-09-19T07:02:05.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.925+0000 7feb1e4b1740 10 client.19830 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:05.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.925+0000 7feb1e4b1740 10 client.19830 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:02:05.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.925+0000 7feb1e4b1740 20 client.19830 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:05.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.925+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/1687326599 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560d7ed5ab00 0x560d7ed5cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.925+0000 7feb1e4b1740 4 client.19830 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:05.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.925+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1687326599 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d7ecae9d0 con 0x560d7ed5ab00 2024-09-19T07:02:05.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.925+0000 7feb19e74640 1 --2- 172.21.15.103:0/1687326599 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560d7ed5ab00 0x560d7ed5cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.925+0000 7feafd7fa640 10 client.19830 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:05.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.925+0000 7feb19e74640 1 --2- 172.21.15.103:0/1687326599 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560d7ed5ab00 0x560d7ed5cf90 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7dbdab4267b660e server_cookie=917bce2be84c7d7e in_seq=0 out_seq=0 2024-09-19T07:02:05.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.926+0000 7feafd7fa640 1 -- 172.21.15.103:0/1687326599 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d7ecae9d0 con 0x560d7ed5ab00 2024-09-19T07:02:05.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.926+0000 7feafd7fa640 10 client.19830 handle_command_reply: tid=0 2024-09-19T07:02:05.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.926+0000 7feb1e4b1740 1 client.19830 shutdown 2024-09-19T07:02:05.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.926+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1687326599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed46940 msgr2=0x560d7ed44200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:05.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.926+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/1687326599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed46940 0x560d7ed44200 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7feb0c011ed0 tx=0x7feb0c004ab0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.926+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1687326599 shutdown_connections 2024-09-19T07:02:05.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.926+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/1687326599 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560d7ed5ab00 0x560d7ed5cf90 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.926+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/1687326599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed46940 0x560d7ed44200 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.926+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/1687326599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed3c080 0x560d7ed43cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.926+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/1687326599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 0x560d7ed43780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.926+0000 7feafd7fa640 0 client.19830 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:05.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.926+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1687326599 >> 172.21.15.103:0/1687326599 conn(0x560d7ed37cf0 msgr2=0x560d7ed36e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:05.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.926+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1687326599 shutdown_connections 2024-09-19T07:02:05.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.926+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1687326599 wait complete. 2024-09-19T07:02:05.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.926+0000 7feb1e4b1740 20 client.19830 trim_cache size 0 max 16384 2024-09-19T07:02:05.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.943+0000 7feb1e4b1740 1 Processor -- start 2024-09-19T07:02:05.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.943+0000 7feb1e4b1740 1 -- start start 2024-09-19T07:02:05.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.943+0000 7feb1e4b1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 0x560d7ed43060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.943+0000 7feb1e4b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed3c080 0x560d7ed43e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.943+0000 7feb1e4b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed46940 0x560d7ed628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.943+0000 7feb1e4b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb141ab7a0 con 0x560d7ed3c080 2024-09-19T07:02:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.943+0000 7feb1e4b1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feb141ad2e0 con 0x560d7ed3b760 2024-09-19T07:02:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.943+0000 7feb1e4b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb141abe70 con 0x560d7ed46940 2024-09-19T07:02:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.944+0000 7feb19673640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed3c080 0x560d7ed43e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.944+0000 7feb19673640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed3c080 0x560d7ed43e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34580/0 (socket says 172.21.15.103:34580) 2024-09-19T07:02:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.944+0000 7feb19673640 1 -- 172.21.15.103:0/2472966584 learned_addr learned my addr 172.21.15.103:0/2472966584 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.944+0000 7feb19e74640 1 --2- 172.21.15.103:0/2472966584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 0x560d7ed43060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.944+0000 7feb1a675640 1 --2- 172.21.15.103:0/2472966584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed46940 0x560d7ed628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.944+0000 7feb19673640 1 -- 172.21.15.103:0/2472966584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed46940 msgr2=0x560d7ed628b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.944+0000 7feb19673640 1 --2- 172.21.15.103:0/2472966584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed46940 0x560d7ed628b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.944+0000 7feb19673640 1 -- 172.21.15.103:0/2472966584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 msgr2=0x560d7ed43060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.944+0000 7feb19673640 1 --2- 172.21.15.103:0/2472966584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 0x560d7ed43060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.944+0000 7feb19673640 1 -- 172.21.15.103:0/2472966584 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7ed462d0 con 0x560d7ed3c080 2024-09-19T07:02:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.944+0000 7feb1a675640 1 --2- 172.21.15.103:0/2472966584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed46940 0x560d7ed628b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.944+0000 7feb19e74640 1 --2- 172.21.15.103:0/2472966584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 0x560d7ed43060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.944+0000 7feb19673640 1 --2- 172.21.15.103:0/2472966584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed3c080 0x560d7ed43e50 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7feb04005c20 tx=0x7feb0403b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec12de55e577be1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.944+0000 7feaff7fe640 1 -- 172.21.15.103:0/2472966584 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04049040 con 0x560d7ed3c080 2024-09-19T07:02:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.944+0000 7feaff7fe640 1 -- 172.21.15.103:0/2472966584 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb0408cc40 con 0x560d7ed3c080 2024-09-19T07:02:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.944+0000 7feaff7fe640 1 -- 172.21.15.103:0/2472966584 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0403bd00 con 0x560d7ed3c080 2024-09-19T07:02:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.945+0000 7feb1e4b1740 1 -- 172.21.15.103:0/2472966584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed3c080 msgr2=0x560d7ed43e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.945+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/2472966584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed3c080 0x560d7ed43e50 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7feb04005c20 tx=0x7feb0403b720 comp rx=0 tx=0).stop 2024-09-19T07:02:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.945+0000 7feb1e4b1740 1 -- 172.21.15.103:0/2472966584 shutdown_connections 2024-09-19T07:02:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.945+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/2472966584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed46940 0x560d7ed628b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.945+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/2472966584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed3c080 0x560d7ed43e50 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.945+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/2472966584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 0x560d7ed43060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.945+0000 7feb1e4b1740 1 -- 172.21.15.103:0/2472966584 >> 172.21.15.103:0/2472966584 conn(0x560d7ed37cf0 msgr2=0x560d7ed43740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.945+0000 7feb1e4b1740 1 -- 172.21.15.103:0/2472966584 shutdown_connections 2024-09-19T07:02:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.945+0000 7feb1e4b1740 1 -- 172.21.15.103:0/2472966584 wait complete. 2024-09-19T07:02:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.945+0000 7feb1e4b1740 1 -- start start 2024-09-19T07:02:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.945+0000 7feb1e4b1740 1 Processor -- start 2024-09-19T07:02:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb1e4b1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 0x560d7ed4f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb1e4b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed3c080 0x560d7ed4f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb1e4b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed46940 0x560d7edc4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb1e4b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb141acc10 con 0x560d7ed46940 2024-09-19T07:02:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb1e4b1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feb141ac540 con 0x560d7ed3b760 2024-09-19T07:02:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb1e4b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb141a97a0 con 0x560d7ed3c080 2024-09-19T07:02:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb1e4b1740 10 client.0 _ll_register_callbacks cb 0x7feb0c011bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb1a675640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed46940 0x560d7edc4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb1a675640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed46940 0x560d7edc4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34594/0 (socket says 172.21.15.103:34594) 2024-09-19T07:02:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb1a675640 1 -- 172.21.15.103:0/1937881077 learned_addr learned my addr 172.21.15.103:0/1937881077 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb19e74640 1 --2- 172.21.15.103:0/1937881077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 0x560d7ed4f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb19673640 1 --2- 172.21.15.103:0/1937881077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed3c080 0x560d7ed4f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb1a675640 1 -- 172.21.15.103:0/1937881077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed3c080 msgr2=0x560d7ed4f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb1a675640 1 --2- 172.21.15.103:0/1937881077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed3c080 0x560d7ed4f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb1a675640 1 -- 172.21.15.103:0/1937881077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 msgr2=0x560d7ed4f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb1a675640 1 --2- 172.21.15.103:0/1937881077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 0x560d7ed4f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb1a675640 1 -- 172.21.15.103:0/1937881077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7edc4df0 con 0x560d7ed46940 2024-09-19T07:02:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.946+0000 7feb19e74640 1 --2- 172.21.15.103:0/1937881077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 0x560d7ed4f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:02:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.947+0000 7feb1a675640 1 --2- 172.21.15.103:0/1937881077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed46940 0x560d7edc4880 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7feb0c004440 tx=0x7feb0c0048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae6d44b055ebce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.947+0000 7feafd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.947+0000 7feafd7fa640 1 -- 172.21.15.103:0/1937881077 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c003d10 con 0x560d7ed46940 2024-09-19T07:02:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.947+0000 7feafd7fa640 1 -- 172.21.15.103:0/1937881077 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7feb0c003eb0 con 0x560d7ed46940 2024-09-19T07:02:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.947+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1937881077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb0408cc40 con 0x560d7ed46940 2024-09-19T07:02:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.947+0000 7feafd7fa640 1 -- 172.21.15.103:0/1937881077 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feb0c0040b0 con 0x560d7ed46940 2024-09-19T07:02:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.947+0000 7feb1e4b1740 10 client.19836 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.947+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1937881077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560d7edc6020 con 0x560d7ed46940 2024-09-19T07:02:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.947+0000 7feafd7fa640 1 -- 172.21.15.103:0/1937881077 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb0408cc40 con 0x560d7ed46940 2024-09-19T07:02:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.947+0000 7feafd7fa640 1 -- 172.21.15.103:0/1937881077 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feb0c0315a0 con 0x560d7ed46940 2024-09-19T07:02:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.947+0000 7feb1e4b1740 10 client.19836 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.947+0000 7feb1e4b1740 10 client.19836 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:02:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.947+0000 7feb1e4b1740 20 client.19836 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.947+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/1937881077 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560d7edc67e0 0x560d7edc8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.947+0000 7feb1e4b1740 4 client.19836 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}] 2024-09-19T07:02:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.947+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1937881077 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}) -- 0x560d7eca8d20 con 0x560d7edc67e0 2024-09-19T07:02:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.948+0000 7feb19e74640 1 --2- 172.21.15.103:0/1937881077 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560d7edc67e0 0x560d7edc8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.948+0000 7feb19e74640 1 --2- 172.21.15.103:0/1937881077 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560d7edc67e0 0x560d7edc8c20 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b3de05189ace9a7 server_cookie=e25bce56e923a2d3 in_seq=0 out_seq=0 2024-09-19T07:02:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.948+0000 7feafd7fa640 10 client.19836 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:05.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.958+0000 7feafd7fa640 1 -- 172.21.15.103:0/1937881077 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560d7eca8d20 con 0x560d7edc67e0 2024-09-19T07:02:05.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.958+0000 7feafd7fa640 10 client.19836 handle_command_reply: tid=0 2024-09-19T07:02:05.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.958+0000 7feb1e4b1740 1 client.19836 shutdown 2024-09-19T07:02:05.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.958+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1937881077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed46940 msgr2=0x560d7edc4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:05.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.958+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/1937881077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed46940 0x560d7edc4880 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7feb0c004440 tx=0x7feb0c0048e0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.958+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1937881077 shutdown_connections 2024-09-19T07:02:05.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.958+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/1937881077 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560d7edc67e0 0x560d7edc8c20 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.958+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/1937881077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d7ed46940 0x560d7edc4880 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.958+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/1937881077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d7ed3c080 0x560d7ed4f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.958+0000 7feafd7fa640 0 client.19836 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:05.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.958+0000 7feb1e4b1740 1 --2- 172.21.15.103:0/1937881077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d7ed3b760 0x560d7ed4f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.958+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1937881077 >> 172.21.15.103:0/1937881077 conn(0x560d7ed37cf0 msgr2=0x560d7ed43f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:05.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.958+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1937881077 shutdown_connections 2024-09-19T07:02:05.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.959+0000 7feb1e4b1740 1 -- 172.21.15.103:0/1937881077 wait complete. 2024-09-19T07:02:05.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.959+0000 7feb1e4b1740 20 client.19836 trim_cache size 0 max 16384 2024-09-19T07:02:05.962 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:02:05.962 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T07:02:05.962 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T07:02:05.962 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 42, 2024-09-19T07:02:05.962 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:02:05.963 INFO:teuthology.orchestra.run.smithi103.stdout: "93884bb1": { 2024-09-19T07:02:05.963 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 42, 2024-09-19T07:02:05.963 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:02:05.963 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:02:05.963 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:02:05.963 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:02:05.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:02:05.963 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:02:05.963 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 142.2, 2024-09-19T07:02:05.963 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:02:05.963 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:02:05.963 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:02:05.963 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:02:05.963 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:02:05.964 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:02:05.964 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:05.964 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:05.964 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:05.964 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:05.964 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:05.964 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:02:05.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.959+0000 7feb1c0d8640 1 -- 172.21.15.103:0/196283398 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feb000a33b0 msgr2=0x7feb000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:05.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.959+0000 7feb1c0d8640 1 --2- 172.21.15.103:0/196283398 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feb000a33b0 0x7feb000a5870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00a780 tx=0x7feb0c013a50 comp rx=0 tx=0).stop 2024-09-19T07:02:05.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.959+0000 7feb1c0d8640 1 -- 172.21.15.103:0/196283398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb14103670 msgr2=0x7feb14198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:05.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.959+0000 7feb1c0d8640 1 --2- 172.21.15.103:0/196283398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb14103670 0x7feb14198a40 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7feb04005ab0 tx=0x7feb040042f0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.959+0000 7feb1c0d8640 1 -- 172.21.15.103:0/196283398 shutdown_connections 2024-09-19T07:02:05.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.959+0000 7feb1c0d8640 1 --2- 172.21.15.103:0/196283398 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7feb000a33b0 0x7feb000a5870 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.959+0000 7feb1c0d8640 1 --2- 172.21.15.103:0/196283398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb1410c180 0x7feb1419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.959+0000 7feb1c0d8640 1 --2- 172.21.15.103:0/196283398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb14103670 0x7feb14198a40 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.959+0000 7feb1c0d8640 1 --2- 172.21.15.103:0/196283398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb14102d50 0x7feb14198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:05.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.959+0000 7feb1c0d8640 1 -- 172.21.15.103:0/196283398 >> 172.21.15.103:0/196283398 conn(0x7feb140fc7d0 msgr2=0x7feb140fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:05.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.959+0000 7feb1c0d8640 1 -- 172.21.15.103:0/196283398 shutdown_connections 2024-09-19T07:02:05.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:05.960+0000 7feb1c0d8640 1 -- 172.21.15.103:0/196283398 wait complete. 2024-09-19T07:02:06.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:05 smithi174 ceph-mon[35637]: pgmap v400: 97 pgs: 97 active+clean; 2.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T07:02:06.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:05 smithi134 ceph-mon[35356]: pgmap v400: 97 pgs: 97 active+clean; 2.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T07:02:06.803 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '93884bb1', quiesce took 3.5 seconds. Will release after: 18.7 2024-09-19T07:02:07.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:06 smithi174 ceph-mon[35637]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T07:02:07.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:06 smithi103 ceph-mon[31158]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T07:02:07.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:06 smithi134 ceph-mon[35356]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T07:02:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.286+0000 7f3887c82640 1 -- 172.21.15.103:0/3326309852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3880100ab0 msgr2=0x7f3880100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.286+0000 7f3887c82640 1 --2- 172.21.15.103:0/3326309852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3880100ab0 0x7f3880100e90 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f387c009900 tx=0x7f387c02a480 comp rx=0 tx=0).stop 2024-09-19T07:02:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.286+0000 7f3887c82640 1 -- 172.21.15.103:0/3326309852 shutdown_connections 2024-09-19T07:02:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.286+0000 7f3887c82640 1 --2- 172.21.15.103:0/3326309852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3880105ee0 0x7f3880106350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.286+0000 7f3887c82640 1 --2- 172.21.15.103:0/3326309852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38801013d0 0x7f3880105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.286+0000 7f3887c82640 1 --2- 172.21.15.103:0/3326309852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3880100ab0 0x7f3880100e90 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.286+0000 7f3887c82640 1 -- 172.21.15.103:0/3326309852 >> 172.21.15.103:0/3326309852 conn(0x7f38800fc6b0 msgr2=0x7f38800fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.287+0000 7f3887c82640 1 -- 172.21.15.103:0/3326309852 shutdown_connections 2024-09-19T07:02:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.287+0000 7f3887c82640 1 -- 172.21.15.103:0/3326309852 wait complete. 2024-09-19T07:02:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.287+0000 7f3887c82640 1 Processor -- start 2024-09-19T07:02:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.287+0000 7f3887c82640 1 -- start start 2024-09-19T07:02:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.287+0000 7f3887c82640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3880100ab0 0x7f38801a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.287+0000 7f3887c82640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38801013d0 0x7f38801a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.287+0000 7f3887c82640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3880105ee0 0x7f388019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.287+0000 7f3887c82640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3880118540 con 0x7f38801013d0 2024-09-19T07:02:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.287+0000 7f3887c82640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38801183c0 con 0x7f3880100ab0 2024-09-19T07:02:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.287+0000 7f3887c82640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f38801186c0 con 0x7f3880105ee0 2024-09-19T07:02:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.288+0000 7f388521d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38801013d0 0x7f38801a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.288+0000 7f388521d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38801013d0 0x7f38801a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34616/0 (socket says 172.21.15.103:34616) 2024-09-19T07:02:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.288+0000 7f388521d640 1 -- 172.21.15.103:0/3999685673 learned_addr learned my addr 172.21.15.103:0/3999685673 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.288+0000 7f3885a1e640 1 --2- 172.21.15.103:0/3999685673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3880100ab0 0x7f38801a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.288+0000 7f388621f640 1 --2- 172.21.15.103:0/3999685673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3880105ee0 0x7f388019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.288+0000 7f388521d640 1 -- 172.21.15.103:0/3999685673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3880105ee0 msgr2=0x7f388019e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.288+0000 7f388521d640 1 --2- 172.21.15.103:0/3999685673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3880105ee0 0x7f388019e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.288+0000 7f388521d640 1 -- 172.21.15.103:0/3999685673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3880100ab0 msgr2=0x7f38801a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.288+0000 7f388521d640 1 --2- 172.21.15.103:0/3999685673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3880100ab0 0x7f38801a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.288+0000 7f388521d640 1 -- 172.21.15.103:0/3999685673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388019f030 con 0x7f38801013d0 2024-09-19T07:02:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.288+0000 7f388521d640 1 --2- 172.21.15.103:0/3999685673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38801013d0 0x7f38801a4bf0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f387000c970 tx=0x7f387000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.288+0000 7f3876ffd640 1 -- 172.21.15.103:0/3999685673 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870007bb0 con 0x7f38801013d0 2024-09-19T07:02:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.288+0000 7f3876ffd640 1 -- 172.21.15.103:0/3999685673 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3870007d50 con 0x7f38801013d0 2024-09-19T07:02:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.288+0000 7f3887c82640 1 -- 172.21.15.103:0/3999685673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388019f320 con 0x7f38801013d0 2024-09-19T07:02:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.288+0000 7f3876ffd640 1 -- 172.21.15.103:0/3999685673 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870018820 con 0x7f38801013d0 2024-09-19T07:02:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.288+0000 7f3887c82640 1 -- 172.21.15.103:0/3999685673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38801ab630 con 0x7f38801013d0 2024-09-19T07:02:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.290+0000 7f3876ffd640 1 -- 172.21.15.103:0/3999685673 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f38700189c0 con 0x7f38801013d0 2024-09-19T07:02:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.291+0000 7f388aa22740 1 Processor -- start 2024-09-19T07:02:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.291+0000 7f388aa22740 1 -- start start 2024-09-19T07:02:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.291+0000 7f388aa22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e2851af0 0x5620e2851ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.291+0000 7f388aa22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e2852410 0x5620e285c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.291+0000 7f388aa22740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e285ccb0 0x5620e285f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.291+0000 7f388aa22740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5620e27b62f0 con 0x5620e285ccb0 2024-09-19T07:02:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.291+0000 7f388aa22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5620e275e270 con 0x5620e2851af0 2024-09-19T07:02:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.291+0000 7f388aa22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5620e27b22b0 con 0x5620e2852410 2024-09-19T07:02:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.291+0000 7f3876ffd640 1 --2- 172.21.15.103:0/3999685673 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f386c0a34e0 0x7f386c0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f388521d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e285ccb0 0x5620e285f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f388621f640 1 --2- 172.21.15.103:0/3999685673 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f386c0a34e0 0x7f386c0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f388521d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e285ccb0 0x5620e285f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39374/0 (socket says 172.21.15.103:39374) 2024-09-19T07:02:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f388521d640 1 -- 172.21.15.103:0/730620705 learned_addr learned my addr 172.21.15.103:0/730620705 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f388621f640 1 --2- 172.21.15.103:0/730620705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e2851af0 0x5620e2851ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f3876ffd640 1 -- 172.21.15.103:0/3999685673 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f38700cfd80 con 0x7f38801013d0 2024-09-19T07:02:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f3885a1e640 1 --2- 172.21.15.103:0/730620705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e2852410 0x5620e285c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f388521d640 1 -- 172.21.15.103:0/730620705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e2852410 msgr2=0x5620e285c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f388521d640 1 --2- 172.21.15.103:0/730620705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e2852410 0x5620e285c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f388521d640 1 -- 172.21.15.103:0/730620705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e2851af0 msgr2=0x5620e2851ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f388521d640 1 --2- 172.21.15.103:0/730620705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e2851af0 0x5620e2851ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f388521d640 1 -- 172.21.15.103:0/730620705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620e275d340 con 0x5620e285ccb0 2024-09-19T07:02:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f388621f640 1 --2- 172.21.15.103:0/730620705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e2851af0 0x5620e2851ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f388621f640 1 --2- 172.21.15.103:0/3999685673 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f386c0a34e0 0x7f386c0a59a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f387800a760 tx=0x7f3878009a10 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f388521d640 1 --2- 172.21.15.103:0/730620705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e285ccb0 0x5620e285f0a0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f38700989e0 tx=0x7f38700c6de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73a96fed23699ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f386bfff640 1 -- 172.21.15.103:0/730620705 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387009a5d0 con 0x5620e285ccb0 2024-09-19T07:02:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f386bfff640 1 -- 172.21.15.103:0/730620705 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3870066e60 con 0x5620e285ccb0 2024-09-19T07:02:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.292+0000 7f386bfff640 1 -- 172.21.15.103:0/730620705 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870018cb0 con 0x5620e285ccb0 2024-09-19T07:02:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.293+0000 7f388aa22740 1 -- 172.21.15.103:0/730620705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e285ccb0 msgr2=0x5620e285f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.293+0000 7f388aa22740 1 --2- 172.21.15.103:0/730620705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e285ccb0 0x5620e285f0a0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f38700989e0 tx=0x7f38700c6de0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.293+0000 7f388aa22740 1 -- 172.21.15.103:0/730620705 shutdown_connections 2024-09-19T07:02:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.293+0000 7f388aa22740 1 --2- 172.21.15.103:0/730620705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e285ccb0 0x5620e285f0a0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.293+0000 7f388aa22740 1 --2- 172.21.15.103:0/730620705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e2852410 0x5620e285c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.293+0000 7f388aa22740 1 --2- 172.21.15.103:0/730620705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e2851af0 0x5620e2851ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.293+0000 7f388aa22740 1 -- 172.21.15.103:0/730620705 >> 172.21.15.103:0/730620705 conn(0x5620e284e060 msgr2=0x5620e284e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.293+0000 7f388aa22740 1 -- 172.21.15.103:0/730620705 shutdown_connections 2024-09-19T07:02:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.293+0000 7f388aa22740 1 -- 172.21.15.103:0/730620705 wait complete. 2024-09-19T07:02:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.293+0000 7f388aa22740 1 -- start start 2024-09-19T07:02:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f388aa22740 1 Processor -- start 2024-09-19T07:02:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f388aa22740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e2859c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f388aa22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e2852410 0x5620e285a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f388aa22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e285ccb0 0x5620e285a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f388aa22740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5620e285f880 con 0x5620e2851af0 2024-09-19T07:02:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f388aa22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5620e273f290 con 0x5620e2852410 2024-09-19T07:02:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f388aa22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5620e285fa00 con 0x5620e285ccb0 2024-09-19T07:02:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f388aa22740 10 client.0 _ll_register_callbacks cb 0x5620e2770140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f3885a1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e2859c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f3885a1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e2859c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39390/0 (socket says 172.21.15.103:39390) 2024-09-19T07:02:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f3885a1e640 1 -- 172.21.15.103:0/1343063800 learned_addr learned my addr 172.21.15.103:0/1343063800 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f3885a1e640 1 -- 172.21.15.103:0/1343063800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e285ccb0 msgr2=0x5620e285a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f388621f640 1 --2- 172.21.15.103:0/1343063800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e285ccb0 0x5620e285a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f388521d640 1 --2- 172.21.15.103:0/1343063800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e2852410 0x5620e285a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f3885a1e640 1 --2- 172.21.15.103:0/1343063800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e285ccb0 0x5620e285a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f3885a1e640 1 -- 172.21.15.103:0/1343063800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e2852410 msgr2=0x5620e285a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f3885a1e640 1 --2- 172.21.15.103:0/1343063800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e2852410 0x5620e285a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f3885a1e640 1 -- 172.21.15.103:0/1343063800 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620e24c9210 con 0x5620e2851af0 2024-09-19T07:02:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.294+0000 7f388521d640 1 --2- 172.21.15.103:0/1343063800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e2852410 0x5620e285a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.295+0000 7f3885a1e640 1 --2- 172.21.15.103:0/1343063800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e2859c00 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f387c009900 tx=0x7f387c031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38e35570c69a51b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.295+0000 7f38697fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.295+0000 7f38697fa640 1 -- 172.21.15.103:0/1343063800 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c035040 con 0x5620e2851af0 2024-09-19T07:02:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.295+0000 7f388aa22740 1 -- 172.21.15.103:0/1343063800 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3870066e60 con 0x5620e2851af0 2024-09-19T07:02:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.295+0000 7f38697fa640 1 -- 172.21.15.103:0/1343063800 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f387c004380 con 0x5620e2851af0 2024-09-19T07:02:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.295+0000 7f38697fa640 1 -- 172.21.15.103:0/1343063800 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f387c005020 con 0x5620e2851af0 2024-09-19T07:02:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.295+0000 7f388aa22740 10 client.19848 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.295+0000 7f388aa22740 1 -- 172.21.15.103:0/1343063800 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5620e2595620 con 0x5620e2851af0 2024-09-19T07:02:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.295+0000 7f38697fa640 1 -- 172.21.15.103:0/1343063800 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3870066e60 con 0x5620e2851af0 2024-09-19T07:02:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.295+0000 7f38697fa640 1 -- 172.21.15.103:0/1343063800 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f387c0035f0 con 0x5620e2851af0 2024-09-19T07:02:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.295+0000 7f388aa22740 10 client.19848 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.295+0000 7f388aa22740 10 client.19848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.295+0000 7f388aa22740 20 client.19848 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.295+0000 7f388aa22740 1 --2- 172.21.15.103:0/1343063800 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5620e286cc70 0x5620e286f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.295+0000 7f388aa22740 4 client.19848 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.295+0000 7f388aa22740 1 -- 172.21.15.103:0/1343063800 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620e27c3a20 con 0x5620e286cc70 2024-09-19T07:02:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.296+0000 7f388521d640 1 --2- 172.21.15.103:0/1343063800 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5620e286cc70 0x5620e286f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.296+0000 7f38697fa640 10 client.19848 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.296+0000 7f388521d640 1 --2- 172.21.15.103:0/1343063800 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5620e286cc70 0x5620e286f060 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff4517efa53c935b server_cookie=3d4567acba1e4128 in_seq=0 out_seq=0 2024-09-19T07:02:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.297+0000 7f38697fa640 1 -- 172.21.15.103:0/1343063800 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620e27c3a20 con 0x5620e286cc70 2024-09-19T07:02:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.297+0000 7f38697fa640 10 client.19848 handle_command_reply: tid=0 2024-09-19T07:02:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.297+0000 7f388aa22740 1 client.19848 shutdown 2024-09-19T07:02:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.297+0000 7f388aa22740 1 -- 172.21.15.103:0/1343063800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 msgr2=0x5620e2859c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.297+0000 7f388aa22740 1 --2- 172.21.15.103:0/1343063800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e2859c00 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f387c009900 tx=0x7f387c031a60 comp rx=0 tx=0).stop 2024-09-19T07:02:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.297+0000 7f388aa22740 1 -- 172.21.15.103:0/1343063800 shutdown_connections 2024-09-19T07:02:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.297+0000 7f388aa22740 1 --2- 172.21.15.103:0/1343063800 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5620e286cc70 0x5620e286f060 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.297+0000 7f388aa22740 1 --2- 172.21.15.103:0/1343063800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e285ccb0 0x5620e285a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.297+0000 7f388aa22740 1 --2- 172.21.15.103:0/1343063800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e2852410 0x5620e285a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.297+0000 7f388aa22740 1 --2- 172.21.15.103:0/1343063800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e2859c00 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.297+0000 7f38697fa640 0 client.19848 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.297+0000 7f388aa22740 1 -- 172.21.15.103:0/1343063800 >> 172.21.15.103:0/1343063800 conn(0x5620e284e060 msgr2=0x5620e2853dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.297+0000 7f388aa22740 1 -- 172.21.15.103:0/1343063800 shutdown_connections 2024-09-19T07:02:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.297+0000 7f388aa22740 1 -- 172.21.15.103:0/1343063800 wait complete. 2024-09-19T07:02:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.297+0000 7f388aa22740 20 client.19848 trim_cache size 0 max 16384 2024-09-19T07:02:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.311+0000 7f388aa22740 1 Processor -- start 2024-09-19T07:02:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.311+0000 7f388aa22740 1 -- start start 2024-09-19T07:02:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.311+0000 7f388aa22740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e285a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.311+0000 7f388aa22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e2852410 0x5620e2874970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.311+0000 7f388aa22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e285ccb0 0x5620e2874eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.311+0000 7f388aa22740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f38801b3dd0 con 0x5620e2851af0 2024-09-19T07:02:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.311+0000 7f388aa22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38801b5910 con 0x5620e2852410 2024-09-19T07:02:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.311+0000 7f388aa22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f38801b44a0 con 0x5620e285ccb0 2024-09-19T07:02:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.311+0000 7f3885a1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e285a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.311+0000 7f3885a1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e285a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39400/0 (socket says 172.21.15.103:39400) 2024-09-19T07:02:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.311+0000 7f3885a1e640 1 -- 172.21.15.103:0/2181472145 learned_addr learned my addr 172.21.15.103:0/2181472145 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.311+0000 7f3885a1e640 1 -- 172.21.15.103:0/2181472145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e285ccb0 msgr2=0x5620e2874eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.311+0000 7f3885a1e640 1 --2- 172.21.15.103:0/2181472145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e285ccb0 0x5620e2874eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.312+0000 7f3885a1e640 1 -- 172.21.15.103:0/2181472145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e2852410 msgr2=0x5620e2874970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.312+0000 7f3885a1e640 1 --2- 172.21.15.103:0/2181472145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e2852410 0x5620e2874970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.312+0000 7f3885a1e640 1 -- 172.21.15.103:0/2181472145 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620e286f3d0 con 0x5620e2851af0 2024-09-19T07:02:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.312+0000 7f3885a1e640 1 --2- 172.21.15.103:0/2181472145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e285a4b0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f387c003990 tx=0x7f387c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87b0af4ca305ad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.312+0000 7f386b7fe640 1 -- 172.21.15.103:0/2181472145 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c035070 con 0x5620e2851af0 2024-09-19T07:02:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.312+0000 7f386b7fe640 1 -- 172.21.15.103:0/2181472145 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f387c0051c0 con 0x5620e2851af0 2024-09-19T07:02:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.312+0000 7f386b7fe640 1 -- 172.21.15.103:0/2181472145 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c004b10 con 0x5620e2851af0 2024-09-19T07:02:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.312+0000 7f388aa22740 1 -- 172.21.15.103:0/2181472145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 msgr2=0x5620e285a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.312+0000 7f388aa22740 1 --2- 172.21.15.103:0/2181472145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e285a4b0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f387c003990 tx=0x7f387c02a480 comp rx=0 tx=0).stop 2024-09-19T07:02:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.312+0000 7f388aa22740 1 -- 172.21.15.103:0/2181472145 shutdown_connections 2024-09-19T07:02:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.312+0000 7f388aa22740 1 --2- 172.21.15.103:0/2181472145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e285ccb0 0x5620e2874eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.312+0000 7f388aa22740 1 --2- 172.21.15.103:0/2181472145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e2852410 0x5620e2874970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.312+0000 7f388aa22740 1 --2- 172.21.15.103:0/2181472145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e285a4b0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.312+0000 7f388aa22740 1 -- 172.21.15.103:0/2181472145 >> 172.21.15.103:0/2181472145 conn(0x5620e284e060 msgr2=0x5620e285bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.312+0000 7f388aa22740 1 -- 172.21.15.103:0/2181472145 shutdown_connections 2024-09-19T07:02:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.313+0000 7f388aa22740 1 -- 172.21.15.103:0/2181472145 wait complete. 2024-09-19T07:02:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.313+0000 7f388aa22740 1 -- start start 2024-09-19T07:02:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.313+0000 7f388aa22740 1 Processor -- start 2024-09-19T07:02:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.313+0000 7f388aa22740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e2865420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.313+0000 7f388aa22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e2852410 0x5620e28607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.313+0000 7f388aa22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e285ccb0 0x5620e2860d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.313+0000 7f388aa22740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f38801b5240 con 0x5620e2851af0 2024-09-19T07:02:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.313+0000 7f388aa22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38801b4b70 con 0x5620e285ccb0 2024-09-19T07:02:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.313+0000 7f388aa22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f38801b1dd0 con 0x5620e2852410 2024-09-19T07:02:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.313+0000 7f388aa22740 10 client.0 _ll_register_callbacks cb 0x7f387c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.313+0000 7f3885a1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e2865420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.313+0000 7f3885a1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e2865420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39404/0 (socket says 172.21.15.103:39404) 2024-09-19T07:02:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.313+0000 7f3885a1e640 1 -- 172.21.15.103:0/3062659632 learned_addr learned my addr 172.21.15.103:0/3062659632 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f388521d640 1 --2- 172.21.15.103:0/3062659632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e2852410 0x5620e28607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f388621f640 1 --2- 172.21.15.103:0/3062659632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e285ccb0 0x5620e2860d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f3885a1e640 1 -- 172.21.15.103:0/3062659632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e2852410 msgr2=0x5620e28607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f3885a1e640 1 --2- 172.21.15.103:0/3062659632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e2852410 0x5620e28607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f3885a1e640 1 -- 172.21.15.103:0/3062659632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e285ccb0 msgr2=0x5620e2860d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f3885a1e640 1 --2- 172.21.15.103:0/3062659632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e285ccb0 0x5620e2860d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f3885a1e640 1 -- 172.21.15.103:0/3062659632 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620e2861520 con 0x5620e2851af0 2024-09-19T07:02:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f3885a1e640 1 --2- 172.21.15.103:0/3062659632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e2865420 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f387c009900 tx=0x7f387c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c11d94fbf613a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f38697fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f38697fa640 1 -- 172.21.15.103:0/3062659632 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c035070 con 0x5620e2851af0 2024-09-19T07:02:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f38697fa640 1 -- 172.21.15.103:0/3062659632 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8600+0+0 (secure 0 0 0) 0x7f387c004380 con 0x5620e2851af0 2024-09-19T07:02:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f388aa22740 1 -- 172.21.15.103:0/3062659632 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f387c0051c0 con 0x5620e2851af0 2024-09-19T07:02:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f38697fa640 1 -- 172.21.15.103:0/3062659632 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f387c068050 con 0x5620e2851af0 2024-09-19T07:02:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f388aa22740 10 client.19854 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f388aa22740 1 -- 172.21.15.103:0/3062659632 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5620e28d7980 con 0x5620e2851af0 2024-09-19T07:02:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f38697fa640 1 -- 172.21.15.103:0/3062659632 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f387c0051c0 con 0x5620e2851af0 2024-09-19T07:02:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f38697fa640 1 -- 172.21.15.103:0/3062659632 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f387c046750 con 0x5620e2851af0 2024-09-19T07:02:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f388aa22740 10 client.19854 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f388aa22740 10 client.19854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f388aa22740 20 client.19854 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.314+0000 7f388aa22740 1 --2- 172.21.15.103:0/3062659632 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5620e28d7f30 0x5620e28da320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.315+0000 7f388aa22740 4 client.19854 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:02:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.315+0000 7f388aa22740 1 -- 172.21.15.103:0/3062659632 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620e27bdd70 con 0x5620e28d7f30 2024-09-19T07:02:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.315+0000 7f388521d640 1 --2- 172.21.15.103:0/3062659632 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5620e28d7f30 0x5620e28da320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.315+0000 7f388521d640 1 --2- 172.21.15.103:0/3062659632 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5620e28d7f30 0x5620e28da320 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9934044ce73f0df3 server_cookie=ab4eb3d4cac875d5 in_seq=0 out_seq=0 2024-09-19T07:02:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.315+0000 7f38697fa640 10 client.19854 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f38697fa640 1 -- 172.21.15.103:0/3062659632 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5620e27bdd70 con 0x5620e28d7f30 2024-09-19T07:02:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f38697fa640 10 client.19854 handle_command_reply: tid=0 2024-09-19T07:02:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f388aa22740 1 client.19854 shutdown 2024-09-19T07:02:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f388aa22740 1 -- 172.21.15.103:0/3062659632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 msgr2=0x5620e2865420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f388aa22740 1 --2- 172.21.15.103:0/3062659632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e2865420 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f387c009900 tx=0x7f387c0035f0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f388aa22740 1 -- 172.21.15.103:0/3062659632 shutdown_connections 2024-09-19T07:02:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f388aa22740 1 --2- 172.21.15.103:0/3062659632 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5620e28d7f30 0x5620e28da320 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f388aa22740 1 --2- 172.21.15.103:0/3062659632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620e285ccb0 0x5620e2860d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f388aa22740 1 --2- 172.21.15.103:0/3062659632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5620e2852410 0x5620e28607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f38697fa640 0 client.19854 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f388aa22740 1 --2- 172.21.15.103:0/3062659632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620e2851af0 0x5620e2865420 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f388aa22740 1 -- 172.21.15.103:0/3062659632 >> 172.21.15.103:0/3062659632 conn(0x5620e284e060 msgr2=0x5620e284c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f388aa22740 1 -- 172.21.15.103:0/3062659632 shutdown_connections 2024-09-19T07:02:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f388aa22740 1 -- 172.21.15.103:0/3062659632 wait complete. 2024-09-19T07:02:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f388aa22740 20 client.19854 trim_cache size 0 max 16384 2024-09-19T07:02:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:02:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:02:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:02:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:02:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f3887c82640 1 -- 172.21.15.103:0/3999685673 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f386c0a34e0 msgr2=0x7f386c0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f3887c82640 1 --2- 172.21.15.103:0/3999685673 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f386c0a34e0 0x7f386c0a59a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f387800a760 tx=0x7f3878009a10 comp rx=0 tx=0).stop 2024-09-19T07:02:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f3887c82640 1 -- 172.21.15.103:0/3999685673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38801013d0 msgr2=0x7f38801a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f3887c82640 1 --2- 172.21.15.103:0/3999685673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38801013d0 0x7f38801a4bf0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f387000c970 tx=0x7f387000ce30 comp rx=0 tx=0).stop 2024-09-19T07:02:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f3887c82640 1 -- 172.21.15.103:0/3999685673 shutdown_connections 2024-09-19T07:02:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f3887c82640 1 --2- 172.21.15.103:0/3999685673 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f386c0a34e0 0x7f386c0a59a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f3887c82640 1 --2- 172.21.15.103:0/3999685673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3880105ee0 0x7f388019e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f3887c82640 1 --2- 172.21.15.103:0/3999685673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38801013d0 0x7f38801a4bf0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f3887c82640 1 --2- 172.21.15.103:0/3999685673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3880100ab0 0x7f38801a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f3887c82640 1 -- 172.21.15.103:0/3999685673 >> 172.21.15.103:0/3999685673 conn(0x7f38800fc6b0 msgr2=0x7f3880103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f3887c82640 1 -- 172.21.15.103:0/3999685673 shutdown_connections 2024-09-19T07:02:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:07.329+0000 7f3887c82640 1 -- 172.21.15.103:0/3999685673 wait complete. 2024-09-19T07:02:08.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:07 smithi174 ceph-mon[35637]: pgmap v402: 97 pgs: 97 active+clean; 2.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-09-19T07:02:08.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:07 smithi103 ceph-mon[31158]: pgmap v402: 97 pgs: 97 active+clean; 2.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-09-19T07:02:08.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:07 smithi134 ceph-mon[35356]: pgmap v402: 97 pgs: 97 active+clean; 2.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-09-19T07:02:10.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:09 smithi174 ceph-mon[35637]: pgmap v403: 97 pgs: 97 active+clean; 2.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-09-19T07:02:10.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:09 smithi103 ceph-mon[31158]: pgmap v403: 97 pgs: 97 active+clean; 2.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-09-19T07:02:10.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:09 smithi134 ceph-mon[35356]: pgmap v403: 97 pgs: 97 active+clean; 2.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-09-19T07:02:12.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:11 smithi174 ceph-mon[35637]: pgmap v404: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 89 KiB/s wr, 37 op/s 2024-09-19T07:02:12.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:11 smithi174 ceph-mon[35637]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T07:02:12.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:11 smithi103 ceph-mon[31158]: pgmap v404: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 89 KiB/s wr, 37 op/s 2024-09-19T07:02:12.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:11 smithi103 ceph-mon[31158]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T07:02:12.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:11 smithi134 ceph-mon[35356]: pgmap v404: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 89 KiB/s wr, 37 op/s 2024-09-19T07:02:12.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:11 smithi134 ceph-mon[35356]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T07:02:14.000 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:02:14.027 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:13 smithi103 ceph-mon[31158]: pgmap v406: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 111 KiB/s wr, 46 op/s 2024-09-19T07:02:14.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:13 smithi174 ceph-mon[35637]: pgmap v406: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 111 KiB/s wr, 46 op/s 2024-09-19T07:02:14.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:13 smithi134 ceph-mon[35356]: pgmap v406: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 111 KiB/s wr, 46 op/s 2024-09-19T07:02:14.263 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:14.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.751+0000 7ff864a2d640 1 -- 172.21.15.103:0/3052089174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff860100e20 msgr2=0x7ff860101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:14.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.751+0000 7ff864a2d640 1 --2- 172.21.15.103:0/3052089174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff860100e20 0x7ff860101200 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7ff850009900 tx=0x7ff85002a480 comp rx=0 tx=0).stop 2024-09-19T07:02:14.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.751+0000 7ff864a2d640 1 -- 172.21.15.103:0/3052089174 shutdown_connections 2024-09-19T07:02:14.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.751+0000 7ff864a2d640 1 --2- 172.21.15.103:0/3052089174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff860102ef0 0x7ff86010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.751+0000 7ff864a2d640 1 --2- 172.21.15.103:0/3052089174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff860102570 0x7ff8601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.751+0000 7ff864a2d640 1 --2- 172.21.15.103:0/3052089174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff860100e20 0x7ff860101200 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.751+0000 7ff864a2d640 1 -- 172.21.15.103:0/3052089174 >> 172.21.15.103:0/3052089174 conn(0x7ff8600fc7d0 msgr2=0x7ff8600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:14.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.751+0000 7ff864a2d640 1 -- 172.21.15.103:0/3052089174 shutdown_connections 2024-09-19T07:02:14.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.751+0000 7ff864a2d640 1 -- 172.21.15.103:0/3052089174 wait complete. 2024-09-19T07:02:14.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.752+0000 7ff864a2d640 1 Processor -- start 2024-09-19T07:02:14.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.752+0000 7ff864a2d640 1 -- start start 2024-09-19T07:02:14.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.752+0000 7ff864a2d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff860100e20 0x7ff8601970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.752+0000 7ff864a2d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff860102570 0x7ff860197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.752+0000 7ff864a2d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff860102ef0 0x7ff860197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.752+0000 7ff864a2d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff860110f80 con 0x7ff860100e20 2024-09-19T07:02:14.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.752+0000 7ff864a2d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff860110e00 con 0x7ff860102ef0 2024-09-19T07:02:14.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.752+0000 7ff864a2d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff860111100 con 0x7ff860102570 2024-09-19T07:02:14.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.753+0000 7ff85e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff860100e20 0x7ff8601970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.753+0000 7ff85e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff860100e20 0x7ff8601970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39424/0 (socket says 172.21.15.103:39424) 2024-09-19T07:02:14.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.753+0000 7ff85e59c640 1 -- 172.21.15.103:0/2341839752 learned_addr learned my addr 172.21.15.103:0/2341839752 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:14.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.753+0000 7ff85dd9b640 1 --2- 172.21.15.103:0/2341839752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff860102570 0x7ff860197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.753+0000 7ff85ed9d640 1 --2- 172.21.15.103:0/2341839752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff860102ef0 0x7ff860197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.753+0000 7ff85e59c640 1 -- 172.21.15.103:0/2341839752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff860102570 msgr2=0x7ff860197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:14.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.753+0000 7ff85e59c640 1 --2- 172.21.15.103:0/2341839752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff860102570 0x7ff860197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.753+0000 7ff85e59c640 1 -- 172.21.15.103:0/2341839752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff860102ef0 msgr2=0x7ff860197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:14.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.753+0000 7ff85e59c640 1 --2- 172.21.15.103:0/2341839752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff860102ef0 0x7ff860197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.753+0000 7ff85e59c640 1 -- 172.21.15.103:0/2341839752 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8601a6950 con 0x7ff860100e20 2024-09-19T07:02:14.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.753+0000 7ff85e59c640 1 --2- 172.21.15.103:0/2341839752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff860100e20 0x7ff8601970e0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7ff850006010 tx=0x7ff850003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:14.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.753+0000 7ff84f7fe640 1 -- 172.21.15.103:0/2341839752 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85003f070 con 0x7ff860100e20 2024-09-19T07:02:14.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.753+0000 7ff84f7fe640 1 -- 172.21.15.103:0/2341839752 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff850031c30 con 0x7ff860100e20 2024-09-19T07:02:14.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.753+0000 7ff864a2d640 1 -- 172.21.15.103:0/2341839752 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8601a6be0 con 0x7ff860100e20 2024-09-19T07:02:14.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.753+0000 7ff84f7fe640 1 -- 172.21.15.103:0/2341839752 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850035030 con 0x7ff860100e20 2024-09-19T07:02:14.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.753+0000 7ff864a2d640 1 -- 172.21.15.103:0/2341839752 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8601a70c0 con 0x7ff860100e20 2024-09-19T07:02:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.755+0000 7ff84f7fe640 1 -- 172.21.15.103:0/2341839752 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff850042070 con 0x7ff860100e20 2024-09-19T07:02:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.756+0000 7ff866e06740 1 Processor -- start 2024-09-19T07:02:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.756+0000 7ff866e06740 1 -- start start 2024-09-19T07:02:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.756+0000 7ff84f7fe640 1 --2- 172.21.15.103:0/2341839752 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff8480a33b0 0x7ff8480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.756+0000 7ff866e06740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e4564a0 0x56355e456880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.756+0000 7ff866e06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e456dc0 0x56355e461140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.756+0000 7ff866e06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 0x56355e463a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.756+0000 7ff866e06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56355e3bc2e0 con 0x56355e461680 2024-09-19T07:02:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.756+0000 7ff866e06740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56355e363ce0 con 0x56355e4564a0 2024-09-19T07:02:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.756+0000 7ff866e06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56355e3b82a0 con 0x56355e456dc0 2024-09-19T07:02:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.756+0000 7ff85ed9d640 1 --2- 172.21.15.103:0/2341839752 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff8480a33b0 0x7ff8480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff84f7fe640 1 -- 172.21.15.103:0/2341839752 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7ff8500f04f0 con 0x7ff860100e20 2024-09-19T07:02:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff85ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 0x56355e463a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff85dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e456dc0 0x56355e461140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff85ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 0x56355e463a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39426/0 (socket says 172.21.15.103:39426) 2024-09-19T07:02:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff85e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e4564a0 0x56355e456880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff85ed9d640 1 -- 172.21.15.103:0/1155268002 learned_addr learned my addr 172.21.15.103:0/1155268002 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:14.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff85e59c640 1 -- 172.21.15.103:0/1155268002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e456dc0 msgr2=0x56355e461140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:14.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff85e59c640 1 --2- 172.21.15.103:0/1155268002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e456dc0 0x56355e461140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff85e59c640 1 -- 172.21.15.103:0/1155268002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 msgr2=0x56355e463a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:14.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff85e59c640 1 --2- 172.21.15.103:0/1155268002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 0x56355e463a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff85e59c640 1 -- 172.21.15.103:0/1155268002 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56355e348440 con 0x56355e4564a0 2024-09-19T07:02:14.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff85ed9d640 1 --2- 172.21.15.103:0/1155268002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 0x56355e463a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:02:14.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff85ed9d640 1 --2- 172.21.15.103:0/2341839752 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff8480a33b0 0x7ff8480a5870 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff858014c80 tx=0x7ff858007e80 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:14.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff85e59c640 1 --2- 172.21.15.103:0/1155268002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e4564a0 0x56355e456880 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff8500edc20 tx=0x7ff8500b8500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4314c329041483e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:14.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff84cff9640 1 -- 172.21.15.103:0/1155268002 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850049070 con 0x56355e4564a0 2024-09-19T07:02:14.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff84cff9640 1 -- 172.21.15.103:0/1155268002 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff850031de0 con 0x56355e4564a0 2024-09-19T07:02:14.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.757+0000 7ff84cff9640 1 -- 172.21.15.103:0/1155268002 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8500b8ed0 con 0x56355e4564a0 2024-09-19T07:02:14.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.758+0000 7ff866e06740 1 -- 172.21.15.103:0/1155268002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e4564a0 msgr2=0x56355e456880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:14.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.758+0000 7ff866e06740 1 --2- 172.21.15.103:0/1155268002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e4564a0 0x56355e456880 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff8500edc20 tx=0x7ff8500b8500 comp rx=0 tx=0).stop 2024-09-19T07:02:14.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.758+0000 7ff866e06740 1 -- 172.21.15.103:0/1155268002 shutdown_connections 2024-09-19T07:02:14.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.758+0000 7ff866e06740 1 --2- 172.21.15.103:0/1155268002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 0x56355e463a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.758+0000 7ff866e06740 1 --2- 172.21.15.103:0/1155268002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e456dc0 0x56355e461140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.758+0000 7ff866e06740 1 --2- 172.21.15.103:0/1155268002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e4564a0 0x56355e456880 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.758+0000 7ff866e06740 1 -- 172.21.15.103:0/1155268002 >> 172.21.15.103:0/1155268002 conn(0x56355e452a30 msgr2=0x56355e452e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:14.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.758+0000 7ff866e06740 1 -- 172.21.15.103:0/1155268002 shutdown_connections 2024-09-19T07:02:14.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.758+0000 7ff866e06740 1 -- 172.21.15.103:0/1155268002 wait complete. 2024-09-19T07:02:14.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.759+0000 7ff866e06740 1 -- start start 2024-09-19T07:02:14.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.759+0000 7ff866e06740 1 Processor -- start 2024-09-19T07:02:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.759+0000 7ff866e06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e4564a0 0x56355e460f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.759+0000 7ff866e06740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e456dc0 0x56355e45e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.759+0000 7ff866e06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 0x56355e45e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.759+0000 7ff866e06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56355e464010 con 0x56355e461680 2024-09-19T07:02:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.759+0000 7ff866e06740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56355e377050 con 0x56355e456dc0 2024-09-19T07:02:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.759+0000 7ff866e06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56355e464190 con 0x56355e4564a0 2024-09-19T07:02:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.759+0000 7ff866e06740 10 client.0 _ll_register_callbacks cb 0x56355e3b7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.759+0000 7ff85ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 0x56355e45e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.759+0000 7ff85ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 0x56355e45e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39442/0 (socket says 172.21.15.103:39442) 2024-09-19T07:02:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.759+0000 7ff85ed9d640 1 -- 172.21.15.103:0/2365047908 learned_addr learned my addr 172.21.15.103:0/2365047908 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.759+0000 7ff85dd9b640 1 --2- 172.21.15.103:0/2365047908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e456dc0 0x56355e45e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.759+0000 7ff85e59c640 1 --2- 172.21.15.103:0/2365047908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e4564a0 0x56355e460f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff85ed9d640 1 -- 172.21.15.103:0/2365047908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e4564a0 msgr2=0x56355e460f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff85ed9d640 1 --2- 172.21.15.103:0/2365047908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e4564a0 0x56355e460f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff85ed9d640 1 -- 172.21.15.103:0/2365047908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e456dc0 msgr2=0x56355e45e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff85ed9d640 1 --2- 172.21.15.103:0/2365047908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e456dc0 0x56355e45e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff85ed9d640 1 -- 172.21.15.103:0/2365047908 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355e12d3c0 con 0x56355e461680 2024-09-19T07:02:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff85ed9d640 1 --2- 172.21.15.103:0/2365047908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 0x56355e45e770 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7ff8580126b0 tx=0x7ff858007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0942a637c98a93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff845ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff845ffb640 1 -- 172.21.15.103:0/2365047908 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858026070 con 0x56355e461680 2024-09-19T07:02:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff845ffb640 1 -- 172.21.15.103:0/2365047908 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7ff858004d20 con 0x56355e461680 2024-09-19T07:02:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff866e06740 1 -- 172.21.15.103:0/2365047908 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff850031de0 con 0x56355e461680 2024-09-19T07:02:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff845ffb640 1 -- 172.21.15.103:0/2365047908 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff858031050 con 0x56355e461680 2024-09-19T07:02:14.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff866e06740 10 client.19866 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:14.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff866e06740 1 -- 172.21.15.103:0/2365047908 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56355e46f8f0 con 0x56355e461680 2024-09-19T07:02:14.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff845ffb640 1 -- 172.21.15.103:0/2365047908 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff850031de0 con 0x56355e461680 2024-09-19T07:02:14.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff845ffb640 1 -- 172.21.15.103:0/2365047908 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff8580050b0 con 0x56355e461680 2024-09-19T07:02:14.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff866e06740 10 client.19866 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:14.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.760+0000 7ff866e06740 10 client.19866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:14.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.761+0000 7ff866e06740 20 client.19866 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:14.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.761+0000 7ff866e06740 1 --2- 172.21.15.103:0/2365047908 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56355e470d50 0x56355e4731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.761+0000 7ff866e06740 4 client.19866 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:14.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.761+0000 7ff866e06740 1 -- 172.21.15.103:0/2365047908 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56355e3c9850 con 0x56355e470d50 2024-09-19T07:02:14.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.761+0000 7ff85e59c640 1 --2- 172.21.15.103:0/2365047908 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56355e470d50 0x56355e4731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.761+0000 7ff85e59c640 1 --2- 172.21.15.103:0/2365047908 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56355e470d50 0x56355e4731d0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35f01e1ef6d34c96 server_cookie=57c0413e3ef7382d in_seq=0 out_seq=0 2024-09-19T07:02:14.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.761+0000 7ff845ffb640 10 client.19866 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:14.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.762+0000 7ff845ffb640 1 -- 172.21.15.103:0/2365047908 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56355e3c9850 con 0x56355e470d50 2024-09-19T07:02:14.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.762+0000 7ff845ffb640 10 client.19866 handle_command_reply: tid=0 2024-09-19T07:02:14.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.762+0000 7ff866e06740 1 client.19866 shutdown 2024-09-19T07:02:14.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.762+0000 7ff866e06740 1 -- 172.21.15.103:0/2365047908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 msgr2=0x56355e45e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:14.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.762+0000 7ff866e06740 1 --2- 172.21.15.103:0/2365047908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 0x56355e45e770 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7ff8580126b0 tx=0x7ff858007520 comp rx=0 tx=0).stop 2024-09-19T07:02:14.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.762+0000 7ff866e06740 1 -- 172.21.15.103:0/2365047908 shutdown_connections 2024-09-19T07:02:14.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.762+0000 7ff866e06740 1 --2- 172.21.15.103:0/2365047908 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56355e470d50 0x56355e4731d0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.762+0000 7ff866e06740 1 --2- 172.21.15.103:0/2365047908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 0x56355e45e770 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.762+0000 7ff866e06740 1 --2- 172.21.15.103:0/2365047908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e456dc0 0x56355e45e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.762+0000 7ff866e06740 1 --2- 172.21.15.103:0/2365047908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e4564a0 0x56355e460f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.762+0000 7ff866e06740 1 -- 172.21.15.103:0/2365047908 >> 172.21.15.103:0/2365047908 conn(0x56355e452a30 msgr2=0x56355e458760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:14.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.762+0000 7ff845ffb640 0 client.19866 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:14.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.762+0000 7ff866e06740 1 -- 172.21.15.103:0/2365047908 shutdown_connections 2024-09-19T07:02:14.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.762+0000 7ff866e06740 1 -- 172.21.15.103:0/2365047908 wait complete. 2024-09-19T07:02:14.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.762+0000 7ff866e06740 20 client.19866 trim_cache size 0 max 16384 2024-09-19T07:02:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.779+0000 7ff866e06740 1 Processor -- start 2024-09-19T07:02:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.779+0000 7ff866e06740 1 -- start start 2024-09-19T07:02:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.779+0000 7ff866e06740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e4564a0 0x56355e472d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.779+0000 7ff866e06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e456dc0 0x56355e4601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.779+0000 7ff866e06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 0x56355e478aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.779+0000 7ff866e06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff8601afa70 con 0x56355e461680 2024-09-19T07:02:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.779+0000 7ff866e06740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8601b15b0 con 0x56355e4564a0 2024-09-19T07:02:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.779+0000 7ff866e06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff8601b0140 con 0x56355e456dc0 2024-09-19T07:02:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.779+0000 7ff85e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e4564a0 0x56355e472d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.780+0000 7ff85ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 0x56355e478aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.780+0000 7ff85e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e4564a0 0x56355e472d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:38466/0 (socket says 172.21.15.103:38466) 2024-09-19T07:02:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.780+0000 7ff85e59c640 1 -- 172.21.15.103:0/4219628797 learned_addr learned my addr 172.21.15.103:0/4219628797 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.780+0000 7ff85dd9b640 1 --2- 172.21.15.103:0/4219628797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e456dc0 0x56355e4601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.780+0000 7ff85e59c640 1 -- 172.21.15.103:0/4219628797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e456dc0 msgr2=0x56355e4601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.780+0000 7ff85e59c640 1 --2- 172.21.15.103:0/4219628797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e456dc0 0x56355e4601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.780+0000 7ff85e59c640 1 -- 172.21.15.103:0/4219628797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 msgr2=0x56355e478aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.780+0000 7ff85e59c640 1 --2- 172.21.15.103:0/4219628797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 0x56355e478aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.780+0000 7ff85e59c640 1 -- 172.21.15.103:0/4219628797 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56355e45d1e0 con 0x56355e4564a0 2024-09-19T07:02:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.780+0000 7ff85e59c640 1 --2- 172.21.15.103:0/4219628797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e4564a0 0x56355e472d60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff850005820 tx=0x7ff85002a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddce3302a1b8bbe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.780+0000 7ff847fff640 1 -- 172.21.15.103:0/4219628797 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850049040 con 0x56355e4564a0 2024-09-19T07:02:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.780+0000 7ff847fff640 1 -- 172.21.15.103:0/4219628797 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8500c0b60 con 0x56355e4564a0 2024-09-19T07:02:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.780+0000 7ff847fff640 1 -- 172.21.15.103:0/4219628797 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8500bcac0 con 0x56355e4564a0 2024-09-19T07:02:14.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.781+0000 7ff866e06740 1 -- 172.21.15.103:0/4219628797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e4564a0 msgr2=0x56355e472d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:14.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.781+0000 7ff866e06740 1 --2- 172.21.15.103:0/4219628797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e4564a0 0x56355e472d60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff850005820 tx=0x7ff85002a480 comp rx=0 tx=0).stop 2024-09-19T07:02:14.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.781+0000 7ff866e06740 1 -- 172.21.15.103:0/4219628797 shutdown_connections 2024-09-19T07:02:14.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.781+0000 7ff866e06740 1 --2- 172.21.15.103:0/4219628797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e461680 0x56355e478aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.781+0000 7ff866e06740 1 --2- 172.21.15.103:0/4219628797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e456dc0 0x56355e4601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.781+0000 7ff866e06740 1 --2- 172.21.15.103:0/4219628797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e4564a0 0x56355e472d60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.781+0000 7ff866e06740 1 -- 172.21.15.103:0/4219628797 >> 172.21.15.103:0/4219628797 conn(0x56355e452a30 msgr2=0x56355e45fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.781+0000 7ff866e06740 1 -- 172.21.15.103:0/4219628797 shutdown_connections 2024-09-19T07:02:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.781+0000 7ff866e06740 1 -- 172.21.15.103:0/4219628797 wait complete. 2024-09-19T07:02:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.781+0000 7ff866e06740 1 -- start start 2024-09-19T07:02:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.781+0000 7ff866e06740 1 Processor -- start 2024-09-19T07:02:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff866e06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e4564a0 0x56355e469920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff866e06740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e456dc0 0x56355e466c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff866e06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e461680 0x56355e4671b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff866e06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff8601b0ee0 con 0x56355e4564a0 2024-09-19T07:02:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff866e06740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8601b0810 con 0x56355e456dc0 2024-09-19T07:02:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff866e06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff8601ada70 con 0x56355e461680 2024-09-19T07:02:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff866e06740 10 client.0 _ll_register_callbacks cb 0x7ff858011f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff85e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e4564a0 0x56355e469920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff85e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e4564a0 0x56355e469920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39464/0 (socket says 172.21.15.103:39464) 2024-09-19T07:02:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff85e59c640 1 -- 172.21.15.103:0/1408283008 learned_addr learned my addr 172.21.15.103:0/1408283008 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff85dd9b640 1 --2- 172.21.15.103:0/1408283008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e456dc0 0x56355e466c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff85ed9d640 1 --2- 172.21.15.103:0/1408283008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e461680 0x56355e4671b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff85e59c640 1 -- 172.21.15.103:0/1408283008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e461680 msgr2=0x56355e4671b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff85e59c640 1 --2- 172.21.15.103:0/1408283008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e461680 0x56355e4671b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff85e59c640 1 -- 172.21.15.103:0/1408283008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e456dc0 msgr2=0x56355e466c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff85e59c640 1 --2- 172.21.15.103:0/1408283008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e456dc0 0x56355e466c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff85e59c640 1 -- 172.21.15.103:0/1408283008 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355e456bf0 con 0x56355e4564a0 2024-09-19T07:02:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff85dd9b640 1 --2- 172.21.15.103:0/1408283008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e456dc0 0x56355e466c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff85e59c640 1 --2- 172.21.15.103:0/1408283008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e4564a0 0x56355e469920 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff8500bab10 tx=0x7ff8500edcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34daa2d728aecb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff845ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff845ffb640 1 -- 172.21.15.103:0/1408283008 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8500d0020 con 0x56355e4564a0 2024-09-19T07:02:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff845ffb640 1 -- 172.21.15.103:0/1408283008 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7ff8500cc150 con 0x56355e4564a0 2024-09-19T07:02:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff866e06740 1 -- 172.21.15.103:0/1408283008 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8500c0b60 con 0x56355e4564a0 2024-09-19T07:02:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff845ffb640 1 -- 172.21.15.103:0/1408283008 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff850049040 con 0x56355e4564a0 2024-09-19T07:02:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff866e06740 10 client.19872 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.782+0000 7ff866e06740 1 -- 172.21.15.103:0/1408283008 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56355e4db820 con 0x56355e4564a0 2024-09-19T07:02:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.783+0000 7ff845ffb640 1 -- 172.21.15.103:0/1408283008 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8500c0b60 con 0x56355e4564a0 2024-09-19T07:02:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.783+0000 7ff845ffb640 1 -- 172.21.15.103:0/1408283008 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff8500bcb50 con 0x56355e4564a0 2024-09-19T07:02:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.783+0000 7ff866e06740 10 client.19872 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.783+0000 7ff866e06740 10 client.19872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.783+0000 7ff866e06740 20 client.19872 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.783+0000 7ff866e06740 1 --2- 172.21.15.103:0/1408283008 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56355e4dbbb0 0x56355e4ddfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.783+0000 7ff866e06740 4 client.19872 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:02:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.783+0000 7ff866e06740 1 -- 172.21.15.103:0/1408283008 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56355e3c3d60 con 0x56355e4dbbb0 2024-09-19T07:02:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.783+0000 7ff85dd9b640 1 --2- 172.21.15.103:0/1408283008 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56355e4dbbb0 0x56355e4ddfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.783+0000 7ff85dd9b640 1 --2- 172.21.15.103:0/1408283008 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56355e4dbbb0 0x56355e4ddfa0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31e7d4e28ef42776 server_cookie=253841f88c05e4cf in_seq=0 out_seq=0 2024-09-19T07:02:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.783+0000 7ff845ffb640 10 client.19872 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.784+0000 7ff845ffb640 1 -- 172.21.15.103:0/1408283008 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56355e3c3d60 con 0x56355e4dbbb0 2024-09-19T07:02:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.784+0000 7ff845ffb640 10 client.19872 handle_command_reply: tid=0 2024-09-19T07:02:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.784+0000 7ff866e06740 1 client.19872 shutdown 2024-09-19T07:02:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.784+0000 7ff866e06740 1 -- 172.21.15.103:0/1408283008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e4564a0 msgr2=0x56355e469920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.784+0000 7ff866e06740 1 --2- 172.21.15.103:0/1408283008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e4564a0 0x56355e469920 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff8500bab10 tx=0x7ff8500edcd0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.784+0000 7ff866e06740 1 -- 172.21.15.103:0/1408283008 shutdown_connections 2024-09-19T07:02:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.784+0000 7ff866e06740 1 --2- 172.21.15.103:0/1408283008 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56355e4dbbb0 0x56355e4ddfa0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.784+0000 7ff866e06740 1 --2- 172.21.15.103:0/1408283008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56355e461680 0x56355e4671b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.784+0000 7ff866e06740 1 --2- 172.21.15.103:0/1408283008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56355e456dc0 0x56355e466c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.784+0000 7ff866e06740 1 --2- 172.21.15.103:0/1408283008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56355e4564a0 0x56355e469920 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.784+0000 7ff866e06740 1 -- 172.21.15.103:0/1408283008 >> 172.21.15.103:0/1408283008 conn(0x56355e452a30 msgr2=0x56355e4602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:14.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.784+0000 7ff845ffb640 0 client.19872 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:14.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.784+0000 7ff866e06740 1 -- 172.21.15.103:0/1408283008 shutdown_connections 2024-09-19T07:02:14.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.784+0000 7ff866e06740 1 -- 172.21.15.103:0/1408283008 wait complete. 2024-09-19T07:02:14.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.784+0000 7ff866e06740 20 client.19872 trim_cache size 0 max 16384 2024-09-19T07:02:14.791 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:02:14.791 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:02:14.791 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:02:14.791 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:02:14.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.785+0000 7ff864a2d640 1 -- 172.21.15.103:0/2341839752 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff8480a33b0 msgr2=0x7ff8480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:14.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.785+0000 7ff864a2d640 1 --2- 172.21.15.103:0/2341839752 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff8480a33b0 0x7ff8480a5870 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff858014c80 tx=0x7ff858007e80 comp rx=0 tx=0).stop 2024-09-19T07:02:14.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.785+0000 7ff864a2d640 1 -- 172.21.15.103:0/2341839752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff860100e20 msgr2=0x7ff8601970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:14.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.785+0000 7ff864a2d640 1 --2- 172.21.15.103:0/2341839752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff860100e20 0x7ff8601970e0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7ff850006010 tx=0x7ff850003c10 comp rx=0 tx=0).stop 2024-09-19T07:02:14.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.785+0000 7ff864a2d640 1 -- 172.21.15.103:0/2341839752 shutdown_connections 2024-09-19T07:02:14.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.785+0000 7ff864a2d640 1 --2- 172.21.15.103:0/2341839752 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff8480a33b0 0x7ff8480a5870 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.785+0000 7ff864a2d640 1 --2- 172.21.15.103:0/2341839752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff860102ef0 0x7ff860197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.785+0000 7ff864a2d640 1 --2- 172.21.15.103:0/2341839752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff860102570 0x7ff860197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.785+0000 7ff864a2d640 1 --2- 172.21.15.103:0/2341839752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff860100e20 0x7ff8601970e0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:14.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.785+0000 7ff864a2d640 1 -- 172.21.15.103:0/2341839752 >> 172.21.15.103:0/2341839752 conn(0x7ff8600fc7d0 msgr2=0x7ff8600fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:14.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.785+0000 7ff864a2d640 1 -- 172.21.15.103:0/2341839752 shutdown_connections 2024-09-19T07:02:14.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:14.785+0000 7ff864a2d640 1 -- 172.21.15.103:0/2341839752 wait complete. 2024-09-19T07:02:15.471 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:02:15.471 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:02:15.757 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:16.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:15 smithi103 ceph-mon[31158]: pgmap v407: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-09-19T07:02:16.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:15 smithi174 ceph-mon[35637]: pgmap v407: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-09-19T07:02:16.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:15 smithi134 ceph-mon[35356]: pgmap v407: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-09-19T07:02:16.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.234+0000 7f510d151640 1 -- 172.21.15.103:0/1069842733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5108101420 msgr2=0x7f510810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:16.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.234+0000 7f510d151640 1 --2- 172.21.15.103:0/1069842733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5108101420 0x7f510810d950 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009930 tx=0x7f50f802a420 comp rx=0 tx=0).stop 2024-09-19T07:02:16.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.234+0000 7f510d151640 1 -- 172.21.15.103:0/1069842733 shutdown_connections 2024-09-19T07:02:16.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.234+0000 7f510d151640 1 --2- 172.21.15.103:0/1069842733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5108101420 0x7f510810d950 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:16.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.234+0000 7f510d151640 1 --2- 172.21.15.103:0/1069842733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5108100a80 0x7f5108100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:16.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.234+0000 7f510d151640 1 --2- 172.21.15.103:0/1069842733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5108106a80 0x7f5108106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:16.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.234+0000 7f510d151640 1 -- 172.21.15.103:0/1069842733 >> 172.21.15.103:0/1069842733 conn(0x7f51080fc7a0 msgr2=0x7f51080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:16.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.234+0000 7f510d151640 1 -- 172.21.15.103:0/1069842733 shutdown_connections 2024-09-19T07:02:16.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.234+0000 7f510d151640 1 -- 172.21.15.103:0/1069842733 wait complete. 2024-09-19T07:02:16.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f510d151640 1 Processor -- start 2024-09-19T07:02:16.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f510d151640 1 -- start start 2024-09-19T07:02:16.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f510d151640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5108100a80 0x7f510819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:16.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f510d151640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5108101420 0x7f510819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:16.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f510d151640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5108106a80 0x7f510819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:16.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f510d151640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5108110f20 con 0x7f5108100a80 2024-09-19T07:02:16.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f510d151640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5108110da0 con 0x7f5108106a80 2024-09-19T07:02:16.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f510d151640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f51081110a0 con 0x7f5108101420 2024-09-19T07:02:16.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f5106d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5108100a80 0x7f510819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:16.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f5106d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5108100a80 0x7f510819a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39472/0 (socket says 172.21.15.103:39472) 2024-09-19T07:02:16.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f5106d9d640 1 -- 172.21.15.103:0/2600333373 learned_addr learned my addr 172.21.15.103:0/2600333373 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:16.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f510659c640 1 --2- 172.21.15.103:0/2600333373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5108101420 0x7f510819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:16.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f510759e640 1 --2- 172.21.15.103:0/2600333373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5108106a80 0x7f510819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:16.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f5106d9d640 1 -- 172.21.15.103:0/2600333373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5108101420 msgr2=0x7f510819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:16.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f5106d9d640 1 --2- 172.21.15.103:0/2600333373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5108101420 0x7f510819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:16.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f5106d9d640 1 -- 172.21.15.103:0/2600333373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5108106a80 msgr2=0x7f510819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:16.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f5106d9d640 1 --2- 172.21.15.103:0/2600333373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5108106a80 0x7f510819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:16.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f5106d9d640 1 -- 172.21.15.103:0/2600333373 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510819f6f0 con 0x7f5108100a80 2024-09-19T07:02:16.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.235+0000 7f510759e640 1 --2- 172.21.15.103:0/2600333373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5108106a80 0x7f510819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:16.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.236+0000 7f5106d9d640 1 --2- 172.21.15.103:0/2600333373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5108100a80 0x7f510819a740 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f510000ba50 tx=0x7f510000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:16.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.236+0000 7f50fffff640 1 -- 172.21.15.103:0/2600333373 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100019070 con 0x7f5108100a80 2024-09-19T07:02:16.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.236+0000 7f510d151640 1 -- 172.21.15.103:0/2600333373 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510819f9e0 con 0x7f5108100a80 2024-09-19T07:02:16.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.236+0000 7f50fffff640 1 -- 172.21.15.103:0/2600333373 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5100014440 con 0x7f5108100a80 2024-09-19T07:02:16.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.236+0000 7f50fffff640 1 -- 172.21.15.103:0/2600333373 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51000075d0 con 0x7f5108100a80 2024-09-19T07:02:16.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.236+0000 7f510d151640 1 -- 172.21.15.103:0/2600333373 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51081a72c0 con 0x7f5108100a80 2024-09-19T07:02:16.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.237+0000 7f510d151640 1 -- 172.21.15.103:0/2600333373 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5108100f60 con 0x7f5108100a80 2024-09-19T07:02:16.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.238+0000 7f50fffff640 1 -- 172.21.15.103:0/2600333373 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f510001c070 con 0x7f5108100a80 2024-09-19T07:02:16.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.239+0000 7f50fffff640 1 --2- 172.21.15.103:0/2600333373 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f50f40a33b0 0x7f50f40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:16.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.239+0000 7f510659c640 1 --2- 172.21.15.103:0/2600333373 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f50f40a33b0 0x7f50f40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:16.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.240+0000 7f50fffff640 1 -- 172.21.15.103:0/2600333373 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f51000cf280 con 0x7f5108100a80 2024-09-19T07:02:16.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.240+0000 7f510659c640 1 --2- 172.21.15.103:0/2600333373 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f50f40a33b0 0x7f50f40a5870 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f51080ff210 tx=0x7f50f0009110 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:16.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.244+0000 7f50fffff640 1 -- 172.21.15.103:0/2600333373 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f510009bd50 con 0x7f5108100a80 2024-09-19T07:02:16.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.445+0000 7f510d151640 1 -- 172.21.15.103:0/2600333373 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f510819baa0 con 0x7f5108100a80 2024-09-19T07:02:16.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.446+0000 7f50fffff640 1 -- 172.21.15.103:0/2600333373 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f510009b650 con 0x7f5108100a80 2024-09-19T07:02:16.447 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:02:16.447 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:02:16.447 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:02:16.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.448+0000 7f510d151640 1 -- 172.21.15.103:0/2600333373 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f50f40a33b0 msgr2=0x7f50f40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:16.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.448+0000 7f510d151640 1 --2- 172.21.15.103:0/2600333373 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f50f40a33b0 0x7f50f40a5870 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f51080ff210 tx=0x7f50f0009110 comp rx=0 tx=0).stop 2024-09-19T07:02:16.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.448+0000 7f510d151640 1 -- 172.21.15.103:0/2600333373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5108100a80 msgr2=0x7f510819a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:16.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.448+0000 7f510d151640 1 --2- 172.21.15.103:0/2600333373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5108100a80 0x7f510819a740 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f510000ba50 tx=0x7f510000be50 comp rx=0 tx=0).stop 2024-09-19T07:02:16.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.448+0000 7f510d151640 1 -- 172.21.15.103:0/2600333373 shutdown_connections 2024-09-19T07:02:16.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.448+0000 7f510d151640 1 --2- 172.21.15.103:0/2600333373 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f50f40a33b0 0x7f50f40a5870 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:16.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.448+0000 7f510d151640 1 --2- 172.21.15.103:0/2600333373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5108106a80 0x7f510819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:16.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.448+0000 7f510d151640 1 --2- 172.21.15.103:0/2600333373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5108101420 0x7f510819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:16.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.448+0000 7f510d151640 1 --2- 172.21.15.103:0/2600333373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5108100a80 0x7f510819a740 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:16.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.448+0000 7f510d151640 1 -- 172.21.15.103:0/2600333373 >> 172.21.15.103:0/2600333373 conn(0x7f51080fc7a0 msgr2=0x7f510806a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:16.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.448+0000 7f510d151640 1 -- 172.21.15.103:0/2600333373 shutdown_connections 2024-09-19T07:02:16.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:16.448+0000 7f510d151640 1 -- 172.21.15.103:0/2600333373 wait complete. 2024-09-19T07:02:17.037 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:16 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2600333373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:02:17.063 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:02:17.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:16 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2600333373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:02:17.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:16 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2600333373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:02:17.311 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:17.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.769+0000 7f8497a23640 1 -- 172.21.15.103:0/1457631242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8490106f40 msgr2=0x7f849010eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:17.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.769+0000 7f8497a23640 1 --2- 172.21.15.103:0/1457631242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8490106f40 0x7f849010eac0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f8488009900 tx=0x7f848802a410 comp rx=0 tx=0).stop 2024-09-19T07:02:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.769+0000 7f8497a23640 1 -- 172.21.15.103:0/1457631242 shutdown_connections 2024-09-19T07:02:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.769+0000 7f8497a23640 1 --2- 172.21.15.103:0/1457631242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8490106f40 0x7f849010eac0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.769+0000 7f8497a23640 1 --2- 172.21.15.103:0/1457631242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84901027c0 0x7f8490106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.769+0000 7f8497a23640 1 --2- 172.21.15.103:0/1457631242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8490101e10 0x7f84901021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.769+0000 7f8497a23640 1 -- 172.21.15.103:0/1457631242 >> 172.21.15.103:0/1457631242 conn(0x7f84900fd8d0 msgr2=0x7f84900ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.770+0000 7f8497a23640 1 -- 172.21.15.103:0/1457631242 shutdown_connections 2024-09-19T07:02:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.770+0000 7f8497a23640 1 -- 172.21.15.103:0/1457631242 wait complete. 2024-09-19T07:02:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.770+0000 7f8497a23640 1 Processor -- start 2024-09-19T07:02:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.770+0000 7f8497a23640 1 -- start start 2024-09-19T07:02:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.770+0000 7f8497a23640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8490101e10 0x7f84900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.770+0000 7f8497a23640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84901027c0 0x7f8490074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.770+0000 7f8497a23640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8490106f40 0x7f8490074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.770+0000 7f8497a23640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8490111fe0 con 0x7f8490101e10 2024-09-19T07:02:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.770+0000 7f8497a23640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8490111e60 con 0x7f8490106f40 2024-09-19T07:02:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.770+0000 7f8497a23640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8490112160 con 0x7f84901027c0 2024-09-19T07:02:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.770+0000 7f84957bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8490101e10 0x7f84900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.771+0000 7f84957bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8490101e10 0x7f84900744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47550/0 (socket says 172.21.15.103:47550) 2024-09-19T07:02:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.771+0000 7f84957bf640 1 -- 172.21.15.103:0/4240207601 learned_addr learned my addr 172.21.15.103:0/4240207601 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.771+0000 7f8495fc0640 1 --2- 172.21.15.103:0/4240207601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8490106f40 0x7f8490074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.771+0000 7f8494fbe640 1 --2- 172.21.15.103:0/4240207601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84901027c0 0x7f8490074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.771+0000 7f84957bf640 1 -- 172.21.15.103:0/4240207601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84901027c0 msgr2=0x7f8490074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.771+0000 7f84957bf640 1 --2- 172.21.15.103:0/4240207601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84901027c0 0x7f8490074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.771+0000 7f84957bf640 1 -- 172.21.15.103:0/4240207601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8490106f40 msgr2=0x7f8490074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.771+0000 7f84957bf640 1 --2- 172.21.15.103:0/4240207601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8490106f40 0x7f8490074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.771+0000 7f84957bf640 1 -- 172.21.15.103:0/4240207601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84901a6980 con 0x7f8490101e10 2024-09-19T07:02:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.771+0000 7f8495fc0640 1 --2- 172.21.15.103:0/4240207601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8490106f40 0x7f8490074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.771+0000 7f84957bf640 1 --2- 172.21.15.103:0/4240207601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8490101e10 0x7f84900744f0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f848000ba50 tx=0x7f848000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.771+0000 7f848e7fc640 1 -- 172.21.15.103:0/4240207601 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480019070 con 0x7f8490101e10 2024-09-19T07:02:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.771+0000 7f8497a23640 1 -- 172.21.15.103:0/4240207601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84901a6c70 con 0x7f8490101e10 2024-09-19T07:02:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.771+0000 7f848e7fc640 1 -- 172.21.15.103:0/4240207601 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f848000cbe0 con 0x7f8490101e10 2024-09-19T07:02:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.771+0000 7f848e7fc640 1 -- 172.21.15.103:0/4240207601 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84800076e0 con 0x7f8490101e10 2024-09-19T07:02:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.771+0000 7f8497a23640 1 -- 172.21.15.103:0/4240207601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84901a71b0 con 0x7f8490101e10 2024-09-19T07:02:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.773+0000 7f848e7fc640 1 -- 172.21.15.103:0/4240207601 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f848001c070 con 0x7f8490101e10 2024-09-19T07:02:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.774+0000 7f8499dfc740 1 Processor -- start 2024-09-19T07:02:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.774+0000 7f8499dfc740 1 -- start start 2024-09-19T07:02:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.774+0000 7f8499dfc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bb3b40 0x5605c9bb2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.774+0000 7f8499dfc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb25d0 0x5605c9bb2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.774+0000 7f8499dfc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bbd370 0x5605c9bbf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.774+0000 7f8499dfc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5605c9b182f0 con 0x5605c9bbd370 2024-09-19T07:02:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.774+0000 7f8499dfc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5605c9abfcf0 con 0x5605c9bb25d0 2024-09-19T07:02:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.774+0000 7f848e7fc640 1 --2- 172.21.15.103:0/4240207601 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f84780a33b0 0x7f84780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.774+0000 7f8499dfc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5605c9b142b0 con 0x5605c9bb3b40 2024-09-19T07:02:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.774+0000 7f8494fbe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bbd370 0x5605c9bbf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.774+0000 7f8494fbe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bbd370 0x5605c9bbf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47564/0 (socket says 172.21.15.103:47564) 2024-09-19T07:02:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.774+0000 7f8495fc0640 1 --2- 172.21.15.103:0/4240207601 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f84780a33b0 0x7f84780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.774+0000 7f8494fbe640 1 -- 172.21.15.103:0/3750815236 learned_addr learned my addr 172.21.15.103:0/3750815236 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.774+0000 7f848e7fc640 1 -- 172.21.15.103:0/4240207601 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f84800cf1e0 con 0x7f8490101e10 2024-09-19T07:02:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.774+0000 7f8495fc0640 1 --2- 172.21.15.103:0/3750815236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bb3b40 0x5605c9bb2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.774+0000 7f84957bf640 1 --2- 172.21.15.103:0/3750815236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb25d0 0x5605c9bb2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.775+0000 7f8494fbe640 1 -- 172.21.15.103:0/3750815236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bb3b40 msgr2=0x5605c9bb2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.775+0000 7f8494fbe640 1 --2- 172.21.15.103:0/3750815236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bb3b40 0x5605c9bb2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.775+0000 7f8494fbe640 1 -- 172.21.15.103:0/3750815236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb25d0 msgr2=0x5605c9bb2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.775+0000 7f8494fbe640 1 --2- 172.21.15.103:0/3750815236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb25d0 0x5605c9bb2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.775+0000 7f8494fbe640 1 -- 172.21.15.103:0/3750815236 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605c9aa43a0 con 0x5605c9bbd370 2024-09-19T07:02:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.775+0000 7f8494fbe640 1 --2- 172.21.15.103:0/3750815236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bbd370 0x5605c9bbf760 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f848400b740 tx=0x7f848400bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c646a0e9783d69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.775+0000 7f8495fc0640 1 --2- 172.21.15.103:0/4240207601 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f84780a33b0 0x7f84780a5870 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8488031550 tx=0x7f84880058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.775+0000 7f84777fe640 1 -- 172.21.15.103:0/3750815236 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484003dd0 con 0x5605c9bbd370 2024-09-19T07:02:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.775+0000 7f84777fe640 1 -- 172.21.15.103:0/3750815236 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8484015070 con 0x5605c9bbd370 2024-09-19T07:02:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.775+0000 7f84777fe640 1 -- 172.21.15.103:0/3750815236 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484004d80 con 0x5605c9bbd370 2024-09-19T07:02:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.776+0000 7f8499dfc740 1 -- 172.21.15.103:0/3750815236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bbd370 msgr2=0x5605c9bbf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.776+0000 7f8499dfc740 1 --2- 172.21.15.103:0/3750815236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bbd370 0x5605c9bbf760 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f848400b740 tx=0x7f848400bc00 comp rx=0 tx=0).stop 2024-09-19T07:02:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.776+0000 7f8499dfc740 1 -- 172.21.15.103:0/3750815236 shutdown_connections 2024-09-19T07:02:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.776+0000 7f8499dfc740 1 --2- 172.21.15.103:0/3750815236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bbd370 0x5605c9bbf760 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.776+0000 7f8499dfc740 1 --2- 172.21.15.103:0/3750815236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb25d0 0x5605c9bb2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.776+0000 7f8499dfc740 1 --2- 172.21.15.103:0/3750815236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bb3b40 0x5605c9bb2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.776+0000 7f8499dfc740 1 -- 172.21.15.103:0/3750815236 >> 172.21.15.103:0/3750815236 conn(0x5605c9b7b310 msgr2=0x5605c9baec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.776+0000 7f8499dfc740 1 -- 172.21.15.103:0/3750815236 shutdown_connections 2024-09-19T07:02:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.776+0000 7f8499dfc740 1 -- 172.21.15.103:0/3750815236 wait complete. 2024-09-19T07:02:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.776+0000 7f8499dfc740 1 -- start start 2024-09-19T07:02:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.776+0000 7f8499dfc740 1 Processor -- start 2024-09-19T07:02:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f8499dfc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb25d0 0x5605c9bbcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f8499dfc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb3b40 0x5605c9bb9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f8499dfc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bbd370 0x5605c9bba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f8499dfc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5605c9bc0070 con 0x5605c9bb25d0 2024-09-19T07:02:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f8499dfc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5605c9ad3060 con 0x5605c9bb3b40 2024-09-19T07:02:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f8499dfc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5605c9bc01f0 con 0x5605c9bbd370 2024-09-19T07:02:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f8499dfc740 10 client.0 _ll_register_callbacks cb 0x5605c98268c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f84957bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb25d0 0x5605c9bbcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f84957bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb25d0 0x5605c9bbcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47580/0 (socket says 172.21.15.103:47580) 2024-09-19T07:02:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f84957bf640 1 -- 172.21.15.103:0/1197737023 learned_addr learned my addr 172.21.15.103:0/1197737023 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f8495fc0640 1 --2- 172.21.15.103:0/1197737023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bbd370 0x5605c9bba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f8494fbe640 1 --2- 172.21.15.103:0/1197737023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb3b40 0x5605c9bb9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f84957bf640 1 -- 172.21.15.103:0/1197737023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bbd370 msgr2=0x5605c9bba4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f84957bf640 1 --2- 172.21.15.103:0/1197737023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bbd370 0x5605c9bba4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f84957bf640 1 -- 172.21.15.103:0/1197737023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb3b40 msgr2=0x5605c9bb9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f84957bf640 1 --2- 172.21.15.103:0/1197737023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb3b40 0x5605c9bb9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f84957bf640 1 -- 172.21.15.103:0/1197737023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605c98893c0 con 0x5605c9bb25d0 2024-09-19T07:02:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f8494fbe640 1 --2- 172.21.15.103:0/1197737023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb3b40 0x5605c9bb9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f84957bf640 1 --2- 172.21.15.103:0/1197737023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb25d0 0x5605c9bbcce0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f848009b5f0 tx=0x7f848009b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648bfd5cb31f419e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f8474ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f8474ff9640 1 -- 172.21.15.103:0/1197737023 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84800b3020 con 0x5605c9bb25d0 2024-09-19T07:02:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f8474ff9640 1 -- 172.21.15.103:0/1197737023 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f84800a3d90 con 0x5605c9bb25d0 2024-09-19T07:02:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f8499dfc740 1 -- 172.21.15.103:0/1197737023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8484015070 con 0x5605c9bb25d0 2024-09-19T07:02:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.777+0000 7f8474ff9640 1 -- 172.21.15.103:0/1197737023 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f84800a4220 con 0x5605c9bb25d0 2024-09-19T07:02:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.778+0000 7f8499dfc740 10 client.19890 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.778+0000 7f8499dfc740 1 -- 172.21.15.103:0/1197737023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5605c9bcbae0 con 0x5605c9bb25d0 2024-09-19T07:02:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.778+0000 7f8474ff9640 1 -- 172.21.15.103:0/1197737023 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8484015070 con 0x5605c9bb25d0 2024-09-19T07:02:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.778+0000 7f8474ff9640 1 -- 172.21.15.103:0/1197737023 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f84800973b0 con 0x5605c9bb25d0 2024-09-19T07:02:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.778+0000 7f8499dfc740 10 client.19890 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.778+0000 7f8499dfc740 10 client.19890 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:02:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.778+0000 7f8499dfc740 20 client.19890 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.778+0000 7f8499dfc740 1 --2- 172.21.15.103:0/1197737023 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5605c9bccef0 0x5605c9bcf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.778+0000 7f8499dfc740 4 client.19890 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.778+0000 7f8499dfc740 1 -- 172.21.15.103:0/1197737023 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605c9b25860 con 0x5605c9bccef0 2024-09-19T07:02:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.778+0000 7f8494fbe640 1 --2- 172.21.15.103:0/1197737023 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5605c9bccef0 0x5605c9bcf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.779+0000 7f8474ff9640 10 client.19890 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:02:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.779+0000 7f8494fbe640 1 --2- 172.21.15.103:0/1197737023 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5605c9bccef0 0x5605c9bcf360 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4d5dedbaedcebf03 server_cookie=ee86092b89732ca in_seq=0 out_seq=0 2024-09-19T07:02:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.779+0000 7f8474ff9640 1 -- 172.21.15.103:0/1197737023 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605c9b25860 con 0x5605c9bccef0 2024-09-19T07:02:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.779+0000 7f8474ff9640 10 client.19890 handle_command_reply: tid=0 2024-09-19T07:02:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.779+0000 7f8499dfc740 1 client.19890 shutdown 2024-09-19T07:02:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.779+0000 7f8499dfc740 1 -- 172.21.15.103:0/1197737023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb25d0 msgr2=0x5605c9bbcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.779+0000 7f8499dfc740 1 --2- 172.21.15.103:0/1197737023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb25d0 0x5605c9bbcce0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f848009b5f0 tx=0x7f848009b260 comp rx=0 tx=0).stop 2024-09-19T07:02:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.779+0000 7f8499dfc740 1 -- 172.21.15.103:0/1197737023 shutdown_connections 2024-09-19T07:02:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.779+0000 7f8499dfc740 1 --2- 172.21.15.103:0/1197737023 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5605c9bccef0 0x5605c9bcf360 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.779+0000 7f8499dfc740 1 --2- 172.21.15.103:0/1197737023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bbd370 0x5605c9bba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.779+0000 7f8499dfc740 1 --2- 172.21.15.103:0/1197737023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb3b40 0x5605c9bb9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.779+0000 7f8474ff9640 0 client.19890 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:02:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.779+0000 7f8499dfc740 1 --2- 172.21.15.103:0/1197737023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb25d0 0x5605c9bbcce0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.779+0000 7f8499dfc740 1 -- 172.21.15.103:0/1197737023 >> 172.21.15.103:0/1197737023 conn(0x5605c9b7b310 msgr2=0x5605c9bb4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.780+0000 7f8499dfc740 1 -- 172.21.15.103:0/1197737023 shutdown_connections 2024-09-19T07:02:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.780+0000 7f8499dfc740 1 -- 172.21.15.103:0/1197737023 wait complete. 2024-09-19T07:02:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.780+0000 7f8499dfc740 20 client.19890 trim_cache size 0 max 16384 2024-09-19T07:02:17.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.793+0000 7f8499dfc740 1 Processor -- start 2024-09-19T07:02:17.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.793+0000 7f8499dfc740 1 -- start start 2024-09-19T07:02:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.793+0000 7f8499dfc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb25d0 0x5605c9bb5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.793+0000 7f8499dfc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb3b40 0x5605c9bb5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.793+0000 7f8499dfc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bbd370 0x5605c9bd4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.793+0000 7f8499dfc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84901afb10 con 0x5605c9bb3b40 2024-09-19T07:02:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.793+0000 7f8499dfc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84901b1650 con 0x5605c9bb25d0 2024-09-19T07:02:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.793+0000 7f8499dfc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84901b01e0 con 0x5605c9bbd370 2024-09-19T07:02:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.794+0000 7f8494fbe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb3b40 0x5605c9bb5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.794+0000 7f8494fbe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb3b40 0x5605c9bb5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47594/0 (socket says 172.21.15.103:47594) 2024-09-19T07:02:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.794+0000 7f8494fbe640 1 -- 172.21.15.103:0/1840772259 learned_addr learned my addr 172.21.15.103:0/1840772259 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.794+0000 7f8495fc0640 1 --2- 172.21.15.103:0/1840772259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bbd370 0x5605c9bd4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.794+0000 7f84957bf640 1 --2- 172.21.15.103:0/1840772259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb25d0 0x5605c9bb5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.794+0000 7f8494fbe640 1 -- 172.21.15.103:0/1840772259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bbd370 msgr2=0x5605c9bd4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.794+0000 7f8494fbe640 1 --2- 172.21.15.103:0/1840772259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bbd370 0x5605c9bd4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.794+0000 7f8494fbe640 1 -- 172.21.15.103:0/1840772259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb25d0 msgr2=0x5605c9bb5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.794+0000 7f8494fbe640 1 --2- 172.21.15.103:0/1840772259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb25d0 0x5605c9bb5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.794+0000 7f8494fbe640 1 -- 172.21.15.103:0/1840772259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605c9bb8f40 con 0x5605c9bb3b40 2024-09-19T07:02:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.794+0000 7f84957bf640 1 --2- 172.21.15.103:0/1840772259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb25d0 0x5605c9bb5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.794+0000 7f8494fbe640 1 --2- 172.21.15.103:0/1840772259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb3b40 0x5605c9bb5660 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f848400c010 tx=0x7f8484013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5bebcc263e4bfb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.794+0000 7f8476ffd640 1 -- 172.21.15.103:0/1840772259 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484003bd0 con 0x5605c9bb3b40 2024-09-19T07:02:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.794+0000 7f8476ffd640 1 -- 172.21.15.103:0/1840772259 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8484015030 con 0x5605c9bb3b40 2024-09-19T07:02:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.794+0000 7f8476ffd640 1 -- 172.21.15.103:0/1840772259 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484011440 con 0x5605c9bb3b40 2024-09-19T07:02:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.795+0000 7f8499dfc740 1 -- 172.21.15.103:0/1840772259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb3b40 msgr2=0x5605c9bb5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.795+0000 7f8499dfc740 1 --2- 172.21.15.103:0/1840772259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb3b40 0x5605c9bb5660 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f848400c010 tx=0x7f8484013040 comp rx=0 tx=0).stop 2024-09-19T07:02:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.795+0000 7f8499dfc740 1 -- 172.21.15.103:0/1840772259 shutdown_connections 2024-09-19T07:02:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.795+0000 7f8499dfc740 1 --2- 172.21.15.103:0/1840772259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bbd370 0x5605c9bd4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.795+0000 7f8499dfc740 1 --2- 172.21.15.103:0/1840772259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb3b40 0x5605c9bb5660 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.795+0000 7f8499dfc740 1 --2- 172.21.15.103:0/1840772259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb25d0 0x5605c9bb5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.795+0000 7f8499dfc740 1 -- 172.21.15.103:0/1840772259 >> 172.21.15.103:0/1840772259 conn(0x5605c9b7b310 msgr2=0x5605c9badd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.795+0000 7f8499dfc740 1 -- 172.21.15.103:0/1840772259 shutdown_connections 2024-09-19T07:02:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.795+0000 7f8499dfc740 1 -- 172.21.15.103:0/1840772259 wait complete. 2024-09-19T07:02:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.795+0000 7f8499dfc740 1 -- start start 2024-09-19T07:02:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.795+0000 7f8499dfc740 1 Processor -- start 2024-09-19T07:02:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f8499dfc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb25d0 0x5605c9bc5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f8499dfc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb3b40 0x5605c9bc5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f8499dfc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bbd370 0x5605c9bc2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f8499dfc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84901b0f80 con 0x5605c9bb25d0 2024-09-19T07:02:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f8499dfc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84901b08b0 con 0x5605c9bb3b40 2024-09-19T07:02:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f8499dfc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84901adb10 con 0x5605c9bbd370 2024-09-19T07:02:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f8499dfc740 10 client.0 _ll_register_callbacks cb 0x7f8480097290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f84957bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb25d0 0x5605c9bc5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f84957bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb25d0 0x5605c9bc5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47608/0 (socket says 172.21.15.103:47608) 2024-09-19T07:02:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f84957bf640 1 -- 172.21.15.103:0/1251034191 learned_addr learned my addr 172.21.15.103:0/1251034191 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f8494fbe640 1 --2- 172.21.15.103:0/1251034191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb3b40 0x5605c9bc5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f8495fc0640 1 --2- 172.21.15.103:0/1251034191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bbd370 0x5605c9bc2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f84957bf640 1 -- 172.21.15.103:0/1251034191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bbd370 msgr2=0x5605c9bc2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f84957bf640 1 --2- 172.21.15.103:0/1251034191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bbd370 0x5605c9bc2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f84957bf640 1 -- 172.21.15.103:0/1251034191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb3b40 msgr2=0x5605c9bc5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f84957bf640 1 --2- 172.21.15.103:0/1251034191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb3b40 0x5605c9bc5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f84957bf640 1 -- 172.21.15.103:0/1251034191 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605c9bc2b60 con 0x5605c9bb25d0 2024-09-19T07:02:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f8495fc0640 1 --2- 172.21.15.103:0/1251034191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bbd370 0x5605c9bc2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f84957bf640 1 --2- 172.21.15.103:0/1251034191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb25d0 0x5605c9bc5660 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f8480016000 tx=0x7f848009b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5eda0d27016730 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.796+0000 7f8474ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.797+0000 7f8474ff9640 1 -- 172.21.15.103:0/1251034191 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84800b3070 con 0x5605c9bb25d0 2024-09-19T07:02:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.797+0000 7f8474ff9640 1 -- 172.21.15.103:0/1251034191 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f84800a3d90 con 0x5605c9bb25d0 2024-09-19T07:02:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.797+0000 7f8499dfc740 1 -- 172.21.15.103:0/1251034191 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8484015030 con 0x5605c9bb25d0 2024-09-19T07:02:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.797+0000 7f8474ff9640 1 -- 172.21.15.103:0/1251034191 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8480097900 con 0x5605c9bb25d0 2024-09-19T07:02:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.797+0000 7f8499dfc740 10 client.19896 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.797+0000 7f8499dfc740 1 -- 172.21.15.103:0/1251034191 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5605c9c38570 con 0x5605c9bb25d0 2024-09-19T07:02:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.797+0000 7f8474ff9640 1 -- 172.21.15.103:0/1251034191 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8484015030 con 0x5605c9bb25d0 2024-09-19T07:02:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.797+0000 7f8474ff9640 1 -- 172.21.15.103:0/1251034191 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f848009ab40 con 0x5605c9bb25d0 2024-09-19T07:02:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.797+0000 7f8499dfc740 10 client.19896 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.797+0000 7f8499dfc740 10 client.19896 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:02:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.797+0000 7f8499dfc740 20 client.19896 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.797+0000 7f8499dfc740 1 --2- 172.21.15.103:0/1251034191 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5605c9c38c30 0x5605c9c3b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.797+0000 7f8499dfc740 4 client.19896 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:02:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.797+0000 7f8499dfc740 1 -- 172.21.15.103:0/1251034191 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605c9b1fd70 con 0x5605c9c38c30 2024-09-19T07:02:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.797+0000 7f8494fbe640 1 --2- 172.21.15.103:0/1251034191 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5605c9c38c30 0x5605c9c3b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8474ff9640 10 client.19896 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:02:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8494fbe640 1 --2- 172.21.15.103:0/1251034191 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5605c9c38c30 0x5605c9c3b050 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=33e3eccca211197 server_cookie=b8354d5f8603063f in_seq=0 out_seq=0 2024-09-19T07:02:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8474ff9640 1 -- 172.21.15.103:0/1251034191 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605c9b1fd70 con 0x5605c9c38c30 2024-09-19T07:02:17.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8474ff9640 10 client.19896 handle_command_reply: tid=0 2024-09-19T07:02:17.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8499dfc740 1 client.19896 shutdown 2024-09-19T07:02:17.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8499dfc740 1 -- 172.21.15.103:0/1251034191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb25d0 msgr2=0x5605c9bc5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:17.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8499dfc740 1 --2- 172.21.15.103:0/1251034191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb25d0 0x5605c9bc5660 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f8480016000 tx=0x7f848009b2d0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8499dfc740 1 -- 172.21.15.103:0/1251034191 shutdown_connections 2024-09-19T07:02:17.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8499dfc740 1 --2- 172.21.15.103:0/1251034191 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x5605c9c38c30 0x5605c9c3b050 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8499dfc740 1 --2- 172.21.15.103:0/1251034191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c9bbd370 0x5605c9bc2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8499dfc740 1 --2- 172.21.15.103:0/1251034191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c9bb3b40 0x5605c9bc5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8474ff9640 0 client.19896 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:02:17.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8499dfc740 1 --2- 172.21.15.103:0/1251034191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c9bb25d0 0x5605c9bc5660 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8499dfc740 1 -- 172.21.15.103:0/1251034191 >> 172.21.15.103:0/1251034191 conn(0x5605c9b7b310 msgr2=0x5605c9bbb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:17.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8499dfc740 1 -- 172.21.15.103:0/1251034191 shutdown_connections 2024-09-19T07:02:17.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8499dfc740 1 -- 172.21.15.103:0/1251034191 wait complete. 2024-09-19T07:02:17.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.798+0000 7f8499dfc740 20 client.19896 trim_cache size 0 max 16384 2024-09-19T07:02:17.811 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:02:17.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.799+0000 7f8497a23640 1 -- 172.21.15.103:0/4240207601 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f84780a33b0 msgr2=0x7f84780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:17.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.799+0000 7f8497a23640 1 --2- 172.21.15.103:0/4240207601 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f84780a33b0 0x7f84780a5870 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8488031550 tx=0x7f84880058d0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.799+0000 7f8497a23640 1 -- 172.21.15.103:0/4240207601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8490101e10 msgr2=0x7f84900744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:17.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.799+0000 7f8497a23640 1 --2- 172.21.15.103:0/4240207601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8490101e10 0x7f84900744f0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f848000ba50 tx=0x7f848000bf10 comp rx=0 tx=0).stop 2024-09-19T07:02:17.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.799+0000 7f8497a23640 1 -- 172.21.15.103:0/4240207601 shutdown_connections 2024-09-19T07:02:17.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.799+0000 7f8497a23640 1 --2- 172.21.15.103:0/4240207601 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f84780a33b0 0x7f84780a5870 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.799+0000 7f8497a23640 1 --2- 172.21.15.103:0/4240207601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8490106f40 0x7f8490074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.799+0000 7f8497a23640 1 --2- 172.21.15.103:0/4240207601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84901027c0 0x7f8490074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.799+0000 7f8497a23640 1 --2- 172.21.15.103:0/4240207601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8490101e10 0x7f84900744f0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:17.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.799+0000 7f8497a23640 1 -- 172.21.15.103:0/4240207601 >> 172.21.15.103:0/4240207601 conn(0x7f84900fd8d0 msgr2=0x7f84900fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:17.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.799+0000 7f8497a23640 1 -- 172.21.15.103:0/4240207601 shutdown_connections 2024-09-19T07:02:17.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:17.799+0000 7f8497a23640 1 -- 172.21.15.103:0/4240207601 wait complete. 2024-09-19T07:02:18.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:17 smithi174 ceph-mon[35637]: pgmap v408: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-19T07:02:18.110 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:17 smithi103 ceph-mon[31158]: pgmap v408: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-19T07:02:18.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:17 smithi134 ceph-mon[35356]: pgmap v408: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-19T07:02:18.466 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:02:18.712 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:19.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.205+0000 7fb1c7289640 1 -- 172.21.15.103:0/2100459785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c0102da0 msgr2=0x7fb1c0103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.205+0000 7fb1c7289640 1 --2- 172.21.15.103:0/2100459785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c0102da0 0x7fb1c0103180 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009900 tx=0x7fb1bc02a480 comp rx=0 tx=0).stop 2024-09-19T07:02:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.205+0000 7fb1c7289640 1 -- 172.21.15.103:0/2100459785 shutdown_connections 2024-09-19T07:02:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.205+0000 7fb1c7289640 1 --2- 172.21.15.103:0/2100459785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c010c180 0x7fb1c010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.205+0000 7fb1c7289640 1 --2- 172.21.15.103:0/2100459785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c01036c0 0x7fb1c010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.205+0000 7fb1c7289640 1 --2- 172.21.15.103:0/2100459785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c0102da0 0x7fb1c0103180 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.205+0000 7fb1c7289640 1 -- 172.21.15.103:0/2100459785 >> 172.21.15.103:0/2100459785 conn(0x7fb1c00fc7a0 msgr2=0x7fb1c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.205+0000 7fb1c7289640 1 -- 172.21.15.103:0/2100459785 shutdown_connections 2024-09-19T07:02:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.205+0000 7fb1c7289640 1 -- 172.21.15.103:0/2100459785 wait complete. 2024-09-19T07:02:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.206+0000 7fb1c7289640 1 Processor -- start 2024-09-19T07:02:19.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.206+0000 7fb1c7289640 1 -- start start 2024-09-19T07:02:19.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.206+0000 7fb1c7289640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c0102da0 0x7fb1c0100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.206+0000 7fb1c7289640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c01036c0 0x7fb1c0100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.206+0000 7fb1c7289640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c010c180 0x7fb1c0101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.206+0000 7fb1c7289640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c0111f60 con 0x7fb1c01036c0 2024-09-19T07:02:19.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.206+0000 7fb1c7289640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c0111de0 con 0x7fb1c0102da0 2024-09-19T07:02:19.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.206+0000 7fb1c7289640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c01120e0 con 0x7fb1c010c180 2024-09-19T07:02:19.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.206+0000 7fb1c4824640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c01036c0 0x7fb1c0100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:19.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.206+0000 7fb1c4824640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c01036c0 0x7fb1c0100f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47622/0 (socket says 172.21.15.103:47622) 2024-09-19T07:02:19.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.206+0000 7fb1c4824640 1 -- 172.21.15.103:0/3202043277 learned_addr learned my addr 172.21.15.103:0/3202043277 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:19.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.206+0000 7fb1c5025640 1 --2- 172.21.15.103:0/3202043277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c0102da0 0x7fb1c0100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:19.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.207+0000 7fb1c5826640 1 --2- 172.21.15.103:0/3202043277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c010c180 0x7fb1c0101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:19.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.207+0000 7fb1c4824640 1 -- 172.21.15.103:0/3202043277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c010c180 msgr2=0x7fb1c0101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:19.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.207+0000 7fb1c4824640 1 --2- 172.21.15.103:0/3202043277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c010c180 0x7fb1c0101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.207+0000 7fb1c4824640 1 -- 172.21.15.103:0/3202043277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c0102da0 msgr2=0x7fb1c0100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:19.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.207+0000 7fb1c4824640 1 --2- 172.21.15.103:0/3202043277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c0102da0 0x7fb1c0100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.207+0000 7fb1c4824640 1 -- 172.21.15.103:0/3202043277 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c01a6840 con 0x7fb1c01036c0 2024-09-19T07:02:19.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.207+0000 7fb1c5025640 1 --2- 172.21.15.103:0/3202043277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c0102da0 0x7fb1c0100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:19.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.207+0000 7fb1c5826640 1 --2- 172.21.15.103:0/3202043277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c010c180 0x7fb1c0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:19.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.207+0000 7fb1c4824640 1 --2- 172.21.15.103:0/3202043277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c01036c0 0x7fb1c0100f40 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000c890 tx=0x7fb1b000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:19.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.207+0000 7fb1b67fc640 1 -- 172.21.15.103:0/3202043277 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0004220 con 0x7fb1c01036c0 2024-09-19T07:02:19.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.207+0000 7fb1b67fc640 1 -- 172.21.15.103:0/3202043277 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1b00043c0 con 0x7fb1c01036c0 2024-09-19T07:02:19.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.207+0000 7fb1c7289640 1 -- 172.21.15.103:0/3202043277 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c01a6b30 con 0x7fb1c01036c0 2024-09-19T07:02:19.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.207+0000 7fb1b67fc640 1 -- 172.21.15.103:0/3202043277 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b00049e0 con 0x7fb1c01036c0 2024-09-19T07:02:19.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.207+0000 7fb1c7289640 1 -- 172.21.15.103:0/3202043277 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c01a7070 con 0x7fb1c01036c0 2024-09-19T07:02:19.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.209+0000 7fb1b67fc640 1 -- 172.21.15.103:0/3202043277 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fb1b00079b0 con 0x7fb1c01036c0 2024-09-19T07:02:19.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.210+0000 7fb1c9662740 1 Processor -- start 2024-09-19T07:02:19.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.210+0000 7fb1c9662740 1 -- start start 2024-09-19T07:02:19.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.210+0000 7fb1c9662740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e43eb40 0x558e3e43d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.210+0000 7fb1c9662740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43d5d0 0x558e3e43da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.210+0000 7fb1c9662740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e448370 0x558e3e44a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.210+0000 7fb1c9662740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e3e3a32f0 con 0x558e3e43d5d0 2024-09-19T07:02:19.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.210+0000 7fb1c9662740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558e3e34acf0 con 0x558e3e448370 2024-09-19T07:02:19.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.210+0000 7fb1c9662740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e3e39f2b0 con 0x558e3e43eb40 2024-09-19T07:02:19.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.210+0000 7fb1c5025640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43d5d0 0x558e3e43da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:19.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.210+0000 7fb1b67fc640 1 --2- 172.21.15.103:0/3202043277 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb1ac0a33b0 0x7fb1ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.210+0000 7fb1c5025640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43d5d0 0x558e3e43da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47630/0 (socket says 172.21.15.103:47630) 2024-09-19T07:02:19.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.210+0000 7fb1c5025640 1 -- 172.21.15.103:0/4244615053 learned_addr learned my addr 172.21.15.103:0/4244615053 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:19.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.210+0000 7fb1c5826640 1 --2- 172.21.15.103:0/4244615053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e43eb40 0x558e3e43d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:19.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.211+0000 7fb1c4824640 1 --2- 172.21.15.103:0/4244615053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e448370 0x558e3e44a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:19.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.211+0000 7fb1c5826640 1 --2- 172.21.15.103:0/3202043277 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb1ac0a33b0 0x7fb1ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:19.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.211+0000 7fb1b67fc640 1 -- 172.21.15.103:0/3202043277 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7fb1b00cfeb0 con 0x7fb1c01036c0 2024-09-19T07:02:19.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.211+0000 7fb1c5025640 1 -- 172.21.15.103:0/4244615053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e43eb40 msgr2=0x558e3e43d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:19.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.211+0000 7fb1c5025640 1 --2- 172.21.15.103:0/4244615053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e43eb40 0x558e3e43d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.211+0000 7fb1c5025640 1 -- 172.21.15.103:0/4244615053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e448370 msgr2=0x558e3e44a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:19.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.211+0000 7fb1c5025640 1 --2- 172.21.15.103:0/4244615053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e448370 0x558e3e44a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.211+0000 7fb1c5025640 1 -- 172.21.15.103:0/4244615053 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e3e32f3a0 con 0x558e3e43d5d0 2024-09-19T07:02:19.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.211+0000 7fb1c5025640 1 --2- 172.21.15.103:0/4244615053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43d5d0 0x558e3e43da10 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009900 tx=0x7fb1bc0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fad7171c9317948 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:19.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.211+0000 7fb1c5826640 1 --2- 172.21.15.103:0/3202043277 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb1ac0a33b0 0x7fb1ac0a5870 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800afd0 tx=0x7fb1b8005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:19.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.211+0000 7fb1ab7fe640 1 -- 172.21.15.103:0/4244615053 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc03f070 con 0x558e3e43d5d0 2024-09-19T07:02:19.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.211+0000 7fb1ab7fe640 1 -- 172.21.15.103:0/4244615053 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1bc0035f0 con 0x558e3e43d5d0 2024-09-19T07:02:19.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.211+0000 7fb1ab7fe640 1 -- 172.21.15.103:0/4244615053 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc035070 con 0x558e3e43d5d0 2024-09-19T07:02:19.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.212+0000 7fb1c9662740 1 -- 172.21.15.103:0/4244615053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43d5d0 msgr2=0x558e3e43da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:19.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.212+0000 7fb1c9662740 1 --2- 172.21.15.103:0/4244615053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43d5d0 0x558e3e43da10 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009900 tx=0x7fb1bc0316e0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.212+0000 7fb1c9662740 1 -- 172.21.15.103:0/4244615053 shutdown_connections 2024-09-19T07:02:19.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.212+0000 7fb1c9662740 1 --2- 172.21.15.103:0/4244615053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e448370 0x558e3e44a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.212+0000 7fb1c9662740 1 --2- 172.21.15.103:0/4244615053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43d5d0 0x558e3e43da10 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.212+0000 7fb1c9662740 1 --2- 172.21.15.103:0/4244615053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e43eb40 0x558e3e43d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.212+0000 7fb1c9662740 1 -- 172.21.15.103:0/4244615053 >> 172.21.15.103:0/4244615053 conn(0x558e3e406310 msgr2=0x558e3e439c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:19.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.212+0000 7fb1c9662740 1 -- 172.21.15.103:0/4244615053 shutdown_connections 2024-09-19T07:02:19.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.212+0000 7fb1c9662740 1 -- 172.21.15.103:0/4244615053 wait complete. 2024-09-19T07:02:19.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.212+0000 7fb1c9662740 1 -- start start 2024-09-19T07:02:19.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.212+0000 7fb1c9662740 1 Processor -- start 2024-09-19T07:02:19.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c9662740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43d5d0 0x558e3e447d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c9662740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e43eb40 0x558e3e444fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c9662740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e448370 0x558e3e4455d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c9662740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e3e44b010 con 0x558e3e43d5d0 2024-09-19T07:02:19.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c9662740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558e3e35e060 con 0x558e3e43eb40 2024-09-19T07:02:19.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c9662740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e3e44b190 con 0x558e3e448370 2024-09-19T07:02:19.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c9662740 10 client.0 _ll_register_callbacks cb 0x558e3e0b18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:19.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c5025640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43d5d0 0x558e3e447d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:19.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c5025640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43d5d0 0x558e3e447d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47642/0 (socket says 172.21.15.103:47642) 2024-09-19T07:02:19.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c5025640 1 -- 172.21.15.103:0/3015699469 learned_addr learned my addr 172.21.15.103:0/3015699469 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:19.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c5025640 1 -- 172.21.15.103:0/3015699469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e448370 msgr2=0x558e3e4455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:19.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c5025640 1 --2- 172.21.15.103:0/3015699469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e448370 0x558e3e4455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c5025640 1 -- 172.21.15.103:0/3015699469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e43eb40 msgr2=0x558e3e444fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:19.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c5025640 1 --2- 172.21.15.103:0/3015699469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e43eb40 0x558e3e444fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c5025640 1 -- 172.21.15.103:0/3015699469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e3e1143c0 con 0x558e3e43d5d0 2024-09-19T07:02:19.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c5025640 1 --2- 172.21.15.103:0/3015699469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43d5d0 0x558e3e447d30 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc005940 tx=0x7fb1bc004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c479fbdd374d2c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:19.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:19.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1a8ff9640 1 -- 172.21.15.103:0/3015699469 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc035070 con 0x558e3e43d5d0 2024-09-19T07:02:19.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1a8ff9640 1 -- 172.21.15.103:0/3015699469 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7fb1bc0318d0 con 0x558e3e43d5d0 2024-09-19T07:02:19.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.213+0000 7fb1c9662740 1 -- 172.21.15.103:0/3015699469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1bc0035f0 con 0x558e3e43d5d0 2024-09-19T07:02:19.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.214+0000 7fb1a8ff9640 1 -- 172.21.15.103:0/3015699469 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb1bc054050 con 0x558e3e43d5d0 2024-09-19T07:02:19.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.214+0000 7fb1c9662740 10 client.19908 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:19.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.214+0000 7fb1c9662740 1 -- 172.21.15.103:0/3015699469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e3e456a70 con 0x558e3e43d5d0 2024-09-19T07:02:19.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.214+0000 7fb1a8ff9640 1 -- 172.21.15.103:0/3015699469 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1bc0035f0 con 0x558e3e43d5d0 2024-09-19T07:02:19.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.214+0000 7fb1a8ff9640 1 -- 172.21.15.103:0/3015699469 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb1bc043400 con 0x558e3e43d5d0 2024-09-19T07:02:19.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.214+0000 7fb1c9662740 10 client.19908 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:19.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.214+0000 7fb1c9662740 10 client.19908 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:02:19.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.214+0000 7fb1c9662740 20 client.19908 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:19.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.214+0000 7fb1c9662740 1 --2- 172.21.15.103:0/3015699469 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x558e3e457dc0 0x558e3e45a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.214+0000 7fb1c9662740 4 client.19908 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:19.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.214+0000 7fb1c9662740 1 -- 172.21.15.103:0/3015699469 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e3e3b0860 con 0x558e3e457dc0 2024-09-19T07:02:19.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.214+0000 7fb1c4824640 1 --2- 172.21.15.103:0/3015699469 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x558e3e457dc0 0x558e3e45a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:19.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.215+0000 7fb1a8ff9640 10 client.19908 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:02:19.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.215+0000 7fb1c4824640 1 --2- 172.21.15.103:0/3015699469 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x558e3e457dc0 0x558e3e45a1b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2edf01fa2a7a33a4 server_cookie=5c7a48eaceaaba63 in_seq=0 out_seq=0 2024-09-19T07:02:19.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.216+0000 7fb1a8ff9640 1 -- 172.21.15.103:0/3015699469 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e3e3b0860 con 0x558e3e457dc0 2024-09-19T07:02:19.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.216+0000 7fb1a8ff9640 10 client.19908 handle_command_reply: tid=0 2024-09-19T07:02:19.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.216+0000 7fb1c9662740 1 client.19908 shutdown 2024-09-19T07:02:19.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.216+0000 7fb1c9662740 1 -- 172.21.15.103:0/3015699469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43d5d0 msgr2=0x558e3e447d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:19.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.216+0000 7fb1c9662740 1 --2- 172.21.15.103:0/3015699469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43d5d0 0x558e3e447d30 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc005940 tx=0x7fb1bc004250 comp rx=0 tx=0).stop 2024-09-19T07:02:19.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.216+0000 7fb1c9662740 1 -- 172.21.15.103:0/3015699469 shutdown_connections 2024-09-19T07:02:19.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.216+0000 7fb1c9662740 1 --2- 172.21.15.103:0/3015699469 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x558e3e457dc0 0x558e3e45a1b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.216+0000 7fb1c9662740 1 --2- 172.21.15.103:0/3015699469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e448370 0x558e3e4455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.216+0000 7fb1c9662740 1 --2- 172.21.15.103:0/3015699469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e43eb40 0x558e3e444fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.216+0000 7fb1c9662740 1 --2- 172.21.15.103:0/3015699469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43d5d0 0x558e3e447d30 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.216+0000 7fb1c9662740 1 -- 172.21.15.103:0/3015699469 >> 172.21.15.103:0/3015699469 conn(0x558e3e406310 msgr2=0x558e3e43f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:19.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.216+0000 7fb1a8ff9640 0 client.19908 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:02:19.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.216+0000 7fb1c9662740 1 -- 172.21.15.103:0/3015699469 shutdown_connections 2024-09-19T07:02:19.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.216+0000 7fb1c9662740 1 -- 172.21.15.103:0/3015699469 wait complete. 2024-09-19T07:02:19.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.216+0000 7fb1c9662740 20 client.19908 trim_cache size 0 max 16384 2024-09-19T07:02:19.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c9662740 1 Processor -- start 2024-09-19T07:02:19.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c9662740 1 -- start start 2024-09-19T07:02:19.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c9662740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e43d5d0 0x558e3e4542b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c9662740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43eb40 0x558e3e439700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c9662740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e448370 0x558e3e439c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c9662740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c01afa50 con 0x558e3e43eb40 2024-09-19T07:02:19.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c9662740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c01b1590 con 0x558e3e448370 2024-09-19T07:02:19.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c9662740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c01b0120 con 0x558e3e43d5d0 2024-09-19T07:02:19.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c4824640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43eb40 0x558e3e439700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:19.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c4824640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43eb40 0x558e3e439700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47652/0 (socket says 172.21.15.103:47652) 2024-09-19T07:02:19.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c4824640 1 -- 172.21.15.103:0/1873235041 learned_addr learned my addr 172.21.15.103:0/1873235041 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:19.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c5025640 1 --2- 172.21.15.103:0/1873235041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e43d5d0 0x558e3e4542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:19.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c4824640 1 -- 172.21.15.103:0/1873235041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e43d5d0 msgr2=0x558e3e4542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:19.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c4824640 1 --2- 172.21.15.103:0/1873235041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e43d5d0 0x558e3e4542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c4824640 1 -- 172.21.15.103:0/1873235041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e448370 msgr2=0x558e3e439c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:19.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c4824640 1 --2- 172.21.15.103:0/1873235041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e448370 0x558e3e439c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c4824640 1 -- 172.21.15.103:0/1873235041 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e3e43a180 con 0x558e3e43eb40 2024-09-19T07:02:19.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.229+0000 7fb1c4824640 1 --2- 172.21.15.103:0/1873235041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43eb40 0x558e3e439700 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fb1b009be10 tx=0x7fb1b006bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900e95139a827172 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:19.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.230+0000 7fb1aaffd640 1 -- 172.21.15.103:0/1873235041 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b001d040 con 0x558e3e43eb40 2024-09-19T07:02:19.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.230+0000 7fb1aaffd640 1 -- 172.21.15.103:0/1873235041 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1b00a04c0 con 0x558e3e43eb40 2024-09-19T07:02:19.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.230+0000 7fb1aaffd640 1 -- 172.21.15.103:0/1873235041 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b00bc1d0 con 0x558e3e43eb40 2024-09-19T07:02:19.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.230+0000 7fb1c9662740 1 -- 172.21.15.103:0/1873235041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43eb40 msgr2=0x558e3e439700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:19.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.230+0000 7fb1c9662740 1 --2- 172.21.15.103:0/1873235041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43eb40 0x558e3e439700 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fb1b009be10 tx=0x7fb1b006bd30 comp rx=0 tx=0).stop 2024-09-19T07:02:19.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.230+0000 7fb1c9662740 1 -- 172.21.15.103:0/1873235041 shutdown_connections 2024-09-19T07:02:19.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.230+0000 7fb1c9662740 1 --2- 172.21.15.103:0/1873235041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e448370 0x558e3e439c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.230+0000 7fb1c9662740 1 --2- 172.21.15.103:0/1873235041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e43eb40 0x558e3e439700 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.230+0000 7fb1c9662740 1 --2- 172.21.15.103:0/1873235041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e43d5d0 0x558e3e4542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.230+0000 7fb1c9662740 1 -- 172.21.15.103:0/1873235041 >> 172.21.15.103:0/1873235041 conn(0x558e3e406310 msgr2=0x558e3e440310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:19.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.230+0000 7fb1c9662740 1 -- 172.21.15.103:0/1873235041 shutdown_connections 2024-09-19T07:02:19.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.230+0000 7fb1c9662740 1 -- 172.21.15.103:0/1873235041 wait complete. 2024-09-19T07:02:19.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.230+0000 7fb1c9662740 1 -- start start 2024-09-19T07:02:19.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c9662740 1 Processor -- start 2024-09-19T07:02:19.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c9662740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e43d5d0 0x558e3e4509a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c9662740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e43eb40 0x558e3e44bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c9662740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e448370 0x558e3e44c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c9662740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c01b0ec0 con 0x558e3e448370 2024-09-19T07:02:19.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c9662740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c01b07f0 con 0x558e3e43d5d0 2024-09-19T07:02:19.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c9662740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c01ada50 con 0x558e3e43eb40 2024-09-19T07:02:19.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c9662740 10 client.0 _ll_register_callbacks cb 0x7fb1b800ad10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:19.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c5826640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e448370 0x558e3e44c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:19.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c5826640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e448370 0x558e3e44c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47660/0 (socket says 172.21.15.103:47660) 2024-09-19T07:02:19.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c5826640 1 -- 172.21.15.103:0/2556838343 learned_addr learned my addr 172.21.15.103:0/2556838343 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:19.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c5025640 1 --2- 172.21.15.103:0/2556838343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e43d5d0 0x558e3e4509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:19.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c4824640 1 --2- 172.21.15.103:0/2556838343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e43eb40 0x558e3e44bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:19.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c5826640 1 -- 172.21.15.103:0/2556838343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e43eb40 msgr2=0x558e3e44bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:19.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c5826640 1 --2- 172.21.15.103:0/2556838343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e43eb40 0x558e3e44bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c5826640 1 -- 172.21.15.103:0/2556838343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e43d5d0 msgr2=0x558e3e4509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:19.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c5826640 1 --2- 172.21.15.103:0/2556838343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e43d5d0 0x558e3e4509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c5826640 1 -- 172.21.15.103:0/2556838343 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e3e44c850 con 0x558e3e448370 2024-09-19T07:02:19.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.231+0000 7fb1c5025640 1 --2- 172.21.15.103:0/2556838343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e43d5d0 0x558e3e4509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:19.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.232+0000 7fb1c5826640 1 --2- 172.21.15.103:0/2556838343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e448370 0x558e3e44c210 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800fab0 tx=0x7fb1b800ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d9c04d005aaae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:19.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.232+0000 7fb1a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:19.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.232+0000 7fb1a8ff9640 1 -- 172.21.15.103:0/2556838343 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8022070 con 0x558e3e448370 2024-09-19T07:02:19.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.232+0000 7fb1a8ff9640 1 -- 172.21.15.103:0/2556838343 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7fb1b800dd10 con 0x558e3e448370 2024-09-19T07:02:19.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.232+0000 7fb1c9662740 1 -- 172.21.15.103:0/2556838343 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1b00a04c0 con 0x558e3e448370 2024-09-19T07:02:19.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.232+0000 7fb1a8ff9640 1 -- 172.21.15.103:0/2556838343 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb1b8027460 con 0x558e3e448370 2024-09-19T07:02:19.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.232+0000 7fb1c9662740 10 client.19914 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:19.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.232+0000 7fb1c9662740 1 -- 172.21.15.103:0/2556838343 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e3e4c25e0 con 0x558e3e448370 2024-09-19T07:02:19.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.232+0000 7fb1a8ff9640 1 -- 172.21.15.103:0/2556838343 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1b00a04c0 con 0x558e3e448370 2024-09-19T07:02:19.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.232+0000 7fb1a8ff9640 1 -- 172.21.15.103:0/2556838343 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb1b801e640 con 0x558e3e448370 2024-09-19T07:02:19.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.232+0000 7fb1c9662740 10 client.19914 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:19.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.232+0000 7fb1c9662740 10 client.19914 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:02:19.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.232+0000 7fb1c9662740 20 client.19914 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:19.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.232+0000 7fb1c9662740 1 --2- 172.21.15.103:0/2556838343 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x558e3e4c2dc0 0x558e3e4c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.232+0000 7fb1c9662740 4 client.19914 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:02:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.232+0000 7fb1c9662740 1 -- 172.21.15.103:0/2556838343 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e3e3aad70 con 0x558e3e4c2dc0 2024-09-19T07:02:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1c5025640 1 --2- 172.21.15.103:0/2556838343 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x558e3e4c2dc0 0x558e3e4c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1c5025640 1 --2- 172.21.15.103:0/2556838343 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x558e3e4c2dc0 0x558e3e4c51e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5174e6445da0f774 server_cookie=53f27749d722b619 in_seq=0 out_seq=0 2024-09-19T07:02:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1a8ff9640 10 client.19914 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:02:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1a8ff9640 1 -- 172.21.15.103:0/2556838343 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e3e3aad70 con 0x558e3e4c2dc0 2024-09-19T07:02:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1a8ff9640 10 client.19914 handle_command_reply: tid=0 2024-09-19T07:02:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1c9662740 1 client.19914 shutdown 2024-09-19T07:02:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1c9662740 1 -- 172.21.15.103:0/2556838343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e448370 msgr2=0x558e3e44c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1c9662740 1 --2- 172.21.15.103:0/2556838343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e448370 0x558e3e44c210 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800fab0 tx=0x7fb1b800ac20 comp rx=0 tx=0).stop 2024-09-19T07:02:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1c9662740 1 -- 172.21.15.103:0/2556838343 shutdown_connections 2024-09-19T07:02:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1c9662740 1 --2- 172.21.15.103:0/2556838343 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x558e3e4c2dc0 0x558e3e4c51e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1c9662740 1 --2- 172.21.15.103:0/2556838343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3e448370 0x558e3e44c210 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1c9662740 1 --2- 172.21.15.103:0/2556838343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e3e43eb40 0x558e3e44bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1c9662740 1 --2- 172.21.15.103:0/2556838343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e3e43d5d0 0x558e3e4509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1c9662740 1 -- 172.21.15.103:0/2556838343 >> 172.21.15.103:0/2556838343 conn(0x558e3e406310 msgr2=0x558e3e440310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1a8ff9640 0 client.19914 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:02:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1c9662740 1 -- 172.21.15.103:0/2556838343 shutdown_connections 2024-09-19T07:02:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.233+0000 7fb1c9662740 1 -- 172.21.15.103:0/2556838343 wait complete. 2024-09-19T07:02:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.234+0000 7fb1c9662740 20 client.19914 trim_cache size 0 max 16384 2024-09-19T07:02:19.265 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:02:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.234+0000 7fb1c7289640 1 -- 172.21.15.103:0/3202043277 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb1ac0a33b0 msgr2=0x7fb1ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.234+0000 7fb1c7289640 1 --2- 172.21.15.103:0/3202043277 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb1ac0a33b0 0x7fb1ac0a5870 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800afd0 tx=0x7fb1b8005f90 comp rx=0 tx=0).stop 2024-09-19T07:02:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.234+0000 7fb1c7289640 1 -- 172.21.15.103:0/3202043277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c01036c0 msgr2=0x7fb1c0100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.234+0000 7fb1c7289640 1 --2- 172.21.15.103:0/3202043277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c01036c0 0x7fb1c0100f40 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000c890 tx=0x7fb1b000cd50 comp rx=0 tx=0).stop 2024-09-19T07:02:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.234+0000 7fb1c7289640 1 -- 172.21.15.103:0/3202043277 shutdown_connections 2024-09-19T07:02:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.234+0000 7fb1c7289640 1 --2- 172.21.15.103:0/3202043277 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb1ac0a33b0 0x7fb1ac0a5870 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.234+0000 7fb1c7289640 1 --2- 172.21.15.103:0/3202043277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c010c180 0x7fb1c0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.234+0000 7fb1c7289640 1 --2- 172.21.15.103:0/3202043277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c01036c0 0x7fb1c0100f40 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.234+0000 7fb1c7289640 1 --2- 172.21.15.103:0/3202043277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c0102da0 0x7fb1c0100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:19.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.234+0000 7fb1c7289640 1 -- 172.21.15.103:0/3202043277 >> 172.21.15.103:0/3202043277 conn(0x7fb1c00fc7a0 msgr2=0x7fb1c00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:19.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.234+0000 7fb1c7289640 1 -- 172.21.15.103:0/3202043277 shutdown_connections 2024-09-19T07:02:19.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:19.234+0000 7fb1c7289640 1 -- 172.21.15.103:0/3202043277 wait complete. 2024-09-19T07:02:19.932 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:02:20.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:19 smithi174 ceph-mon[35637]: pgmap v409: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-19T07:02:20.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:19 smithi103 ceph-mon[31158]: pgmap v409: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-19T07:02:20.183 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:20.186 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:19 smithi134 ceph-mon[35356]: pgmap v409: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-19T07:02:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.418+0000 7ff7c5017640 1 -- 172.21.15.103:0/3180282477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c0101390 msgr2=0x7ff7c010d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.418+0000 7ff7c5017640 1 --2- 172.21.15.103:0/3180282477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c0101390 0x7ff7c010d8c0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009cd0 tx=0x7ff7b802a8e0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.418+0000 7ff7c5017640 1 -- 172.21.15.103:0/3180282477 shutdown_connections 2024-09-19T07:02:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.418+0000 7ff7c5017640 1 --2- 172.21.15.103:0/3180282477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c0101390 0x7ff7c010d8c0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.418+0000 7ff7c5017640 1 --2- 172.21.15.103:0/3180282477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c01009f0 0x7ff7c0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.418+0000 7ff7c5017640 1 --2- 172.21.15.103:0/3180282477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c0106a00 0x7ff7c0106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.418+0000 7ff7c5017640 1 -- 172.21.15.103:0/3180282477 >> 172.21.15.103:0/3180282477 conn(0x7ff7c00fc6b0 msgr2=0x7ff7c00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.418+0000 7ff7c5017640 1 -- 172.21.15.103:0/3180282477 shutdown_connections 2024-09-19T07:02:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.418+0000 7ff7c5017640 1 -- 172.21.15.103:0/3180282477 wait complete. 2024-09-19T07:02:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.419+0000 7ff7c5017640 1 Processor -- start 2024-09-19T07:02:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.419+0000 7ff7c5017640 1 -- start start 2024-09-19T07:02:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.419+0000 7ff7c5017640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c01009f0 0x7ff7c0199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.419+0000 7ff7c5017640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c0101390 0x7ff7c0199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.419+0000 7ff7c5017640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c0106a00 0x7ff7c01a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.419+0000 7ff7c5017640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0110e40 con 0x7ff7c0101390 2024-09-19T07:02:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.419+0000 7ff7c5017640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0110cc0 con 0x7ff7c0106a00 2024-09-19T07:02:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.419+0000 7ff7c5017640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0110fc0 con 0x7ff7c01009f0 2024-09-19T07:02:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.419+0000 7ff7be59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c0101390 0x7ff7c0199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.419+0000 7ff7be59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c0101390 0x7ff7c0199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47688/0 (socket says 172.21.15.103:47688) 2024-09-19T07:02:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.419+0000 7ff7be59c640 1 -- 172.21.15.103:0/3516589275 learned_addr learned my addr 172.21.15.103:0/3516589275 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.419+0000 7ff7bf59e640 1 --2- 172.21.15.103:0/3516589275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c0106a00 0x7ff7c01a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.420+0000 7ff7bed9d640 1 --2- 172.21.15.103:0/3516589275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c01009f0 0x7ff7c0199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.420+0000 7ff7be59c640 1 -- 172.21.15.103:0/3516589275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c01009f0 msgr2=0x7ff7c0199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.420+0000 7ff7be59c640 1 --2- 172.21.15.103:0/3516589275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c01009f0 0x7ff7c0199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.420+0000 7ff7be59c640 1 -- 172.21.15.103:0/3516589275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c0106a00 msgr2=0x7ff7c01a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.420+0000 7ff7be59c640 1 --2- 172.21.15.103:0/3516589275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c0106a00 0x7ff7c01a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.420+0000 7ff7be59c640 1 -- 172.21.15.103:0/3516589275 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c01a47d0 con 0x7ff7c0101390 2024-09-19T07:02:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.420+0000 7ff7bf59e640 1 --2- 172.21.15.103:0/3516589275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c0106a00 0x7ff7c01a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.420+0000 7ff7be59c640 1 --2- 172.21.15.103:0/3516589275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c0101390 0x7ff7c0199750 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400b4a0 tx=0x7ff7b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.420+0000 7ff7a7fff640 1 -- 172.21.15.103:0/3516589275 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4014070 con 0x7ff7c0101390 2024-09-19T07:02:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.420+0000 7ff7c5017640 1 -- 172.21.15.103:0/3516589275 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c01a49a0 con 0x7ff7c0101390 2024-09-19T07:02:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.420+0000 7ff7a7fff640 1 -- 172.21.15.103:0/3516589275 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b4010af0 con 0x7ff7c0101390 2024-09-19T07:02:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.420+0000 7ff7a7fff640 1 -- 172.21.15.103:0/3516589275 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4017d10 con 0x7ff7c0101390 2024-09-19T07:02:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.420+0000 7ff7c5017640 1 -- 172.21.15.103:0/3516589275 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c01a4f10 con 0x7ff7c0101390 2024-09-19T07:02:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.423+0000 7ff7a7fff640 1 -- 172.21.15.103:0/3516589275 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff7b4017070 con 0x7ff7c0101390 2024-09-19T07:02:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.423+0000 7ff7c7e88740 1 Processor -- start 2024-09-19T07:02:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.423+0000 7ff7c7e88740 1 -- start start 2024-09-19T07:02:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.423+0000 7ff7c7e88740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33803af0 0x564a33803ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.423+0000 7ff7c7e88740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a33804410 0x564a3380e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.423+0000 7ff7c7e88740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a3380ecb0 0x564a338110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.423+0000 7ff7c7e88740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a337682f0 con 0x564a33803af0 2024-09-19T07:02:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.423+0000 7ff7c7e88740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564a33710270 con 0x564a3380ecb0 2024-09-19T07:02:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.423+0000 7ff7c7e88740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564a337642b0 con 0x564a33804410 2024-09-19T07:02:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.424+0000 7ff7a7fff640 1 --2- 172.21.15.103:0/3516589275 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7a80a3460 0x7ff7a80a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.424+0000 7ff7bf59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33803af0 0x564a33803ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.424+0000 7ff7bf59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33803af0 0x564a33803ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47700/0 (socket says 172.21.15.103:47700) 2024-09-19T07:02:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.424+0000 7ff7bf59e640 1 -- 172.21.15.103:0/2438401421 learned_addr learned my addr 172.21.15.103:0/2438401421 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.424+0000 7ff7a7fff640 1 -- 172.21.15.103:0/3516589275 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7ff7b40ceff0 con 0x7ff7c0101390 2024-09-19T07:02:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.424+0000 7ff7bf59e640 1 --2- 172.21.15.103:0/3516589275 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7a80a3460 0x7ff7a80a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.424+0000 7ff7bed9d640 1 --2- 172.21.15.103:0/2438401421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a33804410 0x564a3380e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.424+0000 7ff7be59c640 1 --2- 172.21.15.103:0/2438401421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a3380ecb0 0x564a338110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.424+0000 7ff7bf59e640 1 -- 172.21.15.103:0/2438401421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a33804410 msgr2=0x564a3380e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.424+0000 7ff7bf59e640 1 --2- 172.21.15.103:0/2438401421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a33804410 0x564a3380e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.424+0000 7ff7bf59e640 1 -- 172.21.15.103:0/2438401421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a3380ecb0 msgr2=0x564a338110a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.424+0000 7ff7bf59e640 1 --2- 172.21.15.103:0/2438401421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a3380ecb0 0x564a338110a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.424+0000 7ff7bf59e640 1 -- 172.21.15.103:0/2438401421 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a3370f340 con 0x564a33803af0 2024-09-19T07:02:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.425+0000 7ff7bf59e640 1 --2- 172.21.15.103:0/2438401421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33803af0 0x564a33803ed0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8031d00 tx=0x7ff7b8031d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c351336602c96652 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.425+0000 7ff7bf59e640 1 --2- 172.21.15.103:0/3516589275 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7a80a3460 0x7ff7a80a5920 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8031da0 tx=0x7ff7b8048000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.425+0000 7ff7a57fa640 1 -- 172.21.15.103:0/2438401421 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b804e070 con 0x564a33803af0 2024-09-19T07:02:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.425+0000 7ff7a57fa640 1 -- 172.21.15.103:0/2438401421 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b8036040 con 0x564a33803af0 2024-09-19T07:02:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.425+0000 7ff7a57fa640 1 -- 172.21.15.103:0/2438401421 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b80307e0 con 0x564a33803af0 2024-09-19T07:02:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.425+0000 7ff7c7e88740 1 -- 172.21.15.103:0/2438401421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33803af0 msgr2=0x564a33803ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.425+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/2438401421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33803af0 0x564a33803ed0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8031d00 tx=0x7ff7b8031d30 comp rx=0 tx=0).stop 2024-09-19T07:02:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7c7e88740 1 -- 172.21.15.103:0/2438401421 shutdown_connections 2024-09-19T07:02:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/2438401421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a3380ecb0 0x564a338110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/2438401421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a33804410 0x564a3380e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/2438401421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33803af0 0x564a33803ed0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7c7e88740 1 -- 172.21.15.103:0/2438401421 >> 172.21.15.103:0/2438401421 conn(0x564a33800060 msgr2=0x564a33800440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7c7e88740 1 -- 172.21.15.103:0/2438401421 shutdown_connections 2024-09-19T07:02:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7c7e88740 1 -- 172.21.15.103:0/2438401421 wait complete. 2024-09-19T07:02:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7c7e88740 1 -- start start 2024-09-19T07:02:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7c7e88740 1 Processor -- start 2024-09-19T07:02:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7c7e88740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a33803af0 0x564a3380bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7c7e88740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33804410 0x564a3380c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7c7e88740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a3380ecb0 0x564a3380c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7c7e88740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a33815c90 con 0x564a33804410 2024-09-19T07:02:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7c7e88740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564a336f1290 con 0x564a3380ecb0 2024-09-19T07:02:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7c7e88740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564a33815e10 con 0x564a33803af0 2024-09-19T07:02:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7c7e88740 10 client.0 _ll_register_callbacks cb 0x564a33722140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.426+0000 7ff7be59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33804410 0x564a3380c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7be59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33804410 0x564a3380c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47710/0 (socket says 172.21.15.103:47710) 2024-09-19T07:02:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7be59c640 1 -- 172.21.15.103:0/2241700891 learned_addr learned my addr 172.21.15.103:0/2241700891 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7bed9d640 1 --2- 172.21.15.103:0/2241700891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a33803af0 0x564a3380bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7bf59e640 1 --2- 172.21.15.103:0/2241700891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a3380ecb0 0x564a3380c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7be59c640 1 -- 172.21.15.103:0/2241700891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a33803af0 msgr2=0x564a3380bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7be59c640 1 --2- 172.21.15.103:0/2241700891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a33803af0 0x564a3380bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7be59c640 1 -- 172.21.15.103:0/2241700891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a3380ecb0 msgr2=0x564a3380c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7be59c640 1 --2- 172.21.15.103:0/2241700891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a3380ecb0 0x564a3380c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7be59c640 1 -- 172.21.15.103:0/2241700891 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a3347b210 con 0x564a33804410 2024-09-19T07:02:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7bf59e640 1 --2- 172.21.15.103:0/2241700891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a3380ecb0 0x564a3380c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7be59c640 1 --2- 172.21.15.103:0/2241700891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33804410 0x564a3380c120 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7ff7b409b2f0 tx=0x7ff7b409ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f89b0f9935e7edd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7a2ff5640 1 -- 172.21.15.103:0/2241700891 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b401e050 con 0x564a33804410 2024-09-19T07:02:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7c7e88740 1 -- 172.21.15.103:0/2241700891 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7b8036040 con 0x564a33804410 2024-09-19T07:02:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7a2ff5640 1 -- 172.21.15.103:0/2241700891 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7ff7b40b42d0 con 0x564a33804410 2024-09-19T07:02:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7a2ff5640 1 -- 172.21.15.103:0/2241700891 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff7b40a3fd0 con 0x564a33804410 2024-09-19T07:02:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7c7e88740 10 client.19926 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.427+0000 7ff7c7e88740 1 -- 172.21.15.103:0/2241700891 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564a33547620 con 0x564a33804410 2024-09-19T07:02:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.428+0000 7ff7a2ff5640 1 -- 172.21.15.103:0/2241700891 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b8036040 con 0x564a33804410 2024-09-19T07:02:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.428+0000 7ff7a2ff5640 1 -- 172.21.15.103:0/2241700891 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff7b409a7b0 con 0x564a33804410 2024-09-19T07:02:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.428+0000 7ff7c7e88740 10 client.19926 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.428+0000 7ff7c7e88740 10 client.19926 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.428+0000 7ff7c7e88740 20 client.19926 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.428+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/2241700891 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a33823050 0x564a33825440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.428+0000 7ff7c7e88740 4 client.19926 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.428+0000 7ff7c7e88740 1 -- 172.21.15.103:0/2241700891 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a33775a20 con 0x564a33823050 2024-09-19T07:02:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.428+0000 7ff7bed9d640 1 --2- 172.21.15.103:0/2241700891 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a33823050 0x564a33825440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.429+0000 7ff7a2ff5640 10 client.19926 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.429+0000 7ff7bed9d640 1 --2- 172.21.15.103:0/2241700891 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a33823050 0x564a33825440 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c22c0187a5841f54 server_cookie=2e7da8a8bda1ad31 in_seq=0 out_seq=0 2024-09-19T07:02:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.429+0000 7ff7a2ff5640 1 -- 172.21.15.103:0/2241700891 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a33775a20 con 0x564a33823050 2024-09-19T07:02:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.429+0000 7ff7a2ff5640 10 client.19926 handle_command_reply: tid=0 2024-09-19T07:02:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.430+0000 7ff7c7e88740 1 client.19926 shutdown 2024-09-19T07:02:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.430+0000 7ff7c7e88740 1 -- 172.21.15.103:0/2241700891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33804410 msgr2=0x564a3380c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.430+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/2241700891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33804410 0x564a3380c120 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7ff7b409b2f0 tx=0x7ff7b409ba60 comp rx=0 tx=0).stop 2024-09-19T07:02:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.430+0000 7ff7c7e88740 1 -- 172.21.15.103:0/2241700891 shutdown_connections 2024-09-19T07:02:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.430+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/2241700891 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a33823050 0x564a33825440 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.430+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/2241700891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a3380ecb0 0x564a3380c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.430+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/2241700891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33804410 0x564a3380c120 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.430+0000 7ff7a2ff5640 0 client.19926 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.430+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/2241700891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a33803af0 0x564a3380bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.430+0000 7ff7c7e88740 1 -- 172.21.15.103:0/2241700891 >> 172.21.15.103:0/2241700891 conn(0x564a33800060 msgr2=0x564a337ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.430+0000 7ff7c7e88740 1 -- 172.21.15.103:0/2241700891 shutdown_connections 2024-09-19T07:02:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.430+0000 7ff7c7e88740 1 -- 172.21.15.103:0/2241700891 wait complete. 2024-09-19T07:02:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.430+0000 7ff7c7e88740 20 client.19926 trim_cache size 0 max 16384 2024-09-19T07:02:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.444+0000 7ff7c7e88740 1 Processor -- start 2024-09-19T07:02:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7c7e88740 1 -- start start 2024-09-19T07:02:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7c7e88740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a33803af0 0x564a3380c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7c7e88740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a33804410 0x564a3382acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7c7e88740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a3380ecb0 0x564a3382b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7c7e88740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c01ab710 con 0x564a3380ecb0 2024-09-19T07:02:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7c7e88740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c01ad250 con 0x564a33803af0 2024-09-19T07:02:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7c7e88740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c01abde0 con 0x564a33804410 2024-09-19T07:02:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7bf59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a3380ecb0 0x564a3382b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7bf59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a3380ecb0 0x564a3382b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47712/0 (socket says 172.21.15.103:47712) 2024-09-19T07:02:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7bf59e640 1 -- 172.21.15.103:0/3596752145 learned_addr learned my addr 172.21.15.103:0/3596752145 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7be59c640 1 --2- 172.21.15.103:0/3596752145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a33804410 0x564a3382acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7bed9d640 1 --2- 172.21.15.103:0/3596752145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a33803af0 0x564a3380c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7bf59e640 1 -- 172.21.15.103:0/3596752145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a33804410 msgr2=0x564a3382acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7bf59e640 1 --2- 172.21.15.103:0/3596752145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a33804410 0x564a3382acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7bf59e640 1 -- 172.21.15.103:0/3596752145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a33803af0 msgr2=0x564a3380c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7bf59e640 1 --2- 172.21.15.103:0/3596752145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a33803af0 0x564a3380c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7bf59e640 1 -- 172.21.15.103:0/3596752145 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a337fa2f0 con 0x564a3380ecb0 2024-09-19T07:02:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7bed9d640 1 --2- 172.21.15.103:0/3596752145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a33803af0 0x564a3380c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7bf59e640 1 --2- 172.21.15.103:0/3596752145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a3380ecb0 0x564a3382b200 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8035000 tx=0x7ff7b8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07f827704a201c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.445+0000 7ff7a4ff9640 1 -- 172.21.15.103:0/3596752145 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8057030 con 0x564a3380ecb0 2024-09-19T07:02:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.446+0000 7ff7a4ff9640 1 -- 172.21.15.103:0/3596752145 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b8031e50 con 0x564a3380ecb0 2024-09-19T07:02:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.446+0000 7ff7a4ff9640 1 -- 172.21.15.103:0/3596752145 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b804e040 con 0x564a3380ecb0 2024-09-19T07:02:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.446+0000 7ff7c7e88740 1 -- 172.21.15.103:0/3596752145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a3380ecb0 msgr2=0x564a3382b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.446+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/3596752145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a3380ecb0 0x564a3382b200 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8035000 tx=0x7ff7b8005ef0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.446+0000 7ff7c7e88740 1 -- 172.21.15.103:0/3596752145 shutdown_connections 2024-09-19T07:02:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.446+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/3596752145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a3380ecb0 0x564a3382b200 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.446+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/3596752145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a33804410 0x564a3382acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.446+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/3596752145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a33803af0 0x564a3380c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.446+0000 7ff7c7e88740 1 -- 172.21.15.103:0/3596752145 >> 172.21.15.103:0/3596752145 conn(0x564a33800060 msgr2=0x564a3380dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.446+0000 7ff7c7e88740 1 -- 172.21.15.103:0/3596752145 shutdown_connections 2024-09-19T07:02:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.446+0000 7ff7c7e88740 1 -- 172.21.15.103:0/3596752145 wait complete. 2024-09-19T07:02:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7c7e88740 1 -- start start 2024-09-19T07:02:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7c7e88740 1 Processor -- start 2024-09-19T07:02:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7c7e88740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33803af0 0x564a3381b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7c7e88740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a33804410 0x564a3381b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7c7e88740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a3380ecb0 0x564a33818070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7c7e88740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c01acb80 con 0x564a33803af0 2024-09-19T07:02:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7c7e88740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c01ac4b0 con 0x564a33804410 2024-09-19T07:02:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7c7e88740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c01a96f0 con 0x564a3380ecb0 2024-09-19T07:02:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7c7e88740 10 client.0 _ll_register_callbacks cb 0x7ff7b4017f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7bed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33803af0 0x564a3381b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7bed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33803af0 0x564a3381b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47728/0 (socket says 172.21.15.103:47728) 2024-09-19T07:02:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7bed9d640 1 -- 172.21.15.103:0/1717830926 learned_addr learned my addr 172.21.15.103:0/1717830926 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7bed9d640 1 -- 172.21.15.103:0/1717830926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a3380ecb0 msgr2=0x564a33818070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7bed9d640 1 --2- 172.21.15.103:0/1717830926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a3380ecb0 0x564a33818070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7bed9d640 1 -- 172.21.15.103:0/1717830926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a33804410 msgr2=0x564a3381b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7bed9d640 1 --2- 172.21.15.103:0/1717830926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a33804410 0x564a3381b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7bed9d640 1 -- 172.21.15.103:0/1717830926 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a3381bc70 con 0x564a33803af0 2024-09-19T07:02:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7bed9d640 1 --2- 172.21.15.103:0/1717830926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33803af0 0x564a3381b0a0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x564a33825c00 tx=0x7ff7b000bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d81e109bdb4f3e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.447+0000 7ff7a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.448+0000 7ff7a2ff5640 1 -- 172.21.15.103:0/1717830926 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b002a060 con 0x564a33803af0 2024-09-19T07:02:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.448+0000 7ff7a2ff5640 1 -- 172.21.15.103:0/1717830926 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7ff7b0011440 con 0x564a33803af0 2024-09-19T07:02:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.448+0000 7ff7c7e88740 1 -- 172.21.15.103:0/1717830926 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7b8031e50 con 0x564a33803af0 2024-09-19T07:02:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.448+0000 7ff7a2ff5640 1 -- 172.21.15.103:0/1717830926 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff7b0011640 con 0x564a33803af0 2024-09-19T07:02:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.448+0000 7ff7c7e88740 10 client.19932 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.448+0000 7ff7c7e88740 1 -- 172.21.15.103:0/1717830926 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564a3388e210 con 0x564a33803af0 2024-09-19T07:02:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.448+0000 7ff7a2ff5640 1 -- 172.21.15.103:0/1717830926 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b8031e50 con 0x564a33803af0 2024-09-19T07:02:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.448+0000 7ff7a2ff5640 1 -- 172.21.15.103:0/1717830926 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff7b0024030 con 0x564a33803af0 2024-09-19T07:02:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.448+0000 7ff7c7e88740 10 client.19932 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.448+0000 7ff7c7e88740 10 client.19932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.448+0000 7ff7c7e88740 20 client.19932 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.448+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/1717830926 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a3388e990 0x564a33890df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.448+0000 7ff7c7e88740 4 client.19932 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:02:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.448+0000 7ff7c7e88740 1 -- 172.21.15.103:0/1717830926 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a3376fd70 con 0x564a3388e990 2024-09-19T07:02:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.448+0000 7ff7be59c640 1 --2- 172.21.15.103:0/1717830926 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a3388e990 0x564a33890df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.449+0000 7ff7a2ff5640 10 client.19932 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.449+0000 7ff7be59c640 1 --2- 172.21.15.103:0/1717830926 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a3388e990 0x564a33890df0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c1decb8f0cb8428 server_cookie=5a214e9898da6647 in_seq=0 out_seq=0 2024-09-19T07:02:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.465+0000 7ff7a2ff5640 1 -- 172.21.15.103:0/1717830926 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a3376fd70 con 0x564a3388e990 2024-09-19T07:02:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.465+0000 7ff7a2ff5640 10 client.19932 handle_command_reply: tid=0 2024-09-19T07:02:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.465+0000 7ff7c7e88740 1 client.19932 shutdown 2024-09-19T07:02:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.465+0000 7ff7c7e88740 1 -- 172.21.15.103:0/1717830926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33803af0 msgr2=0x564a3381b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.465+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/1717830926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33803af0 0x564a3381b0a0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x564a33825c00 tx=0x7ff7b000bd10 comp rx=0 tx=0).stop 2024-09-19T07:02:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.465+0000 7ff7c7e88740 1 -- 172.21.15.103:0/1717830926 shutdown_connections 2024-09-19T07:02:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.465+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/1717830926 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a3388e990 0x564a33890df0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.465+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/1717830926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a3380ecb0 0x564a33818070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.465+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/1717830926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a33804410 0x564a3381b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.465+0000 7ff7a2ff5640 0 client.19932 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.465+0000 7ff7c7e88740 1 --2- 172.21.15.103:0/1717830926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a33803af0 0x564a3381b0a0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.465+0000 7ff7c7e88740 1 -- 172.21.15.103:0/1717830926 >> 172.21.15.103:0/1717830926 conn(0x564a33800060 msgr2=0x564a33805550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.465+0000 7ff7c7e88740 1 -- 172.21.15.103:0/1717830926 shutdown_connections 2024-09-19T07:02:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.465+0000 7ff7c7e88740 1 -- 172.21.15.103:0/1717830926 wait complete. 2024-09-19T07:02:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.465+0000 7ff7c7e88740 20 client.19932 trim_cache size 0 max 16384 2024-09-19T07:02:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:02:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:02:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:02:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:02:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.466+0000 7ff7c5017640 1 -- 172.21.15.103:0/3516589275 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7a80a3460 msgr2=0x7ff7a80a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.466+0000 7ff7c5017640 1 --2- 172.21.15.103:0/3516589275 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7a80a3460 0x7ff7a80a5920 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8031da0 tx=0x7ff7b8048000 comp rx=0 tx=0).stop 2024-09-19T07:02:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.466+0000 7ff7c5017640 1 -- 172.21.15.103:0/3516589275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c0101390 msgr2=0x7ff7c0199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.466+0000 7ff7c5017640 1 --2- 172.21.15.103:0/3516589275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c0101390 0x7ff7c0199750 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400b4a0 tx=0x7ff7b400b960 comp rx=0 tx=0).stop 2024-09-19T07:02:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.466+0000 7ff7c5017640 1 -- 172.21.15.103:0/3516589275 shutdown_connections 2024-09-19T07:02:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.466+0000 7ff7c5017640 1 --2- 172.21.15.103:0/3516589275 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7a80a3460 0x7ff7a80a5920 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.466+0000 7ff7c5017640 1 --2- 172.21.15.103:0/3516589275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c0106a00 0x7ff7c01a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.466+0000 7ff7c5017640 1 --2- 172.21.15.103:0/3516589275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c0101390 0x7ff7c0199750 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.466+0000 7ff7c5017640 1 --2- 172.21.15.103:0/3516589275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c01009f0 0x7ff7c0199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.466+0000 7ff7c5017640 1 -- 172.21.15.103:0/3516589275 >> 172.21.15.103:0/3516589275 conn(0x7ff7c00fc6b0 msgr2=0x7ff7c00fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.466+0000 7ff7c5017640 1 -- 172.21.15.103:0/3516589275 shutdown_connections 2024-09-19T07:02:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:20.466+0000 7ff7c5017640 1 -- 172.21.15.103:0/3516589275 wait complete. 2024-09-19T07:02:20.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.647+0000 7f96b7727640 1 -- 172.21.15.103:0/1872244678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b01008e0 msgr2=0x7f96b0100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.647+0000 7f96b7727640 1 --2- 172.21.15.103:0/1872244678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b01008e0 0x7f96b0100cc0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009930 tx=0x7f96a002a420 comp rx=0 tx=0).stop 2024-09-19T07:02:20.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.647+0000 7f96b7727640 1 -- 172.21.15.103:0/1872244678 shutdown_connections 2024-09-19T07:02:20.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.647+0000 7f96b7727640 1 --2- 172.21.15.103:0/1872244678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96b0108160 0x7f96b010eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.647+0000 7f96b7727640 1 --2- 172.21.15.103:0/1872244678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b0101290 0x7f96b0107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.647+0000 7f96b7727640 1 --2- 172.21.15.103:0/1872244678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b01008e0 0x7f96b0100cc0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.647+0000 7f96b7727640 1 -- 172.21.15.103:0/1872244678 >> 172.21.15.103:0/1872244678 conn(0x7f96b0077210 msgr2=0x7f96b0077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:20.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.647+0000 7f96b7727640 1 -- 172.21.15.103:0/1872244678 shutdown_connections 2024-09-19T07:02:20.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.647+0000 7f96b7727640 1 -- 172.21.15.103:0/1872244678 wait complete. 2024-09-19T07:02:20.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.647+0000 7f96b7727640 1 Processor -- start 2024-09-19T07:02:20.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.647+0000 7f96b7727640 1 -- start start 2024-09-19T07:02:20.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.647+0000 7f96b7727640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b01008e0 0x7f96b019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.647+0000 7f96b7727640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96b0101290 0x7f96b019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b7727640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b0108160 0x7f96b019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b7727640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96b0112050 con 0x7f96b0108160 2024-09-19T07:02:20.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b7727640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96b0111ed0 con 0x7f96b0101290 2024-09-19T07:02:20.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b7727640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01121d0 con 0x7f96b01008e0 2024-09-19T07:02:20.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b5cc4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b0108160 0x7f96b019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b5cc4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b0108160 0x7f96b019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47734/0 (socket says 172.21.15.103:47734) 2024-09-19T07:02:20.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b5cc4640 1 -- 172.21.15.103:0/4040083046 learned_addr learned my addr 172.21.15.103:0/4040083046 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:20.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b54c3640 1 --2- 172.21.15.103:0/4040083046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b01008e0 0x7f96b019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b4cc2640 1 --2- 172.21.15.103:0/4040083046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96b0101290 0x7f96b019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b5cc4640 1 -- 172.21.15.103:0/4040083046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b01008e0 msgr2=0x7f96b019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b5cc4640 1 --2- 172.21.15.103:0/4040083046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b01008e0 0x7f96b019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b5cc4640 1 -- 172.21.15.103:0/4040083046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96b0101290 msgr2=0x7f96b019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b5cc4640 1 --2- 172.21.15.103:0/4040083046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96b0101290 0x7f96b019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b5cc4640 1 -- 172.21.15.103:0/4040083046 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b019f650 con 0x7f96b0108160 2024-09-19T07:02:20.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b5cc4640 1 --2- 172.21.15.103:0/4040083046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b0108160 0x7f96b019ef70 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f96a800b970 tx=0x7f96a800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:20.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96ae7fc640 1 -- 172.21.15.103:0/4040083046 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8019070 con 0x7f96b0108160 2024-09-19T07:02:20.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96ae7fc640 1 -- 172.21.15.103:0/4040083046 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a800cb40 con 0x7f96b0108160 2024-09-19T07:02:20.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b7727640 1 -- 172.21.15.103:0/4040083046 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b019f940 con 0x7f96b0108160 2024-09-19T07:02:20.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96ae7fc640 1 -- 172.21.15.103:0/4040083046 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8007650 con 0x7f96b0108160 2024-09-19T07:02:20.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.648+0000 7f96b7727640 1 -- 172.21.15.103:0/4040083046 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b01a7220 con 0x7f96b0108160 2024-09-19T07:02:20.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.651+0000 7f96ae7fc640 1 -- 172.21.15.103:0/4040083046 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f96a801c070 con 0x7f96b0108160 2024-09-19T07:02:20.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.651+0000 7f96b9b00740 1 Processor -- start 2024-09-19T07:02:20.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.651+0000 7f96b9b00740 1 -- start start 2024-09-19T07:02:20.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b9b00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a4b40 0x55c8989a3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b9b00740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989a35d0 0x55c8989a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b9b00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989ae370 0x55c8989b0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b9b00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c8989092f0 con 0x55c8989a4b40 2024-09-19T07:02:20.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b9b00740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c8988b0cf0 con 0x55c8989a35d0 2024-09-19T07:02:20.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b9b00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c8989052b0 con 0x55c8989ae370 2024-09-19T07:02:20.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b4cc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a4b40 0x55c8989a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b4cc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a4b40 0x55c8989a3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47736/0 (socket says 172.21.15.103:47736) 2024-09-19T07:02:20.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b4cc2640 1 -- 172.21.15.103:0/902209353 learned_addr learned my addr 172.21.15.103:0/902209353 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:20.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b54c3640 1 --2- 172.21.15.103:0/902209353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989a35d0 0x55c8989a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96ae7fc640 1 --2- 172.21.15.103:0/4040083046 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96980a33b0 0x7f96980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b4cc2640 1 -- 172.21.15.103:0/902209353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989ae370 msgr2=0x55c8989b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:20.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b4cc2640 1 --2- 172.21.15.103:0/902209353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989ae370 0x55c8989b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b4cc2640 1 -- 172.21.15.103:0/902209353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989a35d0 msgr2=0x55c8989a3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:20.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b4cc2640 1 --2- 172.21.15.103:0/902209353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989a35d0 0x55c8989a3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b4cc2640 1 -- 172.21.15.103:0/902209353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8988953a0 con 0x55c8989a4b40 2024-09-19T07:02:20.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b5cc4640 1 --2- 172.21.15.103:0/4040083046 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96980a33b0 0x7f96980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96b4cc2640 1 --2- 172.21.15.103:0/902209353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a4b40 0x55c8989a3090 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f96a400cc70 tx=0x7f96a400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeecfacad4d9204d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:20.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.652+0000 7f96ae7fc640 1 -- 172.21.15.103:0/4040083046 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f96a80cf280 con 0x7f96b0108160 2024-09-19T07:02:20.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.653+0000 7f96977fe640 1 -- 172.21.15.103:0/902209353 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4005220 con 0x55c8989a4b40 2024-09-19T07:02:20.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.653+0000 7f96977fe640 1 -- 172.21.15.103:0/902209353 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a4004c80 con 0x55c8989a4b40 2024-09-19T07:02:20.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.653+0000 7f96977fe640 1 -- 172.21.15.103:0/902209353 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a40107f0 con 0x55c8989a4b40 2024-09-19T07:02:20.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.653+0000 7f96b9b00740 1 -- 172.21.15.103:0/902209353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a4b40 msgr2=0x55c8989a3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.653+0000 7f96b5cc4640 1 --2- 172.21.15.103:0/4040083046 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96980a33b0 0x7f96980a5870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f96a80a3d30 tx=0x7f96a806d420 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:20.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.653+0000 7f96b9b00740 1 --2- 172.21.15.103:0/902209353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a4b40 0x55c8989a3090 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f96a400cc70 tx=0x7f96a400e720 comp rx=0 tx=0).stop 2024-09-19T07:02:20.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.653+0000 7f96b9b00740 1 -- 172.21.15.103:0/902209353 shutdown_connections 2024-09-19T07:02:20.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.653+0000 7f96b9b00740 1 --2- 172.21.15.103:0/902209353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989ae370 0x55c8989b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.653+0000 7f96b9b00740 1 --2- 172.21.15.103:0/902209353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989a35d0 0x55c8989a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.653+0000 7f96b9b00740 1 --2- 172.21.15.103:0/902209353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a4b40 0x55c8989a3090 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.653+0000 7f96b9b00740 1 -- 172.21.15.103:0/902209353 >> 172.21.15.103:0/902209353 conn(0x55c89896c310 msgr2=0x55c89899fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:20.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.653+0000 7f96b9b00740 1 -- 172.21.15.103:0/902209353 shutdown_connections 2024-09-19T07:02:20.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.653+0000 7f96b9b00740 1 -- 172.21.15.103:0/902209353 wait complete. 2024-09-19T07:02:20.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.654+0000 7f96b9b00740 1 -- start start 2024-09-19T07:02:20.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.654+0000 7f96b9b00740 1 Processor -- start 2024-09-19T07:02:20.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.654+0000 7f96b9b00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c8989add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.654+0000 7f96b9b00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989a4b40 0x55c8989aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.654+0000 7f96b9b00740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989ae370 0x55c8989ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.654+0000 7f96b9b00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c8989b1070 con 0x55c8989a35d0 2024-09-19T07:02:20.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.654+0000 7f96b9b00740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c8988c4060 con 0x55c8989ae370 2024-09-19T07:02:20.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.654+0000 7f96b9b00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c8989b11f0 con 0x55c8989a4b40 2024-09-19T07:02:20.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.654+0000 7f96b9b00740 10 client.0 _ll_register_callbacks cb 0x55c8986178c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:20.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.654+0000 7f96b4cc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c8989add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.654+0000 7f96b4cc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c8989add30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47744/0 (socket says 172.21.15.103:47744) 2024-09-19T07:02:20.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.654+0000 7f96b4cc2640 1 -- 172.21.15.103:0/2754068230 learned_addr learned my addr 172.21.15.103:0/2754068230 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:20.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.655+0000 7f96b54c3640 1 --2- 172.21.15.103:0/2754068230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989a4b40 0x55c8989aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.655+0000 7f96b4cc2640 1 --2- 172.21.15.103:0/2754068230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989ae370 0x55c8989ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.655+0000 7f96b4cc2640 1 -- 172.21.15.103:0/2754068230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989a4b40 msgr2=0x55c8989aafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:20.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.655+0000 7f96b4cc2640 1 --2- 172.21.15.103:0/2754068230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989a4b40 0x55c8989aafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.655+0000 7f96b4cc2640 1 -- 172.21.15.103:0/2754068230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989ae370 msgr2=0x55c8989ab5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:20.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.655+0000 7f96b4cc2640 1 --2- 172.21.15.103:0/2754068230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989ae370 0x55c8989ab5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.655+0000 7f96b4cc2640 1 -- 172.21.15.103:0/2754068230 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c89867a3c0 con 0x55c8989a35d0 2024-09-19T07:02:20.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.655+0000 7f96b4cc2640 1 --2- 172.21.15.103:0/2754068230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c8989add30 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c370 tx=0x7f96a4004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61df6cab1f329711 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:20.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.655+0000 7f9694ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:20.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.655+0000 7f9694ff9640 1 -- 172.21.15.103:0/2754068230 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4010730 con 0x55c8989a35d0 2024-09-19T07:02:20.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.655+0000 7f9694ff9640 1 -- 172.21.15.103:0/2754068230 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f96a40108d0 con 0x55c8989a35d0 2024-09-19T07:02:20.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.655+0000 7f96b9b00740 1 -- 172.21.15.103:0/2754068230 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96a4004c80 con 0x55c8989a35d0 2024-09-19T07:02:20.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.655+0000 7f9694ff9640 1 -- 172.21.15.103:0/2754068230 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f96a4010ad0 con 0x55c8989a35d0 2024-09-19T07:02:20.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.655+0000 7f96b9b00740 10 client.19944 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:20.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.655+0000 7f96b9b00740 1 -- 172.21.15.103:0/2754068230 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c8989bcad0 con 0x55c8989a35d0 2024-09-19T07:02:20.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.655+0000 7f9694ff9640 1 -- 172.21.15.103:0/2754068230 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a4004c80 con 0x55c8989a35d0 2024-09-19T07:02:20.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.656+0000 7f9694ff9640 1 -- 172.21.15.103:0/2754068230 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f96a402c4f0 con 0x55c8989a35d0 2024-09-19T07:02:20.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.656+0000 7f96b9b00740 10 client.19944 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:20.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.656+0000 7f96b9b00740 10 client.19944 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:02:20.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.656+0000 7f96b9b00740 20 client.19944 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:20.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.656+0000 7f96b9b00740 1 --2- 172.21.15.103:0/2754068230 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55c8989bde20 0x55c8989c0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.656+0000 7f96b9b00740 4 client.19944 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:20.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.656+0000 7f96b9b00740 1 -- 172.21.15.103:0/2754068230 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c898916860 con 0x55c8989bde20 2024-09-19T07:02:20.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.656+0000 7f96b54c3640 1 --2- 172.21.15.103:0/2754068230 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55c8989bde20 0x55c8989c0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.656+0000 7f9694ff9640 10 client.19944 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:02:20.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.656+0000 7f96b54c3640 1 --2- 172.21.15.103:0/2754068230 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55c8989bde20 0x55c8989c0210 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cced5fb8b49dbfd server_cookie=4b1ed47afda5b035 in_seq=0 out_seq=0 2024-09-19T07:02:20.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.657+0000 7f9694ff9640 1 -- 172.21.15.103:0/2754068230 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c898916860 con 0x55c8989bde20 2024-09-19T07:02:20.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.657+0000 7f9694ff9640 10 client.19944 handle_command_reply: tid=0 2024-09-19T07:02:20.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.657+0000 7f96b9b00740 1 client.19944 shutdown 2024-09-19T07:02:20.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.657+0000 7f96b9b00740 1 -- 172.21.15.103:0/2754068230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 msgr2=0x55c8989add30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.657+0000 7f96b9b00740 1 --2- 172.21.15.103:0/2754068230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c8989add30 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c370 tx=0x7f96a4004e70 comp rx=0 tx=0).stop 2024-09-19T07:02:20.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.657+0000 7f96b9b00740 1 -- 172.21.15.103:0/2754068230 shutdown_connections 2024-09-19T07:02:20.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.657+0000 7f96b9b00740 1 --2- 172.21.15.103:0/2754068230 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55c8989bde20 0x55c8989c0210 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.657+0000 7f96b9b00740 1 --2- 172.21.15.103:0/2754068230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989ae370 0x55c8989ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.657+0000 7f96b9b00740 1 --2- 172.21.15.103:0/2754068230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989a4b40 0x55c8989aafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.657+0000 7f96b9b00740 1 --2- 172.21.15.103:0/2754068230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c8989add30 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.657+0000 7f96b9b00740 1 -- 172.21.15.103:0/2754068230 >> 172.21.15.103:0/2754068230 conn(0x55c89896c310 msgr2=0x55c8989a5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:20.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.657+0000 7f9694ff9640 0 client.19944 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:02:20.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.657+0000 7f96b9b00740 1 -- 172.21.15.103:0/2754068230 shutdown_connections 2024-09-19T07:02:20.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.658+0000 7f96b9b00740 1 -- 172.21.15.103:0/2754068230 wait complete. 2024-09-19T07:02:20.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.658+0000 7f96b9b00740 20 client.19944 trim_cache size 0 max 16384 2024-09-19T07:02:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.670+0000 7f96b9b00740 1 Processor -- start 2024-09-19T07:02:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.670+0000 7f96b9b00740 1 -- start start 2024-09-19T07:02:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.670+0000 7f96b9b00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c89899ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.670+0000 7f96b9b00740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989a4b40 0x55c8989a6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.670+0000 7f96b9b00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989ae370 0x55c8989a6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.670+0000 7f96b9b00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01afa30 con 0x55c8989a35d0 2024-09-19T07:02:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.670+0000 7f96b9b00740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01b1570 con 0x55c8989a4b40 2024-09-19T07:02:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.670+0000 7f96b9b00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01b0100 con 0x55c8989ae370 2024-09-19T07:02:20.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.671+0000 7f96b4cc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c89899ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.671+0000 7f96b4cc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c89899ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47750/0 (socket says 172.21.15.103:47750) 2024-09-19T07:02:20.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.671+0000 7f96b4cc2640 1 -- 172.21.15.103:0/1642919988 learned_addr learned my addr 172.21.15.103:0/1642919988 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:20.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.671+0000 7f96b54c3640 1 --2- 172.21.15.103:0/1642919988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989a4b40 0x55c8989a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.671+0000 7f96b4cc2640 1 -- 172.21.15.103:0/1642919988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989ae370 msgr2=0x55c8989a6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:20.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.671+0000 7f96b4cc2640 1 --2- 172.21.15.103:0/1642919988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989ae370 0x55c8989a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.671+0000 7f96b4cc2640 1 -- 172.21.15.103:0/1642919988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989a4b40 msgr2=0x55c8989a6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:20.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.671+0000 7f96b4cc2640 1 --2- 172.21.15.103:0/1642919988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989a4b40 0x55c8989a6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.671+0000 7f96b4cc2640 1 -- 172.21.15.103:0/1642919988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8989a0960 con 0x55c8989a35d0 2024-09-19T07:02:20.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.671+0000 7f96b4cc2640 1 --2- 172.21.15.103:0/1642919988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c89899ec10 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f96a400d010 tx=0x7f96a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0786c432249851 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:20.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.671+0000 7f9696ffd640 1 -- 172.21.15.103:0/1642919988 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4036070 con 0x55c8989a35d0 2024-09-19T07:02:20.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.671+0000 7f9696ffd640 1 -- 172.21.15.103:0/1642919988 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a401f030 con 0x55c8989a35d0 2024-09-19T07:02:20.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.671+0000 7f9696ffd640 1 -- 172.21.15.103:0/1642919988 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a402cd90 con 0x55c8989a35d0 2024-09-19T07:02:20.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.672+0000 7f96b9b00740 1 -- 172.21.15.103:0/1642919988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 msgr2=0x55c89899ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.672+0000 7f96b9b00740 1 --2- 172.21.15.103:0/1642919988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c89899ec10 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f96a400d010 tx=0x7f96a40057f0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.672+0000 7f96b9b00740 1 -- 172.21.15.103:0/1642919988 shutdown_connections 2024-09-19T07:02:20.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.672+0000 7f96b9b00740 1 --2- 172.21.15.103:0/1642919988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989ae370 0x55c8989a6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.672+0000 7f96b9b00740 1 --2- 172.21.15.103:0/1642919988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989a4b40 0x55c8989a6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.672+0000 7f96b9b00740 1 --2- 172.21.15.103:0/1642919988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c89899ec10 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.672+0000 7f96b9b00740 1 -- 172.21.15.103:0/1642919988 >> 172.21.15.103:0/1642919988 conn(0x55c89896c310 msgr2=0x55c89899e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:20.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.672+0000 7f96b9b00740 1 -- 172.21.15.103:0/1642919988 shutdown_connections 2024-09-19T07:02:20.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.672+0000 7f96b9b00740 1 -- 172.21.15.103:0/1642919988 wait complete. 2024-09-19T07:02:20.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.672+0000 7f96b9b00740 1 -- start start 2024-09-19T07:02:20.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.672+0000 7f96b9b00740 1 Processor -- start 2024-09-19T07:02:20.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b9b00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c8989c06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b9b00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989a4b40 0x55c8989c0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b9b00740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989ae370 0x55c898a1f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b9b00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01b0ea0 con 0x55c8989a35d0 2024-09-19T07:02:20.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b9b00740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01b07d0 con 0x55c8989ae370 2024-09-19T07:02:20.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b9b00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01ada30 con 0x55c8989a4b40 2024-09-19T07:02:20.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b9b00740 10 client.0 _ll_register_callbacks cb 0x7f96a4014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:20.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b4cc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c8989c06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b4cc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c8989c06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47760/0 (socket says 172.21.15.103:47760) 2024-09-19T07:02:20.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b4cc2640 1 -- 172.21.15.103:0/3945470525 learned_addr learned my addr 172.21.15.103:0/3945470525 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:20.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b54c3640 1 --2- 172.21.15.103:0/3945470525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989a4b40 0x55c8989c0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b4cc2640 1 --2- 172.21.15.103:0/3945470525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989ae370 0x55c898a1f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b4cc2640 1 -- 172.21.15.103:0/3945470525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989a4b40 msgr2=0x55c8989c0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:20.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b4cc2640 1 --2- 172.21.15.103:0/3945470525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989a4b40 0x55c8989c0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b4cc2640 1 -- 172.21.15.103:0/3945470525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989ae370 msgr2=0x55c898a1f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:20.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b4cc2640 1 --2- 172.21.15.103:0/3945470525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989ae370 0x55c898a1f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b4cc2640 1 -- 172.21.15.103:0/3945470525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c898a1fdb0 con 0x55c8989a35d0 2024-09-19T07:02:20.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f96b4cc2640 1 --2- 172.21.15.103:0/3945470525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c8989c06b0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c9c0 tx=0x7f96a400efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4897c797a703895a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:20.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f9694ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:20.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.673+0000 7f9694ff9640 1 -- 172.21.15.103:0/3945470525 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4036030 con 0x55c8989a35d0 2024-09-19T07:02:20.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.674+0000 7f9694ff9640 1 -- 172.21.15.103:0/3945470525 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f96a4004b10 con 0x55c8989a35d0 2024-09-19T07:02:20.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.674+0000 7f96b9b00740 1 -- 172.21.15.103:0/3945470525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96a401f030 con 0x55c8989a35d0 2024-09-19T07:02:20.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.674+0000 7f9694ff9640 1 -- 172.21.15.103:0/3945470525 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f96a4004d10 con 0x55c8989a35d0 2024-09-19T07:02:20.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.674+0000 7f96b9b00740 10 client.19950 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:20.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.674+0000 7f96b9b00740 1 -- 172.21.15.103:0/3945470525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c898a28320 con 0x55c8989a35d0 2024-09-19T07:02:20.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.674+0000 7f9694ff9640 1 -- 172.21.15.103:0/3945470525 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a401f030 con 0x55c8989a35d0 2024-09-19T07:02:20.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.674+0000 7f9694ff9640 1 -- 172.21.15.103:0/3945470525 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f96a402a430 con 0x55c8989a35d0 2024-09-19T07:02:20.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.674+0000 7f96b9b00740 10 client.19950 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:20.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.674+0000 7f96b9b00740 10 client.19950 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:02:20.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.674+0000 7f96b9b00740 20 client.19950 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:20.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.674+0000 7f96b9b00740 1 --2- 172.21.15.103:0/3945470525 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55c898a289d0 0x55c898a2adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:20.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.674+0000 7f96b9b00740 4 client.19950 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:02:20.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.674+0000 7f96b9b00740 1 -- 172.21.15.103:0/3945470525 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c898910d70 con 0x55c898a289d0 2024-09-19T07:02:20.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.674+0000 7f96b54c3640 1 --2- 172.21.15.103:0/3945470525 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55c898a289d0 0x55c898a2adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:20.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f96b54c3640 1 --2- 172.21.15.103:0/3945470525 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55c898a289d0 0x55c898a2adc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53ea34181a5cb39c server_cookie=e20404d0e3a3cfbd in_seq=0 out_seq=0 2024-09-19T07:02:20.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f9694ff9640 10 client.19950 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:02:20.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f9694ff9640 1 -- 172.21.15.103:0/3945470525 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c898910d70 con 0x55c898a289d0 2024-09-19T07:02:20.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f9694ff9640 10 client.19950 handle_command_reply: tid=0 2024-09-19T07:02:20.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f96b9b00740 1 client.19950 shutdown 2024-09-19T07:02:20.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f96b9b00740 1 -- 172.21.15.103:0/3945470525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 msgr2=0x55c8989c06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f96b9b00740 1 --2- 172.21.15.103:0/3945470525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c8989c06b0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c9c0 tx=0x7f96a400efb0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f96b9b00740 1 -- 172.21.15.103:0/3945470525 shutdown_connections 2024-09-19T07:02:20.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f96b9b00740 1 --2- 172.21.15.103:0/3945470525 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55c898a289d0 0x55c898a2adc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f96b9b00740 1 --2- 172.21.15.103:0/3945470525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c8989ae370 0x55c898a1f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f96b9b00740 1 --2- 172.21.15.103:0/3945470525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8989a4b40 0x55c8989c0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f96b9b00740 1 --2- 172.21.15.103:0/3945470525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8989a35d0 0x55c8989c06b0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f96b9b00740 1 -- 172.21.15.103:0/3945470525 >> 172.21.15.103:0/3945470525 conn(0x55c89896c310 msgr2=0x55c89899ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:20.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f9694ff9640 0 client.19950 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:02:20.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f96b9b00740 1 -- 172.21.15.103:0/3945470525 shutdown_connections 2024-09-19T07:02:20.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f96b9b00740 1 -- 172.21.15.103:0/3945470525 wait complete. 2024-09-19T07:02:20.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.675+0000 7f96b9b00740 20 client.19950 trim_cache size 0 max 16384 2024-09-19T07:02:20.682 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:02:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.676+0000 7f96b7727640 1 -- 172.21.15.103:0/4040083046 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96980a33b0 msgr2=0x7f96980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.676+0000 7f96b7727640 1 --2- 172.21.15.103:0/4040083046 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96980a33b0 0x7f96980a5870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f96a80a3d30 tx=0x7f96a806d420 comp rx=0 tx=0).stop 2024-09-19T07:02:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.676+0000 7f96b7727640 1 -- 172.21.15.103:0/4040083046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b0108160 msgr2=0x7f96b019ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.676+0000 7f96b7727640 1 --2- 172.21.15.103:0/4040083046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b0108160 0x7f96b019ef70 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f96a800b970 tx=0x7f96a800be30 comp rx=0 tx=0).stop 2024-09-19T07:02:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.676+0000 7f96b7727640 1 -- 172.21.15.103:0/4040083046 shutdown_connections 2024-09-19T07:02:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.676+0000 7f96b7727640 1 --2- 172.21.15.103:0/4040083046 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f96980a33b0 0x7f96980a5870 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.676+0000 7f96b7727640 1 --2- 172.21.15.103:0/4040083046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96b0108160 0x7f96b019ef70 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.676+0000 7f96b7727640 1 --2- 172.21.15.103:0/4040083046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96b0101290 0x7f96b019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.676+0000 7f96b7727640 1 --2- 172.21.15.103:0/4040083046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b01008e0 0x7f96b019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.676+0000 7f96b7727640 1 -- 172.21.15.103:0/4040083046 >> 172.21.15.103:0/4040083046 conn(0x7f96b0077210 msgr2=0x7f96b00ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:20.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.676+0000 7f96b7727640 1 -- 172.21.15.103:0/4040083046 shutdown_connections 2024-09-19T07:02:20.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:20.676+0000 7f96b7727640 1 -- 172.21.15.103:0/4040083046 wait complete. 2024-09-19T07:02:21.340 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:02:21.585 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:22.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:21 smithi103 ceph-mon[31158]: pgmap v410: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-09-19T07:02:22.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.006+0000 7f4b350a5640 1 -- 172.21.15.103:0/3088505257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b30101de0 msgr2=0x7f4b301021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:22.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.006+0000 7f4b350a5640 1 --2- 172.21.15.103:0/3088505257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b30101de0 0x7f4b301021c0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009900 tx=0x7f4b2002a410 comp rx=0 tx=0).stop 2024-09-19T07:02:22.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.006+0000 7f4b350a5640 1 -- 172.21.15.103:0/3088505257 shutdown_connections 2024-09-19T07:02:22.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.006+0000 7f4b350a5640 1 --2- 172.21.15.103:0/3088505257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b30106f10 0x7f4b3010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.006+0000 7f4b350a5640 1 --2- 172.21.15.103:0/3088505257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b30102790 0x7f4b301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.006+0000 7f4b350a5640 1 --2- 172.21.15.103:0/3088505257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b30101de0 0x7f4b301021c0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.006+0000 7f4b350a5640 1 -- 172.21.15.103:0/3088505257 >> 172.21.15.103:0/3088505257 conn(0x7f4b300fd8c0 msgr2=0x7f4b300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:22.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.006+0000 7f4b350a5640 1 -- 172.21.15.103:0/3088505257 shutdown_connections 2024-09-19T07:02:22.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.006+0000 7f4b350a5640 1 -- 172.21.15.103:0/3088505257 wait complete. 2024-09-19T07:02:22.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b350a5640 1 Processor -- start 2024-09-19T07:02:22.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b350a5640 1 -- start start 2024-09-19T07:02:22.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b350a5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b30101de0 0x7f4b301985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b350a5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b30102790 0x7f4b30198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b350a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b30106f10 0x7f4b3019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b350a5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30112010 con 0x7f4b30101de0 2024-09-19T07:02:22.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b350a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30111e90 con 0x7f4b30106f10 2024-09-19T07:02:22.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b350a5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30112190 con 0x7f4b30102790 2024-09-19T07:02:22.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b2ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b30101de0 0x7f4b301985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:22.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b2ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b30101de0 0x7f4b301985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47778/0 (socket says 172.21.15.103:47778) 2024-09-19T07:02:22.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b2ed9d640 1 -- 172.21.15.103:0/3328004662 learned_addr learned my addr 172.21.15.103:0/3328004662 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:22.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b2ed9d640 1 -- 172.21.15.103:0/3328004662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b30102790 msgr2=0x7f4b30198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:22.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b2e59c640 1 --2- 172.21.15.103:0/3328004662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b30102790 0x7f4b30198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:22.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b2f59e640 1 --2- 172.21.15.103:0/3328004662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b30106f10 0x7f4b3019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:22.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b2ed9d640 1 --2- 172.21.15.103:0/3328004662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b30102790 0x7f4b30198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b2ed9d640 1 -- 172.21.15.103:0/3328004662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b30106f10 msgr2=0x7f4b3019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:22.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b2ed9d640 1 --2- 172.21.15.103:0/3328004662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b30106f10 0x7f4b3019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b2ed9d640 1 -- 172.21.15.103:0/3328004662 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3019d560 con 0x7f4b30101de0 2024-09-19T07:02:22.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.007+0000 7f4b2e59c640 1 --2- 172.21.15.103:0/3328004662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b30102790 0x7f4b30198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:22.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.008+0000 7f4b2ed9d640 1 --2- 172.21.15.103:0/3328004662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b30101de0 0x7f4b301985b0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f4b20005ab0 tx=0x7f4b200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:22.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.008+0000 7f4b17fff640 1 -- 172.21.15.103:0/3328004662 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2003f030 con 0x7f4b30101de0 2024-09-19T07:02:22.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.008+0000 7f4b350a5640 1 -- 172.21.15.103:0/3328004662 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3019d7f0 con 0x7f4b30101de0 2024-09-19T07:02:22.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.008+0000 7f4b17fff640 1 -- 172.21.15.103:0/3328004662 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b20003ce0 con 0x7f4b30101de0 2024-09-19T07:02:22.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.008+0000 7f4b17fff640 1 -- 172.21.15.103:0/3328004662 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20035070 con 0x7f4b30101de0 2024-09-19T07:02:22.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.008+0000 7f4b350a5640 1 -- 172.21.15.103:0/3328004662 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b301a5130 con 0x7f4b30101de0 2024-09-19T07:02:22.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.010+0000 7f4b17fff640 1 -- 172.21.15.103:0/3328004662 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4b20042030 con 0x7f4b30101de0 2024-09-19T07:02:22.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.010+0000 7f4b3747e740 1 Processor -- start 2024-09-19T07:02:22.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.010+0000 7f4b3747e740 1 -- start start 2024-09-19T07:02:22.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b3747e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c63b40 0x55bf75c62090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b3747e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c625d0 0x55bf75c62a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b3747e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c6d370 0x55bf75c6f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b3747e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bf75bc82f0 con 0x55bf75c63b40 2024-09-19T07:02:22.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b3747e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bf75b6fcf0 con 0x55bf75c625d0 2024-09-19T07:02:22.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b3747e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf75bc42b0 con 0x55bf75c6d370 2024-09-19T07:02:22.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b2f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c63b40 0x55bf75c62090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:22.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b2f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c63b40 0x55bf75c62090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47786/0 (socket says 172.21.15.103:47786) 2024-09-19T07:02:22.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b2f59e640 1 -- 172.21.15.103:0/3483546802 learned_addr learned my addr 172.21.15.103:0/3483546802 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:22.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b17fff640 1 --2- 172.21.15.103:0/3328004662 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4b180a33b0 0x7f4b180a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b2ed9d640 1 --2- 172.21.15.103:0/3483546802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c625d0 0x55bf75c62a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:22.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b2f59e640 1 --2- 172.21.15.103:0/3328004662 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4b180a33b0 0x7f4b180a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:22.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b2e59c640 1 --2- 172.21.15.103:0/3483546802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c6d370 0x55bf75c6f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:22.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b17fff640 1 -- 172.21.15.103:0/3328004662 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f4b200f0720 con 0x7f4b30101de0 2024-09-19T07:02:22.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b2f59e640 1 -- 172.21.15.103:0/3483546802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c6d370 msgr2=0x55bf75c6f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:22.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b2f59e640 1 --2- 172.21.15.103:0/3483546802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c6d370 0x55bf75c6f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b2f59e640 1 -- 172.21.15.103:0/3483546802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c625d0 msgr2=0x55bf75c62a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:22.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b2f59e640 1 --2- 172.21.15.103:0/3483546802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c625d0 0x55bf75c62a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.011+0000 7f4b2f59e640 1 -- 172.21.15.103:0/3483546802 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf75b543a0 con 0x55bf75c63b40 2024-09-19T07:02:22.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.012+0000 7f4b2f59e640 1 --2- 172.21.15.103:0/3483546802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c63b40 0x55bf75c62090 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f4b28013ca0 tx=0x7f4b28015710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497e4878dd9ac08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:22.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.012+0000 7f4b2f59e640 1 --2- 172.21.15.103:0/3328004662 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4b180a33b0 0x7f4b180a5870 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4b28015c10 tx=0x7f4b28019000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:22.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.012+0000 7f4b157fa640 1 -- 172.21.15.103:0/3483546802 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28024070 con 0x55bf75c63b40 2024-09-19T07:02:22.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.012+0000 7f4b157fa640 1 -- 172.21.15.103:0/3483546802 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2800c030 con 0x55bf75c63b40 2024-09-19T07:02:22.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.012+0000 7f4b157fa640 1 -- 172.21.15.103:0/3483546802 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2801fb90 con 0x55bf75c63b40 2024-09-19T07:02:22.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.012+0000 7f4b3747e740 1 -- 172.21.15.103:0/3483546802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c63b40 msgr2=0x55bf75c62090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:22.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.012+0000 7f4b3747e740 1 --2- 172.21.15.103:0/3483546802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c63b40 0x55bf75c62090 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f4b28013ca0 tx=0x7f4b28015710 comp rx=0 tx=0).stop 2024-09-19T07:02:22.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.012+0000 7f4b3747e740 1 -- 172.21.15.103:0/3483546802 shutdown_connections 2024-09-19T07:02:22.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b3747e740 1 --2- 172.21.15.103:0/3483546802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c6d370 0x55bf75c6f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b3747e740 1 --2- 172.21.15.103:0/3483546802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c625d0 0x55bf75c62a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b3747e740 1 --2- 172.21.15.103:0/3483546802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c63b40 0x55bf75c62090 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b3747e740 1 -- 172.21.15.103:0/3483546802 >> 172.21.15.103:0/3483546802 conn(0x55bf75c2b310 msgr2=0x55bf75c5ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:22.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b3747e740 1 -- 172.21.15.103:0/3483546802 shutdown_connections 2024-09-19T07:02:22.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b3747e740 1 -- 172.21.15.103:0/3483546802 wait complete. 2024-09-19T07:02:22.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b3747e740 1 -- start start 2024-09-19T07:02:22.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b3747e740 1 Processor -- start 2024-09-19T07:02:22.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b3747e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c625d0 0x55bf75c6cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b3747e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c63b40 0x55bf75c6a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b3747e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c6d370 0x55bf75c6a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b3747e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bf75c70010 con 0x55bf75c6d370 2024-09-19T07:02:22.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b3747e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bf75b83060 con 0x55bf75c625d0 2024-09-19T07:02:22.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b3747e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf75c70190 con 0x55bf75c63b40 2024-09-19T07:02:22.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b3747e740 10 client.0 _ll_register_callbacks cb 0x55bf758d68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:22.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b2f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c6d370 0x55bf75c6a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:22.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b2f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c6d370 0x55bf75c6a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47798/0 (socket says 172.21.15.103:47798) 2024-09-19T07:02:22.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b2f59e640 1 -- 172.21.15.103:0/800169136 learned_addr learned my addr 172.21.15.103:0/800169136 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:22.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.013+0000 7f4b2ed9d640 1 --2- 172.21.15.103:0/800169136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c625d0 0x55bf75c6cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:22.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.014+0000 7f4b2e59c640 1 --2- 172.21.15.103:0/800169136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c63b40 0x55bf75c6a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:22.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.014+0000 7f4b2f59e640 1 -- 172.21.15.103:0/800169136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c63b40 msgr2=0x55bf75c6a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:22.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.014+0000 7f4b2f59e640 1 --2- 172.21.15.103:0/800169136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c63b40 0x55bf75c6a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.014+0000 7f4b2f59e640 1 -- 172.21.15.103:0/800169136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c625d0 msgr2=0x55bf75c6cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:22.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.014+0000 7f4b2f59e640 1 --2- 172.21.15.103:0/800169136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c625d0 0x55bf75c6cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.014+0000 7f4b2f59e640 1 -- 172.21.15.103:0/800169136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf759393c0 con 0x55bf75c6d370 2024-09-19T07:02:22.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.014+0000 7f4b2f59e640 1 --2- 172.21.15.103:0/800169136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c6d370 0x55bf75c6a800 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f4b28017cd0 tx=0x7f4b28020710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b77d334e1b7dc35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:22.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.014+0000 7f4b12ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:22.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.014+0000 7f4b12ff5640 1 -- 172.21.15.103:0/800169136 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2802f020 con 0x55bf75c6d370 2024-09-19T07:02:22.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.014+0000 7f4b12ff5640 1 -- 172.21.15.103:0/800169136 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f4b28020d10 con 0x55bf75c6d370 2024-09-19T07:02:22.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.014+0000 7f4b3747e740 1 -- 172.21.15.103:0/800169136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b2800c030 con 0x55bf75c6d370 2024-09-19T07:02:22.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.014+0000 7f4b12ff5640 1 -- 172.21.15.103:0/800169136 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b2803a050 con 0x55bf75c6d370 2024-09-19T07:02:22.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.014+0000 7f4b3747e740 10 client.19962 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:22.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.014+0000 7f4b3747e740 1 -- 172.21.15.103:0/800169136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf75c7bb20 con 0x55bf75c6d370 2024-09-19T07:02:22.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.014+0000 7f4b12ff5640 1 -- 172.21.15.103:0/800169136 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2800c030 con 0x55bf75c6d370 2024-09-19T07:02:22.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.014+0000 7f4b12ff5640 1 -- 172.21.15.103:0/800169136 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4b2802bbc0 con 0x55bf75c6d370 2024-09-19T07:02:22.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.015+0000 7f4b3747e740 10 client.19962 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:22.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.015+0000 7f4b3747e740 10 client.19962 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:02:22.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.015+0000 7f4b3747e740 20 client.19962 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:22.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.015+0000 7f4b3747e740 1 --2- 172.21.15.103:0/800169136 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55bf75c7cfe0 0x55bf75c7f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.015+0000 7f4b3747e740 4 client.19962 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:22.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.015+0000 7f4b3747e740 1 -- 172.21.15.103:0/800169136 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf75bd5860 con 0x55bf75c7cfe0 2024-09-19T07:02:22.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.015+0000 7f4b2ed9d640 1 --2- 172.21.15.103:0/800169136 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55bf75c7cfe0 0x55bf75c7f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:22.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.015+0000 7f4b12ff5640 10 client.19962 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:02:22.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.015+0000 7f4b2ed9d640 1 --2- 172.21.15.103:0/800169136 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55bf75c7cfe0 0x55bf75c7f3d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=589ee973ef673908 server_cookie=ea85c73dbfa3d2f9 in_seq=0 out_seq=0 2024-09-19T07:02:22.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.016+0000 7f4b12ff5640 1 -- 172.21.15.103:0/800169136 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf75bd5860 con 0x55bf75c7cfe0 2024-09-19T07:02:22.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.016+0000 7f4b12ff5640 10 client.19962 handle_command_reply: tid=0 2024-09-19T07:02:22.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.016+0000 7f4b3747e740 1 client.19962 shutdown 2024-09-19T07:02:22.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.016+0000 7f4b3747e740 1 -- 172.21.15.103:0/800169136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c6d370 msgr2=0x55bf75c6a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:22.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.016+0000 7f4b3747e740 1 --2- 172.21.15.103:0/800169136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c6d370 0x55bf75c6a800 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f4b28017cd0 tx=0x7f4b28020710 comp rx=0 tx=0).stop 2024-09-19T07:02:22.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.016+0000 7f4b3747e740 1 -- 172.21.15.103:0/800169136 shutdown_connections 2024-09-19T07:02:22.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.016+0000 7f4b3747e740 1 --2- 172.21.15.103:0/800169136 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55bf75c7cfe0 0x55bf75c7f3d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.016+0000 7f4b3747e740 1 --2- 172.21.15.103:0/800169136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c6d370 0x55bf75c6a800 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.016+0000 7f4b12ff5640 0 client.19962 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:02:22.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.016+0000 7f4b3747e740 1 --2- 172.21.15.103:0/800169136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c63b40 0x55bf75c6a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.016+0000 7f4b3747e740 1 --2- 172.21.15.103:0/800169136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c625d0 0x55bf75c6cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.016+0000 7f4b3747e740 1 -- 172.21.15.103:0/800169136 >> 172.21.15.103:0/800169136 conn(0x55bf75c2b310 msgr2=0x55bf75c64960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:22.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.016+0000 7f4b3747e740 1 -- 172.21.15.103:0/800169136 shutdown_connections 2024-09-19T07:02:22.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.016+0000 7f4b3747e740 1 -- 172.21.15.103:0/800169136 wait complete. 2024-09-19T07:02:22.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.016+0000 7f4b3747e740 20 client.19962 trim_cache size 0 max 16384 2024-09-19T07:02:22.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.029+0000 7f4b3747e740 1 Processor -- start 2024-09-19T07:02:22.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.029+0000 7f4b3747e740 1 -- start start 2024-09-19T07:02:22.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.029+0000 7f4b3747e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c625d0 0x55bf75c69050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.029+0000 7f4b3747e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c63b40 0x55bf75c75800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.029+0000 7f4b3747e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c6d370 0x55bf75c75d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.029+0000 7f4b3747e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301ada20 con 0x55bf75c625d0 2024-09-19T07:02:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.029+0000 7f4b3747e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301af560 con 0x55bf75c63b40 2024-09-19T07:02:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.029+0000 7f4b3747e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301ae0f0 con 0x55bf75c6d370 2024-09-19T07:02:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.029+0000 7f4b2ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c625d0 0x55bf75c69050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.029+0000 7f4b2ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c625d0 0x55bf75c69050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47812/0 (socket says 172.21.15.103:47812) 2024-09-19T07:02:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.029+0000 7f4b2ed9d640 1 -- 172.21.15.103:0/2693582694 learned_addr learned my addr 172.21.15.103:0/2693582694 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.029+0000 7f4b2ed9d640 1 -- 172.21.15.103:0/2693582694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c6d370 msgr2=0x55bf75c75d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.029+0000 7f4b2ed9d640 1 --2- 172.21.15.103:0/2693582694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c6d370 0x55bf75c75d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.029+0000 7f4b2ed9d640 1 -- 172.21.15.103:0/2693582694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c63b40 msgr2=0x55bf75c75800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.029+0000 7f4b2ed9d640 1 --2- 172.21.15.103:0/2693582694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c63b40 0x55bf75c75800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.029+0000 7f4b2ed9d640 1 -- 172.21.15.103:0/2693582694 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf75c7fa00 con 0x55bf75c625d0 2024-09-19T07:02:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.030+0000 7f4b2ed9d640 1 --2- 172.21.15.103:0/2693582694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c625d0 0x55bf75c69050 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4b20005ec0 tx=0x7f4b20085f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4320622b7999ee87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.030+0000 7f4b14ff9640 1 -- 172.21.15.103:0/2693582694 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20049040 con 0x55bf75c625d0 2024-09-19T07:02:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.030+0000 7f4b14ff9640 1 -- 172.21.15.103:0/2693582694 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b200c0d30 con 0x55bf75c625d0 2024-09-19T07:02:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.030+0000 7f4b14ff9640 1 -- 172.21.15.103:0/2693582694 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b200b8ff0 con 0x55bf75c625d0 2024-09-19T07:02:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.030+0000 7f4b3747e740 1 -- 172.21.15.103:0/2693582694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c625d0 msgr2=0x55bf75c69050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.030+0000 7f4b3747e740 1 --2- 172.21.15.103:0/2693582694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c625d0 0x55bf75c69050 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4b20005ec0 tx=0x7f4b20085f90 comp rx=0 tx=0).stop 2024-09-19T07:02:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.030+0000 7f4b3747e740 1 -- 172.21.15.103:0/2693582694 shutdown_connections 2024-09-19T07:02:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.030+0000 7f4b3747e740 1 --2- 172.21.15.103:0/2693582694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c6d370 0x55bf75c75d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.030+0000 7f4b3747e740 1 --2- 172.21.15.103:0/2693582694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c63b40 0x55bf75c75800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.030+0000 7f4b3747e740 1 --2- 172.21.15.103:0/2693582694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c625d0 0x55bf75c69050 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.030+0000 7f4b3747e740 1 -- 172.21.15.103:0/2693582694 >> 172.21.15.103:0/2693582694 conn(0x55bf75c2b310 msgr2=0x55bf75c68940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.030+0000 7f4b3747e740 1 -- 172.21.15.103:0/2693582694 shutdown_connections 2024-09-19T07:02:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b3747e740 1 -- 172.21.15.103:0/2693582694 wait complete. 2024-09-19T07:02:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b3747e740 1 -- start start 2024-09-19T07:02:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b3747e740 1 Processor -- start 2024-09-19T07:02:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b3747e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c625d0 0x55bf75cdfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b3747e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c63b40 0x55bf75ce0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b3747e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c6d370 0x55bf75ce0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b3747e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301aee90 con 0x55bf75c63b40 2024-09-19T07:02:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b3747e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301ae7c0 con 0x55bf75c6d370 2024-09-19T07:02:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b3747e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301aba20 con 0x55bf75c625d0 2024-09-19T07:02:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b3747e740 10 client.0 _ll_register_callbacks cb 0x7f4b28022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b2e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c63b40 0x55bf75ce0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b2e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c63b40 0x55bf75ce0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47824/0 (socket says 172.21.15.103:47824) 2024-09-19T07:02:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b2e59c640 1 -- 172.21.15.103:0/3967744376 learned_addr learned my addr 172.21.15.103:0/3967744376 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b2e59c640 1 -- 172.21.15.103:0/3967744376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c625d0 msgr2=0x55bf75cdfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b2ed9d640 1 --2- 172.21.15.103:0/3967744376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c625d0 0x55bf75cdfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b2f59e640 1 --2- 172.21.15.103:0/3967744376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c6d370 0x55bf75ce0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.031+0000 7f4b2e59c640 1 --2- 172.21.15.103:0/3967744376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c625d0 0x55bf75cdfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b2e59c640 1 -- 172.21.15.103:0/3967744376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c6d370 msgr2=0x55bf75ce0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b2e59c640 1 --2- 172.21.15.103:0/3967744376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c6d370 0x55bf75ce0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b2e59c640 1 -- 172.21.15.103:0/3967744376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf75c5e510 con 0x55bf75c63b40 2024-09-19T07:02:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b2ed9d640 1 --2- 172.21.15.103:0/3967744376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c625d0 0x55bf75cdfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b2f59e640 1 --2- 172.21.15.103:0/3967744376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c6d370 0x55bf75ce0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b2e59c640 1 --2- 172.21.15.103:0/3967744376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c63b40 0x55bf75ce0100 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x55bf75c63070 tx=0x7f4b24011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585ce48113cc021 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b12ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b12ff5640 1 -- 172.21.15.103:0/3967744376 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24019070 con 0x55bf75c63b40 2024-09-19T07:02:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b12ff5640 1 -- 172.21.15.103:0/3967744376 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f4b24015660 con 0x55bf75c63b40 2024-09-19T07:02:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b3747e740 1 -- 172.21.15.103:0/3967744376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b200c0d30 con 0x55bf75c63b40 2024-09-19T07:02:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b12ff5640 1 -- 172.21.15.103:0/3967744376 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b24025050 con 0x55bf75c63b40 2024-09-19T07:02:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b3747e740 10 client.19968 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:22.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b3747e740 1 -- 172.21.15.103:0/3967744376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf75cebf30 con 0x55bf75c63b40 2024-09-19T07:02:22.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b12ff5640 1 -- 172.21.15.103:0/3967744376 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b200c0d30 con 0x55bf75c63b40 2024-09-19T07:02:22.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b12ff5640 1 -- 172.21.15.103:0/3967744376 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4b240148d0 con 0x55bf75c63b40 2024-09-19T07:02:22.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b3747e740 10 client.19968 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:22.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b3747e740 10 client.19968 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:02:22.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b3747e740 20 client.19968 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:22.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b3747e740 1 --2- 172.21.15.103:0/3967744376 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55bf75cec6e0 0x55bf75ceeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:22.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b3747e740 4 client.19968 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:02:22.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.032+0000 7f4b3747e740 1 -- 172.21.15.103:0/3967744376 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf75bcfd70 con 0x55bf75cec6e0 2024-09-19T07:02:22.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.033+0000 7f4b2ed9d640 1 --2- 172.21.15.103:0/3967744376 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55bf75cec6e0 0x55bf75ceeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:22.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.033+0000 7f4b2ed9d640 1 --2- 172.21.15.103:0/3967744376 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55bf75cec6e0 0x55bf75ceeb00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8119286130f19f01 server_cookie=56bebece2ec832c5 in_seq=0 out_seq=0 2024-09-19T07:02:22.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.033+0000 7f4b12ff5640 10 client.19968 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:02:22.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.033+0000 7f4b12ff5640 1 -- 172.21.15.103:0/3967744376 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf75bcfd70 con 0x55bf75cec6e0 2024-09-19T07:02:22.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.033+0000 7f4b12ff5640 10 client.19968 handle_command_reply: tid=0 2024-09-19T07:02:22.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.033+0000 7f4b3747e740 1 client.19968 shutdown 2024-09-19T07:02:22.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.033+0000 7f4b3747e740 1 -- 172.21.15.103:0/3967744376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c63b40 msgr2=0x55bf75ce0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:22.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.033+0000 7f4b3747e740 1 --2- 172.21.15.103:0/3967744376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c63b40 0x55bf75ce0100 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x55bf75c63070 tx=0x7f4b24011630 comp rx=0 tx=0).stop 2024-09-19T07:02:22.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.033+0000 7f4b3747e740 1 -- 172.21.15.103:0/3967744376 shutdown_connections 2024-09-19T07:02:22.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.033+0000 7f4b3747e740 1 --2- 172.21.15.103:0/3967744376 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55bf75cec6e0 0x55bf75ceeb00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.033+0000 7f4b3747e740 1 --2- 172.21.15.103:0/3967744376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bf75c6d370 0x55bf75ce0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.033+0000 7f4b3747e740 1 --2- 172.21.15.103:0/3967744376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf75c63b40 0x55bf75ce0100 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.033+0000 7f4b3747e740 1 --2- 172.21.15.103:0/3967744376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf75c625d0 0x55bf75cdfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.033+0000 7f4b3747e740 1 -- 172.21.15.103:0/3967744376 >> 172.21.15.103:0/3967744376 conn(0x55bf75c2b310 msgr2=0x55bf75c69160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:22.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.033+0000 7f4b12ff5640 0 client.19968 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:02:22.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.034+0000 7f4b3747e740 1 -- 172.21.15.103:0/3967744376 shutdown_connections 2024-09-19T07:02:22.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.034+0000 7f4b3747e740 1 -- 172.21.15.103:0/3967744376 wait complete. 2024-09-19T07:02:22.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.034+0000 7f4b3747e740 20 client.19968 trim_cache size 0 max 16384 2024-09-19T07:02:22.052 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:02:22.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.034+0000 7f4b350a5640 1 -- 172.21.15.103:0/3328004662 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4b180a33b0 msgr2=0x7f4b180a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:22.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.034+0000 7f4b350a5640 1 --2- 172.21.15.103:0/3328004662 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4b180a33b0 0x7f4b180a5870 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4b28015c10 tx=0x7f4b28019000 comp rx=0 tx=0).stop 2024-09-19T07:02:22.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.034+0000 7f4b350a5640 1 -- 172.21.15.103:0/3328004662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b30101de0 msgr2=0x7f4b301985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:22.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.034+0000 7f4b350a5640 1 --2- 172.21.15.103:0/3328004662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b30101de0 0x7f4b301985b0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f4b20005ab0 tx=0x7f4b200042f0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.034+0000 7f4b350a5640 1 -- 172.21.15.103:0/3328004662 shutdown_connections 2024-09-19T07:02:22.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.034+0000 7f4b350a5640 1 --2- 172.21.15.103:0/3328004662 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4b180a33b0 0x7f4b180a5870 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.034+0000 7f4b350a5640 1 --2- 172.21.15.103:0/3328004662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b30106f10 0x7f4b3019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.034+0000 7f4b350a5640 1 --2- 172.21.15.103:0/3328004662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b30102790 0x7f4b30198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.034+0000 7f4b350a5640 1 --2- 172.21.15.103:0/3328004662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b30101de0 0x7f4b301985b0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:22.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.034+0000 7f4b350a5640 1 -- 172.21.15.103:0/3328004662 >> 172.21.15.103:0/3328004662 conn(0x7f4b300fd8c0 msgr2=0x7f4b300fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:22.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.034+0000 7f4b350a5640 1 -- 172.21.15.103:0/3328004662 shutdown_connections 2024-09-19T07:02:22.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:22.034+0000 7f4b350a5640 1 -- 172.21.15.103:0/3328004662 wait complete. 2024-09-19T07:02:22.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:21 smithi174 ceph-mon[35637]: pgmap v410: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-09-19T07:02:22.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:21 smithi134 ceph-mon[35356]: pgmap v410: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-09-19T07:02:22.651 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:02:22.897 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:23.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.332+0000 7f2128e63640 1 -- 172.21.15.103:0/411000383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106f10 msgr2=0x7f212410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:23.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.332+0000 7f2128e63640 1 --2- 172.21.15.103:0/411000383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106f10 0x7f212410ea90 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f2114009900 tx=0x7f211402a410 comp rx=0 tx=0).stop 2024-09-19T07:02:23.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.332+0000 7f2128e63640 1 -- 172.21.15.103:0/411000383 shutdown_connections 2024-09-19T07:02:23.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.332+0000 7f2128e63640 1 --2- 172.21.15.103:0/411000383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106f10 0x7f212410ea90 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.332+0000 7f2128e63640 1 --2- 172.21.15.103:0/411000383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2124102790 0x7f21241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.332+0000 7f2128e63640 1 --2- 172.21.15.103:0/411000383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2124101de0 0x7f21241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.332+0000 7f2128e63640 1 -- 172.21.15.103:0/411000383 >> 172.21.15.103:0/411000383 conn(0x7f21240fd8c0 msgr2=0x7f21240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:23.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.332+0000 7f2128e63640 1 -- 172.21.15.103:0/411000383 shutdown_connections 2024-09-19T07:02:23.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.332+0000 7f2128e63640 1 -- 172.21.15.103:0/411000383 wait complete. 2024-09-19T07:02:23.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.333+0000 7f2128e63640 1 Processor -- start 2024-09-19T07:02:23.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.333+0000 7f2128e63640 1 -- start start 2024-09-19T07:02:23.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.333+0000 7f2128e63640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2124101de0 0x7f21241985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.333+0000 7f2128e63640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2124102790 0x7f2124198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.333+0000 7f2128e63640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106f10 0x7f212419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.333+0000 7f2128e63640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2124112010 con 0x7f2124106f10 2024-09-19T07:02:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.333+0000 7f2128e63640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2124111e90 con 0x7f2124101de0 2024-09-19T07:02:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.333+0000 7f2128e63640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2124112190 con 0x7f2124102790 2024-09-19T07:02:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.333+0000 7f2122d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106f10 0x7f212419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.333+0000 7f2122d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106f10 0x7f212419cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47844/0 (socket says 172.21.15.103:47844) 2024-09-19T07:02:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.333+0000 7f2122d9d640 1 -- 172.21.15.103:0/564667587 learned_addr learned my addr 172.21.15.103:0/564667587 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.334+0000 7f212259c640 1 --2- 172.21.15.103:0/564667587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2124101de0 0x7f21241985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.334+0000 7f2121d9b640 1 --2- 172.21.15.103:0/564667587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2124102790 0x7f2124198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.334+0000 7f2122d9d640 1 -- 172.21.15.103:0/564667587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2124102790 msgr2=0x7f2124198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.334+0000 7f2122d9d640 1 --2- 172.21.15.103:0/564667587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2124102790 0x7f2124198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.334+0000 7f2122d9d640 1 -- 172.21.15.103:0/564667587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2124101de0 msgr2=0x7f21241985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.334+0000 7f2122d9d640 1 --2- 172.21.15.103:0/564667587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2124101de0 0x7f21241985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.334+0000 7f2122d9d640 1 -- 172.21.15.103:0/564667587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212419d5a0 con 0x7f2124106f10 2024-09-19T07:02:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.334+0000 7f212259c640 1 --2- 172.21.15.103:0/564667587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2124101de0 0x7f21241985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.334+0000 7f2122d9d640 1 --2- 172.21.15.103:0/564667587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106f10 0x7f212419cec0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f2114009230 tx=0x7f21140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.334+0000 7f211b7fe640 1 -- 172.21.15.103:0/564667587 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211403f030 con 0x7f2124106f10 2024-09-19T07:02:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.334+0000 7f211b7fe640 1 -- 172.21.15.103:0/564667587 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2114003ce0 con 0x7f2124106f10 2024-09-19T07:02:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.334+0000 7f2128e63640 1 -- 172.21.15.103:0/564667587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212419d830 con 0x7f2124106f10 2024-09-19T07:02:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.334+0000 7f211b7fe640 1 -- 172.21.15.103:0/564667587 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114035070 con 0x7f2124106f10 2024-09-19T07:02:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.334+0000 7f2128e63640 1 -- 172.21.15.103:0/564667587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21241a5170 con 0x7f2124106f10 2024-09-19T07:02:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.337+0000 7f211b7fe640 1 -- 172.21.15.103:0/564667587 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f2114042030 con 0x7f2124106f10 2024-09-19T07:02:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.337+0000 7f212b23c740 1 Processor -- start 2024-09-19T07:02:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f212b23c740 1 -- start start 2024-09-19T07:02:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f211b7fe640 1 --2- 172.21.15.103:0/564667587 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21100a3500 0x7f21100a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f212b23c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd54b40 0x55ea2bd53090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f212b23c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd535d0 0x55ea2bd53a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f212b23c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd5e370 0x55ea2bd60760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f212b23c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ea2bcb92f0 con 0x55ea2bd5e370 2024-09-19T07:02:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f212b23c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ea2bc60cf0 con 0x55ea2bd54b40 2024-09-19T07:02:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f212b23c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ea2bcb52b0 con 0x55ea2bd535d0 2024-09-19T07:02:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f2121d9b640 1 --2- 172.21.15.103:0/564667587 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21100a3500 0x7f21100a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f211b7fe640 1 -- 172.21.15.103:0/564667587 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f21140f0220 con 0x7f2124106f10 2024-09-19T07:02:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f2122d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd5e370 0x55ea2bd60760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f2122d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd5e370 0x55ea2bd60760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47852/0 (socket says 172.21.15.103:47852) 2024-09-19T07:02:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f2122d9d640 1 -- 172.21.15.103:0/3588747232 learned_addr learned my addr 172.21.15.103:0/3588747232 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f212259c640 1 --2- 172.21.15.103:0/3588747232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd54b40 0x55ea2bd53090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f2122d9d640 1 -- 172.21.15.103:0/3588747232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd535d0 msgr2=0x55ea2bd53a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f2122d9d640 1 --2- 172.21.15.103:0/3588747232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd535d0 0x55ea2bd53a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f2122d9d640 1 -- 172.21.15.103:0/3588747232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd54b40 msgr2=0x55ea2bd53090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f2122d9d640 1 --2- 172.21.15.103:0/3588747232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd54b40 0x55ea2bd53090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f2122d9d640 1 -- 172.21.15.103:0/3588747232 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea2bc453a0 con 0x55ea2bd5e370 2024-09-19T07:02:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f212259c640 1 --2- 172.21.15.103:0/3588747232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd54b40 0x55ea2bd53090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.338+0000 7f2121d9b640 1 --2- 172.21.15.103:0/564667587 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21100a3500 0x7f21100a59c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f21240ffe70 tx=0x7f210c009c70 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.339+0000 7f2122d9d640 1 --2- 172.21.15.103:0/3588747232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd5e370 0x55ea2bd60760 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f21140c4c20 tx=0x7f211402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98dc766500520ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.339+0000 7f2118ff9640 1 -- 172.21.15.103:0/3588747232 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114049040 con 0x55ea2bd5e370 2024-09-19T07:02:23.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.339+0000 7f2118ff9640 1 -- 172.21.15.103:0/3588747232 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21140bcbe0 con 0x55ea2bd5e370 2024-09-19T07:02:23.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.339+0000 7f2118ff9640 1 -- 172.21.15.103:0/3588747232 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21140edbd0 con 0x55ea2bd5e370 2024-09-19T07:02:23.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.339+0000 7f212b23c740 1 -- 172.21.15.103:0/3588747232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd5e370 msgr2=0x55ea2bd60760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:23.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.339+0000 7f212b23c740 1 --2- 172.21.15.103:0/3588747232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd5e370 0x55ea2bd60760 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f21140c4c20 tx=0x7f211402a410 comp rx=0 tx=0).stop 2024-09-19T07:02:23.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.339+0000 7f212b23c740 1 -- 172.21.15.103:0/3588747232 shutdown_connections 2024-09-19T07:02:23.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.339+0000 7f212b23c740 1 --2- 172.21.15.103:0/3588747232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd5e370 0x55ea2bd60760 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.339+0000 7f212b23c740 1 --2- 172.21.15.103:0/3588747232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd535d0 0x55ea2bd53a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.339+0000 7f212b23c740 1 --2- 172.21.15.103:0/3588747232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd54b40 0x55ea2bd53090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.339+0000 7f212b23c740 1 -- 172.21.15.103:0/3588747232 >> 172.21.15.103:0/3588747232 conn(0x55ea2bd1c310 msgr2=0x55ea2bd4fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:23.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.340+0000 7f212b23c740 1 -- 172.21.15.103:0/3588747232 shutdown_connections 2024-09-19T07:02:23.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.340+0000 7f212b23c740 1 -- 172.21.15.103:0/3588747232 wait complete. 2024-09-19T07:02:23.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.340+0000 7f212b23c740 1 -- start start 2024-09-19T07:02:23.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.340+0000 7f212b23c740 1 Processor -- start 2024-09-19T07:02:23.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.340+0000 7f212b23c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd535d0 0x55ea2bd5ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.340+0000 7f212b23c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd5b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.340+0000 7f212b23c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd5e370 0x55ea2bd5b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.340+0000 7f212b23c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ea2bd60e30 con 0x55ea2bd54b40 2024-09-19T07:02:23.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.340+0000 7f212b23c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ea2bc74060 con 0x55ea2bd535d0 2024-09-19T07:02:23.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.340+0000 7f212b23c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ea2bd60fb0 con 0x55ea2bd5e370 2024-09-19T07:02:23.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.340+0000 7f212b23c740 10 client.0 _ll_register_callbacks cb 0x55ea2b9c78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:23.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.340+0000 7f2121d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd5b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.340+0000 7f2121d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd5b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47856/0 (socket says 172.21.15.103:47856) 2024-09-19T07:02:23.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.340+0000 7f2121d9b640 1 -- 172.21.15.103:0/381852291 learned_addr learned my addr 172.21.15.103:0/381852291 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:23.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f212259c640 1 --2- 172.21.15.103:0/381852291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd535d0 0x55ea2bd5ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f2122d9d640 1 --2- 172.21.15.103:0/381852291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd5e370 0x55ea2bd5b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f2121d9b640 1 -- 172.21.15.103:0/381852291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd5e370 msgr2=0x55ea2bd5b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:23.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f2121d9b640 1 --2- 172.21.15.103:0/381852291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd5e370 0x55ea2bd5b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f2121d9b640 1 -- 172.21.15.103:0/381852291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd535d0 msgr2=0x55ea2bd5ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:23.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f2121d9b640 1 --2- 172.21.15.103:0/381852291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd535d0 0x55ea2bd5ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f2121d9b640 1 -- 172.21.15.103:0/381852291 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea2ba2a3c0 con 0x55ea2bd54b40 2024-09-19T07:02:23.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f212259c640 1 --2- 172.21.15.103:0/381852291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd535d0 0x55ea2bd5ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:23.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f2121d9b640 1 --2- 172.21.15.103:0/381852291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd5b070 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f210c0196b0 tx=0x7f210c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92c0782ae955210 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:23.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f2109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:23.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f2109ffb640 1 -- 172.21.15.103:0/381852291 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c026060 con 0x55ea2bd54b40 2024-09-19T07:02:23.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f2109ffb640 1 -- 172.21.15.103:0/381852291 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f210c00f070 con 0x55ea2bd54b40 2024-09-19T07:02:23.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f212b23c740 1 -- 172.21.15.103:0/381852291 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21140bcbe0 con 0x55ea2bd54b40 2024-09-19T07:02:23.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f2109ffb640 1 -- 172.21.15.103:0/381852291 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f210c014750 con 0x55ea2bd54b40 2024-09-19T07:02:23.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f212b23c740 10 client.19980 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:23.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f212b23c740 1 -- 172.21.15.103:0/381852291 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea2bd6c8e0 con 0x55ea2bd54b40 2024-09-19T07:02:23.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.341+0000 7f2109ffb640 1 -- 172.21.15.103:0/381852291 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21140bcbe0 con 0x55ea2bd54b40 2024-09-19T07:02:23.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.342+0000 7f2109ffb640 1 -- 172.21.15.103:0/381852291 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f210c021c50 con 0x55ea2bd54b40 2024-09-19T07:02:23.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.342+0000 7f212b23c740 10 client.19980 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:23.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.342+0000 7f212b23c740 10 client.19980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:23.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.342+0000 7f212b23c740 20 client.19980 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:23.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.342+0000 7f212b23c740 1 --2- 172.21.15.103:0/381852291 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ea2bd6dda0 0x55ea2bd701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.342+0000 7f212b23c740 4 client.19980 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:23.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.342+0000 7f212b23c740 1 -- 172.21.15.103:0/381852291 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea2bcc6860 con 0x55ea2bd6dda0 2024-09-19T07:02:23.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.342+0000 7f212259c640 1 --2- 172.21.15.103:0/381852291 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ea2bd6dda0 0x55ea2bd701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.342+0000 7f212259c640 1 --2- 172.21.15.103:0/381852291 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ea2bd6dda0 0x55ea2bd701f0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6383ca2e348fa99e server_cookie=5bf315a8f85b22c4 in_seq=0 out_seq=0 2024-09-19T07:02:23.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.342+0000 7f2109ffb640 10 client.19980 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:23.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.343+0000 7f2109ffb640 1 -- 172.21.15.103:0/381852291 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea2bcc6860 con 0x55ea2bd6dda0 2024-09-19T07:02:23.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.343+0000 7f2109ffb640 10 client.19980 handle_command_reply: tid=0 2024-09-19T07:02:23.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.343+0000 7f212b23c740 1 client.19980 shutdown 2024-09-19T07:02:23.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.343+0000 7f212b23c740 1 -- 172.21.15.103:0/381852291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 msgr2=0x55ea2bd5b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:23.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.343+0000 7f212b23c740 1 --2- 172.21.15.103:0/381852291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd5b070 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f210c0196b0 tx=0x7f210c0047d0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.343+0000 7f212b23c740 1 -- 172.21.15.103:0/381852291 shutdown_connections 2024-09-19T07:02:23.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.343+0000 7f212b23c740 1 --2- 172.21.15.103:0/381852291 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ea2bd6dda0 0x55ea2bd701f0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.343+0000 7f212b23c740 1 --2- 172.21.15.103:0/381852291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd5e370 0x55ea2bd5b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.343+0000 7f212b23c740 1 --2- 172.21.15.103:0/381852291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd5b070 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.343+0000 7f2109ffb640 0 client.19980 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:23.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.343+0000 7f212b23c740 1 --2- 172.21.15.103:0/381852291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd535d0 0x55ea2bd5ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.343+0000 7f212b23c740 1 -- 172.21.15.103:0/381852291 >> 172.21.15.103:0/381852291 conn(0x55ea2bd1c310 msgr2=0x55ea2bd55990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:23.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.343+0000 7f212b23c740 1 -- 172.21.15.103:0/381852291 shutdown_connections 2024-09-19T07:02:23.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.343+0000 7f212b23c740 1 -- 172.21.15.103:0/381852291 wait complete. 2024-09-19T07:02:23.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.343+0000 7f212b23c740 20 client.19980 trim_cache size 0 max 16384 2024-09-19T07:02:23.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.356+0000 7f212b23c740 1 Processor -- start 2024-09-19T07:02:23.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.356+0000 7f212b23c740 1 -- start start 2024-09-19T07:02:23.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.356+0000 7f212b23c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd535d0 0x55ea2bd4fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.356+0000 7f212b23c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd54300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.356+0000 7f212b23c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd5e370 0x55ea2bd5a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.356+0000 7f212b23c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21241ad990 con 0x55ea2bd54b40 2024-09-19T07:02:23.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.356+0000 7f212b23c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21241af4d0 con 0x55ea2bd535d0 2024-09-19T07:02:23.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.356+0000 7f212b23c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21241ae060 con 0x55ea2bd5e370 2024-09-19T07:02:23.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.356+0000 7f2121d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd54300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.356+0000 7f2121d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd54300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47872/0 (socket says 172.21.15.103:47872) 2024-09-19T07:02:23.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.356+0000 7f2121d9b640 1 -- 172.21.15.103:0/3374810685 learned_addr learned my addr 172.21.15.103:0/3374810685 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:23.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.357+0000 7f212259c640 1 --2- 172.21.15.103:0/3374810685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd535d0 0x55ea2bd4fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.357+0000 7f2121d9b640 1 -- 172.21.15.103:0/3374810685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd5e370 msgr2=0x55ea2bd5a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:23.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.357+0000 7f2122d9d640 1 --2- 172.21.15.103:0/3374810685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd5e370 0x55ea2bd5a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.357+0000 7f2121d9b640 1 --2- 172.21.15.103:0/3374810685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd5e370 0x55ea2bd5a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.357+0000 7f2121d9b640 1 -- 172.21.15.103:0/3374810685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd535d0 msgr2=0x55ea2bd4fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:23.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.357+0000 7f2121d9b640 1 --2- 172.21.15.103:0/3374810685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd535d0 0x55ea2bd4fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.357+0000 7f2121d9b640 1 -- 172.21.15.103:0/3374810685 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea2bd5e130 con 0x55ea2bd54b40 2024-09-19T07:02:23.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.357+0000 7f2122d9d640 1 --2- 172.21.15.103:0/3374810685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd5e370 0x55ea2bd5a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:23.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.357+0000 7f2121d9b640 1 --2- 172.21.15.103:0/3374810685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd54300 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f210c0196b0 tx=0x7f210c00d700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb317e77b0435c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:23.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.357+0000 7f210bfff640 1 -- 172.21.15.103:0/3374810685 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c0059c0 con 0x55ea2bd54b40 2024-09-19T07:02:23.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.357+0000 7f210bfff640 1 -- 172.21.15.103:0/3374810685 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210c014dd0 con 0x55ea2bd54b40 2024-09-19T07:02:23.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.357+0000 7f210bfff640 1 -- 172.21.15.103:0/3374810685 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c016b20 con 0x55ea2bd54b40 2024-09-19T07:02:23.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 -- 172.21.15.103:0/3374810685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 msgr2=0x55ea2bd54300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:23.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 --2- 172.21.15.103:0/3374810685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd54300 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f210c0196b0 tx=0x7f210c00d700 comp rx=0 tx=0).stop 2024-09-19T07:02:23.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 -- 172.21.15.103:0/3374810685 shutdown_connections 2024-09-19T07:02:23.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 --2- 172.21.15.103:0/3374810685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd5e370 0x55ea2bd5a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 --2- 172.21.15.103:0/3374810685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd54300 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 --2- 172.21.15.103:0/3374810685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd535d0 0x55ea2bd4fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 -- 172.21.15.103:0/3374810685 >> 172.21.15.103:0/3374810685 conn(0x55ea2bd1c310 msgr2=0x55ea2bd4f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:23.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 -- 172.21.15.103:0/3374810685 shutdown_connections 2024-09-19T07:02:23.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 -- 172.21.15.103:0/3374810685 wait complete. 2024-09-19T07:02:23.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 -- start start 2024-09-19T07:02:23.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 Processor -- start 2024-09-19T07:02:23.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd535d0 0x55ea2bd61fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd5e370 0x55ea2bd62a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21241aee00 con 0x55ea2bd54b40 2024-09-19T07:02:23.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21241ae730 con 0x55ea2bd5e370 2024-09-19T07:02:23.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.358+0000 7f212b23c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21241ab990 con 0x55ea2bd535d0 2024-09-19T07:02:23.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f212b23c740 10 client.0 _ll_register_callbacks cb 0x7f210c009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:23.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f2121d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f2121d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd624f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47878/0 (socket says 172.21.15.103:47878) 2024-09-19T07:02:23.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f2121d9b640 1 -- 172.21.15.103:0/4218182946 learned_addr learned my addr 172.21.15.103:0/4218182946 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:23.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f212259c640 1 --2- 172.21.15.103:0/4218182946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd535d0 0x55ea2bd61fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f2122d9d640 1 --2- 172.21.15.103:0/4218182946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd5e370 0x55ea2bd62a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f2121d9b640 1 -- 172.21.15.103:0/4218182946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd535d0 msgr2=0x55ea2bd61fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:23.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f2121d9b640 1 --2- 172.21.15.103:0/4218182946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd535d0 0x55ea2bd61fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f2121d9b640 1 -- 172.21.15.103:0/4218182946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd5e370 msgr2=0x55ea2bd62a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:23.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f2121d9b640 1 --2- 172.21.15.103:0/4218182946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd5e370 0x55ea2bd62a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f2121d9b640 1 -- 172.21.15.103:0/4218182946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea2bdd7ae0 con 0x55ea2bd54b40 2024-09-19T07:02:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f2122d9d640 1 --2- 172.21.15.103:0/4218182946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd5e370 0x55ea2bd62a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f2121d9b640 1 --2- 172.21.15.103:0/4218182946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd624f0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f210c0196b0 tx=0x7f210c005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9654663ca4f04cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f2109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f2109ffb640 1 -- 172.21.15.103:0/4218182946 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c016b70 con 0x55ea2bd54b40 2024-09-19T07:02:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f2109ffb640 1 -- 172.21.15.103:0/4218182946 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f210c016d10 con 0x55ea2bd54b40 2024-09-19T07:02:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f212b23c740 1 -- 172.21.15.103:0/4218182946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f210c014dd0 con 0x55ea2bd54b40 2024-09-19T07:02:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f2109ffb640 1 -- 172.21.15.103:0/4218182946 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f210c026020 con 0x55ea2bd54b40 2024-09-19T07:02:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f212b23c740 10 client.19986 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.359+0000 7f212b23c740 1 -- 172.21.15.103:0/4218182946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea2bdd8d30 con 0x55ea2bd54b40 2024-09-19T07:02:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.360+0000 7f2109ffb640 1 -- 172.21.15.103:0/4218182946 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210c014dd0 con 0x55ea2bd54b40 2024-09-19T07:02:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.360+0000 7f2109ffb640 1 -- 172.21.15.103:0/4218182946 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f210c02a430 con 0x55ea2bd54b40 2024-09-19T07:02:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.360+0000 7f212b23c740 10 client.19986 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.360+0000 7f212b23c740 10 client.19986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.360+0000 7f212b23c740 20 client.19986 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.360+0000 7f212b23c740 1 --2- 172.21.15.103:0/4218182946 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ea2bd6dda0 0x55ea2bddb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.360+0000 7f212b23c740 4 client.19986 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:02:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.360+0000 7f212b23c740 1 -- 172.21.15.103:0/4218182946 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea2bcc0d70 con 0x55ea2bd6dda0 2024-09-19T07:02:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.360+0000 7f212259c640 1 --2- 172.21.15.103:0/4218182946 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ea2bd6dda0 0x55ea2bddb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.360+0000 7f212259c640 1 --2- 172.21.15.103:0/4218182946 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ea2bd6dda0 0x55ea2bddb430 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=383feef979c57d9e server_cookie=9dc98562bb84a503 in_seq=0 out_seq=0 2024-09-19T07:02:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.360+0000 7f2109ffb640 10 client.19986 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.361+0000 7f2109ffb640 1 -- 172.21.15.103:0/4218182946 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea2bcc0d70 con 0x55ea2bd6dda0 2024-09-19T07:02:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.361+0000 7f2109ffb640 10 client.19986 handle_command_reply: tid=0 2024-09-19T07:02:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.361+0000 7f212b23c740 1 client.19986 shutdown 2024-09-19T07:02:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.361+0000 7f212b23c740 1 -- 172.21.15.103:0/4218182946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 msgr2=0x55ea2bd624f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.361+0000 7f212b23c740 1 --2- 172.21.15.103:0/4218182946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd624f0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f210c0196b0 tx=0x7f210c005c70 comp rx=0 tx=0).stop 2024-09-19T07:02:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.361+0000 7f212b23c740 1 -- 172.21.15.103:0/4218182946 shutdown_connections 2024-09-19T07:02:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.361+0000 7f212b23c740 1 --2- 172.21.15.103:0/4218182946 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ea2bd6dda0 0x55ea2bddb430 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.361+0000 7f212b23c740 1 --2- 172.21.15.103:0/4218182946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea2bd5e370 0x55ea2bd62a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.361+0000 7f2109ffb640 0 client.19986 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:23.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.361+0000 7f212b23c740 1 --2- 172.21.15.103:0/4218182946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea2bd54b40 0x55ea2bd624f0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.361+0000 7f212b23c740 1 --2- 172.21.15.103:0/4218182946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea2bd535d0 0x55ea2bd61fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.361+0000 7f212b23c740 1 -- 172.21.15.103:0/4218182946 >> 172.21.15.103:0/4218182946 conn(0x55ea2bd1c310 msgr2=0x55ea2bd5d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:23.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.361+0000 7f212b23c740 1 -- 172.21.15.103:0/4218182946 shutdown_connections 2024-09-19T07:02:23.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.361+0000 7f212b23c740 1 -- 172.21.15.103:0/4218182946 wait complete. 2024-09-19T07:02:23.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.361+0000 7f212b23c740 20 client.19986 trim_cache size 0 max 16384 2024-09-19T07:02:23.367 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:02:23.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.362+0000 7f2128e63640 1 -- 172.21.15.103:0/564667587 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21100a3500 msgr2=0x7f21100a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:23.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.362+0000 7f2128e63640 1 --2- 172.21.15.103:0/564667587 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21100a3500 0x7f21100a59c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f21240ffe70 tx=0x7f210c009c70 comp rx=0 tx=0).stop 2024-09-19T07:02:23.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.362+0000 7f2128e63640 1 -- 172.21.15.103:0/564667587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106f10 msgr2=0x7f212419cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:23.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.362+0000 7f2128e63640 1 --2- 172.21.15.103:0/564667587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106f10 0x7f212419cec0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f2114009230 tx=0x7f21140042f0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.362+0000 7f2128e63640 1 -- 172.21.15.103:0/564667587 shutdown_connections 2024-09-19T07:02:23.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.362+0000 7f2128e63640 1 --2- 172.21.15.103:0/564667587 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21100a3500 0x7f21100a59c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.362+0000 7f2128e63640 1 --2- 172.21.15.103:0/564667587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106f10 0x7f212419cec0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.362+0000 7f2128e63640 1 --2- 172.21.15.103:0/564667587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2124102790 0x7f2124198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.362+0000 7f2128e63640 1 --2- 172.21.15.103:0/564667587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2124101de0 0x7f21241985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:23.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.362+0000 7f2128e63640 1 -- 172.21.15.103:0/564667587 >> 172.21.15.103:0/564667587 conn(0x7f21240fd8c0 msgr2=0x7f21240fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:23.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.362+0000 7f2128e63640 1 -- 172.21.15.103:0/564667587 shutdown_connections 2024-09-19T07:02:23.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:23.362+0000 7f2128e63640 1 -- 172.21.15.103:0/564667587 wait complete. 2024-09-19T07:02:24.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:23 smithi174 ceph-mon[35637]: pgmap v411: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-19T07:02:24.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:23 smithi103 ceph-mon[31158]: pgmap v411: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-19T07:02:24.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:23 smithi134 ceph-mon[35356]: pgmap v411: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-19T07:02:25.006 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:02:25.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:02:25.266 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:25.505 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:02:25.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.722+0000 7fa1bfc29640 1 -- 172.21.15.103:0/3048542379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1b81082a0 msgr2=0x7fa1b810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:25.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.722+0000 7fa1bfc29640 1 --2- 172.21.15.103:0/3048542379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1b81082a0 0x7fa1b810ec40 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009cd0 tx=0x7fa1b002a8e0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.722+0000 7fa1bfc29640 1 -- 172.21.15.103:0/3048542379 shutdown_connections 2024-09-19T07:02:25.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.722+0000 7fa1bfc29640 1 --2- 172.21.15.103:0/3048542379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1b81082a0 0x7fa1b810ec40 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.722+0000 7fa1bfc29640 1 --2- 172.21.15.103:0/3048542379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1b81036e0 0x7fa1b8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.722+0000 7fa1bfc29640 1 --2- 172.21.15.103:0/3048542379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1b8102d30 0x7fa1b8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.722+0000 7fa1bfc29640 1 -- 172.21.15.103:0/3048542379 >> 172.21.15.103:0/3048542379 conn(0x7fa1b80fc7d0 msgr2=0x7fa1b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.722+0000 7fa1bfc29640 1 -- 172.21.15.103:0/3048542379 shutdown_connections 2024-09-19T07:02:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.722+0000 7fa1bfc29640 1 -- 172.21.15.103:0/3048542379 wait complete. 2024-09-19T07:02:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.723+0000 7fa1bfc29640 1 Processor -- start 2024-09-19T07:02:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.723+0000 7fa1bfc29640 1 -- start start 2024-09-19T07:02:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.723+0000 7fa1bfc29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1b8102d30 0x7fa1b81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.723+0000 7fa1bfc29640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1b81036e0 0x7fa1b8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.723+0000 7fa1bfc29640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1b81082a0 0x7fa1b819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.723+0000 7fa1bfc29640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81120d0 con 0x7fa1b8102d30 2024-09-19T07:02:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.723+0000 7fa1bfc29640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8111f50 con 0x7fa1b81082a0 2024-09-19T07:02:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.723+0000 7fa1bfc29640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8112250 con 0x7fa1b81036e0 2024-09-19T07:02:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.723+0000 7fa1bd9c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1b8102d30 0x7fa1b81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.723+0000 7fa1bd9c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1b8102d30 0x7fa1b81984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47884/0 (socket says 172.21.15.103:47884) 2024-09-19T07:02:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.723+0000 7fa1bd9c5640 1 -- 172.21.15.103:0/2095678108 learned_addr learned my addr 172.21.15.103:0/2095678108 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:25.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.724+0000 7fa1bd1c4640 1 --2- 172.21.15.103:0/2095678108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1b81036e0 0x7fa1b8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.724+0000 7fa1be1c6640 1 --2- 172.21.15.103:0/2095678108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1b81082a0 0x7fa1b819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.724+0000 7fa1bd9c5640 1 -- 172.21.15.103:0/2095678108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1b81036e0 msgr2=0x7fa1b8198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:25.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.724+0000 7fa1bd9c5640 1 --2- 172.21.15.103:0/2095678108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1b81036e0 0x7fa1b8198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.724+0000 7fa1bd9c5640 1 -- 172.21.15.103:0/2095678108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1b81082a0 msgr2=0x7fa1b819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:25.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.724+0000 7fa1bd9c5640 1 --2- 172.21.15.103:0/2095678108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1b81082a0 0x7fa1b819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.724+0000 7fa1bd9c5640 1 -- 172.21.15.103:0/2095678108 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b819d480 con 0x7fa1b8102d30 2024-09-19T07:02:25.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.724+0000 7fa1bd1c4640 1 --2- 172.21.15.103:0/2095678108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1b81036e0 0x7fa1b8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:25.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.724+0000 7fa1bd9c5640 1 --2- 172.21.15.103:0/2095678108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1b8102d30 0x7fa1b81984d0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400c890 tx=0x7fa1b400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:25.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.724+0000 7fa1aeffd640 1 -- 172.21.15.103:0/2095678108 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4004220 con 0x7fa1b8102d30 2024-09-19T07:02:25.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.724+0000 7fa1aeffd640 1 -- 172.21.15.103:0/2095678108 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1b40043c0 con 0x7fa1b8102d30 2024-09-19T07:02:25.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.724+0000 7fa1bfc29640 1 -- 172.21.15.103:0/2095678108 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b819d770 con 0x7fa1b8102d30 2024-09-19T07:02:25.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.724+0000 7fa1aeffd640 1 -- 172.21.15.103:0/2095678108 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b40049e0 con 0x7fa1b8102d30 2024-09-19T07:02:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.724+0000 7fa1bfc29640 1 -- 172.21.15.103:0/2095678108 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b81a5050 con 0x7fa1b8102d30 2024-09-19T07:02:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.727+0000 7fa1c2002740 1 Processor -- start 2024-09-19T07:02:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.727+0000 7fa1c2002740 1 -- start start 2024-09-19T07:02:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.727+0000 7fa1aeffd640 1 -- 172.21.15.103:0/2095678108 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa1b40079b0 con 0x7fa1b8102d30 2024-09-19T07:02:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.727+0000 7fa1c2002740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9e8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.727+0000 7fa1c2002740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9e9480 0x55b3dd9f3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.727+0000 7fa1c2002740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9f3d40 0x55b3dd9f6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.727+0000 7fa1c2002740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b3dd94e280 con 0x55b3dd9e9480 2024-09-19T07:02:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.727+0000 7fa1c2002740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b3dd8f5b10 con 0x55b3dd9f3d40 2024-09-19T07:02:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.727+0000 7fa1c2002740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b3dd94a240 con 0x55b3dd9e8b60 2024-09-19T07:02:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.728+0000 7fa1bd9c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9e9480 0x55b3dd9f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.728+0000 7fa1bd9c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9e9480 0x55b3dd9f3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47900/0 (socket says 172.21.15.103:47900) 2024-09-19T07:02:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.728+0000 7fa1bd9c5640 1 -- 172.21.15.103:0/1708903198 learned_addr learned my addr 172.21.15.103:0/1708903198 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.728+0000 7fa1be1c6640 1 --2- 172.21.15.103:0/1708903198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9e8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.728+0000 7fa1bd1c4640 1 --2- 172.21.15.103:0/1708903198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9f3d40 0x55b3dd9f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.728+0000 7fa1bd9c5640 1 -- 172.21.15.103:0/1708903198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e8b60 msgr2=0x55b3dd9e8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.728+0000 7fa1bd9c5640 1 --2- 172.21.15.103:0/1708903198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9e8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.728+0000 7fa1bd9c5640 1 -- 172.21.15.103:0/1708903198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9f3d40 msgr2=0x55b3dd9f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.728+0000 7fa1bd9c5640 1 --2- 172.21.15.103:0/1708903198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9f3d40 0x55b3dd9f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.728+0000 7fa1bd9c5640 1 -- 172.21.15.103:0/1708903198 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3dd9021f0 con 0x55b3dd9e9480 2024-09-19T07:02:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.728+0000 7fa1bd1c4640 1 --2- 172.21.15.103:0/1708903198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9f3d40 0x55b3dd9f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.728+0000 7fa1aeffd640 1 --2- 172.21.15.103:0/2095678108 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa1a40a33b0 0x7fa1a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.728+0000 7fa1bd9c5640 1 --2- 172.21.15.103:0/1708903198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9e9480 0x55b3dd9f3800 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40cd4f0 tx=0x7fa1b40d8270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8db9e407182951 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.728+0000 7fa1be1c6640 1 --2- 172.21.15.103:0/2095678108 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa1a40a33b0 0x7fa1a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.728+0000 7fa1aeffd640 1 -- 172.21.15.103:0/2095678108 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7fa1b40cfe50 con 0x7fa1b8102d30 2024-09-19T07:02:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.728+0000 7fa1a3fff640 1 -- 172.21.15.103:0/1708903198 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b401d020 con 0x55b3dd9e9480 2024-09-19T07:02:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.729+0000 7fa1a3fff640 1 -- 172.21.15.103:0/1708903198 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1b409c810 con 0x55b3dd9e9480 2024-09-19T07:02:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.729+0000 7fa1a3fff640 1 -- 172.21.15.103:0/1708903198 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b409bee0 con 0x55b3dd9e9480 2024-09-19T07:02:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.729+0000 7fa1c2002740 1 -- 172.21.15.103:0/1708903198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9e9480 msgr2=0x55b3dd9f3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.729+0000 7fa1be1c6640 1 --2- 172.21.15.103:0/2095678108 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa1a40a33b0 0x7fa1a40a5870 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009a60 tx=0x7fa1b0009360 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.729+0000 7fa1c2002740 1 --2- 172.21.15.103:0/1708903198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9e9480 0x55b3dd9f3800 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40cd4f0 tx=0x7fa1b40d8270 comp rx=0 tx=0).stop 2024-09-19T07:02:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.729+0000 7fa1c2002740 1 -- 172.21.15.103:0/1708903198 shutdown_connections 2024-09-19T07:02:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.729+0000 7fa1c2002740 1 --2- 172.21.15.103:0/1708903198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9f3d40 0x55b3dd9f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.729+0000 7fa1c2002740 1 --2- 172.21.15.103:0/1708903198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9e9480 0x55b3dd9f3800 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.729+0000 7fa1c2002740 1 --2- 172.21.15.103:0/1708903198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.729+0000 7fa1c2002740 1 -- 172.21.15.103:0/1708903198 >> 172.21.15.103:0/1708903198 conn(0x55b3dd9e50f0 msgr2=0x55b3dd9e54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.729+0000 7fa1c2002740 1 -- 172.21.15.103:0/1708903198 shutdown_connections 2024-09-19T07:02:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.729+0000 7fa1c2002740 1 -- 172.21.15.103:0/1708903198 wait complete. 2024-09-19T07:02:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.730+0000 7fa1c2002740 1 -- start start 2024-09-19T07:02:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.730+0000 7fa1c2002740 1 Processor -- start 2024-09-19T07:02:25.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.730+0000 7fa1c2002740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9f0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.730+0000 7fa1c2002740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e9480 0x55b3dd9f1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.730+0000 7fa1c2002740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9f3d40 0x55b3dd9f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.730+0000 7fa1c2002740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b3dd9f6910 con 0x55b3dd9e8b60 2024-09-19T07:02:25.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.730+0000 7fa1c2002740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b3dd908f10 con 0x55b3dd9f3d40 2024-09-19T07:02:25.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.730+0000 7fa1c2002740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b3dd9f6a90 con 0x55b3dd9e9480 2024-09-19T07:02:25.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.730+0000 7fa1c2002740 10 client.0 _ll_register_callbacks cb 0x55b3dd8d1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:25.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.730+0000 7fa1bd9c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9f0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.730+0000 7fa1bd9c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9f0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47912/0 (socket says 172.21.15.103:47912) 2024-09-19T07:02:25.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.730+0000 7fa1bd9c5640 1 -- 172.21.15.103:0/2176908519 learned_addr learned my addr 172.21.15.103:0/2176908519 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:25.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.730+0000 7fa1bd1c4640 1 --2- 172.21.15.103:0/2176908519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e9480 0x55b3dd9f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.731+0000 7fa1be1c6640 1 --2- 172.21.15.103:0/2176908519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9f3d40 0x55b3dd9f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.731+0000 7fa1bd9c5640 1 -- 172.21.15.103:0/2176908519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e9480 msgr2=0x55b3dd9f1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:25.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.731+0000 7fa1bd9c5640 1 --2- 172.21.15.103:0/2176908519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e9480 0x55b3dd9f1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.731+0000 7fa1bd9c5640 1 -- 172.21.15.103:0/2176908519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9f3d40 msgr2=0x55b3dd9f16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.731+0000 7fa1bd9c5640 1 --2- 172.21.15.103:0/2176908519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9f3d40 0x55b3dd9f16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.731+0000 7fa1bd9c5640 1 -- 172.21.15.103:0/2176908519 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3dd6bf6a0 con 0x55b3dd9e8b60 2024-09-19T07:02:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.731+0000 7fa1be1c6640 1 --2- 172.21.15.103:0/2176908519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9f3d40 0x55b3dd9f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.731+0000 7fa1bd9c5640 1 --2- 172.21.15.103:0/2176908519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9f0c50 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fa1b409c9c0 tx=0x7fa1b40057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a67bb39f6e5b42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.731+0000 7fa1a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.731+0000 7fa1a17fa640 1 -- 172.21.15.103:0/2176908519 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b401d040 con 0x55b3dd9e8b60 2024-09-19T07:02:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.731+0000 7fa1a17fa640 1 -- 172.21.15.103:0/2176908519 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7fa1b40bc0b0 con 0x55b3dd9e8b60 2024-09-19T07:02:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.731+0000 7fa1c2002740 1 -- 172.21.15.103:0/2176908519 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1b409c810 con 0x55b3dd9e8b60 2024-09-19T07:02:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.731+0000 7fa1a17fa640 1 -- 172.21.15.103:0/2176908519 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa1b409ba70 con 0x55b3dd9e8b60 2024-09-19T07:02:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.731+0000 7fa1c2002740 10 client.19998 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.731+0000 7fa1c2002740 1 -- 172.21.15.103:0/2176908519 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3dd9f6910 con 0x55b3dd9e8b60 2024-09-19T07:02:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.731+0000 7fa1a17fa640 1 -- 172.21.15.103:0/2176908519 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1b409c810 con 0x55b3dd9e8b60 2024-09-19T07:02:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.732+0000 7fa1a17fa640 1 -- 172.21.15.103:0/2176908519 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa1b40a0130 con 0x55b3dd9e8b60 2024-09-19T07:02:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.732+0000 7fa1c2002740 10 client.19998 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.732+0000 7fa1c2002740 10 client.19998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.732+0000 7fa1c2002740 20 client.19998 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.732+0000 7fa1c2002740 1 --2- 172.21.15.103:0/2176908519 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b3dda036c0 0x55b3dda05ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.732+0000 7fa1c2002740 4 client.19998 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.732+0000 7fa1c2002740 1 -- 172.21.15.103:0/2176908519 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3dd95b7f0 con 0x55b3dda036c0 2024-09-19T07:02:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.732+0000 7fa1bd1c4640 1 --2- 172.21.15.103:0/2176908519 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b3dda036c0 0x55b3dda05ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.732+0000 7fa1a17fa640 10 client.19998 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.732+0000 7fa1bd1c4640 1 --2- 172.21.15.103:0/2176908519 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b3dda036c0 0x55b3dda05ab0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ab8fe08c37a21cb server_cookie=c560dbc11e76a3bd in_seq=0 out_seq=0 2024-09-19T07:02:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.733+0000 7fa1a17fa640 1 -- 172.21.15.103:0/2176908519 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3dd95b7f0 con 0x55b3dda036c0 2024-09-19T07:02:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.733+0000 7fa1a17fa640 10 client.19998 handle_command_reply: tid=0 2024-09-19T07:02:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.733+0000 7fa1c2002740 1 client.19998 shutdown 2024-09-19T07:02:25.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.733+0000 7fa1c2002740 1 -- 172.21.15.103:0/2176908519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9e8b60 msgr2=0x55b3dd9f0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:25.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.733+0000 7fa1c2002740 1 --2- 172.21.15.103:0/2176908519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9f0c50 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fa1b409c9c0 tx=0x7fa1b40057a0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.733+0000 7fa1c2002740 1 -- 172.21.15.103:0/2176908519 shutdown_connections 2024-09-19T07:02:25.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.733+0000 7fa1c2002740 1 --2- 172.21.15.103:0/2176908519 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b3dda036c0 0x55b3dda05ab0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.733+0000 7fa1a17fa640 0 client.19998 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:25.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.733+0000 7fa1c2002740 1 --2- 172.21.15.103:0/2176908519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9f3d40 0x55b3dd9f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.733+0000 7fa1c2002740 1 --2- 172.21.15.103:0/2176908519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e9480 0x55b3dd9f1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.733+0000 7fa1c2002740 1 --2- 172.21.15.103:0/2176908519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9f0c50 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.733+0000 7fa1c2002740 1 -- 172.21.15.103:0/2176908519 >> 172.21.15.103:0/2176908519 conn(0x55b3dd9e50f0 msgr2=0x55b3dd9eadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:25.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.733+0000 7fa1c2002740 1 -- 172.21.15.103:0/2176908519 shutdown_connections 2024-09-19T07:02:25.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.733+0000 7fa1c2002740 1 -- 172.21.15.103:0/2176908519 wait complete. 2024-09-19T07:02:25.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.733+0000 7fa1c2002740 20 client.19998 trim_cache size 0 max 16384 2024-09-19T07:02:25.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.752+0000 7fa1c2002740 1 Processor -- start 2024-09-19T07:02:25.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.752+0000 7fa1c2002740 1 -- start start 2024-09-19T07:02:25.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.752+0000 7fa1c2002740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.752+0000 7fa1c2002740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9e9480 0x55b3dda0b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.752+0000 7fa1c2002740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9f3d40 0x55b3dda0b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.752+0000 7fa1c2002740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81ad9c0 con 0x55b3dd9f3d40 2024-09-19T07:02:25.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.752+0000 7fa1c2002740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81af500 con 0x55b3dd9e9480 2024-09-19T07:02:25.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.752+0000 7fa1c2002740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81ae090 con 0x55b3dd9e8b60 2024-09-19T07:02:25.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.752+0000 7fa1be1c6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9f3d40 0x55b3dda0b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.752+0000 7fa1be1c6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9f3d40 0x55b3dda0b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47924/0 (socket says 172.21.15.103:47924) 2024-09-19T07:02:25.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.752+0000 7fa1be1c6640 1 -- 172.21.15.103:0/2812415675 learned_addr learned my addr 172.21.15.103:0/2812415675 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:25.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.752+0000 7fa1bd9c5640 1 --2- 172.21.15.103:0/2812415675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.752+0000 7fa1bd1c4640 1 --2- 172.21.15.103:0/2812415675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9e9480 0x55b3dda0b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.753+0000 7fa1be1c6640 1 -- 172.21.15.103:0/2812415675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e8b60 msgr2=0x55b3dd9f2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:25.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.753+0000 7fa1be1c6640 1 --2- 172.21.15.103:0/2812415675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9f2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.753+0000 7fa1be1c6640 1 -- 172.21.15.103:0/2812415675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9e9480 msgr2=0x55b3dda0b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:25.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.753+0000 7fa1be1c6640 1 --2- 172.21.15.103:0/2812415675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9e9480 0x55b3dda0b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.753+0000 7fa1be1c6640 1 -- 172.21.15.103:0/2812415675 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3dd9f3520 con 0x55b3dd9f3d40 2024-09-19T07:02:25.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.753+0000 7fa1be1c6640 1 --2- 172.21.15.103:0/2812415675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9f3d40 0x55b3dda0b730 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0035000 tx=0x7fa1b003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f82da6c919b59ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:25.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.753+0000 7fa1a37fe640 1 -- 172.21.15.103:0/2812415675 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0004240 con 0x55b3dd9f3d40 2024-09-19T07:02:25.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.753+0000 7fa1a37fe640 1 -- 172.21.15.103:0/2812415675 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1b0036040 con 0x55b3dd9f3d40 2024-09-19T07:02:25.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.753+0000 7fa1a37fe640 1 -- 172.21.15.103:0/2812415675 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0030ed0 con 0x55b3dd9f3d40 2024-09-19T07:02:25.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.754+0000 7fa1c2002740 1 -- 172.21.15.103:0/2812415675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9f3d40 msgr2=0x55b3dda0b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:25.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.754+0000 7fa1c2002740 1 --2- 172.21.15.103:0/2812415675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9f3d40 0x55b3dda0b730 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0035000 tx=0x7fa1b003f000 comp rx=0 tx=0).stop 2024-09-19T07:02:25.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.754+0000 7fa1c2002740 1 -- 172.21.15.103:0/2812415675 shutdown_connections 2024-09-19T07:02:25.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.754+0000 7fa1c2002740 1 --2- 172.21.15.103:0/2812415675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9f3d40 0x55b3dda0b730 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.754+0000 7fa1c2002740 1 --2- 172.21.15.103:0/2812415675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9e9480 0x55b3dda0b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.754+0000 7fa1c2002740 1 --2- 172.21.15.103:0/2812415675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9f2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.754+0000 7fa1c2002740 1 -- 172.21.15.103:0/2812415675 >> 172.21.15.103:0/2812415675 conn(0x55b3dd9e50f0 msgr2=0x55b3dd9f2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:25.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.754+0000 7fa1c2002740 1 -- 172.21.15.103:0/2812415675 shutdown_connections 2024-09-19T07:02:25.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.754+0000 7fa1c2002740 1 -- 172.21.15.103:0/2812415675 wait complete. 2024-09-19T07:02:25.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.754+0000 7fa1c2002740 1 -- start start 2024-09-19T07:02:25.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.754+0000 7fa1c2002740 1 Processor -- start 2024-09-19T07:02:25.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1c2002740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1c2002740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e9480 0x55b3dd9f8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1c2002740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9f3d40 0x55b3dda6d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1c2002740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81aee30 con 0x55b3dd9f3d40 2024-09-19T07:02:25.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1c2002740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81ae760 con 0x55b3dd9e8b60 2024-09-19T07:02:25.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1c2002740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81ab9c0 con 0x55b3dd9e9480 2024-09-19T07:02:25.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1c2002740 10 client.0 _ll_register_callbacks cb 0x7fa1b0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:25.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1be1c6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9f3d40 0x55b3dda6d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1be1c6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9f3d40 0x55b3dda6d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47930/0 (socket says 172.21.15.103:47930) 2024-09-19T07:02:25.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1be1c6640 1 -- 172.21.15.103:0/4220878256 learned_addr learned my addr 172.21.15.103:0/4220878256 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:25.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1bd9c5640 1 --2- 172.21.15.103:0/4220878256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1bd1c4640 1 --2- 172.21.15.103:0/4220878256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e9480 0x55b3dd9f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1be1c6640 1 -- 172.21.15.103:0/4220878256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e9480 msgr2=0x55b3dd9f8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:25.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1be1c6640 1 --2- 172.21.15.103:0/4220878256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e9480 0x55b3dd9f8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1be1c6640 1 -- 172.21.15.103:0/4220878256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9e8b60 msgr2=0x55b3dd9f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:25.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1be1c6640 1 --2- 172.21.15.103:0/4220878256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.755+0000 7fa1be1c6640 1 -- 172.21.15.103:0/4220878256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3dda6d700 con 0x55b3dd9f3d40 2024-09-19T07:02:25.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.756+0000 7fa1be1c6640 1 --2- 172.21.15.103:0/4220878256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9f3d40 0x55b3dda6d1c0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fa1b002a920 tx=0x7fa1b00307d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bce9f7045520cca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:25.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.756+0000 7fa1a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:25.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.756+0000 7fa1a17fa640 1 -- 172.21.15.103:0/4220878256 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0058070 con 0x55b3dd9f3d40 2024-09-19T07:02:25.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.756+0000 7fa1a17fa640 1 -- 172.21.15.103:0/4220878256 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7fa1b0004450 con 0x55b3dd9f3d40 2024-09-19T07:02:25.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.756+0000 7fa1c2002740 1 -- 172.21.15.103:0/4220878256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1b0036040 con 0x55b3dd9f3d40 2024-09-19T07:02:25.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.756+0000 7fa1a17fa640 1 -- 172.21.15.103:0/4220878256 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa1b0064050 con 0x55b3dd9f3d40 2024-09-19T07:02:25.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.756+0000 7fa1c2002740 10 client.20004 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:25.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.756+0000 7fa1c2002740 1 -- 172.21.15.103:0/4220878256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3dda6e9a0 con 0x55b3dd9f3d40 2024-09-19T07:02:25.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.756+0000 7fa1a17fa640 1 -- 172.21.15.103:0/4220878256 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1b0036040 con 0x55b3dd9f3d40 2024-09-19T07:02:25.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.756+0000 7fa1a17fa640 1 -- 172.21.15.103:0/4220878256 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa1b0030070 con 0x55b3dd9f3d40 2024-09-19T07:02:25.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.756+0000 7fa1c2002740 10 client.20004 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:25.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.756+0000 7fa1c2002740 10 client.20004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:25.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.756+0000 7fa1c2002740 20 client.20004 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:25.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.756+0000 7fa1c2002740 1 --2- 172.21.15.103:0/4220878256 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b3dda036c0 0x55b3dda71020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:25.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.757+0000 7fa1c2002740 4 client.20004 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:02:25.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.757+0000 7fa1c2002740 1 -- 172.21.15.103:0/4220878256 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b3dd955d00 con 0x55b3dda036c0 2024-09-19T07:02:25.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.757+0000 7fa1bd9c5640 1 --2- 172.21.15.103:0/4220878256 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b3dda036c0 0x55b3dda71020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:25.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.757+0000 7fa1a17fa640 10 client.20004 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:25.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.757+0000 7fa1bd9c5640 1 --2- 172.21.15.103:0/4220878256 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b3dda036c0 0x55b3dda71020 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e84ba9e15dd0fe73 server_cookie=a58435aa7fb13ec in_seq=0 out_seq=0 2024-09-19T07:02:25.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.758+0000 7fa1a17fa640 1 -- 172.21.15.103:0/4220878256 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b3dd955d00 con 0x55b3dda036c0 2024-09-19T07:02:25.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.758+0000 7fa1a17fa640 10 client.20004 handle_command_reply: tid=0 2024-09-19T07:02:25.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.758+0000 7fa1c2002740 1 client.20004 shutdown 2024-09-19T07:02:25.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.758+0000 7fa1c2002740 1 -- 172.21.15.103:0/4220878256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9f3d40 msgr2=0x55b3dda6d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:25.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.758+0000 7fa1c2002740 1 --2- 172.21.15.103:0/4220878256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9f3d40 0x55b3dda6d1c0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fa1b002a920 tx=0x7fa1b00307d0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.758+0000 7fa1c2002740 1 -- 172.21.15.103:0/4220878256 shutdown_connections 2024-09-19T07:02:25.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.758+0000 7fa1c2002740 1 --2- 172.21.15.103:0/4220878256 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b3dda036c0 0x55b3dda71020 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.758+0000 7fa1c2002740 1 --2- 172.21.15.103:0/4220878256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3dd9f3d40 0x55b3dda6d1c0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.759+0000 7fa1a17fa640 0 client.20004 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:25.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.758+0000 7fa1c2002740 1 --2- 172.21.15.103:0/4220878256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3dd9e9480 0x55b3dd9f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.759+0000 7fa1c2002740 1 --2- 172.21.15.103:0/4220878256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3dd9e8b60 0x55b3dd9f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.759+0000 7fa1c2002740 1 -- 172.21.15.103:0/4220878256 >> 172.21.15.103:0/4220878256 conn(0x55b3dd9e50f0 msgr2=0x55b3dd9e4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:25.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.759+0000 7fa1c2002740 1 -- 172.21.15.103:0/4220878256 shutdown_connections 2024-09-19T07:02:25.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.759+0000 7fa1c2002740 1 -- 172.21.15.103:0/4220878256 wait complete. 2024-09-19T07:02:25.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.759+0000 7fa1c2002740 20 client.20004 trim_cache size 0 max 16384 2024-09-19T07:02:25.764 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:02:25.765 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:02:25.765 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "478db48c-b5aa-4d57-ad26-f644b37e7850", 2024-09-19T07:02:25.765 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:02:25.765 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:02:25.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.759+0000 7fa1bfc29640 1 -- 172.21.15.103:0/2095678108 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa1a40a33b0 msgr2=0x7fa1a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:25.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.759+0000 7fa1bfc29640 1 --2- 172.21.15.103:0/2095678108 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa1a40a33b0 0x7fa1a40a5870 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009a60 tx=0x7fa1b0009360 comp rx=0 tx=0).stop 2024-09-19T07:02:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.759+0000 7fa1bfc29640 1 -- 172.21.15.103:0/2095678108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1b8102d30 msgr2=0x7fa1b81984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.759+0000 7fa1bfc29640 1 --2- 172.21.15.103:0/2095678108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1b8102d30 0x7fa1b81984d0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400c890 tx=0x7fa1b400cd50 comp rx=0 tx=0).stop 2024-09-19T07:02:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.759+0000 7fa1bfc29640 1 -- 172.21.15.103:0/2095678108 shutdown_connections 2024-09-19T07:02:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.759+0000 7fa1bfc29640 1 --2- 172.21.15.103:0/2095678108 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa1a40a33b0 0x7fa1a40a5870 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.759+0000 7fa1bfc29640 1 --2- 172.21.15.103:0/2095678108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1b81082a0 0x7fa1b819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.759+0000 7fa1bfc29640 1 --2- 172.21.15.103:0/2095678108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1b81036e0 0x7fa1b8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.759+0000 7fa1bfc29640 1 --2- 172.21.15.103:0/2095678108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1b8102d30 0x7fa1b81984d0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.759+0000 7fa1bfc29640 1 -- 172.21.15.103:0/2095678108 >> 172.21.15.103:0/2095678108 conn(0x7fa1b80fc7d0 msgr2=0x7fa1b80fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.760+0000 7fa1bfc29640 1 -- 172.21.15.103:0/2095678108 shutdown_connections 2024-09-19T07:02:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:25.760+0000 7fa1bfc29640 1 -- 172.21.15.103:0/2095678108 wait complete. 2024-09-19T07:02:26.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:25 smithi174 ceph-mon[35637]: pgmap v412: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-09-19T07:02:26.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:25 smithi103 ceph-mon[31158]: pgmap v412: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-09-19T07:02:26.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:25 smithi134 ceph-mon[35356]: pgmap v412: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-09-19T07:02:26.402 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:02:26.452 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:26.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.957+0000 7f0403a15640 1 -- 172.21.15.103:0/2373868249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc102570 msgr2=0x7f03fc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:26.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.957+0000 7f0403a15640 1 --2- 172.21.15.103:0/2373868249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc102570 0x7f03fc1029b0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009900 tx=0x7f03ec02a480 comp rx=0 tx=0).stop 2024-09-19T07:02:26.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.957+0000 7f0403a15640 1 -- 172.21.15.103:0/2373868249 shutdown_connections 2024-09-19T07:02:26.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.957+0000 7f0403a15640 1 --2- 172.21.15.103:0/2373868249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc102ef0 0x7f03fc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:26.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.957+0000 7f0403a15640 1 --2- 172.21.15.103:0/2373868249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc102570 0x7f03fc1029b0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:26.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.957+0000 7f0403a15640 1 --2- 172.21.15.103:0/2373868249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03fc100e20 0x7f03fc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:26.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.957+0000 7f0403a15640 1 -- 172.21.15.103:0/2373868249 >> 172.21.15.103:0/2373868249 conn(0x7f03fc0fc7d0 msgr2=0x7f03fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:26.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.957+0000 7f0403a15640 1 -- 172.21.15.103:0/2373868249 shutdown_connections 2024-09-19T07:02:26.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.957+0000 7f0403a15640 1 -- 172.21.15.103:0/2373868249 wait complete. 2024-09-19T07:02:26.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.958+0000 7f0403a15640 1 Processor -- start 2024-09-19T07:02:26.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.958+0000 7f0403a15640 1 -- start start 2024-09-19T07:02:26.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.958+0000 7f0403a15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc100e20 0x7f03fc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:26.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.958+0000 7f0403a15640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc102570 0x7f03fc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:26.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.958+0000 7f0403a15640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03fc102ef0 0x7f03fc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:26.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.958+0000 7f0403a15640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc110f50 con 0x7f03fc100e20 2024-09-19T07:02:26.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.958+0000 7f0403a15640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc110dd0 con 0x7f03fc102570 2024-09-19T07:02:26.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.958+0000 7f0403a15640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1110d0 con 0x7f03fc102ef0 2024-09-19T07:02:26.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.958+0000 7f04017b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc100e20 0x7f03fc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:26.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.958+0000 7f04017b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc100e20 0x7f03fc0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47940/0 (socket says 172.21.15.103:47940) 2024-09-19T07:02:26.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.958+0000 7f04017b1640 1 -- 172.21.15.103:0/1862702413 learned_addr learned my addr 172.21.15.103:0/1862702413 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:26.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.958+0000 7f0400fb0640 1 --2- 172.21.15.103:0/1862702413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc102570 0x7f03fc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:26.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.959+0000 7f0401fb2640 1 --2- 172.21.15.103:0/1862702413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03fc102ef0 0x7f03fc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:26.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.959+0000 7f04017b1640 1 -- 172.21.15.103:0/1862702413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03fc102ef0 msgr2=0x7f03fc0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:26.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.959+0000 7f04017b1640 1 --2- 172.21.15.103:0/1862702413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03fc102ef0 0x7f03fc0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:26.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.959+0000 7f04017b1640 1 -- 172.21.15.103:0/1862702413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc102570 msgr2=0x7f03fc0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:26.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.959+0000 7f04017b1640 1 --2- 172.21.15.103:0/1862702413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc102570 0x7f03fc0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:26.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.959+0000 7f04017b1640 1 -- 172.21.15.103:0/1862702413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc1a6950 con 0x7f03fc100e20 2024-09-19T07:02:26.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.959+0000 7f04017b1640 1 --2- 172.21.15.103:0/1862702413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc100e20 0x7f03fc0fcc80 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f03f800c920 tx=0x7f03f800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:26.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.959+0000 7f03f27fc640 1 -- 172.21.15.103:0/1862702413 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8007c70 con 0x7f03fc100e20 2024-09-19T07:02:26.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.959+0000 7f03f27fc640 1 -- 172.21.15.103:0/1862702413 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f8007e10 con 0x7f03fc100e20 2024-09-19T07:02:26.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.959+0000 7f03f27fc640 1 -- 172.21.15.103:0/1862702413 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8018890 con 0x7f03fc100e20 2024-09-19T07:02:26.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.959+0000 7f0403a15640 1 -- 172.21.15.103:0/1862702413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03fc1a6c40 con 0x7f03fc100e20 2024-09-19T07:02:26.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.959+0000 7f0403a15640 1 -- 172.21.15.103:0/1862702413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03fc1a7100 con 0x7f03fc100e20 2024-09-19T07:02:26.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.960+0000 7f0403a15640 1 -- 172.21.15.103:0/1862702413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03fc101200 con 0x7f03fc100e20 2024-09-19T07:02:26.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.961+0000 7f03f27fc640 1 -- 172.21.15.103:0/1862702413 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f03f8018a70 con 0x7f03fc100e20 2024-09-19T07:02:26.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.962+0000 7f03f27fc640 1 --2- 172.21.15.103:0/1862702413 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03e80a3430 0x7f03e80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:26.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.962+0000 7f0400fb0640 1 --2- 172.21.15.103:0/1862702413 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03e80a3430 0x7f03e80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:26.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.962+0000 7f03f27fc640 1 -- 172.21.15.103:0/1862702413 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f03f80cfdb0 con 0x7f03fc100e20 2024-09-19T07:02:26.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.963+0000 7f0400fb0640 1 --2- 172.21.15.103:0/1862702413 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03e80a3430 0x7f03e80a58f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009230 tx=0x7f03ec0058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:26.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:26.966+0000 7f03f27fc640 1 -- 172.21.15.103:0/1862702413 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f03f80d4070 con 0x7f03fc100e20 2024-09-19T07:02:27.022 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:27.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:26 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T07:02:27.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:26 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:02:27.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:26 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T07:02:27.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:26 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:02:27.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:26 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T07:02:27.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:26 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:02:27.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:26 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T07:02:27.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:26 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:02:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:26 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T07:02:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:26 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:02:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:26 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T07:02:27.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:26 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:02:27.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:27.190+0000 7f0403a15640 1 -- 172.21.15.103:0/1862702413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03fc0fe1b0 con 0x7f03fc100e20 2024-09-19T07:02:27.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:27.191+0000 7f03f27fc640 1 -- 172.21.15.103:0/1862702413 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f03f8014030 con 0x7f03fc100e20 2024-09-19T07:02:27.192 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:02:27.192 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:02:27.192 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:02:27.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:27.193+0000 7f0403a15640 1 -- 172.21.15.103:0/1862702413 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03e80a3430 msgr2=0x7f03e80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:27.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:27.193+0000 7f0403a15640 1 --2- 172.21.15.103:0/1862702413 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03e80a3430 0x7f03e80a58f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009230 tx=0x7f03ec0058d0 comp rx=0 tx=0).stop 2024-09-19T07:02:27.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:27.193+0000 7f0403a15640 1 -- 172.21.15.103:0/1862702413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc100e20 msgr2=0x7f03fc0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:27.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:27.193+0000 7f0403a15640 1 --2- 172.21.15.103:0/1862702413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc100e20 0x7f03fc0fcc80 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f03f800c920 tx=0x7f03f800cde0 comp rx=0 tx=0).stop 2024-09-19T07:02:27.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:27.193+0000 7f0403a15640 1 -- 172.21.15.103:0/1862702413 shutdown_connections 2024-09-19T07:02:27.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:27.193+0000 7f0403a15640 1 --2- 172.21.15.103:0/1862702413 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f03e80a3430 0x7f03e80a58f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:27.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:27.193+0000 7f0403a15640 1 --2- 172.21.15.103:0/1862702413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03fc102ef0 0x7f03fc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:27.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:27.193+0000 7f0403a15640 1 --2- 172.21.15.103:0/1862702413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc102570 0x7f03fc0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:27.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:27.193+0000 7f0403a15640 1 --2- 172.21.15.103:0/1862702413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc100e20 0x7f03fc0fcc80 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:27.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:27.193+0000 7f0403a15640 1 -- 172.21.15.103:0/1862702413 >> 172.21.15.103:0/1862702413 conn(0x7f03fc0fc7d0 msgr2=0x7f03fc06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:27.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:27.193+0000 7f0403a15640 1 -- 172.21.15.103:0/1862702413 shutdown_connections 2024-09-19T07:02:27.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:27.193+0000 7f0403a15640 1 -- 172.21.15.103:0/1862702413 wait complete. 2024-09-19T07:02:27.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:27 smithi103 ceph-mon[31158]: pgmap v413: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-09-19T07:02:27.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:27 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1862702413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:02:28.011 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 93884bb1 --release --await' 2024-09-19T07:02:28.011 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 93884bb1 --release --await 2024-09-19T07:02:28.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:27 smithi174 ceph-mon[35637]: pgmap v413: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-09-19T07:02:28.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:27 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1862702413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:02:28.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:27 smithi134 ceph-mon[35356]: pgmap v413: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-09-19T07:02:28.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:27 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1862702413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:02:28.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.204+0000 7f75bdd54640 1 -- 172.21.15.103:0/3271812215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102ef0 msgr2=0x7f75b810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:28.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.205+0000 7f75bdd54640 1 --2- 172.21.15.103:0/3271812215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102ef0 0x7f75b810da40 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009ca0 tx=0x7f75b002a950 comp rx=0 tx=0).stop 2024-09-19T07:02:28.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.205+0000 7f75bdd54640 1 -- 172.21.15.103:0/3271812215 shutdown_connections 2024-09-19T07:02:28.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.205+0000 7f75bdd54640 1 --2- 172.21.15.103:0/3271812215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102ef0 0x7f75b810da40 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.205+0000 7f75bdd54640 1 --2- 172.21.15.103:0/3271812215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b8102570 0x7f75b81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.205+0000 7f75bdd54640 1 --2- 172.21.15.103:0/3271812215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b8100e20 0x7f75b8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.205+0000 7f75bdd54640 1 -- 172.21.15.103:0/3271812215 >> 172.21.15.103:0/3271812215 conn(0x7f75b80fc7d0 msgr2=0x7f75b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:28.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.205+0000 7f75bdd54640 1 -- 172.21.15.103:0/3271812215 shutdown_connections 2024-09-19T07:02:28.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.205+0000 7f75bdd54640 1 -- 172.21.15.103:0/3271812215 wait complete. 2024-09-19T07:02:28.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.205+0000 7f75bdd54640 1 Processor -- start 2024-09-19T07:02:28.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.205+0000 7f75bdd54640 1 -- start start 2024-09-19T07:02:28.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75bdd54640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b8100e20 0x7f75b8199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75bdd54640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102570 0x7f75b8199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75bdd54640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b8102ef0 0x7f75b81a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75bdd54640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8110f80 con 0x7f75b8102570 2024-09-19T07:02:28.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75bdd54640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8110e00 con 0x7f75b8102ef0 2024-09-19T07:02:28.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75bdd54640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8111100 con 0x7f75b8100e20 2024-09-19T07:02:28.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102570 0x7f75b8199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102570 0x7f75b8199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60940/0 (socket says 172.21.15.103:60940) 2024-09-19T07:02:28.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75b6ffd640 1 -- 172.21.15.103:0/1738294518 learned_addr learned my addr 172.21.15.103:0/1738294518 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:28.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75b77fe640 1 --2- 172.21.15.103:0/1738294518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b8100e20 0x7f75b8199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75b7fff640 1 --2- 172.21.15.103:0/1738294518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b8102ef0 0x7f75b81a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75b6ffd640 1 -- 172.21.15.103:0/1738294518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b8100e20 msgr2=0x7f75b8199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:28.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75b6ffd640 1 --2- 172.21.15.103:0/1738294518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b8100e20 0x7f75b8199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75b6ffd640 1 -- 172.21.15.103:0/1738294518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b8102ef0 msgr2=0x7f75b81a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:28.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75b6ffd640 1 --2- 172.21.15.103:0/1738294518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b8102ef0 0x7f75b81a42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75b6ffd640 1 -- 172.21.15.103:0/1738294518 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b81a47f0 con 0x7f75b8102570 2024-09-19T07:02:28.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75b77fe640 1 --2- 172.21.15.103:0/1738294518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b8100e20 0x7f75b8199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:28.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75b6ffd640 1 --2- 172.21.15.103:0/1738294518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102570 0x7f75b8199770 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00d950 tx=0x7f75ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:28.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75b4ff9640 1 -- 172.21.15.103:0/1738294518 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac017070 con 0x7f75b8102570 2024-09-19T07:02:28.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75b4ff9640 1 -- 172.21.15.103:0/1738294518 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75ac009c30 con 0x7f75b8102570 2024-09-19T07:02:28.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75bdd54640 1 -- 172.21.15.103:0/1738294518 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b81a49c0 con 0x7f75b8102570 2024-09-19T07:02:28.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75b4ff9640 1 -- 172.21.15.103:0/1738294518 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac00f6b0 con 0x7f75b8102570 2024-09-19T07:02:28.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.206+0000 7f75bdd54640 1 -- 172.21.15.103:0/1738294518 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b81a4f50 con 0x7f75b8102570 2024-09-19T07:02:28.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.209+0000 7f75b4ff9640 1 -- 172.21.15.103:0/1738294518 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f75ac0109d0 con 0x7f75b8102570 2024-09-19T07:02:28.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.209+0000 7f75c012d740 1 Processor -- start 2024-09-19T07:02:28.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.209+0000 7f75c012d740 1 -- start start 2024-09-19T07:02:28.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.209+0000 7f75c012d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef14834a0 0x55fef1483880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.209+0000 7f75c012d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef1483dc0 0x55fef148e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.209+0000 7f75c012d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef148e680 0x55fef1490a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.209+0000 7f75c012d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fef13e92e0 con 0x55fef1483dc0 2024-09-19T07:02:28.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.209+0000 7f75c012d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fef1390ce0 con 0x55fef14834a0 2024-09-19T07:02:28.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.209+0000 7f75c012d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fef13e52a0 con 0x55fef148e680 2024-09-19T07:02:28.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.209+0000 7f75b4ff9640 1 --2- 172.21.15.103:0/1738294518 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75a00a3430 0x7f75a00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.209+0000 7f75b77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef1483dc0 0x55fef148e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.209+0000 7f75b77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef1483dc0 0x55fef148e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60954/0 (socket says 172.21.15.103:60954) 2024-09-19T07:02:28.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.209+0000 7f75b77fe640 1 -- 172.21.15.103:0/668982859 learned_addr learned my addr 172.21.15.103:0/668982859 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:28.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.210+0000 7f75b7fff640 1 --2- 172.21.15.103:0/1738294518 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75a00a3430 0x7f75a00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.210+0000 7f75b4ff9640 1 -- 172.21.15.103:0/1738294518 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f75ac0cf940 con 0x7f75b8102570 2024-09-19T07:02:28.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.210+0000 7f75b6ffd640 1 --2- 172.21.15.103:0/668982859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef148e680 0x55fef1490a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.210+0000 7f75b7fff640 1 --2- 172.21.15.103:0/668982859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef14834a0 0x55fef1483880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.210+0000 7f75b77fe640 1 -- 172.21.15.103:0/668982859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef148e680 msgr2=0x55fef1490a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:28.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.210+0000 7f75b77fe640 1 --2- 172.21.15.103:0/668982859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef148e680 0x55fef1490a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.210+0000 7f75b77fe640 1 -- 172.21.15.103:0/668982859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef14834a0 msgr2=0x55fef1483880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:28.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.210+0000 7f75b77fe640 1 --2- 172.21.15.103:0/668982859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef14834a0 0x55fef1483880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.210+0000 7f75b77fe640 1 -- 172.21.15.103:0/668982859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fef1375440 con 0x55fef1483dc0 2024-09-19T07:02:28.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.210+0000 7f75b6ffd640 1 --2- 172.21.15.103:0/668982859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef148e680 0x55fef1490a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:28.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.210+0000 7f75b77fe640 1 --2- 172.21.15.103:0/668982859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef1483dc0 0x55fef148e140 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f75a800bc20 tx=0x7f75a800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17df0e708ade1828 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:28.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.210+0000 7f75b7fff640 1 --2- 172.21.15.103:0/1738294518 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75a00a3430 0x7f75a00a58f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f75b0005e30 tx=0x7f75b0005d80 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:28.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.210+0000 7f759dffb640 1 -- 172.21.15.103:0/668982859 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8017620 con 0x55fef1483dc0 2024-09-19T07:02:28.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.210+0000 7f759dffb640 1 -- 172.21.15.103:0/668982859 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a8017ca0 con 0x55fef1483dc0 2024-09-19T07:02:28.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.210+0000 7f759dffb640 1 -- 172.21.15.103:0/668982859 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a800fce0 con 0x55fef1483dc0 2024-09-19T07:02:28.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.211+0000 7f75c012d740 1 -- 172.21.15.103:0/668982859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef1483dc0 msgr2=0x55fef148e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:28.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.211+0000 7f75c012d740 1 --2- 172.21.15.103:0/668982859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef1483dc0 0x55fef148e140 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f75a800bc20 tx=0x7f75a800d710 comp rx=0 tx=0).stop 2024-09-19T07:02:28.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.211+0000 7f75c012d740 1 -- 172.21.15.103:0/668982859 shutdown_connections 2024-09-19T07:02:28.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.211+0000 7f75c012d740 1 --2- 172.21.15.103:0/668982859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef148e680 0x55fef1490a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.211+0000 7f75c012d740 1 --2- 172.21.15.103:0/668982859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef1483dc0 0x55fef148e140 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.211+0000 7f75c012d740 1 --2- 172.21.15.103:0/668982859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef14834a0 0x55fef1483880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.211+0000 7f75c012d740 1 -- 172.21.15.103:0/668982859 >> 172.21.15.103:0/668982859 conn(0x55fef147fa30 msgr2=0x55fef147fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:28.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.211+0000 7f75c012d740 1 -- 172.21.15.103:0/668982859 shutdown_connections 2024-09-19T07:02:28.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.211+0000 7f75c012d740 1 -- 172.21.15.103:0/668982859 wait complete. 2024-09-19T07:02:28.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.211+0000 7f75c012d740 1 -- start start 2024-09-19T07:02:28.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75c012d740 1 Processor -- start 2024-09-19T07:02:28.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75c012d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef14834a0 0x55fef148ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75c012d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef1483dc0 0x55fef148b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75c012d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef148e680 0x55fef148b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75c012d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fef1491250 con 0x55fef148e680 2024-09-19T07:02:28.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75c012d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fef13a4050 con 0x55fef14834a0 2024-09-19T07:02:28.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75c012d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fef14913d0 con 0x55fef1483dc0 2024-09-19T07:02:28.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75c012d740 10 client.0 _ll_register_callbacks cb 0x55fef13e4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:28.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75b7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef148e680 0x55fef148b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75b7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef148e680 0x55fef148b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60960/0 (socket says 172.21.15.103:60960) 2024-09-19T07:02:28.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75b7fff640 1 -- 172.21.15.103:0/2590807694 learned_addr learned my addr 172.21.15.103:0/2590807694 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:28.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75b7fff640 1 -- 172.21.15.103:0/2590807694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef1483dc0 msgr2=0x55fef148b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:28.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75b6ffd640 1 --2- 172.21.15.103:0/2590807694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef1483dc0 0x55fef148b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75b77fe640 1 --2- 172.21.15.103:0/2590807694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef14834a0 0x55fef148ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75b7fff640 1 --2- 172.21.15.103:0/2590807694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef1483dc0 0x55fef148b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75b7fff640 1 -- 172.21.15.103:0/2590807694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef14834a0 msgr2=0x55fef148ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:28.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75b7fff640 1 --2- 172.21.15.103:0/2590807694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef14834a0 0x55fef148ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75b7fff640 1 -- 172.21.15.103:0/2590807694 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fef115a3c0 con 0x55fef148e680 2024-09-19T07:02:28.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75b6ffd640 1 --2- 172.21.15.103:0/2590807694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef1483dc0 0x55fef148b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:28.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f75b7fff640 1 --2- 172.21.15.103:0/2590807694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef148e680 0x55fef148b6c0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f75b002f8d0 tx=0x7f75b002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a802f68e1511e3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:28.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.212+0000 7f7596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:28.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.213+0000 7f7596ffd640 1 -- 172.21.15.103:0/2590807694 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b004d070 con 0x55fef148e680 2024-09-19T07:02:28.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.213+0000 7f75c012d740 1 -- 172.21.15.103:0/2590807694 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75a8017ca0 con 0x55fef148e680 2024-09-19T07:02:28.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.213+0000 7f7596ffd640 1 -- 172.21.15.103:0/2590807694 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f75b0004450 con 0x55fef148e680 2024-09-19T07:02:28.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.213+0000 7f7596ffd640 1 -- 172.21.15.103:0/2590807694 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f75b0034040 con 0x55fef148e680 2024-09-19T07:02:28.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.213+0000 7f75c012d740 10 client.20022 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:28.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.213+0000 7f75c012d740 1 -- 172.21.15.103:0/2590807694 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fef149ca90 con 0x55fef148e680 2024-09-19T07:02:28.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.213+0000 7f7596ffd640 1 -- 172.21.15.103:0/2590807694 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a8017ca0 con 0x55fef148e680 2024-09-19T07:02:28.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.213+0000 7f7596ffd640 1 -- 172.21.15.103:0/2590807694 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f75b0030440 con 0x55fef148e680 2024-09-19T07:02:28.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.213+0000 7f75c012d740 10 client.20022 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:28.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.213+0000 7f75c012d740 10 client.20022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:28.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.213+0000 7f75c012d740 20 client.20022 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:28.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.213+0000 7f75c012d740 1 --2- 172.21.15.103:0/2590807694 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fef149dea0 0x55fef14a0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.213+0000 7f75c012d740 4 client.20022 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:28.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.213+0000 7f75c012d740 1 -- 172.21.15.103:0/2590807694 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fef13f6850 con 0x55fef149dea0 2024-09-19T07:02:28.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.213+0000 7f75b77fe640 1 --2- 172.21.15.103:0/2590807694 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fef149dea0 0x55fef14a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.214+0000 7f75b77fe640 1 --2- 172.21.15.103:0/2590807694 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fef149dea0 0x55fef14a0310 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de97b5b9f9dc2a94 server_cookie=6ee6299955d34cdf in_seq=0 out_seq=0 2024-09-19T07:02:28.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.214+0000 7f7596ffd640 10 client.20022 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:28.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.215+0000 7f7596ffd640 1 -- 172.21.15.103:0/2590807694 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fef13f6850 con 0x55fef149dea0 2024-09-19T07:02:28.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.215+0000 7f7596ffd640 10 client.20022 handle_command_reply: tid=0 2024-09-19T07:02:28.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.215+0000 7f75c012d740 1 client.20022 shutdown 2024-09-19T07:02:28.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.215+0000 7f75c012d740 1 -- 172.21.15.103:0/2590807694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef148e680 msgr2=0x55fef148b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:28.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.215+0000 7f75c012d740 1 --2- 172.21.15.103:0/2590807694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef148e680 0x55fef148b6c0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f75b002f8d0 tx=0x7f75b002a950 comp rx=0 tx=0).stop 2024-09-19T07:02:28.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.215+0000 7f75c012d740 1 -- 172.21.15.103:0/2590807694 shutdown_connections 2024-09-19T07:02:28.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.215+0000 7f75c012d740 1 --2- 172.21.15.103:0/2590807694 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fef149dea0 0x55fef14a0310 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.215+0000 7f75c012d740 1 --2- 172.21.15.103:0/2590807694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef148e680 0x55fef148b6c0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.215+0000 7f75c012d740 1 --2- 172.21.15.103:0/2590807694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef1483dc0 0x55fef148b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.215+0000 7f7596ffd640 0 client.20022 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:28.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.215+0000 7f75c012d740 1 --2- 172.21.15.103:0/2590807694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef14834a0 0x55fef148ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.215+0000 7f75c012d740 1 -- 172.21.15.103:0/2590807694 >> 172.21.15.103:0/2590807694 conn(0x55fef147fa30 msgr2=0x55fef1485760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:28.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.215+0000 7f75c012d740 1 -- 172.21.15.103:0/2590807694 shutdown_connections 2024-09-19T07:02:28.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.215+0000 7f75c012d740 1 -- 172.21.15.103:0/2590807694 wait complete. 2024-09-19T07:02:28.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.215+0000 7f75c012d740 20 client.20022 trim_cache size 0 max 16384 2024-09-19T07:02:28.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75c012d740 1 Processor -- start 2024-09-19T07:02:28.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75c012d740 1 -- start start 2024-09-19T07:02:28.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75c012d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef147fa30 0x55fef148d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75c012d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef14834a0 0x55fef14a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75c012d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef1483dc0 0x55fef14a6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75c012d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81ad860 con 0x55fef1483dc0 2024-09-19T07:02:28.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75c012d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81af3a0 con 0x55fef147fa30 2024-09-19T07:02:28.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75c012d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81adf30 con 0x55fef14834a0 2024-09-19T07:02:28.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75b7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef1483dc0 0x55fef14a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75b7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef1483dc0 0x55fef14a6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60964/0 (socket says 172.21.15.103:60964) 2024-09-19T07:02:28.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75b7fff640 1 -- 172.21.15.103:0/77655632 learned_addr learned my addr 172.21.15.103:0/77655632 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:28.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75b6ffd640 1 --2- 172.21.15.103:0/77655632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef14834a0 0x55fef14a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75b77fe640 1 --2- 172.21.15.103:0/77655632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef147fa30 0x55fef148d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75b7fff640 1 -- 172.21.15.103:0/77655632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef14834a0 msgr2=0x55fef14a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:28.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75b7fff640 1 --2- 172.21.15.103:0/77655632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef14834a0 0x55fef14a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75b7fff640 1 -- 172.21.15.103:0/77655632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef147fa30 msgr2=0x55fef148d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:28.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75b7fff640 1 --2- 172.21.15.103:0/77655632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef147fa30 0x55fef148d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.232+0000 7f75b7fff640 1 -- 172.21.15.103:0/77655632 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fef147f860 con 0x55fef1483dc0 2024-09-19T07:02:28.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.233+0000 7f75b7fff640 1 --2- 172.21.15.103:0/77655632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef1483dc0 0x55fef14a6120 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f75b002ae70 tx=0x7f75b0003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4f9fd559ced0db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:28.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.233+0000 7f759d7fa640 1 -- 172.21.15.103:0/77655632 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b005f030 con 0x55fef1483dc0 2024-09-19T07:02:28.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.233+0000 7f759d7fa640 1 -- 172.21.15.103:0/77655632 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b0004730 con 0x55fef1483dc0 2024-09-19T07:02:28.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.233+0000 7f759d7fa640 1 -- 172.21.15.103:0/77655632 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b004d040 con 0x55fef1483dc0 2024-09-19T07:02:28.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.233+0000 7f75c012d740 1 -- 172.21.15.103:0/77655632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef1483dc0 msgr2=0x55fef14a6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:28.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.233+0000 7f75c012d740 1 --2- 172.21.15.103:0/77655632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef1483dc0 0x55fef14a6120 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f75b002ae70 tx=0x7f75b0003dc0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.233+0000 7f75c012d740 1 -- 172.21.15.103:0/77655632 shutdown_connections 2024-09-19T07:02:28.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.233+0000 7f75c012d740 1 --2- 172.21.15.103:0/77655632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef1483dc0 0x55fef14a6120 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.233+0000 7f75c012d740 1 --2- 172.21.15.103:0/77655632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef14834a0 0x55fef14a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.233+0000 7f75c012d740 1 --2- 172.21.15.103:0/77655632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef147fa30 0x55fef148d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.233+0000 7f75c012d740 1 -- 172.21.15.103:0/77655632 >> 172.21.15.103:0/77655632 conn(0x55fef148b820 msgr2=0x55fef148c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:28.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.234+0000 7f75c012d740 1 -- 172.21.15.103:0/77655632 shutdown_connections 2024-09-19T07:02:28.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.234+0000 7f75c012d740 1 -- 172.21.15.103:0/77655632 wait complete. 2024-09-19T07:02:28.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.234+0000 7f75c012d740 1 -- start start 2024-09-19T07:02:28.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.234+0000 7f75c012d740 1 Processor -- start 2024-09-19T07:02:28.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.234+0000 7f75c012d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef147fa30 0x55fef1496c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.234+0000 7f75c012d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef14834a0 0x55fef1493f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.234+0000 7f75c012d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef1483dc0 0x55fef14944d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.234+0000 7f75c012d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81aecd0 con 0x55fef147fa30 2024-09-19T07:02:28.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.234+0000 7f75c012d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81ae600 con 0x55fef14834a0 2024-09-19T07:02:28.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.234+0000 7f75c012d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81ab860 con 0x55fef1483dc0 2024-09-19T07:02:28.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.234+0000 7f75c012d740 10 client.0 _ll_register_callbacks cb 0x7f75b0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:28.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f75b77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef147fa30 0x55fef1496c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f75b77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef147fa30 0x55fef1496c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60968/0 (socket says 172.21.15.103:60968) 2024-09-19T07:02:28.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f75b77fe640 1 -- 172.21.15.103:0/2848440123 learned_addr learned my addr 172.21.15.103:0/2848440123 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:28.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f75b6ffd640 1 --2- 172.21.15.103:0/2848440123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef14834a0 0x55fef1493f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f75b7fff640 1 --2- 172.21.15.103:0/2848440123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef1483dc0 0x55fef14944d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f75b77fe640 1 -- 172.21.15.103:0/2848440123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef1483dc0 msgr2=0x55fef14944d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:28.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f75b77fe640 1 --2- 172.21.15.103:0/2848440123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef1483dc0 0x55fef14944d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f75b77fe640 1 -- 172.21.15.103:0/2848440123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef14834a0 msgr2=0x55fef1493f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:28.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f75b77fe640 1 --2- 172.21.15.103:0/2848440123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef14834a0 0x55fef1493f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f75b77fe640 1 -- 172.21.15.103:0/2848440123 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fef1494bc0 con 0x55fef147fa30 2024-09-19T07:02:28.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f75b77fe640 1 --2- 172.21.15.103:0/2848440123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef147fa30 0x55fef1496c40 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f75a800f910 tx=0x7f75a800f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1456aafa3d6f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:28.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f7596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:28.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f7596ffd640 1 -- 172.21.15.103:0/2848440123 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8030060 con 0x55fef147fa30 2024-09-19T07:02:28.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f7596ffd640 1 -- 172.21.15.103:0/2848440123 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f75a800f9b0 con 0x55fef147fa30 2024-09-19T07:02:28.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f75c012d740 1 -- 172.21.15.103:0/2848440123 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75b0004730 con 0x55fef147fa30 2024-09-19T07:02:28.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f7596ffd640 1 -- 172.21.15.103:0/2848440123 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f75a8005e50 con 0x55fef147fa30 2024-09-19T07:02:28.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f75c012d740 10 client.20028 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:28.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.235+0000 7f75c012d740 1 -- 172.21.15.103:0/2848440123 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fef1508af0 con 0x55fef147fa30 2024-09-19T07:02:28.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.236+0000 7f7596ffd640 1 -- 172.21.15.103:0/2848440123 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a800b3c0 con 0x55fef147fa30 2024-09-19T07:02:28.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.236+0000 7f7596ffd640 1 -- 172.21.15.103:0/2848440123 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f75a8004770 con 0x55fef147fa30 2024-09-19T07:02:28.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.236+0000 7f75c012d740 10 client.20028 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:28.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.236+0000 7f75c012d740 10 client.20028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:28.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.236+0000 7f75c012d740 20 client.20028 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:28.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.236+0000 7f75c012d740 1 --2- 172.21.15.103:0/2848440123 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fef148e680 0x55fef150b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:28.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.236+0000 7f75c012d740 4 client.20028 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:02:28.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.236+0000 7f75c012d740 1 -- 172.21.15.103:0/2848440123 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f75b0004730 con 0x55fef148e680 2024-09-19T07:02:28.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.237+0000 7f75b6ffd640 1 --2- 172.21.15.103:0/2848440123 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fef148e680 0x55fef150b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:28.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.237+0000 7f7596ffd640 10 client.20028 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:28.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.237+0000 7f75b6ffd640 1 --2- 172.21.15.103:0/2848440123 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fef148e680 0x55fef150b0f0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6374a31bef2fe6b5 server_cookie=b75dd73d064737e9 in_seq=0 out_seq=0 2024-09-19T07:02:28.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.237+0000 7f7596ffd640 1 -- 172.21.15.103:0/2848440123 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f75b0004730 con 0x55fef148e680 2024-09-19T07:02:28.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.237+0000 7f7596ffd640 10 client.20028 handle_command_reply: tid=0 2024-09-19T07:02:28.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.237+0000 7f75c012d740 1 client.20028 shutdown 2024-09-19T07:02:28.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.238+0000 7f75c012d740 1 -- 172.21.15.103:0/2848440123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef147fa30 msgr2=0x55fef1496c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:28.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.238+0000 7f75c012d740 1 --2- 172.21.15.103:0/2848440123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef147fa30 0x55fef1496c40 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f75a800f910 tx=0x7f75a800f8a0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.238+0000 7f75c012d740 1 -- 172.21.15.103:0/2848440123 shutdown_connections 2024-09-19T07:02:28.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.238+0000 7f75c012d740 1 --2- 172.21.15.103:0/2848440123 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fef148e680 0x55fef150b0f0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.238+0000 7f75c012d740 1 --2- 172.21.15.103:0/2848440123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fef1483dc0 0x55fef14944d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.238+0000 7f75c012d740 1 --2- 172.21.15.103:0/2848440123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fef14834a0 0x55fef1493f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.238+0000 7f75c012d740 1 --2- 172.21.15.103:0/2848440123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fef147fa30 0x55fef1496c40 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.238+0000 7f75c012d740 1 -- 172.21.15.103:0/2848440123 >> 172.21.15.103:0/2848440123 conn(0x55fef148b820 msgr2=0x55fef147e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:28.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.238+0000 7f7596ffd640 0 client.20028 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:28.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.238+0000 7f75c012d740 1 -- 172.21.15.103:0/2848440123 shutdown_connections 2024-09-19T07:02:28.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.238+0000 7f75c012d740 1 -- 172.21.15.103:0/2848440123 wait complete. 2024-09-19T07:02:28.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.238+0000 7f75c012d740 20 client.20028 trim_cache size 0 max 16384 2024-09-19T07:02:28.244 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:02:28.244 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:02:28.245 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:02:28.245 INFO:teuthology.orchestra.run.smithi103.stdout: "478db48c-b5aa-4d57-ad26-f644b37e7850": { 2024-09-19T07:02:28.245 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:02:28.245 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "478db48c-b5aa-4d57-ad26-f644b37e7850", 2024-09-19T07:02:28.245 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:02:28.245 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:28.245 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:28.245 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:02:28.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.239+0000 7f75bdd54640 1 -- 172.21.15.103:0/1738294518 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75a00a3430 msgr2=0x7f75a00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:28.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.239+0000 7f75bdd54640 1 --2- 172.21.15.103:0/1738294518 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75a00a3430 0x7f75a00a58f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f75b0005e30 tx=0x7f75b0005d80 comp rx=0 tx=0).stop 2024-09-19T07:02:28.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.239+0000 7f75bdd54640 1 -- 172.21.15.103:0/1738294518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102570 msgr2=0x7f75b8199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:28.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.239+0000 7f75bdd54640 1 --2- 172.21.15.103:0/1738294518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102570 0x7f75b8199770 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00d950 tx=0x7f75ac00de10 comp rx=0 tx=0).stop 2024-09-19T07:02:28.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.239+0000 7f75bdd54640 1 -- 172.21.15.103:0/1738294518 shutdown_connections 2024-09-19T07:02:28.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.239+0000 7f75bdd54640 1 --2- 172.21.15.103:0/1738294518 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75a00a3430 0x7f75a00a58f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.239+0000 7f75bdd54640 1 --2- 172.21.15.103:0/1738294518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b8102ef0 0x7f75b81a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.239+0000 7f75bdd54640 1 --2- 172.21.15.103:0/1738294518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102570 0x7f75b8199770 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.239+0000 7f75bdd54640 1 --2- 172.21.15.103:0/1738294518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b8100e20 0x7f75b8199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:28.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.239+0000 7f75bdd54640 1 -- 172.21.15.103:0/1738294518 >> 172.21.15.103:0/1738294518 conn(0x7f75b80fc7d0 msgr2=0x7f75b81067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:28.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.239+0000 7f75bdd54640 1 -- 172.21.15.103:0/1738294518 shutdown_connections 2024-09-19T07:02:28.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:28.239+0000 7f75bdd54640 1 -- 172.21.15.103:0/1738294518 wait complete. 2024-09-19T07:02:28.830 INFO:tasks.cephfs.filesystem:scrub status for tag:478db48c-b5aa-4d57-ad26-f644b37e7850 - {'path': '/', 'tag': '478db48c-b5aa-4d57-ad26-f644b37e7850', 'options': 'recursive,force'} 2024-09-19T07:02:28.936 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:29.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 -- 172.21.15.103:0/3170871695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec102790 msgr2=0x7f3fec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:29.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 --2- 172.21.15.103:0/3170871695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec102790 0x7f3fec1067e0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009900 tx=0x7f3fd402a410 comp rx=0 tx=0).stop 2024-09-19T07:02:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 -- 172.21.15.103:0/3170871695 shutdown_connections 2024-09-19T07:02:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 --2- 172.21.15.103:0/3170871695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fec106f10 0x7f3fec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 --2- 172.21.15.103:0/3170871695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec102790 0x7f3fec1067e0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 --2- 172.21.15.103:0/3170871695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fec101de0 0x7f3fec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 -- 172.21.15.103:0/3170871695 >> 172.21.15.103:0/3170871695 conn(0x7f3fec0fd8c0 msgr2=0x7f3fec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 -- 172.21.15.103:0/3170871695 shutdown_connections 2024-09-19T07:02:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 -- 172.21.15.103:0/3170871695 wait complete. 2024-09-19T07:02:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 Processor -- start 2024-09-19T07:02:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 -- start start 2024-09-19T07:02:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec101de0 0x7f3fec1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fec102790 0x7f3fec198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fec106f10 0x7f3fec19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec112010 con 0x7f3fec101de0 2024-09-19T07:02:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec111e90 con 0x7f3fec106f10 2024-09-19T07:02:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.417+0000 7f3ff30a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec112190 con 0x7f3fec102790 2024-09-19T07:02:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3ff0e3e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec101de0 0x7f3fec1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3ff0e3e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec101de0 0x7f3fec1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60994/0 (socket says 172.21.15.103:60994) 2024-09-19T07:02:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3ff0e3e640 1 -- 172.21.15.103:0/918967505 learned_addr learned my addr 172.21.15.103:0/918967505 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3ff163f640 1 --2- 172.21.15.103:0/918967505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fec106f10 0x7f3fec19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3febfff640 1 --2- 172.21.15.103:0/918967505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fec102790 0x7f3fec198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3ff0e3e640 1 -- 172.21.15.103:0/918967505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fec102790 msgr2=0x7f3fec198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3ff0e3e640 1 --2- 172.21.15.103:0/918967505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fec102790 0x7f3fec198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3ff0e3e640 1 -- 172.21.15.103:0/918967505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fec106f10 msgr2=0x7f3fec19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3ff0e3e640 1 --2- 172.21.15.103:0/918967505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fec106f10 0x7f3fec19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3ff0e3e640 1 -- 172.21.15.103:0/918967505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fec19d5a0 con 0x7f3fec101de0 2024-09-19T07:02:29.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3febfff640 1 --2- 172.21.15.103:0/918967505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fec102790 0x7f3fec198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:29.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3ff0e3e640 1 --2- 172.21.15.103:0/918967505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec101de0 0x7f3fec1985f0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00b4a0 tx=0x7f3fdc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:29.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3fe9ffb640 1 -- 172.21.15.103:0/918967505 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc014070 con 0x7f3fec101de0 2024-09-19T07:02:29.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3fe9ffb640 1 -- 172.21.15.103:0/918967505 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fdc010af0 con 0x7f3fec101de0 2024-09-19T07:02:29.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3ff30a2640 1 -- 172.21.15.103:0/918967505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fec19d890 con 0x7f3fec101de0 2024-09-19T07:02:29.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3fe9ffb640 1 -- 172.21.15.103:0/918967505 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc017d10 con 0x7f3fec101de0 2024-09-19T07:02:29.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.418+0000 7f3ff30a2640 1 -- 172.21.15.103:0/918967505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fec1a5170 con 0x7f3fec101de0 2024-09-19T07:02:29.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.420+0000 7f3fe9ffb640 1 -- 172.21.15.103:0/918967505 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3fdc017070 con 0x7f3fec101de0 2024-09-19T07:02:29.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.421+0000 7f3ff547b740 1 Processor -- start 2024-09-19T07:02:29.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.421+0000 7f3ff547b740 1 -- start start 2024-09-19T07:02:29.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.421+0000 7f3fe9ffb640 1 --2- 172.21.15.103:0/918967505 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3fd80a33b0 0x7f3fd80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.421+0000 7f3ff547b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4086060 0x55f7f4086440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.421+0000 7f3ff547b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 0x55f7f4090d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.421+0000 7f3ff547b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4091240 0x55f7f4093630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.421+0000 7f3ff547b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f7f3feb620 con 0x55f7f4091240 2024-09-19T07:02:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.421+0000 7f3ff547b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f7f3f92e50 con 0x55f7f4086060 2024-09-19T07:02:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.421+0000 7f3ff547b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f7f3fe75e0 con 0x55f7f4086980 2024-09-19T07:02:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.421+0000 7f3fe9ffb640 1 -- 172.21.15.103:0/918967505 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f3fdc0cf000 con 0x7f3fec101de0 2024-09-19T07:02:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.421+0000 7f3ff163f640 1 --2- 172.21.15.103:0/918967505 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3fd80a33b0 0x7f3fd80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.421+0000 7f3ff163f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4091240 0x55f7f4093630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.421+0000 7f3ff0e3e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4086060 0x55f7f4086440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.422+0000 7f3ff163f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4091240 0x55f7f4093630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32768/0 (socket says 172.21.15.103:32768) 2024-09-19T07:02:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.422+0000 7f3ff163f640 1 -- 172.21.15.103:0/2914346812 learned_addr learned my addr 172.21.15.103:0/2914346812 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.422+0000 7f3ff0e3e640 1 --2- 172.21.15.103:0/2914346812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4086060 0x55f7f4086440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45518/0 (socket says 172.21.15.103:45518) 2024-09-19T07:02:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.422+0000 7f3febfff640 1 --2- 172.21.15.103:0/2914346812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 0x55f7f4090d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.422+0000 7f3ff0e3e640 1 -- 172.21.15.103:0/2914346812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 msgr2=0x55f7f4090d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.422+0000 7f3ff0e3e640 1 --2- 172.21.15.103:0/2914346812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 0x55f7f4090d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.422+0000 7f3ff0e3e640 1 -- 172.21.15.103:0/2914346812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4091240 msgr2=0x55f7f4093630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.422+0000 7f3ff0e3e640 1 --2- 172.21.15.103:0/2914346812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4091240 0x55f7f4093630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.422+0000 7f3ff0e3e640 1 -- 172.21.15.103:0/2914346812 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7f3f83d40 con 0x55f7f4086060 2024-09-19T07:02:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.422+0000 7f3ff163f640 1 --2- 172.21.15.103:0/2914346812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4091240 0x55f7f4093630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:02:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.422+0000 7f3ff163f640 1 --2- 172.21.15.103:0/918967505 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3fd80a33b0 0x7f3fd80a5870 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400bdd0 tx=0x7f3fe400b410 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.422+0000 7f3ff0e3e640 1 --2- 172.21.15.103:0/2914346812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4086060 0x55f7f4086440 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc0cc840 tx=0x7f3fdc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7967f15748c7c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.422+0000 7f3fe2ffd640 1 -- 172.21.15.103:0/2914346812 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc01e070 con 0x55f7f4086060 2024-09-19T07:02:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.422+0000 7f3fe2ffd640 1 -- 172.21.15.103:0/2914346812 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fdc049e50 con 0x55f7f4086060 2024-09-19T07:02:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.422+0000 7f3fe2ffd640 1 -- 172.21.15.103:0/2914346812 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc010ca0 con 0x55f7f4086060 2024-09-19T07:02:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.423+0000 7f3ff547b740 1 -- 172.21.15.103:0/2914346812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4086060 msgr2=0x55f7f4086440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.423+0000 7f3ff547b740 1 --2- 172.21.15.103:0/2914346812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4086060 0x55f7f4086440 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc0cc840 tx=0x7f3fdc0046c0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.423+0000 7f3ff547b740 1 -- 172.21.15.103:0/2914346812 shutdown_connections 2024-09-19T07:02:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.423+0000 7f3ff547b740 1 --2- 172.21.15.103:0/2914346812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4091240 0x55f7f4093630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.423+0000 7f3ff547b740 1 --2- 172.21.15.103:0/2914346812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 0x55f7f4090d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.423+0000 7f3ff547b740 1 --2- 172.21.15.103:0/2914346812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4086060 0x55f7f4086440 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.423+0000 7f3ff547b740 1 -- 172.21.15.103:0/2914346812 >> 172.21.15.103:0/2914346812 conn(0x55f7f40825f0 msgr2=0x55f7f40829d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.423+0000 7f3ff547b740 1 -- 172.21.15.103:0/2914346812 shutdown_connections 2024-09-19T07:02:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.423+0000 7f3ff547b740 1 -- 172.21.15.103:0/2914346812 wait complete. 2024-09-19T07:02:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.423+0000 7f3ff547b740 1 -- start start 2024-09-19T07:02:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff547b740 1 Processor -- start 2024-09-19T07:02:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff547b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4086060 0x55f7f408e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff547b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 0x55f7f408e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff547b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4091240 0x55f7f408ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff547b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f7f4093e10 con 0x55f7f4086060 2024-09-19T07:02:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff547b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f7f3f73f10 con 0x55f7f4091240 2024-09-19T07:02:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff547b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f7f4093f90 con 0x55f7f4086980 2024-09-19T07:02:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff547b740 10 client.0 _ll_register_callbacks cb 0x55f7f3f97db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff0e3e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4086060 0x55f7f408e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff0e3e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4086060 0x55f7f408e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32772/0 (socket says 172.21.15.103:32772) 2024-09-19T07:02:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff0e3e640 1 -- 172.21.15.103:0/2868385313 learned_addr learned my addr 172.21.15.103:0/2868385313 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3febfff640 1 --2- 172.21.15.103:0/2868385313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 0x55f7f408e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff163f640 1 --2- 172.21.15.103:0/2868385313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4091240 0x55f7f408ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff0e3e640 1 -- 172.21.15.103:0/2868385313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 msgr2=0x55f7f408e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff0e3e640 1 --2- 172.21.15.103:0/2868385313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 0x55f7f408e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff0e3e640 1 -- 172.21.15.103:0/2868385313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4091240 msgr2=0x55f7f408ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff0e3e640 1 --2- 172.21.15.103:0/2868385313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4091240 0x55f7f408ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff0e3e640 1 -- 172.21.15.103:0/2868385313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7f3cfa580 con 0x55f7f4086060 2024-09-19T07:02:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3ff0e3e640 1 --2- 172.21.15.103:0/2868385313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4086060 0x55f7f408e170 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00be60 tx=0x7f3fdc017590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7279dc2b7323306 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.424+0000 7f3fcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.425+0000 7f3fcbfff640 1 -- 172.21.15.103:0/2868385313 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc01e070 con 0x55f7f4086060 2024-09-19T07:02:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.425+0000 7f3fcbfff640 1 -- 172.21.15.103:0/2868385313 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f3fdc0cc9b0 con 0x55f7f4086060 2024-09-19T07:02:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.425+0000 7f3ff547b740 1 -- 172.21.15.103:0/2868385313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fdc049e50 con 0x55f7f4086060 2024-09-19T07:02:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.425+0000 7f3fcbfff640 1 -- 172.21.15.103:0/2868385313 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3fdc09bad0 con 0x55f7f4086060 2024-09-19T07:02:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.425+0000 7f3ff547b740 10 client.20040 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.425+0000 7f3ff547b740 1 -- 172.21.15.103:0/2868385313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f7f3dc4960 con 0x55f7f4086060 2024-09-19T07:02:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.425+0000 7f3fcbfff640 1 -- 172.21.15.103:0/2868385313 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fdc049e50 con 0x55f7f4086060 2024-09-19T07:02:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.425+0000 7f3fcbfff640 1 -- 172.21.15.103:0/2868385313 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3fdc0b4210 con 0x55f7f4086060 2024-09-19T07:02:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.425+0000 7f3ff547b740 10 client.20040 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.425+0000 7f3ff547b740 10 client.20040 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:02:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.425+0000 7f3ff547b740 20 client.20040 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.425+0000 7f3ff547b740 1 --2- 172.21.15.103:0/2868385313 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f7f40a0c90 0x55f7f40a3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.425+0000 7f3ff547b740 4 client.20040 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.425+0000 7f3ff547b740 1 -- 172.21.15.103:0/2868385313 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7f3ff8b90 con 0x55f7f40a0c90 2024-09-19T07:02:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.426+0000 7f3febfff640 1 --2- 172.21.15.103:0/2868385313 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f7f40a0c90 0x55f7f40a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.426+0000 7f3fcbfff640 10 client.20040 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.426+0000 7f3febfff640 1 --2- 172.21.15.103:0/2868385313 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f7f40a0c90 0x55f7f40a3080 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bcd10b598d46531 server_cookie=9d56b179d7e32833 in_seq=0 out_seq=0 2024-09-19T07:02:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.427+0000 7f3fcbfff640 1 -- 172.21.15.103:0/2868385313 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7f3ff8b90 con 0x55f7f40a0c90 2024-09-19T07:02:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.427+0000 7f3fcbfff640 10 client.20040 handle_command_reply: tid=0 2024-09-19T07:02:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.427+0000 7f3ff547b740 1 client.20040 shutdown 2024-09-19T07:02:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.427+0000 7f3ff547b740 1 -- 172.21.15.103:0/2868385313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4086060 msgr2=0x55f7f408e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.427+0000 7f3ff547b740 1 --2- 172.21.15.103:0/2868385313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4086060 0x55f7f408e170 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00be60 tx=0x7f3fdc017590 comp rx=0 tx=0).stop 2024-09-19T07:02:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.427+0000 7f3ff547b740 1 -- 172.21.15.103:0/2868385313 shutdown_connections 2024-09-19T07:02:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.427+0000 7f3ff547b740 1 --2- 172.21.15.103:0/2868385313 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f7f40a0c90 0x55f7f40a3080 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.427+0000 7f3ff547b740 1 --2- 172.21.15.103:0/2868385313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4091240 0x55f7f408ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.427+0000 7f3ff547b740 1 --2- 172.21.15.103:0/2868385313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 0x55f7f408e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.427+0000 7f3fcbfff640 0 client.20040 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.427+0000 7f3ff547b740 1 --2- 172.21.15.103:0/2868385313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4086060 0x55f7f408e170 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.427+0000 7f3ff547b740 1 -- 172.21.15.103:0/2868385313 >> 172.21.15.103:0/2868385313 conn(0x55f7f40825f0 msgr2=0x55f7f4088320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.427+0000 7f3ff547b740 1 -- 172.21.15.103:0/2868385313 shutdown_connections 2024-09-19T07:02:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.427+0000 7f3ff547b740 1 -- 172.21.15.103:0/2868385313 wait complete. 2024-09-19T07:02:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.427+0000 7f3ff547b740 20 client.20040 trim_cache size 0 max 16384 2024-09-19T07:02:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.444+0000 7f3ff547b740 1 Processor -- start 2024-09-19T07:02:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.444+0000 7f3ff547b740 1 -- start start 2024-09-19T07:02:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.444+0000 7f3ff547b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f40825f0 0x55f7f4081a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.444+0000 7f3ff547b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4086060 0x55f7f409c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.444+0000 7f3ff547b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 0x55f7f40a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.444+0000 7f3ff547b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec1ada80 con 0x55f7f4086060 2024-09-19T07:02:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.444+0000 7f3ff547b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec1af5c0 con 0x55f7f40825f0 2024-09-19T07:02:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.444+0000 7f3ff547b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec1ae150 con 0x55f7f4086980 2024-09-19T07:02:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.444+0000 7f3febfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4086060 0x55f7f409c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.444+0000 7f3febfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4086060 0x55f7f409c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32776/0 (socket says 172.21.15.103:32776) 2024-09-19T07:02:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.444+0000 7f3febfff640 1 -- 172.21.15.103:0/686398002 learned_addr learned my addr 172.21.15.103:0/686398002 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.444+0000 7f3febfff640 1 -- 172.21.15.103:0/686398002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 msgr2=0x55f7f40a60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.444+0000 7f3febfff640 1 --2- 172.21.15.103:0/686398002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 0x55f7f40a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.444+0000 7f3febfff640 1 -- 172.21.15.103:0/686398002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f40825f0 msgr2=0x55f7f4081a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.444+0000 7f3febfff640 1 --2- 172.21.15.103:0/686398002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f40825f0 0x55f7f4081a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.444+0000 7f3febfff640 1 -- 172.21.15.103:0/686398002 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7f40836a0 con 0x55f7f4086060 2024-09-19T07:02:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.445+0000 7f3febfff640 1 --2- 172.21.15.103:0/686398002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4086060 0x55f7f409c8f0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4005ab0 tx=0x7f3fd4005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4eabb2b09c027f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.445+0000 7f3fe27fc640 1 -- 172.21.15.103:0/686398002 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4035040 con 0x55f7f4086060 2024-09-19T07:02:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.445+0000 7f3fe27fc640 1 -- 172.21.15.103:0/686398002 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fd4005520 con 0x55f7f4086060 2024-09-19T07:02:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.445+0000 7f3fe27fc640 1 -- 172.21.15.103:0/686398002 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4004cb0 con 0x55f7f4086060 2024-09-19T07:02:29.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.445+0000 7f3ff547b740 1 -- 172.21.15.103:0/686398002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4086060 msgr2=0x55f7f409c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:29.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.445+0000 7f3ff547b740 1 --2- 172.21.15.103:0/686398002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4086060 0x55f7f409c8f0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4005ab0 tx=0x7f3fd4005a40 comp rx=0 tx=0).stop 2024-09-19T07:02:29.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.445+0000 7f3ff547b740 1 -- 172.21.15.103:0/686398002 shutdown_connections 2024-09-19T07:02:29.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.445+0000 7f3ff547b740 1 --2- 172.21.15.103:0/686398002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 0x55f7f40a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.445+0000 7f3ff547b740 1 --2- 172.21.15.103:0/686398002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f4086060 0x55f7f409c8f0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.445+0000 7f3ff547b740 1 --2- 172.21.15.103:0/686398002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f40825f0 0x55f7f4081a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.446+0000 7f3ff547b740 1 -- 172.21.15.103:0/686398002 >> 172.21.15.103:0/686398002 conn(0x55f7f4090600 msgr2=0x55f7f409c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:29.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.446+0000 7f3ff547b740 1 -- 172.21.15.103:0/686398002 shutdown_connections 2024-09-19T07:02:29.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.446+0000 7f3ff547b740 1 -- 172.21.15.103:0/686398002 wait complete. 2024-09-19T07:02:29.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.446+0000 7f3ff547b740 1 -- start start 2024-09-19T07:02:29.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.446+0000 7f3ff547b740 1 Processor -- start 2024-09-19T07:02:29.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.446+0000 7f3ff547b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f40825f0 0x55f7f4103340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.446+0000 7f3ff547b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4086060 0x55f7f4103880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.446+0000 7f3ff547b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 0x55f7f4107950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.446+0000 7f3ff547b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec1aeef0 con 0x55f7f40825f0 2024-09-19T07:02:29.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.446+0000 7f3ff547b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec1ae820 con 0x55f7f4086060 2024-09-19T07:02:29.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.446+0000 7f3ff547b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec1aba80 con 0x55f7f4086980 2024-09-19T07:02:29.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.446+0000 7f3ff547b740 10 client.0 _ll_register_callbacks cb 0x7f3fdc09af10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:29.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.446+0000 7f3ff0e3e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f40825f0 0x55f7f4103340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:29.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.446+0000 7f3ff0e3e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f40825f0 0x55f7f4103340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32780/0 (socket says 172.21.15.103:32780) 2024-09-19T07:02:29.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.446+0000 7f3ff0e3e640 1 -- 172.21.15.103:0/1454298375 learned_addr learned my addr 172.21.15.103:0/1454298375 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3febfff640 1 --2- 172.21.15.103:0/1454298375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4086060 0x55f7f4103880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3ff163f640 1 --2- 172.21.15.103:0/1454298375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 0x55f7f4107950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3ff0e3e640 1 -- 172.21.15.103:0/1454298375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 msgr2=0x55f7f4107950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3ff0e3e640 1 --2- 172.21.15.103:0/1454298375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 0x55f7f4107950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3ff0e3e640 1 -- 172.21.15.103:0/1454298375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4086060 msgr2=0x55f7f4103880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3ff0e3e640 1 --2- 172.21.15.103:0/1454298375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4086060 0x55f7f4103880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3ff0e3e640 1 -- 172.21.15.103:0/1454298375 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7f4107ff0 con 0x55f7f40825f0 2024-09-19T07:02:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3ff163f640 1 --2- 172.21.15.103:0/1454298375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 0x55f7f4107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3ff0e3e640 1 --2- 172.21.15.103:0/1454298375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f40825f0 0x55f7f4103340 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc09b0d0 tx=0x7f3fdc0998e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fdf79c671fb475 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3fcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3fcbfff640 1 -- 172.21.15.103:0/1454298375 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc01e070 con 0x55f7f40825f0 2024-09-19T07:02:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3fcbfff640 1 -- 172.21.15.103:0/1454298375 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f3fdc09a820 con 0x55f7f40825f0 2024-09-19T07:02:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3ff547b740 1 -- 172.21.15.103:0/1454298375 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fd4005520 con 0x55f7f40825f0 2024-09-19T07:02:29.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3fcbfff640 1 -- 172.21.15.103:0/1454298375 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3fdc0499f0 con 0x55f7f40825f0 2024-09-19T07:02:29.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3ff547b740 10 client.20046 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:29.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3ff547b740 1 -- 172.21.15.103:0/1454298375 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f7f41105c0 con 0x55f7f40825f0 2024-09-19T07:02:29.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.447+0000 7f3fcbfff640 1 -- 172.21.15.103:0/1454298375 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fd4005520 con 0x55f7f40825f0 2024-09-19T07:02:29.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.448+0000 7f3fcbfff640 1 -- 172.21.15.103:0/1454298375 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3fdc097120 con 0x55f7f40825f0 2024-09-19T07:02:29.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.448+0000 7f3ff547b740 10 client.20046 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:29.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.448+0000 7f3ff547b740 10 client.20046 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:02:29.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.448+0000 7f3ff547b740 20 client.20046 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:29.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.448+0000 7f3ff547b740 1 --2- 172.21.15.103:0/1454298375 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f7f4091240 0x55f7f4112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:29.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.448+0000 7f3ff547b740 4 client.20046 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "93884bb1", "release": true, "await": true}] 2024-09-19T07:02:29.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.448+0000 7f3ff547b740 1 -- 172.21.15.103:0/1454298375 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "set_id": "93884bb1", "release": true, "await": true}) -- 0x55f7f3ff30a0 con 0x55f7f4091240 2024-09-19T07:02:29.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.448+0000 7f3febfff640 1 --2- 172.21.15.103:0/1454298375 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f7f4091240 0x55f7f4112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:29.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.448+0000 7f3fcbfff640 10 client.20046 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:29.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.448+0000 7f3febfff640 1 --2- 172.21.15.103:0/1454298375 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f7f4091240 0x55f7f4112ab0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d727927d7169fdf6 server_cookie=455610e71588657e in_seq=0 out_seq=0 2024-09-19T07:02:29.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.454+0000 7f3fcbfff640 1 -- 172.21.15.103:0/1454298375 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f7f3ff30a0 con 0x55f7f4091240 2024-09-19T07:02:29.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.454+0000 7f3fcbfff640 10 client.20046 handle_command_reply: tid=0 2024-09-19T07:02:29.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.454+0000 7f3ff547b740 1 client.20046 shutdown 2024-09-19T07:02:29.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.454+0000 7f3ff547b740 1 -- 172.21.15.103:0/1454298375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f40825f0 msgr2=0x55f7f4103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:29.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.454+0000 7f3ff547b740 1 --2- 172.21.15.103:0/1454298375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f40825f0 0x55f7f4103340 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc09b0d0 tx=0x7f3fdc0998e0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.454+0000 7f3ff547b740 1 -- 172.21.15.103:0/1454298375 shutdown_connections 2024-09-19T07:02:29.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.454+0000 7f3ff547b740 1 --2- 172.21.15.103:0/1454298375 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f7f4091240 0x55f7f4112ab0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.454+0000 7f3ff547b740 1 --2- 172.21.15.103:0/1454298375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7f4086980 0x55f7f4107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.454+0000 7f3ff547b740 1 --2- 172.21.15.103:0/1454298375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7f4086060 0x55f7f4103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.454+0000 7f3ff547b740 1 --2- 172.21.15.103:0/1454298375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f7f40825f0 0x55f7f4103340 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.454+0000 7f3ff547b740 1 -- 172.21.15.103:0/1454298375 >> 172.21.15.103:0/1454298375 conn(0x55f7f4090600 msgr2=0x55f7f409ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:29.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.454+0000 7f3fcbfff640 0 client.20046 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:29.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.454+0000 7f3ff547b740 1 -- 172.21.15.103:0/1454298375 shutdown_connections 2024-09-19T07:02:29.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.454+0000 7f3ff547b740 1 -- 172.21.15.103:0/1454298375 wait complete. 2024-09-19T07:02:29.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.454+0000 7f3ff547b740 20 client.20046 trim_cache size 0 max 16384 2024-09-19T07:02:29.457 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:02:29.457 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T07:02:29.457 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T07:02:29.457 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 44, 2024-09-19T07:02:29.457 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:02:29.457 INFO:teuthology.orchestra.run.smithi103.stdout: "93884bb1": { 2024-09-19T07:02:29.457 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 44, 2024-09-19T07:02:29.458 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:02:29.458 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:02:29.458 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:02:29.458 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:02:29.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:02:29.458 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:02:29.458 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 142.2, 2024-09-19T07:02:29.458 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:02:29.458 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:02:29.458 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:02:29.458 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:02:29.458 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:02:29.458 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:02:29.458 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:29.459 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:29.459 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:29.459 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:29.459 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:29.459 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:02:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.455+0000 7f3ff30a2640 1 -- 172.21.15.103:0/918967505 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3fd80a33b0 msgr2=0x7f3fd80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.455+0000 7f3ff30a2640 1 --2- 172.21.15.103:0/918967505 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3fd80a33b0 0x7f3fd80a5870 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400bdd0 tx=0x7f3fe400b410 comp rx=0 tx=0).stop 2024-09-19T07:02:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.455+0000 7f3ff30a2640 1 -- 172.21.15.103:0/918967505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec101de0 msgr2=0x7f3fec1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.455+0000 7f3ff30a2640 1 --2- 172.21.15.103:0/918967505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec101de0 0x7f3fec1985f0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00b4a0 tx=0x7f3fdc00b960 comp rx=0 tx=0).stop 2024-09-19T07:02:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.455+0000 7f3ff30a2640 1 -- 172.21.15.103:0/918967505 shutdown_connections 2024-09-19T07:02:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.455+0000 7f3ff30a2640 1 --2- 172.21.15.103:0/918967505 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3fd80a33b0 0x7f3fd80a5870 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.455+0000 7f3ff30a2640 1 --2- 172.21.15.103:0/918967505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fec106f10 0x7f3fec19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.455+0000 7f3ff30a2640 1 --2- 172.21.15.103:0/918967505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fec102790 0x7f3fec198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.455+0000 7f3ff30a2640 1 --2- 172.21.15.103:0/918967505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec101de0 0x7f3fec1985f0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.455+0000 7f3ff30a2640 1 -- 172.21.15.103:0/918967505 >> 172.21.15.103:0/918967505 conn(0x7f3fec0fd8c0 msgr2=0x7f3fec0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.455+0000 7f3ff30a2640 1 -- 172.21.15.103:0/918967505 shutdown_connections 2024-09-19T07:02:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:29.455+0000 7f3ff30a2640 1 -- 172.21.15.103:0/918967505 wait complete. 2024-09-19T07:02:30.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:29 smithi174 ceph-mon[35637]: pgmap v414: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-09-19T07:02:30.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:29 smithi103 ceph-mon[31158]: pgmap v414: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-09-19T07:02:30.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:29 smithi134 ceph-mon[35356]: pgmap v414: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-09-19T07:02:30.190 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '93884bb1', total seconds elapsed: 26.9 2024-09-19T07:02:30.190 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.2 2024-09-19T07:02:30.190 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:02:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:30.806 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:30 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T07:02:30.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.885+0000 7f860b15a640 1 -- 172.21.15.103:0/2300898248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8604106f10 msgr2=0x7f860410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:30.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.885+0000 7f860b15a640 1 --2- 172.21.15.103:0/2300898248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8604106f10 0x7f860410ea90 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009ca0 tx=0x7f85fc02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:02:30.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.885+0000 7f860b15a640 1 -- 172.21.15.103:0/2300898248 shutdown_connections 2024-09-19T07:02:30.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.885+0000 7f860b15a640 1 --2- 172.21.15.103:0/2300898248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8604106f10 0x7f860410ea90 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:30.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.885+0000 7f860b15a640 1 --2- 172.21.15.103:0/2300898248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8604102790 0x7f86041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:30.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.885+0000 7f860b15a640 1 --2- 172.21.15.103:0/2300898248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8604101de0 0x7f86041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:30.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.885+0000 7f860b15a640 1 -- 172.21.15.103:0/2300898248 >> 172.21.15.103:0/2300898248 conn(0x7f86040fd8c0 msgr2=0x7f86040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:30.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.885+0000 7f860b15a640 1 -- 172.21.15.103:0/2300898248 shutdown_connections 2024-09-19T07:02:30.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.885+0000 7f860b15a640 1 -- 172.21.15.103:0/2300898248 wait complete. 2024-09-19T07:02:30.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.886+0000 7f860b15a640 1 Processor -- start 2024-09-19T07:02:30.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.886+0000 7f860b15a640 1 -- start start 2024-09-19T07:02:30.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.886+0000 7f860b15a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8604101de0 0x7f860419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:30.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.886+0000 7f860b15a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8604102790 0x7f860419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:30.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.886+0000 7f860b15a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8604106f10 0x7f860419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:30.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.886+0000 7f860b15a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8604111f70 con 0x7f8604101de0 2024-09-19T07:02:30.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.886+0000 7f860b15a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8604111df0 con 0x7f8604106f10 2024-09-19T07:02:30.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.886+0000 7f860b15a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86041120f0 con 0x7f8604102790 2024-09-19T07:02:30.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.886+0000 7f8608ef6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8604101de0 0x7f860419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:30.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.886+0000 7f8608ef6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8604101de0 0x7f860419a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32792/0 (socket says 172.21.15.103:32792) 2024-09-19T07:02:30.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.886+0000 7f8608ef6640 1 -- 172.21.15.103:0/3616250012 learned_addr learned my addr 172.21.15.103:0/3616250012 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:30.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.886+0000 7f8608ef6640 1 -- 172.21.15.103:0/3616250012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8604102790 msgr2=0x7f860419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:30.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.886+0000 7f8603fff640 1 --2- 172.21.15.103:0/3616250012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8604102790 0x7f860419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:30.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.887+0000 7f86096f7640 1 --2- 172.21.15.103:0/3616250012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8604106f10 0x7f860419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:30.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.887+0000 7f8608ef6640 1 --2- 172.21.15.103:0/3616250012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8604102790 0x7f860419aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:30.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.887+0000 7f8608ef6640 1 -- 172.21.15.103:0/3616250012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8604106f10 msgr2=0x7f860419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:30.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.887+0000 7f8608ef6640 1 --2- 172.21.15.103:0/3616250012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8604106f10 0x7f860419ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:30.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.887+0000 7f8608ef6640 1 -- 172.21.15.103:0/3616250012 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860419f530 con 0x7f8604101de0 2024-09-19T07:02:30.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.887+0000 7f8608ef6640 1 --2- 172.21.15.103:0/3616250012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8604101de0 0x7f860419a580 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c920 tx=0x7f85f800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:30.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.887+0000 7f8601ffb640 1 -- 172.21.15.103:0/3616250012 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8007c70 con 0x7f8604101de0 2024-09-19T07:02:30.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.887+0000 7f8601ffb640 1 -- 172.21.15.103:0/3616250012 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85f8007e10 con 0x7f8604101de0 2024-09-19T07:02:30.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.887+0000 7f860b15a640 1 -- 172.21.15.103:0/3616250012 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860419f820 con 0x7f8604101de0 2024-09-19T07:02:30.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.887+0000 7f8601ffb640 1 -- 172.21.15.103:0/3616250012 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8018890 con 0x7f8604101de0 2024-09-19T07:02:30.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.887+0000 7f860b15a640 1 -- 172.21.15.103:0/3616250012 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86041a7100 con 0x7f8604101de0 2024-09-19T07:02:30.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.888+0000 7f860b15a640 1 -- 172.21.15.103:0/3616250012 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86041021c0 con 0x7f8604101de0 2024-09-19T07:02:30.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.889+0000 7f8601ffb640 1 -- 172.21.15.103:0/3616250012 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f85f8018a70 con 0x7f8604101de0 2024-09-19T07:02:30.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.890+0000 7f8601ffb640 1 --2- 172.21.15.103:0/3616250012 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f85f40a33b0 0x7f85f40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:30.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.890+0000 7f8603fff640 1 --2- 172.21.15.103:0/3616250012 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f85f40a33b0 0x7f85f40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:30.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.890+0000 7f8601ffb640 1 -- 172.21.15.103:0/3616250012 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f85f80cff40 con 0x7f8604101de0 2024-09-19T07:02:30.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.890+0000 7f8603fff640 1 --2- 172.21.15.103:0/3616250012 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f85f40a33b0 0x7f85f40a5870 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8604076480 tx=0x7f85f0005ed0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:30.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:30.894+0000 7f8601ffb640 1 -- 172.21.15.103:0/3616250012 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f85f8014030 con 0x7f8604101de0 2024-09-19T07:02:31.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:30 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T07:02:31.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:31.095+0000 7f860b15a640 1 -- 172.21.15.103:0/3616250012 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f860419b8e0 con 0x7f8604101de0 2024-09-19T07:02:31.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:31.096+0000 7f8601ffb640 1 -- 172.21.15.103:0/3616250012 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f85f800de20 con 0x7f8604101de0 2024-09-19T07:02:31.097 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:02:31.097 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:02:31.097 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:02:31.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:31.098+0000 7f860b15a640 1 -- 172.21.15.103:0/3616250012 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f85f40a33b0 msgr2=0x7f85f40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:31.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:31.098+0000 7f860b15a640 1 --2- 172.21.15.103:0/3616250012 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f85f40a33b0 0x7f85f40a5870 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8604076480 tx=0x7f85f0005ed0 comp rx=0 tx=0).stop 2024-09-19T07:02:31.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:31.098+0000 7f860b15a640 1 -- 172.21.15.103:0/3616250012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8604101de0 msgr2=0x7f860419a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:31.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:31.098+0000 7f860b15a640 1 --2- 172.21.15.103:0/3616250012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8604101de0 0x7f860419a580 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c920 tx=0x7f85f800cde0 comp rx=0 tx=0).stop 2024-09-19T07:02:31.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:31.098+0000 7f860b15a640 1 -- 172.21.15.103:0/3616250012 shutdown_connections 2024-09-19T07:02:31.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:31.098+0000 7f860b15a640 1 --2- 172.21.15.103:0/3616250012 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f85f40a33b0 0x7f85f40a5870 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:31.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:31.098+0000 7f860b15a640 1 --2- 172.21.15.103:0/3616250012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8604106f10 0x7f860419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:31.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:31.098+0000 7f860b15a640 1 --2- 172.21.15.103:0/3616250012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8604102790 0x7f860419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:31.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:31.098+0000 7f860b15a640 1 --2- 172.21.15.103:0/3616250012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8604101de0 0x7f860419a580 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:31.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:31.098+0000 7f860b15a640 1 -- 172.21.15.103:0/3616250012 >> 172.21.15.103:0/3616250012 conn(0x7f86040fd8c0 msgr2=0x7f86040fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:31.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:31.098+0000 7f860b15a640 1 -- 172.21.15.103:0/3616250012 shutdown_connections 2024-09-19T07:02:31.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:31.098+0000 7f860b15a640 1 -- 172.21.15.103:0/3616250012 wait complete. 2024-09-19T07:02:31.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:30 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T07:02:31.705 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0' 2024-09-19T07:02:31.705 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0 2024-09-19T07:02:31.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:31 smithi103 ceph-mon[31158]: pgmap v415: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.9 KiB/s wr, 16 op/s 2024-09-19T07:02:31.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:31 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3616250012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:02:31.951 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:32.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:31 smithi174 ceph-mon[35637]: pgmap v415: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.9 KiB/s wr, 16 op/s 2024-09-19T07:02:32.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:31 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3616250012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:02:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:31 smithi134 ceph-mon[35356]: pgmap v415: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.9 KiB/s wr, 16 op/s 2024-09-19T07:02:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:31 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3616250012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:02:32.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.428+0000 7fe28fda1640 1 -- 172.21.15.103:0/1859625934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288103760 msgr2=0x7fe288107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:32.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.428+0000 7fe28fda1640 1 --2- 172.21.15.103:0/1859625934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288103760 0x7fe288107ba0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009930 tx=0x7fe27c02a420 comp rx=0 tx=0).stop 2024-09-19T07:02:32.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.428+0000 7fe28fda1640 1 -- 172.21.15.103:0/1859625934 shutdown_connections 2024-09-19T07:02:32.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.428+0000 7fe28fda1640 1 --2- 172.21.15.103:0/1859625934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2881082e0 0x7fe28810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.428+0000 7fe28fda1640 1 --2- 172.21.15.103:0/1859625934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288103760 0x7fe288107ba0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.428+0000 7fe28fda1640 1 --2- 172.21.15.103:0/1859625934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe288102db0 0x7fe288103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.428+0000 7fe28fda1640 1 -- 172.21.15.103:0/1859625934 >> 172.21.15.103:0/1859625934 conn(0x7fe2880fc7d0 msgr2=0x7fe2880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:32.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.428+0000 7fe28fda1640 1 -- 172.21.15.103:0/1859625934 shutdown_connections 2024-09-19T07:02:32.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.428+0000 7fe28fda1640 1 -- 172.21.15.103:0/1859625934 wait complete. 2024-09-19T07:02:32.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28fda1640 1 Processor -- start 2024-09-19T07:02:32.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28fda1640 1 -- start start 2024-09-19T07:02:32.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28fda1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe288102db0 0x7fe288100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28fda1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288103760 0x7fe288100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28fda1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2881082e0 0x7fe288101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28fda1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe288112070 con 0x7fe288103760 2024-09-19T07:02:32.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28fda1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe288111ef0 con 0x7fe288102db0 2024-09-19T07:02:32.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28fda1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881121f0 con 0x7fe2881082e0 2024-09-19T07:02:32.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28d33c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288103760 0x7fe288100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28d33c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288103760 0x7fe288100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32816/0 (socket says 172.21.15.103:32816) 2024-09-19T07:02:32.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28d33c640 1 -- 172.21.15.103:0/3100612241 learned_addr learned my addr 172.21.15.103:0/3100612241 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:32.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28e33e640 1 --2- 172.21.15.103:0/3100612241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2881082e0 0x7fe288101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28db3d640 1 --2- 172.21.15.103:0/3100612241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe288102db0 0x7fe288100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28d33c640 1 -- 172.21.15.103:0/3100612241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2881082e0 msgr2=0x7fe288101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:32.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28d33c640 1 --2- 172.21.15.103:0/3100612241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2881082e0 0x7fe288101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28d33c640 1 -- 172.21.15.103:0/3100612241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe288102db0 msgr2=0x7fe288100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:32.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28d33c640 1 --2- 172.21.15.103:0/3100612241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe288102db0 0x7fe288100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28d33c640 1 -- 172.21.15.103:0/3100612241 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2881a6a40 con 0x7fe288103760 2024-09-19T07:02:32.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.429+0000 7fe28d33c640 1 --2- 172.21.15.103:0/3100612241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288103760 0x7fe288100f50 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fe27c005940 tx=0x7fe27c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:32.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.430+0000 7fe286ffd640 1 -- 172.21.15.103:0/3100612241 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c03f070 con 0x7fe288103760 2024-09-19T07:02:32.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.430+0000 7fe286ffd640 1 -- 172.21.15.103:0/3100612241 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe27c031c30 con 0x7fe288103760 2024-09-19T07:02:32.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.430+0000 7fe28fda1640 1 -- 172.21.15.103:0/3100612241 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2881a6cd0 con 0x7fe288103760 2024-09-19T07:02:32.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.430+0000 7fe28fda1640 1 -- 172.21.15.103:0/3100612241 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2881a71b0 con 0x7fe288103760 2024-09-19T07:02:32.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.430+0000 7fe286ffd640 1 -- 172.21.15.103:0/3100612241 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c035030 con 0x7fe288103760 2024-09-19T07:02:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.432+0000 7fe286ffd640 1 -- 172.21.15.103:0/3100612241 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fe27c042070 con 0x7fe288103760 2024-09-19T07:02:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.432+0000 7fe29217a740 1 Processor -- start 2024-09-19T07:02:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe29217a740 1 -- start start 2024-09-19T07:02:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe29217a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026ed760 0x5606026edb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe29217a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026ee080 0x5606026f8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe29217a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026f8940 0x5606026fad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe29217a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5606026532a0 con 0x5606026ee080 2024-09-19T07:02:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe29217a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5606025fab10 con 0x5606026f8940 2024-09-19T07:02:32.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe29217a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56060264f260 con 0x5606026ed760 2024-09-19T07:02:32.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe286ffd640 1 --2- 172.21.15.103:0/3100612241 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe2700a3430 0x7fe2700a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe28db3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026ee080 0x5606026f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe28e33e640 1 --2- 172.21.15.103:0/3100612241 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe2700a3430 0x7fe2700a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe28db3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026ee080 0x5606026f8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32828/0 (socket says 172.21.15.103:32828) 2024-09-19T07:02:32.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe28db3d640 1 -- 172.21.15.103:0/3386724238 learned_addr learned my addr 172.21.15.103:0/3386724238 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:32.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe28d33c640 1 --2- 172.21.15.103:0/3386724238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026f8940 0x5606026fad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe28e33e640 1 --2- 172.21.15.103:0/3386724238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026ed760 0x5606026edb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe286ffd640 1 -- 172.21.15.103:0/3100612241 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7fe27c0f0490 con 0x7fe288103760 2024-09-19T07:02:32.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe28db3d640 1 -- 172.21.15.103:0/3386724238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026ed760 msgr2=0x5606026edb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:32.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe28db3d640 1 --2- 172.21.15.103:0/3386724238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026ed760 0x5606026edb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe28db3d640 1 -- 172.21.15.103:0/3386724238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026f8940 msgr2=0x5606026fad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:32.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe28db3d640 1 --2- 172.21.15.103:0/3386724238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026f8940 0x5606026fad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.433+0000 7fe28db3d640 1 -- 172.21.15.103:0/3386724238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606023c4b90 con 0x5606026ee080 2024-09-19T07:02:32.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.434+0000 7fe28e33e640 1 --2- 172.21.15.103:0/3100612241 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe2700a3430 0x7fe2700a58f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe280005dd0 tx=0x7fe280005d20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:32.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.434+0000 7fe28db3d640 1 --2- 172.21.15.103:0/3386724238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026ee080 0x5606026f8400 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fe278009f60 tx=0x7fe278009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67167f084f83824c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:32.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.434+0000 7fe26ffff640 1 -- 172.21.15.103:0/3386724238 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278016070 con 0x5606026ee080 2024-09-19T07:02:32.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.434+0000 7fe26ffff640 1 -- 172.21.15.103:0/3386724238 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe278004d20 con 0x5606026ee080 2024-09-19T07:02:32.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.434+0000 7fe26ffff640 1 -- 172.21.15.103:0/3386724238 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278012870 con 0x5606026ee080 2024-09-19T07:02:32.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.435+0000 7fe29217a740 1 -- 172.21.15.103:0/3386724238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026ee080 msgr2=0x5606026f8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:32.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.435+0000 7fe29217a740 1 --2- 172.21.15.103:0/3386724238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026ee080 0x5606026f8400 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fe278009f60 tx=0x7fe278009f90 comp rx=0 tx=0).stop 2024-09-19T07:02:32.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.435+0000 7fe29217a740 1 -- 172.21.15.103:0/3386724238 shutdown_connections 2024-09-19T07:02:32.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.435+0000 7fe29217a740 1 --2- 172.21.15.103:0/3386724238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026f8940 0x5606026fad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.435+0000 7fe29217a740 1 --2- 172.21.15.103:0/3386724238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026ee080 0x5606026f8400 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.435+0000 7fe29217a740 1 --2- 172.21.15.103:0/3386724238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026ed760 0x5606026edb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.435+0000 7fe29217a740 1 -- 172.21.15.103:0/3386724238 >> 172.21.15.103:0/3386724238 conn(0x5606026e9cf0 msgr2=0x5606026ea0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.435+0000 7fe29217a740 1 -- 172.21.15.103:0/3386724238 shutdown_connections 2024-09-19T07:02:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.435+0000 7fe29217a740 1 -- 172.21.15.103:0/3386724238 wait complete. 2024-09-19T07:02:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.435+0000 7fe29217a740 1 -- start start 2024-09-19T07:02:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.435+0000 7fe29217a740 1 Processor -- start 2024-09-19T07:02:32.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe29217a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026ed760 0x5606026f5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe29217a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026ee080 0x5606026f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe29217a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026f8940 0x5606026f6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe29217a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5606026fb510 con 0x5606026f8940 2024-09-19T07:02:32.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe29217a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5606025db6e0 con 0x5606026ed760 2024-09-19T07:02:32.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe29217a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5606026fb690 con 0x5606026ee080 2024-09-19T07:02:32.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe29217a740 10 client.0 _ll_register_callbacks cb 0x5606025ffa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:32.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe28e33e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026f8940 0x5606026f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe28e33e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026f8940 0x5606026f6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32838/0 (socket says 172.21.15.103:32838) 2024-09-19T07:02:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe28e33e640 1 -- 172.21.15.103:0/1086812400 learned_addr learned my addr 172.21.15.103:0/1086812400 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe28d33c640 1 --2- 172.21.15.103:0/1086812400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026ee080 0x5606026f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe28db3d640 1 --2- 172.21.15.103:0/1086812400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026ed760 0x5606026f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe28e33e640 1 -- 172.21.15.103:0/1086812400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026ee080 msgr2=0x5606026f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe28e33e640 1 --2- 172.21.15.103:0/1086812400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026ee080 0x5606026f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe28e33e640 1 -- 172.21.15.103:0/1086812400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026ed760 msgr2=0x5606026f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe28e33e640 1 --2- 172.21.15.103:0/1086812400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026ed760 0x5606026f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe28e33e640 1 -- 172.21.15.103:0/1086812400 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606027049b0 con 0x5606026f8940 2024-09-19T07:02:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe28db3d640 1 --2- 172.21.15.103:0/1086812400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026ed760 0x5606026f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe28e33e640 1 --2- 172.21.15.103:0/1086812400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026f8940 0x5606026f6200 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fe280004260 tx=0x7fe280004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d047a62cdf42af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.436+0000 7fe26d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.437+0000 7fe26d7fa640 1 -- 172.21.15.103:0/1086812400 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2800035f0 con 0x5606026f8940 2024-09-19T07:02:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.437+0000 7fe26d7fa640 1 -- 172.21.15.103:0/1086812400 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7fe280003790 con 0x5606026f8940 2024-09-19T07:02:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.437+0000 7fe29217a740 1 -- 172.21.15.103:0/1086812400 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe278004d20 con 0x5606026f8940 2024-09-19T07:02:32.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.437+0000 7fe26d7fa640 1 -- 172.21.15.103:0/1086812400 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe280003990 con 0x5606026f8940 2024-09-19T07:02:32.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.437+0000 7fe29217a740 10 client.20064 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:32.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.437+0000 7fe29217a740 1 -- 172.21.15.103:0/1086812400 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560602707370 con 0x5606026f8940 2024-09-19T07:02:32.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.437+0000 7fe26d7fa640 1 -- 172.21.15.103:0/1086812400 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe278004d20 con 0x5606026f8940 2024-09-19T07:02:32.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.437+0000 7fe26d7fa640 1 -- 172.21.15.103:0/1086812400 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe28001c040 con 0x5606026f8940 2024-09-19T07:02:32.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.437+0000 7fe29217a740 10 client.20064 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:32.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.437+0000 7fe29217a740 10 client.20064 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:02:32.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.437+0000 7fe29217a740 20 client.20064 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:32.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.437+0000 7fe29217a740 1 --2- 172.21.15.103:0/1086812400 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560602708780 0x56060270ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.437+0000 7fe29217a740 4 client.20064 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:32.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.437+0000 7fe29217a740 1 -- 172.21.15.103:0/1086812400 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606026609d0 con 0x560602708780 2024-09-19T07:02:32.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.438+0000 7fe28db3d640 1 --2- 172.21.15.103:0/1086812400 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560602708780 0x56060270ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.438+0000 7fe26d7fa640 10 client.20064 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:32.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.438+0000 7fe28db3d640 1 --2- 172.21.15.103:0/1086812400 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560602708780 0x56060270ac40 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dd30a8afd67839b server_cookie=443ecdcfdd7d4e9 in_seq=0 out_seq=0 2024-09-19T07:02:32.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.439+0000 7fe26d7fa640 1 -- 172.21.15.103:0/1086812400 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606026609d0 con 0x560602708780 2024-09-19T07:02:32.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.439+0000 7fe26d7fa640 10 client.20064 handle_command_reply: tid=0 2024-09-19T07:02:32.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.439+0000 7fe29217a740 1 client.20064 shutdown 2024-09-19T07:02:32.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.439+0000 7fe29217a740 1 -- 172.21.15.103:0/1086812400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026f8940 msgr2=0x5606026f6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:32.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.439+0000 7fe29217a740 1 --2- 172.21.15.103:0/1086812400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026f8940 0x5606026f6200 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fe280004260 tx=0x7fe280004730 comp rx=0 tx=0).stop 2024-09-19T07:02:32.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.439+0000 7fe29217a740 1 -- 172.21.15.103:0/1086812400 shutdown_connections 2024-09-19T07:02:32.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.439+0000 7fe29217a740 1 --2- 172.21.15.103:0/1086812400 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560602708780 0x56060270ac40 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.439+0000 7fe29217a740 1 --2- 172.21.15.103:0/1086812400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026f8940 0x5606026f6200 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.439+0000 7fe26d7fa640 0 client.20064 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:32.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.439+0000 7fe29217a740 1 --2- 172.21.15.103:0/1086812400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026ee080 0x5606026f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.439+0000 7fe29217a740 1 --2- 172.21.15.103:0/1086812400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026ed760 0x5606026f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.439+0000 7fe29217a740 1 -- 172.21.15.103:0/1086812400 >> 172.21.15.103:0/1086812400 conn(0x5606026e9cf0 msgr2=0x5606026e8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:32.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.439+0000 7fe29217a740 1 -- 172.21.15.103:0/1086812400 shutdown_connections 2024-09-19T07:02:32.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.439+0000 7fe29217a740 1 -- 172.21.15.103:0/1086812400 wait complete. 2024-09-19T07:02:32.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.439+0000 7fe29217a740 20 client.20064 trim_cache size 0 max 16384 2024-09-19T07:02:32.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.456+0000 7fe29217a740 1 Processor -- start 2024-09-19T07:02:32.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.456+0000 7fe29217a740 1 -- start start 2024-09-19T07:02:32.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.456+0000 7fe29217a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026ed760 0x5606026f01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.456+0000 7fe29217a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026ee080 0x560602710560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.456+0000 7fe29217a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026f8940 0x560602710aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.456+0000 7fe29217a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881afab0 con 0x5606026f8940 2024-09-19T07:02:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.456+0000 7fe29217a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881b15f0 con 0x5606026ee080 2024-09-19T07:02:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.456+0000 7fe29217a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881b0180 con 0x5606026ed760 2024-09-19T07:02:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.456+0000 7fe28e33e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026f8940 0x560602710aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.456+0000 7fe28e33e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026f8940 0x560602710aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32842/0 (socket says 172.21.15.103:32842) 2024-09-19T07:02:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.456+0000 7fe28e33e640 1 -- 172.21.15.103:0/335751964 learned_addr learned my addr 172.21.15.103:0/335751964 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.456+0000 7fe28d33c640 1 --2- 172.21.15.103:0/335751964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026ee080 0x560602710560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.456+0000 7fe28db3d640 1 --2- 172.21.15.103:0/335751964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026ed760 0x5606026f01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.457+0000 7fe28e33e640 1 -- 172.21.15.103:0/335751964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026ed760 msgr2=0x5606026f01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.457+0000 7fe28e33e640 1 --2- 172.21.15.103:0/335751964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026ed760 0x5606026f01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.457+0000 7fe28e33e640 1 -- 172.21.15.103:0/335751964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026ee080 msgr2=0x560602710560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.457+0000 7fe28e33e640 1 --2- 172.21.15.103:0/335751964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026ee080 0x560602710560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.457+0000 7fe28e33e640 1 -- 172.21.15.103:0/335751964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606026e9b20 con 0x5606026f8940 2024-09-19T07:02:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.457+0000 7fe28db3d640 1 --2- 172.21.15.103:0/335751964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026ed760 0x5606026f01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.457+0000 7fe28e33e640 1 --2- 172.21.15.103:0/335751964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026f8940 0x560602710aa0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fe280005940 tx=0x7fe28000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a4f8e3170dcc26 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:32.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.457+0000 7fe26f7fe640 1 -- 172.21.15.103:0/335751964 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe280035030 con 0x5606026f8940 2024-09-19T07:02:32.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.457+0000 7fe26f7fe640 1 -- 172.21.15.103:0/335751964 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe280037070 con 0x5606026f8940 2024-09-19T07:02:32.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.457+0000 7fe26f7fe640 1 -- 172.21.15.103:0/335751964 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe280010630 con 0x5606026f8940 2024-09-19T07:02:32.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.457+0000 7fe29217a740 1 -- 172.21.15.103:0/335751964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026f8940 msgr2=0x560602710aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:32.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.457+0000 7fe29217a740 1 --2- 172.21.15.103:0/335751964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026f8940 0x560602710aa0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fe280005940 tx=0x7fe28000a340 comp rx=0 tx=0).stop 2024-09-19T07:02:32.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.458+0000 7fe29217a740 1 -- 172.21.15.103:0/335751964 shutdown_connections 2024-09-19T07:02:32.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.458+0000 7fe29217a740 1 --2- 172.21.15.103:0/335751964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026f8940 0x560602710aa0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.458+0000 7fe29217a740 1 --2- 172.21.15.103:0/335751964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026ee080 0x560602710560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.458+0000 7fe29217a740 1 --2- 172.21.15.103:0/335751964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026ed760 0x5606026f01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.458+0000 7fe29217a740 1 -- 172.21.15.103:0/335751964 >> 172.21.15.103:0/335751964 conn(0x5606026e9cf0 msgr2=0x5606026f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:32.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.458+0000 7fe29217a740 1 -- 172.21.15.103:0/335751964 shutdown_connections 2024-09-19T07:02:32.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.458+0000 7fe29217a740 1 -- 172.21.15.103:0/335751964 wait complete. 2024-09-19T07:02:32.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.458+0000 7fe29217a740 1 -- start start 2024-09-19T07:02:32.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.458+0000 7fe29217a740 1 Processor -- start 2024-09-19T07:02:32.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.458+0000 7fe29217a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026ed760 0x5606026fcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.458+0000 7fe29217a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026ee080 0x5606026fd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.458+0000 7fe29217a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026f8940 0x560602772530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.458+0000 7fe29217a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881b0f20 con 0x5606026ed760 2024-09-19T07:02:32.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.458+0000 7fe29217a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881b0850 con 0x5606026ee080 2024-09-19T07:02:32.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.458+0000 7fe29217a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881adab0 con 0x5606026f8940 2024-09-19T07:02:32.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.458+0000 7fe29217a740 10 client.0 _ll_register_callbacks cb 0x7fe28000a620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:32.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe28db3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026ed760 0x5606026fcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe28db3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026ed760 0x5606026fcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32850/0 (socket says 172.21.15.103:32850) 2024-09-19T07:02:32.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe28db3d640 1 -- 172.21.15.103:0/3502578775 learned_addr learned my addr 172.21.15.103:0/3502578775 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:32.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe28d33c640 1 --2- 172.21.15.103:0/3502578775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026ee080 0x5606026fd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe28e33e640 1 --2- 172.21.15.103:0/3502578775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026f8940 0x560602772530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe28db3d640 1 -- 172.21.15.103:0/3502578775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026f8940 msgr2=0x560602772530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:32.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe28db3d640 1 --2- 172.21.15.103:0/3502578775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026f8940 0x560602772530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe28db3d640 1 -- 172.21.15.103:0/3502578775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026ee080 msgr2=0x5606026fd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:32.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe28db3d640 1 --2- 172.21.15.103:0/3502578775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026ee080 0x5606026fd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe28db3d640 1 -- 172.21.15.103:0/3502578775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606026e5260 con 0x5606026ed760 2024-09-19T07:02:32.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe26d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:32.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe28db3d640 1 --2- 172.21.15.103:0/3502578775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026ed760 0x5606026fcb00 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fe278010c90 tx=0x7fe278005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a55fffd82f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:32.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe26d7fa640 1 -- 172.21.15.103:0/3502578775 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278016040 con 0x5606026ed760 2024-09-19T07:02:32.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe26d7fa640 1 -- 172.21.15.103:0/3502578775 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7fe2780257c0 con 0x5606026ed760 2024-09-19T07:02:32.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe29217a740 1 -- 172.21.15.103:0/3502578775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe280037070 con 0x5606026ed760 2024-09-19T07:02:32.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe26d7fa640 1 -- 172.21.15.103:0/3502578775 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe278033050 con 0x5606026ed760 2024-09-19T07:02:32.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.459+0000 7fe29217a740 10 client.20070 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:32.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.460+0000 7fe29217a740 1 -- 172.21.15.103:0/3502578775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560602773ac0 con 0x5606026ed760 2024-09-19T07:02:32.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.460+0000 7fe26d7fa640 1 -- 172.21.15.103:0/3502578775 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe280037070 con 0x5606026ed760 2024-09-19T07:02:32.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.460+0000 7fe26d7fa640 1 -- 172.21.15.103:0/3502578775 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe27800f040 con 0x5606026ed760 2024-09-19T07:02:32.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.460+0000 7fe29217a740 10 client.20070 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:32.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.460+0000 7fe29217a740 10 client.20070 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:02:32.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.460+0000 7fe29217a740 20 client.20070 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:32.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.460+0000 7fe29217a740 1 --2- 172.21.15.103:0/3502578775 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560602773fd0 0x5606027763f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:32.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.460+0000 7fe29217a740 4 client.20070 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}] 2024-09-19T07:02:32.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.460+0000 7fe29217a740 1 -- 172.21.15.103:0/3502578775 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}) -- 0x56060265ad20 con 0x560602773fd0 2024-09-19T07:02:32.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.460+0000 7fe28d33c640 1 --2- 172.21.15.103:0/3502578775 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560602773fd0 0x5606027763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:32.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.461+0000 7fe28d33c640 1 --2- 172.21.15.103:0/3502578775 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560602773fd0 0x5606027763f0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27cf01c4efbefe14 server_cookie=26482a457e7e7328 in_seq=0 out_seq=0 2024-09-19T07:02:32.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.461+0000 7fe26d7fa640 10 client.20070 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:32.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.472+0000 7fe26d7fa640 1 -- 172.21.15.103:0/3502578775 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56060265ad20 con 0x560602773fd0 2024-09-19T07:02:32.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.472+0000 7fe26d7fa640 10 client.20070 handle_command_reply: tid=0 2024-09-19T07:02:32.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.472+0000 7fe29217a740 1 client.20070 shutdown 2024-09-19T07:02:32.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.472+0000 7fe29217a740 1 -- 172.21.15.103:0/3502578775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026ed760 msgr2=0x5606026fcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:32.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.472+0000 7fe29217a740 1 --2- 172.21.15.103:0/3502578775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026ed760 0x5606026fcb00 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fe278010c90 tx=0x7fe278005ac0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.472+0000 7fe29217a740 1 -- 172.21.15.103:0/3502578775 shutdown_connections 2024-09-19T07:02:32.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.472+0000 7fe29217a740 1 --2- 172.21.15.103:0/3502578775 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560602773fd0 0x5606027763f0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.472+0000 7fe29217a740 1 --2- 172.21.15.103:0/3502578775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606026f8940 0x560602772530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.472+0000 7fe29217a740 1 --2- 172.21.15.103:0/3502578775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606026ee080 0x5606026fd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.472+0000 7fe26d7fa640 0 client.20070 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:32.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.472+0000 7fe29217a740 1 --2- 172.21.15.103:0/3502578775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606026ed760 0x5606026fcb00 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.472+0000 7fe29217a740 1 -- 172.21.15.103:0/3502578775 >> 172.21.15.103:0/3502578775 conn(0x5606026e9cf0 msgr2=0x5606026e8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:32.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.472+0000 7fe29217a740 1 -- 172.21.15.103:0/3502578775 shutdown_connections 2024-09-19T07:02:32.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.472+0000 7fe29217a740 1 -- 172.21.15.103:0/3502578775 wait complete. 2024-09-19T07:02:32.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.473+0000 7fe29217a740 20 client.20070 trim_cache size 0 max 16384 2024-09-19T07:02:32.475 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:02:32.475 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T07:02:32.475 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T07:02:32.475 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 46, 2024-09-19T07:02:32.475 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:02:32.475 INFO:teuthology.orchestra.run.smithi103.stdout: "6244df2c": { 2024-09-19T07:02:32.475 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 46, 2024-09-19T07:02:32.475 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:02:32.475 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:02:32.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:02:32.475 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:02:32.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:02:32.476 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:02:32.476 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 142.2, 2024-09-19T07:02:32.476 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:02:32.476 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:02:32.476 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:02:32.476 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:02:32.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:02:32.476 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:02:32.476 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:32.476 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:32.476 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:32.476 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:32.476 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:32.476 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:02:32.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.473+0000 7fe28fda1640 1 -- 172.21.15.103:0/3100612241 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe2700a3430 msgr2=0x7fe2700a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:32.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.473+0000 7fe28fda1640 1 --2- 172.21.15.103:0/3100612241 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe2700a3430 0x7fe2700a58f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe280005dd0 tx=0x7fe280005d20 comp rx=0 tx=0).stop 2024-09-19T07:02:32.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.473+0000 7fe28fda1640 1 -- 172.21.15.103:0/3100612241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288103760 msgr2=0x7fe288100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:32.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.473+0000 7fe28fda1640 1 --2- 172.21.15.103:0/3100612241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288103760 0x7fe288100f50 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fe27c005940 tx=0x7fe27c003c10 comp rx=0 tx=0).stop 2024-09-19T07:02:32.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.473+0000 7fe28fda1640 1 -- 172.21.15.103:0/3100612241 shutdown_connections 2024-09-19T07:02:32.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.473+0000 7fe28fda1640 1 --2- 172.21.15.103:0/3100612241 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe2700a3430 0x7fe2700a58f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.473+0000 7fe28fda1640 1 --2- 172.21.15.103:0/3100612241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2881082e0 0x7fe288101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.473+0000 7fe28fda1640 1 --2- 172.21.15.103:0/3100612241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288103760 0x7fe288100f50 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.473+0000 7fe28fda1640 1 --2- 172.21.15.103:0/3100612241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe288102db0 0x7fe288100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:32.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.473+0000 7fe28fda1640 1 -- 172.21.15.103:0/3100612241 >> 172.21.15.103:0/3100612241 conn(0x7fe2880fc7d0 msgr2=0x7fe288105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:32.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.473+0000 7fe28fda1640 1 -- 172.21.15.103:0/3100612241 shutdown_connections 2024-09-19T07:02:32.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:32.473+0000 7fe28fda1640 1 -- 172.21.15.103:0/3100612241 wait complete. 2024-09-19T07:02:33.105 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6244df2c', quiesce took 2.9 seconds. Will release after: 19.3 2024-09-19T07:02:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.556+0000 7f2bf6e1e640 1 -- 172.21.15.103:0/4023823241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bf0101350 msgr2=0x7f2bf010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.556+0000 7f2bf6e1e640 1 --2- 172.21.15.103:0/4023823241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bf0101350 0x7f2bf010d880 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009cd0 tx=0x7f2be802a8e0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.556+0000 7f2bf6e1e640 1 -- 172.21.15.103:0/4023823241 shutdown_connections 2024-09-19T07:02:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.556+0000 7f2bf6e1e640 1 --2- 172.21.15.103:0/4023823241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bf0101350 0x7f2bf010d880 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.556+0000 7f2bf6e1e640 1 --2- 172.21.15.103:0/4023823241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bf01009b0 0x7f2bf0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.556+0000 7f2bf6e1e640 1 --2- 172.21.15.103:0/4023823241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bf01069c0 0x7f2bf0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.556+0000 7f2bf6e1e640 1 -- 172.21.15.103:0/4023823241 >> 172.21.15.103:0/4023823241 conn(0x7f2bf0077990 msgr2=0x7f2bf00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.556+0000 7f2bf6e1e640 1 -- 172.21.15.103:0/4023823241 shutdown_connections 2024-09-19T07:02:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.556+0000 7f2bf6e1e640 1 -- 172.21.15.103:0/4023823241 wait complete. 2024-09-19T07:02:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.556+0000 7f2bf6e1e640 1 Processor -- start 2024-09-19T07:02:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf6e1e640 1 -- start start 2024-09-19T07:02:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf6e1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bf01009b0 0x7f2bf0197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf6e1e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bf0101350 0x7f2bf0197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf6e1e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bf01069c0 0x7f2bf019bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf6e1e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0110e50 con 0x7f2bf01009b0 2024-09-19T07:02:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf6e1e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0110cd0 con 0x7f2bf0101350 2024-09-19T07:02:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf6e1e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0110fd0 con 0x7f2bf01069c0 2024-09-19T07:02:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf4bba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bf01009b0 0x7f2bf0197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf4bba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bf01009b0 0x7f2bf0197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32872/0 (socket says 172.21.15.103:32872) 2024-09-19T07:02:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf4bba640 1 -- 172.21.15.103:0/898024863 learned_addr learned my addr 172.21.15.103:0/898024863 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf53bb640 1 --2- 172.21.15.103:0/898024863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bf01069c0 0x7f2bf019bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2beffff640 1 --2- 172.21.15.103:0/898024863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bf0101350 0x7f2bf0197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf4bba640 1 -- 172.21.15.103:0/898024863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bf01069c0 msgr2=0x7f2bf019bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf4bba640 1 --2- 172.21.15.103:0/898024863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bf01069c0 0x7f2bf019bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf4bba640 1 -- 172.21.15.103:0/898024863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bf0101350 msgr2=0x7f2bf0197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf4bba640 1 --2- 172.21.15.103:0/898024863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bf0101350 0x7f2bf0197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf4bba640 1 -- 172.21.15.103:0/898024863 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf019c2f0 con 0x7f2bf01009b0 2024-09-19T07:02:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf4bba640 1 --2- 172.21.15.103:0/898024863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bf01009b0 0x7f2bf0197300 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f2be400e9f0 tx=0x7f2be400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bedffb640 1 -- 172.21.15.103:0/898024863 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be400cd80 con 0x7f2bf01009b0 2024-09-19T07:02:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bedffb640 1 -- 172.21.15.103:0/898024863 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be4005500 con 0x7f2bf01009b0 2024-09-19T07:02:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bedffb640 1 -- 172.21.15.103:0/898024863 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4010690 con 0x7f2bf01009b0 2024-09-19T07:02:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.557+0000 7f2bf6e1e640 1 -- 172.21.15.103:0/898024863 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf019c5e0 con 0x7f2bf01009b0 2024-09-19T07:02:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.558+0000 7f2bf6e1e640 1 -- 172.21.15.103:0/898024863 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf01a3e70 con 0x7f2bf01009b0 2024-09-19T07:02:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.560+0000 7f2bedffb640 1 -- 172.21.15.103:0/898024863 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f2be40046c0 con 0x7f2bf01009b0 2024-09-19T07:02:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.560+0000 7f2bf9c8c740 1 Processor -- start 2024-09-19T07:02:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.560+0000 7f2bf9c8c740 1 -- start start 2024-09-19T07:02:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2bedffb640 1 --2- 172.21.15.103:0/898024863 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2be00a34e0 0x7f2be00a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2bf9c8c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557565719af0 0x557565719ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2bf9c8c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756571a410 0x557565724770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2bf9c8c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557565724cb0 0x5575657270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2bf9c8c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55756567e2f0 con 0x557565724cb0 2024-09-19T07:02:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2bf9c8c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557565626270 con 0x557565719af0 2024-09-19T07:02:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2bf9c8c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55756567a2b0 con 0x55756571a410 2024-09-19T07:02:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2bf53bb640 1 --2- 172.21.15.103:0/898024863 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2be00a34e0 0x7f2be00a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2bedffb640 1 -- 172.21.15.103:0/898024863 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f2be40d00e0 con 0x7f2bf01009b0 2024-09-19T07:02:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2beffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756571a410 0x557565724770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2bf53bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557565724cb0 0x5575657270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2beffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756571a410 0x557565724770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:39012/0 (socket says 172.21.15.103:39012) 2024-09-19T07:02:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2beffff640 1 -- 172.21.15.103:0/2027575171 learned_addr learned my addr 172.21.15.103:0/2027575171 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2bf4bba640 1 --2- 172.21.15.103:0/2027575171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557565719af0 0x557565719ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2beffff640 1 -- 172.21.15.103:0/2027575171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557565719af0 msgr2=0x557565719ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2beffff640 1 --2- 172.21.15.103:0/2027575171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557565719af0 0x557565719ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2beffff640 1 -- 172.21.15.103:0/2027575171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557565724cb0 msgr2=0x5575657270a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2beffff640 1 --2- 172.21.15.103:0/2027575171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557565724cb0 0x5575657270a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2beffff640 1 -- 172.21.15.103:0/2027575171 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557565625340 con 0x55756571a410 2024-09-19T07:02:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2bf53bb640 1 --2- 172.21.15.103:0/898024863 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2be00a34e0 0x7f2be00a59a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2be8005e30 tx=0x7f2be8009930 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.561+0000 7f2beffff640 1 --2- 172.21.15.103:0/2027575171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756571a410 0x557565724770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc005bc0 tx=0x7f2bdc00e2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f4efb88578ce6e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.562+0000 7f2bdaffd640 1 -- 172.21.15.103:0/2027575171 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc016070 con 0x55756571a410 2024-09-19T07:02:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.562+0000 7f2bdaffd640 1 -- 172.21.15.103:0/2027575171 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bdc005500 con 0x55756571a410 2024-09-19T07:02:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.562+0000 7f2bdaffd640 1 -- 172.21.15.103:0/2027575171 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc011470 con 0x55756571a410 2024-09-19T07:02:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.562+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/2027575171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756571a410 msgr2=0x557565724770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.562+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/2027575171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756571a410 0x557565724770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc005bc0 tx=0x7f2bdc00e2f0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.562+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/2027575171 shutdown_connections 2024-09-19T07:02:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.562+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/2027575171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557565724cb0 0x5575657270a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.562+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/2027575171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756571a410 0x557565724770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.562+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/2027575171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557565719af0 0x557565719ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.562+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/2027575171 >> 172.21.15.103:0/2027575171 conn(0x557565716060 msgr2=0x557565716440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.562+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/2027575171 shutdown_connections 2024-09-19T07:02:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.562+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/2027575171 wait complete. 2024-09-19T07:02:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2bf9c8c740 1 -- start start 2024-09-19T07:02:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2bf9c8c740 1 Processor -- start 2024-09-19T07:02:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2bf9c8c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557565719af0 0x557565727cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2bf9c8c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756571a410 0x557565728210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2bf9c8c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557565724cb0 0x557565728750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2bf9c8c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557565727880 con 0x55756571a410 2024-09-19T07:02:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2bf9c8c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557565607290 con 0x557565724cb0 2024-09-19T07:02:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2bf9c8c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557565727a00 con 0x557565719af0 2024-09-19T07:02:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2bf9c8c740 10 client.0 _ll_register_callbacks cb 0x557565638140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2beffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756571a410 0x557565728210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2beffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756571a410 0x557565728210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32890/0 (socket says 172.21.15.103:32890) 2024-09-19T07:02:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2beffff640 1 -- 172.21.15.103:0/2247225776 learned_addr learned my addr 172.21.15.103:0/2247225776 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2bf4bba640 1 --2- 172.21.15.103:0/2247225776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557565719af0 0x557565727cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2bf53bb640 1 --2- 172.21.15.103:0/2247225776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557565724cb0 0x557565728750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2beffff640 1 -- 172.21.15.103:0/2247225776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557565719af0 msgr2=0x557565727cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2beffff640 1 --2- 172.21.15.103:0/2247225776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557565719af0 0x557565727cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.563+0000 7f2beffff640 1 -- 172.21.15.103:0/2247225776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557565724cb0 msgr2=0x557565728750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2beffff640 1 --2- 172.21.15.103:0/2247225776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557565724cb0 0x557565728750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2beffff640 1 -- 172.21.15.103:0/2247225776 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557565391210 con 0x55756571a410 2024-09-19T07:02:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2beffff640 1 --2- 172.21.15.103:0/2247225776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756571a410 0x557565728210 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc004a20 tx=0x7f2bdc004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5484fb79fb217c8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2bcffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2bcffff640 1 -- 172.21.15.103:0/2247225776 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc028060 con 0x55756571a410 2024-09-19T07:02:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2bcffff640 1 -- 172.21.15.103:0/2247225776 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f2bdc011680 con 0x55756571a410 2024-09-19T07:02:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/2247225776 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bdc005500 con 0x55756571a410 2024-09-19T07:02:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2bcffff640 1 -- 172.21.15.103:0/2247225776 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2bdc016040 con 0x55756571a410 2024-09-19T07:02:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2bf9c8c740 10 client.20082 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/2247225776 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55756545d620 con 0x55756571a410 2024-09-19T07:02:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2bcffff640 1 -- 172.21.15.103:0/2247225776 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bdc005500 con 0x55756571a410 2024-09-19T07:02:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2bcffff640 1 -- 172.21.15.103:0/2247225776 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2bdc00f040 con 0x55756571a410 2024-09-19T07:02:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2bf9c8c740 10 client.20082 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2bf9c8c740 10 client.20082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2bf9c8c740 20 client.20082 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/2247225776 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557565736db0 0x5575657391a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2bf9c8c740 4 client.20082 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.564+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/2247225776 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55756568ba20 con 0x557565736db0 2024-09-19T07:02:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.565+0000 7f2bf4bba640 1 --2- 172.21.15.103:0/2247225776 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557565736db0 0x5575657391a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.565+0000 7f2bcffff640 10 client.20082 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.565+0000 7f2bf4bba640 1 --2- 172.21.15.103:0/2247225776 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557565736db0 0x5575657391a0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d1663df7d95f53f server_cookie=f4e5c24e5cdc2388 in_seq=0 out_seq=0 2024-09-19T07:02:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.566+0000 7f2bcffff640 1 -- 172.21.15.103:0/2247225776 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55756568ba20 con 0x557565736db0 2024-09-19T07:02:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.566+0000 7f2bcffff640 10 client.20082 handle_command_reply: tid=0 2024-09-19T07:02:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.566+0000 7f2bf9c8c740 1 client.20082 shutdown 2024-09-19T07:02:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.566+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/2247225776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756571a410 msgr2=0x557565728210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.566+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/2247225776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756571a410 0x557565728210 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc004a20 tx=0x7f2bdc004e50 comp rx=0 tx=0).stop 2024-09-19T07:02:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.566+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/2247225776 shutdown_connections 2024-09-19T07:02:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.566+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/2247225776 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557565736db0 0x5575657391a0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.566+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/2247225776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557565724cb0 0x557565728750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.566+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/2247225776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756571a410 0x557565728210 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.566+0000 7f2bcffff640 0 client.20082 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.566+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/2247225776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557565719af0 0x557565727cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.566+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/2247225776 >> 172.21.15.103:0/2247225776 conn(0x557565716060 msgr2=0x55756571bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.566+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/2247225776 shutdown_connections 2024-09-19T07:02:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.566+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/2247225776 wait complete. 2024-09-19T07:02:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.566+0000 7f2bf9c8c740 20 client.20082 trim_cache size 0 max 16384 2024-09-19T07:02:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.580+0000 7f2bf9c8c740 1 Processor -- start 2024-09-19T07:02:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.580+0000 7f2bf9c8c740 1 -- start start 2024-09-19T07:02:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bf9c8c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557565719af0 0x55756573e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bf9c8c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756571a410 0x55756573ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bf9c8c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557565724cb0 0x557565720f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bf9c8c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01ab680 con 0x557565724cb0 2024-09-19T07:02:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bf9c8c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01ad1c0 con 0x55756571a410 2024-09-19T07:02:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bf9c8c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01abd50 con 0x557565719af0 2024-09-19T07:02:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bf53bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557565724cb0 0x557565720f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bf53bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557565724cb0 0x557565720f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32906/0 (socket says 172.21.15.103:32906) 2024-09-19T07:02:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bf53bb640 1 -- 172.21.15.103:0/542556593 learned_addr learned my addr 172.21.15.103:0/542556593 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bf4bba640 1 --2- 172.21.15.103:0/542556593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557565719af0 0x55756573e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2beffff640 1 --2- 172.21.15.103:0/542556593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756571a410 0x55756573ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bf53bb640 1 -- 172.21.15.103:0/542556593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557565719af0 msgr2=0x55756573e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bf53bb640 1 --2- 172.21.15.103:0/542556593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557565719af0 0x55756573e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bf53bb640 1 -- 172.21.15.103:0/542556593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756571a410 msgr2=0x55756573ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bf53bb640 1 --2- 172.21.15.103:0/542556593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756571a410 0x55756573ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bf53bb640 1 -- 172.21.15.103:0/542556593 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557565719870 con 0x557565724cb0 2024-09-19T07:02:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2beffff640 1 --2- 172.21.15.103:0/542556593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756571a410 0x55756573ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bf53bb640 1 --2- 172.21.15.103:0/542556593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557565724cb0 0x557565720f20 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f2be802aed0 tx=0x7f2be802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62946c21f23e138b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bda7fc640 1 -- 172.21.15.103:0/542556593 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8003e00 con 0x557565724cb0 2024-09-19T07:02:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bda7fc640 1 -- 172.21.15.103:0/542556593 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be8031c30 con 0x557565724cb0 2024-09-19T07:02:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.581+0000 7f2bda7fc640 1 -- 172.21.15.103:0/542556593 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8030e90 con 0x557565724cb0 2024-09-19T07:02:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.582+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/542556593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557565724cb0 msgr2=0x557565720f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.582+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/542556593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557565724cb0 0x557565720f20 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f2be802aed0 tx=0x7f2be802a8e0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.582+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/542556593 shutdown_connections 2024-09-19T07:02:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.582+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/542556593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557565724cb0 0x557565720f20 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.582+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/542556593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756571a410 0x55756573ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.582+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/542556593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557565719af0 0x55756573e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.582+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/542556593 >> 172.21.15.103:0/542556593 conn(0x557565716060 msgr2=0x5575657151c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.582+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/542556593 shutdown_connections 2024-09-19T07:02:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.582+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/542556593 wait complete. 2024-09-19T07:02:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.582+0000 7f2bf9c8c740 1 -- start start 2024-09-19T07:02:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2bf9c8c740 1 Processor -- start 2024-09-19T07:02:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2bf9c8c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557565719af0 0x55756572d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2bf9c8c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756571a410 0x55756572a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2bf9c8c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557565724cb0 0x55756572aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2bf9c8c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01acaf0 con 0x55756571a410 2024-09-19T07:02:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2bf9c8c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01ac420 con 0x557565719af0 2024-09-19T07:02:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2bf9c8c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01a9680 con 0x557565724cb0 2024-09-19T07:02:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2bf9c8c740 10 client.0 _ll_register_callbacks cb 0x7f2be8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2beffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756571a410 0x55756572a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2beffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756571a410 0x55756572a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32908/0 (socket says 172.21.15.103:32908) 2024-09-19T07:02:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2beffff640 1 -- 172.21.15.103:0/3761519188 learned_addr learned my addr 172.21.15.103:0/3761519188 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2bf53bb640 1 --2- 172.21.15.103:0/3761519188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557565724cb0 0x55756572aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2bf4bba640 1 --2- 172.21.15.103:0/3761519188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557565719af0 0x55756572d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2beffff640 1 -- 172.21.15.103:0/3761519188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557565724cb0 msgr2=0x55756572aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2beffff640 1 --2- 172.21.15.103:0/3761519188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557565724cb0 0x55756572aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2beffff640 1 -- 172.21.15.103:0/3761519188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557565719af0 msgr2=0x55756572d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2beffff640 1 --2- 172.21.15.103:0/3761519188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557565719af0 0x55756572d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2beffff640 1 -- 172.21.15.103:0/3761519188 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55756572b0e0 con 0x55756571a410 2024-09-19T07:02:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.583+0000 7f2bf4bba640 1 --2- 172.21.15.103:0/3761519188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557565719af0 0x55756572d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.584+0000 7f2beffff640 1 --2- 172.21.15.103:0/3761519188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756571a410 0x55756572a560 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc01a580 tx=0x7f2bdc009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e015767bc1d92a30 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.584+0000 7f2bcffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.584+0000 7f2bcffff640 1 -- 172.21.15.103:0/3761519188 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc000e60 con 0x55756571a410 2024-09-19T07:02:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.584+0000 7f2bcffff640 1 -- 172.21.15.103:0/3761519188 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f2bdc0054a0 con 0x55756571a410 2024-09-19T07:02:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.584+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/3761519188 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be8031c30 con 0x55756571a410 2024-09-19T07:02:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.584+0000 7f2bcffff640 1 -- 172.21.15.103:0/3761519188 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2bdc03b050 con 0x55756571a410 2024-09-19T07:02:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.584+0000 7f2bf9c8c740 10 client.20088 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.584+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/3761519188 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575657a1590 con 0x55756571a410 2024-09-19T07:02:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.584+0000 7f2bcffff640 1 -- 172.21.15.103:0/3761519188 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be8031c30 con 0x55756571a410 2024-09-19T07:02:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.584+0000 7f2bcffff640 1 -- 172.21.15.103:0/3761519188 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2bdc011680 con 0x55756571a410 2024-09-19T07:02:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.584+0000 7f2bf9c8c740 10 client.20088 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.584+0000 7f2bf9c8c740 10 client.20088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.584+0000 7f2bf9c8c740 20 client.20088 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.584+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/3761519188 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5575657a1d00 0x5575657a4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.584+0000 7f2bf9c8c740 4 client.20088 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:02:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.584+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/3761519188 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557565685d70 con 0x5575657a1d00 2024-09-19T07:02:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.585+0000 7f2bf4bba640 1 --2- 172.21.15.103:0/3761519188 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5575657a1d00 0x5575657a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.585+0000 7f2bf4bba640 1 --2- 172.21.15.103:0/3761519188 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5575657a1d00 0x5575657a4120 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31bb39e9aec1f04e server_cookie=d869e61d3f610b81 in_seq=0 out_seq=0 2024-09-19T07:02:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.585+0000 7f2bcffff640 10 client.20088 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.594+0000 7f2bcffff640 1 -- 172.21.15.103:0/3761519188 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557565685d70 con 0x5575657a1d00 2024-09-19T07:02:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.594+0000 7f2bcffff640 10 client.20088 handle_command_reply: tid=0 2024-09-19T07:02:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.594+0000 7f2bf9c8c740 1 client.20088 shutdown 2024-09-19T07:02:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.594+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/3761519188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756571a410 msgr2=0x55756572a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.594+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/3761519188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756571a410 0x55756572a560 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc01a580 tx=0x7f2bdc009a70 comp rx=0 tx=0).stop 2024-09-19T07:02:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.594+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/3761519188 shutdown_connections 2024-09-19T07:02:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.594+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/3761519188 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5575657a1d00 0x5575657a4120 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.594+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/3761519188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557565724cb0 0x55756572aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.594+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/3761519188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756571a410 0x55756572a560 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.594+0000 7f2bf9c8c740 1 --2- 172.21.15.103:0/3761519188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557565719af0 0x55756572d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.594+0000 7f2bcffff640 0 client.20088 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.594+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/3761519188 >> 172.21.15.103:0/3761519188 conn(0x557565716060 msgr2=0x557565731b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.595+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/3761519188 shutdown_connections 2024-09-19T07:02:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.595+0000 7f2bf9c8c740 1 -- 172.21.15.103:0/3761519188 wait complete. 2024-09-19T07:02:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.595+0000 7f2bf9c8c740 20 client.20088 trim_cache size 0 max 16384 2024-09-19T07:02:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:02:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:02:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:02:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:02:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.595+0000 7f2bf6e1e640 1 -- 172.21.15.103:0/898024863 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2be00a34e0 msgr2=0x7f2be00a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.595+0000 7f2bf6e1e640 1 --2- 172.21.15.103:0/898024863 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2be00a34e0 0x7f2be00a59a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2be8005e30 tx=0x7f2be8009930 comp rx=0 tx=0).stop 2024-09-19T07:02:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.595+0000 7f2bf6e1e640 1 -- 172.21.15.103:0/898024863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bf01009b0 msgr2=0x7f2bf0197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.595+0000 7f2bf6e1e640 1 --2- 172.21.15.103:0/898024863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bf01009b0 0x7f2bf0197300 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f2be400e9f0 tx=0x7f2be400edf0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.595+0000 7f2bf6e1e640 1 -- 172.21.15.103:0/898024863 shutdown_connections 2024-09-19T07:02:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.595+0000 7f2bf6e1e640 1 --2- 172.21.15.103:0/898024863 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f2be00a34e0 0x7f2be00a59a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.595+0000 7f2bf6e1e640 1 --2- 172.21.15.103:0/898024863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bf01069c0 0x7f2bf019bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.595+0000 7f2bf6e1e640 1 --2- 172.21.15.103:0/898024863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bf0101350 0x7f2bf0197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.595+0000 7f2bf6e1e640 1 --2- 172.21.15.103:0/898024863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bf01009b0 0x7f2bf0197300 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.595+0000 7f2bf6e1e640 1 -- 172.21.15.103:0/898024863 >> 172.21.15.103:0/898024863 conn(0x7f2bf0077990 msgr2=0x7f2bf0103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.595+0000 7f2bf6e1e640 1 -- 172.21.15.103:0/898024863 shutdown_connections 2024-09-19T07:02:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:33.595+0000 7f2bf6e1e640 1 -- 172.21.15.103:0/898024863 wait complete. 2024-09-19T07:02:34.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:33 smithi174 ceph-mon[35637]: pgmap v416: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-09-19T07:02:34.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:33 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:02:34.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:33 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:02:34.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:33 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:02:34.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:33 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:02:34.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:33 smithi103 ceph-mon[31158]: pgmap v416: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-09-19T07:02:34.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:33 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:02:34.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:33 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:02:34.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:33 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:02:34.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:33 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:02:34.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:33 smithi134 ceph-mon[35356]: pgmap v416: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-09-19T07:02:34.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:33 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:02:34.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:33 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:02:34.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:33 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:02:34.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:33 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:02:36.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:35 smithi174 ceph-mon[35637]: pgmap v417: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-09-19T07:02:36.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:35 smithi103 ceph-mon[31158]: pgmap v417: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-09-19T07:02:36.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:35 smithi134 ceph-mon[35356]: pgmap v417: 97 pgs: 97 active+clean; 3.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-09-19T07:02:38.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:37 smithi174 ceph-mon[35637]: pgmap v418: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-09-19T07:02:38.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:37 smithi103 ceph-mon[31158]: pgmap v418: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-09-19T07:02:38.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:37 smithi134 ceph-mon[35356]: pgmap v418: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-09-19T07:02:40.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:39 smithi174 ceph-mon[35637]: pgmap v419: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-19T07:02:40.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:39 smithi103 ceph-mon[31158]: pgmap v419: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-19T07:02:40.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:39 smithi134 ceph-mon[35356]: pgmap v419: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-19T07:02:42.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:41 smithi174 ceph-mon[35637]: pgmap v420: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-09-19T07:02:42.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:41 smithi103 ceph-mon[31158]: pgmap v420: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-09-19T07:02:42.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:41 smithi134 ceph-mon[35356]: pgmap v420: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-09-19T07:02:44.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:43 smithi174 ceph-mon[35637]: pgmap v421: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-19T07:02:44.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:43 smithi103 ceph-mon[31158]: pgmap v421: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-19T07:02:44.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:43 smithi134 ceph-mon[35356]: pgmap v421: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-19T07:02:46.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:45 smithi174 ceph-mon[35637]: pgmap v422: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-19T07:02:46.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:45 smithi103 ceph-mon[31158]: pgmap v422: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-19T07:02:46.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:45 smithi134 ceph-mon[35356]: pgmap v422: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-19T07:02:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.690+0000 7f357a647640 1 -- 172.21.15.103:0/103674144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3574102d00 msgr2=0x7f35741030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.690+0000 7f357a647640 1 --2- 172.21.15.103:0/103674144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3574102d00 0x7f35741030e0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f3564009900 tx=0x7f356402a480 comp rx=0 tx=0).stop 2024-09-19T07:02:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.690+0000 7f357a647640 1 -- 172.21.15.103:0/103674144 shutdown_connections 2024-09-19T07:02:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.690+0000 7f357a647640 1 --2- 172.21.15.103:0/103674144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f357410c0d0 0x7f357410e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.690+0000 7f357a647640 1 --2- 172.21.15.103:0/103674144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3574103620 0x7f357410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.690+0000 7f357a647640 1 --2- 172.21.15.103:0/103674144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3574102d00 0x7f35741030e0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.690+0000 7f357a647640 1 -- 172.21.15.103:0/103674144 >> 172.21.15.103:0/103674144 conn(0x7f35740fc6f0 msgr2=0x7f35740feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.690+0000 7f357a647640 1 -- 172.21.15.103:0/103674144 shutdown_connections 2024-09-19T07:02:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.690+0000 7f357a647640 1 -- 172.21.15.103:0/103674144 wait complete. 2024-09-19T07:02:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.691+0000 7f357a647640 1 Processor -- start 2024-09-19T07:02:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.691+0000 7f357a647640 1 -- start start 2024-09-19T07:02:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.691+0000 7f357a647640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3574102d00 0x7f357419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.691+0000 7f357a647640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3574103620 0x7f357419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.691+0000 7f357a647640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f357410c0d0 0x7f357419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.691+0000 7f357a647640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f35741140a0 con 0x7f3574103620 2024-09-19T07:02:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.691+0000 7f357a647640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3574113f20 con 0x7f3574102d00 2024-09-19T07:02:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.691+0000 7f357a647640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3574114220 con 0x7f357410c0d0 2024-09-19T07:02:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.691+0000 7f35737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3574103620 0x7f357419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.691+0000 7f35737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3574103620 0x7f357419aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36058/0 (socket says 172.21.15.103:36058) 2024-09-19T07:02:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.691+0000 7f35737fe640 1 -- 172.21.15.103:0/3058792075 learned_addr learned my addr 172.21.15.103:0/3058792075 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.691+0000 7f3578be4640 1 --2- 172.21.15.103:0/3058792075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f357410c0d0 0x7f357419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.691+0000 7f3573fff640 1 --2- 172.21.15.103:0/3058792075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3574102d00 0x7f357419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.692+0000 7f35737fe640 1 -- 172.21.15.103:0/3058792075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f357410c0d0 msgr2=0x7f357419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.692+0000 7f35737fe640 1 --2- 172.21.15.103:0/3058792075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f357410c0d0 0x7f357419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.692+0000 7f35737fe640 1 -- 172.21.15.103:0/3058792075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3574102d00 msgr2=0x7f357419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.692+0000 7f35737fe640 1 --2- 172.21.15.103:0/3058792075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3574102d00 0x7f357419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.692+0000 7f35737fe640 1 -- 172.21.15.103:0/3058792075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357419f500 con 0x7f3574103620 2024-09-19T07:02:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.692+0000 7f3578be4640 1 --2- 172.21.15.103:0/3058792075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f357410c0d0 0x7f357419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.692+0000 7f35737fe640 1 --2- 172.21.15.103:0/3058792075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3574103620 0x7f357419aa50 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f356800d950 tx=0x7f356800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.692+0000 7f35717fa640 1 -- 172.21.15.103:0/3058792075 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3568017070 con 0x7f3574103620 2024-09-19T07:02:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.692+0000 7f357a647640 1 -- 172.21.15.103:0/3058792075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f357419f7f0 con 0x7f3574103620 2024-09-19T07:02:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.692+0000 7f35717fa640 1 -- 172.21.15.103:0/3058792075 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3568009c30 con 0x7f3574103620 2024-09-19T07:02:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.692+0000 7f35717fa640 1 -- 172.21.15.103:0/3058792075 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356800f6b0 con 0x7f3574103620 2024-09-19T07:02:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.692+0000 7f357a647640 1 -- 172.21.15.103:0/3058792075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35741a7080 con 0x7f3574103620 2024-09-19T07:02:46.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.694+0000 7f35717fa640 1 -- 172.21.15.103:0/3058792075 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f35680109d0 con 0x7f3574103620 2024-09-19T07:02:46.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.694+0000 7f357d3de740 1 Processor -- start 2024-09-19T07:02:46.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f357d3de740 1 -- start start 2024-09-19T07:02:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f357d3de740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d32daf0 0x55710d32ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f357d3de740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d32e410 0x55710d338770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f357d3de740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d338cb0 0x55710d33b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f357d3de740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55710d2922f0 con 0x55710d338cb0 2024-09-19T07:02:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f357d3de740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55710d23a270 con 0x55710d32e410 2024-09-19T07:02:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f357d3de740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55710d28e2b0 con 0x55710d32daf0 2024-09-19T07:02:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f35717fa640 1 --2- 172.21.15.103:0/3058792075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f355c0a3460 0x7f355c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f35737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d338cb0 0x55710d33b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f35737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d338cb0 0x55710d33b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36062/0 (socket says 172.21.15.103:36062) 2024-09-19T07:02:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f35737fe640 1 -- 172.21.15.103:0/1739637107 learned_addr learned my addr 172.21.15.103:0/1739637107 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f3573fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d32e410 0x55710d338770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f3578be4640 1 --2- 172.21.15.103:0/1739637107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d32daf0 0x55710d32ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f3578be4640 1 --2- 172.21.15.103:0/3058792075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f355c0a3460 0x7f355c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f35717fa640 1 -- 172.21.15.103:0/3058792075 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f35680cfa00 con 0x7f3574103620 2024-09-19T07:02:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f35737fe640 1 -- 172.21.15.103:0/1739637107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d32daf0 msgr2=0x55710d32ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f35737fe640 1 --2- 172.21.15.103:0/1739637107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d32daf0 0x55710d32ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f35737fe640 1 -- 172.21.15.103:0/1739637107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d32e410 msgr2=0x55710d338770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f35737fe640 1 --2- 172.21.15.103:0/1739637107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d32e410 0x55710d338770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.695+0000 7f35737fe640 1 -- 172.21.15.103:0/1739637107 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55710d239340 con 0x55710d338cb0 2024-09-19T07:02:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.696+0000 7f35737fe640 1 --2- 172.21.15.103:0/1739637107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d338cb0 0x55710d33b0a0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f35680a4400 tx=0x7f35680865b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccef02cf8bb42de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.696+0000 7f3578be4640 1 --2- 172.21.15.103:0/3058792075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f355c0a3460 0x7f355c0a5920 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f356c005c40 tx=0x7f356c009280 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.696+0000 7f355a7fc640 1 -- 172.21.15.103:0/1739637107 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3568020070 con 0x55710d338cb0 2024-09-19T07:02:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.696+0000 7f355a7fc640 1 -- 172.21.15.103:0/1739637107 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f356809bcc0 con 0x55710d338cb0 2024-09-19T07:02:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.696+0000 7f355a7fc640 1 -- 172.21.15.103:0/1739637107 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3568086250 con 0x55710d338cb0 2024-09-19T07:02:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.696+0000 7f357d3de740 1 -- 172.21.15.103:0/1739637107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d338cb0 msgr2=0x55710d33b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.696+0000 7f357d3de740 1 --2- 172.21.15.103:0/1739637107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d338cb0 0x55710d33b0a0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f35680a4400 tx=0x7f35680865b0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.697+0000 7f357d3de740 1 -- 172.21.15.103:0/1739637107 shutdown_connections 2024-09-19T07:02:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.697+0000 7f357d3de740 1 --2- 172.21.15.103:0/1739637107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d338cb0 0x55710d33b0a0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.697+0000 7f357d3de740 1 --2- 172.21.15.103:0/1739637107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d32e410 0x55710d338770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.697+0000 7f357d3de740 1 --2- 172.21.15.103:0/1739637107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d32daf0 0x55710d32ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.697+0000 7f357d3de740 1 -- 172.21.15.103:0/1739637107 >> 172.21.15.103:0/1739637107 conn(0x55710d32a060 msgr2=0x55710d32a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.697+0000 7f357d3de740 1 -- 172.21.15.103:0/1739637107 shutdown_connections 2024-09-19T07:02:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.697+0000 7f357d3de740 1 -- 172.21.15.103:0/1739637107 wait complete. 2024-09-19T07:02:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.697+0000 7f357d3de740 1 -- start start 2024-09-19T07:02:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.697+0000 7f357d3de740 1 Processor -- start 2024-09-19T07:02:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.697+0000 7f357d3de740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d32daf0 0x55710d335bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f357d3de740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32e410 0x55710d3360f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f357d3de740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d338cb0 0x55710d336630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f357d3de740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55710d33b820 con 0x55710d32e410 2024-09-19T07:02:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f357d3de740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55710d21b290 con 0x55710d32daf0 2024-09-19T07:02:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f357d3de740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55710d33b9a0 con 0x55710d338cb0 2024-09-19T07:02:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f357d3de740 10 client.0 _ll_register_callbacks cb 0x55710d24c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f35737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32e410 0x55710d3360f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f35737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32e410 0x55710d3360f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36072/0 (socket says 172.21.15.103:36072) 2024-09-19T07:02:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f35737fe640 1 -- 172.21.15.103:0/3013114251 learned_addr learned my addr 172.21.15.103:0/3013114251 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f3578be4640 1 --2- 172.21.15.103:0/3013114251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d338cb0 0x55710d336630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f3573fff640 1 --2- 172.21.15.103:0/3013114251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d32daf0 0x55710d335bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f35737fe640 1 -- 172.21.15.103:0/3013114251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d338cb0 msgr2=0x55710d336630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f35737fe640 1 --2- 172.21.15.103:0/3013114251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d338cb0 0x55710d336630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f35737fe640 1 -- 172.21.15.103:0/3013114251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d32daf0 msgr2=0x55710d335bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f35737fe640 1 --2- 172.21.15.103:0/3013114251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d32daf0 0x55710d335bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f35737fe640 1 -- 172.21.15.103:0/3013114251 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55710cfa5210 con 0x55710d32e410 2024-09-19T07:02:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f35737fe640 1 --2- 172.21.15.103:0/3013114251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32e410 0x55710d3360f0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f35680cd1e0 tx=0x7f35680cd660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b2c136ec8e75df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f35537fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f35537fe640 1 -- 172.21.15.103:0/3013114251 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3568020070 con 0x55710d32e410 2024-09-19T07:02:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f35537fe640 1 -- 172.21.15.103:0/3013114251 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f3568086250 con 0x55710d32e410 2024-09-19T07:02:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f357d3de740 1 -- 172.21.15.103:0/3013114251 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55710d29fa20 con 0x55710d32e410 2024-09-19T07:02:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f35537fe640 1 -- 172.21.15.103:0/3013114251 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f356809c3c0 con 0x55710d32e410 2024-09-19T07:02:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.698+0000 7f357d3de740 10 client.20100 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.699+0000 7f357d3de740 1 -- 172.21.15.103:0/3013114251 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55710d071620 con 0x55710d32e410 2024-09-19T07:02:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.699+0000 7f35537fe640 1 -- 172.21.15.103:0/3013114251 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55710d29fa20 con 0x55710d32e410 2024-09-19T07:02:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.699+0000 7f35537fe640 1 -- 172.21.15.103:0/3013114251 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3568010460 con 0x55710d32e410 2024-09-19T07:02:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.699+0000 7f357d3de740 10 client.20100 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.699+0000 7f357d3de740 10 client.20100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.699+0000 7f357d3de740 20 client.20100 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.699+0000 7f357d3de740 1 --2- 172.21.15.103:0/3013114251 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55710d348a50 0x55710d34ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.699+0000 7f357d3de740 4 client.20100 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.699+0000 7f357d3de740 1 -- 172.21.15.103:0/3013114251 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55710d299d70 con 0x55710d348a50 2024-09-19T07:02:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.699+0000 7f3573fff640 1 --2- 172.21.15.103:0/3013114251 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55710d348a50 0x55710d34ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.700+0000 7f3573fff640 1 --2- 172.21.15.103:0/3013114251 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55710d348a50 0x55710d34ae40 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a0d4ae6952c58f9 server_cookie=ae92b9788b550697 in_seq=0 out_seq=0 2024-09-19T07:02:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.700+0000 7f35537fe640 10 client.20100 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.700+0000 7f35537fe640 1 -- 172.21.15.103:0/3013114251 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55710d299d70 con 0x55710d348a50 2024-09-19T07:02:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.700+0000 7f35537fe640 10 client.20100 handle_command_reply: tid=0 2024-09-19T07:02:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.700+0000 7f357d3de740 1 client.20100 shutdown 2024-09-19T07:02:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.700+0000 7f357d3de740 1 -- 172.21.15.103:0/3013114251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32e410 msgr2=0x55710d3360f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.700+0000 7f357d3de740 1 --2- 172.21.15.103:0/3013114251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32e410 0x55710d3360f0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f35680cd1e0 tx=0x7f35680cd660 comp rx=0 tx=0).stop 2024-09-19T07:02:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.700+0000 7f357d3de740 1 -- 172.21.15.103:0/3013114251 shutdown_connections 2024-09-19T07:02:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.700+0000 7f357d3de740 1 --2- 172.21.15.103:0/3013114251 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55710d348a50 0x55710d34ae40 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.700+0000 7f357d3de740 1 --2- 172.21.15.103:0/3013114251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d338cb0 0x55710d336630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.700+0000 7f357d3de740 1 --2- 172.21.15.103:0/3013114251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32e410 0x55710d3360f0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.700+0000 7f35537fe640 0 client.20100 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.700+0000 7f357d3de740 1 --2- 172.21.15.103:0/3013114251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d32daf0 0x55710d335bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.701+0000 7f357d3de740 1 -- 172.21.15.103:0/3013114251 >> 172.21.15.103:0/3013114251 conn(0x55710d32a060 msgr2=0x55710d32fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.701+0000 7f357d3de740 1 -- 172.21.15.103:0/3013114251 shutdown_connections 2024-09-19T07:02:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.701+0000 7f357d3de740 1 -- 172.21.15.103:0/3013114251 wait complete. 2024-09-19T07:02:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.701+0000 7f357d3de740 20 client.20100 trim_cache size 0 max 16384 2024-09-19T07:02:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.715+0000 7f357d3de740 1 Processor -- start 2024-09-19T07:02:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.715+0000 7f357d3de740 1 -- start start 2024-09-19T07:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.715+0000 7f357d3de740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d32daf0 0x55710d330560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.715+0000 7f357d3de740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32e410 0x55710d336460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.715+0000 7f357d3de740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d338cb0 0x55710d34de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.715+0000 7f357d3de740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f35741afa80 con 0x55710d32e410 2024-09-19T07:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.715+0000 7f357d3de740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f35741b15c0 con 0x55710d338cb0 2024-09-19T07:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.715+0000 7f357d3de740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35741b0150 con 0x55710d32daf0 2024-09-19T07:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.715+0000 7f35737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32e410 0x55710d336460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.715+0000 7f35737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32e410 0x55710d336460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36082/0 (socket says 172.21.15.103:36082) 2024-09-19T07:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.715+0000 7f35737fe640 1 -- 172.21.15.103:0/2513597943 learned_addr learned my addr 172.21.15.103:0/2513597943 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.715+0000 7f35737fe640 1 -- 172.21.15.103:0/2513597943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d32daf0 msgr2=0x55710d330560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.715+0000 7f35737fe640 1 --2- 172.21.15.103:0/2513597943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d32daf0 0x55710d330560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.715+0000 7f35737fe640 1 -- 172.21.15.103:0/2513597943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d338cb0 msgr2=0x55710d34de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.715+0000 7f35737fe640 1 --2- 172.21.15.103:0/2513597943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d338cb0 0x55710d34de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.715+0000 7f35737fe640 1 -- 172.21.15.103:0/2513597943 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55710d329e90 con 0x55710d32e410 2024-09-19T07:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.716+0000 7f35737fe640 1 --2- 172.21.15.103:0/2513597943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32e410 0x55710d336460 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f3568098690 tx=0x7f3568005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf3195ba442481 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.716+0000 7f3559ffb640 1 -- 172.21.15.103:0/2513597943 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3568020070 con 0x55710d32e410 2024-09-19T07:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.716+0000 7f3559ffb640 1 -- 172.21.15.103:0/2513597943 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35680a0070 con 0x55710d32e410 2024-09-19T07:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.716+0000 7f3559ffb640 1 -- 172.21.15.103:0/2513597943 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356809bf20 con 0x55710d32e410 2024-09-19T07:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.716+0000 7f357d3de740 1 -- 172.21.15.103:0/2513597943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32e410 msgr2=0x55710d336460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.716+0000 7f357d3de740 1 --2- 172.21.15.103:0/2513597943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32e410 0x55710d336460 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f3568098690 tx=0x7f3568005880 comp rx=0 tx=0).stop 2024-09-19T07:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.716+0000 7f357d3de740 1 -- 172.21.15.103:0/2513597943 shutdown_connections 2024-09-19T07:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.716+0000 7f357d3de740 1 --2- 172.21.15.103:0/2513597943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d338cb0 0x55710d34de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.716+0000 7f357d3de740 1 --2- 172.21.15.103:0/2513597943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32e410 0x55710d336460 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.716+0000 7f357d3de740 1 --2- 172.21.15.103:0/2513597943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d32daf0 0x55710d330560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.716+0000 7f357d3de740 1 -- 172.21.15.103:0/2513597943 >> 172.21.15.103:0/2513597943 conn(0x55710d32a060 msgr2=0x55710d337c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.716+0000 7f357d3de740 1 -- 172.21.15.103:0/2513597943 shutdown_connections 2024-09-19T07:02:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.716+0000 7f357d3de740 1 -- 172.21.15.103:0/2513597943 wait complete. 2024-09-19T07:02:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.717+0000 7f357d3de740 1 -- start start 2024-09-19T07:02:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.717+0000 7f357d3de740 1 Processor -- start 2024-09-19T07:02:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.717+0000 7f357d3de740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32daf0 0x55710d34e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.717+0000 7f357d3de740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d32e410 0x55710d34b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.717+0000 7f357d3de740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d338cb0 0x55710d34bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.717+0000 7f357d3de740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f35741b0ef0 con 0x55710d32daf0 2024-09-19T07:02:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.717+0000 7f357d3de740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f35741b0820 con 0x55710d338cb0 2024-09-19T07:02:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.717+0000 7f357d3de740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35741ada80 con 0x55710d32e410 2024-09-19T07:02:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.717+0000 7f357d3de740 10 client.0 _ll_register_callbacks cb 0x7f3564009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.717+0000 7f3573fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32daf0 0x55710d34e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.717+0000 7f3573fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32daf0 0x55710d34e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36092/0 (socket says 172.21.15.103:36092) 2024-09-19T07:02:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.717+0000 7f3573fff640 1 -- 172.21.15.103:0/2267003641 learned_addr learned my addr 172.21.15.103:0/2267003641 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f3578be4640 1 --2- 172.21.15.103:0/2267003641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d338cb0 0x55710d34bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f35737fe640 1 --2- 172.21.15.103:0/2267003641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d32e410 0x55710d34b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f3573fff640 1 -- 172.21.15.103:0/2267003641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d32e410 msgr2=0x55710d34b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f3573fff640 1 --2- 172.21.15.103:0/2267003641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d32e410 0x55710d34b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f3573fff640 1 -- 172.21.15.103:0/2267003641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d338cb0 msgr2=0x55710d34bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f3573fff640 1 --2- 172.21.15.103:0/2267003641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d338cb0 0x55710d34bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f3573fff640 1 -- 172.21.15.103:0/2267003641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55710d34c270 con 0x55710d32daf0 2024-09-19T07:02:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f35737fe640 1 --2- 172.21.15.103:0/2267003641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d32e410 0x55710d34b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f3578be4640 1 --2- 172.21.15.103:0/2267003641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d338cb0 0x55710d34bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:46.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f3573fff640 1 --2- 172.21.15.103:0/2267003641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32daf0 0x55710d34e3b0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f356402a9a0 tx=0x7f3564009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4eac10d0cdd1f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f35537fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f35537fe640 1 -- 172.21.15.103:0/2267003641 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564049060 con 0x55710d32daf0 2024-09-19T07:02:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f35537fe640 1 -- 172.21.15.103:0/2267003641 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f3564005040 con 0x55710d32daf0 2024-09-19T07:02:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f357d3de740 1 -- 172.21.15.103:0/2267003641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35680a0070 con 0x55710d32daf0 2024-09-19T07:02:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f35537fe640 1 -- 172.21.15.103:0/2267003641 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3564035040 con 0x55710d32daf0 2024-09-19T07:02:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f357d3de740 10 client.20106 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f357d3de740 1 -- 172.21.15.103:0/2267003641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55710d3b3200 con 0x55710d32daf0 2024-09-19T07:02:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f35537fe640 1 -- 172.21.15.103:0/2267003641 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35680a0070 con 0x55710d32daf0 2024-09-19T07:02:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.718+0000 7f35537fe640 1 -- 172.21.15.103:0/2267003641 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f35640035f0 con 0x55710d32daf0 2024-09-19T07:02:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.719+0000 7f357d3de740 10 client.20106 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.719+0000 7f357d3de740 10 client.20106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.719+0000 7f357d3de740 20 client.20106 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.719+0000 7f357d3de740 1 --2- 172.21.15.103:0/2267003641 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55710d3b3930 0x55710d3b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.719+0000 7f357d3de740 4 client.20106 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:02:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.719+0000 7f357d3de740 1 -- 172.21.15.103:0/2267003641 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55710d216900 con 0x55710d3b3930 2024-09-19T07:02:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.719+0000 7f35737fe640 1 --2- 172.21.15.103:0/2267003641 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55710d3b3930 0x55710d3b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.719+0000 7f35537fe640 10 client.20106 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.719+0000 7f35737fe640 1 --2- 172.21.15.103:0/2267003641 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55710d3b3930 0x55710d3b5d50 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cd996d90f17fb27 server_cookie=1ae6ed28721ca474 in_seq=0 out_seq=0 2024-09-19T07:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.729+0000 7f35537fe640 1 -- 172.21.15.103:0/2267003641 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55710d216900 con 0x55710d3b3930 2024-09-19T07:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.729+0000 7f35537fe640 10 client.20106 handle_command_reply: tid=0 2024-09-19T07:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.729+0000 7f357d3de740 1 client.20106 shutdown 2024-09-19T07:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.729+0000 7f357d3de740 1 -- 172.21.15.103:0/2267003641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32daf0 msgr2=0x55710d34e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.729+0000 7f357d3de740 1 --2- 172.21.15.103:0/2267003641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32daf0 0x55710d34e3b0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f356402a9a0 tx=0x7f3564009610 comp rx=0 tx=0).stop 2024-09-19T07:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357d3de740 1 -- 172.21.15.103:0/2267003641 shutdown_connections 2024-09-19T07:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357d3de740 1 --2- 172.21.15.103:0/2267003641 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55710d3b3930 0x55710d3b5d50 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357d3de740 1 --2- 172.21.15.103:0/2267003641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55710d338cb0 0x55710d34bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357d3de740 1 --2- 172.21.15.103:0/2267003641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55710d32e410 0x55710d34b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f35537fe640 0 client.20106 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357d3de740 1 --2- 172.21.15.103:0/2267003641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55710d32daf0 0x55710d34e3b0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357d3de740 1 -- 172.21.15.103:0/2267003641 >> 172.21.15.103:0/2267003641 conn(0x55710d32a060 msgr2=0x55710d3289a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357d3de740 1 -- 172.21.15.103:0/2267003641 shutdown_connections 2024-09-19T07:02:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357d3de740 1 -- 172.21.15.103:0/2267003641 wait complete. 2024-09-19T07:02:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357d3de740 20 client.20106 trim_cache size 0 max 16384 2024-09-19T07:02:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:02:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:02:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:02:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:02:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357a647640 1 -- 172.21.15.103:0/3058792075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f355c0a3460 msgr2=0x7f355c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357a647640 1 --2- 172.21.15.103:0/3058792075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f355c0a3460 0x7f355c0a5920 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f356c005c40 tx=0x7f356c009280 comp rx=0 tx=0).stop 2024-09-19T07:02:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357a647640 1 -- 172.21.15.103:0/3058792075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3574103620 msgr2=0x7f357419aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357a647640 1 --2- 172.21.15.103:0/3058792075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3574103620 0x7f357419aa50 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f356800d950 tx=0x7f356800de10 comp rx=0 tx=0).stop 2024-09-19T07:02:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357a647640 1 -- 172.21.15.103:0/3058792075 shutdown_connections 2024-09-19T07:02:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357a647640 1 --2- 172.21.15.103:0/3058792075 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f355c0a3460 0x7f355c0a5920 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357a647640 1 --2- 172.21.15.103:0/3058792075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f357410c0d0 0x7f357419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357a647640 1 --2- 172.21.15.103:0/3058792075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3574103620 0x7f357419aa50 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357a647640 1 --2- 172.21.15.103:0/3058792075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3574102d00 0x7f357419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357a647640 1 -- 172.21.15.103:0/3058792075 >> 172.21.15.103:0/3058792075 conn(0x7f35740fc6f0 msgr2=0x7f35740fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357a647640 1 -- 172.21.15.103:0/3058792075 shutdown_connections 2024-09-19T07:02:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:46.730+0000 7f357a647640 1 -- 172.21.15.103:0/3058792075 wait complete. 2024-09-19T07:02:48.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:47 smithi174 ceph-mon[35637]: pgmap v423: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-09-19T07:02:48.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:47 smithi103 ceph-mon[31158]: pgmap v423: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-09-19T07:02:48.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:47 smithi134 ceph-mon[35356]: pgmap v423: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-09-19T07:02:50.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:49 smithi174 ceph-mon[35637]: pgmap v424: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:02:50.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:49 smithi103 ceph-mon[31158]: pgmap v424: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:02:50.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:49 smithi134 ceph-mon[35356]: pgmap v424: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:02:52.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:51 smithi174 ceph-mon[35637]: pgmap v425: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2024-09-19T07:02:52.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:51 smithi103 ceph-mon[31158]: pgmap v425: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2024-09-19T07:02:52.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:51 smithi134 ceph-mon[35356]: pgmap v425: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2024-09-19T07:02:52.406 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:02:52.668 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:53.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.119+0000 7fd29dcb4640 1 -- 172.21.15.103:0/629147698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd298102570 msgr2=0x7fd2981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.119+0000 7fd29dcb4640 1 --2- 172.21.15.103:0/629147698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd298102570 0x7fd2981029b0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fd280009900 tx=0x7fd28002a480 comp rx=0 tx=0).stop 2024-09-19T07:02:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.119+0000 7fd29dcb4640 1 -- 172.21.15.103:0/629147698 shutdown_connections 2024-09-19T07:02:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.119+0000 7fd29dcb4640 1 --2- 172.21.15.103:0/629147698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd298102ef0 0x7fd29810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.119+0000 7fd29dcb4640 1 --2- 172.21.15.103:0/629147698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd298102570 0x7fd2981029b0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.119+0000 7fd29dcb4640 1 --2- 172.21.15.103:0/629147698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd298100e20 0x7fd298101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.119+0000 7fd29dcb4640 1 -- 172.21.15.103:0/629147698 >> 172.21.15.103:0/629147698 conn(0x7fd2980fc7d0 msgr2=0x7fd2980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.119+0000 7fd29dcb4640 1 -- 172.21.15.103:0/629147698 shutdown_connections 2024-09-19T07:02:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.119+0000 7fd29dcb4640 1 -- 172.21.15.103:0/629147698 wait complete. 2024-09-19T07:02:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.119+0000 7fd29dcb4640 1 Processor -- start 2024-09-19T07:02:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd29dcb4640 1 -- start start 2024-09-19T07:02:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd29dcb4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd298100e20 0x7fd298199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd29dcb4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd298102570 0x7fd2981998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd29dcb4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd298102ef0 0x7fd2981a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd29dcb4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd298110f80 con 0x7fd298100e20 2024-09-19T07:02:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd29dcb4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd298110e00 con 0x7fd298102ef0 2024-09-19T07:02:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd29dcb4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd298111100 con 0x7fd298102570 2024-09-19T07:02:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd2977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd298100e20 0x7fd298199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd2977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd298100e20 0x7fd298199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58384/0 (socket says 172.21.15.103:58384) 2024-09-19T07:02:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd2977fe640 1 -- 172.21.15.103:0/499685682 learned_addr learned my addr 172.21.15.103:0/499685682 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd28ffff640 1 --2- 172.21.15.103:0/499685682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd298102570 0x7fd2981998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd297fff640 1 --2- 172.21.15.103:0/499685682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd298102ef0 0x7fd2981a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd2977fe640 1 -- 172.21.15.103:0/499685682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd298102570 msgr2=0x7fd2981998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:53.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd2977fe640 1 --2- 172.21.15.103:0/499685682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd298102570 0x7fd2981998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:53.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd2977fe640 1 -- 172.21.15.103:0/499685682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd298102ef0 msgr2=0x7fd2981a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:53.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd2977fe640 1 --2- 172.21.15.103:0/499685682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd298102ef0 0x7fd2981a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:53.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd2977fe640 1 -- 172.21.15.103:0/499685682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2981a4950 con 0x7fd298100e20 2024-09-19T07:02:53.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd28ffff640 1 --2- 172.21.15.103:0/499685682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd298102570 0x7fd2981998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:53.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd297fff640 1 --2- 172.21.15.103:0/499685682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd298102ef0 0x7fd2981a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:53.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd2977fe640 1 --2- 172.21.15.103:0/499685682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd298100e20 0x7fd298199390 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fd28800c970 tx=0x7fd28800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:53.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.120+0000 7fd2957fa640 1 -- 172.21.15.103:0/499685682 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288007bb0 con 0x7fd298100e20 2024-09-19T07:02:53.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.121+0000 7fd2957fa640 1 -- 172.21.15.103:0/499685682 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd288007d50 con 0x7fd298100e20 2024-09-19T07:02:53.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.121+0000 7fd29dcb4640 1 -- 172.21.15.103:0/499685682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2981a4b20 con 0x7fd298100e20 2024-09-19T07:02:53.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.121+0000 7fd2957fa640 1 -- 172.21.15.103:0/499685682 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28800f450 con 0x7fd298100e20 2024-09-19T07:02:53.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.121+0000 7fd29dcb4640 1 -- 172.21.15.103:0/499685682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2981a5030 con 0x7fd298100e20 2024-09-19T07:02:53.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.122+0000 7fd29dcb4640 1 -- 172.21.15.103:0/499685682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd298101200 con 0x7fd298100e20 2024-09-19T07:02:53.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.123+0000 7fd2957fa640 1 -- 172.21.15.103:0/499685682 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd288005020 con 0x7fd298100e20 2024-09-19T07:02:53.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.123+0000 7fd2957fa640 1 --2- 172.21.15.103:0/499685682 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2840a3430 0x7fd2840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:53.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.124+0000 7fd28ffff640 1 --2- 172.21.15.103:0/499685682 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2840a3430 0x7fd2840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:53.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.124+0000 7fd2957fa640 1 -- 172.21.15.103:0/499685682 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7fd2880cfb60 con 0x7fd298100e20 2024-09-19T07:02:53.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.124+0000 7fd28ffff640 1 --2- 172.21.15.103:0/499685682 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2840a3430 0x7fd2840a58f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd280000c00 tx=0x7fd2800041e0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:53.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.128+0000 7fd2957fa640 1 -- 172.21.15.103:0/499685682 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd288014030 con 0x7fd298100e20 2024-09-19T07:02:53.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.328+0000 7fd29dcb4640 1 -- 172.21.15.103:0/499685682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd29806a560 con 0x7fd298100e20 2024-09-19T07:02:53.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.329+0000 7fd2957fa640 1 -- 172.21.15.103:0/499685682 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fd28809c520 con 0x7fd298100e20 2024-09-19T07:02:53.331 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:02:53.331 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:02:53.331 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:02:53.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.331+0000 7fd29dcb4640 1 -- 172.21.15.103:0/499685682 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2840a3430 msgr2=0x7fd2840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:53.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.331+0000 7fd29dcb4640 1 --2- 172.21.15.103:0/499685682 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2840a3430 0x7fd2840a58f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd280000c00 tx=0x7fd2800041e0 comp rx=0 tx=0).stop 2024-09-19T07:02:53.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.331+0000 7fd29dcb4640 1 -- 172.21.15.103:0/499685682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd298100e20 msgr2=0x7fd298199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:53.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.331+0000 7fd29dcb4640 1 --2- 172.21.15.103:0/499685682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd298100e20 0x7fd298199390 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fd28800c970 tx=0x7fd28800ce30 comp rx=0 tx=0).stop 2024-09-19T07:02:53.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.331+0000 7fd29dcb4640 1 -- 172.21.15.103:0/499685682 shutdown_connections 2024-09-19T07:02:53.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.331+0000 7fd29dcb4640 1 --2- 172.21.15.103:0/499685682 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2840a3430 0x7fd2840a58f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:53.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.331+0000 7fd29dcb4640 1 --2- 172.21.15.103:0/499685682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd298102ef0 0x7fd2981a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:53.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.331+0000 7fd29dcb4640 1 --2- 172.21.15.103:0/499685682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd298102570 0x7fd2981998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:53.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.331+0000 7fd29dcb4640 1 --2- 172.21.15.103:0/499685682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd298100e20 0x7fd298199390 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:53.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.331+0000 7fd29dcb4640 1 -- 172.21.15.103:0/499685682 >> 172.21.15.103:0/499685682 conn(0x7fd2980fc7d0 msgr2=0x7fd298105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:53.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.331+0000 7fd29dcb4640 1 -- 172.21.15.103:0/499685682 shutdown_connections 2024-09-19T07:02:53.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:53.331+0000 7fd29dcb4640 1 -- 172.21.15.103:0/499685682 wait complete. 2024-09-19T07:02:54.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:53 smithi174 ceph-mon[35637]: pgmap v426: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-09-19T07:02:54.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:53 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/499685682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:02:54.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:53 smithi103 ceph-mon[31158]: pgmap v426: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-09-19T07:02:54.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:53 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/499685682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:02:54.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:53 smithi134 ceph-mon[35356]: pgmap v426: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-09-19T07:02:54.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:53 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/499685682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:02:54.222 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 6244df2c --release --await' 2024-09-19T07:02:54.222 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 6244df2c --release --await 2024-09-19T07:02:54.474 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:54.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.935+0000 7f85181d7640 1 -- 172.21.15.103:0/2952497974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8510106a80 msgr2=0x7f8510106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:54.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.935+0000 7f85181d7640 1 --2- 172.21.15.103:0/2952497974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8510106a80 0x7f8510106e60 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f850c009930 tx=0x7f850c02a420 comp rx=0 tx=0).stop 2024-09-19T07:02:54.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.935+0000 7f85181d7640 1 -- 172.21.15.103:0/2952497974 shutdown_connections 2024-09-19T07:02:54.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.935+0000 7f85181d7640 1 --2- 172.21.15.103:0/2952497974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8510101420 0x7f851010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.935+0000 7f85181d7640 1 --2- 172.21.15.103:0/2952497974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8510100a80 0x7f8510100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.935+0000 7f85181d7640 1 --2- 172.21.15.103:0/2952497974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8510106a80 0x7f8510106e60 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.935+0000 7f85181d7640 1 -- 172.21.15.103:0/2952497974 >> 172.21.15.103:0/2952497974 conn(0x7f85100fc7a0 msgr2=0x7f85100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:54.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.935+0000 7f85181d7640 1 -- 172.21.15.103:0/2952497974 shutdown_connections 2024-09-19T07:02:54.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.935+0000 7f85181d7640 1 -- 172.21.15.103:0/2952497974 wait complete. 2024-09-19T07:02:54.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f85181d7640 1 Processor -- start 2024-09-19T07:02:54.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f85181d7640 1 -- start start 2024-09-19T07:02:54.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f85181d7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8510100a80 0x7f85101a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f85181d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8510101420 0x7f85101a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f85181d7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8510106a80 0x7f851019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f85181d7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f851010fe10 con 0x7f8510101420 2024-09-19T07:02:54.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f85181d7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f851010fc90 con 0x7f8510100a80 2024-09-19T07:02:54.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f85181d7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f851010ff90 con 0x7f8510106a80 2024-09-19T07:02:54.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f8515772640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8510101420 0x7f85101a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f8515772640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8510101420 0x7f85101a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58404/0 (socket says 172.21.15.103:58404) 2024-09-19T07:02:54.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f8515772640 1 -- 172.21.15.103:0/1792649945 learned_addr learned my addr 172.21.15.103:0/1792649945 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:54.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f8515f73640 1 --2- 172.21.15.103:0/1792649945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8510100a80 0x7f85101a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f8516774640 1 --2- 172.21.15.103:0/1792649945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8510106a80 0x7f851019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f8515772640 1 -- 172.21.15.103:0/1792649945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8510106a80 msgr2=0x7f851019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:54.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f8515772640 1 --2- 172.21.15.103:0/1792649945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8510106a80 0x7f851019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f8515772640 1 -- 172.21.15.103:0/1792649945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8510100a80 msgr2=0x7f85101a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:54.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f8515772640 1 --2- 172.21.15.103:0/1792649945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8510100a80 0x7f85101a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f8515772640 1 -- 172.21.15.103:0/1792649945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851019ad20 con 0x7f8510101420 2024-09-19T07:02:54.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f8516774640 1 --2- 172.21.15.103:0/1792649945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8510106a80 0x7f851019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:54.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.936+0000 7f8515f73640 1 --2- 172.21.15.103:0/1792649945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8510100a80 0x7f85101a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:54.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.937+0000 7f8515772640 1 --2- 172.21.15.103:0/1792649945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8510101420 0x7f85101a0970 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f850000b4a0 tx=0x7f850000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:54.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.937+0000 7f8506ffd640 1 -- 172.21.15.103:0/1792649945 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500014070 con 0x7f8510101420 2024-09-19T07:02:54.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.937+0000 7f85181d7640 1 -- 172.21.15.103:0/1792649945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851019b010 con 0x7f8510101420 2024-09-19T07:02:54.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.937+0000 7f8506ffd640 1 -- 172.21.15.103:0/1792649945 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8500010af0 con 0x7f8510101420 2024-09-19T07:02:54.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.937+0000 7f8506ffd640 1 -- 172.21.15.103:0/1792649945 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500017d10 con 0x7f8510101420 2024-09-19T07:02:54.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.937+0000 7f85181d7640 1 -- 172.21.15.103:0/1792649945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85101a7300 con 0x7f8510101420 2024-09-19T07:02:54.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.939+0000 7f8506ffd640 1 -- 172.21.15.103:0/1792649945 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8500017070 con 0x7f8510101420 2024-09-19T07:02:54.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.939+0000 7f851a5b0740 1 Processor -- start 2024-09-19T07:02:54.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f851a5b0740 1 -- start start 2024-09-19T07:02:54.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f851a5b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2060 0x5565e24b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f851a5b0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2980 0x5565e24bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f851a5b0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 0x5565e24bf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f851a5b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5565e2417620 con 0x5565e24b2060 2024-09-19T07:02:54.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f851a5b0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565e23bee50 con 0x5565e24bd240 2024-09-19T07:02:54.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f851a5b0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5565e24135e0 con 0x5565e24b2980 2024-09-19T07:02:54.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f8516774640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2060 0x5565e24b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f8516774640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2060 0x5565e24b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58408/0 (socket says 172.21.15.103:58408) 2024-09-19T07:02:54.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f8516774640 1 -- 172.21.15.103:0/3333437414 learned_addr learned my addr 172.21.15.103:0/3333437414 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:54.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f8515f73640 1 --2- 172.21.15.103:0/3333437414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2980 0x5565e24bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f8506ffd640 1 --2- 172.21.15.103:0/1792649945 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f84fc0a33b0 0x7f84fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f8515772640 1 --2- 172.21.15.103:0/3333437414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 0x5565e24bf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f8516774640 1 -- 172.21.15.103:0/3333437414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2980 msgr2=0x5565e24bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:54.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f8516774640 1 --2- 172.21.15.103:0/3333437414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2980 0x5565e24bcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f8516774640 1 -- 172.21.15.103:0/3333437414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 msgr2=0x5565e24bf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:54.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f8516774640 1 --2- 172.21.15.103:0/3333437414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 0x5565e24bf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f8516774640 1 -- 172.21.15.103:0/3333437414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e23afd40 con 0x5565e24b2060 2024-09-19T07:02:54.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f8515772640 1 --2- 172.21.15.103:0/3333437414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 0x5565e24bf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:54.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f8506ffd640 1 -- 172.21.15.103:0/1792649945 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f85000cf110 con 0x7f8510101420 2024-09-19T07:02:54.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f8516774640 1 --2- 172.21.15.103:0/3333437414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2060 0x5565e24b2440 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f8508007f30 tx=0x7f850800c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59b3f477045af3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:54.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.940+0000 7f8516774640 1 --2- 172.21.15.103:0/1792649945 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f84fc0a33b0 0x7f84fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.941+0000 7f84fbfff640 1 -- 172.21.15.103:0/3333437414 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508014070 con 0x5565e24b2060 2024-09-19T07:02:54.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.941+0000 7f84fbfff640 1 -- 172.21.15.103:0/3333437414 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8508019930 con 0x5565e24b2060 2024-09-19T07:02:54.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.941+0000 7f84fbfff640 1 -- 172.21.15.103:0/3333437414 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850800ec30 con 0x5565e24b2060 2024-09-19T07:02:54.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.941+0000 7f851a5b0740 1 -- 172.21.15.103:0/3333437414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2060 msgr2=0x5565e24b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:54.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.941+0000 7f8516774640 1 --2- 172.21.15.103:0/1792649945 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f84fc0a33b0 0x7f84fc0a5870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8508016000 tx=0x7f8508017360 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:54.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.941+0000 7f851a5b0740 1 --2- 172.21.15.103:0/3333437414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2060 0x5565e24b2440 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f8508007f30 tx=0x7f850800c750 comp rx=0 tx=0).stop 2024-09-19T07:02:54.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.941+0000 7f851a5b0740 1 -- 172.21.15.103:0/3333437414 shutdown_connections 2024-09-19T07:02:54.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.941+0000 7f851a5b0740 1 --2- 172.21.15.103:0/3333437414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 0x5565e24bf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.941+0000 7f851a5b0740 1 --2- 172.21.15.103:0/3333437414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2980 0x5565e24bcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.941+0000 7f851a5b0740 1 --2- 172.21.15.103:0/3333437414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2060 0x5565e24b2440 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.941+0000 7f851a5b0740 1 -- 172.21.15.103:0/3333437414 >> 172.21.15.103:0/3333437414 conn(0x5565e24ae5f0 msgr2=0x5565e24ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:54.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.942+0000 7f851a5b0740 1 -- 172.21.15.103:0/3333437414 shutdown_connections 2024-09-19T07:02:54.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.942+0000 7f851a5b0740 1 -- 172.21.15.103:0/3333437414 wait complete. 2024-09-19T07:02:54.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.942+0000 7f851a5b0740 1 -- start start 2024-09-19T07:02:54.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.942+0000 7f851a5b0740 1 Processor -- start 2024-09-19T07:02:54.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.942+0000 7f851a5b0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2060 0x5565e24ba0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.942+0000 7f851a5b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2980 0x5565e24ba610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f851a5b0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 0x5565e24bab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f851a5b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5565e24bfe10 con 0x5565e24b2980 2024-09-19T07:02:54.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f851a5b0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565e239ff10 con 0x5565e24bd240 2024-09-19T07:02:54.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f851a5b0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5565e24bff90 con 0x5565e24b2060 2024-09-19T07:02:54.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f851a5b0740 10 client.0 _ll_register_callbacks cb 0x5565e23c3db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:54.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f8515772640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2980 0x5565e24ba610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f8515772640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2980 0x5565e24ba610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58410/0 (socket says 172.21.15.103:58410) 2024-09-19T07:02:54.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f8515772640 1 -- 172.21.15.103:0/333256174 learned_addr learned my addr 172.21.15.103:0/333256174 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:54.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f8515f73640 1 --2- 172.21.15.103:0/333256174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2060 0x5565e24ba0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f8516774640 1 --2- 172.21.15.103:0/333256174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 0x5565e24bab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f8515772640 1 -- 172.21.15.103:0/333256174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2060 msgr2=0x5565e24ba0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:54.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f8515772640 1 --2- 172.21.15.103:0/333256174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2060 0x5565e24ba0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f8515772640 1 -- 172.21.15.103:0/333256174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 msgr2=0x5565e24bab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:54.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f8515772640 1 --2- 172.21.15.103:0/333256174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 0x5565e24bab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f8515772640 1 -- 172.21.15.103:0/333256174 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e2126580 con 0x5565e24b2980 2024-09-19T07:02:54.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f8516774640 1 --2- 172.21.15.103:0/333256174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 0x5565e24bab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:54.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f8515772640 1 --2- 172.21.15.103:0/333256174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2980 0x5565e24ba610 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f850009b410 tx=0x7f8500017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4461bc3fcad555 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:54.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f84f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:54.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f84f97fa640 1 -- 172.21.15.103:0/333256174 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850001e050 con 0x5565e24b2980 2024-09-19T07:02:54.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f84f97fa640 1 -- 172.21.15.103:0/333256174 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f8500097af0 con 0x5565e24b2980 2024-09-19T07:02:54.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f851a5b0740 1 -- 172.21.15.103:0/333256174 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8508019930 con 0x5565e24b2980 2024-09-19T07:02:54.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f84f97fa640 1 -- 172.21.15.103:0/333256174 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8500009230 con 0x5565e24b2980 2024-09-19T07:02:54.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f851a5b0740 10 client.20124 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:54.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.943+0000 7f851a5b0740 1 -- 172.21.15.103:0/333256174 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5565e21f0960 con 0x5565e24b2980 2024-09-19T07:02:54.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.944+0000 7f84f97fa640 1 -- 172.21.15.103:0/333256174 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8508019930 con 0x5565e24b2980 2024-09-19T07:02:54.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.944+0000 7f84f97fa640 1 -- 172.21.15.103:0/333256174 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f85000cca60 con 0x5565e24b2980 2024-09-19T07:02:54.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.944+0000 7f851a5b0740 10 client.20124 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:54.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.944+0000 7f851a5b0740 10 client.20124 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:02:54.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.944+0000 7f851a5b0740 20 client.20124 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:54.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.944+0000 7f851a5b0740 1 --2- 172.21.15.103:0/333256174 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565e24ccd10 0x5565e24cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.944+0000 7f851a5b0740 4 client.20124 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:54.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.944+0000 7f851a5b0740 1 -- 172.21.15.103:0/333256174 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565e2424b90 con 0x5565e24ccd10 2024-09-19T07:02:54.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.944+0000 7f8515f73640 1 --2- 172.21.15.103:0/333256174 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565e24ccd10 0x5565e24cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.944+0000 7f8515f73640 1 --2- 172.21.15.103:0/333256174 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565e24ccd10 0x5565e24cf120 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=388228d77086d20e server_cookie=13fadec7a684d250 in_seq=0 out_seq=0 2024-09-19T07:02:54.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.944+0000 7f84f97fa640 10 client.20124 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:54.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.945+0000 7f84f97fa640 1 -- 172.21.15.103:0/333256174 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565e2424b90 con 0x5565e24ccd10 2024-09-19T07:02:54.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.945+0000 7f84f97fa640 10 client.20124 handle_command_reply: tid=0 2024-09-19T07:02:54.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.945+0000 7f851a5b0740 1 client.20124 shutdown 2024-09-19T07:02:54.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.945+0000 7f851a5b0740 1 -- 172.21.15.103:0/333256174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2980 msgr2=0x5565e24ba610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:54.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.945+0000 7f851a5b0740 1 --2- 172.21.15.103:0/333256174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2980 0x5565e24ba610 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f850009b410 tx=0x7f8500017730 comp rx=0 tx=0).stop 2024-09-19T07:02:54.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.945+0000 7f851a5b0740 1 -- 172.21.15.103:0/333256174 shutdown_connections 2024-09-19T07:02:54.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.945+0000 7f851a5b0740 1 --2- 172.21.15.103:0/333256174 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565e24ccd10 0x5565e24cf120 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.945+0000 7f851a5b0740 1 --2- 172.21.15.103:0/333256174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 0x5565e24bab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.945+0000 7f851a5b0740 1 --2- 172.21.15.103:0/333256174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2980 0x5565e24ba610 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.945+0000 7f851a5b0740 1 --2- 172.21.15.103:0/333256174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2060 0x5565e24ba0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.945+0000 7f84f97fa640 0 client.20124 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:54.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.945+0000 7f851a5b0740 1 -- 172.21.15.103:0/333256174 >> 172.21.15.103:0/333256174 conn(0x5565e24ae5f0 msgr2=0x5565e24b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:54.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.946+0000 7f851a5b0740 1 -- 172.21.15.103:0/333256174 shutdown_connections 2024-09-19T07:02:54.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.946+0000 7f851a5b0740 1 -- 172.21.15.103:0/333256174 wait complete. 2024-09-19T07:02:54.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.946+0000 7f851a5b0740 20 client.20124 trim_cache size 0 max 16384 2024-09-19T07:02:54.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f851a5b0740 1 Processor -- start 2024-09-19T07:02:54.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f851a5b0740 1 -- start start 2024-09-19T07:02:54.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f851a5b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2060 0x5565e24b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f851a5b0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2980 0x5565e24ad640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f851a5b0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 0x5565e24d49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f851a5b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f85101afbe0 con 0x5565e24b2060 2024-09-19T07:02:54.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f851a5b0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85101b1720 con 0x5565e24bd240 2024-09-19T07:02:54.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f851a5b0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85101b02b0 con 0x5565e24b2980 2024-09-19T07:02:54.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f8515f73640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2060 0x5565e24b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f8515f73640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2060 0x5565e24b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58426/0 (socket says 172.21.15.103:58426) 2024-09-19T07:02:54.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f8515f73640 1 -- 172.21.15.103:0/229769384 learned_addr learned my addr 172.21.15.103:0/229769384 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:54.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f8516774640 1 --2- 172.21.15.103:0/229769384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 0x5565e24d49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f8515772640 1 --2- 172.21.15.103:0/229769384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2980 0x5565e24ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f8515f73640 1 -- 172.21.15.103:0/229769384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2980 msgr2=0x5565e24ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:54.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f8515f73640 1 --2- 172.21.15.103:0/229769384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2980 0x5565e24ad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f8515f73640 1 -- 172.21.15.103:0/229769384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 msgr2=0x5565e24d49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f8515f73640 1 --2- 172.21.15.103:0/229769384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 0x5565e24d49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f8515f73640 1 -- 172.21.15.103:0/229769384 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e24ae090 con 0x5565e24b2060 2024-09-19T07:02:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f8515772640 1 --2- 172.21.15.103:0/229769384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2980 0x5565e24ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.963+0000 7f8515f73640 1 --2- 172.21.15.103:0/229769384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2060 0x5565e24b4ac0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f850c009930 tx=0x7f850c02aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebeac5bbd5102079 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.964+0000 7f84fb7fe640 1 -- 172.21.15.103:0/229769384 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c04b020 con 0x5565e24b2060 2024-09-19T07:02:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.964+0000 7f84fb7fe640 1 -- 172.21.15.103:0/229769384 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f850c005520 con 0x5565e24b2060 2024-09-19T07:02:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.964+0000 7f84fb7fe640 1 -- 172.21.15.103:0/229769384 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c035040 con 0x5565e24b2060 2024-09-19T07:02:54.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.964+0000 7f851a5b0740 1 -- 172.21.15.103:0/229769384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2060 msgr2=0x5565e24b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:54.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.964+0000 7f851a5b0740 1 --2- 172.21.15.103:0/229769384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2060 0x5565e24b4ac0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f850c009930 tx=0x7f850c02aa60 comp rx=0 tx=0).stop 2024-09-19T07:02:54.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.964+0000 7f851a5b0740 1 -- 172.21.15.103:0/229769384 shutdown_connections 2024-09-19T07:02:54.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.964+0000 7f851a5b0740 1 --2- 172.21.15.103:0/229769384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24bd240 0x5565e24d49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.964+0000 7f851a5b0740 1 --2- 172.21.15.103:0/229769384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2980 0x5565e24ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.964+0000 7f851a5b0740 1 --2- 172.21.15.103:0/229769384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24b2060 0x5565e24b4ac0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.964+0000 7f851a5b0740 1 -- 172.21.15.103:0/229769384 >> 172.21.15.103:0/229769384 conn(0x5565e24ae5f0 msgr2=0x5565e24acf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:54.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.964+0000 7f851a5b0740 1 -- 172.21.15.103:0/229769384 shutdown_connections 2024-09-19T07:02:54.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.964+0000 7f851a5b0740 1 -- 172.21.15.103:0/229769384 wait complete. 2024-09-19T07:02:54.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.965+0000 7f851a5b0740 1 -- start start 2024-09-19T07:02:54.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.965+0000 7f851a5b0740 1 Processor -- start 2024-09-19T07:02:54.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.965+0000 7f851a5b0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2060 0x5565e24c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.965+0000 7f851a5b0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24b2980 0x5565e24c5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.965+0000 7f851a5b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24bd240 0x5565e24c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.965+0000 7f851a5b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f85101b1050 con 0x5565e24bd240 2024-09-19T07:02:54.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.965+0000 7f851a5b0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85101b0980 con 0x5565e24b2980 2024-09-19T07:02:54.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.965+0000 7f851a5b0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85101adbe0 con 0x5565e24b2060 2024-09-19T07:02:54.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.965+0000 7f851a5b0740 10 client.0 _ll_register_callbacks cb 0x7f850009b090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:54.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.965+0000 7f8516774640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24bd240 0x5565e24c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.965+0000 7f8516774640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24bd240 0x5565e24c21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58436/0 (socket says 172.21.15.103:58436) 2024-09-19T07:02:54.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.965+0000 7f8516774640 1 -- 172.21.15.103:0/3214427738 learned_addr learned my addr 172.21.15.103:0/3214427738 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:54.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.965+0000 7f8515772640 1 --2- 172.21.15.103:0/3214427738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24b2980 0x5565e24c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.966+0000 7f8515f73640 1 --2- 172.21.15.103:0/3214427738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2060 0x5565e24c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.966+0000 7f8516774640 1 -- 172.21.15.103:0/3214427738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2060 msgr2=0x5565e24c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:54.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.966+0000 7f8516774640 1 --2- 172.21.15.103:0/3214427738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2060 0x5565e24c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.966+0000 7f8516774640 1 -- 172.21.15.103:0/3214427738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24b2980 msgr2=0x5565e24c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.966+0000 7f8516774640 1 --2- 172.21.15.103:0/3214427738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24b2980 0x5565e24c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.966+0000 7f8516774640 1 -- 172.21.15.103:0/3214427738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e24c2840 con 0x5565e24bd240 2024-09-19T07:02:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.966+0000 7f8516774640 1 --2- 172.21.15.103:0/3214427738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24bd240 0x5565e24c21b0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f8508007f30 tx=0x7f8508017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86a2ee484b43c16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.966+0000 7f84f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.966+0000 7f84f97fa640 1 -- 172.21.15.103:0/3214427738 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508014040 con 0x5565e24bd240 2024-09-19T07:02:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.966+0000 7f84f97fa640 1 -- 172.21.15.103:0/3214427738 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f850800f710 con 0x5565e24bd240 2024-09-19T07:02:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.966+0000 7f851a5b0740 1 -- 172.21.15.103:0/3214427738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f850c005520 con 0x5565e24bd240 2024-09-19T07:02:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.966+0000 7f84f97fa640 1 -- 172.21.15.103:0/3214427738 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8508007b00 con 0x5565e24bd240 2024-09-19T07:02:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.966+0000 7f851a5b0740 10 client.20130 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.966+0000 7f851a5b0740 1 -- 172.21.15.103:0/3214427738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5565e25382b0 con 0x5565e24bd240 2024-09-19T07:02:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.966+0000 7f84f97fa640 1 -- 172.21.15.103:0/3214427738 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f850c005520 con 0x5565e24bd240 2024-09-19T07:02:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.967+0000 7f84f97fa640 1 -- 172.21.15.103:0/3214427738 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8508019680 con 0x5565e24bd240 2024-09-19T07:02:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.967+0000 7f851a5b0740 10 client.20130 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.967+0000 7f851a5b0740 10 client.20130 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:02:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.967+0000 7f851a5b0740 20 client.20130 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.967+0000 7f851a5b0740 1 --2- 172.21.15.103:0/3214427738 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565e2538940 0x5565e253ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.967+0000 7f851a5b0740 4 client.20130 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6244df2c", "release": true, "await": true}] 2024-09-19T07:02:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.967+0000 7f851a5b0740 1 -- 172.21.15.103:0/3214427738 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6244df2c", "release": true, "await": true}) -- 0x5565e241f0a0 con 0x5565e2538940 2024-09-19T07:02:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.967+0000 7f8515f73640 1 --2- 172.21.15.103:0/3214427738 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565e2538940 0x5565e253ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.967+0000 7f8515f73640 1 --2- 172.21.15.103:0/3214427738 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565e2538940 0x5565e253ad60 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5674bf8f3bc72d43 server_cookie=8d660a3f16bf8fcb in_seq=0 out_seq=0 2024-09-19T07:02:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.967+0000 7f84f97fa640 10 client.20130 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.970+0000 7f84f97fa640 1 -- 172.21.15.103:0/3214427738 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5565e241f0a0 con 0x5565e2538940 2024-09-19T07:02:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.970+0000 7f84f97fa640 10 client.20130 handle_command_reply: tid=0 2024-09-19T07:02:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.970+0000 7f851a5b0740 1 client.20130 shutdown 2024-09-19T07:02:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.970+0000 7f851a5b0740 1 -- 172.21.15.103:0/3214427738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24bd240 msgr2=0x5565e24c21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.970+0000 7f851a5b0740 1 --2- 172.21.15.103:0/3214427738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24bd240 0x5565e24c21b0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f8508007f30 tx=0x7f8508017000 comp rx=0 tx=0).stop 2024-09-19T07:02:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.970+0000 7f851a5b0740 1 -- 172.21.15.103:0/3214427738 shutdown_connections 2024-09-19T07:02:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.970+0000 7f851a5b0740 1 --2- 172.21.15.103:0/3214427738 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5565e2538940 0x5565e253ad60 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.970+0000 7f851a5b0740 1 --2- 172.21.15.103:0/3214427738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565e24bd240 0x5565e24c21b0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.970+0000 7f851a5b0740 1 --2- 172.21.15.103:0/3214427738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565e24b2980 0x5565e24c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.970+0000 7f851a5b0740 1 --2- 172.21.15.103:0/3214427738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565e24b2060 0x5565e24c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.970+0000 7f84f97fa640 0 client.20130 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:54.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.970+0000 7f851a5b0740 1 -- 172.21.15.103:0/3214427738 >> 172.21.15.103:0/3214427738 conn(0x5565e24ae5f0 msgr2=0x5565e24ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:54.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.970+0000 7f851a5b0740 1 -- 172.21.15.103:0/3214427738 shutdown_connections 2024-09-19T07:02:54.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.970+0000 7f851a5b0740 1 -- 172.21.15.103:0/3214427738 wait complete. 2024-09-19T07:02:54.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.971+0000 7f851a5b0740 20 client.20130 trim_cache size 0 max 16384 2024-09-19T07:02:54.990 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:02:54.991 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T07:02:54.991 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T07:02:54.991 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 48, 2024-09-19T07:02:54.991 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:02:54.991 INFO:teuthology.orchestra.run.smithi103.stdout: "6244df2c": { 2024-09-19T07:02:54.991 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 48, 2024-09-19T07:02:54.991 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:02:54.991 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:02:54.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:02:54.991 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:02:54.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:02:54.992 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:02:54.992 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 142.2, 2024-09-19T07:02:54.992 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:02:54.992 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:02:54.992 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:02:54.992 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:02:54.992 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:02:54.992 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:02:54.992 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:54.993 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:54.993 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:54.993 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:54.993 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:02:54.993 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:02:54.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.971+0000 7f85181d7640 1 -- 172.21.15.103:0/1792649945 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f84fc0a33b0 msgr2=0x7f84fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:54.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.971+0000 7f85181d7640 1 --2- 172.21.15.103:0/1792649945 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f84fc0a33b0 0x7f84fc0a5870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8508016000 tx=0x7f8508017360 comp rx=0 tx=0).stop 2024-09-19T07:02:54.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.971+0000 7f85181d7640 1 -- 172.21.15.103:0/1792649945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8510101420 msgr2=0x7f85101a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:54.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.971+0000 7f85181d7640 1 --2- 172.21.15.103:0/1792649945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8510101420 0x7f85101a0970 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f850000b4a0 tx=0x7f850000b960 comp rx=0 tx=0).stop 2024-09-19T07:02:54.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.971+0000 7f85181d7640 1 -- 172.21.15.103:0/1792649945 shutdown_connections 2024-09-19T07:02:54.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.971+0000 7f85181d7640 1 --2- 172.21.15.103:0/1792649945 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f84fc0a33b0 0x7f84fc0a5870 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.971+0000 7f85181d7640 1 --2- 172.21.15.103:0/1792649945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8510106a80 0x7f851019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.971+0000 7f85181d7640 1 --2- 172.21.15.103:0/1792649945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8510101420 0x7f85101a0970 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.971+0000 7f85181d7640 1 --2- 172.21.15.103:0/1792649945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8510100a80 0x7f85101a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:54.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.971+0000 7f85181d7640 1 -- 172.21.15.103:0/1792649945 >> 172.21.15.103:0/1792649945 conn(0x7f85100fc7a0 msgr2=0x7f85100fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:54.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.971+0000 7f85181d7640 1 -- 172.21.15.103:0/1792649945 shutdown_connections 2024-09-19T07:02:54.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:54.971+0000 7f85181d7640 1 -- 172.21.15.103:0/1792649945 wait complete. 2024-09-19T07:02:55.557 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6244df2c', total seconds elapsed: 25.4 2024-09-19T07:02:56.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:55 smithi174 ceph-mon[35637]: pgmap v427: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-09-19T07:02:56.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:55 smithi103 ceph-mon[31158]: pgmap v427: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-09-19T07:02:56.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:55 smithi134 ceph-mon[35356]: pgmap v427: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-09-19T07:02:58.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:57 smithi174 ceph-mon[35637]: pgmap v428: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.9 KiB/s wr, 14 op/s 2024-09-19T07:02:58.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:57 smithi103 ceph-mon[31158]: pgmap v428: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.9 KiB/s wr, 14 op/s 2024-09-19T07:02:58.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:57 smithi134 ceph-mon[35356]: pgmap v428: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.9 KiB/s wr, 14 op/s 2024-09-19T07:02:58.831 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:02:59.095 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:02:59.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.568+0000 7f47ee75a640 1 -- 172.21.15.103:0/3902075056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e8101dd0 msgr2=0x7f47e81021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.568+0000 7f47ee75a640 1 --2- 172.21.15.103:0/3902075056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e8101dd0 0x7f47e81021b0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009900 tx=0x7f47d802a3f0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.568+0000 7f47ee75a640 1 -- 172.21.15.103:0/3902075056 shutdown_connections 2024-09-19T07:02:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.568+0000 7f47ee75a640 1 --2- 172.21.15.103:0/3902075056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e8106f00 0x7f47e810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.568+0000 7f47ee75a640 1 --2- 172.21.15.103:0/3902075056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47e8102780 0x7f47e81067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.568+0000 7f47ee75a640 1 --2- 172.21.15.103:0/3902075056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e8101dd0 0x7f47e81021b0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.568+0000 7f47ee75a640 1 -- 172.21.15.103:0/3902075056 >> 172.21.15.103:0/3902075056 conn(0x7f47e80fd8b0 msgr2=0x7f47e80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.568+0000 7f47ee75a640 1 -- 172.21.15.103:0/3902075056 shutdown_connections 2024-09-19T07:02:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.568+0000 7f47ee75a640 1 -- 172.21.15.103:0/3902075056 wait complete. 2024-09-19T07:02:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47ee75a640 1 Processor -- start 2024-09-19T07:02:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47ee75a640 1 -- start start 2024-09-19T07:02:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47ee75a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e8101dd0 0x7f47e80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47ee75a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47e8102780 0x7f47e8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47ee75a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e8106f00 0x7f47e8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47ee75a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8111f20 con 0x7f47e8101dd0 2024-09-19T07:02:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47ee75a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8111da0 con 0x7f47e8102780 2024-09-19T07:02:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47ee75a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81120a0 con 0x7f47e8106f00 2024-09-19T07:02:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e8101dd0 0x7f47e80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e8101dd0 0x7f47e80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34886/0 (socket says 172.21.15.103:34886) 2024-09-19T07:02:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47e7fff640 1 -- 172.21.15.103:0/3497918154 learned_addr learned my addr 172.21.15.103:0/3497918154 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47eccf7640 1 --2- 172.21.15.103:0/3497918154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e8106f00 0x7f47e8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47dffff640 1 --2- 172.21.15.103:0/3497918154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47e8102780 0x7f47e8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47e7fff640 1 -- 172.21.15.103:0/3497918154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e8106f00 msgr2=0x7f47e8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47e7fff640 1 --2- 172.21.15.103:0/3497918154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e8106f00 0x7f47e8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47e7fff640 1 -- 172.21.15.103:0/3497918154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47e8102780 msgr2=0x7f47e8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47e7fff640 1 --2- 172.21.15.103:0/3497918154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47e8102780 0x7f47e8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.569+0000 7f47e7fff640 1 -- 172.21.15.103:0/3497918154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e81a6800 con 0x7f47e8101dd0 2024-09-19T07:02:59.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.570+0000 7f47e7fff640 1 --2- 172.21.15.103:0/3497918154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e8101dd0 0x7f47e80744f0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f47d8005ab0 tx=0x7f47d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.570+0000 7f47e5ffb640 1 -- 172.21.15.103:0/3497918154 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d803f030 con 0x7f47e8101dd0 2024-09-19T07:02:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.570+0000 7f47ee75a640 1 -- 172.21.15.103:0/3497918154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e81a6a90 con 0x7f47e8101dd0 2024-09-19T07:02:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.570+0000 7f47e5ffb640 1 -- 172.21.15.103:0/3497918154 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d8003ce0 con 0x7f47e8101dd0 2024-09-19T07:02:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.570+0000 7f47e5ffb640 1 -- 172.21.15.103:0/3497918154 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8035070 con 0x7f47e8101dd0 2024-09-19T07:02:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.570+0000 7f47ee75a640 1 -- 172.21.15.103:0/3497918154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e81a6fc0 con 0x7f47e8101dd0 2024-09-19T07:02:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.572+0000 7f47e5ffb640 1 -- 172.21.15.103:0/3497918154 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f47d8042030 con 0x7f47e8101dd0 2024-09-19T07:02:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47f0b33740 1 Processor -- start 2024-09-19T07:02:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47f0b33740 1 -- start start 2024-09-19T07:02:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47f0b33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea24a0 0x55730aea2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47f0b33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aea2dc0 0x55730aead140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47f0b33740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aead680 0x55730aeafa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47f0b33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55730ae082e0 con 0x55730aea24a0 2024-09-19T07:02:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47f0b33740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55730adafce0 con 0x55730aead680 2024-09-19T07:02:59.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47f0b33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55730ae042a0 con 0x55730aea2dc0 2024-09-19T07:02:59.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47eccf7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea24a0 0x55730aea2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47eccf7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea24a0 0x55730aea2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34892/0 (socket says 172.21.15.103:34892) 2024-09-19T07:02:59.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47eccf7640 1 -- 172.21.15.103:0/3840329885 learned_addr learned my addr 172.21.15.103:0/3840329885 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:59.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47dffff640 1 --2- 172.21.15.103:0/3840329885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aead680 0x55730aeafa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47e7fff640 1 --2- 172.21.15.103:0/3840329885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aea2dc0 0x55730aead140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47e5ffb640 1 --2- 172.21.15.103:0/3497918154 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47d40a33b0 0x7f47d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47eccf7640 1 -- 172.21.15.103:0/3840329885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aea2dc0 msgr2=0x55730aead140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:59.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47eccf7640 1 --2- 172.21.15.103:0/3840329885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aea2dc0 0x55730aead140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47eccf7640 1 -- 172.21.15.103:0/3840329885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aead680 msgr2=0x55730aeafa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:59.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47eccf7640 1 --2- 172.21.15.103:0/3840329885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aead680 0x55730aeafa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.573+0000 7f47eccf7640 1 -- 172.21.15.103:0/3840329885 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55730ad94440 con 0x55730aea24a0 2024-09-19T07:02:59.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.574+0000 7f47e5ffb640 1 -- 172.21.15.103:0/3497918154 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f47d80f0630 con 0x7f47e8101dd0 2024-09-19T07:02:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.574+0000 7f47eccf7640 1 --2- 172.21.15.103:0/3497918154 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47d40a33b0 0x7f47d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.574+0000 7f47eccf7640 1 --2- 172.21.15.103:0/3840329885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea24a0 0x55730aea2880 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f47e0007ec0 tx=0x7f47e000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6c68008e8ef52d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.574+0000 7f47de7fc640 1 -- 172.21.15.103:0/3840329885 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0010ac0 con 0x55730aea24a0 2024-09-19T07:02:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.574+0000 7f47de7fc640 1 -- 172.21.15.103:0/3840329885 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e001ae00 con 0x55730aea24a0 2024-09-19T07:02:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.574+0000 7f47de7fc640 1 -- 172.21.15.103:0/3840329885 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0019c60 con 0x55730aea24a0 2024-09-19T07:02:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.575+0000 7f47f0b33740 1 -- 172.21.15.103:0/3840329885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea24a0 msgr2=0x55730aea2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.575+0000 7f47eccf7640 1 --2- 172.21.15.103:0/3497918154 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47d40a33b0 0x7f47d40a5870 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f47e001e000 tx=0x7f47e001f000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.575+0000 7f47f0b33740 1 --2- 172.21.15.103:0/3840329885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea24a0 0x55730aea2880 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f47e0007ec0 tx=0x7f47e000f790 comp rx=0 tx=0).stop 2024-09-19T07:02:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.575+0000 7f47f0b33740 1 -- 172.21.15.103:0/3840329885 shutdown_connections 2024-09-19T07:02:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.575+0000 7f47f0b33740 1 --2- 172.21.15.103:0/3840329885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aead680 0x55730aeafa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.575+0000 7f47f0b33740 1 --2- 172.21.15.103:0/3840329885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aea2dc0 0x55730aead140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.575+0000 7f47f0b33740 1 --2- 172.21.15.103:0/3840329885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea24a0 0x55730aea2880 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.575+0000 7f47f0b33740 1 -- 172.21.15.103:0/3840329885 >> 172.21.15.103:0/3840329885 conn(0x55730ae9ea30 msgr2=0x55730ae9ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.575+0000 7f47f0b33740 1 -- 172.21.15.103:0/3840329885 shutdown_connections 2024-09-19T07:02:59.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.575+0000 7f47f0b33740 1 -- 172.21.15.103:0/3840329885 wait complete. 2024-09-19T07:02:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.575+0000 7f47f0b33740 1 -- start start 2024-09-19T07:02:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47f0b33740 1 Processor -- start 2024-09-19T07:02:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47f0b33740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aea24a0 0x55730aeacf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47f0b33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea2dc0 0x55730aeaa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47f0b33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aead680 0x55730aeaa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47f0b33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55730aeb0250 con 0x55730aea2dc0 2024-09-19T07:02:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47f0b33740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55730adc3050 con 0x55730aea24a0 2024-09-19T07:02:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47f0b33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55730aeb03d0 con 0x55730aead680 2024-09-19T07:02:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47f0b33740 10 client.0 _ll_register_callbacks cb 0x55730ae03160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47dffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea2dc0 0x55730aeaa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47dffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea2dc0 0x55730aeaa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34898/0 (socket says 172.21.15.103:34898) 2024-09-19T07:02:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47dffff640 1 -- 172.21.15.103:0/146498364 learned_addr learned my addr 172.21.15.103:0/146498364 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47e7fff640 1 --2- 172.21.15.103:0/146498364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aea24a0 0x55730aeacf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47eccf7640 1 --2- 172.21.15.103:0/146498364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aead680 0x55730aeaa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47dffff640 1 -- 172.21.15.103:0/146498364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aead680 msgr2=0x55730aeaa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47dffff640 1 --2- 172.21.15.103:0/146498364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aead680 0x55730aeaa720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47dffff640 1 -- 172.21.15.103:0/146498364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aea24a0 msgr2=0x55730aeacf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47dffff640 1 --2- 172.21.15.103:0/146498364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aea24a0 0x55730aeacf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.576+0000 7f47dffff640 1 -- 172.21.15.103:0/146498364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55730ab793c0 con 0x55730aea2dc0 2024-09-19T07:02:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.577+0000 7f47dffff640 1 --2- 172.21.15.103:0/146498364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea2dc0 0x55730aeaa1e0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f47d0004650 tx=0x7f47d00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d93996a342c48b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.577+0000 7f47c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.577+0000 7f47c77fe640 1 -- 172.21.15.103:0/146498364 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d0011910 con 0x55730aea2dc0 2024-09-19T07:02:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.577+0000 7f47c77fe640 1 -- 172.21.15.103:0/146498364 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f47d0011ab0 con 0x55730aea2dc0 2024-09-19T07:02:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.577+0000 7f47f0b33740 1 -- 172.21.15.103:0/146498364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e001ae00 con 0x55730aea2dc0 2024-09-19T07:02:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.577+0000 7f47c77fe640 1 -- 172.21.15.103:0/146498364 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f47d0011cb0 con 0x55730aea2dc0 2024-09-19T07:02:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.577+0000 7f47f0b33740 10 client.20142 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.577+0000 7f47f0b33740 1 -- 172.21.15.103:0/146498364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55730aebbbd0 con 0x55730aea2dc0 2024-09-19T07:02:59.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.577+0000 7f47c77fe640 1 -- 172.21.15.103:0/146498364 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e001ae00 con 0x55730aea2dc0 2024-09-19T07:02:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.577+0000 7f47c77fe640 1 -- 172.21.15.103:0/146498364 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f47d0018070 con 0x55730aea2dc0 2024-09-19T07:02:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.577+0000 7f47f0b33740 10 client.20142 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.577+0000 7f47f0b33740 10 client.20142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.577+0000 7f47f0b33740 20 client.20142 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.578+0000 7f47f0b33740 1 --2- 172.21.15.103:0/146498364 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55730aebd130 0x55730aebf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.578+0000 7f47f0b33740 4 client.20142 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.578+0000 7f47f0b33740 1 -- 172.21.15.103:0/146498364 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55730ae15850 con 0x55730aebd130 2024-09-19T07:02:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.578+0000 7f47e7fff640 1 --2- 172.21.15.103:0/146498364 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55730aebd130 0x55730aebf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.578+0000 7f47e7fff640 1 --2- 172.21.15.103:0/146498364 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55730aebd130 0x55730aebf5d0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac0f96c036a8912e server_cookie=1ac44ee3b11dd704 in_seq=0 out_seq=0 2024-09-19T07:02:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.578+0000 7f47c77fe640 10 client.20142 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.579+0000 7f47c77fe640 1 -- 172.21.15.103:0/146498364 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55730ae15850 con 0x55730aebd130 2024-09-19T07:02:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.579+0000 7f47c77fe640 10 client.20142 handle_command_reply: tid=0 2024-09-19T07:02:59.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.579+0000 7f47f0b33740 1 client.20142 shutdown 2024-09-19T07:02:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.579+0000 7f47f0b33740 1 -- 172.21.15.103:0/146498364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea2dc0 msgr2=0x55730aeaa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.579+0000 7f47f0b33740 1 --2- 172.21.15.103:0/146498364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea2dc0 0x55730aeaa1e0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f47d0004650 tx=0x7f47d00039c0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.579+0000 7f47f0b33740 1 -- 172.21.15.103:0/146498364 shutdown_connections 2024-09-19T07:02:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.579+0000 7f47f0b33740 1 --2- 172.21.15.103:0/146498364 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55730aebd130 0x55730aebf5d0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.579+0000 7f47f0b33740 1 --2- 172.21.15.103:0/146498364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aead680 0x55730aeaa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.579+0000 7f47c77fe640 0 client.20142 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.579+0000 7f47f0b33740 1 --2- 172.21.15.103:0/146498364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea2dc0 0x55730aeaa1e0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.579+0000 7f47f0b33740 1 --2- 172.21.15.103:0/146498364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aea24a0 0x55730aeacf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.579+0000 7f47f0b33740 1 -- 172.21.15.103:0/146498364 >> 172.21.15.103:0/146498364 conn(0x55730ae9ea30 msgr2=0x55730aea4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.579+0000 7f47f0b33740 1 -- 172.21.15.103:0/146498364 shutdown_connections 2024-09-19T07:02:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.579+0000 7f47f0b33740 1 -- 172.21.15.103:0/146498364 wait complete. 2024-09-19T07:02:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.579+0000 7f47f0b33740 20 client.20142 trim_cache size 0 max 16384 2024-09-19T07:02:59.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.596+0000 7f47f0b33740 1 Processor -- start 2024-09-19T07:02:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.596+0000 7f47f0b33740 1 -- start start 2024-09-19T07:02:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.596+0000 7f47f0b33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea24a0 0x55730aeb94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.596+0000 7f47f0b33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aea2dc0 0x55730aeac150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.596+0000 7f47f0b33740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aead680 0x55730aec4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.596+0000 7f47f0b33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81af970 con 0x55730aea24a0 2024-09-19T07:02:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.596+0000 7f47f0b33740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81b14b0 con 0x55730aead680 2024-09-19T07:02:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.596+0000 7f47f0b33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81b0040 con 0x55730aea2dc0 2024-09-19T07:02:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.597+0000 7f47e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea24a0 0x55730aeb94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.597+0000 7f47e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea24a0 0x55730aeb94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34902/0 (socket says 172.21.15.103:34902) 2024-09-19T07:02:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.597+0000 7f47e7fff640 1 -- 172.21.15.103:0/4241735560 learned_addr learned my addr 172.21.15.103:0/4241735560 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.597+0000 7f47dffff640 1 --2- 172.21.15.103:0/4241735560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aea2dc0 0x55730aeac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.597+0000 7f47e7fff640 1 -- 172.21.15.103:0/4241735560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aea2dc0 msgr2=0x55730aeac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.597+0000 7f47e7fff640 1 --2- 172.21.15.103:0/4241735560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aea2dc0 0x55730aeac150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.597+0000 7f47eccf7640 1 --2- 172.21.15.103:0/4241735560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aead680 0x55730aec4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.597+0000 7f47e7fff640 1 -- 172.21.15.103:0/4241735560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aead680 msgr2=0x55730aec4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.597+0000 7f47e7fff640 1 --2- 172.21.15.103:0/4241735560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aead680 0x55730aec4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.597+0000 7f47e7fff640 1 -- 172.21.15.103:0/4241735560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55730ae9e4d0 con 0x55730aea24a0 2024-09-19T07:02:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.597+0000 7f47eccf7640 1 --2- 172.21.15.103:0/4241735560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aead680 0x55730aec4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.597+0000 7f47e7fff640 1 --2- 172.21.15.103:0/4241735560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea24a0 0x55730aeb94b0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f47d80b92c0 tx=0x7f47d8009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d550ed0612fe6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.597+0000 7f47ddffb640 1 -- 172.21.15.103:0/4241735560 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8049040 con 0x55730aea24a0 2024-09-19T07:02:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.597+0000 7f47ddffb640 1 -- 172.21.15.103:0/4241735560 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d80c0ca0 con 0x55730aea24a0 2024-09-19T07:02:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.597+0000 7f47ddffb640 1 -- 172.21.15.103:0/4241735560 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d80b8f60 con 0x55730aea24a0 2024-09-19T07:02:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.598+0000 7f47f0b33740 1 -- 172.21.15.103:0/4241735560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea24a0 msgr2=0x55730aeb94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.598+0000 7f47f0b33740 1 --2- 172.21.15.103:0/4241735560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea24a0 0x55730aeb94b0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f47d80b92c0 tx=0x7f47d8009680 comp rx=0 tx=0).stop 2024-09-19T07:02:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.598+0000 7f47f0b33740 1 -- 172.21.15.103:0/4241735560 shutdown_connections 2024-09-19T07:02:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.598+0000 7f47f0b33740 1 --2- 172.21.15.103:0/4241735560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aead680 0x55730aec4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.598+0000 7f47f0b33740 1 --2- 172.21.15.103:0/4241735560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aea2dc0 0x55730aeac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.598+0000 7f47f0b33740 1 --2- 172.21.15.103:0/4241735560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aea24a0 0x55730aeb94b0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.598+0000 7f47f0b33740 1 -- 172.21.15.103:0/4241735560 >> 172.21.15.103:0/4241735560 conn(0x55730ae9ea30 msgr2=0x55730aeaba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.598+0000 7f47f0b33740 1 -- 172.21.15.103:0/4241735560 shutdown_connections 2024-09-19T07:02:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.598+0000 7f47f0b33740 1 -- 172.21.15.103:0/4241735560 wait complete. 2024-09-19T07:02:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.598+0000 7f47f0b33740 1 -- start start 2024-09-19T07:02:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47f0b33740 1 Processor -- start 2024-09-19T07:02:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47f0b33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aea24a0 0x55730aeb19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47f0b33740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aea2dc0 0x55730aeb1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47f0b33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aead680 0x55730af26e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47f0b33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81b0de0 con 0x55730aead680 2024-09-19T07:02:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47f0b33740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81b0710 con 0x55730aea2dc0 2024-09-19T07:02:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47f0b33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81ad970 con 0x55730aea24a0 2024-09-19T07:02:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47f0b33740 10 client.0 _ll_register_callbacks cb 0x7f47d0004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:59.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47eccf7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aead680 0x55730af26e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47eccf7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aead680 0x55730af26e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34904/0 (socket says 172.21.15.103:34904) 2024-09-19T07:02:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47eccf7640 1 -- 172.21.15.103:0/3759867046 learned_addr learned my addr 172.21.15.103:0/3759867046 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47dffff640 1 --2- 172.21.15.103:0/3759867046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aea2dc0 0x55730aeb1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47e7fff640 1 --2- 172.21.15.103:0/3759867046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aea24a0 0x55730aeb19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47eccf7640 1 -- 172.21.15.103:0/3759867046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aea24a0 msgr2=0x55730aeb19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47eccf7640 1 --2- 172.21.15.103:0/3759867046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aea24a0 0x55730aeb19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47eccf7640 1 -- 172.21.15.103:0/3759867046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aea2dc0 msgr2=0x55730aeb1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47eccf7640 1 --2- 172.21.15.103:0/3759867046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aea2dc0 0x55730aeb1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47eccf7640 1 -- 172.21.15.103:0/3759867046 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55730af273b0 con 0x55730aead680 2024-09-19T07:02:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47dffff640 1 --2- 172.21.15.103:0/3759867046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aea2dc0 0x55730aeb1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47eccf7640 1 --2- 172.21.15.103:0/3759867046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aead680 0x55730af26e70 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f47e000fc90 tx=0x7f47e0019c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03c8827eb2e9587 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.599+0000 7f47c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.600+0000 7f47c77fe640 1 -- 172.21.15.103:0/3759867046 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0035020 con 0x55730aead680 2024-09-19T07:02:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.600+0000 7f47c77fe640 1 -- 172.21.15.103:0/3759867046 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f47e00078a0 con 0x55730aead680 2024-09-19T07:02:59.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.600+0000 7f47f0b33740 1 -- 172.21.15.103:0/3759867046 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47d80c0ca0 con 0x55730aead680 2024-09-19T07:02:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.600+0000 7f47c77fe640 1 -- 172.21.15.103:0/3759867046 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f47e0040050 con 0x55730aead680 2024-09-19T07:02:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.600+0000 7f47f0b33740 10 client.20148 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.600+0000 7f47f0b33740 1 -- 172.21.15.103:0/3759867046 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55730af28530 con 0x55730aead680 2024-09-19T07:02:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.600+0000 7f47c77fe640 1 -- 172.21.15.103:0/3759867046 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d80c0ca0 con 0x55730aead680 2024-09-19T07:02:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.600+0000 7f47c77fe640 1 -- 172.21.15.103:0/3759867046 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f47e001a8a0 con 0x55730aead680 2024-09-19T07:02:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.600+0000 7f47f0b33740 10 client.20148 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.600+0000 7f47f0b33740 10 client.20148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.600+0000 7f47f0b33740 20 client.20148 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.600+0000 7f47f0b33740 1 --2- 172.21.15.103:0/3759867046 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55730af28a20 0x55730af2ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.600+0000 7f47f0b33740 4 client.20148 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:02:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.600+0000 7f47f0b33740 1 -- 172.21.15.103:0/3759867046 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55730ae0fd60 con 0x55730af28a20 2024-09-19T07:02:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.600+0000 7f47e7fff640 1 --2- 172.21.15.103:0/3759867046 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55730af28a20 0x55730af2ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47c77fe640 10 client.20148 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:59.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47e7fff640 1 --2- 172.21.15.103:0/3759867046 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55730af28a20 0x55730af2ae40 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdafcfb1ec1d82ed server_cookie=fef989a26593c826 in_seq=0 out_seq=0 2024-09-19T07:02:59.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47c77fe640 1 -- 172.21.15.103:0/3759867046 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55730ae0fd60 con 0x55730af28a20 2024-09-19T07:02:59.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47c77fe640 10 client.20148 handle_command_reply: tid=0 2024-09-19T07:02:59.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47f0b33740 1 client.20148 shutdown 2024-09-19T07:02:59.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47f0b33740 1 -- 172.21.15.103:0/3759867046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aead680 msgr2=0x55730af26e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47f0b33740 1 --2- 172.21.15.103:0/3759867046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aead680 0x55730af26e70 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f47e000fc90 tx=0x7f47e0019c60 comp rx=0 tx=0).stop 2024-09-19T07:02:59.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47f0b33740 1 -- 172.21.15.103:0/3759867046 shutdown_connections 2024-09-19T07:02:59.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47f0b33740 1 --2- 172.21.15.103:0/3759867046 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55730af28a20 0x55730af2ae40 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47f0b33740 1 --2- 172.21.15.103:0/3759867046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55730aead680 0x55730af26e70 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47f0b33740 1 --2- 172.21.15.103:0/3759867046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55730aea2dc0 0x55730aeb1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47f0b33740 1 --2- 172.21.15.103:0/3759867046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55730aea24a0 0x55730aeb19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47f0b33740 1 -- 172.21.15.103:0/3759867046 >> 172.21.15.103:0/3759867046 conn(0x55730ae9ea30 msgr2=0x55730aeac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:59.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47c77fe640 0 client.20148 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:59.621 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:02:59.621 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:02:59.621 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:02:59.621 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:02:59.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47f0b33740 1 -- 172.21.15.103:0/3759867046 shutdown_connections 2024-09-19T07:02:59.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47f0b33740 1 -- 172.21.15.103:0/3759867046 wait complete. 2024-09-19T07:02:59.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.601+0000 7f47f0b33740 20 client.20148 trim_cache size 0 max 16384 2024-09-19T07:02:59.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.602+0000 7f47ee75a640 1 -- 172.21.15.103:0/3497918154 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47d40a33b0 msgr2=0x7f47d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.602+0000 7f47ee75a640 1 --2- 172.21.15.103:0/3497918154 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47d40a33b0 0x7f47d40a5870 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f47e001e000 tx=0x7f47e001f000 comp rx=0 tx=0).stop 2024-09-19T07:02:59.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.602+0000 7f47ee75a640 1 -- 172.21.15.103:0/3497918154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e8101dd0 msgr2=0x7f47e80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.602+0000 7f47ee75a640 1 --2- 172.21.15.103:0/3497918154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e8101dd0 0x7f47e80744f0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f47d8005ab0 tx=0x7f47d80042f0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.602+0000 7f47ee75a640 1 -- 172.21.15.103:0/3497918154 shutdown_connections 2024-09-19T07:02:59.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.602+0000 7f47ee75a640 1 --2- 172.21.15.103:0/3497918154 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f47d40a33b0 0x7f47d40a5870 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.602+0000 7f47ee75a640 1 --2- 172.21.15.103:0/3497918154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e8106f00 0x7f47e8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.602+0000 7f47ee75a640 1 --2- 172.21.15.103:0/3497918154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47e8102780 0x7f47e8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.602+0000 7f47ee75a640 1 --2- 172.21.15.103:0/3497918154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47e8101dd0 0x7f47e80744f0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.602+0000 7f47ee75a640 1 -- 172.21.15.103:0/3497918154 >> 172.21.15.103:0/3497918154 conn(0x7f47e80fd8b0 msgr2=0x7f47e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:59.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.602+0000 7f47ee75a640 1 -- 172.21.15.103:0/3497918154 shutdown_connections 2024-09-19T07:02:59.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:02:59.602+0000 7f47ee75a640 1 -- 172.21.15.103:0/3497918154 wait complete. 2024-09-19T07:02:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.820+0000 7f1b7da1e640 1 -- 172.21.15.103:0/1532142757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b78101ad0 msgr2=0x7f1b78101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.820+0000 7f1b7da1e640 1 --2- 172.21.15.103:0/1532142757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b78101ad0 0x7f1b78101f30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009900 tx=0x7f1b6c02a410 comp rx=0 tx=0).stop 2024-09-19T07:02:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.820+0000 7f1b7da1e640 1 -- 172.21.15.103:0/1532142757 shutdown_connections 2024-09-19T07:02:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.820+0000 7f1b7da1e640 1 --2- 172.21.15.103:0/1532142757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b78102470 0x7f1b7810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.820+0000 7f1b7da1e640 1 --2- 172.21.15.103:0/1532142757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b78101ad0 0x7f1b78101f30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.820+0000 7f1b7da1e640 1 --2- 172.21.15.103:0/1532142757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b78107b70 0x7f1b78107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.820+0000 7f1b7da1e640 1 -- 172.21.15.103:0/1532142757 >> 172.21.15.103:0/1532142757 conn(0x7f1b780fd7d0 msgr2=0x7f1b780ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.820+0000 7f1b7da1e640 1 -- 172.21.15.103:0/1532142757 shutdown_connections 2024-09-19T07:02:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.820+0000 7f1b7da1e640 1 -- 172.21.15.103:0/1532142757 wait complete. 2024-09-19T07:02:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.821+0000 7f1b7da1e640 1 Processor -- start 2024-09-19T07:02:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.821+0000 7f1b7da1e640 1 -- start start 2024-09-19T07:02:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.821+0000 7f1b7da1e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b78101ad0 0x7f1b781982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.821+0000 7f1b7da1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b78102470 0x7f1b78198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.821+0000 7f1b7da1e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b78107b70 0x7f1b7819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.821+0000 7f1b7da1e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b78111f20 con 0x7f1b78102470 2024-09-19T07:02:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.821+0000 7f1b7da1e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1b78111da0 con 0x7f1b78107b70 2024-09-19T07:02:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.821+0000 7f1b7da1e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781120a0 con 0x7f1b78101ad0 2024-09-19T07:02:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.821+0000 7f1b767fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b78102470 0x7f1b78198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.821+0000 7f1b767fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b78102470 0x7f1b78198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34914/0 (socket says 172.21.15.103:34914) 2024-09-19T07:02:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.821+0000 7f1b767fc640 1 -- 172.21.15.103:0/3650947826 learned_addr learned my addr 172.21.15.103:0/3650947826 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.821+0000 7f1b777fe640 1 --2- 172.21.15.103:0/3650947826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b78107b70 0x7f1b7819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.821+0000 7f1b76ffd640 1 --2- 172.21.15.103:0/3650947826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b78101ad0 0x7f1b781982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.822+0000 7f1b767fc640 1 -- 172.21.15.103:0/3650947826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b78101ad0 msgr2=0x7f1b781982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.822+0000 7f1b767fc640 1 --2- 172.21.15.103:0/3650947826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b78101ad0 0x7f1b781982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.822+0000 7f1b767fc640 1 -- 172.21.15.103:0/3650947826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b78107b70 msgr2=0x7f1b7819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.822+0000 7f1b767fc640 1 --2- 172.21.15.103:0/3650947826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b78107b70 0x7f1b7819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.822+0000 7f1b767fc640 1 -- 172.21.15.103:0/3650947826 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7819d2c0 con 0x7f1b78102470 2024-09-19T07:02:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.822+0000 7f1b767fc640 1 --2- 172.21.15.103:0/3650947826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b78102470 0x7f1b78198810 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c006010 tx=0x7f1b6c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.822+0000 7f1b7ca1c640 1 -- 172.21.15.103:0/3650947826 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c03f070 con 0x7f1b78102470 2024-09-19T07:02:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.822+0000 7f1b7da1e640 1 -- 172.21.15.103:0/3650947826 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7819d550 con 0x7f1b78102470 2024-09-19T07:02:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.822+0000 7f1b7ca1c640 1 -- 172.21.15.103:0/3650947826 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b6c031c30 con 0x7f1b78102470 2024-09-19T07:02:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.822+0000 7f1b7ca1c640 1 -- 172.21.15.103:0/3650947826 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c035030 con 0x7f1b78102470 2024-09-19T07:02:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.822+0000 7f1b7da1e640 1 -- 172.21.15.103:0/3650947826 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b781a4e40 con 0x7f1b78102470 2024-09-19T07:02:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.825+0000 7f1b7ca1c640 1 -- 172.21.15.103:0/3650947826 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1b6c042070 con 0x7f1b78102470 2024-09-19T07:02:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.825+0000 7f1b80890740 1 Processor -- start 2024-09-19T07:02:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.825+0000 7f1b80890740 1 -- start start 2024-09-19T07:02:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.825+0000 7f1b80890740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81bd990 0x55b9b81bdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.825+0000 7f1b80890740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81be2b0 0x55b9b81ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.825+0000 7f1b80890740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81cd0c0 0x55b9b81cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.825+0000 7f1b80890740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b9b81222f0 con 0x55b9b81cd0c0 2024-09-19T07:02:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.825+0000 7f1b80890740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b9b80ca270 con 0x55b9b81be2b0 2024-09-19T07:02:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.825+0000 7f1b80890740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b9b811e2b0 con 0x55b9b81bd990 2024-09-19T07:02:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.825+0000 7f1b7ca1c640 1 --2- 172.21.15.103:0/3650947826 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1b600a3460 0x7f1b600a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.826+0000 7f1b767fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81cd0c0 0x55b9b81cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.826+0000 7f1b76ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81be2b0 0x55b9b81ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.826+0000 7f1b76ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81be2b0 0x55b9b81ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:38274/0 (socket says 172.21.15.103:38274) 2024-09-19T07:02:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.826+0000 7f1b76ffd640 1 -- 172.21.15.103:0/1097425887 learned_addr learned my addr 172.21.15.103:0/1097425887 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.826+0000 7f1b767fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81cd0c0 0x55b9b81cf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34930/0 (socket says 172.21.15.103:34930) 2024-09-19T07:02:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.826+0000 7f1b777fe640 1 --2- 172.21.15.103:0/3650947826 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1b600a3460 0x7f1b600a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.826+0000 7f1b777fe640 1 --2- 172.21.15.103:0/1097425887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81bd990 0x55b9b81bdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.826+0000 7f1b76ffd640 1 -- 172.21.15.103:0/1097425887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81bd990 msgr2=0x55b9b81bdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.826+0000 7f1b76ffd640 1 --2- 172.21.15.103:0/1097425887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81bd990 0x55b9b81bdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.826+0000 7f1b76ffd640 1 -- 172.21.15.103:0/1097425887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81cd0c0 msgr2=0x55b9b81cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.826+0000 7f1b76ffd640 1 --2- 172.21.15.103:0/1097425887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81cd0c0 0x55b9b81cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.826+0000 7f1b76ffd640 1 -- 172.21.15.103:0/1097425887 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9b80c9340 con 0x55b9b81be2b0 2024-09-19T07:02:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.826+0000 7f1b777fe640 1 --2- 172.21.15.103:0/1097425887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81bd990 0x55b9b81bdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.826+0000 7f1b76ffd640 1 --2- 172.21.15.103:0/1097425887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81be2b0 0x55b9b81ccb80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009c70 tx=0x7f1b68011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=753dfd0c9c6a4b9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.826+0000 7f1b7ca1c640 1 -- 172.21.15.103:0/3650947826 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f1b6c0f0040 con 0x7f1b78102470 2024-09-19T07:02:59.830 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:02:59 smithi103 ceph-mon[31158]: pgmap v429: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-09-19T07:02:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.827+0000 7f1b777fe640 1 --2- 172.21.15.103:0/3650947826 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1b600a3460 0x7f1b600a5920 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1b70005fd0 tx=0x7f1b70004a40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.829+0000 7f1b5dffb640 1 -- 172.21.15.103:0/1097425887 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68004230 con 0x55b9b81be2b0 2024-09-19T07:02:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.829+0000 7f1b5dffb640 1 -- 172.21.15.103:0/1097425887 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b680043d0 con 0x55b9b81be2b0 2024-09-19T07:02:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.829+0000 7f1b5dffb640 1 -- 172.21.15.103:0/1097425887 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68019790 con 0x55b9b81be2b0 2024-09-19T07:02:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.830+0000 7f1b80890740 1 -- 172.21.15.103:0/1097425887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81be2b0 msgr2=0x55b9b81ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.830+0000 7f1b80890740 1 --2- 172.21.15.103:0/1097425887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81be2b0 0x55b9b81ccb80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009c70 tx=0x7f1b68011810 comp rx=0 tx=0).stop 2024-09-19T07:02:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.830+0000 7f1b80890740 1 -- 172.21.15.103:0/1097425887 shutdown_connections 2024-09-19T07:02:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.830+0000 7f1b80890740 1 --2- 172.21.15.103:0/1097425887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81cd0c0 0x55b9b81cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.830+0000 7f1b80890740 1 --2- 172.21.15.103:0/1097425887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81be2b0 0x55b9b81ccb80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.830+0000 7f1b80890740 1 --2- 172.21.15.103:0/1097425887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81bd990 0x55b9b81bdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.830+0000 7f1b80890740 1 -- 172.21.15.103:0/1097425887 >> 172.21.15.103:0/1097425887 conn(0x55b9b81ba060 msgr2=0x55b9b81ba440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.830+0000 7f1b80890740 1 -- 172.21.15.103:0/1097425887 shutdown_connections 2024-09-19T07:02:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.830+0000 7f1b80890740 1 -- 172.21.15.103:0/1097425887 wait complete. 2024-09-19T07:02:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.831+0000 7f1b80890740 1 -- start start 2024-09-19T07:02:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.831+0000 7f1b80890740 1 Processor -- start 2024-09-19T07:02:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.831+0000 7f1b80890740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81bd990 0x55b9b81c6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.831+0000 7f1b80890740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81be2b0 0x55b9b81c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.831+0000 7f1b80890740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81cd0c0 0x55b9b81cab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.831+0000 7f1b80890740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b9b81c0950 con 0x55b9b81bd990 2024-09-19T07:02:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.831+0000 7f1b80890740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b9b80ab290 con 0x55b9b81cd0c0 2024-09-19T07:02:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.831+0000 7f1b80890740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b9b81c0ad0 con 0x55b9b81be2b0 2024-09-19T07:02:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.831+0000 7f1b80890740 10 client.0 _ll_register_callbacks cb 0x55b9b80dc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.831+0000 7f1b76ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81bd990 0x55b9b81c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.832+0000 7f1b76ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81bd990 0x55b9b81c6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34932/0 (socket says 172.21.15.103:34932) 2024-09-19T07:02:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.832+0000 7f1b76ffd640 1 -- 172.21.15.103:0/2954404723 learned_addr learned my addr 172.21.15.103:0/2954404723 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.832+0000 7f1b777fe640 1 --2- 172.21.15.103:0/2954404723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81cd0c0 0x55b9b81cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.832+0000 7f1b767fc640 1 --2- 172.21.15.103:0/2954404723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81be2b0 0x55b9b81c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.832+0000 7f1b76ffd640 1 -- 172.21.15.103:0/2954404723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81be2b0 msgr2=0x55b9b81c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.832+0000 7f1b76ffd640 1 --2- 172.21.15.103:0/2954404723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81be2b0 0x55b9b81c67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.832+0000 7f1b76ffd640 1 -- 172.21.15.103:0/2954404723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81cd0c0 msgr2=0x55b9b81cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.832+0000 7f1b76ffd640 1 --2- 172.21.15.103:0/2954404723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81cd0c0 0x55b9b81cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.832+0000 7f1b76ffd640 1 -- 172.21.15.103:0/2954404723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9b7e35210 con 0x55b9b81bd990 2024-09-19T07:02:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.832+0000 7f1b777fe640 1 --2- 172.21.15.103:0/2954404723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81cd0c0 0x55b9b81cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:02:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.832+0000 7f1b76ffd640 1 --2- 172.21.15.103:0/2954404723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81bd990 0x55b9b81c6280 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f1b68003f20 tx=0x7f1b68005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9690bf0d97c920a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.832+0000 7f1b56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.832+0000 7f1b56ffd640 1 -- 172.21.15.103:0/2954404723 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68027070 con 0x55b9b81bd990 2024-09-19T07:02:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.832+0000 7f1b56ffd640 1 -- 172.21.15.103:0/2954404723 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f1b680035f0 con 0x55b9b81bd990 2024-09-19T07:02:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.832+0000 7f1b80890740 1 -- 172.21.15.103:0/2954404723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b680043d0 con 0x55b9b81bd990 2024-09-19T07:02:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.833+0000 7f1b56ffd640 1 -- 172.21.15.103:0/2954404723 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b68031050 con 0x55b9b81bd990 2024-09-19T07:02:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.833+0000 7f1b80890740 10 client.20160 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.833+0000 7f1b80890740 1 -- 172.21.15.103:0/2954404723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b9b7f01620 con 0x55b9b81bd990 2024-09-19T07:02:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.833+0000 7f1b56ffd640 1 -- 172.21.15.103:0/2954404723 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b680043d0 con 0x55b9b81bd990 2024-09-19T07:02:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.833+0000 7f1b56ffd640 1 -- 172.21.15.103:0/2954404723 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1b68018070 con 0x55b9b81bd990 2024-09-19T07:02:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.833+0000 7f1b80890740 10 client.20160 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.833+0000 7f1b80890740 10 client.20160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.833+0000 7f1b80890740 20 client.20160 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.833+0000 7f1b80890740 1 --2- 172.21.15.103:0/2954404723 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b9b81d8b50 0x55b9b81daf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.833+0000 7f1b80890740 4 client.20160 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:02:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.833+0000 7f1b80890740 1 -- 172.21.15.103:0/2954404723 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9b812fa20 con 0x55b9b81d8b50 2024-09-19T07:02:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.833+0000 7f1b767fc640 1 --2- 172.21.15.103:0/2954404723 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b9b81d8b50 0x55b9b81daf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.834+0000 7f1b767fc640 1 --2- 172.21.15.103:0/2954404723 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b9b81d8b50 0x55b9b81daf40 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74cea0047f9e4edc server_cookie=31c397b8792dfb2d in_seq=0 out_seq=0 2024-09-19T07:02:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.834+0000 7f1b56ffd640 10 client.20160 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.834+0000 7f1b56ffd640 1 -- 172.21.15.103:0/2954404723 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9b812fa20 con 0x55b9b81d8b50 2024-09-19T07:02:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.834+0000 7f1b56ffd640 10 client.20160 handle_command_reply: tid=0 2024-09-19T07:02:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.834+0000 7f1b80890740 1 client.20160 shutdown 2024-09-19T07:02:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.835+0000 7f1b80890740 1 -- 172.21.15.103:0/2954404723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81bd990 msgr2=0x55b9b81c6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.835+0000 7f1b80890740 1 --2- 172.21.15.103:0/2954404723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81bd990 0x55b9b81c6280 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f1b68003f20 tx=0x7f1b68005dc0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.835+0000 7f1b80890740 1 -- 172.21.15.103:0/2954404723 shutdown_connections 2024-09-19T07:02:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.835+0000 7f1b80890740 1 --2- 172.21.15.103:0/2954404723 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b9b81d8b50 0x55b9b81daf40 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.835+0000 7f1b80890740 1 --2- 172.21.15.103:0/2954404723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81cd0c0 0x55b9b81cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.835+0000 7f1b56ffd640 0 client.20160 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.835+0000 7f1b80890740 1 --2- 172.21.15.103:0/2954404723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81be2b0 0x55b9b81c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.835+0000 7f1b80890740 1 --2- 172.21.15.103:0/2954404723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81bd990 0x55b9b81c6280 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.835+0000 7f1b80890740 1 -- 172.21.15.103:0/2954404723 >> 172.21.15.103:0/2954404723 conn(0x55b9b81ba060 msgr2=0x55b9b81b91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.835+0000 7f1b80890740 1 -- 172.21.15.103:0/2954404723 shutdown_connections 2024-09-19T07:02:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.835+0000 7f1b80890740 1 -- 172.21.15.103:0/2954404723 wait complete. 2024-09-19T07:02:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.835+0000 7f1b80890740 20 client.20160 trim_cache size 0 max 16384 2024-09-19T07:02:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.849+0000 7f1b80890740 1 Processor -- start 2024-09-19T07:02:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.849+0000 7f1b80890740 1 -- start start 2024-09-19T07:02:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.849+0000 7f1b80890740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81bd990 0x55b9b81cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.849+0000 7f1b80890740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81be2b0 0x55b9b81b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.849+0000 7f1b80890740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81cd0c0 0x55b9b81e0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.849+0000 7f1b80890740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781ab580 con 0x55b9b81be2b0 2024-09-19T07:02:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.849+0000 7f1b80890740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781ad0c0 con 0x55b9b81bd990 2024-09-19T07:02:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.849+0000 7f1b80890740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781abc50 con 0x55b9b81cd0c0 2024-09-19T07:02:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.849+0000 7f1b767fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81be2b0 0x55b9b81b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.849+0000 7f1b767fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81be2b0 0x55b9b81b90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34948/0 (socket says 172.21.15.103:34948) 2024-09-19T07:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.849+0000 7f1b767fc640 1 -- 172.21.15.103:0/178908940 learned_addr learned my addr 172.21.15.103:0/178908940 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.849+0000 7f1b767fc640 1 -- 172.21.15.103:0/178908940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81cd0c0 msgr2=0x55b9b81e0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.849+0000 7f1b767fc640 1 --2- 172.21.15.103:0/178908940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81cd0c0 0x55b9b81e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.849+0000 7f1b767fc640 1 -- 172.21.15.103:0/178908940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81bd990 msgr2=0x55b9b81cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.849+0000 7f1b76ffd640 1 --2- 172.21.15.103:0/178908940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81bd990 0x55b9b81cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.849+0000 7f1b767fc640 1 --2- 172.21.15.103:0/178908940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81bd990 0x55b9b81cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.850+0000 7f1b767fc640 1 -- 172.21.15.103:0/178908940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9b81c6b30 con 0x55b9b81be2b0 2024-09-19T07:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.850+0000 7f1b76ffd640 1 --2- 172.21.15.103:0/178908940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81bd990 0x55b9b81cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.850+0000 7f1b767fc640 1 --2- 172.21.15.103:0/178908940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81be2b0 0x55b9b81b90b0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c005ab0 tx=0x7f1b6c0f2550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13706fcf27b7946d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.850+0000 7f1b5d7fa640 1 -- 172.21.15.103:0/178908940 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c049040 con 0x55b9b81be2b0 2024-09-19T07:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.850+0000 7f1b5d7fa640 1 -- 172.21.15.103:0/178908940 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b6c0c4f70 con 0x55b9b81be2b0 2024-09-19T07:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.850+0000 7f1b5d7fa640 1 -- 172.21.15.103:0/178908940 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c0b8a20 con 0x55b9b81be2b0 2024-09-19T07:02:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 -- 172.21.15.103:0/178908940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81be2b0 msgr2=0x55b9b81b90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 --2- 172.21.15.103:0/178908940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81be2b0 0x55b9b81b90b0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c005ab0 tx=0x7f1b6c0f2550 comp rx=0 tx=0).stop 2024-09-19T07:02:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 -- 172.21.15.103:0/178908940 shutdown_connections 2024-09-19T07:02:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 --2- 172.21.15.103:0/178908940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81cd0c0 0x55b9b81e0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 --2- 172.21.15.103:0/178908940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81be2b0 0x55b9b81b90b0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 --2- 172.21.15.103:0/178908940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81bd990 0x55b9b81cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 -- 172.21.15.103:0/178908940 >> 172.21.15.103:0/178908940 conn(0x55b9b81ba060 msgr2=0x55b9b81b89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 -- 172.21.15.103:0/178908940 shutdown_connections 2024-09-19T07:02:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 -- 172.21.15.103:0/178908940 wait complete. 2024-09-19T07:02:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 -- start start 2024-09-19T07:02:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 Processor -- start 2024-09-19T07:02:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81bd990 0x55b9b81d50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81be2b0 0x55b9b81c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81cd0c0 0x55b9b81c33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781ac9f0 con 0x55b9b81be2b0 2024-09-19T07:02:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781ac320 con 0x55b9b81bd990 2024-09-19T07:02:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781a95a0 con 0x55b9b81cd0c0 2024-09-19T07:02:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.851+0000 7f1b80890740 10 client.0 _ll_register_callbacks cb 0x7f1b6c0bc180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:02:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b767fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81be2b0 0x55b9b81c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b767fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81be2b0 0x55b9b81c2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34958/0 (socket says 172.21.15.103:34958) 2024-09-19T07:02:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b767fc640 1 -- 172.21.15.103:0/1971802387 learned_addr learned my addr 172.21.15.103:0/1971802387 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:02:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b76ffd640 1 --2- 172.21.15.103:0/1971802387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81bd990 0x55b9b81d50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b777fe640 1 --2- 172.21.15.103:0/1971802387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81cd0c0 0x55b9b81c33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b767fc640 1 -- 172.21.15.103:0/1971802387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81cd0c0 msgr2=0x55b9b81c33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b767fc640 1 --2- 172.21.15.103:0/1971802387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81cd0c0 0x55b9b81c33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b767fc640 1 -- 172.21.15.103:0/1971802387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81bd990 msgr2=0x55b9b81d50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:02:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b767fc640 1 --2- 172.21.15.103:0/1971802387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81bd990 0x55b9b81d50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b767fc640 1 -- 172.21.15.103:0/1971802387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9b81c3ab0 con 0x55b9b81be2b0 2024-09-19T07:02:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b76ffd640 1 --2- 172.21.15.103:0/1971802387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81bd990 0x55b9b81d50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:02:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b767fc640 1 --2- 172.21.15.103:0/1971802387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81be2b0 0x55b9b81c2e80 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0edd70 tx=0x7f1b6c0c0830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444d0f2aad0a6e71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:02:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:02:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b56ffd640 1 -- 172.21.15.103:0/1971802387 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c0d5060 con 0x55b9b81be2b0 2024-09-19T07:02:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b56ffd640 1 -- 172.21.15.103:0/1971802387 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f1b6c0eda60 con 0x55b9b81be2b0 2024-09-19T07:02:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b80890740 1 -- 172.21.15.103:0/1971802387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b6c0c4f70 con 0x55b9b81be2b0 2024-09-19T07:02:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b56ffd640 1 -- 172.21.15.103:0/1971802387 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b6c049040 con 0x55b9b81be2b0 2024-09-19T07:02:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b80890740 10 client.20166 fetch_fsmap learned FSMap version 27 2024-09-19T07:02:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b80890740 1 -- 172.21.15.103:0/1971802387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b9b8243470 con 0x55b9b81be2b0 2024-09-19T07:02:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.852+0000 7f1b56ffd640 1 -- 172.21.15.103:0/1971802387 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b6c0cb070 con 0x55b9b81be2b0 2024-09-19T07:02:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.853+0000 7f1b56ffd640 1 -- 172.21.15.103:0/1971802387 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1b6c0c4b70 con 0x55b9b81be2b0 2024-09-19T07:02:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.853+0000 7f1b80890740 10 client.20166 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:02:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.853+0000 7f1b80890740 10 client.20166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:02:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.853+0000 7f1b80890740 20 client.20166 populate_metadata read hostname 'smithi103' 2024-09-19T07:02:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.853+0000 7f1b80890740 1 --2- 172.21.15.103:0/1971802387 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b9b8243b40 0x55b9b8245f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:02:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.853+0000 7f1b80890740 4 client.20166 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:02:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.853+0000 7f1b80890740 1 -- 172.21.15.103:0/1971802387 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9b81c1090 con 0x55b9b8243b40 2024-09-19T07:02:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.853+0000 7f1b76ffd640 1 --2- 172.21.15.103:0/1971802387 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b9b8243b40 0x55b9b8245f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:02:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.853+0000 7f1b76ffd640 1 --2- 172.21.15.103:0/1971802387 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b9b8243b40 0x55b9b8245f60 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf9107c616e9a492 server_cookie=83f0d8dae298d18a in_seq=0 out_seq=0 2024-09-19T07:02:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.853+0000 7f1b56ffd640 10 client.20166 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.863+0000 7f1b56ffd640 1 -- 172.21.15.103:0/1971802387 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b9b81c1090 con 0x55b9b8243b40 2024-09-19T07:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.863+0000 7f1b56ffd640 10 client.20166 handle_command_reply: tid=0 2024-09-19T07:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.863+0000 7f1b80890740 1 client.20166 shutdown 2024-09-19T07:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.863+0000 7f1b80890740 1 -- 172.21.15.103:0/1971802387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81be2b0 msgr2=0x55b9b81c2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.863+0000 7f1b80890740 1 --2- 172.21.15.103:0/1971802387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81be2b0 0x55b9b81c2e80 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0edd70 tx=0x7f1b6c0c0830 comp rx=0 tx=0).stop 2024-09-19T07:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.863+0000 7f1b80890740 1 -- 172.21.15.103:0/1971802387 shutdown_connections 2024-09-19T07:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.863+0000 7f1b80890740 1 --2- 172.21.15.103:0/1971802387 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b9b8243b40 0x55b9b8245f60 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.863+0000 7f1b80890740 1 --2- 172.21.15.103:0/1971802387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9b81cd0c0 0x55b9b81c33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.863+0000 7f1b80890740 1 --2- 172.21.15.103:0/1971802387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b9b81be2b0 0x55b9b81c2e80 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.863+0000 7f1b56ffd640 0 client.20166 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.863+0000 7f1b80890740 1 --2- 172.21.15.103:0/1971802387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9b81bd990 0x55b9b81d50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.863+0000 7f1b80890740 1 -- 172.21.15.103:0/1971802387 >> 172.21.15.103:0/1971802387 conn(0x55b9b81ba060 msgr2=0x55b9b81c6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.863+0000 7f1b80890740 1 -- 172.21.15.103:0/1971802387 shutdown_connections 2024-09-19T07:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.863+0000 7f1b80890740 1 -- 172.21.15.103:0/1971802387 wait complete. 2024-09-19T07:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.864+0000 7f1b80890740 20 client.20166 trim_cache size 0 max 16384 2024-09-19T07:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.864+0000 7f1b7da1e640 1 -- 172.21.15.103:0/3650947826 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1b600a3460 msgr2=0x7f1b600a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.864+0000 7f1b7da1e640 1 --2- 172.21.15.103:0/3650947826 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1b600a3460 0x7f1b600a5920 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1b70005fd0 tx=0x7f1b70004a40 comp rx=0 tx=0).stop 2024-09-19T07:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.864+0000 7f1b7da1e640 1 -- 172.21.15.103:0/3650947826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b78102470 msgr2=0x7f1b78198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.864+0000 7f1b7da1e640 1 --2- 172.21.15.103:0/3650947826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b78102470 0x7f1b78198810 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c006010 tx=0x7f1b6c003c10 comp rx=0 tx=0).stop 2024-09-19T07:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.864+0000 7f1b7da1e640 1 -- 172.21.15.103:0/3650947826 shutdown_connections 2024-09-19T07:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.864+0000 7f1b7da1e640 1 --2- 172.21.15.103:0/3650947826 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1b600a3460 0x7f1b600a5920 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.864+0000 7f1b7da1e640 1 --2- 172.21.15.103:0/3650947826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b78107b70 0x7f1b7819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.864+0000 7f1b7da1e640 1 --2- 172.21.15.103:0/3650947826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b78102470 0x7f1b78198810 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.864+0000 7f1b7da1e640 1 --2- 172.21.15.103:0/3650947826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b78101ad0 0x7f1b781982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:02:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.864+0000 7f1b7da1e640 1 -- 172.21.15.103:0/3650947826 >> 172.21.15.103:0/3650947826 conn(0x7f1b780fd7d0 msgr2=0x7f1b780fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:02:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.864+0000 7f1b7da1e640 1 -- 172.21.15.103:0/3650947826 shutdown_connections 2024-09-19T07:02:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:02:59.864+0000 7f1b7da1e640 1 -- 172.21.15.103:0/3650947826 wait complete. 2024-09-19T07:03:00.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:02:59 smithi174 ceph-mon[35637]: pgmap v429: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-09-19T07:03:00.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:02:59 smithi134 ceph-mon[35356]: pgmap v429: 97 pgs: 97 active+clean; 3.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-09-19T07:03:00.276 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:03:00.277 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:03:00.581 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:01.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.040+0000 7f68434d0640 1 -- 172.21.15.103:0/4154059309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f683c100a80 msgr2=0x7f683c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:01.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.040+0000 7f68434d0640 1 --2- 172.21.15.103:0/4154059309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f683c100a80 0x7f683c100ee0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f682c009930 tx=0x7f682c02a420 comp rx=0 tx=0).stop 2024-09-19T07:03:01.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.040+0000 7f68434d0640 1 -- 172.21.15.103:0/4154059309 shutdown_connections 2024-09-19T07:03:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.040+0000 7f68434d0640 1 --2- 172.21.15.103:0/4154059309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683c101420 0x7f683c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.040+0000 7f68434d0640 1 --2- 172.21.15.103:0/4154059309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f683c100a80 0x7f683c100ee0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.040+0000 7f68434d0640 1 --2- 172.21.15.103:0/4154059309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f683c106a80 0x7f683c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.040+0000 7f68434d0640 1 -- 172.21.15.103:0/4154059309 >> 172.21.15.103:0/4154059309 conn(0x7f683c0fc7a0 msgr2=0x7f683c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.040+0000 7f68434d0640 1 -- 172.21.15.103:0/4154059309 shutdown_connections 2024-09-19T07:03:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.040+0000 7f68434d0640 1 -- 172.21.15.103:0/4154059309 wait complete. 2024-09-19T07:03:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.041+0000 7f68434d0640 1 Processor -- start 2024-09-19T07:03:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.041+0000 7f68434d0640 1 -- start start 2024-09-19T07:03:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.041+0000 7f68434d0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683c100a80 0x7f683c0fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.041+0000 7f68434d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f683c101420 0x7f683c0fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.041+0000 7f68434d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f683c106a80 0x7f683c0fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.041+0000 7f68434d0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f683c110ea0 con 0x7f683c101420 2024-09-19T07:03:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.041+0000 7f68434d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f683c110d20 con 0x7f683c106a80 2024-09-19T07:03:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.041+0000 7f68434d0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f683c111020 con 0x7f683c100a80 2024-09-19T07:03:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.041+0000 7f6840a6b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f683c101420 0x7f683c0fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.041+0000 7f6840a6b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f683c101420 0x7f683c0fd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34966/0 (socket says 172.21.15.103:34966) 2024-09-19T07:03:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.041+0000 7f6840a6b640 1 -- 172.21.15.103:0/3327267175 learned_addr learned my addr 172.21.15.103:0/3327267175 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.042+0000 7f6841a6d640 1 --2- 172.21.15.103:0/3327267175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f683c106a80 0x7f683c0fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.042+0000 7f684126c640 1 --2- 172.21.15.103:0/3327267175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683c100a80 0x7f683c0fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.042+0000 7f6840a6b640 1 -- 172.21.15.103:0/3327267175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683c100a80 msgr2=0x7f683c0fcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.042+0000 7f6840a6b640 1 --2- 172.21.15.103:0/3327267175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683c100a80 0x7f683c0fcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.042+0000 7f6840a6b640 1 -- 172.21.15.103:0/3327267175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f683c106a80 msgr2=0x7f683c0fd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.042+0000 7f6840a6b640 1 --2- 172.21.15.103:0/3327267175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f683c106a80 0x7f683c0fd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.042+0000 7f6840a6b640 1 -- 172.21.15.103:0/3327267175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c1a6900 con 0x7f683c101420 2024-09-19T07:03:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.042+0000 7f6840a6b640 1 --2- 172.21.15.103:0/3327267175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f683c101420 0x7f683c0fd280 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f682c009a50 tx=0x7f682c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:01.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.042+0000 7f68327fc640 1 -- 172.21.15.103:0/3327267175 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c03f030 con 0x7f683c101420 2024-09-19T07:03:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.042+0000 7f68434d0640 1 -- 172.21.15.103:0/3327267175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683c1a6b90 con 0x7f683c101420 2024-09-19T07:03:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.042+0000 7f68327fc640 1 -- 172.21.15.103:0/3327267175 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f682c003ce0 con 0x7f683c101420 2024-09-19T07:03:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.042+0000 7f68327fc640 1 -- 172.21.15.103:0/3327267175 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c035070 con 0x7f683c101420 2024-09-19T07:03:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.042+0000 7f68434d0640 1 -- 172.21.15.103:0/3327267175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f683c1a7070 con 0x7f683c101420 2024-09-19T07:03:01.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.043+0000 7f68434d0640 1 -- 172.21.15.103:0/3327267175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f683c100ee0 con 0x7f683c101420 2024-09-19T07:03:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.045+0000 7f68327fc640 1 -- 172.21.15.103:0/3327267175 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f682c042030 con 0x7f683c101420 2024-09-19T07:03:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.045+0000 7f68327fc640 1 --2- 172.21.15.103:0/3327267175 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68280a33b0 0x7f68280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.045+0000 7f684126c640 1 --2- 172.21.15.103:0/3327267175 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68280a33b0 0x7f68280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.045+0000 7f68327fc640 1 -- 172.21.15.103:0/3327267175 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f682c0f06f0 con 0x7f683c101420 2024-09-19T07:03:01.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.046+0000 7f684126c640 1 --2- 172.21.15.103:0/3327267175 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68280a33b0 0x7f68280a5870 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6838005e10 tx=0x7f6838005da0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:01.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.049+0000 7f68327fc640 1 -- 172.21.15.103:0/3327267175 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f682c0bd0b0 con 0x7f683c101420 2024-09-19T07:03:01.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.249+0000 7f68434d0640 1 -- 172.21.15.103:0/3327267175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f683c10a020 con 0x7f683c101420 2024-09-19T07:03:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.250+0000 7f68327fc640 1 -- 172.21.15.103:0/3327267175 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f682c0bc9b0 con 0x7f683c101420 2024-09-19T07:03:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:03:01.251 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:03:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:03:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.253+0000 7f68434d0640 1 -- 172.21.15.103:0/3327267175 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68280a33b0 msgr2=0x7f68280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.253+0000 7f68434d0640 1 --2- 172.21.15.103:0/3327267175 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68280a33b0 0x7f68280a5870 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6838005e10 tx=0x7f6838005da0 comp rx=0 tx=0).stop 2024-09-19T07:03:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.253+0000 7f68434d0640 1 -- 172.21.15.103:0/3327267175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f683c101420 msgr2=0x7f683c0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.253+0000 7f68434d0640 1 --2- 172.21.15.103:0/3327267175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f683c101420 0x7f683c0fd280 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f682c009a50 tx=0x7f682c004270 comp rx=0 tx=0).stop 2024-09-19T07:03:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.253+0000 7f68434d0640 1 -- 172.21.15.103:0/3327267175 shutdown_connections 2024-09-19T07:03:01.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.253+0000 7f68434d0640 1 --2- 172.21.15.103:0/3327267175 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68280a33b0 0x7f68280a5870 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.253+0000 7f68434d0640 1 --2- 172.21.15.103:0/3327267175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f683c106a80 0x7f683c0fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.253+0000 7f68434d0640 1 --2- 172.21.15.103:0/3327267175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f683c101420 0x7f683c0fd280 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.253+0000 7f68434d0640 1 --2- 172.21.15.103:0/3327267175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683c100a80 0x7f683c0fcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.253+0000 7f68434d0640 1 -- 172.21.15.103:0/3327267175 >> 172.21.15.103:0/3327267175 conn(0x7f683c0fc7a0 msgr2=0x7f683c06a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.253+0000 7f68434d0640 1 -- 172.21.15.103:0/3327267175 shutdown_connections 2024-09-19T07:03:01.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:01.253+0000 7f68434d0640 1 -- 172.21.15.103:0/3327267175 wait complete. 2024-09-19T07:03:01.889 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:03:02.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:01 smithi174 ceph-mon[35637]: pgmap v430: 97 pgs: 97 active+clean; 3.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-09-19T07:03:02.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:01 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3327267175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:03:02.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:01 smithi103 ceph-mon[31158]: pgmap v430: 97 pgs: 97 active+clean; 3.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-09-19T07:03:02.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:01 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3327267175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:03:02.138 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:02.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:01 smithi134 ceph-mon[35356]: pgmap v430: 97 pgs: 97 active+clean; 3.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-09-19T07:03:02.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:01 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3327267175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:03:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.525+0000 7fd86857b640 1 -- 172.21.15.103:0/4260993519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd860102d40 msgr2=0x7fd860103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.525+0000 7fd86857b640 1 --2- 172.21.15.103:0/4260993519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd860102d40 0x7fd860103120 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009900 tx=0x7fd85c02a480 comp rx=0 tx=0).stop 2024-09-19T07:03:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.525+0000 7fd86857b640 1 -- 172.21.15.103:0/4260993519 shutdown_connections 2024-09-19T07:03:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.525+0000 7fd86857b640 1 --2- 172.21.15.103:0/4260993519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd86010c170 0x7fd86010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.525+0000 7fd86857b640 1 --2- 172.21.15.103:0/4260993519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd860103660 0x7fd86010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.525+0000 7fd86857b640 1 --2- 172.21.15.103:0/4260993519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd860102d40 0x7fd860103120 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.525+0000 7fd86857b640 1 -- 172.21.15.103:0/4260993519 >> 172.21.15.103:0/4260993519 conn(0x7fd8600fc7a0 msgr2=0x7fd8600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:02.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.525+0000 7fd86857b640 1 -- 172.21.15.103:0/4260993519 shutdown_connections 2024-09-19T07:03:02.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.525+0000 7fd86857b640 1 -- 172.21.15.103:0/4260993519 wait complete. 2024-09-19T07:03:02.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.526+0000 7fd86857b640 1 Processor -- start 2024-09-19T07:03:02.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.526+0000 7fd86857b640 1 -- start start 2024-09-19T07:03:02.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.526+0000 7fd86857b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd860102d40 0x7fd860198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.526+0000 7fd86857b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd860103660 0x7fd8601988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.526+0000 7fd86857b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd86010c170 0x7fd86019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.526+0000 7fd86857b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd860111f70 con 0x7fd860102d40 2024-09-19T07:03:02.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.526+0000 7fd86857b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd860111df0 con 0x7fd86010c170 2024-09-19T07:03:02.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.526+0000 7fd86857b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8601120f0 con 0x7fd860103660 2024-09-19T07:03:02.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.526+0000 7fd866317640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd860102d40 0x7fd860198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:02.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.526+0000 7fd866317640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd860102d40 0x7fd860198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34988/0 (socket says 172.21.15.103:34988) 2024-09-19T07:03:02.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.526+0000 7fd866317640 1 -- 172.21.15.103:0/1085995764 learned_addr learned my addr 172.21.15.103:0/1085995764 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:02.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.526+0000 7fd865b16640 1 --2- 172.21.15.103:0/1085995764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd860103660 0x7fd8601988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:02.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.526+0000 7fd866b18640 1 --2- 172.21.15.103:0/1085995764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd86010c170 0x7fd86019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:02.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.527+0000 7fd866317640 1 -- 172.21.15.103:0/1085995764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd860103660 msgr2=0x7fd8601988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:02.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.527+0000 7fd866317640 1 --2- 172.21.15.103:0/1085995764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd860103660 0x7fd8601988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.527+0000 7fd866317640 1 -- 172.21.15.103:0/1085995764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd86010c170 msgr2=0x7fd86019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:02.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.527+0000 7fd866317640 1 --2- 172.21.15.103:0/1085995764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd86010c170 0x7fd86019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.527+0000 7fd866317640 1 -- 172.21.15.103:0/1085995764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86019d310 con 0x7fd860102d40 2024-09-19T07:03:02.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.527+0000 7fd866b18640 1 --2- 172.21.15.103:0/1085995764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd86010c170 0x7fd86019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:02.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.527+0000 7fd866317640 1 --2- 172.21.15.103:0/1085995764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd860102d40 0x7fd860198360 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fd85c005ab0 tx=0x7fd85c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:02.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.527+0000 7fd8577fe640 1 -- 172.21.15.103:0/1085995764 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c03f030 con 0x7fd860102d40 2024-09-19T07:03:02.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.527+0000 7fd86857b640 1 -- 172.21.15.103:0/1085995764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86019d5a0 con 0x7fd860102d40 2024-09-19T07:03:02.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.527+0000 7fd8577fe640 1 -- 172.21.15.103:0/1085995764 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd85c003ce0 con 0x7fd860102d40 2024-09-19T07:03:02.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.527+0000 7fd8577fe640 1 -- 172.21.15.103:0/1085995764 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c035070 con 0x7fd860102d40 2024-09-19T07:03:02.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.527+0000 7fd86857b640 1 -- 172.21.15.103:0/1085995764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8601a4ee0 con 0x7fd860102d40 2024-09-19T07:03:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.529+0000 7fd8577fe640 1 -- 172.21.15.103:0/1085995764 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd85c042030 con 0x7fd860102d40 2024-09-19T07:03:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.530+0000 7fd86a954740 1 Processor -- start 2024-09-19T07:03:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.530+0000 7fd86a954740 1 -- start start 2024-09-19T07:03:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.530+0000 7fd8577fe640 1 --2- 172.21.15.103:0/1085995764 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd84c0a33b0 0x7fd84c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.530+0000 7fd86a954740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a2fcb40 0x561f8a2fb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.530+0000 7fd86a954740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fb5d0 0x561f8a2fba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.530+0000 7fd86a954740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a306370 0x561f8a308760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.530+0000 7fd86a954740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561f8a2612f0 con 0x561f8a2fcb40 2024-09-19T07:03:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.530+0000 7fd86a954740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561f8a208cf0 con 0x561f8a306370 2024-09-19T07:03:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.530+0000 7fd86a954740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561f8a25d2b0 con 0x561f8a2fb5d0 2024-09-19T07:03:02.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.530+0000 7fd866b18640 1 --2- 172.21.15.103:0/1085995764 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd84c0a33b0 0x7fd84c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.530+0000 7fd8577fe640 1 -- 172.21.15.103:0/1085995764 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7fd85c0f06b0 con 0x7fd860102d40 2024-09-19T07:03:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.530+0000 7fd866317640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a2fcb40 0x561f8a2fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.530+0000 7fd866317640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a2fcb40 0x561f8a2fb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34998/0 (socket says 172.21.15.103:34998) 2024-09-19T07:03:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.530+0000 7fd866317640 1 -- 172.21.15.103:0/244904184 learned_addr learned my addr 172.21.15.103:0/244904184 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.531+0000 7fd865b16640 1 --2- 172.21.15.103:0/244904184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fb5d0 0x561f8a2fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.531+0000 7fd866317640 1 -- 172.21.15.103:0/244904184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fb5d0 msgr2=0x561f8a2fba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.531+0000 7fd866b18640 1 --2- 172.21.15.103:0/244904184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a306370 0x561f8a308760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.531+0000 7fd866317640 1 --2- 172.21.15.103:0/244904184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fb5d0 0x561f8a2fba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.531+0000 7fd866317640 1 -- 172.21.15.103:0/244904184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a306370 msgr2=0x561f8a308760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.531+0000 7fd866317640 1 --2- 172.21.15.103:0/244904184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a306370 0x561f8a308760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.531+0000 7fd866317640 1 -- 172.21.15.103:0/244904184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f8a1ed3a0 con 0x561f8a2fcb40 2024-09-19T07:03:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.531+0000 7fd866b18640 1 --2- 172.21.15.103:0/244904184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a306370 0x561f8a308760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.531+0000 7fd866b18640 1 --2- 172.21.15.103:0/1085995764 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd84c0a33b0 0x7fd84c0a5870 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd858005cc0 tx=0x7fd85800d710 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.531+0000 7fd866317640 1 --2- 172.21.15.103:0/244904184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a2fcb40 0x561f8a2fb090 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0b92e0 tx=0x7fd85c0bc7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5d64c11b90f5eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.531+0000 7fd854ff9640 1 -- 172.21.15.103:0/244904184 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c049040 con 0x561f8a2fcb40 2024-09-19T07:03:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.531+0000 7fd854ff9640 1 -- 172.21.15.103:0/244904184 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd85c0bc910 con 0x561f8a2fcb40 2024-09-19T07:03:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.531+0000 7fd854ff9640 1 -- 172.21.15.103:0/244904184 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0a8400 con 0x561f8a2fcb40 2024-09-19T07:03:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.532+0000 7fd86a954740 1 -- 172.21.15.103:0/244904184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a2fcb40 msgr2=0x561f8a2fb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.532+0000 7fd86a954740 1 --2- 172.21.15.103:0/244904184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a2fcb40 0x561f8a2fb090 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0b92e0 tx=0x7fd85c0bc7a0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.532+0000 7fd86a954740 1 -- 172.21.15.103:0/244904184 shutdown_connections 2024-09-19T07:03:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.532+0000 7fd86a954740 1 --2- 172.21.15.103:0/244904184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a306370 0x561f8a308760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.532+0000 7fd86a954740 1 --2- 172.21.15.103:0/244904184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fb5d0 0x561f8a2fba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.532+0000 7fd86a954740 1 --2- 172.21.15.103:0/244904184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a2fcb40 0x561f8a2fb090 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.532+0000 7fd86a954740 1 -- 172.21.15.103:0/244904184 >> 172.21.15.103:0/244904184 conn(0x561f8a2c4310 msgr2=0x561f8a2f7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.532+0000 7fd86a954740 1 -- 172.21.15.103:0/244904184 shutdown_connections 2024-09-19T07:03:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.532+0000 7fd86a954740 1 -- 172.21.15.103:0/244904184 wait complete. 2024-09-19T07:03:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.532+0000 7fd86a954740 1 -- start start 2024-09-19T07:03:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.532+0000 7fd86a954740 1 Processor -- start 2024-09-19T07:03:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd86a954740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a2fb5d0 0x561f8a305d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd86a954740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fcb40 0x561f8a302ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd86a954740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a306370 0x561f8a3035e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd86a954740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561f8a30d240 con 0x561f8a306370 2024-09-19T07:03:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd86a954740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561f8a21c060 con 0x561f8a2fb5d0 2024-09-19T07:03:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd86a954740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561f8a30d3c0 con 0x561f8a2fcb40 2024-09-19T07:03:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd86a954740 10 client.0 _ll_register_callbacks cb 0x561f89f6f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd866b18640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a306370 0x561f8a3035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd866b18640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a306370 0x561f8a3035e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35004/0 (socket says 172.21.15.103:35004) 2024-09-19T07:03:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd866b18640 1 -- 172.21.15.103:0/2796406993 learned_addr learned my addr 172.21.15.103:0/2796406993 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd866317640 1 --2- 172.21.15.103:0/2796406993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a2fb5d0 0x561f8a305d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd865b16640 1 --2- 172.21.15.103:0/2796406993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fcb40 0x561f8a302ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd866b18640 1 -- 172.21.15.103:0/2796406993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fcb40 msgr2=0x561f8a302ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd866b18640 1 --2- 172.21.15.103:0/2796406993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fcb40 0x561f8a302ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd866b18640 1 -- 172.21.15.103:0/2796406993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a2fb5d0 msgr2=0x561f8a305d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd866b18640 1 --2- 172.21.15.103:0/2796406993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a2fb5d0 0x561f8a305d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd866b18640 1 -- 172.21.15.103:0/2796406993 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f89fd23c0 con 0x561f8a306370 2024-09-19T07:03:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd866b18640 1 --2- 172.21.15.103:0/2796406993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a306370 0x561f8a3035e0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fd85800fe00 tx=0x7fd858004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cb2c8c1c86d85f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd849ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd849ffb640 1 -- 172.21.15.103:0/2796406993 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858022030 con 0x561f8a306370 2024-09-19T07:03:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd86a954740 1 -- 172.21.15.103:0/2796406993 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd85c0bc910 con 0x561f8a306370 2024-09-19T07:03:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.533+0000 7fd849ffb640 1 -- 172.21.15.103:0/2796406993 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7fd858004b20 con 0x561f8a306370 2024-09-19T07:03:02.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.534+0000 7fd849ffb640 1 -- 172.21.15.103:0/2796406993 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd85802c050 con 0x561f8a306370 2024-09-19T07:03:02.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.534+0000 7fd86a954740 10 client.20184 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:02.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.534+0000 7fd86a954740 1 -- 172.21.15.103:0/2796406993 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f8a318c10 con 0x561f8a306370 2024-09-19T07:03:02.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.534+0000 7fd849ffb640 1 -- 172.21.15.103:0/2796406993 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd85c0bc910 con 0x561f8a306370 2024-09-19T07:03:02.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.534+0000 7fd849ffb640 1 -- 172.21.15.103:0/2796406993 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd85801e450 con 0x561f8a306370 2024-09-19T07:03:02.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.534+0000 7fd86a954740 10 client.20184 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:02.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.534+0000 7fd86a954740 10 client.20184 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:03:02.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.534+0000 7fd86a954740 20 client.20184 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:02.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.534+0000 7fd86a954740 1 --2- 172.21.15.103:0/2796406993 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x561f8a31a020 0x561f8a31c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.534+0000 7fd86a954740 4 client.20184 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:02.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.534+0000 7fd86a954740 1 -- 172.21.15.103:0/2796406993 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f8a26e860 con 0x561f8a31a020 2024-09-19T07:03:02.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.534+0000 7fd866317640 1 --2- 172.21.15.103:0/2796406993 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x561f8a31a020 0x561f8a31c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:02.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.535+0000 7fd849ffb640 10 client.20184 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:03:02.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.535+0000 7fd866317640 1 --2- 172.21.15.103:0/2796406993 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x561f8a31a020 0x561f8a31c4a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e6c9992a5faa8301 server_cookie=8c8654b9dc320774 in_seq=0 out_seq=0 2024-09-19T07:03:02.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.535+0000 7fd849ffb640 1 -- 172.21.15.103:0/2796406993 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f8a26e860 con 0x561f8a31a020 2024-09-19T07:03:02.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.535+0000 7fd849ffb640 10 client.20184 handle_command_reply: tid=0 2024-09-19T07:03:02.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.535+0000 7fd86a954740 1 client.20184 shutdown 2024-09-19T07:03:02.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.536+0000 7fd86a954740 1 -- 172.21.15.103:0/2796406993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a306370 msgr2=0x561f8a3035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:02.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.536+0000 7fd86a954740 1 --2- 172.21.15.103:0/2796406993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a306370 0x561f8a3035e0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fd85800fe00 tx=0x7fd858004ab0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.536+0000 7fd86a954740 1 -- 172.21.15.103:0/2796406993 shutdown_connections 2024-09-19T07:03:02.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.536+0000 7fd86a954740 1 --2- 172.21.15.103:0/2796406993 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x561f8a31a020 0x561f8a31c4a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.536+0000 7fd86a954740 1 --2- 172.21.15.103:0/2796406993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a306370 0x561f8a3035e0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.536+0000 7fd86a954740 1 --2- 172.21.15.103:0/2796406993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fcb40 0x561f8a302ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.536+0000 7fd86a954740 1 --2- 172.21.15.103:0/2796406993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a2fb5d0 0x561f8a305d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.536+0000 7fd849ffb640 0 client.20184 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:03:02.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.536+0000 7fd86a954740 1 -- 172.21.15.103:0/2796406993 >> 172.21.15.103:0/2796406993 conn(0x561f8a2c4310 msgr2=0x561f8a2fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:02.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.536+0000 7fd86a954740 1 -- 172.21.15.103:0/2796406993 shutdown_connections 2024-09-19T07:03:02.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.536+0000 7fd86a954740 1 -- 172.21.15.103:0/2796406993 wait complete. 2024-09-19T07:03:02.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.536+0000 7fd86a954740 20 client.20184 trim_cache size 0 max 16384 2024-09-19T07:03:02.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.548+0000 7fd86a954740 1 Processor -- start 2024-09-19T07:03:02.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.548+0000 7fd86a954740 1 -- start start 2024-09-19T07:03:02.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.548+0000 7fd86a954740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fb5d0 0x561f8a31c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.548+0000 7fd86a954740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a2fcb40 0x561f8a304f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.548+0000 7fd86a954740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a306370 0x561f8a2fe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.548+0000 7fd86a954740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8601ab5f0 con 0x561f8a306370 2024-09-19T07:03:02.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.548+0000 7fd86a954740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8601ad130 con 0x561f8a2fcb40 2024-09-19T07:03:02.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.548+0000 7fd86a954740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8601abcc0 con 0x561f8a2fb5d0 2024-09-19T07:03:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.549+0000 7fd866b18640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a306370 0x561f8a2fe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.549+0000 7fd866b18640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a306370 0x561f8a2fe180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35020/0 (socket says 172.21.15.103:35020) 2024-09-19T07:03:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.549+0000 7fd866b18640 1 -- 172.21.15.103:0/3487634443 learned_addr learned my addr 172.21.15.103:0/3487634443 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.549+0000 7fd865b16640 1 --2- 172.21.15.103:0/3487634443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a2fcb40 0x561f8a304f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.549+0000 7fd866317640 1 --2- 172.21.15.103:0/3487634443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fb5d0 0x561f8a31c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.549+0000 7fd866b18640 1 -- 172.21.15.103:0/3487634443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fb5d0 msgr2=0x561f8a31c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.549+0000 7fd866b18640 1 --2- 172.21.15.103:0/3487634443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fb5d0 0x561f8a31c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.549+0000 7fd866b18640 1 -- 172.21.15.103:0/3487634443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a2fcb40 msgr2=0x561f8a304f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.549+0000 7fd866b18640 1 --2- 172.21.15.103:0/3487634443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a2fcb40 0x561f8a304f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.549+0000 7fd866b18640 1 -- 172.21.15.103:0/3487634443 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f8a303be0 con 0x561f8a306370 2024-09-19T07:03:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.549+0000 7fd866b18640 1 --2- 172.21.15.103:0/3487634443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a306370 0x561f8a2fe180 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fd85801e870 tx=0x7fd85800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7dad86be205411f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.549+0000 7fd84bfff640 1 -- 172.21.15.103:0/3487634443 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858034070 con 0x561f8a306370 2024-09-19T07:03:02.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.549+0000 7fd84bfff640 1 -- 172.21.15.103:0/3487634443 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd85801eae0 con 0x561f8a306370 2024-09-19T07:03:02.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.549+0000 7fd84bfff640 1 -- 172.21.15.103:0/3487634443 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858022040 con 0x561f8a306370 2024-09-19T07:03:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.550+0000 7fd86a954740 1 -- 172.21.15.103:0/3487634443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a306370 msgr2=0x561f8a2fe180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.550+0000 7fd86a954740 1 --2- 172.21.15.103:0/3487634443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a306370 0x561f8a2fe180 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fd85801e870 tx=0x7fd85800fab0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.550+0000 7fd86a954740 1 -- 172.21.15.103:0/3487634443 shutdown_connections 2024-09-19T07:03:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.550+0000 7fd86a954740 1 --2- 172.21.15.103:0/3487634443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a306370 0x561f8a2fe180 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.550+0000 7fd86a954740 1 --2- 172.21.15.103:0/3487634443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a2fcb40 0x561f8a304f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.550+0000 7fd86a954740 1 --2- 172.21.15.103:0/3487634443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fb5d0 0x561f8a31c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.550+0000 7fd86a954740 1 -- 172.21.15.103:0/3487634443 >> 172.21.15.103:0/3487634443 conn(0x561f8a2c4310 msgr2=0x561f8a304850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.550+0000 7fd86a954740 1 -- 172.21.15.103:0/3487634443 shutdown_connections 2024-09-19T07:03:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.550+0000 7fd86a954740 1 -- 172.21.15.103:0/3487634443 wait complete. 2024-09-19T07:03:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.550+0000 7fd86a954740 1 -- start start 2024-09-19T07:03:02.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.550+0000 7fd86a954740 1 Processor -- start 2024-09-19T07:03:02.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd86a954740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a2fb5d0 0x561f8a37d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd86a954740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fcb40 0x561f8a37d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd86a954740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a306370 0x561f8a30dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd86a954740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8601aca60 con 0x561f8a2fb5d0 2024-09-19T07:03:02.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd86a954740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8601ac390 con 0x561f8a306370 2024-09-19T07:03:02.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd86a954740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8601a95f0 con 0x561f8a2fcb40 2024-09-19T07:03:02.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd86a954740 10 client.0 _ll_register_callbacks cb 0x7fd858015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:02.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd866317640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a2fb5d0 0x561f8a37d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:02.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd866317640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a2fb5d0 0x561f8a37d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35026/0 (socket says 172.21.15.103:35026) 2024-09-19T07:03:02.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd866317640 1 -- 172.21.15.103:0/1529594604 learned_addr learned my addr 172.21.15.103:0/1529594604 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:02.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd866317640 1 -- 172.21.15.103:0/1529594604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fcb40 msgr2=0x561f8a37d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:02.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd866317640 1 --2- 172.21.15.103:0/1529594604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fcb40 0x561f8a37d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd866317640 1 -- 172.21.15.103:0/1529594604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a306370 msgr2=0x561f8a30dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:02.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd866317640 1 --2- 172.21.15.103:0/1529594604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a306370 0x561f8a30dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd866317640 1 -- 172.21.15.103:0/1529594604 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f8a30e3b0 con 0x561f8a2fb5d0 2024-09-19T07:03:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd866317640 1 --2- 172.21.15.103:0/1529594604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a2fb5d0 0x561f8a37d0a0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fd85c000c00 tx=0x7fd85c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb92923719342cac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd849ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd849ffb640 1 -- 172.21.15.103:0/1529594604 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0b6060 con 0x561f8a2fb5d0 2024-09-19T07:03:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd849ffb640 1 -- 172.21.15.103:0/1529594604 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7fd85c03bd70 con 0x561f8a2fb5d0 2024-09-19T07:03:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.551+0000 7fd86a954740 1 -- 172.21.15.103:0/1529594604 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd85801eae0 con 0x561f8a2fb5d0 2024-09-19T07:03:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.552+0000 7fd849ffb640 1 -- 172.21.15.103:0/1529594604 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd85c049040 con 0x561f8a2fb5d0 2024-09-19T07:03:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.552+0000 7fd86a954740 10 client.20190 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.552+0000 7fd86a954740 1 -- 172.21.15.103:0/1529594604 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f8a3848a0 con 0x561f8a2fb5d0 2024-09-19T07:03:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.552+0000 7fd849ffb640 1 -- 172.21.15.103:0/1529594604 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd85801eae0 con 0x561f8a2fb5d0 2024-09-19T07:03:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.552+0000 7fd849ffb640 1 -- 172.21.15.103:0/1529594604 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd85c0ee000 con 0x561f8a2fb5d0 2024-09-19T07:03:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.552+0000 7fd86a954740 10 client.20190 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.552+0000 7fd86a954740 10 client.20190 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:03:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.552+0000 7fd86a954740 20 client.20190 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.552+0000 7fd86a954740 1 --2- 172.21.15.103:0/1529594604 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x561f8a384ca0 0x561f8a3870d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.552+0000 7fd86a954740 4 client.20190 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:03:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.552+0000 7fd86a954740 1 -- 172.21.15.103:0/1529594604 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561f8a268d70 con 0x561f8a384ca0 2024-09-19T07:03:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.552+0000 7fd865b16640 1 --2- 172.21.15.103:0/1529594604 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x561f8a384ca0 0x561f8a3870d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd849ffb640 10 client.20190 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:03:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd865b16640 1 --2- 172.21.15.103:0/1529594604 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x561f8a384ca0 0x561f8a3870d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3a2ab86cacbadc2 server_cookie=33a882ed29b5d7cb in_seq=0 out_seq=0 2024-09-19T07:03:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd849ffb640 1 -- 172.21.15.103:0/1529594604 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561f8a268d70 con 0x561f8a384ca0 2024-09-19T07:03:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd849ffb640 10 client.20190 handle_command_reply: tid=0 2024-09-19T07:03:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd86a954740 1 client.20190 shutdown 2024-09-19T07:03:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd86a954740 1 -- 172.21.15.103:0/1529594604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a2fb5d0 msgr2=0x561f8a37d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd86a954740 1 --2- 172.21.15.103:0/1529594604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a2fb5d0 0x561f8a37d0a0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fd85c000c00 tx=0x7fd85c0045e0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd86a954740 1 -- 172.21.15.103:0/1529594604 shutdown_connections 2024-09-19T07:03:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd86a954740 1 --2- 172.21.15.103:0/1529594604 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x561f8a384ca0 0x561f8a3870d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd849ffb640 0 client.20190 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:03:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd86a954740 1 --2- 172.21.15.103:0/1529594604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f8a306370 0x561f8a30dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd86a954740 1 --2- 172.21.15.103:0/1529594604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f8a2fcb40 0x561f8a37d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd86a954740 1 --2- 172.21.15.103:0/1529594604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f8a2fb5d0 0x561f8a37d0a0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd86a954740 1 -- 172.21.15.103:0/1529594604 >> 172.21.15.103:0/1529594604 conn(0x561f8a2c4310 msgr2=0x561f8a2f6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd86a954740 1 -- 172.21.15.103:0/1529594604 shutdown_connections 2024-09-19T07:03:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd86a954740 1 -- 172.21.15.103:0/1529594604 wait complete. 2024-09-19T07:03:02.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.553+0000 7fd86a954740 20 client.20190 trim_cache size 0 max 16384 2024-09-19T07:03:02.559 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:03:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.554+0000 7fd86857b640 1 -- 172.21.15.103:0/1085995764 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd84c0a33b0 msgr2=0x7fd84c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.554+0000 7fd86857b640 1 --2- 172.21.15.103:0/1085995764 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd84c0a33b0 0x7fd84c0a5870 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd858005cc0 tx=0x7fd85800d710 comp rx=0 tx=0).stop 2024-09-19T07:03:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.554+0000 7fd86857b640 1 -- 172.21.15.103:0/1085995764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd860102d40 msgr2=0x7fd860198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.554+0000 7fd86857b640 1 --2- 172.21.15.103:0/1085995764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd860102d40 0x7fd860198360 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fd85c005ab0 tx=0x7fd85c0042f0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.554+0000 7fd86857b640 1 -- 172.21.15.103:0/1085995764 shutdown_connections 2024-09-19T07:03:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.554+0000 7fd86857b640 1 --2- 172.21.15.103:0/1085995764 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd84c0a33b0 0x7fd84c0a5870 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.554+0000 7fd86857b640 1 --2- 172.21.15.103:0/1085995764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd86010c170 0x7fd86019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.554+0000 7fd86857b640 1 --2- 172.21.15.103:0/1085995764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd860103660 0x7fd8601988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.554+0000 7fd86857b640 1 --2- 172.21.15.103:0/1085995764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd860102d40 0x7fd860198360 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.554+0000 7fd86857b640 1 -- 172.21.15.103:0/1085995764 >> 172.21.15.103:0/1085995764 conn(0x7fd8600fc7a0 msgr2=0x7fd8600fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.554+0000 7fd86857b640 1 -- 172.21.15.103:0/1085995764 shutdown_connections 2024-09-19T07:03:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:02.554+0000 7fd86857b640 1 -- 172.21.15.103:0/1085995764 wait complete. 2024-09-19T07:03:03.277 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:03:03.525 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:03.937 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:03 smithi103 ceph-mon[31158]: pgmap v431: 97 pgs: 97 active+clean; 3.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-19T07:03:03.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.943+0000 7f219dc69640 1 -- 172.21.15.103:0/2911220283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2198101420 msgr2=0x7f219810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:03.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.943+0000 7f219dc69640 1 --2- 172.21.15.103:0/2911220283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2198101420 0x7f219810d950 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f2190009cd0 tx=0x7f219002a8e0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.943+0000 7f219dc69640 1 -- 172.21.15.103:0/2911220283 shutdown_connections 2024-09-19T07:03:03.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.943+0000 7f219dc69640 1 --2- 172.21.15.103:0/2911220283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2198101420 0x7f219810d950 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.943+0000 7f219dc69640 1 --2- 172.21.15.103:0/2911220283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198100a80 0x7f2198100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.943+0000 7f219dc69640 1 --2- 172.21.15.103:0/2911220283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2198106a80 0x7f2198106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.943+0000 7f219dc69640 1 -- 172.21.15.103:0/2911220283 >> 172.21.15.103:0/2911220283 conn(0x7f21980fc7a0 msgr2=0x7f21980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:03.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.943+0000 7f219dc69640 1 -- 172.21.15.103:0/2911220283 shutdown_connections 2024-09-19T07:03:03.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.943+0000 7f219dc69640 1 -- 172.21.15.103:0/2911220283 wait complete. 2024-09-19T07:03:03.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.943+0000 7f219dc69640 1 Processor -- start 2024-09-19T07:03:03.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.943+0000 7f219dc69640 1 -- start start 2024-09-19T07:03:03.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f219dc69640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198100a80 0x7f2198199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f219dc69640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2198101420 0x7f21981997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f219dc69640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2198106a80 0x7f21981a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f219dc69640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2198110e80 con 0x7f2198101420 2024-09-19T07:03:03.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f219dc69640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2198110d00 con 0x7f2198100a80 2024-09-19T07:03:03.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f219dc69640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2198111000 con 0x7f2198106a80 2024-09-19T07:03:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f218ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2198101420 0x7f21981997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f218ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2198101420 0x7f21981997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35042/0 (socket says 172.21.15.103:35042) 2024-09-19T07:03:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f218ffff640 1 -- 172.21.15.103:0/823128077 learned_addr learned my addr 172.21.15.103:0/823128077 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f21977fe640 1 --2- 172.21.15.103:0/823128077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198100a80 0x7f2198199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f2197fff640 1 --2- 172.21.15.103:0/823128077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2198106a80 0x7f21981a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f218ffff640 1 -- 172.21.15.103:0/823128077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2198106a80 msgr2=0x7f21981a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f218ffff640 1 --2- 172.21.15.103:0/823128077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2198106a80 0x7f21981a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f218ffff640 1 -- 172.21.15.103:0/823128077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198100a80 msgr2=0x7f2198199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f218ffff640 1 --2- 172.21.15.103:0/823128077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198100a80 0x7f2198199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f218ffff640 1 -- 172.21.15.103:0/823128077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21981a4840 con 0x7f2198101420 2024-09-19T07:03:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f2197fff640 1 --2- 172.21.15.103:0/823128077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2198106a80 0x7f21981a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f218ffff640 1 --2- 172.21.15.103:0/823128077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2198101420 0x7f21981997c0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f218000c890 tx=0x7f218000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f21957fa640 1 -- 172.21.15.103:0/823128077 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180004220 con 0x7f2198101420 2024-09-19T07:03:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f21957fa640 1 -- 172.21.15.103:0/823128077 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21800043c0 con 0x7f2198101420 2024-09-19T07:03:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f219dc69640 1 -- 172.21.15.103:0/823128077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21981a4a10 con 0x7f2198101420 2024-09-19T07:03:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.944+0000 7f21957fa640 1 -- 172.21.15.103:0/823128077 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21800049e0 con 0x7f2198101420 2024-09-19T07:03:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.945+0000 7f219dc69640 1 -- 172.21.15.103:0/823128077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21981a4f50 con 0x7f2198101420 2024-09-19T07:03:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.947+0000 7f21957fa640 1 -- 172.21.15.103:0/823128077 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f21800079b0 con 0x7f2198101420 2024-09-19T07:03:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.947+0000 7f21a0042740 1 Processor -- start 2024-09-19T07:03:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.947+0000 7f21a0042740 1 -- start start 2024-09-19T07:03:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.947+0000 7f21a0042740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b01b40 0x5591c0b00090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.947+0000 7f21a0042740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b005d0 0x5591c0b00a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.947+0000 7f21a0042740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b0b370 0x5591c0b0d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.947+0000 7f21a0042740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5591c0a662f0 con 0x5591c0b005d0 2024-09-19T07:03:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.947+0000 7f21a0042740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5591c0a0dcf0 con 0x5591c0b01b40 2024-09-19T07:03:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.947+0000 7f21a0042740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5591c0a622b0 con 0x5591c0b0b370 2024-09-19T07:03:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f21977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b005d0 0x5591c0b00a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f21977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b005d0 0x5591c0b00a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35046/0 (socket says 172.21.15.103:35046) 2024-09-19T07:03:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f21977fe640 1 -- 172.21.15.103:0/314136822 learned_addr learned my addr 172.21.15.103:0/314136822 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f2197fff640 1 --2- 172.21.15.103:0/314136822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b01b40 0x5591c0b00090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f218ffff640 1 --2- 172.21.15.103:0/314136822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b0b370 0x5591c0b0d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f21957fa640 1 --2- 172.21.15.103:0/823128077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21840a3430 0x7f21840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f21977fe640 1 -- 172.21.15.103:0/314136822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b0b370 msgr2=0x5591c0b0d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f21977fe640 1 --2- 172.21.15.103:0/314136822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b0b370 0x5591c0b0d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f21977fe640 1 -- 172.21.15.103:0/314136822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b01b40 msgr2=0x5591c0b00090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f21977fe640 1 --2- 172.21.15.103:0/314136822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b01b40 0x5591c0b00090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f21977fe640 1 -- 172.21.15.103:0/314136822 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591c09f23a0 con 0x5591c0b005d0 2024-09-19T07:03:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f2197fff640 1 --2- 172.21.15.103:0/823128077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21840a3430 0x7f21840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f21957fa640 1 -- 172.21.15.103:0/823128077 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f21800cfe10 con 0x7f2198101420 2024-09-19T07:03:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f21977fe640 1 --2- 172.21.15.103:0/314136822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b005d0 0x5591c0b00a10 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f218800dc30 tx=0x7f218800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9deb5c3516cda8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f218dffb640 1 -- 172.21.15.103:0/314136822 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188011440 con 0x5591c0b005d0 2024-09-19T07:03:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f218dffb640 1 -- 172.21.15.103:0/314136822 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2188017070 con 0x5591c0b005d0 2024-09-19T07:03:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.948+0000 7f218dffb640 1 -- 172.21.15.103:0/314136822 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21880109c0 con 0x5591c0b005d0 2024-09-19T07:03:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.949+0000 7f21a0042740 1 -- 172.21.15.103:0/314136822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b005d0 msgr2=0x5591c0b00a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.949+0000 7f2197fff640 1 --2- 172.21.15.103:0/823128077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21840a3430 0x7f21840a58f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2190009430 tx=0x7f219003f040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.949+0000 7f21a0042740 1 --2- 172.21.15.103:0/314136822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b005d0 0x5591c0b00a10 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f218800dc30 tx=0x7f218800f730 comp rx=0 tx=0).stop 2024-09-19T07:03:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.949+0000 7f21a0042740 1 -- 172.21.15.103:0/314136822 shutdown_connections 2024-09-19T07:03:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.949+0000 7f21a0042740 1 --2- 172.21.15.103:0/314136822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b0b370 0x5591c0b0d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.949+0000 7f21a0042740 1 --2- 172.21.15.103:0/314136822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b005d0 0x5591c0b00a10 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.949+0000 7f21a0042740 1 --2- 172.21.15.103:0/314136822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b01b40 0x5591c0b00090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.949+0000 7f21a0042740 1 -- 172.21.15.103:0/314136822 >> 172.21.15.103:0/314136822 conn(0x5591c0ac9310 msgr2=0x5591c0afcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:03.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.949+0000 7f21a0042740 1 -- 172.21.15.103:0/314136822 shutdown_connections 2024-09-19T07:03:03.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.949+0000 7f21a0042740 1 -- 172.21.15.103:0/314136822 wait complete. 2024-09-19T07:03:03.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.949+0000 7f21a0042740 1 -- start start 2024-09-19T07:03:03.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f21a0042740 1 Processor -- start 2024-09-19T07:03:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f21a0042740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b005d0 0x5591c0b0ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f21a0042740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b01b40 0x5591c0b07fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f21a0042740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b0b370 0x5591c0b08560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f21a0042740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5591c0b0e070 con 0x5591c0b01b40 2024-09-19T07:03:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f21a0042740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5591c0a21060 con 0x5591c0b0b370 2024-09-19T07:03:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f21a0042740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5591c0b0e1f0 con 0x5591c0b005d0 2024-09-19T07:03:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f21a0042740 10 client.0 _ll_register_callbacks cb 0x5591c07748c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f218ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b01b40 0x5591c0b07fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f218ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b01b40 0x5591c0b07fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35048/0 (socket says 172.21.15.103:35048) 2024-09-19T07:03:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f218ffff640 1 -- 172.21.15.103:0/2135898535 learned_addr learned my addr 172.21.15.103:0/2135898535 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f21977fe640 1 --2- 172.21.15.103:0/2135898535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b005d0 0x5591c0b0ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f2197fff640 1 --2- 172.21.15.103:0/2135898535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b0b370 0x5591c0b08560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f218ffff640 1 -- 172.21.15.103:0/2135898535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b005d0 msgr2=0x5591c0b0ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f218ffff640 1 --2- 172.21.15.103:0/2135898535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b005d0 0x5591c0b0ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f218ffff640 1 -- 172.21.15.103:0/2135898535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b0b370 msgr2=0x5591c0b08560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.950+0000 7f218ffff640 1 --2- 172.21.15.103:0/2135898535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b0b370 0x5591c0b08560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f218ffff640 1 -- 172.21.15.103:0/2135898535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591c07d73c0 con 0x5591c0b01b40 2024-09-19T07:03:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f218ffff640 1 --2- 172.21.15.103:0/2135898535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b01b40 0x5591c0b07fc0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f2180098aa0 tx=0x7f21800054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678f972d69dfae20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f2176ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f2176ffd640 1 -- 172.21.15.103:0/2135898535 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218001d030 con 0x5591c0b01b40 2024-09-19T07:03:03.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f2176ffd640 1 -- 172.21.15.103:0/2135898535 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f21800a0120 con 0x5591c0b01b40 2024-09-19T07:03:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f21a0042740 1 -- 172.21.15.103:0/2135898535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2188017070 con 0x5591c0b01b40 2024-09-19T07:03:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f2176ffd640 1 -- 172.21.15.103:0/2135898535 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f21800a4d40 con 0x5591c0b01b40 2024-09-19T07:03:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f21a0042740 10 client.20202 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f21a0042740 1 -- 172.21.15.103:0/2135898535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5591c0b19ac0 con 0x5591c0b01b40 2024-09-19T07:03:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f2176ffd640 1 -- 172.21.15.103:0/2135898535 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2188017070 con 0x5591c0b01b40 2024-09-19T07:03:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f2176ffd640 1 -- 172.21.15.103:0/2135898535 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2180007ca0 con 0x5591c0b01b40 2024-09-19T07:03:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f21a0042740 10 client.20202 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f21a0042740 10 client.20202 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:03:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f21a0042740 20 client.20202 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f21a0042740 1 --2- 172.21.15.103:0/2135898535 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5591c0b1afd0 0x5591c0b1d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f21a0042740 4 client.20202 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.951+0000 7f21a0042740 1 -- 172.21.15.103:0/2135898535 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591c0a73860 con 0x5591c0b1afd0 2024-09-19T07:03:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.952+0000 7f21977fe640 1 --2- 172.21.15.103:0/2135898535 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5591c0b1afd0 0x5591c0b1d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.952+0000 7f21977fe640 1 --2- 172.21.15.103:0/2135898535 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5591c0b1afd0 0x5591c0b1d3c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8fc001c94e6cffbe server_cookie=a6ad531f5fa59c1e in_seq=0 out_seq=0 2024-09-19T07:03:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.952+0000 7f2176ffd640 10 client.20202 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:03:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.953+0000 7f2176ffd640 1 -- 172.21.15.103:0/2135898535 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5591c0a73860 con 0x5591c0b1afd0 2024-09-19T07:03:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.953+0000 7f2176ffd640 10 client.20202 handle_command_reply: tid=0 2024-09-19T07:03:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.953+0000 7f21a0042740 1 client.20202 shutdown 2024-09-19T07:03:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.953+0000 7f21a0042740 1 -- 172.21.15.103:0/2135898535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b01b40 msgr2=0x5591c0b07fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.953+0000 7f21a0042740 1 --2- 172.21.15.103:0/2135898535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b01b40 0x5591c0b07fc0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f2180098aa0 tx=0x7f21800054a0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.953+0000 7f21a0042740 1 -- 172.21.15.103:0/2135898535 shutdown_connections 2024-09-19T07:03:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.953+0000 7f21a0042740 1 --2- 172.21.15.103:0/2135898535 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5591c0b1afd0 0x5591c0b1d3c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.953+0000 7f21a0042740 1 --2- 172.21.15.103:0/2135898535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b0b370 0x5591c0b08560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.953+0000 7f21a0042740 1 --2- 172.21.15.103:0/2135898535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b01b40 0x5591c0b07fc0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.953+0000 7f2176ffd640 0 client.20202 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:03:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.953+0000 7f21a0042740 1 --2- 172.21.15.103:0/2135898535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b005d0 0x5591c0b0ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.953+0000 7f21a0042740 1 -- 172.21.15.103:0/2135898535 >> 172.21.15.103:0/2135898535 conn(0x5591c0ac9310 msgr2=0x5591c0b02930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.953+0000 7f21a0042740 1 -- 172.21.15.103:0/2135898535 shutdown_connections 2024-09-19T07:03:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.954+0000 7f21a0042740 1 -- 172.21.15.103:0/2135898535 wait complete. 2024-09-19T07:03:03.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.954+0000 7f21a0042740 20 client.20202 trim_cache size 0 max 16384 2024-09-19T07:03:03.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.966+0000 7f21a0042740 1 Processor -- start 2024-09-19T07:03:03.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.966+0000 7f21a0042740 1 -- start start 2024-09-19T07:03:03.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f21a0042740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b005d0 0x5591c0afce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f21a0042740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b01b40 0x5591c0b17310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f21a0042740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b0b370 0x5591c0b22c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f21a0042740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21981ad930 con 0x5591c0b01b40 2024-09-19T07:03:03.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f21a0042740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21981af470 con 0x5591c0b005d0 2024-09-19T07:03:03.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f21a0042740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21981ae000 con 0x5591c0b0b370 2024-09-19T07:03:03.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f218ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b01b40 0x5591c0b17310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f218ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b01b40 0x5591c0b17310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35058/0 (socket says 172.21.15.103:35058) 2024-09-19T07:03:03.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f218ffff640 1 -- 172.21.15.103:0/1627940637 learned_addr learned my addr 172.21.15.103:0/1627940637 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:03.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f2197fff640 1 --2- 172.21.15.103:0/1627940637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b0b370 0x5591c0b22c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f218ffff640 1 -- 172.21.15.103:0/1627940637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b0b370 msgr2=0x5591c0b22c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:03.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f218ffff640 1 --2- 172.21.15.103:0/1627940637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b0b370 0x5591c0b22c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f218ffff640 1 -- 172.21.15.103:0/1627940637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b005d0 msgr2=0x5591c0afce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:03.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f21977fe640 1 --2- 172.21.15.103:0/1627940637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b005d0 0x5591c0afce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f218ffff640 1 --2- 172.21.15.103:0/1627940637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b005d0 0x5591c0afce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f218ffff640 1 -- 172.21.15.103:0/1627940637 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591c0b03120 con 0x5591c0b01b40 2024-09-19T07:03:03.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f218ffff640 1 --2- 172.21.15.103:0/1627940637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b01b40 0x5591c0b17310 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f2180016000 tx=0x7f21800057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1067a297c654f45 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:03.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f218d7fa640 1 -- 172.21.15.103:0/1627940637 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218001d070 con 0x5591c0b01b40 2024-09-19T07:03:03.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f218d7fa640 1 -- 172.21.15.103:0/1627940637 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21800a0480 con 0x5591c0b01b40 2024-09-19T07:03:03.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.967+0000 7f218d7fa640 1 -- 172.21.15.103:0/1627940637 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180007ca0 con 0x5591c0b01b40 2024-09-19T07:03:03.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.968+0000 7f21a0042740 1 -- 172.21.15.103:0/1627940637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b01b40 msgr2=0x5591c0b17310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:03.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.968+0000 7f21a0042740 1 --2- 172.21.15.103:0/1627940637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b01b40 0x5591c0b17310 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f2180016000 tx=0x7f21800057a0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.968+0000 7f21a0042740 1 -- 172.21.15.103:0/1627940637 shutdown_connections 2024-09-19T07:03:03.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.968+0000 7f21a0042740 1 --2- 172.21.15.103:0/1627940637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b0b370 0x5591c0b22c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.968+0000 7f21a0042740 1 --2- 172.21.15.103:0/1627940637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b01b40 0x5591c0b17310 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.968+0000 7f21a0042740 1 --2- 172.21.15.103:0/1627940637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b005d0 0x5591c0afce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.968+0000 7f21a0042740 1 -- 172.21.15.103:0/1627940637 >> 172.21.15.103:0/1627940637 conn(0x5591c0ac9310 msgr2=0x5591c0afc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:03.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.968+0000 7f21a0042740 1 -- 172.21.15.103:0/1627940637 shutdown_connections 2024-09-19T07:03:03.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.968+0000 7f21a0042740 1 -- 172.21.15.103:0/1627940637 wait complete. 2024-09-19T07:03:03.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.968+0000 7f21a0042740 1 -- start start 2024-09-19T07:03:03.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21a0042740 1 Processor -- start 2024-09-19T07:03:03.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21a0042740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b005d0 0x5591c0b13660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21a0042740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b01b40 0x5591c0b13ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21a0042740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b0b370 0x5591c0b10450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21a0042740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21981aeda0 con 0x5591c0b005d0 2024-09-19T07:03:03.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21a0042740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21981ae6d0 con 0x5591c0b01b40 2024-09-19T07:03:03.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21a0042740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21981ab930 con 0x5591c0b0b370 2024-09-19T07:03:03.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21a0042740 10 client.0 _ll_register_callbacks cb 0x7f2180004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:03.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b005d0 0x5591c0b13660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b005d0 0x5591c0b13660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35066/0 (socket says 172.21.15.103:35066) 2024-09-19T07:03:03.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21977fe640 1 -- 172.21.15.103:0/249379401 learned_addr learned my addr 172.21.15.103:0/249379401 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:03.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f218ffff640 1 --2- 172.21.15.103:0/249379401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b01b40 0x5591c0b13ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f2197fff640 1 --2- 172.21.15.103:0/249379401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b0b370 0x5591c0b10450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21977fe640 1 -- 172.21.15.103:0/249379401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b0b370 msgr2=0x5591c0b10450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:03.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21977fe640 1 --2- 172.21.15.103:0/249379401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b0b370 0x5591c0b10450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21977fe640 1 -- 172.21.15.103:0/249379401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b01b40 msgr2=0x5591c0b13ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:03.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21977fe640 1 --2- 172.21.15.103:0/249379401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b01b40 0x5591c0b13ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21977fe640 1 -- 172.21.15.103:0/249379401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591c0b10ae0 con 0x5591c0b005d0 2024-09-19T07:03:03.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f2197fff640 1 --2- 172.21.15.103:0/249379401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b0b370 0x5591c0b10450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:03.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f21977fe640 1 --2- 172.21.15.103:0/249379401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b005d0 0x5591c0b13660 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x5591c0afdac0 tx=0x7f21880105a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc893850b77f8f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:03.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.969+0000 7f2176ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:03.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.970+0000 7f2176ffd640 1 -- 172.21.15.103:0/249379401 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188004240 con 0x5591c0b005d0 2024-09-19T07:03:03.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.970+0000 7f2176ffd640 1 -- 172.21.15.103:0/249379401 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8556+0+0 (secure 0 0 0) 0x7f21880043e0 con 0x5591c0b005d0 2024-09-19T07:03:03.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.970+0000 7f21a0042740 1 -- 172.21.15.103:0/249379401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21800a0480 con 0x5591c0b005d0 2024-09-19T07:03:03.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.970+0000 7f2176ffd640 1 -- 172.21.15.103:0/249379401 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2188039050 con 0x5591c0b005d0 2024-09-19T07:03:03.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.970+0000 7f21a0042740 10 client.20208 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:03.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.970+0000 7f21a0042740 1 -- 172.21.15.103:0/249379401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5591c0b86650 con 0x5591c0b005d0 2024-09-19T07:03:03.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.970+0000 7f2176ffd640 1 -- 172.21.15.103:0/249379401 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21800a0480 con 0x5591c0b005d0 2024-09-19T07:03:03.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.970+0000 7f2176ffd640 1 -- 172.21.15.103:0/249379401 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2188004840 con 0x5591c0b005d0 2024-09-19T07:03:03.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.970+0000 7f21a0042740 10 client.20208 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:03.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.970+0000 7f21a0042740 10 client.20208 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:03:03.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.970+0000 7f21a0042740 20 client.20208 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:03.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.970+0000 7f21a0042740 1 --2- 172.21.15.103:0/249379401 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5591c0b86b20 0x5591c0b88f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:03.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.970+0000 7f21a0042740 4 client.20208 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:03:03.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.970+0000 7f21a0042740 1 -- 172.21.15.103:0/249379401 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591c0a6dd70 con 0x5591c0b86b20 2024-09-19T07:03:03.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.971+0000 7f218ffff640 1 --2- 172.21.15.103:0/249379401 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5591c0b86b20 0x5591c0b88f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:03.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.971+0000 7f218ffff640 1 --2- 172.21.15.103:0/249379401 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5591c0b86b20 0x5591c0b88f40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d4dd847d62d7ef9d server_cookie=c8bd27e052188525 in_seq=0 out_seq=0 2024-09-19T07:03:03.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.971+0000 7f2176ffd640 10 client.20208 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:03:03.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.971+0000 7f2176ffd640 1 -- 172.21.15.103:0/249379401 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591c0a6dd70 con 0x5591c0b86b20 2024-09-19T07:03:03.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.971+0000 7f2176ffd640 10 client.20208 handle_command_reply: tid=0 2024-09-19T07:03:03.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.971+0000 7f21a0042740 1 client.20208 shutdown 2024-09-19T07:03:03.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.971+0000 7f21a0042740 1 -- 172.21.15.103:0/249379401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b005d0 msgr2=0x5591c0b13660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:03.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.971+0000 7f21a0042740 1 --2- 172.21.15.103:0/249379401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b005d0 0x5591c0b13660 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x5591c0afdac0 tx=0x7f21880105a0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.971+0000 7f21a0042740 1 -- 172.21.15.103:0/249379401 shutdown_connections 2024-09-19T07:03:03.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.971+0000 7f21a0042740 1 --2- 172.21.15.103:0/249379401 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5591c0b86b20 0x5591c0b88f40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.971+0000 7f21a0042740 1 --2- 172.21.15.103:0/249379401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591c0b0b370 0x5591c0b10450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f21a0042740 1 --2- 172.21.15.103:0/249379401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591c0b01b40 0x5591c0b13ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f21a0042740 1 --2- 172.21.15.103:0/249379401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591c0b005d0 0x5591c0b13660 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f21a0042740 1 -- 172.21.15.103:0/249379401 >> 172.21.15.103:0/249379401 conn(0x5591c0ac9310 msgr2=0x5591c0afb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:03.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f2176ffd640 0 client.20208 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:03:03.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f21a0042740 1 -- 172.21.15.103:0/249379401 shutdown_connections 2024-09-19T07:03:03.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f21a0042740 1 -- 172.21.15.103:0/249379401 wait complete. 2024-09-19T07:03:03.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f21a0042740 20 client.20208 trim_cache size 0 max 16384 2024-09-19T07:03:03.979 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:03:03.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f219dc69640 1 -- 172.21.15.103:0/823128077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21840a3430 msgr2=0x7f21840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:03.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f219dc69640 1 --2- 172.21.15.103:0/823128077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21840a3430 0x7f21840a58f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2190009430 tx=0x7f219003f040 comp rx=0 tx=0).stop 2024-09-19T07:03:03.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f219dc69640 1 -- 172.21.15.103:0/823128077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2198101420 msgr2=0x7f21981997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:03.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f219dc69640 1 --2- 172.21.15.103:0/823128077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2198101420 0x7f21981997c0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f218000c890 tx=0x7f218000cd50 comp rx=0 tx=0).stop 2024-09-19T07:03:03.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f219dc69640 1 -- 172.21.15.103:0/823128077 shutdown_connections 2024-09-19T07:03:03.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f219dc69640 1 --2- 172.21.15.103:0/823128077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21840a3430 0x7f21840a58f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f219dc69640 1 --2- 172.21.15.103:0/823128077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2198106a80 0x7f21981a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f219dc69640 1 --2- 172.21.15.103:0/823128077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2198101420 0x7f21981997c0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f219dc69640 1 --2- 172.21.15.103:0/823128077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198100a80 0x7f2198199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:03.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f219dc69640 1 -- 172.21.15.103:0/823128077 >> 172.21.15.103:0/823128077 conn(0x7f21980fc7a0 msgr2=0x7f21980fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:03.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f219dc69640 1 -- 172.21.15.103:0/823128077 shutdown_connections 2024-09-19T07:03:03.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:03.972+0000 7f219dc69640 1 -- 172.21.15.103:0/823128077 wait complete. 2024-09-19T07:03:04.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:03 smithi134 ceph-mon[35356]: pgmap v431: 97 pgs: 97 active+clean; 3.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-19T07:03:04.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:03 smithi174 ceph-mon[35637]: pgmap v431: 97 pgs: 97 active+clean; 3.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-19T07:03:04.706 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:03:04.956 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:05.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.386+0000 7f59a880e640 1 -- 172.21.15.103:0/3094358667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59a4101420 msgr2=0x7f59a410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.386+0000 7f59a880e640 1 --2- 172.21.15.103:0/3094358667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59a4101420 0x7f59a410d950 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f599c009cd0 tx=0x7f599c02a8e0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.386+0000 7f59a880e640 1 -- 172.21.15.103:0/3094358667 shutdown_connections 2024-09-19T07:03:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.386+0000 7f59a880e640 1 --2- 172.21.15.103:0/3094358667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59a4101420 0x7f59a410d950 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.386+0000 7f59a880e640 1 --2- 172.21.15.103:0/3094358667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59a4100a80 0x7f59a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.386+0000 7f59a880e640 1 --2- 172.21.15.103:0/3094358667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59a4106a80 0x7f59a4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.386+0000 7f59a880e640 1 -- 172.21.15.103:0/3094358667 >> 172.21.15.103:0/3094358667 conn(0x7f59a40fc7a0 msgr2=0x7f59a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.386+0000 7f59a880e640 1 -- 172.21.15.103:0/3094358667 shutdown_connections 2024-09-19T07:03:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.386+0000 7f59a880e640 1 -- 172.21.15.103:0/3094358667 wait complete. 2024-09-19T07:03:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.387+0000 7f59a880e640 1 Processor -- start 2024-09-19T07:03:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.387+0000 7f59a880e640 1 -- start start 2024-09-19T07:03:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.387+0000 7f59a880e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59a4100a80 0x7f59a4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.387+0000 7f59a880e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59a4101420 0x7f59a4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.387+0000 7f59a880e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59a4106a80 0x7f59a419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.387+0000 7f59a880e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4110e80 con 0x7f59a4101420 2024-09-19T07:03:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.387+0000 7f59a880e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4110d00 con 0x7f59a4100a80 2024-09-19T07:03:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f59a880e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4111000 con 0x7f59a4106a80 2024-09-19T07:03:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f599bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59a4101420 0x7f59a4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f599bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59a4101420 0x7f59a4198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35078/0 (socket says 172.21.15.103:35078) 2024-09-19T07:03:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f599bfff640 1 -- 172.21.15.103:0/2692665295 learned_addr learned my addr 172.21.15.103:0/2692665295 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f59a259c640 1 --2- 172.21.15.103:0/2692665295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59a4100a80 0x7f59a4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f59a2d9d640 1 --2- 172.21.15.103:0/2692665295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59a4106a80 0x7f59a419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f599bfff640 1 -- 172.21.15.103:0/2692665295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59a4106a80 msgr2=0x7f59a419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f599bfff640 1 --2- 172.21.15.103:0/2692665295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59a4106a80 0x7f59a419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f599bfff640 1 -- 172.21.15.103:0/2692665295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59a4100a80 msgr2=0x7f59a4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f599bfff640 1 --2- 172.21.15.103:0/2692665295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59a4100a80 0x7f59a4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f599bfff640 1 -- 172.21.15.103:0/2692665295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a419d3e0 con 0x7f59a4101420 2024-09-19T07:03:05.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f59a2d9d640 1 --2- 172.21.15.103:0/2692665295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59a4106a80 0x7f59a419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:05.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f599bfff640 1 --2- 172.21.15.103:0/2692665295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59a4101420 0x7f59a4198970 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f598c00c920 tx=0x7f598c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:05.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f599b7fe640 1 -- 172.21.15.103:0/2692665295 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c007c70 con 0x7f59a4101420 2024-09-19T07:03:05.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f599b7fe640 1 -- 172.21.15.103:0/2692665295 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f598c007e10 con 0x7f59a4101420 2024-09-19T07:03:05.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f59a880e640 1 -- 172.21.15.103:0/2692665295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a419d6d0 con 0x7f59a4101420 2024-09-19T07:03:05.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f599b7fe640 1 -- 172.21.15.103:0/2692665295 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c018890 con 0x7f59a4101420 2024-09-19T07:03:05.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.388+0000 7f59a880e640 1 -- 172.21.15.103:0/2692665295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a41a4fb0 con 0x7f59a4101420 2024-09-19T07:03:05.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.391+0000 7f599b7fe640 1 -- 172.21.15.103:0/2692665295 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f598c018a70 con 0x7f59a4101420 2024-09-19T07:03:05.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.391+0000 7f59aabe7740 1 Processor -- start 2024-09-19T07:03:05.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.391+0000 7f59aabe7740 1 -- start start 2024-09-19T07:03:05.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.391+0000 7f59aabe7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.391+0000 7f59aabe7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613ad5d0 0x5634613ada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.391+0000 7f59aabe7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613b8370 0x5634613ba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.391+0000 7f59aabe7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5634613132f0 con 0x5634613aeb40 2024-09-19T07:03:05.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.391+0000 7f59aabe7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5634612bacf0 con 0x5634613ad5d0 2024-09-19T07:03:05.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.391+0000 7f59aabe7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56346130f2b0 con 0x5634613b8370 2024-09-19T07:03:05.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f59a2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f59a2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35088/0 (socket says 172.21.15.103:35088) 2024-09-19T07:03:05.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f59a2d9d640 1 -- 172.21.15.103:0/3742374747 learned_addr learned my addr 172.21.15.103:0/3742374747 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:05.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f59a259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613ad5d0 0x5634613ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f599b7fe640 1 --2- 172.21.15.103:0/2692665295 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59900a33b0 0x7f59900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f599bfff640 1 --2- 172.21.15.103:0/3742374747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613b8370 0x5634613ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f59a2d9d640 1 --2- 172.21.15.103:0/2692665295 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59900a33b0 0x7f59900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f59a259c640 1 -- 172.21.15.103:0/3742374747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613b8370 msgr2=0x5634613ba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:05.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f59a259c640 1 --2- 172.21.15.103:0/3742374747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613b8370 0x5634613ba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f59a259c640 1 -- 172.21.15.103:0/3742374747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 msgr2=0x5634613ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:05.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f59a259c640 1 --2- 172.21.15.103:0/3742374747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f59a259c640 1 -- 172.21.15.103:0/3742374747 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56346129f3a0 con 0x5634613ad5d0 2024-09-19T07:03:05.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f599b7fe640 1 -- 172.21.15.103:0/2692665295 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f598c0cfec0 con 0x7f59a4101420 2024-09-19T07:03:05.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f59a2d9d640 1 --2- 172.21.15.103:0/3742374747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:03:05.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f59a259c640 1 --2- 172.21.15.103:0/3742374747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613ad5d0 0x5634613ada10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f599400aac0 tx=0x7f599400af80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8e8107b9387b73d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:05.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f59a2d9d640 1 --2- 172.21.15.103:0/2692665295 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59900a33b0 0x7f59900a5870 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f599c035000 tx=0x7f599c02a8e0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:05.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.392+0000 7f5998ff9640 1 -- 172.21.15.103:0/3742374747 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994004020 con 0x5634613ad5d0 2024-09-19T07:03:05.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.393+0000 7f5998ff9640 1 -- 172.21.15.103:0/3742374747 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59940041c0 con 0x5634613ad5d0 2024-09-19T07:03:05.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.393+0000 7f5998ff9640 1 -- 172.21.15.103:0/3742374747 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59940187c0 con 0x5634613ad5d0 2024-09-19T07:03:05.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.393+0000 7f59aabe7740 1 -- 172.21.15.103:0/3742374747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613ad5d0 msgr2=0x5634613ada10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:05.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.393+0000 7f59aabe7740 1 --2- 172.21.15.103:0/3742374747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613ad5d0 0x5634613ada10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f599400aac0 tx=0x7f599400af80 comp rx=0 tx=0).stop 2024-09-19T07:03:05.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.393+0000 7f59aabe7740 1 -- 172.21.15.103:0/3742374747 shutdown_connections 2024-09-19T07:03:05.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.393+0000 7f59aabe7740 1 --2- 172.21.15.103:0/3742374747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613b8370 0x5634613ba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.393+0000 7f59aabe7740 1 --2- 172.21.15.103:0/3742374747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613ad5d0 0x5634613ada10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.393+0000 7f59aabe7740 1 --2- 172.21.15.103:0/3742374747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.393+0000 7f59aabe7740 1 -- 172.21.15.103:0/3742374747 >> 172.21.15.103:0/3742374747 conn(0x563461376310 msgr2=0x5634613a9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:05.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.393+0000 7f59aabe7740 1 -- 172.21.15.103:0/3742374747 shutdown_connections 2024-09-19T07:03:05.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.393+0000 7f59aabe7740 1 -- 172.21.15.103:0/3742374747 wait complete. 2024-09-19T07:03:05.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f59aabe7740 1 -- start start 2024-09-19T07:03:05.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f59aabe7740 1 Processor -- start 2024-09-19T07:03:05.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f59aabe7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613ad5d0 0x5634613b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f59aabe7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f59aabe7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613b8370 0x5634613b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f59aabe7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5634613bb070 con 0x5634613aeb40 2024-09-19T07:03:05.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f59aabe7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5634612ce060 con 0x5634613ad5d0 2024-09-19T07:03:05.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f59aabe7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5634613bb1f0 con 0x5634613b8370 2024-09-19T07:03:05.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f59aabe7740 10 client.0 _ll_register_callbacks cb 0x5634610218c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:05.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f599bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f599bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613b4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35100/0 (socket says 172.21.15.103:35100) 2024-09-19T07:03:05.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f599bfff640 1 -- 172.21.15.103:0/937695262 learned_addr learned my addr 172.21.15.103:0/937695262 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:05.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f59a259c640 1 --2- 172.21.15.103:0/937695262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613ad5d0 0x5634613b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f59a2d9d640 1 --2- 172.21.15.103:0/937695262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613b8370 0x5634613b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f599bfff640 1 -- 172.21.15.103:0/937695262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613b8370 msgr2=0x5634613b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:05.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f599bfff640 1 --2- 172.21.15.103:0/937695262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613b8370 0x5634613b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f599bfff640 1 -- 172.21.15.103:0/937695262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613ad5d0 msgr2=0x5634613b7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:05.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f599bfff640 1 --2- 172.21.15.103:0/937695262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613ad5d0 0x5634613b7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.394+0000 7f599bfff640 1 -- 172.21.15.103:0/937695262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634610843c0 con 0x5634613aeb40 2024-09-19T07:03:05.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.395+0000 7f599bfff640 1 --2- 172.21.15.103:0/937695262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613b4fb0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f598c007590 tx=0x7f598c09a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1cba4e89a7776f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:05.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.395+0000 7f5989ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:05.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.395+0000 7f5989ffb640 1 -- 172.21.15.103:0/937695262 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c0a0500 con 0x5634613aeb40 2024-09-19T07:03:05.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.395+0000 7f5989ffb640 1 -- 172.21.15.103:0/937695262 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f598c0a0170 con 0x5634613aeb40 2024-09-19T07:03:05.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.395+0000 7f59aabe7740 1 -- 172.21.15.103:0/937695262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59940041c0 con 0x5634613aeb40 2024-09-19T07:03:05.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.395+0000 7f5989ffb640 1 -- 172.21.15.103:0/937695262 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f598c0a4dc0 con 0x5634613aeb40 2024-09-19T07:03:05.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.395+0000 7f59aabe7740 10 client.20220 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:05.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.395+0000 7f59aabe7740 1 -- 172.21.15.103:0/937695262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5634613c6b00 con 0x5634613aeb40 2024-09-19T07:03:05.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.395+0000 7f5989ffb640 1 -- 172.21.15.103:0/937695262 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59940041c0 con 0x5634613aeb40 2024-09-19T07:03:05.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.395+0000 7f5989ffb640 1 -- 172.21.15.103:0/937695262 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f598c0a4930 con 0x5634613aeb40 2024-09-19T07:03:05.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.395+0000 7f59aabe7740 10 client.20220 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:05.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.395+0000 7f59aabe7740 10 client.20220 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:03:05.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.395+0000 7f59aabe7740 20 client.20220 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:05.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.395+0000 7f59aabe7740 1 --2- 172.21.15.103:0/937695262 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x5634613c7f10 0x5634613ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.395+0000 7f59aabe7740 4 client.20220 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:05.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.395+0000 7f59aabe7740 1 -- 172.21.15.103:0/937695262 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563461320860 con 0x5634613c7f10 2024-09-19T07:03:05.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.396+0000 7f59a259c640 1 --2- 172.21.15.103:0/937695262 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x5634613c7f10 0x5634613ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.396+0000 7f5989ffb640 10 client.20220 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:03:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.396+0000 7f59a259c640 1 --2- 172.21.15.103:0/937695262 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x5634613c7f10 0x5634613ca300 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=455ffb5a2b5f03c6 server_cookie=6991905e00f589b4 in_seq=0 out_seq=0 2024-09-19T07:03:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.397+0000 7f5989ffb640 1 -- 172.21.15.103:0/937695262 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563461320860 con 0x5634613c7f10 2024-09-19T07:03:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.397+0000 7f5989ffb640 10 client.20220 handle_command_reply: tid=0 2024-09-19T07:03:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.397+0000 7f59aabe7740 1 client.20220 shutdown 2024-09-19T07:03:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.397+0000 7f59aabe7740 1 -- 172.21.15.103:0/937695262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 msgr2=0x5634613b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:05.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.397+0000 7f59aabe7740 1 --2- 172.21.15.103:0/937695262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613b4fb0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f598c007590 tx=0x7f598c09a660 comp rx=0 tx=0).stop 2024-09-19T07:03:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.397+0000 7f59aabe7740 1 -- 172.21.15.103:0/937695262 shutdown_connections 2024-09-19T07:03:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.397+0000 7f59aabe7740 1 --2- 172.21.15.103:0/937695262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613b8370 0x5634613b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.397+0000 7f59aabe7740 1 --2- 172.21.15.103:0/937695262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613b4fb0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.397+0000 7f59aabe7740 1 --2- 172.21.15.103:0/937695262 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x5634613c7f10 0x5634613ca300 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.397+0000 7f59aabe7740 1 --2- 172.21.15.103:0/937695262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613ad5d0 0x5634613b7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.397+0000 7f5989ffb640 0 client.20220 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:03:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.397+0000 7f59aabe7740 1 -- 172.21.15.103:0/937695262 >> 172.21.15.103:0/937695262 conn(0x563461376310 msgr2=0x5634613af960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.397+0000 7f59aabe7740 1 -- 172.21.15.103:0/937695262 shutdown_connections 2024-09-19T07:03:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.397+0000 7f59aabe7740 1 -- 172.21.15.103:0/937695262 wait complete. 2024-09-19T07:03:05.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.397+0000 7f59aabe7740 20 client.20220 trim_cache size 0 max 16384 2024-09-19T07:03:05.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f59aabe7740 1 Processor -- start 2024-09-19T07:03:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f59aabe7740 1 -- start start 2024-09-19T07:03:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f59aabe7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613ad5d0 0x5634613c4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f59aabe7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613ae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f59aabe7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613b8370 0x5634613a9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f59aabe7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41ad8c0 con 0x5634613aeb40 2024-09-19T07:03:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f59aabe7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41af400 con 0x5634613b8370 2024-09-19T07:03:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f59aabe7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41adf90 con 0x5634613ad5d0 2024-09-19T07:03:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f599bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f599bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613ae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35106/0 (socket says 172.21.15.103:35106) 2024-09-19T07:03:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f599bfff640 1 -- 172.21.15.103:0/1688436453 learned_addr learned my addr 172.21.15.103:0/1688436453 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f59a259c640 1 --2- 172.21.15.103:0/1688436453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613ad5d0 0x5634613c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f59a2d9d640 1 --2- 172.21.15.103:0/1688436453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613b8370 0x5634613a9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f599bfff640 1 -- 172.21.15.103:0/1688436453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613ad5d0 msgr2=0x5634613c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f599bfff640 1 --2- 172.21.15.103:0/1688436453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613ad5d0 0x5634613c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f599bfff640 1 -- 172.21.15.103:0/1688436453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613b8370 msgr2=0x5634613a9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f599bfff640 1 --2- 172.21.15.103:0/1688436453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613b8370 0x5634613a9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f599bfff640 1 -- 172.21.15.103:0/1688436453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634613b0120 con 0x5634613aeb40 2024-09-19T07:03:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.410+0000 7f59a259c640 1 --2- 172.21.15.103:0/1688436453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613ad5d0 0x5634613c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.411+0000 7f599bfff640 1 --2- 172.21.15.103:0/1688436453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613ae300 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f598c098b20 tx=0x7f598c00ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950bb0d610e10449 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.411+0000 7f598bfff640 1 -- 172.21.15.103:0/1688436453 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c0a0500 con 0x5634613aeb40 2024-09-19T07:03:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.411+0000 7f598bfff640 1 -- 172.21.15.103:0/1688436453 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f598c09c150 con 0x5634613aeb40 2024-09-19T07:03:05.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.411+0000 7f598bfff640 1 -- 172.21.15.103:0/1688436453 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c0a4930 con 0x5634613aeb40 2024-09-19T07:03:05.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.411+0000 7f59aabe7740 1 -- 172.21.15.103:0/1688436453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 msgr2=0x5634613ae300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:05.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.411+0000 7f59aabe7740 1 --2- 172.21.15.103:0/1688436453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613ae300 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f598c098b20 tx=0x7f598c00ff30 comp rx=0 tx=0).stop 2024-09-19T07:03:05.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.411+0000 7f59aabe7740 1 -- 172.21.15.103:0/1688436453 shutdown_connections 2024-09-19T07:03:05.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.411+0000 7f59aabe7740 1 --2- 172.21.15.103:0/1688436453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613b8370 0x5634613a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.411+0000 7f59aabe7740 1 --2- 172.21.15.103:0/1688436453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613aeb40 0x5634613ae300 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.411+0000 7f59aabe7740 1 --2- 172.21.15.103:0/1688436453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613ad5d0 0x5634613c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.411+0000 7f59aabe7740 1 -- 172.21.15.103:0/1688436453 >> 172.21.15.103:0/1688436453 conn(0x563461376310 msgr2=0x5634613a4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:05.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.411+0000 7f59aabe7740 1 -- 172.21.15.103:0/1688436453 shutdown_connections 2024-09-19T07:03:05.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.411+0000 7f59aabe7740 1 -- 172.21.15.103:0/1688436453 wait complete. 2024-09-19T07:03:05.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59aabe7740 1 -- start start 2024-09-19T07:03:05.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59aabe7740 1 Processor -- start 2024-09-19T07:03:05.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59aabe7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613ad5d0 0x5634613c0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59aabe7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613aeb40 0x5634613b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59aabe7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613b8370 0x5634613b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59aabe7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41aed30 con 0x5634613ad5d0 2024-09-19T07:03:05.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59aabe7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41ae660 con 0x5634613b8370 2024-09-19T07:03:05.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59aabe7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41ab8c0 con 0x5634613aeb40 2024-09-19T07:03:05.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59aabe7740 10 client.0 _ll_register_callbacks cb 0x7f598c09bd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:05.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59a259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613ad5d0 0x5634613c0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59a259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613ad5d0 0x5634613c0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35116/0 (socket says 172.21.15.103:35116) 2024-09-19T07:03:05.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59a259c640 1 -- 172.21.15.103:0/1066074377 learned_addr learned my addr 172.21.15.103:0/1066074377 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:05.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f599bfff640 1 --2- 172.21.15.103:0/1066074377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613aeb40 0x5634613b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59a259c640 1 -- 172.21.15.103:0/1066074377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613aeb40 msgr2=0x5634613b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:05.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59a259c640 1 --2- 172.21.15.103:0/1066074377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613aeb40 0x5634613b1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59a259c640 1 -- 172.21.15.103:0/1066074377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613b8370 msgr2=0x5634613b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:05.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59a259c640 1 --2- 172.21.15.103:0/1066074377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613b8370 0x5634613b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f59a259c640 1 -- 172.21.15.103:0/1066074377 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634613b2b10 con 0x5634613ad5d0 2024-09-19T07:03:05.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.412+0000 7f599bfff640 1 --2- 172.21.15.103:0/1066074377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613aeb40 0x5634613b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:05.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.413+0000 7f59a259c640 1 --2- 172.21.15.103:0/1066074377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613ad5d0 0x5634613c0c60 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f599400aa10 tx=0x7f5994005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57641475798ea6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:05.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.413+0000 7f5989ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:05.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.413+0000 7f5989ffb640 1 -- 172.21.15.103:0/1066074377 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599402f070 con 0x5634613ad5d0 2024-09-19T07:03:05.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.413+0000 7f5989ffb640 1 -- 172.21.15.103:0/1066074377 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f59940043d0 con 0x5634613ad5d0 2024-09-19T07:03:05.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.413+0000 7f59aabe7740 1 -- 172.21.15.103:0/1066074377 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f598c09c150 con 0x5634613ad5d0 2024-09-19T07:03:05.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.413+0000 7f5989ffb640 1 -- 172.21.15.103:0/1066074377 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f599403b050 con 0x5634613ad5d0 2024-09-19T07:03:05.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.413+0000 7f59aabe7740 10 client.20226 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:05.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.413+0000 7f59aabe7740 1 -- 172.21.15.103:0/1066074377 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56346143b3f0 con 0x5634613ad5d0 2024-09-19T07:03:05.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.413+0000 7f5989ffb640 1 -- 172.21.15.103:0/1066074377 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f598c09c150 con 0x5634613ad5d0 2024-09-19T07:03:05.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.413+0000 7f5989ffb640 1 -- 172.21.15.103:0/1066074377 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5994003890 con 0x5634613ad5d0 2024-09-19T07:03:05.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.413+0000 7f59aabe7740 10 client.20226 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:05.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.413+0000 7f59aabe7740 10 client.20226 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:03:05.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.413+0000 7f59aabe7740 20 client.20226 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:05.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.413+0000 7f59aabe7740 1 --2- 172.21.15.103:0/1066074377 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56346143b890 0x56346143dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:05.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.413+0000 7f59aabe7740 4 client.20226 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:03:05.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.413+0000 7f59aabe7740 1 -- 172.21.15.103:0/1066074377 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56346131ad70 con 0x56346143b890 2024-09-19T07:03:05.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f599bfff640 1 --2- 172.21.15.103:0/1066074377 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56346143b890 0x56346143dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:05.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f5989ffb640 10 client.20226 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:03:05.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f599bfff640 1 --2- 172.21.15.103:0/1066074377 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56346143b890 0x56346143dc80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a64e30b815a499d server_cookie=be2460ba94067749 in_seq=0 out_seq=0 2024-09-19T07:03:05.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f5989ffb640 1 -- 172.21.15.103:0/1066074377 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56346131ad70 con 0x56346143b890 2024-09-19T07:03:05.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f5989ffb640 10 client.20226 handle_command_reply: tid=0 2024-09-19T07:03:05.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f59aabe7740 1 client.20226 shutdown 2024-09-19T07:03:05.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f59aabe7740 1 -- 172.21.15.103:0/1066074377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613ad5d0 msgr2=0x5634613c0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:05.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f59aabe7740 1 --2- 172.21.15.103:0/1066074377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613ad5d0 0x5634613c0c60 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f599400aa10 tx=0x7f5994005da0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f59aabe7740 1 -- 172.21.15.103:0/1066074377 shutdown_connections 2024-09-19T07:03:05.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f59aabe7740 1 --2- 172.21.15.103:0/1066074377 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x56346143b890 0x56346143dc80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f59aabe7740 1 --2- 172.21.15.103:0/1066074377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634613b8370 0x5634613b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f59aabe7740 1 --2- 172.21.15.103:0/1066074377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634613aeb40 0x5634613b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f59aabe7740 1 --2- 172.21.15.103:0/1066074377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634613ad5d0 0x5634613c0c60 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f59aabe7740 1 -- 172.21.15.103:0/1066074377 >> 172.21.15.103:0/1066074377 conn(0x563461376310 msgr2=0x5634613a9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:05.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f5989ffb640 0 client.20226 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:03:05.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f59aabe7740 1 -- 172.21.15.103:0/1066074377 shutdown_connections 2024-09-19T07:03:05.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.414+0000 7f59aabe7740 1 -- 172.21.15.103:0/1066074377 wait complete. 2024-09-19T07:03:05.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.415+0000 7f59aabe7740 20 client.20226 trim_cache size 0 max 16384 2024-09-19T07:03:05.422 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:03:05.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.415+0000 7f59a880e640 1 -- 172.21.15.103:0/2692665295 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59900a33b0 msgr2=0x7f59900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:05.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.415+0000 7f59a880e640 1 --2- 172.21.15.103:0/2692665295 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59900a33b0 0x7f59900a5870 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f599c035000 tx=0x7f599c02a8e0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.415+0000 7f59a880e640 1 -- 172.21.15.103:0/2692665295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59a4101420 msgr2=0x7f59a4198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:05.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.415+0000 7f59a880e640 1 --2- 172.21.15.103:0/2692665295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59a4101420 0x7f59a4198970 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f598c00c920 tx=0x7f598c00cde0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.415+0000 7f59a880e640 1 -- 172.21.15.103:0/2692665295 shutdown_connections 2024-09-19T07:03:05.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.415+0000 7f59a880e640 1 --2- 172.21.15.103:0/2692665295 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59900a33b0 0x7f59900a5870 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.415+0000 7f59a880e640 1 --2- 172.21.15.103:0/2692665295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59a4106a80 0x7f59a419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.415+0000 7f59a880e640 1 --2- 172.21.15.103:0/2692665295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59a4101420 0x7f59a4198970 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.415+0000 7f59a880e640 1 --2- 172.21.15.103:0/2692665295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59a4100a80 0x7f59a4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:05.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.415+0000 7f59a880e640 1 -- 172.21.15.103:0/2692665295 >> 172.21.15.103:0/2692665295 conn(0x7f59a40fc7a0 msgr2=0x7f59a406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:05.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.415+0000 7f59a880e640 1 -- 172.21.15.103:0/2692665295 shutdown_connections 2024-09-19T07:03:05.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:05.415+0000 7f59a880e640 1 -- 172.21.15.103:0/2692665295 wait complete. 2024-09-19T07:03:06.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:05 smithi103 ceph-mon[31158]: pgmap v432: 97 pgs: 97 active+clean; 3.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-19T07:03:06.036 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:05 smithi103 ceph-mon[31158]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T07:03:06.064 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:03:06.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:05 smithi134 ceph-mon[35356]: pgmap v432: 97 pgs: 97 active+clean; 3.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-19T07:03:06.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:05 smithi134 ceph-mon[35356]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T07:03:06.310 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:06.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:05 smithi174 ceph-mon[35637]: pgmap v432: 97 pgs: 97 active+clean; 3.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-19T07:03:06.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:05 smithi174 ceph-mon[35637]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T07:03:06.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.779+0000 7f1a06bed640 1 -- 172.21.15.103:0/1792259749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a00100e20 msgr2=0x7f1a00101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:06.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.779+0000 7f1a06bed640 1 --2- 172.21.15.103:0/1792259749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a00100e20 0x7f1a00101200 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009900 tx=0x7f19f402a480 comp rx=0 tx=0).stop 2024-09-19T07:03:06.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.779+0000 7f1a06bed640 1 -- 172.21.15.103:0/1792259749 shutdown_connections 2024-09-19T07:03:06.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.779+0000 7f1a06bed640 1 --2- 172.21.15.103:0/1792259749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a00102ef0 0x7f1a0010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.779+0000 7f1a06bed640 1 --2- 172.21.15.103:0/1792259749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a00102570 0x7f1a001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.779+0000 7f1a06bed640 1 --2- 172.21.15.103:0/1792259749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a00100e20 0x7f1a00101200 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.779+0000 7f1a06bed640 1 -- 172.21.15.103:0/1792259749 >> 172.21.15.103:0/1792259749 conn(0x7f1a000fc7d0 msgr2=0x7f1a000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:06.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.779+0000 7f1a06bed640 1 -- 172.21.15.103:0/1792259749 shutdown_connections 2024-09-19T07:03:06.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.779+0000 7f1a06bed640 1 -- 172.21.15.103:0/1792259749 wait complete. 2024-09-19T07:03:06.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.779+0000 7f1a06bed640 1 Processor -- start 2024-09-19T07:03:06.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.779+0000 7f1a06bed640 1 -- start start 2024-09-19T07:03:06.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f1a06bed640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a00100e20 0x7f1a00198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f1a06bed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a00102570 0x7f1a00198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f1a06bed640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a00102ef0 0x7f1a0019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f1a06bed640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001120a0 con 0x7f1a00102570 2024-09-19T07:03:06.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f1a06bed640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00111f20 con 0x7f1a00102ef0 2024-09-19T07:03:06.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f1a06bed640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00112220 con 0x7f1a00100e20 2024-09-19T07:03:06.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f19fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a00102570 0x7f1a00198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:06.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f19fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a00102570 0x7f1a00198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35138/0 (socket says 172.21.15.103:35138) 2024-09-19T07:03:06.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f19fffff640 1 -- 172.21.15.103:0/1241095385 learned_addr learned my addr 172.21.15.103:0/1241095385 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:06.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f1a0518a640 1 --2- 172.21.15.103:0/1241095385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a00102ef0 0x7f1a0019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:06.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f1a04989640 1 --2- 172.21.15.103:0/1241095385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a00100e20 0x7f1a00198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:06.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f19fffff640 1 -- 172.21.15.103:0/1241095385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a00100e20 msgr2=0x7f1a00198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:06.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f19fffff640 1 --2- 172.21.15.103:0/1241095385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a00100e20 0x7f1a00198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f19fffff640 1 -- 172.21.15.103:0/1241095385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a00102ef0 msgr2=0x7f1a0019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:06.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f19fffff640 1 --2- 172.21.15.103:0/1241095385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a00102ef0 0x7f1a0019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f19fffff640 1 -- 172.21.15.103:0/1241095385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0019d4b0 con 0x7f1a00102570 2024-09-19T07:03:06.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f19fffff640 1 --2- 172.21.15.103:0/1241095385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a00102570 0x7f1a00198a40 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00c890 tx=0x7f19ec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:06.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f19fdffb640 1 -- 172.21.15.103:0/1241095385 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec004220 con 0x7f1a00102570 2024-09-19T07:03:06.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f1a06bed640 1 -- 172.21.15.103:0/1241095385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0019d7a0 con 0x7f1a00102570 2024-09-19T07:03:06.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f19fdffb640 1 -- 172.21.15.103:0/1241095385 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19ec0043c0 con 0x7f1a00102570 2024-09-19T07:03:06.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.780+0000 7f19fdffb640 1 -- 172.21.15.103:0/1241095385 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0049e0 con 0x7f1a00102570 2024-09-19T07:03:06.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.781+0000 7f1a06bed640 1 -- 172.21.15.103:0/1241095385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a001a5080 con 0x7f1a00102570 2024-09-19T07:03:06.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.783+0000 7f19fdffb640 1 -- 172.21.15.103:0/1241095385 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f19ec0079b0 con 0x7f1a00102570 2024-09-19T07:03:06.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.783+0000 7f1a08fc6740 1 Processor -- start 2024-09-19T07:03:06.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.783+0000 7f1a08fc6740 1 -- start start 2024-09-19T07:03:06.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.783+0000 7f19fdffb640 1 --2- 172.21.15.103:0/1241095385 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19f00a33b0 0x7f19f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.783+0000 7f1a08fc6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a713b40 0x557a4a712090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.783+0000 7f1a08fc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a7125d0 0x557a4a712a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.783+0000 7f1a08fc6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 0x557a4a723b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.783+0000 7f1a08fc6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557a4a6782f0 con 0x557a4a713b40 2024-09-19T07:03:06.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.783+0000 7f1a08fc6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557a4a61fcf0 con 0x557a4a721780 2024-09-19T07:03:06.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.783+0000 7f1a08fc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557a4a6742b0 con 0x557a4a7125d0 2024-09-19T07:03:06.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.783+0000 7f1a0518a640 1 --2- 172.21.15.103:0/1241095385 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19f00a33b0 0x7f19f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:06.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.783+0000 7f1a04989640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a713b40 0x557a4a712090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:06.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.783+0000 7f19fdffb640 1 -- 172.21.15.103:0/1241095385 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f19ec0cfdd0 con 0x7f1a00102570 2024-09-19T07:03:06.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.783+0000 7f1a04989640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a713b40 0x557a4a712090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35148/0 (socket says 172.21.15.103:35148) 2024-09-19T07:03:06.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.783+0000 7f1a04989640 1 -- 172.21.15.103:0/2436844544 learned_addr learned my addr 172.21.15.103:0/2436844544 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:06.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.784+0000 7f19fffff640 1 --2- 172.21.15.103:0/2436844544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a7125d0 0x557a4a712a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:06.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.784+0000 7f1a0518a640 1 --2- 172.21.15.103:0/2436844544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 0x557a4a723b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:06.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.784+0000 7f1a04989640 1 -- 172.21.15.103:0/2436844544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a7125d0 msgr2=0x557a4a712a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:06.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.784+0000 7f1a04989640 1 --2- 172.21.15.103:0/2436844544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a7125d0 0x557a4a712a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.784+0000 7f1a04989640 1 -- 172.21.15.103:0/2436844544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 msgr2=0x557a4a723b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.784+0000 7f1a04989640 1 --2- 172.21.15.103:0/2436844544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 0x557a4a723b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.784+0000 7f1a04989640 1 -- 172.21.15.103:0/2436844544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4a6043a0 con 0x557a4a713b40 2024-09-19T07:03:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.784+0000 7f1a0518a640 1 --2- 172.21.15.103:0/1241095385 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19f00a33b0 0x7f19f00a5870 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f19f8005fd0 tx=0x7f19f8014bf0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.784+0000 7f1a04989640 1 --2- 172.21.15.103:0/2436844544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a713b40 0x557a4a712090 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f19f40314e0 tx=0x7f19f4004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ccd4d8b220ebcb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.784+0000 7f19eaffd640 1 -- 172.21.15.103:0/2436844544 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4035030 con 0x557a4a713b40 2024-09-19T07:03:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.784+0000 7f19eaffd640 1 -- 172.21.15.103:0/2436844544 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f4005520 con 0x557a4a713b40 2024-09-19T07:03:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.784+0000 7f19eaffd640 1 -- 172.21.15.103:0/2436844544 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4004e60 con 0x557a4a713b40 2024-09-19T07:03:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.785+0000 7f1a08fc6740 1 -- 172.21.15.103:0/2436844544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a713b40 msgr2=0x557a4a712090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.785+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/2436844544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a713b40 0x557a4a712090 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f19f40314e0 tx=0x7f19f4004170 comp rx=0 tx=0).stop 2024-09-19T07:03:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.785+0000 7f1a08fc6740 1 -- 172.21.15.103:0/2436844544 shutdown_connections 2024-09-19T07:03:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.785+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/2436844544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 0x557a4a723b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.785+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/2436844544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a7125d0 0x557a4a712a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.785+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/2436844544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a713b40 0x557a4a712090 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.785+0000 7f1a08fc6740 1 -- 172.21.15.103:0/2436844544 >> 172.21.15.103:0/2436844544 conn(0x557a4a6db310 msgr2=0x557a4a70ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.785+0000 7f1a08fc6740 1 -- 172.21.15.103:0/2436844544 shutdown_connections 2024-09-19T07:03:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.785+0000 7f1a08fc6740 1 -- 172.21.15.103:0/2436844544 wait complete. 2024-09-19T07:03:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.785+0000 7f1a08fc6740 1 -- start start 2024-09-19T07:03:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.785+0000 7f1a08fc6740 1 Processor -- start 2024-09-19T07:03:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a08fc6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a7125d0 0x557a4a721130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a08fc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a713b40 0x557a4a71e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a08fc6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 0x557a4a71e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a08fc6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557a4a724420 con 0x557a4a7125d0 2024-09-19T07:03:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a08fc6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557a4a633060 con 0x557a4a721780 2024-09-19T07:03:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a08fc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557a4a7245a0 con 0x557a4a713b40 2024-09-19T07:03:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a08fc6740 10 client.0 _ll_register_callbacks cb 0x557a4a3868c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a04989640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a7125d0 0x557a4a721130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a04989640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a7125d0 0x557a4a721130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35152/0 (socket says 172.21.15.103:35152) 2024-09-19T07:03:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a04989640 1 -- 172.21.15.103:0/2968090564 learned_addr learned my addr 172.21.15.103:0/2968090564 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f19fffff640 1 --2- 172.21.15.103:0/2968090564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a713b40 0x557a4a71e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a04989640 1 -- 172.21.15.103:0/2968090564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a713b40 msgr2=0x557a4a71e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a04989640 1 --2- 172.21.15.103:0/2968090564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a713b40 0x557a4a71e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a04989640 1 -- 172.21.15.103:0/2968090564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 msgr2=0x557a4a71e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a04989640 1 --2- 172.21.15.103:0/2968090564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 0x557a4a71e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a04989640 1 -- 172.21.15.103:0/2968090564 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a4a3e93c0 con 0x557a4a7125d0 2024-09-19T07:03:06.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a04989640 1 --2- 172.21.15.103:0/2968090564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a7125d0 0x557a4a721130 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f19f402a4c0 tx=0x7f19f4005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519695c1242a6713 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:06.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f19dffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:06.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f19dffff640 1 -- 172.21.15.103:0/2968090564 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4035030 con 0x557a4a7125d0 2024-09-19T07:03:06.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f19dffff640 1 -- 172.21.15.103:0/2968090564 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f19f4005050 con 0x557a4a7125d0 2024-09-19T07:03:06.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.786+0000 7f1a08fc6740 1 -- 172.21.15.103:0/2968090564 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f4005520 con 0x557a4a7125d0 2024-09-19T07:03:06.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.787+0000 7f19dffff640 1 -- 172.21.15.103:0/2968090564 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f19f4005250 con 0x557a4a7125d0 2024-09-19T07:03:06.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.787+0000 7f1a08fc6740 10 client.20238 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:06.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.787+0000 7f1a08fc6740 1 -- 172.21.15.103:0/2968090564 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557a4a72fe70 con 0x557a4a7125d0 2024-09-19T07:03:06.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.787+0000 7f19dffff640 1 -- 172.21.15.103:0/2968090564 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f4005520 con 0x557a4a7125d0 2024-09-19T07:03:06.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.787+0000 7f19dffff640 1 -- 172.21.15.103:0/2968090564 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f19f40317b0 con 0x557a4a7125d0 2024-09-19T07:03:06.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.787+0000 7f1a08fc6740 10 client.20238 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:06.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.787+0000 7f1a08fc6740 10 client.20238 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:03:06.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.787+0000 7f1a08fc6740 20 client.20238 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:06.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.787+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/2968090564 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x557a4a7311c0 0x557a4a7335b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.787+0000 7f1a08fc6740 4 client.20238 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:06.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.787+0000 7f1a08fc6740 1 -- 172.21.15.103:0/2968090564 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a4a685860 con 0x557a4a7311c0 2024-09-19T07:03:06.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.787+0000 7f19fffff640 1 --2- 172.21.15.103:0/2968090564 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x557a4a7311c0 0x557a4a7335b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:06.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.788+0000 7f19fffff640 1 --2- 172.21.15.103:0/2968090564 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x557a4a7311c0 0x557a4a7335b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e243d8db1f987a88 server_cookie=5fefdc1c78f99ebe in_seq=0 out_seq=0 2024-09-19T07:03:06.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.788+0000 7f19dffff640 10 client.20238 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:03:06.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.789+0000 7f19dffff640 1 -- 172.21.15.103:0/2968090564 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a4a685860 con 0x557a4a7311c0 2024-09-19T07:03:06.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.789+0000 7f19dffff640 10 client.20238 handle_command_reply: tid=0 2024-09-19T07:03:06.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.789+0000 7f1a08fc6740 1 client.20238 shutdown 2024-09-19T07:03:06.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.789+0000 7f1a08fc6740 1 -- 172.21.15.103:0/2968090564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a7125d0 msgr2=0x557a4a721130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:06.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.789+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/2968090564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a7125d0 0x557a4a721130 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f19f402a4c0 tx=0x7f19f4005730 comp rx=0 tx=0).stop 2024-09-19T07:03:06.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.789+0000 7f1a08fc6740 1 -- 172.21.15.103:0/2968090564 shutdown_connections 2024-09-19T07:03:06.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.789+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/2968090564 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x557a4a7311c0 0x557a4a7335b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.789+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/2968090564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 0x557a4a71e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.789+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/2968090564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a713b40 0x557a4a71e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.789+0000 7f19dffff640 0 client.20238 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:03:06.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.789+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/2968090564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a7125d0 0x557a4a721130 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.789+0000 7f1a08fc6740 1 -- 172.21.15.103:0/2968090564 >> 172.21.15.103:0/2968090564 conn(0x557a4a6db310 msgr2=0x557a4a714960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:06.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.789+0000 7f1a08fc6740 1 -- 172.21.15.103:0/2968090564 shutdown_connections 2024-09-19T07:03:06.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.789+0000 7f1a08fc6740 1 -- 172.21.15.103:0/2968090564 wait complete. 2024-09-19T07:03:06.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.789+0000 7f1a08fc6740 20 client.20238 trim_cache size 0 max 16384 2024-09-19T07:03:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.802+0000 7f1a08fc6740 1 Processor -- start 2024-09-19T07:03:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.802+0000 7f1a08fc6740 1 -- start start 2024-09-19T07:03:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.802+0000 7f1a08fc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a7125d0 0x557a4a72d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.802+0000 7f1a08fc6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a713b40 0x557a4a71d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.802+0000 7f1a08fc6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 0x557a4a738bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.802+0000 7f1a08fc6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001ab6e0 con 0x557a4a713b40 2024-09-19T07:03:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.802+0000 7f1a08fc6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001ad220 con 0x557a4a721780 2024-09-19T07:03:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.802+0000 7f1a08fc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001abdb0 con 0x557a4a7125d0 2024-09-19T07:03:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.802+0000 7f19fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a713b40 0x557a4a71d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.802+0000 7f19fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a713b40 0x557a4a71d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35154/0 (socket says 172.21.15.103:35154) 2024-09-19T07:03:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.802+0000 7f19fffff640 1 -- 172.21.15.103:0/309034402 learned_addr learned my addr 172.21.15.103:0/309034402 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.802+0000 7f19fffff640 1 -- 172.21.15.103:0/309034402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a7125d0 msgr2=0x557a4a72d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.802+0000 7f19fffff640 1 --2- 172.21.15.103:0/309034402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a7125d0 0x557a4a72d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.802+0000 7f19fffff640 1 -- 172.21.15.103:0/309034402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 msgr2=0x557a4a738bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.802+0000 7f19fffff640 1 --2- 172.21.15.103:0/309034402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 0x557a4a738bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.802+0000 7f19fffff640 1 -- 172.21.15.103:0/309034402 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4a70f960 con 0x557a4a713b40 2024-09-19T07:03:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.803+0000 7f19fffff640 1 --2- 172.21.15.103:0/309034402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a713b40 0x557a4a71d3c0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f19ec016000 tx=0x7f19ec005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207e91bfa92bc47c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.803+0000 7f19ea7fc640 1 -- 172.21.15.103:0/309034402 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec01d040 con 0x557a4a713b40 2024-09-19T07:03:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.803+0000 7f19ea7fc640 1 -- 172.21.15.103:0/309034402 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19ec09c060 con 0x557a4a713b40 2024-09-19T07:03:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.803+0000 7f19ea7fc640 1 -- 172.21.15.103:0/309034402 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec09a3f0 con 0x557a4a713b40 2024-09-19T07:03:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.803+0000 7f1a08fc6740 1 -- 172.21.15.103:0/309034402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a713b40 msgr2=0x557a4a71d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.803+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/309034402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a713b40 0x557a4a71d3c0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f19ec016000 tx=0x7f19ec005300 comp rx=0 tx=0).stop 2024-09-19T07:03:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.803+0000 7f1a08fc6740 1 -- 172.21.15.103:0/309034402 shutdown_connections 2024-09-19T07:03:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.803+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/309034402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 0x557a4a738bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.803+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/309034402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a713b40 0x557a4a71d3c0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.803+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/309034402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a7125d0 0x557a4a72d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.803+0000 7f1a08fc6740 1 -- 172.21.15.103:0/309034402 >> 172.21.15.103:0/309034402 conn(0x557a4a6db310 msgr2=0x557a4a71ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.803+0000 7f1a08fc6740 1 -- 172.21.15.103:0/309034402 shutdown_connections 2024-09-19T07:03:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.803+0000 7f1a08fc6740 1 -- 172.21.15.103:0/309034402 wait complete. 2024-09-19T07:03:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.804+0000 7f1a08fc6740 1 -- start start 2024-09-19T07:03:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.804+0000 7f1a08fc6740 1 Processor -- start 2024-09-19T07:03:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.804+0000 7f1a08fc6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a7125d0 0x557a4a729f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.804+0000 7f1a08fc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a713b40 0x557a4a7252e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.804+0000 7f1a08fc6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 0x557a4a725820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.804+0000 7f1a08fc6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001acb50 con 0x557a4a7125d0 2024-09-19T07:03:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.804+0000 7f1a08fc6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001ac480 con 0x557a4a721780 2024-09-19T07:03:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.804+0000 7f1a08fc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001a9700 con 0x557a4a713b40 2024-09-19T07:03:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.804+0000 7f1a08fc6740 10 client.0 _ll_register_callbacks cb 0x7f19ec0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:06.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.804+0000 7f1a04989640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a7125d0 0x557a4a729f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:06.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.804+0000 7f1a04989640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a7125d0 0x557a4a729f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35164/0 (socket says 172.21.15.103:35164) 2024-09-19T07:03:06.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.804+0000 7f1a04989640 1 -- 172.21.15.103:0/294106692 learned_addr learned my addr 172.21.15.103:0/294106692 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:06.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.804+0000 7f1a0518a640 1 --2- 172.21.15.103:0/294106692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 0x557a4a725820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:06.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.805+0000 7f19fffff640 1 --2- 172.21.15.103:0/294106692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a713b40 0x557a4a7252e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:06.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.805+0000 7f1a04989640 1 -- 172.21.15.103:0/294106692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a713b40 msgr2=0x557a4a7252e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:06.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.805+0000 7f1a04989640 1 --2- 172.21.15.103:0/294106692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a713b40 0x557a4a7252e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.805+0000 7f1a04989640 1 -- 172.21.15.103:0/294106692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 msgr2=0x557a4a725820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:06.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.805+0000 7f1a04989640 1 --2- 172.21.15.103:0/294106692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 0x557a4a725820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.805+0000 7f1a04989640 1 -- 172.21.15.103:0/294106692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a4a726050 con 0x557a4a7125d0 2024-09-19T07:03:06.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.805+0000 7f1a0518a640 1 --2- 172.21.15.103:0/294106692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 0x557a4a725820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:06.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.805+0000 7f1a04989640 1 --2- 172.21.15.103:0/294106692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a7125d0 0x557a4a729f60 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f19f402a9a0 tx=0x7f19f40049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7e9cda84dfe7dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:06.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.805+0000 7f19dffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:06.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.805+0000 7f19dffff640 1 -- 172.21.15.103:0/294106692 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4059030 con 0x557a4a7125d0 2024-09-19T07:03:06.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.805+0000 7f19dffff640 1 -- 172.21.15.103:0/294106692 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f19f4049040 con 0x557a4a7125d0 2024-09-19T07:03:06.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.805+0000 7f1a08fc6740 1 -- 172.21.15.103:0/294106692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19ec09c060 con 0x557a4a7125d0 2024-09-19T07:03:06.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.805+0000 7f19dffff640 1 -- 172.21.15.103:0/294106692 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f19f4035070 con 0x557a4a7125d0 2024-09-19T07:03:06.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.805+0000 7f1a08fc6740 10 client.20244 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:06.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.805+0000 7f1a08fc6740 1 -- 172.21.15.103:0/294106692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557a4a79bb70 con 0x557a4a7125d0 2024-09-19T07:03:06.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.805+0000 7f19dffff640 1 -- 172.21.15.103:0/294106692 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19ec09c060 con 0x557a4a7125d0 2024-09-19T07:03:06.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.806+0000 7f19dffff640 1 -- 172.21.15.103:0/294106692 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f19f4003bc0 con 0x557a4a7125d0 2024-09-19T07:03:06.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.806+0000 7f1a08fc6740 10 client.20244 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:06.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.806+0000 7f1a08fc6740 10 client.20244 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:03:06.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.806+0000 7f1a08fc6740 20 client.20244 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:06.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.806+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/294106692 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x557a4a79c290 0x557a4a79e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:06.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.806+0000 7f1a08fc6740 4 client.20244 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:03:06.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.806+0000 7f1a08fc6740 1 -- 172.21.15.103:0/294106692 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a4a67fd70 con 0x557a4a79c290 2024-09-19T07:03:06.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.806+0000 7f19fffff640 1 --2- 172.21.15.103:0/294106692 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x557a4a79c290 0x557a4a79e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:06.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.806+0000 7f19fffff640 1 --2- 172.21.15.103:0/294106692 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x557a4a79c290 0x557a4a79e6b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce5ff04c3fb5be25 server_cookie=8c81236732744c70 in_seq=0 out_seq=0 2024-09-19T07:03:06.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.806+0000 7f19dffff640 10 client.20244 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:03:06.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.806+0000 7f19dffff640 1 -- 172.21.15.103:0/294106692 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a4a67fd70 con 0x557a4a79c290 2024-09-19T07:03:06.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.806+0000 7f19dffff640 10 client.20244 handle_command_reply: tid=0 2024-09-19T07:03:06.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.806+0000 7f1a08fc6740 1 client.20244 shutdown 2024-09-19T07:03:06.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a08fc6740 1 -- 172.21.15.103:0/294106692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a7125d0 msgr2=0x557a4a729f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:06.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/294106692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a7125d0 0x557a4a729f60 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f19f402a9a0 tx=0x7f19f40049c0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a08fc6740 1 -- 172.21.15.103:0/294106692 shutdown_connections 2024-09-19T07:03:06.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/294106692 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x557a4a79c290 0x557a4a79e6b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/294106692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a4a721780 0x557a4a725820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/294106692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4a713b40 0x557a4a7252e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f19dffff640 0 client.20244 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:03:06.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a08fc6740 1 --2- 172.21.15.103:0/294106692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4a7125d0 0x557a4a729f60 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a08fc6740 1 -- 172.21.15.103:0/294106692 >> 172.21.15.103:0/294106692 conn(0x557a4a6db310 msgr2=0x557a4a71d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:06.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a08fc6740 1 -- 172.21.15.103:0/294106692 shutdown_connections 2024-09-19T07:03:06.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a08fc6740 1 -- 172.21.15.103:0/294106692 wait complete. 2024-09-19T07:03:06.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a08fc6740 20 client.20244 trim_cache size 0 max 16384 2024-09-19T07:03:06.813 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:03:06.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a06bed640 1 -- 172.21.15.103:0/1241095385 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19f00a33b0 msgr2=0x7f19f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:06.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a06bed640 1 --2- 172.21.15.103:0/1241095385 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19f00a33b0 0x7f19f00a5870 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f19f8005fd0 tx=0x7f19f8014bf0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a06bed640 1 -- 172.21.15.103:0/1241095385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a00102570 msgr2=0x7f1a00198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:06.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a06bed640 1 --2- 172.21.15.103:0/1241095385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a00102570 0x7f1a00198a40 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00c890 tx=0x7f19ec00cd50 comp rx=0 tx=0).stop 2024-09-19T07:03:06.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a06bed640 1 -- 172.21.15.103:0/1241095385 shutdown_connections 2024-09-19T07:03:06.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a06bed640 1 --2- 172.21.15.103:0/1241095385 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19f00a33b0 0x7f19f00a5870 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a06bed640 1 --2- 172.21.15.103:0/1241095385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a00102ef0 0x7f1a0019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a06bed640 1 --2- 172.21.15.103:0/1241095385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a00102570 0x7f1a00198a40 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a06bed640 1 --2- 172.21.15.103:0/1241095385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a00100e20 0x7f1a00198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:06.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.807+0000 7f1a06bed640 1 -- 172.21.15.103:0/1241095385 >> 172.21.15.103:0/1241095385 conn(0x7f1a000fc7d0 msgr2=0x7f1a001068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:06.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.808+0000 7f1a06bed640 1 -- 172.21.15.103:0/1241095385 shutdown_connections 2024-09-19T07:03:06.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:06.808+0000 7f1a06bed640 1 -- 172.21.15.103:0/1241095385 wait complete. 2024-09-19T07:03:07.573 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:03:07.819 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:08.094 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:07 smithi103 ceph-mon[31158]: pgmap v434: 97 pgs: 97 active+clean; 3.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 96 KiB/s wr, 38 op/s 2024-09-19T07:03:08.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:07 smithi134 ceph-mon[35356]: pgmap v434: 97 pgs: 97 active+clean; 3.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 96 KiB/s wr, 38 op/s 2024-09-19T07:03:08.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.289+0000 7f4ff0c1d640 1 -- 172.21.15.103:0/1902378100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fec106a80 msgr2=0x7f4fec106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:08.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.289+0000 7f4ff0c1d640 1 --2- 172.21.15.103:0/1902378100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fec106a80 0x7f4fec106e60 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009930 tx=0x7f4fdc02a420 comp rx=0 tx=0).stop 2024-09-19T07:03:08.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.289+0000 7f4ff0c1d640 1 -- 172.21.15.103:0/1902378100 shutdown_connections 2024-09-19T07:03:08.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.289+0000 7f4ff0c1d640 1 --2- 172.21.15.103:0/1902378100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fec101420 0x7f4fec10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.289+0000 7f4ff0c1d640 1 --2- 172.21.15.103:0/1902378100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fec100a80 0x7f4fec100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.289+0000 7f4ff0c1d640 1 --2- 172.21.15.103:0/1902378100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fec106a80 0x7f4fec106e60 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.289+0000 7f4ff0c1d640 1 -- 172.21.15.103:0/1902378100 >> 172.21.15.103:0/1902378100 conn(0x7f4fec0fc7a0 msgr2=0x7f4fec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:08.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.289+0000 7f4ff0c1d640 1 -- 172.21.15.103:0/1902378100 shutdown_connections 2024-09-19T07:03:08.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.289+0000 7f4ff0c1d640 1 -- 172.21.15.103:0/1902378100 wait complete. 2024-09-19T07:03:08.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.290+0000 7f4ff0c1d640 1 Processor -- start 2024-09-19T07:03:08.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.290+0000 7f4ff0c1d640 1 -- start start 2024-09-19T07:03:08.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.290+0000 7f4ff0c1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fec100a80 0x7f4fec198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.290+0000 7f4ff0c1d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fec101420 0x7f4fec198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.290+0000 7f4ff0c1d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fec106a80 0x7f4fec19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.290+0000 7f4ff0c1d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec110e80 con 0x7f4fec100a80 2024-09-19T07:03:08.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.290+0000 7f4ff0c1d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec110d00 con 0x7f4fec106a80 2024-09-19T07:03:08.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.290+0000 7f4ff0c1d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec111000 con 0x7f4fec101420 2024-09-19T07:03:08.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.290+0000 7f4fea59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fec100a80 0x7f4fec198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.290+0000 7f4fea59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fec100a80 0x7f4fec198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38044/0 (socket says 172.21.15.103:38044) 2024-09-19T07:03:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.290+0000 7f4fea59c640 1 -- 172.21.15.103:0/2988588743 learned_addr learned my addr 172.21.15.103:0/2988588743 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.291+0000 7f4fe3fff640 1 --2- 172.21.15.103:0/2988588743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fec101420 0x7f4fec198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.291+0000 7f4fead9d640 1 --2- 172.21.15.103:0/2988588743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fec106a80 0x7f4fec19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.291+0000 7f4fea59c640 1 -- 172.21.15.103:0/2988588743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fec101420 msgr2=0x7f4fec198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.291+0000 7f4fea59c640 1 --2- 172.21.15.103:0/2988588743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fec101420 0x7f4fec198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.291+0000 7f4fea59c640 1 -- 172.21.15.103:0/2988588743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fec106a80 msgr2=0x7f4fec19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.291+0000 7f4fea59c640 1 --2- 172.21.15.103:0/2988588743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fec106a80 0x7f4fec19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.291+0000 7f4fea59c640 1 -- 172.21.15.103:0/2988588743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fec19d3f0 con 0x7f4fec100a80 2024-09-19T07:03:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.291+0000 7f4fead9d640 1 --2- 172.21.15.103:0/2988588743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fec106a80 0x7f4fec19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.291+0000 7f4fea59c640 1 --2- 172.21.15.103:0/2988588743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fec100a80 0x7f4fec198440 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009230 tx=0x7f4fdc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.291+0000 7f4fe37fe640 1 -- 172.21.15.103:0/2988588743 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc03f030 con 0x7f4fec100a80 2024-09-19T07:03:08.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.291+0000 7f4fe37fe640 1 -- 172.21.15.103:0/2988588743 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fdc003ad0 con 0x7f4fec100a80 2024-09-19T07:03:08.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.291+0000 7f4ff0c1d640 1 -- 172.21.15.103:0/2988588743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fec19d680 con 0x7f4fec100a80 2024-09-19T07:03:08.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.291+0000 7f4fe37fe640 1 -- 172.21.15.103:0/2988588743 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc035070 con 0x7f4fec100a80 2024-09-19T07:03:08.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.291+0000 7f4ff0c1d640 1 -- 172.21.15.103:0/2988588743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fec1a4fc0 con 0x7f4fec100a80 2024-09-19T07:03:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.293+0000 7f4fe37fe640 1 -- 172.21.15.103:0/2988588743 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4fdc042030 con 0x7f4fec100a80 2024-09-19T07:03:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.294+0000 7f4ff2ff6740 1 Processor -- start 2024-09-19T07:03:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.294+0000 7f4ff2ff6740 1 -- start start 2024-09-19T07:03:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.294+0000 7f4ff2ff6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56083b40 0x564e56082090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.294+0000 7f4ff2ff6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e560825d0 0x564e56082a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.294+0000 7f4ff2ff6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e56091780 0x564e56093b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.294+0000 7f4ff2ff6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564e55fe82f0 con 0x564e56083b40 2024-09-19T07:03:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.294+0000 7f4ff2ff6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564e55f8fcf0 con 0x564e560825d0 2024-09-19T07:03:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.294+0000 7f4ff2ff6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564e55fe42b0 con 0x564e56091780 2024-09-19T07:03:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.294+0000 7f4fe37fe640 1 --2- 172.21.15.103:0/2988588743 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fd80a33b0 0x7f4fd80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.294+0000 7f4fead9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56083b40 0x564e56082090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.294+0000 7f4fead9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56083b40 0x564e56082090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38048/0 (socket says 172.21.15.103:38048) 2024-09-19T07:03:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.294+0000 7f4fead9d640 1 -- 172.21.15.103:0/1813091214 learned_addr learned my addr 172.21.15.103:0/1813091214 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.294+0000 7f4fea59c640 1 --2- 172.21.15.103:0/1813091214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e560825d0 0x564e56082a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.295+0000 7f4fe3fff640 1 --2- 172.21.15.103:0/1813091214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e56091780 0x564e56093b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.295+0000 7f4fe37fe640 1 -- 172.21.15.103:0/2988588743 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f4fdc0f02d0 con 0x7f4fec100a80 2024-09-19T07:03:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.295+0000 7f4fead9d640 1 --2- 172.21.15.103:0/2988588743 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fd80a33b0 0x7f4fd80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.295+0000 7f4fead9d640 1 -- 172.21.15.103:0/1813091214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e56091780 msgr2=0x564e56093b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.295+0000 7f4fead9d640 1 --2- 172.21.15.103:0/1813091214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e56091780 0x564e56093b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.295+0000 7f4fead9d640 1 -- 172.21.15.103:0/1813091214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e560825d0 msgr2=0x564e56082a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.295+0000 7f4fead9d640 1 --2- 172.21.15.103:0/1813091214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e560825d0 0x564e56082a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.295+0000 7f4fead9d640 1 -- 172.21.15.103:0/1813091214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e55f743a0 con 0x564e56083b40 2024-09-19T07:03:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.295+0000 7f4fea59c640 1 --2- 172.21.15.103:0/1813091214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e560825d0 0x564e56082a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.295+0000 7f4fead9d640 1 --2- 172.21.15.103:0/1813091214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56083b40 0x564e56082090 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4014a10 tx=0x7f4fe4014ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab29374f916df3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.295+0000 7f4fe0ff9640 1 -- 172.21.15.103:0/1813091214 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4014250 con 0x564e56083b40 2024-09-19T07:03:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.295+0000 7f4fead9d640 1 --2- 172.21.15.103:0/2988588743 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fd80a33b0 0x7f4fd80a5870 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4022000 tx=0x7f4fe4020000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.295+0000 7f4fe0ff9640 1 -- 172.21.15.103:0/1813091214 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe4023070 con 0x564e56083b40 2024-09-19T07:03:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.295+0000 7f4fe0ff9640 1 -- 172.21.15.103:0/1813091214 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4013b30 con 0x564e56083b40 2024-09-19T07:03:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.296+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1813091214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56083b40 msgr2=0x564e56082090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.296+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/1813091214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56083b40 0x564e56082090 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4014a10 tx=0x7f4fe4014ed0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.296+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1813091214 shutdown_connections 2024-09-19T07:03:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.296+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/1813091214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e56091780 0x564e56093b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.296+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/1813091214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e560825d0 0x564e56082a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.296+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/1813091214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56083b40 0x564e56082090 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.296+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1813091214 >> 172.21.15.103:0/1813091214 conn(0x564e5604b310 msgr2=0x564e5607ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.296+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1813091214 shutdown_connections 2024-09-19T07:03:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.296+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1813091214 wait complete. 2024-09-19T07:03:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.296+0000 7f4ff2ff6740 1 -- start start 2024-09-19T07:03:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.296+0000 7f4ff2ff6740 1 Processor -- start 2024-09-19T07:03:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4ff2ff6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e560825d0 0x564e560911b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4ff2ff6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56083b40 0x564e5608e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4ff2ff6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e56091780 0x564e5608ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4ff2ff6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564e56094480 con 0x564e56083b40 2024-09-19T07:03:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4ff2ff6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564e55fa3060 con 0x564e56091780 2024-09-19T07:03:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4ff2ff6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564e56094600 con 0x564e560825d0 2024-09-19T07:03:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4ff2ff6740 10 client.0 _ll_register_callbacks cb 0x564e55cf68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:08.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4fe3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56083b40 0x564e5608e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4fe3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56083b40 0x564e5608e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38056/0 (socket says 172.21.15.103:38056) 2024-09-19T07:03:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4fe3fff640 1 -- 172.21.15.103:0/1559693480 learned_addr learned my addr 172.21.15.103:0/1559693480 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4fea59c640 1 --2- 172.21.15.103:0/1559693480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e560825d0 0x564e560911b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4fead9d640 1 --2- 172.21.15.103:0/1559693480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e56091780 0x564e5608ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4fe3fff640 1 -- 172.21.15.103:0/1559693480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e560825d0 msgr2=0x564e560911b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:08.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4fe3fff640 1 --2- 172.21.15.103:0/1559693480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e560825d0 0x564e560911b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4fe3fff640 1 -- 172.21.15.103:0/1559693480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e56091780 msgr2=0x564e5608ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4fe3fff640 1 --2- 172.21.15.103:0/1559693480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e56091780 0x564e5608ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4fe3fff640 1 -- 172.21.15.103:0/1559693480 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e55d593c0 con 0x564e56083b40 2024-09-19T07:03:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4fea59c640 1 --2- 172.21.15.103:0/1559693480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e560825d0 0x564e560911b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4fead9d640 1 --2- 172.21.15.103:0/1559693480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e56091780 0x564e5608ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4fe3fff640 1 --2- 172.21.15.103:0/1559693480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56083b40 0x564e5608e460 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400bad0 tx=0x7f4fd400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b807a2db1c6337 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4fd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.297+0000 7f4fd1ffb640 1 -- 172.21.15.103:0/1559693480 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4011440 con 0x564e56083b40 2024-09-19T07:03:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.298+0000 7f4fd1ffb640 1 -- 172.21.15.103:0/1559693480 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f4fd40115e0 con 0x564e56083b40 2024-09-19T07:03:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.298+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1559693480 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe4023070 con 0x564e56083b40 2024-09-19T07:03:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.298+0000 7f4fd1ffb640 1 -- 172.21.15.103:0/1559693480 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4fd40117e0 con 0x564e56083b40 2024-09-19T07:03:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.298+0000 7f4ff2ff6740 10 client.20256 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:08.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.298+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1559693480 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e5609ffc0 con 0x564e56083b40 2024-09-19T07:03:08.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.298+0000 7f4fd1ffb640 1 -- 172.21.15.103:0/1559693480 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe4023070 con 0x564e56083b40 2024-09-19T07:03:08.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.298+0000 7f4fd1ffb640 1 -- 172.21.15.103:0/1559693480 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4fd401ba50 con 0x564e56083b40 2024-09-19T07:03:08.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.298+0000 7f4ff2ff6740 10 client.20256 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:08.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.298+0000 7f4ff2ff6740 10 client.20256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:08.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.298+0000 7f4ff2ff6740 20 client.20256 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:08.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.298+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/1559693480 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564e560a1520 0x564e560a3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.298+0000 7f4ff2ff6740 4 client.20256 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:08.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.298+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1559693480 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e55ff5860 con 0x564e560a1520 2024-09-19T07:03:08.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.299+0000 7f4fea59c640 1 --2- 172.21.15.103:0/1559693480 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564e560a1520 0x564e560a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.299+0000 7f4fd1ffb640 10 client.20256 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:08.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.299+0000 7f4fea59c640 1 --2- 172.21.15.103:0/1559693480 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564e560a1520 0x564e560a3960 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8bfe3cfb90e66e3 server_cookie=1340f355cd4bae43 in_seq=0 out_seq=0 2024-09-19T07:03:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.300+0000 7f4fd1ffb640 1 -- 172.21.15.103:0/1559693480 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e55ff5860 con 0x564e560a1520 2024-09-19T07:03:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.300+0000 7f4fd1ffb640 10 client.20256 handle_command_reply: tid=0 2024-09-19T07:03:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.300+0000 7f4ff2ff6740 1 client.20256 shutdown 2024-09-19T07:03:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.300+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1559693480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56083b40 msgr2=0x564e5608e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.300+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/1559693480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56083b40 0x564e5608e460 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400bad0 tx=0x7f4fd400bf90 comp rx=0 tx=0).stop 2024-09-19T07:03:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.300+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1559693480 shutdown_connections 2024-09-19T07:03:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.300+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/1559693480 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564e560a1520 0x564e560a3960 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.300+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/1559693480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e56091780 0x564e5608ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.300+0000 7f4fd1ffb640 0 client.20256 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.300+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/1559693480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56083b40 0x564e5608e460 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.300+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/1559693480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e560825d0 0x564e560911b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.300+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1559693480 >> 172.21.15.103:0/1559693480 conn(0x564e5604b310 msgr2=0x564e56084930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.300+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1559693480 shutdown_connections 2024-09-19T07:03:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.300+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1559693480 wait complete. 2024-09-19T07:03:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.300+0000 7f4ff2ff6740 20 client.20256 trim_cache size 0 max 16384 2024-09-19T07:03:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4ff2ff6740 1 Processor -- start 2024-09-19T07:03:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4ff2ff6740 1 -- start start 2024-09-19T07:03:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4ff2ff6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e560825d0 0x564e5607ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4ff2ff6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e56083b40 0x564e560a9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4ff2ff6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56091780 0x564e560a9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4ff2ff6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1ab780 con 0x564e56091780 2024-09-19T07:03:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4ff2ff6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1ad2c0 con 0x564e56083b40 2024-09-19T07:03:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4ff2ff6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1abe50 con 0x564e560825d0 2024-09-19T07:03:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4fead9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56091780 0x564e560a9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4fead9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56091780 0x564e560a9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38060/0 (socket says 172.21.15.103:38060) 2024-09-19T07:03:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4fead9d640 1 -- 172.21.15.103:0/4229859828 learned_addr learned my addr 172.21.15.103:0/4229859828 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4fea59c640 1 --2- 172.21.15.103:0/4229859828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e560825d0 0x564e5607ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4fe3fff640 1 --2- 172.21.15.103:0/4229859828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e56083b40 0x564e560a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4fead9d640 1 -- 172.21.15.103:0/4229859828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e560825d0 msgr2=0x564e5607ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4fead9d640 1 --2- 172.21.15.103:0/4229859828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e560825d0 0x564e5607ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4fead9d640 1 -- 172.21.15.103:0/4229859828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e56083b40 msgr2=0x564e560a9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4fead9d640 1 --2- 172.21.15.103:0/4229859828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e56083b40 0x564e560a9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4fead9d640 1 -- 172.21.15.103:0/4229859828 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e5607d310 con 0x564e56091780 2024-09-19T07:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4fea59c640 1 --2- 172.21.15.103:0/4229859828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e560825d0 0x564e5607ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.313+0000 7f4fead9d640 1 --2- 172.21.15.103:0/4229859828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56091780 0x564e560a9770 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009a10 tx=0x7f4fe4009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65706619d6e747f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.314+0000 7f4fd3fff640 1 -- 172.21.15.103:0/4229859828 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe40120c0 con 0x564e56091780 2024-09-19T07:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.314+0000 7f4fd3fff640 1 -- 172.21.15.103:0/4229859828 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe4023030 con 0x564e56091780 2024-09-19T07:03:08.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.314+0000 7f4fd3fff640 1 -- 172.21.15.103:0/4229859828 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4013a20 con 0x564e56091780 2024-09-19T07:03:08.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.314+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/4229859828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56091780 msgr2=0x564e560a9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:08.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.314+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/4229859828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56091780 0x564e560a9770 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009a10 tx=0x7f4fe4009690 comp rx=0 tx=0).stop 2024-09-19T07:03:08.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.314+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/4229859828 shutdown_connections 2024-09-19T07:03:08.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.314+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/4229859828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e56091780 0x564e560a9770 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.314+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/4229859828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e56083b40 0x564e560a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.314+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/4229859828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e560825d0 0x564e5607ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.314+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/4229859828 >> 172.21.15.103:0/4229859828 conn(0x564e5604b310 msgr2=0x564e5607e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:08.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.314+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/4229859828 shutdown_connections 2024-09-19T07:03:08.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.314+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/4229859828 wait complete. 2024-09-19T07:03:08.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.314+0000 7f4ff2ff6740 1 -- start start 2024-09-19T07:03:08.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4ff2ff6740 1 Processor -- start 2024-09-19T07:03:08.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4ff2ff6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e560825d0 0x564e5609a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4ff2ff6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e56083b40 0x564e560953a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4ff2ff6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e56091780 0x564e56095900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4ff2ff6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1acbf0 con 0x564e560825d0 2024-09-19T07:03:08.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4ff2ff6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1ac520 con 0x564e56083b40 2024-09-19T07:03:08.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4ff2ff6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1a9780 con 0x564e56091780 2024-09-19T07:03:08.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4ff2ff6740 10 client.0 _ll_register_callbacks cb 0x7f4fd4004060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:08.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4fea59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e560825d0 0x564e5609a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4fea59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e560825d0 0x564e5609a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38068/0 (socket says 172.21.15.103:38068) 2024-09-19T07:03:08.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4fea59c640 1 -- 172.21.15.103:0/1855660571 learned_addr learned my addr 172.21.15.103:0/1855660571 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:08.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4fead9d640 1 --2- 172.21.15.103:0/1855660571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e56091780 0x564e56095900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4fea59c640 1 -- 172.21.15.103:0/1855660571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e56091780 msgr2=0x564e56095900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:08.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4fea59c640 1 --2- 172.21.15.103:0/1855660571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e56091780 0x564e56095900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4fe3fff640 1 --2- 172.21.15.103:0/1855660571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e56083b40 0x564e560953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4fea59c640 1 -- 172.21.15.103:0/1855660571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e56083b40 msgr2=0x564e560953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:08.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4fea59c640 1 --2- 172.21.15.103:0/1855660571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e56083b40 0x564e560953a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4fea59c640 1 -- 172.21.15.103:0/1855660571 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e56096130 con 0x564e560825d0 2024-09-19T07:03:08.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4fea59c640 1 --2- 172.21.15.103:0/1855660571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e560825d0 0x564e5609a070 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0bc200 tx=0x7f4fdc087390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7edcacf6184ded21 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:08.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.315+0000 7f4fd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:08.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.316+0000 7f4fd1ffb640 1 -- 172.21.15.103:0/1855660571 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0a5020 con 0x564e560825d0 2024-09-19T07:03:08.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.316+0000 7f4fd1ffb640 1 -- 172.21.15.103:0/1855660571 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f4fdc0b8b70 con 0x564e560825d0 2024-09-19T07:03:08.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.316+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1855660571 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe4023030 con 0x564e560825d0 2024-09-19T07:03:08.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.316+0000 7f4fd1ffb640 1 -- 172.21.15.103:0/1855660571 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4fdc0d5050 con 0x564e560825d0 2024-09-19T07:03:08.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.316+0000 7f4ff2ff6740 10 client.20262 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:08.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.316+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1855660571 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e5610c260 con 0x564e560825d0 2024-09-19T07:03:08.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.316+0000 7f4fd1ffb640 1 -- 172.21.15.103:0/1855660571 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe4023030 con 0x564e560825d0 2024-09-19T07:03:08.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.316+0000 7f4fd1ffb640 1 -- 172.21.15.103:0/1855660571 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4fdc0edbf0 con 0x564e560825d0 2024-09-19T07:03:08.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.316+0000 7f4ff2ff6740 10 client.20262 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:08.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.316+0000 7f4ff2ff6740 10 client.20262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:08.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.316+0000 7f4ff2ff6740 20 client.20262 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:08.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.316+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/1855660571 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564e560a1520 0x564e5610e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:08.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.316+0000 7f4ff2ff6740 4 client.20262 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:03:08.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.316+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1855660571 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e55fefd70 con 0x564e560a1520 2024-09-19T07:03:08.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.316+0000 7f4fe3fff640 1 --2- 172.21.15.103:0/1855660571 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564e560a1520 0x564e5610e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:08.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4fd1ffb640 10 client.20262 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:08.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4fe3fff640 1 --2- 172.21.15.103:0/1855660571 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564e560a1520 0x564e5610e940 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7568dc7b9924aee0 server_cookie=e6ee39a828616193 in_seq=0 out_seq=0 2024-09-19T07:03:08.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4fd1ffb640 1 -- 172.21.15.103:0/1855660571 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e55fefd70 con 0x564e560a1520 2024-09-19T07:03:08.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4fd1ffb640 10 client.20262 handle_command_reply: tid=0 2024-09-19T07:03:08.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4ff2ff6740 1 client.20262 shutdown 2024-09-19T07:03:08.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1855660571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e560825d0 msgr2=0x564e5609a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:08.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/1855660571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e560825d0 0x564e5609a070 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0bc200 tx=0x7f4fdc087390 comp rx=0 tx=0).stop 2024-09-19T07:03:08.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1855660571 shutdown_connections 2024-09-19T07:03:08.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/1855660571 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564e560a1520 0x564e5610e940 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/1855660571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e56091780 0x564e56095900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/1855660571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564e56083b40 0x564e560953a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4ff2ff6740 1 --2- 172.21.15.103:0/1855660571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e560825d0 0x564e5609a070 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1855660571 >> 172.21.15.103:0/1855660571 conn(0x564e5604b310 msgr2=0x564e5608fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:08.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4fd1ffb640 0 client.20262 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:08.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1855660571 shutdown_connections 2024-09-19T07:03:08.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4ff2ff6740 1 -- 172.21.15.103:0/1855660571 wait complete. 2024-09-19T07:03:08.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.317+0000 7f4ff2ff6740 20 client.20262 trim_cache size 0 max 16384 2024-09-19T07:03:08.325 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:03:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.318+0000 7f4ff0c1d640 1 -- 172.21.15.103:0/2988588743 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fd80a33b0 msgr2=0x7f4fd80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.318+0000 7f4ff0c1d640 1 --2- 172.21.15.103:0/2988588743 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fd80a33b0 0x7f4fd80a5870 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4022000 tx=0x7f4fe4020000 comp rx=0 tx=0).stop 2024-09-19T07:03:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.318+0000 7f4ff0c1d640 1 -- 172.21.15.103:0/2988588743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fec100a80 msgr2=0x7f4fec198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.318+0000 7f4ff0c1d640 1 --2- 172.21.15.103:0/2988588743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fec100a80 0x7f4fec198440 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009230 tx=0x7f4fdc004270 comp rx=0 tx=0).stop 2024-09-19T07:03:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.318+0000 7f4ff0c1d640 1 -- 172.21.15.103:0/2988588743 shutdown_connections 2024-09-19T07:03:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.318+0000 7f4ff0c1d640 1 --2- 172.21.15.103:0/2988588743 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4fd80a33b0 0x7f4fd80a5870 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.318+0000 7f4ff0c1d640 1 --2- 172.21.15.103:0/2988588743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fec106a80 0x7f4fec19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.318+0000 7f4ff0c1d640 1 --2- 172.21.15.103:0/2988588743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fec101420 0x7f4fec198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.318+0000 7f4ff0c1d640 1 --2- 172.21.15.103:0/2988588743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fec100a80 0x7f4fec198440 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.318+0000 7f4ff0c1d640 1 -- 172.21.15.103:0/2988588743 >> 172.21.15.103:0/2988588743 conn(0x7f4fec0fc7a0 msgr2=0x7f4fec06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:08.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.318+0000 7f4ff0c1d640 1 -- 172.21.15.103:0/2988588743 shutdown_connections 2024-09-19T07:03:08.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:08.318+0000 7f4ff0c1d640 1 -- 172.21.15.103:0/2988588743 wait complete. 2024-09-19T07:03:08.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:07 smithi174 ceph-mon[35637]: pgmap v434: 97 pgs: 97 active+clean; 3.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 96 KiB/s wr, 38 op/s 2024-09-19T07:03:10.041 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:03:10.042 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:03:10.069 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:09 smithi103 ceph-mon[31158]: pgmap v435: 97 pgs: 97 active+clean; 3.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 96 KiB/s wr, 38 op/s 2024-09-19T07:03:10.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:09 smithi134 ceph-mon[35356]: pgmap v435: 97 pgs: 97 active+clean; 3.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 96 KiB/s wr, 38 op/s 2024-09-19T07:03:10.304 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:10.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:09 smithi174 ceph-mon[35637]: pgmap v435: 97 pgs: 97 active+clean; 3.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 96 KiB/s wr, 38 op/s 2024-09-19T07:03:10.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.792+0000 7f178b2eb640 1 -- 172.21.15.103:0/2014728389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1784105590 msgr2=0x7f178410d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:10.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.792+0000 7f178b2eb640 1 --2- 172.21.15.103:0/2014728389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1784105590 0x7f178410d110 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f177c009ca0 tx=0x7f177c02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.792+0000 7f178b2eb640 1 -- 172.21.15.103:0/2014728389 shutdown_connections 2024-09-19T07:03:10.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.792+0000 7f178b2eb640 1 --2- 172.21.15.103:0/2014728389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1784105590 0x7f178410d110 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.792+0000 7f178b2eb640 1 --2- 172.21.15.103:0/2014728389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1784100e10 0x7f1784104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.792+0000 7f178b2eb640 1 --2- 172.21.15.103:0/2014728389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1784100460 0x7f1784100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.792+0000 7f178b2eb640 1 -- 172.21.15.103:0/2014728389 >> 172.21.15.103:0/2014728389 conn(0x7f17840fbee0 msgr2=0x7f17840fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:10.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.792+0000 7f178b2eb640 1 -- 172.21.15.103:0/2014728389 shutdown_connections 2024-09-19T07:03:10.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.792+0000 7f178b2eb640 1 -- 172.21.15.103:0/2014728389 wait complete. 2024-09-19T07:03:10.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.793+0000 7f178b2eb640 1 Processor -- start 2024-09-19T07:03:10.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.793+0000 7f178b2eb640 1 -- start start 2024-09-19T07:03:10.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.793+0000 7f178b2eb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1784100460 0x7f1784199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.793+0000 7f178b2eb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1784100e10 0x7f17841996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.793+0000 7f178b2eb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1784105590 0x7f17841a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.793+0000 7f178b2eb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17841105a0 con 0x7f1784100460 2024-09-19T07:03:10.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.793+0000 7f178b2eb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1784110420 con 0x7f1784105590 2024-09-19T07:03:10.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.793+0000 7f178b2eb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1784110720 con 0x7f1784100e10 2024-09-19T07:03:10.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.793+0000 7f1789087640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1784100460 0x7f1784199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:10.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.794+0000 7f1789087640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1784100460 0x7f1784199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38086/0 (socket says 172.21.15.103:38086) 2024-09-19T07:03:10.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.794+0000 7f1789087640 1 -- 172.21.15.103:0/1595248949 learned_addr learned my addr 172.21.15.103:0/1595248949 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:10.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.794+0000 7f1788886640 1 --2- 172.21.15.103:0/1595248949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1784100e10 0x7f17841996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:10.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.794+0000 7f1789888640 1 --2- 172.21.15.103:0/1595248949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1784105590 0x7f17841a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:10.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.794+0000 7f1789087640 1 -- 172.21.15.103:0/1595248949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1784100e10 msgr2=0x7f17841996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:10.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.794+0000 7f1789087640 1 --2- 172.21.15.103:0/1595248949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1784100e10 0x7f17841996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.794+0000 7f1789087640 1 -- 172.21.15.103:0/1595248949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1784105590 msgr2=0x7f17841a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:10.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.794+0000 7f1789087640 1 --2- 172.21.15.103:0/1595248949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1784105590 0x7f17841a4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.794+0000 7f1789087640 1 -- 172.21.15.103:0/1595248949 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17841a4740 con 0x7f1784100460 2024-09-19T07:03:10.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.794+0000 7f1789087640 1 --2- 172.21.15.103:0/1595248949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1784100460 0x7f1784199180 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f178000d950 tx=0x7f178000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:10.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.794+0000 7f177a7fc640 1 -- 172.21.15.103:0/1595248949 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780017070 con 0x7f1784100460 2024-09-19T07:03:10.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.794+0000 7f177a7fc640 1 -- 172.21.15.103:0/1595248949 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178000de80 con 0x7f1784100460 2024-09-19T07:03:10.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.794+0000 7f178b2eb640 1 -- 172.21.15.103:0/1595248949 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17841a4910 con 0x7f1784100460 2024-09-19T07:03:10.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.794+0000 7f177a7fc640 1 -- 172.21.15.103:0/1595248949 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178000f620 con 0x7f1784100460 2024-09-19T07:03:10.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.794+0000 7f178b2eb640 1 -- 172.21.15.103:0/1595248949 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17841a4e20 con 0x7f1784100460 2024-09-19T07:03:10.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.796+0000 7f177a7fc640 1 -- 172.21.15.103:0/1595248949 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1780009c30 con 0x7f1784100460 2024-09-19T07:03:10.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.797+0000 7f178d6c4740 1 Processor -- start 2024-09-19T07:03:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.797+0000 7f178d6c4740 1 -- start start 2024-09-19T07:03:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.797+0000 7f177a7fc640 1 --2- 172.21.15.103:0/1595248949 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17700a33b0 0x7f17700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.797+0000 7f178d6c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e67a40 0x556995e67e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.797+0000 7f178d6c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e68360 0x556995e76c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.797+0000 7f178d6c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e77150 0x556995e79540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.797+0000 7f178d6c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556995dcd280 con 0x556995e77150 2024-09-19T07:03:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.797+0000 7f178d6c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556995d74b10 con 0x556995e68360 2024-09-19T07:03:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.797+0000 7f178d6c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556995dc9240 con 0x556995e67a40 2024-09-19T07:03:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.797+0000 7f1789888640 1 --2- 172.21.15.103:0/1595248949 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17700a33b0 0x7f17700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.798+0000 7f177a7fc640 1 -- 172.21.15.103:0/1595248949 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f17800cf9d0 con 0x7f1784100460 2024-09-19T07:03:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.798+0000 7f1789888640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e77150 0x556995e79540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.798+0000 7f1789888640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e77150 0x556995e79540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38088/0 (socket says 172.21.15.103:38088) 2024-09-19T07:03:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.798+0000 7f1789888640 1 -- 172.21.15.103:0/4018791758 learned_addr learned my addr 172.21.15.103:0/4018791758 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:10.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.798+0000 7f1789087640 1 --2- 172.21.15.103:0/4018791758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e67a40 0x556995e67e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:10.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.798+0000 7f1788886640 1 --2- 172.21.15.103:0/4018791758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e68360 0x556995e76c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:10.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.798+0000 7f1789087640 1 -- 172.21.15.103:0/4018791758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e68360 msgr2=0x556995e76c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:10.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.798+0000 7f1789087640 1 --2- 172.21.15.103:0/4018791758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e68360 0x556995e76c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.798+0000 7f1789087640 1 -- 172.21.15.103:0/4018791758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e77150 msgr2=0x556995e79540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:10.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.798+0000 7f1789087640 1 --2- 172.21.15.103:0/4018791758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e77150 0x556995e79540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.798+0000 7f1789087640 1 -- 172.21.15.103:0/4018791758 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556995d811f0 con 0x556995e67a40 2024-09-19T07:03:10.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.798+0000 7f1789888640 1 --2- 172.21.15.103:0/1595248949 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17700a33b0 0x7f17700a5870 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f177c004af0 tx=0x7f177c02f500 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:10.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.798+0000 7f1789087640 1 --2- 172.21.15.103:0/4018791758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e67a40 0x556995e67e20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f17800cd070 tx=0x7f178009c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=612956ed3d70136a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:10.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.798+0000 7f176f7fe640 1 -- 172.21.15.103:0/4018791758 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780020070 con 0x556995e67a40 2024-09-19T07:03:10.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.798+0000 7f176f7fe640 1 -- 172.21.15.103:0/4018791758 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17800a4870 con 0x556995e67a40 2024-09-19T07:03:10.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.798+0000 7f176f7fe640 1 -- 172.21.15.103:0/4018791758 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17800860b0 con 0x556995e67a40 2024-09-19T07:03:10.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.799+0000 7f178d6c4740 1 -- 172.21.15.103:0/4018791758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e67a40 msgr2=0x556995e67e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:10.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.799+0000 7f178d6c4740 1 --2- 172.21.15.103:0/4018791758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e67a40 0x556995e67e20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f17800cd070 tx=0x7f178009c440 comp rx=0 tx=0).stop 2024-09-19T07:03:10.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.799+0000 7f178d6c4740 1 -- 172.21.15.103:0/4018791758 shutdown_connections 2024-09-19T07:03:10.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.799+0000 7f178d6c4740 1 --2- 172.21.15.103:0/4018791758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e77150 0x556995e79540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.799+0000 7f178d6c4740 1 --2- 172.21.15.103:0/4018791758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e68360 0x556995e76c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.799+0000 7f178d6c4740 1 --2- 172.21.15.103:0/4018791758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e67a40 0x556995e67e20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.799+0000 7f178d6c4740 1 -- 172.21.15.103:0/4018791758 >> 172.21.15.103:0/4018791758 conn(0x556995e640f0 msgr2=0x556995e644d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:10.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.799+0000 7f178d6c4740 1 -- 172.21.15.103:0/4018791758 shutdown_connections 2024-09-19T07:03:10.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.799+0000 7f178d6c4740 1 -- 172.21.15.103:0/4018791758 wait complete. 2024-09-19T07:03:10.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.800+0000 7f178d6c4740 1 -- start start 2024-09-19T07:03:10.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.800+0000 7f178d6c4740 1 Processor -- start 2024-09-19T07:03:10.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.800+0000 7f178d6c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e67a40 0x556995e740b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.800+0000 7f178d6c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e68360 0x556995e745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.800+0000 7f178d6c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e77150 0x556995e74b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.800+0000 7f178d6c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556995e79d20 con 0x556995e77150 2024-09-19T07:03:10.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.800+0000 7f178d6c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556995d87f10 con 0x556995e67a40 2024-09-19T07:03:10.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.800+0000 7f178d6c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556995e79ea0 con 0x556995e68360 2024-09-19T07:03:10.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.800+0000 7f178d6c4740 10 client.0 _ll_register_callbacks cb 0x556995d50a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:10.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.800+0000 7f1789888640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e77150 0x556995e74b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:10.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.800+0000 7f1789888640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e77150 0x556995e74b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38098/0 (socket says 172.21.15.103:38098) 2024-09-19T07:03:10.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.800+0000 7f1788886640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e68360 0x556995e745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:10.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.800+0000 7f1789888640 1 -- 172.21.15.103:0/1871570735 learned_addr learned my addr 172.21.15.103:0/1871570735 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:10.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.800+0000 7f1789087640 1 --2- 172.21.15.103:0/1871570735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e67a40 0x556995e740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:10.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.801+0000 7f1789888640 1 -- 172.21.15.103:0/1871570735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e68360 msgr2=0x556995e745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:10.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.801+0000 7f1789888640 1 --2- 172.21.15.103:0/1871570735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e68360 0x556995e745f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.801+0000 7f1789888640 1 -- 172.21.15.103:0/1871570735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e67a40 msgr2=0x556995e740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:10.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.801+0000 7f1789888640 1 --2- 172.21.15.103:0/1871570735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e67a40 0x556995e740b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.801+0000 7f1789888640 1 -- 172.21.15.103:0/1871570735 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556995b3e6a0 con 0x556995e77150 2024-09-19T07:03:10.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.801+0000 7f1788886640 1 --2- 172.21.15.103:0/1871570735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e68360 0x556995e745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:03:10.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.801+0000 7f1789888640 1 --2- 172.21.15.103:0/1871570735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e77150 0x556995e74b30 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f177c033040 tx=0x7f177c048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b111c42846863007 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:10.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.801+0000 7f176cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:10.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.801+0000 7f176cff9640 1 -- 172.21.15.103:0/1871570735 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c04f070 con 0x556995e77150 2024-09-19T07:03:10.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.801+0000 7f176cff9640 1 -- 172.21.15.103:0/1871570735 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f177c003e00 con 0x556995e77150 2024-09-19T07:03:10.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.801+0000 7f178d6c4740 1 -- 172.21.15.103:0/1871570735 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17800a4870 con 0x556995e77150 2024-09-19T07:03:10.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.801+0000 7f176cff9640 1 -- 172.21.15.103:0/1871570735 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f177c0047d0 con 0x556995e77150 2024-09-19T07:03:10.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.801+0000 7f178d6c4740 10 client.20274 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:10.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.801+0000 7f178d6c4740 1 -- 172.21.15.103:0/1871570735 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556995e859e0 con 0x556995e77150 2024-09-19T07:03:10.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.801+0000 7f176cff9640 1 -- 172.21.15.103:0/1871570735 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17800a4870 con 0x556995e77150 2024-09-19T07:03:10.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.802+0000 7f176cff9640 1 -- 172.21.15.103:0/1871570735 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f177c030830 con 0x556995e77150 2024-09-19T07:03:10.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.802+0000 7f178d6c4740 10 client.20274 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:10.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.802+0000 7f178d6c4740 10 client.20274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:10.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.802+0000 7f178d6c4740 20 client.20274 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:10.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.802+0000 7f178d6c4740 1 --2- 172.21.15.103:0/1871570735 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556995e86ea0 0x556995e89300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.802+0000 7f178d6c4740 4 client.20274 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:10.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.802+0000 7f178d6c4740 1 -- 172.21.15.103:0/1871570735 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556995dda7f0 con 0x556995e86ea0 2024-09-19T07:03:10.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.802+0000 7f1789087640 1 --2- 172.21.15.103:0/1871570735 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556995e86ea0 0x556995e89300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:10.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.802+0000 7f176cff9640 10 client.20274 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:10.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.802+0000 7f1789087640 1 --2- 172.21.15.103:0/1871570735 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556995e86ea0 0x556995e89300 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=211288850b382aab server_cookie=83401c27bef53ca in_seq=0 out_seq=0 2024-09-19T07:03:10.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.803+0000 7f176cff9640 1 -- 172.21.15.103:0/1871570735 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556995dda7f0 con 0x556995e86ea0 2024-09-19T07:03:10.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.803+0000 7f176cff9640 10 client.20274 handle_command_reply: tid=0 2024-09-19T07:03:10.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.803+0000 7f178d6c4740 1 client.20274 shutdown 2024-09-19T07:03:10.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.803+0000 7f178d6c4740 1 -- 172.21.15.103:0/1871570735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e77150 msgr2=0x556995e74b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:10.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.803+0000 7f178d6c4740 1 --2- 172.21.15.103:0/1871570735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e77150 0x556995e74b30 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f177c033040 tx=0x7f177c048000 comp rx=0 tx=0).stop 2024-09-19T07:03:10.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.803+0000 7f178d6c4740 1 -- 172.21.15.103:0/1871570735 shutdown_connections 2024-09-19T07:03:10.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.803+0000 7f178d6c4740 1 --2- 172.21.15.103:0/1871570735 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556995e86ea0 0x556995e89300 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.803+0000 7f178d6c4740 1 --2- 172.21.15.103:0/1871570735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e77150 0x556995e74b30 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.803+0000 7f178d6c4740 1 --2- 172.21.15.103:0/1871570735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e68360 0x556995e745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.803+0000 7f178d6c4740 1 --2- 172.21.15.103:0/1871570735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e67a40 0x556995e740b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.803+0000 7f176cff9640 0 client.20274 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:10.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.803+0000 7f178d6c4740 1 -- 172.21.15.103:0/1871570735 >> 172.21.15.103:0/1871570735 conn(0x556995e640f0 msgr2=0x556995e69e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:10.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.803+0000 7f178d6c4740 1 -- 172.21.15.103:0/1871570735 shutdown_connections 2024-09-19T07:03:10.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.803+0000 7f178d6c4740 1 -- 172.21.15.103:0/1871570735 wait complete. 2024-09-19T07:03:10.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.804+0000 7f178d6c4740 20 client.20274 trim_cache size 0 max 16384 2024-09-19T07:03:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.820+0000 7f178d6c4740 1 Processor -- start 2024-09-19T07:03:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.820+0000 7f178d6c4740 1 -- start start 2024-09-19T07:03:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.820+0000 7f178d6c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e67a40 0x556995e8ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.820+0000 7f178d6c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e68360 0x556995e8f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.820+0000 7f178d6c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e77150 0x556995e71310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.820+0000 7f178d6c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17841ab730 con 0x556995e68360 2024-09-19T07:03:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.820+0000 7f178d6c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17841ad270 con 0x556995e77150 2024-09-19T07:03:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.820+0000 7f178d6c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17841abe00 con 0x556995e67a40 2024-09-19T07:03:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.821+0000 7f1788886640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e68360 0x556995e8f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.821+0000 7f1788886640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e68360 0x556995e8f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38112/0 (socket says 172.21.15.103:38112) 2024-09-19T07:03:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.821+0000 7f1788886640 1 -- 172.21.15.103:0/3643932326 learned_addr learned my addr 172.21.15.103:0/3643932326 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.821+0000 7f1788886640 1 -- 172.21.15.103:0/3643932326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e67a40 msgr2=0x556995e8ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.821+0000 7f1788886640 1 --2- 172.21.15.103:0/3643932326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e67a40 0x556995e8ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.821+0000 7f1788886640 1 -- 172.21.15.103:0/3643932326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e77150 msgr2=0x556995e71310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.821+0000 7f1788886640 1 --2- 172.21.15.103:0/3643932326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e77150 0x556995e71310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.821+0000 7f1788886640 1 -- 172.21.15.103:0/3643932326 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556995e6a560 con 0x556995e68360 2024-09-19T07:03:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.821+0000 7f1788886640 1 --2- 172.21.15.103:0/3643932326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e68360 0x556995e8f110 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f17740104b0 tx=0x7f177400a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecb1433320d12c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.821+0000 7f176effd640 1 -- 172.21.15.103:0/3643932326 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177401e030 con 0x556995e68360 2024-09-19T07:03:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.821+0000 7f176effd640 1 -- 172.21.15.103:0/3643932326 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1774005500 con 0x556995e68360 2024-09-19T07:03:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.821+0000 7f176effd640 1 -- 172.21.15.103:0/3643932326 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177400f910 con 0x556995e68360 2024-09-19T07:03:10.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.822+0000 7f178d6c4740 1 -- 172.21.15.103:0/3643932326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e68360 msgr2=0x556995e8f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:10.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.822+0000 7f178d6c4740 1 --2- 172.21.15.103:0/3643932326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e68360 0x556995e8f110 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f17740104b0 tx=0x7f177400a2d0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.822+0000 7f178d6c4740 1 -- 172.21.15.103:0/3643932326 shutdown_connections 2024-09-19T07:03:10.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.822+0000 7f178d6c4740 1 --2- 172.21.15.103:0/3643932326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e77150 0x556995e71310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.822+0000 7f178d6c4740 1 --2- 172.21.15.103:0/3643932326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e68360 0x556995e8f110 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.822+0000 7f178d6c4740 1 --2- 172.21.15.103:0/3643932326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e67a40 0x556995e8ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.822+0000 7f178d6c4740 1 -- 172.21.15.103:0/3643932326 >> 172.21.15.103:0/3643932326 conn(0x556995e640f0 msgr2=0x556995e76020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:10.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.822+0000 7f178d6c4740 1 -- 172.21.15.103:0/3643932326 shutdown_connections 2024-09-19T07:03:10.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.822+0000 7f178d6c4740 1 -- 172.21.15.103:0/3643932326 wait complete. 2024-09-19T07:03:10.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.822+0000 7f178d6c4740 1 -- start start 2024-09-19T07:03:10.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.822+0000 7f178d6c4740 1 Processor -- start 2024-09-19T07:03:10.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f178d6c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e67a40 0x556995e7b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f178d6c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e68360 0x556995e7b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f178d6c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e77150 0x556995e7bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f178d6c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17841acba0 con 0x556995e67a40 2024-09-19T07:03:10.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f178d6c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17841ac4d0 con 0x556995e77150 2024-09-19T07:03:10.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f178d6c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17841a9730 con 0x556995e68360 2024-09-19T07:03:10.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f178d6c4740 10 client.0 _ll_register_callbacks cb 0x7f177c031a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:10.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f1789087640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e67a40 0x556995e7b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:10.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f1789087640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e67a40 0x556995e7b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38126/0 (socket says 172.21.15.103:38126) 2024-09-19T07:03:10.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f1789087640 1 -- 172.21.15.103:0/2098759154 learned_addr learned my addr 172.21.15.103:0/2098759154 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:10.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f1788886640 1 --2- 172.21.15.103:0/2098759154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e68360 0x556995e7b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:10.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f1789888640 1 --2- 172.21.15.103:0/2098759154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e77150 0x556995e7bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:10.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f1789087640 1 -- 172.21.15.103:0/2098759154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e68360 msgr2=0x556995e7b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:10.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f1789087640 1 --2- 172.21.15.103:0/2098759154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e68360 0x556995e7b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f1789087640 1 -- 172.21.15.103:0/2098759154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e77150 msgr2=0x556995e7bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:10.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f1789087640 1 --2- 172.21.15.103:0/2098759154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e77150 0x556995e7bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f1789087640 1 -- 172.21.15.103:0/2098759154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556995ef0da0 con 0x556995e67a40 2024-09-19T07:03:10.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f1789888640 1 --2- 172.21.15.103:0/2098759154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e77150 0x556995e7bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f1789087640 1 --2- 172.21.15.103:0/2098759154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e67a40 0x556995e7b0c0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f178009bb40 tx=0x7f1780000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411e97f0fa994669 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f176cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f176cff9640 1 -- 172.21.15.103:0/2098759154 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17800b2060 con 0x556995e67a40 2024-09-19T07:03:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f176cff9640 1 -- 172.21.15.103:0/2098759154 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8600+0+0 (secure 0 0 0) 0x7f17800c62f0 con 0x556995e67a40 2024-09-19T07:03:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.823+0000 7f178d6c4740 1 -- 172.21.15.103:0/2098759154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1774005500 con 0x556995e67a40 2024-09-19T07:03:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.824+0000 7f176cff9640 1 -- 172.21.15.103:0/2098759154 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f17800b9050 con 0x556995e67a40 2024-09-19T07:03:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.824+0000 7f178d6c4740 10 client.20280 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.824+0000 7f178d6c4740 1 -- 172.21.15.103:0/2098759154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556995ef20a0 con 0x556995e67a40 2024-09-19T07:03:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.824+0000 7f176cff9640 1 -- 172.21.15.103:0/2098759154 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1774005500 con 0x556995e67a40 2024-09-19T07:03:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.824+0000 7f176cff9640 1 -- 172.21.15.103:0/2098759154 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f17800cd2f0 con 0x556995e67a40 2024-09-19T07:03:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.824+0000 7f178d6c4740 10 client.20280 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.824+0000 7f178d6c4740 10 client.20280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.824+0000 7f178d6c4740 20 client.20280 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.824+0000 7f178d6c4740 1 --2- 172.21.15.103:0/2098759154 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556995ef23e0 0x556995ef47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.824+0000 7f178d6c4740 4 client.20280 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:03:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.824+0000 7f178d6c4740 1 -- 172.21.15.103:0/2098759154 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556995dd4d00 con 0x556995ef23e0 2024-09-19T07:03:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.824+0000 7f1788886640 1 --2- 172.21.15.103:0/2098759154 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556995ef23e0 0x556995ef47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.825+0000 7f1788886640 1 --2- 172.21.15.103:0/2098759154 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556995ef23e0 0x556995ef47d0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=936738d76def1789 server_cookie=cddc49053b0e0e18 in_seq=0 out_seq=0 2024-09-19T07:03:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.825+0000 7f176cff9640 10 client.20280 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.826+0000 7f176cff9640 1 -- 172.21.15.103:0/2098759154 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556995dd4d00 con 0x556995ef23e0 2024-09-19T07:03:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.826+0000 7f176cff9640 10 client.20280 handle_command_reply: tid=0 2024-09-19T07:03:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.826+0000 7f178d6c4740 1 client.20280 shutdown 2024-09-19T07:03:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.826+0000 7f178d6c4740 1 -- 172.21.15.103:0/2098759154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e67a40 msgr2=0x556995e7b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.826+0000 7f178d6c4740 1 --2- 172.21.15.103:0/2098759154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e67a40 0x556995e7b0c0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f178009bb40 tx=0x7f1780000bc0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.826+0000 7f178d6c4740 1 -- 172.21.15.103:0/2098759154 shutdown_connections 2024-09-19T07:03:10.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.826+0000 7f178d6c4740 1 --2- 172.21.15.103:0/2098759154 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x556995ef23e0 0x556995ef47d0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.826+0000 7f178d6c4740 1 --2- 172.21.15.103:0/2098759154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556995e77150 0x556995e7bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.826+0000 7f178d6c4740 1 --2- 172.21.15.103:0/2098759154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556995e68360 0x556995e7b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.826+0000 7f176cff9640 0 client.20280 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:10.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.826+0000 7f178d6c4740 1 --2- 172.21.15.103:0/2098759154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556995e67a40 0x556995e7b0c0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.826+0000 7f178d6c4740 1 -- 172.21.15.103:0/2098759154 >> 172.21.15.103:0/2098759154 conn(0x556995e640f0 msgr2=0x556995e62a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:10.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.826+0000 7f178d6c4740 1 -- 172.21.15.103:0/2098759154 shutdown_connections 2024-09-19T07:03:10.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.826+0000 7f178d6c4740 1 -- 172.21.15.103:0/2098759154 wait complete. 2024-09-19T07:03:10.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.826+0000 7f178d6c4740 20 client.20280 trim_cache size 0 max 16384 2024-09-19T07:03:10.832 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:03:10.832 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:03:10.832 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "8a457ac8-8e33-4cff-a968-8c2e6435a530", 2024-09-19T07:03:10.832 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:03:10.832 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:03:10.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.827+0000 7f178b2eb640 1 -- 172.21.15.103:0/1595248949 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17700a33b0 msgr2=0x7f17700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:10.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.827+0000 7f178b2eb640 1 --2- 172.21.15.103:0/1595248949 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17700a33b0 0x7f17700a5870 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f177c004af0 tx=0x7f177c02f500 comp rx=0 tx=0).stop 2024-09-19T07:03:10.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.827+0000 7f178b2eb640 1 -- 172.21.15.103:0/1595248949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1784100460 msgr2=0x7f1784199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:10.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.827+0000 7f178b2eb640 1 --2- 172.21.15.103:0/1595248949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1784100460 0x7f1784199180 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f178000d950 tx=0x7f178000dd50 comp rx=0 tx=0).stop 2024-09-19T07:03:10.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.827+0000 7f178b2eb640 1 -- 172.21.15.103:0/1595248949 shutdown_connections 2024-09-19T07:03:10.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.827+0000 7f178b2eb640 1 --2- 172.21.15.103:0/1595248949 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17700a33b0 0x7f17700a5870 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.827+0000 7f178b2eb640 1 --2- 172.21.15.103:0/1595248949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1784105590 0x7f17841a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.827+0000 7f178b2eb640 1 --2- 172.21.15.103:0/1595248949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1784100e10 0x7f17841996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.827+0000 7f178b2eb640 1 --2- 172.21.15.103:0/1595248949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1784100460 0x7f1784199180 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:10.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.827+0000 7f178b2eb640 1 -- 172.21.15.103:0/1595248949 >> 172.21.15.103:0/1595248949 conn(0x7f17840fbee0 msgr2=0x7f1784103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:10.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.827+0000 7f178b2eb640 1 -- 172.21.15.103:0/1595248949 shutdown_connections 2024-09-19T07:03:10.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:10.827+0000 7f178b2eb640 1 -- 172.21.15.103:0/1595248949 wait complete. 2024-09-19T07:03:11.492 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:03:11.779 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:12.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:11 smithi103 ceph-mon[31158]: pgmap v436: 97 pgs: 97 active+clean; 3.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 97 KiB/s wr, 35 op/s 2024-09-19T07:03:12.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:11 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T07:03:12.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:11 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:03:12.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:11 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T07:03:12.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:11 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:03:12.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:11 smithi103 ceph-mon[31158]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T07:03:12.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:11 smithi134 ceph-mon[35356]: pgmap v436: 97 pgs: 97 active+clean; 3.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 97 KiB/s wr, 35 op/s 2024-09-19T07:03:12.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:11 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T07:03:12.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:11 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:03:12.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:11 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T07:03:12.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:11 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:03:12.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:11 smithi134 ceph-mon[35356]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T07:03:12.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.252+0000 7fc7fbb05640 1 -- 172.21.15.103:0/3776675676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7f4106a80 msgr2=0x7fc7f4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:12.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.252+0000 7fc7fbb05640 1 --2- 172.21.15.103:0/3776675676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7f4106a80 0x7fc7f4106e60 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0009930 tx=0x7fc7f002a420 comp rx=0 tx=0).stop 2024-09-19T07:03:12.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.252+0000 7fc7fbb05640 1 -- 172.21.15.103:0/3776675676 shutdown_connections 2024-09-19T07:03:12.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.252+0000 7fc7fbb05640 1 --2- 172.21.15.103:0/3776675676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7f4101420 0x7fc7f410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.252+0000 7fc7fbb05640 1 --2- 172.21.15.103:0/3776675676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f4100a80 0x7fc7f4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.252+0000 7fc7fbb05640 1 --2- 172.21.15.103:0/3776675676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7f4106a80 0x7fc7f4106e60 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.252+0000 7fc7fbb05640 1 -- 172.21.15.103:0/3776675676 >> 172.21.15.103:0/3776675676 conn(0x7fc7f40fc7a0 msgr2=0x7fc7f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:12.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.252+0000 7fc7fbb05640 1 -- 172.21.15.103:0/3776675676 shutdown_connections 2024-09-19T07:03:12.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.252+0000 7fc7fbb05640 1 -- 172.21.15.103:0/3776675676 wait complete. 2024-09-19T07:03:12.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.253+0000 7fc7fbb05640 1 Processor -- start 2024-09-19T07:03:12.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.253+0000 7fc7fbb05640 1 -- start start 2024-09-19T07:03:12.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.253+0000 7fc7fbb05640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f4100a80 0x7fc7f4197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7fbb05640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7f4101420 0x7fc7f41978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7fbb05640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7f4106a80 0x7fc7f419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7fbb05640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4110ea0 con 0x7fc7f4101420 2024-09-19T07:03:12.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7fbb05640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4110d20 con 0x7fc7f4100a80 2024-09-19T07:03:12.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7fbb05640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4111020 con 0x7fc7f4106a80 2024-09-19T07:03:12.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7f90a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7f4101420 0x7fc7f41978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7f90a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7f4101420 0x7fc7f41978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38138/0 (socket says 172.21.15.103:38138) 2024-09-19T07:03:12.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7f90a0640 1 -- 172.21.15.103:0/2667429779 learned_addr learned my addr 172.21.15.103:0/2667429779 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:12.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7f98a1640 1 --2- 172.21.15.103:0/2667429779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f4100a80 0x7fc7f4197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7fa0a2640 1 --2- 172.21.15.103:0/2667429779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7f4106a80 0x7fc7f419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7f90a0640 1 -- 172.21.15.103:0/2667429779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7f4106a80 msgr2=0x7fc7f419bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:12.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7f90a0640 1 --2- 172.21.15.103:0/2667429779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7f4106a80 0x7fc7f419bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7f90a0640 1 -- 172.21.15.103:0/2667429779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f4100a80 msgr2=0x7fc7f4197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:12.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7f90a0640 1 --2- 172.21.15.103:0/2667429779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f4100a80 0x7fc7f4197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7f90a0640 1 -- 172.21.15.103:0/2667429779 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f419c310 con 0x7fc7f4101420 2024-09-19T07:03:12.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7f98a1640 1 --2- 172.21.15.103:0/2667429779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f4100a80 0x7fc7f4197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7f90a0640 1 --2- 172.21.15.103:0/2667429779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7f4101420 0x7fc7f41978a0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400d950 tx=0x7fc7e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7eaffd640 1 -- 172.21.15.103:0/2667429779 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4017070 con 0x7fc7f4101420 2024-09-19T07:03:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7fbb05640 1 -- 172.21.15.103:0/2667429779 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f419c600 con 0x7fc7f4101420 2024-09-19T07:03:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7eaffd640 1 -- 172.21.15.103:0/2667429779 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e4009c30 con 0x7fc7f4101420 2024-09-19T07:03:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7eaffd640 1 -- 172.21.15.103:0/2667429779 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e400f6b0 con 0x7fc7f4101420 2024-09-19T07:03:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.254+0000 7fc7fbb05640 1 -- 172.21.15.103:0/2667429779 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f41a3ee0 con 0x7fc7f4101420 2024-09-19T07:03:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.256+0000 7fc7eaffd640 1 -- 172.21.15.103:0/2667429779 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc7e40109d0 con 0x7fc7f4101420 2024-09-19T07:03:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.257+0000 7fc7fdede740 1 Processor -- start 2024-09-19T07:03:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.257+0000 7fc7fdede740 1 -- start start 2024-09-19T07:03:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.257+0000 7fc7fdede740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7068320 0x55f3e7068700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.257+0000 7fc7fdede740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7075350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.257+0000 7fc7fdede740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7075890 0x55f3e7077c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.257+0000 7fc7fdede740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f3e6fce2e0 con 0x55f3e7068c40 2024-09-19T07:03:12.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.257+0000 7fc7fdede740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f3e6f75ce0 con 0x55f3e7075890 2024-09-19T07:03:12.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.257+0000 7fc7fdede740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f3e6fca2a0 con 0x55f3e7068320 2024-09-19T07:03:12.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.257+0000 7fc7f98a1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7075350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.257+0000 7fc7eaffd640 1 --2- 172.21.15.103:0/2667429779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7e00a33b0 0x7fc7e00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.257+0000 7fc7f98a1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7075350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38142/0 (socket says 172.21.15.103:38142) 2024-09-19T07:03:12.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.257+0000 7fc7f98a1640 1 -- 172.21.15.103:0/2544107264 learned_addr learned my addr 172.21.15.103:0/2544107264 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:12.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.257+0000 7fc7fa0a2640 1 --2- 172.21.15.103:0/2667429779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7e00a33b0 0x7fc7e00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.258+0000 7fc7fa0a2640 1 --2- 172.21.15.103:0/2544107264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7068320 0x55f3e7068700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.258+0000 7fc7eaffd640 1 -- 172.21.15.103:0/2667429779 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7fc7e40cfa80 con 0x7fc7f4101420 2024-09-19T07:03:12.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.258+0000 7fc7f90a0640 1 --2- 172.21.15.103:0/2544107264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7075890 0x55f3e7077c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.258+0000 7fc7f98a1640 1 -- 172.21.15.103:0/2544107264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7068320 msgr2=0x55f3e7068700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:12.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.258+0000 7fc7f98a1640 1 --2- 172.21.15.103:0/2544107264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7068320 0x55f3e7068700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.258+0000 7fc7f98a1640 1 -- 172.21.15.103:0/2544107264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7075890 msgr2=0x55f3e7077c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:12.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.258+0000 7fc7f98a1640 1 --2- 172.21.15.103:0/2544107264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7075890 0x55f3e7077c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.258+0000 7fc7f98a1640 1 -- 172.21.15.103:0/2544107264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3e6f5a440 con 0x55f3e7068c40 2024-09-19T07:03:12.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.258+0000 7fc7f90a0640 1 --2- 172.21.15.103:0/2544107264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7075890 0x55f3e7077c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:12.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.258+0000 7fc7fa0a2640 1 --2- 172.21.15.103:0/2667429779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7e00a33b0 0x7fc7e00a5870 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec006d20 tx=0x7fc7ec005f40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:12.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.258+0000 7fc7f98a1640 1 --2- 172.21.15.103:0/2544107264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7075350 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0006010 tx=0x7fc7f0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee72e5fd38eab8db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:12.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.258+0000 7fc7dffff640 1 -- 172.21.15.103:0/2544107264 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f003f070 con 0x55f3e7068c40 2024-09-19T07:03:12.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.258+0000 7fc7dffff640 1 -- 172.21.15.103:0/2544107264 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7f00317b0 con 0x55f3e7068c40 2024-09-19T07:03:12.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.258+0000 7fc7dffff640 1 -- 172.21.15.103:0/2544107264 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0035070 con 0x55f3e7068c40 2024-09-19T07:03:12.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.259+0000 7fc7fdede740 1 -- 172.21.15.103:0/2544107264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 msgr2=0x55f3e7075350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:12.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.259+0000 7fc7fdede740 1 --2- 172.21.15.103:0/2544107264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7075350 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0006010 tx=0x7fc7f0009680 comp rx=0 tx=0).stop 2024-09-19T07:03:12.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.259+0000 7fc7fdede740 1 -- 172.21.15.103:0/2544107264 shutdown_connections 2024-09-19T07:03:12.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.259+0000 7fc7fdede740 1 --2- 172.21.15.103:0/2544107264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7075890 0x55f3e7077c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.259+0000 7fc7fdede740 1 --2- 172.21.15.103:0/2544107264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7075350 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.259+0000 7fc7fdede740 1 --2- 172.21.15.103:0/2544107264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7068320 0x55f3e7068700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.259+0000 7fc7fdede740 1 -- 172.21.15.103:0/2544107264 >> 172.21.15.103:0/2544107264 conn(0x55f3e7064a30 msgr2=0x55f3e7064e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.259+0000 7fc7fdede740 1 -- 172.21.15.103:0/2544107264 shutdown_connections 2024-09-19T07:03:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.259+0000 7fc7fdede740 1 -- 172.21.15.103:0/2544107264 wait complete. 2024-09-19T07:03:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.259+0000 7fc7fdede740 1 -- start start 2024-09-19T07:03:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.259+0000 7fc7fdede740 1 Processor -- start 2024-09-19T07:03:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7fdede740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7068320 0x55f3e70751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7fdede740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7072490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7fdede740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7075890 0x55f3e70729f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7fdede740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f3e7078460 con 0x55f3e7068c40 2024-09-19T07:03:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7fdede740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f3e6f89050 con 0x55f3e7068320 2024-09-19T07:03:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7fdede740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f3e70785e0 con 0x55f3e7075890 2024-09-19T07:03:12.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7fdede740 10 client.0 _ll_register_callbacks cb 0x55f3e6fc9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:12.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7f90a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7072490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7f90a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7072490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38144/0 (socket says 172.21.15.103:38144) 2024-09-19T07:03:12.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7f90a0640 1 -- 172.21.15.103:0/3391659721 learned_addr learned my addr 172.21.15.103:0/3391659721 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:12.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7fa0a2640 1 --2- 172.21.15.103:0/3391659721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7075890 0x55f3e70729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7f98a1640 1 --2- 172.21.15.103:0/3391659721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7068320 0x55f3e70751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7f90a0640 1 -- 172.21.15.103:0/3391659721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7075890 msgr2=0x55f3e70729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:12.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7f90a0640 1 --2- 172.21.15.103:0/3391659721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7075890 0x55f3e70729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7f90a0640 1 -- 172.21.15.103:0/3391659721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7068320 msgr2=0x55f3e70751e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:12.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7f90a0640 1 --2- 172.21.15.103:0/3391659721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7068320 0x55f3e70751e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7f90a0640 1 -- 172.21.15.103:0/3391659721 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3e6d3f3c0 con 0x55f3e7068c40 2024-09-19T07:03:12.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7f98a1640 1 --2- 172.21.15.103:0/3391659721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7068320 0x55f3e70751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:12.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7f90a0640 1 --2- 172.21.15.103:0/3391659721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7072490 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40981a0 tx=0x7fc7e4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154d42c4b181bbc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:12.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.260+0000 7fc7dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:12.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.261+0000 7fc7dd7fa640 1 -- 172.21.15.103:0/3391659721 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4020070 con 0x55f3e7068c40 2024-09-19T07:03:12.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.261+0000 7fc7dd7fa640 1 -- 172.21.15.103:0/3391659721 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7fc7e409b190 con 0x55f3e7068c40 2024-09-19T07:03:12.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.261+0000 7fc7fdede740 1 -- 172.21.15.103:0/3391659721 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7f00317b0 con 0x55f3e7068c40 2024-09-19T07:03:12.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.261+0000 7fc7dd7fa640 1 -- 172.21.15.103:0/3391659721 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc7e40a00f0 con 0x55f3e7068c40 2024-09-19T07:03:12.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.261+0000 7fc7fdede740 10 client.20292 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:12.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.261+0000 7fc7fdede740 1 -- 172.21.15.103:0/3391659721 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f3e7083e40 con 0x55f3e7068c40 2024-09-19T07:03:12.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.261+0000 7fc7dd7fa640 1 -- 172.21.15.103:0/3391659721 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7f00317b0 con 0x55f3e7068c40 2024-09-19T07:03:12.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.261+0000 7fc7dd7fa640 1 -- 172.21.15.103:0/3391659721 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc7e40b2080 con 0x55f3e7068c40 2024-09-19T07:03:12.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.261+0000 7fc7fdede740 10 client.20292 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:12.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.261+0000 7fc7fdede740 10 client.20292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:12.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.261+0000 7fc7fdede740 20 client.20292 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:12.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.261+0000 7fc7fdede740 1 --2- 172.21.15.103:0/3391659721 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f3e7085250 0x55f3e70876a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.261+0000 7fc7fdede740 4 client.20292 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:12.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.261+0000 7fc7fdede740 1 -- 172.21.15.103:0/3391659721 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3e6fdb850 con 0x55f3e7085250 2024-09-19T07:03:12.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.261+0000 7fc7f98a1640 1 --2- 172.21.15.103:0/3391659721 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f3e7085250 0x55f3e70876a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.262+0000 7fc7f98a1640 1 --2- 172.21.15.103:0/3391659721 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f3e7085250 0x55f3e70876a0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61e47ce237f56bfc server_cookie=92aa3953808cae9f in_seq=0 out_seq=0 2024-09-19T07:03:12.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.262+0000 7fc7dd7fa640 10 client.20292 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:12.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.262+0000 7fc7dd7fa640 1 -- 172.21.15.103:0/3391659721 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3e6fdb850 con 0x55f3e7085250 2024-09-19T07:03:12.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.262+0000 7fc7dd7fa640 10 client.20292 handle_command_reply: tid=0 2024-09-19T07:03:12.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.262+0000 7fc7fdede740 1 client.20292 shutdown 2024-09-19T07:03:12.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.263+0000 7fc7fdede740 1 -- 172.21.15.103:0/3391659721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 msgr2=0x55f3e7072490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:12.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.263+0000 7fc7fdede740 1 --2- 172.21.15.103:0/3391659721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7072490 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40981a0 tx=0x7fc7e4000bc0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.263+0000 7fc7fdede740 1 -- 172.21.15.103:0/3391659721 shutdown_connections 2024-09-19T07:03:12.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.263+0000 7fc7fdede740 1 --2- 172.21.15.103:0/3391659721 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f3e7085250 0x55f3e70876a0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.263+0000 7fc7fdede740 1 --2- 172.21.15.103:0/3391659721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7075890 0x55f3e70729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.263+0000 7fc7fdede740 1 --2- 172.21.15.103:0/3391659721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7072490 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.263+0000 7fc7fdede740 1 --2- 172.21.15.103:0/3391659721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7068320 0x55f3e70751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.263+0000 7fc7dd7fa640 0 client.20292 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:12.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.263+0000 7fc7fdede740 1 -- 172.21.15.103:0/3391659721 >> 172.21.15.103:0/3391659721 conn(0x55f3e7064a30 msgr2=0x55f3e706a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:12.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.263+0000 7fc7fdede740 1 -- 172.21.15.103:0/3391659721 shutdown_connections 2024-09-19T07:03:12.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.263+0000 7fc7fdede740 1 -- 172.21.15.103:0/3391659721 wait complete. 2024-09-19T07:03:12.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.263+0000 7fc7fdede740 20 client.20292 trim_cache size 0 max 16384 2024-09-19T07:03:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7fdede740 1 Processor -- start 2024-09-19T07:03:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7fdede740 1 -- start start 2024-09-19T07:03:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7fdede740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7068320 0x55f3e70816c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7fdede740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e708cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7fdede740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7075890 0x55f3e708d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7fdede740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41ab630 con 0x55f3e7068c40 2024-09-19T07:03:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7fdede740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41ad170 con 0x55f3e7075890 2024-09-19T07:03:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7fdede740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41abd00 con 0x55f3e7068320 2024-09-19T07:03:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7f90a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e708cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7f90a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e708cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38148/0 (socket says 172.21.15.103:38148) 2024-09-19T07:03:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7f90a0640 1 -- 172.21.15.103:0/2517787895 learned_addr learned my addr 172.21.15.103:0/2517787895 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7f90a0640 1 -- 172.21.15.103:0/2517787895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7068320 msgr2=0x55f3e70816c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7f90a0640 1 --2- 172.21.15.103:0/2517787895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7068320 0x55f3e70816c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7f90a0640 1 -- 172.21.15.103:0/2517787895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7075890 msgr2=0x55f3e708d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7f90a0640 1 --2- 172.21.15.103:0/2517787895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7075890 0x55f3e708d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7f90a0640 1 -- 172.21.15.103:0/2517787895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3e7068a70 con 0x55f3e7068c40 2024-09-19T07:03:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.280+0000 7fc7f90a0640 1 --2- 172.21.15.103:0/2517787895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e708cf70 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009610 tx=0x7fc7e409bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d55f50e57419e36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.281+0000 7fc7df7fe640 1 -- 172.21.15.103:0/2517787895 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4020040 con 0x55f3e7068c40 2024-09-19T07:03:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.281+0000 7fc7df7fe640 1 -- 172.21.15.103:0/2517787895 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e4010460 con 0x55f3e7068c40 2024-09-19T07:03:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.281+0000 7fc7df7fe640 1 -- 172.21.15.103:0/2517787895 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e409b190 con 0x55f3e7068c40 2024-09-19T07:03:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.281+0000 7fc7fdede740 1 -- 172.21.15.103:0/2517787895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 msgr2=0x55f3e708cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.281+0000 7fc7fdede740 1 --2- 172.21.15.103:0/2517787895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e708cf70 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009610 tx=0x7fc7e409bec0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.281+0000 7fc7fdede740 1 -- 172.21.15.103:0/2517787895 shutdown_connections 2024-09-19T07:03:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.281+0000 7fc7fdede740 1 --2- 172.21.15.103:0/2517787895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7075890 0x55f3e708d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.281+0000 7fc7fdede740 1 --2- 172.21.15.103:0/2517787895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e708cf70 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.281+0000 7fc7fdede740 1 --2- 172.21.15.103:0/2517787895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7068320 0x55f3e70816c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.281+0000 7fc7fdede740 1 -- 172.21.15.103:0/2517787895 >> 172.21.15.103:0/2517787895 conn(0x55f3e7064a30 msgr2=0x55f3e7071580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.281+0000 7fc7fdede740 1 -- 172.21.15.103:0/2517787895 shutdown_connections 2024-09-19T07:03:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.281+0000 7fc7fdede740 1 -- 172.21.15.103:0/2517787895 wait complete. 2024-09-19T07:03:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.282+0000 7fc7fdede740 1 -- start start 2024-09-19T07:03:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.282+0000 7fc7fdede740 1 Processor -- start 2024-09-19T07:03:12.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.282+0000 7fc7fdede740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7068320 0x55f3e70795c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.282+0000 7fc7fdede740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7079b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.282+0000 7fc7fdede740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7075890 0x55f3e707a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.282+0000 7fc7fdede740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41acaa0 con 0x55f3e7068c40 2024-09-19T07:03:12.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.282+0000 7fc7fdede740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41ac3d0 con 0x55f3e7075890 2024-09-19T07:03:12.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.282+0000 7fc7fdede740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41a9630 con 0x55f3e7068320 2024-09-19T07:03:12.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.282+0000 7fc7fdede740 10 client.0 _ll_register_callbacks cb 0x7fc7e400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:12.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.282+0000 7fc7f90a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7079b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.282+0000 7fc7f90a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7079b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38154/0 (socket says 172.21.15.103:38154) 2024-09-19T07:03:12.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.282+0000 7fc7f90a0640 1 -- 172.21.15.103:0/662651071 learned_addr learned my addr 172.21.15.103:0/662651071 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:12.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7f98a1640 1 --2- 172.21.15.103:0/662651071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7068320 0x55f3e70795c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7f90a0640 1 -- 172.21.15.103:0/662651071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7068320 msgr2=0x55f3e70795c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:12.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7f90a0640 1 --2- 172.21.15.103:0/662651071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7068320 0x55f3e70795c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7f90a0640 1 -- 172.21.15.103:0/662651071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7075890 msgr2=0x55f3e707a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:12.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7fa0a2640 1 --2- 172.21.15.103:0/662651071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7075890 0x55f3e707a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7f90a0640 1 --2- 172.21.15.103:0/662651071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7075890 0x55f3e707a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7f90a0640 1 -- 172.21.15.103:0/662651071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3e70eef90 con 0x55f3e7068c40 2024-09-19T07:03:12.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7f90a0640 1 --2- 172.21.15.103:0/662651071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7079b00 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40981a0 tx=0x7fc7e409aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f3714e8b5daa33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:12.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:12.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7dd7fa640 1 -- 172.21.15.103:0/662651071 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4064070 con 0x55f3e7068c40 2024-09-19T07:03:12.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7dd7fa640 1 -- 172.21.15.103:0/662651071 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7fc7e4097a90 con 0x55f3e7068c40 2024-09-19T07:03:12.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7fdede740 1 -- 172.21.15.103:0/662651071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7e4010460 con 0x55f3e7068c40 2024-09-19T07:03:12.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7dd7fa640 1 -- 172.21.15.103:0/662651071 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc7e4020070 con 0x55f3e7068c40 2024-09-19T07:03:12.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7fdede740 10 client.20298 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:12.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7fdede740 1 -- 172.21.15.103:0/662651071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f3e70eef90 con 0x55f3e7068c40 2024-09-19T07:03:12.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7dd7fa640 1 -- 172.21.15.103:0/662651071 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e4010460 con 0x55f3e7068c40 2024-09-19T07:03:12.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7dd7fa640 1 -- 172.21.15.103:0/662651071 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc7e40b2070 con 0x55f3e7068c40 2024-09-19T07:03:12.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7fdede740 10 client.20298 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:12.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.283+0000 7fc7fdede740 10 client.20298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:12.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.284+0000 7fc7fdede740 20 client.20298 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:12.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.284+0000 7fc7fdede740 1 --2- 172.21.15.103:0/662651071 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f3e70f0590 0x55f3e70f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.284+0000 7fc7fdede740 4 client.20298 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:03:12.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.284+0000 7fc7fdede740 1 -- 172.21.15.103:0/662651071 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f3e6fd5d60 con 0x55f3e70f0590 2024-09-19T07:03:12.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.284+0000 7fc7f98a1640 1 --2- 172.21.15.103:0/662651071 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f3e70f0590 0x55f3e70f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.284+0000 7fc7f98a1640 1 --2- 172.21.15.103:0/662651071 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f3e70f0590 0x55f3e70f29b0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffe6ba3009fc7c26 server_cookie=7cd42f28a9796524 in_seq=0 out_seq=0 2024-09-19T07:03:12.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.284+0000 7fc7dd7fa640 10 client.20298 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:12.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7dd7fa640 1 -- 172.21.15.103:0/662651071 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f3e6fd5d60 con 0x55f3e70f0590 2024-09-19T07:03:12.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7dd7fa640 10 client.20298 handle_command_reply: tid=0 2024-09-19T07:03:12.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7fdede740 1 client.20298 shutdown 2024-09-19T07:03:12.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7fdede740 1 -- 172.21.15.103:0/662651071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 msgr2=0x55f3e7079b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:12.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7fdede740 1 --2- 172.21.15.103:0/662651071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7079b00 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40981a0 tx=0x7fc7e409aea0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7fdede740 1 -- 172.21.15.103:0/662651071 shutdown_connections 2024-09-19T07:03:12.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7fdede740 1 --2- 172.21.15.103:0/662651071 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f3e70f0590 0x55f3e70f29b0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7fdede740 1 --2- 172.21.15.103:0/662651071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3e7075890 0x55f3e707a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7fdede740 1 --2- 172.21.15.103:0/662651071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e7068c40 0x55f3e7079b00 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7dd7fa640 0 client.20298 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:12.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7fdede740 1 --2- 172.21.15.103:0/662651071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3e7068320 0x55f3e70795c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7fdede740 1 -- 172.21.15.103:0/662651071 >> 172.21.15.103:0/662651071 conn(0x55f3e7064a30 msgr2=0x55f3e7073cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:12.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7fdede740 1 -- 172.21.15.103:0/662651071 shutdown_connections 2024-09-19T07:03:12.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7fdede740 1 -- 172.21.15.103:0/662651071 wait complete. 2024-09-19T07:03:12.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7fdede740 20 client.20298 trim_cache size 0 max 16384 2024-09-19T07:03:12.300 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:03:12.300 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:03:12.300 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:03:12.300 INFO:teuthology.orchestra.run.smithi103.stdout: "8a457ac8-8e33-4cff-a968-8c2e6435a530": { 2024-09-19T07:03:12.300 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:03:12.300 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "8a457ac8-8e33-4cff-a968-8c2e6435a530", 2024-09-19T07:03:12.300 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:03:12.300 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:03:12.300 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:03:12.300 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:03:12.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7fbb05640 1 -- 172.21.15.103:0/2667429779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7e00a33b0 msgr2=0x7fc7e00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:12.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7fbb05640 1 --2- 172.21.15.103:0/2667429779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7e00a33b0 0x7fc7e00a5870 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec006d20 tx=0x7fc7ec005f40 comp rx=0 tx=0).stop 2024-09-19T07:03:12.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7fbb05640 1 -- 172.21.15.103:0/2667429779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7f4101420 msgr2=0x7fc7f41978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:12.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.285+0000 7fc7fbb05640 1 --2- 172.21.15.103:0/2667429779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7f4101420 0x7fc7f41978a0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400d950 tx=0x7fc7e400de10 comp rx=0 tx=0).stop 2024-09-19T07:03:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.286+0000 7fc7fbb05640 1 -- 172.21.15.103:0/2667429779 shutdown_connections 2024-09-19T07:03:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.286+0000 7fc7fbb05640 1 --2- 172.21.15.103:0/2667429779 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7e00a33b0 0x7fc7e00a5870 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.286+0000 7fc7fbb05640 1 --2- 172.21.15.103:0/2667429779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7f4106a80 0x7fc7f419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.286+0000 7fc7fbb05640 1 --2- 172.21.15.103:0/2667429779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7f4101420 0x7fc7f41978a0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.286+0000 7fc7fbb05640 1 --2- 172.21.15.103:0/2667429779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f4100a80 0x7fc7f4197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.286+0000 7fc7fbb05640 1 -- 172.21.15.103:0/2667429779 >> 172.21.15.103:0/2667429779 conn(0x7fc7f40fc7a0 msgr2=0x7fc7f40fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.286+0000 7fc7fbb05640 1 -- 172.21.15.103:0/2667429779 shutdown_connections 2024-09-19T07:03:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:12.286+0000 7fc7fbb05640 1 -- 172.21.15.103:0/2667429779 wait complete. 2024-09-19T07:03:12.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:11 smithi174 ceph-mon[35637]: pgmap v436: 97 pgs: 97 active+clean; 3.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 97 KiB/s wr, 35 op/s 2024-09-19T07:03:12.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:11 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T07:03:12.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:11 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:03:12.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:11 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T07:03:12.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:11 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:03:12.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:11 smithi174 ceph-mon[35637]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T07:03:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.954+0000 7f555ce06640 1 -- 172.21.15.103:0/2827097744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558106ea0 msgr2=0x7f555810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.954+0000 7f555ce06640 1 --2- 172.21.15.103:0/2827097744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558106ea0 0x7f555810d910 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f5550009ca0 tx=0x7f555002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.955+0000 7f555ce06640 1 -- 172.21.15.103:0/2827097744 shutdown_connections 2024-09-19T07:03:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.955+0000 7f555ce06640 1 --2- 172.21.15.103:0/2827097744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558106ea0 0x7f555810d910 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.955+0000 7f555ce06640 1 --2- 172.21.15.103:0/2827097744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5558102310 0x7f5558106740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.955+0000 7f555ce06640 1 --2- 172.21.15.103:0/2827097744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558101960 0x7f5558101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.955+0000 7f555ce06640 1 -- 172.21.15.103:0/2827097744 >> 172.21.15.103:0/2827097744 conn(0x7f5558075db0 msgr2=0x7f55580761c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.955+0000 7f555ce06640 1 -- 172.21.15.103:0/2827097744 shutdown_connections 2024-09-19T07:03:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.955+0000 7f555ce06640 1 -- 172.21.15.103:0/2827097744 wait complete. 2024-09-19T07:03:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.955+0000 7f555ce06640 1 Processor -- start 2024-09-19T07:03:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.955+0000 7f555ce06640 1 -- start start 2024-09-19T07:03:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.955+0000 7f555ce06640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558101960 0x7f5558198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.955+0000 7f555ce06640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558106ea0 0x7f55581989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f555ce06640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f555819cd20 0x7f555819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f555ce06640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5558110ee0 con 0x7f5558106ea0 2024-09-19T07:03:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f555ce06640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5558110d60 con 0x7f555819cd20 2024-09-19T07:03:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f555ce06640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5558111060 con 0x7f5558101960 2024-09-19T07:03:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f5555d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558106ea0 0x7f55581989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f5555d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558106ea0 0x7f55581989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38172/0 (socket says 172.21.15.103:38172) 2024-09-19T07:03:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f5555d9b640 1 -- 172.21.15.103:0/2573414765 learned_addr learned my addr 172.21.15.103:0/2573414765 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f555659c640 1 --2- 172.21.15.103:0/2573414765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558101960 0x7f5558198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f5556d9d640 1 --2- 172.21.15.103:0/2573414765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f555819cd20 0x7f555819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f5555d9b640 1 -- 172.21.15.103:0/2573414765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558101960 msgr2=0x7f5558198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f5555d9b640 1 --2- 172.21.15.103:0/2573414765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558101960 0x7f5558198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f5555d9b640 1 -- 172.21.15.103:0/2573414765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f555819cd20 msgr2=0x7f555819d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f5555d9b640 1 --2- 172.21.15.103:0/2573414765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f555819cd20 0x7f555819d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f5555d9b640 1 -- 172.21.15.103:0/2573414765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555819d810 con 0x7f5558106ea0 2024-09-19T07:03:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f5555d9b640 1 --2- 172.21.15.103:0/2573414765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558106ea0 0x7f55581989a0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f554c00b450 tx=0x7f554c00b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f55477fe640 1 -- 172.21.15.103:0/2573414765 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c015020 con 0x7f5558106ea0 2024-09-19T07:03:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f55477fe640 1 -- 172.21.15.103:0/2573414765 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f554c011d10 con 0x7f5558106ea0 2024-09-19T07:03:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f555ce06640 1 -- 172.21.15.103:0/2573414765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555819db00 con 0x7f5558106ea0 2024-09-19T07:03:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f55477fe640 1 -- 172.21.15.103:0/2573414765 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c00f2e0 con 0x7f5558106ea0 2024-09-19T07:03:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.956+0000 7f555ce06640 1 -- 172.21.15.103:0/2573414765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55581a5260 con 0x7f5558106ea0 2024-09-19T07:03:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.958+0000 7f55477fe640 1 -- 172.21.15.103:0/2573414765 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f554c018020 con 0x7f5558106ea0 2024-09-19T07:03:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.959+0000 7f555fc82740 1 Processor -- start 2024-09-19T07:03:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.959+0000 7f555fc82740 1 -- start start 2024-09-19T07:03:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.959+0000 7f555fc82740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c807af0 0x56525c807ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.959+0000 7f555fc82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c808410 0x56525c812770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.959+0000 7f555fc82740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c8150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.959+0000 7f555fc82740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56525c76c2f0 con 0x56525c812cb0 2024-09-19T07:03:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.959+0000 7f555fc82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56525c714270 con 0x56525c808410 2024-09-19T07:03:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.959+0000 7f555fc82740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56525c7682b0 con 0x56525c807af0 2024-09-19T07:03:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.959+0000 7f55477fe640 1 --2- 172.21.15.103:0/2573414765 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55400a3460 0x7f55400a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.959+0000 7f5555d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c8150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.959+0000 7f5555d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c8150a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38180/0 (socket says 172.21.15.103:38180) 2024-09-19T07:03:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.959+0000 7f5555d9b640 1 -- 172.21.15.103:0/1381188261 learned_addr learned my addr 172.21.15.103:0/1381188261 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.960+0000 7f5556d9d640 1 --2- 172.21.15.103:0/2573414765 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55400a3460 0x7f55400a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.960+0000 7f555659c640 1 --2- 172.21.15.103:0/1381188261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c808410 0x56525c812770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.960+0000 7f5556d9d640 1 --2- 172.21.15.103:0/1381188261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c807af0 0x56525c807ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.960+0000 7f55477fe640 1 -- 172.21.15.103:0/2573414765 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f554c0cf550 con 0x7f5558106ea0 2024-09-19T07:03:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.960+0000 7f5555d9b640 1 -- 172.21.15.103:0/1381188261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c807af0 msgr2=0x56525c807ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.960+0000 7f5555d9b640 1 --2- 172.21.15.103:0/1381188261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c807af0 0x56525c807ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.960+0000 7f5555d9b640 1 -- 172.21.15.103:0/1381188261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c808410 msgr2=0x56525c812770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.960+0000 7f5555d9b640 1 --2- 172.21.15.103:0/1381188261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c808410 0x56525c812770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.960+0000 7f5555d9b640 1 -- 172.21.15.103:0/1381188261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56525c713340 con 0x56525c812cb0 2024-09-19T07:03:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.960+0000 7f5556d9d640 1 --2- 172.21.15.103:0/2573414765 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55400a3460 0x7f55400a5920 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f555002f540 tx=0x7f555002f4b0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.960+0000 7f5555d9b640 1 --2- 172.21.15.103:0/1381188261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c8150a0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f554c09bae0 tx=0x7f554c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece83e0f52e92e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.960+0000 7f5544ff9640 1 -- 172.21.15.103:0/1381188261 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c09b460 con 0x56525c812cb0 2024-09-19T07:03:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.960+0000 7f5544ff9640 1 -- 172.21.15.103:0/1381188261 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f554c09c020 con 0x56525c812cb0 2024-09-19T07:03:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.960+0000 7f5544ff9640 1 -- 172.21.15.103:0/1381188261 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c0860e0 con 0x56525c812cb0 2024-09-19T07:03:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.961+0000 7f555fc82740 1 -- 172.21.15.103:0/1381188261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 msgr2=0x56525c8150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.961+0000 7f555fc82740 1 --2- 172.21.15.103:0/1381188261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c8150a0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f554c09bae0 tx=0x7f554c0047d0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.961+0000 7f555fc82740 1 -- 172.21.15.103:0/1381188261 shutdown_connections 2024-09-19T07:03:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.961+0000 7f555fc82740 1 --2- 172.21.15.103:0/1381188261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c8150a0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.961+0000 7f555fc82740 1 --2- 172.21.15.103:0/1381188261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c808410 0x56525c812770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.961+0000 7f555fc82740 1 --2- 172.21.15.103:0/1381188261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c807af0 0x56525c807ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.961+0000 7f555fc82740 1 -- 172.21.15.103:0/1381188261 >> 172.21.15.103:0/1381188261 conn(0x56525c804060 msgr2=0x56525c804440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.961+0000 7f555fc82740 1 -- 172.21.15.103:0/1381188261 shutdown_connections 2024-09-19T07:03:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.961+0000 7f555fc82740 1 -- 172.21.15.103:0/1381188261 wait complete. 2024-09-19T07:03:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.961+0000 7f555fc82740 1 -- start start 2024-09-19T07:03:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.961+0000 7f555fc82740 1 Processor -- start 2024-09-19T07:03:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f555fc82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c807af0 0x56525c80fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f555fc82740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c808410 0x56525c8100d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f555fc82740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c810610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f555fc82740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56525c815880 con 0x56525c812cb0 2024-09-19T07:03:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f555fc82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56525c6f5290 con 0x56525c807af0 2024-09-19T07:03:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f555fc82740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56525c815a00 con 0x56525c808410 2024-09-19T07:03:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f555fc82740 10 client.0 _ll_register_callbacks cb 0x56525c726140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f5556d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c810610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f5556d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c810610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38190/0 (socket says 172.21.15.103:38190) 2024-09-19T07:03:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f5556d9d640 1 -- 172.21.15.103:0/3073630460 learned_addr learned my addr 172.21.15.103:0/3073630460 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f5555d9b640 1 --2- 172.21.15.103:0/3073630460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c808410 0x56525c8100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f555659c640 1 --2- 172.21.15.103:0/3073630460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c807af0 0x56525c80fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f5556d9d640 1 -- 172.21.15.103:0/3073630460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c808410 msgr2=0x56525c8100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f5556d9d640 1 --2- 172.21.15.103:0/3073630460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c808410 0x56525c8100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f5556d9d640 1 -- 172.21.15.103:0/3073630460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c807af0 msgr2=0x56525c80fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f5556d9d640 1 --2- 172.21.15.103:0/3073630460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c807af0 0x56525c80fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f5556d9d640 1 -- 172.21.15.103:0/3073630460 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56525c47f210 con 0x56525c812cb0 2024-09-19T07:03:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f5555d9b640 1 --2- 172.21.15.103:0/3073630460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c808410 0x56525c8100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f5556d9d640 1 --2- 172.21.15.103:0/3073630460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c810610 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f5550031990 tx=0x7f5550047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fddd2099899e110e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f553dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f553dffb640 1 -- 172.21.15.103:0/3073630460 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555004f030 con 0x56525c812cb0 2024-09-19T07:03:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f553dffb640 1 -- 172.21.15.103:0/3073630460 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f5550046040 con 0x56525c812cb0 2024-09-19T07:03:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f555fc82740 1 -- 172.21.15.103:0/3073630460 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f554c09c020 con 0x56525c812cb0 2024-09-19T07:03:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f553dffb640 1 -- 172.21.15.103:0/3073630460 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5550004030 con 0x56525c812cb0 2024-09-19T07:03:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f555fc82740 10 client.20310 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.962+0000 7f555fc82740 1 -- 172.21.15.103:0/3073630460 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56525c54b620 con 0x56525c812cb0 2024-09-19T07:03:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.963+0000 7f553dffb640 1 -- 172.21.15.103:0/3073630460 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f554c09c020 con 0x56525c812cb0 2024-09-19T07:03:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.963+0000 7f553dffb640 1 -- 172.21.15.103:0/3073630460 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f555004a070 con 0x56525c812cb0 2024-09-19T07:03:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.963+0000 7f555fc82740 10 client.20310 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.963+0000 7f555fc82740 10 client.20310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.963+0000 7f555fc82740 20 client.20310 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.963+0000 7f555fc82740 1 --2- 172.21.15.103:0/3073630460 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56525c822c30 0x56525c825060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.963+0000 7f555fc82740 4 client.20310 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.963+0000 7f555fc82740 1 -- 172.21.15.103:0/3073630460 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56525c779a20 con 0x56525c822c30 2024-09-19T07:03:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.963+0000 7f555659c640 1 --2- 172.21.15.103:0/3073630460 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56525c822c30 0x56525c825060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.964+0000 7f555659c640 1 --2- 172.21.15.103:0/3073630460 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56525c822c30 0x56525c825060 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f536edfc011adb server_cookie=45f7f92a9e301427 in_seq=0 out_seq=0 2024-09-19T07:03:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.964+0000 7f553dffb640 10 client.20310 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.964+0000 7f553dffb640 1 -- 172.21.15.103:0/3073630460 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56525c779a20 con 0x56525c822c30 2024-09-19T07:03:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.964+0000 7f553dffb640 10 client.20310 handle_command_reply: tid=0 2024-09-19T07:03:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.964+0000 7f555fc82740 1 client.20310 shutdown 2024-09-19T07:03:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.964+0000 7f555fc82740 1 -- 172.21.15.103:0/3073630460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 msgr2=0x56525c810610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.964+0000 7f555fc82740 1 --2- 172.21.15.103:0/3073630460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c810610 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f5550031990 tx=0x7f5550047000 comp rx=0 tx=0).stop 2024-09-19T07:03:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.965+0000 7f555fc82740 1 -- 172.21.15.103:0/3073630460 shutdown_connections 2024-09-19T07:03:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.965+0000 7f555fc82740 1 --2- 172.21.15.103:0/3073630460 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56525c822c30 0x56525c825060 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.965+0000 7f555fc82740 1 --2- 172.21.15.103:0/3073630460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c810610 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.965+0000 7f555fc82740 1 --2- 172.21.15.103:0/3073630460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c808410 0x56525c8100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.965+0000 7f553dffb640 0 client.20310 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.965+0000 7f555fc82740 1 --2- 172.21.15.103:0/3073630460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c807af0 0x56525c80fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.965+0000 7f555fc82740 1 -- 172.21.15.103:0/3073630460 >> 172.21.15.103:0/3073630460 conn(0x56525c804060 msgr2=0x56525c809d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.965+0000 7f555fc82740 1 -- 172.21.15.103:0/3073630460 shutdown_connections 2024-09-19T07:03:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.965+0000 7f555fc82740 1 -- 172.21.15.103:0/3073630460 wait complete. 2024-09-19T07:03:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.965+0000 7f555fc82740 20 client.20310 trim_cache size 0 max 16384 2024-09-19T07:03:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.978+0000 7f555fc82740 1 Processor -- start 2024-09-19T07:03:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.978+0000 7f555fc82740 1 -- start start 2024-09-19T07:03:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f555fc82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c807af0 0x56525c80a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f555fc82740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c808410 0x56525c810440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f555fc82740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c82a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f555fc82740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55581ad900 con 0x56525c812cb0 2024-09-19T07:03:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f555fc82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55581af440 con 0x56525c807af0 2024-09-19T07:03:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f555fc82740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55581adfd0 con 0x56525c808410 2024-09-19T07:03:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f5556d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c82a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f5556d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c82a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38196/0 (socket says 172.21.15.103:38196) 2024-09-19T07:03:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f5556d9d640 1 -- 172.21.15.103:0/3179811189 learned_addr learned my addr 172.21.15.103:0/3179811189 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f5556d9d640 1 -- 172.21.15.103:0/3179811189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c808410 msgr2=0x56525c810440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f5556d9d640 1 --2- 172.21.15.103:0/3179811189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c808410 0x56525c810440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f5556d9d640 1 -- 172.21.15.103:0/3179811189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c807af0 msgr2=0x56525c80a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f555659c640 1 --2- 172.21.15.103:0/3179811189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c807af0 0x56525c80a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f5556d9d640 1 --2- 172.21.15.103:0/3179811189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c807af0 0x56525c80a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f5556d9d640 1 -- 172.21.15.103:0/3179811189 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56525c8256e0 con 0x56525c812cb0 2024-09-19T07:03:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f5556d9d640 1 --2- 172.21.15.103:0/3179811189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c82a970 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f55500315a0 tx=0x7f555002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17904d1a272b32fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f553ffff640 1 -- 172.21.15.103:0/3179811189 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555004f070 con 0x56525c812cb0 2024-09-19T07:03:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f553ffff640 1 -- 172.21.15.103:0/3179811189 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5550036040 con 0x56525c812cb0 2024-09-19T07:03:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.979+0000 7f553ffff640 1 -- 172.21.15.103:0/3179811189 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550004430 con 0x56525c812cb0 2024-09-19T07:03:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.980+0000 7f555fc82740 1 -- 172.21.15.103:0/3179811189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 msgr2=0x56525c82a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.980+0000 7f555fc82740 1 --2- 172.21.15.103:0/3179811189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c82a970 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f55500315a0 tx=0x7f555002f380 comp rx=0 tx=0).stop 2024-09-19T07:03:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.980+0000 7f555fc82740 1 -- 172.21.15.103:0/3179811189 shutdown_connections 2024-09-19T07:03:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.980+0000 7f555fc82740 1 --2- 172.21.15.103:0/3179811189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c812cb0 0x56525c82a970 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.980+0000 7f555fc82740 1 --2- 172.21.15.103:0/3179811189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c808410 0x56525c810440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.980+0000 7f555fc82740 1 --2- 172.21.15.103:0/3179811189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c807af0 0x56525c80a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.980+0000 7f555fc82740 1 -- 172.21.15.103:0/3179811189 >> 172.21.15.103:0/3179811189 conn(0x56525c804060 msgr2=0x56525c811c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.980+0000 7f555fc82740 1 -- 172.21.15.103:0/3179811189 shutdown_connections 2024-09-19T07:03:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.980+0000 7f555fc82740 1 -- 172.21.15.103:0/3179811189 wait complete. 2024-09-19T07:03:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.980+0000 7f555fc82740 1 -- start start 2024-09-19T07:03:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.980+0000 7f555fc82740 1 Processor -- start 2024-09-19T07:03:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f555fc82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c807af0 0x56525c81b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f555fc82740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c808410 0x56525c816790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f555fc82740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c812cb0 0x56525c816cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f555fc82740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55581aed70 con 0x56525c808410 2024-09-19T07:03:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f555fc82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55581ae6a0 con 0x56525c807af0 2024-09-19T07:03:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f555fc82740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55581ab900 con 0x56525c812cb0 2024-09-19T07:03:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f555fc82740 10 client.0 _ll_register_callbacks cb 0x7f5550031d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f5555d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c808410 0x56525c816790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f5555d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c808410 0x56525c816790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38204/0 (socket says 172.21.15.103:38204) 2024-09-19T07:03:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f5555d9b640 1 -- 172.21.15.103:0/870250969 learned_addr learned my addr 172.21.15.103:0/870250969 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f555659c640 1 --2- 172.21.15.103:0/870250969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c807af0 0x56525c81b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f5555d9b640 1 -- 172.21.15.103:0/870250969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c812cb0 msgr2=0x56525c816cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f5555d9b640 1 --2- 172.21.15.103:0/870250969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c812cb0 0x56525c816cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f5555d9b640 1 -- 172.21.15.103:0/870250969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c807af0 msgr2=0x56525c81b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f5555d9b640 1 --2- 172.21.15.103:0/870250969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c807af0 0x56525c81b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f5555d9b640 1 -- 172.21.15.103:0/870250969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56525c804cf0 con 0x56525c808410 2024-09-19T07:03:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f5555d9b640 1 --2- 172.21.15.103:0/870250969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c808410 0x56525c816790 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f554c0108c0 tx=0x7f554c0cf8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722c06ff08504011 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f553dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f553dffb640 1 -- 172.21.15.103:0/870250969 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c09b920 con 0x56525c808410 2024-09-19T07:03:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f553dffb640 1 -- 172.21.15.103:0/870250969 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f554c017b50 con 0x56525c808410 2024-09-19T07:03:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f555fc82740 1 -- 172.21.15.103:0/870250969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5550036040 con 0x56525c808410 2024-09-19T07:03:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f553dffb640 1 -- 172.21.15.103:0/870250969 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f554c09b280 con 0x56525c808410 2024-09-19T07:03:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f555fc82740 10 client.20316 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.981+0000 7f555fc82740 1 -- 172.21.15.103:0/870250969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56525c891c10 con 0x56525c808410 2024-09-19T07:03:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.982+0000 7f553dffb640 1 -- 172.21.15.103:0/870250969 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5550036040 con 0x56525c808410 2024-09-19T07:03:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.982+0000 7f553dffb640 1 -- 172.21.15.103:0/870250969 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f554c060be0 con 0x56525c808410 2024-09-19T07:03:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.982+0000 7f555fc82740 10 client.20316 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.982+0000 7f555fc82740 10 client.20316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.982+0000 7f555fc82740 20 client.20316 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.982+0000 7f555fc82740 1 --2- 172.21.15.103:0/870250969 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56525c891fa0 0x56525c894390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.982+0000 7f555fc82740 4 client.20316 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:03:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.982+0000 7f555fc82740 1 -- 172.21.15.103:0/870250969 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56525c773d70 con 0x56525c891fa0 2024-09-19T07:03:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.982+0000 7f555659c640 1 --2- 172.21.15.103:0/870250969 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56525c891fa0 0x56525c894390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.982+0000 7f555659c640 1 --2- 172.21.15.103:0/870250969 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56525c891fa0 0x56525c894390 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d6f89ef55d4300c server_cookie=ee49fe953ead9ca1 in_seq=0 out_seq=0 2024-09-19T07:03:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.982+0000 7f553dffb640 10 client.20316 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.996+0000 7f553dffb640 1 -- 172.21.15.103:0/870250969 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56525c773d70 con 0x56525c891fa0 2024-09-19T07:03:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.996+0000 7f553dffb640 10 client.20316 handle_command_reply: tid=0 2024-09-19T07:03:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.996+0000 7f555fc82740 1 client.20316 shutdown 2024-09-19T07:03:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.996+0000 7f555fc82740 1 -- 172.21.15.103:0/870250969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c808410 msgr2=0x56525c816790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.996+0000 7f555fc82740 1 --2- 172.21.15.103:0/870250969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c808410 0x56525c816790 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f554c0108c0 tx=0x7f554c0cf8c0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.996+0000 7f555fc82740 1 -- 172.21.15.103:0/870250969 shutdown_connections 2024-09-19T07:03:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.996+0000 7f555fc82740 1 --2- 172.21.15.103:0/870250969 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56525c891fa0 0x56525c894390 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.996+0000 7f555fc82740 1 --2- 172.21.15.103:0/870250969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56525c812cb0 0x56525c816cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.996+0000 7f553dffb640 0 client.20316 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.996+0000 7f555fc82740 1 --2- 172.21.15.103:0/870250969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56525c808410 0x56525c816790 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.996+0000 7f555fc82740 1 --2- 172.21.15.103:0/870250969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56525c807af0 0x56525c81b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.996+0000 7f555fc82740 1 -- 172.21.15.103:0/870250969 >> 172.21.15.103:0/870250969 conn(0x56525c804060 msgr2=0x56525c8029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.996+0000 7f555fc82740 1 -- 172.21.15.103:0/870250969 shutdown_connections 2024-09-19T07:03:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.996+0000 7f555fc82740 1 -- 172.21.15.103:0/870250969 wait complete. 2024-09-19T07:03:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.996+0000 7f555fc82740 20 client.20316 trim_cache size 0 max 16384 2024-09-19T07:03:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:03:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:03:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:03:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:03:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.997+0000 7f555ce06640 1 -- 172.21.15.103:0/2573414765 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55400a3460 msgr2=0x7f55400a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.997+0000 7f555ce06640 1 --2- 172.21.15.103:0/2573414765 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55400a3460 0x7f55400a5920 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f555002f540 tx=0x7f555002f4b0 comp rx=0 tx=0).stop 2024-09-19T07:03:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.997+0000 7f555ce06640 1 -- 172.21.15.103:0/2573414765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558106ea0 msgr2=0x7f55581989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.997+0000 7f555ce06640 1 --2- 172.21.15.103:0/2573414765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558106ea0 0x7f55581989a0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f554c00b450 tx=0x7f554c00b850 comp rx=0 tx=0).stop 2024-09-19T07:03:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.997+0000 7f555ce06640 1 -- 172.21.15.103:0/2573414765 shutdown_connections 2024-09-19T07:03:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.997+0000 7f555ce06640 1 --2- 172.21.15.103:0/2573414765 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f55400a3460 0x7f55400a5920 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.997+0000 7f555ce06640 1 --2- 172.21.15.103:0/2573414765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f555819cd20 0x7f555819d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.997+0000 7f555ce06640 1 --2- 172.21.15.103:0/2573414765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5558106ea0 0x7f55581989a0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.997+0000 7f555ce06640 1 --2- 172.21.15.103:0/2573414765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558101960 0x7f5558198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.997+0000 7f555ce06640 1 -- 172.21.15.103:0/2573414765 >> 172.21.15.103:0/2573414765 conn(0x7f5558075db0 msgr2=0x7f55580fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.997+0000 7f555ce06640 1 -- 172.21.15.103:0/2573414765 shutdown_connections 2024-09-19T07:03:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:12.997+0000 7f555ce06640 1 -- 172.21.15.103:0/2573414765 wait complete. 2024-09-19T07:03:13.028 INFO:tasks.cephfs.filesystem:scrub status for tag:8a457ac8-8e33-4cff-a968-8c2e6435a530 - {'path': '/', 'tag': '8a457ac8-8e33-4cff-a968-8c2e6435a530', 'options': 'recursive,force'} 2024-09-19T07:03:14.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:13 smithi103 ceph-mon[31158]: pgmap v438: 97 pgs: 97 active+clean; 3.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 121 KiB/s wr, 44 op/s 2024-09-19T07:03:14.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:13 smithi134 ceph-mon[35356]: pgmap v438: 97 pgs: 97 active+clean; 3.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 121 KiB/s wr, 44 op/s 2024-09-19T07:03:14.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:13 smithi174 ceph-mon[35637]: pgmap v438: 97 pgs: 97 active+clean; 3.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 121 KiB/s wr, 44 op/s 2024-09-19T07:03:16.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:15 smithi103 ceph-mon[31158]: pgmap v439: 97 pgs: 97 active+clean; 3.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-09-19T07:03:16.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:15 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T07:03:16.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:15 smithi134 ceph-mon[35356]: pgmap v439: 97 pgs: 97 active+clean; 3.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-09-19T07:03:16.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:15 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T07:03:16.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:15 smithi174 ceph-mon[35637]: pgmap v439: 97 pgs: 97 active+clean; 3.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-09-19T07:03:16.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:15 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T07:03:18.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:17 smithi103 ceph-mon[31158]: pgmap v440: 97 pgs: 97 active+clean; 4.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-09-19T07:03:18.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:17 smithi134 ceph-mon[35356]: pgmap v440: 97 pgs: 97 active+clean; 4.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-09-19T07:03:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:17 smithi174 ceph-mon[35637]: pgmap v440: 97 pgs: 97 active+clean; 4.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-09-19T07:03:20.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:19 smithi103 ceph-mon[31158]: pgmap v441: 97 pgs: 97 active+clean; 4.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-09-19T07:03:20.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:19 smithi134 ceph-mon[35356]: pgmap v441: 97 pgs: 97 active+clean; 4.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-09-19T07:03:20.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:19 smithi174 ceph-mon[35637]: pgmap v441: 97 pgs: 97 active+clean; 4.1 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-09-19T07:03:22.135 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:21 smithi174 ceph-mon[35637]: pgmap v442: 97 pgs: 97 active+clean; 4.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 20 op/s 2024-09-19T07:03:22.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:21 smithi103 ceph-mon[31158]: pgmap v442: 97 pgs: 97 active+clean; 4.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 20 op/s 2024-09-19T07:03:22.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:21 smithi134 ceph-mon[35356]: pgmap v442: 97 pgs: 97 active+clean; 4.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 20 op/s 2024-09-19T07:03:24.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:23 smithi103 ceph-mon[31158]: pgmap v443: 97 pgs: 97 active+clean; 4.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-09-19T07:03:24.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:23 smithi134 ceph-mon[35356]: pgmap v443: 97 pgs: 97 active+clean; 4.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-09-19T07:03:24.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:23 smithi174 ceph-mon[35637]: pgmap v443: 97 pgs: 97 active+clean; 4.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-09-19T07:03:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.095+0000 7f418fbfd640 1 -- 172.21.15.103:0/174452340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4188106a00 msgr2=0x7f4188106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.095+0000 7f418fbfd640 1 --2- 172.21.15.103:0/174452340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4188106a00 0x7f4188106de0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f4184009cd0 tx=0x7f418402a8e0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.095+0000 7f418fbfd640 1 -- 172.21.15.103:0/174452340 shutdown_connections 2024-09-19T07:03:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.095+0000 7f418fbfd640 1 --2- 172.21.15.103:0/174452340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4188101390 0x7f418810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.095+0000 7f418fbfd640 1 --2- 172.21.15.103:0/174452340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41881009f0 0x7f4188100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.095+0000 7f418fbfd640 1 --2- 172.21.15.103:0/174452340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4188106a00 0x7f4188106de0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.095+0000 7f418fbfd640 1 -- 172.21.15.103:0/174452340 >> 172.21.15.103:0/174452340 conn(0x7f41880fc6b0 msgr2=0x7f41880fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.095+0000 7f418fbfd640 1 -- 172.21.15.103:0/174452340 shutdown_connections 2024-09-19T07:03:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.095+0000 7f418fbfd640 1 -- 172.21.15.103:0/174452340 wait complete. 2024-09-19T07:03:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.095+0000 7f418fbfd640 1 Processor -- start 2024-09-19T07:03:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.095+0000 7f418fbfd640 1 -- start start 2024-09-19T07:03:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418fbfd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41881009f0 0x7f41881991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418fbfd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4188101390 0x7f4188199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418fbfd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4188106a00 0x7f41881a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418fbfd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4188110e40 con 0x7f41881009f0 2024-09-19T07:03:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418fbfd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4188110cc0 con 0x7f4188101390 2024-09-19T07:03:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418fbfd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4188110fc0 con 0x7f4188106a00 2024-09-19T07:03:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418d999640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41881009f0 0x7f41881991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418d999640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41881009f0 0x7f41881991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58354/0 (socket says 172.21.15.103:58354) 2024-09-19T07:03:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418d999640 1 -- 172.21.15.103:0/2232836567 learned_addr learned my addr 172.21.15.103:0/2232836567 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418d198640 1 --2- 172.21.15.103:0/2232836567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4188101390 0x7f4188199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418e19a640 1 --2- 172.21.15.103:0/2232836567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4188106a00 0x7f41881a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418d999640 1 -- 172.21.15.103:0/2232836567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4188106a00 msgr2=0x7f41881a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418d999640 1 --2- 172.21.15.103:0/2232836567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4188106a00 0x7f41881a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418d999640 1 -- 172.21.15.103:0/2232836567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4188101390 msgr2=0x7f4188199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418d999640 1 --2- 172.21.15.103:0/2232836567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4188101390 0x7f4188199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418d999640 1 -- 172.21.15.103:0/2232836567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41881a47b0 con 0x7f41881009f0 2024-09-19T07:03:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418e19a640 1 --2- 172.21.15.103:0/2232836567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4188106a00 0x7f41881a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.096+0000 7f418d999640 1 --2- 172.21.15.103:0/2232836567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41881009f0 0x7f41881991f0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f418402ae00 tx=0x7f4184003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.097+0000 7f417effd640 1 -- 172.21.15.103:0/2232836567 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184004040 con 0x7f41881009f0 2024-09-19T07:03:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.097+0000 7f417effd640 1 -- 172.21.15.103:0/2232836567 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41840046c0 con 0x7f41881009f0 2024-09-19T07:03:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.097+0000 7f417effd640 1 -- 172.21.15.103:0/2232836567 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184046480 con 0x7f41881009f0 2024-09-19T07:03:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.097+0000 7f418fbfd640 1 -- 172.21.15.103:0/2232836567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41881a4980 con 0x7f41881009f0 2024-09-19T07:03:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.097+0000 7f418fbfd640 1 -- 172.21.15.103:0/2232836567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41881a4e40 con 0x7f41881009f0 2024-09-19T07:03:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.099+0000 7f417effd640 1 -- 172.21.15.103:0/2232836567 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f41840305a0 con 0x7f41881009f0 2024-09-19T07:03:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.100+0000 7f4192a7d740 1 Processor -- start 2024-09-19T07:03:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.100+0000 7f4192a7d740 1 -- start start 2024-09-19T07:03:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.100+0000 7f4192a7d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd32daf0 0x55eddd32ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.100+0000 7f4192a7d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32e410 0x55eddd338770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.100+0000 7f4192a7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd338cb0 0x55eddd33b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.100+0000 7f4192a7d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eddd2922f0 con 0x55eddd32e410 2024-09-19T07:03:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.100+0000 7f4192a7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55eddd23a270 con 0x55eddd338cb0 2024-09-19T07:03:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.100+0000 7f4192a7d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eddd28e2b0 con 0x55eddd32daf0 2024-09-19T07:03:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.100+0000 7f418d999640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32e410 0x55eddd338770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f417effd640 1 --2- 172.21.15.103:0/2232836567 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f41740a3460 0x7f41740a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f418d999640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32e410 0x55eddd338770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58364/0 (socket says 172.21.15.103:58364) 2024-09-19T07:03:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f418d999640 1 -- 172.21.15.103:0/4012047433 learned_addr learned my addr 172.21.15.103:0/4012047433 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f418e19a640 1 --2- 172.21.15.103:0/4012047433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd32daf0 0x55eddd32ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f418d198640 1 --2- 172.21.15.103:0/4012047433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd338cb0 0x55eddd33b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f418e19a640 1 --2- 172.21.15.103:0/2232836567 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f41740a3460 0x7f41740a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f417effd640 1 -- 172.21.15.103:0/2232836567 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f418402e030 con 0x7f41881009f0 2024-09-19T07:03:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f418d999640 1 -- 172.21.15.103:0/4012047433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd32daf0 msgr2=0x55eddd32ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f418d999640 1 --2- 172.21.15.103:0/4012047433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd32daf0 0x55eddd32ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f418d999640 1 -- 172.21.15.103:0/4012047433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd338cb0 msgr2=0x55eddd33b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f418d999640 1 --2- 172.21.15.103:0/4012047433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd338cb0 0x55eddd33b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f418d999640 1 -- 172.21.15.103:0/4012047433 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eddd239340 con 0x55eddd32e410 2024-09-19T07:03:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f418d999640 1 --2- 172.21.15.103:0/4012047433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32e410 0x55eddd338770 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f41840c1690 tx=0x7f41840095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf0a185a036b27c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f418e19a640 1 --2- 172.21.15.103:0/2232836567 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f41740a3460 0x7f41740a5920 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4180005fd0 tx=0x7f4180005850 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f4173fff640 1 -- 172.21.15.103:0/4012047433 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41840bd100 con 0x55eddd32e410 2024-09-19T07:03:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f4173fff640 1 -- 172.21.15.103:0/4012047433 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4184036050 con 0x55eddd32e410 2024-09-19T07:03:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.101+0000 7f4173fff640 1 -- 172.21.15.103:0/4012047433 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41840a6aa0 con 0x55eddd32e410 2024-09-19T07:03:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.102+0000 7f4192a7d740 1 -- 172.21.15.103:0/4012047433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32e410 msgr2=0x55eddd338770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.102+0000 7f4192a7d740 1 --2- 172.21.15.103:0/4012047433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32e410 0x55eddd338770 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f41840c1690 tx=0x7f41840095c0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.102+0000 7f4192a7d740 1 -- 172.21.15.103:0/4012047433 shutdown_connections 2024-09-19T07:03:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.102+0000 7f4192a7d740 1 --2- 172.21.15.103:0/4012047433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd338cb0 0x55eddd33b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.102+0000 7f4192a7d740 1 --2- 172.21.15.103:0/4012047433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32e410 0x55eddd338770 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.102+0000 7f4192a7d740 1 --2- 172.21.15.103:0/4012047433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd32daf0 0x55eddd32ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.102+0000 7f4192a7d740 1 -- 172.21.15.103:0/4012047433 >> 172.21.15.103:0/4012047433 conn(0x55eddd32a060 msgr2=0x55eddd32a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.102+0000 7f4192a7d740 1 -- 172.21.15.103:0/4012047433 shutdown_connections 2024-09-19T07:03:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.102+0000 7f4192a7d740 1 -- 172.21.15.103:0/4012047433 wait complete. 2024-09-19T07:03:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.103+0000 7f4192a7d740 1 -- start start 2024-09-19T07:03:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.103+0000 7f4192a7d740 1 Processor -- start 2024-09-19T07:03:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.103+0000 7f4192a7d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32daf0 0x55eddd335ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.103+0000 7f4192a7d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd32e410 0x55eddd3361e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.103+0000 7f4192a7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd338cb0 0x55eddd336720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.103+0000 7f4192a7d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eddd33b640 con 0x55eddd32daf0 2024-09-19T07:03:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.103+0000 7f4192a7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55eddd21b290 con 0x55eddd338cb0 2024-09-19T07:03:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.103+0000 7f4192a7d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eddd33b7c0 con 0x55eddd32e410 2024-09-19T07:03:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.103+0000 7f4192a7d740 10 client.0 _ll_register_callbacks cb 0x55eddd24c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.103+0000 7f418d999640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32daf0 0x55eddd335ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.103+0000 7f418d999640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32daf0 0x55eddd335ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58372/0 (socket says 172.21.15.103:58372) 2024-09-19T07:03:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.103+0000 7f418d999640 1 -- 172.21.15.103:0/4261646367 learned_addr learned my addr 172.21.15.103:0/4261646367 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.103+0000 7f418e19a640 1 --2- 172.21.15.103:0/4261646367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd338cb0 0x55eddd336720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f418d198640 1 --2- 172.21.15.103:0/4261646367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd32e410 0x55eddd3361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f418d999640 1 -- 172.21.15.103:0/4261646367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd32e410 msgr2=0x55eddd3361e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f418d999640 1 --2- 172.21.15.103:0/4261646367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd32e410 0x55eddd3361e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f418d999640 1 -- 172.21.15.103:0/4261646367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd338cb0 msgr2=0x55eddd336720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f418d999640 1 --2- 172.21.15.103:0/4261646367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd338cb0 0x55eddd336720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f418d999640 1 -- 172.21.15.103:0/4261646367 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eddcfa5210 con 0x55eddd32daf0 2024-09-19T07:03:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f418d999640 1 --2- 172.21.15.103:0/4261646367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32daf0 0x55eddd335ca0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f41840eecc0 tx=0x7f41840b9b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e88c57eb09a2938 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f41717fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f41717fa640 1 -- 172.21.15.103:0/4261646367 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41840bdea0 con 0x55eddd32daf0 2024-09-19T07:03:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f41717fa640 1 -- 172.21.15.103:0/4261646367 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f41840c6190 con 0x55eddd32daf0 2024-09-19T07:03:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f4192a7d740 1 -- 172.21.15.103:0/4261646367 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4184036050 con 0x55eddd32daf0 2024-09-19T07:03:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f41717fa640 1 -- 172.21.15.103:0/4261646367 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4184030890 con 0x55eddd32daf0 2024-09-19T07:03:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f4192a7d740 10 client.20328 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f4192a7d740 1 -- 172.21.15.103:0/4261646367 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eddd071620 con 0x55eddd32daf0 2024-09-19T07:03:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f41717fa640 1 -- 172.21.15.103:0/4261646367 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4184036050 con 0x55eddd32daf0 2024-09-19T07:03:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f41717fa640 1 -- 172.21.15.103:0/4261646367 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f41840f1030 con 0x55eddd32daf0 2024-09-19T07:03:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f4192a7d740 10 client.20328 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f4192a7d740 10 client.20328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f4192a7d740 20 client.20328 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.104+0000 7f4192a7d740 1 --2- 172.21.15.103:0/4261646367 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eddd3488a0 0x55eddd34ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.105+0000 7f4192a7d740 4 client.20328 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.105+0000 7f4192a7d740 1 -- 172.21.15.103:0/4261646367 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eddd29fa20 con 0x55eddd3488a0 2024-09-19T07:03:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.105+0000 7f418d198640 1 --2- 172.21.15.103:0/4261646367 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eddd3488a0 0x55eddd34ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.105+0000 7f41717fa640 10 client.20328 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.105+0000 7f418d198640 1 --2- 172.21.15.103:0/4261646367 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eddd3488a0 0x55eddd34ac90 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5d152330b95999d server_cookie=2b1a192a506c73a6 in_seq=0 out_seq=0 2024-09-19T07:03:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.106+0000 7f41717fa640 1 -- 172.21.15.103:0/4261646367 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eddd29fa20 con 0x55eddd3488a0 2024-09-19T07:03:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.106+0000 7f41717fa640 10 client.20328 handle_command_reply: tid=0 2024-09-19T07:03:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.106+0000 7f4192a7d740 1 client.20328 shutdown 2024-09-19T07:03:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.106+0000 7f4192a7d740 1 -- 172.21.15.103:0/4261646367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32daf0 msgr2=0x55eddd335ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.106+0000 7f4192a7d740 1 --2- 172.21.15.103:0/4261646367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32daf0 0x55eddd335ca0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f41840eecc0 tx=0x7f41840b9b30 comp rx=0 tx=0).stop 2024-09-19T07:03:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.106+0000 7f4192a7d740 1 -- 172.21.15.103:0/4261646367 shutdown_connections 2024-09-19T07:03:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.106+0000 7f4192a7d740 1 --2- 172.21.15.103:0/4261646367 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eddd3488a0 0x55eddd34ac90 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.106+0000 7f4192a7d740 1 --2- 172.21.15.103:0/4261646367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd338cb0 0x55eddd336720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.106+0000 7f4192a7d740 1 --2- 172.21.15.103:0/4261646367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd32e410 0x55eddd3361e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.106+0000 7f41717fa640 0 client.20328 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.106+0000 7f4192a7d740 1 --2- 172.21.15.103:0/4261646367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32daf0 0x55eddd335ca0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.106+0000 7f4192a7d740 1 -- 172.21.15.103:0/4261646367 >> 172.21.15.103:0/4261646367 conn(0x55eddd32a060 msgr2=0x55eddd32fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.106+0000 7f4192a7d740 1 -- 172.21.15.103:0/4261646367 shutdown_connections 2024-09-19T07:03:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.106+0000 7f4192a7d740 1 -- 172.21.15.103:0/4261646367 wait complete. 2024-09-19T07:03:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.106+0000 7f4192a7d740 20 client.20328 trim_cache size 0 max 16384 2024-09-19T07:03:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f4192a7d740 1 Processor -- start 2024-09-19T07:03:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f4192a7d740 1 -- start start 2024-09-19T07:03:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f4192a7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd32daf0 0x55eddd336550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f4192a7d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd32e410 0x55eddd3505a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f4192a7d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd338cb0 0x55eddd350ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f4192a7d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f41881ad7c0 con 0x55eddd338cb0 2024-09-19T07:03:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f4192a7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f41881af300 con 0x55eddd32daf0 2024-09-19T07:03:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f4192a7d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41881ade90 con 0x55eddd32e410 2024-09-19T07:03:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f418e19a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd338cb0 0x55eddd350ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f418e19a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd338cb0 0x55eddd350ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58374/0 (socket says 172.21.15.103:58374) 2024-09-19T07:03:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f418e19a640 1 -- 172.21.15.103:0/4094541102 learned_addr learned my addr 172.21.15.103:0/4094541102 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f418d198640 1 --2- 172.21.15.103:0/4094541102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd32e410 0x55eddd3505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f418d999640 1 --2- 172.21.15.103:0/4094541102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd32daf0 0x55eddd336550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f418e19a640 1 -- 172.21.15.103:0/4094541102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd32e410 msgr2=0x55eddd3505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f418e19a640 1 --2- 172.21.15.103:0/4094541102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd32e410 0x55eddd3505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f418e19a640 1 -- 172.21.15.103:0/4094541102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd32daf0 msgr2=0x55eddd336550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f418e19a640 1 --2- 172.21.15.103:0/4094541102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd32daf0 0x55eddd336550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f418e19a640 1 -- 172.21.15.103:0/4094541102 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eddd32e240 con 0x55eddd338cb0 2024-09-19T07:03:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.121+0000 7f418d198640 1 --2- 172.21.15.103:0/4094541102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd32e410 0x55eddd3505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.122+0000 7f418e19a640 1 --2- 172.21.15.103:0/4094541102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd338cb0 0x55eddd350ae0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f4180010ad0 tx=0x7f418000e3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74bd9368c171a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.122+0000 7f41737fe640 1 -- 172.21.15.103:0/4094541102 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418001fbd0 con 0x55eddd338cb0 2024-09-19T07:03:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.122+0000 7f41737fe640 1 -- 172.21.15.103:0/4094541102 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f418001fd70 con 0x55eddd338cb0 2024-09-19T07:03:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.122+0000 7f41737fe640 1 -- 172.21.15.103:0/4094541102 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180028690 con 0x55eddd338cb0 2024-09-19T07:03:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.122+0000 7f4192a7d740 1 -- 172.21.15.103:0/4094541102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd338cb0 msgr2=0x55eddd350ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.122+0000 7f4192a7d740 1 --2- 172.21.15.103:0/4094541102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd338cb0 0x55eddd350ae0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f4180010ad0 tx=0x7f418000e3d0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f4192a7d740 1 -- 172.21.15.103:0/4094541102 shutdown_connections 2024-09-19T07:03:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f4192a7d740 1 --2- 172.21.15.103:0/4094541102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd338cb0 0x55eddd350ae0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f4192a7d740 1 --2- 172.21.15.103:0/4094541102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd32e410 0x55eddd3505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f4192a7d740 1 --2- 172.21.15.103:0/4094541102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd32daf0 0x55eddd336550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f4192a7d740 1 -- 172.21.15.103:0/4094541102 >> 172.21.15.103:0/4094541102 conn(0x55eddd32a060 msgr2=0x55eddd337d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f4192a7d740 1 -- 172.21.15.103:0/4094541102 shutdown_connections 2024-09-19T07:03:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f4192a7d740 1 -- 172.21.15.103:0/4094541102 wait complete. 2024-09-19T07:03:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f4192a7d740 1 -- start start 2024-09-19T07:03:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f4192a7d740 1 Processor -- start 2024-09-19T07:03:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f4192a7d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32daf0 0x55eddd341210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f4192a7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd32e410 0x55eddd33c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f4192a7d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd338cb0 0x55eddd33cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f4192a7d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f41881aec30 con 0x55eddd32daf0 2024-09-19T07:03:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f4192a7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f41881ae560 con 0x55eddd32e410 2024-09-19T07:03:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f4192a7d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41881ab7c0 con 0x55eddd338cb0 2024-09-19T07:03:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f4192a7d740 10 client.0 _ll_register_callbacks cb 0x7f4178004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f418d999640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32daf0 0x55eddd341210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.123+0000 7f418d999640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32daf0 0x55eddd341210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58376/0 (socket says 172.21.15.103:58376) 2024-09-19T07:03:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f418d999640 1 -- 172.21.15.103:0/3280992648 learned_addr learned my addr 172.21.15.103:0/3280992648 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f418e19a640 1 --2- 172.21.15.103:0/3280992648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd338cb0 0x55eddd33cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f418d198640 1 --2- 172.21.15.103:0/3280992648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd32e410 0x55eddd33c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f418d999640 1 -- 172.21.15.103:0/3280992648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd338cb0 msgr2=0x55eddd33cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f418d999640 1 --2- 172.21.15.103:0/3280992648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd338cb0 0x55eddd33cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f418d999640 1 -- 172.21.15.103:0/3280992648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd32e410 msgr2=0x55eddd33c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f418d999640 1 --2- 172.21.15.103:0/3280992648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd32e410 0x55eddd33c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f418d999640 1 -- 172.21.15.103:0/3280992648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eddd33d2e0 con 0x55eddd32daf0 2024-09-19T07:03:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f418d999640 1 --2- 172.21.15.103:0/3280992648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32daf0 0x55eddd341210 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f41840bd4a0 tx=0x7f41840f1360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133ed93c39551662 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f41717fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f41717fa640 1 -- 172.21.15.103:0/3280992648 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41840bd100 con 0x55eddd32daf0 2024-09-19T07:03:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f41717fa640 1 -- 172.21.15.103:0/3280992648 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f41840c13f0 con 0x55eddd32daf0 2024-09-19T07:03:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f4192a7d740 1 -- 172.21.15.103:0/3280992648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f418001fd70 con 0x55eddd32daf0 2024-09-19T07:03:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f41717fa640 1 -- 172.21.15.103:0/3280992648 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f41840bd570 con 0x55eddd32daf0 2024-09-19T07:03:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f4192a7d740 10 client.20334 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f4192a7d740 1 -- 172.21.15.103:0/3280992648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eddd33d2e0 con 0x55eddd32daf0 2024-09-19T07:03:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.124+0000 7f41717fa640 1 -- 172.21.15.103:0/3280992648 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f418001fd70 con 0x55eddd32daf0 2024-09-19T07:03:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.125+0000 7f41717fa640 1 -- 172.21.15.103:0/3280992648 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f41840f1040 con 0x55eddd32daf0 2024-09-19T07:03:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.125+0000 7f4192a7d740 10 client.20334 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.125+0000 7f4192a7d740 10 client.20334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.125+0000 7f4192a7d740 20 client.20334 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.125+0000 7f4192a7d740 1 --2- 172.21.15.103:0/3280992648 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eddd3b38a0 0x55eddd3b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.125+0000 7f4192a7d740 4 client.20334 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:03:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.125+0000 7f4192a7d740 1 -- 172.21.15.103:0/3280992648 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eddd299d70 con 0x55eddd3b38a0 2024-09-19T07:03:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.125+0000 7f418d198640 1 --2- 172.21.15.103:0/3280992648 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eddd3b38a0 0x55eddd3b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.125+0000 7f41717fa640 10 client.20334 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.125+0000 7f418d198640 1 --2- 172.21.15.103:0/3280992648 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eddd3b38a0 0x55eddd3b5c90 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b6ed23e23d641d server_cookie=df701ff6f6c81814 in_seq=0 out_seq=0 2024-09-19T07:03:26.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:25 smithi103 ceph-mon[31158]: pgmap v444: 97 pgs: 97 active+clean; 4.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-09-19T07:03:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.138+0000 7f41717fa640 1 -- 172.21.15.103:0/3280992648 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eddd299d70 con 0x55eddd3b38a0 2024-09-19T07:03:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.138+0000 7f41717fa640 10 client.20334 handle_command_reply: tid=0 2024-09-19T07:03:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.138+0000 7f4192a7d740 1 client.20334 shutdown 2024-09-19T07:03:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.138+0000 7f4192a7d740 1 -- 172.21.15.103:0/3280992648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32daf0 msgr2=0x55eddd341210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.138+0000 7f4192a7d740 1 --2- 172.21.15.103:0/3280992648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32daf0 0x55eddd341210 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f41840bd4a0 tx=0x7f41840f1360 comp rx=0 tx=0).stop 2024-09-19T07:03:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f4192a7d740 1 -- 172.21.15.103:0/3280992648 shutdown_connections 2024-09-19T07:03:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f4192a7d740 1 --2- 172.21.15.103:0/3280992648 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eddd3b38a0 0x55eddd3b5c90 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f4192a7d740 1 --2- 172.21.15.103:0/3280992648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eddd338cb0 0x55eddd33cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f4192a7d740 1 --2- 172.21.15.103:0/3280992648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eddd32e410 0x55eddd33c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f4192a7d740 1 --2- 172.21.15.103:0/3280992648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eddd32daf0 0x55eddd341210 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f41717fa640 0 client.20334 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f4192a7d740 1 -- 172.21.15.103:0/3280992648 >> 172.21.15.103:0/3280992648 conn(0x55eddd32a060 msgr2=0x55eddd3289a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f4192a7d740 1 -- 172.21.15.103:0/3280992648 shutdown_connections 2024-09-19T07:03:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f4192a7d740 1 -- 172.21.15.103:0/3280992648 wait complete. 2024-09-19T07:03:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f4192a7d740 20 client.20334 trim_cache size 0 max 16384 2024-09-19T07:03:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:03:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:03:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:03:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:03:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f418fbfd640 1 -- 172.21.15.103:0/2232836567 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f41740a3460 msgr2=0x7f41740a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f418fbfd640 1 --2- 172.21.15.103:0/2232836567 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f41740a3460 0x7f41740a5920 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4180005fd0 tx=0x7f4180005850 comp rx=0 tx=0).stop 2024-09-19T07:03:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f418fbfd640 1 -- 172.21.15.103:0/2232836567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41881009f0 msgr2=0x7f41881991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f418fbfd640 1 --2- 172.21.15.103:0/2232836567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41881009f0 0x7f41881991f0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f418402ae00 tx=0x7f4184003970 comp rx=0 tx=0).stop 2024-09-19T07:03:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f418fbfd640 1 -- 172.21.15.103:0/2232836567 shutdown_connections 2024-09-19T07:03:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f418fbfd640 1 --2- 172.21.15.103:0/2232836567 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f41740a3460 0x7f41740a5920 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f418fbfd640 1 --2- 172.21.15.103:0/2232836567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4188106a00 0x7f41881a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f418fbfd640 1 --2- 172.21.15.103:0/2232836567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4188101390 0x7f4188199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f418fbfd640 1 --2- 172.21.15.103:0/2232836567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41881009f0 0x7f41881991f0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f418fbfd640 1 -- 172.21.15.103:0/2232836567 >> 172.21.15.103:0/2232836567 conn(0x7f41880fc6b0 msgr2=0x7f41880fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f418fbfd640 1 -- 172.21.15.103:0/2232836567 shutdown_connections 2024-09-19T07:03:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:26.139+0000 7f418fbfd640 1 -- 172.21.15.103:0/2232836567 wait complete. 2024-09-19T07:03:26.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:25 smithi134 ceph-mon[35356]: pgmap v444: 97 pgs: 97 active+clean; 4.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-09-19T07:03:26.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:25 smithi174 ceph-mon[35637]: pgmap v444: 97 pgs: 97 active+clean; 4.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-09-19T07:03:28.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:27 smithi103 ceph-mon[31158]: pgmap v445: 97 pgs: 97 active+clean; 4.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-19T07:03:28.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:27 smithi134 ceph-mon[35356]: pgmap v445: 97 pgs: 97 active+clean; 4.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-19T07:03:28.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:27 smithi174 ceph-mon[35637]: pgmap v445: 97 pgs: 97 active+clean; 4.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-09-19T07:03:30.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:29 smithi103 ceph-mon[31158]: pgmap v446: 97 pgs: 97 active+clean; 4.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-09-19T07:03:30.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:29 smithi134 ceph-mon[35356]: pgmap v446: 97 pgs: 97 active+clean; 4.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-09-19T07:03:30.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:29 smithi174 ceph-mon[35637]: pgmap v446: 97 pgs: 97 active+clean; 4.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-09-19T07:03:32.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:31 smithi103 ceph-mon[31158]: pgmap v447: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-09-19T07:03:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:31 smithi134 ceph-mon[35356]: pgmap v447: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-09-19T07:03:32.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:31 smithi174 ceph-mon[35637]: pgmap v447: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-09-19T07:03:33.300 INFO:tasks.workunit.client.0.smithi103.stdout:437/600: open fd = 3 2024-09-19T07:03:33.301 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.301 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.301 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.301 INFO:tasks.workunit.client.0.smithi103.stdout:438/600: open fd = 3 2024-09-19T07:03:33.301 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.301 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.301 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.301 INFO:tasks.workunit.client.0.smithi103.stdout:439/600: open fd = 3 2024-09-19T07:03:33.301 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.301 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.302 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.302 INFO:tasks.workunit.client.0.smithi103.stdout:440/600: open fd = 3 2024-09-19T07:03:33.302 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.302 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.302 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.302 INFO:tasks.workunit.client.0.smithi103.stdout:441/600: open fd = 3 2024-09-19T07:03:33.302 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.302 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.302 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.302 INFO:tasks.workunit.client.0.smithi103.stdout:442/600: open fd = 3 2024-09-19T07:03:33.302 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.302 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.303 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.303 INFO:tasks.workunit.client.0.smithi103.stdout:443/600: open fd = 3 2024-09-19T07:03:33.303 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.303 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.303 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.303 INFO:tasks.workunit.client.0.smithi103.stdout:444/600: open fd = 3 2024-09-19T07:03:33.303 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.303 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.303 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.303 INFO:tasks.workunit.client.0.smithi103.stdout:445/600: open fd = 3 2024-09-19T07:03:33.303 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.303 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.303 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.304 INFO:tasks.workunit.client.0.smithi103.stdout:446/600: open fd = 3 2024-09-19T07:03:33.304 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.304 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.304 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.304 INFO:tasks.workunit.client.0.smithi103.stdout:447/600: open fd = 3 2024-09-19T07:03:33.304 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.304 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.304 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.304 INFO:tasks.workunit.client.0.smithi103.stdout:448/600: open fd = 3 2024-09-19T07:03:33.304 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.304 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.304 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.304 INFO:tasks.workunit.client.0.smithi103.stdout:449/600: open fd = 3 2024-09-19T07:03:33.304 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.305 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.305 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.305 INFO:tasks.workunit.client.0.smithi103.stdout:450/600: open fd = 3 2024-09-19T07:03:33.305 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.305 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.305 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.305 INFO:tasks.workunit.client.0.smithi103.stdout:451/600: open fd = 3 2024-09-19T07:03:33.305 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.305 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.305 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.305 INFO:tasks.workunit.client.0.smithi103.stdout:452/600: open fd = 3 2024-09-19T07:03:33.305 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.305 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.305 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.306 INFO:tasks.workunit.client.0.smithi103.stdout:453/600: open fd = 3 2024-09-19T07:03:33.306 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.306 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.306 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.306 INFO:tasks.workunit.client.0.smithi103.stdout:454/600: open fd = 3 2024-09-19T07:03:33.306 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.306 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.306 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.306 INFO:tasks.workunit.client.0.smithi103.stdout:455/600: open fd = 3 2024-09-19T07:03:33.306 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.306 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.306 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.306 INFO:tasks.workunit.client.0.smithi103.stdout:456/600: open fd = 3 2024-09-19T07:03:33.306 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.306 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.307 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.307 INFO:tasks.workunit.client.0.smithi103.stdout:457/600: open fd = 3 2024-09-19T07:03:33.307 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.307 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.307 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.307 INFO:tasks.workunit.client.0.smithi103.stdout:458/600: open fd = 3 2024-09-19T07:03:33.307 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.307 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.307 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.307 INFO:tasks.workunit.client.0.smithi103.stdout:459/600: open fd = 3 2024-09-19T07:03:33.307 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.307 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.307 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.307 INFO:tasks.workunit.client.0.smithi103.stdout:460/600: open fd = 3 2024-09-19T07:03:33.307 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.308 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.308 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.308 INFO:tasks.workunit.client.0.smithi103.stdout:461/600: open fd = 3 2024-09-19T07:03:33.308 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.308 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.308 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.308 INFO:tasks.workunit.client.0.smithi103.stdout:462/600: open fd = 3 2024-09-19T07:03:33.308 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.308 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.308 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.308 INFO:tasks.workunit.client.0.smithi103.stdout:463/600: open fd = 3 2024-09-19T07:03:33.308 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.308 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.309 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.309 INFO:tasks.workunit.client.0.smithi103.stdout:464/600: open fd = 3 2024-09-19T07:03:33.309 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.309 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.309 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.309 INFO:tasks.workunit.client.0.smithi103.stdout:465/600: open fd = 3 2024-09-19T07:03:33.309 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.309 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.309 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.309 INFO:tasks.workunit.client.0.smithi103.stdout:466/600: open fd = 3 2024-09-19T07:03:33.309 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.309 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.309 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.309 INFO:tasks.workunit.client.0.smithi103.stdout:467/600: open fd = 3 2024-09-19T07:03:33.310 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.310 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.310 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.310 INFO:tasks.workunit.client.0.smithi103.stdout:468/600: open fd = 3 2024-09-19T07:03:33.310 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.310 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.310 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.310 INFO:tasks.workunit.client.0.smithi103.stdout:469/600: open fd = 3 2024-09-19T07:03:33.310 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.310 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.310 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.310 INFO:tasks.workunit.client.0.smithi103.stdout:470/600: open fd = 3 2024-09-19T07:03:33.310 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.310 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.311 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.311 INFO:tasks.workunit.client.0.smithi103.stdout:471/600: open fd = 3 2024-09-19T07:03:33.311 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.311 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.311 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.311 INFO:tasks.workunit.client.0.smithi103.stdout:472/600: open fd = 3 2024-09-19T07:03:33.312 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.312 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.312 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.312 INFO:tasks.workunit.client.0.smithi103.stdout:473/600: open fd = 3 2024-09-19T07:03:33.312 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.312 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.312 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.312 INFO:tasks.workunit.client.0.smithi103.stdout:474/600: open fd = 3 2024-09-19T07:03:33.312 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.313 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.313 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.313 INFO:tasks.workunit.client.0.smithi103.stdout:475/600: open fd = 3 2024-09-19T07:03:33.313 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.313 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.313 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.313 INFO:tasks.workunit.client.0.smithi103.stdout:476/600: open fd = 3 2024-09-19T07:03:33.313 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.314 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.314 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.314 INFO:tasks.workunit.client.0.smithi103.stdout:477/600: open fd = 3 2024-09-19T07:03:33.314 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.314 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.314 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.314 INFO:tasks.workunit.client.0.smithi103.stdout:478/600: open fd = 3 2024-09-19T07:03:33.314 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.315 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.315 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.315 INFO:tasks.workunit.client.0.smithi103.stdout:479/600: open fd = 3 2024-09-19T07:03:33.315 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.315 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.315 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.315 INFO:tasks.workunit.client.0.smithi103.stdout:480/600: open fd = 3 2024-09-19T07:03:33.315 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.315 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.315 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.315 INFO:tasks.workunit.client.0.smithi103.stdout:481/600: open fd = 3 2024-09-19T07:03:33.316 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.316 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.316 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.316 INFO:tasks.workunit.client.0.smithi103.stdout:482/600: open fd = 3 2024-09-19T07:03:33.316 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.316 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.316 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.316 INFO:tasks.workunit.client.0.smithi103.stdout:483/600: open fd = 3 2024-09-19T07:03:33.316 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.316 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.316 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.316 INFO:tasks.workunit.client.0.smithi103.stdout:484/600: open fd = 3 2024-09-19T07:03:33.316 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.316 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.317 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.317 INFO:tasks.workunit.client.0.smithi103.stdout:485/600: open fd = 3 2024-09-19T07:03:33.317 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.317 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.317 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.317 INFO:tasks.workunit.client.0.smithi103.stdout:486/600: open fd = 3 2024-09-19T07:03:33.317 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.317 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.317 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.317 INFO:tasks.workunit.client.0.smithi103.stdout:487/600: open fd = 3 2024-09-19T07:03:33.317 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.317 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.317 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.317 INFO:tasks.workunit.client.0.smithi103.stdout:488/600: open fd = 3 2024-09-19T07:03:33.318 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.318 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.318 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.318 INFO:tasks.workunit.client.0.smithi103.stdout:489/600: open fd = 3 2024-09-19T07:03:33.318 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.318 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.318 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.318 INFO:tasks.workunit.client.0.smithi103.stdout:490/600: open fd = 3 2024-09-19T07:03:33.318 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.318 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.318 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.318 INFO:tasks.workunit.client.0.smithi103.stdout:491/600: open fd = 3 2024-09-19T07:03:33.318 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.318 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.319 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.319 INFO:tasks.workunit.client.0.smithi103.stdout:492/600: open fd = 3 2024-09-19T07:03:33.319 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.319 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.319 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.319 INFO:tasks.workunit.client.0.smithi103.stdout:493/600: open fd = 3 2024-09-19T07:03:33.319 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.319 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.319 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.319 INFO:tasks.workunit.client.0.smithi103.stdout:494/600: open fd = 3 2024-09-19T07:03:33.319 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.319 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.319 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.319 INFO:tasks.workunit.client.0.smithi103.stdout:495/600: open fd = 3 2024-09-19T07:03:33.319 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.320 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.320 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.320 INFO:tasks.workunit.client.0.smithi103.stdout:496/600: open fd = 3 2024-09-19T07:03:33.320 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.320 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.320 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.320 INFO:tasks.workunit.client.0.smithi103.stdout:497/600: open fd = 3 2024-09-19T07:03:33.320 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.320 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.320 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:33.320 INFO:tasks.workunit.client.0.smithi103.stdout:498/600: open fd = 3 2024-09-19T07:03:33.320 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.321 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:03:33.321 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:03:34.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:33 smithi103 ceph-mon[31158]: pgmap v448: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-19T07:03:34.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:33 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:03:34.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:33 smithi134 ceph-mon[35356]: pgmap v448: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-19T07:03:34.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:33 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:03:34.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:33 smithi174 ceph-mon[35637]: pgmap v448: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-19T07:03:34.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:33 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:03:35.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:34 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:03:35.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:34 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:03:35.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:34 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:03:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:34 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:03:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:34 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:03:35.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:34 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:03:35.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:34 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:03:35.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:34 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:03:35.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:34 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:03:36.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:36 smithi174 ceph-mon[35637]: pgmap v449: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-19T07:03:36.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:36 smithi103 ceph-mon[31158]: pgmap v449: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-19T07:03:36.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:36 smithi134 ceph-mon[35356]: pgmap v449: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-09-19T07:03:38.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:38 smithi174 ceph-mon[35637]: pgmap v450: 97 pgs: 97 active+clean; 4.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-09-19T07:03:38.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:38 smithi103 ceph-mon[31158]: pgmap v450: 97 pgs: 97 active+clean; 4.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-09-19T07:03:38.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:38 smithi134 ceph-mon[35356]: pgmap v450: 97 pgs: 97 active+clean; 4.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-09-19T07:03:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.235+0000 7f703388d640 1 -- 172.21.15.103:0/3668035881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f702c10c0c0 msgr2=0x7f702c10e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.235+0000 7f703388d640 1 --2- 172.21.15.103:0/3668035881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f702c10c0c0 0x7f702c10e580 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f7024009ca0 tx=0x7f702402a950 comp rx=0 tx=0).stop 2024-09-19T07:03:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.235+0000 7f703388d640 1 -- 172.21.15.103:0/3668035881 shutdown_connections 2024-09-19T07:03:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.235+0000 7f703388d640 1 --2- 172.21.15.103:0/3668035881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f702c10c0c0 0x7f702c10e580 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.235+0000 7f703388d640 1 --2- 172.21.15.103:0/3668035881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f702c1035c0 0x7f702c10baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.235+0000 7f703388d640 1 --2- 172.21.15.103:0/3668035881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f702c102ca0 0x7f702c103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.235+0000 7f703388d640 1 -- 172.21.15.103:0/3668035881 >> 172.21.15.103:0/3668035881 conn(0x7f702c0fc6f0 msgr2=0x7f702c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.235+0000 7f703388d640 1 -- 172.21.15.103:0/3668035881 shutdown_connections 2024-09-19T07:03:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.235+0000 7f703388d640 1 -- 172.21.15.103:0/3668035881 wait complete. 2024-09-19T07:03:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.235+0000 7f703388d640 1 Processor -- start 2024-09-19T07:03:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.235+0000 7f703388d640 1 -- start start 2024-09-19T07:03:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f703388d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f702c102ca0 0x7f702c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f703388d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f702c1035c0 0x7f702c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f703388d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f702c10c0c0 0x7f702c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f703388d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f702c111ee0 con 0x7f702c1035c0 2024-09-19T07:03:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f703388d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f702c111d60 con 0x7f702c10c0c0 2024-09-19T07:03:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f703388d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f702c112060 con 0x7f702c102ca0 2024-09-19T07:03:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f7030e28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f702c1035c0 0x7f702c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f7030e28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f702c1035c0 0x7f702c1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34840/0 (socket says 172.21.15.103:34840) 2024-09-19T07:03:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f7030e28640 1 -- 172.21.15.103:0/2951263192 learned_addr learned my addr 172.21.15.103:0/2951263192 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f7031629640 1 --2- 172.21.15.103:0/2951263192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f702c102ca0 0x7f702c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f7031e2a640 1 --2- 172.21.15.103:0/2951263192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f702c10c0c0 0x7f702c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f7030e28640 1 -- 172.21.15.103:0/2951263192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f702c102ca0 msgr2=0x7f702c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f7030e28640 1 --2- 172.21.15.103:0/2951263192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f702c102ca0 0x7f702c1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f7030e28640 1 -- 172.21.15.103:0/2951263192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f702c10c0c0 msgr2=0x7f702c19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f7030e28640 1 --2- 172.21.15.103:0/2951263192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f702c10c0c0 0x7f702c19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f7030e28640 1 -- 172.21.15.103:0/2951263192 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702c19d2a0 con 0x7f702c1035c0 2024-09-19T07:03:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f7031e2a640 1 --2- 172.21.15.103:0/2951263192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f702c10c0c0 0x7f702c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f7030e28640 1 --2- 172.21.15.103:0/2951263192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f702c1035c0 0x7f702c1987f0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f701c00c920 tx=0x7f701c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f70227fc640 1 -- 172.21.15.103:0/2951263192 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c007c70 con 0x7f702c1035c0 2024-09-19T07:03:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f70227fc640 1 -- 172.21.15.103:0/2951263192 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f701c007e10 con 0x7f702c1035c0 2024-09-19T07:03:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f703388d640 1 -- 172.21.15.103:0/2951263192 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702c19d590 con 0x7f702c1035c0 2024-09-19T07:03:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.236+0000 7f70227fc640 1 -- 172.21.15.103:0/2951263192 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c018890 con 0x7f702c1035c0 2024-09-19T07:03:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.237+0000 7f703388d640 1 -- 172.21.15.103:0/2951263192 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f702c1a4e20 con 0x7f702c1035c0 2024-09-19T07:03:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.239+0000 7f70227fc640 1 -- 172.21.15.103:0/2951263192 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f701c018a70 con 0x7f702c1035c0 2024-09-19T07:03:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.239+0000 7f7036629740 1 Processor -- start 2024-09-19T07:03:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7036629740 1 -- start start 2024-09-19T07:03:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7036629740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615e8af0 0x55bc615e8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7036629740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e9410 0x55bc615f3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7036629740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 0x55bc615f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7036629740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bc6154d2f0 con 0x55bc615e9410 2024-09-19T07:03:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7036629740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bc614f5270 con 0x55bc615e8af0 2024-09-19T07:03:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7036629740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc615492b0 con 0x55bc615f3cb0 2024-09-19T07:03:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f70227fc640 1 --2- 172.21.15.103:0/2951263192 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70180a3460 0x7f70180a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7031629640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e9410 0x55bc615f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7031629640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e9410 0x55bc615f3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34852/0 (socket says 172.21.15.103:34852) 2024-09-19T07:03:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7031629640 1 -- 172.21.15.103:0/322941466 learned_addr learned my addr 172.21.15.103:0/322941466 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7031e2a640 1 --2- 172.21.15.103:0/2951263192 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70180a3460 0x7f70180a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f70227fc640 1 -- 172.21.15.103:0/2951263192 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f701c0cfe70 con 0x7f702c1035c0 2024-09-19T07:03:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7030e28640 1 --2- 172.21.15.103:0/322941466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 0x55bc615f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7031e2a640 1 --2- 172.21.15.103:0/322941466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615e8af0 0x55bc615e8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7031629640 1 -- 172.21.15.103:0/322941466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 msgr2=0x55bc615f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7031629640 1 --2- 172.21.15.103:0/322941466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 0x55bc615f60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7031629640 1 -- 172.21.15.103:0/322941466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615e8af0 msgr2=0x55bc615e8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7031629640 1 --2- 172.21.15.103:0/322941466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615e8af0 0x55bc615e8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7031629640 1 -- 172.21.15.103:0/322941466 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc614f4340 con 0x55bc615e9410 2024-09-19T07:03:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7030e28640 1 --2- 172.21.15.103:0/322941466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 0x55bc615f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.240+0000 7f7031e2a640 1 --2- 172.21.15.103:0/322941466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615e8af0 0x55bc615e8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.241+0000 7f7031629640 1 --2- 172.21.15.103:0/322941466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e9410 0x55bc615f3770 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f702800b760 tx=0x7f702800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154e17ed1edb07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.241+0000 7f7031e2a640 1 --2- 172.21.15.103:0/2951263192 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70180a3460 0x7f70180a5920 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7024009a60 tx=0x7f702403d040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.241+0000 7f70177fe640 1 -- 172.21.15.103:0/322941466 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028003dd0 con 0x55bc615e9410 2024-09-19T07:03:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.241+0000 7f70177fe640 1 -- 172.21.15.103:0/322941466 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7028015070 con 0x55bc615e9410 2024-09-19T07:03:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.241+0000 7f70177fe640 1 -- 172.21.15.103:0/322941466 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028004dc0 con 0x55bc615e9410 2024-09-19T07:03:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.241+0000 7f7036629740 1 -- 172.21.15.103:0/322941466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e9410 msgr2=0x55bc615f3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.241+0000 7f7036629740 1 --2- 172.21.15.103:0/322941466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e9410 0x55bc615f3770 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f702800b760 tx=0x7f702800bc20 comp rx=0 tx=0).stop 2024-09-19T07:03:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.241+0000 7f7036629740 1 -- 172.21.15.103:0/322941466 shutdown_connections 2024-09-19T07:03:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.241+0000 7f7036629740 1 --2- 172.21.15.103:0/322941466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 0x55bc615f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.241+0000 7f7036629740 1 --2- 172.21.15.103:0/322941466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e9410 0x55bc615f3770 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.241+0000 7f7036629740 1 --2- 172.21.15.103:0/322941466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615e8af0 0x55bc615e8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.241+0000 7f7036629740 1 -- 172.21.15.103:0/322941466 >> 172.21.15.103:0/322941466 conn(0x55bc615e5060 msgr2=0x55bc615e5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.242+0000 7f7036629740 1 -- 172.21.15.103:0/322941466 shutdown_connections 2024-09-19T07:03:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.242+0000 7f7036629740 1 -- 172.21.15.103:0/322941466 wait complete. 2024-09-19T07:03:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.242+0000 7f7036629740 1 -- start start 2024-09-19T07:03:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.242+0000 7f7036629740 1 Processor -- start 2024-09-19T07:03:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.242+0000 7f7036629740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e8af0 0x55bc615f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.242+0000 7f7036629740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615e9410 0x55bc615f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.242+0000 7f7036629740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 0x55bc615f1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.242+0000 7f7036629740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bc615f6880 con 0x55bc615e8af0 2024-09-19T07:03:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.242+0000 7f7036629740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bc614d6290 con 0x55bc615e9410 2024-09-19T07:03:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.242+0000 7f7036629740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc615f6a00 con 0x55bc615f3cb0 2024-09-19T07:03:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7036629740 10 client.0 _ll_register_callbacks cb 0x55bc61507140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7031629640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e8af0 0x55bc615f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7031629640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e8af0 0x55bc615f0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34858/0 (socket says 172.21.15.103:34858) 2024-09-19T07:03:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7031629640 1 -- 172.21.15.103:0/3627185262 learned_addr learned my addr 172.21.15.103:0/3627185262 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7030e28640 1 --2- 172.21.15.103:0/3627185262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615e9410 0x55bc615f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7031e2a640 1 --2- 172.21.15.103:0/3627185262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 0x55bc615f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7031629640 1 -- 172.21.15.103:0/3627185262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 msgr2=0x55bc615f1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7031629640 1 --2- 172.21.15.103:0/3627185262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 0x55bc615f1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7031629640 1 -- 172.21.15.103:0/3627185262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615e9410 msgr2=0x55bc615f0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7031629640 1 --2- 172.21.15.103:0/3627185262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615e9410 0x55bc615f0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7031629640 1 -- 172.21.15.103:0/3627185262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc61260210 con 0x55bc615e8af0 2024-09-19T07:03:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7031629640 1 --2- 172.21.15.103:0/3627185262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e8af0 0x55bc615f0ab0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f702800bfa0 tx=0x7f702800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ba1cf64c9ff5cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7014ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7014ff9640 1 -- 172.21.15.103:0/3627185262 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028028060 con 0x55bc615e8af0 2024-09-19T07:03:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7036629740 1 -- 172.21.15.103:0/3627185262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7028015070 con 0x55bc615e8af0 2024-09-19T07:03:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7014ff9640 1 -- 172.21.15.103:0/3627185262 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f7028011640 con 0x55bc615e8af0 2024-09-19T07:03:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7014ff9640 1 -- 172.21.15.103:0/3627185262 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7028011840 con 0x55bc615e8af0 2024-09-19T07:03:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7036629740 10 client.20346 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.243+0000 7f7036629740 1 -- 172.21.15.103:0/3627185262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bc6132c620 con 0x55bc615e8af0 2024-09-19T07:03:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.244+0000 7f7014ff9640 1 -- 172.21.15.103:0/3627185262 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7028015070 con 0x55bc615e8af0 2024-09-19T07:03:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.244+0000 7f7014ff9640 1 -- 172.21.15.103:0/3627185262 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7028022070 con 0x55bc615e8af0 2024-09-19T07:03:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.244+0000 7f7036629740 10 client.20346 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.244+0000 7f7036629740 10 client.20346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.244+0000 7f7036629740 20 client.20346 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.244+0000 7f7036629740 1 --2- 172.21.15.103:0/3627185262 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55bc61603a80 0x55bc61605e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.244+0000 7f7036629740 4 client.20346 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.244+0000 7f7036629740 1 -- 172.21.15.103:0/3627185262 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc6155aa20 con 0x55bc61603a80 2024-09-19T07:03:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.244+0000 7f7030e28640 1 --2- 172.21.15.103:0/3627185262 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55bc61603a80 0x55bc61605e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.245+0000 7f7014ff9640 10 client.20346 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.245+0000 7f7030e28640 1 --2- 172.21.15.103:0/3627185262 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55bc61603a80 0x55bc61605e70 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3053e53b668e6a92 server_cookie=8181249d05133cc4 in_seq=0 out_seq=0 2024-09-19T07:03:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.246+0000 7f7014ff9640 1 -- 172.21.15.103:0/3627185262 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc6155aa20 con 0x55bc61603a80 2024-09-19T07:03:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.246+0000 7f7014ff9640 10 client.20346 handle_command_reply: tid=0 2024-09-19T07:03:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.246+0000 7f7036629740 1 client.20346 shutdown 2024-09-19T07:03:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.246+0000 7f7036629740 1 -- 172.21.15.103:0/3627185262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e8af0 msgr2=0x55bc615f0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.246+0000 7f7036629740 1 --2- 172.21.15.103:0/3627185262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e8af0 0x55bc615f0ab0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f702800bfa0 tx=0x7f702800bfd0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.246+0000 7f7036629740 1 -- 172.21.15.103:0/3627185262 shutdown_connections 2024-09-19T07:03:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.246+0000 7f7036629740 1 --2- 172.21.15.103:0/3627185262 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55bc61603a80 0x55bc61605e70 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.246+0000 7f7036629740 1 --2- 172.21.15.103:0/3627185262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 0x55bc615f1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.246+0000 7f7036629740 1 --2- 172.21.15.103:0/3627185262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615e9410 0x55bc615f0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.246+0000 7f7036629740 1 --2- 172.21.15.103:0/3627185262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e8af0 0x55bc615f0ab0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.246+0000 7f7014ff9640 0 client.20346 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.246+0000 7f7036629740 1 -- 172.21.15.103:0/3627185262 >> 172.21.15.103:0/3627185262 conn(0x55bc615e5060 msgr2=0x55bc615eadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.246+0000 7f7036629740 1 -- 172.21.15.103:0/3627185262 shutdown_connections 2024-09-19T07:03:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.246+0000 7f7036629740 1 -- 172.21.15.103:0/3627185262 wait complete. 2024-09-19T07:03:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.246+0000 7f7036629740 20 client.20346 trim_cache size 0 max 16384 2024-09-19T07:03:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.260+0000 7f7036629740 1 Processor -- start 2024-09-19T07:03:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.260+0000 7f7036629740 1 -- start start 2024-09-19T07:03:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.260+0000 7f7036629740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615e8af0 0x55bc615eb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.260+0000 7f7036629740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e9410 0x55bc615ef360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.260+0000 7f7036629740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615f3cb0 0x55bc615f1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.260+0000 7f7036629740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1ad740 con 0x55bc615e9410 2024-09-19T07:03:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7036629740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1af280 con 0x55bc615f3cb0 2024-09-19T07:03:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7036629740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1ade10 con 0x55bc615e8af0 2024-09-19T07:03:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7030e28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e9410 0x55bc615ef360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7030e28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e9410 0x55bc615ef360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34862/0 (socket says 172.21.15.103:34862) 2024-09-19T07:03:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7030e28640 1 -- 172.21.15.103:0/3290298688 learned_addr learned my addr 172.21.15.103:0/3290298688 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7031629640 1 --2- 172.21.15.103:0/3290298688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615e8af0 0x55bc615eb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7031e2a640 1 --2- 172.21.15.103:0/3290298688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615f3cb0 0x55bc615f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7030e28640 1 -- 172.21.15.103:0/3290298688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615e8af0 msgr2=0x55bc615eb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7030e28640 1 --2- 172.21.15.103:0/3290298688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615e8af0 0x55bc615eb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7030e28640 1 -- 172.21.15.103:0/3290298688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615f3cb0 msgr2=0x55bc615f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7030e28640 1 --2- 172.21.15.103:0/3290298688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615f3cb0 0x55bc615f1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7030e28640 1 -- 172.21.15.103:0/3290298688 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc615e8870 con 0x55bc615e9410 2024-09-19T07:03:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7031e2a640 1 --2- 172.21.15.103:0/3290298688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615f3cb0 0x55bc615f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7030e28640 1 --2- 172.21.15.103:0/3290298688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e9410 0x55bc615ef360 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f701c0cd5a0 tx=0x7f701c0983b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac08cef03c8d03e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7016ffd640 1 -- 172.21.15.103:0/3290298688 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c018d60 con 0x55bc615e9410 2024-09-19T07:03:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7016ffd640 1 -- 172.21.15.103:0/3290298688 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f701c0bc0b0 con 0x55bc615e9410 2024-09-19T07:03:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.261+0000 7f7016ffd640 1 -- 172.21.15.103:0/3290298688 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c0cd7c0 con 0x55bc615e9410 2024-09-19T07:03:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.262+0000 7f7036629740 1 -- 172.21.15.103:0/3290298688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e9410 msgr2=0x55bc615ef360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.262+0000 7f7036629740 1 --2- 172.21.15.103:0/3290298688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e9410 0x55bc615ef360 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f701c0cd5a0 tx=0x7f701c0983b0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.262+0000 7f7036629740 1 -- 172.21.15.103:0/3290298688 shutdown_connections 2024-09-19T07:03:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.262+0000 7f7036629740 1 --2- 172.21.15.103:0/3290298688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615f3cb0 0x55bc615f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.262+0000 7f7036629740 1 --2- 172.21.15.103:0/3290298688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e9410 0x55bc615ef360 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.262+0000 7f7036629740 1 --2- 172.21.15.103:0/3290298688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615e8af0 0x55bc615eb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.262+0000 7f7036629740 1 -- 172.21.15.103:0/3290298688 >> 172.21.15.103:0/3290298688 conn(0x55bc615e5060 msgr2=0x55bc615f2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.262+0000 7f7036629740 1 -- 172.21.15.103:0/3290298688 shutdown_connections 2024-09-19T07:03:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.262+0000 7f7036629740 1 -- 172.21.15.103:0/3290298688 wait complete. 2024-09-19T07:03:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.262+0000 7f7036629740 1 -- start start 2024-09-19T07:03:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.262+0000 7f7036629740 1 Processor -- start 2024-09-19T07:03:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7036629740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e8af0 0x55bc616093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7036629740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615e9410 0x55bc616067b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7036629740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 0x55bc61606cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7036629740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1aebb0 con 0x55bc615e8af0 2024-09-19T07:03:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7036629740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1ae4e0 con 0x55bc615e9410 2024-09-19T07:03:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7036629740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1ab740 con 0x55bc615f3cb0 2024-09-19T07:03:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7036629740 10 client.0 _ll_register_callbacks cb 0x7f7028004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7031629640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e8af0 0x55bc616093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7031629640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e8af0 0x55bc616093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34868/0 (socket says 172.21.15.103:34868) 2024-09-19T07:03:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7031629640 1 -- 172.21.15.103:0/431881355 learned_addr learned my addr 172.21.15.103:0/431881355 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7030e28640 1 --2- 172.21.15.103:0/431881355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615e9410 0x55bc616067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7031e2a640 1 --2- 172.21.15.103:0/431881355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 0x55bc61606cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7031629640 1 -- 172.21.15.103:0/431881355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 msgr2=0x55bc61606cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7031629640 1 --2- 172.21.15.103:0/431881355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 0x55bc61606cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7031629640 1 -- 172.21.15.103:0/431881355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615e9410 msgr2=0x55bc616067b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7031629640 1 --2- 172.21.15.103:0/431881355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615e9410 0x55bc616067b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7031629640 1 -- 172.21.15.103:0/431881355 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc616072e0 con 0x55bc615e8af0 2024-09-19T07:03:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7031e2a640 1 --2- 172.21.15.103:0/431881355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 0x55bc61606cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7031629640 1 --2- 172.21.15.103:0/431881355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e8af0 0x55bc616093d0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f7028001eb0 tx=0x7f7028003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b9ee98fadb6ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7014ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7014ff9640 1 -- 172.21.15.103:0/431881355 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028011df0 con 0x55bc615e8af0 2024-09-19T07:03:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7014ff9640 1 -- 172.21.15.103:0/431881355 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f7028019420 con 0x55bc615e8af0 2024-09-19T07:03:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.263+0000 7f7036629740 1 -- 172.21.15.103:0/431881355 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f701c0bc0b0 con 0x55bc615e8af0 2024-09-19T07:03:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.264+0000 7f7014ff9640 1 -- 172.21.15.103:0/431881355 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7028042050 con 0x55bc615e8af0 2024-09-19T07:03:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.264+0000 7f7036629740 10 client.20352 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.264+0000 7f7036629740 1 -- 172.21.15.103:0/431881355 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bc6166e120 con 0x55bc615e8af0 2024-09-19T07:03:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.264+0000 7f7014ff9640 1 -- 172.21.15.103:0/431881355 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f701c0bc0b0 con 0x55bc615e8af0 2024-09-19T07:03:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.264+0000 7f7014ff9640 1 -- 172.21.15.103:0/431881355 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7028022040 con 0x55bc615e8af0 2024-09-19T07:03:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.264+0000 7f7036629740 10 client.20352 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.264+0000 7f7036629740 10 client.20352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.264+0000 7f7036629740 20 client.20352 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.264+0000 7f7036629740 1 --2- 172.21.15.103:0/431881355 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55bc6166e860 0x55bc61670c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.264+0000 7f7036629740 4 client.20352 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:03:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.264+0000 7f7036629740 1 -- 172.21.15.103:0/431881355 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc61554d70 con 0x55bc6166e860 2024-09-19T07:03:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.264+0000 7f7030e28640 1 --2- 172.21.15.103:0/431881355 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55bc6166e860 0x55bc61670c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.265+0000 7f7030e28640 1 --2- 172.21.15.103:0/431881355 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55bc6166e860 0x55bc61670c80 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d94cc9709b4a7d0d server_cookie=8baa130466e7279b in_seq=0 out_seq=0 2024-09-19T07:03:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.265+0000 7f7014ff9640 10 client.20352 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.279+0000 7f7014ff9640 1 -- 172.21.15.103:0/431881355 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bc61554d70 con 0x55bc6166e860 2024-09-19T07:03:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.279+0000 7f7014ff9640 10 client.20352 handle_command_reply: tid=0 2024-09-19T07:03:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.279+0000 7f7036629740 1 client.20352 shutdown 2024-09-19T07:03:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.279+0000 7f7036629740 1 -- 172.21.15.103:0/431881355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e8af0 msgr2=0x55bc616093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.279+0000 7f7036629740 1 --2- 172.21.15.103:0/431881355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e8af0 0x55bc616093d0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f7028001eb0 tx=0x7f7028003c50 comp rx=0 tx=0).stop 2024-09-19T07:03:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f7036629740 1 -- 172.21.15.103:0/431881355 shutdown_connections 2024-09-19T07:03:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f7036629740 1 --2- 172.21.15.103:0/431881355 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55bc6166e860 0x55bc61670c80 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f7036629740 1 --2- 172.21.15.103:0/431881355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc615f3cb0 0x55bc61606cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f7014ff9640 0 client.20352 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f7036629740 1 --2- 172.21.15.103:0/431881355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc615e9410 0x55bc616067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f7036629740 1 --2- 172.21.15.103:0/431881355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615e8af0 0x55bc616093d0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f7036629740 1 -- 172.21.15.103:0/431881355 >> 172.21.15.103:0/431881355 conn(0x55bc615e5060 msgr2=0x55bc615e39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:39.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f7036629740 1 -- 172.21.15.103:0/431881355 shutdown_connections 2024-09-19T07:03:39.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f7036629740 1 -- 172.21.15.103:0/431881355 wait complete. 2024-09-19T07:03:39.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f7036629740 20 client.20352 trim_cache size 0 max 16384 2024-09-19T07:03:39.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:03:39.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:03:39.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:03:39.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:03:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f703388d640 1 -- 172.21.15.103:0/2951263192 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70180a3460 msgr2=0x7f70180a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f703388d640 1 --2- 172.21.15.103:0/2951263192 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70180a3460 0x7f70180a5920 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7024009a60 tx=0x7f702403d040 comp rx=0 tx=0).stop 2024-09-19T07:03:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f703388d640 1 -- 172.21.15.103:0/2951263192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f702c1035c0 msgr2=0x7f702c1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f703388d640 1 --2- 172.21.15.103:0/2951263192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f702c1035c0 0x7f702c1987f0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f701c00c920 tx=0x7f701c00cde0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f703388d640 1 -- 172.21.15.103:0/2951263192 shutdown_connections 2024-09-19T07:03:39.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f703388d640 1 --2- 172.21.15.103:0/2951263192 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f70180a3460 0x7f70180a5920 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f703388d640 1 --2- 172.21.15.103:0/2951263192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f702c10c0c0 0x7f702c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f703388d640 1 --2- 172.21.15.103:0/2951263192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f702c1035c0 0x7f702c1987f0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f703388d640 1 --2- 172.21.15.103:0/2951263192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f702c102ca0 0x7f702c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:39.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f703388d640 1 -- 172.21.15.103:0/2951263192 >> 172.21.15.103:0/2951263192 conn(0x7f702c0fc6f0 msgr2=0x7f702c0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:39.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f703388d640 1 -- 172.21.15.103:0/2951263192 shutdown_connections 2024-09-19T07:03:39.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:39.280+0000 7f703388d640 1 -- 172.21.15.103:0/2951263192 wait complete. 2024-09-19T07:03:40.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:40 smithi174 ceph-mon[35637]: pgmap v451: 97 pgs: 97 active+clean; 4.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T07:03:40.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:40 smithi103 ceph-mon[31158]: pgmap v451: 97 pgs: 97 active+clean; 4.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T07:03:40.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:40 smithi134 ceph-mon[35356]: pgmap v451: 97 pgs: 97 active+clean; 4.9 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T07:03:42.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:42 smithi174 ceph-mon[35637]: pgmap v452: 97 pgs: 97 active+clean; 5.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 24 op/s 2024-09-19T07:03:42.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:42 smithi103 ceph-mon[31158]: pgmap v452: 97 pgs: 97 active+clean; 5.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 24 op/s 2024-09-19T07:03:42.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:42 smithi134 ceph-mon[35356]: pgmap v452: 97 pgs: 97 active+clean; 5.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 66 KiB/s wr, 24 op/s 2024-09-19T07:03:43.029 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:03:43.290 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:43.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.750+0000 7ff0de7e1640 1 -- 172.21.15.103:0/3602381280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0d8100a80 msgr2=0x7ff0d8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:43.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.750+0000 7ff0de7e1640 1 --2- 172.21.15.103:0/3602381280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0d8100a80 0x7ff0d8100ee0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009930 tx=0x7ff0c002a420 comp rx=0 tx=0).stop 2024-09-19T07:03:43.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.750+0000 7ff0de7e1640 1 -- 172.21.15.103:0/3602381280 shutdown_connections 2024-09-19T07:03:43.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.750+0000 7ff0de7e1640 1 --2- 172.21.15.103:0/3602381280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0d8101420 0x7ff0d810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.750+0000 7ff0de7e1640 1 --2- 172.21.15.103:0/3602381280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0d8100a80 0x7ff0d8100ee0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.750+0000 7ff0de7e1640 1 --2- 172.21.15.103:0/3602381280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0d8106a80 0x7ff0d8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.750+0000 7ff0de7e1640 1 -- 172.21.15.103:0/3602381280 >> 172.21.15.103:0/3602381280 conn(0x7ff0d80fc7a0 msgr2=0x7ff0d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:43.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.750+0000 7ff0de7e1640 1 -- 172.21.15.103:0/3602381280 shutdown_connections 2024-09-19T07:03:43.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.750+0000 7ff0de7e1640 1 -- 172.21.15.103:0/3602381280 wait complete. 2024-09-19T07:03:43.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.751+0000 7ff0de7e1640 1 Processor -- start 2024-09-19T07:03:43.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.751+0000 7ff0de7e1640 1 -- start start 2024-09-19T07:03:43.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.751+0000 7ff0de7e1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0d8100a80 0x7ff0d8197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.751+0000 7ff0de7e1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0d8101420 0x7ff0d8197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.751+0000 7ff0de7e1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0d8106a80 0x7ff0d819bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.751+0000 7ff0de7e1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d8110ea0 con 0x7ff0d8101420 2024-09-19T07:03:43.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.751+0000 7ff0de7e1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d8110d20 con 0x7ff0d8106a80 2024-09-19T07:03:43.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.751+0000 7ff0de7e1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d8111020 con 0x7ff0d8100a80 2024-09-19T07:03:43.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.752+0000 7ff0cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0d8101420 0x7ff0d8197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.752+0000 7ff0cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0d8101420 0x7ff0d8197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34882/0 (socket says 172.21.15.103:34882) 2024-09-19T07:03:43.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.752+0000 7ff0cffff640 1 -- 172.21.15.103:0/3027256971 learned_addr learned my addr 172.21.15.103:0/3027256971 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:43.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.752+0000 7ff0d7fff640 1 --2- 172.21.15.103:0/3027256971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0d8100a80 0x7ff0d8197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.752+0000 7ff0dcd7e640 1 --2- 172.21.15.103:0/3027256971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0d8106a80 0x7ff0d819bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.752+0000 7ff0cffff640 1 -- 172.21.15.103:0/3027256971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0d8100a80 msgr2=0x7ff0d8197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:43.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.752+0000 7ff0cffff640 1 --2- 172.21.15.103:0/3027256971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0d8100a80 0x7ff0d8197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.752+0000 7ff0cffff640 1 -- 172.21.15.103:0/3027256971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0d8106a80 msgr2=0x7ff0d819bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:43.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.752+0000 7ff0cffff640 1 --2- 172.21.15.103:0/3027256971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0d8106a80 0x7ff0d819bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.752+0000 7ff0cffff640 1 -- 172.21.15.103:0/3027256971 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d819c3b0 con 0x7ff0d8101420 2024-09-19T07:03:43.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.752+0000 7ff0cffff640 1 --2- 172.21.15.103:0/3027256971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0d8101420 0x7ff0d8197940 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009230 tx=0x7ff0c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:43.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.752+0000 7ff0d5ffb640 1 -- 172.21.15.103:0/3027256971 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c003f030 con 0x7ff0d8101420 2024-09-19T07:03:43.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.752+0000 7ff0de7e1640 1 -- 172.21.15.103:0/3027256971 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d819c640 con 0x7ff0d8101420 2024-09-19T07:03:43.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.752+0000 7ff0d5ffb640 1 -- 172.21.15.103:0/3027256971 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0c0003ad0 con 0x7ff0d8101420 2024-09-19T07:03:43.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.752+0000 7ff0d5ffb640 1 -- 172.21.15.103:0/3027256971 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0035070 con 0x7ff0d8101420 2024-09-19T07:03:43.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.752+0000 7ff0de7e1640 1 -- 172.21.15.103:0/3027256971 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d81a3f80 con 0x7ff0d8101420 2024-09-19T07:03:43.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.755+0000 7ff0d5ffb640 1 -- 172.21.15.103:0/3027256971 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff0c0042030 con 0x7ff0d8101420 2024-09-19T07:03:43.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.755+0000 7ff0e0bba740 1 Processor -- start 2024-09-19T07:03:43.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.755+0000 7ff0e0bba740 1 -- start start 2024-09-19T07:03:43.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.755+0000 7ff0e0bba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f74f380 0x558a1f74f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.755+0000 7ff0e0bba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f74fca0 0x558a1f75c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.755+0000 7ff0e0bba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f75c890 0x558a1f75ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.755+0000 7ff0e0bba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558a1f6b52e0 con 0x558a1f75c890 2024-09-19T07:03:43.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.755+0000 7ff0e0bba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558a1f65cce0 con 0x558a1f74f380 2024-09-19T07:03:43.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.755+0000 7ff0e0bba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558a1f6b12a0 con 0x558a1f74fca0 2024-09-19T07:03:43.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0d5ffb640 1 --2- 172.21.15.103:0/3027256971 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff0c40a33b0 0x7ff0c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f75c890 0x558a1f75ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0dcd7e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f74f380 0x558a1f74f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f75c890 0x558a1f75ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34888/0 (socket says 172.21.15.103:34888) 2024-09-19T07:03:43.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0cffff640 1 -- 172.21.15.103:0/1237101845 learned_addr learned my addr 172.21.15.103:0/1237101845 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:43.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0d7fff640 1 --2- 172.21.15.103:0/1237101845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f74fca0 0x558a1f75c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0dcd7e640 1 --2- 172.21.15.103:0/3027256971 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff0c40a33b0 0x7ff0c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0d5ffb640 1 -- 172.21.15.103:0/3027256971 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7ff0c00f0170 con 0x7ff0d8101420 2024-09-19T07:03:43.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0d7fff640 1 -- 172.21.15.103:0/1237101845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f74f380 msgr2=0x558a1f74f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:43.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0d7fff640 1 --2- 172.21.15.103:0/1237101845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f74f380 0x558a1f74f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0d7fff640 1 -- 172.21.15.103:0/1237101845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f75c890 msgr2=0x558a1f75ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:43.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0d7fff640 1 --2- 172.21.15.103:0/1237101845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f75c890 0x558a1f75ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0d7fff640 1 -- 172.21.15.103:0/1237101845 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a1f641440 con 0x558a1f74fca0 2024-09-19T07:03:43.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0cffff640 1 --2- 172.21.15.103:0/1237101845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f75c890 0x558a1f75ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:43.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0dcd7e640 1 --2- 172.21.15.103:0/3027256971 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff0c40a33b0 0x7ff0c40a5870 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0005fd0 tx=0x7ff0d0009600 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:43.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0d7fff640 1 --2- 172.21.15.103:0/1237101845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f74fca0 0x558a1f75c350 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80099a0 tx=0x7ff0c8009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49b9bd7cdba90f5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:43.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.756+0000 7ff0ce7fc640 1 -- 172.21.15.103:0/1237101845 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8017070 con 0x558a1f74fca0 2024-09-19T07:03:43.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.757+0000 7ff0ce7fc640 1 -- 172.21.15.103:0/1237101845 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0c8013470 con 0x558a1f74fca0 2024-09-19T07:03:43.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.757+0000 7ff0ce7fc640 1 -- 172.21.15.103:0/1237101845 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8012610 con 0x558a1f74fca0 2024-09-19T07:03:43.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.757+0000 7ff0e0bba740 1 -- 172.21.15.103:0/1237101845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f74fca0 msgr2=0x558a1f75c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:43.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.757+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/1237101845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f74fca0 0x558a1f75c350 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80099a0 tx=0x7ff0c8009e60 comp rx=0 tx=0).stop 2024-09-19T07:03:43.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.757+0000 7ff0e0bba740 1 -- 172.21.15.103:0/1237101845 shutdown_connections 2024-09-19T07:03:43.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.757+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/1237101845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f75c890 0x558a1f75ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.757+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/1237101845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f74fca0 0x558a1f75c350 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.757+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/1237101845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f74f380 0x558a1f74f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.757+0000 7ff0e0bba740 1 -- 172.21.15.103:0/1237101845 >> 172.21.15.103:0/1237101845 conn(0x558a1f74ba30 msgr2=0x558a1f74be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:43.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.757+0000 7ff0e0bba740 1 -- 172.21.15.103:0/1237101845 shutdown_connections 2024-09-19T07:03:43.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.757+0000 7ff0e0bba740 1 -- 172.21.15.103:0/1237101845 wait complete. 2024-09-19T07:03:43.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.758+0000 7ff0e0bba740 1 -- start start 2024-09-19T07:03:43.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.758+0000 7ff0e0bba740 1 Processor -- start 2024-09-19T07:03:43.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.758+0000 7ff0e0bba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f74f380 0x558a1f7578d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.758+0000 7ff0e0bba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74fca0 0x558a1f757e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.758+0000 7ff0e0bba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f75c890 0x558a1f758350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.758+0000 7ff0e0bba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558a1f752320 con 0x558a1f74fca0 2024-09-19T07:03:43.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.758+0000 7ff0e0bba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558a1f670050 con 0x558a1f75c890 2024-09-19T07:03:43.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.758+0000 7ff0e0bba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558a1f7524a0 con 0x558a1f74f380 2024-09-19T07:03:43.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.758+0000 7ff0e0bba740 10 client.0 _ll_register_callbacks cb 0x558a1f6b0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:43.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.758+0000 7ff0cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74fca0 0x558a1f757e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.758+0000 7ff0d7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f74f380 0x558a1f7578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.758+0000 7ff0cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74fca0 0x558a1f757e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34896/0 (socket says 172.21.15.103:34896) 2024-09-19T07:03:43.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.758+0000 7ff0cffff640 1 -- 172.21.15.103:0/3485885300 learned_addr learned my addr 172.21.15.103:0/3485885300 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:43.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0dcd7e640 1 --2- 172.21.15.103:0/3485885300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f75c890 0x558a1f758350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0d7fff640 1 -- 172.21.15.103:0/3485885300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f75c890 msgr2=0x558a1f758350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:43.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0d7fff640 1 --2- 172.21.15.103:0/3485885300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f75c890 0x558a1f758350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0d7fff640 1 -- 172.21.15.103:0/3485885300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74fca0 msgr2=0x558a1f757e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:43.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0d7fff640 1 --2- 172.21.15.103:0/3485885300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74fca0 0x558a1f757e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0d7fff640 1 -- 172.21.15.103:0/3485885300 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a1f4263c0 con 0x558a1f74f380 2024-09-19T07:03:43.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0cffff640 1 --2- 172.21.15.103:0/3485885300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74fca0 0x558a1f757e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:03:43.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0dcd7e640 1 --2- 172.21.15.103:0/3485885300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f75c890 0x558a1f758350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:43.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0d7fff640 1 --2- 172.21.15.103:0/3485885300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f74f380 0x558a1f7578d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8004700 tx=0x7ff0c8006ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a998c55f332b659d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:43.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:03:43.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0b77fe640 1 -- 172.21.15.103:0/3485885300 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8017050 con 0x558a1f74f380 2024-09-19T07:03:43.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0e0bba740 1 -- 172.21.15.103:0/3485885300 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0c8013470 con 0x558a1f74f380 2024-09-19T07:03:43.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0b77fe640 1 -- 172.21.15.103:0/3485885300 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7ff0c801c450 con 0x558a1f74f380 2024-09-19T07:03:43.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0b77fe640 1 -- 172.21.15.103:0/3485885300 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff0c801c650 con 0x558a1f74f380 2024-09-19T07:03:43.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0e0bba740 10 client.28162 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:43.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0e0bba740 1 -- 172.21.15.103:0/3485885300 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558a1f76d070 con 0x558a1f74f380 2024-09-19T07:03:43.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.759+0000 7ff0b77fe640 1 -- 172.21.15.103:0/3485885300 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0c8013470 con 0x558a1f74f380 2024-09-19T07:03:43.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.760+0000 7ff0b77fe640 1 -- 172.21.15.103:0/3485885300 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff0c80267b0 con 0x558a1f74f380 2024-09-19T07:03:43.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.760+0000 7ff0e0bba740 10 client.28162 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:43.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.760+0000 7ff0e0bba740 10 client.28162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:43.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.760+0000 7ff0e0bba740 20 client.28162 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:43.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.760+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/3485885300 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558a1f76e530 0x558a1f770920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.760+0000 7ff0e0bba740 4 client.28162 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:43.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.760+0000 7ff0e0bba740 1 -- 172.21.15.103:0/3485885300 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a1f6c2850 con 0x558a1f76e530 2024-09-19T07:03:43.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.760+0000 7ff0cffff640 1 --2- 172.21.15.103:0/3485885300 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558a1f76e530 0x558a1f770920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.760+0000 7ff0b77fe640 10 client.28162 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:43.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.760+0000 7ff0cffff640 1 --2- 172.21.15.103:0/3485885300 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558a1f76e530 0x558a1f770920 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e5143ddb950e6 server_cookie=cee8fb7c880ccc37 in_seq=0 out_seq=0 2024-09-19T07:03:43.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.761+0000 7ff0b77fe640 1 -- 172.21.15.103:0/3485885300 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a1f6c2850 con 0x558a1f76e530 2024-09-19T07:03:43.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.761+0000 7ff0b77fe640 10 client.28162 handle_command_reply: tid=0 2024-09-19T07:03:43.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.761+0000 7ff0e0bba740 1 client.28162 shutdown 2024-09-19T07:03:43.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.761+0000 7ff0e0bba740 1 -- 172.21.15.103:0/3485885300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f74f380 msgr2=0x558a1f7578d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:43.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.761+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/3485885300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f74f380 0x558a1f7578d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8004700 tx=0x7ff0c8006ec0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.761+0000 7ff0e0bba740 1 -- 172.21.15.103:0/3485885300 shutdown_connections 2024-09-19T07:03:43.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.761+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/3485885300 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558a1f76e530 0x558a1f770920 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.761+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/3485885300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f75c890 0x558a1f758350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.761+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/3485885300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74fca0 0x558a1f757e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.761+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/3485885300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f74f380 0x558a1f7578d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.761+0000 7ff0b77fe640 0 client.28162 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:43.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.761+0000 7ff0e0bba740 1 -- 172.21.15.103:0/3485885300 >> 172.21.15.103:0/3485885300 conn(0x558a1f74ba30 msgr2=0x558a1f751790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:43.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.762+0000 7ff0e0bba740 1 -- 172.21.15.103:0/3485885300 shutdown_connections 2024-09-19T07:03:43.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.762+0000 7ff0e0bba740 1 -- 172.21.15.103:0/3485885300 wait complete. 2024-09-19T07:03:43.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.762+0000 7ff0e0bba740 20 client.28162 trim_cache size 0 max 16384 2024-09-19T07:03:43.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.778+0000 7ff0e0bba740 1 Processor -- start 2024-09-19T07:03:43.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0e0bba740 1 -- start start 2024-09-19T07:03:43.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0e0bba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74f380 0x558a1f775f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0e0bba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f74fca0 0x558a1f776490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0e0bba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f75c890 0x558a1f754b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0e0bba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d81ab670 con 0x558a1f74f380 2024-09-19T07:03:43.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0e0bba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d81ad1b0 con 0x558a1f74fca0 2024-09-19T07:03:43.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0e0bba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d81abd40 con 0x558a1f75c890 2024-09-19T07:03:43.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0d7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74f380 0x558a1f775f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0d7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74f380 0x558a1f775f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34912/0 (socket says 172.21.15.103:34912) 2024-09-19T07:03:43.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0d7fff640 1 -- 172.21.15.103:0/2569652057 learned_addr learned my addr 172.21.15.103:0/2569652057 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:43.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0cffff640 1 --2- 172.21.15.103:0/2569652057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f74fca0 0x558a1f776490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0dcd7e640 1 --2- 172.21.15.103:0/2569652057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f75c890 0x558a1f754b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0d7fff640 1 -- 172.21.15.103:0/2569652057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f75c890 msgr2=0x558a1f754b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:43.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0d7fff640 1 --2- 172.21.15.103:0/2569652057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f75c890 0x558a1f754b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0d7fff640 1 -- 172.21.15.103:0/2569652057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f74fca0 msgr2=0x558a1f776490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:43.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0d7fff640 1 --2- 172.21.15.103:0/2569652057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f74fca0 0x558a1f776490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0d7fff640 1 -- 172.21.15.103:0/2569652057 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a1f74b280 con 0x558a1f74f380 2024-09-19T07:03:43.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0d7fff640 1 --2- 172.21.15.103:0/2569652057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74f380 0x558a1f775f50 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8006d20 tx=0x7ff0c8009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e5038ed305e9ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:43.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.779+0000 7ff0cdffb640 1 -- 172.21.15.103:0/2569652057 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8033070 con 0x558a1f74f380 2024-09-19T07:03:43.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.780+0000 7ff0cdffb640 1 -- 172.21.15.103:0/2569652057 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0c8013470 con 0x558a1f74f380 2024-09-19T07:03:43.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.780+0000 7ff0cdffb640 1 -- 172.21.15.103:0/2569652057 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8017040 con 0x558a1f74f380 2024-09-19T07:03:43.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.780+0000 7ff0e0bba740 1 -- 172.21.15.103:0/2569652057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74f380 msgr2=0x558a1f775f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:43.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.780+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/2569652057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74f380 0x558a1f775f50 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8006d20 tx=0x7ff0c8009110 comp rx=0 tx=0).stop 2024-09-19T07:03:43.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.780+0000 7ff0e0bba740 1 -- 172.21.15.103:0/2569652057 shutdown_connections 2024-09-19T07:03:43.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.780+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/2569652057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f75c890 0x558a1f754b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.780+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/2569652057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f74fca0 0x558a1f776490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.780+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/2569652057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74f380 0x558a1f775f50 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.780+0000 7ff0e0bba740 1 -- 172.21.15.103:0/2569652057 >> 172.21.15.103:0/2569652057 conn(0x558a1f74ba30 msgr2=0x558a1f74ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:43.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.780+0000 7ff0e0bba740 1 -- 172.21.15.103:0/2569652057 shutdown_connections 2024-09-19T07:03:43.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.780+0000 7ff0e0bba740 1 -- 172.21.15.103:0/2569652057 wait complete. 2024-09-19T07:03:43.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.781+0000 7ff0e0bba740 1 -- start start 2024-09-19T07:03:43.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.781+0000 7ff0e0bba740 1 Processor -- start 2024-09-19T07:03:43.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.781+0000 7ff0e0bba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f74f380 0x558a1f769ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.781+0000 7ff0e0bba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74fca0 0x558a1f764fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.781+0000 7ff0e0bba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f75c890 0x558a1f765510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.781+0000 7ff0e0bba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d81acae0 con 0x558a1f74fca0 2024-09-19T07:03:43.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.781+0000 7ff0e0bba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d81ac410 con 0x558a1f74f380 2024-09-19T07:03:43.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.781+0000 7ff0e0bba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d81a9670 con 0x558a1f75c890 2024-09-19T07:03:43.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.781+0000 7ff0e0bba740 10 client.0 _ll_register_callbacks cb 0x7ff0d0012790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:43.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.781+0000 7ff0cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74fca0 0x558a1f764fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.781+0000 7ff0cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74fca0 0x558a1f764fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34916/0 (socket says 172.21.15.103:34916) 2024-09-19T07:03:43.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.781+0000 7ff0cffff640 1 -- 172.21.15.103:0/363735706 learned_addr learned my addr 172.21.15.103:0/363735706 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:43.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.781+0000 7ff0d7fff640 1 --2- 172.21.15.103:0/363735706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f74f380 0x558a1f769ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0cffff640 1 -- 172.21.15.103:0/363735706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f75c890 msgr2=0x558a1f765510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:43.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0dcd7e640 1 --2- 172.21.15.103:0/363735706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f75c890 0x558a1f765510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0cffff640 1 --2- 172.21.15.103:0/363735706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f75c890 0x558a1f765510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0cffff640 1 -- 172.21.15.103:0/363735706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f74f380 msgr2=0x558a1f769ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:43.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0cffff640 1 --2- 172.21.15.103:0/363735706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f74f380 0x558a1f769ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0cffff640 1 -- 172.21.15.103:0/363735706 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a1f765a50 con 0x558a1f74fca0 2024-09-19T07:03:43.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0dcd7e640 1 --2- 172.21.15.103:0/363735706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f75c890 0x558a1f765510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:43.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0cffff640 1 --2- 172.21.15.103:0/363735706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74fca0 0x558a1f764fd0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00ed950 tx=0x7ff0c0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd86e79b7186e8c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:43.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:43.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0b77fe640 1 -- 172.21.15.103:0/363735706 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0049040 con 0x558a1f74fca0 2024-09-19T07:03:43.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0b77fe640 1 -- 172.21.15.103:0/363735706 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7ff0c00edb20 con 0x558a1f74fca0 2024-09-19T07:03:43.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0e0bba740 1 -- 172.21.15.103:0/363735706 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0c8013470 con 0x558a1f74fca0 2024-09-19T07:03:43.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0b77fe640 1 -- 172.21.15.103:0/363735706 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff0c00c07e0 con 0x558a1f74fca0 2024-09-19T07:03:43.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0e0bba740 10 client.20370 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:43.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0e0bba740 1 -- 172.21.15.103:0/363735706 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558a1f7d8d10 con 0x558a1f74fca0 2024-09-19T07:03:43.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0b77fe640 1 -- 172.21.15.103:0/363735706 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0c8013470 con 0x558a1f74fca0 2024-09-19T07:03:43.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0b77fe640 1 -- 172.21.15.103:0/363735706 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff0c00b8aa0 con 0x558a1f74fca0 2024-09-19T07:03:43.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.782+0000 7ff0e0bba740 10 client.20370 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:43.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.783+0000 7ff0e0bba740 10 client.20370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:43.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.783+0000 7ff0e0bba740 20 client.20370 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:43.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.783+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/363735706 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558a1f7d9440 0x558a1f7db860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:43.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.783+0000 7ff0e0bba740 4 client.20370 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:03:43.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.783+0000 7ff0e0bba740 1 -- 172.21.15.103:0/363735706 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558a1f6bcd60 con 0x558a1f7d9440 2024-09-19T07:03:43.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.783+0000 7ff0d7fff640 1 --2- 172.21.15.103:0/363735706 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558a1f7d9440 0x558a1f7db860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:43.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.783+0000 7ff0b77fe640 10 client.20370 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:43.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.783+0000 7ff0d7fff640 1 --2- 172.21.15.103:0/363735706 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558a1f7d9440 0x558a1f7db860 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58081f939a82fc08 server_cookie=226257e64368c10d in_seq=0 out_seq=0 2024-09-19T07:03:43.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.783+0000 7ff0b77fe640 1 -- 172.21.15.103:0/363735706 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558a1f6bcd60 con 0x558a1f7d9440 2024-09-19T07:03:43.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.783+0000 7ff0b77fe640 10 client.20370 handle_command_reply: tid=0 2024-09-19T07:03:43.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.783+0000 7ff0e0bba740 1 client.20370 shutdown 2024-09-19T07:03:43.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0e0bba740 1 -- 172.21.15.103:0/363735706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74fca0 msgr2=0x558a1f764fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:43.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/363735706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74fca0 0x558a1f764fd0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00ed950 tx=0x7ff0c0005e50 comp rx=0 tx=0).stop 2024-09-19T07:03:43.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0e0bba740 1 -- 172.21.15.103:0/363735706 shutdown_connections 2024-09-19T07:03:43.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/363735706 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558a1f7d9440 0x558a1f7db860 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/363735706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a1f75c890 0x558a1f765510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/363735706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a1f74fca0 0x558a1f764fd0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0e0bba740 1 --2- 172.21.15.103:0/363735706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a1f74f380 0x558a1f769ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0e0bba740 1 -- 172.21.15.103:0/363735706 >> 172.21.15.103:0/363735706 conn(0x558a1f74ba30 msgr2=0x558a1f750e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:43.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0b77fe640 0 client.20370 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:43.808 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:03:43.808 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:03:43.809 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:03:43.809 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:03:43.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0e0bba740 1 -- 172.21.15.103:0/363735706 shutdown_connections 2024-09-19T07:03:43.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0e0bba740 1 -- 172.21.15.103:0/363735706 wait complete. 2024-09-19T07:03:43.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0e0bba740 20 client.20370 trim_cache size 0 max 16384 2024-09-19T07:03:43.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0de7e1640 1 -- 172.21.15.103:0/3027256971 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff0c40a33b0 msgr2=0x7ff0c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:43.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0de7e1640 1 --2- 172.21.15.103:0/3027256971 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff0c40a33b0 0x7ff0c40a5870 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0005fd0 tx=0x7ff0d0009600 comp rx=0 tx=0).stop 2024-09-19T07:03:43.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0de7e1640 1 -- 172.21.15.103:0/3027256971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0d8101420 msgr2=0x7ff0d8197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:43.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0de7e1640 1 --2- 172.21.15.103:0/3027256971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0d8101420 0x7ff0d8197940 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009230 tx=0x7ff0c0004270 comp rx=0 tx=0).stop 2024-09-19T07:03:43.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0de7e1640 1 -- 172.21.15.103:0/3027256971 shutdown_connections 2024-09-19T07:03:43.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0de7e1640 1 --2- 172.21.15.103:0/3027256971 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff0c40a33b0 0x7ff0c40a5870 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0de7e1640 1 --2- 172.21.15.103:0/3027256971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0d8106a80 0x7ff0d819bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0de7e1640 1 --2- 172.21.15.103:0/3027256971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0d8101420 0x7ff0d8197940 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0de7e1640 1 --2- 172.21.15.103:0/3027256971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0d8100a80 0x7ff0d8197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:43.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.784+0000 7ff0de7e1640 1 -- 172.21.15.103:0/3027256971 >> 172.21.15.103:0/3027256971 conn(0x7ff0d80fc7a0 msgr2=0x7ff0d80fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:43.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.785+0000 7ff0de7e1640 1 -- 172.21.15.103:0/3027256971 shutdown_connections 2024-09-19T07:03:43.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:43.785+0000 7ff0de7e1640 1 -- 172.21.15.103:0/3027256971 wait complete. 2024-09-19T07:03:44.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:44 smithi174 ceph-mon[35637]: pgmap v453: 97 pgs: 97 active+clean; 5.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T07:03:44.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:44 smithi103 ceph-mon[31158]: pgmap v453: 97 pgs: 97 active+clean; 5.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T07:03:44.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:44 smithi134 ceph-mon[35356]: pgmap v453: 97 pgs: 97 active+clean; 5.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T07:03:44.566 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:03:44.567 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:03:44.821 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:45.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.320+0000 7f690b343640 1 -- 172.21.15.103:0/1041019554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6904101de0 msgr2=0x7f69041021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:45.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.321+0000 7f690b343640 1 --2- 172.21.15.103:0/1041019554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6904101de0 0x7f69041021c0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009ca0 tx=0x7f68f402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:03:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.321+0000 7f690b343640 1 -- 172.21.15.103:0/1041019554 shutdown_connections 2024-09-19T07:03:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.321+0000 7f690b343640 1 --2- 172.21.15.103:0/1041019554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6904106f10 0x7f690410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.321+0000 7f690b343640 1 --2- 172.21.15.103:0/1041019554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6904102790 0x7f69041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.321+0000 7f690b343640 1 --2- 172.21.15.103:0/1041019554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6904101de0 0x7f69041021c0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.321+0000 7f690b343640 1 -- 172.21.15.103:0/1041019554 >> 172.21.15.103:0/1041019554 conn(0x7f69040fd8c0 msgr2=0x7f69040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.321+0000 7f690b343640 1 -- 172.21.15.103:0/1041019554 shutdown_connections 2024-09-19T07:03:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.321+0000 7f690b343640 1 -- 172.21.15.103:0/1041019554 wait complete. 2024-09-19T07:03:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.321+0000 7f690b343640 1 Processor -- start 2024-09-19T07:03:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.321+0000 7f690b343640 1 -- start start 2024-09-19T07:03:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f690b343640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6904101de0 0x7f6904198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f690b343640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6904102790 0x7f6904198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f690b343640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6904106f10 0x7f690419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:45.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f690b343640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6904111f30 con 0x7f6904101de0 2024-09-19T07:03:45.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f690b343640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6904111db0 con 0x7f6904102790 2024-09-19T07:03:45.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f690b343640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f69041120b0 con 0x7f6904106f10 2024-09-19T07:03:45.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f69090df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6904101de0 0x7f6904198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:45.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f69090df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6904101de0 0x7f6904198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34930/0 (socket says 172.21.15.103:34930) 2024-09-19T07:03:45.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f69090df640 1 -- 172.21.15.103:0/2148104592 learned_addr learned my addr 172.21.15.103:0/2148104592 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:45.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f69088de640 1 --2- 172.21.15.103:0/2148104592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6904102790 0x7f6904198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:45.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f69098e0640 1 --2- 172.21.15.103:0/2148104592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6904106f10 0x7f690419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:45.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f69090df640 1 -- 172.21.15.103:0/2148104592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6904106f10 msgr2=0x7f690419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:45.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f69090df640 1 --2- 172.21.15.103:0/2148104592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6904106f10 0x7f690419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:45.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f69090df640 1 -- 172.21.15.103:0/2148104592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6904102790 msgr2=0x7f6904198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:45.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f69090df640 1 --2- 172.21.15.103:0/2148104592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6904102790 0x7f6904198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:45.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f69090df640 1 -- 172.21.15.103:0/2148104592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690419d3e0 con 0x7f6904101de0 2024-09-19T07:03:45.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f69088de640 1 --2- 172.21.15.103:0/2148104592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6904102790 0x7f6904198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:45.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f69098e0640 1 --2- 172.21.15.103:0/2148104592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6904106f10 0x7f690419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:45.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f69090df640 1 --2- 172.21.15.103:0/2148104592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6904101de0 0x7f6904198430 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f68f40097b0 tx=0x7f68f4031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:45.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.322+0000 7f69027fc640 1 -- 172.21.15.103:0/2148104592 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f402fba0 con 0x7f6904101de0 2024-09-19T07:03:45.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.323+0000 7f69027fc640 1 -- 172.21.15.103:0/2148104592 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68f402fd40 con 0x7f6904101de0 2024-09-19T07:03:45.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.323+0000 7f690b343640 1 -- 172.21.15.103:0/2148104592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690419d670 con 0x7f6904101de0 2024-09-19T07:03:45.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.323+0000 7f69027fc640 1 -- 172.21.15.103:0/2148104592 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f403b930 con 0x7f6904101de0 2024-09-19T07:03:45.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.323+0000 7f690b343640 1 -- 172.21.15.103:0/2148104592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69041a4fb0 con 0x7f6904101de0 2024-09-19T07:03:45.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.324+0000 7f690b343640 1 -- 172.21.15.103:0/2148104592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69041021c0 con 0x7f6904101de0 2024-09-19T07:03:45.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.325+0000 7f69027fc640 1 -- 172.21.15.103:0/2148104592 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f68f4030cc0 con 0x7f6904101de0 2024-09-19T07:03:45.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.325+0000 7f69027fc640 1 --2- 172.21.15.103:0/2148104592 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68ec0a3430 0x7f68ec0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:45.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.326+0000 7f69088de640 1 --2- 172.21.15.103:0/2148104592 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68ec0a3430 0x7f68ec0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:45.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.326+0000 7f69027fc640 1 -- 172.21.15.103:0/2148104592 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f68f40f1170 con 0x7f6904101de0 2024-09-19T07:03:45.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.326+0000 7f69088de640 1 --2- 172.21.15.103:0/2148104592 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68ec0a3430 0x7f68ec0a58f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f69040fdd00 tx=0x7f68f8005d90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:45.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.330+0000 7f69027fc640 1 -- 172.21.15.103:0/2148104592 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f68f40bdc40 con 0x7f6904101de0 2024-09-19T07:03:45.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.530+0000 7f690b343640 1 -- 172.21.15.103:0/2148104592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6904199790 con 0x7f6904101de0 2024-09-19T07:03:45.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.531+0000 7f69027fc640 1 -- 172.21.15.103:0/2148104592 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f68f40bd540 con 0x7f6904101de0 2024-09-19T07:03:45.537 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:03:45.537 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:03:45.537 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:03:45.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.533+0000 7f690b343640 1 -- 172.21.15.103:0/2148104592 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68ec0a3430 msgr2=0x7f68ec0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:45.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.533+0000 7f690b343640 1 --2- 172.21.15.103:0/2148104592 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68ec0a3430 0x7f68ec0a58f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f69040fdd00 tx=0x7f68f8005d90 comp rx=0 tx=0).stop 2024-09-19T07:03:45.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.533+0000 7f690b343640 1 -- 172.21.15.103:0/2148104592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6904101de0 msgr2=0x7f6904198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:45.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.533+0000 7f690b343640 1 --2- 172.21.15.103:0/2148104592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6904101de0 0x7f6904198430 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f68f40097b0 tx=0x7f68f4031b10 comp rx=0 tx=0).stop 2024-09-19T07:03:45.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.533+0000 7f690b343640 1 -- 172.21.15.103:0/2148104592 shutdown_connections 2024-09-19T07:03:45.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.533+0000 7f690b343640 1 --2- 172.21.15.103:0/2148104592 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68ec0a3430 0x7f68ec0a58f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:45.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.533+0000 7f690b343640 1 --2- 172.21.15.103:0/2148104592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6904106f10 0x7f690419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:45.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.533+0000 7f690b343640 1 --2- 172.21.15.103:0/2148104592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6904102790 0x7f6904198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:45.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.533+0000 7f690b343640 1 --2- 172.21.15.103:0/2148104592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6904101de0 0x7f6904198430 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:45.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.533+0000 7f690b343640 1 -- 172.21.15.103:0/2148104592 >> 172.21.15.103:0/2148104592 conn(0x7f69040fd8c0 msgr2=0x7f69040feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:45.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.534+0000 7f690b343640 1 -- 172.21.15.103:0/2148104592 shutdown_connections 2024-09-19T07:03:45.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:45.534+0000 7f690b343640 1 -- 172.21.15.103:0/2148104592 wait complete. 2024-09-19T07:03:46.178 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:46 smithi103 ceph-mon[31158]: pgmap v454: 97 pgs: 97 active+clean; 5.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T07:03:46.178 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:46 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2148104592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:03:46.208 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:03:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:46 smithi174 ceph-mon[35637]: pgmap v454: 97 pgs: 97 active+clean; 5.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T07:03:46.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:46 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2148104592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:03:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:46 smithi134 ceph-mon[35356]: pgmap v454: 97 pgs: 97 active+clean; 5.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T07:03:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:46 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2148104592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:03:46.459 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:46.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 -- 172.21.15.103:0/3934755532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b20101bc0 msgr2=0x7f3b2010d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:46.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 --2- 172.21.15.103:0/3934755532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b20101bc0 0x7f3b2010d910 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009ca0 tx=0x7f3b1802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 -- 172.21.15.103:0/3934755532 shutdown_connections 2024-09-19T07:03:46.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 --2- 172.21.15.103:0/3934755532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b20101bc0 0x7f3b2010d910 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 --2- 172.21.15.103:0/3934755532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b20101240 0x7f3b20101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 --2- 172.21.15.103:0/3934755532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b20105a90 0x7f3b20105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 -- 172.21.15.103:0/3934755532 >> 172.21.15.103:0/3934755532 conn(0x7f3b20077210 msgr2=0x7f3b200fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:46.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 -- 172.21.15.103:0/3934755532 shutdown_connections 2024-09-19T07:03:46.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 -- 172.21.15.103:0/3934755532 wait complete. 2024-09-19T07:03:46.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 Processor -- start 2024-09-19T07:03:46.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 -- start start 2024-09-19T07:03:46.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b20101240 0x7f3b20198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b20101bc0 0x7f3b20198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b20105a90 0x7f3b2019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b20110fd0 con 0x7f3b20101bc0 2024-09-19T07:03:46.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b20110e50 con 0x7f3b20105a90 2024-09-19T07:03:46.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.908+0000 7f3b26d49640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b20111150 con 0x7f3b20101240 2024-09-19T07:03:46.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b17fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b20101bc0 0x7f3b20198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b17fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b20101bc0 0x7f3b20198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34962/0 (socket says 172.21.15.103:34962) 2024-09-19T07:03:46.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b17fff640 1 -- 172.21.15.103:0/437852223 learned_addr learned my addr 172.21.15.103:0/437852223 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:46.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b252e6640 1 --2- 172.21.15.103:0/437852223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b20105a90 0x7f3b2019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b24ae5640 1 --2- 172.21.15.103:0/437852223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b20101240 0x7f3b20198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b17fff640 1 -- 172.21.15.103:0/437852223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b20101240 msgr2=0x7f3b20198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:46.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b17fff640 1 --2- 172.21.15.103:0/437852223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b20101240 0x7f3b20198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b17fff640 1 -- 172.21.15.103:0/437852223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b20105a90 msgr2=0x7f3b2019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:46.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b17fff640 1 --2- 172.21.15.103:0/437852223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b20105a90 0x7f3b2019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b17fff640 1 -- 172.21.15.103:0/437852223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2019d4e0 con 0x7f3b20101bc0 2024-09-19T07:03:46.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b24ae5640 1 --2- 172.21.15.103:0/437852223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b20101240 0x7f3b20198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:46.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b17fff640 1 --2- 172.21.15.103:0/437852223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b20101bc0 0x7f3b20198a70 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000c970 tx=0x7f3b1000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:46.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b15ffb640 1 -- 172.21.15.103:0/437852223 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10007bb0 con 0x7f3b20101bc0 2024-09-19T07:03:46.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b15ffb640 1 -- 172.21.15.103:0/437852223 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b10007d50 con 0x7f3b20101bc0 2024-09-19T07:03:46.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b15ffb640 1 -- 172.21.15.103:0/437852223 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1000f450 con 0x7f3b20101bc0 2024-09-19T07:03:46.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b26d49640 1 -- 172.21.15.103:0/437852223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2019d7d0 con 0x7f3b20101bc0 2024-09-19T07:03:46.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.909+0000 7f3b26d49640 1 -- 172.21.15.103:0/437852223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b201a50b0 con 0x7f3b20101bc0 2024-09-19T07:03:46.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.911+0000 7f3b15ffb640 1 -- 172.21.15.103:0/437852223 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3b10005020 con 0x7f3b20101bc0 2024-09-19T07:03:46.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.912+0000 7f3b29122740 1 Processor -- start 2024-09-19T07:03:46.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.912+0000 7f3b29122740 1 -- start start 2024-09-19T07:03:46.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.912+0000 7f3b15ffb640 1 --2- 172.21.15.103:0/437852223 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3b080a33b0 0x7f3b080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.912+0000 7f3b29122740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914cb40 0x55a1f914b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.912+0000 7f3b29122740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f914b5d0 0x55a1f914ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.912+0000 7f3b29122740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f9156370 0x55a1f9158760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.912+0000 7f3b29122740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a1f90b12f0 con 0x55a1f914cb40 2024-09-19T07:03:46.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.912+0000 7f3b29122740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a1f9058cf0 con 0x55a1f914b5d0 2024-09-19T07:03:46.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.912+0000 7f3b29122740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a1f90ad2b0 con 0x55a1f9156370 2024-09-19T07:03:46.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.912+0000 7f3b24ae5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914cb40 0x55a1f914b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.912+0000 7f3b252e6640 1 --2- 172.21.15.103:0/437852223 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3b080a33b0 0x7f3b080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.912+0000 7f3b24ae5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914cb40 0x55a1f914b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34964/0 (socket says 172.21.15.103:34964) 2024-09-19T07:03:46.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.912+0000 7f3b24ae5640 1 -- 172.21.15.103:0/3262146315 learned_addr learned my addr 172.21.15.103:0/3262146315 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:46.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.912+0000 7f3b15ffb640 1 -- 172.21.15.103:0/437852223 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f3b100cfb80 con 0x7f3b20101bc0 2024-09-19T07:03:46.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.913+0000 7f3b17fff640 1 --2- 172.21.15.103:0/3262146315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f914b5d0 0x55a1f914ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.913+0000 7f3b252e6640 1 --2- 172.21.15.103:0/3262146315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f9156370 0x55a1f9158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.913+0000 7f3b24ae5640 1 -- 172.21.15.103:0/3262146315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f9156370 msgr2=0x55a1f9158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:46.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.913+0000 7f3b24ae5640 1 --2- 172.21.15.103:0/3262146315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f9156370 0x55a1f9158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.913+0000 7f3b24ae5640 1 -- 172.21.15.103:0/3262146315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f914b5d0 msgr2=0x55a1f914ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:46.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.913+0000 7f3b24ae5640 1 --2- 172.21.15.103:0/3262146315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f914b5d0 0x55a1f914ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.913+0000 7f3b24ae5640 1 -- 172.21.15.103:0/3262146315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1f903d3a0 con 0x55a1f914cb40 2024-09-19T07:03:46.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.913+0000 7f3b17fff640 1 --2- 172.21.15.103:0/3262146315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f914b5d0 0x55a1f914ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:46.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.913+0000 7f3b252e6640 1 --2- 172.21.15.103:0/437852223 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3b080a33b0 0x7f3b080a5870 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3b180098d0 tx=0x7f3b18031db0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:46.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.913+0000 7f3b24ae5640 1 --2- 172.21.15.103:0/3262146315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914cb40 0x55a1f914b090 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c007850 tx=0x7f3b1c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c1e68dab08b1b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:46.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.913+0000 7f3b06ffd640 1 -- 172.21.15.103:0/3262146315 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c00d950 con 0x55a1f914cb40 2024-09-19T07:03:46.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.913+0000 7f3b06ffd640 1 -- 172.21.15.103:0/3262146315 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b1c018070 con 0x55a1f914cb40 2024-09-19T07:03:46.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.913+0000 7f3b06ffd640 1 -- 172.21.15.103:0/3262146315 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c0123e0 con 0x55a1f914cb40 2024-09-19T07:03:46.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.914+0000 7f3b29122740 1 -- 172.21.15.103:0/3262146315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914cb40 msgr2=0x55a1f914b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:46.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.914+0000 7f3b29122740 1 --2- 172.21.15.103:0/3262146315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914cb40 0x55a1f914b090 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c007850 tx=0x7f3b1c007d10 comp rx=0 tx=0).stop 2024-09-19T07:03:46.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.914+0000 7f3b29122740 1 -- 172.21.15.103:0/3262146315 shutdown_connections 2024-09-19T07:03:46.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.914+0000 7f3b29122740 1 --2- 172.21.15.103:0/3262146315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f9156370 0x55a1f9158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.914+0000 7f3b29122740 1 --2- 172.21.15.103:0/3262146315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f914b5d0 0x55a1f914ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.914+0000 7f3b29122740 1 --2- 172.21.15.103:0/3262146315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914cb40 0x55a1f914b090 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.914+0000 7f3b29122740 1 -- 172.21.15.103:0/3262146315 >> 172.21.15.103:0/3262146315 conn(0x55a1f9114310 msgr2=0x55a1f9147c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:46.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.914+0000 7f3b29122740 1 -- 172.21.15.103:0/3262146315 shutdown_connections 2024-09-19T07:03:46.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.914+0000 7f3b29122740 1 -- 172.21.15.103:0/3262146315 wait complete. 2024-09-19T07:03:46.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.914+0000 7f3b29122740 1 -- start start 2024-09-19T07:03:46.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.914+0000 7f3b29122740 1 Processor -- start 2024-09-19T07:03:46.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b29122740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914b5d0 0x55a1f9155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b29122740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f914cb40 0x55a1f9152f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b29122740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f9156370 0x55a1f91534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b29122740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a1f9159070 con 0x55a1f914b5d0 2024-09-19T07:03:46.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b29122740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a1f906c060 con 0x55a1f9156370 2024-09-19T07:03:46.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b29122740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a1f91591f0 con 0x55a1f914cb40 2024-09-19T07:03:46.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b29122740 10 client.0 _ll_register_callbacks cb 0x55a1f8dbf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:46.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b24ae5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914b5d0 0x55a1f9155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b24ae5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914b5d0 0x55a1f9155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34972/0 (socket says 172.21.15.103:34972) 2024-09-19T07:03:46.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b24ae5640 1 -- 172.21.15.103:0/1845297103 learned_addr learned my addr 172.21.15.103:0/1845297103 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:46.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b17fff640 1 --2- 172.21.15.103:0/1845297103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f914cb40 0x55a1f9152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b252e6640 1 --2- 172.21.15.103:0/1845297103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f9156370 0x55a1f91534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b24ae5640 1 -- 172.21.15.103:0/1845297103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f914cb40 msgr2=0x55a1f9152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:46.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b24ae5640 1 --2- 172.21.15.103:0/1845297103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f914cb40 0x55a1f9152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b24ae5640 1 -- 172.21.15.103:0/1845297103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f9156370 msgr2=0x55a1f91534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:46.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b24ae5640 1 --2- 172.21.15.103:0/1845297103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f9156370 0x55a1f91534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b24ae5640 1 -- 172.21.15.103:0/1845297103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1f8e223c0 con 0x55a1f914b5d0 2024-09-19T07:03:46.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b24ae5640 1 --2- 172.21.15.103:0/1845297103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914b5d0 0x55a1f9155cd0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c000c00 tx=0x7f3b1c013870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3cb5a9b08654fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:46.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3afffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:46.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3afffff640 1 -- 172.21.15.103:0/1845297103 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c00dd60 con 0x55a1f914b5d0 2024-09-19T07:03:46.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3afffff640 1 -- 172.21.15.103:0/1845297103 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f3b1c024420 con 0x55a1f914b5d0 2024-09-19T07:03:46.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.915+0000 7f3b29122740 1 -- 172.21.15.103:0/1845297103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b1c018070 con 0x55a1f914b5d0 2024-09-19T07:03:46.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.916+0000 7f3afffff640 1 -- 172.21.15.103:0/1845297103 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b1c024620 con 0x55a1f914b5d0 2024-09-19T07:03:46.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.916+0000 7f3b29122740 10 client.20388 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:46.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.916+0000 7f3b29122740 1 -- 172.21.15.103:0/1845297103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a1f9164a80 con 0x55a1f914b5d0 2024-09-19T07:03:46.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.916+0000 7f3afffff640 1 -- 172.21.15.103:0/1845297103 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b1c018070 con 0x55a1f914b5d0 2024-09-19T07:03:46.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.916+0000 7f3afffff640 1 -- 172.21.15.103:0/1845297103 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3b1c02d8c0 con 0x55a1f914b5d0 2024-09-19T07:03:46.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.916+0000 7f3b29122740 10 client.20388 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:46.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.916+0000 7f3b29122740 10 client.20388 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:03:46.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.916+0000 7f3b29122740 20 client.20388 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:46.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.916+0000 7f3b29122740 1 --2- 172.21.15.103:0/1845297103 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55a1f9165ee0 0x55a1f91682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.916+0000 7f3b29122740 4 client.20388 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:46.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.916+0000 7f3b29122740 1 -- 172.21.15.103:0/1845297103 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1f90be860 con 0x55a1f9165ee0 2024-09-19T07:03:46.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.916+0000 7f3b17fff640 1 --2- 172.21.15.103:0/1845297103 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55a1f9165ee0 0x55a1f91682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.917+0000 7f3b17fff640 1 --2- 172.21.15.103:0/1845297103 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55a1f9165ee0 0x55a1f91682d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=676365873fb12051 server_cookie=c0f96697e79a84b9 in_seq=0 out_seq=0 2024-09-19T07:03:46.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.917+0000 7f3afffff640 10 client.20388 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:03:46.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.918+0000 7f3afffff640 1 -- 172.21.15.103:0/1845297103 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1f90be860 con 0x55a1f9165ee0 2024-09-19T07:03:46.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.918+0000 7f3afffff640 10 client.20388 handle_command_reply: tid=0 2024-09-19T07:03:46.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.918+0000 7f3b29122740 1 client.20388 shutdown 2024-09-19T07:03:46.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.918+0000 7f3b29122740 1 -- 172.21.15.103:0/1845297103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914b5d0 msgr2=0x55a1f9155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:46.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.918+0000 7f3b29122740 1 --2- 172.21.15.103:0/1845297103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914b5d0 0x55a1f9155cd0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c000c00 tx=0x7f3b1c013870 comp rx=0 tx=0).stop 2024-09-19T07:03:46.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.918+0000 7f3b29122740 1 -- 172.21.15.103:0/1845297103 shutdown_connections 2024-09-19T07:03:46.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.918+0000 7f3b29122740 1 --2- 172.21.15.103:0/1845297103 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55a1f9165ee0 0x55a1f91682d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.918+0000 7f3b29122740 1 --2- 172.21.15.103:0/1845297103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f9156370 0x55a1f91534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.918+0000 7f3b29122740 1 --2- 172.21.15.103:0/1845297103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f914cb40 0x55a1f9152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.918+0000 7f3afffff640 0 client.20388 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:03:46.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.918+0000 7f3b29122740 1 --2- 172.21.15.103:0/1845297103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914b5d0 0x55a1f9155cd0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.918+0000 7f3b29122740 1 -- 172.21.15.103:0/1845297103 >> 172.21.15.103:0/1845297103 conn(0x55a1f9114310 msgr2=0x55a1f914d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:46.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.918+0000 7f3b29122740 1 -- 172.21.15.103:0/1845297103 shutdown_connections 2024-09-19T07:03:46.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.918+0000 7f3b29122740 1 -- 172.21.15.103:0/1845297103 wait complete. 2024-09-19T07:03:46.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.918+0000 7f3b29122740 20 client.20388 trim_cache size 0 max 16384 2024-09-19T07:03:46.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b29122740 1 Processor -- start 2024-09-19T07:03:46.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b29122740 1 -- start start 2024-09-19T07:03:46.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b29122740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f914b5d0 0x55a1f9162310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b29122740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914cb40 0x55a1f914e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b29122740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f9156370 0x55a1f914e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b29122740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b201ad940 con 0x55a1f914cb40 2024-09-19T07:03:46.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b29122740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b201af480 con 0x55a1f9156370 2024-09-19T07:03:46.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b29122740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b201ae010 con 0x55a1f914b5d0 2024-09-19T07:03:46.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b17fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914cb40 0x55a1f914e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b17fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914cb40 0x55a1f914e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34982/0 (socket says 172.21.15.103:34982) 2024-09-19T07:03:46.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b17fff640 1 -- 172.21.15.103:0/1163704030 learned_addr learned my addr 172.21.15.103:0/1163704030 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:46.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b24ae5640 1 --2- 172.21.15.103:0/1163704030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f914b5d0 0x55a1f9162310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b252e6640 1 --2- 172.21.15.103:0/1163704030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f9156370 0x55a1f914e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b17fff640 1 -- 172.21.15.103:0/1163704030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f914b5d0 msgr2=0x55a1f9162310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:46.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b17fff640 1 --2- 172.21.15.103:0/1163704030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f914b5d0 0x55a1f9162310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b17fff640 1 -- 172.21.15.103:0/1163704030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f9156370 msgr2=0x55a1f914e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:46.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b17fff640 1 --2- 172.21.15.103:0/1163704030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f9156370 0x55a1f914e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b17fff640 1 -- 172.21.15.103:0/1163704030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1f91514d0 con 0x55a1f914cb40 2024-09-19T07:03:46.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b252e6640 1 --2- 172.21.15.103:0/1163704030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f9156370 0x55a1f914e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:46.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.931+0000 7f3b17fff640 1 --2- 172.21.15.103:0/1163704030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914cb40 0x55a1f914e120 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f3b1009bd70 tx=0x7f3b10004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5fdc08d54f9d01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:46.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.932+0000 7f3b067fc640 1 -- 172.21.15.103:0/1163704030 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1001d040 con 0x55a1f914cb40 2024-09-19T07:03:46.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.932+0000 7f3b067fc640 1 -- 172.21.15.103:0/1163704030 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b100a4a50 con 0x55a1f914cb40 2024-09-19T07:03:46.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.932+0000 7f3b067fc640 1 -- 172.21.15.103:0/1163704030 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10005310 con 0x55a1f914cb40 2024-09-19T07:03:46.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.932+0000 7f3b29122740 1 -- 172.21.15.103:0/1163704030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914cb40 msgr2=0x55a1f914e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:46.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.932+0000 7f3b29122740 1 --2- 172.21.15.103:0/1163704030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914cb40 0x55a1f914e120 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f3b1009bd70 tx=0x7f3b10004a20 comp rx=0 tx=0).stop 2024-09-19T07:03:46.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.932+0000 7f3b29122740 1 -- 172.21.15.103:0/1163704030 shutdown_connections 2024-09-19T07:03:46.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.932+0000 7f3b29122740 1 --2- 172.21.15.103:0/1163704030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f9156370 0x55a1f914e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.932+0000 7f3b29122740 1 --2- 172.21.15.103:0/1163704030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f914cb40 0x55a1f914e120 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.932+0000 7f3b29122740 1 --2- 172.21.15.103:0/1163704030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f914b5d0 0x55a1f9162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.932+0000 7f3b29122740 1 -- 172.21.15.103:0/1163704030 >> 172.21.15.103:0/1163704030 conn(0x55a1f9114310 msgr2=0x55a1f9152070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:46.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.932+0000 7f3b29122740 1 -- 172.21.15.103:0/1163704030 shutdown_connections 2024-09-19T07:03:46.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.932+0000 7f3b29122740 1 -- 172.21.15.103:0/1163704030 wait complete. 2024-09-19T07:03:46.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.933+0000 7f3b29122740 1 -- start start 2024-09-19T07:03:46.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.933+0000 7f3b29122740 1 Processor -- start 2024-09-19T07:03:46.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.933+0000 7f3b29122740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f914b5d0 0x55a1f915eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.933+0000 7f3b29122740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f914cb40 0x55a1f9159e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.933+0000 7f3b29122740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f9156370 0x55a1f915a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.933+0000 7f3b29122740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b201aedb0 con 0x55a1f9156370 2024-09-19T07:03:46.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.933+0000 7f3b29122740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b201ae6e0 con 0x55a1f914cb40 2024-09-19T07:03:46.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.933+0000 7f3b29122740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b201ab940 con 0x55a1f914b5d0 2024-09-19T07:03:46.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.933+0000 7f3b29122740 10 client.0 _ll_register_callbacks cb 0x7f3b1c0076d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:46.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.933+0000 7f3b252e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f9156370 0x55a1f915a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.933+0000 7f3b252e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f9156370 0x55a1f915a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34990/0 (socket says 172.21.15.103:34990) 2024-09-19T07:03:46.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.933+0000 7f3b252e6640 1 -- 172.21.15.103:0/931155259 learned_addr learned my addr 172.21.15.103:0/931155259 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:46.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.933+0000 7f3b24ae5640 1 --2- 172.21.15.103:0/931155259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f914b5d0 0x55a1f915eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.933+0000 7f3b17fff640 1 --2- 172.21.15.103:0/931155259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f914cb40 0x55a1f9159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.934+0000 7f3b252e6640 1 -- 172.21.15.103:0/931155259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f914b5d0 msgr2=0x55a1f915eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:46.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.934+0000 7f3b252e6640 1 --2- 172.21.15.103:0/931155259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f914b5d0 0x55a1f915eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.934+0000 7f3b252e6640 1 -- 172.21.15.103:0/931155259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f914cb40 msgr2=0x55a1f9159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:46.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.934+0000 7f3b252e6640 1 --2- 172.21.15.103:0/931155259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f914cb40 0x55a1f9159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.934+0000 7f3b252e6640 1 -- 172.21.15.103:0/931155259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1f915aba0 con 0x55a1f9156370 2024-09-19T07:03:46.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.934+0000 7f3b252e6640 1 --2- 172.21.15.103:0/931155259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f9156370 0x55a1f915a3a0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009ca0 tx=0x7f3b18046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9829980c44d6c0bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:46.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.934+0000 7f3afffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:46.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.934+0000 7f3afffff640 1 -- 172.21.15.103:0/931155259 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1804f070 con 0x55a1f9156370 2024-09-19T07:03:46.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.934+0000 7f3afffff640 1 -- 172.21.15.103:0/931155259 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f3b180045d0 con 0x55a1f9156370 2024-09-19T07:03:46.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.934+0000 7f3b29122740 1 -- 172.21.15.103:0/931155259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b100a4a50 con 0x55a1f9156370 2024-09-19T07:03:46.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.934+0000 7f3afffff640 1 -- 172.21.15.103:0/931155259 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b180047d0 con 0x55a1f9156370 2024-09-19T07:03:46.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.934+0000 7f3b29122740 10 client.20394 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:46.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.934+0000 7f3b29122740 1 -- 172.21.15.103:0/931155259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a1f91d07e0 con 0x55a1f9156370 2024-09-19T07:03:46.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.934+0000 7f3afffff640 1 -- 172.21.15.103:0/931155259 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b100a4a50 con 0x55a1f9156370 2024-09-19T07:03:46.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.934+0000 7f3afffff640 1 -- 172.21.15.103:0/931155259 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3b1802f880 con 0x55a1f9156370 2024-09-19T07:03:46.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.935+0000 7f3b29122740 10 client.20394 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:46.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.935+0000 7f3b29122740 10 client.20394 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:03:46.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.935+0000 7f3b29122740 20 client.20394 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:46.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.935+0000 7f3b29122740 1 --2- 172.21.15.103:0/931155259 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55a1f91d0f60 0x55a1f91d3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:46.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.935+0000 7f3b29122740 4 client.20394 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:03:46.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.935+0000 7f3b29122740 1 -- 172.21.15.103:0/931155259 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1f90b8d70 con 0x55a1f91d0f60 2024-09-19T07:03:46.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.935+0000 7f3b24ae5640 1 --2- 172.21.15.103:0/931155259 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55a1f91d0f60 0x55a1f91d3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:46.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.935+0000 7f3afffff640 10 client.20394 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:03:46.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.935+0000 7f3b24ae5640 1 --2- 172.21.15.103:0/931155259 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55a1f91d0f60 0x55a1f91d3380 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=683e9c2d2c05df2e server_cookie=400896c3e7a2e74a in_seq=0 out_seq=0 2024-09-19T07:03:46.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.935+0000 7f3afffff640 1 -- 172.21.15.103:0/931155259 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1f90b8d70 con 0x55a1f91d0f60 2024-09-19T07:03:46.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.935+0000 7f3afffff640 10 client.20394 handle_command_reply: tid=0 2024-09-19T07:03:46.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.935+0000 7f3b29122740 1 client.20394 shutdown 2024-09-19T07:03:46.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.936+0000 7f3b29122740 1 -- 172.21.15.103:0/931155259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f9156370 msgr2=0x55a1f915a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:46.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.936+0000 7f3b29122740 1 --2- 172.21.15.103:0/931155259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f9156370 0x55a1f915a3a0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009ca0 tx=0x7f3b18046000 comp rx=0 tx=0).stop 2024-09-19T07:03:46.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.936+0000 7f3b29122740 1 -- 172.21.15.103:0/931155259 shutdown_connections 2024-09-19T07:03:46.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.936+0000 7f3b29122740 1 --2- 172.21.15.103:0/931155259 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55a1f91d0f60 0x55a1f91d3380 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.936+0000 7f3b29122740 1 --2- 172.21.15.103:0/931155259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1f9156370 0x55a1f915a3a0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.936+0000 7f3b29122740 1 --2- 172.21.15.103:0/931155259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1f914cb40 0x55a1f9159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.936+0000 7f3afffff640 0 client.20394 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:03:46.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.936+0000 7f3b29122740 1 --2- 172.21.15.103:0/931155259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a1f914b5d0 0x55a1f915eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.936+0000 7f3b29122740 1 -- 172.21.15.103:0/931155259 >> 172.21.15.103:0/931155259 conn(0x55a1f9114310 msgr2=0x55a1f91464e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:46.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.936+0000 7f3b29122740 1 -- 172.21.15.103:0/931155259 shutdown_connections 2024-09-19T07:03:46.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.936+0000 7f3b29122740 1 -- 172.21.15.103:0/931155259 wait complete. 2024-09-19T07:03:46.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.936+0000 7f3b29122740 20 client.20394 trim_cache size 0 max 16384 2024-09-19T07:03:46.950 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:03:46.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.937+0000 7f3b26d49640 1 -- 172.21.15.103:0/437852223 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3b080a33b0 msgr2=0x7f3b080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:46.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.937+0000 7f3b26d49640 1 --2- 172.21.15.103:0/437852223 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3b080a33b0 0x7f3b080a5870 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3b180098d0 tx=0x7f3b18031db0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.937+0000 7f3b26d49640 1 -- 172.21.15.103:0/437852223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b20101bc0 msgr2=0x7f3b20198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:46.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.937+0000 7f3b26d49640 1 --2- 172.21.15.103:0/437852223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b20101bc0 0x7f3b20198a70 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000c970 tx=0x7f3b1000ce30 comp rx=0 tx=0).stop 2024-09-19T07:03:46.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.937+0000 7f3b26d49640 1 -- 172.21.15.103:0/437852223 shutdown_connections 2024-09-19T07:03:46.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.937+0000 7f3b26d49640 1 --2- 172.21.15.103:0/437852223 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3b080a33b0 0x7f3b080a5870 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.937+0000 7f3b26d49640 1 --2- 172.21.15.103:0/437852223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b20105a90 0x7f3b2019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.937+0000 7f3b26d49640 1 --2- 172.21.15.103:0/437852223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b20101bc0 0x7f3b20198a70 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.937+0000 7f3b26d49640 1 --2- 172.21.15.103:0/437852223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b20101240 0x7f3b20198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:46.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.937+0000 7f3b26d49640 1 -- 172.21.15.103:0/437852223 >> 172.21.15.103:0/437852223 conn(0x7f3b20077210 msgr2=0x7f3b200fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:46.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.937+0000 7f3b26d49640 1 -- 172.21.15.103:0/437852223 shutdown_connections 2024-09-19T07:03:46.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:46.937+0000 7f3b26d49640 1 -- 172.21.15.103:0/437852223 wait complete. 2024-09-19T07:03:47.586 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:03:47.833 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:48.287 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:48 smithi103 ceph-mon[31158]: pgmap v455: 97 pgs: 97 active+clean; 5.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 63 KiB/s wr, 23 op/s 2024-09-19T07:03:48.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:48 smithi174 ceph-mon[35637]: pgmap v455: 97 pgs: 97 active+clean; 5.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 63 KiB/s wr, 23 op/s 2024-09-19T07:03:48.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.367+0000 7ff5fd4e5640 1 -- 172.21.15.103:0/585610651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5f8106a80 msgr2=0x7ff5f8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:48.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.367+0000 7ff5fd4e5640 1 --2- 172.21.15.103:0/585610651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5f8106a80 0x7ff5f8106e60 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009930 tx=0x7ff5e802a420 comp rx=0 tx=0).stop 2024-09-19T07:03:48.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.367+0000 7ff5fd4e5640 1 -- 172.21.15.103:0/585610651 shutdown_connections 2024-09-19T07:03:48.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.367+0000 7ff5fd4e5640 1 --2- 172.21.15.103:0/585610651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5f8101420 0x7ff5f810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.367+0000 7ff5fd4e5640 1 --2- 172.21.15.103:0/585610651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5f8100a80 0x7ff5f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.367+0000 7ff5fd4e5640 1 --2- 172.21.15.103:0/585610651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5f8106a80 0x7ff5f8106e60 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.367+0000 7ff5fd4e5640 1 -- 172.21.15.103:0/585610651 >> 172.21.15.103:0/585610651 conn(0x7ff5f80fc7a0 msgr2=0x7ff5f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:48.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.367+0000 7ff5fd4e5640 1 -- 172.21.15.103:0/585610651 shutdown_connections 2024-09-19T07:03:48.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.367+0000 7ff5fd4e5640 1 -- 172.21.15.103:0/585610651 wait complete. 2024-09-19T07:03:48.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5fd4e5640 1 Processor -- start 2024-09-19T07:03:48.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5fd4e5640 1 -- start start 2024-09-19T07:03:48.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5fd4e5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5f8100a80 0x7ff5f81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5fd4e5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5f8101420 0x7ff5f81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5fd4e5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5f8106a80 0x7ff5f819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5fd4e5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f8111fb0 con 0x7ff5f8100a80 2024-09-19T07:03:48.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5fd4e5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f8111e30 con 0x7ff5f8106a80 2024-09-19T07:03:48.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5fd4e5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f8112130 con 0x7ff5f8101420 2024-09-19T07:03:48.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5f6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5f8100a80 0x7ff5f81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5f6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5f8100a80 0x7ff5f81984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52584/0 (socket says 172.21.15.103:52584) 2024-09-19T07:03:48.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5f6ffd640 1 -- 172.21.15.103:0/3880135678 learned_addr learned my addr 172.21.15.103:0/3880135678 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:48.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5f77fe640 1 --2- 172.21.15.103:0/3880135678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5f8106a80 0x7ff5f819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5effff640 1 --2- 172.21.15.103:0/3880135678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5f8101420 0x7ff5f81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5f6ffd640 1 -- 172.21.15.103:0/3880135678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5f8101420 msgr2=0x7ff5f81989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:48.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5f6ffd640 1 --2- 172.21.15.103:0/3880135678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5f8101420 0x7ff5f81989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5f6ffd640 1 -- 172.21.15.103:0/3880135678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5f8106a80 msgr2=0x7ff5f819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:48.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5f6ffd640 1 --2- 172.21.15.103:0/3880135678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5f8106a80 0x7ff5f819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.368+0000 7ff5f6ffd640 1 -- 172.21.15.103:0/3880135678 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f819d460 con 0x7ff5f8100a80 2024-09-19T07:03:48.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.369+0000 7ff5f6ffd640 1 --2- 172.21.15.103:0/3880135678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5f8100a80 0x7ff5f81984b0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009230 tx=0x7ff5e8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:48.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.369+0000 7ff5f4ff9640 1 -- 172.21.15.103:0/3880135678 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e803f030 con 0x7ff5f8100a80 2024-09-19T07:03:48.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.369+0000 7ff5fd4e5640 1 -- 172.21.15.103:0/3880135678 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f819d6f0 con 0x7ff5f8100a80 2024-09-19T07:03:48.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.369+0000 7ff5f4ff9640 1 -- 172.21.15.103:0/3880135678 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e8003ad0 con 0x7ff5f8100a80 2024-09-19T07:03:48.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.369+0000 7ff5f4ff9640 1 -- 172.21.15.103:0/3880135678 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8035070 con 0x7ff5f8100a80 2024-09-19T07:03:48.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.369+0000 7ff5fd4e5640 1 -- 172.21.15.103:0/3880135678 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f81a5030 con 0x7ff5f8100a80 2024-09-19T07:03:48.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.371+0000 7ff5f4ff9640 1 -- 172.21.15.103:0/3880135678 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff5e8042030 con 0x7ff5f8100a80 2024-09-19T07:03:48.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.371+0000 7ff5ff8be740 1 Processor -- start 2024-09-19T07:03:48.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.371+0000 7ff5ff8be740 1 -- start start 2024-09-19T07:03:48.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5ff8be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2bb40 0x564eb1f2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5ff8be740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f2a5d0 0x564eb1f2aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5ff8be740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f35370 0x564eb1f37760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5ff8be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564eb1e902f0 con 0x564eb1f2bb40 2024-09-19T07:03:48.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5ff8be740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564eb1e37cf0 con 0x564eb1f2a5d0 2024-09-19T07:03:48.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5ff8be740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564eb1e8c2b0 con 0x564eb1f35370 2024-09-19T07:03:48.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5f77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2bb40 0x564eb1f2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5f77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2bb40 0x564eb1f2a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52592/0 (socket says 172.21.15.103:52592) 2024-09-19T07:03:48.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5f77fe640 1 -- 172.21.15.103:0/1110377221 learned_addr learned my addr 172.21.15.103:0/1110377221 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:48.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5effff640 1 --2- 172.21.15.103:0/1110377221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f35370 0x564eb1f37760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5f4ff9640 1 --2- 172.21.15.103:0/3880135678 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff5e40a33b0 0x7ff5e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5f6ffd640 1 --2- 172.21.15.103:0/1110377221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f2a5d0 0x564eb1f2aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5f77fe640 1 -- 172.21.15.103:0/1110377221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f35370 msgr2=0x564eb1f37760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:48.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5f77fe640 1 --2- 172.21.15.103:0/1110377221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f35370 0x564eb1f37760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5f77fe640 1 -- 172.21.15.103:0/1110377221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f2a5d0 msgr2=0x564eb1f2aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:48.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5f77fe640 1 --2- 172.21.15.103:0/1110377221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f2a5d0 0x564eb1f2aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5f77fe640 1 -- 172.21.15.103:0/1110377221 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eb1e1c3a0 con 0x564eb1f2bb40 2024-09-19T07:03:48.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5f77fe640 1 --2- 172.21.15.103:0/3880135678 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff5e40a33b0 0x7ff5e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5f4ff9640 1 -- 172.21.15.103:0/3880135678 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7ff5e80f0250 con 0x7ff5f8100a80 2024-09-19T07:03:48.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.372+0000 7ff5f77fe640 1 --2- 172.21.15.103:0/1110377221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2bb40 0x564eb1f2a090 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000a5d0 tx=0x7ff5f000aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06505088ec97fac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:48.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.373+0000 7ff5ed7fa640 1 -- 172.21.15.103:0/1110377221 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f001a040 con 0x564eb1f2bb40 2024-09-19T07:03:48.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.373+0000 7ff5ed7fa640 1 -- 172.21.15.103:0/1110377221 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f0021ba0 con 0x564eb1f2bb40 2024-09-19T07:03:48.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.373+0000 7ff5ed7fa640 1 -- 172.21.15.103:0/1110377221 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f001f350 con 0x564eb1f2bb40 2024-09-19T07:03:48.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.373+0000 7ff5f77fe640 1 --2- 172.21.15.103:0/3880135678 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff5e40a33b0 0x7ff5e40a5870 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0020000 tx=0x7ff5f0021000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:48.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.373+0000 7ff5ff8be740 1 -- 172.21.15.103:0/1110377221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2bb40 msgr2=0x564eb1f2a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:48.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.373+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/1110377221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2bb40 0x564eb1f2a090 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000a5d0 tx=0x7ff5f000aa90 comp rx=0 tx=0).stop 2024-09-19T07:03:48.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.373+0000 7ff5ff8be740 1 -- 172.21.15.103:0/1110377221 shutdown_connections 2024-09-19T07:03:48.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.373+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/1110377221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f35370 0x564eb1f37760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.373+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/1110377221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f2a5d0 0x564eb1f2aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.373+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/1110377221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2bb40 0x564eb1f2a090 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.373+0000 7ff5ff8be740 1 -- 172.21.15.103:0/1110377221 >> 172.21.15.103:0/1110377221 conn(0x564eb1ef3310 msgr2=0x564eb1f26c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:48.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.374+0000 7ff5ff8be740 1 -- 172.21.15.103:0/1110377221 shutdown_connections 2024-09-19T07:03:48.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.374+0000 7ff5ff8be740 1 -- 172.21.15.103:0/1110377221 wait complete. 2024-09-19T07:03:48.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.374+0000 7ff5ff8be740 1 -- start start 2024-09-19T07:03:48.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.374+0000 7ff5ff8be740 1 Processor -- start 2024-09-19T07:03:48.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.374+0000 7ff5ff8be740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f2a5d0 0x564eb1f34d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.374+0000 7ff5ff8be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2bb40 0x564eb1f31fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.374+0000 7ff5ff8be740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f35370 0x564eb1f325d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.374+0000 7ff5ff8be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564eb1f38070 con 0x564eb1f2bb40 2024-09-19T07:03:48.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.374+0000 7ff5ff8be740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564eb1e4b060 con 0x564eb1f2a5d0 2024-09-19T07:03:48.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.374+0000 7ff5ff8be740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564eb1f381f0 con 0x564eb1f35370 2024-09-19T07:03:48.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.374+0000 7ff5ff8be740 10 client.0 _ll_register_callbacks cb 0x564eb1b9e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:48.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.374+0000 7ff5effff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2bb40 0x564eb1f31fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.374+0000 7ff5effff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2bb40 0x564eb1f31fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52602/0 (socket says 172.21.15.103:52602) 2024-09-19T07:03:48.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.374+0000 7ff5effff640 1 -- 172.21.15.103:0/3558664053 learned_addr learned my addr 172.21.15.103:0/3558664053 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:48.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.375+0000 7ff5f77fe640 1 --2- 172.21.15.103:0/3558664053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f35370 0x564eb1f325d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.375+0000 7ff5f6ffd640 1 --2- 172.21.15.103:0/3558664053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f2a5d0 0x564eb1f34d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.375+0000 7ff5effff640 1 -- 172.21.15.103:0/3558664053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f35370 msgr2=0x564eb1f325d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:48.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.375+0000 7ff5effff640 1 --2- 172.21.15.103:0/3558664053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f35370 0x564eb1f325d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.375+0000 7ff5effff640 1 -- 172.21.15.103:0/3558664053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f2a5d0 msgr2=0x564eb1f34d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:48.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.375+0000 7ff5effff640 1 --2- 172.21.15.103:0/3558664053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f2a5d0 0x564eb1f34d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.375+0000 7ff5effff640 1 -- 172.21.15.103:0/3558664053 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eb1c013c0 con 0x564eb1f2bb40 2024-09-19T07:03:48.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.375+0000 7ff5effff640 1 --2- 172.21.15.103:0/3558664053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2bb40 0x564eb1f31fe0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000beb0 tx=0x7ff5e0012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1014758d207510 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:48.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.375+0000 7ff5de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:48.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.375+0000 7ff5de7fc640 1 -- 172.21.15.103:0/3558664053 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0004230 con 0x564eb1f2bb40 2024-09-19T07:03:48.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.375+0000 7ff5de7fc640 1 -- 172.21.15.103:0/3558664053 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7ff5e00043d0 con 0x564eb1f2bb40 2024-09-19T07:03:48.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.375+0000 7ff5ff8be740 1 -- 172.21.15.103:0/3558664053 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5f0021ba0 con 0x564eb1f2bb40 2024-09-19T07:03:48.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.375+0000 7ff5de7fc640 1 -- 172.21.15.103:0/3558664053 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5e00106a0 con 0x564eb1f2bb40 2024-09-19T07:03:48.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.375+0000 7ff5ff8be740 10 client.20406 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:48.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.375+0000 7ff5ff8be740 1 -- 172.21.15.103:0/3558664053 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564eb1f43b30 con 0x564eb1f2bb40 2024-09-19T07:03:48.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.375+0000 7ff5de7fc640 1 -- 172.21.15.103:0/3558664053 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f0021ba0 con 0x564eb1f2bb40 2024-09-19T07:03:48.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.376+0000 7ff5de7fc640 1 -- 172.21.15.103:0/3558664053 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff5e0019070 con 0x564eb1f2bb40 2024-09-19T07:03:48.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.376+0000 7ff5ff8be740 10 client.20406 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:48.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.376+0000 7ff5ff8be740 10 client.20406 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:03:48.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.376+0000 7ff5ff8be740 20 client.20406 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:48.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.376+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/3558664053 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x564eb1f45090 0x564eb1f474d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.376+0000 7ff5ff8be740 4 client.20406 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:48.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.376+0000 7ff5ff8be740 1 -- 172.21.15.103:0/3558664053 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564eb1e9d860 con 0x564eb1f45090 2024-09-19T07:03:48.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.376+0000 7ff5f6ffd640 1 --2- 172.21.15.103:0/3558664053 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x564eb1f45090 0x564eb1f474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.376+0000 7ff5de7fc640 10 client.20406 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:03:48.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.376+0000 7ff5f6ffd640 1 --2- 172.21.15.103:0/3558664053 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x564eb1f45090 0x564eb1f474d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ed077f875a70f8f0 server_cookie=7b85988811ad7642 in_seq=0 out_seq=0 2024-09-19T07:03:48.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.377+0000 7ff5de7fc640 1 -- 172.21.15.103:0/3558664053 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564eb1e9d860 con 0x564eb1f45090 2024-09-19T07:03:48.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.377+0000 7ff5de7fc640 10 client.20406 handle_command_reply: tid=0 2024-09-19T07:03:48.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.377+0000 7ff5ff8be740 1 client.20406 shutdown 2024-09-19T07:03:48.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.378+0000 7ff5ff8be740 1 -- 172.21.15.103:0/3558664053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2bb40 msgr2=0x564eb1f31fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:48.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.378+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/3558664053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2bb40 0x564eb1f31fe0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000beb0 tx=0x7ff5e0012820 comp rx=0 tx=0).stop 2024-09-19T07:03:48.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.378+0000 7ff5ff8be740 1 -- 172.21.15.103:0/3558664053 shutdown_connections 2024-09-19T07:03:48.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.378+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/3558664053 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x564eb1f45090 0x564eb1f474d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.378+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/3558664053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f35370 0x564eb1f325d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.378+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/3558664053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2bb40 0x564eb1f31fe0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.378+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/3558664053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f2a5d0 0x564eb1f34d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.378+0000 7ff5de7fc640 0 client.20406 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:03:48.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.378+0000 7ff5ff8be740 1 -- 172.21.15.103:0/3558664053 >> 172.21.15.103:0/3558664053 conn(0x564eb1ef3310 msgr2=0x564eb1f2c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:48.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.378+0000 7ff5ff8be740 1 -- 172.21.15.103:0/3558664053 shutdown_connections 2024-09-19T07:03:48.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.378+0000 7ff5ff8be740 1 -- 172.21.15.103:0/3558664053 wait complete. 2024-09-19T07:03:48.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.378+0000 7ff5ff8be740 20 client.20406 trim_cache size 0 max 16384 2024-09-19T07:03:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.391+0000 7ff5ff8be740 1 Processor -- start 2024-09-19T07:03:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.391+0000 7ff5ff8be740 1 -- start start 2024-09-19T07:03:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.391+0000 7ff5ff8be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2a5d0 0x564eb1f318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.391+0000 7ff5ff8be740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f2bb40 0x564eb1f33f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.391+0000 7ff5ff8be740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f35370 0x564eb1f2d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.391+0000 7ff5ff8be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f81ad970 con 0x564eb1f2a5d0 2024-09-19T07:03:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.391+0000 7ff5ff8be740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f81af4b0 con 0x564eb1f35370 2024-09-19T07:03:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.391+0000 7ff5ff8be740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f81ae040 con 0x564eb1f2bb40 2024-09-19T07:03:48.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.391+0000 7ff5f6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2a5d0 0x564eb1f318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.391+0000 7ff5f6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2a5d0 0x564eb1f318a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52618/0 (socket says 172.21.15.103:52618) 2024-09-19T07:03:48.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.391+0000 7ff5f6ffd640 1 -- 172.21.15.103:0/2756690259 learned_addr learned my addr 172.21.15.103:0/2756690259 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:48.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.392+0000 7ff5f77fe640 1 --2- 172.21.15.103:0/2756690259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f35370 0x564eb1f2d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.392+0000 7ff5effff640 1 --2- 172.21.15.103:0/2756690259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f2bb40 0x564eb1f33f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.392+0000 7ff5f6ffd640 1 -- 172.21.15.103:0/2756690259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f2bb40 msgr2=0x564eb1f33f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:48.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.392+0000 7ff5f6ffd640 1 --2- 172.21.15.103:0/2756690259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f2bb40 0x564eb1f33f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.392+0000 7ff5f6ffd640 1 -- 172.21.15.103:0/2756690259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f35370 msgr2=0x564eb1f2d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:48.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.392+0000 7ff5f6ffd640 1 --2- 172.21.15.103:0/2756690259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f35370 0x564eb1f2d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.392+0000 7ff5f6ffd640 1 -- 172.21.15.103:0/2756690259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eb1f30f90 con 0x564eb1f2a5d0 2024-09-19T07:03:48.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.392+0000 7ff5effff640 1 --2- 172.21.15.103:0/2756690259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f2bb40 0x564eb1f33f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:48.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.392+0000 7ff5f6ffd640 1 --2- 172.21.15.103:0/2756690259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2a5d0 0x564eb1f318a0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80bc210 tx=0x7ff5e8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dae0d89453ebd59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:48.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.392+0000 7ff5ecff9640 1 -- 172.21.15.103:0/2756690259 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8049040 con 0x564eb1f2a5d0 2024-09-19T07:03:48.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.392+0000 7ff5ecff9640 1 -- 172.21.15.103:0/2756690259 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e80c08c0 con 0x564eb1f2a5d0 2024-09-19T07:03:48.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.392+0000 7ff5ecff9640 1 -- 172.21.15.103:0/2756690259 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80b8b80 con 0x564eb1f2a5d0 2024-09-19T07:03:48.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.393+0000 7ff5ff8be740 1 -- 172.21.15.103:0/2756690259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2a5d0 msgr2=0x564eb1f318a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:48.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.393+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/2756690259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2a5d0 0x564eb1f318a0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80bc210 tx=0x7ff5e8009260 comp rx=0 tx=0).stop 2024-09-19T07:03:48.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.393+0000 7ff5ff8be740 1 -- 172.21.15.103:0/2756690259 shutdown_connections 2024-09-19T07:03:48.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.393+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/2756690259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f35370 0x564eb1f2d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.393+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/2756690259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f2bb40 0x564eb1f33f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.393+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/2756690259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2a5d0 0x564eb1f318a0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.393+0000 7ff5ff8be740 1 -- 172.21.15.103:0/2756690259 >> 172.21.15.103:0/2756690259 conn(0x564eb1ef3310 msgr2=0x564eb1f33840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:48.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.393+0000 7ff5ff8be740 1 -- 172.21.15.103:0/2756690259 shutdown_connections 2024-09-19T07:03:48.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.393+0000 7ff5ff8be740 1 -- 172.21.15.103:0/2756690259 wait complete. 2024-09-19T07:03:48.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.393+0000 7ff5ff8be740 1 -- start start 2024-09-19T07:03:48.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5ff8be740 1 Processor -- start 2024-09-19T07:03:48.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5ff8be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2a5d0 0x564eb1f393f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5ff8be740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f2bb40 0x564eb1f39930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5ff8be740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f35370 0x564eb1f39e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5ff8be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f81aede0 con 0x564eb1f2a5d0 2024-09-19T07:03:48.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5ff8be740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f81ae710 con 0x564eb1f35370 2024-09-19T07:03:48.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5ff8be740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f81ab970 con 0x564eb1f2bb40 2024-09-19T07:03:48.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5ff8be740 10 client.0 _ll_register_callbacks cb 0x7ff5e000b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:48.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5f6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2a5d0 0x564eb1f393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5f6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2a5d0 0x564eb1f393f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52620/0 (socket says 172.21.15.103:52620) 2024-09-19T07:03:48.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5f6ffd640 1 -- 172.21.15.103:0/3935036060 learned_addr learned my addr 172.21.15.103:0/3935036060 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:48.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5effff640 1 --2- 172.21.15.103:0/3935036060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f2bb40 0x564eb1f39930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5f77fe640 1 --2- 172.21.15.103:0/3935036060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f35370 0x564eb1f39e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5f6ffd640 1 -- 172.21.15.103:0/3935036060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f2bb40 msgr2=0x564eb1f39930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:48.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5f6ffd640 1 --2- 172.21.15.103:0/3935036060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f2bb40 0x564eb1f39930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5f6ffd640 1 -- 172.21.15.103:0/3935036060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f35370 msgr2=0x564eb1f39e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:48.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5f6ffd640 1 --2- 172.21.15.103:0/3935036060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f35370 0x564eb1f39e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5f6ffd640 1 -- 172.21.15.103:0/3935036060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eb1faef70 con 0x564eb1f2a5d0 2024-09-19T07:03:48.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5f77fe640 1 --2- 172.21.15.103:0/3935036060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f35370 0x564eb1f39e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:48.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5f6ffd640 1 --2- 172.21.15.103:0/3935036060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2a5d0 0x564eb1f393f0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80bc4a0 tx=0x7ff5e802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66a0af1fe43c841 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:48.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.394+0000 7ff5de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:48.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.395+0000 7ff5de7fc640 1 -- 172.21.15.103:0/3935036060 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80d4020 con 0x564eb1f2a5d0 2024-09-19T07:03:48.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.395+0000 7ff5de7fc640 1 -- 172.21.15.103:0/3935036060 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7ff5e80edc00 con 0x564eb1f2a5d0 2024-09-19T07:03:48.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.395+0000 7ff5ff8be740 1 -- 172.21.15.103:0/3935036060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e80c08c0 con 0x564eb1f2a5d0 2024-09-19T07:03:48.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.395+0000 7ff5de7fc640 1 -- 172.21.15.103:0/3935036060 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5e80a5050 con 0x564eb1f2a5d0 2024-09-19T07:03:48.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.395+0000 7ff5ff8be740 10 client.20412 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:48.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.395+0000 7ff5ff8be740 1 -- 172.21.15.103:0/3935036060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564eb1fb01c0 con 0x564eb1f2a5d0 2024-09-19T07:03:48.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.395+0000 7ff5de7fc640 1 -- 172.21.15.103:0/3935036060 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e80c08c0 con 0x564eb1f2a5d0 2024-09-19T07:03:48.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.395+0000 7ff5de7fc640 1 -- 172.21.15.103:0/3935036060 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff5e8031bd0 con 0x564eb1f2a5d0 2024-09-19T07:03:48.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.395+0000 7ff5ff8be740 10 client.20412 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:48.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.395+0000 7ff5ff8be740 10 client.20412 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:03:48.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.395+0000 7ff5ff8be740 20 client.20412 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:48.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.395+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/3935036060 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x564eb1fb08d0 0x564eb1fb2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:48.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.395+0000 7ff5ff8be740 4 client.20412 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:03:48.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.395+0000 7ff5ff8be740 1 -- 172.21.15.103:0/3935036060 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564eb1e97d70 con 0x564eb1fb08d0 2024-09-19T07:03:48.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.396+0000 7ff5effff640 1 --2- 172.21.15.103:0/3935036060 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x564eb1fb08d0 0x564eb1fb2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:48.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.396+0000 7ff5de7fc640 10 client.20412 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:03:48.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.396+0000 7ff5effff640 1 --2- 172.21.15.103:0/3935036060 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x564eb1fb08d0 0x564eb1fb2cc0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b60c3848e0586807 server_cookie=f124d9bb901528f in_seq=0 out_seq=0 2024-09-19T07:03:48.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.396+0000 7ff5de7fc640 1 -- 172.21.15.103:0/3935036060 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564eb1e97d70 con 0x564eb1fb08d0 2024-09-19T07:03:48.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.396+0000 7ff5de7fc640 10 client.20412 handle_command_reply: tid=0 2024-09-19T07:03:48.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.396+0000 7ff5ff8be740 1 client.20412 shutdown 2024-09-19T07:03:48.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.396+0000 7ff5ff8be740 1 -- 172.21.15.103:0/3935036060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2a5d0 msgr2=0x564eb1f393f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:48.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.396+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/3935036060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2a5d0 0x564eb1f393f0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80bc4a0 tx=0x7ff5e802a420 comp rx=0 tx=0).stop 2024-09-19T07:03:48.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.396+0000 7ff5ff8be740 1 -- 172.21.15.103:0/3935036060 shutdown_connections 2024-09-19T07:03:48.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.396+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/3935036060 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x564eb1fb08d0 0x564eb1fb2cc0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.396+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/3935036060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eb1f35370 0x564eb1f39e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.396+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/3935036060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564eb1f2bb40 0x564eb1f39930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.396+0000 7ff5ff8be740 1 --2- 172.21.15.103:0/3935036060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564eb1f2a5d0 0x564eb1f393f0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.396+0000 7ff5de7fc640 0 client.20412 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:03:48.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.396+0000 7ff5ff8be740 1 -- 172.21.15.103:0/3935036060 >> 172.21.15.103:0/3935036060 conn(0x564eb1ef3310 msgr2=0x564eb1f34060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:48.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.397+0000 7ff5ff8be740 1 -- 172.21.15.103:0/3935036060 shutdown_connections 2024-09-19T07:03:48.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.397+0000 7ff5ff8be740 1 -- 172.21.15.103:0/3935036060 wait complete. 2024-09-19T07:03:48.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.397+0000 7ff5ff8be740 20 client.20412 trim_cache size 0 max 16384 2024-09-19T07:03:48.414 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:03:48.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.397+0000 7ff5fd4e5640 1 -- 172.21.15.103:0/3880135678 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff5e40a33b0 msgr2=0x7ff5e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:48.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.397+0000 7ff5fd4e5640 1 --2- 172.21.15.103:0/3880135678 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff5e40a33b0 0x7ff5e40a5870 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0020000 tx=0x7ff5f0021000 comp rx=0 tx=0).stop 2024-09-19T07:03:48.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.397+0000 7ff5fd4e5640 1 -- 172.21.15.103:0/3880135678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5f8100a80 msgr2=0x7ff5f81984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:48.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.397+0000 7ff5fd4e5640 1 --2- 172.21.15.103:0/3880135678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5f8100a80 0x7ff5f81984b0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009230 tx=0x7ff5e8004270 comp rx=0 tx=0).stop 2024-09-19T07:03:48.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.397+0000 7ff5fd4e5640 1 -- 172.21.15.103:0/3880135678 shutdown_connections 2024-09-19T07:03:48.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.397+0000 7ff5fd4e5640 1 --2- 172.21.15.103:0/3880135678 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff5e40a33b0 0x7ff5e40a5870 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.397+0000 7ff5fd4e5640 1 --2- 172.21.15.103:0/3880135678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5f8106a80 0x7ff5f819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.397+0000 7ff5fd4e5640 1 --2- 172.21.15.103:0/3880135678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5f8101420 0x7ff5f81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.397+0000 7ff5fd4e5640 1 --2- 172.21.15.103:0/3880135678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5f8100a80 0x7ff5f81984b0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:48.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.397+0000 7ff5fd4e5640 1 -- 172.21.15.103:0/3880135678 >> 172.21.15.103:0/3880135678 conn(0x7ff5f80fc7a0 msgr2=0x7ff5f80fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:48.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.397+0000 7ff5fd4e5640 1 -- 172.21.15.103:0/3880135678 shutdown_connections 2024-09-19T07:03:48.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:48.397+0000 7ff5fd4e5640 1 -- 172.21.15.103:0/3880135678 wait complete. 2024-09-19T07:03:48.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:48 smithi134 ceph-mon[35356]: pgmap v455: 97 pgs: 97 active+clean; 5.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 63 KiB/s wr, 23 op/s 2024-09-19T07:03:49.036 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:03:49.282 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.759+0000 7fbdb6ebf640 1 -- 172.21.15.103:0/1383346089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdb0100ca0 msgr2=0x7fbdb0106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.759+0000 7fbdb6ebf640 1 --2- 172.21.15.103:0/1383346089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdb0100ca0 0x7fbdb0106840 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009930 tx=0x7fbd9c02a420 comp rx=0 tx=0).stop 2024-09-19T07:03:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.759+0000 7fbdb6ebf640 1 -- 172.21.15.103:0/1383346089 shutdown_connections 2024-09-19T07:03:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.759+0000 7fbdb6ebf640 1 --2- 172.21.15.103:0/1383346089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdb0106f50 0x7fbdb010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.759+0000 7fbdb6ebf640 1 --2- 172.21.15.103:0/1383346089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdb0100ca0 0x7fbdb0106840 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.759+0000 7fbdb6ebf640 1 --2- 172.21.15.103:0/1383346089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdb01002f0 0x7fbdb01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.759+0000 7fbdb6ebf640 1 -- 172.21.15.103:0/1383346089 >> 172.21.15.103:0/1383346089 conn(0x7fbdb0077210 msgr2=0x7fbdb00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.759+0000 7fbdb6ebf640 1 -- 172.21.15.103:0/1383346089 shutdown_connections 2024-09-19T07:03:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.759+0000 7fbdb6ebf640 1 -- 172.21.15.103:0/1383346089 wait complete. 2024-09-19T07:03:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.760+0000 7fbdb6ebf640 1 Processor -- start 2024-09-19T07:03:49.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.760+0000 7fbdb6ebf640 1 -- start start 2024-09-19T07:03:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.760+0000 7fbdb6ebf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdb01002f0 0x7fbdb0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.760+0000 7fbdb6ebf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdb0100ca0 0x7fbdb0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.760+0000 7fbdb6ebf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdb0106f50 0x7fbdb019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.760+0000 7fbdb6ebf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb0111f60 con 0x7fbdb0100ca0 2024-09-19T07:03:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.760+0000 7fbdb6ebf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb0111de0 con 0x7fbdb01002f0 2024-09-19T07:03:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.760+0000 7fbdb6ebf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01120e0 con 0x7fbdb0106f50 2024-09-19T07:03:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.761+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdb0100ca0 0x7fbdb0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.761+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdb0100ca0 0x7fbdb0198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52634/0 (socket says 172.21.15.103:52634) 2024-09-19T07:03:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.761+0000 7fbdaffff640 1 -- 172.21.15.103:0/3344969042 learned_addr learned my addr 172.21.15.103:0/3344969042 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.761+0000 7fbdb545c640 1 --2- 172.21.15.103:0/3344969042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdb0106f50 0x7fbdb019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.761+0000 7fbdb4c5b640 1 --2- 172.21.15.103:0/3344969042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdb01002f0 0x7fbdb0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.761+0000 7fbdaffff640 1 -- 172.21.15.103:0/3344969042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdb0106f50 msgr2=0x7fbdb019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.761+0000 7fbdaffff640 1 --2- 172.21.15.103:0/3344969042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdb0106f50 0x7fbdb019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.761+0000 7fbdaffff640 1 -- 172.21.15.103:0/3344969042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdb01002f0 msgr2=0x7fbdb0198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.761+0000 7fbdaffff640 1 --2- 172.21.15.103:0/3344969042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdb01002f0 0x7fbdb0198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.761+0000 7fbdaffff640 1 -- 172.21.15.103:0/3344969042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb019d3f0 con 0x7fbdb0100ca0 2024-09-19T07:03:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.761+0000 7fbdaffff640 1 --2- 172.21.15.103:0/3344969042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdb0100ca0 0x7fbdb0198980 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c005940 tx=0x7fbd9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.761+0000 7fbdadffb640 1 -- 172.21.15.103:0/3344969042 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c03f030 con 0x7fbdb0100ca0 2024-09-19T07:03:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.761+0000 7fbdb6ebf640 1 -- 172.21.15.103:0/3344969042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb019d680 con 0x7fbdb0100ca0 2024-09-19T07:03:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.761+0000 7fbdadffb640 1 -- 172.21.15.103:0/3344969042 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd9c003ce0 con 0x7fbdb0100ca0 2024-09-19T07:03:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.761+0000 7fbdadffb640 1 -- 172.21.15.103:0/3344969042 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c035070 con 0x7fbdb0100ca0 2024-09-19T07:03:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.762+0000 7fbdb6ebf640 1 -- 172.21.15.103:0/3344969042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb01a4fc0 con 0x7fbdb0100ca0 2024-09-19T07:03:49.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.764+0000 7fbdadffb640 1 -- 172.21.15.103:0/3344969042 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fbd9c042030 con 0x7fbdb0100ca0 2024-09-19T07:03:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.764+0000 7fbdb9298740 1 Processor -- start 2024-09-19T07:03:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.764+0000 7fbdb9298740 1 -- start start 2024-09-19T07:03:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdb9298740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff874f50 0x561fff8734a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdb9298740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff8739e0 0x561fff873e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdb9298740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff87e780 0x561fff880b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdb9298740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fff7d52f0 con 0x561fff87e780 2024-09-19T07:03:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdb9298740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561fff77ccf0 con 0x561fff8739e0 2024-09-19T07:03:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdb9298740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561fff7d12b0 con 0x561fff874f50 2024-09-19T07:03:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff87e780 0x561fff880b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff87e780 0x561fff880b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52644/0 (socket says 172.21.15.103:52644) 2024-09-19T07:03:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdaffff640 1 -- 172.21.15.103:0/3499494745 learned_addr learned my addr 172.21.15.103:0/3499494745 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:49.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdb545c640 1 --2- 172.21.15.103:0/3499494745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff874f50 0x561fff8734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdadffb640 1 --2- 172.21.15.103:0/3344969042 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbda00a33b0 0x7fbda00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdb4c5b640 1 --2- 172.21.15.103:0/3499494745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff8739e0 0x561fff873e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdaffff640 1 -- 172.21.15.103:0/3499494745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff874f50 msgr2=0x561fff8734a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:49.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdaffff640 1 --2- 172.21.15.103:0/3499494745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff874f50 0x561fff8734a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdaffff640 1 -- 172.21.15.103:0/3499494745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff8739e0 msgr2=0x561fff873e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:49.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdaffff640 1 --2- 172.21.15.103:0/3499494745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff8739e0 0x561fff873e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdb545c640 1 --2- 172.21.15.103:0/3344969042 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbda00a33b0 0x7fbda00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdadffb640 1 -- 172.21.15.103:0/3344969042 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7fbd9c0f01f0 con 0x7fbdb0100ca0 2024-09-19T07:03:49.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdaffff640 1 -- 172.21.15.103:0/3499494745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fff7613a0 con 0x561fff87e780 2024-09-19T07:03:49.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdb4c5b640 1 --2- 172.21.15.103:0/3499494745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff8739e0 0x561fff873e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:49.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.765+0000 7fbdaffff640 1 --2- 172.21.15.103:0/3499494745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff87e780 0x561fff880b70 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0b8e80 tx=0x7fbd9c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d138770c64387 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:49.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.766+0000 7fbd9affd640 1 -- 172.21.15.103:0/3499494745 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c049030 con 0x561fff87e780 2024-09-19T07:03:49.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.766+0000 7fbd9affd640 1 -- 172.21.15.103:0/3499494745 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd9c03b460 con 0x561fff87e780 2024-09-19T07:03:49.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.766+0000 7fbdb545c640 1 --2- 172.21.15.103:0/3344969042 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbda00a33b0 0x7fbda00a5870 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbda8005fd0 tx=0x7fbda8004ab0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:49.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.766+0000 7fbd9affd640 1 -- 172.21.15.103:0/3499494745 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0bc4b0 con 0x561fff87e780 2024-09-19T07:03:49.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.766+0000 7fbdb9298740 1 -- 172.21.15.103:0/3499494745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff87e780 msgr2=0x561fff880b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:49.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.766+0000 7fbdb9298740 1 --2- 172.21.15.103:0/3499494745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff87e780 0x561fff880b70 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0b8e80 tx=0x7fbd9c02a420 comp rx=0 tx=0).stop 2024-09-19T07:03:49.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.766+0000 7fbdb9298740 1 -- 172.21.15.103:0/3499494745 shutdown_connections 2024-09-19T07:03:49.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.766+0000 7fbdb9298740 1 --2- 172.21.15.103:0/3499494745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff87e780 0x561fff880b70 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.766+0000 7fbdb9298740 1 --2- 172.21.15.103:0/3499494745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff8739e0 0x561fff873e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.766+0000 7fbdb9298740 1 --2- 172.21.15.103:0/3499494745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff874f50 0x561fff8734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.766+0000 7fbdb9298740 1 -- 172.21.15.103:0/3499494745 >> 172.21.15.103:0/3499494745 conn(0x561fff838310 msgr2=0x561fff870070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:49.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.766+0000 7fbdb9298740 1 -- 172.21.15.103:0/3499494745 shutdown_connections 2024-09-19T07:03:49.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.766+0000 7fbdb9298740 1 -- 172.21.15.103:0/3499494745 wait complete. 2024-09-19T07:03:49.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.767+0000 7fbdb9298740 1 -- start start 2024-09-19T07:03:49.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.767+0000 7fbdb9298740 1 Processor -- start 2024-09-19T07:03:49.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.767+0000 7fbdb9298740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff8739e0 0x561fff87e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.767+0000 7fbdb9298740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff874f50 0x561fff87b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.767+0000 7fbdb9298740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff87e780 0x561fff87b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.767+0000 7fbdb9298740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fff881240 con 0x561fff874f50 2024-09-19T07:03:49.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.767+0000 7fbdb9298740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561fff790060 con 0x561fff8739e0 2024-09-19T07:03:49.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.767+0000 7fbdb9298740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561fff8813c0 con 0x561fff87e780 2024-09-19T07:03:49.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.767+0000 7fbdb9298740 10 client.0 _ll_register_callbacks cb 0x561fff4e38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:49.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.767+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff874f50 0x561fff87b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.767+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff874f50 0x561fff87b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52658/0 (socket says 172.21.15.103:52658) 2024-09-19T07:03:49.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.767+0000 7fbdaffff640 1 -- 172.21.15.103:0/1582858945 learned_addr learned my addr 172.21.15.103:0/1582858945 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:49.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbdb545c640 1 --2- 172.21.15.103:0/1582858945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff87e780 0x561fff87b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbdb4c5b640 1 --2- 172.21.15.103:0/1582858945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff8739e0 0x561fff87e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbdaffff640 1 -- 172.21.15.103:0/1582858945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff87e780 msgr2=0x561fff87b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:49.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbdaffff640 1 --2- 172.21.15.103:0/1582858945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff87e780 0x561fff87b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbdaffff640 1 -- 172.21.15.103:0/1582858945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff8739e0 msgr2=0x561fff87e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:49.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbdaffff640 1 --2- 172.21.15.103:0/1582858945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff8739e0 0x561fff87e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbdaffff640 1 -- 172.21.15.103:0/1582858945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fff5463c0 con 0x561fff874f50 2024-09-19T07:03:49.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbdb545c640 1 --2- 172.21.15.103:0/1582858945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff87e780 0x561fff87b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:49.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbdaffff640 1 --2- 172.21.15.103:0/1582858945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff874f50 0x561fff87b410 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0b8910 tx=0x7fbd9c0bbff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5beff9bc602b0ae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:49.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbd8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:49.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbd8ffff640 1 -- 172.21.15.103:0/1582858945 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c049070 con 0x561fff874f50 2024-09-19T07:03:49.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbd8ffff640 1 -- 172.21.15.103:0/1582858945 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7fbd9c0edba0 con 0x561fff874f50 2024-09-19T07:03:49.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbdb9298740 1 -- 172.21.15.103:0/1582858945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561fff7e2860 con 0x561fff874f50 2024-09-19T07:03:49.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbd8ffff640 1 -- 172.21.15.103:0/1582858945 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbd9c0bcbb0 con 0x561fff874f50 2024-09-19T07:03:49.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbdb9298740 10 client.20424 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:49.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbdb9298740 1 -- 172.21.15.103:0/1582858945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561fff881240 con 0x561fff874f50 2024-09-19T07:03:49.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.768+0000 7fbd8ffff640 1 -- 172.21.15.103:0/1582858945 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561fff7e2860 con 0x561fff874f50 2024-09-19T07:03:49.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.769+0000 7fbd8ffff640 1 -- 172.21.15.103:0/1582858945 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbd9c0c4c90 con 0x561fff874f50 2024-09-19T07:03:49.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.769+0000 7fbdb9298740 10 client.20424 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:49.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.769+0000 7fbdb9298740 10 client.20424 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:03:49.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.769+0000 7fbdb9298740 20 client.20424 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:49.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.769+0000 7fbdb9298740 1 --2- 172.21.15.103:0/1582858945 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561fff88ddf0 0x561fff890210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.769+0000 7fbdb9298740 4 client.20424 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:49.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.769+0000 7fbdb9298740 1 -- 172.21.15.103:0/1582858945 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fff7dcd70 con 0x561fff88ddf0 2024-09-19T07:03:49.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.769+0000 7fbdb4c5b640 1 --2- 172.21.15.103:0/1582858945 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561fff88ddf0 0x561fff890210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.770+0000 7fbdb4c5b640 1 --2- 172.21.15.103:0/1582858945 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561fff88ddf0 0x561fff890210 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e59677fe5cfc1f7 server_cookie=f60b77357d54a47b in_seq=0 out_seq=0 2024-09-19T07:03:49.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.770+0000 7fbd8ffff640 10 client.20424 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:03:49.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.770+0000 7fbd8ffff640 1 -- 172.21.15.103:0/1582858945 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fff7dcd70 con 0x561fff88ddf0 2024-09-19T07:03:49.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.770+0000 7fbd8ffff640 10 client.20424 handle_command_reply: tid=0 2024-09-19T07:03:49.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.770+0000 7fbdb9298740 1 client.20424 shutdown 2024-09-19T07:03:49.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.770+0000 7fbdb9298740 1 -- 172.21.15.103:0/1582858945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff874f50 msgr2=0x561fff87b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:49.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.770+0000 7fbdb9298740 1 --2- 172.21.15.103:0/1582858945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff874f50 0x561fff87b410 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0b8910 tx=0x7fbd9c0bbff0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.770+0000 7fbdb9298740 1 -- 172.21.15.103:0/1582858945 shutdown_connections 2024-09-19T07:03:49.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.770+0000 7fbdb9298740 1 --2- 172.21.15.103:0/1582858945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff87e780 0x561fff87b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.770+0000 7fbdb9298740 1 --2- 172.21.15.103:0/1582858945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff874f50 0x561fff87b410 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.770+0000 7fbdb9298740 1 --2- 172.21.15.103:0/1582858945 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561fff88ddf0 0x561fff890210 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.770+0000 7fbd8ffff640 0 client.20424 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:03:49.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.770+0000 7fbdb9298740 1 --2- 172.21.15.103:0/1582858945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff8739e0 0x561fff87e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.770+0000 7fbdb9298740 1 -- 172.21.15.103:0/1582858945 >> 172.21.15.103:0/1582858945 conn(0x561fff838310 msgr2=0x561fff875da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:49.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.771+0000 7fbdb9298740 1 -- 172.21.15.103:0/1582858945 shutdown_connections 2024-09-19T07:03:49.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.771+0000 7fbdb9298740 1 -- 172.21.15.103:0/1582858945 wait complete. 2024-09-19T07:03:49.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.771+0000 7fbdb9298740 20 client.20424 trim_cache size 0 max 16384 2024-09-19T07:03:49.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.783+0000 7fbdb9298740 1 Processor -- start 2024-09-19T07:03:49.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.783+0000 7fbdb9298740 1 -- start start 2024-09-19T07:03:49.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.783+0000 7fbdb9298740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff838310 0x561fff88a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdb9298740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff8739e0 0x561fff86abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdb9298740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff874f50 0x561fff895ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdb9298740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01ab6c0 con 0x561fff8739e0 2024-09-19T07:03:49.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdb9298740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01ad200 con 0x561fff874f50 2024-09-19T07:03:49.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdb9298740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01abd90 con 0x561fff838310 2024-09-19T07:03:49.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff8739e0 0x561fff86abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff8739e0 0x561fff86abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52672/0 (socket says 172.21.15.103:52672) 2024-09-19T07:03:49.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdaffff640 1 -- 172.21.15.103:0/1845116688 learned_addr learned my addr 172.21.15.103:0/1845116688 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:49.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdb4c5b640 1 --2- 172.21.15.103:0/1845116688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff838310 0x561fff88a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdb545c640 1 --2- 172.21.15.103:0/1845116688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff874f50 0x561fff895ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdaffff640 1 -- 172.21.15.103:0/1845116688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff838310 msgr2=0x561fff88a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:49.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdaffff640 1 --2- 172.21.15.103:0/1845116688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff838310 0x561fff88a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdaffff640 1 -- 172.21.15.103:0/1845116688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff874f50 msgr2=0x561fff895ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:49.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdaffff640 1 --2- 172.21.15.103:0/1845116688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff874f50 0x561fff895ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdaffff640 1 -- 172.21.15.103:0/1845116688 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fff870310 con 0x561fff8739e0 2024-09-19T07:03:49.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdb545c640 1 --2- 172.21.15.103:0/1845116688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff874f50 0x561fff895ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:49.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbdaffff640 1 --2- 172.21.15.103:0/1845116688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff8739e0 0x561fff86abf0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0ba760 tx=0x7fbd9c03b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bfc0d64b1d3f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:49.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbd9a7fc640 1 -- 172.21.15.103:0/1845116688 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c049070 con 0x561fff8739e0 2024-09-19T07:03:49.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbd9a7fc640 1 -- 172.21.15.103:0/1845116688 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd9c0bc4b0 con 0x561fff8739e0 2024-09-19T07:03:49.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.784+0000 7fbd9a7fc640 1 -- 172.21.15.103:0/1845116688 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0c4c90 con 0x561fff8739e0 2024-09-19T07:03:49.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.785+0000 7fbdb9298740 1 -- 172.21.15.103:0/1845116688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff8739e0 msgr2=0x561fff86abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:49.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.785+0000 7fbdb9298740 1 --2- 172.21.15.103:0/1845116688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff8739e0 0x561fff86abf0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0ba760 tx=0x7fbd9c03b640 comp rx=0 tx=0).stop 2024-09-19T07:03:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.785+0000 7fbdb9298740 1 -- 172.21.15.103:0/1845116688 shutdown_connections 2024-09-19T07:03:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.785+0000 7fbdb9298740 1 --2- 172.21.15.103:0/1845116688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff874f50 0x561fff895ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.785+0000 7fbdb9298740 1 --2- 172.21.15.103:0/1845116688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff8739e0 0x561fff86abf0 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.785+0000 7fbdb9298740 1 --2- 172.21.15.103:0/1845116688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff838310 0x561fff88a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.785+0000 7fbdb9298740 1 -- 172.21.15.103:0/1845116688 >> 172.21.15.103:0/1845116688 conn(0x561fff876e00 msgr2=0x561fff86a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.785+0000 7fbdb9298740 1 -- 172.21.15.103:0/1845116688 shutdown_connections 2024-09-19T07:03:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.785+0000 7fbdb9298740 1 -- 172.21.15.103:0/1845116688 wait complete. 2024-09-19T07:03:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.785+0000 7fbdb9298740 1 -- start start 2024-09-19T07:03:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.785+0000 7fbdb9298740 1 Processor -- start 2024-09-19T07:03:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdb9298740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff838310 0x561fff886a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdb9298740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff8739e0 0x561fff881d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdb9298740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff874f50 0x561fff8822c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdb9298740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01acb30 con 0x561fff8739e0 2024-09-19T07:03:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdb9298740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01ac460 con 0x561fff874f50 2024-09-19T07:03:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdb9298740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01a96e0 con 0x561fff838310 2024-09-19T07:03:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff8739e0 0x561fff881d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdb9298740 10 client.0 _ll_register_callbacks cb 0x7fbd9c0bc280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdb4c5b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff838310 0x561fff886a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff8739e0 0x561fff881d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52680/0 (socket says 172.21.15.103:52680) 2024-09-19T07:03:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdb4c5b640 1 --2- 172.21.15.103:0/2174245492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff838310 0x561fff886a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:47674/0 (socket says 172.21.15.103:47674) 2024-09-19T07:03:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdaffff640 1 -- 172.21.15.103:0/2174245492 learned_addr learned my addr 172.21.15.103:0/2174245492 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdb545c640 1 --2- 172.21.15.103:0/2174245492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff874f50 0x561fff8822c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdaffff640 1 -- 172.21.15.103:0/2174245492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff838310 msgr2=0x561fff886a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdaffff640 1 --2- 172.21.15.103:0/2174245492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff838310 0x561fff886a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdaffff640 1 -- 172.21.15.103:0/2174245492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff874f50 msgr2=0x561fff8822c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdaffff640 1 --2- 172.21.15.103:0/2174245492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff874f50 0x561fff8822c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdaffff640 1 -- 172.21.15.103:0/2174245492 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fff882800 con 0x561fff8739e0 2024-09-19T07:03:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdb4c5b640 1 --2- 172.21.15.103:0/2174245492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff838310 0x561fff886a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:03:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbdaffff640 1 --2- 172.21.15.103:0/2174245492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff8739e0 0x561fff881d80 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c000c00 tx=0x7fbd9c0bbff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3d8ad1f3d5f00c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.786+0000 7fbd8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.787+0000 7fbd8ffff640 1 -- 172.21.15.103:0/2174245492 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c049040 con 0x561fff8739e0 2024-09-19T07:03:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.787+0000 7fbd8ffff640 1 -- 172.21.15.103:0/2174245492 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7fbd9c0c4e90 con 0x561fff8739e0 2024-09-19T07:03:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.787+0000 7fbdb9298740 1 -- 172.21.15.103:0/2174245492 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd9c0bc4b0 con 0x561fff8739e0 2024-09-19T07:03:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.787+0000 7fbd8ffff640 1 -- 172.21.15.103:0/2174245492 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbd9c0c5120 con 0x561fff8739e0 2024-09-19T07:03:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.787+0000 7fbdb9298740 10 client.20430 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.787+0000 7fbdb9298740 1 -- 172.21.15.103:0/2174245492 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561fff882800 con 0x561fff8739e0 2024-09-19T07:03:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.787+0000 7fbd8ffff640 1 -- 172.21.15.103:0/2174245492 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd9c0bc4b0 con 0x561fff8739e0 2024-09-19T07:03:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.787+0000 7fbd8ffff640 1 -- 172.21.15.103:0/2174245492 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbd9c0b8400 con 0x561fff8739e0 2024-09-19T07:03:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.787+0000 7fbdb9298740 10 client.20430 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.787+0000 7fbdb9298740 10 client.20430 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:03:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.787+0000 7fbdb9298740 20 client.20430 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.787+0000 7fbdb9298740 1 --2- 172.21.15.103:0/2174245492 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561fff87e780 0x561fff8faba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.787+0000 7fbdb9298740 4 client.20430 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:03:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.787+0000 7fbdb9298740 1 -- 172.21.15.103:0/2174245492 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fff759170 con 0x561fff87e780 2024-09-19T07:03:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.787+0000 7fbdb4c5b640 1 --2- 172.21.15.103:0/2174245492 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561fff87e780 0x561fff8faba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbdb4c5b640 1 --2- 172.21.15.103:0/2174245492 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561fff87e780 0x561fff8faba0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1180c11ef20b0391 server_cookie=256e616f4dc4a0c7 in_seq=0 out_seq=0 2024-09-19T07:03:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbd8ffff640 10 client.20430 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:03:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbd8ffff640 1 -- 172.21.15.103:0/2174245492 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fff759170 con 0x561fff87e780 2024-09-19T07:03:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbd8ffff640 10 client.20430 handle_command_reply: tid=0 2024-09-19T07:03:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbdb9298740 1 client.20430 shutdown 2024-09-19T07:03:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbdb9298740 1 -- 172.21.15.103:0/2174245492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff8739e0 msgr2=0x561fff881d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbdb9298740 1 --2- 172.21.15.103:0/2174245492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff8739e0 0x561fff881d80 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c000c00 tx=0x7fbd9c0bbff0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbdb9298740 1 -- 172.21.15.103:0/2174245492 shutdown_connections 2024-09-19T07:03:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbdb9298740 1 --2- 172.21.15.103:0/2174245492 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561fff87e780 0x561fff8faba0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbdb9298740 1 --2- 172.21.15.103:0/2174245492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fff874f50 0x561fff8822c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbdb9298740 1 --2- 172.21.15.103:0/2174245492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fff8739e0 0x561fff881d80 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbdb9298740 1 --2- 172.21.15.103:0/2174245492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fff838310 0x561fff886a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbdb9298740 1 -- 172.21.15.103:0/2174245492 >> 172.21.15.103:0/2174245492 conn(0x561fff876e00 msgr2=0x561fff86a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:49.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbd8ffff640 0 client.20430 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:03:49.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbdb9298740 1 -- 172.21.15.103:0/2174245492 shutdown_connections 2024-09-19T07:03:49.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbdb9298740 1 -- 172.21.15.103:0/2174245492 wait complete. 2024-09-19T07:03:49.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.788+0000 7fbdb9298740 20 client.20430 trim_cache size 0 max 16384 2024-09-19T07:03:49.796 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:03:49.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.789+0000 7fbdb6ebf640 1 -- 172.21.15.103:0/3344969042 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbda00a33b0 msgr2=0x7fbda00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:49.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.789+0000 7fbdb6ebf640 1 --2- 172.21.15.103:0/3344969042 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbda00a33b0 0x7fbda00a5870 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbda8005fd0 tx=0x7fbda8004ab0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.789+0000 7fbdb6ebf640 1 -- 172.21.15.103:0/3344969042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdb0100ca0 msgr2=0x7fbdb0198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:49.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.789+0000 7fbdb6ebf640 1 --2- 172.21.15.103:0/3344969042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdb0100ca0 0x7fbdb0198980 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c005940 tx=0x7fbd9c0042f0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.789+0000 7fbdb6ebf640 1 -- 172.21.15.103:0/3344969042 shutdown_connections 2024-09-19T07:03:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.789+0000 7fbdb6ebf640 1 --2- 172.21.15.103:0/3344969042 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fbda00a33b0 0x7fbda00a5870 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.789+0000 7fbdb6ebf640 1 --2- 172.21.15.103:0/3344969042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdb0106f50 0x7fbdb019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.789+0000 7fbdb6ebf640 1 --2- 172.21.15.103:0/3344969042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdb0100ca0 0x7fbdb0198980 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.789+0000 7fbdb6ebf640 1 --2- 172.21.15.103:0/3344969042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdb01002f0 0x7fbdb0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.789+0000 7fbdb6ebf640 1 -- 172.21.15.103:0/3344969042 >> 172.21.15.103:0/3344969042 conn(0x7fbdb0077210 msgr2=0x7fbdb00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.789+0000 7fbdb6ebf640 1 -- 172.21.15.103:0/3344969042 shutdown_connections 2024-09-19T07:03:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:49.789+0000 7fbdb6ebf640 1 -- 172.21.15.103:0/3344969042 wait complete. 2024-09-19T07:03:50.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:50 smithi174 ceph-mon[35637]: pgmap v456: 97 pgs: 97 active+clean; 5.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-09-19T07:03:50.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:50 smithi103 ceph-mon[31158]: pgmap v456: 97 pgs: 97 active+clean; 5.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-09-19T07:03:50.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:50 smithi134 ceph-mon[35356]: pgmap v456: 97 pgs: 97 active+clean; 5.3 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-09-19T07:03:50.480 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:03:50.729 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:51.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.112+0000 7f17b4c53640 1 -- 172.21.15.103:0/3605953630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b0102ef0 msgr2=0x7f17b010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:51.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.112+0000 7f17b4c53640 1 --2- 172.21.15.103:0/3605953630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b0102ef0 0x7f17b010da40 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f17a8009900 tx=0x7f17a802a480 comp rx=0 tx=0).stop 2024-09-19T07:03:51.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.112+0000 7f17b4c53640 1 -- 172.21.15.103:0/3605953630 shutdown_connections 2024-09-19T07:03:51.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.112+0000 7f17b4c53640 1 --2- 172.21.15.103:0/3605953630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b0102ef0 0x7f17b010da40 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.112+0000 7f17b4c53640 1 --2- 172.21.15.103:0/3605953630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17b0102570 0x7f17b01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.112+0000 7f17b4c53640 1 --2- 172.21.15.103:0/3605953630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b0100e20 0x7f17b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.112+0000 7f17b4c53640 1 -- 172.21.15.103:0/3605953630 >> 172.21.15.103:0/3605953630 conn(0x7f17b00fc7d0 msgr2=0x7f17b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:51.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.112+0000 7f17b4c53640 1 -- 172.21.15.103:0/3605953630 shutdown_connections 2024-09-19T07:03:51.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.112+0000 7f17b4c53640 1 -- 172.21.15.103:0/3605953630 wait complete. 2024-09-19T07:03:51.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.113+0000 7f17b4c53640 1 Processor -- start 2024-09-19T07:03:51.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.113+0000 7f17b4c53640 1 -- start start 2024-09-19T07:03:51.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.113+0000 7f17b4c53640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b0100e20 0x7f17b0198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.113+0000 7f17b4c53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b0102570 0x7f17b0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.113+0000 7f17b4c53640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17b0102ef0 0x7f17b019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.113+0000 7f17b4c53640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17b01120a0 con 0x7f17b0102570 2024-09-19T07:03:51.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.113+0000 7f17b4c53640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17b0111f20 con 0x7f17b0102ef0 2024-09-19T07:03:51.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.113+0000 7f17b4c53640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17b0112220 con 0x7f17b0100e20 2024-09-19T07:03:51.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.113+0000 7f17aed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17b0102ef0 0x7f17b019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:51.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.113+0000 7f17add9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b0102570 0x7f17b0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:51.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.113+0000 7f17aed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17b0102ef0 0x7f17b019ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:34466/0 (socket says 172.21.15.103:34466) 2024-09-19T07:03:51.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.113+0000 7f17add9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b0102570 0x7f17b0198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52700/0 (socket says 172.21.15.103:52700) 2024-09-19T07:03:51.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.113+0000 7f17aed9d640 1 -- 172.21.15.103:0/176995184 learned_addr learned my addr 172.21.15.103:0/176995184 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:51.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.113+0000 7f17ae59c640 1 --2- 172.21.15.103:0/176995184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b0100e20 0x7f17b0198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:51.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.114+0000 7f17add9b640 1 -- 172.21.15.103:0/176995184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b0100e20 msgr2=0x7f17b0198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:51.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.114+0000 7f17add9b640 1 --2- 172.21.15.103:0/176995184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b0100e20 0x7f17b0198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.114+0000 7f17add9b640 1 -- 172.21.15.103:0/176995184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17b0102ef0 msgr2=0x7f17b019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:51.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.114+0000 7f17add9b640 1 --2- 172.21.15.103:0/176995184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17b0102ef0 0x7f17b019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.114+0000 7f17add9b640 1 -- 172.21.15.103:0/176995184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b019d4f0 con 0x7f17b0102570 2024-09-19T07:03:51.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.114+0000 7f17add9b640 1 --2- 172.21.15.103:0/176995184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b0102570 0x7f17b0198a80 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f17a400c970 tx=0x7f17a400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:51.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.114+0000 7f179f7fe640 1 -- 172.21.15.103:0/176995184 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4007bb0 con 0x7f17b0102570 2024-09-19T07:03:51.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.114+0000 7f179f7fe640 1 -- 172.21.15.103:0/176995184 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a4007d50 con 0x7f17b0102570 2024-09-19T07:03:51.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.114+0000 7f17b4c53640 1 -- 172.21.15.103:0/176995184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b019d7e0 con 0x7f17b0102570 2024-09-19T07:03:51.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.114+0000 7f179f7fe640 1 -- 172.21.15.103:0/176995184 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a400f450 con 0x7f17b0102570 2024-09-19T07:03:51.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.114+0000 7f17b4c53640 1 -- 172.21.15.103:0/176995184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b01a50c0 con 0x7f17b0102570 2024-09-19T07:03:51.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.117+0000 7f179f7fe640 1 -- 172.21.15.103:0/176995184 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f17a4004ef0 con 0x7f17b0102570 2024-09-19T07:03:51.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.117+0000 7f17b702c740 1 Processor -- start 2024-09-19T07:03:51.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.117+0000 7f17b702c740 1 -- start start 2024-09-19T07:03:51.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17b702c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c098994b40 0x55c098993090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17b702c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989935d0 0x55c098993a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17b702c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0989a2780 0x55c0989a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17b702c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c0988f92f0 con 0x55c0989935d0 2024-09-19T07:03:51.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17b702c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c0988a0cf0 con 0x55c098994b40 2024-09-19T07:03:51.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17b702c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c0988f52b0 con 0x55c0989a2780 2024-09-19T07:03:51.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17ae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989935d0 0x55c098993a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:51.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17ae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989935d0 0x55c098993a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52708/0 (socket says 172.21.15.103:52708) 2024-09-19T07:03:51.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17ae59c640 1 -- 172.21.15.103:0/686518823 learned_addr learned my addr 172.21.15.103:0/686518823 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:51.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17add9b640 1 --2- 172.21.15.103:0/686518823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0989a2780 0x55c0989a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:51.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17aed9d640 1 --2- 172.21.15.103:0/686518823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c098994b40 0x55c098993090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:51.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f179f7fe640 1 --2- 172.21.15.103:0/176995184 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17980a3430 0x7f17980a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17ae59c640 1 -- 172.21.15.103:0/686518823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0989a2780 msgr2=0x55c0989a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:51.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17ae59c640 1 --2- 172.21.15.103:0/686518823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0989a2780 0x55c0989a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17ae59c640 1 -- 172.21.15.103:0/686518823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c098994b40 msgr2=0x55c098993090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:51.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17ae59c640 1 --2- 172.21.15.103:0/686518823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c098994b40 0x55c098993090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17ae59c640 1 -- 172.21.15.103:0/686518823 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0988853a0 con 0x55c0989935d0 2024-09-19T07:03:51.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17aed9d640 1 --2- 172.21.15.103:0/176995184 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17980a3430 0x7f17980a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:51.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f179f7fe640 1 -- 172.21.15.103:0/176995184 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f17a40cf860 con 0x7f17b0102570 2024-09-19T07:03:51.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.118+0000 7f17ae59c640 1 --2- 172.21.15.103:0/686518823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989935d0 0x55c098993a10 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f17a000a4b0 tx=0x7f17a00096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e35f3f457700969 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:51.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.119+0000 7f179cff9640 1 -- 172.21.15.103:0/686518823 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0018020 con 0x55c0989935d0 2024-09-19T07:03:51.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.119+0000 7f179cff9640 1 -- 172.21.15.103:0/686518823 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a0014cf0 con 0x55c0989935d0 2024-09-19T07:03:51.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.119+0000 7f179cff9640 1 -- 172.21.15.103:0/686518823 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0013d50 con 0x55c0989935d0 2024-09-19T07:03:51.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.119+0000 7f17b702c740 1 -- 172.21.15.103:0/686518823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989935d0 msgr2=0x55c098993a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:51.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.119+0000 7f17aed9d640 1 --2- 172.21.15.103:0/176995184 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17980a3430 0x7f17980a58f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f17a80096c0 tx=0x7f17a8009260 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:51.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.119+0000 7f17b702c740 1 --2- 172.21.15.103:0/686518823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989935d0 0x55c098993a10 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f17a000a4b0 tx=0x7f17a00096d0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.119+0000 7f17b702c740 1 -- 172.21.15.103:0/686518823 shutdown_connections 2024-09-19T07:03:51.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.119+0000 7f17b702c740 1 --2- 172.21.15.103:0/686518823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0989a2780 0x55c0989a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17b702c740 1 --2- 172.21.15.103:0/686518823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989935d0 0x55c098993a10 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17b702c740 1 --2- 172.21.15.103:0/686518823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c098994b40 0x55c098993090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17b702c740 1 -- 172.21.15.103:0/686518823 >> 172.21.15.103:0/686518823 conn(0x55c09895c310 msgr2=0x55c09898fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:51.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17b702c740 1 -- 172.21.15.103:0/686518823 shutdown_connections 2024-09-19T07:03:51.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17b702c740 1 -- 172.21.15.103:0/686518823 wait complete. 2024-09-19T07:03:51.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17b702c740 1 -- start start 2024-09-19T07:03:51.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17b702c740 1 Processor -- start 2024-09-19T07:03:51.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17b702c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989935d0 0x55c0989a20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17b702c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c098994b40 0x55c09899f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17b702c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0989a2780 0x55c09899f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17b702c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c0989a5480 con 0x55c0989935d0 2024-09-19T07:03:51.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17b702c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c0988b4060 con 0x55c0989a2780 2024-09-19T07:03:51.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17b702c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c0989a5600 con 0x55c098994b40 2024-09-19T07:03:51.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17ae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989935d0 0x55c0989a20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:51.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17b702c740 10 client.0 _ll_register_callbacks cb 0x55c0986078c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:51.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17ae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989935d0 0x55c0989a20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52718/0 (socket says 172.21.15.103:52718) 2024-09-19T07:03:51.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.120+0000 7f17ae59c640 1 -- 172.21.15.103:0/4010201253 learned_addr learned my addr 172.21.15.103:0/4010201253 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:51.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f17aed9d640 1 --2- 172.21.15.103:0/4010201253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0989a2780 0x55c09899f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:51.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f17add9b640 1 --2- 172.21.15.103:0/4010201253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c098994b40 0x55c09899f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:51.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f17ae59c640 1 -- 172.21.15.103:0/4010201253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c098994b40 msgr2=0x55c09899f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:51.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f17ae59c640 1 --2- 172.21.15.103:0/4010201253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c098994b40 0x55c09899f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f17ae59c640 1 -- 172.21.15.103:0/4010201253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0989a2780 msgr2=0x55c09899f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:51.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f17ae59c640 1 --2- 172.21.15.103:0/4010201253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0989a2780 0x55c09899f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f17ae59c640 1 -- 172.21.15.103:0/4010201253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c09866a3c0 con 0x55c0989935d0 2024-09-19T07:03:51.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f17aed9d640 1 --2- 172.21.15.103:0/4010201253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0989a2780 0x55c09899f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:51.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f17ae59c640 1 --2- 172.21.15.103:0/4010201253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989935d0 0x55c0989a20d0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f17a0014f50 tx=0x7f17a0013d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e5a803af5f953 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:51.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f1795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:51.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f1795ffb640 1 -- 172.21.15.103:0/4010201253 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0018030 con 0x55c0989935d0 2024-09-19T07:03:51.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f1795ffb640 1 -- 172.21.15.103:0/4010201253 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f17a0014810 con 0x55c0989935d0 2024-09-19T07:03:51.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f17b702c740 1 -- 172.21.15.103:0/4010201253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17a0014cf0 con 0x55c0989935d0 2024-09-19T07:03:51.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f1795ffb640 1 -- 172.21.15.103:0/4010201253 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f17a0014a10 con 0x55c0989935d0 2024-09-19T07:03:51.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f17b702c740 10 client.20442 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:51.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f17b702c740 1 -- 172.21.15.103:0/4010201253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c0989b0e90 con 0x55c0989935d0 2024-09-19T07:03:51.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.121+0000 7f1795ffb640 1 -- 172.21.15.103:0/4010201253 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a0014cf0 con 0x55c0989935d0 2024-09-19T07:03:51.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.122+0000 7f1795ffb640 1 -- 172.21.15.103:0/4010201253 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f17a001d420 con 0x55c0989935d0 2024-09-19T07:03:51.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.122+0000 7f17b702c740 10 client.20442 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:51.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.122+0000 7f17b702c740 10 client.20442 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:03:51.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.122+0000 7f17b702c740 20 client.20442 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:51.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.122+0000 7f17b702c740 1 --2- 172.21.15.103:0/4010201253 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c0989b21e0 0x55c0989b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.122+0000 7f17b702c740 4 client.20442 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:51.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.122+0000 7f17b702c740 1 -- 172.21.15.103:0/4010201253 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c098906860 con 0x55c0989b21e0 2024-09-19T07:03:51.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.122+0000 7f17add9b640 1 --2- 172.21.15.103:0/4010201253 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c0989b21e0 0x55c0989b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:51.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.122+0000 7f17add9b640 1 --2- 172.21.15.103:0/4010201253 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c0989b21e0 0x55c0989b45d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abf34dda352dcc95 server_cookie=888e8c2eeaf2ea34 in_seq=0 out_seq=0 2024-09-19T07:03:51.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.122+0000 7f1795ffb640 10 client.20442 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:03:51.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.123+0000 7f1795ffb640 1 -- 172.21.15.103:0/4010201253 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c098906860 con 0x55c0989b21e0 2024-09-19T07:03:51.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.123+0000 7f1795ffb640 10 client.20442 handle_command_reply: tid=0 2024-09-19T07:03:51.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.123+0000 7f17b702c740 1 client.20442 shutdown 2024-09-19T07:03:51.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.123+0000 7f17b702c740 1 -- 172.21.15.103:0/4010201253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989935d0 msgr2=0x55c0989a20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:51.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.123+0000 7f17b702c740 1 --2- 172.21.15.103:0/4010201253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989935d0 0x55c0989a20d0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f17a0014f50 tx=0x7f17a0013d80 comp rx=0 tx=0).stop 2024-09-19T07:03:51.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.123+0000 7f17b702c740 1 -- 172.21.15.103:0/4010201253 shutdown_connections 2024-09-19T07:03:51.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.123+0000 7f17b702c740 1 --2- 172.21.15.103:0/4010201253 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c0989b21e0 0x55c0989b45d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.123+0000 7f17b702c740 1 --2- 172.21.15.103:0/4010201253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0989a2780 0x55c09899f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.123+0000 7f17b702c740 1 --2- 172.21.15.103:0/4010201253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c098994b40 0x55c09899f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.123+0000 7f17b702c740 1 --2- 172.21.15.103:0/4010201253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989935d0 0x55c0989a20d0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.123+0000 7f17b702c740 1 -- 172.21.15.103:0/4010201253 >> 172.21.15.103:0/4010201253 conn(0x55c09895c310 msgr2=0x55c098995960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:51.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.123+0000 7f1795ffb640 0 client.20442 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:03:51.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.123+0000 7f17b702c740 1 -- 172.21.15.103:0/4010201253 shutdown_connections 2024-09-19T07:03:51.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.123+0000 7f17b702c740 1 -- 172.21.15.103:0/4010201253 wait complete. 2024-09-19T07:03:51.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.123+0000 7f17b702c740 20 client.20442 trim_cache size 0 max 16384 2024-09-19T07:03:51.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.136+0000 7f17b702c740 1 Processor -- start 2024-09-19T07:03:51.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.136+0000 7f17b702c740 1 -- start start 2024-09-19T07:03:51.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.136+0000 7f17b702c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0989935d0 0x55c09898fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.136+0000 7f17b702c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c098994b40 0x55c0989ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.136+0000 7f17b702c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0989a2780 0x55c0989b9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.136+0000 7f17b702c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17b01ab720 con 0x55c098994b40 2024-09-19T07:03:51.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.136+0000 7f17b702c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17b01ad260 con 0x55c0989a2780 2024-09-19T07:03:51.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.136+0000 7f17b702c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17b01abdf0 con 0x55c0989935d0 2024-09-19T07:03:51.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.136+0000 7f17add9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c098994b40 0x55c0989ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:51.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.136+0000 7f17add9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c098994b40 0x55c0989ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52728/0 (socket says 172.21.15.103:52728) 2024-09-19T07:03:51.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.136+0000 7f17add9b640 1 -- 172.21.15.103:0/3903201158 learned_addr learned my addr 172.21.15.103:0/3903201158 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:51.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.137+0000 7f17add9b640 1 -- 172.21.15.103:0/3903201158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0989935d0 msgr2=0x55c09898fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:51.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.137+0000 7f17add9b640 1 --2- 172.21.15.103:0/3903201158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0989935d0 0x55c09898fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.137+0000 7f17add9b640 1 -- 172.21.15.103:0/3903201158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0989a2780 msgr2=0x55c0989b9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:51.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.137+0000 7f17add9b640 1 --2- 172.21.15.103:0/3903201158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0989a2780 0x55c0989b9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.137+0000 7f17add9b640 1 -- 172.21.15.103:0/3903201158 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c09899d8f0 con 0x55c098994b40 2024-09-19T07:03:51.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.137+0000 7f17add9b640 1 --2- 172.21.15.103:0/3903201158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c098994b40 0x55c0989ae720 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f17a40058c0 tx=0x7f17a40d1d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1975374ad72db872 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:51.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.137+0000 7f1797fff640 1 -- 172.21.15.103:0/3903201158 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a401d040 con 0x55c098994b40 2024-09-19T07:03:51.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.137+0000 7f1797fff640 1 -- 172.21.15.103:0/3903201158 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a409ce50 con 0x55c098994b40 2024-09-19T07:03:51.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.137+0000 7f1797fff640 1 -- 172.21.15.103:0/3903201158 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4098240 con 0x55c098994b40 2024-09-19T07:03:51.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.137+0000 7f17b702c740 1 -- 172.21.15.103:0/3903201158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c098994b40 msgr2=0x55c0989ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:51.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.137+0000 7f17b702c740 1 --2- 172.21.15.103:0/3903201158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c098994b40 0x55c0989ae720 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f17a40058c0 tx=0x7f17a40d1d70 comp rx=0 tx=0).stop 2024-09-19T07:03:51.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17b702c740 1 -- 172.21.15.103:0/3903201158 shutdown_connections 2024-09-19T07:03:51.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17b702c740 1 --2- 172.21.15.103:0/3903201158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0989a2780 0x55c0989b9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17b702c740 1 --2- 172.21.15.103:0/3903201158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c098994b40 0x55c0989ae720 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17b702c740 1 --2- 172.21.15.103:0/3903201158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0989935d0 0x55c09898fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17b702c740 1 -- 172.21.15.103:0/3903201158 >> 172.21.15.103:0/3903201158 conn(0x55c09895c310 msgr2=0x55c09898f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:51.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17b702c740 1 -- 172.21.15.103:0/3903201158 shutdown_connections 2024-09-19T07:03:51.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17b702c740 1 -- 172.21.15.103:0/3903201158 wait complete. 2024-09-19T07:03:51.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17b702c740 1 -- start start 2024-09-19T07:03:51.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17b702c740 1 Processor -- start 2024-09-19T07:03:51.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17b702c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0989935d0 0x55c0989aa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17b702c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c098994b40 0x55c0989aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17b702c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989a2780 0x55c09899b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17b702c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17b01acb90 con 0x55c0989a2780 2024-09-19T07:03:51.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17b702c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17b01ac4c0 con 0x55c098994b40 2024-09-19T07:03:51.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17b702c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17b01a9740 con 0x55c0989935d0 2024-09-19T07:03:51.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17b702c740 10 client.0 _ll_register_callbacks cb 0x7f17a0009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:51.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17aed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989a2780 0x55c09899b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:51.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17aed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989a2780 0x55c09899b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52732/0 (socket says 172.21.15.103:52732) 2024-09-19T07:03:51.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.138+0000 7f17aed9d640 1 -- 172.21.15.103:0/3206046295 learned_addr learned my addr 172.21.15.103:0/3206046295 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:51.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.139+0000 7f17add9b640 1 --2- 172.21.15.103:0/3206046295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c098994b40 0x55c0989aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:51.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.139+0000 7f17ae59c640 1 --2- 172.21.15.103:0/3206046295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0989935d0 0x55c0989aa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:51.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.139+0000 7f17aed9d640 1 -- 172.21.15.103:0/3206046295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0989935d0 msgr2=0x55c0989aa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:51.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.139+0000 7f17aed9d640 1 --2- 172.21.15.103:0/3206046295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0989935d0 0x55c0989aa890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.139+0000 7f17aed9d640 1 -- 172.21.15.103:0/3206046295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c098994b40 msgr2=0x55c0989aadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:51.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.139+0000 7f17aed9d640 1 --2- 172.21.15.103:0/3206046295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c098994b40 0x55c0989aadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.139+0000 7f17aed9d640 1 -- 172.21.15.103:0/3206046295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c09899bf10 con 0x55c0989a2780 2024-09-19T07:03:51.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.139+0000 7f17aed9d640 1 --2- 172.21.15.103:0/3206046295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989a2780 0x55c09899b9d0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f17a8006010 tx=0x7f17a8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db8018a3ffcbedc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:51.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.139+0000 7f1795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:51.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.139+0000 7f1795ffb640 1 -- 172.21.15.103:0/3206046295 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8035040 con 0x55c0989a2780 2024-09-19T07:03:51.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.139+0000 7f1795ffb640 1 -- 172.21.15.103:0/3206046295 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f17a8003800 con 0x55c0989a2780 2024-09-19T07:03:51.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.139+0000 7f17b702c740 1 -- 172.21.15.103:0/3206046295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17a409ce50 con 0x55c0989a2780 2024-09-19T07:03:51.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.139+0000 7f1795ffb640 1 -- 172.21.15.103:0/3206046295 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f17a8003a00 con 0x55c0989a2780 2024-09-19T07:03:51.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.139+0000 7f17b702c740 10 client.20448 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:51.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.139+0000 7f17b702c740 1 -- 172.21.15.103:0/3206046295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c098a1c460 con 0x55c0989a2780 2024-09-19T07:03:51.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.139+0000 7f1795ffb640 1 -- 172.21.15.103:0/3206046295 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a409ce50 con 0x55c0989a2780 2024-09-19T07:03:51.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.140+0000 7f1795ffb640 1 -- 172.21.15.103:0/3206046295 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f17a805bbf0 con 0x55c0989a2780 2024-09-19T07:03:51.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.140+0000 7f17b702c740 10 client.20448 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:51.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.140+0000 7f17b702c740 10 client.20448 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:03:51.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.140+0000 7f17b702c740 20 client.20448 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:51.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.140+0000 7f17b702c740 1 --2- 172.21.15.103:0/3206046295 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c098a1cbe0 0x55c098a1f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:51.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.140+0000 7f17b702c740 4 client.20448 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:03:51.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.140+0000 7f17b702c740 1 -- 172.21.15.103:0/3206046295 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c098900d70 con 0x55c098a1cbe0 2024-09-19T07:03:51.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.140+0000 7f17ae59c640 1 --2- 172.21.15.103:0/3206046295 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c098a1cbe0 0x55c098a1f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:51.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.140+0000 7f17ae59c640 1 --2- 172.21.15.103:0/3206046295 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c098a1cbe0 0x55c098a1f000 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=166d094f4748b823 server_cookie=164f2227f15ab019 in_seq=0 out_seq=0 2024-09-19T07:03:51.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.140+0000 7f1795ffb640 10 client.20448 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:03:51.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.140+0000 7f1795ffb640 1 -- 172.21.15.103:0/3206046295 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c098900d70 con 0x55c098a1cbe0 2024-09-19T07:03:51.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f1795ffb640 10 client.20448 handle_command_reply: tid=0 2024-09-19T07:03:51.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b702c740 1 client.20448 shutdown 2024-09-19T07:03:51.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b702c740 1 -- 172.21.15.103:0/3206046295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989a2780 msgr2=0x55c09899b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:51.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b702c740 1 --2- 172.21.15.103:0/3206046295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989a2780 0x55c09899b9d0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f17a8006010 tx=0x7f17a8005a20 comp rx=0 tx=0).stop 2024-09-19T07:03:51.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b702c740 1 -- 172.21.15.103:0/3206046295 shutdown_connections 2024-09-19T07:03:51.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b702c740 1 --2- 172.21.15.103:0/3206046295 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c098a1cbe0 0x55c098a1f000 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b702c740 1 --2- 172.21.15.103:0/3206046295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0989a2780 0x55c09899b9d0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b702c740 1 --2- 172.21.15.103:0/3206046295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c098994b40 0x55c0989aadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b702c740 1 --2- 172.21.15.103:0/3206046295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0989935d0 0x55c0989aa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b702c740 1 -- 172.21.15.103:0/3206046295 >> 172.21.15.103:0/3206046295 conn(0x55c09895c310 msgr2=0x55c09899dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:51.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f1795ffb640 0 client.20448 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:03:51.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b702c740 1 -- 172.21.15.103:0/3206046295 shutdown_connections 2024-09-19T07:03:51.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b702c740 1 -- 172.21.15.103:0/3206046295 wait complete. 2024-09-19T07:03:51.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b702c740 20 client.20448 trim_cache size 0 max 16384 2024-09-19T07:03:51.149 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:03:51.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b4c53640 1 -- 172.21.15.103:0/176995184 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17980a3430 msgr2=0x7f17980a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:51.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b4c53640 1 --2- 172.21.15.103:0/176995184 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17980a3430 0x7f17980a58f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f17a80096c0 tx=0x7f17a8009260 comp rx=0 tx=0).stop 2024-09-19T07:03:51.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b4c53640 1 -- 172.21.15.103:0/176995184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b0102570 msgr2=0x7f17b0198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:51.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b4c53640 1 --2- 172.21.15.103:0/176995184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b0102570 0x7f17b0198a80 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f17a400c970 tx=0x7f17a400ce30 comp rx=0 tx=0).stop 2024-09-19T07:03:51.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b4c53640 1 -- 172.21.15.103:0/176995184 shutdown_connections 2024-09-19T07:03:51.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b4c53640 1 --2- 172.21.15.103:0/176995184 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f17980a3430 0x7f17980a58f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b4c53640 1 --2- 172.21.15.103:0/176995184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17b0102ef0 0x7f17b019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b4c53640 1 --2- 172.21.15.103:0/176995184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b0102570 0x7f17b0198a80 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b4c53640 1 --2- 172.21.15.103:0/176995184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b0100e20 0x7f17b0198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:51.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.141+0000 7f17b4c53640 1 -- 172.21.15.103:0/176995184 >> 172.21.15.103:0/176995184 conn(0x7f17b00fc7d0 msgr2=0x7f17b0106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:51.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.142+0000 7f17b4c53640 1 -- 172.21.15.103:0/176995184 shutdown_connections 2024-09-19T07:03:51.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:51.142+0000 7f17b4c53640 1 -- 172.21.15.103:0/176995184 wait complete. 2024-09-19T07:03:51.848 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:03:52.095 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:52.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:52 smithi174 ceph-mon[35637]: pgmap v457: 97 pgs: 97 active+clean; 5.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T07:03:52.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:52 smithi103 ceph-mon[31158]: pgmap v457: 97 pgs: 97 active+clean; 5.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T07:03:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.370+0000 7f0b1a8e4640 1 -- 172.21.15.103:0/2645593262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b14103600 msgr2=0x7f0b1410bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.370+0000 7f0b1a8e4640 1 --2- 172.21.15.103:0/2645593262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b14103600 0x7f0b1410bae0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009900 tx=0x7f0afc02a480 comp rx=0 tx=0).stop 2024-09-19T07:03:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.370+0000 7f0b1a8e4640 1 -- 172.21.15.103:0/2645593262 shutdown_connections 2024-09-19T07:03:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.370+0000 7f0b1a8e4640 1 --2- 172.21.15.103:0/2645593262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1410c0b0 0x7f0b1410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.370+0000 7f0b1a8e4640 1 --2- 172.21.15.103:0/2645593262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b14103600 0x7f0b1410bae0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.370+0000 7f0b1a8e4640 1 --2- 172.21.15.103:0/2645593262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b14102ce0 0x7f0b141030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.370+0000 7f0b1a8e4640 1 -- 172.21.15.103:0/2645593262 >> 172.21.15.103:0/2645593262 conn(0x7f0b14077990 msgr2=0x7f0b140fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.370+0000 7f0b1a8e4640 1 -- 172.21.15.103:0/2645593262 shutdown_connections 2024-09-19T07:03:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.370+0000 7f0b1a8e4640 1 -- 172.21.15.103:0/2645593262 wait complete. 2024-09-19T07:03:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.370+0000 7f0b1a8e4640 1 Processor -- start 2024-09-19T07:03:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b1a8e4640 1 -- start start 2024-09-19T07:03:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b1a8e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b14102ce0 0x7f0b141982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b1a8e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b14103600 0x7f0b14198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b1a8e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1410c0b0 0x7f0b1419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b1a8e4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b14111eb0 con 0x7f0b14102ce0 2024-09-19T07:03:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b1a8e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b14111d30 con 0x7f0b14103600 2024-09-19T07:03:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b1a8e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b14112030 con 0x7f0b1410c0b0 2024-09-19T07:03:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b13fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b14102ce0 0x7f0b141982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b13fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b14102ce0 0x7f0b141982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52736/0 (socket says 172.21.15.103:52736) 2024-09-19T07:03:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b13fff640 1 -- 172.21.15.103:0/4002271696 learned_addr learned my addr 172.21.15.103:0/4002271696 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b18e81640 1 --2- 172.21.15.103:0/4002271696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1410c0b0 0x7f0b1419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b13fff640 1 -- 172.21.15.103:0/4002271696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1410c0b0 msgr2=0x7f0b1419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b13fff640 1 --2- 172.21.15.103:0/4002271696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1410c0b0 0x7f0b1419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b13fff640 1 -- 172.21.15.103:0/4002271696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b14103600 msgr2=0x7f0b14198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b13fff640 1 --2- 172.21.15.103:0/4002271696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b14103600 0x7f0b14198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b13fff640 1 -- 172.21.15.103:0/4002271696 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1419d2d0 con 0x7f0b14102ce0 2024-09-19T07:03:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b13fff640 1 --2- 172.21.15.103:0/4002271696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b14102ce0 0x7f0b141982e0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400d950 tx=0x7f0b0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b11ffb640 1 -- 172.21.15.103:0/4002271696 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04017030 con 0x7f0b14102ce0 2024-09-19T07:03:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b1a8e4640 1 -- 172.21.15.103:0/4002271696 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b1419d5c0 con 0x7f0b14102ce0 2024-09-19T07:03:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b11ffb640 1 -- 172.21.15.103:0/4002271696 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b04004020 con 0x7f0b14102ce0 2024-09-19T07:03:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.371+0000 7f0b11ffb640 1 -- 172.21.15.103:0/4002271696 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b040047a0 con 0x7f0b14102ce0 2024-09-19T07:03:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.372+0000 7f0b1a8e4640 1 -- 172.21.15.103:0/4002271696 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b141a4e50 con 0x7f0b14102ce0 2024-09-19T07:03:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.374+0000 7f0b11ffb640 1 -- 172.21.15.103:0/4002271696 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0b040041d0 con 0x7f0b14102ce0 2024-09-19T07:03:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.374+0000 7f0b11ffb640 1 --2- 172.21.15.103:0/4002271696 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b000a3460 0x7f0b000a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b1d687740 1 Processor -- start 2024-09-19T07:03:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b11ffb640 1 -- 172.21.15.103:0/4002271696 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f0b040cf7d0 con 0x7f0b14102ce0 2024-09-19T07:03:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b0bfff640 1 --2- 172.21.15.103:0/4002271696 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b000a3460 0x7f0b000a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b1d687740 1 -- start start 2024-09-19T07:03:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b1d687740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294150af0 0x55b294150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b1d687740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b294151410 0x55b29415b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b1d687740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b29415bcb0 0x55b29415e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b1d687740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b2940b52f0 con 0x55b294150af0 2024-09-19T07:03:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b1d687740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b29405d270 con 0x55b29415bcb0 2024-09-19T07:03:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b1d687740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b2940b12b0 con 0x55b294151410 2024-09-19T07:03:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b13fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294150af0 0x55b294150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b13fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294150af0 0x55b294150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52744/0 (socket says 172.21.15.103:52744) 2024-09-19T07:03:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b13fff640 1 -- 172.21.15.103:0/1896873742 learned_addr learned my addr 172.21.15.103:0/1896873742 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b0bfff640 1 --2- 172.21.15.103:0/4002271696 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b000a3460 0x7f0b000a5920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0afc02a9a0 tx=0x7f0afc000bc0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b0bfff640 1 --2- 172.21.15.103:0/1896873742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b294151410 0x55b29415b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b13fff640 1 -- 172.21.15.103:0/1896873742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b294151410 msgr2=0x55b29415b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b13fff640 1 --2- 172.21.15.103:0/1896873742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b294151410 0x55b29415b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b13fff640 1 -- 172.21.15.103:0/1896873742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b29415bcb0 msgr2=0x55b29415e0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:03:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b13fff640 1 --2- 172.21.15.103:0/1896873742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b29415bcb0 0x55b29415e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b13fff640 1 -- 172.21.15.103:0/1896873742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b29405c340 con 0x55b294150af0 2024-09-19T07:03:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.375+0000 7f0b0bfff640 1 --2- 172.21.15.103:0/1896873742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b294151410 0x55b29415b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.376+0000 7f0b13fff640 1 --2- 172.21.15.103:0/1896873742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294150af0 0x55b294150ed0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f0b04098460 tx=0x7f0b0409b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a822a454544329f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.376+0000 7f0b0a7fc640 1 -- 172.21.15.103:0/1896873742 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0409fe40 con 0x55b294150af0 2024-09-19T07:03:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.376+0000 7f0b0a7fc640 1 -- 172.21.15.103:0/1896873742 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b040a4700 con 0x55b294150af0 2024-09-19T07:03:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.376+0000 7f0b0a7fc640 1 -- 172.21.15.103:0/1896873742 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b040a4270 con 0x55b294150af0 2024-09-19T07:03:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.376+0000 7f0b1d687740 1 -- 172.21.15.103:0/1896873742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294150af0 msgr2=0x55b294150ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.376+0000 7f0b1d687740 1 --2- 172.21.15.103:0/1896873742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294150af0 0x55b294150ed0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f0b04098460 tx=0x7f0b0409b990 comp rx=0 tx=0).stop 2024-09-19T07:03:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.377+0000 7f0b1d687740 1 -- 172.21.15.103:0/1896873742 shutdown_connections 2024-09-19T07:03:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.377+0000 7f0b1d687740 1 --2- 172.21.15.103:0/1896873742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b29415bcb0 0x55b29415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.377+0000 7f0b1d687740 1 --2- 172.21.15.103:0/1896873742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b294151410 0x55b29415b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.377+0000 7f0b1d687740 1 --2- 172.21.15.103:0/1896873742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294150af0 0x55b294150ed0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.377+0000 7f0b1d687740 1 -- 172.21.15.103:0/1896873742 >> 172.21.15.103:0/1896873742 conn(0x55b29414d060 msgr2=0x55b29414d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.377+0000 7f0b1d687740 1 -- 172.21.15.103:0/1896873742 shutdown_connections 2024-09-19T07:03:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.377+0000 7f0b1d687740 1 -- 172.21.15.103:0/1896873742 wait complete. 2024-09-19T07:03:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.377+0000 7f0b1d687740 1 -- start start 2024-09-19T07:03:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.377+0000 7f0b1d687740 1 Processor -- start 2024-09-19T07:03:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.377+0000 7f0b1d687740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b294150af0 0x55b294158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.377+0000 7f0b1d687740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b294151410 0x55b294159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.377+0000 7f0b1d687740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b29415bcb0 0x55b294159640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.377+0000 7f0b1d687740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b29415e820 con 0x55b29415bcb0 2024-09-19T07:03:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.377+0000 7f0b1d687740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b29403e290 con 0x55b294151410 2024-09-19T07:03:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.377+0000 7f0b1d687740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b29415e9a0 con 0x55b294150af0 2024-09-19T07:03:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0b1d687740 10 client.0 _ll_register_callbacks cb 0x55b29406f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0b18e81640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b29415bcb0 0x55b294159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0b18e81640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b29415bcb0 0x55b294159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52750/0 (socket says 172.21.15.103:52750) 2024-09-19T07:03:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0b18e81640 1 -- 172.21.15.103:0/1388825168 learned_addr learned my addr 172.21.15.103:0/1388825168 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0b13fff640 1 --2- 172.21.15.103:0/1388825168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b294150af0 0x55b294158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0b0bfff640 1 --2- 172.21.15.103:0/1388825168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b294151410 0x55b294159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0b18e81640 1 -- 172.21.15.103:0/1388825168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b294150af0 msgr2=0x55b294158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0b18e81640 1 --2- 172.21.15.103:0/1388825168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b294150af0 0x55b294158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0b18e81640 1 -- 172.21.15.103:0/1388825168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b294151410 msgr2=0x55b294159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0b18e81640 1 --2- 172.21.15.103:0/1388825168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b294151410 0x55b294159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0b18e81640 1 -- 172.21.15.103:0/1388825168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b293dc8210 con 0x55b29415bcb0 2024-09-19T07:03:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0b0bfff640 1 --2- 172.21.15.103:0/1388825168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b294151410 0x55b294159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0b18e81640 1 --2- 172.21.15.103:0/1388825168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b29415bcb0 0x55b294159640 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00e7c0 tx=0x7f0b0c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67fcea4c7b34526 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0af37fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0af37fe640 1 -- 172.21.15.103:0/1388825168 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c004d20 con 0x55b29415bcb0 2024-09-19T07:03:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0af37fe640 1 -- 172.21.15.103:0/1388825168 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f0b0c00cac0 con 0x55b29415bcb0 2024-09-19T07:03:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0b1d687740 1 -- 172.21.15.103:0/1388825168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b040a4700 con 0x55b29415bcb0 2024-09-19T07:03:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0af37fe640 1 -- 172.21.15.103:0/1388825168 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0b0c021050 con 0x55b29415bcb0 2024-09-19T07:03:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0b1d687740 10 client.20460 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.378+0000 7f0b1d687740 1 -- 172.21.15.103:0/1388825168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b293e94620 con 0x55b29415bcb0 2024-09-19T07:03:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.379+0000 7f0af37fe640 1 -- 172.21.15.103:0/1388825168 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b040a4700 con 0x55b29415bcb0 2024-09-19T07:03:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.379+0000 7f0af37fe640 1 -- 172.21.15.103:0/1388825168 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0b0c0041f0 con 0x55b29415bcb0 2024-09-19T07:03:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.379+0000 7f0b1d687740 10 client.20460 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.379+0000 7f0b1d687740 10 client.20460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.379+0000 7f0b1d687740 20 client.20460 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.379+0000 7f0b1d687740 1 --2- 172.21.15.103:0/1388825168 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b29416bbc0 0x55b29416dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.379+0000 7f0b1d687740 4 client.20460 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.379+0000 7f0b1d687740 1 -- 172.21.15.103:0/1388825168 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2940c2a20 con 0x55b29416bbc0 2024-09-19T07:03:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.379+0000 7f0b13fff640 1 --2- 172.21.15.103:0/1388825168 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b29416bbc0 0x55b29416dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.379+0000 7f0af37fe640 10 client.20460 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.379+0000 7f0b13fff640 1 --2- 172.21.15.103:0/1388825168 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b29416bbc0 0x55b29416dff0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=693df2761efc2c4e server_cookie=4887cfc1f693a513 in_seq=0 out_seq=0 2024-09-19T07:03:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.380+0000 7f0af37fe640 1 -- 172.21.15.103:0/1388825168 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2940c2a20 con 0x55b29416bbc0 2024-09-19T07:03:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.380+0000 7f0af37fe640 10 client.20460 handle_command_reply: tid=0 2024-09-19T07:03:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.380+0000 7f0b1d687740 1 client.20460 shutdown 2024-09-19T07:03:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.380+0000 7f0b1d687740 1 -- 172.21.15.103:0/1388825168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b29415bcb0 msgr2=0x55b294159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.380+0000 7f0b1d687740 1 --2- 172.21.15.103:0/1388825168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b29415bcb0 0x55b294159640 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00e7c0 tx=0x7f0b0c00ec80 comp rx=0 tx=0).stop 2024-09-19T07:03:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.380+0000 7f0b1d687740 1 -- 172.21.15.103:0/1388825168 shutdown_connections 2024-09-19T07:03:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.380+0000 7f0b1d687740 1 --2- 172.21.15.103:0/1388825168 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b29416bbc0 0x55b29416dff0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.380+0000 7f0b1d687740 1 --2- 172.21.15.103:0/1388825168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b29415bcb0 0x55b294159640 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.380+0000 7f0b1d687740 1 --2- 172.21.15.103:0/1388825168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b294151410 0x55b294159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.380+0000 7f0b1d687740 1 --2- 172.21.15.103:0/1388825168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b294150af0 0x55b294158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.380+0000 7f0af37fe640 0 client.20460 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.380+0000 7f0b1d687740 1 -- 172.21.15.103:0/1388825168 >> 172.21.15.103:0/1388825168 conn(0x55b29414d060 msgr2=0x55b294152d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.381+0000 7f0b1d687740 1 -- 172.21.15.103:0/1388825168 shutdown_connections 2024-09-19T07:03:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.381+0000 7f0b1d687740 1 -- 172.21.15.103:0/1388825168 wait complete. 2024-09-19T07:03:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.381+0000 7f0b1d687740 20 client.20460 trim_cache size 0 max 16384 2024-09-19T07:03:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.405+0000 7f0b1d687740 1 Processor -- start 2024-09-19T07:03:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.405+0000 7f0b1d687740 1 -- start start 2024-09-19T07:03:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.405+0000 7f0b1d687740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294150af0 0x55b294153560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.405+0000 7f0b1d687740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b294151410 0x55b294159470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.405+0000 7f0b1d687740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b29415bcb0 0x55b294173900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.405+0000 7f0b1d687740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b141ad8b0 con 0x55b294150af0 2024-09-19T07:03:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.405+0000 7f0b1d687740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b141af3f0 con 0x55b294151410 2024-09-19T07:03:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.405+0000 7f0b1d687740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b141adf80 con 0x55b29415bcb0 2024-09-19T07:03:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.405+0000 7f0b13fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294150af0 0x55b294153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.405+0000 7f0b13fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294150af0 0x55b294153560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52752/0 (socket says 172.21.15.103:52752) 2024-09-19T07:03:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.405+0000 7f0b13fff640 1 -- 172.21.15.103:0/3782115001 learned_addr learned my addr 172.21.15.103:0/3782115001 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.405+0000 7f0b13fff640 1 -- 172.21.15.103:0/3782115001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b29415bcb0 msgr2=0x55b294173900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.405+0000 7f0b18e81640 1 --2- 172.21.15.103:0/3782115001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b29415bcb0 0x55b294173900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.406+0000 7f0b0bfff640 1 --2- 172.21.15.103:0/3782115001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b294151410 0x55b294159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.406+0000 7f0b13fff640 1 --2- 172.21.15.103:0/3782115001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b29415bcb0 0x55b294173900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.406+0000 7f0b13fff640 1 -- 172.21.15.103:0/3782115001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b294151410 msgr2=0x55b294159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.406+0000 7f0b13fff640 1 --2- 172.21.15.103:0/3782115001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b294151410 0x55b294159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.406+0000 7f0b13fff640 1 -- 172.21.15.103:0/3782115001 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b294150920 con 0x55b294150af0 2024-09-19T07:03:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.406+0000 7f0b18e81640 1 --2- 172.21.15.103:0/3782115001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b29415bcb0 0x55b294173900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.406+0000 7f0b0bfff640 1 --2- 172.21.15.103:0/3782115001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b294151410 0x55b294159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.406+0000 7f0b13fff640 1 --2- 172.21.15.103:0/3782115001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294150af0 0x55b294153560 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f0b04099d40 tx=0x7f0b04005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbe3af2c729dffa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.406+0000 7f0b09ffb640 1 -- 172.21.15.103:0/3782115001 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b040cd180 con 0x55b294150af0 2024-09-19T07:03:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.406+0000 7f0b09ffb640 1 -- 172.21.15.103:0/3782115001 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b04098100 con 0x55b294150af0 2024-09-19T07:03:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.406+0000 7f0b09ffb640 1 -- 172.21.15.103:0/3782115001 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0409aee0 con 0x55b294150af0 2024-09-19T07:03:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.406+0000 7f0b1d687740 1 -- 172.21.15.103:0/3782115001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294150af0 msgr2=0x55b294153560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.406+0000 7f0b1d687740 1 --2- 172.21.15.103:0/3782115001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294150af0 0x55b294153560 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f0b04099d40 tx=0x7f0b04005bf0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b1d687740 1 -- 172.21.15.103:0/3782115001 shutdown_connections 2024-09-19T07:03:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b1d687740 1 --2- 172.21.15.103:0/3782115001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b29415bcb0 0x55b294173900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b1d687740 1 --2- 172.21.15.103:0/3782115001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b294151410 0x55b294159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b1d687740 1 --2- 172.21.15.103:0/3782115001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294150af0 0x55b294153560 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b1d687740 1 -- 172.21.15.103:0/3782115001 >> 172.21.15.103:0/3782115001 conn(0x55b29414d060 msgr2=0x55b29415ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b1d687740 1 -- 172.21.15.103:0/3782115001 shutdown_connections 2024-09-19T07:03:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b1d687740 1 -- 172.21.15.103:0/3782115001 wait complete. 2024-09-19T07:03:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b1d687740 1 -- start start 2024-09-19T07:03:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b1d687740 1 Processor -- start 2024-09-19T07:03:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b1d687740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b294150af0 0x55b294164140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b1d687740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294151410 0x55b29415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b1d687740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b29415bcb0 0x55b29415f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b1d687740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b141aed20 con 0x55b294151410 2024-09-19T07:03:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b1d687740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b141ae650 con 0x55b294150af0 2024-09-19T07:03:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b1d687740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b141ab8b0 con 0x55b29415bcb0 2024-09-19T07:03:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b1d687740 10 client.0 _ll_register_callbacks cb 0x7f0b0c007460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.407+0000 7f0b0bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294151410 0x55b29415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0b0bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294151410 0x55b29415f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52764/0 (socket says 172.21.15.103:52764) 2024-09-19T07:03:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0b0bfff640 1 -- 172.21.15.103:0/3527150530 learned_addr learned my addr 172.21.15.103:0/3527150530 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0b0bfff640 1 -- 172.21.15.103:0/3527150530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b29415bcb0 msgr2=0x55b29415f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0b18e81640 1 --2- 172.21.15.103:0/3527150530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b29415bcb0 0x55b29415f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0b0bfff640 1 --2- 172.21.15.103:0/3527150530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b29415bcb0 0x55b29415f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0b0bfff640 1 -- 172.21.15.103:0/3527150530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b294150af0 msgr2=0x55b294164140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:03:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0b0bfff640 1 --2- 172.21.15.103:0/3527150530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b294150af0 0x55b294164140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0b0bfff640 1 -- 172.21.15.103:0/3527150530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b29415ffb0 con 0x55b294151410 2024-09-19T07:03:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0b0bfff640 1 --2- 172.21.15.103:0/3527150530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294151410 0x55b29415f4b0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009900 tx=0x7f0afc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1028799961ed0519 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0af37fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0af37fe640 1 -- 172.21.15.103:0/3527150530 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc04c070 con 0x55b294151410 2024-09-19T07:03:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0af37fe640 1 -- 172.21.15.103:0/3527150530 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f0afc0037f0 con 0x55b294151410 2024-09-19T07:03:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0b1d687740 1 -- 172.21.15.103:0/3527150530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b04098100 con 0x55b294151410 2024-09-19T07:03:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0af37fe640 1 -- 172.21.15.103:0/3527150530 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0afc035040 con 0x55b294151410 2024-09-19T07:03:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0b1d687740 10 client.20466 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0b1d687740 1 -- 172.21.15.103:0/3527150530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b2941d6610 con 0x55b294151410 2024-09-19T07:03:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.408+0000 7f0af37fe640 1 -- 172.21.15.103:0/3527150530 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b04098100 con 0x55b294151410 2024-09-19T07:03:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.409+0000 7f0af37fe640 1 -- 172.21.15.103:0/3527150530 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0afc045050 con 0x55b294151410 2024-09-19T07:03:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.409+0000 7f0b1d687740 10 client.20466 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.409+0000 7f0b1d687740 10 client.20466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.409+0000 7f0b1d687740 20 client.20466 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.409+0000 7f0b1d687740 1 --2- 172.21.15.103:0/3527150530 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b2941d6df0 0x55b2941d9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.409+0000 7f0b1d687740 4 client.20466 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:03:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.409+0000 7f0b1d687740 1 -- 172.21.15.103:0/3527150530 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2940bcd70 con 0x55b2941d6df0 2024-09-19T07:03:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.412+0000 7f0b13fff640 1 --2- 172.21.15.103:0/3527150530 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b2941d6df0 0x55b2941d9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.412+0000 7f0b13fff640 1 --2- 172.21.15.103:0/3527150530 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b2941d6df0 0x55b2941d9210 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1599a6685e91eb84 server_cookie=36a0bd128b37da2e in_seq=0 out_seq=0 2024-09-19T07:03:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.412+0000 7f0af37fe640 10 client.20466 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.425+0000 7f0af37fe640 1 -- 172.21.15.103:0/3527150530 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b2940bcd70 con 0x55b2941d6df0 2024-09-19T07:03:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.425+0000 7f0af37fe640 10 client.20466 handle_command_reply: tid=0 2024-09-19T07:03:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.425+0000 7f0b1d687740 1 client.20466 shutdown 2024-09-19T07:03:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.425+0000 7f0b1d687740 1 -- 172.21.15.103:0/3527150530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294151410 msgr2=0x55b29415f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.425+0000 7f0b1d687740 1 --2- 172.21.15.103:0/3527150530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294151410 0x55b29415f4b0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009900 tx=0x7f0afc009260 comp rx=0 tx=0).stop 2024-09-19T07:03:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.425+0000 7f0b1d687740 1 -- 172.21.15.103:0/3527150530 shutdown_connections 2024-09-19T07:03:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.425+0000 7f0b1d687740 1 --2- 172.21.15.103:0/3527150530 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b2941d6df0 0x55b2941d9210 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.425+0000 7f0b1d687740 1 --2- 172.21.15.103:0/3527150530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b29415bcb0 0x55b29415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.425+0000 7f0b1d687740 1 --2- 172.21.15.103:0/3527150530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b294151410 0x55b29415f4b0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.425+0000 7f0b1d687740 1 --2- 172.21.15.103:0/3527150530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b294150af0 0x55b294164140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.425+0000 7f0af37fe640 0 client.20466 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.425+0000 7f0b1d687740 1 -- 172.21.15.103:0/3527150530 >> 172.21.15.103:0/3527150530 conn(0x55b29414d060 msgr2=0x55b29414c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.425+0000 7f0b1d687740 1 -- 172.21.15.103:0/3527150530 shutdown_connections 2024-09-19T07:03:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.425+0000 7f0b1d687740 1 -- 172.21.15.103:0/3527150530 wait complete. 2024-09-19T07:03:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.425+0000 7f0b1d687740 20 client.20466 trim_cache size 0 max 16384 2024-09-19T07:03:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:03:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:03:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:03:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:03:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.426+0000 7f0b1a8e4640 1 -- 172.21.15.103:0/4002271696 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b000a3460 msgr2=0x7f0b000a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.426+0000 7f0b1a8e4640 1 --2- 172.21.15.103:0/4002271696 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b000a3460 0x7f0b000a5920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0afc02a9a0 tx=0x7f0afc000bc0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.426+0000 7f0b1a8e4640 1 -- 172.21.15.103:0/4002271696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b14102ce0 msgr2=0x7f0b141982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.426+0000 7f0b1a8e4640 1 --2- 172.21.15.103:0/4002271696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b14102ce0 0x7f0b141982e0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400d950 tx=0x7f0b0400de10 comp rx=0 tx=0).stop 2024-09-19T07:03:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.426+0000 7f0b1a8e4640 1 -- 172.21.15.103:0/4002271696 shutdown_connections 2024-09-19T07:03:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.426+0000 7f0b1a8e4640 1 --2- 172.21.15.103:0/4002271696 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b000a3460 0x7f0b000a5920 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.426+0000 7f0b1a8e4640 1 --2- 172.21.15.103:0/4002271696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1410c0b0 0x7f0b1419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.426+0000 7f0b1a8e4640 1 --2- 172.21.15.103:0/4002271696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b14103600 0x7f0b14198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.426+0000 7f0b1a8e4640 1 --2- 172.21.15.103:0/4002271696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b14102ce0 0x7f0b141982e0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.426+0000 7f0b1a8e4640 1 -- 172.21.15.103:0/4002271696 >> 172.21.15.103:0/4002271696 conn(0x7f0b14077990 msgr2=0x7f0b140fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.426+0000 7f0b1a8e4640 1 -- 172.21.15.103:0/4002271696 shutdown_connections 2024-09-19T07:03:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:03:52.426+0000 7f0b1a8e4640 1 -- 172.21.15.103:0/4002271696 wait complete. 2024-09-19T07:03:52.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:52 smithi134 ceph-mon[35356]: pgmap v457: 97 pgs: 97 active+clean; 5.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-09-19T07:03:52.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.562+0000 7f732ba00640 1 -- 172.21.15.103:0/3045250093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7324102790 msgr2=0x7f73241067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.562+0000 7f732ba00640 1 --2- 172.21.15.103:0/3045250093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7324102790 0x7f73241067e0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f7320009900 tx=0x7f732002a410 comp rx=0 tx=0).stop 2024-09-19T07:03:52.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.562+0000 7f732ba00640 1 -- 172.21.15.103:0/3045250093 shutdown_connections 2024-09-19T07:03:52.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.562+0000 7f732ba00640 1 --2- 172.21.15.103:0/3045250093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7324106f10 0x7f732410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.562+0000 7f732ba00640 1 --2- 172.21.15.103:0/3045250093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7324102790 0x7f73241067e0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.562+0000 7f732ba00640 1 --2- 172.21.15.103:0/3045250093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7324101de0 0x7f73241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.562+0000 7f732ba00640 1 -- 172.21.15.103:0/3045250093 >> 172.21.15.103:0/3045250093 conn(0x7f73240fd8c0 msgr2=0x7f73240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:52.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.562+0000 7f732ba00640 1 -- 172.21.15.103:0/3045250093 shutdown_connections 2024-09-19T07:03:52.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.562+0000 7f732ba00640 1 -- 172.21.15.103:0/3045250093 wait complete. 2024-09-19T07:03:52.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f732ba00640 1 Processor -- start 2024-09-19T07:03:52.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f732ba00640 1 -- start start 2024-09-19T07:03:52.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f732ba00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7324101de0 0x7f7324198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f732ba00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7324102790 0x7f7324198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f732ba00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7324106f10 0x7f732419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f732ba00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7324111f90 con 0x7f7324102790 2024-09-19T07:03:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f732ba00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7324111e10 con 0x7f7324101de0 2024-09-19T07:03:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f732ba00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7324112110 con 0x7f7324106f10 2024-09-19T07:03:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f7328f9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7324102790 0x7f7324198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f7328f9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7324102790 0x7f7324198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52782/0 (socket says 172.21.15.103:52782) 2024-09-19T07:03:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f7328f9b640 1 -- 172.21.15.103:0/3475607871 learned_addr learned my addr 172.21.15.103:0/3475607871 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f732979c640 1 --2- 172.21.15.103:0/3475607871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7324101de0 0x7f7324198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f7329f9d640 1 --2- 172.21.15.103:0/3475607871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7324106f10 0x7f732419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f7328f9b640 1 -- 172.21.15.103:0/3475607871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7324106f10 msgr2=0x7f732419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f7328f9b640 1 --2- 172.21.15.103:0/3475607871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7324106f10 0x7f732419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f7328f9b640 1 -- 172.21.15.103:0/3475607871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7324101de0 msgr2=0x7f7324198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f7328f9b640 1 --2- 172.21.15.103:0/3475607871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7324101de0 0x7f7324198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f7328f9b640 1 -- 172.21.15.103:0/3475607871 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732419d4c0 con 0x7f7324102790 2024-09-19T07:03:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.563+0000 7f732979c640 1 --2- 172.21.15.103:0/3475607871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7324101de0 0x7f7324198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:52.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.564+0000 7f7328f9b640 1 --2- 172.21.15.103:0/3475607871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7324102790 0x7f7324198a50 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f7320005ab0 tx=0x7f73200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:52.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.564+0000 7f73167fc640 1 -- 172.21.15.103:0/3475607871 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732003f030 con 0x7f7324102790 2024-09-19T07:03:52.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.564+0000 7f732ba00640 1 -- 172.21.15.103:0/3475607871 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732419d750 con 0x7f7324102790 2024-09-19T07:03:52.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.564+0000 7f73167fc640 1 -- 172.21.15.103:0/3475607871 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7320003ce0 con 0x7f7324102790 2024-09-19T07:03:52.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.564+0000 7f73167fc640 1 -- 172.21.15.103:0/3475607871 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320035070 con 0x7f7324102790 2024-09-19T07:03:52.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.564+0000 7f732ba00640 1 -- 172.21.15.103:0/3475607871 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73241a5090 con 0x7f7324102790 2024-09-19T07:03:52.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.566+0000 7f73167fc640 1 -- 172.21.15.103:0/3475607871 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7320042030 con 0x7f7324102790 2024-09-19T07:03:52.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.566+0000 7f732ddd9740 1 Processor -- start 2024-09-19T07:03:52.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f732ddd9740 1 -- start start 2024-09-19T07:03:52.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f732ddd9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55758377cb40 0x55758377b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f732ddd9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377b5d0 0x55758377ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f732ddd9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557583786370 0x557583788760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f732ddd9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5575836e12f0 con 0x55758377cb40 2024-09-19T07:03:52.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f732ddd9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557583688cf0 con 0x55758377b5d0 2024-09-19T07:03:52.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f732ddd9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5575836dd2b0 con 0x557583786370 2024-09-19T07:03:52.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f73167fc640 1 --2- 172.21.15.103:0/3475607871 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f73100a33b0 0x7f73100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f7329f9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55758377cb40 0x55758377b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f7328f9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557583786370 0x557583788760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f7329f9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55758377cb40 0x55758377b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52786/0 (socket says 172.21.15.103:52786) 2024-09-19T07:03:52.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f7329f9d640 1 -- 172.21.15.103:0/1252618869 learned_addr learned my addr 172.21.15.103:0/1252618869 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:52.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f732979c640 1 --2- 172.21.15.103:0/1252618869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377b5d0 0x55758377ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f7329f9d640 1 --2- 172.21.15.103:0/3475607871 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f73100a33b0 0x7f73100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f73167fc640 1 -- 172.21.15.103:0/3475607871 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f73200f0740 con 0x7f7324102790 2024-09-19T07:03:52.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f7328f9b640 1 -- 172.21.15.103:0/1252618869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377b5d0 msgr2=0x55758377ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:52.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f7328f9b640 1 --2- 172.21.15.103:0/1252618869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377b5d0 0x55758377ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f7328f9b640 1 -- 172.21.15.103:0/1252618869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55758377cb40 msgr2=0x55758377b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:52.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f7328f9b640 1 --2- 172.21.15.103:0/1252618869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55758377cb40 0x55758377b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f7328f9b640 1 -- 172.21.15.103:0/1252618869 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55758366d3a0 con 0x557583786370 2024-09-19T07:03:52.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.567+0000 7f7329f9d640 1 --2- 172.21.15.103:0/1252618869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55758377cb40 0x55758377b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:03:52.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.568+0000 7f7328f9b640 1 --2- 172.21.15.103:0/1252618869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557583786370 0x557583788760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f73200ede10 tx=0x7f73200b1e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c43db83e4f2255b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:52.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.568+0000 7f7329f9d640 1 --2- 172.21.15.103:0/3475607871 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f73100a33b0 0x7f73100a5870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f731c00afd0 tx=0x7f731c00a970 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:52.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.568+0000 7f730f7fe640 1 -- 172.21.15.103:0/1252618869 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320049070 con 0x557583786370 2024-09-19T07:03:52.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.568+0000 7f730f7fe640 1 -- 172.21.15.103:0/1252618869 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73200bc9a0 con 0x557583786370 2024-09-19T07:03:52.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.568+0000 7f730f7fe640 1 -- 172.21.15.103:0/1252618869 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732003b460 con 0x557583786370 2024-09-19T07:03:52.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.569+0000 7f732ddd9740 1 -- 172.21.15.103:0/1252618869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557583786370 msgr2=0x557583788760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.569+0000 7f732ddd9740 1 --2- 172.21.15.103:0/1252618869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557583786370 0x557583788760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f73200ede10 tx=0x7f73200b1e30 comp rx=0 tx=0).stop 2024-09-19T07:03:52.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.569+0000 7f732ddd9740 1 -- 172.21.15.103:0/1252618869 shutdown_connections 2024-09-19T07:03:52.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.569+0000 7f732ddd9740 1 --2- 172.21.15.103:0/1252618869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557583786370 0x557583788760 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.569+0000 7f732ddd9740 1 --2- 172.21.15.103:0/1252618869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377b5d0 0x55758377ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.569+0000 7f732ddd9740 1 --2- 172.21.15.103:0/1252618869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55758377cb40 0x55758377b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.569+0000 7f732ddd9740 1 -- 172.21.15.103:0/1252618869 >> 172.21.15.103:0/1252618869 conn(0x557583744310 msgr2=0x557583777c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:52.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.569+0000 7f732ddd9740 1 -- 172.21.15.103:0/1252618869 shutdown_connections 2024-09-19T07:03:52.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.569+0000 7f732ddd9740 1 -- 172.21.15.103:0/1252618869 wait complete. 2024-09-19T07:03:52.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.569+0000 7f732ddd9740 1 -- start start 2024-09-19T07:03:52.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.569+0000 7f732ddd9740 1 Processor -- start 2024-09-19T07:03:52.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.569+0000 7f732ddd9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55758377b5d0 0x557583785dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f732ddd9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55758377cb40 0x557583783070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f732ddd9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557583786370 0x557583783800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f732ddd9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557583788e30 con 0x55758377cb40 2024-09-19T07:03:52.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f732ddd9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55758369c060 con 0x557583786370 2024-09-19T07:03:52.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f732ddd9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557583788fb0 con 0x55758377b5d0 2024-09-19T07:03:52.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f732ddd9740 10 client.0 _ll_register_callbacks cb 0x5575833ef8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:52.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f7328f9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55758377cb40 0x557583783070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f7328f9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55758377cb40 0x557583783070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52798/0 (socket says 172.21.15.103:52798) 2024-09-19T07:03:52.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f7328f9b640 1 -- 172.21.15.103:0/3517825685 learned_addr learned my addr 172.21.15.103:0/3517825685 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:52.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f732979c640 1 --2- 172.21.15.103:0/3517825685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55758377b5d0 0x557583785dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f7329f9d640 1 --2- 172.21.15.103:0/3517825685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557583786370 0x557583783800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f7328f9b640 1 -- 172.21.15.103:0/3517825685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55758377b5d0 msgr2=0x557583785dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:52.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f7328f9b640 1 --2- 172.21.15.103:0/3517825685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55758377b5d0 0x557583785dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f7328f9b640 1 -- 172.21.15.103:0/3517825685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557583786370 msgr2=0x557583783800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:52.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f7328f9b640 1 --2- 172.21.15.103:0/3517825685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557583786370 0x557583783800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f7328f9b640 1 -- 172.21.15.103:0/3517825685 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575834523c0 con 0x55758377cb40 2024-09-19T07:03:52.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f7329f9d640 1 --2- 172.21.15.103:0/3517825685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557583786370 0x557583783800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:52.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f7328f9b640 1 --2- 172.21.15.103:0/3517825685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55758377cb40 0x557583783070 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f7320003fd0 tx=0x7f7320004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdecb03d04a7bde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:52.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f730cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:52.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f730cff9640 1 -- 172.21.15.103:0/3517825685 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320049050 con 0x55758377cb40 2024-09-19T07:03:52.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f730cff9640 1 -- 172.21.15.103:0/3517825685 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f732003b460 con 0x55758377cb40 2024-09-19T07:03:52.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f732ddd9740 1 -- 172.21.15.103:0/3517825685 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73200bc9a0 con 0x55758377cb40 2024-09-19T07:03:52.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f730cff9640 1 -- 172.21.15.103:0/3517825685 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f73200c5610 con 0x55758377cb40 2024-09-19T07:03:52.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f732ddd9740 10 client.20478 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:52.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.570+0000 7f732ddd9740 1 -- 172.21.15.103:0/3517825685 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557583788e30 con 0x55758377cb40 2024-09-19T07:03:52.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.571+0000 7f730cff9640 1 -- 172.21.15.103:0/3517825685 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73200bc9a0 con 0x55758377cb40 2024-09-19T07:03:52.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.571+0000 7f730cff9640 1 -- 172.21.15.103:0/3517825685 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f73200c5180 con 0x55758377cb40 2024-09-19T07:03:52.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.571+0000 7f732ddd9740 10 client.20478 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:52.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.571+0000 7f732ddd9740 10 client.20478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:52.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.571+0000 7f732ddd9740 20 client.20478 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:52.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.571+0000 7f732ddd9740 1 --2- 172.21.15.103:0/3517825685 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557583795a70 0x557583797e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.571+0000 7f732ddd9740 4 client.20478 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:52.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.571+0000 7f732ddd9740 1 -- 172.21.15.103:0/3517825685 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575836ee860 con 0x557583795a70 2024-09-19T07:03:52.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.571+0000 7f732979c640 1 --2- 172.21.15.103:0/3517825685 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557583795a70 0x557583797e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.571+0000 7f730cff9640 10 client.20478 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:52.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.571+0000 7f732979c640 1 --2- 172.21.15.103:0/3517825685 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557583795a70 0x557583797e90 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3fb14c562388bee server_cookie=af80d0e42623cf1e in_seq=0 out_seq=0 2024-09-19T07:03:52.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.572+0000 7f730cff9640 1 -- 172.21.15.103:0/3517825685 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575836ee860 con 0x557583795a70 2024-09-19T07:03:52.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.572+0000 7f730cff9640 10 client.20478 handle_command_reply: tid=0 2024-09-19T07:03:52.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.572+0000 7f732ddd9740 1 client.20478 shutdown 2024-09-19T07:03:52.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.572+0000 7f732ddd9740 1 -- 172.21.15.103:0/3517825685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55758377cb40 msgr2=0x557583783070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.572+0000 7f732ddd9740 1 --2- 172.21.15.103:0/3517825685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55758377cb40 0x557583783070 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f7320003fd0 tx=0x7f7320004540 comp rx=0 tx=0).stop 2024-09-19T07:03:52.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.572+0000 7f732ddd9740 1 -- 172.21.15.103:0/3517825685 shutdown_connections 2024-09-19T07:03:52.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.572+0000 7f732ddd9740 1 --2- 172.21.15.103:0/3517825685 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557583795a70 0x557583797e90 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.572+0000 7f732ddd9740 1 --2- 172.21.15.103:0/3517825685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557583786370 0x557583783800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.572+0000 7f732ddd9740 1 --2- 172.21.15.103:0/3517825685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55758377cb40 0x557583783070 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.572+0000 7f732ddd9740 1 --2- 172.21.15.103:0/3517825685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55758377b5d0 0x557583785dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.572+0000 7f732ddd9740 1 -- 172.21.15.103:0/3517825685 >> 172.21.15.103:0/3517825685 conn(0x557583744310 msgr2=0x55758377d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:52.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.572+0000 7f730cff9640 0 client.20478 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:52.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.572+0000 7f732ddd9740 1 -- 172.21.15.103:0/3517825685 shutdown_connections 2024-09-19T07:03:52.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.572+0000 7f732ddd9740 1 -- 172.21.15.103:0/3517825685 wait complete. 2024-09-19T07:03:52.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.573+0000 7f732ddd9740 20 client.20478 trim_cache size 0 max 16384 2024-09-19T07:03:52.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.585+0000 7f732ddd9740 1 Processor -- start 2024-09-19T07:03:52.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.585+0000 7f732ddd9740 1 -- start start 2024-09-19T07:03:52.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f732ddd9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55758377b5d0 0x55758378e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f732ddd9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377cb40 0x55758378ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f732ddd9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557583786370 0x55758377e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f732ddd9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f73241ad8c0 con 0x557583786370 2024-09-19T07:03:52.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f732ddd9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f73241af400 con 0x55758377cb40 2024-09-19T07:03:52.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f732ddd9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f73241adf90 con 0x55758377b5d0 2024-09-19T07:03:52.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f7329f9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557583786370 0x55758377e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f7329f9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557583786370 0x55758377e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52806/0 (socket says 172.21.15.103:52806) 2024-09-19T07:03:52.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f7329f9d640 1 -- 172.21.15.103:0/205183247 learned_addr learned my addr 172.21.15.103:0/205183247 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:52.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f7328f9b640 1 --2- 172.21.15.103:0/205183247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377cb40 0x55758378ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f732979c640 1 --2- 172.21.15.103:0/205183247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55758377b5d0 0x55758378e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f7329f9d640 1 -- 172.21.15.103:0/205183247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55758377b5d0 msgr2=0x55758378e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:52.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f7329f9d640 1 --2- 172.21.15.103:0/205183247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55758377b5d0 0x55758378e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f7329f9d640 1 -- 172.21.15.103:0/205183247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377cb40 msgr2=0x55758378ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:52.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f7329f9d640 1 --2- 172.21.15.103:0/205183247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377cb40 0x55758378ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f7329f9d640 1 -- 172.21.15.103:0/205183247 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557583786130 con 0x557583786370 2024-09-19T07:03:52.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f732979c640 1 --2- 172.21.15.103:0/205183247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55758377b5d0 0x55758378e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:52.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f7328f9b640 1 --2- 172.21.15.103:0/205183247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377cb40 0x55758378ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:03:52.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f7329f9d640 1 --2- 172.21.15.103:0/205183247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557583786370 0x55758377e120 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f731c005cc0 tx=0x7f731c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce4c7a1cc4cbbd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:52.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f730effd640 1 -- 172.21.15.103:0/205183247 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c004d20 con 0x557583786370 2024-09-19T07:03:52.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f730effd640 1 -- 172.21.15.103:0/205183247 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f731c003e50 con 0x557583786370 2024-09-19T07:03:52.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.586+0000 7f730effd640 1 -- 172.21.15.103:0/205183247 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c004350 con 0x557583786370 2024-09-19T07:03:52.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.587+0000 7f732ddd9740 1 -- 172.21.15.103:0/205183247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557583786370 msgr2=0x55758377e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.587+0000 7f732ddd9740 1 --2- 172.21.15.103:0/205183247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557583786370 0x55758377e120 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f731c005cc0 tx=0x7f731c01e000 comp rx=0 tx=0).stop 2024-09-19T07:03:52.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.587+0000 7f732ddd9740 1 -- 172.21.15.103:0/205183247 shutdown_connections 2024-09-19T07:03:52.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.587+0000 7f732ddd9740 1 --2- 172.21.15.103:0/205183247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557583786370 0x55758377e120 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.587+0000 7f732ddd9740 1 --2- 172.21.15.103:0/205183247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377cb40 0x55758378ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.587+0000 7f732ddd9740 1 --2- 172.21.15.103:0/205183247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55758377b5d0 0x55758378e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.587+0000 7f732ddd9740 1 -- 172.21.15.103:0/205183247 >> 172.21.15.103:0/205183247 conn(0x557583744310 msgr2=0x5575837850f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:52.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.587+0000 7f732ddd9740 1 -- 172.21.15.103:0/205183247 shutdown_connections 2024-09-19T07:03:52.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.587+0000 7f732ddd9740 1 -- 172.21.15.103:0/205183247 wait complete. 2024-09-19T07:03:52.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.587+0000 7f732ddd9740 1 -- start start 2024-09-19T07:03:52.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f732ddd9740 1 Processor -- start 2024-09-19T07:03:52.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f732ddd9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377b5d0 0x5575837802f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f732ddd9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55758377cb40 0x557583780830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f732ddd9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557583786370 0x557583780d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f732ddd9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f73241aed30 con 0x557583786370 2024-09-19T07:03:52.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f732ddd9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f73241ae660 con 0x55758377b5d0 2024-09-19T07:03:52.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f732ddd9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f73241ab8c0 con 0x55758377cb40 2024-09-19T07:03:52.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f732ddd9740 10 client.0 _ll_register_callbacks cb 0x7f731c00ae00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:52.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f7329f9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557583786370 0x557583780d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f7329f9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557583786370 0x557583780d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52822/0 (socket says 172.21.15.103:52822) 2024-09-19T07:03:52.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f7329f9d640 1 -- 172.21.15.103:0/4037233994 learned_addr learned my addr 172.21.15.103:0/4037233994 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:52.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f7328f9b640 1 --2- 172.21.15.103:0/4037233994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55758377cb40 0x557583780830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f732979c640 1 --2- 172.21.15.103:0/4037233994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377b5d0 0x5575837802f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f7329f9d640 1 -- 172.21.15.103:0/4037233994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55758377cb40 msgr2=0x557583780830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:52.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f7329f9d640 1 --2- 172.21.15.103:0/4037233994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55758377cb40 0x557583780830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f7329f9d640 1 -- 172.21.15.103:0/4037233994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377b5d0 msgr2=0x5575837802f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:52.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f7329f9d640 1 --2- 172.21.15.103:0/4037233994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377b5d0 0x5575837802f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f7329f9d640 1 -- 172.21.15.103:0/4037233994 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557583776780 con 0x557583786370 2024-09-19T07:03:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.588+0000 7f732979c640 1 --2- 172.21.15.103:0/4037233994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377b5d0 0x5575837802f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.589+0000 7f7329f9d640 1 --2- 172.21.15.103:0/4037233994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557583786370 0x557583780d70 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f731c01fb50 tx=0x7f731c01fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80bc91e9b5577e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.589+0000 7f730cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.589+0000 7f730cff9640 1 -- 172.21.15.103:0/4037233994 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c038060 con 0x557583786370 2024-09-19T07:03:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.589+0000 7f730cff9640 1 -- 172.21.15.103:0/4037233994 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f731c0137a0 con 0x557583786370 2024-09-19T07:03:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.589+0000 7f732ddd9740 1 -- 172.21.15.103:0/4037233994 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f731c003e50 con 0x557583786370 2024-09-19T07:03:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.589+0000 7f730cff9640 1 -- 172.21.15.103:0/4037233994 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f731c00a530 con 0x557583786370 2024-09-19T07:03:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.589+0000 7f732ddd9740 10 client.20484 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.589+0000 7f732ddd9740 1 -- 172.21.15.103:0/4037233994 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557583777a80 con 0x557583786370 2024-09-19T07:03:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.589+0000 7f730cff9640 1 -- 172.21.15.103:0/4037233994 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f731c003e50 con 0x557583786370 2024-09-19T07:03:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.589+0000 7f730cff9640 1 -- 172.21.15.103:0/4037233994 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f731c01c070 con 0x557583786370 2024-09-19T07:03:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.589+0000 7f732ddd9740 10 client.20484 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.589+0000 7f732ddd9740 10 client.20484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:52.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.589+0000 7f732ddd9740 20 client.20484 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.590+0000 7f732ddd9740 1 --2- 172.21.15.103:0/4037233994 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557583805770 0x557583807b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.590+0000 7f732ddd9740 4 client.20484 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:03:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.590+0000 7f732ddd9740 1 -- 172.21.15.103:0/4037233994 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575836e8d70 con 0x557583805770 2024-09-19T07:03:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.590+0000 7f732979c640 1 --2- 172.21.15.103:0/4037233994 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557583805770 0x557583807b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.590+0000 7f732979c640 1 --2- 172.21.15.103:0/4037233994 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557583805770 0x557583807b60 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ead1a20c2688be78 server_cookie=ab01687b26e1e127 in_seq=0 out_seq=0 2024-09-19T07:03:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.590+0000 7f730cff9640 10 client.20484 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.590+0000 7f730cff9640 1 -- 172.21.15.103:0/4037233994 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575836e8d70 con 0x557583805770 2024-09-19T07:03:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.590+0000 7f730cff9640 10 client.20484 handle_command_reply: tid=0 2024-09-19T07:03:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.590+0000 7f732ddd9740 1 client.20484 shutdown 2024-09-19T07:03:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ddd9740 1 -- 172.21.15.103:0/4037233994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557583786370 msgr2=0x557583780d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ddd9740 1 --2- 172.21.15.103:0/4037233994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557583786370 0x557583780d70 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f731c01fb50 tx=0x7f731c01fb80 comp rx=0 tx=0).stop 2024-09-19T07:03:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ddd9740 1 -- 172.21.15.103:0/4037233994 shutdown_connections 2024-09-19T07:03:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ddd9740 1 --2- 172.21.15.103:0/4037233994 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557583805770 0x557583807b60 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ddd9740 1 --2- 172.21.15.103:0/4037233994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557583786370 0x557583780d70 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ddd9740 1 --2- 172.21.15.103:0/4037233994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55758377cb40 0x557583780830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ddd9740 1 --2- 172.21.15.103:0/4037233994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55758377b5d0 0x5575837802f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ddd9740 1 -- 172.21.15.103:0/4037233994 >> 172.21.15.103:0/4037233994 conn(0x557583744310 msgr2=0x557583781810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:52.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f730cff9640 0 client.20484 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:52.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ddd9740 1 -- 172.21.15.103:0/4037233994 shutdown_connections 2024-09-19T07:03:52.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ddd9740 1 -- 172.21.15.103:0/4037233994 wait complete. 2024-09-19T07:03:52.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ddd9740 20 client.20484 trim_cache size 0 max 16384 2024-09-19T07:03:52.604 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:03:52.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ba00640 1 -- 172.21.15.103:0/3475607871 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f73100a33b0 msgr2=0x7f73100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ba00640 1 --2- 172.21.15.103:0/3475607871 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f73100a33b0 0x7f73100a5870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f731c00afd0 tx=0x7f731c00a970 comp rx=0 tx=0).stop 2024-09-19T07:03:52.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ba00640 1 -- 172.21.15.103:0/3475607871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7324102790 msgr2=0x7f7324198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:52.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ba00640 1 --2- 172.21.15.103:0/3475607871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7324102790 0x7f7324198a50 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f7320005ab0 tx=0x7f73200042f0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ba00640 1 -- 172.21.15.103:0/3475607871 shutdown_connections 2024-09-19T07:03:52.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ba00640 1 --2- 172.21.15.103:0/3475607871 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f73100a33b0 0x7f73100a5870 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ba00640 1 --2- 172.21.15.103:0/3475607871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7324106f10 0x7f732419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ba00640 1 --2- 172.21.15.103:0/3475607871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7324102790 0x7f7324198a50 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ba00640 1 --2- 172.21.15.103:0/3475607871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7324101de0 0x7f7324198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:52.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.591+0000 7f732ba00640 1 -- 172.21.15.103:0/3475607871 >> 172.21.15.103:0/3475607871 conn(0x7f73240fd8c0 msgr2=0x7f73240fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:52.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.592+0000 7f732ba00640 1 -- 172.21.15.103:0/3475607871 shutdown_connections 2024-09-19T07:03:52.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:52.592+0000 7f732ba00640 1 -- 172.21.15.103:0/3475607871 wait complete. 2024-09-19T07:03:54.318 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:03:54.318 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:03:54.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:54 smithi174 ceph-mon[35637]: pgmap v458: 97 pgs: 97 active+clean; 5.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-19T07:03:54.346 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:54 smithi103 ceph-mon[31158]: pgmap v458: 97 pgs: 97 active+clean; 5.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-19T07:03:54.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:54 smithi134 ceph-mon[35356]: pgmap v458: 97 pgs: 97 active+clean; 5.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-19T07:03:54.584 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:55.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.094+0000 7f8d970aa640 1 -- 172.21.15.103:0/1756619003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d90102d40 msgr2=0x7f8d90103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:55.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.094+0000 7f8d970aa640 1 --2- 172.21.15.103:0/1756619003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d90102d40 0x7f8d90103120 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009900 tx=0x7f8d8402a480 comp rx=0 tx=0).stop 2024-09-19T07:03:55.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.094+0000 7f8d970aa640 1 -- 172.21.15.103:0/1756619003 shutdown_connections 2024-09-19T07:03:55.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.094+0000 7f8d970aa640 1 --2- 172.21.15.103:0/1756619003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d9010c170 0x7f8d9010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.094+0000 7f8d970aa640 1 --2- 172.21.15.103:0/1756619003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d90103660 0x7f8d9010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.094+0000 7f8d970aa640 1 --2- 172.21.15.103:0/1756619003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d90102d40 0x7f8d90103120 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.094+0000 7f8d970aa640 1 -- 172.21.15.103:0/1756619003 >> 172.21.15.103:0/1756619003 conn(0x7f8d900fc7a0 msgr2=0x7f8d900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:55.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.094+0000 7f8d970aa640 1 -- 172.21.15.103:0/1756619003 shutdown_connections 2024-09-19T07:03:55.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.094+0000 7f8d970aa640 1 -- 172.21.15.103:0/1756619003 wait complete. 2024-09-19T07:03:55.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d970aa640 1 Processor -- start 2024-09-19T07:03:55.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d970aa640 1 -- start start 2024-09-19T07:03:55.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d970aa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d90102d40 0x7f8d90198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d970aa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d90103660 0x7f8d901989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d970aa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d9010c170 0x7f8d9019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d970aa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d90111fb0 con 0x7f8d90102d40 2024-09-19T07:03:55.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d970aa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d90111e30 con 0x7f8d9010c170 2024-09-19T07:03:55.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d970aa640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d90112130 con 0x7f8d90103660 2024-09-19T07:03:55.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d94e46640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d90102d40 0x7f8d90198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:55.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d94e46640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d90102d40 0x7f8d90198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52836/0 (socket says 172.21.15.103:52836) 2024-09-19T07:03:55.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d94e46640 1 -- 172.21.15.103:0/2485193775 learned_addr learned my addr 172.21.15.103:0/2485193775 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:55.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d8ffff640 1 --2- 172.21.15.103:0/2485193775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d90103660 0x7f8d901989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:55.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d94e46640 1 -- 172.21.15.103:0/2485193775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d90103660 msgr2=0x7f8d901989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:55.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d94e46640 1 --2- 172.21.15.103:0/2485193775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d90103660 0x7f8d901989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d95647640 1 --2- 172.21.15.103:0/2485193775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d9010c170 0x7f8d9019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:55.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d94e46640 1 -- 172.21.15.103:0/2485193775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d9010c170 msgr2=0x7f8d9019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:55.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d94e46640 1 --2- 172.21.15.103:0/2485193775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d9010c170 0x7f8d9019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d94e46640 1 -- 172.21.15.103:0/2485193775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9019d440 con 0x7f8d90102d40 2024-09-19T07:03:55.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d8ffff640 1 --2- 172.21.15.103:0/2485193775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d90103660 0x7f8d901989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:55.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.095+0000 7f8d95647640 1 --2- 172.21.15.103:0/2485193775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d9010c170 0x7f8d9019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:55.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.096+0000 7f8d94e46640 1 --2- 172.21.15.103:0/2485193775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d90102d40 0x7f8d90198490 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f8d84005ab0 tx=0x7f8d840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:55.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.096+0000 7f8d8dffb640 1 -- 172.21.15.103:0/2485193775 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8403f030 con 0x7f8d90102d40 2024-09-19T07:03:55.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.096+0000 7f8d970aa640 1 -- 172.21.15.103:0/2485193775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d9019d6d0 con 0x7f8d90102d40 2024-09-19T07:03:55.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.096+0000 7f8d8dffb640 1 -- 172.21.15.103:0/2485193775 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d84003ce0 con 0x7f8d90102d40 2024-09-19T07:03:55.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.096+0000 7f8d8dffb640 1 -- 172.21.15.103:0/2485193775 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84035070 con 0x7f8d90102d40 2024-09-19T07:03:55.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.096+0000 7f8d970aa640 1 -- 172.21.15.103:0/2485193775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d901a5010 con 0x7f8d90102d40 2024-09-19T07:03:55.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.099+0000 7f8d8dffb640 1 -- 172.21.15.103:0/2485193775 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8d84042030 con 0x7f8d90102d40 2024-09-19T07:03:55.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.099+0000 7f8d99483740 1 Processor -- start 2024-09-19T07:03:55.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.099+0000 7f8d99483740 1 -- start start 2024-09-19T07:03:55.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.099+0000 7f8d99483740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e57b60 0x558c40e57f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.099+0000 7f8d99483740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e58480 0x558c40e62800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.099+0000 7f8d99483740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e62d40 0x558c40e65130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.099+0000 7f8d99483740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558c40dbd280 con 0x558c40e62d40 2024-09-19T07:03:55.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.099+0000 7f8d99483740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558c40d64b10 con 0x558c40e57b60 2024-09-19T07:03:55.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.099+0000 7f8d99483740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558c40db9240 con 0x558c40e58480 2024-09-19T07:03:55.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.099+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e62d40 0x558c40e65130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:55.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.099+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e62d40 0x558c40e65130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52844/0 (socket says 172.21.15.103:52844) 2024-09-19T07:03:55.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.099+0000 7f8d8ffff640 1 -- 172.21.15.103:0/783234844 learned_addr learned my addr 172.21.15.103:0/783234844 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:55.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.099+0000 7f8d8dffb640 1 --2- 172.21.15.103:0/2485193775 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8d800a33b0 0x7f8d800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.099+0000 7f8d94e46640 1 --2- 172.21.15.103:0/783234844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e58480 0x558c40e62800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:55.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.100+0000 7f8d95647640 1 --2- 172.21.15.103:0/783234844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e57b60 0x558c40e57f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:55.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.100+0000 7f8d95647640 1 --2- 172.21.15.103:0/2485193775 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8d800a33b0 0x7f8d800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:55.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.100+0000 7f8d8ffff640 1 -- 172.21.15.103:0/783234844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e58480 msgr2=0x558c40e62800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:55.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.100+0000 7f8d8ffff640 1 --2- 172.21.15.103:0/783234844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e58480 0x558c40e62800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.100+0000 7f8d8ffff640 1 -- 172.21.15.103:0/783234844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e57b60 msgr2=0x558c40e57f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:55.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.100+0000 7f8d8ffff640 1 --2- 172.21.15.103:0/783234844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e57b60 0x558c40e57f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.100+0000 7f8d8ffff640 1 -- 172.21.15.103:0/783234844 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c40d711f0 con 0x558c40e62d40 2024-09-19T07:03:55.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.100+0000 7f8d8dffb640 1 -- 172.21.15.103:0/2485193775 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f8d840f06c0 con 0x7f8d90102d40 2024-09-19T07:03:55.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.100+0000 7f8d8ffff640 1 --2- 172.21.15.103:0/783234844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e62d40 0x558c40e65130 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00bb30 tx=0x7f8d7c00d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb2bdc180c636c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:55.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.100+0000 7f8d7affd640 1 -- 172.21.15.103:0/783234844 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c017420 con 0x558c40e62d40 2024-09-19T07:03:55.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.100+0000 7f8d95647640 1 --2- 172.21.15.103:0/2485193775 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8d800a33b0 0x7f8d800a5870 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800e5d0 tx=0x7f8d88008220 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:55.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.100+0000 7f8d7affd640 1 -- 172.21.15.103:0/783234844 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d7c017aa0 con 0x558c40e62d40 2024-09-19T07:03:55.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.100+0000 7f8d7affd640 1 -- 172.21.15.103:0/783234844 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c00fc10 con 0x558c40e62d40 2024-09-19T07:03:55.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.101+0000 7f8d99483740 1 -- 172.21.15.103:0/783234844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e62d40 msgr2=0x558c40e65130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:55.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.101+0000 7f8d99483740 1 --2- 172.21.15.103:0/783234844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e62d40 0x558c40e65130 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00bb30 tx=0x7f8d7c00d340 comp rx=0 tx=0).stop 2024-09-19T07:03:55.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.101+0000 7f8d99483740 1 -- 172.21.15.103:0/783234844 shutdown_connections 2024-09-19T07:03:55.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.101+0000 7f8d99483740 1 --2- 172.21.15.103:0/783234844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e62d40 0x558c40e65130 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.101+0000 7f8d99483740 1 --2- 172.21.15.103:0/783234844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e58480 0x558c40e62800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.101+0000 7f8d99483740 1 --2- 172.21.15.103:0/783234844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e57b60 0x558c40e57f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.101+0000 7f8d99483740 1 -- 172.21.15.103:0/783234844 >> 172.21.15.103:0/783234844 conn(0x558c40e540f0 msgr2=0x558c40e544d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:55.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.101+0000 7f8d99483740 1 -- 172.21.15.103:0/783234844 shutdown_connections 2024-09-19T07:03:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.101+0000 7f8d99483740 1 -- 172.21.15.103:0/783234844 wait complete. 2024-09-19T07:03:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.101+0000 7f8d99483740 1 -- start start 2024-09-19T07:03:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.101+0000 7f8d99483740 1 Processor -- start 2024-09-19T07:03:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d99483740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e57b60 0x558c40e5fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d99483740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e60130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d99483740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e62d40 0x558c40e60670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d99483740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558c40e65910 con 0x558c40e58480 2024-09-19T07:03:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d99483740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558c40d77f10 con 0x558c40e62d40 2024-09-19T07:03:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d99483740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558c40e65a90 con 0x558c40e57b60 2024-09-19T07:03:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d99483740 10 client.0 _ll_register_callbacks cb 0x558c40d40a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e60130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e60130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52854/0 (socket says 172.21.15.103:52854) 2024-09-19T07:03:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d8ffff640 1 -- 172.21.15.103:0/737312838 learned_addr learned my addr 172.21.15.103:0/737312838 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d8ffff640 1 -- 172.21.15.103:0/737312838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e57b60 msgr2=0x558c40e5fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d8ffff640 1 --2- 172.21.15.103:0/737312838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e57b60 0x558c40e5fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d8ffff640 1 -- 172.21.15.103:0/737312838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e62d40 msgr2=0x558c40e60670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d8ffff640 1 --2- 172.21.15.103:0/737312838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e62d40 0x558c40e60670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d8ffff640 1 -- 172.21.15.103:0/737312838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c40b2e6a0 con 0x558c40e58480 2024-09-19T07:03:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d8ffff640 1 --2- 172.21.15.103:0/737312838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e60130 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00f830 tx=0x7f8d7c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4519d1e6336236 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d6ffff640 1 -- 172.21.15.103:0/737312838 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c026020 con 0x558c40e58480 2024-09-19T07:03:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d6ffff640 1 -- 172.21.15.103:0/737312838 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f8d7c00dd80 con 0x558c40e58480 2024-09-19T07:03:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.102+0000 7f8d99483740 1 -- 172.21.15.103:0/737312838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d7c017aa0 con 0x558c40e58480 2024-09-19T07:03:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.103+0000 7f8d6ffff640 1 -- 172.21.15.103:0/737312838 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d7c031050 con 0x558c40e58480 2024-09-19T07:03:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.103+0000 7f8d99483740 10 client.20496 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.103+0000 7f8d99483740 1 -- 172.21.15.103:0/737312838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558c40e714e0 con 0x558c40e58480 2024-09-19T07:03:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.103+0000 7f8d6ffff640 1 -- 172.21.15.103:0/737312838 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d7c017aa0 con 0x558c40e58480 2024-09-19T07:03:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.103+0000 7f8d6ffff640 1 -- 172.21.15.103:0/737312838 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8d7c021b90 con 0x558c40e58480 2024-09-19T07:03:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.103+0000 7f8d99483740 10 client.20496 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.103+0000 7f8d99483740 10 client.20496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.103+0000 7f8d99483740 20 client.20496 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.103+0000 7f8d99483740 1 --2- 172.21.15.103:0/737312838 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558c40e72940 0x558c40e74d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.103+0000 7f8d99483740 4 client.20496 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.103+0000 7f8d99483740 1 -- 172.21.15.103:0/737312838 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c40dca7f0 con 0x558c40e72940 2024-09-19T07:03:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.103+0000 7f8d94e46640 1 --2- 172.21.15.103:0/737312838 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558c40e72940 0x558c40e74d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.104+0000 7f8d94e46640 1 --2- 172.21.15.103:0/737312838 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558c40e72940 0x558c40e74d30 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be86f7352386a697 server_cookie=fa6658eec11bd512 in_seq=0 out_seq=0 2024-09-19T07:03:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.104+0000 7f8d6ffff640 10 client.20496 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.104+0000 7f8d6ffff640 1 -- 172.21.15.103:0/737312838 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c40dca7f0 con 0x558c40e72940 2024-09-19T07:03:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.104+0000 7f8d6ffff640 10 client.20496 handle_command_reply: tid=0 2024-09-19T07:03:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.104+0000 7f8d99483740 1 client.20496 shutdown 2024-09-19T07:03:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.105+0000 7f8d99483740 1 -- 172.21.15.103:0/737312838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 msgr2=0x558c40e60130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.105+0000 7f8d99483740 1 --2- 172.21.15.103:0/737312838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e60130 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00f830 tx=0x7f8d7c00b870 comp rx=0 tx=0).stop 2024-09-19T07:03:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.105+0000 7f8d99483740 1 -- 172.21.15.103:0/737312838 shutdown_connections 2024-09-19T07:03:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.105+0000 7f8d99483740 1 --2- 172.21.15.103:0/737312838 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558c40e72940 0x558c40e74d30 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.105+0000 7f8d99483740 1 --2- 172.21.15.103:0/737312838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e62d40 0x558c40e60670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.105+0000 7f8d99483740 1 --2- 172.21.15.103:0/737312838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e60130 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.105+0000 7f8d6ffff640 0 client.20496 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.105+0000 7f8d99483740 1 --2- 172.21.15.103:0/737312838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e57b60 0x558c40e5fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.105+0000 7f8d99483740 1 -- 172.21.15.103:0/737312838 >> 172.21.15.103:0/737312838 conn(0x558c40e540f0 msgr2=0x558c40e59dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.105+0000 7f8d99483740 1 -- 172.21.15.103:0/737312838 shutdown_connections 2024-09-19T07:03:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.105+0000 7f8d99483740 1 -- 172.21.15.103:0/737312838 wait complete. 2024-09-19T07:03:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.105+0000 7f8d99483740 20 client.20496 trim_cache size 0 max 16384 2024-09-19T07:03:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d99483740 1 Processor -- start 2024-09-19T07:03:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d99483740 1 -- start start 2024-09-19T07:03:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d99483740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e57b60 0x558c40e5eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d99483740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e7a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d99483740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e62d40 0x558c40e7a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d99483740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901ad970 con 0x558c40e58480 2024-09-19T07:03:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d99483740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901af4b0 con 0x558c40e57b60 2024-09-19T07:03:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d99483740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901ae040 con 0x558c40e62d40 2024-09-19T07:03:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e7a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e7a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52870/0 (socket says 172.21.15.103:52870) 2024-09-19T07:03:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d8ffff640 1 -- 172.21.15.103:0/313425416 learned_addr learned my addr 172.21.15.103:0/313425416 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d8ffff640 1 -- 172.21.15.103:0/313425416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e62d40 msgr2=0x558c40e7a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d8ffff640 1 --2- 172.21.15.103:0/313425416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e62d40 0x558c40e7a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d94e46640 1 --2- 172.21.15.103:0/313425416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e57b60 0x558c40e5eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:55.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d8ffff640 1 -- 172.21.15.103:0/313425416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e57b60 msgr2=0x558c40e5eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:55.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d8ffff640 1 --2- 172.21.15.103:0/313425416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e57b60 0x558c40e5eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.122+0000 7f8d8ffff640 1 -- 172.21.15.103:0/313425416 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c40e53f20 con 0x558c40e58480 2024-09-19T07:03:55.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.123+0000 7f8d8ffff640 1 --2- 172.21.15.103:0/313425416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e7a3e0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c021f60 tx=0x7f8d7c022570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1c4de0c80a6b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:55.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.123+0000 7f8d7a7fc640 1 -- 172.21.15.103:0/313425416 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c026070 con 0x558c40e58480 2024-09-19T07:03:55.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.123+0000 7f8d7a7fc640 1 -- 172.21.15.103:0/313425416 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d7c017aa0 con 0x558c40e58480 2024-09-19T07:03:55.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.123+0000 7f8d7a7fc640 1 -- 172.21.15.103:0/313425416 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c021450 con 0x558c40e58480 2024-09-19T07:03:55.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.123+0000 7f8d99483740 1 -- 172.21.15.103:0/313425416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 msgr2=0x558c40e7a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:55.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.123+0000 7f8d99483740 1 --2- 172.21.15.103:0/313425416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e7a3e0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c021f60 tx=0x7f8d7c022570 comp rx=0 tx=0).stop 2024-09-19T07:03:55.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.123+0000 7f8d99483740 1 -- 172.21.15.103:0/313425416 shutdown_connections 2024-09-19T07:03:55.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.123+0000 7f8d99483740 1 --2- 172.21.15.103:0/313425416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e62d40 0x558c40e7a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.123+0000 7f8d99483740 1 --2- 172.21.15.103:0/313425416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e7a3e0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.123+0000 7f8d99483740 1 --2- 172.21.15.103:0/313425416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e57b60 0x558c40e5eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.123+0000 7f8d99483740 1 -- 172.21.15.103:0/313425416 >> 172.21.15.103:0/313425416 conn(0x558c40e540f0 msgr2=0x558c40e5e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.123+0000 7f8d99483740 1 -- 172.21.15.103:0/313425416 shutdown_connections 2024-09-19T07:03:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.123+0000 7f8d99483740 1 -- 172.21.15.103:0/313425416 wait complete. 2024-09-19T07:03:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.124+0000 7f8d99483740 1 -- start start 2024-09-19T07:03:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.124+0000 7f8d99483740 1 Processor -- start 2024-09-19T07:03:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.124+0000 7f8d99483740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e57b60 0x558c40e675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.124+0000 7f8d99483740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e67ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.124+0000 7f8d99483740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e62d40 0x558c40ed5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.124+0000 7f8d99483740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901aede0 con 0x558c40e58480 2024-09-19T07:03:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.124+0000 7f8d99483740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901ae710 con 0x558c40e62d40 2024-09-19T07:03:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.124+0000 7f8d99483740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901ab950 con 0x558c40e57b60 2024-09-19T07:03:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.124+0000 7f8d99483740 10 client.0 _ll_register_callbacks cb 0x7f8d840bc750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:55.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.124+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e67ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:55.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.124+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e67ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52882/0 (socket says 172.21.15.103:52882) 2024-09-19T07:03:55.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.124+0000 7f8d8ffff640 1 -- 172.21.15.103:0/2343131906 learned_addr learned my addr 172.21.15.103:0/2343131906 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:55.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.124+0000 7f8d95647640 1 --2- 172.21.15.103:0/2343131906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e62d40 0x558c40ed5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:55.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d8ffff640 1 -- 172.21.15.103:0/2343131906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e57b60 msgr2=0x558c40e675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:55.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d94e46640 1 --2- 172.21.15.103:0/2343131906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e57b60 0x558c40e675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:55.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d8ffff640 1 --2- 172.21.15.103:0/2343131906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e57b60 0x558c40e675a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d8ffff640 1 -- 172.21.15.103:0/2343131906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e62d40 msgr2=0x558c40ed5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:55.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d8ffff640 1 --2- 172.21.15.103:0/2343131906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e62d40 0x558c40ed5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d8ffff640 1 -- 172.21.15.103:0/2343131906 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c40e60bc0 con 0x558c40e58480 2024-09-19T07:03:55.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d94e46640 1 --2- 172.21.15.103:0/2343131906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e57b60 0x558c40e675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:55.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d95647640 1 --2- 172.21.15.103:0/2343131906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e62d40 0x558c40ed5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:55.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d8ffff640 1 --2- 172.21.15.103:0/2343131906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e67ae0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0058c0 tx=0x7f8d7c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50824fcdf482d4ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:55.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:55.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d6ffff640 1 -- 172.21.15.103:0/2343131906 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c042060 con 0x558c40e58480 2024-09-19T07:03:55.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d99483740 1 -- 172.21.15.103:0/2343131906 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d7c017aa0 con 0x558c40e58480 2024-09-19T07:03:55.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d6ffff640 1 -- 172.21.15.103:0/2343131906 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f8d7c017830 con 0x558c40e58480 2024-09-19T07:03:55.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d6ffff640 1 -- 172.21.15.103:0/2343131906 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d7c026040 con 0x558c40e58480 2024-09-19T07:03:55.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d99483740 10 client.20502 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:55.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d99483740 1 -- 172.21.15.103:0/2343131906 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558c40eddd90 con 0x558c40e58480 2024-09-19T07:03:55.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d6ffff640 1 -- 172.21.15.103:0/2343131906 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d7c017aa0 con 0x558c40e58480 2024-09-19T07:03:55.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.125+0000 7f8d6ffff640 1 -- 172.21.15.103:0/2343131906 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8d7c0047c0 con 0x558c40e58480 2024-09-19T07:03:55.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.126+0000 7f8d99483740 10 client.20502 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:55.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.126+0000 7f8d99483740 10 client.20502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:55.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.126+0000 7f8d99483740 20 client.20502 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:55.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.126+0000 7f8d99483740 1 --2- 172.21.15.103:0/2343131906 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558c40ede3a0 0x558c40ee0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:55.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.126+0000 7f8d99483740 4 client.20502 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:03:55.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.126+0000 7f8d99483740 1 -- 172.21.15.103:0/2343131906 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558c40dc4d00 con 0x558c40ede3a0 2024-09-19T07:03:55.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.126+0000 7f8d94e46640 1 --2- 172.21.15.103:0/2343131906 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558c40ede3a0 0x558c40ee0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:55.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.126+0000 7f8d6ffff640 10 client.20502 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:55.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.126+0000 7f8d94e46640 1 --2- 172.21.15.103:0/2343131906 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558c40ede3a0 0x558c40ee0790 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89351675d3d8738 server_cookie=c881406819553438 in_seq=0 out_seq=0 2024-09-19T07:03:55.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.127+0000 7f8d6ffff640 1 -- 172.21.15.103:0/2343131906 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558c40dc4d00 con 0x558c40ede3a0 2024-09-19T07:03:55.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.127+0000 7f8d6ffff640 10 client.20502 handle_command_reply: tid=0 2024-09-19T07:03:55.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.127+0000 7f8d99483740 1 client.20502 shutdown 2024-09-19T07:03:55.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.127+0000 7f8d99483740 1 -- 172.21.15.103:0/2343131906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 msgr2=0x558c40e67ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:55.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.127+0000 7f8d99483740 1 --2- 172.21.15.103:0/2343131906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e67ae0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0058c0 tx=0x7f8d7c005bc0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.127+0000 7f8d99483740 1 -- 172.21.15.103:0/2343131906 shutdown_connections 2024-09-19T07:03:55.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.127+0000 7f8d99483740 1 --2- 172.21.15.103:0/2343131906 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x558c40ede3a0 0x558c40ee0790 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.127+0000 7f8d99483740 1 --2- 172.21.15.103:0/2343131906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c40e62d40 0x558c40ed5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.127+0000 7f8d99483740 1 --2- 172.21.15.103:0/2343131906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c40e58480 0x558c40e67ae0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.127+0000 7f8d99483740 1 --2- 172.21.15.103:0/2343131906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c40e57b60 0x558c40e675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.127+0000 7f8d99483740 1 -- 172.21.15.103:0/2343131906 >> 172.21.15.103:0/2343131906 conn(0x558c40e540f0 msgr2=0x558c40e603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:55.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.127+0000 7f8d6ffff640 0 client.20502 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:55.173 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:03:55.173 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:03:55.173 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "5f45b32a-9a28-421a-b165-7c11748fda05", 2024-09-19T07:03:55.173 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:03:55.174 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:03:55.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.128+0000 7f8d99483740 1 -- 172.21.15.103:0/2343131906 shutdown_connections 2024-09-19T07:03:55.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.128+0000 7f8d99483740 1 -- 172.21.15.103:0/2343131906 wait complete. 2024-09-19T07:03:55.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.128+0000 7f8d99483740 20 client.20502 trim_cache size 0 max 16384 2024-09-19T07:03:55.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.128+0000 7f8d970aa640 1 -- 172.21.15.103:0/2485193775 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8d800a33b0 msgr2=0x7f8d800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:55.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.128+0000 7f8d970aa640 1 --2- 172.21.15.103:0/2485193775 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8d800a33b0 0x7f8d800a5870 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800e5d0 tx=0x7f8d88008220 comp rx=0 tx=0).stop 2024-09-19T07:03:55.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.128+0000 7f8d970aa640 1 -- 172.21.15.103:0/2485193775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d90102d40 msgr2=0x7f8d90198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:55.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.128+0000 7f8d970aa640 1 --2- 172.21.15.103:0/2485193775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d90102d40 0x7f8d90198490 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f8d84005ab0 tx=0x7f8d840042f0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.128+0000 7f8d970aa640 1 -- 172.21.15.103:0/2485193775 shutdown_connections 2024-09-19T07:03:55.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.128+0000 7f8d970aa640 1 --2- 172.21.15.103:0/2485193775 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8d800a33b0 0x7f8d800a5870 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.128+0000 7f8d970aa640 1 --2- 172.21.15.103:0/2485193775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d9010c170 0x7f8d9019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.128+0000 7f8d970aa640 1 --2- 172.21.15.103:0/2485193775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d90103660 0x7f8d901989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.128+0000 7f8d970aa640 1 --2- 172.21.15.103:0/2485193775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d90102d40 0x7f8d90198490 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:55.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.128+0000 7f8d970aa640 1 -- 172.21.15.103:0/2485193775 >> 172.21.15.103:0/2485193775 conn(0x7f8d900fc7a0 msgr2=0x7f8d900fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:55.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.128+0000 7f8d970aa640 1 -- 172.21.15.103:0/2485193775 shutdown_connections 2024-09-19T07:03:55.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:55.128+0000 7f8d970aa640 1 -- 172.21.15.103:0/2485193775 wait complete. 2024-09-19T07:03:55.819 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:03:56.111 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:03:56.342 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:56 smithi103 ceph-mon[31158]: pgmap v459: 97 pgs: 97 active+clean; 5.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-19T07:03:56.342 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:56 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T07:03:56.342 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:56 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:03:56.342 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:56 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T07:03:56.342 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:56 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:03:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:56 smithi134 ceph-mon[35356]: pgmap v459: 97 pgs: 97 active+clean; 5.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-19T07:03:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:56 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T07:03:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:56 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:03:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:56 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T07:03:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:56 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:03:56.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.535+0000 7f59f05af640 1 -- 172.21.15.103:0/1553767229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e8102d40 msgr2=0x7f59e8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:56.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.535+0000 7f59f05af640 1 --2- 172.21.15.103:0/1553767229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e8102d40 0x7f59e8103120 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009900 tx=0x7f59e402a480 comp rx=0 tx=0).stop 2024-09-19T07:03:56.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.536+0000 7f59f05af640 1 -- 172.21.15.103:0/1553767229 shutdown_connections 2024-09-19T07:03:56.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.536+0000 7f59f05af640 1 --2- 172.21.15.103:0/1553767229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59e810c170 0x7f59e810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.536+0000 7f59f05af640 1 --2- 172.21.15.103:0/1553767229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59e8103660 0x7f59e810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.536+0000 7f59f05af640 1 --2- 172.21.15.103:0/1553767229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e8102d40 0x7f59e8103120 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.536+0000 7f59f05af640 1 -- 172.21.15.103:0/1553767229 >> 172.21.15.103:0/1553767229 conn(0x7f59e80fc7a0 msgr2=0x7f59e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:56.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.536+0000 7f59f05af640 1 -- 172.21.15.103:0/1553767229 shutdown_connections 2024-09-19T07:03:56.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.536+0000 7f59f05af640 1 -- 172.21.15.103:0/1553767229 wait complete. 2024-09-19T07:03:56.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.536+0000 7f59f05af640 1 Processor -- start 2024-09-19T07:03:56.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.536+0000 7f59f05af640 1 -- start start 2024-09-19T07:03:56.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.536+0000 7f59f05af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e8102d40 0x7f59e8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.536+0000 7f59f05af640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59e8103660 0x7f59e8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59f05af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59e810c170 0x7f59e819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59f05af640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59e8111fb0 con 0x7f59e8102d40 2024-09-19T07:03:56.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59f05af640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59e8111e30 con 0x7f59e810c170 2024-09-19T07:03:56.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59f05af640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59e8112130 con 0x7f59e8103660 2024-09-19T07:03:56.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59ee34b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e8102d40 0x7f59e8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:56.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59ee34b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e8102d40 0x7f59e8198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52906/0 (socket says 172.21.15.103:52906) 2024-09-19T07:03:56.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59ee34b640 1 -- 172.21.15.103:0/276114332 learned_addr learned my addr 172.21.15.103:0/276114332 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:56.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59ee34b640 1 -- 172.21.15.103:0/276114332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59e8103660 msgr2=0x7f59e8198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:56.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59edb4a640 1 --2- 172.21.15.103:0/276114332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59e8103660 0x7f59e8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:56.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59eeb4c640 1 --2- 172.21.15.103:0/276114332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59e810c170 0x7f59e819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:56.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59ee34b640 1 --2- 172.21.15.103:0/276114332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59e8103660 0x7f59e8198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59ee34b640 1 -- 172.21.15.103:0/276114332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59e810c170 msgr2=0x7f59e819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:56.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59ee34b640 1 --2- 172.21.15.103:0/276114332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59e810c170 0x7f59e819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59ee34b640 1 -- 172.21.15.103:0/276114332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e819d520 con 0x7f59e8102d40 2024-09-19T07:03:56.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59edb4a640 1 --2- 172.21.15.103:0/276114332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59e8103660 0x7f59e8198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:56.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59eeb4c640 1 --2- 172.21.15.103:0/276114332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59e810c170 0x7f59e819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:03:56.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59ee34b640 1 --2- 172.21.15.103:0/276114332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e8102d40 0x7f59e8198570 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f59e4005ab0 tx=0x7f59e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:56.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59df7fe640 1 -- 172.21.15.103:0/276114332 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e403f030 con 0x7f59e8102d40 2024-09-19T07:03:56.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59f05af640 1 -- 172.21.15.103:0/276114332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e819d7b0 con 0x7f59e8102d40 2024-09-19T07:03:56.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59df7fe640 1 -- 172.21.15.103:0/276114332 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59e4003ce0 con 0x7f59e8102d40 2024-09-19T07:03:56.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59df7fe640 1 -- 172.21.15.103:0/276114332 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4035070 con 0x7f59e8102d40 2024-09-19T07:03:56.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.537+0000 7f59f05af640 1 -- 172.21.15.103:0/276114332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e81a50f0 con 0x7f59e8102d40 2024-09-19T07:03:56.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.540+0000 7f59df7fe640 1 -- 172.21.15.103:0/276114332 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f59e4042030 con 0x7f59e8102d40 2024-09-19T07:03:56.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.540+0000 7f59f2988740 1 Processor -- start 2024-09-19T07:03:56.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.540+0000 7f59f2988740 1 -- start start 2024-09-19T07:03:56.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.540+0000 7f59f2988740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76f4a0 0x55b68d76f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.540+0000 7f59f2988740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76fdc0 0x55b68d77a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.540+0000 7f59f2988740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 0x55b68d77ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.540+0000 7f59f2988740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b68d6d52e0 con 0x55b68d76fdc0 2024-09-19T07:03:56.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.540+0000 7f59f2988740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b68d67cce0 con 0x55b68d77a680 2024-09-19T07:03:56.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.540+0000 7f59f2988740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b68d6d12a0 con 0x55b68d76f4a0 2024-09-19T07:03:56.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59ee34b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76fdc0 0x55b68d77a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:56.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59ee34b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76fdc0 0x55b68d77a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52920/0 (socket says 172.21.15.103:52920) 2024-09-19T07:03:56.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59ee34b640 1 -- 172.21.15.103:0/1062655513 learned_addr learned my addr 172.21.15.103:0/1062655513 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:56.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59eeb4c640 1 --2- 172.21.15.103:0/1062655513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76f4a0 0x55b68d76f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:56.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59edb4a640 1 --2- 172.21.15.103:0/1062655513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 0x55b68d77ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:56.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59df7fe640 1 --2- 172.21.15.103:0/276114332 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59d40a33b0 0x7f59d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59ee34b640 1 -- 172.21.15.103:0/1062655513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76f4a0 msgr2=0x55b68d76f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:56.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59ee34b640 1 --2- 172.21.15.103:0/1062655513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76f4a0 0x55b68d76f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59ee34b640 1 -- 172.21.15.103:0/1062655513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 msgr2=0x55b68d77ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:56.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59ee34b640 1 --2- 172.21.15.103:0/1062655513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 0x55b68d77ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59ee34b640 1 -- 172.21.15.103:0/1062655513 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b68d661440 con 0x55b68d76fdc0 2024-09-19T07:03:56.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59eeb4c640 1 --2- 172.21.15.103:0/276114332 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59d40a33b0 0x7f59d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:56.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59df7fe640 1 -- 172.21.15.103:0/276114332 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f59e40f06c0 con 0x7f59e8102d40 2024-09-19T07:03:56.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59ee34b640 1 --2- 172.21.15.103:0/1062655513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76fdc0 0x55b68d77a140 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f59e40edd60 tx=0x7f59e40e7750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34d64004e8d9cb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:56.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59eeb4c640 1 --2- 172.21.15.103:0/276114332 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59d40a33b0 0x7f59d40a5870 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f59e0005fd0 tx=0x7f59e0004a40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:56.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59dcff9640 1 -- 172.21.15.103:0/1062655513 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4049070 con 0x55b68d76fdc0 2024-09-19T07:03:56.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59dcff9640 1 -- 172.21.15.103:0/1062655513 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59e40bd080 con 0x55b68d76fdc0 2024-09-19T07:03:56.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.541+0000 7f59dcff9640 1 -- 172.21.15.103:0/1062655513 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e40bc4c0 con 0x55b68d76fdc0 2024-09-19T07:03:56.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.542+0000 7f59f2988740 1 -- 172.21.15.103:0/1062655513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76fdc0 msgr2=0x55b68d77a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:56.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.542+0000 7f59f2988740 1 --2- 172.21.15.103:0/1062655513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76fdc0 0x55b68d77a140 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f59e40edd60 tx=0x7f59e40e7750 comp rx=0 tx=0).stop 2024-09-19T07:03:56.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.542+0000 7f59f2988740 1 -- 172.21.15.103:0/1062655513 shutdown_connections 2024-09-19T07:03:56.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.542+0000 7f59f2988740 1 --2- 172.21.15.103:0/1062655513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 0x55b68d77ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.542+0000 7f59f2988740 1 --2- 172.21.15.103:0/1062655513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76fdc0 0x55b68d77a140 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.542+0000 7f59f2988740 1 --2- 172.21.15.103:0/1062655513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76f4a0 0x55b68d76f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.542+0000 7f59f2988740 1 -- 172.21.15.103:0/1062655513 >> 172.21.15.103:0/1062655513 conn(0x55b68d76ba30 msgr2=0x55b68d76be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:56.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.542+0000 7f59f2988740 1 -- 172.21.15.103:0/1062655513 shutdown_connections 2024-09-19T07:03:56.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.542+0000 7f59f2988740 1 -- 172.21.15.103:0/1062655513 wait complete. 2024-09-19T07:03:56.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59f2988740 1 -- start start 2024-09-19T07:03:56.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59f2988740 1 Processor -- start 2024-09-19T07:03:56.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59f2988740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76f4a0 0x55b68d779f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59f2988740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76fdc0 0x55b68d777220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59f2988740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 0x55b68d777760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59f2988740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b68d77d010 con 0x55b68d76fdc0 2024-09-19T07:03:56.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59f2988740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b68d690050 con 0x55b68d77a680 2024-09-19T07:03:56.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59f2988740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b68d77d190 con 0x55b68d76f4a0 2024-09-19T07:03:56.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59f2988740 10 client.0 _ll_register_callbacks cb 0x55b68d6d0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:56.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59edb4a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76fdc0 0x55b68d777220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:56.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59edb4a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76fdc0 0x55b68d777220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52922/0 (socket says 172.21.15.103:52922) 2024-09-19T07:03:56.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59edb4a640 1 -- 172.21.15.103:0/3707931378 learned_addr learned my addr 172.21.15.103:0/3707931378 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:56.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59ee34b640 1 --2- 172.21.15.103:0/3707931378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76f4a0 0x55b68d779f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:56.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59eeb4c640 1 --2- 172.21.15.103:0/3707931378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 0x55b68d777760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:56.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59edb4a640 1 -- 172.21.15.103:0/3707931378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76f4a0 msgr2=0x55b68d779f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:56.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59edb4a640 1 --2- 172.21.15.103:0/3707931378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76f4a0 0x55b68d779f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59edb4a640 1 -- 172.21.15.103:0/3707931378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 msgr2=0x55b68d777760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:56.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59edb4a640 1 --2- 172.21.15.103:0/3707931378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 0x55b68d777760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59edb4a640 1 -- 172.21.15.103:0/3707931378 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b68d4463c0 con 0x55b68d76fdc0 2024-09-19T07:03:56.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59eeb4c640 1 --2- 172.21.15.103:0/3707931378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 0x55b68d777760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:56.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.543+0000 7f59ee34b640 1 --2- 172.21.15.103:0/3707931378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76f4a0 0x55b68d779f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:03:56.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.544+0000 7f59edb4a640 1 --2- 172.21.15.103:0/3707931378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76fdc0 0x55b68d777220 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f59d800dec0 tx=0x7f59d800f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92edaf671978cb07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:56.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.544+0000 7f59d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:56.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.544+0000 7f59d1ffb640 1 -- 172.21.15.103:0/3707931378 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8014680 con 0x55b68d76fdc0 2024-09-19T07:03:56.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.544+0000 7f59d1ffb640 1 -- 172.21.15.103:0/3707931378 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f59d8014820 con 0x55b68d76fdc0 2024-09-19T07:03:56.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.544+0000 7f59f2988740 1 -- 172.21.15.103:0/3707931378 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59e40bd080 con 0x55b68d76fdc0 2024-09-19T07:03:56.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.544+0000 7f59d1ffb640 1 -- 172.21.15.103:0/3707931378 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f59d8023050 con 0x55b68d76fdc0 2024-09-19T07:03:56.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.544+0000 7f59f2988740 10 client.20514 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:56.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.544+0000 7f59f2988740 1 -- 172.21.15.103:0/3707931378 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b68d788950 con 0x55b68d76fdc0 2024-09-19T07:03:56.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.544+0000 7f59d1ffb640 1 -- 172.21.15.103:0/3707931378 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59e40bd080 con 0x55b68d76fdc0 2024-09-19T07:03:56.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.544+0000 7f59d1ffb640 1 -- 172.21.15.103:0/3707931378 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f59d801c3e0 con 0x55b68d76fdc0 2024-09-19T07:03:56.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.544+0000 7f59f2988740 10 client.20514 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:56.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.544+0000 7f59f2988740 10 client.20514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:56.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.544+0000 7f59f2988740 20 client.20514 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:56.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.544+0000 7f59f2988740 1 --2- 172.21.15.103:0/3707931378 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b68d789eb0 0x55b68d78c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.544+0000 7f59f2988740 4 client.20514 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:03:56.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.544+0000 7f59f2988740 1 -- 172.21.15.103:0/3707931378 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b68d6e2850 con 0x55b68d789eb0 2024-09-19T07:03:56.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.545+0000 7f59ee34b640 1 --2- 172.21.15.103:0/3707931378 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b68d789eb0 0x55b68d78c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:56.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.545+0000 7f59ee34b640 1 --2- 172.21.15.103:0/3707931378 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b68d789eb0 0x55b68d78c2f0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b9030e720b5abe1 server_cookie=52448040d56fb0c6 in_seq=0 out_seq=0 2024-09-19T07:03:56.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.545+0000 7f59d1ffb640 10 client.20514 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:56.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.546+0000 7f59d1ffb640 1 -- 172.21.15.103:0/3707931378 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b68d6e2850 con 0x55b68d789eb0 2024-09-19T07:03:56.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.546+0000 7f59d1ffb640 10 client.20514 handle_command_reply: tid=0 2024-09-19T07:03:56.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.546+0000 7f59f2988740 1 client.20514 shutdown 2024-09-19T07:03:56.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.546+0000 7f59f2988740 1 -- 172.21.15.103:0/3707931378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76fdc0 msgr2=0x55b68d777220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:56.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.546+0000 7f59f2988740 1 --2- 172.21.15.103:0/3707931378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76fdc0 0x55b68d777220 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f59d800dec0 tx=0x7f59d800f5b0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.546+0000 7f59f2988740 1 -- 172.21.15.103:0/3707931378 shutdown_connections 2024-09-19T07:03:56.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.546+0000 7f59f2988740 1 --2- 172.21.15.103:0/3707931378 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b68d789eb0 0x55b68d78c2f0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.546+0000 7f59d1ffb640 0 client.20514 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:56.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.546+0000 7f59f2988740 1 --2- 172.21.15.103:0/3707931378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 0x55b68d777760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.546+0000 7f59f2988740 1 --2- 172.21.15.103:0/3707931378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76fdc0 0x55b68d777220 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.546+0000 7f59f2988740 1 --2- 172.21.15.103:0/3707931378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76f4a0 0x55b68d779f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.546+0000 7f59f2988740 1 -- 172.21.15.103:0/3707931378 >> 172.21.15.103:0/3707931378 conn(0x55b68d76ba30 msgr2=0x55b68d771760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:56.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.546+0000 7f59f2988740 1 -- 172.21.15.103:0/3707931378 shutdown_connections 2024-09-19T07:03:56.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.546+0000 7f59f2988740 1 -- 172.21.15.103:0/3707931378 wait complete. 2024-09-19T07:03:56.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.546+0000 7f59f2988740 20 client.20514 trim_cache size 0 max 16384 2024-09-19T07:03:56.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.563+0000 7f59f2988740 1 Processor -- start 2024-09-19T07:03:56.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.563+0000 7f59f2988740 1 -- start start 2024-09-19T07:03:56.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.563+0000 7f59f2988740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76f4a0 0x55b68d786100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.563+0000 7f59f2988740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76fdc0 0x55b68d791bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.563+0000 7f59f2988740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 0x55b68d792100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.563+0000 7f59f2988740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81ad950 con 0x55b68d76f4a0 2024-09-19T07:03:56.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.563+0000 7f59f2988740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81af490 con 0x55b68d77a680 2024-09-19T07:03:56.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.563+0000 7f59f2988740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81ae020 con 0x55b68d76fdc0 2024-09-19T07:03:56.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.563+0000 7f59ee34b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76f4a0 0x55b68d786100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:56.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.563+0000 7f59ee34b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76f4a0 0x55b68d786100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52936/0 (socket says 172.21.15.103:52936) 2024-09-19T07:03:56.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.563+0000 7f59ee34b640 1 -- 172.21.15.103:0/3192292078 learned_addr learned my addr 172.21.15.103:0/3192292078 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:56.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.564+0000 7f59ee34b640 1 -- 172.21.15.103:0/3192292078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76fdc0 msgr2=0x55b68d791bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:56.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.564+0000 7f59ee34b640 1 --2- 172.21.15.103:0/3192292078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76fdc0 0x55b68d791bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.564+0000 7f59ee34b640 1 -- 172.21.15.103:0/3192292078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 msgr2=0x55b68d792100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:56.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.564+0000 7f59ee34b640 1 --2- 172.21.15.103:0/3192292078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 0x55b68d792100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.564+0000 7f59ee34b640 1 -- 172.21.15.103:0/3192292078 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b68d76b860 con 0x55b68d76f4a0 2024-09-19T07:03:56.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.564+0000 7f59ee34b640 1 --2- 172.21.15.103:0/3192292078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76f4a0 0x55b68d786100 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f59e4031550 tx=0x7f59e4005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844e36a0c4923f26 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:56.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.564+0000 7f59d3fff640 1 -- 172.21.15.103:0/3192292078 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4049040 con 0x55b68d76f4a0 2024-09-19T07:03:56.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.564+0000 7f59d3fff640 1 -- 172.21.15.103:0/3192292078 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59e40bd080 con 0x55b68d76f4a0 2024-09-19T07:03:56.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.564+0000 7f59d3fff640 1 -- 172.21.15.103:0/3192292078 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e40bbdd0 con 0x55b68d76f4a0 2024-09-19T07:03:56.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 -- 172.21.15.103:0/3192292078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76f4a0 msgr2=0x55b68d786100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:56.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 --2- 172.21.15.103:0/3192292078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76f4a0 0x55b68d786100 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f59e4031550 tx=0x7f59e4005890 comp rx=0 tx=0).stop 2024-09-19T07:03:56.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 -- 172.21.15.103:0/3192292078 shutdown_connections 2024-09-19T07:03:56.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 --2- 172.21.15.103:0/3192292078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 0x55b68d792100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 --2- 172.21.15.103:0/3192292078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76fdc0 0x55b68d791bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 --2- 172.21.15.103:0/3192292078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76f4a0 0x55b68d786100 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 -- 172.21.15.103:0/3192292078 >> 172.21.15.103:0/3192292078 conn(0x55b68d76ba30 msgr2=0x55b68d7728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:56.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 -- 172.21.15.103:0/3192292078 shutdown_connections 2024-09-19T07:03:56.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 -- 172.21.15.103:0/3192292078 wait complete. 2024-09-19T07:03:56.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 -- start start 2024-09-19T07:03:56.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 Processor -- start 2024-09-19T07:03:56.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76f4a0 0x55b68d7823d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76fdc0 0x55b68d782910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 0x55b68d77d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81aedc0 con 0x55b68d76f4a0 2024-09-19T07:03:56.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81ae6f0 con 0x55b68d77a680 2024-09-19T07:03:56.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81ab950 con 0x55b68d76fdc0 2024-09-19T07:03:56.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.565+0000 7f59f2988740 10 client.0 _ll_register_callbacks cb 0x7f59e806b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:03:56.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59ee34b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76f4a0 0x55b68d7823d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:56.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59ee34b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76f4a0 0x55b68d7823d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52938/0 (socket says 172.21.15.103:52938) 2024-09-19T07:03:56.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59ee34b640 1 -- 172.21.15.103:0/3969848767 learned_addr learned my addr 172.21.15.103:0/3969848767 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:03:56.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59edb4a640 1 --2- 172.21.15.103:0/3969848767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76fdc0 0x55b68d782910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:56.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59eeb4c640 1 --2- 172.21.15.103:0/3969848767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 0x55b68d77d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:56.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59ee34b640 1 -- 172.21.15.103:0/3969848767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76fdc0 msgr2=0x55b68d782910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:56.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59ee34b640 1 --2- 172.21.15.103:0/3969848767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76fdc0 0x55b68d782910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59ee34b640 1 -- 172.21.15.103:0/3969848767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 msgr2=0x55b68d77d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:03:56.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59ee34b640 1 --2- 172.21.15.103:0/3969848767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 0x55b68d77d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59ee34b640 1 -- 172.21.15.103:0/3969848767 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b68d77dfc0 con 0x55b68d76f4a0 2024-09-19T07:03:56.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59ee34b640 1 --2- 172.21.15.103:0/3969848767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76f4a0 0x55b68d7823d0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f59e4006010 tx=0x7f59e40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7289de35aacfc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:03:56.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:03:56.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59d1ffb640 1 -- 172.21.15.103:0/3969848767 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4085060 con 0x55b68d76f4a0 2024-09-19T07:03:56.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59d1ffb640 1 -- 172.21.15.103:0/3969848767 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8556+0+0 (secure 0 0 0) 0x7f59e40b87d0 con 0x55b68d76f4a0 2024-09-19T07:03:56.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59f2988740 1 -- 172.21.15.103:0/3969848767 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59e40bd080 con 0x55b68d76f4a0 2024-09-19T07:03:56.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59d1ffb640 1 -- 172.21.15.103:0/3969848767 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f59e4049030 con 0x55b68d76f4a0 2024-09-19T07:03:56.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59f2988740 10 client.20520 fetch_fsmap learned FSMap version 27 2024-09-19T07:03:56.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59f2988740 1 -- 172.21.15.103:0/3969848767 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b68d7f45b0 con 0x55b68d76f4a0 2024-09-19T07:03:56.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.566+0000 7f59d1ffb640 1 -- 172.21.15.103:0/3969848767 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59e40bd080 con 0x55b68d76f4a0 2024-09-19T07:03:56.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.567+0000 7f59d1ffb640 1 -- 172.21.15.103:0/3969848767 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f59e40ee070 con 0x55b68d76f4a0 2024-09-19T07:03:56.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.567+0000 7f59f2988740 10 client.20520 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:03:56.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.567+0000 7f59f2988740 10 client.20520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:03:56.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.567+0000 7f59f2988740 20 client.20520 populate_metadata read hostname 'smithi103' 2024-09-19T07:03:56.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.567+0000 7f59f2988740 1 --2- 172.21.15.103:0/3969848767 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b68d789eb0 0x55b68d7f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:03:56.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.567+0000 7f59f2988740 4 client.20520 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:03:56.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.567+0000 7f59f2988740 1 -- 172.21.15.103:0/3969848767 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b68d6dcd60 con 0x55b68d789eb0 2024-09-19T07:03:56.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.567+0000 7f59edb4a640 1 --2- 172.21.15.103:0/3969848767 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b68d789eb0 0x55b68d7f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:03:56.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.567+0000 7f59edb4a640 1 --2- 172.21.15.103:0/3969848767 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b68d789eb0 0x55b68d7f68f0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3bc98d336bbd151 server_cookie=d04c29fe2f848392 in_seq=0 out_seq=0 2024-09-19T07:03:56.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.567+0000 7f59d1ffb640 10 client.20520 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:56.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.567+0000 7f59d1ffb640 1 -- 172.21.15.103:0/3969848767 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b68d6dcd60 con 0x55b68d789eb0 2024-09-19T07:03:56.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59d1ffb640 10 client.20520 handle_command_reply: tid=0 2024-09-19T07:03:56.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f2988740 1 client.20520 shutdown 2024-09-19T07:03:56.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f2988740 1 -- 172.21.15.103:0/3969848767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76f4a0 msgr2=0x55b68d7823d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:56.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f2988740 1 --2- 172.21.15.103:0/3969848767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76f4a0 0x55b68d7823d0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f59e4006010 tx=0x7f59e40045e0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f2988740 1 -- 172.21.15.103:0/3969848767 shutdown_connections 2024-09-19T07:03:56.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f2988740 1 --2- 172.21.15.103:0/3969848767 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b68d789eb0 0x55b68d7f68f0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f2988740 1 --2- 172.21.15.103:0/3969848767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b68d77a680 0x55b68d77d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f2988740 1 --2- 172.21.15.103:0/3969848767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b68d76fdc0 0x55b68d782910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f2988740 1 --2- 172.21.15.103:0/3969848767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b68d76f4a0 0x55b68d7823d0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f2988740 1 -- 172.21.15.103:0/3969848767 >> 172.21.15.103:0/3969848767 conn(0x55b68d76ba30 msgr2=0x55b68d76a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:56.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59d1ffb640 0 client.20520 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:03:56.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f2988740 1 -- 172.21.15.103:0/3969848767 shutdown_connections 2024-09-19T07:03:56.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f2988740 1 -- 172.21.15.103:0/3969848767 wait complete. 2024-09-19T07:03:56.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f2988740 20 client.20520 trim_cache size 0 max 16384 2024-09-19T07:03:56.575 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:03:56.575 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:03:56.575 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:03:56.575 INFO:teuthology.orchestra.run.smithi103.stdout: "5f45b32a-9a28-421a-b165-7c11748fda05": { 2024-09-19T07:03:56.575 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:03:56.575 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "5f45b32a-9a28-421a-b165-7c11748fda05", 2024-09-19T07:03:56.575 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:03:56.575 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:03:56.575 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:03:56.575 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:03:56.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f05af640 1 -- 172.21.15.103:0/276114332 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59d40a33b0 msgr2=0x7f59d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:56.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f05af640 1 --2- 172.21.15.103:0/276114332 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59d40a33b0 0x7f59d40a5870 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f59e0005fd0 tx=0x7f59e0004a40 comp rx=0 tx=0).stop 2024-09-19T07:03:56.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f05af640 1 -- 172.21.15.103:0/276114332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e8102d40 msgr2=0x7f59e8198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:03:56.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f05af640 1 --2- 172.21.15.103:0/276114332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e8102d40 0x7f59e8198570 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f59e4005ab0 tx=0x7f59e40042f0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f05af640 1 -- 172.21.15.103:0/276114332 shutdown_connections 2024-09-19T07:03:56.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f05af640 1 --2- 172.21.15.103:0/276114332 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f59d40a33b0 0x7f59d40a5870 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f05af640 1 --2- 172.21.15.103:0/276114332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59e810c170 0x7f59e819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f05af640 1 --2- 172.21.15.103:0/276114332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59e8103660 0x7f59e8198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f05af640 1 --2- 172.21.15.103:0/276114332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e8102d40 0x7f59e8198570 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:03:56.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.568+0000 7f59f05af640 1 -- 172.21.15.103:0/276114332 >> 172.21.15.103:0/276114332 conn(0x7f59e80fc7a0 msgr2=0x7f59e80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:03:56.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.569+0000 7f59f05af640 1 -- 172.21.15.103:0/276114332 shutdown_connections 2024-09-19T07:03:56.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:03:56.569+0000 7f59f05af640 1 -- 172.21.15.103:0/276114332 wait complete. 2024-09-19T07:03:56.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:56 smithi174 ceph-mon[35637]: pgmap v459: 97 pgs: 97 active+clean; 5.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-19T07:03:56.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:56 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T07:03:56.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:56 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:03:56.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:56 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T07:03:56.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:56 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:03:57.218 INFO:tasks.cephfs.filesystem:scrub status for tag:5f45b32a-9a28-421a-b165-7c11748fda05 - {'path': '/', 'tag': '5f45b32a-9a28-421a-b165-7c11748fda05', 'options': 'recursive,force'} 2024-09-19T07:03:58.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:03:58 smithi103 ceph-mon[31158]: pgmap v460: 97 pgs: 97 active+clean; 1.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-19T07:03:58.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:03:58 smithi134 ceph-mon[35356]: pgmap v460: 97 pgs: 97 active+clean; 1.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-19T07:03:58.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:03:58 smithi174 ceph-mon[35637]: pgmap v460: 97 pgs: 97 active+clean; 1.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-19T07:04:00.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:00 smithi103 ceph-mon[31158]: pgmap v461: 97 pgs: 97 active+clean; 1.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-09-19T07:04:00.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:00 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T07:04:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:00 smithi134 ceph-mon[35356]: pgmap v461: 97 pgs: 97 active+clean; 1.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-09-19T07:04:00.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:00 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T07:04:00.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:00 smithi174 ceph-mon[35637]: pgmap v461: 97 pgs: 97 active+clean; 1.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-09-19T07:04:00.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:00 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T07:04:02.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:02 smithi103 ceph-mon[31158]: pgmap v462: 97 pgs: 97 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-09-19T07:04:02.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:02 smithi134 ceph-mon[35356]: pgmap v462: 97 pgs: 97 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-09-19T07:04:02.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:02 smithi174 ceph-mon[35637]: pgmap v462: 97 pgs: 97 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-09-19T07:04:04.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:04 smithi103 ceph-mon[31158]: pgmap v463: 97 pgs: 97 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-09-19T07:04:04.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:04 smithi134 ceph-mon[35356]: pgmap v463: 97 pgs: 97 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-09-19T07:04:04.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:04 smithi174 ceph-mon[35637]: pgmap v463: 97 pgs: 97 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-09-19T07:04:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.526+0000 7f664b08d640 1 -- 172.21.15.103:0/940652757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6644102cc0 msgr2=0x7f66441030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.526+0000 7f664b08d640 1 --2- 172.21.15.103:0/940652757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6644102cc0 0x7f66441030a0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f6638009900 tx=0x7f663802a480 comp rx=0 tx=0).stop 2024-09-19T07:04:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.526+0000 7f664b08d640 1 -- 172.21.15.103:0/940652757 shutdown_connections 2024-09-19T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.526+0000 7f664b08d640 1 --2- 172.21.15.103:0/940652757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f664410c110 0x7f664410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.526+0000 7f664b08d640 1 --2- 172.21.15.103:0/940652757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66441035e0 0x7f664410bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.526+0000 7f664b08d640 1 --2- 172.21.15.103:0/940652757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6644102cc0 0x7f66441030a0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.526+0000 7f664b08d640 1 -- 172.21.15.103:0/940652757 >> 172.21.15.103:0/940652757 conn(0x7f66440fc6b0 msgr2=0x7f66440fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.526+0000 7f664b08d640 1 -- 172.21.15.103:0/940652757 shutdown_connections 2024-09-19T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.526+0000 7f664b08d640 1 -- 172.21.15.103:0/940652757 wait complete. 2024-09-19T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.526+0000 7f664b08d640 1 Processor -- start 2024-09-19T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f664b08d640 1 -- start start 2024-09-19T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f664b08d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6644102cc0 0x7f6644100910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f664b08d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66441035e0 0x7f6644100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f664b08d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f664410c110 0x7f6644101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f664b08d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6644111f30 con 0x7f664410c110 2024-09-19T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f664b08d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6644111db0 con 0x7f66441035e0 2024-09-19T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f664b08d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66441120b0 con 0x7f6644102cc0 2024-09-19T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f664962a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f664410c110 0x7f6644101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f664962a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f664410c110 0x7f6644101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43172/0 (socket says 172.21.15.103:43172) 2024-09-19T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f664962a640 1 -- 172.21.15.103:0/1967364299 learned_addr learned my addr 172.21.15.103:0/1967364299 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f6648e29640 1 --2- 172.21.15.103:0/1967364299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6644102cc0 0x7f6644100910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f6643fff640 1 --2- 172.21.15.103:0/1967364299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66441035e0 0x7f6644100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f664962a640 1 -- 172.21.15.103:0/1967364299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6644102cc0 msgr2=0x7f6644100910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f664962a640 1 --2- 172.21.15.103:0/1967364299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6644102cc0 0x7f6644100910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f664962a640 1 -- 172.21.15.103:0/1967364299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66441035e0 msgr2=0x7f6644100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f664962a640 1 --2- 172.21.15.103:0/1967364299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66441035e0 0x7f6644100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f664962a640 1 -- 172.21.15.103:0/1967364299 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66441a6910 con 0x7f664410c110 2024-09-19T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f6643fff640 1 --2- 172.21.15.103:0/1967364299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66441035e0 0x7f6644100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f664962a640 1 --2- 172.21.15.103:0/1967364299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f664410c110 0x7f6644101390 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f663c00cce0 tx=0x7f663c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.527+0000 7f6641ffb640 1 -- 172.21.15.103:0/1967364299 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c00f450 con 0x7f664410c110 2024-09-19T07:04:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.528+0000 7f6641ffb640 1 -- 172.21.15.103:0/1967364299 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c00fad0 con 0x7f664410c110 2024-09-19T07:04:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.528+0000 7f664b08d640 1 -- 172.21.15.103:0/1967364299 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66441a6c00 con 0x7f664410c110 2024-09-19T07:04:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.528+0000 7f6641ffb640 1 -- 172.21.15.103:0/1967364299 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c018c10 con 0x7f664410c110 2024-09-19T07:04:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.528+0000 7f664b08d640 1 -- 172.21.15.103:0/1967364299 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66441a7150 con 0x7f664410c110 2024-09-19T07:04:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.530+0000 7f6641ffb640 1 -- 172.21.15.103:0/1967364299 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f663c00f5f0 con 0x7f664410c110 2024-09-19T07:04:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.530+0000 7f664de26740 1 Processor -- start 2024-09-19T07:04:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f664de26740 1 -- start start 2024-09-19T07:04:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f6641ffb640 1 --2- 172.21.15.103:0/1967364299 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66340a34e0 0x7f66340a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f664de26740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a30af0 0x562264a30ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f664de26740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 0x562264a3b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f664de26740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a3bcb0 0x562264a3e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f664de26740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5622649952f0 con 0x562264a3bcb0 2024-09-19T07:04:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f664de26740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56226493d270 con 0x562264a30af0 2024-09-19T07:04:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f664de26740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5622649912b0 con 0x562264a31410 2024-09-19T07:04:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f6643fff640 1 --2- 172.21.15.103:0/1967364299 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66340a34e0 0x7f66340a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f6641ffb640 1 -- 172.21.15.103:0/1967364299 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f663c0d04c0 con 0x7f664410c110 2024-09-19T07:04:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f664962a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a3bcb0 0x562264a3e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f664962a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a3bcb0 0x562264a3e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43180/0 (socket says 172.21.15.103:43180) 2024-09-19T07:04:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f664962a640 1 -- 172.21.15.103:0/392938931 learned_addr learned my addr 172.21.15.103:0/392938931 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f6643fff640 1 --2- 172.21.15.103:0/392938931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 0x562264a3b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f6648e29640 1 --2- 172.21.15.103:0/392938931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a30af0 0x562264a30ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f664962a640 1 -- 172.21.15.103:0/392938931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 msgr2=0x562264a3b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f664962a640 1 --2- 172.21.15.103:0/392938931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 0x562264a3b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f664962a640 1 -- 172.21.15.103:0/392938931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a30af0 msgr2=0x562264a30ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f664962a640 1 --2- 172.21.15.103:0/392938931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a30af0 0x562264a30ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f664962a640 1 -- 172.21.15.103:0/392938931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56226493c340 con 0x562264a3bcb0 2024-09-19T07:04:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f6643fff640 1 --2- 172.21.15.103:0/392938931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 0x562264a3b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f6648e29640 1 --2- 172.21.15.103:0/392938931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a30af0 0x562264a30ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f6643fff640 1 --2- 172.21.15.103:0/1967364299 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66340a34e0 0x7f66340a59a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6630006fd0 tx=0x7f6630013650 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.531+0000 7f664962a640 1 --2- 172.21.15.103:0/392938931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a3bcb0 0x562264a3e0a0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f663c098b50 tx=0x7f663c09c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd69efe53678782 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.532+0000 7f662effd640 1 -- 172.21.15.103:0/392938931 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c0a0aa0 con 0x562264a3bcb0 2024-09-19T07:04:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.532+0000 7f662effd640 1 -- 172.21.15.103:0/392938931 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c0a5360 con 0x562264a3bcb0 2024-09-19T07:04:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.532+0000 7f662effd640 1 -- 172.21.15.103:0/392938931 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c0cdde0 con 0x562264a3bcb0 2024-09-19T07:04:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.532+0000 7f664de26740 1 -- 172.21.15.103:0/392938931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a3bcb0 msgr2=0x562264a3e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.532+0000 7f664de26740 1 --2- 172.21.15.103:0/392938931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a3bcb0 0x562264a3e0a0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f663c098b50 tx=0x7f663c09c050 comp rx=0 tx=0).stop 2024-09-19T07:04:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.532+0000 7f664de26740 1 -- 172.21.15.103:0/392938931 shutdown_connections 2024-09-19T07:04:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.532+0000 7f664de26740 1 --2- 172.21.15.103:0/392938931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a3bcb0 0x562264a3e0a0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.532+0000 7f664de26740 1 --2- 172.21.15.103:0/392938931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 0x562264a3b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.532+0000 7f664de26740 1 --2- 172.21.15.103:0/392938931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a30af0 0x562264a30ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.532+0000 7f664de26740 1 -- 172.21.15.103:0/392938931 >> 172.21.15.103:0/392938931 conn(0x562264a2d060 msgr2=0x562264a2d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.532+0000 7f664de26740 1 -- 172.21.15.103:0/392938931 shutdown_connections 2024-09-19T07:04:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.532+0000 7f664de26740 1 -- 172.21.15.103:0/392938931 wait complete. 2024-09-19T07:04:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.533+0000 7f664de26740 1 -- start start 2024-09-19T07:04:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.533+0000 7f664de26740 1 Processor -- start 2024-09-19T07:04:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.533+0000 7f664de26740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a38be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.533+0000 7f664de26740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 0x562264a39120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.533+0000 7f664de26740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a3bcb0 0x562264a39660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.533+0000 7f664de26740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562264a3e880 con 0x562264a30af0 2024-09-19T07:04:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.533+0000 7f664de26740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56226491e290 con 0x562264a3bcb0 2024-09-19T07:04:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.533+0000 7f664de26740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562264a3ea00 con 0x562264a31410 2024-09-19T07:04:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.533+0000 7f664de26740 10 client.0 _ll_register_callbacks cb 0x56226494f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.533+0000 7f6648e29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a38be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.533+0000 7f6648e29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a38be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43182/0 (socket says 172.21.15.103:43182) 2024-09-19T07:04:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.533+0000 7f6648e29640 1 -- 172.21.15.103:0/2158594106 learned_addr learned my addr 172.21.15.103:0/2158594106 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f6643fff640 1 --2- 172.21.15.103:0/2158594106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 0x562264a39120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f664962a640 1 --2- 172.21.15.103:0/2158594106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a3bcb0 0x562264a39660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f6648e29640 1 -- 172.21.15.103:0/2158594106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 msgr2=0x562264a39120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f6648e29640 1 --2- 172.21.15.103:0/2158594106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 0x562264a39120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f6648e29640 1 -- 172.21.15.103:0/2158594106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a3bcb0 msgr2=0x562264a39660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f6648e29640 1 --2- 172.21.15.103:0/2158594106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a3bcb0 0x562264a39660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f6648e29640 1 -- 172.21.15.103:0/2158594106 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622646a8210 con 0x562264a30af0 2024-09-19T07:04:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f6643fff640 1 --2- 172.21.15.103:0/2158594106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 0x562264a39120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f664962a640 1 --2- 172.21.15.103:0/2158594106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a3bcb0 0x562264a39660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f6648e29640 1 --2- 172.21.15.103:0/2158594106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a38be0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f6638033000 tx=0x7f6638004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d5e8bc07ef27f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f6623fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f6623fff640 1 -- 172.21.15.103:0/2158594106 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638035070 con 0x562264a30af0 2024-09-19T07:04:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f6623fff640 1 -- 172.21.15.103:0/2158594106 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f6638005220 con 0x562264a30af0 2024-09-19T07:04:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f664de26740 1 -- 172.21.15.103:0/2158594106 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f663c0a5360 con 0x562264a30af0 2024-09-19T07:04:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f6623fff640 1 -- 172.21.15.103:0/2158594106 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6638005d50 con 0x562264a30af0 2024-09-19T07:04:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f664de26740 10 client.20532 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f664de26740 1 -- 172.21.15.103:0/2158594106 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562264774620 con 0x562264a30af0 2024-09-19T07:04:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.534+0000 7f6623fff640 1 -- 172.21.15.103:0/2158594106 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c0a5360 con 0x562264a30af0 2024-09-19T07:04:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.535+0000 7f6623fff640 1 -- 172.21.15.103:0/2158594106 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f66380036c0 con 0x562264a30af0 2024-09-19T07:04:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.535+0000 7f664de26740 10 client.20532 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.535+0000 7f664de26740 10 client.20532 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.535+0000 7f664de26740 20 client.20532 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.535+0000 7f664de26740 1 --2- 172.21.15.103:0/2158594106 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562264a4bc90 0x562264a4e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.535+0000 7f664de26740 4 client.20532 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.535+0000 7f664de26740 1 -- 172.21.15.103:0/2158594106 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622649a2a20 con 0x562264a4bc90 2024-09-19T07:04:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.535+0000 7f6643fff640 1 --2- 172.21.15.103:0/2158594106 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562264a4bc90 0x562264a4e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.535+0000 7f6643fff640 1 --2- 172.21.15.103:0/2158594106 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562264a4bc90 0x562264a4e0a0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe30c401ee9ac20 server_cookie=252a63976f3bd28 in_seq=0 out_seq=0 2024-09-19T07:04:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.535+0000 7f6623fff640 10 client.20532 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.536+0000 7f6623fff640 1 -- 172.21.15.103:0/2158594106 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622649a2a20 con 0x562264a4bc90 2024-09-19T07:04:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.536+0000 7f6623fff640 10 client.20532 handle_command_reply: tid=0 2024-09-19T07:04:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.536+0000 7f664de26740 1 client.20532 shutdown 2024-09-19T07:04:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.536+0000 7f664de26740 1 -- 172.21.15.103:0/2158594106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 msgr2=0x562264a38be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.536+0000 7f664de26740 1 --2- 172.21.15.103:0/2158594106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a38be0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f6638033000 tx=0x7f6638004270 comp rx=0 tx=0).stop 2024-09-19T07:04:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.536+0000 7f664de26740 1 -- 172.21.15.103:0/2158594106 shutdown_connections 2024-09-19T07:04:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.536+0000 7f664de26740 1 --2- 172.21.15.103:0/2158594106 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562264a4bc90 0x562264a4e0a0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.536+0000 7f664de26740 1 --2- 172.21.15.103:0/2158594106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a3bcb0 0x562264a39660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.536+0000 7f664de26740 1 --2- 172.21.15.103:0/2158594106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 0x562264a39120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.536+0000 7f664de26740 1 --2- 172.21.15.103:0/2158594106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a38be0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.536+0000 7f6623fff640 0 client.20532 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.536+0000 7f664de26740 1 -- 172.21.15.103:0/2158594106 >> 172.21.15.103:0/2158594106 conn(0x562264a2d060 msgr2=0x562264a32d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.536+0000 7f664de26740 1 -- 172.21.15.103:0/2158594106 shutdown_connections 2024-09-19T07:04:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.537+0000 7f664de26740 1 -- 172.21.15.103:0/2158594106 wait complete. 2024-09-19T07:04:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.537+0000 7f664de26740 20 client.20532 trim_cache size 0 max 16384 2024-09-19T07:04:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.551+0000 7f664de26740 1 Processor -- start 2024-09-19T07:04:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.551+0000 7f664de26740 1 -- start start 2024-09-19T07:04:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.551+0000 7f664de26740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a39490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.551+0000 7f664de26740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a31410 0x562264a539b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.551+0000 7f664de26740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a3bcb0 0x562264a53ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.551+0000 7f664de26740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66441af9b0 con 0x562264a30af0 2024-09-19T07:04:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.551+0000 7f664de26740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66441b14f0 con 0x562264a31410 2024-09-19T07:04:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.551+0000 7f664de26740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66441b0080 con 0x562264a3bcb0 2024-09-19T07:04:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.552+0000 7f6648e29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a39490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.552+0000 7f6648e29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a39490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43192/0 (socket says 172.21.15.103:43192) 2024-09-19T07:04:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.552+0000 7f6643fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a31410 0x562264a539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.552+0000 7f6648e29640 1 -- 172.21.15.103:0/899361380 learned_addr learned my addr 172.21.15.103:0/899361380 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.552+0000 7f664962a640 1 --2- 172.21.15.103:0/899361380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a3bcb0 0x562264a53ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.552+0000 7f6643fff640 1 -- 172.21.15.103:0/899361380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a3bcb0 msgr2=0x562264a53ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.552+0000 7f6643fff640 1 --2- 172.21.15.103:0/899361380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a3bcb0 0x562264a53ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.552+0000 7f6643fff640 1 -- 172.21.15.103:0/899361380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 msgr2=0x562264a39490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.552+0000 7f6643fff640 1 --2- 172.21.15.103:0/899361380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a39490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.552+0000 7f6643fff640 1 -- 172.21.15.103:0/899361380 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562264a4e720 con 0x562264a31410 2024-09-19T07:04:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.552+0000 7f6648e29640 1 --2- 172.21.15.103:0/899361380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a39490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.552+0000 7f6643fff640 1 --2- 172.21.15.103:0/899361380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a31410 0x562264a539b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x562264a4e950 tx=0x7f6630019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e416edb22a46878d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.552+0000 7f662e7fc640 1 -- 172.21.15.103:0/899361380 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630032070 con 0x562264a31410 2024-09-19T07:04:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.553+0000 7f662e7fc640 1 -- 172.21.15.103:0/899361380 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663001f040 con 0x562264a31410 2024-09-19T07:04:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.553+0000 7f662e7fc640 1 -- 172.21.15.103:0/899361380 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663001e040 con 0x562264a31410 2024-09-19T07:04:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.553+0000 7f664de26740 1 -- 172.21.15.103:0/899361380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a31410 msgr2=0x562264a539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.553+0000 7f664de26740 1 --2- 172.21.15.103:0/899361380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a31410 0x562264a539b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x562264a4e950 tx=0x7f6630019000 comp rx=0 tx=0).stop 2024-09-19T07:04:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.553+0000 7f664de26740 1 -- 172.21.15.103:0/899361380 shutdown_connections 2024-09-19T07:04:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.553+0000 7f664de26740 1 --2- 172.21.15.103:0/899361380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a3bcb0 0x562264a53ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.553+0000 7f664de26740 1 --2- 172.21.15.103:0/899361380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a31410 0x562264a539b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.553+0000 7f664de26740 1 --2- 172.21.15.103:0/899361380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a39490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.553+0000 7f664de26740 1 -- 172.21.15.103:0/899361380 >> 172.21.15.103:0/899361380 conn(0x562264a2d060 msgr2=0x562264a3ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.553+0000 7f664de26740 1 -- 172.21.15.103:0/899361380 shutdown_connections 2024-09-19T07:04:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.553+0000 7f664de26740 1 -- 172.21.15.103:0/899361380 wait complete. 2024-09-19T07:04:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.554+0000 7f664de26740 1 -- start start 2024-09-19T07:04:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.554+0000 7f664de26740 1 Processor -- start 2024-09-19T07:04:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.554+0000 7f664de26740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a35ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.554+0000 7f664de26740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 0x562264a361e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.554+0000 7f664de26740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a3bcb0 0x562264a36720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.554+0000 7f664de26740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66441b0e20 con 0x562264a30af0 2024-09-19T07:04:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.554+0000 7f664de26740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66441b0750 con 0x562264a3bcb0 2024-09-19T07:04:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.554+0000 7f664de26740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66441ad9b0 con 0x562264a31410 2024-09-19T07:04:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.554+0000 7f664de26740 10 client.0 _ll_register_callbacks cb 0x7f6638009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.554+0000 7f6648e29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a35ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.554+0000 7f6648e29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a35ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43196/0 (socket says 172.21.15.103:43196) 2024-09-19T07:04:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.554+0000 7f6648e29640 1 -- 172.21.15.103:0/1607641681 learned_addr learned my addr 172.21.15.103:0/1607641681 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.555+0000 7f664962a640 1 --2- 172.21.15.103:0/1607641681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a3bcb0 0x562264a36720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.555+0000 7f6643fff640 1 --2- 172.21.15.103:0/1607641681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 0x562264a361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.555+0000 7f6648e29640 1 -- 172.21.15.103:0/1607641681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 msgr2=0x562264a361e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.555+0000 7f6648e29640 1 --2- 172.21.15.103:0/1607641681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 0x562264a361e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.555+0000 7f6648e29640 1 -- 172.21.15.103:0/1607641681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a3bcb0 msgr2=0x562264a36720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.555+0000 7f6648e29640 1 --2- 172.21.15.103:0/1607641681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a3bcb0 0x562264a36720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.555+0000 7f6648e29640 1 -- 172.21.15.103:0/1607641681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562264a3b770 con 0x562264a30af0 2024-09-19T07:04:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.555+0000 7f6648e29640 1 --2- 172.21.15.103:0/1607641681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a35ca0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f6638003b10 tx=0x7f6638031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df9a2c9c877c2f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.555+0000 7f6623fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.555+0000 7f6623fff640 1 -- 172.21.15.103:0/1607641681 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638035070 con 0x562264a30af0 2024-09-19T07:04:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.555+0000 7f6623fff640 1 -- 172.21.15.103:0/1607641681 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f66380037f0 con 0x562264a30af0 2024-09-19T07:04:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.555+0000 7f664de26740 1 -- 172.21.15.103:0/1607641681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f663001f040 con 0x562264a30af0 2024-09-19T07:04:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.555+0000 7f6623fff640 1 -- 172.21.15.103:0/1607641681 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6638050070 con 0x562264a30af0 2024-09-19T07:04:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.555+0000 7f664de26740 10 client.20538 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.556+0000 7f664de26740 1 -- 172.21.15.103:0/1607641681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562264ab6b90 con 0x562264a30af0 2024-09-19T07:04:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.556+0000 7f6623fff640 1 -- 172.21.15.103:0/1607641681 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663001f040 con 0x562264a30af0 2024-09-19T07:04:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.556+0000 7f6623fff640 1 -- 172.21.15.103:0/1607641681 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f663803b550 con 0x562264a30af0 2024-09-19T07:04:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.556+0000 7f664de26740 10 client.20538 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.556+0000 7f664de26740 10 client.20538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.556+0000 7f664de26740 20 client.20538 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.556+0000 7f664de26740 1 --2- 172.21.15.103:0/1607641681 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562264ab7340 0x562264ab9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.556+0000 7f664de26740 4 client.20538 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:04:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.556+0000 7f664de26740 1 -- 172.21.15.103:0/1607641681 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56226499cd70 con 0x562264ab7340 2024-09-19T07:04:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.556+0000 7f6643fff640 1 --2- 172.21.15.103:0/1607641681 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562264ab7340 0x562264ab9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.557+0000 7f6623fff640 10 client.20538 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.557+0000 7f6643fff640 1 --2- 172.21.15.103:0/1607641681 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562264ab7340 0x562264ab9760 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2185a169126e596 server_cookie=fd22aeb36b70302d in_seq=0 out_seq=0 2024-09-19T07:04:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.569+0000 7f6623fff640 1 -- 172.21.15.103:0/1607641681 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56226499cd70 con 0x562264ab7340 2024-09-19T07:04:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.569+0000 7f6623fff640 10 client.20538 handle_command_reply: tid=0 2024-09-19T07:04:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.569+0000 7f664de26740 1 client.20538 shutdown 2024-09-19T07:04:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.569+0000 7f664de26740 1 -- 172.21.15.103:0/1607641681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 msgr2=0x562264a35ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.569+0000 7f664de26740 1 --2- 172.21.15.103:0/1607641681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a35ca0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f6638003b10 tx=0x7f6638031510 comp rx=0 tx=0).stop 2024-09-19T07:04:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.569+0000 7f664de26740 1 -- 172.21.15.103:0/1607641681 shutdown_connections 2024-09-19T07:04:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.569+0000 7f664de26740 1 --2- 172.21.15.103:0/1607641681 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562264ab7340 0x562264ab9760 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.569+0000 7f664de26740 1 --2- 172.21.15.103:0/1607641681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562264a3bcb0 0x562264a36720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.569+0000 7f664de26740 1 --2- 172.21.15.103:0/1607641681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562264a31410 0x562264a361e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.569+0000 7f6623fff640 0 client.20538 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.569+0000 7f664de26740 1 --2- 172.21.15.103:0/1607641681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562264a30af0 0x562264a35ca0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.569+0000 7f664de26740 1 -- 172.21.15.103:0/1607641681 >> 172.21.15.103:0/1607641681 conn(0x562264a2d060 msgr2=0x562264a2b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.569+0000 7f664de26740 1 -- 172.21.15.103:0/1607641681 shutdown_connections 2024-09-19T07:04:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.569+0000 7f664de26740 1 -- 172.21.15.103:0/1607641681 wait complete. 2024-09-19T07:04:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.569+0000 7f664de26740 20 client.20538 trim_cache size 0 max 16384 2024-09-19T07:04:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:04:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:04:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:04:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:04:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.570+0000 7f664b08d640 1 -- 172.21.15.103:0/1967364299 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66340a34e0 msgr2=0x7f66340a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.570+0000 7f664b08d640 1 --2- 172.21.15.103:0/1967364299 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66340a34e0 0x7f66340a59a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6630006fd0 tx=0x7f6630013650 comp rx=0 tx=0).stop 2024-09-19T07:04:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.570+0000 7f664b08d640 1 -- 172.21.15.103:0/1967364299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f664410c110 msgr2=0x7f6644101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.570+0000 7f664b08d640 1 --2- 172.21.15.103:0/1967364299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f664410c110 0x7f6644101390 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f663c00cce0 tx=0x7f663c00a620 comp rx=0 tx=0).stop 2024-09-19T07:04:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.570+0000 7f664b08d640 1 -- 172.21.15.103:0/1967364299 shutdown_connections 2024-09-19T07:04:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.570+0000 7f664b08d640 1 --2- 172.21.15.103:0/1967364299 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66340a34e0 0x7f66340a59a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.570+0000 7f664b08d640 1 --2- 172.21.15.103:0/1967364299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f664410c110 0x7f6644101390 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.570+0000 7f664b08d640 1 --2- 172.21.15.103:0/1967364299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66441035e0 0x7f6644100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.570+0000 7f664b08d640 1 --2- 172.21.15.103:0/1967364299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6644102cc0 0x7f6644100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.570+0000 7f664b08d640 1 -- 172.21.15.103:0/1967364299 >> 172.21.15.103:0/1967364299 conn(0x7f66440fc6b0 msgr2=0x7f66440fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.570+0000 7f664b08d640 1 -- 172.21.15.103:0/1967364299 shutdown_connections 2024-09-19T07:04:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:05.570+0000 7f664b08d640 1 -- 172.21.15.103:0/1967364299 wait complete. 2024-09-19T07:04:06.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:06 smithi103 ceph-mon[31158]: pgmap v464: 97 pgs: 97 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-09-19T07:04:06.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:06 smithi103 ceph-mon[31158]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T07:04:06.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:06 smithi134 ceph-mon[35356]: pgmap v464: 97 pgs: 97 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-09-19T07:04:06.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:06 smithi134 ceph-mon[35356]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T07:04:06.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:06 smithi174 ceph-mon[35637]: pgmap v464: 97 pgs: 97 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-09-19T07:04:06.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:06 smithi174 ceph-mon[35637]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T07:04:08.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:08 smithi103 ceph-mon[31158]: pgmap v466: 97 pgs: 97 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 99 KiB/s wr, 37 op/s 2024-09-19T07:04:08.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:08 smithi134 ceph-mon[35356]: pgmap v466: 97 pgs: 97 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 99 KiB/s wr, 37 op/s 2024-09-19T07:04:08.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:08 smithi174 ceph-mon[35637]: pgmap v466: 97 pgs: 97 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 99 KiB/s wr, 37 op/s 2024-09-19T07:04:10.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:10 smithi103 ceph-mon[31158]: pgmap v467: 97 pgs: 97 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 99 KiB/s wr, 37 op/s 2024-09-19T07:04:10.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:10 smithi134 ceph-mon[35356]: pgmap v467: 97 pgs: 97 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 99 KiB/s wr, 37 op/s 2024-09-19T07:04:10.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:10 smithi174 ceph-mon[35637]: pgmap v467: 97 pgs: 97 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 99 KiB/s wr, 37 op/s 2024-09-19T07:04:12.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:12 smithi103 ceph-mon[31158]: pgmap v468: 97 pgs: 97 active+clean; 2.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 108 KiB/s wr, 38 op/s 2024-09-19T07:04:12.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:12 smithi103 ceph-mon[31158]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T07:04:12.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:12 smithi134 ceph-mon[35356]: pgmap v468: 97 pgs: 97 active+clean; 2.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 108 KiB/s wr, 38 op/s 2024-09-19T07:04:12.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:12 smithi134 ceph-mon[35356]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T07:04:12.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:12 smithi174 ceph-mon[35637]: pgmap v468: 97 pgs: 97 active+clean; 2.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 108 KiB/s wr, 38 op/s 2024-09-19T07:04:12.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:12 smithi174 ceph-mon[35637]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T07:04:14.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:14 smithi103 ceph-mon[31158]: pgmap v470: 97 pgs: 97 active+clean; 2.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 134 KiB/s wr, 48 op/s 2024-09-19T07:04:14.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:14 smithi134 ceph-mon[35356]: pgmap v470: 97 pgs: 97 active+clean; 2.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 134 KiB/s wr, 48 op/s 2024-09-19T07:04:14.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:14 smithi174 ceph-mon[35637]: pgmap v470: 97 pgs: 97 active+clean; 2.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 134 KiB/s wr, 48 op/s 2024-09-19T07:04:16.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:16 smithi103 ceph-mon[31158]: pgmap v471: 97 pgs: 97 active+clean; 2.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-09-19T07:04:16.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:16 smithi134 ceph-mon[35356]: pgmap v471: 97 pgs: 97 active+clean; 2.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-09-19T07:04:16.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:16 smithi174 ceph-mon[35637]: pgmap v471: 97 pgs: 97 active+clean; 2.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-09-19T07:04:18.015 INFO:tasks.quiescer.fs.[cephfs]:Total duration (58.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T07:04:18.015 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.45 2024-09-19T07:04:18.015 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:04:18.281 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:18.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:18 smithi103 ceph-mon[31158]: pgmap v472: 97 pgs: 97 active+clean; 2.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-09-19T07:04:18.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:18 smithi134 ceph-mon[35356]: pgmap v472: 97 pgs: 97 active+clean; 2.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-09-19T07:04:18.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:18 smithi174 ceph-mon[35637]: pgmap v472: 97 pgs: 97 active+clean; 2.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-09-19T07:04:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.659+0000 7fabbeb52640 1 -- 172.21.15.103:0/4035004214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8101350 msgr2=0x7fabb810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.659+0000 7fabbeb52640 1 --2- 172.21.15.103:0/4035004214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8101350 0x7fabb810d880 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009cd0 tx=0x7fabb002a8e0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.659+0000 7fabbeb52640 1 -- 172.21.15.103:0/4035004214 shutdown_connections 2024-09-19T07:04:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.659+0000 7fabbeb52640 1 --2- 172.21.15.103:0/4035004214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8101350 0x7fabb810d880 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.659+0000 7fabbeb52640 1 --2- 172.21.15.103:0/4035004214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabb81009b0 0x7fabb8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.659+0000 7fabbeb52640 1 --2- 172.21.15.103:0/4035004214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb81069c0 0x7fabb8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.659+0000 7fabbeb52640 1 -- 172.21.15.103:0/4035004214 >> 172.21.15.103:0/4035004214 conn(0x7fabb8077990 msgr2=0x7fabb80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.659+0000 7fabbeb52640 1 -- 172.21.15.103:0/4035004214 shutdown_connections 2024-09-19T07:04:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.659+0000 7fabbeb52640 1 -- 172.21.15.103:0/4035004214 wait complete. 2024-09-19T07:04:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabbeb52640 1 Processor -- start 2024-09-19T07:04:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabbeb52640 1 -- start start 2024-09-19T07:04:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabbeb52640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb81009b0 0x7fabb8199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabbeb52640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8101350 0x7fabb81996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabbeb52640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabb81069c0 0x7fabb81a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabbeb52640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8110e00 con 0x7fabb8101350 2024-09-19T07:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabbeb52640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8110c80 con 0x7fabb81009b0 2024-09-19T07:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabbeb52640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8110f80 con 0x7fabb81069c0 2024-09-19T07:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabb7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8101350 0x7fabb81996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabb7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8101350 0x7fabb81996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42524/0 (socket says 172.21.15.103:42524) 2024-09-19T07:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabb7fff640 1 -- 172.21.15.103:0/2812925701 learned_addr learned my addr 172.21.15.103:0/2812925701 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabbc8ee640 1 --2- 172.21.15.103:0/2812925701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb81009b0 0x7fabb8199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabbd0ef640 1 --2- 172.21.15.103:0/2812925701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabb81069c0 0x7fabb81a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabb7fff640 1 -- 172.21.15.103:0/2812925701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabb81069c0 msgr2=0x7fabb81a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabb7fff640 1 --2- 172.21.15.103:0/2812925701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabb81069c0 0x7fabb81a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabb7fff640 1 -- 172.21.15.103:0/2812925701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb81009b0 msgr2=0x7fabb8199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabb7fff640 1 --2- 172.21.15.103:0/2812925701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb81009b0 0x7fabb8199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.660+0000 7fabb7fff640 1 -- 172.21.15.103:0/2812925701 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb81a4730 con 0x7fabb8101350 2024-09-19T07:04:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.661+0000 7fabb7fff640 1 --2- 172.21.15.103:0/2812925701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8101350 0x7fabb81996b0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7faba400c970 tx=0x7faba400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.661+0000 7fabb5ffb640 1 -- 172.21.15.103:0/2812925701 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4007bb0 con 0x7fabb8101350 2024-09-19T07:04:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.661+0000 7fabbeb52640 1 -- 172.21.15.103:0/2812925701 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb81a4900 con 0x7fabb8101350 2024-09-19T07:04:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.661+0000 7fabb5ffb640 1 -- 172.21.15.103:0/2812925701 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba4007d50 con 0x7fabb8101350 2024-09-19T07:04:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.661+0000 7fabb5ffb640 1 -- 172.21.15.103:0/2812925701 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba400f450 con 0x7fabb8101350 2024-09-19T07:04:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.661+0000 7fabbeb52640 1 -- 172.21.15.103:0/2812925701 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb81a4dd0 con 0x7fabb8101350 2024-09-19T07:04:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.663+0000 7fabb5ffb640 1 -- 172.21.15.103:0/2812925701 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7faba4005020 con 0x7fabb8101350 2024-09-19T07:04:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.664+0000 7fabc19cb740 1 Processor -- start 2024-09-19T07:04:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.664+0000 7fabb5ffb640 1 --2- 172.21.15.103:0/2812925701 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faba80a3460 0x7faba80a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.664+0000 7fabc19cb740 1 -- start start 2024-09-19T07:04:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.664+0000 7fabc19cb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af517af0 0x5626af517ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.664+0000 7fabc19cb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af518410 0x5626af522770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.664+0000 7fabbd0ef640 1 --2- 172.21.15.103:0/2812925701 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faba80a3460 0x7faba80a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.664+0000 7fabc19cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af522cb0 0x5626af5250a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.664+0000 7fabc19cb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5626af47c2f0 con 0x5626af518410 2024-09-19T07:04:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.664+0000 7fabc19cb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5626af424270 con 0x5626af517af0 2024-09-19T07:04:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.664+0000 7fabb5ffb640 1 -- 172.21.15.103:0/2812925701 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7faba40cfb30 con 0x7fabb8101350 2024-09-19T07:04:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.664+0000 7fabc19cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5626af4782b0 con 0x5626af522cb0 2024-09-19T07:04:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.664+0000 7fabb7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af518410 0x5626af522770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.664+0000 7fabbc8ee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af517af0 0x5626af517ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.664+0000 7fabb7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af518410 0x5626af522770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42528/0 (socket says 172.21.15.103:42528) 2024-09-19T07:04:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.664+0000 7fabb7fff640 1 -- 172.21.15.103:0/1311609935 learned_addr learned my addr 172.21.15.103:0/1311609935 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.664+0000 7fabbd0ef640 1 --2- 172.21.15.103:0/1311609935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af522cb0 0x5626af5250a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.665+0000 7fabb7fff640 1 -- 172.21.15.103:0/1311609935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af522cb0 msgr2=0x5626af5250a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.665+0000 7fabb7fff640 1 --2- 172.21.15.103:0/1311609935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af522cb0 0x5626af5250a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.665+0000 7fabb7fff640 1 -- 172.21.15.103:0/1311609935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af517af0 msgr2=0x5626af517ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.665+0000 7fabb7fff640 1 --2- 172.21.15.103:0/1311609935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af517af0 0x5626af517ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.665+0000 7fabb7fff640 1 -- 172.21.15.103:0/1311609935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626af423340 con 0x5626af518410 2024-09-19T07:04:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.665+0000 7fabbc8ee640 1 --2- 172.21.15.103:0/1311609935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af517af0 0x5626af517ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.665+0000 7fabb7fff640 1 --2- 172.21.15.103:0/1311609935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af518410 0x5626af522770 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7faba40987c0 tx=0x7faba409c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8099f857f7fe5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.665+0000 7fabbd0ef640 1 --2- 172.21.15.103:0/2812925701 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faba80a3460 0x7faba80a5920 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fabb0006010 tx=0x7fabb0031770 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.665+0000 7faba2ffd640 1 -- 172.21.15.103:0/1311609935 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba401d040 con 0x5626af518410 2024-09-19T07:04:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.665+0000 7faba2ffd640 1 -- 172.21.15.103:0/1311609935 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba40a4a60 con 0x5626af518410 2024-09-19T07:04:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.665+0000 7faba2ffd640 1 -- 172.21.15.103:0/1311609935 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba40985b0 con 0x5626af518410 2024-09-19T07:04:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.666+0000 7fabc19cb740 1 -- 172.21.15.103:0/1311609935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af518410 msgr2=0x5626af522770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.666+0000 7fabc19cb740 1 --2- 172.21.15.103:0/1311609935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af518410 0x5626af522770 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7faba40987c0 tx=0x7faba409c630 comp rx=0 tx=0).stop 2024-09-19T07:04:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.666+0000 7fabc19cb740 1 -- 172.21.15.103:0/1311609935 shutdown_connections 2024-09-19T07:04:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.666+0000 7fabc19cb740 1 --2- 172.21.15.103:0/1311609935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af522cb0 0x5626af5250a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.666+0000 7fabc19cb740 1 --2- 172.21.15.103:0/1311609935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af518410 0x5626af522770 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.666+0000 7fabc19cb740 1 --2- 172.21.15.103:0/1311609935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af517af0 0x5626af517ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.666+0000 7fabc19cb740 1 -- 172.21.15.103:0/1311609935 >> 172.21.15.103:0/1311609935 conn(0x5626af514060 msgr2=0x5626af514440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.666+0000 7fabc19cb740 1 -- 172.21.15.103:0/1311609935 shutdown_connections 2024-09-19T07:04:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.666+0000 7fabc19cb740 1 -- 172.21.15.103:0/1311609935 wait complete. 2024-09-19T07:04:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.666+0000 7fabc19cb740 1 -- start start 2024-09-19T07:04:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabc19cb740 1 Processor -- start 2024-09-19T07:04:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabc19cb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af51fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabc19cb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af518410 0x5626af5200b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabc19cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af522cb0 0x5626af5205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabc19cb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5626af525880 con 0x5626af517af0 2024-09-19T07:04:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabc19cb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5626af405290 con 0x5626af518410 2024-09-19T07:04:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabc19cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5626af525a00 con 0x5626af522cb0 2024-09-19T07:04:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabc19cb740 10 client.0 _ll_register_callbacks cb 0x5626af436140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabbc8ee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af51fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabbc8ee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af51fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42530/0 (socket says 172.21.15.103:42530) 2024-09-19T07:04:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabbc8ee640 1 -- 172.21.15.103:0/201474252 learned_addr learned my addr 172.21.15.103:0/201474252 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabbd0ef640 1 --2- 172.21.15.103:0/201474252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af522cb0 0x5626af5205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabb7fff640 1 --2- 172.21.15.103:0/201474252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af518410 0x5626af5200b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabbc8ee640 1 -- 172.21.15.103:0/201474252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af522cb0 msgr2=0x5626af5205f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabbc8ee640 1 --2- 172.21.15.103:0/201474252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af522cb0 0x5626af5205f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabbc8ee640 1 -- 172.21.15.103:0/201474252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af518410 msgr2=0x5626af5200b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabbc8ee640 1 --2- 172.21.15.103:0/201474252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af518410 0x5626af5200b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.667+0000 7fabbc8ee640 1 -- 172.21.15.103:0/201474252 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626af18f210 con 0x5626af517af0 2024-09-19T07:04:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.668+0000 7fabbc8ee640 1 --2- 172.21.15.103:0/201474252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af51fb70 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fabac00ba30 tx=0x7fabac00bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212dd4568ae849d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.668+0000 7fab97fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.668+0000 7fab97fff640 1 -- 172.21.15.103:0/201474252 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac011440 con 0x5626af517af0 2024-09-19T07:04:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.668+0000 7fab97fff640 1 -- 172.21.15.103:0/201474252 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fabac0115e0 con 0x5626af517af0 2024-09-19T07:04:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.668+0000 7fabc19cb740 1 -- 172.21.15.103:0/201474252 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba40a4a60 con 0x5626af517af0 2024-09-19T07:04:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.668+0000 7fab97fff640 1 -- 172.21.15.103:0/201474252 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fabac0117e0 con 0x5626af517af0 2024-09-19T07:04:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.668+0000 7fabc19cb740 10 client.20550 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.668+0000 7fabc19cb740 1 -- 172.21.15.103:0/201474252 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5626af25b620 con 0x5626af517af0 2024-09-19T07:04:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.669+0000 7fab97fff640 1 -- 172.21.15.103:0/201474252 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba40a4a60 con 0x5626af517af0 2024-09-19T07:04:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.669+0000 7fab97fff640 1 -- 172.21.15.103:0/201474252 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fabac025a10 con 0x5626af517af0 2024-09-19T07:04:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.669+0000 7fabc19cb740 10 client.20550 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.669+0000 7fabc19cb740 10 client.20550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.669+0000 7fabc19cb740 20 client.20550 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.669+0000 7fabc19cb740 1 --2- 172.21.15.103:0/201474252 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5626af532be0 0x5626af535010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.669+0000 7fabc19cb740 4 client.20550 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.669+0000 7fabc19cb740 1 -- 172.21.15.103:0/201474252 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626af489a20 con 0x5626af532be0 2024-09-19T07:04:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.669+0000 7fabb7fff640 1 --2- 172.21.15.103:0/201474252 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5626af532be0 0x5626af535010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.670+0000 7fab97fff640 10 client.20550 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.670+0000 7fabb7fff640 1 --2- 172.21.15.103:0/201474252 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5626af532be0 0x5626af535010 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9c6bbbb05865484 server_cookie=e54ea73f88559434 in_seq=0 out_seq=0 2024-09-19T07:04:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.670+0000 7fab97fff640 1 -- 172.21.15.103:0/201474252 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626af489a20 con 0x5626af532be0 2024-09-19T07:04:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.670+0000 7fab97fff640 10 client.20550 handle_command_reply: tid=0 2024-09-19T07:04:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.670+0000 7fabc19cb740 1 client.20550 shutdown 2024-09-19T07:04:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.671+0000 7fabc19cb740 1 -- 172.21.15.103:0/201474252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 msgr2=0x5626af51fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.671+0000 7fabc19cb740 1 --2- 172.21.15.103:0/201474252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af51fb70 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fabac00ba30 tx=0x7fabac00bef0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.671+0000 7fabc19cb740 1 -- 172.21.15.103:0/201474252 shutdown_connections 2024-09-19T07:04:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.671+0000 7fabc19cb740 1 --2- 172.21.15.103:0/201474252 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5626af532be0 0x5626af535010 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.671+0000 7fabc19cb740 1 --2- 172.21.15.103:0/201474252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af522cb0 0x5626af5205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.671+0000 7fab97fff640 0 client.20550 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.671+0000 7fabc19cb740 1 --2- 172.21.15.103:0/201474252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af518410 0x5626af5200b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.671+0000 7fabc19cb740 1 --2- 172.21.15.103:0/201474252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af51fb70 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.671+0000 7fabc19cb740 1 -- 172.21.15.103:0/201474252 >> 172.21.15.103:0/201474252 conn(0x5626af514060 msgr2=0x5626af519dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.671+0000 7fabc19cb740 1 -- 172.21.15.103:0/201474252 shutdown_connections 2024-09-19T07:04:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.671+0000 7fabc19cb740 1 -- 172.21.15.103:0/201474252 wait complete. 2024-09-19T07:04:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.671+0000 7fabc19cb740 20 client.20550 trim_cache size 0 max 16384 2024-09-19T07:04:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.685+0000 7fabc19cb740 1 Processor -- start 2024-09-19T07:04:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.685+0000 7fabc19cb740 1 -- start start 2024-09-19T07:04:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabc19cb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af520420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabc19cb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af518410 0x5626af519c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabc19cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af522cb0 0x5626af53a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabc19cb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81ad850 con 0x5626af517af0 2024-09-19T07:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabc19cb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81af390 con 0x5626af518410 2024-09-19T07:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabc19cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81adf20 con 0x5626af522cb0 2024-09-19T07:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabbc8ee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af520420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabbc8ee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af520420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42534/0 (socket says 172.21.15.103:42534) 2024-09-19T07:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabbc8ee640 1 -- 172.21.15.103:0/2783171919 learned_addr learned my addr 172.21.15.103:0/2783171919 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabb7fff640 1 --2- 172.21.15.103:0/2783171919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af518410 0x5626af519c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabbd0ef640 1 --2- 172.21.15.103:0/2783171919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af522cb0 0x5626af53a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabbc8ee640 1 -- 172.21.15.103:0/2783171919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af522cb0 msgr2=0x5626af53a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabbc8ee640 1 --2- 172.21.15.103:0/2783171919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af522cb0 0x5626af53a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabbc8ee640 1 -- 172.21.15.103:0/2783171919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af518410 msgr2=0x5626af519c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabbc8ee640 1 --2- 172.21.15.103:0/2783171919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af518410 0x5626af519c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabbc8ee640 1 -- 172.21.15.103:0/2783171919 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626af513e90 con 0x5626af517af0 2024-09-19T07:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabb7fff640 1 --2- 172.21.15.103:0/2783171919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af518410 0x5626af519c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7fabbc8ee640 1 --2- 172.21.15.103:0/2783171919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af520420 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fabac000c00 tx=0x7fabac003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f134a8a5acc08c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.686+0000 7faba27fc640 1 -- 172.21.15.103:0/2783171919 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac011500 con 0x5626af517af0 2024-09-19T07:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.687+0000 7faba27fc640 1 -- 172.21.15.103:0/2783171919 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabac02a070 con 0x5626af517af0 2024-09-19T07:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.687+0000 7faba27fc640 1 -- 172.21.15.103:0/2783171919 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac0256d0 con 0x5626af517af0 2024-09-19T07:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.687+0000 7fabc19cb740 1 -- 172.21.15.103:0/2783171919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 msgr2=0x5626af520420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.687+0000 7fabc19cb740 1 --2- 172.21.15.103:0/2783171919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af520420 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fabac000c00 tx=0x7fabac003ee0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.687+0000 7fabc19cb740 1 -- 172.21.15.103:0/2783171919 shutdown_connections 2024-09-19T07:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.687+0000 7fabc19cb740 1 --2- 172.21.15.103:0/2783171919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af522cb0 0x5626af53a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.687+0000 7fabc19cb740 1 --2- 172.21.15.103:0/2783171919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af518410 0x5626af519c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.687+0000 7fabc19cb740 1 --2- 172.21.15.103:0/2783171919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af520420 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.687+0000 7fabc19cb740 1 -- 172.21.15.103:0/2783171919 >> 172.21.15.103:0/2783171919 conn(0x5626af514060 msgr2=0x5626af519550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.687+0000 7fabc19cb740 1 -- 172.21.15.103:0/2783171919 shutdown_connections 2024-09-19T07:04:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.687+0000 7fabc19cb740 1 -- 172.21.15.103:0/2783171919 wait complete. 2024-09-19T07:04:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.688+0000 7fabc19cb740 1 -- start start 2024-09-19T07:04:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.688+0000 7fabc19cb740 1 Processor -- start 2024-09-19T07:04:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.688+0000 7fabc19cb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af51ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.688+0000 7fabc19cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af518410 0x5626af51cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.688+0000 7fabc19cb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af522cb0 0x5626af51d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.688+0000 7fabc19cb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81aecc0 con 0x5626af517af0 2024-09-19T07:04:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.688+0000 7fabc19cb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81ae5f0 con 0x5626af522cb0 2024-09-19T07:04:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.688+0000 7fabc19cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81ab850 con 0x5626af518410 2024-09-19T07:04:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.688+0000 7fabc19cb740 10 client.0 _ll_register_callbacks cb 0x7fabac004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.688+0000 7fabbc8ee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af51ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.688+0000 7fabbc8ee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af51ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42546/0 (socket says 172.21.15.103:42546) 2024-09-19T07:04:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.688+0000 7fabbc8ee640 1 -- 172.21.15.103:0/2820393257 learned_addr learned my addr 172.21.15.103:0/2820393257 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.688+0000 7fabbd0ef640 1 --2- 172.21.15.103:0/2820393257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af522cb0 0x5626af51d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.688+0000 7fabb7fff640 1 --2- 172.21.15.103:0/2820393257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af518410 0x5626af51cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.689+0000 7fabbc8ee640 1 -- 172.21.15.103:0/2820393257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af518410 msgr2=0x5626af51cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.689+0000 7fabbc8ee640 1 --2- 172.21.15.103:0/2820393257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af518410 0x5626af51cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.689+0000 7fabbc8ee640 1 -- 172.21.15.103:0/2820393257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af522cb0 msgr2=0x5626af51d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.689+0000 7fabbc8ee640 1 --2- 172.21.15.103:0/2820393257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af522cb0 0x5626af51d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.689+0000 7fabbc8ee640 1 -- 172.21.15.103:0/2820393257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626af59c940 con 0x5626af517af0 2024-09-19T07:04:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.689+0000 7fabbc8ee640 1 --2- 172.21.15.103:0/2820393257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af51ca90 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fabac028040 tx=0x7fabac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732bf608499d3657 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.689+0000 7fab97fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.689+0000 7fab97fff640 1 -- 172.21.15.103:0/2820393257 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac0055c0 con 0x5626af517af0 2024-09-19T07:04:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.689+0000 7fabc19cb740 1 -- 172.21.15.103:0/2820393257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabac02a070 con 0x5626af517af0 2024-09-19T07:04:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.689+0000 7fab97fff640 1 -- 172.21.15.103:0/2820393257 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fabac017040 con 0x5626af517af0 2024-09-19T07:04:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.689+0000 7fab97fff640 1 -- 172.21.15.103:0/2820393257 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fabac025470 con 0x5626af517af0 2024-09-19T07:04:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.689+0000 7fabc19cb740 10 client.20556 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.689+0000 7fabc19cb740 1 -- 172.21.15.103:0/2820393257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5626af59db50 con 0x5626af517af0 2024-09-19T07:04:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.689+0000 7fab97fff640 1 -- 172.21.15.103:0/2820393257 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabac02a070 con 0x5626af517af0 2024-09-19T07:04:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.689+0000 7fab97fff640 1 -- 172.21.15.103:0/2820393257 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fabac01b810 con 0x5626af517af0 2024-09-19T07:04:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.690+0000 7fabc19cb740 10 client.20556 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.690+0000 7fabc19cb740 10 client.20556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.690+0000 7fabc19cb740 20 client.20556 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.690+0000 7fabc19cb740 1 --2- 172.21.15.103:0/2820393257 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5626af59e160 0x5626af5a0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.690+0000 7fabc19cb740 4 client.20556 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:04:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.690+0000 7fabc19cb740 1 -- 172.21.15.103:0/2820393257 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626af483d70 con 0x5626af59e160 2024-09-19T07:04:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.690+0000 7fabb7fff640 1 --2- 172.21.15.103:0/2820393257 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5626af59e160 0x5626af5a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.690+0000 7fab97fff640 10 client.20556 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.690+0000 7fabb7fff640 1 --2- 172.21.15.103:0/2820393257 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5626af59e160 0x5626af5a0580 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbcc43286f6c84be server_cookie=d7a52f04dcd29422 in_seq=0 out_seq=0 2024-09-19T07:04:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.702+0000 7fab97fff640 1 -- 172.21.15.103:0/2820393257 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5626af483d70 con 0x5626af59e160 2024-09-19T07:04:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.702+0000 7fab97fff640 10 client.20556 handle_command_reply: tid=0 2024-09-19T07:04:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.702+0000 7fabc19cb740 1 client.20556 shutdown 2024-09-19T07:04:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.702+0000 7fabc19cb740 1 -- 172.21.15.103:0/2820393257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 msgr2=0x5626af51ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.702+0000 7fabc19cb740 1 --2- 172.21.15.103:0/2820393257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af51ca90 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fabac028040 tx=0x7fabac0042f0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.702+0000 7fabc19cb740 1 -- 172.21.15.103:0/2820393257 shutdown_connections 2024-09-19T07:04:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.702+0000 7fabc19cb740 1 --2- 172.21.15.103:0/2820393257 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5626af59e160 0x5626af5a0580 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.702+0000 7fabc19cb740 1 --2- 172.21.15.103:0/2820393257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626af522cb0 0x5626af51d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.702+0000 7fabc19cb740 1 --2- 172.21.15.103:0/2820393257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5626af518410 0x5626af51cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.702+0000 7fabc19cb740 1 --2- 172.21.15.103:0/2820393257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626af517af0 0x5626af51ca90 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.702+0000 7fab97fff640 0 client.20556 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.702+0000 7fabc19cb740 1 -- 172.21.15.103:0/2820393257 >> 172.21.15.103:0/2820393257 conn(0x5626af514060 msgr2=0x5626af519550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.702+0000 7fabc19cb740 1 -- 172.21.15.103:0/2820393257 shutdown_connections 2024-09-19T07:04:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.702+0000 7fabc19cb740 1 -- 172.21.15.103:0/2820393257 wait complete. 2024-09-19T07:04:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.702+0000 7fabc19cb740 20 client.20556 trim_cache size 0 max 16384 2024-09-19T07:04:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:04:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:04:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:04:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:04:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.703+0000 7fabbeb52640 1 -- 172.21.15.103:0/2812925701 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faba80a3460 msgr2=0x7faba80a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.703+0000 7fabbeb52640 1 --2- 172.21.15.103:0/2812925701 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faba80a3460 0x7faba80a5920 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fabb0006010 tx=0x7fabb0031770 comp rx=0 tx=0).stop 2024-09-19T07:04:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.703+0000 7fabbeb52640 1 -- 172.21.15.103:0/2812925701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8101350 msgr2=0x7fabb81996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.703+0000 7fabbeb52640 1 --2- 172.21.15.103:0/2812925701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8101350 0x7fabb81996b0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7faba400c970 tx=0x7faba400ce30 comp rx=0 tx=0).stop 2024-09-19T07:04:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.703+0000 7fabbeb52640 1 -- 172.21.15.103:0/2812925701 shutdown_connections 2024-09-19T07:04:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.703+0000 7fabbeb52640 1 --2- 172.21.15.103:0/2812925701 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7faba80a3460 0x7faba80a5920 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.703+0000 7fabbeb52640 1 --2- 172.21.15.103:0/2812925701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabb81069c0 0x7fabb81a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.703+0000 7fabbeb52640 1 --2- 172.21.15.103:0/2812925701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabb8101350 0x7fabb81996b0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.703+0000 7fabbeb52640 1 --2- 172.21.15.103:0/2812925701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb81009b0 0x7fabb8199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.703+0000 7fabbeb52640 1 -- 172.21.15.103:0/2812925701 >> 172.21.15.103:0/2812925701 conn(0x7fabb8077990 msgr2=0x7fabb8103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.703+0000 7fabbeb52640 1 -- 172.21.15.103:0/2812925701 shutdown_connections 2024-09-19T07:04:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:18.703+0000 7fabbeb52640 1 -- 172.21.15.103:0/2812925701 wait complete. 2024-09-19T07:04:18.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.728+0000 7f26d30de640 1 -- 172.21.15.103:0/1774047453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26cc100b30 msgr2=0x7f26cc100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:18.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.728+0000 7f26d30de640 1 --2- 172.21.15.103:0/1774047453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26cc100b30 0x7f26cc100f10 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009900 tx=0x7f26c402a470 comp rx=0 tx=0).stop 2024-09-19T07:04:18.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.728+0000 7f26d30de640 1 -- 172.21.15.103:0/1774047453 shutdown_connections 2024-09-19T07:04:18.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.728+0000 7f26d30de640 1 --2- 172.21.15.103:0/1774047453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26cc105f40 0x7f26cc1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.728+0000 7f26d30de640 1 --2- 172.21.15.103:0/1774047453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26cc101450 0x7f26cc105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.728+0000 7f26d30de640 1 --2- 172.21.15.103:0/1774047453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26cc100b30 0x7f26cc100f10 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.728+0000 7f26d30de640 1 -- 172.21.15.103:0/1774047453 >> 172.21.15.103:0/1774047453 conn(0x7f26cc0fc7a0 msgr2=0x7f26cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:18.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.728+0000 7f26d30de640 1 -- 172.21.15.103:0/1774047453 shutdown_connections 2024-09-19T07:04:18.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.728+0000 7f26d30de640 1 -- 172.21.15.103:0/1774047453 wait complete. 2024-09-19T07:04:18.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.728+0000 7f26d30de640 1 Processor -- start 2024-09-19T07:04:18.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.728+0000 7f26d30de640 1 -- start start 2024-09-19T07:04:18.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d30de640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26cc100b30 0x7f26cc1a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d30de640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26cc101450 0x7f26cc1a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d30de640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26cc105f40 0x7f26cc19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d30de640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc1185e0 con 0x7f26cc100b30 2024-09-19T07:04:18.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d30de640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc118460 con 0x7f26cc101450 2024-09-19T07:04:18.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d30de640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc118760 con 0x7f26cc105f40 2024-09-19T07:04:18.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d0e7a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26cc100b30 0x7f26cc1a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d0e7a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26cc100b30 0x7f26cc1a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42566/0 (socket says 172.21.15.103:42566) 2024-09-19T07:04:18.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d0e7a640 1 -- 172.21.15.103:0/99916195 learned_addr learned my addr 172.21.15.103:0/99916195 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:18.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26cbfff640 1 --2- 172.21.15.103:0/99916195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26cc101450 0x7f26cc1a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d167b640 1 --2- 172.21.15.103:0/99916195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26cc105f40 0x7f26cc19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d0e7a640 1 -- 172.21.15.103:0/99916195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26cc105f40 msgr2=0x7f26cc19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:18.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d0e7a640 1 --2- 172.21.15.103:0/99916195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26cc105f40 0x7f26cc19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d0e7a640 1 -- 172.21.15.103:0/99916195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26cc101450 msgr2=0x7f26cc1a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:18.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d0e7a640 1 --2- 172.21.15.103:0/99916195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26cc101450 0x7f26cc1a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d0e7a640 1 -- 172.21.15.103:0/99916195 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26cc19f080 con 0x7f26cc100b30 2024-09-19T07:04:18.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d167b640 1 --2- 172.21.15.103:0/99916195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26cc105f40 0x7f26cc19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:18.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26d0e7a640 1 --2- 172.21.15.103:0/99916195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26cc100b30 0x7f26cc1a4790 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f26c4006010 tx=0x7f26c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:18.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.729+0000 7f26c9ffb640 1 -- 172.21.15.103:0/99916195 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c403f030 con 0x7f26cc100b30 2024-09-19T07:04:18.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.730+0000 7f26d30de640 1 -- 172.21.15.103:0/99916195 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26cc19f310 con 0x7f26cc100b30 2024-09-19T07:04:18.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.730+0000 7f26c9ffb640 1 -- 172.21.15.103:0/99916195 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c4003ce0 con 0x7f26cc100b30 2024-09-19T07:04:18.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.730+0000 7f26c9ffb640 1 -- 172.21.15.103:0/99916195 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4035070 con 0x7f26cc100b30 2024-09-19T07:04:18.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.730+0000 7f26d30de640 1 -- 172.21.15.103:0/99916195 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26cc1ab660 con 0x7f26cc100b30 2024-09-19T07:04:18.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.731+0000 7f26d30de640 1 -- 172.21.15.103:0/99916195 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26cc105920 con 0x7f26cc100b30 2024-09-19T07:04:18.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.732+0000 7f26c9ffb640 1 -- 172.21.15.103:0/99916195 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f26c4042030 con 0x7f26cc100b30 2024-09-19T07:04:18.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.733+0000 7f26c9ffb640 1 --2- 172.21.15.103:0/99916195 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26b80a33b0 0x7f26b80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:18.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.733+0000 7f26c9ffb640 1 -- 172.21.15.103:0/99916195 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f26c40f06d0 con 0x7f26cc100b30 2024-09-19T07:04:18.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.733+0000 7f26cbfff640 1 --2- 172.21.15.103:0/99916195 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26b80a33b0 0x7f26b80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:18.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.733+0000 7f26cbfff640 1 --2- 172.21.15.103:0/99916195 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26b80a33b0 0x7f26b80a5870 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0fff00 tx=0x7f26bc00d040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:18.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.736+0000 7f26c9ffb640 1 -- 172.21.15.103:0/99916195 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f26c40bd150 con 0x7f26cc100b30 2024-09-19T07:04:18.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.938+0000 7f26d30de640 1 -- 172.21.15.103:0/99916195 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26cc1ab890 con 0x7f26cc100b30 2024-09-19T07:04:18.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.939+0000 7f26c9ffb640 1 -- 172.21.15.103:0/99916195 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f26c40bca50 con 0x7f26cc100b30 2024-09-19T07:04:18.941 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:04:18.942 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:04:18.943 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:04:18.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.941+0000 7f26d30de640 1 -- 172.21.15.103:0/99916195 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26b80a33b0 msgr2=0x7f26b80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:18.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.941+0000 7f26d30de640 1 --2- 172.21.15.103:0/99916195 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26b80a33b0 0x7f26b80a5870 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0fff00 tx=0x7f26bc00d040 comp rx=0 tx=0).stop 2024-09-19T07:04:18.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.941+0000 7f26d30de640 1 -- 172.21.15.103:0/99916195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26cc100b30 msgr2=0x7f26cc1a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:18.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.941+0000 7f26d30de640 1 --2- 172.21.15.103:0/99916195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26cc100b30 0x7f26cc1a4790 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f26c4006010 tx=0x7f26c40042f0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.941+0000 7f26d30de640 1 -- 172.21.15.103:0/99916195 shutdown_connections 2024-09-19T07:04:18.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.941+0000 7f26d30de640 1 --2- 172.21.15.103:0/99916195 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26b80a33b0 0x7f26b80a5870 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.941+0000 7f26d30de640 1 --2- 172.21.15.103:0/99916195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26cc105f40 0x7f26cc19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.941+0000 7f26d30de640 1 --2- 172.21.15.103:0/99916195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26cc101450 0x7f26cc1a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.941+0000 7f26d30de640 1 --2- 172.21.15.103:0/99916195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26cc100b30 0x7f26cc1a4790 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.941+0000 7f26d30de640 1 -- 172.21.15.103:0/99916195 >> 172.21.15.103:0/99916195 conn(0x7f26cc0fc7a0 msgr2=0x7f26cc1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.941+0000 7f26d30de640 1 -- 172.21.15.103:0/99916195 shutdown_connections 2024-09-19T07:04:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:18.941+0000 7f26d30de640 1 -- 172.21.15.103:0/99916195 wait complete. 2024-09-19T07:04:19.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:19 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/99916195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:19.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:19 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/99916195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:19.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:19 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/99916195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:19.623 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0' 2024-09-19T07:04:19.624 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0 2024-09-19T07:04:19.875 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:20.255 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:20 smithi103 ceph-mon[31158]: pgmap v473: 97 pgs: 97 active+clean; 2.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-09-19T07:04:20.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.344+0000 7f940cc8d640 1 -- 172.21.15.103:0/46962380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9408101420 msgr2=0x7f940810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.344+0000 7f940cc8d640 1 --2- 172.21.15.103:0/46962380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9408101420 0x7f940810d950 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f9400009cd0 tx=0x7f940002a8e0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.344+0000 7f940cc8d640 1 -- 172.21.15.103:0/46962380 shutdown_connections 2024-09-19T07:04:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.344+0000 7f940cc8d640 1 --2- 172.21.15.103:0/46962380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9408101420 0x7f940810d950 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.344+0000 7f940cc8d640 1 --2- 172.21.15.103:0/46962380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9408100a80 0x7f9408100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.344+0000 7f940cc8d640 1 --2- 172.21.15.103:0/46962380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9408106a80 0x7f9408106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.344+0000 7f940cc8d640 1 -- 172.21.15.103:0/46962380 >> 172.21.15.103:0/46962380 conn(0x7f94080fc7a0 msgr2=0x7f94080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.344+0000 7f940cc8d640 1 -- 172.21.15.103:0/46962380 shutdown_connections 2024-09-19T07:04:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.344+0000 7f940cc8d640 1 -- 172.21.15.103:0/46962380 wait complete. 2024-09-19T07:04:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.345+0000 7f940cc8d640 1 Processor -- start 2024-09-19T07:04:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.345+0000 7f940cc8d640 1 -- start start 2024-09-19T07:04:20.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f940cc8d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9408100a80 0x7f9408198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f940cc8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9408101420 0x7f9408198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f940cc8d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9408106a80 0x7f940819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f940cc8d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9408111fe0 con 0x7f9408101420 2024-09-19T07:04:20.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f940cc8d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9408111e60 con 0x7f9408100a80 2024-09-19T07:04:20.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f940cc8d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9408112160 con 0x7f9408106a80 2024-09-19T07:04:20.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f9405d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9408101420 0x7f9408198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f9405d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9408101420 0x7f9408198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42576/0 (socket says 172.21.15.103:42576) 2024-09-19T07:04:20.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f9405d9b640 1 -- 172.21.15.103:0/3109720387 learned_addr learned my addr 172.21.15.103:0/3109720387 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:20.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f9406d9d640 1 --2- 172.21.15.103:0/3109720387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9408106a80 0x7f940819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f940659c640 1 --2- 172.21.15.103:0/3109720387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9408100a80 0x7f9408198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f9405d9b640 1 -- 172.21.15.103:0/3109720387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9408106a80 msgr2=0x7f940819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:20.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f9405d9b640 1 --2- 172.21.15.103:0/3109720387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9408106a80 0x7f940819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f9405d9b640 1 -- 172.21.15.103:0/3109720387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9408100a80 msgr2=0x7f9408198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f9405d9b640 1 --2- 172.21.15.103:0/3109720387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9408100a80 0x7f9408198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f9405d9b640 1 -- 172.21.15.103:0/3109720387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940819d2e0 con 0x7f9408101420 2024-09-19T07:04:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f940659c640 1 --2- 172.21.15.103:0/3109720387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9408100a80 0x7f9408198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f9405d9b640 1 --2- 172.21.15.103:0/3109720387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9408101420 0x7f9408198870 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f93fc00c920 tx=0x7f93fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f93f77fe640 1 -- 172.21.15.103:0/3109720387 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc007c70 con 0x7f9408101420 2024-09-19T07:04:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f93f77fe640 1 -- 172.21.15.103:0/3109720387 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93fc007e10 con 0x7f9408101420 2024-09-19T07:04:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f940cc8d640 1 -- 172.21.15.103:0/3109720387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940819d5d0 con 0x7f9408101420 2024-09-19T07:04:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f93f77fe640 1 -- 172.21.15.103:0/3109720387 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc018890 con 0x7f9408101420 2024-09-19T07:04:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.346+0000 7f940cc8d640 1 -- 172.21.15.103:0/3109720387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94081a4eb0 con 0x7f9408101420 2024-09-19T07:04:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.349+0000 7f93f77fe640 1 -- 172.21.15.103:0/3109720387 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f93fc018a70 con 0x7f9408101420 2024-09-19T07:04:20.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.349+0000 7f940f066740 1 Processor -- start 2024-09-19T07:04:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.349+0000 7f940f066740 1 -- start start 2024-09-19T07:04:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.349+0000 7f940f066740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f94675c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.349+0000 7f940f066740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94676170 0x560f946804f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.349+0000 7f940f066740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94680a30 0x560f94682e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.349+0000 7f940f066740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560f945da160 con 0x560f94675850 2024-09-19T07:04:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.349+0000 7f940f066740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560f94581720 con 0x560f94680a30 2024-09-19T07:04:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.349+0000 7f940f066740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560f945d6120 con 0x560f94676170 2024-09-19T07:04:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.349+0000 7f93f77fe640 1 --2- 172.21.15.103:0/3109720387 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f93f00a33b0 0x7f93f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.349+0000 7f9406d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f94675c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.350+0000 7f9406d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f94675c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42588/0 (socket says 172.21.15.103:42588) 2024-09-19T07:04:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.350+0000 7f9406d9d640 1 -- 172.21.15.103:0/2842674320 learned_addr learned my addr 172.21.15.103:0/2842674320 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.350+0000 7f940659c640 1 --2- 172.21.15.103:0/2842674320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94676170 0x560f946804f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.350+0000 7f9405d9b640 1 --2- 172.21.15.103:0/2842674320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94680a30 0x560f94682e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.350+0000 7f9406d9d640 1 --2- 172.21.15.103:0/3109720387 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f93f00a33b0 0x7f93f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.350+0000 7f93f77fe640 1 -- 172.21.15.103:0/3109720387 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f93fc0cff40 con 0x7f9408101420 2024-09-19T07:04:20.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.350+0000 7f9406d9d640 1 -- 172.21.15.103:0/2842674320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94676170 msgr2=0x560f946804f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:20.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.350+0000 7f9406d9d640 1 --2- 172.21.15.103:0/2842674320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94676170 0x560f946804f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.350+0000 7f9406d9d640 1 -- 172.21.15.103:0/2842674320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94680a30 msgr2=0x560f94682e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:20.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.350+0000 7f9406d9d640 1 --2- 172.21.15.103:0/2842674320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94680a30 0x560f94682e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.350+0000 7f9406d9d640 1 -- 172.21.15.103:0/2842674320 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9434bba0 con 0x560f94675850 2024-09-19T07:04:20.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.350+0000 7f9406d9d640 1 --2- 172.21.15.103:0/2842674320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f94675c30 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f9400035000 tx=0x7f9400031d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb6dfd05566c1b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:20.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.350+0000 7f9406d9d640 1 --2- 172.21.15.103:0/3109720387 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f93f00a33b0 0x7f93f00a5870 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9400031dd0 tx=0x7f9400048000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:20.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.350+0000 7f93f4ff9640 1 -- 172.21.15.103:0/2842674320 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400031630 con 0x560f94675850 2024-09-19T07:04:20.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.350+0000 7f93f4ff9640 1 -- 172.21.15.103:0/2842674320 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9400036040 con 0x560f94675850 2024-09-19T07:04:20.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.350+0000 7f93f4ff9640 1 -- 172.21.15.103:0/2842674320 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400030890 con 0x560f94675850 2024-09-19T07:04:20.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.351+0000 7f940f066740 1 -- 172.21.15.103:0/2842674320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 msgr2=0x560f94675c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:20.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.351+0000 7f940f066740 1 --2- 172.21.15.103:0/2842674320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f94675c30 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f9400035000 tx=0x7f9400031d60 comp rx=0 tx=0).stop 2024-09-19T07:04:20.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.351+0000 7f940f066740 1 -- 172.21.15.103:0/2842674320 shutdown_connections 2024-09-19T07:04:20.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.351+0000 7f940f066740 1 --2- 172.21.15.103:0/2842674320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94680a30 0x560f94682e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.351+0000 7f940f066740 1 --2- 172.21.15.103:0/2842674320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94676170 0x560f946804f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.351+0000 7f940f066740 1 --2- 172.21.15.103:0/2842674320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f94675c30 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.351+0000 7f940f066740 1 -- 172.21.15.103:0/2842674320 >> 172.21.15.103:0/2842674320 conn(0x560f94671de0 msgr2=0x560f946721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:20.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.351+0000 7f940f066740 1 -- 172.21.15.103:0/2842674320 shutdown_connections 2024-09-19T07:04:20.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.351+0000 7f940f066740 1 -- 172.21.15.103:0/2842674320 wait complete. 2024-09-19T07:04:20.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.351+0000 7f940f066740 1 -- start start 2024-09-19T07:04:20.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f940f066740 1 Processor -- start 2024-09-19T07:04:20.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f940f066740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94675850 0x560f9467d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f940f066740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94676170 0x560f9467df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f940f066740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94680a30 0x560f9467e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f940f066740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560f94683600 con 0x560f94680a30 2024-09-19T07:04:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f940f066740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560f94562900 con 0x560f94676170 2024-09-19T07:04:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f940f066740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560f94683780 con 0x560f94675850 2024-09-19T07:04:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f940f066740 10 client.0 _ll_register_callbacks cb 0x560f94560780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f9406d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94680a30 0x560f9467e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f9406d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94680a30 0x560f9467e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42594/0 (socket says 172.21.15.103:42594) 2024-09-19T07:04:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f9406d9d640 1 -- 172.21.15.103:0/1939379101 learned_addr learned my addr 172.21.15.103:0/1939379101 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f940659c640 1 --2- 172.21.15.103:0/1939379101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94675850 0x560f9467d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f9405d9b640 1 --2- 172.21.15.103:0/1939379101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94676170 0x560f9467df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f9406d9d640 1 -- 172.21.15.103:0/1939379101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94675850 msgr2=0x560f9467d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f9406d9d640 1 --2- 172.21.15.103:0/1939379101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94675850 0x560f9467d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f9406d9d640 1 -- 172.21.15.103:0/1939379101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94676170 msgr2=0x560f9467df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f9406d9d640 1 --2- 172.21.15.103:0/1939379101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94676170 0x560f9467df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f9406d9d640 1 -- 172.21.15.103:0/1939379101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f945a2c50 con 0x560f94680a30 2024-09-19T07:04:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f940659c640 1 --2- 172.21.15.103:0/1939379101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94675850 0x560f9467d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f9405d9b640 1 --2- 172.21.15.103:0/1939379101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94676170 0x560f9467df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f9406d9d640 1 --2- 172.21.15.103:0/1939379101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94680a30 0x560f9467e440 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f9400031ef0 tx=0x7f9400031430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfed1387cb9c8cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.352+0000 7f93edffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.353+0000 7f93edffb640 1 -- 172.21.15.103:0/1939379101 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400004350 con 0x560f94680a30 2024-09-19T07:04:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.353+0000 7f93edffb640 1 -- 172.21.15.103:0/1939379101 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f94000044f0 con 0x560f94680a30 2024-09-19T07:04:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.353+0000 7f940f066740 1 -- 172.21.15.103:0/1939379101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9400036040 con 0x560f94680a30 2024-09-19T07:04:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.353+0000 7f93edffb640 1 -- 172.21.15.103:0/1939379101 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9400030610 con 0x560f94680a30 2024-09-19T07:04:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.353+0000 7f940f066740 10 client.20574 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.353+0000 7f940f066740 1 -- 172.21.15.103:0/1939379101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560f9468f350 con 0x560f94680a30 2024-09-19T07:04:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.353+0000 7f93edffb640 1 -- 172.21.15.103:0/1939379101 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9400036040 con 0x560f94680a30 2024-09-19T07:04:20.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.353+0000 7f93edffb640 1 -- 172.21.15.103:0/1939379101 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9400030070 con 0x560f94680a30 2024-09-19T07:04:20.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.353+0000 7f940f066740 10 client.20574 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:20.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.353+0000 7f940f066740 10 client.20574 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:04:20.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.353+0000 7f940f066740 20 client.20574 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:20.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.353+0000 7f940f066740 1 --2- 172.21.15.103:0/1939379101 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560f946906a0 0x560f94692a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.353+0000 7f940f066740 4 client.20574 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:20.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.353+0000 7f940f066740 1 -- 172.21.15.103:0/1939379101 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f945e7890 con 0x560f946906a0 2024-09-19T07:04:20.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.354+0000 7f940659c640 1 --2- 172.21.15.103:0/1939379101 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560f946906a0 0x560f94692a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.354+0000 7f940659c640 1 --2- 172.21.15.103:0/1939379101 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560f946906a0 0x560f94692a90 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a455c8db6696d60f server_cookie=324f8c75bfdfb304 in_seq=0 out_seq=0 2024-09-19T07:04:20.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.354+0000 7f93edffb640 10 client.20574 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:20.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.355+0000 7f93edffb640 1 -- 172.21.15.103:0/1939379101 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f945e7890 con 0x560f946906a0 2024-09-19T07:04:20.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.355+0000 7f93edffb640 10 client.20574 handle_command_reply: tid=0 2024-09-19T07:04:20.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.355+0000 7f940f066740 1 client.20574 shutdown 2024-09-19T07:04:20.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.355+0000 7f940f066740 1 -- 172.21.15.103:0/1939379101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94680a30 msgr2=0x560f9467e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.355+0000 7f940f066740 1 --2- 172.21.15.103:0/1939379101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94680a30 0x560f9467e440 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f9400031ef0 tx=0x7f9400031430 comp rx=0 tx=0).stop 2024-09-19T07:04:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.355+0000 7f940f066740 1 -- 172.21.15.103:0/1939379101 shutdown_connections 2024-09-19T07:04:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.355+0000 7f940f066740 1 --2- 172.21.15.103:0/1939379101 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560f946906a0 0x560f94692a90 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.355+0000 7f940f066740 1 --2- 172.21.15.103:0/1939379101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94680a30 0x560f9467e440 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.355+0000 7f93edffb640 0 client.20574 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.355+0000 7f940f066740 1 --2- 172.21.15.103:0/1939379101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94676170 0x560f9467df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.355+0000 7f940f066740 1 --2- 172.21.15.103:0/1939379101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94675850 0x560f9467d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.355+0000 7f940f066740 1 -- 172.21.15.103:0/1939379101 >> 172.21.15.103:0/1939379101 conn(0x560f94671de0 msgr2=0x560f94677ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.355+0000 7f940f066740 1 -- 172.21.15.103:0/1939379101 shutdown_connections 2024-09-19T07:04:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.355+0000 7f940f066740 1 -- 172.21.15.103:0/1939379101 wait complete. 2024-09-19T07:04:20.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.355+0000 7f940f066740 20 client.20574 trim_cache size 0 max 16384 2024-09-19T07:04:20.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f940f066740 1 Processor -- start 2024-09-19T07:04:20.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f940f066740 1 -- start start 2024-09-19T07:04:20.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f940f066740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f9467cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f940f066740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94676170 0x560f94677910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f940f066740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94680a30 0x560f9467e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f940f066740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94081ab6c0 con 0x560f94675850 2024-09-19T07:04:20.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f940f066740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f94081ad200 con 0x560f94680a30 2024-09-19T07:04:20.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f940f066740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94081abd90 con 0x560f94676170 2024-09-19T07:04:20.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f940659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f9467cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f940659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f9467cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42596/0 (socket says 172.21.15.103:42596) 2024-09-19T07:04:20.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f940659c640 1 -- 172.21.15.103:0/1961437840 learned_addr learned my addr 172.21.15.103:0/1961437840 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:20.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f9405d9b640 1 --2- 172.21.15.103:0/1961437840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94676170 0x560f94677910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f940659c640 1 -- 172.21.15.103:0/1961437840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94676170 msgr2=0x560f94677910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:20.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f940659c640 1 --2- 172.21.15.103:0/1961437840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94676170 0x560f94677910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f9406d9d640 1 --2- 172.21.15.103:0/1961437840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94680a30 0x560f9467e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f940659c640 1 -- 172.21.15.103:0/1961437840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94680a30 msgr2=0x560f9467e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:20.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f940659c640 1 --2- 172.21.15.103:0/1961437840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94680a30 0x560f9467e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f940659c640 1 -- 172.21.15.103:0/1961437840 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f94672e90 con 0x560f94675850 2024-09-19T07:04:20.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.372+0000 7f9406d9d640 1 --2- 172.21.15.103:0/1961437840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94680a30 0x560f9467e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:20.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.373+0000 7f940659c640 1 --2- 172.21.15.103:0/1961437840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f9467cce0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f93f8004700 tx=0x7f93f8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d48651022a5185f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:20.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.373+0000 7f93effff640 1 -- 172.21.15.103:0/1961437840 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8004020 con 0x560f94675850 2024-09-19T07:04:20.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.373+0000 7f93effff640 1 -- 172.21.15.103:0/1961437840 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f8005500 con 0x560f94675850 2024-09-19T07:04:20.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.373+0000 7f93effff640 1 -- 172.21.15.103:0/1961437840 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8004d30 con 0x560f94675850 2024-09-19T07:04:20.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.373+0000 7f940f066740 1 -- 172.21.15.103:0/1961437840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 msgr2=0x560f9467cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:20.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.373+0000 7f940f066740 1 --2- 172.21.15.103:0/1961437840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f9467cce0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f93f8004700 tx=0x7f93f8010800 comp rx=0 tx=0).stop 2024-09-19T07:04:20.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.373+0000 7f940f066740 1 -- 172.21.15.103:0/1961437840 shutdown_connections 2024-09-19T07:04:20.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.373+0000 7f940f066740 1 --2- 172.21.15.103:0/1961437840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94680a30 0x560f9467e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.373+0000 7f940f066740 1 --2- 172.21.15.103:0/1961437840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94676170 0x560f94677910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.373+0000 7f940f066740 1 --2- 172.21.15.103:0/1961437840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f9467cce0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.373+0000 7f940f066740 1 -- 172.21.15.103:0/1961437840 >> 172.21.15.103:0/1961437840 conn(0x560f94671de0 msgr2=0x560f9467f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:20.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.373+0000 7f940f066740 1 -- 172.21.15.103:0/1961437840 shutdown_connections 2024-09-19T07:04:20.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.373+0000 7f940f066740 1 -- 172.21.15.103:0/1961437840 wait complete. 2024-09-19T07:04:20.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940f066740 1 -- start start 2024-09-19T07:04:20.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940f066740 1 Processor -- start 2024-09-19T07:04:20.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940f066740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f94693fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940f066740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94676170 0x560f94694500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940f066740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94680a30 0x560f946f9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940f066740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94081acb30 con 0x560f94675850 2024-09-19T07:04:20.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940f066740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f94081ac460 con 0x560f94676170 2024-09-19T07:04:20.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940f066740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94081a96c0 con 0x560f94680a30 2024-09-19T07:04:20.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940f066740 10 client.0 _ll_register_callbacks cb 0x7f93fc00a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:20.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f94693fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f94693fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42600/0 (socket says 172.21.15.103:42600) 2024-09-19T07:04:20.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940659c640 1 -- 172.21.15.103:0/1604323818 learned_addr learned my addr 172.21.15.103:0/1604323818 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:20.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f9406d9d640 1 --2- 172.21.15.103:0/1604323818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94680a30 0x560f946f9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f9405d9b640 1 --2- 172.21.15.103:0/1604323818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94676170 0x560f94694500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940659c640 1 -- 172.21.15.103:0/1604323818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94680a30 msgr2=0x560f946f9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:20.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940659c640 1 --2- 172.21.15.103:0/1604323818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94680a30 0x560f946f9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940659c640 1 -- 172.21.15.103:0/1604323818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94676170 msgr2=0x560f94694500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:20.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940659c640 1 --2- 172.21.15.103:0/1604323818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94676170 0x560f94694500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.374+0000 7f940659c640 1 -- 172.21.15.103:0/1604323818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f94672a70 con 0x560f94675850 2024-09-19T07:04:20.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f9406d9d640 1 --2- 172.21.15.103:0/1604323818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94680a30 0x560f946f9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f9405d9b640 1 --2- 172.21.15.103:0/1604323818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94676170 0x560f94694500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f940659c640 1 --2- 172.21.15.103:0/1604323818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f94693fc0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f93f800ca90 tx=0x7f93f800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d091fe537f5ba511 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f93edffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f93edffb640 1 -- 172.21.15.103:0/1604323818 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f80043f0 con 0x560f94675850 2024-09-19T07:04:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f940f066740 1 -- 172.21.15.103:0/1604323818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93f8005500 con 0x560f94675850 2024-09-19T07:04:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f93edffb640 1 -- 172.21.15.103:0/1604323818 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f93f80035f0 con 0x560f94675850 2024-09-19T07:04:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f93edffb640 1 -- 172.21.15.103:0/1604323818 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f93f8035050 con 0x560f94675850 2024-09-19T07:04:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f940f066740 10 client.20580 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f940f066740 1 -- 172.21.15.103:0/1604323818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560f946fb630 con 0x560f94675850 2024-09-19T07:04:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f93edffb640 1 -- 172.21.15.103:0/1604323818 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f8005500 con 0x560f94675850 2024-09-19T07:04:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f93edffb640 1 -- 172.21.15.103:0/1604323818 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f93f8004e50 con 0x560f94675850 2024-09-19T07:04:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f940f066740 10 client.20580 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:20.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f940f066740 10 client.20580 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:04:20.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f940f066740 20 client.20580 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:20.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f940f066740 1 --2- 172.21.15.103:0/1604323818 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560f946fbd60 0x560f946fe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:20.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f940f066740 4 client.20580 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}] 2024-09-19T07:04:20.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.375+0000 7f940f066740 1 -- 172.21.15.103:0/1604323818 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}) -- 0x560f945e1be0 con 0x560f946fbd60 2024-09-19T07:04:20.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.376+0000 7f9405d9b640 1 --2- 172.21.15.103:0/1604323818 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560f946fbd60 0x560f946fe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:20.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.376+0000 7f9405d9b640 1 --2- 172.21.15.103:0/1604323818 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560f946fbd60 0x560f946fe150 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53b5dad9c1902d7b server_cookie=6e2dfe316262955 in_seq=0 out_seq=0 2024-09-19T07:04:20.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.376+0000 7f93edffb640 10 client.20580 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:20.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.387+0000 7f93edffb640 1 -- 172.21.15.103:0/1604323818 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560f945e1be0 con 0x560f946fbd60 2024-09-19T07:04:20.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.387+0000 7f93edffb640 10 client.20580 handle_command_reply: tid=0 2024-09-19T07:04:20.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.387+0000 7f940f066740 1 client.20580 shutdown 2024-09-19T07:04:20.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.387+0000 7f940f066740 1 -- 172.21.15.103:0/1604323818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 msgr2=0x560f94693fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:20.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.387+0000 7f940f066740 1 --2- 172.21.15.103:0/1604323818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f94693fc0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f93f800ca90 tx=0x7f93f800cac0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.387+0000 7f940f066740 1 -- 172.21.15.103:0/1604323818 shutdown_connections 2024-09-19T07:04:20.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.387+0000 7f940f066740 1 --2- 172.21.15.103:0/1604323818 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560f946fbd60 0x560f946fe150 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.387+0000 7f940f066740 1 --2- 172.21.15.103:0/1604323818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f94680a30 0x560f946f9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.387+0000 7f940f066740 1 --2- 172.21.15.103:0/1604323818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f94676170 0x560f94694500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.387+0000 7f940f066740 1 --2- 172.21.15.103:0/1604323818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f94675850 0x560f94693fc0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.387+0000 7f940f066740 1 -- 172.21.15.103:0/1604323818 >> 172.21.15.103:0/1604323818 conn(0x560f94671de0 msgr2=0x560f9467fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:20.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.387+0000 7f93edffb640 0 client.20580 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:20.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.387+0000 7f940f066740 1 -- 172.21.15.103:0/1604323818 shutdown_connections 2024-09-19T07:04:20.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.387+0000 7f940f066740 1 -- 172.21.15.103:0/1604323818 wait complete. 2024-09-19T07:04:20.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.387+0000 7f940f066740 20 client.20580 trim_cache size 0 max 16384 2024-09-19T07:04:20.390 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:04:20.390 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T07:04:20.390 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T07:04:20.390 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 50, 2024-09-19T07:04:20.390 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:04:20.390 INFO:teuthology.orchestra.run.smithi103.stdout: "a3c7bd9c": { 2024-09-19T07:04:20.390 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 50, 2024-09-19T07:04:20.390 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:04:20.390 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:04:20.390 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:04:20.390 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:04:20.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:04:20.390 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:04:20.391 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 149.4, 2024-09-19T07:04:20.391 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:04:20.391 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:04:20.391 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:04:20.391 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:04:20.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:04:20.391 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:04:20.391 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:04:20.391 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:04:20.391 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:04:20.391 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:04:20.391 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:04:20.391 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:04:20.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.388+0000 7f940cc8d640 1 -- 172.21.15.103:0/3109720387 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f93f00a33b0 msgr2=0x7f93f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:20.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.388+0000 7f940cc8d640 1 --2- 172.21.15.103:0/3109720387 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f93f00a33b0 0x7f93f00a5870 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9400031dd0 tx=0x7f9400048000 comp rx=0 tx=0).stop 2024-09-19T07:04:20.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.388+0000 7f940cc8d640 1 -- 172.21.15.103:0/3109720387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9408101420 msgr2=0x7f9408198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:20.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.388+0000 7f940cc8d640 1 --2- 172.21.15.103:0/3109720387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9408101420 0x7f9408198870 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f93fc00c920 tx=0x7f93fc00cde0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.388+0000 7f940cc8d640 1 -- 172.21.15.103:0/3109720387 shutdown_connections 2024-09-19T07:04:20.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.388+0000 7f940cc8d640 1 --2- 172.21.15.103:0/3109720387 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f93f00a33b0 0x7f93f00a5870 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.388+0000 7f940cc8d640 1 --2- 172.21.15.103:0/3109720387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9408106a80 0x7f940819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.388+0000 7f940cc8d640 1 --2- 172.21.15.103:0/3109720387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9408101420 0x7f9408198870 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.388+0000 7f940cc8d640 1 --2- 172.21.15.103:0/3109720387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9408100a80 0x7f9408198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:20.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.388+0000 7f940cc8d640 1 -- 172.21.15.103:0/3109720387 >> 172.21.15.103:0/3109720387 conn(0x7f94080fc7a0 msgr2=0x7f94080fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:20.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.388+0000 7f940cc8d640 1 -- 172.21.15.103:0/3109720387 shutdown_connections 2024-09-19T07:04:20.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:20.388+0000 7f940cc8d640 1 -- 172.21.15.103:0/3109720387 wait complete. 2024-09-19T07:04:20.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:20 smithi134 ceph-mon[35356]: pgmap v473: 97 pgs: 97 active+clean; 2.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-09-19T07:04:20.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:20 smithi174 ceph-mon[35637]: pgmap v473: 97 pgs: 97 active+clean; 2.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-09-19T07:04:21.099 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a3c7bd9c', quiesce took 3.1 seconds. Will release after: 26.349999999999998 2024-09-19T07:04:22.385 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:22 smithi174 ceph-mon[35637]: pgmap v474: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T07:04:22.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:22 smithi103 ceph-mon[31158]: pgmap v474: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T07:04:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:22 smithi134 ceph-mon[35356]: pgmap v474: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-09-19T07:04:23.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:23 smithi103 ceph-mon[31158]: pgmap v475: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-09-19T07:04:23.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:23 smithi134 ceph-mon[35356]: pgmap v475: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-09-19T07:04:23.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:23 smithi174 ceph-mon[35637]: pgmap v475: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-09-19T07:04:25.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:25 smithi174 ceph-mon[35637]: pgmap v476: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T07:04:25.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:25 smithi103 ceph-mon[31158]: pgmap v476: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T07:04:25.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:25 smithi134 ceph-mon[35356]: pgmap v476: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-19T07:04:27.220 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:04:27.483 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:27.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:27 smithi174 ceph-mon[35637]: pgmap v477: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-19T07:04:27.603 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:27 smithi103 ceph-mon[31158]: pgmap v477: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-19T07:04:27.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:27 smithi134 ceph-mon[35356]: pgmap v477: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-19T07:04:28.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.005+0000 7fd1c172d640 1 -- 172.21.15.103:0/1373316012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1bc100e20 msgr2=0x7fd1bc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:28.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.005+0000 7fd1c172d640 1 --2- 172.21.15.103:0/1373316012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1bc100e20 0x7fd1bc101200 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009900 tx=0x7fd1ac02a480 comp rx=0 tx=0).stop 2024-09-19T07:04:28.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.005+0000 7fd1c172d640 1 -- 172.21.15.103:0/1373316012 shutdown_connections 2024-09-19T07:04:28.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.005+0000 7fd1c172d640 1 --2- 172.21.15.103:0/1373316012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1bc102ef0 0x7fd1bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.005+0000 7fd1c172d640 1 --2- 172.21.15.103:0/1373316012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1bc102570 0x7fd1bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.005+0000 7fd1c172d640 1 --2- 172.21.15.103:0/1373316012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1bc100e20 0x7fd1bc101200 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.005+0000 7fd1c172d640 1 -- 172.21.15.103:0/1373316012 >> 172.21.15.103:0/1373316012 conn(0x7fd1bc0fc7d0 msgr2=0x7fd1bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:28.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.006+0000 7fd1c172d640 1 -- 172.21.15.103:0/1373316012 shutdown_connections 2024-09-19T07:04:28.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.006+0000 7fd1c172d640 1 -- 172.21.15.103:0/1373316012 wait complete. 2024-09-19T07:04:28.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.006+0000 7fd1c172d640 1 Processor -- start 2024-09-19T07:04:28.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.006+0000 7fd1c172d640 1 -- start start 2024-09-19T07:04:28.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.006+0000 7fd1c172d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1bc100e20 0x7fd1bc199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.006+0000 7fd1c172d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1bc102570 0x7fd1bc199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1c172d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1bc102ef0 0x7fd1bc1a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1c172d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc110fa0 con 0x7fd1bc102ef0 2024-09-19T07:04:28.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1c172d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc110e20 con 0x7fd1bc100e20 2024-09-19T07:04:28.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1c172d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc111120 con 0x7fd1bc102570 2024-09-19T07:04:28.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1bb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1bc102ef0 0x7fd1bc1a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:28.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1bb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1bc102ef0 0x7fd1bc1a43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45576/0 (socket says 172.21.15.103:45576) 2024-09-19T07:04:28.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1bb7fe640 1 -- 172.21.15.103:0/3604989414 learned_addr learned my addr 172.21.15.103:0/3604989414 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:28.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1ba7fc640 1 --2- 172.21.15.103:0/3604989414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1bc102570 0x7fd1bc199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:28.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1baffd640 1 --2- 172.21.15.103:0/3604989414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1bc100e20 0x7fd1bc199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:28.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1bb7fe640 1 -- 172.21.15.103:0/3604989414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1bc102570 msgr2=0x7fd1bc199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:28.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1bb7fe640 1 --2- 172.21.15.103:0/3604989414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1bc102570 0x7fd1bc199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1bb7fe640 1 -- 172.21.15.103:0/3604989414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1bc100e20 msgr2=0x7fd1bc199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:28.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1bb7fe640 1 --2- 172.21.15.103:0/3604989414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1bc100e20 0x7fd1bc199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1bb7fe640 1 -- 172.21.15.103:0/3604989414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc1a48f0 con 0x7fd1bc102ef0 2024-09-19T07:04:28.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1bb7fe640 1 --2- 172.21.15.103:0/3604989414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1bc102ef0 0x7fd1bc1a43b0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400b970 tx=0x7fd1b400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:28.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1a3fff640 1 -- 172.21.15.103:0/3604989414 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4019070 con 0x7fd1bc102ef0 2024-09-19T07:04:28.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1c172d640 1 -- 172.21.15.103:0/3604989414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1bc1a4ac0 con 0x7fd1bc102ef0 2024-09-19T07:04:28.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1a3fff640 1 -- 172.21.15.103:0/3604989414 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b4014440 con 0x7fd1bc102ef0 2024-09-19T07:04:28.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1a3fff640 1 -- 172.21.15.103:0/3604989414 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b40075d0 con 0x7fd1bc102ef0 2024-09-19T07:04:28.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.007+0000 7fd1c172d640 1 -- 172.21.15.103:0/3604989414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1bc1a4fd0 con 0x7fd1bc102ef0 2024-09-19T07:04:28.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.009+0000 7fd1a3fff640 1 -- 172.21.15.103:0/3604989414 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd1b401c070 con 0x7fd1bc102ef0 2024-09-19T07:04:28.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.010+0000 7fd1c3b06740 1 Processor -- start 2024-09-19T07:04:28.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.010+0000 7fd1c3b06740 1 -- start start 2024-09-19T07:04:28.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.010+0000 7fd1c3b06740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473e44a0 0x5611473e4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.010+0000 7fd1c3b06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e4dc0 0x5611473ef140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.010+0000 7fd1c3b06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473ef680 0x5611473f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.010+0000 7fd1c3b06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56114734a2e0 con 0x5611473e4dc0 2024-09-19T07:04:28.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.010+0000 7fd1c3b06740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5611472f1ce0 con 0x5611473e44a0 2024-09-19T07:04:28.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.010+0000 7fd1c3b06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5611473462a0 con 0x5611473ef680 2024-09-19T07:04:28.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.010+0000 7fd1a3fff640 1 --2- 172.21.15.103:0/3604989414 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1a40a33b0 0x7fd1a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.010+0000 7fd1baffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e4dc0 0x5611473ef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:28.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1baffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e4dc0 0x5611473ef140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45582/0 (socket says 172.21.15.103:45582) 2024-09-19T07:04:28.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1baffd640 1 -- 172.21.15.103:0/3880064257 learned_addr learned my addr 172.21.15.103:0/3880064257 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:28.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1ba7fc640 1 --2- 172.21.15.103:0/3880064257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473ef680 0x5611473f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:28.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1bb7fe640 1 --2- 172.21.15.103:0/3604989414 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1a40a33b0 0x7fd1a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:28.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1ba7fc640 1 --2- 172.21.15.103:0/3880064257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473e44a0 0x5611473e4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:28.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1a3fff640 1 -- 172.21.15.103:0/3604989414 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fd1b40cf2d0 con 0x7fd1bc102ef0 2024-09-19T07:04:28.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1baffd640 1 -- 172.21.15.103:0/3880064257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473ef680 msgr2=0x5611473f1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:28.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1baffd640 1 --2- 172.21.15.103:0/3880064257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473ef680 0x5611473f1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1baffd640 1 -- 172.21.15.103:0/3880064257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473e44a0 msgr2=0x5611473e4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:28.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1baffd640 1 --2- 172.21.15.103:0/3880064257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473e44a0 0x5611473e4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1baffd640 1 -- 172.21.15.103:0/3880064257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611472d6440 con 0x5611473e4dc0 2024-09-19T07:04:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1ba7fc640 1 --2- 172.21.15.103:0/3880064257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473ef680 0x5611473f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1bb7fe640 1 --2- 172.21.15.103:0/3604989414 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1a40a33b0 0x7fd1a40a5870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd1b406e440 tx=0x7fd1b406e470 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1baffd640 1 --2- 172.21.15.103:0/3880064257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e4dc0 0x5611473ef140 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0314e0 tx=0x7fd1ac004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13e5632a2248bb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1a17fa640 1 -- 172.21.15.103:0/3880064257 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac035030 con 0x5611473e4dc0 2024-09-19T07:04:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1a17fa640 1 -- 172.21.15.103:0/3880064257 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ac003ad0 con 0x5611473e4dc0 2024-09-19T07:04:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.011+0000 7fd1a17fa640 1 -- 172.21.15.103:0/3880064257 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac031c40 con 0x5611473e4dc0 2024-09-19T07:04:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.012+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3880064257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e4dc0 msgr2=0x5611473ef140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.012+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/3880064257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e4dc0 0x5611473ef140 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0314e0 tx=0x7fd1ac004170 comp rx=0 tx=0).stop 2024-09-19T07:04:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.012+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3880064257 shutdown_connections 2024-09-19T07:04:28.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.012+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/3880064257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473ef680 0x5611473f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.012+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/3880064257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e4dc0 0x5611473ef140 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.012+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/3880064257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473e44a0 0x5611473e4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.012+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3880064257 >> 172.21.15.103:0/3880064257 conn(0x5611473e0a30 msgr2=0x5611473e0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:28.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.012+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3880064257 shutdown_connections 2024-09-19T07:04:28.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.012+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3880064257 wait complete. 2024-09-19T07:04:28.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.012+0000 7fd1c3b06740 1 -- start start 2024-09-19T07:04:28.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1c3b06740 1 Processor -- start 2024-09-19T07:04:28.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1c3b06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e44a0 0x5611473eef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1c3b06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473e4dc0 0x5611473ec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1c3b06740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473ef680 0x5611473ec730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1c3b06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5611473f2250 con 0x5611473e44a0 2024-09-19T07:04:28.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1c3b06740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561147305050 con 0x5611473ef680 2024-09-19T07:04:28.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1c3b06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5611473f23d0 con 0x5611473e4dc0 2024-09-19T07:04:28.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1c3b06740 10 client.0 _ll_register_callbacks cb 0x561147345160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:28.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1ba7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e44a0 0x5611473eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:28.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1ba7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e44a0 0x5611473eef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45590/0 (socket says 172.21.15.103:45590) 2024-09-19T07:04:28.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1ba7fc640 1 -- 172.21.15.103:0/3776736454 learned_addr learned my addr 172.21.15.103:0/3776736454 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:28.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1ba7fc640 1 -- 172.21.15.103:0/3776736454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473e4dc0 msgr2=0x5611473ec1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:28.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1ba7fc640 1 --2- 172.21.15.103:0/3776736454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473e4dc0 0x5611473ec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1ba7fc640 1 -- 172.21.15.103:0/3776736454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473ef680 msgr2=0x5611473ec730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:28.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1ba7fc640 1 --2- 172.21.15.103:0/3776736454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473ef680 0x5611473ec730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1ba7fc640 1 -- 172.21.15.103:0/3776736454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611470bb3c0 con 0x5611473e44a0 2024-09-19T07:04:28.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd1ba7fc640 1 --2- 172.21.15.103:0/3776736454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e44a0 0x5611473eef40 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000ad20 tx=0x7fd1b000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c27ab9970488bff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:28.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.013+0000 7fd19eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:28.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.014+0000 7fd19eff5640 1 -- 172.21.15.103:0/3776736454 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b000e670 con 0x5611473e44a0 2024-09-19T07:04:28.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.014+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3776736454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1ac003ad0 con 0x5611473e44a0 2024-09-19T07:04:28.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.014+0000 7fd19eff5640 1 -- 172.21.15.103:0/3776736454 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fd1b000e810 con 0x5611473e44a0 2024-09-19T07:04:28.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.014+0000 7fd19eff5640 1 -- 172.21.15.103:0/3776736454 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd1b000ec20 con 0x5611473e44a0 2024-09-19T07:04:28.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.014+0000 7fd1c3b06740 10 client.20592 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:28.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.014+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3776736454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5611473fdc20 con 0x5611473e44a0 2024-09-19T07:04:28.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.014+0000 7fd19eff5640 1 -- 172.21.15.103:0/3776736454 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ac003ad0 con 0x5611473e44a0 2024-09-19T07:04:28.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.014+0000 7fd19eff5640 1 -- 172.21.15.103:0/3776736454 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd1b001e480 con 0x5611473e44a0 2024-09-19T07:04:28.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.014+0000 7fd1c3b06740 10 client.20592 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:28.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.014+0000 7fd1c3b06740 10 client.20592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:28.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.014+0000 7fd1c3b06740 20 client.20592 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:28.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.014+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/3776736454 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5611473ff080 0x561147401540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.014+0000 7fd1c3b06740 4 client.20592 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:28.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.014+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3776736454 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561147357850 con 0x5611473ff080 2024-09-19T07:04:28.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.014+0000 7fd1baffd640 1 --2- 172.21.15.103:0/3776736454 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5611473ff080 0x561147401540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:28.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.016+0000 7fd19eff5640 10 client.20592 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:28.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.016+0000 7fd1baffd640 1 --2- 172.21.15.103:0/3776736454 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5611473ff080 0x561147401540 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f815a2ffc8adee9c server_cookie=25912b69a52e32d4 in_seq=0 out_seq=0 2024-09-19T07:04:28.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.016+0000 7fd19eff5640 1 -- 172.21.15.103:0/3776736454 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561147357850 con 0x5611473ff080 2024-09-19T07:04:28.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.017+0000 7fd19eff5640 10 client.20592 handle_command_reply: tid=0 2024-09-19T07:04:28.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.017+0000 7fd1c3b06740 1 client.20592 shutdown 2024-09-19T07:04:28.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.017+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3776736454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e44a0 msgr2=0x5611473eef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:28.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.017+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/3776736454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e44a0 0x5611473eef40 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000ad20 tx=0x7fd1b000f7f0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.017+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3776736454 shutdown_connections 2024-09-19T07:04:28.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.017+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/3776736454 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5611473ff080 0x561147401540 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.017+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/3776736454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473ef680 0x5611473ec730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.017+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/3776736454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473e4dc0 0x5611473ec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.017+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/3776736454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e44a0 0x5611473eef40 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.017+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3776736454 >> 172.21.15.103:0/3776736454 conn(0x5611473e0a30 msgr2=0x5611473dfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:28.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.017+0000 7fd19eff5640 0 client.20592 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:28.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.017+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3776736454 shutdown_connections 2024-09-19T07:04:28.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.017+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3776736454 wait complete. 2024-09-19T07:04:28.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.017+0000 7fd1c3b06740 20 client.20592 trim_cache size 0 max 16384 2024-09-19T07:04:28.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.034+0000 7fd1c3b06740 1 Processor -- start 2024-09-19T07:04:28.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.034+0000 7fd1c3b06740 1 -- start start 2024-09-19T07:04:28.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.034+0000 7fd1c3b06740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473e44a0 0x5611473fb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.034+0000 7fd1c3b06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473e4dc0 0x561147406e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.034+0000 7fd1c3b06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473ef680 0x561147407350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.034+0000 7fd1c3b06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc1ad8c0 con 0x5611473ef680 2024-09-19T07:04:28.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.034+0000 7fd1c3b06740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc1af400 con 0x5611473e44a0 2024-09-19T07:04:28.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.034+0000 7fd1c3b06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc1adf90 con 0x5611473e4dc0 2024-09-19T07:04:28.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.035+0000 7fd1ba7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473ef680 0x561147407350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:28.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.035+0000 7fd1ba7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473ef680 0x561147407350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45592/0 (socket says 172.21.15.103:45592) 2024-09-19T07:04:28.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.035+0000 7fd1ba7fc640 1 -- 172.21.15.103:0/2320811640 learned_addr learned my addr 172.21.15.103:0/2320811640 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:28.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.035+0000 7fd1ba7fc640 1 --2- 172.21.15.103:0/2320811640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473e44a0 0x5611473fb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:28.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.035+0000 7fd1baffd640 1 --2- 172.21.15.103:0/2320811640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473e4dc0 0x561147406e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:28.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.035+0000 7fd1ba7fc640 1 -- 172.21.15.103:0/2320811640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473e4dc0 msgr2=0x561147406e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:28.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.035+0000 7fd1ba7fc640 1 --2- 172.21.15.103:0/2320811640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473e4dc0 0x561147406e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.035+0000 7fd1ba7fc640 1 -- 172.21.15.103:0/2320811640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473e44a0 msgr2=0x5611473fb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:28.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.035+0000 7fd1ba7fc640 1 --2- 172.21.15.103:0/2320811640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473e44a0 0x5611473fb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.035+0000 7fd1ba7fc640 1 -- 172.21.15.103:0/2320811640 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611473e5330 con 0x5611473ef680 2024-09-19T07:04:28.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.035+0000 7fd1ba7fc640 1 --2- 172.21.15.103:0/2320811640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473ef680 0x561147407350 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000a700 tx=0x7fd1b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d42452e7e9b488 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:28.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.035+0000 7fd1a0ff9640 1 -- 172.21.15.103:0/2320811640 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b001ee60 con 0x5611473ef680 2024-09-19T07:04:28.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.035+0000 7fd1a0ff9640 1 -- 172.21.15.103:0/2320811640 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b001e3f0 con 0x5611473ef680 2024-09-19T07:04:28.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.035+0000 7fd1a0ff9640 1 -- 172.21.15.103:0/2320811640 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0010670 con 0x5611473ef680 2024-09-19T07:04:28.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.036+0000 7fd1c3b06740 1 -- 172.21.15.103:0/2320811640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473ef680 msgr2=0x561147407350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.036+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/2320811640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473ef680 0x561147407350 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000a700 tx=0x7fd1b00046c0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.036+0000 7fd1c3b06740 1 -- 172.21.15.103:0/2320811640 shutdown_connections 2024-09-19T07:04:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.036+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/2320811640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473ef680 0x561147407350 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.036+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/2320811640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473e4dc0 0x561147406e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.036+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/2320811640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473e44a0 0x5611473fb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.036+0000 7fd1c3b06740 1 -- 172.21.15.103:0/2320811640 >> 172.21.15.103:0/2320811640 conn(0x5611473e0a30 msgr2=0x5611473e77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.036+0000 7fd1c3b06740 1 -- 172.21.15.103:0/2320811640 shutdown_connections 2024-09-19T07:04:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.036+0000 7fd1c3b06740 1 -- 172.21.15.103:0/2320811640 wait complete. 2024-09-19T07:04:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.036+0000 7fd1c3b06740 1 -- start start 2024-09-19T07:04:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.036+0000 7fd1c3b06740 1 Processor -- start 2024-09-19T07:04:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1c3b06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e44a0 0x5611473f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1c3b06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473e4dc0 0x5611473f3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1c3b06740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473ef680 0x5611473f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1c3b06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc1aed30 con 0x5611473e44a0 2024-09-19T07:04:28.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1c3b06740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc1ae660 con 0x5611473ef680 2024-09-19T07:04:28.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1c3b06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc1ab8c0 con 0x5611473e4dc0 2024-09-19T07:04:28.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1c3b06740 10 client.0 _ll_register_callbacks cb 0x7fd1b0012b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:28.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1ba7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e44a0 0x5611473f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:28.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1ba7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e44a0 0x5611473f33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45600/0 (socket says 172.21.15.103:45600) 2024-09-19T07:04:28.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1ba7fc640 1 -- 172.21.15.103:0/3092656155 learned_addr learned my addr 172.21.15.103:0/3092656155 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:28.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1ba7fc640 1 -- 172.21.15.103:0/3092656155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473e4dc0 msgr2=0x5611473f3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:28.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1baffd640 1 --2- 172.21.15.103:0/3092656155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473e4dc0 0x5611473f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:28.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1ba7fc640 1 --2- 172.21.15.103:0/3092656155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473e4dc0 0x5611473f3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1ba7fc640 1 -- 172.21.15.103:0/3092656155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473ef680 msgr2=0x5611473f3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:28.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1ba7fc640 1 --2- 172.21.15.103:0/3092656155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473ef680 0x5611473f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1ba7fc640 1 -- 172.21.15.103:0/3092656155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561147468e30 con 0x5611473e44a0 2024-09-19T07:04:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd1ba7fc640 1 --2- 172.21.15.103:0/3092656155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e44a0 0x5611473f33f0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000ad20 tx=0x7fd1b000a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a5413674261f38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd19eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.037+0000 7fd19eff5640 1 -- 172.21.15.103:0/3092656155 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0005050 con 0x5611473e44a0 2024-09-19T07:04:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.038+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3092656155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1b001e3f0 con 0x5611473e44a0 2024-09-19T07:04:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.038+0000 7fd19eff5640 1 -- 172.21.15.103:0/3092656155 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fd1b00051f0 con 0x5611473e44a0 2024-09-19T07:04:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.038+0000 7fd19eff5640 1 -- 172.21.15.103:0/3092656155 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd1b00053f0 con 0x5611473e44a0 2024-09-19T07:04:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.038+0000 7fd1c3b06740 10 client.20598 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.038+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3092656155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56114746a120 con 0x5611473e44a0 2024-09-19T07:04:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.038+0000 7fd19eff5640 1 -- 172.21.15.103:0/3092656155 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b001e3f0 con 0x5611473e44a0 2024-09-19T07:04:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.038+0000 7fd19eff5640 1 -- 172.21.15.103:0/3092656155 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd1b003e4e0 con 0x5611473e44a0 2024-09-19T07:04:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.038+0000 7fd1c3b06740 10 client.20598 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.038+0000 7fd1c3b06740 10 client.20598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.038+0000 7fd1c3b06740 20 client.20598 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.038+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/3092656155 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56114746a780 0x56114746cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.038+0000 7fd1c3b06740 4 client.20598 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:04:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.038+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3092656155 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561147351d60 con 0x56114746a780 2024-09-19T07:04:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.038+0000 7fd1baffd640 1 --2- 172.21.15.103:0/3092656155 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56114746a780 0x56114746cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd1baffd640 1 --2- 172.21.15.103:0/3092656155 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56114746a780 0x56114746cb70 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e84199d8b4b40274 server_cookie=8727640fb9993128 in_seq=0 out_seq=0 2024-09-19T07:04:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd19eff5640 10 client.20598 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd19eff5640 1 -- 172.21.15.103:0/3092656155 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561147351d60 con 0x56114746a780 2024-09-19T07:04:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd19eff5640 10 client.20598 handle_command_reply: tid=0 2024-09-19T07:04:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd1c3b06740 1 client.20598 shutdown 2024-09-19T07:04:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3092656155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e44a0 msgr2=0x5611473f33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/3092656155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e44a0 0x5611473f33f0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000ad20 tx=0x7fd1b000a850 comp rx=0 tx=0).stop 2024-09-19T07:04:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3092656155 shutdown_connections 2024-09-19T07:04:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/3092656155 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56114746a780 0x56114746cb70 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/3092656155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611473ef680 0x5611473f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/3092656155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611473e4dc0 0x5611473f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd19eff5640 0 client.20598 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:28.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd1c3b06740 1 --2- 172.21.15.103:0/3092656155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611473e44a0 0x5611473f33f0 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3092656155 >> 172.21.15.103:0/3092656155 conn(0x5611473e0a30 msgr2=0x5611473e5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:28.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3092656155 shutdown_connections 2024-09-19T07:04:28.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd1c3b06740 1 -- 172.21.15.103:0/3092656155 wait complete. 2024-09-19T07:04:28.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.039+0000 7fd1c3b06740 20 client.20598 trim_cache size 0 max 16384 2024-09-19T07:04:28.046 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:04:28.046 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:04:28.046 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:04:28.046 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:04:28.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.040+0000 7fd1c172d640 1 -- 172.21.15.103:0/3604989414 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1a40a33b0 msgr2=0x7fd1a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:28.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.040+0000 7fd1c172d640 1 --2- 172.21.15.103:0/3604989414 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1a40a33b0 0x7fd1a40a5870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd1b406e440 tx=0x7fd1b406e470 comp rx=0 tx=0).stop 2024-09-19T07:04:28.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.040+0000 7fd1c172d640 1 -- 172.21.15.103:0/3604989414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1bc102ef0 msgr2=0x7fd1bc1a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:28.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.040+0000 7fd1c172d640 1 --2- 172.21.15.103:0/3604989414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1bc102ef0 0x7fd1bc1a43b0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400b970 tx=0x7fd1b400bd70 comp rx=0 tx=0).stop 2024-09-19T07:04:28.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.040+0000 7fd1c172d640 1 -- 172.21.15.103:0/3604989414 shutdown_connections 2024-09-19T07:04:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.040+0000 7fd1c172d640 1 --2- 172.21.15.103:0/3604989414 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd1a40a33b0 0x7fd1a40a5870 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.040+0000 7fd1c172d640 1 --2- 172.21.15.103:0/3604989414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1bc102ef0 0x7fd1bc1a43b0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.040+0000 7fd1c172d640 1 --2- 172.21.15.103:0/3604989414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1bc102570 0x7fd1bc199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.040+0000 7fd1c172d640 1 --2- 172.21.15.103:0/3604989414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1bc100e20 0x7fd1bc199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.040+0000 7fd1c172d640 1 -- 172.21.15.103:0/3604989414 >> 172.21.15.103:0/3604989414 conn(0x7fd1bc0fc7d0 msgr2=0x7fd1bc106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.040+0000 7fd1c172d640 1 -- 172.21.15.103:0/3604989414 shutdown_connections 2024-09-19T07:04:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:28.040+0000 7fd1c172d640 1 -- 172.21.15.103:0/3604989414 wait complete. 2024-09-19T07:04:28.782 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:04:28.783 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:04:29.033 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:29.467 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:29 smithi103 ceph-mon[31158]: pgmap v478: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-19T07:04:29.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.472+0000 7fea83681640 1 -- 172.21.15.103:0/43733645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea7c100e20 msgr2=0x7fea7c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:29.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.472+0000 7fea83681640 1 --2- 172.21.15.103:0/43733645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea7c100e20 0x7fea7c101200 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fea78009900 tx=0x7fea7802a480 comp rx=0 tx=0).stop 2024-09-19T07:04:29.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.472+0000 7fea83681640 1 -- 172.21.15.103:0/43733645 shutdown_connections 2024-09-19T07:04:29.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.472+0000 7fea83681640 1 --2- 172.21.15.103:0/43733645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea7c102ef0 0x7fea7c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:29.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.472+0000 7fea83681640 1 --2- 172.21.15.103:0/43733645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea7c102570 0x7fea7c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:29.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.472+0000 7fea83681640 1 --2- 172.21.15.103:0/43733645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea7c100e20 0x7fea7c101200 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:29.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.472+0000 7fea83681640 1 -- 172.21.15.103:0/43733645 >> 172.21.15.103:0/43733645 conn(0x7fea7c0fc7d0 msgr2=0x7fea7c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:29.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.472+0000 7fea83681640 1 -- 172.21.15.103:0/43733645 shutdown_connections 2024-09-19T07:04:29.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.472+0000 7fea83681640 1 -- 172.21.15.103:0/43733645 wait complete. 2024-09-19T07:04:29.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.472+0000 7fea83681640 1 Processor -- start 2024-09-19T07:04:29.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.472+0000 7fea83681640 1 -- start start 2024-09-19T07:04:29.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea83681640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea7c100e20 0x7fea7c197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:29.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea83681640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea7c102570 0x7fea7c197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:29.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea83681640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea7c102ef0 0x7fea7c197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:29.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea83681640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c110fa0 con 0x7fea7c102ef0 2024-09-19T07:04:29.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea83681640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c110e20 con 0x7fea7c102570 2024-09-19T07:04:29.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea83681640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c111120 con 0x7fea7c100e20 2024-09-19T07:04:29.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea81c1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea7c102ef0 0x7fea7c197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea81c1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea7c102ef0 0x7fea7c197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45624/0 (socket says 172.21.15.103:45624) 2024-09-19T07:04:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea81c1e640 1 -- 172.21.15.103:0/258858345 learned_addr learned my addr 172.21.15.103:0/258858345 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea80c1c640 1 --2- 172.21.15.103:0/258858345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea7c102570 0x7fea7c197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea8141d640 1 --2- 172.21.15.103:0/258858345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea7c100e20 0x7fea7c197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea81c1e640 1 -- 172.21.15.103:0/258858345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea7c100e20 msgr2=0x7fea7c197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea81c1e640 1 --2- 172.21.15.103:0/258858345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea7c100e20 0x7fea7c197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea81c1e640 1 -- 172.21.15.103:0/258858345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea7c102570 msgr2=0x7fea7c197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea81c1e640 1 --2- 172.21.15.103:0/258858345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea7c102570 0x7fea7c197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea81c1e640 1 -- 172.21.15.103:0/258858345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c1a69c0 con 0x7fea7c102ef0 2024-09-19T07:04:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea80c1c640 1 --2- 172.21.15.103:0/258858345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea7c102570 0x7fea7c197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea81c1e640 1 --2- 172.21.15.103:0/258858345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea7c102ef0 0x7fea7c197bd0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fea7400eaa0 tx=0x7fea7400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.473+0000 7fea727fc640 1 -- 172.21.15.103:0/258858345 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7400cdb0 con 0x7fea7c102ef0 2024-09-19T07:04:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.474+0000 7fea83681640 1 -- 172.21.15.103:0/258858345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7c1a6cb0 con 0x7fea7c102ef0 2024-09-19T07:04:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.474+0000 7fea727fc640 1 -- 172.21.15.103:0/258858345 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea74005020 con 0x7fea7c102ef0 2024-09-19T07:04:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.474+0000 7fea727fc640 1 -- 172.21.15.103:0/258858345 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea740116b0 con 0x7fea7c102ef0 2024-09-19T07:04:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.474+0000 7fea83681640 1 -- 172.21.15.103:0/258858345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea7c1a7220 con 0x7fea7c102ef0 2024-09-19T07:04:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.475+0000 7fea83681640 1 -- 172.21.15.103:0/258858345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea7c101200 con 0x7fea7c102ef0 2024-09-19T07:04:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.476+0000 7fea727fc640 1 -- 172.21.15.103:0/258858345 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fea74005560 con 0x7fea7c102ef0 2024-09-19T07:04:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.476+0000 7fea727fc640 1 --2- 172.21.15.103:0/258858345 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fea680a33b0 0x7fea680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.476+0000 7fea8141d640 1 --2- 172.21.15.103:0/258858345 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fea680a33b0 0x7fea680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.476+0000 7fea727fc640 1 -- 172.21.15.103:0/258858345 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fea740cff90 con 0x7fea7c102ef0 2024-09-19T07:04:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.477+0000 7fea8141d640 1 --2- 172.21.15.103:0/258858345 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fea680a33b0 0x7fea680a5870 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fea78009230 tx=0x7fea780058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:29.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.481+0000 7fea727fc640 1 -- 172.21.15.103:0/258858345 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fea74016030 con 0x7fea7c102ef0 2024-09-19T07:04:29.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:29 smithi174 ceph-mon[35637]: pgmap v478: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-19T07:04:29.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.683+0000 7fea83681640 1 -- 172.21.15.103:0/258858345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea7c1a74b0 con 0x7fea7c102ef0 2024-09-19T07:04:29.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.683+0000 7fea727fc640 1 -- 172.21.15.103:0/258858345 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fea7409c950 con 0x7fea7c102ef0 2024-09-19T07:04:29.685 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:04:29.685 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:04:29.685 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:04:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.686+0000 7fea83681640 1 -- 172.21.15.103:0/258858345 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fea680a33b0 msgr2=0x7fea680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.686+0000 7fea83681640 1 --2- 172.21.15.103:0/258858345 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fea680a33b0 0x7fea680a5870 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fea78009230 tx=0x7fea780058d0 comp rx=0 tx=0).stop 2024-09-19T07:04:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.686+0000 7fea83681640 1 -- 172.21.15.103:0/258858345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea7c102ef0 msgr2=0x7fea7c197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.686+0000 7fea83681640 1 --2- 172.21.15.103:0/258858345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea7c102ef0 0x7fea7c197bd0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fea7400eaa0 tx=0x7fea7400ef60 comp rx=0 tx=0).stop 2024-09-19T07:04:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.686+0000 7fea83681640 1 -- 172.21.15.103:0/258858345 shutdown_connections 2024-09-19T07:04:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.686+0000 7fea83681640 1 --2- 172.21.15.103:0/258858345 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fea680a33b0 0x7fea680a5870 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.686+0000 7fea83681640 1 --2- 172.21.15.103:0/258858345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea7c102ef0 0x7fea7c197bd0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.686+0000 7fea83681640 1 --2- 172.21.15.103:0/258858345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea7c102570 0x7fea7c197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.686+0000 7fea83681640 1 --2- 172.21.15.103:0/258858345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea7c100e20 0x7fea7c197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:29.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.686+0000 7fea83681640 1 -- 172.21.15.103:0/258858345 >> 172.21.15.103:0/258858345 conn(0x7fea7c0fc7d0 msgr2=0x7fea7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:29.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.686+0000 7fea83681640 1 -- 172.21.15.103:0/258858345 shutdown_connections 2024-09-19T07:04:29.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:29.686+0000 7fea83681640 1 -- 172.21.15.103:0/258858345 wait complete. 2024-09-19T07:04:29.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:29 smithi134 ceph-mon[35356]: pgmap v478: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-19T07:04:30.359 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:04:30.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:30 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/258858345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:30.604 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:30.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:30 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/258858345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:30.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:30 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/258858345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:31.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.106+0000 7f26dbed1640 1 -- 172.21.15.103:0/327090755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d4102790 msgr2=0x7f26d41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.106+0000 7f26dbed1640 1 --2- 172.21.15.103:0/327090755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d4102790 0x7f26d41067e0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009900 tx=0x7f26c402a410 comp rx=0 tx=0).stop 2024-09-19T07:04:31.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.106+0000 7f26dbed1640 1 -- 172.21.15.103:0/327090755 shutdown_connections 2024-09-19T07:04:31.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.106+0000 7f26dbed1640 1 --2- 172.21.15.103:0/327090755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26d4106f10 0x7f26d410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.106+0000 7f26dbed1640 1 --2- 172.21.15.103:0/327090755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d4102790 0x7f26d41067e0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.106+0000 7f26dbed1640 1 --2- 172.21.15.103:0/327090755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d4101de0 0x7f26d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.106+0000 7f26dbed1640 1 -- 172.21.15.103:0/327090755 >> 172.21.15.103:0/327090755 conn(0x7f26d40fd8c0 msgr2=0x7f26d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:31.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.106+0000 7f26dbed1640 1 -- 172.21.15.103:0/327090755 shutdown_connections 2024-09-19T07:04:31.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.107+0000 7f26dbed1640 1 -- 172.21.15.103:0/327090755 wait complete. 2024-09-19T07:04:31.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.107+0000 7f26dbed1640 1 Processor -- start 2024-09-19T07:04:31.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.107+0000 7f26dbed1640 1 -- start start 2024-09-19T07:04:31.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.107+0000 7f26dbed1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d4101de0 0x7f26d41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.107+0000 7f26dbed1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26d4102790 0x7f26d41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.107+0000 7f26dbed1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d4106f10 0x7f26d419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.107+0000 7f26dbed1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26d4111f70 con 0x7f26d4106f10 2024-09-19T07:04:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.107+0000 7f26dbed1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26d4111df0 con 0x7f26d4102790 2024-09-19T07:04:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.107+0000 7f26dbed1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41120f0 con 0x7f26d4101de0 2024-09-19T07:04:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.107+0000 7f26da46e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d4106f10 0x7f26d419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.107+0000 7f26da46e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d4106f10 0x7f26d419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45632/0 (socket says 172.21.15.103:45632) 2024-09-19T07:04:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.107+0000 7f26da46e640 1 -- 172.21.15.103:0/3833601361 learned_addr learned my addr 172.21.15.103:0/3833601361 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.108+0000 7f26d9c6d640 1 --2- 172.21.15.103:0/3833601361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d4101de0 0x7f26d41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.108+0000 7f26d946c640 1 --2- 172.21.15.103:0/3833601361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26d4102790 0x7f26d41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.108+0000 7f26da46e640 1 -- 172.21.15.103:0/3833601361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d4101de0 msgr2=0x7f26d41984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.108+0000 7f26da46e640 1 --2- 172.21.15.103:0/3833601361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d4101de0 0x7f26d41984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.108+0000 7f26da46e640 1 -- 172.21.15.103:0/3833601361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26d4102790 msgr2=0x7f26d41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.108+0000 7f26da46e640 1 --2- 172.21.15.103:0/3833601361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26d4102790 0x7f26d41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.108+0000 7f26da46e640 1 -- 172.21.15.103:0/3833601361 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d419d460 con 0x7f26d4106f10 2024-09-19T07:04:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.108+0000 7f26da46e640 1 --2- 172.21.15.103:0/3833601361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d4106f10 0x7f26d419cd80 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00eaa0 tx=0x7f26cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.108+0000 7f26caffd640 1 -- 172.21.15.103:0/3833601361 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc00cdb0 con 0x7f26d4106f10 2024-09-19T07:04:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.108+0000 7f26caffd640 1 -- 172.21.15.103:0/3833601361 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26cc005020 con 0x7f26d4106f10 2024-09-19T07:04:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.108+0000 7f26caffd640 1 -- 172.21.15.103:0/3833601361 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc0116b0 con 0x7f26d4106f10 2024-09-19T07:04:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.108+0000 7f26dbed1640 1 -- 172.21.15.103:0/3833601361 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d419d750 con 0x7f26d4106f10 2024-09-19T07:04:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.108+0000 7f26dbed1640 1 -- 172.21.15.103:0/3833601361 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d41a5030 con 0x7f26d4106f10 2024-09-19T07:04:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.110+0000 7f26caffd640 1 -- 172.21.15.103:0/3833601361 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f26cc005560 con 0x7f26d4106f10 2024-09-19T07:04:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.111+0000 7f26de2aa740 1 Processor -- start 2024-09-19T07:04:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.111+0000 7f26de2aa740 1 -- start start 2024-09-19T07:04:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.111+0000 7f26de2aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdced9b40 0x555bdced8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.111+0000 7f26de2aa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdced8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.111+0000 7f26de2aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdcee3370 0x555bdcee5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.111+0000 7f26de2aa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555bdce3e2f0 con 0x555bdced85d0 2024-09-19T07:04:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.111+0000 7f26de2aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555bdcde5cf0 con 0x555bdced9b40 2024-09-19T07:04:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.111+0000 7f26de2aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555bdce3a2b0 con 0x555bdcee3370 2024-09-19T07:04:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.111+0000 7f26caffd640 1 --2- 172.21.15.103:0/3833601361 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26c00a33b0 0x7f26c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.111+0000 7f26d9c6d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdced8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.111+0000 7f26d9c6d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdced8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45644/0 (socket says 172.21.15.103:45644) 2024-09-19T07:04:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.111+0000 7f26d9c6d640 1 -- 172.21.15.103:0/3831533782 learned_addr learned my addr 172.21.15.103:0/3831533782 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.111+0000 7f26d946c640 1 --2- 172.21.15.103:0/3831533782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdcee3370 0x555bdcee5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.111+0000 7f26caffd640 1 -- 172.21.15.103:0/3833601361 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f26cc0cfe80 con 0x7f26d4106f10 2024-09-19T07:04:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.111+0000 7f26da46e640 1 --2- 172.21.15.103:0/3833601361 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26c00a33b0 0x7f26c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.112+0000 7f26d946c640 1 --2- 172.21.15.103:0/3831533782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdced9b40 0x555bdced8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.112+0000 7f26d9c6d640 1 -- 172.21.15.103:0/3831533782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdcee3370 msgr2=0x555bdcee5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.112+0000 7f26d9c6d640 1 --2- 172.21.15.103:0/3831533782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdcee3370 0x555bdcee5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.112+0000 7f26d9c6d640 1 -- 172.21.15.103:0/3831533782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdced9b40 msgr2=0x555bdced8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.112+0000 7f26d9c6d640 1 --2- 172.21.15.103:0/3831533782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdced9b40 0x555bdced8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.112+0000 7f26d9c6d640 1 -- 172.21.15.103:0/3831533782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bdcdca3a0 con 0x555bdced85d0 2024-09-19T07:04:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.112+0000 7f26d946c640 1 --2- 172.21.15.103:0/3831533782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdced9b40 0x555bdced8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.112+0000 7f26da46e640 1 --2- 172.21.15.103:0/3833601361 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26c00a33b0 0x7f26c00a5870 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f26cc09c180 tx=0x7f26cc009270 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.112+0000 7f26d9c6d640 1 --2- 172.21.15.103:0/3831533782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdced8a10 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f26d000b760 tx=0x7f26d000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d44e566b74e6473 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.112+0000 7f26bffff640 1 -- 172.21.15.103:0/3831533782 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0003dd0 con 0x555bdced85d0 2024-09-19T07:04:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.112+0000 7f26bffff640 1 -- 172.21.15.103:0/3831533782 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d0015070 con 0x555bdced85d0 2024-09-19T07:04:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.112+0000 7f26bffff640 1 -- 172.21.15.103:0/3831533782 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0004e30 con 0x555bdced85d0 2024-09-19T07:04:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.113+0000 7f26de2aa740 1 -- 172.21.15.103:0/3831533782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 msgr2=0x555bdced8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.113+0000 7f26de2aa740 1 --2- 172.21.15.103:0/3831533782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdced8a10 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f26d000b760 tx=0x7f26d000bc20 comp rx=0 tx=0).stop 2024-09-19T07:04:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.113+0000 7f26de2aa740 1 -- 172.21.15.103:0/3831533782 shutdown_connections 2024-09-19T07:04:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.113+0000 7f26de2aa740 1 --2- 172.21.15.103:0/3831533782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdcee3370 0x555bdcee5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.113+0000 7f26de2aa740 1 --2- 172.21.15.103:0/3831533782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdced8a10 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.113+0000 7f26de2aa740 1 --2- 172.21.15.103:0/3831533782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdced9b40 0x555bdced8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.113+0000 7f26de2aa740 1 -- 172.21.15.103:0/3831533782 >> 172.21.15.103:0/3831533782 conn(0x555bdcea1310 msgr2=0x555bdced4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.113+0000 7f26de2aa740 1 -- 172.21.15.103:0/3831533782 shutdown_connections 2024-09-19T07:04:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.113+0000 7f26de2aa740 1 -- 172.21.15.103:0/3831533782 wait complete. 2024-09-19T07:04:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.113+0000 7f26de2aa740 1 -- start start 2024-09-19T07:04:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26de2aa740 1 Processor -- start 2024-09-19T07:04:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26de2aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdced85d0 0x555bdcee2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26de2aa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced9b40 0x555bdcee0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26de2aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdcee3370 0x555bdcee05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26de2aa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555bdcee6070 con 0x555bdced9b40 2024-09-19T07:04:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26de2aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555bdcdf9060 con 0x555bdcee3370 2024-09-19T07:04:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26de2aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555bdcee61f0 con 0x555bdced85d0 2024-09-19T07:04:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26de2aa740 10 client.0 _ll_register_callbacks cb 0x555bdcb4c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26d9c6d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced9b40 0x555bdcee0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26d9c6d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced9b40 0x555bdcee0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45660/0 (socket says 172.21.15.103:45660) 2024-09-19T07:04:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26d9c6d640 1 -- 172.21.15.103:0/1106350022 learned_addr learned my addr 172.21.15.103:0/1106350022 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26d946c640 1 --2- 172.21.15.103:0/1106350022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdced85d0 0x555bdcee2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26d946c640 1 --2- 172.21.15.103:0/1106350022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdcee3370 0x555bdcee05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26d9c6d640 1 -- 172.21.15.103:0/1106350022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdced85d0 msgr2=0x555bdcee2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26d9c6d640 1 --2- 172.21.15.103:0/1106350022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdced85d0 0x555bdcee2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26d9c6d640 1 -- 172.21.15.103:0/1106350022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdcee3370 msgr2=0x555bdcee05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26d9c6d640 1 --2- 172.21.15.103:0/1106350022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdcee3370 0x555bdcee05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26d9c6d640 1 -- 172.21.15.103:0/1106350022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bdcbaf3c0 con 0x555bdced9b40 2024-09-19T07:04:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.114+0000 7f26d946c640 1 --2- 172.21.15.103:0/1106350022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdcee3370 0x555bdcee05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.115+0000 7f26d9c6d640 1 --2- 172.21.15.103:0/1106350022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced9b40 0x555bdcee0000 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f26d0006010 tx=0x7f26d00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4202ff4a1d81f3b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.115+0000 7f26bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.115+0000 7f26bd7fa640 1 -- 172.21.15.103:0/1106350022 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0028060 con 0x555bdced9b40 2024-09-19T07:04:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.115+0000 7f26bd7fa640 1 -- 172.21.15.103:0/1106350022 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f26d0011640 con 0x555bdced9b40 2024-09-19T07:04:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.115+0000 7f26de2aa740 1 -- 172.21.15.103:0/1106350022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26d0015070 con 0x555bdced9b40 2024-09-19T07:04:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.115+0000 7f26bd7fa640 1 -- 172.21.15.103:0/1106350022 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f26d0011840 con 0x555bdced9b40 2024-09-19T07:04:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.115+0000 7f26de2aa740 10 client.20616 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.115+0000 7f26de2aa740 1 -- 172.21.15.103:0/1106350022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555bdcef1af0 con 0x555bdced9b40 2024-09-19T07:04:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.115+0000 7f26bd7fa640 1 -- 172.21.15.103:0/1106350022 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d0015070 con 0x555bdced9b40 2024-09-19T07:04:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.115+0000 7f26bd7fa640 1 -- 172.21.15.103:0/1106350022 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f26d0022070 con 0x555bdced9b40 2024-09-19T07:04:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.115+0000 7f26de2aa740 10 client.20616 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.115+0000 7f26de2aa740 10 client.20616 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:04:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.115+0000 7f26de2aa740 20 client.20616 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.115+0000 7f26de2aa740 1 --2- 172.21.15.103:0/1106350022 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x555bdcef2ef0 0x555bdcef52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.115+0000 7f26de2aa740 4 client.20616 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.115+0000 7f26de2aa740 1 -- 172.21.15.103:0/1106350022 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bdce4b860 con 0x555bdcef2ef0 2024-09-19T07:04:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.116+0000 7f26d946c640 1 --2- 172.21.15.103:0/1106350022 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x555bdcef2ef0 0x555bdcef52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.116+0000 7f26d946c640 1 --2- 172.21.15.103:0/1106350022 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x555bdcef2ef0 0x555bdcef52e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e1048bef5eef7908 server_cookie=94255338ec629cfb in_seq=0 out_seq=0 2024-09-19T07:04:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.116+0000 7f26bd7fa640 10 client.20616 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:04:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.117+0000 7f26bd7fa640 1 -- 172.21.15.103:0/1106350022 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555bdce4b860 con 0x555bdcef2ef0 2024-09-19T07:04:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.117+0000 7f26bd7fa640 10 client.20616 handle_command_reply: tid=0 2024-09-19T07:04:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.117+0000 7f26de2aa740 1 client.20616 shutdown 2024-09-19T07:04:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.117+0000 7f26de2aa740 1 -- 172.21.15.103:0/1106350022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced9b40 msgr2=0x555bdcee0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.117+0000 7f26de2aa740 1 --2- 172.21.15.103:0/1106350022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced9b40 0x555bdcee0000 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f26d0006010 tx=0x7f26d00048f0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.117+0000 7f26de2aa740 1 -- 172.21.15.103:0/1106350022 shutdown_connections 2024-09-19T07:04:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.117+0000 7f26de2aa740 1 --2- 172.21.15.103:0/1106350022 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x555bdcef2ef0 0x555bdcef52e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.117+0000 7f26de2aa740 1 --2- 172.21.15.103:0/1106350022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdcee3370 0x555bdcee05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.117+0000 7f26de2aa740 1 --2- 172.21.15.103:0/1106350022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced9b40 0x555bdcee0000 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.117+0000 7f26bd7fa640 0 client.20616 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:04:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.117+0000 7f26de2aa740 1 --2- 172.21.15.103:0/1106350022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdced85d0 0x555bdcee2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.117+0000 7f26de2aa740 1 -- 172.21.15.103:0/1106350022 >> 172.21.15.103:0/1106350022 conn(0x555bdcea1310 msgr2=0x555bdceda960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.117+0000 7f26de2aa740 1 -- 172.21.15.103:0/1106350022 shutdown_connections 2024-09-19T07:04:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.117+0000 7f26de2aa740 1 -- 172.21.15.103:0/1106350022 wait complete. 2024-09-19T07:04:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.117+0000 7f26de2aa740 20 client.20616 trim_cache size 0 max 16384 2024-09-19T07:04:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.130+0000 7f26de2aa740 1 Processor -- start 2024-09-19T07:04:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.130+0000 7f26de2aa740 1 -- start start 2024-09-19T07:04:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.130+0000 7f26de2aa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdceef310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.130+0000 7f26de2aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdced9b40 0x555bdcedefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.130+0000 7f26de2aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdcee3370 0x555bdcefa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.130+0000 7f26de2aa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41ad890 con 0x555bdced85d0 2024-09-19T07:04:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.130+0000 7f26de2aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41af3d0 con 0x555bdced9b40 2024-09-19T07:04:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.130+0000 7f26de2aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41adf60 con 0x555bdcee3370 2024-09-19T07:04:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.131+0000 7f26d946c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdceef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.131+0000 7f26d946c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdceef310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45668/0 (socket says 172.21.15.103:45668) 2024-09-19T07:04:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.131+0000 7f26d946c640 1 -- 172.21.15.103:0/23456778 learned_addr learned my addr 172.21.15.103:0/23456778 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.131+0000 7f26d946c640 1 --2- 172.21.15.103:0/23456778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdcee3370 0x555bdcefa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.131+0000 7f26d9c6d640 1 --2- 172.21.15.103:0/23456778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdced9b40 0x555bdcedefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.131+0000 7f26d946c640 1 -- 172.21.15.103:0/23456778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdcee3370 msgr2=0x555bdcefa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.131+0000 7f26d946c640 1 --2- 172.21.15.103:0/23456778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdcee3370 0x555bdcefa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.131+0000 7f26d946c640 1 -- 172.21.15.103:0/23456778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdced9b40 msgr2=0x555bdcedefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.131+0000 7f26d946c640 1 --2- 172.21.15.103:0/23456778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdced9b40 0x555bdcedefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.131+0000 7f26d946c640 1 -- 172.21.15.103:0/23456778 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bdced3310 con 0x555bdced85d0 2024-09-19T07:04:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.131+0000 7f26d946c640 1 --2- 172.21.15.103:0/23456778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdceef310 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009230 tx=0x7f26c40058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e6a6ceba0a8d38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.131+0000 7f26bf7fe640 1 -- 172.21.15.103:0/23456778 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4050070 con 0x555bdced85d0 2024-09-19T07:04:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.131+0000 7f26bf7fe640 1 -- 172.21.15.103:0/23456778 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c4031d10 con 0x555bdced85d0 2024-09-19T07:04:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.131+0000 7f26bf7fe640 1 -- 172.21.15.103:0/23456778 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4035040 con 0x555bdced85d0 2024-09-19T07:04:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.132+0000 7f26de2aa740 1 -- 172.21.15.103:0/23456778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 msgr2=0x555bdceef310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.132+0000 7f26de2aa740 1 --2- 172.21.15.103:0/23456778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdceef310 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009230 tx=0x7f26c40058d0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.132+0000 7f26de2aa740 1 -- 172.21.15.103:0/23456778 shutdown_connections 2024-09-19T07:04:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.132+0000 7f26de2aa740 1 --2- 172.21.15.103:0/23456778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdcee3370 0x555bdcefa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.132+0000 7f26de2aa740 1 --2- 172.21.15.103:0/23456778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdced9b40 0x555bdcedefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.132+0000 7f26de2aa740 1 --2- 172.21.15.103:0/23456778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdceef310 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.132+0000 7f26de2aa740 1 -- 172.21.15.103:0/23456778 >> 172.21.15.103:0/23456778 conn(0x555bdcea1310 msgr2=0x555bdcede8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.132+0000 7f26de2aa740 1 -- 172.21.15.103:0/23456778 shutdown_connections 2024-09-19T07:04:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.132+0000 7f26de2aa740 1 -- 172.21.15.103:0/23456778 wait complete. 2024-09-19T07:04:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.132+0000 7f26de2aa740 1 -- start start 2024-09-19T07:04:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.132+0000 7f26de2aa740 1 Processor -- start 2024-09-19T07:04:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26de2aa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdceeb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26de2aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdced9b40 0x555bdceebdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26de2aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdcee3370 0x555bdcedcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26de2aa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41aed00 con 0x555bdced85d0 2024-09-19T07:04:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26de2aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41ae630 con 0x555bdcee3370 2024-09-19T07:04:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26de2aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41ab890 con 0x555bdced9b40 2024-09-19T07:04:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26de2aa740 10 client.0 _ll_register_callbacks cb 0x7f26c40095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26d946c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdceeb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26d946c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdceeb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45674/0 (socket says 172.21.15.103:45674) 2024-09-19T07:04:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26d946c640 1 -- 172.21.15.103:0/3789697538 learned_addr learned my addr 172.21.15.103:0/3789697538 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26d946c640 1 -- 172.21.15.103:0/3789697538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdced9b40 msgr2=0x555bdceebdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26d946c640 1 --2- 172.21.15.103:0/3789697538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdced9b40 0x555bdceebdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26d946c640 1 -- 172.21.15.103:0/3789697538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdcee3370 msgr2=0x555bdcedcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26d946c640 1 --2- 172.21.15.103:0/3789697538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdcee3370 0x555bdcedcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26d946c640 1 -- 172.21.15.103:0/3789697538 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bdcedd490 con 0x555bdced85d0 2024-09-19T07:04:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26d946c640 1 --2- 172.21.15.103:0/3789697538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdceeb870 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f26c402a930 tx=0x7f26c4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeab3a129c3a07cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26bd7fa640 1 -- 172.21.15.103:0/3789697538 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4035070 con 0x555bdced85d0 2024-09-19T07:04:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26bd7fa640 1 -- 172.21.15.103:0/3789697538 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f26c4031830 con 0x555bdced85d0 2024-09-19T07:04:31.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.133+0000 7f26de2aa740 1 -- 172.21.15.103:0/3789697538 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c4031d10 con 0x555bdced85d0 2024-09-19T07:04:31.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.134+0000 7f26bd7fa640 1 -- 172.21.15.103:0/3789697538 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f26c405b050 con 0x555bdced85d0 2024-09-19T07:04:31.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.134+0000 7f26de2aa740 10 client.20622 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:31.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.134+0000 7f26de2aa740 1 -- 172.21.15.103:0/3789697538 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555bdcf5d5b0 con 0x555bdced85d0 2024-09-19T07:04:31.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.134+0000 7f26bd7fa640 1 -- 172.21.15.103:0/3789697538 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c4031d10 con 0x555bdced85d0 2024-09-19T07:04:31.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.134+0000 7f26bd7fa640 1 -- 172.21.15.103:0/3789697538 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f26c40051d0 con 0x555bdced85d0 2024-09-19T07:04:31.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.134+0000 7f26de2aa740 10 client.20622 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:31.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.134+0000 7f26de2aa740 10 client.20622 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:04:31.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.134+0000 7f26de2aa740 20 client.20622 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:31.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.134+0000 7f26de2aa740 1 --2- 172.21.15.103:0/3789697538 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x555bdcf5dc70 0x555bdcf60090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.134+0000 7f26de2aa740 4 client.20622 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:04:31.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.134+0000 7f26de2aa740 1 -- 172.21.15.103:0/3789697538 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555bdce45d70 con 0x555bdcf5dc70 2024-09-19T07:04:31.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.134+0000 7f26d9c6d640 1 --2- 172.21.15.103:0/3789697538 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x555bdcf5dc70 0x555bdcf60090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26d9c6d640 1 --2- 172.21.15.103:0/3789697538 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x555bdcf5dc70 0x555bdcf60090 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=29467050f9d6980d server_cookie=87c0481ca27dc487 in_seq=0 out_seq=0 2024-09-19T07:04:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26bd7fa640 10 client.20622 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:04:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26bd7fa640 1 -- 172.21.15.103:0/3789697538 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555bdce45d70 con 0x555bdcf5dc70 2024-09-19T07:04:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26bd7fa640 10 client.20622 handle_command_reply: tid=0 2024-09-19T07:04:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26de2aa740 1 client.20622 shutdown 2024-09-19T07:04:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26de2aa740 1 -- 172.21.15.103:0/3789697538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 msgr2=0x555bdceeb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26de2aa740 1 --2- 172.21.15.103:0/3789697538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdceeb870 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f26c402a930 tx=0x7f26c4009260 comp rx=0 tx=0).stop 2024-09-19T07:04:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26de2aa740 1 -- 172.21.15.103:0/3789697538 shutdown_connections 2024-09-19T07:04:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26de2aa740 1 --2- 172.21.15.103:0/3789697538 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x555bdcf5dc70 0x555bdcf60090 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26de2aa740 1 --2- 172.21.15.103:0/3789697538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bdcee3370 0x555bdcedcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26bd7fa640 0 client.20622 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:04:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26de2aa740 1 --2- 172.21.15.103:0/3789697538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bdced9b40 0x555bdceebdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26de2aa740 1 --2- 172.21.15.103:0/3789697538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bdced85d0 0x555bdceeb870 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26de2aa740 1 -- 172.21.15.103:0/3789697538 >> 172.21.15.103:0/3789697538 conn(0x555bdcea1310 msgr2=0x555bdcedf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:31.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26de2aa740 1 -- 172.21.15.103:0/3789697538 shutdown_connections 2024-09-19T07:04:31.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26de2aa740 1 -- 172.21.15.103:0/3789697538 wait complete. 2024-09-19T07:04:31.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.135+0000 7f26de2aa740 20 client.20622 trim_cache size 0 max 16384 2024-09-19T07:04:31.153 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:04:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.136+0000 7f26dbed1640 1 -- 172.21.15.103:0/3833601361 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26c00a33b0 msgr2=0x7f26c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.136+0000 7f26dbed1640 1 --2- 172.21.15.103:0/3833601361 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26c00a33b0 0x7f26c00a5870 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f26cc09c180 tx=0x7f26cc009270 comp rx=0 tx=0).stop 2024-09-19T07:04:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.136+0000 7f26dbed1640 1 -- 172.21.15.103:0/3833601361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d4106f10 msgr2=0x7f26d419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.136+0000 7f26dbed1640 1 --2- 172.21.15.103:0/3833601361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d4106f10 0x7f26d419cd80 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00eaa0 tx=0x7f26cc00ef60 comp rx=0 tx=0).stop 2024-09-19T07:04:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.136+0000 7f26dbed1640 1 -- 172.21.15.103:0/3833601361 shutdown_connections 2024-09-19T07:04:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.136+0000 7f26dbed1640 1 --2- 172.21.15.103:0/3833601361 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26c00a33b0 0x7f26c00a5870 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.136+0000 7f26dbed1640 1 --2- 172.21.15.103:0/3833601361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d4106f10 0x7f26d419cd80 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.136+0000 7f26dbed1640 1 --2- 172.21.15.103:0/3833601361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26d4102790 0x7f26d41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.136+0000 7f26dbed1640 1 --2- 172.21.15.103:0/3833601361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d4101de0 0x7f26d41984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.136+0000 7f26dbed1640 1 -- 172.21.15.103:0/3833601361 >> 172.21.15.103:0/3833601361 conn(0x7f26d40fd8c0 msgr2=0x7f26d40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.136+0000 7f26dbed1640 1 -- 172.21.15.103:0/3833601361 shutdown_connections 2024-09-19T07:04:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:31.136+0000 7f26dbed1640 1 -- 172.21.15.103:0/3833601361 wait complete. 2024-09-19T07:04:31.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:31 smithi103 ceph-mon[31158]: pgmap v479: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 18 op/s 2024-09-19T07:04:31.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:31 smithi174 ceph-mon[35637]: pgmap v479: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 18 op/s 2024-09-19T07:04:31.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:31 smithi134 ceph-mon[35356]: pgmap v479: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 18 op/s 2024-09-19T07:04:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.791+0000 7f0b21496640 1 -- 172.21.15.103:0/3771914091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b1c101370 msgr2=0x7f0b1c10d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.791+0000 7f0b21496640 1 --2- 172.21.15.103:0/3771914091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b1c101370 0x7f0b1c10d8a0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009cd0 tx=0x7f0b1402a8e0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.791+0000 7f0b21496640 1 -- 172.21.15.103:0/3771914091 shutdown_connections 2024-09-19T07:04:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.791+0000 7f0b21496640 1 --2- 172.21.15.103:0/3771914091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b1c101370 0x7f0b1c10d8a0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.791+0000 7f0b21496640 1 --2- 172.21.15.103:0/3771914091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1c1009d0 0x7f0b1c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.791+0000 7f0b21496640 1 --2- 172.21.15.103:0/3771914091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c1069e0 0x7f0b1c106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.791+0000 7f0b21496640 1 -- 172.21.15.103:0/3771914091 >> 172.21.15.103:0/3771914091 conn(0x7f0b1c0fc6b0 msgr2=0x7f0b1c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.791+0000 7f0b21496640 1 -- 172.21.15.103:0/3771914091 shutdown_connections 2024-09-19T07:04:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.791+0000 7f0b21496640 1 -- 172.21.15.103:0/3771914091 wait complete. 2024-09-19T07:04:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.791+0000 7f0b21496640 1 Processor -- start 2024-09-19T07:04:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b21496640 1 -- start start 2024-09-19T07:04:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b21496640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c1009d0 0x7f0b1c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b21496640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b1c101370 0x7f0b1c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b21496640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1c1069e0 0x7f0b1c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b21496640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c110e20 con 0x7f0b1c101370 2024-09-19T07:04:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b21496640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c110ca0 con 0x7f0b1c1009d0 2024-09-19T07:04:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b21496640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c110fa0 con 0x7f0b1c1069e0 2024-09-19T07:04:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b1a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b1c101370 0x7f0b1c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b1a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b1c101370 0x7f0b1c1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45702/0 (socket says 172.21.15.103:45702) 2024-09-19T07:04:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b1a7fc640 1 -- 172.21.15.103:0/57112115 learned_addr learned my addr 172.21.15.103:0/57112115 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b1affd640 1 --2- 172.21.15.103:0/57112115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c1009d0 0x7f0b1c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b1b7fe640 1 --2- 172.21.15.103:0/57112115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1c1069e0 0x7f0b1c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b1a7fc640 1 -- 172.21.15.103:0/57112115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1c1069e0 msgr2=0x7f0b1c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b1a7fc640 1 --2- 172.21.15.103:0/57112115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1c1069e0 0x7f0b1c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b1a7fc640 1 -- 172.21.15.103:0/57112115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c1009d0 msgr2=0x7f0b1c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b1a7fc640 1 --2- 172.21.15.103:0/57112115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c1009d0 0x7f0b1c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b1a7fc640 1 -- 172.21.15.103:0/57112115 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1c19d390 con 0x7f0b1c101370 2024-09-19T07:04:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b1b7fe640 1 --2- 172.21.15.103:0/57112115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1c1069e0 0x7f0b1c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.792+0000 7f0b1affd640 1 --2- 172.21.15.103:0/57112115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c1009d0 0x7f0b1c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.793+0000 7f0b1a7fc640 1 --2- 172.21.15.103:0/57112115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b1c101370 0x7f0b1c1988e0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400b4a0 tx=0x7f0b0400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.793+0000 7f0b13fff640 1 -- 172.21.15.103:0/57112115 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04014070 con 0x7f0b1c101370 2024-09-19T07:04:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.793+0000 7f0b13fff640 1 -- 172.21.15.103:0/57112115 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b04010af0 con 0x7f0b1c101370 2024-09-19T07:04:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.793+0000 7f0b21496640 1 -- 172.21.15.103:0/57112115 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b1c19d680 con 0x7f0b1c101370 2024-09-19T07:04:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.793+0000 7f0b13fff640 1 -- 172.21.15.103:0/57112115 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04017d10 con 0x7f0b1c101370 2024-09-19T07:04:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.793+0000 7f0b21496640 1 -- 172.21.15.103:0/57112115 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b1c1a4f10 con 0x7f0b1c101370 2024-09-19T07:04:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.795+0000 7f0b2422f740 1 Processor -- start 2024-09-19T07:04:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.795+0000 7f0b2422f740 1 -- start start 2024-09-19T07:04:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.795+0000 7f0b13fff640 1 -- 172.21.15.103:0/57112115 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0b04017070 con 0x7f0b1c101370 2024-09-19T07:04:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.795+0000 7f0b2422f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee5caf0 0x564a9ee5ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.795+0000 7f0b2422f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5d410 0x564a9ee67770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.795+0000 7f0b2422f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee67cb0 0x564a9ee6a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.796+0000 7f0b2422f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a9edc12f0 con 0x564a9ee67cb0 2024-09-19T07:04:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.796+0000 7f0b2422f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564a9ed69270 con 0x564a9ee5d410 2024-09-19T07:04:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.796+0000 7f0b2422f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564a9edbd2b0 con 0x564a9ee5caf0 2024-09-19T07:04:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.796+0000 7f0b1a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee67cb0 0x564a9ee6a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.796+0000 7f0b1affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5d410 0x564a9ee67770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.796+0000 7f0b1a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee67cb0 0x564a9ee6a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45706/0 (socket says 172.21.15.103:45706) 2024-09-19T07:04:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.796+0000 7f0b1a7fc640 1 -- 172.21.15.103:0/3868012196 learned_addr learned my addr 172.21.15.103:0/3868012196 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.796+0000 7f0b1b7fe640 1 --2- 172.21.15.103:0/3868012196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee5caf0 0x564a9ee5ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.796+0000 7f0b1affd640 1 -- 172.21.15.103:0/3868012196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee5caf0 msgr2=0x564a9ee5ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.796+0000 7f0b1affd640 1 --2- 172.21.15.103:0/3868012196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee5caf0 0x564a9ee5ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.796+0000 7f0b1affd640 1 -- 172.21.15.103:0/3868012196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee67cb0 msgr2=0x564a9ee6a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.796+0000 7f0b1affd640 1 --2- 172.21.15.103:0/3868012196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee67cb0 0x564a9ee6a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.796+0000 7f0b1affd640 1 -- 172.21.15.103:0/3868012196 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a9ed68340 con 0x564a9ee5d410 2024-09-19T07:04:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.796+0000 7f0b1affd640 1 --2- 172.21.15.103:0/3868012196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5d410 0x564a9ee67770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00cbd0 tx=0x7f0b0c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa644fcd35d90ce1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.796+0000 7f0b13fff640 1 --2- 172.21.15.103:0/57112115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b080a3460 0x7f0b080a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.796+0000 7f0b117fa640 1 -- 172.21.15.103:0/3868012196 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c005220 con 0x564a9ee5d410 2024-09-19T07:04:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.797+0000 7f0b117fa640 1 -- 172.21.15.103:0/3868012196 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b0c004c80 con 0x564a9ee5d410 2024-09-19T07:04:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.797+0000 7f0b117fa640 1 -- 172.21.15.103:0/3868012196 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c010830 con 0x564a9ee5d410 2024-09-19T07:04:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.797+0000 7f0b1b7fe640 1 --2- 172.21.15.103:0/57112115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b080a3460 0x7f0b080a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.797+0000 7f0b13fff640 1 -- 172.21.15.103:0/57112115 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f0b040cf070 con 0x7f0b1c101370 2024-09-19T07:04:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.798+0000 7f0b1b7fe640 1 --2- 172.21.15.103:0/57112115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b080a3460 0x7f0b080a5920 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009430 tx=0x7f0b14009360 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.798+0000 7f0b2422f740 1 -- 172.21.15.103:0/3868012196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5d410 msgr2=0x564a9ee67770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.798+0000 7f0b2422f740 1 --2- 172.21.15.103:0/3868012196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5d410 0x564a9ee67770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00cbd0 tx=0x7f0b0c00e720 comp rx=0 tx=0).stop 2024-09-19T07:04:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.798+0000 7f0b2422f740 1 -- 172.21.15.103:0/3868012196 shutdown_connections 2024-09-19T07:04:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.798+0000 7f0b2422f740 1 --2- 172.21.15.103:0/3868012196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee67cb0 0x564a9ee6a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.798+0000 7f0b2422f740 1 --2- 172.21.15.103:0/3868012196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5d410 0x564a9ee67770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.798+0000 7f0b2422f740 1 --2- 172.21.15.103:0/3868012196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee5caf0 0x564a9ee5ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.798+0000 7f0b2422f740 1 -- 172.21.15.103:0/3868012196 >> 172.21.15.103:0/3868012196 conn(0x564a9ee59060 msgr2=0x564a9ee59440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.798+0000 7f0b2422f740 1 -- 172.21.15.103:0/3868012196 shutdown_connections 2024-09-19T07:04:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.798+0000 7f0b2422f740 1 -- 172.21.15.103:0/3868012196 wait complete. 2024-09-19T07:04:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.798+0000 7f0b2422f740 1 -- start start 2024-09-19T07:04:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.798+0000 7f0b2422f740 1 Processor -- start 2024-09-19T07:04:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b2422f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee5caf0 0x564a9ee64b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b2422f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5d410 0x564a9ee65050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b2422f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee67cb0 0x564a9ee65590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b2422f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a9ee6a880 con 0x564a9ee5caf0 2024-09-19T07:04:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b2422f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564a9ed4a290 con 0x564a9ee5d410 2024-09-19T07:04:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b2422f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564a9ee6aa00 con 0x564a9ee67cb0 2024-09-19T07:04:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b2422f740 10 client.0 _ll_register_callbacks cb 0x564a9ed7b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b1affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee5caf0 0x564a9ee64b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b1affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee5caf0 0x564a9ee64b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45722/0 (socket says 172.21.15.103:45722) 2024-09-19T07:04:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b1affd640 1 -- 172.21.15.103:0/1268039125 learned_addr learned my addr 172.21.15.103:0/1268039125 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b1b7fe640 1 --2- 172.21.15.103:0/1268039125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee67cb0 0x564a9ee65590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b1affd640 1 -- 172.21.15.103:0/1268039125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee67cb0 msgr2=0x564a9ee65590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b1affd640 1 --2- 172.21.15.103:0/1268039125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee67cb0 0x564a9ee65590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b1affd640 1 -- 172.21.15.103:0/1268039125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5d410 msgr2=0x564a9ee65050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:31.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b1a7fc640 1 --2- 172.21.15.103:0/1268039125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5d410 0x564a9ee65050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b1affd640 1 --2- 172.21.15.103:0/1268039125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5d410 0x564a9ee65050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b1affd640 1 -- 172.21.15.103:0/1268039125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a9ead4210 con 0x564a9ee5caf0 2024-09-19T07:04:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b1b7fe640 1 --2- 172.21.15.103:0/1268039125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee67cb0 0x564a9ee65590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b1affd640 1 --2- 172.21.15.103:0/1268039125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee5caf0 0x564a9ee64b10 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0057b0 tx=0x7f0b0c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b846b565f3466bb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b027fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.799+0000 7f0b027fc640 1 -- 172.21.15.103:0/1268039125 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c02a060 con 0x564a9ee5caf0 2024-09-19T07:04:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.800+0000 7f0b027fc640 1 -- 172.21.15.103:0/1268039125 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f0b0c0048d0 con 0x564a9ee5caf0 2024-09-19T07:04:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.800+0000 7f0b2422f740 1 -- 172.21.15.103:0/1268039125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b0c004c80 con 0x564a9ee5caf0 2024-09-19T07:04:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.800+0000 7f0b027fc640 1 -- 172.21.15.103:0/1268039125 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0b0c004ad0 con 0x564a9ee5caf0 2024-09-19T07:04:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.800+0000 7f0b2422f740 10 client.20634 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.800+0000 7f0b2422f740 1 -- 172.21.15.103:0/1268039125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564a9eba0620 con 0x564a9ee5caf0 2024-09-19T07:04:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.800+0000 7f0b027fc640 1 -- 172.21.15.103:0/1268039125 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b0c004c80 con 0x564a9ee5caf0 2024-09-19T07:04:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.800+0000 7f0b027fc640 1 -- 172.21.15.103:0/1268039125 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0b0c01a6a0 con 0x564a9ee5caf0 2024-09-19T07:04:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.800+0000 7f0b2422f740 10 client.20634 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.800+0000 7f0b2422f740 10 client.20634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.800+0000 7f0b2422f740 20 client.20634 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.800+0000 7f0b2422f740 1 --2- 172.21.15.103:0/1268039125 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a9ee77b40 0x564a9ee79f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.800+0000 7f0b2422f740 4 client.20634 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.800+0000 7f0b2422f740 1 -- 172.21.15.103:0/1268039125 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a9edcea20 con 0x564a9ee77b40 2024-09-19T07:04:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.801+0000 7f0b1a7fc640 1 --2- 172.21.15.103:0/1268039125 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a9ee77b40 0x564a9ee79f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.801+0000 7f0b1a7fc640 1 --2- 172.21.15.103:0/1268039125 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a9ee77b40 0x564a9ee79f30 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cae6740393feb420 server_cookie=575513fa47d25617 in_seq=0 out_seq=0 2024-09-19T07:04:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.801+0000 7f0b027fc640 10 client.20634 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.802+0000 7f0b027fc640 1 -- 172.21.15.103:0/1268039125 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a9edcea20 con 0x564a9ee77b40 2024-09-19T07:04:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.802+0000 7f0b027fc640 10 client.20634 handle_command_reply: tid=0 2024-09-19T07:04:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.802+0000 7f0b2422f740 1 client.20634 shutdown 2024-09-19T07:04:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.802+0000 7f0b2422f740 1 -- 172.21.15.103:0/1268039125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee5caf0 msgr2=0x564a9ee64b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.802+0000 7f0b2422f740 1 --2- 172.21.15.103:0/1268039125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee5caf0 0x564a9ee64b10 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0057b0 tx=0x7f0b0c00ced0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.802+0000 7f0b2422f740 1 -- 172.21.15.103:0/1268039125 shutdown_connections 2024-09-19T07:04:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.802+0000 7f0b2422f740 1 --2- 172.21.15.103:0/1268039125 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a9ee77b40 0x564a9ee79f30 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.802+0000 7f0b2422f740 1 --2- 172.21.15.103:0/1268039125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee67cb0 0x564a9ee65590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.802+0000 7f0b2422f740 1 --2- 172.21.15.103:0/1268039125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5d410 0x564a9ee65050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.802+0000 7f0b027fc640 0 client.20634 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.802+0000 7f0b2422f740 1 --2- 172.21.15.103:0/1268039125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee5caf0 0x564a9ee64b10 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.802+0000 7f0b2422f740 1 -- 172.21.15.103:0/1268039125 >> 172.21.15.103:0/1268039125 conn(0x564a9ee59060 msgr2=0x564a9ee5ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.802+0000 7f0b2422f740 1 -- 172.21.15.103:0/1268039125 shutdown_connections 2024-09-19T07:04:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.802+0000 7f0b2422f740 1 -- 172.21.15.103:0/1268039125 wait complete. 2024-09-19T07:04:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.802+0000 7f0b2422f740 20 client.20634 trim_cache size 0 max 16384 2024-09-19T07:04:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.829+0000 7f0b2422f740 1 Processor -- start 2024-09-19T07:04:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.829+0000 7f0b2422f740 1 -- start start 2024-09-19T07:04:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.829+0000 7f0b2422f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee5caf0 0x564a9ee65040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.829+0000 7f0b2422f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5d410 0x564a9ee65580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.829+0000 7f0b2422f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee67cb0 0x564a9ee7f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.829+0000 7f0b2422f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c1ad8d0 con 0x564a9ee67cb0 2024-09-19T07:04:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.829+0000 7f0b2422f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c1af410 con 0x564a9ee5d410 2024-09-19T07:04:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.829+0000 7f0b2422f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c1adfa0 con 0x564a9ee5caf0 2024-09-19T07:04:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.830+0000 7f0b1b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee67cb0 0x564a9ee7f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.830+0000 7f0b1b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee67cb0 0x564a9ee7f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45738/0 (socket says 172.21.15.103:45738) 2024-09-19T07:04:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.830+0000 7f0b1b7fe640 1 -- 172.21.15.103:0/574351340 learned_addr learned my addr 172.21.15.103:0/574351340 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.830+0000 7f0b1affd640 1 --2- 172.21.15.103:0/574351340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee5caf0 0x564a9ee65040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.830+0000 7f0b1a7fc640 1 --2- 172.21.15.103:0/574351340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5d410 0x564a9ee65580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.830+0000 7f0b1b7fe640 1 -- 172.21.15.103:0/574351340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee5caf0 msgr2=0x564a9ee65040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.830+0000 7f0b1b7fe640 1 --2- 172.21.15.103:0/574351340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee5caf0 0x564a9ee65040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.830+0000 7f0b1b7fe640 1 -- 172.21.15.103:0/574351340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5d410 msgr2=0x564a9ee65580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.830+0000 7f0b1b7fe640 1 --2- 172.21.15.103:0/574351340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5d410 0x564a9ee65580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.830+0000 7f0b1b7fe640 1 -- 172.21.15.103:0/574351340 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a9ee5d240 con 0x564a9ee67cb0 2024-09-19T07:04:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.830+0000 7f0b1affd640 1 --2- 172.21.15.103:0/574351340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee5caf0 0x564a9ee65040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.830+0000 7f0b1b7fe640 1 --2- 172.21.15.103:0/574351340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee67cb0 0x564a9ee7f530 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f0b14031860 tx=0x7f0b1403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee15b0b66bb63e9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.830+0000 7f0b10ff9640 1 -- 172.21.15.103:0/574351340 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14004240 con 0x564a9ee67cb0 2024-09-19T07:04:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.831+0000 7f0b10ff9640 1 -- 172.21.15.103:0/574351340 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b14036040 con 0x564a9ee67cb0 2024-09-19T07:04:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.831+0000 7f0b10ff9640 1 -- 172.21.15.103:0/574351340 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14051480 con 0x564a9ee67cb0 2024-09-19T07:04:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.831+0000 7f0b2422f740 1 -- 172.21.15.103:0/574351340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee67cb0 msgr2=0x564a9ee7f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.831+0000 7f0b2422f740 1 --2- 172.21.15.103:0/574351340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee67cb0 0x564a9ee7f530 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f0b14031860 tx=0x7f0b1403f000 comp rx=0 tx=0).stop 2024-09-19T07:04:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b2422f740 1 -- 172.21.15.103:0/574351340 shutdown_connections 2024-09-19T07:04:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b2422f740 1 --2- 172.21.15.103:0/574351340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee67cb0 0x564a9ee7f530 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b2422f740 1 --2- 172.21.15.103:0/574351340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5d410 0x564a9ee65580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b2422f740 1 --2- 172.21.15.103:0/574351340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee5caf0 0x564a9ee65040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b2422f740 1 -- 172.21.15.103:0/574351340 >> 172.21.15.103:0/574351340 conn(0x564a9ee59060 msgr2=0x564a9ee66b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b2422f740 1 -- 172.21.15.103:0/574351340 shutdown_connections 2024-09-19T07:04:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b2422f740 1 -- 172.21.15.103:0/574351340 wait complete. 2024-09-19T07:04:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b2422f740 1 -- start start 2024-09-19T07:04:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b2422f740 1 Processor -- start 2024-09-19T07:04:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b2422f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5caf0 0x564a9ee702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b2422f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee5d410 0x564a9ee616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b2422f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee67cb0 0x564a9ee61bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b2422f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c1aed40 con 0x564a9ee5d410 2024-09-19T07:04:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b2422f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c1ae670 con 0x564a9ee5caf0 2024-09-19T07:04:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b2422f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c1ab8d0 con 0x564a9ee67cb0 2024-09-19T07:04:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b2422f740 10 client.0 _ll_register_callbacks cb 0x7f0b0c014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b1a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee5d410 0x564a9ee616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b1a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee5d410 0x564a9ee616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45744/0 (socket says 172.21.15.103:45744) 2024-09-19T07:04:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.832+0000 7f0b1a7fc640 1 -- 172.21.15.103:0/2980933113 learned_addr learned my addr 172.21.15.103:0/2980933113 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b1affd640 1 --2- 172.21.15.103:0/2980933113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5caf0 0x564a9ee702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b1a7fc640 1 -- 172.21.15.103:0/2980933113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee67cb0 msgr2=0x564a9ee61bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b1a7fc640 1 --2- 172.21.15.103:0/2980933113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee67cb0 0x564a9ee61bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b1a7fc640 1 -- 172.21.15.103:0/2980933113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5caf0 msgr2=0x564a9ee702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b1a7fc640 1 --2- 172.21.15.103:0/2980933113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5caf0 0x564a9ee702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b1a7fc640 1 -- 172.21.15.103:0/2980933113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a9ee62290 con 0x564a9ee5d410 2024-09-19T07:04:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b1affd640 1 --2- 172.21.15.103:0/2980933113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5caf0 0x564a9ee702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b1a7fc640 1 --2- 172.21.15.103:0/2980933113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee5d410 0x564a9ee616b0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f0b040cc960 tx=0x7f0b04017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b283c01605638ca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b027fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b027fc640 1 -- 172.21.15.103:0/2980933113 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0401e040 con 0x564a9ee5d410 2024-09-19T07:04:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b027fc640 1 -- 172.21.15.103:0/2980933113 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f0b040d2d70 con 0x564a9ee5d410 2024-09-19T07:04:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b2422f740 1 -- 172.21.15.103:0/2980933113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b14036040 con 0x564a9ee5d410 2024-09-19T07:04:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b027fc640 1 -- 172.21.15.103:0/2980933113 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0b0409ada0 con 0x564a9ee5d410 2024-09-19T07:04:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b2422f740 10 client.20640 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b2422f740 1 -- 172.21.15.103:0/2980933113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564a9eee2420 con 0x564a9ee5d410 2024-09-19T07:04:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b027fc640 1 -- 172.21.15.103:0/2980933113 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b14036040 con 0x564a9ee5d410 2024-09-19T07:04:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b027fc640 1 -- 172.21.15.103:0/2980933113 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0b040cca20 con 0x564a9ee5d410 2024-09-19T07:04:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b2422f740 10 client.20640 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b2422f740 10 client.20640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b2422f740 20 client.20640 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b2422f740 1 --2- 172.21.15.103:0/2980933113 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a9eee2b70 0x564a9eee4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b2422f740 4 client.20640 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:04:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.833+0000 7f0b2422f740 1 -- 172.21.15.103:0/2980933113 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a9edc8d70 con 0x564a9eee2b70 2024-09-19T07:04:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.834+0000 7f0b1affd640 1 --2- 172.21.15.103:0/2980933113 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a9eee2b70 0x564a9eee4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.834+0000 7f0b027fc640 10 client.20640 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.834+0000 7f0b1affd640 1 --2- 172.21.15.103:0/2980933113 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a9eee2b70 0x564a9eee4f90 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8f54c2cdd1099d server_cookie=2c083795eed283d3 in_seq=0 out_seq=0 2024-09-19T07:04:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.847+0000 7f0b027fc640 1 -- 172.21.15.103:0/2980933113 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a9edc8d70 con 0x564a9eee2b70 2024-09-19T07:04:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.847+0000 7f0b027fc640 10 client.20640 handle_command_reply: tid=0 2024-09-19T07:04:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.847+0000 7f0b2422f740 1 client.20640 shutdown 2024-09-19T07:04:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.847+0000 7f0b2422f740 1 -- 172.21.15.103:0/2980933113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee5d410 msgr2=0x564a9ee616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.847+0000 7f0b2422f740 1 --2- 172.21.15.103:0/2980933113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee5d410 0x564a9ee616b0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f0b040cc960 tx=0x7f0b04017730 comp rx=0 tx=0).stop 2024-09-19T07:04:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.847+0000 7f0b2422f740 1 -- 172.21.15.103:0/2980933113 shutdown_connections 2024-09-19T07:04:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.847+0000 7f0b2422f740 1 --2- 172.21.15.103:0/2980933113 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x564a9eee2b70 0x564a9eee4f90 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.847+0000 7f0b2422f740 1 --2- 172.21.15.103:0/2980933113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9ee67cb0 0x564a9ee61bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.847+0000 7f0b027fc640 0 client.20640 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.847+0000 7f0b2422f740 1 --2- 172.21.15.103:0/2980933113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a9ee5d410 0x564a9ee616b0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.847+0000 7f0b2422f740 1 --2- 172.21.15.103:0/2980933113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a9ee5caf0 0x564a9ee702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.847+0000 7f0b2422f740 1 -- 172.21.15.103:0/2980933113 >> 172.21.15.103:0/2980933113 conn(0x564a9ee59060 msgr2=0x564a9ee579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.848+0000 7f0b2422f740 1 -- 172.21.15.103:0/2980933113 shutdown_connections 2024-09-19T07:04:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.848+0000 7f0b2422f740 1 -- 172.21.15.103:0/2980933113 wait complete. 2024-09-19T07:04:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.848+0000 7f0b2422f740 20 client.20640 trim_cache size 0 max 16384 2024-09-19T07:04:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:04:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:04:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:04:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:04:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.848+0000 7f0b21496640 1 -- 172.21.15.103:0/57112115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b080a3460 msgr2=0x7f0b080a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.848+0000 7f0b21496640 1 --2- 172.21.15.103:0/57112115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b080a3460 0x7f0b080a5920 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009430 tx=0x7f0b14009360 comp rx=0 tx=0).stop 2024-09-19T07:04:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.848+0000 7f0b21496640 1 -- 172.21.15.103:0/57112115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b1c101370 msgr2=0x7f0b1c1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.848+0000 7f0b21496640 1 --2- 172.21.15.103:0/57112115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b1c101370 0x7f0b1c1988e0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400b4a0 tx=0x7f0b0400b960 comp rx=0 tx=0).stop 2024-09-19T07:04:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.848+0000 7f0b21496640 1 -- 172.21.15.103:0/57112115 shutdown_connections 2024-09-19T07:04:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.848+0000 7f0b21496640 1 --2- 172.21.15.103:0/57112115 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0b080a3460 0x7f0b080a5920 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.848+0000 7f0b21496640 1 --2- 172.21.15.103:0/57112115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1c1069e0 0x7f0b1c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.848+0000 7f0b21496640 1 --2- 172.21.15.103:0/57112115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b1c101370 0x7f0b1c1988e0 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.848+0000 7f0b21496640 1 --2- 172.21.15.103:0/57112115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c1009d0 0x7f0b1c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.848+0000 7f0b21496640 1 -- 172.21.15.103:0/57112115 >> 172.21.15.103:0/57112115 conn(0x7f0b1c0fc6b0 msgr2=0x7f0b1c06a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.849+0000 7f0b21496640 1 -- 172.21.15.103:0/57112115 shutdown_connections 2024-09-19T07:04:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:31.849+0000 7f0b21496640 1 -- 172.21.15.103:0/57112115 wait complete. 2024-09-19T07:04:31.875 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:04:32.130 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:32.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.625+0000 7fe49a8a4640 1 -- 172.21.15.103:0/4173099812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe494106f10 msgr2=0x7fe49410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:32.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.625+0000 7fe49a8a4640 1 --2- 172.21.15.103:0/4173099812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe494106f10 0x7fe49410ea90 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009ca0 tx=0x7fe48c02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.625+0000 7fe49a8a4640 1 -- 172.21.15.103:0/4173099812 shutdown_connections 2024-09-19T07:04:32.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.625+0000 7fe49a8a4640 1 --2- 172.21.15.103:0/4173099812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe494106f10 0x7fe49410ea90 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.625+0000 7fe49a8a4640 1 --2- 172.21.15.103:0/4173099812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe494102790 0x7fe4941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.625+0000 7fe49a8a4640 1 --2- 172.21.15.103:0/4173099812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe494101de0 0x7fe4941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.625+0000 7fe49a8a4640 1 -- 172.21.15.103:0/4173099812 >> 172.21.15.103:0/4173099812 conn(0x7fe4940fd8c0 msgr2=0x7fe4940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:32.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.625+0000 7fe49a8a4640 1 -- 172.21.15.103:0/4173099812 shutdown_connections 2024-09-19T07:04:32.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.625+0000 7fe49a8a4640 1 -- 172.21.15.103:0/4173099812 wait complete. 2024-09-19T07:04:32.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.626+0000 7fe49a8a4640 1 Processor -- start 2024-09-19T07:04:32.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.626+0000 7fe49a8a4640 1 -- start start 2024-09-19T07:04:32.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe49a8a4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe494101de0 0x7fe494198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe49a8a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe494102790 0x7fe494198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe49a8a4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe494106f10 0x7fe49419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe49a8a4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe494112010 con 0x7fe494106f10 2024-09-19T07:04:32.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe49a8a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe494111e90 con 0x7fe494102790 2024-09-19T07:04:32.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe49a8a4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe494112190 con 0x7fe494101de0 2024-09-19T07:04:32.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe498e41640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe494106f10 0x7fe49419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe498e41640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe494106f10 0x7fe49419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45756/0 (socket says 172.21.15.103:45756) 2024-09-19T07:04:32.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe498e41640 1 -- 172.21.15.103:0/4098703253 learned_addr learned my addr 172.21.15.103:0/4098703253 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:32.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe493fff640 1 --2- 172.21.15.103:0/4098703253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe494101de0 0x7fe494198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe4937fe640 1 --2- 172.21.15.103:0/4098703253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe494102790 0x7fe494198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe498e41640 1 -- 172.21.15.103:0/4098703253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe494101de0 msgr2=0x7fe494198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:32.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe498e41640 1 --2- 172.21.15.103:0/4098703253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe494101de0 0x7fe494198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe498e41640 1 -- 172.21.15.103:0/4098703253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe494102790 msgr2=0x7fe494198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:32.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe498e41640 1 --2- 172.21.15.103:0/4098703253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe494102790 0x7fe494198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe498e41640 1 -- 172.21.15.103:0/4098703253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49419d520 con 0x7fe494106f10 2024-09-19T07:04:32.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe4937fe640 1 --2- 172.21.15.103:0/4098703253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe494102790 0x7fe494198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:32.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe498e41640 1 --2- 172.21.15.103:0/4098703253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe494106f10 0x7fe49419ce40 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fe48c031a60 tx=0x7fe48c031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:32.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe4917fa640 1 -- 172.21.15.103:0/4098703253 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c003b70 con 0x7fe494106f10 2024-09-19T07:04:32.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe4917fa640 1 -- 172.21.15.103:0/4098703253 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe48c003d10 con 0x7fe494106f10 2024-09-19T07:04:32.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe49a8a4640 1 -- 172.21.15.103:0/4098703253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49419d7b0 con 0x7fe494106f10 2024-09-19T07:04:32.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe4917fa640 1 -- 172.21.15.103:0/4098703253 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c004770 con 0x7fe494106f10 2024-09-19T07:04:32.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.627+0000 7fe49a8a4640 1 -- 172.21.15.103:0/4098703253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4941a50f0 con 0x7fe494106f10 2024-09-19T07:04:32.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.630+0000 7fe4917fa640 1 -- 172.21.15.103:0/4098703253 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fe48c02f790 con 0x7fe494106f10 2024-09-19T07:04:32.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.630+0000 7fe49cc7d740 1 Processor -- start 2024-09-19T07:04:32.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.630+0000 7fe49cc7d740 1 -- start start 2024-09-19T07:04:32.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.630+0000 7fe49cc7d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abeeb40 0x561e5abed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.630+0000 7fe49cc7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abed5d0 0x561e5abeda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.630+0000 7fe49cc7d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abf8370 0x561e5abfa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.630+0000 7fe49cc7d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561e5ab532f0 con 0x561e5abf8370 2024-09-19T07:04:32.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.630+0000 7fe49cc7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561e5aafacf0 con 0x561e5abed5d0 2024-09-19T07:04:32.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.630+0000 7fe49cc7d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561e5ab4f2b0 con 0x561e5abeeb40 2024-09-19T07:04:32.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.630+0000 7fe4937fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abf8370 0x561e5abfa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.630+0000 7fe4917fa640 1 --2- 172.21.15.103:0/4098703253 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe47c0a3430 0x7fe47c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.630+0000 7fe4937fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abf8370 0x561e5abfa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45758/0 (socket says 172.21.15.103:45758) 2024-09-19T07:04:32.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.631+0000 7fe4937fe640 1 -- 172.21.15.103:0/554175043 learned_addr learned my addr 172.21.15.103:0/554175043 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:32.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.631+0000 7fe493fff640 1 --2- 172.21.15.103:0/554175043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abed5d0 0x561e5abeda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.631+0000 7fe4937fe640 1 --2- 172.21.15.103:0/554175043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abeeb40 0x561e5abed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.631+0000 7fe498e41640 1 --2- 172.21.15.103:0/4098703253 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe47c0a3430 0x7fe47c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.631+0000 7fe4937fe640 1 -- 172.21.15.103:0/554175043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abeeb40 msgr2=0x561e5abed090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:32.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.631+0000 7fe4937fe640 1 --2- 172.21.15.103:0/554175043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abeeb40 0x561e5abed090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.631+0000 7fe4917fa640 1 -- 172.21.15.103:0/4098703253 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fe48c0f0f80 con 0x7fe494106f10 2024-09-19T07:04:32.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.631+0000 7fe4937fe640 1 -- 172.21.15.103:0/554175043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abed5d0 msgr2=0x561e5abeda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:32.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.631+0000 7fe4937fe640 1 --2- 172.21.15.103:0/554175043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abed5d0 0x561e5abeda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.631+0000 7fe4937fe640 1 -- 172.21.15.103:0/554175043 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5aadf3a0 con 0x561e5abf8370 2024-09-19T07:04:32.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.631+0000 7fe493fff640 1 --2- 172.21.15.103:0/554175043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abed5d0 0x561e5abeda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:32.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.631+0000 7fe4937fe640 1 --2- 172.21.15.103:0/554175043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abf8370 0x561e5abfa760 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fe488016560 tx=0x7fe48800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8458c1fdad1ab75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:32.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.631+0000 7fe47a7fc640 1 -- 172.21.15.103:0/554175043 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488018430 con 0x561e5abf8370 2024-09-19T07:04:32.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.631+0000 7fe498e41640 1 --2- 172.21.15.103:0/4098703253 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe47c0a3430 0x7fe47c0a58f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0bd110 tx=0x7fe48c0b8f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:32.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.631+0000 7fe47a7fc640 1 -- 172.21.15.103:0/554175043 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe488018ab0 con 0x561e5abf8370 2024-09-19T07:04:32.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.631+0000 7fe47a7fc640 1 -- 172.21.15.103:0/554175043 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488017870 con 0x561e5abf8370 2024-09-19T07:04:32.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.632+0000 7fe49cc7d740 1 -- 172.21.15.103:0/554175043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abf8370 msgr2=0x561e5abfa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:32.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.632+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/554175043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abf8370 0x561e5abfa760 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fe488016560 tx=0x7fe48800a340 comp rx=0 tx=0).stop 2024-09-19T07:04:32.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.632+0000 7fe49cc7d740 1 -- 172.21.15.103:0/554175043 shutdown_connections 2024-09-19T07:04:32.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.632+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/554175043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abf8370 0x561e5abfa760 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.632+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/554175043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abed5d0 0x561e5abeda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.632+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/554175043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abeeb40 0x561e5abed090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.632+0000 7fe49cc7d740 1 -- 172.21.15.103:0/554175043 >> 172.21.15.103:0/554175043 conn(0x561e5abb6310 msgr2=0x561e5abe9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:32.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.632+0000 7fe49cc7d740 1 -- 172.21.15.103:0/554175043 shutdown_connections 2024-09-19T07:04:32.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.632+0000 7fe49cc7d740 1 -- 172.21.15.103:0/554175043 wait complete. 2024-09-19T07:04:32.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.632+0000 7fe49cc7d740 1 -- start start 2024-09-19T07:04:32.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe49cc7d740 1 Processor -- start 2024-09-19T07:04:32.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe49cc7d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abed5d0 0x561e5abf52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe49cc7d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abeeb40 0x561e5abf5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe49cc7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abf5d70 0x561e5ac08810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe49cc7d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561e5abff360 con 0x561e5abeeb40 2024-09-19T07:04:32.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe49cc7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561e5ab0e060 con 0x561e5abf5d70 2024-09-19T07:04:32.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe49cc7d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561e5abff4e0 con 0x561e5abed5d0 2024-09-19T07:04:32.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe49cc7d740 10 client.0 _ll_register_callbacks cb 0x561e5a8618c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:32.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe493fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abeeb40 0x561e5abf5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe493fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abeeb40 0x561e5abf5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45770/0 (socket says 172.21.15.103:45770) 2024-09-19T07:04:32.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe493fff640 1 -- 172.21.15.103:0/2449065049 learned_addr learned my addr 172.21.15.103:0/2449065049 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:32.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe4937fe640 1 --2- 172.21.15.103:0/2449065049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abf5d70 0x561e5ac08810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe4937fe640 1 --2- 172.21.15.103:0/2449065049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abed5d0 0x561e5abf52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe493fff640 1 -- 172.21.15.103:0/2449065049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abed5d0 msgr2=0x561e5abf52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:32.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe493fff640 1 --2- 172.21.15.103:0/2449065049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abed5d0 0x561e5abf52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe493fff640 1 -- 172.21.15.103:0/2449065049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abf5d70 msgr2=0x561e5ac08810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:32.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe493fff640 1 --2- 172.21.15.103:0/2449065049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abf5d70 0x561e5ac08810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.633+0000 7fe493fff640 1 -- 172.21.15.103:0/2449065049 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e5a8c43c0 con 0x561e5abeeb40 2024-09-19T07:04:32.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.634+0000 7fe493fff640 1 --2- 172.21.15.103:0/2449065049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abeeb40 0x561e5abf5830 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fe484011540 tx=0x7fe484011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75792493f2be0f86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:32.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.634+0000 7fe4737fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:32.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.634+0000 7fe4737fe640 1 -- 172.21.15.103:0/2449065049 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48400f6d0 con 0x561e5abeeb40 2024-09-19T07:04:32.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.634+0000 7fe49cc7d740 1 -- 172.21.15.103:0/2449065049 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe488018ab0 con 0x561e5abeeb40 2024-09-19T07:04:32.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.634+0000 7fe4737fe640 1 -- 172.21.15.103:0/2449065049 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fe48400f870 con 0x561e5abeeb40 2024-09-19T07:04:32.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.634+0000 7fe4737fe640 1 -- 172.21.15.103:0/2449065049 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe48400fa70 con 0x561e5abeeb40 2024-09-19T07:04:32.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.634+0000 7fe49cc7d740 10 client.20652 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:32.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.634+0000 7fe49cc7d740 1 -- 172.21.15.103:0/2449065049 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e5ac0b510 con 0x561e5abeeb40 2024-09-19T07:04:32.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.634+0000 7fe4737fe640 1 -- 172.21.15.103:0/2449065049 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe488018ab0 con 0x561e5abeeb40 2024-09-19T07:04:32.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.634+0000 7fe4737fe640 1 -- 172.21.15.103:0/2449065049 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe48401d400 con 0x561e5abeeb40 2024-09-19T07:04:32.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.634+0000 7fe49cc7d740 10 client.20652 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:32.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.634+0000 7fe49cc7d740 10 client.20652 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:04:32.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.634+0000 7fe49cc7d740 20 client.20652 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:32.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.634+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/2449065049 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x561e5ac0ca20 0x561e5ac0ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.634+0000 7fe49cc7d740 4 client.20652 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:32.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.634+0000 7fe49cc7d740 1 -- 172.21.15.103:0/2449065049 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e5ab60860 con 0x561e5ac0ca20 2024-09-19T07:04:32.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.635+0000 7fe4937fe640 1 --2- 172.21.15.103:0/2449065049 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x561e5ac0ca20 0x561e5ac0ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.635+0000 7fe4937fe640 1 --2- 172.21.15.103:0/2449065049 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x561e5ac0ca20 0x561e5ac0ee40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f83ac4a8d3a3249c server_cookie=4fa9810d440a8eb7 in_seq=0 out_seq=0 2024-09-19T07:04:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.635+0000 7fe4737fe640 10 client.20652 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:04:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.636+0000 7fe4737fe640 1 -- 172.21.15.103:0/2449065049 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e5ab60860 con 0x561e5ac0ca20 2024-09-19T07:04:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.636+0000 7fe4737fe640 10 client.20652 handle_command_reply: tid=0 2024-09-19T07:04:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.636+0000 7fe49cc7d740 1 client.20652 shutdown 2024-09-19T07:04:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.636+0000 7fe49cc7d740 1 -- 172.21.15.103:0/2449065049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abeeb40 msgr2=0x561e5abf5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.636+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/2449065049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abeeb40 0x561e5abf5830 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fe484011540 tx=0x7fe484011a00 comp rx=0 tx=0).stop 2024-09-19T07:04:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.636+0000 7fe49cc7d740 1 -- 172.21.15.103:0/2449065049 shutdown_connections 2024-09-19T07:04:32.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.636+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/2449065049 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x561e5ac0ca20 0x561e5ac0ee40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.636+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/2449065049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abf5d70 0x561e5ac08810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.636+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/2449065049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abeeb40 0x561e5abf5830 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.636+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/2449065049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abed5d0 0x561e5abf52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.636+0000 7fe4737fe640 0 client.20652 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:04:32.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.636+0000 7fe49cc7d740 1 -- 172.21.15.103:0/2449065049 >> 172.21.15.103:0/2449065049 conn(0x561e5abb6310 msgr2=0x561e5abef960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:32.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.636+0000 7fe49cc7d740 1 -- 172.21.15.103:0/2449065049 shutdown_connections 2024-09-19T07:04:32.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.636+0000 7fe49cc7d740 1 -- 172.21.15.103:0/2449065049 wait complete. 2024-09-19T07:04:32.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.636+0000 7fe49cc7d740 20 client.20652 trim_cache size 0 max 16384 2024-09-19T07:04:32.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.649+0000 7fe49cc7d740 1 Processor -- start 2024-09-19T07:04:32.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.649+0000 7fe49cc7d740 1 -- start start 2024-09-19T07:04:32.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.649+0000 7fe49cc7d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abed5d0 0x561e5ac085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe49cc7d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abeeb40 0x561e5ac146f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe49cc7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abf5d70 0x561e5ac14c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe49cc7d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941ab7c0 con 0x561e5abeeb40 2024-09-19T07:04:32.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe49cc7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941ad300 con 0x561e5abf5d70 2024-09-19T07:04:32.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe49cc7d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941abe90 con 0x561e5abed5d0 2024-09-19T07:04:32.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe493fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abeeb40 0x561e5ac146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe493fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abeeb40 0x561e5ac146f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45784/0 (socket says 172.21.15.103:45784) 2024-09-19T07:04:32.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe493fff640 1 -- 172.21.15.103:0/867193740 learned_addr learned my addr 172.21.15.103:0/867193740 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:32.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe4937fe640 1 --2- 172.21.15.103:0/867193740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abed5d0 0x561e5ac085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe493fff640 1 -- 172.21.15.103:0/867193740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abed5d0 msgr2=0x561e5ac085c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:32.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe493fff640 1 --2- 172.21.15.103:0/867193740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abed5d0 0x561e5ac085c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe493fff640 1 -- 172.21.15.103:0/867193740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abf5d70 msgr2=0x561e5ac14c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:32.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe493fff640 1 --2- 172.21.15.103:0/867193740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abf5d70 0x561e5ac14c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe493fff640 1 -- 172.21.15.103:0/867193740 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5abe94f0 con 0x561e5abeeb40 2024-09-19T07:04:32.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe4937fe640 1 --2- 172.21.15.103:0/867193740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abed5d0 0x561e5ac085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:32.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe493fff640 1 --2- 172.21.15.103:0/867193740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abeeb40 0x561e5ac146f0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fe484009520 tx=0x7fe484009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69dbb04c84b9d071 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:32.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe479ffb640 1 -- 172.21.15.103:0/867193740 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48402a070 con 0x561e5abeeb40 2024-09-19T07:04:32.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe479ffb640 1 -- 172.21.15.103:0/867193740 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe48401d400 con 0x561e5abeeb40 2024-09-19T07:04:32.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.650+0000 7fe479ffb640 1 -- 172.21.15.103:0/867193740 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484026930 con 0x561e5abeeb40 2024-09-19T07:04:32.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.651+0000 7fe49cc7d740 1 -- 172.21.15.103:0/867193740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abeeb40 msgr2=0x561e5ac146f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:32.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.651+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/867193740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abeeb40 0x561e5ac146f0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fe484009520 tx=0x7fe484009220 comp rx=0 tx=0).stop 2024-09-19T07:04:32.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.651+0000 7fe49cc7d740 1 -- 172.21.15.103:0/867193740 shutdown_connections 2024-09-19T07:04:32.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.651+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/867193740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abf5d70 0x561e5ac14c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.651+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/867193740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abeeb40 0x561e5ac146f0 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.651+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/867193740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abed5d0 0x561e5ac085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.651+0000 7fe49cc7d740 1 -- 172.21.15.103:0/867193740 >> 172.21.15.103:0/867193740 conn(0x561e5abb6310 msgr2=0x561e5abf4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:32.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.651+0000 7fe49cc7d740 1 -- 172.21.15.103:0/867193740 shutdown_connections 2024-09-19T07:04:32.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.651+0000 7fe49cc7d740 1 -- 172.21.15.103:0/867193740 wait complete. 2024-09-19T07:04:32.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.651+0000 7fe49cc7d740 1 -- start start 2024-09-19T07:04:32.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe49cc7d740 1 Processor -- start 2024-09-19T07:04:32.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe49cc7d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abed5d0 0x561e5ac04db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe49cc7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abeeb40 0x561e5abf1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe49cc7d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abf5d70 0x561e5abf24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe49cc7d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941acc30 con 0x561e5abed5d0 2024-09-19T07:04:32.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe49cc7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941ac560 con 0x561e5abeeb40 2024-09-19T07:04:32.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe49cc7d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941a97c0 con 0x561e5abf5d70 2024-09-19T07:04:32.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe49cc7d740 10 client.0 _ll_register_callbacks cb 0x7fe4840095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:32.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe4937fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abed5d0 0x561e5ac04db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe4937fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abed5d0 0x561e5ac04db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45792/0 (socket says 172.21.15.103:45792) 2024-09-19T07:04:32.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe4937fe640 1 -- 172.21.15.103:0/1111898259 learned_addr learned my addr 172.21.15.103:0/1111898259 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:32.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe4937fe640 1 --2- 172.21.15.103:0/1111898259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abf5d70 0x561e5abf24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe493fff640 1 --2- 172.21.15.103:0/1111898259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abeeb40 0x561e5abf1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe4937fe640 1 -- 172.21.15.103:0/1111898259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abf5d70 msgr2=0x561e5abf24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:32.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe4937fe640 1 --2- 172.21.15.103:0/1111898259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abf5d70 0x561e5abf24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe4937fe640 1 -- 172.21.15.103:0/1111898259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abeeb40 msgr2=0x561e5abf1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:32.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe4937fe640 1 --2- 172.21.15.103:0/1111898259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abeeb40 0x561e5abf1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe4937fe640 1 -- 172.21.15.103:0/1111898259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e5abf2be0 con 0x561e5abed5d0 2024-09-19T07:04:32.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe493fff640 1 --2- 172.21.15.103:0/1111898259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abeeb40 0x561e5abf1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:32.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe4937fe640 1 --2- 172.21.15.103:0/1111898259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abed5d0 0x561e5ac04db0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fe488006aa0 tx=0x7fe488006a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d76760822bf000 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:32.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.652+0000 7fe4737fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:32.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.653+0000 7fe4737fe640 1 -- 172.21.15.103:0/1111898259 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488018630 con 0x561e5abed5d0 2024-09-19T07:04:32.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.653+0000 7fe49cc7d740 1 -- 172.21.15.103:0/1111898259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe48401d400 con 0x561e5abed5d0 2024-09-19T07:04:32.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.653+0000 7fe4737fe640 1 -- 172.21.15.103:0/1111898259 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fe4880187d0 con 0x561e5abed5d0 2024-09-19T07:04:32.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.653+0000 7fe4737fe640 1 -- 172.21.15.103:0/1111898259 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4880189d0 con 0x561e5abed5d0 2024-09-19T07:04:32.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.653+0000 7fe49cc7d740 10 client.20658 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:32.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.653+0000 7fe49cc7d740 1 -- 172.21.15.103:0/1111898259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e5ac776c0 con 0x561e5abed5d0 2024-09-19T07:04:32.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.653+0000 7fe4737fe640 1 -- 172.21.15.103:0/1111898259 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe48401d400 con 0x561e5abed5d0 2024-09-19T07:04:32.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.653+0000 7fe4737fe640 1 -- 172.21.15.103:0/1111898259 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe488012050 con 0x561e5abed5d0 2024-09-19T07:04:32.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.653+0000 7fe49cc7d740 10 client.20658 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:32.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.653+0000 7fe49cc7d740 10 client.20658 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:04:32.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.653+0000 7fe49cc7d740 20 client.20658 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:32.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.653+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/1111898259 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x561e5ac77b40 0x561e5ac79f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:32.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.653+0000 7fe49cc7d740 4 client.20658 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:04:32.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.653+0000 7fe49cc7d740 1 -- 172.21.15.103:0/1111898259 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e5ab5ad70 con 0x561e5ac77b40 2024-09-19T07:04:32.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.653+0000 7fe493fff640 1 --2- 172.21.15.103:0/1111898259 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x561e5ac77b40 0x561e5ac79f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:32.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe493fff640 1 --2- 172.21.15.103:0/1111898259 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x561e5ac77b40 0x561e5ac79f30 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4e23323918e10639 server_cookie=37aac0d7780ac131 in_seq=0 out_seq=0 2024-09-19T07:04:32.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe4737fe640 10 client.20658 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:04:32.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe4737fe640 1 -- 172.21.15.103:0/1111898259 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e5ab5ad70 con 0x561e5ac77b40 2024-09-19T07:04:32.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe4737fe640 10 client.20658 handle_command_reply: tid=0 2024-09-19T07:04:32.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe49cc7d740 1 client.20658 shutdown 2024-09-19T07:04:32.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe49cc7d740 1 -- 172.21.15.103:0/1111898259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abed5d0 msgr2=0x561e5ac04db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:32.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/1111898259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abed5d0 0x561e5ac04db0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fe488006aa0 tx=0x7fe488006a30 comp rx=0 tx=0).stop 2024-09-19T07:04:32.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe49cc7d740 1 -- 172.21.15.103:0/1111898259 shutdown_connections 2024-09-19T07:04:32.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/1111898259 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x561e5ac77b40 0x561e5ac79f30 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/1111898259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5abf5d70 0x561e5abf24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/1111898259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e5abeeb40 0x561e5abf1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe49cc7d740 1 --2- 172.21.15.103:0/1111898259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e5abed5d0 0x561e5ac04db0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe49cc7d740 1 -- 172.21.15.103:0/1111898259 >> 172.21.15.103:0/1111898259 conn(0x561e5abb6310 msgr2=0x561e5abf6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:32.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe4737fe640 0 client.20658 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:04:32.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe49cc7d740 1 -- 172.21.15.103:0/1111898259 shutdown_connections 2024-09-19T07:04:32.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe49cc7d740 1 -- 172.21.15.103:0/1111898259 wait complete. 2024-09-19T07:04:32.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.654+0000 7fe49cc7d740 20 client.20658 trim_cache size 0 max 16384 2024-09-19T07:04:32.675 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:04:32.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.655+0000 7fe49a8a4640 1 -- 172.21.15.103:0/4098703253 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe47c0a3430 msgr2=0x7fe47c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:32.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.655+0000 7fe49a8a4640 1 --2- 172.21.15.103:0/4098703253 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe47c0a3430 0x7fe47c0a58f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0bd110 tx=0x7fe48c0b8f90 comp rx=0 tx=0).stop 2024-09-19T07:04:32.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.655+0000 7fe49a8a4640 1 -- 172.21.15.103:0/4098703253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe494106f10 msgr2=0x7fe49419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:32.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.655+0000 7fe49a8a4640 1 --2- 172.21.15.103:0/4098703253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe494106f10 0x7fe49419ce40 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fe48c031a60 tx=0x7fe48c031a90 comp rx=0 tx=0).stop 2024-09-19T07:04:32.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.655+0000 7fe49a8a4640 1 -- 172.21.15.103:0/4098703253 shutdown_connections 2024-09-19T07:04:32.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.655+0000 7fe49a8a4640 1 --2- 172.21.15.103:0/4098703253 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe47c0a3430 0x7fe47c0a58f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.655+0000 7fe49a8a4640 1 --2- 172.21.15.103:0/4098703253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe494106f10 0x7fe49419ce40 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.655+0000 7fe49a8a4640 1 --2- 172.21.15.103:0/4098703253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe494102790 0x7fe494198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.655+0000 7fe49a8a4640 1 --2- 172.21.15.103:0/4098703253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe494101de0 0x7fe494198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:32.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.655+0000 7fe49a8a4640 1 -- 172.21.15.103:0/4098703253 >> 172.21.15.103:0/4098703253 conn(0x7fe4940fd8c0 msgr2=0x7fe4940fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:32.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.655+0000 7fe49a8a4640 1 -- 172.21.15.103:0/4098703253 shutdown_connections 2024-09-19T07:04:32.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:32.655+0000 7fe49a8a4640 1 -- 172.21.15.103:0/4098703253 wait complete. 2024-09-19T07:04:33.368 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:04:33.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:33 smithi174 ceph-mon[35637]: pgmap v480: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-19T07:04:33.615 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:33.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:33 smithi103 ceph-mon[31158]: pgmap v480: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-19T07:04:33.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:33 smithi134 ceph-mon[35356]: pgmap v480: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-19T07:04:34.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.033+0000 7f0445153640 1 -- 172.21.15.103:0/2144568886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0440101de0 msgr2=0x7f04401021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:34.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.033+0000 7f0445153640 1 --2- 172.21.15.103:0/2144568886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0440101de0 0x7f04401021c0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f0430009900 tx=0x7f043002a410 comp rx=0 tx=0).stop 2024-09-19T07:04:34.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.033+0000 7f0445153640 1 -- 172.21.15.103:0/2144568886 shutdown_connections 2024-09-19T07:04:34.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.033+0000 7f0445153640 1 --2- 172.21.15.103:0/2144568886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0440106f10 0x7f044010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.033+0000 7f0445153640 1 --2- 172.21.15.103:0/2144568886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0440102790 0x7f04401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.033+0000 7f0445153640 1 --2- 172.21.15.103:0/2144568886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0440101de0 0x7f04401021c0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.033+0000 7f0445153640 1 -- 172.21.15.103:0/2144568886 >> 172.21.15.103:0/2144568886 conn(0x7f04400fd8c0 msgr2=0x7f04400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:34.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.033+0000 7f0445153640 1 -- 172.21.15.103:0/2144568886 shutdown_connections 2024-09-19T07:04:34.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.033+0000 7f0445153640 1 -- 172.21.15.103:0/2144568886 wait complete. 2024-09-19T07:04:34.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.034+0000 7f0445153640 1 Processor -- start 2024-09-19T07:04:34.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.034+0000 7f0445153640 1 -- start start 2024-09-19T07:04:34.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.035+0000 7f0445153640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0440101de0 0x7f0440198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.035+0000 7f0445153640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0440102790 0x7f04401988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.035+0000 7f0445153640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0440106f10 0x7f044019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.035+0000 7f0445153640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0440111f70 con 0x7f0440102790 2024-09-19T07:04:34.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.035+0000 7f0445153640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0440111df0 con 0x7f0440101de0 2024-09-19T07:04:34.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.035+0000 7f0445153640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04401120f0 con 0x7f0440106f10 2024-09-19T07:04:34.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.035+0000 7f043659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0440102790 0x7f04401988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:34.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.035+0000 7f043659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0440102790 0x7f04401988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45806/0 (socket says 172.21.15.103:45806) 2024-09-19T07:04:34.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.035+0000 7f043659c640 1 -- 172.21.15.103:0/1592130750 learned_addr learned my addr 172.21.15.103:0/1592130750 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:34.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.035+0000 7f043ed9d640 1 --2- 172.21.15.103:0/1592130750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0440101de0 0x7f0440198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:34.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.035+0000 7f043f59e640 1 --2- 172.21.15.103:0/1592130750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0440106f10 0x7f044019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:34.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.035+0000 7f043659c640 1 -- 172.21.15.103:0/1592130750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0440106f10 msgr2=0x7f044019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:34.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.035+0000 7f043659c640 1 --2- 172.21.15.103:0/1592130750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0440106f10 0x7f044019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.035+0000 7f043659c640 1 -- 172.21.15.103:0/1592130750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0440101de0 msgr2=0x7f0440198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:34.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.035+0000 7f043659c640 1 --2- 172.21.15.103:0/1592130750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0440101de0 0x7f0440198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.035+0000 7f043659c640 1 -- 172.21.15.103:0/1592130750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044019d320 con 0x7f0440102790 2024-09-19T07:04:34.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.036+0000 7f043659c640 1 --2- 172.21.15.103:0/1592130750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0440102790 0x7f04401988b0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f042800c920 tx=0x7f042800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:34.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.036+0000 7f043cd99640 1 -- 172.21.15.103:0/1592130750 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428007c70 con 0x7f0440102790 2024-09-19T07:04:34.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.036+0000 7f043cd99640 1 -- 172.21.15.103:0/1592130750 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0428007e10 con 0x7f0440102790 2024-09-19T07:04:34.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.036+0000 7f0445153640 1 -- 172.21.15.103:0/1592130750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044019d610 con 0x7f0440102790 2024-09-19T07:04:34.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.036+0000 7f043cd99640 1 -- 172.21.15.103:0/1592130750 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428018890 con 0x7f0440102790 2024-09-19T07:04:34.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.036+0000 7f0445153640 1 -- 172.21.15.103:0/1592130750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04401a4ef0 con 0x7f0440102790 2024-09-19T07:04:34.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.038+0000 7f043cd99640 1 -- 172.21.15.103:0/1592130750 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0428018a70 con 0x7f0440102790 2024-09-19T07:04:34.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.038+0000 7f044752c740 1 Processor -- start 2024-09-19T07:04:34.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.038+0000 7f044752c740 1 -- start start 2024-09-19T07:04:34.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f044752c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e36b40 0x561098e35090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f044752c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e355d0 0x561098e35a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f044752c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e40370 0x561098e42760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f044752c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561098d9b2f0 con 0x561098e355d0 2024-09-19T07:04:34.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f044752c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561098d42cf0 con 0x561098e36b40 2024-09-19T07:04:34.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f044752c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561098d972b0 con 0x561098e40370 2024-09-19T07:04:34.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f043ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e355d0 0x561098e35a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:34.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f043ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e355d0 0x561098e35a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45816/0 (socket says 172.21.15.103:45816) 2024-09-19T07:04:34.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f043ed9d640 1 -- 172.21.15.103:0/1011737461 learned_addr learned my addr 172.21.15.103:0/1011737461 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:34.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f043f59e640 1 --2- 172.21.15.103:0/1011737461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e36b40 0x561098e35090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:34.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f043659c640 1 --2- 172.21.15.103:0/1011737461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e40370 0x561098e42760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:34.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f043ed9d640 1 -- 172.21.15.103:0/1011737461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e40370 msgr2=0x561098e42760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:34.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f043ed9d640 1 --2- 172.21.15.103:0/1011737461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e40370 0x561098e42760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f043ed9d640 1 -- 172.21.15.103:0/1011737461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e36b40 msgr2=0x561098e35090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:34.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f043ed9d640 1 --2- 172.21.15.103:0/1011737461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e36b40 0x561098e35090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f043ed9d640 1 -- 172.21.15.103:0/1011737461 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561098d273a0 con 0x561098e355d0 2024-09-19T07:04:34.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f043cd99640 1 --2- 172.21.15.103:0/1592130750 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f042c0a33b0 0x7f042c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f043f59e640 1 --2- 172.21.15.103:0/1592130750 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f042c0a33b0 0x7f042c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:34.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f043ed9d640 1 --2- 172.21.15.103:0/1011737461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e355d0 0x561098e35a10 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f0430009900 tx=0x7f0430031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2eb8c0600c0f493 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:34.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f043cd99640 1 -- 172.21.15.103:0/1592130750 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f04280cfde0 con 0x7f0440102790 2024-09-19T07:04:34.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.039+0000 7f043559a640 1 -- 172.21.15.103:0/1011737461 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430040070 con 0x561098e355d0 2024-09-19T07:04:34.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.040+0000 7f043559a640 1 -- 172.21.15.103:0/1011737461 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0430005020 con 0x561098e355d0 2024-09-19T07:04:34.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.040+0000 7f043559a640 1 -- 172.21.15.103:0/1011737461 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430035040 con 0x561098e355d0 2024-09-19T07:04:34.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.040+0000 7f044752c740 1 -- 172.21.15.103:0/1011737461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e355d0 msgr2=0x561098e35a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:34.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.040+0000 7f043f59e640 1 --2- 172.21.15.103:0/1592130750 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f042c0a33b0 0x7f042c0a5870 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0438005fd0 tx=0x7f0438009600 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:34.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.040+0000 7f044752c740 1 --2- 172.21.15.103:0/1011737461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e355d0 0x561098e35a10 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f0430009900 tx=0x7f0430031630 comp rx=0 tx=0).stop 2024-09-19T07:04:34.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.040+0000 7f044752c740 1 -- 172.21.15.103:0/1011737461 shutdown_connections 2024-09-19T07:04:34.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.040+0000 7f044752c740 1 --2- 172.21.15.103:0/1011737461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e40370 0x561098e42760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.040+0000 7f044752c740 1 --2- 172.21.15.103:0/1011737461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e355d0 0x561098e35a10 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.040+0000 7f044752c740 1 --2- 172.21.15.103:0/1011737461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e36b40 0x561098e35090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.040+0000 7f044752c740 1 -- 172.21.15.103:0/1011737461 >> 172.21.15.103:0/1011737461 conn(0x561098dfe310 msgr2=0x561098e31c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:34.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.040+0000 7f044752c740 1 -- 172.21.15.103:0/1011737461 shutdown_connections 2024-09-19T07:04:34.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.040+0000 7f044752c740 1 -- 172.21.15.103:0/1011737461 wait complete. 2024-09-19T07:04:34.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.041+0000 7f044752c740 1 -- start start 2024-09-19T07:04:34.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.041+0000 7f044752c740 1 Processor -- start 2024-09-19T07:04:34.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.041+0000 7f044752c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e355d0 0x561098e3fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.041+0000 7f044752c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e36b40 0x561098e3cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.041+0000 7f044752c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e40370 0x561098e3d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.041+0000 7f044752c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561098e43010 con 0x561098e40370 2024-09-19T07:04:34.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.041+0000 7f044752c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561098d56060 con 0x561098e36b40 2024-09-19T07:04:34.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.041+0000 7f044752c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561098e43190 con 0x561098e355d0 2024-09-19T07:04:34.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.041+0000 7f044752c740 10 client.0 _ll_register_callbacks cb 0x561098aa98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:34.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.041+0000 7f043f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e40370 0x561098e3d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:34.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.041+0000 7f043f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e40370 0x561098e3d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45822/0 (socket says 172.21.15.103:45822) 2024-09-19T07:04:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.041+0000 7f043f59e640 1 -- 172.21.15.103:0/3617391582 learned_addr learned my addr 172.21.15.103:0/3617391582 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f043659c640 1 --2- 172.21.15.103:0/3617391582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e36b40 0x561098e3cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f043ed9d640 1 --2- 172.21.15.103:0/3617391582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e355d0 0x561098e3fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f043f59e640 1 -- 172.21.15.103:0/3617391582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e355d0 msgr2=0x561098e3fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f043f59e640 1 --2- 172.21.15.103:0/3617391582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e355d0 0x561098e3fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f043f59e640 1 -- 172.21.15.103:0/3617391582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e36b40 msgr2=0x561098e3cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f043f59e640 1 --2- 172.21.15.103:0/3617391582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e36b40 0x561098e3cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f043f59e640 1 -- 172.21.15.103:0/3617391582 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561098b0c3c0 con 0x561098e40370 2024-09-19T07:04:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f043659c640 1 --2- 172.21.15.103:0/3617391582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e36b40 0x561098e3cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f043f59e640 1 --2- 172.21.15.103:0/3617391582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e40370 0x561098e3d4d0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f0438011d40 tx=0x7f0438004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a582a091b0a14c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f04267fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f04267fc640 1 -- 172.21.15.103:0/3617391582 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438004d20 con 0x561098e40370 2024-09-19T07:04:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f04267fc640 1 -- 172.21.15.103:0/3617391582 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f0438005020 con 0x561098e40370 2024-09-19T07:04:34.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f044752c740 1 -- 172.21.15.103:0/3617391582 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0430005020 con 0x561098e40370 2024-09-19T07:04:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f04267fc640 1 -- 172.21.15.103:0/3617391582 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0438005260 con 0x561098e40370 2024-09-19T07:04:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f044752c740 10 client.20670 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f044752c740 1 -- 172.21.15.103:0/3617391582 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561098e4ea80 con 0x561098e40370 2024-09-19T07:04:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.042+0000 7f04267fc640 1 -- 172.21.15.103:0/3617391582 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0430005020 con 0x561098e40370 2024-09-19T07:04:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.043+0000 7f04267fc640 1 -- 172.21.15.103:0/3617391582 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f043800f040 con 0x561098e40370 2024-09-19T07:04:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.043+0000 7f044752c740 10 client.20670 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.043+0000 7f044752c740 10 client.20670 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:04:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.043+0000 7f044752c740 20 client.20670 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.043+0000 7f044752c740 1 --2- 172.21.15.103:0/3617391582 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561098e4ff40 0x561098e523b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.043+0000 7f044752c740 4 client.20670 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.043+0000 7f044752c740 1 -- 172.21.15.103:0/3617391582 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561098da8860 con 0x561098e4ff40 2024-09-19T07:04:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.043+0000 7f043ed9d640 1 --2- 172.21.15.103:0/3617391582 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561098e4ff40 0x561098e523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.043+0000 7f043ed9d640 1 --2- 172.21.15.103:0/3617391582 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561098e4ff40 0x561098e523b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=26fd6bef737d6d06 server_cookie=a415b8898bb869bf in_seq=0 out_seq=0 2024-09-19T07:04:34.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.043+0000 7f04267fc640 10 client.20670 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:04:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.044+0000 7f04267fc640 1 -- 172.21.15.103:0/3617391582 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561098da8860 con 0x561098e4ff40 2024-09-19T07:04:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.044+0000 7f04267fc640 10 client.20670 handle_command_reply: tid=0 2024-09-19T07:04:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.044+0000 7f044752c740 1 client.20670 shutdown 2024-09-19T07:04:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.044+0000 7f044752c740 1 -- 172.21.15.103:0/3617391582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e40370 msgr2=0x561098e3d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.044+0000 7f044752c740 1 --2- 172.21.15.103:0/3617391582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e40370 0x561098e3d4d0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f0438011d40 tx=0x7f0438004a40 comp rx=0 tx=0).stop 2024-09-19T07:04:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.044+0000 7f044752c740 1 -- 172.21.15.103:0/3617391582 shutdown_connections 2024-09-19T07:04:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.044+0000 7f044752c740 1 --2- 172.21.15.103:0/3617391582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e40370 0x561098e3d4d0 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.044+0000 7f044752c740 1 --2- 172.21.15.103:0/3617391582 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561098e4ff40 0x561098e523b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.044+0000 7f044752c740 1 --2- 172.21.15.103:0/3617391582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e36b40 0x561098e3cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.044+0000 7f04267fc640 0 client.20670 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:04:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.044+0000 7f044752c740 1 --2- 172.21.15.103:0/3617391582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e355d0 0x561098e3fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.044+0000 7f044752c740 1 -- 172.21.15.103:0/3617391582 >> 172.21.15.103:0/3617391582 conn(0x561098dfe310 msgr2=0x561098e37960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.044+0000 7f044752c740 1 -- 172.21.15.103:0/3617391582 shutdown_connections 2024-09-19T07:04:34.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.044+0000 7f044752c740 1 -- 172.21.15.103:0/3617391582 wait complete. 2024-09-19T07:04:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.044+0000 7f044752c740 20 client.20670 trim_cache size 0 max 16384 2024-09-19T07:04:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.057+0000 7f044752c740 1 Processor -- start 2024-09-19T07:04:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.057+0000 7f044752c740 1 -- start start 2024-09-19T07:04:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f044752c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e355d0 0x561098e31e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f044752c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e36b40 0x561098e4c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f044752c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e40370 0x561098e3c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f044752c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f04401ad860 con 0x561098e40370 2024-09-19T07:04:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f044752c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04401af3a0 con 0x561098e355d0 2024-09-19T07:04:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f044752c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04401adf30 con 0x561098e36b40 2024-09-19T07:04:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f043f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e40370 0x561098e3c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f043f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e40370 0x561098e3c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45838/0 (socket says 172.21.15.103:45838) 2024-09-19T07:04:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f043f59e640 1 -- 172.21.15.103:0/3541431292 learned_addr learned my addr 172.21.15.103:0/3541431292 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:34.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f043ed9d640 1 --2- 172.21.15.103:0/3541431292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e355d0 0x561098e31e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f043f59e640 1 -- 172.21.15.103:0/3541431292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e36b40 msgr2=0x561098e4c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f043f59e640 1 --2- 172.21.15.103:0/3541431292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e36b40 0x561098e4c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f043f59e640 1 -- 172.21.15.103:0/3541431292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e355d0 msgr2=0x561098e31e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f043f59e640 1 --2- 172.21.15.103:0/3541431292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e355d0 0x561098e31e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f043f59e640 1 -- 172.21.15.103:0/3541431292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561098e38b70 con 0x561098e40370 2024-09-19T07:04:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f043f59e640 1 --2- 172.21.15.103:0/3541431292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e40370 0x561098e3c830 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f0438013dd0 tx=0x7f04380119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bbd25082ebc0db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f0434d99640 1 -- 172.21.15.103:0/3541431292 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438030bb0 con 0x561098e40370 2024-09-19T07:04:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f0434d99640 1 -- 172.21.15.103:0/3541431292 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0438003e50 con 0x561098e40370 2024-09-19T07:04:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.058+0000 7f0434d99640 1 -- 172.21.15.103:0/3541431292 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438004ff0 con 0x561098e40370 2024-09-19T07:04:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.059+0000 7f044752c740 1 -- 172.21.15.103:0/3541431292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e40370 msgr2=0x561098e3c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.059+0000 7f044752c740 1 --2- 172.21.15.103:0/3541431292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e40370 0x561098e3c830 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f0438013dd0 tx=0x7f04380119b0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.059+0000 7f044752c740 1 -- 172.21.15.103:0/3541431292 shutdown_connections 2024-09-19T07:04:34.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.059+0000 7f044752c740 1 --2- 172.21.15.103:0/3541431292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e40370 0x561098e3c830 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.059+0000 7f044752c740 1 --2- 172.21.15.103:0/3541431292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e36b40 0x561098e4c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.059+0000 7f044752c740 1 --2- 172.21.15.103:0/3541431292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e355d0 0x561098e31e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.059+0000 7f044752c740 1 -- 172.21.15.103:0/3541431292 >> 172.21.15.103:0/3541431292 conn(0x561098dfe310 msgr2=0x561098e31700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.059+0000 7f044752c740 1 -- 172.21.15.103:0/3541431292 shutdown_connections 2024-09-19T07:04:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.059+0000 7f044752c740 1 -- 172.21.15.103:0/3541431292 wait complete. 2024-09-19T07:04:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.059+0000 7f044752c740 1 -- start start 2024-09-19T07:04:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f044752c740 1 Processor -- start 2024-09-19T07:04:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f044752c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e355d0 0x561098e3a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f044752c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e36b40 0x561098e3a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f044752c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e40370 0x561098e3ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f044752c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f04401aecd0 con 0x561098e355d0 2024-09-19T07:04:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f044752c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04401ae600 con 0x561098e36b40 2024-09-19T07:04:34.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f044752c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04401ab860 con 0x561098e40370 2024-09-19T07:04:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f044752c740 10 client.0 _ll_register_callbacks cb 0x7f0438009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f043ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e355d0 0x561098e3a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f043ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e355d0 0x561098e3a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45846/0 (socket says 172.21.15.103:45846) 2024-09-19T07:04:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f043ed9d640 1 -- 172.21.15.103:0/860736163 learned_addr learned my addr 172.21.15.103:0/860736163 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f043ed9d640 1 -- 172.21.15.103:0/860736163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e40370 msgr2=0x561098e3ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f043ed9d640 1 --2- 172.21.15.103:0/860736163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e40370 0x561098e3ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f043ed9d640 1 -- 172.21.15.103:0/860736163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e36b40 msgr2=0x561098e3a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f043ed9d640 1 --2- 172.21.15.103:0/860736163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e36b40 0x561098e3a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f043ed9d640 1 -- 172.21.15.103:0/860736163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561098eb9e00 con 0x561098e355d0 2024-09-19T07:04:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f043ed9d640 1 --2- 172.21.15.103:0/860736163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e355d0 0x561098e3a0b0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f0430009900 tx=0x7f0430031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6213a70368267cb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f04267fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f04267fc640 1 -- 172.21.15.103:0/860736163 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430054060 con 0x561098e355d0 2024-09-19T07:04:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f04267fc640 1 -- 172.21.15.103:0/860736163 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f0430004450 con 0x561098e355d0 2024-09-19T07:04:34.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.060+0000 7f044752c740 1 -- 172.21.15.103:0/860736163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0438003e50 con 0x561098e355d0 2024-09-19T07:04:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.061+0000 7f04267fc640 1 -- 172.21.15.103:0/860736163 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0430040040 con 0x561098e355d0 2024-09-19T07:04:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.061+0000 7f044752c740 10 client.20676 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.061+0000 7f044752c740 1 -- 172.21.15.103:0/860736163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561098ebb100 con 0x561098e355d0 2024-09-19T07:04:34.080 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:04:34.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.061+0000 7f04267fc640 1 -- 172.21.15.103:0/860736163 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0438003e50 con 0x561098e355d0 2024-09-19T07:04:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.061+0000 7f04267fc640 1 -- 172.21.15.103:0/860736163 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f043004f8c0 con 0x561098e355d0 2024-09-19T07:04:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.061+0000 7f044752c740 10 client.20676 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.061+0000 7f044752c740 10 client.20676 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:04:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.061+0000 7f044752c740 20 client.20676 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.061+0000 7f044752c740 1 --2- 172.21.15.103:0/860736163 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561098e4ff40 0x561098ebd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.061+0000 7f044752c740 4 client.20676 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:04:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.061+0000 7f044752c740 1 -- 172.21.15.103:0/860736163 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561098da2d70 con 0x561098e4ff40 2024-09-19T07:04:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.061+0000 7f043659c640 1 --2- 172.21.15.103:0/860736163 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561098e4ff40 0x561098ebd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f04267fc640 10 client.20676 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:04:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f043659c640 1 --2- 172.21.15.103:0/860736163 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561098e4ff40 0x561098ebd450 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e272d7403964295 server_cookie=678ac0fa6eddaa5d in_seq=0 out_seq=0 2024-09-19T07:04:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f04267fc640 1 -- 172.21.15.103:0/860736163 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561098da2d70 con 0x561098e4ff40 2024-09-19T07:04:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f04267fc640 10 client.20676 handle_command_reply: tid=0 2024-09-19T07:04:34.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f044752c740 1 client.20676 shutdown 2024-09-19T07:04:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f044752c740 1 -- 172.21.15.103:0/860736163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e355d0 msgr2=0x561098e3a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f044752c740 1 --2- 172.21.15.103:0/860736163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e355d0 0x561098e3a0b0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f0430009900 tx=0x7f0430031a50 comp rx=0 tx=0).stop 2024-09-19T07:04:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f044752c740 1 -- 172.21.15.103:0/860736163 shutdown_connections 2024-09-19T07:04:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f044752c740 1 --2- 172.21.15.103:0/860736163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561098e40370 0x561098e3ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f044752c740 1 --2- 172.21.15.103:0/860736163 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x561098e4ff40 0x561098ebd450 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f044752c740 1 --2- 172.21.15.103:0/860736163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561098e36b40 0x561098e3a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f044752c740 1 --2- 172.21.15.103:0/860736163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561098e355d0 0x561098e3a0b0 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f04267fc640 0 client.20676 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:04:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f044752c740 1 -- 172.21.15.103:0/860736163 >> 172.21.15.103:0/860736163 conn(0x561098dfe310 msgr2=0x561098e3e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f044752c740 1 -- 172.21.15.103:0/860736163 shutdown_connections 2024-09-19T07:04:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f044752c740 1 -- 172.21.15.103:0/860736163 wait complete. 2024-09-19T07:04:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.062+0000 7f044752c740 20 client.20676 trim_cache size 0 max 16384 2024-09-19T07:04:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.063+0000 7f0445153640 1 -- 172.21.15.103:0/1592130750 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f042c0a33b0 msgr2=0x7f042c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:34.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.063+0000 7f0445153640 1 --2- 172.21.15.103:0/1592130750 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f042c0a33b0 0x7f042c0a5870 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0438005fd0 tx=0x7f0438009600 comp rx=0 tx=0).stop 2024-09-19T07:04:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.063+0000 7f0445153640 1 -- 172.21.15.103:0/1592130750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0440102790 msgr2=0x7f04401988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.063+0000 7f0445153640 1 --2- 172.21.15.103:0/1592130750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0440102790 0x7f04401988b0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f042800c920 tx=0x7f042800cde0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.063+0000 7f0445153640 1 -- 172.21.15.103:0/1592130750 shutdown_connections 2024-09-19T07:04:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.063+0000 7f0445153640 1 --2- 172.21.15.103:0/1592130750 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f042c0a33b0 0x7f042c0a5870 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.063+0000 7f0445153640 1 --2- 172.21.15.103:0/1592130750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0440106f10 0x7f044019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.063+0000 7f0445153640 1 --2- 172.21.15.103:0/1592130750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0440102790 0x7f04401988b0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.063+0000 7f0445153640 1 --2- 172.21.15.103:0/1592130750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0440101de0 0x7f0440198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.063+0000 7f0445153640 1 -- 172.21.15.103:0/1592130750 >> 172.21.15.103:0/1592130750 conn(0x7f04400fd8c0 msgr2=0x7f04400fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.063+0000 7f0445153640 1 -- 172.21.15.103:0/1592130750 shutdown_connections 2024-09-19T07:04:34.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:34.063+0000 7f0445153640 1 -- 172.21.15.103:0/1592130750 wait complete. 2024-09-19T07:04:34.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:34 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:04:34.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:34 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:04:34.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:34 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:04:34.729 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:04:34.987 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:35.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:35 smithi103 ceph-mon[31158]: pgmap v481: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-19T07:04:35.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:35 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:04:35.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:35 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:04:35.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:35 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:04:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.401+0000 7f1c88847640 1 -- 172.21.15.103:0/1227586069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c84102ef0 msgr2=0x7f1c8410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.401+0000 7f1c88847640 1 --2- 172.21.15.103:0/1227586069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c84102ef0 0x7f1c8410da40 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009ca0 tx=0x7f1c7c02a950 comp rx=0 tx=0).stop 2024-09-19T07:04:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.401+0000 7f1c88847640 1 -- 172.21.15.103:0/1227586069 shutdown_connections 2024-09-19T07:04:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.401+0000 7f1c88847640 1 --2- 172.21.15.103:0/1227586069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c84102ef0 0x7f1c8410da40 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.401+0000 7f1c88847640 1 --2- 172.21.15.103:0/1227586069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c84102570 0x7f1c841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.401+0000 7f1c88847640 1 --2- 172.21.15.103:0/1227586069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c84100e20 0x7f1c84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.401+0000 7f1c88847640 1 -- 172.21.15.103:0/1227586069 >> 172.21.15.103:0/1227586069 conn(0x7f1c840fc7d0 msgr2=0x7f1c840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.401+0000 7f1c88847640 1 -- 172.21.15.103:0/1227586069 shutdown_connections 2024-09-19T07:04:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.401+0000 7f1c88847640 1 -- 172.21.15.103:0/1227586069 wait complete. 2024-09-19T07:04:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.402+0000 7f1c88847640 1 Processor -- start 2024-09-19T07:04:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.402+0000 7f1c88847640 1 -- start start 2024-09-19T07:04:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.402+0000 7f1c88847640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c84100e20 0x7f1c84197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.402+0000 7f1c88847640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c84102570 0x7f1c84197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c88847640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c84102ef0 0x7f1c8419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c88847640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c84110f80 con 0x7f1c84102570 2024-09-19T07:04:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c88847640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1c84110e00 con 0x7f1c84102ef0 2024-09-19T07:04:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c88847640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c84111100 con 0x7f1c84100e20 2024-09-19T07:04:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c79d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c84102570 0x7f1c84197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c79d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c84102570 0x7f1c84197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45866/0 (socket says 172.21.15.103:45866) 2024-09-19T07:04:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c79d9b640 1 -- 172.21.15.103:0/4031300869 learned_addr learned my addr 172.21.15.103:0/4031300869 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c82d9d640 1 --2- 172.21.15.103:0/4031300869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c84102ef0 0x7f1c8419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c8259c640 1 --2- 172.21.15.103:0/4031300869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c84100e20 0x7f1c84197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c79d9b640 1 -- 172.21.15.103:0/4031300869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c84100e20 msgr2=0x7f1c84197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c79d9b640 1 --2- 172.21.15.103:0/4031300869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c84100e20 0x7f1c84197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c79d9b640 1 -- 172.21.15.103:0/4031300869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c84102ef0 msgr2=0x7f1c8419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c79d9b640 1 --2- 172.21.15.103:0/4031300869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c84102ef0 0x7f1c8419bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c79d9b640 1 -- 172.21.15.103:0/4031300869 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8419c2b0 con 0x7f1c84102570 2024-09-19T07:04:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c82d9d640 1 --2- 172.21.15.103:0/4031300869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c84102ef0 0x7f1c8419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c79d9b640 1 --2- 172.21.15.103:0/4031300869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c84102570 0x7f1c84197840 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00c890 tx=0x7f1c6c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c7bfff640 1 -- 172.21.15.103:0/4031300869 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c004220 con 0x7f1c84102570 2024-09-19T07:04:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c88847640 1 -- 172.21.15.103:0/4031300869 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c8419c5a0 con 0x7f1c84102570 2024-09-19T07:04:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c7bfff640 1 -- 172.21.15.103:0/4031300869 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c6c0043c0 con 0x7f1c84102570 2024-09-19T07:04:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c7bfff640 1 -- 172.21.15.103:0/4031300869 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c004a00 con 0x7f1c84102570 2024-09-19T07:04:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.403+0000 7f1c88847640 1 -- 172.21.15.103:0/4031300869 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c841a3e80 con 0x7f1c84102570 2024-09-19T07:04:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.406+0000 7f1c7bfff640 1 -- 172.21.15.103:0/4031300869 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f1c6c0079b0 con 0x7f1c84102570 2024-09-19T07:04:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.406+0000 7f1c8ac20740 1 Processor -- start 2024-09-19T07:04:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.406+0000 7f1c8ac20740 1 -- start start 2024-09-19T07:04:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.406+0000 7f1c8ac20740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 0x55a49d27a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.406+0000 7f1c8ac20740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27a5d0 0x55a49d27aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.406+0000 7f1c8ac20740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d287580 0x55a49d289970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.406+0000 7f1c8ac20740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a49d1e02f0 con 0x55a49d27a5d0 2024-09-19T07:04:35.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.406+0000 7f1c8ac20740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a49d187cf0 con 0x55a49d287580 2024-09-19T07:04:35.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.406+0000 7f1c8ac20740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a49d1dc2b0 con 0x55a49d27bb40 2024-09-19T07:04:35.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c8259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27a5d0 0x55a49d27aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c8259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27a5d0 0x55a49d27aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45872/0 (socket says 172.21.15.103:45872) 2024-09-19T07:04:35.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c8259c640 1 -- 172.21.15.103:0/3754627851 learned_addr learned my addr 172.21.15.103:0/3754627851 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:35.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c82d9d640 1 --2- 172.21.15.103:0/3754627851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 0x55a49d27a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c79d9b640 1 --2- 172.21.15.103:0/3754627851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d287580 0x55a49d289970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c7bfff640 1 --2- 172.21.15.103:0/4031300869 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1c700a33b0 0x7f1c700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c8259c640 1 -- 172.21.15.103:0/3754627851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 msgr2=0x55a49d27a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c8259c640 1 --2- 172.21.15.103:0/3754627851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 0x55a49d27a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c8259c640 1 -- 172.21.15.103:0/3754627851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d287580 msgr2=0x55a49d289970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c8259c640 1 --2- 172.21.15.103:0/3754627851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d287580 0x55a49d289970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c8259c640 1 -- 172.21.15.103:0/3754627851 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a49d16c3a0 con 0x55a49d27a5d0 2024-09-19T07:04:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c82d9d640 1 --2- 172.21.15.103:0/3754627851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 0x55a49d27a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c82d9d640 1 --2- 172.21.15.103:0/4031300869 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1c700a33b0 0x7f1c700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c7bfff640 1 -- 172.21.15.103:0/4031300869 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f1c6c0cfaf0 con 0x7f1c84102570 2024-09-19T07:04:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c8259c640 1 --2- 172.21.15.103:0/3754627851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27a5d0 0x55a49d27aa10 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400cc60 tx=0x7f1c7400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d682d9c2f7c45a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c78d99640 1 -- 172.21.15.103:0/3754627851 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74014070 con 0x55a49d27a5d0 2024-09-19T07:04:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c78d99640 1 -- 172.21.15.103:0/3754627851 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c740047a0 con 0x55a49d27a5d0 2024-09-19T07:04:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.407+0000 7f1c78d99640 1 -- 172.21.15.103:0/3754627851 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74010750 con 0x55a49d27a5d0 2024-09-19T07:04:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.408+0000 7f1c8ac20740 1 -- 172.21.15.103:0/3754627851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27a5d0 msgr2=0x55a49d27aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.408+0000 7f1c82d9d640 1 --2- 172.21.15.103:0/4031300869 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1c700a33b0 0x7f1c700a5870 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c030040 tx=0x7f1c7c03e040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.408+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/3754627851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27a5d0 0x55a49d27aa10 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400cc60 tx=0x7f1c7400e720 comp rx=0 tx=0).stop 2024-09-19T07:04:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.408+0000 7f1c8ac20740 1 -- 172.21.15.103:0/3754627851 shutdown_connections 2024-09-19T07:04:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.408+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/3754627851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d287580 0x55a49d289970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.408+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/3754627851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27a5d0 0x55a49d27aa10 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.408+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/3754627851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 0x55a49d27a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.408+0000 7f1c8ac20740 1 -- 172.21.15.103:0/3754627851 >> 172.21.15.103:0/3754627851 conn(0x55a49d243310 msgr2=0x55a49d276c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.408+0000 7f1c8ac20740 1 -- 172.21.15.103:0/3754627851 shutdown_connections 2024-09-19T07:04:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.408+0000 7f1c8ac20740 1 -- 172.21.15.103:0/3754627851 wait complete. 2024-09-19T07:04:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c8ac20740 1 -- start start 2024-09-19T07:04:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c8ac20740 1 Processor -- start 2024-09-19T07:04:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c8ac20740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d27a5d0 0x55a49d282680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c8ac20740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 0x55a49d282bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c8ac20740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d287580 0x55a49d283100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c8ac20740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a49d27d550 con 0x55a49d287580 2024-09-19T07:04:35.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c8ac20740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a49d19b060 con 0x55a49d27a5d0 2024-09-19T07:04:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c8ac20740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a49d27d6d0 con 0x55a49d27bb40 2024-09-19T07:04:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c8ac20740 10 client.0 _ll_register_callbacks cb 0x55a49ceee8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c82d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d287580 0x55a49d283100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c82d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d287580 0x55a49d283100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45886/0 (socket says 172.21.15.103:45886) 2024-09-19T07:04:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c82d9d640 1 -- 172.21.15.103:0/1692352311 learned_addr learned my addr 172.21.15.103:0/1692352311 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c79d9b640 1 --2- 172.21.15.103:0/1692352311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 0x55a49d282bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c8259c640 1 --2- 172.21.15.103:0/1692352311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d27a5d0 0x55a49d282680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c82d9d640 1 -- 172.21.15.103:0/1692352311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 msgr2=0x55a49d282bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c82d9d640 1 --2- 172.21.15.103:0/1692352311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 0x55a49d282bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c82d9d640 1 -- 172.21.15.103:0/1692352311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d27a5d0 msgr2=0x55a49d282680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c82d9d640 1 --2- 172.21.15.103:0/1692352311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d27a5d0 0x55a49d282680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.409+0000 7f1c82d9d640 1 -- 172.21.15.103:0/1692352311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a49cf513c0 con 0x55a49d287580 2024-09-19T07:04:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.410+0000 7f1c8259c640 1 --2- 172.21.15.103:0/1692352311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d27a5d0 0x55a49d282680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.410+0000 7f1c79d9b640 1 --2- 172.21.15.103:0/1692352311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 0x55a49d282bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.410+0000 7f1c82d9d640 1 --2- 172.21.15.103:0/1692352311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d287580 0x55a49d283100 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c02f620 tx=0x7f1c7c02fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6677ddb8f546bd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.410+0000 7f1c69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.410+0000 7f1c69ffb640 1 -- 172.21.15.103:0/1692352311 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c034040 con 0x55a49d287580 2024-09-19T07:04:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.410+0000 7f1c69ffb640 1 -- 172.21.15.103:0/1692352311 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f1c7c03cbd0 con 0x55a49d287580 2024-09-19T07:04:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.410+0000 7f1c8ac20740 1 -- 172.21.15.103:0/1692352311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c740047a0 con 0x55a49d287580 2024-09-19T07:04:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.410+0000 7f1c69ffb640 1 -- 172.21.15.103:0/1692352311 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c7c02f710 con 0x55a49d287580 2024-09-19T07:04:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.410+0000 7f1c8ac20740 10 client.20688 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.410+0000 7f1c8ac20740 1 -- 172.21.15.103:0/1692352311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a49d293b00 con 0x55a49d287580 2024-09-19T07:04:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.410+0000 7f1c69ffb640 1 -- 172.21.15.103:0/1692352311 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c740047a0 con 0x55a49d287580 2024-09-19T07:04:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.410+0000 7f1c69ffb640 1 -- 172.21.15.103:0/1692352311 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1c7c05c5a0 con 0x55a49d287580 2024-09-19T07:04:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.410+0000 7f1c8ac20740 10 client.20688 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.410+0000 7f1c8ac20740 10 client.20688 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:04:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.410+0000 7f1c8ac20740 20 client.20688 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.411+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/1692352311 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55a49d294f10 0x55a49d2973b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.411+0000 7f1c8ac20740 4 client.20688 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.411+0000 7f1c8ac20740 1 -- 172.21.15.103:0/1692352311 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a49d1ed860 con 0x55a49d294f10 2024-09-19T07:04:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.411+0000 7f1c8259c640 1 --2- 172.21.15.103:0/1692352311 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55a49d294f10 0x55a49d2973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.411+0000 7f1c8259c640 1 --2- 172.21.15.103:0/1692352311 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55a49d294f10 0x55a49d2973b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48c13a5ee8bacdb0 server_cookie=3fd60734798eb774 in_seq=0 out_seq=0 2024-09-19T07:04:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.411+0000 7f1c69ffb640 10 client.20688 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:04:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.412+0000 7f1c69ffb640 1 -- 172.21.15.103:0/1692352311 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a49d1ed860 con 0x55a49d294f10 2024-09-19T07:04:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.412+0000 7f1c69ffb640 10 client.20688 handle_command_reply: tid=0 2024-09-19T07:04:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.412+0000 7f1c8ac20740 1 client.20688 shutdown 2024-09-19T07:04:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.412+0000 7f1c8ac20740 1 -- 172.21.15.103:0/1692352311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d287580 msgr2=0x55a49d283100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.412+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/1692352311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d287580 0x55a49d283100 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c02f620 tx=0x7f1c7c02fea0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.412+0000 7f1c8ac20740 1 -- 172.21.15.103:0/1692352311 shutdown_connections 2024-09-19T07:04:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.412+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/1692352311 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55a49d294f10 0x55a49d2973b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.412+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/1692352311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d287580 0x55a49d283100 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.412+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/1692352311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 0x55a49d282bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.412+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/1692352311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d27a5d0 0x55a49d282680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.412+0000 7f1c69ffb640 0 client.20688 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:04:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.412+0000 7f1c8ac20740 1 -- 172.21.15.103:0/1692352311 >> 172.21.15.103:0/1692352311 conn(0x55a49d243310 msgr2=0x55a49d27c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.412+0000 7f1c8ac20740 1 -- 172.21.15.103:0/1692352311 shutdown_connections 2024-09-19T07:04:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.412+0000 7f1c8ac20740 1 -- 172.21.15.103:0/1692352311 wait complete. 2024-09-19T07:04:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.412+0000 7f1c8ac20740 20 client.20688 trim_cache size 0 max 16384 2024-09-19T07:04:35.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.425+0000 7f1c8ac20740 1 Processor -- start 2024-09-19T07:04:35.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.425+0000 7f1c8ac20740 1 -- start start 2024-09-19T07:04:35.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.425+0000 7f1c8ac20740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27a5d0 0x55a49d282d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.425+0000 7f1c8ac20740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 0x55a49d29cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.425+0000 7f1c8ac20740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d287580 0x55a49d29d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.425+0000 7f1c8ac20740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841ab5f0 con 0x55a49d27a5d0 2024-09-19T07:04:35.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.425+0000 7f1c8ac20740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841ad130 con 0x55a49d287580 2024-09-19T07:04:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.425+0000 7f1c8ac20740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841abcc0 con 0x55a49d27bb40 2024-09-19T07:04:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.425+0000 7f1c8259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27a5d0 0x55a49d282d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.425+0000 7f1c8259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27a5d0 0x55a49d282d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45888/0 (socket says 172.21.15.103:45888) 2024-09-19T07:04:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.425+0000 7f1c8259c640 1 -- 172.21.15.103:0/1475390291 learned_addr learned my addr 172.21.15.103:0/1475390291 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.426+0000 7f1c79d9b640 1 --2- 172.21.15.103:0/1475390291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 0x55a49d29cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.426+0000 7f1c8259c640 1 -- 172.21.15.103:0/1475390291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 msgr2=0x55a49d29cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.426+0000 7f1c8259c640 1 --2- 172.21.15.103:0/1475390291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 0x55a49d29cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.426+0000 7f1c8259c640 1 -- 172.21.15.103:0/1475390291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d287580 msgr2=0x55a49d29d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.426+0000 7f1c8259c640 1 --2- 172.21.15.103:0/1475390291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d287580 0x55a49d29d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.426+0000 7f1c8259c640 1 -- 172.21.15.103:0/1475390291 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a49d281650 con 0x55a49d27a5d0 2024-09-19T07:04:35.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.426+0000 7f1c8259c640 1 --2- 172.21.15.103:0/1475390291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27a5d0 0x55a49d282d50 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400cb80 tx=0x7f1c74004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece05fefc9f70ed7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:35.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.426+0000 7f1c6bfff640 1 -- 172.21.15.103:0/1475390291 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7402b070 con 0x55a49d27a5d0 2024-09-19T07:04:35.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.426+0000 7f1c6bfff640 1 -- 172.21.15.103:0/1475390291 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c74024b50 con 0x55a49d27a5d0 2024-09-19T07:04:35.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.426+0000 7f1c6bfff640 1 -- 172.21.15.103:0/1475390291 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74014040 con 0x55a49d27a5d0 2024-09-19T07:04:35.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.426+0000 7f1c8ac20740 1 -- 172.21.15.103:0/1475390291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27a5d0 msgr2=0x55a49d282d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:35.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.426+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/1475390291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27a5d0 0x55a49d282d50 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400cb80 tx=0x7f1c74004790 comp rx=0 tx=0).stop 2024-09-19T07:04:35.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8ac20740 1 -- 172.21.15.103:0/1475390291 shutdown_connections 2024-09-19T07:04:35.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/1475390291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d287580 0x55a49d29d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/1475390291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d27bb40 0x55a49d29cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/1475390291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27a5d0 0x55a49d282d50 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8ac20740 1 -- 172.21.15.103:0/1475390291 >> 172.21.15.103:0/1475390291 conn(0x55a49d243310 msgr2=0x55a49d282640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:35.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8ac20740 1 -- 172.21.15.103:0/1475390291 shutdown_connections 2024-09-19T07:04:35.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8ac20740 1 -- 172.21.15.103:0/1475390291 wait complete. 2024-09-19T07:04:35.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8ac20740 1 -- start start 2024-09-19T07:04:35.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8ac20740 1 Processor -- start 2024-09-19T07:04:35.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8ac20740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d27a5d0 0x55a49d28c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8ac20740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27bb40 0x55a49d28cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8ac20740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d287580 0x55a49d2f8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8ac20740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841aca60 con 0x55a49d27bb40 2024-09-19T07:04:35.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8ac20740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841ac390 con 0x55a49d27a5d0 2024-09-19T07:04:35.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8ac20740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841a95f0 con 0x55a49d287580 2024-09-19T07:04:35.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8ac20740 10 client.0 _ll_register_callbacks cb 0x7f1c7c02fc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:35.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c79d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27bb40 0x55a49d28cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c79d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27bb40 0x55a49d28cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45896/0 (socket says 172.21.15.103:45896) 2024-09-19T07:04:35.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c79d9b640 1 -- 172.21.15.103:0/3614790921 learned_addr learned my addr 172.21.15.103:0/3614790921 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:35.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c8259c640 1 --2- 172.21.15.103:0/3614790921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d27a5d0 0x55a49d28c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.427+0000 7f1c82d9d640 1 --2- 172.21.15.103:0/3614790921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d287580 0x55a49d2f8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.428+0000 7f1c79d9b640 1 -- 172.21.15.103:0/3614790921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d287580 msgr2=0x55a49d2f8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:35.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.428+0000 7f1c79d9b640 1 --2- 172.21.15.103:0/3614790921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d287580 0x55a49d2f8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.428+0000 7f1c79d9b640 1 -- 172.21.15.103:0/3614790921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d27a5d0 msgr2=0x55a49d28c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:35.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.428+0000 7f1c79d9b640 1 --2- 172.21.15.103:0/3614790921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d27a5d0 0x55a49d28c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.428+0000 7f1c79d9b640 1 -- 172.21.15.103:0/3614790921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a49d2f9280 con 0x55a49d27bb40 2024-09-19T07:04:35.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.428+0000 7f1c8259c640 1 --2- 172.21.15.103:0/3614790921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d27a5d0 0x55a49d28c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:35.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.428+0000 7f1c79d9b640 1 --2- 172.21.15.103:0/3614790921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27bb40 0x55a49d28cd50 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c0cd820 tx=0x7f1c6c00a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115ccd1381c49599 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.428+0000 7f1c69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.428+0000 7f1c69ffb640 1 -- 172.21.15.103:0/3614790921 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c01d040 con 0x55a49d27bb40 2024-09-19T07:04:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.428+0000 7f1c69ffb640 1 -- 172.21.15.103:0/3614790921 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f1c6c007ca0 con 0x55a49d27bb40 2024-09-19T07:04:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.428+0000 7f1c8ac20740 1 -- 172.21.15.103:0/3614790921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c74024b50 con 0x55a49d27bb40 2024-09-19T07:04:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.428+0000 7f1c69ffb640 1 -- 172.21.15.103:0/3614790921 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c6c09c4b0 con 0x55a49d27bb40 2024-09-19T07:04:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.428+0000 7f1c8ac20740 10 client.20694 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.428+0000 7f1c8ac20740 1 -- 172.21.15.103:0/3614790921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a49d3005e0 con 0x55a49d27bb40 2024-09-19T07:04:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.428+0000 7f1c69ffb640 1 -- 172.21.15.103:0/3614790921 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c74024b50 con 0x55a49d27bb40 2024-09-19T07:04:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.429+0000 7f1c69ffb640 1 -- 172.21.15.103:0/3614790921 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1c6c07f170 con 0x55a49d27bb40 2024-09-19T07:04:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.429+0000 7f1c8ac20740 10 client.20694 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.429+0000 7f1c8ac20740 10 client.20694 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:04:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.429+0000 7f1c8ac20740 20 client.20694 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.429+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/3614790921 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55a49d300d10 0x55a49d303130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.429+0000 7f1c8ac20740 4 client.20694 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:04:35.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.429+0000 7f1c8ac20740 1 -- 172.21.15.103:0/3614790921 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a49d1e7d70 con 0x55a49d300d10 2024-09-19T07:04:35.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.429+0000 7f1c8259c640 1 --2- 172.21.15.103:0/3614790921 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55a49d300d10 0x55a49d303130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:35.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.429+0000 7f1c8259c640 1 --2- 172.21.15.103:0/3614790921 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55a49d300d10 0x55a49d303130 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9cb2e2d2d5c9f645 server_cookie=6316e6f05ccd3f42 in_seq=0 out_seq=0 2024-09-19T07:04:35.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.429+0000 7f1c69ffb640 10 client.20694 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:04:35.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.429+0000 7f1c69ffb640 1 -- 172.21.15.103:0/3614790921 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a49d1e7d70 con 0x55a49d300d10 2024-09-19T07:04:35.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.429+0000 7f1c69ffb640 10 client.20694 handle_command_reply: tid=0 2024-09-19T07:04:35.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.429+0000 7f1c8ac20740 1 client.20694 shutdown 2024-09-19T07:04:35.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c8ac20740 1 -- 172.21.15.103:0/3614790921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27bb40 msgr2=0x55a49d28cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:35.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/3614790921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27bb40 0x55a49d28cd50 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c0cd820 tx=0x7f1c6c00a5e0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c8ac20740 1 -- 172.21.15.103:0/3614790921 shutdown_connections 2024-09-19T07:04:35.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/3614790921 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55a49d300d10 0x55a49d303130 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/3614790921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a49d287580 0x55a49d2f8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c69ffb640 0 client.20694 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:04:35.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/3614790921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a49d27bb40 0x55a49d28cd50 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c8ac20740 1 --2- 172.21.15.103:0/3614790921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a49d27a5d0 0x55a49d28c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c8ac20740 1 -- 172.21.15.103:0/3614790921 >> 172.21.15.103:0/3614790921 conn(0x55a49d243310 msgr2=0x55a49d286710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:35.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c8ac20740 1 -- 172.21.15.103:0/3614790921 shutdown_connections 2024-09-19T07:04:35.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c8ac20740 1 -- 172.21.15.103:0/3614790921 wait complete. 2024-09-19T07:04:35.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c8ac20740 20 client.20694 trim_cache size 0 max 16384 2024-09-19T07:04:35.443 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:04:35.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c88847640 1 -- 172.21.15.103:0/4031300869 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1c700a33b0 msgr2=0x7f1c700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:35.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c88847640 1 --2- 172.21.15.103:0/4031300869 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1c700a33b0 0x7f1c700a5870 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c030040 tx=0x7f1c7c03e040 comp rx=0 tx=0).stop 2024-09-19T07:04:35.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c88847640 1 -- 172.21.15.103:0/4031300869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c84102570 msgr2=0x7f1c84197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:35.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c88847640 1 --2- 172.21.15.103:0/4031300869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c84102570 0x7f1c84197840 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00c890 tx=0x7f1c6c00cd50 comp rx=0 tx=0).stop 2024-09-19T07:04:35.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c88847640 1 -- 172.21.15.103:0/4031300869 shutdown_connections 2024-09-19T07:04:35.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c88847640 1 --2- 172.21.15.103:0/4031300869 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f1c700a33b0 0x7f1c700a5870 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c88847640 1 --2- 172.21.15.103:0/4031300869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c84102ef0 0x7f1c8419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c88847640 1 --2- 172.21.15.103:0/4031300869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c84102570 0x7f1c84197840 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c88847640 1 --2- 172.21.15.103:0/4031300869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c84100e20 0x7f1c84197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:35.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.430+0000 7f1c88847640 1 -- 172.21.15.103:0/4031300869 >> 172.21.15.103:0/4031300869 conn(0x7f1c840fc7d0 msgr2=0x7f1c841067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:35.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.431+0000 7f1c88847640 1 -- 172.21.15.103:0/4031300869 shutdown_connections 2024-09-19T07:04:35.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:35.431+0000 7f1c88847640 1 -- 172.21.15.103:0/4031300869 wait complete. 2024-09-19T07:04:35.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:35 smithi174 ceph-mon[35637]: pgmap v481: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-19T07:04:35.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:35 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:04:35.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:35 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:04:35.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:35 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:04:35.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:35 smithi134 ceph-mon[35356]: pgmap v481: 97 pgs: 97 active+clean; 2.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-19T07:04:35.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:35 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:04:35.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:35 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:04:35.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:35 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:04:36.212 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:04:36.460 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:36.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.887+0000 7f85499c2640 1 -- 172.21.15.103:0/460618982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8544106f10 msgr2=0x7f854410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:36.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.887+0000 7f85499c2640 1 --2- 172.21.15.103:0/460618982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8544106f10 0x7f854410ea90 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f853c009ca0 tx=0x7f853c02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.887+0000 7f85499c2640 1 -- 172.21.15.103:0/460618982 shutdown_connections 2024-09-19T07:04:36.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.887+0000 7f85499c2640 1 --2- 172.21.15.103:0/460618982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8544106f10 0x7f854410ea90 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.887+0000 7f85499c2640 1 --2- 172.21.15.103:0/460618982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8544102790 0x7f85441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.887+0000 7f85499c2640 1 --2- 172.21.15.103:0/460618982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8544101de0 0x7f85441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.887+0000 7f85499c2640 1 -- 172.21.15.103:0/460618982 >> 172.21.15.103:0/460618982 conn(0x7f85440fd8c0 msgr2=0x7f85440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.887+0000 7f85499c2640 1 -- 172.21.15.103:0/460618982 shutdown_connections 2024-09-19T07:04:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.887+0000 7f85499c2640 1 -- 172.21.15.103:0/460618982 wait complete. 2024-09-19T07:04:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.888+0000 7f85499c2640 1 Processor -- start 2024-09-19T07:04:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.888+0000 7f85499c2640 1 -- start start 2024-09-19T07:04:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.888+0000 7f85499c2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8544101de0 0x7f8544198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.888+0000 7f85499c2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8544102790 0x7f8544198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.888+0000 7f85499c2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8544106f10 0x7f854419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.888+0000 7f85499c2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8544111f90 con 0x7f8544106f10 2024-09-19T07:04:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.888+0000 7f85499c2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8544111e10 con 0x7f8544101de0 2024-09-19T07:04:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.888+0000 7f85499c2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8544112110 con 0x7f8544102790 2024-09-19T07:04:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.888+0000 7f85437fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8544106f10 0x7f854419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.889+0000 7f85437fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8544106f10 0x7f854419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45912/0 (socket says 172.21.15.103:45912) 2024-09-19T07:04:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.889+0000 7f85437fe640 1 -- 172.21.15.103:0/3275409929 learned_addr learned my addr 172.21.15.103:0/3275409929 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.889+0000 7f8542ffd640 1 --2- 172.21.15.103:0/3275409929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8544101de0 0x7f8544198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.889+0000 7f85427fc640 1 --2- 172.21.15.103:0/3275409929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8544102790 0x7f8544198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.889+0000 7f85437fe640 1 -- 172.21.15.103:0/3275409929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8544102790 msgr2=0x7f8544198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.889+0000 7f85437fe640 1 --2- 172.21.15.103:0/3275409929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8544102790 0x7f8544198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.889+0000 7f85437fe640 1 -- 172.21.15.103:0/3275409929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8544101de0 msgr2=0x7f8544198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.889+0000 7f85437fe640 1 --2- 172.21.15.103:0/3275409929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8544101de0 0x7f8544198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.889+0000 7f85437fe640 1 -- 172.21.15.103:0/3275409929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854419d400 con 0x7f8544106f10 2024-09-19T07:04:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.889+0000 7f85437fe640 1 --2- 172.21.15.103:0/3275409929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8544106f10 0x7f854419cd20 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f853c031a20 tx=0x7f853c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.889+0000 7f85489c0640 1 -- 172.21.15.103:0/3275409929 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c003b70 con 0x7f8544106f10 2024-09-19T07:04:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.889+0000 7f85499c2640 1 -- 172.21.15.103:0/3275409929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854419d690 con 0x7f8544106f10 2024-09-19T07:04:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.889+0000 7f85489c0640 1 -- 172.21.15.103:0/3275409929 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f853c003d10 con 0x7f8544106f10 2024-09-19T07:04:36.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.889+0000 7f85489c0640 1 -- 172.21.15.103:0/3275409929 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c02fc80 con 0x7f8544106f10 2024-09-19T07:04:36.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.889+0000 7f85499c2640 1 -- 172.21.15.103:0/3275409929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85441a4fd0 con 0x7f8544106f10 2024-09-19T07:04:36.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.892+0000 7f85489c0640 1 -- 172.21.15.103:0/3275409929 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f853c0046a0 con 0x7f8544106f10 2024-09-19T07:04:36.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.892+0000 7f854bd9b740 1 Processor -- start 2024-09-19T07:04:36.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.892+0000 7f854bd9b740 1 -- start start 2024-09-19T07:04:36.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f854bd9b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950df9ab40 0x55950df99090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f854bd9b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df995d0 0x55950df99a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f854bd9b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950dfa4370 0x55950dfa6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f854bd9b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55950deff2f0 con 0x55950df995d0 2024-09-19T07:04:36.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f854bd9b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55950dea6cf0 con 0x55950dfa4370 2024-09-19T07:04:36.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f854bd9b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55950defb2b0 con 0x55950df9ab40 2024-09-19T07:04:36.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f8542ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df995d0 0x55950df99a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f8542ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df995d0 0x55950df99a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45922/0 (socket says 172.21.15.103:45922) 2024-09-19T07:04:36.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f8542ffd640 1 -- 172.21.15.103:0/2851477572 learned_addr learned my addr 172.21.15.103:0/2851477572 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:36.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f85427fc640 1 --2- 172.21.15.103:0/2851477572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950df9ab40 0x55950df99090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f85427fc640 1 --2- 172.21.15.103:0/2851477572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950dfa4370 0x55950dfa6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f85489c0640 1 --2- 172.21.15.103:0/3275409929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f852c0a33b0 0x7f852c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f8542ffd640 1 -- 172.21.15.103:0/2851477572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950df9ab40 msgr2=0x55950df99090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:36.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f8542ffd640 1 --2- 172.21.15.103:0/2851477572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950df9ab40 0x55950df99090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f8542ffd640 1 -- 172.21.15.103:0/2851477572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950dfa4370 msgr2=0x55950dfa6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:36.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f8542ffd640 1 --2- 172.21.15.103:0/2851477572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950dfa4370 0x55950dfa6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f8542ffd640 1 -- 172.21.15.103:0/2851477572 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55950de8b3a0 con 0x55950df995d0 2024-09-19T07:04:36.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f85437fe640 1 --2- 172.21.15.103:0/3275409929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f852c0a33b0 0x7f852c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f85489c0640 1 -- 172.21.15.103:0/3275409929 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f853c0f0de0 con 0x7f8544106f10 2024-09-19T07:04:36.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.893+0000 7f8542ffd640 1 --2- 172.21.15.103:0/2851477572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df995d0 0x55950df99a10 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f853400bd10 tx=0x7f85340107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c920218a9476f054 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:36.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.894+0000 7f8529ffb640 1 -- 172.21.15.103:0/2851477572 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534004030 con 0x55950df995d0 2024-09-19T07:04:36.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.894+0000 7f8529ffb640 1 -- 172.21.15.103:0/2851477572 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85340041d0 con 0x55950df995d0 2024-09-19T07:04:36.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.894+0000 7f8529ffb640 1 -- 172.21.15.103:0/2851477572 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534011800 con 0x55950df995d0 2024-09-19T07:04:36.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.894+0000 7f854bd9b740 1 -- 172.21.15.103:0/2851477572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df995d0 msgr2=0x55950df99a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:36.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.894+0000 7f85437fe640 1 --2- 172.21.15.103:0/3275409929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f852c0a33b0 0x7f852c0a5870 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f853c0bd1d0 tx=0x7f853c02a8c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:36.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.894+0000 7f854bd9b740 1 --2- 172.21.15.103:0/2851477572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df995d0 0x55950df99a10 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f853400bd10 tx=0x7f85340107a0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.894+0000 7f854bd9b740 1 -- 172.21.15.103:0/2851477572 shutdown_connections 2024-09-19T07:04:36.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.894+0000 7f854bd9b740 1 --2- 172.21.15.103:0/2851477572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950dfa4370 0x55950dfa6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.894+0000 7f854bd9b740 1 --2- 172.21.15.103:0/2851477572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df995d0 0x55950df99a10 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.894+0000 7f854bd9b740 1 --2- 172.21.15.103:0/2851477572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950df9ab40 0x55950df99090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.894+0000 7f854bd9b740 1 -- 172.21.15.103:0/2851477572 >> 172.21.15.103:0/2851477572 conn(0x55950df62310 msgr2=0x55950df95c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:36.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.894+0000 7f854bd9b740 1 -- 172.21.15.103:0/2851477572 shutdown_connections 2024-09-19T07:04:36.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.894+0000 7f854bd9b740 1 -- 172.21.15.103:0/2851477572 wait complete. 2024-09-19T07:04:36.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.895+0000 7f854bd9b740 1 -- start start 2024-09-19T07:04:36.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.895+0000 7f854bd9b740 1 Processor -- start 2024-09-19T07:04:36.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.895+0000 7f854bd9b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950df995d0 0x55950dfa3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.895+0000 7f854bd9b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df9ab40 0x55950dfa1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.895+0000 7f854bd9b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950dfa4370 0x55950dfa15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.895+0000 7f854bd9b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55950dfab480 con 0x55950df9ab40 2024-09-19T07:04:36.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.895+0000 7f854bd9b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55950deba060 con 0x55950df995d0 2024-09-19T07:04:36.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.895+0000 7f854bd9b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55950dfab600 con 0x55950dfa4370 2024-09-19T07:04:36.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.895+0000 7f854bd9b740 10 client.0 _ll_register_callbacks cb 0x55950dc0d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:36.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.895+0000 7f8542ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df9ab40 0x55950dfa1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.895+0000 7f85427fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950df995d0 0x55950dfa3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.895+0000 7f8542ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df9ab40 0x55950dfa1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45928/0 (socket says 172.21.15.103:45928) 2024-09-19T07:04:36.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.895+0000 7f8542ffd640 1 -- 172.21.15.103:0/1820699332 learned_addr learned my addr 172.21.15.103:0/1820699332 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:36.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.896+0000 7f85427fc640 1 --2- 172.21.15.103:0/1820699332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950dfa4370 0x55950dfa15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.896+0000 7f8542ffd640 1 -- 172.21.15.103:0/1820699332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950dfa4370 msgr2=0x55950dfa15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:36.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.896+0000 7f8542ffd640 1 --2- 172.21.15.103:0/1820699332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950dfa4370 0x55950dfa15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.896+0000 7f8542ffd640 1 -- 172.21.15.103:0/1820699332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950df995d0 msgr2=0x55950dfa3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:36.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.896+0000 7f8542ffd640 1 --2- 172.21.15.103:0/1820699332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950df995d0 0x55950dfa3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.896+0000 7f8542ffd640 1 -- 172.21.15.103:0/1820699332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55950dc703c0 con 0x55950df9ab40 2024-09-19T07:04:36.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.896+0000 7f85427fc640 1 --2- 172.21.15.103:0/1820699332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950df995d0 0x55950dfa3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:36.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.896+0000 7f8542ffd640 1 --2- 172.21.15.103:0/1820699332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df9ab40 0x55950dfa1000 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f8534003cc0 tx=0x7f8534003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f6519a7cf7634d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:36.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.896+0000 7f8522ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:36.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.896+0000 7f8522ffd640 1 -- 172.21.15.103:0/1820699332 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534029060 con 0x55950df9ab40 2024-09-19T07:04:36.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.896+0000 7f8522ffd640 1 -- 172.21.15.103:0/1820699332 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f8534004380 con 0x55950df9ab40 2024-09-19T07:04:36.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.896+0000 7f854bd9b740 1 -- 172.21.15.103:0/1820699332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85340041d0 con 0x55950df9ab40 2024-09-19T07:04:36.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.896+0000 7f8522ffd640 1 -- 172.21.15.103:0/1820699332 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f85340035f0 con 0x55950df9ab40 2024-09-19T07:04:36.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.896+0000 7f854bd9b740 10 client.20706 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:36.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.896+0000 7f854bd9b740 1 -- 172.21.15.103:0/1820699332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55950dfb6f00 con 0x55950df9ab40 2024-09-19T07:04:36.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.896+0000 7f8522ffd640 1 -- 172.21.15.103:0/1820699332 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85340041d0 con 0x55950df9ab40 2024-09-19T07:04:36.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.897+0000 7f8522ffd640 1 -- 172.21.15.103:0/1820699332 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8534004750 con 0x55950df9ab40 2024-09-19T07:04:36.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.897+0000 7f854bd9b740 10 client.20706 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:36.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.897+0000 7f854bd9b740 10 client.20706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:36.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.897+0000 7f854bd9b740 20 client.20706 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:36.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.897+0000 7f854bd9b740 1 --2- 172.21.15.103:0/1820699332 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55950dfb8300 0x55950dfba6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.897+0000 7f854bd9b740 4 client.20706 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:36.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.897+0000 7f854bd9b740 1 -- 172.21.15.103:0/1820699332 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55950df0c860 con 0x55950dfb8300 2024-09-19T07:04:36.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.897+0000 7f85427fc640 1 --2- 172.21.15.103:0/1820699332 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55950dfb8300 0x55950dfba6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.897+0000 7f8522ffd640 10 client.20706 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:36.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.897+0000 7f85427fc640 1 --2- 172.21.15.103:0/1820699332 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55950dfb8300 0x55950dfba6f0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20e0fce6a273b327 server_cookie=17c9f577b8fdb6be in_seq=0 out_seq=0 2024-09-19T07:04:36.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.898+0000 7f8522ffd640 1 -- 172.21.15.103:0/1820699332 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55950df0c860 con 0x55950dfb8300 2024-09-19T07:04:36.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.898+0000 7f8522ffd640 10 client.20706 handle_command_reply: tid=0 2024-09-19T07:04:36.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.898+0000 7f854bd9b740 1 client.20706 shutdown 2024-09-19T07:04:36.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.898+0000 7f854bd9b740 1 -- 172.21.15.103:0/1820699332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df9ab40 msgr2=0x55950dfa1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:36.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.898+0000 7f854bd9b740 1 --2- 172.21.15.103:0/1820699332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df9ab40 0x55950dfa1000 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f8534003cc0 tx=0x7f8534003cf0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.898+0000 7f854bd9b740 1 -- 172.21.15.103:0/1820699332 shutdown_connections 2024-09-19T07:04:36.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.898+0000 7f854bd9b740 1 --2- 172.21.15.103:0/1820699332 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55950dfb8300 0x55950dfba6f0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.898+0000 7f8522ffd640 0 client.20706 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:36.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.898+0000 7f854bd9b740 1 --2- 172.21.15.103:0/1820699332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950dfa4370 0x55950dfa15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.898+0000 7f854bd9b740 1 --2- 172.21.15.103:0/1820699332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df9ab40 0x55950dfa1000 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.898+0000 7f854bd9b740 1 --2- 172.21.15.103:0/1820699332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950df995d0 0x55950dfa3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.898+0000 7f854bd9b740 1 -- 172.21.15.103:0/1820699332 >> 172.21.15.103:0/1820699332 conn(0x55950df62310 msgr2=0x55950df9b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:36.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.899+0000 7f854bd9b740 1 -- 172.21.15.103:0/1820699332 shutdown_connections 2024-09-19T07:04:36.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.899+0000 7f854bd9b740 1 -- 172.21.15.103:0/1820699332 wait complete. 2024-09-19T07:04:36.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.899+0000 7f854bd9b740 20 client.20706 trim_cache size 0 max 16384 2024-09-19T07:04:36.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.911+0000 7f854bd9b740 1 Processor -- start 2024-09-19T07:04:36.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.911+0000 7f854bd9b740 1 -- start start 2024-09-19T07:04:36.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.911+0000 7f854bd9b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df995d0 0x55950dfb4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.911+0000 7f854bd9b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950df9ab40 0x55950dfa5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.911+0000 7f854bd9b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950dfa4370 0x55950dfa5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.911+0000 7f854bd9b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f85441ab6d0 con 0x55950df995d0 2024-09-19T07:04:36.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.911+0000 7f854bd9b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85441ad210 con 0x55950dfa4370 2024-09-19T07:04:36.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.911+0000 7f854bd9b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85441abda0 con 0x55950df9ab40 2024-09-19T07:04:36.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.912+0000 7f85427fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df995d0 0x55950dfb4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.912+0000 7f85427fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df995d0 0x55950dfb4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45932/0 (socket says 172.21.15.103:45932) 2024-09-19T07:04:36.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.912+0000 7f85427fc640 1 -- 172.21.15.103:0/1761126753 learned_addr learned my addr 172.21.15.103:0/1761126753 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:36.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.912+0000 7f8542ffd640 1 --2- 172.21.15.103:0/1761126753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950df9ab40 0x55950dfa5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.912+0000 7f85427fc640 1 --2- 172.21.15.103:0/1761126753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950dfa4370 0x55950dfa5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.912+0000 7f85427fc640 1 -- 172.21.15.103:0/1761126753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950df9ab40 msgr2=0x55950dfa5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:36.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.912+0000 7f85427fc640 1 --2- 172.21.15.103:0/1761126753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950df9ab40 0x55950dfa5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.912+0000 7f85427fc640 1 -- 172.21.15.103:0/1761126753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950dfa4370 msgr2=0x55950dfa5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:36.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.912+0000 7f85427fc640 1 --2- 172.21.15.103:0/1761126753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950dfa4370 0x55950dfa5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.912+0000 7f85427fc640 1 -- 172.21.15.103:0/1761126753 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55950df96960 con 0x55950df995d0 2024-09-19T07:04:36.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.912+0000 7f85427fc640 1 --2- 172.21.15.103:0/1761126753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df995d0 0x55950dfb4720 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f85440fdce0 tx=0x7f8538008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de86902b2eaf63d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:36.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.912+0000 7f85297fa640 1 -- 172.21.15.103:0/1761126753 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853802b070 con 0x55950df995d0 2024-09-19T07:04:36.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.912+0000 7f85297fa640 1 -- 172.21.15.103:0/1761126753 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8538023040 con 0x55950df995d0 2024-09-19T07:04:36.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.912+0000 7f85297fa640 1 -- 172.21.15.103:0/1761126753 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853801f030 con 0x55950df995d0 2024-09-19T07:04:36.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.913+0000 7f854bd9b740 1 -- 172.21.15.103:0/1761126753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df995d0 msgr2=0x55950dfb4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:36.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.913+0000 7f854bd9b740 1 --2- 172.21.15.103:0/1761126753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df995d0 0x55950dfb4720 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f85440fdce0 tx=0x7f8538008040 comp rx=0 tx=0).stop 2024-09-19T07:04:36.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.913+0000 7f854bd9b740 1 -- 172.21.15.103:0/1761126753 shutdown_connections 2024-09-19T07:04:36.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.913+0000 7f854bd9b740 1 --2- 172.21.15.103:0/1761126753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950dfa4370 0x55950dfa5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.913+0000 7f854bd9b740 1 --2- 172.21.15.103:0/1761126753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950df9ab40 0x55950dfa5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.913+0000 7f854bd9b740 1 --2- 172.21.15.103:0/1761126753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950df995d0 0x55950dfb4720 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.913+0000 7f854bd9b740 1 -- 172.21.15.103:0/1761126753 >> 172.21.15.103:0/1761126753 conn(0x55950df62310 msgr2=0x55950df9c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:36.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.913+0000 7f854bd9b740 1 -- 172.21.15.103:0/1761126753 shutdown_connections 2024-09-19T07:04:36.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.913+0000 7f854bd9b740 1 -- 172.21.15.103:0/1761126753 wait complete. 2024-09-19T07:04:36.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.913+0000 7f854bd9b740 1 -- start start 2024-09-19T07:04:36.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.913+0000 7f854bd9b740 1 Processor -- start 2024-09-19T07:04:36.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f854bd9b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950df995d0 0x55950dfb0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f854bd9b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950df9ab40 0x55950dfbf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f854bd9b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950dfa4370 0x55950dfbf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f854bd9b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f85441acb40 con 0x55950dfa4370 2024-09-19T07:04:36.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f854bd9b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85441ac470 con 0x55950df9ab40 2024-09-19T07:04:36.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f854bd9b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85441a96b0 con 0x55950df995d0 2024-09-19T07:04:36.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f854bd9b740 10 client.0 _ll_register_callbacks cb 0x7f8538009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:36.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f85427fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950dfa4370 0x55950dfbf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f85427fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950df995d0 0x55950dfb0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f85427fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950dfa4370 0x55950dfbf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45938/0 (socket says 172.21.15.103:45938) 2024-09-19T07:04:36.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f85427fc640 1 -- 172.21.15.103:0/200200820 learned_addr learned my addr 172.21.15.103:0/200200820 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:36.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f8542ffd640 1 --2- 172.21.15.103:0/200200820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950df9ab40 0x55950dfbf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f85427fc640 1 -- 172.21.15.103:0/200200820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950df995d0 msgr2=0x55950dfb0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:36.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f85427fc640 1 --2- 172.21.15.103:0/200200820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950df995d0 0x55950dfb0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f85427fc640 1 -- 172.21.15.103:0/200200820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950df9ab40 msgr2=0x55950dfbf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:36.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f85427fc640 1 --2- 172.21.15.103:0/200200820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950df9ab40 0x55950dfbf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f85427fc640 1 -- 172.21.15.103:0/200200820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55950dfbfcf0 con 0x55950dfa4370 2024-09-19T07:04:36.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f85427fc640 1 --2- 172.21.15.103:0/200200820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950dfa4370 0x55950dfbf7b0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f85380099d0 tx=0x7f85380156b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e352ad3f077250 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:36.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f8522ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:36.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f8522ffd640 1 -- 172.21.15.103:0/200200820 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853801f070 con 0x55950dfa4370 2024-09-19T07:04:36.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f8522ffd640 1 -- 172.21.15.103:0/200200820 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f8538018ad0 con 0x55950dfa4370 2024-09-19T07:04:36.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.914+0000 7f854bd9b740 1 -- 172.21.15.103:0/200200820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8538023040 con 0x55950dfa4370 2024-09-19T07:04:36.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.915+0000 7f8522ffd640 1 -- 172.21.15.103:0/200200820 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f853802b040 con 0x55950dfa4370 2024-09-19T07:04:36.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.915+0000 7f854bd9b740 10 client.20712 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:36.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.915+0000 7f854bd9b740 1 -- 172.21.15.103:0/200200820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55950dfaf530 con 0x55950dfa4370 2024-09-19T07:04:36.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.915+0000 7f8522ffd640 1 -- 172.21.15.103:0/200200820 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8538023040 con 0x55950dfa4370 2024-09-19T07:04:36.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.915+0000 7f8522ffd640 1 -- 172.21.15.103:0/200200820 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8538031040 con 0x55950dfa4370 2024-09-19T07:04:36.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.915+0000 7f854bd9b740 10 client.20712 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:36.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.915+0000 7f854bd9b740 10 client.20712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:36.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.915+0000 7f854bd9b740 20 client.20712 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:36.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.915+0000 7f854bd9b740 1 --2- 172.21.15.103:0/200200820 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55950dfb8300 0x55950e025040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:36.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.915+0000 7f854bd9b740 4 client.20712 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:04:36.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.915+0000 7f854bd9b740 1 -- 172.21.15.103:0/200200820 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55950df06d70 con 0x55950dfb8300 2024-09-19T07:04:36.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.915+0000 7f8542ffd640 1 --2- 172.21.15.103:0/200200820 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55950dfb8300 0x55950e025040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:36.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f8542ffd640 1 --2- 172.21.15.103:0/200200820 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55950dfb8300 0x55950e025040 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=389170d646c91ed4 server_cookie=963b2416b092994b in_seq=0 out_seq=0 2024-09-19T07:04:36.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f8522ffd640 10 client.20712 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:36.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f8522ffd640 1 -- 172.21.15.103:0/200200820 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55950df06d70 con 0x55950dfb8300 2024-09-19T07:04:36.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f8522ffd640 10 client.20712 handle_command_reply: tid=0 2024-09-19T07:04:36.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f854bd9b740 1 client.20712 shutdown 2024-09-19T07:04:36.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f854bd9b740 1 -- 172.21.15.103:0/200200820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950dfa4370 msgr2=0x55950dfbf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:36.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f854bd9b740 1 --2- 172.21.15.103:0/200200820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950dfa4370 0x55950dfbf7b0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f85380099d0 tx=0x7f85380156b0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f854bd9b740 1 -- 172.21.15.103:0/200200820 shutdown_connections 2024-09-19T07:04:36.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f854bd9b740 1 --2- 172.21.15.103:0/200200820 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55950dfb8300 0x55950e025040 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f854bd9b740 1 --2- 172.21.15.103:0/200200820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55950dfa4370 0x55950dfbf7b0 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f854bd9b740 1 --2- 172.21.15.103:0/200200820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55950df9ab40 0x55950dfbf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f854bd9b740 1 --2- 172.21.15.103:0/200200820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950df995d0 0x55950dfb0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f854bd9b740 1 -- 172.21.15.103:0/200200820 >> 172.21.15.103:0/200200820 conn(0x55950df62310 msgr2=0x55950dfa00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:36.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f8522ffd640 0 client.20712 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:36.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f854bd9b740 1 -- 172.21.15.103:0/200200820 shutdown_connections 2024-09-19T07:04:36.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f854bd9b740 1 -- 172.21.15.103:0/200200820 wait complete. 2024-09-19T07:04:36.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.916+0000 7f854bd9b740 20 client.20712 trim_cache size 0 max 16384 2024-09-19T07:04:36.924 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:04:36.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.917+0000 7f85499c2640 1 -- 172.21.15.103:0/3275409929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f852c0a33b0 msgr2=0x7f852c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:36.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.917+0000 7f85499c2640 1 --2- 172.21.15.103:0/3275409929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f852c0a33b0 0x7f852c0a5870 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f853c0bd1d0 tx=0x7f853c02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.917+0000 7f85499c2640 1 -- 172.21.15.103:0/3275409929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8544106f10 msgr2=0x7f854419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:36.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.917+0000 7f85499c2640 1 --2- 172.21.15.103:0/3275409929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8544106f10 0x7f854419cd20 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f853c031a20 tx=0x7f853c031a50 comp rx=0 tx=0).stop 2024-09-19T07:04:36.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.917+0000 7f85499c2640 1 -- 172.21.15.103:0/3275409929 shutdown_connections 2024-09-19T07:04:36.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.917+0000 7f85499c2640 1 --2- 172.21.15.103:0/3275409929 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f852c0a33b0 0x7f852c0a5870 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.917+0000 7f85499c2640 1 --2- 172.21.15.103:0/3275409929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8544106f10 0x7f854419cd20 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.917+0000 7f85499c2640 1 --2- 172.21.15.103:0/3275409929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8544102790 0x7f8544198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.917+0000 7f85499c2640 1 --2- 172.21.15.103:0/3275409929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8544101de0 0x7f8544198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:36.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.917+0000 7f85499c2640 1 -- 172.21.15.103:0/3275409929 >> 172.21.15.103:0/3275409929 conn(0x7f85440fd8c0 msgr2=0x7f85440fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:36.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.917+0000 7f85499c2640 1 -- 172.21.15.103:0/3275409929 shutdown_connections 2024-09-19T07:04:36.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:36.917+0000 7f85499c2640 1 -- 172.21.15.103:0/3275409929 wait complete. 2024-09-19T07:04:37.573 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:37 smithi103 ceph-mon[31158]: pgmap v482: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-09-19T07:04:37.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:37 smithi174 ceph-mon[35637]: pgmap v482: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-09-19T07:04:37.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:37 smithi134 ceph-mon[35356]: pgmap v482: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-09-19T07:04:38.682 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:04:38.682 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:04:38.940 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:39.299 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:39 smithi103 ceph-mon[31158]: pgmap v483: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-09-19T07:04:39.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.382+0000 7f3c5c4f8640 1 -- 172.21.15.103:0/2340408391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c541002f0 msgr2=0x7f3c541006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:39.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.382+0000 7f3c5c4f8640 1 --2- 172.21.15.103:0/2340408391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c541002f0 0x7f3c541006d0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009930 tx=0x7f3c5002a420 comp rx=0 tx=0).stop 2024-09-19T07:04:39.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.382+0000 7f3c5c4f8640 1 -- 172.21.15.103:0/2340408391 shutdown_connections 2024-09-19T07:04:39.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.382+0000 7f3c5c4f8640 1 --2- 172.21.15.103:0/2340408391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c54106f50 0x7f3c5410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.382+0000 7f3c5c4f8640 1 --2- 172.21.15.103:0/2340408391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c54100ca0 0x7f3c54106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.382+0000 7f3c5c4f8640 1 --2- 172.21.15.103:0/2340408391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c541002f0 0x7f3c541006d0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.382+0000 7f3c5c4f8640 1 -- 172.21.15.103:0/2340408391 >> 172.21.15.103:0/2340408391 conn(0x7f3c54077210 msgr2=0x7f3c540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:39.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.382+0000 7f3c5c4f8640 1 -- 172.21.15.103:0/2340408391 shutdown_connections 2024-09-19T07:04:39.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.382+0000 7f3c5c4f8640 1 -- 172.21.15.103:0/2340408391 wait complete. 2024-09-19T07:04:39.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5c4f8640 1 Processor -- start 2024-09-19T07:04:39.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5c4f8640 1 -- start start 2024-09-19T07:04:39.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5c4f8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c541002f0 0x7f3c54101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5c4f8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c54100ca0 0x7f3c541017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5c4f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c54106f50 0x7f3c54101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5c4f8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c54111fa0 con 0x7f3c54106f50 2024-09-19T07:04:39.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5c4f8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c54111e20 con 0x7f3c541002f0 2024-09-19T07:04:39.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5c4f8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c54112120 con 0x7f3c54100ca0 2024-09-19T07:04:39.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5aa95640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c54106f50 0x7f3c54101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:39.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5aa95640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c54106f50 0x7f3c54101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56622/0 (socket says 172.21.15.103:56622) 2024-09-19T07:04:39.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5aa95640 1 -- 172.21.15.103:0/3851428348 learned_addr learned my addr 172.21.15.103:0/3851428348 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:39.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c59a93640 1 --2- 172.21.15.103:0/3851428348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c54100ca0 0x7f3c541017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:39.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5a294640 1 --2- 172.21.15.103:0/3851428348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c541002f0 0x7f3c54101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:39.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5aa95640 1 -- 172.21.15.103:0/3851428348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c54100ca0 msgr2=0x7f3c541017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:39.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5aa95640 1 --2- 172.21.15.103:0/3851428348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c54100ca0 0x7f3c541017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5aa95640 1 -- 172.21.15.103:0/3851428348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c541002f0 msgr2=0x7f3c54101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:39.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5aa95640 1 --2- 172.21.15.103:0/3851428348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c541002f0 0x7f3c54101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.383+0000 7f3c5aa95640 1 -- 172.21.15.103:0/3851428348 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c541a6a00 con 0x7f3c54106f50 2024-09-19T07:04:39.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.384+0000 7f3c5aa95640 1 --2- 172.21.15.103:0/3851428348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c54106f50 0x7f3c54101cf0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00cad0 tx=0x7f3c4c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:39.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.384+0000 7f3c4b7fe640 1 -- 172.21.15.103:0/3851428348 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00ae20 con 0x7f3c54106f50 2024-09-19T07:04:39.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.384+0000 7f3c4b7fe640 1 -- 172.21.15.103:0/3851428348 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c4c005020 con 0x7f3c54106f50 2024-09-19T07:04:39.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.384+0000 7f3c5c4f8640 1 -- 172.21.15.103:0/3851428348 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c541a6cf0 con 0x7f3c54106f50 2024-09-19T07:04:39.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.384+0000 7f3c4b7fe640 1 -- 172.21.15.103:0/3851428348 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00f6f0 con 0x7f3c54106f50 2024-09-19T07:04:39.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.384+0000 7f3c5c4f8640 1 -- 172.21.15.103:0/3851428348 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c541a7230 con 0x7f3c54106f50 2024-09-19T07:04:39.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.386+0000 7f3c4b7fe640 1 -- 172.21.15.103:0/3851428348 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3c4c0051d0 con 0x7f3c54106f50 2024-09-19T07:04:39.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.386+0000 7f3c5e8d1740 1 Processor -- start 2024-09-19T07:04:39.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.386+0000 7f3c5e8d1740 1 -- start start 2024-09-19T07:04:39.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5e8d1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8d2b60 0x55f46e8d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5e8d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d3480 0x55f46e8dd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5e8d1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8ddd40 0x55f46e8e0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5e8d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f46e838280 con 0x55f46e8d3480 2024-09-19T07:04:39.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5e8d1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f46e7dfb10 con 0x55f46e8ddd40 2024-09-19T07:04:39.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5e8d1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f46e834240 con 0x55f46e8d2b60 2024-09-19T07:04:39.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c4b7fe640 1 --2- 172.21.15.103:0/3851428348 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c400a3430 0x7f3c400a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5a294640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d3480 0x55f46e8dd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:39.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5a294640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d3480 0x55f46e8dd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56630/0 (socket says 172.21.15.103:56630) 2024-09-19T07:04:39.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5a294640 1 -- 172.21.15.103:0/2118461603 learned_addr learned my addr 172.21.15.103:0/2118461603 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:39.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5aa95640 1 --2- 172.21.15.103:0/3851428348 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c400a3430 0x7f3c400a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:39.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c59a93640 1 --2- 172.21.15.103:0/2118461603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8d2b60 0x55f46e8d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:39.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c59a93640 1 --2- 172.21.15.103:0/2118461603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8ddd40 0x55f46e8e0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:39.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c4b7fe640 1 -- 172.21.15.103:0/3851428348 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f3c4c0cfce0 con 0x7f3c54106f50 2024-09-19T07:04:39.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5a294640 1 -- 172.21.15.103:0/2118461603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8d2b60 msgr2=0x55f46e8d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:39.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5a294640 1 --2- 172.21.15.103:0/2118461603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8d2b60 0x55f46e8d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5a294640 1 -- 172.21.15.103:0/2118461603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8ddd40 msgr2=0x55f46e8e0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:39.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5a294640 1 --2- 172.21.15.103:0/2118461603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8ddd40 0x55f46e8e0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5a294640 1 -- 172.21.15.103:0/2118461603 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f46e7ec1f0 con 0x55f46e8d3480 2024-09-19T07:04:39.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c59a93640 1 --2- 172.21.15.103:0/2118461603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8ddd40 0x55f46e8e0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:39.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5a294640 1 --2- 172.21.15.103:0/2118461603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d3480 0x55f46e8dd800 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009930 tx=0x7f3c50032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a5ac82412dbab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:39.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.387+0000 7f3c5aa95640 1 --2- 172.21.15.103:0/3851428348 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c400a3430 0x7f3c400a58f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c098400 tx=0x7f3c4c06dd20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:39.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.388+0000 7f3c48ff9640 1 -- 172.21.15.103:0/2118461603 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5003f070 con 0x55f46e8d3480 2024-09-19T07:04:39.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.388+0000 7f3c48ff9640 1 -- 172.21.15.103:0/2118461603 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c500317b0 con 0x55f46e8d3480 2024-09-19T07:04:39.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.388+0000 7f3c48ff9640 1 -- 172.21.15.103:0/2118461603 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50035070 con 0x55f46e8d3480 2024-09-19T07:04:39.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.388+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/2118461603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d3480 msgr2=0x55f46e8dd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:39.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.388+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/2118461603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d3480 0x55f46e8dd800 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009930 tx=0x7f3c50032ce0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.388+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/2118461603 shutdown_connections 2024-09-19T07:04:39.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.388+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/2118461603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8ddd40 0x55f46e8e0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.388+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/2118461603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d3480 0x55f46e8dd800 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.388+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/2118461603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8d2b60 0x55f46e8d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.388+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/2118461603 >> 172.21.15.103:0/2118461603 conn(0x55f46e8cf0f0 msgr2=0x55f46e8cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:39.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.388+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/2118461603 shutdown_connections 2024-09-19T07:04:39.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.388+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/2118461603 wait complete. 2024-09-19T07:04:39.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.389+0000 7f3c5e8d1740 1 -- start start 2024-09-19T07:04:39.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.389+0000 7f3c5e8d1740 1 Processor -- start 2024-09-19T07:04:39.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.389+0000 7f3c5e8d1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8d2b60 0x55f46e8dab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.389+0000 7f3c5e8d1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8d3480 0x55f46e8db0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.389+0000 7f3c5e8d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8ddd40 0x55f46e8db610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.389+0000 7f3c5e8d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f46e8e0910 con 0x55f46e8ddd40 2024-09-19T07:04:39.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.389+0000 7f3c5e8d1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f46e7f2f10 con 0x55f46e8d3480 2024-09-19T07:04:39.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.389+0000 7f3c5e8d1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f46e8e0a90 con 0x55f46e8d2b60 2024-09-19T07:04:39.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.389+0000 7f3c5e8d1740 10 client.0 _ll_register_callbacks cb 0x55f46e7bba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:39.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.389+0000 7f3c59a93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8ddd40 0x55f46e8db610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:39.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.389+0000 7f3c59a93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8ddd40 0x55f46e8db610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56644/0 (socket says 172.21.15.103:56644) 2024-09-19T07:04:39.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.389+0000 7f3c59a93640 1 -- 172.21.15.103:0/1628955987 learned_addr learned my addr 172.21.15.103:0/1628955987 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:39.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.389+0000 7f3c59a93640 1 --2- 172.21.15.103:0/1628955987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8d2b60 0x55f46e8dab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:39.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.389+0000 7f3c5a294640 1 --2- 172.21.15.103:0/1628955987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8d3480 0x55f46e8db0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:39.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.390+0000 7f3c59a93640 1 -- 172.21.15.103:0/1628955987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8d2b60 msgr2=0x55f46e8dab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:39.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.390+0000 7f3c59a93640 1 --2- 172.21.15.103:0/1628955987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8d2b60 0x55f46e8dab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.390+0000 7f3c59a93640 1 -- 172.21.15.103:0/1628955987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8d3480 msgr2=0x55f46e8db0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:39.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.390+0000 7f3c59a93640 1 --2- 172.21.15.103:0/1628955987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8d3480 0x55f46e8db0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.390+0000 7f3c59a93640 1 -- 172.21.15.103:0/1628955987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f46e5a96a0 con 0x55f46e8ddd40 2024-09-19T07:04:39.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.390+0000 7f3c59a93640 1 --2- 172.21.15.103:0/1628955987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8ddd40 0x55f46e8db610 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f3c44003670 tx=0x7f3c440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebee7de33d5f50c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:39.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.390+0000 7f3c3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:39.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.390+0000 7f3c3dffb640 1 -- 172.21.15.103:0/1628955987 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c440126f0 con 0x55f46e8ddd40 2024-09-19T07:04:39.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.390+0000 7f3c3dffb640 1 -- 172.21.15.103:0/1628955987 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f3c44012890 con 0x55f46e8ddd40 2024-09-19T07:04:39.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.390+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/1628955987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c500317b0 con 0x55f46e8ddd40 2024-09-19T07:04:39.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.390+0000 7f3c3dffb640 1 -- 172.21.15.103:0/1628955987 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c44012a90 con 0x55f46e8ddd40 2024-09-19T07:04:39.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.390+0000 7f3c5e8d1740 10 client.20724 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:39.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.390+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/1628955987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f46e8ec4d0 con 0x55f46e8ddd40 2024-09-19T07:04:39.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.390+0000 7f3c3dffb640 1 -- 172.21.15.103:0/1628955987 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c500317b0 con 0x55f46e8ddd40 2024-09-19T07:04:39.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.391+0000 7f3c3dffb640 1 -- 172.21.15.103:0/1628955987 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3c44020c30 con 0x55f46e8ddd40 2024-09-19T07:04:39.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.391+0000 7f3c5e8d1740 10 client.20724 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:39.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.391+0000 7f3c5e8d1740 10 client.20724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:39.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.391+0000 7f3c5e8d1740 20 client.20724 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:39.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.391+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/1628955987 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f46e8ed8e0 0x55f46e8efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.391+0000 7f3c5e8d1740 4 client.20724 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:39.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.391+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/1628955987 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f46e8457f0 con 0x55f46e8ed8e0 2024-09-19T07:04:39.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.391+0000 7f3c5a294640 1 --2- 172.21.15.103:0/1628955987 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f46e8ed8e0 0x55f46e8efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:39.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.391+0000 7f3c5a294640 1 --2- 172.21.15.103:0/1628955987 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f46e8ed8e0 0x55f46e8efcd0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c2d142d765ce545 server_cookie=aa82f4fecd3bc928 in_seq=0 out_seq=0 2024-09-19T07:04:39.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.391+0000 7f3c3dffb640 10 client.20724 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:39.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.392+0000 7f3c3dffb640 1 -- 172.21.15.103:0/1628955987 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f46e8457f0 con 0x55f46e8ed8e0 2024-09-19T07:04:39.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.392+0000 7f3c3dffb640 10 client.20724 handle_command_reply: tid=0 2024-09-19T07:04:39.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.392+0000 7f3c5e8d1740 1 client.20724 shutdown 2024-09-19T07:04:39.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.392+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/1628955987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8ddd40 msgr2=0x55f46e8db610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:39.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.392+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/1628955987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8ddd40 0x55f46e8db610 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f3c44003670 tx=0x7f3c440046c0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.392+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/1628955987 shutdown_connections 2024-09-19T07:04:39.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.392+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/1628955987 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f46e8ed8e0 0x55f46e8efcd0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.392+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/1628955987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8ddd40 0x55f46e8db610 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.392+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/1628955987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8d3480 0x55f46e8db0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.392+0000 7f3c3dffb640 0 client.20724 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:39.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.392+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/1628955987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8d2b60 0x55f46e8dab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.392+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/1628955987 >> 172.21.15.103:0/1628955987 conn(0x55f46e8cf0f0 msgr2=0x55f46e8ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:39.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.392+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/1628955987 shutdown_connections 2024-09-19T07:04:39.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.392+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/1628955987 wait complete. 2024-09-19T07:04:39.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.392+0000 7f3c5e8d1740 20 client.20724 trim_cache size 0 max 16384 2024-09-19T07:04:39.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.409+0000 7f3c5e8d1740 1 Processor -- start 2024-09-19T07:04:39.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.409+0000 7f3c5e8d1740 1 -- start start 2024-09-19T07:04:39.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.409+0000 7f3c5e8d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d2b60 0x55f46e8d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.409+0000 7f3c5e8d1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8d3480 0x55f46e8f5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.409+0000 7f3c5e8d1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8ddd40 0x55f46e8f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.409+0000 7f3c5e8d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c541afbc0 con 0x55f46e8d2b60 2024-09-19T07:04:39.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.409+0000 7f3c5e8d1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c541b1700 con 0x55f46e8d3480 2024-09-19T07:04:39.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.409+0000 7f3c5e8d1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c541b0290 con 0x55f46e8ddd40 2024-09-19T07:04:39.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.409+0000 7f3c59a93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d2b60 0x55f46e8d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:39.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.410+0000 7f3c59a93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d2b60 0x55f46e8d55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56648/0 (socket says 172.21.15.103:56648) 2024-09-19T07:04:39.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.410+0000 7f3c59a93640 1 -- 172.21.15.103:0/3383992220 learned_addr learned my addr 172.21.15.103:0/3383992220 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:39.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.410+0000 7f3c59a93640 1 -- 172.21.15.103:0/3383992220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8ddd40 msgr2=0x55f46e8f5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:39.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.410+0000 7f3c59a93640 1 --2- 172.21.15.103:0/3383992220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8ddd40 0x55f46e8f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.410+0000 7f3c59a93640 1 -- 172.21.15.103:0/3383992220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8d3480 msgr2=0x55f46e8f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:39.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.410+0000 7f3c59a93640 1 --2- 172.21.15.103:0/3383992220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8d3480 0x55f46e8f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.410+0000 7f3c59a93640 1 -- 172.21.15.103:0/3383992220 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f46e8ceb90 con 0x55f46e8d2b60 2024-09-19T07:04:39.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.410+0000 7f3c59a93640 1 --2- 172.21.15.103:0/3383992220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d2b60 0x55f46e8d55c0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f3c44005d00 tx=0x7f3c4400f7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea6289a22d19ce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:39.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.410+0000 7f3c3ffff640 1 -- 172.21.15.103:0/3383992220 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4402c070 con 0x55f46e8d2b60 2024-09-19T07:04:39.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.410+0000 7f3c3ffff640 1 -- 172.21.15.103:0/3383992220 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c4400fe40 con 0x55f46e8d2b60 2024-09-19T07:04:39.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.410+0000 7f3c3ffff640 1 -- 172.21.15.103:0/3383992220 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c440126f0 con 0x55f46e8d2b60 2024-09-19T07:04:39.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.411+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/3383992220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d2b60 msgr2=0x55f46e8d55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:39.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.411+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/3383992220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d2b60 0x55f46e8d55c0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f3c44005d00 tx=0x7f3c4400f7a0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.411+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/3383992220 shutdown_connections 2024-09-19T07:04:39.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.411+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/3383992220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8ddd40 0x55f46e8f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.411+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/3383992220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8d3480 0x55f46e8f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.411+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/3383992220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d2b60 0x55f46e8d55c0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.411+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/3383992220 >> 172.21.15.103:0/3383992220 conn(0x55f46e8cf0f0 msgr2=0x55f46e8dcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:39.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.411+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/3383992220 shutdown_connections 2024-09-19T07:04:39.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.411+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/3383992220 wait complete. 2024-09-19T07:04:39.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.411+0000 7f3c5e8d1740 1 -- start start 2024-09-19T07:04:39.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.411+0000 7f3c5e8d1740 1 Processor -- start 2024-09-19T07:04:39.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c5e8d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d2b60 0x55f46e8e5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c5e8d1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8d3480 0x55f46e8e6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c5e8d1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8ddd40 0x55f46e8e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c5e8d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c541b1030 con 0x55f46e8d2b60 2024-09-19T07:04:39.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c5e8d1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c541b0960 con 0x55f46e8d3480 2024-09-19T07:04:39.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c5e8d1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c541adbc0 con 0x55f46e8ddd40 2024-09-19T07:04:39.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c5e8d1740 10 client.0 _ll_register_callbacks cb 0x7f3c440040c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c59a93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d2b60 0x55f46e8e5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c59a93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d2b60 0x55f46e8e5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56660/0 (socket says 172.21.15.103:56660) 2024-09-19T07:04:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c59a93640 1 -- 172.21.15.103:0/3503237105 learned_addr learned my addr 172.21.15.103:0/3503237105 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c5a294640 1 --2- 172.21.15.103:0/3503237105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8d3480 0x55f46e8e6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c59a93640 1 -- 172.21.15.103:0/3503237105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8ddd40 msgr2=0x55f46e8e2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c59a93640 1 --2- 172.21.15.103:0/3503237105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8ddd40 0x55f46e8e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c59a93640 1 -- 172.21.15.103:0/3503237105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8d3480 msgr2=0x55f46e8e6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c59a93640 1 --2- 172.21.15.103:0/3503237105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8d3480 0x55f46e8e6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c59a93640 1 -- 172.21.15.103:0/3503237105 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f46e8e3250 con 0x55f46e8d2b60 2024-09-19T07:04:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c59a93640 1 --2- 172.21.15.103:0/3503237105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d2b60 0x55f46e8e5d40 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f3c44011040 tx=0x7f3c4400f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3dc43589424765a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c3dffb640 1 -- 172.21.15.103:0/3503237105 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44034020 con 0x55f46e8d2b60 2024-09-19T07:04:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c3dffb640 1 -- 172.21.15.103:0/3503237105 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f3c4400fb60 con 0x55f46e8d2b60 2024-09-19T07:04:39.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/3503237105 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f46e83fd00 con 0x55f46e8d2b60 2024-09-19T07:04:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c3dffb640 1 -- 172.21.15.103:0/3503237105 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c4403f050 con 0x55f46e8d2b60 2024-09-19T07:04:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c5e8d1740 10 client.20730 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.412+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/3503237105 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f46e958d20 con 0x55f46e8d2b60 2024-09-19T07:04:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.413+0000 7f3c3dffb640 1 -- 172.21.15.103:0/3503237105 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f46e83fd00 con 0x55f46e8d2b60 2024-09-19T07:04:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.413+0000 7f3c3dffb640 1 -- 172.21.15.103:0/3503237105 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3c44004690 con 0x55f46e8d2b60 2024-09-19T07:04:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.413+0000 7f3c5e8d1740 10 client.20730 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.413+0000 7f3c5e8d1740 10 client.20730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.413+0000 7f3c5e8d1740 20 client.20730 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.413+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/3503237105 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f46e9593d0 0x55f46e95b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.413+0000 7f3c5e8d1740 4 client.20730 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:04:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.413+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/3503237105 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f46e7bc110 con 0x55f46e9593d0 2024-09-19T07:04:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.413+0000 7f3c5a294640 1 --2- 172.21.15.103:0/3503237105 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f46e9593d0 0x55f46e95b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:39.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.413+0000 7f3c5a294640 1 --2- 172.21.15.103:0/3503237105 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f46e9593d0 0x55f46e95b7f0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fab9283eb921b287 server_cookie=a27827f9f332ee76 in_seq=0 out_seq=0 2024-09-19T07:04:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.413+0000 7f3c3dffb640 10 client.20730 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.414+0000 7f3c3dffb640 1 -- 172.21.15.103:0/3503237105 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f46e7bc110 con 0x55f46e9593d0 2024-09-19T07:04:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.414+0000 7f3c3dffb640 10 client.20730 handle_command_reply: tid=0 2024-09-19T07:04:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.414+0000 7f3c5e8d1740 1 client.20730 shutdown 2024-09-19T07:04:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/3503237105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d2b60 msgr2=0x55f46e8e5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/3503237105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d2b60 0x55f46e8e5d40 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f3c44011040 tx=0x7f3c4400f810 comp rx=0 tx=0).stop 2024-09-19T07:04:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/3503237105 shutdown_connections 2024-09-19T07:04:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/3503237105 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55f46e9593d0 0x55f46e95b7f0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/3503237105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f46e8ddd40 0x55f46e8e2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/3503237105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f46e8d3480 0x55f46e8e6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5e8d1740 1 --2- 172.21.15.103:0/3503237105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f46e8d2b60 0x55f46e8e5d40 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/3503237105 >> 172.21.15.103:0/3503237105 conn(0x55f46e8cf0f0 msgr2=0x55f46e8dab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:39.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c3dffb640 0 client.20730 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:39.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/3503237105 shutdown_connections 2024-09-19T07:04:39.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5e8d1740 1 -- 172.21.15.103:0/3503237105 wait complete. 2024-09-19T07:04:39.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5e8d1740 20 client.20730 trim_cache size 0 max 16384 2024-09-19T07:04:39.422 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:04:39.422 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:04:39.422 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "f0dfd048-c810-4173-84a1-3eb30e77fd3a", 2024-09-19T07:04:39.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:04:39.422 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:04:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5c4f8640 1 -- 172.21.15.103:0/3851428348 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c400a3430 msgr2=0x7f3c400a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5c4f8640 1 --2- 172.21.15.103:0/3851428348 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c400a3430 0x7f3c400a58f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c098400 tx=0x7f3c4c06dd20 comp rx=0 tx=0).stop 2024-09-19T07:04:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5c4f8640 1 -- 172.21.15.103:0/3851428348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c54106f50 msgr2=0x7f3c54101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5c4f8640 1 --2- 172.21.15.103:0/3851428348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c54106f50 0x7f3c54101cf0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00cad0 tx=0x7f3c4c00cf90 comp rx=0 tx=0).stop 2024-09-19T07:04:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5c4f8640 1 -- 172.21.15.103:0/3851428348 shutdown_connections 2024-09-19T07:04:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5c4f8640 1 --2- 172.21.15.103:0/3851428348 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c400a3430 0x7f3c400a58f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5c4f8640 1 --2- 172.21.15.103:0/3851428348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c54106f50 0x7f3c54101cf0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5c4f8640 1 --2- 172.21.15.103:0/3851428348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c54100ca0 0x7f3c541017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5c4f8640 1 --2- 172.21.15.103:0/3851428348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c541002f0 0x7f3c54101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.415+0000 7f3c5c4f8640 1 -- 172.21.15.103:0/3851428348 >> 172.21.15.103:0/3851428348 conn(0x7f3c54077210 msgr2=0x7f3c54105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.416+0000 7f3c5c4f8640 1 -- 172.21.15.103:0/3851428348 shutdown_connections 2024-09-19T07:04:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:39.416+0000 7f3c5c4f8640 1 -- 172.21.15.103:0/3851428348 wait complete. 2024-09-19T07:04:39.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:39 smithi174 ceph-mon[35637]: pgmap v483: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-09-19T07:04:39.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:39 smithi134 ceph-mon[35356]: pgmap v483: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-09-19T07:04:40.176 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:04:40.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:40 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T07:04:40.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:40 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:04:40.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:40 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T07:04:40.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:40 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:04:40.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:40 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T07:04:40.426 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:40.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:40 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T07:04:40.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:40 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:04:40.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:40 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T07:04:40.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:40 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:04:40.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:40 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T07:04:40.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:40 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T07:04:40.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:40 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:04:40.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:40 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T07:04:40.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:40 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:04:40.688 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:40 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T07:04:40.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.831+0000 7eff3f107640 1 -- 172.21.15.103:0/3199387756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff38100e20 msgr2=0x7eff38101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:40.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.831+0000 7eff3f107640 1 --2- 172.21.15.103:0/3199387756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff38100e20 0x7eff38101200 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7eff30009900 tx=0x7eff3002a480 comp rx=0 tx=0).stop 2024-09-19T07:04:40.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.831+0000 7eff3f107640 1 -- 172.21.15.103:0/3199387756 shutdown_connections 2024-09-19T07:04:40.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.831+0000 7eff3f107640 1 --2- 172.21.15.103:0/3199387756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff38102ef0 0x7eff3810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.831+0000 7eff3f107640 1 --2- 172.21.15.103:0/3199387756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff38102570 0x7eff381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.831+0000 7eff3f107640 1 --2- 172.21.15.103:0/3199387756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff38100e20 0x7eff38101200 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.831+0000 7eff3f107640 1 -- 172.21.15.103:0/3199387756 >> 172.21.15.103:0/3199387756 conn(0x7eff380fc7d0 msgr2=0x7eff380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:40.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.831+0000 7eff3f107640 1 -- 172.21.15.103:0/3199387756 shutdown_connections 2024-09-19T07:04:40.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.831+0000 7eff3f107640 1 -- 172.21.15.103:0/3199387756 wait complete. 2024-09-19T07:04:40.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.831+0000 7eff3f107640 1 Processor -- start 2024-09-19T07:04:40.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.831+0000 7eff3f107640 1 -- start start 2024-09-19T07:04:40.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff3f107640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff38100e20 0x7eff38103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff3f107640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff38102570 0x7eff38103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff3f107640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff38102ef0 0x7eff381041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff3f107640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7eff38110f80 con 0x7eff38102570 2024-09-19T07:04:40.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff3f107640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff38110e00 con 0x7eff38100e20 2024-09-19T07:04:40.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff3f107640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7eff38111100 con 0x7eff38102ef0 2024-09-19T07:04:40.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff37fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff38102570 0x7eff38103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff37fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff38102570 0x7eff38103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56678/0 (socket says 172.21.15.103:56678) 2024-09-19T07:04:40.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff37fff640 1 -- 172.21.15.103:0/3624981842 learned_addr learned my addr 172.21.15.103:0/3624981842 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:40.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff3cea3640 1 --2- 172.21.15.103:0/3624981842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff38100e20 0x7eff38103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff3d6a4640 1 --2- 172.21.15.103:0/3624981842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff38102ef0 0x7eff381041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff37fff640 1 -- 172.21.15.103:0/3624981842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff38102ef0 msgr2=0x7eff381041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:40.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff37fff640 1 --2- 172.21.15.103:0/3624981842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff38102ef0 0x7eff381041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff37fff640 1 -- 172.21.15.103:0/3624981842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff38100e20 msgr2=0x7eff38103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:40.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff37fff640 1 --2- 172.21.15.103:0/3624981842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff38100e20 0x7eff38103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff37fff640 1 -- 172.21.15.103:0/3624981842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff381a6950 con 0x7eff38102570 2024-09-19T07:04:40.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff37fff640 1 --2- 172.21.15.103:0/3624981842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff38102570 0x7eff38103c90 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7eff2000cce0 tx=0x7eff2000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:40.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff35ffb640 1 -- 172.21.15.103:0/3624981842 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20004b20 con 0x7eff38102570 2024-09-19T07:04:40.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff35ffb640 1 -- 172.21.15.103:0/3624981842 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff20004cc0 con 0x7eff38102570 2024-09-19T07:04:40.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.832+0000 7eff35ffb640 1 -- 172.21.15.103:0/3624981842 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2000f7d0 con 0x7eff38102570 2024-09-19T07:04:40.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.833+0000 7eff3f107640 1 -- 172.21.15.103:0/3624981842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff381a6c40 con 0x7eff38102570 2024-09-19T07:04:40.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.833+0000 7eff3f107640 1 -- 172.21.15.103:0/3624981842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff381a7180 con 0x7eff38102570 2024-09-19T07:04:40.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.835+0000 7eff35ffb640 1 -- 172.21.15.103:0/3624981842 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7eff2000f970 con 0x7eff38102570 2024-09-19T07:04:40.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.835+0000 7eff414e0740 1 Processor -- start 2024-09-19T07:04:40.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.835+0000 7eff414e0740 1 -- start start 2024-09-19T07:04:40.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.835+0000 7eff35ffb640 1 --2- 172.21.15.103:0/3624981842 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff240a3430 0x7eff240a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.835+0000 7eff414e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee82880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.835+0000 7eff414e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee82dc0 0x55ed9ee8d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.835+0000 7eff414e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee8d680 0x55ed9ee8fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff414e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ed9ede82e0 con 0x55ed9ee824a0 2024-09-19T07:04:40.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff414e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ed9ed8fce0 con 0x55ed9ee8d680 2024-09-19T07:04:40.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff414e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ed9ede42a0 con 0x55ed9ee82dc0 2024-09-19T07:04:40.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff35ffb640 1 -- 172.21.15.103:0/3624981842 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7eff200cfde0 con 0x7eff38102570 2024-09-19T07:04:40.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff3cea3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee82880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff3d6a4640 1 --2- 172.21.15.103:0/3624981842 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff240a3430 0x7eff240a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff3cea3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee82880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56684/0 (socket says 172.21.15.103:56684) 2024-09-19T07:04:40.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff3cea3640 1 -- 172.21.15.103:0/2693597060 learned_addr learned my addr 172.21.15.103:0/2693597060 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:40.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff37fff640 1 --2- 172.21.15.103:0/2693597060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee82dc0 0x55ed9ee8d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff3d6a4640 1 --2- 172.21.15.103:0/2693597060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee8d680 0x55ed9ee8fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff3cea3640 1 -- 172.21.15.103:0/2693597060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee82dc0 msgr2=0x55ed9ee8d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:40.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff3cea3640 1 --2- 172.21.15.103:0/2693597060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee82dc0 0x55ed9ee8d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff3cea3640 1 -- 172.21.15.103:0/2693597060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee8d680 msgr2=0x55ed9ee8fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:40.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff3cea3640 1 --2- 172.21.15.103:0/2693597060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee8d680 0x55ed9ee8fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff3cea3640 1 -- 172.21.15.103:0/2693597060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed9ed74440 con 0x55ed9ee824a0 2024-09-19T07:04:40.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff3d6a4640 1 --2- 172.21.15.103:0/3624981842 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff240a3430 0x7eff240a58f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7eff28005fd0 tx=0x7eff2800b710 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:40.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff3cea3640 1 --2- 172.21.15.103:0/2693597060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee82880 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7eff30009900 tx=0x7eff30004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d98a73f2bb636de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:40.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.836+0000 7eff2effd640 1 -- 172.21.15.103:0/2693597060 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30035030 con 0x55ed9ee824a0 2024-09-19T07:04:40.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.837+0000 7eff2effd640 1 -- 172.21.15.103:0/2693597060 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff30040070 con 0x55ed9ee824a0 2024-09-19T07:04:40.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.837+0000 7eff2effd640 1 -- 172.21.15.103:0/2693597060 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30031b30 con 0x55ed9ee824a0 2024-09-19T07:04:40.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.837+0000 7eff414e0740 1 -- 172.21.15.103:0/2693597060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 msgr2=0x55ed9ee82880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:40.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.837+0000 7eff414e0740 1 --2- 172.21.15.103:0/2693597060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee82880 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7eff30009900 tx=0x7eff30004200 comp rx=0 tx=0).stop 2024-09-19T07:04:40.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.837+0000 7eff414e0740 1 -- 172.21.15.103:0/2693597060 shutdown_connections 2024-09-19T07:04:40.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.837+0000 7eff414e0740 1 --2- 172.21.15.103:0/2693597060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee8d680 0x55ed9ee8fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.837+0000 7eff414e0740 1 --2- 172.21.15.103:0/2693597060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee82dc0 0x55ed9ee8d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.837+0000 7eff414e0740 1 --2- 172.21.15.103:0/2693597060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee82880 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.837+0000 7eff414e0740 1 -- 172.21.15.103:0/2693597060 >> 172.21.15.103:0/2693597060 conn(0x55ed9ee7ea30 msgr2=0x55ed9ee7ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:40.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.837+0000 7eff414e0740 1 -- 172.21.15.103:0/2693597060 shutdown_connections 2024-09-19T07:04:40.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.837+0000 7eff414e0740 1 -- 172.21.15.103:0/2693597060 wait complete. 2024-09-19T07:04:40.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.838+0000 7eff414e0740 1 -- start start 2024-09-19T07:04:40.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.838+0000 7eff414e0740 1 Processor -- start 2024-09-19T07:04:40.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.838+0000 7eff414e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee8cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.838+0000 7eff414e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee82dc0 0x55ed9ee8a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.838+0000 7eff414e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee8d680 0x55ed9ee8a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.838+0000 7eff414e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ed9ee90250 con 0x55ed9ee824a0 2024-09-19T07:04:40.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.838+0000 7eff414e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ed9eda3050 con 0x55ed9ee8d680 2024-09-19T07:04:40.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.838+0000 7eff414e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ed9ee903d0 con 0x55ed9ee82dc0 2024-09-19T07:04:40.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.838+0000 7eff414e0740 10 client.0 _ll_register_callbacks cb 0x55ed9ede3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:40.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.838+0000 7eff3cea3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee8cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.838+0000 7eff3cea3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee8cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56686/0 (socket says 172.21.15.103:56686) 2024-09-19T07:04:40.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.838+0000 7eff3cea3640 1 -- 172.21.15.103:0/1731773200 learned_addr learned my addr 172.21.15.103:0/1731773200 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:40.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.838+0000 7eff37fff640 1 --2- 172.21.15.103:0/1731773200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee82dc0 0x55ed9ee8a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff3d6a4640 1 --2- 172.21.15.103:0/1731773200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee8d680 0x55ed9ee8a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff3cea3640 1 -- 172.21.15.103:0/1731773200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee82dc0 msgr2=0x55ed9ee8a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:40.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff3cea3640 1 --2- 172.21.15.103:0/1731773200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee82dc0 0x55ed9ee8a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff3cea3640 1 -- 172.21.15.103:0/1731773200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee8d680 msgr2=0x55ed9ee8a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:40.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff3cea3640 1 --2- 172.21.15.103:0/1731773200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee8d680 0x55ed9ee8a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff3cea3640 1 -- 172.21.15.103:0/1731773200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed9eb593c0 con 0x55ed9ee824a0 2024-09-19T07:04:40.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff3d6a4640 1 --2- 172.21.15.103:0/1731773200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee8d680 0x55ed9ee8a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:40.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff3cea3640 1 --2- 172.21.15.103:0/1731773200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee8cf70 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7eff30031810 tx=0x7eff30003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebaeb742884bc49c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:40.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff17fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:40.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff17fff640 1 -- 172.21.15.103:0/1731773200 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30035070 con 0x55ed9ee824a0 2024-09-19T07:04:40.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff17fff640 1 -- 172.21.15.103:0/1731773200 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7eff300037f0 con 0x55ed9ee824a0 2024-09-19T07:04:40.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff414e0740 1 -- 172.21.15.103:0/1731773200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff30040070 con 0x55ed9ee824a0 2024-09-19T07:04:40.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff17fff640 1 -- 172.21.15.103:0/1731773200 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff30009520 con 0x55ed9ee824a0 2024-09-19T07:04:40.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff414e0740 10 client.20742 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:40.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff414e0740 1 -- 172.21.15.103:0/1731773200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed9ee9bbc0 con 0x55ed9ee824a0 2024-09-19T07:04:40.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff17fff640 1 -- 172.21.15.103:0/1731773200 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff30040070 con 0x55ed9ee824a0 2024-09-19T07:04:40.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.839+0000 7eff17fff640 1 -- 172.21.15.103:0/1731773200 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7eff300459e0 con 0x55ed9ee824a0 2024-09-19T07:04:40.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.840+0000 7eff414e0740 10 client.20742 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:40.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.840+0000 7eff414e0740 10 client.20742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:40.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.840+0000 7eff414e0740 20 client.20742 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:40.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.840+0000 7eff414e0740 1 --2- 172.21.15.103:0/1731773200 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ed9ee9cf10 0x55ed9ee9f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.840+0000 7eff414e0740 4 client.20742 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:40.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.840+0000 7eff414e0740 1 -- 172.21.15.103:0/1731773200 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed9edf5850 con 0x55ed9ee9cf10 2024-09-19T07:04:40.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.840+0000 7eff37fff640 1 --2- 172.21.15.103:0/1731773200 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ed9ee9cf10 0x55ed9ee9f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.840+0000 7eff37fff640 1 --2- 172.21.15.103:0/1731773200 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ed9ee9cf10 0x55ed9ee9f300 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d33321b6370b8b62 server_cookie=13b1f90dc3e47de7 in_seq=0 out_seq=0 2024-09-19T07:04:40.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.840+0000 7eff17fff640 10 client.20742 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:40.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.841+0000 7eff17fff640 1 -- 172.21.15.103:0/1731773200 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed9edf5850 con 0x55ed9ee9cf10 2024-09-19T07:04:40.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.841+0000 7eff17fff640 10 client.20742 handle_command_reply: tid=0 2024-09-19T07:04:40.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.841+0000 7eff414e0740 1 client.20742 shutdown 2024-09-19T07:04:40.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.841+0000 7eff414e0740 1 -- 172.21.15.103:0/1731773200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 msgr2=0x55ed9ee8cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:40.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.841+0000 7eff414e0740 1 --2- 172.21.15.103:0/1731773200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee8cf70 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7eff30031810 tx=0x7eff30003fd0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.841+0000 7eff414e0740 1 -- 172.21.15.103:0/1731773200 shutdown_connections 2024-09-19T07:04:40.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.841+0000 7eff414e0740 1 --2- 172.21.15.103:0/1731773200 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ed9ee9cf10 0x55ed9ee9f300 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.841+0000 7eff414e0740 1 --2- 172.21.15.103:0/1731773200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee8d680 0x55ed9ee8a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.841+0000 7eff414e0740 1 --2- 172.21.15.103:0/1731773200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee82dc0 0x55ed9ee8a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.841+0000 7eff414e0740 1 --2- 172.21.15.103:0/1731773200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee8cf70 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.841+0000 7eff414e0740 1 -- 172.21.15.103:0/1731773200 >> 172.21.15.103:0/1731773200 conn(0x55ed9ee7ea30 msgr2=0x55ed9ee84710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:40.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.841+0000 7eff17fff640 0 client.20742 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:40.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.842+0000 7eff414e0740 1 -- 172.21.15.103:0/1731773200 shutdown_connections 2024-09-19T07:04:40.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.842+0000 7eff414e0740 1 -- 172.21.15.103:0/1731773200 wait complete. 2024-09-19T07:04:40.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.842+0000 7eff414e0740 20 client.20742 trim_cache size 0 max 16384 2024-09-19T07:04:40.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.858+0000 7eff414e0740 1 Processor -- start 2024-09-19T07:04:40.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.858+0000 7eff414e0740 1 -- start start 2024-09-19T07:04:40.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.858+0000 7eff414e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee88af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff414e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee82dc0 0x55ed9ee7da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff414e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee8d680 0x55ed9eea1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff414e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7eff381afac0 con 0x55ed9ee824a0 2024-09-19T07:04:40.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff414e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff381b1600 con 0x55ed9ee82dc0 2024-09-19T07:04:40.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff414e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7eff381b0190 con 0x55ed9ee8d680 2024-09-19T07:04:40.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff3cea3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee88af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff3cea3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee88af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56702/0 (socket says 172.21.15.103:56702) 2024-09-19T07:04:40.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff3cea3640 1 -- 172.21.15.103:0/3195257570 learned_addr learned my addr 172.21.15.103:0/3195257570 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:40.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff3cea3640 1 -- 172.21.15.103:0/3195257570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee8d680 msgr2=0x55ed9eea1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:40.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff3d6a4640 1 --2- 172.21.15.103:0/3195257570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee8d680 0x55ed9eea1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff37fff640 1 --2- 172.21.15.103:0/3195257570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee82dc0 0x55ed9ee7da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff3cea3640 1 --2- 172.21.15.103:0/3195257570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee8d680 0x55ed9eea1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff3cea3640 1 -- 172.21.15.103:0/3195257570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee82dc0 msgr2=0x55ed9ee7da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:40.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff3cea3640 1 --2- 172.21.15.103:0/3195257570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee82dc0 0x55ed9ee7da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff3cea3640 1 -- 172.21.15.103:0/3195257570 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed9ee891d0 con 0x55ed9ee824a0 2024-09-19T07:04:40.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff37fff640 1 --2- 172.21.15.103:0/3195257570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee82dc0 0x55ed9ee7da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:40.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff3cea3640 1 --2- 172.21.15.103:0/3195257570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee88af0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7eff30009850 tx=0x7eff30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd575e09b0a4e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:40.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff2e7fc640 1 -- 172.21.15.103:0/3195257570 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30035070 con 0x55ed9ee824a0 2024-09-19T07:04:40.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff2e7fc640 1 -- 172.21.15.103:0/3195257570 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff30040040 con 0x55ed9ee824a0 2024-09-19T07:04:40.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.859+0000 7eff2e7fc640 1 -- 172.21.15.103:0/3195257570 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30005280 con 0x55ed9ee824a0 2024-09-19T07:04:40.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.860+0000 7eff414e0740 1 -- 172.21.15.103:0/3195257570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 msgr2=0x55ed9ee88af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:40.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.860+0000 7eff414e0740 1 --2- 172.21.15.103:0/3195257570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee88af0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7eff30009850 tx=0x7eff30000bc0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.860+0000 7eff414e0740 1 -- 172.21.15.103:0/3195257570 shutdown_connections 2024-09-19T07:04:40.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.860+0000 7eff414e0740 1 --2- 172.21.15.103:0/3195257570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee8d680 0x55ed9eea1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.860+0000 7eff414e0740 1 --2- 172.21.15.103:0/3195257570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee82dc0 0x55ed9ee7da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.860+0000 7eff414e0740 1 --2- 172.21.15.103:0/3195257570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee824a0 0x55ed9ee88af0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.860+0000 7eff414e0740 1 -- 172.21.15.103:0/3195257570 >> 172.21.15.103:0/3195257570 conn(0x55ed9ee7ea30 msgr2=0x55ed9ee7d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:40.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.860+0000 7eff414e0740 1 -- 172.21.15.103:0/3195257570 shutdown_connections 2024-09-19T07:04:40.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.860+0000 7eff414e0740 1 -- 172.21.15.103:0/3195257570 wait complete. 2024-09-19T07:04:40.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.860+0000 7eff414e0740 1 -- start start 2024-09-19T07:04:40.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.860+0000 7eff414e0740 1 Processor -- start 2024-09-19T07:04:40.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff414e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee824a0 0x55ed9eea16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff414e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee82dc0 0x55ed9eea1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff414e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee8d680 0x55ed9eeff900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff414e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7eff381b0f30 con 0x55ed9ee82dc0 2024-09-19T07:04:40.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff414e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff381b0860 con 0x55ed9ee8d680 2024-09-19T07:04:40.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff414e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7eff381adac0 con 0x55ed9ee824a0 2024-09-19T07:04:40.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff414e0740 10 client.0 _ll_register_callbacks cb 0x7eff20005840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:40.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff37fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee82dc0 0x55ed9eea1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff37fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee82dc0 0x55ed9eea1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56716/0 (socket says 172.21.15.103:56716) 2024-09-19T07:04:40.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff37fff640 1 -- 172.21.15.103:0/257517043 learned_addr learned my addr 172.21.15.103:0/257517043 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:40.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff3cea3640 1 --2- 172.21.15.103:0/257517043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee824a0 0x55ed9eea16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff3d6a4640 1 --2- 172.21.15.103:0/257517043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee8d680 0x55ed9eeff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff37fff640 1 -- 172.21.15.103:0/257517043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee824a0 msgr2=0x55ed9eea16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:40.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff37fff640 1 --2- 172.21.15.103:0/257517043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee824a0 0x55ed9eea16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff37fff640 1 -- 172.21.15.103:0/257517043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee8d680 msgr2=0x55ed9eeff900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:40.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff37fff640 1 --2- 172.21.15.103:0/257517043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee8d680 0x55ed9eeff900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff37fff640 1 -- 172.21.15.103:0/257517043 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed9eefffa0 con 0x55ed9ee82dc0 2024-09-19T07:04:40.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff3cea3640 1 --2- 172.21.15.103:0/257517043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee824a0 0x55ed9eea16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:40.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff37fff640 1 --2- 172.21.15.103:0/257517043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee82dc0 0x55ed9eea1be0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7eff2009bd40 tx=0x7eff2000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f843fb9a8fff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:40.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff17fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:40.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.861+0000 7eff17fff640 1 -- 172.21.15.103:0/257517043 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2001d040 con 0x55ed9ee82dc0 2024-09-19T07:04:40.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.862+0000 7eff17fff640 1 -- 172.21.15.103:0/257517043 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7eff2009a340 con 0x55ed9ee82dc0 2024-09-19T07:04:40.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.862+0000 7eff414e0740 1 -- 172.21.15.103:0/257517043 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff30040040 con 0x55ed9ee82dc0 2024-09-19T07:04:40.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.862+0000 7eff17fff640 1 -- 172.21.15.103:0/257517043 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff2007f050 con 0x55ed9ee82dc0 2024-09-19T07:04:40.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.862+0000 7eff414e0740 10 client.20748 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:40.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.862+0000 7eff414e0740 1 -- 172.21.15.103:0/257517043 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed9ef072f0 con 0x55ed9ee82dc0 2024-09-19T07:04:40.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.862+0000 7eff17fff640 1 -- 172.21.15.103:0/257517043 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff30040040 con 0x55ed9ee82dc0 2024-09-19T07:04:40.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.862+0000 7eff17fff640 1 -- 172.21.15.103:0/257517043 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7eff200cd730 con 0x55ed9ee82dc0 2024-09-19T07:04:40.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.862+0000 7eff414e0740 10 client.20748 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:40.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.862+0000 7eff414e0740 10 client.20748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:40.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.862+0000 7eff414e0740 20 client.20748 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:40.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.862+0000 7eff414e0740 1 --2- 172.21.15.103:0/257517043 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ed9ef07a40 0x55ed9ef09e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:40.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.862+0000 7eff414e0740 4 client.20748 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:04:40.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.862+0000 7eff414e0740 1 -- 172.21.15.103:0/257517043 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ed9edefd60 con 0x55ed9ef07a40 2024-09-19T07:04:40.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.862+0000 7eff3cea3640 1 --2- 172.21.15.103:0/257517043 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ed9ef07a40 0x55ed9ef09e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:40.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.863+0000 7eff17fff640 10 client.20748 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:40.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.863+0000 7eff3cea3640 1 --2- 172.21.15.103:0/257517043 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ed9ef07a40 0x55ed9ef09e60 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=418e4b8fc3cd8fd2 server_cookie=8b85e1d257445902 in_seq=0 out_seq=0 2024-09-19T07:04:40.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.863+0000 7eff17fff640 1 -- 172.21.15.103:0/257517043 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ed9edefd60 con 0x55ed9ef07a40 2024-09-19T07:04:40.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.863+0000 7eff17fff640 10 client.20748 handle_command_reply: tid=0 2024-09-19T07:04:40.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.863+0000 7eff414e0740 1 client.20748 shutdown 2024-09-19T07:04:40.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.863+0000 7eff414e0740 1 -- 172.21.15.103:0/257517043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee82dc0 msgr2=0x55ed9eea1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:40.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.863+0000 7eff414e0740 1 --2- 172.21.15.103:0/257517043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee82dc0 0x55ed9eea1be0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7eff2009bd40 tx=0x7eff2000e440 comp rx=0 tx=0).stop 2024-09-19T07:04:40.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.863+0000 7eff414e0740 1 -- 172.21.15.103:0/257517043 shutdown_connections 2024-09-19T07:04:40.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.863+0000 7eff414e0740 1 --2- 172.21.15.103:0/257517043 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55ed9ef07a40 0x55ed9ef09e60 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.863+0000 7eff414e0740 1 --2- 172.21.15.103:0/257517043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed9ee8d680 0x55ed9eeff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.863+0000 7eff414e0740 1 --2- 172.21.15.103:0/257517043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed9ee82dc0 0x55ed9eea1be0 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.863+0000 7eff414e0740 1 --2- 172.21.15.103:0/257517043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed9ee824a0 0x55ed9eea16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.863+0000 7eff414e0740 1 -- 172.21.15.103:0/257517043 >> 172.21.15.103:0/257517043 conn(0x55ed9ee7ea30 msgr2=0x55ed9ee7d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:40.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.863+0000 7eff17fff640 0 client.20748 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:40.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.864+0000 7eff414e0740 1 -- 172.21.15.103:0/257517043 shutdown_connections 2024-09-19T07:04:40.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.864+0000 7eff414e0740 1 -- 172.21.15.103:0/257517043 wait complete. 2024-09-19T07:04:40.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.864+0000 7eff414e0740 20 client.20748 trim_cache size 0 max 16384 2024-09-19T07:04:40.872 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:04:40.872 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:04:40.872 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:04:40.872 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:04:40.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.864+0000 7eff3f107640 1 -- 172.21.15.103:0/3624981842 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff240a3430 msgr2=0x7eff240a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:40.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.864+0000 7eff3f107640 1 --2- 172.21.15.103:0/3624981842 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff240a3430 0x7eff240a58f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7eff28005fd0 tx=0x7eff2800b710 comp rx=0 tx=0).stop 2024-09-19T07:04:40.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.864+0000 7eff3f107640 1 -- 172.21.15.103:0/3624981842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff38102570 msgr2=0x7eff38103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:40.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.864+0000 7eff3f107640 1 --2- 172.21.15.103:0/3624981842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff38102570 0x7eff38103c90 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7eff2000cce0 tx=0x7eff2000a620 comp rx=0 tx=0).stop 2024-09-19T07:04:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.864+0000 7eff3f107640 1 -- 172.21.15.103:0/3624981842 shutdown_connections 2024-09-19T07:04:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.864+0000 7eff3f107640 1 --2- 172.21.15.103:0/3624981842 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7eff240a3430 0x7eff240a58f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.864+0000 7eff3f107640 1 --2- 172.21.15.103:0/3624981842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff38102ef0 0x7eff381041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.864+0000 7eff3f107640 1 --2- 172.21.15.103:0/3624981842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff38102570 0x7eff38103c90 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.864+0000 7eff3f107640 1 --2- 172.21.15.103:0/3624981842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff38100e20 0x7eff38103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.864+0000 7eff3f107640 1 -- 172.21.15.103:0/3624981842 >> 172.21.15.103:0/3624981842 conn(0x7eff380fc7d0 msgr2=0x7eff3806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.864+0000 7eff3f107640 1 -- 172.21.15.103:0/3624981842 shutdown_connections 2024-09-19T07:04:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:40.864+0000 7eff3f107640 1 -- 172.21.15.103:0/3624981842 wait complete. 2024-09-19T07:04:41.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:41 smithi174 ceph-mon[35637]: pgmap v484: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 16 op/s 2024-09-19T07:04:41.608 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:41 smithi103 ceph-mon[31158]: pgmap v484: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 16 op/s 2024-09-19T07:04:41.638 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:04:41.638 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:04:41.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:41 smithi134 ceph-mon[35356]: pgmap v484: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 16 op/s 2024-09-19T07:04:41.888 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:42.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.381+0000 7f19e2c70640 1 -- 172.21.15.103:0/677064209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19dc102790 msgr2=0x7f19dc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:42.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.381+0000 7f19e2c70640 1 --2- 172.21.15.103:0/677064209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19dc102790 0x7f19dc1067e0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009900 tx=0x7f19d002a410 comp rx=0 tx=0).stop 2024-09-19T07:04:42.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.381+0000 7f19e2c70640 1 -- 172.21.15.103:0/677064209 shutdown_connections 2024-09-19T07:04:42.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.381+0000 7f19e2c70640 1 --2- 172.21.15.103:0/677064209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19dc106f10 0x7f19dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:42.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.381+0000 7f19e2c70640 1 --2- 172.21.15.103:0/677064209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19dc102790 0x7f19dc1067e0 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:42.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.381+0000 7f19e2c70640 1 --2- 172.21.15.103:0/677064209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19dc101de0 0x7f19dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:42.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.381+0000 7f19e2c70640 1 -- 172.21.15.103:0/677064209 >> 172.21.15.103:0/677064209 conn(0x7f19dc0fd8c0 msgr2=0x7f19dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:42.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.381+0000 7f19e2c70640 1 -- 172.21.15.103:0/677064209 shutdown_connections 2024-09-19T07:04:42.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.382+0000 7f19e2c70640 1 -- 172.21.15.103:0/677064209 wait complete. 2024-09-19T07:04:42.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.382+0000 7f19e2c70640 1 Processor -- start 2024-09-19T07:04:42.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.382+0000 7f19e2c70640 1 -- start start 2024-09-19T07:04:42.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.382+0000 7f19e2c70640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19dc101de0 0x7f19dc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:42.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.382+0000 7f19e2c70640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19dc102790 0x7f19dc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:42.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.382+0000 7f19e2c70640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19dc106f10 0x7f19dc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:42.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.382+0000 7f19e2c70640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc111f70 con 0x7f19dc101de0 2024-09-19T07:04:42.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.382+0000 7f19e2c70640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc111df0 con 0x7f19dc106f10 2024-09-19T07:04:42.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.382+0000 7f19e2c70640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc1120f0 con 0x7f19dc102790 2024-09-19T07:04:42.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.382+0000 7f19e0a0c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19dc101de0 0x7f19dc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:42.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.382+0000 7f19e0a0c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19dc101de0 0x7f19dc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56740/0 (socket says 172.21.15.103:56740) 2024-09-19T07:04:42.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.382+0000 7f19e0a0c640 1 -- 172.21.15.103:0/2645951325 learned_addr learned my addr 172.21.15.103:0/2645951325 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:42.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.383+0000 7f19dbfff640 1 --2- 172.21.15.103:0/2645951325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19dc102790 0x7f19dc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:42.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.383+0000 7f19e120d640 1 --2- 172.21.15.103:0/2645951325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19dc106f10 0x7f19dc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:42.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.383+0000 7f19e0a0c640 1 -- 172.21.15.103:0/2645951325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19dc102790 msgr2=0x7f19dc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:42.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.383+0000 7f19e0a0c640 1 --2- 172.21.15.103:0/2645951325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19dc102790 0x7f19dc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:42.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.383+0000 7f19e0a0c640 1 -- 172.21.15.103:0/2645951325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19dc106f10 msgr2=0x7f19dc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:42.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.383+0000 7f19e0a0c640 1 --2- 172.21.15.103:0/2645951325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19dc106f10 0x7f19dc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:42.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.383+0000 7f19e0a0c640 1 -- 172.21.15.103:0/2645951325 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc1a6930 con 0x7f19dc101de0 2024-09-19T07:04:42.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.383+0000 7f19e0a0c640 1 --2- 172.21.15.103:0/2645951325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19dc101de0 0x7f19dc0744f0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f19cc00c970 tx=0x7f19cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:42.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.383+0000 7f19d9ffb640 1 -- 172.21.15.103:0/2645951325 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc007bb0 con 0x7f19dc101de0 2024-09-19T07:04:42.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.383+0000 7f19d9ffb640 1 -- 172.21.15.103:0/2645951325 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19cc007d50 con 0x7f19dc101de0 2024-09-19T07:04:42.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.383+0000 7f19d9ffb640 1 -- 172.21.15.103:0/2645951325 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc00f450 con 0x7f19dc101de0 2024-09-19T07:04:42.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.383+0000 7f19e2c70640 1 -- 172.21.15.103:0/2645951325 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19dc1a6c20 con 0x7f19dc101de0 2024-09-19T07:04:42.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.383+0000 7f19e2c70640 1 -- 172.21.15.103:0/2645951325 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19dc1a70e0 con 0x7f19dc101de0 2024-09-19T07:04:42.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.384+0000 7f19e2c70640 1 -- 172.21.15.103:0/2645951325 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19dc1021c0 con 0x7f19dc101de0 2024-09-19T07:04:42.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.385+0000 7f19d9ffb640 1 -- 172.21.15.103:0/2645951325 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f19cc005020 con 0x7f19dc101de0 2024-09-19T07:04:42.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.386+0000 7f19d9ffb640 1 --2- 172.21.15.103:0/2645951325 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19c40a33b0 0x7f19c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:42.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.386+0000 7f19dbfff640 1 --2- 172.21.15.103:0/2645951325 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19c40a33b0 0x7f19c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:42.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.387+0000 7f19d9ffb640 1 -- 172.21.15.103:0/2645951325 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f19cc0cfb30 con 0x7f19dc101de0 2024-09-19T07:04:42.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.387+0000 7f19dbfff640 1 --2- 172.21.15.103:0/2645951325 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19c40a33b0 0x7f19c40a5870 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f19d0000c00 tx=0x7f19d00058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:42.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.391+0000 7f19d9ffb640 1 -- 172.21.15.103:0/2645951325 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f19cc014030 con 0x7f19dc101de0 2024-09-19T07:04:42.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.598+0000 7f19e2c70640 1 -- 172.21.15.103:0/2645951325 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19dc02d7c0 con 0x7f19dc101de0 2024-09-19T07:04:42.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.599+0000 7f19d9ffb640 1 -- 172.21.15.103:0/2645951325 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f19cc09c4f0 con 0x7f19dc101de0 2024-09-19T07:04:42.601 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:04:42.601 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:04:42.601 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:04:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.601+0000 7f19e2c70640 1 -- 172.21.15.103:0/2645951325 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19c40a33b0 msgr2=0x7f19c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.601+0000 7f19e2c70640 1 --2- 172.21.15.103:0/2645951325 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19c40a33b0 0x7f19c40a5870 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f19d0000c00 tx=0x7f19d00058d0 comp rx=0 tx=0).stop 2024-09-19T07:04:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.601+0000 7f19e2c70640 1 -- 172.21.15.103:0/2645951325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19dc101de0 msgr2=0x7f19dc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.601+0000 7f19e2c70640 1 --2- 172.21.15.103:0/2645951325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19dc101de0 0x7f19dc0744f0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f19cc00c970 tx=0x7f19cc00ce30 comp rx=0 tx=0).stop 2024-09-19T07:04:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.601+0000 7f19e2c70640 1 -- 172.21.15.103:0/2645951325 shutdown_connections 2024-09-19T07:04:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.601+0000 7f19e2c70640 1 --2- 172.21.15.103:0/2645951325 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f19c40a33b0 0x7f19c40a5870 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.601+0000 7f19e2c70640 1 --2- 172.21.15.103:0/2645951325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19dc106f10 0x7f19dc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.601+0000 7f19e2c70640 1 --2- 172.21.15.103:0/2645951325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19dc102790 0x7f19dc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.601+0000 7f19e2c70640 1 --2- 172.21.15.103:0/2645951325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19dc101de0 0x7f19dc0744f0 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.601+0000 7f19e2c70640 1 -- 172.21.15.103:0/2645951325 >> 172.21.15.103:0/2645951325 conn(0x7f19dc0fd8c0 msgr2=0x7f19dc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.601+0000 7f19e2c70640 1 -- 172.21.15.103:0/2645951325 shutdown_connections 2024-09-19T07:04:42.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:42.601+0000 7f19e2c70640 1 -- 172.21.15.103:0/2645951325 wait complete. 2024-09-19T07:04:43.272 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:04:43.518 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:43.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:43 smithi174 ceph-mon[35637]: pgmap v485: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-09-19T07:04:43.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:43 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2645951325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:43.635 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:43 smithi103 ceph-mon[31158]: pgmap v485: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-09-19T07:04:43.635 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:43 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2645951325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:43.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:43 smithi134 ceph-mon[35356]: pgmap v485: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-09-19T07:04:43.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:43 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2645951325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:43.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.944+0000 7fe4eb27c640 1 -- 172.21.15.103:0/2017217687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4e410c1a0 msgr2=0x7fe4e410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:43.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.944+0000 7fe4eb27c640 1 --2- 172.21.15.103:0/2017217687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4e410c1a0 0x7fe4e410e610 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009ca0 tx=0x7fe4dc02a950 comp rx=0 tx=0).stop 2024-09-19T07:04:43.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.944+0000 7fe4eb27c640 1 -- 172.21.15.103:0/2017217687 shutdown_connections 2024-09-19T07:04:43.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.944+0000 7fe4eb27c640 1 --2- 172.21.15.103:0/2017217687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4e410c1a0 0x7fe4e410e610 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.944+0000 7fe4eb27c640 1 --2- 172.21.15.103:0/2017217687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4e4103720 0x7fe4e410bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.944+0000 7fe4eb27c640 1 --2- 172.21.15.103:0/2017217687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e4102e00 0x7fe4e41031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.944+0000 7fe4eb27c640 1 -- 172.21.15.103:0/2017217687 >> 172.21.15.103:0/2017217687 conn(0x7fe4e40fc7a0 msgr2=0x7fe4e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.944+0000 7fe4eb27c640 1 -- 172.21.15.103:0/2017217687 shutdown_connections 2024-09-19T07:04:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.944+0000 7fe4eb27c640 1 -- 172.21.15.103:0/2017217687 wait complete. 2024-09-19T07:04:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4eb27c640 1 Processor -- start 2024-09-19T07:04:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4eb27c640 1 -- start start 2024-09-19T07:04:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4eb27c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e4102e00 0x7fe4e4100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4eb27c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4e4103720 0x7fe4e4100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4eb27c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4e410c1a0 0x7fe4e41014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4eb27c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e4112030 con 0x7fe4e410c1a0 2024-09-19T07:04:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4eb27c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e4111eb0 con 0x7fe4e4102e00 2024-09-19T07:04:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4eb27c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41121b0 con 0x7fe4e4103720 2024-09-19T07:04:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4e9819640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4e410c1a0 0x7fe4e41014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4e9819640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4e410c1a0 0x7fe4e41014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56768/0 (socket says 172.21.15.103:56768) 2024-09-19T07:04:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4e9819640 1 -- 172.21.15.103:0/2172171437 learned_addr learned my addr 172.21.15.103:0/2172171437 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4e9018640 1 --2- 172.21.15.103:0/2172171437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e4102e00 0x7fe4e4100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4e9819640 1 -- 172.21.15.103:0/2172171437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4e4103720 msgr2=0x7fe4e4100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4e9819640 1 --2- 172.21.15.103:0/2172171437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4e4103720 0x7fe4e4100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4e9819640 1 -- 172.21.15.103:0/2172171437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e4102e00 msgr2=0x7fe4e4100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4e9819640 1 --2- 172.21.15.103:0/2172171437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e4102e00 0x7fe4e4100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.945+0000 7fe4e9819640 1 -- 172.21.15.103:0/2172171437 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e41a69f0 con 0x7fe4e410c1a0 2024-09-19T07:04:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.946+0000 7fe4e9018640 1 --2- 172.21.15.103:0/2172171437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e4102e00 0x7fe4e4100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.946+0000 7fe4e9819640 1 --2- 172.21.15.103:0/2172171437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4e410c1a0 0x7fe4e41014e0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc006010 tx=0x7fe4dc02f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.946+0000 7fe4da7fc640 1 -- 172.21.15.103:0/2172171437 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc034040 con 0x7fe4e410c1a0 2024-09-19T07:04:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.946+0000 7fe4da7fc640 1 -- 172.21.15.103:0/2172171437 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc003e00 con 0x7fe4e410c1a0 2024-09-19T07:04:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.946+0000 7fe4da7fc640 1 -- 172.21.15.103:0/2172171437 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc0456e0 con 0x7fe4e410c1a0 2024-09-19T07:04:43.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.946+0000 7fe4eb27c640 1 -- 172.21.15.103:0/2172171437 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e41a6c80 con 0x7fe4e410c1a0 2024-09-19T07:04:43.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.946+0000 7fe4eb27c640 1 -- 172.21.15.103:0/2172171437 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e41a71b0 con 0x7fe4e410c1a0 2024-09-19T07:04:43.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.948+0000 7fe4da7fc640 1 -- 172.21.15.103:0/2172171437 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fe4dc037020 con 0x7fe4e410c1a0 2024-09-19T07:04:43.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.948+0000 7fe4ed655740 1 Processor -- start 2024-09-19T07:04:43.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4ed655740 1 -- start start 2024-09-19T07:04:43.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4ed655740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91447b40 0x558a91446090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4ed655740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a914465d0 0x558a91446a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4ed655740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a91451370 0x558a91453760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4ed655740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558a913ac2f0 con 0x558a91447b40 2024-09-19T07:04:43.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4ed655740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558a91353cf0 con 0x558a91451370 2024-09-19T07:04:43.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4ed655740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558a913a82b0 con 0x558a914465d0 2024-09-19T07:04:43.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4da7fc640 1 --2- 172.21.15.103:0/2172171437 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe4d00a3430 0x7fe4d00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4e8817640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91447b40 0x558a91446090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:43.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4e8817640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91447b40 0x558a91446090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56772/0 (socket says 172.21.15.103:56772) 2024-09-19T07:04:43.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4e8817640 1 -- 172.21.15.103:0/3281109304 learned_addr learned my addr 172.21.15.103:0/3281109304 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:43.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4e9819640 1 --2- 172.21.15.103:0/2172171437 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe4d00a3430 0x7fe4d00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:43.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4e9018640 1 --2- 172.21.15.103:0/3281109304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a914465d0 0x558a91446a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:43.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4da7fc640 1 -- 172.21.15.103:0/2172171437 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fe4dc0f0e10 con 0x7fe4e410c1a0 2024-09-19T07:04:43.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4e8817640 1 --2- 172.21.15.103:0/3281109304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a91451370 0x558a91453760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:43.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4e8817640 1 -- 172.21.15.103:0/3281109304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a914465d0 msgr2=0x558a91446a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:43.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4e8817640 1 --2- 172.21.15.103:0/3281109304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a914465d0 0x558a91446a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4e8817640 1 -- 172.21.15.103:0/3281109304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a91451370 msgr2=0x558a91453760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:43.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4e8817640 1 --2- 172.21.15.103:0/3281109304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a91451370 0x558a91453760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4e8817640 1 -- 172.21.15.103:0/3281109304 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a913383a0 con 0x558a91447b40 2024-09-19T07:04:43.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.949+0000 7fe4e8817640 1 --2- 172.21.15.103:0/3281109304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91447b40 0x558a91446090 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009a90 tx=0x7fe4d40092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fac8673b0c5192 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:43.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.950+0000 7fe4e9819640 1 --2- 172.21.15.103:0/2172171437 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe4d00a3430 0x7fe4d00a58f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0ee700 tx=0x7fe4dc087fc0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:43.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.950+0000 7fe4cf7fe640 1 -- 172.21.15.103:0/3281109304 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4019070 con 0x558a91447b40 2024-09-19T07:04:43.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.950+0000 7fe4cf7fe640 1 -- 172.21.15.103:0/3281109304 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d4010040 con 0x558a91447b40 2024-09-19T07:04:43.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.950+0000 7fe4cf7fe640 1 -- 172.21.15.103:0/3281109304 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d400c050 con 0x558a91447b40 2024-09-19T07:04:43.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.950+0000 7fe4ed655740 1 -- 172.21.15.103:0/3281109304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91447b40 msgr2=0x558a91446090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:43.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.950+0000 7fe4ed655740 1 --2- 172.21.15.103:0/3281109304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91447b40 0x558a91446090 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009a90 tx=0x7fe4d40092d0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.950+0000 7fe4ed655740 1 -- 172.21.15.103:0/3281109304 shutdown_connections 2024-09-19T07:04:43.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.950+0000 7fe4ed655740 1 --2- 172.21.15.103:0/3281109304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a91451370 0x558a91453760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.950+0000 7fe4ed655740 1 --2- 172.21.15.103:0/3281109304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a914465d0 0x558a91446a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.950+0000 7fe4ed655740 1 --2- 172.21.15.103:0/3281109304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91447b40 0x558a91446090 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.950+0000 7fe4ed655740 1 -- 172.21.15.103:0/3281109304 >> 172.21.15.103:0/3281109304 conn(0x558a9140f310 msgr2=0x558a91442c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:43.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.950+0000 7fe4ed655740 1 -- 172.21.15.103:0/3281109304 shutdown_connections 2024-09-19T07:04:43.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.951+0000 7fe4ed655740 1 -- 172.21.15.103:0/3281109304 wait complete. 2024-09-19T07:04:43.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.951+0000 7fe4ed655740 1 -- start start 2024-09-19T07:04:43.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.951+0000 7fe4ed655740 1 Processor -- start 2024-09-19T07:04:43.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.951+0000 7fe4ed655740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a914465d0 0x558a91450dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.951+0000 7fe4ed655740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a91447b40 0x558a9144e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.951+0000 7fe4ed655740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a91451370 0x558a9144e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.951+0000 7fe4ed655740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558a91454070 con 0x558a914465d0 2024-09-19T07:04:43.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.951+0000 7fe4ed655740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558a91367060 con 0x558a91447b40 2024-09-19T07:04:43.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.951+0000 7fe4ed655740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558a914541f0 con 0x558a91451370 2024-09-19T07:04:43.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.951+0000 7fe4ed655740 10 client.0 _ll_register_callbacks cb 0x558a910ba8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:43.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.951+0000 7fe4e8817640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a914465d0 0x558a91450dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:43.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.951+0000 7fe4e8817640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a914465d0 0x558a91450dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56776/0 (socket says 172.21.15.103:56776) 2024-09-19T07:04:43.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.951+0000 7fe4e8817640 1 -- 172.21.15.103:0/3733995945 learned_addr learned my addr 172.21.15.103:0/3733995945 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:43.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4e9018640 1 --2- 172.21.15.103:0/3733995945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a91447b40 0x558a9144e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:43.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4e8817640 1 -- 172.21.15.103:0/3733995945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a91451370 msgr2=0x558a9144e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:43.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4e8817640 1 --2- 172.21.15.103:0/3733995945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a91451370 0x558a9144e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4e8817640 1 -- 172.21.15.103:0/3733995945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a91447b40 msgr2=0x558a9144e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:43.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4e8817640 1 --2- 172.21.15.103:0/3733995945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a91447b40 0x558a9144e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4e8817640 1 -- 172.21.15.103:0/3733995945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a9111d3c0 con 0x558a914465d0 2024-09-19T07:04:43.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4e8817640 1 --2- 172.21.15.103:0/3733995945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a914465d0 0x558a91450dd0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40092a0 tx=0x7fe4d4003690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf39fe2493fedfb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:43.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:43.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4ccff9640 1 -- 172.21.15.103:0/3733995945 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4019040 con 0x558a914465d0 2024-09-19T07:04:43.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4ccff9640 1 -- 172.21.15.103:0/3733995945 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fe4d4014430 con 0x558a914465d0 2024-09-19T07:04:43.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4ed655740 1 -- 172.21.15.103:0/3733995945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d4010040 con 0x558a914465d0 2024-09-19T07:04:43.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4ccff9640 1 -- 172.21.15.103:0/3733995945 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4d402b050 con 0x558a914465d0 2024-09-19T07:04:43.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4ed655740 10 client.20766 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:43.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4ed655740 1 -- 172.21.15.103:0/3733995945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558a9145fbc0 con 0x558a914465d0 2024-09-19T07:04:43.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4ccff9640 1 -- 172.21.15.103:0/3733995945 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d4010040 con 0x558a914465d0 2024-09-19T07:04:43.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4ccff9640 1 -- 172.21.15.103:0/3733995945 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe4d4003e40 con 0x558a914465d0 2024-09-19T07:04:43.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.952+0000 7fe4ed655740 10 client.20766 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:43.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.953+0000 7fe4ed655740 10 client.20766 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:04:43.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.953+0000 7fe4ed655740 20 client.20766 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:43.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.953+0000 7fe4ed655740 1 --2- 172.21.15.103:0/3733995945 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x558a91460f10 0x558a91463300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.953+0000 7fe4ed655740 4 client.20766 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:43.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.953+0000 7fe4ed655740 1 -- 172.21.15.103:0/3733995945 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a913b9860 con 0x558a91460f10 2024-09-19T07:04:43.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.953+0000 7fe4e9018640 1 --2- 172.21.15.103:0/3733995945 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x558a91460f10 0x558a91463300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:43.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.953+0000 7fe4ccff9640 10 client.20766 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:04:43.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.953+0000 7fe4e9018640 1 --2- 172.21.15.103:0/3733995945 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x558a91460f10 0x558a91463300 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9907606d827c50aa server_cookie=39a660ca4da05878 in_seq=0 out_seq=0 2024-09-19T07:04:43.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.954+0000 7fe4ccff9640 1 -- 172.21.15.103:0/3733995945 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a913b9860 con 0x558a91460f10 2024-09-19T07:04:43.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.954+0000 7fe4ccff9640 10 client.20766 handle_command_reply: tid=0 2024-09-19T07:04:43.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.954+0000 7fe4ed655740 1 client.20766 shutdown 2024-09-19T07:04:43.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.954+0000 7fe4ed655740 1 -- 172.21.15.103:0/3733995945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a914465d0 msgr2=0x558a91450dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:43.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.954+0000 7fe4ed655740 1 --2- 172.21.15.103:0/3733995945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a914465d0 0x558a91450dd0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40092a0 tx=0x7fe4d4003690 comp rx=0 tx=0).stop 2024-09-19T07:04:43.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.954+0000 7fe4ed655740 1 -- 172.21.15.103:0/3733995945 shutdown_connections 2024-09-19T07:04:43.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.954+0000 7fe4ed655740 1 --2- 172.21.15.103:0/3733995945 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x558a91460f10 0x558a91463300 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.954+0000 7fe4ed655740 1 --2- 172.21.15.103:0/3733995945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a91451370 0x558a9144e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.954+0000 7fe4ed655740 1 --2- 172.21.15.103:0/3733995945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a91447b40 0x558a9144e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.954+0000 7fe4ccff9640 0 client.20766 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:04:43.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.954+0000 7fe4ed655740 1 --2- 172.21.15.103:0/3733995945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a914465d0 0x558a91450dd0 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.954+0000 7fe4ed655740 1 -- 172.21.15.103:0/3733995945 >> 172.21.15.103:0/3733995945 conn(0x558a9140f310 msgr2=0x558a91448930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:43.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.954+0000 7fe4ed655740 1 -- 172.21.15.103:0/3733995945 shutdown_connections 2024-09-19T07:04:43.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.954+0000 7fe4ed655740 1 -- 172.21.15.103:0/3733995945 wait complete. 2024-09-19T07:04:43.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.954+0000 7fe4ed655740 20 client.20766 trim_cache size 0 max 16384 2024-09-19T07:04:43.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.967+0000 7fe4ed655740 1 Processor -- start 2024-09-19T07:04:43.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.967+0000 7fe4ed655740 1 -- start start 2024-09-19T07:04:43.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.967+0000 7fe4ed655740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a914465d0 0x558a9144d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.967+0000 7fe4ed655740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a91447b40 0x558a914688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.967+0000 7fe4ed655740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91451370 0x558a91468e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.967+0000 7fe4ed655740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41afb60 con 0x558a91451370 2024-09-19T07:04:43.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.967+0000 7fe4ed655740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41b16a0 con 0x558a914465d0 2024-09-19T07:04:43.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.967+0000 7fe4ed655740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41b0230 con 0x558a91447b40 2024-09-19T07:04:43.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.967+0000 7fe4e8817640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91451370 0x558a91468e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:43.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.967+0000 7fe4e8817640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91451370 0x558a91468e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56792/0 (socket says 172.21.15.103:56792) 2024-09-19T07:04:43.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.967+0000 7fe4e8817640 1 -- 172.21.15.103:0/3929587147 learned_addr learned my addr 172.21.15.103:0/3929587147 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:43.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.968+0000 7fe4e8817640 1 --2- 172.21.15.103:0/3929587147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a914465d0 0x558a9144d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:43.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.968+0000 7fe4e8817640 1 -- 172.21.15.103:0/3929587147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a91447b40 msgr2=0x558a914688d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:43.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.968+0000 7fe4e8817640 1 --2- 172.21.15.103:0/3929587147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a91447b40 0x558a914688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.968+0000 7fe4e8817640 1 -- 172.21.15.103:0/3929587147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a914465d0 msgr2=0x558a9144d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:43.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.968+0000 7fe4e8817640 1 --2- 172.21.15.103:0/3929587147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a914465d0 0x558a9144d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.968+0000 7fe4e8817640 1 -- 172.21.15.103:0/3929587147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9144eb40 con 0x558a91451370 2024-09-19T07:04:43.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.968+0000 7fe4e8817640 1 --2- 172.21.15.103:0/3929587147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91451370 0x558a91468e10 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40099e0 tx=0x7fe4d4009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ce0b3abf508020 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:43.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.968+0000 7fe4ceffd640 1 -- 172.21.15.103:0/3929587147 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4019040 con 0x558a91451370 2024-09-19T07:04:43.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.968+0000 7fe4ceffd640 1 -- 172.21.15.103:0/3929587147 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d4004450 con 0x558a91451370 2024-09-19T07:04:43.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.968+0000 7fe4ceffd640 1 -- 172.21.15.103:0/3929587147 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d400c040 con 0x558a91451370 2024-09-19T07:04:43.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.969+0000 7fe4ed655740 1 -- 172.21.15.103:0/3929587147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91451370 msgr2=0x558a91468e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:43.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.969+0000 7fe4ed655740 1 --2- 172.21.15.103:0/3929587147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91451370 0x558a91468e10 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40099e0 tx=0x7fe4d4009fb0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.969+0000 7fe4ed655740 1 -- 172.21.15.103:0/3929587147 shutdown_connections 2024-09-19T07:04:43.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.969+0000 7fe4ed655740 1 --2- 172.21.15.103:0/3929587147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91451370 0x558a91468e10 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.969+0000 7fe4ed655740 1 --2- 172.21.15.103:0/3929587147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a91447b40 0x558a914688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.969+0000 7fe4ed655740 1 --2- 172.21.15.103:0/3929587147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a914465d0 0x558a9144d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.969+0000 7fe4ed655740 1 -- 172.21.15.103:0/3929587147 >> 172.21.15.103:0/3929587147 conn(0x558a9140f310 msgr2=0x558a9144c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:43.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.969+0000 7fe4ed655740 1 -- 172.21.15.103:0/3929587147 shutdown_connections 2024-09-19T07:04:43.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.969+0000 7fe4ed655740 1 -- 172.21.15.103:0/3929587147 wait complete. 2024-09-19T07:04:43.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.969+0000 7fe4ed655740 1 -- start start 2024-09-19T07:04:43.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.969+0000 7fe4ed655740 1 Processor -- start 2024-09-19T07:04:43.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4ed655740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a914465d0 0x558a9144b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4ed655740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91447b40 0x558a9144b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4ed655740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a91451370 0x558a9144baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4ed655740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41b0fd0 con 0x558a91447b40 2024-09-19T07:04:43.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4ed655740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41b0900 con 0x558a914465d0 2024-09-19T07:04:43.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4ed655740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41adb60 con 0x558a91451370 2024-09-19T07:04:43.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4ed655740 10 client.0 _ll_register_callbacks cb 0x7fe4d4017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:43.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4e9018640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91447b40 0x558a9144b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:43.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4e9018640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91447b40 0x558a9144b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56804/0 (socket says 172.21.15.103:56804) 2024-09-19T07:04:43.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4e9018640 1 -- 172.21.15.103:0/2604277574 learned_addr learned my addr 172.21.15.103:0/2604277574 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:43.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4e8817640 1 --2- 172.21.15.103:0/2604277574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a914465d0 0x558a9144b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:43.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4e8817640 1 --2- 172.21.15.103:0/2604277574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a91451370 0x558a9144baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:43.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4e9018640 1 -- 172.21.15.103:0/2604277574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a91451370 msgr2=0x558a9144baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:43.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4e9018640 1 --2- 172.21.15.103:0/2604277574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a91451370 0x558a9144baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4e9018640 1 -- 172.21.15.103:0/2604277574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a914465d0 msgr2=0x558a9144b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:43.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4e9018640 1 --2- 172.21.15.103:0/2604277574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a914465d0 0x558a9144b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4e9018640 1 -- 172.21.15.103:0/2604277574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a914ca8a0 con 0x558a91447b40 2024-09-19T07:04:43.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4e8817640 1 --2- 172.21.15.103:0/2604277574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a914465d0 0x558a9144b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:43.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4e9018640 1 --2- 172.21.15.103:0/2604277574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91447b40 0x558a9144b560 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0000c00 tx=0x7fe4e000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aad710db6fd9789 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:43.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:43.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4ccff9640 1 -- 172.21.15.103:0/2604277574 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0015c60 con 0x558a91447b40 2024-09-19T07:04:43.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4ccff9640 1 -- 172.21.15.103:0/2604277574 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fe4e0015e00 con 0x558a91447b40 2024-09-19T07:04:43.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.970+0000 7fe4ed655740 1 -- 172.21.15.103:0/2604277574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d4004450 con 0x558a91447b40 2024-09-19T07:04:43.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.971+0000 7fe4ccff9640 1 -- 172.21.15.103:0/2604277574 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4e0023420 con 0x558a91447b40 2024-09-19T07:04:43.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.971+0000 7fe4ed655740 10 client.20772 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:43.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.971+0000 7fe4ed655740 1 -- 172.21.15.103:0/2604277574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558a914cbbb0 con 0x558a91447b40 2024-09-19T07:04:43.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.971+0000 7fe4ccff9640 1 -- 172.21.15.103:0/2604277574 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d4004450 con 0x558a91447b40 2024-09-19T07:04:43.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.971+0000 7fe4ccff9640 1 -- 172.21.15.103:0/2604277574 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe4e0022810 con 0x558a91447b40 2024-09-19T07:04:43.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.971+0000 7fe4ed655740 10 client.20772 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:43.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.971+0000 7fe4ed655740 10 client.20772 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:04:43.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.971+0000 7fe4ed655740 20 client.20772 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:43.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.971+0000 7fe4ed655740 1 --2- 172.21.15.103:0/2604277574 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x558a91460f10 0x558a914ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:43.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.971+0000 7fe4ed655740 4 client.20772 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:04:43.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.971+0000 7fe4ed655740 1 -- 172.21.15.103:0/2604277574 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a913b3d70 con 0x558a91460f10 2024-09-19T07:04:43.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.971+0000 7fe4e8817640 1 --2- 172.21.15.103:0/2604277574 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x558a91460f10 0x558a914ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:43.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4e8817640 1 --2- 172.21.15.103:0/2604277574 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x558a91460f10 0x558a914ce3f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fc5d6b2c40dc955d server_cookie=9418c8cb7dec90c7 in_seq=0 out_seq=0 2024-09-19T07:04:43.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4ccff9640 10 client.20772 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:04:43.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4ccff9640 1 -- 172.21.15.103:0/2604277574 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a913b3d70 con 0x558a91460f10 2024-09-19T07:04:43.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4ccff9640 10 client.20772 handle_command_reply: tid=0 2024-09-19T07:04:43.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4ed655740 1 client.20772 shutdown 2024-09-19T07:04:43.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4ed655740 1 -- 172.21.15.103:0/2604277574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91447b40 msgr2=0x558a9144b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:43.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4ed655740 1 --2- 172.21.15.103:0/2604277574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91447b40 0x558a9144b560 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0000c00 tx=0x7fe4e000cf90 comp rx=0 tx=0).stop 2024-09-19T07:04:43.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4ed655740 1 -- 172.21.15.103:0/2604277574 shutdown_connections 2024-09-19T07:04:43.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4ed655740 1 --2- 172.21.15.103:0/2604277574 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x558a91460f10 0x558a914ce3f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4ed655740 1 --2- 172.21.15.103:0/2604277574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a91451370 0x558a9144baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4ed655740 1 --2- 172.21.15.103:0/2604277574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a91447b40 0x558a9144b560 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4ccff9640 0 client.20772 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:04:43.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4ed655740 1 --2- 172.21.15.103:0/2604277574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a914465d0 0x558a9144b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4ed655740 1 -- 172.21.15.103:0/2604277574 >> 172.21.15.103:0/2604277574 conn(0x558a9140f310 msgr2=0x558a914414e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:43.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4ed655740 1 -- 172.21.15.103:0/2604277574 shutdown_connections 2024-09-19T07:04:43.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4ed655740 1 -- 172.21.15.103:0/2604277574 wait complete. 2024-09-19T07:04:43.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.972+0000 7fe4ed655740 20 client.20772 trim_cache size 0 max 16384 2024-09-19T07:04:43.980 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:04:43.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.973+0000 7fe4eb27c640 1 -- 172.21.15.103:0/2172171437 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe4d00a3430 msgr2=0x7fe4d00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:43.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.973+0000 7fe4eb27c640 1 --2- 172.21.15.103:0/2172171437 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe4d00a3430 0x7fe4d00a58f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0ee700 tx=0x7fe4dc087fc0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.973+0000 7fe4eb27c640 1 -- 172.21.15.103:0/2172171437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4e410c1a0 msgr2=0x7fe4e41014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:43.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.973+0000 7fe4eb27c640 1 --2- 172.21.15.103:0/2172171437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4e410c1a0 0x7fe4e41014e0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc006010 tx=0x7fe4dc02f710 comp rx=0 tx=0).stop 2024-09-19T07:04:43.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.973+0000 7fe4eb27c640 1 -- 172.21.15.103:0/2172171437 shutdown_connections 2024-09-19T07:04:43.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.973+0000 7fe4eb27c640 1 --2- 172.21.15.103:0/2172171437 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe4d00a3430 0x7fe4d00a58f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.973+0000 7fe4eb27c640 1 --2- 172.21.15.103:0/2172171437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4e410c1a0 0x7fe4e41014e0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.973+0000 7fe4eb27c640 1 --2- 172.21.15.103:0/2172171437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4e4103720 0x7fe4e4100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.973+0000 7fe4eb27c640 1 --2- 172.21.15.103:0/2172171437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e4102e00 0x7fe4e4100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:43.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.973+0000 7fe4eb27c640 1 -- 172.21.15.103:0/2172171437 >> 172.21.15.103:0/2172171437 conn(0x7fe4e40fc7a0 msgr2=0x7fe4e40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:43.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.973+0000 7fe4eb27c640 1 -- 172.21.15.103:0/2172171437 shutdown_connections 2024-09-19T07:04:43.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:43.973+0000 7fe4eb27c640 1 -- 172.21.15.103:0/2172171437 wait complete. 2024-09-19T07:04:44.614 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:04:44.861 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.945+0000 7f7cb6cf1640 1 -- 172.21.15.103:0/2860059126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cb01035b0 msgr2=0x7f7cb010ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.945+0000 7f7cb6cf1640 1 --2- 172.21.15.103:0/2860059126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cb01035b0 0x7f7cb010ba90 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009900 tx=0x7f7ca002a480 comp rx=0 tx=0).stop 2024-09-19T07:04:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.945+0000 7f7cb6cf1640 1 -- 172.21.15.103:0/2860059126 shutdown_connections 2024-09-19T07:04:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.945+0000 7f7cb6cf1640 1 --2- 172.21.15.103:0/2860059126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cb010c0b0 0x7f7cb010e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.945+0000 7f7cb6cf1640 1 --2- 172.21.15.103:0/2860059126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cb01035b0 0x7f7cb010ba90 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.945+0000 7f7cb6cf1640 1 --2- 172.21.15.103:0/2860059126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cb0102c90 0x7f7cb0103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.945+0000 7f7cb6cf1640 1 -- 172.21.15.103:0/2860059126 >> 172.21.15.103:0/2860059126 conn(0x7f7cb00fc6e0 msgr2=0x7f7cb00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.945+0000 7f7cb6cf1640 1 -- 172.21.15.103:0/2860059126 shutdown_connections 2024-09-19T07:04:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.945+0000 7f7cb6cf1640 1 -- 172.21.15.103:0/2860059126 wait complete. 2024-09-19T07:04:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.945+0000 7f7cb6cf1640 1 Processor -- start 2024-09-19T07:04:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.945+0000 7f7cb6cf1640 1 -- start start 2024-09-19T07:04:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.945+0000 7f7cb6cf1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cb0102c90 0x7f7cb0198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.945+0000 7f7cb6cf1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cb01035b0 0x7f7cb0198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7cb6cf1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cb010c0b0 0x7f7cb019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7cb6cf1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb0111ed0 con 0x7f7cb0102c90 2024-09-19T07:04:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7cb6cf1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb0111d50 con 0x7f7cb01035b0 2024-09-19T07:04:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7cb6cf1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb0112050 con 0x7f7cb010c0b0 2024-09-19T07:04:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7cb4a8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cb0102c90 0x7f7cb0198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7cb4a8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cb0102c90 0x7f7cb0198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56818/0 (socket says 172.21.15.103:56818) 2024-09-19T07:04:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7cb4a8d640 1 -- 172.21.15.103:0/137428870 learned_addr learned my addr 172.21.15.103:0/137428870 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7ca7fff640 1 --2- 172.21.15.103:0/137428870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cb01035b0 0x7f7cb0198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7cb528e640 1 --2- 172.21.15.103:0/137428870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cb010c0b0 0x7f7cb019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7cb4a8d640 1 -- 172.21.15.103:0/137428870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cb010c0b0 msgr2=0x7f7cb019cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7cb4a8d640 1 --2- 172.21.15.103:0/137428870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cb010c0b0 0x7f7cb019cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7cb4a8d640 1 -- 172.21.15.103:0/137428870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cb01035b0 msgr2=0x7f7cb0198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7cb4a8d640 1 --2- 172.21.15.103:0/137428870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cb01035b0 0x7f7cb0198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7cb4a8d640 1 -- 172.21.15.103:0/137428870 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb019d300 con 0x7f7cb0102c90 2024-09-19T07:04:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7cb4a8d640 1 --2- 172.21.15.103:0/137428870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cb0102c90 0x7f7cb0198310 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00c890 tx=0x7f7cac00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7ca5ffb640 1 -- 172.21.15.103:0/137428870 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac004220 con 0x7f7cb0102c90 2024-09-19T07:04:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7ca5ffb640 1 -- 172.21.15.103:0/137428870 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cac0043c0 con 0x7f7cb0102c90 2024-09-19T07:04:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7ca5ffb640 1 -- 172.21.15.103:0/137428870 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac0049e0 con 0x7f7cb0102c90 2024-09-19T07:04:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.946+0000 7f7cb6cf1640 1 -- 172.21.15.103:0/137428870 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb019d5f0 con 0x7f7cb0102c90 2024-09-19T07:04:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.947+0000 7f7cb6cf1640 1 -- 172.21.15.103:0/137428870 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb01a4e80 con 0x7f7cb0102c90 2024-09-19T07:04:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.949+0000 7f7ca5ffb640 1 -- 172.21.15.103:0/137428870 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7cac0079b0 con 0x7f7cb0102c90 2024-09-19T07:04:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.949+0000 7f7cb9a90740 1 Processor -- start 2024-09-19T07:04:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7cb9a90740 1 -- start start 2024-09-19T07:04:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7cb9a90740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd6f49b0 0x5567dd6f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7ca5ffb640 1 --2- 172.21.15.103:0/137428870 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c980a3460 0x7f7c980a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7cb9a90740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f52d0 0x5567dd703b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7cb9a90740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd7040c0 0x5567dd7064b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7cb9a90740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5567dd6592f0 con 0x5567dd6f52d0 2024-09-19T07:04:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7cb9a90740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5567dd601270 con 0x5567dd6f49b0 2024-09-19T07:04:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7cb9a90740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5567dd6552b0 con 0x5567dd7040c0 2024-09-19T07:04:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7ca7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f52d0 0x5567dd703b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7cb4a8d640 1 --2- 172.21.15.103:0/137428870 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c980a3460 0x7f7c980a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7ca7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f52d0 0x5567dd703b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56828/0 (socket says 172.21.15.103:56828) 2024-09-19T07:04:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7ca7fff640 1 -- 172.21.15.103:0/2389846807 learned_addr learned my addr 172.21.15.103:0/2389846807 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7ca5ffb640 1 -- 172.21.15.103:0/137428870 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f7cac0cfda0 con 0x7f7cb0102c90 2024-09-19T07:04:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7cb528e640 1 --2- 172.21.15.103:0/2389846807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd6f49b0 0x5567dd6f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7cb528e640 1 --2- 172.21.15.103:0/2389846807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd7040c0 0x5567dd7064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7ca7fff640 1 -- 172.21.15.103:0/2389846807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd7040c0 msgr2=0x5567dd7064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7ca7fff640 1 --2- 172.21.15.103:0/2389846807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd7040c0 0x5567dd7064b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7ca7fff640 1 -- 172.21.15.103:0/2389846807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd6f49b0 msgr2=0x5567dd6f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7ca7fff640 1 --2- 172.21.15.103:0/2389846807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd6f49b0 0x5567dd6f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7ca7fff640 1 -- 172.21.15.103:0/2389846807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567dd600340 con 0x5567dd6f52d0 2024-09-19T07:04:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7cb4a8d640 1 --2- 172.21.15.103:0/137428870 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c980a3460 0x7f7c980a5920 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7cac0cdad0 tx=0x7f7cac005300 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.950+0000 7f7ca7fff640 1 --2- 172.21.15.103:0/2389846807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f52d0 0x5567dd703b80 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009900 tx=0x7f7ca0031d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228fbb0f0a456616 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.951+0000 7f7c96ffd640 1 -- 172.21.15.103:0/2389846807 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0040070 con 0x5567dd6f52d0 2024-09-19T07:04:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.951+0000 7f7c96ffd640 1 -- 172.21.15.103:0/2389846807 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ca00317b0 con 0x5567dd6f52d0 2024-09-19T07:04:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.951+0000 7f7c96ffd640 1 -- 172.21.15.103:0/2389846807 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0035040 con 0x5567dd6f52d0 2024-09-19T07:04:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.951+0000 7f7cb9a90740 1 -- 172.21.15.103:0/2389846807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f52d0 msgr2=0x5567dd703b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.951+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/2389846807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f52d0 0x5567dd703b80 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009900 tx=0x7f7ca0031d10 comp rx=0 tx=0).stop 2024-09-19T07:04:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.951+0000 7f7cb9a90740 1 -- 172.21.15.103:0/2389846807 shutdown_connections 2024-09-19T07:04:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.951+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/2389846807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd7040c0 0x5567dd7064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.951+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/2389846807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f52d0 0x5567dd703b80 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.951+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/2389846807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd6f49b0 0x5567dd6f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.951+0000 7f7cb9a90740 1 -- 172.21.15.103:0/2389846807 >> 172.21.15.103:0/2389846807 conn(0x5567dd6f1060 msgr2=0x5567dd6f1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.951+0000 7f7cb9a90740 1 -- 172.21.15.103:0/2389846807 shutdown_connections 2024-09-19T07:04:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.951+0000 7f7cb9a90740 1 -- 172.21.15.103:0/2389846807 wait complete. 2024-09-19T07:04:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.952+0000 7f7cb9a90740 1 -- start start 2024-09-19T07:04:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.952+0000 7f7cb9a90740 1 Processor -- start 2024-09-19T07:04:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.952+0000 7f7cb9a90740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd6f49b0 0x5567dd700fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.952+0000 7f7cb9a90740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd6f52d0 0x5567dd7014e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.952+0000 7f7cb9a90740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd7040c0 0x5567dd701a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.952+0000 7f7cb9a90740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5567dd706c30 con 0x5567dd7040c0 2024-09-19T07:04:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.952+0000 7f7cb9a90740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5567dd5e2290 con 0x5567dd6f52d0 2024-09-19T07:04:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.952+0000 7f7cb9a90740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5567dd706db0 con 0x5567dd6f49b0 2024-09-19T07:04:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.952+0000 7f7cb9a90740 10 client.0 _ll_register_callbacks cb 0x5567dd613140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.952+0000 7f7cb528e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd7040c0 0x5567dd701a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.952+0000 7f7cb528e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd7040c0 0x5567dd701a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56842/0 (socket says 172.21.15.103:56842) 2024-09-19T07:04:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.952+0000 7f7cb528e640 1 -- 172.21.15.103:0/2290021709 learned_addr learned my addr 172.21.15.103:0/2290021709 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.952+0000 7f7cb528e640 1 --2- 172.21.15.103:0/2290021709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd6f49b0 0x5567dd700fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.952+0000 7f7ca7fff640 1 --2- 172.21.15.103:0/2290021709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd6f52d0 0x5567dd7014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.953+0000 7f7cb528e640 1 -- 172.21.15.103:0/2290021709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd6f49b0 msgr2=0x5567dd700fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.953+0000 7f7cb528e640 1 --2- 172.21.15.103:0/2290021709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd6f49b0 0x5567dd700fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.953+0000 7f7cb528e640 1 -- 172.21.15.103:0/2290021709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd6f52d0 msgr2=0x5567dd7014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.953+0000 7f7cb528e640 1 --2- 172.21.15.103:0/2290021709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd6f52d0 0x5567dd7014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.953+0000 7f7cb528e640 1 -- 172.21.15.103:0/2290021709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567dd36c210 con 0x5567dd7040c0 2024-09-19T07:04:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.953+0000 7f7cb528e640 1 --2- 172.21.15.103:0/2290021709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd7040c0 0x5567dd701a20 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8010740 tx=0x7f7ca8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c38a1d6432ffa84 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.953+0000 7f7c8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.953+0000 7f7c8ffff640 1 -- 172.21.15.103:0/2290021709 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80130d0 con 0x5567dd7040c0 2024-09-19T07:04:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.953+0000 7f7cb9a90740 1 -- 172.21.15.103:0/2290021709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ca00317b0 con 0x5567dd7040c0 2024-09-19T07:04:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.953+0000 7f7c8ffff640 1 -- 172.21.15.103:0/2290021709 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f7ca8013270 con 0x5567dd7040c0 2024-09-19T07:04:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.953+0000 7f7c8ffff640 1 -- 172.21.15.103:0/2290021709 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ca8004b20 con 0x5567dd7040c0 2024-09-19T07:04:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.953+0000 7f7cb9a90740 10 client.20784 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.953+0000 7f7cb9a90740 1 -- 172.21.15.103:0/2290021709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567dd438620 con 0x5567dd7040c0 2024-09-19T07:04:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.953+0000 7f7c8ffff640 1 -- 172.21.15.103:0/2290021709 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ca00317b0 con 0x5567dd7040c0 2024-09-19T07:04:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.953+0000 7f7c8ffff640 1 -- 172.21.15.103:0/2290021709 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7ca80117d0 con 0x5567dd7040c0 2024-09-19T07:04:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.954+0000 7f7cb9a90740 10 client.20784 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.954+0000 7f7cb9a90740 10 client.20784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.954+0000 7f7cb9a90740 20 client.20784 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.954+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/2290021709 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5567dd713f80 0x5567dd716390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.954+0000 7f7cb9a90740 4 client.20784 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.954+0000 7f7cb9a90740 1 -- 172.21.15.103:0/2290021709 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567dd666a20 con 0x5567dd713f80 2024-09-19T07:04:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.954+0000 7f7ca7fff640 1 --2- 172.21.15.103:0/2290021709 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5567dd713f80 0x5567dd716390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.954+0000 7f7ca7fff640 1 --2- 172.21.15.103:0/2290021709 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5567dd713f80 0x5567dd716390 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4f5935f5969f373 server_cookie=d19ed591df36d9b6 in_seq=0 out_seq=0 2024-09-19T07:04:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.954+0000 7f7c8ffff640 10 client.20784 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.955+0000 7f7c8ffff640 1 -- 172.21.15.103:0/2290021709 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5567dd666a20 con 0x5567dd713f80 2024-09-19T07:04:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.955+0000 7f7c8ffff640 10 client.20784 handle_command_reply: tid=0 2024-09-19T07:04:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.955+0000 7f7cb9a90740 1 client.20784 shutdown 2024-09-19T07:04:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.955+0000 7f7cb9a90740 1 -- 172.21.15.103:0/2290021709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd7040c0 msgr2=0x5567dd701a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.955+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/2290021709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd7040c0 0x5567dd701a20 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8010740 tx=0x7f7ca8004ab0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.955+0000 7f7cb9a90740 1 -- 172.21.15.103:0/2290021709 shutdown_connections 2024-09-19T07:04:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.955+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/2290021709 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5567dd713f80 0x5567dd716390 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.955+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/2290021709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd7040c0 0x5567dd701a20 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.955+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/2290021709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd6f52d0 0x5567dd7014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.955+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/2290021709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd6f49b0 0x5567dd700fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.955+0000 7f7c8ffff640 0 client.20784 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.955+0000 7f7cb9a90740 1 -- 172.21.15.103:0/2290021709 >> 172.21.15.103:0/2290021709 conn(0x5567dd6f1060 msgr2=0x5567dd6f6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.955+0000 7f7cb9a90740 1 -- 172.21.15.103:0/2290021709 shutdown_connections 2024-09-19T07:04:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.955+0000 7f7cb9a90740 1 -- 172.21.15.103:0/2290021709 wait complete. 2024-09-19T07:04:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.955+0000 7f7cb9a90740 20 client.20784 trim_cache size 0 max 16384 2024-09-19T07:04:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.969+0000 7f7cb9a90740 1 Processor -- start 2024-09-19T07:04:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.969+0000 7f7cb9a90740 1 -- start start 2024-09-19T07:04:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7cb9a90740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f49b0 0x5567dd71bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7cb9a90740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd6f52d0 0x5567dd71c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7cb9a90740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd7040c0 0x5567dd6fe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7cb9a90740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb01ab630 con 0x5567dd6f49b0 2024-09-19T07:04:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7cb9a90740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb01ad170 con 0x5567dd7040c0 2024-09-19T07:04:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7cb9a90740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb01abd00 con 0x5567dd6f52d0 2024-09-19T07:04:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7cb528e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f49b0 0x5567dd71bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7cb528e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f49b0 0x5567dd71bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56852/0 (socket says 172.21.15.103:56852) 2024-09-19T07:04:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7cb528e640 1 -- 172.21.15.103:0/4122143804 learned_addr learned my addr 172.21.15.103:0/4122143804 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7ca7fff640 1 --2- 172.21.15.103:0/4122143804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd6f52d0 0x5567dd71c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7cb528e640 1 -- 172.21.15.103:0/4122143804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd6f52d0 msgr2=0x5567dd71c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7cb528e640 1 --2- 172.21.15.103:0/4122143804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd6f52d0 0x5567dd71c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7cb528e640 1 -- 172.21.15.103:0/4122143804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd7040c0 msgr2=0x5567dd6fe1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7cb528e640 1 --2- 172.21.15.103:0/4122143804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd7040c0 0x5567dd6fe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7cb528e640 1 -- 172.21.15.103:0/4122143804 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567dd6f5100 con 0x5567dd6f49b0 2024-09-19T07:04:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7cb528e640 1 --2- 172.21.15.103:0/4122143804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f49b0 0x5567dd71bca0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8011010 tx=0x7f7ca800aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb394fcc726a8a71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7c967fc640 1 -- 172.21.15.103:0/4122143804 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca802c070 con 0x5567dd6f49b0 2024-09-19T07:04:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7c967fc640 1 -- 172.21.15.103:0/4122143804 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ca8011920 con 0x5567dd6f49b0 2024-09-19T07:04:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.970+0000 7f7c967fc640 1 -- 172.21.15.103:0/4122143804 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca800ccd0 con 0x5567dd6f49b0 2024-09-19T07:04:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.971+0000 7f7cb9a90740 1 -- 172.21.15.103:0/4122143804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f49b0 msgr2=0x5567dd71bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.971+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/4122143804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f49b0 0x5567dd71bca0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8011010 tx=0x7f7ca800aed0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.971+0000 7f7cb9a90740 1 -- 172.21.15.103:0/4122143804 shutdown_connections 2024-09-19T07:04:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.971+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/4122143804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd7040c0 0x5567dd6fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.971+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/4122143804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd6f52d0 0x5567dd71c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.971+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/4122143804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f49b0 0x5567dd71bca0 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.971+0000 7f7cb9a90740 1 -- 172.21.15.103:0/4122143804 >> 172.21.15.103:0/4122143804 conn(0x5567dd6f1060 msgr2=0x5567dd703020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.971+0000 7f7cb9a90740 1 -- 172.21.15.103:0/4122143804 shutdown_connections 2024-09-19T07:04:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.971+0000 7f7cb9a90740 1 -- 172.21.15.103:0/4122143804 wait complete. 2024-09-19T07:04:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.971+0000 7f7cb9a90740 1 -- start start 2024-09-19T07:04:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.971+0000 7f7cb9a90740 1 Processor -- start 2024-09-19T07:04:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb9a90740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f49b0 0x5567dd708670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb9a90740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd6f52d0 0x5567dd708bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb9a90740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd7040c0 0x5567dd777c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb9a90740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb01acaa0 con 0x5567dd6f49b0 2024-09-19T07:04:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb9a90740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb01ac3d0 con 0x5567dd7040c0 2024-09-19T07:04:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb9a90740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb01a9630 con 0x5567dd6f52d0 2024-09-19T07:04:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb9a90740 10 client.0 _ll_register_callbacks cb 0x7f7ca800ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb528e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f49b0 0x5567dd708670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb528e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f49b0 0x5567dd708670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56864/0 (socket says 172.21.15.103:56864) 2024-09-19T07:04:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb528e640 1 -- 172.21.15.103:0/141722635 learned_addr learned my addr 172.21.15.103:0/141722635 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7ca7fff640 1 --2- 172.21.15.103:0/141722635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd6f52d0 0x5567dd708bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb528e640 1 -- 172.21.15.103:0/141722635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd6f52d0 msgr2=0x5567dd708bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb528e640 1 --2- 172.21.15.103:0/141722635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd6f52d0 0x5567dd708bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb528e640 1 -- 172.21.15.103:0/141722635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd7040c0 msgr2=0x5567dd777c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb528e640 1 --2- 172.21.15.103:0/141722635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd7040c0 0x5567dd777c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb528e640 1 -- 172.21.15.103:0/141722635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567dd778280 con 0x5567dd6f49b0 2024-09-19T07:04:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb528e640 1 --2- 172.21.15.103:0/141722635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f49b0 0x5567dd708670 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800ac00 tx=0x7f7ca800add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24119e22710405c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7c8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7c8ffff640 1 -- 172.21.15.103:0/141722635 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca802c040 con 0x5567dd6f49b0 2024-09-19T07:04:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7c8ffff640 1 -- 172.21.15.103:0/141722635 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f7ca80132d0 con 0x5567dd6f49b0 2024-09-19T07:04:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb9a90740 1 -- 172.21.15.103:0/141722635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ca8011920 con 0x5567dd6f49b0 2024-09-19T07:04:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7c8ffff640 1 -- 172.21.15.103:0/141722635 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ca803e050 con 0x5567dd6f49b0 2024-09-19T07:04:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb9a90740 10 client.20790 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.972+0000 7f7cb9a90740 1 -- 172.21.15.103:0/141722635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567dd77f640 con 0x5567dd6f49b0 2024-09-19T07:04:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.973+0000 7f7c8ffff640 1 -- 172.21.15.103:0/141722635 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ca8011920 con 0x5567dd6f49b0 2024-09-19T07:04:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.973+0000 7f7c8ffff640 1 -- 172.21.15.103:0/141722635 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7ca8030450 con 0x5567dd6f49b0 2024-09-19T07:04:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.973+0000 7f7cb9a90740 10 client.20790 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.973+0000 7f7cb9a90740 10 client.20790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.973+0000 7f7cb9a90740 20 client.20790 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.973+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/141722635 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5567dd77fbf0 0x5567dd781fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.973+0000 7f7cb9a90740 4 client.20790 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:04:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.973+0000 7f7cb9a90740 1 -- 172.21.15.103:0/141722635 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5567dd660d70 con 0x5567dd77fbf0 2024-09-19T07:04:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.973+0000 7f7ca7fff640 1 --2- 172.21.15.103:0/141722635 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5567dd77fbf0 0x5567dd781fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.973+0000 7f7ca7fff640 1 --2- 172.21.15.103:0/141722635 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5567dd77fbf0 0x5567dd781fe0 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c52bf6727dd52f56 server_cookie=d4796d510e0d76d9 in_seq=0 out_seq=0 2024-09-19T07:04:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.973+0000 7f7c8ffff640 10 client.20790 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.982+0000 7f7c8ffff640 1 -- 172.21.15.103:0/141722635 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5567dd660d70 con 0x5567dd77fbf0 2024-09-19T07:04:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.982+0000 7f7c8ffff640 10 client.20790 handle_command_reply: tid=0 2024-09-19T07:04:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.982+0000 7f7cb9a90740 1 client.20790 shutdown 2024-09-19T07:04:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.983+0000 7f7cb9a90740 1 -- 172.21.15.103:0/141722635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f49b0 msgr2=0x5567dd708670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.983+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/141722635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f49b0 0x5567dd708670 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800ac00 tx=0x7f7ca800add0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.983+0000 7f7cb9a90740 1 -- 172.21.15.103:0/141722635 shutdown_connections 2024-09-19T07:04:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.983+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/141722635 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5567dd77fbf0 0x5567dd781fe0 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.983+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/141722635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5567dd7040c0 0x5567dd777c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.983+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/141722635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567dd6f52d0 0x5567dd708bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.983+0000 7f7c8ffff640 0 client.20790 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.983+0000 7f7cb9a90740 1 --2- 172.21.15.103:0/141722635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567dd6f49b0 0x5567dd708670 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.983+0000 7f7cb9a90740 1 -- 172.21.15.103:0/141722635 >> 172.21.15.103:0/141722635 conn(0x5567dd6f1060 msgr2=0x5567dd6ef9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.983+0000 7f7cb9a90740 1 -- 172.21.15.103:0/141722635 shutdown_connections 2024-09-19T07:04:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.983+0000 7f7cb9a90740 1 -- 172.21.15.103:0/141722635 wait complete. 2024-09-19T07:04:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.983+0000 7f7cb9a90740 20 client.20790 trim_cache size 0 max 16384 2024-09-19T07:04:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:04:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:04:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:04:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:04:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.984+0000 7f7cb6cf1640 1 -- 172.21.15.103:0/137428870 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c980a3460 msgr2=0x7f7c980a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.984+0000 7f7cb6cf1640 1 --2- 172.21.15.103:0/137428870 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c980a3460 0x7f7c980a5920 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7cac0cdad0 tx=0x7f7cac005300 comp rx=0 tx=0).stop 2024-09-19T07:04:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.984+0000 7f7cb6cf1640 1 -- 172.21.15.103:0/137428870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cb0102c90 msgr2=0x7f7cb0198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.984+0000 7f7cb6cf1640 1 --2- 172.21.15.103:0/137428870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cb0102c90 0x7f7cb0198310 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00c890 tx=0x7f7cac00cd50 comp rx=0 tx=0).stop 2024-09-19T07:04:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.984+0000 7f7cb6cf1640 1 -- 172.21.15.103:0/137428870 shutdown_connections 2024-09-19T07:04:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.984+0000 7f7cb6cf1640 1 --2- 172.21.15.103:0/137428870 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c980a3460 0x7f7c980a5920 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.984+0000 7f7cb6cf1640 1 --2- 172.21.15.103:0/137428870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cb010c0b0 0x7f7cb019cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.984+0000 7f7cb6cf1640 1 --2- 172.21.15.103:0/137428870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cb01035b0 0x7f7cb0198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.984+0000 7f7cb6cf1640 1 --2- 172.21.15.103:0/137428870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cb0102c90 0x7f7cb0198310 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.984+0000 7f7cb6cf1640 1 -- 172.21.15.103:0/137428870 >> 172.21.15.103:0/137428870 conn(0x7f7cb00fc6e0 msgr2=0x7f7cb00fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.984+0000 7f7cb6cf1640 1 -- 172.21.15.103:0/137428870 shutdown_connections 2024-09-19T07:04:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:44.984+0000 7f7cb6cf1640 1 -- 172.21.15.103:0/137428870 wait complete. 2024-09-19T07:04:45.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.295+0000 7f483001a640 1 -- 172.21.15.103:0/1974666310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4828101de0 msgr2=0x7f48281021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:45.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.295+0000 7f483001a640 1 --2- 172.21.15.103:0/1974666310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4828101de0 0x7f48281021c0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f4824009900 tx=0x7f482402a410 comp rx=0 tx=0).stop 2024-09-19T07:04:45.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.295+0000 7f483001a640 1 -- 172.21.15.103:0/1974666310 shutdown_connections 2024-09-19T07:04:45.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.295+0000 7f483001a640 1 --2- 172.21.15.103:0/1974666310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4828106f10 0x7f482810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.295+0000 7f483001a640 1 --2- 172.21.15.103:0/1974666310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4828102790 0x7f48281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.295+0000 7f483001a640 1 --2- 172.21.15.103:0/1974666310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4828101de0 0x7f48281021c0 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.295+0000 7f483001a640 1 -- 172.21.15.103:0/1974666310 >> 172.21.15.103:0/1974666310 conn(0x7f48280fd8c0 msgr2=0x7f48280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:45.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.295+0000 7f483001a640 1 -- 172.21.15.103:0/1974666310 shutdown_connections 2024-09-19T07:04:45.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.295+0000 7f483001a640 1 -- 172.21.15.103:0/1974666310 wait complete. 2024-09-19T07:04:45.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.295+0000 7f483001a640 1 Processor -- start 2024-09-19T07:04:45.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.295+0000 7f483001a640 1 -- start start 2024-09-19T07:04:45.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f483001a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4828101de0 0x7f4828198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f483001a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4828102790 0x7f4828198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f483001a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4828106f10 0x7f482819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f483001a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4828111f70 con 0x7f4828102790 2024-09-19T07:04:45.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f483001a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4828111df0 con 0x7f4828106f10 2024-09-19T07:04:45.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f483001a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f48281120f0 con 0x7f4828101de0 2024-09-19T07:04:45.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f482d5b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4828102790 0x7f4828198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f482d5b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4828102790 0x7f4828198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56888/0 (socket says 172.21.15.103:56888) 2024-09-19T07:04:45.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f482d5b5640 1 -- 172.21.15.103:0/1850262098 learned_addr learned my addr 172.21.15.103:0/1850262098 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:45.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f482e5b7640 1 --2- 172.21.15.103:0/1850262098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4828106f10 0x7f482819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f482ddb6640 1 --2- 172.21.15.103:0/1850262098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4828101de0 0x7f4828198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f482d5b5640 1 -- 172.21.15.103:0/1850262098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4828101de0 msgr2=0x7f4828198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:45.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f482d5b5640 1 --2- 172.21.15.103:0/1850262098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4828101de0 0x7f4828198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f482d5b5640 1 -- 172.21.15.103:0/1850262098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4828106f10 msgr2=0x7f482819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:45.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f482d5b5640 1 --2- 172.21.15.103:0/1850262098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4828106f10 0x7f482819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f482d5b5640 1 -- 172.21.15.103:0/1850262098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482819d400 con 0x7f4828102790 2024-09-19T07:04:45.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f482e5b7640 1 --2- 172.21.15.103:0/1850262098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4828106f10 0x7f482819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:45.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f482d5b5640 1 --2- 172.21.15.103:0/1850262098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4828102790 0x7f4828198990 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f481800c890 tx=0x7f481800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:45.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f481effd640 1 -- 172.21.15.103:0/1850262098 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4818004220 con 0x7f4828102790 2024-09-19T07:04:45.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f481effd640 1 -- 172.21.15.103:0/1850262098 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48180043c0 con 0x7f4828102790 2024-09-19T07:04:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f483001a640 1 -- 172.21.15.103:0/1850262098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482819d6f0 con 0x7f4828102790 2024-09-19T07:04:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f481effd640 1 -- 172.21.15.103:0/1850262098 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48180049e0 con 0x7f4828102790 2024-09-19T07:04:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.296+0000 7f483001a640 1 -- 172.21.15.103:0/1850262098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48281a4fd0 con 0x7f4828102790 2024-09-19T07:04:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.299+0000 7f481effd640 1 -- 172.21.15.103:0/1850262098 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f48180079b0 con 0x7f4828102790 2024-09-19T07:04:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.299+0000 7f48323f3740 1 Processor -- start 2024-09-19T07:04:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.299+0000 7f48323f3740 1 -- start start 2024-09-19T07:04:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.299+0000 7f48323f3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6bb40 0x55df44f6a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.299+0000 7f48323f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f6a5d0 0x55df44f6aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f48323f3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f79780 0x55df44f7bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f48323f3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55df44ed02f0 con 0x55df44f6bb40 2024-09-19T07:04:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f48323f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55df44e77cf0 con 0x55df44f6a5d0 2024-09-19T07:04:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f48323f3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55df44ecc2b0 con 0x55df44f79780 2024-09-19T07:04:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f482e5b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6bb40 0x55df44f6a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f482e5b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6bb40 0x55df44f6a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56894/0 (socket says 172.21.15.103:56894) 2024-09-19T07:04:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f482e5b7640 1 -- 172.21.15.103:0/2596876386 learned_addr learned my addr 172.21.15.103:0/2596876386 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f482d5b5640 1 --2- 172.21.15.103:0/2596876386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f79780 0x55df44f7bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f481effd640 1 --2- 172.21.15.103:0/1850262098 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48140a33b0 0x7f48140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f482ddb6640 1 --2- 172.21.15.103:0/2596876386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f6a5d0 0x55df44f6aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f482e5b7640 1 -- 172.21.15.103:0/2596876386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f79780 msgr2=0x55df44f7bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f482e5b7640 1 --2- 172.21.15.103:0/2596876386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f79780 0x55df44f7bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f482e5b7640 1 -- 172.21.15.103:0/2596876386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f6a5d0 msgr2=0x55df44f6aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f482e5b7640 1 --2- 172.21.15.103:0/2596876386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f6a5d0 0x55df44f6aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f482e5b7640 1 -- 172.21.15.103:0/2596876386 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df44e5c3a0 con 0x55df44f6bb40 2024-09-19T07:04:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f482e5b7640 1 --2- 172.21.15.103:0/1850262098 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48140a33b0 0x7f48140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f481effd640 1 -- 172.21.15.103:0/1850262098 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f48180cfeb0 con 0x7f4828102790 2024-09-19T07:04:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f482e5b7640 1 --2- 172.21.15.103:0/2596876386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6bb40 0x55df44f6a090 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f4820010420 tx=0x7f48200108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba5a0289ed9aee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.300+0000 7f4813fff640 1 -- 172.21.15.103:0/2596876386 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820023070 con 0x55df44f6bb40 2024-09-19T07:04:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.301+0000 7f4813fff640 1 -- 172.21.15.103:0/2596876386 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f482001c470 con 0x55df44f6bb40 2024-09-19T07:04:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.301+0000 7f4813fff640 1 -- 172.21.15.103:0/2596876386 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48200136e0 con 0x55df44f6bb40 2024-09-19T07:04:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.301+0000 7f482e5b7640 1 --2- 172.21.15.103:0/1850262098 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48140a33b0 0x7f48140a5870 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4820020000 tx=0x7f482001a000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.301+0000 7f48323f3740 1 -- 172.21.15.103:0/2596876386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6bb40 msgr2=0x55df44f6a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.301+0000 7f48323f3740 1 --2- 172.21.15.103:0/2596876386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6bb40 0x55df44f6a090 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f4820010420 tx=0x7f48200108e0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.301+0000 7f48323f3740 1 -- 172.21.15.103:0/2596876386 shutdown_connections 2024-09-19T07:04:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.301+0000 7f48323f3740 1 --2- 172.21.15.103:0/2596876386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f79780 0x55df44f7bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.301+0000 7f48323f3740 1 --2- 172.21.15.103:0/2596876386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f6a5d0 0x55df44f6aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.301+0000 7f48323f3740 1 --2- 172.21.15.103:0/2596876386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6bb40 0x55df44f6a090 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.301+0000 7f48323f3740 1 -- 172.21.15.103:0/2596876386 >> 172.21.15.103:0/2596876386 conn(0x55df44f33310 msgr2=0x55df44f66c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.301+0000 7f48323f3740 1 -- 172.21.15.103:0/2596876386 shutdown_connections 2024-09-19T07:04:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.301+0000 7f48323f3740 1 -- 172.21.15.103:0/2596876386 wait complete. 2024-09-19T07:04:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.302+0000 7f48323f3740 1 -- start start 2024-09-19T07:04:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.302+0000 7f48323f3740 1 Processor -- start 2024-09-19T07:04:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.302+0000 7f48323f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f6a5d0 0x55df44f791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.302+0000 7f48323f3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6bb40 0x55df44f764a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.302+0000 7f48323f3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f79780 0x55df44f76be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.302+0000 7f48323f3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55df44f7c480 con 0x55df44f6bb40 2024-09-19T07:04:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.302+0000 7f48323f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55df44e8b060 con 0x55df44f6a5d0 2024-09-19T07:04:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.302+0000 7f48323f3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55df44f7c600 con 0x55df44f79780 2024-09-19T07:04:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.302+0000 7f48323f3740 10 client.0 _ll_register_callbacks cb 0x55df44bde8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.302+0000 7f482d5b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6bb40 0x55df44f764a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.302+0000 7f482d5b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6bb40 0x55df44f764a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56902/0 (socket says 172.21.15.103:56902) 2024-09-19T07:04:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.302+0000 7f482d5b5640 1 -- 172.21.15.103:0/3840023439 learned_addr learned my addr 172.21.15.103:0/3840023439 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.302+0000 7f482e5b7640 1 --2- 172.21.15.103:0/3840023439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f79780 0x55df44f76be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.302+0000 7f482ddb6640 1 --2- 172.21.15.103:0/3840023439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f6a5d0 0x55df44f791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f482d5b5640 1 -- 172.21.15.103:0/3840023439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f79780 msgr2=0x55df44f76be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:45.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f482d5b5640 1 --2- 172.21.15.103:0/3840023439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f79780 0x55df44f76be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f482d5b5640 1 -- 172.21.15.103:0/3840023439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f6a5d0 msgr2=0x55df44f791f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:45.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f482d5b5640 1 --2- 172.21.15.103:0/3840023439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f6a5d0 0x55df44f791f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f482d5b5640 1 -- 172.21.15.103:0/3840023439 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df44c413c0 con 0x55df44f6bb40 2024-09-19T07:04:45.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f482e5b7640 1 --2- 172.21.15.103:0/3840023439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f79780 0x55df44f76be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:45.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f482d5b5640 1 --2- 172.21.15.103:0/3840023439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6bb40 0x55df44f764a0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f4818000c00 tx=0x7f481800a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4caef360311dcf01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:45.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f48117fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:45.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f48117fa640 1 -- 172.21.15.103:0/3840023439 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481801d030 con 0x55df44f6bb40 2024-09-19T07:04:45.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f48117fa640 1 -- 172.21.15.103:0/3840023439 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f4818007ca0 con 0x55df44f6bb40 2024-09-19T07:04:45.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f48323f3740 1 -- 172.21.15.103:0/3840023439 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f482001c470 con 0x55df44f6bb40 2024-09-19T07:04:45.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f48117fa640 1 -- 172.21.15.103:0/3840023439 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f481809c810 con 0x55df44f6bb40 2024-09-19T07:04:45.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f48323f3740 10 client.20802 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:45.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f48323f3740 1 -- 172.21.15.103:0/3840023439 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df44f88060 con 0x55df44f6bb40 2024-09-19T07:04:45.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f48117fa640 1 -- 172.21.15.103:0/3840023439 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f482001c470 con 0x55df44f6bb40 2024-09-19T07:04:45.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f48117fa640 1 -- 172.21.15.103:0/3840023439 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f48180b0210 con 0x55df44f6bb40 2024-09-19T07:04:45.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f48323f3740 10 client.20802 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:45.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f48323f3740 10 client.20802 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:04:45.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f48323f3740 20 client.20802 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:45.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f48323f3740 1 --2- 172.21.15.103:0/3840023439 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55df44f89520 0x55df44f8b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f48323f3740 4 client.20802 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:45.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.303+0000 7f48323f3740 1 -- 172.21.15.103:0/3840023439 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df44edd860 con 0x55df44f89520 2024-09-19T07:04:45.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.304+0000 7f482ddb6640 1 --2- 172.21.15.103:0/3840023439 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55df44f89520 0x55df44f8b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.304+0000 7f48117fa640 10 client.20802 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:04:45.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.304+0000 7f482ddb6640 1 --2- 172.21.15.103:0/3840023439 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55df44f89520 0x55df44f8b910 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8847f29935f2c28f server_cookie=9e225f6dccc74d8c in_seq=0 out_seq=0 2024-09-19T07:04:45.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.305+0000 7f48117fa640 1 -- 172.21.15.103:0/3840023439 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df44edd860 con 0x55df44f89520 2024-09-19T07:04:45.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.305+0000 7f48117fa640 10 client.20802 handle_command_reply: tid=0 2024-09-19T07:04:45.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.305+0000 7f48323f3740 1 client.20802 shutdown 2024-09-19T07:04:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.305+0000 7f48323f3740 1 -- 172.21.15.103:0/3840023439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6bb40 msgr2=0x55df44f764a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.305+0000 7f48323f3740 1 --2- 172.21.15.103:0/3840023439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6bb40 0x55df44f764a0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f4818000c00 tx=0x7f481800a520 comp rx=0 tx=0).stop 2024-09-19T07:04:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.305+0000 7f48323f3740 1 -- 172.21.15.103:0/3840023439 shutdown_connections 2024-09-19T07:04:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.305+0000 7f48323f3740 1 --2- 172.21.15.103:0/3840023439 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55df44f89520 0x55df44f8b910 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.305+0000 7f48323f3740 1 --2- 172.21.15.103:0/3840023439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f79780 0x55df44f76be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.305+0000 7f48323f3740 1 --2- 172.21.15.103:0/3840023439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6bb40 0x55df44f764a0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.305+0000 7f48117fa640 0 client.20802 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:04:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.305+0000 7f48323f3740 1 --2- 172.21.15.103:0/3840023439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f6a5d0 0x55df44f791f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.305+0000 7f48323f3740 1 -- 172.21.15.103:0/3840023439 >> 172.21.15.103:0/3840023439 conn(0x55df44f33310 msgr2=0x55df44f6c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.305+0000 7f48323f3740 1 -- 172.21.15.103:0/3840023439 shutdown_connections 2024-09-19T07:04:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.305+0000 7f48323f3740 1 -- 172.21.15.103:0/3840023439 wait complete. 2024-09-19T07:04:45.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.305+0000 7f48323f3740 20 client.20802 trim_cache size 0 max 16384 2024-09-19T07:04:45.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f48323f3740 1 Processor -- start 2024-09-19T07:04:45.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f48323f3740 1 -- start start 2024-09-19T07:04:45.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f48323f3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6a5d0 0x55df44f66700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f48323f3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f6bb40 0x55df44f85720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f48323f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f79780 0x55df44f91190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f48323f3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f48281ab740 con 0x55df44f6a5d0 2024-09-19T07:04:45.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f48323f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f48281ad280 con 0x55df44f79780 2024-09-19T07:04:45.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f48323f3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f48281abe10 con 0x55df44f6bb40 2024-09-19T07:04:45.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f482ddb6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6a5d0 0x55df44f66700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f482ddb6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6a5d0 0x55df44f66700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56916/0 (socket says 172.21.15.103:56916) 2024-09-19T07:04:45.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f482ddb6640 1 -- 172.21.15.103:0/2756510037 learned_addr learned my addr 172.21.15.103:0/2756510037 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:45.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f482d5b5640 1 --2- 172.21.15.103:0/2756510037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f6bb40 0x55df44f85720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f482e5b7640 1 --2- 172.21.15.103:0/2756510037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f79780 0x55df44f91190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f482ddb6640 1 -- 172.21.15.103:0/2756510037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f6bb40 msgr2=0x55df44f85720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:45.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f482ddb6640 1 --2- 172.21.15.103:0/2756510037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f6bb40 0x55df44f85720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f482ddb6640 1 -- 172.21.15.103:0/2756510037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f79780 msgr2=0x55df44f91190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:45.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f482ddb6640 1 --2- 172.21.15.103:0/2756510037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f79780 0x55df44f91190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f482ddb6640 1 -- 172.21.15.103:0/2756510037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df44f66ed0 con 0x55df44f6a5d0 2024-09-19T07:04:45.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.319+0000 7f482d5b5640 1 --2- 172.21.15.103:0/2756510037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f6bb40 0x55df44f85720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:45.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.320+0000 7f482ddb6640 1 --2- 172.21.15.103:0/2756510037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6a5d0 0x55df44f66700 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f4824031550 tx=0x7f4824004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e850d89e063b381b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:45.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.320+0000 7f48137fe640 1 -- 172.21.15.103:0/2756510037 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824048070 con 0x55df44f6a5d0 2024-09-19T07:04:45.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.320+0000 7f48137fe640 1 -- 172.21.15.103:0/2756510037 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4824005520 con 0x55df44f6a5d0 2024-09-19T07:04:45.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.320+0000 7f48137fe640 1 -- 172.21.15.103:0/2756510037 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824035070 con 0x55df44f6a5d0 2024-09-19T07:04:45.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.321+0000 7f48323f3740 1 -- 172.21.15.103:0/2756510037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6a5d0 msgr2=0x55df44f66700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:45.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.321+0000 7f48323f3740 1 --2- 172.21.15.103:0/2756510037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6a5d0 0x55df44f66700 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f4824031550 tx=0x7f4824004280 comp rx=0 tx=0).stop 2024-09-19T07:04:45.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.321+0000 7f48323f3740 1 -- 172.21.15.103:0/2756510037 shutdown_connections 2024-09-19T07:04:45.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.321+0000 7f48323f3740 1 --2- 172.21.15.103:0/2756510037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f79780 0x55df44f91190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.321+0000 7f48323f3740 1 --2- 172.21.15.103:0/2756510037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f6bb40 0x55df44f85720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.321+0000 7f48323f3740 1 --2- 172.21.15.103:0/2756510037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f6a5d0 0x55df44f66700 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.321+0000 7f48323f3740 1 -- 172.21.15.103:0/2756510037 >> 172.21.15.103:0/2756510037 conn(0x55df44f33310 msgr2=0x55df44f78520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:45.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.321+0000 7f48323f3740 1 -- 172.21.15.103:0/2756510037 shutdown_connections 2024-09-19T07:04:45.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.321+0000 7f48323f3740 1 -- 172.21.15.103:0/2756510037 wait complete. 2024-09-19T07:04:45.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.321+0000 7f48323f3740 1 -- start start 2024-09-19T07:04:45.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.321+0000 7f48323f3740 1 Processor -- start 2024-09-19T07:04:45.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f48323f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f6a5d0 0x55df44f7d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f48323f3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f6bb40 0x55df44f7dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f48323f3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f79780 0x55df44f7e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f48323f3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f48281acbb0 con 0x55df44f79780 2024-09-19T07:04:45.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f48323f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f48281ac4e0 con 0x55df44f6a5d0 2024-09-19T07:04:45.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f48323f3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f48281a9740 con 0x55df44f6bb40 2024-09-19T07:04:45.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f48323f3740 10 client.0 _ll_register_callbacks cb 0x7f48180057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:45.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f482e5b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f79780 0x55df44f7e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f482d5b5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f6bb40 0x55df44f7dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f482e5b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f79780 0x55df44f7e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56920/0 (socket says 172.21.15.103:56920) 2024-09-19T07:04:45.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f482e5b7640 1 -- 172.21.15.103:0/3476360871 learned_addr learned my addr 172.21.15.103:0/3476360871 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:45.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f482ddb6640 1 --2- 172.21.15.103:0/3476360871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f6a5d0 0x55df44f7d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f482e5b7640 1 -- 172.21.15.103:0/3476360871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f6bb40 msgr2=0x55df44f7dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:45.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f482e5b7640 1 --2- 172.21.15.103:0/3476360871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f6bb40 0x55df44f7dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f482e5b7640 1 -- 172.21.15.103:0/3476360871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f6a5d0 msgr2=0x55df44f7d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:45.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f482e5b7640 1 --2- 172.21.15.103:0/3476360871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f6a5d0 0x55df44f7d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f482e5b7640 1 -- 172.21.15.103:0/3476360871 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df44ff32c0 con 0x55df44f79780 2024-09-19T07:04:45.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f482d5b5640 1 --2- 172.21.15.103:0/3476360871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f6bb40 0x55df44f7dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:45.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f482e5b7640 1 --2- 172.21.15.103:0/3476360871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f79780 0x55df44f7e1c0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f4820005c40 tx=0x7f4820009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6264cf980b6a2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:45.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f48117fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:45.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f48117fa640 1 -- 172.21.15.103:0/3476360871 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820023040 con 0x55df44f79780 2024-09-19T07:04:45.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f48117fa640 1 -- 172.21.15.103:0/3476360871 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f482001dc00 con 0x55df44f79780 2024-09-19T07:04:45.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.322+0000 7f48323f3740 1 -- 172.21.15.103:0/3476360871 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4824005520 con 0x55df44f79780 2024-09-19T07:04:45.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.323+0000 7f48117fa640 1 -- 172.21.15.103:0/3476360871 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f48200136c0 con 0x55df44f79780 2024-09-19T07:04:45.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.323+0000 7f48323f3740 10 client.20808 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:45.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.323+0000 7f48323f3740 1 -- 172.21.15.103:0/3476360871 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df44ff4580 con 0x55df44f79780 2024-09-19T07:04:45.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.323+0000 7f48117fa640 1 -- 172.21.15.103:0/3476360871 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4824005520 con 0x55df44f79780 2024-09-19T07:04:45.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.323+0000 7f48117fa640 1 -- 172.21.15.103:0/3476360871 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4820031020 con 0x55df44f79780 2024-09-19T07:04:45.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.323+0000 7f48323f3740 10 client.20808 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:45.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.323+0000 7f48323f3740 10 client.20808 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:04:45.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.323+0000 7f48323f3740 20 client.20808 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:45.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.323+0000 7f48323f3740 1 --2- 172.21.15.103:0/3476360871 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55df44ff4b00 0x55df44ff6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:45.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.323+0000 7f48323f3740 4 client.20808 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:04:45.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.323+0000 7f48323f3740 1 -- 172.21.15.103:0/3476360871 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df44ed7d70 con 0x55df44ff4b00 2024-09-19T07:04:45.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.323+0000 7f482ddb6640 1 --2- 172.21.15.103:0/3476360871 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55df44ff4b00 0x55df44ff6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:45.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f482ddb6640 1 --2- 172.21.15.103:0/3476360871 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55df44ff4b00 0x55df44ff6ef0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=46168ff8ce3315d2 server_cookie=a4e25b74fe3e0524 in_seq=0 out_seq=0 2024-09-19T07:04:45.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f48117fa640 10 client.20808 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:04:45.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f48117fa640 1 -- 172.21.15.103:0/3476360871 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df44ed7d70 con 0x55df44ff4b00 2024-09-19T07:04:45.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f48117fa640 10 client.20808 handle_command_reply: tid=0 2024-09-19T07:04:45.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f48323f3740 1 client.20808 shutdown 2024-09-19T07:04:45.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f48323f3740 1 -- 172.21.15.103:0/3476360871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f79780 msgr2=0x55df44f7e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:45.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f48323f3740 1 --2- 172.21.15.103:0/3476360871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f79780 0x55df44f7e1c0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f4820005c40 tx=0x7f4820009c30 comp rx=0 tx=0).stop 2024-09-19T07:04:45.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f48323f3740 1 -- 172.21.15.103:0/3476360871 shutdown_connections 2024-09-19T07:04:45.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f48323f3740 1 --2- 172.21.15.103:0/3476360871 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55df44ff4b00 0x55df44ff6ef0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f48323f3740 1 --2- 172.21.15.103:0/3476360871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df44f79780 0x55df44f7e1c0 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f48323f3740 1 --2- 172.21.15.103:0/3476360871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df44f6bb40 0x55df44f7dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f48323f3740 1 --2- 172.21.15.103:0/3476360871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df44f6a5d0 0x55df44f7d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f48117fa640 0 client.20808 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:04:45.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f48323f3740 1 -- 172.21.15.103:0/3476360871 >> 172.21.15.103:0/3476360871 conn(0x55df44f33310 msgr2=0x55df44f74c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:45.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f48323f3740 1 -- 172.21.15.103:0/3476360871 shutdown_connections 2024-09-19T07:04:45.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f48323f3740 1 -- 172.21.15.103:0/3476360871 wait complete. 2024-09-19T07:04:45.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.324+0000 7f48323f3740 20 client.20808 trim_cache size 0 max 16384 2024-09-19T07:04:45.336 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:04:45.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.325+0000 7f483001a640 1 -- 172.21.15.103:0/1850262098 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48140a33b0 msgr2=0x7f48140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:45.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.325+0000 7f483001a640 1 --2- 172.21.15.103:0/1850262098 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48140a33b0 0x7f48140a5870 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4820020000 tx=0x7f482001a000 comp rx=0 tx=0).stop 2024-09-19T07:04:45.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.325+0000 7f483001a640 1 -- 172.21.15.103:0/1850262098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4828102790 msgr2=0x7f4828198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:45.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.325+0000 7f483001a640 1 --2- 172.21.15.103:0/1850262098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4828102790 0x7f4828198990 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f481800c890 tx=0x7f481800cd50 comp rx=0 tx=0).stop 2024-09-19T07:04:45.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.325+0000 7f483001a640 1 -- 172.21.15.103:0/1850262098 shutdown_connections 2024-09-19T07:04:45.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.325+0000 7f483001a640 1 --2- 172.21.15.103:0/1850262098 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f48140a33b0 0x7f48140a5870 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.325+0000 7f483001a640 1 --2- 172.21.15.103:0/1850262098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4828106f10 0x7f482819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.325+0000 7f483001a640 1 --2- 172.21.15.103:0/1850262098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4828102790 0x7f4828198990 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.325+0000 7f483001a640 1 --2- 172.21.15.103:0/1850262098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4828101de0 0x7f4828198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:45.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.325+0000 7f483001a640 1 -- 172.21.15.103:0/1850262098 >> 172.21.15.103:0/1850262098 conn(0x7f48280fd8c0 msgr2=0x7f48280fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:45.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.325+0000 7f483001a640 1 -- 172.21.15.103:0/1850262098 shutdown_connections 2024-09-19T07:04:45.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:45.325+0000 7f483001a640 1 -- 172.21.15.103:0/1850262098 wait complete. 2024-09-19T07:04:45.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:45 smithi174 ceph-mon[35637]: pgmap v486: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-09-19T07:04:45.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:45 smithi103 ceph-mon[31158]: pgmap v486: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-09-19T07:04:45.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:45 smithi134 ceph-mon[35356]: pgmap v486: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-09-19T07:04:45.999 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:04:46.245 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.719+0000 7f8b25a79640 1 -- 172.21.15.103:0/294512586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b201011c0 msgr2=0x7f8b20101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.719+0000 7f8b25a79640 1 --2- 172.21.15.103:0/294512586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b201011c0 0x7f8b20101620 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009900 tx=0x7f8b1402a410 comp rx=0 tx=0).stop 2024-09-19T07:04:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.719+0000 7f8b25a79640 1 -- 172.21.15.103:0/294512586 shutdown_connections 2024-09-19T07:04:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.719+0000 7f8b25a79640 1 --2- 172.21.15.103:0/294512586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b20101b60 0x7f8b2010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.719+0000 7f8b25a79640 1 --2- 172.21.15.103:0/294512586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b201011c0 0x7f8b20101620 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.719+0000 7f8b25a79640 1 --2- 172.21.15.103:0/294512586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b20105ad0 0x7f8b20105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.719+0000 7f8b25a79640 1 -- 172.21.15.103:0/294512586 >> 172.21.15.103:0/294512586 conn(0x7f8b20077210 msgr2=0x7f8b200fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:46.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.719+0000 7f8b25a79640 1 -- 172.21.15.103:0/294512586 shutdown_connections 2024-09-19T07:04:46.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.719+0000 7f8b25a79640 1 -- 172.21.15.103:0/294512586 wait complete. 2024-09-19T07:04:46.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.720+0000 7f8b25a79640 1 Processor -- start 2024-09-19T07:04:46.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.720+0000 7f8b25a79640 1 -- start start 2024-09-19T07:04:46.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.720+0000 7f8b25a79640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b201011c0 0x7f8b20198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.720+0000 7f8b25a79640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b20101b60 0x7f8b20198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.720+0000 7f8b25a79640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b20105ad0 0x7f8b2019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.720+0000 7f8b25a79640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8b20110ef0 con 0x7f8b20105ad0 2024-09-19T07:04:46.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.720+0000 7f8b25a79640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b20110d70 con 0x7f8b201011c0 2024-09-19T07:04:46.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.720+0000 7f8b25a79640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8b20111070 con 0x7f8b20101b60 2024-09-19T07:04:46.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.720+0000 7f8b1ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b20105ad0 0x7f8b2019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.720+0000 7f8b1ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b20105ad0 0x7f8b2019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56934/0 (socket says 172.21.15.103:56934) 2024-09-19T07:04:46.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.720+0000 7f8b1ffff640 1 -- 172.21.15.103:0/1459037147 learned_addr learned my addr 172.21.15.103:0/1459037147 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:46.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.720+0000 7f8b1f7fe640 1 --2- 172.21.15.103:0/1459037147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b201011c0 0x7f8b20198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.721+0000 7f8b1effd640 1 --2- 172.21.15.103:0/1459037147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b20101b60 0x7f8b20198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.721+0000 7f8b1ffff640 1 -- 172.21.15.103:0/1459037147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b20101b60 msgr2=0x7f8b20198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:46.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.721+0000 7f8b1ffff640 1 --2- 172.21.15.103:0/1459037147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b20101b60 0x7f8b20198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.721+0000 7f8b1ffff640 1 -- 172.21.15.103:0/1459037147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b201011c0 msgr2=0x7f8b20198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:46.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.721+0000 7f8b1ffff640 1 --2- 172.21.15.103:0/1459037147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b201011c0 0x7f8b20198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.721+0000 7f8b1ffff640 1 -- 172.21.15.103:0/1459037147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2019d3d0 con 0x7f8b20105ad0 2024-09-19T07:04:46.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.721+0000 7f8b1ffff640 1 --2- 172.21.15.103:0/1459037147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b20105ad0 0x7f8b2019ccf0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800e9d0 tx=0x7f8b1800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:46.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.721+0000 7f8b1cff9640 1 -- 172.21.15.103:0/1459037147 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1800cd80 con 0x7f8b20105ad0 2024-09-19T07:04:46.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.721+0000 7f8b25a79640 1 -- 172.21.15.103:0/1459037147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2019d6c0 con 0x7f8b20105ad0 2024-09-19T07:04:46.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.721+0000 7f8b1cff9640 1 -- 172.21.15.103:0/1459037147 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b18005020 con 0x7f8b20105ad0 2024-09-19T07:04:46.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.721+0000 7f8b1cff9640 1 -- 172.21.15.103:0/1459037147 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b180116d0 con 0x7f8b20105ad0 2024-09-19T07:04:46.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.721+0000 7f8b25a79640 1 -- 172.21.15.103:0/1459037147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b201a4fa0 con 0x7f8b20105ad0 2024-09-19T07:04:46.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.723+0000 7f8b1cff9640 1 -- 172.21.15.103:0/1459037147 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8b180051d0 con 0x7f8b20105ad0 2024-09-19T07:04:46.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b27e52740 1 Processor -- start 2024-09-19T07:04:46.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b27e52740 1 -- start start 2024-09-19T07:04:46.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b1cff9640 1 --2- 172.21.15.103:0/1459037147 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8b080a33b0 0x7f8b080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b27e52740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca04b40 0x558f2ca03090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b27e52740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca035d0 0x558f2ca03a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b27e52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca0e370 0x558f2ca10760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b27e52740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558f2c9692f0 con 0x558f2ca035d0 2024-09-19T07:04:46.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b27e52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558f2c910cf0 con 0x558f2ca0e370 2024-09-19T07:04:46.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b27e52740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558f2c9652b0 con 0x558f2ca04b40 2024-09-19T07:04:46.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b1cff9640 1 -- 172.21.15.103:0/1459037147 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f8b180d0330 con 0x7f8b20105ad0 2024-09-19T07:04:46.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b1effd640 1 --2- 172.21.15.103:0/1459037147 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8b080a33b0 0x7f8b080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b1effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca035d0 0x558f2ca03a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b1effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca035d0 0x558f2ca03a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56950/0 (socket says 172.21.15.103:56950) 2024-09-19T07:04:46.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b1effd640 1 -- 172.21.15.103:0/4082217316 learned_addr learned my addr 172.21.15.103:0/4082217316 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:46.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b1ffff640 1 --2- 172.21.15.103:0/4082217316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca0e370 0x558f2ca10760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b1f7fe640 1 --2- 172.21.15.103:0/4082217316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca04b40 0x558f2ca03090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b1ffff640 1 -- 172.21.15.103:0/4082217316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca04b40 msgr2=0x558f2ca03090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:46.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b1ffff640 1 --2- 172.21.15.103:0/4082217316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca04b40 0x558f2ca03090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b1ffff640 1 -- 172.21.15.103:0/4082217316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca035d0 msgr2=0x558f2ca03a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:46.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b1ffff640 1 --2- 172.21.15.103:0/4082217316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca035d0 0x558f2ca03a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b1ffff640 1 -- 172.21.15.103:0/4082217316 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f2c8f53a0 con 0x558f2ca0e370 2024-09-19T07:04:46.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.724+0000 7f8b1effd640 1 --2- 172.21.15.103:0/4082217316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca035d0 0x558f2ca03a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:46.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.725+0000 7f8b1effd640 1 --2- 172.21.15.103:0/1459037147 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8b080a33b0 0x7f8b080a5870 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x558f2ca00ac0 tx=0x7f8b14000bc0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:46.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.725+0000 7f8b1ffff640 1 --2- 172.21.15.103:0/4082217316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca0e370 0x558f2ca10760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8b180cda60 tx=0x7f8b1809c240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7d8ef69e73be089 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:46.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.725+0000 7f8b05ffb640 1 -- 172.21.15.103:0/4082217316 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1801f030 con 0x558f2ca0e370 2024-09-19T07:04:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.725+0000 7f8b05ffb640 1 -- 172.21.15.103:0/4082217316 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b1809ccf0 con 0x558f2ca0e370 2024-09-19T07:04:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.725+0000 7f8b05ffb640 1 -- 172.21.15.103:0/4082217316 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b18098c60 con 0x558f2ca0e370 2024-09-19T07:04:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.725+0000 7f8b27e52740 1 -- 172.21.15.103:0/4082217316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca0e370 msgr2=0x558f2ca10760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.725+0000 7f8b27e52740 1 --2- 172.21.15.103:0/4082217316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca0e370 0x558f2ca10760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8b180cda60 tx=0x7f8b1809c240 comp rx=0 tx=0).stop 2024-09-19T07:04:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.725+0000 7f8b27e52740 1 -- 172.21.15.103:0/4082217316 shutdown_connections 2024-09-19T07:04:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.725+0000 7f8b27e52740 1 --2- 172.21.15.103:0/4082217316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca0e370 0x558f2ca10760 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.725+0000 7f8b27e52740 1 --2- 172.21.15.103:0/4082217316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca035d0 0x558f2ca03a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.725+0000 7f8b27e52740 1 --2- 172.21.15.103:0/4082217316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca04b40 0x558f2ca03090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.725+0000 7f8b27e52740 1 -- 172.21.15.103:0/4082217316 >> 172.21.15.103:0/4082217316 conn(0x558f2c9cc310 msgr2=0x558f2c9ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.726+0000 7f8b27e52740 1 -- 172.21.15.103:0/4082217316 shutdown_connections 2024-09-19T07:04:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.726+0000 7f8b27e52740 1 -- 172.21.15.103:0/4082217316 wait complete. 2024-09-19T07:04:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.726+0000 7f8b27e52740 1 -- start start 2024-09-19T07:04:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.726+0000 7f8b27e52740 1 Processor -- start 2024-09-19T07:04:46.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.726+0000 7f8b27e52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca035d0 0x558f2ca0ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.726+0000 7f8b27e52740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca04b40 0x558f2ca0b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.726+0000 7f8b27e52740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca0e370 0x558f2ca0b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.726+0000 7f8b27e52740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558f2ca11010 con 0x558f2ca0e370 2024-09-19T07:04:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.726+0000 7f8b27e52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558f2c924060 con 0x558f2ca035d0 2024-09-19T07:04:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.726+0000 7f8b27e52740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558f2ca11190 con 0x558f2ca04b40 2024-09-19T07:04:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.726+0000 7f8b27e52740 10 client.0 _ll_register_callbacks cb 0x558f2c6778c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.726+0000 7f8b1ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca0e370 0x558f2ca0b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.726+0000 7f8b1ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca0e370 0x558f2ca0b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56964/0 (socket says 172.21.15.103:56964) 2024-09-19T07:04:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.726+0000 7f8b1ffff640 1 -- 172.21.15.103:0/508177560 learned_addr learned my addr 172.21.15.103:0/508177560 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8b1f7fe640 1 --2- 172.21.15.103:0/508177560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca035d0 0x558f2ca0ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8b1effd640 1 --2- 172.21.15.103:0/508177560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca04b40 0x558f2ca0b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8b1ffff640 1 -- 172.21.15.103:0/508177560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca04b40 msgr2=0x558f2ca0b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:46.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8b1ffff640 1 --2- 172.21.15.103:0/508177560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca04b40 0x558f2ca0b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8b1ffff640 1 -- 172.21.15.103:0/508177560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca035d0 msgr2=0x558f2ca0ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8b1ffff640 1 --2- 172.21.15.103:0/508177560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca035d0 0x558f2ca0ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8b1ffff640 1 -- 172.21.15.103:0/508177560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f2c6da3c0 con 0x558f2ca0e370 2024-09-19T07:04:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8b1effd640 1 --2- 172.21.15.103:0/508177560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca04b40 0x558f2ca0b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8b1ffff640 1 --2- 172.21.15.103:0/508177560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca0e370 0x558f2ca0b7f0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f8b180cdac0 tx=0x7f8b18098e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3c982bd565e3bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8afeffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8afeffd640 1 -- 172.21.15.103:0/508177560 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1801f040 con 0x558f2ca0e370 2024-09-19T07:04:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8afeffd640 1 -- 172.21.15.103:0/508177560 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f8b180a4fe0 con 0x558f2ca0e370 2024-09-19T07:04:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8b27e52740 1 -- 172.21.15.103:0/508177560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b1809ccf0 con 0x558f2ca0e370 2024-09-19T07:04:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8afeffd640 1 -- 172.21.15.103:0/508177560 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8b1809c5f0 con 0x558f2ca0e370 2024-09-19T07:04:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8b27e52740 10 client.20820 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:46.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8b27e52740 1 -- 172.21.15.103:0/508177560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558f2ca1cb00 con 0x558f2ca0e370 2024-09-19T07:04:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.727+0000 7f8afeffd640 1 -- 172.21.15.103:0/508177560 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b1809ccf0 con 0x558f2ca0e370 2024-09-19T07:04:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.728+0000 7f8afeffd640 1 -- 172.21.15.103:0/508177560 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8b1800f040 con 0x558f2ca0e370 2024-09-19T07:04:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.728+0000 7f8b27e52740 10 client.20820 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.728+0000 7f8b27e52740 10 client.20820 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:04:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.728+0000 7f8b27e52740 20 client.20820 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.728+0000 7f8b27e52740 1 --2- 172.21.15.103:0/508177560 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558f2ca1de50 0x558f2ca20240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.728+0000 7f8b27e52740 4 client.20820 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.728+0000 7f8b27e52740 1 -- 172.21.15.103:0/508177560 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f2c976860 con 0x558f2ca1de50 2024-09-19T07:04:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.728+0000 7f8b1f7fe640 1 --2- 172.21.15.103:0/508177560 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558f2ca1de50 0x558f2ca20240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.728+0000 7f8afeffd640 10 client.20820 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:04:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.728+0000 7f8b1f7fe640 1 --2- 172.21.15.103:0/508177560 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558f2ca1de50 0x558f2ca20240 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2202e2a84ed222bf server_cookie=a15f37605eded477 in_seq=0 out_seq=0 2024-09-19T07:04:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.729+0000 7f8afeffd640 1 -- 172.21.15.103:0/508177560 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f2c976860 con 0x558f2ca1de50 2024-09-19T07:04:46.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.729+0000 7f8afeffd640 10 client.20820 handle_command_reply: tid=0 2024-09-19T07:04:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.729+0000 7f8b27e52740 1 client.20820 shutdown 2024-09-19T07:04:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.729+0000 7f8b27e52740 1 -- 172.21.15.103:0/508177560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca0e370 msgr2=0x558f2ca0b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.729+0000 7f8b27e52740 1 --2- 172.21.15.103:0/508177560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca0e370 0x558f2ca0b7f0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f8b180cdac0 tx=0x7f8b18098e90 comp rx=0 tx=0).stop 2024-09-19T07:04:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.729+0000 7f8b27e52740 1 -- 172.21.15.103:0/508177560 shutdown_connections 2024-09-19T07:04:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.729+0000 7f8b27e52740 1 --2- 172.21.15.103:0/508177560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca0e370 0x558f2ca0b7f0 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.729+0000 7f8b27e52740 1 --2- 172.21.15.103:0/508177560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca04b40 0x558f2ca0b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.729+0000 7f8b27e52740 1 --2- 172.21.15.103:0/508177560 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558f2ca1de50 0x558f2ca20240 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.729+0000 7f8afeffd640 0 client.20820 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:04:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.729+0000 7f8b27e52740 1 --2- 172.21.15.103:0/508177560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca035d0 0x558f2ca0ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.729+0000 7f8b27e52740 1 -- 172.21.15.103:0/508177560 >> 172.21.15.103:0/508177560 conn(0x558f2c9cc310 msgr2=0x558f2ca059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.729+0000 7f8b27e52740 1 -- 172.21.15.103:0/508177560 shutdown_connections 2024-09-19T07:04:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.729+0000 7f8b27e52740 1 -- 172.21.15.103:0/508177560 wait complete. 2024-09-19T07:04:46.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.730+0000 7f8b27e52740 20 client.20820 trim_cache size 0 max 16384 2024-09-19T07:04:46.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.742+0000 7f8b27e52740 1 Processor -- start 2024-09-19T07:04:46.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.742+0000 7f8b27e52740 1 -- start start 2024-09-19T07:04:46.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.742+0000 7f8b27e52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca035d0 0x558f2c9ffd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.742+0000 7f8b27e52740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca04b40 0x558f2ca0a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.742+0000 7f8b27e52740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca0e370 0x558f2ca23650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.742+0000 7f8b27e52740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8b201ad920 con 0x558f2ca04b40 2024-09-19T07:04:46.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.742+0000 7f8b27e52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b201af460 con 0x558f2ca035d0 2024-09-19T07:04:46.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.742+0000 7f8b27e52740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8b201adff0 con 0x558f2ca0e370 2024-09-19T07:04:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.742+0000 7f8b1effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca04b40 0x558f2ca0a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.742+0000 7f8b1effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca04b40 0x558f2ca0a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56978/0 (socket says 172.21.15.103:56978) 2024-09-19T07:04:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.742+0000 7f8b1effd640 1 -- 172.21.15.103:0/3348900785 learned_addr learned my addr 172.21.15.103:0/3348900785 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.743+0000 7f8b1ffff640 1 --2- 172.21.15.103:0/3348900785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca0e370 0x558f2ca23650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.743+0000 7f8b1effd640 1 -- 172.21.15.103:0/3348900785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca0e370 msgr2=0x558f2ca23650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.743+0000 7f8b1effd640 1 --2- 172.21.15.103:0/3348900785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca0e370 0x558f2ca23650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.743+0000 7f8b1effd640 1 -- 172.21.15.103:0/3348900785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca035d0 msgr2=0x558f2c9ffd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.743+0000 7f8b1effd640 1 --2- 172.21.15.103:0/3348900785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca035d0 0x558f2c9ffd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.743+0000 7f8b1effd640 1 -- 172.21.15.103:0/3348900785 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f2ca0c3e0 con 0x558f2ca04b40 2024-09-19T07:04:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.743+0000 7f8b1effd640 1 --2- 172.21.15.103:0/3348900785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca04b40 0x558f2ca0a040 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f8b14005770 tx=0x7f8b14005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4ec1861bce40ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.743+0000 7f8b057fa640 1 -- 172.21.15.103:0/3348900785 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14048040 con 0x558f2ca04b40 2024-09-19T07:04:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.743+0000 7f8b057fa640 1 -- 172.21.15.103:0/3348900785 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b14051070 con 0x558f2ca04b40 2024-09-19T07:04:46.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.743+0000 7f8b057fa640 1 -- 172.21.15.103:0/3348900785 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14035040 con 0x558f2ca04b40 2024-09-19T07:04:46.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 -- 172.21.15.103:0/3348900785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca04b40 msgr2=0x558f2ca0a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:46.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 --2- 172.21.15.103:0/3348900785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca04b40 0x558f2ca0a040 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f8b14005770 tx=0x7f8b14005fd0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 -- 172.21.15.103:0/3348900785 shutdown_connections 2024-09-19T07:04:46.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 --2- 172.21.15.103:0/3348900785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca0e370 0x558f2ca23650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 --2- 172.21.15.103:0/3348900785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca04b40 0x558f2ca0a040 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 --2- 172.21.15.103:0/3348900785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca035d0 0x558f2c9ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 -- 172.21.15.103:0/3348900785 >> 172.21.15.103:0/3348900785 conn(0x558f2c9cc310 msgr2=0x558f2ca09930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:46.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 -- 172.21.15.103:0/3348900785 shutdown_connections 2024-09-19T07:04:46.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 -- 172.21.15.103:0/3348900785 wait complete. 2024-09-19T07:04:46.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 -- start start 2024-09-19T07:04:46.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 Processor -- start 2024-09-19T07:04:46.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca035d0 0x558f2ca7f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca04b40 0x558f2ca7fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca0e370 0x558f2ca83c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8b201aed90 con 0x558f2ca04b40 2024-09-19T07:04:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b201ae6c0 con 0x558f2ca0e370 2024-09-19T07:04:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.744+0000 7f8b27e52740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8b201ab920 con 0x558f2ca035d0 2024-09-19T07:04:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8b27e52740 10 client.0 _ll_register_callbacks cb 0x7f8b14009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8b1effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca04b40 0x558f2ca7fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8b1effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca04b40 0x558f2ca7fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56988/0 (socket says 172.21.15.103:56988) 2024-09-19T07:04:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8b1effd640 1 -- 172.21.15.103:0/2824037344 learned_addr learned my addr 172.21.15.103:0/2824037344 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8b1f7fe640 1 --2- 172.21.15.103:0/2824037344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca035d0 0x558f2ca7f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8b1ffff640 1 --2- 172.21.15.103:0/2824037344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca0e370 0x558f2ca83c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8b1effd640 1 -- 172.21.15.103:0/2824037344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca035d0 msgr2=0x558f2ca7f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8b1effd640 1 --2- 172.21.15.103:0/2824037344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca035d0 0x558f2ca7f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8b1effd640 1 -- 172.21.15.103:0/2824037344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca0e370 msgr2=0x558f2ca83c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8b1effd640 1 --2- 172.21.15.103:0/2824037344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca0e370 0x558f2ca83c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8b1effd640 1 -- 172.21.15.103:0/2824037344 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f2ca842c0 con 0x558f2ca04b40 2024-09-19T07:04:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8b1ffff640 1 --2- 172.21.15.103:0/2824037344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca0e370 0x558f2ca83c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8b1effd640 1 --2- 172.21.15.103:0/2824037344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca04b40 0x558f2ca7fb50 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f8b140048d0 tx=0x7f8b14005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54d2c5566015860 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8afeffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8afeffd640 1 -- 172.21.15.103:0/2824037344 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14035040 con 0x558f2ca04b40 2024-09-19T07:04:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8afeffd640 1 -- 172.21.15.103:0/2824037344 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f8b14003cd0 con 0x558f2ca04b40 2024-09-19T07:04:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.745+0000 7f8b27e52740 1 -- 172.21.15.103:0/2824037344 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b14051070 con 0x558f2ca04b40 2024-09-19T07:04:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.746+0000 7f8afeffd640 1 -- 172.21.15.103:0/2824037344 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8b14004970 con 0x558f2ca04b40 2024-09-19T07:04:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.746+0000 7f8b27e52740 10 client.20826 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.746+0000 7f8b27e52740 1 -- 172.21.15.103:0/2824037344 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558f2ca8c850 con 0x558f2ca04b40 2024-09-19T07:04:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.746+0000 7f8afeffd640 1 -- 172.21.15.103:0/2824037344 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b14051070 con 0x558f2ca04b40 2024-09-19T07:04:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.746+0000 7f8afeffd640 1 -- 172.21.15.103:0/2824037344 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8b1404b070 con 0x558f2ca04b40 2024-09-19T07:04:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.746+0000 7f8b27e52740 10 client.20826 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.746+0000 7f8b27e52740 10 client.20826 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:04:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.746+0000 7f8b27e52740 20 client.20826 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.746+0000 7f8b27e52740 1 --2- 172.21.15.103:0/2824037344 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558f2ca1de50 0x558f2ca8efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.746+0000 7f8b27e52740 4 client.20826 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:04:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.746+0000 7f8b27e52740 1 -- 172.21.15.103:0/2824037344 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f2c970d70 con 0x558f2ca1de50 2024-09-19T07:04:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.746+0000 7f8b1f7fe640 1 --2- 172.21.15.103:0/2824037344 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558f2ca1de50 0x558f2ca8efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.747+0000 7f8afeffd640 10 client.20826 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:04:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.747+0000 7f8b1f7fe640 1 --2- 172.21.15.103:0/2824037344 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558f2ca1de50 0x558f2ca8efb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f718a0c187654c07 server_cookie=95581d55c27de198 in_seq=0 out_seq=0 2024-09-19T07:04:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.747+0000 7f8afeffd640 1 -- 172.21.15.103:0/2824037344 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f2c970d70 con 0x558f2ca1de50 2024-09-19T07:04:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.747+0000 7f8afeffd640 10 client.20826 handle_command_reply: tid=0 2024-09-19T07:04:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.747+0000 7f8b27e52740 1 client.20826 shutdown 2024-09-19T07:04:46.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.747+0000 7f8b27e52740 1 -- 172.21.15.103:0/2824037344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca04b40 msgr2=0x558f2ca7fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:46.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.747+0000 7f8b27e52740 1 --2- 172.21.15.103:0/2824037344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca04b40 0x558f2ca7fb50 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f8b140048d0 tx=0x7f8b14005a40 comp rx=0 tx=0).stop 2024-09-19T07:04:46.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.747+0000 7f8b27e52740 1 -- 172.21.15.103:0/2824037344 shutdown_connections 2024-09-19T07:04:46.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.747+0000 7f8b27e52740 1 --2- 172.21.15.103:0/2824037344 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x558f2ca1de50 0x558f2ca8efb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.747+0000 7f8b27e52740 1 --2- 172.21.15.103:0/2824037344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f2ca0e370 0x558f2ca83c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.747+0000 7f8b27e52740 1 --2- 172.21.15.103:0/2824037344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f2ca04b40 0x558f2ca7fb50 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.747+0000 7f8b27e52740 1 --2- 172.21.15.103:0/2824037344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2ca035d0 0x558f2ca7f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.747+0000 7f8b27e52740 1 -- 172.21.15.103:0/2824037344 >> 172.21.15.103:0/2824037344 conn(0x558f2c9cc310 msgr2=0x558f2c9fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:46.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.747+0000 7f8afeffd640 0 client.20826 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:04:46.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.747+0000 7f8b27e52740 1 -- 172.21.15.103:0/2824037344 shutdown_connections 2024-09-19T07:04:46.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.747+0000 7f8b27e52740 1 -- 172.21.15.103:0/2824037344 wait complete. 2024-09-19T07:04:46.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.748+0000 7f8b27e52740 20 client.20826 trim_cache size 0 max 16384 2024-09-19T07:04:46.767 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:04:46.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.748+0000 7f8b25a79640 1 -- 172.21.15.103:0/1459037147 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8b080a33b0 msgr2=0x7f8b080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:46.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.748+0000 7f8b25a79640 1 --2- 172.21.15.103:0/1459037147 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8b080a33b0 0x7f8b080a5870 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x558f2ca00ac0 tx=0x7f8b14000bc0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.748+0000 7f8b25a79640 1 -- 172.21.15.103:0/1459037147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b20105ad0 msgr2=0x7f8b2019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:46.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.748+0000 7f8b25a79640 1 --2- 172.21.15.103:0/1459037147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b20105ad0 0x7f8b2019ccf0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800e9d0 tx=0x7f8b1800ee90 comp rx=0 tx=0).stop 2024-09-19T07:04:46.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.748+0000 7f8b25a79640 1 -- 172.21.15.103:0/1459037147 shutdown_connections 2024-09-19T07:04:46.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.748+0000 7f8b25a79640 1 --2- 172.21.15.103:0/1459037147 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8b080a33b0 0x7f8b080a5870 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.748+0000 7f8b25a79640 1 --2- 172.21.15.103:0/1459037147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b20105ad0 0x7f8b2019ccf0 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.748+0000 7f8b25a79640 1 --2- 172.21.15.103:0/1459037147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b20101b60 0x7f8b20198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.748+0000 7f8b25a79640 1 --2- 172.21.15.103:0/1459037147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b201011c0 0x7f8b20198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:46.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.748+0000 7f8b25a79640 1 -- 172.21.15.103:0/1459037147 >> 172.21.15.103:0/1459037147 conn(0x7f8b20077210 msgr2=0x7f8b20100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:46.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.748+0000 7f8b25a79640 1 -- 172.21.15.103:0/1459037147 shutdown_connections 2024-09-19T07:04:46.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:46.748+0000 7f8b25a79640 1 -- 172.21.15.103:0/1459037147 wait complete. 2024-09-19T07:04:47.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:47 smithi103 ceph-mon[31158]: pgmap v487: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-09-19T07:04:47.450 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:04:47.508 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:04:47.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:47 smithi174 ceph-mon[35637]: pgmap v487: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-09-19T07:04:47.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:47 smithi134 ceph-mon[35356]: pgmap v487: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-09-19T07:04:47.712 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:48.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.189+0000 7f4934a77640 1 -- 172.21.15.103:0/1243552123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4930100100 msgr2=0x7f4930100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:48.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.189+0000 7f4934a77640 1 --2- 172.21.15.103:0/1243552123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4930100100 0x7f4930100560 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f4924009900 tx=0x7f492402a410 comp rx=0 tx=0).stop 2024-09-19T07:04:48.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.189+0000 7f4934a77640 1 -- 172.21.15.103:0/1243552123 shutdown_connections 2024-09-19T07:04:48.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.189+0000 7f4934a77640 1 --2- 172.21.15.103:0/1243552123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4930100aa0 0x7f493010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:48.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.189+0000 7f4934a77640 1 --2- 172.21.15.103:0/1243552123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4930100100 0x7f4930100560 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:48.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.189+0000 7f4934a77640 1 --2- 172.21.15.103:0/1243552123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4930106b80 0x7f4930106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:48.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.189+0000 7f4934a77640 1 -- 172.21.15.103:0/1243552123 >> 172.21.15.103:0/1243552123 conn(0x7f4930077230 msgr2=0x7f49300fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:48.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.190+0000 7f4934a77640 1 -- 172.21.15.103:0/1243552123 shutdown_connections 2024-09-19T07:04:48.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.190+0000 7f4934a77640 1 -- 172.21.15.103:0/1243552123 wait complete. 2024-09-19T07:04:48.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.190+0000 7f4934a77640 1 Processor -- start 2024-09-19T07:04:48.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.190+0000 7f4934a77640 1 -- start start 2024-09-19T07:04:48.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.190+0000 7f4934a77640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4930100100 0x7f4930198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:48.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.190+0000 7f4934a77640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4930100aa0 0x7f4930198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:48.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.191+0000 7f4934a77640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4930106b80 0x7f493019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:48.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.191+0000 7f4934a77640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4930112070 con 0x7f4930106b80 2024-09-19T07:04:48.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.191+0000 7f4934a77640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4930111ef0 con 0x7f4930100100 2024-09-19T07:04:48.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.191+0000 7f4934a77640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f49301121f0 con 0x7f4930100aa0 2024-09-19T07:04:48.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.191+0000 7f492ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4930106b80 0x7f493019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:48.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.191+0000 7f492dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4930100aa0 0x7f4930198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:48.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.191+0000 7f492ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4930106b80 0x7f493019ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53070/0 (socket says 172.21.15.103:53070) 2024-09-19T07:04:48.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.191+0000 7f492ed9d640 1 -- 172.21.15.103:0/2741252867 learned_addr learned my addr 172.21.15.103:0/2741252867 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:48.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.191+0000 7f492e59c640 1 --2- 172.21.15.103:0/2741252867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4930100100 0x7f4930198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:48.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.191+0000 7f492ed9d640 1 -- 172.21.15.103:0/2741252867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4930100aa0 msgr2=0x7f4930198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:48.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.191+0000 7f492ed9d640 1 --2- 172.21.15.103:0/2741252867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4930100aa0 0x7f4930198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:48.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.191+0000 7f492ed9d640 1 -- 172.21.15.103:0/2741252867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4930100100 msgr2=0x7f4930198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:48.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.191+0000 7f492ed9d640 1 --2- 172.21.15.103:0/2741252867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4930100100 0x7f4930198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:48.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.191+0000 7f492ed9d640 1 -- 172.21.15.103:0/2741252867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493019d500 con 0x7f4930106b80 2024-09-19T07:04:48.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.191+0000 7f492ed9d640 1 --2- 172.21.15.103:0/2741252867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4930106b80 0x7f493019ce20 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f492800cad0 tx=0x7f492800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:48.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.191+0000 7f491f7fe640 1 -- 172.21.15.103:0/2741252867 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492800ad60 con 0x7f4930106b80 2024-09-19T07:04:48.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.192+0000 7f491f7fe640 1 -- 172.21.15.103:0/2741252867 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4928004030 con 0x7f4930106b80 2024-09-19T07:04:48.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.192+0000 7f4934a77640 1 -- 172.21.15.103:0/2741252867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493019d7f0 con 0x7f4930106b80 2024-09-19T07:04:48.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.192+0000 7f491f7fe640 1 -- 172.21.15.103:0/2741252867 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492800f6b0 con 0x7f4930106b80 2024-09-19T07:04:48.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.192+0000 7f4934a77640 1 -- 172.21.15.103:0/2741252867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49301a50d0 con 0x7f4930106b80 2024-09-19T07:04:48.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.193+0000 7f4934a77640 1 -- 172.21.15.103:0/2741252867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4930100560 con 0x7f4930106b80 2024-09-19T07:04:48.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.194+0000 7f491f7fe640 1 -- 172.21.15.103:0/2741252867 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f49280041e0 con 0x7f4930106b80 2024-09-19T07:04:48.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.194+0000 7f491f7fe640 1 --2- 172.21.15.103:0/2741252867 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f49180a33b0 0x7f49180a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:48.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.194+0000 7f491f7fe640 1 -- 172.21.15.103:0/2741252867 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f49280cfce0 con 0x7f4930106b80 2024-09-19T07:04:48.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.194+0000 7f492e59c640 1 --2- 172.21.15.103:0/2741252867 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f49180a33b0 0x7f49180a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:48.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.195+0000 7f492e59c640 1 --2- 172.21.15.103:0/2741252867 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f49180a33b0 0x7f49180a5870 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4920009990 tx=0x7f4920009110 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:48.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.199+0000 7f491f7fe640 1 -- 172.21.15.103:0/2741252867 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4928014070 con 0x7f4930106b80 2024-09-19T07:04:48.223 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:48.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.422+0000 7f4934a77640 1 -- 172.21.15.103:0/2741252867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49301a5330 con 0x7f4930106b80 2024-09-19T07:04:48.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.423+0000 7f491f7fe640 1 -- 172.21.15.103:0/2741252867 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f492809c6a0 con 0x7f4930106b80 2024-09-19T07:04:48.424 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:04:48.424 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:04:48.424 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:04:48.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.425+0000 7f4934a77640 1 -- 172.21.15.103:0/2741252867 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f49180a33b0 msgr2=0x7f49180a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:48.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.425+0000 7f4934a77640 1 --2- 172.21.15.103:0/2741252867 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f49180a33b0 0x7f49180a5870 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4920009990 tx=0x7f4920009110 comp rx=0 tx=0).stop 2024-09-19T07:04:48.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.425+0000 7f4934a77640 1 -- 172.21.15.103:0/2741252867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4930106b80 msgr2=0x7f493019ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:48.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.425+0000 7f4934a77640 1 --2- 172.21.15.103:0/2741252867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4930106b80 0x7f493019ce20 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f492800cad0 tx=0x7f492800ced0 comp rx=0 tx=0).stop 2024-09-19T07:04:48.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.425+0000 7f4934a77640 1 -- 172.21.15.103:0/2741252867 shutdown_connections 2024-09-19T07:04:48.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.425+0000 7f4934a77640 1 --2- 172.21.15.103:0/2741252867 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f49180a33b0 0x7f49180a5870 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:48.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.425+0000 7f4934a77640 1 --2- 172.21.15.103:0/2741252867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4930106b80 0x7f493019ce20 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:48.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.425+0000 7f4934a77640 1 --2- 172.21.15.103:0/2741252867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4930100aa0 0x7f4930198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:48.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.425+0000 7f4934a77640 1 --2- 172.21.15.103:0/2741252867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4930100100 0x7f4930198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:48.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.425+0000 7f4934a77640 1 -- 172.21.15.103:0/2741252867 >> 172.21.15.103:0/2741252867 conn(0x7f4930077230 msgr2=0x7f4930104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:48.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.425+0000 7f4934a77640 1 -- 172.21.15.103:0/2741252867 shutdown_connections 2024-09-19T07:04:48.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:48.425+0000 7f4934a77640 1 -- 172.21.15.103:0/2741252867 wait complete. 2024-09-19T07:04:49.087 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id a3c7bd9c --release --await' 2024-09-19T07:04:49.088 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id a3c7bd9c --release --await 2024-09-19T07:04:49.309 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:49 smithi103 ceph-mon[31158]: pgmap v488: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-09-19T07:04:49.310 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:49 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2741252867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:49.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.304+0000 7fef286eb640 1 -- 172.21.15.103:0/3626589999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef20100e20 msgr2=0x7fef20101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:49.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.304+0000 7fef286eb640 1 --2- 172.21.15.103:0/3626589999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef20100e20 0x7fef20101200 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009900 tx=0x7fef1c02a480 comp rx=0 tx=0).stop 2024-09-19T07:04:49.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.304+0000 7fef286eb640 1 -- 172.21.15.103:0/3626589999 shutdown_connections 2024-09-19T07:04:49.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.304+0000 7fef286eb640 1 --2- 172.21.15.103:0/3626589999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef20102ef0 0x7fef2010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.304+0000 7fef286eb640 1 --2- 172.21.15.103:0/3626589999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef20102570 0x7fef201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.304+0000 7fef286eb640 1 --2- 172.21.15.103:0/3626589999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef20100e20 0x7fef20101200 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.304+0000 7fef286eb640 1 -- 172.21.15.103:0/3626589999 >> 172.21.15.103:0/3626589999 conn(0x7fef200fc7d0 msgr2=0x7fef200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.304+0000 7fef286eb640 1 -- 172.21.15.103:0/3626589999 shutdown_connections 2024-09-19T07:04:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.304+0000 7fef286eb640 1 -- 172.21.15.103:0/3626589999 wait complete. 2024-09-19T07:04:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef286eb640 1 Processor -- start 2024-09-19T07:04:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef286eb640 1 -- start start 2024-09-19T07:04:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef286eb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef20100e20 0x7fef201985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef286eb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef20102570 0x7fef20198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef286eb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef20102ef0 0x7fef2019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef286eb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fef20112060 con 0x7fef20102ef0 2024-09-19T07:04:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef286eb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef20111ee0 con 0x7fef20102570 2024-09-19T07:04:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef286eb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef201121e0 con 0x7fef20100e20 2024-09-19T07:04:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef26c88640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef20102ef0 0x7fef2019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef26c88640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef20102ef0 0x7fef2019ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53090/0 (socket says 172.21.15.103:53090) 2024-09-19T07:04:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef26c88640 1 -- 172.21.15.103:0/182977896 learned_addr learned my addr 172.21.15.103:0/182977896 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef26487640 1 --2- 172.21.15.103:0/182977896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef20100e20 0x7fef201985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef26c88640 1 -- 172.21.15.103:0/182977896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef20100e20 msgr2=0x7fef201985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef26c88640 1 --2- 172.21.15.103:0/182977896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef20100e20 0x7fef201985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef26c88640 1 -- 172.21.15.103:0/182977896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef20102570 msgr2=0x7fef20198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef25c86640 1 --2- 172.21.15.103:0/182977896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef20102570 0x7fef20198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef26c88640 1 --2- 172.21.15.103:0/182977896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef20102570 0x7fef20198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef26c88640 1 -- 172.21.15.103:0/182977896 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef2019d590 con 0x7fef20102ef0 2024-09-19T07:04:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef26487640 1 --2- 172.21.15.103:0/182977896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef20100e20 0x7fef201985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.305+0000 7fef26c88640 1 --2- 172.21.15.103:0/182977896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef20102ef0 0x7fef2019ceb0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fef1800cad0 tx=0x7fef1800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.306+0000 7fef177fe640 1 -- 172.21.15.103:0/182977896 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1800ae20 con 0x7fef20102ef0 2024-09-19T07:04:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.306+0000 7fef177fe640 1 -- 172.21.15.103:0/182977896 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef18005020 con 0x7fef20102ef0 2024-09-19T07:04:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.306+0000 7fef177fe640 1 -- 172.21.15.103:0/182977896 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1800f6f0 con 0x7fef20102ef0 2024-09-19T07:04:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.306+0000 7fef286eb640 1 -- 172.21.15.103:0/182977896 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef2019d880 con 0x7fef20102ef0 2024-09-19T07:04:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.306+0000 7fef286eb640 1 -- 172.21.15.103:0/182977896 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef201a5160 con 0x7fef20102ef0 2024-09-19T07:04:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.308+0000 7fef2aac4740 1 Processor -- start 2024-09-19T07:04:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.309+0000 7fef177fe640 1 -- 172.21.15.103:0/182977896 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fef180051d0 con 0x7fef20102ef0 2024-09-19T07:04:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.309+0000 7fef2aac4740 1 -- start start 2024-09-19T07:04:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef2aac4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdcb40 0x55d329bdb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef2aac4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdb5d0 0x55d329bdba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef2aac4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329be6370 0x55d329be8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef2aac4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d329b412f0 con 0x55d329bdb5d0 2024-09-19T07:04:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef2aac4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d329ae8cf0 con 0x55d329be6370 2024-09-19T07:04:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef2aac4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d329b3d2b0 con 0x55d329bdcb40 2024-09-19T07:04:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef26487640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdb5d0 0x55d329bdba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef26487640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdb5d0 0x55d329bdba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53096/0 (socket says 172.21.15.103:53096) 2024-09-19T07:04:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef26487640 1 -- 172.21.15.103:0/1467687990 learned_addr learned my addr 172.21.15.103:0/1467687990 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef25c86640 1 --2- 172.21.15.103:0/1467687990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdcb40 0x55d329bdb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef25c86640 1 --2- 172.21.15.103:0/1467687990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329be6370 0x55d329be8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef177fe640 1 --2- 172.21.15.103:0/182977896 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fef0c0a3430 0x7fef0c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef26487640 1 -- 172.21.15.103:0/1467687990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdcb40 msgr2=0x55d329bdb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef26487640 1 --2- 172.21.15.103:0/1467687990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdcb40 0x55d329bdb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef26487640 1 -- 172.21.15.103:0/1467687990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329be6370 msgr2=0x55d329be8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef26487640 1 --2- 172.21.15.103:0/1467687990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329be6370 0x55d329be8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef26487640 1 -- 172.21.15.103:0/1467687990 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d329acd3a0 con 0x55d329bdb5d0 2024-09-19T07:04:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef25c86640 1 --2- 172.21.15.103:0/1467687990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329be6370 0x55d329be8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.310+0000 7fef26c88640 1 --2- 172.21.15.103:0/182977896 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fef0c0a3430 0x7fef0c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.311+0000 7fef177fe640 1 -- 172.21.15.103:0/182977896 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fef180cfd70 con 0x7fef20102ef0 2024-09-19T07:04:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.311+0000 7fef26487640 1 --2- 172.21.15.103:0/1467687990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdb5d0 0x55d329bdba10 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0314e0 tx=0x7fef1c004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9dc41046ef3eb3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.311+0000 7fef14ff9640 1 -- 172.21.15.103:0/1467687990 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c035030 con 0x55d329bdb5d0 2024-09-19T07:04:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.311+0000 7fef14ff9640 1 -- 172.21.15.103:0/1467687990 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef1c003ad0 con 0x55d329bdb5d0 2024-09-19T07:04:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.311+0000 7fef14ff9640 1 -- 172.21.15.103:0/1467687990 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c031c70 con 0x55d329bdb5d0 2024-09-19T07:04:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.312+0000 7fef2aac4740 1 -- 172.21.15.103:0/1467687990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdb5d0 msgr2=0x55d329bdba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.312+0000 7fef2aac4740 1 --2- 172.21.15.103:0/1467687990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdb5d0 0x55d329bdba10 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0314e0 tx=0x7fef1c004200 comp rx=0 tx=0).stop 2024-09-19T07:04:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.312+0000 7fef26c88640 1 --2- 172.21.15.103:0/182977896 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fef0c0a3430 0x7fef0c0a58f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fef1809bd30 tx=0x7fef18005960 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.312+0000 7fef2aac4740 1 -- 172.21.15.103:0/1467687990 shutdown_connections 2024-09-19T07:04:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.312+0000 7fef2aac4740 1 --2- 172.21.15.103:0/1467687990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329be6370 0x55d329be8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.312+0000 7fef2aac4740 1 --2- 172.21.15.103:0/1467687990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdb5d0 0x55d329bdba10 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.312+0000 7fef2aac4740 1 --2- 172.21.15.103:0/1467687990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdcb40 0x55d329bdb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.312+0000 7fef2aac4740 1 -- 172.21.15.103:0/1467687990 >> 172.21.15.103:0/1467687990 conn(0x55d329ba4310 msgr2=0x55d329bd7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.312+0000 7fef2aac4740 1 -- 172.21.15.103:0/1467687990 shutdown_connections 2024-09-19T07:04:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.312+0000 7fef2aac4740 1 -- 172.21.15.103:0/1467687990 wait complete. 2024-09-19T07:04:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.312+0000 7fef2aac4740 1 -- start start 2024-09-19T07:04:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.312+0000 7fef2aac4740 1 Processor -- start 2024-09-19T07:04:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef2aac4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdb5d0 0x55d329be5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef2aac4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329bdcb40 0x55d329be3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef2aac4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329be6370 0x55d329be35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef2aac4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d329be9010 con 0x55d329be6370 2024-09-19T07:04:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef2aac4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d329afc060 con 0x55d329bdcb40 2024-09-19T07:04:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef2aac4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d329be9190 con 0x55d329bdb5d0 2024-09-19T07:04:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef2aac4740 10 client.0 _ll_register_callbacks cb 0x55d32984f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef25c86640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329be6370 0x55d329be35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef25c86640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329be6370 0x55d329be35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53098/0 (socket says 172.21.15.103:53098) 2024-09-19T07:04:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef25c86640 1 -- 172.21.15.103:0/20782239 learned_addr learned my addr 172.21.15.103:0/20782239 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef26487640 1 --2- 172.21.15.103:0/20782239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329bdcb40 0x55d329be3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef25c86640 1 --2- 172.21.15.103:0/20782239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdb5d0 0x55d329be5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef25c86640 1 -- 172.21.15.103:0/20782239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdb5d0 msgr2=0x55d329be5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef25c86640 1 --2- 172.21.15.103:0/20782239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdb5d0 0x55d329be5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef25c86640 1 -- 172.21.15.103:0/20782239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329bdcb40 msgr2=0x55d329be3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef25c86640 1 --2- 172.21.15.103:0/20782239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329bdcb40 0x55d329be3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef25c86640 1 -- 172.21.15.103:0/20782239 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3298b23c0 con 0x55d329be6370 2024-09-19T07:04:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef25c86640 1 --2- 172.21.15.103:0/20782239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329be6370 0x55d329be35d0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fef10012bb0 tx=0x7fef100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9857fd018fe94ebe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef09ffb640 1 -- 172.21.15.103:0/20782239 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10011a60 con 0x55d329be6370 2024-09-19T07:04:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef2aac4740 1 -- 172.21.15.103:0/20782239 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef1c003ad0 con 0x55d329be6370 2024-09-19T07:04:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef09ffb640 1 -- 172.21.15.103:0/20782239 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fef10011c00 con 0x55d329be6370 2024-09-19T07:04:49.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.313+0000 7fef09ffb640 1 -- 172.21.15.103:0/20782239 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fef10011e00 con 0x55d329be6370 2024-09-19T07:04:49.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.314+0000 7fef2aac4740 10 client.20844 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:49.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.314+0000 7fef2aac4740 1 -- 172.21.15.103:0/20782239 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d329bf4af0 con 0x55d329be6370 2024-09-19T07:04:49.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.314+0000 7fef09ffb640 1 -- 172.21.15.103:0/20782239 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef1c003ad0 con 0x55d329be6370 2024-09-19T07:04:49.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.314+0000 7fef09ffb640 1 -- 172.21.15.103:0/20782239 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fef10004ca0 con 0x55d329be6370 2024-09-19T07:04:49.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.314+0000 7fef2aac4740 10 client.20844 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:49.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.314+0000 7fef2aac4740 10 client.20844 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:04:49.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.314+0000 7fef2aac4740 20 client.20844 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:49.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.314+0000 7fef2aac4740 1 --2- 172.21.15.103:0/20782239 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d329bf5f00 0x55d329bf82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.314+0000 7fef2aac4740 4 client.20844 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:49.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.314+0000 7fef2aac4740 1 -- 172.21.15.103:0/20782239 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d329b4e860 con 0x55d329bf5f00 2024-09-19T07:04:49.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.314+0000 7fef26487640 1 --2- 172.21.15.103:0/20782239 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d329bf5f00 0x55d329bf82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.315+0000 7fef09ffb640 10 client.20844 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:04:49.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.315+0000 7fef26487640 1 --2- 172.21.15.103:0/20782239 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d329bf5f00 0x55d329bf82f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a62f52cc41142ba server_cookie=c5d33fe2c8c4ba61 in_seq=0 out_seq=0 2024-09-19T07:04:49.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.315+0000 7fef09ffb640 1 -- 172.21.15.103:0/20782239 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d329b4e860 con 0x55d329bf5f00 2024-09-19T07:04:49.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.315+0000 7fef09ffb640 10 client.20844 handle_command_reply: tid=0 2024-09-19T07:04:49.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.316+0000 7fef2aac4740 1 client.20844 shutdown 2024-09-19T07:04:49.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.316+0000 7fef2aac4740 1 -- 172.21.15.103:0/20782239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329be6370 msgr2=0x55d329be35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:49.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.316+0000 7fef2aac4740 1 --2- 172.21.15.103:0/20782239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329be6370 0x55d329be35d0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fef10012bb0 tx=0x7fef100046c0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.316+0000 7fef2aac4740 1 -- 172.21.15.103:0/20782239 shutdown_connections 2024-09-19T07:04:49.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.316+0000 7fef2aac4740 1 --2- 172.21.15.103:0/20782239 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d329bf5f00 0x55d329bf82f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.316+0000 7fef09ffb640 0 client.20844 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:04:49.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.316+0000 7fef2aac4740 1 --2- 172.21.15.103:0/20782239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329be6370 0x55d329be35d0 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.316+0000 7fef2aac4740 1 --2- 172.21.15.103:0/20782239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329bdcb40 0x55d329be3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.316+0000 7fef2aac4740 1 --2- 172.21.15.103:0/20782239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdb5d0 0x55d329be5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.316+0000 7fef2aac4740 1 -- 172.21.15.103:0/20782239 >> 172.21.15.103:0/20782239 conn(0x55d329ba4310 msgr2=0x55d329bdd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:49.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.316+0000 7fef2aac4740 1 -- 172.21.15.103:0/20782239 shutdown_connections 2024-09-19T07:04:49.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.316+0000 7fef2aac4740 1 -- 172.21.15.103:0/20782239 wait complete. 2024-09-19T07:04:49.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.316+0000 7fef2aac4740 20 client.20844 trim_cache size 0 max 16384 2024-09-19T07:04:49.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.329+0000 7fef2aac4740 1 Processor -- start 2024-09-19T07:04:49.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.329+0000 7fef2aac4740 1 -- start start 2024-09-19T07:04:49.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.329+0000 7fef2aac4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdb5d0 0x55d329bf22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.329+0000 7fef2aac4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdcb40 0x55d329be7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.329+0000 7fef2aac4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329be6370 0x55d329be7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.329+0000 7fef2aac4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fef201ad8f0 con 0x55d329bdcb40 2024-09-19T07:04:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.329+0000 7fef2aac4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef201af430 con 0x55d329be6370 2024-09-19T07:04:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.329+0000 7fef2aac4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef201adfc0 con 0x55d329bdb5d0 2024-09-19T07:04:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.329+0000 7fef26487640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdcb40 0x55d329be7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.329+0000 7fef26487640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdcb40 0x55d329be7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53108/0 (socket says 172.21.15.103:53108) 2024-09-19T07:04:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.329+0000 7fef26487640 1 -- 172.21.15.103:0/1899059620 learned_addr learned my addr 172.21.15.103:0/1899059620 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.329+0000 7fef25c86640 1 --2- 172.21.15.103:0/1899059620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdb5d0 0x55d329bf22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.329+0000 7fef25c86640 1 --2- 172.21.15.103:0/1899059620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329be6370 0x55d329be7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.330+0000 7fef26487640 1 -- 172.21.15.103:0/1899059620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdb5d0 msgr2=0x55d329bf22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.330+0000 7fef26487640 1 --2- 172.21.15.103:0/1899059620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdb5d0 0x55d329bf22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.330+0000 7fef26487640 1 -- 172.21.15.103:0/1899059620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329be6370 msgr2=0x55d329be7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.330+0000 7fef26487640 1 --2- 172.21.15.103:0/1899059620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329be6370 0x55d329be7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.330+0000 7fef26487640 1 -- 172.21.15.103:0/1899059620 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d329bf8920 con 0x55d329bdcb40 2024-09-19T07:04:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.330+0000 7fef26487640 1 --2- 172.21.15.103:0/1899059620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdcb40 0x55d329be7760 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fef1c006010 tx=0x7fef1c031870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156b1078fa610599 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.330+0000 7fef0bfff640 1 -- 172.21.15.103:0/1899059620 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c054030 con 0x55d329bdcb40 2024-09-19T07:04:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.330+0000 7fef0bfff640 1 -- 172.21.15.103:0/1899059620 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef1c031b90 con 0x55d329bdcb40 2024-09-19T07:04:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.330+0000 7fef0bfff640 1 -- 172.21.15.103:0/1899059620 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c035040 con 0x55d329bdcb40 2024-09-19T07:04:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.331+0000 7fef2aac4740 1 -- 172.21.15.103:0/1899059620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdcb40 msgr2=0x55d329be7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.331+0000 7fef2aac4740 1 --2- 172.21.15.103:0/1899059620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdcb40 0x55d329be7760 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fef1c006010 tx=0x7fef1c031870 comp rx=0 tx=0).stop 2024-09-19T07:04:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.331+0000 7fef2aac4740 1 -- 172.21.15.103:0/1899059620 shutdown_connections 2024-09-19T07:04:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.331+0000 7fef2aac4740 1 --2- 172.21.15.103:0/1899059620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329be6370 0x55d329be7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.331+0000 7fef2aac4740 1 --2- 172.21.15.103:0/1899059620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdcb40 0x55d329be7760 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.331+0000 7fef2aac4740 1 --2- 172.21.15.103:0/1899059620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdb5d0 0x55d329bf22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.331+0000 7fef2aac4740 1 -- 172.21.15.103:0/1899059620 >> 172.21.15.103:0/1899059620 conn(0x55d329ba4310 msgr2=0x55d329bde230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.331+0000 7fef2aac4740 1 -- 172.21.15.103:0/1899059620 shutdown_connections 2024-09-19T07:04:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.331+0000 7fef2aac4740 1 -- 172.21.15.103:0/1899059620 wait complete. 2024-09-19T07:04:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.331+0000 7fef2aac4740 1 -- start start 2024-09-19T07:04:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.331+0000 7fef2aac4740 1 Processor -- start 2024-09-19T07:04:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.331+0000 7fef2aac4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdb5d0 0x55d329bfd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.331+0000 7fef2aac4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdcb40 0x55d329bfd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.331+0000 7fef2aac4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329be6370 0x55d329bed420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.331+0000 7fef2aac4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fef201aed60 con 0x55d329bdb5d0 2024-09-19T07:04:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef2aac4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef201ae690 con 0x55d329be6370 2024-09-19T07:04:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef2aac4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef201ab8f0 con 0x55d329bdcb40 2024-09-19T07:04:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef25c86640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdb5d0 0x55d329bfd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef2aac4740 10 client.0 _ll_register_callbacks cb 0x7fef1000f860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef25c86640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdb5d0 0x55d329bfd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53110/0 (socket says 172.21.15.103:53110) 2024-09-19T07:04:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef25c86640 1 -- 172.21.15.103:0/3658447854 learned_addr learned my addr 172.21.15.103:0/3658447854 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef26487640 1 --2- 172.21.15.103:0/3658447854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdcb40 0x55d329bfd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef25c86640 1 --2- 172.21.15.103:0/3658447854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329be6370 0x55d329bed420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef25c86640 1 -- 172.21.15.103:0/3658447854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdcb40 msgr2=0x55d329bfd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef25c86640 1 --2- 172.21.15.103:0/3658447854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdcb40 0x55d329bfd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef25c86640 1 -- 172.21.15.103:0/3658447854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329be6370 msgr2=0x55d329bed420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef25c86640 1 --2- 172.21.15.103:0/3658447854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329be6370 0x55d329bed420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef25c86640 1 -- 172.21.15.103:0/3658447854 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d329bdc3a0 con 0x55d329bdb5d0 2024-09-19T07:04:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef25c86640 1 --2- 172.21.15.103:0/3658447854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdb5d0 0x55d329bfd2e0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fef10012bb0 tx=0x7fef100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dad55c2a6fea7ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef09ffb640 1 -- 172.21.15.103:0/3658447854 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1002b050 con 0x55d329bdb5d0 2024-09-19T07:04:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef2aac4740 1 -- 172.21.15.103:0/3658447854 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef1c031b90 con 0x55d329bdb5d0 2024-09-19T07:04:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef09ffb640 1 -- 172.21.15.103:0/3658447854 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fef1001f870 con 0x55d329bdb5d0 2024-09-19T07:04:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef09ffb640 1 -- 172.21.15.103:0/3658447854 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fef10005a10 con 0x55d329bdb5d0 2024-09-19T07:04:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef2aac4740 10 client.20850 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.332+0000 7fef2aac4740 1 -- 172.21.15.103:0/3658447854 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d329beeae0 con 0x55d329bdb5d0 2024-09-19T07:04:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.333+0000 7fef09ffb640 1 -- 172.21.15.103:0/3658447854 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef1c031b90 con 0x55d329bdb5d0 2024-09-19T07:04:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.333+0000 7fef09ffb640 1 -- 172.21.15.103:0/3658447854 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fef10004730 con 0x55d329bdb5d0 2024-09-19T07:04:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.333+0000 7fef2aac4740 10 client.20850 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.333+0000 7fef2aac4740 10 client.20850 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:04:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.333+0000 7fef2aac4740 20 client.20850 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.333+0000 7fef2aac4740 1 --2- 172.21.15.103:0/3658447854 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d329c64f30 0x55d329c67320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.333+0000 7fef2aac4740 4 client.20850 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:04:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.333+0000 7fef2aac4740 1 -- 172.21.15.103:0/3658447854 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d329b48d70 con 0x55d329c64f30 2024-09-19T07:04:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.333+0000 7fef26487640 1 --2- 172.21.15.103:0/3658447854 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d329c64f30 0x55d329c67320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.333+0000 7fef26487640 1 --2- 172.21.15.103:0/3658447854 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d329c64f30 0x55d329c67320 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9935e7225edde02c server_cookie=72f7844f6442aba7 in_seq=0 out_seq=0 2024-09-19T07:04:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.333+0000 7fef09ffb640 10 client.20850 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:04:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef09ffb640 1 -- 172.21.15.103:0/3658447854 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d329b48d70 con 0x55d329c64f30 2024-09-19T07:04:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef09ffb640 10 client.20850 handle_command_reply: tid=0 2024-09-19T07:04:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef2aac4740 1 client.20850 shutdown 2024-09-19T07:04:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef2aac4740 1 -- 172.21.15.103:0/3658447854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdb5d0 msgr2=0x55d329bfd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef2aac4740 1 --2- 172.21.15.103:0/3658447854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdb5d0 0x55d329bfd2e0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fef10012bb0 tx=0x7fef100046c0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef2aac4740 1 -- 172.21.15.103:0/3658447854 shutdown_connections 2024-09-19T07:04:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef2aac4740 1 --2- 172.21.15.103:0/3658447854 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d329c64f30 0x55d329c67320 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef2aac4740 1 --2- 172.21.15.103:0/3658447854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d329be6370 0x55d329bed420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef2aac4740 1 --2- 172.21.15.103:0/3658447854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d329bdcb40 0x55d329bfd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef2aac4740 1 --2- 172.21.15.103:0/3658447854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d329bdb5d0 0x55d329bfd2e0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef09ffb640 0 client.20850 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:04:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef2aac4740 1 -- 172.21.15.103:0/3658447854 >> 172.21.15.103:0/3658447854 conn(0x55d329ba4310 msgr2=0x55d329be1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef2aac4740 1 -- 172.21.15.103:0/3658447854 shutdown_connections 2024-09-19T07:04:49.357 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:04:49.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef2aac4740 1 -- 172.21.15.103:0/3658447854 wait complete. 2024-09-19T07:04:49.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef2aac4740 20 client.20850 trim_cache size 0 max 16384 2024-09-19T07:04:49.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef286eb640 1 -- 172.21.15.103:0/182977896 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fef0c0a3430 msgr2=0x7fef0c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:49.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef286eb640 1 --2- 172.21.15.103:0/182977896 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fef0c0a3430 0x7fef0c0a58f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fef1809bd30 tx=0x7fef18005960 comp rx=0 tx=0).stop 2024-09-19T07:04:49.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef286eb640 1 -- 172.21.15.103:0/182977896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef20102ef0 msgr2=0x7fef2019ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef286eb640 1 --2- 172.21.15.103:0/182977896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef20102ef0 0x7fef2019ceb0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fef1800cad0 tx=0x7fef1800cf90 comp rx=0 tx=0).stop 2024-09-19T07:04:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef286eb640 1 -- 172.21.15.103:0/182977896 shutdown_connections 2024-09-19T07:04:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef286eb640 1 --2- 172.21.15.103:0/182977896 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fef0c0a3430 0x7fef0c0a58f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef286eb640 1 --2- 172.21.15.103:0/182977896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef20102ef0 0x7fef2019ceb0 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.334+0000 7fef286eb640 1 --2- 172.21.15.103:0/182977896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef20102570 0x7fef20198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.335+0000 7fef286eb640 1 --2- 172.21.15.103:0/182977896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef20100e20 0x7fef201985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.335+0000 7fef286eb640 1 -- 172.21.15.103:0/182977896 >> 172.21.15.103:0/182977896 conn(0x7fef200fc7d0 msgr2=0x7fef200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.335+0000 7fef286eb640 1 -- 172.21.15.103:0/182977896 shutdown_connections 2024-09-19T07:04:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:49.335+0000 7fef286eb640 1 -- 172.21.15.103:0/182977896 wait complete. 2024-09-19T07:04:49.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:49 smithi174 ceph-mon[35637]: pgmap v488: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-09-19T07:04:49.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:49 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2741252867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:49.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:49 smithi134 ceph-mon[35356]: pgmap v488: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-09-19T07:04:49.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:49 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2741252867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:50.031 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:04:50.078 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:50.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.561+0000 7ff98164e640 1 -- 172.21.15.103:0/2430745786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106b80 msgr2=0x7ff97c106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:50.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.561+0000 7ff98164e640 1 --2- 172.21.15.103:0/2430745786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106b80 0x7ff97c106f60 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009900 tx=0x7ff96c02a410 comp rx=0 tx=0).stop 2024-09-19T07:04:50.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.561+0000 7ff98164e640 1 -- 172.21.15.103:0/2430745786 shutdown_connections 2024-09-19T07:04:50.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.561+0000 7ff98164e640 1 --2- 172.21.15.103:0/2430745786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff97c100aa0 0x7ff97c10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.561+0000 7ff98164e640 1 --2- 172.21.15.103:0/2430745786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff97c100100 0x7ff97c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.561+0000 7ff98164e640 1 --2- 172.21.15.103:0/2430745786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106b80 0x7ff97c106f60 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.561+0000 7ff98164e640 1 -- 172.21.15.103:0/2430745786 >> 172.21.15.103:0/2430745786 conn(0x7ff97c077230 msgr2=0x7ff97c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:50.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.561+0000 7ff98164e640 1 -- 172.21.15.103:0/2430745786 shutdown_connections 2024-09-19T07:04:50.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.561+0000 7ff98164e640 1 -- 172.21.15.103:0/2430745786 wait complete. 2024-09-19T07:04:50.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.562+0000 7ff98164e640 1 Processor -- start 2024-09-19T07:04:50.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.562+0000 7ff98164e640 1 -- start start 2024-09-19T07:04:50.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.562+0000 7ff98164e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff97c100100 0x7ff97c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.562+0000 7ff98164e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff97c100aa0 0x7ff97c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.562+0000 7ff98164e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106b80 0x7ff97c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.562+0000 7ff98164e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c112050 con 0x7ff97c106b80 2024-09-19T07:04:50.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.562+0000 7ff98164e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c111ed0 con 0x7ff97c100100 2024-09-19T07:04:50.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.562+0000 7ff98164e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c1121d0 con 0x7ff97c100aa0 2024-09-19T07:04:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.563+0000 7ff97b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106b80 0x7ff97c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.563+0000 7ff97b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106b80 0x7ff97c19cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53130/0 (socket says 172.21.15.103:53130) 2024-09-19T07:04:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.563+0000 7ff97affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff97c100100 0x7ff97c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.563+0000 7ff97b7fe640 1 -- 172.21.15.103:0/2633405040 learned_addr learned my addr 172.21.15.103:0/2633405040 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.563+0000 7ff97a7fc640 1 --2- 172.21.15.103:0/2633405040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff97c100aa0 0x7ff97c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.563+0000 7ff97b7fe640 1 -- 172.21.15.103:0/2633405040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff97c100aa0 msgr2=0x7ff97c198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.563+0000 7ff97b7fe640 1 --2- 172.21.15.103:0/2633405040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff97c100aa0 0x7ff97c198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.563+0000 7ff97b7fe640 1 -- 172.21.15.103:0/2633405040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff97c100100 msgr2=0x7ff97c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.563+0000 7ff97b7fe640 1 --2- 172.21.15.103:0/2633405040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff97c100100 0x7ff97c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.563+0000 7ff97b7fe640 1 -- 172.21.15.103:0/2633405040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97c19d4a0 con 0x7ff97c106b80 2024-09-19T07:04:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.563+0000 7ff97b7fe640 1 --2- 172.21.15.103:0/2633405040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106b80 0x7ff97c19cdc0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7ff97400e9d0 tx=0x7ff97400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.563+0000 7ff963fff640 1 -- 172.21.15.103:0/2633405040 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97400cd80 con 0x7ff97c106b80 2024-09-19T07:04:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.564+0000 7ff98164e640 1 -- 172.21.15.103:0/2633405040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97c19d790 con 0x7ff97c106b80 2024-09-19T07:04:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.564+0000 7ff963fff640 1 -- 172.21.15.103:0/2633405040 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff974005020 con 0x7ff97c106b80 2024-09-19T07:04:50.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.564+0000 7ff963fff640 1 -- 172.21.15.103:0/2633405040 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9740116d0 con 0x7ff97c106b80 2024-09-19T07:04:50.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.564+0000 7ff98164e640 1 -- 172.21.15.103:0/2633405040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97c1a5070 con 0x7ff97c106b80 2024-09-19T07:04:50.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.567+0000 7ff963fff640 1 -- 172.21.15.103:0/2633405040 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff9740051d0 con 0x7ff97c106b80 2024-09-19T07:04:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.567+0000 7ff983a27740 1 Processor -- start 2024-09-19T07:04:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.567+0000 7ff983a27740 1 -- start start 2024-09-19T07:04:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.567+0000 7ff963fff640 1 --2- 172.21.15.103:0/2633405040 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9640a3430 0x7ff9640a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.567+0000 7ff983a27740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa913060 0x5652aa913440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.567+0000 7ff983a27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913980 0x5652aa91dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.567+0000 7ff983a27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa91e240 0x5652aa920630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.567+0000 7ff983a27740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5652aa878620 con 0x5652aa913060 2024-09-19T07:04:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.567+0000 7ff983a27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5652aa81fe50 con 0x5652aa91e240 2024-09-19T07:04:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.567+0000 7ff983a27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5652aa8745e0 con 0x5652aa913980 2024-09-19T07:04:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.568+0000 7ff963fff640 1 -- 172.21.15.103:0/2633405040 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7ff9740d0150 con 0x7ff97c106b80 2024-09-19T07:04:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.568+0000 7ff97affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa913060 0x5652aa913440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.568+0000 7ff97affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa913060 0x5652aa913440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53142/0 (socket says 172.21.15.103:53142) 2024-09-19T07:04:50.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.568+0000 7ff97affd640 1 -- 172.21.15.103:0/1091980425 learned_addr learned my addr 172.21.15.103:0/1091980425 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:50.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.568+0000 7ff97a7fc640 1 --2- 172.21.15.103:0/2633405040 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9640a3430 0x7ff9640a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.568+0000 7ff97b7fe640 1 --2- 172.21.15.103:0/1091980425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa91e240 0x5652aa920630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.568+0000 7ff97a7fc640 1 --2- 172.21.15.103:0/1091980425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913980 0x5652aa91dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.568+0000 7ff97affd640 1 -- 172.21.15.103:0/1091980425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913980 msgr2=0x5652aa91dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:50.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.568+0000 7ff97affd640 1 --2- 172.21.15.103:0/1091980425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913980 0x5652aa91dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.568+0000 7ff97affd640 1 -- 172.21.15.103:0/1091980425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa91e240 msgr2=0x5652aa920630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:50.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.568+0000 7ff97affd640 1 --2- 172.21.15.103:0/1091980425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa91e240 0x5652aa920630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.568+0000 7ff97affd640 1 -- 172.21.15.103:0/1091980425 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652aa810d40 con 0x5652aa913060 2024-09-19T07:04:50.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.568+0000 7ff97a7fc640 1 --2- 172.21.15.103:0/2633405040 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9640a3430 0x7ff9640a58f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x5652aa910450 tx=0x7ff970009920 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:50.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.568+0000 7ff97affd640 1 --2- 172.21.15.103:0/1091980425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa913060 0x5652aa913440 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009900 tx=0x7ff96c005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fdc95ec768674 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:50.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.569+0000 7ff9617fa640 1 -- 172.21.15.103:0/1091980425 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c040070 con 0x5652aa913060 2024-09-19T07:04:50.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.569+0000 7ff9617fa640 1 -- 172.21.15.103:0/1091980425 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff96c0318f0 con 0x5652aa913060 2024-09-19T07:04:50.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.569+0000 7ff9617fa640 1 -- 172.21.15.103:0/1091980425 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c035040 con 0x5652aa913060 2024-09-19T07:04:50.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.570+0000 7ff983a27740 1 -- 172.21.15.103:0/1091980425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa913060 msgr2=0x5652aa913440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:50.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.570+0000 7ff983a27740 1 --2- 172.21.15.103:0/1091980425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa913060 0x5652aa913440 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009900 tx=0x7ff96c005e00 comp rx=0 tx=0).stop 2024-09-19T07:04:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.570+0000 7ff983a27740 1 -- 172.21.15.103:0/1091980425 shutdown_connections 2024-09-19T07:04:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.570+0000 7ff983a27740 1 --2- 172.21.15.103:0/1091980425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa91e240 0x5652aa920630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.570+0000 7ff983a27740 1 --2- 172.21.15.103:0/1091980425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913980 0x5652aa91dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.570+0000 7ff983a27740 1 --2- 172.21.15.103:0/1091980425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa913060 0x5652aa913440 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.570+0000 7ff983a27740 1 -- 172.21.15.103:0/1091980425 >> 172.21.15.103:0/1091980425 conn(0x5652aa90f5f0 msgr2=0x5652aa90f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.570+0000 7ff983a27740 1 -- 172.21.15.103:0/1091980425 shutdown_connections 2024-09-19T07:04:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.570+0000 7ff983a27740 1 -- 172.21.15.103:0/1091980425 wait complete. 2024-09-19T07:04:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.570+0000 7ff983a27740 1 -- start start 2024-09-19T07:04:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff983a27740 1 Processor -- start 2024-09-19T07:04:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff983a27740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa913060 0x5652aa91b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff983a27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa913980 0x5652aa91b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff983a27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa91e240 0x5652aa91bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff983a27740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5652aa920e10 con 0x5652aa913060 2024-09-19T07:04:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff983a27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5652aa800f10 con 0x5652aa913980 2024-09-19T07:04:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff983a27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5652aa920f90 con 0x5652aa91e240 2024-09-19T07:04:50.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff983a27740 10 client.0 _ll_register_callbacks cb 0x5652aa824db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff97affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa913060 0x5652aa91b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff97affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa913060 0x5652aa91b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53154/0 (socket says 172.21.15.103:53154) 2024-09-19T07:04:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff97affd640 1 -- 172.21.15.103:0/2854354484 learned_addr learned my addr 172.21.15.103:0/2854354484 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff97b7fe640 1 --2- 172.21.15.103:0/2854354484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa91e240 0x5652aa91bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff97a7fc640 1 --2- 172.21.15.103:0/2854354484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa913980 0x5652aa91b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff97affd640 1 -- 172.21.15.103:0/2854354484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa91e240 msgr2=0x5652aa91bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff97affd640 1 --2- 172.21.15.103:0/2854354484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa91e240 0x5652aa91bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff97affd640 1 -- 172.21.15.103:0/2854354484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa913980 msgr2=0x5652aa91b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff97affd640 1 --2- 172.21.15.103:0/2854354484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa913980 0x5652aa91b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.571+0000 7ff97affd640 1 -- 172.21.15.103:0/2854354484 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652aa587580 con 0x5652aa913060 2024-09-19T07:04:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.572+0000 7ff97affd640 1 --2- 172.21.15.103:0/2854354484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa913060 0x5652aa91b160 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0042f0 tx=0x7ff96c005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fb97f86181d97a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.572+0000 7ff95eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.572+0000 7ff95eff5640 1 -- 172.21.15.103:0/2854354484 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c04b070 con 0x5652aa913060 2024-09-19T07:04:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.572+0000 7ff95eff5640 1 -- 172.21.15.103:0/2854354484 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7ff96c004450 con 0x5652aa913060 2024-09-19T07:04:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.572+0000 7ff983a27740 1 -- 172.21.15.103:0/2854354484 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff96c0318f0 con 0x5652aa913060 2024-09-19T07:04:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.572+0000 7ff95eff5640 1 -- 172.21.15.103:0/2854354484 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff96c057050 con 0x5652aa913060 2024-09-19T07:04:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.572+0000 7ff983a27740 10 client.20862 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:50.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.572+0000 7ff983a27740 1 -- 172.21.15.103:0/2854354484 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5652aa651960 con 0x5652aa913060 2024-09-19T07:04:50.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.572+0000 7ff95eff5640 1 -- 172.21.15.103:0/2854354484 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff96c0318f0 con 0x5652aa913060 2024-09-19T07:04:50.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.572+0000 7ff95eff5640 1 -- 172.21.15.103:0/2854354484 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff96c0035f0 con 0x5652aa913060 2024-09-19T07:04:50.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.573+0000 7ff983a27740 10 client.20862 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:50.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.573+0000 7ff983a27740 10 client.20862 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:04:50.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.573+0000 7ff983a27740 20 client.20862 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:50.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.573+0000 7ff983a27740 1 --2- 172.21.15.103:0/2854354484 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5652aa92dc70 0x5652aa930060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.573+0000 7ff983a27740 4 client.20862 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:50.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.573+0000 7ff983a27740 1 -- 172.21.15.103:0/2854354484 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652aa885b90 con 0x5652aa92dc70 2024-09-19T07:04:50.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.573+0000 7ff97a7fc640 1 --2- 172.21.15.103:0/2854354484 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5652aa92dc70 0x5652aa930060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.573+0000 7ff95eff5640 10 client.20862 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:50.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.573+0000 7ff97a7fc640 1 --2- 172.21.15.103:0/2854354484 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5652aa92dc70 0x5652aa930060 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88f3431f91acdda0 server_cookie=f9696c660af55974 in_seq=0 out_seq=0 2024-09-19T07:04:50.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.574+0000 7ff95eff5640 1 -- 172.21.15.103:0/2854354484 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652aa885b90 con 0x5652aa92dc70 2024-09-19T07:04:50.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.574+0000 7ff95eff5640 10 client.20862 handle_command_reply: tid=0 2024-09-19T07:04:50.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.574+0000 7ff983a27740 1 client.20862 shutdown 2024-09-19T07:04:50.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.574+0000 7ff983a27740 1 -- 172.21.15.103:0/2854354484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa913060 msgr2=0x5652aa91b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.574+0000 7ff983a27740 1 --2- 172.21.15.103:0/2854354484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa913060 0x5652aa91b160 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0042f0 tx=0x7ff96c005250 comp rx=0 tx=0).stop 2024-09-19T07:04:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.574+0000 7ff983a27740 1 -- 172.21.15.103:0/2854354484 shutdown_connections 2024-09-19T07:04:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.574+0000 7ff983a27740 1 --2- 172.21.15.103:0/2854354484 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5652aa92dc70 0x5652aa930060 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.574+0000 7ff95eff5640 0 client.20862 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.574+0000 7ff983a27740 1 --2- 172.21.15.103:0/2854354484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa91e240 0x5652aa91bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.574+0000 7ff983a27740 1 --2- 172.21.15.103:0/2854354484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa913980 0x5652aa91b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.574+0000 7ff983a27740 1 --2- 172.21.15.103:0/2854354484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa913060 0x5652aa91b160 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.574+0000 7ff983a27740 1 -- 172.21.15.103:0/2854354484 >> 172.21.15.103:0/2854354484 conn(0x5652aa90f5f0 msgr2=0x5652aa9152d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.574+0000 7ff983a27740 1 -- 172.21.15.103:0/2854354484 shutdown_connections 2024-09-19T07:04:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.574+0000 7ff983a27740 1 -- 172.21.15.103:0/2854354484 wait complete. 2024-09-19T07:04:50.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.575+0000 7ff983a27740 20 client.20862 trim_cache size 0 max 16384 2024-09-19T07:04:50.587 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:50.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.592+0000 7ff983a27740 1 Processor -- start 2024-09-19T07:04:50.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.592+0000 7ff983a27740 1 -- start start 2024-09-19T07:04:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.592+0000 7ff983a27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa913060 0x5652aa9151c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.592+0000 7ff983a27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913980 0x5652aa9356c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.592+0000 7ff983a27740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa91e240 0x5652aa935c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.592+0000 7ff983a27740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c1ad880 con 0x5652aa91e240 2024-09-19T07:04:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.592+0000 7ff983a27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c1af3c0 con 0x5652aa913060 2024-09-19T07:04:50.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.592+0000 7ff983a27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c1adf50 con 0x5652aa913980 2024-09-19T07:04:50.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.592+0000 7ff97a7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913980 0x5652aa9356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.592+0000 7ff97b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa91e240 0x5652aa935c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.592+0000 7ff97a7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913980 0x5652aa9356c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:34682/0 (socket says 172.21.15.103:34682) 2024-09-19T07:04:50.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.592+0000 7ff97a7fc640 1 -- 172.21.15.103:0/4230936425 learned_addr learned my addr 172.21.15.103:0/4230936425 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:50.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.592+0000 7ff97affd640 1 --2- 172.21.15.103:0/4230936425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa913060 0x5652aa9151c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.592+0000 7ff97b7fe640 1 -- 172.21.15.103:0/4230936425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913980 msgr2=0x5652aa9356c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:50.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.593+0000 7ff97b7fe640 1 --2- 172.21.15.103:0/4230936425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913980 0x5652aa9356c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.593+0000 7ff97b7fe640 1 -- 172.21.15.103:0/4230936425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa913060 msgr2=0x5652aa9151c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:50.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.593+0000 7ff97b7fe640 1 --2- 172.21.15.103:0/4230936425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa913060 0x5652aa9151c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.593+0000 7ff97b7fe640 1 -- 172.21.15.103:0/4230936425 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652aa912e90 con 0x5652aa91e240 2024-09-19T07:04:50.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.593+0000 7ff97a7fc640 1 --2- 172.21.15.103:0/4230936425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913980 0x5652aa9356c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:50.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.593+0000 7ff97b7fe640 1 --2- 172.21.15.103:0/4230936425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa91e240 0x5652aa935c00 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7ff9740cd880 tx=0x7ff9740980a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6367a28bc4240d08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.593+0000 7ff960ff9640 1 -- 172.21.15.103:0/4230936425 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97401f040 con 0x5652aa91e240 2024-09-19T07:04:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.593+0000 7ff960ff9640 1 -- 172.21.15.103:0/4230936425 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9740a07c0 con 0x5652aa91e240 2024-09-19T07:04:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.593+0000 7ff960ff9640 1 -- 172.21.15.103:0/4230936425 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff974098b30 con 0x5652aa91e240 2024-09-19T07:04:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.594+0000 7ff983a27740 1 -- 172.21.15.103:0/4230936425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa91e240 msgr2=0x5652aa935c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.594+0000 7ff983a27740 1 --2- 172.21.15.103:0/4230936425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa91e240 0x5652aa935c00 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7ff9740cd880 tx=0x7ff9740980a0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.594+0000 7ff983a27740 1 -- 172.21.15.103:0/4230936425 shutdown_connections 2024-09-19T07:04:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.594+0000 7ff983a27740 1 --2- 172.21.15.103:0/4230936425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa91e240 0x5652aa935c00 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.594+0000 7ff983a27740 1 --2- 172.21.15.103:0/4230936425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913980 0x5652aa9356c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.594+0000 7ff983a27740 1 --2- 172.21.15.103:0/4230936425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa913060 0x5652aa9151c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.594+0000 7ff983a27740 1 -- 172.21.15.103:0/4230936425 >> 172.21.15.103:0/4230936425 conn(0x5652aa90f5f0 msgr2=0x5652aa914ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.594+0000 7ff983a27740 1 -- 172.21.15.103:0/4230936425 shutdown_connections 2024-09-19T07:04:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.594+0000 7ff983a27740 1 -- 172.21.15.103:0/4230936425 wait complete. 2024-09-19T07:04:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.595+0000 7ff983a27740 1 -- start start 2024-09-19T07:04:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.595+0000 7ff983a27740 1 Processor -- start 2024-09-19T07:04:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.595+0000 7ff983a27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913060 0x5652aa9253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.595+0000 7ff983a27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa913980 0x5652aa925900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.595+0000 7ff983a27740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa91e240 0x5652aa925f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.595+0000 7ff97b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa91e240 0x5652aa925f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.595+0000 7ff97b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa91e240 0x5652aa925f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53162/0 (socket says 172.21.15.103:53162) 2024-09-19T07:04:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.595+0000 7ff97affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913060 0x5652aa9253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.595+0000 7ff97a7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa913980 0x5652aa925900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.595+0000 7ff97affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913060 0x5652aa9253c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:34688/0 (socket says 172.21.15.103:34688) 2024-09-19T07:04:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.595+0000 7ff97b7fe640 1 -- 172.21.15.103:0/3913275557 learned_addr learned my addr 172.21.15.103:0/3913275557 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.595+0000 7ff983a27740 1 -- 172.21.15.103:0/3913275557 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c1aecf0 con 0x5652aa91e240 2024-09-19T07:04:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.595+0000 7ff983a27740 1 -- 172.21.15.103:0/3913275557 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c1ae620 con 0x5652aa913980 2024-09-19T07:04:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.595+0000 7ff983a27740 1 -- 172.21.15.103:0/3913275557 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c1ab880 con 0x5652aa913060 2024-09-19T07:04:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.596+0000 7ff983a27740 10 client.0 _ll_register_callbacks cb 0x7ff96c005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.596+0000 7ff97b7fe640 1 -- 172.21.15.103:0/3913275557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913060 msgr2=0x5652aa9253c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.596+0000 7ff97b7fe640 1 --2- 172.21.15.103:0/3913275557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913060 0x5652aa9253c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.596+0000 7ff97b7fe640 1 -- 172.21.15.103:0/3913275557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa913980 msgr2=0x5652aa925900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.596+0000 7ff97b7fe640 1 --2- 172.21.15.103:0/3913275557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa913980 0x5652aa925900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.596+0000 7ff97b7fe640 1 -- 172.21.15.103:0/3913275557 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652aa926750 con 0x5652aa91e240 2024-09-19T07:04:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.596+0000 7ff97affd640 1 --2- 172.21.15.103:0/3913275557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913060 0x5652aa9253c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.596+0000 7ff97a7fc640 1 --2- 172.21.15.103:0/3913275557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa913980 0x5652aa925900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.596+0000 7ff97b7fe640 1 --2- 172.21.15.103:0/3913275557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa91e240 0x5652aa925f80 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7ff9740cde80 tx=0x7ff97409c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967be93de82e8bd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.596+0000 7ff95eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.596+0000 7ff95eff5640 1 -- 172.21.15.103:0/3913275557 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97401f030 con 0x5652aa91e240 2024-09-19T07:04:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.596+0000 7ff983a27740 1 -- 172.21.15.103:0/3913275557 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9740a07c0 con 0x5652aa91e240 2024-09-19T07:04:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.596+0000 7ff95eff5640 1 -- 172.21.15.103:0/3913275557 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7ff9740cdb00 con 0x5652aa91e240 2024-09-19T07:04:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.596+0000 7ff95eff5640 1 -- 172.21.15.103:0/3913275557 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff974098b30 con 0x5652aa91e240 2024-09-19T07:04:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.597+0000 7ff983a27740 10 client.20868 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.597+0000 7ff983a27740 1 -- 172.21.15.103:0/3913275557 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5652aa9950c0 con 0x5652aa91e240 2024-09-19T07:04:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.597+0000 7ff95eff5640 1 -- 172.21.15.103:0/3913275557 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9740a07c0 con 0x5652aa91e240 2024-09-19T07:04:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.597+0000 7ff95eff5640 1 -- 172.21.15.103:0/3913275557 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff97400f040 con 0x5652aa91e240 2024-09-19T07:04:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.597+0000 7ff983a27740 10 client.20868 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.597+0000 7ff983a27740 10 client.20868 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:04:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.597+0000 7ff983a27740 20 client.20868 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.597+0000 7ff983a27740 1 --2- 172.21.15.103:0/3913275557 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5652aa92dc70 0x5652aa9977b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.597+0000 7ff983a27740 4 client.20868 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a3c7bd9c", "release": true, "await": true}] 2024-09-19T07:04:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.597+0000 7ff983a27740 1 -- 172.21.15.103:0/3913275557 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a3c7bd9c", "release": true, "await": true}) -- 0x5652aa8800a0 con 0x5652aa92dc70 2024-09-19T07:04:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.597+0000 7ff97affd640 1 --2- 172.21.15.103:0/3913275557 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5652aa92dc70 0x5652aa9977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.598+0000 7ff97affd640 1 --2- 172.21.15.103:0/3913275557 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5652aa92dc70 0x5652aa9977b0 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9730ad2a1cb0f2c4 server_cookie=cc8297f071835f49 in_seq=0 out_seq=0 2024-09-19T07:04:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.598+0000 7ff95eff5640 10 client.20868 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.601+0000 7ff95eff5640 1 -- 172.21.15.103:0/3913275557 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5652aa8800a0 con 0x5652aa92dc70 2024-09-19T07:04:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.601+0000 7ff95eff5640 10 client.20868 handle_command_reply: tid=0 2024-09-19T07:04:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.601+0000 7ff983a27740 1 client.20868 shutdown 2024-09-19T07:04:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.601+0000 7ff983a27740 1 -- 172.21.15.103:0/3913275557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa91e240 msgr2=0x5652aa925f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.601+0000 7ff983a27740 1 --2- 172.21.15.103:0/3913275557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa91e240 0x5652aa925f80 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7ff9740cde80 tx=0x7ff97409c3b0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff983a27740 1 -- 172.21.15.103:0/3913275557 shutdown_connections 2024-09-19T07:04:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff983a27740 1 --2- 172.21.15.103:0/3913275557 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5652aa92dc70 0x5652aa9977b0 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff983a27740 1 --2- 172.21.15.103:0/3913275557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652aa91e240 0x5652aa925f80 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff983a27740 1 --2- 172.21.15.103:0/3913275557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652aa913980 0x5652aa925900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff983a27740 1 --2- 172.21.15.103:0/3913275557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652aa913060 0x5652aa9253c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff95eff5640 0 client.20868 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff983a27740 1 -- 172.21.15.103:0/3913275557 >> 172.21.15.103:0/3913275557 conn(0x5652aa90f5f0 msgr2=0x5652aa914ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:50.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff983a27740 1 -- 172.21.15.103:0/3913275557 shutdown_connections 2024-09-19T07:04:50.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff983a27740 1 -- 172.21.15.103:0/3913275557 wait complete. 2024-09-19T07:04:50.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff983a27740 20 client.20868 trim_cache size 0 max 16384 2024-09-19T07:04:50.610 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:04:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T07:04:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T07:04:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 52, 2024-09-19T07:04:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:04:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "a3c7bd9c": { 2024-09-19T07:04:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 52, 2024-09-19T07:04:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:04:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:04:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:04:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:04:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:04:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:04:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 149.4, 2024-09-19T07:04:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:04:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:04:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:04:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:04:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:04:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:04:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:04:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:04:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:04:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:04:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:04:50.612 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:04:50.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff98164e640 1 -- 172.21.15.103:0/2633405040 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9640a3430 msgr2=0x7ff9640a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:50.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff98164e640 1 --2- 172.21.15.103:0/2633405040 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9640a3430 0x7ff9640a58f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x5652aa910450 tx=0x7ff970009920 comp rx=0 tx=0).stop 2024-09-19T07:04:50.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff98164e640 1 -- 172.21.15.103:0/2633405040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106b80 msgr2=0x7ff97c19cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff98164e640 1 --2- 172.21.15.103:0/2633405040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106b80 0x7ff97c19cdc0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7ff97400e9d0 tx=0x7ff97400ee90 comp rx=0 tx=0).stop 2024-09-19T07:04:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff98164e640 1 -- 172.21.15.103:0/2633405040 shutdown_connections 2024-09-19T07:04:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff98164e640 1 --2- 172.21.15.103:0/2633405040 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff9640a3430 0x7ff9640a58f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff98164e640 1 --2- 172.21.15.103:0/2633405040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff97c106b80 0x7ff97c19cdc0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff98164e640 1 --2- 172.21.15.103:0/2633405040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff97c100aa0 0x7ff97c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.602+0000 7ff98164e640 1 --2- 172.21.15.103:0/2633405040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff97c100100 0x7ff97c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.603+0000 7ff98164e640 1 -- 172.21.15.103:0/2633405040 >> 172.21.15.103:0/2633405040 conn(0x7ff97c077230 msgr2=0x7ff97c104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.603+0000 7ff98164e640 1 -- 172.21.15.103:0/2633405040 shutdown_connections 2024-09-19T07:04:50.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:50.603+0000 7ff98164e640 1 -- 172.21.15.103:0/2633405040 wait complete. 2024-09-19T07:04:51.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.222+0000 7ff7c8b52640 1 -- 172.21.15.103:0/2588013860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c4101420 msgr2=0x7ff7c410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:51.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.222+0000 7ff7c8b52640 1 --2- 172.21.15.103:0/2588013860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c4101420 0x7ff7c410d950 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009cd0 tx=0x7ff7bc02a8e0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.222+0000 7ff7c8b52640 1 -- 172.21.15.103:0/2588013860 shutdown_connections 2024-09-19T07:04:51.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.222+0000 7ff7c8b52640 1 --2- 172.21.15.103:0/2588013860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c4101420 0x7ff7c410d950 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.222+0000 7ff7c8b52640 1 --2- 172.21.15.103:0/2588013860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c4100a80 0x7ff7c4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.222+0000 7ff7c8b52640 1 --2- 172.21.15.103:0/2588013860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c4106a80 0x7ff7c4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.222+0000 7ff7c8b52640 1 -- 172.21.15.103:0/2588013860 >> 172.21.15.103:0/2588013860 conn(0x7ff7c40fc7a0 msgr2=0x7ff7c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:51.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.222+0000 7ff7c8b52640 1 -- 172.21.15.103:0/2588013860 shutdown_connections 2024-09-19T07:04:51.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.222+0000 7ff7c8b52640 1 -- 172.21.15.103:0/2588013860 wait complete. 2024-09-19T07:04:51.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.223+0000 7ff7c8b52640 1 Processor -- start 2024-09-19T07:04:51.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.223+0000 7ff7c8b52640 1 -- start start 2024-09-19T07:04:51.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.223+0000 7ff7c8b52640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c4100a80 0x7ff7c419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c8b52640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c4101420 0x7ff7c419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c8b52640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c4106a80 0x7ff7c4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c8b52640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c4110e80 con 0x7ff7c4100a80 2024-09-19T07:04:51.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c8b52640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c4110d00 con 0x7ff7c4106a80 2024-09-19T07:04:51.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c8b52640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c4111000 con 0x7ff7c4101420 2024-09-19T07:04:51.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c4100a80 0x7ff7c419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c4100a80 0x7ff7c419d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53176/0 (socket says 172.21.15.103:53176) 2024-09-19T07:04:51.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c259c640 1 -- 172.21.15.103:0/2617467040 learned_addr learned my addr 172.21.15.103:0/2617467040 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:51.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7b9d9b640 1 --2- 172.21.15.103:0/2617467040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c4101420 0x7ff7c419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c2d9d640 1 --2- 172.21.15.103:0/2617467040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c4106a80 0x7ff7c4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c259c640 1 -- 172.21.15.103:0/2617467040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c4101420 msgr2=0x7ff7c419d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:51.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c259c640 1 --2- 172.21.15.103:0/2617467040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c4101420 0x7ff7c419d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c259c640 1 -- 172.21.15.103:0/2617467040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c4106a80 msgr2=0x7ff7c4198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:51.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c259c640 1 --2- 172.21.15.103:0/2617467040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c4106a80 0x7ff7c4198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c259c640 1 -- 172.21.15.103:0/2617467040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c4198ca0 con 0x7ff7c4100a80 2024-09-19T07:04:51.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c2d9d640 1 --2- 172.21.15.103:0/2617467040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c4106a80 0x7ff7c4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:51.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c259c640 1 --2- 172.21.15.103:0/2617467040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c4100a80 0x7ff7c419d000 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400c890 tx=0x7ff7b400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:51.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7bbfff640 1 -- 172.21.15.103:0/2617467040 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4004220 con 0x7ff7c4100a80 2024-09-19T07:04:51.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.224+0000 7ff7c8b52640 1 -- 172.21.15.103:0/2617467040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c4198f90 con 0x7ff7c4100a80 2024-09-19T07:04:51.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.225+0000 7ff7bbfff640 1 -- 172.21.15.103:0/2617467040 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b40043c0 con 0x7ff7c4100a80 2024-09-19T07:04:51.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.225+0000 7ff7bbfff640 1 -- 172.21.15.103:0/2617467040 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4004a00 con 0x7ff7c4100a80 2024-09-19T07:04:51.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.225+0000 7ff7c8b52640 1 -- 172.21.15.103:0/2617467040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c41a4f30 con 0x7ff7c4100a80 2024-09-19T07:04:51.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.227+0000 7ff7bbfff640 1 -- 172.21.15.103:0/2617467040 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff7b40079b0 con 0x7ff7c4100a80 2024-09-19T07:04:51.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.227+0000 7ff7caf2b740 1 Processor -- start 2024-09-19T07:04:51.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.227+0000 7ff7caf2b740 1 -- start start 2024-09-19T07:04:51.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7caf2b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d17b40 0x557488d16090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7caf2b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d165d0 0x557488d16a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7caf2b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d21370 0x557488d23760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7caf2b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557488c7c2f0 con 0x557488d21370 2024-09-19T07:04:51.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7caf2b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557488c23cf0 con 0x557488d165d0 2024-09-19T07:04:51.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7caf2b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557488c782b0 con 0x557488d17b40 2024-09-19T07:04:51.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7b9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d21370 0x557488d23760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7b9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d21370 0x557488d23760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53190/0 (socket says 172.21.15.103:53190) 2024-09-19T07:04:51.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7b9d9b640 1 -- 172.21.15.103:0/397621153 learned_addr learned my addr 172.21.15.103:0/397621153 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:51.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7c259c640 1 --2- 172.21.15.103:0/397621153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d165d0 0x557488d16a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7c2d9d640 1 --2- 172.21.15.103:0/397621153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d17b40 0x557488d16090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7bbfff640 1 --2- 172.21.15.103:0/2617467040 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7b00a33b0 0x7ff7b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7c2d9d640 1 --2- 172.21.15.103:0/2617467040 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7b00a33b0 0x7ff7b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7b9d9b640 1 -- 172.21.15.103:0/397621153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d17b40 msgr2=0x557488d16090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:51.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7b9d9b640 1 --2- 172.21.15.103:0/397621153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d17b40 0x557488d16090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7b9d9b640 1 -- 172.21.15.103:0/397621153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d165d0 msgr2=0x557488d16a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:51.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7b9d9b640 1 --2- 172.21.15.103:0/397621153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d165d0 0x557488d16a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7b9d9b640 1 -- 172.21.15.103:0/397621153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557488c083a0 con 0x557488d21370 2024-09-19T07:04:51.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7c2d9d640 1 --2- 172.21.15.103:0/397621153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d17b40 0x557488d16090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:51.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7c259c640 1 --2- 172.21.15.103:0/397621153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d165d0 0x557488d16a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:51.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.228+0000 7ff7bbfff640 1 -- 172.21.15.103:0/2617467040 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7ff7b40cf9d0 con 0x7ff7c4100a80 2024-09-19T07:04:51.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.229+0000 7ff7b9d9b640 1 --2- 172.21.15.103:0/397621153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d21370 0x557488d23760 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00b6c0 tx=0x7ff7ac00bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f13a85f6b0f3f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:51.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.229+0000 7ff7c2d9d640 1 --2- 172.21.15.103:0/2617467040 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7b00a33b0 0x7ff7b00a5870 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc006010 tx=0x7ff7bc041040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:51.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.229+0000 7ff7b8d99640 1 -- 172.21.15.103:0/397621153 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac003bd0 con 0x557488d21370 2024-09-19T07:04:51.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.229+0000 7ff7b8d99640 1 -- 172.21.15.103:0/397621153 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7ac015070 con 0x557488d21370 2024-09-19T07:04:51.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.229+0000 7ff7b8d99640 1 -- 172.21.15.103:0/397621153 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac004e00 con 0x557488d21370 2024-09-19T07:04:51.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 -- 172.21.15.103:0/397621153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d21370 msgr2=0x557488d23760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:51.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/397621153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d21370 0x557488d23760 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00b6c0 tx=0x7ff7ac00bb80 comp rx=0 tx=0).stop 2024-09-19T07:04:51.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 -- 172.21.15.103:0/397621153 shutdown_connections 2024-09-19T07:04:51.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/397621153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d21370 0x557488d23760 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/397621153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d165d0 0x557488d16a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/397621153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d17b40 0x557488d16090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 -- 172.21.15.103:0/397621153 >> 172.21.15.103:0/397621153 conn(0x557488cdf310 msgr2=0x557488d12c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:51.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 -- 172.21.15.103:0/397621153 shutdown_connections 2024-09-19T07:04:51.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 -- 172.21.15.103:0/397621153 wait complete. 2024-09-19T07:04:51.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 -- start start 2024-09-19T07:04:51.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 Processor -- start 2024-09-19T07:04:51.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d165d0 0x557488d20c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d17b40 0x557488d1df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d21370 0x557488d1e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557488d24070 con 0x557488d165d0 2024-09-19T07:04:51.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557488c37060 con 0x557488d21370 2024-09-19T07:04:51.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.230+0000 7ff7caf2b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557488d241f0 con 0x557488d17b40 2024-09-19T07:04:51.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7caf2b740 10 client.0 _ll_register_callbacks cb 0x55748898a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:51.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7c259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d165d0 0x557488d20c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7c259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d165d0 0x557488d20c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53204/0 (socket says 172.21.15.103:53204) 2024-09-19T07:04:51.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7c259c640 1 -- 172.21.15.103:0/1387487021 learned_addr learned my addr 172.21.15.103:0/1387487021 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:51.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7c2d9d640 1 --2- 172.21.15.103:0/1387487021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d21370 0x557488d1e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7b9d9b640 1 --2- 172.21.15.103:0/1387487021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d17b40 0x557488d1df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7c259c640 1 -- 172.21.15.103:0/1387487021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d17b40 msgr2=0x557488d1df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:51.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7c259c640 1 --2- 172.21.15.103:0/1387487021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d17b40 0x557488d1df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7c259c640 1 -- 172.21.15.103:0/1387487021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d21370 msgr2=0x557488d1e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:51.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7c259c640 1 --2- 172.21.15.103:0/1387487021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d21370 0x557488d1e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7c259c640 1 -- 172.21.15.103:0/1387487021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574889ed3c0 con 0x557488d165d0 2024-09-19T07:04:51.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7c259c640 1 --2- 172.21.15.103:0/1387487021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d165d0 0x557488d20c40 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40cd700 tx=0x7ff7b4097ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcf01f87fda13cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:51.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:51.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7a9ffb640 1 -- 172.21.15.103:0/1387487021 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b401d040 con 0x557488d165d0 2024-09-19T07:04:51.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7caf2b740 1 -- 172.21.15.103:0/1387487021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7ac015070 con 0x557488d165d0 2024-09-19T07:04:51.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7a9ffb640 1 -- 172.21.15.103:0/1387487021 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7ff7b409b0e0 con 0x557488d165d0 2024-09-19T07:04:51.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7a9ffb640 1 -- 172.21.15.103:0/1387487021 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff7b409c390 con 0x557488d165d0 2024-09-19T07:04:51.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7caf2b740 10 client.20880 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:51.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.231+0000 7ff7caf2b740 1 -- 172.21.15.103:0/1387487021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557488d2fa60 con 0x557488d165d0 2024-09-19T07:04:51.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.232+0000 7ff7a9ffb640 1 -- 172.21.15.103:0/1387487021 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7ac015070 con 0x557488d165d0 2024-09-19T07:04:51.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.232+0000 7ff7a9ffb640 1 -- 172.21.15.103:0/1387487021 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff7b40860c0 con 0x557488d165d0 2024-09-19T07:04:51.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.232+0000 7ff7caf2b740 10 client.20880 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:51.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.232+0000 7ff7caf2b740 10 client.20880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:51.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.232+0000 7ff7caf2b740 20 client.20880 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:51.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.232+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/1387487021 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557488d30f20 0x557488d33330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.232+0000 7ff7caf2b740 4 client.20880 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:51.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.232+0000 7ff7caf2b740 1 -- 172.21.15.103:0/1387487021 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557488c89860 con 0x557488d30f20 2024-09-19T07:04:51.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.232+0000 7ff7b9d9b640 1 --2- 172.21.15.103:0/1387487021 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557488d30f20 0x557488d33330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.233+0000 7ff7b9d9b640 1 --2- 172.21.15.103:0/1387487021 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557488d30f20 0x557488d33330 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cd2eb2e63517838 server_cookie=63d1a9ef24921ab8 in_seq=0 out_seq=0 2024-09-19T07:04:51.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.233+0000 7ff7a9ffb640 10 client.20880 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:51.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.233+0000 7ff7a9ffb640 1 -- 172.21.15.103:0/1387487021 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557488c89860 con 0x557488d30f20 2024-09-19T07:04:51.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.233+0000 7ff7a9ffb640 10 client.20880 handle_command_reply: tid=0 2024-09-19T07:04:51.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.233+0000 7ff7caf2b740 1 client.20880 shutdown 2024-09-19T07:04:51.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.233+0000 7ff7caf2b740 1 -- 172.21.15.103:0/1387487021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d165d0 msgr2=0x557488d20c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:51.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.233+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/1387487021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d165d0 0x557488d20c40 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40cd700 tx=0x7ff7b4097ff0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.233+0000 7ff7caf2b740 1 -- 172.21.15.103:0/1387487021 shutdown_connections 2024-09-19T07:04:51.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.233+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/1387487021 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557488d30f20 0x557488d33330 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.233+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/1387487021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d21370 0x557488d1e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.233+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/1387487021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d17b40 0x557488d1df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.233+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/1387487021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d165d0 0x557488d20c40 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.233+0000 7ff7a9ffb640 0 client.20880 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:51.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.233+0000 7ff7caf2b740 1 -- 172.21.15.103:0/1387487021 >> 172.21.15.103:0/1387487021 conn(0x557488cdf310 msgr2=0x557488d18960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:51.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.234+0000 7ff7caf2b740 1 -- 172.21.15.103:0/1387487021 shutdown_connections 2024-09-19T07:04:51.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.234+0000 7ff7caf2b740 1 -- 172.21.15.103:0/1387487021 wait complete. 2024-09-19T07:04:51.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.234+0000 7ff7caf2b740 20 client.20880 trim_cache size 0 max 16384 2024-09-19T07:04:51.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.246+0000 7ff7caf2b740 1 Processor -- start 2024-09-19T07:04:51.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.246+0000 7ff7caf2b740 1 -- start start 2024-09-19T07:04:51.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7caf2b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d165d0 0x557488d2d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7caf2b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d17b40 0x557488d18820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7caf2b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d21370 0x557488d18d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7caf2b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c41afb10 con 0x557488d165d0 2024-09-19T07:04:51.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7caf2b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c41b1650 con 0x557488d21370 2024-09-19T07:04:51.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7caf2b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c41b01e0 con 0x557488d17b40 2024-09-19T07:04:51.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7c259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d165d0 0x557488d2d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7c259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d165d0 0x557488d2d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53218/0 (socket says 172.21.15.103:53218) 2024-09-19T07:04:51.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7c259c640 1 -- 172.21.15.103:0/70697717 learned_addr learned my addr 172.21.15.103:0/70697717 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:51.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7b9d9b640 1 --2- 172.21.15.103:0/70697717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d17b40 0x557488d18820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7c2d9d640 1 --2- 172.21.15.103:0/70697717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d21370 0x557488d18d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7c259c640 1 -- 172.21.15.103:0/70697717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d17b40 msgr2=0x557488d18820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:51.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7c259c640 1 --2- 172.21.15.103:0/70697717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d17b40 0x557488d18820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7c259c640 1 -- 172.21.15.103:0/70697717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d21370 msgr2=0x557488d18d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:51.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7c259c640 1 --2- 172.21.15.103:0/70697717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d21370 0x557488d18d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7c259c640 1 -- 172.21.15.103:0/70697717 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557488d13960 con 0x557488d165d0 2024-09-19T07:04:51.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7c2d9d640 1 --2- 172.21.15.103:0/70697717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d21370 0x557488d18d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:51.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7c259c640 1 --2- 172.21.15.103:0/70697717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d165d0 0x557488d2d340 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7ff7b409b990 tx=0x7ff7b4005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c16b3d19cae63a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:51.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7abfff640 1 -- 172.21.15.103:0/70697717 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b401d040 con 0x557488d165d0 2024-09-19T07:04:51.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7abfff640 1 -- 172.21.15.103:0/70697717 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b409bc90 con 0x557488d165d0 2024-09-19T07:04:51.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.247+0000 7ff7abfff640 1 -- 172.21.15.103:0/70697717 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b409ab50 con 0x557488d165d0 2024-09-19T07:04:51.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.248+0000 7ff7caf2b740 1 -- 172.21.15.103:0/70697717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d165d0 msgr2=0x557488d2d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:51.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.248+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/70697717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d165d0 0x557488d2d340 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7ff7b409b990 tx=0x7ff7b4005300 comp rx=0 tx=0).stop 2024-09-19T07:04:51.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.248+0000 7ff7caf2b740 1 -- 172.21.15.103:0/70697717 shutdown_connections 2024-09-19T07:04:51.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.248+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/70697717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d21370 0x557488d18d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.248+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/70697717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d17b40 0x557488d18820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.248+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/70697717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d165d0 0x557488d2d340 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.248+0000 7ff7caf2b740 1 -- 172.21.15.103:0/70697717 >> 172.21.15.103:0/70697717 conn(0x557488cdf310 msgr2=0x557488d18110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:51.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.248+0000 7ff7caf2b740 1 -- 172.21.15.103:0/70697717 shutdown_connections 2024-09-19T07:04:51.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.248+0000 7ff7caf2b740 1 -- 172.21.15.103:0/70697717 wait complete. 2024-09-19T07:04:51.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.248+0000 7ff7caf2b740 1 -- start start 2024-09-19T07:04:51.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.248+0000 7ff7caf2b740 1 Processor -- start 2024-09-19T07:04:51.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7caf2b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d165d0 0x557488d25130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7caf2b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d17b40 0x557488d25670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7caf2b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d21370 0x557488d25bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7caf2b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c41b0f80 con 0x557488d17b40 2024-09-19T07:04:51.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7b9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d17b40 0x557488d25670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7caf2b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c41b08b0 con 0x557488d165d0 2024-09-19T07:04:51.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7caf2b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c41adb30 con 0x557488d21370 2024-09-19T07:04:51.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7b9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d17b40 0x557488d25670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53230/0 (socket says 172.21.15.103:53230) 2024-09-19T07:04:51.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7b9d9b640 1 -- 172.21.15.103:0/3695878701 learned_addr learned my addr 172.21.15.103:0/3695878701 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:51.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7caf2b740 10 client.0 _ll_register_callbacks cb 0x7ff7b40058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:51.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7c2d9d640 1 --2- 172.21.15.103:0/3695878701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d21370 0x557488d25bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7c259c640 1 --2- 172.21.15.103:0/3695878701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d165d0 0x557488d25130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7b9d9b640 1 -- 172.21.15.103:0/3695878701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d21370 msgr2=0x557488d25bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:51.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7b9d9b640 1 --2- 172.21.15.103:0/3695878701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d21370 0x557488d25bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7b9d9b640 1 -- 172.21.15.103:0/3695878701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d165d0 msgr2=0x557488d25130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:51.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7b9d9b640 1 --2- 172.21.15.103:0/3695878701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d165d0 0x557488d25130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7b9d9b640 1 -- 172.21.15.103:0/3695878701 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557488d9aba0 con 0x557488d17b40 2024-09-19T07:04:51.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7c259c640 1 --2- 172.21.15.103:0/3695878701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d165d0 0x557488d25130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:51.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7b9d9b640 1 --2- 172.21.15.103:0/3695878701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d17b40 0x557488d25670 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x557488d17070 tx=0x7ff7ac004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45eaa4e7c8376fbb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:51.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:51.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7a9ffb640 1 -- 172.21.15.103:0/3695878701 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac003dd0 con 0x557488d17b40 2024-09-19T07:04:51.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7a9ffb640 1 -- 172.21.15.103:0/3695878701 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7ff7ac011440 con 0x557488d17b40 2024-09-19T07:04:51.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.249+0000 7ff7caf2b740 1 -- 172.21.15.103:0/3695878701 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7b409bc90 con 0x557488d17b40 2024-09-19T07:04:51.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.250+0000 7ff7a9ffb640 1 -- 172.21.15.103:0/3695878701 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff7ac011640 con 0x557488d17b40 2024-09-19T07:04:51.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.250+0000 7ff7caf2b740 10 client.20886 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:51.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.250+0000 7ff7caf2b740 1 -- 172.21.15.103:0/3695878701 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557488d9bdb0 con 0x557488d17b40 2024-09-19T07:04:51.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.250+0000 7ff7a9ffb640 1 -- 172.21.15.103:0/3695878701 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b409bc90 con 0x557488d17b40 2024-09-19T07:04:51.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.250+0000 7ff7a9ffb640 1 -- 172.21.15.103:0/3695878701 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff7ac02d070 con 0x557488d17b40 2024-09-19T07:04:51.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.250+0000 7ff7caf2b740 10 client.20886 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:51.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.250+0000 7ff7caf2b740 10 client.20886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:51.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.250+0000 7ff7caf2b740 20 client.20886 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:51.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.250+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/3695878701 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557488d9c3d0 0x557488d9e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:51.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.250+0000 7ff7caf2b740 4 client.20886 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:04:51.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.250+0000 7ff7caf2b740 1 -- 172.21.15.103:0/3695878701 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557488c83d70 con 0x557488d9c3d0 2024-09-19T07:04:51.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.250+0000 7ff7c259c640 1 --2- 172.21.15.103:0/3695878701 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557488d9c3d0 0x557488d9e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:51.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.251+0000 7ff7a9ffb640 10 client.20886 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:51.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.251+0000 7ff7c259c640 1 --2- 172.21.15.103:0/3695878701 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557488d9c3d0 0x557488d9e7f0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6288b7d6f19994d1 server_cookie=d1f64bb039f09a61 in_seq=0 out_seq=0 2024-09-19T07:04:51.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.251+0000 7ff7a9ffb640 1 -- 172.21.15.103:0/3695878701 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557488c83d70 con 0x557488d9c3d0 2024-09-19T07:04:51.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.251+0000 7ff7a9ffb640 10 client.20886 handle_command_reply: tid=0 2024-09-19T07:04:51.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.251+0000 7ff7caf2b740 1 client.20886 shutdown 2024-09-19T07:04:51.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.251+0000 7ff7caf2b740 1 -- 172.21.15.103:0/3695878701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d17b40 msgr2=0x557488d25670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:51.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.251+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/3695878701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d17b40 0x557488d25670 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x557488d17070 tx=0x7ff7ac004870 comp rx=0 tx=0).stop 2024-09-19T07:04:51.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.251+0000 7ff7caf2b740 1 -- 172.21.15.103:0/3695878701 shutdown_connections 2024-09-19T07:04:51.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.251+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/3695878701 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557488d9c3d0 0x557488d9e7f0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.251+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/3695878701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557488d21370 0x557488d25bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.251+0000 7ff7a9ffb640 0 client.20886 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:51.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.251+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/3695878701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557488d17b40 0x557488d25670 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.251+0000 7ff7caf2b740 1 --2- 172.21.15.103:0/3695878701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557488d165d0 0x557488d25130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.251+0000 7ff7caf2b740 1 -- 172.21.15.103:0/3695878701 >> 172.21.15.103:0/3695878701 conn(0x557488cdf310 msgr2=0x557488d11d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:51.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.251+0000 7ff7caf2b740 1 -- 172.21.15.103:0/3695878701 shutdown_connections 2024-09-19T07:04:51.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.251+0000 7ff7caf2b740 1 -- 172.21.15.103:0/3695878701 wait complete. 2024-09-19T07:04:51.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.252+0000 7ff7caf2b740 20 client.20886 trim_cache size 0 max 16384 2024-09-19T07:04:51.261 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:04:51.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.252+0000 7ff7c8b52640 1 -- 172.21.15.103:0/2617467040 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7b00a33b0 msgr2=0x7ff7b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:51.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.252+0000 7ff7c8b52640 1 --2- 172.21.15.103:0/2617467040 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7b00a33b0 0x7ff7b00a5870 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc006010 tx=0x7ff7bc041040 comp rx=0 tx=0).stop 2024-09-19T07:04:51.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.252+0000 7ff7c8b52640 1 -- 172.21.15.103:0/2617467040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c4100a80 msgr2=0x7ff7c419d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:51.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.252+0000 7ff7c8b52640 1 --2- 172.21.15.103:0/2617467040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c4100a80 0x7ff7c419d000 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400c890 tx=0x7ff7b400cd50 comp rx=0 tx=0).stop 2024-09-19T07:04:51.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.252+0000 7ff7c8b52640 1 -- 172.21.15.103:0/2617467040 shutdown_connections 2024-09-19T07:04:51.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.252+0000 7ff7c8b52640 1 --2- 172.21.15.103:0/2617467040 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7b00a33b0 0x7ff7b00a5870 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.252+0000 7ff7c8b52640 1 --2- 172.21.15.103:0/2617467040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7c4106a80 0x7ff7c4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.252+0000 7ff7c8b52640 1 --2- 172.21.15.103:0/2617467040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c4101420 0x7ff7c419d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.252+0000 7ff7c8b52640 1 --2- 172.21.15.103:0/2617467040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7c4100a80 0x7ff7c419d000 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:51.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.252+0000 7ff7c8b52640 1 -- 172.21.15.103:0/2617467040 >> 172.21.15.103:0/2617467040 conn(0x7ff7c40fc7a0 msgr2=0x7ff7c4104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:51.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.252+0000 7ff7c8b52640 1 -- 172.21.15.103:0/2617467040 shutdown_connections 2024-09-19T07:04:51.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:51.252+0000 7ff7c8b52640 1 -- 172.21.15.103:0/2617467040 wait complete. 2024-09-19T07:04:51.571 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:51 smithi103 ceph-mon[31158]: pgmap v489: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 KiB/s wr, 15 op/s 2024-09-19T07:04:51.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:51 smithi174 ceph-mon[35637]: pgmap v489: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 KiB/s wr, 15 op/s 2024-09-19T07:04:51.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:51 smithi134 ceph-mon[35356]: pgmap v489: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 KiB/s wr, 15 op/s 2024-09-19T07:04:52.083 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a3c7bd9c', total seconds elapsed: 34.1 2024-09-19T07:04:52.083 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.45 2024-09-19T07:04:52.084 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:04:52.769 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.185+0000 7ff774b44640 1 -- 172.21.15.103:0/2989645859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff770100a80 msgr2=0x7ff770100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.185+0000 7ff774b44640 1 --2- 172.21.15.103:0/2989645859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff770100a80 0x7ff770100ee0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7ff764009930 tx=0x7ff76402a420 comp rx=0 tx=0).stop 2024-09-19T07:04:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.185+0000 7ff774b44640 1 -- 172.21.15.103:0/2989645859 shutdown_connections 2024-09-19T07:04:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.185+0000 7ff774b44640 1 --2- 172.21.15.103:0/2989645859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff770101420 0x7ff77010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.185+0000 7ff774b44640 1 --2- 172.21.15.103:0/2989645859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff770100a80 0x7ff770100ee0 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.185+0000 7ff774b44640 1 --2- 172.21.15.103:0/2989645859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff770106a80 0x7ff770106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.185+0000 7ff774b44640 1 -- 172.21.15.103:0/2989645859 >> 172.21.15.103:0/2989645859 conn(0x7ff7700fc7a0 msgr2=0x7ff7700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:53.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.185+0000 7ff774b44640 1 -- 172.21.15.103:0/2989645859 shutdown_connections 2024-09-19T07:04:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.185+0000 7ff774b44640 1 -- 172.21.15.103:0/2989645859 wait complete. 2024-09-19T07:04:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.186+0000 7ff774b44640 1 Processor -- start 2024-09-19T07:04:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.186+0000 7ff774b44640 1 -- start start 2024-09-19T07:04:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.186+0000 7ff774b44640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff770100a80 0x7ff770198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.186+0000 7ff774b44640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff770101420 0x7ff770198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff774b44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff770106a80 0x7ff77019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff774b44640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff770111f60 con 0x7ff770106a80 2024-09-19T07:04:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff774b44640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff770111de0 con 0x7ff770101420 2024-09-19T07:04:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff774b44640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7701120e0 con 0x7ff770100a80 2024-09-19T07:04:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff76ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff770106a80 0x7ff77019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff76ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff770106a80 0x7ff77019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53242/0 (socket says 172.21.15.103:53242) 2024-09-19T07:04:53.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff76ed9d640 1 -- 172.21.15.103:0/439508405 learned_addr learned my addr 172.21.15.103:0/439508405 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff76e59c640 1 --2- 172.21.15.103:0/439508405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff770100a80 0x7ff770198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff76dd9b640 1 --2- 172.21.15.103:0/439508405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff770101420 0x7ff770198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff76ed9d640 1 -- 172.21.15.103:0/439508405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff770100a80 msgr2=0x7ff770198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff76ed9d640 1 --2- 172.21.15.103:0/439508405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff770100a80 0x7ff770198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff76ed9d640 1 -- 172.21.15.103:0/439508405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff770101420 msgr2=0x7ff770198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff76ed9d640 1 --2- 172.21.15.103:0/439508405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff770101420 0x7ff770198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff76ed9d640 1 -- 172.21.15.103:0/439508405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77019d3b0 con 0x7ff770106a80 2024-09-19T07:04:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff76dd9b640 1 --2- 172.21.15.103:0/439508405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff770101420 0x7ff770198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff76ed9d640 1 --2- 172.21.15.103:0/439508405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff770106a80 0x7ff77019ccd0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7ff76800cad0 tx=0x7ff76800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff75f7fe640 1 -- 172.21.15.103:0/439508405 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76800ad60 con 0x7ff770106a80 2024-09-19T07:04:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff75f7fe640 1 -- 172.21.15.103:0/439508405 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff768005020 con 0x7ff770106a80 2024-09-19T07:04:53.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff774b44640 1 -- 172.21.15.103:0/439508405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77019d6a0 con 0x7ff770106a80 2024-09-19T07:04:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff75f7fe640 1 -- 172.21.15.103:0/439508405 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76800f6b0 con 0x7ff770106a80 2024-09-19T07:04:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.187+0000 7ff774b44640 1 -- 172.21.15.103:0/439508405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7701a4f80 con 0x7ff770106a80 2024-09-19T07:04:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.188+0000 7ff774b44640 1 -- 172.21.15.103:0/439508405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff770100ee0 con 0x7ff770106a80 2024-09-19T07:04:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.190+0000 7ff75f7fe640 1 -- 172.21.15.103:0/439508405 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff768005560 con 0x7ff770106a80 2024-09-19T07:04:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.190+0000 7ff75f7fe640 1 --2- 172.21.15.103:0/439508405 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7580a33b0 0x7ff7580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.191+0000 7ff76e59c640 1 --2- 172.21.15.103:0/439508405 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7580a33b0 0x7ff7580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.191+0000 7ff75f7fe640 1 -- 172.21.15.103:0/439508405 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7ff7680cfde0 con 0x7ff770106a80 2024-09-19T07:04:53.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.191+0000 7ff76e59c640 1 --2- 172.21.15.103:0/439508405 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7580a33b0 0x7ff7580a5870 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff76000ab60 tx=0x7ff76000a2d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:53.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.195+0000 7ff75f7fe640 1 -- 172.21.15.103:0/439508405 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff768014030 con 0x7ff770106a80 2024-09-19T07:04:53.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.399+0000 7ff774b44640 1 -- 172.21.15.103:0/439508405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff77002d7c0 con 0x7ff770106a80 2024-09-19T07:04:53.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.400+0000 7ff75f7fe640 1 -- 172.21.15.103:0/439508405 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7ff76809c7a0 con 0x7ff770106a80 2024-09-19T07:04:53.401 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:04:53.401 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:04:53.401 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:04:53.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.402+0000 7ff774b44640 1 -- 172.21.15.103:0/439508405 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7580a33b0 msgr2=0x7ff7580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:53.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.402+0000 7ff774b44640 1 --2- 172.21.15.103:0/439508405 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7580a33b0 0x7ff7580a5870 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff76000ab60 tx=0x7ff76000a2d0 comp rx=0 tx=0).stop 2024-09-19T07:04:53.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.402+0000 7ff774b44640 1 -- 172.21.15.103:0/439508405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff770106a80 msgr2=0x7ff77019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:53.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.402+0000 7ff774b44640 1 --2- 172.21.15.103:0/439508405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff770106a80 0x7ff77019ccd0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7ff76800cad0 tx=0x7ff76800ced0 comp rx=0 tx=0).stop 2024-09-19T07:04:53.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.402+0000 7ff774b44640 1 -- 172.21.15.103:0/439508405 shutdown_connections 2024-09-19T07:04:53.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.402+0000 7ff774b44640 1 --2- 172.21.15.103:0/439508405 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff7580a33b0 0x7ff7580a5870 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:53.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.402+0000 7ff774b44640 1 --2- 172.21.15.103:0/439508405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff770106a80 0x7ff77019ccd0 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:53.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.402+0000 7ff774b44640 1 --2- 172.21.15.103:0/439508405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff770101420 0x7ff770198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:53.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.402+0000 7ff774b44640 1 --2- 172.21.15.103:0/439508405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff770100a80 0x7ff770198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:53.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.402+0000 7ff774b44640 1 -- 172.21.15.103:0/439508405 >> 172.21.15.103:0/439508405 conn(0x7ff7700fc7a0 msgr2=0x7ff7700fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:53.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.402+0000 7ff774b44640 1 -- 172.21.15.103:0/439508405 shutdown_connections 2024-09-19T07:04:53.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:53.402+0000 7ff774b44640 1 -- 172.21.15.103:0/439508405 wait complete. 2024-09-19T07:04:53.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:53 smithi174 ceph-mon[35637]: pgmap v490: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-09-19T07:04:53.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:53 smithi103 ceph-mon[31158]: pgmap v490: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-09-19T07:04:53.681 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:04:53.681 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:04:53.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:53 smithi134 ceph-mon[35356]: pgmap v490: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-09-19T07:04:54.031 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0' 2024-09-19T07:04:54.031 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0 2024-09-19T07:04:54.066 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:54.575 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:54 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/439508405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:54.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:54 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/439508405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:54.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.586+0000 7fca48bf2640 1 -- 172.21.15.103:0/3016599902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca44102ef0 msgr2=0x7fca4410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:54.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.586+0000 7fca48bf2640 1 --2- 172.21.15.103:0/3016599902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca44102ef0 0x7fca4410da40 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009ca0 tx=0x7fca3c02a950 comp rx=0 tx=0).stop 2024-09-19T07:04:54.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.586+0000 7fca48bf2640 1 -- 172.21.15.103:0/3016599902 shutdown_connections 2024-09-19T07:04:54.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.586+0000 7fca48bf2640 1 --2- 172.21.15.103:0/3016599902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca44102ef0 0x7fca4410da40 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.586+0000 7fca48bf2640 1 --2- 172.21.15.103:0/3016599902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca44102570 0x7fca441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.586+0000 7fca48bf2640 1 --2- 172.21.15.103:0/3016599902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca44100e20 0x7fca44101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.586+0000 7fca48bf2640 1 -- 172.21.15.103:0/3016599902 >> 172.21.15.103:0/3016599902 conn(0x7fca440fc7d0 msgr2=0x7fca440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:54.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.586+0000 7fca48bf2640 1 -- 172.21.15.103:0/3016599902 shutdown_connections 2024-09-19T07:04:54.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.587+0000 7fca48bf2640 1 -- 172.21.15.103:0/3016599902 wait complete. 2024-09-19T07:04:54.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.588+0000 7fca48bf2640 1 Processor -- start 2024-09-19T07:04:54.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.588+0000 7fca48bf2640 1 -- start start 2024-09-19T07:04:54.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.588+0000 7fca48bf2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca44100e20 0x7fca44199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.588+0000 7fca48bf2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca44102570 0x7fca441998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.588+0000 7fca48bf2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca44102ef0 0x7fca441a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.588+0000 7fca48bf2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca44110f80 con 0x7fca44102ef0 2024-09-19T07:04:54.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.588+0000 7fca48bf2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca44110e00 con 0x7fca44100e20 2024-09-19T07:04:54.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.588+0000 7fca48bf2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca44111100 con 0x7fca44102570 2024-09-19T07:04:54.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.588+0000 7fca42d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca44102ef0 0x7fca441a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.588+0000 7fca42d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca44102ef0 0x7fca441a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53256/0 (socket says 172.21.15.103:53256) 2024-09-19T07:04:54.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.588+0000 7fca42d9d640 1 -- 172.21.15.103:0/3328891347 learned_addr learned my addr 172.21.15.103:0/3328891347 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:54.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.588+0000 7fca41d9b640 1 --2- 172.21.15.103:0/3328891347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca44102570 0x7fca441998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.588+0000 7fca4259c640 1 --2- 172.21.15.103:0/3328891347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca44100e20 0x7fca44199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.589+0000 7fca42d9d640 1 -- 172.21.15.103:0/3328891347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca44102570 msgr2=0x7fca441998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:54.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.589+0000 7fca42d9d640 1 --2- 172.21.15.103:0/3328891347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca44102570 0x7fca441998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.589+0000 7fca42d9d640 1 -- 172.21.15.103:0/3328891347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca44100e20 msgr2=0x7fca44199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:54.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.589+0000 7fca42d9d640 1 --2- 172.21.15.103:0/3328891347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca44100e20 0x7fca44199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.589+0000 7fca42d9d640 1 -- 172.21.15.103:0/3328891347 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca441a4930 con 0x7fca44102ef0 2024-09-19T07:04:54.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.589+0000 7fca42d9d640 1 --2- 172.21.15.103:0/3328891347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca44102ef0 0x7fca441a43f0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7fca3c02ae70 tx=0x7fca3c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:54.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.589+0000 7fca337fe640 1 -- 172.21.15.103:0/3328891347 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c034040 con 0x7fca44102ef0 2024-09-19T07:04:54.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.589+0000 7fca337fe640 1 -- 172.21.15.103:0/3328891347 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca3c030920 con 0x7fca44102ef0 2024-09-19T07:04:54.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.589+0000 7fca48bf2640 1 -- 172.21.15.103:0/3328891347 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca441a4b00 con 0x7fca44102ef0 2024-09-19T07:04:54.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.589+0000 7fca337fe640 1 -- 172.21.15.103:0/3328891347 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c045480 con 0x7fca44102ef0 2024-09-19T07:04:54.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.590+0000 7fca48bf2640 1 -- 172.21.15.103:0/3328891347 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca441a4f50 con 0x7fca44102ef0 2024-09-19T07:04:54.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.592+0000 7fca4afcb740 1 Processor -- start 2024-09-19T07:04:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.592+0000 7fca337fe640 1 -- 172.21.15.103:0/3328891347 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fca3c037020 con 0x7fca44102ef0 2024-09-19T07:04:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca4afcb740 1 -- start start 2024-09-19T07:04:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca4afcb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2cab60 0x5635fb2caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca4afcb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cb480 0x5635fb2d5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca4afcb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2d5d40 0x5635fb2d8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca4afcb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5635fb230280 con 0x5635fb2d5d40 2024-09-19T07:04:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca4afcb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5635fb1d7b10 con 0x5635fb2cab60 2024-09-19T07:04:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca4afcb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5635fb22c240 con 0x5635fb2cb480 2024-09-19T07:04:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca41d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2d5d40 0x5635fb2d8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca41d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2d5d40 0x5635fb2d8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53264/0 (socket says 172.21.15.103:53264) 2024-09-19T07:04:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca41d9b640 1 -- 172.21.15.103:0/3307745822 learned_addr learned my addr 172.21.15.103:0/3307745822 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca4259c640 1 --2- 172.21.15.103:0/3307745822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cb480 0x5635fb2d5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca41d9b640 1 --2- 172.21.15.103:0/3307745822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2cab60 0x5635fb2caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca337fe640 1 --2- 172.21.15.103:0/3328891347 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fca2c0a33b0 0x7fca2c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca4259c640 1 -- 172.21.15.103:0/3307745822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2cab60 msgr2=0x5635fb2caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca4259c640 1 --2- 172.21.15.103:0/3307745822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2cab60 0x5635fb2caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca4259c640 1 -- 172.21.15.103:0/3307745822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2d5d40 msgr2=0x5635fb2d8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca4259c640 1 --2- 172.21.15.103:0/3307745822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2d5d40 0x5635fb2d8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.593+0000 7fca4259c640 1 -- 172.21.15.103:0/3307745822 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635fb1e41f0 con 0x5635fb2cb480 2024-09-19T07:04:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.594+0000 7fca337fe640 1 -- 172.21.15.103:0/3328891347 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fca3c0f0a90 con 0x7fca44102ef0 2024-09-19T07:04:54.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.594+0000 7fca42d9d640 1 --2- 172.21.15.103:0/3328891347 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fca2c0a33b0 0x7fca2c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.594+0000 7fca41d9b640 1 --2- 172.21.15.103:0/3307745822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2cab60 0x5635fb2caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:54.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.594+0000 7fca4259c640 1 --2- 172.21.15.103:0/3307745822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cb480 0x5635fb2d5800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fca3400bbf0 tx=0x7fca340107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74dfaf4abf5dcccf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.594+0000 7fca30ff9640 1 -- 172.21.15.103:0/3307745822 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34004030 con 0x5635fb2cb480 2024-09-19T07:04:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.594+0000 7fca30ff9640 1 -- 172.21.15.103:0/3307745822 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca340041d0 con 0x5635fb2cb480 2024-09-19T07:04:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.594+0000 7fca30ff9640 1 -- 172.21.15.103:0/3307745822 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca340116c0 con 0x5635fb2cb480 2024-09-19T07:04:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.595+0000 7fca4afcb740 1 -- 172.21.15.103:0/3307745822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cb480 msgr2=0x5635fb2d5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.595+0000 7fca4afcb740 1 --2- 172.21.15.103:0/3307745822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cb480 0x5635fb2d5800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fca3400bbf0 tx=0x7fca340107a0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.595+0000 7fca42d9d640 1 --2- 172.21.15.103:0/3328891347 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fca2c0a33b0 0x7fca2c0a5870 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0c55a0 tx=0x7fca3c0095c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.595+0000 7fca4afcb740 1 -- 172.21.15.103:0/3307745822 shutdown_connections 2024-09-19T07:04:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.595+0000 7fca4afcb740 1 --2- 172.21.15.103:0/3307745822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2d5d40 0x5635fb2d8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.595+0000 7fca4afcb740 1 --2- 172.21.15.103:0/3307745822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cb480 0x5635fb2d5800 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.595+0000 7fca4afcb740 1 --2- 172.21.15.103:0/3307745822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2cab60 0x5635fb2caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.595+0000 7fca4afcb740 1 -- 172.21.15.103:0/3307745822 >> 172.21.15.103:0/3307745822 conn(0x5635fb2c70f0 msgr2=0x5635fb2c74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.595+0000 7fca4afcb740 1 -- 172.21.15.103:0/3307745822 shutdown_connections 2024-09-19T07:04:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.596+0000 7fca4afcb740 1 -- 172.21.15.103:0/3307745822 wait complete. 2024-09-19T07:04:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.596+0000 7fca4afcb740 1 -- start start 2024-09-19T07:04:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.596+0000 7fca4afcb740 1 Processor -- start 2024-09-19T07:04:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.596+0000 7fca4afcb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cab60 0x5635fb2d2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.596+0000 7fca4afcb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cb480 0x5635fb2d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.596+0000 7fca4afcb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2d5d40 0x5635fb2d3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.596+0000 7fca4afcb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5635fb2dcd20 con 0x5635fb2cab60 2024-09-19T07:04:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.596+0000 7fca4afcb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5635fb1eaf10 con 0x5635fb2d5d40 2024-09-19T07:04:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.596+0000 7fca4afcb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5635fb2dcea0 con 0x5635fb2cb480 2024-09-19T07:04:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.596+0000 7fca4afcb740 10 client.0 _ll_register_callbacks cb 0x5635fb1b3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.596+0000 7fca41d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cab60 0x5635fb2d2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.596+0000 7fca41d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cab60 0x5635fb2d2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53270/0 (socket says 172.21.15.103:53270) 2024-09-19T07:04:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.596+0000 7fca41d9b640 1 -- 172.21.15.103:0/1131170797 learned_addr learned my addr 172.21.15.103:0/1131170797 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.597+0000 7fca4259c640 1 --2- 172.21.15.103:0/1131170797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cb480 0x5635fb2d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.597+0000 7fca41d9b640 1 --2- 172.21.15.103:0/1131170797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2d5d40 0x5635fb2d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.597+0000 7fca41d9b640 1 -- 172.21.15.103:0/1131170797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cb480 msgr2=0x5635fb2d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.597+0000 7fca41d9b640 1 --2- 172.21.15.103:0/1131170797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cb480 0x5635fb2d3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.597+0000 7fca41d9b640 1 -- 172.21.15.103:0/1131170797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2d5d40 msgr2=0x5635fb2d3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.597+0000 7fca41d9b640 1 --2- 172.21.15.103:0/1131170797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2d5d40 0x5635fb2d3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.597+0000 7fca41d9b640 1 -- 172.21.15.103:0/1131170797 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635fafa16a0 con 0x5635fb2cab60 2024-09-19T07:04:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.597+0000 7fca4259c640 1 --2- 172.21.15.103:0/1131170797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cb480 0x5635fb2d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.597+0000 7fca41d9b640 1 --2- 172.21.15.103:0/1131170797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cab60 0x5635fb2d2be0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fca38012a90 tx=0x7fca38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5f0f4b024f22a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.597+0000 7fca29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.597+0000 7fca29ffb640 1 -- 172.21.15.103:0/1131170797 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38011c70 con 0x5635fb2cab60 2024-09-19T07:04:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.597+0000 7fca29ffb640 1 -- 172.21.15.103:0/1131170797 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fca38011e10 con 0x5635fb2cab60 2024-09-19T07:04:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.597+0000 7fca4afcb740 1 -- 172.21.15.103:0/1131170797 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca340041d0 con 0x5635fb2cab60 2024-09-19T07:04:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.597+0000 7fca29ffb640 1 -- 172.21.15.103:0/1131170797 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca38028050 con 0x5635fb2cab60 2024-09-19T07:04:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.597+0000 7fca4afcb740 10 client.20904 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.597+0000 7fca4afcb740 1 -- 172.21.15.103:0/1131170797 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635fb2e8980 con 0x5635fb2cab60 2024-09-19T07:04:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.598+0000 7fca29ffb640 1 -- 172.21.15.103:0/1131170797 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca340041d0 con 0x5635fb2cab60 2024-09-19T07:04:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.598+0000 7fca29ffb640 1 -- 172.21.15.103:0/1131170797 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fca380047a0 con 0x5635fb2cab60 2024-09-19T07:04:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.598+0000 7fca4afcb740 10 client.20904 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.598+0000 7fca4afcb740 10 client.20904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.598+0000 7fca4afcb740 20 client.20904 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.598+0000 7fca4afcb740 1 --2- 172.21.15.103:0/1131170797 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5635fb2ea1c0 0x5635fb2ec640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.598+0000 7fca4afcb740 4 client.20904 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.598+0000 7fca4afcb740 1 -- 172.21.15.103:0/1131170797 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635fb23d7f0 con 0x5635fb2ea1c0 2024-09-19T07:04:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.598+0000 7fca4259c640 1 --2- 172.21.15.103:0/1131170797 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5635fb2ea1c0 0x5635fb2ec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.599+0000 7fca29ffb640 10 client.20904 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.599+0000 7fca4259c640 1 --2- 172.21.15.103:0/1131170797 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5635fb2ea1c0 0x5635fb2ec640 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b971241f80335b03 server_cookie=12cb162fdb0f4ece in_seq=0 out_seq=0 2024-09-19T07:04:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.599+0000 7fca29ffb640 1 -- 172.21.15.103:0/1131170797 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635fb23d7f0 con 0x5635fb2ea1c0 2024-09-19T07:04:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.599+0000 7fca29ffb640 10 client.20904 handle_command_reply: tid=0 2024-09-19T07:04:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.599+0000 7fca4afcb740 1 client.20904 shutdown 2024-09-19T07:04:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.600+0000 7fca4afcb740 1 -- 172.21.15.103:0/1131170797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cab60 msgr2=0x5635fb2d2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.600+0000 7fca4afcb740 1 --2- 172.21.15.103:0/1131170797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cab60 0x5635fb2d2be0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fca38012a90 tx=0x7fca38004730 comp rx=0 tx=0).stop 2024-09-19T07:04:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.600+0000 7fca4afcb740 1 -- 172.21.15.103:0/1131170797 shutdown_connections 2024-09-19T07:04:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.600+0000 7fca4afcb740 1 --2- 172.21.15.103:0/1131170797 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5635fb2ea1c0 0x5635fb2ec640 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.600+0000 7fca4afcb740 1 --2- 172.21.15.103:0/1131170797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2d5d40 0x5635fb2d3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.600+0000 7fca4afcb740 1 --2- 172.21.15.103:0/1131170797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cb480 0x5635fb2d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.600+0000 7fca29ffb640 0 client.20904 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.600+0000 7fca4afcb740 1 --2- 172.21.15.103:0/1131170797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cab60 0x5635fb2d2be0 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.600+0000 7fca4afcb740 1 -- 172.21.15.103:0/1131170797 >> 172.21.15.103:0/1131170797 conn(0x5635fb2c70f0 msgr2=0x5635fb2c6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.600+0000 7fca4afcb740 1 -- 172.21.15.103:0/1131170797 shutdown_connections 2024-09-19T07:04:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.600+0000 7fca4afcb740 1 -- 172.21.15.103:0/1131170797 wait complete. 2024-09-19T07:04:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.600+0000 7fca4afcb740 20 client.20904 trim_cache size 0 max 16384 2024-09-19T07:04:54.616 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:54.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.618+0000 7fca4afcb740 1 Processor -- start 2024-09-19T07:04:54.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.618+0000 7fca4afcb740 1 -- start start 2024-09-19T07:04:54.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.618+0000 7fca4afcb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cab60 0x5635fb2cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.618+0000 7fca4afcb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cb480 0x5635fb2d4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.618+0000 7fca4afcb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2d5d40 0x5635fb2f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.618+0000 7fca4afcb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca441ab650 con 0x5635fb2cb480 2024-09-19T07:04:54.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.618+0000 7fca4afcb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca441ad190 con 0x5635fb2d5d40 2024-09-19T07:04:54.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.618+0000 7fca4afcb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca441abd20 con 0x5635fb2cab60 2024-09-19T07:04:54.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.619+0000 7fca4259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cb480 0x5635fb2d4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.619+0000 7fca4259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cb480 0x5635fb2d4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53274/0 (socket says 172.21.15.103:53274) 2024-09-19T07:04:54.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.619+0000 7fca4259c640 1 -- 172.21.15.103:0/29177605 learned_addr learned my addr 172.21.15.103:0/29177605 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.619+0000 7fca41d9b640 1 --2- 172.21.15.103:0/29177605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cab60 0x5635fb2cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.619+0000 7fca41d9b640 1 --2- 172.21.15.103:0/29177605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2d5d40 0x5635fb2f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.619+0000 7fca4259c640 1 -- 172.21.15.103:0/29177605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cab60 msgr2=0x5635fb2cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.619+0000 7fca4259c640 1 --2- 172.21.15.103:0/29177605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cab60 0x5635fb2cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.619+0000 7fca4259c640 1 -- 172.21.15.103:0/29177605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2d5d40 msgr2=0x5635fb2f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.619+0000 7fca4259c640 1 --2- 172.21.15.103:0/29177605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2d5d40 0x5635fb2f1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.619+0000 7fca4259c640 1 -- 172.21.15.103:0/29177605 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635fb2d54b0 con 0x5635fb2cb480 2024-09-19T07:04:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.619+0000 7fca41d9b640 1 --2- 172.21.15.103:0/29177605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cab60 0x5635fb2cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:54.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.619+0000 7fca4259c640 1 --2- 172.21.15.103:0/29177605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cb480 0x5635fb2d4a40 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x5635fb2c7f50 tx=0x7fca34003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372c71e4b121d717 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:54.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.619+0000 7fca2bfff640 1 -- 172.21.15.103:0/29177605 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3402c020 con 0x5635fb2cb480 2024-09-19T07:04:54.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.619+0000 7fca2bfff640 1 -- 172.21.15.103:0/29177605 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca340035f0 con 0x5635fb2cb480 2024-09-19T07:04:54.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.619+0000 7fca2bfff640 1 -- 172.21.15.103:0/29177605 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca340116c0 con 0x5635fb2cb480 2024-09-19T07:04:54.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.620+0000 7fca4afcb740 1 -- 172.21.15.103:0/29177605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cb480 msgr2=0x5635fb2d4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:54.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.620+0000 7fca4afcb740 1 --2- 172.21.15.103:0/29177605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cb480 0x5635fb2d4a40 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x5635fb2c7f50 tx=0x7fca34003dc0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.620+0000 7fca4afcb740 1 -- 172.21.15.103:0/29177605 shutdown_connections 2024-09-19T07:04:54.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.620+0000 7fca4afcb740 1 --2- 172.21.15.103:0/29177605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2d5d40 0x5635fb2f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.620+0000 7fca4afcb740 1 --2- 172.21.15.103:0/29177605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cb480 0x5635fb2d4a40 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.620+0000 7fca4afcb740 1 --2- 172.21.15.103:0/29177605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2cab60 0x5635fb2cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.620+0000 7fca4afcb740 1 -- 172.21.15.103:0/29177605 >> 172.21.15.103:0/29177605 conn(0x5635fb2c70f0 msgr2=0x5635fb2d4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:54.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.620+0000 7fca4afcb740 1 -- 172.21.15.103:0/29177605 shutdown_connections 2024-09-19T07:04:54.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.620+0000 7fca4afcb740 1 -- 172.21.15.103:0/29177605 wait complete. 2024-09-19T07:04:54.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca4afcb740 1 -- start start 2024-09-19T07:04:54.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca4afcb740 1 Processor -- start 2024-09-19T07:04:54.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca4afcb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cab60 0x5635fb2dddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca4afcb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2cb480 0x5635fb2de310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca4afcb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2d5d40 0x5635fb2de850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca4afcb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca441acac0 con 0x5635fb2cab60 2024-09-19T07:04:54.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca4afcb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca441ac3f0 con 0x5635fb2cb480 2024-09-19T07:04:54.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca4afcb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca441a9630 con 0x5635fb2d5d40 2024-09-19T07:04:54.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca4afcb740 10 client.0 _ll_register_callbacks cb 0x7fca3800bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca41d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cab60 0x5635fb2dddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca41d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cab60 0x5635fb2dddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53276/0 (socket says 172.21.15.103:53276) 2024-09-19T07:04:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca41d9b640 1 -- 172.21.15.103:0/1871100551 learned_addr learned my addr 172.21.15.103:0/1871100551 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca4259c640 1 --2- 172.21.15.103:0/1871100551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2cb480 0x5635fb2de310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca41d9b640 1 -- 172.21.15.103:0/1871100551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2d5d40 msgr2=0x5635fb2de850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca41d9b640 1 --2- 172.21.15.103:0/1871100551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2d5d40 0x5635fb2de850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca41d9b640 1 -- 172.21.15.103:0/1871100551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2cb480 msgr2=0x5635fb2de310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.621+0000 7fca41d9b640 1 --2- 172.21.15.103:0/1871100551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2cb480 0x5635fb2de310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.622+0000 7fca41d9b640 1 -- 172.21.15.103:0/1871100551 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635fb353f20 con 0x5635fb2cab60 2024-09-19T07:04:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.622+0000 7fca41d9b640 1 --2- 172.21.15.103:0/1871100551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cab60 0x5635fb2dddd0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fca38004d10 tx=0x7fca38004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0a7632c98306ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.622+0000 7fca29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.622+0000 7fca29ffb640 1 -- 172.21.15.103:0/1871100551 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38011070 con 0x5635fb2cab60 2024-09-19T07:04:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.622+0000 7fca29ffb640 1 -- 172.21.15.103:0/1871100551 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fca38011c70 con 0x5635fb2cab60 2024-09-19T07:04:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.622+0000 7fca4afcb740 1 -- 172.21.15.103:0/1871100551 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca340035f0 con 0x5635fb2cab60 2024-09-19T07:04:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.622+0000 7fca29ffb640 1 -- 172.21.15.103:0/1871100551 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca38011210 con 0x5635fb2cab60 2024-09-19T07:04:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.622+0000 7fca4afcb740 10 client.20910 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.622+0000 7fca4afcb740 1 -- 172.21.15.103:0/1871100551 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635fb3551a0 con 0x5635fb2cab60 2024-09-19T07:04:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.622+0000 7fca29ffb640 1 -- 172.21.15.103:0/1871100551 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca340035f0 con 0x5635fb2cab60 2024-09-19T07:04:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.623+0000 7fca29ffb640 1 -- 172.21.15.103:0/1871100551 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fca38021540 con 0x5635fb2cab60 2024-09-19T07:04:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.623+0000 7fca4afcb740 10 client.20910 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.623+0000 7fca4afcb740 10 client.20910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.623+0000 7fca4afcb740 20 client.20910 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.623+0000 7fca4afcb740 1 --2- 172.21.15.103:0/1871100551 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5635fb3558f0 0x5635fb357d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.623+0000 7fca4afcb740 4 client.20910 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:04:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.623+0000 7fca4afcb740 1 -- 172.21.15.103:0/1871100551 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5635fb237d00 con 0x5635fb3558f0 2024-09-19T07:04:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.623+0000 7fca4259c640 1 --2- 172.21.15.103:0/1871100551 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5635fb3558f0 0x5635fb357d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.624+0000 7fca29ffb640 10 client.20910 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:54.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.624+0000 7fca4259c640 1 --2- 172.21.15.103:0/1871100551 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5635fb3558f0 0x5635fb357d10 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=132f84fcdcca3d77 server_cookie=3bed2d79b0d55409 in_seq=0 out_seq=0 2024-09-19T07:04:54.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.625+0000 7fca29ffb640 1 -- 172.21.15.103:0/1871100551 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5635fb237d00 con 0x5635fb3558f0 2024-09-19T07:04:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.625+0000 7fca29ffb640 10 client.20910 handle_command_reply: tid=0 2024-09-19T07:04:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.625+0000 7fca4afcb740 1 client.20910 shutdown 2024-09-19T07:04:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.625+0000 7fca4afcb740 1 -- 172.21.15.103:0/1871100551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cab60 msgr2=0x5635fb2dddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.625+0000 7fca4afcb740 1 --2- 172.21.15.103:0/1871100551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cab60 0x5635fb2dddd0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fca38004d10 tx=0x7fca38004e90 comp rx=0 tx=0).stop 2024-09-19T07:04:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.625+0000 7fca4afcb740 1 -- 172.21.15.103:0/1871100551 shutdown_connections 2024-09-19T07:04:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.625+0000 7fca4afcb740 1 --2- 172.21.15.103:0/1871100551 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5635fb3558f0 0x5635fb357d10 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.625+0000 7fca4afcb740 1 --2- 172.21.15.103:0/1871100551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635fb2d5d40 0x5635fb2de850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.625+0000 7fca4afcb740 1 --2- 172.21.15.103:0/1871100551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635fb2cb480 0x5635fb2de310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.625+0000 7fca29ffb640 0 client.20910 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.625+0000 7fca4afcb740 1 --2- 172.21.15.103:0/1871100551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635fb2cab60 0x5635fb2dddd0 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.625+0000 7fca4afcb740 1 -- 172.21.15.103:0/1871100551 >> 172.21.15.103:0/1871100551 conn(0x5635fb2c70f0 msgr2=0x5635fb2c6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.625+0000 7fca4afcb740 1 -- 172.21.15.103:0/1871100551 shutdown_connections 2024-09-19T07:04:54.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.625+0000 7fca4afcb740 1 -- 172.21.15.103:0/1871100551 wait complete. 2024-09-19T07:04:54.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.625+0000 7fca4afcb740 20 client.20910 trim_cache size 0 max 16384 2024-09-19T07:04:54.631 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:04:54.631 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:04:54.631 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "6bd7a26f-4d25-4d88-8d00-eea6000384a7", 2024-09-19T07:04:54.631 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:04:54.631 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:04:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.626+0000 7fca48bf2640 1 -- 172.21.15.103:0/3328891347 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fca2c0a33b0 msgr2=0x7fca2c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.626+0000 7fca48bf2640 1 --2- 172.21.15.103:0/3328891347 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fca2c0a33b0 0x7fca2c0a5870 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0c55a0 tx=0x7fca3c0095c0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.626+0000 7fca48bf2640 1 -- 172.21.15.103:0/3328891347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca44102ef0 msgr2=0x7fca441a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.626+0000 7fca48bf2640 1 --2- 172.21.15.103:0/3328891347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca44102ef0 0x7fca441a43f0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7fca3c02ae70 tx=0x7fca3c02f5d0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.626+0000 7fca48bf2640 1 -- 172.21.15.103:0/3328891347 shutdown_connections 2024-09-19T07:04:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.626+0000 7fca48bf2640 1 --2- 172.21.15.103:0/3328891347 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fca2c0a33b0 0x7fca2c0a5870 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.626+0000 7fca48bf2640 1 --2- 172.21.15.103:0/3328891347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca44102ef0 0x7fca441a43f0 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.626+0000 7fca48bf2640 1 --2- 172.21.15.103:0/3328891347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca44102570 0x7fca441998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.626+0000 7fca48bf2640 1 --2- 172.21.15.103:0/3328891347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca44100e20 0x7fca44199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.626+0000 7fca48bf2640 1 -- 172.21.15.103:0/3328891347 >> 172.21.15.103:0/3328891347 conn(0x7fca440fc7d0 msgr2=0x7fca440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.626+0000 7fca48bf2640 1 -- 172.21.15.103:0/3328891347 shutdown_connections 2024-09-19T07:04:54.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:54.626+0000 7fca48bf2640 1 -- 172.21.15.103:0/3328891347 wait complete. 2024-09-19T07:04:54.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:54 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/439508405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:55.393 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:04:55.575 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:55 smithi103 ceph-mon[31158]: pgmap v491: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-09-19T07:04:55.575 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:55 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T07:04:55.575 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:55 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:04:55.575 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:55 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T07:04:55.575 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:55 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:04:55.575 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:55 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T07:04:55.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:55 smithi174 ceph-mon[35637]: pgmap v491: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-09-19T07:04:55.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:55 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T07:04:55.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:55 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:04:55.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:55 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T07:04:55.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:55 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:04:55.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:55 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T07:04:55.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:55 smithi134 ceph-mon[35356]: pgmap v491: 97 pgs: 97 active+clean; 3.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-09-19T07:04:55.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:55 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T07:04:55.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:55 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:04:55.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:55 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T07:04:55.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:55 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:04:55.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:55 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T07:04:55.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.781+0000 7fc07fb19640 1 -- 172.21.15.103:0/1611569976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc078102570 msgr2=0x7fc0781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:55.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.781+0000 7fc07fb19640 1 --2- 172.21.15.103:0/1611569976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc078102570 0x7fc0781029b0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fc068009900 tx=0x7fc06802a480 comp rx=0 tx=0).stop 2024-09-19T07:04:55.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.782+0000 7fc07fb19640 1 -- 172.21.15.103:0/1611569976 shutdown_connections 2024-09-19T07:04:55.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.782+0000 7fc07fb19640 1 --2- 172.21.15.103:0/1611569976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc078102ef0 0x7fc07810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.782+0000 7fc07fb19640 1 --2- 172.21.15.103:0/1611569976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc078102570 0x7fc0781029b0 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.782+0000 7fc07fb19640 1 --2- 172.21.15.103:0/1611569976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc078100e20 0x7fc078101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.782+0000 7fc07fb19640 1 -- 172.21.15.103:0/1611569976 >> 172.21.15.103:0/1611569976 conn(0x7fc0780fc7d0 msgr2=0x7fc0780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:55.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.782+0000 7fc07fb19640 1 -- 172.21.15.103:0/1611569976 shutdown_connections 2024-09-19T07:04:55.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.782+0000 7fc07fb19640 1 -- 172.21.15.103:0/1611569976 wait complete. 2024-09-19T07:04:55.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.782+0000 7fc07fb19640 1 Processor -- start 2024-09-19T07:04:55.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.782+0000 7fc07fb19640 1 -- start start 2024-09-19T07:04:55.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.782+0000 7fc07fb19640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc078100e20 0x7fc078198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.782+0000 7fc07fb19640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc078102570 0x7fc078198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07fb19640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc078102ef0 0x7fc07819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07fb19640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc078110fa0 con 0x7fc078102570 2024-09-19T07:04:55.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07fb19640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc078110e20 con 0x7fc078102ef0 2024-09-19T07:04:55.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07fb19640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc078111120 con 0x7fc078100e20 2024-09-19T07:04:55.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07d0b4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc078102570 0x7fc078198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07d0b4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc078102570 0x7fc078198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53286/0 (socket says 172.21.15.103:53286) 2024-09-19T07:04:55.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07d0b4640 1 -- 172.21.15.103:0/39529372 learned_addr learned my addr 172.21.15.103:0/39529372 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:55.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07d8b5640 1 --2- 172.21.15.103:0/39529372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc078100e20 0x7fc078198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07e0b6640 1 --2- 172.21.15.103:0/39529372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc078102ef0 0x7fc07819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07d0b4640 1 -- 172.21.15.103:0/39529372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc078100e20 msgr2=0x7fc078198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:55.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07d0b4640 1 --2- 172.21.15.103:0/39529372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc078100e20 0x7fc078198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07d0b4640 1 -- 172.21.15.103:0/39529372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc078102ef0 msgr2=0x7fc07819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:55.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07d0b4640 1 --2- 172.21.15.103:0/39529372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc078102ef0 0x7fc07819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07d0b4640 1 -- 172.21.15.103:0/39529372 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07819d4b0 con 0x7fc078102570 2024-09-19T07:04:55.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07d8b5640 1 --2- 172.21.15.103:0/39529372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc078100e20 0x7fc078198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:55.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07e0b6640 1 --2- 172.21.15.103:0/39529372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc078102ef0 0x7fc07819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:55.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07d0b4640 1 --2- 172.21.15.103:0/39529372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc078102570 0x7fc078198a40 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fc068006010 tx=0x7fc068003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:55.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc06effd640 1 -- 172.21.15.103:0/39529372 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06803f070 con 0x7fc078102570 2024-09-19T07:04:55.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc07fb19640 1 -- 172.21.15.103:0/39529372 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07819d740 con 0x7fc078102570 2024-09-19T07:04:55.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc06effd640 1 -- 172.21.15.103:0/39529372 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc068031e30 con 0x7fc078102570 2024-09-19T07:04:55.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.783+0000 7fc06effd640 1 -- 172.21.15.103:0/39529372 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068035030 con 0x7fc078102570 2024-09-19T07:04:55.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.784+0000 7fc07fb19640 1 -- 172.21.15.103:0/39529372 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0781a5080 con 0x7fc078102570 2024-09-19T07:04:55.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.786+0000 7fc06effd640 1 -- 172.21.15.103:0/39529372 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc068042070 con 0x7fc078102570 2024-09-19T07:04:55.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.786+0000 7fc081ef2740 1 Processor -- start 2024-09-19T07:04:55.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.786+0000 7fc081ef2740 1 -- start start 2024-09-19T07:04:55.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.786+0000 7fc081ef2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a53730 0x55b923a53b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.786+0000 7fc081ef2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a54050 0x55b923a62900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.786+0000 7fc081ef2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a62e40 0x55b923a65230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.786+0000 7fc081ef2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b9239b8160 con 0x55b923a54050 2024-09-19T07:04:55.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.786+0000 7fc081ef2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b92395f720 con 0x55b923a62e40 2024-09-19T07:04:55.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.786+0000 7fc081ef2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b9239b4120 con 0x55b923a53730 2024-09-19T07:04:55.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.786+0000 7fc06effd640 1 --2- 172.21.15.103:0/39529372 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0640a33b0 0x7fc0640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.787+0000 7fc07e0b6640 1 --2- 172.21.15.103:0/39529372 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0640a33b0 0x7fc0640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.787+0000 7fc07d0b4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a62e40 0x55b923a65230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.787+0000 7fc07d8b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a54050 0x55b923a62900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.787+0000 7fc07d0b4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a62e40 0x55b923a65230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56398/0 (socket says 172.21.15.103:56398) 2024-09-19T07:04:55.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.787+0000 7fc07d0b4640 1 -- 172.21.15.103:0/3371555421 learned_addr learned my addr 172.21.15.103:0/3371555421 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:55.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.787+0000 7fc07e0b6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a53730 0x55b923a53b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.787+0000 7fc06effd640 1 -- 172.21.15.103:0/39529372 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fc0680f0740 con 0x7fc078102570 2024-09-19T07:04:55.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.787+0000 7fc07e0b6640 1 -- 172.21.15.103:0/3371555421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a62e40 msgr2=0x55b923a65230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:55.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.787+0000 7fc07e0b6640 1 --2- 172.21.15.103:0/3371555421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a62e40 0x55b923a65230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.787+0000 7fc07e0b6640 1 -- 172.21.15.103:0/3371555421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a54050 msgr2=0x55b923a62900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:55.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.787+0000 7fc07e0b6640 1 --2- 172.21.15.103:0/3371555421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a54050 0x55b923a62900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.787+0000 7fc07e0b6640 1 -- 172.21.15.103:0/3371555421 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b923729ba0 con 0x55b923a53730 2024-09-19T07:04:55.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.787+0000 7fc07d0b4640 1 --2- 172.21.15.103:0/3371555421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a62e40 0x55b923a65230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:55.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.787+0000 7fc07d8b5640 1 --2- 172.21.15.103:0/3371555421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a54050 0x55b923a62900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:55.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.787+0000 7fc07e0b6640 1 --2- 172.21.15.103:0/39529372 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0640a33b0 0x7fc0640a5870 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc070005fd0 tx=0x7fc07000ad30 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:55.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.787+0000 7fc07e0b6640 1 --2- 172.21.15.103:0/3371555421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a53730 0x55b923a53b10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc070012fd0 tx=0x7fc070019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab5b24faac479cde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:55.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.788+0000 7fc063fff640 1 -- 172.21.15.103:0/3371555421 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070022070 con 0x55b923a53730 2024-09-19T07:04:55.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.788+0000 7fc063fff640 1 -- 172.21.15.103:0/3371555421 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc07000dde0 con 0x55b923a53730 2024-09-19T07:04:55.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.788+0000 7fc063fff640 1 -- 172.21.15.103:0/3371555421 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070010630 con 0x55b923a53730 2024-09-19T07:04:55.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.788+0000 7fc081ef2740 1 -- 172.21.15.103:0/3371555421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a53730 msgr2=0x55b923a53b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:55.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.788+0000 7fc081ef2740 1 --2- 172.21.15.103:0/3371555421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a53730 0x55b923a53b10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc070012fd0 tx=0x7fc070019000 comp rx=0 tx=0).stop 2024-09-19T07:04:55.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.788+0000 7fc081ef2740 1 -- 172.21.15.103:0/3371555421 shutdown_connections 2024-09-19T07:04:55.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.788+0000 7fc081ef2740 1 --2- 172.21.15.103:0/3371555421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a62e40 0x55b923a65230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.788+0000 7fc081ef2740 1 --2- 172.21.15.103:0/3371555421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a54050 0x55b923a62900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.788+0000 7fc081ef2740 1 --2- 172.21.15.103:0/3371555421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a53730 0x55b923a53b10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.788+0000 7fc081ef2740 1 -- 172.21.15.103:0/3371555421 >> 172.21.15.103:0/3371555421 conn(0x55b923a4fde0 msgr2=0x55b923a501c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:55.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.788+0000 7fc081ef2740 1 -- 172.21.15.103:0/3371555421 shutdown_connections 2024-09-19T07:04:55.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.788+0000 7fc081ef2740 1 -- 172.21.15.103:0/3371555421 wait complete. 2024-09-19T07:04:55.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.789+0000 7fc081ef2740 1 -- start start 2024-09-19T07:04:55.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.789+0000 7fc081ef2740 1 Processor -- start 2024-09-19T07:04:55.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.789+0000 7fc081ef2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a53730 0x55b923a5fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.789+0000 7fc081ef2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a54050 0x55b923a602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.789+0000 7fc081ef2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a62e40 0x55b923a60820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.789+0000 7fc081ef2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b923a65a10 con 0x55b923a53730 2024-09-19T07:04:55.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.789+0000 7fc081ef2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b923940900 con 0x55b923a54050 2024-09-19T07:04:55.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.789+0000 7fc081ef2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b923a65b90 con 0x55b923a62e40 2024-09-19T07:04:55.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.789+0000 7fc081ef2740 10 client.0 _ll_register_callbacks cb 0x55b92393e780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:55.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.789+0000 7fc07d8b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a53730 0x55b923a5fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.789+0000 7fc07d8b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a53730 0x55b923a5fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53314/0 (socket says 172.21.15.103:53314) 2024-09-19T07:04:55.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.789+0000 7fc07d8b5640 1 -- 172.21.15.103:0/1919589766 learned_addr learned my addr 172.21.15.103:0/1919589766 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:55.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.789+0000 7fc07d0b4640 1 --2- 172.21.15.103:0/1919589766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a54050 0x55b923a602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.789+0000 7fc07e0b6640 1 --2- 172.21.15.103:0/1919589766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a62e40 0x55b923a60820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.790+0000 7fc07d8b5640 1 -- 172.21.15.103:0/1919589766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a62e40 msgr2=0x55b923a60820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:55.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.790+0000 7fc07d8b5640 1 --2- 172.21.15.103:0/1919589766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a62e40 0x55b923a60820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.790+0000 7fc07d8b5640 1 -- 172.21.15.103:0/1919589766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a54050 msgr2=0x55b923a602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:55.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.790+0000 7fc07d8b5640 1 --2- 172.21.15.103:0/1919589766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a54050 0x55b923a602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.790+0000 7fc07d8b5640 1 -- 172.21.15.103:0/1919589766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b923980c50 con 0x55b923a53730 2024-09-19T07:04:55.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.790+0000 7fc07d0b4640 1 --2- 172.21.15.103:0/1919589766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a54050 0x55b923a602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:04:55.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.790+0000 7fc07d8b5640 1 --2- 172.21.15.103:0/1919589766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a53730 0x55b923a5fda0 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fc07400d4a0 tx=0x7fc07400d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5769cf3469d1247 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:55.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.790+0000 7fc0617fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:55.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.790+0000 7fc0617fa640 1 -- 172.21.15.103:0/1919589766 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07401c060 con 0x55b923a53730 2024-09-19T07:04:55.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.790+0000 7fc0617fa640 1 -- 172.21.15.103:0/1919589766 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fc0740146f0 con 0x55b923a53730 2024-09-19T07:04:55.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.790+0000 7fc081ef2740 1 -- 172.21.15.103:0/1919589766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc07000dde0 con 0x55b923a53730 2024-09-19T07:04:55.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.791+0000 7fc0617fa640 1 -- 172.21.15.103:0/1919589766 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc0740148f0 con 0x55b923a53730 2024-09-19T07:04:55.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.791+0000 7fc081ef2740 10 client.20922 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:55.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.791+0000 7fc081ef2740 1 -- 172.21.15.103:0/1919589766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b923a71870 con 0x55b923a53730 2024-09-19T07:04:55.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.791+0000 7fc0617fa640 1 -- 172.21.15.103:0/1919589766 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc07000dde0 con 0x55b923a53730 2024-09-19T07:04:55.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.791+0000 7fc0617fa640 1 -- 172.21.15.103:0/1919589766 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc07401fc80 con 0x55b923a53730 2024-09-19T07:04:55.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.791+0000 7fc081ef2740 10 client.20922 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:55.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.791+0000 7fc081ef2740 10 client.20922 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:04:55.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.791+0000 7fc081ef2740 20 client.20922 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:55.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.791+0000 7fc081ef2740 1 --2- 172.21.15.103:0/1919589766 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b923a72d30 0x55b923a751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.791+0000 7fc081ef2740 4 client.20922 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:55.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.791+0000 7fc081ef2740 1 -- 172.21.15.103:0/1919589766 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9239c5890 con 0x55b923a72d30 2024-09-19T07:04:55.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.791+0000 7fc07d0b4640 1 --2- 172.21.15.103:0/1919589766 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b923a72d30 0x55b923a751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.792+0000 7fc0617fa640 10 client.20922 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:55.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.792+0000 7fc07d0b4640 1 --2- 172.21.15.103:0/1919589766 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b923a72d30 0x55b923a751d0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9fa1b73be5b177d server_cookie=b6cf115d4dc0a63c in_seq=0 out_seq=0 2024-09-19T07:04:55.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.792+0000 7fc0617fa640 1 -- 172.21.15.103:0/1919589766 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9239c5890 con 0x55b923a72d30 2024-09-19T07:04:55.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.792+0000 7fc0617fa640 10 client.20922 handle_command_reply: tid=0 2024-09-19T07:04:55.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.792+0000 7fc081ef2740 1 client.20922 shutdown 2024-09-19T07:04:55.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.792+0000 7fc081ef2740 1 -- 172.21.15.103:0/1919589766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a53730 msgr2=0x55b923a5fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:55.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.792+0000 7fc081ef2740 1 --2- 172.21.15.103:0/1919589766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a53730 0x55b923a5fda0 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fc07400d4a0 tx=0x7fc07400d960 comp rx=0 tx=0).stop 2024-09-19T07:04:55.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.793+0000 7fc081ef2740 1 -- 172.21.15.103:0/1919589766 shutdown_connections 2024-09-19T07:04:55.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.793+0000 7fc081ef2740 1 --2- 172.21.15.103:0/1919589766 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b923a72d30 0x55b923a751d0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.793+0000 7fc081ef2740 1 --2- 172.21.15.103:0/1919589766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a62e40 0x55b923a60820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.793+0000 7fc081ef2740 1 --2- 172.21.15.103:0/1919589766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a54050 0x55b923a602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.793+0000 7fc081ef2740 1 --2- 172.21.15.103:0/1919589766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a53730 0x55b923a5fda0 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.793+0000 7fc0617fa640 0 client.20922 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:55.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.793+0000 7fc081ef2740 1 -- 172.21.15.103:0/1919589766 >> 172.21.15.103:0/1919589766 conn(0x55b923a4fde0 msgr2=0x55b923a55b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:55.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.793+0000 7fc081ef2740 1 -- 172.21.15.103:0/1919589766 shutdown_connections 2024-09-19T07:04:55.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.793+0000 7fc081ef2740 1 -- 172.21.15.103:0/1919589766 wait complete. 2024-09-19T07:04:55.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.793+0000 7fc081ef2740 20 client.20922 trim_cache size 0 max 16384 2024-09-19T07:04:55.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.809+0000 7fc081ef2740 1 Processor -- start 2024-09-19T07:04:55.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.809+0000 7fc081ef2740 1 -- start start 2024-09-19T07:04:55.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.809+0000 7fc081ef2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a53730 0x55b923a7aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.809+0000 7fc081ef2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a54050 0x55b923a7afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.809+0000 7fc081ef2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a62e40 0x55b923a5d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.809+0000 7fc081ef2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781ab5e0 con 0x55b923a53730 2024-09-19T07:04:55.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.809+0000 7fc081ef2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781ad120 con 0x55b923a62e40 2024-09-19T07:04:55.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.809+0000 7fc081ef2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781abcb0 con 0x55b923a54050 2024-09-19T07:04:55.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.809+0000 7fc07d8b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a53730 0x55b923a7aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.810+0000 7fc07d8b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a53730 0x55b923a7aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53318/0 (socket says 172.21.15.103:53318) 2024-09-19T07:04:55.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.810+0000 7fc07d8b5640 1 -- 172.21.15.103:0/235166208 learned_addr learned my addr 172.21.15.103:0/235166208 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:55.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.810+0000 7fc07d0b4640 1 --2- 172.21.15.103:0/235166208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a54050 0x55b923a7afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.810+0000 7fc07d8b5640 1 -- 172.21.15.103:0/235166208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a54050 msgr2=0x55b923a7afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:55.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.810+0000 7fc07d8b5640 1 --2- 172.21.15.103:0/235166208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a54050 0x55b923a7afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.810+0000 7fc07d8b5640 1 -- 172.21.15.103:0/235166208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a62e40 msgr2=0x55b923a5d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:55.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.810+0000 7fc07d8b5640 1 --2- 172.21.15.103:0/235166208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a62e40 0x55b923a5d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.810+0000 7fc07d8b5640 1 -- 172.21.15.103:0/235166208 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b923a5ed50 con 0x55b923a53730 2024-09-19T07:04:55.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.810+0000 7fc07d8b5640 1 --2- 172.21.15.103:0/235166208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a53730 0x55b923a7aaa0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fc074015610 tx=0x7fc074008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf625722a506ea9f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:55.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.810+0000 7fc0637fe640 1 -- 172.21.15.103:0/235166208 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07402c030 con 0x55b923a53730 2024-09-19T07:04:55.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.810+0000 7fc0637fe640 1 -- 172.21.15.103:0/235166208 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc074007d10 con 0x55b923a53730 2024-09-19T07:04:55.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.810+0000 7fc0637fe640 1 -- 172.21.15.103:0/235166208 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074015c60 con 0x55b923a53730 2024-09-19T07:04:55.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.811+0000 7fc081ef2740 1 -- 172.21.15.103:0/235166208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a53730 msgr2=0x55b923a7aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:55.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.811+0000 7fc081ef2740 1 --2- 172.21.15.103:0/235166208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a53730 0x55b923a7aaa0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fc074015610 tx=0x7fc074008040 comp rx=0 tx=0).stop 2024-09-19T07:04:55.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.811+0000 7fc081ef2740 1 -- 172.21.15.103:0/235166208 shutdown_connections 2024-09-19T07:04:55.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.811+0000 7fc081ef2740 1 --2- 172.21.15.103:0/235166208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a62e40 0x55b923a5d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.811+0000 7fc081ef2740 1 --2- 172.21.15.103:0/235166208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a54050 0x55b923a7afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.811+0000 7fc081ef2740 1 --2- 172.21.15.103:0/235166208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a53730 0x55b923a7aaa0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.811+0000 7fc081ef2740 1 -- 172.21.15.103:0/235166208 >> 172.21.15.103:0/235166208 conn(0x55b923a4fde0 msgr2=0x55b923a55ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:55.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.811+0000 7fc081ef2740 1 -- 172.21.15.103:0/235166208 shutdown_connections 2024-09-19T07:04:55.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.811+0000 7fc081ef2740 1 -- 172.21.15.103:0/235166208 wait complete. 2024-09-19T07:04:55.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.811+0000 7fc081ef2740 1 -- start start 2024-09-19T07:04:55.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.811+0000 7fc081ef2740 1 Processor -- start 2024-09-19T07:04:55.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc081ef2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a53730 0x55b923a5cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc081ef2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a54050 0x55b923a5d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc081ef2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a62e40 0x55b923a5d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc081ef2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781aca50 con 0x55b923a62e40 2024-09-19T07:04:55.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc081ef2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781ac380 con 0x55b923a54050 2024-09-19T07:04:55.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc081ef2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781a95e0 con 0x55b923a53730 2024-09-19T07:04:55.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc07e0b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a62e40 0x55b923a5d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc07e0b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a62e40 0x55b923a5d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53332/0 (socket says 172.21.15.103:53332) 2024-09-19T07:04:55.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc07e0b6640 1 -- 172.21.15.103:0/3524414087 learned_addr learned my addr 172.21.15.103:0/3524414087 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:55.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc081ef2740 10 client.0 _ll_register_callbacks cb 0x7fc074007500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:55.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc07d8b5640 1 --2- 172.21.15.103:0/3524414087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a53730 0x55b923a5cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc07d0b4640 1 --2- 172.21.15.103:0/3524414087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a54050 0x55b923a5d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc07e0b6640 1 -- 172.21.15.103:0/3524414087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a53730 msgr2=0x55b923a5cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:55.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc07e0b6640 1 --2- 172.21.15.103:0/3524414087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a53730 0x55b923a5cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc07e0b6640 1 -- 172.21.15.103:0/3524414087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a54050 msgr2=0x55b923a5d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:55.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc07e0b6640 1 --2- 172.21.15.103:0/3524414087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a54050 0x55b923a5d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc07e0b6640 1 -- 172.21.15.103:0/3524414087 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b923adcc70 con 0x55b923a62e40 2024-09-19T07:04:55.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.812+0000 7fc07d0b4640 1 --2- 172.21.15.103:0/3524414087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a54050 0x55b923a5d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:55.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.813+0000 7fc07e0b6640 1 --2- 172.21.15.103:0/3524414087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a62e40 0x55b923a5d9f0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fc07001ec80 tx=0x7fc07000ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422b3735bdbfa06f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:55.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.813+0000 7fc0617fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:55.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.813+0000 7fc0617fa640 1 -- 172.21.15.103:0/3524414087 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070038060 con 0x55b923a62e40 2024-09-19T07:04:55.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.813+0000 7fc0617fa640 1 -- 172.21.15.103:0/3524414087 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fc070012470 con 0x55b923a62e40 2024-09-19T07:04:55.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.813+0000 7fc081ef2740 1 -- 172.21.15.103:0/3524414087 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc074007d10 con 0x55b923a62e40 2024-09-19T07:04:55.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.813+0000 7fc0617fa640 1 -- 172.21.15.103:0/3524414087 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc070022040 con 0x55b923a62e40 2024-09-19T07:04:55.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.813+0000 7fc081ef2740 10 client.20928 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:55.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.813+0000 7fc081ef2740 1 -- 172.21.15.103:0/3524414087 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b923addef0 con 0x55b923a62e40 2024-09-19T07:04:55.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.813+0000 7fc0617fa640 1 -- 172.21.15.103:0/3524414087 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc074007d10 con 0x55b923a62e40 2024-09-19T07:04:55.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.813+0000 7fc0617fa640 1 -- 172.21.15.103:0/3524414087 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc070027450 con 0x55b923a62e40 2024-09-19T07:04:55.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.813+0000 7fc081ef2740 10 client.20928 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:55.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.813+0000 7fc081ef2740 10 client.20928 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:04:55.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.813+0000 7fc081ef2740 20 client.20928 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:55.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.813+0000 7fc081ef2740 1 --2- 172.21.15.103:0/3524414087 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b923ade490 0x55b923ae0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:55.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.813+0000 7fc081ef2740 4 client.20928 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}] 2024-09-19T07:04:55.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.813+0000 7fc081ef2740 1 -- 172.21.15.103:0/3524414087 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}) -- 0x55b9239bfbe0 con 0x55b923ade490 2024-09-19T07:04:55.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.814+0000 7fc07d8b5640 1 --2- 172.21.15.103:0/3524414087 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b923ade490 0x55b923ae0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:55.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.814+0000 7fc0617fa640 10 client.20928 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:55.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.814+0000 7fc07d8b5640 1 --2- 172.21.15.103:0/3524414087 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b923ade490 0x55b923ae0880 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbdd0f3becb5efd4 server_cookie=f5a208926b3d3ff1 in_seq=0 out_seq=0 2024-09-19T07:04:55.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.821+0000 7fc0617fa640 1 -- 172.21.15.103:0/3524414087 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b9239bfbe0 con 0x55b923ade490 2024-09-19T07:04:55.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.821+0000 7fc0617fa640 10 client.20928 handle_command_reply: tid=0 2024-09-19T07:04:55.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.821+0000 7fc081ef2740 1 client.20928 shutdown 2024-09-19T07:04:55.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.822+0000 7fc081ef2740 1 -- 172.21.15.103:0/3524414087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a62e40 msgr2=0x55b923a5d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:55.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.822+0000 7fc081ef2740 1 --2- 172.21.15.103:0/3524414087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a62e40 0x55b923a5d9f0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fc07001ec80 tx=0x7fc07000ffa0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.822+0000 7fc081ef2740 1 -- 172.21.15.103:0/3524414087 shutdown_connections 2024-09-19T07:04:55.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.822+0000 7fc081ef2740 1 --2- 172.21.15.103:0/3524414087 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55b923ade490 0x55b923ae0880 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.822+0000 7fc081ef2740 1 --2- 172.21.15.103:0/3524414087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b923a62e40 0x55b923a5d9f0 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.822+0000 7fc081ef2740 1 --2- 172.21.15.103:0/3524414087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b923a54050 0x55b923a5d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.822+0000 7fc0617fa640 0 client.20928 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:55.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.822+0000 7fc081ef2740 1 --2- 172.21.15.103:0/3524414087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b923a53730 0x55b923a5cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.822+0000 7fc081ef2740 1 -- 172.21.15.103:0/3524414087 >> 172.21.15.103:0/3524414087 conn(0x55b923a4fde0 msgr2=0x55b923a4e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:55.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.822+0000 7fc081ef2740 1 -- 172.21.15.103:0/3524414087 shutdown_connections 2024-09-19T07:04:55.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.822+0000 7fc081ef2740 1 -- 172.21.15.103:0/3524414087 wait complete. 2024-09-19T07:04:55.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.822+0000 7fc081ef2740 20 client.20928 trim_cache size 0 max 16384 2024-09-19T07:04:55.824 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:04:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T07:04:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T07:04:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 54, 2024-09-19T07:04:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:04:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "87781512": { 2024-09-19T07:04:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 54, 2024-09-19T07:04:55.824 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:04:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:04:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:04:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:04:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:04:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:04:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 149.4, 2024-09-19T07:04:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:04:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:04:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:04:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:04:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:04:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:04:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:04:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:04:55.825 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:04:55.826 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:04:55.826 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:04:55.826 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:04:55.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.822+0000 7fc07fb19640 1 -- 172.21.15.103:0/39529372 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0640a33b0 msgr2=0x7fc0640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:55.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.822+0000 7fc07fb19640 1 --2- 172.21.15.103:0/39529372 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0640a33b0 0x7fc0640a5870 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc070005fd0 tx=0x7fc07000ad30 comp rx=0 tx=0).stop 2024-09-19T07:04:55.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.822+0000 7fc07fb19640 1 -- 172.21.15.103:0/39529372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc078102570 msgr2=0x7fc078198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:55.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.822+0000 7fc07fb19640 1 --2- 172.21.15.103:0/39529372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc078102570 0x7fc078198a40 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fc068006010 tx=0x7fc068003c80 comp rx=0 tx=0).stop 2024-09-19T07:04:55.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.823+0000 7fc07fb19640 1 -- 172.21.15.103:0/39529372 shutdown_connections 2024-09-19T07:04:55.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.823+0000 7fc07fb19640 1 --2- 172.21.15.103:0/39529372 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc0640a33b0 0x7fc0640a5870 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.823+0000 7fc07fb19640 1 --2- 172.21.15.103:0/39529372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc078102ef0 0x7fc07819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.823+0000 7fc07fb19640 1 --2- 172.21.15.103:0/39529372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc078102570 0x7fc078198a40 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.823+0000 7fc07fb19640 1 --2- 172.21.15.103:0/39529372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc078100e20 0x7fc078198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:55.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.823+0000 7fc07fb19640 1 -- 172.21.15.103:0/39529372 >> 172.21.15.103:0/39529372 conn(0x7fc0780fc7d0 msgr2=0x7fc07806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:55.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.823+0000 7fc07fb19640 1 -- 172.21.15.103:0/39529372 shutdown_connections 2024-09-19T07:04:55.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:55.823+0000 7fc07fb19640 1 -- 172.21.15.103:0/39529372 wait complete. 2024-09-19T07:04:56.593 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '87781512', quiesce took 4.5 seconds. Will release after: 24.95 2024-09-19T07:04:56.642 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:57.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.240+0000 7fedf2871640 1 -- 172.21.15.103:0/3663832940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedec1082a0 msgr2=0x7fedec10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:57.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.240+0000 7fedf2871640 1 --2- 172.21.15.103:0/3663832940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedec1082a0 0x7fedec10ec40 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fede4009cd0 tx=0x7fede402a8e0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.240+0000 7fedf2871640 1 -- 172.21.15.103:0/3663832940 shutdown_connections 2024-09-19T07:04:57.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.240+0000 7fedf2871640 1 --2- 172.21.15.103:0/3663832940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedec1082a0 0x7fedec10ec40 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.240+0000 7fedf2871640 1 --2- 172.21.15.103:0/3663832940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedec1036e0 0x7fedec107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.240+0000 7fedf2871640 1 --2- 172.21.15.103:0/3663832940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedec102d30 0x7fedec103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.240+0000 7fedf2871640 1 -- 172.21.15.103:0/3663832940 >> 172.21.15.103:0/3663832940 conn(0x7fedec0fc7d0 msgr2=0x7fedec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:57.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.240+0000 7fedf2871640 1 -- 172.21.15.103:0/3663832940 shutdown_connections 2024-09-19T07:04:57.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.240+0000 7fedf2871640 1 -- 172.21.15.103:0/3663832940 wait complete. 2024-09-19T07:04:57.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.241+0000 7fedf2871640 1 Processor -- start 2024-09-19T07:04:57.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.241+0000 7fedf2871640 1 -- start start 2024-09-19T07:04:57.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.241+0000 7fedf2871640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedec102d30 0x7fedec100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.241+0000 7fedf2871640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedec1036e0 0x7fedec100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.241+0000 7fedf2871640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedec1082a0 0x7fedec101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.241+0000 7fedf2871640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1120a0 con 0x7fedec1036e0 2024-09-19T07:04:57.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.241+0000 7fedf2871640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fedec111f20 con 0x7fedec102d30 2024-09-19T07:04:57.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.241+0000 7fedf2871640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fedec112220 con 0x7fedec1082a0 2024-09-19T07:04:57.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.241+0000 7fedeb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedec1036e0 0x7fedec100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.241+0000 7fedeb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedec1036e0 0x7fedec100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53350/0 (socket says 172.21.15.103:53350) 2024-09-19T07:04:57.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.241+0000 7fedeb7fe640 1 -- 172.21.15.103:0/1783828502 learned_addr learned my addr 172.21.15.103:0/1783828502 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:57.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.241+0000 7fedf0e0e640 1 --2- 172.21.15.103:0/1783828502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedec1082a0 0x7fedec101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.242+0000 7fedebfff640 1 --2- 172.21.15.103:0/1783828502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedec102d30 0x7fedec100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.242+0000 7fedeb7fe640 1 -- 172.21.15.103:0/1783828502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedec1082a0 msgr2=0x7fedec101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.242+0000 7fedeb7fe640 1 --2- 172.21.15.103:0/1783828502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedec1082a0 0x7fedec101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.242+0000 7fedeb7fe640 1 -- 172.21.15.103:0/1783828502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedec102d30 msgr2=0x7fedec100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.242+0000 7fedeb7fe640 1 --2- 172.21.15.103:0/1783828502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedec102d30 0x7fedec100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.242+0000 7fedeb7fe640 1 -- 172.21.15.103:0/1783828502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedec1a6990 con 0x7fedec1036e0 2024-09-19T07:04:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.242+0000 7fedebfff640 1 --2- 172.21.15.103:0/1783828502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedec102d30 0x7fedec100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.242+0000 7fedeb7fe640 1 --2- 172.21.15.103:0/1783828502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedec1036e0 0x7fedec100ed0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fede000d740 tx=0x7fede000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.242+0000 7fede97fa640 1 -- 172.21.15.103:0/1783828502 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0017070 con 0x7fedec1036e0 2024-09-19T07:04:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.242+0000 7fede97fa640 1 -- 172.21.15.103:0/1783828502 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede000de50 con 0x7fedec1036e0 2024-09-19T07:04:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.242+0000 7fedf2871640 1 -- 172.21.15.103:0/1783828502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedec1a6c80 con 0x7fedec1036e0 2024-09-19T07:04:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.242+0000 7fede97fa640 1 -- 172.21.15.103:0/1783828502 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede000f6b0 con 0x7fedec1036e0 2024-09-19T07:04:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.242+0000 7fedf2871640 1 -- 172.21.15.103:0/1783828502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedec1a71c0 con 0x7fedec1036e0 2024-09-19T07:04:57.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.244+0000 7fede97fa640 1 -- 172.21.15.103:0/1783828502 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fede001a070 con 0x7fedec1036e0 2024-09-19T07:04:57.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.245+0000 7fedf4c4a740 1 Processor -- start 2024-09-19T07:04:57.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.245+0000 7fedf4c4a740 1 -- start start 2024-09-19T07:04:57.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.245+0000 7fedf4c4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47c4a0 0x561fdd47c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.245+0000 7fedf4c4a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47cdc0 0x561fdd487140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.245+0000 7fedf4c4a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd487680 0x561fdd489a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.245+0000 7fede97fa640 1 --2- 172.21.15.103:0/1783828502 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fedd40a33b0 0x7fedd40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.245+0000 7fedf4c4a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fdd3e22e0 con 0x561fdd47cdc0 2024-09-19T07:04:57.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.245+0000 7fedf4c4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561fdd389ce0 con 0x561fdd47c4a0 2024-09-19T07:04:57.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.245+0000 7fedf4c4a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561fdd3de2a0 con 0x561fdd487680 2024-09-19T07:04:57.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.245+0000 7fedf0e0e640 1 --2- 172.21.15.103:0/1783828502 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fedd40a33b0 0x7fedd40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.245+0000 7fedebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47cdc0 0x561fdd487140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.245+0000 7fede97fa640 1 -- 172.21.15.103:0/1783828502 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fede00cf6f0 con 0x7fedec1036e0 2024-09-19T07:04:57.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.245+0000 7fedebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47cdc0 0x561fdd487140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53352/0 (socket says 172.21.15.103:53352) 2024-09-19T07:04:57.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.245+0000 7fedebfff640 1 -- 172.21.15.103:0/3873324801 learned_addr learned my addr 172.21.15.103:0/3873324801 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:57.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.245+0000 7fedeb7fe640 1 --2- 172.21.15.103:0/3873324801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd487680 0x561fdd489a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.245+0000 7fedf0e0e640 1 --2- 172.21.15.103:0/3873324801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47c4a0 0x561fdd47c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.246+0000 7fedebfff640 1 -- 172.21.15.103:0/3873324801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd487680 msgr2=0x561fdd489a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:57.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.246+0000 7fedebfff640 1 --2- 172.21.15.103:0/3873324801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd487680 0x561fdd489a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.246+0000 7fedebfff640 1 -- 172.21.15.103:0/3873324801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47c4a0 msgr2=0x561fdd47c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:57.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.246+0000 7fedebfff640 1 --2- 172.21.15.103:0/3873324801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47c4a0 0x561fdd47c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.246+0000 7fedebfff640 1 -- 172.21.15.103:0/3873324801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fdd36e440 con 0x561fdd47cdc0 2024-09-19T07:04:57.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.246+0000 7fedf0e0e640 1 --2- 172.21.15.103:0/3873324801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47c4a0 0x561fdd47c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:57.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.246+0000 7fedf0e0e640 1 --2- 172.21.15.103:0/1783828502 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fedd40a33b0 0x7fedd40a5870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fede4009430 tx=0x7fede40316c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:57.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.246+0000 7fedebfff640 1 --2- 172.21.15.103:0/3873324801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47cdc0 0x561fdd487140 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7feddc00bc20 tx=0x7feddc00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09559a5f80e9e4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:57.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.246+0000 7fedd27fc640 1 -- 172.21.15.103:0/3873324801 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc017620 con 0x561fdd47cdc0 2024-09-19T07:04:57.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.246+0000 7fedd27fc640 1 -- 172.21.15.103:0/3873324801 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feddc017ca0 con 0x561fdd47cdc0 2024-09-19T07:04:57.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.246+0000 7fedd27fc640 1 -- 172.21.15.103:0/3873324801 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc00fcb0 con 0x561fdd47cdc0 2024-09-19T07:04:57.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.247+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3873324801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47cdc0 msgr2=0x561fdd487140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:57.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.247+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3873324801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47cdc0 0x561fdd487140 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7feddc00bc20 tx=0x7feddc00d710 comp rx=0 tx=0).stop 2024-09-19T07:04:57.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.247+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3873324801 shutdown_connections 2024-09-19T07:04:57.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.247+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3873324801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd487680 0x561fdd489a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.247+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3873324801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47cdc0 0x561fdd487140 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.247+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3873324801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47c4a0 0x561fdd47c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.247+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3873324801 >> 172.21.15.103:0/3873324801 conn(0x561fdd478a30 msgr2=0x561fdd478e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:57.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.247+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3873324801 shutdown_connections 2024-09-19T07:04:57.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.247+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3873324801 wait complete. 2024-09-19T07:04:57.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.247+0000 7fedf4c4a740 1 -- start start 2024-09-19T07:04:57.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.247+0000 7fedf4c4a740 1 Processor -- start 2024-09-19T07:04:57.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedf4c4a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd47c4a0 0x561fdd486f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedf4c4a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47cdc0 0x561fdd4841d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedf4c4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd487680 0x561fdd484710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedf4c4a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fdd48a250 con 0x561fdd47cdc0 2024-09-19T07:04:57.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedf4c4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561fdd39d050 con 0x561fdd487680 2024-09-19T07:04:57.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedf4c4a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561fdd48a3d0 con 0x561fdd47c4a0 2024-09-19T07:04:57.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedf4c4a740 10 client.0 _ll_register_callbacks cb 0x561fdd3dd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:57.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedeb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47cdc0 0x561fdd4841d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedeb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47cdc0 0x561fdd4841d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53358/0 (socket says 172.21.15.103:53358) 2024-09-19T07:04:57.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedeb7fe640 1 -- 172.21.15.103:0/3645871363 learned_addr learned my addr 172.21.15.103:0/3645871363 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:57.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedebfff640 1 --2- 172.21.15.103:0/3645871363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd47c4a0 0x561fdd486f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedf0e0e640 1 --2- 172.21.15.103:0/3645871363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd487680 0x561fdd484710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedeb7fe640 1 -- 172.21.15.103:0/3645871363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd47c4a0 msgr2=0x561fdd486f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedeb7fe640 1 --2- 172.21.15.103:0/3645871363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd47c4a0 0x561fdd486f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedeb7fe640 1 -- 172.21.15.103:0/3645871363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd487680 msgr2=0x561fdd484710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedeb7fe640 1 --2- 172.21.15.103:0/3645871363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd487680 0x561fdd484710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedeb7fe640 1 -- 172.21.15.103:0/3645871363 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fdd1533c0 con 0x561fdd47cdc0 2024-09-19T07:04:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedeb7fe640 1 --2- 172.21.15.103:0/3645871363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47cdc0 0x561fdd4841d0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fede0098380 tx=0x7fede00b4e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8abe08bd59a5dae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedcb7fe640 1 -- 172.21.15.103:0/3645871363 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0021070 con 0x561fdd47cdc0 2024-09-19T07:04:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedcb7fe640 1 -- 172.21.15.103:0/3645871363 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fede0098020 con 0x561fdd47cdc0 2024-09-19T07:04:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.248+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3645871363 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feddc017ca0 con 0x561fdd47cdc0 2024-09-19T07:04:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.249+0000 7fedcb7fe640 1 -- 172.21.15.103:0/3645871363 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fede009b9b0 con 0x561fdd47cdc0 2024-09-19T07:04:57.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.249+0000 7fedf4c4a740 10 client.20940 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:57.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.249+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3645871363 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561fdd495b70 con 0x561fdd47cdc0 2024-09-19T07:04:57.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.249+0000 7fedcb7fe640 1 -- 172.21.15.103:0/3645871363 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feddc017ca0 con 0x561fdd47cdc0 2024-09-19T07:04:57.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.249+0000 7fedcb7fe640 1 -- 172.21.15.103:0/3645871363 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fede00b6030 con 0x561fdd47cdc0 2024-09-19T07:04:57.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.249+0000 7fedf4c4a740 10 client.20940 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:57.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.249+0000 7fedf4c4a740 10 client.20940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:57.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.249+0000 7fedf4c4a740 20 client.20940 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:57.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.249+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3645871363 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561fdd496fd0 0x561fdd499400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.249+0000 7fedf4c4a740 4 client.20940 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:57.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.249+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3645871363 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fdd3ef850 con 0x561fdd496fd0 2024-09-19T07:04:57.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.249+0000 7fedebfff640 1 --2- 172.21.15.103:0/3645871363 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561fdd496fd0 0x561fdd499400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.250+0000 7fedcb7fe640 10 client.20940 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:57.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.250+0000 7fedebfff640 1 --2- 172.21.15.103:0/3645871363 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561fdd496fd0 0x561fdd499400 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8dfc457d25f15d server_cookie=326168a468b52407 in_seq=0 out_seq=0 2024-09-19T07:04:57.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.250+0000 7fedcb7fe640 1 -- 172.21.15.103:0/3645871363 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fdd3ef850 con 0x561fdd496fd0 2024-09-19T07:04:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.250+0000 7fedcb7fe640 10 client.20940 handle_command_reply: tid=0 2024-09-19T07:04:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.250+0000 7fedf4c4a740 1 client.20940 shutdown 2024-09-19T07:04:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.250+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3645871363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47cdc0 msgr2=0x561fdd4841d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.251+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3645871363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47cdc0 0x561fdd4841d0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fede0098380 tx=0x7fede00b4e80 comp rx=0 tx=0).stop 2024-09-19T07:04:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.251+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3645871363 shutdown_connections 2024-09-19T07:04:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.251+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3645871363 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561fdd496fd0 0x561fdd499400 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.251+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3645871363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd487680 0x561fdd484710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.251+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3645871363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47cdc0 0x561fdd4841d0 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.251+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3645871363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd47c4a0 0x561fdd486f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.251+0000 7fedcb7fe640 0 client.20940 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.251+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3645871363 >> 172.21.15.103:0/3645871363 conn(0x561fdd478a30 msgr2=0x561fdd47e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.251+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3645871363 shutdown_connections 2024-09-19T07:04:57.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.251+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3645871363 wait complete. 2024-09-19T07:04:57.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.251+0000 7fedf4c4a740 20 client.20940 trim_cache size 0 max 16384 2024-09-19T07:04:57.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.267+0000 7fedf4c4a740 1 Processor -- start 2024-09-19T07:04:57.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedf4c4a740 1 -- start start 2024-09-19T07:04:57.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedf4c4a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47c4a0 0x561fdd478e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedf4c4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47cdc0 0x561fdd486140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedf4c4a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd487680 0x561fdd49ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedf4c4a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1afa80 con 0x561fdd47c4a0 2024-09-19T07:04:57.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedf4c4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1b15c0 con 0x561fdd47cdc0 2024-09-19T07:04:57.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedf4c4a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1b0150 con 0x561fdd487680 2024-09-19T07:04:57.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47c4a0 0x561fdd478e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47c4a0 0x561fdd478e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53362/0 (socket says 172.21.15.103:53362) 2024-09-19T07:04:57.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedebfff640 1 -- 172.21.15.103:0/3552891393 learned_addr learned my addr 172.21.15.103:0/3552891393 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:57.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedeb7fe640 1 --2- 172.21.15.103:0/3552891393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47cdc0 0x561fdd486140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedf0e0e640 1 --2- 172.21.15.103:0/3552891393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd487680 0x561fdd49ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedebfff640 1 -- 172.21.15.103:0/3552891393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd487680 msgr2=0x561fdd49ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:57.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedebfff640 1 --2- 172.21.15.103:0/3552891393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd487680 0x561fdd49ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedebfff640 1 -- 172.21.15.103:0/3552891393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47cdc0 msgr2=0x561fdd486140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:57.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedebfff640 1 --2- 172.21.15.103:0/3552891393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47cdc0 0x561fdd486140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedebfff640 1 -- 172.21.15.103:0/3552891393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fdd483180 con 0x561fdd47c4a0 2024-09-19T07:04:57.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedf0e0e640 1 --2- 172.21.15.103:0/3552891393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd487680 0x561fdd49ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:57.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedeb7fe640 1 --2- 172.21.15.103:0/3552891393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47cdc0 0x561fdd486140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:57.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.268+0000 7fedebfff640 1 --2- 172.21.15.103:0/3552891393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47c4a0 0x561fdd478e10 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7feddc00f8e0 tx=0x7feddc00f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483e856f605bf16f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:57.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.269+0000 7fedd1ffb640 1 -- 172.21.15.103:0/3552891393 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc00f980 con 0x561fdd47c4a0 2024-09-19T07:04:57.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.269+0000 7fedd1ffb640 1 -- 172.21.15.103:0/3552891393 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feddc005040 con 0x561fdd47c4a0 2024-09-19T07:04:57.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.269+0000 7fedd1ffb640 1 -- 172.21.15.103:0/3552891393 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc0048c0 con 0x561fdd47c4a0 2024-09-19T07:04:57.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.269+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3552891393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47c4a0 msgr2=0x561fdd478e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:57.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.269+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3552891393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47c4a0 0x561fdd478e10 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7feddc00f8e0 tx=0x7feddc00f870 comp rx=0 tx=0).stop 2024-09-19T07:04:57.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.269+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3552891393 shutdown_connections 2024-09-19T07:04:57.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.269+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3552891393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd487680 0x561fdd49ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.269+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3552891393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47cdc0 0x561fdd486140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.269+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3552891393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd47c4a0 0x561fdd478e10 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.269+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3552891393 >> 172.21.15.103:0/3552891393 conn(0x561fdd478a30 msgr2=0x561fdd485a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:57.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.269+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3552891393 shutdown_connections 2024-09-19T07:04:57.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.269+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3552891393 wait complete. 2024-09-19T07:04:57.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.270+0000 7fedf4c4a740 1 -- start start 2024-09-19T07:04:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.270+0000 7fedf4c4a740 1 Processor -- start 2024-09-19T07:04:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.270+0000 7fedf4c4a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd47c4a0 0x561fdd48d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.270+0000 7fedf4c4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47cdc0 0x561fdd48d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.270+0000 7fedf4c4a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd487680 0x561fdd48dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.270+0000 7fedf4c4a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1b0ef0 con 0x561fdd487680 2024-09-19T07:04:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.270+0000 7fedf4c4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1b0820 con 0x561fdd47cdc0 2024-09-19T07:04:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.270+0000 7fedf4c4a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1ada80 con 0x561fdd47c4a0 2024-09-19T07:04:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.270+0000 7fedf4c4a740 10 client.0 _ll_register_callbacks cb 0x7fede009b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.270+0000 7fedf0e0e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd487680 0x561fdd48dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.270+0000 7fedf0e0e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd487680 0x561fdd48dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53366/0 (socket says 172.21.15.103:53366) 2024-09-19T07:04:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.270+0000 7fedf0e0e640 1 -- 172.21.15.103:0/3097719756 learned_addr learned my addr 172.21.15.103:0/3097719756 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.270+0000 7fedebfff640 1 --2- 172.21.15.103:0/3097719756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd47c4a0 0x561fdd48d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.270+0000 7fedeb7fe640 1 --2- 172.21.15.103:0/3097719756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47cdc0 0x561fdd48d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.270+0000 7fedf0e0e640 1 -- 172.21.15.103:0/3097719756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd47c4a0 msgr2=0x561fdd48d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:57.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.270+0000 7fedf0e0e640 1 --2- 172.21.15.103:0/3097719756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd47c4a0 0x561fdd48d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedf0e0e640 1 -- 172.21.15.103:0/3097719756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47cdc0 msgr2=0x561fdd48d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:57.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedf0e0e640 1 --2- 172.21.15.103:0/3097719756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47cdc0 0x561fdd48d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedf0e0e640 1 -- 172.21.15.103:0/3097719756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fdd500cb0 con 0x561fdd487680 2024-09-19T07:04:57.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedf0e0e640 1 --2- 172.21.15.103:0/3097719756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd487680 0x561fdd48dd50 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fede4009cd0 tx=0x7fede4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b752cf5e60d9c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:57.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:57.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedcb7fe640 1 -- 172.21.15.103:0/3097719756 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4057060 con 0x561fdd487680 2024-09-19T07:04:57.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedcb7fe640 1 -- 172.21.15.103:0/3097719756 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fede4045040 con 0x561fdd487680 2024-09-19T07:04:57.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3097719756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feddc005040 con 0x561fdd487680 2024-09-19T07:04:57.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedcb7fe640 1 -- 172.21.15.103:0/3097719756 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fede4004490 con 0x561fdd487680 2024-09-19T07:04:57.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedf4c4a740 10 client.20946 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:57.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3097719756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561fdd501e50 con 0x561fdd487680 2024-09-19T07:04:57.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedcb7fe640 1 -- 172.21.15.103:0/3097719756 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feddc005040 con 0x561fdd487680 2024-09-19T07:04:57.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedcb7fe640 1 -- 172.21.15.103:0/3097719756 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fede4051070 con 0x561fdd487680 2024-09-19T07:04:57.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedf4c4a740 10 client.20946 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:57.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedf4c4a740 10 client.20946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:57.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedf4c4a740 20 client.20946 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:57.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3097719756 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561fdd5025d0 0x561fdd5049f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:57.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedf4c4a740 4 client.20946 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:04:57.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.271+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3097719756 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561fdd3e9d60 con 0x561fdd5025d0 2024-09-19T07:04:57.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.272+0000 7fedebfff640 1 --2- 172.21.15.103:0/3097719756 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561fdd5025d0 0x561fdd5049f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:57.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.272+0000 7fedcb7fe640 10 client.20946 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:57.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.272+0000 7fedebfff640 1 --2- 172.21.15.103:0/3097719756 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561fdd5025d0 0x561fdd5049f0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d14c2b3d7a285f74 server_cookie=6b40c00ea300a5f3 in_seq=0 out_seq=0 2024-09-19T07:04:57.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.272+0000 7fedcb7fe640 1 -- 172.21.15.103:0/3097719756 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561fdd3e9d60 con 0x561fdd5025d0 2024-09-19T07:04:57.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.272+0000 7fedcb7fe640 10 client.20946 handle_command_reply: tid=0 2024-09-19T07:04:57.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.272+0000 7fedf4c4a740 1 client.20946 shutdown 2024-09-19T07:04:57.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.272+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3097719756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd487680 msgr2=0x561fdd48dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:57.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.272+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3097719756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd487680 0x561fdd48dd50 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fede4009cd0 tx=0x7fede4004a40 comp rx=0 tx=0).stop 2024-09-19T07:04:57.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3097719756 shutdown_connections 2024-09-19T07:04:57.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3097719756 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561fdd5025d0 0x561fdd5049f0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3097719756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdd487680 0x561fdd48dd50 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3097719756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fdd47cdc0 0x561fdd48d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedcb7fe640 0 client.20946 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:57.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf4c4a740 1 --2- 172.21.15.103:0/3097719756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd47c4a0 0x561fdd48d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3097719756 >> 172.21.15.103:0/3097719756 conn(0x561fdd478a30 msgr2=0x561fdd486250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:57.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3097719756 shutdown_connections 2024-09-19T07:04:57.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf4c4a740 1 -- 172.21.15.103:0/3097719756 wait complete. 2024-09-19T07:04:57.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf4c4a740 20 client.20946 trim_cache size 0 max 16384 2024-09-19T07:04:57.282 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:04:57.282 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:04:57.282 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:04:57.282 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:04:57.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf2871640 1 -- 172.21.15.103:0/1783828502 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fedd40a33b0 msgr2=0x7fedd40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:57.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf2871640 1 --2- 172.21.15.103:0/1783828502 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fedd40a33b0 0x7fedd40a5870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fede4009430 tx=0x7fede40316c0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf2871640 1 -- 172.21.15.103:0/1783828502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedec1036e0 msgr2=0x7fedec100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:57.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf2871640 1 --2- 172.21.15.103:0/1783828502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedec1036e0 0x7fedec100ed0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fede000d740 tx=0x7fede000dc00 comp rx=0 tx=0).stop 2024-09-19T07:04:57.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf2871640 1 -- 172.21.15.103:0/1783828502 shutdown_connections 2024-09-19T07:04:57.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf2871640 1 --2- 172.21.15.103:0/1783828502 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fedd40a33b0 0x7fedd40a5870 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf2871640 1 --2- 172.21.15.103:0/1783828502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedec1082a0 0x7fedec101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf2871640 1 --2- 172.21.15.103:0/1783828502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedec1036e0 0x7fedec100ed0 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf2871640 1 --2- 172.21.15.103:0/1783828502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedec102d30 0x7fedec100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:57.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf2871640 1 -- 172.21.15.103:0/1783828502 >> 172.21.15.103:0/1783828502 conn(0x7fedec0fc7d0 msgr2=0x7fedec106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:57.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf2871640 1 -- 172.21.15.103:0/1783828502 shutdown_connections 2024-09-19T07:04:57.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:57.273+0000 7fedf2871640 1 -- 172.21.15.103:0/1783828502 wait complete. 2024-09-19T07:04:57.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:57 smithi103 ceph-mon[31158]: pgmap v492: 97 pgs: 97 active+clean; 3.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-09-19T07:04:57.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:57 smithi174 ceph-mon[35637]: pgmap v492: 97 pgs: 97 active+clean; 3.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-09-19T07:04:57.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:57 smithi134 ceph-mon[35356]: pgmap v492: 97 pgs: 97 active+clean; 3.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-09-19T07:04:58.010 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:04:58.010 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:04:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.076+0000 7f6829f3b640 1 -- 172.21.15.103:0/2457324931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68241069c0 msgr2=0x7f6824106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.076+0000 7f6829f3b640 1 --2- 172.21.15.103:0/2457324931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68241069c0 0x7f6824106da0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f6814009930 tx=0x7f681402a420 comp rx=0 tx=0).stop 2024-09-19T07:04:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.076+0000 7f6829f3b640 1 -- 172.21.15.103:0/2457324931 shutdown_connections 2024-09-19T07:04:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.076+0000 7f6829f3b640 1 --2- 172.21.15.103:0/2457324931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6824101350 0x7f682410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.076+0000 7f6829f3b640 1 --2- 172.21.15.103:0/2457324931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68241009b0 0x7f6824100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.076+0000 7f6829f3b640 1 --2- 172.21.15.103:0/2457324931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68241069c0 0x7f6824106da0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.076+0000 7f6829f3b640 1 -- 172.21.15.103:0/2457324931 >> 172.21.15.103:0/2457324931 conn(0x7f6824077990 msgr2=0x7f68240fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.076+0000 7f6829f3b640 1 -- 172.21.15.103:0/2457324931 shutdown_connections 2024-09-19T07:04:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.076+0000 7f6829f3b640 1 -- 172.21.15.103:0/2457324931 wait complete. 2024-09-19T07:04:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.076+0000 7f6829f3b640 1 Processor -- start 2024-09-19T07:04:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.076+0000 7f6829f3b640 1 -- start start 2024-09-19T07:04:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f6829f3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68241009b0 0x7f6824199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f6829f3b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6824101350 0x7f6824199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f6829f3b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68241069c0 0x7f68241a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f6829f3b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6824110e50 con 0x7f68241009b0 2024-09-19T07:04:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f6829f3b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6824110cd0 con 0x7f6824101350 2024-09-19T07:04:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f6829f3b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6824110fd0 con 0x7f68241069c0 2024-09-19T07:04:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f68237fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68241009b0 0x7f6824199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f68237fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68241009b0 0x7f6824199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60002/0 (socket says 172.21.15.103:60002) 2024-09-19T07:04:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f68237fe640 1 -- 172.21.15.103:0/3634639406 learned_addr learned my addr 172.21.15.103:0/3634639406 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f6822ffd640 1 --2- 172.21.15.103:0/3634639406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6824101350 0x7f6824199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f6823fff640 1 --2- 172.21.15.103:0/3634639406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68241069c0 0x7f68241a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f68237fe640 1 -- 172.21.15.103:0/3634639406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68241069c0 msgr2=0x7f68241a41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f68237fe640 1 --2- 172.21.15.103:0/3634639406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68241069c0 0x7f68241a41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f68237fe640 1 -- 172.21.15.103:0/3634639406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6824101350 msgr2=0x7f6824199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f68237fe640 1 --2- 172.21.15.103:0/3634639406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6824101350 0x7f6824199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f68237fe640 1 -- 172.21.15.103:0/3634639406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68241a46e0 con 0x7f68241009b0 2024-09-19T07:04:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.077+0000 7f68237fe640 1 --2- 172.21.15.103:0/3634639406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68241009b0 0x7f6824199120 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f681402a940 tx=0x7f6814003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.078+0000 7f6820ff9640 1 -- 172.21.15.103:0/3634639406 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681403f070 con 0x7f68241009b0 2024-09-19T07:04:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.078+0000 7f6829f3b640 1 -- 172.21.15.103:0/3634639406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68241a48b0 con 0x7f68241009b0 2024-09-19T07:04:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.078+0000 7f6820ff9640 1 -- 172.21.15.103:0/3634639406 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6814031c90 con 0x7f68241009b0 2024-09-19T07:04:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.078+0000 7f6820ff9640 1 -- 172.21.15.103:0/3634639406 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814035030 con 0x7f68241009b0 2024-09-19T07:04:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.078+0000 7f6829f3b640 1 -- 172.21.15.103:0/3634639406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68241a4d10 con 0x7f68241009b0 2024-09-19T07:04:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.080+0000 7f682ccd7740 1 Processor -- start 2024-09-19T07:04:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.080+0000 7f682ccd7740 1 -- start start 2024-09-19T07:04:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.080+0000 7f682ccd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622814f19d0 0x5622814f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.080+0000 7f682ccd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x562281500b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.080+0000 7f682ccd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622815010c0 0x5622815034b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.080+0000 7f682ccd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5622814562f0 con 0x5622814f22f0 2024-09-19T07:04:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.080+0000 7f682ccd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5622813fe270 con 0x5622815010c0 2024-09-19T07:04:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.080+0000 7f682ccd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5622814522b0 con 0x5622814f19d0 2024-09-19T07:04:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.081+0000 7f6822ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622815010c0 0x5622815034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.081+0000 7f6822ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622815010c0 0x5622815034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56346/0 (socket says 172.21.15.103:56346) 2024-09-19T07:04:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.081+0000 7f6822ffd640 1 -- 172.21.15.103:0/1238397512 learned_addr learned my addr 172.21.15.103:0/1238397512 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.081+0000 7f6823fff640 1 --2- 172.21.15.103:0/1238397512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622814f19d0 0x5622814f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.081+0000 7f6820ff9640 1 -- 172.21.15.103:0/3634639406 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f6814042070 con 0x7f68241009b0 2024-09-19T07:04:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.081+0000 7f6822ffd640 1 -- 172.21.15.103:0/1238397512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622814f19d0 msgr2=0x5622814f1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.081+0000 7f6822ffd640 1 --2- 172.21.15.103:0/1238397512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622814f19d0 0x5622814f1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.081+0000 7f6822ffd640 1 -- 172.21.15.103:0/1238397512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 msgr2=0x562281500b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.081+0000 7f68237fe640 1 --2- 172.21.15.103:0/1238397512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x562281500b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.081+0000 7f6822ffd640 1 --2- 172.21.15.103:0/1238397512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x562281500b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.081+0000 7f6822ffd640 1 -- 172.21.15.103:0/1238397512 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622813fd340 con 0x5622815010c0 2024-09-19T07:04:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.081+0000 7f68237fe640 1 --2- 172.21.15.103:0/1238397512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x562281500b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.081+0000 7f6822ffd640 1 --2- 172.21.15.103:0/1238397512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622815010c0 0x5622815034b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f681800b740 tx=0x7f681800bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f70d8ddb59a6734 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.081+0000 7f6820ff9640 1 --2- 172.21.15.103:0/3634639406 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f680c0a3460 0x7f680c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.081+0000 7f6809ffb640 1 -- 172.21.15.103:0/1238397512 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818003dd0 con 0x5622815010c0 2024-09-19T07:04:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.082+0000 7f6809ffb640 1 -- 172.21.15.103:0/1238397512 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6818015070 con 0x5622815010c0 2024-09-19T07:04:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.082+0000 7f6809ffb640 1 -- 172.21.15.103:0/1238397512 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818011440 con 0x5622815010c0 2024-09-19T07:04:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.082+0000 7f6823fff640 1 --2- 172.21.15.103:0/3634639406 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f680c0a3460 0x7f680c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.082+0000 7f6820ff9640 1 -- 172.21.15.103:0/3634639406 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f68140f0140 con 0x7f68241009b0 2024-09-19T07:04:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.082+0000 7f682ccd7740 1 -- 172.21.15.103:0/1238397512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622815010c0 msgr2=0x5622815034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.082+0000 7f6823fff640 1 --2- 172.21.15.103:0/3634639406 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f680c0a3460 0x7f680c0a5920 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f681c007e50 tx=0x7f681c007600 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.082+0000 7f682ccd7740 1 --2- 172.21.15.103:0/1238397512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622815010c0 0x5622815034b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f681800b740 tx=0x7f681800bc00 comp rx=0 tx=0).stop 2024-09-19T07:04:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.082+0000 7f682ccd7740 1 -- 172.21.15.103:0/1238397512 shutdown_connections 2024-09-19T07:04:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.082+0000 7f682ccd7740 1 --2- 172.21.15.103:0/1238397512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622815010c0 0x5622815034b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.082+0000 7f682ccd7740 1 --2- 172.21.15.103:0/1238397512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x562281500b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.082+0000 7f682ccd7740 1 --2- 172.21.15.103:0/1238397512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622814f19d0 0x5622814f1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.082+0000 7f682ccd7740 1 -- 172.21.15.103:0/1238397512 >> 172.21.15.103:0/1238397512 conn(0x5622814ee060 msgr2=0x5622814ee440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.082+0000 7f682ccd7740 1 -- 172.21.15.103:0/1238397512 shutdown_connections 2024-09-19T07:04:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.082+0000 7f682ccd7740 1 -- 172.21.15.103:0/1238397512 wait complete. 2024-09-19T07:04:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.083+0000 7f682ccd7740 1 -- start start 2024-09-19T07:04:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.083+0000 7f682ccd7740 1 Processor -- start 2024-09-19T07:04:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.083+0000 7f682ccd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622814f19d0 0x5622814fe020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.083+0000 7f682ccd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x5622814fe560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.083+0000 7f682ccd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622815010c0 0x5622814feaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.083+0000 7f682ccd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562281503cf0 con 0x5622814f22f0 2024-09-19T07:04:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.083+0000 7f682ccd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5622813df290 con 0x5622814f19d0 2024-09-19T07:04:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.083+0000 7f682ccd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562281503e70 con 0x5622815010c0 2024-09-19T07:04:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.083+0000 7f682ccd7740 10 client.0 _ll_register_callbacks cb 0x562281410140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.083+0000 7f6822ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x5622814fe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.083+0000 7f6822ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x5622814fe560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60018/0 (socket says 172.21.15.103:60018) 2024-09-19T07:04:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.083+0000 7f6822ffd640 1 -- 172.21.15.103:0/1034685587 learned_addr learned my addr 172.21.15.103:0/1034685587 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.083+0000 7f68237fe640 1 --2- 172.21.15.103:0/1034685587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622814f19d0 0x5622814fe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.083+0000 7f6823fff640 1 --2- 172.21.15.103:0/1034685587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622815010c0 0x5622814feaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.084+0000 7f6822ffd640 1 -- 172.21.15.103:0/1034685587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622815010c0 msgr2=0x5622814feaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.084+0000 7f6822ffd640 1 --2- 172.21.15.103:0/1034685587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622815010c0 0x5622814feaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.084+0000 7f6822ffd640 1 -- 172.21.15.103:0/1034685587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622814f19d0 msgr2=0x5622814fe020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.084+0000 7f6822ffd640 1 --2- 172.21.15.103:0/1034685587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622814f19d0 0x5622814fe020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.084+0000 7f6822ffd640 1 -- 172.21.15.103:0/1034685587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562281169210 con 0x5622814f22f0 2024-09-19T07:04:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.084+0000 7f6822ffd640 1 --2- 172.21.15.103:0/1034685587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x5622814fe560 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f68180047e0 tx=0x7f6818003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798767ac6b330244 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.084+0000 7f6802ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.084+0000 7f6802ffd640 1 -- 172.21.15.103:0/1034685587 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818028060 con 0x5622814f22f0 2024-09-19T07:04:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.084+0000 7f6802ffd640 1 -- 172.21.15.103:0/1034685587 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f6818011650 con 0x5622814f22f0 2024-09-19T07:04:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.084+0000 7f682ccd7740 1 -- 172.21.15.103:0/1034685587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6818015070 con 0x5622814f22f0 2024-09-19T07:04:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.084+0000 7f6802ffd640 1 -- 172.21.15.103:0/1034685587 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f681800be50 con 0x5622814f22f0 2024-09-19T07:04:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.084+0000 7f682ccd7740 10 client.20955 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.084+0000 7f682ccd7740 1 -- 172.21.15.103:0/1034685587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562281235620 con 0x5622814f22f0 2024-09-19T07:04:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.084+0000 7f6802ffd640 1 -- 172.21.15.103:0/1034685587 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6818015070 con 0x5622814f22f0 2024-09-19T07:04:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.085+0000 7f6802ffd640 1 -- 172.21.15.103:0/1034685587 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6818019920 con 0x5622814f22f0 2024-09-19T07:04:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.085+0000 7f682ccd7740 10 client.20955 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.085+0000 7f682ccd7740 10 client.20955 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.085+0000 7f682ccd7740 20 client.20955 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.085+0000 7f682ccd7740 1 --2- 172.21.15.103:0/1034685587 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562281510f40 0x562281513330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.085+0000 7f682ccd7740 4 client.20955 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:04:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.085+0000 7f682ccd7740 1 -- 172.21.15.103:0/1034685587 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562281463a20 con 0x562281510f40 2024-09-19T07:04:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.085+0000 7f68237fe640 1 --2- 172.21.15.103:0/1034685587 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562281510f40 0x562281513330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.085+0000 7f6802ffd640 10 client.20955 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.085+0000 7f68237fe640 1 --2- 172.21.15.103:0/1034685587 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562281510f40 0x562281513330 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12a8657e909d66db server_cookie=681fb1dad12debb5 in_seq=0 out_seq=0 2024-09-19T07:04:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.086+0000 7f6802ffd640 1 -- 172.21.15.103:0/1034685587 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562281463a20 con 0x562281510f40 2024-09-19T07:04:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.086+0000 7f6802ffd640 10 client.20955 handle_command_reply: tid=0 2024-09-19T07:04:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.086+0000 7f682ccd7740 1 client.20955 shutdown 2024-09-19T07:04:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.086+0000 7f682ccd7740 1 -- 172.21.15.103:0/1034685587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 msgr2=0x5622814fe560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.086+0000 7f682ccd7740 1 --2- 172.21.15.103:0/1034685587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x5622814fe560 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f68180047e0 tx=0x7f6818003e00 comp rx=0 tx=0).stop 2024-09-19T07:04:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.086+0000 7f682ccd7740 1 -- 172.21.15.103:0/1034685587 shutdown_connections 2024-09-19T07:04:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.086+0000 7f682ccd7740 1 --2- 172.21.15.103:0/1034685587 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562281510f40 0x562281513330 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.086+0000 7f682ccd7740 1 --2- 172.21.15.103:0/1034685587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622815010c0 0x5622814feaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.086+0000 7f682ccd7740 1 --2- 172.21.15.103:0/1034685587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x5622814fe560 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.086+0000 7f682ccd7740 1 --2- 172.21.15.103:0/1034685587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622814f19d0 0x5622814fe020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.086+0000 7f6802ffd640 0 client.20955 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.086+0000 7f682ccd7740 1 -- 172.21.15.103:0/1034685587 >> 172.21.15.103:0/1034685587 conn(0x5622814ee060 msgr2=0x5622814ed1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.086+0000 7f682ccd7740 1 -- 172.21.15.103:0/1034685587 shutdown_connections 2024-09-19T07:04:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.086+0000 7f682ccd7740 1 -- 172.21.15.103:0/1034685587 wait complete. 2024-09-19T07:04:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.086+0000 7f682ccd7740 20 client.20955 trim_cache size 0 max 16384 2024-09-19T07:04:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.100+0000 7f682ccd7740 1 Processor -- start 2024-09-19T07:04:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.100+0000 7f682ccd7740 1 -- start start 2024-09-19T07:04:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f682ccd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622814f19d0 0x5622814fff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f682ccd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x562281518900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f682ccd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622815010c0 0x562281518e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f682ccd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f68241ab540 con 0x5622814f22f0 2024-09-19T07:04:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f682ccd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68241ad080 con 0x5622814f19d0 2024-09-19T07:04:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f682ccd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68241abc10 con 0x5622815010c0 2024-09-19T07:04:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f6822ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x562281518900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f6822ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x562281518900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60028/0 (socket says 172.21.15.103:60028) 2024-09-19T07:04:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f6822ffd640 1 -- 172.21.15.103:0/3248762433 learned_addr learned my addr 172.21.15.103:0/3248762433 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f6823fff640 1 --2- 172.21.15.103:0/3248762433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622815010c0 0x562281518e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f6822ffd640 1 -- 172.21.15.103:0/3248762433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622815010c0 msgr2=0x562281518e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f6822ffd640 1 --2- 172.21.15.103:0/3248762433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622815010c0 0x562281518e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f6822ffd640 1 -- 172.21.15.103:0/3248762433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622814f19d0 msgr2=0x5622814fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f68237fe640 1 --2- 172.21.15.103:0/3248762433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622814f19d0 0x5622814fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f6822ffd640 1 --2- 172.21.15.103:0/3248762433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622814f19d0 0x5622814fff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f6822ffd640 1 -- 172.21.15.103:0/3248762433 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622814fc550 con 0x5622814f22f0 2024-09-19T07:04:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f6822ffd640 1 --2- 172.21.15.103:0/3248762433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x562281518900 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f681800bc40 tx=0x7f6818005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cca277464a1bd03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f68097fa640 1 -- 172.21.15.103:0/3248762433 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818028040 con 0x5622814f22f0 2024-09-19T07:04:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f68097fa640 1 -- 172.21.15.103:0/3248762433 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6818015030 con 0x5622814f22f0 2024-09-19T07:04:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.101+0000 7f68097fa640 1 -- 172.21.15.103:0/3248762433 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818004eb0 con 0x5622814f22f0 2024-09-19T07:04:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.102+0000 7f682ccd7740 1 -- 172.21.15.103:0/3248762433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 msgr2=0x562281518900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.102+0000 7f682ccd7740 1 --2- 172.21.15.103:0/3248762433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x562281518900 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f681800bc40 tx=0x7f6818005c50 comp rx=0 tx=0).stop 2024-09-19T07:04:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.102+0000 7f682ccd7740 1 -- 172.21.15.103:0/3248762433 shutdown_connections 2024-09-19T07:04:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.102+0000 7f682ccd7740 1 --2- 172.21.15.103:0/3248762433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622815010c0 0x562281518e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.102+0000 7f682ccd7740 1 --2- 172.21.15.103:0/3248762433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622814f22f0 0x562281518900 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.102+0000 7f682ccd7740 1 --2- 172.21.15.103:0/3248762433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622814f19d0 0x5622814fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.102+0000 7f682ccd7740 1 -- 172.21.15.103:0/3248762433 >> 172.21.15.103:0/3248762433 conn(0x5622814ee060 msgr2=0x5622814ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.102+0000 7f682ccd7740 1 -- 172.21.15.103:0/3248762433 shutdown_connections 2024-09-19T07:04:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.102+0000 7f682ccd7740 1 -- 172.21.15.103:0/3248762433 wait complete. 2024-09-19T07:04:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f682ccd7740 1 -- start start 2024-09-19T07:04:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f682ccd7740 1 Processor -- start 2024-09-19T07:04:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f682ccd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622814f19d0 0x562281509980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f682ccd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622814f22f0 0x5622814fae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f682ccd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622815010c0 0x5622814fb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f682ccd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f68241ac9b0 con 0x5622815010c0 2024-09-19T07:04:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f682ccd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68241ac2e0 con 0x5622814f22f0 2024-09-19T07:04:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f682ccd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68241a9560 con 0x5622814f19d0 2024-09-19T07:04:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f682ccd7740 10 client.0 _ll_register_callbacks cb 0x7f681c0074a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:04:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f6823fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622815010c0 0x5622814fb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f6823fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622815010c0 0x5622814fb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60034/0 (socket says 172.21.15.103:60034) 2024-09-19T07:04:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f6823fff640 1 -- 172.21.15.103:0/3329152644 learned_addr learned my addr 172.21.15.103:0/3329152644 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f68237fe640 1 --2- 172.21.15.103:0/3329152644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622814f19d0 0x562281509980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f6822ffd640 1 --2- 172.21.15.103:0/3329152644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622814f22f0 0x5622814fae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f6823fff640 1 -- 172.21.15.103:0/3329152644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622814f19d0 msgr2=0x562281509980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f6823fff640 1 --2- 172.21.15.103:0/3329152644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622814f19d0 0x562281509980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.103+0000 7f6823fff640 1 -- 172.21.15.103:0/3329152644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622814f22f0 msgr2=0x5622814fae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:04:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f6823fff640 1 --2- 172.21.15.103:0/3329152644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622814f22f0 0x5622814fae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f6823fff640 1 -- 172.21.15.103:0/3329152644 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622814fbc30 con 0x5622815010c0 2024-09-19T07:04:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f6822ffd640 1 --2- 172.21.15.103:0/3329152644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622814f22f0 0x5622814fae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:04:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f6823fff640 1 --2- 172.21.15.103:0/3329152644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622815010c0 0x5622814fb430 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f681c01de90 tx=0x7f681c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ffc044455784a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f6802ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:04:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f6802ffd640 1 -- 172.21.15.103:0/3329152644 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c01cb50 con 0x5622815010c0 2024-09-19T07:04:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f682ccd7740 1 -- 172.21.15.103:0/3329152644 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6818015030 con 0x5622815010c0 2024-09-19T07:04:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f6802ffd640 1 -- 172.21.15.103:0/3329152644 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f681c01ccf0 con 0x5622815010c0 2024-09-19T07:04:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f6802ffd640 1 -- 172.21.15.103:0/3329152644 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f681c034050 con 0x5622815010c0 2024-09-19T07:04:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f682ccd7740 10 client.20961 fetch_fsmap learned FSMap version 27 2024-09-19T07:04:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f682ccd7740 1 -- 172.21.15.103:0/3329152644 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56228157ba20 con 0x5622815010c0 2024-09-19T07:04:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f6802ffd640 1 -- 172.21.15.103:0/3329152644 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6818015030 con 0x5622815010c0 2024-09-19T07:04:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f6802ffd640 1 -- 172.21.15.103:0/3329152644 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f681c00e3b0 con 0x5622815010c0 2024-09-19T07:04:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f682ccd7740 10 client.20961 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:04:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f682ccd7740 10 client.20961 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:04:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f682ccd7740 20 client.20961 populate_metadata read hostname 'smithi103' 2024-09-19T07:04:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f682ccd7740 1 --2- 172.21.15.103:0/3329152644 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56228157c1d0 0x56228157e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f682ccd7740 4 client.20961 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:04:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.104+0000 7f682ccd7740 1 -- 172.21.15.103:0/3329152644 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56228145dd70 con 0x56228157c1d0 2024-09-19T07:04:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.105+0000 7f68237fe640 1 --2- 172.21.15.103:0/3329152644 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56228157c1d0 0x56228157e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.105+0000 7f6802ffd640 10 client.20961 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.105+0000 7f68237fe640 1 --2- 172.21.15.103:0/3329152644 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56228157c1d0 0x56228157e5f0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=430d47322d4675d4 server_cookie=22066fa5a0fe9c8a in_seq=0 out_seq=0 2024-09-19T07:04:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.115+0000 7f6802ffd640 1 -- 172.21.15.103:0/3329152644 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56228145dd70 con 0x56228157c1d0 2024-09-19T07:04:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.115+0000 7f6802ffd640 10 client.20961 handle_command_reply: tid=0 2024-09-19T07:04:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.115+0000 7f682ccd7740 1 client.20961 shutdown 2024-09-19T07:04:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.115+0000 7f682ccd7740 1 -- 172.21.15.103:0/3329152644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622815010c0 msgr2=0x5622814fb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.115+0000 7f682ccd7740 1 --2- 172.21.15.103:0/3329152644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622815010c0 0x5622814fb430 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f681c01de90 tx=0x7f681c004a40 comp rx=0 tx=0).stop 2024-09-19T07:04:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.115+0000 7f682ccd7740 1 -- 172.21.15.103:0/3329152644 shutdown_connections 2024-09-19T07:04:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.115+0000 7f682ccd7740 1 --2- 172.21.15.103:0/3329152644 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56228157c1d0 0x56228157e5f0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.115+0000 7f682ccd7740 1 --2- 172.21.15.103:0/3329152644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622815010c0 0x5622814fb430 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.115+0000 7f6802ffd640 0 client.20961 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:04:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.115+0000 7f682ccd7740 1 --2- 172.21.15.103:0/3329152644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622814f22f0 0x5622814fae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.115+0000 7f682ccd7740 1 --2- 172.21.15.103:0/3329152644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622814f19d0 0x562281509980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.115+0000 7f682ccd7740 1 -- 172.21.15.103:0/3329152644 >> 172.21.15.103:0/3329152644 conn(0x5622814ee060 msgr2=0x5622814fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.115+0000 7f682ccd7740 1 -- 172.21.15.103:0/3329152644 shutdown_connections 2024-09-19T07:04:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.115+0000 7f682ccd7740 1 -- 172.21.15.103:0/3329152644 wait complete. 2024-09-19T07:04:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.116+0000 7f682ccd7740 20 client.20961 trim_cache size 0 max 16384 2024-09-19T07:04:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:04:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:04:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:04:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:04:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.116+0000 7f6829f3b640 1 -- 172.21.15.103:0/3634639406 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f680c0a3460 msgr2=0x7f680c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.116+0000 7f6829f3b640 1 --2- 172.21.15.103:0/3634639406 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f680c0a3460 0x7f680c0a5920 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f681c007e50 tx=0x7f681c007600 comp rx=0 tx=0).stop 2024-09-19T07:04:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.116+0000 7f6829f3b640 1 -- 172.21.15.103:0/3634639406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68241009b0 msgr2=0x7f6824199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.116+0000 7f6829f3b640 1 --2- 172.21.15.103:0/3634639406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68241009b0 0x7f6824199120 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f681402a940 tx=0x7f6814003b90 comp rx=0 tx=0).stop 2024-09-19T07:04:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.116+0000 7f6829f3b640 1 -- 172.21.15.103:0/3634639406 shutdown_connections 2024-09-19T07:04:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.116+0000 7f6829f3b640 1 --2- 172.21.15.103:0/3634639406 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f680c0a3460 0x7f680c0a5920 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.116+0000 7f6829f3b640 1 --2- 172.21.15.103:0/3634639406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68241069c0 0x7f68241a41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.116+0000 7f6829f3b640 1 --2- 172.21.15.103:0/3634639406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6824101350 0x7f6824199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.116+0000 7f6829f3b640 1 --2- 172.21.15.103:0/3634639406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68241009b0 0x7f6824199120 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.116+0000 7f6829f3b640 1 -- 172.21.15.103:0/3634639406 >> 172.21.15.103:0/3634639406 conn(0x7f6824077990 msgr2=0x7f6824103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:58.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.116+0000 7f6829f3b640 1 -- 172.21.15.103:0/3634639406 shutdown_connections 2024-09-19T07:04:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:04:58.116+0000 7f6829f3b640 1 -- 172.21.15.103:0/3634639406 wait complete. 2024-09-19T07:04:58.267 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:04:58.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.770+0000 7fafc858c640 1 -- 172.21.15.103:0/2508450586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafc01082a0 msgr2=0x7fafc010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:58.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.770+0000 7fafc858c640 1 --2- 172.21.15.103:0/2508450586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafc01082a0 0x7fafc010ec40 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009cd0 tx=0x7fafb002a8e0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.771+0000 7fafc858c640 1 -- 172.21.15.103:0/2508450586 shutdown_connections 2024-09-19T07:04:58.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.771+0000 7fafc858c640 1 --2- 172.21.15.103:0/2508450586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafc01082a0 0x7fafc010ec40 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.771+0000 7fafc858c640 1 --2- 172.21.15.103:0/2508450586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafc01036e0 0x7fafc0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.771+0000 7fafc858c640 1 --2- 172.21.15.103:0/2508450586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafc0102d30 0x7fafc0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.771+0000 7fafc858c640 1 -- 172.21.15.103:0/2508450586 >> 172.21.15.103:0/2508450586 conn(0x7fafc00fc7d0 msgr2=0x7fafc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:58.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.771+0000 7fafc858c640 1 -- 172.21.15.103:0/2508450586 shutdown_connections 2024-09-19T07:04:58.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.771+0000 7fafc858c640 1 -- 172.21.15.103:0/2508450586 wait complete. 2024-09-19T07:04:58.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.771+0000 7fafc858c640 1 Processor -- start 2024-09-19T07:04:58.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.771+0000 7fafc858c640 1 -- start start 2024-09-19T07:04:58.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.771+0000 7fafc858c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafc0102d30 0x7fafc01a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.771+0000 7fafc858c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafc01036e0 0x7fafc01a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.771+0000 7fafc858c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafc01082a0 0x7fafc019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.771+0000 7fafc858c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fafc0112100 con 0x7fafc01082a0 2024-09-19T07:04:58.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.771+0000 7fafc858c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fafc0111f80 con 0x7fafc01036e0 2024-09-19T07:04:58.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.771+0000 7fafc858c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fafc0112280 con 0x7fafc0102d30 2024-09-19T07:04:58.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafc6b29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafc01082a0 0x7fafc019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafc6b29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafc01082a0 0x7fafc019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60050/0 (socket says 172.21.15.103:60050) 2024-09-19T07:04:58.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafc6b29640 1 -- 172.21.15.103:0/1527328965 learned_addr learned my addr 172.21.15.103:0/1527328965 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:04:58.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafc5b27640 1 --2- 172.21.15.103:0/1527328965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafc01036e0 0x7fafc01a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafc6328640 1 --2- 172.21.15.103:0/1527328965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafc0102d30 0x7fafc01a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafc6b29640 1 -- 172.21.15.103:0/1527328965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafc0102d30 msgr2=0x7fafc01a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:58.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafc6b29640 1 --2- 172.21.15.103:0/1527328965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafc0102d30 0x7fafc01a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafc6b29640 1 -- 172.21.15.103:0/1527328965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafc01036e0 msgr2=0x7fafc01a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:58.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafc6b29640 1 --2- 172.21.15.103:0/1527328965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafc01036e0 0x7fafc01a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafc6b29640 1 -- 172.21.15.103:0/1527328965 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc019acf0 con 0x7fafc01082a0 2024-09-19T07:04:58.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafc5b27640 1 --2- 172.21.15.103:0/1527328965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafc01036e0 0x7fafc01a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:04:58.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafc6b29640 1 --2- 172.21.15.103:0/1527328965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafc01082a0 0x7fafc019a5c0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fafb0005cc0 tx=0x7fafb0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:58.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafbf7fe640 1 -- 172.21.15.103:0/1527328965 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0004040 con 0x7fafc01082a0 2024-09-19T07:04:58.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafbf7fe640 1 -- 172.21.15.103:0/1527328965 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafb00046c0 con 0x7fafc01082a0 2024-09-19T07:04:58.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafc858c640 1 -- 172.21.15.103:0/1527328965 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc019af80 con 0x7fafc01082a0 2024-09-19T07:04:58.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafbf7fe640 1 -- 172.21.15.103:0/1527328965 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0046480 con 0x7fafc01082a0 2024-09-19T07:04:58.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.772+0000 7fafc858c640 1 -- 172.21.15.103:0/1527328965 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc01a72d0 con 0x7fafc01082a0 2024-09-19T07:04:58.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.773+0000 7fafc858c640 1 -- 172.21.15.103:0/1527328965 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc0103110 con 0x7fafc01082a0 2024-09-19T07:04:58.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.775+0000 7fafbf7fe640 1 -- 172.21.15.103:0/1527328965 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fafb00305a0 con 0x7fafc01082a0 2024-09-19T07:04:58.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.776+0000 7fafbf7fe640 1 --2- 172.21.15.103:0/1527328965 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fafa80a3430 0x7fafa80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:04:58.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.776+0000 7fafc6328640 1 --2- 172.21.15.103:0/1527328965 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fafa80a3430 0x7fafa80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:04:58.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.776+0000 7fafbf7fe640 1 -- 172.21.15.103:0/1527328965 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fafb002e030 con 0x7fafc01082a0 2024-09-19T07:04:58.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.776+0000 7fafc6328640 1 --2- 172.21.15.103:0/1527328965 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fafa80a3430 0x7fafa80a58f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009890 tx=0x7fafb8008040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:04:58.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.779+0000 7fafbf7fe640 1 -- 172.21.15.103:0/1527328965 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fafb004b060 con 0x7fafc01082a0 2024-09-19T07:04:58.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.984+0000 7fafc858c640 1 -- 172.21.15.103:0/1527328965 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafc010c2c0 con 0x7fafc01082a0 2024-09-19T07:04:58.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.985+0000 7fafbf7fe640 1 -- 172.21.15.103:0/1527328965 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fafb00be0a0 con 0x7fafc01082a0 2024-09-19T07:04:58.986 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:04:58.986 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:04:58.986 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:04:58.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.987+0000 7fafc858c640 1 -- 172.21.15.103:0/1527328965 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fafa80a3430 msgr2=0x7fafa80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:58.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.987+0000 7fafc858c640 1 --2- 172.21.15.103:0/1527328965 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fafa80a3430 0x7fafa80a58f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009890 tx=0x7fafb8008040 comp rx=0 tx=0).stop 2024-09-19T07:04:58.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.987+0000 7fafc858c640 1 -- 172.21.15.103:0/1527328965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafc01082a0 msgr2=0x7fafc019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:04:58.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.987+0000 7fafc858c640 1 --2- 172.21.15.103:0/1527328965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafc01082a0 0x7fafc019a5c0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fafb0005cc0 tx=0x7fafb0003970 comp rx=0 tx=0).stop 2024-09-19T07:04:58.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.987+0000 7fafc858c640 1 -- 172.21.15.103:0/1527328965 shutdown_connections 2024-09-19T07:04:58.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.987+0000 7fafc858c640 1 --2- 172.21.15.103:0/1527328965 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fafa80a3430 0x7fafa80a58f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.987+0000 7fafc858c640 1 --2- 172.21.15.103:0/1527328965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafc01082a0 0x7fafc019a5c0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.987+0000 7fafc858c640 1 --2- 172.21.15.103:0/1527328965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafc01036e0 0x7fafc01a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.987+0000 7fafc858c640 1 --2- 172.21.15.103:0/1527328965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafc0102d30 0x7fafc01a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:04:58.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.987+0000 7fafc858c640 1 -- 172.21.15.103:0/1527328965 >> 172.21.15.103:0/1527328965 conn(0x7fafc00fc7d0 msgr2=0x7fafc0101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:04:58.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.987+0000 7fafc858c640 1 -- 172.21.15.103:0/1527328965 shutdown_connections 2024-09-19T07:04:58.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:04:58.987+0000 7fafc858c640 1 -- 172.21.15.103:0/1527328965 wait complete. 2024-09-19T07:04:59.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:59 smithi174 ceph-mon[35637]: pgmap v493: 97 pgs: 97 active+clean; 3.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-09-19T07:04:59.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:04:59 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1527328965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:59.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:59 smithi103 ceph-mon[31158]: pgmap v493: 97 pgs: 97 active+clean; 3.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-09-19T07:04:59.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:04:59 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1527328965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:59.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:59 smithi134 ceph-mon[35356]: pgmap v493: 97 pgs: 97 active+clean; 3.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-09-19T07:04:59.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:04:59 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1527328965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:04:59.708 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:04:59.951 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:00.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.451+0000 7f56d64a8640 1 -- 172.21.15.103:0/3180915626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56d0106f20 msgr2=0x7f56d010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:00.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.451+0000 7f56d64a8640 1 --2- 172.21.15.103:0/3180915626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56d0106f20 0x7f56d010eaa0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009ca0 tx=0x7f56c802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.451+0000 7f56d64a8640 1 -- 172.21.15.103:0/3180915626 shutdown_connections 2024-09-19T07:05:00.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.451+0000 7f56d64a8640 1 --2- 172.21.15.103:0/3180915626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56d0106f20 0x7f56d010eaa0 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.451+0000 7f56d64a8640 1 --2- 172.21.15.103:0/3180915626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56d01027a0 0x7f56d01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.451+0000 7f56d64a8640 1 --2- 172.21.15.103:0/3180915626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56d0101df0 0x7f56d01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.451+0000 7f56d64a8640 1 -- 172.21.15.103:0/3180915626 >> 172.21.15.103:0/3180915626 conn(0x7f56d00fd8b0 msgr2=0x7f56d00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:00.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.451+0000 7f56d64a8640 1 -- 172.21.15.103:0/3180915626 shutdown_connections 2024-09-19T07:05:00.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.451+0000 7f56d64a8640 1 -- 172.21.15.103:0/3180915626 wait complete. 2024-09-19T07:05:00.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.452+0000 7f56d64a8640 1 Processor -- start 2024-09-19T07:05:00.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.452+0000 7f56d64a8640 1 -- start start 2024-09-19T07:05:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.452+0000 7f56d64a8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56d0101df0 0x7f56d00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.452+0000 7f56d64a8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56d01027a0 0x7f56d0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.452+0000 7f56d64a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56d0106f20 0x7f56d0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.452+0000 7f56d64a8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0111f80 con 0x7f56d01027a0 2024-09-19T07:05:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.452+0000 7f56d64a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0111e00 con 0x7f56d0106f20 2024-09-19T07:05:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.452+0000 7f56d64a8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0112100 con 0x7f56d0101df0 2024-09-19T07:05:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.452+0000 7f56cf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56d01027a0 0x7f56d0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.452+0000 7f56cf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56d01027a0 0x7f56d0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60058/0 (socket says 172.21.15.103:60058) 2024-09-19T07:05:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.452+0000 7f56cf7fe640 1 -- 172.21.15.103:0/4134580077 learned_addr learned my addr 172.21.15.103:0/4134580077 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.452+0000 7f56cffff640 1 --2- 172.21.15.103:0/4134580077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56d0101df0 0x7f56d00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.452+0000 7f56d4a45640 1 --2- 172.21.15.103:0/4134580077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56d0106f20 0x7f56d0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.453+0000 7f56cf7fe640 1 -- 172.21.15.103:0/4134580077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56d0101df0 msgr2=0x7f56d00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.453+0000 7f56cf7fe640 1 --2- 172.21.15.103:0/4134580077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56d0101df0 0x7f56d00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.453+0000 7f56cf7fe640 1 -- 172.21.15.103:0/4134580077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56d0106f20 msgr2=0x7f56d0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:00.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.453+0000 7f56cf7fe640 1 --2- 172.21.15.103:0/4134580077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56d0106f20 0x7f56d0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.453+0000 7f56cf7fe640 1 -- 172.21.15.103:0/4134580077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d01a6820 con 0x7f56d01027a0 2024-09-19T07:05:00.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.453+0000 7f56d4a45640 1 --2- 172.21.15.103:0/4134580077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56d0106f20 0x7f56d0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:00.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.453+0000 7f56cf7fe640 1 --2- 172.21.15.103:0/4134580077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56d01027a0 0x7f56d0074a30 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f56c400d770 tx=0x7f56c400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:00.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.453+0000 7f56cd7fa640 1 -- 172.21.15.103:0/4134580077 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4017070 con 0x7f56d01027a0 2024-09-19T07:05:00.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.453+0000 7f56cd7fa640 1 -- 172.21.15.103:0/4134580077 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c4009820 con 0x7f56d01027a0 2024-09-19T07:05:00.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.453+0000 7f56d64a8640 1 -- 172.21.15.103:0/4134580077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d01a6b10 con 0x7f56d01027a0 2024-09-19T07:05:00.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.453+0000 7f56cd7fa640 1 -- 172.21.15.103:0/4134580077 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4004cd0 con 0x7f56d01027a0 2024-09-19T07:05:00.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.453+0000 7f56d64a8640 1 -- 172.21.15.103:0/4134580077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d01a7000 con 0x7f56d01027a0 2024-09-19T07:05:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.455+0000 7f56cd7fa640 1 -- 172.21.15.103:0/4134580077 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f56c40099d0 con 0x7f56d01027a0 2024-09-19T07:05:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.456+0000 7f56d8881740 1 Processor -- start 2024-09-19T07:05:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.456+0000 7f56d8881740 1 -- start start 2024-09-19T07:05:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.456+0000 7f56d8881740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d8ab40 0x564295d89090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.456+0000 7f56d8881740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d895d0 0x564295d89a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.456+0000 7f56d8881740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d96760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.456+0000 7f56d8881740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564295cef2f0 con 0x564295d94370 2024-09-19T07:05:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.456+0000 7f56d8881740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564295c96cf0 con 0x564295d8ab40 2024-09-19T07:05:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.456+0000 7f56d8881740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564295ceb2b0 con 0x564295d895d0 2024-09-19T07:05:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.456+0000 7f56cd7fa640 1 --2- 172.21.15.103:0/4134580077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f56b80a33b0 0x7f56b80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.456+0000 7f56cf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d96760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.456+0000 7f56cf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d96760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60064/0 (socket says 172.21.15.103:60064) 2024-09-19T07:05:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.456+0000 7f56cf7fe640 1 -- 172.21.15.103:0/753919790 learned_addr learned my addr 172.21.15.103:0/753919790 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.456+0000 7f56d4a45640 1 --2- 172.21.15.103:0/4134580077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f56b80a33b0 0x7f56b80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.456+0000 7f56cffff640 1 --2- 172.21.15.103:0/753919790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d895d0 0x564295d89a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.456+0000 7f56d4a45640 1 --2- 172.21.15.103:0/753919790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d8ab40 0x564295d89090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.456+0000 7f56cd7fa640 1 -- 172.21.15.103:0/4134580077 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f56c40cf740 con 0x7f56d01027a0 2024-09-19T07:05:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.457+0000 7f56cf7fe640 1 -- 172.21.15.103:0/753919790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d895d0 msgr2=0x564295d89a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.457+0000 7f56cf7fe640 1 --2- 172.21.15.103:0/753919790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d895d0 0x564295d89a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.457+0000 7f56cf7fe640 1 -- 172.21.15.103:0/753919790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d8ab40 msgr2=0x564295d89090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.457+0000 7f56cf7fe640 1 --2- 172.21.15.103:0/753919790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d8ab40 0x564295d89090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.457+0000 7f56cf7fe640 1 -- 172.21.15.103:0/753919790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564295c7b3a0 con 0x564295d94370 2024-09-19T07:05:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.457+0000 7f56cf7fe640 1 --2- 172.21.15.103:0/753919790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d96760 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f56c40ccf20 tx=0x7f56c409bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ed5fca70a4b24a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.457+0000 7f56d4a45640 1 --2- 172.21.15.103:0/4134580077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f56b80a33b0 0x7f56b80a5870 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009a60 tx=0x7f56c8031d50 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.457+0000 7f56b67fc640 1 -- 172.21.15.103:0/753919790 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4020030 con 0x564295d94370 2024-09-19T07:05:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.457+0000 7f56b67fc640 1 -- 172.21.15.103:0/753919790 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c40860f0 con 0x564295d94370 2024-09-19T07:05:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.457+0000 7f56b67fc640 1 -- 172.21.15.103:0/753919790 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4098070 con 0x564295d94370 2024-09-19T07:05:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.458+0000 7f56d8881740 1 -- 172.21.15.103:0/753919790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 msgr2=0x564295d96760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.458+0000 7f56d8881740 1 --2- 172.21.15.103:0/753919790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d96760 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f56c40ccf20 tx=0x7f56c409bac0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.458+0000 7f56d8881740 1 -- 172.21.15.103:0/753919790 shutdown_connections 2024-09-19T07:05:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.458+0000 7f56d8881740 1 --2- 172.21.15.103:0/753919790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d96760 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.458+0000 7f56d8881740 1 --2- 172.21.15.103:0/753919790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d895d0 0x564295d89a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.458+0000 7f56d8881740 1 --2- 172.21.15.103:0/753919790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d8ab40 0x564295d89090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.458+0000 7f56d8881740 1 -- 172.21.15.103:0/753919790 >> 172.21.15.103:0/753919790 conn(0x564295d52310 msgr2=0x564295d85c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.458+0000 7f56d8881740 1 -- 172.21.15.103:0/753919790 shutdown_connections 2024-09-19T07:05:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.458+0000 7f56d8881740 1 -- 172.21.15.103:0/753919790 wait complete. 2024-09-19T07:05:00.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.458+0000 7f56d8881740 1 -- start start 2024-09-19T07:05:00.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.458+0000 7f56d8881740 1 Processor -- start 2024-09-19T07:05:00.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56d8881740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d895d0 0x564295d93d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56d8881740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 0x564295d91010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56d8881740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d94370 0x564295d91600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56d8881740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564295d97010 con 0x564295d895d0 2024-09-19T07:05:00.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56d8881740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564295caa060 con 0x564295d94370 2024-09-19T07:05:00.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56d8881740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564295d97190 con 0x564295d8ab40 2024-09-19T07:05:00.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56d8881740 10 client.0 _ll_register_callbacks cb 0x5642959fd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:00.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d895d0 0x564295d93d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56cffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d895d0 0x564295d93d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60080/0 (socket says 172.21.15.103:60080) 2024-09-19T07:05:00.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56cffff640 1 -- 172.21.15.103:0/592938643 learned_addr learned my addr 172.21.15.103:0/592938643 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:00.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56cf7fe640 1 --2- 172.21.15.103:0/592938643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 0x564295d91010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56d4a45640 1 --2- 172.21.15.103:0/592938643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d94370 0x564295d91600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56cffff640 1 -- 172.21.15.103:0/592938643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 msgr2=0x564295d91010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56cffff640 1 --2- 172.21.15.103:0/592938643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 0x564295d91010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56cffff640 1 -- 172.21.15.103:0/592938643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d94370 msgr2=0x564295d91600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56cffff640 1 --2- 172.21.15.103:0/592938643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d94370 0x564295d91600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56cffff640 1 -- 172.21.15.103:0/592938643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564295a603c0 con 0x564295d895d0 2024-09-19T07:05:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56d4a45640 1 --2- 172.21.15.103:0/592938643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d94370 0x564295d91600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56cffff640 1 --2- 172.21.15.103:0/592938643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d895d0 0x564295d93d60 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f56c0010a50 tx=0x7f56c0010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ebb3b45b1914fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56af7fe640 1 -- 172.21.15.103:0/592938643 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c001e060 con 0x564295d895d0 2024-09-19T07:05:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56af7fe640 1 -- 172.21.15.103:0/592938643 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f56c000fb10 con 0x564295d895d0 2024-09-19T07:05:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.459+0000 7f56d8881740 1 -- 172.21.15.103:0/592938643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56c40860f0 con 0x564295d895d0 2024-09-19T07:05:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.460+0000 7f56af7fe640 1 -- 172.21.15.103:0/592938643 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f56c000fd10 con 0x564295d895d0 2024-09-19T07:05:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.460+0000 7f56d8881740 10 client.20979 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:00.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.460+0000 7f56d8881740 1 -- 172.21.15.103:0/592938643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564295da2b10 con 0x564295d895d0 2024-09-19T07:05:00.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.460+0000 7f56af7fe640 1 -- 172.21.15.103:0/592938643 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c40860f0 con 0x564295d895d0 2024-09-19T07:05:00.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.460+0000 7f56af7fe640 1 -- 172.21.15.103:0/592938643 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f56c000e560 con 0x564295d895d0 2024-09-19T07:05:00.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.460+0000 7f56d8881740 10 client.20979 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:00.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.460+0000 7f56d8881740 10 client.20979 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:05:00.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.460+0000 7f56d8881740 20 client.20979 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:00.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.460+0000 7f56d8881740 1 --2- 172.21.15.103:0/592938643 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x564295da3fd0 0x564295da63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.460+0000 7f56d8881740 4 client.20979 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:00.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.460+0000 7f56d8881740 1 -- 172.21.15.103:0/592938643 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564295cfc860 con 0x564295da3fd0 2024-09-19T07:05:00.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.460+0000 7f56cf7fe640 1 --2- 172.21.15.103:0/592938643 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x564295da3fd0 0x564295da63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.461+0000 7f56af7fe640 10 client.20979 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:05:00.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.461+0000 7f56cf7fe640 1 --2- 172.21.15.103:0/592938643 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x564295da3fd0 0x564295da63c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d4081020ea98f685 server_cookie=cbcd6578044f1e3a in_seq=0 out_seq=0 2024-09-19T07:05:00.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.462+0000 7f56af7fe640 1 -- 172.21.15.103:0/592938643 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564295cfc860 con 0x564295da3fd0 2024-09-19T07:05:00.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.462+0000 7f56af7fe640 10 client.20979 handle_command_reply: tid=0 2024-09-19T07:05:00.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.462+0000 7f56d8881740 1 client.20979 shutdown 2024-09-19T07:05:00.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.462+0000 7f56d8881740 1 -- 172.21.15.103:0/592938643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d895d0 msgr2=0x564295d93d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:00.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.462+0000 7f56d8881740 1 --2- 172.21.15.103:0/592938643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d895d0 0x564295d93d60 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f56c0010a50 tx=0x7f56c0010f10 comp rx=0 tx=0).stop 2024-09-19T07:05:00.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.462+0000 7f56d8881740 1 -- 172.21.15.103:0/592938643 shutdown_connections 2024-09-19T07:05:00.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.462+0000 7f56d8881740 1 --2- 172.21.15.103:0/592938643 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x564295da3fd0 0x564295da63c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.462+0000 7f56d8881740 1 --2- 172.21.15.103:0/592938643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d94370 0x564295d91600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.462+0000 7f56d8881740 1 --2- 172.21.15.103:0/592938643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 0x564295d91010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.462+0000 7f56d8881740 1 --2- 172.21.15.103:0/592938643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d895d0 0x564295d93d60 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.462+0000 7f56d8881740 1 -- 172.21.15.103:0/592938643 >> 172.21.15.103:0/592938643 conn(0x564295d52310 msgr2=0x564295d8b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:00.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.462+0000 7f56af7fe640 0 client.20979 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:05:00.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.462+0000 7f56d8881740 1 -- 172.21.15.103:0/592938643 shutdown_connections 2024-09-19T07:05:00.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.462+0000 7f56d8881740 1 -- 172.21.15.103:0/592938643 wait complete. 2024-09-19T07:05:00.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.462+0000 7f56d8881740 20 client.20979 trim_cache size 0 max 16384 2024-09-19T07:05:00.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56d8881740 1 Processor -- start 2024-09-19T07:05:00.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56d8881740 1 -- start start 2024-09-19T07:05:00.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56d8881740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d895d0 0x564295d85e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56d8881740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 0x564295da02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56d8881740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d8c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56d8881740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01afa50 con 0x564295d94370 2024-09-19T07:05:00.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56d8881740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01b1590 con 0x564295d895d0 2024-09-19T07:05:00.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56d8881740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01b0120 con 0x564295d8ab40 2024-09-19T07:05:00.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56d4a45640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d8c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56d4a45640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d8c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60082/0 (socket says 172.21.15.103:60082) 2024-09-19T07:05:00.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56d4a45640 1 -- 172.21.15.103:0/2128425415 learned_addr learned my addr 172.21.15.103:0/2128425415 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:00.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56cffff640 1 --2- 172.21.15.103:0/2128425415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d895d0 0x564295d85e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56cf7fe640 1 --2- 172.21.15.103:0/2128425415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 0x564295da02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56d4a45640 1 -- 172.21.15.103:0/2128425415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 msgr2=0x564295da02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:00.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56d4a45640 1 --2- 172.21.15.103:0/2128425415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 0x564295da02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56d4a45640 1 -- 172.21.15.103:0/2128425415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d895d0 msgr2=0x564295d85e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:00.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56d4a45640 1 --2- 172.21.15.103:0/2128425415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d895d0 0x564295d85e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56d4a45640 1 -- 172.21.15.103:0/2128425415 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564295d94090 con 0x564295d94370 2024-09-19T07:05:00.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56cffff640 1 --2- 172.21.15.103:0/2128425415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d895d0 0x564295d85e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:00.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.475+0000 7f56cf7fe640 1 --2- 172.21.15.103:0/2128425415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 0x564295da02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:00.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.476+0000 7f56d4a45640 1 --2- 172.21.15.103:0/2128425415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d8c120 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f56c8035000 tx=0x7f56c8005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ebeefa1d7daa69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:00.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.476+0000 7f56b5ffb640 1 -- 172.21.15.103:0/2128425415 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c802f610 con 0x564295d94370 2024-09-19T07:05:00.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.476+0000 7f56b5ffb640 1 -- 172.21.15.103:0/2128425415 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c8036040 con 0x564295d94370 2024-09-19T07:05:00.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.476+0000 7f56b5ffb640 1 -- 172.21.15.103:0/2128425415 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8030ce0 con 0x564295d94370 2024-09-19T07:05:00.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.476+0000 7f56d8881740 1 -- 172.21.15.103:0/2128425415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 msgr2=0x564295d8c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:00.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.476+0000 7f56d8881740 1 --2- 172.21.15.103:0/2128425415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d8c120 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f56c8035000 tx=0x7f56c8005c30 comp rx=0 tx=0).stop 2024-09-19T07:05:00.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.476+0000 7f56d8881740 1 -- 172.21.15.103:0/2128425415 shutdown_connections 2024-09-19T07:05:00.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.476+0000 7f56d8881740 1 --2- 172.21.15.103:0/2128425415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d8c120 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.476+0000 7f56d8881740 1 --2- 172.21.15.103:0/2128425415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 0x564295da02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.476+0000 7f56d8881740 1 --2- 172.21.15.103:0/2128425415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d895d0 0x564295d85e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.476+0000 7f56d8881740 1 -- 172.21.15.103:0/2128425415 >> 172.21.15.103:0/2128425415 conn(0x564295d52310 msgr2=0x564295d85700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:00.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.476+0000 7f56d8881740 1 -- 172.21.15.103:0/2128425415 shutdown_connections 2024-09-19T07:05:00.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.476+0000 7f56d8881740 1 -- 172.21.15.103:0/2128425415 wait complete. 2024-09-19T07:05:00.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.477+0000 7f56d8881740 1 -- start start 2024-09-19T07:05:00.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.477+0000 7f56d8881740 1 Processor -- start 2024-09-19T07:05:00.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.477+0000 7f56d8881740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d895d0 0x564295d9c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.477+0000 7f56d8881740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 0x564295d9c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.477+0000 7f56d8881740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d8d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.477+0000 7f56d8881740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01b0ec0 con 0x564295d94370 2024-09-19T07:05:00.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.477+0000 7f56d8881740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01b07f0 con 0x564295d895d0 2024-09-19T07:05:00.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.477+0000 7f56d8881740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56d01ada50 con 0x564295d8ab40 2024-09-19T07:05:00.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.477+0000 7f56d8881740 10 client.0 _ll_register_callbacks cb 0x7f56c000a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:00.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.477+0000 7f56d4a45640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d8d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.477+0000 7f56d4a45640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d8d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60088/0 (socket says 172.21.15.103:60088) 2024-09-19T07:05:00.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.477+0000 7f56d4a45640 1 -- 172.21.15.103:0/1912758958 learned_addr learned my addr 172.21.15.103:0/1912758958 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:00.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.477+0000 7f56cffff640 1 --2- 172.21.15.103:0/1912758958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d895d0 0x564295d9c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.478+0000 7f56cf7fe640 1 --2- 172.21.15.103:0/1912758958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 0x564295d9c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.478+0000 7f56d4a45640 1 -- 172.21.15.103:0/1912758958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 msgr2=0x564295d9c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:00.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.478+0000 7f56d4a45640 1 --2- 172.21.15.103:0/1912758958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 0x564295d9c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.478+0000 7f56d4a45640 1 -- 172.21.15.103:0/1912758958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d895d0 msgr2=0x564295d9c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:00.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.478+0000 7f56d4a45640 1 --2- 172.21.15.103:0/1912758958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d895d0 0x564295d9c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.478+0000 7f56d4a45640 1 -- 172.21.15.103:0/1912758958 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564295d8dbb0 con 0x564295d94370 2024-09-19T07:05:00.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.478+0000 7f56cf7fe640 1 --2- 172.21.15.103:0/1912758958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 0x564295d9c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:00.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.478+0000 7f56d4a45640 1 --2- 172.21.15.103:0/1912758958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d8d640 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009ca0 tx=0x7f56c8030b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395b20e2362e760 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:00.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.478+0000 7f56af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:00.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.478+0000 7f56af7fe640 1 -- 172.21.15.103:0/1912758958 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c802f810 con 0x564295d94370 2024-09-19T07:05:00.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.478+0000 7f56af7fe640 1 -- 172.21.15.103:0/1912758958 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f56c802f9b0 con 0x564295d94370 2024-09-19T07:05:00.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.478+0000 7f56d8881740 1 -- 172.21.15.103:0/1912758958 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56c8036040 con 0x564295d94370 2024-09-19T07:05:00.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.478+0000 7f56af7fe640 1 -- 172.21.15.103:0/1912758958 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f56c8066050 con 0x564295d94370 2024-09-19T07:05:00.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.478+0000 7f56d8881740 10 client.20985 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:00.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.478+0000 7f56d8881740 1 -- 172.21.15.103:0/1912758958 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564295e0e3c0 con 0x564295d94370 2024-09-19T07:05:00.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.478+0000 7f56af7fe640 1 -- 172.21.15.103:0/1912758958 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c8036040 con 0x564295d94370 2024-09-19T07:05:00.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.479+0000 7f56af7fe640 1 -- 172.21.15.103:0/1912758958 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f56c804a070 con 0x564295d94370 2024-09-19T07:05:00.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.479+0000 7f56d8881740 10 client.20985 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:00.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.479+0000 7f56d8881740 10 client.20985 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:05:00.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.479+0000 7f56d8881740 20 client.20985 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:00.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.479+0000 7f56d8881740 1 --2- 172.21.15.103:0/1912758958 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x564295e0ea30 0x564295e10e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:00.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.479+0000 7f56d8881740 4 client.20985 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:05:00.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.479+0000 7f56d8881740 1 -- 172.21.15.103:0/1912758958 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564295cf6d70 con 0x564295e0ea30 2024-09-19T07:05:00.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.479+0000 7f56cffff640 1 --2- 172.21.15.103:0/1912758958 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x564295e0ea30 0x564295e10e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:00.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.479+0000 7f56cffff640 1 --2- 172.21.15.103:0/1912758958 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x564295e0ea30 0x564295e10e50 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cc2a53b22d13a95d server_cookie=c7a481e9e40ba8ea in_seq=0 out_seq=0 2024-09-19T07:05:00.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.479+0000 7f56af7fe640 10 client.20985 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:05:00.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.479+0000 7f56af7fe640 1 -- 172.21.15.103:0/1912758958 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564295cf6d70 con 0x564295e0ea30 2024-09-19T07:05:00.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.479+0000 7f56af7fe640 10 client.20985 handle_command_reply: tid=0 2024-09-19T07:05:00.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.479+0000 7f56d8881740 1 client.20985 shutdown 2024-09-19T07:05:00.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d8881740 1 -- 172.21.15.103:0/1912758958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 msgr2=0x564295d8d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:00.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d8881740 1 --2- 172.21.15.103:0/1912758958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d8d640 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009ca0 tx=0x7f56c8030b90 comp rx=0 tx=0).stop 2024-09-19T07:05:00.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d8881740 1 -- 172.21.15.103:0/1912758958 shutdown_connections 2024-09-19T07:05:00.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d8881740 1 --2- 172.21.15.103:0/1912758958 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x564295e0ea30 0x564295e10e50 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d8881740 1 --2- 172.21.15.103:0/1912758958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564295d94370 0x564295d8d640 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d8881740 1 --2- 172.21.15.103:0/1912758958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564295d8ab40 0x564295d9c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d8881740 1 --2- 172.21.15.103:0/1912758958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564295d895d0 0x564295d9c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d8881740 1 -- 172.21.15.103:0/1912758958 >> 172.21.15.103:0/1912758958 conn(0x564295d52310 msgr2=0x564295d92870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:00.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56af7fe640 0 client.20985 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:05:00.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d8881740 1 -- 172.21.15.103:0/1912758958 shutdown_connections 2024-09-19T07:05:00.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d8881740 1 -- 172.21.15.103:0/1912758958 wait complete. 2024-09-19T07:05:00.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d8881740 20 client.20985 trim_cache size 0 max 16384 2024-09-19T07:05:00.486 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:05:00.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d64a8640 1 -- 172.21.15.103:0/4134580077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f56b80a33b0 msgr2=0x7f56b80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:00.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d64a8640 1 --2- 172.21.15.103:0/4134580077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f56b80a33b0 0x7f56b80a5870 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009a60 tx=0x7f56c8031d50 comp rx=0 tx=0).stop 2024-09-19T07:05:00.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d64a8640 1 -- 172.21.15.103:0/4134580077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56d01027a0 msgr2=0x7f56d0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:00.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d64a8640 1 --2- 172.21.15.103:0/4134580077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56d01027a0 0x7f56d0074a30 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f56c400d770 tx=0x7f56c400db70 comp rx=0 tx=0).stop 2024-09-19T07:05:00.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d64a8640 1 -- 172.21.15.103:0/4134580077 shutdown_connections 2024-09-19T07:05:00.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d64a8640 1 --2- 172.21.15.103:0/4134580077 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f56b80a33b0 0x7f56b80a5870 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d64a8640 1 --2- 172.21.15.103:0/4134580077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56d0106f20 0x7f56d0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d64a8640 1 --2- 172.21.15.103:0/4134580077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56d01027a0 0x7f56d0074a30 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d64a8640 1 --2- 172.21.15.103:0/4134580077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56d0101df0 0x7f56d00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:00.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.480+0000 7f56d64a8640 1 -- 172.21.15.103:0/4134580077 >> 172.21.15.103:0/4134580077 conn(0x7f56d00fd8b0 msgr2=0x7f56d00fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:00.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.481+0000 7f56d64a8640 1 -- 172.21.15.103:0/4134580077 shutdown_connections 2024-09-19T07:05:00.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:00.481+0000 7f56d64a8640 1 -- 172.21.15.103:0/4134580077 wait complete. 2024-09-19T07:05:01.204 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:05:01.458 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:01.482 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:01 smithi103 ceph-mon[31158]: pgmap v494: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-09-19T07:05:01.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:01 smithi174 ceph-mon[35637]: pgmap v494: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-09-19T07:05:01.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:01 smithi134 ceph-mon[35356]: pgmap v494: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-09-19T07:05:01.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.888+0000 7fa7da860640 1 -- 172.21.15.103:0/1428051105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7d410e3c0 msgr2=0x7fa7d4110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:01.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.888+0000 7fa7da860640 1 --2- 172.21.15.103:0/1428051105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7d410e3c0 0x7fa7d4110810 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009ca0 tx=0x7fa7cc02a950 comp rx=0 tx=0).stop 2024-09-19T07:05:01.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.888+0000 7fa7da860640 1 -- 172.21.15.103:0/1428051105 shutdown_connections 2024-09-19T07:05:01.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.888+0000 7fa7da860640 1 --2- 172.21.15.103:0/1428051105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7d410e3c0 0x7fa7d4110810 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.888+0000 7fa7da860640 1 --2- 172.21.15.103:0/1428051105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7d4074f40 0x7fa7d410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.888+0000 7fa7da860640 1 --2- 172.21.15.103:0/1428051105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7d4074620 0x7fa7d4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.888+0000 7fa7da860640 1 -- 172.21.15.103:0/1428051105 >> 172.21.15.103:0/1428051105 conn(0x7fa7d40fd880 msgr2=0x7fa7d40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:01.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.888+0000 7fa7da860640 1 -- 172.21.15.103:0/1428051105 shutdown_connections 2024-09-19T07:05:01.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.888+0000 7fa7da860640 1 -- 172.21.15.103:0/1428051105 wait complete. 2024-09-19T07:05:01.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.889+0000 7fa7da860640 1 Processor -- start 2024-09-19T07:05:01.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.889+0000 7fa7da860640 1 -- start start 2024-09-19T07:05:01.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.889+0000 7fa7da860640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7d4074620 0x7fa7d419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.889+0000 7fa7da860640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7d4074f40 0x7fa7d419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.889+0000 7fa7da860640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7d410e3c0 0x7fa7d419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.889+0000 7fa7da860640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d4114160 con 0x7fa7d410e3c0 2024-09-19T07:05:01.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.889+0000 7fa7da860640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d4113fe0 con 0x7fa7d4074f40 2024-09-19T07:05:01.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.889+0000 7fa7da860640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41142e0 con 0x7fa7d4074620 2024-09-19T07:05:01.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.890+0000 7fa7d8dfd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7d410e3c0 0x7fa7d419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.890+0000 7fa7d8dfd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7d410e3c0 0x7fa7d419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60112/0 (socket says 172.21.15.103:60112) 2024-09-19T07:05:01.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.890+0000 7fa7d8dfd640 1 -- 172.21.15.103:0/3512218695 learned_addr learned my addr 172.21.15.103:0/3512218695 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:01.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.890+0000 7fa7d37fe640 1 --2- 172.21.15.103:0/3512218695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7d4074f40 0x7fa7d419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.890+0000 7fa7d3fff640 1 --2- 172.21.15.103:0/3512218695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7d4074620 0x7fa7d419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.890+0000 7fa7d8dfd640 1 -- 172.21.15.103:0/3512218695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7d4074620 msgr2=0x7fa7d419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:01.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.890+0000 7fa7d8dfd640 1 --2- 172.21.15.103:0/3512218695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7d4074620 0x7fa7d419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.890+0000 7fa7d8dfd640 1 -- 172.21.15.103:0/3512218695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7d4074f40 msgr2=0x7fa7d419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:01.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.890+0000 7fa7d8dfd640 1 --2- 172.21.15.103:0/3512218695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7d4074f40 0x7fa7d419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.890+0000 7fa7d8dfd640 1 -- 172.21.15.103:0/3512218695 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d419f5b0 con 0x7fa7d410e3c0 2024-09-19T07:05:01.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.890+0000 7fa7d8dfd640 1 --2- 172.21.15.103:0/3512218695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7d410e3c0 0x7fa7d419eed0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc02ae70 tx=0x7fa7cc02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:01.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.890+0000 7fa7d17fa640 1 -- 172.21.15.103:0/3512218695 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc034040 con 0x7fa7d410e3c0 2024-09-19T07:05:01.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.890+0000 7fa7da860640 1 -- 172.21.15.103:0/3512218695 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d419f840 con 0x7fa7d410e3c0 2024-09-19T07:05:01.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.890+0000 7fa7d17fa640 1 -- 172.21.15.103:0/3512218695 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7cc030920 con 0x7fa7d410e3c0 2024-09-19T07:05:01.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.890+0000 7fa7d17fa640 1 -- 172.21.15.103:0/3512218695 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc045480 con 0x7fa7d410e3c0 2024-09-19T07:05:01.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.890+0000 7fa7da860640 1 -- 172.21.15.103:0/3512218695 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d41a7180 con 0x7fa7d410e3c0 2024-09-19T07:05:01.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.892+0000 7fa7d17fa640 1 -- 172.21.15.103:0/3512218695 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa7cc037020 con 0x7fa7d410e3c0 2024-09-19T07:05:01.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.893+0000 7fa7d17fa640 1 --2- 172.21.15.103:0/3512218695 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7bc0a3430 0x7fa7bc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.893+0000 7fa7d3fff640 1 --2- 172.21.15.103:0/3512218695 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7bc0a3430 0x7fa7bc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.893+0000 7fa7d17fa640 1 -- 172.21.15.103:0/3512218695 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fa7cc0f0ac0 con 0x7fa7d410e3c0 2024-09-19T07:05:01.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.894+0000 7fa7d3fff640 1 --2- 172.21.15.103:0/3512218695 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7bc0a3430 0x7fa7bc0a58f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4005fd0 tx=0x7fa7c400b390 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:01.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.894+0000 7fa7dcc39740 1 Processor -- start 2024-09-19T07:05:01.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.894+0000 7fa7dcc39740 1 -- start start 2024-09-19T07:05:01.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.894+0000 7fa7dcc39740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57fb40 0x5578ec57e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.894+0000 7fa7dcc39740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec57e5d0 0x5578ec57ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.894+0000 7fa7dcc39740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec589370 0x5578ec58b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.894+0000 7fa7dcc39740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5578ec4e42f0 con 0x5578ec589370 2024-09-19T07:05:01.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.894+0000 7fa7dcc39740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5578ec48bcf0 con 0x5578ec57fb40 2024-09-19T07:05:01.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.894+0000 7fa7dcc39740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578ec4e02b0 con 0x5578ec57e5d0 2024-09-19T07:05:01.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.894+0000 7fa7d8dfd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec589370 0x5578ec58b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.894+0000 7fa7d8dfd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec589370 0x5578ec58b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60114/0 (socket says 172.21.15.103:60114) 2024-09-19T07:05:01.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.894+0000 7fa7d8dfd640 1 -- 172.21.15.103:0/920065193 learned_addr learned my addr 172.21.15.103:0/920065193 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:01.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.894+0000 7fa7d3fff640 1 --2- 172.21.15.103:0/920065193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57fb40 0x5578ec57e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.895+0000 7fa7d37fe640 1 --2- 172.21.15.103:0/920065193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec57e5d0 0x5578ec57ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.895+0000 7fa7d8dfd640 1 -- 172.21.15.103:0/920065193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec57e5d0 msgr2=0x5578ec57ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:01.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.895+0000 7fa7d8dfd640 1 --2- 172.21.15.103:0/920065193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec57e5d0 0x5578ec57ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.895+0000 7fa7d8dfd640 1 -- 172.21.15.103:0/920065193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57fb40 msgr2=0x5578ec57e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:01.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.895+0000 7fa7d8dfd640 1 --2- 172.21.15.103:0/920065193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57fb40 0x5578ec57e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.895+0000 7fa7d8dfd640 1 -- 172.21.15.103:0/920065193 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578ec4703a0 con 0x5578ec589370 2024-09-19T07:05:01.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.895+0000 7fa7d3fff640 1 --2- 172.21.15.103:0/920065193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57fb40 0x5578ec57e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:01.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.895+0000 7fa7d37fe640 1 --2- 172.21.15.103:0/920065193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec57e5d0 0x5578ec57ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:01.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.895+0000 7fa7d8dfd640 1 --2- 172.21.15.103:0/920065193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec589370 0x5578ec58b760 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0c55d0 tx=0x7fa7cc02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2d9fca71f8341 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:01.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.895+0000 7fa7ba7fc640 1 -- 172.21.15.103:0/920065193 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc02c040 con 0x5578ec589370 2024-09-19T07:05:01.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.895+0000 7fa7ba7fc640 1 -- 172.21.15.103:0/920065193 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7cc0098d0 con 0x5578ec589370 2024-09-19T07:05:01.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.895+0000 7fa7ba7fc640 1 -- 172.21.15.103:0/920065193 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc004040 con 0x5578ec589370 2024-09-19T07:05:01.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.896+0000 7fa7dcc39740 1 -- 172.21.15.103:0/920065193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec589370 msgr2=0x5578ec58b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:01.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.896+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/920065193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec589370 0x5578ec58b760 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0c55d0 tx=0x7fa7cc02a950 comp rx=0 tx=0).stop 2024-09-19T07:05:01.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.896+0000 7fa7dcc39740 1 -- 172.21.15.103:0/920065193 shutdown_connections 2024-09-19T07:05:01.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.896+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/920065193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec589370 0x5578ec58b760 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.896+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/920065193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec57e5d0 0x5578ec57ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.896+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/920065193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57fb40 0x5578ec57e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.896+0000 7fa7dcc39740 1 -- 172.21.15.103:0/920065193 >> 172.21.15.103:0/920065193 conn(0x5578ec547310 msgr2=0x5578ec57ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:01.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.896+0000 7fa7dcc39740 1 -- 172.21.15.103:0/920065193 shutdown_connections 2024-09-19T07:05:01.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.896+0000 7fa7dcc39740 1 -- 172.21.15.103:0/920065193 wait complete. 2024-09-19T07:05:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.897+0000 7fa7dcc39740 1 -- start start 2024-09-19T07:05:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.897+0000 7fa7dcc39740 1 Processor -- start 2024-09-19T07:05:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.897+0000 7fa7dcc39740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57e5d0 0x5578ec588d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.897+0000 7fa7dcc39740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec57fb40 0x5578ec586000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.897+0000 7fa7dcc39740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec589370 0x5578ec5865d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.897+0000 7fa7dcc39740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5578ec58be30 con 0x5578ec589370 2024-09-19T07:05:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.897+0000 7fa7dcc39740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5578ec49f060 con 0x5578ec57e5d0 2024-09-19T07:05:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.897+0000 7fa7dcc39740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578ec58bfb0 con 0x5578ec57fb40 2024-09-19T07:05:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.897+0000 7fa7d8dfd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec589370 0x5578ec5865d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7dcc39740 10 client.0 _ll_register_callbacks cb 0x5578ec1f28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7d8dfd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec589370 0x5578ec5865d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60124/0 (socket says 172.21.15.103:60124) 2024-09-19T07:05:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7d8dfd640 1 -- 172.21.15.103:0/1807311746 learned_addr learned my addr 172.21.15.103:0/1807311746 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:01.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7d37fe640 1 --2- 172.21.15.103:0/1807311746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec57fb40 0x5578ec586000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7d3fff640 1 --2- 172.21.15.103:0/1807311746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57e5d0 0x5578ec588d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7d8dfd640 1 -- 172.21.15.103:0/1807311746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec57fb40 msgr2=0x5578ec586000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7d8dfd640 1 --2- 172.21.15.103:0/1807311746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec57fb40 0x5578ec586000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7d8dfd640 1 -- 172.21.15.103:0/1807311746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57e5d0 msgr2=0x5578ec588d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7d8dfd640 1 --2- 172.21.15.103:0/1807311746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57e5d0 0x5578ec588d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7d8dfd640 1 -- 172.21.15.103:0/1807311746 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578ec2553c0 con 0x5578ec589370 2024-09-19T07:05:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7d8dfd640 1 --2- 172.21.15.103:0/1807311746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec589370 0x5578ec5865d0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009430 tx=0x7fa7cc0bcf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bbd924666d2c52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7b37fe640 1 -- 172.21.15.103:0/1807311746 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc02c040 con 0x5578ec589370 2024-09-19T07:05:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7b37fe640 1 -- 172.21.15.103:0/1807311746 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fa7cc0bd300 con 0x5578ec589370 2024-09-19T07:05:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7dcc39740 1 -- 172.21.15.103:0/1807311746 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5578ec4f1860 con 0x5578ec589370 2024-09-19T07:05:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7b37fe640 1 -- 172.21.15.103:0/1807311746 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa7cc0c1240 con 0x5578ec589370 2024-09-19T07:05:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7dcc39740 10 client.20997 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:01.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.898+0000 7fa7dcc39740 1 -- 172.21.15.103:0/1807311746 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578ec58be30 con 0x5578ec589370 2024-09-19T07:05:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.899+0000 7fa7b37fe640 1 -- 172.21.15.103:0/1807311746 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5578ec4f1860 con 0x5578ec589370 2024-09-19T07:05:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.899+0000 7fa7b37fe640 1 -- 172.21.15.103:0/1807311746 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa7cc0bc2e0 con 0x5578ec589370 2024-09-19T07:05:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.899+0000 7fa7dcc39740 10 client.20997 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.899+0000 7fa7dcc39740 10 client.20997 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:05:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.899+0000 7fa7dcc39740 20 client.20997 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.899+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/1807311746 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5578ec598b70 0x5578ec59af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.899+0000 7fa7dcc39740 4 client.20997 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.899+0000 7fa7dcc39740 1 -- 172.21.15.103:0/1807311746 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578ec4ebd70 con 0x5578ec598b70 2024-09-19T07:05:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.899+0000 7fa7d3fff640 1 --2- 172.21.15.103:0/1807311746 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5578ec598b70 0x5578ec59af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.900+0000 7fa7d3fff640 1 --2- 172.21.15.103:0/1807311746 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5578ec598b70 0x5578ec59af80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e0584366040465be server_cookie=996a346e318a1c5a in_seq=0 out_seq=0 2024-09-19T07:05:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.900+0000 7fa7b37fe640 10 client.20997 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:05:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.900+0000 7fa7b37fe640 1 -- 172.21.15.103:0/1807311746 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578ec4ebd70 con 0x5578ec598b70 2024-09-19T07:05:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.900+0000 7fa7b37fe640 10 client.20997 handle_command_reply: tid=0 2024-09-19T07:05:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.900+0000 7fa7dcc39740 1 client.20997 shutdown 2024-09-19T07:05:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.900+0000 7fa7dcc39740 1 -- 172.21.15.103:0/1807311746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec589370 msgr2=0x5578ec5865d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.900+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/1807311746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec589370 0x5578ec5865d0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009430 tx=0x7fa7cc0bcf00 comp rx=0 tx=0).stop 2024-09-19T07:05:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.901+0000 7fa7dcc39740 1 -- 172.21.15.103:0/1807311746 shutdown_connections 2024-09-19T07:05:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.901+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/1807311746 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5578ec598b70 0x5578ec59af80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.901+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/1807311746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec589370 0x5578ec5865d0 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.901+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/1807311746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec57fb40 0x5578ec586000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.901+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/1807311746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57e5d0 0x5578ec588d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.901+0000 7fa7dcc39740 1 -- 172.21.15.103:0/1807311746 >> 172.21.15.103:0/1807311746 conn(0x5578ec547310 msgr2=0x5578ec580990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:01.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.901+0000 7fa7b37fe640 0 client.20997 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:05:01.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.901+0000 7fa7dcc39740 1 -- 172.21.15.103:0/1807311746 shutdown_connections 2024-09-19T07:05:01.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.901+0000 7fa7dcc39740 1 -- 172.21.15.103:0/1807311746 wait complete. 2024-09-19T07:05:01.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.901+0000 7fa7dcc39740 20 client.20997 trim_cache size 0 max 16384 2024-09-19T07:05:01.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.914+0000 7fa7dcc39740 1 Processor -- start 2024-09-19T07:05:01.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.914+0000 7fa7dcc39740 1 -- start start 2024-09-19T07:05:01.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.914+0000 7fa7dcc39740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57e5d0 0x5578ec595090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.914+0000 7fa7dcc39740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec57fb40 0x5578ec5848d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.914+0000 7fa7dcc39740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec589370 0x5578ec584e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.914+0000 7fa7dcc39740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41afa70 con 0x5578ec57fb40 2024-09-19T07:05:01.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.914+0000 7fa7dcc39740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41b15b0 con 0x5578ec57e5d0 2024-09-19T07:05:01.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.914+0000 7fa7dcc39740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41b0140 con 0x5578ec589370 2024-09-19T07:05:01.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.914+0000 7fa7d37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec57fb40 0x5578ec5848d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.914+0000 7fa7d37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec57fb40 0x5578ec5848d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60132/0 (socket says 172.21.15.103:60132) 2024-09-19T07:05:01.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.914+0000 7fa7d37fe640 1 -- 172.21.15.103:0/1737568154 learned_addr learned my addr 172.21.15.103:0/1737568154 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:01.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.914+0000 7fa7d8dfd640 1 --2- 172.21.15.103:0/1737568154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec589370 0x5578ec584e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.914+0000 7fa7d3fff640 1 --2- 172.21.15.103:0/1737568154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57e5d0 0x5578ec595090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.915+0000 7fa7d37fe640 1 -- 172.21.15.103:0/1737568154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec589370 msgr2=0x5578ec584e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:01.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.915+0000 7fa7d37fe640 1 --2- 172.21.15.103:0/1737568154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec589370 0x5578ec584e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.915+0000 7fa7d37fe640 1 -- 172.21.15.103:0/1737568154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57e5d0 msgr2=0x5578ec595090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:01.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.915+0000 7fa7d37fe640 1 --2- 172.21.15.103:0/1737568154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57e5d0 0x5578ec595090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.915+0000 7fa7d37fe640 1 -- 172.21.15.103:0/1737568154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578ec57aed0 con 0x5578ec57fb40 2024-09-19T07:05:01.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.915+0000 7fa7d3fff640 1 --2- 172.21.15.103:0/1737568154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57e5d0 0x5578ec595090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:01.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.915+0000 7fa7d37fe640 1 --2- 172.21.15.103:0/1737568154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec57fb40 0x5578ec5848d0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8004550 tx=0x7fa7c800ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5584154d4f162f38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:01.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.915+0000 7fa7b9ffb640 1 -- 172.21.15.103:0/1737568154 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8003710 con 0x5578ec57fb40 2024-09-19T07:05:01.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.915+0000 7fa7b9ffb640 1 -- 172.21.15.103:0/1737568154 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7c8003d90 con 0x5578ec57fb40 2024-09-19T07:05:01.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.915+0000 7fa7b9ffb640 1 -- 172.21.15.103:0/1737568154 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8010430 con 0x5578ec57fb40 2024-09-19T07:05:01.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.916+0000 7fa7dcc39740 1 -- 172.21.15.103:0/1737568154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec57fb40 msgr2=0x5578ec5848d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:01.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.916+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/1737568154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec57fb40 0x5578ec5848d0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8004550 tx=0x7fa7c800ba60 comp rx=0 tx=0).stop 2024-09-19T07:05:01.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.916+0000 7fa7dcc39740 1 -- 172.21.15.103:0/1737568154 shutdown_connections 2024-09-19T07:05:01.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.916+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/1737568154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec589370 0x5578ec584e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.916+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/1737568154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec57fb40 0x5578ec5848d0 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.916+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/1737568154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57e5d0 0x5578ec595090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.916+0000 7fa7dcc39740 1 -- 172.21.15.103:0/1737568154 >> 172.21.15.103:0/1737568154 conn(0x5578ec547310 msgr2=0x5578ec585fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:01.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.916+0000 7fa7dcc39740 1 -- 172.21.15.103:0/1737568154 shutdown_connections 2024-09-19T07:05:01.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.916+0000 7fa7dcc39740 1 -- 172.21.15.103:0/1737568154 wait complete. 2024-09-19T07:05:01.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.916+0000 7fa7dcc39740 1 -- start start 2024-09-19T07:05:01.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.917+0000 7fa7dcc39740 1 Processor -- start 2024-09-19T07:05:01.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.917+0000 7fa7dcc39740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57e5d0 0x5578ec58c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.917+0000 7fa7dcc39740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec57fb40 0x5578ec58c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.917+0000 7fa7dcc39740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec589370 0x5578ec58cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.917+0000 7fa7dcc39740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41b0ee0 con 0x5578ec57fb40 2024-09-19T07:05:01.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.917+0000 7fa7dcc39740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41b0810 con 0x5578ec57e5d0 2024-09-19T07:05:01.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.917+0000 7fa7dcc39740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41ada70 con 0x5578ec589370 2024-09-19T07:05:01.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.917+0000 7fa7dcc39740 10 client.0 _ll_register_callbacks cb 0x7fa7cc0bc8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:01.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.917+0000 7fa7d37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec57fb40 0x5578ec58c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.917+0000 7fa7d37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec57fb40 0x5578ec58c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60138/0 (socket says 172.21.15.103:60138) 2024-09-19T07:05:01.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.917+0000 7fa7d8dfd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec589370 0x5578ec58cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.917+0000 7fa7d37fe640 1 -- 172.21.15.103:0/2278421112 learned_addr learned my addr 172.21.15.103:0/2278421112 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:01.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.917+0000 7fa7d3fff640 1 --2- 172.21.15.103:0/2278421112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57e5d0 0x5578ec58c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.918+0000 7fa7d37fe640 1 -- 172.21.15.103:0/2278421112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec589370 msgr2=0x5578ec58cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:01.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.918+0000 7fa7d37fe640 1 --2- 172.21.15.103:0/2278421112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec589370 0x5578ec58cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.918+0000 7fa7d37fe640 1 -- 172.21.15.103:0/2278421112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57e5d0 msgr2=0x5578ec58c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:01.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.918+0000 7fa7d37fe640 1 --2- 172.21.15.103:0/2278421112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57e5d0 0x5578ec58c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.918+0000 7fa7d37fe640 1 -- 172.21.15.103:0/2278421112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578ec601ab0 con 0x5578ec57fb40 2024-09-19T07:05:01.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.918+0000 7fa7d8dfd640 1 --2- 172.21.15.103:0/2278421112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec589370 0x5578ec58cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:05:01.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.918+0000 7fa7d37fe640 1 --2- 172.21.15.103:0/2278421112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec57fb40 0x5578ec58c740 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800b620 tx=0x7fa7c80116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346c06d36eb8ecb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:01.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.918+0000 7fa7b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:01.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.918+0000 7fa7b37fe640 1 -- 172.21.15.103:0/2278421112 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8003920 con 0x5578ec57fb40 2024-09-19T07:05:01.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.918+0000 7fa7b37fe640 1 -- 172.21.15.103:0/2278421112 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fa7c8018040 con 0x5578ec57fb40 2024-09-19T07:05:01.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.918+0000 7fa7dcc39740 1 -- 172.21.15.103:0/2278421112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7c8003d90 con 0x5578ec57fb40 2024-09-19T07:05:01.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.918+0000 7fa7b37fe640 1 -- 172.21.15.103:0/2278421112 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa7c8033050 con 0x5578ec57fb40 2024-09-19T07:05:01.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.918+0000 7fa7dcc39740 10 client.21003 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:01.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.918+0000 7fa7dcc39740 1 -- 172.21.15.103:0/2278421112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578ec602d70 con 0x5578ec57fb40 2024-09-19T07:05:01.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.918+0000 7fa7b37fe640 1 -- 172.21.15.103:0/2278421112 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7c802a070 con 0x5578ec57fb40 2024-09-19T07:05:01.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.919+0000 7fa7b37fe640 1 -- 172.21.15.103:0/2278421112 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa7c801c420 con 0x5578ec57fb40 2024-09-19T07:05:01.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.919+0000 7fa7dcc39740 10 client.21003 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:01.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.919+0000 7fa7dcc39740 10 client.21003 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:05:01.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.919+0000 7fa7dcc39740 20 client.21003 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:01.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.919+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/2278421112 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5578ec603460 0x5578ec605880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:01.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.919+0000 7fa7dcc39740 4 client.21003 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:05:01.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.919+0000 7fa7dcc39740 1 -- 172.21.15.103:0/2278421112 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa7c8003d90 con 0x5578ec603460 2024-09-19T07:05:01.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.919+0000 7fa7d3fff640 1 --2- 172.21.15.103:0/2278421112 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5578ec603460 0x5578ec605880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:01.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.919+0000 7fa7b37fe640 10 client.21003 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:05:01.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.919+0000 7fa7d3fff640 1 --2- 172.21.15.103:0/2278421112 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5578ec603460 0x5578ec605880 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f602894c6901a418 server_cookie=17acd8f94afebe8f in_seq=0 out_seq=0 2024-09-19T07:05:01.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.920+0000 7fa7b37fe640 1 -- 172.21.15.103:0/2278421112 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa7c8003d90 con 0x5578ec603460 2024-09-19T07:05:01.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.920+0000 7fa7b37fe640 10 client.21003 handle_command_reply: tid=0 2024-09-19T07:05:01.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.920+0000 7fa7dcc39740 1 client.21003 shutdown 2024-09-19T07:05:01.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.920+0000 7fa7dcc39740 1 -- 172.21.15.103:0/2278421112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec57fb40 msgr2=0x5578ec58c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:01.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.920+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/2278421112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec57fb40 0x5578ec58c740 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800b620 tx=0x7fa7c80116f0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.920+0000 7fa7dcc39740 1 -- 172.21.15.103:0/2278421112 shutdown_connections 2024-09-19T07:05:01.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.920+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/2278421112 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5578ec603460 0x5578ec605880 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.920+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/2278421112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578ec589370 0x5578ec58cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.920+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/2278421112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578ec57fb40 0x5578ec58c740 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.920+0000 7fa7dcc39740 1 --2- 172.21.15.103:0/2278421112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578ec57e5d0 0x5578ec58c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.920+0000 7fa7dcc39740 1 -- 172.21.15.103:0/2278421112 >> 172.21.15.103:0/2278421112 conn(0x5578ec547310 msgr2=0x5578ec584890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:01.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.920+0000 7fa7b37fe640 0 client.21003 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:05:01.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.920+0000 7fa7dcc39740 1 -- 172.21.15.103:0/2278421112 shutdown_connections 2024-09-19T07:05:01.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.920+0000 7fa7dcc39740 1 -- 172.21.15.103:0/2278421112 wait complete. 2024-09-19T07:05:01.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.920+0000 7fa7dcc39740 20 client.21003 trim_cache size 0 max 16384 2024-09-19T07:05:01.926 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:05:01.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.921+0000 7fa7da860640 1 -- 172.21.15.103:0/3512218695 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7bc0a3430 msgr2=0x7fa7bc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:01.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.921+0000 7fa7da860640 1 --2- 172.21.15.103:0/3512218695 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7bc0a3430 0x7fa7bc0a58f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4005fd0 tx=0x7fa7c400b390 comp rx=0 tx=0).stop 2024-09-19T07:05:01.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.921+0000 7fa7da860640 1 -- 172.21.15.103:0/3512218695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7d410e3c0 msgr2=0x7fa7d419eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:01.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.921+0000 7fa7da860640 1 --2- 172.21.15.103:0/3512218695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7d410e3c0 0x7fa7d419eed0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc02ae70 tx=0x7fa7cc02f5b0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.921+0000 7fa7da860640 1 -- 172.21.15.103:0/3512218695 shutdown_connections 2024-09-19T07:05:01.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.921+0000 7fa7da860640 1 --2- 172.21.15.103:0/3512218695 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa7bc0a3430 0x7fa7bc0a58f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.921+0000 7fa7da860640 1 --2- 172.21.15.103:0/3512218695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7d410e3c0 0x7fa7d419eed0 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.921+0000 7fa7da860640 1 --2- 172.21.15.103:0/3512218695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7d4074f40 0x7fa7d419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.921+0000 7fa7da860640 1 --2- 172.21.15.103:0/3512218695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7d4074620 0x7fa7d419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:01.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.921+0000 7fa7da860640 1 -- 172.21.15.103:0/3512218695 >> 172.21.15.103:0/3512218695 conn(0x7fa7d40fd880 msgr2=0x7fa7d40fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:01.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.921+0000 7fa7da860640 1 -- 172.21.15.103:0/3512218695 shutdown_connections 2024-09-19T07:05:01.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:01.921+0000 7fa7da860640 1 -- 172.21.15.103:0/3512218695 wait complete. 2024-09-19T07:05:02.728 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:05:02.977 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:03.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:03 smithi103 ceph-mon[31158]: pgmap v495: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-09-19T07:05:03.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.387+0000 7fdbaf6fd640 1 -- 172.21.15.103:0/2988624011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba8102d30 msgr2=0x7fdba8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:03.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.387+0000 7fdbaf6fd640 1 --2- 172.21.15.103:0/2988624011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba8102d30 0x7fdba8103110 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009cd0 tx=0x7fdb9802a8e0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.387+0000 7fdbaf6fd640 1 -- 172.21.15.103:0/2988624011 shutdown_connections 2024-09-19T07:05:03.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.387+0000 7fdbaf6fd640 1 --2- 172.21.15.103:0/2988624011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba81082a0 0x7fdba810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.387+0000 7fdbaf6fd640 1 --2- 172.21.15.103:0/2988624011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdba81036e0 0x7fdba8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.387+0000 7fdbaf6fd640 1 --2- 172.21.15.103:0/2988624011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba8102d30 0x7fdba8103110 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.387+0000 7fdbaf6fd640 1 -- 172.21.15.103:0/2988624011 >> 172.21.15.103:0/2988624011 conn(0x7fdba80fc7d0 msgr2=0x7fdba80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:03.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.387+0000 7fdbaf6fd640 1 -- 172.21.15.103:0/2988624011 shutdown_connections 2024-09-19T07:05:03.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.387+0000 7fdbaf6fd640 1 -- 172.21.15.103:0/2988624011 wait complete. 2024-09-19T07:05:03.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.387+0000 7fdbaf6fd640 1 Processor -- start 2024-09-19T07:05:03.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.387+0000 7fdbaf6fd640 1 -- start start 2024-09-19T07:05:03.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbaf6fd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdba8102d30 0x7fdba8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbaf6fd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba81036e0 0x7fdba8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbaf6fd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba81082a0 0x7fdba8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbaf6fd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdba81120a0 con 0x7fdba81036e0 2024-09-19T07:05:03.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbaf6fd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdba8111f20 con 0x7fdba8102d30 2024-09-19T07:05:03.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbaf6fd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdba8112220 con 0x7fdba81082a0 2024-09-19T07:05:03.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbacc98640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba81036e0 0x7fdba8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbacc98640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba81036e0 0x7fdba8100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60160/0 (socket says 172.21.15.103:60160) 2024-09-19T07:05:03.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbacc98640 1 -- 172.21.15.103:0/3043816911 learned_addr learned my addr 172.21.15.103:0/3043816911 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:03.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbadc9a640 1 --2- 172.21.15.103:0/3043816911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba81082a0 0x7fdba8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbad499640 1 --2- 172.21.15.103:0/3043816911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdba8102d30 0x7fdba8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbacc98640 1 -- 172.21.15.103:0/3043816911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba81082a0 msgr2=0x7fdba8101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:03.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbacc98640 1 --2- 172.21.15.103:0/3043816911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba81082a0 0x7fdba8101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbacc98640 1 -- 172.21.15.103:0/3043816911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdba8102d30 msgr2=0x7fdba8100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:03.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbacc98640 1 --2- 172.21.15.103:0/3043816911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdba8102d30 0x7fdba8100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbacc98640 1 -- 172.21.15.103:0/3043816911 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba81a69e0 con 0x7fdba81036e0 2024-09-19T07:05:03.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbad499640 1 --2- 172.21.15.103:0/3043816911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdba8102d30 0x7fdba8100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:03.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbacc98640 1 --2- 172.21.15.103:0/3043816911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba81036e0 0x7fdba8100ed0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c00d740 tx=0x7fdb9c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:03.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdba67fc640 1 -- 172.21.15.103:0/3043816911 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c017070 con 0x7fdba81036e0 2024-09-19T07:05:03.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdba67fc640 1 -- 172.21.15.103:0/3043816911 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb9c00de50 con 0x7fdba81036e0 2024-09-19T07:05:03.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdba67fc640 1 -- 172.21.15.103:0/3043816911 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c00f6b0 con 0x7fdba81036e0 2024-09-19T07:05:03.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbaf6fd640 1 -- 172.21.15.103:0/3043816911 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba81a6cd0 con 0x7fdba81036e0 2024-09-19T07:05:03.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.388+0000 7fdbaf6fd640 1 -- 172.21.15.103:0/3043816911 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba81a7200 con 0x7fdba81036e0 2024-09-19T07:05:03.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.391+0000 7fdba67fc640 1 -- 172.21.15.103:0/3043816911 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fdb9c01a070 con 0x7fdba81036e0 2024-09-19T07:05:03.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.392+0000 7fdbb1ad6740 1 Processor -- start 2024-09-19T07:05:03.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.392+0000 7fdbb1ad6740 1 -- start start 2024-09-19T07:05:03.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.392+0000 7fdbb1ad6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72019b40 0x556e72018090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.392+0000 7fdbb1ad6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e720185d0 0x556e72018a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.392+0000 7fdbb1ad6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e72023370 0x556e72025760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.392+0000 7fdbb1ad6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556e71f7e2f0 con 0x556e72019b40 2024-09-19T07:05:03.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.392+0000 7fdbb1ad6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556e71f25cf0 con 0x556e720185d0 2024-09-19T07:05:03.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.392+0000 7fdbb1ad6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556e71f7a2b0 con 0x556e72023370 2024-09-19T07:05:03.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.392+0000 7fdba67fc640 1 --2- 172.21.15.103:0/3043816911 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdb900a3430 0x7fdb900a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.392+0000 7fdbadc9a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72019b40 0x556e72018090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.392+0000 7fdbadc9a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72019b40 0x556e72018090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60174/0 (socket says 172.21.15.103:60174) 2024-09-19T07:05:03.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.392+0000 7fdbadc9a640 1 -- 172.21.15.103:0/1625789454 learned_addr learned my addr 172.21.15.103:0/1625789454 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:03.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.392+0000 7fdbacc98640 1 --2- 172.21.15.103:0/1625789454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e72023370 0x556e72025760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.392+0000 7fdbadc9a640 1 --2- 172.21.15.103:0/3043816911 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdb900a3430 0x7fdb900a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.393+0000 7fdbad499640 1 --2- 172.21.15.103:0/1625789454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e720185d0 0x556e72018a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.393+0000 7fdbadc9a640 1 -- 172.21.15.103:0/1625789454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e72023370 msgr2=0x556e72025760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:03.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.393+0000 7fdbadc9a640 1 --2- 172.21.15.103:0/1625789454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e72023370 0x556e72025760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.393+0000 7fdbadc9a640 1 -- 172.21.15.103:0/1625789454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e720185d0 msgr2=0x556e72018a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:03.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.393+0000 7fdbadc9a640 1 --2- 172.21.15.103:0/1625789454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e720185d0 0x556e72018a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.393+0000 7fdbadc9a640 1 -- 172.21.15.103:0/1625789454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e71f0a3a0 con 0x556e72019b40 2024-09-19T07:05:03.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.393+0000 7fdba67fc640 1 -- 172.21.15.103:0/3043816911 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fdb9c0cf750 con 0x7fdba81036e0 2024-09-19T07:05:03.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.393+0000 7fdbad499640 1 --2- 172.21.15.103:0/1625789454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e720185d0 0x556e72018a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:03.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.393+0000 7fdbadc9a640 1 --2- 172.21.15.103:0/1625789454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72019b40 0x556e72018090 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fdba00127b0 tx=0x7fdba0012c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca669a46029dca56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:03.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.393+0000 7fdb8f7fe640 1 -- 172.21.15.103:0/1625789454 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0022020 con 0x556e72019b40 2024-09-19T07:05:03.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.393+0000 7fdb8f7fe640 1 -- 172.21.15.103:0/1625789454 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba000cbc0 con 0x556e72019b40 2024-09-19T07:05:03.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.393+0000 7fdb8f7fe640 1 -- 172.21.15.103:0/1625789454 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba001e690 con 0x556e72019b40 2024-09-19T07:05:03.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.394+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/1625789454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72019b40 msgr2=0x556e72018090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:03.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.394+0000 7fdbadc9a640 1 --2- 172.21.15.103:0/3043816911 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdb900a3430 0x7fdb900a58f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdba0020000 tx=0x7fdba001f000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:03.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.394+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/1625789454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72019b40 0x556e72018090 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fdba00127b0 tx=0x7fdba0012c70 comp rx=0 tx=0).stop 2024-09-19T07:05:03.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.394+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/1625789454 shutdown_connections 2024-09-19T07:05:03.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.394+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/1625789454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e72023370 0x556e72025760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.394+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/1625789454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e720185d0 0x556e72018a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.394+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/1625789454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72019b40 0x556e72018090 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.394+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/1625789454 >> 172.21.15.103:0/1625789454 conn(0x556e71fe1310 msgr2=0x556e72014c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:03.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.394+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/1625789454 shutdown_connections 2024-09-19T07:05:03.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.394+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/1625789454 wait complete. 2024-09-19T07:05:03.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.394+0000 7fdbb1ad6740 1 -- start start 2024-09-19T07:05:03.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.394+0000 7fdbb1ad6740 1 Processor -- start 2024-09-19T07:05:03.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbb1ad6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e720185d0 0x556e72022c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbb1ad6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e72019b40 0x556e7201ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbb1ad6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72023370 0x556e720204d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbb1ad6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556e72026070 con 0x556e72023370 2024-09-19T07:05:03.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbb1ad6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556e71f39060 con 0x556e720185d0 2024-09-19T07:05:03.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbb1ad6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556e720261f0 con 0x556e72019b40 2024-09-19T07:05:03.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbb1ad6740 10 client.0 _ll_register_callbacks cb 0x556e71c8c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:03.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbadc9a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72023370 0x556e720204d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbadc9a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72023370 0x556e720204d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60180/0 (socket says 172.21.15.103:60180) 2024-09-19T07:05:03.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbadc9a640 1 -- 172.21.15.103:0/3650177840 learned_addr learned my addr 172.21.15.103:0/3650177840 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:03.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbad499640 1 --2- 172.21.15.103:0/3650177840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e720185d0 0x556e72022c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbacc98640 1 --2- 172.21.15.103:0/3650177840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e72019b40 0x556e7201ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbadc9a640 1 -- 172.21.15.103:0/3650177840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e72019b40 msgr2=0x556e7201ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:03.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbadc9a640 1 --2- 172.21.15.103:0/3650177840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e72019b40 0x556e7201ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbadc9a640 1 -- 172.21.15.103:0/3650177840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e720185d0 msgr2=0x556e72022c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:03.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbadc9a640 1 --2- 172.21.15.103:0/3650177840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e720185d0 0x556e72022c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbadc9a640 1 -- 172.21.15.103:0/3650177840 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e71cef3c0 con 0x556e72023370 2024-09-19T07:05:03.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbacc98640 1 --2- 172.21.15.103:0/3650177840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e72019b40 0x556e7201ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:03.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbadc9a640 1 --2- 172.21.15.103:0/3650177840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72023370 0x556e720204d0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fdba001ec80 tx=0x7fdba0005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9fb11987ca0061 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:03.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdb8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:03.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdb8cff9640 1 -- 172.21.15.103:0/3650177840 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0034060 con 0x556e72023370 2024-09-19T07:05:03.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/3650177840 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdba000cbc0 con 0x556e72023370 2024-09-19T07:05:03.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.395+0000 7fdb8cff9640 1 -- 172.21.15.103:0/3650177840 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fdba0004240 con 0x556e72023370 2024-09-19T07:05:03.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.396+0000 7fdb8cff9640 1 -- 172.21.15.103:0/3650177840 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdba0022070 con 0x556e72023370 2024-09-19T07:05:03.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.396+0000 7fdbb1ad6740 10 client.21015 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:03.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.396+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/3650177840 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e72031a30 con 0x556e72023370 2024-09-19T07:05:03.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.396+0000 7fdb8cff9640 1 -- 172.21.15.103:0/3650177840 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba000cbc0 con 0x556e72023370 2024-09-19T07:05:03.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.396+0000 7fdb8cff9640 1 -- 172.21.15.103:0/3650177840 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdba0004b60 con 0x556e72023370 2024-09-19T07:05:03.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.396+0000 7fdbb1ad6740 10 client.21015 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:03.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.396+0000 7fdbb1ad6740 10 client.21015 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:05:03.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.396+0000 7fdbb1ad6740 20 client.21015 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:03.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.396+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/3650177840 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x556e72032e40 0x556e72035230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.396+0000 7fdbb1ad6740 4 client.21015 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:03.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.396+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/3650177840 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e71f8b860 con 0x556e72032e40 2024-09-19T07:05:03.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.396+0000 7fdbad499640 1 --2- 172.21.15.103:0/3650177840 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x556e72032e40 0x556e72035230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.397+0000 7fdbad499640 1 --2- 172.21.15.103:0/3650177840 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x556e72032e40 0x556e72035230 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e64bf9a372c8786 server_cookie=73be686be47bf73 in_seq=0 out_seq=0 2024-09-19T07:05:03.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.397+0000 7fdb8cff9640 10 client.21015 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:05:03.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.397+0000 7fdb8cff9640 1 -- 172.21.15.103:0/3650177840 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e71f8b860 con 0x556e72032e40 2024-09-19T07:05:03.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.397+0000 7fdb8cff9640 10 client.21015 handle_command_reply: tid=0 2024-09-19T07:05:03.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.398+0000 7fdbb1ad6740 1 client.21015 shutdown 2024-09-19T07:05:03.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.398+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/3650177840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72023370 msgr2=0x556e720204d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:03.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.398+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/3650177840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72023370 0x556e720204d0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fdba001ec80 tx=0x7fdba0005520 comp rx=0 tx=0).stop 2024-09-19T07:05:03.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.398+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/3650177840 shutdown_connections 2024-09-19T07:05:03.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.398+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/3650177840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72023370 0x556e720204d0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.398+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/3650177840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e72019b40 0x556e7201ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.398+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/3650177840 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x556e72032e40 0x556e72035230 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.398+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/3650177840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e720185d0 0x556e72022c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.398+0000 7fdb8cff9640 0 client.21015 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:05:03.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.398+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/3650177840 >> 172.21.15.103:0/3650177840 conn(0x556e71fe1310 msgr2=0x556e7201a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:03.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.398+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/3650177840 shutdown_connections 2024-09-19T07:05:03.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.398+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/3650177840 wait complete. 2024-09-19T07:05:03.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.398+0000 7fdbb1ad6740 20 client.21015 trim_cache size 0 max 16384 2024-09-19T07:05:03.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.412+0000 7fdbb1ad6740 1 Processor -- start 2024-09-19T07:05:03.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.412+0000 7fdbb1ad6740 1 -- start start 2024-09-19T07:05:03.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbb1ad6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e720185d0 0x556e7202f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbb1ad6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72019b40 0x556e7201b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbb1ad6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e72023370 0x556e7201b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbb1ad6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdba81afb50 con 0x556e72019b40 2024-09-19T07:05:03.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbb1ad6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdba81b1690 con 0x556e72023370 2024-09-19T07:05:03.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbb1ad6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdba81b0220 con 0x556e720185d0 2024-09-19T07:05:03.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbacc98640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72019b40 0x556e7201b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbacc98640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72019b40 0x556e7201b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60182/0 (socket says 172.21.15.103:60182) 2024-09-19T07:05:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbacc98640 1 -- 172.21.15.103:0/3330128453 learned_addr learned my addr 172.21.15.103:0/3330128453 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbad499640 1 --2- 172.21.15.103:0/3330128453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e720185d0 0x556e7202f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbadc9a640 1 --2- 172.21.15.103:0/3330128453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e72023370 0x556e7201b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbacc98640 1 -- 172.21.15.103:0/3330128453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e720185d0 msgr2=0x556e7202f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbacc98640 1 --2- 172.21.15.103:0/3330128453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e720185d0 0x556e7202f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbacc98640 1 -- 172.21.15.103:0/3330128453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e72023370 msgr2=0x556e7201b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbacc98640 1 --2- 172.21.15.103:0/3330128453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e72023370 0x556e7201b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbacc98640 1 -- 172.21.15.103:0/3330128453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e7201bba0 con 0x556e72019b40 2024-09-19T07:05:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdbacc98640 1 --2- 172.21.15.103:0/3330128453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72019b40 0x556e7201b120 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c097e70 tx=0x7fdb9c016b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8711a39d11d4f879 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.413+0000 7fdb8effd640 1 -- 172.21.15.103:0/3330128453 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c021040 con 0x556e72019b40 2024-09-19T07:05:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.414+0000 7fdb8effd640 1 -- 172.21.15.103:0/3330128453 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb9c09ba10 con 0x556e72019b40 2024-09-19T07:05:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.414+0000 7fdb8effd640 1 -- 172.21.15.103:0/3330128453 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c098080 con 0x556e72019b40 2024-09-19T07:05:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.414+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/3330128453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72019b40 msgr2=0x556e7201b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:03.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.414+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/3330128453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72019b40 0x556e7201b120 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c097e70 tx=0x7fdb9c016b40 comp rx=0 tx=0).stop 2024-09-19T07:05:03.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.414+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/3330128453 shutdown_connections 2024-09-19T07:05:03.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.414+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/3330128453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e72023370 0x556e7201b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.414+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/3330128453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e72019b40 0x556e7201b120 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.414+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/3330128453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e720185d0 0x556e7202f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.414+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/3330128453 >> 172.21.15.103:0/3330128453 conn(0x556e71fe1310 msgr2=0x556e7201f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:03.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.414+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/3330128453 shutdown_connections 2024-09-19T07:05:03.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.414+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/3330128453 wait complete. 2024-09-19T07:05:03.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.415+0000 7fdbb1ad6740 1 -- start start 2024-09-19T07:05:03.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.415+0000 7fdbb1ad6740 1 Processor -- start 2024-09-19T07:05:03.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.415+0000 7fdbb1ad6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e720185d0 0x556e7202ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.415+0000 7fdbb1ad6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e72019b40 0x556e7201cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.415+0000 7fdbb1ad6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e72023370 0x556e7201d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.415+0000 7fdbb1ad6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdba81b0fc0 con 0x556e720185d0 2024-09-19T07:05:03.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.415+0000 7fdbb1ad6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdba81b08f0 con 0x556e72019b40 2024-09-19T07:05:03.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.415+0000 7fdbb1ad6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdba81adb50 con 0x556e72023370 2024-09-19T07:05:03.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.415+0000 7fdbb1ad6740 10 client.0 _ll_register_callbacks cb 0x7fdba0012580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:03.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.415+0000 7fdbad499640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e720185d0 0x556e7202ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.415+0000 7fdbad499640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e720185d0 0x556e7202ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60194/0 (socket says 172.21.15.103:60194) 2024-09-19T07:05:03.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.415+0000 7fdbad499640 1 -- 172.21.15.103:0/838968845 learned_addr learned my addr 172.21.15.103:0/838968845 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:03.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.415+0000 7fdbadc9a640 1 --2- 172.21.15.103:0/838968845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e72023370 0x556e7201d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.416+0000 7fdbacc98640 1 --2- 172.21.15.103:0/838968845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e72019b40 0x556e7201cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.416+0000 7fdbad499640 1 -- 172.21.15.103:0/838968845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e72023370 msgr2=0x556e7201d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.416+0000 7fdbad499640 1 --2- 172.21.15.103:0/838968845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e72023370 0x556e7201d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.416+0000 7fdbad499640 1 -- 172.21.15.103:0/838968845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e72019b40 msgr2=0x556e7201cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.416+0000 7fdbad499640 1 --2- 172.21.15.103:0/838968845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e72019b40 0x556e7201cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.416+0000 7fdbad499640 1 -- 172.21.15.103:0/838968845 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e7201d730 con 0x556e720185d0 2024-09-19T07:05:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.416+0000 7fdbad499640 1 --2- 172.21.15.103:0/838968845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e720185d0 0x556e7202ba20 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009ca0 tx=0x7fdb980318c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e06ac4e182b478 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.416+0000 7fdb8cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.416+0000 7fdb8cff9640 1 -- 172.21.15.103:0/838968845 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98031d00 con 0x556e720185d0 2024-09-19T07:05:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.416+0000 7fdb8cff9640 1 -- 172.21.15.103:0/838968845 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fdb98003970 con 0x556e720185d0 2024-09-19T07:05:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.416+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/838968845 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb9c09ba10 con 0x556e720185d0 2024-09-19T07:05:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.416+0000 7fdb8cff9640 1 -- 172.21.15.103:0/838968845 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdb98057050 con 0x556e720185d0 2024-09-19T07:05:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.416+0000 7fdbb1ad6740 10 client.21021 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.416+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/838968845 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e7209d710 con 0x556e720185d0 2024-09-19T07:05:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.416+0000 7fdb8cff9640 1 -- 172.21.15.103:0/838968845 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb9c09ba10 con 0x556e720185d0 2024-09-19T07:05:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.416+0000 7fdb8cff9640 1 -- 172.21.15.103:0/838968845 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdb9804e070 con 0x556e720185d0 2024-09-19T07:05:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdbb1ad6740 10 client.21021 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdbb1ad6740 10 client.21021 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:05:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdbb1ad6740 20 client.21021 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/838968845 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x556e72032e40 0x556e7209fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdbb1ad6740 4 client.21021 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:05:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/838968845 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e71f85d70 con 0x556e72032e40 2024-09-19T07:05:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdbacc98640 1 --2- 172.21.15.103:0/838968845 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x556e72032e40 0x556e7209fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdbacc98640 1 --2- 172.21.15.103:0/838968845 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x556e72032e40 0x556e7209fe80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25ae1560f9437afd server_cookie=10a16abbe847610e in_seq=0 out_seq=0 2024-09-19T07:05:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdb8cff9640 10 client.21021 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:05:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdb8cff9640 1 -- 172.21.15.103:0/838968845 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e71f85d70 con 0x556e72032e40 2024-09-19T07:05:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdb8cff9640 10 client.21021 handle_command_reply: tid=0 2024-09-19T07:05:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdbb1ad6740 1 client.21021 shutdown 2024-09-19T07:05:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/838968845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e720185d0 msgr2=0x556e7202ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:03.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/838968845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e720185d0 0x556e7202ba20 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009ca0 tx=0x7fdb980318c0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/838968845 shutdown_connections 2024-09-19T07:05:03.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/838968845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556e72023370 0x556e7201d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.417+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/838968845 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x556e72032e40 0x556e7209fe80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/838968845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e72019b40 0x556e7201cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdbb1ad6740 1 --2- 172.21.15.103:0/838968845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e720185d0 0x556e7202ba20 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdb8cff9640 0 client.21021 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:05:03.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/838968845 >> 172.21.15.103:0/838968845 conn(0x556e71fe1310 msgr2=0x556e72013310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:03.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/838968845 shutdown_connections 2024-09-19T07:05:03.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdbb1ad6740 1 -- 172.21.15.103:0/838968845 wait complete. 2024-09-19T07:05:03.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdbb1ad6740 20 client.21021 trim_cache size 0 max 16384 2024-09-19T07:05:03.430 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:05:03.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdbaf6fd640 1 -- 172.21.15.103:0/3043816911 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdb900a3430 msgr2=0x7fdb900a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:03.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdbaf6fd640 1 --2- 172.21.15.103:0/3043816911 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdb900a3430 0x7fdb900a58f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdba0020000 tx=0x7fdba001f000 comp rx=0 tx=0).stop 2024-09-19T07:05:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdbaf6fd640 1 -- 172.21.15.103:0/3043816911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba81036e0 msgr2=0x7fdba8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdbaf6fd640 1 --2- 172.21.15.103:0/3043816911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba81036e0 0x7fdba8100ed0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c00d740 tx=0x7fdb9c00dc00 comp rx=0 tx=0).stop 2024-09-19T07:05:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdbaf6fd640 1 -- 172.21.15.103:0/3043816911 shutdown_connections 2024-09-19T07:05:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdbaf6fd640 1 --2- 172.21.15.103:0/3043816911 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fdb900a3430 0x7fdb900a58f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdbaf6fd640 1 --2- 172.21.15.103:0/3043816911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba81082a0 0x7fdba8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdbaf6fd640 1 --2- 172.21.15.103:0/3043816911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba81036e0 0x7fdba8100ed0 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdbaf6fd640 1 --2- 172.21.15.103:0/3043816911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdba8102d30 0x7fdba8100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.418+0000 7fdbaf6fd640 1 -- 172.21.15.103:0/3043816911 >> 172.21.15.103:0/3043816911 conn(0x7fdba80fc7d0 msgr2=0x7fdba8105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.419+0000 7fdbaf6fd640 1 -- 172.21.15.103:0/3043816911 shutdown_connections 2024-09-19T07:05:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:03.419+0000 7fdbaf6fd640 1 -- 172.21.15.103:0/3043816911 wait complete. 2024-09-19T07:05:03.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:03 smithi174 ceph-mon[35637]: pgmap v495: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-09-19T07:05:03.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:03 smithi134 ceph-mon[35356]: pgmap v495: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-09-19T07:05:04.129 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:05:04.377 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:04.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.769+0000 7f26b99e9640 1 -- 172.21.15.103:0/1544129470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26b4101410 msgr2=0x7f26b410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:04.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.769+0000 7f26b99e9640 1 --2- 172.21.15.103:0/1544129470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26b4101410 0x7f26b410d940 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009930 tx=0x7f26a402a420 comp rx=0 tx=0).stop 2024-09-19T07:05:04.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.769+0000 7f26b99e9640 1 -- 172.21.15.103:0/1544129470 shutdown_connections 2024-09-19T07:05:04.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.769+0000 7f26b99e9640 1 --2- 172.21.15.103:0/1544129470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26b4101410 0x7f26b410d940 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.769+0000 7f26b99e9640 1 --2- 172.21.15.103:0/1544129470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b4100a70 0x7f26b4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.769+0000 7f26b99e9640 1 --2- 172.21.15.103:0/1544129470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b4106a70 0x7f26b4106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.769+0000 7f26b99e9640 1 -- 172.21.15.103:0/1544129470 >> 172.21.15.103:0/1544129470 conn(0x7f26b40fc790 msgr2=0x7f26b40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:04.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.769+0000 7f26b99e9640 1 -- 172.21.15.103:0/1544129470 shutdown_connections 2024-09-19T07:05:04.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.769+0000 7f26b99e9640 1 -- 172.21.15.103:0/1544129470 wait complete. 2024-09-19T07:05:04.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.769+0000 7f26b99e9640 1 Processor -- start 2024-09-19T07:05:04.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.769+0000 7f26b99e9640 1 -- start start 2024-09-19T07:05:04.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b99e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26b4100a70 0x7f26b41992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b99e9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b4101410 0x7f26b41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b99e9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b4106a70 0x7f26b41a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b99e9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4110e70 con 0x7f26b4100a70 2024-09-19T07:05:04.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b99e9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4110cf0 con 0x7f26b4101410 2024-09-19T07:05:04.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b99e9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4110ff0 con 0x7f26b4106a70 2024-09-19T07:05:04.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26b4100a70 0x7f26b41992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26b4100a70 0x7f26b41992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60200/0 (socket says 172.21.15.103:60200) 2024-09-19T07:05:04.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b2ffd640 1 -- 172.21.15.103:0/288069207 learned_addr learned my addr 172.21.15.103:0/288069207 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:04.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b27fc640 1 --2- 172.21.15.103:0/288069207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b4101410 0x7f26b41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b37fe640 1 --2- 172.21.15.103:0/288069207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b4106a70 0x7f26b41a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b2ffd640 1 -- 172.21.15.103:0/288069207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b4106a70 msgr2=0x7f26b41a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:04.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b2ffd640 1 --2- 172.21.15.103:0/288069207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b4106a70 0x7f26b41a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b2ffd640 1 -- 172.21.15.103:0/288069207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b4101410 msgr2=0x7f26b41997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:04.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b2ffd640 1 --2- 172.21.15.103:0/288069207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b4101410 0x7f26b41997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b2ffd640 1 -- 172.21.15.103:0/288069207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b41a4860 con 0x7f26b4100a70 2024-09-19T07:05:04.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b2ffd640 1 --2- 172.21.15.103:0/288069207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26b4100a70 0x7f26b41992a0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00cce0 tx=0x7f26ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:04.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.770+0000 7f26b89e7640 1 -- 172.21.15.103:0/288069207 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac004b20 con 0x7f26b4100a70 2024-09-19T07:05:04.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.771+0000 7f26b99e9640 1 -- 172.21.15.103:0/288069207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b41a4a30 con 0x7f26b4100a70 2024-09-19T07:05:04.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.771+0000 7f26b89e7640 1 -- 172.21.15.103:0/288069207 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac004cc0 con 0x7f26b4100a70 2024-09-19T07:05:04.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.771+0000 7f26b89e7640 1 -- 172.21.15.103:0/288069207 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00f8c0 con 0x7f26b4100a70 2024-09-19T07:05:04.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.771+0000 7f26b99e9640 1 -- 172.21.15.103:0/288069207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b41a4f40 con 0x7f26b4100a70 2024-09-19T07:05:04.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.774+0000 7f26bbdc2740 1 Processor -- start 2024-09-19T07:05:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.774+0000 7f26bbdc2740 1 -- start start 2024-09-19T07:05:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.774+0000 7f26b89e7640 1 -- 172.21.15.103:0/288069207 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f26ac00fa60 con 0x7f26b4100a70 2024-09-19T07:05:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.774+0000 7f26bbdc2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa59b40 0x55d03aa58090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.774+0000 7f26bbdc2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa58a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.774+0000 7f26bbdc2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa67780 0x55d03aa69b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.774+0000 7f26bbdc2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d03a9be2f0 con 0x55d03aa585d0 2024-09-19T07:05:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.774+0000 7f26bbdc2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d03a965cf0 con 0x55d03aa59b40 2024-09-19T07:05:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.774+0000 7f26bbdc2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d03a9ba2b0 con 0x55d03aa67780 2024-09-19T07:05:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.774+0000 7f26b2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa58a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.774+0000 7f26b2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa58a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60206/0 (socket says 172.21.15.103:60206) 2024-09-19T07:05:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.774+0000 7f26b2ffd640 1 -- 172.21.15.103:0/3973313770 learned_addr learned my addr 172.21.15.103:0/3973313770 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.774+0000 7f26b27fc640 1 --2- 172.21.15.103:0/3973313770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa67780 0x55d03aa69b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.774+0000 7f26b37fe640 1 --2- 172.21.15.103:0/3973313770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa59b40 0x55d03aa58090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.775+0000 7f26b2ffd640 1 -- 172.21.15.103:0/3973313770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa67780 msgr2=0x55d03aa69b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.775+0000 7f26b2ffd640 1 --2- 172.21.15.103:0/3973313770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa67780 0x55d03aa69b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.775+0000 7f26b2ffd640 1 -- 172.21.15.103:0/3973313770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa59b40 msgr2=0x55d03aa58090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.775+0000 7f26b2ffd640 1 --2- 172.21.15.103:0/3973313770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa59b40 0x55d03aa58090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.775+0000 7f26b2ffd640 1 -- 172.21.15.103:0/3973313770 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03a94a3a0 con 0x55d03aa585d0 2024-09-19T07:05:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.775+0000 7f26b37fe640 1 --2- 172.21.15.103:0/3973313770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa59b40 0x55d03aa58090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.775+0000 7f26b2ffd640 1 --2- 172.21.15.103:0/3973313770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa58a10 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0cd390 tx=0x7f26ac0d8830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745bbfd695b1f71a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.775+0000 7f26a9ffb640 1 -- 172.21.15.103:0/3973313770 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac0e5070 con 0x55d03aa585d0 2024-09-19T07:05:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.775+0000 7f26b89e7640 1 --2- 172.21.15.103:0/288069207 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26a00a33b0 0x7f26a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.775+0000 7f26a9ffb640 1 -- 172.21.15.103:0/3973313770 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac0d6d90 con 0x55d03aa585d0 2024-09-19T07:05:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.775+0000 7f26a9ffb640 1 -- 172.21.15.103:0/3973313770 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac09bba0 con 0x55d03aa585d0 2024-09-19T07:05:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.775+0000 7f26b37fe640 1 --2- 172.21.15.103:0/288069207 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26a00a33b0 0x7f26a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.775+0000 7f26b89e7640 1 -- 172.21.15.103:0/288069207 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f26ac0cfcf0 con 0x7f26b4100a70 2024-09-19T07:05:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.776+0000 7f26bbdc2740 1 -- 172.21.15.103:0/3973313770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 msgr2=0x55d03aa58a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.776+0000 7f26b37fe640 1 --2- 172.21.15.103:0/288069207 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26a00a33b0 0x7f26a00a5870 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f26a4006010 tx=0x7f26a40095e0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.776+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/3973313770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa58a10 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0cd390 tx=0x7f26ac0d8830 comp rx=0 tx=0).stop 2024-09-19T07:05:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.776+0000 7f26bbdc2740 1 -- 172.21.15.103:0/3973313770 shutdown_connections 2024-09-19T07:05:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.776+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/3973313770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa67780 0x55d03aa69b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.776+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/3973313770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa58a10 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.776+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/3973313770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa59b40 0x55d03aa58090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.776+0000 7f26bbdc2740 1 -- 172.21.15.103:0/3973313770 >> 172.21.15.103:0/3973313770 conn(0x55d03aa21310 msgr2=0x55d03aa54c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.776+0000 7f26bbdc2740 1 -- 172.21.15.103:0/3973313770 shutdown_connections 2024-09-19T07:05:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.776+0000 7f26bbdc2740 1 -- 172.21.15.103:0/3973313770 wait complete. 2024-09-19T07:05:04.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.776+0000 7f26bbdc2740 1 -- start start 2024-09-19T07:05:04.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26bbdc2740 1 Processor -- start 2024-09-19T07:05:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26bbdc2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa67160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26bbdc2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa59b40 0x55d03aa64410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26bbdc2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa67780 0x55d03aa649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26bbdc2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d03aa6a480 con 0x55d03aa585d0 2024-09-19T07:05:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26bbdc2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d03a979060 con 0x55d03aa59b40 2024-09-19T07:05:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26bbdc2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d03aa6a600 con 0x55d03aa67780 2024-09-19T07:05:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26bbdc2740 10 client.0 _ll_register_callbacks cb 0x55d03a6cc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26b2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa67160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26b2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa67160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60220/0 (socket says 172.21.15.103:60220) 2024-09-19T07:05:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26b2ffd640 1 -- 172.21.15.103:0/730983804 learned_addr learned my addr 172.21.15.103:0/730983804 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26b37fe640 1 --2- 172.21.15.103:0/730983804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa67780 0x55d03aa649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26b27fc640 1 --2- 172.21.15.103:0/730983804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa59b40 0x55d03aa64410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26b2ffd640 1 -- 172.21.15.103:0/730983804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa67780 msgr2=0x55d03aa649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26b2ffd640 1 --2- 172.21.15.103:0/730983804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa67780 0x55d03aa649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26b2ffd640 1 -- 172.21.15.103:0/730983804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa59b40 msgr2=0x55d03aa64410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26b2ffd640 1 --2- 172.21.15.103:0/730983804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa59b40 0x55d03aa64410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.777+0000 7f26b2ffd640 1 -- 172.21.15.103:0/730983804 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03a72f3c0 con 0x55d03aa585d0 2024-09-19T07:05:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.778+0000 7f26b2ffd640 1 --2- 172.21.15.103:0/730983804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa67160 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0cd4d0 tx=0x7f26ac09bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9cf8ade2a14c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.778+0000 7f2692ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.778+0000 7f2692ffd640 1 -- 172.21.15.103:0/730983804 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac09a620 con 0x55d03aa585d0 2024-09-19T07:05:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.778+0000 7f2692ffd640 1 -- 172.21.15.103:0/730983804 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f26ac0d3350 con 0x55d03aa585d0 2024-09-19T07:05:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.778+0000 7f26bbdc2740 1 -- 172.21.15.103:0/730983804 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26ac0d6d90 con 0x55d03aa585d0 2024-09-19T07:05:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.778+0000 7f2692ffd640 1 -- 172.21.15.103:0/730983804 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f26ac09a810 con 0x55d03aa585d0 2024-09-19T07:05:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.778+0000 7f26bbdc2740 10 client.21033 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.778+0000 7f26bbdc2740 1 -- 172.21.15.103:0/730983804 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d03aa6a480 con 0x55d03aa585d0 2024-09-19T07:05:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.778+0000 7f2692ffd640 1 -- 172.21.15.103:0/730983804 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac0d6d90 con 0x55d03aa585d0 2024-09-19T07:05:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.778+0000 7f2692ffd640 1 -- 172.21.15.103:0/730983804 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f26ac0986d0 con 0x55d03aa585d0 2024-09-19T07:05:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.778+0000 7f26bbdc2740 10 client.21033 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.778+0000 7f26bbdc2740 10 client.21033 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:05:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.778+0000 7f26bbdc2740 20 client.21033 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.778+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/730983804 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d03aa770b0 0x55d03aa794a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.778+0000 7f26bbdc2740 4 client.21033 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.778+0000 7f26bbdc2740 1 -- 172.21.15.103:0/730983804 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d03a9cb860 con 0x55d03aa770b0 2024-09-19T07:05:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.779+0000 7f26b27fc640 1 --2- 172.21.15.103:0/730983804 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d03aa770b0 0x55d03aa794a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.779+0000 7f2692ffd640 10 client.21033 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:05:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.779+0000 7f26b27fc640 1 --2- 172.21.15.103:0/730983804 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d03aa770b0 0x55d03aa794a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67ae7a9141295382 server_cookie=ae32ddd9623d2d3f in_seq=0 out_seq=0 2024-09-19T07:05:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.780+0000 7f2692ffd640 1 -- 172.21.15.103:0/730983804 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d03a9cb860 con 0x55d03aa770b0 2024-09-19T07:05:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.780+0000 7f2692ffd640 10 client.21033 handle_command_reply: tid=0 2024-09-19T07:05:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.780+0000 7f26bbdc2740 1 client.21033 shutdown 2024-09-19T07:05:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.780+0000 7f26bbdc2740 1 -- 172.21.15.103:0/730983804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 msgr2=0x55d03aa67160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.780+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/730983804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa67160 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0cd4d0 tx=0x7f26ac09bf10 comp rx=0 tx=0).stop 2024-09-19T07:05:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.780+0000 7f26bbdc2740 1 -- 172.21.15.103:0/730983804 shutdown_connections 2024-09-19T07:05:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.780+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/730983804 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d03aa770b0 0x55d03aa794a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.780+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/730983804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa67780 0x55d03aa649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.780+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/730983804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa59b40 0x55d03aa64410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.780+0000 7f2692ffd640 0 client.21033 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:05:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.780+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/730983804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa67160 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.780+0000 7f26bbdc2740 1 -- 172.21.15.103:0/730983804 >> 172.21.15.103:0/730983804 conn(0x55d03aa21310 msgr2=0x55d03aa5a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.780+0000 7f26bbdc2740 1 -- 172.21.15.103:0/730983804 shutdown_connections 2024-09-19T07:05:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.780+0000 7f26bbdc2740 1 -- 172.21.15.103:0/730983804 wait complete. 2024-09-19T07:05:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.780+0000 7f26bbdc2740 20 client.21033 trim_cache size 0 max 16384 2024-09-19T07:05:04.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26bbdc2740 1 Processor -- start 2024-09-19T07:05:04.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26bbdc2740 1 -- start start 2024-09-19T07:05:04.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26bbdc2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa73750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26bbdc2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa59b40 0x55d03aa68b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26bbdc2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa67780 0x55d03aa690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26bbdc2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41ab740 con 0x55d03aa585d0 2024-09-19T07:05:04.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26bbdc2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41ad280 con 0x55d03aa67780 2024-09-19T07:05:04.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26bbdc2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41abe10 con 0x55d03aa59b40 2024-09-19T07:05:04.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26b2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa73750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26b2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa73750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60224/0 (socket says 172.21.15.103:60224) 2024-09-19T07:05:04.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26b2ffd640 1 -- 172.21.15.103:0/492977151 learned_addr learned my addr 172.21.15.103:0/492977151 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:04.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26b27fc640 1 --2- 172.21.15.103:0/492977151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa59b40 0x55d03aa68b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26b37fe640 1 --2- 172.21.15.103:0/492977151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa67780 0x55d03aa690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26b2ffd640 1 -- 172.21.15.103:0/492977151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa59b40 msgr2=0x55d03aa68b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:04.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26b2ffd640 1 --2- 172.21.15.103:0/492977151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa59b40 0x55d03aa68b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26b2ffd640 1 -- 172.21.15.103:0/492977151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa67780 msgr2=0x55d03aa690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:04.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.793+0000 7f26b2ffd640 1 --2- 172.21.15.103:0/492977151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa67780 0x55d03aa690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.794+0000 7f26b2ffd640 1 -- 172.21.15.103:0/492977151 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03aa674d0 con 0x55d03aa585d0 2024-09-19T07:05:04.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.794+0000 7f26b2ffd640 1 --2- 172.21.15.103:0/492977151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa73750 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0984c0 tx=0x7f26ac0d8830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafd3dc6e3d62ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:04.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.794+0000 7f26a97fa640 1 -- 172.21.15.103:0/492977151 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac0dc070 con 0x55d03aa585d0 2024-09-19T07:05:04.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.794+0000 7f26a97fa640 1 -- 172.21.15.103:0/492977151 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac09a9b0 con 0x55d03aa585d0 2024-09-19T07:05:04.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.794+0000 7f26a97fa640 1 -- 172.21.15.103:0/492977151 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac01d050 con 0x55d03aa585d0 2024-09-19T07:05:04.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.794+0000 7f26bbdc2740 1 -- 172.21.15.103:0/492977151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 msgr2=0x55d03aa73750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:04.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.794+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/492977151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa73750 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0984c0 tx=0x7f26ac0d8830 comp rx=0 tx=0).stop 2024-09-19T07:05:04.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.794+0000 7f26bbdc2740 1 -- 172.21.15.103:0/492977151 shutdown_connections 2024-09-19T07:05:04.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.794+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/492977151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa67780 0x55d03aa690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.794+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/492977151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa59b40 0x55d03aa68b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.794+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/492977151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa585d0 0x55d03aa73750 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.794+0000 7f26bbdc2740 1 -- 172.21.15.103:0/492977151 >> 172.21.15.103:0/492977151 conn(0x55d03aa21310 msgr2=0x55d03aa5f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:04.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.795+0000 7f26bbdc2740 1 -- 172.21.15.103:0/492977151 shutdown_connections 2024-09-19T07:05:04.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.795+0000 7f26bbdc2740 1 -- 172.21.15.103:0/492977151 wait complete. 2024-09-19T07:05:04.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.795+0000 7f26bbdc2740 1 -- start start 2024-09-19T07:05:04.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.795+0000 7f26bbdc2740 1 Processor -- start 2024-09-19T07:05:04.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.795+0000 7f26bbdc2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa585d0 0x55d03aa6fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.795+0000 7f26bbdc2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa59b40 0x55d03aa70070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.795+0000 7f26bbdc2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa67780 0x55d03aa7e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.795+0000 7f26bbdc2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41acbb0 con 0x55d03aa59b40 2024-09-19T07:05:04.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.795+0000 7f26bbdc2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41ac4e0 con 0x55d03aa67780 2024-09-19T07:05:04.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.795+0000 7f26bbdc2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41a9740 con 0x55d03aa585d0 2024-09-19T07:05:04.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.795+0000 7f26bbdc2740 10 client.0 _ll_register_callbacks cb 0x7f26ac0e3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:04.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.795+0000 7f26b27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa59b40 0x55d03aa70070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.795+0000 7f26b27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa59b40 0x55d03aa70070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60230/0 (socket says 172.21.15.103:60230) 2024-09-19T07:05:04.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.795+0000 7f26b27fc640 1 -- 172.21.15.103:0/3816543479 learned_addr learned my addr 172.21.15.103:0/3816543479 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:04.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.796+0000 7f26b37fe640 1 --2- 172.21.15.103:0/3816543479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa67780 0x55d03aa7e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.796+0000 7f26b2ffd640 1 --2- 172.21.15.103:0/3816543479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa585d0 0x55d03aa6fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.796+0000 7f26b27fc640 1 -- 172.21.15.103:0/3816543479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa585d0 msgr2=0x55d03aa6fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:04.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.796+0000 7f26b27fc640 1 --2- 172.21.15.103:0/3816543479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa585d0 0x55d03aa6fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.796+0000 7f26b27fc640 1 -- 172.21.15.103:0/3816543479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa67780 msgr2=0x55d03aa7e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:04.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.796+0000 7f26b27fc640 1 --2- 172.21.15.103:0/3816543479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa67780 0x55d03aa7e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.796+0000 7f26b27fc640 1 -- 172.21.15.103:0/3816543479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03aa7e9c0 con 0x55d03aa59b40 2024-09-19T07:05:04.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.796+0000 7f26b27fc640 1 --2- 172.21.15.103:0/3816543479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa59b40 0x55d03aa70070 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f269c000c00 tx=0x7f269c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945074138aa9ebd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:04.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.796+0000 7f2692ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:04.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.796+0000 7f2692ffd640 1 -- 172.21.15.103:0/3816543479 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c005220 con 0x55d03aa59b40 2024-09-19T07:05:04.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.796+0000 7f2692ffd640 1 -- 172.21.15.103:0/3816543479 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f269c0053c0 con 0x55d03aa59b40 2024-09-19T07:05:04.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.796+0000 7f26bbdc2740 1 -- 172.21.15.103:0/3816543479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26ac09a9b0 con 0x55d03aa59b40 2024-09-19T07:05:04.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.796+0000 7f2692ffd640 1 -- 172.21.15.103:0/3816543479 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f269c0055e0 con 0x55d03aa59b40 2024-09-19T07:05:04.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.796+0000 7f26bbdc2740 10 client.21039 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:04.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.796+0000 7f26bbdc2740 1 -- 172.21.15.103:0/3816543479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d03aa6f450 con 0x55d03aa59b40 2024-09-19T07:05:04.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.797+0000 7f2692ffd640 1 -- 172.21.15.103:0/3816543479 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac09a9b0 con 0x55d03aa59b40 2024-09-19T07:05:04.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.797+0000 7f2692ffd640 1 -- 172.21.15.103:0/3816543479 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f269c02d850 con 0x55d03aa59b40 2024-09-19T07:05:04.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.797+0000 7f26bbdc2740 10 client.21039 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:04.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.797+0000 7f26bbdc2740 10 client.21039 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:05:04.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.797+0000 7f26bbdc2740 20 client.21039 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:04.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.797+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/3816543479 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d03aae1e90 0x55d03aae42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:04.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.797+0000 7f26bbdc2740 4 client.21039 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:05:04.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.797+0000 7f26bbdc2740 1 -- 172.21.15.103:0/3816543479 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d03a9c5d70 con 0x55d03aae1e90 2024-09-19T07:05:04.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.797+0000 7f26b2ffd640 1 --2- 172.21.15.103:0/3816543479 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d03aae1e90 0x55d03aae42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:04.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f2692ffd640 10 client.21039 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:05:04.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f26b2ffd640 1 --2- 172.21.15.103:0/3816543479 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d03aae1e90 0x55d03aae42b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99b5951259c7f80f server_cookie=c325811719e96e4f in_seq=0 out_seq=0 2024-09-19T07:05:04.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f2692ffd640 1 -- 172.21.15.103:0/3816543479 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d03a9c5d70 con 0x55d03aae1e90 2024-09-19T07:05:04.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f2692ffd640 10 client.21039 handle_command_reply: tid=0 2024-09-19T07:05:04.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f26bbdc2740 1 client.21039 shutdown 2024-09-19T07:05:04.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f26bbdc2740 1 -- 172.21.15.103:0/3816543479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa59b40 msgr2=0x55d03aa70070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:04.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/3816543479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa59b40 0x55d03aa70070 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f269c000c00 tx=0x7f269c00bf90 comp rx=0 tx=0).stop 2024-09-19T07:05:04.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f26bbdc2740 1 -- 172.21.15.103:0/3816543479 shutdown_connections 2024-09-19T07:05:04.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/3816543479 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55d03aae1e90 0x55d03aae42b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/3816543479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d03aa67780 0x55d03aa7e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/3816543479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d03aa59b40 0x55d03aa70070 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f26bbdc2740 1 --2- 172.21.15.103:0/3816543479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d03aa585d0 0x55d03aa6fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f26bbdc2740 1 -- 172.21.15.103:0/3816543479 >> 172.21.15.103:0/3816543479 conn(0x55d03aa21310 msgr2=0x55d03aa62ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:04.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f2692ffd640 0 client.21039 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:05:04.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f26bbdc2740 1 -- 172.21.15.103:0/3816543479 shutdown_connections 2024-09-19T07:05:04.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f26bbdc2740 1 -- 172.21.15.103:0/3816543479 wait complete. 2024-09-19T07:05:04.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.798+0000 7f26bbdc2740 20 client.21039 trim_cache size 0 max 16384 2024-09-19T07:05:04.805 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:05:04.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.799+0000 7f26b99e9640 1 -- 172.21.15.103:0/288069207 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26a00a33b0 msgr2=0x7f26a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:04.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.799+0000 7f26b99e9640 1 --2- 172.21.15.103:0/288069207 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26a00a33b0 0x7f26a00a5870 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f26a4006010 tx=0x7f26a40095e0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.799+0000 7f26b99e9640 1 -- 172.21.15.103:0/288069207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26b4100a70 msgr2=0x7f26b41992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:04.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.799+0000 7f26b99e9640 1 --2- 172.21.15.103:0/288069207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26b4100a70 0x7f26b41992a0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00cce0 tx=0x7f26ac00a620 comp rx=0 tx=0).stop 2024-09-19T07:05:04.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.799+0000 7f26b99e9640 1 -- 172.21.15.103:0/288069207 shutdown_connections 2024-09-19T07:05:04.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.799+0000 7f26b99e9640 1 --2- 172.21.15.103:0/288069207 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26a00a33b0 0x7f26a00a5870 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.799+0000 7f26b99e9640 1 --2- 172.21.15.103:0/288069207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b4106a70 0x7f26b41a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.799+0000 7f26b99e9640 1 --2- 172.21.15.103:0/288069207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b4101410 0x7f26b41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.799+0000 7f26b99e9640 1 --2- 172.21.15.103:0/288069207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26b4100a70 0x7f26b41992a0 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:04.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.799+0000 7f26b99e9640 1 -- 172.21.15.103:0/288069207 >> 172.21.15.103:0/288069207 conn(0x7f26b40fc790 msgr2=0x7f26b40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:04.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.799+0000 7f26b99e9640 1 -- 172.21.15.103:0/288069207 shutdown_connections 2024-09-19T07:05:04.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:04.799+0000 7f26b99e9640 1 -- 172.21.15.103:0/288069207 wait complete. 2024-09-19T07:05:05.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:05 smithi103 ceph-mon[31158]: pgmap v496: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-09-19T07:05:05.512 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:05:05.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:05 smithi174 ceph-mon[35637]: pgmap v496: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-09-19T07:05:05.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:05 smithi134 ceph-mon[35356]: pgmap v496: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-09-19T07:05:05.757 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:06.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.154+0000 7efc51f6f640 1 -- 172.21.15.103:0/1280422804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc4c1082a0 msgr2=0x7efc4c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:06.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.154+0000 7efc51f6f640 1 --2- 172.21.15.103:0/1280422804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc4c1082a0 0x7efc4c10ec40 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7efc44009cd0 tx=0x7efc4402a8e0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.154+0000 7efc51f6f640 1 -- 172.21.15.103:0/1280422804 shutdown_connections 2024-09-19T07:05:06.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.154+0000 7efc51f6f640 1 --2- 172.21.15.103:0/1280422804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc4c1082a0 0x7efc4c10ec40 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.154+0000 7efc51f6f640 1 --2- 172.21.15.103:0/1280422804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc4c1036e0 0x7efc4c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.154+0000 7efc51f6f640 1 --2- 172.21.15.103:0/1280422804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc4c102d30 0x7efc4c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.154+0000 7efc51f6f640 1 -- 172.21.15.103:0/1280422804 >> 172.21.15.103:0/1280422804 conn(0x7efc4c0fc7d0 msgr2=0x7efc4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:06.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.154+0000 7efc51f6f640 1 -- 172.21.15.103:0/1280422804 shutdown_connections 2024-09-19T07:05:06.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.154+0000 7efc51f6f640 1 -- 172.21.15.103:0/1280422804 wait complete. 2024-09-19T07:05:06.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.154+0000 7efc51f6f640 1 Processor -- start 2024-09-19T07:05:06.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.154+0000 7efc51f6f640 1 -- start start 2024-09-19T07:05:06.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc51f6f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc4c102d30 0x7efc4c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc51f6f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc4c1036e0 0x7efc4c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc51f6f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc4c1082a0 0x7efc4c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc51f6f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c112100 con 0x7efc4c102d30 2024-09-19T07:05:06.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc51f6f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c111f80 con 0x7efc4c1082a0 2024-09-19T07:05:06.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc51f6f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c112280 con 0x7efc4c1036e0 2024-09-19T07:05:06.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc4b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc4c102d30 0x7efc4c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc4b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc4c102d30 0x7efc4c198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60240/0 (socket says 172.21.15.103:60240) 2024-09-19T07:05:06.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc4b7fe640 1 -- 172.21.15.103:0/1731375745 learned_addr learned my addr 172.21.15.103:0/1731375745 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:06.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc43fff640 1 --2- 172.21.15.103:0/1731375745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc4c1036e0 0x7efc4c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc4bfff640 1 --2- 172.21.15.103:0/1731375745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc4c1082a0 0x7efc4c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc4b7fe640 1 -- 172.21.15.103:0/1731375745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc4c1036e0 msgr2=0x7efc4c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:06.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc4b7fe640 1 --2- 172.21.15.103:0/1731375745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc4c1036e0 0x7efc4c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc4b7fe640 1 -- 172.21.15.103:0/1731375745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc4c1082a0 msgr2=0x7efc4c19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:06.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc4b7fe640 1 --2- 172.21.15.103:0/1731375745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc4c1082a0 0x7efc4c19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc4b7fe640 1 -- 172.21.15.103:0/1731375745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c19d500 con 0x7efc4c102d30 2024-09-19T07:05:06.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc4bfff640 1 --2- 172.21.15.103:0/1731375745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc4c1082a0 0x7efc4c19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:06.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.155+0000 7efc4b7fe640 1 --2- 172.21.15.103:0/1731375745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc4c102d30 0x7efc4c198550 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00b4a0 tx=0x7efc3c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:06.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.156+0000 7efc497fa640 1 -- 172.21.15.103:0/1731375745 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c015020 con 0x7efc4c102d30 2024-09-19T07:05:06.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.156+0000 7efc51f6f640 1 -- 172.21.15.103:0/1731375745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4c19d7f0 con 0x7efc4c102d30 2024-09-19T07:05:06.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.156+0000 7efc497fa640 1 -- 172.21.15.103:0/1731375745 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc3c011d10 con 0x7efc4c102d30 2024-09-19T07:05:06.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.156+0000 7efc497fa640 1 -- 172.21.15.103:0/1731375745 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c00f2e0 con 0x7efc4c102d30 2024-09-19T07:05:06.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.156+0000 7efc51f6f640 1 -- 172.21.15.103:0/1731375745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4c1a50d0 con 0x7efc4c102d30 2024-09-19T07:05:06.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.158+0000 7efc497fa640 1 -- 172.21.15.103:0/1731375745 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7efc3c018020 con 0x7efc4c102d30 2024-09-19T07:05:06.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.158+0000 7efc54348740 1 Processor -- start 2024-09-19T07:05:06.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.158+0000 7efc54348740 1 -- start start 2024-09-19T07:05:06.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.158+0000 7efc54348740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f7cb40 0x560ea7f7b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.158+0000 7efc54348740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7b5d0 0x560ea7f7ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.158+0000 7efc54348740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f8a780 0x560ea7f8cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc54348740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560ea7ee12f0 con 0x560ea7f7cb40 2024-09-19T07:05:06.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc54348740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560ea7e88cf0 con 0x560ea7f8a780 2024-09-19T07:05:06.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc54348740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560ea7edd2b0 con 0x560ea7f7b5d0 2024-09-19T07:05:06.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f7cb40 0x560ea7f7b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f7cb40 0x560ea7f7b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60242/0 (socket says 172.21.15.103:60242) 2024-09-19T07:05:06.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc4bfff640 1 -- 172.21.15.103:0/3964317603 learned_addr learned my addr 172.21.15.103:0/3964317603 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:06.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc497fa640 1 --2- 172.21.15.103:0/1731375745 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efc380a3430 0x7efc380a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc4b7fe640 1 --2- 172.21.15.103:0/3964317603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7b5d0 0x560ea7f7ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc43fff640 1 --2- 172.21.15.103:0/3964317603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f8a780 0x560ea7f8cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc4bfff640 1 -- 172.21.15.103:0/3964317603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7b5d0 msgr2=0x560ea7f7ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:06.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc4bfff640 1 --2- 172.21.15.103:0/3964317603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7b5d0 0x560ea7f7ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc4bfff640 1 -- 172.21.15.103:0/3964317603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f8a780 msgr2=0x560ea7f8cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:06.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc4bfff640 1 --2- 172.21.15.103:0/3964317603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f8a780 0x560ea7f8cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc4bfff640 1 -- 172.21.15.103:0/3964317603 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ea7e6d3a0 con 0x560ea7f7cb40 2024-09-19T07:05:06.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc497fa640 1 -- 172.21.15.103:0/1731375745 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7efc3c0cf560 con 0x7efc4c102d30 2024-09-19T07:05:06.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc4bfff640 1 --2- 172.21.15.103:0/1731375745 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efc380a3430 0x7efc380a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc4bfff640 1 --2- 172.21.15.103:0/3964317603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f7cb40 0x560ea7f7b090 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7efc44031650 tx=0x7efc44031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17440394077e1e4c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:06.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.159+0000 7efc41ffb640 1 -- 172.21.15.103:0/3964317603 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44003e40 con 0x560ea7f7cb40 2024-09-19T07:05:06.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.160+0000 7efc41ffb640 1 -- 172.21.15.103:0/3964317603 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc44036040 con 0x560ea7f7cb40 2024-09-19T07:05:06.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.160+0000 7efc41ffb640 1 -- 172.21.15.103:0/3964317603 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44030880 con 0x560ea7f7cb40 2024-09-19T07:05:06.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.160+0000 7efc4bfff640 1 --2- 172.21.15.103:0/1731375745 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efc380a3430 0x7efc380a58f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efc4404b000 tx=0x7efc44048000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:06.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.160+0000 7efc54348740 1 -- 172.21.15.103:0/3964317603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f7cb40 msgr2=0x560ea7f7b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:06.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.160+0000 7efc54348740 1 --2- 172.21.15.103:0/3964317603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f7cb40 0x560ea7f7b090 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7efc44031650 tx=0x7efc44031680 comp rx=0 tx=0).stop 2024-09-19T07:05:06.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.160+0000 7efc54348740 1 -- 172.21.15.103:0/3964317603 shutdown_connections 2024-09-19T07:05:06.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.160+0000 7efc54348740 1 --2- 172.21.15.103:0/3964317603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f8a780 0x560ea7f8cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.160+0000 7efc54348740 1 --2- 172.21.15.103:0/3964317603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7b5d0 0x560ea7f7ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.160+0000 7efc54348740 1 --2- 172.21.15.103:0/3964317603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f7cb40 0x560ea7f7b090 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.160+0000 7efc54348740 1 -- 172.21.15.103:0/3964317603 >> 172.21.15.103:0/3964317603 conn(0x560ea7f44310 msgr2=0x560ea7f77c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:06.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.160+0000 7efc54348740 1 -- 172.21.15.103:0/3964317603 shutdown_connections 2024-09-19T07:05:06.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.160+0000 7efc54348740 1 -- 172.21.15.103:0/3964317603 wait complete. 2024-09-19T07:05:06.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.161+0000 7efc54348740 1 -- start start 2024-09-19T07:05:06.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.161+0000 7efc54348740 1 Processor -- start 2024-09-19T07:05:06.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.161+0000 7efc54348740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f7b5d0 0x560ea7f8a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.161+0000 7efc54348740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7cb40 0x560ea7f873f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.161+0000 7efc54348740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f8a780 0x560ea7f879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.161+0000 7efc54348740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560ea7f8d480 con 0x560ea7f8a780 2024-09-19T07:05:06.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.161+0000 7efc54348740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560ea7e9c060 con 0x560ea7f7b5d0 2024-09-19T07:05:06.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.161+0000 7efc54348740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560ea7f8d600 con 0x560ea7f7cb40 2024-09-19T07:05:06.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.161+0000 7efc54348740 10 client.0 _ll_register_callbacks cb 0x560ea7bef8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:06.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.161+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f8a780 0x560ea7f879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.161+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f8a780 0x560ea7f879e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60252/0 (socket says 172.21.15.103:60252) 2024-09-19T07:05:06.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.161+0000 7efc4bfff640 1 -- 172.21.15.103:0/1451345765 learned_addr learned my addr 172.21.15.103:0/1451345765 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:06.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.161+0000 7efc43fff640 1 --2- 172.21.15.103:0/1451345765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7cb40 0x560ea7f873f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.161+0000 7efc4b7fe640 1 --2- 172.21.15.103:0/1451345765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f7b5d0 0x560ea7f8a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc4bfff640 1 -- 172.21.15.103:0/1451345765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7cb40 msgr2=0x560ea7f873f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:06.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc4bfff640 1 --2- 172.21.15.103:0/1451345765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7cb40 0x560ea7f873f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc4bfff640 1 -- 172.21.15.103:0/1451345765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f7b5d0 msgr2=0x560ea7f8a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:06.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc4bfff640 1 --2- 172.21.15.103:0/1451345765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f7b5d0 0x560ea7f8a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc4bfff640 1 -- 172.21.15.103:0/1451345765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ea7c523c0 con 0x560ea7f8a780 2024-09-19T07:05:06.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc4b7fe640 1 --2- 172.21.15.103:0/1451345765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f7b5d0 0x560ea7f8a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:06.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc4bfff640 1 --2- 172.21.15.103:0/1451345765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f8a780 0x560ea7f879e0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7efc44009570 tx=0x7efc440048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c260bb8fb2ab33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:06.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc2affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:06.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc2affd640 1 -- 172.21.15.103:0/1451345765 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44030d30 con 0x560ea7f8a780 2024-09-19T07:05:06.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc54348740 1 -- 172.21.15.103:0/1451345765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc44036040 con 0x560ea7f8a780 2024-09-19T07:05:06.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc2affd640 1 -- 172.21.15.103:0/1451345765 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7efc4404d050 con 0x560ea7f8a780 2024-09-19T07:05:06.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc2affd640 1 -- 172.21.15.103:0/1451345765 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efc44030ed0 con 0x560ea7f8a780 2024-09-19T07:05:06.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc54348740 10 client.21051 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:06.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc54348740 1 -- 172.21.15.103:0/1451345765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ea7f8d480 con 0x560ea7f8a780 2024-09-19T07:05:06.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc2affd640 1 -- 172.21.15.103:0/1451345765 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc44036040 con 0x560ea7f8a780 2024-09-19T07:05:06.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc2affd640 1 -- 172.21.15.103:0/1451345765 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7efc44030070 con 0x560ea7f8a780 2024-09-19T07:05:06.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc54348740 10 client.21051 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:06.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc54348740 10 client.21051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:06.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc54348740 20 client.21051 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:06.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc54348740 1 --2- 172.21.15.103:0/1451345765 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560ea7f9a060 0x560ea7f9c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc54348740 4 client.21051 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:06.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.162+0000 7efc54348740 1 -- 172.21.15.103:0/1451345765 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ea7eee860 con 0x560ea7f9a060 2024-09-19T07:05:06.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.163+0000 7efc4b7fe640 1 --2- 172.21.15.103:0/1451345765 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560ea7f9a060 0x560ea7f9c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.163+0000 7efc2affd640 10 client.21051 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:06.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.163+0000 7efc4b7fe640 1 --2- 172.21.15.103:0/1451345765 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560ea7f9a060 0x560ea7f9c450 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=480b2ec40c2b2127 server_cookie=8c13da97a69b1185 in_seq=0 out_seq=0 2024-09-19T07:05:06.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.164+0000 7efc2affd640 1 -- 172.21.15.103:0/1451345765 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ea7eee860 con 0x560ea7f9a060 2024-09-19T07:05:06.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.164+0000 7efc2affd640 10 client.21051 handle_command_reply: tid=0 2024-09-19T07:05:06.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.164+0000 7efc54348740 1 client.21051 shutdown 2024-09-19T07:05:06.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.164+0000 7efc54348740 1 -- 172.21.15.103:0/1451345765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f8a780 msgr2=0x560ea7f879e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:06.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.164+0000 7efc54348740 1 --2- 172.21.15.103:0/1451345765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f8a780 0x560ea7f879e0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7efc44009570 tx=0x7efc440048d0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.164+0000 7efc54348740 1 -- 172.21.15.103:0/1451345765 shutdown_connections 2024-09-19T07:05:06.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.164+0000 7efc54348740 1 --2- 172.21.15.103:0/1451345765 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560ea7f9a060 0x560ea7f9c450 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.164+0000 7efc54348740 1 --2- 172.21.15.103:0/1451345765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f8a780 0x560ea7f879e0 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.164+0000 7efc54348740 1 --2- 172.21.15.103:0/1451345765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7cb40 0x560ea7f873f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.164+0000 7efc2affd640 0 client.21051 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:06.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.164+0000 7efc54348740 1 --2- 172.21.15.103:0/1451345765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f7b5d0 0x560ea7f8a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.164+0000 7efc54348740 1 -- 172.21.15.103:0/1451345765 >> 172.21.15.103:0/1451345765 conn(0x560ea7f44310 msgr2=0x560ea7f7d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:06.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.165+0000 7efc54348740 1 -- 172.21.15.103:0/1451345765 shutdown_connections 2024-09-19T07:05:06.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.165+0000 7efc54348740 1 -- 172.21.15.103:0/1451345765 wait complete. 2024-09-19T07:05:06.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.165+0000 7efc54348740 20 client.21051 trim_cache size 0 max 16384 2024-09-19T07:05:06.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.177+0000 7efc54348740 1 Processor -- start 2024-09-19T07:05:06.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.177+0000 7efc54348740 1 -- start start 2024-09-19T07:05:06.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.177+0000 7efc54348740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f7b5d0 0x560ea7f96720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.177+0000 7efc54348740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7cb40 0x560ea7f89360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.177+0000 7efc54348740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f8a780 0x560ea7fa1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.177+0000 7efc54348740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c1ab740 con 0x560ea7f7b5d0 2024-09-19T07:05:06.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.177+0000 7efc54348740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c1ad280 con 0x560ea7f8a780 2024-09-19T07:05:06.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.177+0000 7efc54348740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c1abe10 con 0x560ea7f7cb40 2024-09-19T07:05:06.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.177+0000 7efc4b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f7b5d0 0x560ea7f96720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.178+0000 7efc4b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f7b5d0 0x560ea7f96720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60258/0 (socket says 172.21.15.103:60258) 2024-09-19T07:05:06.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.178+0000 7efc4b7fe640 1 -- 172.21.15.103:0/3915225106 learned_addr learned my addr 172.21.15.103:0/3915225106 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:06.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.178+0000 7efc4bfff640 1 --2- 172.21.15.103:0/3915225106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f8a780 0x560ea7fa1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.178+0000 7efc4b7fe640 1 -- 172.21.15.103:0/3915225106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7cb40 msgr2=0x560ea7f89360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:06.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.178+0000 7efc4b7fe640 1 --2- 172.21.15.103:0/3915225106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7cb40 0x560ea7f89360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.178+0000 7efc4b7fe640 1 -- 172.21.15.103:0/3915225106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f8a780 msgr2=0x560ea7fa1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:06.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.178+0000 7efc4b7fe640 1 --2- 172.21.15.103:0/3915225106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f8a780 0x560ea7fa1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.178+0000 7efc4b7fe640 1 -- 172.21.15.103:0/3915225106 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ea7f86350 con 0x560ea7f7b5d0 2024-09-19T07:05:06.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.178+0000 7efc4bfff640 1 --2- 172.21.15.103:0/3915225106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f8a780 0x560ea7fa1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:06.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.178+0000 7efc4b7fe640 1 --2- 172.21.15.103:0/3915225106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f7b5d0 0x560ea7f96720 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7efc3c000c00 tx=0x7efc3c0d1930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efb7494215811dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:06.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.178+0000 7efc417fa640 1 -- 172.21.15.103:0/3915225106 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c09bfd0 con 0x560ea7f7b5d0 2024-09-19T07:05:06.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.178+0000 7efc417fa640 1 -- 172.21.15.103:0/3915225106 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc3c09b8d0 con 0x560ea7f7b5d0 2024-09-19T07:05:06.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.178+0000 7efc417fa640 1 -- 172.21.15.103:0/3915225106 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c097f40 con 0x560ea7f7b5d0 2024-09-19T07:05:06.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.179+0000 7efc54348740 1 -- 172.21.15.103:0/3915225106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f7b5d0 msgr2=0x560ea7f96720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:06.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.179+0000 7efc54348740 1 --2- 172.21.15.103:0/3915225106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f7b5d0 0x560ea7f96720 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7efc3c000c00 tx=0x7efc3c0d1930 comp rx=0 tx=0).stop 2024-09-19T07:05:06.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.179+0000 7efc54348740 1 -- 172.21.15.103:0/3915225106 shutdown_connections 2024-09-19T07:05:06.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.179+0000 7efc54348740 1 --2- 172.21.15.103:0/3915225106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f8a780 0x560ea7fa1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.179+0000 7efc54348740 1 --2- 172.21.15.103:0/3915225106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7cb40 0x560ea7f89360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.179+0000 7efc54348740 1 --2- 172.21.15.103:0/3915225106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f7b5d0 0x560ea7f96720 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.179+0000 7efc54348740 1 -- 172.21.15.103:0/3915225106 >> 172.21.15.103:0/3915225106 conn(0x560ea7f44310 msgr2=0x560ea7f88c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:06.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.179+0000 7efc54348740 1 -- 172.21.15.103:0/3915225106 shutdown_connections 2024-09-19T07:05:06.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.179+0000 7efc54348740 1 -- 172.21.15.103:0/3915225106 wait complete. 2024-09-19T07:05:06.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.179+0000 7efc54348740 1 -- start start 2024-09-19T07:05:06.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.179+0000 7efc54348740 1 Processor -- start 2024-09-19T07:05:06.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc54348740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7b5d0 0x560ea7f92e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc54348740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f7cb40 0x560ea7f84000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc54348740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f8a780 0x560ea7f84540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc54348740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c1acbb0 con 0x560ea7f8a780 2024-09-19T07:05:06.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc54348740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c1ac4e0 con 0x560ea7f7cb40 2024-09-19T07:05:06.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc54348740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c1a9740 con 0x560ea7f7b5d0 2024-09-19T07:05:06.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc54348740 10 client.0 _ll_register_callbacks cb 0x7efc4404a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:06.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f8a780 0x560ea7f84540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f8a780 0x560ea7f84540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60270/0 (socket says 172.21.15.103:60270) 2024-09-19T07:05:06.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc4bfff640 1 -- 172.21.15.103:0/3426370223 learned_addr learned my addr 172.21.15.103:0/3426370223 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:06.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc4b7fe640 1 --2- 172.21.15.103:0/3426370223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7b5d0 0x560ea7f92e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc43fff640 1 --2- 172.21.15.103:0/3426370223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f7cb40 0x560ea7f84000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc4bfff640 1 -- 172.21.15.103:0/3426370223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7b5d0 msgr2=0x560ea7f92e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:06.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc4bfff640 1 --2- 172.21.15.103:0/3426370223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7b5d0 0x560ea7f92e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc4bfff640 1 -- 172.21.15.103:0/3426370223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f7cb40 msgr2=0x560ea7f84000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:06.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc4bfff640 1 --2- 172.21.15.103:0/3426370223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f7cb40 0x560ea7f84000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc4bfff640 1 -- 172.21.15.103:0/3426370223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ea7f84bb0 con 0x560ea7f8a780 2024-09-19T07:05:06.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc4b7fe640 1 --2- 172.21.15.103:0/3426370223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7b5d0 0x560ea7f92e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:06.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc4bfff640 1 --2- 172.21.15.103:0/3426370223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f8a780 0x560ea7f84540 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7efc44009430 tx=0x7efc44062650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8f1c0b70ba5b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:06.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc2affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:06.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc2affd640 1 -- 172.21.15.103:0/3426370223 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44030d30 con 0x560ea7f8a780 2024-09-19T07:05:06.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc2affd640 1 -- 172.21.15.103:0/3426370223 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7efc44056040 con 0x560ea7f8a780 2024-09-19T07:05:06.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.180+0000 7efc54348740 1 -- 172.21.15.103:0/3426370223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc3c09b8d0 con 0x560ea7f8a780 2024-09-19T07:05:06.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.181+0000 7efc2affd640 1 -- 172.21.15.103:0/3426370223 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efc44004250 con 0x560ea7f8a780 2024-09-19T07:05:06.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.181+0000 7efc54348740 10 client.21057 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:06.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.181+0000 7efc54348740 1 -- 172.21.15.103:0/3426370223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ea80049d0 con 0x560ea7f8a780 2024-09-19T07:05:06.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.181+0000 7efc2affd640 1 -- 172.21.15.103:0/3426370223 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc3c09b8d0 con 0x560ea7f8a780 2024-09-19T07:05:06.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.181+0000 7efc2affd640 1 -- 172.21.15.103:0/3426370223 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7efc44069020 con 0x560ea7f8a780 2024-09-19T07:05:06.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.181+0000 7efc54348740 10 client.21057 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:06.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.181+0000 7efc54348740 10 client.21057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:06.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.181+0000 7efc54348740 20 client.21057 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:06.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.181+0000 7efc54348740 1 --2- 172.21.15.103:0/3426370223 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560ea80050e0 0x560ea8007500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:06.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.181+0000 7efc54348740 4 client.21057 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:05:06.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.181+0000 7efc54348740 1 -- 172.21.15.103:0/3426370223 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ea7ee8d70 con 0x560ea80050e0 2024-09-19T07:05:06.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.181+0000 7efc4b7fe640 1 --2- 172.21.15.103:0/3426370223 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560ea80050e0 0x560ea8007500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:06.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc4b7fe640 1 --2- 172.21.15.103:0/3426370223 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560ea80050e0 0x560ea8007500 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c365fb0daaf5d352 server_cookie=a25956dda549283f in_seq=0 out_seq=0 2024-09-19T07:05:06.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc2affd640 10 client.21057 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:06.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc2affd640 1 -- 172.21.15.103:0/3426370223 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ea7ee8d70 con 0x560ea80050e0 2024-09-19T07:05:06.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc2affd640 10 client.21057 handle_command_reply: tid=0 2024-09-19T07:05:06.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc54348740 1 client.21057 shutdown 2024-09-19T07:05:06.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc54348740 1 -- 172.21.15.103:0/3426370223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f8a780 msgr2=0x560ea7f84540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:06.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc54348740 1 --2- 172.21.15.103:0/3426370223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f8a780 0x560ea7f84540 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7efc44009430 tx=0x7efc44062650 comp rx=0 tx=0).stop 2024-09-19T07:05:06.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc54348740 1 -- 172.21.15.103:0/3426370223 shutdown_connections 2024-09-19T07:05:06.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc54348740 1 --2- 172.21.15.103:0/3426370223 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560ea80050e0 0x560ea8007500 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc2affd640 0 client.21057 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:06.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc54348740 1 --2- 172.21.15.103:0/3426370223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ea7f8a780 0x560ea7f84540 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc54348740 1 --2- 172.21.15.103:0/3426370223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ea7f7cb40 0x560ea7f84000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc54348740 1 --2- 172.21.15.103:0/3426370223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ea7f7b5d0 0x560ea7f92e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc54348740 1 -- 172.21.15.103:0/3426370223 >> 172.21.15.103:0/3426370223 conn(0x560ea7f44310 msgr2=0x560ea7f89470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:06.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc54348740 1 -- 172.21.15.103:0/3426370223 shutdown_connections 2024-09-19T07:05:06.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc54348740 1 -- 172.21.15.103:0/3426370223 wait complete. 2024-09-19T07:05:06.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.182+0000 7efc54348740 20 client.21057 trim_cache size 0 max 16384 2024-09-19T07:05:06.199 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:05:06.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.183+0000 7efc51f6f640 1 -- 172.21.15.103:0/1731375745 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efc380a3430 msgr2=0x7efc380a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:06.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.183+0000 7efc51f6f640 1 --2- 172.21.15.103:0/1731375745 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efc380a3430 0x7efc380a58f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efc4404b000 tx=0x7efc44048000 comp rx=0 tx=0).stop 2024-09-19T07:05:06.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.183+0000 7efc51f6f640 1 -- 172.21.15.103:0/1731375745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc4c102d30 msgr2=0x7efc4c198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:06.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.183+0000 7efc51f6f640 1 --2- 172.21.15.103:0/1731375745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc4c102d30 0x7efc4c198550 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00b4a0 tx=0x7efc3c00b8a0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.183+0000 7efc51f6f640 1 -- 172.21.15.103:0/1731375745 shutdown_connections 2024-09-19T07:05:06.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.183+0000 7efc51f6f640 1 --2- 172.21.15.103:0/1731375745 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7efc380a3430 0x7efc380a58f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.183+0000 7efc51f6f640 1 --2- 172.21.15.103:0/1731375745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc4c1082a0 0x7efc4c19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.183+0000 7efc51f6f640 1 --2- 172.21.15.103:0/1731375745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc4c1036e0 0x7efc4c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.183+0000 7efc51f6f640 1 --2- 172.21.15.103:0/1731375745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc4c102d30 0x7efc4c198550 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:06.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.183+0000 7efc51f6f640 1 -- 172.21.15.103:0/1731375745 >> 172.21.15.103:0/1731375745 conn(0x7efc4c0fc7d0 msgr2=0x7efc4c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:06.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.183+0000 7efc51f6f640 1 -- 172.21.15.103:0/1731375745 shutdown_connections 2024-09-19T07:05:06.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:06.183+0000 7efc51f6f640 1 -- 172.21.15.103:0/1731375745 wait complete. 2024-09-19T07:05:07.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:07 smithi174 ceph-mon[35637]: pgmap v497: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 48 KiB/s wr, 33 op/s 2024-09-19T07:05:07.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:07 smithi103 ceph-mon[31158]: pgmap v497: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 48 KiB/s wr, 33 op/s 2024-09-19T07:05:07.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:07 smithi134 ceph-mon[35356]: pgmap v497: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 48 KiB/s wr, 33 op/s 2024-09-19T07:05:07.980 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:05:07.980 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:05:08.231 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:08.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.672+0000 7f7ec2be2640 1 -- 172.21.15.103:0/2534288051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ebc100a80 msgr2=0x7f7ebc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:08.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.672+0000 7f7ec2be2640 1 --2- 172.21.15.103:0/2534288051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ebc100a80 0x7f7ebc100ee0 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009930 tx=0x7f7ea802a420 comp rx=0 tx=0).stop 2024-09-19T07:05:08.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.672+0000 7f7ec2be2640 1 -- 172.21.15.103:0/2534288051 shutdown_connections 2024-09-19T07:05:08.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.672+0000 7f7ec2be2640 1 --2- 172.21.15.103:0/2534288051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ebc101420 0x7f7ebc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.672+0000 7f7ec2be2640 1 --2- 172.21.15.103:0/2534288051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ebc100a80 0x7f7ebc100ee0 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.672+0000 7f7ec2be2640 1 --2- 172.21.15.103:0/2534288051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ebc106a80 0x7f7ebc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.672+0000 7f7ec2be2640 1 -- 172.21.15.103:0/2534288051 >> 172.21.15.103:0/2534288051 conn(0x7f7ebc0fc7a0 msgr2=0x7f7ebc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:08.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.672+0000 7f7ec2be2640 1 -- 172.21.15.103:0/2534288051 shutdown_connections 2024-09-19T07:05:08.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.672+0000 7f7ec2be2640 1 -- 172.21.15.103:0/2534288051 wait complete. 2024-09-19T07:05:08.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.673+0000 7f7ec2be2640 1 Processor -- start 2024-09-19T07:05:08.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.673+0000 7f7ec2be2640 1 -- start start 2024-09-19T07:05:08.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.673+0000 7f7ec2be2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ebc100a80 0x7f7ebc199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.673+0000 7f7ec2be2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ebc101420 0x7f7ebc199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.673+0000 7f7ec2be2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ebc106a80 0x7f7ebc1a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.673+0000 7f7ec2be2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc110e80 con 0x7f7ebc106a80 2024-09-19T07:05:08.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.673+0000 7f7ec2be2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc110d00 con 0x7f7ebc100a80 2024-09-19T07:05:08.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.673+0000 7f7ec2be2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc111000 con 0x7f7ebc101420 2024-09-19T07:05:08.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.673+0000 7f7ec117f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ebc106a80 0x7f7ebc1a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.674+0000 7f7ec117f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ebc106a80 0x7f7ebc1a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42400/0 (socket says 172.21.15.103:42400) 2024-09-19T07:05:08.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.674+0000 7f7ec097e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ebc100a80 0x7f7ebc199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.674+0000 7f7ebbfff640 1 --2- 172.21.15.103:0/1278008737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ebc101420 0x7f7ebc199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.674+0000 7f7ec117f640 1 -- 172.21.15.103:0/1278008737 learned_addr learned my addr 172.21.15.103:0/1278008737 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:08.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.674+0000 7f7ec117f640 1 -- 172.21.15.103:0/1278008737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ebc101420 msgr2=0x7f7ebc199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:08.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.674+0000 7f7ec117f640 1 --2- 172.21.15.103:0/1278008737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ebc101420 0x7f7ebc199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.674+0000 7f7ec117f640 1 -- 172.21.15.103:0/1278008737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ebc100a80 msgr2=0x7f7ebc199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:08.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.674+0000 7f7ec117f640 1 --2- 172.21.15.103:0/1278008737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ebc100a80 0x7f7ebc199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.674+0000 7f7ec117f640 1 -- 172.21.15.103:0/1278008737 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc1a48d0 con 0x7f7ebc106a80 2024-09-19T07:05:08.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.674+0000 7f7ec117f640 1 --2- 172.21.15.103:0/1278008737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ebc106a80 0x7f7ebc1a4390 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400eaa0 tx=0x7f7eb400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:08.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.674+0000 7f7eb9ffb640 1 -- 172.21.15.103:0/1278008737 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb400cdb0 con 0x7f7ebc106a80 2024-09-19T07:05:08.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.674+0000 7f7eb9ffb640 1 -- 172.21.15.103:0/1278008737 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb4005020 con 0x7f7ebc106a80 2024-09-19T07:05:08.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.674+0000 7f7ec2be2640 1 -- 172.21.15.103:0/1278008737 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ebc1a4aa0 con 0x7f7ebc106a80 2024-09-19T07:05:08.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.674+0000 7f7eb9ffb640 1 -- 172.21.15.103:0/1278008737 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb40116b0 con 0x7f7ebc106a80 2024-09-19T07:05:08.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.675+0000 7f7ec2be2640 1 -- 172.21.15.103:0/1278008737 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ebc1a4f60 con 0x7f7ebc106a80 2024-09-19T07:05:08.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.677+0000 7f7eb9ffb640 1 -- 172.21.15.103:0/1278008737 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7eb4005560 con 0x7f7ebc106a80 2024-09-19T07:05:08.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.677+0000 7f7ec4fbb740 1 Processor -- start 2024-09-19T07:05:08.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.677+0000 7f7ec4fbb740 1 -- start start 2024-09-19T07:05:08.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.677+0000 7f7eb9ffb640 1 --2- 172.21.15.103:0/1278008737 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7eac0a33b0 0x7f7eac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.677+0000 7f7ec4fbb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf7314b60 0x557bf7314f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.677+0000 7f7ec4fbb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7315480 0x557bf731f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.677+0000 7f7ec4fbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7322130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.677+0000 7f7ec4fbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557bf727a280 con 0x557bf731fd40 2024-09-19T07:05:08.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.677+0000 7f7ec4fbb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557bf7221b10 con 0x557bf7315480 2024-09-19T07:05:08.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.677+0000 7f7ec4fbb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557bf7276240 con 0x557bf7314b60 2024-09-19T07:05:08.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.677+0000 7f7ebbfff640 1 --2- 172.21.15.103:0/1278008737 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7eac0a33b0 0x7f7eac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.677+0000 7f7ec117f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7322130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.678+0000 7f7ec117f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7322130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42410/0 (socket says 172.21.15.103:42410) 2024-09-19T07:05:08.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.678+0000 7f7ec117f640 1 -- 172.21.15.103:0/2660496512 learned_addr learned my addr 172.21.15.103:0/2660496512 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:08.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.678+0000 7f7eb9ffb640 1 -- 172.21.15.103:0/1278008737 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f7eb40cff40 con 0x7f7ebc106a80 2024-09-19T07:05:08.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.678+0000 7f7ec097e640 1 --2- 172.21.15.103:0/2660496512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf7314b60 0x557bf7314f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.678+0000 7f7ebbfff640 1 --2- 172.21.15.103:0/2660496512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7315480 0x557bf731f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.678+0000 7f7ec117f640 1 -- 172.21.15.103:0/2660496512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf7314b60 msgr2=0x557bf7314f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:08.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.678+0000 7f7ec117f640 1 --2- 172.21.15.103:0/2660496512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf7314b60 0x557bf7314f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.678+0000 7f7ec117f640 1 -- 172.21.15.103:0/2660496512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7315480 msgr2=0x557bf731f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:08.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.678+0000 7f7ec117f640 1 --2- 172.21.15.103:0/2660496512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7315480 0x557bf731f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.678+0000 7f7ec117f640 1 -- 172.21.15.103:0/2660496512 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bf722e1f0 con 0x557bf731fd40 2024-09-19T07:05:08.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.678+0000 7f7ec097e640 1 --2- 172.21.15.103:0/2660496512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf7314b60 0x557bf7314f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:05:08.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.678+0000 7f7ebbfff640 1 --2- 172.21.15.103:0/1278008737 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7eac0a33b0 0x7f7eac0a5870 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009900 tx=0x7f7ea8031800 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:08.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.678+0000 7f7ec117f640 1 --2- 172.21.15.103:0/2660496512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7322130 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f7eb409c1e0 tx=0x7f7eb4015840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5768d6cfea76604 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:08.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.678+0000 7f7ea6ffd640 1 -- 172.21.15.103:0/2660496512 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4079220 con 0x557bf731fd40 2024-09-19T07:05:08.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.678+0000 7f7ea6ffd640 1 -- 172.21.15.103:0/2660496512 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb409c250 con 0x557bf731fd40 2024-09-19T07:05:08.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.678+0000 7f7ea6ffd640 1 -- 172.21.15.103:0/2660496512 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4092c70 con 0x557bf731fd40 2024-09-19T07:05:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.679+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/2660496512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 msgr2=0x557bf7322130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.679+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/2660496512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7322130 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f7eb409c1e0 tx=0x7f7eb4015840 comp rx=0 tx=0).stop 2024-09-19T07:05:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.679+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/2660496512 shutdown_connections 2024-09-19T07:05:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.679+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/2660496512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7322130 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.679+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/2660496512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7315480 0x557bf731f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.679+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/2660496512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf7314b60 0x557bf7314f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.679+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/2660496512 >> 172.21.15.103:0/2660496512 conn(0x557bf73110f0 msgr2=0x557bf73114d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:08.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.679+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/2660496512 shutdown_connections 2024-09-19T07:05:08.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.679+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/2660496512 wait complete. 2024-09-19T07:05:08.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.679+0000 7f7ec4fbb740 1 -- start start 2024-09-19T07:05:08.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.679+0000 7f7ec4fbb740 1 Processor -- start 2024-09-19T07:05:08.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ec4fbb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7314b60 0x557bf731cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ec4fbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf7315480 0x557bf731d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ec4fbb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf731fd40 0x557bf731d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ec4fbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557bf73228b0 con 0x557bf7315480 2024-09-19T07:05:08.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ec4fbb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557bf7234f10 con 0x557bf7314b60 2024-09-19T07:05:08.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ec4fbb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557bf7322a30 con 0x557bf731fd40 2024-09-19T07:05:08.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ec4fbb740 10 client.0 _ll_register_callbacks cb 0x557bf71fda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:08.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ebbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf7315480 0x557bf731d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ebbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf7315480 0x557bf731d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42422/0 (socket says 172.21.15.103:42422) 2024-09-19T07:05:08.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ebbfff640 1 -- 172.21.15.103:0/825086407 learned_addr learned my addr 172.21.15.103:0/825086407 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:08.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ec097e640 1 --2- 172.21.15.103:0/825086407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7314b60 0x557bf731cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ec117f640 1 --2- 172.21.15.103:0/825086407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf731fd40 0x557bf731d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ebbfff640 1 -- 172.21.15.103:0/825086407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf731fd40 msgr2=0x557bf731d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:08.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ebbfff640 1 --2- 172.21.15.103:0/825086407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf731fd40 0x557bf731d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ebbfff640 1 -- 172.21.15.103:0/825086407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7314b60 msgr2=0x557bf731cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:08.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ebbfff640 1 --2- 172.21.15.103:0/825086407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7314b60 0x557bf731cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.680+0000 7f7ebbfff640 1 -- 172.21.15.103:0/825086407 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bf6feb6a0 con 0x557bf7315480 2024-09-19T07:05:08.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.681+0000 7f7ebbfff640 1 --2- 172.21.15.103:0/825086407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf7315480 0x557bf731d160 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009930 tx=0x7f7ea802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38979592ed5c32d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:08.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.681+0000 7f7e9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:08.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.681+0000 7f7e9bfff640 1 -- 172.21.15.103:0/825086407 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea804e070 con 0x557bf7315480 2024-09-19T07:05:08.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.681+0000 7f7e9bfff640 1 -- 172.21.15.103:0/825086407 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f7ea8003b80 con 0x557bf7315480 2024-09-19T07:05:08.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.681+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/825086407 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eb409c250 con 0x557bf7315480 2024-09-19T07:05:08.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.681+0000 7f7e9bfff640 1 -- 172.21.15.103:0/825086407 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ea805a050 con 0x557bf7315480 2024-09-19T07:05:08.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.681+0000 7f7ec4fbb740 10 client.21069 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:08.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.681+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/825086407 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557bf732e4b0 con 0x557bf7315480 2024-09-19T07:05:08.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.681+0000 7f7e9bfff640 1 -- 172.21.15.103:0/825086407 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb409c250 con 0x557bf7315480 2024-09-19T07:05:08.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.681+0000 7f7e9bfff640 1 -- 172.21.15.103:0/825086407 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7ea803d3f0 con 0x557bf7315480 2024-09-19T07:05:08.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.681+0000 7f7ec4fbb740 10 client.21069 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:08.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.681+0000 7f7ec4fbb740 10 client.21069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:08.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.681+0000 7f7ec4fbb740 20 client.21069 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:08.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.681+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/825086407 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557bf732f8c0 0x557bf7331cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.681+0000 7f7ec4fbb740 4 client.21069 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:08.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.681+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/825086407 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bf72877f0 con 0x557bf732f8c0 2024-09-19T07:05:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.682+0000 7f7ec097e640 1 --2- 172.21.15.103:0/825086407 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557bf732f8c0 0x557bf7331cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.682+0000 7f7e9bfff640 10 client.21069 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.682+0000 7f7ec097e640 1 --2- 172.21.15.103:0/825086407 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557bf732f8c0 0x557bf7331cf0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a4bf40037d188be server_cookie=89f89f409b5e6336 in_seq=0 out_seq=0 2024-09-19T07:05:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.683+0000 7f7e9bfff640 1 -- 172.21.15.103:0/825086407 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557bf72877f0 con 0x557bf732f8c0 2024-09-19T07:05:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.683+0000 7f7e9bfff640 10 client.21069 handle_command_reply: tid=0 2024-09-19T07:05:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.683+0000 7f7ec4fbb740 1 client.21069 shutdown 2024-09-19T07:05:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.683+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/825086407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf7315480 msgr2=0x557bf731d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.683+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/825086407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf7315480 0x557bf731d160 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009930 tx=0x7f7ea802a420 comp rx=0 tx=0).stop 2024-09-19T07:05:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.683+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/825086407 shutdown_connections 2024-09-19T07:05:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.683+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/825086407 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557bf732f8c0 0x557bf7331cf0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.683+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/825086407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf731fd40 0x557bf731d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.683+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/825086407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf7315480 0x557bf731d160 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.683+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/825086407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7314b60 0x557bf731cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.683+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/825086407 >> 172.21.15.103:0/825086407 conn(0x557bf73110f0 msgr2=0x557bf7316e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.683+0000 7f7e9bfff640 0 client.21069 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.683+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/825086407 shutdown_connections 2024-09-19T07:05:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.683+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/825086407 wait complete. 2024-09-19T07:05:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.683+0000 7f7ec4fbb740 20 client.21069 trim_cache size 0 max 16384 2024-09-19T07:05:08.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.700+0000 7f7ec4fbb740 1 Processor -- start 2024-09-19T07:05:08.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.700+0000 7f7ec4fbb740 1 -- start start 2024-09-19T07:05:08.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.700+0000 7f7ec4fbb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7314b60 0x557bf731ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.700+0000 7f7ec4fbb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf7315480 0x557bf73375c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.700+0000 7f7ec4fbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7337b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.700+0000 7f7ec4fbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1ad9d0 con 0x557bf731fd40 2024-09-19T07:05:08.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.700+0000 7f7ec4fbb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1af510 con 0x557bf7314b60 2024-09-19T07:05:08.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.700+0000 7f7ec4fbb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1ae0a0 con 0x557bf7315480 2024-09-19T07:05:08.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.701+0000 7f7ec117f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7337b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.701+0000 7f7ec117f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7337b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42426/0 (socket says 172.21.15.103:42426) 2024-09-19T07:05:08.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.701+0000 7f7ec117f640 1 -- 172.21.15.103:0/1812313211 learned_addr learned my addr 172.21.15.103:0/1812313211 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:08.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.701+0000 7f7ebbfff640 1 --2- 172.21.15.103:0/1812313211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf7315480 0x557bf73375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.701+0000 7f7ec097e640 1 --2- 172.21.15.103:0/1812313211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7314b60 0x557bf731ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.701+0000 7f7ec117f640 1 -- 172.21.15.103:0/1812313211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf7315480 msgr2=0x557bf73375c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:08.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.701+0000 7f7ec117f640 1 --2- 172.21.15.103:0/1812313211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf7315480 0x557bf73375c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.701+0000 7f7ec117f640 1 -- 172.21.15.103:0/1812313211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7314b60 msgr2=0x557bf731ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:08.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.701+0000 7f7ec117f640 1 --2- 172.21.15.103:0/1812313211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7314b60 0x557bf731ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.701+0000 7f7ec117f640 1 -- 172.21.15.103:0/1812313211 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bf730c830 con 0x557bf731fd40 2024-09-19T07:05:08.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.701+0000 7f7ec097e640 1 --2- 172.21.15.103:0/1812313211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7314b60 0x557bf731ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:08.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.701+0000 7f7ec117f640 1 --2- 172.21.15.103:0/1812313211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7337b00 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400efd0 tx=0x7f7eb4079510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6dc0fbf77d9ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:08.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.701+0000 7f7ea67fc640 1 -- 172.21.15.103:0/1812313211 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb40b5070 con 0x557bf731fd40 2024-09-19T07:05:08.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.701+0000 7f7ea67fc640 1 -- 172.21.15.103:0/1812313211 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb409a550 con 0x557bf731fd40 2024-09-19T07:05:08.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.701+0000 7f7ea67fc640 1 -- 172.21.15.103:0/1812313211 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb401f040 con 0x557bf731fd40 2024-09-19T07:05:08.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.702+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/1812313211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 msgr2=0x557bf7337b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:08.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.702+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/1812313211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7337b00 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400efd0 tx=0x7f7eb4079510 comp rx=0 tx=0).stop 2024-09-19T07:05:08.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.702+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/1812313211 shutdown_connections 2024-09-19T07:05:08.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.702+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/1812313211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7337b00 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.702+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/1812313211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf7315480 0x557bf73375c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.702+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/1812313211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7314b60 0x557bf731ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.702+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/1812313211 >> 172.21.15.103:0/1812313211 conn(0x557bf73110f0 msgr2=0x557bf731e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:08.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.702+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/1812313211 shutdown_connections 2024-09-19T07:05:08.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.702+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/1812313211 wait complete. 2024-09-19T07:05:08.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.702+0000 7f7ec4fbb740 1 -- start start 2024-09-19T07:05:08.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.702+0000 7f7ec4fbb740 1 Processor -- start 2024-09-19T07:05:08.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec4fbb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7314b60 0x557bf7327c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec4fbb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf7315480 0x557bf73281c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec4fbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7324c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec4fbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1aee40 con 0x557bf731fd40 2024-09-19T07:05:08.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec4fbb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1ae770 con 0x557bf7314b60 2024-09-19T07:05:08.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec4fbb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1ab9d0 con 0x557bf7315480 2024-09-19T07:05:08.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec4fbb740 10 client.0 _ll_register_callbacks cb 0x7f7ea8005df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:08.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec117f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7324c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec117f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7324c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42434/0 (socket says 172.21.15.103:42434) 2024-09-19T07:05:08.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec117f640 1 -- 172.21.15.103:0/2773059399 learned_addr learned my addr 172.21.15.103:0/2773059399 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:08.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec097e640 1 --2- 172.21.15.103:0/2773059399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7314b60 0x557bf7327c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ebbfff640 1 --2- 172.21.15.103:0/2773059399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf7315480 0x557bf73281c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec117f640 1 -- 172.21.15.103:0/2773059399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf7315480 msgr2=0x557bf73281c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:08.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec117f640 1 --2- 172.21.15.103:0/2773059399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf7315480 0x557bf73281c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec117f640 1 -- 172.21.15.103:0/2773059399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7314b60 msgr2=0x557bf7327c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:08.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec117f640 1 --2- 172.21.15.103:0/2773059399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7314b60 0x557bf7327c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec117f640 1 -- 172.21.15.103:0/2773059399 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bf73152b0 con 0x557bf731fd40 2024-09-19T07:05:08.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7ec117f640 1 --2- 172.21.15.103:0/2773059399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7324c50 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f7eb409bf50 tx=0x7f7eb40a4f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516a3aee64de6e8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:08.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.703+0000 7f7e9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:08.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.704+0000 7f7e9bfff640 1 -- 172.21.15.103:0/2773059399 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb401f030 con 0x557bf731fd40 2024-09-19T07:05:08.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.704+0000 7f7e9bfff640 1 -- 172.21.15.103:0/2773059399 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f7eb40bc0c0 con 0x557bf731fd40 2024-09-19T07:05:08.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.704+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/2773059399 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eb409a550 con 0x557bf731fd40 2024-09-19T07:05:08.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.704+0000 7f7e9bfff640 1 -- 172.21.15.103:0/2773059399 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7eb4081050 con 0x557bf731fd40 2024-09-19T07:05:08.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.704+0000 7f7ec4fbb740 10 client.21075 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:08.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.704+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/2773059399 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557bf739efa0 con 0x557bf731fd40 2024-09-19T07:05:08.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.704+0000 7f7e9bfff640 1 -- 172.21.15.103:0/2773059399 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb409a550 con 0x557bf731fd40 2024-09-19T07:05:08.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.704+0000 7f7e9bfff640 1 -- 172.21.15.103:0/2773059399 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7eb400f040 con 0x557bf731fd40 2024-09-19T07:05:08.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.704+0000 7f7ec4fbb740 10 client.21075 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:08.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.704+0000 7f7ec4fbb740 10 client.21075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:08.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.704+0000 7f7ec4fbb740 20 client.21075 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:08.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.704+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/2773059399 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557bf739f1d0 0x557bf73a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:08.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.704+0000 7f7ec4fbb740 4 client.21075 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:05:08.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.704+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/2773059399 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557bf7281d00 con 0x557bf739f1d0 2024-09-19T07:05:08.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.704+0000 7f7ec097e640 1 --2- 172.21.15.103:0/2773059399 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557bf739f1d0 0x557bf73a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:08.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.705+0000 7f7e9bfff640 10 client.21075 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:08.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.705+0000 7f7ec097e640 1 --2- 172.21.15.103:0/2773059399 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557bf739f1d0 0x557bf73a15c0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db71f58fbf9d1d36 server_cookie=c09d23d516c5b0a2 in_seq=0 out_seq=0 2024-09-19T07:05:08.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.706+0000 7f7e9bfff640 1 -- 172.21.15.103:0/2773059399 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557bf7281d00 con 0x557bf739f1d0 2024-09-19T07:05:08.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.706+0000 7f7e9bfff640 10 client.21075 handle_command_reply: tid=0 2024-09-19T07:05:08.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.706+0000 7f7ec4fbb740 1 client.21075 shutdown 2024-09-19T07:05:08.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.706+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/2773059399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 msgr2=0x557bf7324c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:08.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.706+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/2773059399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7324c50 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f7eb409bf50 tx=0x7f7eb40a4f50 comp rx=0 tx=0).stop 2024-09-19T07:05:08.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.706+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/2773059399 shutdown_connections 2024-09-19T07:05:08.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.706+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/2773059399 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557bf739f1d0 0x557bf73a15c0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.706+0000 7f7e9bfff640 0 client.21075 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:08.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.706+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/2773059399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557bf731fd40 0x557bf7324c50 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.706+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/2773059399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bf7315480 0x557bf73281c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.706+0000 7f7ec4fbb740 1 --2- 172.21.15.103:0/2773059399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557bf7314b60 0x557bf7327c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.706+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/2773059399 >> 172.21.15.103:0/2773059399 conn(0x557bf73110f0 msgr2=0x557bf731d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:08.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.706+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/2773059399 shutdown_connections 2024-09-19T07:05:08.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.706+0000 7f7ec4fbb740 1 -- 172.21.15.103:0/2773059399 wait complete. 2024-09-19T07:05:08.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.706+0000 7f7ec4fbb740 20 client.21075 trim_cache size 0 max 16384 2024-09-19T07:05:08.713 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:05:08.713 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:05:08.713 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "a1252bf4-6202-4048-a6bc-18b2dc67de64", 2024-09-19T07:05:08.713 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:05:08.713 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:05:08.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.707+0000 7f7ec2be2640 1 -- 172.21.15.103:0/1278008737 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7eac0a33b0 msgr2=0x7f7eac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:08.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.707+0000 7f7ec2be2640 1 --2- 172.21.15.103:0/1278008737 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7eac0a33b0 0x7f7eac0a5870 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009900 tx=0x7f7ea8031800 comp rx=0 tx=0).stop 2024-09-19T07:05:08.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.707+0000 7f7ec2be2640 1 -- 172.21.15.103:0/1278008737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ebc106a80 msgr2=0x7f7ebc1a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:08.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.707+0000 7f7ec2be2640 1 --2- 172.21.15.103:0/1278008737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ebc106a80 0x7f7ebc1a4390 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400eaa0 tx=0x7f7eb400ef60 comp rx=0 tx=0).stop 2024-09-19T07:05:08.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.707+0000 7f7ec2be2640 1 -- 172.21.15.103:0/1278008737 shutdown_connections 2024-09-19T07:05:08.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.707+0000 7f7ec2be2640 1 --2- 172.21.15.103:0/1278008737 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7eac0a33b0 0x7f7eac0a5870 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.707+0000 7f7ec2be2640 1 --2- 172.21.15.103:0/1278008737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ebc106a80 0x7f7ebc1a4390 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.707+0000 7f7ec2be2640 1 --2- 172.21.15.103:0/1278008737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ebc101420 0x7f7ebc199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.707+0000 7f7ec2be2640 1 --2- 172.21.15.103:0/1278008737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ebc100a80 0x7f7ebc199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:08.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.707+0000 7f7ec2be2640 1 -- 172.21.15.103:0/1278008737 >> 172.21.15.103:0/1278008737 conn(0x7f7ebc0fc7a0 msgr2=0x7f7ebc0fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:08.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.707+0000 7f7ec2be2640 1 -- 172.21.15.103:0/1278008737 shutdown_connections 2024-09-19T07:05:08.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:08.707+0000 7f7ec2be2640 1 -- 172.21.15.103:0/1278008737 wait complete. 2024-09-19T07:05:09.491 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:05:09.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:09 smithi174 ceph-mon[35637]: pgmap v498: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-09-19T07:05:09.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:09 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T07:05:09.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:09 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:05:09.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:09 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T07:05:09.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:09 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:05:09.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:09 smithi103 ceph-mon[31158]: pgmap v498: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-09-19T07:05:09.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:09 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T07:05:09.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:09 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:05:09.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:09 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T07:05:09.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:09 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:05:09.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:09 smithi134 ceph-mon[35356]: pgmap v498: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-09-19T07:05:09.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:09 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T07:05:09.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:09 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:05:09.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:09 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T07:05:09.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:09 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:05:09.740 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:10.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.220+0000 7fa53b26d640 1 -- 172.21.15.103:0/2337785581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa534102790 msgr2=0x7fa5341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:10.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.220+0000 7fa53b26d640 1 --2- 172.21.15.103:0/2337785581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa534102790 0x7fa5341067e0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7fa530009900 tx=0x7fa53002a410 comp rx=0 tx=0).stop 2024-09-19T07:05:10.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.220+0000 7fa53b26d640 1 -- 172.21.15.103:0/2337785581 shutdown_connections 2024-09-19T07:05:10.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.220+0000 7fa53b26d640 1 --2- 172.21.15.103:0/2337785581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa534106f10 0x7fa53410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.220+0000 7fa53b26d640 1 --2- 172.21.15.103:0/2337785581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa534102790 0x7fa5341067e0 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.220+0000 7fa53b26d640 1 --2- 172.21.15.103:0/2337785581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa534101de0 0x7fa5341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.220+0000 7fa53b26d640 1 -- 172.21.15.103:0/2337785581 >> 172.21.15.103:0/2337785581 conn(0x7fa5340fd8c0 msgr2=0x7fa5340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:10.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.220+0000 7fa53b26d640 1 -- 172.21.15.103:0/2337785581 shutdown_connections 2024-09-19T07:05:10.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.220+0000 7fa53b26d640 1 -- 172.21.15.103:0/2337785581 wait complete. 2024-09-19T07:05:10.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa53b26d640 1 Processor -- start 2024-09-19T07:05:10.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa53b26d640 1 -- start start 2024-09-19T07:05:10.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa53b26d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa534101de0 0x7fa5340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa53b26d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa534102790 0x7fa534074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa53b26d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa534106f10 0x7fa534074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa53b26d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa534111f90 con 0x7fa534106f10 2024-09-19T07:05:10.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa53b26d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa534111e10 con 0x7fa534102790 2024-09-19T07:05:10.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa53b26d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa534112110 con 0x7fa534101de0 2024-09-19T07:05:10.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa53980a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa534106f10 0x7fa534074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:10.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa53980a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa534106f10 0x7fa534074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42454/0 (socket says 172.21.15.103:42454) 2024-09-19T07:05:10.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa53980a640 1 -- 172.21.15.103:0/3909124871 learned_addr learned my addr 172.21.15.103:0/3909124871 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:10.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa538808640 1 --2- 172.21.15.103:0/3909124871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa534102790 0x7fa534074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:10.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa539009640 1 --2- 172.21.15.103:0/3909124871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa534101de0 0x7fa5340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:10.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa53980a640 1 -- 172.21.15.103:0/3909124871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa534101de0 msgr2=0x7fa5340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:10.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa53980a640 1 --2- 172.21.15.103:0/3909124871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa534101de0 0x7fa5340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa53980a640 1 -- 172.21.15.103:0/3909124871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa534102790 msgr2=0x7fa534074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:10.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa53980a640 1 --2- 172.21.15.103:0/3909124871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa534102790 0x7fa534074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa53980a640 1 -- 172.21.15.103:0/3909124871 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5341a6980 con 0x7fa534106f10 2024-09-19T07:05:10.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.221+0000 7fa539009640 1 --2- 172.21.15.103:0/3909124871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa534101de0 0x7fa5340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:10.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.222+0000 7fa53980a640 1 --2- 172.21.15.103:0/3909124871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa534106f10 0x7fa534074f70 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00cad0 tx=0x7fa52c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:10.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.222+0000 7fa5267fc640 1 -- 172.21.15.103:0/3909124871 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c00ae20 con 0x7fa534106f10 2024-09-19T07:05:10.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.222+0000 7fa5267fc640 1 -- 172.21.15.103:0/3909124871 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa52c005020 con 0x7fa534106f10 2024-09-19T07:05:10.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.222+0000 7fa53b26d640 1 -- 172.21.15.103:0/3909124871 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5341a6c70 con 0x7fa534106f10 2024-09-19T07:05:10.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.222+0000 7fa5267fc640 1 -- 172.21.15.103:0/3909124871 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c00f6f0 con 0x7fa534106f10 2024-09-19T07:05:10.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.222+0000 7fa53b26d640 1 -- 172.21.15.103:0/3909124871 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5341a7160 con 0x7fa534106f10 2024-09-19T07:05:10.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.224+0000 7fa5267fc640 1 -- 172.21.15.103:0/3909124871 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa52c0051d0 con 0x7fa534106f10 2024-09-19T07:05:10.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.225+0000 7fa53d646740 1 Processor -- start 2024-09-19T07:05:10.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.225+0000 7fa53d646740 1 -- start start 2024-09-19T07:05:10.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.225+0000 7fa53d646740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea64a0 0x5625d1ea6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.225+0000 7fa53d646740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1ea6dc0 0x5625d1eb1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.225+0000 7fa53d646740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1eb1680 0x5625d1eb3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.225+0000 7fa53d646740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5625d1e0c2e0 con 0x5625d1eb1680 2024-09-19T07:05:10.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.225+0000 7fa53d646740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5625d1db3ce0 con 0x5625d1ea64a0 2024-09-19T07:05:10.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.225+0000 7fa53d646740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5625d1e082a0 con 0x5625d1ea6dc0 2024-09-19T07:05:10.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.225+0000 7fa5267fc640 1 --2- 172.21.15.103:0/3909124871 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5200a33b0 0x7fa5200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.225+0000 7fa538808640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1eb1680 0x5625d1eb3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:10.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.225+0000 7fa538808640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1eb1680 0x5625d1eb3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42460/0 (socket says 172.21.15.103:42460) 2024-09-19T07:05:10.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.225+0000 7fa538808640 1 -- 172.21.15.103:0/2418285125 learned_addr learned my addr 172.21.15.103:0/2418285125 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:10.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.225+0000 7fa538808640 1 --2- 172.21.15.103:0/2418285125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea64a0 0x5625d1ea6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:10.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.225+0000 7fa539009640 1 --2- 172.21.15.103:0/2418285125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1ea6dc0 0x5625d1eb1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:10.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.225+0000 7fa53980a640 1 --2- 172.21.15.103:0/3909124871 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5200a33b0 0x7fa5200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:10.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.225+0000 7fa5267fc640 1 -- 172.21.15.103:0/3909124871 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fa52c0cfeb0 con 0x7fa534106f10 2024-09-19T07:05:10.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.226+0000 7fa538808640 1 -- 172.21.15.103:0/2418285125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1ea6dc0 msgr2=0x5625d1eb1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:10.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.226+0000 7fa538808640 1 --2- 172.21.15.103:0/2418285125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1ea6dc0 0x5625d1eb1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.226+0000 7fa538808640 1 -- 172.21.15.103:0/2418285125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea64a0 msgr2=0x5625d1ea6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:10.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.226+0000 7fa538808640 1 --2- 172.21.15.103:0/2418285125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea64a0 0x5625d1ea6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.226+0000 7fa538808640 1 -- 172.21.15.103:0/2418285125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625d1d98440 con 0x5625d1eb1680 2024-09-19T07:05:10.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.226+0000 7fa53980a640 1 --2- 172.21.15.103:0/3909124871 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5200a33b0 0x7fa5200a5870 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0cd690 tx=0x7fa52c06efa0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:10.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.226+0000 7fa538808640 1 --2- 172.21.15.103:0/2418285125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1eb1680 0x5625d1eb3a70 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7fa530031550 tx=0x7fa5300058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab57affde66a965e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:10.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.226+0000 7fa51f7fe640 1 -- 172.21.15.103:0/2418285125 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530035040 con 0x5625d1eb1680 2024-09-19T07:05:10.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.226+0000 7fa51f7fe640 1 -- 172.21.15.103:0/2418285125 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5300319e0 con 0x5625d1eb1680 2024-09-19T07:05:10.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.226+0000 7fa51f7fe640 1 -- 172.21.15.103:0/2418285125 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530004a00 con 0x5625d1eb1680 2024-09-19T07:05:10.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.227+0000 7fa53d646740 1 -- 172.21.15.103:0/2418285125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1eb1680 msgr2=0x5625d1eb3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:10.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.227+0000 7fa53d646740 1 --2- 172.21.15.103:0/2418285125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1eb1680 0x5625d1eb3a70 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7fa530031550 tx=0x7fa5300058d0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.227+0000 7fa53d646740 1 -- 172.21.15.103:0/2418285125 shutdown_connections 2024-09-19T07:05:10.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.227+0000 7fa53d646740 1 --2- 172.21.15.103:0/2418285125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1eb1680 0x5625d1eb3a70 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.227+0000 7fa53d646740 1 --2- 172.21.15.103:0/2418285125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1ea6dc0 0x5625d1eb1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.227+0000 7fa53d646740 1 --2- 172.21.15.103:0/2418285125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea64a0 0x5625d1ea6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.227+0000 7fa53d646740 1 -- 172.21.15.103:0/2418285125 >> 172.21.15.103:0/2418285125 conn(0x5625d1ea2a30 msgr2=0x5625d1ea2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:10.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.227+0000 7fa53d646740 1 -- 172.21.15.103:0/2418285125 shutdown_connections 2024-09-19T07:05:10.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.227+0000 7fa53d646740 1 -- 172.21.15.103:0/2418285125 wait complete. 2024-09-19T07:05:10.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.227+0000 7fa53d646740 1 -- start start 2024-09-19T07:05:10.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.227+0000 7fa53d646740 1 Processor -- start 2024-09-19T07:05:10.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa53d646740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1ea64a0 0x5625d1eb1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa53d646740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea6dc0 0x5625d1eae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa53d646740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1eb1680 0x5625d1eae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa53d646740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5625d1eb4250 con 0x5625d1ea64a0 2024-09-19T07:05:10.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa53d646740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5625d1dc7050 con 0x5625d1ea6dc0 2024-09-19T07:05:10.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa53d646740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5625d1eb43d0 con 0x5625d1eb1680 2024-09-19T07:05:10.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa53d646740 10 client.0 _ll_register_callbacks cb 0x5625d1e07160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:10.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa538808640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1ea64a0 0x5625d1eb1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:10.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa538808640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1ea64a0 0x5625d1eb1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42466/0 (socket says 172.21.15.103:42466) 2024-09-19T07:05:10.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa538808640 1 -- 172.21.15.103:0/1028704756 learned_addr learned my addr 172.21.15.103:0/1028704756 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:10.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa539009640 1 --2- 172.21.15.103:0/1028704756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea6dc0 0x5625d1eae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:10.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa538808640 1 -- 172.21.15.103:0/1028704756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1eb1680 msgr2=0x5625d1eae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:10.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa538808640 1 --2- 172.21.15.103:0/1028704756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1eb1680 0x5625d1eae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa538808640 1 -- 172.21.15.103:0/1028704756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea6dc0 msgr2=0x5625d1eae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:10.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa538808640 1 --2- 172.21.15.103:0/1028704756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea6dc0 0x5625d1eae2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa538808640 1 -- 172.21.15.103:0/1028704756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625d1b7d3c0 con 0x5625d1ea64a0 2024-09-19T07:05:10.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa538808640 1 --2- 172.21.15.103:0/1028704756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1ea64a0 0x5625d1eb1010 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7fa5300315f0 tx=0x7fa5300314d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f39109dba01100 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:10.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.228+0000 7fa51cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:10.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.229+0000 7fa51cff9640 1 -- 172.21.15.103:0/1028704756 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53004e030 con 0x5625d1ea64a0 2024-09-19T07:05:10.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.229+0000 7fa53d646740 1 -- 172.21.15.103:0/1028704756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5300319e0 con 0x5625d1ea64a0 2024-09-19T07:05:10.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.229+0000 7fa51cff9640 1 -- 172.21.15.103:0/1028704756 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fa5300049b0 con 0x5625d1ea64a0 2024-09-19T07:05:10.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.229+0000 7fa51cff9640 1 -- 172.21.15.103:0/1028704756 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa530058050 con 0x5625d1ea64a0 2024-09-19T07:05:10.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.229+0000 7fa53d646740 10 client.21087 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:10.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.229+0000 7fa53d646740 1 -- 172.21.15.103:0/1028704756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5625d1ebfc40 con 0x5625d1ea64a0 2024-09-19T07:05:10.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.229+0000 7fa51cff9640 1 -- 172.21.15.103:0/1028704756 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5300319e0 con 0x5625d1ea64a0 2024-09-19T07:05:10.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.229+0000 7fa51cff9640 1 -- 172.21.15.103:0/1028704756 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa53003a040 con 0x5625d1ea64a0 2024-09-19T07:05:10.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.229+0000 7fa53d646740 10 client.21087 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:10.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.229+0000 7fa53d646740 10 client.21087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:10.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.229+0000 7fa53d646740 20 client.21087 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:10.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.229+0000 7fa53d646740 1 --2- 172.21.15.103:0/1028704756 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5625d1ec0f90 0x5625d1ec3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.229+0000 7fa53d646740 4 client.21087 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:10.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.229+0000 7fa53d646740 1 -- 172.21.15.103:0/1028704756 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625d1e19850 con 0x5625d1ec0f90 2024-09-19T07:05:10.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.229+0000 7fa539009640 1 --2- 172.21.15.103:0/1028704756 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5625d1ec0f90 0x5625d1ec3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:10.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.230+0000 7fa51cff9640 10 client.21087 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:10.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.230+0000 7fa539009640 1 --2- 172.21.15.103:0/1028704756 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5625d1ec0f90 0x5625d1ec3380 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8518c896280bcaf server_cookie=8baa1b6ceb370f50 in_seq=0 out_seq=0 2024-09-19T07:05:10.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.231+0000 7fa51cff9640 1 -- 172.21.15.103:0/1028704756 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625d1e19850 con 0x5625d1ec0f90 2024-09-19T07:05:10.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.231+0000 7fa51cff9640 10 client.21087 handle_command_reply: tid=0 2024-09-19T07:05:10.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.231+0000 7fa53d646740 1 client.21087 shutdown 2024-09-19T07:05:10.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.231+0000 7fa53d646740 1 -- 172.21.15.103:0/1028704756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1ea64a0 msgr2=0x5625d1eb1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:10.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.231+0000 7fa53d646740 1 --2- 172.21.15.103:0/1028704756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1ea64a0 0x5625d1eb1010 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7fa5300315f0 tx=0x7fa5300314d0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.231+0000 7fa53d646740 1 -- 172.21.15.103:0/1028704756 shutdown_connections 2024-09-19T07:05:10.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.231+0000 7fa53d646740 1 --2- 172.21.15.103:0/1028704756 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5625d1ec0f90 0x5625d1ec3380 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.231+0000 7fa53d646740 1 --2- 172.21.15.103:0/1028704756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1eb1680 0x5625d1eae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.231+0000 7fa53d646740 1 --2- 172.21.15.103:0/1028704756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea6dc0 0x5625d1eae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.231+0000 7fa53d646740 1 --2- 172.21.15.103:0/1028704756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1ea64a0 0x5625d1eb1010 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.231+0000 7fa53d646740 1 -- 172.21.15.103:0/1028704756 >> 172.21.15.103:0/1028704756 conn(0x5625d1ea2a30 msgr2=0x5625d1ea8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:10.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.231+0000 7fa51cff9640 0 client.21087 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:10.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.231+0000 7fa53d646740 1 -- 172.21.15.103:0/1028704756 shutdown_connections 2024-09-19T07:05:10.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.231+0000 7fa53d646740 1 -- 172.21.15.103:0/1028704756 wait complete. 2024-09-19T07:05:10.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.231+0000 7fa53d646740 20 client.21087 trim_cache size 0 max 16384 2024-09-19T07:05:10.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa53d646740 1 Processor -- start 2024-09-19T07:05:10.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa53d646740 1 -- start start 2024-09-19T07:05:10.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa53d646740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea64a0 0x5625d1ea1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa53d646740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1ea6dc0 0x5625d1ec5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa53d646740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1eb1680 0x5625d1ec6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa53d646740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa5341afb20 con 0x5625d1ea6dc0 2024-09-19T07:05:10.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa53d646740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5341b1660 con 0x5625d1ea64a0 2024-09-19T07:05:10.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa53d646740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa5341b01f0 con 0x5625d1eb1680 2024-09-19T07:05:10.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa539009640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1ea6dc0 0x5625d1ec5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:10.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa539009640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1ea6dc0 0x5625d1ec5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42480/0 (socket says 172.21.15.103:42480) 2024-09-19T07:05:10.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa539009640 1 -- 172.21.15.103:0/192698862 learned_addr learned my addr 172.21.15.103:0/192698862 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:10.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa538808640 1 --2- 172.21.15.103:0/192698862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1eb1680 0x5625d1ec6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:10.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa539009640 1 -- 172.21.15.103:0/192698862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1eb1680 msgr2=0x5625d1ec6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:10.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa539009640 1 --2- 172.21.15.103:0/192698862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1eb1680 0x5625d1ec6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa539009640 1 -- 172.21.15.103:0/192698862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea64a0 msgr2=0x5625d1ea1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:10.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa539009640 1 --2- 172.21.15.103:0/192698862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea64a0 0x5625d1ea1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa539009640 1 -- 172.21.15.103:0/192698862 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625d1eaebb0 con 0x5625d1ea6dc0 2024-09-19T07:05:10.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.248+0000 7fa538808640 1 --2- 172.21.15.103:0/192698862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1eb1680 0x5625d1ec6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:10.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.249+0000 7fa539009640 1 --2- 172.21.15.103:0/192698862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1ea6dc0 0x5625d1ec5fc0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x5625d1ea3890 tx=0x7fa52800af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827bfb5729767d89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:10.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.249+0000 7fa51effd640 1 -- 172.21.15.103:0/192698862 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528016a60 con 0x5625d1ea6dc0 2024-09-19T07:05:10.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.249+0000 7fa51effd640 1 -- 172.21.15.103:0/192698862 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa528016c00 con 0x5625d1ea6dc0 2024-09-19T07:05:10.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.249+0000 7fa51effd640 1 -- 172.21.15.103:0/192698862 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5280147d0 con 0x5625d1ea6dc0 2024-09-19T07:05:10.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.249+0000 7fa53d646740 1 -- 172.21.15.103:0/192698862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1ea6dc0 msgr2=0x5625d1ec5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:10.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.249+0000 7fa53d646740 1 --2- 172.21.15.103:0/192698862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1ea6dc0 0x5625d1ec5fc0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x5625d1ea3890 tx=0x7fa52800af60 comp rx=0 tx=0).stop 2024-09-19T07:05:10.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.249+0000 7fa53d646740 1 -- 172.21.15.103:0/192698862 shutdown_connections 2024-09-19T07:05:10.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.249+0000 7fa53d646740 1 --2- 172.21.15.103:0/192698862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1eb1680 0x5625d1ec6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.249+0000 7fa53d646740 1 --2- 172.21.15.103:0/192698862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1ea6dc0 0x5625d1ec5fc0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.249+0000 7fa53d646740 1 --2- 172.21.15.103:0/192698862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea64a0 0x5625d1ea1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.249+0000 7fa53d646740 1 -- 172.21.15.103:0/192698862 >> 172.21.15.103:0/192698862 conn(0x5625d1ea2a30 msgr2=0x5625d1ea1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:10.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.249+0000 7fa53d646740 1 -- 172.21.15.103:0/192698862 shutdown_connections 2024-09-19T07:05:10.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.250+0000 7fa53d646740 1 -- 172.21.15.103:0/192698862 wait complete. 2024-09-19T07:05:10.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.250+0000 7fa53d646740 1 -- start start 2024-09-19T07:05:10.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.250+0000 7fa53d646740 1 Processor -- start 2024-09-19T07:05:10.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.250+0000 7fa53d646740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea64a0 0x5625d1ec5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.250+0000 7fa53d646740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1ea6dc0 0x5625d1ec6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.250+0000 7fa53d646740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1eb1680 0x5625d1ec3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.250+0000 7fa53d646740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa5341b0f90 con 0x5625d1eb1680 2024-09-19T07:05:10.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.250+0000 7fa53d646740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5341b08c0 con 0x5625d1ea64a0 2024-09-19T07:05:10.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.250+0000 7fa53d646740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa5341adb20 con 0x5625d1ea6dc0 2024-09-19T07:05:10.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.250+0000 7fa53d646740 10 client.0 _ll_register_callbacks cb 0x7fa530032ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:10.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.250+0000 7fa538808640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1eb1680 0x5625d1ec3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:10.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa539009640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1ea6dc0 0x5625d1ec6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:10.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa538808640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1eb1680 0x5625d1ec3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42494/0 (socket says 172.21.15.103:42494) 2024-09-19T07:05:10.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa538808640 1 -- 172.21.15.103:0/3361464100 learned_addr learned my addr 172.21.15.103:0/3361464100 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:10.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa538808640 1 --2- 172.21.15.103:0/3361464100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea64a0 0x5625d1ec5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:10.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa538808640 1 -- 172.21.15.103:0/3361464100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1ea6dc0 msgr2=0x5625d1ec6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:10.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa538808640 1 --2- 172.21.15.103:0/3361464100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1ea6dc0 0x5625d1ec6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa538808640 1 -- 172.21.15.103:0/3361464100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea64a0 msgr2=0x5625d1ec5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:10.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa538808640 1 --2- 172.21.15.103:0/3361464100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea64a0 0x5625d1ec5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa538808640 1 -- 172.21.15.103:0/3361464100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625d1ec3c80 con 0x5625d1eb1680 2024-09-19T07:05:10.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa538808640 1 --2- 172.21.15.103:0/3361464100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1eb1680 0x5625d1ec3540 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fa53003dd80 tx=0x7fa530031ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e8bfa6115ab3a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:10.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa51cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:10.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa51cff9640 1 -- 172.21.15.103:0/3361464100 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53004e040 con 0x5625d1eb1680 2024-09-19T07:05:10.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa51cff9640 1 -- 172.21.15.103:0/3361464100 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fa530053a50 con 0x5625d1eb1680 2024-09-19T07:05:10.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa53d646740 1 -- 172.21.15.103:0/3361464100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa528016c00 con 0x5625d1eb1680 2024-09-19T07:05:10.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa51cff9640 1 -- 172.21.15.103:0/3361464100 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa530035070 con 0x5625d1eb1680 2024-09-19T07:05:10.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa53d646740 10 client.21093 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:10.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.251+0000 7fa53d646740 1 -- 172.21.15.103:0/3361464100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5625d1f2b390 con 0x5625d1eb1680 2024-09-19T07:05:10.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.252+0000 7fa51cff9640 1 -- 172.21.15.103:0/3361464100 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa528016c00 con 0x5625d1eb1680 2024-09-19T07:05:10.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.252+0000 7fa51cff9640 1 -- 172.21.15.103:0/3361464100 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa53005b040 con 0x5625d1eb1680 2024-09-19T07:05:10.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.252+0000 7fa53d646740 10 client.21093 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:10.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.252+0000 7fa53d646740 10 client.21093 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:10.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.252+0000 7fa53d646740 20 client.21093 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:10.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.252+0000 7fa53d646740 1 --2- 172.21.15.103:0/3361464100 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5625d1f2b920 0x5625d1f2dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:10.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.252+0000 7fa53d646740 4 client.21093 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:05:10.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.252+0000 7fa53d646740 1 -- 172.21.15.103:0/3361464100 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5625d1e13d60 con 0x5625d1f2b920 2024-09-19T07:05:10.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.252+0000 7fa539009640 1 --2- 172.21.15.103:0/3361464100 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5625d1f2b920 0x5625d1f2dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:10.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.253+0000 7fa51cff9640 10 client.21093 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:10.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.252+0000 7fa539009640 1 --2- 172.21.15.103:0/3361464100 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5625d1f2b920 0x5625d1f2dd10 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b988b83a97e551f server_cookie=649456bafb447278 in_seq=0 out_seq=0 2024-09-19T07:05:10.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.253+0000 7fa51cff9640 1 -- 172.21.15.103:0/3361464100 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5625d1e13d60 con 0x5625d1f2b920 2024-09-19T07:05:10.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.253+0000 7fa51cff9640 10 client.21093 handle_command_reply: tid=0 2024-09-19T07:05:10.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.253+0000 7fa53d646740 1 client.21093 shutdown 2024-09-19T07:05:10.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.253+0000 7fa53d646740 1 -- 172.21.15.103:0/3361464100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1eb1680 msgr2=0x5625d1ec3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:10.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.253+0000 7fa53d646740 1 --2- 172.21.15.103:0/3361464100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1eb1680 0x5625d1ec3540 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fa53003dd80 tx=0x7fa530031ae0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.253+0000 7fa53d646740 1 -- 172.21.15.103:0/3361464100 shutdown_connections 2024-09-19T07:05:10.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.253+0000 7fa53d646740 1 --2- 172.21.15.103:0/3361464100 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5625d1f2b920 0x5625d1f2dd10 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.253+0000 7fa53d646740 1 --2- 172.21.15.103:0/3361464100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d1eb1680 0x5625d1ec3540 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.253+0000 7fa51cff9640 0 client.21093 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:10.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.253+0000 7fa53d646740 1 --2- 172.21.15.103:0/3361464100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d1ea6dc0 0x5625d1ec6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.253+0000 7fa53d646740 1 --2- 172.21.15.103:0/3361464100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d1ea64a0 0x5625d1ec5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.253+0000 7fa53d646740 1 -- 172.21.15.103:0/3361464100 >> 172.21.15.103:0/3361464100 conn(0x5625d1ea2a30 msgr2=0x5625d1ea9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:10.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.253+0000 7fa53d646740 1 -- 172.21.15.103:0/3361464100 shutdown_connections 2024-09-19T07:05:10.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.253+0000 7fa53d646740 1 -- 172.21.15.103:0/3361464100 wait complete. 2024-09-19T07:05:10.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.253+0000 7fa53d646740 20 client.21093 trim_cache size 0 max 16384 2024-09-19T07:05:10.260 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:05:10.260 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:05:10.260 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:05:10.260 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:05:10.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.254+0000 7fa53b26d640 1 -- 172.21.15.103:0/3909124871 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5200a33b0 msgr2=0x7fa5200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:10.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.254+0000 7fa53b26d640 1 --2- 172.21.15.103:0/3909124871 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5200a33b0 0x7fa5200a5870 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0cd690 tx=0x7fa52c06efa0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.254+0000 7fa53b26d640 1 -- 172.21.15.103:0/3909124871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa534106f10 msgr2=0x7fa534074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:10.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.254+0000 7fa53b26d640 1 --2- 172.21.15.103:0/3909124871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa534106f10 0x7fa534074f70 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00cad0 tx=0x7fa52c00cf90 comp rx=0 tx=0).stop 2024-09-19T07:05:10.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.254+0000 7fa53b26d640 1 -- 172.21.15.103:0/3909124871 shutdown_connections 2024-09-19T07:05:10.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.254+0000 7fa53b26d640 1 --2- 172.21.15.103:0/3909124871 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa5200a33b0 0x7fa5200a5870 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.254+0000 7fa53b26d640 1 --2- 172.21.15.103:0/3909124871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa534106f10 0x7fa534074f70 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.254+0000 7fa53b26d640 1 --2- 172.21.15.103:0/3909124871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa534102790 0x7fa534074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.254+0000 7fa53b26d640 1 --2- 172.21.15.103:0/3909124871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa534101de0 0x7fa5340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:10.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.254+0000 7fa53b26d640 1 -- 172.21.15.103:0/3909124871 >> 172.21.15.103:0/3909124871 conn(0x7fa5340fd8c0 msgr2=0x7fa5340fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:10.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.254+0000 7fa53b26d640 1 -- 172.21.15.103:0/3909124871 shutdown_connections 2024-09-19T07:05:10.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:10.254+0000 7fa53b26d640 1 -- 172.21.15.103:0/3909124871 wait complete. 2024-09-19T07:05:10.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:10 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T07:05:10.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:10 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T07:05:10.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:10 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T07:05:10.932 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:05:10.932 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:05:11.182 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.212+0000 7f65058a5640 1 -- 172.21.15.103:0/232479632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6500106a00 msgr2=0x7f6500106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.212+0000 7f65058a5640 1 --2- 172.21.15.103:0/232479632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6500106a00 0x7f6500106de0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009930 tx=0x7f64f002a420 comp rx=0 tx=0).stop 2024-09-19T07:05:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.212+0000 7f65058a5640 1 -- 172.21.15.103:0/232479632 shutdown_connections 2024-09-19T07:05:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.212+0000 7f65058a5640 1 --2- 172.21.15.103:0/232479632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6500101390 0x7f650010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.212+0000 7f65058a5640 1 --2- 172.21.15.103:0/232479632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65001009f0 0x7f6500100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.212+0000 7f65058a5640 1 --2- 172.21.15.103:0/232479632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6500106a00 0x7f6500106de0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.212+0000 7f65058a5640 1 -- 172.21.15.103:0/232479632 >> 172.21.15.103:0/232479632 conn(0x7f65000fc6b0 msgr2=0x7f65000fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.212+0000 7f65058a5640 1 -- 172.21.15.103:0/232479632 shutdown_connections 2024-09-19T07:05:11.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.212+0000 7f65058a5640 1 -- 172.21.15.103:0/232479632 wait complete. 2024-09-19T07:05:11.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.212+0000 7f65058a5640 1 Processor -- start 2024-09-19T07:05:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f65058a5640 1 -- start start 2024-09-19T07:05:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f65058a5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65001009f0 0x7f6500199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f65058a5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6500101390 0x7f6500199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f65058a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6500106a00 0x7f65001a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f65058a5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6500110e40 con 0x7f65001009f0 2024-09-19T07:05:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f65058a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6500110cc0 con 0x7f6500106a00 2024-09-19T07:05:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f65058a5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6500110fc0 con 0x7f6500101390 2024-09-19T07:05:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f64feffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65001009f0 0x7f6500199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f64feffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65001009f0 0x7f6500199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42514/0 (socket says 172.21.15.103:42514) 2024-09-19T07:05:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f64feffd640 1 -- 172.21.15.103:0/3715000930 learned_addr learned my addr 172.21.15.103:0/3715000930 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f64fe7fc640 1 --2- 172.21.15.103:0/3715000930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6500101390 0x7f6500199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f64ff7fe640 1 --2- 172.21.15.103:0/3715000930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6500106a00 0x7f65001a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f64feffd640 1 -- 172.21.15.103:0/3715000930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6500101390 msgr2=0x7f6500199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f64feffd640 1 --2- 172.21.15.103:0/3715000930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6500101390 0x7f6500199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f64feffd640 1 -- 172.21.15.103:0/3715000930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6500106a00 msgr2=0x7f65001a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f64feffd640 1 --2- 172.21.15.103:0/3715000930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6500106a00 0x7f65001a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.213+0000 7f64feffd640 1 -- 172.21.15.103:0/3715000930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65001a47f0 con 0x7f65001009f0 2024-09-19T07:05:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.214+0000 7f64feffd640 1 --2- 172.21.15.103:0/3715000930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65001009f0 0x7f6500199230 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009230 tx=0x7f64f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.214+0000 7f65048a3640 1 -- 172.21.15.103:0/3715000930 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f003f030 con 0x7f65001009f0 2024-09-19T07:05:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.214+0000 7f65058a5640 1 -- 172.21.15.103:0/3715000930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65001a49c0 con 0x7f65001009f0 2024-09-19T07:05:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.214+0000 7f65048a3640 1 -- 172.21.15.103:0/3715000930 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f0003ad0 con 0x7f65001009f0 2024-09-19T07:05:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.214+0000 7f65048a3640 1 -- 172.21.15.103:0/3715000930 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0035070 con 0x7f65001009f0 2024-09-19T07:05:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.214+0000 7f65058a5640 1 -- 172.21.15.103:0/3715000930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65001a4e20 con 0x7f65001009f0 2024-09-19T07:05:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.216+0000 7f65048a3640 1 -- 172.21.15.103:0/3715000930 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f64f0042030 con 0x7f65001009f0 2024-09-19T07:05:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.216+0000 7f6508645740 1 Processor -- start 2024-09-19T07:05:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.216+0000 7f6508645740 1 -- start start 2024-09-19T07:05:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f6508645740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667a9a10 0x55fb667a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f6508645740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667aa330 0x55fb667b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f6508645740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb667bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f6508645740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fb6670e2f0 con 0x55fb667a9a10 2024-09-19T07:05:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f6508645740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fb666b6270 con 0x55fb667aa330 2024-09-19T07:05:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f6508645740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fb6670a2b0 con 0x55fb667b90c0 2024-09-19T07:05:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f64ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667a9a10 0x55fb667a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f64ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667a9a10 0x55fb667a9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42516/0 (socket says 172.21.15.103:42516) 2024-09-19T07:05:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f64ff7fe640 1 -- 172.21.15.103:0/1761794981 learned_addr learned my addr 172.21.15.103:0/1761794981 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f64feffd640 1 --2- 172.21.15.103:0/1761794981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667aa330 0x55fb667b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f65048a3640 1 --2- 172.21.15.103:0/3715000930 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f64e80a3460 0x7f64e80a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f64fe7fc640 1 --2- 172.21.15.103:0/1761794981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb667bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f64ff7fe640 1 -- 172.21.15.103:0/1761794981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 msgr2=0x55fb667bb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f64ff7fe640 1 --2- 172.21.15.103:0/1761794981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb667bb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f64ff7fe640 1 -- 172.21.15.103:0/1761794981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667aa330 msgr2=0x55fb667b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f64ff7fe640 1 --2- 172.21.15.103:0/1761794981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667aa330 0x55fb667b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f64ff7fe640 1 -- 172.21.15.103:0/1761794981 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb666b5340 con 0x55fb667a9a10 2024-09-19T07:05:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f64feffd640 1 --2- 172.21.15.103:0/1761794981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667aa330 0x55fb667b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f65048a3640 1 -- 172.21.15.103:0/3715000930 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f64f00f0170 con 0x7f65001009f0 2024-09-19T07:05:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f64ff7fe640 1 --2- 172.21.15.103:0/3715000930 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f64e80a3460 0x7f64e80a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.217+0000 7f64ff7fe640 1 --2- 172.21.15.103:0/1761794981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667a9a10 0x55fb667a9df0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f64f80104a0 tx=0x7f64f8010960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646155f5d32128ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.218+0000 7f64e5ffb640 1 -- 172.21.15.103:0/1761794981 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f801d910 con 0x55fb667a9a10 2024-09-19T07:05:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.218+0000 7f64e5ffb640 1 -- 172.21.15.103:0/1761794981 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f8011440 con 0x55fb667a9a10 2024-09-19T07:05:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.218+0000 7f64e5ffb640 1 -- 172.21.15.103:0/1761794981 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f801c660 con 0x55fb667a9a10 2024-09-19T07:05:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.218+0000 7f6508645740 1 -- 172.21.15.103:0/1761794981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667a9a10 msgr2=0x55fb667a9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.218+0000 7f6508645740 1 --2- 172.21.15.103:0/1761794981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667a9a10 0x55fb667a9df0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f64f80104a0 tx=0x7f64f8010960 comp rx=0 tx=0).stop 2024-09-19T07:05:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.218+0000 7f64ff7fe640 1 --2- 172.21.15.103:0/3715000930 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f64e80a3460 0x7f64e80a5920 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f64f8021000 tx=0x7f64f801e000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.218+0000 7f6508645740 1 -- 172.21.15.103:0/1761794981 shutdown_connections 2024-09-19T07:05:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.218+0000 7f6508645740 1 --2- 172.21.15.103:0/1761794981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb667bb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.218+0000 7f6508645740 1 --2- 172.21.15.103:0/1761794981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667aa330 0x55fb667b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.218+0000 7f6508645740 1 --2- 172.21.15.103:0/1761794981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667a9a10 0x55fb667a9df0 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.218+0000 7f6508645740 1 -- 172.21.15.103:0/1761794981 >> 172.21.15.103:0/1761794981 conn(0x55fb667a6060 msgr2=0x55fb667a6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.218+0000 7f6508645740 1 -- 172.21.15.103:0/1761794981 shutdown_connections 2024-09-19T07:05:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.218+0000 7f6508645740 1 -- 172.21.15.103:0/1761794981 wait complete. 2024-09-19T07:05:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.219+0000 7f6508645740 1 -- start start 2024-09-19T07:05:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.219+0000 7f6508645740 1 Processor -- start 2024-09-19T07:05:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.219+0000 7f6508645740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667a9a10 0x55fb667b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.219+0000 7f6508645740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667aa330 0x55fb667b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.219+0000 7f6508645740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb667b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.219+0000 7f6508645740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fb667bbc90 con 0x55fb667aa330 2024-09-19T07:05:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.219+0000 7f6508645740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fb66697290 con 0x55fb667a9a10 2024-09-19T07:05:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.219+0000 7f6508645740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fb667bbe10 con 0x55fb667b90c0 2024-09-19T07:05:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.219+0000 7f6508645740 10 client.0 _ll_register_callbacks cb 0x55fb666c8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.219+0000 7f64fe7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667aa330 0x55fb667b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.219+0000 7f64fe7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667aa330 0x55fb667b64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42526/0 (socket says 172.21.15.103:42526) 2024-09-19T07:05:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.219+0000 7f64fe7fc640 1 -- 172.21.15.103:0/2825430176 learned_addr learned my addr 172.21.15.103:0/2825430176 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.219+0000 7f64fe7fc640 1 -- 172.21.15.103:0/2825430176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 msgr2=0x55fb667b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f64ff7fe640 1 --2- 172.21.15.103:0/2825430176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb667b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f64feffd640 1 --2- 172.21.15.103:0/2825430176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667a9a10 0x55fb667b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f64fe7fc640 1 --2- 172.21.15.103:0/2825430176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb667b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f64fe7fc640 1 -- 172.21.15.103:0/2825430176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667a9a10 msgr2=0x55fb667b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f64fe7fc640 1 --2- 172.21.15.103:0/2825430176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667a9a10 0x55fb667b5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f64fe7fc640 1 -- 172.21.15.103:0/2825430176 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb66421210 con 0x55fb667aa330 2024-09-19T07:05:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f64ff7fe640 1 --2- 172.21.15.103:0/2825430176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb667b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f64feffd640 1 --2- 172.21.15.103:0/2825430176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667a9a10 0x55fb667b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f64fe7fc640 1 --2- 172.21.15.103:0/2825430176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667aa330 0x55fb667b64a0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f64f400ba50 tx=0x7f64f400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c033cbc590424d3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f64deffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f64deffd640 1 -- 172.21.15.103:0/2825430176 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f401b440 con 0x55fb667aa330 2024-09-19T07:05:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f64deffd640 1 -- 172.21.15.103:0/2825430176 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f64f401b5e0 con 0x55fb667aa330 2024-09-19T07:05:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f6508645740 1 -- 172.21.15.103:0/2825430176 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64f8011440 con 0x55fb667aa330 2024-09-19T07:05:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f64deffd640 1 -- 172.21.15.103:0/2825430176 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f64f401b7e0 con 0x55fb667aa330 2024-09-19T07:05:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f6508645740 10 client.21105 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f6508645740 1 -- 172.21.15.103:0/2825430176 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fb664ed620 con 0x55fb667aa330 2024-09-19T07:05:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f64deffd640 1 -- 172.21.15.103:0/2825430176 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f8011440 con 0x55fb667aa330 2024-09-19T07:05:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f64deffd640 1 -- 172.21.15.103:0/2825430176 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f64f4019020 con 0x55fb667aa330 2024-09-19T07:05:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f6508645740 10 client.21105 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.220+0000 7f6508645740 10 client.21105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.221+0000 7f6508645740 20 client.21105 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.221+0000 7f6508645740 1 --2- 172.21.15.103:0/2825430176 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fb667c9100 0x55fb667cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.221+0000 7f6508645740 4 client.21105 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.221+0000 7f6508645740 1 -- 172.21.15.103:0/2825430176 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb6671ba20 con 0x55fb667c9100 2024-09-19T07:05:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.221+0000 7f64feffd640 1 --2- 172.21.15.103:0/2825430176 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fb667c9100 0x55fb667cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.221+0000 7f64feffd640 1 --2- 172.21.15.103:0/2825430176 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fb667c9100 0x55fb667cb510 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27fa5fd7ad6ba719 server_cookie=c97aed8fa51c7923 in_seq=0 out_seq=0 2024-09-19T07:05:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.221+0000 7f64deffd640 10 client.21105 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.222+0000 7f64deffd640 1 -- 172.21.15.103:0/2825430176 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb6671ba20 con 0x55fb667c9100 2024-09-19T07:05:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.222+0000 7f64deffd640 10 client.21105 handle_command_reply: tid=0 2024-09-19T07:05:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.222+0000 7f6508645740 1 client.21105 shutdown 2024-09-19T07:05:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.222+0000 7f6508645740 1 -- 172.21.15.103:0/2825430176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667aa330 msgr2=0x55fb667b64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.222+0000 7f6508645740 1 --2- 172.21.15.103:0/2825430176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667aa330 0x55fb667b64a0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f64f400ba50 tx=0x7f64f400bf10 comp rx=0 tx=0).stop 2024-09-19T07:05:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.222+0000 7f6508645740 1 -- 172.21.15.103:0/2825430176 shutdown_connections 2024-09-19T07:05:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.222+0000 7f6508645740 1 --2- 172.21.15.103:0/2825430176 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fb667c9100 0x55fb667cb510 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.222+0000 7f6508645740 1 --2- 172.21.15.103:0/2825430176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb667b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.222+0000 7f6508645740 1 --2- 172.21.15.103:0/2825430176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667aa330 0x55fb667b64a0 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.222+0000 7f64deffd640 0 client.21105 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.222+0000 7f6508645740 1 --2- 172.21.15.103:0/2825430176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667a9a10 0x55fb667b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.222+0000 7f6508645740 1 -- 172.21.15.103:0/2825430176 >> 172.21.15.103:0/2825430176 conn(0x55fb667a6060 msgr2=0x55fb667abd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.222+0000 7f6508645740 1 -- 172.21.15.103:0/2825430176 shutdown_connections 2024-09-19T07:05:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.222+0000 7f6508645740 1 -- 172.21.15.103:0/2825430176 wait complete. 2024-09-19T07:05:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.222+0000 7f6508645740 20 client.21105 trim_cache size 0 max 16384 2024-09-19T07:05:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f6508645740 1 Processor -- start 2024-09-19T07:05:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f6508645740 1 -- start start 2024-09-19T07:05:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f6508645740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667a9a10 0x55fb667b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f6508645740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667aa330 0x55fb667d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f6508645740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb667d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f6508645740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f65001ab5e0 con 0x55fb667a9a10 2024-09-19T07:05:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f6508645740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f65001ad120 con 0x55fb667aa330 2024-09-19T07:05:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f6508645740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f65001abcb0 con 0x55fb667b90c0 2024-09-19T07:05:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f64feffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667a9a10 0x55fb667b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f64feffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667a9a10 0x55fb667b7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42536/0 (socket says 172.21.15.103:42536) 2024-09-19T07:05:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f64feffd640 1 -- 172.21.15.103:0/1975676843 learned_addr learned my addr 172.21.15.103:0/1975676843 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f64feffd640 1 -- 172.21.15.103:0/1975676843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 msgr2=0x55fb667d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f64feffd640 1 --2- 172.21.15.103:0/1975676843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb667d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f64feffd640 1 -- 172.21.15.103:0/1975676843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667aa330 msgr2=0x55fb667d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f64feffd640 1 --2- 172.21.15.103:0/1975676843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667aa330 0x55fb667d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f64feffd640 1 -- 172.21.15.103:0/1975676843 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb667aa160 con 0x55fb667a9a10 2024-09-19T07:05:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.237+0000 7f64feffd640 1 --2- 172.21.15.103:0/1975676843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667a9a10 0x55fb667b7ed0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f64f00b8eb0 tx=0x7f64f0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a97f6c0d3332841 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.238+0000 7f64e57fa640 1 -- 172.21.15.103:0/1975676843 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0049040 con 0x55fb667a9a10 2024-09-19T07:05:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.238+0000 7f64e57fa640 1 -- 172.21.15.103:0/1975676843 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f00ba7e0 con 0x55fb667a9a10 2024-09-19T07:05:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.238+0000 7f64e57fa640 1 -- 172.21.15.103:0/1975676843 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f00b8b50 con 0x55fb667a9a10 2024-09-19T07:05:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.238+0000 7f6508645740 1 -- 172.21.15.103:0/1975676843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667a9a10 msgr2=0x55fb667b7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.238+0000 7f6508645740 1 --2- 172.21.15.103:0/1975676843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667a9a10 0x55fb667b7ed0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f64f00b8eb0 tx=0x7f64f0005e50 comp rx=0 tx=0).stop 2024-09-19T07:05:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.238+0000 7f6508645740 1 -- 172.21.15.103:0/1975676843 shutdown_connections 2024-09-19T07:05:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.238+0000 7f6508645740 1 --2- 172.21.15.103:0/1975676843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb667d1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.238+0000 7f6508645740 1 --2- 172.21.15.103:0/1975676843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667aa330 0x55fb667d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.238+0000 7f6508645740 1 --2- 172.21.15.103:0/1975676843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667a9a10 0x55fb667b7ed0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.238+0000 7f6508645740 1 -- 172.21.15.103:0/1975676843 >> 172.21.15.103:0/1975676843 conn(0x55fb667a6060 msgr2=0x55fb667b77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.238+0000 7f6508645740 1 -- 172.21.15.103:0/1975676843 shutdown_connections 2024-09-19T07:05:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.238+0000 7f6508645740 1 -- 172.21.15.103:0/1975676843 wait complete. 2024-09-19T07:05:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f6508645740 1 -- start start 2024-09-19T07:05:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f6508645740 1 Processor -- start 2024-09-19T07:05:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f6508645740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667a9a10 0x55fb667bda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f6508645740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667aa330 0x55fb667bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f6508645740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb6682bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f6508645740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f65001aca50 con 0x55fb667aa330 2024-09-19T07:05:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f6508645740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f65001ac380 con 0x55fb667a9a10 2024-09-19T07:05:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f6508645740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f65001a95c0 con 0x55fb667b90c0 2024-09-19T07:05:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f6508645740 10 client.0 _ll_register_callbacks cb 0x7f64f4004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f64fe7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667aa330 0x55fb667bdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f64fe7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667aa330 0x55fb667bdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42542/0 (socket says 172.21.15.103:42542) 2024-09-19T07:05:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f64fe7fc640 1 -- 172.21.15.103:0/374094328 learned_addr learned my addr 172.21.15.103:0/374094328 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f64feffd640 1 --2- 172.21.15.103:0/374094328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667a9a10 0x55fb667bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f64ff7fe640 1 --2- 172.21.15.103:0/374094328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb6682bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f64fe7fc640 1 -- 172.21.15.103:0/374094328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 msgr2=0x55fb6682bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f64fe7fc640 1 --2- 172.21.15.103:0/374094328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb6682bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f64fe7fc640 1 -- 172.21.15.103:0/374094328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667a9a10 msgr2=0x55fb667bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f64fe7fc640 1 --2- 172.21.15.103:0/374094328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667a9a10 0x55fb667bda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f64fe7fc640 1 -- 172.21.15.103:0/374094328 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb6682c560 con 0x55fb667aa330 2024-09-19T07:05:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f64feffd640 1 --2- 172.21.15.103:0/374094328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667a9a10 0x55fb667bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.239+0000 7f64ff7fe640 1 --2- 172.21.15.103:0/374094328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb6682bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.240+0000 7f64fe7fc640 1 --2- 172.21.15.103:0/374094328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667aa330 0x55fb667bdf40 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x55fb667b75a0 tx=0x7f64f400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad646b7e821fcf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.240+0000 7f64deffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.240+0000 7f64deffd640 1 -- 172.21.15.103:0/374094328 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f4027040 con 0x55fb667aa330 2024-09-19T07:05:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.240+0000 7f64deffd640 1 -- 172.21.15.103:0/374094328 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f64f401b980 con 0x55fb667aa330 2024-09-19T07:05:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.240+0000 7f6508645740 1 -- 172.21.15.103:0/374094328 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64f00ba7e0 con 0x55fb667aa330 2024-09-19T07:05:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.240+0000 7f64deffd640 1 -- 172.21.15.103:0/374094328 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f64f401bb80 con 0x55fb667aa330 2024-09-19T07:05:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.240+0000 7f6508645740 10 client.21111 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.240+0000 7f6508645740 1 -- 172.21.15.103:0/374094328 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fb66834980 con 0x55fb667aa330 2024-09-19T07:05:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.240+0000 7f64deffd640 1 -- 172.21.15.103:0/374094328 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f00ba7e0 con 0x55fb667aa330 2024-09-19T07:05:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.240+0000 7f64deffd640 1 -- 172.21.15.103:0/374094328 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f64f4019040 con 0x55fb667aa330 2024-09-19T07:05:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.240+0000 7f6508645740 10 client.21111 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.240+0000 7f6508645740 10 client.21111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.240+0000 7f6508645740 20 client.21111 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.240+0000 7f6508645740 1 --2- 172.21.15.103:0/374094328 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fb66835180 0x55fb668375a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.240+0000 7f6508645740 4 client.21111 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:05:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.240+0000 7f6508645740 1 -- 172.21.15.103:0/374094328 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb66715d70 con 0x55fb66835180 2024-09-19T07:05:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.241+0000 7f64feffd640 1 --2- 172.21.15.103:0/374094328 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fb66835180 0x55fb668375a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.241+0000 7f64deffd640 10 client.21111 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.241+0000 7f64feffd640 1 --2- 172.21.15.103:0/374094328 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fb66835180 0x55fb668375a0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2015be9066f7fb8 server_cookie=3e37f8e1a08a5a73 in_seq=0 out_seq=0 2024-09-19T07:05:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.250+0000 7f64deffd640 1 -- 172.21.15.103:0/374094328 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb66715d70 con 0x55fb66835180 2024-09-19T07:05:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.250+0000 7f64deffd640 10 client.21111 handle_command_reply: tid=0 2024-09-19T07:05:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.250+0000 7f6508645740 1 client.21111 shutdown 2024-09-19T07:05:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.250+0000 7f6508645740 1 -- 172.21.15.103:0/374094328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667aa330 msgr2=0x55fb667bdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.250+0000 7f6508645740 1 --2- 172.21.15.103:0/374094328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667aa330 0x55fb667bdf40 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x55fb667b75a0 tx=0x7f64f400bf10 comp rx=0 tx=0).stop 2024-09-19T07:05:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.250+0000 7f6508645740 1 -- 172.21.15.103:0/374094328 shutdown_connections 2024-09-19T07:05:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.250+0000 7f6508645740 1 --2- 172.21.15.103:0/374094328 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55fb66835180 0x55fb668375a0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.250+0000 7f6508645740 1 --2- 172.21.15.103:0/374094328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb667b90c0 0x55fb6682bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.250+0000 7f6508645740 1 --2- 172.21.15.103:0/374094328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb667aa330 0x55fb667bdf40 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.250+0000 7f6508645740 1 --2- 172.21.15.103:0/374094328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb667a9a10 0x55fb667bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.250+0000 7f6508645740 1 -- 172.21.15.103:0/374094328 >> 172.21.15.103:0/374094328 conn(0x55fb667a6060 msgr2=0x55fb667b6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.251+0000 7f64deffd640 0 client.21111 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.251+0000 7f6508645740 1 -- 172.21.15.103:0/374094328 shutdown_connections 2024-09-19T07:05:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.251+0000 7f6508645740 1 -- 172.21.15.103:0/374094328 wait complete. 2024-09-19T07:05:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.251+0000 7f6508645740 20 client.21111 trim_cache size 0 max 16384 2024-09-19T07:05:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:05:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:05:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:05:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:05:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.251+0000 7f65058a5640 1 -- 172.21.15.103:0/3715000930 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f64e80a3460 msgr2=0x7f64e80a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.251+0000 7f65058a5640 1 --2- 172.21.15.103:0/3715000930 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f64e80a3460 0x7f64e80a5920 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f64f8021000 tx=0x7f64f801e000 comp rx=0 tx=0).stop 2024-09-19T07:05:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.251+0000 7f65058a5640 1 -- 172.21.15.103:0/3715000930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65001009f0 msgr2=0x7f6500199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.251+0000 7f65058a5640 1 --2- 172.21.15.103:0/3715000930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65001009f0 0x7f6500199230 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009230 tx=0x7f64f0004270 comp rx=0 tx=0).stop 2024-09-19T07:05:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.251+0000 7f65058a5640 1 -- 172.21.15.103:0/3715000930 shutdown_connections 2024-09-19T07:05:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.251+0000 7f65058a5640 1 --2- 172.21.15.103:0/3715000930 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f64e80a3460 0x7f64e80a5920 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.251+0000 7f65058a5640 1 --2- 172.21.15.103:0/3715000930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6500106a00 0x7f65001a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.251+0000 7f65058a5640 1 --2- 172.21.15.103:0/3715000930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6500101390 0x7f6500199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.251+0000 7f65058a5640 1 --2- 172.21.15.103:0/3715000930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65001009f0 0x7f6500199230 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.251+0000 7f65058a5640 1 -- 172.21.15.103:0/3715000930 >> 172.21.15.103:0/3715000930 conn(0x7f65000fc6b0 msgr2=0x7f65000fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.251+0000 7f65058a5640 1 -- 172.21.15.103:0/3715000930 shutdown_connections 2024-09-19T07:05:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:11.251+0000 7f65058a5640 1 -- 172.21.15.103:0/3715000930 wait complete. 2024-09-19T07:05:11.574 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:11 smithi103 ceph-mon[31158]: pgmap v499: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 32 KiB/s wr, 31 op/s 2024-09-19T07:05:11.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:11 smithi174 ceph-mon[35637]: pgmap v499: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 32 KiB/s wr, 31 op/s 2024-09-19T07:05:11.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.652+0000 7f0090bbe640 1 -- 172.21.15.103:0/2095648363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f008c101420 msgr2=0x7f008c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:11.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.652+0000 7f0090bbe640 1 --2- 172.21.15.103:0/2095648363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f008c101420 0x7f008c10d950 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f0084009cd0 tx=0x7f008402a8e0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.652+0000 7f0090bbe640 1 -- 172.21.15.103:0/2095648363 shutdown_connections 2024-09-19T07:05:11.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.652+0000 7f0090bbe640 1 --2- 172.21.15.103:0/2095648363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f008c101420 0x7f008c10d950 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.652+0000 7f0090bbe640 1 --2- 172.21.15.103:0/2095648363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f008c100a80 0x7f008c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.652+0000 7f0090bbe640 1 --2- 172.21.15.103:0/2095648363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f008c106a80 0x7f008c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.652+0000 7f0090bbe640 1 -- 172.21.15.103:0/2095648363 >> 172.21.15.103:0/2095648363 conn(0x7f008c0fc7a0 msgr2=0x7f008c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:11.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.652+0000 7f0090bbe640 1 -- 172.21.15.103:0/2095648363 shutdown_connections 2024-09-19T07:05:11.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.652+0000 7f0090bbe640 1 -- 172.21.15.103:0/2095648363 wait complete. 2024-09-19T07:05:11.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.653+0000 7f0090bbe640 1 Processor -- start 2024-09-19T07:05:11.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.653+0000 7f0090bbe640 1 -- start start 2024-09-19T07:05:11.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.653+0000 7f0090bbe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f008c100a80 0x7f008c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.653+0000 7f0090bbe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f008c101420 0x7f008c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.653+0000 7f0090bbe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f008c106a80 0x7f008c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.653+0000 7f0090bbe640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f008c110e80 con 0x7f008c100a80 2024-09-19T07:05:11.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f0090bbe640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f008c110d00 con 0x7f008c101420 2024-09-19T07:05:11.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f0090bbe640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f008c111000 con 0x7f008c106a80 2024-09-19T07:05:11.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f008a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f008c100a80 0x7f008c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f008a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f008c100a80 0x7f008c1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42572/0 (socket says 172.21.15.103:42572) 2024-09-19T07:05:11.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f008a59c640 1 -- 172.21.15.103:0/521563004 learned_addr learned my addr 172.21.15.103:0/521563004 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:11.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f0089d9b640 1 --2- 172.21.15.103:0/521563004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f008c101420 0x7f008c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f008ad9d640 1 --2- 172.21.15.103:0/521563004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f008c106a80 0x7f008c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f008a59c640 1 -- 172.21.15.103:0/521563004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f008c106a80 msgr2=0x7f008c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:11.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f008a59c640 1 --2- 172.21.15.103:0/521563004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f008c106a80 0x7f008c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f008a59c640 1 -- 172.21.15.103:0/521563004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f008c101420 msgr2=0x7f008c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:11.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f008a59c640 1 --2- 172.21.15.103:0/521563004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f008c101420 0x7f008c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f008a59c640 1 -- 172.21.15.103:0/521563004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008c19d450 con 0x7f008c100a80 2024-09-19T07:05:11.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f008a59c640 1 --2- 172.21.15.103:0/521563004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f008c100a80 0x7f008c1984a0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f007c00d950 tx=0x7f007c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f007b7fe640 1 -- 172.21.15.103:0/521563004 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c017070 con 0x7f008c100a80 2024-09-19T07:05:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f0090bbe640 1 -- 172.21.15.103:0/521563004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f008c19d740 con 0x7f008c100a80 2024-09-19T07:05:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f007b7fe640 1 -- 172.21.15.103:0/521563004 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f007c009c30 con 0x7f008c100a80 2024-09-19T07:05:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f007b7fe640 1 -- 172.21.15.103:0/521563004 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c00f6b0 con 0x7f008c100a80 2024-09-19T07:05:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.654+0000 7f0090bbe640 1 -- 172.21.15.103:0/521563004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f008c1a5020 con 0x7f008c100a80 2024-09-19T07:05:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.655+0000 7f0090bbe640 1 -- 172.21.15.103:0/521563004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f008c100ee0 con 0x7f008c100a80 2024-09-19T07:05:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.656+0000 7f007b7fe640 1 -- 172.21.15.103:0/521563004 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f007c0109d0 con 0x7f008c100a80 2024-09-19T07:05:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.657+0000 7f007b7fe640 1 --2- 172.21.15.103:0/521563004 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f00740a33b0 0x7f00740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.657+0000 7f0089d9b640 1 --2- 172.21.15.103:0/521563004 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f00740a33b0 0x7f00740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.657+0000 7f007b7fe640 1 -- 172.21.15.103:0/521563004 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f007c0cf850 con 0x7f008c100a80 2024-09-19T07:05:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.658+0000 7f0089d9b640 1 --2- 172.21.15.103:0/521563004 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f00740a33b0 0x7f00740a5870 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f008c1051e0 tx=0x7f008000b390 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:11.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.661+0000 7f007b7fe640 1 -- 172.21.15.103:0/521563004 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f007c009de0 con 0x7f008c100a80 2024-09-19T07:05:11.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:11 smithi134 ceph-mon[35356]: pgmap v499: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 32 KiB/s wr, 31 op/s 2024-09-19T07:05:11.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.862+0000 7f0090bbe640 1 -- 172.21.15.103:0/521563004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f008c199800 con 0x7f008c100a80 2024-09-19T07:05:11.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.863+0000 7f007b7fe640 1 -- 172.21.15.103:0/521563004 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f007c09bbc0 con 0x7f008c100a80 2024-09-19T07:05:11.864 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:05:11.864 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:05:11.864 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:05:11.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.865+0000 7f0090bbe640 1 -- 172.21.15.103:0/521563004 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f00740a33b0 msgr2=0x7f00740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:11.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.865+0000 7f0090bbe640 1 --2- 172.21.15.103:0/521563004 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f00740a33b0 0x7f00740a5870 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f008c1051e0 tx=0x7f008000b390 comp rx=0 tx=0).stop 2024-09-19T07:05:11.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.865+0000 7f0090bbe640 1 -- 172.21.15.103:0/521563004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f008c100a80 msgr2=0x7f008c1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:11.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.865+0000 7f0090bbe640 1 --2- 172.21.15.103:0/521563004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f008c100a80 0x7f008c1984a0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f007c00d950 tx=0x7f007c00de10 comp rx=0 tx=0).stop 2024-09-19T07:05:11.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.865+0000 7f0090bbe640 1 -- 172.21.15.103:0/521563004 shutdown_connections 2024-09-19T07:05:11.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.865+0000 7f0090bbe640 1 --2- 172.21.15.103:0/521563004 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f00740a33b0 0x7f00740a5870 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.865+0000 7f0090bbe640 1 --2- 172.21.15.103:0/521563004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f008c106a80 0x7f008c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.865+0000 7f0090bbe640 1 --2- 172.21.15.103:0/521563004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f008c101420 0x7f008c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.865+0000 7f0090bbe640 1 --2- 172.21.15.103:0/521563004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f008c100a80 0x7f008c1984a0 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:11.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.865+0000 7f0090bbe640 1 -- 172.21.15.103:0/521563004 >> 172.21.15.103:0/521563004 conn(0x7f008c0fc7a0 msgr2=0x7f008c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:11.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.865+0000 7f0090bbe640 1 -- 172.21.15.103:0/521563004 shutdown_connections 2024-09-19T07:05:11.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:11.865+0000 7f0090bbe640 1 -- 172.21.15.103:0/521563004 wait complete. 2024-09-19T07:05:12.541 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:12 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/521563004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:05:12.582 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:05:12.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:12 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/521563004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:05:12.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:12 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/521563004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:05:12.868 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:13.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.323+0000 7fe384c94640 1 -- 172.21.15.103:0/1462607233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3801002f0 msgr2=0x7fe3801006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:13.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.323+0000 7fe384c94640 1 --2- 172.21.15.103:0/1462607233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3801002f0 0x7fe3801006d0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7fe370009930 tx=0x7fe37002a420 comp rx=0 tx=0).stop 2024-09-19T07:05:13.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.323+0000 7fe384c94640 1 -- 172.21.15.103:0/1462607233 shutdown_connections 2024-09-19T07:05:13.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.323+0000 7fe384c94640 1 --2- 172.21.15.103:0/1462607233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe380106f50 0x7fe38010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.323+0000 7fe384c94640 1 --2- 172.21.15.103:0/1462607233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe380100ca0 0x7fe380106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.323+0000 7fe384c94640 1 --2- 172.21.15.103:0/1462607233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3801002f0 0x7fe3801006d0 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.323+0000 7fe384c94640 1 -- 172.21.15.103:0/1462607233 >> 172.21.15.103:0/1462607233 conn(0x7fe380077210 msgr2=0x7fe3800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:13.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.323+0000 7fe384c94640 1 -- 172.21.15.103:0/1462607233 shutdown_connections 2024-09-19T07:05:13.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.323+0000 7fe384c94640 1 -- 172.21.15.103:0/1462607233 wait complete. 2024-09-19T07:05:13.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.324+0000 7fe384c94640 1 Processor -- start 2024-09-19T07:05:13.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.324+0000 7fe384c94640 1 -- start start 2024-09-19T07:05:13.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.324+0000 7fe384c94640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3801002f0 0x7fe380101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.324+0000 7fe384c94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe380100ca0 0x7fe3801017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.324+0000 7fe384c94640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe380106f50 0x7fe380101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.324+0000 7fe384c94640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe380112000 con 0x7fe380106f50 2024-09-19T07:05:13.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.324+0000 7fe384c94640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe380111e80 con 0x7fe380100ca0 2024-09-19T07:05:13.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.324+0000 7fe384c94640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe380112180 con 0x7fe3801002f0 2024-09-19T07:05:13.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.324+0000 7fe37ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe380106f50 0x7fe380101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:13.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.324+0000 7fe37ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe380106f50 0x7fe380101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42590/0 (socket says 172.21.15.103:42590) 2024-09-19T07:05:13.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.324+0000 7fe37ed9d640 1 -- 172.21.15.103:0/2055735687 learned_addr learned my addr 172.21.15.103:0/2055735687 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:13.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.324+0000 7fe37dd9b640 1 --2- 172.21.15.103:0/2055735687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe380100ca0 0x7fe3801017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:13.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.325+0000 7fe37e59c640 1 --2- 172.21.15.103:0/2055735687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3801002f0 0x7fe380101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:13.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.325+0000 7fe37ed9d640 1 -- 172.21.15.103:0/2055735687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3801002f0 msgr2=0x7fe380101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:13.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.325+0000 7fe37ed9d640 1 --2- 172.21.15.103:0/2055735687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3801002f0 0x7fe380101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.325+0000 7fe37ed9d640 1 -- 172.21.15.103:0/2055735687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe380100ca0 msgr2=0x7fe3801017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:13.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.325+0000 7fe37ed9d640 1 --2- 172.21.15.103:0/2055735687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe380100ca0 0x7fe3801017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.325+0000 7fe37ed9d640 1 -- 172.21.15.103:0/2055735687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3801a6a60 con 0x7fe380106f50 2024-09-19T07:05:13.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.325+0000 7fe37ed9d640 1 --2- 172.21.15.103:0/2055735687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe380106f50 0x7fe380101cf0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7fe37800b970 tx=0x7fe37800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:13.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.325+0000 7fe36f7fe640 1 -- 172.21.15.103:0/2055735687 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe378019070 con 0x7fe380106f50 2024-09-19T07:05:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.325+0000 7fe36f7fe640 1 -- 172.21.15.103:0/2055735687 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe37800cb40 con 0x7fe380106f50 2024-09-19T07:05:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.325+0000 7fe384c94640 1 -- 172.21.15.103:0/2055735687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3801a6d50 con 0x7fe380106f50 2024-09-19T07:05:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.325+0000 7fe36f7fe640 1 -- 172.21.15.103:0/2055735687 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe378007650 con 0x7fe380106f50 2024-09-19T07:05:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.325+0000 7fe384c94640 1 -- 172.21.15.103:0/2055735687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3801a7290 con 0x7fe380106f50 2024-09-19T07:05:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.328+0000 7fe38706d740 1 Processor -- start 2024-09-19T07:05:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.328+0000 7fe36f7fe640 1 -- 172.21.15.103:0/2055735687 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fe37801c070 con 0x7fe380106f50 2024-09-19T07:05:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.328+0000 7fe38706d740 1 -- start start 2024-09-19T07:05:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe38706d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442d2b40 0x557a442d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe38706d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d15d0 0x557a442d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe38706d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442dc370 0x557a442de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe38706d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557a442372f0 con 0x557a442dc370 2024-09-19T07:05:13.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe38706d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557a441decf0 con 0x557a442d15d0 2024-09-19T07:05:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe38706d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557a442332b0 con 0x557a442d2b40 2024-09-19T07:05:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe37dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442dc370 0x557a442de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe37dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442dc370 0x557a442de760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42606/0 (socket says 172.21.15.103:42606) 2024-09-19T07:05:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe37dd9b640 1 -- 172.21.15.103:0/1963703500 learned_addr learned my addr 172.21.15.103:0/1963703500 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe37dd9b640 1 --2- 172.21.15.103:0/1963703500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442d2b40 0x557a442d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe37e59c640 1 --2- 172.21.15.103:0/1963703500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d15d0 0x557a442d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe37dd9b640 1 -- 172.21.15.103:0/1963703500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442d2b40 msgr2=0x557a442d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe37dd9b640 1 --2- 172.21.15.103:0/1963703500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442d2b40 0x557a442d1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe37dd9b640 1 -- 172.21.15.103:0/1963703500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d15d0 msgr2=0x557a442d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe37dd9b640 1 --2- 172.21.15.103:0/1963703500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d15d0 0x557a442d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe36f7fe640 1 --2- 172.21.15.103:0/2055735687 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe3680a33b0 0x7fe3680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe37dd9b640 1 -- 172.21.15.103:0/1963703500 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a441c33a0 con 0x557a442dc370 2024-09-19T07:05:13.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe36f7fe640 1 -- 172.21.15.103:0/2055735687 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fe3780cf120 con 0x7fe380106f50 2024-09-19T07:05:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe37dd9b640 1 --2- 172.21.15.103:0/2055735687 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe3680a33b0 0x7fe3680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.329+0000 7fe37dd9b640 1 --2- 172.21.15.103:0/1963703500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442dc370 0x557a442de760 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7fe374012770 tx=0x7fe374004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48fca6626267011 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.330+0000 7fe36cff9640 1 -- 172.21.15.103:0/1963703500 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374026070 con 0x557a442dc370 2024-09-19T07:05:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.330+0000 7fe36cff9640 1 -- 172.21.15.103:0/1963703500 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe374020b00 con 0x557a442dc370 2024-09-19T07:05:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.330+0000 7fe36cff9640 1 -- 172.21.15.103:0/1963703500 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37401fa00 con 0x557a442dc370 2024-09-19T07:05:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 -- 172.21.15.103:0/1963703500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442dc370 msgr2=0x557a442de760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe37dd9b640 1 --2- 172.21.15.103:0/2055735687 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe3680a33b0 0x7fe3680a5870 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe374023000 tx=0x7fe374020000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 --2- 172.21.15.103:0/1963703500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442dc370 0x557a442de760 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7fe374012770 tx=0x7fe374004730 comp rx=0 tx=0).stop 2024-09-19T07:05:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 -- 172.21.15.103:0/1963703500 shutdown_connections 2024-09-19T07:05:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 --2- 172.21.15.103:0/1963703500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442dc370 0x557a442de760 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 --2- 172.21.15.103:0/1963703500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d15d0 0x557a442d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 --2- 172.21.15.103:0/1963703500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442d2b40 0x557a442d1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 -- 172.21.15.103:0/1963703500 >> 172.21.15.103:0/1963703500 conn(0x557a4429a310 msgr2=0x557a442cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 -- 172.21.15.103:0/1963703500 shutdown_connections 2024-09-19T07:05:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 -- 172.21.15.103:0/1963703500 wait complete. 2024-09-19T07:05:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 -- start start 2024-09-19T07:05:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 Processor -- start 2024-09-19T07:05:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d15d0 0x557a442dbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d2b40 0x557a442d9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442dc370 0x557a442d9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557a442df070 con 0x557a442d15d0 2024-09-19T07:05:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557a441f2060 con 0x557a442d2b40 2024-09-19T07:05:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.331+0000 7fe38706d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557a442df1f0 con 0x557a442dc370 2024-09-19T07:05:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.332+0000 7fe38706d740 10 client.0 _ll_register_callbacks cb 0x557a43f458c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:13.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.332+0000 7fe37e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d15d0 0x557a442dbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:13.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.332+0000 7fe37e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d15d0 0x557a442dbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42608/0 (socket says 172.21.15.103:42608) 2024-09-19T07:05:13.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.332+0000 7fe37e59c640 1 -- 172.21.15.103:0/3224137219 learned_addr learned my addr 172.21.15.103:0/3224137219 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:13.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.332+0000 7fe37dd9b640 1 --2- 172.21.15.103:0/3224137219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d2b40 0x557a442d9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:13.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.332+0000 7fe37ed9d640 1 --2- 172.21.15.103:0/3224137219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442dc370 0x557a442d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:13.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.332+0000 7fe37e59c640 1 -- 172.21.15.103:0/3224137219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442dc370 msgr2=0x557a442d9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:13.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.332+0000 7fe37e59c640 1 --2- 172.21.15.103:0/3224137219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442dc370 0x557a442d9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.332+0000 7fe37e59c640 1 -- 172.21.15.103:0/3224137219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d2b40 msgr2=0x557a442d9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:13.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.332+0000 7fe37e59c640 1 --2- 172.21.15.103:0/3224137219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d2b40 0x557a442d9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.332+0000 7fe37e59c640 1 -- 172.21.15.103:0/3224137219 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a43fa83c0 con 0x557a442d15d0 2024-09-19T07:05:13.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.332+0000 7fe37e59c640 1 --2- 172.21.15.103:0/3224137219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d15d0 0x557a442dbd60 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fe370009930 tx=0x7fe3700319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c702e2d4010177b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:13.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.332+0000 7fe365ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:13.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.335+0000 7fe365ffb640 1 -- 172.21.15.103:0/3224137219 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370035040 con 0x557a442d15d0 2024-09-19T07:05:13.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.335+0000 7fe365ffb640 1 -- 172.21.15.103:0/3224137219 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fe370004240 con 0x557a442d15d0 2024-09-19T07:05:13.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.335+0000 7fe38706d740 1 -- 172.21.15.103:0/3224137219 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe374020b00 con 0x557a442d15d0 2024-09-19T07:05:13.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.335+0000 7fe365ffb640 1 -- 172.21.15.103:0/3224137219 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe370004440 con 0x557a442d15d0 2024-09-19T07:05:13.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.335+0000 7fe38706d740 10 client.21129 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:13.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.335+0000 7fe38706d740 1 -- 172.21.15.103:0/3224137219 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557a442eab40 con 0x557a442d15d0 2024-09-19T07:05:13.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.335+0000 7fe365ffb640 1 -- 172.21.15.103:0/3224137219 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe374020b00 con 0x557a442d15d0 2024-09-19T07:05:13.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.335+0000 7fe365ffb640 1 -- 172.21.15.103:0/3224137219 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe370004c30 con 0x557a442d15d0 2024-09-19T07:05:13.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.335+0000 7fe38706d740 10 client.21129 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:13.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.335+0000 7fe38706d740 10 client.21129 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:05:13.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.335+0000 7fe38706d740 20 client.21129 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:13.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.335+0000 7fe38706d740 1 --2- 172.21.15.103:0/3224137219 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x557a442ebf50 0x557a442ee340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.335+0000 7fe38706d740 4 client.21129 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:13.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.335+0000 7fe38706d740 1 -- 172.21.15.103:0/3224137219 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a44244860 con 0x557a442ebf50 2024-09-19T07:05:13.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.336+0000 7fe37dd9b640 1 --2- 172.21.15.103:0/3224137219 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x557a442ebf50 0x557a442ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:13.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.336+0000 7fe37dd9b640 1 --2- 172.21.15.103:0/3224137219 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x557a442ebf50 0x557a442ee340 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b54660a5e9cfa91e server_cookie=9a7193cfc37a9417 in_seq=0 out_seq=0 2024-09-19T07:05:13.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.336+0000 7fe365ffb640 10 client.21129 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:05:13.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.337+0000 7fe365ffb640 1 -- 172.21.15.103:0/3224137219 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a44244860 con 0x557a442ebf50 2024-09-19T07:05:13.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.337+0000 7fe365ffb640 10 client.21129 handle_command_reply: tid=0 2024-09-19T07:05:13.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.337+0000 7fe38706d740 1 client.21129 shutdown 2024-09-19T07:05:13.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.337+0000 7fe38706d740 1 -- 172.21.15.103:0/3224137219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d15d0 msgr2=0x557a442dbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:13.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.337+0000 7fe38706d740 1 --2- 172.21.15.103:0/3224137219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d15d0 0x557a442dbd60 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fe370009930 tx=0x7fe3700319b0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.337+0000 7fe38706d740 1 -- 172.21.15.103:0/3224137219 shutdown_connections 2024-09-19T07:05:13.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.337+0000 7fe38706d740 1 --2- 172.21.15.103:0/3224137219 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x557a442ebf50 0x557a442ee340 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.337+0000 7fe38706d740 1 --2- 172.21.15.103:0/3224137219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442dc370 0x557a442d9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.337+0000 7fe38706d740 1 --2- 172.21.15.103:0/3224137219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d2b40 0x557a442d9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.337+0000 7fe365ffb640 0 client.21129 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:05:13.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.337+0000 7fe38706d740 1 --2- 172.21.15.103:0/3224137219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d15d0 0x557a442dbd60 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.337+0000 7fe38706d740 1 -- 172.21.15.103:0/3224137219 >> 172.21.15.103:0/3224137219 conn(0x557a4429a310 msgr2=0x557a442d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:13.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.337+0000 7fe38706d740 1 -- 172.21.15.103:0/3224137219 shutdown_connections 2024-09-19T07:05:13.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.337+0000 7fe38706d740 1 -- 172.21.15.103:0/3224137219 wait complete. 2024-09-19T07:05:13.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.337+0000 7fe38706d740 20 client.21129 trim_cache size 0 max 16384 2024-09-19T07:05:13.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe38706d740 1 Processor -- start 2024-09-19T07:05:13.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe38706d740 1 -- start start 2024-09-19T07:05:13.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe38706d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d15d0 0x557a442ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe38706d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d2b40 0x557a442cd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe38706d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442dc370 0x557a442dd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe38706d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3801afb70 con 0x557a442d2b40 2024-09-19T07:05:13.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe38706d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3801b16b0 con 0x557a442d15d0 2024-09-19T07:05:13.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe38706d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3801b0240 con 0x557a442dc370 2024-09-19T07:05:13.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe37dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d2b40 0x557a442cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:13.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe37dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d2b40 0x557a442cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42616/0 (socket says 172.21.15.103:42616) 2024-09-19T07:05:13.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe37dd9b640 1 -- 172.21.15.103:0/2450433723 learned_addr learned my addr 172.21.15.103:0/2450433723 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:13.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe37e59c640 1 --2- 172.21.15.103:0/2450433723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d15d0 0x557a442ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:13.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe37dd9b640 1 -- 172.21.15.103:0/2450433723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442dc370 msgr2=0x557a442dd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:13.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe37dd9b640 1 --2- 172.21.15.103:0/2450433723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442dc370 0x557a442dd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe37dd9b640 1 -- 172.21.15.103:0/2450433723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d15d0 msgr2=0x557a442ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:13.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe37dd9b640 1 --2- 172.21.15.103:0/2450433723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d15d0 0x557a442ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe37dd9b640 1 -- 172.21.15.103:0/2450433723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a442cded0 con 0x557a442d2b40 2024-09-19T07:05:13.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.350+0000 7fe37e59c640 1 --2- 172.21.15.103:0/2450433723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d15d0 0x557a442ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:13.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.351+0000 7fe37dd9b640 1 --2- 172.21.15.103:0/2450433723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d2b40 0x557a442cd130 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fe374022010 tx=0x7fe37401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f9631485cc81ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:13.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.351+0000 7fe367fff640 1 -- 172.21.15.103:0/2450433723 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374026040 con 0x557a442d2b40 2024-09-19T07:05:13.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.351+0000 7fe367fff640 1 -- 172.21.15.103:0/2450433723 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe374016040 con 0x557a442d2b40 2024-09-19T07:05:13.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.351+0000 7fe367fff640 1 -- 172.21.15.103:0/2450433723 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374004040 con 0x557a442d2b40 2024-09-19T07:05:13.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.351+0000 7fe38706d740 1 -- 172.21.15.103:0/2450433723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d2b40 msgr2=0x557a442cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:13.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.351+0000 7fe38706d740 1 --2- 172.21.15.103:0/2450433723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d2b40 0x557a442cd130 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fe374022010 tx=0x7fe37401b040 comp rx=0 tx=0).stop 2024-09-19T07:05:13.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.351+0000 7fe38706d740 1 -- 172.21.15.103:0/2450433723 shutdown_connections 2024-09-19T07:05:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.351+0000 7fe38706d740 1 --2- 172.21.15.103:0/2450433723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442dc370 0x557a442dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.351+0000 7fe38706d740 1 --2- 172.21.15.103:0/2450433723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d2b40 0x557a442cd130 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.351+0000 7fe38706d740 1 --2- 172.21.15.103:0/2450433723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d15d0 0x557a442ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.351+0000 7fe38706d740 1 -- 172.21.15.103:0/2450433723 >> 172.21.15.103:0/2450433723 conn(0x557a4429a310 msgr2=0x557a442cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.351+0000 7fe38706d740 1 -- 172.21.15.103:0/2450433723 shutdown_connections 2024-09-19T07:05:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.351+0000 7fe38706d740 1 -- 172.21.15.103:0/2450433723 wait complete. 2024-09-19T07:05:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe38706d740 1 -- start start 2024-09-19T07:05:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe38706d740 1 Processor -- start 2024-09-19T07:05:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe38706d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d15d0 0x557a442f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe38706d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d2b40 0x557a442f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe38706d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442dc370 0x557a442f4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe38706d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3801b0fe0 con 0x557a442d2b40 2024-09-19T07:05:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe38706d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3801b0910 con 0x557a442d15d0 2024-09-19T07:05:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe38706d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3801adb70 con 0x557a442dc370 2024-09-19T07:05:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe38706d740 10 client.0 _ll_register_callbacks cb 0x7fe3700314d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe37dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d2b40 0x557a442f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe37dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d2b40 0x557a442f3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42622/0 (socket says 172.21.15.103:42622) 2024-09-19T07:05:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe37dd9b640 1 -- 172.21.15.103:0/443488214 learned_addr learned my addr 172.21.15.103:0/443488214 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe37ed9d640 1 --2- 172.21.15.103:0/443488214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442dc370 0x557a442f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe37dd9b640 1 -- 172.21.15.103:0/443488214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442dc370 msgr2=0x557a442f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe37dd9b640 1 --2- 172.21.15.103:0/443488214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442dc370 0x557a442f4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe37dd9b640 1 -- 172.21.15.103:0/443488214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d15d0 msgr2=0x557a442f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe37dd9b640 1 --2- 172.21.15.103:0/443488214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d15d0 0x557a442f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe37dd9b640 1 -- 172.21.15.103:0/443488214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a442e2970 con 0x557a442d2b40 2024-09-19T07:05:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.352+0000 7fe37ed9d640 1 --2- 172.21.15.103:0/443488214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442dc370 0x557a442f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.353+0000 7fe37dd9b640 1 --2- 172.21.15.103:0/443488214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d2b40 0x557a442f3d00 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fe37401f400 tx=0x7fe37400b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33862be80d412136 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.353+0000 7fe365ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.353+0000 7fe365ffb640 1 -- 172.21.15.103:0/443488214 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374041060 con 0x557a442d2b40 2024-09-19T07:05:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.353+0000 7fe365ffb640 1 -- 172.21.15.103:0/443488214 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fe37401f700 con 0x557a442d2b40 2024-09-19T07:05:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.353+0000 7fe38706d740 1 -- 172.21.15.103:0/443488214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe374016040 con 0x557a442d2b40 2024-09-19T07:05:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.353+0000 7fe365ffb640 1 -- 172.21.15.103:0/443488214 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe374026040 con 0x557a442d2b40 2024-09-19T07:05:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.353+0000 7fe38706d740 10 client.21135 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.353+0000 7fe38706d740 1 -- 172.21.15.103:0/443488214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557a44356df0 con 0x557a442d2b40 2024-09-19T07:05:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.353+0000 7fe365ffb640 1 -- 172.21.15.103:0/443488214 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe374016040 con 0x557a442d2b40 2024-09-19T07:05:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.353+0000 7fe365ffb640 1 -- 172.21.15.103:0/443488214 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe374005450 con 0x557a442d2b40 2024-09-19T07:05:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.353+0000 7fe38706d740 10 client.21135 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.353+0000 7fe38706d740 10 client.21135 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:05:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.353+0000 7fe38706d740 20 client.21135 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.353+0000 7fe38706d740 1 --2- 172.21.15.103:0/443488214 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x557a44357400 0x557a44359820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.353+0000 7fe38706d740 4 client.21135 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:05:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.353+0000 7fe38706d740 1 -- 172.21.15.103:0/443488214 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a4423ed70 con 0x557a44357400 2024-09-19T07:05:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe37e59c640 1 --2- 172.21.15.103:0/443488214 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x557a44357400 0x557a44359820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe37e59c640 1 --2- 172.21.15.103:0/443488214 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x557a44357400 0x557a44359820 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b0034db311b9d07a server_cookie=aa1d15508e531674 in_seq=0 out_seq=0 2024-09-19T07:05:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe365ffb640 10 client.21135 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:05:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe365ffb640 1 -- 172.21.15.103:0/443488214 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a4423ed70 con 0x557a44357400 2024-09-19T07:05:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe365ffb640 10 client.21135 handle_command_reply: tid=0 2024-09-19T07:05:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe38706d740 1 client.21135 shutdown 2024-09-19T07:05:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe38706d740 1 -- 172.21.15.103:0/443488214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d2b40 msgr2=0x557a442f3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe38706d740 1 --2- 172.21.15.103:0/443488214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d2b40 0x557a442f3d00 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fe37401f400 tx=0x7fe37400b7e0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe38706d740 1 -- 172.21.15.103:0/443488214 shutdown_connections 2024-09-19T07:05:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe38706d740 1 --2- 172.21.15.103:0/443488214 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x557a44357400 0x557a44359820 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe38706d740 1 --2- 172.21.15.103:0/443488214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a442dc370 0x557a442f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe38706d740 1 --2- 172.21.15.103:0/443488214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a442d2b40 0x557a442f3d00 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe365ffb640 0 client.21135 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:05:13.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe38706d740 1 --2- 172.21.15.103:0/443488214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a442d15d0 0x557a442f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe38706d740 1 -- 172.21.15.103:0/443488214 >> 172.21.15.103:0/443488214 conn(0x557a4429a310 msgr2=0x557a442ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:13.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe38706d740 1 -- 172.21.15.103:0/443488214 shutdown_connections 2024-09-19T07:05:13.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe38706d740 1 -- 172.21.15.103:0/443488214 wait complete. 2024-09-19T07:05:13.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.354+0000 7fe38706d740 20 client.21135 trim_cache size 0 max 16384 2024-09-19T07:05:13.361 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:05:13.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.355+0000 7fe384c94640 1 -- 172.21.15.103:0/2055735687 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe3680a33b0 msgr2=0x7fe3680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:13.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.355+0000 7fe384c94640 1 --2- 172.21.15.103:0/2055735687 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe3680a33b0 0x7fe3680a5870 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe374023000 tx=0x7fe374020000 comp rx=0 tx=0).stop 2024-09-19T07:05:13.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.355+0000 7fe384c94640 1 -- 172.21.15.103:0/2055735687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe380106f50 msgr2=0x7fe380101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:13.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.355+0000 7fe384c94640 1 --2- 172.21.15.103:0/2055735687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe380106f50 0x7fe380101cf0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7fe37800b970 tx=0x7fe37800be30 comp rx=0 tx=0).stop 2024-09-19T07:05:13.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.355+0000 7fe384c94640 1 -- 172.21.15.103:0/2055735687 shutdown_connections 2024-09-19T07:05:13.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.355+0000 7fe384c94640 1 --2- 172.21.15.103:0/2055735687 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe3680a33b0 0x7fe3680a5870 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.355+0000 7fe384c94640 1 --2- 172.21.15.103:0/2055735687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe380106f50 0x7fe380101cf0 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.355+0000 7fe384c94640 1 --2- 172.21.15.103:0/2055735687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe380100ca0 0x7fe3801017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.355+0000 7fe384c94640 1 --2- 172.21.15.103:0/2055735687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3801002f0 0x7fe380101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:13.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.355+0000 7fe384c94640 1 -- 172.21.15.103:0/2055735687 >> 172.21.15.103:0/2055735687 conn(0x7fe380077210 msgr2=0x7fe3801052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:13.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.355+0000 7fe384c94640 1 -- 172.21.15.103:0/2055735687 shutdown_connections 2024-09-19T07:05:13.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:13.355+0000 7fe384c94640 1 -- 172.21.15.103:0/2055735687 wait complete. 2024-09-19T07:05:13.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:13 smithi103 ceph-mon[31158]: pgmap v500: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-19T07:05:13.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:13 smithi134 ceph-mon[35356]: pgmap v500: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-19T07:05:13.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:13 smithi174 ceph-mon[35637]: pgmap v500: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-19T07:05:14.075 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:05:14.332 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:14.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.755+0000 7f9cd85f1640 1 -- 172.21.15.103:0/372051441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0100a80 msgr2=0x7f9cd0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:14.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.755+0000 7f9cd85f1640 1 --2- 172.21.15.103:0/372051441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0100a80 0x7f9cd0100ee0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009930 tx=0x7f9cc402a420 comp rx=0 tx=0).stop 2024-09-19T07:05:14.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.755+0000 7f9cd85f1640 1 -- 172.21.15.103:0/372051441 shutdown_connections 2024-09-19T07:05:14.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.755+0000 7f9cd85f1640 1 --2- 172.21.15.103:0/372051441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cd0101420 0x7f9cd010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.755+0000 7f9cd85f1640 1 --2- 172.21.15.103:0/372051441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0100a80 0x7f9cd0100ee0 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.755+0000 7f9cd85f1640 1 --2- 172.21.15.103:0/372051441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd0106a80 0x7f9cd0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.755+0000 7f9cd85f1640 1 -- 172.21.15.103:0/372051441 >> 172.21.15.103:0/372051441 conn(0x7f9cd00fc7a0 msgr2=0x7f9cd00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.755+0000 7f9cd85f1640 1 -- 172.21.15.103:0/372051441 shutdown_connections 2024-09-19T07:05:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.755+0000 7f9cd85f1640 1 -- 172.21.15.103:0/372051441 wait complete. 2024-09-19T07:05:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.756+0000 7f9cd85f1640 1 Processor -- start 2024-09-19T07:05:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.756+0000 7f9cd85f1640 1 -- start start 2024-09-19T07:05:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.756+0000 7f9cd85f1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd0100a80 0x7f9cd0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.756+0000 7f9cd85f1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cd0101420 0x7f9cd01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.756+0000 7f9cd85f1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0106a80 0x7f9cd019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.756+0000 7f9cd85f1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd0110ea0 con 0x7f9cd0106a80 2024-09-19T07:05:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.756+0000 7f9cd85f1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd0110d20 con 0x7f9cd0101420 2024-09-19T07:05:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.756+0000 7f9cd85f1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd0111020 con 0x7f9cd0100a80 2024-09-19T07:05:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.756+0000 7f9cd6b8e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0106a80 0x7f9cd019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.756+0000 7f9cd6b8e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0106a80 0x7f9cd019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42640/0 (socket says 172.21.15.103:42640) 2024-09-19T07:05:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.756+0000 7f9cd6b8e640 1 -- 172.21.15.103:0/3041479936 learned_addr learned my addr 172.21.15.103:0/3041479936 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.756+0000 7f9cd638d640 1 --2- 172.21.15.103:0/3041479936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd0100a80 0x7f9cd0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.756+0000 7f9cd5b8c640 1 --2- 172.21.15.103:0/3041479936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cd0101420 0x7f9cd01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.757+0000 7f9cd6b8e640 1 -- 172.21.15.103:0/3041479936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd0100a80 msgr2=0x7f9cd0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.757+0000 7f9cd6b8e640 1 --2- 172.21.15.103:0/3041479936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd0100a80 0x7f9cd0198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.757+0000 7f9cd6b8e640 1 -- 172.21.15.103:0/3041479936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cd0101420 msgr2=0x7f9cd01989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.757+0000 7f9cd6b8e640 1 --2- 172.21.15.103:0/3041479936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cd0101420 0x7f9cd01989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.757+0000 7f9cd6b8e640 1 -- 172.21.15.103:0/3041479936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd019d440 con 0x7f9cd0106a80 2024-09-19T07:05:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.757+0000 7f9cd638d640 1 --2- 172.21.15.103:0/3041479936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd0100a80 0x7f9cd0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.757+0000 7f9cd6b8e640 1 --2- 172.21.15.103:0/3041479936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0106a80 0x7f9cd019cd60 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000cce0 tx=0x7f9cc000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.757+0000 7f9ccf7fe640 1 -- 172.21.15.103:0/3041479936 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000f450 con 0x7f9cd0106a80 2024-09-19T07:05:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.757+0000 7f9cd85f1640 1 -- 172.21.15.103:0/3041479936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd019d730 con 0x7f9cd0106a80 2024-09-19T07:05:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.757+0000 7f9ccf7fe640 1 -- 172.21.15.103:0/3041479936 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cc000fad0 con 0x7f9cd0106a80 2024-09-19T07:05:14.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.757+0000 7f9ccf7fe640 1 -- 172.21.15.103:0/3041479936 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0018c10 con 0x7f9cd0106a80 2024-09-19T07:05:14.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.757+0000 7f9cd85f1640 1 -- 172.21.15.103:0/3041479936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd01a5010 con 0x7f9cd0106a80 2024-09-19T07:05:14.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.759+0000 7f9ccf7fe640 1 -- 172.21.15.103:0/3041479936 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9cc000f5f0 con 0x7f9cd0106a80 2024-09-19T07:05:14.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.760+0000 7f9ccf7fe640 1 --2- 172.21.15.103:0/3041479936 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9cb80a3430 0x7f9cb80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.760+0000 7f9cd638d640 1 --2- 172.21.15.103:0/3041479936 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9cb80a3430 0x7f9cb80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.760+0000 7f9ccf7fe640 1 -- 172.21.15.103:0/3041479936 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f9cc00d0390 con 0x7f9cd0106a80 2024-09-19T07:05:14.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.760+0000 7f9cda9ca740 1 Processor -- start 2024-09-19T07:05:14.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.760+0000 7f9cd638d640 1 --2- 172.21.15.103:0/3041479936 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9cb80a3430 0x7f9cb80a58f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8006fd0 tx=0x7f9cc8008040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:14.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cda9ca740 1 -- start start 2024-09-19T07:05:14.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cda9ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d351ab40 0x55a5d3519090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cda9ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d35195d0 0x55a5d3519a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cda9ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d3526760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cda9ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a5d347f2f0 con 0x55a5d3524370 2024-09-19T07:05:14.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cda9ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a5d3426cf0 con 0x55a5d351ab40 2024-09-19T07:05:14.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cda9ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a5d347b2b0 con 0x55a5d35195d0 2024-09-19T07:05:14.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cd6b8e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d3526760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cd6b8e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d3526760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42650/0 (socket says 172.21.15.103:42650) 2024-09-19T07:05:14.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cd6b8e640 1 -- 172.21.15.103:0/1487115455 learned_addr learned my addr 172.21.15.103:0/1487115455 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:14.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cd5b8c640 1 --2- 172.21.15.103:0/1487115455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d35195d0 0x55a5d3519a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cd638d640 1 --2- 172.21.15.103:0/1487115455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d351ab40 0x55a5d3519090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cd6b8e640 1 -- 172.21.15.103:0/1487115455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d35195d0 msgr2=0x55a5d3519a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:14.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cd6b8e640 1 --2- 172.21.15.103:0/1487115455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d35195d0 0x55a5d3519a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cd6b8e640 1 -- 172.21.15.103:0/1487115455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d351ab40 msgr2=0x55a5d3519090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cd6b8e640 1 --2- 172.21.15.103:0/1487115455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d351ab40 0x55a5d3519090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cd6b8e640 1 -- 172.21.15.103:0/1487115455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d340b3a0 con 0x55a5d3524370 2024-09-19T07:05:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cd5b8c640 1 --2- 172.21.15.103:0/1487115455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d35195d0 0x55a5d3519a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cd6b8e640 1 --2- 172.21.15.103:0/1487115455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d3526760 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00a4d90 tx=0x7f9cc009c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69775c1a90dcd19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.761+0000 7f9cccff9640 1 -- 172.21.15.103:0/1487115455 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00b5070 con 0x55a5d3524370 2024-09-19T07:05:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.762+0000 7f9cccff9640 1 -- 172.21.15.103:0/1487115455 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cc00a0a00 con 0x55a5d3524370 2024-09-19T07:05:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.762+0000 7f9cccff9640 1 -- 172.21.15.103:0/1487115455 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00cded0 con 0x55a5d3524370 2024-09-19T07:05:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.762+0000 7f9cda9ca740 1 -- 172.21.15.103:0/1487115455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 msgr2=0x55a5d3526760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.762+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/1487115455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d3526760 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00a4d90 tx=0x7f9cc009c070 comp rx=0 tx=0).stop 2024-09-19T07:05:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.762+0000 7f9cda9ca740 1 -- 172.21.15.103:0/1487115455 shutdown_connections 2024-09-19T07:05:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.762+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/1487115455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d3526760 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.762+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/1487115455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d35195d0 0x55a5d3519a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.762+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/1487115455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d351ab40 0x55a5d3519090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.762+0000 7f9cda9ca740 1 -- 172.21.15.103:0/1487115455 >> 172.21.15.103:0/1487115455 conn(0x55a5d34e2310 msgr2=0x55a5d3515c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.762+0000 7f9cda9ca740 1 -- 172.21.15.103:0/1487115455 shutdown_connections 2024-09-19T07:05:14.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.762+0000 7f9cda9ca740 1 -- 172.21.15.103:0/1487115455 wait complete. 2024-09-19T07:05:14.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.763+0000 7f9cda9ca740 1 -- start start 2024-09-19T07:05:14.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.763+0000 7f9cda9ca740 1 Processor -- start 2024-09-19T07:05:14.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.763+0000 7f9cda9ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d35195d0 0x55a5d3523de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.763+0000 7f9cda9ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d351ab40 0x55a5d3521090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.763+0000 7f9cda9ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d35217d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.763+0000 7f9cda9ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a5d3527070 con 0x55a5d3524370 2024-09-19T07:05:14.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.763+0000 7f9cda9ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a5d343a060 con 0x55a5d35195d0 2024-09-19T07:05:14.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.763+0000 7f9cda9ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a5d35271f0 con 0x55a5d351ab40 2024-09-19T07:05:14.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.763+0000 7f9cda9ca740 10 client.0 _ll_register_callbacks cb 0x55a5d318d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:14.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.763+0000 7f9cd6b8e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d35217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.763+0000 7f9cd6b8e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d35217d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42652/0 (socket says 172.21.15.103:42652) 2024-09-19T07:05:14.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.763+0000 7f9cd6b8e640 1 -- 172.21.15.103:0/4161853217 learned_addr learned my addr 172.21.15.103:0/4161853217 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:14.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.763+0000 7f9cd5b8c640 1 --2- 172.21.15.103:0/4161853217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d351ab40 0x55a5d3521090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.763+0000 7f9cd638d640 1 --2- 172.21.15.103:0/4161853217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d35195d0 0x55a5d3523de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.764+0000 7f9cd6b8e640 1 -- 172.21.15.103:0/4161853217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d351ab40 msgr2=0x55a5d3521090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:14.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.764+0000 7f9cd6b8e640 1 --2- 172.21.15.103:0/4161853217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d351ab40 0x55a5d3521090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.764+0000 7f9cd6b8e640 1 -- 172.21.15.103:0/4161853217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d35195d0 msgr2=0x55a5d3523de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:14.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.764+0000 7f9cd6b8e640 1 --2- 172.21.15.103:0/4161853217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d35195d0 0x55a5d3523de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.764+0000 7f9cd6b8e640 1 -- 172.21.15.103:0/4161853217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d31f03c0 con 0x55a5d3524370 2024-09-19T07:05:14.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.764+0000 7f9cd6b8e640 1 --2- 172.21.15.103:0/4161853217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d35217d0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0098a60 tx=0x7f9cc000feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c6f476c29beb7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:14.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.764+0000 7f9cb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:14.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.764+0000 7f9cb5ffb640 1 -- 172.21.15.103:0/4161853217 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00b9070 con 0x55a5d3524370 2024-09-19T07:05:14.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.764+0000 7f9cb5ffb640 1 -- 172.21.15.103:0/4161853217 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f9cc00a5040 con 0x55a5d3524370 2024-09-19T07:05:14.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.764+0000 7f9cda9ca740 1 -- 172.21.15.103:0/4161853217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cc00a0a00 con 0x55a5d3524370 2024-09-19T07:05:14.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.764+0000 7f9cb5ffb640 1 -- 172.21.15.103:0/4161853217 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9cc00b5040 con 0x55a5d3524370 2024-09-19T07:05:14.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.764+0000 7f9cda9ca740 10 client.21147 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:14.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.764+0000 7f9cda9ca740 1 -- 172.21.15.103:0/4161853217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a5d3532ba0 con 0x55a5d3524370 2024-09-19T07:05:14.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.764+0000 7f9cb5ffb640 1 -- 172.21.15.103:0/4161853217 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cc00a0a00 con 0x55a5d3524370 2024-09-19T07:05:14.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.765+0000 7f9cb5ffb640 1 -- 172.21.15.103:0/4161853217 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9cc009bc30 con 0x55a5d3524370 2024-09-19T07:05:14.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.765+0000 7f9cda9ca740 10 client.21147 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.765+0000 7f9cda9ca740 10 client.21147 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:05:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.765+0000 7f9cda9ca740 20 client.21147 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.765+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/4161853217 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55a5d3533ef0 0x55a5d35362e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.765+0000 7f9cda9ca740 4 client.21147 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.765+0000 7f9cda9ca740 1 -- 172.21.15.103:0/4161853217 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5d348c860 con 0x55a5d3533ef0 2024-09-19T07:05:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.765+0000 7f9cd638d640 1 --2- 172.21.15.103:0/4161853217 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55a5d3533ef0 0x55a5d35362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.765+0000 7f9cb5ffb640 10 client.21147 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:05:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.765+0000 7f9cd638d640 1 --2- 172.21.15.103:0/4161853217 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55a5d3533ef0 0x55a5d35362e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bc2a9feb342a8ac7 server_cookie=5157ea5280534d62 in_seq=0 out_seq=0 2024-09-19T07:05:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.766+0000 7f9cb5ffb640 1 -- 172.21.15.103:0/4161853217 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5d348c860 con 0x55a5d3533ef0 2024-09-19T07:05:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.766+0000 7f9cb5ffb640 10 client.21147 handle_command_reply: tid=0 2024-09-19T07:05:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.766+0000 7f9cda9ca740 1 client.21147 shutdown 2024-09-19T07:05:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.766+0000 7f9cda9ca740 1 -- 172.21.15.103:0/4161853217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 msgr2=0x55a5d35217d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.766+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/4161853217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d35217d0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0098a60 tx=0x7f9cc000feb0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.766+0000 7f9cda9ca740 1 -- 172.21.15.103:0/4161853217 shutdown_connections 2024-09-19T07:05:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.766+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/4161853217 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55a5d3533ef0 0x55a5d35362e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.766+0000 7f9cb5ffb640 0 client.21147 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:05:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.766+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/4161853217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d35217d0 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.766+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/4161853217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d351ab40 0x55a5d3521090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.766+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/4161853217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d35195d0 0x55a5d3523de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.766+0000 7f9cda9ca740 1 -- 172.21.15.103:0/4161853217 >> 172.21.15.103:0/4161853217 conn(0x55a5d34e2310 msgr2=0x55a5d351b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.766+0000 7f9cda9ca740 1 -- 172.21.15.103:0/4161853217 shutdown_connections 2024-09-19T07:05:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.766+0000 7f9cda9ca740 1 -- 172.21.15.103:0/4161853217 wait complete. 2024-09-19T07:05:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.766+0000 7f9cda9ca740 20 client.21147 trim_cache size 0 max 16384 2024-09-19T07:05:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.779+0000 7f9cda9ca740 1 Processor -- start 2024-09-19T07:05:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.779+0000 7f9cda9ca740 1 -- start start 2024-09-19T07:05:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cda9ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d35195d0 0x55a5d3530490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cda9ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d351ab40 0x55a5d3520950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cda9ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d3523000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cda9ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01ad910 con 0x55a5d3524370 2024-09-19T07:05:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cda9ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01af450 con 0x55a5d351ab40 2024-09-19T07:05:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cda9ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01adfe0 con 0x55a5d35195d0 2024-09-19T07:05:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cd6b8e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d3523000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cd6b8e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d3523000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42666/0 (socket says 172.21.15.103:42666) 2024-09-19T07:05:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cd6b8e640 1 -- 172.21.15.103:0/2012834817 learned_addr learned my addr 172.21.15.103:0/2012834817 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cd5b8c640 1 --2- 172.21.15.103:0/2012834817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d351ab40 0x55a5d3520950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cd638d640 1 --2- 172.21.15.103:0/2012834817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d35195d0 0x55a5d3530490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cd6b8e640 1 -- 172.21.15.103:0/2012834817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d35195d0 msgr2=0x55a5d3530490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cd6b8e640 1 --2- 172.21.15.103:0/2012834817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d35195d0 0x55a5d3530490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cd6b8e640 1 -- 172.21.15.103:0/2012834817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d351ab40 msgr2=0x55a5d3520950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cd6b8e640 1 --2- 172.21.15.103:0/2012834817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d351ab40 0x55a5d3520950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cd6b8e640 1 -- 172.21.15.103:0/2012834817 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d3516960 con 0x55a5d3524370 2024-09-19T07:05:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cd638d640 1 --2- 172.21.15.103:0/2012834817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d35195d0 0x55a5d3530490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cd6b8e640 1 --2- 172.21.15.103:0/2012834817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d3523000 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00ce0c0 tx=0x7f9cc000ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20ea733d7ce62ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cb7fff640 1 -- 172.21.15.103:0/2012834817 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00b9070 con 0x55a5d3524370 2024-09-19T07:05:14.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cb7fff640 1 -- 172.21.15.103:0/2012834817 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cc000f8e0 con 0x55a5d3524370 2024-09-19T07:05:14.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.780+0000 7f9cb7fff640 1 -- 172.21.15.103:0/2012834817 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00a1d30 con 0x55a5d3524370 2024-09-19T07:05:14.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.781+0000 7f9cda9ca740 1 -- 172.21.15.103:0/2012834817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 msgr2=0x55a5d3523000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:14.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.781+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/2012834817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d3523000 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00ce0c0 tx=0x7f9cc000ac50 comp rx=0 tx=0).stop 2024-09-19T07:05:14.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.781+0000 7f9cda9ca740 1 -- 172.21.15.103:0/2012834817 shutdown_connections 2024-09-19T07:05:14.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.781+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/2012834817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d3524370 0x55a5d3523000 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.781+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/2012834817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d351ab40 0x55a5d3520950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.781+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/2012834817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d35195d0 0x55a5d3530490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.781+0000 7f9cda9ca740 1 -- 172.21.15.103:0/2012834817 >> 172.21.15.103:0/2012834817 conn(0x55a5d34e2310 msgr2=0x55a5d35228f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.781+0000 7f9cda9ca740 1 -- 172.21.15.103:0/2012834817 shutdown_connections 2024-09-19T07:05:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.781+0000 7f9cda9ca740 1 -- 172.21.15.103:0/2012834817 wait complete. 2024-09-19T07:05:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.781+0000 7f9cda9ca740 1 -- start start 2024-09-19T07:05:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.781+0000 7f9cda9ca740 1 Processor -- start 2024-09-19T07:05:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cda9ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d35195d0 0x55a5d3539440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cda9ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d351ab40 0x55a5d3539980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cda9ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d3524370 0x55a5d3536890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cda9ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01aed80 con 0x55a5d35195d0 2024-09-19T07:05:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cda9ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01ae6b0 con 0x55a5d351ab40 2024-09-19T07:05:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cda9ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01ab910 con 0x55a5d3524370 2024-09-19T07:05:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cda9ca740 10 client.0 _ll_register_callbacks cb 0x7f9cc00b3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cd638d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d35195d0 0x55a5d3539440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cd638d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d35195d0 0x55a5d3539440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42682/0 (socket says 172.21.15.103:42682) 2024-09-19T07:05:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cd638d640 1 -- 172.21.15.103:0/2342364627 learned_addr learned my addr 172.21.15.103:0/2342364627 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cd6b8e640 1 --2- 172.21.15.103:0/2342364627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d3524370 0x55a5d3536890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cd638d640 1 -- 172.21.15.103:0/2342364627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d3524370 msgr2=0x55a5d3536890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cd5b8c640 1 --2- 172.21.15.103:0/2342364627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d351ab40 0x55a5d3539980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cd638d640 1 --2- 172.21.15.103:0/2342364627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d3524370 0x55a5d3536890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cd638d640 1 -- 172.21.15.103:0/2342364627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d351ab40 msgr2=0x55a5d3539980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cd638d640 1 --2- 172.21.15.103:0/2342364627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d351ab40 0x55a5d3539980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cd638d640 1 -- 172.21.15.103:0/2342364627 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d35154f0 con 0x55a5d35195d0 2024-09-19T07:05:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cd6b8e640 1 --2- 172.21.15.103:0/2342364627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d3524370 0x55a5d3536890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cd5b8c640 1 --2- 172.21.15.103:0/2342364627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d351ab40 0x55a5d3539980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cd638d640 1 --2- 172.21.15.103:0/2342364627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d35195d0 0x55a5d3539440 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800a1a0 tx=0x7f9cc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84f4cb1bb2db9a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cb5ffb640 1 -- 172.21.15.103:0/2342364627 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8022ab0 con 0x55a5d35195d0 2024-09-19T07:05:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.782+0000 7f9cb5ffb640 1 -- 172.21.15.103:0/2342364627 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f9cc8022c50 con 0x55a5d35195d0 2024-09-19T07:05:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.783+0000 7f9cda9ca740 1 -- 172.21.15.103:0/2342364627 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cc000f8e0 con 0x55a5d35195d0 2024-09-19T07:05:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.783+0000 7f9cb5ffb640 1 -- 172.21.15.103:0/2342364627 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9cc8022e50 con 0x55a5d35195d0 2024-09-19T07:05:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.783+0000 7f9cda9ca740 10 client.21153 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.783+0000 7f9cda9ca740 1 -- 172.21.15.103:0/2342364627 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a5d359e120 con 0x55a5d35195d0 2024-09-19T07:05:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.783+0000 7f9cb5ffb640 1 -- 172.21.15.103:0/2342364627 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cc000f8e0 con 0x55a5d35195d0 2024-09-19T07:05:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.783+0000 7f9cb5ffb640 1 -- 172.21.15.103:0/2342364627 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9cc80135e0 con 0x55a5d35195d0 2024-09-19T07:05:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.783+0000 7f9cda9ca740 10 client.21153 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.783+0000 7f9cda9ca740 10 client.21153 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:05:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.783+0000 7f9cda9ca740 20 client.21153 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:14.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.783+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/2342364627 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55a5d3533ef0 0x55a5d35a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.783+0000 7f9cda9ca740 4 client.21153 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:05:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.783+0000 7f9cda9ca740 1 -- 172.21.15.103:0/2342364627 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5d3486d70 con 0x55a5d3533ef0 2024-09-19T07:05:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.783+0000 7f9cd5b8c640 1 --2- 172.21.15.103:0/2342364627 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55a5d3533ef0 0x55a5d35a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cb5ffb640 10 client.21153 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:05:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cd5b8c640 1 --2- 172.21.15.103:0/2342364627 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55a5d3533ef0 0x55a5d35a08e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5d7ad2b9df577e40 server_cookie=deaa332575626e3a in_seq=0 out_seq=0 2024-09-19T07:05:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cb5ffb640 1 -- 172.21.15.103:0/2342364627 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5d3486d70 con 0x55a5d3533ef0 2024-09-19T07:05:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cb5ffb640 10 client.21153 handle_command_reply: tid=0 2024-09-19T07:05:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cda9ca740 1 client.21153 shutdown 2024-09-19T07:05:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cda9ca740 1 -- 172.21.15.103:0/2342364627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d35195d0 msgr2=0x55a5d3539440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/2342364627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d35195d0 0x55a5d3539440 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800a1a0 tx=0x7f9cc80046c0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cda9ca740 1 -- 172.21.15.103:0/2342364627 shutdown_connections 2024-09-19T07:05:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/2342364627 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55a5d3533ef0 0x55a5d35a08e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/2342364627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5d3524370 0x55a5d3536890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cb5ffb640 0 client.21153 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:05:14.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/2342364627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5d351ab40 0x55a5d3539980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cda9ca740 1 --2- 172.21.15.103:0/2342364627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5d35195d0 0x55a5d3539440 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cda9ca740 1 -- 172.21.15.103:0/2342364627 >> 172.21.15.103:0/2342364627 conn(0x55a5d34e2310 msgr2=0x55a5d3514d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:14.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cda9ca740 1 -- 172.21.15.103:0/2342364627 shutdown_connections 2024-09-19T07:05:14.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cda9ca740 1 -- 172.21.15.103:0/2342364627 wait complete. 2024-09-19T07:05:14.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.784+0000 7f9cda9ca740 20 client.21153 trim_cache size 0 max 16384 2024-09-19T07:05:14.791 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:05:14.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.785+0000 7f9cd85f1640 1 -- 172.21.15.103:0/3041479936 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9cb80a3430 msgr2=0x7f9cb80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:14.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.785+0000 7f9cd85f1640 1 --2- 172.21.15.103:0/3041479936 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9cb80a3430 0x7f9cb80a58f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8006fd0 tx=0x7f9cc8008040 comp rx=0 tx=0).stop 2024-09-19T07:05:14.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.785+0000 7f9cd85f1640 1 -- 172.21.15.103:0/3041479936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0106a80 msgr2=0x7f9cd019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:14.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.785+0000 7f9cd85f1640 1 --2- 172.21.15.103:0/3041479936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0106a80 0x7f9cd019cd60 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000cce0 tx=0x7f9cc000a620 comp rx=0 tx=0).stop 2024-09-19T07:05:14.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.785+0000 7f9cd85f1640 1 -- 172.21.15.103:0/3041479936 shutdown_connections 2024-09-19T07:05:14.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.785+0000 7f9cd85f1640 1 --2- 172.21.15.103:0/3041479936 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9cb80a3430 0x7f9cb80a58f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.785+0000 7f9cd85f1640 1 --2- 172.21.15.103:0/3041479936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cd0106a80 0x7f9cd019cd60 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.785+0000 7f9cd85f1640 1 --2- 172.21.15.103:0/3041479936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cd0101420 0x7f9cd01989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.785+0000 7f9cd85f1640 1 --2- 172.21.15.103:0/3041479936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd0100a80 0x7f9cd0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:14.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.785+0000 7f9cd85f1640 1 -- 172.21.15.103:0/3041479936 >> 172.21.15.103:0/3041479936 conn(0x7f9cd00fc7a0 msgr2=0x7f9cd006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:14.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.785+0000 7f9cd85f1640 1 -- 172.21.15.103:0/3041479936 shutdown_connections 2024-09-19T07:05:14.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:14.785+0000 7f9cd85f1640 1 -- 172.21.15.103:0/3041479936 wait complete. 2024-09-19T07:05:15.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:15 smithi103 ceph-mon[31158]: pgmap v501: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-19T07:05:15.491 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:05:15.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:15 smithi134 ceph-mon[35356]: pgmap v501: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-19T07:05:15.736 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:15.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:15 smithi174 ceph-mon[35637]: pgmap v501: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-19T07:05:16.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.156+0000 7fc7d51ee640 1 -- 172.21.15.103:0/1941582797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7d01036c0 msgr2=0x7fc7d010bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:16.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.156+0000 7fc7d51ee640 1 --2- 172.21.15.103:0/1941582797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7d01036c0 0x7fc7d010bbb0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009900 tx=0x7fc7c402a480 comp rx=0 tx=0).stop 2024-09-19T07:05:16.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.156+0000 7fc7d51ee640 1 -- 172.21.15.103:0/1941582797 shutdown_connections 2024-09-19T07:05:16.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.156+0000 7fc7d51ee640 1 --2- 172.21.15.103:0/1941582797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7d010c180 0x7fc7d010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.156+0000 7fc7d51ee640 1 --2- 172.21.15.103:0/1941582797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7d01036c0 0x7fc7d010bbb0 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.156+0000 7fc7d51ee640 1 --2- 172.21.15.103:0/1941582797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7d0102da0 0x7fc7d0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.156+0000 7fc7d51ee640 1 -- 172.21.15.103:0/1941582797 >> 172.21.15.103:0/1941582797 conn(0x7fc7d00fc7a0 msgr2=0x7fc7d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:16.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.156+0000 7fc7d51ee640 1 -- 172.21.15.103:0/1941582797 shutdown_connections 2024-09-19T07:05:16.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.156+0000 7fc7d51ee640 1 -- 172.21.15.103:0/1941582797 wait complete. 2024-09-19T07:05:16.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.157+0000 7fc7d51ee640 1 Processor -- start 2024-09-19T07:05:16.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.157+0000 7fc7d51ee640 1 -- start start 2024-09-19T07:05:16.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.157+0000 7fc7d51ee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7d0102da0 0x7fc7d019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.157+0000 7fc7d51ee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7d01036c0 0x7fc7d019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.157+0000 7fc7d51ee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7d010c180 0x7fc7d019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.157+0000 7fc7d51ee640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d0111f60 con 0x7fc7d01036c0 2024-09-19T07:05:16.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.157+0000 7fc7d51ee640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d0111de0 con 0x7fc7d0102da0 2024-09-19T07:05:16.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.157+0000 7fc7d51ee640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d01120e0 con 0x7fc7d010c180 2024-09-19T07:05:16.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7d01036c0 0x7fc7d019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7d01036c0 0x7fc7d019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42706/0 (socket says 172.21.15.103:42706) 2024-09-19T07:05:16.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7ce59c640 1 -- 172.21.15.103:0/728996768 learned_addr learned my addr 172.21.15.103:0/728996768 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:16.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7ced9d640 1 --2- 172.21.15.103:0/728996768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7d0102da0 0x7fc7d019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7cf59e640 1 --2- 172.21.15.103:0/728996768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7d010c180 0x7fc7d019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7ce59c640 1 -- 172.21.15.103:0/728996768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7d010c180 msgr2=0x7fc7d019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:16.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7ce59c640 1 --2- 172.21.15.103:0/728996768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7d010c180 0x7fc7d019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7ce59c640 1 -- 172.21.15.103:0/728996768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7d0102da0 msgr2=0x7fc7d019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:16.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7ce59c640 1 --2- 172.21.15.103:0/728996768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7d0102da0 0x7fc7d019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7ce59c640 1 -- 172.21.15.103:0/728996768 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d019f590 con 0x7fc7d01036c0 2024-09-19T07:05:16.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7ced9d640 1 --2- 172.21.15.103:0/728996768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7d0102da0 0x7fc7d019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:16.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7ce59c640 1 --2- 172.21.15.103:0/728996768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7d01036c0 0x7fc7d019ab20 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4006010 tx=0x7fc7c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:16.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7b7fff640 1 -- 172.21.15.103:0/728996768 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c403f030 con 0x7fc7d01036c0 2024-09-19T07:05:16.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7d51ee640 1 -- 172.21.15.103:0/728996768 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d019f820 con 0x7fc7d01036c0 2024-09-19T07:05:16.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7b7fff640 1 -- 172.21.15.103:0/728996768 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c4003ce0 con 0x7fc7d01036c0 2024-09-19T07:05:16.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7b7fff640 1 -- 172.21.15.103:0/728996768 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4035070 con 0x7fc7d01036c0 2024-09-19T07:05:16.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.158+0000 7fc7d51ee640 1 -- 172.21.15.103:0/728996768 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d01a7160 con 0x7fc7d01036c0 2024-09-19T07:05:16.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.161+0000 7fc7b7fff640 1 -- 172.21.15.103:0/728996768 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fc7c4042030 con 0x7fc7d01036c0 2024-09-19T07:05:16.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.161+0000 7fc7d75c7740 1 Processor -- start 2024-09-19T07:05:16.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.161+0000 7fc7d75c7740 1 -- start start 2024-09-19T07:05:16.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.161+0000 7fc7d75c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec77b40 0x55f5bec76090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.161+0000 7fc7d75c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec76a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.161+0000 7fc7d75c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec81370 0x55f5bec83760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.161+0000 7fc7d75c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f5bebdc2f0 con 0x55f5bec765d0 2024-09-19T07:05:16.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.161+0000 7fc7d75c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f5beb83cf0 con 0x55f5bec77b40 2024-09-19T07:05:16.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.161+0000 7fc7d75c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f5bebd82b0 con 0x55f5bec81370 2024-09-19T07:05:16.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.161+0000 7fc7b7fff640 1 --2- 172.21.15.103:0/728996768 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7b80a33b0 0x7fc7b80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.161+0000 7fc7ced9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec76a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.161+0000 7fc7ced9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec76a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42714/0 (socket says 172.21.15.103:42714) 2024-09-19T07:05:16.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.161+0000 7fc7ced9d640 1 -- 172.21.15.103:0/3790778410 learned_addr learned my addr 172.21.15.103:0/3790778410 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:16.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.162+0000 7fc7cf59e640 1 --2- 172.21.15.103:0/728996768 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7b80a33b0 0x7fc7b80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.162+0000 7fc7cf59e640 1 --2- 172.21.15.103:0/3790778410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec77b40 0x55f5bec76090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.162+0000 7fc7ce59c640 1 --2- 172.21.15.103:0/3790778410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec81370 0x55f5bec83760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.162+0000 7fc7b7fff640 1 -- 172.21.15.103:0/728996768 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fc7c40f04b0 con 0x7fc7d01036c0 2024-09-19T07:05:16.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.162+0000 7fc7ced9d640 1 -- 172.21.15.103:0/3790778410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec81370 msgr2=0x55f5bec83760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:16.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.162+0000 7fc7ced9d640 1 --2- 172.21.15.103:0/3790778410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec81370 0x55f5bec83760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.162+0000 7fc7ced9d640 1 -- 172.21.15.103:0/3790778410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec77b40 msgr2=0x55f5bec76090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:16.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.162+0000 7fc7ced9d640 1 --2- 172.21.15.103:0/3790778410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec77b40 0x55f5bec76090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.162+0000 7fc7ced9d640 1 -- 172.21.15.103:0/3790778410 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5beb683a0 con 0x55f5bec765d0 2024-09-19T07:05:16.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.162+0000 7fc7ced9d640 1 --2- 172.21.15.103:0/3790778410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec76a10 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8008c60 tx=0x7fc7c800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5f0e0d4dadc5ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:16.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.162+0000 7fc7cf59e640 1 --2- 172.21.15.103:0/728996768 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7b80a33b0 0x7fc7b80a5870 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00092f0 tx=0x7fc7c0009280 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:16.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.162+0000 7fc7b57fa640 1 -- 172.21.15.103:0/3790778410 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8014070 con 0x55f5bec765d0 2024-09-19T07:05:16.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.162+0000 7fc7b57fa640 1 -- 172.21.15.103:0/3790778410 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c80047a0 con 0x55f5bec765d0 2024-09-19T07:05:16.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.162+0000 7fc7b57fa640 1 -- 172.21.15.103:0/3790778410 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8010760 con 0x55f5bec765d0 2024-09-19T07:05:16.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.163+0000 7fc7d75c7740 1 -- 172.21.15.103:0/3790778410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 msgr2=0x55f5bec76a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:16.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.163+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/3790778410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec76a10 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8008c60 tx=0x7fc7c800e720 comp rx=0 tx=0).stop 2024-09-19T07:05:16.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.163+0000 7fc7d75c7740 1 -- 172.21.15.103:0/3790778410 shutdown_connections 2024-09-19T07:05:16.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.163+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/3790778410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec81370 0x55f5bec83760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.163+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/3790778410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec76a10 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.163+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/3790778410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec77b40 0x55f5bec76090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.163+0000 7fc7d75c7740 1 -- 172.21.15.103:0/3790778410 >> 172.21.15.103:0/3790778410 conn(0x55f5bec3f310 msgr2=0x55f5bec72c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:16.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.163+0000 7fc7d75c7740 1 -- 172.21.15.103:0/3790778410 shutdown_connections 2024-09-19T07:05:16.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.163+0000 7fc7d75c7740 1 -- 172.21.15.103:0/3790778410 wait complete. 2024-09-19T07:05:16.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.163+0000 7fc7d75c7740 1 -- start start 2024-09-19T07:05:16.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.163+0000 7fc7d75c7740 1 Processor -- start 2024-09-19T07:05:16.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7d75c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec80d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7d75c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec77b40 0x55f5bec7dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7d75c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec81370 0x55f5bec7e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7d75c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f5bec84070 con 0x55f5bec765d0 2024-09-19T07:05:16.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7d75c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f5beb97060 con 0x55f5bec81370 2024-09-19T07:05:16.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7d75c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f5bec841f0 con 0x55f5bec77b40 2024-09-19T07:05:16.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7d75c7740 10 client.0 _ll_register_callbacks cb 0x55f5be8ea8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:16.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7ced9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec80d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7ced9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec80d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42726/0 (socket says 172.21.15.103:42726) 2024-09-19T07:05:16.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7ced9d640 1 -- 172.21.15.103:0/696630455 learned_addr learned my addr 172.21.15.103:0/696630455 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:16.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7ce59c640 1 --2- 172.21.15.103:0/696630455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec77b40 0x55f5bec7dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7cf59e640 1 --2- 172.21.15.103:0/696630455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec81370 0x55f5bec7e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7ced9d640 1 -- 172.21.15.103:0/696630455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec77b40 msgr2=0x55f5bec7dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:16.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7ced9d640 1 --2- 172.21.15.103:0/696630455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec77b40 0x55f5bec7dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7ced9d640 1 -- 172.21.15.103:0/696630455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec81370 msgr2=0x55f5bec7e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:16.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7ced9d640 1 --2- 172.21.15.103:0/696630455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec81370 0x55f5bec7e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7ced9d640 1 -- 172.21.15.103:0/696630455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5be94d3c0 con 0x55f5bec765d0 2024-09-19T07:05:16.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7cf59e640 1 --2- 172.21.15.103:0/696630455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec81370 0x55f5bec7e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:16.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7ced9d640 1 --2- 172.21.15.103:0/696630455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec80d40 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800f010 tx=0x7fc7c8005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7badb873a59c74a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:16.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:16.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7b2ff5640 1 -- 172.21.15.103:0/696630455 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8027060 con 0x55f5bec765d0 2024-09-19T07:05:16.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7b2ff5640 1 -- 172.21.15.103:0/696630455 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fc7c80054e0 con 0x55f5bec765d0 2024-09-19T07:05:16.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.164+0000 7fc7d75c7740 1 -- 172.21.15.103:0/696630455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7c80047a0 con 0x55f5bec765d0 2024-09-19T07:05:16.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.165+0000 7fc7b2ff5640 1 -- 172.21.15.103:0/696630455 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc7c8014070 con 0x55f5bec765d0 2024-09-19T07:05:16.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.165+0000 7fc7d75c7740 10 client.21165 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:16.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.165+0000 7fc7d75c7740 1 -- 172.21.15.103:0/696630455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5bec8fae0 con 0x55f5bec765d0 2024-09-19T07:05:16.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.165+0000 7fc7b2ff5640 1 -- 172.21.15.103:0/696630455 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c80047a0 con 0x55f5bec765d0 2024-09-19T07:05:16.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.165+0000 7fc7b2ff5640 1 -- 172.21.15.103:0/696630455 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc7c80184a0 con 0x55f5bec765d0 2024-09-19T07:05:16.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.165+0000 7fc7d75c7740 10 client.21165 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:16.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.165+0000 7fc7d75c7740 10 client.21165 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:05:16.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.165+0000 7fc7d75c7740 20 client.21165 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:16.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.165+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/696630455 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55f5bec90e30 0x55f5bec93220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.165+0000 7fc7d75c7740 4 client.21165 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:16.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.165+0000 7fc7d75c7740 1 -- 172.21.15.103:0/696630455 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5bebe9860 con 0x55f5bec90e30 2024-09-19T07:05:16.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.165+0000 7fc7ce59c640 1 --2- 172.21.15.103:0/696630455 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55f5bec90e30 0x55f5bec93220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.166+0000 7fc7b2ff5640 10 client.21165 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:05:16.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.166+0000 7fc7ce59c640 1 --2- 172.21.15.103:0/696630455 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55f5bec90e30 0x55f5bec93220 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a5353895fd7378e server_cookie=52dd67e7402cdb04 in_seq=0 out_seq=0 2024-09-19T07:05:16.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.166+0000 7fc7b2ff5640 1 -- 172.21.15.103:0/696630455 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5bebe9860 con 0x55f5bec90e30 2024-09-19T07:05:16.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.166+0000 7fc7b2ff5640 10 client.21165 handle_command_reply: tid=0 2024-09-19T07:05:16.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.166+0000 7fc7d75c7740 1 client.21165 shutdown 2024-09-19T07:05:16.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.167+0000 7fc7d75c7740 1 -- 172.21.15.103:0/696630455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 msgr2=0x55f5bec80d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:16.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.167+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/696630455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec80d40 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800f010 tx=0x7fc7c8005160 comp rx=0 tx=0).stop 2024-09-19T07:05:16.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.167+0000 7fc7d75c7740 1 -- 172.21.15.103:0/696630455 shutdown_connections 2024-09-19T07:05:16.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.167+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/696630455 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55f5bec90e30 0x55f5bec93220 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.167+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/696630455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec81370 0x55f5bec7e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.167+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/696630455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec77b40 0x55f5bec7dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.167+0000 7fc7b2ff5640 0 client.21165 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:05:16.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.167+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/696630455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec80d40 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.167+0000 7fc7d75c7740 1 -- 172.21.15.103:0/696630455 >> 172.21.15.103:0/696630455 conn(0x55f5bec3f310 msgr2=0x55f5bec78960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:16.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.167+0000 7fc7d75c7740 1 -- 172.21.15.103:0/696630455 shutdown_connections 2024-09-19T07:05:16.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.167+0000 7fc7d75c7740 1 -- 172.21.15.103:0/696630455 wait complete. 2024-09-19T07:05:16.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.167+0000 7fc7d75c7740 20 client.21165 trim_cache size 0 max 16384 2024-09-19T07:05:16.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.179+0000 7fc7d75c7740 1 Processor -- start 2024-09-19T07:05:16.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.179+0000 7fc7d75c7740 1 -- start start 2024-09-19T07:05:16.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7d75c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec765d0 0x55f5bec7d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7d75c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec77b40 0x55f5bec71a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7d75c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec81370 0x55f5bec71f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7d75c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d01afad0 con 0x55f5bec77b40 2024-09-19T07:05:16.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7d75c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d01b1610 con 0x55f5bec81370 2024-09-19T07:05:16.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7d75c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d01b01a0 con 0x55f5bec765d0 2024-09-19T07:05:16.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec77b40 0x55f5bec71a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec77b40 0x55f5bec71a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42730/0 (socket says 172.21.15.103:42730) 2024-09-19T07:05:16.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7ce59c640 1 -- 172.21.15.103:0/1911125052 learned_addr learned my addr 172.21.15.103:0/1911125052 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:16.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7ced9d640 1 --2- 172.21.15.103:0/1911125052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec765d0 0x55f5bec7d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7ce59c640 1 -- 172.21.15.103:0/1911125052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec765d0 msgr2=0x55f5bec7d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:16.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7ce59c640 1 --2- 172.21.15.103:0/1911125052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec765d0 0x55f5bec7d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7ce59c640 1 -- 172.21.15.103:0/1911125052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec81370 msgr2=0x55f5bec71f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:16.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7cf59e640 1 --2- 172.21.15.103:0/1911125052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec81370 0x55f5bec71f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7ce59c640 1 --2- 172.21.15.103:0/1911125052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec81370 0x55f5bec71f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7ce59c640 1 -- 172.21.15.103:0/1911125052 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5bec93590 con 0x55f5bec77b40 2024-09-19T07:05:16.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7ced9d640 1 --2- 172.21.15.103:0/1911125052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec765d0 0x55f5bec7d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:16.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7cf59e640 1 --2- 172.21.15.103:0/1911125052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec81370 0x55f5bec71f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:16.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7ce59c640 1 --2- 172.21.15.103:0/1911125052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec77b40 0x55f5bec71a20 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40edc40 tx=0x7fc7c4036b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9070e65986ae4d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:16.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7b4ff9640 1 -- 172.21.15.103:0/1911125052 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4049070 con 0x55f5bec77b40 2024-09-19T07:05:16.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7b4ff9640 1 -- 172.21.15.103:0/1911125052 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c403b580 con 0x55f5bec77b40 2024-09-19T07:05:16.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.180+0000 7fc7b4ff9640 1 -- 172.21.15.103:0/1911125052 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40bc830 con 0x55f5bec77b40 2024-09-19T07:05:16.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.181+0000 7fc7d75c7740 1 -- 172.21.15.103:0/1911125052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec77b40 msgr2=0x55f5bec71a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:16.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.181+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/1911125052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec77b40 0x55f5bec71a20 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40edc40 tx=0x7fc7c4036b60 comp rx=0 tx=0).stop 2024-09-19T07:05:16.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.181+0000 7fc7d75c7740 1 -- 172.21.15.103:0/1911125052 shutdown_connections 2024-09-19T07:05:16.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.181+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/1911125052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec81370 0x55f5bec71f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.181+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/1911125052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec77b40 0x55f5bec71a20 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.181+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/1911125052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec765d0 0x55f5bec7d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.181+0000 7fc7d75c7740 1 -- 172.21.15.103:0/1911125052 >> 172.21.15.103:0/1911125052 conn(0x55f5bec3f310 msgr2=0x55f5bec71310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:16.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.181+0000 7fc7d75c7740 1 -- 172.21.15.103:0/1911125052 shutdown_connections 2024-09-19T07:05:16.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.181+0000 7fc7d75c7740 1 -- 172.21.15.103:0/1911125052 wait complete. 2024-09-19T07:05:16.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.181+0000 7fc7d75c7740 1 -- start start 2024-09-19T07:05:16.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.181+0000 7fc7d75c7740 1 Processor -- start 2024-09-19T07:05:16.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7d75c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec89a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7d75c7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec77b40 0x55f5bec84d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7d75c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec81370 0x55f5bec85280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7d75c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d01b0f40 con 0x55f5bec765d0 2024-09-19T07:05:16.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7d75c7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d01b0870 con 0x55f5bec77b40 2024-09-19T07:05:16.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7d75c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d01adad0 con 0x55f5bec81370 2024-09-19T07:05:16.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7d75c7740 10 client.0 _ll_register_callbacks cb 0x7fc7c0009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:16.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7ced9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec89a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7ced9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec89a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42732/0 (socket says 172.21.15.103:42732) 2024-09-19T07:05:16.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7ced9d640 1 -- 172.21.15.103:0/2491615726 learned_addr learned my addr 172.21.15.103:0/2491615726 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:16.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7ced9d640 1 -- 172.21.15.103:0/2491615726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec81370 msgr2=0x55f5bec85280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:16.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7ced9d640 1 --2- 172.21.15.103:0/2491615726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec81370 0x55f5bec85280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7ced9d640 1 -- 172.21.15.103:0/2491615726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec77b40 msgr2=0x55f5bec84d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:16.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7ce59c640 1 --2- 172.21.15.103:0/2491615726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec77b40 0x55f5bec84d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7ced9d640 1 --2- 172.21.15.103:0/2491615726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec77b40 0x55f5bec84d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7ced9d640 1 -- 172.21.15.103:0/2491615726 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5bec858c0 con 0x55f5bec765d0 2024-09-19T07:05:16.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7ced9d640 1 --2- 172.21.15.103:0/2491615726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec89a10 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8005e10 tx=0x7fc7c800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cfb671a2699a04 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:16.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:16.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7b2ff5640 1 -- 172.21.15.103:0/2491615726 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c801c020 con 0x55f5bec765d0 2024-09-19T07:05:16.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7b2ff5640 1 -- 172.21.15.103:0/2491615726 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fc7c8008710 con 0x55f5bec765d0 2024-09-19T07:05:16.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.182+0000 7fc7d75c7740 1 -- 172.21.15.103:0/2491615726 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f5bebe3d70 con 0x55f5bec765d0 2024-09-19T07:05:16.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.183+0000 7fc7b2ff5640 1 -- 172.21.15.103:0/2491615726 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc7c803b050 con 0x55f5bec765d0 2024-09-19T07:05:16.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.183+0000 7fc7d75c7740 10 client.21171 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:16.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.183+0000 7fc7d75c7740 1 -- 172.21.15.103:0/2491615726 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5becfb530 con 0x55f5bec765d0 2024-09-19T07:05:16.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.183+0000 7fc7b2ff5640 1 -- 172.21.15.103:0/2491615726 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f5bebe3d70 con 0x55f5bec765d0 2024-09-19T07:05:16.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.183+0000 7fc7b2ff5640 1 -- 172.21.15.103:0/2491615726 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc7c80035f0 con 0x55f5bec765d0 2024-09-19T07:05:16.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.183+0000 7fc7d75c7740 10 client.21171 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:16.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.183+0000 7fc7d75c7740 10 client.21171 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:05:16.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.183+0000 7fc7d75c7740 20 client.21171 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:16.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.183+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/2491615726 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55f5becfbc50 0x55f5becfe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:16.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.183+0000 7fc7d75c7740 4 client.21171 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:05:16.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.183+0000 7fc7d75c7740 1 -- 172.21.15.103:0/2491615726 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5beb60170 con 0x55f5becfbc50 2024-09-19T07:05:16.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.183+0000 7fc7ce59c640 1 --2- 172.21.15.103:0/2491615726 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55f5becfbc50 0x55f5becfe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:16.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7ce59c640 1 --2- 172.21.15.103:0/2491615726 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55f5becfbc50 0x55f5becfe090 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3de3662dc105d614 server_cookie=d6056d472480578b in_seq=0 out_seq=0 2024-09-19T07:05:16.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7b2ff5640 10 client.21171 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:05:16.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7b2ff5640 1 -- 172.21.15.103:0/2491615726 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5beb60170 con 0x55f5becfbc50 2024-09-19T07:05:16.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7b2ff5640 10 client.21171 handle_command_reply: tid=0 2024-09-19T07:05:16.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7d75c7740 1 client.21171 shutdown 2024-09-19T07:05:16.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7d75c7740 1 -- 172.21.15.103:0/2491615726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 msgr2=0x55f5bec89a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:16.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/2491615726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec89a10 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8005e10 tx=0x7fc7c800ed50 comp rx=0 tx=0).stop 2024-09-19T07:05:16.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7d75c7740 1 -- 172.21.15.103:0/2491615726 shutdown_connections 2024-09-19T07:05:16.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/2491615726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5bec81370 0x55f5bec85280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/2491615726 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55f5becfbc50 0x55f5becfe090 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/2491615726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f5bec77b40 0x55f5bec84d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7d75c7740 1 --2- 172.21.15.103:0/2491615726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bec765d0 0x55f5bec89a10 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7b2ff5640 0 client.21171 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:05:16.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7d75c7740 1 -- 172.21.15.103:0/2491615726 >> 172.21.15.103:0/2491615726 conn(0x55f5bec3f310 msgr2=0x55f5bec71b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:16.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7d75c7740 1 -- 172.21.15.103:0/2491615726 shutdown_connections 2024-09-19T07:05:16.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7d75c7740 1 -- 172.21.15.103:0/2491615726 wait complete. 2024-09-19T07:05:16.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.184+0000 7fc7d75c7740 20 client.21171 trim_cache size 0 max 16384 2024-09-19T07:05:16.191 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:05:16.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.185+0000 7fc7d51ee640 1 -- 172.21.15.103:0/728996768 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7b80a33b0 msgr2=0x7fc7b80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:16.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.185+0000 7fc7d51ee640 1 --2- 172.21.15.103:0/728996768 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7b80a33b0 0x7fc7b80a5870 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00092f0 tx=0x7fc7c0009280 comp rx=0 tx=0).stop 2024-09-19T07:05:16.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.185+0000 7fc7d51ee640 1 -- 172.21.15.103:0/728996768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7d01036c0 msgr2=0x7fc7d019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:16.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.185+0000 7fc7d51ee640 1 --2- 172.21.15.103:0/728996768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7d01036c0 0x7fc7d019ab20 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4006010 tx=0x7fc7c40042f0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.185+0000 7fc7d51ee640 1 -- 172.21.15.103:0/728996768 shutdown_connections 2024-09-19T07:05:16.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.185+0000 7fc7d51ee640 1 --2- 172.21.15.103:0/728996768 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fc7b80a33b0 0x7fc7b80a5870 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.185+0000 7fc7d51ee640 1 --2- 172.21.15.103:0/728996768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7d010c180 0x7fc7d019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.185+0000 7fc7d51ee640 1 --2- 172.21.15.103:0/728996768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7d01036c0 0x7fc7d019ab20 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.185+0000 7fc7d51ee640 1 --2- 172.21.15.103:0/728996768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7d0102da0 0x7fc7d019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:16.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.185+0000 7fc7d51ee640 1 -- 172.21.15.103:0/728996768 >> 172.21.15.103:0/728996768 conn(0x7fc7d00fc7a0 msgr2=0x7fc7d0101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:16.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.185+0000 7fc7d51ee640 1 -- 172.21.15.103:0/728996768 shutdown_connections 2024-09-19T07:05:16.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:16.185+0000 7fc7d51ee640 1 -- 172.21.15.103:0/728996768 wait complete. 2024-09-19T07:05:16.850 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:05:17.101 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:17.502 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:17 smithi103 ceph-mon[31158]: pgmap v502: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 30 op/s 2024-09-19T07:05:17.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.583+0000 7f0869b7d640 1 -- 172.21.15.103:0/3420624243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0864102ef0 msgr2=0x7f086410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:17.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.583+0000 7f0869b7d640 1 --2- 172.21.15.103:0/3420624243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0864102ef0 0x7f086410da40 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f085c009900 tx=0x7f085c02a480 comp rx=0 tx=0).stop 2024-09-19T07:05:17.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.583+0000 7f0869b7d640 1 -- 172.21.15.103:0/3420624243 shutdown_connections 2024-09-19T07:05:17.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.583+0000 7f0869b7d640 1 --2- 172.21.15.103:0/3420624243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0864102ef0 0x7f086410da40 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.583+0000 7f0869b7d640 1 --2- 172.21.15.103:0/3420624243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0864102570 0x7f08641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.583+0000 7f0869b7d640 1 --2- 172.21.15.103:0/3420624243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0864100e20 0x7f0864101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.583+0000 7f0869b7d640 1 -- 172.21.15.103:0/3420624243 >> 172.21.15.103:0/3420624243 conn(0x7f08640fc7d0 msgr2=0x7f08640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:17.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.583+0000 7f0869b7d640 1 -- 172.21.15.103:0/3420624243 shutdown_connections 2024-09-19T07:05:17.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.583+0000 7f0869b7d640 1 -- 172.21.15.103:0/3420624243 wait complete. 2024-09-19T07:05:17.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.583+0000 7f0869b7d640 1 Processor -- start 2024-09-19T07:05:17.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.583+0000 7f0869b7d640 1 -- start start 2024-09-19T07:05:17.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0869b7d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0864100e20 0x7f08640fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0869b7d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0864102570 0x7f08640fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0869b7d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0864102ef0 0x7f08640fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0869b7d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0864111020 con 0x7f0864102570 2024-09-19T07:05:17.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0869b7d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0864110ea0 con 0x7f0864100e20 2024-09-19T07:05:17.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0869b7d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08641111a0 con 0x7f0864102ef0 2024-09-19T07:05:17.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0862ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0864102570 0x7f08640fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0862ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0864102570 0x7f08640fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60184/0 (socket says 172.21.15.103:60184) 2024-09-19T07:05:17.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0862ffd640 1 -- 172.21.15.103:0/323135044 learned_addr learned my addr 172.21.15.103:0/323135044 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:17.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f08637fe640 1 --2- 172.21.15.103:0/323135044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0864100e20 0x7f08640fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0863fff640 1 --2- 172.21.15.103:0/323135044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0864102ef0 0x7f08640fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0862ffd640 1 -- 172.21.15.103:0/323135044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0864102ef0 msgr2=0x7f08640fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:17.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0862ffd640 1 --2- 172.21.15.103:0/323135044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0864102ef0 0x7f08640fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0862ffd640 1 -- 172.21.15.103:0/323135044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0864100e20 msgr2=0x7f08640fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:17.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0862ffd640 1 --2- 172.21.15.103:0/323135044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0864100e20 0x7f08640fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0862ffd640 1 -- 172.21.15.103:0/323135044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08641a6aa0 con 0x7f0864102570 2024-09-19T07:05:17.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0862ffd640 1 --2- 172.21.15.103:0/323135044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0864102570 0x7f08640fd1c0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f085800b4a0 tx=0x7f085800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:17.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0860ff9640 1 -- 172.21.15.103:0/323135044 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858014070 con 0x7f0864102570 2024-09-19T07:05:17.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0860ff9640 1 -- 172.21.15.103:0/323135044 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0858010af0 con 0x7f0864102570 2024-09-19T07:05:17.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.584+0000 7f0869b7d640 1 -- 172.21.15.103:0/323135044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08641a6d90 con 0x7f0864102570 2024-09-19T07:05:17.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.585+0000 7f0860ff9640 1 -- 172.21.15.103:0/323135044 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858017d10 con 0x7f0864102570 2024-09-19T07:05:17.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.585+0000 7f0869b7d640 1 -- 172.21.15.103:0/323135044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08641a7270 con 0x7f0864102570 2024-09-19T07:05:17.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.587+0000 7f0860ff9640 1 -- 172.21.15.103:0/323135044 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0858017070 con 0x7f0864102570 2024-09-19T07:05:17.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.587+0000 7f086bf56740 1 Processor -- start 2024-09-19T07:05:17.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.587+0000 7f086bf56740 1 -- start start 2024-09-19T07:05:17.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.587+0000 7f086bf56740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 0x56463963d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.587+0000 7f086bf56740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56463963d5d0 0x56463963da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.587+0000 7f086bf56740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564639648370 0x56463964a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.587+0000 7f086bf56740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5646395a32f0 con 0x564639648370 2024-09-19T07:05:17.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.587+0000 7f086bf56740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56463954acf0 con 0x56463963d5d0 2024-09-19T07:05:17.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.587+0000 7f086bf56740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56463959f2b0 con 0x56463963eb40 2024-09-19T07:05:17.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.587+0000 7f0860ff9640 1 --2- 172.21.15.103:0/323135044 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f084c0a33b0 0x7f084c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.587+0000 7f0862ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564639648370 0x56463964a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.587+0000 7f0862ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564639648370 0x56463964a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60196/0 (socket says 172.21.15.103:60196) 2024-09-19T07:05:17.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.587+0000 7f0862ffd640 1 -- 172.21.15.103:0/1730545949 learned_addr learned my addr 172.21.15.103:0/1730545949 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:17.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.588+0000 7f0863fff640 1 --2- 172.21.15.103:0/323135044 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f084c0a33b0 0x7f084c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.588+0000 7f08637fe640 1 --2- 172.21.15.103:0/1730545949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56463963d5d0 0x56463963da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.588+0000 7f0860ff9640 1 -- 172.21.15.103:0/323135044 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f08580cefa0 con 0x7f0864102570 2024-09-19T07:05:17.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.588+0000 7f0863fff640 1 --2- 172.21.15.103:0/1730545949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 0x56463963d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.588+0000 7f0862ffd640 1 -- 172.21.15.103:0/1730545949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 msgr2=0x56463963d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:17.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.588+0000 7f0862ffd640 1 --2- 172.21.15.103:0/1730545949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 0x56463963d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.588+0000 7f0862ffd640 1 -- 172.21.15.103:0/1730545949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56463963d5d0 msgr2=0x56463963da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:17.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.588+0000 7f0862ffd640 1 --2- 172.21.15.103:0/1730545949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56463963d5d0 0x56463963da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.588+0000 7f0862ffd640 1 -- 172.21.15.103:0/1730545949 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463952f3a0 con 0x564639648370 2024-09-19T07:05:17.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.588+0000 7f08637fe640 1 --2- 172.21.15.103:0/1730545949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56463963d5d0 0x56463963da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:17.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.588+0000 7f0863fff640 1 --2- 172.21.15.103:0/1730545949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 0x56463963d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:17.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.588+0000 7f0862ffd640 1 --2- 172.21.15.103:0/1730545949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564639648370 0x56463964a760 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f0858097d40 tx=0x7f0858009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beeccd7cdd9d2e2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:17.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.588+0000 7f0849ffb640 1 -- 172.21.15.103:0/1730545949 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085801e070 con 0x564639648370 2024-09-19T07:05:17.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.588+0000 7f0849ffb640 1 -- 172.21.15.103:0/1730545949 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08580a3fe0 con 0x564639648370 2024-09-19T07:05:17.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.588+0000 7f0849ffb640 1 -- 172.21.15.103:0/1730545949 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08580979e0 con 0x564639648370 2024-09-19T07:05:17.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.589+0000 7f086bf56740 1 -- 172.21.15.103:0/1730545949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564639648370 msgr2=0x56463964a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:17.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.589+0000 7f086bf56740 1 --2- 172.21.15.103:0/1730545949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564639648370 0x56463964a760 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f0858097d40 tx=0x7f0858009390 comp rx=0 tx=0).stop 2024-09-19T07:05:17.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.589+0000 7f0863fff640 1 --2- 172.21.15.103:0/323135044 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f084c0a33b0 0x7f084c0a5870 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f085c0314e0 tx=0x7f085c0058b0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:17.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.589+0000 7f086bf56740 1 -- 172.21.15.103:0/1730545949 shutdown_connections 2024-09-19T07:05:17.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.589+0000 7f086bf56740 1 --2- 172.21.15.103:0/1730545949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564639648370 0x56463964a760 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.589+0000 7f086bf56740 1 --2- 172.21.15.103:0/1730545949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56463963d5d0 0x56463963da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.589+0000 7f086bf56740 1 --2- 172.21.15.103:0/1730545949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 0x56463963d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.589+0000 7f086bf56740 1 -- 172.21.15.103:0/1730545949 >> 172.21.15.103:0/1730545949 conn(0x564639606310 msgr2=0x564639639c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:17.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.589+0000 7f086bf56740 1 -- 172.21.15.103:0/1730545949 shutdown_connections 2024-09-19T07:05:17.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.589+0000 7f086bf56740 1 -- 172.21.15.103:0/1730545949 wait complete. 2024-09-19T07:05:17.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.589+0000 7f086bf56740 1 -- start start 2024-09-19T07:05:17.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f086bf56740 1 Processor -- start 2024-09-19T07:05:17.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f086bf56740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463963d5d0 0x564639647d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f086bf56740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 0x564639645020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f086bf56740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564639648370 0x564639645610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f086bf56740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56463964afb0 con 0x56463963d5d0 2024-09-19T07:05:17.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f086bf56740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56463955e060 con 0x564639648370 2024-09-19T07:05:17.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f086bf56740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56463964b130 con 0x56463963eb40 2024-09-19T07:05:17.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f086bf56740 10 client.0 _ll_register_callbacks cb 0x5646392b18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:17.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f08637fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463963d5d0 0x564639647d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f08637fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463963d5d0 0x564639647d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60208/0 (socket says 172.21.15.103:60208) 2024-09-19T07:05:17.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f08637fe640 1 -- 172.21.15.103:0/3986253425 learned_addr learned my addr 172.21.15.103:0/3986253425 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:17.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f0863fff640 1 --2- 172.21.15.103:0/3986253425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564639648370 0x564639645610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f0862ffd640 1 --2- 172.21.15.103:0/3986253425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 0x564639645020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f08637fe640 1 -- 172.21.15.103:0/3986253425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 msgr2=0x564639645020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:17.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f08637fe640 1 --2- 172.21.15.103:0/3986253425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 0x564639645020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f08637fe640 1 -- 172.21.15.103:0/3986253425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564639648370 msgr2=0x564639645610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:17.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f08637fe640 1 --2- 172.21.15.103:0/3986253425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564639648370 0x564639645610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f08637fe640 1 -- 172.21.15.103:0/3986253425 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646393143c0 con 0x56463963d5d0 2024-09-19T07:05:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f0862ffd640 1 --2- 172.21.15.103:0/3986253425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 0x564639645020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f08637fe640 1 --2- 172.21.15.103:0/3986253425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463963d5d0 0x564639647d70 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f085400ab90 tx=0x7f0854007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33215e76dafceb25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.590+0000 7f0842ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.591+0000 7f0842ffd640 1 -- 172.21.15.103:0/3986253425 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854016610 con 0x56463963d5d0 2024-09-19T07:05:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.591+0000 7f0842ffd640 1 -- 172.21.15.103:0/3986253425 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f08540167b0 con 0x56463963d5d0 2024-09-19T07:05:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.591+0000 7f086bf56740 1 -- 172.21.15.103:0/3986253425 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08580a3fe0 con 0x56463963d5d0 2024-09-19T07:05:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.591+0000 7f0842ffd640 1 -- 172.21.15.103:0/3986253425 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f08540169b0 con 0x56463963d5d0 2024-09-19T07:05:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.591+0000 7f086bf56740 10 client.21183 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.591+0000 7f086bf56740 1 -- 172.21.15.103:0/3986253425 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564639656a80 con 0x56463963d5d0 2024-09-19T07:05:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.591+0000 7f0842ffd640 1 -- 172.21.15.103:0/3986253425 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08580a3fe0 con 0x56463963d5d0 2024-09-19T07:05:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.591+0000 7f0842ffd640 1 -- 172.21.15.103:0/3986253425 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f085401fb30 con 0x56463963d5d0 2024-09-19T07:05:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.591+0000 7f086bf56740 10 client.21183 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:17.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.591+0000 7f086bf56740 10 client.21183 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:05:17.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.591+0000 7f086bf56740 20 client.21183 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:17.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.591+0000 7f086bf56740 1 --2- 172.21.15.103:0/3986253425 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x564639657f90 0x56463965a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.591+0000 7f086bf56740 4 client.21183 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:17.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.591+0000 7f086bf56740 1 -- 172.21.15.103:0/3986253425 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646395b0860 con 0x564639657f90 2024-09-19T07:05:17.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.592+0000 7f0862ffd640 1 --2- 172.21.15.103:0/3986253425 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x564639657f90 0x56463965a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.592+0000 7f0842ffd640 10 client.21183 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:05:17.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.592+0000 7f0862ffd640 1 --2- 172.21.15.103:0/3986253425 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x564639657f90 0x56463965a410 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=484770d7f366772d server_cookie=36aa4cef7299871a in_seq=0 out_seq=0 2024-09-19T07:05:17.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.593+0000 7f0842ffd640 1 -- 172.21.15.103:0/3986253425 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646395b0860 con 0x564639657f90 2024-09-19T07:05:17.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.593+0000 7f0842ffd640 10 client.21183 handle_command_reply: tid=0 2024-09-19T07:05:17.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.593+0000 7f086bf56740 1 client.21183 shutdown 2024-09-19T07:05:17.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.593+0000 7f086bf56740 1 -- 172.21.15.103:0/3986253425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463963d5d0 msgr2=0x564639647d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:17.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.593+0000 7f086bf56740 1 --2- 172.21.15.103:0/3986253425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463963d5d0 0x564639647d70 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f085400ab90 tx=0x7f0854007680 comp rx=0 tx=0).stop 2024-09-19T07:05:17.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.593+0000 7f086bf56740 1 -- 172.21.15.103:0/3986253425 shutdown_connections 2024-09-19T07:05:17.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.593+0000 7f086bf56740 1 --2- 172.21.15.103:0/3986253425 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x564639657f90 0x56463965a410 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.593+0000 7f086bf56740 1 --2- 172.21.15.103:0/3986253425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564639648370 0x564639645610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.593+0000 7f086bf56740 1 --2- 172.21.15.103:0/3986253425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 0x564639645020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.593+0000 7f086bf56740 1 --2- 172.21.15.103:0/3986253425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463963d5d0 0x564639647d70 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.593+0000 7f0842ffd640 0 client.21183 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:05:17.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.593+0000 7f086bf56740 1 -- 172.21.15.103:0/3986253425 >> 172.21.15.103:0/3986253425 conn(0x564639606310 msgr2=0x56463963f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:17.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.593+0000 7f086bf56740 1 -- 172.21.15.103:0/3986253425 shutdown_connections 2024-09-19T07:05:17.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.593+0000 7f086bf56740 1 -- 172.21.15.103:0/3986253425 wait complete. 2024-09-19T07:05:17.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.593+0000 7f086bf56740 20 client.21183 trim_cache size 0 max 16384 2024-09-19T07:05:17.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f086bf56740 1 Processor -- start 2024-09-19T07:05:17.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f086bf56740 1 -- start start 2024-09-19T07:05:17.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f086bf56740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963d5d0 0x564639639e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f086bf56740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463963eb40 0x5646396542b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f086bf56740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564639648370 0x56463965fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f086bf56740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f08641afba0 con 0x56463963eb40 2024-09-19T07:05:17.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f086bf56740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f08641b16e0 con 0x564639648370 2024-09-19T07:05:17.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f086bf56740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08641b0270 con 0x56463963d5d0 2024-09-19T07:05:17.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f0862ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463963eb40 0x5646396542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f0862ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463963eb40 0x5646396542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60214/0 (socket says 172.21.15.103:60214) 2024-09-19T07:05:17.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f0862ffd640 1 -- 172.21.15.103:0/3779768779 learned_addr learned my addr 172.21.15.103:0/3779768779 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:17.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f08637fe640 1 --2- 172.21.15.103:0/3779768779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963d5d0 0x564639639e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f0862ffd640 1 -- 172.21.15.103:0/3779768779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963d5d0 msgr2=0x564639639e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:17.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f0862ffd640 1 --2- 172.21.15.103:0/3779768779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963d5d0 0x564639639e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f0862ffd640 1 -- 172.21.15.103:0/3779768779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564639648370 msgr2=0x56463965fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:17.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f0862ffd640 1 --2- 172.21.15.103:0/3779768779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564639648370 0x56463965fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.606+0000 7f0862ffd640 1 -- 172.21.15.103:0/3779768779 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646396463a0 con 0x56463963eb40 2024-09-19T07:05:17.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.607+0000 7f0862ffd640 1 --2- 172.21.15.103:0/3779768779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463963eb40 0x5646396542b0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f08580977d0 tx=0x7f085809b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ecf78c412fe424 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:17.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.607+0000 7f08497fa640 1 -- 172.21.15.103:0/3779768779 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085801e040 con 0x56463963eb40 2024-09-19T07:05:17.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.607+0000 7f08497fa640 1 -- 172.21.15.103:0/3779768779 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08580a3fe0 con 0x56463963eb40 2024-09-19T07:05:17.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.607+0000 7f08497fa640 1 -- 172.21.15.103:0/3779768779 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085809a7c0 con 0x56463963eb40 2024-09-19T07:05:17.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.607+0000 7f086bf56740 1 -- 172.21.15.103:0/3779768779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463963eb40 msgr2=0x5646396542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:17.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.607+0000 7f086bf56740 1 --2- 172.21.15.103:0/3779768779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463963eb40 0x5646396542b0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f08580977d0 tx=0x7f085809b4f0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.607+0000 7f086bf56740 1 -- 172.21.15.103:0/3779768779 shutdown_connections 2024-09-19T07:05:17.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.607+0000 7f086bf56740 1 --2- 172.21.15.103:0/3779768779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564639648370 0x56463965fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.607+0000 7f086bf56740 1 --2- 172.21.15.103:0/3779768779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463963eb40 0x5646396542b0 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.607+0000 7f086bf56740 1 --2- 172.21.15.103:0/3779768779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963d5d0 0x564639639e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.607+0000 7f086bf56740 1 -- 172.21.15.103:0/3779768779 >> 172.21.15.103:0/3779768779 conn(0x564639606310 msgr2=0x564639639700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:17.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.608+0000 7f086bf56740 1 -- 172.21.15.103:0/3779768779 shutdown_connections 2024-09-19T07:05:17.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.608+0000 7f086bf56740 1 -- 172.21.15.103:0/3779768779 wait complete. 2024-09-19T07:05:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.608+0000 7f086bf56740 1 -- start start 2024-09-19T07:05:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.608+0000 7f086bf56740 1 Processor -- start 2024-09-19T07:05:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.608+0000 7f086bf56740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56463963d5d0 0x564639650d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.608+0000 7f086bf56740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 0x56463964c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.608+0000 7f086bf56740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564639648370 0x56463964c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.608+0000 7f086bf56740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f08641b1010 con 0x564639648370 2024-09-19T07:05:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.608+0000 7f086bf56740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f08641b0940 con 0x56463963d5d0 2024-09-19T07:05:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.608+0000 7f086bf56740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08641adba0 con 0x56463963eb40 2024-09-19T07:05:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.608+0000 7f086bf56740 10 client.0 _ll_register_callbacks cb 0x7f0854009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f0863fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564639648370 0x56463964c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f0863fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564639648370 0x56463964c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60226/0 (socket says 172.21.15.103:60226) 2024-09-19T07:05:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f0863fff640 1 -- 172.21.15.103:0/1744199909 learned_addr learned my addr 172.21.15.103:0/1744199909 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f08637fe640 1 --2- 172.21.15.103:0/1744199909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56463963d5d0 0x564639650d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f0862ffd640 1 --2- 172.21.15.103:0/1744199909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 0x56463964c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f0863fff640 1 -- 172.21.15.103:0/1744199909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 msgr2=0x56463964c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f0863fff640 1 --2- 172.21.15.103:0/1744199909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 0x56463964c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f0863fff640 1 -- 172.21.15.103:0/1744199909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56463963d5d0 msgr2=0x564639650d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f0863fff640 1 --2- 172.21.15.103:0/1744199909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56463963d5d0 0x564639650d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f0863fff640 1 -- 172.21.15.103:0/1744199909 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646396c1cb0 con 0x564639648370 2024-09-19T07:05:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f0863fff640 1 --2- 172.21.15.103:0/1744199909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564639648370 0x56463964c860 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f085c009900 tx=0x7f085c003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6538b26945e488d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f0842ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f0842ffd640 1 -- 172.21.15.103:0/1744199909 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c04e070 con 0x564639648370 2024-09-19T07:05:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f0842ffd640 1 -- 172.21.15.103:0/1744199909 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f085c03d450 con 0x564639648370 2024-09-19T07:05:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f086bf56740 1 -- 172.21.15.103:0/1744199909 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08580a3fe0 con 0x564639648370 2024-09-19T07:05:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f0842ffd640 1 -- 172.21.15.103:0/1744199909 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f085c035030 con 0x564639648370 2024-09-19T07:05:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f086bf56740 10 client.21189 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.609+0000 7f086bf56740 1 -- 172.21.15.103:0/1744199909 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5646396c2e90 con 0x564639648370 2024-09-19T07:05:17.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.610+0000 7f0842ffd640 1 -- 172.21.15.103:0/1744199909 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08580a3fe0 con 0x564639648370 2024-09-19T07:05:17.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.610+0000 7f0842ffd640 1 -- 172.21.15.103:0/1744199909 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f085c04a530 con 0x564639648370 2024-09-19T07:05:17.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.610+0000 7f086bf56740 10 client.21189 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:17.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.610+0000 7f086bf56740 10 client.21189 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:05:17.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.610+0000 7f086bf56740 20 client.21189 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:17.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.610+0000 7f086bf56740 1 --2- 172.21.15.103:0/1744199909 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5646396c3610 0x5646396c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:17.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.610+0000 7f086bf56740 4 client.21189 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:05:17.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.610+0000 7f086bf56740 1 -- 172.21.15.103:0/1744199909 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646395aad70 con 0x5646396c3610 2024-09-19T07:05:17.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.610+0000 7f08637fe640 1 --2- 172.21.15.103:0/1744199909 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5646396c3610 0x5646396c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:17.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.610+0000 7f0842ffd640 10 client.21189 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:05:17.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.610+0000 7f08637fe640 1 --2- 172.21.15.103:0/1744199909 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5646396c3610 0x5646396c5a30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be860759a883fea3 server_cookie=db4678d5f237e145 in_seq=0 out_seq=0 2024-09-19T07:05:17.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.611+0000 7f0842ffd640 1 -- 172.21.15.103:0/1744199909 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646395aad70 con 0x5646396c3610 2024-09-19T07:05:17.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.611+0000 7f0842ffd640 10 client.21189 handle_command_reply: tid=0 2024-09-19T07:05:17.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.611+0000 7f086bf56740 1 client.21189 shutdown 2024-09-19T07:05:17.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.611+0000 7f086bf56740 1 -- 172.21.15.103:0/1744199909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564639648370 msgr2=0x56463964c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:17.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.611+0000 7f086bf56740 1 --2- 172.21.15.103:0/1744199909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564639648370 0x56463964c860 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f085c009900 tx=0x7f085c003db0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.611+0000 7f086bf56740 1 -- 172.21.15.103:0/1744199909 shutdown_connections 2024-09-19T07:05:17.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.611+0000 7f086bf56740 1 --2- 172.21.15.103:0/1744199909 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x5646396c3610 0x5646396c5a30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.611+0000 7f086bf56740 1 --2- 172.21.15.103:0/1744199909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564639648370 0x56463964c860 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.611+0000 7f086bf56740 1 --2- 172.21.15.103:0/1744199909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56463963eb40 0x56463964c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.611+0000 7f086bf56740 1 --2- 172.21.15.103:0/1744199909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56463963d5d0 0x564639650d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.611+0000 7f086bf56740 1 -- 172.21.15.103:0/1744199909 >> 172.21.15.103:0/1744199909 conn(0x564639606310 msgr2=0x56463963f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:17.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.611+0000 7f0842ffd640 0 client.21189 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:05:17.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.611+0000 7f086bf56740 1 -- 172.21.15.103:0/1744199909 shutdown_connections 2024-09-19T07:05:17.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.611+0000 7f086bf56740 1 -- 172.21.15.103:0/1744199909 wait complete. 2024-09-19T07:05:17.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.611+0000 7f086bf56740 20 client.21189 trim_cache size 0 max 16384 2024-09-19T07:05:17.618 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:05:17.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.612+0000 7f0869b7d640 1 -- 172.21.15.103:0/323135044 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f084c0a33b0 msgr2=0x7f084c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.612+0000 7f0869b7d640 1 --2- 172.21.15.103:0/323135044 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f084c0a33b0 0x7f084c0a5870 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f085c0314e0 tx=0x7f085c0058b0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.612+0000 7f0869b7d640 1 -- 172.21.15.103:0/323135044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0864102570 msgr2=0x7f08640fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.612+0000 7f0869b7d640 1 --2- 172.21.15.103:0/323135044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0864102570 0x7f08640fd1c0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f085800b4a0 tx=0x7f085800b960 comp rx=0 tx=0).stop 2024-09-19T07:05:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.612+0000 7f0869b7d640 1 -- 172.21.15.103:0/323135044 shutdown_connections 2024-09-19T07:05:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.612+0000 7f0869b7d640 1 --2- 172.21.15.103:0/323135044 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f084c0a33b0 0x7f084c0a5870 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.612+0000 7f0869b7d640 1 --2- 172.21.15.103:0/323135044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0864102ef0 0x7f08640fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.612+0000 7f0869b7d640 1 --2- 172.21.15.103:0/323135044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0864102570 0x7f08640fd1c0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.612+0000 7f0869b7d640 1 --2- 172.21.15.103:0/323135044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0864100e20 0x7f08640fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.612+0000 7f0869b7d640 1 -- 172.21.15.103:0/323135044 >> 172.21.15.103:0/323135044 conn(0x7f08640fc7d0 msgr2=0x7f086406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.612+0000 7f0869b7d640 1 -- 172.21.15.103:0/323135044 shutdown_connections 2024-09-19T07:05:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:17.612+0000 7f0869b7d640 1 -- 172.21.15.103:0/323135044 wait complete. 2024-09-19T07:05:17.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:17 smithi134 ceph-mon[35356]: pgmap v502: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 30 op/s 2024-09-19T07:05:17.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:17 smithi174 ceph-mon[35637]: pgmap v502: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 30 op/s 2024-09-19T07:05:18.291 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:05:18.537 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:18.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.975+0000 7f4e0e9e1640 1 -- 172.21.15.103:0/2311617260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e08106a80 msgr2=0x7f4e08106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:18.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.975+0000 7f4e0e9e1640 1 --2- 172.21.15.103:0/2311617260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e08106a80 0x7f4e08106e60 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009930 tx=0x7f4df802a420 comp rx=0 tx=0).stop 2024-09-19T07:05:18.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.975+0000 7f4e0e9e1640 1 -- 172.21.15.103:0/2311617260 shutdown_connections 2024-09-19T07:05:18.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.975+0000 7f4e0e9e1640 1 --2- 172.21.15.103:0/2311617260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e08101420 0x7f4e0810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:18.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.975+0000 7f4e0e9e1640 1 --2- 172.21.15.103:0/2311617260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e08100a80 0x7f4e08100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:18.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.975+0000 7f4e0e9e1640 1 --2- 172.21.15.103:0/2311617260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e08106a80 0x7f4e08106e60 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:18.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.975+0000 7f4e0e9e1640 1 -- 172.21.15.103:0/2311617260 >> 172.21.15.103:0/2311617260 conn(0x7f4e080fc7a0 msgr2=0x7f4e080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:18.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.975+0000 7f4e0e9e1640 1 -- 172.21.15.103:0/2311617260 shutdown_connections 2024-09-19T07:05:18.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.975+0000 7f4e0e9e1640 1 -- 172.21.15.103:0/2311617260 wait complete. 2024-09-19T07:05:18.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.975+0000 7f4e0e9e1640 1 Processor -- start 2024-09-19T07:05:18.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.975+0000 7f4e0e9e1640 1 -- start start 2024-09-19T07:05:18.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4e0e9e1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e08100a80 0x7f4e081973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:18.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4e0e9e1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e08101420 0x7f4e081978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:18.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4e0e9e1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e08106a80 0x7f4e0819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:18.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4e0e9e1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08110ed0 con 0x7f4e08101420 2024-09-19T07:05:18.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4e0e9e1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08110d50 con 0x7f4e08100a80 2024-09-19T07:05:18.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4e0e9e1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08111050 con 0x7f4e08106a80 2024-09-19T07:05:18.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4dfffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e08101420 0x7f4e081978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:18.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4dfffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e08101420 0x7f4e081978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60250/0 (socket says 172.21.15.103:60250) 2024-09-19T07:05:18.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4dfffff640 1 -- 172.21.15.103:0/2374573140 learned_addr learned my addr 172.21.15.103:0/2374573140 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:18.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4e07fff640 1 --2- 172.21.15.103:0/2374573140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e08100a80 0x7f4e081973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:18.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4e0cf7e640 1 --2- 172.21.15.103:0/2374573140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e08106a80 0x7f4e0819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:18.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4dfffff640 1 -- 172.21.15.103:0/2374573140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e08106a80 msgr2=0x7f4e0819bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:18.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4dfffff640 1 --2- 172.21.15.103:0/2374573140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e08106a80 0x7f4e0819bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:18.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4dfffff640 1 -- 172.21.15.103:0/2374573140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e08100a80 msgr2=0x7f4e081973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:18.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4dfffff640 1 --2- 172.21.15.103:0/2374573140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e08100a80 0x7f4e081973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:18.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4dfffff640 1 -- 172.21.15.103:0/2374573140 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0819c360 con 0x7f4e08101420 2024-09-19T07:05:18.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.976+0000 7f4dfffff640 1 --2- 172.21.15.103:0/2374573140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e08101420 0x7f4e081978f0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f4df000d770 tx=0x7f4df000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:18.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.977+0000 7f4e05ffb640 1 -- 172.21.15.103:0/2374573140 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0017070 con 0x7f4e08101420 2024-09-19T07:05:18.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.977+0000 7f4e05ffb640 1 -- 172.21.15.103:0/2374573140 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df0005500 con 0x7f4e08101420 2024-09-19T07:05:18.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.977+0000 7f4e05ffb640 1 -- 172.21.15.103:0/2374573140 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0004e00 con 0x7f4e08101420 2024-09-19T07:05:18.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.977+0000 7f4e0e9e1640 1 -- 172.21.15.103:0/2374573140 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0819c650 con 0x7f4e08101420 2024-09-19T07:05:18.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.977+0000 7f4e0e9e1640 1 -- 172.21.15.103:0/2374573140 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e0806a790 con 0x7f4e08101420 2024-09-19T07:05:18.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.979+0000 7f4e05ffb640 1 -- 172.21.15.103:0/2374573140 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4df0004fa0 con 0x7f4e08101420 2024-09-19T07:05:18.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.979+0000 7f4e10dba740 1 Processor -- start 2024-09-19T07:05:18.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.979+0000 7f4e10dba740 1 -- start start 2024-09-19T07:05:18.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e10dba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 0x5643da57f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:18.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e10dba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da57fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:18.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e10dba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 0x5643da58c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:18.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e10dba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5643da4e52f0 con 0x5643da57f5d0 2024-09-19T07:05:18.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e10dba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5643da48ccf0 con 0x5643da580b40 2024-09-19T07:05:18.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e10dba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5643da4e12b0 con 0x5643da58a370 2024-09-19T07:05:18.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e07fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da57fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:18.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e07fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da57fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60254/0 (socket says 172.21.15.103:60254) 2024-09-19T07:05:18.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e07fff640 1 -- 172.21.15.103:0/2918121179 learned_addr learned my addr 172.21.15.103:0/2918121179 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:18.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e0cf7e640 1 --2- 172.21.15.103:0/2918121179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 0x5643da57f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:18.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e05ffb640 1 --2- 172.21.15.103:0/2374573140 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4df40a3430 0x7f4df40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:18.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4dfffff640 1 --2- 172.21.15.103:0/2918121179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 0x5643da58c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:18.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e0cf7e640 1 --2- 172.21.15.103:0/2374573140 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4df40a3430 0x7f4df40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:18.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e07fff640 1 -- 172.21.15.103:0/2918121179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 msgr2=0x5643da58c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:18.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e07fff640 1 --2- 172.21.15.103:0/2918121179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 0x5643da58c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:18.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e05ffb640 1 -- 172.21.15.103:0/2374573140 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f4df00cf3d0 con 0x7f4e08101420 2024-09-19T07:05:18.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e07fff640 1 -- 172.21.15.103:0/2918121179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 msgr2=0x5643da57f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:18.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e07fff640 1 --2- 172.21.15.103:0/2918121179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 0x5643da57f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:18.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e07fff640 1 -- 172.21.15.103:0/2918121179 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643da4713a0 con 0x5643da57f5d0 2024-09-19T07:05:18.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4dfffff640 1 --2- 172.21.15.103:0/2918121179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 0x5643da58c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:18.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.980+0000 7f4e07fff640 1 --2- 172.21.15.103:0/2918121179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da57fa10 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009930 tx=0x7f4df8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41583a8109ad311 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:18.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.981+0000 7f4dfe7fc640 1 -- 172.21.15.103:0/2918121179 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8046070 con 0x5643da57f5d0 2024-09-19T07:05:18.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.981+0000 7f4e0cf7e640 1 --2- 172.21.15.103:0/2374573140 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4df40a3430 0x7f4df40a58f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4e00005fd0 tx=0x7f4e00004ab0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:18.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.981+0000 7f4dfe7fc640 1 -- 172.21.15.103:0/2918121179 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df8005520 con 0x5643da57f5d0 2024-09-19T07:05:18.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.981+0000 7f4dfe7fc640 1 -- 172.21.15.103:0/2918121179 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8035030 con 0x5643da57f5d0 2024-09-19T07:05:18.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.981+0000 7f4e10dba740 1 -- 172.21.15.103:0/2918121179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 msgr2=0x5643da57fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:18.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.981+0000 7f4e10dba740 1 --2- 172.21.15.103:0/2918121179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da57fa10 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009930 tx=0x7f4df8004040 comp rx=0 tx=0).stop 2024-09-19T07:05:18.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.981+0000 7f4e10dba740 1 -- 172.21.15.103:0/2918121179 shutdown_connections 2024-09-19T07:05:18.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.981+0000 7f4e10dba740 1 --2- 172.21.15.103:0/2918121179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 0x5643da58c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:18.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.981+0000 7f4e10dba740 1 --2- 172.21.15.103:0/2918121179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da57fa10 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:18.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.981+0000 7f4e10dba740 1 --2- 172.21.15.103:0/2918121179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 0x5643da57f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:18.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.981+0000 7f4e10dba740 1 -- 172.21.15.103:0/2918121179 >> 172.21.15.103:0/2918121179 conn(0x5643da548310 msgr2=0x5643da57bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:18.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.981+0000 7f4e10dba740 1 -- 172.21.15.103:0/2918121179 shutdown_connections 2024-09-19T07:05:18.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.981+0000 7f4e10dba740 1 -- 172.21.15.103:0/2918121179 wait complete. 2024-09-19T07:05:18.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.982+0000 7f4e10dba740 1 -- start start 2024-09-19T07:05:18.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.982+0000 7f4e10dba740 1 Processor -- start 2024-09-19T07:05:18.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.982+0000 7f4e10dba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da57f5d0 0x5643da589d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:18.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.982+0000 7f4e10dba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da580b40 0x5643da586fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:18.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.982+0000 7f4e10dba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da58a370 0x5643da587530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:18.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.982+0000 7f4e10dba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5643da591420 con 0x5643da580b40 2024-09-19T07:05:18.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.982+0000 7f4e10dba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5643da4a0060 con 0x5643da58a370 2024-09-19T07:05:18.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.982+0000 7f4e10dba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5643da5915a0 con 0x5643da57f5d0 2024-09-19T07:05:18.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.982+0000 7f4e10dba740 10 client.0 _ll_register_callbacks cb 0x5643da1f38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:18.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.982+0000 7f4dfffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da580b40 0x5643da586fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:18.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.982+0000 7f4dfffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da580b40 0x5643da586fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60260/0 (socket says 172.21.15.103:60260) 2024-09-19T07:05:18.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.982+0000 7f4dfffff640 1 -- 172.21.15.103:0/1398336288 learned_addr learned my addr 172.21.15.103:0/1398336288 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:18.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.982+0000 7f4e07fff640 1 --2- 172.21.15.103:0/1398336288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da57f5d0 0x5643da589d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:18.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.983+0000 7f4e0cf7e640 1 --2- 172.21.15.103:0/1398336288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da58a370 0x5643da587530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:18.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.983+0000 7f4dfffff640 1 -- 172.21.15.103:0/1398336288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da57f5d0 msgr2=0x5643da589d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:18.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.983+0000 7f4dfffff640 1 --2- 172.21.15.103:0/1398336288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da57f5d0 0x5643da589d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:18.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.983+0000 7f4dfffff640 1 -- 172.21.15.103:0/1398336288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da58a370 msgr2=0x5643da587530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:18.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.983+0000 7f4dfffff640 1 --2- 172.21.15.103:0/1398336288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da58a370 0x5643da587530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:18.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.983+0000 7f4dfffff640 1 -- 172.21.15.103:0/1398336288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643da2563c0 con 0x5643da580b40 2024-09-19T07:05:18.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.983+0000 7f4e0cf7e640 1 --2- 172.21.15.103:0/1398336288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da58a370 0x5643da587530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:18.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.983+0000 7f4dfffff640 1 --2- 172.21.15.103:0/1398336288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da580b40 0x5643da586fb0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f4df009b670 tx=0x7f4df009b100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ad27e9137a7059 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:18.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.983+0000 7f4de77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:18.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.983+0000 7f4de77fe640 1 -- 172.21.15.103:0/1398336288 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0049030 con 0x5643da580b40 2024-09-19T07:05:18.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.983+0000 7f4de77fe640 1 -- 172.21.15.103:0/1398336288 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f4df0097d50 con 0x5643da580b40 2024-09-19T07:05:18.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.983+0000 7f4e10dba740 1 -- 172.21.15.103:0/1398336288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4df8005520 con 0x5643da580b40 2024-09-19T07:05:18.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.983+0000 7f4de77fe640 1 -- 172.21.15.103:0/1398336288 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4df0020040 con 0x5643da580b40 2024-09-19T07:05:18.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.983+0000 7f4e10dba740 10 client.21201 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:18.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.983+0000 7f4e10dba740 1 -- 172.21.15.103:0/1398336288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5643da59ce70 con 0x5643da580b40 2024-09-19T07:05:18.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.983+0000 7f4de77fe640 1 -- 172.21.15.103:0/1398336288 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df8005520 con 0x5643da580b40 2024-09-19T07:05:18.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.984+0000 7f4de77fe640 1 -- 172.21.15.103:0/1398336288 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4df0023060 con 0x5643da580b40 2024-09-19T07:05:18.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.984+0000 7f4e10dba740 10 client.21201 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:18.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.984+0000 7f4e10dba740 10 client.21201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:18.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.984+0000 7f4e10dba740 20 client.21201 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:18.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.984+0000 7f4e10dba740 1 --2- 172.21.15.103:0/1398336288 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5643da59e2d0 0x5643da5a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:18.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.984+0000 7f4e10dba740 4 client.21201 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:18.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.984+0000 7f4e10dba740 1 -- 172.21.15.103:0/1398336288 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643da4f2860 con 0x5643da59e2d0 2024-09-19T07:05:18.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.984+0000 7f4e07fff640 1 --2- 172.21.15.103:0/1398336288 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5643da59e2d0 0x5643da5a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:18.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.984+0000 7f4de77fe640 10 client.21201 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:18.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.984+0000 7f4e07fff640 1 --2- 172.21.15.103:0/1398336288 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5643da59e2d0 0x5643da5a06c0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67e075a6e3182a12 server_cookie=11f2bf1f67f8b1ab in_seq=0 out_seq=0 2024-09-19T07:05:18.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.985+0000 7f4de77fe640 1 -- 172.21.15.103:0/1398336288 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643da4f2860 con 0x5643da59e2d0 2024-09-19T07:05:18.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.985+0000 7f4de77fe640 10 client.21201 handle_command_reply: tid=0 2024-09-19T07:05:18.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.985+0000 7f4e10dba740 1 client.21201 shutdown 2024-09-19T07:05:18.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.985+0000 7f4e10dba740 1 -- 172.21.15.103:0/1398336288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da580b40 msgr2=0x5643da586fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:18.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.985+0000 7f4e10dba740 1 --2- 172.21.15.103:0/1398336288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da580b40 0x5643da586fb0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f4df009b670 tx=0x7f4df009b100 comp rx=0 tx=0).stop 2024-09-19T07:05:18.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.985+0000 7f4e10dba740 1 -- 172.21.15.103:0/1398336288 shutdown_connections 2024-09-19T07:05:18.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.985+0000 7f4e10dba740 1 --2- 172.21.15.103:0/1398336288 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5643da59e2d0 0x5643da5a06c0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:18.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.985+0000 7f4e10dba740 1 --2- 172.21.15.103:0/1398336288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da58a370 0x5643da587530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:18.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.985+0000 7f4e10dba740 1 --2- 172.21.15.103:0/1398336288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da580b40 0x5643da586fb0 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:18.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.985+0000 7f4de77fe640 0 client.21201 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:18.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.985+0000 7f4e10dba740 1 --2- 172.21.15.103:0/1398336288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da57f5d0 0x5643da589d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:18.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.985+0000 7f4e10dba740 1 -- 172.21.15.103:0/1398336288 >> 172.21.15.103:0/1398336288 conn(0x5643da548310 msgr2=0x5643da581930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:18.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.985+0000 7f4e10dba740 1 -- 172.21.15.103:0/1398336288 shutdown_connections 2024-09-19T07:05:18.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.985+0000 7f4e10dba740 1 -- 172.21.15.103:0/1398336288 wait complete. 2024-09-19T07:05:18.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.985+0000 7f4e10dba740 20 client.21201 trim_cache size 0 max 16384 2024-09-19T07:05:19.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.998+0000 7f4e10dba740 1 Processor -- start 2024-09-19T07:05:19.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.998+0000 7f4e10dba740 1 -- start start 2024-09-19T07:05:19.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.998+0000 7f4e10dba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da59a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:19.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.998+0000 7f4e10dba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 0x5643da580300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:19.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.998+0000 7f4e10dba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 0x5643da57b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:19.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.998+0000 7f4e10dba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e081ab6c0 con 0x5643da57f5d0 2024-09-19T07:05:19.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.998+0000 7f4e10dba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e081ad200 con 0x5643da580b40 2024-09-19T07:05:19.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.998+0000 7f4e10dba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4e081abd90 con 0x5643da58a370 2024-09-19T07:05:19.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.998+0000 7f4e07fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da59a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:19.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.998+0000 7f4e07fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da59a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60276/0 (socket says 172.21.15.103:60276) 2024-09-19T07:05:19.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.998+0000 7f4e07fff640 1 -- 172.21.15.103:0/1557959552 learned_addr learned my addr 172.21.15.103:0/1557959552 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:19.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.999+0000 7f4e0cf7e640 1 --2- 172.21.15.103:0/1557959552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 0x5643da57b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:19.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.999+0000 7f4e07fff640 1 -- 172.21.15.103:0/1557959552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 msgr2=0x5643da57b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:19.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.999+0000 7f4e07fff640 1 --2- 172.21.15.103:0/1557959552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 0x5643da57b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.999+0000 7f4e07fff640 1 -- 172.21.15.103:0/1557959552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 msgr2=0x5643da580300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:19.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.999+0000 7f4dfffff640 1 --2- 172.21.15.103:0/1557959552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 0x5643da580300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:19.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.999+0000 7f4e07fff640 1 --2- 172.21.15.103:0/1557959552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 0x5643da580300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.999+0000 7f4e07fff640 1 -- 172.21.15.103:0/1557959552 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643da582120 con 0x5643da57f5d0 2024-09-19T07:05:19.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.999+0000 7f4e0cf7e640 1 --2- 172.21.15.103:0/1557959552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 0x5643da57b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:19.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.999+0000 7f4dfffff640 1 --2- 172.21.15.103:0/1557959552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 0x5643da580300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:19.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.999+0000 7f4e07fff640 1 --2- 172.21.15.103:0/1557959552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da59a6c0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f4df8044040 tx=0x7f4df80049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfbe26265ab9d32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:19.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.999+0000 7f4dfdffb640 1 -- 172.21.15.103:0/1557959552 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8035040 con 0x5643da57f5d0 2024-09-19T07:05:19.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.999+0000 7f4dfdffb640 1 -- 172.21.15.103:0/1557959552 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df8051070 con 0x5643da57f5d0 2024-09-19T07:05:19.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:18.999+0000 7f4dfdffb640 1 -- 172.21.15.103:0/1557959552 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8046040 con 0x5643da57f5d0 2024-09-19T07:05:19.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.000+0000 7f4e10dba740 1 -- 172.21.15.103:0/1557959552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 msgr2=0x5643da59a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:19.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.000+0000 7f4e10dba740 1 --2- 172.21.15.103:0/1557959552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da59a6c0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f4df8044040 tx=0x7f4df80049d0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.000+0000 7f4e10dba740 1 -- 172.21.15.103:0/1557959552 shutdown_connections 2024-09-19T07:05:19.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.000+0000 7f4e10dba740 1 --2- 172.21.15.103:0/1557959552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 0x5643da57b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.000+0000 7f4e10dba740 1 --2- 172.21.15.103:0/1557959552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 0x5643da580300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.000+0000 7f4e10dba740 1 --2- 172.21.15.103:0/1557959552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da59a6c0 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.000+0000 7f4e10dba740 1 -- 172.21.15.103:0/1557959552 >> 172.21.15.103:0/1557959552 conn(0x5643da548310 msgr2=0x5643da576a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:19.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.000+0000 7f4e10dba740 1 -- 172.21.15.103:0/1557959552 shutdown_connections 2024-09-19T07:05:19.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.000+0000 7f4e10dba740 1 -- 172.21.15.103:0/1557959552 wait complete. 2024-09-19T07:05:19.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.000+0000 7f4e10dba740 1 -- start start 2024-09-19T07:05:19.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.000+0000 7f4e10dba740 1 Processor -- start 2024-09-19T07:05:19.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e10dba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da596c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:19.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e10dba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 0x5643da591f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:19.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e10dba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 0x5643da5924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:19.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e10dba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e081acb30 con 0x5643da57f5d0 2024-09-19T07:05:19.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e10dba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e081ac460 con 0x5643da580b40 2024-09-19T07:05:19.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e10dba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4e081a96c0 con 0x5643da58a370 2024-09-19T07:05:19.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e10dba740 10 client.0 _ll_register_callbacks cb 0x7f4df00056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:19.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e07fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da596c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:19.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e07fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da596c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60280/0 (socket says 172.21.15.103:60280) 2024-09-19T07:05:19.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e07fff640 1 -- 172.21.15.103:0/4126840353 learned_addr learned my addr 172.21.15.103:0/4126840353 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:19.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4dfffff640 1 --2- 172.21.15.103:0/4126840353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 0x5643da591f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:19.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e07fff640 1 -- 172.21.15.103:0/4126840353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 msgr2=0x5643da5924d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:19.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e0cf7e640 1 --2- 172.21.15.103:0/4126840353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 0x5643da5924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:19.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e07fff640 1 --2- 172.21.15.103:0/4126840353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 0x5643da5924d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e07fff640 1 -- 172.21.15.103:0/4126840353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 msgr2=0x5643da591f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:19.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e07fff640 1 --2- 172.21.15.103:0/4126840353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 0x5643da591f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e07fff640 1 -- 172.21.15.103:0/4126840353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643da592a10 con 0x5643da57f5d0 2024-09-19T07:05:19.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e0cf7e640 1 --2- 172.21.15.103:0/4126840353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 0x5643da5924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:19.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4dfffff640 1 --2- 172.21.15.103:0/4126840353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 0x5643da591f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:19.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e07fff640 1 --2- 172.21.15.103:0/4126840353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da596c60 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f4df8005880 tx=0x7f4df80058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e93ba08ef0b2cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:19.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4de77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:19.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4de77fe640 1 -- 172.21.15.103:0/4126840353 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8035070 con 0x5643da57f5d0 2024-09-19T07:05:19.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4e10dba740 1 -- 172.21.15.103:0/4126840353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4df8051070 con 0x5643da57f5d0 2024-09-19T07:05:19.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4de77fe640 1 -- 172.21.15.103:0/4126840353 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f4df8056620 con 0x5643da57f5d0 2024-09-19T07:05:19.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.001+0000 7f4de77fe640 1 -- 172.21.15.103:0/4126840353 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4df8046040 con 0x5643da57f5d0 2024-09-19T07:05:19.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.002+0000 7f4e10dba740 10 client.21207 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:19.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.002+0000 7f4e10dba740 1 -- 172.21.15.103:0/4126840353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5643da608bc0 con 0x5643da57f5d0 2024-09-19T07:05:19.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.002+0000 7f4de77fe640 1 -- 172.21.15.103:0/4126840353 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df8051070 con 0x5643da57f5d0 2024-09-19T07:05:19.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.002+0000 7f4de77fe640 1 -- 172.21.15.103:0/4126840353 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4df803a040 con 0x5643da57f5d0 2024-09-19T07:05:19.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.002+0000 7f4e10dba740 10 client.21207 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:19.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.002+0000 7f4e10dba740 10 client.21207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:19.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.002+0000 7f4e10dba740 20 client.21207 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:19.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.002+0000 7f4e10dba740 1 --2- 172.21.15.103:0/4126840353 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5643da608e40 0x5643da60b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:19.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.002+0000 7f4e10dba740 4 client.21207 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:05:19.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.002+0000 7f4e10dba740 1 -- 172.21.15.103:0/4126840353 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643da4ecd70 con 0x5643da608e40 2024-09-19T07:05:19.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.002+0000 7f4dfffff640 1 --2- 172.21.15.103:0/4126840353 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5643da608e40 0x5643da60b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:19.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.002+0000 7f4de77fe640 10 client.21207 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:19.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.002+0000 7f4dfffff640 1 --2- 172.21.15.103:0/4126840353 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5643da608e40 0x5643da60b230 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ed8da69e274c887 server_cookie=fc9d52ba86d32eaa in_seq=0 out_seq=0 2024-09-19T07:05:19.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.003+0000 7f4de77fe640 1 -- 172.21.15.103:0/4126840353 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643da4ecd70 con 0x5643da608e40 2024-09-19T07:05:19.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.003+0000 7f4de77fe640 10 client.21207 handle_command_reply: tid=0 2024-09-19T07:05:19.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.003+0000 7f4e10dba740 1 client.21207 shutdown 2024-09-19T07:05:19.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.003+0000 7f4e10dba740 1 -- 172.21.15.103:0/4126840353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 msgr2=0x5643da596c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:19.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.003+0000 7f4e10dba740 1 --2- 172.21.15.103:0/4126840353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da596c60 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f4df8005880 tx=0x7f4df80058b0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.003+0000 7f4e10dba740 1 -- 172.21.15.103:0/4126840353 shutdown_connections 2024-09-19T07:05:19.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.003+0000 7f4e10dba740 1 --2- 172.21.15.103:0/4126840353 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5643da608e40 0x5643da60b230 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.003+0000 7f4e10dba740 1 --2- 172.21.15.103:0/4126840353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643da58a370 0x5643da5924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.003+0000 7f4e10dba740 1 --2- 172.21.15.103:0/4126840353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643da580b40 0x5643da591f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.003+0000 7f4e10dba740 1 --2- 172.21.15.103:0/4126840353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643da57f5d0 0x5643da596c60 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.003+0000 7f4e10dba740 1 -- 172.21.15.103:0/4126840353 >> 172.21.15.103:0/4126840353 conn(0x5643da548310 msgr2=0x5643da585480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:19.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.003+0000 7f4de77fe640 0 client.21207 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:19.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.003+0000 7f4e10dba740 1 -- 172.21.15.103:0/4126840353 shutdown_connections 2024-09-19T07:05:19.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.003+0000 7f4e10dba740 1 -- 172.21.15.103:0/4126840353 wait complete. 2024-09-19T07:05:19.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.003+0000 7f4e10dba740 20 client.21207 trim_cache size 0 max 16384 2024-09-19T07:05:19.013 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:05:19.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.004+0000 7f4e0e9e1640 1 -- 172.21.15.103:0/2374573140 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4df40a3430 msgr2=0x7f4df40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:19.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.004+0000 7f4e0e9e1640 1 --2- 172.21.15.103:0/2374573140 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4df40a3430 0x7f4df40a58f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4e00005fd0 tx=0x7f4e00004ab0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.004+0000 7f4e0e9e1640 1 -- 172.21.15.103:0/2374573140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e08101420 msgr2=0x7f4e081978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:19.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.004+0000 7f4e0e9e1640 1 --2- 172.21.15.103:0/2374573140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e08101420 0x7f4e081978f0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f4df000d770 tx=0x7f4df000dc30 comp rx=0 tx=0).stop 2024-09-19T07:05:19.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.004+0000 7f4e0e9e1640 1 -- 172.21.15.103:0/2374573140 shutdown_connections 2024-09-19T07:05:19.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.004+0000 7f4e0e9e1640 1 --2- 172.21.15.103:0/2374573140 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4df40a3430 0x7f4df40a58f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.004+0000 7f4e0e9e1640 1 --2- 172.21.15.103:0/2374573140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e08106a80 0x7f4e0819bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.004+0000 7f4e0e9e1640 1 --2- 172.21.15.103:0/2374573140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e08101420 0x7f4e081978f0 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.004+0000 7f4e0e9e1640 1 --2- 172.21.15.103:0/2374573140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e08100a80 0x7f4e081973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:19.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.004+0000 7f4e0e9e1640 1 -- 172.21.15.103:0/2374573140 >> 172.21.15.103:0/2374573140 conn(0x7f4e080fc7a0 msgr2=0x7f4e080fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:19.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.004+0000 7f4e0e9e1640 1 -- 172.21.15.103:0/2374573140 shutdown_connections 2024-09-19T07:05:19.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:19.004+0000 7f4e0e9e1640 1 -- 172.21.15.103:0/2374573140 wait complete. 2024-09-19T07:05:19.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:19 smithi103 ceph-mon[31158]: pgmap v503: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-19T07:05:19.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:19 smithi134 ceph-mon[35356]: pgmap v503: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-19T07:05:19.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:19 smithi174 ceph-mon[35637]: pgmap v503: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-19T07:05:20.677 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:05:20.677 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:05:20.931 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:21.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.387+0000 7f34725a7640 1 -- 172.21.15.103:0/1218103527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f346c101420 msgr2=0x7f346c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:21.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.387+0000 7f34725a7640 1 --2- 172.21.15.103:0/1218103527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f346c101420 0x7f346c10d950 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f3464009cd0 tx=0x7f346402a8e0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.387+0000 7f34725a7640 1 -- 172.21.15.103:0/1218103527 shutdown_connections 2024-09-19T07:05:21.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.387+0000 7f34725a7640 1 --2- 172.21.15.103:0/1218103527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f346c101420 0x7f346c10d950 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.387+0000 7f34725a7640 1 --2- 172.21.15.103:0/1218103527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f346c100a80 0x7f346c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.387+0000 7f34725a7640 1 --2- 172.21.15.103:0/1218103527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f346c106a80 0x7f346c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.387+0000 7f34725a7640 1 -- 172.21.15.103:0/1218103527 >> 172.21.15.103:0/1218103527 conn(0x7f346c0fc7a0 msgr2=0x7f346c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:21.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.387+0000 7f34725a7640 1 -- 172.21.15.103:0/1218103527 shutdown_connections 2024-09-19T07:05:21.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.387+0000 7f34725a7640 1 -- 172.21.15.103:0/1218103527 wait complete. 2024-09-19T07:05:21.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.387+0000 7f34725a7640 1 Processor -- start 2024-09-19T07:05:21.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.387+0000 7f34725a7640 1 -- start start 2024-09-19T07:05:21.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f34725a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f346c100a80 0x7f346c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f34725a7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f346c101420 0x7f346c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f34725a7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f346c106a80 0x7f346c19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f34725a7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f346c110e80 con 0x7f346c100a80 2024-09-19T07:05:21.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f34725a7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f346c110d00 con 0x7f346c101420 2024-09-19T07:05:21.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f34725a7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f346c111000 con 0x7f346c106a80 2024-09-19T07:05:21.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f346bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f346c100a80 0x7f346c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f346bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f346c100a80 0x7f346c1972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60290/0 (socket says 172.21.15.103:60290) 2024-09-19T07:05:21.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f346bfff640 1 -- 172.21.15.103:0/1924736362 learned_addr learned my addr 172.21.15.103:0/1924736362 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:21.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f346b7fe640 1 --2- 172.21.15.103:0/1924736362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f346c101420 0x7f346c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f3470b44640 1 --2- 172.21.15.103:0/1924736362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f346c106a80 0x7f346c19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f346bfff640 1 -- 172.21.15.103:0/1924736362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f346c106a80 msgr2=0x7f346c19bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:21.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f346bfff640 1 --2- 172.21.15.103:0/1924736362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f346c106a80 0x7f346c19bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f346bfff640 1 -- 172.21.15.103:0/1924736362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f346c101420 msgr2=0x7f346c197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:21.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f346bfff640 1 --2- 172.21.15.103:0/1924736362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f346c101420 0x7f346c197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f346bfff640 1 -- 172.21.15.103:0/1924736362 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346c19c290 con 0x7f346c100a80 2024-09-19T07:05:21.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f346b7fe640 1 --2- 172.21.15.103:0/1924736362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f346c101420 0x7f346c197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:21.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f346bfff640 1 --2- 172.21.15.103:0/1924736362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f346c100a80 0x7f346c1972e0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f345c00d950 tx=0x7f345c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:21.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f34697fa640 1 -- 172.21.15.103:0/1924736362 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c017030 con 0x7f346c100a80 2024-09-19T07:05:21.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f34725a7640 1 -- 172.21.15.103:0/1924736362 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346c19c580 con 0x7f346c100a80 2024-09-19T07:05:21.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.388+0000 7f34697fa640 1 -- 172.21.15.103:0/1924736362 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f345c004020 con 0x7f346c100a80 2024-09-19T07:05:21.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.389+0000 7f34697fa640 1 -- 172.21.15.103:0/1924736362 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c0047a0 con 0x7f346c100a80 2024-09-19T07:05:21.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.389+0000 7f34725a7640 1 -- 172.21.15.103:0/1924736362 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346c06a790 con 0x7f346c100a80 2024-09-19T07:05:21.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.391+0000 7f34697fa640 1 -- 172.21.15.103:0/1924736362 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f345c0041d0 con 0x7f346c100a80 2024-09-19T07:05:21.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.391+0000 7f3474980740 1 Processor -- start 2024-09-19T07:05:21.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.391+0000 7f3474980740 1 -- start start 2024-09-19T07:05:21.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.391+0000 7f3474980740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458aa40 0x55d54458ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.391+0000 7f34697fa640 1 --2- 172.21.15.103:0/1924736362 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34580a33b0 0x7f34580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.391+0000 7f3474980740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458b360 0x55d544599c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.391+0000 7f3474980740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 0x55d54459c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.391+0000 7f3474980740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d5444f0280 con 0x55d54458aa40 2024-09-19T07:05:21.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.391+0000 7f3474980740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d544497b10 con 0x55d54459a150 2024-09-19T07:05:21.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.391+0000 7f3474980740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5444ec240 con 0x55d54458b360 2024-09-19T07:05:21.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f3470b44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458aa40 0x55d54458ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f3470b44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458aa40 0x55d54458ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60300/0 (socket says 172.21.15.103:60300) 2024-09-19T07:05:21.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f3470b44640 1 -- 172.21.15.103:0/4020763161 learned_addr learned my addr 172.21.15.103:0/4020763161 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:21.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f34697fa640 1 -- 172.21.15.103:0/1924736362 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f345c0cf790 con 0x7f346c100a80 2024-09-19T07:05:21.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f346bfff640 1 --2- 172.21.15.103:0/1924736362 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34580a33b0 0x7f34580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f346b7fe640 1 --2- 172.21.15.103:0/4020763161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458b360 0x55d544599c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f3470b44640 1 --2- 172.21.15.103:0/4020763161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 0x55d54459c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f3470b44640 1 -- 172.21.15.103:0/4020763161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458b360 msgr2=0x55d544599c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:21.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f3470b44640 1 --2- 172.21.15.103:0/4020763161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458b360 0x55d544599c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f3470b44640 1 -- 172.21.15.103:0/4020763161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 msgr2=0x55d54459c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:21.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f3470b44640 1 --2- 172.21.15.103:0/4020763161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 0x55d54459c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f3470b44640 1 -- 172.21.15.103:0/4020763161 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5444a41f0 con 0x55d54458aa40 2024-09-19T07:05:21.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f3470b44640 1 --2- 172.21.15.103:0/4020763161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458aa40 0x55d54458ae20 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f3464005fe0 tx=0x7f3464005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3487fd58c628d04 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:21.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f346bfff640 1 --2- 172.21.15.103:0/1924736362 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34580a33b0 0x7f34580a5870 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f345c00dfd0 tx=0x7f345c086040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:21.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f34627fc640 1 -- 172.21.15.103:0/4020763161 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464044070 con 0x55d54458aa40 2024-09-19T07:05:21.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f34627fc640 1 -- 172.21.15.103:0/4020763161 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3464036040 con 0x55d54458aa40 2024-09-19T07:05:21.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.392+0000 7f34627fc640 1 -- 172.21.15.103:0/4020763161 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464004500 con 0x55d54458aa40 2024-09-19T07:05:21.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.393+0000 7f3474980740 1 -- 172.21.15.103:0/4020763161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458aa40 msgr2=0x55d54458ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:21.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.393+0000 7f3474980740 1 --2- 172.21.15.103:0/4020763161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458aa40 0x55d54458ae20 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f3464005fe0 tx=0x7f3464005da0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.393+0000 7f3474980740 1 -- 172.21.15.103:0/4020763161 shutdown_connections 2024-09-19T07:05:21.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.393+0000 7f3474980740 1 --2- 172.21.15.103:0/4020763161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 0x55d54459c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.393+0000 7f3474980740 1 --2- 172.21.15.103:0/4020763161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458b360 0x55d544599c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.393+0000 7f3474980740 1 --2- 172.21.15.103:0/4020763161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458aa40 0x55d54458ae20 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.393+0000 7f3474980740 1 -- 172.21.15.103:0/4020763161 >> 172.21.15.103:0/4020763161 conn(0x55d5445870f0 msgr2=0x55d5445874d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:21.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.393+0000 7f3474980740 1 -- 172.21.15.103:0/4020763161 shutdown_connections 2024-09-19T07:05:21.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.393+0000 7f3474980740 1 -- 172.21.15.103:0/4020763161 wait complete. 2024-09-19T07:05:21.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.393+0000 7f3474980740 1 -- start start 2024-09-19T07:05:21.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f3474980740 1 Processor -- start 2024-09-19T07:05:21.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f3474980740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458aa40 0x55d544597040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f3474980740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458b360 0x55d544597580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f3474980740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 0x55d544597ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f3474980740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d54459cd20 con 0x55d54458b360 2024-09-19T07:05:21.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f3474980740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d5444aaf10 con 0x55d54459a150 2024-09-19T07:05:21.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f3474980740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d54459cea0 con 0x55d54458aa40 2024-09-19T07:05:21.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f3474980740 10 client.0 _ll_register_callbacks cb 0x55d544473a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:21.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458b360 0x55d544597580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458b360 0x55d544597580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60310/0 (socket says 172.21.15.103:60310) 2024-09-19T07:05:21.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f346b7fe640 1 -- 172.21.15.103:0/3157894662 learned_addr learned my addr 172.21.15.103:0/3157894662 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:21.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f3470b44640 1 --2- 172.21.15.103:0/3157894662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458aa40 0x55d544597040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f3470b44640 1 --2- 172.21.15.103:0/3157894662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 0x55d544597ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f346b7fe640 1 -- 172.21.15.103:0/3157894662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458aa40 msgr2=0x55d544597040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:21.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f346b7fe640 1 --2- 172.21.15.103:0/3157894662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458aa40 0x55d544597040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f346b7fe640 1 -- 172.21.15.103:0/3157894662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 msgr2=0x55d544597ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:21.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f346b7fe640 1 --2- 172.21.15.103:0/3157894662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 0x55d544597ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f346b7fe640 1 -- 172.21.15.103:0/3157894662 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5442616a0 con 0x55d54458b360 2024-09-19T07:05:21.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.394+0000 7f3470b44640 1 --2- 172.21.15.103:0/3157894662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 0x55d544597ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:21.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.395+0000 7f346b7fe640 1 --2- 172.21.15.103:0/3157894662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458b360 0x55d544597580 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f345400bf80 tx=0x7f34540128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1b44f5322f6747 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:21.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.395+0000 7f344b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:21.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.395+0000 7f344b7fe640 1 -- 172.21.15.103:0/3157894662 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34540035f0 con 0x55d54458b360 2024-09-19T07:05:21.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.395+0000 7f344b7fe640 1 -- 172.21.15.103:0/3157894662 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f3454003790 con 0x55d54458b360 2024-09-19T07:05:21.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.395+0000 7f3474980740 1 -- 172.21.15.103:0/3157894662 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3464036040 con 0x55d54458b360 2024-09-19T07:05:21.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.395+0000 7f344b7fe640 1 -- 172.21.15.103:0/3157894662 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3454003990 con 0x55d54458b360 2024-09-19T07:05:21.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.395+0000 7f3474980740 10 client.21219 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:21.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.395+0000 7f3474980740 1 -- 172.21.15.103:0/3157894662 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d5445a89d0 con 0x55d54458b360 2024-09-19T07:05:21.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.395+0000 7f344b7fe640 1 -- 172.21.15.103:0/3157894662 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3464036040 con 0x55d54458b360 2024-09-19T07:05:21.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.395+0000 7f344b7fe640 1 -- 172.21.15.103:0/3157894662 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3454019070 con 0x55d54458b360 2024-09-19T07:05:21.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.395+0000 7f3474980740 10 client.21219 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:21.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.395+0000 7f3474980740 10 client.21219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:21.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.395+0000 7f3474980740 20 client.21219 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:21.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.395+0000 7f3474980740 1 --2- 172.21.15.103:0/3157894662 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d5445a9f30 0x55d5445ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.395+0000 7f3474980740 4 client.21219 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:21.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.395+0000 7f3474980740 1 -- 172.21.15.103:0/3157894662 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5444fd7f0 con 0x55d5445a9f30 2024-09-19T07:05:21.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.396+0000 7f3470b44640 1 --2- 172.21.15.103:0/3157894662 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d5445a9f30 0x55d5445ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.396+0000 7f344b7fe640 10 client.21219 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:21.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.396+0000 7f3470b44640 1 --2- 172.21.15.103:0/3157894662 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d5445a9f30 0x55d5445ac350 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd6544d3d181d4de server_cookie=3b14a0dba247d37f in_seq=0 out_seq=0 2024-09-19T07:05:21.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.397+0000 7f344b7fe640 1 -- 172.21.15.103:0/3157894662 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5444fd7f0 con 0x55d5445a9f30 2024-09-19T07:05:21.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.397+0000 7f344b7fe640 10 client.21219 handle_command_reply: tid=0 2024-09-19T07:05:21.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.397+0000 7f3474980740 1 client.21219 shutdown 2024-09-19T07:05:21.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.397+0000 7f3474980740 1 -- 172.21.15.103:0/3157894662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458b360 msgr2=0x55d544597580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:21.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.397+0000 7f3474980740 1 --2- 172.21.15.103:0/3157894662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458b360 0x55d544597580 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f345400bf80 tx=0x7f34540128e0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.397+0000 7f3474980740 1 -- 172.21.15.103:0/3157894662 shutdown_connections 2024-09-19T07:05:21.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.397+0000 7f3474980740 1 --2- 172.21.15.103:0/3157894662 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d5445a9f30 0x55d5445ac350 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.397+0000 7f3474980740 1 --2- 172.21.15.103:0/3157894662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 0x55d544597ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.397+0000 7f3474980740 1 --2- 172.21.15.103:0/3157894662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458b360 0x55d544597580 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.397+0000 7f344b7fe640 0 client.21219 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:21.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.397+0000 7f3474980740 1 --2- 172.21.15.103:0/3157894662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458aa40 0x55d544597040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.397+0000 7f3474980740 1 -- 172.21.15.103:0/3157894662 >> 172.21.15.103:0/3157894662 conn(0x55d5445870f0 msgr2=0x55d54458ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:21.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.397+0000 7f3474980740 1 -- 172.21.15.103:0/3157894662 shutdown_connections 2024-09-19T07:05:21.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.397+0000 7f3474980740 1 -- 172.21.15.103:0/3157894662 wait complete. 2024-09-19T07:05:21.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.397+0000 7f3474980740 20 client.21219 trim_cache size 0 max 16384 2024-09-19T07:05:21.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.414+0000 7f3474980740 1 Processor -- start 2024-09-19T07:05:21.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.414+0000 7f3474980740 1 -- start start 2024-09-19T07:05:21.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3474980740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458aa40 0x55d544597910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3474980740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458b360 0x55d5445b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3474980740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 0x55d5445b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3474980740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ab710 con 0x55d54458aa40 2024-09-19T07:05:21.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3474980740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ad250 con 0x55d54459a150 2024-09-19T07:05:21.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3474980740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1abde0 con 0x55d54458b360 2024-09-19T07:05:21.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3470b44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458aa40 0x55d544597910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3470b44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458aa40 0x55d544597910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60324/0 (socket says 172.21.15.103:60324) 2024-09-19T07:05:21.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3470b44640 1 -- 172.21.15.103:0/3207573872 learned_addr learned my addr 172.21.15.103:0/3207573872 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:21.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f346b7fe640 1 --2- 172.21.15.103:0/3207573872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458b360 0x55d5445b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3470b44640 1 -- 172.21.15.103:0/3207573872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458b360 msgr2=0x55d5445b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:21.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3470b44640 1 --2- 172.21.15.103:0/3207573872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458b360 0x55d5445b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3470b44640 1 -- 172.21.15.103:0/3207573872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 msgr2=0x55d5445b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:21.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3470b44640 1 --2- 172.21.15.103:0/3207573872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 0x55d5445b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3470b44640 1 -- 172.21.15.103:0/3207573872 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5445ac9b0 con 0x55d54458aa40 2024-09-19T07:05:21.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f346b7fe640 1 --2- 172.21.15.103:0/3207573872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458b360 0x55d5445b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:21.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3470b44640 1 --2- 172.21.15.103:0/3207573872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458aa40 0x55d544597910 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f3464006010 tx=0x7f3464005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4e95bdef594bb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:21.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3461ffb640 1 -- 172.21.15.103:0/3207573872 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464044040 con 0x55d54458aa40 2024-09-19T07:05:21.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3461ffb640 1 -- 172.21.15.103:0/3207573872 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3464036040 con 0x55d54458aa40 2024-09-19T07:05:21.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.415+0000 7f3461ffb640 1 -- 172.21.15.103:0/3207573872 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464004750 con 0x55d54458aa40 2024-09-19T07:05:21.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.416+0000 7f3474980740 1 -- 172.21.15.103:0/3207573872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458aa40 msgr2=0x55d544597910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:21.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.416+0000 7f3474980740 1 --2- 172.21.15.103:0/3207573872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458aa40 0x55d544597910 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f3464006010 tx=0x7f3464005da0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.416+0000 7f3474980740 1 -- 172.21.15.103:0/3207573872 shutdown_connections 2024-09-19T07:05:21.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.416+0000 7f3474980740 1 --2- 172.21.15.103:0/3207573872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 0x55d5445b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.416+0000 7f3474980740 1 --2- 172.21.15.103:0/3207573872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458b360 0x55d5445b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.416+0000 7f3474980740 1 --2- 172.21.15.103:0/3207573872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458aa40 0x55d544597910 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.416+0000 7f3474980740 1 -- 172.21.15.103:0/3207573872 >> 172.21.15.103:0/3207573872 conn(0x55d5445870f0 msgr2=0x55d544598fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:21.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.416+0000 7f3474980740 1 -- 172.21.15.103:0/3207573872 shutdown_connections 2024-09-19T07:05:21.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.416+0000 7f3474980740 1 -- 172.21.15.103:0/3207573872 wait complete. 2024-09-19T07:05:21.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f3474980740 1 -- start start 2024-09-19T07:05:21.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f3474980740 1 Processor -- start 2024-09-19T07:05:21.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f3474980740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458aa40 0x55d5445a2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f3474980740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458b360 0x55d5445a2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f3474980740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 0x55d54459f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f3474980740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1acb80 con 0x55d54458b360 2024-09-19T07:05:21.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f3474980740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ac4b0 con 0x55d54459a150 2024-09-19T07:05:21.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f3474980740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1a9710 con 0x55d54458aa40 2024-09-19T07:05:21.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f3474980740 10 client.0 _ll_register_callbacks cb 0x7f345400b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:21.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458b360 0x55d5445a2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458b360 0x55d5445a2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60330/0 (socket says 172.21.15.103:60330) 2024-09-19T07:05:21.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f346b7fe640 1 -- 172.21.15.103:0/4186876724 learned_addr learned my addr 172.21.15.103:0/4186876724 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:21.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f3470b44640 1 --2- 172.21.15.103:0/4186876724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 0x55d54459f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f3470b44640 1 --2- 172.21.15.103:0/4186876724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458aa40 0x55d5445a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f346b7fe640 1 -- 172.21.15.103:0/4186876724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458aa40 msgr2=0x55d5445a2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:21.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f346b7fe640 1 --2- 172.21.15.103:0/4186876724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458aa40 0x55d5445a2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f346b7fe640 1 -- 172.21.15.103:0/4186876724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 msgr2=0x55d54459f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:21.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f346b7fe640 1 --2- 172.21.15.103:0/4186876724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 0x55d54459f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.417+0000 7f346b7fe640 1 -- 172.21.15.103:0/4186876724 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d54459f640 con 0x55d54458b360 2024-09-19T07:05:21.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.418+0000 7f346b7fe640 1 --2- 172.21.15.103:0/4186876724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458b360 0x55d5445a2670 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f3454010ee0 tx=0x7f345400bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc942790d34879a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:21.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.418+0000 7f344b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:21.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.418+0000 7f344b7fe640 1 -- 172.21.15.103:0/4186876724 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454003d30 con 0x55d54458b360 2024-09-19T07:05:21.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.418+0000 7f3474980740 1 -- 172.21.15.103:0/4186876724 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3464036040 con 0x55d54458b360 2024-09-19T07:05:21.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.418+0000 7f344b7fe640 1 -- 172.21.15.103:0/4186876724 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f3454010490 con 0x55d54458b360 2024-09-19T07:05:21.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.418+0000 7f344b7fe640 1 -- 172.21.15.103:0/4186876724 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f34540106b0 con 0x55d54458b360 2024-09-19T07:05:21.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.418+0000 7f3474980740 10 client.21225 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:21.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.418+0000 7f3474980740 1 -- 172.21.15.103:0/4186876724 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d544615330 con 0x55d54458b360 2024-09-19T07:05:21.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.418+0000 7f344b7fe640 1 -- 172.21.15.103:0/4186876724 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3464036040 con 0x55d54458b360 2024-09-19T07:05:21.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.418+0000 7f344b7fe640 1 -- 172.21.15.103:0/4186876724 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3454019040 con 0x55d54458b360 2024-09-19T07:05:21.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.418+0000 7f3474980740 10 client.21225 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:21.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.418+0000 7f3474980740 10 client.21225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:21.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.418+0000 7f3474980740 20 client.21225 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:21.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.418+0000 7f3474980740 1 --2- 172.21.15.103:0/4186876724 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d544615a80 0x55d544617ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:21.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.418+0000 7f3474980740 4 client.21225 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:05:21.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.418+0000 7f3474980740 1 -- 172.21.15.103:0/4186876724 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d5444f7d00 con 0x55d544615a80 2024-09-19T07:05:21.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.419+0000 7f3470b44640 1 --2- 172.21.15.103:0/4186876724 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d544615a80 0x55d544617ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:21.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.419+0000 7f344b7fe640 10 client.21225 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:21.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.419+0000 7f3470b44640 1 --2- 172.21.15.103:0/4186876724 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d544615a80 0x55d544617ea0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcddd1fd1a963ab3 server_cookie=bb30173b37ccf6e6 in_seq=0 out_seq=0 2024-09-19T07:05:21.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.420+0000 7f344b7fe640 1 -- 172.21.15.103:0/4186876724 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d5444f7d00 con 0x55d544615a80 2024-09-19T07:05:21.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.420+0000 7f344b7fe640 10 client.21225 handle_command_reply: tid=0 2024-09-19T07:05:21.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.420+0000 7f3474980740 1 client.21225 shutdown 2024-09-19T07:05:21.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.420+0000 7f3474980740 1 -- 172.21.15.103:0/4186876724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458b360 msgr2=0x55d5445a2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:21.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.420+0000 7f3474980740 1 --2- 172.21.15.103:0/4186876724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458b360 0x55d5445a2670 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f3454010ee0 tx=0x7f345400bc30 comp rx=0 tx=0).stop 2024-09-19T07:05:21.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.420+0000 7f3474980740 1 -- 172.21.15.103:0/4186876724 shutdown_connections 2024-09-19T07:05:21.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.420+0000 7f3474980740 1 --2- 172.21.15.103:0/4186876724 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55d544615a80 0x55d544617ea0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f3474980740 1 --2- 172.21.15.103:0/4186876724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d54459a150 0x55d54459f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f3474980740 1 --2- 172.21.15.103:0/4186876724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d54458b360 0x55d5445a2670 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f3474980740 1 --2- 172.21.15.103:0/4186876724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d54458aa40 0x55d5445a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f3474980740 1 -- 172.21.15.103:0/4186876724 >> 172.21.15.103:0/4186876724 conn(0x55d5445870f0 msgr2=0x55d544585a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:21.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f344b7fe640 0 client.21225 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:21.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f3474980740 1 -- 172.21.15.103:0/4186876724 shutdown_connections 2024-09-19T07:05:21.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f3474980740 1 -- 172.21.15.103:0/4186876724 wait complete. 2024-09-19T07:05:21.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f3474980740 20 client.21225 trim_cache size 0 max 16384 2024-09-19T07:05:21.451 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:05:21.451 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:05:21.451 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "70c3565e-7f0a-493e-b524-142863861290", 2024-09-19T07:05:21.452 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:05:21.452 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:05:21.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f34725a7640 1 -- 172.21.15.103:0/1924736362 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34580a33b0 msgr2=0x7f34580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:21.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f34725a7640 1 --2- 172.21.15.103:0/1924736362 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34580a33b0 0x7f34580a5870 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f345c00dfd0 tx=0x7f345c086040 comp rx=0 tx=0).stop 2024-09-19T07:05:21.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f34725a7640 1 -- 172.21.15.103:0/1924736362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f346c100a80 msgr2=0x7f346c1972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:21.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f34725a7640 1 --2- 172.21.15.103:0/1924736362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f346c100a80 0x7f346c1972e0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f345c00d950 tx=0x7f345c00de10 comp rx=0 tx=0).stop 2024-09-19T07:05:21.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f34725a7640 1 -- 172.21.15.103:0/1924736362 shutdown_connections 2024-09-19T07:05:21.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f34725a7640 1 --2- 172.21.15.103:0/1924736362 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34580a33b0 0x7f34580a5870 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f34725a7640 1 --2- 172.21.15.103:0/1924736362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f346c106a80 0x7f346c19bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f34725a7640 1 --2- 172.21.15.103:0/1924736362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f346c101420 0x7f346c197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f34725a7640 1 --2- 172.21.15.103:0/1924736362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f346c100a80 0x7f346c1972e0 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:21.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f34725a7640 1 -- 172.21.15.103:0/1924736362 >> 172.21.15.103:0/1924736362 conn(0x7f346c0fc7a0 msgr2=0x7f346c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:21.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f34725a7640 1 -- 172.21.15.103:0/1924736362 shutdown_connections 2024-09-19T07:05:21.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:21.421+0000 7f34725a7640 1 -- 172.21.15.103:0/1924736362 wait complete. 2024-09-19T07:05:21.544 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:05:21.567 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:21 smithi103 ceph-mon[31158]: pgmap v504: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-09-19T07:05:21.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:21 smithi134 ceph-mon[35356]: pgmap v504: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-09-19T07:05:21.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:21 smithi174 ceph-mon[35637]: pgmap v504: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-09-19T07:05:22.086 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:05:22.183 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:22.636 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:22 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T07:05:22.636 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:22 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:05:22.636 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:22 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T07:05:22.636 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:22 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:05:22.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.650+0000 7f312b4b5640 1 -- 172.21.15.103:0/517584613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3124101e20 msgr2=0x7f31241056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:22.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.650+0000 7f312b4b5640 1 --2- 172.21.15.103:0/517584613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3124101e20 0x7f31241056b0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f3120009900 tx=0x7f312002a410 comp rx=0 tx=0).stop 2024-09-19T07:05:22.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.650+0000 7f312b4b5640 1 -- 172.21.15.103:0/517584613 shutdown_connections 2024-09-19T07:05:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.650+0000 7f312b4b5640 1 --2- 172.21.15.103:0/517584613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3124105de0 0x7f312410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.650+0000 7f312b4b5640 1 --2- 172.21.15.103:0/517584613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3124101e20 0x7f31241056b0 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.650+0000 7f312b4b5640 1 --2- 172.21.15.103:0/517584613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3124101470 0x7f3124101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.650+0000 7f312b4b5640 1 -- 172.21.15.103:0/517584613 >> 172.21.15.103:0/517584613 conn(0x7f3124077230 msgr2=0x7f31240fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.650+0000 7f312b4b5640 1 -- 172.21.15.103:0/517584613 shutdown_connections 2024-09-19T07:05:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.650+0000 7f312b4b5640 1 -- 172.21.15.103:0/517584613 wait complete. 2024-09-19T07:05:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.650+0000 7f312b4b5640 1 Processor -- start 2024-09-19T07:05:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f312b4b5640 1 -- start start 2024-09-19T07:05:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f312b4b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3124101470 0x7f31240fc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f312b4b5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3124101e20 0x7f31240fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f312b4b5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3124105de0 0x7f31240fd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f312b4b5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3124110e40 con 0x7f3124101470 2024-09-19T07:05:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f312b4b5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3124110cc0 con 0x7f3124105de0 2024-09-19T07:05:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f312b4b5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3124110fc0 con 0x7f3124101e20 2024-09-19T07:05:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f3129251640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3124101470 0x7f31240fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f3129251640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3124101470 0x7f31240fc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60360/0 (socket says 172.21.15.103:60360) 2024-09-19T07:05:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f3129251640 1 -- 172.21.15.103:0/3153813134 learned_addr learned my addr 172.21.15.103:0/3153813134 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f3128a50640 1 --2- 172.21.15.103:0/3153813134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3124101e20 0x7f31240fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f3129a52640 1 --2- 172.21.15.103:0/3153813134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3124105de0 0x7f31240fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f3129251640 1 -- 172.21.15.103:0/3153813134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3124101e20 msgr2=0x7f31240fcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f3129251640 1 --2- 172.21.15.103:0/3153813134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3124101e20 0x7f31240fcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:22.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f3129251640 1 -- 172.21.15.103:0/3153813134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3124105de0 msgr2=0x7f31240fd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f3129251640 1 --2- 172.21.15.103:0/3153813134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3124105de0 0x7f31240fd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f3129251640 1 -- 172.21.15.103:0/3153813134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31241a6860 con 0x7f3124101470 2024-09-19T07:05:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.651+0000 7f3129a52640 1 --2- 172.21.15.103:0/3153813134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3124105de0 0x7f31240fd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.652+0000 7f3129251640 1 --2- 172.21.15.103:0/3153813134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3124101470 0x7f31240fc700 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f311400c920 tx=0x7f311400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.652+0000 7f311a7fc640 1 -- 172.21.15.103:0/3153813134 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114007c70 con 0x7f3124101470 2024-09-19T07:05:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.652+0000 7f311a7fc640 1 -- 172.21.15.103:0/3153813134 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3114007e10 con 0x7f3124101470 2024-09-19T07:05:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.652+0000 7f312b4b5640 1 -- 172.21.15.103:0/3153813134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31241a6b50 con 0x7f3124101470 2024-09-19T07:05:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.652+0000 7f311a7fc640 1 -- 172.21.15.103:0/3153813134 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114018890 con 0x7f3124101470 2024-09-19T07:05:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.652+0000 7f312b4b5640 1 -- 172.21.15.103:0/3153813134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31241a7060 con 0x7f3124101470 2024-09-19T07:05:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.653+0000 7f312b4b5640 1 -- 172.21.15.103:0/3153813134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3124102220 con 0x7f3124101470 2024-09-19T07:05:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.654+0000 7f311a7fc640 1 -- 172.21.15.103:0/3153813134 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3114018a50 con 0x7f3124101470 2024-09-19T07:05:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.655+0000 7f311a7fc640 1 --2- 172.21.15.103:0/3153813134 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f31100a33b0 0x7f31100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.655+0000 7f311a7fc640 1 -- 172.21.15.103:0/3153813134 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f31140d0110 con 0x7f3124101470 2024-09-19T07:05:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.655+0000 7f3128a50640 1 --2- 172.21.15.103:0/3153813134 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f31100a33b0 0x7f31100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:22.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.656+0000 7f3128a50640 1 --2- 172.21.15.103:0/3153813134 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f31100a33b0 0x7f31100a5870 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3120009230 tx=0x7f31200058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.661+0000 7f311a7fc640 1 -- 172.21.15.103:0/3153813134 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3114014030 con 0x7f3124101470 2024-09-19T07:05:22.679 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:22.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:22 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T07:05:22.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:22 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:05:22.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:22 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T07:05:22.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:22 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:05:22.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:22 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T07:05:22.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:22 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:05:22.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:22 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T07:05:22.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:22 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:05:22.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.872+0000 7f312b4b5640 1 -- 172.21.15.103:0/3153813134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3124101850 con 0x7f3124101470 2024-09-19T07:05:22.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.872+0000 7f311a7fc640 1 -- 172.21.15.103:0/3153813134 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f3124101850 con 0x7f3124101470 2024-09-19T07:05:22.874 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:05:22.874 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:05:22.874 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:05:22.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.875+0000 7f312b4b5640 1 -- 172.21.15.103:0/3153813134 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f31100a33b0 msgr2=0x7f31100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:22.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.875+0000 7f312b4b5640 1 --2- 172.21.15.103:0/3153813134 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f31100a33b0 0x7f31100a5870 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3120009230 tx=0x7f31200058d0 comp rx=0 tx=0).stop 2024-09-19T07:05:22.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.875+0000 7f312b4b5640 1 -- 172.21.15.103:0/3153813134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3124101470 msgr2=0x7f31240fc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:22.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.875+0000 7f312b4b5640 1 --2- 172.21.15.103:0/3153813134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3124101470 0x7f31240fc700 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f311400c920 tx=0x7f311400cde0 comp rx=0 tx=0).stop 2024-09-19T07:05:22.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.875+0000 7f312b4b5640 1 -- 172.21.15.103:0/3153813134 shutdown_connections 2024-09-19T07:05:22.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.875+0000 7f312b4b5640 1 --2- 172.21.15.103:0/3153813134 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f31100a33b0 0x7f31100a5870 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:22.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.875+0000 7f312b4b5640 1 --2- 172.21.15.103:0/3153813134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3124105de0 0x7f31240fd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:22.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.875+0000 7f312b4b5640 1 --2- 172.21.15.103:0/3153813134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3124101e20 0x7f31240fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:22.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.875+0000 7f312b4b5640 1 --2- 172.21.15.103:0/3153813134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3124101470 0x7f31240fc700 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:22.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.875+0000 7f312b4b5640 1 -- 172.21.15.103:0/3153813134 >> 172.21.15.103:0/3153813134 conn(0x7f3124077230 msgr2=0x7f3124100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:22.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.875+0000 7f312b4b5640 1 -- 172.21.15.103:0/3153813134 shutdown_connections 2024-09-19T07:05:22.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:22.875+0000 7f312b4b5640 1 -- 172.21.15.103:0/3153813134 wait complete. 2024-09-19T07:05:23.493 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:23 smithi103 ceph-mon[31158]: pgmap v505: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-19T07:05:23.493 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:23 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3153813134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:05:23.520 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 87781512 --release --await' 2024-09-19T07:05:23.521 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 87781512 --release --await 2024-09-19T07:05:23.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.643+0000 7f66b02f0640 1 -- 172.21.15.103:0/2254186604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66a8102d90 msgr2=0x7f66a8103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:23.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.643+0000 7f66b02f0640 1 --2- 172.21.15.103:0/2254186604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66a8102d90 0x7f66a8103170 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f669c009930 tx=0x7f669c02a420 comp rx=0 tx=0).stop 2024-09-19T07:05:23.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.643+0000 7f66b02f0640 1 -- 172.21.15.103:0/2254186604 shutdown_connections 2024-09-19T07:05:23.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.643+0000 7f66b02f0640 1 --2- 172.21.15.103:0/2254186604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66a8108290 0x7f66a810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.643+0000 7f66b02f0640 1 --2- 172.21.15.103:0/2254186604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a8103740 0x7f66a8107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.643+0000 7f66b02f0640 1 --2- 172.21.15.103:0/2254186604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66a8102d90 0x7f66a8103170 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.643+0000 7f66b02f0640 1 -- 172.21.15.103:0/2254186604 >> 172.21.15.103:0/2254186604 conn(0x7f66a80fc7d0 msgr2=0x7f66a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:23.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.643+0000 7f66b02f0640 1 -- 172.21.15.103:0/2254186604 shutdown_connections 2024-09-19T07:05:23.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.643+0000 7f66b02f0640 1 -- 172.21.15.103:0/2254186604 wait complete. 2024-09-19T07:05:23.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.643+0000 7f66b02f0640 1 Processor -- start 2024-09-19T07:05:23.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.643+0000 7f66b02f0640 1 -- start start 2024-09-19T07:05:23.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66b02f0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66a8102d90 0x7f66a81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66b02f0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66a8103740 0x7f66a8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66b02f0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a8108290 0x7f66a819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66b02f0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81120f0 con 0x7f66a8103740 2024-09-19T07:05:23.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66b02f0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66a8111f70 con 0x7f66a8108290 2024-09-19T07:05:23.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66b02f0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66a8112270 con 0x7f66a8102d90 2024-09-19T07:05:23.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66ad88b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66a8103740 0x7f66a8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66ad88b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66a8103740 0x7f66a8198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60378/0 (socket says 172.21.15.103:60378) 2024-09-19T07:05:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66ad88b640 1 -- 172.21.15.103:0/1437774334 learned_addr learned my addr 172.21.15.103:0/1437774334 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66ae08c640 1 --2- 172.21.15.103:0/1437774334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66a8102d90 0x7f66a81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66ae88d640 1 --2- 172.21.15.103:0/1437774334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a8108290 0x7f66a819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66ad88b640 1 -- 172.21.15.103:0/1437774334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66a8102d90 msgr2=0x7f66a81986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66ad88b640 1 --2- 172.21.15.103:0/1437774334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66a8102d90 0x7f66a81986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66ad88b640 1 -- 172.21.15.103:0/1437774334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a8108290 msgr2=0x7f66a819cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66ad88b640 1 --2- 172.21.15.103:0/1437774334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a8108290 0x7f66a819cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66ad88b640 1 -- 172.21.15.103:0/1437774334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a819d680 con 0x7f66a8103740 2024-09-19T07:05:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66ae88d640 1 --2- 172.21.15.103:0/1437774334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a8108290 0x7f66a819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f66ad88b640 1 --2- 172.21.15.103:0/1437774334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66a8103740 0x7f66a8198c10 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f66a400b450 tx=0x7f66a400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.644+0000 7f669b7fe640 1 -- 172.21.15.103:0/1437774334 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a4015020 con 0x7f66a8103740 2024-09-19T07:05:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.645+0000 7f669b7fe640 1 -- 172.21.15.103:0/1437774334 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66a4011d70 con 0x7f66a8103740 2024-09-19T07:05:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.645+0000 7f66b02f0640 1 -- 172.21.15.103:0/1437774334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a819d970 con 0x7f66a8103740 2024-09-19T07:05:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.645+0000 7f669b7fe640 1 -- 172.21.15.103:0/1437774334 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a400f340 con 0x7f66a8103740 2024-09-19T07:05:23.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.645+0000 7f66b02f0640 1 -- 172.21.15.103:0/1437774334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a81a5250 con 0x7f66a8103740 2024-09-19T07:05:23.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.647+0000 7f669b7fe640 1 -- 172.21.15.103:0/1437774334 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f66a4018020 con 0x7f66a8103740 2024-09-19T07:05:23.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.647+0000 7f66b26c9740 1 Processor -- start 2024-09-19T07:05:23.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.647+0000 7f66b26c9740 1 -- start start 2024-09-19T07:05:23.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66b26c9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095b83c0 0x5635095b87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66b26c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b8ce0 0x5635095c7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66b26c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095c7a90 0x5635095c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66b26c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56350951e2e0 con 0x5635095b8ce0 2024-09-19T07:05:23.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66b26c9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5635094c5ce0 con 0x5635095b83c0 2024-09-19T07:05:23.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66b26c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56350951a2a0 con 0x5635095c7a90 2024-09-19T07:05:23.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66ae08c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b8ce0 0x5635095c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66ae08c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b8ce0 0x5635095c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60384/0 (socket says 172.21.15.103:60384) 2024-09-19T07:05:23.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66ae08c640 1 -- 172.21.15.103:0/1276969330 learned_addr learned my addr 172.21.15.103:0/1276969330 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:23.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66ad88b640 1 --2- 172.21.15.103:0/1276969330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095c7a90 0x5635095c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66ae88d640 1 --2- 172.21.15.103:0/1276969330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095b83c0 0x5635095b87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f669b7fe640 1 --2- 172.21.15.103:0/1437774334 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66940a3430 0x7f66940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66ae08c640 1 -- 172.21.15.103:0/1276969330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095c7a90 msgr2=0x5635095c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:23.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66ae08c640 1 --2- 172.21.15.103:0/1276969330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095c7a90 0x5635095c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66ae08c640 1 -- 172.21.15.103:0/1276969330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095b83c0 msgr2=0x5635095b87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:23.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66ae08c640 1 --2- 172.21.15.103:0/1276969330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095b83c0 0x5635095b87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66ae08c640 1 -- 172.21.15.103:0/1276969330 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635094aa440 con 0x5635095b8ce0 2024-09-19T07:05:23.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66ad88b640 1 --2- 172.21.15.103:0/1276969330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095c7a90 0x5635095c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:23.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f66ae88d640 1 --2- 172.21.15.103:0/1437774334 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66940a3430 0x7f66940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.648+0000 7f669b7fe640 1 -- 172.21.15.103:0/1437774334 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f66a40cf670 con 0x7f66a8103740 2024-09-19T07:05:23.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.649+0000 7f66ae08c640 1 --2- 172.21.15.103:0/1276969330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b8ce0 0x5635095c7550 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f669c009930 tx=0x7f669c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a4591443190272 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:23.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.649+0000 7f6698dea640 1 -- 172.21.15.103:0/1276969330 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c040070 con 0x5635095b8ce0 2024-09-19T07:05:23.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.649+0000 7f66ae88d640 1 --2- 172.21.15.103:0/1437774334 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66940a3430 0x7f66940a58f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f66a0005fd0 tx=0x7f66a0004a40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:23.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.649+0000 7f6698dea640 1 -- 172.21.15.103:0/1276969330 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669c005020 con 0x5635095b8ce0 2024-09-19T07:05:23.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.649+0000 7f6698dea640 1 -- 172.21.15.103:0/1276969330 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c035040 con 0x5635095b8ce0 2024-09-19T07:05:23.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.650+0000 7f66b26c9740 1 -- 172.21.15.103:0/1276969330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b8ce0 msgr2=0x5635095c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:23.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.650+0000 7f66b26c9740 1 --2- 172.21.15.103:0/1276969330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b8ce0 0x5635095c7550 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f669c009930 tx=0x7f669c031630 comp rx=0 tx=0).stop 2024-09-19T07:05:23.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.650+0000 7f66b26c9740 1 -- 172.21.15.103:0/1276969330 shutdown_connections 2024-09-19T07:05:23.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.650+0000 7f66b26c9740 1 --2- 172.21.15.103:0/1276969330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095c7a90 0x5635095c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.650+0000 7f66b26c9740 1 --2- 172.21.15.103:0/1276969330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b8ce0 0x5635095c7550 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.650+0000 7f66b26c9740 1 --2- 172.21.15.103:0/1276969330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095b83c0 0x5635095b87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.650+0000 7f66b26c9740 1 -- 172.21.15.103:0/1276969330 >> 172.21.15.103:0/1276969330 conn(0x5635095b4a30 msgr2=0x5635095b4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:23.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.650+0000 7f66b26c9740 1 -- 172.21.15.103:0/1276969330 shutdown_connections 2024-09-19T07:05:23.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.650+0000 7f66b26c9740 1 -- 172.21.15.103:0/1276969330 wait complete. 2024-09-19T07:05:23.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.650+0000 7f66b26c9740 1 -- start start 2024-09-19T07:05:23.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.650+0000 7f66b26c9740 1 Processor -- start 2024-09-19T07:05:23.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66b26c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095b83c0 0x5635095c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66b26c9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095b8ce0 0x5635095c4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66b26c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095c7a90 0x5635095c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66b26c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5635095ca5a0 con 0x5635095c7a90 2024-09-19T07:05:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66b26c9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5635094d9050 con 0x5635095b8ce0 2024-09-19T07:05:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66b26c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5635095ca720 con 0x5635095b83c0 2024-09-19T07:05:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66ae88d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095c7a90 0x5635095c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66b26c9740 10 client.0 _ll_register_callbacks cb 0x563509519160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66ae88d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095c7a90 0x5635095c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60396/0 (socket says 172.21.15.103:60396) 2024-09-19T07:05:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66ae88d640 1 -- 172.21.15.103:0/3589307246 learned_addr learned my addr 172.21.15.103:0/3589307246 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66ae08c640 1 --2- 172.21.15.103:0/3589307246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095b83c0 0x5635095c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66ad88b640 1 --2- 172.21.15.103:0/3589307246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095b8ce0 0x5635095c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66ae88d640 1 -- 172.21.15.103:0/3589307246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095b83c0 msgr2=0x5635095c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66ae88d640 1 --2- 172.21.15.103:0/3589307246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095b83c0 0x5635095c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66ae88d640 1 -- 172.21.15.103:0/3589307246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095b8ce0 msgr2=0x5635095c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66ae88d640 1 --2- 172.21.15.103:0/3589307246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095b8ce0 0x5635095c4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66ae88d640 1 -- 172.21.15.103:0/3589307246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350928f3c0 con 0x5635095c7a90 2024-09-19T07:05:23.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66ad88b640 1 --2- 172.21.15.103:0/3589307246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095b8ce0 0x5635095c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:23.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f66ae88d640 1 --2- 172.21.15.103:0/3589307246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095c7a90 0x5635095c4b70 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f66a000f960 tx=0x7f66a0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c740f483d5ef3b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:23.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f6691ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:23.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.651+0000 7f6691ffb640 1 -- 172.21.15.103:0/3589307246 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a001ece0 con 0x5635095c7a90 2024-09-19T07:05:23.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.652+0000 7f66b26c9740 1 -- 172.21.15.103:0/3589307246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669c005020 con 0x5635095c7a90 2024-09-19T07:05:23.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.652+0000 7f6691ffb640 1 -- 172.21.15.103:0/3589307246 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f66a001b410 con 0x5635095c7a90 2024-09-19T07:05:23.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.652+0000 7f6691ffb640 1 -- 172.21.15.103:0/3589307246 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66a001b610 con 0x5635095c7a90 2024-09-19T07:05:23.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.652+0000 7f66b26c9740 10 client.21243 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:23.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.652+0000 7f66b26c9740 1 -- 172.21.15.103:0/3589307246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635095d5fb0 con 0x5635095c7a90 2024-09-19T07:05:23.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.652+0000 7f6691ffb640 1 -- 172.21.15.103:0/3589307246 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669c005020 con 0x5635095c7a90 2024-09-19T07:05:23.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.652+0000 7f6691ffb640 1 -- 172.21.15.103:0/3589307246 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f66a0025070 con 0x5635095c7a90 2024-09-19T07:05:23.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.652+0000 7f66b26c9740 10 client.21243 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:23.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.652+0000 7f66b26c9740 10 client.21243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:23.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.652+0000 7f66b26c9740 20 client.21243 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:23.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.652+0000 7f66b26c9740 1 --2- 172.21.15.103:0/3589307246 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5635095d7470 0x5635095d9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.652+0000 7f66b26c9740 4 client.21243 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:23.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.652+0000 7f66b26c9740 1 -- 172.21.15.103:0/3589307246 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56350952b850 con 0x5635095d7470 2024-09-19T07:05:23.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.653+0000 7f66ae08c640 1 --2- 172.21.15.103:0/3589307246 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5635095d7470 0x5635095d9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.653+0000 7f6691ffb640 10 client.21243 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:23.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.653+0000 7f66ae08c640 1 --2- 172.21.15.103:0/3589307246 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5635095d7470 0x5635095d9910 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76594930e644fcce server_cookie=b7378bc5e297825c in_seq=0 out_seq=0 2024-09-19T07:05:23.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.654+0000 7f6691ffb640 1 -- 172.21.15.103:0/3589307246 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56350952b850 con 0x5635095d7470 2024-09-19T07:05:23.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.654+0000 7f6691ffb640 10 client.21243 handle_command_reply: tid=0 2024-09-19T07:05:23.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.654+0000 7f66b26c9740 1 client.21243 shutdown 2024-09-19T07:05:23.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.654+0000 7f66b26c9740 1 -- 172.21.15.103:0/3589307246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095c7a90 msgr2=0x5635095c4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:23.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.654+0000 7f66b26c9740 1 --2- 172.21.15.103:0/3589307246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095c7a90 0x5635095c4b70 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f66a000f960 tx=0x7f66a0004ab0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.654+0000 7f66b26c9740 1 -- 172.21.15.103:0/3589307246 shutdown_connections 2024-09-19T07:05:23.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.654+0000 7f66b26c9740 1 --2- 172.21.15.103:0/3589307246 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5635095d7470 0x5635095d9910 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.654+0000 7f6691ffb640 0 client.21243 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:23.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.654+0000 7f66b26c9740 1 --2- 172.21.15.103:0/3589307246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095c7a90 0x5635095c4b70 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.654+0000 7f66b26c9740 1 --2- 172.21.15.103:0/3589307246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095b8ce0 0x5635095c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.654+0000 7f66b26c9740 1 --2- 172.21.15.103:0/3589307246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095b83c0 0x5635095c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.654+0000 7f66b26c9740 1 -- 172.21.15.103:0/3589307246 >> 172.21.15.103:0/3589307246 conn(0x5635095b4a30 msgr2=0x5635095ba6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:23.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.654+0000 7f66b26c9740 1 -- 172.21.15.103:0/3589307246 shutdown_connections 2024-09-19T07:05:23.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.654+0000 7f66b26c9740 1 -- 172.21.15.103:0/3589307246 wait complete. 2024-09-19T07:05:23.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.654+0000 7f66b26c9740 20 client.21243 trim_cache size 0 max 16384 2024-09-19T07:05:23.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.671+0000 7f66b26c9740 1 Processor -- start 2024-09-19T07:05:23.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.671+0000 7f66b26c9740 1 -- start start 2024-09-19T07:05:23.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66b26c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b83c0 0x5635095d3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66b26c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095b8ce0 0x5635095df230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66b26c9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095c7a90 0x5635095df770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66b26c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81ab810 con 0x5635095b83c0 2024-09-19T07:05:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66b26c9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81ad350 con 0x5635095c7a90 2024-09-19T07:05:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66b26c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81abee0 con 0x5635095b8ce0 2024-09-19T07:05:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66ae08c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b83c0 0x5635095d3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66ae08c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b83c0 0x5635095d3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60402/0 (socket says 172.21.15.103:60402) 2024-09-19T07:05:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66ae08c640 1 -- 172.21.15.103:0/4254406130 learned_addr learned my addr 172.21.15.103:0/4254406130 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66ad88b640 1 --2- 172.21.15.103:0/4254406130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095b8ce0 0x5635095df230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66ae88d640 1 --2- 172.21.15.103:0/4254406130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095c7a90 0x5635095df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66ae08c640 1 -- 172.21.15.103:0/4254406130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095b8ce0 msgr2=0x5635095df230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66ae08c640 1 --2- 172.21.15.103:0/4254406130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095b8ce0 0x5635095df230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66ae08c640 1 -- 172.21.15.103:0/4254406130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095c7a90 msgr2=0x5635095df770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66ae08c640 1 --2- 172.21.15.103:0/4254406130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095c7a90 0x5635095df770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66ae08c640 1 -- 172.21.15.103:0/4254406130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635095c76f0 con 0x5635095b83c0 2024-09-19T07:05:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.672+0000 7f66ae08c640 1 --2- 172.21.15.103:0/4254406130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b83c0 0x5635095d3840 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f669c009900 tx=0x7f669c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b135e9b2502ea810 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.673+0000 7f6693fff640 1 -- 172.21.15.103:0/4254406130 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c054020 con 0x5635095b83c0 2024-09-19T07:05:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.673+0000 7f6693fff640 1 -- 172.21.15.103:0/4254406130 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669c04fb50 con 0x5635095b83c0 2024-09-19T07:05:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.673+0000 7f6693fff640 1 -- 172.21.15.103:0/4254406130 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c040070 con 0x5635095b83c0 2024-09-19T07:05:23.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.673+0000 7f66b26c9740 1 -- 172.21.15.103:0/4254406130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b83c0 msgr2=0x5635095d3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:23.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.673+0000 7f66b26c9740 1 --2- 172.21.15.103:0/4254406130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b83c0 0x5635095d3840 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f669c009900 tx=0x7f669c031630 comp rx=0 tx=0).stop 2024-09-19T07:05:23.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.673+0000 7f66b26c9740 1 -- 172.21.15.103:0/4254406130 shutdown_connections 2024-09-19T07:05:23.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.673+0000 7f66b26c9740 1 --2- 172.21.15.103:0/4254406130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095c7a90 0x5635095df770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.673+0000 7f66b26c9740 1 --2- 172.21.15.103:0/4254406130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095b8ce0 0x5635095df230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.673+0000 7f66b26c9740 1 --2- 172.21.15.103:0/4254406130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b83c0 0x5635095d3840 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.673+0000 7f66b26c9740 1 -- 172.21.15.103:0/4254406130 >> 172.21.15.103:0/4254406130 conn(0x5635095b4a30 msgr2=0x5635095bfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:23.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.673+0000 7f66b26c9740 1 -- 172.21.15.103:0/4254406130 shutdown_connections 2024-09-19T07:05:23.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.673+0000 7f66b26c9740 1 -- 172.21.15.103:0/4254406130 wait complete. 2024-09-19T07:05:23.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.674+0000 7f66b26c9740 1 -- start start 2024-09-19T07:05:23.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.674+0000 7f66b26c9740 1 Processor -- start 2024-09-19T07:05:23.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.674+0000 7f66b26c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b83c0 0x5635095cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.674+0000 7f66b26c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095b8ce0 0x5635095cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.674+0000 7f66b26c9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095c7a90 0x5635095cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.674+0000 7f66b26c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81acc80 con 0x5635095b83c0 2024-09-19T07:05:23.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.674+0000 7f66b26c9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81ac5b0 con 0x5635095c7a90 2024-09-19T07:05:23.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.674+0000 7f66b26c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81a9810 con 0x5635095b8ce0 2024-09-19T07:05:23.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.674+0000 7f66b26c9740 10 client.0 _ll_register_callbacks cb 0x7f66a000f510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:23.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.674+0000 7f66ae08c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b83c0 0x5635095cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.674+0000 7f66ae08c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b83c0 0x5635095cf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60404/0 (socket says 172.21.15.103:60404) 2024-09-19T07:05:23.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.674+0000 7f66ae08c640 1 -- 172.21.15.103:0/1207598440 learned_addr learned my addr 172.21.15.103:0/1207598440 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:23.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.674+0000 7f66ae88d640 1 --2- 172.21.15.103:0/1207598440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095c7a90 0x5635095cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.675+0000 7f66ad88b640 1 --2- 172.21.15.103:0/1207598440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095b8ce0 0x5635095cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.675+0000 7f66ae08c640 1 -- 172.21.15.103:0/1207598440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095b8ce0 msgr2=0x5635095cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:23.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.675+0000 7f66ae08c640 1 --2- 172.21.15.103:0/1207598440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095b8ce0 0x5635095cfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.675+0000 7f66ae08c640 1 -- 172.21.15.103:0/1207598440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095c7a90 msgr2=0x5635095cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:23.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.675+0000 7f66ae08c640 1 --2- 172.21.15.103:0/1207598440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095c7a90 0x5635095cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.675+0000 7f66ae08c640 1 -- 172.21.15.103:0/1207598440 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635095ccec0 con 0x5635095b83c0 2024-09-19T07:05:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.675+0000 7f66ae88d640 1 --2- 172.21.15.103:0/1207598440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095c7a90 0x5635095cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.675+0000 7f66ae08c640 1 --2- 172.21.15.103:0/1207598440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b83c0 0x5635095cf9b0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f669c009340 tx=0x7f669c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b238481a512d2b29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.675+0000 7f6691ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.675+0000 7f6691ffb640 1 -- 172.21.15.103:0/1207598440 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c05d060 con 0x5635095b83c0 2024-09-19T07:05:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.675+0000 7f6691ffb640 1 -- 172.21.15.103:0/1207598440 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f669c004030 con 0x5635095b83c0 2024-09-19T07:05:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.675+0000 7f66b26c9740 1 -- 172.21.15.103:0/1207598440 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669c04fb50 con 0x5635095b83c0 2024-09-19T07:05:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.675+0000 7f6691ffb640 1 -- 172.21.15.103:0/1207598440 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f669c040070 con 0x5635095b83c0 2024-09-19T07:05:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.675+0000 7f66b26c9740 10 client.21249 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.675+0000 7f66b26c9740 1 -- 172.21.15.103:0/1207598440 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563509642960 con 0x5635095b83c0 2024-09-19T07:05:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.675+0000 7f6691ffb640 1 -- 172.21.15.103:0/1207598440 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669c04fb50 con 0x5635095b83c0 2024-09-19T07:05:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.676+0000 7f6691ffb640 1 -- 172.21.15.103:0/1207598440 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f669c03ca50 con 0x5635095b83c0 2024-09-19T07:05:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.676+0000 7f66b26c9740 10 client.21249 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.676+0000 7f66b26c9740 10 client.21249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.676+0000 7f66b26c9740 20 client.21249 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.676+0000 7f66b26c9740 1 --2- 172.21.15.103:0/1207598440 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563509642b90 0x563509644f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.676+0000 7f66b26c9740 4 client.21249 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:05:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.676+0000 7f66b26c9740 1 -- 172.21.15.103:0/1207598440 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563509525d60 con 0x563509642b90 2024-09-19T07:05:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.676+0000 7f66ad88b640 1 --2- 172.21.15.103:0/1207598440 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563509642b90 0x563509644f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.676+0000 7f66ad88b640 1 --2- 172.21.15.103:0/1207598440 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563509642b90 0x563509644f80 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c63d190d117e4097 server_cookie=97d7c7e0cfaa5156 in_seq=0 out_seq=0 2024-09-19T07:05:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.676+0000 7f6691ffb640 10 client.21249 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f6691ffb640 1 -- 172.21.15.103:0/1207598440 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563509525d60 con 0x563509642b90 2024-09-19T07:05:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f6691ffb640 10 client.21249 handle_command_reply: tid=0 2024-09-19T07:05:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f66b26c9740 1 client.21249 shutdown 2024-09-19T07:05:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f66b26c9740 1 -- 172.21.15.103:0/1207598440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b83c0 msgr2=0x5635095cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f66b26c9740 1 --2- 172.21.15.103:0/1207598440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b83c0 0x5635095cf9b0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f669c009340 tx=0x7f669c005850 comp rx=0 tx=0).stop 2024-09-19T07:05:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f66b26c9740 1 -- 172.21.15.103:0/1207598440 shutdown_connections 2024-09-19T07:05:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f66b26c9740 1 --2- 172.21.15.103:0/1207598440 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563509642b90 0x563509644f80 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f66b26c9740 1 --2- 172.21.15.103:0/1207598440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635095c7a90 0x5635095cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f66b26c9740 1 --2- 172.21.15.103:0/1207598440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635095b8ce0 0x5635095cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f66b26c9740 1 --2- 172.21.15.103:0/1207598440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635095b83c0 0x5635095cf9b0 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f6691ffb640 0 client.21249 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f66b26c9740 1 -- 172.21.15.103:0/1207598440 >> 172.21.15.103:0/1207598440 conn(0x5635095b4a30 msgr2=0x5635095c5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f66b26c9740 1 -- 172.21.15.103:0/1207598440 shutdown_connections 2024-09-19T07:05:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f66b26c9740 1 -- 172.21.15.103:0/1207598440 wait complete. 2024-09-19T07:05:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f66b26c9740 20 client.21249 trim_cache size 0 max 16384 2024-09-19T07:05:23.683 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:05:23.683 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:05:23.683 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:05:23.684 INFO:teuthology.orchestra.run.smithi103.stdout: "70c3565e-7f0a-493e-b524-142863861290": { 2024-09-19T07:05:23.684 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:05:23.684 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "70c3565e-7f0a-493e-b524-142863861290", 2024-09-19T07:05:23.684 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:05:23.684 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:05:23.684 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:05:23.684 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:05:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f66b02f0640 1 -- 172.21.15.103:0/1437774334 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66940a3430 msgr2=0x7f66940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f66b02f0640 1 --2- 172.21.15.103:0/1437774334 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66940a3430 0x7f66940a58f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f66a0005fd0 tx=0x7f66a0004a40 comp rx=0 tx=0).stop 2024-09-19T07:05:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f66b02f0640 1 -- 172.21.15.103:0/1437774334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66a8103740 msgr2=0x7f66a8198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.677+0000 7f66b02f0640 1 --2- 172.21.15.103:0/1437774334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66a8103740 0x7f66a8198c10 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f66a400b450 tx=0x7f66a400b910 comp rx=0 tx=0).stop 2024-09-19T07:05:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.678+0000 7f66b02f0640 1 -- 172.21.15.103:0/1437774334 shutdown_connections 2024-09-19T07:05:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.678+0000 7f66b02f0640 1 --2- 172.21.15.103:0/1437774334 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f66940a3430 0x7f66940a58f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.678+0000 7f66b02f0640 1 --2- 172.21.15.103:0/1437774334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a8108290 0x7f66a819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.678+0000 7f66b02f0640 1 --2- 172.21.15.103:0/1437774334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66a8103740 0x7f66a8198c10 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.678+0000 7f66b02f0640 1 --2- 172.21.15.103:0/1437774334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66a8102d90 0x7f66a81986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.678+0000 7f66b02f0640 1 -- 172.21.15.103:0/1437774334 >> 172.21.15.103:0/1437774334 conn(0x7f66a80fc7d0 msgr2=0x7f66a8105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.678+0000 7f66b02f0640 1 -- 172.21.15.103:0/1437774334 shutdown_connections 2024-09-19T07:05:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:23.678+0000 7f66b02f0640 1 -- 172.21.15.103:0/1437774334 wait complete. 2024-09-19T07:05:23.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:23 smithi134 ceph-mon[35356]: pgmap v505: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-19T07:05:23.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:23 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3153813134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:05:23.734 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:23 smithi174 ceph-mon[35637]: pgmap v505: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-19T07:05:23.734 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:23 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3153813134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:05:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.339+0000 7f8fb2af1640 1 -- 172.21.15.103:0/3755110795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fac102d20 msgr2=0x7f8fac103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.339+0000 7f8fb2af1640 1 --2- 172.21.15.103:0/3755110795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fac102d20 0x7f8fac103100 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009900 tx=0x7f8fa002a480 comp rx=0 tx=0).stop 2024-09-19T07:05:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.339+0000 7f8fb2af1640 1 -- 172.21.15.103:0/3755110795 shutdown_connections 2024-09-19T07:05:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.339+0000 7f8fb2af1640 1 --2- 172.21.15.103:0/3755110795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fac10c0f0 0x7f8fac10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.339+0000 7f8fb2af1640 1 --2- 172.21.15.103:0/3755110795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fac103640 0x7f8fac10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.339+0000 7f8fb2af1640 1 --2- 172.21.15.103:0/3755110795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fac102d20 0x7f8fac103100 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.339+0000 7f8fb2af1640 1 -- 172.21.15.103:0/3755110795 >> 172.21.15.103:0/3755110795 conn(0x7f8fac0fc6b0 msgr2=0x7f8fac0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.339+0000 7f8fb2af1640 1 -- 172.21.15.103:0/3755110795 shutdown_connections 2024-09-19T07:05:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.339+0000 7f8fb2af1640 1 -- 172.21.15.103:0/3755110795 wait complete. 2024-09-19T07:05:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.339+0000 7f8fb2af1640 1 Processor -- start 2024-09-19T07:05:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.339+0000 7f8fb2af1640 1 -- start start 2024-09-19T07:05:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fb2af1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fac102d20 0x7f8fac198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fb2af1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fac103640 0x7f8fac198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fb2af1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fac10c0f0 0x7f8fac19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fb2af1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac111ef0 con 0x7f8fac103640 2024-09-19T07:05:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fb2af1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac111d70 con 0x7f8fac10c0f0 2024-09-19T07:05:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fb2af1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac112070 con 0x7f8fac102d20 2024-09-19T07:05:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fabfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fac103640 0x7f8fac198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fabfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fac103640 0x7f8fac198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60424/0 (socket says 172.21.15.103:60424) 2024-09-19T07:05:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fabfff640 1 -- 172.21.15.103:0/143362802 learned_addr learned my addr 172.21.15.103:0/143362802 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fb088d640 1 --2- 172.21.15.103:0/143362802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fac102d20 0x7f8fac198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fb108e640 1 --2- 172.21.15.103:0/143362802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fac10c0f0 0x7f8fac19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fabfff640 1 -- 172.21.15.103:0/143362802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fac102d20 msgr2=0x7f8fac198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fabfff640 1 --2- 172.21.15.103:0/143362802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fac102d20 0x7f8fac198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fabfff640 1 -- 172.21.15.103:0/143362802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fac10c0f0 msgr2=0x7f8fac19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fabfff640 1 --2- 172.21.15.103:0/143362802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fac10c0f0 0x7f8fac19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fabfff640 1 -- 172.21.15.103:0/143362802 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fac19d330 con 0x7f8fac103640 2024-09-19T07:05:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fb108e640 1 --2- 172.21.15.103:0/143362802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fac10c0f0 0x7f8fac19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fabfff640 1 --2- 172.21.15.103:0/143362802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fac103640 0x7f8fac198880 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800c890 tx=0x7f8f9800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fa9ffb640 1 -- 172.21.15.103:0/143362802 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98004220 con 0x7f8fac103640 2024-09-19T07:05:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fa9ffb640 1 -- 172.21.15.103:0/143362802 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f980043c0 con 0x7f8fac103640 2024-09-19T07:05:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fb2af1640 1 -- 172.21.15.103:0/143362802 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fac19d620 con 0x7f8fac103640 2024-09-19T07:05:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.340+0000 7f8fa9ffb640 1 -- 172.21.15.103:0/143362802 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f980049e0 con 0x7f8fac103640 2024-09-19T07:05:24.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.341+0000 7f8fb2af1640 1 -- 172.21.15.103:0/143362802 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fac1a4eb0 con 0x7f8fac103640 2024-09-19T07:05:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.343+0000 7f8fa9ffb640 1 -- 172.21.15.103:0/143362802 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8f980079b0 con 0x7f8fac103640 2024-09-19T07:05:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.343+0000 7f8fb5890740 1 Processor -- start 2024-09-19T07:05:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.343+0000 7f8fb5890740 1 -- start start 2024-09-19T07:05:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.343+0000 7f8fb5890740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017fd6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.343+0000 7f8fa9ffb640 1 --2- 172.21.15.103:0/143362802 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8f9c0a3460 0x7f8f9c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.343+0000 7f8fb5890740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fd72f0 0x560017fe5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.343+0000 7f8fb5890740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fe60c0 0x560017fe84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.343+0000 7f8fb5890740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560017f3b2f0 con 0x560017fd69d0 2024-09-19T07:05:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.343+0000 7f8fb5890740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560017ee3270 con 0x560017fd72f0 2024-09-19T07:05:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.343+0000 7f8fb5890740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560017f372b0 con 0x560017fe60c0 2024-09-19T07:05:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.343+0000 7f8fb108e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017fd6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.343+0000 7f8fb088d640 1 --2- 172.21.15.103:0/143362802 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8f9c0a3460 0x7f8f9c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.343+0000 7f8fb108e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017fd6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60434/0 (socket says 172.21.15.103:60434) 2024-09-19T07:05:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.343+0000 7f8fb108e640 1 -- 172.21.15.103:0/2090975700 learned_addr learned my addr 172.21.15.103:0/2090975700 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.344+0000 7f8fabfff640 1 --2- 172.21.15.103:0/2090975700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fd72f0 0x560017fe5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.344+0000 7f8fa9ffb640 1 -- 172.21.15.103:0/143362802 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f8f980cfda0 con 0x7f8fac103640 2024-09-19T07:05:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.344+0000 7f8fb108e640 1 --2- 172.21.15.103:0/2090975700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fe60c0 0x560017fe84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.344+0000 7f8fb108e640 1 -- 172.21.15.103:0/2090975700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fe60c0 msgr2=0x560017fe84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.344+0000 7f8fb108e640 1 --2- 172.21.15.103:0/2090975700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fe60c0 0x560017fe84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.344+0000 7f8fb108e640 1 -- 172.21.15.103:0/2090975700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fd72f0 msgr2=0x560017fe5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.344+0000 7f8fb108e640 1 --2- 172.21.15.103:0/2090975700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fd72f0 0x560017fe5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.344+0000 7f8fb108e640 1 -- 172.21.15.103:0/2090975700 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560017ee2340 con 0x560017fd69d0 2024-09-19T07:05:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.344+0000 7f8fb088d640 1 --2- 172.21.15.103:0/143362802 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8f9c0a3460 0x7f8f9c0a5920 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009230 tx=0x7f8fa00058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.344+0000 7f8fb108e640 1 --2- 172.21.15.103:0/2090975700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017fd6db0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40145e0 tx=0x7f8fa4009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c9eb156dabb0a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.344+0000 7f8f96ffd640 1 -- 172.21.15.103:0/2090975700 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4012750 con 0x560017fd69d0 2024-09-19T07:05:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.344+0000 7f8f96ffd640 1 -- 172.21.15.103:0/2090975700 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa4018070 con 0x560017fd69d0 2024-09-19T07:05:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.344+0000 7f8f96ffd640 1 -- 172.21.15.103:0/2090975700 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa401dd90 con 0x560017fd69d0 2024-09-19T07:05:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.345+0000 7f8fb5890740 1 -- 172.21.15.103:0/2090975700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 msgr2=0x560017fd6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.345+0000 7f8fb5890740 1 --2- 172.21.15.103:0/2090975700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017fd6db0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40145e0 tx=0x7f8fa4009910 comp rx=0 tx=0).stop 2024-09-19T07:05:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.345+0000 7f8fb5890740 1 -- 172.21.15.103:0/2090975700 shutdown_connections 2024-09-19T07:05:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.345+0000 7f8fb5890740 1 --2- 172.21.15.103:0/2090975700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fe60c0 0x560017fe84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.345+0000 7f8fb5890740 1 --2- 172.21.15.103:0/2090975700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fd72f0 0x560017fe5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.345+0000 7f8fb5890740 1 --2- 172.21.15.103:0/2090975700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017fd6db0 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.345+0000 7f8fb5890740 1 -- 172.21.15.103:0/2090975700 >> 172.21.15.103:0/2090975700 conn(0x560017fd3060 msgr2=0x560017fd3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.345+0000 7f8fb5890740 1 -- 172.21.15.103:0/2090975700 shutdown_connections 2024-09-19T07:05:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.345+0000 7f8fb5890740 1 -- 172.21.15.103:0/2090975700 wait complete. 2024-09-19T07:05:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.345+0000 7f8fb5890740 1 -- start start 2024-09-19T07:05:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.345+0000 7f8fb5890740 1 Processor -- start 2024-09-19T07:05:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb5890740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017fe3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb5890740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fd72f0 0x560017fe3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb5890740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fe60c0 0x560017fe3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb5890740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560017fe8cf0 con 0x560017fd69d0 2024-09-19T07:05:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb5890740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560017ec4290 con 0x560017fd72f0 2024-09-19T07:05:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb5890740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560017fe8e70 con 0x560017fe60c0 2024-09-19T07:05:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb5890740 10 client.0 _ll_register_callbacks cb 0x560017ef5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb088d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017fe3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb088d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017fe3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60440/0 (socket says 172.21.15.103:60440) 2024-09-19T07:05:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb088d640 1 -- 172.21.15.103:0/357352679 learned_addr learned my addr 172.21.15.103:0/357352679 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fabfff640 1 --2- 172.21.15.103:0/357352679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fd72f0 0x560017fe3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb108e640 1 --2- 172.21.15.103:0/357352679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fe60c0 0x560017fe3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb088d640 1 -- 172.21.15.103:0/357352679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fe60c0 msgr2=0x560017fe3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb088d640 1 --2- 172.21.15.103:0/357352679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fe60c0 0x560017fe3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb088d640 1 -- 172.21.15.103:0/357352679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fd72f0 msgr2=0x560017fe3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb088d640 1 --2- 172.21.15.103:0/357352679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fd72f0 0x560017fe3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb088d640 1 -- 172.21.15.103:0/357352679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560017c4e210 con 0x560017fd69d0 2024-09-19T07:05:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb108e640 1 --2- 172.21.15.103:0/357352679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fe60c0 0x560017fe3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8fb088d640 1 --2- 172.21.15.103:0/357352679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017fe3050 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009900 tx=0x7f8fa0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edbb24eb7c21445 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.346+0000 7f8f8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.347+0000 7f8f8bfff640 1 -- 172.21.15.103:0/357352679 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa004c070 con 0x560017fd69d0 2024-09-19T07:05:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.347+0000 7f8f8bfff640 1 -- 172.21.15.103:0/357352679 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f8fa00035f0 con 0x560017fd69d0 2024-09-19T07:05:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.347+0000 7f8fb5890740 1 -- 172.21.15.103:0/357352679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fa4018070 con 0x560017fd69d0 2024-09-19T07:05:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.347+0000 7f8f8bfff640 1 -- 172.21.15.103:0/357352679 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8fa0058050 con 0x560017fd69d0 2024-09-19T07:05:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.347+0000 7f8fb5890740 10 client.21261 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.347+0000 7f8fb5890740 1 -- 172.21.15.103:0/357352679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560017d1a620 con 0x560017fd69d0 2024-09-19T07:05:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.347+0000 7f8f8bfff640 1 -- 172.21.15.103:0/357352679 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa4018070 con 0x560017fd69d0 2024-09-19T07:05:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.347+0000 7f8f8bfff640 1 -- 172.21.15.103:0/357352679 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8fa003c860 con 0x560017fd69d0 2024-09-19T07:05:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.347+0000 7f8fb5890740 10 client.21261 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.347+0000 7f8fb5890740 10 client.21261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.347+0000 7f8fb5890740 20 client.21261 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.347+0000 7f8fb5890740 1 --2- 172.21.15.103:0/357352679 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560017ff64e0 0x560017ff8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.347+0000 7f8fb5890740 4 client.21261 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.347+0000 7f8fb5890740 1 -- 172.21.15.103:0/357352679 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560017f48a20 con 0x560017ff64e0 2024-09-19T07:05:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.347+0000 7f8fabfff640 1 --2- 172.21.15.103:0/357352679 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560017ff64e0 0x560017ff8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.348+0000 7f8fabfff640 1 --2- 172.21.15.103:0/357352679 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560017ff64e0 0x560017ff8980 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=800d8343ee164e4 server_cookie=374930266ecc5aa3 in_seq=0 out_seq=0 2024-09-19T07:05:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.348+0000 7f8f8bfff640 10 client.21261 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.348+0000 7f8f8bfff640 1 -- 172.21.15.103:0/357352679 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560017f48a20 con 0x560017ff64e0 2024-09-19T07:05:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.348+0000 7f8f8bfff640 10 client.21261 handle_command_reply: tid=0 2024-09-19T07:05:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.348+0000 7f8fb5890740 1 client.21261 shutdown 2024-09-19T07:05:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.348+0000 7f8fb5890740 1 -- 172.21.15.103:0/357352679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 msgr2=0x560017fe3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.348+0000 7f8fb5890740 1 --2- 172.21.15.103:0/357352679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017fe3050 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009900 tx=0x7f8fa0009260 comp rx=0 tx=0).stop 2024-09-19T07:05:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.349+0000 7f8fb5890740 1 -- 172.21.15.103:0/357352679 shutdown_connections 2024-09-19T07:05:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.349+0000 7f8fb5890740 1 --2- 172.21.15.103:0/357352679 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560017ff64e0 0x560017ff8980 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.349+0000 7f8fb5890740 1 --2- 172.21.15.103:0/357352679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fe60c0 0x560017fe3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.349+0000 7f8fb5890740 1 --2- 172.21.15.103:0/357352679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fd72f0 0x560017fe3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.349+0000 7f8f8bfff640 0 client.21261 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.349+0000 7f8fb5890740 1 --2- 172.21.15.103:0/357352679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017fe3050 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.349+0000 7f8fb5890740 1 -- 172.21.15.103:0/357352679 >> 172.21.15.103:0/357352679 conn(0x560017fd3060 msgr2=0x560017fd8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.349+0000 7f8fb5890740 1 -- 172.21.15.103:0/357352679 shutdown_connections 2024-09-19T07:05:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.349+0000 7f8fb5890740 1 -- 172.21.15.103:0/357352679 wait complete. 2024-09-19T07:05:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.349+0000 7f8fb5890740 20 client.21261 trim_cache size 0 max 16384 2024-09-19T07:05:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb5890740 1 Processor -- start 2024-09-19T07:05:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb5890740 1 -- start start 2024-09-19T07:05:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb5890740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017ffe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb5890740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fd72f0 0x560017ffe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb5890740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fe60c0 0x560017fe02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb5890740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac1ab6c0 con 0x560017fd69d0 2024-09-19T07:05:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb5890740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac1ad200 con 0x560017fe60c0 2024-09-19T07:05:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb5890740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac1abd90 con 0x560017fd72f0 2024-09-19T07:05:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb088d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017ffe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb088d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017ffe290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60444/0 (socket says 172.21.15.103:60444) 2024-09-19T07:05:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb088d640 1 -- 172.21.15.103:0/807591883 learned_addr learned my addr 172.21.15.103:0/807591883 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fabfff640 1 --2- 172.21.15.103:0/807591883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fd72f0 0x560017ffe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb108e640 1 --2- 172.21.15.103:0/807591883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fe60c0 0x560017fe02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb088d640 1 -- 172.21.15.103:0/807591883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fd72f0 msgr2=0x560017ffe7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb088d640 1 --2- 172.21.15.103:0/807591883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fd72f0 0x560017ffe7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb088d640 1 -- 172.21.15.103:0/807591883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fe60c0 msgr2=0x560017fe02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb088d640 1 --2- 172.21.15.103:0/807591883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fe60c0 0x560017fe02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb088d640 1 -- 172.21.15.103:0/807591883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560017fe1fe0 con 0x560017fd69d0 2024-09-19T07:05:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fabfff640 1 --2- 172.21.15.103:0/807591883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fd72f0 0x560017ffe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.364+0000 7f8fb088d640 1 --2- 172.21.15.103:0/807591883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017ffe290 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f8fa003cc30 tx=0x7f8fa0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b0d9884e717cf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.365+0000 7f8f967fc640 1 -- 172.21.15.103:0/807591883 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0060020 con 0x560017fd69d0 2024-09-19T07:05:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.365+0000 7f8f967fc640 1 -- 172.21.15.103:0/807591883 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa003c920 con 0x560017fd69d0 2024-09-19T07:05:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.365+0000 7f8f967fc640 1 -- 172.21.15.103:0/807591883 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa004c040 con 0x560017fd69d0 2024-09-19T07:05:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.365+0000 7f8fb5890740 1 -- 172.21.15.103:0/807591883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 msgr2=0x560017ffe290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.365+0000 7f8fb5890740 1 --2- 172.21.15.103:0/807591883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017ffe290 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f8fa003cc30 tx=0x7f8fa0005a20 comp rx=0 tx=0).stop 2024-09-19T07:05:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.365+0000 7f8fb5890740 1 -- 172.21.15.103:0/807591883 shutdown_connections 2024-09-19T07:05:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.365+0000 7f8fb5890740 1 --2- 172.21.15.103:0/807591883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fe60c0 0x560017fe02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.366+0000 7f8fb5890740 1 --2- 172.21.15.103:0/807591883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fd72f0 0x560017ffe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.366+0000 7f8fb5890740 1 --2- 172.21.15.103:0/807591883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fd69d0 0x560017ffe290 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.366+0000 7f8fb5890740 1 -- 172.21.15.103:0/807591883 >> 172.21.15.103:0/807591883 conn(0x560017fd3060 msgr2=0x560017fe50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.366+0000 7f8fb5890740 1 -- 172.21.15.103:0/807591883 shutdown_connections 2024-09-19T07:05:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.366+0000 7f8fb5890740 1 -- 172.21.15.103:0/807591883 wait complete. 2024-09-19T07:05:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.366+0000 7f8fb5890740 1 -- start start 2024-09-19T07:05:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.366+0000 7f8fb5890740 1 Processor -- start 2024-09-19T07:05:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.366+0000 7f8fb5890740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fd69d0 0x560017fee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.366+0000 7f8fb5890740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fd72f0 0x560017fee600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.366+0000 7f8fb5890740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fe60c0 0x560017feb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.366+0000 7f8fb5890740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac1acb30 con 0x560017fe60c0 2024-09-19T07:05:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.366+0000 7f8fb5890740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac1ac460 con 0x560017fd72f0 2024-09-19T07:05:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.366+0000 7f8fb5890740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac1a96c0 con 0x560017fd69d0 2024-09-19T07:05:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.366+0000 7f8fb5890740 10 client.0 _ll_register_callbacks cb 0x7f8fa0040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8fb108e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fe60c0 0x560017feb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8fb108e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fe60c0 0x560017feb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60446/0 (socket says 172.21.15.103:60446) 2024-09-19T07:05:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8fb108e640 1 -- 172.21.15.103:0/2530341264 learned_addr learned my addr 172.21.15.103:0/2530341264 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8fabfff640 1 --2- 172.21.15.103:0/2530341264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fd72f0 0x560017fee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8fb088d640 1 --2- 172.21.15.103:0/2530341264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fd69d0 0x560017fee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8fb108e640 1 -- 172.21.15.103:0/2530341264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fd69d0 msgr2=0x560017fee0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8fb108e640 1 --2- 172.21.15.103:0/2530341264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fd69d0 0x560017fee0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8fb108e640 1 -- 172.21.15.103:0/2530341264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fd72f0 msgr2=0x560017fee600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8fb108e640 1 --2- 172.21.15.103:0/2530341264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fd72f0 0x560017fee600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8fb108e640 1 -- 172.21.15.103:0/2530341264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560017feeca0 con 0x560017fe60c0 2024-09-19T07:05:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8fb108e640 1 --2- 172.21.15.103:0/2530341264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fe60c0 0x560017feb090 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009810 tx=0x7f8fa4009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa495ce0c3d6d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8f8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8f8bfff640 1 -- 172.21.15.103:0/2530341264 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa402c060 con 0x560017fe60c0 2024-09-19T07:05:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8f8bfff640 1 -- 172.21.15.103:0/2530341264 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f8fa4012550 con 0x560017fe60c0 2024-09-19T07:05:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8fb5890740 1 -- 172.21.15.103:0/2530341264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fa003c920 con 0x560017fe60c0 2024-09-19T07:05:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8f8bfff640 1 -- 172.21.15.103:0/2530341264 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8fa4012790 con 0x560017fe60c0 2024-09-19T07:05:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8fb5890740 10 client.21267 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.367+0000 7f8fb5890740 1 -- 172.21.15.103:0/2530341264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560018061820 con 0x560017fe60c0 2024-09-19T07:05:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.368+0000 7f8f8bfff640 1 -- 172.21.15.103:0/2530341264 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa003c920 con 0x560017fe60c0 2024-09-19T07:05:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.368+0000 7f8f8bfff640 1 -- 172.21.15.103:0/2530341264 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8fa400f040 con 0x560017fe60c0 2024-09-19T07:05:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.368+0000 7f8fb5890740 10 client.21267 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.368+0000 7f8fb5890740 10 client.21267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.368+0000 7f8fb5890740 20 client.21267 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.368+0000 7f8fb5890740 1 --2- 172.21.15.103:0/2530341264 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560018061cf0 0x560018064110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.368+0000 7f8fb5890740 4 client.21267 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:05:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.368+0000 7f8fb5890740 1 -- 172.21.15.103:0/2530341264 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560017f42d70 con 0x560018061cf0 2024-09-19T07:05:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.368+0000 7f8fb088d640 1 --2- 172.21.15.103:0/2530341264 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560018061cf0 0x560018064110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.369+0000 7f8f8bfff640 10 client.21267 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.369+0000 7f8fb088d640 1 --2- 172.21.15.103:0/2530341264 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560018061cf0 0x560018064110 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1dbf1265637521 server_cookie=af6714def37ce692 in_seq=0 out_seq=0 2024-09-19T07:05:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.376+0000 7f8f8bfff640 1 -- 172.21.15.103:0/2530341264 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560017f42d70 con 0x560018061cf0 2024-09-19T07:05:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.376+0000 7f8f8bfff640 10 client.21267 handle_command_reply: tid=0 2024-09-19T07:05:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.376+0000 7f8fb5890740 1 client.21267 shutdown 2024-09-19T07:05:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.376+0000 7f8fb5890740 1 -- 172.21.15.103:0/2530341264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fe60c0 msgr2=0x560017feb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.376+0000 7f8fb5890740 1 --2- 172.21.15.103:0/2530341264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fe60c0 0x560017feb090 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009810 tx=0x7f8fa4009a50 comp rx=0 tx=0).stop 2024-09-19T07:05:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.376+0000 7f8fb5890740 1 -- 172.21.15.103:0/2530341264 shutdown_connections 2024-09-19T07:05:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.376+0000 7f8fb5890740 1 --2- 172.21.15.103:0/2530341264 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x560018061cf0 0x560018064110 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.376+0000 7f8fb5890740 1 --2- 172.21.15.103:0/2530341264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560017fe60c0 0x560017feb090 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.376+0000 7f8fb5890740 1 --2- 172.21.15.103:0/2530341264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017fd72f0 0x560017fee600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.376+0000 7f8f8bfff640 0 client.21267 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.376+0000 7f8fb5890740 1 --2- 172.21.15.103:0/2530341264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560017fd69d0 0x560017fee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.376+0000 7f8fb5890740 1 -- 172.21.15.103:0/2530341264 >> 172.21.15.103:0/2530341264 conn(0x560017fd3060 msgr2=0x560017fd19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.376+0000 7f8fb5890740 1 -- 172.21.15.103:0/2530341264 shutdown_connections 2024-09-19T07:05:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.376+0000 7f8fb5890740 1 -- 172.21.15.103:0/2530341264 wait complete. 2024-09-19T07:05:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.376+0000 7f8fb5890740 20 client.21267 trim_cache size 0 max 16384 2024-09-19T07:05:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:05:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:05:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:05:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:05:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.377+0000 7f8fb2af1640 1 -- 172.21.15.103:0/143362802 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8f9c0a3460 msgr2=0x7f8f9c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.377+0000 7f8fb2af1640 1 --2- 172.21.15.103:0/143362802 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8f9c0a3460 0x7f8f9c0a5920 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009230 tx=0x7f8fa00058d0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.377+0000 7f8fb2af1640 1 -- 172.21.15.103:0/143362802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fac103640 msgr2=0x7f8fac198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.377+0000 7f8fb2af1640 1 --2- 172.21.15.103:0/143362802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fac103640 0x7f8fac198880 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800c890 tx=0x7f8f9800cd50 comp rx=0 tx=0).stop 2024-09-19T07:05:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.377+0000 7f8fb2af1640 1 -- 172.21.15.103:0/143362802 shutdown_connections 2024-09-19T07:05:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.377+0000 7f8fb2af1640 1 --2- 172.21.15.103:0/143362802 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8f9c0a3460 0x7f8f9c0a5920 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.377+0000 7f8fb2af1640 1 --2- 172.21.15.103:0/143362802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fac10c0f0 0x7f8fac19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.377+0000 7f8fb2af1640 1 --2- 172.21.15.103:0/143362802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fac103640 0x7f8fac198880 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.377+0000 7f8fb2af1640 1 --2- 172.21.15.103:0/143362802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fac102d20 0x7f8fac198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.377+0000 7f8fb2af1640 1 -- 172.21.15.103:0/143362802 >> 172.21.15.103:0/143362802 conn(0x7f8fac0fc6b0 msgr2=0x7f8fac0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.377+0000 7f8fb2af1640 1 -- 172.21.15.103:0/143362802 shutdown_connections 2024-09-19T07:05:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:24.377+0000 7f8fb2af1640 1 -- 172.21.15.103:0/143362802 wait complete. 2024-09-19T07:05:24.387 INFO:tasks.cephfs.filesystem:scrub status for tag:70c3565e-7f0a-493e-b524-142863861290 - {'path': '/', 'tag': '70c3565e-7f0a-493e-b524-142863861290', 'options': 'recursive,force'} 2024-09-19T07:05:24.395 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.821+0000 7f3c4ebb6640 1 -- 172.21.15.103:0/1264284353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c48102570 msgr2=0x7f3c481029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.821+0000 7f3c4ebb6640 1 --2- 172.21.15.103:0/1264284353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c48102570 0x7f3c481029b0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009900 tx=0x7f3c3402a480 comp rx=0 tx=0).stop 2024-09-19T07:05:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.821+0000 7f3c4ebb6640 1 -- 172.21.15.103:0/1264284353 shutdown_connections 2024-09-19T07:05:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.821+0000 7f3c4ebb6640 1 --2- 172.21.15.103:0/1264284353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c48102ef0 0x7f3c4810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.821+0000 7f3c4ebb6640 1 --2- 172.21.15.103:0/1264284353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c48102570 0x7f3c481029b0 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.821+0000 7f3c4ebb6640 1 --2- 172.21.15.103:0/1264284353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c48100e20 0x7f3c48101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.821+0000 7f3c4ebb6640 1 -- 172.21.15.103:0/1264284353 >> 172.21.15.103:0/1264284353 conn(0x7f3c480fc7d0 msgr2=0x7f3c480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.821+0000 7f3c4ebb6640 1 -- 172.21.15.103:0/1264284353 shutdown_connections 2024-09-19T07:05:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.821+0000 7f3c4ebb6640 1 -- 172.21.15.103:0/1264284353 wait complete. 2024-09-19T07:05:24.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c4ebb6640 1 Processor -- start 2024-09-19T07:05:24.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c4ebb6640 1 -- start start 2024-09-19T07:05:24.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c4ebb6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c48100e20 0x7f3c480fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c4ebb6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c48102570 0x7f3c480fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c4ebb6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c48102ef0 0x7f3c480fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c4ebb6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48110fa0 con 0x7f3c48102570 2024-09-19T07:05:24.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c4ebb6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48110e20 con 0x7f3c48102ef0 2024-09-19T07:05:24.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c4ebb6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48111120 con 0x7f3c48100e20 2024-09-19T07:05:24.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c48102570 0x7f3c480fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c48102570 0x7f3c480fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60464/0 (socket says 172.21.15.103:60464) 2024-09-19T07:05:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c47fff640 1 -- 172.21.15.103:0/1203931506 learned_addr learned my addr 172.21.15.103:0/1203931506 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c4d153640 1 --2- 172.21.15.103:0/1203931506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c48102ef0 0x7f3c480fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c4c952640 1 --2- 172.21.15.103:0/1203931506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c48100e20 0x7f3c480fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c47fff640 1 -- 172.21.15.103:0/1203931506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c48100e20 msgr2=0x7f3c480fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c47fff640 1 --2- 172.21.15.103:0/1203931506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c48100e20 0x7f3c480fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c47fff640 1 -- 172.21.15.103:0/1203931506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c48102ef0 msgr2=0x7f3c480fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c47fff640 1 --2- 172.21.15.103:0/1203931506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c48102ef0 0x7f3c480fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.822+0000 7f3c47fff640 1 -- 172.21.15.103:0/1203931506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c481a69c0 con 0x7f3c48102570 2024-09-19T07:05:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.823+0000 7f3c47fff640 1 --2- 172.21.15.103:0/1203931506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c48102570 0x7f3c480fd1c0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f3c34006010 tx=0x7f3c34003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.823+0000 7f3c45ffb640 1 -- 172.21.15.103:0/1203931506 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3403f070 con 0x7f3c48102570 2024-09-19T07:05:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.823+0000 7f3c45ffb640 1 -- 172.21.15.103:0/1203931506 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c34031e30 con 0x7f3c48102570 2024-09-19T07:05:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.823+0000 7f3c4ebb6640 1 -- 172.21.15.103:0/1203931506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c481a6c50 con 0x7f3c48102570 2024-09-19T07:05:24.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.823+0000 7f3c45ffb640 1 -- 172.21.15.103:0/1203931506 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34035030 con 0x7f3c48102570 2024-09-19T07:05:24.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.823+0000 7f3c4ebb6640 1 -- 172.21.15.103:0/1203931506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c481a7130 con 0x7f3c48102570 2024-09-19T07:05:24.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.825+0000 7f3c45ffb640 1 -- 172.21.15.103:0/1203931506 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3c34042070 con 0x7f3c48102570 2024-09-19T07:05:24.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c50f8f740 1 Processor -- start 2024-09-19T07:05:24.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c50f8f740 1 -- start start 2024-09-19T07:05:24.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c50f8f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996b9a060 0x559996b9a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c45ffb640 1 --2- 172.21.15.103:0/1203931506 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c380a33b0 0x7f3c380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c50f8f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a980 0x559996ba4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c50f8f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996ba5240 0x559996ba7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c50f8f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559996aff620 con 0x559996b9a980 2024-09-19T07:05:24.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c50f8f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559996aa6e50 con 0x559996ba5240 2024-09-19T07:05:24.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c50f8f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559996afb5e0 con 0x559996b9a060 2024-09-19T07:05:24.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c4c952640 1 --2- 172.21.15.103:0/1203931506 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c380a33b0 0x7f3c380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a980 0x559996ba4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c45ffb640 1 -- 172.21.15.103:0/1203931506 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f3c340f0690 con 0x7f3c48102570 2024-09-19T07:05:24.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a980 0x559996ba4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60472/0 (socket says 172.21.15.103:60472) 2024-09-19T07:05:24.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c47fff640 1 -- 172.21.15.103:0/11694084 learned_addr learned my addr 172.21.15.103:0/11694084 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:24.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c4d153640 1 --2- 172.21.15.103:0/11694084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996ba5240 0x559996ba7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c4d153640 1 --2- 172.21.15.103:0/11694084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996b9a060 0x559996b9a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c47fff640 1 -- 172.21.15.103:0/11694084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996b9a060 msgr2=0x559996b9a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:24.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c47fff640 1 --2- 172.21.15.103:0/11694084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996b9a060 0x559996b9a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c47fff640 1 -- 172.21.15.103:0/11694084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996ba5240 msgr2=0x559996ba7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:24.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c47fff640 1 --2- 172.21.15.103:0/11694084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996ba5240 0x559996ba7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.826+0000 7f3c47fff640 1 -- 172.21.15.103:0/11694084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559996a97d40 con 0x559996b9a980 2024-09-19T07:05:24.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.827+0000 7f3c4c952640 1 --2- 172.21.15.103:0/1203931506 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c380a33b0 0x7f3c380a5870 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009890 tx=0x7f3c3c008040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:24.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.827+0000 7f3c47fff640 1 --2- 172.21.15.103:0/11694084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a980 0x559996ba4d00 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f3c340ee3c0 tx=0x7f3c3402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983a4edfd739d471 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:24.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.827+0000 7f3c32ffd640 1 -- 172.21.15.103:0/11694084 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34049070 con 0x559996b9a980 2024-09-19T07:05:24.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.827+0000 7f3c32ffd640 1 -- 172.21.15.103:0/11694084 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c340a70e0 con 0x559996b9a980 2024-09-19T07:05:24.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.827+0000 7f3c32ffd640 1 -- 172.21.15.103:0/11694084 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c340c1c30 con 0x559996b9a980 2024-09-19T07:05:24.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.827+0000 7f3c50f8f740 1 -- 172.21.15.103:0/11694084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a980 msgr2=0x559996ba4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.827+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/11694084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a980 0x559996ba4d00 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f3c340ee3c0 tx=0x7f3c3402a480 comp rx=0 tx=0).stop 2024-09-19T07:05:24.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.828+0000 7f3c50f8f740 1 -- 172.21.15.103:0/11694084 shutdown_connections 2024-09-19T07:05:24.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.828+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/11694084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996ba5240 0x559996ba7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.828+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/11694084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a980 0x559996ba4d00 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.828+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/11694084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996b9a060 0x559996b9a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.828+0000 7f3c50f8f740 1 -- 172.21.15.103:0/11694084 >> 172.21.15.103:0/11694084 conn(0x559996b965f0 msgr2=0x559996b969d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:24.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.828+0000 7f3c50f8f740 1 -- 172.21.15.103:0/11694084 shutdown_connections 2024-09-19T07:05:24.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.828+0000 7f3c50f8f740 1 -- 172.21.15.103:0/11694084 wait complete. 2024-09-19T07:05:24.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.828+0000 7f3c50f8f740 1 -- start start 2024-09-19T07:05:24.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.828+0000 7f3c50f8f740 1 Processor -- start 2024-09-19T07:05:24.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.828+0000 7f3c50f8f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a060 0x559996ba20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.828+0000 7f3c50f8f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996b9a980 0x559996ba2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.828+0000 7f3c50f8f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996ba5240 0x559996ba2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.828+0000 7f3c50f8f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559996ba7bd0 con 0x559996b9a060 2024-09-19T07:05:24.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.828+0000 7f3c50f8f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559996a87f10 con 0x559996ba5240 2024-09-19T07:05:24.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.828+0000 7f3c50f8f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559996ba7d50 con 0x559996b9a980 2024-09-19T07:05:24.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.828+0000 7f3c50f8f740 10 client.0 _ll_register_callbacks cb 0x559996aabdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:24.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c4c952640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a060 0x559996ba20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c4c952640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a060 0x559996ba20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60482/0 (socket says 172.21.15.103:60482) 2024-09-19T07:05:24.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c4c952640 1 -- 172.21.15.103:0/1186576125 learned_addr learned my addr 172.21.15.103:0/1186576125 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:24.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c4d153640 1 --2- 172.21.15.103:0/1186576125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996ba5240 0x559996ba2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c47fff640 1 --2- 172.21.15.103:0/1186576125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996b9a980 0x559996ba2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c4c952640 1 -- 172.21.15.103:0/1186576125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996b9a980 msgr2=0x559996ba2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:24.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c4c952640 1 --2- 172.21.15.103:0/1186576125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996b9a980 0x559996ba2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c4c952640 1 -- 172.21.15.103:0/1186576125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996ba5240 msgr2=0x559996ba2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:24.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c4c952640 1 --2- 172.21.15.103:0/1186576125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996ba5240 0x559996ba2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c4c952640 1 -- 172.21.15.103:0/1186576125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55999680e580 con 0x559996b9a060 2024-09-19T07:05:24.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c4c952640 1 --2- 172.21.15.103:0/1186576125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a060 0x559996ba20e0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00de10 tx=0x7f3c3c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947526b8dcb0404d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:24.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c27fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:24.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c27fff640 1 -- 172.21.15.103:0/1186576125 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c024060 con 0x559996b9a060 2024-09-19T07:05:24.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c27fff640 1 -- 172.21.15.103:0/1186576125 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f3c3c01f400 con 0x559996b9a060 2024-09-19T07:05:24.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c50f8f740 1 -- 172.21.15.103:0/1186576125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c340a70e0 con 0x559996b9a060 2024-09-19T07:05:24.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c27fff640 1 -- 172.21.15.103:0/1186576125 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c3c01f620 con 0x559996b9a060 2024-09-19T07:05:24.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c50f8f740 10 client.21279 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:24.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.829+0000 7f3c50f8f740 1 -- 172.21.15.103:0/1186576125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5599968d8960 con 0x559996b9a060 2024-09-19T07:05:24.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.830+0000 7f3c27fff640 1 -- 172.21.15.103:0/1186576125 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c340a70e0 con 0x559996b9a060 2024-09-19T07:05:24.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.830+0000 7f3c27fff640 1 -- 172.21.15.103:0/1186576125 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3c3c013070 con 0x559996b9a060 2024-09-19T07:05:24.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.830+0000 7f3c50f8f740 10 client.21279 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:24.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.830+0000 7f3c50f8f740 10 client.21279 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:05:24.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.830+0000 7f3c50f8f740 20 client.21279 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:24.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.830+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/1186576125 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559996bb4b40 0x559996bb6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.830+0000 7f3c50f8f740 4 client.21279 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:24.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.830+0000 7f3c50f8f740 1 -- 172.21.15.103:0/1186576125 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559996b0cb90 con 0x559996bb4b40 2024-09-19T07:05:24.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.830+0000 7f3c47fff640 1 --2- 172.21.15.103:0/1186576125 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559996bb4b40 0x559996bb6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.831+0000 7f3c27fff640 10 client.21279 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:24.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.831+0000 7f3c47fff640 1 --2- 172.21.15.103:0/1186576125 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559996bb4b40 0x559996bb6f30 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=716544e7998a1dfa server_cookie=ab2cc44763640ff1 in_seq=0 out_seq=0 2024-09-19T07:05:24.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.831+0000 7f3c27fff640 1 -- 172.21.15.103:0/1186576125 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559996b0cb90 con 0x559996bb4b40 2024-09-19T07:05:24.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.831+0000 7f3c27fff640 10 client.21279 handle_command_reply: tid=0 2024-09-19T07:05:24.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.831+0000 7f3c50f8f740 1 client.21279 shutdown 2024-09-19T07:05:24.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.831+0000 7f3c50f8f740 1 -- 172.21.15.103:0/1186576125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a060 msgr2=0x559996ba20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.831+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/1186576125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a060 0x559996ba20e0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00de10 tx=0x7f3c3c0047d0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.831+0000 7f3c50f8f740 1 -- 172.21.15.103:0/1186576125 shutdown_connections 2024-09-19T07:05:24.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.831+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/1186576125 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559996bb4b40 0x559996bb6f30 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.832+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/1186576125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996ba5240 0x559996ba2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.832+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/1186576125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996b9a980 0x559996ba2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.832+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/1186576125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a060 0x559996ba20e0 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.832+0000 7f3c50f8f740 1 -- 172.21.15.103:0/1186576125 >> 172.21.15.103:0/1186576125 conn(0x559996b965f0 msgr2=0x559996b9c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:24.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.832+0000 7f3c27fff640 0 client.21279 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:24.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.832+0000 7f3c50f8f740 1 -- 172.21.15.103:0/1186576125 shutdown_connections 2024-09-19T07:05:24.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.832+0000 7f3c50f8f740 1 -- 172.21.15.103:0/1186576125 wait complete. 2024-09-19T07:05:24.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.832+0000 7f3c50f8f740 20 client.21279 trim_cache size 0 max 16384 2024-09-19T07:05:24.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.848+0000 7f3c50f8f740 1 Processor -- start 2024-09-19T07:05:24.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.848+0000 7f3c50f8f740 1 -- start start 2024-09-19T07:05:24.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.848+0000 7f3c50f8f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996b9a060 0x559996b9cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.848+0000 7f3c50f8f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996b9a980 0x559996b9d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.848+0000 7f3c50f8f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996ba5240 0x559996b95640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.848+0000 7f3c50f8f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481afa30 con 0x559996ba5240 2024-09-19T07:05:24.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.848+0000 7f3c50f8f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481b1570 con 0x559996b9a980 2024-09-19T07:05:24.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.848+0000 7f3c50f8f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481b0100 con 0x559996b9a060 2024-09-19T07:05:24.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.848+0000 7f3c4d153640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996ba5240 0x559996b95640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.849+0000 7f3c4d153640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996ba5240 0x559996b95640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60484/0 (socket says 172.21.15.103:60484) 2024-09-19T07:05:24.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.849+0000 7f3c4d153640 1 -- 172.21.15.103:0/2652243611 learned_addr learned my addr 172.21.15.103:0/2652243611 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:24.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.849+0000 7f3c47fff640 1 --2- 172.21.15.103:0/2652243611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996b9a980 0x559996b9d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.849+0000 7f3c4c952640 1 --2- 172.21.15.103:0/2652243611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996b9a060 0x559996b9cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.849+0000 7f3c4d153640 1 -- 172.21.15.103:0/2652243611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996b9a060 msgr2=0x559996b9cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.849+0000 7f3c4d153640 1 --2- 172.21.15.103:0/2652243611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996b9a060 0x559996b9cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.849+0000 7f3c4d153640 1 -- 172.21.15.103:0/2652243611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996b9a980 msgr2=0x559996b9d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.849+0000 7f3c4d153640 1 --2- 172.21.15.103:0/2652243611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996b9a980 0x559996b9d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.849+0000 7f3c4d153640 1 -- 172.21.15.103:0/2652243611 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559996ba4020 con 0x559996ba5240 2024-09-19T07:05:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.849+0000 7f3c47fff640 1 --2- 172.21.15.103:0/2652243611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996b9a980 0x559996b9d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.849+0000 7f3c4c952640 1 --2- 172.21.15.103:0/2652243611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996b9a060 0x559996b9cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.849+0000 7f3c4d153640 1 --2- 172.21.15.103:0/2652243611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996ba5240 0x559996b95640 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f3c400142c0 tx=0x7f3c40007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ddc3374303a1da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.849+0000 7f3c327fc640 1 -- 172.21.15.103:0/2652243611 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40003970 con 0x559996ba5240 2024-09-19T07:05:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.849+0000 7f3c327fc640 1 -- 172.21.15.103:0/2652243611 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c40012950 con 0x559996ba5240 2024-09-19T07:05:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.849+0000 7f3c327fc640 1 -- 172.21.15.103:0/2652243611 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400046d0 con 0x559996ba5240 2024-09-19T07:05:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.850+0000 7f3c50f8f740 1 -- 172.21.15.103:0/2652243611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996ba5240 msgr2=0x559996b95640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.850+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/2652243611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996ba5240 0x559996b95640 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f3c400142c0 tx=0x7f3c40007d80 comp rx=0 tx=0).stop 2024-09-19T07:05:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.850+0000 7f3c50f8f740 1 -- 172.21.15.103:0/2652243611 shutdown_connections 2024-09-19T07:05:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.850+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/2652243611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996ba5240 0x559996b95640 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.850+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/2652243611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996b9a980 0x559996b9d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.850+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/2652243611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996b9a060 0x559996b9cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.850+0000 7f3c50f8f740 1 -- 172.21.15.103:0/2652243611 >> 172.21.15.103:0/2652243611 conn(0x559996b965f0 msgr2=0x559996b94f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.850+0000 7f3c50f8f740 1 -- 172.21.15.103:0/2652243611 shutdown_connections 2024-09-19T07:05:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.850+0000 7f3c50f8f740 1 -- 172.21.15.103:0/2652243611 wait complete. 2024-09-19T07:05:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.850+0000 7f3c50f8f740 1 -- start start 2024-09-19T07:05:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.850+0000 7f3c50f8f740 1 Processor -- start 2024-09-19T07:05:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c50f8f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996b9a060 0x559996bace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c50f8f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a980 0x559996bad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c50f8f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996ba5240 0x559996ba7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c50f8f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481b0ea0 con 0x559996b9a980 2024-09-19T07:05:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c50f8f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481b07d0 con 0x559996b9a060 2024-09-19T07:05:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c50f8f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481ada30 con 0x559996ba5240 2024-09-19T07:05:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c50f8f740 10 client.0 _ll_register_callbacks cb 0x7f3c3c00dac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a980 0x559996bad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a980 0x559996bad3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60490/0 (socket says 172.21.15.103:60490) 2024-09-19T07:05:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c47fff640 1 -- 172.21.15.103:0/2826886651 learned_addr learned my addr 172.21.15.103:0/2826886651 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c4d153640 1 --2- 172.21.15.103:0/2826886651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996ba5240 0x559996ba7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c4c952640 1 --2- 172.21.15.103:0/2826886651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996b9a060 0x559996bace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c47fff640 1 -- 172.21.15.103:0/2826886651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996ba5240 msgr2=0x559996ba7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c47fff640 1 --2- 172.21.15.103:0/2826886651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996ba5240 0x559996ba7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c47fff640 1 -- 172.21.15.103:0/2826886651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996b9a060 msgr2=0x559996bace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:24.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c47fff640 1 --2- 172.21.15.103:0/2826886651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996b9a060 0x559996bace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c47fff640 1 -- 172.21.15.103:0/2826886651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559996ba8360 con 0x559996b9a980 2024-09-19T07:05:24.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c4c952640 1 --2- 172.21.15.103:0/2826886651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996b9a060 0x559996bace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:24.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c47fff640 1 --2- 172.21.15.103:0/2826886651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a980 0x559996bad3b0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f3c340c5090 tx=0x7f3c340bcad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466ed601938defc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:24.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c27fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:24.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c27fff640 1 -- 172.21.15.103:0/2826886651 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34049070 con 0x559996b9a980 2024-09-19T07:05:24.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c27fff640 1 -- 172.21.15.103:0/2826886651 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f3c340bac50 con 0x559996b9a980 2024-09-19T07:05:24.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.851+0000 7f3c50f8f740 1 -- 172.21.15.103:0/2826886651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c40012950 con 0x559996b9a980 2024-09-19T07:05:24.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.852+0000 7f3c27fff640 1 -- 172.21.15.103:0/2826886651 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c340bc2b0 con 0x559996b9a980 2024-09-19T07:05:24.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.852+0000 7f3c50f8f740 10 client.21285 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:24.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.852+0000 7f3c50f8f740 1 -- 172.21.15.103:0/2826886651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559996c1fe20 con 0x559996b9a980 2024-09-19T07:05:24.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.852+0000 7f3c27fff640 1 -- 172.21.15.103:0/2826886651 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c40012950 con 0x559996b9a980 2024-09-19T07:05:24.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.852+0000 7f3c27fff640 1 -- 172.21.15.103:0/2826886651 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3c340ee040 con 0x559996b9a980 2024-09-19T07:05:24.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.852+0000 7f3c50f8f740 10 client.21285 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:24.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.852+0000 7f3c50f8f740 10 client.21285 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:05:24.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.852+0000 7f3c50f8f740 20 client.21285 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:24.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.852+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/2826886651 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559996c203e0 0x559996c22800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:24.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.852+0000 7f3c50f8f740 4 client.21285 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "87781512", "release": true, "await": true}] 2024-09-19T07:05:24.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.852+0000 7f3c50f8f740 1 -- 172.21.15.103:0/2826886651 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "set_id": "87781512", "release": true, "await": true}) -- 0x559996b070a0 con 0x559996c203e0 2024-09-19T07:05:24.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.852+0000 7f3c4c952640 1 --2- 172.21.15.103:0/2826886651 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559996c203e0 0x559996c22800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:24.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.853+0000 7f3c27fff640 10 client.21285 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:24.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.853+0000 7f3c4c952640 1 --2- 172.21.15.103:0/2826886651 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559996c203e0 0x559996c22800 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e40e0cc71ca014c server_cookie=d024d259824c005d in_seq=0 out_seq=0 2024-09-19T07:05:24.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.856+0000 7f3c27fff640 1 -- 172.21.15.103:0/2826886651 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559996b070a0 con 0x559996c203e0 2024-09-19T07:05:24.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.856+0000 7f3c27fff640 10 client.21285 handle_command_reply: tid=0 2024-09-19T07:05:24.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.856+0000 7f3c50f8f740 1 client.21285 shutdown 2024-09-19T07:05:24.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.856+0000 7f3c50f8f740 1 -- 172.21.15.103:0/2826886651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a980 msgr2=0x559996bad3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.856+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/2826886651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a980 0x559996bad3b0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f3c340c5090 tx=0x7f3c340bcad0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.856+0000 7f3c50f8f740 1 -- 172.21.15.103:0/2826886651 shutdown_connections 2024-09-19T07:05:24.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.856+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/2826886651 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559996c203e0 0x559996c22800 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.856+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/2826886651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559996ba5240 0x559996ba7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.856+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/2826886651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559996b9a980 0x559996bad3b0 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.856+0000 7f3c27fff640 0 client.21285 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:24.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.856+0000 7f3c50f8f740 1 --2- 172.21.15.103:0/2826886651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559996b9a060 0x559996bace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.856+0000 7f3c50f8f740 1 -- 172.21.15.103:0/2826886651 >> 172.21.15.103:0/2826886651 conn(0x559996b965f0 msgr2=0x559996ba2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:24.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.857+0000 7f3c50f8f740 1 -- 172.21.15.103:0/2826886651 shutdown_connections 2024-09-19T07:05:24.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.857+0000 7f3c50f8f740 1 -- 172.21.15.103:0/2826886651 wait complete. 2024-09-19T07:05:24.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.857+0000 7f3c50f8f740 20 client.21285 trim_cache size 0 max 16384 2024-09-19T07:05:24.861 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:05:24.861 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T07:05:24.862 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T07:05:24.862 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 56, 2024-09-19T07:05:24.862 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:05:24.862 INFO:teuthology.orchestra.run.smithi103.stdout: "87781512": { 2024-09-19T07:05:24.862 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 56, 2024-09-19T07:05:24.862 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:05:24.862 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:05:24.862 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:05:24.862 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:05:24.862 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:05:24.862 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:05:24.862 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 149.4, 2024-09-19T07:05:24.862 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:05:24.862 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:05:24.863 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:05:24.863 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:05:24.863 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:05:24.863 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:05:24.863 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:05:24.863 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:05:24.863 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:05:24.863 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:05:24.863 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:05:24.863 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:05:24.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.857+0000 7f3c4ebb6640 1 -- 172.21.15.103:0/1203931506 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c380a33b0 msgr2=0x7f3c380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.857+0000 7f3c4ebb6640 1 --2- 172.21.15.103:0/1203931506 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c380a33b0 0x7f3c380a5870 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009890 tx=0x7f3c3c008040 comp rx=0 tx=0).stop 2024-09-19T07:05:24.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.857+0000 7f3c4ebb6640 1 -- 172.21.15.103:0/1203931506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c48102570 msgr2=0x7f3c480fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:24.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.857+0000 7f3c4ebb6640 1 --2- 172.21.15.103:0/1203931506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c48102570 0x7f3c480fd1c0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f3c34006010 tx=0x7f3c34003c80 comp rx=0 tx=0).stop 2024-09-19T07:05:24.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.857+0000 7f3c4ebb6640 1 -- 172.21.15.103:0/1203931506 shutdown_connections 2024-09-19T07:05:24.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.857+0000 7f3c4ebb6640 1 --2- 172.21.15.103:0/1203931506 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3c380a33b0 0x7f3c380a5870 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.857+0000 7f3c4ebb6640 1 --2- 172.21.15.103:0/1203931506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c48102ef0 0x7f3c480fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.857+0000 7f3c4ebb6640 1 --2- 172.21.15.103:0/1203931506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c48102570 0x7f3c480fd1c0 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.857+0000 7f3c4ebb6640 1 --2- 172.21.15.103:0/1203931506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c48100e20 0x7f3c480fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:24.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.857+0000 7f3c4ebb6640 1 -- 172.21.15.103:0/1203931506 >> 172.21.15.103:0/1203931506 conn(0x7f3c480fc7d0 msgr2=0x7f3c4806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:24.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.857+0000 7f3c4ebb6640 1 -- 172.21.15.103:0/1203931506 shutdown_connections 2024-09-19T07:05:24.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:24.857+0000 7f3c4ebb6640 1 -- 172.21.15.103:0/1203931506 wait complete. 2024-09-19T07:05:25.591 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:25 smithi103 ceph-mon[31158]: pgmap v506: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-19T07:05:25.591 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:25 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T07:05:25.618 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '87781512', total seconds elapsed: 33.5 2024-09-19T07:05:25.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:25 smithi134 ceph-mon[35356]: pgmap v506: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-19T07:05:25.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:25 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T07:05:25.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:25 smithi174 ceph-mon[35637]: pgmap v506: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-19T07:05:25.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:25 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T07:05:27.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:27 smithi103 ceph-mon[31158]: pgmap v507: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.3 KiB/s wr, 17 op/s 2024-09-19T07:05:27.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:27 smithi134 ceph-mon[35356]: pgmap v507: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.3 KiB/s wr, 17 op/s 2024-09-19T07:05:27.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:27 smithi174 ceph-mon[35637]: pgmap v507: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.3 KiB/s wr, 17 op/s 2024-09-19T07:05:29.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:29 smithi103 ceph-mon[31158]: pgmap v508: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-09-19T07:05:29.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:29 smithi134 ceph-mon[35356]: pgmap v508: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-09-19T07:05:29.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:29 smithi174 ceph-mon[35637]: pgmap v508: 97 pgs: 97 active+clean; 3.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-09-19T07:05:31.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:31 smithi103 ceph-mon[31158]: pgmap v509: 97 pgs: 97 active+clean; 3.5 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-19T07:05:31.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:31 smithi134 ceph-mon[35356]: pgmap v509: 97 pgs: 97 active+clean; 3.5 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-19T07:05:31.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:31 smithi174 ceph-mon[35637]: pgmap v509: 97 pgs: 97 active+clean; 3.5 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-19T07:05:33.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:33 smithi103 ceph-mon[31158]: pgmap v510: 97 pgs: 97 active+clean; 3.5 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-09-19T07:05:33.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:33 smithi134 ceph-mon[35356]: pgmap v510: 97 pgs: 97 active+clean; 3.5 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-09-19T07:05:33.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:33 smithi174 ceph-mon[35637]: pgmap v510: 97 pgs: 97 active+clean; 3.5 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-09-19T07:05:33.939 INFO:tasks.workunit.client.0.smithi103.stdout:499/600: open fd = 3 2024-09-19T07:05:33.940 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.940 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.940 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.940 INFO:tasks.workunit.client.0.smithi103.stdout:500/600: open fd = 3 2024-09-19T07:05:33.940 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.940 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.940 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.940 INFO:tasks.workunit.client.0.smithi103.stdout:501/600: open fd = 3 2024-09-19T07:05:33.940 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.940 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.940 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.941 INFO:tasks.workunit.client.0.smithi103.stdout:502/600: open fd = 3 2024-09-19T07:05:33.941 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.941 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.941 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.941 INFO:tasks.workunit.client.0.smithi103.stdout:503/600: open fd = 3 2024-09-19T07:05:33.941 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.941 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.941 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.941 INFO:tasks.workunit.client.0.smithi103.stdout:504/600: open fd = 3 2024-09-19T07:05:33.941 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.941 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.941 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.941 INFO:tasks.workunit.client.0.smithi103.stdout:505/600: open fd = 3 2024-09-19T07:05:33.942 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.942 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.942 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.942 INFO:tasks.workunit.client.0.smithi103.stdout:506/600: open fd = 3 2024-09-19T07:05:33.942 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.942 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.942 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.942 INFO:tasks.workunit.client.0.smithi103.stdout:507/600: open fd = 3 2024-09-19T07:05:33.942 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.942 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.942 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.942 INFO:tasks.workunit.client.0.smithi103.stdout:508/600: open fd = 3 2024-09-19T07:05:33.943 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.943 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.943 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.943 INFO:tasks.workunit.client.0.smithi103.stdout:509/600: open fd = 3 2024-09-19T07:05:33.943 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.943 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.943 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.943 INFO:tasks.workunit.client.0.smithi103.stdout:510/600: open fd = 3 2024-09-19T07:05:33.943 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.943 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.943 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.943 INFO:tasks.workunit.client.0.smithi103.stdout:511/600: open fd = 3 2024-09-19T07:05:33.943 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.943 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.944 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.944 INFO:tasks.workunit.client.0.smithi103.stdout:512/600: open fd = 3 2024-09-19T07:05:33.944 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.944 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.944 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.944 INFO:tasks.workunit.client.0.smithi103.stdout:513/600: open fd = 3 2024-09-19T07:05:33.944 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.944 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.944 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.944 INFO:tasks.workunit.client.0.smithi103.stdout:514/600: open fd = 3 2024-09-19T07:05:33.944 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.944 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.944 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.945 INFO:tasks.workunit.client.0.smithi103.stdout:515/600: open fd = 3 2024-09-19T07:05:33.945 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.945 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.945 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.945 INFO:tasks.workunit.client.0.smithi103.stdout:516/600: open fd = 3 2024-09-19T07:05:33.945 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.945 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.945 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.945 INFO:tasks.workunit.client.0.smithi103.stdout:517/600: open fd = 3 2024-09-19T07:05:33.945 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.945 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.945 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.945 INFO:tasks.workunit.client.0.smithi103.stdout:518/600: open fd = 3 2024-09-19T07:05:33.945 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.946 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.946 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.946 INFO:tasks.workunit.client.0.smithi103.stdout:519/600: open fd = 3 2024-09-19T07:05:33.946 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.946 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.946 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.946 INFO:tasks.workunit.client.0.smithi103.stdout:520/600: open fd = 3 2024-09-19T07:05:33.946 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.946 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.946 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.947 INFO:tasks.workunit.client.0.smithi103.stdout:521/600: open fd = 3 2024-09-19T07:05:33.947 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.947 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.947 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.947 INFO:tasks.workunit.client.0.smithi103.stdout:522/600: open fd = 3 2024-09-19T07:05:33.947 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.947 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.947 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.947 INFO:tasks.workunit.client.0.smithi103.stdout:523/600: open fd = 3 2024-09-19T07:05:33.947 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.947 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.947 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.947 INFO:tasks.workunit.client.0.smithi103.stdout:524/600: open fd = 3 2024-09-19T07:05:33.947 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.948 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.948 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.948 INFO:tasks.workunit.client.0.smithi103.stdout:525/600: open fd = 3 2024-09-19T07:05:33.948 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.948 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.948 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.948 INFO:tasks.workunit.client.0.smithi103.stdout:526/600: open fd = 3 2024-09-19T07:05:33.948 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.948 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.948 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.948 INFO:tasks.workunit.client.0.smithi103.stdout:527/600: open fd = 3 2024-09-19T07:05:33.948 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.948 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.948 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.949 INFO:tasks.workunit.client.0.smithi103.stdout:528/600: open fd = 3 2024-09-19T07:05:33.949 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.949 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.949 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.949 INFO:tasks.workunit.client.0.smithi103.stdout:529/600: open fd = 3 2024-09-19T07:05:33.949 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.949 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.949 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.949 INFO:tasks.workunit.client.0.smithi103.stdout:530/600: open fd = 3 2024-09-19T07:05:33.949 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.949 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.949 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.949 INFO:tasks.workunit.client.0.smithi103.stdout:531/600: open fd = 3 2024-09-19T07:05:33.949 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.950 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.950 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.950 INFO:tasks.workunit.client.0.smithi103.stdout:532/600: open fd = 3 2024-09-19T07:05:33.950 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.950 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.950 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.950 INFO:tasks.workunit.client.0.smithi103.stdout:533/600: open fd = 3 2024-09-19T07:05:33.950 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.950 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.950 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.951 INFO:tasks.workunit.client.0.smithi103.stdout:534/600: open fd = 3 2024-09-19T07:05:33.951 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.951 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.951 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.951 INFO:tasks.workunit.client.0.smithi103.stdout:535/600: open fd = 3 2024-09-19T07:05:33.951 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.951 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.951 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.951 INFO:tasks.workunit.client.0.smithi103.stdout:536/600: open fd = 3 2024-09-19T07:05:33.951 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.951 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.951 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.951 INFO:tasks.workunit.client.0.smithi103.stdout:537/600: open fd = 3 2024-09-19T07:05:33.951 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.952 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.952 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.952 INFO:tasks.workunit.client.0.smithi103.stdout:538/600: open fd = 3 2024-09-19T07:05:33.952 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.952 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.952 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.952 INFO:tasks.workunit.client.0.smithi103.stdout:539/600: open fd = 3 2024-09-19T07:05:33.952 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.952 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.952 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.952 INFO:tasks.workunit.client.0.smithi103.stdout:540/600: open fd = 3 2024-09-19T07:05:33.952 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.952 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.952 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.953 INFO:tasks.workunit.client.0.smithi103.stdout:541/600: open fd = 3 2024-09-19T07:05:33.953 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.953 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.953 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.953 INFO:tasks.workunit.client.0.smithi103.stdout:542/600: open fd = 3 2024-09-19T07:05:33.953 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.953 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.953 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.953 INFO:tasks.workunit.client.0.smithi103.stdout:543/600: open fd = 3 2024-09-19T07:05:33.953 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.953 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.953 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.953 INFO:tasks.workunit.client.0.smithi103.stdout:544/600: open fd = 3 2024-09-19T07:05:33.953 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.953 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.954 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.954 INFO:tasks.workunit.client.0.smithi103.stdout:545/600: open fd = 3 2024-09-19T07:05:33.954 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.954 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.954 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.954 INFO:tasks.workunit.client.0.smithi103.stdout:546/600: open fd = 3 2024-09-19T07:05:33.954 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.954 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.954 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.954 INFO:tasks.workunit.client.0.smithi103.stdout:547/600: open fd = 3 2024-09-19T07:05:33.954 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.954 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.954 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.954 INFO:tasks.workunit.client.0.smithi103.stdout:548/600: open fd = 3 2024-09-19T07:05:33.955 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.955 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.955 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.955 INFO:tasks.workunit.client.0.smithi103.stdout:549/600: open fd = 3 2024-09-19T07:05:33.955 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.955 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.955 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.955 INFO:tasks.workunit.client.0.smithi103.stdout:550/600: open fd = 3 2024-09-19T07:05:33.955 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.955 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.955 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.955 INFO:tasks.workunit.client.0.smithi103.stdout:551/600: open fd = 3 2024-09-19T07:05:33.955 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.955 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.955 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.956 INFO:tasks.workunit.client.0.smithi103.stdout:552/600: open fd = 3 2024-09-19T07:05:33.956 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.956 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.956 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.956 INFO:tasks.workunit.client.0.smithi103.stdout:553/600: open fd = 3 2024-09-19T07:05:33.956 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.956 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.956 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.956 INFO:tasks.workunit.client.0.smithi103.stdout:554/600: open fd = 3 2024-09-19T07:05:33.956 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.956 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.957 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.957 INFO:tasks.workunit.client.0.smithi103.stdout:555/600: open fd = 3 2024-09-19T07:05:33.957 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.957 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.957 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.957 INFO:tasks.workunit.client.0.smithi103.stdout:556/600: open fd = 3 2024-09-19T07:05:33.957 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.957 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.957 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.957 INFO:tasks.workunit.client.0.smithi103.stdout:557/600: open fd = 3 2024-09-19T07:05:33.957 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.957 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.957 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.957 INFO:tasks.workunit.client.0.smithi103.stdout:558/600: open fd = 3 2024-09-19T07:05:33.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.958 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.958 INFO:tasks.workunit.client.0.smithi103.stdout:559/600: open fd = 3 2024-09-19T07:05:33.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.958 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:33.958 INFO:tasks.workunit.client.0.smithi103.stdout:560/600: open fd = 3 2024-09-19T07:05:33.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.958 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:05:33.958 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:05:35.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:35 smithi103 ceph-mon[31158]: pgmap v511: 97 pgs: 97 active+clean; 3.5 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-09-19T07:05:35.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:35 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:05:35.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:35 smithi134 ceph-mon[35356]: pgmap v511: 97 pgs: 97 active+clean; 3.5 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-09-19T07:05:35.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:35 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:05:35.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:35 smithi174 ceph-mon[35637]: pgmap v511: 97 pgs: 97 active+clean; 3.5 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-09-19T07:05:35.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:35 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:05:36.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:36 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:05:36.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:36 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:05:36.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:36 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:05:36.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:36 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:05:36.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:36 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:05:36.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:36 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:05:36.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:36 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:05:36.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:36 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:05:36.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:36 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:05:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.473+0000 7f3083482640 1 -- 172.21.15.103:0/2665473080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f307c1035e0 msgr2=0x7f307c10bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.473+0000 7f3083482640 1 --2- 172.21.15.103:0/2665473080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f307c1035e0 0x7f307c10bac0 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f306c009900 tx=0x7f306c02a480 comp rx=0 tx=0).stop 2024-09-19T07:05:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.473+0000 7f3083482640 1 -- 172.21.15.103:0/2665473080 shutdown_connections 2024-09-19T07:05:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.473+0000 7f3083482640 1 --2- 172.21.15.103:0/2665473080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f307c10c110 0x7f307c10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.473+0000 7f3083482640 1 --2- 172.21.15.103:0/2665473080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f307c1035e0 0x7f307c10bac0 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.473+0000 7f3083482640 1 --2- 172.21.15.103:0/2665473080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f307c102cc0 0x7f307c1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.473+0000 7f3083482640 1 -- 172.21.15.103:0/2665473080 >> 172.21.15.103:0/2665473080 conn(0x7f307c0fc6b0 msgr2=0x7f307c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.474+0000 7f3083482640 1 -- 172.21.15.103:0/2665473080 shutdown_connections 2024-09-19T07:05:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.474+0000 7f3083482640 1 -- 172.21.15.103:0/2665473080 wait complete. 2024-09-19T07:05:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.474+0000 7f3083482640 1 Processor -- start 2024-09-19T07:05:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.474+0000 7f3083482640 1 -- start start 2024-09-19T07:05:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.474+0000 7f3083482640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f307c102cc0 0x7f307c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.474+0000 7f3083482640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f307c1035e0 0x7f307c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.474+0000 7f3083482640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f307c10c110 0x7f307c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.474+0000 7f3083482640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f307c111ef0 con 0x7f307c102cc0 2024-09-19T07:05:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.474+0000 7f3083482640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f307c111d70 con 0x7f307c10c110 2024-09-19T07:05:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.474+0000 7f3083482640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f307c112070 con 0x7f307c1035e0 2024-09-19T07:05:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f308121e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f307c102cc0 0x7f307c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f308121e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f307c102cc0 0x7f307c1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51924/0 (socket says 172.21.15.103:51924) 2024-09-19T07:05:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f308121e640 1 -- 172.21.15.103:0/2895503125 learned_addr learned my addr 172.21.15.103:0/2895503125 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f3081a1f640 1 --2- 172.21.15.103:0/2895503125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f307c10c110 0x7f307c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f3080a1d640 1 --2- 172.21.15.103:0/2895503125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f307c1035e0 0x7f307c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f308121e640 1 -- 172.21.15.103:0/2895503125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f307c1035e0 msgr2=0x7f307c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f308121e640 1 --2- 172.21.15.103:0/2895503125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f307c1035e0 0x7f307c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f308121e640 1 -- 172.21.15.103:0/2895503125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f307c10c110 msgr2=0x7f307c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f308121e640 1 --2- 172.21.15.103:0/2895503125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f307c10c110 0x7f307c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f308121e640 1 -- 172.21.15.103:0/2895503125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307c19d3b0 con 0x7f307c102cc0 2024-09-19T07:05:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f3080a1d640 1 --2- 172.21.15.103:0/2895503125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f307c1035e0 0x7f307c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f3081a1f640 1 --2- 172.21.15.103:0/2895503125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f307c10c110 0x7f307c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f308121e640 1 --2- 172.21.15.103:0/2895503125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f307c102cc0 0x7f307c1983c0 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f307800d770 tx=0x7f307800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f30727fc640 1 -- 172.21.15.103:0/2895503125 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078017070 con 0x7f307c102cc0 2024-09-19T07:05:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f30727fc640 1 -- 172.21.15.103:0/2895503125 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3078005500 con 0x7f307c102cc0 2024-09-19T07:05:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f30727fc640 1 -- 172.21.15.103:0/2895503125 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078004e00 con 0x7f307c102cc0 2024-09-19T07:05:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f3083482640 1 -- 172.21.15.103:0/2895503125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307c19d6a0 con 0x7f307c102cc0 2024-09-19T07:05:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.475+0000 7f3083482640 1 -- 172.21.15.103:0/2895503125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f307c1a4f30 con 0x7f307c102cc0 2024-09-19T07:05:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.478+0000 7f30727fc640 1 -- 172.21.15.103:0/2895503125 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3078004fa0 con 0x7f307c102cc0 2024-09-19T07:05:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.478+0000 7f3086220740 1 Processor -- start 2024-09-19T07:05:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.478+0000 7f3086220740 1 -- start start 2024-09-19T07:05:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.478+0000 7f3086220740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 0x5589d21d8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.478+0000 7f3086220740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21d9410 0x5589d21e3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.478+0000 7f3086220740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21e3cb0 0x5589d21e60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.478+0000 7f3086220740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5589d213d2f0 con 0x5589d21e3cb0 2024-09-19T07:05:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.478+0000 7f3086220740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5589d20e5270 con 0x5589d21d9410 2024-09-19T07:05:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f3086220740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5589d21392b0 con 0x5589d21d8af0 2024-09-19T07:05:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f30727fc640 1 --2- 172.21.15.103:0/2895503125 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f30680a3460 0x7f30680a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f3080a1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21e3cb0 0x5589d21e60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f3080a1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21e3cb0 0x5589d21e60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51940/0 (socket says 172.21.15.103:51940) 2024-09-19T07:05:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f3080a1d640 1 -- 172.21.15.103:0/1106405176 learned_addr learned my addr 172.21.15.103:0/1106405176 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f3081a1f640 1 --2- 172.21.15.103:0/1106405176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 0x5589d21d8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f3081a1f640 1 --2- 172.21.15.103:0/2895503125 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f30680a3460 0x7f30680a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f308121e640 1 --2- 172.21.15.103:0/1106405176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21d9410 0x5589d21e3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f30727fc640 1 -- 172.21.15.103:0/2895503125 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f30780cf870 con 0x7f307c102cc0 2024-09-19T07:05:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f3080a1d640 1 -- 172.21.15.103:0/1106405176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 msgr2=0x5589d21d8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f3080a1d640 1 --2- 172.21.15.103:0/1106405176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 0x5589d21d8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f3080a1d640 1 -- 172.21.15.103:0/1106405176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21d9410 msgr2=0x5589d21e3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f3080a1d640 1 --2- 172.21.15.103:0/1106405176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21d9410 0x5589d21e3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f3080a1d640 1 -- 172.21.15.103:0/1106405176 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589d20e4340 con 0x5589d21e3cb0 2024-09-19T07:05:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f308121e640 1 --2- 172.21.15.103:0/1106405176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21d9410 0x5589d21e3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f3081a1f640 1 --2- 172.21.15.103:0/2895503125 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f30680a3460 0x7f30680a5920 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3074007310 tx=0x7f30740072a0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f3080a1d640 1 --2- 172.21.15.103:0/1106405176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21e3cb0 0x5589d21e60a0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f306c031550 tx=0x7f306c032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7f4ab5960925a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.479+0000 7f30677fe640 1 -- 172.21.15.103:0/1106405176 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c03f070 con 0x5589d21e3cb0 2024-09-19T07:05:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.480+0000 7f30677fe640 1 -- 172.21.15.103:0/1106405176 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f306c032e20 con 0x5589d21e3cb0 2024-09-19T07:05:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.480+0000 7f30677fe640 1 -- 172.21.15.103:0/1106405176 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c035070 con 0x5589d21e3cb0 2024-09-19T07:05:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.480+0000 7f3086220740 1 -- 172.21.15.103:0/1106405176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21e3cb0 msgr2=0x5589d21e60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.480+0000 7f3086220740 1 --2- 172.21.15.103:0/1106405176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21e3cb0 0x5589d21e60a0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f306c031550 tx=0x7f306c032c30 comp rx=0 tx=0).stop 2024-09-19T07:05:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.480+0000 7f3086220740 1 -- 172.21.15.103:0/1106405176 shutdown_connections 2024-09-19T07:05:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.480+0000 7f3086220740 1 --2- 172.21.15.103:0/1106405176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21e3cb0 0x5589d21e60a0 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.480+0000 7f3086220740 1 --2- 172.21.15.103:0/1106405176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21d9410 0x5589d21e3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.480+0000 7f3086220740 1 --2- 172.21.15.103:0/1106405176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 0x5589d21d8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.480+0000 7f3086220740 1 -- 172.21.15.103:0/1106405176 >> 172.21.15.103:0/1106405176 conn(0x5589d21d5060 msgr2=0x5589d21d5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.480+0000 7f3086220740 1 -- 172.21.15.103:0/1106405176 shutdown_connections 2024-09-19T07:05:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.480+0000 7f3086220740 1 -- 172.21.15.103:0/1106405176 wait complete. 2024-09-19T07:05:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.481+0000 7f3086220740 1 -- start start 2024-09-19T07:05:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.481+0000 7f3086220740 1 Processor -- start 2024-09-19T07:05:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.481+0000 7f3086220740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d8af0 0x5589d21e0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.481+0000 7f3086220740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d9410 0x5589d21e1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.481+0000 7f3086220740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21e3cb0 0x5589d21e1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.481+0000 7f3086220740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5589d21e6880 con 0x5589d21d8af0 2024-09-19T07:05:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.481+0000 7f3086220740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5589d20c6290 con 0x5589d21e3cb0 2024-09-19T07:05:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.481+0000 7f3086220740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5589d21e6a00 con 0x5589d21d9410 2024-09-19T07:05:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.481+0000 7f3086220740 10 client.0 _ll_register_callbacks cb 0x5589d20f7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.481+0000 7f308121e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d8af0 0x5589d21e0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.481+0000 7f308121e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d8af0 0x5589d21e0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51946/0 (socket says 172.21.15.103:51946) 2024-09-19T07:05:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.481+0000 7f308121e640 1 -- 172.21.15.103:0/4200693863 learned_addr learned my addr 172.21.15.103:0/4200693863 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.481+0000 7f3081a1f640 1 --2- 172.21.15.103:0/4200693863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21e3cb0 0x5589d21e1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.481+0000 7f3080a1d640 1 --2- 172.21.15.103:0/4200693863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d9410 0x5589d21e1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f308121e640 1 -- 172.21.15.103:0/4200693863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d9410 msgr2=0x5589d21e1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f308121e640 1 --2- 172.21.15.103:0/4200693863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d9410 0x5589d21e1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f308121e640 1 -- 172.21.15.103:0/4200693863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21e3cb0 msgr2=0x5589d21e1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f308121e640 1 --2- 172.21.15.103:0/4200693863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21e3cb0 0x5589d21e1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f308121e640 1 -- 172.21.15.103:0/4200693863 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589d1e50210 con 0x5589d21d8af0 2024-09-19T07:05:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f3081a1f640 1 --2- 172.21.15.103:0/4200693863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21e3cb0 0x5589d21e1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f308121e640 1 --2- 172.21.15.103:0/4200693863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d8af0 0x5589d21e0bc0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f3078097f90 tx=0x7f3078018cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c212d8863e42804b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f3064ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f3064ff9640 1 -- 172.21.15.103:0/4200693863 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078020050 con 0x5589d21d8af0 2024-09-19T07:05:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f3064ff9640 1 -- 172.21.15.103:0/4200693863 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f3078018d60 con 0x5589d21d8af0 2024-09-19T07:05:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f3086220740 1 -- 172.21.15.103:0/4200693863 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f306c032e20 con 0x5589d21d8af0 2024-09-19T07:05:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f3064ff9640 1 -- 172.21.15.103:0/4200693863 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f307809c230 con 0x5589d21d8af0 2024-09-19T07:05:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f3086220740 10 client.21297 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f3086220740 1 -- 172.21.15.103:0/4200693863 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5589d1f1c620 con 0x5589d21d8af0 2024-09-19T07:05:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f3064ff9640 1 -- 172.21.15.103:0/4200693863 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f306c032e20 con 0x5589d21d8af0 2024-09-19T07:05:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f3064ff9640 1 -- 172.21.15.103:0/4200693863 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f30780cd220 con 0x5589d21d8af0 2024-09-19T07:05:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f3086220740 10 client.21297 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f3086220740 10 client.21297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f3086220740 20 client.21297 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f3086220740 1 --2- 172.21.15.103:0/4200693863 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5589d21f3d10 0x5589d21f6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f3086220740 4 client.21297 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.482+0000 7f3086220740 1 -- 172.21.15.103:0/4200693863 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589d214aa20 con 0x5589d21f3d10 2024-09-19T07:05:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.483+0000 7f3080a1d640 1 --2- 172.21.15.103:0/4200693863 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5589d21f3d10 0x5589d21f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.483+0000 7f3080a1d640 1 --2- 172.21.15.103:0/4200693863 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5589d21f3d10 0x5589d21f6100 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8534133ca6e7d92a server_cookie=db619b92dcd6a303 in_seq=0 out_seq=0 2024-09-19T07:05:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.483+0000 7f3064ff9640 10 client.21297 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.484+0000 7f3064ff9640 1 -- 172.21.15.103:0/4200693863 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589d214aa20 con 0x5589d21f3d10 2024-09-19T07:05:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.484+0000 7f3064ff9640 10 client.21297 handle_command_reply: tid=0 2024-09-19T07:05:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.484+0000 7f3086220740 1 client.21297 shutdown 2024-09-19T07:05:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.484+0000 7f3086220740 1 -- 172.21.15.103:0/4200693863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d8af0 msgr2=0x5589d21e0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.484+0000 7f3086220740 1 --2- 172.21.15.103:0/4200693863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d8af0 0x5589d21e0bc0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f3078097f90 tx=0x7f3078018cf0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.484+0000 7f3086220740 1 -- 172.21.15.103:0/4200693863 shutdown_connections 2024-09-19T07:05:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.484+0000 7f3086220740 1 --2- 172.21.15.103:0/4200693863 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5589d21f3d10 0x5589d21f6100 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.484+0000 7f3086220740 1 --2- 172.21.15.103:0/4200693863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21e3cb0 0x5589d21e1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.484+0000 7f3086220740 1 --2- 172.21.15.103:0/4200693863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d9410 0x5589d21e1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.484+0000 7f3064ff9640 0 client.21297 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.484+0000 7f3086220740 1 --2- 172.21.15.103:0/4200693863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d8af0 0x5589d21e0bc0 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.484+0000 7f3086220740 1 -- 172.21.15.103:0/4200693863 >> 172.21.15.103:0/4200693863 conn(0x5589d21d5060 msgr2=0x5589d21dad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.484+0000 7f3086220740 1 -- 172.21.15.103:0/4200693863 shutdown_connections 2024-09-19T07:05:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.484+0000 7f3086220740 1 -- 172.21.15.103:0/4200693863 wait complete. 2024-09-19T07:05:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.484+0000 7f3086220740 20 client.21297 trim_cache size 0 max 16384 2024-09-19T07:05:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3086220740 1 Processor -- start 2024-09-19T07:05:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3086220740 1 -- start start 2024-09-19T07:05:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3086220740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 0x5589d21e1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3086220740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d9410 0x5589d21fb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3086220740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21e3cb0 0x5589d21fbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3086220740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1ad7e0 con 0x5589d21d9410 2024-09-19T07:05:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3086220740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1af320 con 0x5589d21e3cb0 2024-09-19T07:05:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3086220740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1adeb0 con 0x5589d21d8af0 2024-09-19T07:05:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3080a1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d9410 0x5589d21fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3080a1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d9410 0x5589d21fb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51952/0 (socket says 172.21.15.103:51952) 2024-09-19T07:05:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3080a1d640 1 -- 172.21.15.103:0/3806183877 learned_addr learned my addr 172.21.15.103:0/3806183877 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f308121e640 1 --2- 172.21.15.103:0/3806183877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 0x5589d21e1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3081a1f640 1 --2- 172.21.15.103:0/3806183877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21e3cb0 0x5589d21fbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3080a1d640 1 -- 172.21.15.103:0/3806183877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 msgr2=0x5589d21e1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3080a1d640 1 --2- 172.21.15.103:0/3806183877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 0x5589d21e1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3080a1d640 1 -- 172.21.15.103:0/3806183877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21e3cb0 msgr2=0x5589d21fbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3080a1d640 1 --2- 172.21.15.103:0/3806183877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21e3cb0 0x5589d21fbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3080a1d640 1 -- 172.21.15.103:0/3806183877 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589d21e6410 con 0x5589d21d9410 2024-09-19T07:05:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f308121e640 1 --2- 172.21.15.103:0/3806183877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 0x5589d21e1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.499+0000 7f3081a1f640 1 --2- 172.21.15.103:0/3806183877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21e3cb0 0x5589d21fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.500+0000 7f3080a1d640 1 --2- 172.21.15.103:0/3806183877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d9410 0x5589d21fb930 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f306c0057c0 tx=0x7f306c0051e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79be1050734bade4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.500+0000 7f3066ffd640 1 -- 172.21.15.103:0/3806183877 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c03f050 con 0x5589d21d9410 2024-09-19T07:05:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.500+0000 7f3066ffd640 1 -- 172.21.15.103:0/3806183877 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f306c032e20 con 0x5589d21d9410 2024-09-19T07:05:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.500+0000 7f3066ffd640 1 -- 172.21.15.103:0/3806183877 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c035040 con 0x5589d21d9410 2024-09-19T07:05:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.500+0000 7f3086220740 1 -- 172.21.15.103:0/3806183877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d9410 msgr2=0x5589d21fb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.500+0000 7f3086220740 1 --2- 172.21.15.103:0/3806183877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d9410 0x5589d21fb930 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f306c0057c0 tx=0x7f306c0051e0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.500+0000 7f3086220740 1 -- 172.21.15.103:0/3806183877 shutdown_connections 2024-09-19T07:05:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.500+0000 7f3086220740 1 --2- 172.21.15.103:0/3806183877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21e3cb0 0x5589d21fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.500+0000 7f3086220740 1 --2- 172.21.15.103:0/3806183877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d9410 0x5589d21fb930 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.500+0000 7f3086220740 1 --2- 172.21.15.103:0/3806183877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 0x5589d21e1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.500+0000 7f3086220740 1 -- 172.21.15.103:0/3806183877 >> 172.21.15.103:0/3806183877 conn(0x5589d21d5060 msgr2=0x5589d21e2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.500+0000 7f3086220740 1 -- 172.21.15.103:0/3806183877 shutdown_connections 2024-09-19T07:05:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.500+0000 7f3086220740 1 -- 172.21.15.103:0/3806183877 wait complete. 2024-09-19T07:05:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.501+0000 7f3086220740 1 -- start start 2024-09-19T07:05:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.501+0000 7f3086220740 1 Processor -- start 2024-09-19T07:05:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.501+0000 7f3086220740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 0x5589d21ddbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.501+0000 7f3086220740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d9410 0x5589d21de120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.501+0000 7f3086220740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21e3cb0 0x5589d21de660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.501+0000 7f3086220740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1aec50 con 0x5589d21d9410 2024-09-19T07:05:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.501+0000 7f3086220740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1ae580 con 0x5589d21e3cb0 2024-09-19T07:05:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.501+0000 7f3086220740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1ab7e0 con 0x5589d21d8af0 2024-09-19T07:05:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.501+0000 7f3086220740 10 client.0 _ll_register_callbacks cb 0x7f307800de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.501+0000 7f3080a1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d9410 0x5589d21de120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.501+0000 7f3080a1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d9410 0x5589d21de120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51968/0 (socket says 172.21.15.103:51968) 2024-09-19T07:05:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.501+0000 7f3080a1d640 1 -- 172.21.15.103:0/2210964612 learned_addr learned my addr 172.21.15.103:0/2210964612 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.501+0000 7f308121e640 1 --2- 172.21.15.103:0/2210964612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 0x5589d21ddbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3080a1d640 1 -- 172.21.15.103:0/2210964612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 msgr2=0x5589d21ddbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3080a1d640 1 --2- 172.21.15.103:0/2210964612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 0x5589d21ddbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3080a1d640 1 -- 172.21.15.103:0/2210964612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21e3cb0 msgr2=0x5589d21de660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3080a1d640 1 --2- 172.21.15.103:0/2210964612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21e3cb0 0x5589d21de660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3080a1d640 1 -- 172.21.15.103:0/2210964612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589d225d900 con 0x5589d21d9410 2024-09-19T07:05:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f308121e640 1 --2- 172.21.15.103:0/2210964612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 0x5589d21ddbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3080a1d640 1 --2- 172.21.15.103:0/2210964612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d9410 0x5589d21de120 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f306c04e040 tx=0x7f306c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fd8764c193e68f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3064ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3064ff9640 1 -- 172.21.15.103:0/2210964612 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c035050 con 0x5589d21d9410 2024-09-19T07:05:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3064ff9640 1 -- 172.21.15.103:0/2210964612 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f306c0046c0 con 0x5589d21d9410 2024-09-19T07:05:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3086220740 1 -- 172.21.15.103:0/2210964612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f306c032e20 con 0x5589d21d9410 2024-09-19T07:05:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3064ff9640 1 -- 172.21.15.103:0/2210964612 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f306c03f040 con 0x5589d21d9410 2024-09-19T07:05:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3086220740 10 client.21303 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3086220740 1 -- 172.21.15.103:0/2210964612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5589d225eb60 con 0x5589d21d9410 2024-09-19T07:05:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3064ff9640 1 -- 172.21.15.103:0/2210964612 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f306c032e20 con 0x5589d21d9410 2024-09-19T07:05:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3064ff9640 1 -- 172.21.15.103:0/2210964612 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f306c03b5e0 con 0x5589d21d9410 2024-09-19T07:05:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3086220740 10 client.21303 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3086220740 10 client.21303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3086220740 20 client.21303 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3086220740 1 --2- 172.21.15.103:0/2210964612 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5589d225ee50 0x5589d2261240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3086220740 4 client.21303 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:05:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.502+0000 7f3086220740 1 -- 172.21.15.103:0/2210964612 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589d2144d70 con 0x5589d225ee50 2024-09-19T07:05:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.503+0000 7f308121e640 1 --2- 172.21.15.103:0/2210964612 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5589d225ee50 0x5589d2261240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.503+0000 7f308121e640 1 --2- 172.21.15.103:0/2210964612 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5589d225ee50 0x5589d2261240 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2978913f2f2de70c server_cookie=69b30653a9fe434d in_seq=0 out_seq=0 2024-09-19T07:05:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.503+0000 7f3064ff9640 10 client.21303 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.513+0000 7f3064ff9640 1 -- 172.21.15.103:0/2210964612 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589d2144d70 con 0x5589d225ee50 2024-09-19T07:05:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.513+0000 7f3064ff9640 10 client.21303 handle_command_reply: tid=0 2024-09-19T07:05:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.513+0000 7f3086220740 1 client.21303 shutdown 2024-09-19T07:05:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.513+0000 7f3086220740 1 -- 172.21.15.103:0/2210964612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d9410 msgr2=0x5589d21de120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.513+0000 7f3086220740 1 --2- 172.21.15.103:0/2210964612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d9410 0x5589d21de120 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f306c04e040 tx=0x7f306c0035f0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.513+0000 7f3086220740 1 -- 172.21.15.103:0/2210964612 shutdown_connections 2024-09-19T07:05:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.513+0000 7f3086220740 1 --2- 172.21.15.103:0/2210964612 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x5589d225ee50 0x5589d2261240 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.513+0000 7f3086220740 1 --2- 172.21.15.103:0/2210964612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589d21e3cb0 0x5589d21de660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.513+0000 7f3064ff9640 0 client.21303 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.513+0000 7f3086220740 1 --2- 172.21.15.103:0/2210964612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589d21d9410 0x5589d21de120 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.513+0000 7f3086220740 1 --2- 172.21.15.103:0/2210964612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589d21d8af0 0x5589d21ddbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.513+0000 7f3086220740 1 -- 172.21.15.103:0/2210964612 >> 172.21.15.103:0/2210964612 conn(0x5589d21d5060 msgr2=0x5589d21da550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.514+0000 7f3086220740 1 -- 172.21.15.103:0/2210964612 shutdown_connections 2024-09-19T07:05:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.514+0000 7f3086220740 1 -- 172.21.15.103:0/2210964612 wait complete. 2024-09-19T07:05:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.514+0000 7f3086220740 20 client.21303 trim_cache size 0 max 16384 2024-09-19T07:05:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:05:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:05:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:05:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:05:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.514+0000 7f3083482640 1 -- 172.21.15.103:0/2895503125 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f30680a3460 msgr2=0x7f30680a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.514+0000 7f3083482640 1 --2- 172.21.15.103:0/2895503125 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f30680a3460 0x7f30680a5920 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3074007310 tx=0x7f30740072a0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.514+0000 7f3083482640 1 -- 172.21.15.103:0/2895503125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f307c102cc0 msgr2=0x7f307c1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.514+0000 7f3083482640 1 --2- 172.21.15.103:0/2895503125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f307c102cc0 0x7f307c1983c0 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f307800d770 tx=0x7f307800dc30 comp rx=0 tx=0).stop 2024-09-19T07:05:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.514+0000 7f3083482640 1 -- 172.21.15.103:0/2895503125 shutdown_connections 2024-09-19T07:05:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.514+0000 7f3083482640 1 --2- 172.21.15.103:0/2895503125 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f30680a3460 0x7f30680a5920 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.514+0000 7f3083482640 1 --2- 172.21.15.103:0/2895503125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f307c10c110 0x7f307c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.514+0000 7f3083482640 1 --2- 172.21.15.103:0/2895503125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f307c1035e0 0x7f307c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.514+0000 7f3083482640 1 --2- 172.21.15.103:0/2895503125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f307c102cc0 0x7f307c1983c0 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.514+0000 7f3083482640 1 -- 172.21.15.103:0/2895503125 >> 172.21.15.103:0/2895503125 conn(0x7f307c0fc6b0 msgr2=0x7f307c0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.514+0000 7f3083482640 1 -- 172.21.15.103:0/2895503125 shutdown_connections 2024-09-19T07:05:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:37.514+0000 7f3083482640 1 -- 172.21.15.103:0/2895503125 wait complete. 2024-09-19T07:05:37.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:37 smithi134 ceph-mon[35356]: pgmap v512: 97 pgs: 97 active+clean; 3.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 19 op/s 2024-09-19T07:05:37.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:37 smithi174 ceph-mon[35637]: pgmap v512: 97 pgs: 97 active+clean; 3.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 19 op/s 2024-09-19T07:05:37.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:37 smithi103 ceph-mon[31158]: pgmap v512: 97 pgs: 97 active+clean; 3.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 19 op/s 2024-09-19T07:05:39.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:39 smithi134 ceph-mon[35356]: pgmap v513: 97 pgs: 97 active+clean; 3.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-19T07:05:39.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:39 smithi174 ceph-mon[35637]: pgmap v513: 97 pgs: 97 active+clean; 3.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-19T07:05:39.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:39 smithi103 ceph-mon[31158]: pgmap v513: 97 pgs: 97 active+clean; 3.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-19T07:05:41.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:41 smithi134 ceph-mon[35356]: pgmap v514: 97 pgs: 97 active+clean; 4.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-09-19T07:05:41.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:41 smithi174 ceph-mon[35637]: pgmap v514: 97 pgs: 97 active+clean; 4.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-09-19T07:05:41.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:41 smithi103 ceph-mon[31158]: pgmap v514: 97 pgs: 97 active+clean; 4.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-09-19T07:05:43.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:43 smithi134 ceph-mon[35356]: pgmap v515: 97 pgs: 97 active+clean; 4.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T07:05:43.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:43 smithi174 ceph-mon[35637]: pgmap v515: 97 pgs: 97 active+clean; 4.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T07:05:43.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:43 smithi103 ceph-mon[31158]: pgmap v515: 97 pgs: 97 active+clean; 4.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T07:05:45.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:45 smithi174 ceph-mon[35637]: pgmap v516: 97 pgs: 97 active+clean; 4.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T07:05:45.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:45 smithi103 ceph-mon[31158]: pgmap v516: 97 pgs: 97 active+clean; 4.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T07:05:45.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:45 smithi134 ceph-mon[35356]: pgmap v516: 97 pgs: 97 active+clean; 4.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T07:05:47.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:47 smithi174 ceph-mon[35637]: pgmap v517: 97 pgs: 97 active+clean; 4.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 21 op/s 2024-09-19T07:05:47.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:47 smithi103 ceph-mon[31158]: pgmap v517: 97 pgs: 97 active+clean; 4.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 21 op/s 2024-09-19T07:05:47.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:47 smithi134 ceph-mon[35356]: pgmap v517: 97 pgs: 97 active+clean; 4.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 21 op/s 2024-09-19T07:05:49.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:49 smithi174 ceph-mon[35637]: pgmap v518: 97 pgs: 97 active+clean; 4.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T07:05:49.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:49 smithi103 ceph-mon[31158]: pgmap v518: 97 pgs: 97 active+clean; 4.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T07:05:49.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:49 smithi134 ceph-mon[35356]: pgmap v518: 97 pgs: 97 active+clean; 4.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-19T07:05:50.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.611+0000 7fb4b2fd8640 1 -- 172.21.15.103:0/2686811248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4ac102cc0 msgr2=0x7fb4ac1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:50.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.611+0000 7fb4b2fd8640 1 --2- 172.21.15.103:0/2686811248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4ac102cc0 0x7fb4ac1030a0 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009900 tx=0x7fb4a002a480 comp rx=0 tx=0).stop 2024-09-19T07:05:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.612+0000 7fb4b2fd8640 1 -- 172.21.15.103:0/2686811248 shutdown_connections 2024-09-19T07:05:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.612+0000 7fb4b2fd8640 1 --2- 172.21.15.103:0/2686811248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4ac10c110 0x7fb4ac10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.612+0000 7fb4b2fd8640 1 --2- 172.21.15.103:0/2686811248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4ac1035e0 0x7fb4ac10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.612+0000 7fb4b2fd8640 1 --2- 172.21.15.103:0/2686811248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4ac102cc0 0x7fb4ac1030a0 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.612+0000 7fb4b2fd8640 1 -- 172.21.15.103:0/2686811248 >> 172.21.15.103:0/2686811248 conn(0x7fb4ac0fc6b0 msgr2=0x7fb4ac0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.612+0000 7fb4b2fd8640 1 -- 172.21.15.103:0/2686811248 shutdown_connections 2024-09-19T07:05:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.612+0000 7fb4b2fd8640 1 -- 172.21.15.103:0/2686811248 wait complete. 2024-09-19T07:05:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.612+0000 7fb4b2fd8640 1 Processor -- start 2024-09-19T07:05:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.612+0000 7fb4b2fd8640 1 -- start start 2024-09-19T07:05:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.612+0000 7fb4b2fd8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4ac102cc0 0x7fb4ac1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.612+0000 7fb4b2fd8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4ac1035e0 0x7fb4ac198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4b2fd8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4ac10c110 0x7fb4ac19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4b2fd8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac111ef0 con 0x7fb4ac10c110 2024-09-19T07:05:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4b2fd8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac111d70 con 0x7fb4ac1035e0 2024-09-19T07:05:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4b2fd8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac112070 con 0x7fb4ac102cc0 2024-09-19T07:05:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4b1575640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4ac10c110 0x7fb4ac19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4b0d74640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4ac102cc0 0x7fb4ac1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4b1575640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4ac10c110 0x7fb4ac19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57560/0 (socket says 172.21.15.103:57560) 2024-09-19T07:05:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4b0d74640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4ac102cc0 0x7fb4ac1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:43222/0 (socket says 172.21.15.103:43222) 2024-09-19T07:05:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4b1575640 1 -- 172.21.15.103:0/1540240354 learned_addr learned my addr 172.21.15.103:0/1540240354 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4abfff640 1 --2- 172.21.15.103:0/1540240354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4ac1035e0 0x7fb4ac198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4b1575640 1 -- 172.21.15.103:0/1540240354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4ac102cc0 msgr2=0x7fb4ac1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4b1575640 1 --2- 172.21.15.103:0/1540240354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4ac102cc0 0x7fb4ac1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4b1575640 1 -- 172.21.15.103:0/1540240354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4ac1035e0 msgr2=0x7fb4ac198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4b1575640 1 --2- 172.21.15.103:0/1540240354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4ac1035e0 0x7fb4ac198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4b1575640 1 -- 172.21.15.103:0/1540240354 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ac19d3d0 con 0x7fb4ac10c110 2024-09-19T07:05:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4abfff640 1 --2- 172.21.15.103:0/1540240354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4ac1035e0 0x7fb4ac198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4b1575640 1 --2- 172.21.15.103:0/1540240354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4ac10c110 0x7fb4ac19cca0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009fd0 tx=0x7fb4a400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4a9ffb640 1 -- 172.21.15.103:0/1540240354 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4017070 con 0x7fb4ac10c110 2024-09-19T07:05:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4a9ffb640 1 -- 172.21.15.103:0/1540240354 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a4003e50 con 0x7fb4ac10c110 2024-09-19T07:05:50.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4b2fd8640 1 -- 172.21.15.103:0/1540240354 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4ac19d6c0 con 0x7fb4ac10c110 2024-09-19T07:05:50.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.613+0000 7fb4a9ffb640 1 -- 172.21.15.103:0/1540240354 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a40106c0 con 0x7fb4ac10c110 2024-09-19T07:05:50.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.614+0000 7fb4b2fd8640 1 -- 172.21.15.103:0/1540240354 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4ac1a4f50 con 0x7fb4ac10c110 2024-09-19T07:05:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.616+0000 7fb4a9ffb640 1 -- 172.21.15.103:0/1540240354 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fb4a4003970 con 0x7fb4ac10c110 2024-09-19T07:05:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.616+0000 7fb4b5e49740 1 Processor -- start 2024-09-19T07:05:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.616+0000 7fb4b5e49740 1 -- start start 2024-09-19T07:05:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.616+0000 7fb4a9ffb640 1 --2- 172.21.15.103:0/1540240354 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb49c0a34e0 0x7fb49c0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.616+0000 7fb4b5e49740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff61af0 0x55c08ff61ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.616+0000 7fb4b5e49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff62410 0x55c08ff6c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.616+0000 7fb4b5e49740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff6ccb0 0x55c08ff6f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.616+0000 7fb4b5e49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c08fec62f0 con 0x55c08ff62410 2024-09-19T07:05:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.616+0000 7fb4b5e49740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c08fe6e270 con 0x55c08ff61af0 2024-09-19T07:05:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.616+0000 7fb4b5e49740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c08fec22b0 con 0x55c08ff6ccb0 2024-09-19T07:05:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb4abfff640 1 --2- 172.21.15.103:0/1540240354 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb49c0a34e0 0x7fb49c0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb4a9ffb640 1 -- 172.21.15.103:0/1540240354 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fb4a40cf9e0 con 0x7fb4ac10c110 2024-09-19T07:05:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb4abfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff62410 0x55c08ff6c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb4abfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff62410 0x55c08ff6c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57576/0 (socket says 172.21.15.103:57576) 2024-09-19T07:05:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb4abfff640 1 -- 172.21.15.103:0/3858043514 learned_addr learned my addr 172.21.15.103:0/3858043514 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb4b0d74640 1 --2- 172.21.15.103:0/3858043514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff61af0 0x55c08ff61ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb4b1575640 1 --2- 172.21.15.103:0/3858043514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff6ccb0 0x55c08ff6f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb4abfff640 1 -- 172.21.15.103:0/3858043514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff6ccb0 msgr2=0x55c08ff6f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:50.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb4abfff640 1 --2- 172.21.15.103:0/3858043514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff6ccb0 0x55c08ff6f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb4abfff640 1 -- 172.21.15.103:0/3858043514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff61af0 msgr2=0x55c08ff61ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:50.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb4abfff640 1 --2- 172.21.15.103:0/3858043514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff61af0 0x55c08ff61ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb4abfff640 1 -- 172.21.15.103:0/3858043514 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c08fe6d340 con 0x55c08ff62410 2024-09-19T07:05:50.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb4abfff640 1 --2- 172.21.15.103:0/1540240354 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb49c0a34e0 0x7fb49c0a59a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb498014a10 tx=0x7fb49800bc90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:50.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb4abfff640 1 --2- 172.21.15.103:0/3858043514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff62410 0x55c08ff6c770 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fb498011c20 tx=0x7fb49801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37aabeeb770e6ef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:50.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb496ffd640 1 -- 172.21.15.103:0/3858043514 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498022070 con 0x55c08ff62410 2024-09-19T07:05:50.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb496ffd640 1 -- 172.21.15.103:0/3858043514 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb498005500 con 0x55c08ff62410 2024-09-19T07:05:50.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.617+0000 7fb496ffd640 1 -- 172.21.15.103:0/3858043514 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4980047a0 con 0x55c08ff62410 2024-09-19T07:05:50.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.618+0000 7fb4b5e49740 1 -- 172.21.15.103:0/3858043514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff62410 msgr2=0x55c08ff6c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:50.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.618+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/3858043514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff62410 0x55c08ff6c770 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fb498011c20 tx=0x7fb49801b000 comp rx=0 tx=0).stop 2024-09-19T07:05:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.618+0000 7fb4b5e49740 1 -- 172.21.15.103:0/3858043514 shutdown_connections 2024-09-19T07:05:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.618+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/3858043514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff6ccb0 0x55c08ff6f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.618+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/3858043514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff62410 0x55c08ff6c770 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.618+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/3858043514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff61af0 0x55c08ff61ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.618+0000 7fb4b5e49740 1 -- 172.21.15.103:0/3858043514 >> 172.21.15.103:0/3858043514 conn(0x55c08ff5e060 msgr2=0x55c08ff5e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.618+0000 7fb4b5e49740 1 -- 172.21.15.103:0/3858043514 shutdown_connections 2024-09-19T07:05:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.618+0000 7fb4b5e49740 1 -- 172.21.15.103:0/3858043514 wait complete. 2024-09-19T07:05:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.618+0000 7fb4b5e49740 1 -- start start 2024-09-19T07:05:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.619+0000 7fb4b5e49740 1 Processor -- start 2024-09-19T07:05:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.619+0000 7fb4b5e49740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff61af0 0x55c08ff69c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.619+0000 7fb4b5e49740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff62410 0x55c08ff6a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.619+0000 7fb4b5e49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff6ccb0 0x55c08ff6a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.619+0000 7fb4b5e49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c08ff6f880 con 0x55c08ff6ccb0 2024-09-19T07:05:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.619+0000 7fb4b5e49740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c08fe4f290 con 0x55c08ff61af0 2024-09-19T07:05:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.619+0000 7fb4b5e49740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c08ff6fa00 con 0x55c08ff62410 2024-09-19T07:05:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.619+0000 7fb4b5e49740 10 client.0 _ll_register_callbacks cb 0x55c08fe80140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.619+0000 7fb4b1575640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff6ccb0 0x55c08ff6a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.619+0000 7fb4b1575640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff6ccb0 0x55c08ff6a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57582/0 (socket says 172.21.15.103:57582) 2024-09-19T07:05:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.619+0000 7fb4b1575640 1 -- 172.21.15.103:0/3518971207 learned_addr learned my addr 172.21.15.103:0/3518971207 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.619+0000 7fb4abfff640 1 --2- 172.21.15.103:0/3518971207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff62410 0x55c08ff6a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.619+0000 7fb4b0d74640 1 --2- 172.21.15.103:0/3518971207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff61af0 0x55c08ff69c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb4b1575640 1 -- 172.21.15.103:0/3518971207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff62410 msgr2=0x55c08ff6a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb4b1575640 1 --2- 172.21.15.103:0/3518971207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff62410 0x55c08ff6a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb4b1575640 1 -- 172.21.15.103:0/3518971207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff61af0 msgr2=0x55c08ff69c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb4b1575640 1 --2- 172.21.15.103:0/3518971207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff61af0 0x55c08ff69c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb4b1575640 1 -- 172.21.15.103:0/3518971207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c08fbd9210 con 0x55c08ff6ccb0 2024-09-19T07:05:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb4b1575640 1 --2- 172.21.15.103:0/3518971207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff6ccb0 0x55c08ff6a690 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40cd170 tx=0x7fb4a409b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5a2a8789d7684b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb48bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb48bfff640 1 -- 172.21.15.103:0/3518971207 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4020070 con 0x55c08ff6ccb0 2024-09-19T07:05:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb48bfff640 1 -- 172.21.15.103:0/3518971207 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fb4a4098310 con 0x55c08ff6ccb0 2024-09-19T07:05:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb4b5e49740 1 -- 172.21.15.103:0/3518971207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb498005500 con 0x55c08ff6ccb0 2024-09-19T07:05:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb48bfff640 1 -- 172.21.15.103:0/3518971207 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb4a40a0050 con 0x55c08ff6ccb0 2024-09-19T07:05:50.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb4b5e49740 10 client.21315 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:50.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb4b5e49740 1 -- 172.21.15.103:0/3518971207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c08fca5620 con 0x55c08ff6ccb0 2024-09-19T07:05:50.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb48bfff640 1 -- 172.21.15.103:0/3518971207 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb498005500 con 0x55c08ff6ccb0 2024-09-19T07:05:50.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb48bfff640 1 -- 172.21.15.103:0/3518971207 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb4a40860b0 con 0x55c08ff6ccb0 2024-09-19T07:05:50.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb4b5e49740 10 client.21315 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:50.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb4b5e49740 10 client.21315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:50.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb4b5e49740 20 client.21315 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:50.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/3518971207 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c08ff7cd70 0x55c08ff7f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb4b5e49740 4 client.21315 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:50.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.620+0000 7fb4b5e49740 1 -- 172.21.15.103:0/3518971207 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c08fed3a20 con 0x55c08ff7cd70 2024-09-19T07:05:50.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.621+0000 7fb4b0d74640 1 --2- 172.21.15.103:0/3518971207 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c08ff7cd70 0x55c08ff7f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.621+0000 7fb48bfff640 10 client.21315 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:50.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.621+0000 7fb4b0d74640 1 --2- 172.21.15.103:0/3518971207 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c08ff7cd70 0x55c08ff7f160 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24c39600b68806c server_cookie=385a1854647f0307 in_seq=0 out_seq=0 2024-09-19T07:05:50.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.622+0000 7fb48bfff640 1 -- 172.21.15.103:0/3518971207 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c08fed3a20 con 0x55c08ff7cd70 2024-09-19T07:05:50.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.622+0000 7fb48bfff640 10 client.21315 handle_command_reply: tid=0 2024-09-19T07:05:50.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.622+0000 7fb4b5e49740 1 client.21315 shutdown 2024-09-19T07:05:50.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.622+0000 7fb4b5e49740 1 -- 172.21.15.103:0/3518971207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff6ccb0 msgr2=0x55c08ff6a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:50.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.622+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/3518971207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff6ccb0 0x55c08ff6a690 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40cd170 tx=0x7fb4a409b8d0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.622+0000 7fb4b5e49740 1 -- 172.21.15.103:0/3518971207 shutdown_connections 2024-09-19T07:05:50.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.622+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/3518971207 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c08ff7cd70 0x55c08ff7f160 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.622+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/3518971207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff6ccb0 0x55c08ff6a690 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.622+0000 7fb48bfff640 0 client.21315 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:50.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.622+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/3518971207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff62410 0x55c08ff6a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.622+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/3518971207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff61af0 0x55c08ff69c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.622+0000 7fb4b5e49740 1 -- 172.21.15.103:0/3518971207 >> 172.21.15.103:0/3518971207 conn(0x55c08ff5e060 msgr2=0x55c08ff5d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:50.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.622+0000 7fb4b5e49740 1 -- 172.21.15.103:0/3518971207 shutdown_connections 2024-09-19T07:05:50.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.622+0000 7fb4b5e49740 1 -- 172.21.15.103:0/3518971207 wait complete. 2024-09-19T07:05:50.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.623+0000 7fb4b5e49740 20 client.21315 trim_cache size 0 max 16384 2024-09-19T07:05:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4b5e49740 1 Processor -- start 2024-09-19T07:05:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4b5e49740 1 -- start start 2024-09-19T07:05:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4b5e49740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff61af0 0x55c08ff6a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4b5e49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff62410 0x55c08ff84a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4b5e49740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff6ccb0 0x55c08ff84fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4b5e49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1ad840 con 0x55c08ff62410 2024-09-19T07:05:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4b5e49740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1af380 con 0x55c08ff61af0 2024-09-19T07:05:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4b5e49740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1adf10 con 0x55c08ff6ccb0 2024-09-19T07:05:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4abfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff62410 0x55c08ff84a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4abfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff62410 0x55c08ff84a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57596/0 (socket says 172.21.15.103:57596) 2024-09-19T07:05:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4abfff640 1 -- 172.21.15.103:0/175675884 learned_addr learned my addr 172.21.15.103:0/175675884 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4b1575640 1 --2- 172.21.15.103:0/175675884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff6ccb0 0x55c08ff84fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4b0d74640 1 --2- 172.21.15.103:0/175675884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff61af0 0x55c08ff6a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4abfff640 1 -- 172.21.15.103:0/175675884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff6ccb0 msgr2=0x55c08ff84fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4abfff640 1 --2- 172.21.15.103:0/175675884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff6ccb0 0x55c08ff84fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4abfff640 1 -- 172.21.15.103:0/175675884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff61af0 msgr2=0x55c08ff6a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4abfff640 1 --2- 172.21.15.103:0/175675884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff61af0 0x55c08ff6a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4abfff640 1 -- 172.21.15.103:0/175675884 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c08ff68ba0 con 0x55c08ff62410 2024-09-19T07:05:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4b1575640 1 --2- 172.21.15.103:0/175675884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff6ccb0 0x55c08ff84fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.637+0000 7fb4b0d74640 1 --2- 172.21.15.103:0/175675884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff61af0 0x55c08ff6a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.638+0000 7fb4abfff640 1 --2- 172.21.15.103:0/175675884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff62410 0x55c08ff84a70 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fb498011c20 tx=0x7fb498014720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0b5e5b4094e914 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.638+0000 7fb4967fc640 1 -- 172.21.15.103:0/175675884 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498022040 con 0x55c08ff62410 2024-09-19T07:05:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.638+0000 7fb4967fc640 1 -- 172.21.15.103:0/175675884 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49802a070 con 0x55c08ff62410 2024-09-19T07:05:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.638+0000 7fb4967fc640 1 -- 172.21.15.103:0/175675884 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498004680 con 0x55c08ff62410 2024-09-19T07:05:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.638+0000 7fb4b5e49740 1 -- 172.21.15.103:0/175675884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff62410 msgr2=0x55c08ff84a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.638+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/175675884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff62410 0x55c08ff84a70 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fb498011c20 tx=0x7fb498014720 comp rx=0 tx=0).stop 2024-09-19T07:05:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.638+0000 7fb4b5e49740 1 -- 172.21.15.103:0/175675884 shutdown_connections 2024-09-19T07:05:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.638+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/175675884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff6ccb0 0x55c08ff84fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.638+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/175675884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff62410 0x55c08ff84a70 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.638+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/175675884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff61af0 0x55c08ff6a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.638+0000 7fb4b5e49740 1 -- 172.21.15.103:0/175675884 >> 172.21.15.103:0/175675884 conn(0x55c08ff5e060 msgr2=0x55c08ff6bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:50.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.638+0000 7fb4b5e49740 1 -- 172.21.15.103:0/175675884 shutdown_connections 2024-09-19T07:05:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.638+0000 7fb4b5e49740 1 -- 172.21.15.103:0/175675884 wait complete. 2024-09-19T07:05:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.639+0000 7fb4b5e49740 1 -- start start 2024-09-19T07:05:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.639+0000 7fb4b5e49740 1 Processor -- start 2024-09-19T07:05:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.639+0000 7fb4b5e49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff61af0 0x55c08ff709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.639+0000 7fb4b5e49740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff62410 0x55c08ff70f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.639+0000 7fb4b5e49740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff6ccb0 0x55c08ff71460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.639+0000 7fb4b5e49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1aecb0 con 0x55c08ff61af0 2024-09-19T07:05:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.639+0000 7fb4b5e49740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1ae5e0 con 0x55c08ff62410 2024-09-19T07:05:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.639+0000 7fb4b5e49740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1ab840 con 0x55c08ff6ccb0 2024-09-19T07:05:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.639+0000 7fb4b5e49740 10 client.0 _ll_register_callbacks cb 0x7fb4a4097f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.639+0000 7fb4b0d74640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff61af0 0x55c08ff709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.639+0000 7fb4b0d74640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff61af0 0x55c08ff709e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57604/0 (socket says 172.21.15.103:57604) 2024-09-19T07:05:50.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.639+0000 7fb4b0d74640 1 -- 172.21.15.103:0/2429382511 learned_addr learned my addr 172.21.15.103:0/2429382511 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.639+0000 7fb4b1575640 1 --2- 172.21.15.103:0/2429382511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff6ccb0 0x55c08ff71460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.639+0000 7fb4abfff640 1 --2- 172.21.15.103:0/2429382511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff62410 0x55c08ff70f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb4b0d74640 1 -- 172.21.15.103:0/2429382511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff6ccb0 msgr2=0x55c08ff71460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb4b0d74640 1 --2- 172.21.15.103:0/2429382511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff6ccb0 0x55c08ff71460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb4b0d74640 1 -- 172.21.15.103:0/2429382511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff62410 msgr2=0x55c08ff70f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb4b0d74640 1 --2- 172.21.15.103:0/2429382511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff62410 0x55c08ff70f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb4b0d74640 1 -- 172.21.15.103:0/2429382511 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c08ff62240 con 0x55c08ff61af0 2024-09-19T07:05:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb4abfff640 1 --2- 172.21.15.103:0/2429382511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff62410 0x55c08ff70f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb4b0d74640 1 --2- 172.21.15.103:0/2429382511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff61af0 0x55c08ff709e0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fb4a002a9a0 tx=0x7fb4a0031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aedca0e17224161 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb48bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb48bfff640 1 -- 172.21.15.103:0/2429382511 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a004a030 con 0x55c08ff61af0 2024-09-19T07:05:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb48bfff640 1 -- 172.21.15.103:0/2429382511 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fb4a0004250 con 0x55c08ff61af0 2024-09-19T07:05:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb4b5e49740 1 -- 172.21.15.103:0/2429382511 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb49802a070 con 0x55c08ff61af0 2024-09-19T07:05:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb48bfff640 1 -- 172.21.15.103:0/2429382511 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb4a0056050 con 0x55c08ff61af0 2024-09-19T07:05:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb4b5e49740 10 client.21321 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb4b5e49740 1 -- 172.21.15.103:0/2429382511 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c08ffe7ab0 con 0x55c08ff61af0 2024-09-19T07:05:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb48bfff640 1 -- 172.21.15.103:0/2429382511 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49802a070 con 0x55c08ff61af0 2024-09-19T07:05:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb48bfff640 1 -- 172.21.15.103:0/2429382511 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb4a003d750 con 0x55c08ff61af0 2024-09-19T07:05:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb4b5e49740 10 client.21321 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb4b5e49740 10 client.21321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb4b5e49740 20 client.21321 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/2429382511 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c08ffe8210 0x55c08ffea630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb4b5e49740 4 client.21321 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:05:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.640+0000 7fb4b5e49740 1 -- 172.21.15.103:0/2429382511 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c08fecdd70 con 0x55c08ffe8210 2024-09-19T07:05:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.641+0000 7fb4abfff640 1 --2- 172.21.15.103:0/2429382511 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c08ffe8210 0x55c08ffea630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.641+0000 7fb48bfff640 10 client.21321 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.641+0000 7fb4abfff640 1 --2- 172.21.15.103:0/2429382511 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c08ffe8210 0x55c08ffea630 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0e8e498104f42b7 server_cookie=ec6b019a18c40928 in_seq=0 out_seq=0 2024-09-19T07:05:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.653+0000 7fb48bfff640 1 -- 172.21.15.103:0/2429382511 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c08fecdd70 con 0x55c08ffe8210 2024-09-19T07:05:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.653+0000 7fb48bfff640 10 client.21321 handle_command_reply: tid=0 2024-09-19T07:05:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.653+0000 7fb4b5e49740 1 client.21321 shutdown 2024-09-19T07:05:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.653+0000 7fb4b5e49740 1 -- 172.21.15.103:0/2429382511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff61af0 msgr2=0x55c08ff709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.653+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/2429382511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff61af0 0x55c08ff709e0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fb4a002a9a0 tx=0x7fb4a0031a60 comp rx=0 tx=0).stop 2024-09-19T07:05:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.653+0000 7fb4b5e49740 1 -- 172.21.15.103:0/2429382511 shutdown_connections 2024-09-19T07:05:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.653+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/2429382511 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55c08ffe8210 0x55c08ffea630 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.653+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/2429382511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c08ff6ccb0 0x55c08ff71460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.653+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/2429382511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c08ff62410 0x55c08ff70f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.653+0000 7fb4b5e49740 1 --2- 172.21.15.103:0/2429382511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c08ff61af0 0x55c08ff709e0 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.653+0000 7fb4b5e49740 1 -- 172.21.15.103:0/2429382511 >> 172.21.15.103:0/2429382511 conn(0x55c08ff5e060 msgr2=0x55c08ff5c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.653+0000 7fb48bfff640 0 client.21321 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:50.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.653+0000 7fb4b5e49740 1 -- 172.21.15.103:0/2429382511 shutdown_connections 2024-09-19T07:05:50.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.653+0000 7fb4b5e49740 1 -- 172.21.15.103:0/2429382511 wait complete. 2024-09-19T07:05:50.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.653+0000 7fb4b5e49740 20 client.21321 trim_cache size 0 max 16384 2024-09-19T07:05:50.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:05:50.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:05:50.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:05:50.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:05:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.654+0000 7fb4b2fd8640 1 -- 172.21.15.103:0/1540240354 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb49c0a34e0 msgr2=0x7fb49c0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.654+0000 7fb4b2fd8640 1 --2- 172.21.15.103:0/1540240354 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb49c0a34e0 0x7fb49c0a59a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb498014a10 tx=0x7fb49800bc90 comp rx=0 tx=0).stop 2024-09-19T07:05:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.654+0000 7fb4b2fd8640 1 -- 172.21.15.103:0/1540240354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4ac10c110 msgr2=0x7fb4ac19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.654+0000 7fb4b2fd8640 1 --2- 172.21.15.103:0/1540240354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4ac10c110 0x7fb4ac19cca0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009fd0 tx=0x7fb4a400b260 comp rx=0 tx=0).stop 2024-09-19T07:05:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.654+0000 7fb4b2fd8640 1 -- 172.21.15.103:0/1540240354 shutdown_connections 2024-09-19T07:05:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.654+0000 7fb4b2fd8640 1 --2- 172.21.15.103:0/1540240354 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fb49c0a34e0 0x7fb49c0a59a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.654+0000 7fb4b2fd8640 1 --2- 172.21.15.103:0/1540240354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4ac10c110 0x7fb4ac19cca0 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.654+0000 7fb4b2fd8640 1 --2- 172.21.15.103:0/1540240354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4ac1035e0 0x7fb4ac198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.654+0000 7fb4b2fd8640 1 --2- 172.21.15.103:0/1540240354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4ac102cc0 0x7fb4ac1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.654+0000 7fb4b2fd8640 1 -- 172.21.15.103:0/1540240354 >> 172.21.15.103:0/1540240354 conn(0x7fb4ac0fc6b0 msgr2=0x7fb4ac0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.654+0000 7fb4b2fd8640 1 -- 172.21.15.103:0/1540240354 shutdown_connections 2024-09-19T07:05:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:05:50.654+0000 7fb4b2fd8640 1 -- 172.21.15.103:0/1540240354 wait complete. 2024-09-19T07:05:51.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:51 smithi174 ceph-mon[35637]: pgmap v519: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-09-19T07:05:51.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:51 smithi103 ceph-mon[31158]: pgmap v519: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-09-19T07:05:51.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:51 smithi134 ceph-mon[35356]: pgmap v519: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-09-19T07:05:53.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:53 smithi174 ceph-mon[35637]: pgmap v520: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-19T07:05:53.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:53 smithi103 ceph-mon[31158]: pgmap v520: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-19T07:05:53.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:53 smithi134 ceph-mon[35356]: pgmap v520: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-19T07:05:54.389 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:05:54.650 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:55.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.128+0000 7f62c4361640 1 -- 172.21.15.103:0/989504145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62bc100a80 msgr2=0x7f62bc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:55.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.128+0000 7f62c4361640 1 --2- 172.21.15.103:0/989504145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62bc100a80 0x7f62bc100ee0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009930 tx=0x7f62ac02a420 comp rx=0 tx=0).stop 2024-09-19T07:05:55.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.128+0000 7f62c4361640 1 -- 172.21.15.103:0/989504145 shutdown_connections 2024-09-19T07:05:55.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.128+0000 7f62c4361640 1 --2- 172.21.15.103:0/989504145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62bc101420 0x7f62bc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.128+0000 7f62c4361640 1 --2- 172.21.15.103:0/989504145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62bc100a80 0x7f62bc100ee0 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.128+0000 7f62c4361640 1 --2- 172.21.15.103:0/989504145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62bc106a80 0x7f62bc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.128+0000 7f62c4361640 1 -- 172.21.15.103:0/989504145 >> 172.21.15.103:0/989504145 conn(0x7f62bc0fc7a0 msgr2=0x7f62bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:55.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.128+0000 7f62c4361640 1 -- 172.21.15.103:0/989504145 shutdown_connections 2024-09-19T07:05:55.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.128+0000 7f62c4361640 1 -- 172.21.15.103:0/989504145 wait complete. 2024-09-19T07:05:55.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.129+0000 7f62c4361640 1 Processor -- start 2024-09-19T07:05:55.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.129+0000 7f62c4361640 1 -- start start 2024-09-19T07:05:55.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.129+0000 7f62c4361640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62bc100a80 0x7f62bc10e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.129+0000 7f62c4361640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62bc101420 0x7f62bc10e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.129+0000 7f62c4361640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62bc106a80 0x7f62bc10ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.129+0000 7f62c4361640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc111f90 con 0x7f62bc101420 2024-09-19T07:05:55.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.129+0000 7f62c4361640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc111e10 con 0x7f62bc100a80 2024-09-19T07:05:55.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.129+0000 7f62c4361640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc112110 con 0x7f62bc106a80 2024-09-19T07:05:55.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.129+0000 7f62c18fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62bc101420 0x7f62bc10e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.129+0000 7f62c18fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62bc101420 0x7f62bc10e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57620/0 (socket says 172.21.15.103:57620) 2024-09-19T07:05:55.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.129+0000 7f62c18fc640 1 -- 172.21.15.103:0/1589122140 learned_addr learned my addr 172.21.15.103:0/1589122140 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:55.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.130+0000 7f62c28fe640 1 --2- 172.21.15.103:0/1589122140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62bc106a80 0x7f62bc10ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.130+0000 7f62c20fd640 1 --2- 172.21.15.103:0/1589122140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62bc100a80 0x7f62bc10e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.130+0000 7f62c18fc640 1 -- 172.21.15.103:0/1589122140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62bc106a80 msgr2=0x7f62bc10ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:55.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.130+0000 7f62c18fc640 1 --2- 172.21.15.103:0/1589122140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62bc106a80 0x7f62bc10ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.130+0000 7f62c18fc640 1 -- 172.21.15.103:0/1589122140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62bc100a80 msgr2=0x7f62bc10e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:55.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.130+0000 7f62c18fc640 1 --2- 172.21.15.103:0/1589122140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62bc100a80 0x7f62bc10e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.130+0000 7f62c18fc640 1 -- 172.21.15.103:0/1589122140 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62bc1a6930 con 0x7f62bc101420 2024-09-19T07:05:55.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.130+0000 7f62c18fc640 1 --2- 172.21.15.103:0/1589122140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62bc101420 0x7f62bc10e7a0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009230 tx=0x7f62ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:55.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.130+0000 7f62b37fe640 1 -- 172.21.15.103:0/1589122140 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac03f030 con 0x7f62bc101420 2024-09-19T07:05:55.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.130+0000 7f62b37fe640 1 -- 172.21.15.103:0/1589122140 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62ac003ad0 con 0x7f62bc101420 2024-09-19T07:05:55.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.130+0000 7f62c4361640 1 -- 172.21.15.103:0/1589122140 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62bc1a6bc0 con 0x7f62bc101420 2024-09-19T07:05:55.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.130+0000 7f62b37fe640 1 -- 172.21.15.103:0/1589122140 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac035070 con 0x7f62bc101420 2024-09-19T07:05:55.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.130+0000 7f62c4361640 1 -- 172.21.15.103:0/1589122140 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62bc1a70f0 con 0x7f62bc101420 2024-09-19T07:05:55.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.132+0000 7f62b37fe640 1 -- 172.21.15.103:0/1589122140 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f62ac042030 con 0x7f62bc101420 2024-09-19T07:05:55.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.133+0000 7f62c673a740 1 Processor -- start 2024-09-19T07:05:55.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.133+0000 7f62c673a740 1 -- start start 2024-09-19T07:05:55.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.133+0000 7f62b37fe640 1 --2- 172.21.15.103:0/1589122140 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62a80a3430 0x7f62a80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.133+0000 7f62c673a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 0x559b36056880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.133+0000 7f62c673a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36056dc0 0x559b36061140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.133+0000 7f62c673a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 0x559b36063a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.133+0000 7f62c673a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559b35fbc2e0 con 0x559b36061680 2024-09-19T07:05:55.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.133+0000 7f62c673a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559b35f63ce0 con 0x559b360564a0 2024-09-19T07:05:55.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.133+0000 7f62c673a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559b35fb82a0 con 0x559b36056dc0 2024-09-19T07:05:55.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.133+0000 7f62c28fe640 1 --2- 172.21.15.103:0/1589122140 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62a80a3430 0x7f62a80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.133+0000 7f62b37fe640 1 -- 172.21.15.103:0/1589122140 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f62ac0f0240 con 0x7f62bc101420 2024-09-19T07:05:55.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.133+0000 7f62c20fd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 0x559b36056880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.133+0000 7f62c20fd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 0x559b36056880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:59926/0 (socket says 172.21.15.103:59926) 2024-09-19T07:05:55.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.133+0000 7f62c20fd640 1 -- 172.21.15.103:0/4089969944 learned_addr learned my addr 172.21.15.103:0/4089969944 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:55.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.133+0000 7f62c28fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 0x559b36063a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.133+0000 7f62c18fc640 1 --2- 172.21.15.103:0/4089969944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36056dc0 0x559b36061140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.134+0000 7f62c20fd640 1 -- 172.21.15.103:0/4089969944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36056dc0 msgr2=0x559b36061140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:55.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.134+0000 7f62c20fd640 1 --2- 172.21.15.103:0/4089969944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36056dc0 0x559b36061140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.134+0000 7f62c20fd640 1 -- 172.21.15.103:0/4089969944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 msgr2=0x559b36063a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:55.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.134+0000 7f62c20fd640 1 --2- 172.21.15.103:0/4089969944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 0x559b36063a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.134+0000 7f62c20fd640 1 -- 172.21.15.103:0/4089969944 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b35f48440 con 0x559b360564a0 2024-09-19T07:05:55.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.134+0000 7f62c18fc640 1 --2- 172.21.15.103:0/4089969944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36056dc0 0x559b36061140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:55.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.134+0000 7f62c28fe640 1 --2- 172.21.15.103:0/1589122140 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62a80a3430 0x7f62a80a58f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009270 tx=0x7f62b4009c90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:55.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.134+0000 7f62c20fd640 1 --2- 172.21.15.103:0/4089969944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 0x559b36056880 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009b30 tx=0x7f62b800e2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14c5b36dabf73d1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:55.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.134+0000 7f62b0dea640 1 -- 172.21.15.103:0/4089969944 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8016070 con 0x559b360564a0 2024-09-19T07:05:55.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.134+0000 7f62b0dea640 1 -- 172.21.15.103:0/4089969944 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62b8005500 con 0x559b360564a0 2024-09-19T07:05:55.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.134+0000 7f62b0dea640 1 -- 172.21.15.103:0/4089969944 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8011470 con 0x559b360564a0 2024-09-19T07:05:55.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.135+0000 7f62c673a740 1 -- 172.21.15.103:0/4089969944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 msgr2=0x559b36056880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:55.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.135+0000 7f62c673a740 1 --2- 172.21.15.103:0/4089969944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 0x559b36056880 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009b30 tx=0x7f62b800e2f0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.135+0000 7f62c673a740 1 -- 172.21.15.103:0/4089969944 shutdown_connections 2024-09-19T07:05:55.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.135+0000 7f62c673a740 1 --2- 172.21.15.103:0/4089969944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 0x559b36063a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.135+0000 7f62c673a740 1 --2- 172.21.15.103:0/4089969944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36056dc0 0x559b36061140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.135+0000 7f62c673a740 1 --2- 172.21.15.103:0/4089969944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 0x559b36056880 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.135+0000 7f62c673a740 1 -- 172.21.15.103:0/4089969944 >> 172.21.15.103:0/4089969944 conn(0x559b36052a30 msgr2=0x559b36052e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:55.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.135+0000 7f62c673a740 1 -- 172.21.15.103:0/4089969944 shutdown_connections 2024-09-19T07:05:55.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.135+0000 7f62c673a740 1 -- 172.21.15.103:0/4089969944 wait complete. 2024-09-19T07:05:55.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.135+0000 7f62c673a740 1 -- start start 2024-09-19T07:05:55.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.135+0000 7f62c673a740 1 Processor -- start 2024-09-19T07:05:55.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c673a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b360564a0 0x559b36060f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c673a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b36056dc0 0x559b3605e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c673a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 0x559b3605e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c673a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559b36064250 con 0x559b36061680 2024-09-19T07:05:55.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c673a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559b35f77050 con 0x559b36056dc0 2024-09-19T07:05:55.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c673a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559b360643d0 con 0x559b360564a0 2024-09-19T07:05:55.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c673a740 10 client.0 _ll_register_callbacks cb 0x559b35fb7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:55.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c28fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 0x559b3605e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c28fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 0x559b3605e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57646/0 (socket says 172.21.15.103:57646) 2024-09-19T07:05:55.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c28fe640 1 -- 172.21.15.103:0/436217217 learned_addr learned my addr 172.21.15.103:0/436217217 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:55.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c18fc640 1 --2- 172.21.15.103:0/436217217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b36056dc0 0x559b3605e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c20fd640 1 --2- 172.21.15.103:0/436217217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b360564a0 0x559b36060f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c28fe640 1 -- 172.21.15.103:0/436217217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b360564a0 msgr2=0x559b36060f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:55.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c28fe640 1 --2- 172.21.15.103:0/436217217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b360564a0 0x559b36060f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c28fe640 1 -- 172.21.15.103:0/436217217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b36056dc0 msgr2=0x559b3605e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:55.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c28fe640 1 --2- 172.21.15.103:0/436217217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b36056dc0 0x559b3605e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c28fe640 1 -- 172.21.15.103:0/436217217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b35d2d3c0 con 0x559b36061680 2024-09-19T07:05:55.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62c28fe640 1 --2- 172.21.15.103:0/436217217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 0x559b3605e780 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f62b4014310 tx=0x7f62b401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d7624d7593278d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:55.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.136+0000 7f62a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:55.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.137+0000 7f62a5ffb640 1 -- 172.21.15.103:0/436217217 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4025030 con 0x559b36061680 2024-09-19T07:05:55.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.137+0000 7f62a5ffb640 1 -- 172.21.15.103:0/436217217 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f62b4012970 con 0x559b36061680 2024-09-19T07:05:55.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.137+0000 7f62c673a740 1 -- 172.21.15.103:0/436217217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62b8005500 con 0x559b36061680 2024-09-19T07:05:55.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.137+0000 7f62a5ffb640 1 -- 172.21.15.103:0/436217217 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f62b4011ea0 con 0x559b36061680 2024-09-19T07:05:55.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.137+0000 7f62c673a740 10 client.21333 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:55.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.137+0000 7f62c673a740 1 -- 172.21.15.103:0/436217217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559b3606fc30 con 0x559b36061680 2024-09-19T07:05:55.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.137+0000 7f62a5ffb640 1 -- 172.21.15.103:0/436217217 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62b8005500 con 0x559b36061680 2024-09-19T07:05:55.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.137+0000 7f62a5ffb640 1 -- 172.21.15.103:0/436217217 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f62b400f050 con 0x559b36061680 2024-09-19T07:05:55.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.137+0000 7f62c673a740 10 client.21333 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:55.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.137+0000 7f62c673a740 10 client.21333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:55.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.137+0000 7f62c673a740 20 client.21333 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:55.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.137+0000 7f62c673a740 1 --2- 172.21.15.103:0/436217217 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559b36071040 0x559b360734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.137+0000 7f62c673a740 4 client.21333 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:55.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.137+0000 7f62c673a740 1 -- 172.21.15.103:0/436217217 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b35fc9850 con 0x559b36071040 2024-09-19T07:05:55.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.138+0000 7f62c20fd640 1 --2- 172.21.15.103:0/436217217 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559b36071040 0x559b360734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.138+0000 7f62c20fd640 1 --2- 172.21.15.103:0/436217217 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559b36071040 0x559b360734e0 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e271ae37a11f9e8 server_cookie=37738ba7a7de12be in_seq=0 out_seq=0 2024-09-19T07:05:55.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.138+0000 7f62a5ffb640 10 client.21333 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:55.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.139+0000 7f62a5ffb640 1 -- 172.21.15.103:0/436217217 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b35fc9850 con 0x559b36071040 2024-09-19T07:05:55.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.139+0000 7f62a5ffb640 10 client.21333 handle_command_reply: tid=0 2024-09-19T07:05:55.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.139+0000 7f62c673a740 1 client.21333 shutdown 2024-09-19T07:05:55.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.139+0000 7f62c673a740 1 -- 172.21.15.103:0/436217217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 msgr2=0x559b3605e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:55.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.139+0000 7f62c673a740 1 --2- 172.21.15.103:0/436217217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 0x559b3605e780 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f62b4014310 tx=0x7f62b401b000 comp rx=0 tx=0).stop 2024-09-19T07:05:55.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.139+0000 7f62c673a740 1 -- 172.21.15.103:0/436217217 shutdown_connections 2024-09-19T07:05:55.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.139+0000 7f62c673a740 1 --2- 172.21.15.103:0/436217217 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559b36071040 0x559b360734e0 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.139+0000 7f62a5ffb640 0 client.21333 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:55.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.139+0000 7f62c673a740 1 --2- 172.21.15.103:0/436217217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 0x559b3605e780 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.139+0000 7f62c673a740 1 --2- 172.21.15.103:0/436217217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b36056dc0 0x559b3605e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.139+0000 7f62c673a740 1 --2- 172.21.15.103:0/436217217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b360564a0 0x559b36060f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.139+0000 7f62c673a740 1 -- 172.21.15.103:0/436217217 >> 172.21.15.103:0/436217217 conn(0x559b36052a30 msgr2=0x559b36058710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:55.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.139+0000 7f62c673a740 1 -- 172.21.15.103:0/436217217 shutdown_connections 2024-09-19T07:05:55.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.139+0000 7f62c673a740 1 -- 172.21.15.103:0/436217217 wait complete. 2024-09-19T07:05:55.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.139+0000 7f62c673a740 20 client.21333 trim_cache size 0 max 16384 2024-09-19T07:05:55.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.156+0000 7f62c673a740 1 Processor -- start 2024-09-19T07:05:55.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.156+0000 7f62c673a740 1 -- start start 2024-09-19T07:05:55.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.156+0000 7f62c673a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 0x559b3606d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.156+0000 7f62c673a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36056dc0 0x559b36051a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.156+0000 7f62c673a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36061680 0x559b36069680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.156+0000 7f62c673a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc1afb40 con 0x559b36056dc0 2024-09-19T07:05:55.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.156+0000 7f62c673a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc1b1680 con 0x559b360564a0 2024-09-19T07:05:55.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.156+0000 7f62c673a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc1b0210 con 0x559b36061680 2024-09-19T07:05:55.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.156+0000 7f62c18fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36056dc0 0x559b36051a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.156+0000 7f62c18fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36056dc0 0x559b36051a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57658/0 (socket says 172.21.15.103:57658) 2024-09-19T07:05:55.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.156+0000 7f62c18fc640 1 -- 172.21.15.103:0/996917519 learned_addr learned my addr 172.21.15.103:0/996917519 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:55.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.157+0000 7f62c20fd640 1 --2- 172.21.15.103:0/996917519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 0x559b3606d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.157+0000 7f62c18fc640 1 -- 172.21.15.103:0/996917519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36061680 msgr2=0x559b36069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:55.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.157+0000 7f62c28fe640 1 --2- 172.21.15.103:0/996917519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36061680 0x559b36069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.157+0000 7f62c18fc640 1 --2- 172.21.15.103:0/996917519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36061680 0x559b36069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.157+0000 7f62c18fc640 1 -- 172.21.15.103:0/996917519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 msgr2=0x559b3606d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:55.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.157+0000 7f62c18fc640 1 --2- 172.21.15.103:0/996917519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 0x559b3606d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.157+0000 7f62c18fc640 1 -- 172.21.15.103:0/996917519 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b36060930 con 0x559b36056dc0 2024-09-19T07:05:55.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.157+0000 7f62c20fd640 1 --2- 172.21.15.103:0/996917519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 0x559b3606d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:55.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.157+0000 7f62c28fe640 1 --2- 172.21.15.103:0/996917519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36061680 0x559b36069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:55.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.157+0000 7f62c18fc640 1 --2- 172.21.15.103:0/996917519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36056dc0 0x559b36051a60 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009a50 tx=0x7f62ac0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a23612a28f6047 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:55.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.157+0000 7f62a7fff640 1 -- 172.21.15.103:0/996917519 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac049040 con 0x559b36056dc0 2024-09-19T07:05:55.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.157+0000 7f62a7fff640 1 -- 172.21.15.103:0/996917519 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62ac0bce90 con 0x559b36056dc0 2024-09-19T07:05:55.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.157+0000 7f62a7fff640 1 -- 172.21.15.103:0/996917519 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0c53c0 con 0x559b36056dc0 2024-09-19T07:05:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 -- 172.21.15.103:0/996917519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36056dc0 msgr2=0x559b36051a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 --2- 172.21.15.103:0/996917519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36056dc0 0x559b36051a60 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009a50 tx=0x7f62ac0045f0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 -- 172.21.15.103:0/996917519 shutdown_connections 2024-09-19T07:05:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 --2- 172.21.15.103:0/996917519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36061680 0x559b36069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 --2- 172.21.15.103:0/996917519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36056dc0 0x559b36051a60 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 --2- 172.21.15.103:0/996917519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 0x559b3606d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 -- 172.21.15.103:0/996917519 >> 172.21.15.103:0/996917519 conn(0x559b36052a30 msgr2=0x559b36051350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 -- 172.21.15.103:0/996917519 shutdown_connections 2024-09-19T07:05:55.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 -- 172.21.15.103:0/996917519 wait complete. 2024-09-19T07:05:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 -- start start 2024-09-19T07:05:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 Processor -- start 2024-09-19T07:05:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 0x559b36067ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36056dc0 0x559b36068020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 0x559b360dade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc1b0fb0 con 0x559b36061680 2024-09-19T07:05:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc1b08e0 con 0x559b360564a0 2024-09-19T07:05:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc1adb40 con 0x559b36056dc0 2024-09-19T07:05:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.158+0000 7f62c673a740 10 client.0 _ll_register_callbacks cb 0x7f62b4009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62c28fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 0x559b360dade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62c28fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 0x559b360dade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57660/0 (socket says 172.21.15.103:57660) 2024-09-19T07:05:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62c28fe640 1 -- 172.21.15.103:0/1314767293 learned_addr learned my addr 172.21.15.103:0/1314767293 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62c18fc640 1 --2- 172.21.15.103:0/1314767293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36056dc0 0x559b36068020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62c20fd640 1 --2- 172.21.15.103:0/1314767293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 0x559b36067ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62c28fe640 1 -- 172.21.15.103:0/1314767293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36056dc0 msgr2=0x559b36068020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62c28fe640 1 --2- 172.21.15.103:0/1314767293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36056dc0 0x559b36068020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62c28fe640 1 -- 172.21.15.103:0/1314767293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 msgr2=0x559b36067ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62c28fe640 1 --2- 172.21.15.103:0/1314767293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 0x559b36067ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62c28fe640 1 -- 172.21.15.103:0/1314767293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b360db320 con 0x559b36061680 2024-09-19T07:05:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62c20fd640 1 --2- 172.21.15.103:0/1314767293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 0x559b36067ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62c28fe640 1 --2- 172.21.15.103:0/1314767293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 0x559b360dade0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f62b4011770 tx=0x7f62b4011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8224154295517dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62a5ffb640 1 -- 172.21.15.103:0/1314767293 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4036030 con 0x559b36061680 2024-09-19T07:05:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62a5ffb640 1 -- 172.21.15.103:0/1314767293 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f62b402c9f0 con 0x559b36061680 2024-09-19T07:05:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62c673a740 1 -- 172.21.15.103:0/1314767293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62ac0bce90 con 0x559b36061680 2024-09-19T07:05:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62a5ffb640 1 -- 172.21.15.103:0/1314767293 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f62b4041050 con 0x559b36061680 2024-09-19T07:05:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62c673a740 10 client.21339 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:55.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.159+0000 7f62c673a740 1 -- 172.21.15.103:0/1314767293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559b360dc540 con 0x559b36061680 2024-09-19T07:05:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.160+0000 7f62a5ffb640 1 -- 172.21.15.103:0/1314767293 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62ac0bce90 con 0x559b36061680 2024-09-19T07:05:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.160+0000 7f62a5ffb640 1 -- 172.21.15.103:0/1314767293 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f62b400f030 con 0x559b36061680 2024-09-19T07:05:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.160+0000 7f62c673a740 10 client.21339 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.160+0000 7f62c673a740 10 client.21339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:05:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.160+0000 7f62c673a740 20 client.21339 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.160+0000 7f62c673a740 1 --2- 172.21.15.103:0/1314767293 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559b36071040 0x559b360decf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.160+0000 7f62c673a740 4 client.21339 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:05:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.160+0000 7f62c673a740 1 -- 172.21.15.103:0/1314767293 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559b35fc3d60 con 0x559b36071040 2024-09-19T07:05:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.160+0000 7f62c20fd640 1 --2- 172.21.15.103:0/1314767293 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559b36071040 0x559b360decf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62a5ffb640 10 client.21339 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c20fd640 1 --2- 172.21.15.103:0/1314767293 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559b36071040 0x559b360decf0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef4af65887bf4d5 server_cookie=be66a6b910ffee9e in_seq=0 out_seq=0 2024-09-19T07:05:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62a5ffb640 1 -- 172.21.15.103:0/1314767293 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559b35fc3d60 con 0x559b36071040 2024-09-19T07:05:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62a5ffb640 10 client.21339 handle_command_reply: tid=0 2024-09-19T07:05:55.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c673a740 1 client.21339 shutdown 2024-09-19T07:05:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c673a740 1 -- 172.21.15.103:0/1314767293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 msgr2=0x559b360dade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c673a740 1 --2- 172.21.15.103:0/1314767293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 0x559b360dade0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f62b4011770 tx=0x7f62b4011870 comp rx=0 tx=0).stop 2024-09-19T07:05:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c673a740 1 -- 172.21.15.103:0/1314767293 shutdown_connections 2024-09-19T07:05:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c673a740 1 --2- 172.21.15.103:0/1314767293 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x559b36071040 0x559b360decf0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c673a740 1 --2- 172.21.15.103:0/1314767293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b36061680 0x559b360dade0 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c673a740 1 --2- 172.21.15.103:0/1314767293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b36056dc0 0x559b36068020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c673a740 1 --2- 172.21.15.103:0/1314767293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b360564a0 0x559b36067ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c673a740 1 -- 172.21.15.103:0/1314767293 >> 172.21.15.103:0/1314767293 conn(0x559b36052a30 msgr2=0x559b36051b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62a5ffb640 0 client.21339 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:05:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c673a740 1 -- 172.21.15.103:0/1314767293 shutdown_connections 2024-09-19T07:05:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c673a740 1 -- 172.21.15.103:0/1314767293 wait complete. 2024-09-19T07:05:55.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c673a740 20 client.21339 trim_cache size 0 max 16384 2024-09-19T07:05:55.168 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:05:55.168 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:05:55.168 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:05:55.168 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:05:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c4361640 1 -- 172.21.15.103:0/1589122140 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62a80a3430 msgr2=0x7f62a80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c4361640 1 --2- 172.21.15.103:0/1589122140 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62a80a3430 0x7f62a80a58f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009270 tx=0x7f62b4009c90 comp rx=0 tx=0).stop 2024-09-19T07:05:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c4361640 1 -- 172.21.15.103:0/1589122140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62bc101420 msgr2=0x7f62bc10e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.161+0000 7f62c4361640 1 --2- 172.21.15.103:0/1589122140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62bc101420 0x7f62bc10e7a0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009230 tx=0x7f62ac004270 comp rx=0 tx=0).stop 2024-09-19T07:05:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.162+0000 7f62c4361640 1 -- 172.21.15.103:0/1589122140 shutdown_connections 2024-09-19T07:05:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.162+0000 7f62c4361640 1 --2- 172.21.15.103:0/1589122140 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f62a80a3430 0x7f62a80a58f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.162+0000 7f62c4361640 1 --2- 172.21.15.103:0/1589122140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62bc106a80 0x7f62bc10ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.162+0000 7f62c4361640 1 --2- 172.21.15.103:0/1589122140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62bc101420 0x7f62bc10e7a0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.162+0000 7f62c4361640 1 --2- 172.21.15.103:0/1589122140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62bc100a80 0x7f62bc10e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.162+0000 7f62c4361640 1 -- 172.21.15.103:0/1589122140 >> 172.21.15.103:0/1589122140 conn(0x7f62bc0fc7a0 msgr2=0x7f62bc0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:55.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.162+0000 7f62c4361640 1 -- 172.21.15.103:0/1589122140 shutdown_connections 2024-09-19T07:05:55.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:55.162+0000 7f62c4361640 1 -- 172.21.15.103:0/1589122140 wait complete. 2024-09-19T07:05:55.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:55 smithi103 ceph-mon[31158]: pgmap v521: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-19T07:05:55.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:55 smithi174 ceph-mon[35637]: pgmap v521: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-19T07:05:55.929 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:05:55.929 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:05:55.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:55 smithi134 ceph-mon[35356]: pgmap v521: 97 pgs: 97 active+clean; 4.4 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-09-19T07:05:56.190 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.820+0000 7f0a7f11f640 1 -- 172.21.15.103:0/2527387994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a78100e20 msgr2=0x7f0a78101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.820+0000 7f0a7f11f640 1 --2- 172.21.15.103:0/2527387994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a78100e20 0x7f0a78101200 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009900 tx=0x7f0a6002a480 comp rx=0 tx=0).stop 2024-09-19T07:05:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.820+0000 7f0a7f11f640 1 -- 172.21.15.103:0/2527387994 shutdown_connections 2024-09-19T07:05:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.820+0000 7f0a7f11f640 1 --2- 172.21.15.103:0/2527387994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a78102ef0 0x7f0a7810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.820+0000 7f0a7f11f640 1 --2- 172.21.15.103:0/2527387994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a78102570 0x7f0a781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.820+0000 7f0a7f11f640 1 --2- 172.21.15.103:0/2527387994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a78100e20 0x7f0a78101200 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.820+0000 7f0a7f11f640 1 -- 172.21.15.103:0/2527387994 >> 172.21.15.103:0/2527387994 conn(0x7f0a780fc7d0 msgr2=0x7f0a780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.820+0000 7f0a7f11f640 1 -- 172.21.15.103:0/2527387994 shutdown_connections 2024-09-19T07:05:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.820+0000 7f0a7f11f640 1 -- 172.21.15.103:0/2527387994 wait complete. 2024-09-19T07:05:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.820+0000 7f0a7f11f640 1 Processor -- start 2024-09-19T07:05:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7f11f640 1 -- start start 2024-09-19T07:05:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7f11f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a78100e20 0x7f0a78197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7f11f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a78102570 0x7f0a78197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7f11f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a78102ef0 0x7f0a7819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7f11f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a78110fc0 con 0x7f0a78102ef0 2024-09-19T07:05:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7f11f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0a78110e40 con 0x7f0a78100e20 2024-09-19T07:05:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7f11f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a78111140 con 0x7f0a78102570 2024-09-19T07:05:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7d6bc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a78102ef0 0x7f0a7819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7d6bc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a78102ef0 0x7f0a7819bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57672/0 (socket says 172.21.15.103:57672) 2024-09-19T07:05:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7d6bc640 1 -- 172.21.15.103:0/2371663224 learned_addr learned my addr 172.21.15.103:0/2371663224 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a77fff640 1 --2- 172.21.15.103:0/2371663224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a78102570 0x7f0a78197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7cebb640 1 --2- 172.21.15.103:0/2371663224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a78100e20 0x7f0a78197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7d6bc640 1 -- 172.21.15.103:0/2371663224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a78102570 msgr2=0x7f0a78197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7d6bc640 1 --2- 172.21.15.103:0/2371663224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a78102570 0x7f0a78197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7d6bc640 1 -- 172.21.15.103:0/2371663224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a78100e20 msgr2=0x7f0a78197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7d6bc640 1 --2- 172.21.15.103:0/2371663224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a78100e20 0x7f0a78197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7d6bc640 1 -- 172.21.15.103:0/2371663224 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7819c3e0 con 0x7f0a78102ef0 2024-09-19T07:05:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7cebb640 1 --2- 172.21.15.103:0/2371663224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a78100e20 0x7f0a78197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:56.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.821+0000 7f0a7d6bc640 1 --2- 172.21.15.103:0/2371663224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a78102ef0 0x7f0a7819bd00 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009fd0 tx=0x7f0a7000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:56.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.822+0000 7f0a75ffb640 1 -- 172.21.15.103:0/2371663224 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70011830 con 0x7f0a78102ef0 2024-09-19T07:05:56.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.822+0000 7f0a75ffb640 1 -- 172.21.15.103:0/2371663224 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a70016070 con 0x7f0a78102ef0 2024-09-19T07:05:56.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.822+0000 7f0a7f11f640 1 -- 172.21.15.103:0/2371663224 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7819c6d0 con 0x7f0a78102ef0 2024-09-19T07:05:56.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.822+0000 7f0a75ffb640 1 -- 172.21.15.103:0/2371663224 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7000f2e0 con 0x7f0a78102ef0 2024-09-19T07:05:56.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.822+0000 7f0a7f11f640 1 -- 172.21.15.103:0/2371663224 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a781a3fb0 con 0x7f0a78102ef0 2024-09-19T07:05:56.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.823+0000 7f0a7f11f640 1 -- 172.21.15.103:0/2371663224 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a78101200 con 0x7f0a78102ef0 2024-09-19T07:05:56.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.824+0000 7f0a75ffb640 1 -- 172.21.15.103:0/2371663224 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0a700119d0 con 0x7f0a78102ef0 2024-09-19T07:05:56.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.824+0000 7f0a75ffb640 1 --2- 172.21.15.103:0/2371663224 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0a640a3430 0x7f0a640a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:56.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.825+0000 7f0a7cebb640 1 --2- 172.21.15.103:0/2371663224 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0a640a3430 0x7f0a640a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:56.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.825+0000 7f0a75ffb640 1 -- 172.21.15.103:0/2371663224 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f0a700cf260 con 0x7f0a78102ef0 2024-09-19T07:05:56.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.825+0000 7f0a7cebb640 1 --2- 172.21.15.103:0/2371663224 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0a640a3430 0x7f0a640a58f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0a60000c00 tx=0x7f0a60004100 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:56.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:56.829+0000 7f0a75ffb640 1 -- 172.21.15.103:0/2371663224 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0a7009bd30 con 0x7f0a78102ef0 2024-09-19T07:05:57.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:57.033+0000 7f0a7f11f640 1 -- 172.21.15.103:0/2371663224 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a78109090 con 0x7f0a78102ef0 2024-09-19T07:05:57.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:57.033+0000 7f0a75ffb640 1 -- 172.21.15.103:0/2371663224 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f0a7009b630 con 0x7f0a78102ef0 2024-09-19T07:05:57.034 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:05:57.034 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:05:57.035 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:05:57.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:57.035+0000 7f0a7f11f640 1 -- 172.21.15.103:0/2371663224 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0a640a3430 msgr2=0x7f0a640a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:57.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:57.035+0000 7f0a7f11f640 1 --2- 172.21.15.103:0/2371663224 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0a640a3430 0x7f0a640a58f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0a60000c00 tx=0x7f0a60004100 comp rx=0 tx=0).stop 2024-09-19T07:05:57.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:57.035+0000 7f0a7f11f640 1 -- 172.21.15.103:0/2371663224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a78102ef0 msgr2=0x7f0a7819bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:57.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:57.035+0000 7f0a7f11f640 1 --2- 172.21.15.103:0/2371663224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a78102ef0 0x7f0a7819bd00 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009fd0 tx=0x7f0a7000b8e0 comp rx=0 tx=0).stop 2024-09-19T07:05:57.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:57.035+0000 7f0a7f11f640 1 -- 172.21.15.103:0/2371663224 shutdown_connections 2024-09-19T07:05:57.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:57.035+0000 7f0a7f11f640 1 --2- 172.21.15.103:0/2371663224 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0a640a3430 0x7f0a640a58f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:57.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:57.035+0000 7f0a7f11f640 1 --2- 172.21.15.103:0/2371663224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a78102ef0 0x7f0a7819bd00 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:57.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:57.035+0000 7f0a7f11f640 1 --2- 172.21.15.103:0/2371663224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a78102570 0x7f0a78197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:57.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:57.035+0000 7f0a7f11f640 1 --2- 172.21.15.103:0/2371663224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a78100e20 0x7f0a78197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:57.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:57.035+0000 7f0a7f11f640 1 -- 172.21.15.103:0/2371663224 >> 172.21.15.103:0/2371663224 conn(0x7f0a780fc7d0 msgr2=0x7f0a780fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:57.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:57.036+0000 7f0a7f11f640 1 -- 172.21.15.103:0/2371663224 shutdown_connections 2024-09-19T07:05:57.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:57.036+0000 7f0a7f11f640 1 -- 172.21.15.103:0/2371663224 wait complete. 2024-09-19T07:05:57.637 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:57 smithi103 ceph-mon[31158]: pgmap v522: 97 pgs: 97 active+clean; 4.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 21 op/s 2024-09-19T07:05:57.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:57 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2371663224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:05:57.687 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:05:57.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:57 smithi174 ceph-mon[35637]: pgmap v522: 97 pgs: 97 active+clean; 4.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 21 op/s 2024-09-19T07:05:57.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:57 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2371663224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:05:57.936 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:57.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:57 smithi134 ceph-mon[35356]: pgmap v522: 97 pgs: 97 active+clean; 4.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 21 op/s 2024-09-19T07:05:57.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:57 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2371663224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:05:58.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.417+0000 7f767e2b4640 1 -- 172.21.15.103:0/3866794486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7678102d40 msgr2=0x7f7678103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:58.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.417+0000 7f767e2b4640 1 --2- 172.21.15.103:0/3866794486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7678102d40 0x7f7678103120 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f7668009900 tx=0x7f766802a480 comp rx=0 tx=0).stop 2024-09-19T07:05:58.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.417+0000 7f767e2b4640 1 -- 172.21.15.103:0/3866794486 shutdown_connections 2024-09-19T07:05:58.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.417+0000 7f767e2b4640 1 --2- 172.21.15.103:0/3866794486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f767810c170 0x7f767810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.417+0000 7f767e2b4640 1 --2- 172.21.15.103:0/3866794486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7678103660 0x7f767810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.417+0000 7f767e2b4640 1 --2- 172.21.15.103:0/3866794486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7678102d40 0x7f7678103120 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.417+0000 7f767e2b4640 1 -- 172.21.15.103:0/3866794486 >> 172.21.15.103:0/3866794486 conn(0x7f76780fc7a0 msgr2=0x7f76780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:58.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.417+0000 7f767e2b4640 1 -- 172.21.15.103:0/3866794486 shutdown_connections 2024-09-19T07:05:58.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.417+0000 7f767e2b4640 1 -- 172.21.15.103:0/3866794486 wait complete. 2024-09-19T07:05:58.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.418+0000 7f767e2b4640 1 Processor -- start 2024-09-19T07:05:58.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.418+0000 7f767e2b4640 1 -- start start 2024-09-19T07:05:58.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.418+0000 7f767e2b4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7678102d40 0x7f76781009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.418+0000 7f767e2b4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7678103660 0x7f7678100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.418+0000 7f767e2b4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f767810c170 0x7f7678101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.418+0000 7f767e2b4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7678111f70 con 0x7f7678102d40 2024-09-19T07:05:58.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f767e2b4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7678111df0 con 0x7f7678103660 2024-09-19T07:05:58.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f767e2b4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76781120f0 con 0x7f767810c170 2024-09-19T07:05:58.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f7677fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7678102d40 0x7f76781009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f7677fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7678102d40 0x7f76781009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34266/0 (socket says 172.21.15.103:34266) 2024-09-19T07:05:58.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f7677fff640 1 -- 172.21.15.103:0/309987894 learned_addr learned my addr 172.21.15.103:0/309987894 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:58.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f76777fe640 1 --2- 172.21.15.103:0/309987894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7678103660 0x7f7678100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f767c851640 1 --2- 172.21.15.103:0/309987894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f767810c170 0x7f7678101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f7677fff640 1 -- 172.21.15.103:0/309987894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f767810c170 msgr2=0x7f7678101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:58.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f7677fff640 1 --2- 172.21.15.103:0/309987894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f767810c170 0x7f7678101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f7677fff640 1 -- 172.21.15.103:0/309987894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7678103660 msgr2=0x7f7678100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:58.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f7677fff640 1 --2- 172.21.15.103:0/309987894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7678103660 0x7f7678100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f7677fff640 1 -- 172.21.15.103:0/309987894 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76781a6910 con 0x7f7678102d40 2024-09-19T07:05:58.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f7677fff640 1 --2- 172.21.15.103:0/309987894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7678102d40 0x7f76781009a0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f7668005ab0 tx=0x7f76680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:58.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f76757fa640 1 -- 172.21.15.103:0/309987894 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766803f030 con 0x7f7678102d40 2024-09-19T07:05:58.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f76757fa640 1 -- 172.21.15.103:0/309987894 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7668003ce0 con 0x7f7678102d40 2024-09-19T07:05:58.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f767e2b4640 1 -- 172.21.15.103:0/309987894 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76781a6ba0 con 0x7f7678102d40 2024-09-19T07:05:58.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f76757fa640 1 -- 172.21.15.103:0/309987894 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668035070 con 0x7f7678102d40 2024-09-19T07:05:58.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.419+0000 7f767e2b4640 1 -- 172.21.15.103:0/309987894 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76781a7080 con 0x7f7678102d40 2024-09-19T07:05:58.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.421+0000 7f76757fa640 1 -- 172.21.15.103:0/309987894 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7668042030 con 0x7f7678102d40 2024-09-19T07:05:58.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.422+0000 7f768068d740 1 Processor -- start 2024-09-19T07:05:58.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.422+0000 7f768068d740 1 -- start start 2024-09-19T07:05:58.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.422+0000 7f76757fa640 1 --2- 172.21.15.103:0/309987894 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f76600a33b0 0x7f76600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.422+0000 7f768068d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9edb40 0x55d75f9ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.422+0000 7f768068d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9ec5d0 0x55d75f9eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.422+0000 7f768068d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9f7370 0x55d75f9f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.422+0000 7f768068d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d75f9522f0 con 0x55d75f9edb40 2024-09-19T07:05:58.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.422+0000 7f768068d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d75f8f9cf0 con 0x55d75f9f7370 2024-09-19T07:05:58.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.422+0000 7f768068d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d75f94e2b0 con 0x55d75f9ec5d0 2024-09-19T07:05:58.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.422+0000 7f767c851640 1 --2- 172.21.15.103:0/309987894 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f76600a33b0 0x7f76600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.422+0000 7f7677fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9edb40 0x55d75f9ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.422+0000 7f76757fa640 1 -- 172.21.15.103:0/309987894 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f76680f06b0 con 0x7f7678102d40 2024-09-19T07:05:58.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.422+0000 7f7677fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9edb40 0x55d75f9ec090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34280/0 (socket says 172.21.15.103:34280) 2024-09-19T07:05:58.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.422+0000 7f7677fff640 1 -- 172.21.15.103:0/3008290559 learned_addr learned my addr 172.21.15.103:0/3008290559 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:58.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.422+0000 7f76777fe640 1 --2- 172.21.15.103:0/3008290559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9ec5d0 0x55d75f9eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.423+0000 7f767c851640 1 --2- 172.21.15.103:0/3008290559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9f7370 0x55d75f9f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.423+0000 7f7677fff640 1 -- 172.21.15.103:0/3008290559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9ec5d0 msgr2=0x55d75f9eca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:58.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.423+0000 7f7677fff640 1 --2- 172.21.15.103:0/3008290559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9ec5d0 0x55d75f9eca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.423+0000 7f7677fff640 1 -- 172.21.15.103:0/3008290559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9f7370 msgr2=0x55d75f9f9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:58.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.423+0000 7f7677fff640 1 --2- 172.21.15.103:0/3008290559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9f7370 0x55d75f9f9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.423+0000 7f7677fff640 1 -- 172.21.15.103:0/3008290559 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d75f8de3a0 con 0x55d75f9edb40 2024-09-19T07:05:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.423+0000 7f767c851640 1 --2- 172.21.15.103:0/309987894 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f76600a33b0 0x7f76600a5870 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7670005cc0 tx=0x7f76700127c0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.423+0000 7f7677fff640 1 --2- 172.21.15.103:0/3008290559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9edb40 0x55d75f9ec090 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f7668000c00 tx=0x7f766803b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace5295ebe6c0f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.423+0000 7f765e7fc640 1 -- 172.21.15.103:0/3008290559 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668049070 con 0x55d75f9edb40 2024-09-19T07:05:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.423+0000 7f765e7fc640 1 -- 172.21.15.103:0/3008290559 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76680bd010 con 0x55d75f9edb40 2024-09-19T07:05:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.423+0000 7f765e7fc640 1 -- 172.21.15.103:0/3008290559 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76680b8f80 con 0x55d75f9edb40 2024-09-19T07:05:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.424+0000 7f768068d740 1 -- 172.21.15.103:0/3008290559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9edb40 msgr2=0x55d75f9ec090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.424+0000 7f768068d740 1 --2- 172.21.15.103:0/3008290559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9edb40 0x55d75f9ec090 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f7668000c00 tx=0x7f766803b800 comp rx=0 tx=0).stop 2024-09-19T07:05:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.424+0000 7f768068d740 1 -- 172.21.15.103:0/3008290559 shutdown_connections 2024-09-19T07:05:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.424+0000 7f768068d740 1 --2- 172.21.15.103:0/3008290559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9f7370 0x55d75f9f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.424+0000 7f768068d740 1 --2- 172.21.15.103:0/3008290559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9ec5d0 0x55d75f9eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.424+0000 7f768068d740 1 --2- 172.21.15.103:0/3008290559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9edb40 0x55d75f9ec090 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.424+0000 7f768068d740 1 -- 172.21.15.103:0/3008290559 >> 172.21.15.103:0/3008290559 conn(0x55d75f9b5310 msgr2=0x55d75f9e8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:58.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.424+0000 7f768068d740 1 -- 172.21.15.103:0/3008290559 shutdown_connections 2024-09-19T07:05:58.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.424+0000 7f768068d740 1 -- 172.21.15.103:0/3008290559 wait complete. 2024-09-19T07:05:58.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.424+0000 7f768068d740 1 -- start start 2024-09-19T07:05:58.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.424+0000 7f768068d740 1 Processor -- start 2024-09-19T07:05:58.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f768068d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9ec5d0 0x55d75f9f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f768068d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9edb40 0x55d75f9f4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f768068d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9f7370 0x55d75f9f4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f768068d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d75f9f9e30 con 0x55d75f9ec5d0 2024-09-19T07:05:58.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f768068d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d75f90d060 con 0x55d75f9f7370 2024-09-19T07:05:58.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f768068d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d75f9f9fb0 con 0x55d75f9edb40 2024-09-19T07:05:58.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f768068d740 10 client.0 _ll_register_callbacks cb 0x55d75f6608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:58.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f7677fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9ec5d0 0x55d75f9f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f7677fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9ec5d0 0x55d75f9f6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34282/0 (socket says 172.21.15.103:34282) 2024-09-19T07:05:58.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f7677fff640 1 -- 172.21.15.103:0/1718490206 learned_addr learned my addr 172.21.15.103:0/1718490206 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:58.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f76777fe640 1 --2- 172.21.15.103:0/1718490206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9edb40 0x55d75f9f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f767c851640 1 --2- 172.21.15.103:0/1718490206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9f7370 0x55d75f9f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f7677fff640 1 -- 172.21.15.103:0/1718490206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9edb40 msgr2=0x55d75f9f4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:58.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f7677fff640 1 --2- 172.21.15.103:0/1718490206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9edb40 0x55d75f9f4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f7677fff640 1 -- 172.21.15.103:0/1718490206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9f7370 msgr2=0x55d75f9f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:58.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f7677fff640 1 --2- 172.21.15.103:0/1718490206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9f7370 0x55d75f9f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f7677fff640 1 -- 172.21.15.103:0/1718490206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d75f6c33c0 con 0x55d75f9ec5d0 2024-09-19T07:05:58.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f767c851640 1 --2- 172.21.15.103:0/1718490206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9f7370 0x55d75f9f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:05:58.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f7677fff640 1 --2- 172.21.15.103:0/1718490206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9ec5d0 0x55d75f9f6d60 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f7668003fd0 tx=0x7f76680b72f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c70994156fa36da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:58.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f76577fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:58.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f76577fe640 1 -- 172.21.15.103:0/1718490206 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668049070 con 0x55d75f9ec5d0 2024-09-19T07:05:58.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f76577fe640 1 -- 172.21.15.103:0/1718490206 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f76680ee000 con 0x55d75f9ec5d0 2024-09-19T07:05:58.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.425+0000 7f768068d740 1 -- 172.21.15.103:0/1718490206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76680bd010 con 0x55d75f9ec5d0 2024-09-19T07:05:58.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.426+0000 7f76577fe640 1 -- 172.21.15.103:0/1718490206 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f76680bc270 con 0x55d75f9ec5d0 2024-09-19T07:05:58.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.426+0000 7f768068d740 10 client.21357 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:58.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.426+0000 7f768068d740 1 -- 172.21.15.103:0/1718490206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d75f9f9e30 con 0x55d75f9ec5d0 2024-09-19T07:05:58.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.426+0000 7f76577fe640 1 -- 172.21.15.103:0/1718490206 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76680bd010 con 0x55d75f9ec5d0 2024-09-19T07:05:58.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.426+0000 7f76577fe640 1 -- 172.21.15.103:0/1718490206 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f76680bcbe0 con 0x55d75f9ec5d0 2024-09-19T07:05:58.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.426+0000 7f768068d740 10 client.21357 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:58.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.426+0000 7f768068d740 10 client.21357 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:05:58.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.426+0000 7f768068d740 20 client.21357 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:58.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.426+0000 7f768068d740 1 --2- 172.21.15.103:0/1718490206 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55d75fa06980 0x55d75fa08d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.426+0000 7f768068d740 4 client.21357 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:58.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.426+0000 7f768068d740 1 -- 172.21.15.103:0/1718490206 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d75f95f860 con 0x55d75fa06980 2024-09-19T07:05:58.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.426+0000 7f76777fe640 1 --2- 172.21.15.103:0/1718490206 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55d75fa06980 0x55d75fa08d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.427+0000 7f76577fe640 10 client.21357 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:05:58.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.427+0000 7f76777fe640 1 --2- 172.21.15.103:0/1718490206 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55d75fa06980 0x55d75fa08d70 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c971ec1786c3700 server_cookie=eced45e5675e6a01 in_seq=0 out_seq=0 2024-09-19T07:05:58.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.428+0000 7f76577fe640 1 -- 172.21.15.103:0/1718490206 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d75f95f860 con 0x55d75fa06980 2024-09-19T07:05:58.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.428+0000 7f76577fe640 10 client.21357 handle_command_reply: tid=0 2024-09-19T07:05:58.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.428+0000 7f768068d740 1 client.21357 shutdown 2024-09-19T07:05:58.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.428+0000 7f768068d740 1 -- 172.21.15.103:0/1718490206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9ec5d0 msgr2=0x55d75f9f6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:58.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.428+0000 7f768068d740 1 --2- 172.21.15.103:0/1718490206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9ec5d0 0x55d75f9f6d60 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f7668003fd0 tx=0x7f76680b72f0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.428+0000 7f768068d740 1 -- 172.21.15.103:0/1718490206 shutdown_connections 2024-09-19T07:05:58.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.428+0000 7f768068d740 1 --2- 172.21.15.103:0/1718490206 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55d75fa06980 0x55d75fa08d70 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.428+0000 7f768068d740 1 --2- 172.21.15.103:0/1718490206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9f7370 0x55d75f9f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.428+0000 7f768068d740 1 --2- 172.21.15.103:0/1718490206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9edb40 0x55d75f9f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.428+0000 7f768068d740 1 --2- 172.21.15.103:0/1718490206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9ec5d0 0x55d75f9f6d60 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.428+0000 7f768068d740 1 -- 172.21.15.103:0/1718490206 >> 172.21.15.103:0/1718490206 conn(0x55d75f9b5310 msgr2=0x55d75f9ee990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:58.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.428+0000 7f76577fe640 0 client.21357 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:05:58.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.428+0000 7f768068d740 1 -- 172.21.15.103:0/1718490206 shutdown_connections 2024-09-19T07:05:58.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.428+0000 7f768068d740 1 -- 172.21.15.103:0/1718490206 wait complete. 2024-09-19T07:05:58.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.428+0000 7f768068d740 20 client.21357 trim_cache size 0 max 16384 2024-09-19T07:05:58.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f768068d740 1 Processor -- start 2024-09-19T07:05:58.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f768068d740 1 -- start start 2024-09-19T07:05:58.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f768068d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9ec5d0 0x55d75fa03000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f768068d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9edb40 0x55d75f9f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f768068d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9f7370 0x55d75f9ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f768068d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76781afb20 con 0x55d75f9edb40 2024-09-19T07:05:58.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f768068d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f76781b1660 con 0x55d75f9ec5d0 2024-09-19T07:05:58.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f768068d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76781b01f0 con 0x55d75f9f7370 2024-09-19T07:05:58.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f76777fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9edb40 0x55d75f9f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f76777fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9edb40 0x55d75f9f38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34294/0 (socket says 172.21.15.103:34294) 2024-09-19T07:05:58.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f76777fe640 1 -- 172.21.15.103:0/1806289160 learned_addr learned my addr 172.21.15.103:0/1806289160 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:58.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f7677fff640 1 --2- 172.21.15.103:0/1806289160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9ec5d0 0x55d75fa03000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f76777fe640 1 -- 172.21.15.103:0/1806289160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9f7370 msgr2=0x55d75f9ef120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:58.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f76777fe640 1 --2- 172.21.15.103:0/1806289160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9f7370 0x55d75f9ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f76777fe640 1 -- 172.21.15.103:0/1806289160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9ec5d0 msgr2=0x55d75fa03000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:58.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f76777fe640 1 --2- 172.21.15.103:0/1806289160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9ec5d0 0x55d75fa03000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.441+0000 7f76777fe640 1 -- 172.21.15.103:0/1806289160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d75f9f2560 con 0x55d75f9edb40 2024-09-19T07:05:58.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.442+0000 7f76777fe640 1 --2- 172.21.15.103:0/1806289160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9edb40 0x55d75f9f38d0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f766c000c00 tx=0x7f766c005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98ef6df4baa86a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:58.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.442+0000 7f765dffb640 1 -- 172.21.15.103:0/1806289160 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c005220 con 0x55d75f9edb40 2024-09-19T07:05:58.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.442+0000 7f765dffb640 1 -- 172.21.15.103:0/1806289160 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f766c004c80 con 0x55d75f9edb40 2024-09-19T07:05:58.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.442+0000 7f765dffb640 1 -- 172.21.15.103:0/1806289160 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c01e960 con 0x55d75f9edb40 2024-09-19T07:05:58.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.442+0000 7f768068d740 1 -- 172.21.15.103:0/1806289160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9edb40 msgr2=0x55d75f9f38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:58.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.442+0000 7f768068d740 1 --2- 172.21.15.103:0/1806289160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9edb40 0x55d75f9f38d0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f766c000c00 tx=0x7f766c005f70 comp rx=0 tx=0).stop 2024-09-19T07:05:58.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.442+0000 7f768068d740 1 -- 172.21.15.103:0/1806289160 shutdown_connections 2024-09-19T07:05:58.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.442+0000 7f768068d740 1 --2- 172.21.15.103:0/1806289160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9f7370 0x55d75f9ef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.442+0000 7f768068d740 1 --2- 172.21.15.103:0/1806289160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9edb40 0x55d75f9f38d0 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.442+0000 7f768068d740 1 --2- 172.21.15.103:0/1806289160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9ec5d0 0x55d75fa03000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.442+0000 7f768068d740 1 -- 172.21.15.103:0/1806289160 >> 172.21.15.103:0/1806289160 conn(0x55d75f9b5310 msgr2=0x55d75f9f3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:58.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.442+0000 7f768068d740 1 -- 172.21.15.103:0/1806289160 shutdown_connections 2024-09-19T07:05:58.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.442+0000 7f768068d740 1 -- 172.21.15.103:0/1806289160 wait complete. 2024-09-19T07:05:58.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.443+0000 7f768068d740 1 -- start start 2024-09-19T07:05:58.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.443+0000 7f768068d740 1 Processor -- start 2024-09-19T07:05:58.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.443+0000 7f768068d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9ec5d0 0x55d75f9ff310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.443+0000 7f768068d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9edb40 0x55d75f9ff850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.443+0000 7f768068d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9f7370 0x55d75f9fa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.443+0000 7f768068d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76781b0f90 con 0x55d75f9ec5d0 2024-09-19T07:05:58.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.443+0000 7f768068d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f76781b08c0 con 0x55d75f9f7370 2024-09-19T07:05:58.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.443+0000 7f768068d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76781adb20 con 0x55d75f9edb40 2024-09-19T07:05:58.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.443+0000 7f768068d740 10 client.0 _ll_register_callbacks cb 0x7f766c00a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:58.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.443+0000 7f7677fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9ec5d0 0x55d75f9ff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.443+0000 7f7677fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9ec5d0 0x55d75f9ff310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34310/0 (socket says 172.21.15.103:34310) 2024-09-19T07:05:58.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.443+0000 7f7677fff640 1 -- 172.21.15.103:0/2974514167 learned_addr learned my addr 172.21.15.103:0/2974514167 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:58.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.444+0000 7f76777fe640 1 --2- 172.21.15.103:0/2974514167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9edb40 0x55d75f9ff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.444+0000 7f767c851640 1 --2- 172.21.15.103:0/2974514167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9f7370 0x55d75f9fa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.444+0000 7f7677fff640 1 -- 172.21.15.103:0/2974514167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9edb40 msgr2=0x55d75f9ff850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:58.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.444+0000 7f7677fff640 1 --2- 172.21.15.103:0/2974514167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9edb40 0x55d75f9ff850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.444+0000 7f7677fff640 1 -- 172.21.15.103:0/2974514167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9f7370 msgr2=0x55d75f9fa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:58.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.444+0000 7f7677fff640 1 --2- 172.21.15.103:0/2974514167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9f7370 0x55d75f9fa6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.444+0000 7f7677fff640 1 -- 172.21.15.103:0/2974514167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d75f9faef0 con 0x55d75f9ec5d0 2024-09-19T07:05:58.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.444+0000 7f7677fff640 1 --2- 172.21.15.103:0/2974514167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9ec5d0 0x55d75f9ff310 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f76680b92e0 tx=0x7f76680045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84dcc84f627e746 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:58.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.444+0000 7f76577fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:58.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.444+0000 7f76577fe640 1 -- 172.21.15.103:0/2974514167 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668049040 con 0x55d75f9ec5d0 2024-09-19T07:05:58.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.444+0000 7f76577fe640 1 -- 172.21.15.103:0/2974514167 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f76680bbdb0 con 0x55d75f9ec5d0 2024-09-19T07:05:58.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.444+0000 7f768068d740 1 -- 172.21.15.103:0/2974514167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f766c004c80 con 0x55d75f9ec5d0 2024-09-19T07:05:58.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.444+0000 7f76577fe640 1 -- 172.21.15.103:0/2974514167 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f76680b8bb0 con 0x55d75f9ec5d0 2024-09-19T07:05:58.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.444+0000 7f768068d740 10 client.21363 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:58.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.444+0000 7f768068d740 1 -- 172.21.15.103:0/2974514167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d75f9faef0 con 0x55d75f9ec5d0 2024-09-19T07:05:58.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.444+0000 7f76577fe640 1 -- 172.21.15.103:0/2974514167 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f766c004c80 con 0x55d75f9ec5d0 2024-09-19T07:05:58.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.445+0000 7f76577fe640 1 -- 172.21.15.103:0/2974514167 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f76680db1a0 con 0x55d75f9ec5d0 2024-09-19T07:05:58.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.445+0000 7f768068d740 10 client.21363 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:58.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.445+0000 7f768068d740 10 client.21363 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:05:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.445+0000 7f768068d740 20 client.21363 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.445+0000 7f768068d740 1 --2- 172.21.15.103:0/2974514167 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55d75fa71330 0x55d75fa73720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.445+0000 7f768068d740 4 client.21363 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:05:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.445+0000 7f768068d740 1 -- 172.21.15.103:0/2974514167 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d75f959d70 con 0x55d75fa71330 2024-09-19T07:05:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.445+0000 7f76777fe640 1 --2- 172.21.15.103:0/2974514167 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55d75fa71330 0x55d75fa73720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.445+0000 7f76577fe640 10 client.21363 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:05:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.445+0000 7f76777fe640 1 --2- 172.21.15.103:0/2974514167 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55d75fa71330 0x55d75fa73720 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=67008b598d78c645 server_cookie=4e1d17cc7797dd56 in_seq=0 out_seq=0 2024-09-19T07:05:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f76577fe640 1 -- 172.21.15.103:0/2974514167 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d75f959d70 con 0x55d75fa71330 2024-09-19T07:05:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f76577fe640 10 client.21363 handle_command_reply: tid=0 2024-09-19T07:05:58.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f768068d740 1 client.21363 shutdown 2024-09-19T07:05:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f768068d740 1 -- 172.21.15.103:0/2974514167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9ec5d0 msgr2=0x55d75f9ff310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f768068d740 1 --2- 172.21.15.103:0/2974514167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9ec5d0 0x55d75f9ff310 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f76680b92e0 tx=0x7f76680045e0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f768068d740 1 -- 172.21.15.103:0/2974514167 shutdown_connections 2024-09-19T07:05:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f768068d740 1 --2- 172.21.15.103:0/2974514167 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55d75fa71330 0x55d75fa73720 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f768068d740 1 --2- 172.21.15.103:0/2974514167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75f9f7370 0x55d75f9fa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f768068d740 1 --2- 172.21.15.103:0/2974514167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d75f9edb40 0x55d75f9ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f768068d740 1 --2- 172.21.15.103:0/2974514167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d75f9ec5d0 0x55d75f9ff310 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f768068d740 1 -- 172.21.15.103:0/2974514167 >> 172.21.15.103:0/2974514167 conn(0x55d75f9b5310 msgr2=0x55d75f9f5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f76577fe640 0 client.21363 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:05:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f768068d740 1 -- 172.21.15.103:0/2974514167 shutdown_connections 2024-09-19T07:05:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f768068d740 1 -- 172.21.15.103:0/2974514167 wait complete. 2024-09-19T07:05:58.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f768068d740 20 client.21363 trim_cache size 0 max 16384 2024-09-19T07:05:58.460 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:05:58.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f767e2b4640 1 -- 172.21.15.103:0/309987894 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f76600a33b0 msgr2=0x7f76600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:58.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f767e2b4640 1 --2- 172.21.15.103:0/309987894 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f76600a33b0 0x7f76600a5870 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7670005cc0 tx=0x7f76700127c0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f767e2b4640 1 -- 172.21.15.103:0/309987894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7678102d40 msgr2=0x7f76781009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:58.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.446+0000 7f767e2b4640 1 --2- 172.21.15.103:0/309987894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7678102d40 0x7f76781009a0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f7668005ab0 tx=0x7f76680042f0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.447+0000 7f767e2b4640 1 -- 172.21.15.103:0/309987894 shutdown_connections 2024-09-19T07:05:58.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.447+0000 7f767e2b4640 1 --2- 172.21.15.103:0/309987894 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f76600a33b0 0x7f76600a5870 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.447+0000 7f767e2b4640 1 --2- 172.21.15.103:0/309987894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f767810c170 0x7f7678101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.447+0000 7f767e2b4640 1 --2- 172.21.15.103:0/309987894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7678103660 0x7f7678100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.447+0000 7f767e2b4640 1 --2- 172.21.15.103:0/309987894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7678102d40 0x7f76781009a0 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:58.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.447+0000 7f767e2b4640 1 -- 172.21.15.103:0/309987894 >> 172.21.15.103:0/309987894 conn(0x7f76780fc7a0 msgr2=0x7f76780fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:58.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.447+0000 7f767e2b4640 1 -- 172.21.15.103:0/309987894 shutdown_connections 2024-09-19T07:05:58.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:58.447+0000 7f767e2b4640 1 -- 172.21.15.103:0/309987894 wait complete. 2024-09-19T07:05:59.036 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:05:59.284 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:05:59.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.660+0000 7fecadbbb640 1 -- 172.21.15.103:0/3315641506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca8102de0 msgr2=0x7feca81031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:59.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.660+0000 7fecadbbb640 1 --2- 172.21.15.103:0/3315641506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca8102de0 0x7feca81031c0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7fec98009900 tx=0x7fec9802a480 comp rx=0 tx=0).stop 2024-09-19T07:05:59.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.660+0000 7fecadbbb640 1 -- 172.21.15.103:0/3315641506 shutdown_connections 2024-09-19T07:05:59.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.660+0000 7fecadbbb640 1 --2- 172.21.15.103:0/3315641506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feca810c180 0x7feca810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.660+0000 7fecadbbb640 1 --2- 172.21.15.103:0/3315641506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feca8103700 0x7feca810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.660+0000 7fecadbbb640 1 --2- 172.21.15.103:0/3315641506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca8102de0 0x7feca81031c0 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.660+0000 7fecadbbb640 1 -- 172.21.15.103:0/3315641506 >> 172.21.15.103:0/3315641506 conn(0x7feca80fc7a0 msgr2=0x7feca80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:59.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:05:59 smithi103 ceph-mon[31158]: pgmap v523: 97 pgs: 97 active+clean; 4.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T07:05:59.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.660+0000 7fecadbbb640 1 -- 172.21.15.103:0/3315641506 shutdown_connections 2024-09-19T07:05:59.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.660+0000 7fecadbbb640 1 -- 172.21.15.103:0/3315641506 wait complete. 2024-09-19T07:05:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.661+0000 7fecadbbb640 1 Processor -- start 2024-09-19T07:05:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.661+0000 7fecadbbb640 1 -- start start 2024-09-19T07:05:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.661+0000 7fecadbbb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feca8102de0 0x7feca81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.661+0000 7fecadbbb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca8103700 0x7feca8198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.661+0000 7fecadbbb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feca810c180 0x7feca819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.661+0000 7fecadbbb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feca8111f30 con 0x7feca8103700 2024-09-19T07:05:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.661+0000 7fecadbbb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feca8111db0 con 0x7feca810c180 2024-09-19T07:05:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.661+0000 7fecadbbb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feca81120b0 con 0x7feca8102de0 2024-09-19T07:05:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.661+0000 7fec9ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca8103700 0x7feca8198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.661+0000 7fec9ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca8103700 0x7feca8198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34332/0 (socket says 172.21.15.103:34332) 2024-09-19T07:05:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.662+0000 7fec9ffff640 1 -- 172.21.15.103:0/2389497865 learned_addr learned my addr 172.21.15.103:0/2389497865 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.662+0000 7feca77fe640 1 --2- 172.21.15.103:0/2389497865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feca8102de0 0x7feca81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:59.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.662+0000 7feca7fff640 1 --2- 172.21.15.103:0/2389497865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feca810c180 0x7feca819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:59.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.662+0000 7fec9ffff640 1 -- 172.21.15.103:0/2389497865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feca8102de0 msgr2=0x7feca81983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:59.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.662+0000 7fec9ffff640 1 --2- 172.21.15.103:0/2389497865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feca8102de0 0x7feca81983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.662+0000 7fec9ffff640 1 -- 172.21.15.103:0/2389497865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feca810c180 msgr2=0x7feca819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:59.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.662+0000 7fec9ffff640 1 --2- 172.21.15.103:0/2389497865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feca810c180 0x7feca819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.662+0000 7fec9ffff640 1 -- 172.21.15.103:0/2389497865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca819d370 con 0x7feca8103700 2024-09-19T07:05:59.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.662+0000 7fec9ffff640 1 --2- 172.21.15.103:0/2389497865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca8103700 0x7feca8198900 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c920 tx=0x7fec9000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:59.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.662+0000 7feca57fa640 1 -- 172.21.15.103:0/2389497865 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90007c70 con 0x7feca8103700 2024-09-19T07:05:59.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.662+0000 7fecadbbb640 1 -- 172.21.15.103:0/2389497865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca819d660 con 0x7feca8103700 2024-09-19T07:05:59.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.662+0000 7feca57fa640 1 -- 172.21.15.103:0/2389497865 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec90007e10 con 0x7feca8103700 2024-09-19T07:05:59.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.662+0000 7feca57fa640 1 -- 172.21.15.103:0/2389497865 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90018890 con 0x7feca8103700 2024-09-19T07:05:59.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.662+0000 7fecadbbb640 1 -- 172.21.15.103:0/2389497865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca81a4f40 con 0x7feca8103700 2024-09-19T07:05:59.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.665+0000 7feca57fa640 1 -- 172.21.15.103:0/2389497865 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fec90018a70 con 0x7feca8103700 2024-09-19T07:05:59.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.665+0000 7fecaff94740 1 Processor -- start 2024-09-19T07:05:59.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.665+0000 7fecaff94740 1 -- start start 2024-09-19T07:05:59.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.665+0000 7feca57fa640 1 --2- 172.21.15.103:0/2389497865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fec940a3430 0x7fec940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.665+0000 7fecaff94740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1beb40 0x5618af1bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.665+0000 7fecaff94740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1bd5d0 0x5618af1bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.665+0000 7fecaff94740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1c8370 0x5618af1ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.665+0000 7feca57fa640 1 -- 172.21.15.103:0/2389497865 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fec900cfe90 con 0x7feca8103700 2024-09-19T07:05:59.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.665+0000 7fecaff94740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5618af1232f0 con 0x5618af1beb40 2024-09-19T07:05:59.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.665+0000 7fecaff94740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5618af0cacf0 con 0x5618af1bd5d0 2024-09-19T07:05:59.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.665+0000 7fecaff94740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5618af11f2b0 con 0x5618af1c8370 2024-09-19T07:05:59.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.666+0000 7feca7fff640 1 --2- 172.21.15.103:0/2389497865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fec940a3430 0x7fec940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:59.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.666+0000 7feca77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1beb40 0x5618af1bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:59.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.666+0000 7feca77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1beb40 0x5618af1bd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34338/0 (socket says 172.21.15.103:34338) 2024-09-19T07:05:59.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.666+0000 7feca77fe640 1 -- 172.21.15.103:0/1350681329 learned_addr learned my addr 172.21.15.103:0/1350681329 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:59.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.666+0000 7feca77fe640 1 -- 172.21.15.103:0/1350681329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1c8370 msgr2=0x5618af1ca760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:05:59.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.666+0000 7feca77fe640 1 --2- 172.21.15.103:0/1350681329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1c8370 0x5618af1ca760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.666+0000 7feca77fe640 1 -- 172.21.15.103:0/1350681329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1bd5d0 msgr2=0x5618af1bda10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:05:59.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.666+0000 7feca77fe640 1 --2- 172.21.15.103:0/1350681329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1bd5d0 0x5618af1bda10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.666+0000 7feca77fe640 1 -- 172.21.15.103:0/1350681329 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618af0af3a0 con 0x5618af1beb40 2024-09-19T07:05:59.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.666+0000 7feca7fff640 1 --2- 172.21.15.103:0/2389497865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fec940a3430 0x7fec940a58f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feca000ad10 tx=0x7feca000a2b0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:59.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.666+0000 7feca77fe640 1 --2- 172.21.15.103:0/1350681329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1beb40 0x5618af1bd090 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fec98009900 tx=0x7fec980316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3d62935330ba3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:59.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.666+0000 7fec9dffb640 1 -- 172.21.15.103:0/1350681329 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98040070 con 0x5618af1beb40 2024-09-19T07:05:59.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.666+0000 7fec9dffb640 1 -- 172.21.15.103:0/1350681329 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec98005020 con 0x5618af1beb40 2024-09-19T07:05:59.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.666+0000 7fec9dffb640 1 -- 172.21.15.103:0/1350681329 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98035040 con 0x5618af1beb40 2024-09-19T07:05:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.667+0000 7fecaff94740 1 -- 172.21.15.103:0/1350681329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1beb40 msgr2=0x5618af1bd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.667+0000 7fecaff94740 1 --2- 172.21.15.103:0/1350681329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1beb40 0x5618af1bd090 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fec98009900 tx=0x7fec980316e0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.667+0000 7fecaff94740 1 -- 172.21.15.103:0/1350681329 shutdown_connections 2024-09-19T07:05:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.667+0000 7fecaff94740 1 --2- 172.21.15.103:0/1350681329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1c8370 0x5618af1ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.667+0000 7fecaff94740 1 --2- 172.21.15.103:0/1350681329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1bd5d0 0x5618af1bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.667+0000 7fecaff94740 1 --2- 172.21.15.103:0/1350681329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1beb40 0x5618af1bd090 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.667+0000 7fecaff94740 1 -- 172.21.15.103:0/1350681329 >> 172.21.15.103:0/1350681329 conn(0x5618af186310 msgr2=0x5618af1b9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:59.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.667+0000 7fecaff94740 1 -- 172.21.15.103:0/1350681329 shutdown_connections 2024-09-19T07:05:59.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.667+0000 7fecaff94740 1 -- 172.21.15.103:0/1350681329 wait complete. 2024-09-19T07:05:59.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.667+0000 7fecaff94740 1 -- start start 2024-09-19T07:05:59.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7fecaff94740 1 Processor -- start 2024-09-19T07:05:59.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7fecaff94740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1bd5d0 0x5618af1cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7fecaff94740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1beb40 0x5618af1cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7fecaff94740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1c8370 0x5618af1d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7fecaff94740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5618af1cb010 con 0x5618af1beb40 2024-09-19T07:05:59.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7fecaff94740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5618af0de060 con 0x5618af1bd5d0 2024-09-19T07:05:59.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7fecaff94740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5618af1cb190 con 0x5618af1c8370 2024-09-19T07:05:59.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7fecaff94740 10 client.0 _ll_register_callbacks cb 0x5618aee318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:59.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7fec9ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1beb40 0x5618af1cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:59.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7fec9ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1beb40 0x5618af1cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34348/0 (socket says 172.21.15.103:34348) 2024-09-19T07:05:59.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7fec9ffff640 1 -- 172.21.15.103:0/1106050094 learned_addr learned my addr 172.21.15.103:0/1106050094 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:59.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7feca77fe640 1 --2- 172.21.15.103:0/1106050094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1bd5d0 0x5618af1cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:59.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7feca7fff640 1 --2- 172.21.15.103:0/1106050094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1c8370 0x5618af1d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:59.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7fec9ffff640 1 -- 172.21.15.103:0/1106050094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1c8370 msgr2=0x5618af1d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:59.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7fec9ffff640 1 --2- 172.21.15.103:0/1106050094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1c8370 0x5618af1d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7fec9ffff640 1 -- 172.21.15.103:0/1106050094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1bd5d0 msgr2=0x5618af1cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:59.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7fec9ffff640 1 --2- 172.21.15.103:0/1106050094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1bd5d0 0x5618af1cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7fec9ffff640 1 -- 172.21.15.103:0/1106050094 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618aee943c0 con 0x5618af1beb40 2024-09-19T07:05:59.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.668+0000 7feca7fff640 1 --2- 172.21.15.103:0/1106050094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1c8370 0x5618af1d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:59.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.669+0000 7fec9ffff640 1 --2- 172.21.15.103:0/1106050094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1beb40 0x5618af1cccc0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c8f0 tx=0x7fec9009c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747c721eda6099b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:59.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.669+0000 7fec86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:59.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.669+0000 7fec86ffd640 1 -- 172.21.15.103:0/1106050094 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec900b4020 con 0x5618af1beb40 2024-09-19T07:05:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.669+0000 7fec86ffd640 1 -- 172.21.15.103:0/1106050094 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fec900a0170 con 0x5618af1beb40 2024-09-19T07:05:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.669+0000 7fecaff94740 1 -- 172.21.15.103:0/1106050094 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec98005020 con 0x5618af1beb40 2024-09-19T07:05:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.669+0000 7fec86ffd640 1 -- 172.21.15.103:0/1106050094 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fec900a4d60 con 0x5618af1beb40 2024-09-19T07:05:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.669+0000 7fecaff94740 10 client.21375 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.669+0000 7fecaff94740 1 -- 172.21.15.103:0/1106050094 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618af1daf90 con 0x5618af1beb40 2024-09-19T07:05:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.669+0000 7fec86ffd640 1 -- 172.21.15.103:0/1106050094 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec98005020 con 0x5618af1beb40 2024-09-19T07:05:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.669+0000 7fec86ffd640 1 -- 172.21.15.103:0/1106050094 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fec900b6020 con 0x5618af1beb40 2024-09-19T07:05:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.669+0000 7fecaff94740 10 client.21375 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.669+0000 7fecaff94740 10 client.21375 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:05:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.669+0000 7fecaff94740 20 client.21375 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.669+0000 7fecaff94740 1 --2- 172.21.15.103:0/1106050094 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5618af1dc450 0x5618af1de840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.669+0000 7fecaff94740 4 client.21375 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:05:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.669+0000 7fecaff94740 1 -- 172.21.15.103:0/1106050094 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618af130860 con 0x5618af1dc450 2024-09-19T07:05:59.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.670+0000 7feca77fe640 1 --2- 172.21.15.103:0/1106050094 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5618af1dc450 0x5618af1de840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:59.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.670+0000 7feca77fe640 1 --2- 172.21.15.103:0/1106050094 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5618af1dc450 0x5618af1de840 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=63f050e8dce00b76 server_cookie=d14ae45dd2858458 in_seq=0 out_seq=0 2024-09-19T07:05:59.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.670+0000 7fec86ffd640 10 client.21375 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:05:59.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.671+0000 7fec86ffd640 1 -- 172.21.15.103:0/1106050094 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618af130860 con 0x5618af1dc450 2024-09-19T07:05:59.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.671+0000 7fec86ffd640 10 client.21375 handle_command_reply: tid=0 2024-09-19T07:05:59.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.671+0000 7fecaff94740 1 client.21375 shutdown 2024-09-19T07:05:59.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.671+0000 7fecaff94740 1 -- 172.21.15.103:0/1106050094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1beb40 msgr2=0x5618af1cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:59.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.671+0000 7fecaff94740 1 --2- 172.21.15.103:0/1106050094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1beb40 0x5618af1cccc0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c8f0 tx=0x7fec9009c0f0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.671+0000 7fecaff94740 1 -- 172.21.15.103:0/1106050094 shutdown_connections 2024-09-19T07:05:59.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.671+0000 7fecaff94740 1 --2- 172.21.15.103:0/1106050094 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5618af1dc450 0x5618af1de840 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.671+0000 7fecaff94740 1 --2- 172.21.15.103:0/1106050094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1c8370 0x5618af1d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.671+0000 7fecaff94740 1 --2- 172.21.15.103:0/1106050094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1beb40 0x5618af1cccc0 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.671+0000 7fecaff94740 1 --2- 172.21.15.103:0/1106050094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1bd5d0 0x5618af1cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.671+0000 7fecaff94740 1 -- 172.21.15.103:0/1106050094 >> 172.21.15.103:0/1106050094 conn(0x5618af186310 msgr2=0x5618af1bf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:59.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.671+0000 7fec86ffd640 0 client.21375 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:05:59.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.671+0000 7fecaff94740 1 -- 172.21.15.103:0/1106050094 shutdown_connections 2024-09-19T07:05:59.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.671+0000 7fecaff94740 1 -- 172.21.15.103:0/1106050094 wait complete. 2024-09-19T07:05:59.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.671+0000 7fecaff94740 20 client.21375 trim_cache size 0 max 16384 2024-09-19T07:05:59.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.684+0000 7fecaff94740 1 Processor -- start 2024-09-19T07:05:59.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.684+0000 7fecaff94740 1 -- start start 2024-09-19T07:05:59.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.684+0000 7fecaff94740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1bd5d0 0x5618af1d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.684+0000 7fecaff94740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1beb40 0x5618af1b9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.684+0000 7fecaff94740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1c8370 0x5618af1d8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.684+0000 7fecaff94740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feca81ab5f0 con 0x5618af1c8370 2024-09-19T07:05:59.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.684+0000 7fecaff94740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feca81ad130 con 0x5618af1bd5d0 2024-09-19T07:05:59.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.684+0000 7fecaff94740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feca81abcc0 con 0x5618af1beb40 2024-09-19T07:05:59.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.684+0000 7feca7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1c8370 0x5618af1d8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:59.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.684+0000 7feca7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1c8370 0x5618af1d8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34354/0 (socket says 172.21.15.103:34354) 2024-09-19T07:05:59.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.684+0000 7feca7fff640 1 -- 172.21.15.103:0/278112329 learned_addr learned my addr 172.21.15.103:0/278112329 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:59.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.685+0000 7feca7fff640 1 -- 172.21.15.103:0/278112329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1beb40 msgr2=0x5618af1b9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:59.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.685+0000 7feca7fff640 1 --2- 172.21.15.103:0/278112329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1beb40 0x5618af1b9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.685+0000 7feca7fff640 1 -- 172.21.15.103:0/278112329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1bd5d0 msgr2=0x5618af1d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:59.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.685+0000 7feca7fff640 1 --2- 172.21.15.103:0/278112329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1bd5d0 0x5618af1d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.685+0000 7feca7fff640 1 -- 172.21.15.103:0/278112329 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618af1be760 con 0x5618af1c8370 2024-09-19T07:05:59.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.685+0000 7feca7fff640 1 --2- 172.21.15.103:0/278112329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1c8370 0x5618af1d8cc0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7feca000a500 tx=0x7feca0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3af0a2002cb1958 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:59.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.685+0000 7fec9d7fa640 1 -- 172.21.15.103:0/278112329 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0004d20 con 0x5618af1c8370 2024-09-19T07:05:59.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.685+0000 7fec9d7fa640 1 -- 172.21.15.103:0/278112329 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca001db10 con 0x5618af1c8370 2024-09-19T07:05:59.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.685+0000 7fec9d7fa640 1 -- 172.21.15.103:0/278112329 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca00056e0 con 0x5618af1c8370 2024-09-19T07:05:59.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 -- 172.21.15.103:0/278112329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1c8370 msgr2=0x5618af1d8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:59.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 --2- 172.21.15.103:0/278112329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1c8370 0x5618af1d8cc0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7feca000a500 tx=0x7feca0011000 comp rx=0 tx=0).stop 2024-09-19T07:05:59.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 -- 172.21.15.103:0/278112329 shutdown_connections 2024-09-19T07:05:59.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 --2- 172.21.15.103:0/278112329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1c8370 0x5618af1d8cc0 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 --2- 172.21.15.103:0/278112329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1beb40 0x5618af1b9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 --2- 172.21.15.103:0/278112329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1bd5d0 0x5618af1d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 -- 172.21.15.103:0/278112329 >> 172.21.15.103:0/278112329 conn(0x5618af186310 msgr2=0x5618af1b9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:59.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 -- 172.21.15.103:0/278112329 shutdown_connections 2024-09-19T07:05:59.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 -- 172.21.15.103:0/278112329 wait complete. 2024-09-19T07:05:59.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 -- start start 2024-09-19T07:05:59.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 Processor -- start 2024-09-19T07:05:59.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1bd5d0 0x5618af1d8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1beb40 0x5618af1d4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1c8370 0x5618af1d4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feca81aca60 con 0x5618af1bd5d0 2024-09-19T07:05:59.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feca81ac390 con 0x5618af1beb40 2024-09-19T07:05:59.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feca81a95f0 con 0x5618af1c8370 2024-09-19T07:05:59.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.686+0000 7fecaff94740 10 client.0 _ll_register_callbacks cb 0x7fec9000a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:05:59.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7feca77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1bd5d0 0x5618af1d8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:59.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7feca77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1bd5d0 0x5618af1d8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34358/0 (socket says 172.21.15.103:34358) 2024-09-19T07:05:59.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7feca77fe640 1 -- 172.21.15.103:0/391764318 learned_addr learned my addr 172.21.15.103:0/391764318 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:05:59.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7fec9ffff640 1 --2- 172.21.15.103:0/391764318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1beb40 0x5618af1d4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:59.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7feca7fff640 1 --2- 172.21.15.103:0/391764318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1c8370 0x5618af1d4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:59.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7feca77fe640 1 -- 172.21.15.103:0/391764318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1c8370 msgr2=0x5618af1d4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:59.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7feca77fe640 1 --2- 172.21.15.103:0/391764318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1c8370 0x5618af1d4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7feca77fe640 1 -- 172.21.15.103:0/391764318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1beb40 msgr2=0x5618af1d4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:05:59.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7feca77fe640 1 --2- 172.21.15.103:0/391764318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1beb40 0x5618af1d4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7feca77fe640 1 -- 172.21.15.103:0/391764318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618af1d4d70 con 0x5618af1bd5d0 2024-09-19T07:05:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7feca7fff640 1 --2- 172.21.15.103:0/391764318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1c8370 0x5618af1d4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:05:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7feca77fe640 1 --2- 172.21.15.103:0/391764318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1bd5d0 0x5618af1d8d00 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fec98009900 tx=0x7fec9802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80b4159d5b57086 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:05:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7fec86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:05:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7fec86ffd640 1 -- 172.21.15.103:0/391764318 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98040040 con 0x5618af1bd5d0 2024-09-19T07:05:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7fec86ffd640 1 -- 172.21.15.103:0/391764318 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fec98005450 con 0x5618af1bd5d0 2024-09-19T07:05:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7fecaff94740 1 -- 172.21.15.103:0/391764318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca001db10 con 0x5618af1bd5d0 2024-09-19T07:05:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7fec86ffd640 1 -- 172.21.15.103:0/391764318 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fec98035040 con 0x5618af1bd5d0 2024-09-19T07:05:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7fecaff94740 10 client.21381 fetch_fsmap learned FSMap version 27 2024-09-19T07:05:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7fecaff94740 1 -- 172.21.15.103:0/391764318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618af247040 con 0x5618af1bd5d0 2024-09-19T07:05:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.687+0000 7fec86ffd640 1 -- 172.21.15.103:0/391764318 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca001db10 con 0x5618af1bd5d0 2024-09-19T07:05:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.688+0000 7fec86ffd640 1 -- 172.21.15.103:0/391764318 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fec98004e20 con 0x5618af1bd5d0 2024-09-19T07:05:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.688+0000 7fecaff94740 10 client.21381 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:05:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.688+0000 7fecaff94740 10 client.21381 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:05:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.688+0000 7fecaff94740 20 client.21381 populate_metadata read hostname 'smithi103' 2024-09-19T07:05:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.688+0000 7fecaff94740 1 --2- 172.21.15.103:0/391764318 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5618af2473a0 0x5618af249790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:05:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.688+0000 7fecaff94740 4 client.21381 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:05:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.688+0000 7fecaff94740 1 -- 172.21.15.103:0/391764318 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618af12ad70 con 0x5618af2473a0 2024-09-19T07:05:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.688+0000 7fec9ffff640 1 --2- 172.21.15.103:0/391764318 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5618af2473a0 0x5618af249790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:05:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.689+0000 7fec9ffff640 1 --2- 172.21.15.103:0/391764318 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5618af2473a0 0x5618af249790 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6ab4270f8811eead server_cookie=83bed3503b4d57ce in_seq=0 out_seq=0 2024-09-19T07:05:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.689+0000 7fec86ffd640 10 client.21381 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:05:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.689+0000 7fec86ffd640 1 -- 172.21.15.103:0/391764318 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618af12ad70 con 0x5618af2473a0 2024-09-19T07:05:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.689+0000 7fec86ffd640 10 client.21381 handle_command_reply: tid=0 2024-09-19T07:05:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.689+0000 7fecaff94740 1 client.21381 shutdown 2024-09-19T07:05:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.689+0000 7fecaff94740 1 -- 172.21.15.103:0/391764318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1bd5d0 msgr2=0x5618af1d8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.689+0000 7fecaff94740 1 --2- 172.21.15.103:0/391764318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1bd5d0 0x5618af1d8d00 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fec98009900 tx=0x7fec9802a480 comp rx=0 tx=0).stop 2024-09-19T07:05:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.689+0000 7fecaff94740 1 -- 172.21.15.103:0/391764318 shutdown_connections 2024-09-19T07:05:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.689+0000 7fecaff94740 1 --2- 172.21.15.103:0/391764318 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x5618af2473a0 0x5618af249790 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.689+0000 7fecaff94740 1 --2- 172.21.15.103:0/391764318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618af1c8370 0x5618af1d4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.689+0000 7fecaff94740 1 --2- 172.21.15.103:0/391764318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618af1beb40 0x5618af1d4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.689+0000 7fecaff94740 1 --2- 172.21.15.103:0/391764318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618af1bd5d0 0x5618af1d8d00 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.689+0000 7fecaff94740 1 -- 172.21.15.103:0/391764318 >> 172.21.15.103:0/391764318 conn(0x5618af186310 msgr2=0x5618af1bf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:59.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.689+0000 7fec86ffd640 0 client.21381 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:05:59.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.689+0000 7fecaff94740 1 -- 172.21.15.103:0/391764318 shutdown_connections 2024-09-19T07:05:59.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.689+0000 7fecaff94740 1 -- 172.21.15.103:0/391764318 wait complete. 2024-09-19T07:05:59.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.690+0000 7fecaff94740 20 client.21381 trim_cache size 0 max 16384 2024-09-19T07:05:59.695 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:05:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.690+0000 7fecadbbb640 1 -- 172.21.15.103:0/2389497865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fec940a3430 msgr2=0x7fec940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.690+0000 7fecadbbb640 1 --2- 172.21.15.103:0/2389497865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fec940a3430 0x7fec940a58f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feca000ad10 tx=0x7feca000a2b0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.690+0000 7fecadbbb640 1 -- 172.21.15.103:0/2389497865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca8103700 msgr2=0x7feca8198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:05:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.690+0000 7fecadbbb640 1 --2- 172.21.15.103:0/2389497865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca8103700 0x7feca8198900 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c920 tx=0x7fec9000cde0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.690+0000 7fecadbbb640 1 -- 172.21.15.103:0/2389497865 shutdown_connections 2024-09-19T07:05:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.690+0000 7fecadbbb640 1 --2- 172.21.15.103:0/2389497865 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fec940a3430 0x7fec940a58f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.690+0000 7fecadbbb640 1 --2- 172.21.15.103:0/2389497865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feca810c180 0x7feca819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.690+0000 7fecadbbb640 1 --2- 172.21.15.103:0/2389497865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feca8103700 0x7feca8198900 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.690+0000 7fecadbbb640 1 --2- 172.21.15.103:0/2389497865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feca8102de0 0x7feca81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:05:59.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.690+0000 7fecadbbb640 1 -- 172.21.15.103:0/2389497865 >> 172.21.15.103:0/2389497865 conn(0x7feca80fc7a0 msgr2=0x7feca80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:05:59.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.690+0000 7fecadbbb640 1 -- 172.21.15.103:0/2389497865 shutdown_connections 2024-09-19T07:05:59.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:05:59.690+0000 7fecadbbb640 1 -- 172.21.15.103:0/2389497865 wait complete. 2024-09-19T07:05:59.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:05:59 smithi174 ceph-mon[35637]: pgmap v523: 97 pgs: 97 active+clean; 4.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T07:05:59.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:05:59 smithi134 ceph-mon[35356]: pgmap v523: 97 pgs: 97 active+clean; 4.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-09-19T07:06:00.319 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:06:00.565 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:01.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.002+0000 7f21ae9fe640 1 -- 172.21.15.103:0/1084099969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a8106a80 msgr2=0x7f21a8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:01.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.002+0000 7f21ae9fe640 1 --2- 172.21.15.103:0/1084099969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a8106a80 0x7f21a8106e60 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f2198009930 tx=0x7f219802a480 comp rx=0 tx=0).stop 2024-09-19T07:06:01.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.002+0000 7f21ae9fe640 1 -- 172.21.15.103:0/1084099969 shutdown_connections 2024-09-19T07:06:01.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.002+0000 7f21ae9fe640 1 --2- 172.21.15.103:0/1084099969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21a8101420 0x7f21a810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.002+0000 7f21ae9fe640 1 --2- 172.21.15.103:0/1084099969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a8100a80 0x7f21a8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.002+0000 7f21ae9fe640 1 --2- 172.21.15.103:0/1084099969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a8106a80 0x7f21a8106e60 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.002+0000 7f21ae9fe640 1 -- 172.21.15.103:0/1084099969 >> 172.21.15.103:0/1084099969 conn(0x7f21a80fc7a0 msgr2=0x7f21a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:01.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.002+0000 7f21ae9fe640 1 -- 172.21.15.103:0/1084099969 shutdown_connections 2024-09-19T07:06:01.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.002+0000 7f21ae9fe640 1 -- 172.21.15.103:0/1084099969 wait complete. 2024-09-19T07:06:01.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21ae9fe640 1 Processor -- start 2024-09-19T07:06:01.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21ae9fe640 1 -- start start 2024-09-19T07:06:01.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21ae9fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a8100a80 0x7f21a81972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21ae9fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21a8101420 0x7f21a81977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21ae9fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a8106a80 0x7f21a8197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21ae9fe640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21a8110f60 con 0x7f21a8106a80 2024-09-19T07:06:01.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21ae9fe640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21a8110de0 con 0x7f21a8101420 2024-09-19T07:06:01.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21ae9fe640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21a81110e0 con 0x7f21a8100a80 2024-09-19T07:06:01.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21acf9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a8106a80 0x7f21a8197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21acf9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a8106a80 0x7f21a8197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34372/0 (socket says 172.21.15.103:34372) 2024-09-19T07:06:01.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21acf9b640 1 -- 172.21.15.103:0/1314829535 learned_addr learned my addr 172.21.15.103:0/1314829535 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:01.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21a77fe640 1 --2- 172.21.15.103:0/1314829535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21a8101420 0x7f21a81977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21a7fff640 1 --2- 172.21.15.103:0/1314829535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a8100a80 0x7f21a81972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21acf9b640 1 -- 172.21.15.103:0/1314829535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a8100a80 msgr2=0x7f21a81972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:01.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21acf9b640 1 --2- 172.21.15.103:0/1314829535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a8100a80 0x7f21a81972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21acf9b640 1 -- 172.21.15.103:0/1314829535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21a8101420 msgr2=0x7f21a81977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:01.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.003+0000 7f21acf9b640 1 --2- 172.21.15.103:0/1314829535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21a8101420 0x7f21a81977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.004+0000 7f21acf9b640 1 -- 172.21.15.103:0/1314829535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a81a6b10 con 0x7f21a8106a80 2024-09-19T07:06:01.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.004+0000 7f21a77fe640 1 --2- 172.21.15.103:0/1314829535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21a8101420 0x7f21a81977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:06:01.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.004+0000 7f21acf9b640 1 --2- 172.21.15.103:0/1314829535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a8106a80 0x7f21a8197d20 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f21a000b990 tx=0x7f21a000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:01.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.004+0000 7f21a57fa640 1 -- 172.21.15.103:0/1314829535 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0019070 con 0x7f21a8106a80 2024-09-19T07:06:01.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.004+0000 7f21a57fa640 1 -- 172.21.15.103:0/1314829535 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21a000cba0 con 0x7f21a8106a80 2024-09-19T07:06:01.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.004+0000 7f21ae9fe640 1 -- 172.21.15.103:0/1314829535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a81a6e00 con 0x7f21a8106a80 2024-09-19T07:06:01.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.004+0000 7f21a57fa640 1 -- 172.21.15.103:0/1314829535 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0007650 con 0x7f21a8106a80 2024-09-19T07:06:01.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.004+0000 7f21ae9fe640 1 -- 172.21.15.103:0/1314829535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a81a7250 con 0x7f21a8106a80 2024-09-19T07:06:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.006+0000 7f21a57fa640 1 -- 172.21.15.103:0/1314829535 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f21a001c070 con 0x7f21a8106a80 2024-09-19T07:06:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.006+0000 7f21b0dd7740 1 Processor -- start 2024-09-19T07:06:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21b0dd7740 1 -- start start 2024-09-19T07:06:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21b0dd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed59fb40 0x555aed59e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21b0dd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59e5d0 0x555aed59ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21b0dd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed5a9370 0x555aed5ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21b0dd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555aed5042f0 con 0x555aed59e5d0 2024-09-19T07:06:01.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21b0dd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555aed4abcf0 con 0x555aed59fb40 2024-09-19T07:06:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21b0dd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555aed5002b0 con 0x555aed5a9370 2024-09-19T07:06:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21a57fa640 1 --2- 172.21.15.103:0/1314829535 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21940a33b0 0x7f21940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21a7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59e5d0 0x555aed59ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21a7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59e5d0 0x555aed59ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34380/0 (socket says 172.21.15.103:34380) 2024-09-19T07:06:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21a7fff640 1 -- 172.21.15.103:0/1176781472 learned_addr learned my addr 172.21.15.103:0/1176781472 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21acf9b640 1 --2- 172.21.15.103:0/1314829535 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21940a33b0 0x7f21940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21a77fe640 1 --2- 172.21.15.103:0/1176781472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed5a9370 0x555aed5ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21a77fe640 1 --2- 172.21.15.103:0/1176781472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed59fb40 0x555aed59e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21a57fa640 1 -- 172.21.15.103:0/1314829535 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f21a00cf270 con 0x7f21a8106a80 2024-09-19T07:06:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21a7fff640 1 -- 172.21.15.103:0/1176781472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed5a9370 msgr2=0x555aed5ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21a7fff640 1 --2- 172.21.15.103:0/1176781472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed5a9370 0x555aed5ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21a7fff640 1 -- 172.21.15.103:0/1176781472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed59fb40 msgr2=0x555aed59e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:01.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21a7fff640 1 --2- 172.21.15.103:0/1176781472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed59fb40 0x555aed59e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.007+0000 7f21a7fff640 1 -- 172.21.15.103:0/1176781472 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aed4903a0 con 0x555aed59e5d0 2024-09-19T07:06:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.008+0000 7f21a7fff640 1 --2- 172.21.15.103:0/1176781472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59e5d0 0x555aed59ea10 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f2198005c40 tx=0x7f2198032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a762becbeaff78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.008+0000 7f21acf9b640 1 --2- 172.21.15.103:0/1314829535 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21940a33b0 0x7f21940a5870 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f21a00ccb60 tx=0x7f21a006e470 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.008+0000 7f219e7fc640 1 -- 172.21.15.103:0/1176781472 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219803f070 con 0x555aed59e5d0 2024-09-19T07:06:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.008+0000 7f219e7fc640 1 -- 172.21.15.103:0/1176781472 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21980317b0 con 0x555aed59e5d0 2024-09-19T07:06:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.008+0000 7f219e7fc640 1 -- 172.21.15.103:0/1176781472 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198035070 con 0x555aed59e5d0 2024-09-19T07:06:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.008+0000 7f21b0dd7740 1 -- 172.21.15.103:0/1176781472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59e5d0 msgr2=0x555aed59ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.008+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/1176781472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59e5d0 0x555aed59ea10 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f2198005c40 tx=0x7f2198032cb0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.008+0000 7f21b0dd7740 1 -- 172.21.15.103:0/1176781472 shutdown_connections 2024-09-19T07:06:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.008+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/1176781472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed5a9370 0x555aed5ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.008+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/1176781472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59e5d0 0x555aed59ea10 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.008+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/1176781472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed59fb40 0x555aed59e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.009+0000 7f21b0dd7740 1 -- 172.21.15.103:0/1176781472 >> 172.21.15.103:0/1176781472 conn(0x555aed567310 msgr2=0x555aed59ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.009+0000 7f21b0dd7740 1 -- 172.21.15.103:0/1176781472 shutdown_connections 2024-09-19T07:06:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.009+0000 7f21b0dd7740 1 -- 172.21.15.103:0/1176781472 wait complete. 2024-09-19T07:06:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.009+0000 7f21b0dd7740 1 -- start start 2024-09-19T07:06:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.009+0000 7f21b0dd7740 1 Processor -- start 2024-09-19T07:06:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.009+0000 7f21b0dd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59e5d0 0x555aed5a8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.009+0000 7f21b0dd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed59fb40 0x555aed5a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.009+0000 7f21b0dd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed5a9370 0x555aed5a6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.009+0000 7f21b0dd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555aed5ac010 con 0x555aed59e5d0 2024-09-19T07:06:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.009+0000 7f21b0dd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555aed4bf060 con 0x555aed5a9370 2024-09-19T07:06:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.009+0000 7f21b0dd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555aed5ac190 con 0x555aed59fb40 2024-09-19T07:06:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.009+0000 7f21b0dd7740 10 client.0 _ll_register_callbacks cb 0x555aed2128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21a77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59e5d0 0x555aed5a8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21a77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59e5d0 0x555aed5a8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34384/0 (socket says 172.21.15.103:34384) 2024-09-19T07:06:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21a77fe640 1 -- 172.21.15.103:0/2082255211 learned_addr learned my addr 172.21.15.103:0/2082255211 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21a7fff640 1 --2- 172.21.15.103:0/2082255211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed59fb40 0x555aed5a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21a77fe640 1 --2- 172.21.15.103:0/2082255211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed5a9370 0x555aed5a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21a77fe640 1 -- 172.21.15.103:0/2082255211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed59fb40 msgr2=0x555aed5a5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21a77fe640 1 --2- 172.21.15.103:0/2082255211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed59fb40 0x555aed5a5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21a77fe640 1 -- 172.21.15.103:0/2082255211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed5a9370 msgr2=0x555aed5a6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21a77fe640 1 --2- 172.21.15.103:0/2082255211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed5a9370 0x555aed5a6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21a77fe640 1 -- 172.21.15.103:0/2082255211 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aed2753c0 con 0x555aed59e5d0 2024-09-19T07:06:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21a77fe640 1 --2- 172.21.15.103:0/2082255211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59e5d0 0x555aed5a8cf0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f219000aee0 tx=0x7f2190004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86026ad2795eed5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21877fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21877fe640 1 -- 172.21.15.103:0/2082255211 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190014560 con 0x555aed59e5d0 2024-09-19T07:06:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21877fe640 1 -- 172.21.15.103:0/2082255211 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f2190014700 con 0x555aed59e5d0 2024-09-19T07:06:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2082255211 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21980317b0 con 0x555aed59e5d0 2024-09-19T07:06:01.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21877fe640 1 -- 172.21.15.103:0/2082255211 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2190026050 con 0x555aed59e5d0 2024-09-19T07:06:01.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21b0dd7740 10 client.21393 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:01.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.010+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2082255211 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555aed5b7b20 con 0x555aed59e5d0 2024-09-19T07:06:01.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.011+0000 7f21877fe640 1 -- 172.21.15.103:0/2082255211 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21980317b0 con 0x555aed59e5d0 2024-09-19T07:06:01.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.011+0000 7f21877fe640 1 -- 172.21.15.103:0/2082255211 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2190015040 con 0x555aed59e5d0 2024-09-19T07:06:01.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.011+0000 7f21b0dd7740 10 client.21393 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:01.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.011+0000 7f21b0dd7740 10 client.21393 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:06:01.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.011+0000 7f21b0dd7740 20 client.21393 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:01.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.011+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/2082255211 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x555aed5b8fe0 0x555aed5bb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.011+0000 7f21b0dd7740 4 client.21393 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:06:01.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.011+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2082255211 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555aed511860 con 0x555aed5b8fe0 2024-09-19T07:06:01.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.011+0000 7f21a7fff640 1 --2- 172.21.15.103:0/2082255211 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x555aed5b8fe0 0x555aed5bb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.012+0000 7f21877fe640 10 client.21393 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:06:01.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.012+0000 7f21a7fff640 1 --2- 172.21.15.103:0/2082255211 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x555aed5b8fe0 0x555aed5bb3d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b873fa012a77d354 server_cookie=d313e4243181c768 in_seq=0 out_seq=0 2024-09-19T07:06:01.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.012+0000 7f21877fe640 1 -- 172.21.15.103:0/2082255211 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555aed511860 con 0x555aed5b8fe0 2024-09-19T07:06:01.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.012+0000 7f21877fe640 10 client.21393 handle_command_reply: tid=0 2024-09-19T07:06:01.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.012+0000 7f21b0dd7740 1 client.21393 shutdown 2024-09-19T07:06:01.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.012+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2082255211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59e5d0 msgr2=0x555aed5a8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:01.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.012+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/2082255211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59e5d0 0x555aed5a8cf0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f219000aee0 tx=0x7f2190004750 comp rx=0 tx=0).stop 2024-09-19T07:06:01.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.013+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2082255211 shutdown_connections 2024-09-19T07:06:01.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.013+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/2082255211 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x555aed5b8fe0 0x555aed5bb3d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.013+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/2082255211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed5a9370 0x555aed5a6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.013+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/2082255211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed59fb40 0x555aed5a5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.013+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/2082255211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59e5d0 0x555aed5a8cf0 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.013+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2082255211 >> 172.21.15.103:0/2082255211 conn(0x555aed567310 msgr2=0x555aed5a0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:01.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.013+0000 7f21877fe640 0 client.21393 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:06:01.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.013+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2082255211 shutdown_connections 2024-09-19T07:06:01.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.013+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2082255211 wait complete. 2024-09-19T07:06:01.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.013+0000 7f21b0dd7740 20 client.21393 trim_cache size 0 max 16384 2024-09-19T07:06:01.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.025+0000 7f21b0dd7740 1 Processor -- start 2024-09-19T07:06:01.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.025+0000 7f21b0dd7740 1 -- start start 2024-09-19T07:06:01.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21b0dd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed59e5d0 0x555aed5b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21b0dd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59fb40 0x555aed5a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21b0dd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed5a9370 0x555aed5a1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21b0dd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21a81afb60 con 0x555aed59fb40 2024-09-19T07:06:01.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21b0dd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21a81b16a0 con 0x555aed59e5d0 2024-09-19T07:06:01.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21b0dd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21a81b0230 con 0x555aed5a9370 2024-09-19T07:06:01.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21a7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59fb40 0x555aed5a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21a7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59fb40 0x555aed5a12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34386/0 (socket says 172.21.15.103:34386) 2024-09-19T07:06:01.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21a7fff640 1 -- 172.21.15.103:0/2814716793 learned_addr learned my addr 172.21.15.103:0/2814716793 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:01.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21a77fe640 1 --2- 172.21.15.103:0/2814716793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed5a9370 0x555aed5a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21a7fff640 1 -- 172.21.15.103:0/2814716793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed5a9370 msgr2=0x555aed5a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:01.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21a7fff640 1 --2- 172.21.15.103:0/2814716793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed5a9370 0x555aed5a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21a7fff640 1 -- 172.21.15.103:0/2814716793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed59e5d0 msgr2=0x555aed5b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:01.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21a7fff640 1 --2- 172.21.15.103:0/2814716793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed59e5d0 0x555aed5b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21a7fff640 1 -- 172.21.15.103:0/2814716793 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aed59f300 con 0x555aed59fb40 2024-09-19T07:06:01.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21a77fe640 1 --2- 172.21.15.103:0/2814716793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed5a9370 0x555aed5a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:01.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f21a7fff640 1 --2- 172.21.15.103:0/2814716793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59fb40 0x555aed5a12f0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f2198005c40 tx=0x7f2198005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0842b11484dcb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:01.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f219dffb640 1 -- 172.21.15.103:0/2814716793 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219803f040 con 0x555aed59fb40 2024-09-19T07:06:01.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f219dffb640 1 -- 172.21.15.103:0/2814716793 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21980053b0 con 0x555aed59fb40 2024-09-19T07:06:01.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.026+0000 7f219dffb640 1 -- 172.21.15.103:0/2814716793 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198035040 con 0x555aed59fb40 2024-09-19T07:06:01.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.027+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2814716793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59fb40 msgr2=0x555aed5a12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:01.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.027+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/2814716793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59fb40 0x555aed5a12f0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f2198005c40 tx=0x7f2198005340 comp rx=0 tx=0).stop 2024-09-19T07:06:01.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.027+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2814716793 shutdown_connections 2024-09-19T07:06:01.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.027+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/2814716793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed5a9370 0x555aed5a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.027+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/2814716793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59fb40 0x555aed5a12f0 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.027+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/2814716793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed59e5d0 0x555aed5b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.027+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2814716793 >> 172.21.15.103:0/2814716793 conn(0x555aed567310 msgr2=0x555aed59a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:01.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.027+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2814716793 shutdown_connections 2024-09-19T07:06:01.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.027+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2814716793 wait complete. 2024-09-19T07:06:01.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.027+0000 7f21b0dd7740 1 -- start start 2024-09-19T07:06:01.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21b0dd7740 1 Processor -- start 2024-09-19T07:06:01.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21b0dd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed59e5d0 0x555aed5ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21b0dd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59fb40 0x555aed5a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21b0dd7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed5a9370 0x555aed5aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21b0dd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21a81b0fd0 con 0x555aed59fb40 2024-09-19T07:06:01.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21b0dd7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21a81b0900 con 0x555aed5a9370 2024-09-19T07:06:01.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21b0dd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21a81adb60 con 0x555aed59e5d0 2024-09-19T07:06:01.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21b0dd7740 10 client.0 _ll_register_callbacks cb 0x7f219000a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:01.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21a7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59fb40 0x555aed5a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21a7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59fb40 0x555aed5a9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34400/0 (socket says 172.21.15.103:34400) 2024-09-19T07:06:01.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21a7fff640 1 -- 172.21.15.103:0/2566826611 learned_addr learned my addr 172.21.15.103:0/2566826611 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:01.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21a77fe640 1 --2- 172.21.15.103:0/2566826611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed59e5d0 0x555aed5ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21a77fe640 1 --2- 172.21.15.103:0/2566826611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed5a9370 0x555aed5aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21a7fff640 1 -- 172.21.15.103:0/2566826611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed59e5d0 msgr2=0x555aed5ab5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:01.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21a7fff640 1 --2- 172.21.15.103:0/2566826611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed59e5d0 0x555aed5ab5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21a7fff640 1 -- 172.21.15.103:0/2566826611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed5a9370 msgr2=0x555aed5aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:01.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21a7fff640 1 --2- 172.21.15.103:0/2566826611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed5a9370 0x555aed5aa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21a7fff640 1 -- 172.21.15.103:0/2566826611 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aed5b0450 con 0x555aed59fb40 2024-09-19T07:06:01.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21a77fe640 1 --2- 172.21.15.103:0/2566826611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed59e5d0 0x555aed5ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:06:01.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21a7fff640 1 --2- 172.21.15.103:0/2566826611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59fb40 0x555aed5a9b10 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f2198005940 tx=0x7f21980318b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2942203ac3d0800 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:01.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.028+0000 7f21877fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:01.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.029+0000 7f21877fe640 1 -- 172.21.15.103:0/2566826611 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198035030 con 0x555aed59fb40 2024-09-19T07:06:01.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.029+0000 7f21877fe640 1 -- 172.21.15.103:0/2566826611 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f2198031be0 con 0x555aed59fb40 2024-09-19T07:06:01.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.029+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2566826611 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21980053b0 con 0x555aed59fb40 2024-09-19T07:06:01.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.029+0000 7f21877fe640 1 -- 172.21.15.103:0/2566826611 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2198064050 con 0x555aed59fb40 2024-09-19T07:06:01.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.029+0000 7f21b0dd7740 10 client.21399 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:01.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.029+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2566826611 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555aed5b16c0 con 0x555aed59fb40 2024-09-19T07:06:01.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.029+0000 7f21877fe640 1 -- 172.21.15.103:0/2566826611 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21980053b0 con 0x555aed59fb40 2024-09-19T07:06:01.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.029+0000 7f21877fe640 1 -- 172.21.15.103:0/2566826611 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2198044450 con 0x555aed59fb40 2024-09-19T07:06:01.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.029+0000 7f21b0dd7740 10 client.21399 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:01.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.029+0000 7f21b0dd7740 10 client.21399 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:06:01.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.029+0000 7f21b0dd7740 20 client.21399 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:01.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.029+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/2566826611 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x555aed5b19a0 0x555aed5b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:01.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.029+0000 7f21b0dd7740 4 client.21399 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:06:01.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.029+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2566826611 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555aed50bd70 con 0x555aed5b19a0 2024-09-19T07:06:01.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.030+0000 7f21a77fe640 1 --2- 172.21.15.103:0/2566826611 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x555aed5b19a0 0x555aed5b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:01.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.030+0000 7f21a77fe640 1 --2- 172.21.15.103:0/2566826611 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x555aed5b19a0 0x555aed5b1d80 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99f49075a6a08d93 server_cookie=a1f4c1548fb0d33e in_seq=0 out_seq=0 2024-09-19T07:06:01.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.030+0000 7f21877fe640 10 client.21399 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:06:01.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.030+0000 7f21877fe640 1 -- 172.21.15.103:0/2566826611 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555aed50bd70 con 0x555aed5b19a0 2024-09-19T07:06:01.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.030+0000 7f21877fe640 10 client.21399 handle_command_reply: tid=0 2024-09-19T07:06:01.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.030+0000 7f21b0dd7740 1 client.21399 shutdown 2024-09-19T07:06:01.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.030+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2566826611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59fb40 msgr2=0x555aed5a9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:01.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.030+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/2566826611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59fb40 0x555aed5a9b10 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f2198005940 tx=0x7f21980318b0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.030+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2566826611 shutdown_connections 2024-09-19T07:06:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.030+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/2566826611 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x555aed5b19a0 0x555aed5b1d80 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.030+0000 7f21877fe640 0 client.21399 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:06:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.030+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/2566826611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555aed5a9370 0x555aed5aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.030+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/2566826611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aed59fb40 0x555aed5a9b10 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.030+0000 7f21b0dd7740 1 --2- 172.21.15.103:0/2566826611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555aed59e5d0 0x555aed5ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.030+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2566826611 >> 172.21.15.103:0/2566826611 conn(0x555aed567310 msgr2=0x555aed599d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.030+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2566826611 shutdown_connections 2024-09-19T07:06:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.031+0000 7f21b0dd7740 1 -- 172.21.15.103:0/2566826611 wait complete. 2024-09-19T07:06:01.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.031+0000 7f21b0dd7740 20 client.21399 trim_cache size 0 max 16384 2024-09-19T07:06:01.038 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:06:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.031+0000 7f21ae9fe640 1 -- 172.21.15.103:0/1314829535 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21940a33b0 msgr2=0x7f21940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.031+0000 7f21ae9fe640 1 --2- 172.21.15.103:0/1314829535 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21940a33b0 0x7f21940a5870 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f21a00ccb60 tx=0x7f21a006e470 comp rx=0 tx=0).stop 2024-09-19T07:06:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.031+0000 7f21ae9fe640 1 -- 172.21.15.103:0/1314829535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a8106a80 msgr2=0x7f21a8197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.031+0000 7f21ae9fe640 1 --2- 172.21.15.103:0/1314829535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a8106a80 0x7f21a8197d20 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f21a000b990 tx=0x7f21a000be50 comp rx=0 tx=0).stop 2024-09-19T07:06:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.031+0000 7f21ae9fe640 1 -- 172.21.15.103:0/1314829535 shutdown_connections 2024-09-19T07:06:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.031+0000 7f21ae9fe640 1 --2- 172.21.15.103:0/1314829535 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21940a33b0 0x7f21940a5870 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.031+0000 7f21ae9fe640 1 --2- 172.21.15.103:0/1314829535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a8106a80 0x7f21a8197d20 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.031+0000 7f21ae9fe640 1 --2- 172.21.15.103:0/1314829535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21a8101420 0x7f21a81977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.031+0000 7f21ae9fe640 1 --2- 172.21.15.103:0/1314829535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a8100a80 0x7f21a81972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:01.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.031+0000 7f21ae9fe640 1 -- 172.21.15.103:0/1314829535 >> 172.21.15.103:0/1314829535 conn(0x7f21a80fc7a0 msgr2=0x7f21a80fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:01.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.031+0000 7f21ae9fe640 1 -- 172.21.15.103:0/1314829535 shutdown_connections 2024-09-19T07:06:01.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:01.031+0000 7f21ae9fe640 1 -- 172.21.15.103:0/1314829535 wait complete. 2024-09-19T07:06:01.718 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:01 smithi103 ceph-mon[31158]: pgmap v524: 97 pgs: 97 active+clean; 4.9 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 21 op/s 2024-09-19T07:06:01.746 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:06:01.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:01 smithi174 ceph-mon[35637]: pgmap v524: 97 pgs: 97 active+clean; 4.9 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 21 op/s 2024-09-19T07:06:01.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:01 smithi134 ceph-mon[35356]: pgmap v524: 97 pgs: 97 active+clean; 4.9 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 21 op/s 2024-09-19T07:06:01.994 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:02.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.360+0000 7f367039e640 1 -- 172.21.15.103:0/2264035076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3668102ef0 msgr2=0x7f366810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:02.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.360+0000 7f367039e640 1 --2- 172.21.15.103:0/2264035076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3668102ef0 0x7f366810da40 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f3658009ca0 tx=0x7f365802a950 comp rx=0 tx=0).stop 2024-09-19T07:06:02.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.360+0000 7f367039e640 1 -- 172.21.15.103:0/2264035076 shutdown_connections 2024-09-19T07:06:02.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.360+0000 7f367039e640 1 --2- 172.21.15.103:0/2264035076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3668102ef0 0x7f366810da40 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.360+0000 7f367039e640 1 --2- 172.21.15.103:0/2264035076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3668102570 0x7f36681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.360+0000 7f367039e640 1 --2- 172.21.15.103:0/2264035076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3668100e20 0x7f3668101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.360+0000 7f367039e640 1 -- 172.21.15.103:0/2264035076 >> 172.21.15.103:0/2264035076 conn(0x7f36680fc7d0 msgr2=0x7f36680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:02.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.360+0000 7f367039e640 1 -- 172.21.15.103:0/2264035076 shutdown_connections 2024-09-19T07:06:02.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.360+0000 7f367039e640 1 -- 172.21.15.103:0/2264035076 wait complete. 2024-09-19T07:06:02.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.360+0000 7f367039e640 1 Processor -- start 2024-09-19T07:06:02.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f367039e640 1 -- start start 2024-09-19T07:06:02.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f367039e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3668100e20 0x7f3668199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f367039e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3668102570 0x7f3668199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f367039e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3668102ef0 0x7f36681a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f367039e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3668111020 con 0x7f3668102ef0 2024-09-19T07:06:02.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f367039e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3668110ea0 con 0x7f3668100e20 2024-09-19T07:06:02.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f367039e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36681111a0 con 0x7f3668102570 2024-09-19T07:06:02.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f366e93b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3668102ef0 0x7f36681a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:02.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f366e93b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3668102ef0 0x7f36681a44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34416/0 (socket says 172.21.15.103:34416) 2024-09-19T07:06:02.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f366e93b640 1 -- 172.21.15.103:0/1086161414 learned_addr learned my addr 172.21.15.103:0/1086161414 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:02.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f366d939640 1 --2- 172.21.15.103:0/1086161414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3668102570 0x7f3668199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:02.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f366e13a640 1 --2- 172.21.15.103:0/1086161414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3668100e20 0x7f3668199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:02.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f366e93b640 1 -- 172.21.15.103:0/1086161414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3668102570 msgr2=0x7f3668199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:02.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f366e93b640 1 --2- 172.21.15.103:0/1086161414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3668102570 0x7f3668199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f366e93b640 1 -- 172.21.15.103:0/1086161414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3668100e20 msgr2=0x7f3668199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:02.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f366e93b640 1 --2- 172.21.15.103:0/1086161414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3668100e20 0x7f3668199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.361+0000 7f366e93b640 1 -- 172.21.15.103:0/1086161414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36681a4a10 con 0x7f3668102ef0 2024-09-19T07:06:02.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.362+0000 7f366e93b640 1 --2- 172.21.15.103:0/1086161414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3668102ef0 0x7f36681a44d0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f3658006010 tx=0x7f365802f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:02.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.362+0000 7f36677fe640 1 -- 172.21.15.103:0/1086161414 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658034040 con 0x7f3668102ef0 2024-09-19T07:06:02.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.362+0000 7f36677fe640 1 -- 172.21.15.103:0/1086161414 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3658003e00 con 0x7f3668102ef0 2024-09-19T07:06:02.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.362+0000 7f367039e640 1 -- 172.21.15.103:0/1086161414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3668197240 con 0x7f3668102ef0 2024-09-19T07:06:02.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.362+0000 7f36677fe640 1 -- 172.21.15.103:0/1086161414 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658045480 con 0x7f3668102ef0 2024-09-19T07:06:02.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.362+0000 7f367039e640 1 -- 172.21.15.103:0/1086161414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3668197720 con 0x7f3668102ef0 2024-09-19T07:06:02.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.364+0000 7f36677fe640 1 -- 172.21.15.103:0/1086161414 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3658037020 con 0x7f3668102ef0 2024-09-19T07:06:02.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.364+0000 7f3672777740 1 Processor -- start 2024-09-19T07:06:02.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f3672777740 1 -- start start 2024-09-19T07:06:02.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f36677fe640 1 --2- 172.21.15.103:0/1086161414 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f36500a33b0 0x7f36500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f3672777740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f378b40 0x55c36f377090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f3672777740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f3775d0 0x55c36f377a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f3672777740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f382370 0x55c36f384760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f3672777740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c36f2dd2f0 con 0x55c36f378b40 2024-09-19T07:06:02.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f3672777740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c36f284cf0 con 0x55c36f382370 2024-09-19T07:06:02.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f3672777740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c36f2d92b0 con 0x55c36f3775d0 2024-09-19T07:06:02.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f36677fe640 1 -- 172.21.15.103:0/1086161414 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f36580f0cb0 con 0x7f3668102ef0 2024-09-19T07:06:02.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f366e13a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f378b40 0x55c36f377090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:02.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f366d939640 1 --2- 172.21.15.103:0/1086161414 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f36500a33b0 0x7f36500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:02.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f366e13a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f378b40 0x55c36f377090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34418/0 (socket says 172.21.15.103:34418) 2024-09-19T07:06:02.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f366e13a640 1 -- 172.21.15.103:0/1441596723 learned_addr learned my addr 172.21.15.103:0/1441596723 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:02.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f366d939640 1 --2- 172.21.15.103:0/1441596723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f3775d0 0x55c36f377a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:02.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f366e13a640 1 -- 172.21.15.103:0/1441596723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f3775d0 msgr2=0x55c36f377a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:02.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f366e13a640 1 --2- 172.21.15.103:0/1441596723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f3775d0 0x55c36f377a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f366e13a640 1 -- 172.21.15.103:0/1441596723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f382370 msgr2=0x55c36f384760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:02.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f366e93b640 1 --2- 172.21.15.103:0/1441596723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f382370 0x55c36f384760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:02.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f366e13a640 1 --2- 172.21.15.103:0/1441596723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f382370 0x55c36f384760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.365+0000 7f366e13a640 1 -- 172.21.15.103:0/1441596723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c36f2693a0 con 0x55c36f378b40 2024-09-19T07:06:02.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.366+0000 7f366d939640 1 --2- 172.21.15.103:0/1086161414 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f36500a33b0 0x7f36500a5870 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f366806aac0 tx=0x7f365c009920 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:02.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.366+0000 7f366e13a640 1 --2- 172.21.15.103:0/1441596723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f378b40 0x55c36f377090 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f3660009a70 tx=0x7f3660009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8607d389bf84d745 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:02.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.366+0000 7f3664dea640 1 -- 172.21.15.103:0/1441596723 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660017070 con 0x55c36f378b40 2024-09-19T07:06:02.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.366+0000 7f3664dea640 1 -- 172.21.15.103:0/1441596723 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3660013470 con 0x55c36f378b40 2024-09-19T07:06:02.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.366+0000 7f3664dea640 1 -- 172.21.15.103:0/1441596723 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660012630 con 0x55c36f378b40 2024-09-19T07:06:02.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 -- 172.21.15.103:0/1441596723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f378b40 msgr2=0x55c36f377090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:02.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 --2- 172.21.15.103:0/1441596723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f378b40 0x55c36f377090 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f3660009a70 tx=0x7f3660009f30 comp rx=0 tx=0).stop 2024-09-19T07:06:02.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 -- 172.21.15.103:0/1441596723 shutdown_connections 2024-09-19T07:06:02.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 --2- 172.21.15.103:0/1441596723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f382370 0x55c36f384760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 --2- 172.21.15.103:0/1441596723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f3775d0 0x55c36f377a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 --2- 172.21.15.103:0/1441596723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f378b40 0x55c36f377090 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 -- 172.21.15.103:0/1441596723 >> 172.21.15.103:0/1441596723 conn(0x55c36f340310 msgr2=0x55c36f373c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:02.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 -- 172.21.15.103:0/1441596723 shutdown_connections 2024-09-19T07:06:02.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 -- 172.21.15.103:0/1441596723 wait complete. 2024-09-19T07:06:02.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 -- start start 2024-09-19T07:06:02.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 Processor -- start 2024-09-19T07:06:02.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f3775d0 0x55c36f381d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f378b40 0x55c36f37f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f382370 0x55c36f37f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c36f385070 con 0x55c36f3775d0 2024-09-19T07:06:02.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c36f298060 con 0x55c36f382370 2024-09-19T07:06:02.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.367+0000 7f3672777740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c36f3851f0 con 0x55c36f378b40 2024-09-19T07:06:02.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f3672777740 10 client.0 _ll_register_callbacks cb 0x55c36efeb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:02.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f366e13a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f3775d0 0x55c36f381d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:02.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f366e13a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f3775d0 0x55c36f381d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34422/0 (socket says 172.21.15.103:34422) 2024-09-19T07:06:02.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f366e13a640 1 -- 172.21.15.103:0/3820898592 learned_addr learned my addr 172.21.15.103:0/3820898592 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:02.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f366e13a640 1 -- 172.21.15.103:0/3820898592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f378b40 msgr2=0x55c36f37f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:02.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f366e13a640 1 --2- 172.21.15.103:0/3820898592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f378b40 0x55c36f37f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f366e13a640 1 -- 172.21.15.103:0/3820898592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f382370 msgr2=0x55c36f37f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:02.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f366e13a640 1 --2- 172.21.15.103:0/3820898592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f382370 0x55c36f37f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f366e13a640 1 -- 172.21.15.103:0/3820898592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c36f04e3c0 con 0x55c36f3775d0 2024-09-19T07:06:02.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f366e13a640 1 --2- 172.21.15.103:0/3820898592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f3775d0 0x55c36f381d50 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f3660004700 tx=0x7f3660006e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eea3bdecc5ada03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:02.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f364dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:02.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f364dffb640 1 -- 172.21.15.103:0/3820898592 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660029060 con 0x55c36f3775d0 2024-09-19T07:06:02.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f3672777740 1 -- 172.21.15.103:0/3820898592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3660013470 con 0x55c36f3775d0 2024-09-19T07:06:02.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f364dffb640 1 -- 172.21.15.103:0/3820898592 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f36600138f0 con 0x55c36f3775d0 2024-09-19T07:06:02.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f364dffb640 1 -- 172.21.15.103:0/3820898592 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3660017040 con 0x55c36f3775d0 2024-09-19T07:06:02.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f3672777740 10 client.21411 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:02.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.368+0000 7f3672777740 1 -- 172.21.15.103:0/3820898592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c36f390af0 con 0x55c36f3775d0 2024-09-19T07:06:02.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.369+0000 7f364dffb640 1 -- 172.21.15.103:0/3820898592 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3660013470 con 0x55c36f3775d0 2024-09-19T07:06:02.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.369+0000 7f364dffb640 1 -- 172.21.15.103:0/3820898592 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f366002c4f0 con 0x55c36f3775d0 2024-09-19T07:06:02.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.369+0000 7f3672777740 10 client.21411 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:02.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.369+0000 7f3672777740 10 client.21411 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:06:02.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.369+0000 7f3672777740 20 client.21411 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:02.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.369+0000 7f3672777740 1 --2- 172.21.15.103:0/3820898592 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c36f391e40 0x55c36f394230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.369+0000 7f3672777740 4 client.21411 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:06:02.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.369+0000 7f3672777740 1 -- 172.21.15.103:0/3820898592 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c36f2ea860 con 0x55c36f391e40 2024-09-19T07:06:02.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.369+0000 7f366d939640 1 --2- 172.21.15.103:0/3820898592 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c36f391e40 0x55c36f394230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:02.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.370+0000 7f364dffb640 10 client.21411 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:06:02.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.370+0000 7f366d939640 1 --2- 172.21.15.103:0/3820898592 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c36f391e40 0x55c36f394230 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fcfa29aca3a768a3 server_cookie=231745e60d7f8dc7 in_seq=0 out_seq=0 2024-09-19T07:06:02.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.370+0000 7f364dffb640 1 -- 172.21.15.103:0/3820898592 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c36f2ea860 con 0x55c36f391e40 2024-09-19T07:06:02.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.370+0000 7f364dffb640 10 client.21411 handle_command_reply: tid=0 2024-09-19T07:06:02.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.370+0000 7f3672777740 1 client.21411 shutdown 2024-09-19T07:06:02.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.371+0000 7f3672777740 1 -- 172.21.15.103:0/3820898592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f3775d0 msgr2=0x55c36f381d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:02.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.371+0000 7f3672777740 1 --2- 172.21.15.103:0/3820898592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f3775d0 0x55c36f381d50 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f3660004700 tx=0x7f3660006e60 comp rx=0 tx=0).stop 2024-09-19T07:06:02.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.371+0000 7f3672777740 1 -- 172.21.15.103:0/3820898592 shutdown_connections 2024-09-19T07:06:02.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.371+0000 7f3672777740 1 --2- 172.21.15.103:0/3820898592 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c36f391e40 0x55c36f394230 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.371+0000 7f3672777740 1 --2- 172.21.15.103:0/3820898592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f382370 0x55c36f37f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.371+0000 7f3672777740 1 --2- 172.21.15.103:0/3820898592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f378b40 0x55c36f37f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.371+0000 7f3672777740 1 --2- 172.21.15.103:0/3820898592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f3775d0 0x55c36f381d50 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.371+0000 7f364dffb640 0 client.21411 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:06:02.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.371+0000 7f3672777740 1 -- 172.21.15.103:0/3820898592 >> 172.21.15.103:0/3820898592 conn(0x55c36f340310 msgr2=0x55c36f379960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:02.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.371+0000 7f3672777740 1 -- 172.21.15.103:0/3820898592 shutdown_connections 2024-09-19T07:06:02.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.371+0000 7f3672777740 1 -- 172.21.15.103:0/3820898592 wait complete. 2024-09-19T07:06:02.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.371+0000 7f3672777740 20 client.21411 trim_cache size 0 max 16384 2024-09-19T07:06:02.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f3672777740 1 Processor -- start 2024-09-19T07:06:02.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f3672777740 1 -- start start 2024-09-19T07:06:02.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f3672777740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f3775d0 0x55c36f38e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f3672777740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f378b40 0x55c36f37e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f3672777740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f382370 0x55c36f37dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f3672777740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36681afb80 con 0x55c36f378b40 2024-09-19T07:06:02.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f3672777740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f36681b16c0 con 0x55c36f3775d0 2024-09-19T07:06:02.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f3672777740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36681b0250 con 0x55c36f382370 2024-09-19T07:06:02.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f366d939640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f378b40 0x55c36f37e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:02.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f366d939640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f378b40 0x55c36f37e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34426/0 (socket says 172.21.15.103:34426) 2024-09-19T07:06:02.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f366d939640 1 -- 172.21.15.103:0/893123083 learned_addr learned my addr 172.21.15.103:0/893123083 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:02.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f366e93b640 1 --2- 172.21.15.103:0/893123083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f382370 0x55c36f37dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:02.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f366d939640 1 -- 172.21.15.103:0/893123083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f382370 msgr2=0x55c36f37dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:02.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f366e13a640 1 --2- 172.21.15.103:0/893123083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f3775d0 0x55c36f38e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:02.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f366d939640 1 --2- 172.21.15.103:0/893123083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f382370 0x55c36f37dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f366d939640 1 -- 172.21.15.103:0/893123083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f3775d0 msgr2=0x55c36f38e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:02.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f366d939640 1 --2- 172.21.15.103:0/893123083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f3775d0 0x55c36f38e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f366d939640 1 -- 172.21.15.103:0/893123083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c36f37abc0 con 0x55c36f378b40 2024-09-19T07:06:02.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f366e93b640 1 --2- 172.21.15.103:0/893123083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f382370 0x55c36f37dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:02.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.384+0000 7f366e13a640 1 --2- 172.21.15.103:0/893123083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f3775d0 0x55c36f38e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:02.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.385+0000 7f366d939640 1 --2- 172.21.15.103:0/893123083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f378b40 0x55c36f37e8c0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x55c36f37e890 tx=0x7f365c01a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b15e4ecc87a336a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:02.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.385+0000 7f364ffff640 1 -- 172.21.15.103:0/893123083 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c033070 con 0x55c36f378b40 2024-09-19T07:06:02.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.385+0000 7f364ffff640 1 -- 172.21.15.103:0/893123083 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f365c01c040 con 0x55c36f378b40 2024-09-19T07:06:02.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.385+0000 7f364ffff640 1 -- 172.21.15.103:0/893123083 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c01e070 con 0x55c36f378b40 2024-09-19T07:06:02.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.385+0000 7f3672777740 1 -- 172.21.15.103:0/893123083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f378b40 msgr2=0x55c36f37e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:02.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.385+0000 7f3672777740 1 --2- 172.21.15.103:0/893123083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f378b40 0x55c36f37e8c0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x55c36f37e890 tx=0x7f365c01a070 comp rx=0 tx=0).stop 2024-09-19T07:06:02.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.385+0000 7f3672777740 1 -- 172.21.15.103:0/893123083 shutdown_connections 2024-09-19T07:06:02.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.385+0000 7f3672777740 1 --2- 172.21.15.103:0/893123083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f382370 0x55c36f37dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.385+0000 7f3672777740 1 --2- 172.21.15.103:0/893123083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f378b40 0x55c36f37e8c0 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.385+0000 7f3672777740 1 --2- 172.21.15.103:0/893123083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f3775d0 0x55c36f38e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.385+0000 7f3672777740 1 -- 172.21.15.103:0/893123083 >> 172.21.15.103:0/893123083 conn(0x55c36f340310 msgr2=0x55c36f37d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:02.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.385+0000 7f3672777740 1 -- 172.21.15.103:0/893123083 shutdown_connections 2024-09-19T07:06:02.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.385+0000 7f3672777740 1 -- 172.21.15.103:0/893123083 wait complete. 2024-09-19T07:06:02.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f3672777740 1 -- start start 2024-09-19T07:06:02.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f3672777740 1 Processor -- start 2024-09-19T07:06:02.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f3672777740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f3775d0 0x55c36f3859e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f3672777740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f378b40 0x55c36f385f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f3672777740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f382370 0x55c36f3fa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f3672777740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36681b0ff0 con 0x55c36f3775d0 2024-09-19T07:06:02.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f3672777740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f36681b0920 con 0x55c36f382370 2024-09-19T07:06:02.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f3672777740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36681adb80 con 0x55c36f378b40 2024-09-19T07:06:02.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f3672777740 10 client.0 _ll_register_callbacks cb 0x7f365c009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:02.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f366e13a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f3775d0 0x55c36f3859e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:02.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f366e13a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f3775d0 0x55c36f3859e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34440/0 (socket says 172.21.15.103:34440) 2024-09-19T07:06:02.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f366e13a640 1 -- 172.21.15.103:0/3874155818 learned_addr learned my addr 172.21.15.103:0/3874155818 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:02.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f366e93b640 1 --2- 172.21.15.103:0/3874155818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f382370 0x55c36f3fa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:02.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f366e13a640 1 -- 172.21.15.103:0/3874155818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f378b40 msgr2=0x55c36f385f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:06:02.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f366e13a640 1 --2- 172.21.15.103:0/3874155818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f378b40 0x55c36f385f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f366e13a640 1 -- 172.21.15.103:0/3874155818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f382370 msgr2=0x55c36f3fa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:02.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f366e13a640 1 --2- 172.21.15.103:0/3874155818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f382370 0x55c36f3fa980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f366e13a640 1 -- 172.21.15.103:0/3874155818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c36f3faec0 con 0x55c36f3775d0 2024-09-19T07:06:02.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.386+0000 7f366e93b640 1 --2- 172.21.15.103:0/3874155818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f382370 0x55c36f3fa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:02.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.387+0000 7f366e13a640 1 --2- 172.21.15.103:0/3874155818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f3775d0 0x55c36f3859e0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f3660004810 tx=0x7f36600094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30ffce0510973cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:02.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.387+0000 7f364dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:02.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.387+0000 7f364dffb640 1 -- 172.21.15.103:0/3874155818 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660029070 con 0x55c36f3775d0 2024-09-19T07:06:02.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.387+0000 7f364dffb640 1 -- 172.21.15.103:0/3874155818 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f3660013af0 con 0x55c36f3775d0 2024-09-19T07:06:02.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.387+0000 7f3672777740 1 -- 172.21.15.103:0/3874155818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f365c01c040 con 0x55c36f3775d0 2024-09-19T07:06:02.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.387+0000 7f364dffb640 1 -- 172.21.15.103:0/3874155818 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f366002c6a0 con 0x55c36f3775d0 2024-09-19T07:06:02.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.387+0000 7f3672777740 10 client.21417 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:02.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.387+0000 7f3672777740 1 -- 172.21.15.103:0/3874155818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c36f3fc0b0 con 0x55c36f3775d0 2024-09-19T07:06:02.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.387+0000 7f364dffb640 1 -- 172.21.15.103:0/3874155818 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f365c01c040 con 0x55c36f3775d0 2024-09-19T07:06:02.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.387+0000 7f364dffb640 1 -- 172.21.15.103:0/3874155818 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f366002d430 con 0x55c36f3775d0 2024-09-19T07:06:02.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.387+0000 7f3672777740 10 client.21417 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:02.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.387+0000 7f3672777740 10 client.21417 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:06:02.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.387+0000 7f3672777740 20 client.21417 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:02.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.387+0000 7f3672777740 1 --2- 172.21.15.103:0/3874155818 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c36f3fc6f0 0x55c36f3feb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:02.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.387+0000 7f3672777740 4 client.21417 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:06:02.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.387+0000 7f3672777740 1 -- 172.21.15.103:0/3874155818 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c36f2e4d70 con 0x55c36f3fc6f0 2024-09-19T07:06:02.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.388+0000 7f366d939640 1 --2- 172.21.15.103:0/3874155818 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c36f3fc6f0 0x55c36f3feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:02.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.388+0000 7f364dffb640 10 client.21417 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:06:02.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.388+0000 7f366d939640 1 --2- 172.21.15.103:0/3874155818 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c36f3fc6f0 0x55c36f3feb10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e262a82becbc0a2 server_cookie=d45643de85ade41c in_seq=0 out_seq=0 2024-09-19T07:06:02.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.388+0000 7f364dffb640 1 -- 172.21.15.103:0/3874155818 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c36f2e4d70 con 0x55c36f3fc6f0 2024-09-19T07:06:02.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.388+0000 7f364dffb640 10 client.21417 handle_command_reply: tid=0 2024-09-19T07:06:02.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.388+0000 7f3672777740 1 client.21417 shutdown 2024-09-19T07:06:02.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.388+0000 7f3672777740 1 -- 172.21.15.103:0/3874155818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f3775d0 msgr2=0x55c36f3859e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:02.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.388+0000 7f3672777740 1 --2- 172.21.15.103:0/3874155818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f3775d0 0x55c36f3859e0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f3660004810 tx=0x7f36600094a0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.388+0000 7f3672777740 1 -- 172.21.15.103:0/3874155818 shutdown_connections 2024-09-19T07:06:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.388+0000 7f3672777740 1 --2- 172.21.15.103:0/3874155818 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x55c36f3fc6f0 0x55c36f3feb10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f3672777740 1 --2- 172.21.15.103:0/3874155818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c36f382370 0x55c36f3fa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f3672777740 1 --2- 172.21.15.103:0/3874155818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c36f378b40 0x55c36f385f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f364dffb640 0 client.21417 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:06:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f3672777740 1 --2- 172.21.15.103:0/3874155818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c36f3775d0 0x55c36f3859e0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f3672777740 1 -- 172.21.15.103:0/3874155818 >> 172.21.15.103:0/3874155818 conn(0x55c36f340310 msgr2=0x55c36f37e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f3672777740 1 -- 172.21.15.103:0/3874155818 shutdown_connections 2024-09-19T07:06:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f3672777740 1 -- 172.21.15.103:0/3874155818 wait complete. 2024-09-19T07:06:02.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f3672777740 20 client.21417 trim_cache size 0 max 16384 2024-09-19T07:06:02.395 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:06:02.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f367039e640 1 -- 172.21.15.103:0/1086161414 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f36500a33b0 msgr2=0x7f36500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:02.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f367039e640 1 --2- 172.21.15.103:0/1086161414 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f36500a33b0 0x7f36500a5870 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f366806aac0 tx=0x7f365c009920 comp rx=0 tx=0).stop 2024-09-19T07:06:02.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f367039e640 1 -- 172.21.15.103:0/1086161414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3668102ef0 msgr2=0x7f36681a44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:02.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f367039e640 1 --2- 172.21.15.103:0/1086161414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3668102ef0 0x7f36681a44d0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f3658006010 tx=0x7f365802f6f0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f367039e640 1 -- 172.21.15.103:0/1086161414 shutdown_connections 2024-09-19T07:06:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f367039e640 1 --2- 172.21.15.103:0/1086161414 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f36500a33b0 0x7f36500a5870 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f367039e640 1 --2- 172.21.15.103:0/1086161414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3668102ef0 0x7f36681a44d0 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f367039e640 1 --2- 172.21.15.103:0/1086161414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3668102570 0x7f3668199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f367039e640 1 --2- 172.21.15.103:0/1086161414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3668100e20 0x7f3668199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f367039e640 1 -- 172.21.15.103:0/1086161414 >> 172.21.15.103:0/1086161414 conn(0x7f36680fc7d0 msgr2=0x7f36680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f367039e640 1 -- 172.21.15.103:0/1086161414 shutdown_connections 2024-09-19T07:06:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:02.389+0000 7f367039e640 1 -- 172.21.15.103:0/1086161414 wait complete. 2024-09-19T07:06:03.130 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:06:03.378 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:03.730 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:03 smithi103 ceph-mon[31158]: pgmap v525: 97 pgs: 97 active+clean; 4.9 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-19T07:06:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.743+0000 7fa0d275f640 1 -- 172.21.15.103:0/1966124212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0cc1016d0 msgr2=0x7fa0cc101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.743+0000 7fa0d275f640 1 --2- 172.21.15.103:0/1966124212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0cc1016d0 0x7fa0cc101b10 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009900 tx=0x7fa0c002a480 comp rx=0 tx=0).stop 2024-09-19T07:06:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.743+0000 7fa0d275f640 1 -- 172.21.15.103:0/1966124212 shutdown_connections 2024-09-19T07:06:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.743+0000 7fa0d275f640 1 --2- 172.21.15.103:0/1966124212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0cc102050 0x7fa0cc10d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.743+0000 7fa0d275f640 1 --2- 172.21.15.103:0/1966124212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0cc1016d0 0x7fa0cc101b10 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.743+0000 7fa0d275f640 1 --2- 172.21.15.103:0/1966124212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0cc105af0 0x7fa0cc105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.743+0000 7fa0d275f640 1 -- 172.21.15.103:0/1966124212 >> 172.21.15.103:0/1966124212 conn(0x7fa0cc075f20 msgr2=0x7fa0cc076330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.743+0000 7fa0d275f640 1 -- 172.21.15.103:0/1966124212 shutdown_connections 2024-09-19T07:06:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.743+0000 7fa0d275f640 1 -- 172.21.15.103:0/1966124212 wait complete. 2024-09-19T07:06:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.743+0000 7fa0d275f640 1 Processor -- start 2024-09-19T07:06:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.743+0000 7fa0d275f640 1 -- start start 2024-09-19T07:06:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0d275f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0cc1016d0 0x7fa0cc1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0d275f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0cc102050 0x7fa0cc197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0d275f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0cc105af0 0x7fa0cc19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0d275f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc110ed0 con 0x7fa0cc105af0 2024-09-19T07:06:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0d275f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc110d50 con 0x7fa0cc1016d0 2024-09-19T07:06:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0d275f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc111050 con 0x7fa0cc102050 2024-09-19T07:06:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0d0cfc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0cc105af0 0x7fa0cc19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0d0cfc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0cc105af0 0x7fa0cc19bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34448/0 (socket says 172.21.15.103:34448) 2024-09-19T07:06:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0d0cfc640 1 -- 172.21.15.103:0/2898321206 learned_addr learned my addr 172.21.15.103:0/2898321206 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0cb7fe640 1 --2- 172.21.15.103:0/2898321206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0cc102050 0x7fa0cc197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0cbfff640 1 --2- 172.21.15.103:0/2898321206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0cc1016d0 0x7fa0cc1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0d0cfc640 1 -- 172.21.15.103:0/2898321206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0cc102050 msgr2=0x7fa0cc197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0d0cfc640 1 --2- 172.21.15.103:0/2898321206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0cc102050 0x7fa0cc197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0d0cfc640 1 -- 172.21.15.103:0/2898321206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0cc1016d0 msgr2=0x7fa0cc1972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0d0cfc640 1 --2- 172.21.15.103:0/2898321206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0cc1016d0 0x7fa0cc1972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0d0cfc640 1 -- 172.21.15.103:0/2898321206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0cc19c2d0 con 0x7fa0cc105af0 2024-09-19T07:06:03.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0cbfff640 1 --2- 172.21.15.103:0/2898321206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0cc1016d0 0x7fa0cc1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:06:03.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.744+0000 7fa0d0cfc640 1 --2- 172.21.15.103:0/2898321206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0cc105af0 0x7fa0cc19bba0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400b970 tx=0x7fa0c400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:03.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.745+0000 7fa0c97fa640 1 -- 172.21.15.103:0/2898321206 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4019070 con 0x7fa0cc105af0 2024-09-19T07:06:03.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.745+0000 7fa0d275f640 1 -- 172.21.15.103:0/2898321206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0cc19c5c0 con 0x7fa0cc105af0 2024-09-19T07:06:03.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.745+0000 7fa0c97fa640 1 -- 172.21.15.103:0/2898321206 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c400cb40 con 0x7fa0cc105af0 2024-09-19T07:06:03.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.745+0000 7fa0c97fa640 1 -- 172.21.15.103:0/2898321206 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4007650 con 0x7fa0cc105af0 2024-09-19T07:06:03.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.745+0000 7fa0d275f640 1 -- 172.21.15.103:0/2898321206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0cc1a3e50 con 0x7fa0cc105af0 2024-09-19T07:06:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.748+0000 7fa0d55d6740 1 Processor -- start 2024-09-19T07:06:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.748+0000 7fa0c97fa640 1 -- 172.21.15.103:0/2898321206 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fa0c401c070 con 0x7fa0cc105af0 2024-09-19T07:06:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.748+0000 7fa0d55d6740 1 -- start start 2024-09-19T07:06:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.748+0000 7fa0d55d6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee5bc0 0x55cf60ee5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.748+0000 7fa0d55d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ee64e0 0x55cf60ef2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.748+0000 7fa0d55d6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ef32b0 0x55cf60ef56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.748+0000 7fa0d55d6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cf60e482f0 con 0x55cf60ef32b0 2024-09-19T07:06:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.748+0000 7fa0d55d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cf60df0270 con 0x55cf60ee64e0 2024-09-19T07:06:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.748+0000 7fa0d55d6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cf60e442b0 con 0x55cf60ee5bc0 2024-09-19T07:06:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.748+0000 7fa0cb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ef32b0 0x55cf60ef56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.748+0000 7fa0cb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ef32b0 0x55cf60ef56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34460/0 (socket says 172.21.15.103:34460) 2024-09-19T07:06:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.748+0000 7fa0cb7fe640 1 -- 172.21.15.103:0/4284015061 learned_addr learned my addr 172.21.15.103:0/4284015061 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.748+0000 7fa0c97fa640 1 --2- 172.21.15.103:0/2898321206 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa0b40a3460 0x7fa0b40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.748+0000 7fa0cbfff640 1 --2- 172.21.15.103:0/4284015061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ee64e0 0x55cf60ef2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.748+0000 7fa0cb7fe640 1 --2- 172.21.15.103:0/4284015061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee5bc0 0x55cf60ee5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.749+0000 7fa0d0cfc640 1 --2- 172.21.15.103:0/2898321206 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa0b40a3460 0x7fa0b40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.749+0000 7fa0cb7fe640 1 -- 172.21.15.103:0/4284015061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee5bc0 msgr2=0x55cf60ee5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.749+0000 7fa0cb7fe640 1 --2- 172.21.15.103:0/4284015061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee5bc0 0x55cf60ee5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.749+0000 7fa0cb7fe640 1 -- 172.21.15.103:0/4284015061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ee64e0 msgr2=0x55cf60ef2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.749+0000 7fa0cb7fe640 1 --2- 172.21.15.103:0/4284015061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ee64e0 0x55cf60ef2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.749+0000 7fa0cb7fe640 1 -- 172.21.15.103:0/4284015061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf60def340 con 0x55cf60ef32b0 2024-09-19T07:06:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.749+0000 7fa0c97fa640 1 -- 172.21.15.103:0/2898321206 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fa0c40cf2b0 con 0x7fa0cc105af0 2024-09-19T07:06:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.749+0000 7fa0cb7fe640 1 --2- 172.21.15.103:0/4284015061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ef32b0 0x55cf60ef56a0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009900 tx=0x7fa0c0031460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7c2565e109d9af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.749+0000 7fa0d0cfc640 1 --2- 172.21.15.103:0/2898321206 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa0b40a3460 0x7fa0b40a5920 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40ccb50 tx=0x7fa0c406d400 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.749+0000 7fa0b27fc640 1 -- 172.21.15.103:0/4284015061 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0035040 con 0x55cf60ef32b0 2024-09-19T07:06:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.749+0000 7fa0b27fc640 1 -- 172.21.15.103:0/4284015061 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c0005500 con 0x55cf60ef32b0 2024-09-19T07:06:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.749+0000 7fa0b27fc640 1 -- 172.21.15.103:0/4284015061 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0003850 con 0x55cf60ef32b0 2024-09-19T07:06:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.750+0000 7fa0d55d6740 1 -- 172.21.15.103:0/4284015061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ef32b0 msgr2=0x55cf60ef56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.750+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/4284015061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ef32b0 0x55cf60ef56a0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009900 tx=0x7fa0c0031460 comp rx=0 tx=0).stop 2024-09-19T07:06:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.750+0000 7fa0d55d6740 1 -- 172.21.15.103:0/4284015061 shutdown_connections 2024-09-19T07:06:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.750+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/4284015061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ef32b0 0x55cf60ef56a0 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.750+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/4284015061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ee64e0 0x55cf60ef2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.750+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/4284015061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee5bc0 0x55cf60ee5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.750+0000 7fa0d55d6740 1 -- 172.21.15.103:0/4284015061 >> 172.21.15.103:0/4284015061 conn(0x55cf60ee2270 msgr2=0x55cf60ee2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.750+0000 7fa0d55d6740 1 -- 172.21.15.103:0/4284015061 shutdown_connections 2024-09-19T07:06:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.750+0000 7fa0d55d6740 1 -- 172.21.15.103:0/4284015061 wait complete. 2024-09-19T07:06:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.750+0000 7fa0d55d6740 1 -- start start 2024-09-19T07:06:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.750+0000 7fa0d55d6740 1 Processor -- start 2024-09-19T07:06:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0d55d6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ee5bc0 0x55cf60ef0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0d55d6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee64e0 0x55cf60ef0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0d55d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ef32b0 0x55cf60ef0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0d55d6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cf60ef5ca0 con 0x55cf60ee5bc0 2024-09-19T07:06:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0d55d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cf60dd1290 con 0x55cf60ef32b0 2024-09-19T07:06:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0d55d6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cf60ef5e20 con 0x55cf60ee64e0 2024-09-19T07:06:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0d55d6740 10 client.0 _ll_register_callbacks cb 0x55cf60e02140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0cb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ee5bc0 0x55cf60ef0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0cb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ee5bc0 0x55cf60ef0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34472/0 (socket says 172.21.15.103:34472) 2024-09-19T07:06:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0cb7fe640 1 -- 172.21.15.103:0/1899089789 learned_addr learned my addr 172.21.15.103:0/1899089789 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0cb7fe640 1 --2- 172.21.15.103:0/1899089789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ef32b0 0x55cf60ef0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0cbfff640 1 --2- 172.21.15.103:0/1899089789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee64e0 0x55cf60ef0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0cb7fe640 1 -- 172.21.15.103:0/1899089789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee64e0 msgr2=0x55cf60ef0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0cb7fe640 1 --2- 172.21.15.103:0/1899089789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee64e0 0x55cf60ef0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0cb7fe640 1 -- 172.21.15.103:0/1899089789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ef32b0 msgr2=0x55cf60ef0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0cb7fe640 1 --2- 172.21.15.103:0/1899089789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ef32b0 0x55cf60ef0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0cb7fe640 1 -- 172.21.15.103:0/1899089789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf60b5b210 con 0x55cf60ee5bc0 2024-09-19T07:06:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0cb7fe640 1 --2- 172.21.15.103:0/1899089789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ee5bc0 0x55cf60ef0240 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0004650 tx=0x7fa0c0004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c39aaba0b1c94f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0ab7fe640 1 -- 172.21.15.103:0/1899089789 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0051020 con 0x55cf60ee5bc0 2024-09-19T07:06:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0ab7fe640 1 -- 172.21.15.103:0/1899089789 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fa0c00317b0 con 0x55cf60ee5bc0 2024-09-19T07:06:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.751+0000 7fa0d55d6740 1 -- 172.21.15.103:0/1899089789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c0005500 con 0x55cf60ee5bc0 2024-09-19T07:06:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.752+0000 7fa0ab7fe640 1 -- 172.21.15.103:0/1899089789 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa0c0035070 con 0x55cf60ee5bc0 2024-09-19T07:06:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.752+0000 7fa0d55d6740 10 client.21429 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.752+0000 7fa0d55d6740 1 -- 172.21.15.103:0/1899089789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf60c27620 con 0x55cf60ee5bc0 2024-09-19T07:06:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.752+0000 7fa0ab7fe640 1 -- 172.21.15.103:0/1899089789 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c0005500 con 0x55cf60ee5bc0 2024-09-19T07:06:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.752+0000 7fa0ab7fe640 1 -- 172.21.15.103:0/1899089789 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa0c003a040 con 0x55cf60ee5bc0 2024-09-19T07:06:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.752+0000 7fa0d55d6740 10 client.21429 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.752+0000 7fa0d55d6740 10 client.21429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:06:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.752+0000 7fa0d55d6740 20 client.21429 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.752+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/1899089789 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cf60f02f40 0x55cf60f05330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.752+0000 7fa0d55d6740 4 client.21429 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:06:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.752+0000 7fa0d55d6740 1 -- 172.21.15.103:0/1899089789 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf60e55a20 con 0x55cf60f02f40 2024-09-19T07:06:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.752+0000 7fa0cbfff640 1 --2- 172.21.15.103:0/1899089789 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cf60f02f40 0x55cf60f05330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.753+0000 7fa0cbfff640 1 --2- 172.21.15.103:0/1899089789 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cf60f02f40 0x55cf60f05330 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fd99d1cd04e062d server_cookie=6e5f5d8cb471a992 in_seq=0 out_seq=0 2024-09-19T07:06:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.753+0000 7fa0ab7fe640 10 client.21429 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.753+0000 7fa0ab7fe640 1 -- 172.21.15.103:0/1899089789 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf60e55a20 con 0x55cf60f02f40 2024-09-19T07:06:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.753+0000 7fa0ab7fe640 10 client.21429 handle_command_reply: tid=0 2024-09-19T07:06:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.753+0000 7fa0d55d6740 1 client.21429 shutdown 2024-09-19T07:06:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.754+0000 7fa0d55d6740 1 -- 172.21.15.103:0/1899089789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ee5bc0 msgr2=0x55cf60ef0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.754+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/1899089789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ee5bc0 0x55cf60ef0240 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0004650 tx=0x7fa0c0004ec0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.754+0000 7fa0d55d6740 1 -- 172.21.15.103:0/1899089789 shutdown_connections 2024-09-19T07:06:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.754+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/1899089789 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cf60f02f40 0x55cf60f05330 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.754+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/1899089789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ef32b0 0x55cf60ef0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.754+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/1899089789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee64e0 0x55cf60ef0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.754+0000 7fa0ab7fe640 0 client.21429 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.754+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/1899089789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ee5bc0 0x55cf60ef0240 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.754+0000 7fa0d55d6740 1 -- 172.21.15.103:0/1899089789 >> 172.21.15.103:0/1899089789 conn(0x55cf60ee2270 msgr2=0x55cf60ee7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.754+0000 7fa0d55d6740 1 -- 172.21.15.103:0/1899089789 shutdown_connections 2024-09-19T07:06:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.754+0000 7fa0d55d6740 1 -- 172.21.15.103:0/1899089789 wait complete. 2024-09-19T07:06:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.754+0000 7fa0d55d6740 20 client.21429 trim_cache size 0 max 16384 2024-09-19T07:06:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0d55d6740 1 Processor -- start 2024-09-19T07:06:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0d55d6740 1 -- start start 2024-09-19T07:06:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0d55d6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee5bc0 0x55cf60edf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0d55d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ee64e0 0x55cf60ee15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0d55d6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ef32b0 0x55cf60eedd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0d55d6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc1ab5e0 con 0x55cf60ef32b0 2024-09-19T07:06:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0d55d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc1ad120 con 0x55cf60ee64e0 2024-09-19T07:06:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0d55d6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc1abcb0 con 0x55cf60ee5bc0 2024-09-19T07:06:03.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0cb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ef32b0 0x55cf60eedd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0cb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ef32b0 0x55cf60eedd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34476/0 (socket says 172.21.15.103:34476) 2024-09-19T07:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0cb7fe640 1 -- 172.21.15.103:0/333970178 learned_addr learned my addr 172.21.15.103:0/333970178 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0cbfff640 1 --2- 172.21.15.103:0/333970178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ee64e0 0x55cf60ee15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0cb7fe640 1 -- 172.21.15.103:0/333970178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee5bc0 msgr2=0x55cf60edf0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0cb7fe640 1 --2- 172.21.15.103:0/333970178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee5bc0 0x55cf60edf0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0cb7fe640 1 -- 172.21.15.103:0/333970178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ee64e0 msgr2=0x55cf60ee15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0cb7fe640 1 --2- 172.21.15.103:0/333970178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ee64e0 0x55cf60ee15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.768+0000 7fa0cb7fe640 1 -- 172.21.15.103:0/333970178 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf60ee3340 con 0x55cf60ef32b0 2024-09-19T07:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.769+0000 7fa0cb7fe640 1 --2- 172.21.15.103:0/333970178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ef32b0 0x55cf60eedd40 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0004df0 tx=0x7fa0c0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e9530c1cf1dcfd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.769+0000 7fa0b1ffb640 1 -- 172.21.15.103:0/333970178 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0051020 con 0x55cf60ef32b0 2024-09-19T07:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.769+0000 7fa0b1ffb640 1 -- 172.21.15.103:0/333970178 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c0003ad0 con 0x55cf60ef32b0 2024-09-19T07:06:03.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.769+0000 7fa0b1ffb640 1 -- 172.21.15.103:0/333970178 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c00317d0 con 0x55cf60ef32b0 2024-09-19T07:06:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.769+0000 7fa0d55d6740 1 -- 172.21.15.103:0/333970178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ef32b0 msgr2=0x55cf60eedd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.769+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/333970178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ef32b0 0x55cf60eedd40 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0004df0 tx=0x7fa0c0005fd0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.769+0000 7fa0d55d6740 1 -- 172.21.15.103:0/333970178 shutdown_connections 2024-09-19T07:06:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.769+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/333970178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ef32b0 0x55cf60eedd40 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.769+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/333970178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ee64e0 0x55cf60ee15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.769+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/333970178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee5bc0 0x55cf60edf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.769+0000 7fa0d55d6740 1 -- 172.21.15.103:0/333970178 >> 172.21.15.103:0/333970178 conn(0x55cf60ee2270 msgr2=0x55cf60ede9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.770+0000 7fa0d55d6740 1 -- 172.21.15.103:0/333970178 shutdown_connections 2024-09-19T07:06:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.770+0000 7fa0d55d6740 1 -- 172.21.15.103:0/333970178 wait complete. 2024-09-19T07:06:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.770+0000 7fa0d55d6740 1 -- start start 2024-09-19T07:06:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.770+0000 7fa0d55d6740 1 Processor -- start 2024-09-19T07:06:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.770+0000 7fa0d55d6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ee5bc0 0x55cf60f061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.770+0000 7fa0d55d6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee64e0 0x55cf60f06700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.770+0000 7fa0d55d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ef32b0 0x55cf60f06c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.770+0000 7fa0d55d6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc1aca50 con 0x55cf60ee5bc0 2024-09-19T07:06:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.770+0000 7fa0d55d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc1ac380 con 0x55cf60ef32b0 2024-09-19T07:06:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.770+0000 7fa0d55d6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc1a95e0 con 0x55cf60ee64e0 2024-09-19T07:06:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.770+0000 7fa0d55d6740 10 client.0 _ll_register_callbacks cb 0x7fa0c00043a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.770+0000 7fa0cb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ee5bc0 0x55cf60f061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.770+0000 7fa0cb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ee5bc0 0x55cf60f061c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34490/0 (socket says 172.21.15.103:34490) 2024-09-19T07:06:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.770+0000 7fa0cb7fe640 1 -- 172.21.15.103:0/2423120497 learned_addr learned my addr 172.21.15.103:0/2423120497 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.771+0000 7fa0cb7fe640 1 -- 172.21.15.103:0/2423120497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee64e0 msgr2=0x55cf60f06700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.771+0000 7fa0cb7fe640 1 --2- 172.21.15.103:0/2423120497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee64e0 0x55cf60f06700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.771+0000 7fa0cb7fe640 1 -- 172.21.15.103:0/2423120497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ef32b0 msgr2=0x55cf60f06c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.771+0000 7fa0cb7fe640 1 --2- 172.21.15.103:0/2423120497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ef32b0 0x55cf60f06c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.771+0000 7fa0cb7fe640 1 -- 172.21.15.103:0/2423120497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf60f6c980 con 0x55cf60ee5bc0 2024-09-19T07:06:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.771+0000 7fa0cb7fe640 1 --2- 172.21.15.103:0/2423120497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ee5bc0 0x55cf60f061c0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00095c0 tx=0x7fa0c002ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6075b8449f53fc7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.771+0000 7fa0ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.771+0000 7fa0ab7fe640 1 -- 172.21.15.103:0/2423120497 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0051040 con 0x55cf60ee5bc0 2024-09-19T07:06:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.771+0000 7fa0ab7fe640 1 -- 172.21.15.103:0/2423120497 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7fa0c0004e20 con 0x55cf60ee5bc0 2024-09-19T07:06:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.771+0000 7fa0d55d6740 1 -- 172.21.15.103:0/2423120497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c0003ad0 con 0x55cf60ee5bc0 2024-09-19T07:06:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.771+0000 7fa0ab7fe640 1 -- 172.21.15.103:0/2423120497 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa0c00317c0 con 0x55cf60ee5bc0 2024-09-19T07:06:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.771+0000 7fa0d55d6740 10 client.21435 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.771+0000 7fa0d55d6740 1 -- 172.21.15.103:0/2423120497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf60f6dc70 con 0x55cf60ee5bc0 2024-09-19T07:06:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.771+0000 7fa0ab7fe640 1 -- 172.21.15.103:0/2423120497 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c0003ad0 con 0x55cf60ee5bc0 2024-09-19T07:06:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.771+0000 7fa0ab7fe640 1 -- 172.21.15.103:0/2423120497 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa0c005c070 con 0x55cf60ee5bc0 2024-09-19T07:06:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.772+0000 7fa0d55d6740 10 client.21435 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.772+0000 7fa0d55d6740 10 client.21435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:06:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.772+0000 7fa0d55d6740 20 client.21435 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.772+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/2423120497 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cf60f6e1a0 0x55cf60f70590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.772+0000 7fa0d55d6740 4 client.21435 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:06:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.772+0000 7fa0d55d6740 1 -- 172.21.15.103:0/2423120497 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf60e4fd70 con 0x55cf60f6e1a0 2024-09-19T07:06:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.772+0000 7fa0cbfff640 1 --2- 172.21.15.103:0/2423120497 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cf60f6e1a0 0x55cf60f70590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.772+0000 7fa0ab7fe640 10 client.21435 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.772+0000 7fa0cbfff640 1 --2- 172.21.15.103:0/2423120497 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cf60f6e1a0 0x55cf60f70590 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b72b958243856c server_cookie=577b440ddd763661 in_seq=0 out_seq=0 2024-09-19T07:06:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.785+0000 7fa0ab7fe640 1 -- 172.21.15.103:0/2423120497 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf60e4fd70 con 0x55cf60f6e1a0 2024-09-19T07:06:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.785+0000 7fa0ab7fe640 10 client.21435 handle_command_reply: tid=0 2024-09-19T07:06:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.785+0000 7fa0d55d6740 1 client.21435 shutdown 2024-09-19T07:06:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.785+0000 7fa0d55d6740 1 -- 172.21.15.103:0/2423120497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ee5bc0 msgr2=0x55cf60f061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.785+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/2423120497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ee5bc0 0x55cf60f061c0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00095c0 tx=0x7fa0c002ab70 comp rx=0 tx=0).stop 2024-09-19T07:06:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.785+0000 7fa0d55d6740 1 -- 172.21.15.103:0/2423120497 shutdown_connections 2024-09-19T07:06:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.785+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/2423120497 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55cf60f6e1a0 0x55cf60f70590 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.785+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/2423120497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf60ef32b0 0x55cf60f06c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.785+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/2423120497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf60ee64e0 0x55cf60f06700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.785+0000 7fa0d55d6740 1 --2- 172.21.15.103:0/2423120497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60ee5bc0 0x55cf60f061c0 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.785+0000 7fa0ab7fe640 0 client.21435 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.785+0000 7fa0d55d6740 1 -- 172.21.15.103:0/2423120497 >> 172.21.15.103:0/2423120497 conn(0x55cf60ee2270 msgr2=0x55cf60efe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.785+0000 7fa0d55d6740 1 -- 172.21.15.103:0/2423120497 shutdown_connections 2024-09-19T07:06:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.785+0000 7fa0d55d6740 1 -- 172.21.15.103:0/2423120497 wait complete. 2024-09-19T07:06:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.785+0000 7fa0d55d6740 20 client.21435 trim_cache size 0 max 16384 2024-09-19T07:06:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:06:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:06:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:06:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:06:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.786+0000 7fa0d275f640 1 -- 172.21.15.103:0/2898321206 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa0b40a3460 msgr2=0x7fa0b40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.786+0000 7fa0d275f640 1 --2- 172.21.15.103:0/2898321206 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa0b40a3460 0x7fa0b40a5920 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40ccb50 tx=0x7fa0c406d400 comp rx=0 tx=0).stop 2024-09-19T07:06:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.786+0000 7fa0d275f640 1 -- 172.21.15.103:0/2898321206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0cc105af0 msgr2=0x7fa0cc19bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.786+0000 7fa0d275f640 1 --2- 172.21.15.103:0/2898321206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0cc105af0 0x7fa0cc19bba0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400b970 tx=0x7fa0c400be30 comp rx=0 tx=0).stop 2024-09-19T07:06:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.786+0000 7fa0d275f640 1 -- 172.21.15.103:0/2898321206 shutdown_connections 2024-09-19T07:06:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.786+0000 7fa0d275f640 1 --2- 172.21.15.103:0/2898321206 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fa0b40a3460 0x7fa0b40a5920 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.786+0000 7fa0d275f640 1 --2- 172.21.15.103:0/2898321206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0cc105af0 0x7fa0cc19bba0 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.786+0000 7fa0d275f640 1 --2- 172.21.15.103:0/2898321206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0cc102050 0x7fa0cc197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.786+0000 7fa0d275f640 1 --2- 172.21.15.103:0/2898321206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0cc1016d0 0x7fa0cc1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.786+0000 7fa0d275f640 1 -- 172.21.15.103:0/2898321206 >> 172.21.15.103:0/2898321206 conn(0x7fa0cc075f20 msgr2=0x7fa0cc0ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.786+0000 7fa0d275f640 1 -- 172.21.15.103:0/2898321206 shutdown_connections 2024-09-19T07:06:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:03.786+0000 7fa0d275f640 1 -- 172.21.15.103:0/2898321206 wait complete. 2024-09-19T07:06:03.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:03 smithi174 ceph-mon[35637]: pgmap v525: 97 pgs: 97 active+clean; 4.9 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-19T07:06:03.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.816+0000 7f34e9006640 1 -- 172.21.15.103:0/4076641196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e4102790 msgr2=0x7f34e41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.816+0000 7f34e9006640 1 --2- 172.21.15.103:0/4076641196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e4102790 0x7f34e41067e0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009900 tx=0x7f34cc02a410 comp rx=0 tx=0).stop 2024-09-19T07:06:03.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.816+0000 7f34e9006640 1 -- 172.21.15.103:0/4076641196 shutdown_connections 2024-09-19T07:06:03.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.816+0000 7f34e9006640 1 --2- 172.21.15.103:0/4076641196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34e4106f10 0x7f34e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.816+0000 7f34e9006640 1 --2- 172.21.15.103:0/4076641196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e4102790 0x7f34e41067e0 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.816+0000 7f34e9006640 1 --2- 172.21.15.103:0/4076641196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e4101de0 0x7f34e41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.816+0000 7f34e9006640 1 -- 172.21.15.103:0/4076641196 >> 172.21.15.103:0/4076641196 conn(0x7f34e40fd8c0 msgr2=0x7f34e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:03.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.816+0000 7f34e9006640 1 -- 172.21.15.103:0/4076641196 shutdown_connections 2024-09-19T07:06:03.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.816+0000 7f34e9006640 1 -- 172.21.15.103:0/4076641196 wait complete. 2024-09-19T07:06:03.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.817+0000 7f34e9006640 1 Processor -- start 2024-09-19T07:06:03.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.817+0000 7f34e9006640 1 -- start start 2024-09-19T07:06:03.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e9006640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e4101de0 0x7f34e4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e9006640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34e4102790 0x7f34e41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e9006640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e4106f10 0x7f34e419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e9006640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4111f90 con 0x7f34e4101de0 2024-09-19T07:06:03.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e9006640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4111e10 con 0x7f34e4102790 2024-09-19T07:06:03.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e9006640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4112110 con 0x7f34e4106f10 2024-09-19T07:06:03.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e4101de0 0x7f34e4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e4101de0 0x7f34e4198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34514/0 (socket says 172.21.15.103:34514) 2024-09-19T07:06:03.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e2d9d640 1 -- 172.21.15.103:0/761721716 learned_addr learned my addr 172.21.15.103:0/761721716 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:03.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e359e640 1 --2- 172.21.15.103:0/761721716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e4106f10 0x7f34e419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34da59c640 1 --2- 172.21.15.103:0/761721716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34e4102790 0x7f34e41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e2d9d640 1 -- 172.21.15.103:0/761721716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e4106f10 msgr2=0x7f34e419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e2d9d640 1 --2- 172.21.15.103:0/761721716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e4106f10 0x7f34e419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e2d9d640 1 -- 172.21.15.103:0/761721716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34e4102790 msgr2=0x7f34e41989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e2d9d640 1 --2- 172.21.15.103:0/761721716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34e4102790 0x7f34e41989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e2d9d640 1 -- 172.21.15.103:0/761721716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e419d440 con 0x7f34e4101de0 2024-09-19T07:06:03.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e2d9d640 1 --2- 172.21.15.103:0/761721716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e4101de0 0x7f34e4198490 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f34d400b4a0 tx=0x7f34d400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:03.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.818+0000 7f34e0d99640 1 -- 172.21.15.103:0/761721716 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4014070 con 0x7f34e4101de0 2024-09-19T07:06:03.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.819+0000 7f34e9006640 1 -- 172.21.15.103:0/761721716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e419d730 con 0x7f34e4101de0 2024-09-19T07:06:03.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.819+0000 7f34e0d99640 1 -- 172.21.15.103:0/761721716 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d4010af0 con 0x7f34e4101de0 2024-09-19T07:06:03.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.819+0000 7f34e0d99640 1 -- 172.21.15.103:0/761721716 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4017d10 con 0x7f34e4101de0 2024-09-19T07:06:03.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.819+0000 7f34e9006640 1 -- 172.21.15.103:0/761721716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e41a5010 con 0x7f34e4101de0 2024-09-19T07:06:03.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.821+0000 7f34e0d99640 1 -- 172.21.15.103:0/761721716 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f34d4017070 con 0x7f34e4101de0 2024-09-19T07:06:03.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.821+0000 7f34eb3df740 1 Processor -- start 2024-09-19T07:06:03.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.821+0000 7f34eb3df740 1 -- start start 2024-09-19T07:06:03.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.821+0000 7f34eb3df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a2891b40 0x55e4a2890090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.821+0000 7f34eb3df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a28905d0 0x55e4a2890a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.821+0000 7f34eb3df740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a289b370 0x55e4a289d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.821+0000 7f34eb3df740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e4a27f62f0 con 0x55e4a289b370 2024-09-19T07:06:03.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.821+0000 7f34eb3df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e4a279dcf0 con 0x55e4a28905d0 2024-09-19T07:06:03.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.821+0000 7f34eb3df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e4a27f22b0 con 0x55e4a2891b40 2024-09-19T07:06:03.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34da59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a289b370 0x55e4a289d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34da59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a289b370 0x55e4a289d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34516/0 (socket says 172.21.15.103:34516) 2024-09-19T07:06:03.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34da59c640 1 -- 172.21.15.103:0/1658699260 learned_addr learned my addr 172.21.15.103:0/1658699260 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:03.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34e2d9d640 1 --2- 172.21.15.103:0/1658699260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a28905d0 0x55e4a2890a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34e359e640 1 --2- 172.21.15.103:0/1658699260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a2891b40 0x55e4a2890090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34da59c640 1 -- 172.21.15.103:0/1658699260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a2891b40 msgr2=0x55e4a2890090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:03.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34da59c640 1 --2- 172.21.15.103:0/1658699260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a2891b40 0x55e4a2890090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34da59c640 1 -- 172.21.15.103:0/1658699260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a28905d0 msgr2=0x55e4a2890a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:03.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34da59c640 1 --2- 172.21.15.103:0/1658699260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a28905d0 0x55e4a2890a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34da59c640 1 -- 172.21.15.103:0/1658699260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a27823a0 con 0x55e4a289b370 2024-09-19T07:06:03.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34da59c640 1 --2- 172.21.15.103:0/1658699260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a289b370 0x55e4a289d760 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009900 tx=0x7f34cc031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bbaa2f1e55f3df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:03.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34e0d99640 1 --2- 172.21.15.103:0/761721716 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34d00a33b0 0x7f34d00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34d959a640 1 -- 172.21.15.103:0/1658699260 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc040070 con 0x55e4a289b370 2024-09-19T07:06:03.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34e359e640 1 --2- 172.21.15.103:0/761721716 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34d00a33b0 0x7f34d00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34d959a640 1 -- 172.21.15.103:0/1658699260 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34cc005020 con 0x55e4a289b370 2024-09-19T07:06:03.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34d959a640 1 -- 172.21.15.103:0/1658699260 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc035040 con 0x55e4a289b370 2024-09-19T07:06:03.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.822+0000 7f34e0d99640 1 -- 172.21.15.103:0/761721716 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f34d40cefc0 con 0x7f34e4101de0 2024-09-19T07:06:03.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.823+0000 7f34eb3df740 1 -- 172.21.15.103:0/1658699260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a289b370 msgr2=0x55e4a289d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.823+0000 7f34e359e640 1 --2- 172.21.15.103:0/761721716 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34d00a33b0 0x7f34d00a5870 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f34dc005fd0 tx=0x7f34dc009600 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:03.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.823+0000 7f34eb3df740 1 --2- 172.21.15.103:0/1658699260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a289b370 0x55e4a289d760 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009900 tx=0x7f34cc031630 comp rx=0 tx=0).stop 2024-09-19T07:06:03.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.823+0000 7f34eb3df740 1 -- 172.21.15.103:0/1658699260 shutdown_connections 2024-09-19T07:06:03.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.823+0000 7f34eb3df740 1 --2- 172.21.15.103:0/1658699260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a289b370 0x55e4a289d760 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.823+0000 7f34eb3df740 1 --2- 172.21.15.103:0/1658699260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a28905d0 0x55e4a2890a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.823+0000 7f34eb3df740 1 --2- 172.21.15.103:0/1658699260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a2891b40 0x55e4a2890090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.823+0000 7f34eb3df740 1 -- 172.21.15.103:0/1658699260 >> 172.21.15.103:0/1658699260 conn(0x55e4a2859310 msgr2=0x55e4a288cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:03.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.823+0000 7f34eb3df740 1 -- 172.21.15.103:0/1658699260 shutdown_connections 2024-09-19T07:06:03.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.823+0000 7f34eb3df740 1 -- 172.21.15.103:0/1658699260 wait complete. 2024-09-19T07:06:03.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34eb3df740 1 -- start start 2024-09-19T07:06:03.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34eb3df740 1 Processor -- start 2024-09-19T07:06:03.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34eb3df740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a28905d0 0x55e4a289ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34eb3df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a2891b40 0x55e4a2897fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34eb3df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a289b370 0x55e4a2898570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34eb3df740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e4a289e010 con 0x55e4a28905d0 2024-09-19T07:06:03.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34eb3df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e4a27b1060 con 0x55e4a289b370 2024-09-19T07:06:03.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34eb3df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e4a289e190 con 0x55e4a2891b40 2024-09-19T07:06:03.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34eb3df740 10 client.0 _ll_register_callbacks cb 0x55e4a25048c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:03.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34e2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a28905d0 0x55e4a289ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34e2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a28905d0 0x55e4a289ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34522/0 (socket says 172.21.15.103:34522) 2024-09-19T07:06:03.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34e2d9d640 1 -- 172.21.15.103:0/2670033551 learned_addr learned my addr 172.21.15.103:0/2670033551 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:03.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34da59c640 1 --2- 172.21.15.103:0/2670033551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a2891b40 0x55e4a2897fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34e359e640 1 --2- 172.21.15.103:0/2670033551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a289b370 0x55e4a2898570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34e2d9d640 1 -- 172.21.15.103:0/2670033551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a2891b40 msgr2=0x55e4a2897fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:03.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34e2d9d640 1 --2- 172.21.15.103:0/2670033551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a2891b40 0x55e4a2897fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34e2d9d640 1 -- 172.21.15.103:0/2670033551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a289b370 msgr2=0x55e4a2898570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:03.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34e2d9d640 1 --2- 172.21.15.103:0/2670033551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a289b370 0x55e4a2898570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.824+0000 7f34e2d9d640 1 -- 172.21.15.103:0/2670033551 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a25673c0 con 0x55e4a28905d0 2024-09-19T07:06:03.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34e359e640 1 --2- 172.21.15.103:0/2670033551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a289b370 0x55e4a2898570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:03.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34e2d9d640 1 --2- 172.21.15.103:0/2670033551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a28905d0 0x55e4a289ad20 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f34d400be60 tx=0x7f34d409b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e99a31176dd85c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:03.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:03.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34ca7fc640 1 -- 172.21.15.103:0/2670033551 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d401e030 con 0x55e4a28905d0 2024-09-19T07:06:03.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34ca7fc640 1 -- 172.21.15.103:0/2670033551 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f34d4097ab0 con 0x55e4a28905d0 2024-09-19T07:06:03.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34eb3df740 1 -- 172.21.15.103:0/2670033551 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34cc005020 con 0x55e4a28905d0 2024-09-19T07:06:03.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34ca7fc640 1 -- 172.21.15.103:0/2670033551 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f34d40d2620 con 0x55e4a28905d0 2024-09-19T07:06:03.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34eb3df740 10 client.21447 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:03.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34eb3df740 1 -- 172.21.15.103:0/2670033551 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e4a28a9a60 con 0x55e4a28905d0 2024-09-19T07:06:03.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34ca7fc640 1 -- 172.21.15.103:0/2670033551 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34cc005020 con 0x55e4a28905d0 2024-09-19T07:06:03.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34ca7fc640 1 -- 172.21.15.103:0/2670033551 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f34d40d1110 con 0x55e4a28905d0 2024-09-19T07:06:03.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34eb3df740 10 client.21447 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:03.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34eb3df740 10 client.21447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:06:03.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34eb3df740 20 client.21447 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:03.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34eb3df740 1 --2- 172.21.15.103:0/2670033551 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e4a28aaf20 0x55e4a28ad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34eb3df740 4 client.21447 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:06:03.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.825+0000 7f34eb3df740 1 -- 172.21.15.103:0/2670033551 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4a2803860 con 0x55e4a28aaf20 2024-09-19T07:06:03.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.827+0000 7f34da59c640 1 --2- 172.21.15.103:0/2670033551 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e4a28aaf20 0x55e4a28ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.828+0000 7f34ca7fc640 10 client.21447 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:03.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.828+0000 7f34da59c640 1 --2- 172.21.15.103:0/2670033551 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e4a28aaf20 0x55e4a28ad330 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69557ecf329c4920 server_cookie=3feb76e277e709d6 in_seq=0 out_seq=0 2024-09-19T07:06:03.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.829+0000 7f34ca7fc640 1 -- 172.21.15.103:0/2670033551 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4a2803860 con 0x55e4a28aaf20 2024-09-19T07:06:03.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.829+0000 7f34ca7fc640 10 client.21447 handle_command_reply: tid=0 2024-09-19T07:06:03.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.829+0000 7f34eb3df740 1 client.21447 shutdown 2024-09-19T07:06:03.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.829+0000 7f34eb3df740 1 -- 172.21.15.103:0/2670033551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a28905d0 msgr2=0x55e4a289ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.829+0000 7f34eb3df740 1 --2- 172.21.15.103:0/2670033551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a28905d0 0x55e4a289ad20 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f34d400be60 tx=0x7f34d409b2d0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.829+0000 7f34eb3df740 1 -- 172.21.15.103:0/2670033551 shutdown_connections 2024-09-19T07:06:03.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.829+0000 7f34eb3df740 1 --2- 172.21.15.103:0/2670033551 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e4a28aaf20 0x55e4a28ad330 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.829+0000 7f34eb3df740 1 --2- 172.21.15.103:0/2670033551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a289b370 0x55e4a2898570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.829+0000 7f34eb3df740 1 --2- 172.21.15.103:0/2670033551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a2891b40 0x55e4a2897fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.829+0000 7f34ca7fc640 0 client.21447 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:03.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.829+0000 7f34eb3df740 1 --2- 172.21.15.103:0/2670033551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a28905d0 0x55e4a289ad20 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.829+0000 7f34eb3df740 1 -- 172.21.15.103:0/2670033551 >> 172.21.15.103:0/2670033551 conn(0x55e4a2859310 msgr2=0x55e4a2892960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:03.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.829+0000 7f34eb3df740 1 -- 172.21.15.103:0/2670033551 shutdown_connections 2024-09-19T07:06:03.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.829+0000 7f34eb3df740 1 -- 172.21.15.103:0/2670033551 wait complete. 2024-09-19T07:06:03.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.829+0000 7f34eb3df740 20 client.21447 trim_cache size 0 max 16384 2024-09-19T07:06:03.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.842+0000 7f34eb3df740 1 Processor -- start 2024-09-19T07:06:03.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.842+0000 7f34eb3df740 1 -- start start 2024-09-19T07:06:03.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.842+0000 7f34eb3df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a28905d0 0x55e4a289a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.842+0000 7f34eb3df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a2891b40 0x55e4a28a72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.843+0000 7f34eb3df740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a289b370 0x55e4a28b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.843+0000 7f34eb3df740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f34e41ad940 con 0x55e4a289b370 2024-09-19T07:06:03.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.843+0000 7f34eb3df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34e41af480 con 0x55e4a2891b40 2024-09-19T07:06:03.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.843+0000 7f34eb3df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34e41ae010 con 0x55e4a28905d0 2024-09-19T07:06:03.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.843+0000 7f34e359e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a289b370 0x55e4a28b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.843+0000 7f34e359e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a289b370 0x55e4a28b2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34526/0 (socket says 172.21.15.103:34526) 2024-09-19T07:06:03.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.843+0000 7f34e359e640 1 -- 172.21.15.103:0/2187964716 learned_addr learned my addr 172.21.15.103:0/2187964716 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:03.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.843+0000 7f34e359e640 1 -- 172.21.15.103:0/2187964716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a28905d0 msgr2=0x55e4a289a300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:06:03.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.843+0000 7f34e359e640 1 --2- 172.21.15.103:0/2187964716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a28905d0 0x55e4a289a300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.843+0000 7f34e359e640 1 -- 172.21.15.103:0/2187964716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a2891b40 msgr2=0x55e4a28a72b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:06:03.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.843+0000 7f34e359e640 1 --2- 172.21.15.103:0/2187964716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a2891b40 0x55e4a28a72b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.843+0000 7f34e359e640 1 -- 172.21.15.103:0/2187964716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a2896520 con 0x55e4a289b370 2024-09-19T07:06:03.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.843+0000 7f34e359e640 1 --2- 172.21.15.103:0/2187964716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a289b370 0x55e4a28b2ba0 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f34dc01c5f0 tx=0x7f34dc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7576be2637a26b5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:03.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.843+0000 7f34d8d99640 1 -- 172.21.15.103:0/2187964716 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc0104f0 con 0x55e4a289b370 2024-09-19T07:06:03.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.843+0000 7f34d8d99640 1 -- 172.21.15.103:0/2187964716 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34dc010b70 con 0x55e4a289b370 2024-09-19T07:06:03.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.843+0000 7f34d8d99640 1 -- 172.21.15.103:0/2187964716 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc027e00 con 0x55e4a289b370 2024-09-19T07:06:03.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.844+0000 7f34eb3df740 1 -- 172.21.15.103:0/2187964716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a289b370 msgr2=0x55e4a28b2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.844+0000 7f34eb3df740 1 --2- 172.21.15.103:0/2187964716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a289b370 0x55e4a28b2ba0 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f34dc01c5f0 tx=0x7f34dc015000 comp rx=0 tx=0).stop 2024-09-19T07:06:03.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.844+0000 7f34eb3df740 1 -- 172.21.15.103:0/2187964716 shutdown_connections 2024-09-19T07:06:03.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.844+0000 7f34eb3df740 1 --2- 172.21.15.103:0/2187964716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a289b370 0x55e4a28b2ba0 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.844+0000 7f34eb3df740 1 --2- 172.21.15.103:0/2187964716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a2891b40 0x55e4a28a72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.844+0000 7f34eb3df740 1 --2- 172.21.15.103:0/2187964716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a28905d0 0x55e4a289a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.844+0000 7f34eb3df740 1 -- 172.21.15.103:0/2187964716 >> 172.21.15.103:0/2187964716 conn(0x55e4a2859310 msgr2=0x55e4a288bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:03.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.844+0000 7f34eb3df740 1 -- 172.21.15.103:0/2187964716 shutdown_connections 2024-09-19T07:06:03.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.844+0000 7f34eb3df740 1 -- 172.21.15.103:0/2187964716 wait complete. 2024-09-19T07:06:03.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.844+0000 7f34eb3df740 1 -- start start 2024-09-19T07:06:03.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34eb3df740 1 Processor -- start 2024-09-19T07:06:03.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34eb3df740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a28905d0 0x55e4a28a0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34eb3df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a2891b40 0x55e4a28a0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34eb3df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a289b370 0x55e4a28a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34eb3df740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f34e41aedb0 con 0x55e4a28905d0 2024-09-19T07:06:03.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34eb3df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34e41ae6e0 con 0x55e4a2891b40 2024-09-19T07:06:03.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34eb3df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34e41ab940 con 0x55e4a289b370 2024-09-19T07:06:03.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34eb3df740 10 client.0 _ll_register_callbacks cb 0x7f34d4017630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:03.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34e2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a28905d0 0x55e4a28a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34e2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a28905d0 0x55e4a28a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34532/0 (socket says 172.21.15.103:34532) 2024-09-19T07:06:03.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34e2d9d640 1 -- 172.21.15.103:0/3951112400 learned_addr learned my addr 172.21.15.103:0/3951112400 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:03.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34e359e640 1 --2- 172.21.15.103:0/3951112400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a289b370 0x55e4a28a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34e2d9d640 1 -- 172.21.15.103:0/3951112400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a289b370 msgr2=0x55e4a28a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:03.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34e2d9d640 1 --2- 172.21.15.103:0/3951112400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a289b370 0x55e4a28a0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34e2d9d640 1 -- 172.21.15.103:0/3951112400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a2891b40 msgr2=0x55e4a28a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:03.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34e2d9d640 1 --2- 172.21.15.103:0/3951112400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a2891b40 0x55e4a28a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34e2d9d640 1 -- 172.21.15.103:0/3951112400 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a2915e20 con 0x55e4a28905d0 2024-09-19T07:06:03.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34e359e640 1 --2- 172.21.15.103:0/3951112400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a289b370 0x55e4a28a0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:03.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34e2d9d640 1 --2- 172.21.15.103:0/3951112400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a28905d0 0x55e4a28a0430 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f34d40996f0 tx=0x7f34d40ccba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c939a0e4ab87bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:03.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.845+0000 7f34ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:03.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.846+0000 7f34ca7fc640 1 -- 172.21.15.103:0/3951112400 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d401e070 con 0x55e4a28905d0 2024-09-19T07:06:03.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.846+0000 7f34ca7fc640 1 -- 172.21.15.103:0/3951112400 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8556+0+0 (secure 0 0 0) 0x7f34d409ab60 con 0x55e4a28905d0 2024-09-19T07:06:03.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.846+0000 7f34eb3df740 1 -- 172.21.15.103:0/3951112400 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34dc010b70 con 0x55e4a28905d0 2024-09-19T07:06:03.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.846+0000 7f34ca7fc640 1 -- 172.21.15.103:0/3951112400 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f34d409ada0 con 0x55e4a28905d0 2024-09-19T07:06:03.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.846+0000 7f34eb3df740 10 client.21453 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:03.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.846+0000 7f34eb3df740 1 -- 172.21.15.103:0/3951112400 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e4a2917100 con 0x55e4a28905d0 2024-09-19T07:06:03.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.846+0000 7f34ca7fc640 1 -- 172.21.15.103:0/3951112400 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34dc010b70 con 0x55e4a28905d0 2024-09-19T07:06:03.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.846+0000 7f34ca7fc640 1 -- 172.21.15.103:0/3951112400 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f34d4097190 con 0x55e4a28905d0 2024-09-19T07:06:03.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.846+0000 7f34eb3df740 10 client.21453 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:03.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.846+0000 7f34eb3df740 10 client.21453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:06:03.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.846+0000 7f34eb3df740 20 client.21453 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:03.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.846+0000 7f34eb3df740 1 --2- 172.21.15.103:0/3951112400 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e4a2917640 0x55e4a2919a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:03.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.846+0000 7f34eb3df740 4 client.21453 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:06:03.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.846+0000 7f34eb3df740 1 -- 172.21.15.103:0/3951112400 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4a27fdd70 con 0x55e4a2917640 2024-09-19T07:06:03.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.847+0000 7f34da59c640 1 --2- 172.21.15.103:0/3951112400 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e4a2917640 0x55e4a2919a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:03.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.849+0000 7f34da59c640 1 --2- 172.21.15.103:0/3951112400 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e4a2917640 0x55e4a2919a30 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2070725aeb398da server_cookie=df6572f07d27c0a8 in_seq=0 out_seq=0 2024-09-19T07:06:03.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.849+0000 7f34ca7fc640 10 client.21453 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:03.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.850+0000 7f34ca7fc640 1 -- 172.21.15.103:0/3951112400 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4a27fdd70 con 0x55e4a2917640 2024-09-19T07:06:03.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.850+0000 7f34ca7fc640 10 client.21453 handle_command_reply: tid=0 2024-09-19T07:06:03.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.850+0000 7f34eb3df740 1 client.21453 shutdown 2024-09-19T07:06:03.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.850+0000 7f34eb3df740 1 -- 172.21.15.103:0/3951112400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a28905d0 msgr2=0x55e4a28a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.850+0000 7f34eb3df740 1 --2- 172.21.15.103:0/3951112400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a28905d0 0x55e4a28a0430 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f34d40996f0 tx=0x7f34d40ccba0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.850+0000 7f34eb3df740 1 -- 172.21.15.103:0/3951112400 shutdown_connections 2024-09-19T07:06:03.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.850+0000 7f34eb3df740 1 --2- 172.21.15.103:0/3951112400 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55e4a2917640 0x55e4a2919a30 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.850+0000 7f34eb3df740 1 --2- 172.21.15.103:0/3951112400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4a289b370 0x55e4a28a0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.850+0000 7f34eb3df740 1 --2- 172.21.15.103:0/3951112400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4a2891b40 0x55e4a28a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.850+0000 7f34eb3df740 1 --2- 172.21.15.103:0/3951112400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4a28905d0 0x55e4a28a0430 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.850+0000 7f34ca7fc640 0 client.21453 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:03.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.850+0000 7f34eb3df740 1 -- 172.21.15.103:0/3951112400 >> 172.21.15.103:0/3951112400 conn(0x55e4a2859310 msgr2=0x55e4a28a62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:03.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.850+0000 7f34eb3df740 1 -- 172.21.15.103:0/3951112400 shutdown_connections 2024-09-19T07:06:03.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.850+0000 7f34eb3df740 1 -- 172.21.15.103:0/3951112400 wait complete. 2024-09-19T07:06:03.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.851+0000 7f34eb3df740 20 client.21453 trim_cache size 0 max 16384 2024-09-19T07:06:03.860 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:06:03.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.851+0000 7f34e9006640 1 -- 172.21.15.103:0/761721716 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34d00a33b0 msgr2=0x7f34d00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.851+0000 7f34e9006640 1 --2- 172.21.15.103:0/761721716 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34d00a33b0 0x7f34d00a5870 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f34dc005fd0 tx=0x7f34dc009600 comp rx=0 tx=0).stop 2024-09-19T07:06:03.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.851+0000 7f34e9006640 1 -- 172.21.15.103:0/761721716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e4101de0 msgr2=0x7f34e4198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:03.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.851+0000 7f34e9006640 1 --2- 172.21.15.103:0/761721716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e4101de0 0x7f34e4198490 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f34d400b4a0 tx=0x7f34d400b960 comp rx=0 tx=0).stop 2024-09-19T07:06:03.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.851+0000 7f34e9006640 1 -- 172.21.15.103:0/761721716 shutdown_connections 2024-09-19T07:06:03.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.851+0000 7f34e9006640 1 --2- 172.21.15.103:0/761721716 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f34d00a33b0 0x7f34d00a5870 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.851+0000 7f34e9006640 1 --2- 172.21.15.103:0/761721716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e4106f10 0x7f34e419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.851+0000 7f34e9006640 1 --2- 172.21.15.103:0/761721716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34e4102790 0x7f34e41989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.851+0000 7f34e9006640 1 --2- 172.21.15.103:0/761721716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e4101de0 0x7f34e4198490 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:03.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.851+0000 7f34e9006640 1 -- 172.21.15.103:0/761721716 >> 172.21.15.103:0/761721716 conn(0x7f34e40fd8c0 msgr2=0x7f34e40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:03.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.851+0000 7f34e9006640 1 -- 172.21.15.103:0/761721716 shutdown_connections 2024-09-19T07:06:03.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:03.851+0000 7f34e9006640 1 -- 172.21.15.103:0/761721716 wait complete. 2024-09-19T07:06:03.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:03 smithi134 ceph-mon[35356]: pgmap v525: 97 pgs: 97 active+clean; 4.9 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-19T07:06:05.605 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:06:05.606 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:06:05.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:05 smithi174 ceph-mon[35637]: pgmap v526: 97 pgs: 97 active+clean; 4.9 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-19T07:06:05.870 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:05.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:05 smithi103 ceph-mon[31158]: pgmap v526: 97 pgs: 97 active+clean; 4.9 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-19T07:06:05.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:05 smithi134 ceph-mon[35356]: pgmap v526: 97 pgs: 97 active+clean; 4.9 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-09-19T07:06:06.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.302+0000 7fcd5bb1f640 1 -- 172.21.15.103:0/114626166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd54101420 msgr2=0x7fcd5410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:06.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.302+0000 7fcd5bb1f640 1 --2- 172.21.15.103:0/114626166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd54101420 0x7fcd5410d950 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009cd0 tx=0x7fcd4c02a8e0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.303+0000 7fcd5bb1f640 1 -- 172.21.15.103:0/114626166 shutdown_connections 2024-09-19T07:06:06.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.303+0000 7fcd5bb1f640 1 --2- 172.21.15.103:0/114626166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd54101420 0x7fcd5410d950 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.303+0000 7fcd5bb1f640 1 --2- 172.21.15.103:0/114626166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd54100a80 0x7fcd54100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.303+0000 7fcd5bb1f640 1 --2- 172.21.15.103:0/114626166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd54106a80 0x7fcd54106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.303+0000 7fcd5bb1f640 1 -- 172.21.15.103:0/114626166 >> 172.21.15.103:0/114626166 conn(0x7fcd540fc7a0 msgr2=0x7fcd540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:06.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.303+0000 7fcd5bb1f640 1 -- 172.21.15.103:0/114626166 shutdown_connections 2024-09-19T07:06:06.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.303+0000 7fcd5bb1f640 1 -- 172.21.15.103:0/114626166 wait complete. 2024-09-19T07:06:06.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.303+0000 7fcd5bb1f640 1 Processor -- start 2024-09-19T07:06:06.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.303+0000 7fcd5bb1f640 1 -- start start 2024-09-19T07:06:06.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.303+0000 7fcd5bb1f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd54100a80 0x7fcd541971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.303+0000 7fcd5bb1f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd54101420 0x7fcd54197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.303+0000 7fcd5bb1f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd54106a80 0x7fcd54197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.303+0000 7fcd5bb1f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcd54110f20 con 0x7fcd54106a80 2024-09-19T07:06:06.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.303+0000 7fcd5bb1f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcd54110da0 con 0x7fcd54100a80 2024-09-19T07:06:06.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.303+0000 7fcd5bb1f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd541110a0 con 0x7fcd54101420 2024-09-19T07:06:06.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd5a0bc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd54106a80 0x7fcd54197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:06.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd5a0bc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd54106a80 0x7fcd54197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34560/0 (socket says 172.21.15.103:34560) 2024-09-19T07:06:06.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd5a0bc640 1 -- 172.21.15.103:0/807513925 learned_addr learned my addr 172.21.15.103:0/807513925 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:06.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd590ba640 1 --2- 172.21.15.103:0/807513925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd54101420 0x7fcd54197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:06.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd598bb640 1 --2- 172.21.15.103:0/807513925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd54100a80 0x7fcd541971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:06.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd5a0bc640 1 -- 172.21.15.103:0/807513925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd54101420 msgr2=0x7fcd54197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:06.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd5a0bc640 1 --2- 172.21.15.103:0/807513925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd54101420 0x7fcd54197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd5a0bc640 1 -- 172.21.15.103:0/807513925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd54100a80 msgr2=0x7fcd541971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:06.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd5a0bc640 1 --2- 172.21.15.103:0/807513925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd54100a80 0x7fcd541971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd5a0bc640 1 -- 172.21.15.103:0/807513925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd541a6a50 con 0x7fcd54106a80 2024-09-19T07:06:06.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd598bb640 1 --2- 172.21.15.103:0/807513925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd54100a80 0x7fcd541971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:06.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd590ba640 1 --2- 172.21.15.103:0/807513925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd54101420 0x7fcd54197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:06:06.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd5a0bc640 1 --2- 172.21.15.103:0/807513925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd54106a80 0x7fcd54197c60 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c006010 tx=0x7fcd4c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:06.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd4affd640 1 -- 172.21.15.103:0/807513925 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c004450 con 0x7fcd54106a80 2024-09-19T07:06:06.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd5bb1f640 1 -- 172.21.15.103:0/807513925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd541a6ce0 con 0x7fcd54106a80 2024-09-19T07:06:06.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd4affd640 1 -- 172.21.15.103:0/807513925 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd4c003e10 con 0x7fcd54106a80 2024-09-19T07:06:06.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd4affd640 1 -- 172.21.15.103:0/807513925 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0466e0 con 0x7fcd54106a80 2024-09-19T07:06:06.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.304+0000 7fcd5bb1f640 1 -- 172.21.15.103:0/807513925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd541a7220 con 0x7fcd54106a80 2024-09-19T07:06:06.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.307+0000 7fcd4affd640 1 -- 172.21.15.103:0/807513925 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fcd4c0305a0 con 0x7fcd54106a80 2024-09-19T07:06:06.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.307+0000 7fcd5def8740 1 Processor -- start 2024-09-19T07:06:06.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.307+0000 7fcd5def8740 1 -- start start 2024-09-19T07:06:06.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.307+0000 7fcd5def8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e1ab60 0x562342e1af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.307+0000 7fcd5def8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e25800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.307+0000 7fcd4affd640 1 --2- 172.21.15.103:0/807513925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd400a33b0 0x7fcd400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd5def8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e25d40 0x562342e28130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd5def8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562342d80280 con 0x562342e1b480 2024-09-19T07:06:06.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd5def8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562342d27b10 con 0x562342e1ab60 2024-09-19T07:06:06.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd5def8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562342d7c240 con 0x562342e25d40 2024-09-19T07:06:06.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd590ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e25800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:06.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd590ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e25800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34568/0 (socket says 172.21.15.103:34568) 2024-09-19T07:06:06.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd598bb640 1 --2- 172.21.15.103:0/807513925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd400a33b0 0x7fcd400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:06.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd590ba640 1 -- 172.21.15.103:0/4027622761 learned_addr learned my addr 172.21.15.103:0/4027622761 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:06.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd4affd640 1 -- 172.21.15.103:0/807513925 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7fcd4c02e030 con 0x7fcd54106a80 2024-09-19T07:06:06.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd590ba640 1 -- 172.21.15.103:0/4027622761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e25d40 msgr2=0x562342e28130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:06:06.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd590ba640 1 --2- 172.21.15.103:0/4027622761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e25d40 0x562342e28130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd590ba640 1 -- 172.21.15.103:0/4027622761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e1ab60 msgr2=0x562342e1af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:06.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd590ba640 1 --2- 172.21.15.103:0/4027622761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e1ab60 0x562342e1af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd590ba640 1 -- 172.21.15.103:0/4027622761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562342d341f0 con 0x562342e1b480 2024-09-19T07:06:06.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd598bb640 1 --2- 172.21.15.103:0/807513925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd400a33b0 0x7fcd400a5870 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcd50006fd0 tx=0x7fcd50008040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:06.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd590ba640 1 --2- 172.21.15.103:0/4027622761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e25800 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7fcd440078d0 tx=0x7fcd44007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b6a46074e9edc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:06.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.308+0000 7fcd3ffff640 1 -- 172.21.15.103:0/4027622761 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4400d950 con 0x562342e1b480 2024-09-19T07:06:06.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.309+0000 7fcd3ffff640 1 -- 172.21.15.103:0/4027622761 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd44018070 con 0x562342e1b480 2024-09-19T07:06:06.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.309+0000 7fcd3ffff640 1 -- 172.21.15.103:0/4027622761 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd440123e0 con 0x562342e1b480 2024-09-19T07:06:06.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.309+0000 7fcd5def8740 1 -- 172.21.15.103:0/4027622761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 msgr2=0x562342e25800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:06.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.309+0000 7fcd5def8740 1 --2- 172.21.15.103:0/4027622761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e25800 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7fcd440078d0 tx=0x7fcd44007d90 comp rx=0 tx=0).stop 2024-09-19T07:06:06.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.309+0000 7fcd5def8740 1 -- 172.21.15.103:0/4027622761 shutdown_connections 2024-09-19T07:06:06.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.309+0000 7fcd5def8740 1 --2- 172.21.15.103:0/4027622761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e25d40 0x562342e28130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.309+0000 7fcd5def8740 1 --2- 172.21.15.103:0/4027622761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e25800 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.309+0000 7fcd5def8740 1 --2- 172.21.15.103:0/4027622761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e1ab60 0x562342e1af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.309+0000 7fcd5def8740 1 -- 172.21.15.103:0/4027622761 >> 172.21.15.103:0/4027622761 conn(0x562342e170f0 msgr2=0x562342e174d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:06.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.309+0000 7fcd5def8740 1 -- 172.21.15.103:0/4027622761 shutdown_connections 2024-09-19T07:06:06.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.309+0000 7fcd5def8740 1 -- 172.21.15.103:0/4027622761 wait complete. 2024-09-19T07:06:06.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.310+0000 7fcd5def8740 1 -- start start 2024-09-19T07:06:06.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.310+0000 7fcd5def8740 1 Processor -- start 2024-09-19T07:06:06.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.310+0000 7fcd5def8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e1ab60 0x562342e22c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.310+0000 7fcd5def8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e231c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.310+0000 7fcd5def8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e25d40 0x562342e23700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.310+0000 7fcd5def8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562342e28910 con 0x562342e1b480 2024-09-19T07:06:06.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.310+0000 7fcd5def8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562342d3af10 con 0x562342e25d40 2024-09-19T07:06:06.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.310+0000 7fcd5def8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562342e28a90 con 0x562342e1ab60 2024-09-19T07:06:06.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.310+0000 7fcd5def8740 10 client.0 _ll_register_callbacks cb 0x562342d03a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:06.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.310+0000 7fcd590ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e231c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:06.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.310+0000 7fcd590ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e231c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34578/0 (socket says 172.21.15.103:34578) 2024-09-19T07:06:06.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.310+0000 7fcd590ba640 1 -- 172.21.15.103:0/4162232283 learned_addr learned my addr 172.21.15.103:0/4162232283 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:06.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.310+0000 7fcd598bb640 1 --2- 172.21.15.103:0/4162232283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e1ab60 0x562342e22c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:06.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.311+0000 7fcd5a0bc640 1 --2- 172.21.15.103:0/4162232283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e25d40 0x562342e23700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:06.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.311+0000 7fcd590ba640 1 -- 172.21.15.103:0/4162232283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e1ab60 msgr2=0x562342e22c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:06.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.311+0000 7fcd590ba640 1 --2- 172.21.15.103:0/4162232283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e1ab60 0x562342e22c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.311+0000 7fcd590ba640 1 -- 172.21.15.103:0/4162232283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e25d40 msgr2=0x562342e23700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:06.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.311+0000 7fcd590ba640 1 --2- 172.21.15.103:0/4162232283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e25d40 0x562342e23700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.311+0000 7fcd590ba640 1 -- 172.21.15.103:0/4162232283 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562342af16a0 con 0x562342e1b480 2024-09-19T07:06:06.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.311+0000 7fcd590ba640 1 --2- 172.21.15.103:0/4162232283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e231c0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400d530 tx=0x7fcd440137b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5b59630f278c25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:06.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.311+0000 7fcd3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:06.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.311+0000 7fcd3d7fa640 1 -- 172.21.15.103:0/4162232283 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44022020 con 0x562342e1b480 2024-09-19T07:06:06.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.311+0000 7fcd3d7fa640 1 -- 172.21.15.103:0/4162232283 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7fcd4400dd60 con 0x562342e1b480 2024-09-19T07:06:06.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.311+0000 7fcd5def8740 1 -- 172.21.15.103:0/4162232283 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd44018070 con 0x562342e1b480 2024-09-19T07:06:06.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.311+0000 7fcd3d7fa640 1 -- 172.21.15.103:0/4162232283 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcd4402d050 con 0x562342e1b480 2024-09-19T07:06:06.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.311+0000 7fcd5def8740 10 client.21465 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:06.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.311+0000 7fcd5def8740 1 -- 172.21.15.103:0/4162232283 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562342e345a0 con 0x562342e1b480 2024-09-19T07:06:06.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.312+0000 7fcd3d7fa640 1 -- 172.21.15.103:0/4162232283 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd44018070 con 0x562342e1b480 2024-09-19T07:06:06.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.312+0000 7fcd3d7fa640 1 -- 172.21.15.103:0/4162232283 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcd4401c530 con 0x562342e1b480 2024-09-19T07:06:06.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.312+0000 7fcd5def8740 10 client.21465 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:06.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.312+0000 7fcd5def8740 10 client.21465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:06:06.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.312+0000 7fcd5def8740 20 client.21465 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:06.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.312+0000 7fcd5def8740 1 --2- 172.21.15.103:0/4162232283 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562342e35a60 0x562342e37e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.312+0000 7fcd5def8740 4 client.21465 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:06:06.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.312+0000 7fcd5def8740 1 -- 172.21.15.103:0/4162232283 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562342d8d7f0 con 0x562342e35a60 2024-09-19T07:06:06.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.312+0000 7fcd598bb640 1 --2- 172.21.15.103:0/4162232283 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562342e35a60 0x562342e37e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:06.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.312+0000 7fcd3d7fa640 10 client.21465 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:06.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.312+0000 7fcd598bb640 1 --2- 172.21.15.103:0/4162232283 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562342e35a60 0x562342e37e50 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8fc8b4d144b65c9 server_cookie=6c868b0538097602 in_seq=0 out_seq=0 2024-09-19T07:06:06.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.313+0000 7fcd3d7fa640 1 -- 172.21.15.103:0/4162232283 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562342d8d7f0 con 0x562342e35a60 2024-09-19T07:06:06.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.313+0000 7fcd3d7fa640 10 client.21465 handle_command_reply: tid=0 2024-09-19T07:06:06.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.313+0000 7fcd5def8740 1 client.21465 shutdown 2024-09-19T07:06:06.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.314+0000 7fcd5def8740 1 -- 172.21.15.103:0/4162232283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 msgr2=0x562342e231c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:06.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.314+0000 7fcd5def8740 1 --2- 172.21.15.103:0/4162232283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e231c0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400d530 tx=0x7fcd440137b0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.314+0000 7fcd5def8740 1 -- 172.21.15.103:0/4162232283 shutdown_connections 2024-09-19T07:06:06.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.314+0000 7fcd5def8740 1 --2- 172.21.15.103:0/4162232283 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562342e35a60 0x562342e37e50 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.314+0000 7fcd5def8740 1 --2- 172.21.15.103:0/4162232283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e25d40 0x562342e23700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.314+0000 7fcd5def8740 1 --2- 172.21.15.103:0/4162232283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e231c0 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.314+0000 7fcd5def8740 1 --2- 172.21.15.103:0/4162232283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e1ab60 0x562342e22c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.314+0000 7fcd5def8740 1 -- 172.21.15.103:0/4162232283 >> 172.21.15.103:0/4162232283 conn(0x562342e170f0 msgr2=0x562342e1ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:06.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.314+0000 7fcd3d7fa640 0 client.21465 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:06.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.314+0000 7fcd5def8740 1 -- 172.21.15.103:0/4162232283 shutdown_connections 2024-09-19T07:06:06.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.314+0000 7fcd5def8740 1 -- 172.21.15.103:0/4162232283 wait complete. 2024-09-19T07:06:06.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.314+0000 7fcd5def8740 20 client.21465 trim_cache size 0 max 16384 2024-09-19T07:06:06.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd5def8740 1 Processor -- start 2024-09-19T07:06:06.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd5def8740 1 -- start start 2024-09-19T07:06:06.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd5def8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e1ab60 0x562342e23550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd5def8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e3d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd5def8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e25d40 0x562342e3d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd5def8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcd541afb90 con 0x562342e1b480 2024-09-19T07:06:06.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd5def8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcd541b16d0 con 0x562342e1ab60 2024-09-19T07:06:06.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd5def8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd541b0260 con 0x562342e25d40 2024-09-19T07:06:06.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd590ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e3d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:06.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd590ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e3d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34582/0 (socket says 172.21.15.103:34582) 2024-09-19T07:06:06.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd590ba640 1 -- 172.21.15.103:0/2056315378 learned_addr learned my addr 172.21.15.103:0/2056315378 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:06.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd598bb640 1 --2- 172.21.15.103:0/2056315378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e1ab60 0x562342e23550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:06.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd590ba640 1 -- 172.21.15.103:0/2056315378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e25d40 msgr2=0x562342e3d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:06.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd590ba640 1 --2- 172.21.15.103:0/2056315378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e25d40 0x562342e3d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd590ba640 1 -- 172.21.15.103:0/2056315378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e1ab60 msgr2=0x562342e23550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:06.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd590ba640 1 --2- 172.21.15.103:0/2056315378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e1ab60 0x562342e23550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd590ba640 1 -- 172.21.15.103:0/2056315378 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562342e381c0 con 0x562342e1b480 2024-09-19T07:06:06.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.331+0000 7fcd590ba640 1 --2- 172.21.15.103:0/2056315378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e3d450 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7fcd44004810 tx=0x7fcd44004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c5812dfc359f1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:06.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.332+0000 7fcd3f7fe640 1 -- 172.21.15.103:0/2056315378 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44036070 con 0x562342e1b480 2024-09-19T07:06:06.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.332+0000 7fcd3f7fe640 1 -- 172.21.15.103:0/2056315378 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd44018050 con 0x562342e1b480 2024-09-19T07:06:06.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.332+0000 7fcd3f7fe640 1 -- 172.21.15.103:0/2056315378 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44022040 con 0x562342e1b480 2024-09-19T07:06:06.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.332+0000 7fcd5def8740 1 -- 172.21.15.103:0/2056315378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 msgr2=0x562342e3d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:06.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.332+0000 7fcd5def8740 1 --2- 172.21.15.103:0/2056315378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e3d450 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7fcd44004810 tx=0x7fcd44004640 comp rx=0 tx=0).stop 2024-09-19T07:06:06.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.332+0000 7fcd5def8740 1 -- 172.21.15.103:0/2056315378 shutdown_connections 2024-09-19T07:06:06.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.332+0000 7fcd5def8740 1 --2- 172.21.15.103:0/2056315378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e25d40 0x562342e3d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.332+0000 7fcd5def8740 1 --2- 172.21.15.103:0/2056315378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1b480 0x562342e3d450 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.332+0000 7fcd5def8740 1 --2- 172.21.15.103:0/2056315378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e1ab60 0x562342e23550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.332+0000 7fcd5def8740 1 -- 172.21.15.103:0/2056315378 >> 172.21.15.103:0/2056315378 conn(0x562342e170f0 msgr2=0x562342e1cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:06.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.332+0000 7fcd5def8740 1 -- 172.21.15.103:0/2056315378 shutdown_connections 2024-09-19T07:06:06.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.332+0000 7fcd5def8740 1 -- 172.21.15.103:0/2056315378 wait complete. 2024-09-19T07:06:06.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.333+0000 7fcd5def8740 1 -- start start 2024-09-19T07:06:06.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.333+0000 7fcd5def8740 1 Processor -- start 2024-09-19T07:06:06.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.333+0000 7fcd5def8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1ab60 0x562342e29b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.333+0000 7fcd5def8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e1b480 0x562342e2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.333+0000 7fcd5def8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e25d40 0x562342e2a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.333+0000 7fcd5def8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcd541b1000 con 0x562342e1ab60 2024-09-19T07:06:06.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.333+0000 7fcd5def8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcd541b0930 con 0x562342e25d40 2024-09-19T07:06:06.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.333+0000 7fcd5def8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd541adb90 con 0x562342e1b480 2024-09-19T07:06:06.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.333+0000 7fcd5def8740 10 client.0 _ll_register_callbacks cb 0x7fcd44007780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:06.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.333+0000 7fcd598bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1ab60 0x562342e29b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:06.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.333+0000 7fcd598bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1ab60 0x562342e29b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34590/0 (socket says 172.21.15.103:34590) 2024-09-19T07:06:06.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.333+0000 7fcd598bb640 1 -- 172.21.15.103:0/991011262 learned_addr learned my addr 172.21.15.103:0/991011262 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:06.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd5a0bc640 1 --2- 172.21.15.103:0/991011262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e25d40 0x562342e2a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:06.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd590ba640 1 --2- 172.21.15.103:0/991011262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e1b480 0x562342e2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:06.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd598bb640 1 -- 172.21.15.103:0/991011262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e1b480 msgr2=0x562342e2a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:06.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd598bb640 1 --2- 172.21.15.103:0/991011262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e1b480 0x562342e2a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd598bb640 1 -- 172.21.15.103:0/991011262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e25d40 msgr2=0x562342e2a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:06.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd598bb640 1 --2- 172.21.15.103:0/991011262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e25d40 0x562342e2a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd598bb640 1 -- 172.21.15.103:0/991011262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562342e9f520 con 0x562342e1ab60 2024-09-19T07:06:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd5a0bc640 1 --2- 172.21.15.103:0/991011262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e25d40 0x562342e2a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd590ba640 1 --2- 172.21.15.103:0/991011262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e1b480 0x562342e2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd598bb640 1 --2- 172.21.15.103:0/991011262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1ab60 0x562342e29b50 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7fcd50006f80 tx=0x7fcd500047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d61c69be70dcf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd3d7fa640 1 -- 172.21.15.103:0/991011262 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd50020400 con 0x562342e1ab60 2024-09-19T07:06:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd3d7fa640 1 -- 172.21.15.103:0/991011262 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7fcd500205a0 con 0x562342e1ab60 2024-09-19T07:06:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd5def8740 1 -- 172.21.15.103:0/991011262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd44018050 con 0x562342e1ab60 2024-09-19T07:06:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd3d7fa640 1 -- 172.21.15.103:0/991011262 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcd50037050 con 0x562342e1ab60 2024-09-19T07:06:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd5def8740 10 client.21471 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd5def8740 1 -- 172.21.15.103:0/991011262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562342ea0820 con 0x562342e1ab60 2024-09-19T07:06:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.334+0000 7fcd3d7fa640 1 -- 172.21.15.103:0/991011262 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd44018050 con 0x562342e1ab60 2024-09-19T07:06:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.335+0000 7fcd3d7fa640 1 -- 172.21.15.103:0/991011262 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcd50029990 con 0x562342e1ab60 2024-09-19T07:06:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.335+0000 7fcd5def8740 10 client.21471 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.335+0000 7fcd5def8740 10 client.21471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:06:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.335+0000 7fcd5def8740 20 client.21471 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.335+0000 7fcd5def8740 1 --2- 172.21.15.103:0/991011262 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562342ea0fd0 0x562342ea33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.335+0000 7fcd5def8740 4 client.21471 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:06:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.335+0000 7fcd5def8740 1 -- 172.21.15.103:0/991011262 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562342d87d00 con 0x562342ea0fd0 2024-09-19T07:06:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.335+0000 7fcd590ba640 1 --2- 172.21.15.103:0/991011262 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562342ea0fd0 0x562342ea33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.335+0000 7fcd590ba640 1 --2- 172.21.15.103:0/991011262 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562342ea0fd0 0x562342ea33f0 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90a7763081e55b71 server_cookie=a7d20683fc0e277d in_seq=0 out_seq=0 2024-09-19T07:06:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.335+0000 7fcd3d7fa640 10 client.21471 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.336+0000 7fcd3d7fa640 1 -- 172.21.15.103:0/991011262 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562342d87d00 con 0x562342ea0fd0 2024-09-19T07:06:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.336+0000 7fcd3d7fa640 10 client.21471 handle_command_reply: tid=0 2024-09-19T07:06:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.336+0000 7fcd5def8740 1 client.21471 shutdown 2024-09-19T07:06:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.337+0000 7fcd5def8740 1 -- 172.21.15.103:0/991011262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1ab60 msgr2=0x562342e29b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.337+0000 7fcd5def8740 1 --2- 172.21.15.103:0/991011262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1ab60 0x562342e29b50 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7fcd50006f80 tx=0x7fcd500047d0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.337+0000 7fcd5def8740 1 -- 172.21.15.103:0/991011262 shutdown_connections 2024-09-19T07:06:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.337+0000 7fcd5def8740 1 --2- 172.21.15.103:0/991011262 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x562342ea0fd0 0x562342ea33f0 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.337+0000 7fcd3d7fa640 0 client.21471 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.337+0000 7fcd5def8740 1 --2- 172.21.15.103:0/991011262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562342e25d40 0x562342e2a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.337+0000 7fcd5def8740 1 --2- 172.21.15.103:0/991011262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562342e1b480 0x562342e2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.337+0000 7fcd5def8740 1 --2- 172.21.15.103:0/991011262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562342e1ab60 0x562342e29b50 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.337+0000 7fcd5def8740 1 -- 172.21.15.103:0/991011262 >> 172.21.15.103:0/991011262 conn(0x562342e170f0 msgr2=0x562342e24230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.337+0000 7fcd5def8740 1 -- 172.21.15.103:0/991011262 shutdown_connections 2024-09-19T07:06:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.337+0000 7fcd5def8740 1 -- 172.21.15.103:0/991011262 wait complete. 2024-09-19T07:06:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.337+0000 7fcd5def8740 20 client.21471 trim_cache size 0 max 16384 2024-09-19T07:06:06.343 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:06:06.344 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:06:06.344 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "f2fb2dbd-6032-43bb-89d9-554886e20e19", 2024-09-19T07:06:06.344 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:06:06.344 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:06:06.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.337+0000 7fcd5bb1f640 1 -- 172.21.15.103:0/807513925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd400a33b0 msgr2=0x7fcd400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:06.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.337+0000 7fcd5bb1f640 1 --2- 172.21.15.103:0/807513925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd400a33b0 0x7fcd400a5870 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcd50006fd0 tx=0x7fcd50008040 comp rx=0 tx=0).stop 2024-09-19T07:06:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.337+0000 7fcd5bb1f640 1 -- 172.21.15.103:0/807513925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd54106a80 msgr2=0x7fcd54197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.337+0000 7fcd5bb1f640 1 --2- 172.21.15.103:0/807513925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd54106a80 0x7fcd54197c60 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c006010 tx=0x7fcd4c003c20 comp rx=0 tx=0).stop 2024-09-19T07:06:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.338+0000 7fcd5bb1f640 1 -- 172.21.15.103:0/807513925 shutdown_connections 2024-09-19T07:06:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.338+0000 7fcd5bb1f640 1 --2- 172.21.15.103:0/807513925 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fcd400a33b0 0x7fcd400a5870 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.338+0000 7fcd5bb1f640 1 --2- 172.21.15.103:0/807513925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd54106a80 0x7fcd54197c60 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.338+0000 7fcd5bb1f640 1 --2- 172.21.15.103:0/807513925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd54101420 0x7fcd54197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.338+0000 7fcd5bb1f640 1 --2- 172.21.15.103:0/807513925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd54100a80 0x7fcd541971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.338+0000 7fcd5bb1f640 1 -- 172.21.15.103:0/807513925 >> 172.21.15.103:0/807513925 conn(0x7fcd540fc7a0 msgr2=0x7fcd540fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.338+0000 7fcd5bb1f640 1 -- 172.21.15.103:0/807513925 shutdown_connections 2024-09-19T07:06:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:06.338+0000 7fcd5bb1f640 1 -- 172.21.15.103:0/807513925 wait complete. 2024-09-19T07:06:06.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:06 smithi174 ceph-mon[35637]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T07:06:06.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:06 smithi103 ceph-mon[31158]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T07:06:06.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:06 smithi134 ceph-mon[35356]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T07:06:06.976 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:06:07.222 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:07.597 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:07 smithi103 ceph-mon[31158]: pgmap v528: 97 pgs: 97 active+clean; 5.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 110 KiB/s wr, 39 op/s 2024-09-19T07:06:07.597 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:07 smithi103 ceph-mon[31158]: scrub queued for path: / 2024-09-19T07:06:07.597 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:07 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:06:07.597 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:07 smithi103 ceph-mon[31158]: scrub summary: active paths [/] 2024-09-19T07:06:07.597 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:07 smithi103 ceph-mon[31158]: scrub summary: idle+waiting paths [/] 2024-09-19T07:06:07.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.673+0000 7f9077cb9640 1 -- 172.21.15.103:0/2526572016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9070106a80 msgr2=0x7f9070106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:07.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.673+0000 7f9077cb9640 1 --2- 172.21.15.103:0/2526572016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9070106a80 0x7f9070106e60 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f906c009930 tx=0x7f906c02a420 comp rx=0 tx=0).stop 2024-09-19T07:06:07.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.673+0000 7f9077cb9640 1 -- 172.21.15.103:0/2526572016 shutdown_connections 2024-09-19T07:06:07.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.673+0000 7f9077cb9640 1 --2- 172.21.15.103:0/2526572016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9070101420 0x7f907010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.673+0000 7f9077cb9640 1 --2- 172.21.15.103:0/2526572016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9070100a80 0x7f9070100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.673+0000 7f9077cb9640 1 --2- 172.21.15.103:0/2526572016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9070106a80 0x7f9070106e60 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.673+0000 7f9077cb9640 1 -- 172.21.15.103:0/2526572016 >> 172.21.15.103:0/2526572016 conn(0x7f90700fc7a0 msgr2=0x7f90700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:07.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.673+0000 7f9077cb9640 1 -- 172.21.15.103:0/2526572016 shutdown_connections 2024-09-19T07:06:07.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.673+0000 7f9077cb9640 1 -- 172.21.15.103:0/2526572016 wait complete. 2024-09-19T07:06:07.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.674+0000 7f9077cb9640 1 Processor -- start 2024-09-19T07:06:07.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.674+0000 7f9077cb9640 1 -- start start 2024-09-19T07:06:07.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.674+0000 7f9077cb9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9070100a80 0x7f90701973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.674+0000 7f9077cb9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9070101420 0x7f9070197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.674+0000 7f9077cb9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9070106a80 0x7f907019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.674+0000 7f9077cb9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9070110ec0 con 0x7f9070106a80 2024-09-19T07:06:07.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.674+0000 7f9077cb9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9070110d40 con 0x7f9070100a80 2024-09-19T07:06:07.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.674+0000 7f9077cb9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9070111040 con 0x7f9070101420 2024-09-19T07:06:07.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.674+0000 7f9076256640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9070106a80 0x7f907019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:07.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.674+0000 7f9076256640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9070106a80 0x7f907019bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50520/0 (socket says 172.21.15.103:50520) 2024-09-19T07:06:07.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.674+0000 7f9076256640 1 -- 172.21.15.103:0/4148549026 learned_addr learned my addr 172.21.15.103:0/4148549026 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:07.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.675+0000 7f9075a55640 1 --2- 172.21.15.103:0/4148549026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9070100a80 0x7f90701973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:07.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.675+0000 7f9075254640 1 --2- 172.21.15.103:0/4148549026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9070101420 0x7f9070197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:07.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.675+0000 7f9076256640 1 -- 172.21.15.103:0/4148549026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9070101420 msgr2=0x7f9070197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:07.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.675+0000 7f9076256640 1 --2- 172.21.15.103:0/4148549026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9070101420 0x7f9070197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.675+0000 7f9076256640 1 -- 172.21.15.103:0/4148549026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9070100a80 msgr2=0x7f90701973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:07.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.675+0000 7f9076256640 1 --2- 172.21.15.103:0/4148549026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9070100a80 0x7f90701973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.675+0000 7f9076256640 1 -- 172.21.15.103:0/4148549026 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907019c370 con 0x7f9070106a80 2024-09-19T07:06:07.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.675+0000 7f9076256640 1 --2- 172.21.15.103:0/4148549026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9070106a80 0x7f907019bc90 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f906800b970 tx=0x7f906800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:07.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.675+0000 7f9066ffd640 1 -- 172.21.15.103:0/4148549026 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068019070 con 0x7f9070106a80 2024-09-19T07:06:07.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.675+0000 7f9066ffd640 1 -- 172.21.15.103:0/4148549026 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906800cb40 con 0x7f9070106a80 2024-09-19T07:06:07.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.675+0000 7f9077cb9640 1 -- 172.21.15.103:0/4148549026 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907019c660 con 0x7f9070106a80 2024-09-19T07:06:07.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.675+0000 7f9066ffd640 1 -- 172.21.15.103:0/4148549026 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068007650 con 0x7f9070106a80 2024-09-19T07:06:07.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.675+0000 7f9077cb9640 1 -- 172.21.15.103:0/4148549026 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701a3f40 con 0x7f9070106a80 2024-09-19T07:06:07.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.677+0000 7f9066ffd640 1 -- 172.21.15.103:0/4148549026 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f906801c070 con 0x7f9070106a80 2024-09-19T07:06:07.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.678+0000 7f907a092740 1 Processor -- start 2024-09-19T07:06:07.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.678+0000 7f907a092740 1 -- start start 2024-09-19T07:06:07.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.678+0000 7f907a092740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed51b40 0x561f6ed50340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.678+0000 7f907a092740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed50910 0x561f6ed50d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.678+0000 7f907a092740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed5f740 0x561f6ed61b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.678+0000 7f907a092740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561f6ecb62e0 con 0x561f6ed51b40 2024-09-19T07:06:07.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.678+0000 7f907a092740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561f6ec5dce0 con 0x561f6ed50910 2024-09-19T07:06:07.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.678+0000 7f907a092740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561f6ecb22a0 con 0x561f6ed5f740 2024-09-19T07:06:07.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.678+0000 7f9075254640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed51b40 0x561f6ed50340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:07.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.678+0000 7f9075254640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed51b40 0x561f6ed50340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50526/0 (socket says 172.21.15.103:50526) 2024-09-19T07:06:07.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.678+0000 7f9075254640 1 -- 172.21.15.103:0/2964868852 learned_addr learned my addr 172.21.15.103:0/2964868852 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:07.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.678+0000 7f9066ffd640 1 --2- 172.21.15.103:0/4148549026 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f905c0a33b0 0x7f905c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.678+0000 7f9075a55640 1 --2- 172.21.15.103:0/2964868852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed50910 0x561f6ed50d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:07.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.679+0000 7f9075254640 1 --2- 172.21.15.103:0/2964868852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed5f740 0x561f6ed61b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:07.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.679+0000 7f9076256640 1 --2- 172.21.15.103:0/4148549026 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f905c0a33b0 0x7f905c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:07.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.679+0000 7f9075254640 1 -- 172.21.15.103:0/2964868852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed5f740 msgr2=0x561f6ed61b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:07.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.679+0000 7f9075254640 1 --2- 172.21.15.103:0/2964868852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed5f740 0x561f6ed61b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.679+0000 7f9075254640 1 -- 172.21.15.103:0/2964868852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed50910 msgr2=0x561f6ed50d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:07.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.679+0000 7f9075254640 1 --2- 172.21.15.103:0/2964868852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed50910 0x561f6ed50d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.679+0000 7f9075254640 1 -- 172.21.15.103:0/2964868852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f6ec42440 con 0x561f6ed51b40 2024-09-19T07:06:07.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.679+0000 7f9066ffd640 1 -- 172.21.15.103:0/4148549026 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f90680cf350 con 0x7f9070106a80 2024-09-19T07:06:07.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.679+0000 7f9075254640 1 --2- 172.21.15.103:0/2964868852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed51b40 0x561f6ed50340 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f9060005fd0 tx=0x7f906000b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b3e791a98162a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:07.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.679+0000 7f9076256640 1 --2- 172.21.15.103:0/4148549026 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f905c0a33b0 0x7f905c0a5870 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f906809b620 tx=0x7f906806a380 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:07.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.679+0000 7f905bfff640 1 -- 172.21.15.103:0/2964868852 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060012860 con 0x561f6ed51b40 2024-09-19T07:06:07.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.679+0000 7f905bfff640 1 -- 172.21.15.103:0/2964868852 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9060011440 con 0x561f6ed51b40 2024-09-19T07:06:07.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.679+0000 7f905bfff640 1 -- 172.21.15.103:0/2964868852 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060010630 con 0x561f6ed51b40 2024-09-19T07:06:07.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.680+0000 7f907a092740 1 -- 172.21.15.103:0/2964868852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed51b40 msgr2=0x561f6ed50340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:07.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.680+0000 7f907a092740 1 --2- 172.21.15.103:0/2964868852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed51b40 0x561f6ed50340 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f9060005fd0 tx=0x7f906000b470 comp rx=0 tx=0).stop 2024-09-19T07:06:07.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.680+0000 7f907a092740 1 -- 172.21.15.103:0/2964868852 shutdown_connections 2024-09-19T07:06:07.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.680+0000 7f907a092740 1 --2- 172.21.15.103:0/2964868852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed5f740 0x561f6ed61b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.680+0000 7f907a092740 1 --2- 172.21.15.103:0/2964868852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed50910 0x561f6ed50d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.680+0000 7f907a092740 1 --2- 172.21.15.103:0/2964868852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed51b40 0x561f6ed50340 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.680+0000 7f907a092740 1 -- 172.21.15.103:0/2964868852 >> 172.21.15.103:0/2964868852 conn(0x561f6ed4ca30 msgr2=0x561f6ed4ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:07.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.680+0000 7f907a092740 1 -- 172.21.15.103:0/2964868852 shutdown_connections 2024-09-19T07:06:07.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.680+0000 7f907a092740 1 -- 172.21.15.103:0/2964868852 wait complete. 2024-09-19T07:06:07.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.680+0000 7f907a092740 1 -- start start 2024-09-19T07:06:07.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.680+0000 7f907a092740 1 Processor -- start 2024-09-19T07:06:07.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f907a092740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed50910 0x561f6ed5f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f907a092740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed51b40 0x561f6ed5c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f907a092740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed5f740 0x561f6ed5c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f907a092740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561f6ed62630 con 0x561f6ed51b40 2024-09-19T07:06:07.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f907a092740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561f6ec71050 con 0x561f6ed5f740 2024-09-19T07:06:07.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f907a092740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561f6ed627b0 con 0x561f6ed50910 2024-09-19T07:06:07.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f9075a55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed51b40 0x561f6ed5c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:07.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f907a092740 10 client.0 _ll_register_callbacks cb 0x561f6ecb1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:07.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f9075a55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed51b40 0x561f6ed5c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50542/0 (socket says 172.21.15.103:50542) 2024-09-19T07:06:07.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f9075a55640 1 -- 172.21.15.103:0/4232964929 learned_addr learned my addr 172.21.15.103:0/4232964929 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:07.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f9075254640 1 --2- 172.21.15.103:0/4232964929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed5f740 0x561f6ed5c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:07.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f9075254640 1 --2- 172.21.15.103:0/4232964929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed50910 0x561f6ed5f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:07.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f9075a55640 1 -- 172.21.15.103:0/4232964929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed50910 msgr2=0x561f6ed5f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:07.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f9075a55640 1 --2- 172.21.15.103:0/4232964929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed50910 0x561f6ed5f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f9075a55640 1 -- 172.21.15.103:0/4232964929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed5f740 msgr2=0x561f6ed5c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:07.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f9075a55640 1 --2- 172.21.15.103:0/4232964929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed5f740 0x561f6ed5c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f9075a55640 1 -- 172.21.15.103:0/4232964929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f6ea273c0 con 0x561f6ed51b40 2024-09-19T07:06:07.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f9075a55640 1 --2- 172.21.15.103:0/4232964929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed51b40 0x561f6ed5c380 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f906c031550 tx=0x7f906c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58b969e88700070 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:07.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.681+0000 7f90597fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:07.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.682+0000 7f90597fa640 1 -- 172.21.15.103:0/4232964929 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c035040 con 0x561f6ed51b40 2024-09-19T07:06:07.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.682+0000 7f90597fa640 1 -- 172.21.15.103:0/4232964929 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f906c005020 con 0x561f6ed51b40 2024-09-19T07:06:07.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.682+0000 7f907a092740 1 -- 172.21.15.103:0/4232964929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9060011440 con 0x561f6ed51b40 2024-09-19T07:06:07.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.682+0000 7f90597fa640 1 -- 172.21.15.103:0/4232964929 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f906c005220 con 0x561f6ed51b40 2024-09-19T07:06:07.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.682+0000 7f907a092740 10 client.21483 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:07.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.682+0000 7f907a092740 1 -- 172.21.15.103:0/4232964929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f6ed6e040 con 0x561f6ed51b40 2024-09-19T07:06:07.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.682+0000 7f90597fa640 1 -- 172.21.15.103:0/4232964929 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9060011440 con 0x561f6ed51b40 2024-09-19T07:06:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.682+0000 7f90597fa640 1 -- 172.21.15.103:0/4232964929 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f906c003710 con 0x561f6ed51b40 2024-09-19T07:06:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.682+0000 7f907a092740 10 client.21483 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.682+0000 7f907a092740 10 client.21483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:06:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.682+0000 7f907a092740 20 client.21483 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.682+0000 7f907a092740 1 --2- 172.21.15.103:0/4232964929 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561f6ed6f500 0x561f6ed71940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.682+0000 7f907a092740 4 client.21483 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:06:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.682+0000 7f907a092740 1 -- 172.21.15.103:0/4232964929 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f6ecc3850 con 0x561f6ed6f500 2024-09-19T07:06:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.683+0000 7f9075254640 1 --2- 172.21.15.103:0/4232964929 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561f6ed6f500 0x561f6ed71940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.683+0000 7f90597fa640 10 client.21483 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.683+0000 7f9075254640 1 --2- 172.21.15.103:0/4232964929 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561f6ed6f500 0x561f6ed71940 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17252bbfb6223ee5 server_cookie=87713f0e9a02fc2a in_seq=0 out_seq=0 2024-09-19T07:06:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.684+0000 7f90597fa640 1 -- 172.21.15.103:0/4232964929 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f6ecc3850 con 0x561f6ed6f500 2024-09-19T07:06:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.684+0000 7f90597fa640 10 client.21483 handle_command_reply: tid=0 2024-09-19T07:06:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.684+0000 7f907a092740 1 client.21483 shutdown 2024-09-19T07:06:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.684+0000 7f907a092740 1 -- 172.21.15.103:0/4232964929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed51b40 msgr2=0x561f6ed5c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:07.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.684+0000 7f907a092740 1 --2- 172.21.15.103:0/4232964929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed51b40 0x561f6ed5c380 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f906c031550 tx=0x7f906c004270 comp rx=0 tx=0).stop 2024-09-19T07:06:07.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.684+0000 7f907a092740 1 -- 172.21.15.103:0/4232964929 shutdown_connections 2024-09-19T07:06:07.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.684+0000 7f907a092740 1 --2- 172.21.15.103:0/4232964929 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561f6ed6f500 0x561f6ed71940 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.684+0000 7f907a092740 1 --2- 172.21.15.103:0/4232964929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed5f740 0x561f6ed5c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.684+0000 7f907a092740 1 --2- 172.21.15.103:0/4232964929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed51b40 0x561f6ed5c380 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.684+0000 7f90597fa640 0 client.21483 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:07.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.684+0000 7f907a092740 1 --2- 172.21.15.103:0/4232964929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed50910 0x561f6ed5f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.684+0000 7f907a092740 1 -- 172.21.15.103:0/4232964929 >> 172.21.15.103:0/4232964929 conn(0x561f6ed4ca30 msgr2=0x561f6ed52740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:07.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.684+0000 7f907a092740 1 -- 172.21.15.103:0/4232964929 shutdown_connections 2024-09-19T07:06:07.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.684+0000 7f907a092740 1 -- 172.21.15.103:0/4232964929 wait complete. 2024-09-19T07:06:07.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.684+0000 7f907a092740 20 client.21483 trim_cache size 0 max 16384 2024-09-19T07:06:07.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.701+0000 7f907a092740 1 Processor -- start 2024-09-19T07:06:07.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.701+0000 7f907a092740 1 -- start start 2024-09-19T07:06:07.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.701+0000 7f907a092740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed50910 0x561f6ed59f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.701+0000 7f907a092740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed51b40 0x561f6ed6b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.701+0000 7f907a092740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed5f740 0x561f6ed5b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.701+0000 7f907a092740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90701ab7c0 con 0x561f6ed50910 2024-09-19T07:06:07.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.701+0000 7f907a092740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90701ad300 con 0x561f6ed5f740 2024-09-19T07:06:07.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.701+0000 7f907a092740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f90701abe90 con 0x561f6ed51b40 2024-09-19T07:06:07.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.701+0000 7f9075254640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed50910 0x561f6ed59f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:07.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.701+0000 7f9075254640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed50910 0x561f6ed59f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50552/0 (socket says 172.21.15.103:50552) 2024-09-19T07:06:07.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.701+0000 7f9075254640 1 -- 172.21.15.103:0/1563547835 learned_addr learned my addr 172.21.15.103:0/1563547835 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:07.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.701+0000 7f9075254640 1 -- 172.21.15.103:0/1563547835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed51b40 msgr2=0x561f6ed6b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:07.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.701+0000 7f9075254640 1 --2- 172.21.15.103:0/1563547835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed51b40 0x561f6ed6b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.701+0000 7f9075254640 1 -- 172.21.15.103:0/1563547835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed5f740 msgr2=0x561f6ed5b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:07.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.701+0000 7f9075254640 1 --2- 172.21.15.103:0/1563547835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed5f740 0x561f6ed5b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.701+0000 7f9075254640 1 -- 172.21.15.103:0/1563547835 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f6ed550f0 con 0x561f6ed50910 2024-09-19T07:06:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.702+0000 7f9075254640 1 --2- 172.21.15.103:0/1563547835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed50910 0x561f6ed59f20 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f9060011da0 tx=0x7f9060011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e07a3eb173f87e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.702+0000 7f905b7fe640 1 -- 172.21.15.103:0/1563547835 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060005dd0 con 0x561f6ed50910 2024-09-19T07:06:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.702+0000 7f905b7fe640 1 -- 172.21.15.103:0/1563547835 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906000b3c0 con 0x561f6ed50910 2024-09-19T07:06:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.702+0000 7f905b7fe640 1 -- 172.21.15.103:0/1563547835 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060005390 con 0x561f6ed50910 2024-09-19T07:06:07.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.702+0000 7f907a092740 1 -- 172.21.15.103:0/1563547835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed50910 msgr2=0x561f6ed59f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:07.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.702+0000 7f907a092740 1 --2- 172.21.15.103:0/1563547835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed50910 0x561f6ed59f20 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f9060011da0 tx=0x7f9060011dd0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.702+0000 7f907a092740 1 -- 172.21.15.103:0/1563547835 shutdown_connections 2024-09-19T07:06:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.702+0000 7f907a092740 1 --2- 172.21.15.103:0/1563547835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed5f740 0x561f6ed5b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.702+0000 7f907a092740 1 --2- 172.21.15.103:0/1563547835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed51b40 0x561f6ed6b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.702+0000 7f907a092740 1 --2- 172.21.15.103:0/1563547835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed50910 0x561f6ed59f20 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.702+0000 7f907a092740 1 -- 172.21.15.103:0/1563547835 >> 172.21.15.103:0/1563547835 conn(0x561f6ed4ca30 msgr2=0x561f6ed5ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.703+0000 7f907a092740 1 -- 172.21.15.103:0/1563547835 shutdown_connections 2024-09-19T07:06:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.703+0000 7f907a092740 1 -- 172.21.15.103:0/1563547835 wait complete. 2024-09-19T07:06:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.703+0000 7f907a092740 1 -- start start 2024-09-19T07:06:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.703+0000 7f907a092740 1 Processor -- start 2024-09-19T07:06:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.703+0000 7f907a092740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed50910 0x561f6ed68110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.703+0000 7f907a092740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed51b40 0x561f6ed61010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.703+0000 7f907a092740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed5f740 0x561f6ed61550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.703+0000 7f907a092740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90701acc30 con 0x561f6ed5f740 2024-09-19T07:06:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.703+0000 7f907a092740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90701ac560 con 0x561f6ed50910 2024-09-19T07:06:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.703+0000 7f907a092740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f90701a97c0 con 0x561f6ed51b40 2024-09-19T07:06:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.703+0000 7f907a092740 10 client.0 _ll_register_callbacks cb 0x7f906c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.703+0000 7f9075254640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed5f740 0x561f6ed61550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.703+0000 7f9075254640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed5f740 0x561f6ed61550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50564/0 (socket says 172.21.15.103:50564) 2024-09-19T07:06:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.703+0000 7f9075254640 1 -- 172.21.15.103:0/2311698352 learned_addr learned my addr 172.21.15.103:0/2311698352 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.704+0000 7f9075a55640 1 --2- 172.21.15.103:0/2311698352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed51b40 0x561f6ed61010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.704+0000 7f9075254640 1 --2- 172.21.15.103:0/2311698352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed50910 0x561f6ed68110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.704+0000 7f9075254640 1 -- 172.21.15.103:0/2311698352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed51b40 msgr2=0x561f6ed61010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.704+0000 7f9075254640 1 --2- 172.21.15.103:0/2311698352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed51b40 0x561f6ed61010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.704+0000 7f9075254640 1 -- 172.21.15.103:0/2311698352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed50910 msgr2=0x561f6ed68110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.704+0000 7f9075254640 1 --2- 172.21.15.103:0/2311698352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed50910 0x561f6ed68110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.704+0000 7f9075254640 1 -- 172.21.15.103:0/2311698352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f6ed61d50 con 0x561f6ed5f740 2024-09-19T07:06:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.704+0000 7f9075254640 1 --2- 172.21.15.103:0/2311698352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed5f740 0x561f6ed61550 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f906000ba00 tx=0x7f906000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34125e7273df56d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.704+0000 7f90597fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.704+0000 7f90597fa640 1 -- 172.21.15.103:0/2311698352 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906002f040 con 0x561f6ed5f740 2024-09-19T07:06:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.704+0000 7f90597fa640 1 -- 172.21.15.103:0/2311698352 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f9060011440 con 0x561f6ed5f740 2024-09-19T07:06:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.704+0000 7f907a092740 1 -- 172.21.15.103:0/2311698352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f906000b3c0 con 0x561f6ed5f740 2024-09-19T07:06:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.704+0000 7f90597fa640 1 -- 172.21.15.103:0/2311698352 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9060011640 con 0x561f6ed5f740 2024-09-19T07:06:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.704+0000 7f907a092740 10 client.21489 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.704+0000 7f907a092740 1 -- 172.21.15.103:0/2311698352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f6edda100 con 0x561f6ed5f740 2024-09-19T07:06:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.704+0000 7f90597fa640 1 -- 172.21.15.103:0/2311698352 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906000b3c0 con 0x561f6ed5f740 2024-09-19T07:06:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.705+0000 7f90597fa640 1 -- 172.21.15.103:0/2311698352 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9060035070 con 0x561f6ed5f740 2024-09-19T07:06:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.705+0000 7f907a092740 10 client.21489 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.705+0000 7f907a092740 10 client.21489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:06:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.705+0000 7f907a092740 20 client.21489 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:07.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.705+0000 7f907a092740 1 --2- 172.21.15.103:0/2311698352 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561f6edda520 0x561f6eddc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:07.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.705+0000 7f907a092740 4 client.21489 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:06:07.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.705+0000 7f907a092740 1 -- 172.21.15.103:0/2311698352 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f6ecbdd60 con 0x561f6edda520 2024-09-19T07:06:07.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.705+0000 7f9075a55640 1 --2- 172.21.15.103:0/2311698352 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561f6edda520 0x561f6eddc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:07.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.705+0000 7f9075a55640 1 --2- 172.21.15.103:0/2311698352 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561f6edda520 0x561f6eddc910 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86b07e133f8b453 server_cookie=6193d85b5c74c7a3 in_seq=0 out_seq=0 2024-09-19T07:06:07.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.705+0000 7f90597fa640 10 client.21489 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:07.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f90597fa640 1 -- 172.21.15.103:0/2311698352 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561f6ecbdd60 con 0x561f6edda520 2024-09-19T07:06:07.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f90597fa640 10 client.21489 handle_command_reply: tid=0 2024-09-19T07:06:07.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f907a092740 1 client.21489 shutdown 2024-09-19T07:06:07.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f907a092740 1 -- 172.21.15.103:0/2311698352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed5f740 msgr2=0x561f6ed61550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:07.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f907a092740 1 --2- 172.21.15.103:0/2311698352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed5f740 0x561f6ed61550 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f906000ba00 tx=0x7f906000bf60 comp rx=0 tx=0).stop 2024-09-19T07:06:07.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f907a092740 1 -- 172.21.15.103:0/2311698352 shutdown_connections 2024-09-19T07:06:07.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f907a092740 1 --2- 172.21.15.103:0/2311698352 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x561f6edda520 0x561f6eddc910 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f907a092740 1 --2- 172.21.15.103:0/2311698352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f6ed5f740 0x561f6ed61550 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f907a092740 1 --2- 172.21.15.103:0/2311698352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f6ed51b40 0x561f6ed61010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f907a092740 1 --2- 172.21.15.103:0/2311698352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6ed50910 0x561f6ed68110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f90597fa640 0 client.21489 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:07.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f907a092740 1 -- 172.21.15.103:0/2311698352 >> 172.21.15.103:0/2311698352 conn(0x561f6ed4ca30 msgr2=0x561f6ed5b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:07.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f907a092740 1 -- 172.21.15.103:0/2311698352 shutdown_connections 2024-09-19T07:06:07.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f907a092740 1 -- 172.21.15.103:0/2311698352 wait complete. 2024-09-19T07:06:07.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f907a092740 20 client.21489 trim_cache size 0 max 16384 2024-09-19T07:06:07.712 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:06:07.712 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:06:07.712 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:06:07.713 INFO:teuthology.orchestra.run.smithi103.stdout: "f2fb2dbd-6032-43bb-89d9-554886e20e19": { 2024-09-19T07:06:07.713 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:06:07.713 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "f2fb2dbd-6032-43bb-89d9-554886e20e19", 2024-09-19T07:06:07.713 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:06:07.713 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:07.713 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:07.713 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:06:07.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f9077cb9640 1 -- 172.21.15.103:0/4148549026 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f905c0a33b0 msgr2=0x7f905c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:07.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f9077cb9640 1 --2- 172.21.15.103:0/4148549026 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f905c0a33b0 0x7f905c0a5870 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f906809b620 tx=0x7f906806a380 comp rx=0 tx=0).stop 2024-09-19T07:06:07.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f9077cb9640 1 -- 172.21.15.103:0/4148549026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9070106a80 msgr2=0x7f907019bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:07.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f9077cb9640 1 --2- 172.21.15.103:0/4148549026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9070106a80 0x7f907019bc90 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f906800b970 tx=0x7f906800be30 comp rx=0 tx=0).stop 2024-09-19T07:06:07.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f9077cb9640 1 -- 172.21.15.103:0/4148549026 shutdown_connections 2024-09-19T07:06:07.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f9077cb9640 1 --2- 172.21.15.103:0/4148549026 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f905c0a33b0 0x7f905c0a5870 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f9077cb9640 1 --2- 172.21.15.103:0/4148549026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9070106a80 0x7f907019bc90 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f9077cb9640 1 --2- 172.21.15.103:0/4148549026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9070101420 0x7f9070197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.706+0000 7f9077cb9640 1 --2- 172.21.15.103:0/4148549026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9070100a80 0x7f90701973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:07.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.707+0000 7f9077cb9640 1 -- 172.21.15.103:0/4148549026 >> 172.21.15.103:0/4148549026 conn(0x7f90700fc7a0 msgr2=0x7f90700fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:07.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.707+0000 7f9077cb9640 1 -- 172.21.15.103:0/4148549026 shutdown_connections 2024-09-19T07:06:07.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:07.707+0000 7f9077cb9640 1 -- 172.21.15.103:0/4148549026 wait complete. 2024-09-19T07:06:07.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:07 smithi174 ceph-mon[35637]: pgmap v528: 97 pgs: 97 active+clean; 5.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 110 KiB/s wr, 39 op/s 2024-09-19T07:06:07.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:07 smithi174 ceph-mon[35637]: scrub queued for path: / 2024-09-19T07:06:07.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:07 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:06:07.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:07 smithi174 ceph-mon[35637]: scrub summary: active paths [/] 2024-09-19T07:06:07.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:07 smithi174 ceph-mon[35637]: scrub summary: idle+waiting paths [/] 2024-09-19T07:06:07.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:07 smithi134 ceph-mon[35356]: pgmap v528: 97 pgs: 97 active+clean; 5.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 110 KiB/s wr, 39 op/s 2024-09-19T07:06:07.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:07 smithi134 ceph-mon[35356]: scrub queued for path: / 2024-09-19T07:06:07.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:07 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:06:07.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:07 smithi134 ceph-mon[35356]: scrub summary: active paths [/] 2024-09-19T07:06:07.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:07 smithi134 ceph-mon[35356]: scrub summary: idle+waiting paths [/] 2024-09-19T07:06:08.426 INFO:tasks.cephfs.filesystem:scrub status for tag:f2fb2dbd-6032-43bb-89d9-554886e20e19 - {'path': '/', 'tag': 'f2fb2dbd-6032-43bb-89d9-554886e20e19', 'options': 'recursive,force'} 2024-09-19T07:06:09.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:09 smithi174 ceph-mon[35637]: pgmap v529: 97 pgs: 97 active+clean; 5.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 110 KiB/s wr, 39 op/s 2024-09-19T07:06:09.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:09 smithi103 ceph-mon[31158]: pgmap v529: 97 pgs: 97 active+clean; 5.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 110 KiB/s wr, 39 op/s 2024-09-19T07:06:10.025 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:09 smithi134 ceph-mon[35356]: pgmap v529: 97 pgs: 97 active+clean; 5.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 110 KiB/s wr, 39 op/s 2024-09-19T07:06:10.833 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:10 smithi174 ceph-mon[35637]: scrub summary: idle 2024-09-19T07:06:10.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:10 smithi103 ceph-mon[31158]: scrub summary: idle 2024-09-19T07:06:10.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:10 smithi134 ceph-mon[35356]: scrub summary: idle 2024-09-19T07:06:11.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:11 smithi174 ceph-mon[35637]: pgmap v530: 97 pgs: 97 active+clean; 5.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 110 KiB/s wr, 41 op/s 2024-09-19T07:06:11.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:11 smithi103 ceph-mon[31158]: pgmap v530: 97 pgs: 97 active+clean; 5.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 110 KiB/s wr, 41 op/s 2024-09-19T07:06:11.937 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:11 smithi134 ceph-mon[35356]: pgmap v530: 97 pgs: 97 active+clean; 5.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 110 KiB/s wr, 41 op/s 2024-09-19T07:06:13.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:12 smithi174 ceph-mon[35637]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T07:06:13.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:12 smithi103 ceph-mon[31158]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T07:06:13.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:12 smithi134 ceph-mon[35356]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T07:06:14.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:13 smithi174 ceph-mon[35637]: pgmap v532: 97 pgs: 97 active+clean; 5.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 138 KiB/s wr, 51 op/s 2024-09-19T07:06:14.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:13 smithi103 ceph-mon[31158]: pgmap v532: 97 pgs: 97 active+clean; 5.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 138 KiB/s wr, 51 op/s 2024-09-19T07:06:14.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:13 smithi134 ceph-mon[35356]: pgmap v532: 97 pgs: 97 active+clean; 5.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 138 KiB/s wr, 51 op/s 2024-09-19T07:06:14.275 INFO:tasks.workunit.client.0.smithi103.stdout:561/600: open fd = 3 2024-09-19T07:06:14.276 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.276 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.276 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.276 INFO:tasks.workunit.client.0.smithi103.stdout:562/600: open fd = 3 2024-09-19T07:06:14.276 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.276 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.276 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.276 INFO:tasks.workunit.client.0.smithi103.stdout:563/600: open fd = 3 2024-09-19T07:06:14.276 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.276 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.276 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.276 INFO:tasks.workunit.client.0.smithi103.stdout:564/600: open fd = 3 2024-09-19T07:06:14.277 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.277 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.277 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.277 INFO:tasks.workunit.client.0.smithi103.stdout:565/600: open fd = 3 2024-09-19T07:06:14.277 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.277 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.277 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.277 INFO:tasks.workunit.client.0.smithi103.stdout:566/600: open fd = 3 2024-09-19T07:06:14.277 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.277 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.277 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.278 INFO:tasks.workunit.client.0.smithi103.stdout:567/600: open fd = 3 2024-09-19T07:06:14.278 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.278 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.278 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.278 INFO:tasks.workunit.client.0.smithi103.stdout:568/600: open fd = 3 2024-09-19T07:06:14.278 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.278 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.278 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.278 INFO:tasks.workunit.client.0.smithi103.stdout:569/600: open fd = 3 2024-09-19T07:06:14.278 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.278 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.278 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.278 INFO:tasks.workunit.client.0.smithi103.stdout:570/600: open fd = 3 2024-09-19T07:06:14.279 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.279 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.279 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.279 INFO:tasks.workunit.client.0.smithi103.stdout:571/600: open fd = 3 2024-09-19T07:06:14.279 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.279 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.279 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.279 INFO:tasks.workunit.client.0.smithi103.stdout:572/600: open fd = 3 2024-09-19T07:06:14.279 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.279 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.279 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.280 INFO:tasks.workunit.client.0.smithi103.stdout:573/600: open fd = 3 2024-09-19T07:06:14.280 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.280 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.280 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.280 INFO:tasks.workunit.client.0.smithi103.stdout:574/600: open fd = 3 2024-09-19T07:06:14.280 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.280 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.280 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.280 INFO:tasks.workunit.client.0.smithi103.stdout:575/600: open fd = 3 2024-09-19T07:06:14.280 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.280 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.280 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.280 INFO:tasks.workunit.client.0.smithi103.stdout:576/600: open fd = 3 2024-09-19T07:06:14.280 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.281 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.281 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.281 INFO:tasks.workunit.client.0.smithi103.stdout:577/600: open fd = 3 2024-09-19T07:06:14.281 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.281 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.281 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.281 INFO:tasks.workunit.client.0.smithi103.stdout:578/600: open fd = 3 2024-09-19T07:06:14.281 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.281 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.281 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.281 INFO:tasks.workunit.client.0.smithi103.stdout:579/600: open fd = 3 2024-09-19T07:06:14.281 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.282 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.282 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.282 INFO:tasks.workunit.client.0.smithi103.stdout:580/600: open fd = 3 2024-09-19T07:06:14.282 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.282 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.282 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.282 INFO:tasks.workunit.client.0.smithi103.stdout:581/600: open fd = 3 2024-09-19T07:06:14.282 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.282 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.282 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.282 INFO:tasks.workunit.client.0.smithi103.stdout:582/600: open fd = 3 2024-09-19T07:06:14.282 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.282 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.282 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.283 INFO:tasks.workunit.client.0.smithi103.stdout:583/600: open fd = 3 2024-09-19T07:06:14.283 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.283 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.283 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.283 INFO:tasks.workunit.client.0.smithi103.stdout:584/600: open fd = 3 2024-09-19T07:06:14.283 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.283 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.283 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.283 INFO:tasks.workunit.client.0.smithi103.stdout:585/600: open fd = 3 2024-09-19T07:06:14.283 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.283 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.283 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.283 INFO:tasks.workunit.client.0.smithi103.stdout:586/600: open fd = 3 2024-09-19T07:06:14.284 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.284 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.284 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.284 INFO:tasks.workunit.client.0.smithi103.stdout:587/600: open fd = 3 2024-09-19T07:06:14.284 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.284 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.284 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.284 INFO:tasks.workunit.client.0.smithi103.stdout:588/600: open fd = 3 2024-09-19T07:06:14.284 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.284 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.284 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.284 INFO:tasks.workunit.client.0.smithi103.stdout:589/600: open fd = 3 2024-09-19T07:06:14.284 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.284 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.285 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.285 INFO:tasks.workunit.client.0.smithi103.stdout:590/600: open fd = 3 2024-09-19T07:06:14.285 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.285 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.285 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.285 INFO:tasks.workunit.client.0.smithi103.stdout:591/600: open fd = 3 2024-09-19T07:06:14.285 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.285 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.285 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.285 INFO:tasks.workunit.client.0.smithi103.stdout:592/600: open fd = 3 2024-09-19T07:06:14.285 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.285 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.285 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.285 INFO:tasks.workunit.client.0.smithi103.stdout:593/600: open fd = 3 2024-09-19T07:06:14.285 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.286 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.286 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.286 INFO:tasks.workunit.client.0.smithi103.stdout:594/600: open fd = 3 2024-09-19T07:06:14.286 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.286 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.286 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.286 INFO:tasks.workunit.client.0.smithi103.stdout:595/600: open fd = 3 2024-09-19T07:06:14.286 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.286 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.286 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.286 INFO:tasks.workunit.client.0.smithi103.stdout:596/600: open fd = 3 2024-09-19T07:06:14.286 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.286 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.287 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.287 INFO:tasks.workunit.client.0.smithi103.stdout:597/600: open fd = 3 2024-09-19T07:06:14.287 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.287 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.287 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.287 INFO:tasks.workunit.client.0.smithi103.stdout:598/600: open fd = 3 2024-09-19T07:06:14.287 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.287 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.287 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.287 INFO:tasks.workunit.client.0.smithi103.stdout:599/600: open fd = 3 2024-09-19T07:06:14.287 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.287 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.287 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.287 INFO:tasks.workunit.client.0.smithi103.stdout:600/600: open fd = 3 2024-09-19T07:06:14.288 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.288 INFO:tasks.workunit.client.0.smithi103.stdout:write ret = 32 2024-09-19T07:06:14.288 INFO:tasks.workunit.client.0.smithi103.stdout:pread ret = 64 2024-09-19T07:06:14.288 INFO:tasks.workunit.client.0.smithi103.stderr:+ echo OK 2024-09-19T07:06:14.289 INFO:tasks.workunit.client.0.smithi103.stdout:OK 2024-09-19T07:06:14.289 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T07:06:14.289 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T07:06:14.332 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-09-19T07:06:14.332 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T07:06:14.676 DEBUG:teuthology.parallel:result is None 2024-09-19T07:06:14.677 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T07:06:14.709 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T07:06:14.709 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-19T07:06:14.725 INFO:tasks.quiescer:joining Quiescers 2024-09-19T07:06:14.725 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-19T07:06:14.725 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:06:14.996 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:15.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.444+0000 7f69177aa640 1 -- 172.21.15.103:0/696582852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6910101420 msgr2=0x7f691010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:15.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.444+0000 7f69177aa640 1 --2- 172.21.15.103:0/696582852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6910101420 0x7f691010d950 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f6908009cd0 tx=0x7f690802a8e0 comp rx=0 tx=0).stop 2024-09-19T07:06:15.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.444+0000 7f69177aa640 1 -- 172.21.15.103:0/696582852 shutdown_connections 2024-09-19T07:06:15.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.444+0000 7f69177aa640 1 --2- 172.21.15.103:0/696582852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6910101420 0x7f691010d950 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:15.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.444+0000 7f69177aa640 1 --2- 172.21.15.103:0/696582852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6910100a80 0x7f6910100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:15.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.444+0000 7f69177aa640 1 --2- 172.21.15.103:0/696582852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6910106a80 0x7f6910106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:15.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.444+0000 7f69177aa640 1 -- 172.21.15.103:0/696582852 >> 172.21.15.103:0/696582852 conn(0x7f69100fc7a0 msgr2=0x7f69100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:15.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.444+0000 7f69177aa640 1 -- 172.21.15.103:0/696582852 shutdown_connections 2024-09-19T07:06:15.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.444+0000 7f69177aa640 1 -- 172.21.15.103:0/696582852 wait complete. 2024-09-19T07:06:15.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.444+0000 7f69177aa640 1 Processor -- start 2024-09-19T07:06:15.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.444+0000 7f69177aa640 1 -- start start 2024-09-19T07:06:15.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f69177aa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6910100a80 0x7f6910102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:15.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f69177aa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6910101420 0x7f6910102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:15.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f69177aa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6910106a80 0x7f69101030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:15.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f69177aa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6910110ea0 con 0x7f6910106a80 2024-09-19T07:06:15.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f69177aa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6910110d20 con 0x7f6910100a80 2024-09-19T07:06:15.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f69177aa640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6910111020 con 0x7f6910101420 2024-09-19T07:06:15.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f6915d47640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6910106a80 0x7f69101030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:15.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f6915d47640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6910106a80 0x7f69101030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50574/0 (socket says 172.21.15.103:50574) 2024-09-19T07:06:15.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f6915d47640 1 -- 172.21.15.103:0/1919474152 learned_addr learned my addr 172.21.15.103:0/1919474152 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:15.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f6914d45640 1 --2- 172.21.15.103:0/1919474152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6910101420 0x7f6910102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:15.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f6915546640 1 --2- 172.21.15.103:0/1919474152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6910100a80 0x7f6910102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:15.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f6915d47640 1 -- 172.21.15.103:0/1919474152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6910101420 msgr2=0x7f6910102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:15.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f6915d47640 1 --2- 172.21.15.103:0/1919474152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6910101420 0x7f6910102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:15.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f6915d47640 1 -- 172.21.15.103:0/1919474152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6910100a80 msgr2=0x7f6910102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:15.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f6915d47640 1 --2- 172.21.15.103:0/1919474152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6910100a80 0x7f6910102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:15.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f6915d47640 1 -- 172.21.15.103:0/1919474152 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69101a68b0 con 0x7f6910106a80 2024-09-19T07:06:15.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f6915546640 1 --2- 172.21.15.103:0/1919474152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6910100a80 0x7f6910102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:06:15.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f6915d47640 1 --2- 172.21.15.103:0/1919474152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6910106a80 0x7f69101030d0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f6908005cc0 tx=0x7f6908003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:15.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f69067fc640 1 -- 172.21.15.103:0/1919474152 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908004040 con 0x7f6910106a80 2024-09-19T07:06:15.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f69067fc640 1 -- 172.21.15.103:0/1919474152 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69080046c0 con 0x7f6910106a80 2024-09-19T07:06:15.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f69177aa640 1 -- 172.21.15.103:0/1919474152 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69101a6b40 con 0x7f6910106a80 2024-09-19T07:06:15.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.445+0000 7f69067fc640 1 -- 172.21.15.103:0/1919474152 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908046480 con 0x7f6910106a80 2024-09-19T07:06:15.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.446+0000 7f69177aa640 1 -- 172.21.15.103:0/1919474152 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69101a7080 con 0x7f6910106a80 2024-09-19T07:06:15.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.447+0000 7f69177aa640 1 -- 172.21.15.103:0/1919474152 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6910100ee0 con 0x7f6910106a80 2024-09-19T07:06:15.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.448+0000 7f69067fc640 1 -- 172.21.15.103:0/1919474152 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f69080305a0 con 0x7f6910106a80 2024-09-19T07:06:15.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.448+0000 7f69067fc640 1 --2- 172.21.15.103:0/1919474152 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68fc0a33b0 0x7f68fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:15.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.448+0000 7f6915546640 1 --2- 172.21.15.103:0/1919474152 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68fc0a33b0 0x7f68fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:15.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.448+0000 7f69067fc640 1 -- 172.21.15.103:0/1919474152 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f690802e030 con 0x7f6910106a80 2024-09-19T07:06:15.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.449+0000 7f6915546640 1 --2- 172.21.15.103:0/1919474152 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68fc0a33b0 0x7f68fc0a5870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6900009930 tx=0x7f6900008040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:15.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.452+0000 7f69067fc640 1 -- 172.21.15.103:0/1919474152 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f690804b060 con 0x7f6910106a80 2024-09-19T07:06:15.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.654+0000 7f69177aa640 1 -- 172.21.15.103:0/1919474152 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69101a72b0 con 0x7f6910106a80 2024-09-19T07:06:15.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.655+0000 7f69067fc640 1 -- 172.21.15.103:0/1919474152 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f69080bdb40 con 0x7f6910106a80 2024-09-19T07:06:15.656 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:06:15.656 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:06:15.656 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:06:15.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.657+0000 7f69177aa640 1 -- 172.21.15.103:0/1919474152 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68fc0a33b0 msgr2=0x7f68fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:15.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.657+0000 7f69177aa640 1 --2- 172.21.15.103:0/1919474152 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68fc0a33b0 0x7f68fc0a5870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6900009930 tx=0x7f6900008040 comp rx=0 tx=0).stop 2024-09-19T07:06:15.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.657+0000 7f69177aa640 1 -- 172.21.15.103:0/1919474152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6910106a80 msgr2=0x7f69101030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:15.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.657+0000 7f69177aa640 1 --2- 172.21.15.103:0/1919474152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6910106a80 0x7f69101030d0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f6908005cc0 tx=0x7f6908003970 comp rx=0 tx=0).stop 2024-09-19T07:06:15.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.657+0000 7f69177aa640 1 -- 172.21.15.103:0/1919474152 shutdown_connections 2024-09-19T07:06:15.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.657+0000 7f69177aa640 1 --2- 172.21.15.103:0/1919474152 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68fc0a33b0 0x7f68fc0a5870 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:15.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.657+0000 7f69177aa640 1 --2- 172.21.15.103:0/1919474152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6910106a80 0x7f69101030d0 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:15.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.657+0000 7f69177aa640 1 --2- 172.21.15.103:0/1919474152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6910101420 0x7f6910102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:15.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.657+0000 7f69177aa640 1 --2- 172.21.15.103:0/1919474152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6910100a80 0x7f6910102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:15.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.657+0000 7f69177aa640 1 -- 172.21.15.103:0/1919474152 >> 172.21.15.103:0/1919474152 conn(0x7f69100fc7a0 msgr2=0x7f691006a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:15.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.657+0000 7f69177aa640 1 -- 172.21.15.103:0/1919474152 shutdown_connections 2024-09-19T07:06:15.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:15.657+0000 7f69177aa640 1 -- 172.21.15.103:0/1919474152 wait complete. 2024-09-19T07:06:16.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:15 smithi174 ceph-mon[35637]: pgmap v533: 97 pgs: 97 active+clean; 5.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 11 op/s 2024-09-19T07:06:16.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:15 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1919474152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:06:16.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:15 smithi103 ceph-mon[31158]: pgmap v533: 97 pgs: 97 active+clean; 5.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 11 op/s 2024-09-19T07:06:16.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:15 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1919474152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:06:16.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:15 smithi134 ceph-mon[35356]: pgmap v533: 97 pgs: 97 active+clean; 5.4 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 11 op/s 2024-09-19T07:06:16.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:15 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1919474152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:06:16.330 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --cancel --all' 2024-09-19T07:06:16.330 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --cancel --all 2024-09-19T07:06:16.577 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.875+0000 7f26c94e4640 1 -- 172.21.15.103:0/1231227978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26c41009b0 msgr2=0x7f26c4100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.875+0000 7f26c94e4640 1 --2- 172.21.15.103:0/1231227978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26c41009b0 0x7f26c4100e10 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009930 tx=0x7f26b802a420 comp rx=0 tx=0).stop 2024-09-19T07:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.875+0000 7f26c94e4640 1 -- 172.21.15.103:0/1231227978 shutdown_connections 2024-09-19T07:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.875+0000 7f26c94e4640 1 --2- 172.21.15.103:0/1231227978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26c4101350 0x7f26c410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.875+0000 7f26c94e4640 1 --2- 172.21.15.103:0/1231227978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26c41009b0 0x7f26c4100e10 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.875+0000 7f26c94e4640 1 --2- 172.21.15.103:0/1231227978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26c41069c0 0x7f26c4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.875+0000 7f26c94e4640 1 -- 172.21.15.103:0/1231227978 >> 172.21.15.103:0/1231227978 conn(0x7f26c4077990 msgr2=0x7f26c40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.875+0000 7f26c94e4640 1 -- 172.21.15.103:0/1231227978 shutdown_connections 2024-09-19T07:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.875+0000 7f26c94e4640 1 -- 172.21.15.103:0/1231227978 wait complete. 2024-09-19T07:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.876+0000 7f26c94e4640 1 Processor -- start 2024-09-19T07:06:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.876+0000 7f26c94e4640 1 -- start start 2024-09-19T07:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.876+0000 7f26c94e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26c41009b0 0x7f26c4199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.876+0000 7f26c94e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26c4101350 0x7f26c41996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.876+0000 7f26c94e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26c41069c0 0x7f26c41a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.876+0000 7f26c94e4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26c4110e00 con 0x7f26c4101350 2024-09-19T07:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.876+0000 7f26c94e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26c4110c80 con 0x7f26c41069c0 2024-09-19T07:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.876+0000 7f26c94e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26c4110f80 con 0x7f26c41009b0 2024-09-19T07:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.876+0000 7f26c27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26c4101350 0x7f26c41996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.876+0000 7f26c27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26c4101350 0x7f26c41996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50592/0 (socket says 172.21.15.103:50592) 2024-09-19T07:06:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.876+0000 7f26c27fc640 1 -- 172.21.15.103:0/979630642 learned_addr learned my addr 172.21.15.103:0/979630642 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.876+0000 7f26c37fe640 1 --2- 172.21.15.103:0/979630642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26c41069c0 0x7f26c41a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.876+0000 7f26c2ffd640 1 --2- 172.21.15.103:0/979630642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26c41009b0 0x7f26c4199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.877+0000 7f26c27fc640 1 -- 172.21.15.103:0/979630642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26c41009b0 msgr2=0x7f26c4199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.877+0000 7f26c27fc640 1 --2- 172.21.15.103:0/979630642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26c41009b0 0x7f26c4199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.877+0000 7f26c27fc640 1 -- 172.21.15.103:0/979630642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26c41069c0 msgr2=0x7f26c41a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.877+0000 7f26c27fc640 1 --2- 172.21.15.103:0/979630642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26c41069c0 0x7f26c41a4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.877+0000 7f26c27fc640 1 -- 172.21.15.103:0/979630642 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c41a4740 con 0x7f26c4101350 2024-09-19T07:06:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.877+0000 7f26c2ffd640 1 --2- 172.21.15.103:0/979630642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26c41009b0 0x7f26c4199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.877+0000 7f26c27fc640 1 --2- 172.21.15.103:0/979630642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26c4101350 0x7f26c41996c0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009a50 tx=0x7f26b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.877+0000 7f26abfff640 1 -- 172.21.15.103:0/979630642 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b803f030 con 0x7f26c4101350 2024-09-19T07:06:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.877+0000 7f26c94e4640 1 -- 172.21.15.103:0/979630642 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c41a4910 con 0x7f26c4101350 2024-09-19T07:06:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.877+0000 7f26abfff640 1 -- 172.21.15.103:0/979630642 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b8003ce0 con 0x7f26c4101350 2024-09-19T07:06:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.877+0000 7f26abfff640 1 -- 172.21.15.103:0/979630642 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8035070 con 0x7f26c4101350 2024-09-19T07:06:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.877+0000 7f26c94e4640 1 -- 172.21.15.103:0/979630642 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c41a4d70 con 0x7f26c4101350 2024-09-19T07:06:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.879+0000 7f26abfff640 1 -- 172.21.15.103:0/979630642 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f26b8042030 con 0x7f26c4101350 2024-09-19T07:06:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.879+0000 7f26cc285740 1 Processor -- start 2024-09-19T07:06:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.879+0000 7f26cc285740 1 -- start start 2024-09-19T07:06:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26cc285740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdd2af0 0x56116bdd2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26cc285740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdd3410 0x56116bddd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26cc285740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdddcb0 0x56116bde00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26cc285740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56116bd372f0 con 0x56116bdddcb0 2024-09-19T07:06:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26cc285740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56116bcdf270 con 0x56116bdd3410 2024-09-19T07:06:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26cc285740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56116bd332b0 con 0x56116bdd2af0 2024-09-19T07:06:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26c27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdddcb0 0x56116bde00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26c27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdddcb0 0x56116bde00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50604/0 (socket says 172.21.15.103:50604) 2024-09-19T07:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26c27fc640 1 -- 172.21.15.103:0/2052797378 learned_addr learned my addr 172.21.15.103:0/2052797378 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26c2ffd640 1 --2- 172.21.15.103:0/2052797378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdd3410 0x56116bddd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26abfff640 1 --2- 172.21.15.103:0/979630642 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26ac0a3460 0x7f26ac0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26c37fe640 1 --2- 172.21.15.103:0/2052797378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdd2af0 0x56116bdd2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26c37fe640 1 --2- 172.21.15.103:0/979630642 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26ac0a3460 0x7f26ac0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26c2ffd640 1 -- 172.21.15.103:0/2052797378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdd2af0 msgr2=0x56116bdd2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26c2ffd640 1 --2- 172.21.15.103:0/2052797378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdd2af0 0x56116bdd2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26c2ffd640 1 -- 172.21.15.103:0/2052797378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdddcb0 msgr2=0x56116bde00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26c2ffd640 1 --2- 172.21.15.103:0/2052797378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdddcb0 0x56116bde00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26c2ffd640 1 -- 172.21.15.103:0/2052797378 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56116bcde340 con 0x56116bdd3410 2024-09-19T07:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26c27fc640 1 --2- 172.21.15.103:0/2052797378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdddcb0 0x56116bde00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26abfff640 1 -- 172.21.15.103:0/979630642 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f26b80f0370 con 0x7f26c4101350 2024-09-19T07:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.880+0000 7f26c2ffd640 1 --2- 172.21.15.103:0/2052797378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdd3410 0x56116bddd770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f26b400ba30 tx=0x7f26b400bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac6feb219daeda40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.881+0000 7f26c37fe640 1 --2- 172.21.15.103:0/979630642 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26ac0a3460 0x7f26ac0a5920 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f26bc013540 tx=0x7f26bc004a40 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.881+0000 7f26a97fa640 1 -- 172.21.15.103:0/2052797378 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4017420 con 0x56116bdd3410 2024-09-19T07:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.881+0000 7f26a97fa640 1 -- 172.21.15.103:0/2052797378 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b4017aa0 con 0x56116bdd3410 2024-09-19T07:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.881+0000 7f26a97fa640 1 -- 172.21.15.103:0/2052797378 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b400faa0 con 0x56116bdd3410 2024-09-19T07:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.881+0000 7f26cc285740 1 -- 172.21.15.103:0/2052797378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdd3410 msgr2=0x56116bddd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.881+0000 7f26cc285740 1 --2- 172.21.15.103:0/2052797378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdd3410 0x56116bddd770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f26b400ba30 tx=0x7f26b400bef0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.881+0000 7f26cc285740 1 -- 172.21.15.103:0/2052797378 shutdown_connections 2024-09-19T07:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.881+0000 7f26cc285740 1 --2- 172.21.15.103:0/2052797378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdddcb0 0x56116bde00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.881+0000 7f26cc285740 1 --2- 172.21.15.103:0/2052797378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdd3410 0x56116bddd770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.881+0000 7f26cc285740 1 --2- 172.21.15.103:0/2052797378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdd2af0 0x56116bdd2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.881+0000 7f26cc285740 1 -- 172.21.15.103:0/2052797378 >> 172.21.15.103:0/2052797378 conn(0x56116bdcf060 msgr2=0x56116bdcf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.881+0000 7f26cc285740 1 -- 172.21.15.103:0/2052797378 shutdown_connections 2024-09-19T07:06:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.881+0000 7f26cc285740 1 -- 172.21.15.103:0/2052797378 wait complete. 2024-09-19T07:06:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26cc285740 1 -- start start 2024-09-19T07:06:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26cc285740 1 Processor -- start 2024-09-19T07:06:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26cc285740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd2af0 0x56116bddaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26cc285740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdd3410 0x56116bddb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26cc285740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdddcb0 0x56116bddb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26cc285740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56116bde0880 con 0x56116bdd2af0 2024-09-19T07:06:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26cc285740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56116bcc0290 con 0x56116bdd3410 2024-09-19T07:06:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26cc285740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56116bde0a00 con 0x56116bdddcb0 2024-09-19T07:06:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26cc285740 10 client.0 _ll_register_callbacks cb 0x56116bcf1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26c2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd2af0 0x56116bddaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26c2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd2af0 0x56116bddaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50614/0 (socket says 172.21.15.103:50614) 2024-09-19T07:06:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26c2ffd640 1 -- 172.21.15.103:0/1966176251 learned_addr learned my addr 172.21.15.103:0/1966176251 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26c27fc640 1 --2- 172.21.15.103:0/1966176251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdd3410 0x56116bddb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26c2ffd640 1 -- 172.21.15.103:0/1966176251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdddcb0 msgr2=0x56116bddb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26c2ffd640 1 --2- 172.21.15.103:0/1966176251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdddcb0 0x56116bddb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26c2ffd640 1 -- 172.21.15.103:0/1966176251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdd3410 msgr2=0x56116bddb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26c2ffd640 1 --2- 172.21.15.103:0/1966176251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdd3410 0x56116bddb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26c2ffd640 1 -- 172.21.15.103:0/1966176251 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56116ba4a210 con 0x56116bdd2af0 2024-09-19T07:06:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.882+0000 7f26c27fc640 1 --2- 172.21.15.103:0/1966176251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdd3410 0x56116bddb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:06:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.883+0000 7f26c2ffd640 1 --2- 172.21.15.103:0/1966176251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd2af0 0x56116bddaae0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f26b4019040 tx=0x7f26b4017c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88505bb30629721c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.883+0000 7f26a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.883+0000 7f26a6ff5640 1 -- 172.21.15.103:0/1966176251 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4027020 con 0x56116bdd2af0 2024-09-19T07:06:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.883+0000 7f26a6ff5640 1 -- 172.21.15.103:0/1966176251 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f26b4017830 con 0x56116bdd2af0 2024-09-19T07:06:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.883+0000 7f26cc285740 1 -- 172.21.15.103:0/1966176251 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26b4017aa0 con 0x56116bdd2af0 2024-09-19T07:06:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.883+0000 7f26a6ff5640 1 -- 172.21.15.103:0/1966176251 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f26b4032050 con 0x56116bdd2af0 2024-09-19T07:06:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.883+0000 7f26cc285740 10 client.21507 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.883+0000 7f26cc285740 1 -- 172.21.15.103:0/1966176251 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56116bb16620 con 0x56116bdd2af0 2024-09-19T07:06:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.883+0000 7f26a6ff5640 1 -- 172.21.15.103:0/1966176251 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b4017aa0 con 0x56116bdd2af0 2024-09-19T07:06:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.883+0000 7f26a6ff5640 1 -- 172.21.15.103:0/1966176251 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f26b4023910 con 0x56116bdd2af0 2024-09-19T07:06:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.883+0000 7f26cc285740 10 client.21507 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.883+0000 7f26cc285740 10 client.21507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:06:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.883+0000 7f26cc285740 20 client.21507 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.883+0000 7f26cc285740 1 --2- 172.21.15.103:0/1966176251 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56116bdedb40 0x56116bdeff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.883+0000 7f26cc285740 4 client.21507 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:06:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.883+0000 7f26cc285740 1 -- 172.21.15.103:0/1966176251 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56116bd44a20 con 0x56116bdedb40 2024-09-19T07:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.884+0000 7f26c27fc640 1 --2- 172.21.15.103:0/1966176251 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56116bdedb40 0x56116bdeff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.884+0000 7f26a6ff5640 10 client.21507 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.884+0000 7f26c27fc640 1 --2- 172.21.15.103:0/1966176251 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56116bdedb40 0x56116bdeff30 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b352276edfeff837 server_cookie=9042a0a6cb7a44cb in_seq=0 out_seq=0 2024-09-19T07:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.885+0000 7f26a6ff5640 1 -- 172.21.15.103:0/1966176251 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56116bd44a20 con 0x56116bdedb40 2024-09-19T07:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.885+0000 7f26a6ff5640 10 client.21507 handle_command_reply: tid=0 2024-09-19T07:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.885+0000 7f26cc285740 1 client.21507 shutdown 2024-09-19T07:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.885+0000 7f26cc285740 1 -- 172.21.15.103:0/1966176251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd2af0 msgr2=0x56116bddaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.885+0000 7f26cc285740 1 --2- 172.21.15.103:0/1966176251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd2af0 0x56116bddaae0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f26b4019040 tx=0x7f26b4017c50 comp rx=0 tx=0).stop 2024-09-19T07:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.885+0000 7f26cc285740 1 -- 172.21.15.103:0/1966176251 shutdown_connections 2024-09-19T07:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.885+0000 7f26cc285740 1 --2- 172.21.15.103:0/1966176251 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56116bdedb40 0x56116bdeff30 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.885+0000 7f26cc285740 1 --2- 172.21.15.103:0/1966176251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdddcb0 0x56116bddb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.885+0000 7f26a6ff5640 0 client.21507 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.885+0000 7f26cc285740 1 --2- 172.21.15.103:0/1966176251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdd3410 0x56116bddb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.885+0000 7f26cc285740 1 --2- 172.21.15.103:0/1966176251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd2af0 0x56116bddaae0 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.885+0000 7f26cc285740 1 -- 172.21.15.103:0/1966176251 >> 172.21.15.103:0/1966176251 conn(0x56116bdcf060 msgr2=0x56116bdd4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.885+0000 7f26cc285740 1 -- 172.21.15.103:0/1966176251 shutdown_connections 2024-09-19T07:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.885+0000 7f26cc285740 1 -- 172.21.15.103:0/1966176251 wait complete. 2024-09-19T07:06:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.885+0000 7f26cc285740 20 client.21507 trim_cache size 0 max 16384 2024-09-19T07:06:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.899+0000 7f26cc285740 1 Processor -- start 2024-09-19T07:06:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26cc285740 1 -- start start 2024-09-19T07:06:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26cc285740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdd2af0 0x56116bdd9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26cc285740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd3410 0x56116bddb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26cc285740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdddcb0 0x56116bdf5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26cc285740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26c41ad6d0 con 0x56116bdd3410 2024-09-19T07:06:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26cc285740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26c41af210 con 0x56116bdddcb0 2024-09-19T07:06:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26cc285740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26c41adda0 con 0x56116bdd2af0 2024-09-19T07:06:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26c27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd3410 0x56116bddb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26c27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd3410 0x56116bddb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50622/0 (socket says 172.21.15.103:50622) 2024-09-19T07:06:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26c27fc640 1 -- 172.21.15.103:0/111549965 learned_addr learned my addr 172.21.15.103:0/111549965 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26c27fc640 1 -- 172.21.15.103:0/111549965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdd2af0 msgr2=0x56116bdd9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26c27fc640 1 --2- 172.21.15.103:0/111549965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdd2af0 0x56116bdd9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26c27fc640 1 -- 172.21.15.103:0/111549965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdddcb0 msgr2=0x56116bdf5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26c37fe640 1 --2- 172.21.15.103:0/111549965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdddcb0 0x56116bdf5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26c27fc640 1 --2- 172.21.15.103:0/111549965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdddcb0 0x56116bdf5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26c27fc640 1 -- 172.21.15.103:0/111549965 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56116bdcee90 con 0x56116bdd3410 2024-09-19T07:06:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26c37fe640 1 --2- 172.21.15.103:0/111549965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdddcb0 0x56116bdf5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26c27fc640 1 --2- 172.21.15.103:0/111549965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd3410 0x56116bddb390 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f26b80bc5c0 tx=0x7f26b80f3ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bc52d1e6d2211f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26a8ff9640 1 -- 172.21.15.103:0/111549965 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8049040 con 0x56116bdd3410 2024-09-19T07:06:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26a8ff9640 1 -- 172.21.15.103:0/111549965 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b80bcd30 con 0x56116bdd3410 2024-09-19T07:06:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.900+0000 7f26a8ff9640 1 -- 172.21.15.103:0/111549965 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80baa10 con 0x56116bdd3410 2024-09-19T07:06:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.901+0000 7f26cc285740 1 -- 172.21.15.103:0/111549965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd3410 msgr2=0x56116bddb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.901+0000 7f26cc285740 1 --2- 172.21.15.103:0/111549965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd3410 0x56116bddb390 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f26b80bc5c0 tx=0x7f26b80f3ec0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.901+0000 7f26cc285740 1 -- 172.21.15.103:0/111549965 shutdown_connections 2024-09-19T07:06:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.901+0000 7f26cc285740 1 --2- 172.21.15.103:0/111549965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdddcb0 0x56116bdf5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.901+0000 7f26cc285740 1 --2- 172.21.15.103:0/111549965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd3410 0x56116bddb390 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.901+0000 7f26cc285740 1 --2- 172.21.15.103:0/111549965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdd2af0 0x56116bdd9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.901+0000 7f26cc285740 1 -- 172.21.15.103:0/111549965 >> 172.21.15.103:0/111549965 conn(0x56116bdcf060 msgr2=0x56116bddcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.901+0000 7f26cc285740 1 -- 172.21.15.103:0/111549965 shutdown_connections 2024-09-19T07:06:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.901+0000 7f26cc285740 1 -- 172.21.15.103:0/111549965 wait complete. 2024-09-19T07:06:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.901+0000 7f26cc285740 1 -- start start 2024-09-19T07:06:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.901+0000 7f26cc285740 1 Processor -- start 2024-09-19T07:06:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26cc285740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd2af0 0x56116bde64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26cc285740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdd3410 0x56116bddec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26cc285740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdddcb0 0x56116bddf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26cc285740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26c41aeb40 con 0x56116bdd2af0 2024-09-19T07:06:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26cc285740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26c41ae470 con 0x56116bdd3410 2024-09-19T07:06:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26cc285740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26c41ab6d0 con 0x56116bdddcb0 2024-09-19T07:06:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26cc285740 10 client.0 _ll_register_callbacks cb 0x7f26b80f3d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26c2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd2af0 0x56116bde64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26c2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd2af0 0x56116bde64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50630/0 (socket says 172.21.15.103:50630) 2024-09-19T07:06:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26c2ffd640 1 -- 172.21.15.103:0/953959023 learned_addr learned my addr 172.21.15.103:0/953959023 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26c37fe640 1 --2- 172.21.15.103:0/953959023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdddcb0 0x56116bddf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26c2ffd640 1 -- 172.21.15.103:0/953959023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdddcb0 msgr2=0x56116bddf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26c2ffd640 1 --2- 172.21.15.103:0/953959023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdddcb0 0x56116bddf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26c2ffd640 1 -- 172.21.15.103:0/953959023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdd3410 msgr2=0x56116bddec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26c2ffd640 1 --2- 172.21.15.103:0/953959023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdd3410 0x56116bddec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26c2ffd640 1 -- 172.21.15.103:0/953959023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56116bdca2d0 con 0x56116bdd2af0 2024-09-19T07:06:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26c37fe640 1 --2- 172.21.15.103:0/953959023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdddcb0 0x56116bddf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26c2ffd640 1 --2- 172.21.15.103:0/953959023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd2af0 0x56116bde64e0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f26b4005ba0 tx=0x7f26b40216d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bed1942c6aaf2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26a6ff5640 1 -- 172.21.15.103:0/953959023 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4027070 con 0x56116bdd2af0 2024-09-19T07:06:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26a6ff5640 1 -- 172.21.15.103:0/953959023 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f26b400f9f0 con 0x56116bdd2af0 2024-09-19T07:06:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26cc285740 1 -- 172.21.15.103:0/953959023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26b80bcd30 con 0x56116bdd2af0 2024-09-19T07:06:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26a6ff5640 1 -- 172.21.15.103:0/953959023 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f26b400fbf0 con 0x56116bdd2af0 2024-09-19T07:06:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26cc285740 10 client.21513 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.902+0000 7f26cc285740 1 -- 172.21.15.103:0/953959023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56116be584d0 con 0x56116bdd2af0 2024-09-19T07:06:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.903+0000 7f26a6ff5640 1 -- 172.21.15.103:0/953959023 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b80bcd30 con 0x56116bdd2af0 2024-09-19T07:06:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.903+0000 7f26a6ff5640 1 -- 172.21.15.103:0/953959023 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f26b4023470 con 0x56116bdd2af0 2024-09-19T07:06:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.903+0000 7f26cc285740 10 client.21513 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.903+0000 7f26cc285740 10 client.21513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T07:06:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.903+0000 7f26cc285740 20 client.21513 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.903+0000 7f26cc285740 1 --2- 172.21.15.103:0/953959023 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56116be58a80 0x56116be5aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.903+0000 7f26cc285740 4 client.21513 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T07:06:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.903+0000 7f26cc285740 1 -- 172.21.15.103:0/953959023 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56116bd3ed70 con 0x56116be58a80 2024-09-19T07:06:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.903+0000 7f26c27fc640 1 --2- 172.21.15.103:0/953959023 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56116be58a80 0x56116be5aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.903+0000 7f26a6ff5640 10 client.21513 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.903+0000 7f26c27fc640 1 --2- 172.21.15.103:0/953959023 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56116be58a80 0x56116be5aea0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abdacf42a551ae31 server_cookie=c34f39107f1d4919 in_seq=0 out_seq=0 2024-09-19T07:06:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.918+0000 7f26a6ff5640 1 -- 172.21.15.103:0/953959023 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56116bd3ed70 con 0x56116be58a80 2024-09-19T07:06:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.918+0000 7f26a6ff5640 10 client.21513 handle_command_reply: tid=0 2024-09-19T07:06:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.918+0000 7f26cc285740 1 client.21513 shutdown 2024-09-19T07:06:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.918+0000 7f26cc285740 1 -- 172.21.15.103:0/953959023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd2af0 msgr2=0x56116bde64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.918+0000 7f26cc285740 1 --2- 172.21.15.103:0/953959023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd2af0 0x56116bde64e0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f26b4005ba0 tx=0x7f26b40216d0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.918+0000 7f26cc285740 1 -- 172.21.15.103:0/953959023 shutdown_connections 2024-09-19T07:06:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.918+0000 7f26cc285740 1 --2- 172.21.15.103:0/953959023 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x56116be58a80 0x56116be5aea0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.918+0000 7f26cc285740 1 --2- 172.21.15.103:0/953959023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56116bdddcb0 0x56116bddf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.918+0000 7f26a6ff5640 0 client.21513 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.918+0000 7f26cc285740 1 --2- 172.21.15.103:0/953959023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56116bdd3410 0x56116bddec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.918+0000 7f26cc285740 1 --2- 172.21.15.103:0/953959023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56116bdd2af0 0x56116bde64e0 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.918+0000 7f26cc285740 1 -- 172.21.15.103:0/953959023 >> 172.21.15.103:0/953959023 conn(0x56116bdcf060 msgr2=0x56116bdcd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.918+0000 7f26cc285740 1 -- 172.21.15.103:0/953959023 shutdown_connections 2024-09-19T07:06:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.918+0000 7f26cc285740 1 -- 172.21.15.103:0/953959023 wait complete. 2024-09-19T07:06:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.918+0000 7f26cc285740 20 client.21513 trim_cache size 0 max 16384 2024-09-19T07:06:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-19T07:06:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-19T07:06:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-19T07:06:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-19T07:06:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.919+0000 7f26c94e4640 1 -- 172.21.15.103:0/979630642 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26ac0a3460 msgr2=0x7f26ac0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.919+0000 7f26c94e4640 1 --2- 172.21.15.103:0/979630642 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26ac0a3460 0x7f26ac0a5920 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f26bc013540 tx=0x7f26bc004a40 comp rx=0 tx=0).stop 2024-09-19T07:06:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.919+0000 7f26c94e4640 1 -- 172.21.15.103:0/979630642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26c4101350 msgr2=0x7f26c41996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.919+0000 7f26c94e4640 1 --2- 172.21.15.103:0/979630642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26c4101350 0x7f26c41996c0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009a50 tx=0x7f26b8004270 comp rx=0 tx=0).stop 2024-09-19T07:06:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.919+0000 7f26c94e4640 1 -- 172.21.15.103:0/979630642 shutdown_connections 2024-09-19T07:06:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.919+0000 7f26c94e4640 1 --2- 172.21.15.103:0/979630642 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f26ac0a3460 0x7f26ac0a5920 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.919+0000 7f26c94e4640 1 --2- 172.21.15.103:0/979630642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26c41069c0 0x7f26c41a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.919+0000 7f26c94e4640 1 --2- 172.21.15.103:0/979630642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26c4101350 0x7f26c41996c0 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.919+0000 7f26c94e4640 1 --2- 172.21.15.103:0/979630642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26c41009b0 0x7f26c4199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.919+0000 7f26c94e4640 1 -- 172.21.15.103:0/979630642 >> 172.21.15.103:0/979630642 conn(0x7f26c4077990 msgr2=0x7f26c4103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.919+0000 7f26c94e4640 1 -- 172.21.15.103:0/979630642 shutdown_connections 2024-09-19T07:06:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-19T07:06:16.919+0000 7f26c94e4640 1 -- 172.21.15.103:0/979630642 wait complete. 2024-09-19T07:06:17.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.024+0000 7f900775b640 1 -- 172.21.15.103:0/3485573950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f900010e420 msgr2=0x7f9000110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:17.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.024+0000 7f900775b640 1 --2- 172.21.15.103:0/3485573950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f900010e420 0x7f9000110810 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8009ca0 tx=0x7f8ff802a950 comp rx=0 tx=0).stop 2024-09-19T07:06:17.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.024+0000 7f900775b640 1 -- 172.21.15.103:0/3485573950 shutdown_connections 2024-09-19T07:06:17.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.024+0000 7f900775b640 1 --2- 172.21.15.103:0/3485573950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f900010e420 0x7f9000110810 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.024+0000 7f900775b640 1 --2- 172.21.15.103:0/3485573950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9000074ee0 0x7f900010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.024+0000 7f900775b640 1 --2- 172.21.15.103:0/3485573950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90000745c0 0x7f90000749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.024+0000 7f900775b640 1 -- 172.21.15.103:0/3485573950 >> 172.21.15.103:0/3485573950 conn(0x7f90000fd8c0 msgr2=0x7f90000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:17.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.024+0000 7f900775b640 1 -- 172.21.15.103:0/3485573950 shutdown_connections 2024-09-19T07:06:17.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.024+0000 7f900775b640 1 -- 172.21.15.103:0/3485573950 wait complete. 2024-09-19T07:06:17.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.025+0000 7f900775b640 1 Processor -- start 2024-09-19T07:06:17.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.025+0000 7f900775b640 1 -- start start 2024-09-19T07:06:17.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.025+0000 7f900775b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90000745c0 0x7f900019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.025+0000 7f900775b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9000074ee0 0x7f900019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.025+0000 7f900775b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f900010e420 0x7f900019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.025+0000 7f900775b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9000114150 con 0x7f90000745c0 2024-09-19T07:06:17.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.025+0000 7f900775b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9000113fd0 con 0x7f9000074ee0 2024-09-19T07:06:17.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.025+0000 7f900775b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f90001142d0 con 0x7f900010e420 2024-09-19T07:06:17.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.025+0000 7f90054f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90000745c0 0x7f900019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.025+0000 7f90054f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90000745c0 0x7f900019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50644/0 (socket says 172.21.15.103:50644) 2024-09-19T07:06:17.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.025+0000 7f90054f7640 1 -- 172.21.15.103:0/504932215 learned_addr learned my addr 172.21.15.103:0/504932215 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:17.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.025+0000 7f9004cf6640 1 --2- 172.21.15.103:0/504932215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9000074ee0 0x7f900019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.025+0000 7f9005cf8640 1 --2- 172.21.15.103:0/504932215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f900010e420 0x7f900019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.026+0000 7f90054f7640 1 -- 172.21.15.103:0/504932215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f900010e420 msgr2=0x7f900019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:17.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.026+0000 7f90054f7640 1 --2- 172.21.15.103:0/504932215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f900010e420 0x7f900019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.026+0000 7f90054f7640 1 -- 172.21.15.103:0/504932215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9000074ee0 msgr2=0x7f900019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:17.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.026+0000 7f90054f7640 1 --2- 172.21.15.103:0/504932215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9000074ee0 0x7f900019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.026+0000 7f90054f7640 1 -- 172.21.15.103:0/504932215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900019f5e0 con 0x7f90000745c0 2024-09-19T07:06:17.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.026+0000 7f90054f7640 1 --2- 172.21.15.103:0/504932215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90000745c0 0x7f900019a630 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000d950 tx=0x7f8ff000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:17.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.026+0000 7f8ff67fc640 1 -- 172.21.15.103:0/504932215 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0017070 con 0x7f90000745c0 2024-09-19T07:06:17.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.026+0000 7f900775b640 1 -- 172.21.15.103:0/504932215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900019f8d0 con 0x7f90000745c0 2024-09-19T07:06:17.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.026+0000 7f8ff67fc640 1 -- 172.21.15.103:0/504932215 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ff0009c30 con 0x7f90000745c0 2024-09-19T07:06:17.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.026+0000 7f8ff67fc640 1 -- 172.21.15.103:0/504932215 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff000f6b0 con 0x7f90000745c0 2024-09-19T07:06:17.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.026+0000 7f900775b640 1 -- 172.21.15.103:0/504932215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90001a71b0 con 0x7f90000745c0 2024-09-19T07:06:17.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.028+0000 7f8ff67fc640 1 -- 172.21.15.103:0/504932215 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8ff00109d0 con 0x7f90000745c0 2024-09-19T07:06:17.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f9009b34740 1 Processor -- start 2024-09-19T07:06:17.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f9009b34740 1 -- start start 2024-09-19T07:06:17.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f9009b34740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874e9a00 0x55eb874e9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f9009b34740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874ea320 0x55eb874f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f9009b34740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874f4be0 0x55eb874f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f9009b34740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eb8744e200 con 0x55eb874ea320 2024-09-19T07:06:17.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f9009b34740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55eb873f4dc0 con 0x55eb874e9a00 2024-09-19T07:06:17.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f9009b34740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eb8744a1c0 con 0x55eb874f4be0 2024-09-19T07:06:17.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f8ff67fc640 1 --2- 172.21.15.103:0/504932215 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8fec0a33b0 0x7f8fec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f90054f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874ea320 0x55eb874f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f90054f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874ea320 0x55eb874f46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50658/0 (socket says 172.21.15.103:50658) 2024-09-19T07:06:17.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f90054f7640 1 -- 172.21.15.103:0/4094411731 learned_addr learned my addr 172.21.15.103:0/4094411731 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:17.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f9005cf8640 1 --2- 172.21.15.103:0/504932215 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8fec0a33b0 0x7f8fec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f9004cf6640 1 --2- 172.21.15.103:0/4094411731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874f4be0 0x55eb874f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f9005cf8640 1 --2- 172.21.15.103:0/4094411731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874e9a00 0x55eb874e9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f8ff67fc640 1 -- 172.21.15.103:0/504932215 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f8ff00cf900 con 0x7f90000745c0 2024-09-19T07:06:17.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f90054f7640 1 -- 172.21.15.103:0/4094411731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874f4be0 msgr2=0x55eb874f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:17.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f90054f7640 1 --2- 172.21.15.103:0/4094411731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874f4be0 0x55eb874f6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f90054f7640 1 -- 172.21.15.103:0/4094411731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874e9a00 msgr2=0x55eb874e9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:17.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f90054f7640 1 --2- 172.21.15.103:0/4094411731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874e9a00 0x55eb874e9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.029+0000 7f90054f7640 1 -- 172.21.15.103:0/4094411731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb871bf620 con 0x55eb874ea320 2024-09-19T07:06:17.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.030+0000 7f90054f7640 1 --2- 172.21.15.103:0/4094411731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874ea320 0x55eb874f46a0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0063a90 tx=0x7f8ff000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cb073dd16640ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:17.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.030+0000 7f9005cf8640 1 --2- 172.21.15.103:0/504932215 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8fec0a33b0 0x7f8fec0a5870 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8005e30 tx=0x7f8ff802fe10 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:17.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.030+0000 7f8feb7fe640 1 -- 172.21.15.103:0/4094411731 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0020040 con 0x55eb874ea320 2024-09-19T07:06:17.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.030+0000 7f8feb7fe640 1 -- 172.21.15.103:0/4094411731 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ff009bc70 con 0x55eb874ea320 2024-09-19T07:06:17.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.030+0000 7f8feb7fe640 1 -- 172.21.15.103:0/4094411731 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff00cd440 con 0x55eb874ea320 2024-09-19T07:06:17.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.031+0000 7f9009b34740 1 -- 172.21.15.103:0/4094411731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874ea320 msgr2=0x55eb874f46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:17.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.031+0000 7f9009b34740 1 --2- 172.21.15.103:0/4094411731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874ea320 0x55eb874f46a0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0063a90 tx=0x7f8ff000bf60 comp rx=0 tx=0).stop 2024-09-19T07:06:17.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.031+0000 7f9009b34740 1 -- 172.21.15.103:0/4094411731 shutdown_connections 2024-09-19T07:06:17.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.031+0000 7f9009b34740 1 --2- 172.21.15.103:0/4094411731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874f4be0 0x55eb874f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.031+0000 7f9009b34740 1 --2- 172.21.15.103:0/4094411731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874ea320 0x55eb874f46a0 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.031+0000 7f9009b34740 1 --2- 172.21.15.103:0/4094411731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874e9a00 0x55eb874e9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.031+0000 7f9009b34740 1 -- 172.21.15.103:0/4094411731 >> 172.21.15.103:0/4094411731 conn(0x55eb874e5f90 msgr2=0x55eb874e6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:17.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.031+0000 7f9009b34740 1 -- 172.21.15.103:0/4094411731 shutdown_connections 2024-09-19T07:06:17.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.031+0000 7f9009b34740 1 -- 172.21.15.103:0/4094411731 wait complete. 2024-09-19T07:06:17.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.031+0000 7f9009b34740 1 -- start start 2024-09-19T07:06:17.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.031+0000 7f9009b34740 1 Processor -- start 2024-09-19T07:06:17.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9009b34740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874e9a00 0x55eb874f4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9009b34740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874ea320 0x55eb874f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9009b34740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874f4be0 0x55eb874f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9009b34740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eb874f77b0 con 0x55eb874ea320 2024-09-19T07:06:17.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9009b34740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55eb873e8c50 con 0x55eb874f4be0 2024-09-19T07:06:17.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9009b34740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eb874f7930 con 0x55eb874e9a00 2024-09-19T07:06:17.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9009b34740 10 client.0 _ll_register_callbacks cb 0x55eb873faba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:17.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9004cf6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874ea320 0x55eb874f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9004cf6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874ea320 0x55eb874f17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50666/0 (socket says 172.21.15.103:50666) 2024-09-19T07:06:17.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9004cf6640 1 -- 172.21.15.103:0/1433975302 learned_addr learned my addr 172.21.15.103:0/1433975302 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:17.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9005cf8640 1 --2- 172.21.15.103:0/1433975302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874f4be0 0x55eb874f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f90054f7640 1 --2- 172.21.15.103:0/1433975302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874e9a00 0x55eb874f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9004cf6640 1 -- 172.21.15.103:0/1433975302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874e9a00 msgr2=0x55eb874f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9004cf6640 1 --2- 172.21.15.103:0/1433975302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874e9a00 0x55eb874f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9004cf6640 1 -- 172.21.15.103:0/1433975302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874f4be0 msgr2=0x55eb874f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9004cf6640 1 --2- 172.21.15.103:0/1433975302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874f4be0 0x55eb874f1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9004cf6640 1 -- 172.21.15.103:0/1433975302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb874f2540 con 0x55eb874ea320 2024-09-19T07:06:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f90054f7640 1 --2- 172.21.15.103:0/1433975302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874e9a00 0x55eb874f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9004cf6640 1 --2- 172.21.15.103:0/1433975302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874ea320 0x55eb874f17d0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc010770 tx=0x7f8ffc010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4820454e5ccc4320 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f8fe8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f8fe8ff9640 1 -- 172.21.15.103:0/1433975302 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00fc50 con 0x55eb874ea320 2024-09-19T07:06:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f8fe8ff9640 1 -- 172.21.15.103:0/1433975302 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f8ffc00fdf0 con 0x55eb874ea320 2024-09-19T07:06:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.032+0000 7f9009b34740 1 -- 172.21.15.103:0/1433975302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ff009bc70 con 0x55eb874ea320 2024-09-19T07:06:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.033+0000 7f8fe8ff9640 1 -- 172.21.15.103:0/1433975302 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8ffc004020 con 0x55eb874ea320 2024-09-19T07:06:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.033+0000 7f9009b34740 10 client.21525 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.033+0000 7f9009b34740 1 -- 172.21.15.103:0/1433975302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eb87503390 con 0x55eb874ea320 2024-09-19T07:06:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.033+0000 7f8fe8ff9640 1 -- 172.21.15.103:0/1433975302 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ff009bc70 con 0x55eb874ea320 2024-09-19T07:06:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.033+0000 7f8fe8ff9640 1 -- 172.21.15.103:0/1433975302 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8ffc019410 con 0x55eb874ea320 2024-09-19T07:06:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.033+0000 7f9009b34740 10 client.21525 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.033+0000 7f9009b34740 10 client.21525 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:06:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.033+0000 7f9009b34740 20 client.21525 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.033+0000 7f9009b34740 1 --2- 172.21.15.103:0/1433975302 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eb875048f0 0x55eb87506d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.033+0000 7f9009b34740 4 client.21525 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:06:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.033+0000 7f9009b34740 1 -- 172.21.15.103:0/1433975302 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb8745b770 con 0x55eb875048f0 2024-09-19T07:06:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.033+0000 7f90054f7640 1 --2- 172.21.15.103:0/1433975302 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eb875048f0 0x55eb87506d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.034+0000 7f90054f7640 1 --2- 172.21.15.103:0/1433975302 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eb875048f0 0x55eb87506d50 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eda27c62d6a26382 server_cookie=5b34ce11e2b82b5b in_seq=0 out_seq=0 2024-09-19T07:06:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.034+0000 7f8fe8ff9640 10 client.21525 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.034+0000 7f8fe8ff9640 1 -- 172.21.15.103:0/1433975302 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb8745b770 con 0x55eb875048f0 2024-09-19T07:06:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.034+0000 7f8fe8ff9640 10 client.21525 handle_command_reply: tid=0 2024-09-19T07:06:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.034+0000 7f9009b34740 1 client.21525 shutdown 2024-09-19T07:06:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.035+0000 7f9009b34740 1 -- 172.21.15.103:0/1433975302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874ea320 msgr2=0x55eb874f17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.035+0000 7f9009b34740 1 --2- 172.21.15.103:0/1433975302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874ea320 0x55eb874f17d0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc010770 tx=0x7f8ffc010b70 comp rx=0 tx=0).stop 2024-09-19T07:06:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.035+0000 7f9009b34740 1 -- 172.21.15.103:0/1433975302 shutdown_connections 2024-09-19T07:06:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.035+0000 7f9009b34740 1 --2- 172.21.15.103:0/1433975302 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eb875048f0 0x55eb87506d50 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.035+0000 7f9009b34740 1 --2- 172.21.15.103:0/1433975302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874f4be0 0x55eb874f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.035+0000 7f9009b34740 1 --2- 172.21.15.103:0/1433975302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874ea320 0x55eb874f17d0 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.035+0000 7f9009b34740 1 --2- 172.21.15.103:0/1433975302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874e9a00 0x55eb874f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.035+0000 7f9009b34740 1 -- 172.21.15.103:0/1433975302 >> 172.21.15.103:0/1433975302 conn(0x55eb874e5f90 msgr2=0x55eb874e51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.035+0000 7f8fe8ff9640 0 client.21525 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.035+0000 7f9009b34740 1 -- 172.21.15.103:0/1433975302 shutdown_connections 2024-09-19T07:06:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.035+0000 7f9009b34740 1 -- 172.21.15.103:0/1433975302 wait complete. 2024-09-19T07:06:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.035+0000 7f9009b34740 20 client.21525 trim_cache size 0 max 16384 2024-09-19T07:06:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.051+0000 7f9009b34740 1 Processor -- start 2024-09-19T07:06:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.051+0000 7f9009b34740 1 -- start start 2024-09-19T07:06:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f9009b34740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874e9a00 0x55eb87506b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f9009b34740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874ea320 0x55eb87500a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f9009b34740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874f4be0 0x55eb8750c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f9009b34740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90001afc30 con 0x55eb874f4be0 2024-09-19T07:06:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f9009b34740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90001b1770 con 0x55eb874ea320 2024-09-19T07:06:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f9009b34740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f90001b0300 con 0x55eb874e9a00 2024-09-19T07:06:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f9005cf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874f4be0 0x55eb8750c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f9005cf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874f4be0 0x55eb8750c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50674/0 (socket says 172.21.15.103:50674) 2024-09-19T07:06:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f9005cf8640 1 -- 172.21.15.103:0/3080458773 learned_addr learned my addr 172.21.15.103:0/3080458773 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f9004cf6640 1 --2- 172.21.15.103:0/3080458773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874ea320 0x55eb87500a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f9005cf8640 1 -- 172.21.15.103:0/3080458773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874e9a00 msgr2=0x55eb87506b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f9005cf8640 1 --2- 172.21.15.103:0/3080458773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874e9a00 0x55eb87506b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f9005cf8640 1 -- 172.21.15.103:0/3080458773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874ea320 msgr2=0x55eb87500a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f9005cf8640 1 --2- 172.21.15.103:0/3080458773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874ea320 0x55eb87500a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f9005cf8640 1 -- 172.21.15.103:0/3080458773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb874ea150 con 0x55eb874f4be0 2024-09-19T07:06:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f9005cf8640 1 --2- 172.21.15.103:0/3080458773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874f4be0 0x55eb8750c620 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f900006b460 tx=0x7f8ff8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea1b958dc9bf60d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f8feaffd640 1 -- 172.21.15.103:0/3080458773 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8034040 con 0x55eb874f4be0 2024-09-19T07:06:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f8feaffd640 1 -- 172.21.15.103:0/3080458773 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ff804f070 con 0x55eb874f4be0 2024-09-19T07:06:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.052+0000 7f8feaffd640 1 -- 172.21.15.103:0/3080458773 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff804b6b0 con 0x55eb874f4be0 2024-09-19T07:06:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.053+0000 7f9009b34740 1 -- 172.21.15.103:0/3080458773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874f4be0 msgr2=0x55eb8750c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.053+0000 7f9009b34740 1 --2- 172.21.15.103:0/3080458773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874f4be0 0x55eb8750c620 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f900006b460 tx=0x7f8ff8005c50 comp rx=0 tx=0).stop 2024-09-19T07:06:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.053+0000 7f9009b34740 1 -- 172.21.15.103:0/3080458773 shutdown_connections 2024-09-19T07:06:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.053+0000 7f9009b34740 1 --2- 172.21.15.103:0/3080458773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874f4be0 0x55eb8750c620 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.053+0000 7f9009b34740 1 --2- 172.21.15.103:0/3080458773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874ea320 0x55eb87500a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.053+0000 7f9009b34740 1 --2- 172.21.15.103:0/3080458773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874e9a00 0x55eb87506b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.053+0000 7f9009b34740 1 -- 172.21.15.103:0/3080458773 >> 172.21.15.103:0/3080458773 conn(0x55eb874e5f90 msgr2=0x55eb874f08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.053+0000 7f9009b34740 1 -- 172.21.15.103:0/3080458773 shutdown_connections 2024-09-19T07:06:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.053+0000 7f9009b34740 1 -- 172.21.15.103:0/3080458773 wait complete. 2024-09-19T07:06:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.053+0000 7f9009b34740 1 -- start start 2024-09-19T07:06:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.053+0000 7f9009b34740 1 Processor -- start 2024-09-19T07:06:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f9009b34740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874e9a00 0x55eb874f9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f9009b34740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874ea320 0x55eb874fa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f9009b34740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874f4be0 0x55eb874fa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f9009b34740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90001b10a0 con 0x55eb874e9a00 2024-09-19T07:06:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f9009b34740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90001b09d0 con 0x55eb874f4be0 2024-09-19T07:06:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f9009b34740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f90001adc30 con 0x55eb874ea320 2024-09-19T07:06:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f9009b34740 10 client.0 _ll_register_callbacks cb 0x7f8ffc00abf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f90054f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874e9a00 0x55eb874f9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f90054f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874e9a00 0x55eb874f9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50676/0 (socket says 172.21.15.103:50676) 2024-09-19T07:06:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f90054f7640 1 -- 172.21.15.103:0/1366527920 learned_addr learned my addr 172.21.15.103:0/1366527920 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f9005cf8640 1 --2- 172.21.15.103:0/1366527920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874f4be0 0x55eb874fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f9004cf6640 1 --2- 172.21.15.103:0/1366527920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874ea320 0x55eb874fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f90054f7640 1 -- 172.21.15.103:0/1366527920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874ea320 msgr2=0x55eb874fa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f90054f7640 1 --2- 172.21.15.103:0/1366527920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874ea320 0x55eb874fa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f90054f7640 1 -- 172.21.15.103:0/1366527920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874f4be0 msgr2=0x55eb874fa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f90054f7640 1 --2- 172.21.15.103:0/1366527920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874f4be0 0x55eb874fa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f90054f7640 1 -- 172.21.15.103:0/1366527920 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb8756d8e0 con 0x55eb874e9a00 2024-09-19T07:06:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.054+0000 7f9004cf6640 1 --2- 172.21.15.103:0/1366527920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874ea320 0x55eb874fa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:06:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.055+0000 7f90054f7640 1 --2- 172.21.15.103:0/1366527920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874e9a00 0x55eb874f9bc0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0098640 tx=0x7f8ff0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915d73a05730ee8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.055+0000 7f8fe8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.055+0000 7f8fe8ff9640 1 -- 172.21.15.103:0/1366527920 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0014020 con 0x55eb874e9a00 2024-09-19T07:06:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.055+0000 7f8fe8ff9640 1 -- 172.21.15.103:0/1366527920 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f8ff009b0c0 con 0x55eb874e9a00 2024-09-19T07:06:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.055+0000 7f9009b34740 1 -- 172.21.15.103:0/1366527920 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ff804f070 con 0x55eb874e9a00 2024-09-19T07:06:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.055+0000 7f8fe8ff9640 1 -- 172.21.15.103:0/1366527920 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8ff0020040 con 0x55eb874e9a00 2024-09-19T07:06:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.055+0000 7f9009b34740 10 client.21531 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.055+0000 7f9009b34740 1 -- 172.21.15.103:0/1366527920 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eb8756eb80 con 0x55eb874e9a00 2024-09-19T07:06:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.055+0000 7f8fe8ff9640 1 -- 172.21.15.103:0/1366527920 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ff804f070 con 0x55eb874e9a00 2024-09-19T07:06:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.055+0000 7f8fe8ff9640 1 -- 172.21.15.103:0/1366527920 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8ff00979c0 con 0x55eb874e9a00 2024-09-19T07:06:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.055+0000 7f9009b34740 10 client.21531 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.055+0000 7f9009b34740 10 client.21531 resolve_mds: validated gid 24482 aka daemon mds.b 2024-09-19T07:06:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.055+0000 7f9009b34740 20 client.21531 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.055+0000 7f9009b34740 1 --2- 172.21.15.103:0/1366527920 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eb8756ee70 0x55eb87571260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.055+0000 7f9009b34740 4 client.21531 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-19T07:06:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.055+0000 7f9009b34740 1 -- 172.21.15.103:0/1366527920 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55eb87455c80 con 0x55eb8756ee70 2024-09-19T07:06:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.056+0000 7f9004cf6640 1 --2- 172.21.15.103:0/1366527920 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eb8756ee70 0x55eb87571260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.056+0000 7f8fe8ff9640 10 client.21531 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.056+0000 7f9004cf6640 1 --2- 172.21.15.103:0/1366527920 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eb8756ee70 0x55eb87571260 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d49c344e63eeb4c4 server_cookie=9f6e9a26aedcf35b in_seq=0 out_seq=0 2024-09-19T07:06:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.056+0000 7f8fe8ff9640 1 -- 172.21.15.103:0/1366527920 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55eb87455c80 con 0x55eb8756ee70 2024-09-19T07:06:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.056+0000 7f8fe8ff9640 10 client.21531 handle_command_reply: tid=0 2024-09-19T07:06:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.056+0000 7f9009b34740 1 client.21531 shutdown 2024-09-19T07:06:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.056+0000 7f9009b34740 1 -- 172.21.15.103:0/1366527920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874e9a00 msgr2=0x55eb874f9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.056+0000 7f9009b34740 1 --2- 172.21.15.103:0/1366527920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874e9a00 0x55eb874f9bc0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0098640 tx=0x7f8ff0005880 comp rx=0 tx=0).stop 2024-09-19T07:06:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.056+0000 7f9009b34740 1 -- 172.21.15.103:0/1366527920 shutdown_connections 2024-09-19T07:06:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.056+0000 7f9009b34740 1 --2- 172.21.15.103:0/1366527920 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x55eb8756ee70 0x55eb87571260 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.056+0000 7f9009b34740 1 --2- 172.21.15.103:0/1366527920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb874f4be0 0x55eb874fa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.056+0000 7f9009b34740 1 --2- 172.21.15.103:0/1366527920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb874ea320 0x55eb874fa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.056+0000 7f9009b34740 1 --2- 172.21.15.103:0/1366527920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb874e9a00 0x55eb874f9bc0 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.056+0000 7f8fe8ff9640 0 client.21531 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:17.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.056+0000 7f9009b34740 1 -- 172.21.15.103:0/1366527920 >> 172.21.15.103:0/1366527920 conn(0x55eb874e5f90 msgr2=0x55eb874e49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:17.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.057+0000 7f9009b34740 1 -- 172.21.15.103:0/1366527920 shutdown_connections 2024-09-19T07:06:17.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.057+0000 7f9009b34740 1 -- 172.21.15.103:0/1366527920 wait complete. 2024-09-19T07:06:17.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.057+0000 7f9009b34740 20 client.21531 trim_cache size 0 max 16384 2024-09-19T07:06:17.063 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:06:17.063 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-19T07:06:17.063 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24482, 2024-09-19T07:06:17.064 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 56, 2024-09-19T07:06:17.064 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": {} 2024-09-19T07:06:17.064 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:06:17.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.057+0000 7f900775b640 1 -- 172.21.15.103:0/504932215 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8fec0a33b0 msgr2=0x7f8fec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:17.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.057+0000 7f900775b640 1 --2- 172.21.15.103:0/504932215 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8fec0a33b0 0x7f8fec0a5870 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8005e30 tx=0x7f8ff802fe10 comp rx=0 tx=0).stop 2024-09-19T07:06:17.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.057+0000 7f900775b640 1 -- 172.21.15.103:0/504932215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90000745c0 msgr2=0x7f900019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:17.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.057+0000 7f900775b640 1 --2- 172.21.15.103:0/504932215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90000745c0 0x7f900019a630 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000d950 tx=0x7f8ff000de10 comp rx=0 tx=0).stop 2024-09-19T07:06:17.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.057+0000 7f900775b640 1 -- 172.21.15.103:0/504932215 shutdown_connections 2024-09-19T07:06:17.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.057+0000 7f900775b640 1 --2- 172.21.15.103:0/504932215 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8fec0a33b0 0x7f8fec0a5870 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.057+0000 7f900775b640 1 --2- 172.21.15.103:0/504932215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f900010e420 0x7f900019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.057+0000 7f900775b640 1 --2- 172.21.15.103:0/504932215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9000074ee0 0x7f900019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.057+0000 7f900775b640 1 --2- 172.21.15.103:0/504932215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90000745c0 0x7f900019a630 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:17.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.057+0000 7f900775b640 1 -- 172.21.15.103:0/504932215 >> 172.21.15.103:0/504932215 conn(0x7f90000fd8c0 msgr2=0x7f90000fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:17.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.057+0000 7f900775b640 1 -- 172.21.15.103:0/504932215 shutdown_connections 2024-09-19T07:06:17.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:17.057+0000 7f900775b640 1 -- 172.21.15.103:0/504932215 wait complete. 2024-09-19T07:06:17.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:17 smithi103 ceph-mon[31158]: pgmap v534: 97 pgs: 97 active+clean; 5.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-19T07:06:17.943 INFO:tasks.quiescer:done joining Quiescers 2024-09-19T07:06:17.943 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-19T07:06:17.953 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T07:06:17.962 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-19T07:06:18.006 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-19T07:06:18.006 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T07:06:18.006 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T07:06:18.007 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-19T07:06:18.036 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-19T07:06:18.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:17 smithi174 ceph-mon[35637]: pgmap v534: 97 pgs: 97 active+clean; 5.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-19T07:06:18.105 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T07:06:18.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:17 smithi134 ceph-mon[35356]: pgmap v534: 97 pgs: 97 active+clean; 5.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-19T07:06:18.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.247+0000 7f75bd21e640 1 -- 172.21.15.103:0/902168145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102d50 msgr2=0x7f75b8103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:18.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.247+0000 7f75bd21e640 1 --2- 172.21.15.103:0/902168145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102d50 0x7f75b8103130 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009900 tx=0x7f75a802a480 comp rx=0 tx=0).stop 2024-09-19T07:06:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.247+0000 7f75bd21e640 1 -- 172.21.15.103:0/902168145 shutdown_connections 2024-09-19T07:06:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.247+0000 7f75bd21e640 1 --2- 172.21.15.103:0/902168145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b810c0e0 0x7f75b810e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.247+0000 7f75bd21e640 1 --2- 172.21.15.103:0/902168145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b8103670 0x7f75b810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.247+0000 7f75bd21e640 1 --2- 172.21.15.103:0/902168145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102d50 0x7f75b8103130 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.247+0000 7f75bd21e640 1 -- 172.21.15.103:0/902168145 >> 172.21.15.103:0/902168145 conn(0x7f75b80fc6c0 msgr2=0x7f75b80feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.247+0000 7f75bd21e640 1 -- 172.21.15.103:0/902168145 shutdown_connections 2024-09-19T07:06:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.247+0000 7f75bd21e640 1 -- 172.21.15.103:0/902168145 wait complete. 2024-09-19T07:06:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.248+0000 7f75bd21e640 1 Processor -- start 2024-09-19T07:06:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.248+0000 7f75bd21e640 1 -- start start 2024-09-19T07:06:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.248+0000 7f75bd21e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102d50 0x7f75b81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.248+0000 7f75bd21e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b8103670 0x7f75b81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.248+0000 7f75bd21e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b810c0e0 0x7f75b819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.248+0000 7f75bd21e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8111ee0 con 0x7f75b8102d50 2024-09-19T07:06:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.248+0000 7f75bd21e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8111d60 con 0x7f75b810c0e0 2024-09-19T07:06:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.248+0000 7f75bd21e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8112060 con 0x7f75b8103670 2024-09-19T07:06:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.248+0000 7f75b6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102d50 0x7f75b81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.248+0000 7f75b6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102d50 0x7f75b81983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35544/0 (socket says 172.21.15.103:35544) 2024-09-19T07:06:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.248+0000 7f75b6d9d640 1 -- 172.21.15.103:0/1165526625 learned_addr learned my addr 172.21.15.103:0/1165526625 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.249+0000 7f75b659c640 1 --2- 172.21.15.103:0/1165526625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b8103670 0x7f75b81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.249+0000 7f75b759e640 1 --2- 172.21.15.103:0/1165526625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b810c0e0 0x7f75b819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.249+0000 7f75b6d9d640 1 -- 172.21.15.103:0/1165526625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b8103670 msgr2=0x7f75b81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.249+0000 7f75b6d9d640 1 --2- 172.21.15.103:0/1165526625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b8103670 0x7f75b81988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.249+0000 7f75b6d9d640 1 -- 172.21.15.103:0/1165526625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b810c0e0 msgr2=0x7f75b819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.249+0000 7f75b6d9d640 1 --2- 172.21.15.103:0/1165526625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b810c0e0 0x7f75b819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.249+0000 7f75b6d9d640 1 -- 172.21.15.103:0/1165526625 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b819d3a0 con 0x7f75b8102d50 2024-09-19T07:06:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.249+0000 7f75b759e640 1 --2- 172.21.15.103:0/1165526625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b810c0e0 0x7f75b819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.249+0000 7f75b6d9d640 1 --2- 172.21.15.103:0/1165526625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102d50 0x7f75b81983b0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f75a8006010 tx=0x7f75a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.249+0000 7f75affff640 1 -- 172.21.15.103:0/1165526625 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a803f030 con 0x7f75b8102d50 2024-09-19T07:06:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.249+0000 7f75affff640 1 -- 172.21.15.103:0/1165526625 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a8003ad0 con 0x7f75b8102d50 2024-09-19T07:06:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.249+0000 7f75bd21e640 1 -- 172.21.15.103:0/1165526625 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b819d630 con 0x7f75b8102d50 2024-09-19T07:06:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.249+0000 7f75affff640 1 -- 172.21.15.103:0/1165526625 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8035070 con 0x7f75b8102d50 2024-09-19T07:06:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.249+0000 7f75bd21e640 1 -- 172.21.15.103:0/1165526625 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b81a4f20 con 0x7f75b8102d50 2024-09-19T07:06:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.250+0000 7f75bd21e640 1 -- 172.21.15.103:0/1165526625 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b8103130 con 0x7f75b8102d50 2024-09-19T07:06:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.251+0000 7f75affff640 1 -- 172.21.15.103:0/1165526625 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f75a8042030 con 0x7f75b8102d50 2024-09-19T07:06:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.252+0000 7f75affff640 1 --2- 172.21.15.103:0/1165526625 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75a40a3460 0x7f75a40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.253+0000 7f75b659c640 1 --2- 172.21.15.103:0/1165526625 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75a40a3460 0x7f75a40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.253+0000 7f75affff640 1 -- 172.21.15.103:0/1165526625 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f75a80f0520 con 0x7f75b8102d50 2024-09-19T07:06:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.253+0000 7f75b659c640 1 --2- 172.21.15.103:0/1165526625 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75a40a3460 0x7f75a40a5920 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f75a0005fd0 tx=0x7f75a0005850 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:18.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.257+0000 7f75affff640 1 -- 172.21.15.103:0/1165526625 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f75a80bcfa0 con 0x7f75b8102d50 2024-09-19T07:06:18.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.442+0000 7f75bd21e640 1 -- 172.21.15.103:0/1165526625 --> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f75b8062630 con 0x7f75a40a3460 2024-09-19T07:06:18.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.451+0000 7f75affff640 1 -- 172.21.15.103:0/1165526625 <== mgr.15210 v2:172.21.15.103:6800/82882809 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f75b8062630 con 0x7f75a40a3460 2024-09-19T07:06:18.452 INFO:teuthology.orchestra.run.smithi103.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T06:49:23", "first": "2024-09-19T06:50:00", "last": "2024-09-19T07:06:00", "last_pruned": "2024-09-19T07:06:00", "created_count": 17, "pruned_count": 10, "active": true} 2024-09-19T07:06:18.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.453+0000 7f75bd21e640 1 -- 172.21.15.103:0/1165526625 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75a40a3460 msgr2=0x7f75a40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:18.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.453+0000 7f75bd21e640 1 --2- 172.21.15.103:0/1165526625 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75a40a3460 0x7f75a40a5920 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f75a0005fd0 tx=0x7f75a0005850 comp rx=0 tx=0).stop 2024-09-19T07:06:18.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.453+0000 7f75bd21e640 1 -- 172.21.15.103:0/1165526625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102d50 msgr2=0x7f75b81983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:18.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.453+0000 7f75bd21e640 1 --2- 172.21.15.103:0/1165526625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102d50 0x7f75b81983b0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f75a8006010 tx=0x7f75a8004270 comp rx=0 tx=0).stop 2024-09-19T07:06:18.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.453+0000 7f75bd21e640 1 -- 172.21.15.103:0/1165526625 shutdown_connections 2024-09-19T07:06:18.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.453+0000 7f75bd21e640 1 --2- 172.21.15.103:0/1165526625 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f75a40a3460 0x7f75a40a5920 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.453+0000 7f75bd21e640 1 --2- 172.21.15.103:0/1165526625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b810c0e0 0x7f75b819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.453+0000 7f75bd21e640 1 --2- 172.21.15.103:0/1165526625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b8103670 0x7f75b81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.453+0000 7f75bd21e640 1 --2- 172.21.15.103:0/1165526625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75b8102d50 0x7f75b81983b0 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.453+0000 7f75bd21e640 1 -- 172.21.15.103:0/1165526625 >> 172.21.15.103:0/1165526625 conn(0x7f75b80fc6c0 msgr2=0x7f75b80fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:18.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.453+0000 7f75bd21e640 1 -- 172.21.15.103:0/1165526625 shutdown_connections 2024-09-19T07:06:18.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.453+0000 7f75bd21e640 1 -- 172.21.15.103:0/1165526625 wait complete. 2024-09-19T07:06:18.465 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T07:06:18.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.566+0000 7f7c3ee53640 1 -- 172.21.15.103:0/178575841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c381069e0 msgr2=0x7f7c38106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:18.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.566+0000 7f7c3ee53640 1 --2- 172.21.15.103:0/178575841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c381069e0 0x7f7c38106dc0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009930 tx=0x7f7c2c02a420 comp rx=0 tx=0).stop 2024-09-19T07:06:18.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.566+0000 7f7c3ee53640 1 -- 172.21.15.103:0/178575841 shutdown_connections 2024-09-19T07:06:18.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.566+0000 7f7c3ee53640 1 --2- 172.21.15.103:0/178575841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c38101370 0x7f7c3810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.566+0000 7f7c3ee53640 1 --2- 172.21.15.103:0/178575841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c381009d0 0x7f7c38100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.566+0000 7f7c3ee53640 1 --2- 172.21.15.103:0/178575841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c381069e0 0x7f7c38106dc0 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.566+0000 7f7c3ee53640 1 -- 172.21.15.103:0/178575841 >> 172.21.15.103:0/178575841 conn(0x7f7c380fc6f0 msgr2=0x7f7c380feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:18.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.566+0000 7f7c3ee53640 1 -- 172.21.15.103:0/178575841 shutdown_connections 2024-09-19T07:06:18.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.566+0000 7f7c3ee53640 1 -- 172.21.15.103:0/178575841 wait complete. 2024-09-19T07:06:18.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c3ee53640 1 Processor -- start 2024-09-19T07:06:18.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c3ee53640 1 -- start start 2024-09-19T07:06:18.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c3ee53640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c381009d0 0x7f7c381991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:18.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c3ee53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c38101370 0x7f7c38199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:18.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c3ee53640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c381069e0 0x7f7c381a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:18.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c3ee53640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c38110e20 con 0x7f7c38101370 2024-09-19T07:06:18.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c3ee53640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c38110ca0 con 0x7f7c381009d0 2024-09-19T07:06:18.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c3ee53640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c38110fa0 con 0x7f7c381069e0 2024-09-19T07:06:18.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c37fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c38101370 0x7f7c38199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:18.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c37fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c38101370 0x7f7c38199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35556/0 (socket says 172.21.15.103:35556) 2024-09-19T07:06:18.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c37fff640 1 -- 172.21.15.103:0/781347722 learned_addr learned my addr 172.21.15.103:0/781347722 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:18.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c3cbef640 1 --2- 172.21.15.103:0/781347722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c381009d0 0x7f7c381991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:18.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c37fff640 1 -- 172.21.15.103:0/781347722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c381069e0 msgr2=0x7f7c381a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:18.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c37fff640 1 --2- 172.21.15.103:0/781347722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c381069e0 0x7f7c381a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c37fff640 1 -- 172.21.15.103:0/781347722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c381009d0 msgr2=0x7f7c381991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:18.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c37fff640 1 --2- 172.21.15.103:0/781347722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c381009d0 0x7f7c381991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c37fff640 1 -- 172.21.15.103:0/781347722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c381a4790 con 0x7f7c38101370 2024-09-19T07:06:18.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.567+0000 7f7c37fff640 1 --2- 172.21.15.103:0/781347722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c38101370 0x7f7c38199710 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400d770 tx=0x7f7c2400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:18.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.568+0000 7f7c35ffb640 1 -- 172.21.15.103:0/781347722 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c24017070 con 0x7f7c38101370 2024-09-19T07:06:18.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.568+0000 7f7c35ffb640 1 -- 172.21.15.103:0/781347722 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c24005500 con 0x7f7c38101370 2024-09-19T07:06:18.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.568+0000 7f7c3ee53640 1 -- 172.21.15.103:0/781347722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c381a4960 con 0x7f7c38101370 2024-09-19T07:06:18.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.568+0000 7f7c35ffb640 1 -- 172.21.15.103:0/781347722 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c24004e00 con 0x7f7c38101370 2024-09-19T07:06:18.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.568+0000 7f7c3ee53640 1 -- 172.21.15.103:0/781347722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c381a4e80 con 0x7f7c38101370 2024-09-19T07:06:18.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.569+0000 7f7c3ee53640 1 -- 172.21.15.103:0/781347722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c38100e30 con 0x7f7c38101370 2024-09-19T07:06:18.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.570+0000 7f7c35ffb640 1 -- 172.21.15.103:0/781347722 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f7c24004fa0 con 0x7f7c38101370 2024-09-19T07:06:18.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.571+0000 7f7c35ffb640 1 --2- 172.21.15.103:0/781347722 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c280a34e0 0x7f7c280a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:18.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.571+0000 7f7c3cbef640 1 --2- 172.21.15.103:0/781347722 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c280a34e0 0x7f7c280a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:18.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.571+0000 7f7c35ffb640 1 -- 172.21.15.103:0/781347722 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f7c240cf7b0 con 0x7f7c38101370 2024-09-19T07:06:18.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.571+0000 7f7c3cbef640 1 --2- 172.21.15.103:0/781347722 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c280a34e0 0x7f7c280a59a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009900 tx=0x7f7c2c0058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:18.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.575+0000 7f7c35ffb640 1 -- 172.21.15.103:0/781347722 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7c2409c110 con 0x7f7c38101370 2024-09-19T07:06:18.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.760+0000 7f7c3ee53640 1 -- 172.21.15.103:0/781347722 --> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7c380fd470 con 0x7f7c280a34e0 2024-09-19T07:06:18.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.769+0000 7f7c35ffb640 1 -- 172.21.15.103:0/781347722 <== mgr.15210 v2:172.21.15.103:6800/82882809 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7c380fd470 con 0x7f7c280a34e0 2024-09-19T07:06:18.770 INFO:teuthology.orchestra.run.smithi103.stdout:/ 1m 6m3h 2024-09-19T07:06:18.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.771+0000 7f7c3ee53640 1 -- 172.21.15.103:0/781347722 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c280a34e0 msgr2=0x7f7c280a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:18.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.771+0000 7f7c3ee53640 1 --2- 172.21.15.103:0/781347722 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c280a34e0 0x7f7c280a59a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009900 tx=0x7f7c2c0058d0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.771+0000 7f7c3ee53640 1 -- 172.21.15.103:0/781347722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c38101370 msgr2=0x7f7c38199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:18.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.771+0000 7f7c3ee53640 1 --2- 172.21.15.103:0/781347722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c38101370 0x7f7c38199710 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400d770 tx=0x7f7c2400dc30 comp rx=0 tx=0).stop 2024-09-19T07:06:18.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.771+0000 7f7c3ee53640 1 -- 172.21.15.103:0/781347722 shutdown_connections 2024-09-19T07:06:18.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.771+0000 7f7c3ee53640 1 --2- 172.21.15.103:0/781347722 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f7c280a34e0 0x7f7c280a59a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.771+0000 7f7c3ee53640 1 --2- 172.21.15.103:0/781347722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c381069e0 0x7f7c381a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.771+0000 7f7c3ee53640 1 --2- 172.21.15.103:0/781347722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c38101370 0x7f7c38199710 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.771+0000 7f7c3ee53640 1 --2- 172.21.15.103:0/781347722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c381009d0 0x7f7c381991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:18.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.771+0000 7f7c3ee53640 1 -- 172.21.15.103:0/781347722 >> 172.21.15.103:0/781347722 conn(0x7f7c380fc6f0 msgr2=0x7f7c380fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:18.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.771+0000 7f7c3ee53640 1 -- 172.21.15.103:0/781347722 shutdown_connections 2024-09-19T07:06:18.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:18.771+0000 7f7c3ee53640 1 -- 172.21.15.103:0/781347722 wait complete. 2024-09-19T07:06:18.783 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-19T07:06:18.794 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-19T07:06:20.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:19 smithi174 ceph-mon[35637]: pgmap v535: 97 pgs: 97 active+clean; 5.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-19T07:06:20.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:19 smithi174 ceph-mon[35637]: from='client.21537 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:06:20.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:19 smithi174 ceph-mon[35637]: from='client.21543 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:06:20.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:19 smithi103 ceph-mon[31158]: pgmap v535: 97 pgs: 97 active+clean; 5.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-19T07:06:20.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:19 smithi103 ceph-mon[31158]: from='client.21537 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:06:20.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:19 smithi103 ceph-mon[31158]: from='client.21543 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:06:20.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:19 smithi134 ceph-mon[35356]: pgmap v535: 97 pgs: 97 active+clean; 5.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-19T07:06:20.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:19 smithi134 ceph-mon[35356]: from='client.21537 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:06:20.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:19 smithi134 ceph-mon[35356]: from='client.21543 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:06:22.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:21 smithi174 ceph-mon[35637]: pgmap v536: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-09-19T07:06:22.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:21 smithi103 ceph-mon[31158]: pgmap v536: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-09-19T07:06:22.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:21 smithi134 ceph-mon[35356]: pgmap v536: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-09-19T07:06:24.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:23 smithi174 ceph-mon[35637]: pgmap v537: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 30 KiB/s wr, 21 op/s 2024-09-19T07:06:24.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:23 smithi103 ceph-mon[31158]: pgmap v537: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 30 KiB/s wr, 21 op/s 2024-09-19T07:06:24.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:23 smithi134 ceph-mon[35356]: pgmap v537: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 30 KiB/s wr, 21 op/s 2024-09-19T07:06:26.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:25 smithi174 ceph-mon[35637]: pgmap v538: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-09-19T07:06:26.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:25 smithi103 ceph-mon[31158]: pgmap v538: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-09-19T07:06:26.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:25 smithi134 ceph-mon[35356]: pgmap v538: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-09-19T07:06:28.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:27 smithi174 ceph-mon[35637]: pgmap v539: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 KiB/s wr, 23 op/s 2024-09-19T07:06:28.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:27 smithi103 ceph-mon[31158]: pgmap v539: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 KiB/s wr, 23 op/s 2024-09-19T07:06:28.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:27 smithi134 ceph-mon[35356]: pgmap v539: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 KiB/s wr, 23 op/s 2024-09-19T07:06:30.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:29 smithi174 ceph-mon[35637]: pgmap v540: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.5 KiB/s wr, 16 op/s 2024-09-19T07:06:30.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:29 smithi103 ceph-mon[31158]: pgmap v540: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.5 KiB/s wr, 16 op/s 2024-09-19T07:06:30.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:29 smithi134 ceph-mon[35356]: pgmap v540: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.5 KiB/s wr, 16 op/s 2024-09-19T07:06:32.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:31 smithi174 ceph-mon[35637]: pgmap v541: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.5 KiB/s wr, 21 op/s 2024-09-19T07:06:32.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:31 smithi103 ceph-mon[31158]: pgmap v541: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.5 KiB/s wr, 21 op/s 2024-09-19T07:06:32.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:31 smithi134 ceph-mon[35356]: pgmap v541: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.5 KiB/s wr, 21 op/s 2024-09-19T07:06:34.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:33 smithi174 ceph-mon[35637]: pgmap v542: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-19T07:06:34.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:33 smithi103 ceph-mon[31158]: pgmap v542: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-19T07:06:34.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:33 smithi134 ceph-mon[35356]: pgmap v542: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-19T07:06:36.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:35 smithi174 ceph-mon[35637]: pgmap v543: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-19T07:06:36.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:35 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:06:36.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:35 smithi103 ceph-mon[31158]: pgmap v543: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-19T07:06:36.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:35 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:06:36.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:35 smithi134 ceph-mon[35356]: pgmap v543: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-19T07:06:36.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:35 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:06:37.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:36 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:06:37.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:36 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:06:37.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:36 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:06:37.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:36 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:06:37.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:36 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:06:37.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:36 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:06:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:36 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:06:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:36 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:06:37.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:36 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' 2024-09-19T07:06:38.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:37 smithi174 ceph-mon[35637]: pgmap v544: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-19T07:06:38.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:37 smithi103 ceph-mon[31158]: pgmap v544: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-19T07:06:38.427 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:06:38.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:37 smithi134 ceph-mon[35356]: pgmap v544: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-19T07:06:38.690 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:39.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.156+0000 7fca230ac640 1 -- 172.21.15.103:0/1785359106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca1c100a80 msgr2=0x7fca1c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:39.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.156+0000 7fca230ac640 1 --2- 172.21.15.103:0/1785359106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca1c100a80 0x7fca1c100ee0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fca08009930 tx=0x7fca0802a420 comp rx=0 tx=0).stop 2024-09-19T07:06:39.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.156+0000 7fca230ac640 1 -- 172.21.15.103:0/1785359106 shutdown_connections 2024-09-19T07:06:39.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.156+0000 7fca230ac640 1 --2- 172.21.15.103:0/1785359106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca1c101420 0x7fca1c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.156+0000 7fca230ac640 1 --2- 172.21.15.103:0/1785359106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca1c100a80 0x7fca1c100ee0 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.156+0000 7fca230ac640 1 --2- 172.21.15.103:0/1785359106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca1c106a80 0x7fca1c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.156+0000 7fca230ac640 1 -- 172.21.15.103:0/1785359106 >> 172.21.15.103:0/1785359106 conn(0x7fca1c0fc7a0 msgr2=0x7fca1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:39.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.156+0000 7fca230ac640 1 -- 172.21.15.103:0/1785359106 shutdown_connections 2024-09-19T07:06:39.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.156+0000 7fca230ac640 1 -- 172.21.15.103:0/1785359106 wait complete. 2024-09-19T07:06:39.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.157+0000 7fca230ac640 1 Processor -- start 2024-09-19T07:06:39.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.157+0000 7fca230ac640 1 -- start start 2024-09-19T07:06:39.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.157+0000 7fca230ac640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca1c100a80 0x7fca1c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.157+0000 7fca230ac640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca1c101420 0x7fca1c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.157+0000 7fca230ac640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca1c106a80 0x7fca1c1a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.157+0000 7fca230ac640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c110ea0 con 0x7fca1c101420 2024-09-19T07:06:39.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.157+0000 7fca230ac640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c110d20 con 0x7fca1c100a80 2024-09-19T07:06:39.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.157+0000 7fca230ac640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c111020 con 0x7fca1c106a80 2024-09-19T07:06:39.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.157+0000 7fca1bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca1c101420 0x7fca1c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.157+0000 7fca1bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca1c101420 0x7fca1c199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55162/0 (socket says 172.21.15.103:55162) 2024-09-19T07:06:39.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.157+0000 7fca1bfff640 1 -- 172.21.15.103:0/2172729128 learned_addr learned my addr 172.21.15.103:0/2172729128 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:39.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.157+0000 7fca20e48640 1 --2- 172.21.15.103:0/2172729128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca1c100a80 0x7fca1c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.158+0000 7fca21649640 1 --2- 172.21.15.103:0/2172729128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca1c106a80 0x7fca1c1a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.158+0000 7fca1bfff640 1 -- 172.21.15.103:0/2172729128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca1c106a80 msgr2=0x7fca1c1a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:39.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.158+0000 7fca1bfff640 1 --2- 172.21.15.103:0/2172729128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca1c106a80 0x7fca1c1a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.158+0000 7fca1bfff640 1 -- 172.21.15.103:0/2172729128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca1c100a80 msgr2=0x7fca1c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:39.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.158+0000 7fca1bfff640 1 --2- 172.21.15.103:0/2172729128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca1c100a80 0x7fca1c199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.158+0000 7fca1bfff640 1 -- 172.21.15.103:0/2172729128 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1c1a48f0 con 0x7fca1c101420 2024-09-19T07:06:39.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.158+0000 7fca21649640 1 --2- 172.21.15.103:0/2172729128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca1c106a80 0x7fca1c1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:39.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.158+0000 7fca1bfff640 1 --2- 172.21.15.103:0/2172729128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca1c101420 0x7fca1c199870 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fca08009230 tx=0x7fca08004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:39.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.158+0000 7fca19ffb640 1 -- 172.21.15.103:0/2172729128 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0803f030 con 0x7fca1c101420 2024-09-19T07:06:39.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.158+0000 7fca19ffb640 1 -- 172.21.15.103:0/2172729128 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca08003ad0 con 0x7fca1c101420 2024-09-19T07:06:39.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.158+0000 7fca230ac640 1 -- 172.21.15.103:0/2172729128 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1c1a4ac0 con 0x7fca1c101420 2024-09-19T07:06:39.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.158+0000 7fca19ffb640 1 -- 172.21.15.103:0/2172729128 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08035070 con 0x7fca1c101420 2024-09-19T07:06:39.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.158+0000 7fca230ac640 1 -- 172.21.15.103:0/2172729128 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca1c1a4f10 con 0x7fca1c101420 2024-09-19T07:06:39.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.160+0000 7fca19ffb640 1 -- 172.21.15.103:0/2172729128 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fca08042030 con 0x7fca1c101420 2024-09-19T07:06:39.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.161+0000 7fca25485740 1 Processor -- start 2024-09-19T07:06:39.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.161+0000 7fca25485740 1 -- start start 2024-09-19T07:06:39.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.161+0000 7fca25485740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71dd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.161+0000 7fca25485740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71dddc0 0x563bd71e8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.161+0000 7fca25485740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71e8680 0x563bd71eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.161+0000 7fca25485740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563bd71432e0 con 0x563bd71dd4a0 2024-09-19T07:06:39.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.161+0000 7fca25485740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563bd70eace0 con 0x563bd71dddc0 2024-09-19T07:06:39.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.161+0000 7fca25485740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563bd713f2a0 con 0x563bd71e8680 2024-09-19T07:06:39.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.161+0000 7fca19ffb640 1 --2- 172.21.15.103:0/2172729128 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fca0c0a33b0 0x7fca0c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.161+0000 7fca21649640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.161+0000 7fca21649640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71dd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55166/0 (socket says 172.21.15.103:55166) 2024-09-19T07:06:39.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.161+0000 7fca21649640 1 -- 172.21.15.103:0/237388985 learned_addr learned my addr 172.21.15.103:0/237388985 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:39.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.161+0000 7fca21649640 1 --2- 172.21.15.103:0/2172729128 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fca0c0a33b0 0x7fca0c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.161+0000 7fca1bfff640 1 --2- 172.21.15.103:0/237388985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71e8680 0x563bd71eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.161+0000 7fca19ffb640 1 -- 172.21.15.103:0/2172729128 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7fca080f02b0 con 0x7fca1c101420 2024-09-19T07:06:39.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.161+0000 7fca20e48640 1 --2- 172.21.15.103:0/237388985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71dddc0 0x563bd71e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.162+0000 7fca21649640 1 -- 172.21.15.103:0/237388985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71e8680 msgr2=0x563bd71eaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:39.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.162+0000 7fca21649640 1 --2- 172.21.15.103:0/237388985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71e8680 0x563bd71eaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.162+0000 7fca21649640 1 -- 172.21.15.103:0/237388985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71dddc0 msgr2=0x563bd71e8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:39.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.162+0000 7fca21649640 1 --2- 172.21.15.103:0/237388985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71dddc0 0x563bd71e8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.162+0000 7fca21649640 1 -- 172.21.15.103:0/237388985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bd70cf440 con 0x563bd71dd4a0 2024-09-19T07:06:39.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.162+0000 7fca20e48640 1 --2- 172.21.15.103:0/237388985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71dddc0 0x563bd71e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:06:39.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.162+0000 7fca21649640 1 --2- 172.21.15.103:0/237388985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71dd880 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fca1400ed70 tx=0x7fca1400c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809be2c6721d2eb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:39.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.162+0000 7fca21649640 1 --2- 172.21.15.103:0/2172729128 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fca0c0a33b0 0x7fca0c0a5870 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fca1400cc40 tx=0x7fca14019000 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:39.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.162+0000 7fca06ffd640 1 -- 172.21.15.103:0/237388985 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca14018b70 con 0x563bd71dd4a0 2024-09-19T07:06:39.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.162+0000 7fca06ffd640 1 -- 172.21.15.103:0/237388985 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca14018d10 con 0x563bd71dd4a0 2024-09-19T07:06:39.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.162+0000 7fca06ffd640 1 -- 172.21.15.103:0/237388985 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1401ae60 con 0x563bd71dd4a0 2024-09-19T07:06:39.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.163+0000 7fca25485740 1 -- 172.21.15.103:0/237388985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 msgr2=0x563bd71dd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:39.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.163+0000 7fca25485740 1 --2- 172.21.15.103:0/237388985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71dd880 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fca1400ed70 tx=0x7fca1400c740 comp rx=0 tx=0).stop 2024-09-19T07:06:39.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.163+0000 7fca25485740 1 -- 172.21.15.103:0/237388985 shutdown_connections 2024-09-19T07:06:39.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.163+0000 7fca25485740 1 --2- 172.21.15.103:0/237388985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71e8680 0x563bd71eaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.163+0000 7fca25485740 1 --2- 172.21.15.103:0/237388985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71dddc0 0x563bd71e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.163+0000 7fca25485740 1 --2- 172.21.15.103:0/237388985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71dd880 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.163+0000 7fca25485740 1 -- 172.21.15.103:0/237388985 >> 172.21.15.103:0/237388985 conn(0x563bd71d9a30 msgr2=0x563bd71d9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:39.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.163+0000 7fca25485740 1 -- 172.21.15.103:0/237388985 shutdown_connections 2024-09-19T07:06:39.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.163+0000 7fca25485740 1 -- 172.21.15.103:0/237388985 wait complete. 2024-09-19T07:06:39.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.163+0000 7fca25485740 1 -- start start 2024-09-19T07:06:39.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.163+0000 7fca25485740 1 Processor -- start 2024-09-19T07:06:39.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca25485740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71e8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca25485740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71dddc0 0x563bd71e52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca25485740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 0x563bd71e5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca25485740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563bd71eb250 con 0x563bd71dd4a0 2024-09-19T07:06:39.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca25485740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563bd70fe050 con 0x563bd71e8680 2024-09-19T07:06:39.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca25485740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563bd71eb3d0 con 0x563bd71dddc0 2024-09-19T07:06:39.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca25485740 10 client.0 _ll_register_callbacks cb 0x563bd713e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:39.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca20e48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca20e48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71e8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55182/0 (socket says 172.21.15.103:55182) 2024-09-19T07:06:39.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca20e48640 1 -- 172.21.15.103:0/1494710918 learned_addr learned my addr 172.21.15.103:0/1494710918 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:39.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca1bfff640 1 --2- 172.21.15.103:0/1494710918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71dddc0 0x563bd71e52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca21649640 1 --2- 172.21.15.103:0/1494710918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 0x563bd71e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca20e48640 1 -- 172.21.15.103:0/1494710918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71dddc0 msgr2=0x563bd71e52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:39.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca20e48640 1 --2- 172.21.15.103:0/1494710918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71dddc0 0x563bd71e52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca20e48640 1 -- 172.21.15.103:0/1494710918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 msgr2=0x563bd71e5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:39.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca20e48640 1 --2- 172.21.15.103:0/1494710918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 0x563bd71e5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca20e48640 1 -- 172.21.15.103:0/1494710918 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bd6eb43c0 con 0x563bd71dd4a0 2024-09-19T07:06:39.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca1bfff640 1 --2- 172.21.15.103:0/1494710918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71dddc0 0x563bd71e52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:06:39.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca21649640 1 --2- 172.21.15.103:0/1494710918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 0x563bd71e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:39.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca20e48640 1 --2- 172.21.15.103:0/1494710918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71e8030 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fca1000baa0 tx=0x7fca1000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115c6de9d0f6d71a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:39.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fc9fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:39.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fc9fbfff640 1 -- 172.21.15.103:0/1494710918 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1001b640 con 0x563bd71dd4a0 2024-09-19T07:06:39.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fc9fbfff640 1 -- 172.21.15.103:0/1494710918 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7fca10011440 con 0x563bd71dd4a0 2024-09-19T07:06:39.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.164+0000 7fca25485740 1 -- 172.21.15.103:0/1494710918 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca14018d10 con 0x563bd71dd4a0 2024-09-19T07:06:39.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.165+0000 7fc9fbfff640 1 -- 172.21.15.103:0/1494710918 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca1001b7e0 con 0x563bd71dd4a0 2024-09-19T07:06:39.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.165+0000 7fca25485740 10 client.21555 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:39.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.165+0000 7fca25485740 1 -- 172.21.15.103:0/1494710918 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563bd71f6cc0 con 0x563bd71dd4a0 2024-09-19T07:06:39.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.165+0000 7fc9fbfff640 1 -- 172.21.15.103:0/1494710918 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca14018d10 con 0x563bd71dd4a0 2024-09-19T07:06:39.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.165+0000 7fc9fbfff640 1 -- 172.21.15.103:0/1494710918 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fca10019020 con 0x563bd71dd4a0 2024-09-19T07:06:39.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.165+0000 7fca25485740 10 client.21555 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:39.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.165+0000 7fca25485740 10 client.21555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:06:39.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.165+0000 7fca25485740 20 client.21555 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:39.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.165+0000 7fca25485740 1 --2- 172.21.15.103:0/1494710918 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563bd71f8180 0x563bd71fa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.165+0000 7fca25485740 4 client.21555 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:06:39.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.165+0000 7fca25485740 1 -- 172.21.15.103:0/1494710918 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bd7150850 con 0x563bd71f8180 2024-09-19T07:06:39.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.165+0000 7fca1bfff640 1 --2- 172.21.15.103:0/1494710918 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563bd71f8180 0x563bd71fa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.166+0000 7fc9fbfff640 10 client.21555 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:39.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.166+0000 7fca1bfff640 1 --2- 172.21.15.103:0/1494710918 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563bd71f8180 0x563bd71fa5b0 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a3af9d8431c7a8 server_cookie=112e7711be5cf47b in_seq=0 out_seq=0 2024-09-19T07:06:39.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.166+0000 7fc9fbfff640 1 -- 172.21.15.103:0/1494710918 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563bd7150850 con 0x563bd71f8180 2024-09-19T07:06:39.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.166+0000 7fc9fbfff640 10 client.21555 handle_command_reply: tid=0 2024-09-19T07:06:39.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.166+0000 7fca25485740 1 client.21555 shutdown 2024-09-19T07:06:39.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.167+0000 7fca25485740 1 -- 172.21.15.103:0/1494710918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 msgr2=0x563bd71e8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:39.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.167+0000 7fca25485740 1 --2- 172.21.15.103:0/1494710918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71e8030 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fca1000baa0 tx=0x7fca1000bf60 comp rx=0 tx=0).stop 2024-09-19T07:06:39.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.167+0000 7fca25485740 1 -- 172.21.15.103:0/1494710918 shutdown_connections 2024-09-19T07:06:39.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.167+0000 7fca25485740 1 --2- 172.21.15.103:0/1494710918 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563bd71f8180 0x563bd71fa5b0 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.167+0000 7fca25485740 1 --2- 172.21.15.103:0/1494710918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 0x563bd71e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.167+0000 7fca25485740 1 --2- 172.21.15.103:0/1494710918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71dddc0 0x563bd71e52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.167+0000 7fc9fbfff640 0 client.21555 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:39.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.167+0000 7fca25485740 1 --2- 172.21.15.103:0/1494710918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71e8030 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.167+0000 7fca25485740 1 -- 172.21.15.103:0/1494710918 >> 172.21.15.103:0/1494710918 conn(0x563bd71d9a30 msgr2=0x563bd71d8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:39.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.167+0000 7fca25485740 1 -- 172.21.15.103:0/1494710918 shutdown_connections 2024-09-19T07:06:39.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.167+0000 7fca25485740 1 -- 172.21.15.103:0/1494710918 wait complete. 2024-09-19T07:06:39.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.167+0000 7fca25485740 20 client.21555 trim_cache size 0 max 16384 2024-09-19T07:06:39.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.184+0000 7fca25485740 1 Processor -- start 2024-09-19T07:06:39.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.184+0000 7fca25485740 1 -- start start 2024-09-19T07:06:39.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.184+0000 7fca25485740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.184+0000 7fca25485740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71dddc0 0x563bd72003c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.184+0000 7fca25485740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 0x563bd71e2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.184+0000 7fca25485740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1ad910 con 0x563bd71dd4a0 2024-09-19T07:06:39.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.184+0000 7fca25485740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1af450 con 0x563bd71e8680 2024-09-19T07:06:39.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.184+0000 7fca25485740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1adfe0 con 0x563bd71dddc0 2024-09-19T07:06:39.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.184+0000 7fca20e48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.184+0000 7fca20e48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71ffe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55196/0 (socket says 172.21.15.103:55196) 2024-09-19T07:06:39.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.184+0000 7fca20e48640 1 -- 172.21.15.103:0/2104319578 learned_addr learned my addr 172.21.15.103:0/2104319578 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:39.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.185+0000 7fca21649640 1 --2- 172.21.15.103:0/2104319578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 0x563bd71e2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.185+0000 7fca1bfff640 1 --2- 172.21.15.103:0/2104319578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71dddc0 0x563bd72003c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.185+0000 7fca20e48640 1 -- 172.21.15.103:0/2104319578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71dddc0 msgr2=0x563bd72003c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:39.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.185+0000 7fca20e48640 1 --2- 172.21.15.103:0/2104319578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71dddc0 0x563bd72003c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.185+0000 7fca20e48640 1 -- 172.21.15.103:0/2104319578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 msgr2=0x563bd71e2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:39.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.185+0000 7fca20e48640 1 --2- 172.21.15.103:0/2104319578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 0x563bd71e2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.185+0000 7fca20e48640 1 -- 172.21.15.103:0/2104319578 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bd71e6660 con 0x563bd71dd4a0 2024-09-19T07:06:39.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.185+0000 7fca20e48640 1 --2- 172.21.15.103:0/2104319578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71ffe80 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7fca10000c00 tx=0x7fca10003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bb6409445f4fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:39.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.185+0000 7fca067fc640 1 -- 172.21.15.103:0/2104319578 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10030070 con 0x563bd71dd4a0 2024-09-19T07:06:39.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.185+0000 7fca067fc640 1 -- 172.21.15.103:0/2104319578 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca10010500 con 0x563bd71dd4a0 2024-09-19T07:06:39.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.185+0000 7fca067fc640 1 -- 172.21.15.103:0/2104319578 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10027040 con 0x563bd71dd4a0 2024-09-19T07:06:39.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.186+0000 7fca25485740 1 -- 172.21.15.103:0/2104319578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 msgr2=0x563bd71ffe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:39.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.186+0000 7fca25485740 1 --2- 172.21.15.103:0/2104319578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71ffe80 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7fca10000c00 tx=0x7fca10003aa0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.186+0000 7fca25485740 1 -- 172.21.15.103:0/2104319578 shutdown_connections 2024-09-19T07:06:39.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.186+0000 7fca25485740 1 --2- 172.21.15.103:0/2104319578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 0x563bd71e2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.186+0000 7fca25485740 1 --2- 172.21.15.103:0/2104319578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71dddc0 0x563bd72003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.186+0000 7fca25485740 1 --2- 172.21.15.103:0/2104319578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dd4a0 0x563bd71ffe80 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.186+0000 7fca25485740 1 -- 172.21.15.103:0/2104319578 >> 172.21.15.103:0/2104319578 conn(0x563bd71d9a30 msgr2=0x563bd71d8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:39.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.186+0000 7fca25485740 1 -- 172.21.15.103:0/2104319578 shutdown_connections 2024-09-19T07:06:39.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.186+0000 7fca25485740 1 -- 172.21.15.103:0/2104319578 wait complete. 2024-09-19T07:06:39.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.186+0000 7fca25485740 1 -- start start 2024-09-19T07:06:39.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.186+0000 7fca25485740 1 Processor -- start 2024-09-19T07:06:39.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca25485740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71dd4a0 0x563bd71ee540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca25485740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dddc0 0x563bd71eea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca25485740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 0x563bd71eefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca25485740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1aed80 con 0x563bd71dddc0 2024-09-19T07:06:39.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca25485740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1ae6b0 con 0x563bd71e8680 2024-09-19T07:06:39.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca25485740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1ab910 con 0x563bd71dd4a0 2024-09-19T07:06:39.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca25485740 10 client.0 _ll_register_callbacks cb 0x7fca10004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:39.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca1bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dddc0 0x563bd71eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca1bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dddc0 0x563bd71eea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55198/0 (socket says 172.21.15.103:55198) 2024-09-19T07:06:39.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca1bfff640 1 -- 172.21.15.103:0/3155062342 learned_addr learned my addr 172.21.15.103:0/3155062342 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:39.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca20e48640 1 --2- 172.21.15.103:0/3155062342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71dd4a0 0x563bd71ee540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca21649640 1 --2- 172.21.15.103:0/3155062342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 0x563bd71eefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca1bfff640 1 -- 172.21.15.103:0/3155062342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71dd4a0 msgr2=0x563bd71ee540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:39.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca1bfff640 1 --2- 172.21.15.103:0/3155062342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71dd4a0 0x563bd71ee540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca1bfff640 1 -- 172.21.15.103:0/3155062342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 msgr2=0x563bd71eefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:39.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca1bfff640 1 --2- 172.21.15.103:0/3155062342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 0x563bd71eefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca1bfff640 1 -- 172.21.15.103:0/3155062342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bd7262000 con 0x563bd71dddc0 2024-09-19T07:06:39.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca21649640 1 --2- 172.21.15.103:0/3155062342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 0x563bd71eefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:39.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fca1bfff640 1 --2- 172.21.15.103:0/3155062342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dddc0 0x563bd71eea80 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fca080bc4a0 tx=0x7fca0802ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df9bb3f35036852 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:39.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.187+0000 7fc9fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:39.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.188+0000 7fc9fbfff640 1 -- 172.21.15.103:0/3155062342 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca080d5020 con 0x563bd71dddc0 2024-09-19T07:06:39.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.188+0000 7fc9fbfff640 1 -- 172.21.15.103:0/3155062342 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7fca08041d80 con 0x563bd71dddc0 2024-09-19T07:06:39.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.188+0000 7fca25485740 1 -- 172.21.15.103:0/3155062342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca10010500 con 0x563bd71dddc0 2024-09-19T07:06:39.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.188+0000 7fc9fbfff640 1 -- 172.21.15.103:0/3155062342 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca08049040 con 0x563bd71dddc0 2024-09-19T07:06:39.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.188+0000 7fca25485740 10 client.21561 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:39.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.188+0000 7fca25485740 1 -- 172.21.15.103:0/3155062342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563bd72632f0 con 0x563bd71dddc0 2024-09-19T07:06:39.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.188+0000 7fc9fbfff640 1 -- 172.21.15.103:0/3155062342 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca10010500 con 0x563bd71dddc0 2024-09-19T07:06:39.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.188+0000 7fc9fbfff640 1 -- 172.21.15.103:0/3155062342 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fca080edc00 con 0x563bd71dddc0 2024-09-19T07:06:39.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.188+0000 7fca25485740 10 client.21561 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:39.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.188+0000 7fca25485740 10 client.21561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:06:39.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.188+0000 7fca25485740 20 client.21561 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:39.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.188+0000 7fca25485740 1 --2- 172.21.15.103:0/3155062342 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563bd71f8180 0x563bd7265ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:39.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.188+0000 7fca25485740 4 client.21561 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:06:39.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.188+0000 7fca25485740 1 -- 172.21.15.103:0/3155062342 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563bd71d9300 con 0x563bd71f8180 2024-09-19T07:06:39.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.188+0000 7fca20e48640 1 --2- 172.21.15.103:0/3155062342 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563bd71f8180 0x563bd7265ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:39.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fca20e48640 1 --2- 172.21.15.103:0/3155062342 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563bd71f8180 0x563bd7265ae0 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=290df7418390aad8 server_cookie=e3985d9741f30e3d in_seq=0 out_seq=0 2024-09-19T07:06:39.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fc9fbfff640 10 client.21561 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:39.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fc9fbfff640 1 -- 172.21.15.103:0/3155062342 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563bd71d9300 con 0x563bd71f8180 2024-09-19T07:06:39.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fc9fbfff640 10 client.21561 handle_command_reply: tid=0 2024-09-19T07:06:39.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fca25485740 1 client.21561 shutdown 2024-09-19T07:06:39.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fca25485740 1 -- 172.21.15.103:0/3155062342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dddc0 msgr2=0x563bd71eea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:39.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fca25485740 1 --2- 172.21.15.103:0/3155062342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dddc0 0x563bd71eea80 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fca080bc4a0 tx=0x7fca0802ffa0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fca25485740 1 -- 172.21.15.103:0/3155062342 shutdown_connections 2024-09-19T07:06:39.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fca25485740 1 --2- 172.21.15.103:0/3155062342 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x563bd71f8180 0x563bd7265ae0 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fca25485740 1 --2- 172.21.15.103:0/3155062342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd71e8680 0x563bd71eefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fca25485740 1 --2- 172.21.15.103:0/3155062342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bd71dddc0 0x563bd71eea80 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fca25485740 1 --2- 172.21.15.103:0/3155062342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bd71dd4a0 0x563bd71ee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fc9fbfff640 0 client.21561 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:39.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fca25485740 1 -- 172.21.15.103:0/3155062342 >> 172.21.15.103:0/3155062342 conn(0x563bd71d9a30 msgr2=0x563bd71e6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:39.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fca25485740 1 -- 172.21.15.103:0/3155062342 shutdown_connections 2024-09-19T07:06:39.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fca25485740 1 -- 172.21.15.103:0/3155062342 wait complete. 2024-09-19T07:06:39.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.189+0000 7fca25485740 20 client.21561 trim_cache size 0 max 16384 2024-09-19T07:06:39.197 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:06:39.198 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:06:39.199 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:06:39.199 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:06:39.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.190+0000 7fca230ac640 1 -- 172.21.15.103:0/2172729128 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fca0c0a33b0 msgr2=0x7fca0c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:39.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.190+0000 7fca230ac640 1 --2- 172.21.15.103:0/2172729128 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fca0c0a33b0 0x7fca0c0a5870 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fca1400cc40 tx=0x7fca14019000 comp rx=0 tx=0).stop 2024-09-19T07:06:39.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.190+0000 7fca230ac640 1 -- 172.21.15.103:0/2172729128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca1c101420 msgr2=0x7fca1c199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:39.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.190+0000 7fca230ac640 1 --2- 172.21.15.103:0/2172729128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca1c101420 0x7fca1c199870 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fca08009230 tx=0x7fca08004270 comp rx=0 tx=0).stop 2024-09-19T07:06:39.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.190+0000 7fca230ac640 1 -- 172.21.15.103:0/2172729128 shutdown_connections 2024-09-19T07:06:39.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.190+0000 7fca230ac640 1 --2- 172.21.15.103:0/2172729128 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fca0c0a33b0 0x7fca0c0a5870 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.190+0000 7fca230ac640 1 --2- 172.21.15.103:0/2172729128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca1c106a80 0x7fca1c1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.190+0000 7fca230ac640 1 --2- 172.21.15.103:0/2172729128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca1c101420 0x7fca1c199870 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.190+0000 7fca230ac640 1 --2- 172.21.15.103:0/2172729128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca1c100a80 0x7fca1c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:39.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.190+0000 7fca230ac640 1 -- 172.21.15.103:0/2172729128 >> 172.21.15.103:0/2172729128 conn(0x7fca1c0fc7a0 msgr2=0x7fca1c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:39.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.190+0000 7fca230ac640 1 -- 172.21.15.103:0/2172729128 shutdown_connections 2024-09-19T07:06:39.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:39.190+0000 7fca230ac640 1 -- 172.21.15.103:0/2172729128 wait complete. 2024-09-19T07:06:39.899 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:06:39.899 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:06:40.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:39 smithi103 ceph-mon[31158]: pgmap v545: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 8 op/s 2024-09-19T07:06:40.149 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:40.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:39 smithi174 ceph-mon[35637]: pgmap v545: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 8 op/s 2024-09-19T07:06:40.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:39 smithi134 ceph-mon[35356]: pgmap v545: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 8 op/s 2024-09-19T07:06:40.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.619+0000 7ff5a3f24640 1 -- 172.21.15.103:0/3476055465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff59c100a80 msgr2=0x7ff59c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:40.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.619+0000 7ff5a3f24640 1 --2- 172.21.15.103:0/3476055465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff59c100a80 0x7ff59c100ee0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7ff590009cd0 tx=0x7ff59002a8e0 comp rx=0 tx=0).stop 2024-09-19T07:06:40.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.619+0000 7ff5a3f24640 1 -- 172.21.15.103:0/3476055465 shutdown_connections 2024-09-19T07:06:40.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.619+0000 7ff5a3f24640 1 --2- 172.21.15.103:0/3476055465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff59c101420 0x7ff59c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:40.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.619+0000 7ff5a3f24640 1 --2- 172.21.15.103:0/3476055465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff59c100a80 0x7ff59c100ee0 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:40.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.619+0000 7ff5a3f24640 1 --2- 172.21.15.103:0/3476055465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff59c106a80 0x7ff59c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:40.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.619+0000 7ff5a3f24640 1 -- 172.21.15.103:0/3476055465 >> 172.21.15.103:0/3476055465 conn(0x7ff59c0fc7a0 msgr2=0x7ff59c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:40.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.619+0000 7ff5a3f24640 1 -- 172.21.15.103:0/3476055465 shutdown_connections 2024-09-19T07:06:40.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.619+0000 7ff5a3f24640 1 -- 172.21.15.103:0/3476055465 wait complete. 2024-09-19T07:06:40.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.619+0000 7ff5a3f24640 1 Processor -- start 2024-09-19T07:06:40.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.619+0000 7ff5a3f24640 1 -- start start 2024-09-19T07:06:40.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a3f24640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff59c100a80 0x7ff59c1a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:40.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a3f24640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff59c101420 0x7ff59c1a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:40.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a3f24640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff59c106a80 0x7ff59c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:40.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a3f24640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c10fe10 con 0x7ff59c101420 2024-09-19T07:06:40.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a3f24640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c10fc90 con 0x7ff59c100a80 2024-09-19T07:06:40.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a3f24640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c10ff90 con 0x7ff59c106a80 2024-09-19T07:06:40.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a14bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff59c101420 0x7ff59c1a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:40.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a14bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff59c101420 0x7ff59c1a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55218/0 (socket says 172.21.15.103:55218) 2024-09-19T07:06:40.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a14bf640 1 -- 172.21.15.103:0/1094026720 learned_addr learned my addr 172.21.15.103:0/1094026720 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:40.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a1cc0640 1 --2- 172.21.15.103:0/1094026720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff59c100a80 0x7ff59c1a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:40.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a24c1640 1 --2- 172.21.15.103:0/1094026720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff59c106a80 0x7ff59c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:40.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a14bf640 1 -- 172.21.15.103:0/1094026720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff59c106a80 msgr2=0x7ff59c19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:40.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a14bf640 1 --2- 172.21.15.103:0/1094026720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff59c106a80 0x7ff59c19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:40.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a14bf640 1 -- 172.21.15.103:0/1094026720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff59c100a80 msgr2=0x7ff59c1a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:40.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a14bf640 1 --2- 172.21.15.103:0/1094026720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff59c100a80 0x7ff59c1a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:40.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a14bf640 1 -- 172.21.15.103:0/1094026720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59c19ad00 con 0x7ff59c101420 2024-09-19T07:06:40.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a14bf640 1 --2- 172.21.15.103:0/1094026720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff59c101420 0x7ff59c1a0950 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7ff59002ae00 tx=0x7ff590003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:40.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff59affd640 1 -- 172.21.15.103:0/1094026720 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff590004240 con 0x7ff59c101420 2024-09-19T07:06:40.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff59affd640 1 -- 172.21.15.103:0/1094026720 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5900305a0 con 0x7ff59c101420 2024-09-19T07:06:40.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff5a3f24640 1 -- 172.21.15.103:0/1094026720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff59c19af90 con 0x7ff59c101420 2024-09-19T07:06:40.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.620+0000 7ff59affd640 1 -- 172.21.15.103:0/1094026720 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59003a660 con 0x7ff59c101420 2024-09-19T07:06:40.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.621+0000 7ff5a3f24640 1 -- 172.21.15.103:0/1094026720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff59c1a72e0 con 0x7ff59c101420 2024-09-19T07:06:40.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.622+0000 7ff5a3f24640 1 -- 172.21.15.103:0/1094026720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff59c100f60 con 0x7ff59c101420 2024-09-19T07:06:40.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.623+0000 7ff59affd640 1 -- 172.21.15.103:0/1094026720 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff590030ac0 con 0x7ff59c101420 2024-09-19T07:06:40.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.623+0000 7ff59affd640 1 --2- 172.21.15.103:0/1094026720 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff5840a3430 0x7ff5840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:40.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.623+0000 7ff5a1cc0640 1 --2- 172.21.15.103:0/1094026720 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff5840a3430 0x7ff5840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:40.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.623+0000 7ff59affd640 1 -- 172.21.15.103:0/1094026720 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7ff59002e030 con 0x7ff59c101420 2024-09-19T07:06:40.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.624+0000 7ff5a1cc0640 1 --2- 172.21.15.103:0/1094026720 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff5840a3430 0x7ff5840a58f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff594009890 tx=0x7ff594008040 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:40.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.627+0000 7ff59affd640 1 -- 172.21.15.103:0/1094026720 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff5900098b0 con 0x7ff59c101420 2024-09-19T07:06:40.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.829+0000 7ff5a3f24640 1 -- 172.21.15.103:0/1094026720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff59c109070 con 0x7ff59c101420 2024-09-19T07:06:40.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.830+0000 7ff59affd640 1 -- 172.21.15.103:0/1094026720 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7ff5900bd9d0 con 0x7ff59c101420 2024-09-19T07:06:40.873 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:06:40.873 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:06:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:06:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.832+0000 7ff5a3f24640 1 -- 172.21.15.103:0/1094026720 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff5840a3430 msgr2=0x7ff5840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.832+0000 7ff5a3f24640 1 --2- 172.21.15.103:0/1094026720 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff5840a3430 0x7ff5840a58f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff594009890 tx=0x7ff594008040 comp rx=0 tx=0).stop 2024-09-19T07:06:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.832+0000 7ff5a3f24640 1 -- 172.21.15.103:0/1094026720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff59c101420 msgr2=0x7ff59c1a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.832+0000 7ff5a3f24640 1 --2- 172.21.15.103:0/1094026720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff59c101420 0x7ff59c1a0950 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7ff59002ae00 tx=0x7ff590003ba0 comp rx=0 tx=0).stop 2024-09-19T07:06:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.832+0000 7ff5a3f24640 1 -- 172.21.15.103:0/1094026720 shutdown_connections 2024-09-19T07:06:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.832+0000 7ff5a3f24640 1 --2- 172.21.15.103:0/1094026720 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff5840a3430 0x7ff5840a58f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.832+0000 7ff5a3f24640 1 --2- 172.21.15.103:0/1094026720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff59c106a80 0x7ff59c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.832+0000 7ff5a3f24640 1 --2- 172.21.15.103:0/1094026720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff59c101420 0x7ff59c1a0950 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.832+0000 7ff5a3f24640 1 --2- 172.21.15.103:0/1094026720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff59c100a80 0x7ff59c1a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.832+0000 7ff5a3f24640 1 -- 172.21.15.103:0/1094026720 >> 172.21.15.103:0/1094026720 conn(0x7ff59c0fc7a0 msgr2=0x7ff59c0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:40.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.832+0000 7ff5a3f24640 1 -- 172.21.15.103:0/1094026720 shutdown_connections 2024-09-19T07:06:40.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:40.832+0000 7ff5a3f24640 1 -- 172.21.15.103:0/1094026720 wait complete. 2024-09-19T07:06:41.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:40 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1094026720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:06:41.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:41 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1094026720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:06:41.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:40 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1094026720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:06:41.458 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:06:41.706 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:42.083 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:42 smithi103 ceph-mon[31158]: pgmap v546: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 12 op/s 2024-09-19T07:06:42.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.165+0000 7f9eac818640 1 -- 172.21.15.103:0/1322249128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ea8100aa0 msgr2=0x7f9ea810ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:42.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.165+0000 7f9eac818640 1 --2- 172.21.15.103:0/1322249128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ea8100aa0 0x7f9ea810ea20 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009900 tx=0x7f9ea002a410 comp rx=0 tx=0).stop 2024-09-19T07:06:42.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.165+0000 7f9eac818640 1 -- 172.21.15.103:0/1322249128 shutdown_connections 2024-09-19T07:06:42.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.165+0000 7f9eac818640 1 --2- 172.21.15.103:0/1322249128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ea8100aa0 0x7f9ea810ea20 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.165+0000 7f9eac818640 1 --2- 172.21.15.103:0/1322249128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ea8100100 0x7f9ea8100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.165+0000 7f9eac818640 1 --2- 172.21.15.103:0/1322249128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ea8106b80 0x7f9ea8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.165+0000 7f9eac818640 1 -- 172.21.15.103:0/1322249128 >> 172.21.15.103:0/1322249128 conn(0x7f9ea8077230 msgr2=0x7f9ea80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:42.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.165+0000 7f9eac818640 1 -- 172.21.15.103:0/1322249128 shutdown_connections 2024-09-19T07:06:42.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.165+0000 7f9eac818640 1 -- 172.21.15.103:0/1322249128 wait complete. 2024-09-19T07:06:42.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.165+0000 7f9eac818640 1 Processor -- start 2024-09-19T07:06:42.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.165+0000 7f9eac818640 1 -- start start 2024-09-19T07:06:42.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9eac818640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ea8100100 0x7f9ea81012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9eac818640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ea8100aa0 0x7f9ea8101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9eac818640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ea8106b80 0x7f9ea8101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9eac818640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea8112050 con 0x7f9ea8100100 2024-09-19T07:06:42.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9eac818640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea8111ed0 con 0x7f9ea8100aa0 2024-09-19T07:06:42.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9eac818640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81121d0 con 0x7f9ea8106b80 2024-09-19T07:06:42.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9ea659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ea8100100 0x7f9ea81012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:42.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9ea659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ea8100100 0x7f9ea81012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55240/0 (socket says 172.21.15.103:55240) 2024-09-19T07:06:42.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9ea659c640 1 -- 172.21.15.103:0/3926649179 learned_addr learned my addr 172.21.15.103:0/3926649179 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:42.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9ea5d9b640 1 --2- 172.21.15.103:0/3926649179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ea8100aa0 0x7f9ea8101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:42.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9ea6d9d640 1 --2- 172.21.15.103:0/3926649179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ea8106b80 0x7f9ea8101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:42.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9ea659c640 1 -- 172.21.15.103:0/3926649179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ea8106b80 msgr2=0x7f9ea8101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9ea659c640 1 --2- 172.21.15.103:0/3926649179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ea8106b80 0x7f9ea8101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9ea659c640 1 -- 172.21.15.103:0/3926649179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ea8100aa0 msgr2=0x7f9ea8101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9ea659c640 1 --2- 172.21.15.103:0/3926649179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ea8100aa0 0x7f9ea8101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9ea659c640 1 -- 172.21.15.103:0/3926649179 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea81a68b0 con 0x7f9ea8100100 2024-09-19T07:06:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9ea659c640 1 --2- 172.21.15.103:0/3926649179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ea8100100 0x7f9ea81012e0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800cce0 tx=0x7f9e9800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9e9f7fe640 1 -- 172.21.15.103:0/3926649179 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9800f650 con 0x7f9ea8100100 2024-09-19T07:06:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9eac818640 1 -- 172.21.15.103:0/3926649179 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea81a6ba0 con 0x7f9ea8100100 2024-09-19T07:06:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9e9f7fe640 1 -- 172.21.15.103:0/3926649179 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9800fcd0 con 0x7f9ea8100100 2024-09-19T07:06:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.166+0000 7f9e9f7fe640 1 -- 172.21.15.103:0/3926649179 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98018c70 con 0x7f9ea8100100 2024-09-19T07:06:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.167+0000 7f9eac818640 1 -- 172.21.15.103:0/3926649179 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea81a70e0 con 0x7f9ea8100100 2024-09-19T07:06:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.169+0000 7f9e9f7fe640 1 -- 172.21.15.103:0/3926649179 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9e9800f7f0 con 0x7f9ea8100100 2024-09-19T07:06:42.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.169+0000 7f9eaebf1740 1 Processor -- start 2024-09-19T07:06:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.169+0000 7f9eaebf1740 1 -- start start 2024-09-19T07:06:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9eaebf1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de88294b40 0x55de88293090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9eaebf1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de88293a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9eaebf1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de8829e370 0x55de882a0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9eaebf1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55de881f92f0 con 0x55de882935d0 2024-09-19T07:06:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9eaebf1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55de881a0cf0 con 0x55de88294b40 2024-09-19T07:06:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9eaebf1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55de881f52b0 con 0x55de8829e370 2024-09-19T07:06:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9ea659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de88293a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9ea659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de88293a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55242/0 (socket says 172.21.15.103:55242) 2024-09-19T07:06:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9ea659c640 1 -- 172.21.15.103:0/477294288 learned_addr learned my addr 172.21.15.103:0/477294288 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9ea5d9b640 1 --2- 172.21.15.103:0/477294288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de8829e370 0x55de882a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9ea6d9d640 1 --2- 172.21.15.103:0/477294288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de88294b40 0x55de88293090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9e9f7fe640 1 --2- 172.21.15.103:0/3926649179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9e940a33b0 0x7f9e940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9ea659c640 1 -- 172.21.15.103:0/477294288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de8829e370 msgr2=0x55de882a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:42.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9ea659c640 1 --2- 172.21.15.103:0/477294288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de8829e370 0x55de882a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9ea659c640 1 -- 172.21.15.103:0/477294288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de88294b40 msgr2=0x55de88293090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:42.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9ea659c640 1 --2- 172.21.15.103:0/477294288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de88294b40 0x55de88293090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9ea659c640 1 -- 172.21.15.103:0/477294288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de881853a0 con 0x55de882935d0 2024-09-19T07:06:42.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9ea6d9d640 1 --2- 172.21.15.103:0/3926649179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9e940a33b0 0x7f9e940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:42.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9e9f7fe640 1 -- 172.21.15.103:0/3926649179 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f9e980d02a0 con 0x7f9ea8100100 2024-09-19T07:06:42.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.170+0000 7f9ea659c640 1 --2- 172.21.15.103:0/477294288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de88293a10 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f9e980cd8b0 tx=0x7f9e980c72d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aeb5408b40ada0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:42.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.171+0000 7f9e9cff9640 1 -- 172.21.15.103:0/477294288 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9809c040 con 0x55de882935d0 2024-09-19T07:06:42.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.171+0000 7f9e9cff9640 1 -- 172.21.15.103:0/477294288 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9809cc00 con 0x55de882935d0 2024-09-19T07:06:42.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.171+0000 7f9e9cff9640 1 -- 172.21.15.103:0/477294288 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e980d3a60 con 0x55de882935d0 2024-09-19T07:06:42.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.171+0000 7f9eaebf1740 1 -- 172.21.15.103:0/477294288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 msgr2=0x55de88293a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:42.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.171+0000 7f9ea6d9d640 1 --2- 172.21.15.103:0/3926649179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9e940a33b0 0x7f9e940a5870 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009230 tx=0x7f9ea0009260 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:42.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.171+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/477294288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de88293a10 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f9e980cd8b0 tx=0x7f9e980c72d0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.171+0000 7f9eaebf1740 1 -- 172.21.15.103:0/477294288 shutdown_connections 2024-09-19T07:06:42.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.171+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/477294288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de8829e370 0x55de882a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.171+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/477294288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de88293a10 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.171+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/477294288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de88294b40 0x55de88293090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.171+0000 7f9eaebf1740 1 -- 172.21.15.103:0/477294288 >> 172.21.15.103:0/477294288 conn(0x55de8825c310 msgr2=0x55de8828fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:42.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9eaebf1740 1 -- 172.21.15.103:0/477294288 shutdown_connections 2024-09-19T07:06:42.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9eaebf1740 1 -- 172.21.15.103:0/477294288 wait complete. 2024-09-19T07:06:42.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9eaebf1740 1 -- start start 2024-09-19T07:06:42.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9eaebf1740 1 Processor -- start 2024-09-19T07:06:42.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9eaebf1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de8829dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9eaebf1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de88294b40 0x55de8829afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9eaebf1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de8829e370 0x55de8829b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9eaebf1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55de882a1010 con 0x55de882935d0 2024-09-19T07:06:42.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9eaebf1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55de881b4060 con 0x55de88294b40 2024-09-19T07:06:42.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9eaebf1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55de882a1190 con 0x55de8829e370 2024-09-19T07:06:42.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9eaebf1740 10 client.0 _ll_register_callbacks cb 0x55de87f078c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:42.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9ea659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de8829dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:42.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9ea659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de8829dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55248/0 (socket says 172.21.15.103:55248) 2024-09-19T07:06:42.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9ea659c640 1 -- 172.21.15.103:0/1524790185 learned_addr learned my addr 172.21.15.103:0/1524790185 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:42.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9ea659c640 1 -- 172.21.15.103:0/1524790185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de8829e370 msgr2=0x55de8829b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:42.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9ea659c640 1 --2- 172.21.15.103:0/1524790185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de8829e370 0x55de8829b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9ea659c640 1 -- 172.21.15.103:0/1524790185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de88294b40 msgr2=0x55de8829afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:42.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9ea659c640 1 --2- 172.21.15.103:0/1524790185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de88294b40 0x55de8829afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.172+0000 7f9ea659c640 1 -- 172.21.15.103:0/1524790185 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de87f6a3c0 con 0x55de882935d0 2024-09-19T07:06:42.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.173+0000 7f9ea659c640 1 --2- 172.21.15.103:0/1524790185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de8829dd20 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f9e9809cdb0 tx=0x7f9e980d0520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7ac475a406803 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:42.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.173+0000 7f9e8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:42.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.173+0000 7f9e8dffb640 1 -- 172.21.15.103:0/1524790185 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e980648f0 con 0x55de882935d0 2024-09-19T07:06:42.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.173+0000 7f9e8dffb640 1 -- 172.21.15.103:0/1524790185 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f9e980d3a60 con 0x55de882935d0 2024-09-19T07:06:42.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.173+0000 7f9eaebf1740 1 -- 172.21.15.103:0/1524790185 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e9809cc00 con 0x55de882935d0 2024-09-19T07:06:42.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.173+0000 7f9e8dffb640 1 -- 172.21.15.103:0/1524790185 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9e9809be60 con 0x55de882935d0 2024-09-19T07:06:42.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.173+0000 7f9eaebf1740 10 client.21579 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:42.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.173+0000 7f9eaebf1740 1 -- 172.21.15.103:0/1524790185 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55de882aca60 con 0x55de882935d0 2024-09-19T07:06:42.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.173+0000 7f9e8dffb640 1 -- 172.21.15.103:0/1524790185 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9809cc00 con 0x55de882935d0 2024-09-19T07:06:42.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.173+0000 7f9e8dffb640 1 -- 172.21.15.103:0/1524790185 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9e980cdbf0 con 0x55de882935d0 2024-09-19T07:06:42.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.173+0000 7f9eaebf1740 10 client.21579 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:42.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.173+0000 7f9eaebf1740 10 client.21579 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:06:42.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.173+0000 7f9eaebf1740 20 client.21579 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:42.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.173+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/1524790185 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55de882addb0 0x55de882b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.173+0000 7f9eaebf1740 4 client.21579 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:06:42.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.173+0000 7f9eaebf1740 1 -- 172.21.15.103:0/1524790185 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de88206860 con 0x55de882addb0 2024-09-19T07:06:42.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.174+0000 7f9ea5d9b640 1 --2- 172.21.15.103:0/1524790185 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55de882addb0 0x55de882b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:42.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.174+0000 7f9e8dffb640 10 client.21579 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:06:42.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.174+0000 7f9ea5d9b640 1 --2- 172.21.15.103:0/1524790185 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55de882addb0 0x55de882b01a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7981dc3a933d5d4a server_cookie=c8ef001c6227fc56 in_seq=0 out_seq=0 2024-09-19T07:06:42.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.175+0000 7f9e8dffb640 1 -- 172.21.15.103:0/1524790185 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de88206860 con 0x55de882addb0 2024-09-19T07:06:42.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.175+0000 7f9e8dffb640 10 client.21579 handle_command_reply: tid=0 2024-09-19T07:06:42.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.175+0000 7f9eaebf1740 1 client.21579 shutdown 2024-09-19T07:06:42.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.175+0000 7f9eaebf1740 1 -- 172.21.15.103:0/1524790185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 msgr2=0x55de8829dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:42.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.175+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/1524790185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de8829dd20 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f9e9809cdb0 tx=0x7f9e980d0520 comp rx=0 tx=0).stop 2024-09-19T07:06:42.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.175+0000 7f9eaebf1740 1 -- 172.21.15.103:0/1524790185 shutdown_connections 2024-09-19T07:06:42.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.175+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/1524790185 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55de882addb0 0x55de882b01a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.175+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/1524790185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de8829e370 0x55de8829b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.175+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/1524790185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de88294b40 0x55de8829afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.175+0000 7f9e8dffb640 0 client.21579 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:06:42.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.175+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/1524790185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de8829dd20 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.175+0000 7f9eaebf1740 1 -- 172.21.15.103:0/1524790185 >> 172.21.15.103:0/1524790185 conn(0x55de8825c310 msgr2=0x55de88295990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:42.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.175+0000 7f9eaebf1740 1 -- 172.21.15.103:0/1524790185 shutdown_connections 2024-09-19T07:06:42.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.175+0000 7f9eaebf1740 1 -- 172.21.15.103:0/1524790185 wait complete. 2024-09-19T07:06:42.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.175+0000 7f9eaebf1740 20 client.21579 trim_cache size 0 max 16384 2024-09-19T07:06:42.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.188+0000 7f9eaebf1740 1 Processor -- start 2024-09-19T07:06:42.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.188+0000 7f9eaebf1740 1 -- start start 2024-09-19T07:06:42.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.188+0000 7f9eaebf1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de882935d0 0x55de88296120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.188+0000 7f9eaebf1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de88294b40 0x55de88296660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.188+0000 7f9eaebf1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de8829e370 0x55de882b5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.188+0000 7f9eaebf1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81afb00 con 0x55de8829e370 2024-09-19T07:06:42.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.188+0000 7f9eaebf1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81b1640 con 0x55de882935d0 2024-09-19T07:06:42.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.188+0000 7f9eaebf1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81b01d0 con 0x55de88294b40 2024-09-19T07:06:42.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.188+0000 7f9ea6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de8829e370 0x55de882b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:42.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.188+0000 7f9ea6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de8829e370 0x55de882b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55254/0 (socket says 172.21.15.103:55254) 2024-09-19T07:06:42.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.188+0000 7f9ea6d9d640 1 -- 172.21.15.103:0/3757953269 learned_addr learned my addr 172.21.15.103:0/3757953269 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:42.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.188+0000 7f9ea5d9b640 1 --2- 172.21.15.103:0/3757953269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de88294b40 0x55de88296660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:42.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.188+0000 7f9ea6d9d640 1 -- 172.21.15.103:0/3757953269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de88294b40 msgr2=0x55de88296660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:42.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.188+0000 7f9ea6d9d640 1 --2- 172.21.15.103:0/3757953269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de88294b40 0x55de88296660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.188+0000 7f9ea6d9d640 1 -- 172.21.15.103:0/3757953269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de882935d0 msgr2=0x55de88296120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:42.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.188+0000 7f9ea6d9d640 1 --2- 172.21.15.103:0/3757953269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de882935d0 0x55de88296120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.189+0000 7f9ea6d9d640 1 -- 172.21.15.103:0/3757953269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de88299520 con 0x55de8829e370 2024-09-19T07:06:42.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.189+0000 7f9ea6d9d640 1 --2- 172.21.15.103:0/3757953269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de8829e370 0x55de882b5740 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f9ea81077c0 tx=0x7f9ea002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3e355fd34f64ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:42.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.189+0000 7f9e8ffff640 1 -- 172.21.15.103:0/3757953269 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0035040 con 0x55de8829e370 2024-09-19T07:06:42.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.189+0000 7f9e8ffff640 1 -- 172.21.15.103:0/3757953269 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea004e070 con 0x55de8829e370 2024-09-19T07:06:42.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.189+0000 7f9e8ffff640 1 -- 172.21.15.103:0/3757953269 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0004e30 con 0x55de8829e370 2024-09-19T07:06:42.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.189+0000 7f9eaebf1740 1 -- 172.21.15.103:0/3757953269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de8829e370 msgr2=0x55de882b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:42.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.189+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/3757953269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de8829e370 0x55de882b5740 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f9ea81077c0 tx=0x7f9ea002a410 comp rx=0 tx=0).stop 2024-09-19T07:06:42.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.189+0000 7f9eaebf1740 1 -- 172.21.15.103:0/3757953269 shutdown_connections 2024-09-19T07:06:42.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.189+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/3757953269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de8829e370 0x55de882b5740 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.189+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/3757953269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de88294b40 0x55de88296660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.189+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/3757953269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de882935d0 0x55de88296120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.190+0000 7f9eaebf1740 1 -- 172.21.15.103:0/3757953269 >> 172.21.15.103:0/3757953269 conn(0x55de8825c310 msgr2=0x55de8828ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:42.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.190+0000 7f9eaebf1740 1 -- 172.21.15.103:0/3757953269 shutdown_connections 2024-09-19T07:06:42.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.190+0000 7f9eaebf1740 1 -- 172.21.15.103:0/3757953269 wait complete. 2024-09-19T07:06:42.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.190+0000 7f9eaebf1740 1 -- start start 2024-09-19T07:06:42.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.190+0000 7f9eaebf1740 1 Processor -- start 2024-09-19T07:06:42.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.190+0000 7f9eaebf1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de882a6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.190+0000 7f9eaebf1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de88294b40 0x55de882a6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.190+0000 7f9eaebf1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de8829e370 0x55de882a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.190+0000 7f9eaebf1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81b0f70 con 0x55de882935d0 2024-09-19T07:06:42.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.190+0000 7f9eaebf1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81b08a0 con 0x55de88294b40 2024-09-19T07:06:42.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.190+0000 7f9eaebf1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81adb00 con 0x55de8829e370 2024-09-19T07:06:42.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.190+0000 7f9eaebf1740 10 client.0 _ll_register_callbacks cb 0x7f9ea0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:42.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.190+0000 7f9ea659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de882a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:42.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.190+0000 7f9ea659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de882a6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55258/0 (socket says 172.21.15.103:55258) 2024-09-19T07:06:42.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.190+0000 7f9ea659c640 1 -- 172.21.15.103:0/1227889906 learned_addr learned my addr 172.21.15.103:0/1227889906 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:42.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.190+0000 7f9ea5d9b640 1 --2- 172.21.15.103:0/1227889906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de88294b40 0x55de882a6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:42.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9ea6d9d640 1 --2- 172.21.15.103:0/1227889906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de8829e370 0x55de882a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:42.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9ea659c640 1 -- 172.21.15.103:0/1227889906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de8829e370 msgr2=0x55de882a33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:42.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9ea659c640 1 --2- 172.21.15.103:0/1227889906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de8829e370 0x55de882a33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9ea659c640 1 -- 172.21.15.103:0/1227889906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de88294b40 msgr2=0x55de882a6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:42.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9ea659c640 1 --2- 172.21.15.103:0/1227889906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de88294b40 0x55de882a6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9ea659c640 1 -- 172.21.15.103:0/1227889906 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de882a3b30 con 0x55de882935d0 2024-09-19T07:06:42.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9ea6d9d640 1 --2- 172.21.15.103:0/1227889906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de8829e370 0x55de882a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:42.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9ea5d9b640 1 --2- 172.21.15.103:0/1227889906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de88294b40 0x55de882a6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:06:42.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9ea659c640 1 --2- 172.21.15.103:0/1227889906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de882a6600 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f9e980985a0 tx=0x7f9e980b08f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e8e9307aab06df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:42.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9e8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:42.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9e8dffb640 1 -- 172.21.15.103:0/1227889906 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9809c040 con 0x55de882935d0 2024-09-19T07:06:42.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9e8dffb640 1 -- 172.21.15.103:0/1227889906 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f9e9809c960 con 0x55de882935d0 2024-09-19T07:06:42.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9eaebf1740 1 -- 172.21.15.103:0/1227889906 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea004e070 con 0x55de882935d0 2024-09-19T07:06:42.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9e8dffb640 1 -- 172.21.15.103:0/1227889906 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9e9809cc00 con 0x55de882935d0 2024-09-19T07:06:42.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9eaebf1740 10 client.21585 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:42.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9eaebf1740 1 -- 172.21.15.103:0/1227889906 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55de88319120 con 0x55de882935d0 2024-09-19T07:06:42.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9e8dffb640 1 -- 172.21.15.103:0/1227889906 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea004e070 con 0x55de882935d0 2024-09-19T07:06:42.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.191+0000 7f9e8dffb640 1 -- 172.21.15.103:0/1227889906 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9e980b52e0 con 0x55de882935d0 2024-09-19T07:06:42.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.192+0000 7f9eaebf1740 10 client.21585 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:42.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.192+0000 7f9eaebf1740 10 client.21585 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-19T07:06:42.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.192+0000 7f9eaebf1740 20 client.21585 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:42.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.192+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/1227889906 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55de88319650 0x55de8831ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:42.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.192+0000 7f9eaebf1740 4 client.21585 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:06:42.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.192+0000 7f9eaebf1740 1 -- 172.21.15.103:0/1227889906 --> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de88200d70 con 0x55de88319650 2024-09-19T07:06:42.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.192+0000 7f9ea5d9b640 1 --2- 172.21.15.103:0/1227889906 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55de88319650 0x55de8831ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:42.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.192+0000 7f9e8dffb640 10 client.21585 ms_handle_connect on v2:172.21.15.134:6834/3092414367 2024-09-19T07:06:42.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.192+0000 7f9ea5d9b640 1 --2- 172.21.15.103:0/1227889906 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55de88319650 0x55de8831ba40 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2a776c9f631bb377 server_cookie=a673b382d2506326 in_seq=0 out_seq=0 2024-09-19T07:06:42.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.192+0000 7f9e8dffb640 1 -- 172.21.15.103:0/1227889906 <== mds.4 v2:172.21.15.134:6834/3092414367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de88200d70 con 0x55de88319650 2024-09-19T07:06:42.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.192+0000 7f9e8dffb640 10 client.21585 handle_command_reply: tid=0 2024-09-19T07:06:42.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.192+0000 7f9eaebf1740 1 client.21585 shutdown 2024-09-19T07:06:42.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eaebf1740 1 -- 172.21.15.103:0/1227889906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 msgr2=0x55de882a6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:42.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/1227889906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de882a6600 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f9e980985a0 tx=0x7f9e980b08f0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eaebf1740 1 -- 172.21.15.103:0/1227889906 shutdown_connections 2024-09-19T07:06:42.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/1227889906 >> [v2:172.21.15.134:6834/3092414367,v1:172.21.15.134:6835/3092414367] conn(0x55de88319650 0x55de8831ba40 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/1227889906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55de8829e370 0x55de882a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/1227889906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de88294b40 0x55de882a6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eaebf1740 1 --2- 172.21.15.103:0/1227889906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de882935d0 0x55de882a6600 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eaebf1740 1 -- 172.21.15.103:0/1227889906 >> 172.21.15.103:0/1227889906 conn(0x55de8825c310 msgr2=0x55de8829c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:42.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9e8dffb640 0 client.21585 ms_handle_reset on v2:172.21.15.134:6834/3092414367 2024-09-19T07:06:42.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eaebf1740 1 -- 172.21.15.103:0/1227889906 shutdown_connections 2024-09-19T07:06:42.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eaebf1740 1 -- 172.21.15.103:0/1227889906 wait complete. 2024-09-19T07:06:42.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eaebf1740 20 client.21585 trim_cache size 0 max 16384 2024-09-19T07:06:42.206 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:06:42.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eac818640 1 -- 172.21.15.103:0/3926649179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9e940a33b0 msgr2=0x7f9e940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:42.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eac818640 1 --2- 172.21.15.103:0/3926649179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9e940a33b0 0x7f9e940a5870 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009230 tx=0x7f9ea0009260 comp rx=0 tx=0).stop 2024-09-19T07:06:42.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eac818640 1 -- 172.21.15.103:0/3926649179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ea8100100 msgr2=0x7f9ea81012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:42.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eac818640 1 --2- 172.21.15.103:0/3926649179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ea8100100 0x7f9ea81012e0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800cce0 tx=0x7f9e9800a620 comp rx=0 tx=0).stop 2024-09-19T07:06:42.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eac818640 1 -- 172.21.15.103:0/3926649179 shutdown_connections 2024-09-19T07:06:42.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eac818640 1 --2- 172.21.15.103:0/3926649179 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9e940a33b0 0x7f9e940a5870 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eac818640 1 --2- 172.21.15.103:0/3926649179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ea8106b80 0x7f9ea8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eac818640 1 --2- 172.21.15.103:0/3926649179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ea8100aa0 0x7f9ea8101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eac818640 1 --2- 172.21.15.103:0/3926649179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ea8100100 0x7f9ea81012e0 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.193+0000 7f9eac818640 1 -- 172.21.15.103:0/3926649179 >> 172.21.15.103:0/3926649179 conn(0x7f9ea8077230 msgr2=0x7f9ea8104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.194+0000 7f9eac818640 1 -- 172.21.15.103:0/3926649179 shutdown_connections 2024-09-19T07:06:42.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:42.194+0000 7f9eac818640 1 -- 172.21.15.103:0/3926649179 wait complete. 2024-09-19T07:06:42.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:42 smithi174 ceph-mon[35637]: pgmap v546: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 12 op/s 2024-09-19T07:06:42.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:42 smithi134 ceph-mon[35356]: pgmap v546: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 12 op/s 2024-09-19T07:06:42.806 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:06:43.057 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:43.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.513+0000 7fd25d32e640 1 -- 172.21.15.103:0/4236147800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd258100a50 msgr2=0x7fd2581057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:43.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.513+0000 7fd25d32e640 1 --2- 172.21.15.103:0/4236147800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd258100a50 0x7fd2581057e0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fd250009ca0 tx=0x7fd25002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.513+0000 7fd25d32e640 1 -- 172.21.15.103:0/4236147800 shutdown_connections 2024-09-19T07:06:43.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.513+0000 7fd25d32e640 1 --2- 172.21.15.103:0/4236147800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd258100a50 0x7fd2581057e0 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.513+0000 7fd25d32e640 1 --2- 172.21.15.103:0/4236147800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd258100090 0x7fd258100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.513+0000 7fd25d32e640 1 --2- 172.21.15.103:0/4236147800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd258106b70 0x7fd258106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.513+0000 7fd25d32e640 1 -- 172.21.15.103:0/4236147800 >> 172.21.15.103:0/4236147800 conn(0x7fd258077210 msgr2=0x7fd2580fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:43.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.513+0000 7fd25d32e640 1 -- 172.21.15.103:0/4236147800 shutdown_connections 2024-09-19T07:06:43.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.513+0000 7fd25d32e640 1 -- 172.21.15.103:0/4236147800 wait complete. 2024-09-19T07:06:43.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.513+0000 7fd25d32e640 1 Processor -- start 2024-09-19T07:06:43.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.513+0000 7fd25d32e640 1 -- start start 2024-09-19T07:06:43.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd25d32e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd258100090 0x7fd2581a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd25d32e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd258100a50 0x7fd2581a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd25d32e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd258106b70 0x7fd25819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd25d32e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581185c0 con 0x7fd258106b70 2024-09-19T07:06:43.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd25d32e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd258118440 con 0x7fd258100a50 2024-09-19T07:06:43.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd25d32e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd258118740 con 0x7fd258100090 2024-09-19T07:06:43.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd2577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd258106b70 0x7fd25819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:43.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd2577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd258106b70 0x7fd25819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55290/0 (socket says 172.21.15.103:55290) 2024-09-19T07:06:43.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd2577fe640 1 -- 172.21.15.103:0/4285397952 learned_addr learned my addr 172.21.15.103:0/4285397952 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:43.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd256ffd640 1 --2- 172.21.15.103:0/4285397952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd258100090 0x7fd2581a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:43.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd24ffff640 1 --2- 172.21.15.103:0/4285397952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd258100a50 0x7fd2581a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:43.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd2577fe640 1 -- 172.21.15.103:0/4285397952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd258100090 msgr2=0x7fd2581a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:43.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd2577fe640 1 --2- 172.21.15.103:0/4285397952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd258100090 0x7fd2581a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd2577fe640 1 -- 172.21.15.103:0/4285397952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd258100a50 msgr2=0x7fd2581a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:43.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd2577fe640 1 --2- 172.21.15.103:0/4285397952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd258100a50 0x7fd2581a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd2577fe640 1 -- 172.21.15.103:0/4285397952 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25819efd0 con 0x7fd258106b70 2024-09-19T07:06:43.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd2577fe640 1 --2- 172.21.15.103:0/4285397952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd258106b70 0x7fd25819e830 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7fd250030c60 tx=0x7fd250030c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:43.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd254ff9640 1 -- 172.21.15.103:0/4285397952 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250003b70 con 0x7fd258106b70 2024-09-19T07:06:43.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.514+0000 7fd254ff9640 1 -- 172.21.15.103:0/4285397952 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd250031930 con 0x7fd258106b70 2024-09-19T07:06:43.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.515+0000 7fd25d32e640 1 -- 172.21.15.103:0/4285397952 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25819f260 con 0x7fd258106b70 2024-09-19T07:06:43.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.515+0000 7fd254ff9640 1 -- 172.21.15.103:0/4285397952 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2500047d0 con 0x7fd258106b70 2024-09-19T07:06:43.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.515+0000 7fd25d32e640 1 -- 172.21.15.103:0/4285397952 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2581ab510 con 0x7fd258106b70 2024-09-19T07:06:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.517+0000 7fd254ff9640 1 -- 172.21.15.103:0/4285397952 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd250031450 con 0x7fd258106b70 2024-09-19T07:06:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.517+0000 7fd25f707740 1 Processor -- start 2024-09-19T07:06:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.517+0000 7fd25f707740 1 -- start start 2024-09-19T07:06:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd25f707740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4d9eb40 0x55c1a4d9d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd25f707740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4d9da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd25f707740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4da8370 0x55c1a4daa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd25f707740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c1a4d032f0 con 0x55c1a4d9d5d0 2024-09-19T07:06:43.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd25f707740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c1a4caacf0 con 0x55c1a4d9eb40 2024-09-19T07:06:43.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd25f707740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c1a4cff2b0 con 0x55c1a4da8370 2024-09-19T07:06:43.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd254ff9640 1 --2- 172.21.15.103:0/4285397952 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2440a3430 0x7fd2440a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd256ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4d9da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:43.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd2577fe640 1 --2- 172.21.15.103:0/4285397952 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2440a3430 0x7fd2440a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:43.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd256ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4d9da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55304/0 (socket says 172.21.15.103:55304) 2024-09-19T07:06:43.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd256ffd640 1 -- 172.21.15.103:0/4215498675 learned_addr learned my addr 172.21.15.103:0/4215498675 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:43.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd24ffff640 1 --2- 172.21.15.103:0/4215498675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4da8370 0x55c1a4daa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:43.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd254ff9640 1 -- 172.21.15.103:0/4285397952 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7fd2500f10b0 con 0x7fd258106b70 2024-09-19T07:06:43.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd24ffff640 1 --2- 172.21.15.103:0/4215498675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4d9eb40 0x55c1a4d9d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:43.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd256ffd640 1 -- 172.21.15.103:0/4215498675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4da8370 msgr2=0x55c1a4daa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:43.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd256ffd640 1 --2- 172.21.15.103:0/4215498675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4da8370 0x55c1a4daa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd256ffd640 1 -- 172.21.15.103:0/4215498675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4d9eb40 msgr2=0x55c1a4d9d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:43.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd256ffd640 1 --2- 172.21.15.103:0/4215498675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4d9eb40 0x55c1a4d9d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd256ffd640 1 -- 172.21.15.103:0/4215498675 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1a4c8f3a0 con 0x55c1a4d9d5d0 2024-09-19T07:06:43.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd2577fe640 1 --2- 172.21.15.103:0/4285397952 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2440a3430 0x7fd2440a58f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd2500bd120 tx=0x7fd2500b28b0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:43.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.518+0000 7fd256ffd640 1 --2- 172.21.15.103:0/4215498675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4d9da10 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fd24800b820 tx=0x7fd24800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10cd1cc55103b47a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:43.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.519+0000 7fd24d7fa640 1 -- 172.21.15.103:0/4215498675 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24800d8d0 con 0x55c1a4d9d5d0 2024-09-19T07:06:43.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.519+0000 7fd24d7fa640 1 -- 172.21.15.103:0/4215498675 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd248019070 con 0x55c1a4d9d5d0 2024-09-19T07:06:43.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.519+0000 7fd24d7fa640 1 -- 172.21.15.103:0/4215498675 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2480123e0 con 0x55c1a4d9d5d0 2024-09-19T07:06:43.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.519+0000 7fd25f707740 1 -- 172.21.15.103:0/4215498675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4d9d5d0 msgr2=0x55c1a4d9da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:43.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.519+0000 7fd25f707740 1 --2- 172.21.15.103:0/4215498675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4d9da10 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fd24800b820 tx=0x7fd24800bce0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.519+0000 7fd25f707740 1 -- 172.21.15.103:0/4215498675 shutdown_connections 2024-09-19T07:06:43.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.519+0000 7fd25f707740 1 --2- 172.21.15.103:0/4215498675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4da8370 0x55c1a4daa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.519+0000 7fd25f707740 1 --2- 172.21.15.103:0/4215498675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4d9da10 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.519+0000 7fd25f707740 1 --2- 172.21.15.103:0/4215498675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4d9eb40 0x55c1a4d9d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.519+0000 7fd25f707740 1 -- 172.21.15.103:0/4215498675 >> 172.21.15.103:0/4215498675 conn(0x55c1a4d66310 msgr2=0x55c1a4d99c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:43.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.519+0000 7fd25f707740 1 -- 172.21.15.103:0/4215498675 shutdown_connections 2024-09-19T07:06:43.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.519+0000 7fd25f707740 1 -- 172.21.15.103:0/4215498675 wait complete. 2024-09-19T07:06:43.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd25f707740 1 -- start start 2024-09-19T07:06:43.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd25f707740 1 Processor -- start 2024-09-19T07:06:43.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd25f707740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4da7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd25f707740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4d9eb40 0x55c1a4da4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd25f707740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4da8370 0x55c1a4da54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd25f707740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c1a4dab070 con 0x55c1a4da8370 2024-09-19T07:06:43.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd25f707740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c1a4cbe060 con 0x55c1a4d9d5d0 2024-09-19T07:06:43.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd25f707740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c1a4dab1f0 con 0x55c1a4d9eb40 2024-09-19T07:06:43.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd25f707740 10 client.0 _ll_register_callbacks cb 0x55c1a4a118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:43.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd24ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4da8370 0x55c1a4da54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:43.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd24ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4da8370 0x55c1a4da54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55312/0 (socket says 172.21.15.103:55312) 2024-09-19T07:06:43.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd24ffff640 1 -- 172.21.15.103:0/1868598494 learned_addr learned my addr 172.21.15.103:0/1868598494 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:43.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd24ffff640 1 -- 172.21.15.103:0/1868598494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4d9eb40 msgr2=0x55c1a4da4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:43.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd256ffd640 1 --2- 172.21.15.103:0/1868598494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4d9eb40 0x55c1a4da4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:43.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd24ffff640 1 --2- 172.21.15.103:0/1868598494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4d9eb40 0x55c1a4da4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd24ffff640 1 -- 172.21.15.103:0/1868598494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4d9d5d0 msgr2=0x55c1a4da7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:43.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd24ffff640 1 --2- 172.21.15.103:0/1868598494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4da7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.520+0000 7fd24ffff640 1 -- 172.21.15.103:0/1868598494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1a4a743c0 con 0x55c1a4da8370 2024-09-19T07:06:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.521+0000 7fd24ffff640 1 --2- 172.21.15.103:0/1868598494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4da8370 0x55c1a4da54f0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fd240011e60 tx=0x7fd240010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819f71c40f507f29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.521+0000 7fd23e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.521+0000 7fd23e7fc640 1 -- 172.21.15.103:0/1868598494 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240005220 con 0x55c1a4da8370 2024-09-19T07:06:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.521+0000 7fd23e7fc640 1 -- 172.21.15.103:0/1868598494 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7fd2400053c0 con 0x55c1a4da8370 2024-09-19T07:06:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.521+0000 7fd25f707740 1 -- 172.21.15.103:0/1868598494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd248019070 con 0x55c1a4da8370 2024-09-19T07:06:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.521+0000 7fd23e7fc640 1 -- 172.21.15.103:0/1868598494 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd24000ee50 con 0x55c1a4da8370 2024-09-19T07:06:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.521+0000 7fd25f707740 10 client.21597 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.521+0000 7fd25f707740 1 -- 172.21.15.103:0/1868598494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c1a4db6a80 con 0x55c1a4da8370 2024-09-19T07:06:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.521+0000 7fd23e7fc640 1 -- 172.21.15.103:0/1868598494 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd248019070 con 0x55c1a4da8370 2024-09-19T07:06:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.521+0000 7fd23e7fc640 1 -- 172.21.15.103:0/1868598494 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd24001c6d0 con 0x55c1a4da8370 2024-09-19T07:06:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.521+0000 7fd25f707740 10 client.21597 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.521+0000 7fd25f707740 10 client.21597 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:06:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.521+0000 7fd25f707740 20 client.21597 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:43.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.521+0000 7fd25f707740 1 --2- 172.21.15.103:0/1868598494 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c1a4db7f40 0x55c1a4dba390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.521+0000 7fd25f707740 4 client.21597 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:06:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.521+0000 7fd25f707740 1 -- 172.21.15.103:0/1868598494 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1a4d10860 con 0x55c1a4db7f40 2024-09-19T07:06:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.522+0000 7fd256ffd640 1 --2- 172.21.15.103:0/1868598494 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c1a4db7f40 0x55c1a4dba390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.522+0000 7fd256ffd640 1 --2- 172.21.15.103:0/1868598494 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c1a4db7f40 0x55c1a4dba390 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8a93a46c1bd324e8 server_cookie=b7240d13e31fccd4 in_seq=0 out_seq=0 2024-09-19T07:06:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.522+0000 7fd23e7fc640 10 client.21597 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:06:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.523+0000 7fd23e7fc640 1 -- 172.21.15.103:0/1868598494 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1a4d10860 con 0x55c1a4db7f40 2024-09-19T07:06:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.523+0000 7fd23e7fc640 10 client.21597 handle_command_reply: tid=0 2024-09-19T07:06:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.523+0000 7fd25f707740 1 client.21597 shutdown 2024-09-19T07:06:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.523+0000 7fd25f707740 1 -- 172.21.15.103:0/1868598494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4da8370 msgr2=0x55c1a4da54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.523+0000 7fd25f707740 1 --2- 172.21.15.103:0/1868598494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4da8370 0x55c1a4da54f0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fd240011e60 tx=0x7fd240010800 comp rx=0 tx=0).stop 2024-09-19T07:06:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.523+0000 7fd25f707740 1 -- 172.21.15.103:0/1868598494 shutdown_connections 2024-09-19T07:06:43.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.523+0000 7fd25f707740 1 --2- 172.21.15.103:0/1868598494 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c1a4db7f40 0x55c1a4dba390 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.523+0000 7fd25f707740 1 --2- 172.21.15.103:0/1868598494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4da8370 0x55c1a4da54f0 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.523+0000 7fd25f707740 1 --2- 172.21.15.103:0/1868598494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4d9eb40 0x55c1a4da4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.523+0000 7fd25f707740 1 --2- 172.21.15.103:0/1868598494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4da7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.523+0000 7fd25f707740 1 -- 172.21.15.103:0/1868598494 >> 172.21.15.103:0/1868598494 conn(0x55c1a4d66310 msgr2=0x55c1a4d9f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.523+0000 7fd23e7fc640 0 client.21597 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:06:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.523+0000 7fd25f707740 1 -- 172.21.15.103:0/1868598494 shutdown_connections 2024-09-19T07:06:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.523+0000 7fd25f707740 1 -- 172.21.15.103:0/1868598494 wait complete. 2024-09-19T07:06:43.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.523+0000 7fd25f707740 20 client.21597 trim_cache size 0 max 16384 2024-09-19T07:06:43.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.536+0000 7fd25f707740 1 Processor -- start 2024-09-19T07:06:43.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.536+0000 7fd25f707740 1 -- start start 2024-09-19T07:06:43.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.536+0000 7fd25f707740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4db4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.536+0000 7fd25f707740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4d9eb40 0x55c1a4d99700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.536+0000 7fd25f707740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4da8370 0x55c1a4d99c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.536+0000 7fd25f707740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581b3ed0 con 0x55c1a4da8370 2024-09-19T07:06:43.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.536+0000 7fd25f707740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581b5a10 con 0x55c1a4d9d5d0 2024-09-19T07:06:43.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.536+0000 7fd25f707740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581b45a0 con 0x55c1a4d9eb40 2024-09-19T07:06:43.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.537+0000 7fd24ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4da8370 0x55c1a4d99c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.537+0000 7fd24ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4da8370 0x55c1a4d99c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55324/0 (socket says 172.21.15.103:55324) 2024-09-19T07:06:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.537+0000 7fd24ffff640 1 -- 172.21.15.103:0/1290236563 learned_addr learned my addr 172.21.15.103:0/1290236563 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.537+0000 7fd256ffd640 1 --2- 172.21.15.103:0/1290236563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4d9eb40 0x55c1a4d99700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.537+0000 7fd24ffff640 1 --2- 172.21.15.103:0/1290236563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4db4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.537+0000 7fd24ffff640 1 -- 172.21.15.103:0/1290236563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4d9eb40 msgr2=0x55c1a4d99700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.537+0000 7fd24ffff640 1 --2- 172.21.15.103:0/1290236563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4d9eb40 0x55c1a4d99700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.537+0000 7fd24ffff640 1 -- 172.21.15.103:0/1290236563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4d9d5d0 msgr2=0x55c1a4db4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.537+0000 7fd24ffff640 1 --2- 172.21.15.103:0/1290236563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4db4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.537+0000 7fd24ffff640 1 -- 172.21.15.103:0/1290236563 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1a4d9a180 con 0x55c1a4da8370 2024-09-19T07:06:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.537+0000 7fd256ffd640 1 --2- 172.21.15.103:0/1290236563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4d9eb40 0x55c1a4d99700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:06:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.537+0000 7fd24ffff640 1 --2- 172.21.15.103:0/1290236563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4da8370 0x55c1a4d99c40 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fd2400117d0 tx=0x7fd240013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900a4cb2e83c2409 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.537+0000 7fd24cff9640 1 -- 172.21.15.103:0/1290236563 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240028020 con 0x55c1a4da8370 2024-09-19T07:06:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.537+0000 7fd24cff9640 1 -- 172.21.15.103:0/1290236563 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd240004c10 con 0x55c1a4da8370 2024-09-19T07:06:43.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.537+0000 7fd24cff9640 1 -- 172.21.15.103:0/1290236563 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240005460 con 0x55c1a4da8370 2024-09-19T07:06:43.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.538+0000 7fd25f707740 1 -- 172.21.15.103:0/1290236563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4da8370 msgr2=0x55c1a4d99c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:43.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.538+0000 7fd25f707740 1 --2- 172.21.15.103:0/1290236563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4da8370 0x55c1a4d99c40 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fd2400117d0 tx=0x7fd240013040 comp rx=0 tx=0).stop 2024-09-19T07:06:43.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.538+0000 7fd25f707740 1 -- 172.21.15.103:0/1290236563 shutdown_connections 2024-09-19T07:06:43.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.538+0000 7fd25f707740 1 --2- 172.21.15.103:0/1290236563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4da8370 0x55c1a4d99c40 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.538+0000 7fd25f707740 1 --2- 172.21.15.103:0/1290236563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4d9eb40 0x55c1a4d99700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.538+0000 7fd25f707740 1 --2- 172.21.15.103:0/1290236563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4db4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.538+0000 7fd25f707740 1 -- 172.21.15.103:0/1290236563 >> 172.21.15.103:0/1290236563 conn(0x55c1a4d66310 msgr2=0x55c1a4da3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:43.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.538+0000 7fd25f707740 1 -- 172.21.15.103:0/1290236563 shutdown_connections 2024-09-19T07:06:43.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.538+0000 7fd25f707740 1 -- 172.21.15.103:0/1290236563 wait complete. 2024-09-19T07:06:43.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.538+0000 7fd25f707740 1 -- start start 2024-09-19T07:06:43.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd25f707740 1 Processor -- start 2024-09-19T07:06:43.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd25f707740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4db0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd25f707740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4d9eb40 0x55c1a4db09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd25f707740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4da8370 0x55c1a4da15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd25f707740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581b5340 con 0x55c1a4d9d5d0 2024-09-19T07:06:43.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd25f707740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581b4c70 con 0x55c1a4da8370 2024-09-19T07:06:43.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd25f707740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581b1ed0 con 0x55c1a4d9eb40 2024-09-19T07:06:43.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd25f707740 10 client.0 _ll_register_callbacks cb 0x7fd24000e810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:43.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd24ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4db0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:43.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd24ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4db0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55328/0 (socket says 172.21.15.103:55328) 2024-09-19T07:06:43.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd24ffff640 1 -- 172.21.15.103:0/820804004 learned_addr learned my addr 172.21.15.103:0/820804004 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:43.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd24ffff640 1 -- 172.21.15.103:0/820804004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4d9eb40 msgr2=0x55c1a4db09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:43.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd256ffd640 1 --2- 172.21.15.103:0/820804004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4d9eb40 0x55c1a4db09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:43.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd24ffff640 1 --2- 172.21.15.103:0/820804004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4d9eb40 0x55c1a4db09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd24ffff640 1 -- 172.21.15.103:0/820804004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4da8370 msgr2=0x55c1a4da15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:43.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd24ffff640 1 --2- 172.21.15.103:0/820804004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4da8370 0x55c1a4da15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd24ffff640 1 -- 172.21.15.103:0/820804004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1a4da1b00 con 0x55c1a4d9d5d0 2024-09-19T07:06:43.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd24ffff640 1 --2- 172.21.15.103:0/820804004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4db0480 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7fd24000e9c0 tx=0x7fd240011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f3b0c955e88ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:43.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd23e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:43.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.539+0000 7fd23e7fc640 1 -- 172.21.15.103:0/820804004 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240028030 con 0x55c1a4d9d5d0 2024-09-19T07:06:43.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.540+0000 7fd25f707740 1 -- 172.21.15.103:0/820804004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd240004c10 con 0x55c1a4d9d5d0 2024-09-19T07:06:43.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.540+0000 7fd23e7fc640 1 -- 172.21.15.103:0/820804004 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7fd240005160 con 0x55c1a4d9d5d0 2024-09-19T07:06:43.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.540+0000 7fd23e7fc640 1 -- 172.21.15.103:0/820804004 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd24001c840 con 0x55c1a4d9d5d0 2024-09-19T07:06:43.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.540+0000 7fd25f707740 10 client.21603 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:43.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.540+0000 7fd25f707740 1 -- 172.21.15.103:0/820804004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c1a4e22450 con 0x55c1a4d9d5d0 2024-09-19T07:06:43.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.540+0000 7fd23e7fc640 1 -- 172.21.15.103:0/820804004 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd240004c10 con 0x55c1a4d9d5d0 2024-09-19T07:06:43.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.540+0000 7fd23e7fc640 1 -- 172.21.15.103:0/820804004 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd240043030 con 0x55c1a4d9d5d0 2024-09-19T07:06:43.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.540+0000 7fd25f707740 10 client.21603 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:43.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.540+0000 7fd25f707740 10 client.21603 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:06:43.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.540+0000 7fd25f707740 20 client.21603 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:43.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.540+0000 7fd25f707740 1 --2- 172.21.15.103:0/820804004 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c1a4e22ab0 0x55c1a4e24ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:43.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.540+0000 7fd25f707740 4 client.21603 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:06:43.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.540+0000 7fd25f707740 1 -- 172.21.15.103:0/820804004 --> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c1a4d0ad70 con 0x55c1a4e22ab0 2024-09-19T07:06:43.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.540+0000 7fd256ffd640 1 --2- 172.21.15.103:0/820804004 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c1a4e22ab0 0x55c1a4e24ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:43.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd23e7fc640 10 client.21603 ms_handle_connect on v2:172.21.15.174:6836/1053388911 2024-09-19T07:06:43.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd256ffd640 1 --2- 172.21.15.103:0/820804004 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c1a4e22ab0 0x55c1a4e24ea0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=285421db35a586f8 server_cookie=9ce50fff91690c7b in_seq=0 out_seq=0 2024-09-19T07:06:43.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd23e7fc640 1 -- 172.21.15.103:0/820804004 <== mds.3 v2:172.21.15.174:6836/1053388911 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c1a4d0ad70 con 0x55c1a4e22ab0 2024-09-19T07:06:43.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd23e7fc640 10 client.21603 handle_command_reply: tid=0 2024-09-19T07:06:43.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd25f707740 1 client.21603 shutdown 2024-09-19T07:06:43.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd25f707740 1 -- 172.21.15.103:0/820804004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4d9d5d0 msgr2=0x55c1a4db0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:43.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd25f707740 1 --2- 172.21.15.103:0/820804004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4db0480 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7fd24000e9c0 tx=0x7fd240011b40 comp rx=0 tx=0).stop 2024-09-19T07:06:43.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd25f707740 1 -- 172.21.15.103:0/820804004 shutdown_connections 2024-09-19T07:06:43.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd25f707740 1 --2- 172.21.15.103:0/820804004 >> [v2:172.21.15.174:6836/1053388911,v1:172.21.15.174:6837/1053388911] conn(0x55c1a4e22ab0 0x55c1a4e24ea0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd25f707740 1 --2- 172.21.15.103:0/820804004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1a4da8370 0x55c1a4da15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd25f707740 1 --2- 172.21.15.103:0/820804004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1a4d9eb40 0x55c1a4db09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd25f707740 1 --2- 172.21.15.103:0/820804004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1a4d9d5d0 0x55c1a4db0480 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd23e7fc640 0 client.21603 ms_handle_reset on v2:172.21.15.174:6836/1053388911 2024-09-19T07:06:43.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd25f707740 1 -- 172.21.15.103:0/820804004 >> 172.21.15.103:0/820804004 conn(0x55c1a4d66310 msgr2=0x55c1a4da3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:43.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd25f707740 1 -- 172.21.15.103:0/820804004 shutdown_connections 2024-09-19T07:06:43.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd25f707740 1 -- 172.21.15.103:0/820804004 wait complete. 2024-09-19T07:06:43.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.541+0000 7fd25f707740 20 client.21603 trim_cache size 0 max 16384 2024-09-19T07:06:43.553 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:06:43.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.542+0000 7fd25d32e640 1 -- 172.21.15.103:0/4285397952 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2440a3430 msgr2=0x7fd2440a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:43.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.542+0000 7fd25d32e640 1 --2- 172.21.15.103:0/4285397952 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2440a3430 0x7fd2440a58f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd2500bd120 tx=0x7fd2500b28b0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.542+0000 7fd25d32e640 1 -- 172.21.15.103:0/4285397952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd258106b70 msgr2=0x7fd25819e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:43.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.542+0000 7fd25d32e640 1 --2- 172.21.15.103:0/4285397952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd258106b70 0x7fd25819e830 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7fd250030c60 tx=0x7fd250030c90 comp rx=0 tx=0).stop 2024-09-19T07:06:43.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.542+0000 7fd25d32e640 1 -- 172.21.15.103:0/4285397952 shutdown_connections 2024-09-19T07:06:43.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.542+0000 7fd25d32e640 1 --2- 172.21.15.103:0/4285397952 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd2440a3430 0x7fd2440a58f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.542+0000 7fd25d32e640 1 --2- 172.21.15.103:0/4285397952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd258106b70 0x7fd25819e830 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.542+0000 7fd25d32e640 1 --2- 172.21.15.103:0/4285397952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd258100a50 0x7fd2581a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.542+0000 7fd25d32e640 1 --2- 172.21.15.103:0/4285397952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd258100090 0x7fd2581a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:43.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.542+0000 7fd25d32e640 1 -- 172.21.15.103:0/4285397952 >> 172.21.15.103:0/4285397952 conn(0x7fd258077210 msgr2=0x7fd2581037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:43.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.542+0000 7fd25d32e640 1 -- 172.21.15.103:0/4285397952 shutdown_connections 2024-09-19T07:06:43.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:43.542+0000 7fd25d32e640 1 -- 172.21.15.103:0/4285397952 wait complete. 2024-09-19T07:06:44.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:44 smithi103 ceph-mon[31158]: pgmap v547: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:06:44.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:44 smithi174 ceph-mon[35637]: pgmap v547: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:06:44.425 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:06:44.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:44 smithi134 ceph-mon[35356]: pgmap v547: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:06:44.670 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:45.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.130+0000 7f4bbf1ed640 1 -- 172.21.15.103:0/3154675446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bb8106a80 msgr2=0x7f4bb8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:45.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.130+0000 7f4bbf1ed640 1 --2- 172.21.15.103:0/3154675446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bb8106a80 0x7f4bb8106e60 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009930 tx=0x7f4bb402a420 comp rx=0 tx=0).stop 2024-09-19T07:06:45.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.130+0000 7f4bbf1ed640 1 -- 172.21.15.103:0/3154675446 shutdown_connections 2024-09-19T07:06:45.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.130+0000 7f4bbf1ed640 1 --2- 172.21.15.103:0/3154675446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bb8101420 0x7f4bb810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.130+0000 7f4bbf1ed640 1 --2- 172.21.15.103:0/3154675446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bb8100a80 0x7f4bb8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.130+0000 7f4bbf1ed640 1 --2- 172.21.15.103:0/3154675446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bb8106a80 0x7f4bb8106e60 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.130+0000 7f4bbf1ed640 1 -- 172.21.15.103:0/3154675446 >> 172.21.15.103:0/3154675446 conn(0x7f4bb80fc7a0 msgr2=0x7f4bb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:45.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.130+0000 7f4bbf1ed640 1 -- 172.21.15.103:0/3154675446 shutdown_connections 2024-09-19T07:06:45.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.130+0000 7f4bbf1ed640 1 -- 172.21.15.103:0/3154675446 wait complete. 2024-09-19T07:06:45.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.131+0000 7f4bbf1ed640 1 Processor -- start 2024-09-19T07:06:45.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.131+0000 7f4bbf1ed640 1 -- start start 2024-09-19T07:06:45.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.131+0000 7f4bbf1ed640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bb8100a80 0x7f4bb8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.131+0000 7f4bbf1ed640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bb8101420 0x7f4bb8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.131+0000 7f4bbf1ed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bb8106a80 0x7f4bb819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.131+0000 7f4bbf1ed640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb8111fa0 con 0x7f4bb8106a80 2024-09-19T07:06:45.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.131+0000 7f4bbf1ed640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb8111e20 con 0x7f4bb8100a80 2024-09-19T07:06:45.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.131+0000 7f4bbf1ed640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb8112120 con 0x7f4bb8101420 2024-09-19T07:06:45.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.131+0000 7f4bbd78a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bb8106a80 0x7f4bb819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:45.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.131+0000 7f4bbd78a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bb8106a80 0x7f4bb819cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55350/0 (socket says 172.21.15.103:55350) 2024-09-19T07:06:45.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.131+0000 7f4bbd78a640 1 -- 172.21.15.103:0/1660367477 learned_addr learned my addr 172.21.15.103:0/1660367477 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:45.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.131+0000 7f4bbcf89640 1 --2- 172.21.15.103:0/1660367477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bb8100a80 0x7f4bb8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:45.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.132+0000 7f4bbd78a640 1 -- 172.21.15.103:0/1660367477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bb8101420 msgr2=0x7f4bb8198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:45.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.132+0000 7f4baffff640 1 --2- 172.21.15.103:0/1660367477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bb8101420 0x7f4bb8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:45.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.132+0000 7f4bbd78a640 1 --2- 172.21.15.103:0/1660367477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bb8101420 0x7f4bb8198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.132+0000 7f4bbd78a640 1 -- 172.21.15.103:0/1660367477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bb8100a80 msgr2=0x7f4bb8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:45.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.132+0000 7f4bbd78a640 1 --2- 172.21.15.103:0/1660367477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bb8100a80 0x7f4bb8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.132+0000 7f4bbd78a640 1 -- 172.21.15.103:0/1660367477 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb819d3f0 con 0x7f4bb8106a80 2024-09-19T07:06:45.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.132+0000 7f4baffff640 1 --2- 172.21.15.103:0/1660367477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bb8101420 0x7f4bb8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:45.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.132+0000 7f4bbd78a640 1 --2- 172.21.15.103:0/1660367477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bb8106a80 0x7f4bb819cd10 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000eaa0 tx=0x7f4bb000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:45.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.132+0000 7f4badffb640 1 -- 172.21.15.103:0/1660367477 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000cdb0 con 0x7f4bb8106a80 2024-09-19T07:06:45.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.132+0000 7f4badffb640 1 -- 172.21.15.103:0/1660367477 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb0005020 con 0x7f4bb8106a80 2024-09-19T07:06:45.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.132+0000 7f4bbf1ed640 1 -- 172.21.15.103:0/1660367477 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb819d6e0 con 0x7f4bb8106a80 2024-09-19T07:06:45.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.132+0000 7f4badffb640 1 -- 172.21.15.103:0/1660367477 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb00116b0 con 0x7f4bb8106a80 2024-09-19T07:06:45.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.132+0000 7f4bbf1ed640 1 -- 172.21.15.103:0/1660367477 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb81a4fc0 con 0x7f4bb8106a80 2024-09-19T07:06:45.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.134+0000 7f4badffb640 1 -- 172.21.15.103:0/1660367477 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4bb0005560 con 0x7f4bb8106a80 2024-09-19T07:06:45.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.135+0000 7f4bc15c6740 1 Processor -- start 2024-09-19T07:06:45.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.135+0000 7f4bc15c6740 1 -- start start 2024-09-19T07:06:45.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.135+0000 7f4badffb640 1 --2- 172.21.15.103:0/1660367477 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4ba00a33b0 0x7f4ba00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.135+0000 7f4bc15c6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89bfffb40 0x55c89bffe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.135+0000 7f4bc15c6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89bffea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.135+0000 7f4bc15c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89c009370 0x55c89c00b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.135+0000 7f4bc15c6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c89bf642f0 con 0x55c89bffe5d0 2024-09-19T07:06:45.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.135+0000 7f4bc15c6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c89bf0bcf0 con 0x55c89bfffb40 2024-09-19T07:06:45.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.135+0000 7f4bc15c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c89bf602b0 con 0x55c89c009370 2024-09-19T07:06:45.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.135+0000 7f4badffb640 1 -- 172.21.15.103:0/1660367477 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f4bb00cff30 con 0x7f4bb8106a80 2024-09-19T07:06:45.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.135+0000 7f4baffff640 1 --2- 172.21.15.103:0/1660367477 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4ba00a33b0 0x7f4ba00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:45.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.135+0000 7f4baffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89bffea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:45.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.135+0000 7f4baffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89bffea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55358/0 (socket says 172.21.15.103:55358) 2024-09-19T07:06:45.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.135+0000 7f4baffff640 1 -- 172.21.15.103:0/674406783 learned_addr learned my addr 172.21.15.103:0/674406783 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:45.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.135+0000 7f4bbcf89640 1 --2- 172.21.15.103:0/674406783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89bfffb40 0x55c89bffe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:45.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.135+0000 7f4bbd78a640 1 --2- 172.21.15.103:0/674406783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89c009370 0x55c89c00b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:45.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.136+0000 7f4baffff640 1 --2- 172.21.15.103:0/1660367477 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4ba00a33b0 0x7f4ba00a5870 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800bfb0 tx=0x7f4ba8014840 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:45.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.136+0000 7f4baffff640 1 -- 172.21.15.103:0/674406783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89c009370 msgr2=0x55c89c00b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:45.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.136+0000 7f4baffff640 1 --2- 172.21.15.103:0/674406783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89c009370 0x55c89c00b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.136+0000 7f4baffff640 1 -- 172.21.15.103:0/674406783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89bfffb40 msgr2=0x55c89bffe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:45.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.136+0000 7f4baffff640 1 --2- 172.21.15.103:0/674406783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89bfffb40 0x55c89bffe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.136+0000 7f4baffff640 1 -- 172.21.15.103:0/674406783 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c89bef03a0 con 0x55c89bffe5d0 2024-09-19T07:06:45.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.136+0000 7f4baffff640 1 --2- 172.21.15.103:0/674406783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89bffea10 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8014d40 tx=0x7f4ba801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270f95efc43b9118 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:45.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.136+0000 7f4b9effd640 1 -- 172.21.15.103:0/674406783 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8023070 con 0x55c89bffe5d0 2024-09-19T07:06:45.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.136+0000 7f4b9effd640 1 -- 172.21.15.103:0/674406783 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ba8012930 con 0x55c89bffe5d0 2024-09-19T07:06:45.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.136+0000 7f4b9effd640 1 -- 172.21.15.103:0/674406783 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba801e8d0 con 0x55c89bffe5d0 2024-09-19T07:06:45.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.137+0000 7f4bc15c6740 1 -- 172.21.15.103:0/674406783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 msgr2=0x55c89bffea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:45.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.137+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/674406783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89bffea10 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8014d40 tx=0x7f4ba801b000 comp rx=0 tx=0).stop 2024-09-19T07:06:45.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.137+0000 7f4bc15c6740 1 -- 172.21.15.103:0/674406783 shutdown_connections 2024-09-19T07:06:45.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.137+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/674406783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89c009370 0x55c89c00b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.137+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/674406783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89bffea10 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.137+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/674406783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89bfffb40 0x55c89bffe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.137+0000 7f4bc15c6740 1 -- 172.21.15.103:0/674406783 >> 172.21.15.103:0/674406783 conn(0x55c89bfc7310 msgr2=0x55c89bffac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.137+0000 7f4bc15c6740 1 -- 172.21.15.103:0/674406783 shutdown_connections 2024-09-19T07:06:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.137+0000 7f4bc15c6740 1 -- 172.21.15.103:0/674406783 wait complete. 2024-09-19T07:06:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.137+0000 7f4bc15c6740 1 -- start start 2024-09-19T07:06:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.137+0000 7f4bc15c6740 1 Processor -- start 2024-09-19T07:06:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bc15c6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89c008d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bc15c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89bfffb40 0x55c89c006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bc15c6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89c009370 0x55c89c006610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bc15c6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c89c00c070 con 0x55c89bffe5d0 2024-09-19T07:06:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bc15c6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c89bf1f060 con 0x55c89c009370 2024-09-19T07:06:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bc15c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c89c00c1f0 con 0x55c89bfffb40 2024-09-19T07:06:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bc15c6740 10 client.0 _ll_register_callbacks cb 0x55c89bc728c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bbcf89640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89c008d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:45.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bbcf89640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89c008d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55362/0 (socket says 172.21.15.103:55362) 2024-09-19T07:06:45.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bbcf89640 1 -- 172.21.15.103:0/2202456929 learned_addr learned my addr 172.21.15.103:0/2202456929 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:45.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bbd78a640 1 --2- 172.21.15.103:0/2202456929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89c009370 0x55c89c006610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:45.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4baffff640 1 --2- 172.21.15.103:0/2202456929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89bfffb40 0x55c89c006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:45.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bbcf89640 1 -- 172.21.15.103:0/2202456929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89bfffb40 msgr2=0x55c89c006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:45.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bbcf89640 1 --2- 172.21.15.103:0/2202456929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89bfffb40 0x55c89c006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bbcf89640 1 -- 172.21.15.103:0/2202456929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89c009370 msgr2=0x55c89c006610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:45.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bbcf89640 1 --2- 172.21.15.103:0/2202456929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89c009370 0x55c89c006610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bbcf89640 1 -- 172.21.15.103:0/2202456929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c89bcd53c0 con 0x55c89bffe5d0 2024-09-19T07:06:45.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bbcf89640 1 --2- 172.21.15.103:0/2202456929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89c008d70 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f4bb402aa10 tx=0x7f4bb40045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b97598a6c9ad79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:45.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4b97fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:45.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4b97fff640 1 -- 172.21.15.103:0/2202456929 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4035040 con 0x55c89bffe5d0 2024-09-19T07:06:45.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4bc15c6740 1 -- 172.21.15.103:0/2202456929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ba8012930 con 0x55c89bffe5d0 2024-09-19T07:06:45.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.138+0000 7f4b97fff640 1 -- 172.21.15.103:0/2202456929 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f4bb4005550 con 0x55c89bffe5d0 2024-09-19T07:06:45.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.139+0000 7f4b97fff640 1 -- 172.21.15.103:0/2202456929 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4bb40046c0 con 0x55c89bffe5d0 2024-09-19T07:06:45.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.139+0000 7f4bc15c6740 10 client.21615 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:45.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.139+0000 7f4bc15c6740 1 -- 172.21.15.103:0/2202456929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c89c017b60 con 0x55c89bffe5d0 2024-09-19T07:06:45.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.139+0000 7f4b97fff640 1 -- 172.21.15.103:0/2202456929 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ba8012930 con 0x55c89bffe5d0 2024-09-19T07:06:45.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.139+0000 7f4b97fff640 1 -- 172.21.15.103:0/2202456929 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4bb4003c70 con 0x55c89bffe5d0 2024-09-19T07:06:45.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.139+0000 7f4bc15c6740 10 client.21615 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:45.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.139+0000 7f4bc15c6740 10 client.21615 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:06:45.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.139+0000 7f4bc15c6740 20 client.21615 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.139+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/2202456929 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55c89c018f70 0x55c89c01b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.139+0000 7f4bc15c6740 4 client.21615 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:06:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.139+0000 7f4bc15c6740 1 -- 172.21.15.103:0/2202456929 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c89bf71860 con 0x55c89c018f70 2024-09-19T07:06:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.139+0000 7f4baffff640 1 --2- 172.21.15.103:0/2202456929 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55c89c018f70 0x55c89c01b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.140+0000 7f4baffff640 1 --2- 172.21.15.103:0/2202456929 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55c89c018f70 0x55c89c01b360 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a40c712a4983e99a server_cookie=db02d1a0c0196cae in_seq=0 out_seq=0 2024-09-19T07:06:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.140+0000 7f4b97fff640 10 client.21615 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:06:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.140+0000 7f4b97fff640 1 -- 172.21.15.103:0/2202456929 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c89bf71860 con 0x55c89c018f70 2024-09-19T07:06:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.140+0000 7f4b97fff640 10 client.21615 handle_command_reply: tid=0 2024-09-19T07:06:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.140+0000 7f4bc15c6740 1 client.21615 shutdown 2024-09-19T07:06:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.141+0000 7f4bc15c6740 1 -- 172.21.15.103:0/2202456929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 msgr2=0x55c89c008d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.141+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/2202456929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89c008d70 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f4bb402aa10 tx=0x7f4bb40045c0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.141+0000 7f4bc15c6740 1 -- 172.21.15.103:0/2202456929 shutdown_connections 2024-09-19T07:06:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.141+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/2202456929 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55c89c018f70 0x55c89c01b360 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.141+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/2202456929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89c009370 0x55c89c006610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.141+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/2202456929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89bfffb40 0x55c89c006020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.141+0000 7f4b97fff640 0 client.21615 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:06:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.141+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/2202456929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89c008d70 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.141+0000 7f4bc15c6740 1 -- 172.21.15.103:0/2202456929 >> 172.21.15.103:0/2202456929 conn(0x55c89bfc7310 msgr2=0x55c89c000960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.141+0000 7f4bc15c6740 1 -- 172.21.15.103:0/2202456929 shutdown_connections 2024-09-19T07:06:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.141+0000 7f4bc15c6740 1 -- 172.21.15.103:0/2202456929 wait complete. 2024-09-19T07:06:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.141+0000 7f4bc15c6740 20 client.21615 trim_cache size 0 max 16384 2024-09-19T07:06:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bc15c6740 1 Processor -- start 2024-09-19T07:06:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bc15c6740 1 -- start start 2024-09-19T07:06:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bc15c6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89c007f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bc15c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89bfffb40 0x55c89c001120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bc15c6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89c009370 0x55c89c001660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bc15c6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb81ad870 con 0x55c89bffe5d0 2024-09-19T07:06:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bc15c6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb81af3b0 con 0x55c89c009370 2024-09-19T07:06:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bc15c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb81adf40 con 0x55c89bfffb40 2024-09-19T07:06:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bbcf89640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89c007f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bbcf89640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89c007f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55374/0 (socket says 172.21.15.103:55374) 2024-09-19T07:06:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bbcf89640 1 -- 172.21.15.103:0/2461166321 learned_addr learned my addr 172.21.15.103:0/2461166321 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4baffff640 1 --2- 172.21.15.103:0/2461166321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89bfffb40 0x55c89c001120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bbcf89640 1 -- 172.21.15.103:0/2461166321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89bfffb40 msgr2=0x55c89c001120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bbcf89640 1 --2- 172.21.15.103:0/2461166321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89bfffb40 0x55c89c001120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bbcf89640 1 -- 172.21.15.103:0/2461166321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89c009370 msgr2=0x55c89c001660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bbcf89640 1 --2- 172.21.15.103:0/2461166321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89c009370 0x55c89c001660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bbcf89640 1 -- 172.21.15.103:0/2461166321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c89bffb730 con 0x55c89bffe5d0 2024-09-19T07:06:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4baffff640 1 --2- 172.21.15.103:0/2461166321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89bfffb40 0x55c89c001120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:06:45.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.154+0000 7f4bbcf89640 1 --2- 172.21.15.103:0/2461166321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89c007f90 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4004650 tx=0x7f4bb4004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a03f6f90c8831e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:45.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.155+0000 7f4b9e7fc640 1 -- 172.21.15.103:0/2461166321 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4035030 con 0x55c89bffe5d0 2024-09-19T07:06:45.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.155+0000 7f4b9e7fc640 1 -- 172.21.15.103:0/2461166321 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb4041070 con 0x55c89bffe5d0 2024-09-19T07:06:45.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.155+0000 7f4b9e7fc640 1 -- 172.21.15.103:0/2461166321 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4052070 con 0x55c89bffe5d0 2024-09-19T07:06:45.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.155+0000 7f4bc15c6740 1 -- 172.21.15.103:0/2461166321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 msgr2=0x55c89c007f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:45.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.155+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/2461166321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89c007f90 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4004650 tx=0x7f4bb4004150 comp rx=0 tx=0).stop 2024-09-19T07:06:45.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.155+0000 7f4bc15c6740 1 -- 172.21.15.103:0/2461166321 shutdown_connections 2024-09-19T07:06:45.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.155+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/2461166321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89c009370 0x55c89c001660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.155+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/2461166321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89bfffb40 0x55c89c001120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.155+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/2461166321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bffe5d0 0x55c89c007f90 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.155+0000 7f4bc15c6740 1 -- 172.21.15.103:0/2461166321 >> 172.21.15.103:0/2461166321 conn(0x55c89bfc7310 msgr2=0x55c89c007880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:45.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.155+0000 7f4bc15c6740 1 -- 172.21.15.103:0/2461166321 shutdown_connections 2024-09-19T07:06:45.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.155+0000 7f4bc15c6740 1 -- 172.21.15.103:0/2461166321 wait complete. 2024-09-19T07:06:45.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.155+0000 7f4bc15c6740 1 -- start start 2024-09-19T07:06:45.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4bc15c6740 1 Processor -- start 2024-09-19T07:06:45.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4bc15c6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89bffe5d0 0x55c89c011cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4bc15c6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bfffb40 0x55c89c00d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4bc15c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89c009370 0x55c89c00d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4bc15c6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb81aece0 con 0x55c89bfffb40 2024-09-19T07:06:45.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4bc15c6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb81ae610 con 0x55c89bffe5d0 2024-09-19T07:06:45.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4bc15c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb81ab870 con 0x55c89c009370 2024-09-19T07:06:45.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4bc15c6740 10 client.0 _ll_register_callbacks cb 0x7f4bb4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:45.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4baffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bfffb40 0x55c89c00d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:45.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4baffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bfffb40 0x55c89c00d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55390/0 (socket says 172.21.15.103:55390) 2024-09-19T07:06:45.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4baffff640 1 -- 172.21.15.103:0/3624852771 learned_addr learned my addr 172.21.15.103:0/3624852771 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4baffff640 1 -- 172.21.15.103:0/3624852771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89c009370 msgr2=0x55c89c00d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4baffff640 1 --2- 172.21.15.103:0/3624852771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89c009370 0x55c89c00d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4bbcf89640 1 --2- 172.21.15.103:0/3624852771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89bffe5d0 0x55c89c011cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4baffff640 1 -- 172.21.15.103:0/3624852771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89bffe5d0 msgr2=0x55c89c011cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4baffff640 1 --2- 172.21.15.103:0/3624852771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89bffe5d0 0x55c89c011cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4baffff640 1 -- 172.21.15.103:0/3624852771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c89c00de40 con 0x55c89bfffb40 2024-09-19T07:06:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.156+0000 7f4baffff640 1 --2- 172.21.15.103:0/3624852771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bfffb40 0x55c89c00d020 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8014d40 tx=0x7f4ba800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51919a6716508b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.157+0000 7f4b97fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.157+0000 7f4b97fff640 1 -- 172.21.15.103:0/3624852771 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8037020 con 0x55c89bfffb40 2024-09-19T07:06:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.157+0000 7f4b97fff640 1 -- 172.21.15.103:0/3624852771 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f4ba801e6a0 con 0x55c89bfffb40 2024-09-19T07:06:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.157+0000 7f4bc15c6740 1 -- 172.21.15.103:0/3624852771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb4041070 con 0x55c89bfffb40 2024-09-19T07:06:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.157+0000 7f4b97fff640 1 -- 172.21.15.103:0/3624852771 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4ba8042050 con 0x55c89bfffb40 2024-09-19T07:06:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.157+0000 7f4bc15c6740 10 client.21621 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.157+0000 7f4bc15c6740 1 -- 172.21.15.103:0/3624852771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c89c083cf0 con 0x55c89bfffb40 2024-09-19T07:06:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.157+0000 7f4b97fff640 1 -- 172.21.15.103:0/3624852771 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb4041070 con 0x55c89bfffb40 2024-09-19T07:06:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.157+0000 7f4b97fff640 1 -- 172.21.15.103:0/3624852771 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4ba8012570 con 0x55c89bfffb40 2024-09-19T07:06:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.157+0000 7f4bc15c6740 10 client.21621 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.157+0000 7f4bc15c6740 10 client.21621 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T07:06:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.157+0000 7f4bc15c6740 20 client.21621 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.157+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/3624852771 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55c89c0841e0 0x55c89c086600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.157+0000 7f4bc15c6740 4 client.21621 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:06:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.157+0000 7f4bc15c6740 1 -- 172.21.15.103:0/3624852771 --> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c89bf6bd70 con 0x55c89c0841e0 2024-09-19T07:06:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.158+0000 7f4bbcf89640 1 --2- 172.21.15.103:0/3624852771 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55c89c0841e0 0x55c89c086600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.158+0000 7f4bbcf89640 1 --2- 172.21.15.103:0/3624852771 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55c89c0841e0 0x55c89c086600 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68be233592fc41a6 server_cookie=4d65a0b7e09ff32d in_seq=0 out_seq=0 2024-09-19T07:06:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.158+0000 7f4b97fff640 10 client.21621 ms_handle_connect on v2:172.21.15.174:6838/4202229983 2024-09-19T07:06:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.158+0000 7f4b97fff640 1 -- 172.21.15.103:0/3624852771 <== mds.2 v2:172.21.15.174:6838/4202229983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c89bf6bd70 con 0x55c89c0841e0 2024-09-19T07:06:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.158+0000 7f4b97fff640 10 client.21621 handle_command_reply: tid=0 2024-09-19T07:06:45.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.158+0000 7f4bc15c6740 1 client.21621 shutdown 2024-09-19T07:06:45.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.158+0000 7f4bc15c6740 1 -- 172.21.15.103:0/3624852771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bfffb40 msgr2=0x55c89c00d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:45.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.158+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/3624852771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bfffb40 0x55c89c00d020 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8014d40 tx=0x7f4ba800bce0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.158+0000 7f4bc15c6740 1 -- 172.21.15.103:0/3624852771 shutdown_connections 2024-09-19T07:06:45.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.158+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/3624852771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c89c009370 0x55c89c00d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.158+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/3624852771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c89bfffb40 0x55c89c00d020 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.158+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/3624852771 >> [v2:172.21.15.174:6838/4202229983,v1:172.21.15.174:6839/4202229983] conn(0x55c89c0841e0 0x55c89c086600 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.158+0000 7f4bc15c6740 1 --2- 172.21.15.103:0/3624852771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c89bffe5d0 0x55c89c011cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.158+0000 7f4bc15c6740 1 -- 172.21.15.103:0/3624852771 >> 172.21.15.103:0/3624852771 conn(0x55c89bfc7310 msgr2=0x55c89bff9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:45.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.158+0000 7f4b97fff640 0 client.21621 ms_handle_reset on v2:172.21.15.174:6838/4202229983 2024-09-19T07:06:45.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.159+0000 7f4bc15c6740 1 -- 172.21.15.103:0/3624852771 shutdown_connections 2024-09-19T07:06:45.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.159+0000 7f4bc15c6740 1 -- 172.21.15.103:0/3624852771 wait complete. 2024-09-19T07:06:45.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.159+0000 7f4bc15c6740 20 client.21621 trim_cache size 0 max 16384 2024-09-19T07:06:45.177 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:06:45.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.159+0000 7f4bbf1ed640 1 -- 172.21.15.103:0/1660367477 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4ba00a33b0 msgr2=0x7f4ba00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:45.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.159+0000 7f4bbf1ed640 1 --2- 172.21.15.103:0/1660367477 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4ba00a33b0 0x7f4ba00a5870 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800bfb0 tx=0x7f4ba8014840 comp rx=0 tx=0).stop 2024-09-19T07:06:45.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.159+0000 7f4bbf1ed640 1 -- 172.21.15.103:0/1660367477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bb8106a80 msgr2=0x7f4bb819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.159+0000 7f4bbf1ed640 1 --2- 172.21.15.103:0/1660367477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bb8106a80 0x7f4bb819cd10 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000eaa0 tx=0x7f4bb000ef60 comp rx=0 tx=0).stop 2024-09-19T07:06:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.159+0000 7f4bbf1ed640 1 -- 172.21.15.103:0/1660367477 shutdown_connections 2024-09-19T07:06:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.159+0000 7f4bbf1ed640 1 --2- 172.21.15.103:0/1660367477 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4ba00a33b0 0x7f4ba00a5870 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.159+0000 7f4bbf1ed640 1 --2- 172.21.15.103:0/1660367477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bb8106a80 0x7f4bb819cd10 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.159+0000 7f4bbf1ed640 1 --2- 172.21.15.103:0/1660367477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bb8101420 0x7f4bb8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.159+0000 7f4bbf1ed640 1 --2- 172.21.15.103:0/1660367477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bb8100a80 0x7f4bb8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.159+0000 7f4bbf1ed640 1 -- 172.21.15.103:0/1660367477 >> 172.21.15.103:0/1660367477 conn(0x7f4bb80fc7a0 msgr2=0x7f4bb80fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.159+0000 7f4bbf1ed640 1 -- 172.21.15.103:0/1660367477 shutdown_connections 2024-09-19T07:06:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:45.159+0000 7f4bbf1ed640 1 -- 172.21.15.103:0/1660367477 wait complete. 2024-09-19T07:06:45.866 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:06:46.116 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:46.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:46 smithi103 ceph-mon[31158]: pgmap v548: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:06:46.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:46 smithi174 ceph-mon[35637]: pgmap v548: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:06:46.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:46 smithi134 ceph-mon[35356]: pgmap v548: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:06:46.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.592+0000 7f9426911640 1 -- 172.21.15.103:0/3787479921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9420101dd0 msgr2=0x7f94201021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:46.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.592+0000 7f9426911640 1 --2- 172.21.15.103:0/3787479921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9420101dd0 0x7f94201021b0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f9410009900 tx=0x7f941002a410 comp rx=0 tx=0).stop 2024-09-19T07:06:46.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.592+0000 7f9426911640 1 -- 172.21.15.103:0/3787479921 shutdown_connections 2024-09-19T07:06:46.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.592+0000 7f9426911640 1 --2- 172.21.15.103:0/3787479921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9420106f00 0x7f942010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.592+0000 7f9426911640 1 --2- 172.21.15.103:0/3787479921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9420102780 0x7f94201067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.592+0000 7f9426911640 1 --2- 172.21.15.103:0/3787479921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9420101dd0 0x7f94201021b0 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.592+0000 7f9426911640 1 -- 172.21.15.103:0/3787479921 >> 172.21.15.103:0/3787479921 conn(0x7f94200fd890 msgr2=0x7f94200ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:46.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.592+0000 7f9426911640 1 -- 172.21.15.103:0/3787479921 shutdown_connections 2024-09-19T07:06:46.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.592+0000 7f9426911640 1 -- 172.21.15.103:0/3787479921 wait complete. 2024-09-19T07:06:46.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.592+0000 7f9426911640 1 Processor -- start 2024-09-19T07:06:46.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.592+0000 7f9426911640 1 -- start start 2024-09-19T07:06:46.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.593+0000 7f9426911640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9420101dd0 0x7f94201983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.593+0000 7f9426911640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9420102780 0x7f94201988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.593+0000 7f9426911640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9420106f00 0x7f942019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.593+0000 7f9426911640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9420111f60 con 0x7f9420106f00 2024-09-19T07:06:46.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.593+0000 7f9426911640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9420111de0 con 0x7f9420102780 2024-09-19T07:06:46.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.593+0000 7f9426911640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94201120e0 con 0x7f9420101dd0 2024-09-19T07:06:46.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.593+0000 7f9424eae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9420106f00 0x7f942019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:46.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.593+0000 7f9424eae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9420106f00 0x7f942019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55408/0 (socket says 172.21.15.103:55408) 2024-09-19T07:06:46.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.593+0000 7f9424eae640 1 -- 172.21.15.103:0/1731923235 learned_addr learned my addr 172.21.15.103:0/1731923235 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:46.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.593+0000 7f941ffff640 1 --2- 172.21.15.103:0/1731923235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9420101dd0 0x7f94201983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:46.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.593+0000 7f941f7fe640 1 --2- 172.21.15.103:0/1731923235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9420102780 0x7f94201988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:46.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.593+0000 7f9424eae640 1 -- 172.21.15.103:0/1731923235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9420101dd0 msgr2=0x7f94201983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:46.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.593+0000 7f9424eae640 1 --2- 172.21.15.103:0/1731923235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9420101dd0 0x7f94201983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.593+0000 7f9424eae640 1 -- 172.21.15.103:0/1731923235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9420102780 msgr2=0x7f94201988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:46.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.593+0000 7f9424eae640 1 --2- 172.21.15.103:0/1731923235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9420102780 0x7f94201988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.593+0000 7f9424eae640 1 -- 172.21.15.103:0/1731923235 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942019d350 con 0x7f9420106f00 2024-09-19T07:06:46.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.594+0000 7f9424eae640 1 --2- 172.21.15.103:0/1731923235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9420106f00 0x7f942019cc70 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f941800cce0 tx=0x7f941800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:46.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.594+0000 7f941d7fa640 1 -- 172.21.15.103:0/1731923235 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941800f450 con 0x7f9420106f00 2024-09-19T07:06:46.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.594+0000 7f9426911640 1 -- 172.21.15.103:0/1731923235 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942019d640 con 0x7f9420106f00 2024-09-19T07:06:46.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.594+0000 7f941d7fa640 1 -- 172.21.15.103:0/1731923235 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f941800fad0 con 0x7f9420106f00 2024-09-19T07:06:46.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.594+0000 7f941d7fa640 1 -- 172.21.15.103:0/1731923235 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418018c10 con 0x7f9420106f00 2024-09-19T07:06:46.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.594+0000 7f9426911640 1 -- 172.21.15.103:0/1731923235 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94201a4f20 con 0x7f9420106f00 2024-09-19T07:06:46.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.596+0000 7f941d7fa640 1 -- 172.21.15.103:0/1731923235 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f941800f5f0 con 0x7f9420106f00 2024-09-19T07:06:46.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.596+0000 7f9428cea740 1 Processor -- start 2024-09-19T07:06:46.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.596+0000 7f9428cea740 1 -- start start 2024-09-19T07:06:46.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.596+0000 7f9428cea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e14b40 0x559657e13090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.596+0000 7f9428cea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e13a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f9428cea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e1e370 0x559657e20760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f9428cea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559657d792f0 con 0x559657e135d0 2024-09-19T07:06:46.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f941d7fa640 1 --2- 172.21.15.103:0/1731923235 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f940c0a33b0 0x7f940c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f9428cea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559657d20cf0 con 0x559657e14b40 2024-09-19T07:06:46.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f9428cea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559657d752b0 con 0x559657e1e370 2024-09-19T07:06:46.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f941ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e13a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:46.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f941ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e13a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55410/0 (socket says 172.21.15.103:55410) 2024-09-19T07:06:46.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f941ffff640 1 -- 172.21.15.103:0/1120141113 learned_addr learned my addr 172.21.15.103:0/1120141113 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:46.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f941f7fe640 1 --2- 172.21.15.103:0/1120141113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e1e370 0x559657e20760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:46.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f9424eae640 1 --2- 172.21.15.103:0/1731923235 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f940c0a33b0 0x7f940c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:46.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f941f7fe640 1 --2- 172.21.15.103:0/1120141113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e14b40 0x559657e13090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:46.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f941d7fa640 1 -- 172.21.15.103:0/1731923235 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f94180d02c0 con 0x7f9420106f00 2024-09-19T07:06:46.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f941ffff640 1 -- 172.21.15.103:0/1120141113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e1e370 msgr2=0x559657e20760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:46.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f941ffff640 1 --2- 172.21.15.103:0/1120141113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e1e370 0x559657e20760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f941ffff640 1 -- 172.21.15.103:0/1120141113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e14b40 msgr2=0x559657e13090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:46.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f941ffff640 1 --2- 172.21.15.103:0/1120141113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e14b40 0x559657e13090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f941ffff640 1 -- 172.21.15.103:0/1120141113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559657d053a0 con 0x559657e135d0 2024-09-19T07:06:46.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f941ffff640 1 --2- 172.21.15.103:0/1120141113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e13a10 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f9410009900 tx=0x7f94100316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b783266da8962f3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:46.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f9424eae640 1 --2- 172.21.15.103:0/1731923235 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f940c0a33b0 0x7f940c0a5870 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f94180cdaa0 tx=0x7f9418013810 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:46.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.597+0000 7f94167fc640 1 -- 172.21.15.103:0/1120141113 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410040070 con 0x559657e135d0 2024-09-19T07:06:46.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.598+0000 7f94167fc640 1 -- 172.21.15.103:0/1120141113 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9410005020 con 0x559657e135d0 2024-09-19T07:06:46.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.598+0000 7f94167fc640 1 -- 172.21.15.103:0/1120141113 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410035040 con 0x559657e135d0 2024-09-19T07:06:46.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.598+0000 7f9428cea740 1 -- 172.21.15.103:0/1120141113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 msgr2=0x559657e13a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:46.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.598+0000 7f9428cea740 1 --2- 172.21.15.103:0/1120141113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e13a10 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f9410009900 tx=0x7f94100316e0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.598+0000 7f9428cea740 1 -- 172.21.15.103:0/1120141113 shutdown_connections 2024-09-19T07:06:46.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.598+0000 7f9428cea740 1 --2- 172.21.15.103:0/1120141113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e1e370 0x559657e20760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.598+0000 7f9428cea740 1 --2- 172.21.15.103:0/1120141113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e13a10 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.598+0000 7f9428cea740 1 --2- 172.21.15.103:0/1120141113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e14b40 0x559657e13090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.598+0000 7f9428cea740 1 -- 172.21.15.103:0/1120141113 >> 172.21.15.103:0/1120141113 conn(0x559657ddc310 msgr2=0x559657e0fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:46.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.598+0000 7f9428cea740 1 -- 172.21.15.103:0/1120141113 shutdown_connections 2024-09-19T07:06:46.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.598+0000 7f9428cea740 1 -- 172.21.15.103:0/1120141113 wait complete. 2024-09-19T07:06:46.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f9428cea740 1 -- start start 2024-09-19T07:06:46.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f9428cea740 1 Processor -- start 2024-09-19T07:06:46.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f9428cea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e1dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f9428cea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e14b40 0x559657e1af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f9428cea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e1e370 0x559657e1b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f9428cea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559657e21010 con 0x559657e135d0 2024-09-19T07:06:46.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f9428cea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559657d34060 con 0x559657e14b40 2024-09-19T07:06:46.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f9428cea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559657e21190 con 0x559657e1e370 2024-09-19T07:06:46.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f9428cea740 10 client.0 _ll_register_callbacks cb 0x559657a878c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:46.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f941f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e1dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:46.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f941f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e1dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55426/0 (socket says 172.21.15.103:55426) 2024-09-19T07:06:46.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f941f7fe640 1 -- 172.21.15.103:0/536144117 learned_addr learned my addr 172.21.15.103:0/536144117 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:46.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f941ffff640 1 --2- 172.21.15.103:0/536144117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e14b40 0x559657e1af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:46.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f941f7fe640 1 -- 172.21.15.103:0/536144117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e1e370 msgr2=0x559657e1b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:46.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f941f7fe640 1 --2- 172.21.15.103:0/536144117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e1e370 0x559657e1b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f941f7fe640 1 -- 172.21.15.103:0/536144117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e14b40 msgr2=0x559657e1af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:46.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f941f7fe640 1 --2- 172.21.15.103:0/536144117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e14b40 0x559657e1af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f941f7fe640 1 -- 172.21.15.103:0/536144117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559657aea3c0 con 0x559657e135d0 2024-09-19T07:06:46.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.599+0000 7f941ffff640 1 --2- 172.21.15.103:0/536144117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e14b40 0x559657e1af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:46.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.600+0000 7f941f7fe640 1 --2- 172.21.15.103:0/536144117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e1dce0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f940800ae30 tx=0x7f940800e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb0663bcf511015 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:46.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.600+0000 7f93ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:46.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.600+0000 7f93ff7fe640 1 -- 172.21.15.103:0/536144117 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408004020 con 0x559657e135d0 2024-09-19T07:06:46.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.600+0000 7f93ff7fe640 1 -- 172.21.15.103:0/536144117 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f94080041c0 con 0x559657e135d0 2024-09-19T07:06:46.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.600+0000 7f9428cea740 1 -- 172.21.15.103:0/536144117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9410005020 con 0x559657e135d0 2024-09-19T07:06:46.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.600+0000 7f93ff7fe640 1 -- 172.21.15.103:0/536144117 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f94080043c0 con 0x559657e135d0 2024-09-19T07:06:46.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.600+0000 7f9428cea740 10 client.21633 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:46.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.600+0000 7f9428cea740 1 -- 172.21.15.103:0/536144117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559657e2ca20 con 0x559657e135d0 2024-09-19T07:06:46.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.600+0000 7f93ff7fe640 1 -- 172.21.15.103:0/536144117 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9410005020 con 0x559657e135d0 2024-09-19T07:06:46.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.600+0000 7f93ff7fe640 1 -- 172.21.15.103:0/536144117 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9408004c60 con 0x559657e135d0 2024-09-19T07:06:46.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.600+0000 7f9428cea740 10 client.21633 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:46.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.600+0000 7f9428cea740 10 client.21633 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:06:46.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.600+0000 7f9428cea740 20 client.21633 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:46.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.600+0000 7f9428cea740 1 --2- 172.21.15.103:0/536144117 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x559657e2df30 0x559657e30390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.600+0000 7f9428cea740 4 client.21633 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:06:46.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.600+0000 7f9428cea740 1 -- 172.21.15.103:0/536144117 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559657d86860 con 0x559657e2df30 2024-09-19T07:06:46.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.601+0000 7f941ffff640 1 --2- 172.21.15.103:0/536144117 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x559657e2df30 0x559657e30390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:46.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.601+0000 7f93ff7fe640 10 client.21633 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:06:46.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.601+0000 7f941ffff640 1 --2- 172.21.15.103:0/536144117 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x559657e2df30 0x559657e30390 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7227e0c5678dc6ee server_cookie=2ace4ad2efec4933 in_seq=0 out_seq=0 2024-09-19T07:06:46.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.602+0000 7f93ff7fe640 1 -- 172.21.15.103:0/536144117 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559657d86860 con 0x559657e2df30 2024-09-19T07:06:46.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.602+0000 7f93ff7fe640 10 client.21633 handle_command_reply: tid=0 2024-09-19T07:06:46.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.602+0000 7f9428cea740 1 client.21633 shutdown 2024-09-19T07:06:46.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.602+0000 7f9428cea740 1 -- 172.21.15.103:0/536144117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 msgr2=0x559657e1dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:46.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.602+0000 7f9428cea740 1 --2- 172.21.15.103:0/536144117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e1dce0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f940800ae30 tx=0x7f940800e780 comp rx=0 tx=0).stop 2024-09-19T07:06:46.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.602+0000 7f9428cea740 1 -- 172.21.15.103:0/536144117 shutdown_connections 2024-09-19T07:06:46.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.602+0000 7f9428cea740 1 --2- 172.21.15.103:0/536144117 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x559657e2df30 0x559657e30390 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.602+0000 7f9428cea740 1 --2- 172.21.15.103:0/536144117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e1e370 0x559657e1b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.602+0000 7f9428cea740 1 --2- 172.21.15.103:0/536144117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e14b40 0x559657e1af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.602+0000 7f93ff7fe640 0 client.21633 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:06:46.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.602+0000 7f9428cea740 1 --2- 172.21.15.103:0/536144117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e1dce0 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.602+0000 7f9428cea740 1 -- 172.21.15.103:0/536144117 >> 172.21.15.103:0/536144117 conn(0x559657ddc310 msgr2=0x559657e15930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:46.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.602+0000 7f9428cea740 1 -- 172.21.15.103:0/536144117 shutdown_connections 2024-09-19T07:06:46.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.602+0000 7f9428cea740 1 -- 172.21.15.103:0/536144117 wait complete. 2024-09-19T07:06:46.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.602+0000 7f9428cea740 20 client.21633 trim_cache size 0 max 16384 2024-09-19T07:06:46.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.615+0000 7f9428cea740 1 Processor -- start 2024-09-19T07:06:46.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.615+0000 7f9428cea740 1 -- start start 2024-09-19T07:06:46.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.615+0000 7f9428cea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e135d0 0x559657e0fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.615+0000 7f9428cea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e14b40 0x559657e2a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.615+0000 7f9428cea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e1e370 0x559657e35c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.615+0000 7f9428cea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94201ad7e0 con 0x559657e14b40 2024-09-19T07:06:46.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.615+0000 7f9428cea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f94201af320 con 0x559657e135d0 2024-09-19T07:06:46.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.615+0000 7f9428cea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94201adeb0 con 0x559657e1e370 2024-09-19T07:06:46.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.615+0000 7f941ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e14b40 0x559657e2a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:46.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.615+0000 7f941ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e14b40 0x559657e2a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55428/0 (socket says 172.21.15.103:55428) 2024-09-19T07:06:46.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.615+0000 7f941ffff640 1 -- 172.21.15.103:0/1053834619 learned_addr learned my addr 172.21.15.103:0/1053834619 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:46.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.616+0000 7f941ffff640 1 -- 172.21.15.103:0/1053834619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e1e370 msgr2=0x559657e35c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:46.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.616+0000 7f941ffff640 1 --2- 172.21.15.103:0/1053834619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e1e370 0x559657e35c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.616+0000 7f941f7fe640 1 --2- 172.21.15.103:0/1053834619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e135d0 0x559657e0fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:46.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.616+0000 7f941ffff640 1 -- 172.21.15.103:0/1053834619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e135d0 msgr2=0x559657e0fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:46.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.616+0000 7f941ffff640 1 --2- 172.21.15.103:0/1053834619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e135d0 0x559657e0fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.616+0000 7f941ffff640 1 -- 172.21.15.103:0/1053834619 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559657e194e0 con 0x559657e14b40 2024-09-19T07:06:46.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.616+0000 7f941ffff640 1 --2- 172.21.15.103:0/1053834619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e14b40 0x559657e2a2b0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f9410009900 tx=0x7f9410005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adf1db6b0b3fb96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:46.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.616+0000 7f9415ffb640 1 -- 172.21.15.103:0/1053834619 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410035040 con 0x559657e14b40 2024-09-19T07:06:46.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.616+0000 7f9415ffb640 1 -- 172.21.15.103:0/1053834619 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9410005500 con 0x559657e14b40 2024-09-19T07:06:46.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.616+0000 7f9415ffb640 1 -- 172.21.15.103:0/1053834619 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410040040 con 0x559657e14b40 2024-09-19T07:06:46.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.617+0000 7f9428cea740 1 -- 172.21.15.103:0/1053834619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e14b40 msgr2=0x559657e2a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:46.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.617+0000 7f9428cea740 1 --2- 172.21.15.103:0/1053834619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e14b40 0x559657e2a2b0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f9410009900 tx=0x7f9410005fd0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.617+0000 7f9428cea740 1 -- 172.21.15.103:0/1053834619 shutdown_connections 2024-09-19T07:06:46.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.617+0000 7f9428cea740 1 --2- 172.21.15.103:0/1053834619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e1e370 0x559657e35c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.617+0000 7f9428cea740 1 --2- 172.21.15.103:0/1053834619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e14b40 0x559657e2a2b0 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.617+0000 7f9428cea740 1 --2- 172.21.15.103:0/1053834619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e135d0 0x559657e0fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.617+0000 7f9428cea740 1 -- 172.21.15.103:0/1053834619 >> 172.21.15.103:0/1053834619 conn(0x559657ddc310 msgr2=0x559657e0f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:46.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.617+0000 7f9428cea740 1 -- 172.21.15.103:0/1053834619 shutdown_connections 2024-09-19T07:06:46.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.617+0000 7f9428cea740 1 -- 172.21.15.103:0/1053834619 wait complete. 2024-09-19T07:06:46.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.617+0000 7f9428cea740 1 -- start start 2024-09-19T07:06:46.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.617+0000 7f9428cea740 1 Processor -- start 2024-09-19T07:06:46.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f9428cea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e26b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f9428cea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e14b40 0x559657e17c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f9428cea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e1e370 0x559657e181a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f9428cea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94201aec50 con 0x559657e135d0 2024-09-19T07:06:46.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f9428cea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f94201ae580 con 0x559657e1e370 2024-09-19T07:06:46.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f9428cea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94201ab7e0 con 0x559657e14b40 2024-09-19T07:06:46.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f9428cea740 10 client.0 _ll_register_callbacks cb 0x7f940800fab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:46.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f941f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e26b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:46.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f941f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e26b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55432/0 (socket says 172.21.15.103:55432) 2024-09-19T07:06:46.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f941f7fe640 1 -- 172.21.15.103:0/3851503958 learned_addr learned my addr 172.21.15.103:0/3851503958 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:46.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f941ffff640 1 --2- 172.21.15.103:0/3851503958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e14b40 0x559657e17c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:46.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f941f7fe640 1 -- 172.21.15.103:0/3851503958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e14b40 msgr2=0x559657e17c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:46.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f941f7fe640 1 --2- 172.21.15.103:0/3851503958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e14b40 0x559657e17c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f941f7fe640 1 -- 172.21.15.103:0/3851503958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e1e370 msgr2=0x559657e181a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:46.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f941f7fe640 1 --2- 172.21.15.103:0/3851503958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e1e370 0x559657e181a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f941f7fe640 1 -- 172.21.15.103:0/3851503958 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559657e189a0 con 0x559657e135d0 2024-09-19T07:06:46.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f941f7fe640 1 --2- 172.21.15.103:0/3851503958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e26b30 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f940800aae0 tx=0x7f940800a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb31320ec902bf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:46.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f93ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:46.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f93ff7fe640 1 -- 172.21.15.103:0/3851503958 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408005000 con 0x559657e135d0 2024-09-19T07:06:46.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f93ff7fe640 1 -- 172.21.15.103:0/3851503958 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f94080051a0 con 0x559657e135d0 2024-09-19T07:06:46.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.618+0000 7f9428cea740 1 -- 172.21.15.103:0/3851503958 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9410005500 con 0x559657e135d0 2024-09-19T07:06:46.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.619+0000 7f93ff7fe640 1 -- 172.21.15.103:0/3851503958 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f94080053a0 con 0x559657e135d0 2024-09-19T07:06:46.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.619+0000 7f9428cea740 10 client.21639 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:46.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.619+0000 7f9428cea740 1 -- 172.21.15.103:0/3851503958 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559657e98bd0 con 0x559657e135d0 2024-09-19T07:06:46.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.619+0000 7f93ff7fe640 1 -- 172.21.15.103:0/3851503958 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9410005500 con 0x559657e135d0 2024-09-19T07:06:46.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.619+0000 7f93ff7fe640 1 -- 172.21.15.103:0/3851503958 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9408003ba0 con 0x559657e135d0 2024-09-19T07:06:46.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.619+0000 7f9428cea740 10 client.21639 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:46.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.619+0000 7f9428cea740 10 client.21639 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T07:06:46.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.619+0000 7f9428cea740 20 client.21639 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:46.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.619+0000 7f9428cea740 1 --2- 172.21.15.103:0/3851503958 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x559657e99350 0x559657e9b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:46.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.619+0000 7f9428cea740 4 client.21639 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:06:46.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.619+0000 7f9428cea740 1 -- 172.21.15.103:0/3851503958 --> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559657d80d70 con 0x559657e99350 2024-09-19T07:06:46.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.619+0000 7f941ffff640 1 --2- 172.21.15.103:0/3851503958 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x559657e99350 0x559657e9b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:46.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f93ff7fe640 10 client.21639 ms_handle_connect on v2:172.21.15.134:6836/1590453330 2024-09-19T07:06:46.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f941ffff640 1 --2- 172.21.15.103:0/3851503958 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x559657e99350 0x559657e9b770 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a1ba1baefc1368d server_cookie=8108028be5a06eb8 in_seq=0 out_seq=0 2024-09-19T07:06:46.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f93ff7fe640 1 -- 172.21.15.103:0/3851503958 <== mds.1 v2:172.21.15.134:6836/1590453330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559657d80d70 con 0x559657e99350 2024-09-19T07:06:46.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f93ff7fe640 10 client.21639 handle_command_reply: tid=0 2024-09-19T07:06:46.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f9428cea740 1 client.21639 shutdown 2024-09-19T07:06:46.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f9428cea740 1 -- 172.21.15.103:0/3851503958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 msgr2=0x559657e26b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:46.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f9428cea740 1 --2- 172.21.15.103:0/3851503958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e26b30 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f940800aae0 tx=0x7f940800a7c0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.643 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:06:46.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f9428cea740 1 -- 172.21.15.103:0/3851503958 shutdown_connections 2024-09-19T07:06:46.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f9428cea740 1 --2- 172.21.15.103:0/3851503958 >> [v2:172.21.15.134:6836/1590453330,v1:172.21.15.134:6837/1590453330] conn(0x559657e99350 0x559657e9b770 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f9428cea740 1 --2- 172.21.15.103:0/3851503958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559657e1e370 0x559657e181a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f9428cea740 1 --2- 172.21.15.103:0/3851503958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559657e14b40 0x559657e17c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f93ff7fe640 0 client.21639 ms_handle_reset on v2:172.21.15.134:6836/1590453330 2024-09-19T07:06:46.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f9428cea740 1 --2- 172.21.15.103:0/3851503958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559657e135d0 0x559657e26b30 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f9428cea740 1 -- 172.21.15.103:0/3851503958 >> 172.21.15.103:0/3851503958 conn(0x559657ddc310 msgr2=0x559657e15110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:46.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f9428cea740 1 -- 172.21.15.103:0/3851503958 shutdown_connections 2024-09-19T07:06:46.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f9428cea740 1 -- 172.21.15.103:0/3851503958 wait complete. 2024-09-19T07:06:46.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.620+0000 7f9428cea740 20 client.21639 trim_cache size 0 max 16384 2024-09-19T07:06:46.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.621+0000 7f9426911640 1 -- 172.21.15.103:0/1731923235 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f940c0a33b0 msgr2=0x7f940c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:46.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.621+0000 7f9426911640 1 --2- 172.21.15.103:0/1731923235 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f940c0a33b0 0x7f940c0a5870 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f94180cdaa0 tx=0x7f9418013810 comp rx=0 tx=0).stop 2024-09-19T07:06:46.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.621+0000 7f9426911640 1 -- 172.21.15.103:0/1731923235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9420106f00 msgr2=0x7f942019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:46.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.621+0000 7f9426911640 1 --2- 172.21.15.103:0/1731923235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9420106f00 0x7f942019cc70 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f941800cce0 tx=0x7f941800a620 comp rx=0 tx=0).stop 2024-09-19T07:06:46.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.621+0000 7f9426911640 1 -- 172.21.15.103:0/1731923235 shutdown_connections 2024-09-19T07:06:46.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.621+0000 7f9426911640 1 --2- 172.21.15.103:0/1731923235 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f940c0a33b0 0x7f940c0a5870 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.621+0000 7f9426911640 1 --2- 172.21.15.103:0/1731923235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9420106f00 0x7f942019cc70 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.621+0000 7f9426911640 1 --2- 172.21.15.103:0/1731923235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9420102780 0x7f94201988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.621+0000 7f9426911640 1 --2- 172.21.15.103:0/1731923235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9420101dd0 0x7f94201983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:46.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.621+0000 7f9426911640 1 -- 172.21.15.103:0/1731923235 >> 172.21.15.103:0/1731923235 conn(0x7f94200fd890 msgr2=0x7f94200fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:46.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.621+0000 7f9426911640 1 -- 172.21.15.103:0/1731923235 shutdown_connections 2024-09-19T07:06:46.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:46.621+0000 7f9426911640 1 -- 172.21.15.103:0/1731923235 wait complete. 2024-09-19T07:06:47.349 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:06:47.595 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:48.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.044+0000 7f4f5903f640 1 -- 172.21.15.103:0/3885072494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f54105a90 msgr2=0x7f4f54105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:48.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.044+0000 7f4f5903f640 1 --2- 172.21.15.103:0/3885072494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f54105a90 0x7f4f54105e70 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009900 tx=0x7f4f4402a410 comp rx=0 tx=0).stop 2024-09-19T07:06:48.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.044+0000 7f4f5903f640 1 -- 172.21.15.103:0/3885072494 shutdown_connections 2024-09-19T07:06:48.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.044+0000 7f4f5903f640 1 --2- 172.21.15.103:0/3885072494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f54101bc0 0x7f4f5410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.044+0000 7f4f5903f640 1 --2- 172.21.15.103:0/3885072494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f54101240 0x7f4f54101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.044+0000 7f4f5903f640 1 --2- 172.21.15.103:0/3885072494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f54105a90 0x7f4f54105e70 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.044+0000 7f4f5903f640 1 -- 172.21.15.103:0/3885072494 >> 172.21.15.103:0/3885072494 conn(0x7f4f54077210 msgr2=0x7f4f540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:48.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.044+0000 7f4f5903f640 1 -- 172.21.15.103:0/3885072494 shutdown_connections 2024-09-19T07:06:48.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.044+0000 7f4f5903f640 1 -- 172.21.15.103:0/3885072494 wait complete. 2024-09-19T07:06:48.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.045+0000 7f4f5903f640 1 Processor -- start 2024-09-19T07:06:48.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.045+0000 7f4f5903f640 1 -- start start 2024-09-19T07:06:48.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.045+0000 7f4f5903f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f54101240 0x7f4f54199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.045+0000 7f4f5903f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f54101bc0 0x7f4f54199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.045+0000 7f4f5903f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f54105a90 0x7f4f541a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.045+0000 7f4f5903f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54110fb0 con 0x7f4f54101240 2024-09-19T07:06:48.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.045+0000 7f4f5903f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54110e30 con 0x7f4f54101bc0 2024-09-19T07:06:48.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.045+0000 7f4f5903f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54111130 con 0x7f4f54105a90 2024-09-19T07:06:48.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.045+0000 7f4f52d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f54101240 0x7f4f54199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.045+0000 7f4f52d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f54101240 0x7f4f54199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52308/0 (socket says 172.21.15.103:52308) 2024-09-19T07:06:48.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.045+0000 7f4f52d9d640 1 -- 172.21.15.103:0/1052795572 learned_addr learned my addr 172.21.15.103:0/1052795572 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:48.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.045+0000 7f4f5359e640 1 --2- 172.21.15.103:0/1052795572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f54105a90 0x7f4f541a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.045+0000 7f4f5259c640 1 --2- 172.21.15.103:0/1052795572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f54101bc0 0x7f4f54199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.046+0000 7f4f52d9d640 1 -- 172.21.15.103:0/1052795572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f54105a90 msgr2=0x7f4f541a44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:48.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.046+0000 7f4f52d9d640 1 --2- 172.21.15.103:0/1052795572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f54105a90 0x7f4f541a44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.046+0000 7f4f52d9d640 1 -- 172.21.15.103:0/1052795572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f54101bc0 msgr2=0x7f4f54199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:48.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.046+0000 7f4f52d9d640 1 --2- 172.21.15.103:0/1052795572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f54101bc0 0x7f4f54199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.046+0000 7f4f52d9d640 1 -- 172.21.15.103:0/1052795572 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f541a4a00 con 0x7f4f54101240 2024-09-19T07:06:48.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.046+0000 7f4f52d9d640 1 --2- 172.21.15.103:0/1052795572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f54101240 0x7f4f54199440 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f4f44006010 tx=0x7f4f44003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:48.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.046+0000 7f4f3bfff640 1 -- 172.21.15.103:0/1052795572 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4403f070 con 0x7f4f54101240 2024-09-19T07:06:48.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.046+0000 7f4f3bfff640 1 -- 172.21.15.103:0/1052795572 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f44031e30 con 0x7f4f54101240 2024-09-19T07:06:48.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.046+0000 7f4f5903f640 1 -- 172.21.15.103:0/1052795572 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f541a4bd0 con 0x7f4f54101240 2024-09-19T07:06:48.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.046+0000 7f4f3bfff640 1 -- 172.21.15.103:0/1052795572 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44035030 con 0x7f4f54101240 2024-09-19T07:06:48.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.046+0000 7f4f5903f640 1 -- 172.21.15.103:0/1052795572 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f541a5020 con 0x7f4f54101240 2024-09-19T07:06:48.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.048+0000 7f4f3bfff640 1 -- 172.21.15.103:0/1052795572 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f4f44042070 con 0x7f4f54101240 2024-09-19T07:06:48.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5b418740 1 Processor -- start 2024-09-19T07:06:48.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5b418740 1 -- start start 2024-09-19T07:06:48.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5b418740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff91b40 0x557d1ff90090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5b418740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff905d0 0x557d1ff90a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5b418740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff9f780 0x557d1ffa1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5b418740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557d1fef62f0 con 0x557d1ff9f780 2024-09-19T07:06:48.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5b418740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d1fe9dcf0 con 0x557d1ff905d0 2024-09-19T07:06:48.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5b418740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d1fef22b0 con 0x557d1ff91b40 2024-09-19T07:06:48.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff9f780 0x557d1ffa1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff9f780 0x557d1ffa1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52312/0 (socket says 172.21.15.103:52312) 2024-09-19T07:06:48.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5259c640 1 -- 172.21.15.103:0/2712450209 learned_addr learned my addr 172.21.15.103:0/2712450209 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:48.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f3bfff640 1 --2- 172.21.15.103:0/1052795572 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4f3c0a33b0 0x7f4f3c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f52d9d640 1 --2- 172.21.15.103:0/2712450209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff905d0 0x557d1ff90a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5359e640 1 --2- 172.21.15.103:0/2712450209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff91b40 0x557d1ff90090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5359e640 1 --2- 172.21.15.103:0/1052795572 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4f3c0a33b0 0x7f4f3c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5259c640 1 -- 172.21.15.103:0/2712450209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff91b40 msgr2=0x557d1ff90090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:48.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5259c640 1 --2- 172.21.15.103:0/2712450209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff91b40 0x557d1ff90090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5259c640 1 -- 172.21.15.103:0/2712450209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff905d0 msgr2=0x557d1ff90a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:48.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5259c640 1 --2- 172.21.15.103:0/2712450209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff905d0 0x557d1ff90a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.049+0000 7f4f5259c640 1 -- 172.21.15.103:0/2712450209 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d1fe823a0 con 0x557d1ff9f780 2024-09-19T07:06:48.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.050+0000 7f4f3bfff640 1 -- 172.21.15.103:0/1052795572 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f4f440f0690 con 0x7f4f54101240 2024-09-19T07:06:48.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.050+0000 7f4f5259c640 1 --2- 172.21.15.103:0/2712450209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff9f780 0x557d1ffa1b70 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800b7f0 tx=0x7f4f4800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ca13b1d6813029 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:48.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.050+0000 7f4f397fa640 1 -- 172.21.15.103:0/2712450209 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4800d950 con 0x557d1ff9f780 2024-09-19T07:06:48.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.050+0000 7f4f5359e640 1 --2- 172.21.15.103:0/1052795572 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4f3c0a33b0 0x7f4f3c0a5870 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c005c40 tx=0x7f4f4c00a4d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:48.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.050+0000 7f4f397fa640 1 -- 172.21.15.103:0/2712450209 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f48018070 con 0x557d1ff9f780 2024-09-19T07:06:48.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.050+0000 7f4f397fa640 1 -- 172.21.15.103:0/2712450209 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f480123e0 con 0x557d1ff9f780 2024-09-19T07:06:48.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.051+0000 7f4f5b418740 1 -- 172.21.15.103:0/2712450209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff9f780 msgr2=0x557d1ffa1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:48.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.051+0000 7f4f5b418740 1 --2- 172.21.15.103:0/2712450209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff9f780 0x557d1ffa1b70 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800b7f0 tx=0x7f4f4800bcb0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.051+0000 7f4f5b418740 1 -- 172.21.15.103:0/2712450209 shutdown_connections 2024-09-19T07:06:48.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.051+0000 7f4f5b418740 1 --2- 172.21.15.103:0/2712450209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff9f780 0x557d1ffa1b70 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.051+0000 7f4f5b418740 1 --2- 172.21.15.103:0/2712450209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff905d0 0x557d1ff90a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.051+0000 7f4f5b418740 1 --2- 172.21.15.103:0/2712450209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff91b40 0x557d1ff90090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.051+0000 7f4f5b418740 1 -- 172.21.15.103:0/2712450209 >> 172.21.15.103:0/2712450209 conn(0x557d1ff59310 msgr2=0x557d1ff8cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:48.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.051+0000 7f4f5b418740 1 -- 172.21.15.103:0/2712450209 shutdown_connections 2024-09-19T07:06:48.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.051+0000 7f4f5b418740 1 -- 172.21.15.103:0/2712450209 wait complete. 2024-09-19T07:06:48.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.051+0000 7f4f5b418740 1 -- start start 2024-09-19T07:06:48.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.051+0000 7f4f5b418740 1 Processor -- start 2024-09-19T07:06:48.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5b418740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff905d0 0x557d1ff9f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5b418740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff91b40 0x557d1ff9c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5b418740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff9f780 0x557d1ff9c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5b418740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557d1ffa2480 con 0x557d1ff91b40 2024-09-19T07:06:48.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5b418740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d1feb1060 con 0x557d1ff9f780 2024-09-19T07:06:48.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5b418740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d1ffa2600 con 0x557d1ff905d0 2024-09-19T07:06:48.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5b418740 10 client.0 _ll_register_callbacks cb 0x557d1fc048c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:48.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff91b40 0x557d1ff9c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff91b40 0x557d1ff9c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52320/0 (socket says 172.21.15.103:52320) 2024-09-19T07:06:48.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5259c640 1 -- 172.21.15.103:0/3860685282 learned_addr learned my addr 172.21.15.103:0/3860685282 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:48.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f52d9d640 1 --2- 172.21.15.103:0/3860685282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff905d0 0x557d1ff9f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5359e640 1 --2- 172.21.15.103:0/3860685282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff9f780 0x557d1ff9c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5259c640 1 -- 172.21.15.103:0/3860685282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff905d0 msgr2=0x557d1ff9f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:48.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5259c640 1 --2- 172.21.15.103:0/3860685282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff905d0 0x557d1ff9f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5259c640 1 -- 172.21.15.103:0/3860685282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff9f780 msgr2=0x557d1ff9c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:48.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5259c640 1 --2- 172.21.15.103:0/3860685282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff9f780 0x557d1ff9c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5259c640 1 -- 172.21.15.103:0/3860685282 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d1fc673c0 con 0x557d1ff91b40 2024-09-19T07:06:48.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5259c640 1 --2- 172.21.15.103:0/3860685282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff91b40 0x557d1ff9c320 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f4f48008010 tx=0x7f4f48013730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafcf1c65cda3a0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:48.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:48.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f36ff5640 1 -- 172.21.15.103:0/3860685282 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48023070 con 0x557d1ff91b40 2024-09-19T07:06:48.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f36ff5640 1 -- 172.21.15.103:0/3860685282 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f4f4800dd60 con 0x557d1ff91b40 2024-09-19T07:06:48.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.052+0000 7f4f5b418740 1 -- 172.21.15.103:0/3860685282 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f48018070 con 0x557d1ff91b40 2024-09-19T07:06:48.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.053+0000 7f4f36ff5640 1 -- 172.21.15.103:0/3860685282 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4f4802f050 con 0x557d1ff91b40 2024-09-19T07:06:48.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.053+0000 7f4f5b418740 10 client.21651 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:48.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.053+0000 7f4f5b418740 1 -- 172.21.15.103:0/3860685282 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557d1ffade90 con 0x557d1ff91b40 2024-09-19T07:06:48.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.053+0000 7f4f36ff5640 1 -- 172.21.15.103:0/3860685282 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f48018070 con 0x557d1ff91b40 2024-09-19T07:06:48.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.053+0000 7f4f36ff5640 1 -- 172.21.15.103:0/3860685282 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4f4801b540 con 0x557d1ff91b40 2024-09-19T07:06:48.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.053+0000 7f4f5b418740 10 client.21651 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:48.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.053+0000 7f4f5b418740 10 client.21651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:06:48.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.053+0000 7f4f5b418740 20 client.21651 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:48.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.053+0000 7f4f5b418740 1 --2- 172.21.15.103:0/3860685282 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557d1ffaf1e0 0x557d1ffb15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.053+0000 7f4f5b418740 4 client.21651 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:06:48.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.053+0000 7f4f5b418740 1 -- 172.21.15.103:0/3860685282 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d1ff03860 con 0x557d1ffaf1e0 2024-09-19T07:06:48.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.053+0000 7f4f52d9d640 1 --2- 172.21.15.103:0/3860685282 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557d1ffaf1e0 0x557d1ffb15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.054+0000 7f4f52d9d640 1 --2- 172.21.15.103:0/3860685282 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557d1ffaf1e0 0x557d1ffb15d0 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eb54d3766ec7c2a server_cookie=d1e4dd04c944da3c in_seq=0 out_seq=0 2024-09-19T07:06:48.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.054+0000 7f4f36ff5640 10 client.21651 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:48.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.055+0000 7f4f36ff5640 1 -- 172.21.15.103:0/3860685282 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d1ff03860 con 0x557d1ffaf1e0 2024-09-19T07:06:48.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.055+0000 7f4f36ff5640 10 client.21651 handle_command_reply: tid=0 2024-09-19T07:06:48.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.055+0000 7f4f5b418740 1 client.21651 shutdown 2024-09-19T07:06:48.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.055+0000 7f4f5b418740 1 -- 172.21.15.103:0/3860685282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff91b40 msgr2=0x557d1ff9c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:48.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.055+0000 7f4f5b418740 1 --2- 172.21.15.103:0/3860685282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff91b40 0x557d1ff9c320 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f4f48008010 tx=0x7f4f48013730 comp rx=0 tx=0).stop 2024-09-19T07:06:48.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.055+0000 7f4f5b418740 1 -- 172.21.15.103:0/3860685282 shutdown_connections 2024-09-19T07:06:48.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.055+0000 7f4f5b418740 1 --2- 172.21.15.103:0/3860685282 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557d1ffaf1e0 0x557d1ffb15d0 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.055+0000 7f4f5b418740 1 --2- 172.21.15.103:0/3860685282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff9f780 0x557d1ff9c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.055+0000 7f4f5b418740 1 --2- 172.21.15.103:0/3860685282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff91b40 0x557d1ff9c320 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.055+0000 7f4f5b418740 1 --2- 172.21.15.103:0/3860685282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff905d0 0x557d1ff9f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.055+0000 7f4f36ff5640 0 client.21651 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:48.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.055+0000 7f4f5b418740 1 -- 172.21.15.103:0/3860685282 >> 172.21.15.103:0/3860685282 conn(0x557d1ff59310 msgr2=0x557d1ff92960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:48.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.055+0000 7f4f5b418740 1 -- 172.21.15.103:0/3860685282 shutdown_connections 2024-09-19T07:06:48.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.055+0000 7f4f5b418740 1 -- 172.21.15.103:0/3860685282 wait complete. 2024-09-19T07:06:48.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.055+0000 7f4f5b418740 20 client.21651 trim_cache size 0 max 16384 2024-09-19T07:06:48.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5b418740 1 Processor -- start 2024-09-19T07:06:48.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5b418740 1 -- start start 2024-09-19T07:06:48.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5b418740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff905d0 0x557d1ffab780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5b418740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff91b40 0x557d1ffb6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5b418740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff9f780 0x557d1ffb71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5b418740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541ab720 con 0x557d1ff9f780 2024-09-19T07:06:48.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5b418740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541ad260 con 0x557d1ff91b40 2024-09-19T07:06:48.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5b418740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541abdf0 con 0x557d1ff905d0 2024-09-19T07:06:48.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5359e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff9f780 0x557d1ffb71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5359e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff9f780 0x557d1ffb71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52326/0 (socket says 172.21.15.103:52326) 2024-09-19T07:06:48.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5359e640 1 -- 172.21.15.103:0/2812231978 learned_addr learned my addr 172.21.15.103:0/2812231978 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:48.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5259c640 1 --2- 172.21.15.103:0/2812231978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff91b40 0x557d1ffb6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f52d9d640 1 --2- 172.21.15.103:0/2812231978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff905d0 0x557d1ffab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5359e640 1 -- 172.21.15.103:0/2812231978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff905d0 msgr2=0x557d1ffab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:48.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5359e640 1 --2- 172.21.15.103:0/2812231978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff905d0 0x557d1ffab780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5359e640 1 -- 172.21.15.103:0/2812231978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff91b40 msgr2=0x557d1ffb6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:48.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5359e640 1 --2- 172.21.15.103:0/2812231978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff91b40 0x557d1ffb6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5359e640 1 -- 172.21.15.103:0/2812231978 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d1ff9f3e0 con 0x557d1ff9f780 2024-09-19T07:06:48.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.068+0000 7f4f5259c640 1 --2- 172.21.15.103:0/2812231978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff91b40 0x557d1ffb6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:06:48.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.069+0000 7f4f5359e640 1 --2- 172.21.15.103:0/2812231978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff9f780 0x557d1ffb71a0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c012970 tx=0x7f4f4c013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523420756faedd62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:48.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.069+0000 7f4f38ff9640 1 -- 172.21.15.103:0/2812231978 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c003970 con 0x557d1ff9f780 2024-09-19T07:06:48.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.069+0000 7f4f38ff9640 1 -- 172.21.15.103:0/2812231978 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c01c900 con 0x557d1ff9f780 2024-09-19T07:06:48.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.069+0000 7f4f38ff9640 1 -- 172.21.15.103:0/2812231978 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c004770 con 0x557d1ff9f780 2024-09-19T07:06:48.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.069+0000 7f4f5b418740 1 -- 172.21.15.103:0/2812231978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff9f780 msgr2=0x557d1ffb71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:48.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.069+0000 7f4f5b418740 1 --2- 172.21.15.103:0/2812231978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff9f780 0x557d1ffb71a0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c012970 tx=0x7f4f4c013070 comp rx=0 tx=0).stop 2024-09-19T07:06:48.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.069+0000 7f4f5b418740 1 -- 172.21.15.103:0/2812231978 shutdown_connections 2024-09-19T07:06:48.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.069+0000 7f4f5b418740 1 --2- 172.21.15.103:0/2812231978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff9f780 0x557d1ffb71a0 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.069+0000 7f4f5b418740 1 --2- 172.21.15.103:0/2812231978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff91b40 0x557d1ffb6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.069+0000 7f4f5b418740 1 --2- 172.21.15.103:0/2812231978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff905d0 0x557d1ffab780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.069+0000 7f4f5b418740 1 -- 172.21.15.103:0/2812231978 >> 172.21.15.103:0/2812231978 conn(0x557d1ff59310 msgr2=0x557d1ff9e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:48.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.069+0000 7f4f5b418740 1 -- 172.21.15.103:0/2812231978 shutdown_connections 2024-09-19T07:06:48.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.069+0000 7f4f5b418740 1 -- 172.21.15.103:0/2812231978 wait complete. 2024-09-19T07:06:48.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f5b418740 1 -- start start 2024-09-19T07:06:48.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f5b418740 1 Processor -- start 2024-09-19T07:06:48.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f5b418740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff905d0 0x557d1ffa7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f5b418740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff91b40 0x557d1ffa3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f5b418740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff9f780 0x557d1ffa37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f5b418740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541acb90 con 0x557d1ff905d0 2024-09-19T07:06:48.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f5b418740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541ac4c0 con 0x557d1ff9f780 2024-09-19T07:06:48.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f5b418740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541a9700 con 0x557d1ff91b40 2024-09-19T07:06:48.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f5b418740 10 client.0 _ll_register_callbacks cb 0x7f4f4800b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:06:48.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f52d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff905d0 0x557d1ffa7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f52d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff905d0 0x557d1ffa7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52340/0 (socket says 172.21.15.103:52340) 2024-09-19T07:06:48.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f52d9d640 1 -- 172.21.15.103:0/1099956270 learned_addr learned my addr 172.21.15.103:0/1099956270 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:48.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f5359e640 1 --2- 172.21.15.103:0/1099956270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff9f780 0x557d1ffa37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f5259c640 1 --2- 172.21.15.103:0/1099956270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff91b40 0x557d1ffa3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f52d9d640 1 -- 172.21.15.103:0/1099956270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff91b40 msgr2=0x557d1ffa3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:48.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f52d9d640 1 --2- 172.21.15.103:0/1099956270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff91b40 0x557d1ffa3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f52d9d640 1 -- 172.21.15.103:0/1099956270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff9f780 msgr2=0x557d1ffa37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:06:48.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f52d9d640 1 --2- 172.21.15.103:0/1099956270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff9f780 0x557d1ffa37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.070+0000 7f4f52d9d640 1 -- 172.21.15.103:0/1099956270 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d1ffa4000 con 0x557d1ff905d0 2024-09-19T07:06:48.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.071+0000 7f4f52d9d640 1 --2- 172.21.15.103:0/1099956270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff905d0 0x557d1ffa7f60 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f4f440b8db0 tx=0x7f4f440f2ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b61fb006dd9243 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:48.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.071+0000 7f4f36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:06:48.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.071+0000 7f4f36ff5640 1 -- 172.21.15.103:0/1099956270 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44049040 con 0x557d1ff905d0 2024-09-19T07:06:48.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.071+0000 7f4f36ff5640 1 -- 172.21.15.103:0/1099956270 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f4f440b8fc0 con 0x557d1ff905d0 2024-09-19T07:06:48.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.071+0000 7f4f5b418740 1 -- 172.21.15.103:0/1099956270 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f4c01c900 con 0x557d1ff905d0 2024-09-19T07:06:48.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.071+0000 7f4f36ff5640 1 -- 172.21.15.103:0/1099956270 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4f440a70f0 con 0x557d1ff905d0 2024-09-19T07:06:48.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.071+0000 7f4f5b418740 10 client.21657 fetch_fsmap learned FSMap version 27 2024-09-19T07:06:48.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.071+0000 7f4f5b418740 1 -- 172.21.15.103:0/1099956270 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557d20019bc0 con 0x557d1ff905d0 2024-09-19T07:06:48.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.071+0000 7f4f36ff5640 1 -- 172.21.15.103:0/1099956270 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c01c900 con 0x557d1ff905d0 2024-09-19T07:06:48.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.071+0000 7f4f36ff5640 1 -- 172.21.15.103:0/1099956270 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4f4403b5f0 con 0x557d1ff905d0 2024-09-19T07:06:48.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.071+0000 7f4f5b418740 10 client.21657 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:06:48.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.071+0000 7f4f5b418740 10 client.21657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:06:48.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.071+0000 7f4f5b418740 20 client.21657 populate_metadata read hostname 'smithi103' 2024-09-19T07:06:48.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.071+0000 7f4f5b418740 1 --2- 172.21.15.103:0/1099956270 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557d2001a330 0x557d2001c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.071+0000 7f4f5b418740 4 client.21657 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:06:48.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.071+0000 7f4f5b418740 1 -- 172.21.15.103:0/1099956270 --> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d1fefdd70 con 0x557d2001a330 2024-09-19T07:06:48.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f5259c640 1 --2- 172.21.15.103:0/1099956270 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557d2001a330 0x557d2001c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f5259c640 1 --2- 172.21.15.103:0/1099956270 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557d2001a330 0x557d2001c720 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d40ee38ceacb932 server_cookie=56f781064a92ff2f in_seq=0 out_seq=0 2024-09-19T07:06:48.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f36ff5640 10 client.21657 ms_handle_connect on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:48.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f36ff5640 1 -- 172.21.15.103:0/1099956270 <== mds.0 v2:172.21.15.134:6838/2333076106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d1fefdd70 con 0x557d2001a330 2024-09-19T07:06:48.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f36ff5640 10 client.21657 handle_command_reply: tid=0 2024-09-19T07:06:48.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f5b418740 1 client.21657 shutdown 2024-09-19T07:06:48.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f5b418740 1 -- 172.21.15.103:0/1099956270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff905d0 msgr2=0x557d1ffa7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:48.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f5b418740 1 --2- 172.21.15.103:0/1099956270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff905d0 0x557d1ffa7f60 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f4f440b8db0 tx=0x7f4f440f2ba0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f5b418740 1 -- 172.21.15.103:0/1099956270 shutdown_connections 2024-09-19T07:06:48.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f5b418740 1 --2- 172.21.15.103:0/1099956270 >> [v2:172.21.15.134:6838/2333076106,v1:172.21.15.134:6839/2333076106] conn(0x557d2001a330 0x557d2001c720 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f5b418740 1 --2- 172.21.15.103:0/1099956270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d1ff9f780 0x557d1ffa37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f5b418740 1 --2- 172.21.15.103:0/1099956270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d1ff91b40 0x557d1ffa3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f5b418740 1 --2- 172.21.15.103:0/1099956270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d1ff905d0 0x557d1ffa7f60 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f5b418740 1 -- 172.21.15.103:0/1099956270 >> 172.21.15.103:0/1099956270 conn(0x557d1ff59310 msgr2=0x557d1ff8b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:48.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f36ff5640 0 client.21657 ms_handle_reset on v2:172.21.15.134:6838/2333076106 2024-09-19T07:06:48.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f5b418740 1 -- 172.21.15.103:0/1099956270 shutdown_connections 2024-09-19T07:06:48.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f5b418740 1 -- 172.21.15.103:0/1099956270 wait complete. 2024-09-19T07:06:48.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.072+0000 7f4f5b418740 20 client.21657 trim_cache size 0 max 16384 2024-09-19T07:06:48.081 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:06:48.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.073+0000 7f4f5903f640 1 -- 172.21.15.103:0/1052795572 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4f3c0a33b0 msgr2=0x7f4f3c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:48.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.073+0000 7f4f5903f640 1 --2- 172.21.15.103:0/1052795572 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4f3c0a33b0 0x7f4f3c0a5870 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c005c40 tx=0x7f4f4c00a4d0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.073+0000 7f4f5903f640 1 -- 172.21.15.103:0/1052795572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f54101240 msgr2=0x7f4f54199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:48.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.073+0000 7f4f5903f640 1 --2- 172.21.15.103:0/1052795572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f54101240 0x7f4f54199440 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f4f44006010 tx=0x7f4f44003c80 comp rx=0 tx=0).stop 2024-09-19T07:06:48.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.073+0000 7f4f5903f640 1 -- 172.21.15.103:0/1052795572 shutdown_connections 2024-09-19T07:06:48.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.073+0000 7f4f5903f640 1 --2- 172.21.15.103:0/1052795572 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f4f3c0a33b0 0x7f4f3c0a5870 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.073+0000 7f4f5903f640 1 --2- 172.21.15.103:0/1052795572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f54105a90 0x7f4f541a44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.073+0000 7f4f5903f640 1 --2- 172.21.15.103:0/1052795572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f54101bc0 0x7f4f54199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.073+0000 7f4f5903f640 1 --2- 172.21.15.103:0/1052795572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f54101240 0x7f4f54199440 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.073+0000 7f4f5903f640 1 -- 172.21.15.103:0/1052795572 >> 172.21.15.103:0/1052795572 conn(0x7f4f54077210 msgr2=0x7f4f54103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:48.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.073+0000 7f4f5903f640 1 -- 172.21.15.103:0/1052795572 shutdown_connections 2024-09-19T07:06:48.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.073+0000 7f4f5903f640 1 -- 172.21.15.103:0/1052795572 wait complete. 2024-09-19T07:06:48.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:48 smithi174 ceph-mon[35637]: pgmap v549: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T07:06:48.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:48 smithi103 ceph-mon[31158]: pgmap v549: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T07:06:48.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:48 smithi134 ceph-mon[35356]: pgmap v549: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T07:06:48.734 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-19T07:06:48.735 INFO:tasks.fwd_scrub:done joining 2024-09-19T07:06:48.735 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T07:06:48.750 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:06:48.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.850+0000 7fd292c1e640 1 -- 172.21.15.103:0/687632863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd28c106990 msgr2=0x7fd28c106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:48.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.850+0000 7fd292c1e640 1 --2- 172.21.15.103:0/687632863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd28c106990 0x7fd28c106d70 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7fd280009930 tx=0x7fd28002a420 comp rx=0 tx=0).stop 2024-09-19T07:06:48.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.850+0000 7fd292c1e640 1 -- 172.21.15.103:0/687632863 shutdown_connections 2024-09-19T07:06:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.850+0000 7fd292c1e640 1 --2- 172.21.15.103:0/687632863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd28c101320 0x7fd28c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.850+0000 7fd292c1e640 1 --2- 172.21.15.103:0/687632863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd28c100980 0x7fd28c100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.850+0000 7fd292c1e640 1 --2- 172.21.15.103:0/687632863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd28c106990 0x7fd28c106d70 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.850+0000 7fd292c1e640 1 -- 172.21.15.103:0/687632863 >> 172.21.15.103:0/687632863 conn(0x7fd28c077a10 msgr2=0x7fd28c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.850+0000 7fd292c1e640 1 -- 172.21.15.103:0/687632863 shutdown_connections 2024-09-19T07:06:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.850+0000 7fd292c1e640 1 -- 172.21.15.103:0/687632863 wait complete. 2024-09-19T07:06:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd292c1e640 1 Processor -- start 2024-09-19T07:06:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd292c1e640 1 -- start start 2024-09-19T07:06:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd292c1e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd28c100980 0x7fd28c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd292c1e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd28c101320 0x7fd28c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd292c1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd28c106990 0x7fd28c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd292c1e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c111e80 con 0x7fd28c106990 2024-09-19T07:06:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd292c1e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c111d00 con 0x7fd28c100980 2024-09-19T07:06:48.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd292c1e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c112000 con 0x7fd28c101320 2024-09-19T07:06:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd2911bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd28c106990 0x7fd28c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd2911bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd28c106990 0x7fd28c19ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52358/0 (socket says 172.21.15.103:52358) 2024-09-19T07:06:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd2911bb640 1 -- 172.21.15.103:0/378255529 learned_addr learned my addr 172.21.15.103:0/378255529 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd28bfff640 1 --2- 172.21.15.103:0/378255529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd28c101320 0x7fd28c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd2909ba640 1 --2- 172.21.15.103:0/378255529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd28c100980 0x7fd28c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd2911bb640 1 -- 172.21.15.103:0/378255529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd28c101320 msgr2=0x7fd28c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd2911bb640 1 --2- 172.21.15.103:0/378255529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd28c101320 0x7fd28c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd2911bb640 1 -- 172.21.15.103:0/378255529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd28c100980 msgr2=0x7fd28c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd2911bb640 1 --2- 172.21.15.103:0/378255529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd28c100980 0x7fd28c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd2911bb640 1 -- 172.21.15.103:0/378255529 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c19d420 con 0x7fd28c106990 2024-09-19T07:06:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.851+0000 7fd2909ba640 1 --2- 172.21.15.103:0/378255529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd28c100980 0x7fd28c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.852+0000 7fd2911bb640 1 --2- 172.21.15.103:0/378255529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd28c106990 0x7fd28c19ccf0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7fd28400eaa0 tx=0x7fd28400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.852+0000 7fd289ffb640 1 -- 172.21.15.103:0/378255529 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28400cdb0 con 0x7fd28c106990 2024-09-19T07:06:48.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.852+0000 7fd289ffb640 1 -- 172.21.15.103:0/378255529 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd284005020 con 0x7fd28c106990 2024-09-19T07:06:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.852+0000 7fd292c1e640 1 -- 172.21.15.103:0/378255529 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28c19d710 con 0x7fd28c106990 2024-09-19T07:06:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.852+0000 7fd289ffb640 1 -- 172.21.15.103:0/378255529 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2840116b0 con 0x7fd28c106990 2024-09-19T07:06:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.852+0000 7fd292c1e640 1 -- 172.21.15.103:0/378255529 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28c1a4fa0 con 0x7fd28c106990 2024-09-19T07:06:48.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.853+0000 7fd292c1e640 1 -- 172.21.15.103:0/378255529 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd28c100de0 con 0x7fd28c106990 2024-09-19T07:06:48.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.854+0000 7fd289ffb640 1 -- 172.21.15.103:0/378255529 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fd284005560 con 0x7fd28c106990 2024-09-19T07:06:48.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.855+0000 7fd289ffb640 1 --2- 172.21.15.103:0/378255529 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd27c0a33c0 0x7fd27c0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:48.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.855+0000 7fd2909ba640 1 --2- 172.21.15.103:0/378255529 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd27c0a33c0 0x7fd27c0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:48.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.855+0000 7fd289ffb640 1 -- 172.21.15.103:0/378255529 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7fd2840d0000 con 0x7fd28c106990 2024-09-19T07:06:48.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.855+0000 7fd2909ba640 1 --2- 172.21.15.103:0/378255529 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd27c0a33c0 0x7fd27c0a5880 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd280000c00 tx=0x7fd280032c70 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:48.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:48.859+0000 7fd289ffb640 1 -- 172.21.15.103:0/378255529 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd284016030 con 0x7fd28c106990 2024-09-19T07:06:49.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.030+0000 7fd292c1e640 1 -- 172.21.15.103:0/378255529 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd28c108f50 con 0x7fd28c106990 2024-09-19T07:06:49.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.032+0000 7fd289ffb640 1 -- 172.21.15.103:0/378255529 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+23267 (secure 0 0 0) 0x7fd28409c9c0 con 0x7fd28c106990 2024-09-19T07:06:49.033 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:06:49.033 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":103,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","created":"2024-09-19T06:39:46.840574+0000","modified":"2024-09-19T07:06:11.729901+0000","last_up_change":"2024-09-19T06:46:35.652960+0000","last_in_change":"2024-09-19T06:46:11.630585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:42:52.050881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:47:21.192611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:47:23.643291+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":28,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d45dab3-e434-4783-b2e3-f658c3ba583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6803","nonce":3157697740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6805","nonce":3157697740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6809","nonce":3157697740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6807","nonce":3157697740}]},"public_addr":"172.21.15.103:6803/3157697740","cluster_addr":"172.21.15.103:6805/3157697740","heartbeat_back_addr":"172.21.15.103:6809/3157697740","heartbeat_front_addr":"172.21.15.103:6807/3157697740","state":["exists","up"]},{"osd":1,"uuid":"ee0e92bd-15ab-4867-9a18-20abb9f5fc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6801","nonce":1418301195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6803","nonce":1418301195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6807","nonce":1418301195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6805","nonce":1418301195}]},"public_addr":"172.21.15.134:6801/1418301195","cluster_addr":"172.21.15.134:6803/1418301195","heartbeat_back_addr":"172.21.15.134:6807/1418301195","heartbeat_front_addr":"172.21.15.134:6805/1418301195","state":["exists","up"]},{"osd":2,"uuid":"0230caed-025b-46e9-a011-237601d16ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6801","nonce":2292242290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6803","nonce":2292242290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6807","nonce":2292242290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6805","nonce":2292242290}]},"public_addr":"172.21.15.174:6801/2292242290","cluster_addr":"172.21.15.174:6803/2292242290","heartbeat_back_addr":"172.21.15.174:6807/2292242290","heartbeat_front_addr":"172.21.15.174:6805/2292242290","state":["exists","up"]},{"osd":3,"uuid":"c54a8399-bf4b-4d70-b207-a13d85f99590","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6811","nonce":2698986333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6813","nonce":2698986333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6817","nonce":2698986333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6815","nonce":2698986333}]},"public_addr":"172.21.15.103:6811/2698986333","cluster_addr":"172.21.15.103:6813/2698986333","heartbeat_back_addr":"172.21.15.103:6817/2698986333","heartbeat_front_addr":"172.21.15.103:6815/2698986333","state":["exists","up"]},{"osd":4,"uuid":"7c046be6-464a-4ab5-812b-1781121fc317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6809","nonce":3169891968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6811","nonce":3169891968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6815","nonce":3169891968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6813","nonce":3169891968}]},"public_addr":"172.21.15.134:6809/3169891968","cluster_addr":"172.21.15.134:6811/3169891968","heartbeat_back_addr":"172.21.15.134:6815/3169891968","heartbeat_front_addr":"172.21.15.134:6813/3169891968","state":["exists","up"]},{"osd":5,"uuid":"8622596b-6a06-459c-882d-50fba9e296ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6809","nonce":763749644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6811","nonce":763749644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6815","nonce":763749644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6813","nonce":763749644}]},"public_addr":"172.21.15.174:6809/763749644","cluster_addr":"172.21.15.174:6811/763749644","heartbeat_back_addr":"172.21.15.174:6815/763749644","heartbeat_front_addr":"172.21.15.174:6813/763749644","state":["exists","up"]},{"osd":6,"uuid":"64d858d6-a8b3-4f65-a187-af9ab1ccee0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6819","nonce":1396972394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6821","nonce":1396972394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6825","nonce":1396972394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6823","nonce":1396972394}]},"public_addr":"172.21.15.103:6819/1396972394","cluster_addr":"172.21.15.103:6821/1396972394","heartbeat_back_addr":"172.21.15.103:6825/1396972394","heartbeat_front_addr":"172.21.15.103:6823/1396972394","state":["exists","up"]},{"osd":7,"uuid":"d405d1c6-839c-4c92-8261-4adb3163c90f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6817","nonce":2196366765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6819","nonce":2196366765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6823","nonce":2196366765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6821","nonce":2196366765}]},"public_addr":"172.21.15.134:6817/2196366765","cluster_addr":"172.21.15.134:6819/2196366765","heartbeat_back_addr":"172.21.15.134:6823/2196366765","heartbeat_front_addr":"172.21.15.134:6821/2196366765","state":["exists","up"]},{"osd":8,"uuid":"4c452cd9-816c-4f77-9368-2cd4bbeee4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6817","nonce":189977465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6819","nonce":189977465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6823","nonce":189977465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6821","nonce":189977465}]},"public_addr":"172.21.15.174:6817/189977465","cluster_addr":"172.21.15.174:6819/189977465","heartbeat_back_addr":"172.21.15.174:6823/189977465","heartbeat_front_addr":"172.21.15.174:6821/189977465","state":["exists","up"]},{"osd":9,"uuid":"fb3662e0-2d8e-45c4-9119-6d08b35443da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6827","nonce":3161966413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6829","nonce":3161966413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6833","nonce":3161966413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6831","nonce":3161966413}]},"public_addr":"172.21.15.103:6827/3161966413","cluster_addr":"172.21.15.103:6829/3161966413","heartbeat_back_addr":"172.21.15.103:6833/3161966413","heartbeat_front_addr":"172.21.15.103:6831/3161966413","state":["exists","up"]},{"osd":10,"uuid":"e52c3cbe-a6af-441e-b423-b94f3a2da544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6825","nonce":2532758632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6827","nonce":2532758632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6831","nonce":2532758632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6829","nonce":2532758632}]},"public_addr":"172.21.15.134:6825/2532758632","cluster_addr":"172.21.15.134:6827/2532758632","heartbeat_back_addr":"172.21.15.134:6831/2532758632","heartbeat_front_addr":"172.21.15.134:6829/2532758632","state":["exists","up"]},{"osd":11,"uuid":"9040f3f8-3873-4dea-8716-dcda5811c8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6825","nonce":161916442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6827","nonce":161916442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6831","nonce":161916442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6829","nonce":161916442}]},"public_addr":"172.21.15.174:6825/161916442","cluster_addr":"172.21.15.174:6827/161916442","heartbeat_back_addr":"172.21.15.174:6831/161916442","heartbeat_front_addr":"172.21.15.174:6829/161916442","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:41:59.656854+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:24.797551+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:49.685497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:33.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:39.324983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:03.987310+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:28.747971+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:53.827780+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:18.554450+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:43.705691+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:08.401275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:33.502542+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2169657832":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/1596229922":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1642809745":"2024-09-20T06:40:16.780890+0000","172.21.15.103:6800/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/70266056":"2024-09-20T06:49:13.909393+0000","172.21.15.103:6801/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1496153385":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/471230312":"2024-09-20T06:49:13.909393+0000","172.21.15.103:6801/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/2309624637":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/3727259725":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/2590139910":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/2050139492":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/2867049580":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/2590139910":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/1074662744":"2024-09-20T06:49:13.909393+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":28,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:06:49.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.034+0000 7fd292c1e640 1 -- 172.21.15.103:0/378255529 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd27c0a33c0 msgr2=0x7fd27c0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.034+0000 7fd292c1e640 1 --2- 172.21.15.103:0/378255529 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd27c0a33c0 0x7fd27c0a5880 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd280000c00 tx=0x7fd280032c70 comp rx=0 tx=0).stop 2024-09-19T07:06:49.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.034+0000 7fd292c1e640 1 -- 172.21.15.103:0/378255529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd28c106990 msgr2=0x7fd28c19ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.034+0000 7fd292c1e640 1 --2- 172.21.15.103:0/378255529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd28c106990 0x7fd28c19ccf0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7fd28400eaa0 tx=0x7fd28400ef60 comp rx=0 tx=0).stop 2024-09-19T07:06:49.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.034+0000 7fd292c1e640 1 -- 172.21.15.103:0/378255529 shutdown_connections 2024-09-19T07:06:49.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.034+0000 7fd292c1e640 1 --2- 172.21.15.103:0/378255529 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fd27c0a33c0 0x7fd27c0a5880 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.034+0000 7fd292c1e640 1 --2- 172.21.15.103:0/378255529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd28c106990 0x7fd28c19ccf0 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.034+0000 7fd292c1e640 1 --2- 172.21.15.103:0/378255529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd28c101320 0x7fd28c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.034+0000 7fd292c1e640 1 --2- 172.21.15.103:0/378255529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd28c100980 0x7fd28c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.034+0000 7fd292c1e640 1 -- 172.21.15.103:0/378255529 >> 172.21.15.103:0/378255529 conn(0x7fd28c077a10 msgr2=0x7fd28c103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:49.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.034+0000 7fd292c1e640 1 -- 172.21.15.103:0/378255529 shutdown_connections 2024-09-19T07:06:49.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.034+0000 7fd292c1e640 1 -- 172.21.15.103:0/378255529 wait complete. 2024-09-19T07:06:49.046 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T06:42:52.050881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T06:47:21.192611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T06:47:23.643291+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 28, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T07:06:49.046 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T07:06:49.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.154+0000 7fdeae61e640 1 -- 172.21.15.103:0/3764491365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdea8100040 msgr2=0x7fdea8100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.154+0000 7fdeae61e640 1 --2- 172.21.15.103:0/3764491365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdea8100040 0x7fdea8100460 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7fde90009900 tx=0x7fde9002a410 comp rx=0 tx=0).stop 2024-09-19T07:06:49.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.154+0000 7fdeae61e640 1 -- 172.21.15.103:0/3764491365 shutdown_connections 2024-09-19T07:06:49.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.154+0000 7fdeae61e640 1 --2- 172.21.15.103:0/3764491365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdea81009a0 0x7fdea810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.154+0000 7fdeae61e640 1 --2- 172.21.15.103:0/3764491365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdea8100040 0x7fdea8100460 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.154+0000 7fdeae61e640 1 --2- 172.21.15.103:0/3764491365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdea8107bd0 0x7fdea8107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.154+0000 7fdeae61e640 1 -- 172.21.15.103:0/3764491365 >> 172.21.15.103:0/3764491365 conn(0x7fdea8077170 msgr2=0x7fdea80fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:49.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.155+0000 7fdeae61e640 1 -- 172.21.15.103:0/3764491365 shutdown_connections 2024-09-19T07:06:49.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.155+0000 7fdeae61e640 1 -- 172.21.15.103:0/3764491365 wait complete. 2024-09-19T07:06:49.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.155+0000 7fdeae61e640 1 Processor -- start 2024-09-19T07:06:49.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.155+0000 7fdeae61e640 1 -- start start 2024-09-19T07:06:49.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.155+0000 7fdeae61e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdea8100040 0x7fdea8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:49.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.155+0000 7fdeae61e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdea81009a0 0x7fdea81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:49.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.155+0000 7fdeae61e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdea8107bd0 0x7fdea819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:49.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.155+0000 7fdeae61e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8111fa0 con 0x7fdea8107bd0 2024-09-19T07:06:49.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.155+0000 7fdeae61e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8111e20 con 0x7fdea8100040 2024-09-19T07:06:49.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.155+0000 7fdeae61e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8112120 con 0x7fdea81009a0 2024-09-19T07:06:49.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.155+0000 7fdeacbbb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdea8107bd0 0x7fdea819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:49.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.155+0000 7fdeacbbb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdea8107bd0 0x7fdea819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52382/0 (socket says 172.21.15.103:52382) 2024-09-19T07:06:49.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.155+0000 7fdeacbbb640 1 -- 172.21.15.103:0/944005657 learned_addr learned my addr 172.21.15.103:0/944005657 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:49.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.156+0000 7fdea7fff640 1 --2- 172.21.15.103:0/944005657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdea8100040 0x7fdea8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:49.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.156+0000 7fde9ffff640 1 --2- 172.21.15.103:0/944005657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdea81009a0 0x7fdea81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:49.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.156+0000 7fdeacbbb640 1 -- 172.21.15.103:0/944005657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdea81009a0 msgr2=0x7fdea81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.156+0000 7fdeacbbb640 1 --2- 172.21.15.103:0/944005657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdea81009a0 0x7fdea81989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.156+0000 7fdeacbbb640 1 -- 172.21.15.103:0/944005657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdea8100040 msgr2=0x7fdea8198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.156+0000 7fdeacbbb640 1 --2- 172.21.15.103:0/944005657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdea8100040 0x7fdea8198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.156+0000 7fdeacbbb640 1 -- 172.21.15.103:0/944005657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea819d480 con 0x7fdea8107bd0 2024-09-19T07:06:49.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.156+0000 7fdeacbbb640 1 --2- 172.21.15.103:0/944005657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdea8107bd0 0x7fdea819cd50 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fdea000e960 tx=0x7fdea000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:49.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.156+0000 7fdea5ffb640 1 -- 172.21.15.103:0/944005657 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0004d20 con 0x7fdea8107bd0 2024-09-19T07:06:49.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.156+0000 7fdea5ffb640 1 -- 172.21.15.103:0/944005657 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea0005020 con 0x7fdea8107bd0 2024-09-19T07:06:49.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.156+0000 7fdeae61e640 1 -- 172.21.15.103:0/944005657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea819d770 con 0x7fdea8107bd0 2024-09-19T07:06:49.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.156+0000 7fdea5ffb640 1 -- 172.21.15.103:0/944005657 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea00116d0 con 0x7fdea8107bd0 2024-09-19T07:06:49.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.156+0000 7fdeae61e640 1 -- 172.21.15.103:0/944005657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea81a5000 con 0x7fdea8107bd0 2024-09-19T07:06:49.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.157+0000 7fdeae61e640 1 -- 172.21.15.103:0/944005657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea8100460 con 0x7fdea8107bd0 2024-09-19T07:06:49.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.159+0000 7fdea5ffb640 1 -- 172.21.15.103:0/944005657 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fdea0005560 con 0x7fdea8107bd0 2024-09-19T07:06:49.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.159+0000 7fdea5ffb640 1 --2- 172.21.15.103:0/944005657 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fde940a33c0 0x7fde940a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:49.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.159+0000 7fdea7fff640 1 --2- 172.21.15.103:0/944005657 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fde940a33c0 0x7fde940a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:49.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.159+0000 7fdea5ffb640 1 -- 172.21.15.103:0/944005657 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7fdea00d0240 con 0x7fdea8107bd0 2024-09-19T07:06:49.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.160+0000 7fdea7fff640 1 --2- 172.21.15.103:0/944005657 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fde940a33c0 0x7fde940a5880 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fde9800bb60 tx=0x7fde9800b390 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:49.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.162+0000 7fdea5ffb640 1 -- 172.21.15.103:0/944005657 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdea0016030 con 0x7fdea8107bd0 2024-09-19T07:06:49.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.328+0000 7fdeae61e640 1 -- 172.21.15.103:0/944005657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdea8109ac0 con 0x7fdea8107bd0 2024-09-19T07:06:49.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.328+0000 7fdea5ffb640 1 -- 172.21.15.103:0/944005657 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7fdea009c5b0 con 0x7fdea8107bd0 2024-09-19T07:06:49.329 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 1 2024-09-19T07:06:49.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.330+0000 7fdeae61e640 1 -- 172.21.15.103:0/944005657 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fde940a33c0 msgr2=0x7fde940a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.330+0000 7fdeae61e640 1 --2- 172.21.15.103:0/944005657 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fde940a33c0 0x7fde940a5880 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fde9800bb60 tx=0x7fde9800b390 comp rx=0 tx=0).stop 2024-09-19T07:06:49.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.330+0000 7fdeae61e640 1 -- 172.21.15.103:0/944005657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdea8107bd0 msgr2=0x7fdea819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.330+0000 7fdeae61e640 1 --2- 172.21.15.103:0/944005657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdea8107bd0 0x7fdea819cd50 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fdea000e960 tx=0x7fdea000ee20 comp rx=0 tx=0).stop 2024-09-19T07:06:49.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.330+0000 7fdeae61e640 1 -- 172.21.15.103:0/944005657 shutdown_connections 2024-09-19T07:06:49.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.330+0000 7fdeae61e640 1 --2- 172.21.15.103:0/944005657 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fde940a33c0 0x7fde940a5880 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.330+0000 7fdeae61e640 1 --2- 172.21.15.103:0/944005657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdea8107bd0 0x7fdea819cd50 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.330+0000 7fdeae61e640 1 --2- 172.21.15.103:0/944005657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdea81009a0 0x7fdea81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.330+0000 7fdeae61e640 1 --2- 172.21.15.103:0/944005657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdea8100040 0x7fdea8198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.330+0000 7fdeae61e640 1 -- 172.21.15.103:0/944005657 >> 172.21.15.103:0/944005657 conn(0x7fdea8077170 msgr2=0x7fdea80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:49.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.331+0000 7fdeae61e640 1 -- 172.21.15.103:0/944005657 shutdown_connections 2024-09-19T07:06:49.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.331+0000 7fdeae61e640 1 -- 172.21.15.103:0/944005657 wait complete. 2024-09-19T07:06:49.343 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T07:06:49.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.445+0000 7f3a68d3b640 1 -- 172.21.15.103:0/2626954383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a64100990 msgr2=0x7f3a64100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.445+0000 7f3a68d3b640 1 --2- 172.21.15.103:0/2626954383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a64100990 0x7f3a64100df0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f3a58009930 tx=0x7f3a5802a420 comp rx=0 tx=0).stop 2024-09-19T07:06:49.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.445+0000 7f3a68d3b640 1 -- 172.21.15.103:0/2626954383 shutdown_connections 2024-09-19T07:06:49.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.445+0000 7f3a68d3b640 1 --2- 172.21.15.103:0/2626954383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a64101330 0x7f3a6410d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.445+0000 7f3a68d3b640 1 --2- 172.21.15.103:0/2626954383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a64100990 0x7f3a64100df0 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.445+0000 7f3a68d3b640 1 --2- 172.21.15.103:0/2626954383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a641069a0 0x7f3a64106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.445+0000 7f3a68d3b640 1 -- 172.21.15.103:0/2626954383 >> 172.21.15.103:0/2626954383 conn(0x7f3a640fc6b0 msgr2=0x7f3a640fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.445+0000 7f3a68d3b640 1 -- 172.21.15.103:0/2626954383 shutdown_connections 2024-09-19T07:06:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.445+0000 7f3a68d3b640 1 -- 172.21.15.103:0/2626954383 wait complete. 2024-09-19T07:06:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.446+0000 7f3a68d3b640 1 Processor -- start 2024-09-19T07:06:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.446+0000 7f3a68d3b640 1 -- start start 2024-09-19T07:06:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.446+0000 7f3a68d3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a64100990 0x7f3a64197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.446+0000 7f3a68d3b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a64101330 0x7f3a641978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.446+0000 7f3a68d3b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a641069a0 0x7f3a6419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.446+0000 7f3a68d3b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a64110de0 con 0x7f3a64100990 2024-09-19T07:06:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.446+0000 7f3a68d3b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3a64110c60 con 0x7f3a641069a0 2024-09-19T07:06:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.446+0000 7f3a68d3b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a64110f60 con 0x7f3a64101330 2024-09-19T07:06:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.446+0000 7f3a6259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a64100990 0x7f3a64197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.446+0000 7f3a6259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a64100990 0x7f3a64197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52398/0 (socket says 172.21.15.103:52398) 2024-09-19T07:06:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.446+0000 7f3a6259c640 1 -- 172.21.15.103:0/1436951034 learned_addr learned my addr 172.21.15.103:0/1436951034 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.446+0000 7f3a61d9b640 1 --2- 172.21.15.103:0/1436951034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a64101330 0x7f3a641978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.446+0000 7f3a62d9d640 1 --2- 172.21.15.103:0/1436951034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a641069a0 0x7f3a6419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.446+0000 7f3a6259c640 1 -- 172.21.15.103:0/1436951034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a64101330 msgr2=0x7f3a641978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.447+0000 7f3a6259c640 1 --2- 172.21.15.103:0/1436951034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a64101330 0x7f3a641978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.447+0000 7f3a6259c640 1 -- 172.21.15.103:0/1436951034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a641069a0 msgr2=0x7f3a6419bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.447+0000 7f3a6259c640 1 --2- 172.21.15.103:0/1436951034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a641069a0 0x7f3a6419bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.447+0000 7f3a6259c640 1 -- 172.21.15.103:0/1436951034 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6419c360 con 0x7f3a64100990 2024-09-19T07:06:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.447+0000 7f3a62d9d640 1 --2- 172.21.15.103:0/1436951034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a641069a0 0x7f3a6419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:06:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.447+0000 7f3a6259c640 1 --2- 172.21.15.103:0/1436951034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a64100990 0x7f3a64197370 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400d770 tx=0x7f3a5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:49.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.447+0000 7f3a537fe640 1 -- 172.21.15.103:0/1436951034 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54017070 con 0x7f3a64100990 2024-09-19T07:06:49.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.447+0000 7f3a537fe640 1 -- 172.21.15.103:0/1436951034 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a54005500 con 0x7f3a64100990 2024-09-19T07:06:49.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.447+0000 7f3a68d3b640 1 -- 172.21.15.103:0/1436951034 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6419c650 con 0x7f3a64100990 2024-09-19T07:06:49.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.447+0000 7f3a537fe640 1 -- 172.21.15.103:0/1436951034 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54004e00 con 0x7f3a64100990 2024-09-19T07:06:49.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.447+0000 7f3a68d3b640 1 -- 172.21.15.103:0/1436951034 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a641a3ee0 con 0x7f3a64100990 2024-09-19T07:06:49.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.448+0000 7f3a68d3b640 1 -- 172.21.15.103:0/1436951034 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a64100df0 con 0x7f3a64100990 2024-09-19T07:06:49.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.449+0000 7f3a537fe640 1 -- 172.21.15.103:0/1436951034 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f3a54004fa0 con 0x7f3a64100990 2024-09-19T07:06:49.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.450+0000 7f3a537fe640 1 --2- 172.21.15.103:0/1436951034 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3a4c0a33c0 0x7f3a4c0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:49.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.450+0000 7f3a61d9b640 1 --2- 172.21.15.103:0/1436951034 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3a4c0a33c0 0x7f3a4c0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:49.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.450+0000 7f3a537fe640 1 -- 172.21.15.103:0/1436951034 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f3a540cfc50 con 0x7f3a64100990 2024-09-19T07:06:49.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.450+0000 7f3a61d9b640 1 --2- 172.21.15.103:0/1436951034 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3a4c0a33c0 0x7f3a4c0a5880 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3a580096c0 tx=0x7f3a58000bc0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:49.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.453+0000 7f3a537fe640 1 -- 172.21.15.103:0/1436951034 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3a5409c5b0 con 0x7f3a64100990 2024-09-19T07:06:49.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.622+0000 7f3a68d3b640 1 -- 172.21.15.103:0/1436951034 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3a6402d7c0 con 0x7f3a64100990 2024-09-19T07:06:49.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.623+0000 7f3a537fe640 1 -- 172.21.15.103:0/1436951034 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7f3a5409beb0 con 0x7f3a64100990 2024-09-19T07:06:49.624 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 32 2024-09-19T07:06:49.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.625+0000 7f3a68d3b640 1 -- 172.21.15.103:0/1436951034 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3a4c0a33c0 msgr2=0x7f3a4c0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.625+0000 7f3a68d3b640 1 --2- 172.21.15.103:0/1436951034 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3a4c0a33c0 0x7f3a4c0a5880 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3a580096c0 tx=0x7f3a58000bc0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.625+0000 7f3a68d3b640 1 -- 172.21.15.103:0/1436951034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a64100990 msgr2=0x7f3a64197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.625+0000 7f3a68d3b640 1 --2- 172.21.15.103:0/1436951034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a64100990 0x7f3a64197370 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400d770 tx=0x7f3a5400dc30 comp rx=0 tx=0).stop 2024-09-19T07:06:49.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.625+0000 7f3a68d3b640 1 -- 172.21.15.103:0/1436951034 shutdown_connections 2024-09-19T07:06:49.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.625+0000 7f3a68d3b640 1 --2- 172.21.15.103:0/1436951034 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f3a4c0a33c0 0x7f3a4c0a5880 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.625+0000 7f3a68d3b640 1 --2- 172.21.15.103:0/1436951034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a641069a0 0x7f3a6419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.625+0000 7f3a68d3b640 1 --2- 172.21.15.103:0/1436951034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a64101330 0x7f3a641978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.625+0000 7f3a68d3b640 1 --2- 172.21.15.103:0/1436951034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a64100990 0x7f3a64197370 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.625+0000 7f3a68d3b640 1 -- 172.21.15.103:0/1436951034 >> 172.21.15.103:0/1436951034 conn(0x7f3a640fc6b0 msgr2=0x7f3a640fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:49.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.625+0000 7f3a68d3b640 1 -- 172.21.15.103:0/1436951034 shutdown_connections 2024-09-19T07:06:49.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.625+0000 7f3a68d3b640 1 -- 172.21.15.103:0/1436951034 wait complete. 2024-09-19T07:06:49.636 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T07:06:49.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.738+0000 7f8ef284c640 1 -- 172.21.15.103:0/2868045467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec103610 msgr2=0x7f8eec10baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.738+0000 7f8ef284c640 1 --2- 172.21.15.103:0/2868045467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec103610 0x7f8eec10baf0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009900 tx=0x7f8ee002a480 comp rx=0 tx=0).stop 2024-09-19T07:06:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.738+0000 7f8ef284c640 1 -- 172.21.15.103:0/2868045467 shutdown_connections 2024-09-19T07:06:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.738+0000 7f8ef284c640 1 --2- 172.21.15.103:0/2868045467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8eec10c0c0 0x7f8eec10e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.738+0000 7f8ef284c640 1 --2- 172.21.15.103:0/2868045467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec103610 0x7f8eec10baf0 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.738+0000 7f8ef284c640 1 --2- 172.21.15.103:0/2868045467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8eec102cf0 0x7f8eec1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.738+0000 7f8ef284c640 1 -- 172.21.15.103:0/2868045467 >> 172.21.15.103:0/2868045467 conn(0x7f8eec0779d0 msgr2=0x7f8eec0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.738+0000 7f8ef284c640 1 -- 172.21.15.103:0/2868045467 shutdown_connections 2024-09-19T07:06:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.738+0000 7f8ef284c640 1 -- 172.21.15.103:0/2868045467 wait complete. 2024-09-19T07:06:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.738+0000 7f8ef284c640 1 Processor -- start 2024-09-19T07:06:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.738+0000 7f8ef284c640 1 -- start start 2024-09-19T07:06:49.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ef284c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8eec102cf0 0x7f8eec198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ef284c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8eec103610 0x7f8eec1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ef284c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec10c0c0 0x7f8eec19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ef284c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec111ee0 con 0x7f8eec10c0c0 2024-09-19T07:06:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ef284c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec111d60 con 0x7f8eec103610 2024-09-19T07:06:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ef284c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec112060 con 0x7f8eec102cf0 2024-09-19T07:06:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ef0de9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec10c0c0 0x7f8eec19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ef0de9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec10c0c0 0x7f8eec19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52422/0 (socket says 172.21.15.103:52422) 2024-09-19T07:06:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ef0de9640 1 -- 172.21.15.103:0/3654606891 learned_addr learned my addr 172.21.15.103:0/3654606891 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8eebfff640 1 --2- 172.21.15.103:0/3654606891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8eec102cf0 0x7f8eec198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8eeb7fe640 1 --2- 172.21.15.103:0/3654606891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8eec103610 0x7f8eec1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ef0de9640 1 -- 172.21.15.103:0/3654606891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8eec102cf0 msgr2=0x7f8eec198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ef0de9640 1 --2- 172.21.15.103:0/3654606891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8eec102cf0 0x7f8eec198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ef0de9640 1 -- 172.21.15.103:0/3654606891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8eec103610 msgr2=0x7f8eec1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ef0de9640 1 --2- 172.21.15.103:0/3654606891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8eec103610 0x7f8eec1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ef0de9640 1 -- 172.21.15.103:0/3654606891 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eec19d380 con 0x7f8eec10c0c0 2024-09-19T07:06:49.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ef0de9640 1 --2- 172.21.15.103:0/3654606891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec10c0c0 0x7f8eec19cc50 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400cad0 tx=0x7f8ee400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:49.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ee97fa640 1 -- 172.21.15.103:0/3654606891 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee400ae20 con 0x7f8eec10c0c0 2024-09-19T07:06:49.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ee97fa640 1 -- 172.21.15.103:0/3654606891 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee4005020 con 0x7f8eec10c0c0 2024-09-19T07:06:49.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.739+0000 7f8ee97fa640 1 -- 172.21.15.103:0/3654606891 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee400f6f0 con 0x7f8eec10c0c0 2024-09-19T07:06:49.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.740+0000 7f8ef284c640 1 -- 172.21.15.103:0/3654606891 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eec19d670 con 0x7f8eec10c0c0 2024-09-19T07:06:49.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.740+0000 7f8ef284c640 1 -- 172.21.15.103:0/3654606891 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eec1a4f00 con 0x7f8eec10c0c0 2024-09-19T07:06:49.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.741+0000 7f8ef284c640 1 -- 172.21.15.103:0/3654606891 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eec1030d0 con 0x7f8eec10c0c0 2024-09-19T07:06:49.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.742+0000 7f8ee97fa640 1 -- 172.21.15.103:0/3654606891 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8ee40051d0 con 0x7f8eec10c0c0 2024-09-19T07:06:49.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.743+0000 7f8ee97fa640 1 --2- 172.21.15.103:0/3654606891 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8ed40a3440 0x7f8ed40a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:49.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.743+0000 7f8eebfff640 1 --2- 172.21.15.103:0/3654606891 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8ed40a3440 0x7f8ed40a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:49.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.743+0000 7f8ee97fa640 1 -- 172.21.15.103:0/3654606891 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f8ee40cfff0 con 0x7f8eec10c0c0 2024-09-19T07:06:49.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.743+0000 7f8eebfff640 1 --2- 172.21.15.103:0/3654606891 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8ed40a3440 0x7f8ed40a5900 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8edc005e10 tx=0x7f8edc005da0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:49.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.746+0000 7f8ee97fa640 1 -- 172.21.15.103:0/3654606891 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8ee4014030 con 0x7f8eec10c0c0 2024-09-19T07:06:49.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.912+0000 7f8ef284c640 1 -- 172.21.15.103:0/3654606891 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8eec02d7c0 con 0x7f8eec10c0c0 2024-09-19T07:06:49.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.912+0000 7f8ee97fa640 1 -- 172.21.15.103:0/3654606891 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7f8ee409ca30 con 0x7f8eec10c0c0 2024-09-19T07:06:49.913 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 64 2024-09-19T07:06:49.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.914+0000 7f8ef284c640 1 -- 172.21.15.103:0/3654606891 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8ed40a3440 msgr2=0x7f8ed40a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.914+0000 7f8ef284c640 1 --2- 172.21.15.103:0/3654606891 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8ed40a3440 0x7f8ed40a5900 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8edc005e10 tx=0x7f8edc005da0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.914+0000 7f8ef284c640 1 -- 172.21.15.103:0/3654606891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec10c0c0 msgr2=0x7f8eec19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:49.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.914+0000 7f8ef284c640 1 --2- 172.21.15.103:0/3654606891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec10c0c0 0x7f8eec19cc50 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400cad0 tx=0x7f8ee400cf90 comp rx=0 tx=0).stop 2024-09-19T07:06:49.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.915+0000 7f8ef284c640 1 -- 172.21.15.103:0/3654606891 shutdown_connections 2024-09-19T07:06:49.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.915+0000 7f8ef284c640 1 --2- 172.21.15.103:0/3654606891 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8ed40a3440 0x7f8ed40a5900 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.915+0000 7f8ef284c640 1 --2- 172.21.15.103:0/3654606891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec10c0c0 0x7f8eec19cc50 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.915+0000 7f8ef284c640 1 --2- 172.21.15.103:0/3654606891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8eec103610 0x7f8eec1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.915+0000 7f8ef284c640 1 --2- 172.21.15.103:0/3654606891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8eec102cf0 0x7f8eec198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:49.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.915+0000 7f8ef284c640 1 -- 172.21.15.103:0/3654606891 >> 172.21.15.103:0/3654606891 conn(0x7f8eec0779d0 msgr2=0x7f8eec0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:49.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.915+0000 7f8ef284c640 1 -- 172.21.15.103:0/3654606891 shutdown_connections 2024-09-19T07:06:49.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:49.915+0000 7f8ef284c640 1 -- 172.21.15.103:0/3654606891 wait complete. 2024-09-19T07:06:49.927 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T07:06:50.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.027+0000 7f68f247b640 1 -- 172.21.15.103:0/4139112920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68ec101320 msgr2=0x7f68ec10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.027+0000 7f68f247b640 1 --2- 172.21.15.103:0/4139112920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68ec101320 0x7f68ec10d850 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009cd0 tx=0x7f68e402a8e0 comp rx=0 tx=0).stop 2024-09-19T07:06:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.028+0000 7f68f247b640 1 -- 172.21.15.103:0/4139112920 shutdown_connections 2024-09-19T07:06:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.028+0000 7f68f247b640 1 --2- 172.21.15.103:0/4139112920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68ec101320 0x7f68ec10d850 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.028+0000 7f68f247b640 1 --2- 172.21.15.103:0/4139112920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ec100980 0x7f68ec100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.028+0000 7f68f247b640 1 --2- 172.21.15.103:0/4139112920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68ec106990 0x7f68ec106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.028+0000 7f68f247b640 1 -- 172.21.15.103:0/4139112920 >> 172.21.15.103:0/4139112920 conn(0x7f68ec077a10 msgr2=0x7f68ec0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.028+0000 7f68f247b640 1 -- 172.21.15.103:0/4139112920 shutdown_connections 2024-09-19T07:06:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.028+0000 7f68f247b640 1 -- 172.21.15.103:0/4139112920 wait complete. 2024-09-19T07:06:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.028+0000 7f68f247b640 1 Processor -- start 2024-09-19T07:06:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.028+0000 7f68f247b640 1 -- start start 2024-09-19T07:06:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.028+0000 7f68f247b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68ec100980 0x7f68ec199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:50.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.028+0000 7f68f247b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68ec101320 0x7f68ec1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:50.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.028+0000 7f68f247b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ec106990 0x7f68ec1a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:50.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68f247b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec110dd0 con 0x7f68ec101320 2024-09-19T07:06:50.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68f247b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec110c50 con 0x7f68ec106990 2024-09-19T07:06:50.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68f247b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec110f50 con 0x7f68ec100980 2024-09-19T07:06:50.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68ec101320 0x7f68ec1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:50.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68ec101320 0x7f68ec1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52438/0 (socket says 172.21.15.103:52438) 2024-09-19T07:06:50.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68eb7fe640 1 -- 172.21.15.103:0/2162546413 learned_addr learned my addr 172.21.15.103:0/2162546413 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:06:50.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68ebfff640 1 --2- 172.21.15.103:0/2162546413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68ec100980 0x7f68ec199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:50.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68f0a18640 1 --2- 172.21.15.103:0/2162546413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ec106990 0x7f68ec1a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:50.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68eb7fe640 1 -- 172.21.15.103:0/2162546413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68ec100980 msgr2=0x7f68ec199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:50.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68eb7fe640 1 --2- 172.21.15.103:0/2162546413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68ec100980 0x7f68ec199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:50.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68eb7fe640 1 -- 172.21.15.103:0/2162546413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ec106990 msgr2=0x7f68ec1a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:50.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68eb7fe640 1 --2- 172.21.15.103:0/2162546413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ec106990 0x7f68ec1a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:50.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68eb7fe640 1 -- 172.21.15.103:0/2162546413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ec1a4820 con 0x7f68ec101320 2024-09-19T07:06:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68eb7fe640 1 --2- 172.21.15.103:0/2162546413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68ec101320 0x7f68ec1997a0 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f68e000c890 tx=0x7f68e000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68e97fa640 1 -- 172.21.15.103:0/2162546413 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e0004220 con 0x7f68ec101320 2024-09-19T07:06:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68e97fa640 1 -- 172.21.15.103:0/2162546413 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68e00043c0 con 0x7f68ec101320 2024-09-19T07:06:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68f247b640 1 -- 172.21.15.103:0/2162546413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68ec1a49f0 con 0x7f68ec101320 2024-09-19T07:06:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68e97fa640 1 -- 172.21.15.103:0/2162546413 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e00049e0 con 0x7f68ec101320 2024-09-19T07:06:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.029+0000 7f68f247b640 1 -- 172.21.15.103:0/2162546413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68ec1a4f10 con 0x7f68ec101320 2024-09-19T07:06:50.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.030+0000 7f68f247b640 1 -- 172.21.15.103:0/2162546413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68ec1020c0 con 0x7f68ec101320 2024-09-19T07:06:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.032+0000 7f68e97fa640 1 -- 172.21.15.103:0/2162546413 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f68e00079b0 con 0x7f68ec101320 2024-09-19T07:06:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.033+0000 7f68e97fa640 1 --2- 172.21.15.103:0/2162546413 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68d40a33c0 0x7f68d40a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:06:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.033+0000 7f68ebfff640 1 --2- 172.21.15.103:0/2162546413 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68d40a33c0 0x7f68d40a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:06:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.033+0000 7f68e97fa640 1 -- 172.21.15.103:0/2162546413 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f68e00cff10 con 0x7f68ec101320 2024-09-19T07:06:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.033+0000 7f68ebfff640 1 --2- 172.21.15.103:0/2162546413 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68d40a33c0 0x7f68d40a5880 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009b50 tx=0x7f68dc005f90 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:06:50.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.036+0000 7f68e97fa640 1 -- 172.21.15.103:0/2162546413 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f68e0014030 con 0x7f68ec101320 2024-09-19T07:06:50.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.250+0000 7f68f247b640 1 -- 172.21.15.103:0/2162546413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f68ec100de0 con 0x7f68ec101320 2024-09-19T07:06:50.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.258+0000 7f68e97fa640 1 -- 172.21.15.103:0/2162546413 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+742875 (secure 0 0 0) 0x7f68e009c8d0 con 0x7f68ec101320 2024-09-19T07:06:50.271 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:06:50.271 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:06:50.272 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 25, 2024-09-19T07:06:50.272 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.272 INFO:teuthology.orchestra.run.smithi103.stdout: "active_gid": 15210, 2024-09-19T07:06:50.272 INFO:teuthology.orchestra.run.smithi103.stdout: "active_name": "x", 2024-09-19T07:06:50.272 INFO:teuthology.orchestra.run.smithi103.stdout: "active_addrs": { 2024-09-19T07:06:50.272 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-19T07:06:50.272 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.272 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-19T07:06:50.272 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:6800", 2024-09-19T07:06:50.272 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 82882809 2024-09-19T07:06:50.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.272 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.273 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-09-19T07:06:50.273 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:6801", 2024-09-19T07:06:50.273 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 82882809 2024-09-19T07:06:50.273 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.273 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:06:50.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.273 INFO:teuthology.orchestra.run.smithi103.stdout: "active_addr": "172.21.15.103:6801/82882809", 2024-09-19T07:06:50.273 INFO:teuthology.orchestra.run.smithi103.stdout: "active_change": "2024-09-19T06:49:13.909604+0000", 2024-09-19T07:06:50.273 INFO:teuthology.orchestra.run.smithi103.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T07:06:50.273 INFO:teuthology.orchestra.run.smithi103.stdout: "available": true, 2024-09-19T07:06:50.273 INFO:teuthology.orchestra.run.smithi103.stdout: "standbys": [ 2024-09-19T07:06:50.273 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.273 INFO:teuthology.orchestra.run.smithi103.stdout: "gid": 15204, 2024-09-19T07:06:50.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "y", 2024-09-19T07:06:50.274 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_features": 4540701547738038271, 2024-09-19T07:06:50.274 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-09-19T07:06:50.274 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-09-19T07:06:50.274 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.274 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.274 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.274 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:06:50.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:06:50.274 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.275 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.275 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.275 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:06:50.275 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.275 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.275 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.275 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T07:06:50.275 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.275 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.275 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.275 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.275 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.276 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.276 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.276 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.276 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.276 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.276 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.276 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.276 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.276 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.276 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.276 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.277 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.277 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.277 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.277 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.277 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.277 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.277 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.277 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.277 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.277 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.277 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.277 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.277 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.278 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.278 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.278 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.278 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.278 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.278 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.278 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.278 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.278 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.279 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.279 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.279 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.279 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.279 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.279 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.280 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.280 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.280 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.280 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.280 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.280 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.280 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.280 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.281 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.281 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.281 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.281 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.281 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.281 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.281 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.281 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.281 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.281 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.281 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.282 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.282 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.282 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.282 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.282 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.282 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.282 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.282 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.282 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-09-19T07:06:50.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-09-19T07:06:50.282 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.282 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.283 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.283 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.283 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.283 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.283 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.283 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to", 2024-09-19T07:06:50.283 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.283 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.283 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.283 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.283 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-09-19T07:06:50.283 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-09-19T07:06:50.283 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.284 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.284 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.284 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-09-19T07:06:50.284 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.284 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.284 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.284 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-09-19T07:06:50.284 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.284 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.284 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.284 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-09-19T07:06:50.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-09-19T07:06:50.285 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.285 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.285 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.285 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.285 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.285 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.285 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.285 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-09-19T07:06:50.285 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.285 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.285 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.286 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-09-19T07:06:50.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-09-19T07:06:50.286 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.286 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.286 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.286 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.286 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.287 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.287 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.287 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-09-19T07:06:50.287 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.287 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.287 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.287 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.287 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-09-19T07:06:50.288 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-09-19T07:06:50.288 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.288 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.288 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.288 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-09-19T07:06:50.288 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.288 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.288 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.288 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-09-19T07:06:50.288 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.288 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.289 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.289 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.289 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-09-19T07:06:50.289 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-09-19T07:06:50.289 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.289 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.289 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.289 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.289 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.289 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.289 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.289 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-09-19T07:06:50.289 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.290 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.290 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.290 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.290 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-09-19T07:06:50.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-09-19T07:06:50.290 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.290 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.290 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.290 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.290 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.291 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.291 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.291 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T07:06:50.291 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.291 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.291 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.291 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.291 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-09-19T07:06:50.291 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-09-19T07:06:50.291 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.291 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.291 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.292 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.292 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.292 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.292 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.292 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-09-19T07:06:50.292 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.292 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.292 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.292 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.292 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.292 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.292 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-09-19T07:06:50.293 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.293 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.293 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.293 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-09-19T07:06:50.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-09-19T07:06:50.293 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.293 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.293 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.293 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.293 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.293 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.293 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.294 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T07:06:50.294 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.294 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.294 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.294 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.294 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-09-19T07:06:50.294 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-09-19T07:06:50.294 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.294 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.294 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.294 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-09-19T07:06:50.294 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.294 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.294 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.295 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T07:06:50.295 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:06:50.295 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.295 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.295 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-09-19T07:06:50.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-09-19T07:06:50.295 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:06:50.295 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.295 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.295 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:06:50.295 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-19T07:06:50.295 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-19T07:06:50.296 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.296 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T07:06:50.296 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:06:50.296 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.296 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.296 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-09-19T07:06:50.296 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-09-19T07:06:50.296 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:06:50.296 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.297 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.297 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-09-19T07:06:50.297 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:06:50.297 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-09-19T07:06:50.297 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.297 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T07:06:50.297 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.297 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.297 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.297 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.297 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-09-19T07:06:50.297 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-09-19T07:06:50.298 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.298 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.298 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.298 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T07:06:50.298 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.298 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.298 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.298 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T07:06:50.298 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T07:06:50.298 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.298 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.298 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.298 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-09-19T07:06:50.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-09-19T07:06:50.299 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:50.299 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.299 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.299 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-09-19T07:06:50.299 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-09-19T07:06:50.299 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-09-19T07:06:50.299 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.299 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-09-19T07:06:50.299 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T07:06:50.299 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.299 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.299 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-09-19T07:06:50.300 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-09-19T07:06:50.300 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.300 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.300 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.300 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-09-19T07:06:50.300 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.300 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.300 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.300 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T07:06:50.300 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:06:50.300 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.301 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.301 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.301 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-09-19T07:06:50.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-09-19T07:06:50.301 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:06:50.301 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.301 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.301 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:06:50.301 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-19T07:06:50.301 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-19T07:06:50.301 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.302 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T07:06:50.302 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:06:50.302 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.302 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.302 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.302 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.302 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.302 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.302 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.302 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.302 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.302 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.303 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.303 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.303 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.303 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.303 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.303 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.303 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.303 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.303 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.303 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.303 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.303 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.303 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.303 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.304 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.304 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.304 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.304 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.304 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.304 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.304 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.304 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.304 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.304 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.304 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.305 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.305 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.305 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.305 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.305 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.305 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.305 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.305 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.305 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.305 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.305 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.305 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.306 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.306 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.306 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.306 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.306 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.306 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.306 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.306 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.306 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.306 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.306 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.306 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.306 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.307 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.307 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.307 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.307 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.307 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.307 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.307 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.307 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.307 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-09-19T07:06:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-09-19T07:06:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:06:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T07:06:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-19T07:06:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-19T07:06:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-09-19T07:06:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-09-19T07:06:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-09-19T07:06:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-09-19T07:06:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-09-19T07:06:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-09-19T07:06:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-09-19T07:06:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-09-19T07:06:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-09-19T07:06:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T07:06:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:06:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:06:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:06:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T07:06:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-09-19T07:06:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-09-19T07:06:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:06:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:06:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T07:06:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T07:06:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-09-19T07:06:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-09-19T07:06:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:06:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T07:06:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-09-19T07:06:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-09-19T07:06:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-09-19T07:06:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:50.316 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.316 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.316 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-19T07:06:50.316 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.316 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.316 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.316 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T07:06:50.316 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.316 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.316 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.316 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.316 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-09-19T07:06:50.316 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-09-19T07:06:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-09-19T07:06:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T07:06:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-09-19T07:06:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-09-19T07:06:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-09-19T07:06:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T07:06:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-09-19T07:06:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-09-19T07:06:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T07:06:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T07:06:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-09-19T07:06:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-09-19T07:06:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:06:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T07:06:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-09-19T07:06:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T07:06:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-09-19T07:06:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T07:06:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-09-19T07:06:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-09-19T07:06:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-09-19T07:06:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-09-19T07:06:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-09-19T07:06:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T07:06:50.329 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.329 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.329 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.329 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-09-19T07:06:50.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-09-19T07:06:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T07:06:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-09-19T07:06:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-09-19T07:06:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T07:06:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-09-19T07:06:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-09-19T07:06:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T07:06:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-09-19T07:06:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-09-19T07:06:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T07:06:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:06:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-09-19T07:06:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-09-19T07:06:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T07:06:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-09-19T07:06:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-09-19T07:06:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-09-19T07:06:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T07:06:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-09-19T07:06:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-09-19T07:06:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-09-19T07:06:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T07:06:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:06:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-09-19T07:06:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-09-19T07:06:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T07:06:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-09-19T07:06:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-09-19T07:06:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-09-19T07:06:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T07:06:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-09-19T07:06:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-09-19T07:06:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-09-19T07:06:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T07:06:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-09-19T07:06:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-09-19T07:06:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-09-19T07:06:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T07:06:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-09-19T07:06:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-09-19T07:06:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-09-19T07:06:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T07:06:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-09-19T07:06:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-09-19T07:06:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-09-19T07:06:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T07:06:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-09-19T07:06:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-09-19T07:06:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-09-19T07:06:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.364 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T07:06:50.364 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.364 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.364 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.364 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:06:50.364 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.364 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.365 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.365 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-09-19T07:06:50.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-09-19T07:06:50.365 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.365 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T07:06:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-09-19T07:06:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-09-19T07:06:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-09-19T07:06:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T07:06:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:06:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-09-19T07:06:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-09-19T07:06:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T07:06:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-09-19T07:06:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-09-19T07:06:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-09-19T07:06:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T07:06:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-09-19T07:06:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-09-19T07:06:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-09-19T07:06:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T07:06:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-09-19T07:06:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-09-19T07:06:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-09-19T07:06:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T07:06:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-09-19T07:06:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-09-19T07:06:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:06:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T07:06:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-09-19T07:06:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T07:06:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-09-19T07:06:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T07:06:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-09-19T07:06:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-09-19T07:06:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-09-19T07:06:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T07:06:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-09-19T07:06:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-09-19T07:06:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-09-19T07:06:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-09-19T07:06:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-09-19T07:06:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-09-19T07:06:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T07:06:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-09-19T07:06:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-09-19T07:06:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:06:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-09-19T07:06:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-09-19T07:06:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-09-19T07:06:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T07:06:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T07:06:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-09-19T07:06:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-09-19T07:06:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:06:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-09-19T07:06:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-09-19T07:06:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-09-19T07:06:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T07:06:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-09-19T07:06:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-09-19T07:06:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.393 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.393 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.393 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.393 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.393 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T07:06:50.393 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.393 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.394 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.394 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.394 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.394 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.394 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.394 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.394 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.394 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.394 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.394 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.394 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.394 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.394 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.395 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.395 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.395 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.395 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.395 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.395 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.395 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.395 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.395 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.395 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-09-19T07:06:50.395 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-09-19T07:06:50.395 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.395 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.395 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.396 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.396 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.396 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.396 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.396 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T07:06:50.396 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.396 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.396 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.396 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.397 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.397 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.397 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.397 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.397 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.397 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.397 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.397 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T07:06:50.397 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.397 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.397 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.397 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.397 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.398 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.398 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.398 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.398 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.398 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.398 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.398 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.398 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.398 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.398 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.398 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.398 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.398 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.399 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.399 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.399 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.399 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.399 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.399 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.399 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.399 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.399 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.399 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.399 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.399 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.399 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.400 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.400 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.400 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.400 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.400 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.400 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T07:06:50.400 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T07:06:50.400 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.400 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.400 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.400 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.400 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.401 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.401 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.401 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T07:06:50.401 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.401 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.401 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.401 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T07:06:50.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T07:06:50.401 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.401 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.401 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.401 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-09-19T07:06:50.401 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.402 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.402 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.402 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T07:06:50.402 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.402 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.402 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.402 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.402 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-09-19T07:06:50.402 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-09-19T07:06:50.402 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.402 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.402 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.402 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:50.402 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.403 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.403 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.403 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-09-19T07:06:50.403 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.403 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.403 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.403 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-09-19T07:06:50.403 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-09-19T07:06:50.403 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.403 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.403 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.403 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:50.403 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.404 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.404 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.404 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T07:06:50.404 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.404 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.404 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.404 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-09-19T07:06:50.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-09-19T07:06:50.404 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.404 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.404 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.404 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.404 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.405 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.405 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.405 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-09-19T07:06:50.405 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.405 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.405 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.405 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-19T07:06:50.405 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-19T07:06:50.405 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.405 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.405 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.406 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-09-19T07:06:50.406 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.406 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.406 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.406 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-09-19T07:06:50.406 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-09-19T07:06:50.406 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.406 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T07:06:50.406 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.406 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.406 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.406 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-09-19T07:06:50.406 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-09-19T07:06:50.407 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.407 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.407 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.407 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-09-19T07:06:50.407 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.407 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.407 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.407 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T07:06:50.407 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.407 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.407 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.407 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.407 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-09-19T07:06:50.407 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-09-19T07:06:50.408 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.408 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.408 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.408 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T07:06:50.408 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.408 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.408 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.408 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T07:06:50.408 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.408 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.408 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.408 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-09-19T07:06:50.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-09-19T07:06:50.409 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.409 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.409 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.409 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.409 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.409 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.409 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.409 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T07:06:50.409 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.409 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.410 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.410 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-09-19T07:06:50.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-09-19T07:06:50.410 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.410 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.410 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.410 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.410 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.410 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.410 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.410 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T07:06:50.410 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.410 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.411 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.411 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-09-19T07:06:50.411 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-09-19T07:06:50.411 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.411 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.411 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.411 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.411 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.411 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.411 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.411 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T07:06:50.411 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.412 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.412 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.412 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-09-19T07:06:50.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-09-19T07:06:50.412 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.412 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.412 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.412 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.412 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.412 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.412 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.412 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T07:06:50.413 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.413 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.413 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.413 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-09-19T07:06:50.413 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-09-19T07:06:50.413 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.413 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.413 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.413 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.413 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.413 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.413 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.414 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T07:06:50.414 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.414 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.414 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.414 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-09-19T07:06:50.414 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-09-19T07:06:50.414 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.414 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.414 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.414 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-09-19T07:06:50.414 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.414 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.415 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.415 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-09-19T07:06:50.415 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.415 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.415 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.415 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-09-19T07:06:50.415 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-09-19T07:06:50.415 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.415 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.415 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.415 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.415 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.415 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.416 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.416 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T07:06:50.416 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.416 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.416 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.416 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-09-19T07:06:50.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-09-19T07:06:50.416 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.416 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.416 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.416 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.416 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.416 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.417 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.417 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T07:06:50.417 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.417 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.417 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.417 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-09-19T07:06:50.417 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-09-19T07:06:50.417 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.417 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.417 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.417 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.417 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.417 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.418 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.418 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T07:06:50.418 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.418 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.418 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.418 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-09-19T07:06:50.418 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-09-19T07:06:50.418 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.418 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.418 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.418 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.418 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.419 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.419 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.419 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T07:06:50.419 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.419 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.419 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.419 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-09-19T07:06:50.419 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-09-19T07:06:50.419 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.419 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.419 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.419 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.419 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.420 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.420 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.420 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T07:06:50.420 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.420 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.420 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.420 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-09-19T07:06:50.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-09-19T07:06:50.420 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.420 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.420 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.420 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.420 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.421 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.421 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.421 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T07:06:50.421 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.421 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.421 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.421 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.421 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.421 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-09-19T07:06:50.421 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.422 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.422 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.422 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.422 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.422 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.422 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.422 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.422 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.422 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.422 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.422 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.422 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.422 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.423 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.423 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.423 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.423 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.423 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.423 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.423 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.423 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.423 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.423 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.423 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.423 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.423 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.424 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.424 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.424 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.424 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.424 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.424 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.424 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.424 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.424 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.424 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.424 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.424 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.424 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.425 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.425 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.425 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.425 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.425 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.425 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.425 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.425 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.425 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.425 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.425 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.425 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.425 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.425 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.427 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.427 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.427 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.427 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-09-19T07:06:50.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-09-19T07:06:50.427 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.427 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.427 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.427 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-09-19T07:06:50.427 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.427 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.427 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.427 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T07:06:50.428 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.428 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.428 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.428 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-09-19T07:06:50.428 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-09-19T07:06:50.428 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.428 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.428 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.428 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-09-19T07:06:50.428 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.428 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.428 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.429 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T07:06:50.429 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.429 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.429 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.429 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.429 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.429 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-09-19T07:06:50.429 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.429 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.429 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.429 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T07:06:50.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T07:06:50.430 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.430 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.430 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.430 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:50.430 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.430 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.430 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.430 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.430 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.430 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.430 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.430 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.430 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T07:06:50.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T07:06:50.431 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.431 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.431 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.431 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.431 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.431 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.431 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.431 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.431 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.431 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.431 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.431 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T07:06:50.431 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T07:06:50.432 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.432 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.432 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.432 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.432 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.432 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.432 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.432 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.432 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.432 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.432 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.432 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-09-19T07:06:50.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T07:06:50.433 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.433 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.433 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.433 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.433 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.433 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.433 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.433 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.433 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.433 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.433 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.433 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T07:06:50.433 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T07:06:50.434 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.434 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.434 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.434 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.434 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.434 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.434 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.434 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.434 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.434 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.434 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.434 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.434 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T07:06:50.434 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T07:06:50.435 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.435 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.435 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.435 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.435 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.435 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.435 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.435 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.435 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.435 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.435 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.435 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.436 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T07:06:50.436 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T07:06:50.436 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.436 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.436 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.436 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.436 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.436 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.436 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.436 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.436 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.436 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.436 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.437 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T07:06:50.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T07:06:50.437 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.437 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.437 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.437 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.437 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.437 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.437 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.437 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.437 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.437 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.438 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.438 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.438 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T07:06:50.438 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T07:06:50.438 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.438 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.438 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.438 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.438 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.438 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.438 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.438 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.438 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.438 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.439 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.439 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T07:06:50.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T07:06:50.439 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.439 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.439 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.439 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.439 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.439 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.440 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.440 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.440 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.440 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.440 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.440 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T07:06:50.440 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T07:06:50.440 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.440 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.440 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.440 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.440 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.441 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.441 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.441 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.441 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.441 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.441 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.441 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T07:06:50.442 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T07:06:50.442 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.442 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.442 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.442 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.442 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.442 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.442 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.442 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.442 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.442 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.442 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.442 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T07:06:50.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T07:06:50.443 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.443 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.443 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.443 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.443 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.443 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.443 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.444 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.444 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.444 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.444 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.444 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T07:06:50.444 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T07:06:50.444 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.444 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.444 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.444 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.444 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.444 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.444 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.445 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.445 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.445 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.445 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.445 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T07:06:50.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T07:06:50.445 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.445 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.445 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.445 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:06:50.445 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.445 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.445 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.446 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.446 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.446 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.446 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.446 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T07:06:50.446 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T07:06:50.446 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.446 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.446 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.446 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.446 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.446 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.446 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.447 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.447 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.447 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.447 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.447 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.447 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-09-19T07:06:50.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-09-19T07:06:50.447 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.447 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.447 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.447 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.447 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.447 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.448 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.448 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.448 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.448 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.448 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.448 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T07:06:50.448 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T07:06:50.448 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.448 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.448 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.448 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:06:50.448 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.449 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.449 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.449 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.449 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.449 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.449 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.449 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T07:06:50.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T07:06:50.449 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.449 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.449 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.449 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.449 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.450 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.450 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.450 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.450 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.450 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.450 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.450 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T07:06:50.450 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T07:06:50.451 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.451 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.451 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.451 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.451 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.451 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.451 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.451 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.451 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.451 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.451 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.451 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T07:06:50.452 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T07:06:50.452 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.452 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.452 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.452 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.452 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.452 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.452 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.452 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.452 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.452 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.452 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.453 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.453 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T07:06:50.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T07:06:50.453 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.453 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.453 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.453 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.453 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.453 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.453 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.454 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.454 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.454 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.454 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.454 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T07:06:50.454 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T07:06:50.454 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.454 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.454 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.454 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.454 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.454 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.455 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.455 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.455 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.455 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.455 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.455 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T07:06:50.455 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T07:06:50.455 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.455 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.455 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.455 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.455 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.455 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.456 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.456 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.456 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.456 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.456 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.456 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.456 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T07:06:50.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T07:06:50.456 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.456 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.456 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.456 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.456 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.456 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.457 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.457 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.457 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.457 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.457 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.457 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.457 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T07:06:50.457 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T07:06:50.457 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.457 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.457 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.458 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.458 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.458 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.458 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.458 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.458 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.458 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.458 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.458 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T07:06:50.458 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T07:06:50.458 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.458 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.458 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.459 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.459 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.459 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.459 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.459 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.459 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.459 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.459 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.459 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T07:06:50.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T07:06:50.459 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.459 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.459 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.460 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.460 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.460 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.460 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.460 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.460 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.460 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.460 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.460 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.460 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T07:06:50.460 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T07:06:50.460 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.460 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.460 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.461 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.461 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.461 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.461 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.461 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.461 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.461 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.461 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.461 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T07:06:50.461 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T07:06:50.461 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.461 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.461 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.462 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.462 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.462 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.462 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.462 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.462 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.462 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.462 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.462 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.462 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T07:06:50.462 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T07:06:50.462 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.462 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.462 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.463 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.463 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.463 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.463 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.463 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.463 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.463 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.463 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.463 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T07:06:50.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T07:06:50.463 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.463 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.463 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.464 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.464 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.464 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.464 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.464 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.464 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.464 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.464 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.464 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T07:06:50.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T07:06:50.465 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.465 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.465 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.465 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T07:06:50.465 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.465 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.465 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.465 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.465 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.465 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.465 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.465 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.465 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T07:06:50.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T07:06:50.466 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.466 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.466 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.466 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:50.466 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.466 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.466 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.466 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.466 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.466 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.466 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.466 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T07:06:50.467 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T07:06:50.467 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.467 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.467 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.467 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-09-19T07:06:50.467 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.467 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.467 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.467 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.467 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.467 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.467 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.467 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T07:06:50.468 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T07:06:50.468 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.468 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.468 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.468 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-09-19T07:06:50.468 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.468 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.468 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.468 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.468 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.468 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.468 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.469 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T07:06:50.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T07:06:50.469 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.469 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.469 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.469 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.469 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.469 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.469 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.469 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.469 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.469 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.469 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.470 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.470 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T07:06:50.470 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T07:06:50.470 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.470 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.470 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.470 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:06:50.470 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.470 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.470 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.470 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.470 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.470 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.470 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.471 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T07:06:50.471 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T07:06:50.471 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.471 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.471 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.471 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.471 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.471 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.471 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.471 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.471 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.471 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.471 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.472 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.472 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T07:06:50.472 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T07:06:50.472 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.472 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.472 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.472 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.472 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.472 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.472 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.472 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.472 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.472 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.472 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.473 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.473 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T07:06:50.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T07:06:50.473 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.473 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.473 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.473 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:06:50.473 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.473 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.473 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.473 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.473 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.473 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.473 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.474 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T07:06:50.474 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T07:06:50.474 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.474 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.474 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.474 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:50.474 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.474 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.474 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.474 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.474 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.474 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.475 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.475 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T07:06:50.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T07:06:50.475 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.475 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.475 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.475 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:06:50.475 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.475 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.475 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.475 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.475 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.476 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.476 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.476 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-09-19T07:06:50.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-09-19T07:06:50.476 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.476 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.476 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.476 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.476 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.476 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.476 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.476 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.477 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.477 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.477 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.477 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-09-19T07:06:50.477 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-09-19T07:06:50.477 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.477 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.477 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.477 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.477 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.477 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.478 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.478 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.478 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.478 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.478 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.478 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.478 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-09-19T07:06:50.478 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-09-19T07:06:50.478 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.478 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.479 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.479 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.479 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.479 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.479 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.479 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-09-19T07:06:50.479 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.479 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.479 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.479 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-09-19T07:06:50.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-09-19T07:06:50.480 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.480 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.480 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.480 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-09-19T07:06:50.480 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.480 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.480 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.480 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.480 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.480 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.480 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.480 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.480 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-19T07:06:50.481 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-19T07:06:50.481 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.481 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.481 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.481 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.481 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.481 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.481 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.481 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.481 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.481 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.482 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.482 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.482 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.482 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.482 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.482 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.482 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.482 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.482 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.482 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.483 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.483 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.483 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.483 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.483 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.483 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.483 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.483 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.483 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.483 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.483 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.483 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.484 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.484 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.484 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.484 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.484 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.484 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.484 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.484 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.484 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.484 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.485 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.485 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.485 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.485 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.485 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.485 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.485 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.485 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.485 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.485 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.485 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.485 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.485 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.486 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.486 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.486 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.486 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.486 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.486 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.486 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.486 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.486 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.486 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.486 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.487 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.487 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.487 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.487 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.487 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.487 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.487 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.487 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.487 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.487 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.487 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-09-19T07:06:50.487 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-09-19T07:06:50.487 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.488 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.488 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.488 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.488 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.488 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.488 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.488 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-09-19T07:06:50.488 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.488 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.488 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.488 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-09-19T07:06:50.488 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T07:06:50.488 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.489 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.489 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.489 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.489 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.489 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.489 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.489 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.489 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.489 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.489 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.489 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.489 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:06:50.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:06:50.490 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.490 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.490 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.490 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-19T07:06:50.490 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.490 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.490 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.490 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.490 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.490 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.490 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.490 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.490 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:06:50.490 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:06:50.491 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.491 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.491 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.491 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-09-19T07:06:50.491 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.491 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.491 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.491 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.491 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.491 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.491 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.491 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-19T07:06:50.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-19T07:06:50.492 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.492 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.492 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.492 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.492 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.492 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.492 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.492 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.492 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.492 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.492 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.492 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-09-19T07:06:50.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-09-19T07:06:50.493 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.493 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.493 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.493 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-09-19T07:06:50.493 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.493 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.493 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.493 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.493 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.493 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.493 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.493 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.494 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-19T07:06:50.494 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-19T07:06:50.494 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.494 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.494 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.494 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-09-19T07:06:50.494 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.494 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.494 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.494 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.494 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-09-19T07:06:50.494 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.494 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.495 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.495 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.495 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.495 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.495 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-19T07:06:50.495 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-19T07:06:50.495 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.495 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.495 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.495 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-19T07:06:50.495 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-19T07:06:50.495 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-19T07:06:50.495 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.495 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.496 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.496 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.496 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.496 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.496 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-09-19T07:06:50.496 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-09-19T07:06:50.496 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.496 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.496 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.496 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.496 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.496 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.496 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.497 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.497 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.497 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.497 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.497 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.497 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.497 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-19T07:06:50.497 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.497 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.497 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.497 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-09-19T07:06:50.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-09-19T07:06:50.498 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.498 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.498 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.498 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.498 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.498 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.498 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.498 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-09-19T07:06:50.498 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.498 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.498 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.498 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.498 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.499 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.499 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.499 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.499 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.499 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.499 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.499 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.499 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.499 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.499 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.499 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.499 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.499 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.499 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.500 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.500 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.500 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.500 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.500 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.500 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.500 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.500 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.500 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.500 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.500 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.500 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.501 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.501 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.501 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.501 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.501 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.501 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.501 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.501 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.501 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.501 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.501 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.502 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.502 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.502 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.502 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.502 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.502 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.502 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.502 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.502 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.502 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.502 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.502 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.502 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.503 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.503 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.503 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.503 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.503 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.503 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.503 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.503 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.503 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.503 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.503 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.503 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.503 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.503 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.504 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.504 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-09-19T07:06:50.504 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-09-19T07:06:50.504 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.504 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.504 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.504 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-09-19T07:06:50.504 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.504 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.504 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.504 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T07:06:50.504 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.504 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.504 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.505 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-09-19T07:06:50.505 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-09-19T07:06:50.505 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.505 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.505 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.505 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-09-19T07:06:50.505 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.505 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.505 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.505 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T07:06:50.505 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.505 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.506 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.506 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.506 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-09-19T07:06:50.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-09-19T07:06:50.506 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.506 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.506 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.506 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-09-19T07:06:50.506 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.506 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.506 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.506 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-09-19T07:06:50.506 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.506 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.507 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.507 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.507 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-09-19T07:06:50.507 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-09-19T07:06:50.507 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.507 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.507 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.507 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-19T07:06:50.507 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.507 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.507 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.507 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T07:06:50.508 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.508 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.508 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.508 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.508 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-09-19T07:06:50.508 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-09-19T07:06:50.508 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.508 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.508 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.508 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.508 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.508 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.508 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.509 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T07:06:50.509 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.509 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.509 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.509 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:06:50.509 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:06:50.509 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.509 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.509 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.509 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:06:50.509 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.509 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.510 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.510 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T07:06:50.510 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.510 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.510 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.510 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-09-19T07:06:50.510 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-09-19T07:06:50.510 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.510 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.510 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.510 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-09-19T07:06:50.510 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.511 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.511 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.511 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T07:06:50.511 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.511 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.511 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.511 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.511 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.511 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.511 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.511 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-09-19T07:06:50.511 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.511 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.511 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.512 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.512 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.512 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.512 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.512 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.512 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.512 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.512 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.512 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.512 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.512 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.512 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.512 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.513 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.513 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.513 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.513 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.513 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.513 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.513 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.513 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.513 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.513 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.513 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.513 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.513 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.514 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.514 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.514 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.514 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.514 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.514 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.514 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.514 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.514 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.514 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.514 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.514 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.515 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.515 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.515 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.515 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.515 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.515 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.515 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.515 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.515 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.515 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.515 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.515 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.515 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.515 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.516 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.516 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.516 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.516 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.516 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.516 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.516 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.516 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.516 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.516 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.516 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.516 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.516 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.517 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.517 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-09-19T07:06:50.517 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-09-19T07:06:50.517 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.517 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.517 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.517 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-19T07:06:50.517 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.517 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.517 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.517 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.517 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.517 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.517 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.518 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.518 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-09-19T07:06:50.518 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-09-19T07:06:50.518 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.518 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.518 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.518 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-09-19T07:06:50.518 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.518 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.518 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.518 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.518 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.518 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.518 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.519 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.519 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:06:50.519 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:06:50.519 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.519 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.519 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.519 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:06:50.519 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.519 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.519 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.519 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.519 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.519 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.519 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-09-19T07:06:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-09-19T07:06:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-09-19T07:06:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-09-19T07:06:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-09-19T07:06:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-09-19T07:06:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T07:06:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-09-19T07:06:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-09-19T07:06:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.522 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.522 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-09-19T07:06:50.522 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.522 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.522 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.522 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T07:06:50.522 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.522 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.522 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.522 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.522 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-09-19T07:06:50.522 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-09-19T07:06:50.522 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.523 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.523 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.523 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.523 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.523 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.523 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.523 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-09-19T07:06:50.523 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.523 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.523 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.523 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.523 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:06:50.523 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:06:50.524 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.524 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.524 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.524 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-09-19T07:06:50.524 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-09-19T07:06:50.524 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.524 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.524 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T07:06:50.524 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.524 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.524 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.524 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.524 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.525 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.525 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.525 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.525 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.525 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.525 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.525 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.525 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.525 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.525 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.525 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.525 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.525 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.525 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.526 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.526 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.526 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.526 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.526 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.526 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.526 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.526 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.526 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.526 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.526 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.526 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.526 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.526 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.527 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.527 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.527 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.527 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.527 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.527 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.527 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.527 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.527 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.527 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.527 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.527 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.527 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.527 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.528 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.528 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.528 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.528 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.528 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.528 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.528 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.528 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.528 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.528 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.528 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.528 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.528 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.528 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.529 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.529 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.529 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.529 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.529 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.529 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.529 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.529 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.529 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.529 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.529 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.529 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.529 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.529 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-09-19T07:06:50.530 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-09-19T07:06:50.530 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.530 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.530 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.530 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.530 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.530 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.530 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.530 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-09-19T07:06:50.530 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.530 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.530 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.530 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.530 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-09-19T07:06:50.531 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-09-19T07:06:50.531 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.531 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.531 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.531 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-09-19T07:06:50.531 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.531 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.531 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.531 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-09-19T07:06:50.531 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.531 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.531 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.531 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.531 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-19T07:06:50.532 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-19T07:06:50.532 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.532 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.532 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.532 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-09-19T07:06:50.532 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.532 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.532 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.532 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:06:50.532 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.532 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.532 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.532 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.532 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-09-19T07:06:50.533 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-09-19T07:06:50.533 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.533 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.533 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.533 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:06:50.533 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:06:50.533 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-09-19T07:06:50.533 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.533 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T07:06:50.533 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.533 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.533 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.533 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.533 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-09-19T07:06:50.534 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-09-19T07:06:50.534 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.534 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.534 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.534 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.534 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.534 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.534 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.534 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-09-19T07:06:50.534 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.534 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.534 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.534 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.534 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-09-19T07:06:50.535 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-09-19T07:06:50.535 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.535 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.535 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.535 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-09-19T07:06:50.535 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.535 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.535 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.535 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:06:50.535 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.535 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.535 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.536 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.536 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.536 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.536 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.536 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-09-19T07:06:50.536 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.536 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.536 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.536 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.536 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.536 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.536 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.536 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.537 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.537 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.537 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.537 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.537 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.537 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.537 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.537 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.537 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.537 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.537 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.537 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.537 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.538 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.538 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.538 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.538 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.538 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.538 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.538 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.538 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.538 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.538 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.538 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.538 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.538 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.539 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.539 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.539 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.539 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.539 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.539 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.539 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.539 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.539 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.539 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.539 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.539 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.539 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.539 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.540 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.540 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.540 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.540 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.540 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.540 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.540 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.540 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.540 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.540 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.540 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.540 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.540 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.541 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.541 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.541 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.541 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.541 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.541 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.541 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.541 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.541 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.541 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.541 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.541 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.541 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.542 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.542 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.542 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-09-19T07:06:50.542 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.542 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.542 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.542 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.542 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.542 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.542 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.542 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.542 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.542 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.542 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.543 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.543 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.543 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.543 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.543 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.543 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.543 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.543 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.543 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.543 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.543 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.543 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.543 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.543 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.544 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.544 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.544 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.544 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.544 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.544 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.544 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.544 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.544 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.544 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.544 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.544 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.544 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.545 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.545 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.545 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.545 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.545 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.545 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.545 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.545 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.545 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.545 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.545 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.545 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.545 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.545 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.546 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.546 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.546 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.546 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.546 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.546 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.546 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.546 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.546 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.546 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.546 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.546 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.546 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.546 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.547 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.547 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.547 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.547 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.547 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.547 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.547 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.547 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.547 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.547 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.547 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-09-19T07:06:50.547 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.547 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.547 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.548 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-09-19T07:06:50.548 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-09-19T07:06:50.548 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.548 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.548 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.548 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-09-19T07:06:50.548 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.548 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.548 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.548 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T07:06:50.548 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.548 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.548 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.548 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.549 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-09-19T07:06:50.549 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-09-19T07:06:50.549 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.549 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.549 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.549 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:50.549 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-09-19T07:06:50.549 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.549 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.549 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T07:06:50.549 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.550 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.550 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.550 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.550 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.550 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.550 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.550 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.550 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.550 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.550 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.550 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.550 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.550 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.550 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.551 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.551 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.551 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.551 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.551 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.551 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.551 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.551 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.551 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.551 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.551 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.551 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.551 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.551 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.552 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.552 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.552 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.552 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.552 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.552 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.552 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.552 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.552 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.552 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.552 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.552 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.552 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.553 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.553 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.553 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.553 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.553 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.553 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.553 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.553 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.553 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.553 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.553 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.553 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.553 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.554 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.554 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.554 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.554 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.554 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.554 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.554 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.554 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.554 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.554 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.554 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.554 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.554 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.555 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.555 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.555 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.555 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.555 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.555 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.555 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.555 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.555 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-09-19T07:06:50.555 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.555 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.555 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.556 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-09-19T07:06:50.556 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-09-19T07:06:50.556 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.556 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.556 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.556 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-09-19T07:06:50.556 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.556 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.556 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.556 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-09-19T07:06:50.556 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T07:06:50.556 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.557 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.557 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.557 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.557 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.557 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.557 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.557 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.557 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.557 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.557 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.557 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.557 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.557 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.557 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.558 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.558 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.558 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.558 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.558 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.558 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.558 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.558 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.558 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.558 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.558 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.558 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.558 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.558 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.559 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.559 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.559 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.559 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.559 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.559 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.559 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.559 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.559 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.559 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.559 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.559 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.559 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.559 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.560 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.560 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.560 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.560 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.560 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.560 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.560 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.560 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.560 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.560 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.560 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.560 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.560 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.561 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.561 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.561 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.561 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.561 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.561 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.561 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.561 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.561 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.561 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.561 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.561 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.561 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.561 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.562 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.562 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.562 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.562 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-09-19T07:06:50.562 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-09-19T07:06:50.562 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.562 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.562 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.562 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.562 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.562 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.562 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.562 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-09-19T07:06:50.562 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T07:06:50.563 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.563 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.563 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.563 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-09-19T07:06:50.563 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-09-19T07:06:50.563 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.563 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.563 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.563 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-09-19T07:06:50.563 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.563 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.563 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.564 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-09-19T07:06:50.564 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.564 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.564 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.564 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.564 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-09-19T07:06:50.564 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-09-19T07:06:50.564 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.564 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.564 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.564 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-09-19T07:06:50.564 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.564 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.565 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.565 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T07:06:50.565 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.565 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.565 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.565 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.565 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-09-19T07:06:50.565 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-09-19T07:06:50.565 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.565 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.565 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.565 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.565 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.565 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.566 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.566 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-09-19T07:06:50.566 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.566 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.566 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.566 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.566 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-09-19T07:06:50.566 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-09-19T07:06:50.566 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.566 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.566 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.566 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-09-19T07:06:50.566 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.566 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.567 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.567 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T07:06:50.567 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T07:06:50.567 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.567 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.567 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.567 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.567 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.567 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.567 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-09-19T07:06:50.567 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.567 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.567 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.568 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.568 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.568 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.568 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.568 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.568 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.568 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.568 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.568 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.568 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.568 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.568 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.568 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.569 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.569 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.569 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.569 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.569 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.569 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.569 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.569 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.569 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.569 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.569 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.569 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.569 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.569 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.570 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.570 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.570 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.570 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.570 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.570 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.570 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.570 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.570 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.570 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.570 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.570 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.570 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.571 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.571 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.571 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.571 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.571 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.571 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.571 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.571 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.571 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.571 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.571 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.571 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.571 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.571 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.572 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.572 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.572 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.572 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.572 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.572 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.572 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.572 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.572 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.572 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.572 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.572 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.572 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.572 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.573 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.573 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.573 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.573 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.573 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.573 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-09-19T07:06:50.573 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.573 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.573 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.573 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.573 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.573 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.573 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.573 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.574 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.574 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.574 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.574 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.574 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.574 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.574 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.574 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.574 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.574 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.574 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.574 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.574 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.574 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.574 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.575 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.575 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.575 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.575 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.575 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.575 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.575 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.575 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.575 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.575 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.575 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.575 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.575 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.576 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.576 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.576 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.576 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.576 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.576 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.576 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.576 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.576 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.576 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.576 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.576 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.576 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.576 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.577 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.577 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.577 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.577 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.577 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.577 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.577 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.577 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.577 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.577 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.577 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.577 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.578 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.578 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.578 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.578 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.578 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.578 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.578 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.578 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.578 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.578 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.578 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.578 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-09-19T07:06:50.578 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.580 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.580 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.580 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.580 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.580 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.580 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.580 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.580 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.580 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.580 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.580 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.580 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.580 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.581 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.581 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.581 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.581 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.581 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.581 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.581 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.581 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.581 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.581 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.581 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.581 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.581 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.581 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.582 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.582 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.582 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.582 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.582 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.582 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.582 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.582 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.582 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.582 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.582 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.582 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.583 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.583 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.583 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.583 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.583 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.583 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.583 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.583 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.583 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.583 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.583 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.583 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.583 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.583 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.584 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.584 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.584 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.584 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.584 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.584 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.584 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.584 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.584 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-19T07:06:50.584 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.584 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.584 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.584 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-09-19T07:06:50.584 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-09-19T07:06:50.585 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.585 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.585 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.585 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.585 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.585 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.585 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.585 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T07:06:50.585 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.585 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.585 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.585 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.585 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.585 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.586 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.586 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.586 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.586 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.586 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.586 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.586 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.586 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.586 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.586 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.586 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.586 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.586 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.586 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.587 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.587 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.587 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.587 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.587 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.587 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.587 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.587 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.587 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.587 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.587 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.587 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.587 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.588 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.588 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.588 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.588 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.588 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.588 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.588 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.588 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.588 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.588 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.588 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.588 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.588 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.589 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.589 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.589 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.589 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.589 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.589 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.589 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.589 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.589 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.589 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.589 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.589 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.589 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.590 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.590 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.590 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.590 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.590 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.590 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.590 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.590 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.590 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.590 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.590 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.590 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.590 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-09-19T07:06:50.590 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-19T07:06:50.591 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.591 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.591 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.591 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.591 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.591 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.591 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.591 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-19T07:06:50.591 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-09-19T07:06:50.591 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-09-19T07:06:50.591 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.591 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-09-19T07:06:50.591 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.591 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.592 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.592 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.592 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.592 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.592 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.592 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-09-19T07:06:50.592 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.592 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.592 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.592 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.592 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.593 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.593 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.593 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.593 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.593 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.593 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.593 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.593 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.593 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.593 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.593 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.593 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.593 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.593 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.594 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.594 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.594 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.594 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.594 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.594 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.594 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.594 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.594 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.594 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.594 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.594 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.594 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.595 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.595 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.595 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.595 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.595 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.595 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.595 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.595 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.595 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.595 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.595 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.595 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.595 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.596 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.596 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.596 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.596 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.596 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.596 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.596 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.596 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.596 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.596 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.596 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.596 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.596 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.597 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.597 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.597 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.597 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.597 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.597 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.597 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.597 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.597 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.597 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.597 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.597 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.597 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.597 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.598 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.598 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.598 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-09-19T07:06:50.598 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.598 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.598 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.598 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.598 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.598 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.598 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.598 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.598 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.598 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.599 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.599 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.599 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.599 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.599 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.599 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.599 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.599 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.599 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.599 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.599 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.599 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.599 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.599 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.600 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.600 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.600 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.600 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.600 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.600 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.600 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.600 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.600 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.600 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.600 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.600 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.600 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.601 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.601 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.601 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.601 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.601 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.601 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.601 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.601 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.601 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.601 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.601 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.601 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.602 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.602 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.602 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.602 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.602 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.602 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.602 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.602 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.602 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.602 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.602 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.602 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.602 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.602 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.603 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.603 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.603 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.603 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.603 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.603 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.603 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.603 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.603 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.603 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.603 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-09-19T07:06:50.603 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.603 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.603 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.604 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.604 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.604 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.604 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.604 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.604 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.604 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.604 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.604 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.604 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.604 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.604 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.604 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.605 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.605 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.605 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.605 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.605 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.605 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.605 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.605 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.605 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.605 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.605 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.605 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.605 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.606 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.606 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.606 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.606 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.606 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.606 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.606 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.606 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.606 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.606 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.606 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.607 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.607 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.607 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.607 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.607 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.607 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.607 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.607 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.607 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.607 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.607 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.607 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.607 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.607 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.608 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.608 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.608 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.608 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.608 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.608 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.608 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.608 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.608 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.608 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.608 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.608 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.608 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.609 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.609 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.609 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.609 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.609 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:06:50.609 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:06:50.609 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.609 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.609 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.609 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:06:50.609 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.609 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.609 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.609 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-09-19T07:06:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-09-19T07:06:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-19T07:06:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-09-19T07:06:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.610 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-09-19T07:06:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T07:06:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-09-19T07:06:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-09-19T07:06:50.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-09-19T07:06:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T07:06:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.613 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-19T07:06:50.613 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-19T07:06:50.613 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.613 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.613 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.613 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.613 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.613 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.613 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.613 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.613 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.613 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.613 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.614 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.614 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.614 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.614 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.614 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.614 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.614 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.614 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.614 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.614 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.614 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.614 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.614 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.614 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.615 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.615 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.615 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.615 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.615 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.615 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.615 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.615 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.615 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.615 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.615 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.615 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.615 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.616 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.616 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.616 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.616 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.616 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.616 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.616 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.616 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.616 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.616 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.616 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.616 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.616 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.617 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.617 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.617 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.617 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.617 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.617 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.617 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.617 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.617 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.617 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.617 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.617 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.618 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.618 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.618 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.618 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.618 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.618 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.618 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.618 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.618 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.618 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.618 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.618 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.618 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.619 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.619 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.619 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-09-19T07:06:50.619 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-09-19T07:06:50.619 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.619 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.619 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.619 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-09-19T07:06:50.619 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.619 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.619 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.619 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-09-19T07:06:50.619 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.620 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.620 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.620 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:06:50.620 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:06:50.620 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.620 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.620 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.620 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:06:50.620 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.620 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.620 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.620 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-09-19T07:06:50.621 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.621 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.621 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.621 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.621 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.621 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.621 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-09-19T07:06:50.621 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.621 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.621 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.621 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-09-19T07:06:50.621 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-09-19T07:06:50.622 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.622 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.622 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.622 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.622 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.622 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.622 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.622 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.622 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.622 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.622 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.622 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-09-19T07:06:50.622 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-09-19T07:06:50.623 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.623 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.623 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.623 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.623 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.623 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.623 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.623 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T07:06:50.623 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T07:06:50.623 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.623 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.623 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.624 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.624 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.624 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.624 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.624 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.624 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.624 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.624 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.624 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.624 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.624 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.624 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.624 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.625 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.625 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.625 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.625 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.625 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.625 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.625 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.625 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.625 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.625 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.625 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.625 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.626 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.626 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.626 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.626 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.626 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.626 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.626 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.626 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.626 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.626 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.626 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.626 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.626 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.627 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.627 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.627 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.627 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.627 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.627 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.627 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.627 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.627 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.627 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.627 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.627 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.627 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.628 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.628 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.628 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.628 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.628 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.628 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.628 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.628 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.628 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.628 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.628 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.628 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.628 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.629 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-09-19T07:06:50.629 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-09-19T07:06:50.629 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.629 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.629 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.629 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.629 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.629 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.629 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.629 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.629 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.629 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.629 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.630 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T07:06:50.630 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T07:06:50.630 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.630 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.630 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.630 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-09-19T07:06:50.630 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.630 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.630 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.630 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.630 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.630 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.630 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.631 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-09-19T07:06:50.631 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-09-19T07:06:50.631 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:50.631 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.631 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.631 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-09-19T07:06:50.631 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.631 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.631 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.631 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.631 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.631 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.632 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.632 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:06:50.632 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:06:50.632 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.632 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.632 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.632 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-19T07:06:50.632 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.632 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.632 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.632 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T07:06:50.632 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.632 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.633 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.633 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:06:50.633 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:06:50.633 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.633 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.633 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.633 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-09-19T07:06:50.633 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.633 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.633 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.633 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T07:06:50.633 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.633 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.634 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.634 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-09-19T07:06:50.634 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-09-19T07:06:50.634 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.634 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.634 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.634 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-09-19T07:06:50.634 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.634 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.634 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.634 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.635 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.635 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.635 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.635 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-19T07:06:50.635 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-19T07:06:50.635 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.635 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.635 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.635 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-09-19T07:06:50.635 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.635 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.635 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.635 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-09-19T07:06:50.636 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-09-19T07:06:50.636 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.636 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.636 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.636 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.636 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.636 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-19T07:06:50.636 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-19T07:06:50.636 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.636 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.636 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.637 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-19T07:06:50.637 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-19T07:06:50.637 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-19T07:06:50.637 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.637 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.637 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.637 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.637 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.637 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.637 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.637 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.637 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-19T07:06:50.638 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.638 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.638 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.638 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.638 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.638 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.638 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.638 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.638 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.638 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.638 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.638 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.638 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.638 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.639 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.639 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.639 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.639 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.639 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.639 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.639 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.639 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.639 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.639 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.639 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.639 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.640 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.640 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.640 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.640 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.640 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.640 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.640 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.640 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.640 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.640 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.640 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.640 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.641 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.641 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.641 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.641 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.641 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.641 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.641 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.641 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.641 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.641 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.641 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.641 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.641 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.641 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.642 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.642 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.642 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.642 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.642 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.642 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.642 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.642 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.642 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.642 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.642 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.642 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.642 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.643 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.643 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.643 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.643 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.643 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-09-19T07:06:50.643 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-09-19T07:06:50.643 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.643 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.643 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.643 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:50.643 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.643 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.643 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.643 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.644 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.644 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.644 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.644 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-09-19T07:06:50.644 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T07:06:50.644 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.644 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.644 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.644 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.644 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.644 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.644 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.645 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.645 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.645 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.645 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.645 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-09-19T07:06:50.645 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-09-19T07:06:50.645 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.645 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.645 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.645 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.645 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.645 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.645 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.646 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.646 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.646 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.646 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.646 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.646 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.646 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.646 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-09-19T07:06:50.646 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.646 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.646 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.646 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-09-19T07:06:50.647 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-09-19T07:06:50.647 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.647 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.647 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.647 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.647 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.647 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.647 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.647 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.647 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.647 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.647 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.647 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-19T07:06:50.648 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-19T07:06:50.648 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.648 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.648 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.648 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.648 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.648 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.648 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.648 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.648 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.649 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.649 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.649 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.649 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.649 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.649 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.649 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.649 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.649 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.649 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.649 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.649 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.650 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.650 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.650 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.650 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.650 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.650 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.650 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.650 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.650 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.650 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.650 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.650 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.650 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.651 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.651 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.651 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.651 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.651 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.651 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.651 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.651 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.651 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.651 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.651 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.651 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.651 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.652 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.652 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.652 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.652 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.652 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.652 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.652 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.652 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.652 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.652 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.652 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.652 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.652 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.653 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.653 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.653 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.653 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.653 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.653 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.653 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.653 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.653 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.653 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.653 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.653 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.653 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.654 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.654 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.654 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.654 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.654 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:06:50.654 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:06:50.654 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.654 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.654 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.654 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.654 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.654 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.654 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.655 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.655 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.655 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.655 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.655 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:06:50.655 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:06:50.655 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.655 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.655 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.655 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.655 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.655 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.655 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.656 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.656 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.656 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.656 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.656 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.656 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.656 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.656 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-09-19T07:06:50.656 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.656 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.656 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.656 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.656 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.657 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.657 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.657 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.657 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.657 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.657 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.657 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.657 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.657 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.657 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.657 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.657 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.657 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.657 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.658 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.658 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.658 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.658 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.658 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.658 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.658 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.658 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.658 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.658 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.658 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.658 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.659 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.659 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.659 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.659 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.659 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.659 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.659 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.659 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.659 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.659 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.659 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.659 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.660 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.660 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.660 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.660 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.660 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.660 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.660 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.660 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.660 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.660 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.660 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.660 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.660 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.661 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.661 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.661 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.661 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.661 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.661 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.661 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.661 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.661 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.661 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.661 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.661 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.661 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.662 INFO:teuthology.orchestra.run.smithi103.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T07:06:50.662 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T07:06:50.662 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.662 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.662 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.662 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:06:50.662 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.662 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.662 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.662 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T07:06:50.662 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.662 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.663 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.663 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.663 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.663 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.663 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.663 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-09-19T07:06:50.663 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.663 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.663 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.663 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.663 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.663 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.664 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.664 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.664 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.664 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.664 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.664 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.664 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.664 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.664 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.664 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.664 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.664 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.665 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.665 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.665 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.665 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.665 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.665 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.665 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.665 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.665 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.666 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.666 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.666 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.666 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.666 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.666 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.666 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.666 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.666 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.666 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.666 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.666 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.667 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.667 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.667 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.667 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.667 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.667 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.667 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.667 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.667 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.667 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.667 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.667 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.667 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.668 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.668 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.668 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.668 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.668 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.668 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.668 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.668 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.668 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.668 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.668 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.668 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.668 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.668 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.669 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.669 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.669 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.669 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.669 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-09-19T07:06:50.669 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-09-19T07:06:50.669 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.669 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.669 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.669 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-09-19T07:06:50.669 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.669 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.670 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.670 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T07:06:50.670 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.670 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.670 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.670 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.670 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.670 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.670 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-09-19T07:06:50.670 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.670 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.670 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.670 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.671 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.671 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.671 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.671 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.671 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.671 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.671 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.671 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.671 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.671 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.671 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.671 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.671 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.671 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.672 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.672 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.672 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.672 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.672 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.672 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.672 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.672 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.672 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.672 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.672 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.672 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.672 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.673 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.673 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.673 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.673 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.673 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.673 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.673 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.673 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.673 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.673 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.673 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.673 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.673 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.674 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.674 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.674 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.674 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.674 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.674 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.674 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.674 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.674 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.674 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.674 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.674 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.674 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.674 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-09-19T07:06:50.676 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-09-19T07:06:50.676 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.676 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.676 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.676 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.676 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.676 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.676 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.676 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.676 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.676 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.677 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.677 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-09-19T07:06:50.677 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-09-19T07:06:50.677 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.677 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.677 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.677 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-09-19T07:06:50.677 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.677 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.677 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.677 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.678 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.678 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.678 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.678 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-09-19T07:06:50.678 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-09-19T07:06:50.678 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.678 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.678 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.678 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.678 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.678 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.679 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.679 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.679 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.679 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.679 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.679 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-09-19T07:06:50.679 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-09-19T07:06:50.679 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.679 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.679 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.679 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.679 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.679 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.680 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.680 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.680 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.680 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.680 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.680 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-09-19T07:06:50.680 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-09-19T07:06:50.680 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:50.680 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.680 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.680 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.680 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.681 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.681 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.681 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.681 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.681 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.681 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.681 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.681 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-09-19T07:06:50.681 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-09-19T07:06:50.681 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.681 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.681 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.681 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.682 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.682 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.682 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.682 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.682 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.682 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.682 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.682 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.682 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-09-19T07:06:50.682 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-09-19T07:06:50.682 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.682 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.682 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.683 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.683 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.683 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.683 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.683 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.683 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.683 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.683 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.683 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.683 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-09-19T07:06:50.683 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-09-19T07:06:50.683 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.684 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.684 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.684 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.684 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.684 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.684 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.684 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.684 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.684 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.684 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.684 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.684 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-09-19T07:06:50.684 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-09-19T07:06:50.685 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.685 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.685 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.685 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.685 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:06:50.685 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-09-19T07:06:50.685 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.685 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.685 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.685 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.685 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.685 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.685 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-09-19T07:06:50.686 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-09-19T07:06:50.686 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.686 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.686 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.686 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.686 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.686 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.686 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.686 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.686 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.686 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.686 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.686 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.687 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-09-19T07:06:50.687 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-09-19T07:06:50.687 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.687 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.687 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.687 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.687 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.687 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.687 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.687 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.687 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.687 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.687 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.688 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.688 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-09-19T07:06:50.688 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-09-19T07:06:50.688 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.688 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.688 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.688 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.688 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.688 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.688 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.688 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.688 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.688 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.689 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.689 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.689 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.689 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.689 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-19T07:06:50.689 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.689 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.689 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.689 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-09-19T07:06:50.689 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-09-19T07:06:50.689 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.689 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.690 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.690 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.690 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.690 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.690 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.690 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T07:06:50.690 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.690 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.690 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.690 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.690 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-09-19T07:06:50.690 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-09-19T07:06:50.690 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.690 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.691 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.691 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.691 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.691 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.691 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.691 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-09-19T07:06:50.691 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.691 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.691 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.691 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.691 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.691 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.691 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.692 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.692 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.692 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.692 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.692 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.692 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.692 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.692 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.692 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.692 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.692 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.692 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.692 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.692 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.693 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.693 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.693 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.693 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.693 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.693 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.693 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.693 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.693 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.693 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.693 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.693 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.694 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.694 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.694 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.694 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.694 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.694 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.694 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.694 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.694 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.694 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.694 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.694 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.694 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.695 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.695 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.695 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.695 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.695 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.695 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.695 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.695 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.695 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.695 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.695 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.695 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.695 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.695 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.696 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.696 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.696 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.696 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.696 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.696 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.696 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.696 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.696 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.696 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.696 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.696 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.696 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.697 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.697 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.697 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.697 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.697 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-09-19T07:06:50.697 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.697 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.697 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.697 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.697 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.697 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.697 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.697 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.698 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.698 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.698 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.698 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.698 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.698 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.698 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.698 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.698 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.698 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.698 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.698 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.698 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.699 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.699 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.699 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.699 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.699 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.699 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.699 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.699 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.699 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.699 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.699 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.699 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.699 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.700 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.700 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.700 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.700 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.700 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.700 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.700 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.700 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.700 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.700 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.700 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.700 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.700 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.701 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.701 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.701 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.701 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.701 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.701 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.701 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.701 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.701 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.701 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.701 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.701 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.701 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.701 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.702 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.702 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.702 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.702 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.702 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.702 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.702 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.702 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.702 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.702 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.702 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.702 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.702 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.702 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.703 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.703 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-09-19T07:06:50.703 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.703 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.703 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.703 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.703 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.703 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.703 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.703 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.703 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.703 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.703 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.703 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.704 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.704 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.704 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.704 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.704 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.704 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.704 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.704 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.704 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.704 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.704 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.704 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.705 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.705 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.705 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.705 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.705 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.705 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.705 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.705 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.705 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.705 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.705 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.705 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.706 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.706 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.706 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.706 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.706 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.706 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.706 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.706 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.706 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.706 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.706 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.706 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.706 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.706 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.707 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.707 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.707 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.707 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.707 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.707 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.707 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.707 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.707 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.707 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.707 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.707 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.707 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.707 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.707 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.708 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.708 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.708 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.708 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.708 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.708 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.708 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.708 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.708 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.708 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-09-19T07:06:50.708 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.709 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.709 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.709 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-09-19T07:06:50.709 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-09-19T07:06:50.709 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.709 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.709 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.709 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T07:06:50.709 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.709 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.709 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.709 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.709 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.709 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.710 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.710 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.710 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:06:50.710 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:06:50.710 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.710 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.710 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.710 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-09-19T07:06:50.710 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.710 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.710 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.710 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.710 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.711 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.711 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.711 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.711 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.711 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.711 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.711 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.711 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.711 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.711 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.711 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.711 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.711 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.711 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.712 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.712 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.712 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.712 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.712 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.712 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.712 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.712 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.712 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.712 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.712 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.712 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.712 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.712 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.713 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.713 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.713 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.713 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.713 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.713 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.713 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.713 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.713 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.713 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.713 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.713 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.713 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.713 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.714 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.714 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.714 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.714 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.714 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.714 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.714 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.714 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.714 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.714 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.714 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.714 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.714 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.714 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.715 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.715 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.715 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.715 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.715 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.715 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.715 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.715 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.715 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.715 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.715 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.715 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.715 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.715 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.716 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.716 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.716 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.716 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.716 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.716 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.716 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-09-19T07:06:50.716 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.716 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.716 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.716 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-09-19T07:06:50.716 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-09-19T07:06:50.716 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.717 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.717 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.717 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.717 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.717 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.717 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.717 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T07:06:50.717 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.717 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.717 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.717 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.717 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-09-19T07:06:50.717 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-09-19T07:06:50.717 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.718 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.718 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.718 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.718 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.718 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.718 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.718 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T07:06:50.718 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.718 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.718 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.718 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.718 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-09-19T07:06:50.718 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-09-19T07:06:50.718 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.719 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.719 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.719 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.719 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.719 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.719 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.719 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T07:06:50.719 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.719 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.719 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.719 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.719 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-09-19T07:06:50.719 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-09-19T07:06:50.719 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.720 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.720 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.720 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.720 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.720 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.720 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.720 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T07:06:50.720 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.720 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.720 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.720 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.720 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-09-19T07:06:50.720 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-09-19T07:06:50.721 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.721 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.721 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.721 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.721 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.721 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.721 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.721 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T07:06:50.721 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.721 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.721 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.721 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-09-19T07:06:50.722 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-09-19T07:06:50.722 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.722 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.722 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.722 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.722 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.722 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.722 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.722 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.722 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.722 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.722 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.722 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.723 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-09-19T07:06:50.723 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-09-19T07:06:50.723 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.723 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.723 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.723 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.723 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.723 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.723 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.723 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.723 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.723 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.723 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.723 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.724 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-09-19T07:06:50.724 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-09-19T07:06:50.724 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.724 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.724 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.724 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T07:06:50.724 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.724 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.724 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.724 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.724 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.724 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.724 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.725 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.725 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-19T07:06:50.725 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-19T07:06:50.725 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.725 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.725 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.725 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.725 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.725 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.725 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.725 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.725 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.725 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.725 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.726 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.726 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:06:50.726 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:06:50.726 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.726 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.726 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.726 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-09-19T07:06:50.726 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-09-19T07:06:50.726 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.726 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.726 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.726 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.726 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.727 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.727 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.727 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-09-19T07:06:50.727 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-09-19T07:06:50.727 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.727 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.727 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.727 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-09-19T07:06:50.727 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.727 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.727 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.727 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.727 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.727 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.728 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.728 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.728 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-09-19T07:06:50.728 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-09-19T07:06:50.728 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.728 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.728 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.728 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.728 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.728 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.728 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.728 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.728 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.729 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.729 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.729 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.729 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-09-19T07:06:50.729 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-09-19T07:06:50.729 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.729 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.729 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.729 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.729 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.729 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.729 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.729 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.729 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.730 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.730 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.730 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.730 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.730 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.730 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.730 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.730 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.730 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.730 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.730 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.730 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.730 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.730 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.731 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.731 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.731 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.731 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.731 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.731 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.731 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.731 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.731 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.731 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.731 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.731 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.731 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.731 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.732 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.732 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.732 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.732 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.732 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.732 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.732 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.732 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.732 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.732 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.732 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.732 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.732 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.732 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.733 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.733 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.733 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.733 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.733 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.733 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.733 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.733 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.733 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.733 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.733 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.733 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.733 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.734 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.734 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.734 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.734 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.734 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.734 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.734 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.734 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.734 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.734 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.734 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.734 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.734 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.734 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.735 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.735 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.735 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.735 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.735 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-09-19T07:06:50.735 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-09-19T07:06:50.735 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.735 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.735 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.735 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.735 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.735 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.735 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.735 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.736 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.736 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.736 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.736 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.736 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-09-19T07:06:50.736 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-09-19T07:06:50.736 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.736 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.736 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.736 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.736 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.736 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.737 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.737 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.737 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.737 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.737 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.737 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.737 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-09-19T07:06:50.737 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-09-19T07:06:50.737 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.737 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.737 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.737 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T07:06:50.737 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.737 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.738 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.738 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.738 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.738 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.738 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.738 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.738 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.738 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.738 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.738 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-09-19T07:06:50.738 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.738 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.738 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.738 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.739 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.739 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.739 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.739 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.739 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.739 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.739 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.739 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.739 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.739 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.739 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.739 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.739 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.740 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.740 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.740 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.740 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.740 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.740 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.740 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.740 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.740 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.740 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.740 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.740 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.740 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.740 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.741 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.741 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.741 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.741 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.741 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.741 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.741 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.741 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.741 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.741 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.741 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.741 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.741 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.742 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.742 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.742 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.742 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.742 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.742 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.742 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.742 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.742 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.742 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.742 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.742 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.742 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.742 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.743 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.743 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.743 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.743 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.743 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.743 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.743 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.743 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.743 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.743 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.743 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.743 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.743 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.744 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.744 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.744 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.744 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.744 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.744 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-19T07:06:50.744 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.744 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.744 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.744 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.744 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.744 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.744 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.745 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.745 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.745 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.745 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.745 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.745 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.745 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.745 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.745 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.745 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.745 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.745 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.745 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.745 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.746 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.746 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.746 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.746 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.746 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.746 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.746 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.746 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.746 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.746 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.746 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.746 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.746 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.746 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.747 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.747 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.747 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.747 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.747 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.747 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.747 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.747 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.747 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.747 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.747 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.747 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.747 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.747 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.748 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.748 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.748 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.748 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.748 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.748 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.748 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.748 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.748 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.748 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.748 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.748 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.748 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.749 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.749 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.749 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.749 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.749 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.749 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.749 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.749 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.749 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.749 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.749 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-09-19T07:06:50.749 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-09-19T07:06:50.749 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.750 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.750 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.750 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-09-19T07:06:50.750 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.750 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.750 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.750 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T07:06:50.750 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.750 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.750 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.750 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.750 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-09-19T07:06:50.750 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-09-19T07:06:50.751 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.751 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.751 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.751 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.751 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.751 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.751 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.751 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-09-19T07:06:50.751 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.751 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.751 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.751 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.751 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-09-19T07:06:50.752 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-09-19T07:06:50.752 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.752 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.752 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.752 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:06:50.752 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.752 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.752 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.752 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T07:06:50.752 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.752 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.752 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.752 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.753 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-09-19T07:06:50.753 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T07:06:50.753 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.753 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.753 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.753 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.753 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.753 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.753 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.753 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T07:06:50.753 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.753 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.753 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.753 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.754 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.754 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.754 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.754 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-09-19T07:06:50.754 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.754 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.754 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.754 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-09-19T07:06:50.754 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-09-19T07:06:50.754 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:06:50.754 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.754 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.754 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-09-19T07:06:50.755 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.755 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.755 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.755 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.755 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.755 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.755 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.755 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.755 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-09-19T07:06:50.755 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-09-19T07:06:50.755 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.755 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.755 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.756 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.756 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.756 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.756 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.756 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.756 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.756 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.756 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.756 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.756 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:06:50.756 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:06:50.756 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.756 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.756 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.757 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:06:50.757 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.757 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.757 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.757 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.757 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.757 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.757 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.757 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.757 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.757 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.757 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.757 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.757 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.758 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.758 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.758 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.758 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.758 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.758 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.758 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.758 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.758 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.758 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.758 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.758 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.758 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.758 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.759 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.759 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.759 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.759 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.759 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.759 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.759 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.759 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.759 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.759 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.759 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.759 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.759 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.759 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.760 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.760 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.760 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.760 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.760 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.760 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.760 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.760 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.760 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.760 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.760 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.760 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.760 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.760 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.761 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.761 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.761 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.761 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.761 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.761 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.761 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.761 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.761 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.761 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.761 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.761 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.761 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.761 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.762 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.762 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.762 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.762 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.762 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.762 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.762 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.762 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.762 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.762 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-09-19T07:06:50.762 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-09-19T07:06:50.762 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.762 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.762 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.763 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.763 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.763 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.763 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.763 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.763 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.763 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.763 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.763 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.763 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-09-19T07:06:50.763 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-09-19T07:06:50.763 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.763 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.763 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.764 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-09-19T07:06:50.764 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.764 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.764 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.764 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.764 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.764 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.764 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.764 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.764 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-09-19T07:06:50.764 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-09-19T07:06:50.764 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.765 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.765 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.765 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T07:06:50.765 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.765 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.765 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.765 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.765 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.765 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.765 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.765 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.765 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.765 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.765 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:06:50.766 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.766 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.766 INFO:teuthology.orchestra.run.smithi103.stdout: "gid": 24775, 2024-09-19T07:06:50.766 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "z", 2024-09-19T07:06:50.766 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_features": 4540701547738038271, 2024-09-19T07:06:50.766 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-09-19T07:06:50.766 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.766 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-09-19T07:06:50.766 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.766 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.766 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.766 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:06:50.766 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:06:50.766 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.767 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.767 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.767 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:06:50.767 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.767 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.767 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.767 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T07:06:50.767 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.767 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.767 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.767 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.767 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.767 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.767 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.768 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.768 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.768 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.768 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.768 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.768 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.768 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.768 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.768 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.768 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.768 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.768 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.768 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.769 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.769 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.769 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.769 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.769 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.769 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.769 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.769 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.769 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.769 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.769 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.769 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.769 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.770 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.770 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.770 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.770 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.770 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.770 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.770 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.770 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.770 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.770 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.770 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.770 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.770 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.771 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.771 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.771 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.771 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.771 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.771 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.771 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.771 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.771 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.771 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.771 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.771 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.771 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.771 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.772 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.772 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.772 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.772 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.772 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.772 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.772 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.772 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.772 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.772 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.772 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.772 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.772 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.773 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.773 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-09-19T07:06:50.773 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-09-19T07:06:50.773 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.773 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.773 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.773 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.773 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.773 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.773 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.773 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to", 2024-09-19T07:06:50.773 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.773 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.773 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.774 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.774 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-09-19T07:06:50.774 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-09-19T07:06:50.774 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.774 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.774 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.774 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-09-19T07:06:50.774 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.774 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.774 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.774 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-09-19T07:06:50.774 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.774 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.774 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.775 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.775 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-09-19T07:06:50.775 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-09-19T07:06:50.775 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.775 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.775 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.775 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.775 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.775 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.775 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.775 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-09-19T07:06:50.775 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.775 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.776 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.776 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.776 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-09-19T07:06:50.776 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-09-19T07:06:50.776 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.776 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.776 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.776 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.776 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.776 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.776 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.776 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-09-19T07:06:50.776 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.776 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.777 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.777 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.777 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-09-19T07:06:50.777 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-09-19T07:06:50.777 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.777 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.777 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.777 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-09-19T07:06:50.777 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.777 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.777 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.777 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-09-19T07:06:50.777 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.778 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.778 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.778 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.778 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-09-19T07:06:50.778 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-09-19T07:06:50.778 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.778 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.778 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.778 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.778 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.778 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.778 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.778 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-09-19T07:06:50.779 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.779 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.779 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.779 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.779 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-09-19T07:06:50.779 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-09-19T07:06:50.779 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.779 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.779 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.779 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.779 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.779 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.780 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.780 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T07:06:50.780 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.780 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.780 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.780 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.780 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-09-19T07:06:50.780 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-09-19T07:06:50.780 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.780 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.780 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.780 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.780 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.781 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.781 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.781 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-09-19T07:06:50.781 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.781 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.781 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.781 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.781 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.781 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.781 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.781 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-09-19T07:06:50.781 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.781 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.781 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.782 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-09-19T07:06:50.782 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-09-19T07:06:50.782 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.782 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.782 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.782 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.782 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.782 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.782 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.782 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T07:06:50.782 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.782 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.782 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.782 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.783 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-09-19T07:06:50.783 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-09-19T07:06:50.783 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.783 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.783 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.783 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-09-19T07:06:50.783 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.783 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.783 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.783 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T07:06:50.783 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:06:50.783 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.783 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.784 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.784 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-09-19T07:06:50.784 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-09-19T07:06:50.784 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:06:50.784 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.784 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.784 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:06:50.784 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-19T07:06:50.784 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-19T07:06:50.784 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.784 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T07:06:50.784 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:06:50.784 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.784 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.785 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.785 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-09-19T07:06:50.785 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-09-19T07:06:50.785 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:06:50.785 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.785 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.785 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-09-19T07:06:50.785 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:06:50.785 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-09-19T07:06:50.785 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.785 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T07:06:50.785 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.785 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.785 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.786 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.786 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-09-19T07:06:50.786 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-09-19T07:06:50.786 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.786 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.786 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.786 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T07:06:50.786 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.786 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.786 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.786 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T07:06:50.786 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T07:06:50.786 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.786 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.787 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.787 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-09-19T07:06:50.787 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-09-19T07:06:50.787 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:50.787 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.787 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.787 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-09-19T07:06:50.787 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-09-19T07:06:50.787 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-09-19T07:06:50.787 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.787 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-09-19T07:06:50.787 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T07:06:50.787 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.787 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.788 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.788 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-09-19T07:06:50.788 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-09-19T07:06:50.788 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.788 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.788 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.788 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-09-19T07:06:50.788 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.788 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.788 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.788 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T07:06:50.788 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:06:50.788 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.788 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.789 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.789 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-09-19T07:06:50.789 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-09-19T07:06:50.789 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:06:50.789 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.789 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.789 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:06:50.789 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-19T07:06:50.789 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-19T07:06:50.789 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.789 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T07:06:50.789 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:06:50.789 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.790 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.790 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.790 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.790 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.790 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.790 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.790 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.790 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.790 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.790 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.790 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.790 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.790 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.791 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.792 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.793 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.794 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-09-19T07:06:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-09-19T07:06:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:50.795 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:06:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T07:06:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-19T07:06:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-19T07:06:50.796 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-09-19T07:06:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-09-19T07:06:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-09-19T07:06:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-09-19T07:06:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-09-19T07:06:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-09-19T07:06:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.797 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-09-19T07:06:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-09-19T07:06:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-09-19T07:06:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.798 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T07:06:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:06:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:06:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:06:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.799 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T07:06:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-09-19T07:06:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-09-19T07:06:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:06:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:06:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.800 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T07:06:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T07:06:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.801 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.805 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-09-19T07:06:50.805 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-09-19T07:06:50.805 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:06:50.805 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.805 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.806 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:50.806 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.806 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.806 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.806 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T07:06:50.806 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.806 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.806 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.806 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.806 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.806 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.806 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.806 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-09-19T07:06:50.806 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.807 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.807 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.807 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-09-19T07:06:50.807 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-09-19T07:06:50.807 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:50.807 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.807 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.807 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-19T07:06:50.807 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.807 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.807 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.807 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T07:06:50.807 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.807 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.808 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.808 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.808 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-09-19T07:06:50.808 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-09-19T07:06:50.808 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.808 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.808 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.808 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-09-19T07:06:50.808 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.808 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.808 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.808 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T07:06:50.808 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.809 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.809 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.809 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.809 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-09-19T07:06:50.809 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-09-19T07:06:50.809 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.809 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.809 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.809 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-09-19T07:06:50.809 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.809 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.809 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.809 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T07:06:50.810 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.810 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.810 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.810 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.810 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-09-19T07:06:50.810 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-09-19T07:06:50.810 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.810 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.810 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.810 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.810 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.810 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.810 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.811 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T07:06:50.811 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T07:06:50.811 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.811 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.811 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.811 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-09-19T07:06:50.811 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-09-19T07:06:50.811 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.811 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.811 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.811 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:06:50.811 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.811 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.811 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.812 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T07:06:50.812 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.812 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.812 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.812 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.812 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-09-19T07:06:50.812 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T07:06:50.812 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:50.812 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.812 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.812 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-09-19T07:06:50.812 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.812 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.813 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.813 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T07:06:50.813 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.813 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.813 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.813 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.813 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-09-19T07:06:50.813 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-09-19T07:06:50.813 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.813 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.813 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.813 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.813 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.814 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.814 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.814 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-09-19T07:06:50.814 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-09-19T07:06:50.814 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-09-19T07:06:50.814 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.814 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T07:06:50.814 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.814 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.814 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.814 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.814 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-09-19T07:06:50.814 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-09-19T07:06:50.814 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.815 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.815 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.815 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.815 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.815 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.815 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.815 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T07:06:50.815 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.815 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.815 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.815 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.815 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-09-19T07:06:50.815 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-09-19T07:06:50.816 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.816 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.816 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.816 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.816 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.816 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.816 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.816 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T07:06:50.816 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.816 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.816 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.816 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.816 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-09-19T07:06:50.816 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-09-19T07:06:50.817 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.817 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.817 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.817 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.817 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.817 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.817 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.817 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T07:06:50.817 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.817 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.817 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.817 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.817 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-09-19T07:06:50.818 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-09-19T07:06:50.818 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.818 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.818 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.818 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T07:06:50.818 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.818 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.818 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.818 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:06:50.818 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.818 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.818 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.818 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.818 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-09-19T07:06:50.819 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-09-19T07:06:50.819 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.819 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.819 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.819 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T07:06:50.819 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.819 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.819 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.819 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-09-19T07:06:50.819 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.819 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.819 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.819 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.820 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-09-19T07:06:50.820 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-09-19T07:06:50.820 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.820 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.820 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.820 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T07:06:50.820 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.820 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.820 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.820 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-09-19T07:06:50.820 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.820 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.820 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.821 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.821 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-09-19T07:06:50.821 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-09-19T07:06:50.821 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.821 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.821 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.821 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T07:06:50.821 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.821 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.821 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.821 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:06:50.821 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.821 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.822 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.822 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.822 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-09-19T07:06:50.822 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-09-19T07:06:50.822 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.822 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.822 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.822 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T07:06:50.822 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.822 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.822 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.822 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-09-19T07:06:50.823 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.823 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.823 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.823 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.823 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-09-19T07:06:50.823 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-09-19T07:06:50.823 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.823 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.823 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.823 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T07:06:50.823 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.823 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.824 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.824 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-09-19T07:06:50.824 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.824 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.824 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.824 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.824 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-09-19T07:06:50.824 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-09-19T07:06:50.824 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.824 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.824 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.824 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T07:06:50.825 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.825 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.825 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.825 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-09-19T07:06:50.825 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.825 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.825 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.825 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.825 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-09-19T07:06:50.825 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-09-19T07:06:50.825 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.825 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.825 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.826 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T07:06:50.826 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.826 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.826 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.826 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-09-19T07:06:50.826 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.826 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.826 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.826 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.826 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-09-19T07:06:50.826 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-09-19T07:06:50.826 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.826 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.827 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.827 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T07:06:50.827 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.827 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.827 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.827 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-09-19T07:06:50.827 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.827 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.827 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.827 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.827 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-09-19T07:06:50.827 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-09-19T07:06:50.827 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.827 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.828 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.828 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T07:06:50.828 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.828 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.828 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.828 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-09-19T07:06:50.828 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.828 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.828 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.828 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.828 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-09-19T07:06:50.828 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-09-19T07:06:50.828 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.828 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.829 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.829 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T07:06:50.829 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.829 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.829 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.829 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:06:50.829 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.829 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.829 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.829 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.829 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-09-19T07:06:50.829 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-09-19T07:06:50.829 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.829 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.830 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.830 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T07:06:50.830 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.830 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.830 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.830 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-09-19T07:06:50.830 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.830 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.830 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.830 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.830 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-09-19T07:06:50.830 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-09-19T07:06:50.830 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.831 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.831 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.831 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T07:06:50.831 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.831 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.831 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.831 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:06:50.831 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.831 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.831 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.831 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.831 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-09-19T07:06:50.831 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-09-19T07:06:50.831 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.832 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.832 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.832 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T07:06:50.832 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.832 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.832 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.832 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-09-19T07:06:50.832 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.832 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.832 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.832 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.832 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-09-19T07:06:50.832 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-09-19T07:06:50.832 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.833 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.833 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.833 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T07:06:50.833 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.833 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.833 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.833 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-09-19T07:06:50.833 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.833 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.833 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.833 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.833 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-09-19T07:06:50.833 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-09-19T07:06:50.833 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.834 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.834 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.834 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T07:06:50.834 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.834 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.834 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.834 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-09-19T07:06:50.834 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.834 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.834 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.834 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.834 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-09-19T07:06:50.834 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-09-19T07:06:50.834 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.835 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.835 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.835 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.835 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.835 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.835 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.835 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T07:06:50.835 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.835 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.835 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.835 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.835 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-09-19T07:06:50.836 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-09-19T07:06:50.836 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.836 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.836 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.836 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:06:50.836 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.836 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.836 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.836 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T07:06:50.836 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.836 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.836 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.836 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.836 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-09-19T07:06:50.837 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T07:06:50.837 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.837 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.837 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.837 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-09-19T07:06:50.837 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.837 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.837 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.837 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T07:06:50.837 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.837 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.837 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.837 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.838 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-09-19T07:06:50.838 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-09-19T07:06:50.838 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.838 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.838 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.838 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-09-19T07:06:50.838 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.838 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.838 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.838 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T07:06:50.838 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.838 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.838 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.839 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.839 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-09-19T07:06:50.839 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-09-19T07:06:50.839 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.839 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.839 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.839 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-09-19T07:06:50.839 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.839 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.839 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.839 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-09-19T07:06:50.839 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.839 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.839 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.840 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.840 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-09-19T07:06:50.840 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-09-19T07:06:50.840 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.840 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.840 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.840 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.840 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.840 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.840 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.840 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T07:06:50.840 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.840 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.841 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.841 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.841 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-09-19T07:06:50.841 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-09-19T07:06:50.841 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.841 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.841 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.841 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:06:50.841 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.841 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.841 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.841 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-09-19T07:06:50.841 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.841 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.841 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.842 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.842 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-09-19T07:06:50.842 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-09-19T07:06:50.842 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.842 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.842 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.842 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T07:06:50.842 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.842 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.842 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.842 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T07:06:50.842 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.842 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.843 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.843 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.843 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-09-19T07:06:50.843 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-09-19T07:06:50.843 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.843 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.843 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.843 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:06:50.843 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.843 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.843 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.843 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-09-19T07:06:50.843 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.844 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.844 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.844 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.844 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-09-19T07:06:50.844 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-09-19T07:06:50.844 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.844 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.844 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.844 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.844 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.844 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.844 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.844 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T07:06:50.844 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.845 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.845 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.845 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.845 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-09-19T07:06:50.845 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-09-19T07:06:50.845 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.845 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.845 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.845 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.845 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.845 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.845 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.845 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T07:06:50.845 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.846 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.846 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.846 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.846 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.846 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.846 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.846 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.846 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.846 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.846 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.846 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.846 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.846 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.846 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.847 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.847 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.847 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.847 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.847 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.847 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.847 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.847 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.847 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.847 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.847 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-09-19T07:06:50.847 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-09-19T07:06:50.847 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.848 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.848 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.848 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.848 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.848 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.848 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.848 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T07:06:50.848 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.848 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.848 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.848 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.848 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.848 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.849 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.849 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.849 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.849 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.849 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.849 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.849 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.849 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T07:06:50.849 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.849 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.849 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.849 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.849 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.849 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.850 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.850 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.850 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.850 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.850 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.850 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.850 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.850 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.850 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.850 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.850 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.850 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.850 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.851 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.851 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.851 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.851 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.851 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.851 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.851 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.851 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.851 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.851 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.851 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.851 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.851 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.851 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.852 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.852 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.852 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.852 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.852 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.852 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.852 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T07:06:50.852 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T07:06:50.852 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.852 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.852 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.852 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.852 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.853 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.853 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.853 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T07:06:50.853 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.853 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.853 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.853 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T07:06:50.853 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T07:06:50.853 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.853 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.853 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.853 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-09-19T07:06:50.853 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.854 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.854 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.854 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T07:06:50.854 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.854 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.854 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.854 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.854 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-09-19T07:06:50.854 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-09-19T07:06:50.854 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.855 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.855 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.855 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:50.855 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.855 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.855 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.855 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-09-19T07:06:50.855 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.855 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.855 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.855 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-09-19T07:06:50.855 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-09-19T07:06:50.855 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.856 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.856 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.856 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:50.856 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.856 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.856 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.856 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T07:06:50.856 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.856 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.856 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.856 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.856 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-09-19T07:06:50.856 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-09-19T07:06:50.856 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.857 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.857 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.857 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.857 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.857 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.857 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.857 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-09-19T07:06:50.857 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.857 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.857 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.857 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.857 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-19T07:06:50.857 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-19T07:06:50.857 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.858 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.858 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.858 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-09-19T07:06:50.858 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.858 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.858 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.858 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-09-19T07:06:50.858 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-09-19T07:06:50.858 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.858 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T07:06:50.858 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.858 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.858 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.858 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.859 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-09-19T07:06:50.859 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-09-19T07:06:50.859 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.859 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.859 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.859 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-09-19T07:06:50.859 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.859 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.859 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.859 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T07:06:50.859 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.859 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.859 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.859 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.860 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-09-19T07:06:50.860 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-09-19T07:06:50.860 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.860 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.860 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.860 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T07:06:50.860 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.860 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.860 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.860 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T07:06:50.860 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.861 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.861 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.861 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.861 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-09-19T07:06:50.861 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-09-19T07:06:50.861 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.861 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.861 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.861 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.861 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.861 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.861 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.861 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T07:06:50.862 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.862 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.862 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.862 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.862 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-09-19T07:06:50.862 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-09-19T07:06:50.862 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.862 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.862 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.862 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.862 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.862 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.862 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.863 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T07:06:50.863 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.863 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.863 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.863 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.863 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-09-19T07:06:50.863 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-09-19T07:06:50.863 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.863 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.863 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.863 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.863 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.863 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.863 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.864 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T07:06:50.864 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.864 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.864 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.864 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.864 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-09-19T07:06:50.864 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-09-19T07:06:50.864 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.864 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.864 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.864 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.864 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.864 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.865 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.865 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T07:06:50.865 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.865 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.865 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.865 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.865 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-09-19T07:06:50.865 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-09-19T07:06:50.865 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.865 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.865 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.865 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.865 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.865 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.866 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.866 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T07:06:50.866 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.866 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.866 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.866 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.866 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-09-19T07:06:50.866 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-09-19T07:06:50.866 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.866 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.867 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.867 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-09-19T07:06:50.867 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.867 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.867 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.867 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-09-19T07:06:50.867 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.867 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.867 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.867 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.867 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-09-19T07:06:50.867 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-09-19T07:06:50.867 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.868 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.868 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.868 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.868 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.868 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.868 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.868 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T07:06:50.868 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.868 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.868 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.868 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-09-19T07:06:50.868 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-09-19T07:06:50.869 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.869 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.869 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.869 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.869 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.869 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.869 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.869 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T07:06:50.869 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.869 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.869 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.869 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.869 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-09-19T07:06:50.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-09-19T07:06:50.870 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.870 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.870 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.870 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.870 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.870 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.870 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.870 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T07:06:50.870 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.870 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.870 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.870 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.870 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-09-19T07:06:50.870 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-09-19T07:06:50.871 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.871 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.871 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.871 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.871 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.871 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.871 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.871 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T07:06:50.871 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.871 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.871 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.871 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.871 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-09-19T07:06:50.871 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-09-19T07:06:50.872 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.872 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.872 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.872 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.872 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.872 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.872 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.872 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T07:06:50.872 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.872 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.872 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.872 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.872 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-09-19T07:06:50.872 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-09-19T07:06:50.873 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.873 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.873 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.873 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.873 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.873 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.873 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.873 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T07:06:50.873 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.873 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.873 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.873 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.874 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.874 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.874 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.874 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-09-19T07:06:50.874 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.874 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.874 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.874 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.874 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.874 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.874 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.874 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.874 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.874 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.875 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.875 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.875 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.875 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.875 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.875 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.875 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.875 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.875 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.875 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.875 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.875 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.875 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.875 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.876 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.876 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.876 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.876 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.876 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.876 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.876 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.876 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.876 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.876 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.876 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.876 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.876 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.877 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.877 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.877 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.877 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.877 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.877 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.877 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.877 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.877 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.877 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.877 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.877 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.877 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.877 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.878 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.878 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.878 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.878 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.878 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.878 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.878 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.878 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.878 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.878 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.878 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.878 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.878 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.879 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.879 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.879 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.879 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.879 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.879 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.879 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.879 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-09-19T07:06:50.879 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-09-19T07:06:50.879 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.879 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.879 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.880 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-09-19T07:06:50.880 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.880 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.880 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.880 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T07:06:50.880 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.880 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.880 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.880 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.880 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-09-19T07:06:50.880 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-09-19T07:06:50.880 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.880 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.880 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.881 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-09-19T07:06:50.881 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.881 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.881 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.881 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T07:06:50.881 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.881 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.881 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.881 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.881 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.881 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.881 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.881 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-09-19T07:06:50.881 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.882 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.882 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.882 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T07:06:50.882 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T07:06:50.882 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.882 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.882 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.882 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:50.882 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.882 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.882 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.882 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.882 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.882 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.883 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.883 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.883 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T07:06:50.883 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T07:06:50.883 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.883 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.883 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.883 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.883 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.883 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.883 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.883 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.883 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.883 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.884 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.884 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.884 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T07:06:50.884 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T07:06:50.884 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.884 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.884 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.884 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.884 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.884 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.884 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.884 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.884 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.885 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.885 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.885 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.885 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-09-19T07:06:50.885 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T07:06:50.885 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.885 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.885 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.885 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.885 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.885 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.885 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.885 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.885 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.886 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.886 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.886 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T07:06:50.886 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T07:06:50.886 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.886 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.886 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.886 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.886 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.886 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.886 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.886 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.886 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.887 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.887 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.887 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T07:06:50.887 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T07:06:50.887 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.887 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.887 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.887 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.887 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.887 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.887 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.887 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.888 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.888 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.888 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.888 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.888 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T07:06:50.888 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T07:06:50.888 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.888 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.888 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.888 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.888 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.888 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.888 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.889 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.889 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.889 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.889 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.889 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T07:06:50.889 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T07:06:50.889 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.889 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.889 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.889 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.889 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.889 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.889 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.890 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.890 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.890 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.890 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.890 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T07:06:50.890 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T07:06:50.890 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.890 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.890 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.890 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.890 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.890 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.890 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.891 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.891 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.891 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.891 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.891 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.891 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T07:06:50.891 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T07:06:50.891 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.891 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.891 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.891 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.891 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.891 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.892 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.892 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.892 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.892 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.892 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.892 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.892 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T07:06:50.892 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T07:06:50.892 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.892 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.892 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.892 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.892 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.893 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.893 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.893 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.893 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.893 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.893 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.893 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.893 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T07:06:50.893 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T07:06:50.893 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.893 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.893 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.893 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.893 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.894 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.894 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.894 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.894 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.894 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.894 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.894 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.894 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T07:06:50.894 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T07:06:50.894 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.894 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.894 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.894 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.895 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.895 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.895 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.895 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.895 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.895 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.895 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.895 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T07:06:50.895 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T07:06:50.895 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.895 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.895 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.895 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.896 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.896 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.896 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.896 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.896 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.896 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.896 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.896 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.896 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T07:06:50.896 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T07:06:50.896 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.896 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.897 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.897 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:06:50.897 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.897 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.897 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.897 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.897 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.897 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.897 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.897 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.897 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T07:06:50.897 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T07:06:50.897 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.897 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.898 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.898 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.898 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.898 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.898 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.898 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.898 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.898 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.898 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.898 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.898 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-09-19T07:06:50.898 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-09-19T07:06:50.898 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.899 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.899 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.899 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.899 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.899 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.899 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.899 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.899 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.899 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.899 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.899 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.899 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T07:06:50.899 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T07:06:50.899 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.900 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.900 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.900 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:06:50.900 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.900 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.900 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.900 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.900 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.900 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.900 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.900 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.900 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T07:06:50.900 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T07:06:50.900 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.901 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.901 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.901 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.901 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.901 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.901 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.901 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.901 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.901 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.901 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.901 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.901 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T07:06:50.901 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T07:06:50.902 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.902 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.902 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.902 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.902 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.902 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.902 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.902 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.902 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.902 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.902 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.902 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.902 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T07:06:50.903 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T07:06:50.903 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.903 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.903 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.903 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.903 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.903 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.903 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.903 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.903 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.903 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.903 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.903 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.903 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T07:06:50.904 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T07:06:50.904 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.904 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.904 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.904 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.904 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.904 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.904 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.904 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.904 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.904 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.904 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.904 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.905 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T07:06:50.905 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T07:06:50.905 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.905 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.905 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.905 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.905 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.905 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.905 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.905 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.905 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.905 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.905 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.906 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.906 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T07:06:50.906 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T07:06:50.906 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.906 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.906 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.906 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.906 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.906 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.906 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.906 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.906 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.906 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.907 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.907 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.907 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T07:06:50.907 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T07:06:50.907 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.907 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.907 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.907 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.907 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.907 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.907 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.907 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.907 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.907 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.908 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.908 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.908 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T07:06:50.908 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T07:06:50.908 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.908 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.908 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.908 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.908 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.908 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.908 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.908 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.908 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.908 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.909 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.909 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.909 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T07:06:50.909 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T07:06:50.909 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.909 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.909 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.909 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.909 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.909 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.909 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.909 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.909 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.909 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.910 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.910 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.910 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T07:06:50.910 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T07:06:50.910 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.910 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.910 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.910 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.910 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.910 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.910 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.910 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.910 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.910 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.911 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.911 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.911 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T07:06:50.911 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T07:06:50.911 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.911 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.911 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.911 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.911 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.911 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.911 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.911 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.911 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.911 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.912 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.912 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.912 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T07:06:50.912 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T07:06:50.912 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.912 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.912 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.912 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.912 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.912 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.912 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.912 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.912 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.912 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.913 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.913 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.913 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T07:06:50.913 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T07:06:50.913 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.913 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.913 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.913 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.913 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.913 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.913 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.913 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.913 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.913 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.914 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.914 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.914 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T07:06:50.914 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T07:06:50.914 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.914 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.914 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.914 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.914 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.914 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.914 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.914 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.914 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.915 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.915 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.915 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.915 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T07:06:50.915 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T07:06:50.915 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.915 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.915 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.915 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T07:06:50.915 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.915 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.915 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.915 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.916 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.916 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.916 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.916 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.916 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T07:06:50.916 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T07:06:50.916 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.916 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.916 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.916 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:50.916 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.916 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.917 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.917 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.917 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.917 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.917 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.917 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.917 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T07:06:50.917 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T07:06:50.917 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.917 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.917 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.917 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-09-19T07:06:50.917 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.917 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.918 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.918 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.918 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.918 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.918 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.918 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.918 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T07:06:50.918 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T07:06:50.918 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.918 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.918 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.918 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-09-19T07:06:50.918 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.919 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.919 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.919 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.919 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.919 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.919 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.919 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.919 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T07:06:50.919 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T07:06:50.919 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.919 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.919 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.919 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.919 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.920 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.920 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.920 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.920 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.920 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.920 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.920 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.920 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T07:06:50.920 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T07:06:50.920 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.920 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.920 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.920 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:06:50.921 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.921 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.921 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.921 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.921 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.921 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.921 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.921 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.921 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T07:06:50.921 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T07:06:50.921 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.921 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.921 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.921 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.922 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.922 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.922 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.922 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.922 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.922 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.922 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.922 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.922 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T07:06:50.922 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T07:06:50.922 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.922 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.922 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.922 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.923 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.923 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.923 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.923 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.923 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.923 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.923 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.923 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.923 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T07:06:50.923 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T07:06:50.923 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.923 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.923 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.923 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:06:50.924 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.924 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.924 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.924 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.924 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.924 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.924 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.924 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.924 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T07:06:50.924 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T07:06:50.924 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.924 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.924 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.924 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:50.925 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.925 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.925 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.925 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.925 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.925 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.925 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.925 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.925 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T07:06:50.925 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T07:06:50.925 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.925 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.925 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.925 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:06:50.926 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.926 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.926 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.926 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.926 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.926 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.926 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.926 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.926 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-09-19T07:06:50.926 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-09-19T07:06:50.927 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.927 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.927 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.927 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.927 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.927 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.927 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.927 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.927 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.927 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.927 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.927 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.927 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-09-19T07:06:50.927 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-09-19T07:06:50.928 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.928 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.928 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.928 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.928 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.928 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.928 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.928 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.928 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.928 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.928 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.928 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.929 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-09-19T07:06:50.929 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-09-19T07:06:50.929 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.929 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.929 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.929 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.929 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.929 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.929 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.929 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-09-19T07:06:50.929 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.929 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.929 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.929 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.930 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-09-19T07:06:50.930 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-09-19T07:06:50.930 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.930 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.930 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.930 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-09-19T07:06:50.930 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.930 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.930 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.930 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.930 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.930 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.930 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.930 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.931 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-19T07:06:50.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-19T07:06:50.931 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.931 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.931 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.931 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.931 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.931 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.931 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.931 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.931 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.931 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.931 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.932 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.932 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.932 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.932 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.932 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.932 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.932 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.932 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.932 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.932 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.932 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.932 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.932 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.932 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.933 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.933 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.933 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.933 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.933 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.933 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.933 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.933 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.933 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.933 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.933 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.933 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.933 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.933 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.934 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.934 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.934 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.934 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.934 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.934 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.934 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.934 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.934 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.934 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.934 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.934 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.934 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.934 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.935 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.935 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.935 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.935 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.935 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.935 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.935 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.935 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.935 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.935 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.935 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.935 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.935 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.935 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.936 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.936 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.936 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.936 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.936 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.936 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.936 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.936 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.936 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.936 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.936 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.936 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.936 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.936 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.937 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.937 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-09-19T07:06:50.937 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-09-19T07:06:50.937 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.937 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.937 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.937 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.937 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.937 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.937 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.937 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-09-19T07:06:50.937 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.937 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.938 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.938 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.938 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-09-19T07:06:50.938 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T07:06:50.938 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.938 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.938 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.938 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.938 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.938 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.938 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.938 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.938 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.938 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.939 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.939 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.939 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:06:50.939 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:06:50.939 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.939 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.939 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.939 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-19T07:06:50.939 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.939 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.939 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.939 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.939 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.939 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.939 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.940 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.940 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:06:50.940 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:06:50.940 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.940 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.940 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.940 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-09-19T07:06:50.940 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.940 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.940 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.940 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.940 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.940 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.941 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.941 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.941 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-19T07:06:50.941 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-19T07:06:50.941 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.941 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.941 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.941 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.941 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.941 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.941 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.941 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.941 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.941 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.942 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.942 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.942 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-09-19T07:06:50.942 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-09-19T07:06:50.942 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.942 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.942 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.942 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-09-19T07:06:50.942 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.942 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.942 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.942 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.943 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.943 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.943 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.943 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.943 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-19T07:06:50.943 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-19T07:06:50.943 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.943 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.943 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.943 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-09-19T07:06:50.943 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.943 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.943 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.944 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.944 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-09-19T07:06:50.944 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.944 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.944 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.944 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.944 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.944 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.944 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-19T07:06:50.944 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-19T07:06:50.944 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.944 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.944 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.945 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-19T07:06:50.945 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-19T07:06:50.945 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-19T07:06:50.945 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.945 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.945 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.945 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.945 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.945 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.945 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-09-19T07:06:50.945 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-09-19T07:06:50.945 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.945 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.945 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.946 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.946 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.946 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.946 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.946 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.946 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.946 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.946 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.946 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.946 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.946 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.946 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-19T07:06:50.947 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.947 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.947 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.947 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-09-19T07:06:50.947 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-09-19T07:06:50.947 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.947 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.947 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.947 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.947 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.947 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.947 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.947 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-09-19T07:06:50.948 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.948 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.948 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.948 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.948 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.948 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.948 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.948 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.948 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.948 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.948 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.948 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.949 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.949 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.949 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.949 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.949 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.949 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.949 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.949 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.949 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.949 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.949 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.949 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.949 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.950 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.950 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.950 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.950 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.950 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.950 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.950 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.950 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.950 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.950 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.950 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.950 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.950 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.950 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.951 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.951 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.951 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.951 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.951 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.951 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.951 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.951 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.951 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.951 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.951 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.951 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.951 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.951 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.952 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.952 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.952 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.952 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.952 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.952 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.952 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.952 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.952 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.952 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.952 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.952 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.952 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.952 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.953 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.953 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.953 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.953 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.953 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.953 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.953 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-09-19T07:06:50.953 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-09-19T07:06:50.953 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.953 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.953 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.953 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-09-19T07:06:50.953 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.954 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.954 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.954 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T07:06:50.954 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.954 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.954 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.954 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.954 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-09-19T07:06:50.954 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-09-19T07:06:50.954 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.954 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.954 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.954 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-09-19T07:06:50.954 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.955 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.955 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.955 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T07:06:50.955 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.955 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.955 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.955 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.955 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-09-19T07:06:50.955 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-09-19T07:06:50.955 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.955 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.955 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.955 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-09-19T07:06:50.955 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.956 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.956 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.956 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-09-19T07:06:50.956 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.956 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.956 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.956 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.956 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-09-19T07:06:50.956 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-09-19T07:06:50.956 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.956 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.956 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.957 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-19T07:06:50.957 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.957 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.957 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.957 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T07:06:50.957 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.957 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.957 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.957 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.957 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-09-19T07:06:50.957 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-09-19T07:06:50.957 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.957 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.957 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.958 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:50.958 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.958 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.958 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.958 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T07:06:50.958 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.958 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.958 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.958 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.958 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:06:50.958 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:06:50.958 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.958 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.959 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.959 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:06:50.959 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.959 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.959 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.959 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T07:06:50.959 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.959 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.959 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.959 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.959 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-09-19T07:06:50.959 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-09-19T07:06:50.959 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.960 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.960 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.960 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-09-19T07:06:50.960 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.960 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.960 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.960 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T07:06:50.960 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.960 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.960 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.960 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.960 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.960 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.961 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.961 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-09-19T07:06:50.961 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.961 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.961 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.961 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.961 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.961 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.961 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.961 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.961 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.961 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.961 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.962 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.962 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.962 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.962 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.962 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.962 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.962 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.962 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.962 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.962 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.962 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.962 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.962 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.963 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.963 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.963 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.963 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.963 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.963 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.963 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.963 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.963 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.963 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.963 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.963 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.963 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.964 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.964 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.964 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.964 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.964 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.964 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.964 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.964 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.964 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.964 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.964 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.964 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.965 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.965 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.965 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.965 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.965 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.965 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.965 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.965 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.965 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.965 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.965 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.965 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.965 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.965 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.966 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.966 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.966 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.966 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.966 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.966 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.966 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.966 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.966 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.966 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-09-19T07:06:50.966 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-09-19T07:06:50.966 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.966 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.966 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.967 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-19T07:06:50.967 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.967 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.967 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.967 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.967 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.967 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.967 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.967 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.967 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-09-19T07:06:50.967 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-09-19T07:06:50.967 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.967 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.967 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.968 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-09-19T07:06:50.968 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.968 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.968 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.968 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.968 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.968 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.968 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.968 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.968 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:06:50.968 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:06:50.968 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.968 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.968 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.969 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:06:50.969 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.969 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.969 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.969 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.969 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.969 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.969 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.969 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.969 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.969 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.969 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.969 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-09-19T07:06:50.969 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-09-19T07:06:50.970 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-09-19T07:06:50.970 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.970 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-09-19T07:06:50.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-09-19T07:06:50.970 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.970 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.970 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.970 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-09-19T07:06:50.970 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.970 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.970 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.970 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T07:06:50.970 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.971 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.971 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.971 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.971 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-09-19T07:06:50.971 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-09-19T07:06:50.971 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.971 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.971 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.971 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-09-19T07:06:50.971 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.971 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.971 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.971 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T07:06:50.971 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.972 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.972 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.972 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-09-19T07:06:50.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-09-19T07:06:50.972 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.972 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.972 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.972 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.972 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.972 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.972 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.973 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-09-19T07:06:50.973 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.973 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.973 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.973 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.973 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:06:50.973 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:06:50.973 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:50.973 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.973 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.973 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-09-19T07:06:50.973 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-09-19T07:06:50.973 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.974 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.974 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T07:06:50.974 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.974 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.974 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.974 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.974 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.974 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.974 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.974 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.974 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.974 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.975 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.975 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.975 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.975 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.975 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.975 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.975 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.975 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.975 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.975 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.975 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.975 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.975 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.975 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.976 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.976 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.976 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.976 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.976 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.976 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.976 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.976 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.976 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.976 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.976 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.976 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.977 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.977 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.977 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.977 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.977 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.977 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.977 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.977 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.977 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.977 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.977 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.977 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.978 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.978 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.978 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.978 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.978 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.978 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.978 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.978 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.978 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.978 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.978 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.978 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.979 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.979 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.979 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.979 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.979 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.979 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.979 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.979 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.979 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.979 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.979 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-09-19T07:06:50.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-09-19T07:06:50.979 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.979 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.980 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.980 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.980 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.980 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.980 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.980 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-09-19T07:06:50.980 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.980 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.980 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.980 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.980 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-09-19T07:06:50.980 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-09-19T07:06:50.980 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.981 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.981 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.981 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-09-19T07:06:50.981 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.981 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.981 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.981 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-09-19T07:06:50.981 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.981 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.981 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.981 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-19T07:06:50.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-19T07:06:50.981 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-09-19T07:06:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:06:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-09-19T07:06:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-09-19T07:06:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:06:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:06:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-09-19T07:06:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T07:06:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-09-19T07:06:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-09-19T07:06:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-09-19T07:06:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-09-19T07:06:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-09-19T07:06:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-09-19T07:06:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:06:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-09-19T07:06:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-09-19T07:06:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-09-19T07:06:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-09-19T07:06:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-09-19T07:06:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-09-19T07:06:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T07:06:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-09-19T07:06:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-09-19T07:06:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-09-19T07:06:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T07:06:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-09-19T07:06:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-09-19T07:06:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-09-19T07:06:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-09-19T07:06:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.006 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.006 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.006 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-09-19T07:06:51.006 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T07:06:51.006 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.006 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.006 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.006 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.006 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.006 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.006 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.006 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.007 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.007 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.007 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.007 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.007 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.007 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.007 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.007 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.007 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.007 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.007 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.007 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.007 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-09-19T07:06:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-09-19T07:06:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-09-19T07:06:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T07:06:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-09-19T07:06:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-09-19T07:06:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-09-19T07:06:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-09-19T07:06:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-09-19T07:06:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-09-19T07:06:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-09-19T07:06:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T07:06:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-09-19T07:06:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-09-19T07:06:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-09-19T07:06:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-09-19T07:06:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-09-19T07:06:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-09-19T07:06:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T07:06:51.021 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T07:06:51.021 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.021 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.021 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.021 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.021 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.021 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-09-19T07:06:51.021 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.021 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.021 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.021 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.021 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.022 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.022 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.022 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.022 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.022 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.022 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.022 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.022 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.022 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.022 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.022 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.022 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.022 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.022 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.023 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.023 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.023 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.023 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.023 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.023 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.023 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.023 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.023 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.023 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.023 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.023 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.024 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.024 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.024 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.024 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.024 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.024 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.024 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.024 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.024 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.024 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.024 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.025 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.025 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.025 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.025 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.025 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.025 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.025 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.025 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.025 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.025 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.025 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.025 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.026 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.026 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.026 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.026 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.026 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.026 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.026 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.026 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.026 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.026 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.026 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.027 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.027 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.027 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.027 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.027 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-09-19T07:06:51.027 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.027 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.027 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.027 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.028 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.028 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.028 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.028 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.028 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.028 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.028 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.028 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.028 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.028 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.028 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.028 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.028 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.028 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.029 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.029 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.029 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.029 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.029 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.029 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.029 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.029 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.029 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.029 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.029 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.029 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.030 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.030 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.030 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.030 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.030 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.030 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.030 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.030 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.030 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.030 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.030 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.031 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.031 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.031 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.031 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.031 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.031 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.031 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.031 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.031 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.031 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.031 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.031 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.031 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.032 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.032 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.032 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.032 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.032 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.032 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.032 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.032 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.032 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.032 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.032 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.032 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.033 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.033 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.033 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-09-19T07:06:51.033 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.033 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.033 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.033 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.033 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.033 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.033 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.034 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.034 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.034 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.034 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.034 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.034 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.034 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.034 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.034 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.034 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.034 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.034 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.034 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.035 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.035 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.035 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.035 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.035 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.035 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.035 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.035 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.035 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.035 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.035 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.035 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.035 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.036 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.036 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.036 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.036 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.036 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.036 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.036 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.036 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.036 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.036 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.036 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.036 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.036 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.037 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.037 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.037 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.037 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.037 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.037 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.037 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.037 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.037 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.037 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.037 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.037 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.037 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.038 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.038 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.038 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.038 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.038 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.038 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.038 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.038 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.038 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.038 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.038 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-19T07:06:51.039 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.039 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.039 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.039 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-09-19T07:06:51.039 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-09-19T07:06:51.039 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.039 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.039 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.039 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.039 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.039 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.039 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.039 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T07:06:51.039 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.039 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.040 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.040 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.040 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.040 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.040 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.040 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.040 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.040 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.040 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.040 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.040 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.040 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.041 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.041 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.041 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.041 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.041 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.041 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.041 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.041 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.041 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.041 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.041 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.041 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.041 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.041 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.042 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.042 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.042 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.042 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.042 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.042 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.042 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.042 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.042 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.042 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.042 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.042 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.042 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.043 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.043 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.043 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.043 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.043 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.043 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.043 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.043 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.043 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.043 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.043 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.043 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.043 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.043 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.044 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.044 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.044 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.044 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.044 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.044 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.044 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.044 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.044 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.044 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.044 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.044 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.045 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-09-19T07:06:51.045 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-19T07:06:51.045 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.045 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.045 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.045 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.045 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.045 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.045 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.045 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-19T07:06:51.045 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-09-19T07:06:51.045 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-09-19T07:06:51.045 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.045 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-09-19T07:06:51.046 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.046 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.046 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.046 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.046 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.046 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.046 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-09-19T07:06:51.046 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.046 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.046 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.046 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.046 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.046 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.048 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.048 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.048 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.048 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.048 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.048 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.048 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.048 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.048 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.048 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.048 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.049 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.049 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.049 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.049 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.049 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.049 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.049 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.049 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.049 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.049 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.049 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.049 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.050 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.050 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.050 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.050 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.050 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.050 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.050 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.050 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.050 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.050 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.050 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.050 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.050 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.051 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.051 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.051 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.051 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.051 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.051 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.051 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.051 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.051 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.051 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.051 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.051 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.051 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-09-19T07:06:51.052 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.052 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.052 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.052 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.052 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.052 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.052 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.052 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.052 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.052 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.052 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.052 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.052 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.052 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.052 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.053 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.053 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.053 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.053 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.053 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.053 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.053 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.053 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.053 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.053 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.053 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.053 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.053 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.054 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.054 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.054 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.054 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.054 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.054 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.054 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.054 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.054 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.054 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.054 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.054 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.054 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.055 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.055 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.055 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.055 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.055 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.055 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.055 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.055 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.055 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.055 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.055 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.055 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.055 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.055 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.056 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.056 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.056 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.056 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.056 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.056 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.056 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.056 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.056 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.056 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.056 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.056 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.056 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.056 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.057 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.057 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.057 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.057 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.057 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-09-19T07:06:51.057 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.057 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.057 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.057 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.057 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.057 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.058 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.058 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.058 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.058 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.058 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.058 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.058 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.058 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.058 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.058 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.058 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.058 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.058 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.058 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.058 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.059 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.059 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.059 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.059 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.059 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.059 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.059 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.059 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.059 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.059 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.059 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.059 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.059 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.060 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.060 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.060 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.060 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.060 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.060 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.060 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.060 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.060 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.060 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.060 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.060 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.060 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.061 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.061 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.061 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.061 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.061 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.061 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.061 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.061 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.061 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.061 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.061 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.061 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.061 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.062 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.062 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.062 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.062 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.062 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.062 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.062 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.062 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.062 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.062 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:06:51.062 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:06:51.062 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.062 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.062 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.063 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:06:51.063 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.063 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.063 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.063 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.063 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.063 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.063 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.063 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-09-19T07:06:51.063 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-09-19T07:06:51.063 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:51.063 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.063 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.064 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-19T07:06:51.064 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-09-19T07:06:51.064 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.064 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.064 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-09-19T07:06:51.064 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T07:06:51.064 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.064 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.064 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.064 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.064 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.064 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-09-19T07:06:51.064 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-09-19T07:06:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-09-19T07:06:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T07:06:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.066 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-19T07:06:51.066 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-19T07:06:51.066 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.066 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.066 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.066 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.066 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.066 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.066 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.066 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.066 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.066 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.067 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.067 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.067 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.067 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.067 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.067 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.067 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.067 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.067 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.067 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.067 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.067 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.067 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.067 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.068 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.068 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.068 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.068 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.068 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.068 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.068 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.068 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.068 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.068 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.068 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.068 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.068 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.069 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.069 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.069 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.069 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.069 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.069 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.069 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.069 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.069 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.069 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.070 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.070 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.070 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.070 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.070 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.070 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.070 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.070 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.070 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.070 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.070 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.070 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.070 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.070 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.070 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.071 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.071 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.071 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.071 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.071 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.071 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.071 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.071 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.071 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.071 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.071 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.071 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.071 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.071 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.072 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.072 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-09-19T07:06:51.072 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-09-19T07:06:51.072 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.072 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.072 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.072 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-09-19T07:06:51.072 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.072 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.072 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.072 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-09-19T07:06:51.072 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.073 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.073 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.073 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:06:51.073 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:06:51.073 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.073 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.073 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.073 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:06:51.073 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.073 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.073 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.073 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-09-19T07:06:51.073 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.074 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.074 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.074 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.074 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.074 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-09-19T07:06:51.074 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.074 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.074 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.074 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-09-19T07:06:51.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-09-19T07:06:51.074 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.074 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.075 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.075 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.075 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.075 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.075 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.075 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.075 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.075 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.075 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.075 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-09-19T07:06:51.075 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-09-19T07:06:51.075 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.076 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.076 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.076 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.076 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.076 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.076 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.076 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T07:06:51.076 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T07:06:51.076 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.076 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.076 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.076 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.076 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.077 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.077 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.077 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.077 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.077 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.077 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.077 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.077 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.077 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.077 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.077 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.077 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.077 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.077 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.078 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.078 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.078 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.078 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.078 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.078 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.078 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.078 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.078 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.078 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.078 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.078 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.079 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.079 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.079 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.079 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.079 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.079 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.079 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.079 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.079 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.079 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.079 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.079 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.079 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.080 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.080 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.080 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.080 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.080 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.080 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.080 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.080 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.080 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.080 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.080 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.080 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.080 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.081 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.081 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.081 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.081 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.081 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.081 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.081 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.081 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.081 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.081 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.081 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.081 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-09-19T07:06:51.081 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-09-19T07:06:51.082 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.082 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.082 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.082 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.082 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.082 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.082 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.082 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.082 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.082 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.082 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.082 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T07:06:51.083 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T07:06:51.083 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.083 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.083 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.083 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-09-19T07:06:51.083 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.083 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.083 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.083 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.083 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.083 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.083 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.083 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-09-19T07:06:51.084 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-09-19T07:06:51.084 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:51.084 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.084 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.084 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-09-19T07:06:51.084 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.084 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.084 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.084 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.084 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.084 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.084 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.084 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:06:51.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:06:51.085 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.085 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.085 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.085 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-19T07:06:51.085 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.085 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.085 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.085 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T07:06:51.085 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.085 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.085 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.085 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:06:51.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:06:51.086 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.086 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.086 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.086 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-09-19T07:06:51.086 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.086 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.086 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.086 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T07:06:51.086 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.086 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.086 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.086 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-09-19T07:06:51.087 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-09-19T07:06:51.087 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.087 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.087 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.087 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-09-19T07:06:51.087 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.087 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.087 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.087 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.087 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.088 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.088 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.088 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-19T07:06:51.088 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-19T07:06:51.088 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.088 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.088 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.088 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-09-19T07:06:51.088 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.088 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.088 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.088 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-09-19T07:06:51.088 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-09-19T07:06:51.088 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.089 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.089 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.089 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.089 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.089 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-19T07:06:51.089 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-19T07:06:51.089 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.089 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.089 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.089 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-19T07:06:51.089 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-19T07:06:51.089 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-19T07:06:51.089 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.090 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.090 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.090 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.090 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.090 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.090 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.090 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.090 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-19T07:06:51.090 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.090 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.090 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.090 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.090 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.091 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.091 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.091 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.091 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.091 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.091 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.091 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.091 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.091 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.091 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.091 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.091 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.091 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.091 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.092 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.092 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.092 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.092 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.092 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.092 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.092 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.092 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.092 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.092 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.092 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.092 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.092 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.093 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.093 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.093 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.093 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.093 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.093 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.093 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.093 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.093 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.093 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.093 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.093 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.093 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.094 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.094 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.094 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.094 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.094 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.094 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.094 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.094 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.094 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.094 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.094 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.094 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.094 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-09-19T07:06:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-09-19T07:06:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-09-19T07:06:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T07:06:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.098 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.098 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.098 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.098 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-09-19T07:06:51.098 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-09-19T07:06:51.098 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.098 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.098 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.098 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.098 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.098 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.098 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.099 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.099 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.099 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.099 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.099 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.099 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.099 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.099 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-09-19T07:06:51.099 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.099 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.099 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.099 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-09-19T07:06:51.099 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-09-19T07:06:51.100 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.100 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.100 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.100 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.100 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.100 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.100 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.100 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.100 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.101 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.101 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.101 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-19T07:06:51.101 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-19T07:06:51.101 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.101 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.101 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.101 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.102 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.102 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.102 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.102 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.102 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.102 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.102 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.102 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.103 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.103 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.103 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.103 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.103 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.103 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.103 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.103 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.103 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.104 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.104 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.104 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.104 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.104 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.104 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.104 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.104 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.104 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.105 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.105 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.105 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.105 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.105 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.105 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.105 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.106 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.106 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.106 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.106 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.106 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.106 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.106 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.106 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.106 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.106 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.106 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.106 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.107 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.107 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.107 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.107 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.107 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.107 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.107 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.107 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.107 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.107 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.107 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.108 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.108 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.108 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.108 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.108 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.108 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.108 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.108 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.109 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.109 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.109 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.109 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.109 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.109 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.109 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.109 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.109 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:06:51.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:06:51.110 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.110 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.110 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.110 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.110 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.110 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.110 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.110 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.110 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.110 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.110 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.110 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:06:51.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:06:51.111 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.111 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.111 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.111 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.111 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.111 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.111 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.111 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.111 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.112 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.112 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.112 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.112 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.112 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.112 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-09-19T07:06:51.112 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.112 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.112 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.112 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.113 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.113 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.113 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.113 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.113 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.113 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.113 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.113 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.113 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.113 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.113 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.114 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.114 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.114 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.114 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.114 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.114 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.114 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.114 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.115 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.115 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.115 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.115 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.115 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.115 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.116 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.116 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.116 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.116 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.116 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.116 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.116 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.116 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.116 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.117 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.117 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.117 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.117 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.117 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.117 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.117 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.117 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.117 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.117 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.117 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.118 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.118 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.118 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.118 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.118 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.118 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.118 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.119 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.119 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.119 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.119 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.119 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.119 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.119 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.119 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.119 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.119 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.119 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.119 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.120 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.120 INFO:teuthology.orchestra.run.smithi103.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T07:06:51.120 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T07:06:51.120 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.120 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.120 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.120 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:06:51.120 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.120 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.120 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.121 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T07:06:51.121 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.121 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.121 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.121 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.121 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.121 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.121 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-09-19T07:06:51.121 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.121 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.122 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.122 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.122 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.122 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.122 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.122 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.122 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.122 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.122 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.122 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.122 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.122 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.122 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.123 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.123 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.123 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.123 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.123 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.123 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.123 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.123 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.123 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.123 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.123 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.124 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.124 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.124 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.124 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.124 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.124 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.124 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.124 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.124 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.124 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.125 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.125 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.125 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.125 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.125 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.125 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.125 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.125 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.125 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.126 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.126 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.126 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.126 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.126 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.126 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.126 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.126 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.126 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.128 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.128 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-09-19T07:06:51.128 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-09-19T07:06:51.128 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.128 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.128 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.128 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-09-19T07:06:51.128 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.129 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.129 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.129 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T07:06:51.129 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.129 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.129 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.129 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.129 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.130 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.130 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-09-19T07:06:51.130 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.130 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.130 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.130 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.130 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.130 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.130 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.130 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.131 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.131 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.131 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.131 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.131 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.131 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.131 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.131 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.131 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.131 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.131 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.131 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.132 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.132 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.132 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.132 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.132 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.132 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.132 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.132 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.132 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.132 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.133 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.133 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.133 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.133 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.133 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.133 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.133 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.133 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.133 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.133 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.133 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.133 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.134 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.134 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.134 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.134 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.134 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.134 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.134 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.134 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.134 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.134 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.134 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.134 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.135 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.135 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.135 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.135 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.135 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.135 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.135 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.135 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.135 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.135 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.136 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.136 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.136 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.136 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.136 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-09-19T07:06:51.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-09-19T07:06:51.136 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.136 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.136 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.136 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.137 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.137 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.137 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.137 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.137 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.137 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.137 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.138 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-09-19T07:06:51.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-09-19T07:06:51.138 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.138 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.138 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.138 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-09-19T07:06:51.139 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.139 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.139 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.139 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.139 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.139 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.139 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.139 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-09-19T07:06:51.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-09-19T07:06:51.140 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.140 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.140 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.140 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.140 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.140 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.140 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.140 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.140 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.140 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.140 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.141 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-09-19T07:06:51.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-09-19T07:06:51.141 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.141 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.141 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.141 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.141 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.141 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.141 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.141 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.141 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.142 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.142 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.142 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-09-19T07:06:51.142 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-09-19T07:06:51.142 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:51.142 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.142 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.142 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.142 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.142 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.142 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.143 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.143 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.143 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.143 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.143 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-09-19T07:06:51.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-09-19T07:06:51.143 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.143 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.143 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.143 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.143 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.143 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.144 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.144 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.144 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.144 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.144 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.144 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-09-19T07:06:51.144 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-09-19T07:06:51.144 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.145 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.145 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.145 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.145 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.145 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.145 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.145 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.145 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.145 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.145 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.146 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-09-19T07:06:51.146 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-09-19T07:06:51.146 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.146 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.146 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.146 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.146 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.146 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.146 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.146 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.147 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.147 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.147 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.147 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-09-19T07:06:51.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-09-19T07:06:51.147 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.147 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.147 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.147 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.148 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:06:51.148 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-09-19T07:06:51.148 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.148 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.148 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.148 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.148 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.148 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-09-19T07:06:51.148 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-09-19T07:06:51.148 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.148 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.148 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.149 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.149 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.149 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.149 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.149 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.149 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.149 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.149 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.149 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-09-19T07:06:51.149 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-09-19T07:06:51.149 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.149 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.149 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.150 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.150 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.150 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.150 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.150 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.150 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.150 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.150 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.150 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-09-19T07:06:51.150 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-09-19T07:06:51.150 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.150 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.150 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.151 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.151 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.151 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.151 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.151 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.151 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.151 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.151 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.151 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.151 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.151 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.151 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-19T07:06:51.152 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.152 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.152 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.152 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-09-19T07:06:51.152 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-09-19T07:06:51.152 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.152 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.152 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.152 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.152 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.152 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.152 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.153 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T07:06:51.153 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.153 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.153 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.153 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-09-19T07:06:51.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-09-19T07:06:51.153 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.153 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.153 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.154 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.154 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.154 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.154 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.154 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-09-19T07:06:51.154 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.154 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.154 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.154 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.154 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.154 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.155 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.155 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.155 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.155 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.155 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.155 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.155 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.155 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.155 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.155 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.155 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.155 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.155 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.156 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.156 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.156 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.156 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.156 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.156 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.156 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.156 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.156 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.156 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.156 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.156 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.157 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.157 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.157 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.157 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.157 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.157 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.157 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.157 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.157 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.157 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.157 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.157 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.157 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.157 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.158 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.158 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.158 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.158 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.158 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.158 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.158 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.158 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.158 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.158 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.158 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.158 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.159 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.159 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.159 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.159 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.159 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.159 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.159 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.159 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.159 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.159 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.159 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.159 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.159 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.159 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-09-19T07:06:51.160 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.160 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.160 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.160 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.160 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.160 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.160 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.160 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.160 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.160 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.160 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.160 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.160 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.161 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.161 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.161 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.161 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.161 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.161 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.161 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.161 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.161 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.161 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.161 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.162 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.162 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.162 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.162 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.162 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.162 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.162 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.162 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.162 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.162 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.162 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.162 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.162 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.163 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.163 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.163 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.163 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.163 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.163 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.163 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.163 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.163 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.163 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.163 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.163 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.163 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.163 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.164 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.164 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.164 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.164 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.164 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.164 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.164 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.164 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.164 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.164 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.164 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.164 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.165 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.165 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.165 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.165 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.165 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.165 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.165 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.165 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.165 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-09-19T07:06:51.165 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.165 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.165 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.165 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.166 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.166 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.166 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.166 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.166 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.166 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.166 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.166 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.166 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.166 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.166 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.166 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.166 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.166 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.167 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.167 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.167 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.167 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.167 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.167 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.167 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.167 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.167 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.167 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.167 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.167 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.168 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.168 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.168 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.168 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.168 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.168 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.168 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.168 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.168 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.168 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.168 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.168 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.168 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.169 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.169 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.169 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.169 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.169 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.169 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.169 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.169 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.169 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.169 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.169 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.169 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.169 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.170 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.170 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.170 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.170 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.170 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.170 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.170 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.170 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.170 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.170 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.170 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.170 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.170 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.171 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.171 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-09-19T07:06:51.171 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.171 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.171 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.171 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-09-19T07:06:51.171 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-09-19T07:06:51.171 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.171 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.171 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.171 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T07:06:51.171 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.171 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.171 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.172 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.172 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.172 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.172 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.172 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:06:51.172 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:06:51.172 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.172 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.172 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.172 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-09-19T07:06:51.172 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.172 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.172 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.173 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.173 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.173 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.173 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.173 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.173 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.173 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.173 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.173 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.173 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.173 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.174 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.174 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.174 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.174 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.174 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.174 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.174 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.174 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.174 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.174 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.174 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.174 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.174 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.175 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.175 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.175 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.175 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.175 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.175 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.175 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.175 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.175 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.175 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.175 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.175 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.176 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.176 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.176 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.176 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.176 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.176 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.176 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.176 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.176 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.176 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.176 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.176 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.176 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.176 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.177 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.177 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.177 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.177 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.177 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.177 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.177 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.177 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.177 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.177 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.177 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.177 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.177 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.177 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.178 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.178 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.178 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.178 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.178 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.178 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.178 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.178 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-09-19T07:06:51.178 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.178 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.178 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.178 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-09-19T07:06:51.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-09-19T07:06:51.179 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.179 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.179 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.179 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.179 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.179 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.179 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.179 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T07:06:51.179 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.179 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.179 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.179 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-09-19T07:06:51.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-09-19T07:06:51.180 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.180 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.180 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.180 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.180 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.180 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.180 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.180 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T07:06:51.180 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.180 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.180 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.180 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-09-19T07:06:51.181 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-09-19T07:06:51.181 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.181 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.181 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.181 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.181 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.181 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.181 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.181 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T07:06:51.181 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.181 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.181 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.182 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-09-19T07:06:51.182 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-09-19T07:06:51.182 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.182 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.182 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.182 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.182 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.182 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.182 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.182 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T07:06:51.182 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.182 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.182 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.183 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-09-19T07:06:51.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-09-19T07:06:51.183 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.183 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.183 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.183 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.183 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.183 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.183 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.183 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T07:06:51.183 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.183 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.184 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.184 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-09-19T07:06:51.184 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-09-19T07:06:51.184 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.184 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.184 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.184 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.184 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.184 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.184 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.184 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.184 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.185 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.185 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.185 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-09-19T07:06:51.185 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-09-19T07:06:51.185 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.185 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.185 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.185 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.185 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.185 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.185 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.185 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.185 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.186 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.186 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.186 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-09-19T07:06:51.186 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-09-19T07:06:51.186 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.186 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.186 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.186 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T07:06:51.186 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.186 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.186 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.187 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.187 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.187 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.187 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.187 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-19T07:06:51.187 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-19T07:06:51.187 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.187 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.187 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.187 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.187 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.187 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.187 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.188 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.188 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.188 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.188 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.188 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.188 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:06:51.188 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:06:51.188 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.188 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.188 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.188 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-09-19T07:06:51.188 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-09-19T07:06:51.188 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.189 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.189 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.189 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.189 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.189 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.189 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-09-19T07:06:51.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-09-19T07:06:51.189 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.189 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.189 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.189 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-09-19T07:06:51.189 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.190 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.190 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.190 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.190 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.190 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.190 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.190 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-09-19T07:06:51.190 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-09-19T07:06:51.190 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.190 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.190 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.190 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.191 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.191 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.191 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.191 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.191 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.191 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.191 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.191 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-09-19T07:06:51.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-09-19T07:06:51.191 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.191 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.191 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.191 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.192 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.192 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.192 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.192 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.192 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.192 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.192 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.192 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.192 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.192 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.192 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.192 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.192 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.193 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.193 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.193 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.193 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.193 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.193 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.193 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.193 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.193 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.193 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.193 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.193 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.193 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.193 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.194 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.194 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.194 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.194 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.194 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.194 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.194 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.194 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.194 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.194 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.194 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.194 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.195 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.195 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.195 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.195 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.195 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.195 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.195 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.195 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.195 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.195 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.195 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.195 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.195 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.196 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.196 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.196 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.196 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.196 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.196 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.196 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.196 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.196 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.196 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.196 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.196 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.196 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.197 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.197 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.197 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.197 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.197 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.197 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-09-19T07:06:51.197 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-09-19T07:06:51.197 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.197 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.197 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.197 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.197 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.198 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.198 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.198 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.198 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.198 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.198 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.198 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-09-19T07:06:51.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-09-19T07:06:51.198 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.198 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.198 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.198 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.199 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.199 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.199 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.199 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.199 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.199 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.199 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.199 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-09-19T07:06:51.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-09-19T07:06:51.199 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.199 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.199 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.199 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T07:06:51.200 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.200 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.200 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.200 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.200 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.200 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.200 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.200 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.200 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.200 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.200 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-09-19T07:06:51.200 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.201 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.201 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.201 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.201 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.201 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.201 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.201 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.201 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.201 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.201 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.201 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.201 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.201 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.201 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.201 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.202 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.202 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.202 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.202 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.202 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.202 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.202 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.202 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.202 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.202 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.202 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.202 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.203 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.203 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.203 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.203 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.203 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.203 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.203 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.203 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.203 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.203 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.203 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.203 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.203 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.204 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.204 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.204 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.204 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.204 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.204 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.204 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.204 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.204 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.204 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.204 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.204 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.204 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.204 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.205 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.205 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.205 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.205 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.205 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.205 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.205 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.205 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.205 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.205 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.205 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.205 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.205 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.205 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.206 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.206 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-19T07:06:51.206 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.206 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.206 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.206 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.206 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.206 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.206 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.206 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.206 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.207 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.207 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.207 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.207 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.207 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.207 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.207 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.207 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.207 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.207 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.207 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.207 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.207 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.208 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.208 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.208 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.208 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.208 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.208 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.208 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.208 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.208 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.208 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.209 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.209 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.212 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.212 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.212 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.212 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.212 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.213 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.213 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.213 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.213 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.213 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.213 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.213 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.213 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.213 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.213 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.213 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.213 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.213 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.213 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.214 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.214 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.214 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.214 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.214 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.214 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.214 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.214 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.214 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.214 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.214 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.215 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.215 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-09-19T07:06:51.215 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-09-19T07:06:51.215 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.215 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.215 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.215 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-09-19T07:06:51.215 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.215 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.215 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.215 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T07:06:51.216 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.216 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.216 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.216 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-09-19T07:06:51.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-09-19T07:06:51.216 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.216 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.216 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.216 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.216 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.216 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.216 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.217 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-09-19T07:06:51.217 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.217 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.217 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.217 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-09-19T07:06:51.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-09-19T07:06:51.217 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.217 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.217 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.217 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:06:51.217 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.217 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.218 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.218 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T07:06:51.218 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.218 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.218 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.218 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-09-19T07:06:51.218 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T07:06:51.218 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.218 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.218 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.218 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.218 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.218 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.219 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.219 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T07:06:51.219 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.219 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.219 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.219 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.219 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.219 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-09-19T07:06:51.219 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.219 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.219 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.219 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-09-19T07:06:51.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-09-19T07:06:51.220 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:06:51.220 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.220 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.220 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-09-19T07:06:51.220 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.220 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.220 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.220 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.220 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.220 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.220 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.220 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-09-19T07:06:51.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-09-19T07:06:51.221 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.221 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.221 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.221 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.221 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.221 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.221 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.221 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.221 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.221 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.221 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.222 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:06:51.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:06:51.222 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.222 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.222 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.222 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:06:51.222 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.222 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.222 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.222 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.222 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.222 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.222 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.222 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.223 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.223 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.223 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.223 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.223 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.223 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.223 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.223 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.223 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.223 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.223 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.223 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.223 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.223 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.224 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.224 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.224 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.224 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.224 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.224 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.224 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.224 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.224 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.224 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.224 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.224 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.224 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.225 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.225 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.225 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.225 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.225 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.225 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.225 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.225 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.225 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.225 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.225 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.225 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.225 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.226 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.226 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.226 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.226 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.226 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.226 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.226 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.226 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.226 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.226 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.226 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.226 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.226 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.226 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.227 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.227 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.227 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.227 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.227 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.227 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.227 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.227 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.227 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.227 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.227 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-09-19T07:06:51.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-09-19T07:06:51.227 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.228 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.228 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.228 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.228 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.228 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.228 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.228 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.228 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.228 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.228 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.228 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-09-19T07:06:51.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-09-19T07:06:51.228 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.229 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.229 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.229 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-09-19T07:06:51.229 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.229 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.229 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.229 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.229 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.229 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.229 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.229 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-09-19T07:06:51.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-09-19T07:06:51.230 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.230 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.230 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.230 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T07:06:51.230 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.230 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.230 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.230 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.230 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.230 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.230 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.230 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.231 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.231 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.231 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:06:51.231 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.231 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.231 INFO:teuthology.orchestra.run.smithi103.stdout: "modules": [ 2024-09-19T07:06:51.231 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-19T07:06:51.231 INFO:teuthology.orchestra.run.smithi103.stdout: "dashboard", 2024-09-19T07:06:51.231 INFO:teuthology.orchestra.run.smithi103.stdout: "iostat", 2024-09-19T07:06:51.231 INFO:teuthology.orchestra.run.smithi103.stdout: "nfs", 2024-09-19T07:06:51.231 INFO:teuthology.orchestra.run.smithi103.stdout: "restful", 2024-09-19T07:06:51.231 INFO:teuthology.orchestra.run.smithi103.stdout: "snap_schedule" 2024-09-19T07:06:51.231 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.231 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-09-19T07:06:51.231 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-09-19T07:06:51.232 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.232 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.232 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.232 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:06:51.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:06:51.232 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.232 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.232 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.232 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:06:51.232 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.232 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.233 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.233 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T07:06:51.233 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.233 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.233 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.233 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.233 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.233 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.233 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.233 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.233 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.234 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.234 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.234 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.234 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.234 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.234 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.234 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.234 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.234 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.234 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.234 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.234 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.234 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.235 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.235 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.235 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.235 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.235 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.235 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.235 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.235 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.235 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.235 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.235 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.236 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.236 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.236 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.236 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.236 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.236 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.236 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.236 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.236 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.236 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.236 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.236 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.236 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.237 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.237 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.237 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.237 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.237 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.237 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.237 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.237 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.237 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.237 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.237 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.237 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.237 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.238 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.238 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.238 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.238 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.238 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.238 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.238 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.238 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.238 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-09-19T07:06:51.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-09-19T07:06:51.238 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.238 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.238 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.239 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.239 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.239 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.239 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.239 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to", 2024-09-19T07:06:51.239 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.239 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.239 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.239 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-09-19T07:06:51.239 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-09-19T07:06:51.239 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.239 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.239 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.239 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-09-19T07:06:51.240 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.240 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.240 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.240 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-09-19T07:06:51.240 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.240 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.240 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.240 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-09-19T07:06:51.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-09-19T07:06:51.240 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.240 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.240 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.240 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.241 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.241 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.241 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.241 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-09-19T07:06:51.241 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.241 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.241 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.241 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-09-19T07:06:51.241 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-09-19T07:06:51.241 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.241 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.241 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.241 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.242 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.242 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.242 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.242 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-09-19T07:06:51.242 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.242 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.242 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.242 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-09-19T07:06:51.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-09-19T07:06:51.242 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.242 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.242 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.242 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-09-19T07:06:51.243 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.243 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.243 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.243 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-09-19T07:06:51.243 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.243 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.243 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.243 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-09-19T07:06:51.243 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-09-19T07:06:51.243 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.243 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.243 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.243 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.243 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.244 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.244 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.244 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-09-19T07:06:51.244 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.244 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.244 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.244 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-09-19T07:06:51.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-09-19T07:06:51.244 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.244 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.244 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.244 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.245 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.245 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.245 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.245 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T07:06:51.245 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.245 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.245 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.245 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-09-19T07:06:51.245 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-09-19T07:06:51.245 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.245 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.245 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.245 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.246 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.246 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.246 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.246 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-09-19T07:06:51.246 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.246 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.246 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.246 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.246 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.246 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-09-19T07:06:51.246 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.246 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.247 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.247 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-09-19T07:06:51.247 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-09-19T07:06:51.247 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.247 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.247 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.247 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.247 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.247 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.247 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.247 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T07:06:51.247 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.247 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.247 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.248 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-09-19T07:06:51.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-09-19T07:06:51.248 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.248 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.248 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.248 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-09-19T07:06:51.248 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.248 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.248 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.248 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T07:06:51.248 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:06:51.248 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.249 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.249 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-09-19T07:06:51.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-09-19T07:06:51.249 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:06:51.249 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.249 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.249 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:06:51.249 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-19T07:06:51.249 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-19T07:06:51.249 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.249 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T07:06:51.249 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:06:51.249 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.250 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.250 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-09-19T07:06:51.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-09-19T07:06:51.250 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:06:51.250 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.250 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.250 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-09-19T07:06:51.250 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:06:51.250 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-09-19T07:06:51.250 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.250 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T07:06:51.250 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.251 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.251 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.251 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-09-19T07:06:51.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-09-19T07:06:51.251 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.251 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.251 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.251 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T07:06:51.251 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.251 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.251 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.251 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T07:06:51.251 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T07:06:51.252 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.252 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.252 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-09-19T07:06:51.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-09-19T07:06:51.252 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:51.252 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.252 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.252 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-09-19T07:06:51.252 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-09-19T07:06:51.252 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-09-19T07:06:51.252 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.252 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-09-19T07:06:51.252 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T07:06:51.253 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.253 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.253 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-09-19T07:06:51.253 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-09-19T07:06:51.253 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.253 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.253 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.253 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-09-19T07:06:51.253 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.253 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.253 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.253 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T07:06:51.253 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:06:51.254 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.254 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.254 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-09-19T07:06:51.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-09-19T07:06:51.254 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:06:51.254 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.254 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.254 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:06:51.254 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-19T07:06:51.254 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-19T07:06:51.254 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.254 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T07:06:51.254 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:06:51.254 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.255 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.255 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.255 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.255 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.255 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.255 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.255 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.255 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.255 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.255 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.255 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.256 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.256 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.256 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.256 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.256 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.256 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.256 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.256 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.256 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.256 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.256 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.256 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.256 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.256 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.257 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.257 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.257 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.257 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.257 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.257 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.257 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.257 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.257 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.257 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.257 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.257 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.257 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.258 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.258 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.258 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.258 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.258 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.258 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.258 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.258 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.258 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.258 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.258 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.258 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.258 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.259 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.259 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.259 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.259 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.259 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.259 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.259 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.259 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.259 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.259 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.259 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.259 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.259 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.259 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.260 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.260 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-09-19T07:06:51.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-09-19T07:06:51.260 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:51.260 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.260 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.260 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:06:51.260 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.260 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.260 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.260 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T07:06:51.260 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.261 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.261 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.261 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.261 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-19T07:06:51.261 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-19T07:06:51.261 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.261 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.261 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.261 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-09-19T07:06:51.261 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.261 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.261 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.261 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-09-19T07:06:51.261 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-09-19T07:06:51.262 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-09-19T07:06:51.262 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-09-19T07:06:51.262 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-09-19T07:06:51.262 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.262 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-09-19T07:06:51.262 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.262 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.262 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.262 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-09-19T07:06:51.263 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-09-19T07:06:51.263 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.263 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.263 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.263 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.263 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.263 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.263 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.263 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T07:06:51.263 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.263 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.263 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.263 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.264 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:06:51.264 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:06:51.264 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.264 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.264 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.264 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:06:51.264 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.264 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.264 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.264 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T07:06:51.264 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.264 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.264 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.265 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-09-19T07:06:51.265 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-09-19T07:06:51.265 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.265 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.265 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.265 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:06:51.265 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:06:51.265 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.265 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.265 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T07:06:51.265 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T07:06:51.265 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.265 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.266 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-09-19T07:06:51.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-09-19T07:06:51.266 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:06:51.266 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.266 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.266 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:51.266 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.266 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.266 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.266 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T07:06:51.266 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.266 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.266 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.266 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.267 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.267 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.267 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-09-19T07:06:51.267 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.267 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.267 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.267 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-09-19T07:06:51.267 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-09-19T07:06:51.267 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:51.267 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.267 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.267 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-19T07:06:51.267 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.268 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.268 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.268 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T07:06:51.268 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.268 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.268 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.268 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-09-19T07:06:51.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-09-19T07:06:51.268 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.268 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.268 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.269 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-09-19T07:06:51.269 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.269 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.269 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.269 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T07:06:51.269 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.269 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.269 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.269 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-09-19T07:06:51.269 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-09-19T07:06:51.269 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.269 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.269 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.270 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-09-19T07:06:51.270 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.270 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.270 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.270 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T07:06:51.270 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.270 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.270 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.270 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.270 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-09-19T07:06:51.270 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-09-19T07:06:51.270 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.270 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.270 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.270 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.271 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.271 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.271 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.271 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T07:06:51.271 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T07:06:51.271 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.271 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.271 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.271 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-09-19T07:06:51.271 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-09-19T07:06:51.271 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.271 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.271 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.272 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:06:51.272 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.272 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.272 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.272 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T07:06:51.272 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.272 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.272 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.273 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-09-19T07:06:51.273 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T07:06:51.273 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:51.273 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.273 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.273 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-09-19T07:06:51.273 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.273 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.273 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.273 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T07:06:51.273 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.273 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.273 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.274 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-09-19T07:06:51.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-09-19T07:06:51.274 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.274 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.274 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.274 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.274 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.274 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.274 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.274 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-09-19T07:06:51.274 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-09-19T07:06:51.274 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-09-19T07:06:51.274 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.274 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T07:06:51.275 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.275 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.275 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.275 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-09-19T07:06:51.275 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-09-19T07:06:51.275 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.275 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.275 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.275 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.275 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.275 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.275 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.275 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T07:06:51.276 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.276 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.276 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.276 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-09-19T07:06:51.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-09-19T07:06:51.276 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.276 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.276 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.276 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.276 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.276 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.276 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.276 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T07:06:51.277 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.277 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.277 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.277 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.277 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-09-19T07:06:51.277 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-09-19T07:06:51.277 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.277 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.277 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.277 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.277 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.277 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.277 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.277 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T07:06:51.278 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.278 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.278 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.278 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-09-19T07:06:51.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-09-19T07:06:51.278 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.278 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.278 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.278 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T07:06:51.278 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.278 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.278 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.278 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:06:51.278 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.278 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.279 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.279 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.279 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-09-19T07:06:51.279 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-09-19T07:06:51.279 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.279 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.279 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.279 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T07:06:51.279 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.279 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.279 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.279 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-09-19T07:06:51.279 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.279 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.279 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.279 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.280 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-09-19T07:06:51.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-09-19T07:06:51.280 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.280 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.280 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.280 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T07:06:51.280 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.280 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.280 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.280 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-09-19T07:06:51.280 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.280 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.280 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.281 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-09-19T07:06:51.281 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-09-19T07:06:51.281 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.281 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.281 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.281 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T07:06:51.281 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.281 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.281 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.281 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:06:51.281 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.281 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.281 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.281 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.282 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-09-19T07:06:51.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-09-19T07:06:51.282 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.282 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.282 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.282 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T07:06:51.282 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.282 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.282 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.282 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-09-19T07:06:51.282 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.282 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.282 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.283 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-09-19T07:06:51.283 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-09-19T07:06:51.283 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.283 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.283 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.283 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T07:06:51.283 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.283 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.283 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.283 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-09-19T07:06:51.283 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.283 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.283 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.283 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.283 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-09-19T07:06:51.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-09-19T07:06:51.284 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.284 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.284 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.284 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T07:06:51.284 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.284 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.284 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.284 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-09-19T07:06:51.284 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.284 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.284 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.284 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-09-19T07:06:51.285 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-09-19T07:06:51.285 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.285 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.285 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.285 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T07:06:51.285 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.285 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.285 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.285 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-09-19T07:06:51.285 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.285 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.285 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.285 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-09-19T07:06:51.285 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-09-19T07:06:51.286 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.286 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.286 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.286 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T07:06:51.286 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.286 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.286 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.286 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-09-19T07:06:51.286 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.286 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.286 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.286 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.286 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-09-19T07:06:51.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-09-19T07:06:51.287 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.287 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.287 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.287 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T07:06:51.287 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.287 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.287 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.287 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-09-19T07:06:51.287 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.287 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.287 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.287 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.287 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-09-19T07:06:51.287 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-09-19T07:06:51.287 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.288 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.288 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.288 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T07:06:51.288 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.288 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.288 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.288 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:06:51.288 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.288 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.288 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.288 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.288 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-09-19T07:06:51.288 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-09-19T07:06:51.288 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.289 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.289 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.289 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T07:06:51.289 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.289 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.289 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.289 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-09-19T07:06:51.289 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.289 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.289 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.289 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.289 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-09-19T07:06:51.289 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-09-19T07:06:51.289 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.290 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.290 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.290 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T07:06:51.290 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.290 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.290 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.290 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:06:51.290 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.290 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.290 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.290 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.290 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-09-19T07:06:51.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-09-19T07:06:51.290 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.290 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.291 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.291 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T07:06:51.291 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.291 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.291 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.291 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-09-19T07:06:51.291 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.291 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.291 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.291 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.291 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-09-19T07:06:51.291 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-09-19T07:06:51.291 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.292 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.292 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.292 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T07:06:51.292 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.292 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.292 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.292 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-09-19T07:06:51.292 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.292 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.292 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.292 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.292 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-09-19T07:06:51.292 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-09-19T07:06:51.292 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.293 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.293 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.293 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T07:06:51.293 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.293 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.293 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.293 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-09-19T07:06:51.293 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.293 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.293 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.293 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.293 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-09-19T07:06:51.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-09-19T07:06:51.293 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.294 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.294 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.294 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.294 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.294 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.294 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.294 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T07:06:51.294 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.294 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.294 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.294 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.294 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-09-19T07:06:51.294 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-09-19T07:06:51.294 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.295 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.295 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.295 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:06:51.295 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.295 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.295 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.295 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T07:06:51.295 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.295 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.295 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.295 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-09-19T07:06:51.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T07:06:51.295 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.296 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.296 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.296 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-09-19T07:06:51.296 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.296 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.296 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.296 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T07:06:51.296 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.296 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.296 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.296 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-09-19T07:06:51.296 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-09-19T07:06:51.297 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.297 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.297 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.297 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-09-19T07:06:51.297 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.297 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.297 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.297 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T07:06:51.297 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.297 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.297 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.297 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.297 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-09-19T07:06:51.297 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-09-19T07:06:51.298 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.298 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.298 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.298 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-09-19T07:06:51.298 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.298 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.298 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.298 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-09-19T07:06:51.298 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.298 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.298 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.298 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.298 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-09-19T07:06:51.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-09-19T07:06:51.298 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.299 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.299 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.299 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.299 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.299 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.299 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.299 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T07:06:51.299 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.299 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.299 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.299 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-09-19T07:06:51.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-09-19T07:06:51.300 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.300 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.300 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.300 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:06:51.300 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.300 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.300 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.300 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-09-19T07:06:51.300 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.300 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.300 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.300 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-09-19T07:06:51.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-09-19T07:06:51.301 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.301 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.301 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.301 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T07:06:51.301 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.301 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.301 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.301 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T07:06:51.301 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.301 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.301 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.301 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.301 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-09-19T07:06:51.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-09-19T07:06:51.302 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.302 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.302 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.302 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:06:51.302 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.302 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.302 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.302 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-09-19T07:06:51.302 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.302 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.302 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.302 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-09-19T07:06:51.303 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-09-19T07:06:51.303 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.303 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.303 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.303 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.303 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.303 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.303 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.303 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T07:06:51.303 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.303 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.303 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.303 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.303 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-09-19T07:06:51.304 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-09-19T07:06:51.304 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.304 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.304 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.304 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.304 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.304 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.304 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.304 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T07:06:51.304 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.304 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.304 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.304 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.304 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.305 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.305 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.305 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.305 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.305 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.305 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.305 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.305 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.305 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.305 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.305 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.305 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.305 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.305 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.306 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.306 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.306 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.306 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.306 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.306 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-09-19T07:06:51.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-09-19T07:06:51.306 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.306 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.306 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.306 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.306 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.306 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.307 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.307 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T07:06:51.307 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.307 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.307 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.307 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.307 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.307 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.307 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.307 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.307 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.307 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.307 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.307 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.307 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.308 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T07:06:51.308 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.308 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.308 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.308 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.308 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.308 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.308 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.308 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.308 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.308 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.308 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.308 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.308 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.309 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.309 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.309 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.309 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.309 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.309 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.309 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.309 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.309 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.309 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.309 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.309 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.309 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.310 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.310 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.310 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.310 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.310 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.310 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.310 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.310 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.310 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.310 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.310 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.310 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.310 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T07:06:51.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T07:06:51.311 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.311 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.311 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.311 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.311 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.311 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.311 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.311 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T07:06:51.311 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.311 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.311 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.311 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T07:06:51.311 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T07:06:51.311 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.312 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.312 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.312 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-09-19T07:06:51.312 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.312 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.312 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.312 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T07:06:51.312 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.312 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.312 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.312 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-09-19T07:06:51.312 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-09-19T07:06:51.312 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.313 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.313 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.313 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:51.313 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.313 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.313 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.313 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-09-19T07:06:51.313 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.313 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.313 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.313 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-09-19T07:06:51.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-09-19T07:06:51.313 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.314 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.314 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.314 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:51.314 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.314 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.314 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.314 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T07:06:51.314 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.314 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.314 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.314 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-09-19T07:06:51.314 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-09-19T07:06:51.314 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.314 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.315 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.315 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.315 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.315 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.315 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.315 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-09-19T07:06:51.315 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.315 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.315 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.315 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-19T07:06:51.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-19T07:06:51.315 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.315 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.315 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.316 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-09-19T07:06:51.316 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.316 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.316 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.316 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-09-19T07:06:51.316 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-09-19T07:06:51.316 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.316 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T07:06:51.316 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.316 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.316 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.316 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.316 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-09-19T07:06:51.316 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-09-19T07:06:51.317 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.317 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.317 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.317 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-09-19T07:06:51.317 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.317 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.317 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.317 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T07:06:51.317 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.317 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.317 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.317 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-09-19T07:06:51.318 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-09-19T07:06:51.318 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.318 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.318 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.318 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T07:06:51.318 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.318 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.318 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.318 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T07:06:51.318 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.318 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.318 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.318 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-09-19T07:06:51.318 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-09-19T07:06:51.319 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.319 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.319 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.319 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.319 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.319 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.319 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.319 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T07:06:51.319 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.319 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.319 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.319 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-09-19T07:06:51.320 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-09-19T07:06:51.320 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.320 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.320 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.320 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.320 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.320 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.320 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.320 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T07:06:51.320 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.320 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.320 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.321 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-09-19T07:06:51.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-09-19T07:06:51.321 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.321 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.321 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.321 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.321 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.321 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.321 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.321 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T07:06:51.321 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.321 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.322 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.322 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.322 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-09-19T07:06:51.322 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-09-19T07:06:51.322 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.322 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.322 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.322 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.322 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.322 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.322 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.322 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T07:06:51.322 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.322 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.323 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.323 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-09-19T07:06:51.323 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-09-19T07:06:51.323 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.323 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.323 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.323 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.323 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.323 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.323 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.323 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T07:06:51.323 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.324 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.324 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.324 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-09-19T07:06:51.324 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-09-19T07:06:51.324 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.324 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.324 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.324 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-09-19T07:06:51.324 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.324 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.324 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.324 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-09-19T07:06:51.325 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.325 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.325 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.325 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-09-19T07:06:51.325 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-09-19T07:06:51.325 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.325 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.325 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.325 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.326 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.326 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.326 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.326 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T07:06:51.326 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.326 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.326 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.326 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-09-19T07:06:51.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-09-19T07:06:51.326 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.326 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.327 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.327 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.327 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.327 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.327 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.327 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T07:06:51.327 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.327 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.327 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.327 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-09-19T07:06:51.327 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-09-19T07:06:51.327 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.327 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.328 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.328 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.328 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.328 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.328 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.328 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T07:06:51.328 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.328 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.328 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.329 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-09-19T07:06:51.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-09-19T07:06:51.329 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.329 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.329 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.329 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.329 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.329 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.329 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.329 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T07:06:51.329 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.329 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.329 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.330 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-09-19T07:06:51.330 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-09-19T07:06:51.330 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.330 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.330 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.330 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.330 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.330 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.330 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.330 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T07:06:51.330 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.330 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.331 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.331 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-09-19T07:06:51.331 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-09-19T07:06:51.331 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.331 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.331 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.331 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.331 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.331 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.331 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.331 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T07:06:51.331 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.331 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.332 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.332 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.332 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.332 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-09-19T07:06:51.332 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.332 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.332 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.332 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.332 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.332 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.332 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.333 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.333 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.333 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.333 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.333 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.333 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.333 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.333 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.333 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.333 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.333 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.333 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.333 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.333 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.334 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.334 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.334 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.334 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.334 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.334 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.334 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.334 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.334 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.334 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.334 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.335 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.335 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.335 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.335 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.335 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.335 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.335 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.335 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.335 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.335 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.335 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.335 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.336 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.336 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.336 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.336 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.336 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.336 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.336 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.336 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.336 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.336 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.336 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.336 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.336 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.337 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.337 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.337 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.337 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.337 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.337 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.337 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.337 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.337 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.337 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.337 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-09-19T07:06:51.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-09-19T07:06:51.337 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.338 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.338 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.338 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-09-19T07:06:51.338 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.338 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.338 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.338 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T07:06:51.338 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.338 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.338 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.338 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.338 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-09-19T07:06:51.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-09-19T07:06:51.338 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.339 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.339 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.339 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-09-19T07:06:51.339 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.339 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.339 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.339 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T07:06:51.339 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.339 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.339 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.339 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.339 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.339 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-09-19T07:06:51.340 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.340 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.340 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.340 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T07:06:51.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T07:06:51.340 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.340 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.340 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.340 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:51.340 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.341 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.341 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.341 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.341 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.341 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.341 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.341 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.341 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T07:06:51.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T07:06:51.341 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.341 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.341 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.341 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.342 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.342 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.342 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.342 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.342 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.342 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.342 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.342 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T07:06:51.342 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T07:06:51.342 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.342 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.342 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.342 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.343 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.343 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.343 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.343 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.343 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.343 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.343 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.343 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-09-19T07:06:51.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T07:06:51.343 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.343 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.343 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.343 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.344 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.344 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.344 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.344 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.344 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.344 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.344 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.344 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.344 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T07:06:51.344 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T07:06:51.344 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.344 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.344 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.345 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.345 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.345 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.345 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.345 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.345 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.345 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.345 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.345 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.345 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T07:06:51.345 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T07:06:51.345 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.345 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.346 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.346 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.346 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.346 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.346 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.346 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.346 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.346 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.346 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.346 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T07:06:51.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T07:06:51.346 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.347 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.347 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.347 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.347 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.347 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.347 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.347 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.347 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.347 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.347 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.347 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T07:06:51.348 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T07:06:51.348 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.348 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.348 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.348 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.348 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.348 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.348 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.348 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.348 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.348 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.348 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.349 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T07:06:51.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T07:06:51.349 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.349 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.349 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.349 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.349 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.349 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.349 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.349 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.349 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.349 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.349 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.350 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T07:06:51.350 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T07:06:51.350 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.350 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.350 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.350 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.350 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.350 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.350 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.350 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.350 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.350 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.350 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.351 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T07:06:51.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T07:06:51.351 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.351 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.351 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.351 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.351 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.351 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.351 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.351 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.351 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.351 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.351 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.351 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T07:06:51.352 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T07:06:51.352 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.352 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.352 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.352 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.352 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.352 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.352 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.352 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.352 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.352 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.352 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.352 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T07:06:51.353 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T07:06:51.353 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.353 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.353 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.353 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.353 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.353 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.353 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.353 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.353 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.353 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.353 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.353 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T07:06:51.353 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T07:06:51.354 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.354 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.354 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.354 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.354 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.354 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.354 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.354 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.354 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.354 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.354 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.355 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T07:06:51.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T07:06:51.355 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.355 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.355 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.355 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:06:51.355 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.355 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.355 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.355 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.355 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.355 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.355 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.356 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T07:06:51.356 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T07:06:51.356 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.356 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.356 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.356 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.356 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.356 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.356 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.356 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.356 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.356 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.356 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.357 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-09-19T07:06:51.357 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-09-19T07:06:51.357 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.357 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.357 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.357 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.357 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.357 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.357 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.357 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.357 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.357 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.357 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.358 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T07:06:51.358 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T07:06:51.358 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.358 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.358 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.358 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:06:51.358 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.358 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.358 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.358 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.358 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.358 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.358 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.358 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.359 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T07:06:51.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T07:06:51.359 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.359 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.359 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.359 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.359 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.359 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.359 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.359 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.359 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.359 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.359 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.360 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T07:06:51.360 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T07:06:51.360 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.360 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.360 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.360 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.360 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.360 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.360 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.360 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.360 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.360 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.360 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.361 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T07:06:51.361 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T07:06:51.361 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.361 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.361 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.361 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.361 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.361 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.361 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.361 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.361 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.361 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.361 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.362 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T07:06:51.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T07:06:51.362 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.362 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.362 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.362 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.362 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.362 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.362 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.362 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.362 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.362 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.362 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.363 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T07:06:51.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T07:06:51.363 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.363 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.363 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.363 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.363 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.363 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.363 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.363 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.363 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.363 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.364 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.364 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T07:06:51.364 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T07:06:51.364 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.364 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.364 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.364 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.364 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.364 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.364 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.364 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.364 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.364 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.365 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.365 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T07:06:51.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T07:06:51.365 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.365 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.365 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.365 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.365 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.365 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.365 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.365 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.365 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.365 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.366 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.366 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T07:06:51.366 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T07:06:51.366 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.366 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.366 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.366 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.366 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.366 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.366 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.366 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.366 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.366 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.367 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.367 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T07:06:51.367 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T07:06:51.367 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.367 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.367 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.367 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.367 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.367 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.367 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.367 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.367 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.367 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.367 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.368 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T07:06:51.368 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T07:06:51.368 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.368 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.368 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.368 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.368 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.368 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.368 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.368 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.368 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.368 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.368 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.369 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T07:06:51.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T07:06:51.369 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.369 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.369 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.369 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.369 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.369 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.369 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.369 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.369 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.369 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.369 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.370 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T07:06:51.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T07:06:51.370 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.370 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.370 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.370 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.370 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.370 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.370 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.370 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.370 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.370 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.370 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.371 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T07:06:51.371 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T07:06:51.371 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.371 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.371 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.371 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.371 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.371 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.371 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.371 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.371 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.371 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.372 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.372 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T07:06:51.372 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T07:06:51.372 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.372 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.372 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.372 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.372 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.372 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.372 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.372 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.372 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.373 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.373 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.373 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T07:06:51.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T07:06:51.373 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.373 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.373 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.373 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T07:06:51.373 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.373 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.373 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.373 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.374 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.374 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.374 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.374 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T07:06:51.374 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T07:06:51.374 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.374 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.374 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.374 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:51.374 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.374 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.374 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.375 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.375 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.375 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.375 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.375 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T07:06:51.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T07:06:51.375 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.375 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.375 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.375 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-09-19T07:06:51.375 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.375 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.375 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.375 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.376 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.376 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.376 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.376 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T07:06:51.376 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T07:06:51.376 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.376 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.376 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.376 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-09-19T07:06:51.376 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.376 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.376 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.376 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.377 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.377 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.377 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.377 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T07:06:51.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T07:06:51.377 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.377 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.377 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.377 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.377 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.377 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.377 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.378 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.378 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.378 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.378 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.378 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T07:06:51.378 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T07:06:51.378 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.378 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.378 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.378 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:06:51.378 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.378 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.378 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.379 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.379 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.379 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.379 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.379 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T07:06:51.379 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T07:06:51.379 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.379 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.379 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.379 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.379 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.379 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.379 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.380 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.380 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.380 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.380 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.380 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T07:06:51.380 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T07:06:51.380 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.380 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.380 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.380 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.380 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.380 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.380 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.381 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.381 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.381 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.381 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.381 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T07:06:51.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T07:06:51.381 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.381 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.381 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.381 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:06:51.381 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.381 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.381 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.382 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.382 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.382 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.382 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.382 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T07:06:51.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T07:06:51.382 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.382 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.382 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.382 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:51.382 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.382 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.382 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.382 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.383 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.383 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.383 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.383 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T07:06:51.383 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T07:06:51.383 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.383 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.383 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.383 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:06:51.383 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.383 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.383 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.383 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.384 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.384 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.384 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.384 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-09-19T07:06:51.384 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-09-19T07:06:51.384 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.384 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.384 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.384 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.384 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.384 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.384 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.385 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.385 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.385 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.385 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.385 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-09-19T07:06:51.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-09-19T07:06:51.385 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.385 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.385 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.385 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.385 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.385 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.385 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.386 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.386 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.386 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.386 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.386 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-09-19T07:06:51.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-09-19T07:06:51.386 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.386 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.386 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.386 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.386 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.386 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.386 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.387 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-09-19T07:06:51.387 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.387 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.387 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.387 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-09-19T07:06:51.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-09-19T07:06:51.387 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.387 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.387 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.387 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-09-19T07:06:51.387 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.387 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.388 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.388 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.388 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.388 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.388 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.388 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-19T07:06:51.388 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-19T07:06:51.388 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.388 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.388 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.388 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.389 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.389 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.389 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.389 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.389 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.389 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.389 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.389 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.389 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.389 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.389 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.389 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.389 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.389 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.389 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.391 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.391 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.391 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.391 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.391 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.391 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.391 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.391 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.391 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.391 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.391 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.392 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.392 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.392 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.392 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.392 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.392 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.392 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.392 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.392 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.392 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.392 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.392 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.392 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.393 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.393 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.393 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.393 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.393 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.393 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.393 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.393 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.393 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.393 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.393 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.394 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.394 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.394 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.394 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.394 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.394 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.394 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.394 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.394 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.394 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-09-19T07:06:51.394 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-09-19T07:06:51.394 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.394 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.395 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.395 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.395 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.395 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.395 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.395 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-09-19T07:06:51.395 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.395 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.395 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.395 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.395 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-09-19T07:06:51.395 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T07:06:51.395 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.395 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.396 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.396 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.396 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.396 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.396 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.396 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.396 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.396 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.396 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.396 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:06:51.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:06:51.396 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.396 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.397 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.397 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-19T07:06:51.397 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.397 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.397 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.397 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.397 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.397 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.397 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.397 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:06:51.397 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:06:51.397 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.397 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.398 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.398 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-09-19T07:06:51.398 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.398 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.398 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.398 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.398 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.398 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.398 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.398 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-19T07:06:51.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-19T07:06:51.398 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.399 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.399 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.399 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.399 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.399 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.399 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.399 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.399 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.399 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.399 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.399 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-09-19T07:06:51.399 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-09-19T07:06:51.399 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.399 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.400 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.400 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-09-19T07:06:51.400 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.400 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.400 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.400 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.400 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.400 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.400 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.400 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-19T07:06:51.400 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-19T07:06:51.400 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.400 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.400 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.401 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-09-19T07:06:51.401 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.401 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.401 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.401 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.401 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-09-19T07:06:51.401 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.401 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.401 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.401 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.401 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.401 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-19T07:06:51.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-19T07:06:51.402 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.402 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.402 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.402 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-19T07:06:51.402 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-19T07:06:51.402 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-19T07:06:51.402 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.402 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.402 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.402 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.402 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.402 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.402 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-09-19T07:06:51.402 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-09-19T07:06:51.403 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.403 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.403 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.403 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.403 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.403 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.403 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.403 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.403 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.403 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.403 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.403 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.403 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.404 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-19T07:06:51.404 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.404 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.404 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.404 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-09-19T07:06:51.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-09-19T07:06:51.404 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.404 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.404 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.405 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.405 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.405 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.405 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.405 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-09-19T07:06:51.405 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.405 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.405 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.405 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.405 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.405 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.405 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.405 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.406 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.406 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.406 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.406 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.406 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.406 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.406 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.406 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.406 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.406 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.406 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.406 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.406 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.407 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.407 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.407 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.407 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.407 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.407 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.407 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.407 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.407 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.407 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.407 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.407 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.407 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.408 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.408 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.408 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.408 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.408 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.408 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.408 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.408 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.408 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.408 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.408 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.408 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.409 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.409 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.409 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.409 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.409 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.409 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.409 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.409 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.409 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.409 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.409 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.409 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.409 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.410 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.410 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.410 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.410 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.410 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.410 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.410 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.410 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.410 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.410 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-09-19T07:06:51.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-09-19T07:06:51.410 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.411 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.411 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.411 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-09-19T07:06:51.411 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.411 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.411 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.411 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T07:06:51.411 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.411 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.411 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.411 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-09-19T07:06:51.411 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-09-19T07:06:51.411 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.411 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.412 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.412 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-09-19T07:06:51.412 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.412 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.412 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.412 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T07:06:51.412 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.412 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.412 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.412 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-09-19T07:06:51.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-09-19T07:06:51.412 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.412 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.412 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.412 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-09-19T07:06:51.413 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.413 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.413 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.413 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-09-19T07:06:51.413 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.413 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.413 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.413 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-09-19T07:06:51.413 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-09-19T07:06:51.413 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.413 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.413 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.413 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-19T07:06:51.414 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.414 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.414 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.414 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T07:06:51.414 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.414 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.414 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.414 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-09-19T07:06:51.414 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-09-19T07:06:51.414 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.414 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.414 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.414 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.414 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.415 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.415 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.415 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T07:06:51.415 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.415 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.415 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.415 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:06:51.415 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:06:51.415 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.415 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.415 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.415 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:06:51.415 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.415 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.416 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.416 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T07:06:51.416 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.416 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.416 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.416 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-09-19T07:06:51.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-09-19T07:06:51.416 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.416 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.416 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.416 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-09-19T07:06:51.416 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.416 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.417 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.417 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T07:06:51.417 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.417 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.417 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.417 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.417 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.417 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.417 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-09-19T07:06:51.417 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.417 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.417 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.417 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.417 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.418 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.418 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.418 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.418 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.418 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.418 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.418 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.418 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.418 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.418 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.418 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.418 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.418 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.418 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.419 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.419 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.419 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.419 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.419 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.419 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.419 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.419 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.419 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.419 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.419 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.419 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.419 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.419 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.420 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.420 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.420 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.420 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.420 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.420 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.420 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.421 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.421 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.421 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.421 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.421 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.421 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.421 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.421 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.421 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.421 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.421 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.421 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.421 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.422 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.422 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.422 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.422 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.422 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.422 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.422 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.422 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.422 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.422 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.423 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.423 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.423 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.423 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.423 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.423 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-09-19T07:06:51.423 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-09-19T07:06:51.423 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.423 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.423 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.423 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-19T07:06:51.423 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.423 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.423 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.424 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.424 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.424 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.424 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.424 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.424 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-09-19T07:06:51.424 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-09-19T07:06:51.424 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.424 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.424 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.425 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-09-19T07:06:51.425 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.425 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.425 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.425 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.425 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.425 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.425 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.425 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:06:51.425 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:06:51.425 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.425 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.425 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.426 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:06:51.426 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.426 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.426 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.426 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.426 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.426 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.426 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.426 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.426 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.426 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.426 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-09-19T07:06:51.426 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-09-19T07:06:51.427 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-09-19T07:06:51.427 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.427 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-09-19T07:06:51.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-09-19T07:06:51.427 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.427 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.427 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.427 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-09-19T07:06:51.427 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.427 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.427 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.427 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T07:06:51.427 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.428 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.428 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.428 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-09-19T07:06:51.428 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-09-19T07:06:51.428 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.428 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.428 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.428 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-09-19T07:06:51.428 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.428 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.428 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.428 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T07:06:51.428 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.429 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.429 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.429 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-09-19T07:06:51.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-09-19T07:06:51.429 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.429 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.429 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.429 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.429 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.429 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.429 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.429 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-09-19T07:06:51.429 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.430 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.430 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.430 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.430 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:06:51.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:06:51.430 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.430 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.430 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.430 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-09-19T07:06:51.430 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-09-19T07:06:51.430 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.430 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.430 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T07:06:51.430 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.430 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.431 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.431 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.431 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.431 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.431 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.431 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.431 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.431 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.431 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.431 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.431 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.431 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.431 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.432 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.432 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.432 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.432 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.432 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.432 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.432 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.432 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.432 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.432 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.432 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.433 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.433 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.433 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.433 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.433 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.433 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.433 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.433 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.433 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.433 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.433 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.433 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.433 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.434 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.434 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.434 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.434 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.434 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.434 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.434 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.434 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.434 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.434 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.434 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.434 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.434 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.434 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.435 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.435 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.435 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.435 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.435 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.435 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.435 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.435 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.435 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.435 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.435 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.435 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.435 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.435 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.435 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.436 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.436 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.436 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-09-19T07:06:51.436 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-09-19T07:06:51.436 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.436 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.436 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.436 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.436 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.436 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.436 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.436 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-09-19T07:06:51.436 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.437 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.437 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.437 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-09-19T07:06:51.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-09-19T07:06:51.437 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.437 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.437 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.437 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-09-19T07:06:51.437 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.437 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.438 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.438 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-09-19T07:06:51.438 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.438 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.438 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.438 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.438 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-19T07:06:51.438 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-19T07:06:51.438 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.438 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.438 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.438 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-09-19T07:06:51.438 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.439 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.439 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.439 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:06:51.439 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.439 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.439 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.439 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-09-19T07:06:51.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-09-19T07:06:51.439 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.439 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.439 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.439 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:06:51.440 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:06:51.440 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-09-19T07:06:51.440 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.440 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T07:06:51.440 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.440 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.440 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.440 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-09-19T07:06:51.440 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-09-19T07:06:51.440 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.440 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.440 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.441 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.441 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.441 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.441 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.441 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-09-19T07:06:51.441 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.441 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.441 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.441 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-09-19T07:06:51.441 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-09-19T07:06:51.441 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.442 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.442 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.442 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-09-19T07:06:51.442 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.442 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.442 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.442 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:06:51.442 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.442 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.442 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.442 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.442 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.443 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-09-19T07:06:51.443 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.443 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.443 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.443 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.443 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.443 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.443 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.443 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.443 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.443 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.443 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.443 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.444 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.444 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.444 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.444 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.444 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.444 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.444 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.444 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.444 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.444 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.444 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.444 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.445 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.445 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.445 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.445 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.445 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.445 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.445 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.445 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.445 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.445 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.445 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.445 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.446 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.446 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.446 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.446 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.446 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.446 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.446 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.446 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.446 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.446 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.446 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.446 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.446 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.446 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.446 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.447 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.447 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.447 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.447 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.447 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.447 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.447 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.447 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.447 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.447 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.447 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.447 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.447 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.448 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.448 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.448 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.448 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.448 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.448 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.448 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-09-19T07:06:51.448 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.448 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.448 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.448 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.448 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.448 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.449 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.449 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.449 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.449 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.449 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.449 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.449 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.449 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.449 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.449 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.449 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.449 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.449 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.450 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.450 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.450 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.450 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.450 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.450 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.450 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.450 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.450 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.450 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.450 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.450 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.451 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.451 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.451 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.451 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.451 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.451 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.451 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.451 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.451 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.451 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.451 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.452 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.452 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.452 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.452 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.452 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.452 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.452 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.452 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.452 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.452 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.452 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.452 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.452 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.452 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.453 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.453 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.453 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.453 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.453 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.453 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.453 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.453 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.453 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.453 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.453 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.453 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.453 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.454 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.454 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.454 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.454 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-09-19T07:06:51.454 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.454 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.454 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.454 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-09-19T07:06:51.454 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-09-19T07:06:51.454 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.454 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.454 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.454 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-09-19T07:06:51.455 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.455 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.455 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.455 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T07:06:51.455 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.455 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.455 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.455 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-09-19T07:06:51.455 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-09-19T07:06:51.455 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.455 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.456 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.456 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:51.456 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-09-19T07:06:51.456 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.456 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.456 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T07:06:51.456 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.456 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.456 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.456 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.456 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.456 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.456 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.457 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.457 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.457 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.457 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.457 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.457 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.457 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.457 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.457 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.457 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.457 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.457 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.457 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.457 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.458 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.458 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.458 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.458 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.458 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.458 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.458 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.458 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.458 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.458 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.458 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.458 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.458 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.459 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.459 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.459 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.459 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.459 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.459 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.459 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.459 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.459 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.459 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.459 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.459 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.460 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.460 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.460 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.460 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.460 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.460 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.460 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.460 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.460 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.460 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.460 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.460 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.460 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.460 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.461 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.461 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.461 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.461 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.461 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.461 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.461 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.461 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.461 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.461 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.461 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.461 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.461 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-09-19T07:06:51.462 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.462 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.462 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.462 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-09-19T07:06:51.462 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-09-19T07:06:51.462 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.462 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.462 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.462 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-09-19T07:06:51.462 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.462 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.462 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.462 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-09-19T07:06:51.463 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T07:06:51.463 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.463 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.463 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.463 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.463 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.463 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.463 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.463 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.463 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.464 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.464 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.464 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.464 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.464 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.464 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.464 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.464 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.464 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.464 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.464 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.464 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.465 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.465 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.465 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.465 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.465 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.465 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.465 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.465 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.466 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.466 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.466 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.466 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.466 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.466 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.466 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.466 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.466 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.466 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.466 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.467 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.467 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.467 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.467 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.467 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.467 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.467 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.467 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.467 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.467 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.467 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.467 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.467 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.468 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.468 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.468 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.468 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.468 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.468 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.468 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.468 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.468 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.468 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.468 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.468 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.469 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.469 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-09-19T07:06:51.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-09-19T07:06:51.469 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.469 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.469 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.469 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.469 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.469 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.469 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.469 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-09-19T07:06:51.469 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T07:06:51.469 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.470 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.470 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.470 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-09-19T07:06:51.470 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-09-19T07:06:51.470 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.470 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.470 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.470 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-09-19T07:06:51.470 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.470 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.470 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.470 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-09-19T07:06:51.470 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.470 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.471 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.471 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-09-19T07:06:51.471 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-09-19T07:06:51.471 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.471 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.471 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.471 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-09-19T07:06:51.471 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.471 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.471 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.471 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T07:06:51.472 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.472 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.472 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.472 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.472 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-09-19T07:06:51.472 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-09-19T07:06:51.472 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.472 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.472 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.472 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.472 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.472 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.472 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.473 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-09-19T07:06:51.473 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.473 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.473 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.473 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.473 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-09-19T07:06:51.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-09-19T07:06:51.473 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.473 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.473 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.473 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-09-19T07:06:51.473 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.473 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.474 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.474 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T07:06:51.474 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T07:06:51.474 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.474 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.474 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.474 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.474 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.474 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-09-19T07:06:51.474 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.474 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.474 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.475 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.475 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.475 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.475 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.475 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.475 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.475 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.475 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.475 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.475 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.475 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.475 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.476 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.476 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.476 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.476 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.476 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.476 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.476 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.476 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.476 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.477 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.477 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.477 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.477 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.477 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.477 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.477 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.477 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.477 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.477 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.477 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.477 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.478 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.478 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.478 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.478 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.478 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.478 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.478 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.478 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.478 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.478 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.478 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.478 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.478 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.478 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.479 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.479 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.479 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.479 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.479 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.479 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.479 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.479 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.479 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.479 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.479 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.479 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.480 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.480 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.480 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.480 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.480 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.480 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.480 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.480 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.480 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-09-19T07:06:51.480 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.480 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.480 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.480 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.481 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.481 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.481 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.481 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.481 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.481 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.481 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.481 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.481 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.481 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.481 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.481 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.481 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.481 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.482 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.482 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.482 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.482 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.482 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.482 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.482 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.482 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.482 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.482 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.482 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.482 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.483 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.483 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.483 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.483 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.483 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.483 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.483 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.483 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.483 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.483 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.483 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.483 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.483 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.483 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.484 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.484 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.484 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.484 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.484 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.484 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.484 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.484 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.484 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.484 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.484 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.484 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.484 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.485 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.485 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.485 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.485 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.485 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.485 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.485 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.485 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.485 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.485 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.485 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.485 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.485 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.486 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.486 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-09-19T07:06:51.486 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.486 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.486 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.486 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.486 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.486 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.486 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.486 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.486 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.487 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.487 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.487 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.487 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.487 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.487 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.487 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.487 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.487 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.487 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.487 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.487 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.487 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.487 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.488 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.488 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.488 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.488 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.488 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.488 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.488 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.488 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.488 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.488 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.488 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.488 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.488 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.489 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.489 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.489 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.489 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.489 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.489 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.489 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.489 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.489 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.489 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.489 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.489 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.489 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.489 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.490 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.490 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.490 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.490 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.490 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.490 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.490 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.490 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.490 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.490 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.490 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.490 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.490 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.490 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.490 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.491 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.491 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.491 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.491 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.491 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.491 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-19T07:06:51.491 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.491 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.491 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.491 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-09-19T07:06:51.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-09-19T07:06:51.491 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.492 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.492 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.492 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.492 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.492 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.492 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.492 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T07:06:51.492 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.492 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.492 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.492 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.492 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.493 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.493 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.493 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.493 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.493 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.493 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.493 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.493 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.493 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.493 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.493 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.493 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.493 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.494 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.494 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.494 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.494 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.494 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.494 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.494 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.494 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.494 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.494 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.494 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.494 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.494 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.494 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.495 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.495 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.495 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.495 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.495 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.495 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.495 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.495 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.495 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.495 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.495 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.495 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.495 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.495 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.495 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.496 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.496 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.496 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.496 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.496 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.496 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.496 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.496 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.496 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.496 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.496 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.496 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.496 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.497 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.497 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.497 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.497 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.497 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.497 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.497 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.497 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.497 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.497 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.497 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.497 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-09-19T07:06:51.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-19T07:06:51.498 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.498 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.498 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.498 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.498 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.498 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.498 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.498 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-19T07:06:51.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-09-19T07:06:51.498 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-09-19T07:06:51.498 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.498 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-09-19T07:06:51.498 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.498 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.498 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.499 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.499 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.499 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.499 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-09-19T07:06:51.499 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.499 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.499 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.499 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.499 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.499 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.499 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.499 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.500 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.500 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.500 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.500 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.500 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.500 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.500 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.500 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.500 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.500 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.500 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.500 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.500 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.500 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.500 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.501 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.501 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.501 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.501 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.501 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.501 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.501 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.501 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.501 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.501 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.501 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.502 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.502 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.502 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.502 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.502 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.502 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.502 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.502 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.502 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.502 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.502 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.502 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.502 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.503 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.503 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.503 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.503 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.503 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.503 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.503 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.503 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.503 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.503 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.503 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.503 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.503 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.504 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.504 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.504 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.504 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.504 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.504 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.504 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.504 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.504 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.504 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.504 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.504 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.504 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.505 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-09-19T07:06:51.505 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.505 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.505 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.505 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.505 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.505 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.505 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.505 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.505 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.505 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.505 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.505 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.505 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.506 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.506 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.506 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.506 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.506 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.506 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.506 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.506 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.506 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.506 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.506 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.506 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.507 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.507 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.507 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.507 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.507 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.507 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.507 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.507 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.507 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.507 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.507 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.507 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.507 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.508 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.508 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.508 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.508 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.508 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.508 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.508 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.508 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.508 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.508 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.508 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.508 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.508 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.509 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.509 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.509 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.509 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.509 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.509 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.509 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.509 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.509 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.509 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.509 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.509 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.509 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.510 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.510 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.510 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.510 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.510 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.510 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.510 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.510 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.510 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.510 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-09-19T07:06:51.510 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.510 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.510 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.511 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.511 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.511 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.511 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.511 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.511 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.511 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.511 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.511 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.511 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.511 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.511 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.511 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.511 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.512 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.512 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.512 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.512 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.512 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.512 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.512 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.512 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.512 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.512 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.512 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.512 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.512 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.513 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.513 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.513 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.513 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.513 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.513 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.513 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.513 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.513 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.513 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.513 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.513 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.513 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.514 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.514 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.514 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.514 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.514 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.514 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.514 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.514 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.514 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.514 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.514 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.514 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.514 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.514 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.515 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.515 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.515 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.515 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.515 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.515 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.515 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.515 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.515 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.515 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.515 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.515 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:06:51.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:06:51.516 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.516 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.516 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.516 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:06:51.516 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.516 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.516 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.516 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.516 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.516 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.516 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.516 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-09-19T07:06:51.516 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-09-19T07:06:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-19T07:06:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-09-19T07:06:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-09-19T07:06:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T07:06:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.517 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.518 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-09-19T07:06:51.518 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.518 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.518 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.518 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-09-19T07:06:51.518 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-09-19T07:06:51.518 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.518 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.518 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.518 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.518 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.518 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.518 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.518 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T07:06:51.518 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.518 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.519 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.519 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.519 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-19T07:06:51.519 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-19T07:06:51.519 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.519 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.519 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.519 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.519 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.519 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.519 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.519 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.519 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.519 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.520 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.520 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.520 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.520 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.520 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.520 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.520 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.520 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.520 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.520 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.520 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.520 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.520 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.520 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.521 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.521 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.521 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.521 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.521 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.521 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.521 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.521 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.521 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.521 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.521 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.521 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.521 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.522 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.522 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.522 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.522 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.522 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.522 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.522 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.522 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.522 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.522 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.522 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.522 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.522 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.523 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.523 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.523 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.523 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.523 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.523 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.523 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.523 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.523 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.523 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.523 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.523 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.523 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.523 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.524 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.524 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.524 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.524 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.524 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.524 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.524 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.524 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.524 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.524 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.524 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.524 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.524 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.524 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.525 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.525 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.525 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.525 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.525 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-09-19T07:06:51.525 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-09-19T07:06:51.525 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.525 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.525 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.525 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-09-19T07:06:51.525 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.525 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.525 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.526 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-09-19T07:06:51.526 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.526 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.526 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.526 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.526 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:06:51.526 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:06:51.526 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.526 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.526 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.526 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:06:51.526 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.526 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.526 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.527 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-09-19T07:06:51.527 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.527 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.527 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.527 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.527 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.527 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.527 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.527 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-09-19T07:06:51.527 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.527 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.527 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.527 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-09-19T07:06:51.527 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-09-19T07:06:51.528 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.528 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.528 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.528 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.528 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.528 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.528 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.528 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.528 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.528 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.528 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.528 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.528 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-09-19T07:06:51.528 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-09-19T07:06:51.529 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.529 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.529 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.529 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.529 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.529 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.529 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.529 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T07:06:51.529 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T07:06:51.529 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.529 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.529 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.530 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.530 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.530 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.530 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.530 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.530 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.530 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.530 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.530 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.530 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.530 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.530 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.531 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.531 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.531 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.531 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.531 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.531 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.531 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.531 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.531 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.531 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.531 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.531 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.531 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.532 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.532 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.532 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.532 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.532 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.532 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.532 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.532 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.532 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.532 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.532 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.532 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.532 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.532 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.533 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.533 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.533 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.533 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.533 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.533 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.533 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.533 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.533 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.533 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.533 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.533 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.535 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.535 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.535 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.535 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.535 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.535 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.535 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.535 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-09-19T07:06:51.535 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-09-19T07:06:51.535 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.535 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.535 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.536 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.536 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.536 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.536 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.536 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.536 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.536 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.536 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.536 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.536 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T07:06:51.536 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T07:06:51.536 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.536 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.537 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.537 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-09-19T07:06:51.537 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.537 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.537 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.537 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.537 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.537 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.537 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.537 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.537 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-09-19T07:06:51.537 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-09-19T07:06:51.537 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:51.538 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.538 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.538 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-09-19T07:06:51.538 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.538 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.538 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.538 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.538 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.538 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.538 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.538 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.538 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:06:51.538 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:06:51.539 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.539 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.539 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.539 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-19T07:06:51.539 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.539 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.539 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.539 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T07:06:51.539 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.539 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.539 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.539 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.539 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:06:51.540 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:06:51.540 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.540 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.540 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.540 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-09-19T07:06:51.540 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.540 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.540 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.540 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T07:06:51.540 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.540 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.540 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.540 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.541 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-09-19T07:06:51.541 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-09-19T07:06:51.541 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.541 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.541 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.541 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-09-19T07:06:51.541 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.541 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.541 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.541 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.541 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.541 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.541 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.541 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.542 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-19T07:06:51.542 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-19T07:06:51.542 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.542 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.542 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.542 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-09-19T07:06:51.542 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.542 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.542 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.542 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-09-19T07:06:51.542 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-09-19T07:06:51.542 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.543 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.543 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.543 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.543 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.543 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.543 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-19T07:06:51.543 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-19T07:06:51.543 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.543 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.543 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.543 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-19T07:06:51.543 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-19T07:06:51.543 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-19T07:06:51.544 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.544 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.544 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.544 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.544 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.544 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.544 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.544 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.544 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.544 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-19T07:06:51.544 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.544 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.544 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.545 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.545 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.545 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.545 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.545 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.545 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.545 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.545 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.545 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.545 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.545 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.545 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.545 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.545 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.546 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.546 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.546 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.546 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.546 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.546 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.546 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.546 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.546 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.546 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.546 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.546 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.546 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.546 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.547 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.547 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.547 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.547 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.547 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.547 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.547 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.547 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.547 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.547 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.547 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.547 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.547 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.547 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.548 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.548 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.548 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.548 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.548 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.548 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.548 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.548 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.548 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.548 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.548 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.548 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.548 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.549 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.549 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.549 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.549 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.549 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.549 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.549 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.549 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.549 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.549 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.549 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.549 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.549 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.549 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.550 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.550 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-09-19T07:06:51.550 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-09-19T07:06:51.550 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.550 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.550 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.550 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:06:51.550 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.550 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.550 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.550 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.550 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.550 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.550 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.551 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.551 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-09-19T07:06:51.551 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T07:06:51.551 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.551 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.551 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.551 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.551 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.551 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.551 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.551 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.551 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.551 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.551 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.552 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.552 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-09-19T07:06:51.552 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-09-19T07:06:51.552 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.552 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.552 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.552 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.552 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.552 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.552 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.552 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.552 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.552 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.552 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.553 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.553 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.553 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.553 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-09-19T07:06:51.553 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.553 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.553 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.553 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-09-19T07:06:51.553 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-09-19T07:06:51.553 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.553 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.553 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.554 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.554 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.554 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.554 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.554 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.554 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.554 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.554 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.554 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.554 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-19T07:06:51.554 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-19T07:06:51.554 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.554 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.555 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.555 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.555 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.555 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.555 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.555 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.555 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.555 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.555 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.555 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.555 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.555 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.555 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.555 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.556 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.556 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.556 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.556 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.556 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.556 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.556 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.556 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.556 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.556 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.556 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.556 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.556 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.556 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.556 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.557 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.557 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.557 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.557 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.557 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.557 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.557 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.557 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.557 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.557 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.557 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.557 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.557 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.557 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.557 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.558 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.558 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.558 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.558 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.558 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.558 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.558 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.558 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.558 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.558 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.558 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.558 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.558 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.558 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.559 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.559 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.559 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.559 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.559 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.559 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.559 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.559 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.559 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.559 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.559 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.559 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.560 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.560 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.560 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.560 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.560 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.560 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.560 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.560 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.560 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.560 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:06:51.560 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:06:51.560 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.560 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.560 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.561 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.561 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.561 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.561 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.561 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.561 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.561 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.561 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.561 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.561 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:06:51.561 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:06:51.561 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.561 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.561 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.561 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.562 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.562 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.562 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.562 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.562 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.562 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.562 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.562 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.562 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.562 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.562 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.562 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-09-19T07:06:51.562 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.562 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.562 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.563 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.563 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.563 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.563 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.563 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.563 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.563 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.563 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.563 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.563 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.563 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.563 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.563 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.564 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.564 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.564 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.564 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.564 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.564 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.564 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.564 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.564 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.564 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.564 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.564 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.564 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.565 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.565 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.565 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.565 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.565 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.565 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.565 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.565 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.565 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.565 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.565 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.565 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.565 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.565 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.566 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.566 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.566 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.566 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.566 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.566 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.566 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.566 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.566 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.566 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.566 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.566 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.566 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.566 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.567 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.567 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.567 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.567 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.567 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.567 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.567 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.567 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.567 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.567 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.567 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.567 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.567 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.567 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.567 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.568 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.568 INFO:teuthology.orchestra.run.smithi103.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T07:06:51.568 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T07:06:51.568 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.568 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.568 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.568 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:06:51.568 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.568 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.568 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.568 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T07:06:51.568 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.568 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.568 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.569 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.569 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.569 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.569 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.569 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-09-19T07:06:51.569 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.569 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.569 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.569 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.569 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.569 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.569 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.569 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.569 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.569 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.570 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.570 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.570 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.570 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.570 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.570 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.570 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.570 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.570 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.570 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.570 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.570 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.570 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.570 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.571 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.571 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.571 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.571 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.571 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.571 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.571 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.571 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.571 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.571 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.571 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.571 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.571 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.571 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.572 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.572 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.572 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.572 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.572 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.572 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.572 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.572 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.572 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.572 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.572 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.572 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.572 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.572 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.572 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.573 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.573 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.573 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.573 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.573 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.573 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.573 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.573 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.573 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.573 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.573 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.573 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.573 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.573 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.574 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.574 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.574 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.574 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.574 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.574 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.574 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-09-19T07:06:51.574 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-09-19T07:06:51.574 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.574 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.574 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.574 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-09-19T07:06:51.574 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.574 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.574 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.575 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T07:06:51.575 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.575 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.575 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.575 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.575 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.575 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.575 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-09-19T07:06:51.575 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.575 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.575 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.575 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.576 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.576 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.576 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.576 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.576 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.576 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.576 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.576 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.576 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.576 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.576 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.576 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.576 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.576 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.577 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.577 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.577 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.577 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.577 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.577 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.577 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.577 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.577 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.577 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.577 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.577 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.578 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.578 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.578 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.578 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.578 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.578 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.578 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.578 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.578 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.578 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.578 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.578 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.578 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.579 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.579 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.579 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.579 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.579 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.579 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.579 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.579 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.579 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.579 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.579 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.579 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.579 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.580 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.580 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.580 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.580 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.580 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.580 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.580 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.580 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.580 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.580 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.580 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.581 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.581 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.581 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.581 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.581 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.581 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-09-19T07:06:51.581 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-09-19T07:06:51.581 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.581 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.581 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.581 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.581 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.581 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.581 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.582 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.582 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.582 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.582 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.582 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.582 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-09-19T07:06:51.582 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-09-19T07:06:51.582 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.582 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.582 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.582 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-09-19T07:06:51.582 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.582 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.583 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.583 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.583 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.583 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.583 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.583 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.583 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-09-19T07:06:51.583 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-09-19T07:06:51.583 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.583 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.583 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.583 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.583 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.583 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.584 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.584 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.584 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.584 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.584 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.584 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.584 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-09-19T07:06:51.584 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-09-19T07:06:51.584 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.584 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.584 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.584 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.584 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.584 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.585 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.585 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.585 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.585 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.585 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.585 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.585 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-09-19T07:06:51.585 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-09-19T07:06:51.585 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:06:51.585 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.585 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.585 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.585 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.585 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.586 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.586 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.586 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.586 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.586 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.586 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.586 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-09-19T07:06:51.586 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-09-19T07:06:51.586 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.586 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.586 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.586 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.586 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.586 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.586 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.587 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.587 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.587 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.587 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.587 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-09-19T07:06:51.587 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-09-19T07:06:51.587 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.587 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.587 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.587 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.587 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.587 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.588 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.588 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.588 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.588 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.588 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.588 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-09-19T07:06:51.588 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-09-19T07:06:51.588 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.588 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.588 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.588 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.588 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.588 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.589 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.589 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.589 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.589 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.589 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.589 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-09-19T07:06:51.589 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-09-19T07:06:51.589 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.589 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.589 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.589 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.589 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:06:51.590 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-09-19T07:06:51.590 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.590 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.590 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.590 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.590 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.590 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.590 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-09-19T07:06:51.590 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-09-19T07:06:51.590 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.590 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.590 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.590 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.590 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.591 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.591 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.591 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.591 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.591 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.591 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.591 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.591 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-09-19T07:06:51.591 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-09-19T07:06:51.591 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.591 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.591 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.591 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.591 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.591 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.592 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.592 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.592 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.592 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.592 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.592 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.592 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-09-19T07:06:51.592 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-09-19T07:06:51.592 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.592 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.592 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.592 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.592 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.593 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.593 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.593 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.593 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.593 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.593 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.593 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.593 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.593 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.593 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.593 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-19T07:06:51.593 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.593 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.593 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.594 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-09-19T07:06:51.594 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-09-19T07:06:51.594 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.594 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.594 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.594 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.594 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.594 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.594 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.594 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T07:06:51.594 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.594 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.594 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.595 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-09-19T07:06:51.595 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-09-19T07:06:51.595 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.595 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.595 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.595 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.595 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.595 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.595 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.595 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-09-19T07:06:51.595 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.595 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.595 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.595 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.595 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.596 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.596 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.596 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.596 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.596 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.596 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.596 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.596 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.596 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.596 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.596 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.596 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.596 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.597 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.597 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.597 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.597 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.597 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.597 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.597 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.597 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.597 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.597 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.597 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.597 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.597 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.598 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.598 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.598 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.598 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.598 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.598 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.598 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.598 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.598 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.598 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.598 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.598 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.598 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.599 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.599 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.599 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.599 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.599 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.599 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.599 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.599 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.599 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.599 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.599 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.599 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.599 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.599 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.600 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.600 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.600 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.600 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.600 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.600 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.600 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.600 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.600 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.600 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.600 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.600 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.600 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.600 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.601 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.601 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-09-19T07:06:51.601 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.601 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.601 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.601 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.601 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.601 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.601 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.601 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.601 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.602 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.602 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.602 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.602 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.602 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.602 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.602 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.602 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.602 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.602 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.602 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.602 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.602 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.602 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.603 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.603 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.603 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.603 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.603 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.603 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.603 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.603 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.603 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.603 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.603 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.603 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.603 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.603 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.604 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.604 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.604 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.604 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.604 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.604 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.604 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.604 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.604 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.604 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.604 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.604 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.604 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.605 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.605 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.605 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.605 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.605 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.605 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.605 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.605 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.605 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.605 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.605 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.605 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.606 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.606 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.606 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.606 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.606 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.606 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.606 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.606 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.606 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.606 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.606 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-09-19T07:06:51.606 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.606 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.607 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.607 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.607 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.607 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.607 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.607 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.607 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.607 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.607 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.607 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.607 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.607 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.607 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.607 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.607 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.608 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.608 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.608 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.608 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.608 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.608 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.608 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.608 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.608 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.608 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.608 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.608 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.608 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.608 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.609 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.609 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.609 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.609 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.609 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.609 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.609 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.609 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.609 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.609 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.609 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.609 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.609 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.609 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.609 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.610 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.610 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.610 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.610 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.610 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.610 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.610 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.610 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.610 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.610 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.610 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.610 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.610 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.611 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.611 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.611 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.611 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.611 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.611 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.611 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.611 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.611 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.611 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.611 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.611 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.611 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.611 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.612 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.612 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-09-19T07:06:51.612 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.612 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.612 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.612 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-09-19T07:06:51.612 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-09-19T07:06:51.612 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.612 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.612 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.612 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T07:06:51.612 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.612 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.613 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.613 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.613 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.613 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.613 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.613 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:06:51.613 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:06:51.613 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.613 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.613 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.613 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-09-19T07:06:51.613 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.613 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.614 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.614 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.614 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.614 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.614 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.614 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.614 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.614 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.614 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.614 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.614 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.614 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.614 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.614 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.615 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.615 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.615 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.615 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.615 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.615 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.615 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.615 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.615 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.615 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.615 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.615 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.615 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.615 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.616 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.616 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.616 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.616 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.616 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.616 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.616 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.616 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.616 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.616 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.616 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.616 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.616 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.617 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.617 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.617 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.617 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.617 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.617 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.617 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.617 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.617 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.617 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.617 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.617 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.617 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.617 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.617 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.618 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.618 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.618 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.618 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.618 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.618 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.618 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.618 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.618 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.618 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.618 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.618 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.618 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.619 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.619 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.619 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.619 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.619 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.619 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.619 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-09-19T07:06:51.619 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.619 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.619 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.619 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-09-19T07:06:51.619 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-09-19T07:06:51.620 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.620 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.620 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.620 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.620 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.620 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.620 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.620 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T07:06:51.620 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.620 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.620 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.620 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-09-19T07:06:51.621 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-09-19T07:06:51.621 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.621 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.621 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.621 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.621 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.621 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.621 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.621 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T07:06:51.621 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.621 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.621 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.621 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-09-19T07:06:51.622 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-09-19T07:06:51.622 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.622 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.622 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.622 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.622 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.622 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.622 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.622 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T07:06:51.622 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.622 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.622 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.623 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-09-19T07:06:51.623 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-09-19T07:06:51.623 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.623 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.623 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.623 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.623 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.623 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.623 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.623 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T07:06:51.623 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.623 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.623 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.624 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-09-19T07:06:51.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-09-19T07:06:51.624 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.624 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.624 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.624 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.624 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.624 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.624 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.624 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T07:06:51.624 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.624 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.624 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.625 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-09-19T07:06:51.625 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-09-19T07:06:51.625 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.625 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.625 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.625 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.625 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.625 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.625 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.625 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.625 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.625 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.625 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.626 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-09-19T07:06:51.626 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-09-19T07:06:51.626 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.626 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.626 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.626 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.626 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.626 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.626 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.626 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.626 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.626 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.626 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.627 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-09-19T07:06:51.627 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-09-19T07:06:51.627 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.627 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.627 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.627 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T07:06:51.627 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.627 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.627 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.627 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.627 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.627 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.627 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.628 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-19T07:06:51.628 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-19T07:06:51.628 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.628 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.628 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.628 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.628 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.628 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.628 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.628 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.628 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.628 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.629 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.629 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:06:51.629 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:06:51.629 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.629 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.629 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.629 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-09-19T07:06:51.629 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-09-19T07:06:51.629 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.629 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.629 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.630 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.630 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.630 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.630 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-09-19T07:06:51.630 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-09-19T07:06:51.630 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.630 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.630 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.630 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-09-19T07:06:51.630 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.630 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.630 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.630 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.631 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.631 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.631 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.631 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-09-19T07:06:51.631 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-09-19T07:06:51.631 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.631 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.631 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.631 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.631 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.631 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.631 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.632 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.632 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.632 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.632 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.632 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-09-19T07:06:51.632 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-09-19T07:06:51.632 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.632 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.632 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.632 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.632 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.632 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.633 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.633 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.633 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.633 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.633 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.633 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.633 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.633 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.633 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.633 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.633 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.633 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.634 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.634 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.634 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.634 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.634 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.634 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.634 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.634 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.634 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.634 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.634 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.634 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.635 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.635 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.635 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.635 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.635 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.635 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.635 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.635 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.635 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.635 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.635 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.635 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.635 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.636 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.636 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.636 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.636 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.636 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.636 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.636 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.636 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.636 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.636 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.636 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.636 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.637 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.637 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.637 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.637 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.637 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.637 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.637 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.637 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.637 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.637 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.637 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.637 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.638 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.638 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.638 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.638 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.638 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.638 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.638 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.638 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.638 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.638 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.638 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-09-19T07:06:51.638 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-09-19T07:06:51.639 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.639 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.639 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.639 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.639 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.639 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.639 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.639 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.639 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.639 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.639 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.639 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-09-19T07:06:51.639 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-09-19T07:06:51.639 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.640 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.640 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.640 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.640 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.640 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.640 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.640 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.640 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.640 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.640 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.640 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-09-19T07:06:51.640 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-09-19T07:06:51.640 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.640 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.641 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.641 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T07:06:51.641 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.641 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.641 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.641 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.641 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.641 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.641 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.641 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.641 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.641 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.641 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-09-19T07:06:51.642 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.642 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.642 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.642 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.642 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.642 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.642 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.642 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.642 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.642 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.642 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.642 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.642 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.642 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.642 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.643 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.643 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.643 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.643 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.643 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.643 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.643 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.643 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.643 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.643 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.643 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.643 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.643 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.644 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.644 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.644 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.644 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.644 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.644 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.644 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.644 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.644 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.644 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.644 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.644 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.644 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.644 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.645 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.645 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.645 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.645 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.645 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.645 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.645 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.645 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.645 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.645 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.645 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.645 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.645 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.646 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.646 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.646 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.646 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.646 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.646 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.646 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.646 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.646 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.646 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.646 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.647 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.647 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.647 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.647 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.647 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.647 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.647 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-19T07:06:51.647 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.647 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.647 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.647 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.647 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.648 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.648 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.648 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.648 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.648 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.648 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.648 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.648 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.648 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.648 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.648 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.648 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.648 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.648 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.649 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.649 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.649 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.649 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.649 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.649 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.649 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.649 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.649 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.649 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.649 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.649 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.650 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.650 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.650 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.650 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.650 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.650 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.650 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.650 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.650 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.650 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.650 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.650 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.650 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.651 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.651 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.651 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.651 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.651 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.651 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.651 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.651 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.651 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.651 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.651 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.651 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.651 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.652 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.652 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.652 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.652 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.652 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.652 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.652 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.652 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.652 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.652 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.652 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.652 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.652 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-09-19T07:06:51.653 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-09-19T07:06:51.653 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.653 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.653 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.653 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-09-19T07:06:51.653 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.653 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.653 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.653 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T07:06:51.653 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.653 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.653 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.653 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-09-19T07:06:51.653 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-09-19T07:06:51.654 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.654 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.654 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.654 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.654 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.654 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.654 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.654 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-09-19T07:06:51.654 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.654 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.654 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.654 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-09-19T07:06:51.654 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-09-19T07:06:51.655 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.655 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.655 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.655 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:06:51.655 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.655 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.655 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.655 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T07:06:51.655 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.655 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.655 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.655 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-09-19T07:06:51.655 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T07:06:51.655 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.656 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.656 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.656 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:06:51.656 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.656 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.656 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.656 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T07:06:51.656 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.656 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.656 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.656 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.656 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.656 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.657 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-09-19T07:06:51.657 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:06:51.657 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:06:51.657 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:06:51.657 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-09-19T07:06:51.657 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-09-19T07:06:51.657 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:06:51.657 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.657 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.657 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-09-19T07:06:51.657 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.657 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.657 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.658 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.658 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.658 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.658 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.658 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-09-19T07:06:51.658 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-09-19T07:06:51.658 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.658 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.658 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.658 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.658 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.658 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.658 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.659 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.659 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.659 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.659 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.659 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:06:51.659 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:06:51.659 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:06:51.659 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.659 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.659 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:06:51.659 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.659 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.659 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.659 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.660 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.660 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.660 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.660 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:06:51.660 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:06:51.660 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.660 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.660 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.660 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.660 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.660 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.660 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.660 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.661 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.661 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.661 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.661 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.661 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.661 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.661 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.661 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.661 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.661 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.661 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:06:51.661 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:06:51.661 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.662 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.662 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.662 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.662 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.662 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.662 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.662 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.662 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.662 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.662 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.662 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:06:51.662 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:06:51.662 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.662 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.663 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.663 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:06:51.663 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.663 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.663 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:06:51.663 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:06:51.663 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:06:51.663 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:06:51.663 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:06:51.663 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:06:51.663 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:06:51.663 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.663 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.664 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.664 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.664 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.664 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:06:51.664 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:06:51.664 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:06:51.664 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.664 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:06:51.664 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:06:51.664 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.664 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.664 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.665 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.665 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.665 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.665 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.665 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-09-19T07:06:51.665 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-09-19T07:06:51.665 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.665 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.665 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.665 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:06:51.665 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.665 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.665 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.665 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.666 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.666 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.666 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.666 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-09-19T07:06:51.666 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-09-19T07:06:51.666 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:06:51.666 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.666 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.666 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-09-19T07:06:51.666 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.666 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.666 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.667 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.667 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.667 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.667 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.667 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-09-19T07:06:51.667 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-09-19T07:06:51.667 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:06:51.667 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:06:51.667 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:06:51.667 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T07:06:51.667 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:06:51.667 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:06:51.667 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:06:51.668 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:06:51.668 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:06:51.668 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:06:51.668 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:06:51.668 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.668 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.668 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.668 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.668 INFO:teuthology.orchestra.run.smithi103.stdout: "services": { 2024-09-19T07:06:51.668 INFO:teuthology.orchestra.run.smithi103.stdout: "dashboard": "https://172.21.15.103:8443/" 2024-09-19T07:06:51.668 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.668 INFO:teuthology.orchestra.run.smithi103.stdout: "always_on_modules": { 2024-09-19T07:06:51.668 INFO:teuthology.orchestra.run.smithi103.stdout: "octopus": [ 2024-09-19T07:06:51.668 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-19T07:06:51.669 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-19T07:06:51.669 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-19T07:06:51.669 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-19T07:06:51.669 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-19T07:06:51.669 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-19T07:06:51.669 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-19T07:06:51.669 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-19T07:06:51.669 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-19T07:06:51.669 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-19T07:06:51.669 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.669 INFO:teuthology.orchestra.run.smithi103.stdout: "pacific": [ 2024-09-19T07:06:51.669 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-19T07:06:51.669 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-19T07:06:51.670 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-19T07:06:51.670 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-19T07:06:51.670 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-19T07:06:51.670 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-19T07:06:51.670 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-19T07:06:51.670 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-19T07:06:51.670 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-19T07:06:51.670 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-19T07:06:51.670 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.670 INFO:teuthology.orchestra.run.smithi103.stdout: "quincy": [ 2024-09-19T07:06:51.670 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-19T07:06:51.670 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-19T07:06:51.670 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-19T07:06:51.670 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-19T07:06:51.671 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-19T07:06:51.671 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-19T07:06:51.671 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-19T07:06:51.671 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-19T07:06:51.671 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-19T07:06:51.671 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-19T07:06:51.671 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.671 INFO:teuthology.orchestra.run.smithi103.stdout: "reef": [ 2024-09-19T07:06:51.671 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-19T07:06:51.671 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-19T07:06:51.671 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-19T07:06:51.671 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-19T07:06:51.671 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-19T07:06:51.672 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-19T07:06:51.672 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-19T07:06:51.672 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-19T07:06:51.672 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-19T07:06:51.672 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-19T07:06:51.672 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:06:51.672 INFO:teuthology.orchestra.run.smithi103.stdout: "squid": [ 2024-09-19T07:06:51.672 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-19T07:06:51.672 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-19T07:06:51.672 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-19T07:06:51.672 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-19T07:06:51.672 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-19T07:06:51.672 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-19T07:06:51.673 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-19T07:06:51.673 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-19T07:06:51.673 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-19T07:06:51.673 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-19T07:06:51.673 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:06:51.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.673 INFO:teuthology.orchestra.run.smithi103.stdout: "last_failure_osd_epoch": 83, 2024-09-19T07:06:51.673 INFO:teuthology.orchestra.run.smithi103.stdout: "active_clients": [ 2024-09-19T07:06:51.673 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.673 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-19T07:06:51.673 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-19T07:06:51.673 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.673 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-19T07:06:51.674 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-19T07:06:51.674 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 1360760822 2024-09-19T07:06:51.674 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.674 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:06:51.674 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.674 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.674 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "libcephsqlite", 2024-09-19T07:06:51.674 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-19T07:06:51.674 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.674 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-19T07:06:51.674 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-19T07:06:51.674 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 2577554490 2024-09-19T07:06:51.675 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.675 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:06:51.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.675 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.675 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-19T07:06:51.675 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-19T07:06:51.675 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.675 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-19T07:06:51.675 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-19T07:06:51.675 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 940951838 2024-09-19T07:06:51.675 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.675 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:06:51.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.676 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.676 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-19T07:06:51.676 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-19T07:06:51.676 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.676 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-19T07:06:51.676 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-19T07:06:51.676 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 2769581276 2024-09-19T07:06:51.676 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.676 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:06:51.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.676 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.676 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-19T07:06:51.676 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-19T07:06:51.677 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.677 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-19T07:06:51.677 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-19T07:06:51.677 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 3318525402 2024-09-19T07:06:51.677 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.677 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:06:51.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:51.677 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.677 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-19T07:06:51.677 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-19T07:06:51.677 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:06:51.677 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-19T07:06:51.677 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-19T07:06:51.678 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 748868586 2024-09-19T07:06:51.678 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.678 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:06:51.678 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:51.678 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:06:51.678 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:06:51.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.263+0000 7f68f247b640 1 -- 172.21.15.103:0/2162546413 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68d40a33c0 msgr2=0x7f68d40a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:51.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.263+0000 7f68f247b640 1 --2- 172.21.15.103:0/2162546413 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68d40a33c0 0x7f68d40a5880 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009b50 tx=0x7f68dc005f90 comp rx=0 tx=0).stop 2024-09-19T07:06:51.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.263+0000 7f68f247b640 1 -- 172.21.15.103:0/2162546413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68ec101320 msgr2=0x7f68ec1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:06:51.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.263+0000 7f68f247b640 1 --2- 172.21.15.103:0/2162546413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68ec101320 0x7f68ec1997a0 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f68e000c890 tx=0x7f68e000cd50 comp rx=0 tx=0).stop 2024-09-19T07:06:51.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.263+0000 7f68f247b640 1 -- 172.21.15.103:0/2162546413 shutdown_connections 2024-09-19T07:06:51.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.263+0000 7f68f247b640 1 --2- 172.21.15.103:0/2162546413 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f68d40a33c0 0x7f68d40a5880 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:51.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.263+0000 7f68f247b640 1 --2- 172.21.15.103:0/2162546413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ec106990 0x7f68ec1a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:51.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.263+0000 7f68f247b640 1 --2- 172.21.15.103:0/2162546413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68ec101320 0x7f68ec1997a0 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:51.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.263+0000 7f68f247b640 1 --2- 172.21.15.103:0/2162546413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68ec100980 0x7f68ec199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:06:51.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.263+0000 7f68f247b640 1 -- 172.21.15.103:0/2162546413 >> 172.21.15.103:0/2162546413 conn(0x7f68ec077a10 msgr2=0x7f68ec103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:06:51.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.263+0000 7f68f247b640 1 -- 172.21.15.103:0/2162546413 shutdown_connections 2024-09-19T07:06:51.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:06:50.263+0000 7f68f247b640 1 -- 172.21.15.103:0/2162546413 wait complete. 2024-09-19T07:06:51.681 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:50 smithi134 ceph-mon[35356]: pgmap v550: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:06:51.681 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:50 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/378255529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:06:51.681 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:50 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/944005657' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:06:51.681 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:50 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1436951034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T07:06:51.681 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:50 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3654606891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T07:06:51.681 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:51 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2162546413' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T07:06:51.681 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:50 smithi174 ceph-mon[35637]: pgmap v550: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:06:51.681 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:50 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/378255529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:06:51.681 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:50 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/944005657' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:06:51.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:50 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1436951034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T07:06:51.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:50 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3654606891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T07:06:51.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:51 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2162546413' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T07:06:51.682 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:50 smithi103 ceph-mon[31158]: pgmap v550: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:06:51.682 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:50 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/378255529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:06:51.682 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:50 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/944005657' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:06:51.682 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:50 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1436951034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T07:06:51.682 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:50 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3654606891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T07:06:51.746 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T07:06:51.746 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T07:06:51.747 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:51 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2162546413' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T07:06:52.007 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:52.129 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:52 smithi103 ceph-mon[31158]: pgmap v551: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T07:06:52.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:52 smithi174 ceph-mon[35637]: pgmap v551: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T07:06:52.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:52 smithi134 ceph-mon[35356]: pgmap v551: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T07:06:52.494 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:06:52.494 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:06:52.494 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1513, 2024-09-19T07:06:52.494 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1499, 2024-09-19T07:06:52.494 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1912752, 2024-09-19T07:06:52.494 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 566780, 2024-09-19T07:06:52.495 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 5, 2024-09-19T07:06:52.495 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-19T07:06:52.495 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.184997915, 2024-09-19T07:06:52.495 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.067144820, 2024-09-19T07:06:52.495 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.076739023, 2024-09-19T07:06:52.495 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.040158091, 2024-09-19T07:06:52.495 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:06:52.495 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1499, 2024-09-19T07:06:52.495 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.082785414, 2024-09-19T07:06:52.495 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000055227 2024-09-19T07:06:52.495 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.495 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:06:52.495 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.495 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.496 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.496 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.496 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1525008, 2024-09-19T07:06:52.496 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 199808 2024-09-19T07:06:52.496 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.496 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:06:52.496 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1061, 2024-09-19T07:06:52.496 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1300, 2024-09-19T07:06:52.496 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 261486, 2024-09-19T07:06:52.496 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 928366, 2024-09-19T07:06:52.496 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 8, 2024-09-19T07:06:52.496 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 5, 2024-09-19T07:06:52.496 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.120412927, 2024-09-19T07:06:52.496 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.040423915, 2024-09-19T07:06:52.497 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.028256283, 2024-09-19T07:06:52.497 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.041060965, 2024-09-19T07:06:52.497 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:06:52.497 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1297, 2024-09-19T07:06:52.497 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.047147825, 2024-09-19T07:06:52.497 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000036351 2024-09-19T07:06:52.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.497 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:06:52.497 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.497 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.497 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.497 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T07:06:52.498 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 591536 2024-09-19T07:06:52.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.498 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:06:52.498 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1161, 2024-09-19T07:06:52.498 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1163, 2024-09-19T07:06:52.498 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 306851, 2024-09-19T07:06:52.498 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 369199, 2024-09-19T07:06:52.498 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-19T07:06:52.498 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-09-19T07:06:52.498 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.100000286, 2024-09-19T07:06:52.498 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.033407656, 2024-09-19T07:06:52.498 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.027430714, 2024-09-19T07:06:52.498 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.029819713, 2024-09-19T07:06:52.498 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:06:52.499 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1161, 2024-09-19T07:06:52.499 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.040270657, 2024-09-19T07:06:52.499 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000034686 2024-09-19T07:06:52.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.499 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:06:52.499 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.499 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.499 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.499 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:06:52.499 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:06:52.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.499 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-19T07:06:52.499 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-09-19T07:06:52.499 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-19T07:06:52.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.500 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-09-19T07:06:52.500 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-19T07:06:52.500 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-19T07:06:52.500 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 5304, 2024-09-19T07:06:52.500 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.153801862, 2024-09-19T07:06:52.500 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000028997 2024-09-19T07:06:52.500 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:52.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.500 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-09-19T07:06:52.500 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-19T07:06:52.500 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-19T07:06:52.500 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.501 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.501 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.501 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:52.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.501 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-09-19T07:06:52.501 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-09-19T07:06:52.501 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-09-19T07:06:52.501 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-09-19T07:06:52.501 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.501 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.501 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.501 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-09-19T07:06:52.501 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-09-19T07:06:52.502 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-09-19T07:06:52.502 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-09-19T07:06:52.502 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-09-19T07:06:52.502 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-09-19T07:06:52.502 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-09-19T07:06:52.502 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 0, 2024-09-19T07:06:52.502 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-09-19T07:06:52.502 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 0, 2024-09-19T07:06:52.502 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-09-19T07:06:52.502 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 0, 2024-09-19T07:06:52.502 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-09-19T07:06:52.502 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-09-19T07:06:52.502 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-09-19T07:06:52.503 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 1, 2024-09-19T07:06:52.503 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-09-19T07:06:52.503 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-09-19T07:06:52.503 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-09-19T07:06:52.503 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-09-19T07:06:52.503 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:06:52.503 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-09-19T07:06:52.503 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-09-19T07:06:52.503 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-09-19T07:06:52.503 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-09-19T07:06:52.503 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-09-19T07:06:52.503 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-09-19T07:06:52.503 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-09-19T07:06:52.503 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-09-19T07:06:52.504 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-09-19T07:06:52.504 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:06:52.504 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-09-19T07:06:52.504 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2024-09-19T07:06:52.504 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2024-09-19T07:06:52.504 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-09-19T07:06:52.504 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:06:52.504 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-09-19T07:06:52.504 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:06:52.504 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:06:52.504 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:06:52.504 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:06:52.504 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:06:52.504 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-09-19T07:06:52.505 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:06:52.505 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T07:06:52.505 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:06:52.505 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:06:52.505 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:06:52.505 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:06:52.505 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-09-19T07:06:52.505 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:06:52.505 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-09-19T07:06:52.505 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-09-19T07:06:52.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.505 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-09-19T07:06:52.505 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-09-19T07:06:52.505 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-09-19T07:06:52.505 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-09-19T07:06:52.506 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-09-19T07:06:52.506 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-09-19T07:06:52.506 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-09-19T07:06:52.506 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-09-19T07:06:52.506 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:06:52.506 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-09-19T07:06:52.506 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-09-19T07:06:52.506 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-09-19T07:06:52.506 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-09-19T07:06:52.506 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-09-19T07:06:52.506 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:06:52.506 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:06:52.506 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-09-19T07:06:52.506 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-09-19T07:06:52.507 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-09-19T07:06:52.507 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-09-19T07:06:52.507 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:06:52.507 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-09-19T07:06:52.507 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-09-19T07:06:52.507 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:06:52.507 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-09-19T07:06:52.507 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-09-19T07:06:52.507 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.507 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-09-19T07:06:52.507 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-09-19T07:06:52.507 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-09-19T07:06:52.507 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-09-19T07:06:52.507 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-09-19T07:06:52.508 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 10, 2024-09-19T07:06:52.508 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-09-19T07:06:52.508 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-09-19T07:06:52.508 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-09-19T07:06:52.508 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-09-19T07:06:52.508 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-09-19T07:06:52.508 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 2, 2024-09-19T07:06:52.508 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-09-19T07:06:52.508 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-09-19T07:06:52.508 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-09-19T07:06:52.508 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-09-19T07:06:52.508 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 4197028, 2024-09-19T07:06:52.508 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 4196887, 2024-09-19T07:06:52.508 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-09-19T07:06:52.509 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.509 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.509 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.509 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 10 2024-09-19T07:06:52.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.509 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-09-19T07:06:52.509 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 3, 2024-09-19T07:06:52.509 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 3, 2024-09-19T07:06:52.509 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 0, 2024-09-19T07:06:52.509 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 1, 2024-09-19T07:06:52.509 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 1, 2024-09-19T07:06:52.509 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 0, 2024-09-19T07:06:52.510 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 0, 2024-09-19T07:06:52.510 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 0, 2024-09-19T07:06:52.510 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 0, 2024-09-19T07:06:52.510 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-09-19T07:06:52.510 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-09-19T07:06:52.510 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-09-19T07:06:52.510 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 48644, 2024-09-19T07:06:52.510 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215328 2024-09-19T07:06:52.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.510 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-09-19T07:06:52.510 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-09-19T07:06:52.510 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-09-19T07:06:52.510 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-09-19T07:06:52.510 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-09-19T07:06:52.511 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-09-19T07:06:52.511 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-09-19T07:06:52.511 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.511 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.511 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.511 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.511 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-09-19T07:06:52.511 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.511 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.511 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.511 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.511 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-09-19T07:06:52.511 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.511 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.512 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.512 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-09-19T07:06:52.512 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.512 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.512 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.512 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-09-19T07:06:52.512 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.512 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.512 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.512 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-09-19T07:06:52.512 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.513 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.513 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.513 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-09-19T07:06:52.513 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.513 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.513 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.513 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-09-19T07:06:52.513 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.513 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.513 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.513 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-09-19T07:06:52.514 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.514 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.514 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.514 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-09-19T07:06:52.514 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.514 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.514 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.514 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-09-19T07:06:52.514 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.514 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.514 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.515 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-09-19T07:06:52.515 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.515 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.515 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.515 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-09-19T07:06:52.515 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.515 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.515 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.515 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-09-19T07:06:52.515 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.515 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.516 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.516 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-09-19T07:06:52.516 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.516 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.516 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.516 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-09-19T07:06:52.516 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.516 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.516 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.516 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-09-19T07:06:52.516 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.517 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.517 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.517 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.517 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-09-19T07:06:52.517 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.517 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.517 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.517 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.517 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-09-19T07:06:52.517 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.517 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.517 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.517 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.517 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-09-19T07:06:52.518 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.518 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.518 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.518 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.518 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-09-19T07:06:52.518 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.518 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.518 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.518 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.518 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-09-19T07:06:52.518 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.518 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.519 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.519 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.519 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-09-19T07:06:52.519 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.519 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.519 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.519 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.519 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-09-19T07:06:52.519 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.519 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.519 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.519 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.519 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-09-19T07:06:52.519 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.519 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.520 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.520 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.520 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-09-19T07:06:52.520 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.520 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.520 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.520 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.520 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-09-19T07:06:52.520 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.520 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.520 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.520 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.520 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-09-19T07:06:52.520 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.521 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.521 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.521 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.521 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-09-19T07:06:52.521 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.521 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.521 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.521 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.521 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:06:52.521 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:06:52.521 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-09-19T07:06:52.521 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.521 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.521 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.522 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:52.522 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.522 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-09-19T07:06:52.522 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 0, 2024-09-19T07:06:52.522 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 4, 2024-09-19T07:06:52.522 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 4, 2024-09-19T07:06:52.522 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 3, 2024-09-19T07:06:52.522 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-09-19T07:06:52.522 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-09-19T07:06:52.522 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-09-19T07:06:52.522 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 1097, 2024-09-19T07:06:52.522 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-09-19T07:06:52.522 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.522 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-19T07:06:52.523 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:06:52.523 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-19T07:06:52.523 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:06:52.523 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:06:52.523 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:06:52.523 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:06:52.523 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:06:52.523 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:06:52.523 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:06:52.523 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:06:52.523 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:06:52.523 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:06:52.523 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:06:52.523 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:06:52.524 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:06:52.524 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-19T07:06:52.524 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:06:52.524 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-19T07:06:52.524 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:06:52.524 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:06:52.524 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:06:52.524 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:06:52.524 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:06:52.524 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:06:52.524 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:06:52.524 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-19T07:06:52.524 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:06:52.525 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:06:52.525 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:06:52.525 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-19T07:06:52.525 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-19T07:06:52.525 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-19T07:06:52.525 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:06:52.525 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:06:52.525 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:06:52.525 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:06:52.525 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 190195, 2024-09-19T07:06:52.525 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 54, 2024-09-19T07:06:52.525 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:06:52.525 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-19T07:06:52.525 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-19T07:06:52.526 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-19T07:06:52.526 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:06:52.526 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-19T07:06:52.526 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:06:52.526 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-19T07:06:52.526 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8192, 2024-09-19T07:06:52.526 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 180, 2024-09-19T07:06:52.526 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:06:52.526 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:06:52.526 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-19T07:06:52.526 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-19T07:06:52.526 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 8512, 2024-09-19T07:06:52.526 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 13, 2024-09-19T07:06:52.526 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-19T07:06:52.527 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-19T07:06:52.527 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-19T07:06:52.527 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-19T07:06:52.527 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.527 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-09-19T07:06:52.527 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-09-19T07:06:52.527 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-09-19T07:06:52.527 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 3374, 2024-09-19T07:06:52.527 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2024-09-19T07:06:52.527 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2024-09-19T07:06:52.527 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 3374, 2024-09-19T07:06:52.527 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-09-19T07:06:52.527 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3374, 2024-09-19T07:06:52.527 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 2.187958922, 2024-09-19T07:06:52.528 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000648476 2024-09-19T07:06:52.528 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.528 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-09-19T07:06:52.528 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-09-19T07:06:52.528 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3374, 2024-09-19T07:06:52.528 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 3375 2024-09-19T07:06:52.528 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.528 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 3374, 2024-09-19T07:06:52.528 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 3374, 2024-09-19T07:06:52.528 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-09-19T07:06:52.528 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-09-19T07:06:52.528 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-09-19T07:06:52.528 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 2210, 2024-09-19T07:06:52.528 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-09-19T07:06:52.529 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 1162, 2024-09-19T07:06:52.529 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-09-19T07:06:52.529 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-09-19T07:06:52.529 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-09-19T07:06:52.529 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-09-19T07:06:52.529 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-09-19T07:06:52.529 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-09-19T07:06:52.529 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-09-19T07:06:52.529 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-09-19T07:06:52.529 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-09-19T07:06:52.529 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-09-19T07:06:52.529 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2024-09-19T07:06:52.529 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-09-19T07:06:52.529 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:06:52.530 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-09-19T07:06:52.530 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:06:52.530 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-09-19T07:06:52.530 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-09-19T07:06:52.530 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-09-19T07:06:52.530 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:06:52.530 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-09-19T07:06:52.530 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:06:52.530 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-09-19T07:06:52.530 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-09-19T07:06:52.530 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-09-19T07:06:52.530 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-09-19T07:06:52.530 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-09-19T07:06:52.530 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-09-19T07:06:52.531 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-09-19T07:06:52.531 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-09-19T07:06:52.531 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-09-19T07:06:52.531 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-09-19T07:06:52.531 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-09-19T07:06:52.531 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-09-19T07:06:52.531 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-09-19T07:06:52.531 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-09-19T07:06:52.531 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-09-19T07:06:52.531 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-09-19T07:06:52.531 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-09-19T07:06:52.531 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 103, 2024-09-19T07:06:52.531 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-09-19T07:06:52.532 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 23, 2024-09-19T07:06:52.532 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 5, 2024-09-19T07:06:52.532 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 6, 2024-09-19T07:06:52.532 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 1, 2024-09-19T07:06:52.532 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-09-19T07:06:52.532 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-09-19T07:06:52.532 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-09-19T07:06:52.532 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-09-19T07:06:52.532 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.532 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-09-19T07:06:52.532 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-09-19T07:06:52.532 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:06:52.532 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-09-19T07:06:52.532 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-09-19T07:06:52.533 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.533 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-09-19T07:06:52.533 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-09-19T07:06:52.533 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.533 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-09-19T07:06:52.533 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-09-19T07:06:52.533 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:06:52.533 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-09-19T07:06:52.533 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-09-19T07:06:52.533 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-09-19T07:06:52.533 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-09-19T07:06:52.533 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-09-19T07:06:52.533 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.533 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:06:52.534 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:06:52.534 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T07:06:52.534 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:06:52.534 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 3731, 2024-09-19T07:06:52.534 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 2124108, 2024-09-19T07:06:52.534 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:52.534 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 3731, 2024-09-19T07:06:52.534 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T07:06:52.534 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T07:06:52.534 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3731, 2024-09-19T07:06:52.534 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2124108, 2024-09-19T07:06:52.534 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:06:52.534 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.534 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.535 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.535 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:52.535 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.535 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-09-19T07:06:52.535 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:06:52.535 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T07:06:52.535 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:06:52.535 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T07:06:52.535 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T07:06:52.535 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:52.535 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T07:06:52.535 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 3374, 2024-09-19T07:06:52.535 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 2729, 2024-09-19T07:06:52.535 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3374, 2024-09-19T07:06:52.536 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2729, 2024-09-19T07:06:52.536 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:06:52.536 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.536 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.536 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.536 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:52.536 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.536 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-09-19T07:06:52.536 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:06:52.536 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-09-19T07:06:52.536 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:06:52.536 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T07:06:52.536 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T07:06:52.536 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:52.537 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T07:06:52.537 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 3374, 2024-09-19T07:06:52.537 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 3374, 2024-09-19T07:06:52.537 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3374, 2024-09-19T07:06:52.537 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 3374, 2024-09-19T07:06:52.537 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:06:52.537 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.537 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.537 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.537 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:52.537 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.537 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:06:52.537 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:06:52.537 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-19T07:06:52.538 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:06:52.538 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T07:06:52.538 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T07:06:52.538 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:52.538 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T07:06:52.538 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T07:06:52.538 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T07:06:52.538 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-19T07:06:52.538 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-19T07:06:52.538 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:06:52.538 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.538 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.538 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.538 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:52.539 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:52.539 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x55ae8f2ccb40": { 2024-09-19T07:06:52.539 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:06:52.539 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-19T07:06:52.539 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:06:52.539 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T07:06:52.539 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T07:06:52.539 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:52.539 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T07:06:52.539 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T07:06:52.539 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T07:06:52.539 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-19T07:06:52.539 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-19T07:06:52.539 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:06:52.540 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:52.540 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:52.540 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:52.540 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:52.540 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:52.540 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:06:53.271 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-19T07:06:53.271 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-19T07:06:53.272 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-19T07:06:53.272 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:06:53.272 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:06:53.272 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T07:06:53.272 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T07:06:53.519 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:53.972 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:06:53.972 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:06:53.972 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 330, 2024-09-19T07:06:53.973 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 320, 2024-09-19T07:06:53.973 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1479424, 2024-09-19T07:06:53.973 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 197680, 2024-09-19T07:06:53.973 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-19T07:06:53.973 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-09-19T07:06:53.973 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.052380826, 2024-09-19T07:06:53.973 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.023502060, 2024-09-19T07:06:53.973 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.039120497, 2024-09-19T07:06:53.973 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:06:53.973 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:06:53.973 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 319, 2024-09-19T07:06:53.973 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.018884949, 2024-09-19T07:06:53.973 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000059200 2024-09-19T07:06:53.973 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:53.974 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:06:53.974 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:53.974 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:53.974 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:53.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:53.974 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1479424, 2024-09-19T07:06:53.974 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 197680 2024-09-19T07:06:53.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:53.974 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:06:53.974 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 3, 2024-09-19T07:06:53.974 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 245, 2024-09-19T07:06:53.974 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 480, 2024-09-19T07:06:53.974 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 279280, 2024-09-19T07:06:53.974 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-09-19T07:06:53.975 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-09-19T07:06:53.975 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.018991089, 2024-09-19T07:06:53.975 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.013900739, 2024-09-19T07:06:53.975 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.002078849, 2024-09-19T07:06:53.975 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:06:53.975 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:06:53.975 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 242, 2024-09-19T07:06:53.975 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.024046388, 2024-09-19T07:06:53.975 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000099365 2024-09-19T07:06:53.975 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:53.975 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:06:53.975 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:53.975 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:53.976 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:53.976 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:53.976 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-19T07:06:53.976 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 279280 2024-09-19T07:06:53.976 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:53.976 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:06:53.976 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 4, 2024-09-19T07:06:53.976 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 4, 2024-09-19T07:06:53.976 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 3024, 2024-09-19T07:06:53.976 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 448, 2024-09-19T07:06:53.976 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-09-19T07:06:53.976 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-19T07:06:53.976 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.002352990, 2024-09-19T07:06:53.976 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000030000, 2024-09-19T07:06:53.977 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.002286098, 2024-09-19T07:06:53.977 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:06:53.977 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:06:53.977 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3, 2024-09-19T07:06:53.977 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.004238859, 2024-09-19T07:06:53.977 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001412953 2024-09-19T07:06:53.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:53.977 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:06:53.977 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:53.977 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:53.977 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:53.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:53.977 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T07:06:53.977 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T07:06:53.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:53.978 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-19T07:06:53.978 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-09-19T07:06:53.978 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-19T07:06:53.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:53.978 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-19T07:06:53.978 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:06:53.978 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-19T07:06:53.978 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:06:53.978 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:06:53.978 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:06:53.978 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:06:53.978 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:06:53.978 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:06:53.978 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:06:53.979 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:06:53.979 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:06:53.979 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:06:53.979 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:06:53.979 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:06:53.979 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:06:53.979 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-19T07:06:53.979 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:06:53.980 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-19T07:06:53.980 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:06:53.980 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:06:53.980 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:06:53.980 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:06:53.980 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:06:53.980 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:06:53.980 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:06:53.980 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-19T07:06:53.981 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:06:53.981 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:06:53.981 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:06:53.981 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-19T07:06:53.981 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-19T07:06:53.981 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-19T07:06:53.981 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:06:53.981 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:06:53.981 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:06:53.981 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:06:53.982 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 148670, 2024-09-19T07:06:53.982 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 30, 2024-09-19T07:06:53.982 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:06:53.982 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-19T07:06:53.982 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-19T07:06:53.982 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-19T07:06:53.982 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:06:53.982 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-19T07:06:53.982 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:06:53.982 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-19T07:06:53.982 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-09-19T07:06:53.982 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-09-19T07:06:53.982 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:06:53.982 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:06:53.983 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-19T07:06:53.983 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-19T07:06:53.983 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 928, 2024-09-19T07:06:53.983 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-09-19T07:06:53.983 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-19T07:06:53.983 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-19T07:06:53.983 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-19T07:06:53.983 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-19T07:06:53.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:53.983 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:06:53.983 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:06:53.983 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T07:06:53.983 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:06:53.984 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 333, 2024-09-19T07:06:53.984 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1432855, 2024-09-19T07:06:53.984 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:53.984 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 333, 2024-09-19T07:06:53.984 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T07:06:53.984 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T07:06:53.984 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 333, 2024-09-19T07:06:53.984 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1432855, 2024-09-19T07:06:53.984 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:06:53.984 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:53.984 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:53.984 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:53.984 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:53.984 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:53.985 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:06:54.316 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:54 smithi103 ceph-mon[31158]: pgmap v552: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:06:54.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:54 smithi174 ceph-mon[35637]: pgmap v552: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:06:54.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:54 smithi134 ceph-mon[35356]: pgmap v552: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:06:54.702 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-19T07:06:54.702 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-19T07:06:54.702 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-19T07:06:54.703 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-19T07:06:54.703 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-19T07:06:54.703 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T07:06:54.703 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T07:06:54.950 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:55.438 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:06:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:06:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1514, 2024-09-19T07:06:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1501, 2024-09-19T07:06:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1911426, 2024-09-19T07:06:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 567926, 2024-09-19T07:06:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 5, 2024-09-19T07:06:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-19T07:06:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.174694737, 2024-09-19T07:06:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.078415308, 2024-09-19T07:06:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.065808852, 2024-09-19T07:06:55.439 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.030257784, 2024-09-19T07:06:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:06:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1500, 2024-09-19T07:06:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.081681230, 2024-09-19T07:06:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000054454 2024-09-19T07:06:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:06:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1522256, 2024-09-19T07:06:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 200000 2024-09-19T07:06:55.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:06:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2227, 2024-09-19T07:06:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2466, 2024-09-19T07:06:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 569408, 2024-09-19T07:06:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 1300287, 2024-09-19T07:06:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 7, 2024-09-19T07:06:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-09-19T07:06:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.147820253, 2024-09-19T07:06:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.049252185, 2024-09-19T07:06:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.038055754, 2024-09-19T07:06:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.048849332, 2024-09-19T07:06:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:06:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2464, 2024-09-19T07:06:55.441 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.091305843, 2024-09-19T07:06:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000037055 2024-09-19T07:06:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:06:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-19T07:06:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 593952 2024-09-19T07:06:55.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:06:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 4, 2024-09-19T07:06:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 6, 2024-09-19T07:06:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1629, 2024-09-19T07:06:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 1284, 2024-09-19T07:06:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 8, 2024-09-19T07:06:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-09-19T07:06:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.002188106, 2024-09-19T07:06:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000057049, 2024-09-19T07:06:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.001828390, 2024-09-19T07:06:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000074583, 2024-09-19T07:06:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:06:55.443 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4, 2024-09-19T07:06:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.002031377, 2024-09-19T07:06:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000507844 2024-09-19T07:06:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:06:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:06:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:06:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-19T07:06:55.444 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-09-19T07:06:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-19T07:06:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-09-19T07:06:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-19T07:06:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-19T07:06:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 5650, 2024-09-19T07:06:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.122473452, 2024-09-19T07:06:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000021676 2024-09-19T07:06:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.445 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-09-19T07:06:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-19T07:06:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-19T07:06:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-09-19T07:06:55.446 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-09-19T07:06:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-09-19T07:06:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-09-19T07:06:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-09-19T07:06:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-09-19T07:06:55.447 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-09-19T07:06:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-09-19T07:06:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-09-19T07:06:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-09-19T07:06:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-09-19T07:06:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 0, 2024-09-19T07:06:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-09-19T07:06:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 0, 2024-09-19T07:06:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-09-19T07:06:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 0, 2024-09-19T07:06:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-09-19T07:06:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-09-19T07:06:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-09-19T07:06:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 1, 2024-09-19T07:06:55.448 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-09-19T07:06:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-09-19T07:06:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-09-19T07:06:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-09-19T07:06:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:06:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-09-19T07:06:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-09-19T07:06:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-09-19T07:06:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-09-19T07:06:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-09-19T07:06:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-09-19T07:06:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-09-19T07:06:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-09-19T07:06:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-09-19T07:06:55.449 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:06:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-09-19T07:06:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2024-09-19T07:06:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2024-09-19T07:06:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-09-19T07:06:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:06:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-09-19T07:06:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:06:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:06:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:06:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:06:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:06:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-09-19T07:06:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:06:55.450 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T07:06:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:06:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:06:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:06:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:06:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-09-19T07:06:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:06:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-09-19T07:06:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-09-19T07:06:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-09-19T07:06:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-09-19T07:06:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-09-19T07:06:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-09-19T07:06:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-09-19T07:06:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-09-19T07:06:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-09-19T07:06:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-09-19T07:06:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:06:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-09-19T07:06:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-09-19T07:06:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-09-19T07:06:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-09-19T07:06:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-09-19T07:06:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:06:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:06:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-09-19T07:06:55.452 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-09-19T07:06:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-09-19T07:06:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-09-19T07:06:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:06:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-09-19T07:06:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-09-19T07:06:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:06:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-09-19T07:06:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-09-19T07:06:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-09-19T07:06:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-09-19T07:06:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-09-19T07:06:55.453 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-09-19T07:06:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-09-19T07:06:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 4, 2024-09-19T07:06:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-09-19T07:06:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-09-19T07:06:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-09-19T07:06:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-09-19T07:06:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-09-19T07:06:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 2, 2024-09-19T07:06:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-09-19T07:06:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-09-19T07:06:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-09-19T07:06:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-09-19T07:06:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 4195426, 2024-09-19T07:06:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 4195285, 2024-09-19T07:06:55.454 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-09-19T07:06:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 4 2024-09-19T07:06:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-09-19T07:06:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 3, 2024-09-19T07:06:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 3, 2024-09-19T07:06:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 0, 2024-09-19T07:06:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 1, 2024-09-19T07:06:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 1, 2024-09-19T07:06:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 0, 2024-09-19T07:06:55.455 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 0, 2024-09-19T07:06:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 0, 2024-09-19T07:06:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 0, 2024-09-19T07:06:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-09-19T07:06:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-09-19T07:06:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-09-19T07:06:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 48880, 2024-09-19T07:06:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215328 2024-09-19T07:06:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-09-19T07:06:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-09-19T07:06:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-09-19T07:06:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-09-19T07:06:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-09-19T07:06:55.456 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-09-19T07:06:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-09-19T07:06:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-09-19T07:06:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-09-19T07:06:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.457 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-09-19T07:06:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-09-19T07:06:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-09-19T07:06:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.458 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-09-19T07:06:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-09-19T07:06:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.459 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-09-19T07:06:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-09-19T07:06:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.460 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-09-19T07:06:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.461 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-09-19T07:06:55.462 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-09-19T07:06:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-09-19T07:06:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-09-19T07:06:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-09-19T07:06:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-09-19T07:06:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-09-19T07:06:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.465 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-09-19T07:06:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-09-19T07:06:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-09-19T07:06:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.466 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-09-19T07:06:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-09-19T07:06:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-09-19T07:06:55.467 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-09-19T07:06:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-09-19T07:06:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-09-19T07:06:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-09-19T07:06:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-09-19T07:06:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.469 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:06:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:06:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-09-19T07:06:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-09-19T07:06:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 0, 2024-09-19T07:06:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 1, 2024-09-19T07:06:55.470 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 1, 2024-09-19T07:06:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 1, 2024-09-19T07:06:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-09-19T07:06:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-09-19T07:06:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-09-19T07:06:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 1099, 2024-09-19T07:06:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-09-19T07:06:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-19T07:06:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:06:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-19T07:06:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:06:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:06:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:06:55.471 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:06:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:06:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:06:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:06:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:06:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:06:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:06:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:06:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:06:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:06:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-19T07:06:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:06:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-19T07:06:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:06:55.472 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:06:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:06:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:06:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:06:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:06:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:06:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-19T07:06:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:06:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:06:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:06:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-19T07:06:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-19T07:06:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-19T07:06:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:06:55.473 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:06:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:06:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:06:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 198183, 2024-09-19T07:06:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 54, 2024-09-19T07:06:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:06:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-19T07:06:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-19T07:06:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-19T07:06:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:06:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-19T07:06:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:06:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-19T07:06:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8192, 2024-09-19T07:06:55.474 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 180, 2024-09-19T07:06:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:06:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:06:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-19T07:06:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-19T07:06:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 8512, 2024-09-19T07:06:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 13, 2024-09-19T07:06:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-19T07:06:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-19T07:06:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-19T07:06:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-19T07:06:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-09-19T07:06:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-09-19T07:06:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-09-19T07:06:55.475 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 3382, 2024-09-19T07:06:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2024-09-19T07:06:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2024-09-19T07:06:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 3382, 2024-09-19T07:06:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-09-19T07:06:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3382, 2024-09-19T07:06:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 2.470953577, 2024-09-19T07:06:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000730619 2024-09-19T07:06:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-09-19T07:06:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-09-19T07:06:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3382, 2024-09-19T07:06:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 3383 2024-09-19T07:06:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.476 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 3382, 2024-09-19T07:06:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 3382, 2024-09-19T07:06:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-09-19T07:06:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-09-19T07:06:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-09-19T07:06:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 2217, 2024-09-19T07:06:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-09-19T07:06:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 1163, 2024-09-19T07:06:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-09-19T07:06:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-09-19T07:06:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-09-19T07:06:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-09-19T07:06:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-09-19T07:06:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-09-19T07:06:55.477 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-09-19T07:06:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-09-19T07:06:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-09-19T07:06:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-09-19T07:06:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2024-09-19T07:06:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-09-19T07:06:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:06:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-09-19T07:06:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:06:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-09-19T07:06:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-09-19T07:06:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-09-19T07:06:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:06:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-09-19T07:06:55.478 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:06:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-09-19T07:06:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-09-19T07:06:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-09-19T07:06:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-09-19T07:06:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-09-19T07:06:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-09-19T07:06:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-09-19T07:06:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-09-19T07:06:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-09-19T07:06:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-09-19T07:06:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-09-19T07:06:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-09-19T07:06:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-09-19T07:06:55.479 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-09-19T07:06:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-09-19T07:06:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-09-19T07:06:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-09-19T07:06:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 103, 2024-09-19T07:06:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-09-19T07:06:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 23, 2024-09-19T07:06:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 6, 2024-09-19T07:06:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 9, 2024-09-19T07:06:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 3, 2024-09-19T07:06:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-09-19T07:06:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-09-19T07:06:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-09-19T07:06:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-09-19T07:06:55.480 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-09-19T07:06:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-09-19T07:06:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:06:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-09-19T07:06:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-09-19T07:06:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-09-19T07:06:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-09-19T07:06:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-09-19T07:06:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-09-19T07:06:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:06:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-09-19T07:06:55.481 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-09-19T07:06:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-09-19T07:06:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-09-19T07:06:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-09-19T07:06:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:06:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:06:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T07:06:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:06:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 3741, 2024-09-19T07:06:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 2124530, 2024-09-19T07:06:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 3741, 2024-09-19T07:06:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T07:06:55.482 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T07:06:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3741, 2024-09-19T07:06:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2124530, 2024-09-19T07:06:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:06:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-09-19T07:06:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:06:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T07:06:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:06:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T07:06:55.483 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T07:06:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T07:06:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 3382, 2024-09-19T07:06:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 1127, 2024-09-19T07:06:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3382, 2024-09-19T07:06:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1127, 2024-09-19T07:06:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:06:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.484 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-09-19T07:06:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:06:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-09-19T07:06:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:06:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T07:06:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T07:06:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T07:06:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 3382, 2024-09-19T07:06:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 3382, 2024-09-19T07:06:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3382, 2024-09-19T07:06:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 3382, 2024-09-19T07:06:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:06:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.485 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:06:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:06:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-19T07:06:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:06:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T07:06:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T07:06:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T07:06:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T07:06:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T07:06:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-19T07:06:55.486 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-19T07:06:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:06:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x562e536c8b40": { 2024-09-19T07:06:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:06:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-19T07:06:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:06:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T07:06:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T07:06:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:55.487 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T07:06:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T07:06:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T07:06:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-19T07:06:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-19T07:06:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:06:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:55.488 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:55.488 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:06:56.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:56 smithi103 ceph-mon[31158]: pgmap v553: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:06:56.238 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-19T07:06:56.239 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-19T07:06:56.239 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-19T07:06:56.239 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:06:56.239 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:06:56.239 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T07:06:56.239 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T07:06:56.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:56 smithi174 ceph-mon[35637]: pgmap v553: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:06:56.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:56 smithi134 ceph-mon[35356]: pgmap v553: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:06:56.485 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:06:56.950 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:06:56.950 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:06:56.950 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 0, 2024-09-19T07:06:56.950 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2, 2024-09-19T07:06:56.950 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 0, 2024-09-19T07:06:56.950 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 0, 2024-09-19T07:06:56.950 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-19T07:06:56.950 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-09-19T07:06:56.950 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.001119196, 2024-09-19T07:06:56.950 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T07:06:56.950 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.000240978, 2024-09-19T07:06:56.951 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:06:56.951 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:06:56.951 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:56.951 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:56.951 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:56.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:56.951 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:06:56.951 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:56.951 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:56.951 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:56.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:56.951 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:06:56.951 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:06:56.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:56.952 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:06:56.952 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 331, 2024-09-19T07:06:56.952 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 321, 2024-09-19T07:06:56.952 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1479696, 2024-09-19T07:06:56.952 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 198320, 2024-09-19T07:06:56.952 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-09-19T07:06:56.952 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-19T07:06:56.952 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.051821666, 2024-09-19T07:06:56.952 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.021784395, 2024-09-19T07:06:56.952 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.041412057, 2024-09-19T07:06:56.952 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:06:56.953 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:06:56.953 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 320, 2024-09-19T07:06:56.953 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.017098973, 2024-09-19T07:06:56.953 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000053434 2024-09-19T07:06:56.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:56.953 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:06:56.953 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:56.953 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:56.953 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:56.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:56.953 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1479696, 2024-09-19T07:06:56.953 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 198320 2024-09-19T07:06:56.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:56.954 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:06:56.954 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 6, 2024-09-19T07:06:56.954 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 248, 2024-09-19T07:06:56.954 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 3344, 2024-09-19T07:06:56.954 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 201008, 2024-09-19T07:06:56.954 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 5, 2024-09-19T07:06:56.954 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-09-19T07:06:56.955 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.021659558, 2024-09-19T07:06:56.955 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.014569212, 2024-09-19T07:06:56.955 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.004185227, 2024-09-19T07:06:56.955 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:06:56.955 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:06:56.955 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 246, 2024-09-19T07:06:56.955 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.024714101, 2024-09-19T07:06:56.955 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000100463 2024-09-19T07:06:56.955 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:56.955 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:06:56.955 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:56.955 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:56.955 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:56.955 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:56.956 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-19T07:06:56.956 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 201008 2024-09-19T07:06:56.956 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:56.956 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-19T07:06:56.956 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-09-19T07:06:56.956 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-19T07:06:56.956 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:56.956 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-19T07:06:56.956 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:06:56.956 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-19T07:06:56.956 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:06:56.956 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:06:56.956 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:06:56.956 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:06:56.957 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:06:56.957 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:06:56.957 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:06:56.957 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:06:56.957 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:06:56.957 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:06:56.957 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:06:56.957 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:06:56.957 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:06:56.957 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-19T07:06:56.958 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:06:56.958 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-19T07:06:56.958 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:06:56.958 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:06:56.958 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:06:56.958 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:06:56.958 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:06:56.958 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:06:56.958 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:06:56.958 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-19T07:06:56.958 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:06:56.958 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:06:56.958 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:06:56.958 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-19T07:06:56.959 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-19T07:06:56.959 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-19T07:06:56.959 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:06:56.959 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:06:56.959 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:06:56.959 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:06:56.959 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 150025, 2024-09-19T07:06:56.959 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 32, 2024-09-19T07:06:56.959 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:06:56.959 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-19T07:06:56.959 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-19T07:06:56.959 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-19T07:06:56.959 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:06:56.959 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-19T07:06:56.960 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:06:56.960 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-19T07:06:56.960 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-09-19T07:06:56.960 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-09-19T07:06:56.960 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:06:56.960 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:06:56.960 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-19T07:06:56.960 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-19T07:06:56.960 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 928, 2024-09-19T07:06:56.960 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-09-19T07:06:56.960 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-19T07:06:56.960 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-19T07:06:56.960 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-19T07:06:56.960 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-19T07:06:56.961 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:06:56.961 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:06:56.961 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:06:56.961 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T07:06:56.961 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:06:56.961 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 333, 2024-09-19T07:06:56.961 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1432958, 2024-09-19T07:06:56.961 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:56.961 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 333, 2024-09-19T07:06:56.961 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T07:06:56.961 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T07:06:56.961 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 333, 2024-09-19T07:06:56.961 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1432958, 2024-09-19T07:06:56.961 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:06:56.962 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:06:56.962 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:06:56.962 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:06:56.962 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:56.962 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:06:56.962 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:06:57.630 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T07:06:57.630 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T07:06:57.630 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-19T07:06:57.630 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-19T07:06:57.630 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-19T07:06:57.631 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T07:06:57.631 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T07:06:57.888 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:06:58.315 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:06:58 smithi134 ceph-mon[35356]: pgmap v554: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T07:06:58.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:06:58 smithi174 ceph-mon[35637]: pgmap v554: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T07:06:58.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:06:58 smithi103 ceph-mon[31158]: pgmap v554: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T07:06:58.422 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-19T07:06:58.423 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:06:58.423 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 5209, 2024-09-19T07:06:58.423 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 7900, 2024-09-19T07:06:58.423 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1818373, 2024-09-19T07:06:58.423 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 16465610, 2024-09-19T07:06:58.423 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 38, 2024-09-19T07:06:58.423 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-09-19T07:06:58.423 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.557237741, 2024-09-19T07:06:58.423 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.230025405, 2024-09-19T07:06:58.423 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.235670919, 2024-09-19T07:06:58.423 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.069621531, 2024-09-19T07:06:58.423 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:06:58.423 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 7900, 2024-09-19T07:06:58.424 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.523254385, 2024-09-19T07:06:58.424 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000066234 2024-09-19T07:06:58.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.424 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:06:58.424 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6366, 2024-09-19T07:06:58.424 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001553948, 2024-09-19T07:06:58.424 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000244 2024-09-19T07:06:58.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.424 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 57696, 2024-09-19T07:06:58.424 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 503680 2024-09-19T07:06:58.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.424 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:06:58.424 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 2955, 2024-09-19T07:06:58.424 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 2689, 2024-09-19T07:06:58.425 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 716002, 2024-09-19T07:06:58.425 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 14938757, 2024-09-19T07:06:58.425 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 404, 2024-09-19T07:06:58.425 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 11, 2024-09-19T07:06:58.425 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.380863591, 2024-09-19T07:06:58.425 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.105607473, 2024-09-19T07:06:58.425 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.213366087, 2024-09-19T07:06:58.425 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.054426287, 2024-09-19T07:06:58.425 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:06:58.425 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2683, 2024-09-19T07:06:58.425 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.287385948, 2024-09-19T07:06:58.426 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000107113 2024-09-19T07:06:58.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.426 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:06:58.426 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2205, 2024-09-19T07:06:58.426 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000469821, 2024-09-19T07:06:58.426 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000213 2024-09-19T07:06:58.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.426 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 85584, 2024-09-19T07:06:58.426 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 129728 2024-09-19T07:06:58.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.426 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:06:58.426 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 5009, 2024-09-19T07:06:58.426 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 4567, 2024-09-19T07:06:58.426 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 3095087, 2024-09-19T07:06:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 6533328, 2024-09-19T07:06:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 22, 2024-09-19T07:06:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 7, 2024-09-19T07:06:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.508909620, 2024-09-19T07:06:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.173399454, 2024-09-19T07:06:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.279157153, 2024-09-19T07:06:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.096359190, 2024-09-19T07:06:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:06:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4567, 2024-09-19T07:06:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.478279486, 2024-09-19T07:06:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000104725 2024-09-19T07:06:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:06:58.427 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6621, 2024-09-19T07:06:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001396695, 2024-09-19T07:06:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000210 2024-09-19T07:06:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2151184, 2024-09-19T07:06:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 412192 2024-09-19T07:06:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-19T07:06:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-19T07:06:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-19T07:06:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-19T07:06:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-19T07:06:58.428 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-19T07:06:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6939, 2024-09-19T07:06:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.791441875, 2024-09-19T07:06:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000258170 2024-09-19T07:06:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:06:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-19T07:06:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-19T07:06:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-19T07:06:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 15, 2024-09-19T07:06:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000351220, 2024-09-19T07:06:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000023414 2024-09-19T07:06:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:06:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.429 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-19T07:06:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 864, 2024-09-19T07:06:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 864, 2024-09-19T07:06:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-19T07:06:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 864, 2024-09-19T07:06:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 3.424917373, 2024-09-19T07:06:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.003964024 2024-09-19T07:06:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-19T07:06:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-09-19T07:06:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-19T07:06:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-19T07:06:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 504, 2024-09-19T07:06:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-19T07:06:58.430 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-19T07:06:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 49, 2024-09-19T07:06:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 18, 2024-09-19T07:06:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 31, 2024-09-19T07:06:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-09-19T07:06:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 18, 2024-09-19T07:06:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-09-19T07:06:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 3, 2024-09-19T07:06:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 3, 2024-09-19T07:06:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 2, 2024-09-19T07:06:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 972, 2024-09-19T07:06:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 963, 2024-09-19T07:06:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-09-19T07:06:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-09-19T07:06:58.431 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:06:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-19T07:06:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-19T07:06:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-19T07:06:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-19T07:06:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-09-19T07:06:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-09-19T07:06:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-09-19T07:06:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-09-19T07:06:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-19T07:06:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 385, 2024-09-19T07:06:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-19T07:06:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 4, 2024-09-19T07:06:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 234, 2024-09-19T07:06:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 7, 2024-09-19T07:06:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 385, 2024-09-19T07:06:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-19T07:06:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:06:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:06:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 248, 2024-09-19T07:06:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 28, 2024-09-19T07:06:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 220, 2024-09-19T07:06:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 137, 2024-09-19T07:06:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 2, 2024-09-19T07:06:58.433 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 671, 2024-09-19T07:06:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 667, 2024-09-19T07:06:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 599, 2024-09-19T07:06:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:06:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:06:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 790, 2024-09-19T07:06:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 720, 2024-09-19T07:06:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 3, 2024-09-19T07:06:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 627 2024-09-19T07:06:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-19T07:06:58.434 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-19T07:06:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-19T07:06:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-19T07:06:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-09-19T07:06:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 10, 2024-09-19T07:06:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 2, 2024-09-19T07:06:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-19T07:06:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:06:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 3, 2024-09-19T07:06:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 1, 2024-09-19T07:06:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-19T07:06:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-19T07:06:58.435 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-19T07:06:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:06:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:06:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-19T07:06:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-19T07:06:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 14, 2024-09-19T07:06:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 194, 2024-09-19T07:06:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 28, 2024-09-19T07:06:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-09-19T07:06:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-19T07:06:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:06:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-19T07:06:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-09-19T07:06:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.436 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-19T07:06:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-09-19T07:06:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 2638, 2024-09-19T07:06:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 2627, 2024-09-19T07:06:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 2627, 2024-09-19T07:06:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 11, 2024-09-19T07:06:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-19T07:06:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 48, 2024-09-19T07:06:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 84, 2024-09-19T07:06:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 83, 2024-09-19T07:06:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 83, 2024-09-19T07:06:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 1, 2024-09-19T07:06:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 1, 2024-09-19T07:06:58.437 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-19T07:06:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 2, 2024-09-19T07:06:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 29691453, 2024-09-19T07:06:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 29758899, 2024-09-19T07:06:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-09-19T07:06:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-19T07:06:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2544, 2024-09-19T07:06:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 5.547700810, 2024-09-19T07:06:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002180700 2024-09-19T07:06:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-09-19T07:06:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-19T07:06:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 41, 2024-09-19T07:06:58.438 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 70, 2024-09-19T07:06:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 29, 2024-09-19T07:06:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 20, 2024-09-19T07:06:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 20, 2024-09-19T07:06:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 0, 2024-09-19T07:06:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 49, 2024-09-19T07:06:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 49, 2024-09-19T07:06:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 0, 2024-09-19T07:06:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 3, 2024-09-19T07:06:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 20, 2024-09-19T07:06:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 17, 2024-09-19T07:06:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 57200, 2024-09-19T07:06:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 231712 2024-09-19T07:06:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.439 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-19T07:06:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 2137, 2024-09-19T07:06:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 3, 2024-09-19T07:06:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 864, 2024-09-19T07:06:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 44, 2024-09-19T07:06:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 6, 2024-09-19T07:06:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-19T07:06:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 18, 2024-09-19T07:06:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.010173210, 2024-09-19T07:06:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000565178 2024-09-19T07:06:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-19T07:06:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 173, 2024-09-19T07:06:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.027991553, 2024-09-19T07:06:58.440 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000161800 2024-09-19T07:06:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-19T07:06:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-19T07:06:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-19T07:06:58.441 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4, 2024-09-19T07:06:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001198566, 2024-09-19T07:06:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000299641 2024-09-19T07:06:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-19T07:06:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-19T07:06:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-09-19T07:06:58.442 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000128738, 2024-09-19T07:06:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000128738 2024-09-19T07:06:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-19T07:06:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-19T07:06:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-19T07:06:58.443 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-19T07:06:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 17, 2024-09-19T07:06:58.444 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.002619103, 2024-09-19T07:06:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000154064 2024-09-19T07:06:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-19T07:06:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 8, 2024-09-19T07:06:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.013970806, 2024-09-19T07:06:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001746350 2024-09-19T07:06:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-19T07:06:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-19T07:06:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 17, 2024-09-19T07:06:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.084174713, 2024-09-19T07:06:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.004951453 2024-09-19T07:06:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-19T07:06:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-09-19T07:06:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000136282, 2024-09-19T07:06:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000136282 2024-09-19T07:06:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-19T07:06:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-09-19T07:06:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.003268631, 2024-09-19T07:06:58.446 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001634315 2024-09-19T07:06:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-19T07:06:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-09-19T07:06:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001803842, 2024-09-19T07:06:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000901921 2024-09-19T07:06:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-19T07:06:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-19T07:06:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.447 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-19T07:06:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-09-19T07:06:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.007524706, 2024-09-19T07:06:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.003762353 2024-09-19T07:06:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-19T07:06:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 10, 2024-09-19T07:06:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.045458366, 2024-09-19T07:06:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.004545836 2024-09-19T07:06:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-19T07:06:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-19T07:06:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 600, 2024-09-19T07:06:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 3.201912069, 2024-09-19T07:06:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.005336520 2024-09-19T07:06:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-19T07:06:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.449 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-19T07:06:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-19T07:06:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-19T07:06:58.450 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 8, 2024-09-19T07:06:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.022255303, 2024-09-19T07:06:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002781912 2024-09-19T07:06:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-19T07:06:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-19T07:06:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-09-19T07:06:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.002301485, 2024-09-19T07:06:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002301485 2024-09-19T07:06:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.451 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:06:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:06:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-19T07:06:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:06:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-19T07:06:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2024-09-19T07:06:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 6, 2024-09-19T07:06:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 4, 2024-09-19T07:06:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 2, 2024-09-19T07:06:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-19T07:06:58.452 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 89, 2024-09-19T07:06:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 44, 2024-09-19T07:06:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 1037, 2024-09-19T07:06:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-19T07:06:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-19T07:06:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:06:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-19T07:06:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:06:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:06:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:06:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:06:58.453 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:06:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:06:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:06:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:06:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:06:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:06:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:06:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:06:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:06:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-19T07:06:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:06:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-19T07:06:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:06:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:06:58.454 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:06:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:06:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:06:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:06:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:06:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-19T07:06:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:06:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:06:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:06:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-19T07:06:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-19T07:06:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-19T07:06:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:06:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:06:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:06:58.455 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:06:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 319302, 2024-09-19T07:06:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 118, 2024-09-19T07:06:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:06:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-19T07:06:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-19T07:06:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-19T07:06:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:06:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-19T07:06:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:06:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-19T07:06:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 8192, 2024-09-19T07:06:58.456 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 180, 2024-09-19T07:06:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:06:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:06:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-19T07:06:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-19T07:06:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 195325, 2024-09-19T07:06:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 1435, 2024-09-19T07:06:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-19T07:06:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-19T07:06:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-19T07:06:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-19T07:06:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-19T07:06:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-19T07:06:58.457 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-19T07:06:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 5076, 2024-09-19T07:06:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 27357039, 2024-09-19T07:06:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 37, 2024-09-19T07:06:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 5076, 2024-09-19T07:06:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-19T07:06:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 5076, 2024-09-19T07:06:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 26.283384507, 2024-09-19T07:06:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.005177971 2024-09-19T07:06:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-19T07:06:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-19T07:06:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 5076, 2024-09-19T07:06:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 6126 2024-09-19T07:06:58.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 5076, 2024-09-19T07:06:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 442, 2024-09-19T07:06:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 4634, 2024-09-19T07:06:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-19T07:06:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-19T07:06:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 484, 2024-09-19T07:06:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 29, 2024-09-19T07:06:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 29, 2024-09-19T07:06:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 2545, 2024-09-19T07:06:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 264, 2024-09-19T07:06:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-19T07:06:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-19T07:06:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-19T07:06:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-19T07:06:58.459 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 525, 2024-09-19T07:06:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-19T07:06:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-19T07:06:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-19T07:06:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 413, 2024-09-19T07:06:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 58, 2024-09-19T07:06:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:06:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-19T07:06:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:06:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-19T07:06:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-19T07:06:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-19T07:06:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:06:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-19T07:06:58.460 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:06:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 599, 2024-09-19T07:06:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-19T07:06:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-19T07:06:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-19T07:06:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-19T07:06:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-19T07:06:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-19T07:06:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-19T07:06:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-19T07:06:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-19T07:06:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-19T07:06:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-19T07:06:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-19T07:06:58.461 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-19T07:06:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-19T07:06:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-19T07:06:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-19T07:06:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 103, 2024-09-19T07:06:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-19T07:06:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 23, 2024-09-19T07:06:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 12, 2024-09-19T07:06:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 12, 2024-09-19T07:06:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-09-19T07:06:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-19T07:06:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 1165, 2024-09-19T07:06:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2024-09-19T07:06:58.462 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 15 2024-09-19T07:06:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-19T07:06:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-09-19T07:06:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:06:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-19T07:06:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-19T07:06:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-09-19T07:06:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 195 2024-09-19T07:06:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-19T07:06:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-19T07:06:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 1, 2024-09-19T07:06:58.463 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-19T07:06:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 1, 2024-09-19T07:06:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 1, 2024-09-19T07:06:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 1, 2024-09-19T07:06:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-19T07:06:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:06:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:06:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-19T07:06:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:06:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 13169, 2024-09-19T07:06:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 4342109, 2024-09-19T07:06:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:58.464 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 13169, 2024-09-19T07:06:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:06:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:06:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 13169, 2024-09-19T07:06:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 4342109, 2024-09-19T07:06:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:06:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:06:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-19T07:06:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:06:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-19T07:06:58.465 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:06:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:06:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:06:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:06:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 5076, 2024-09-19T07:06:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 27357196, 2024-09-19T07:06:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 5076, 2024-09-19T07:06:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 27357196, 2024-09-19T07:06:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:06:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:06:58.466 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-19T07:06:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:06:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-19T07:06:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:06:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:06:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:06:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:06:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 5076, 2024-09-19T07:06:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 5076, 2024-09-19T07:06:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 5076, 2024-09-19T07:06:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 5076, 2024-09-19T07:06:58.467 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:06:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:06:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:06:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:06:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-19T07:06:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:06:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 1, 2024-09-19T07:06:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 157, 2024-09-19T07:06:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 1, 2024-09-19T07:06:58.468 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:06:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:06:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1, 2024-09-19T07:06:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 157, 2024-09-19T07:06:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:06:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:06:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:06:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x557e7cec2960": { 2024-09-19T07:06:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:06:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-19T07:06:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:06:58.469 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 2638, 2024-09-19T07:06:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 25564595, 2024-09-19T07:06:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:06:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 2638, 2024-09-19T07:06:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:06:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:06:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2544, 2024-09-19T07:06:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 25564595, 2024-09-19T07:06:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:06:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:06:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:06:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:06:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:06:58.470 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:06:58.470 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-19T07:06:59.299 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-19T07:06:59.299 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-19T07:06:59.300 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-09-19T07:06:59.300 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=17 2024-09-19T07:06:59.300 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2024-09-19T07:06:59.300 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T07:06:59.300 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T07:06:59.557 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:07:00.036 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-19T07:07:00.037 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:07:00.037 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 442, 2024-09-19T07:07:00.037 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 433, 2024-09-19T07:07:00.037 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1541131, 2024-09-19T07:07:00.037 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 678160, 2024-09-19T07:07:00.037 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 49, 2024-09-19T07:07:00.037 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-09-19T07:07:00.037 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.082823324, 2024-09-19T07:07:00.037 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.025688958, 2024-09-19T07:07:00.037 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.065851248, 2024-09-19T07:07:00.037 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000229077, 2024-09-19T07:07:00.038 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:00.038 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 429, 2024-09-19T07:07:00.038 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.028312470, 2024-09-19T07:07:00.038 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000065996 2024-09-19T07:07:00.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.038 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:00.038 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 46, 2024-09-19T07:07:00.038 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000004648, 2024-09-19T07:07:00.038 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000101 2024-09-19T07:07:00.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.038 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1532688, 2024-09-19T07:07:00.038 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 208496 2024-09-19T07:07:00.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.039 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:07:00.039 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 780, 2024-09-19T07:07:00.039 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1144, 2024-09-19T07:07:00.039 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 208182, 2024-09-19T07:07:00.039 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 950307, 2024-09-19T07:07:00.039 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 25, 2024-09-19T07:07:00.039 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-19T07:07:00.039 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.120948051, 2024-09-19T07:07:00.039 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.058446897, 2024-09-19T07:07:00.039 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.054762743, 2024-09-19T07:07:00.039 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.005711238, 2024-09-19T07:07:00.039 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:00.039 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1144, 2024-09-19T07:07:00.039 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.067944298, 2024-09-19T07:07:00.040 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000059391 2024-09-19T07:07:00.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.040 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:00.040 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1622, 2024-09-19T07:07:00.040 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000367747, 2024-09-19T07:07:00.040 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000226 2024-09-19T07:07:00.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.040 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T07:07:00.040 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 595648 2024-09-19T07:07:00.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.040 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:07:00.040 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 31, 2024-09-19T07:07:00.041 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 31, 2024-09-19T07:07:00.041 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 9242, 2024-09-19T07:07:00.041 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 107159, 2024-09-19T07:07:00.041 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 16, 2024-09-19T07:07:00.041 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-09-19T07:07:00.041 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.008217542, 2024-09-19T07:07:00.041 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.000514636, 2024-09-19T07:07:00.041 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.006812408, 2024-09-19T07:07:00.041 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000455687, 2024-09-19T07:07:00.041 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:00.041 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 30, 2024-09-19T07:07:00.041 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.022933409, 2024-09-19T07:07:00.042 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000764446 2024-09-19T07:07:00.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.042 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:00.042 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 8, 2024-09-19T07:07:00.042 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000860, 2024-09-19T07:07:00.042 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000107 2024-09-19T07:07:00.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.042 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T07:07:00.042 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T07:07:00.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.042 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-19T07:07:00.042 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-19T07:07:00.042 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-19T07:07:00.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.043 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-19T07:07:00.043 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-19T07:07:00.043 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-19T07:07:00.043 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 38, 2024-09-19T07:07:00.043 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001522180, 2024-09-19T07:07:00.043 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000040057 2024-09-19T07:07:00.043 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:00.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.043 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-19T07:07:00.043 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-19T07:07:00.043 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-19T07:07:00.043 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-09-19T07:07:00.043 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000013263, 2024-09-19T07:07:00.044 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000006631 2024-09-19T07:07:00.044 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:00.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.044 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-19T07:07:00.044 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-09-19T07:07:00.044 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-09-19T07:07:00.044 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-19T07:07:00.044 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.044 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.044 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.044 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-19T07:07:00.045 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-09-19T07:07:00.045 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-19T07:07:00.045 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-19T07:07:00.045 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 11, 2024-09-19T07:07:00.045 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-19T07:07:00.045 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-19T07:07:00.045 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 10, 2024-09-19T07:07:00.045 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-09-19T07:07:00.045 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 10, 2024-09-19T07:07:00.045 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-09-19T07:07:00.045 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 10, 2024-09-19T07:07:00.045 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-09-19T07:07:00.045 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-09-19T07:07:00.046 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-09-19T07:07:00.046 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 1, 2024-09-19T07:07:00.046 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 28, 2024-09-19T07:07:00.046 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 28, 2024-09-19T07:07:00.046 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-09-19T07:07:00.046 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-09-19T07:07:00.046 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:07:00.046 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-19T07:07:00.046 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-19T07:07:00.046 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-19T07:07:00.046 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-19T07:07:00.046 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-09-19T07:07:00.046 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-09-19T07:07:00.047 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-09-19T07:07:00.047 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-09-19T07:07:00.047 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-19T07:07:00.047 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:07:00.047 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-19T07:07:00.047 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 5, 2024-09-19T07:07:00.047 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 298, 2024-09-19T07:07:00.047 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 7, 2024-09-19T07:07:00.047 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:07:00.047 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-19T07:07:00.047 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:07:00.047 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:07:00.047 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:07:00.048 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:07:00.048 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:07:00.048 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-09-19T07:07:00.048 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:07:00.048 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T07:07:00.048 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:07:00.048 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:07:00.048 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:07:00.048 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:07:00.048 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-09-19T07:07:00.048 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:07:00.048 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-09-19T07:07:00.048 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-09-19T07:07:00.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.049 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-19T07:07:00.049 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-19T07:07:00.049 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-19T07:07:00.049 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-19T07:07:00.049 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 1, 2024-09-19T07:07:00.049 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-09-19T07:07:00.049 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-09-19T07:07:00.049 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-19T07:07:00.049 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:07:00.049 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-09-19T07:07:00.049 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-09-19T07:07:00.049 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-19T07:07:00.049 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-19T07:07:00.049 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-19T07:07:00.050 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:07:00.050 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:07:00.050 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-19T07:07:00.050 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-19T07:07:00.050 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 14, 2024-09-19T07:07:00.050 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 14, 2024-09-19T07:07:00.050 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:07:00.050 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-09-19T07:07:00.050 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-19T07:07:00.050 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:07:00.051 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-19T07:07:00.051 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-09-19T07:07:00.051 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.051 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-19T07:07:00.051 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-09-19T07:07:00.051 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 4, 2024-09-19T07:07:00.051 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-19T07:07:00.051 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-19T07:07:00.051 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 4, 2024-09-19T07:07:00.051 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-19T07:07:00.051 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-19T07:07:00.052 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 2, 2024-09-19T07:07:00.052 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-19T07:07:00.052 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-19T07:07:00.052 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2024-09-19T07:07:00.052 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-09-19T07:07:00.052 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-19T07:07:00.052 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-19T07:07:00.053 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-19T07:07:00.053 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 4195426, 2024-09-19T07:07:00.053 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-09-19T07:07:00.053 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-19T07:07:00.053 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3, 2024-09-19T07:07:00.053 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.004910259, 2024-09-19T07:07:00.053 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001636753 2024-09-19T07:07:00.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.053 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-09-19T07:07:00.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.053 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-19T07:07:00.053 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 13, 2024-09-19T07:07:00.053 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 13, 2024-09-19T07:07:00.054 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 0, 2024-09-19T07:07:00.054 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 11, 2024-09-19T07:07:00.054 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 12, 2024-09-19T07:07:00.054 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 1, 2024-09-19T07:07:00.054 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 10, 2024-09-19T07:07:00.054 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 10, 2024-09-19T07:07:00.054 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 0, 2024-09-19T07:07:00.054 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-09-19T07:07:00.054 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-09-19T07:07:00.054 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-09-19T07:07:00.054 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 51492, 2024-09-19T07:07:00.054 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 223520 2024-09-19T07:07:00.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.054 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-19T07:07:00.055 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-09-19T07:07:00.055 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-09-19T07:07:00.055 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-09-19T07:07:00.055 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 4, 2024-09-19T07:07:00.055 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-09-19T07:07:00.055 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-19T07:07:00.055 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.055 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.055 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.055 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.055 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-19T07:07:00.055 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.055 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.055 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.056 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-19T07:07:00.056 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.056 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.056 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.056 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-19T07:07:00.056 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.056 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.056 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.056 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-19T07:07:00.056 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.057 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.057 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.057 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-19T07:07:00.057 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.057 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.057 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.057 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-19T07:07:00.057 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.057 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.057 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.057 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-19T07:07:00.058 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.058 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.058 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.058 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.058 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-19T07:07:00.058 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.058 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.058 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.058 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.058 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-19T07:07:00.058 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.058 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.059 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.059 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-19T07:07:00.059 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.059 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.059 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.059 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-19T07:07:00.060 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.060 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.060 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.060 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.061 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-19T07:07:00.061 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.061 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.061 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.061 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-19T07:07:00.061 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.061 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.061 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.061 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-19T07:07:00.061 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.061 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.062 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.062 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-19T07:07:00.062 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.062 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.062 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.062 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-19T07:07:00.062 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.062 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.062 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.062 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-19T07:07:00.063 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.063 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.063 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.063 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-19T07:07:00.063 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.063 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.063 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.063 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-19T07:07:00.063 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.063 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.063 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.064 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-19T07:07:00.064 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.064 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.064 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.064 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-19T07:07:00.064 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.064 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.064 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.064 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-19T07:07:00.064 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.064 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.064 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.065 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-19T07:07:00.065 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.065 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.065 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.065 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-19T07:07:00.065 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.065 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.065 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.065 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-19T07:07:00.065 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.066 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.066 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-19T07:07:00.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.066 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.066 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-19T07:07:00.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.066 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.066 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-19T07:07:00.067 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.067 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.067 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.067 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:07:00.067 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:07:00.067 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-19T07:07:00.067 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.067 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.067 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.068 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:00.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.068 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-19T07:07:00.068 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 0, 2024-09-19T07:07:00.068 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 1, 2024-09-19T07:07:00.068 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 1, 2024-09-19T07:07:00.068 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 0, 2024-09-19T07:07:00.068 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-19T07:07:00.068 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-09-19T07:07:00.068 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-09-19T07:07:00.068 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 87, 2024-09-19T07:07:00.068 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-19T07:07:00.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.069 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-19T07:07:00.069 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:07:00.069 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-19T07:07:00.069 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:07:00.069 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:07:00.069 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:07:00.069 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:07:00.069 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:07:00.069 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:07:00.069 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:07:00.069 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:07:00.069 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:07:00.069 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:07:00.069 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:07:00.070 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:07:00.070 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:07:00.070 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-19T07:07:00.070 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:07:00.070 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-19T07:07:00.070 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:07:00.070 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:07:00.070 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:07:00.070 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:07:00.070 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:07:00.070 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:07:00.070 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:07:00.070 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-19T07:07:00.071 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:07:00.071 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:07:00.071 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:07:00.071 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-19T07:07:00.071 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-19T07:07:00.071 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-19T07:07:00.071 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:07:00.071 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:07:00.071 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:07:00.071 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:07:00.071 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 238775, 2024-09-19T07:07:00.071 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 75, 2024-09-19T07:07:00.071 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:07:00.072 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-19T07:07:00.072 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-19T07:07:00.072 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-19T07:07:00.072 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:07:00.072 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-19T07:07:00.072 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:07:00.072 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-19T07:07:00.072 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 8192, 2024-09-19T07:07:00.072 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 180, 2024-09-19T07:07:00.072 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:07:00.072 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:07:00.072 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-19T07:07:00.072 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-19T07:07:00.072 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 50872, 2024-09-19T07:07:00.073 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 198, 2024-09-19T07:07:00.073 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-19T07:07:00.073 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-19T07:07:00.073 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-19T07:07:00.073 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-19T07:07:00.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.073 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-19T07:07:00.073 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-19T07:07:00.073 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-19T07:07:00.073 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 39, 2024-09-19T07:07:00.073 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 11424, 2024-09-19T07:07:00.073 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 30, 2024-09-19T07:07:00.073 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 39, 2024-09-19T07:07:00.074 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-19T07:07:00.074 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 39, 2024-09-19T07:07:00.074 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.202995267, 2024-09-19T07:07:00.074 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.005205006 2024-09-19T07:07:00.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.074 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-19T07:07:00.074 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-19T07:07:00.074 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 39, 2024-09-19T07:07:00.074 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 61 2024-09-19T07:07:00.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.074 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 39, 2024-09-19T07:07:00.074 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 0, 2024-09-19T07:07:00.074 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 39, 2024-09-19T07:07:00.075 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-19T07:07:00.075 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-19T07:07:00.075 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-09-19T07:07:00.075 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2024-09-19T07:07:00.075 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 0, 2024-09-19T07:07:00.075 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 3, 2024-09-19T07:07:00.075 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 7, 2024-09-19T07:07:00.075 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-19T07:07:00.075 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-19T07:07:00.075 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-19T07:07:00.075 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-19T07:07:00.075 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 6, 2024-09-19T07:07:00.075 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-19T07:07:00.075 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-19T07:07:00.076 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-19T07:07:00.076 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 0, 2024-09-19T07:07:00.076 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2024-09-19T07:07:00.076 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:07:00.076 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-19T07:07:00.076 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:07:00.076 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-19T07:07:00.076 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-19T07:07:00.076 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-19T07:07:00.076 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:07:00.076 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-19T07:07:00.076 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:07:00.076 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-19T07:07:00.076 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-19T07:07:00.077 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-19T07:07:00.077 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-19T07:07:00.077 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-19T07:07:00.077 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-19T07:07:00.077 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-19T07:07:00.077 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-19T07:07:00.077 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-19T07:07:00.077 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-19T07:07:00.078 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-19T07:07:00.078 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-19T07:07:00.078 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-19T07:07:00.078 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-19T07:07:00.078 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-19T07:07:00.079 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-19T07:07:00.079 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-19T07:07:00.079 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 103, 2024-09-19T07:07:00.079 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-19T07:07:00.079 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 23, 2024-09-19T07:07:00.079 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 9, 2024-09-19T07:07:00.079 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 14, 2024-09-19T07:07:00.080 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 5, 2024-09-19T07:07:00.080 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-19T07:07:00.080 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 14, 2024-09-19T07:07:00.080 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2024-09-19T07:07:00.080 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 1 2024-09-19T07:07:00.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.080 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-19T07:07:00.080 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-09-19T07:07:00.080 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:07:00.081 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-19T07:07:00.081 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-19T07:07:00.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.081 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-09-19T07:07:00.081 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 26 2024-09-19T07:07:00.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.081 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-19T07:07:00.081 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-19T07:07:00.081 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:07:00.081 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-19T07:07:00.081 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-09-19T07:07:00.081 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-09-19T07:07:00.081 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-09-19T07:07:00.082 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-19T07:07:00.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.082 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:07:00.082 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:07:00.082 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-19T07:07:00.082 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:07:00.082 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 1249, 2024-09-19T07:07:00.082 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1623180, 2024-09-19T07:07:00.082 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:00.082 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 1249, 2024-09-19T07:07:00.082 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:07:00.082 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:07:00.082 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1249, 2024-09-19T07:07:00.082 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1623180, 2024-09-19T07:07:00.083 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:07:00.083 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.083 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.083 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.083 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:00.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.083 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-19T07:07:00.083 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:07:00.083 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-19T07:07:00.083 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:07:00.083 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:07:00.083 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:07:00.083 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:00.084 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:00.084 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 39, 2024-09-19T07:07:00.084 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 11424, 2024-09-19T07:07:00.084 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 39, 2024-09-19T07:07:00.084 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 11424, 2024-09-19T07:07:00.084 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:07:00.084 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.084 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.084 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.084 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:00.084 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.084 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-19T07:07:00.084 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:07:00.084 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-19T07:07:00.085 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:07:00.085 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:07:00.085 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:07:00.085 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:00.085 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:00.085 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 39, 2024-09-19T07:07:00.085 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 39, 2024-09-19T07:07:00.085 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 39, 2024-09-19T07:07:00.085 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 39, 2024-09-19T07:07:00.085 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:07:00.085 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.085 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.085 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:07:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:07:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-19T07:07:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:07:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:07:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:07:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:07:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:07:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-19T07:07:00.086 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-19T07:07:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:07:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55c5b3d12960": { 2024-09-19T07:07:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:07:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-19T07:07:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:07:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 4, 2024-09-19T07:07:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1122, 2024-09-19T07:07:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:00.087 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 4, 2024-09-19T07:07:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:07:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:07:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 3, 2024-09-19T07:07:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1122, 2024-09-19T07:07:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:07:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:00.088 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:00.088 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-19T07:07:00.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:00 smithi134 ceph-mon[35356]: pgmap v555: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:07:00.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:00 smithi174 ceph-mon[35637]: pgmap v555: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:07:00.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:00 smithi103 ceph-mon[31158]: pgmap v555: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:07:00.750 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-19T07:07:00.750 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-19T07:07:00.750 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=7 2024-09-19T07:07:00.750 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:07:00.750 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:07:00.750 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T07:07:00.751 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T07:07:00.998 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:07:01.511 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-19T07:07:01.511 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:07:01.511 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1068, 2024-09-19T07:07:01.511 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1091, 2024-09-19T07:07:01.511 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 278732, 2024-09-19T07:07:01.511 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 571117, 2024-09-19T07:07:01.511 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 11, 2024-09-19T07:07:01.511 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-09-19T07:07:01.512 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.136595220, 2024-09-19T07:07:01.512 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.061023651, 2024-09-19T07:07:01.512 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.067203686, 2024-09-19T07:07:01.512 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.006738578, 2024-09-19T07:07:01.512 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:01.512 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1089, 2024-09-19T07:07:01.512 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.067510771, 2024-09-19T07:07:01.512 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000061993 2024-09-19T07:07:01.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.512 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:01.512 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2072, 2024-09-19T07:07:01.512 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000485718, 2024-09-19T07:07:01.513 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000234 2024-09-19T07:07:01.513 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.513 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 4336, 2024-09-19T07:07:01.513 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 20592 2024-09-19T07:07:01.513 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.513 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:07:01.513 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 417, 2024-09-19T07:07:01.513 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 612, 2024-09-19T07:07:01.513 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1551787, 2024-09-19T07:07:01.513 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 806561, 2024-09-19T07:07:01.513 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 19, 2024-09-19T07:07:01.513 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-09-19T07:07:01.513 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.075787959, 2024-09-19T07:07:01.513 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.032855588, 2024-09-19T07:07:01.514 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.053859614, 2024-09-19T07:07:01.514 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000234511, 2024-09-19T07:07:01.514 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:01.514 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 611, 2024-09-19T07:07:01.514 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.050857036, 2024-09-19T07:07:01.514 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000083235 2024-09-19T07:07:01.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.514 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:01.514 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 10, 2024-09-19T07:07:01.514 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000740, 2024-09-19T07:07:01.514 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000074 2024-09-19T07:07:01.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.515 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1547552, 2024-09-19T07:07:01.515 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 679040 2024-09-19T07:07:01.515 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.515 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:07:01.515 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 89, 2024-09-19T07:07:01.515 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 147, 2024-09-19T07:07:01.515 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 17959, 2024-09-19T07:07:01.515 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 756484, 2024-09-19T07:07:01.515 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 60, 2024-09-19T07:07:01.515 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 6, 2024-09-19T07:07:01.515 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.037856690, 2024-09-19T07:07:01.515 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.007551287, 2024-09-19T07:07:01.515 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.027377161, 2024-09-19T07:07:01.515 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000286069, 2024-09-19T07:07:01.516 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:01.516 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 145, 2024-09-19T07:07:01.516 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.013738571, 2024-09-19T07:07:01.516 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000094748 2024-09-19T07:07:01.516 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.516 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:01.516 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 124, 2024-09-19T07:07:01.516 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000023794, 2024-09-19T07:07:01.516 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000191 2024-09-19T07:07:01.516 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.516 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 5840, 2024-09-19T07:07:01.516 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 129376 2024-09-19T07:07:01.516 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.517 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-19T07:07:01.517 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-19T07:07:01.517 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-19T07:07:01.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.517 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-19T07:07:01.517 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-19T07:07:01.517 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-19T07:07:01.517 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 48, 2024-09-19T07:07:01.517 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001660595, 2024-09-19T07:07:01.517 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000034595 2024-09-19T07:07:01.517 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:01.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.517 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-19T07:07:01.518 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-19T07:07:01.518 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-19T07:07:01.518 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-09-19T07:07:01.518 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000014264, 2024-09-19T07:07:01.518 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000007132 2024-09-19T07:07:01.518 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:01.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.518 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-19T07:07:01.518 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 9, 2024-09-19T07:07:01.518 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 6, 2024-09-19T07:07:01.518 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-19T07:07:01.518 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6, 2024-09-19T07:07:01.518 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.002281001, 2024-09-19T07:07:01.519 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000380166 2024-09-19T07:07:01.519 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.519 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-19T07:07:01.519 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 3, 2024-09-19T07:07:01.519 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-19T07:07:01.519 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-19T07:07:01.519 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 11, 2024-09-19T07:07:01.519 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-19T07:07:01.519 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-19T07:07:01.519 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 11, 2024-09-19T07:07:01.519 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-09-19T07:07:01.519 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 11, 2024-09-19T07:07:01.519 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-09-19T07:07:01.519 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 10, 2024-09-19T07:07:01.520 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 1, 2024-09-19T07:07:01.520 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 1, 2024-09-19T07:07:01.520 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 1, 2024-09-19T07:07:01.520 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 2, 2024-09-19T07:07:01.520 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 43, 2024-09-19T07:07:01.520 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 38, 2024-09-19T07:07:01.520 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-09-19T07:07:01.520 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 2, 2024-09-19T07:07:01.520 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:07:01.520 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-19T07:07:01.520 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-19T07:07:01.520 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-19T07:07:01.521 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-19T07:07:01.521 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-09-19T07:07:01.521 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-09-19T07:07:01.521 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-09-19T07:07:01.521 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-09-19T07:07:01.521 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-19T07:07:01.521 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:07:01.521 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 1, 2024-09-19T07:07:01.521 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 5, 2024-09-19T07:07:01.521 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 298, 2024-09-19T07:07:01.521 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 7, 2024-09-19T07:07:01.521 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:07:01.521 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-19T07:07:01.521 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:07:01.522 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:07:01.522 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:07:01.522 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:07:01.522 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:07:01.522 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-09-19T07:07:01.522 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:07:01.522 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 3, 2024-09-19T07:07:01.522 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:07:01.522 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:07:01.522 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:07:01.522 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:07:01.522 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 3, 2024-09-19T07:07:01.522 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:07:01.522 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-09-19T07:07:01.523 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-09-19T07:07:01.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.523 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-19T07:07:01.523 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-19T07:07:01.523 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-19T07:07:01.523 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-19T07:07:01.523 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 3, 2024-09-19T07:07:01.523 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-09-19T07:07:01.523 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-09-19T07:07:01.523 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-19T07:07:01.523 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:07:01.523 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-09-19T07:07:01.523 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-09-19T07:07:01.523 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-19T07:07:01.524 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-19T07:07:01.524 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-19T07:07:01.524 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:07:01.524 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:07:01.524 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-19T07:07:01.524 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-19T07:07:01.524 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 14, 2024-09-19T07:07:01.524 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 28, 2024-09-19T07:07:01.524 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:07:01.524 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-09-19T07:07:01.524 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-19T07:07:01.524 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:07:01.524 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-19T07:07:01.525 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-09-19T07:07:01.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.525 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-19T07:07:01.525 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-09-19T07:07:01.525 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 6, 2024-09-19T07:07:01.525 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-19T07:07:01.525 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-19T07:07:01.525 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 6, 2024-09-19T07:07:01.525 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-19T07:07:01.525 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-19T07:07:01.525 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 2, 2024-09-19T07:07:01.525 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-19T07:07:01.525 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-19T07:07:01.526 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2024-09-19T07:07:01.526 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-09-19T07:07:01.526 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-19T07:07:01.526 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-19T07:07:01.526 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-19T07:07:01.526 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 4196127, 2024-09-19T07:07:01.526 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-09-19T07:07:01.526 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-19T07:07:01.526 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 5, 2024-09-19T07:07:01.526 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.008201587, 2024-09-19T07:07:01.526 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001640317 2024-09-19T07:07:01.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.526 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-09-19T07:07:01.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.527 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-19T07:07:01.527 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 14, 2024-09-19T07:07:01.527 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 14, 2024-09-19T07:07:01.527 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 0, 2024-09-19T07:07:01.527 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 13, 2024-09-19T07:07:01.527 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 14, 2024-09-19T07:07:01.527 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 1, 2024-09-19T07:07:01.527 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 11, 2024-09-19T07:07:01.527 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 12, 2024-09-19T07:07:01.527 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 1, 2024-09-19T07:07:01.527 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 1, 2024-09-19T07:07:01.527 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 3, 2024-09-19T07:07:01.527 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 2, 2024-09-19T07:07:01.528 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 53328, 2024-09-19T07:07:01.528 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 223520 2024-09-19T07:07:01.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.528 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-19T07:07:01.528 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 14, 2024-09-19T07:07:01.528 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-09-19T07:07:01.528 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 9, 2024-09-19T07:07:01.528 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 31, 2024-09-19T07:07:01.528 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 1, 2024-09-19T07:07:01.528 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-19T07:07:01.528 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.528 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.528 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.529 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-19T07:07:01.529 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4, 2024-09-19T07:07:01.529 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001684182, 2024-09-19T07:07:01.529 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000421045 2024-09-19T07:07:01.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.529 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-19T07:07:01.529 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.529 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.529 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.529 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-19T07:07:01.529 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.529 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.530 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.530 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-19T07:07:01.530 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-09-19T07:07:01.530 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000596819, 2024-09-19T07:07:01.530 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000298409 2024-09-19T07:07:01.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.530 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-19T07:07:01.530 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.530 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.530 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.530 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-19T07:07:01.531 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.531 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.531 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.531 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-19T07:07:01.531 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.531 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.531 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.531 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-19T07:07:01.531 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.531 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.531 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.532 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-19T07:07:01.532 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.532 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.532 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.532 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-19T07:07:01.532 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.532 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.532 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.532 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-19T07:07:01.533 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.533 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.533 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.533 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-19T07:07:01.533 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.533 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.534 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.534 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-19T07:07:01.534 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.534 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.534 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.534 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-19T07:07:01.534 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.534 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.534 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.534 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-19T07:07:01.535 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.535 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.535 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.535 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-19T07:07:01.535 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.535 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.535 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.535 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-19T07:07:01.535 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.535 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.535 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.536 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-19T07:07:01.536 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.536 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.536 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.536 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-19T07:07:01.536 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.536 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.536 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.536 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-19T07:07:01.536 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.536 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.537 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.537 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-19T07:07:01.537 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.537 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.537 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.537 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-19T07:07:01.537 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.537 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.537 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.537 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-19T07:07:01.537 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.538 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.538 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.538 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-19T07:07:01.538 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.538 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.538 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.538 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-19T07:07:01.538 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.538 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.538 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.538 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-19T07:07:01.539 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.539 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.539 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.539 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-19T07:07:01.539 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.539 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.539 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.539 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-19T07:07:01.539 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.539 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.539 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.540 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:07:01.540 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:07:01.540 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-19T07:07:01.540 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.540 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.540 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.540 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:01.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.540 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-19T07:07:01.540 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2024-09-19T07:07:01.540 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 3, 2024-09-19T07:07:01.540 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 1, 2024-09-19T07:07:01.540 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 2, 2024-09-19T07:07:01.541 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-19T07:07:01.541 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 4, 2024-09-19T07:07:01.541 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 1, 2024-09-19T07:07:01.541 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 1030, 2024-09-19T07:07:01.541 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-19T07:07:01.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.541 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-19T07:07:01.541 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:07:01.541 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-19T07:07:01.541 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:07:01.541 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:07:01.541 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:07:01.541 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:07:01.541 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:07:01.542 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:07:01.542 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:07:01.542 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:07:01.542 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:07:01.542 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:07:01.542 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:07:01.542 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:07:01.542 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:07:01.542 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-19T07:07:01.542 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:07:01.542 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-19T07:07:01.542 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:07:01.542 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:07:01.542 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:07:01.543 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:07:01.543 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:07:01.543 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:07:01.543 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:07:01.543 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-19T07:07:01.543 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:07:01.543 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:07:01.543 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:07:01.543 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-19T07:07:01.543 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-19T07:07:01.543 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-19T07:07:01.543 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:07:01.543 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:07:01.543 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:07:01.544 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:07:01.544 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 255239, 2024-09-19T07:07:01.544 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 87, 2024-09-19T07:07:01.544 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:07:01.544 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-19T07:07:01.544 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-19T07:07:01.544 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-19T07:07:01.544 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:07:01.544 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-19T07:07:01.544 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:07:01.544 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-19T07:07:01.544 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 8192, 2024-09-19T07:07:01.544 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 180, 2024-09-19T07:07:01.545 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:07:01.545 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:07:01.545 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-19T07:07:01.545 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-19T07:07:01.545 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 56928, 2024-09-19T07:07:01.545 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 228, 2024-09-19T07:07:01.545 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-19T07:07:01.545 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-19T07:07:01.545 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-19T07:07:01.545 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-19T07:07:01.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.545 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-19T07:07:01.545 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-19T07:07:01.545 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-19T07:07:01.546 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 45, 2024-09-19T07:07:01.546 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 12450, 2024-09-19T07:07:01.546 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 32, 2024-09-19T07:07:01.546 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 45, 2024-09-19T07:07:01.546 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-19T07:07:01.546 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 45, 2024-09-19T07:07:01.546 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.218994891, 2024-09-19T07:07:01.546 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.004866553 2024-09-19T07:07:01.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.546 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-19T07:07:01.546 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-19T07:07:01.546 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 45, 2024-09-19T07:07:01.546 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 69 2024-09-19T07:07:01.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.547 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 45, 2024-09-19T07:07:01.547 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 0, 2024-09-19T07:07:01.547 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 45, 2024-09-19T07:07:01.547 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-19T07:07:01.547 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-19T07:07:01.547 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-09-19T07:07:01.547 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2024-09-19T07:07:01.547 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 0, 2024-09-19T07:07:01.547 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 5, 2024-09-19T07:07:01.547 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 9, 2024-09-19T07:07:01.547 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-19T07:07:01.547 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-19T07:07:01.547 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-19T07:07:01.547 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-19T07:07:01.548 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 6, 2024-09-19T07:07:01.548 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-19T07:07:01.548 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-19T07:07:01.548 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-19T07:07:01.548 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 0, 2024-09-19T07:07:01.548 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2024-09-19T07:07:01.548 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:07:01.548 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-19T07:07:01.548 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:07:01.548 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-19T07:07:01.548 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-19T07:07:01.548 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-19T07:07:01.549 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:07:01.549 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-19T07:07:01.549 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:07:01.549 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-19T07:07:01.549 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-19T07:07:01.549 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-19T07:07:01.549 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-19T07:07:01.549 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-19T07:07:01.549 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-19T07:07:01.549 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-19T07:07:01.549 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-19T07:07:01.549 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-19T07:07:01.549 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-19T07:07:01.549 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-19T07:07:01.550 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-19T07:07:01.550 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-19T07:07:01.550 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-19T07:07:01.550 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-19T07:07:01.550 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-19T07:07:01.550 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-19T07:07:01.550 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 103, 2024-09-19T07:07:01.550 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-19T07:07:01.550 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 23, 2024-09-19T07:07:01.550 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 9, 2024-09-19T07:07:01.550 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 12, 2024-09-19T07:07:01.550 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 3, 2024-09-19T07:07:01.550 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-19T07:07:01.551 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 17, 2024-09-19T07:07:01.551 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2024-09-19T07:07:01.551 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 2 2024-09-19T07:07:01.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.551 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-19T07:07:01.551 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-09-19T07:07:01.551 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:07:01.551 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-19T07:07:01.551 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-19T07:07:01.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.551 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-09-19T07:07:01.551 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 39 2024-09-19T07:07:01.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.551 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-19T07:07:01.552 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-19T07:07:01.552 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:07:01.552 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-19T07:07:01.552 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-09-19T07:07:01.552 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-09-19T07:07:01.552 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-09-19T07:07:01.552 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-19T07:07:01.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.552 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:07:01.552 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:07:01.552 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-19T07:07:01.552 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:07:01.552 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 1570, 2024-09-19T07:07:01.552 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1682831, 2024-09-19T07:07:01.553 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:01.553 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 1570, 2024-09-19T07:07:01.553 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:07:01.553 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:07:01.553 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1570, 2024-09-19T07:07:01.553 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1682831, 2024-09-19T07:07:01.553 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:07:01.553 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.553 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.553 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.553 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:01.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.553 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-19T07:07:01.553 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:07:01.554 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-19T07:07:01.554 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:07:01.554 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:07:01.554 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:07:01.554 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:01.554 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:01.554 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 45, 2024-09-19T07:07:01.554 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 12450, 2024-09-19T07:07:01.554 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 45, 2024-09-19T07:07:01.554 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 12450, 2024-09-19T07:07:01.554 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:07:01.554 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.555 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.555 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.555 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:01.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.555 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-19T07:07:01.555 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:07:01.555 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-19T07:07:01.555 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:07:01.555 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:07:01.555 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:07:01.555 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:01.555 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:01.556 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 45, 2024-09-19T07:07:01.556 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 45, 2024-09-19T07:07:01.556 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 45, 2024-09-19T07:07:01.556 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 45, 2024-09-19T07:07:01.556 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:07:01.556 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.556 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.556 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.556 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:01.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.556 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:07:01.556 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:07:01.556 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-19T07:07:01.557 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:07:01.557 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:07:01.557 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:07:01.557 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:01.557 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:01.557 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:07:01.557 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:07:01.557 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-19T07:07:01.557 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-19T07:07:01.557 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:07:01.557 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.557 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.557 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.558 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:01.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:01.558 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x56253672e960": { 2024-09-19T07:07:01.558 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:07:01.558 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-19T07:07:01.558 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:07:01.558 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 6, 2024-09-19T07:07:01.558 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1823, 2024-09-19T07:07:01.558 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:01.558 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 6, 2024-09-19T07:07:01.558 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:07:01.558 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:07:01.558 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 5, 2024-09-19T07:07:01.558 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1823, 2024-09-19T07:07:01.559 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:07:01.559 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:01.559 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:01.559 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:01.559 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:01.559 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:01.559 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-19T07:07:02.281 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:02 smithi134 ceph-mon[35356]: pgmap v556: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T07:07:02.307 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-19T07:07:02.308 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-19T07:07:02.308 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-09-19T07:07:02.308 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:07:02.308 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:07:02.308 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T07:07:02.308 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T07:07:02.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:02 smithi174 ceph-mon[35637]: pgmap v556: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T07:07:02.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:02 smithi103 ceph-mon[31158]: pgmap v556: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T07:07:02.553 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:07:02.992 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-19T07:07:02.992 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:07:02.992 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 3404, 2024-09-19T07:07:02.993 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 3406, 2024-09-19T07:07:02.993 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 963320, 2024-09-19T07:07:02.993 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 1082479, 2024-09-19T07:07:02.993 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2024-09-19T07:07:02.993 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 2, 2024-09-19T07:07:02.993 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.265755949, 2024-09-19T07:07:02.993 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.088185758, 2024-09-19T07:07:02.993 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.063867676, 2024-09-19T07:07:02.993 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.092139571, 2024-09-19T07:07:02.993 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:02.993 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3404, 2024-09-19T07:07:02.993 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.114251301, 2024-09-19T07:07:02.993 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000033563 2024-09-19T07:07:02.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:02.994 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:02.994 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:02.994 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:02.994 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:02.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:02.994 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:07:02.994 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:07:02.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:02.994 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:07:02.994 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 8, 2024-09-19T07:07:02.994 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 252, 2024-09-19T07:07:02.994 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 3900, 2024-09-19T07:07:02.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 597146, 2024-09-19T07:07:02.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2024-09-19T07:07:02.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-09-19T07:07:02.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.016801244, 2024-09-19T07:07:02.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.010046346, 2024-09-19T07:07:02.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.004054462, 2024-09-19T07:07:02.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000062039, 2024-09-19T07:07:02.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:02.995 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 250, 2024-09-19T07:07:02.995 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.027896117, 2024-09-19T07:07:02.995 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000111584 2024-09-19T07:07:02.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:02.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:02.996 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:02.996 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:02.996 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:02.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:02.996 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-19T07:07:02.996 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 596448 2024-09-19T07:07:02.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:02.996 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:07:02.996 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 365, 2024-09-19T07:07:02.996 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 353, 2024-09-19T07:07:02.996 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1524700, 2024-09-19T07:07:02.996 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 205438, 2024-09-19T07:07:02.996 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 4, 2024-09-19T07:07:02.997 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 2, 2024-09-19T07:07:02.997 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.059081471, 2024-09-19T07:07:02.997 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.023909480, 2024-09-19T07:07:02.997 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.048488569, 2024-09-19T07:07:02.997 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000011835, 2024-09-19T07:07:02.997 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:02.997 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 352, 2024-09-19T07:07:02.997 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.022320862, 2024-09-19T07:07:02.997 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000063411 2024-09-19T07:07:02.997 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:02.997 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:02.997 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:02.997 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:02.998 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:02.998 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:02.998 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1524432, 2024-09-19T07:07:02.998 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 205120 2024-09-19T07:07:02.998 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:02.998 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-19T07:07:02.998 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-19T07:07:02.998 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-19T07:07:02.998 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:02.998 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-19T07:07:02.998 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-19T07:07:02.998 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-19T07:07:02.999 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 5697, 2024-09-19T07:07:02.999 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.136670562, 2024-09-19T07:07:02.999 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000023989 2024-09-19T07:07:02.999 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:02.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:02.999 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-19T07:07:02.999 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-19T07:07:02.999 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-19T07:07:02.999 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:02.999 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:02.999 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:02.999 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:02.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:02.999 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-19T07:07:03.000 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-09-19T07:07:03.000 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-09-19T07:07:03.000 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-19T07:07:03.000 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.000 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.000 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.000 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.000 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-19T07:07:03.000 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-09-19T07:07:03.000 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-19T07:07:03.000 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-19T07:07:03.000 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2024-09-19T07:07:03.000 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-19T07:07:03.001 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-19T07:07:03.001 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 0, 2024-09-19T07:07:03.001 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-09-19T07:07:03.001 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 0, 2024-09-19T07:07:03.001 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-09-19T07:07:03.001 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 0, 2024-09-19T07:07:03.001 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-09-19T07:07:03.001 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-09-19T07:07:03.001 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-09-19T07:07:03.001 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 1, 2024-09-19T07:07:03.001 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2024-09-19T07:07:03.001 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2024-09-19T07:07:03.001 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-09-19T07:07:03.001 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-09-19T07:07:03.002 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:07:03.002 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-19T07:07:03.002 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-19T07:07:03.002 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-19T07:07:03.002 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-19T07:07:03.002 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-09-19T07:07:03.002 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-09-19T07:07:03.002 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-09-19T07:07:03.002 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-09-19T07:07:03.002 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-19T07:07:03.002 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:07:03.002 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-19T07:07:03.002 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 0, 2024-09-19T07:07:03.002 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 0, 2024-09-19T07:07:03.003 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-09-19T07:07:03.003 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:07:03.003 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-19T07:07:03.003 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:07:03.003 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:07:03.003 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:07:03.003 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:07:03.003 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:07:03.003 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-09-19T07:07:03.003 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:07:03.003 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T07:07:03.003 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:07:03.003 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:07:03.003 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:07:03.004 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:07:03.004 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-09-19T07:07:03.004 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:07:03.004 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-09-19T07:07:03.004 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-09-19T07:07:03.004 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.004 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-19T07:07:03.004 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-19T07:07:03.004 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-19T07:07:03.004 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-19T07:07:03.004 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-09-19T07:07:03.004 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-09-19T07:07:03.004 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-09-19T07:07:03.005 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-19T07:07:03.005 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:07:03.005 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-09-19T07:07:03.005 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-09-19T07:07:03.005 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-19T07:07:03.005 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-19T07:07:03.005 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-19T07:07:03.005 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:07:03.005 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:07:03.005 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-19T07:07:03.005 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-19T07:07:03.005 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 0, 2024-09-19T07:07:03.005 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 0, 2024-09-19T07:07:03.006 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:07:03.006 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-09-19T07:07:03.006 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-19T07:07:03.006 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:07:03.006 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-19T07:07:03.006 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-09-19T07:07:03.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.006 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-19T07:07:03.006 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-09-19T07:07:03.006 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2024-09-19T07:07:03.006 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-19T07:07:03.006 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-19T07:07:03.006 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 6, 2024-09-19T07:07:03.006 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-19T07:07:03.007 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-19T07:07:03.007 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2024-09-19T07:07:03.007 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-19T07:07:03.007 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-19T07:07:03.007 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2024-09-19T07:07:03.007 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-09-19T07:07:03.007 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-19T07:07:03.007 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-19T07:07:03.007 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-19T07:07:03.007 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 4196127, 2024-09-19T07:07:03.007 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 4195986, 2024-09-19T07:07:03.007 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-19T07:07:03.007 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.007 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.008 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.008 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.008 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 6 2024-09-19T07:07:03.008 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.008 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-19T07:07:03.008 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 3, 2024-09-19T07:07:03.008 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 3, 2024-09-19T07:07:03.008 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 0, 2024-09-19T07:07:03.008 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 1, 2024-09-19T07:07:03.008 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 1, 2024-09-19T07:07:03.008 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 0, 2024-09-19T07:07:03.008 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 0, 2024-09-19T07:07:03.008 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 0, 2024-09-19T07:07:03.008 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 0, 2024-09-19T07:07:03.009 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-09-19T07:07:03.009 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-09-19T07:07:03.009 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-09-19T07:07:03.009 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 47832, 2024-09-19T07:07:03.009 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 215328 2024-09-19T07:07:03.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.009 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-19T07:07:03.009 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-09-19T07:07:03.009 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-09-19T07:07:03.010 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-09-19T07:07:03.010 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2024-09-19T07:07:03.010 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-09-19T07:07:03.010 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-19T07:07:03.010 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.010 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.010 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.010 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-19T07:07:03.010 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.010 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.010 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.011 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-19T07:07:03.011 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.011 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.011 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.011 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-19T07:07:03.011 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.011 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.011 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.011 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-19T07:07:03.011 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.011 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.012 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.012 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-19T07:07:03.012 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.012 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.012 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.012 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-19T07:07:03.012 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.012 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.013 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.013 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-19T07:07:03.013 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.013 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.013 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.013 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-19T07:07:03.013 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.013 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.013 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.013 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-19T07:07:03.013 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.014 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.014 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.014 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-19T07:07:03.014 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.014 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.014 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.014 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-19T07:07:03.014 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.014 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.014 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.015 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-19T07:07:03.015 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.015 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.015 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.015 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-19T07:07:03.015 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.015 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.015 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.015 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-19T07:07:03.015 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.015 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.016 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.016 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.016 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-19T07:07:03.016 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.016 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.016 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.016 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.016 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-19T07:07:03.016 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.016 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.016 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.016 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.016 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-19T07:07:03.017 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.017 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.017 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.017 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-19T07:07:03.017 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.017 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.017 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.017 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-19T07:07:03.017 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.017 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.017 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.018 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-19T07:07:03.018 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.018 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.018 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.018 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-19T07:07:03.018 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.018 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.018 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.018 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-19T07:07:03.018 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.018 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.018 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.019 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-19T07:07:03.019 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.019 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.019 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.019 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-19T07:07:03.019 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.019 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.019 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.019 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-19T07:07:03.019 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.019 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.020 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.020 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-19T07:07:03.020 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.020 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.020 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.020 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-19T07:07:03.020 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.020 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.020 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.020 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-19T07:07:03.021 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.021 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.021 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.021 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:07:03.021 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:07:03.021 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-19T07:07:03.021 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.021 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.021 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.021 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:03.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.021 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-19T07:07:03.021 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2024-09-19T07:07:03.022 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 3, 2024-09-19T07:07:03.022 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 1, 2024-09-19T07:07:03.022 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 3, 2024-09-19T07:07:03.022 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-19T07:07:03.022 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-09-19T07:07:03.022 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-09-19T07:07:03.022 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 2995, 2024-09-19T07:07:03.022 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-19T07:07:03.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.022 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-19T07:07:03.022 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:07:03.022 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-19T07:07:03.022 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:07:03.022 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:07:03.023 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:07:03.023 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:07:03.023 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:07:03.023 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:07:03.023 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:07:03.023 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:07:03.023 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:07:03.023 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:07:03.023 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:07:03.023 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:07:03.023 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:07:03.023 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-19T07:07:03.023 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:07:03.023 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-19T07:07:03.024 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:07:03.024 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:07:03.024 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:07:03.024 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:07:03.024 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:07:03.024 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:07:03.024 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:07:03.024 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-19T07:07:03.024 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:07:03.024 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:07:03.024 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:07:03.024 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-19T07:07:03.025 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-19T07:07:03.025 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-19T07:07:03.025 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:07:03.025 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:07:03.025 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:07:03.025 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:07:03.025 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 182075, 2024-09-19T07:07:03.025 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 49, 2024-09-19T07:07:03.025 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:07:03.025 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-19T07:07:03.025 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-19T07:07:03.025 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-19T07:07:03.026 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:07:03.026 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-19T07:07:03.026 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:07:03.026 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-19T07:07:03.026 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 8192, 2024-09-19T07:07:03.026 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 180, 2024-09-19T07:07:03.026 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:07:03.026 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:07:03.026 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-19T07:07:03.026 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-19T07:07:03.026 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 8512, 2024-09-19T07:07:03.026 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 13, 2024-09-19T07:07:03.026 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-19T07:07:03.027 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-19T07:07:03.027 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-19T07:07:03.027 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-19T07:07:03.027 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.027 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-19T07:07:03.027 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-19T07:07:03.027 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-19T07:07:03.027 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 3407, 2024-09-19T07:07:03.027 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 21, 2024-09-19T07:07:03.027 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 3, 2024-09-19T07:07:03.027 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 3407, 2024-09-19T07:07:03.027 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-19T07:07:03.027 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3407, 2024-09-19T07:07:03.027 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.741958922, 2024-09-19T07:07:03.028 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000511288 2024-09-19T07:07:03.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.028 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-19T07:07:03.028 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-19T07:07:03.028 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3407, 2024-09-19T07:07:03.028 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 3408 2024-09-19T07:07:03.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.028 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 3407, 2024-09-19T07:07:03.028 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 3407, 2024-09-19T07:07:03.028 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2024-09-19T07:07:03.028 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-19T07:07:03.028 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-19T07:07:03.028 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 2232, 2024-09-19T07:07:03.028 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2024-09-19T07:07:03.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 1173, 2024-09-19T07:07:03.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2024-09-19T07:07:03.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2024-09-19T07:07:03.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-19T07:07:03.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-19T07:07:03.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-19T07:07:03.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-19T07:07:03.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2024-09-19T07:07:03.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-19T07:07:03.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-19T07:07:03.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-19T07:07:03.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 1, 2024-09-19T07:07:03.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2024-09-19T07:07:03.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:07:03.030 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-19T07:07:03.030 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:07:03.030 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-19T07:07:03.030 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-19T07:07:03.030 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-19T07:07:03.030 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:07:03.030 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-19T07:07:03.030 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:07:03.030 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-19T07:07:03.030 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-19T07:07:03.030 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-19T07:07:03.030 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-19T07:07:03.030 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-19T07:07:03.030 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-19T07:07:03.031 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-19T07:07:03.031 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-19T07:07:03.031 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-19T07:07:03.031 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-19T07:07:03.031 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-19T07:07:03.031 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-19T07:07:03.031 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-19T07:07:03.031 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-19T07:07:03.031 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-19T07:07:03.031 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-19T07:07:03.031 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-19T07:07:03.031 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 103, 2024-09-19T07:07:03.031 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-19T07:07:03.032 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 23, 2024-09-19T07:07:03.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 4, 2024-09-19T07:07:03.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 5, 2024-09-19T07:07:03.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 1, 2024-09-19T07:07:03.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-19T07:07:03.032 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2024-09-19T07:07:03.032 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-09-19T07:07:03.032 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2024-09-19T07:07:03.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.032 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-19T07:07:03.032 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2024-09-19T07:07:03.032 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:07:03.032 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-19T07:07:03.032 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-19T07:07:03.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.033 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-09-19T07:07:03.033 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-09-19T07:07:03.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.033 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-19T07:07:03.033 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-19T07:07:03.033 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:07:03.033 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-19T07:07:03.033 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-09-19T07:07:03.033 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-09-19T07:07:03.033 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-09-19T07:07:03.033 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-19T07:07:03.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.033 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:07:03.034 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:07:03.034 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-19T07:07:03.034 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:07:03.034 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 3773, 2024-09-19T07:07:03.034 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 2130691, 2024-09-19T07:07:03.034 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:03.034 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 3773, 2024-09-19T07:07:03.034 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:07:03.034 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:07:03.034 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 3773, 2024-09-19T07:07:03.034 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2130691, 2024-09-19T07:07:03.034 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:07:03.034 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.034 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.035 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.035 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:03.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.035 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-19T07:07:03.035 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:07:03.035 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-19T07:07:03.035 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:07:03.035 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:07:03.035 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:07:03.035 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:03.035 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:03.035 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 3407, 2024-09-19T07:07:03.035 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 1828, 2024-09-19T07:07:03.035 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 3407, 2024-09-19T07:07:03.036 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1828, 2024-09-19T07:07:03.036 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:07:03.036 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.036 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.036 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.036 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:03.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.036 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-19T07:07:03.036 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:07:03.036 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-19T07:07:03.036 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:07:03.036 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:07:03.037 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:07:03.037 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:03.037 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:03.037 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 3407, 2024-09-19T07:07:03.037 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 3407, 2024-09-19T07:07:03.037 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 3407, 2024-09-19T07:07:03.037 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 3407, 2024-09-19T07:07:03.037 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:07:03.037 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.037 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.037 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.037 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:03.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.038 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:07:03.038 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:07:03.038 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-19T07:07:03.038 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:07:03.038 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:07:03.038 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:07:03.038 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:03.038 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:03.038 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:07:03.038 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:07:03.038 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-19T07:07:03.038 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-19T07:07:03.038 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:07:03.039 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.039 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.039 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.039 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:03.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:07:03.039 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x556f78eb0b40": { 2024-09-19T07:07:03.039 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:07:03.039 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-19T07:07:03.039 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:07:03.039 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:07:03.039 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:07:03.039 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:03.039 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:03.039 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:07:03.040 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:07:03.040 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-19T07:07:03.040 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-19T07:07:03.040 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:07:03.040 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:07:03.040 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:07:03.040 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:07:03.040 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:03.040 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:07:03.040 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-19T07:07:03.809 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-19T07:07:03.809 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-19T07:07:03.809 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-19T07:07:03.809 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:07:03.810 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:07:03.810 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T07:07:03.810 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T07:07:04.067 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:07:04.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:04 smithi174 ceph-mon[35637]: pgmap v557: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:07:04.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:04 smithi103 ceph-mon[31158]: pgmap v557: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:07:04.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:04 smithi134 ceph-mon[35356]: pgmap v557: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:07:04.634 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-19T07:07:04.634 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:07:04.634 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 99, 2024-09-19T07:07:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 103, 2024-09-19T07:07:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 19479, 2024-09-19T07:07:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 726144, 2024-09-19T07:07:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 69, 2024-09-19T07:07:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-09-19T07:07:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.030604897, 2024-09-19T07:07:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.002474109, 2024-09-19T07:07:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.020936669, 2024-09-19T07:07:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.001023242, 2024-09-19T07:07:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 101, 2024-09-19T07:07:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.013716702, 2024-09-19T07:07:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000135808 2024-09-19T07:07:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 83, 2024-09-19T07:07:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000034926, 2024-09-19T07:07:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000420 2024-09-19T07:07:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3408, 2024-09-19T07:07:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 4256 2024-09-19T07:07:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:07:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 27, 2024-09-19T07:07:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 269, 2024-09-19T07:07:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 8295, 2024-09-19T07:07:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 740312, 2024-09-19T07:07:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 16, 2024-09-19T07:07:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-09-19T07:07:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.022769193, 2024-09-19T07:07:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.009998090, 2024-09-19T07:07:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.009553156, 2024-09-19T07:07:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000497347, 2024-09-19T07:07:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 267, 2024-09-19T07:07:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.064527416, 2024-09-19T07:07:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000241675 2024-09-19T07:07:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 12, 2024-09-19T07:07:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000001629, 2024-09-19T07:07:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000135 2024-09-19T07:07:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 4432, 2024-09-19T07:07:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 685248 2024-09-19T07:07:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:07:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1425, 2024-09-19T07:07:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1374, 2024-09-19T07:07:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1810761, 2024-09-19T07:07:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 442698, 2024-09-19T07:07:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 8, 2024-09-19T07:07:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-09-19T07:07:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.166515992, 2024-09-19T07:07:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.066209478, 2024-09-19T07:07:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.111030667, 2024-09-19T07:07:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.007154904, 2024-09-19T07:07:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1373, 2024-09-19T07:07:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.072068340, 2024-09-19T07:07:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000052489 2024-09-19T07:07:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1985, 2024-09-19T07:07:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000437105, 2024-09-19T07:07:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000220 2024-09-19T07:07:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1540976, 2024-09-19T07:07:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 211344 2024-09-19T07:07:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-19T07:07:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-19T07:07:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-19T07:07:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-19T07:07:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T07:07:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T07:07:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 53, 2024-09-19T07:07:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.002135490, 2024-09-19T07:07:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000040292 2024-09-19T07:07:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-19T07:07:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T07:07:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T07:07:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-09-19T07:07:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000017070, 2024-09-19T07:07:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000008535 2024-09-19T07:07:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-19T07:07:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 6, 2024-09-19T07:07:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 5, 2024-09-19T07:07:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-19T07:07:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5, 2024-09-19T07:07:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.001370915, 2024-09-19T07:07:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000274183 2024-09-19T07:07:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-19T07:07:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 1, 2024-09-19T07:07:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-19T07:07:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-19T07:07:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 11, 2024-09-19T07:07:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-09-19T07:07:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-19T07:07:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 11, 2024-09-19T07:07:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 0, 2024-09-19T07:07:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 11, 2024-09-19T07:07:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-09-19T07:07:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 10, 2024-09-19T07:07:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 1, 2024-09-19T07:07:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-09-19T07:07:04.645 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-09-19T07:07:04.645 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 2, 2024-09-19T07:07:04.645 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 41, 2024-09-19T07:07:04.645 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 36, 2024-09-19T07:07:04.645 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 1, 2024-09-19T07:07:04.645 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 2, 2024-09-19T07:07:04.645 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:07:04.645 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-19T07:07:04.645 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-09-19T07:07:04.646 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-19T07:07:04.646 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-19T07:07:04.646 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-09-19T07:07:04.646 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-09-19T07:07:04.646 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-09-19T07:07:04.646 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-09-19T07:07:04.646 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-19T07:07:04.646 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:07:04.646 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-19T07:07:04.646 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 5, 2024-09-19T07:07:04.647 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 298, 2024-09-19T07:07:04.647 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 7, 2024-09-19T07:07:04.647 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:07:04.647 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-19T07:07:04.647 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:07:04.647 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:07:04.647 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:07:04.647 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:07:04.647 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:07:04.647 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-09-19T07:07:04.647 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:07:04.648 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 4, 2024-09-19T07:07:04.648 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:07:04.648 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:07:04.648 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:07:04.648 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:07:04.648 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 4, 2024-09-19T07:07:04.648 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:07:04.648 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-09-19T07:07:04.648 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-09-19T07:07:04.648 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.649 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-19T07:07:04.649 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-19T07:07:04.649 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-19T07:07:04.649 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-19T07:07:04.649 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 3, 2024-09-19T07:07:04.649 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-09-19T07:07:04.649 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-19T07:07:04.649 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-19T07:07:04.649 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:07:04.650 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-09-19T07:07:04.650 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-09-19T07:07:04.650 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-19T07:07:04.650 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-19T07:07:04.650 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-19T07:07:04.650 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:07:04.650 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:07:04.650 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-19T07:07:04.650 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-19T07:07:04.650 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 14, 2024-09-19T07:07:04.651 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 28, 2024-09-19T07:07:04.651 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:07:04.651 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-09-19T07:07:04.651 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-19T07:07:04.651 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:07:04.651 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-19T07:07:04.651 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-19T07:07:04.651 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.651 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-19T07:07:04.652 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-19T07:07:04.652 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 10, 2024-09-19T07:07:04.652 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-19T07:07:04.652 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-19T07:07:04.652 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 10, 2024-09-19T07:07:04.652 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-19T07:07:04.652 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-19T07:07:04.652 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 2, 2024-09-19T07:07:04.652 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-19T07:07:04.652 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-19T07:07:04.652 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-09-19T07:07:04.652 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-09-19T07:07:04.652 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-19T07:07:04.653 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-19T07:07:04.653 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-09-19T07:07:04.653 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 4197028, 2024-09-19T07:07:04.653 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-09-19T07:07:04.653 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-19T07:07:04.653 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 9, 2024-09-19T07:07:04.653 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.022413296, 2024-09-19T07:07:04.653 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002490366 2024-09-19T07:07:04.653 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.653 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-09-19T07:07:04.653 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.653 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-19T07:07:04.653 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 14, 2024-09-19T07:07:04.654 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 14, 2024-09-19T07:07:04.654 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 0, 2024-09-19T07:07:04.654 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 13, 2024-09-19T07:07:04.654 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 14, 2024-09-19T07:07:04.654 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 1, 2024-09-19T07:07:04.654 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 11, 2024-09-19T07:07:04.654 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 12, 2024-09-19T07:07:04.654 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 1, 2024-09-19T07:07:04.654 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-09-19T07:07:04.654 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 4, 2024-09-19T07:07:04.654 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 4, 2024-09-19T07:07:04.654 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 50984, 2024-09-19T07:07:04.654 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223520 2024-09-19T07:07:04.654 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.655 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-19T07:07:04.655 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 10, 2024-09-19T07:07:04.655 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-09-19T07:07:04.655 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 6, 2024-09-19T07:07:04.655 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 15, 2024-09-19T07:07:04.655 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 1, 2024-09-19T07:07:04.655 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-19T07:07:04.655 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.655 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.655 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.655 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.655 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-19T07:07:04.656 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-09-19T07:07:04.656 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000639230, 2024-09-19T07:07:04.656 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000319615 2024-09-19T07:07:04.656 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.656 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-19T07:07:04.656 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.656 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.656 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.656 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.656 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-19T07:07:04.656 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.656 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.656 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.657 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.657 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-19T07:07:04.657 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3, 2024-09-19T07:07:04.657 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000731685, 2024-09-19T07:07:04.657 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000243895 2024-09-19T07:07:04.657 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.657 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-19T07:07:04.657 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.657 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.657 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.657 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.657 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-19T07:07:04.657 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.658 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.658 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.658 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.658 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-19T07:07:04.658 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.658 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.658 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.658 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.658 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-19T07:07:04.658 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.658 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.658 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.658 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.659 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-19T07:07:04.659 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.659 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.659 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.659 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.659 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-19T07:07:04.659 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.659 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.659 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.659 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.659 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-19T07:07:04.660 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.660 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.660 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.660 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.660 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-19T07:07:04.661 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.661 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.661 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.661 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.661 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-19T07:07:04.661 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.661 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.661 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.661 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.661 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-19T07:07:04.661 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.661 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.661 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.661 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.662 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-19T07:07:04.662 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.662 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.662 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.662 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.662 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-19T07:07:04.662 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.662 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.662 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.662 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.662 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-19T07:07:04.662 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.662 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.663 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.663 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.663 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-19T07:07:04.663 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.663 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.663 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.663 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.663 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-19T07:07:04.663 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.663 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.663 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.663 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.663 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-19T07:07:04.664 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.664 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.664 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.664 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.664 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-19T07:07:04.664 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.664 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.664 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.664 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.664 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-19T07:07:04.664 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.664 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.665 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.665 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.665 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-19T07:07:04.665 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.665 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.665 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.665 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.665 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-19T07:07:04.665 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.665 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.665 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.665 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.665 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-19T07:07:04.665 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.666 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.666 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.666 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.666 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-19T07:07:04.666 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.666 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.666 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.666 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.666 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-19T07:07:04.666 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.666 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.666 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.666 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.666 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-19T07:07:04.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.667 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.667 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.667 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:07:04.667 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:07:04.667 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-19T07:07:04.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.667 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.667 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:04.667 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.667 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-19T07:07:04.668 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 0, 2024-09-19T07:07:04.668 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 4, 2024-09-19T07:07:04.668 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 4, 2024-09-19T07:07:04.668 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 0, 2024-09-19T07:07:04.668 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-19T07:07:04.668 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 4, 2024-09-19T07:07:04.668 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 1, 2024-09-19T07:07:04.668 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 86, 2024-09-19T07:07:04.668 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-19T07:07:04.668 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.668 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-19T07:07:04.668 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:07:04.669 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-19T07:07:04.669 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:07:04.669 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:07:04.669 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:07:04.669 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:07:04.669 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:07:04.669 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:07:04.669 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:07:04.669 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:07:04.669 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:07:04.669 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:07:04.669 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:07:04.669 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:07:04.670 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:07:04.670 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-19T07:07:04.670 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:07:04.670 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-19T07:07:04.670 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:07:04.670 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:07:04.670 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:07:04.670 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:07:04.670 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:07:04.670 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:07:04.670 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:07:04.670 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-19T07:07:04.671 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:07:04.671 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:07:04.671 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:07:04.671 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-19T07:07:04.671 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-19T07:07:04.671 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-19T07:07:04.671 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:07:04.671 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:07:04.671 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:07:04.671 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:07:04.671 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 246775, 2024-09-19T07:07:04.671 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 77, 2024-09-19T07:07:04.671 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:07:04.671 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-19T07:07:04.672 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-19T07:07:04.672 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-19T07:07:04.672 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:07:04.672 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-19T07:07:04.672 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:07:04.672 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-19T07:07:04.672 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8192, 2024-09-19T07:07:04.672 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 180, 2024-09-19T07:07:04.672 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:07:04.672 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:07:04.672 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-19T07:07:04.672 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-19T07:07:04.673 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 56568, 2024-09-19T07:07:04.673 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 227, 2024-09-19T07:07:04.673 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-19T07:07:04.673 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-19T07:07:04.673 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-19T07:07:04.673 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-19T07:07:04.673 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.673 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-19T07:07:04.673 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-19T07:07:04.673 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-19T07:07:04.673 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 47, 2024-09-19T07:07:04.673 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 13171, 2024-09-19T07:07:04.674 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 31, 2024-09-19T07:07:04.674 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 47, 2024-09-19T07:07:04.674 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-19T07:07:04.674 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 47, 2024-09-19T07:07:04.674 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.228997081, 2024-09-19T07:07:04.674 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004872278 2024-09-19T07:07:04.674 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.674 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-19T07:07:04.674 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-19T07:07:04.674 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 47, 2024-09-19T07:07:04.675 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 71 2024-09-19T07:07:04.675 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.675 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 47, 2024-09-19T07:07:04.675 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 0, 2024-09-19T07:07:04.675 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 47, 2024-09-19T07:07:04.675 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-19T07:07:04.675 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-19T07:07:04.675 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-09-19T07:07:04.675 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-09-19T07:07:04.675 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 0, 2024-09-19T07:07:04.675 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 9, 2024-09-19T07:07:04.675 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 7, 2024-09-19T07:07:04.676 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-19T07:07:04.676 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-19T07:07:04.676 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-19T07:07:04.676 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-09-19T07:07:04.676 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 6, 2024-09-19T07:07:04.676 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-19T07:07:04.676 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-19T07:07:04.676 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-19T07:07:04.676 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 0, 2024-09-19T07:07:04.677 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-09-19T07:07:04.677 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:07:04.677 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-19T07:07:04.677 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:07:04.677 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-19T07:07:04.677 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-19T07:07:04.677 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-19T07:07:04.677 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:07:04.677 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-19T07:07:04.677 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:07:04.677 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-19T07:07:04.677 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-19T07:07:04.677 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-19T07:07:04.678 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-19T07:07:04.678 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-19T07:07:04.678 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-19T07:07:04.678 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-19T07:07:04.678 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-19T07:07:04.678 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-19T07:07:04.678 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-19T07:07:04.678 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-19T07:07:04.678 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-19T07:07:04.678 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-19T07:07:04.678 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-19T07:07:04.678 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-19T07:07:04.678 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-19T07:07:04.678 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-19T07:07:04.679 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 103, 2024-09-19T07:07:04.679 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-19T07:07:04.679 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 23, 2024-09-19T07:07:04.679 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 8, 2024-09-19T07:07:04.679 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 13, 2024-09-19T07:07:04.679 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 5, 2024-09-19T07:07:04.679 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-19T07:07:04.679 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 17, 2024-09-19T07:07:04.679 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-09-19T07:07:04.679 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 2 2024-09-19T07:07:04.679 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.679 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-19T07:07:04.679 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-09-19T07:07:04.680 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:07:04.680 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-19T07:07:04.680 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-19T07:07:04.680 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.680 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-19T07:07:04.680 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 39 2024-09-19T07:07:04.680 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.680 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-19T07:07:04.680 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-19T07:07:04.680 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:07:04.680 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-19T07:07:04.680 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-09-19T07:07:04.681 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-09-19T07:07:04.681 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-09-19T07:07:04.681 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-19T07:07:04.681 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.681 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:07:04.681 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:04.681 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T07:07:04.681 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:04.681 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 1547, 2024-09-19T07:07:04.681 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 1675243, 2024-09-19T07:07:04.682 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:04.682 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 1547, 2024-09-19T07:07:04.682 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:07:04.682 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:07:04.682 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 1547, 2024-09-19T07:07:04.682 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1675243, 2024-09-19T07:07:04.682 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:04.682 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.682 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.682 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.682 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:04.682 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.683 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-19T07:07:04.683 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:04.683 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T07:07:04.683 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:04.683 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:07:04.683 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:07:04.683 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:04.683 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:04.683 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 47, 2024-09-19T07:07:04.684 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 13171, 2024-09-19T07:07:04.684 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 47, 2024-09-19T07:07:04.684 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 13171, 2024-09-19T07:07:04.684 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:04.684 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.684 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.684 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.684 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:04.684 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.684 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-19T07:07:04.684 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:04.684 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-19T07:07:04.685 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:04.685 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:07:04.685 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:07:04.685 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:04.685 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:04.685 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 47, 2024-09-19T07:07:04.685 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 47, 2024-09-19T07:07:04.685 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 47, 2024-09-19T07:07:04.685 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 47, 2024-09-19T07:07:04.686 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:04.686 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.686 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.686 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.686 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:04.686 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.686 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:07:04.686 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:04.686 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T07:07:04.686 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:04.686 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:07:04.687 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:07:04.687 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:04.687 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:04.687 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:07:04.687 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:07:04.687 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-19T07:07:04.687 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-19T07:07:04.687 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:04.687 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.688 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.688 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.688 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:04.688 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:04.688 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55f9f5e38960": { 2024-09-19T07:07:04.688 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:04.688 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T07:07:04.688 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:04.688 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 10, 2024-09-19T07:07:04.689 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 2724, 2024-09-19T07:07:04.689 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:04.689 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 10, 2024-09-19T07:07:04.689 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:07:04.689 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:07:04.689 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 9, 2024-09-19T07:07:04.689 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2724, 2024-09-19T07:07:04.689 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:04.689 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:04.689 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:04.690 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:04.690 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:04.690 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:04.690 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-19T07:07:05.442 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-19T07:07:05.443 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-19T07:07:05.443 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-09-19T07:07:05.443 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:07:05.443 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:07:05.443 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T07:07:05.443 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T07:07:05.695 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:07:06.059 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:06 smithi174 ceph-mon[35637]: pgmap v558: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:07:06.163 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-19T07:07:06.163 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:07:06.163 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 2184, 2024-09-19T07:07:06.163 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 2186, 2024-09-19T07:07:06.163 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 9041027, 2024-09-19T07:07:06.163 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 694522, 2024-09-19T07:07:06.163 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 6, 2024-09-19T07:07:06.163 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 2, 2024-09-19T07:07:06.163 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.234990277, 2024-09-19T07:07:06.163 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.068941500, 2024-09-19T07:07:06.163 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.070552383, 2024-09-19T07:07:06.164 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.074218022, 2024-09-19T07:07:06.164 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:06.164 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2184, 2024-09-19T07:07:06.164 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.090328817, 2024-09-19T07:07:06.164 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000041359 2024-09-19T07:07:06.164 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.164 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:06.164 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.164 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.164 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.164 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.164 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:07:06.165 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:07:06.165 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.165 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:07:06.165 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 973, 2024-09-19T07:07:06.165 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1218, 2024-09-19T07:07:06.165 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 9042751, 2024-09-19T07:07:06.165 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 906284, 2024-09-19T07:07:06.165 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 7, 2024-09-19T07:07:06.165 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-09-19T07:07:06.165 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.104500520, 2024-09-19T07:07:06.165 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.037720134, 2024-09-19T07:07:06.165 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.030740473, 2024-09-19T07:07:06.165 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.026592382, 2024-09-19T07:07:06.166 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:06.166 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1215, 2024-09-19T07:07:06.166 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.056261061, 2024-09-19T07:07:06.166 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000046305 2024-09-19T07:07:06.166 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.166 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:06.166 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.166 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.166 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.166 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.166 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T07:07:06.166 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 597440 2024-09-19T07:07:06.166 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.167 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:07:06.167 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1398, 2024-09-19T07:07:06.167 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1384, 2024-09-19T07:07:06.167 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 10173607, 2024-09-19T07:07:06.167 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 531985, 2024-09-19T07:07:06.167 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 4, 2024-09-19T07:07:06.167 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-09-19T07:07:06.167 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.161122732, 2024-09-19T07:07:06.167 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.056785295, 2024-09-19T07:07:06.167 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.079742393, 2024-09-19T07:07:06.167 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.029868074, 2024-09-19T07:07:06.167 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:06.168 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1384, 2024-09-19T07:07:06.168 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.066645452, 2024-09-19T07:07:06.168 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000048154 2024-09-19T07:07:06.168 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.168 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:06.168 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.168 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.168 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.168 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.169 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1526800, 2024-09-19T07:07:06.169 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 203808 2024-09-19T07:07:06.169 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.169 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-19T07:07:06.169 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-19T07:07:06.169 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-19T07:07:06.169 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.169 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-19T07:07:06.169 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T07:07:06.169 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T07:07:06.169 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 7115, 2024-09-19T07:07:06.169 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.199508737, 2024-09-19T07:07:06.169 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000028040 2024-09-19T07:07:06.170 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:06.170 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.170 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-19T07:07:06.170 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T07:07:06.170 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T07:07:06.170 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.170 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.170 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.170 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:06.170 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.170 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-19T07:07:06.170 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 0, 2024-09-19T07:07:06.170 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 0, 2024-09-19T07:07:06.170 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-19T07:07:06.171 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.171 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.171 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.171 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.171 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-19T07:07:06.171 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-09-19T07:07:06.171 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-19T07:07:06.171 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-19T07:07:06.171 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 0, 2024-09-19T07:07:06.171 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-09-19T07:07:06.171 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-19T07:07:06.171 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 37, 2024-09-19T07:07:06.171 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 16, 2024-09-19T07:07:06.172 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 21, 2024-09-19T07:07:06.172 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-09-19T07:07:06.172 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 9, 2024-09-19T07:07:06.172 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2024-09-19T07:07:06.172 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-09-19T07:07:06.172 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-09-19T07:07:06.172 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 2, 2024-09-19T07:07:06.172 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 0, 2024-09-19T07:07:06.172 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 0, 2024-09-19T07:07:06.172 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-09-19T07:07:06.172 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 0, 2024-09-19T07:07:06.172 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:07:06.172 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-19T07:07:06.173 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-09-19T07:07:06.173 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-19T07:07:06.173 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-19T07:07:06.173 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-09-19T07:07:06.173 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-09-19T07:07:06.173 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-09-19T07:07:06.173 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-09-19T07:07:06.173 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-19T07:07:06.173 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:07:06.173 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-19T07:07:06.173 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 4, 2024-09-19T07:07:06.173 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 234, 2024-09-19T07:07:06.174 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 7, 2024-09-19T07:07:06.174 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:07:06.174 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-19T07:07:06.174 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:07:06.174 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:07:06.174 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:07:06.174 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:07:06.174 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:07:06.174 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-09-19T07:07:06.174 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:07:06.174 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T07:07:06.174 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:07:06.174 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:07:06.174 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:07:06.175 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:07:06.175 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-09-19T07:07:06.175 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:07:06.175 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-09-19T07:07:06.175 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-09-19T07:07:06.175 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.175 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-19T07:07:06.175 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-19T07:07:06.175 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-19T07:07:06.175 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-19T07:07:06.175 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 0, 2024-09-19T07:07:06.175 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-09-19T07:07:06.175 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-19T07:07:06.175 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-19T07:07:06.176 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:07:06.176 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-09-19T07:07:06.176 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-09-19T07:07:06.176 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-19T07:07:06.176 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-19T07:07:06.176 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-19T07:07:06.176 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:07:06.176 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:07:06.176 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-19T07:07:06.176 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-19T07:07:06.176 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-09-19T07:07:06.176 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-09-19T07:07:06.176 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:07:06.176 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-09-19T07:07:06.177 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-19T07:07:06.177 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:07:06.177 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-19T07:07:06.177 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-19T07:07:06.177 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.177 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-19T07:07:06.177 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-19T07:07:06.177 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 0, 2024-09-19T07:07:06.177 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-19T07:07:06.177 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-19T07:07:06.177 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 2645, 2024-09-19T07:07:06.177 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-19T07:07:06.177 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-19T07:07:06.177 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 0, 2024-09-19T07:07:06.178 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-19T07:07:06.178 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-19T07:07:06.178 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-09-19T07:07:06.178 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 79, 2024-09-19T07:07:06.178 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-19T07:07:06.178 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-19T07:07:06.178 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 29691453, 2024-09-19T07:07:06.178 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 29768893, 2024-09-19T07:07:06.178 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 29768809, 2024-09-19T07:07:06.178 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-19T07:07:06.178 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.178 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.178 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.179 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.179 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 2645 2024-09-19T07:07:06.179 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.179 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-19T07:07:06.179 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 32, 2024-09-19T07:07:06.179 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 33, 2024-09-19T07:07:06.179 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 1, 2024-09-19T07:07:06.179 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 11, 2024-09-19T07:07:06.179 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 11, 2024-09-19T07:07:06.179 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 0, 2024-09-19T07:07:06.179 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 37, 2024-09-19T07:07:06.179 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 37, 2024-09-19T07:07:06.179 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 0, 2024-09-19T07:07:06.179 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-09-19T07:07:06.180 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 0, 2024-09-19T07:07:06.180 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2024-09-19T07:07:06.180 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 52120, 2024-09-19T07:07:06.180 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 215328 2024-09-19T07:07:06.180 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.180 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-19T07:07:06.180 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 0, 2024-09-19T07:07:06.180 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-09-19T07:07:06.180 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 0, 2024-09-19T07:07:06.180 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 0, 2024-09-19T07:07:06.180 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-09-19T07:07:06.180 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-19T07:07:06.180 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.180 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.181 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.181 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.181 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-19T07:07:06.181 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.181 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.181 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.181 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.181 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-19T07:07:06.181 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.181 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.181 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.181 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.181 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-19T07:07:06.182 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.182 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.182 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.182 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.182 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-19T07:07:06.182 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.182 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.182 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.182 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.182 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-19T07:07:06.182 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.182 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.182 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.183 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.183 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-19T07:07:06.183 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.183 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.183 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.183 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.183 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-19T07:07:06.183 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.183 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.183 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.183 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.183 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-19T07:07:06.183 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.183 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.184 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.184 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.184 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-19T07:07:06.184 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.184 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.184 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.184 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.184 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-19T07:07:06.184 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.184 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.184 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.184 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.184 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-19T07:07:06.185 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.185 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.185 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.185 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.185 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-19T07:07:06.186 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.186 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.186 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.186 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.186 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-19T07:07:06.186 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.186 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.186 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.186 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.186 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-19T07:07:06.186 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.186 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.186 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.187 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.187 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-19T07:07:06.187 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.187 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.187 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.187 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.187 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-19T07:07:06.187 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.187 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.187 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.187 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.187 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-19T07:07:06.187 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.187 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.188 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.188 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.188 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-19T07:07:06.188 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.188 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.188 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.188 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.188 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-19T07:07:06.188 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.188 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.188 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.188 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.188 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-19T07:07:06.188 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.189 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.189 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.189 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.189 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-19T07:07:06.189 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.189 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.189 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.189 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.189 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-19T07:07:06.189 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.189 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.189 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.189 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.190 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-19T07:07:06.190 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.190 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.190 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.190 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.190 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-19T07:07:06.190 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.190 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.190 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.190 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.190 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-19T07:07:06.190 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.190 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.190 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.191 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.191 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-19T07:07:06.191 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.191 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.191 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.191 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.191 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-19T07:07:06.191 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.191 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.191 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.191 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.191 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-19T07:07:06.191 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.192 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.192 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.192 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.192 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:07:06.192 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:07:06.192 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-19T07:07:06.192 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.192 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.192 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.192 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:06.192 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.192 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-19T07:07:06.192 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 2, 2024-09-19T07:07:06.192 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 6, 2024-09-19T07:07:06.193 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 4, 2024-09-19T07:07:06.193 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 5, 2024-09-19T07:07:06.193 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-19T07:07:06.193 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 0, 2024-09-19T07:07:06.193 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2024-09-19T07:07:06.193 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 4257, 2024-09-19T07:07:06.193 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-19T07:07:06.193 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.193 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-19T07:07:06.193 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:07:06.193 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-19T07:07:06.193 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:07:06.193 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:07:06.194 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:07:06.194 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:07:06.194 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:07:06.194 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:07:06.194 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:07:06.194 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:07:06.194 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:07:06.194 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:07:06.194 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:07:06.194 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:07:06.194 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:07:06.194 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-19T07:07:06.194 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:07:06.194 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-19T07:07:06.195 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:07:06.195 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:07:06.195 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:07:06.195 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:07:06.195 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:07:06.195 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:07:06.195 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:07:06.195 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-19T07:07:06.195 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:07:06.195 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:07:06.195 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:07:06.195 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-19T07:07:06.195 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-19T07:07:06.195 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-19T07:07:06.196 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:07:06.196 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:07:06.196 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:07:06.196 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:07:06.196 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 199441, 2024-09-19T07:07:06.196 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 89, 2024-09-19T07:07:06.196 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:07:06.196 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-19T07:07:06.196 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-19T07:07:06.196 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-19T07:07:06.196 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:07:06.196 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-19T07:07:06.196 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:07:06.196 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-19T07:07:06.197 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8192, 2024-09-19T07:07:06.197 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 180, 2024-09-19T07:07:06.197 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:07:06.197 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:07:06.197 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-19T07:07:06.197 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-19T07:07:06.197 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 152860, 2024-09-19T07:07:06.197 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 1194, 2024-09-19T07:07:06.197 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-19T07:07:06.197 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-19T07:07:06.197 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-19T07:07:06.197 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-19T07:07:06.197 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.198 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-19T07:07:06.198 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-19T07:07:06.198 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-19T07:07:06.198 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 4187, 2024-09-19T07:07:06.198 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 26, 2024-09-19T07:07:06.198 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 4, 2024-09-19T07:07:06.198 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 4187, 2024-09-19T07:07:06.198 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-19T07:07:06.198 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4187, 2024-09-19T07:07:06.198 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2.113972992, 2024-09-19T07:07:06.198 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000504889 2024-09-19T07:07:06.198 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.198 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-19T07:07:06.198 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-19T07:07:06.199 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4187, 2024-09-19T07:07:06.199 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 4188 2024-09-19T07:07:06.199 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.199 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 4187, 2024-09-19T07:07:06.199 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 4187, 2024-09-19T07:07:06.199 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 0, 2024-09-19T07:07:06.199 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-19T07:07:06.199 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-19T07:07:06.199 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 2326, 2024-09-19T07:07:06.199 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 0, 2024-09-19T07:07:06.199 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 1858, 2024-09-19T07:07:06.199 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 0, 2024-09-19T07:07:06.199 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 0, 2024-09-19T07:07:06.199 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-19T07:07:06.199 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-19T07:07:06.200 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-19T07:07:06.200 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-09-19T07:07:06.200 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 0, 2024-09-19T07:07:06.200 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-19T07:07:06.200 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-19T07:07:06.200 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-19T07:07:06.200 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 2, 2024-09-19T07:07:06.200 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 0, 2024-09-19T07:07:06.200 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:07:06.200 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-19T07:07:06.200 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:07:06.200 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-19T07:07:06.201 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-19T07:07:06.201 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-19T07:07:06.201 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:07:06.201 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-19T07:07:06.201 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:07:06.201 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-19T07:07:06.201 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-19T07:07:06.201 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-19T07:07:06.201 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-19T07:07:06.201 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-19T07:07:06.201 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-19T07:07:06.201 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-19T07:07:06.201 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-19T07:07:06.201 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-19T07:07:06.202 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-19T07:07:06.202 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-19T07:07:06.202 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-19T07:07:06.202 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-19T07:07:06.202 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-19T07:07:06.202 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-19T07:07:06.202 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-19T07:07:06.202 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-19T07:07:06.202 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 103, 2024-09-19T07:07:06.202 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-19T07:07:06.202 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 23, 2024-09-19T07:07:06.202 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 6, 2024-09-19T07:07:06.202 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 6, 2024-09-19T07:07:06.202 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-19T07:07:06.203 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-19T07:07:06.203 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 0, 2024-09-19T07:07:06.203 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 2, 2024-09-19T07:07:06.203 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 0 2024-09-19T07:07:06.203 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.203 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-19T07:07:06.203 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 0, 2024-09-19T07:07:06.203 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:07:06.203 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-19T07:07:06.203 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-19T07:07:06.203 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.203 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-19T07:07:06.203 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-09-19T07:07:06.204 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.204 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-19T07:07:06.204 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-19T07:07:06.204 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:07:06.204 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-19T07:07:06.204 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-09-19T07:07:06.204 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-09-19T07:07:06.204 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-09-19T07:07:06.204 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-19T07:07:06.204 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.204 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:07:06.204 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:06.204 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T07:07:06.205 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:06.205 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 4551, 2024-09-19T07:07:06.205 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 27826246, 2024-09-19T07:07:06.205 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:06.205 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 4551, 2024-09-19T07:07:06.205 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:07:06.205 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:07:06.205 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 4551, 2024-09-19T07:07:06.205 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 27826246, 2024-09-19T07:07:06.205 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:06.205 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.205 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.205 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.205 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:06.206 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.206 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-19T07:07:06.206 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:06.206 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T07:07:06.206 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:06.206 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:07:06.206 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:07:06.206 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:06.206 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:06.206 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 4187, 2024-09-19T07:07:06.206 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 25574599, 2024-09-19T07:07:06.206 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 4187, 2024-09-19T07:07:06.206 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 25574599, 2024-09-19T07:07:06.206 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:06.207 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.207 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.207 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.207 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:06.207 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.207 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-19T07:07:06.207 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:06.207 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-19T07:07:06.207 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:06.207 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:07:06.207 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:07:06.207 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:06.207 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:06.207 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 4187, 2024-09-19T07:07:06.208 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 4187, 2024-09-19T07:07:06.208 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 4187, 2024-09-19T07:07:06.208 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 4187, 2024-09-19T07:07:06.208 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:06.208 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.208 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.208 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.208 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:06.208 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.208 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:07:06.208 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:06.208 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T07:07:06.208 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:06.208 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:07:06.209 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:07:06.209 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:06.209 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:06.209 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:07:06.209 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:07:06.209 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-19T07:07:06.209 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-19T07:07:06.209 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:06.209 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.209 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.209 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.209 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:06.209 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:06.209 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x556d86d44b40": { 2024-09-19T07:07:06.209 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:06.210 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T07:07:06.210 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:06.210 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:07:06.210 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:07:06.210 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:06.210 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:06.210 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:07:06.210 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:07:06.210 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-19T07:07:06.210 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-19T07:07:06.210 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:06.210 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:06.210 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:06.210 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:06.211 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:06.211 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:06.211 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-19T07:07:06.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:06 smithi103 ceph-mon[31158]: pgmap v558: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:07:06.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:06 smithi134 ceph-mon[35356]: pgmap v558: 97 pgs: 97 active+clean; 1.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:07:06.926 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-19T07:07:06.926 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-19T07:07:06.926 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=7 2024-09-19T07:07:06.926 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:07:06.926 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:07:06.926 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T07:07:06.926 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T07:07:07.176 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:07:07.661 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-19T07:07:07.661 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:07:07.661 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 78, 2024-09-19T07:07:07.661 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 80, 2024-09-19T07:07:07.661 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 18283, 2024-09-19T07:07:07.661 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 432180, 2024-09-19T07:07:07.661 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 46, 2024-09-19T07:07:07.661 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-09-19T07:07:07.662 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.023799883, 2024-09-19T07:07:07.662 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.001701628, 2024-09-19T07:07:07.662 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.015425710, 2024-09-19T07:07:07.662 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000672180, 2024-09-19T07:07:07.662 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:07.662 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 79, 2024-09-19T07:07:07.662 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.018304404, 2024-09-19T07:07:07.662 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000231701 2024-09-19T07:07:07.662 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.662 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:07.662 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 60, 2024-09-19T07:07:07.662 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000031687, 2024-09-19T07:07:07.663 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000528 2024-09-19T07:07:07.663 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.663 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 6304, 2024-09-19T07:07:07.663 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 3680 2024-09-19T07:07:07.663 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.663 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:07:07.663 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1536, 2024-09-19T07:07:07.663 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 507, 2024-09-19T07:07:07.663 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 2006174, 2024-09-19T07:07:07.663 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 582156, 2024-09-19T07:07:07.663 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 34, 2024-09-19T07:07:07.663 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 8, 2024-09-19T07:07:07.664 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.219283701, 2024-09-19T07:07:07.664 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.054173072, 2024-09-19T07:07:07.664 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.107183445, 2024-09-19T07:07:07.664 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.066432464, 2024-09-19T07:07:07.664 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:07.664 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 506, 2024-09-19T07:07:07.664 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.037549746, 2024-09-19T07:07:07.664 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000074208 2024-09-19T07:07:07.664 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.664 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:07.664 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1198, 2024-09-19T07:07:07.664 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000225842, 2024-09-19T07:07:07.664 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000188 2024-09-19T07:07:07.665 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.665 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1543104, 2024-09-19T07:07:07.665 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 206880 2024-09-19T07:07:07.665 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.665 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:07:07.665 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1910, 2024-09-19T07:07:07.665 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1563, 2024-09-19T07:07:07.665 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 490095, 2024-09-19T07:07:07.665 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 1067290, 2024-09-19T07:07:07.665 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 10, 2024-09-19T07:07:07.665 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-09-19T07:07:07.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.149299939, 2024-09-19T07:07:07.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.063867913, 2024-09-19T07:07:07.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.089794749, 2024-09-19T07:07:07.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.007018746, 2024-09-19T07:07:07.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:07.666 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1560, 2024-09-19T07:07:07.666 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.072593455, 2024-09-19T07:07:07.666 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000046534 2024-09-19T07:07:07.666 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:07.666 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3138, 2024-09-19T07:07:07.666 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000648399, 2024-09-19T07:07:07.666 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000206 2024-09-19T07:07:07.667 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.667 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-09-19T07:07:07.667 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 598064 2024-09-19T07:07:07.667 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.667 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-19T07:07:07.667 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-19T07:07:07.667 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-19T07:07:07.667 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.667 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-19T07:07:07.667 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T07:07:07.667 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T07:07:07.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 52, 2024-09-19T07:07:07.667 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.001724416, 2024-09-19T07:07:07.668 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000033161 2024-09-19T07:07:07.668 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:07.668 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.668 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-19T07:07:07.668 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T07:07:07.668 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T07:07:07.668 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-09-19T07:07:07.668 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000008066, 2024-09-19T07:07:07.668 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000004033 2024-09-19T07:07:07.668 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:07.668 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.668 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-19T07:07:07.668 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 6, 2024-09-19T07:07:07.669 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 4, 2024-09-19T07:07:07.669 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-19T07:07:07.669 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4, 2024-09-19T07:07:07.669 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1.847274201, 2024-09-19T07:07:07.669 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.461818550 2024-09-19T07:07:07.669 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.669 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-19T07:07:07.669 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 2, 2024-09-19T07:07:07.669 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-19T07:07:07.669 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-19T07:07:07.669 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 11, 2024-09-19T07:07:07.669 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-09-19T07:07:07.669 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-19T07:07:07.669 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 14, 2024-09-19T07:07:07.670 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 0, 2024-09-19T07:07:07.670 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 14, 2024-09-19T07:07:07.670 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-09-19T07:07:07.670 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 14, 2024-09-19T07:07:07.670 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2024-09-19T07:07:07.670 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 2, 2024-09-19T07:07:07.670 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 2, 2024-09-19T07:07:07.670 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 2, 2024-09-19T07:07:07.670 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 37, 2024-09-19T07:07:07.670 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 33, 2024-09-19T07:07:07.670 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-09-19T07:07:07.670 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 2, 2024-09-19T07:07:07.670 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:07:07.670 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-19T07:07:07.671 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-09-19T07:07:07.671 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-19T07:07:07.671 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-19T07:07:07.671 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-09-19T07:07:07.671 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-09-19T07:07:07.671 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-09-19T07:07:07.671 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-09-19T07:07:07.671 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-19T07:07:07.671 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:07:07.671 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-19T07:07:07.671 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 5, 2024-09-19T07:07:07.671 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 298, 2024-09-19T07:07:07.671 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 7, 2024-09-19T07:07:07.671 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:07:07.672 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-19T07:07:07.672 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:07:07.672 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:07:07.672 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:07:07.672 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:07:07.672 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:07:07.672 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-09-19T07:07:07.672 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:07:07.672 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 6, 2024-09-19T07:07:07.672 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 5, 2024-09-19T07:07:07.672 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:07:07.672 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:07:07.672 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:07:07.672 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 6, 2024-09-19T07:07:07.673 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:07:07.673 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-09-19T07:07:07.673 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 2 2024-09-19T07:07:07.673 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.673 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-19T07:07:07.673 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-19T07:07:07.673 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-19T07:07:07.673 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-19T07:07:07.673 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 3, 2024-09-19T07:07:07.673 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-09-19T07:07:07.673 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-19T07:07:07.673 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-19T07:07:07.673 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:07:07.673 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-09-19T07:07:07.674 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-09-19T07:07:07.674 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-19T07:07:07.674 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-19T07:07:07.674 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-19T07:07:07.674 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:07:07.674 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:07:07.674 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-19T07:07:07.674 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-19T07:07:07.674 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 14, 2024-09-19T07:07:07.674 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 70, 2024-09-19T07:07:07.674 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:07:07.674 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-09-19T07:07:07.674 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-19T07:07:07.674 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:07:07.675 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-19T07:07:07.675 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-19T07:07:07.675 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.675 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-19T07:07:07.675 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-19T07:07:07.675 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 11, 2024-09-19T07:07:07.675 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-19T07:07:07.675 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-19T07:07:07.675 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 11, 2024-09-19T07:07:07.675 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-19T07:07:07.675 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-19T07:07:07.675 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 2, 2024-09-19T07:07:07.675 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-19T07:07:07.675 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-19T07:07:07.676 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-09-19T07:07:07.676 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-09-19T07:07:07.676 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-19T07:07:07.676 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-19T07:07:07.676 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-09-19T07:07:07.676 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 4197336, 2024-09-19T07:07:07.676 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-09-19T07:07:07.676 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-19T07:07:07.676 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 10, 2024-09-19T07:07:07.676 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.028466879, 2024-09-19T07:07:07.676 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002846687 2024-09-19T07:07:07.676 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.676 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-09-19T07:07:07.677 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.677 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-19T07:07:07.677 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 17, 2024-09-19T07:07:07.677 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 17, 2024-09-19T07:07:07.677 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 0, 2024-09-19T07:07:07.677 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 15, 2024-09-19T07:07:07.677 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 16, 2024-09-19T07:07:07.677 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 1, 2024-09-19T07:07:07.677 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 14, 2024-09-19T07:07:07.677 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 14, 2024-09-19T07:07:07.677 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 0, 2024-09-19T07:07:07.677 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 2, 2024-09-19T07:07:07.677 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 3, 2024-09-19T07:07:07.678 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 1, 2024-09-19T07:07:07.678 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 54472, 2024-09-19T07:07:07.678 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223520 2024-09-19T07:07:07.678 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.678 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-19T07:07:07.678 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 10, 2024-09-19T07:07:07.678 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-09-19T07:07:07.678 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 6, 2024-09-19T07:07:07.678 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 29, 2024-09-19T07:07:07.678 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 1, 2024-09-19T07:07:07.678 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-19T07:07:07.678 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.678 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.678 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.679 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.679 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-19T07:07:07.679 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3, 2024-09-19T07:07:07.679 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1.847131246, 2024-09-19T07:07:07.679 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.615710415 2024-09-19T07:07:07.679 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.679 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-19T07:07:07.679 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.679 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.679 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.679 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.679 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-19T07:07:07.679 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.679 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.680 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.680 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.680 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-19T07:07:07.680 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.680 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.680 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.680 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.680 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-19T07:07:07.680 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.680 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.680 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.680 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.680 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-19T07:07:07.681 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1, 2024-09-19T07:07:07.681 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000142955, 2024-09-19T07:07:07.681 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000142955 2024-09-19T07:07:07.681 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.681 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-19T07:07:07.681 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.681 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.681 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.681 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.681 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-19T07:07:07.681 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.681 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.681 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.681 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.682 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-19T07:07:07.682 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.682 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.682 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.682 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.682 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-19T07:07:07.682 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.682 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.682 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.682 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.682 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-19T07:07:07.683 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.683 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.683 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.683 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.683 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-19T07:07:07.683 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.683 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.683 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.683 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.683 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-19T07:07:07.684 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.684 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.684 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.684 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.684 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-19T07:07:07.684 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.684 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.684 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.684 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.684 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-19T07:07:07.684 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.684 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.684 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.684 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.685 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-19T07:07:07.685 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.685 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.685 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.685 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.685 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-19T07:07:07.685 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.685 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.685 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.685 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.685 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-19T07:07:07.685 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.685 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.685 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.686 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.686 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-19T07:07:07.686 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.686 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.686 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.686 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.686 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-19T07:07:07.686 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.686 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.686 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.686 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.686 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-19T07:07:07.686 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.687 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.687 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.687 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.687 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-19T07:07:07.687 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.687 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.687 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.687 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.687 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-19T07:07:07.687 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.687 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.687 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.687 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.687 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-19T07:07:07.688 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.688 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.688 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.688 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.688 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-19T07:07:07.688 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.688 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.688 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.688 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.688 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-19T07:07:07.688 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.688 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.688 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.688 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.689 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-19T07:07:07.689 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.689 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.689 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.689 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.689 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-19T07:07:07.689 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.689 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.689 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.689 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.689 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:07:07.689 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:07:07.689 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-19T07:07:07.690 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.690 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.690 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.690 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:07.690 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.690 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-19T07:07:07.690 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-09-19T07:07:07.690 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 5, 2024-09-19T07:07:07.690 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 4, 2024-09-19T07:07:07.690 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-09-19T07:07:07.690 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-19T07:07:07.690 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 2, 2024-09-19T07:07:07.690 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2024-09-19T07:07:07.690 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 1074, 2024-09-19T07:07:07.691 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-19T07:07:07.691 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.691 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-19T07:07:07.691 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:07:07.691 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-19T07:07:07.691 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:07:07.691 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:07:07.691 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:07:07.691 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:07:07.691 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:07:07.691 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:07:07.691 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:07:07.691 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:07:07.691 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:07:07.692 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:07:07.692 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:07:07.692 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:07:07.692 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:07:07.692 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-19T07:07:07.692 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:07:07.692 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-19T07:07:07.692 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:07:07.692 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:07:07.692 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:07:07.692 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:07:07.692 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:07:07.692 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:07:07.692 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:07:07.693 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-19T07:07:07.693 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:07:07.693 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:07:07.693 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:07:07.693 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-19T07:07:07.693 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-19T07:07:07.693 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-19T07:07:07.693 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:07:07.693 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:07:07.693 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:07:07.693 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:07:07.693 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 255319, 2024-09-19T07:07:07.693 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 86, 2024-09-19T07:07:07.693 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:07:07.694 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-19T07:07:07.694 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-19T07:07:07.694 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-19T07:07:07.694 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:07:07.694 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-19T07:07:07.694 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:07:07.694 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-19T07:07:07.694 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8192, 2024-09-19T07:07:07.694 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 180, 2024-09-19T07:07:07.694 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:07:07.694 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:07:07.694 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-19T07:07:07.694 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-19T07:07:07.694 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 68437, 2024-09-19T07:07:07.695 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 294, 2024-09-19T07:07:07.695 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-19T07:07:07.695 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-19T07:07:07.695 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-19T07:07:07.695 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-19T07:07:07.695 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.695 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-19T07:07:07.695 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-19T07:07:07.695 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-19T07:07:07.695 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 49, 2024-09-19T07:07:07.695 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 13800, 2024-09-19T07:07:07.695 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 30, 2024-09-19T07:07:07.695 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 49, 2024-09-19T07:07:07.695 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-19T07:07:07.696 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 49, 2024-09-19T07:07:07.696 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.208997332, 2024-09-19T07:07:07.696 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004265251 2024-09-19T07:07:07.696 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.696 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-19T07:07:07.696 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-19T07:07:07.696 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 49, 2024-09-19T07:07:07.697 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 74 2024-09-19T07:07:07.697 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.697 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 49, 2024-09-19T07:07:07.697 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 0, 2024-09-19T07:07:07.697 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 49, 2024-09-19T07:07:07.697 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-19T07:07:07.697 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-19T07:07:07.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-09-19T07:07:07.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-09-19T07:07:07.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 0, 2024-09-19T07:07:07.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 10, 2024-09-19T07:07:07.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 7, 2024-09-19T07:07:07.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-19T07:07:07.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-19T07:07:07.698 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-19T07:07:07.698 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-09-19T07:07:07.698 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 6, 2024-09-19T07:07:07.698 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-19T07:07:07.698 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-19T07:07:07.698 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-19T07:07:07.698 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 0, 2024-09-19T07:07:07.698 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-09-19T07:07:07.698 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:07:07.698 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-19T07:07:07.698 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:07:07.698 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-19T07:07:07.698 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-19T07:07:07.699 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-19T07:07:07.699 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:07:07.699 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-19T07:07:07.699 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:07:07.699 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-19T07:07:07.699 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-19T07:07:07.699 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-19T07:07:07.699 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-19T07:07:07.699 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-19T07:07:07.699 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-19T07:07:07.699 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-19T07:07:07.699 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-19T07:07:07.699 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-19T07:07:07.699 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-19T07:07:07.700 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-19T07:07:07.700 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-19T07:07:07.700 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-19T07:07:07.700 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-19T07:07:07.700 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-19T07:07:07.700 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-19T07:07:07.700 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-19T07:07:07.700 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 103, 2024-09-19T07:07:07.700 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-19T07:07:07.700 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 23, 2024-09-19T07:07:07.700 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 8, 2024-09-19T07:07:07.700 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 11, 2024-09-19T07:07:07.700 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 3, 2024-09-19T07:07:07.701 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-19T07:07:07.701 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 19, 2024-09-19T07:07:07.701 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-09-19T07:07:07.701 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 2 2024-09-19T07:07:07.701 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.701 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-19T07:07:07.701 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-09-19T07:07:07.701 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:07:07.701 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-19T07:07:07.701 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-19T07:07:07.701 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.701 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-19T07:07:07.701 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 78 2024-09-19T07:07:07.701 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.702 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-19T07:07:07.702 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-19T07:07:07.702 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:07:07.702 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-19T07:07:07.702 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-09-19T07:07:07.702 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-09-19T07:07:07.702 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-09-19T07:07:07.702 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-19T07:07:07.702 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.702 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:07:07.702 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:07.702 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T07:07:07.702 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:07.702 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 3520, 2024-09-19T07:07:07.703 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 2174100, 2024-09-19T07:07:07.703 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:07.703 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 3520, 2024-09-19T07:07:07.703 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:07:07.703 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:07:07.703 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 3520, 2024-09-19T07:07:07.703 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2174100, 2024-09-19T07:07:07.703 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:07.703 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.703 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.703 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.703 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:07.703 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.703 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-19T07:07:07.703 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:07.704 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T07:07:07.704 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:07.704 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:07:07.704 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:07:07.704 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:07.704 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:07.704 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 49, 2024-09-19T07:07:07.704 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 13800, 2024-09-19T07:07:07.704 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 49, 2024-09-19T07:07:07.704 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 13800, 2024-09-19T07:07:07.704 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:07.704 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.704 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.705 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.705 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:07.705 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.705 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-19T07:07:07.705 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:07.705 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-19T07:07:07.705 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:07.705 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:07:07.705 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:07:07.705 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:07.705 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:07.705 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 49, 2024-09-19T07:07:07.705 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 49, 2024-09-19T07:07:07.705 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 49, 2024-09-19T07:07:07.706 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 49, 2024-09-19T07:07:07.706 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:07.706 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.706 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.706 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.706 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:07.706 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.706 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:07:07.706 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:07.706 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T07:07:07.706 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:07.706 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:07:07.706 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:07:07.706 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:07.707 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:07.707 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:07:07.707 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:07:07.707 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-19T07:07:07.707 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-19T07:07:07.707 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:07.707 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.707 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.707 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.707 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:07.707 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:07.707 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55cb98f7c960": { 2024-09-19T07:07:07.707 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:07.707 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T07:07:07.708 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:07.708 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 11, 2024-09-19T07:07:07.708 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 3032, 2024-09-19T07:07:07.708 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:07.708 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 11, 2024-09-19T07:07:07.708 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:07:07.708 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:07:07.708 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 10, 2024-09-19T07:07:07.708 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 3032, 2024-09-19T07:07:07.708 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:07.708 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:07.708 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:07.708 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:07.708 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:07.709 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:07.709 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-19T07:07:08.314 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:08 smithi174 ceph-mon[35637]: pgmap v559: 97 pgs: 97 active+clean; 1.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T07:07:08.341 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-19T07:07:08.341 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-19T07:07:08.341 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-09-19T07:07:08.341 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:07:08.341 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:07:08.341 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T07:07:08.342 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T07:07:08.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:08 smithi103 ceph-mon[31158]: pgmap v559: 97 pgs: 97 active+clean; 1.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T07:07:08.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:08 smithi134 ceph-mon[35356]: pgmap v559: 97 pgs: 97 active+clean; 1.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T07:07:08.586 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:07:09.017 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-19T07:07:09.017 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:07:09.017 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1184, 2024-09-19T07:07:09.018 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1184, 2024-09-19T07:07:09.018 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 315759, 2024-09-19T07:07:09.018 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 375752, 2024-09-19T07:07:09.018 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 7, 2024-09-19T07:07:09.018 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 2, 2024-09-19T07:07:09.018 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.118432447, 2024-09-19T07:07:09.018 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.037875681, 2024-09-19T07:07:09.018 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.033286308, 2024-09-19T07:07:09.018 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.034767152, 2024-09-19T07:07:09.018 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:09.018 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1183, 2024-09-19T07:07:09.018 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.058077430, 2024-09-19T07:07:09.018 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000049093 2024-09-19T07:07:09.018 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.019 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:09.019 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.019 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.019 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.019 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.019 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T07:07:09.019 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T07:07:09.019 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.019 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:07:09.019 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1443, 2024-09-19T07:07:09.019 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1431, 2024-09-19T07:07:09.019 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1789940, 2024-09-19T07:07:09.019 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 548891, 2024-09-19T07:07:09.019 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 4, 2024-09-19T07:07:09.020 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-09-19T07:07:09.020 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.177179893, 2024-09-19T07:07:09.020 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.056065344, 2024-09-19T07:07:09.020 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.081041666, 2024-09-19T07:07:09.020 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.044823610, 2024-09-19T07:07:09.020 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:09.020 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1430, 2024-09-19T07:07:09.020 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.063154421, 2024-09-19T07:07:09.020 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000044163 2024-09-19T07:07:09.020 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.020 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:09.020 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.020 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.021 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.021 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.021 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1524976, 2024-09-19T07:07:09.021 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 206400 2024-09-19T07:07:09.021 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.021 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:07:09.021 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1173, 2024-09-19T07:07:09.021 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1420, 2024-09-19T07:07:09.021 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 393776, 2024-09-19T07:07:09.021 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 968554, 2024-09-19T07:07:09.021 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 6, 2024-09-19T07:07:09.021 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-09-19T07:07:09.021 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.161911866, 2024-09-19T07:07:09.021 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.055452308, 2024-09-19T07:07:09.022 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.040393758, 2024-09-19T07:07:09.022 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.048487962, 2024-09-19T07:07:09.022 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:07:09.022 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1417, 2024-09-19T07:07:09.022 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.092053679, 2024-09-19T07:07:09.022 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000064963 2024-09-19T07:07:09.022 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.022 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:07:09.022 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.022 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.022 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.022 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.022 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T07:07:09.022 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 596176 2024-09-19T07:07:09.023 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.023 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-19T07:07:09.023 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-19T07:07:09.023 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-19T07:07:09.023 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.023 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-19T07:07:09.023 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T07:07:09.023 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T07:07:09.023 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5661, 2024-09-19T07:07:09.023 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.179835849, 2024-09-19T07:07:09.023 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000031767 2024-09-19T07:07:09.023 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:09.023 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.024 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-19T07:07:09.024 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T07:07:09.024 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T07:07:09.024 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.024 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.024 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.024 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:09.024 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.024 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-19T07:07:09.024 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 0, 2024-09-19T07:07:09.024 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 0, 2024-09-19T07:07:09.024 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-19T07:07:09.024 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.024 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.025 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.025 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.025 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-19T07:07:09.025 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-09-19T07:07:09.025 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-19T07:07:09.025 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-19T07:07:09.025 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 0, 2024-09-19T07:07:09.025 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-09-19T07:07:09.025 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-19T07:07:09.025 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 0, 2024-09-19T07:07:09.025 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 0, 2024-09-19T07:07:09.025 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 0, 2024-09-19T07:07:09.025 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-09-19T07:07:09.025 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 0, 2024-09-19T07:07:09.026 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2024-09-19T07:07:09.026 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-09-19T07:07:09.026 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-09-19T07:07:09.026 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 1, 2024-09-19T07:07:09.026 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 0, 2024-09-19T07:07:09.026 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 0, 2024-09-19T07:07:09.026 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-09-19T07:07:09.026 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 0, 2024-09-19T07:07:09.026 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:07:09.026 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-19T07:07:09.026 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-09-19T07:07:09.026 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-19T07:07:09.026 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-19T07:07:09.026 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-09-19T07:07:09.027 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-09-19T07:07:09.027 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-09-19T07:07:09.027 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-09-19T07:07:09.027 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-19T07:07:09.027 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:07:09.027 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-19T07:07:09.027 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 0, 2024-09-19T07:07:09.027 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 0, 2024-09-19T07:07:09.027 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-09-19T07:07:09.027 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:07:09.027 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-19T07:07:09.027 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:07:09.027 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:07:09.027 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:07:09.028 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:07:09.028 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:07:09.028 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-09-19T07:07:09.028 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:07:09.028 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T07:07:09.028 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:07:09.028 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:07:09.028 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:07:09.028 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:07:09.028 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-09-19T07:07:09.028 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:07:09.028 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-09-19T07:07:09.028 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-09-19T07:07:09.028 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.029 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-19T07:07:09.029 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-19T07:07:09.029 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-19T07:07:09.029 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-19T07:07:09.029 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 0, 2024-09-19T07:07:09.029 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-09-19T07:07:09.029 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-19T07:07:09.029 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-19T07:07:09.029 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:07:09.029 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-09-19T07:07:09.029 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-09-19T07:07:09.029 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-19T07:07:09.029 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-19T07:07:09.029 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-19T07:07:09.030 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:07:09.030 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:07:09.030 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-19T07:07:09.030 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-19T07:07:09.030 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-09-19T07:07:09.030 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-09-19T07:07:09.030 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:07:09.030 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-09-19T07:07:09.030 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-19T07:07:09.030 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:07:09.030 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-19T07:07:09.030 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-19T07:07:09.030 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.030 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-19T07:07:09.031 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-19T07:07:09.031 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 0, 2024-09-19T07:07:09.031 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-19T07:07:09.031 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-19T07:07:09.031 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 11, 2024-09-19T07:07:09.031 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-19T07:07:09.031 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-19T07:07:09.031 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 0, 2024-09-19T07:07:09.031 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-19T07:07:09.031 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-19T07:07:09.031 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-09-19T07:07:09.031 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-09-19T07:07:09.031 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-19T07:07:09.031 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-19T07:07:09.032 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-09-19T07:07:09.032 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 4197336, 2024-09-19T07:07:09.032 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 4197195, 2024-09-19T07:07:09.032 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-19T07:07:09.032 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.032 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.032 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.032 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.032 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 11 2024-09-19T07:07:09.032 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.032 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-19T07:07:09.032 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 3, 2024-09-19T07:07:09.032 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 3, 2024-09-19T07:07:09.033 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 0, 2024-09-19T07:07:09.033 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 1, 2024-09-19T07:07:09.033 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 1, 2024-09-19T07:07:09.033 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 0, 2024-09-19T07:07:09.033 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 0, 2024-09-19T07:07:09.033 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 0, 2024-09-19T07:07:09.033 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 0, 2024-09-19T07:07:09.033 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-09-19T07:07:09.033 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 0, 2024-09-19T07:07:09.033 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2024-09-19T07:07:09.033 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 48900, 2024-09-19T07:07:09.033 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 215328 2024-09-19T07:07:09.033 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.033 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-19T07:07:09.034 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 0, 2024-09-19T07:07:09.034 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-09-19T07:07:09.034 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 0, 2024-09-19T07:07:09.034 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 0, 2024-09-19T07:07:09.034 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-09-19T07:07:09.034 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-19T07:07:09.034 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.034 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.034 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.034 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.034 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-19T07:07:09.034 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.034 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.034 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.034 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.035 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-19T07:07:09.035 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.035 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.035 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.035 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.035 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-19T07:07:09.035 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.035 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.035 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.035 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.035 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-19T07:07:09.035 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.035 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.036 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.036 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.036 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-19T07:07:09.036 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.036 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.036 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.036 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.036 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-19T07:07:09.036 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.036 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.036 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.036 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.036 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-19T07:07:09.036 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.037 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.037 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.037 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.037 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-19T07:07:09.037 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.037 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.037 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.037 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.037 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-19T07:07:09.037 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.037 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.037 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.037 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.037 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-19T07:07:09.038 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.038 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.038 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.038 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.038 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-19T07:07:09.038 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.038 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.039 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.039 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.039 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-19T07:07:09.039 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.039 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.039 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.039 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.039 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-19T07:07:09.039 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.039 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.039 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.039 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.040 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-19T07:07:09.040 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.040 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.040 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.040 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.040 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-19T07:07:09.040 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.040 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.040 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.040 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.040 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-19T07:07:09.040 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.040 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.040 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.041 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.041 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-19T07:07:09.041 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.041 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.041 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.041 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.041 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-19T07:07:09.041 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.041 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.041 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.041 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.041 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-19T07:07:09.041 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.042 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.042 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.042 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.042 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-19T07:07:09.042 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.042 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.042 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.042 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.042 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-19T07:07:09.042 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.042 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.042 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.042 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.043 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-19T07:07:09.043 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.043 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.043 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.043 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.043 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-19T07:07:09.043 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.043 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.043 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.043 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.043 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-19T07:07:09.043 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.043 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.043 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.044 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.044 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-19T07:07:09.044 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.044 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.044 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.044 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.044 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-19T07:07:09.044 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.044 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.044 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.044 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.044 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-19T07:07:09.044 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.045 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.045 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.045 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.045 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-19T07:07:09.045 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.045 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.045 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.045 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.045 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:07:09.045 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:07:09.045 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-19T07:07:09.045 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.045 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.046 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.046 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:09.046 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.046 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-19T07:07:09.046 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-09-19T07:07:09.046 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 5, 2024-09-19T07:07:09.046 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 4, 2024-09-19T07:07:09.046 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 4, 2024-09-19T07:07:09.046 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-19T07:07:09.046 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 0, 2024-09-19T07:07:09.046 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2024-09-19T07:07:09.046 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 10891, 2024-09-19T07:07:09.046 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-19T07:07:09.046 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.047 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-19T07:07:09.047 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:07:09.047 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-19T07:07:09.047 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:07:09.047 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:07:09.047 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:07:09.047 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:07:09.047 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:07:09.047 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:07:09.047 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:07:09.047 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:07:09.047 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:07:09.047 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:07:09.047 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:07:09.048 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:07:09.048 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:07:09.048 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-19T07:07:09.048 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:07:09.048 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-19T07:07:09.048 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:07:09.048 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:07:09.048 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:07:09.048 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:07:09.048 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:07:09.048 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:07:09.048 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:07:09.049 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-19T07:07:09.049 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:07:09.049 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:07:09.049 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:07:09.049 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-19T07:07:09.049 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-19T07:07:09.049 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-19T07:07:09.050 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:07:09.050 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:07:09.050 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:07:09.050 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:07:09.050 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 190177, 2024-09-19T07:07:09.050 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 53, 2024-09-19T07:07:09.050 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:07:09.050 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-19T07:07:09.050 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-19T07:07:09.050 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-19T07:07:09.050 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:07:09.050 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-19T07:07:09.050 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:07:09.050 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-19T07:07:09.051 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8192, 2024-09-19T07:07:09.051 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 180, 2024-09-19T07:07:09.051 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:07:09.051 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:07:09.051 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-19T07:07:09.051 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-19T07:07:09.051 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 8512, 2024-09-19T07:07:09.051 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 13, 2024-09-19T07:07:09.051 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-19T07:07:09.051 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-19T07:07:09.051 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-19T07:07:09.051 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-19T07:07:09.051 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.052 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-19T07:07:09.052 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-19T07:07:09.052 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-19T07:07:09.052 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 3428, 2024-09-19T07:07:09.052 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 21, 2024-09-19T07:07:09.052 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 3, 2024-09-19T07:07:09.052 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 3428, 2024-09-19T07:07:09.052 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-19T07:07:09.052 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3428, 2024-09-19T07:07:09.052 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1.851976367, 2024-09-19T07:07:09.052 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000540249 2024-09-19T07:07:09.052 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.052 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-19T07:07:09.053 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-19T07:07:09.053 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3428, 2024-09-19T07:07:09.053 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 3429 2024-09-19T07:07:09.053 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.053 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 3428, 2024-09-19T07:07:09.053 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 3428, 2024-09-19T07:07:09.053 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 0, 2024-09-19T07:07:09.053 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-19T07:07:09.053 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-19T07:07:09.053 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 2246, 2024-09-19T07:07:09.053 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 0, 2024-09-19T07:07:09.053 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 1180, 2024-09-19T07:07:09.053 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 0, 2024-09-19T07:07:09.053 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 0, 2024-09-19T07:07:09.054 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-19T07:07:09.054 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-19T07:07:09.054 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-19T07:07:09.054 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-09-19T07:07:09.054 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 0, 2024-09-19T07:07:09.054 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-19T07:07:09.054 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-19T07:07:09.054 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-19T07:07:09.054 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 1, 2024-09-19T07:07:09.054 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 0, 2024-09-19T07:07:09.054 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:07:09.054 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-19T07:07:09.054 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:07:09.054 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-19T07:07:09.054 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-19T07:07:09.055 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-19T07:07:09.055 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:07:09.055 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-19T07:07:09.055 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:07:09.055 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-19T07:07:09.055 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-19T07:07:09.055 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-19T07:07:09.055 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-19T07:07:09.055 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-19T07:07:09.055 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-19T07:07:09.055 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-19T07:07:09.055 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-19T07:07:09.055 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-19T07:07:09.055 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-19T07:07:09.056 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-19T07:07:09.056 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-19T07:07:09.056 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-19T07:07:09.056 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-19T07:07:09.056 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-19T07:07:09.056 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-19T07:07:09.056 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-19T07:07:09.056 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 103, 2024-09-19T07:07:09.056 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-19T07:07:09.056 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 23, 2024-09-19T07:07:09.056 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 5, 2024-09-19T07:07:09.056 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 6, 2024-09-19T07:07:09.056 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 1, 2024-09-19T07:07:09.057 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-19T07:07:09.057 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 0, 2024-09-19T07:07:09.057 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 2, 2024-09-19T07:07:09.057 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 0 2024-09-19T07:07:09.057 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.057 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-19T07:07:09.057 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 0, 2024-09-19T07:07:09.057 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:07:09.057 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-19T07:07:09.057 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-19T07:07:09.057 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.057 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-19T07:07:09.057 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-09-19T07:07:09.057 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.058 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-19T07:07:09.058 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-19T07:07:09.058 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:07:09.058 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-19T07:07:09.058 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-09-19T07:07:09.058 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-09-19T07:07:09.058 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-09-19T07:07:09.058 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-19T07:07:09.058 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.058 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:07:09.058 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:09.058 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T07:07:09.058 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:09.058 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 3796, 2024-09-19T07:07:09.058 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 2136072, 2024-09-19T07:07:09.059 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:09.059 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 3796, 2024-09-19T07:07:09.059 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:07:09.059 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:07:09.059 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 3796, 2024-09-19T07:07:09.059 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2136072, 2024-09-19T07:07:09.059 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:09.059 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.059 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.059 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.059 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:09.059 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.059 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-19T07:07:09.059 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:09.060 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T07:07:09.060 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:09.060 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:07:09.060 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:07:09.060 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:09.060 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:09.060 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 3428, 2024-09-19T07:07:09.060 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 3037, 2024-09-19T07:07:09.060 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 3428, 2024-09-19T07:07:09.060 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 3037, 2024-09-19T07:07:09.060 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:09.060 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.060 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.061 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.061 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:09.061 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.061 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-19T07:07:09.061 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:09.061 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-19T07:07:09.061 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:09.061 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:07:09.061 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:07:09.061 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:09.061 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:09.061 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 3428, 2024-09-19T07:07:09.061 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 3428, 2024-09-19T07:07:09.061 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 3428, 2024-09-19T07:07:09.061 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 3428, 2024-09-19T07:07:09.062 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:09.062 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.062 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.062 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.062 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:09.062 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.062 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:07:09.062 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:09.062 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T07:07:09.062 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:09.062 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:07:09.062 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:07:09.062 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:09.062 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:09.063 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:07:09.063 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:07:09.063 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-19T07:07:09.063 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-19T07:07:09.063 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:09.063 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.063 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.063 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.063 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:09.063 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:07:09.063 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x5607fd9e4b40": { 2024-09-19T07:07:09.063 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:07:09.063 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T07:07:09.063 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:07:09.064 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:07:09.064 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:07:09.064 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:07:09.064 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:07:09.064 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:07:09.064 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:07:09.064 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-19T07:07:09.064 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-19T07:07:09.064 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:07:09.064 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:07:09.064 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:07:09.064 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:07:09.064 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:09.065 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:07:09.065 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-19T07:07:09.907 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-19T07:07:09.907 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-19T07:07:09.908 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-19T07:07:09.908 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:07:09.908 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:07:09.908 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-19T07:07:09.920 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-19T07:07:09.920 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:07:09.920 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:07:09.943 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:07:09.943 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:07:10.003 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T07:07:10.310 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:07:10.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:10 smithi174 ceph-mon[35637]: pgmap v560: 97 pgs: 97 active+clean; 1.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T07:07:10.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:10 smithi103 ceph-mon[31158]: pgmap v560: 97 pgs: 97 active+clean; 1.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T07:07:10.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:10 smithi134 ceph-mon[35356]: pgmap v560: 97 pgs: 97 active+clean; 1.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T07:07:10.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.770+0000 7f8e8b71e640 1 -- 172.21.15.103:0/1919507555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e84106f50 msgr2=0x7f8e8410eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:10.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.770+0000 7f8e8b71e640 1 --2- 172.21.15.103:0/1919507555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e84106f50 0x7f8e8410eaf0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009cd0 tx=0x7f8e7c02a8e0 comp rx=0 tx=0).stop 2024-09-19T07:07:10.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.771+0000 7f8e8b71e640 1 -- 172.21.15.103:0/1919507555 shutdown_connections 2024-09-19T07:07:10.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.771+0000 7f8e8b71e640 1 --2- 172.21.15.103:0/1919507555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e84106f50 0x7f8e8410eaf0 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:10.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.771+0000 7f8e8b71e640 1 --2- 172.21.15.103:0/1919507555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e84100ca0 0x7f8e84106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:10.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.771+0000 7f8e8b71e640 1 --2- 172.21.15.103:0/1919507555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e841002f0 0x7f8e841006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:10.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.771+0000 7f8e8b71e640 1 -- 172.21.15.103:0/1919507555 >> 172.21.15.103:0/1919507555 conn(0x7f8e84077210 msgr2=0x7f8e840fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:07:10.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.771+0000 7f8e8b71e640 1 -- 172.21.15.103:0/1919507555 shutdown_connections 2024-09-19T07:07:10.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.771+0000 7f8e8b71e640 1 -- 172.21.15.103:0/1919507555 wait complete. 2024-09-19T07:07:10.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.771+0000 7f8e8b71e640 1 Processor -- start 2024-09-19T07:07:10.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.771+0000 7f8e8b71e640 1 -- start start 2024-09-19T07:07:10.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.771+0000 7f8e8b71e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e841002f0 0x7f8e841983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:10.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.771+0000 7f8e8b71e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e84100ca0 0x7f8e84198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:10.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.772+0000 7f8e8b71e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e84106f50 0x7f8e8419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:10.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.772+0000 7f8e8b71e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84111fa0 con 0x7f8e84106f50 2024-09-19T07:07:10.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.772+0000 7f8e8b71e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84111e20 con 0x7f8e84100ca0 2024-09-19T07:07:10.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.772+0000 7f8e8b71e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84112120 con 0x7f8e841002f0 2024-09-19T07:07:10.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.772+0000 7f8e89cbb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e84106f50 0x7f8e8419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:10.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.772+0000 7f8e894ba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e841002f0 0x7f8e841983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:10.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.772+0000 7f8e89cbb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e84106f50 0x7f8e8419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51654/0 (socket says 172.21.15.103:51654) 2024-09-19T07:07:10.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.772+0000 7f8e89cbb640 1 -- 172.21.15.103:0/38533443 learned_addr learned my addr 172.21.15.103:0/38533443 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:07:10.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.772+0000 7f8e88cb9640 1 --2- 172.21.15.103:0/38533443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e84100ca0 0x7f8e84198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:10.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.772+0000 7f8e89cbb640 1 -- 172.21.15.103:0/38533443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e841002f0 msgr2=0x7f8e841983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:10.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.772+0000 7f8e89cbb640 1 --2- 172.21.15.103:0/38533443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e841002f0 0x7f8e841983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:10.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.772+0000 7f8e89cbb640 1 -- 172.21.15.103:0/38533443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e84100ca0 msgr2=0x7f8e84198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:10.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.772+0000 7f8e89cbb640 1 --2- 172.21.15.103:0/38533443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e84100ca0 0x7f8e84198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:10.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.772+0000 7f8e89cbb640 1 -- 172.21.15.103:0/38533443 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8419d370 con 0x7f8e84106f50 2024-09-19T07:07:10.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.772+0000 7f8e88cb9640 1 --2- 172.21.15.103:0/38533443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e84100ca0 0x7f8e84198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:07:10.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.772+0000 7f8e894ba640 1 --2- 172.21.15.103:0/38533443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e841002f0 0x7f8e841983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:07:10.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.773+0000 7f8e89cbb640 1 --2- 172.21.15.103:0/38533443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e84106f50 0x7f8e8419cc90 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c005cc0 tx=0x7f8e7c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:07:10.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.773+0000 7f8e7a7fc640 1 -- 172.21.15.103:0/38533443 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c004040 con 0x7f8e84106f50 2024-09-19T07:07:10.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.773+0000 7f8e7a7fc640 1 -- 172.21.15.103:0/38533443 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e7c0046c0 con 0x7f8e84106f50 2024-09-19T07:07:10.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.773+0000 7f8e8b71e640 1 -- 172.21.15.103:0/38533443 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8419d600 con 0x7f8e84106f50 2024-09-19T07:07:10.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.773+0000 7f8e7a7fc640 1 -- 172.21.15.103:0/38533443 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c030e70 con 0x7f8e84106f50 2024-09-19T07:07:10.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.773+0000 7f8e8b71e640 1 -- 172.21.15.103:0/38533443 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e841a4f40 con 0x7f8e84106f50 2024-09-19T07:07:10.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.774+0000 7f8e8b71e640 1 -- 172.21.15.103:0/38533443 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e841006d0 con 0x7f8e84106f50 2024-09-19T07:07:10.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.775+0000 7f8e7a7fc640 1 -- 172.21.15.103:0/38533443 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f8e7c0305a0 con 0x7f8e84106f50 2024-09-19T07:07:10.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.776+0000 7f8e7a7fc640 1 --2- 172.21.15.103:0/38533443 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8e700a33b0 0x7f8e700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:10.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.776+0000 7f8e894ba640 1 --2- 172.21.15.103:0/38533443 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8e700a33b0 0x7f8e700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:10.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.776+0000 7f8e7a7fc640 1 -- 172.21.15.103:0/38533443 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8600+0+0 (secure 0 0 0) 0x7f8e7c02e030 con 0x7f8e84106f50 2024-09-19T07:07:10.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.776+0000 7f8e894ba640 1 --2- 172.21.15.103:0/38533443 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8e700a33b0 0x7f8e700a5870 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000bfd0 tx=0x7f8e8000b390 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:07:10.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.780+0000 7f8e7a7fc640 1 -- 172.21.15.103:0/38533443 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8e7c0098b0 con 0x7f8e84106f50 2024-09-19T07:07:10.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.957+0000 7f8e8b71e640 1 -- 172.21.15.103:0/38533443 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8e8402d7c0 con 0x7f8e84106f50 2024-09-19T07:07:10.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.958+0000 7f8e7a7fc640 1 -- 172.21.15.103:0/38533443 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v104) ==== 81+0+1018 (secure 0 0 0) 0x7f8e7c0bd750 con 0x7f8e84106f50 2024-09-19T07:07:10.959 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2169657832 2024-09-20T06:49:13.909393+0000 2024-09-19T07:07:10.959 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1596229922 2024-09-20T06:40:16.780890+0000 2024-09-19T07:07:10.959 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1642809745 2024-09-20T06:40:16.780890+0000 2024-09-19T07:07:10.959 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2003588522 2024-09-20T06:40:16.780890+0000 2024-09-19T07:07:10.959 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/70266056 2024-09-20T06:49:13.909393+0000 2024-09-19T07:07:10.959 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2003588522 2024-09-20T06:40:16.780890+0000 2024-09-19T07:07:10.959 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1496153385 2024-09-20T06:40:39.739638+0000 2024-09-19T07:07:10.959 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/471230312 2024-09-20T06:49:13.909393+0000 2024-09-19T07:07:10.959 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/4122140716 2024-09-20T06:40:39.739638+0000 2024-09-19T07:07:10.959 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/4122140716 2024-09-20T06:40:39.739638+0000 2024-09-19T07:07:10.959 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2309624637 2024-09-20T06:40:16.780890+0000 2024-09-19T07:07:10.960 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3727259725 2024-09-20T06:40:39.739638+0000 2024-09-19T07:07:10.960 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2590139910 2024-09-20T06:49:13.909393+0000 2024-09-19T07:07:10.960 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2050139492 2024-09-20T06:49:13.909393+0000 2024-09-19T07:07:10.960 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2867049580 2024-09-20T06:40:39.739638+0000 2024-09-19T07:07:10.960 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2590139910 2024-09-20T06:49:13.909393+0000 2024-09-19T07:07:10.960 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1074662744 2024-09-20T06:49:13.909393+0000 2024-09-19T07:07:10.960 INFO:teuthology.orchestra.run.smithi103.stderr:listed 17 entries 2024-09-19T07:07:10.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.960+0000 7f8e8b71e640 1 -- 172.21.15.103:0/38533443 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8e700a33b0 msgr2=0x7f8e700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:10.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.960+0000 7f8e8b71e640 1 --2- 172.21.15.103:0/38533443 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8e700a33b0 0x7f8e700a5870 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000bfd0 tx=0x7f8e8000b390 comp rx=0 tx=0).stop 2024-09-19T07:07:10.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.960+0000 7f8e8b71e640 1 -- 172.21.15.103:0/38533443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e84106f50 msgr2=0x7f8e8419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:10.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.960+0000 7f8e8b71e640 1 --2- 172.21.15.103:0/38533443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e84106f50 0x7f8e8419cc90 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c005cc0 tx=0x7f8e7c003970 comp rx=0 tx=0).stop 2024-09-19T07:07:10.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.960+0000 7f8e8b71e640 1 -- 172.21.15.103:0/38533443 shutdown_connections 2024-09-19T07:07:10.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.960+0000 7f8e8b71e640 1 --2- 172.21.15.103:0/38533443 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f8e700a33b0 0x7f8e700a5870 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:10.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.960+0000 7f8e8b71e640 1 --2- 172.21.15.103:0/38533443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e84106f50 0x7f8e8419cc90 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:10.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.960+0000 7f8e8b71e640 1 --2- 172.21.15.103:0/38533443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e84100ca0 0x7f8e84198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:10.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.960+0000 7f8e8b71e640 1 --2- 172.21.15.103:0/38533443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e841002f0 0x7f8e841983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:10.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.960+0000 7f8e8b71e640 1 -- 172.21.15.103:0/38533443 >> 172.21.15.103:0/38533443 conn(0x7f8e84077210 msgr2=0x7f8e840fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:07:10.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.960+0000 7f8e8b71e640 1 -- 172.21.15.103:0/38533443 shutdown_connections 2024-09-19T07:07:10.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:10.960+0000 7f8e8b71e640 1 -- 172.21.15.103:0/38533443 wait complete. 2024-09-19T07:07:11.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:11 smithi174 ceph-mon[35637]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T07:07:11.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:11 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/38533443' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T07:07:11.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:11 smithi103 ceph-mon[31158]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T07:07:11.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:11 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/38533443' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T07:07:11.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:11 smithi134 ceph-mon[35356]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T07:07:11.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:11 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/38533443' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T07:07:11.708 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-19T07:07:11.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:07:11.708 DEBUG:teuthology.orchestra.run.smithi103:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-19T07:07:11.753 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T07:07:11.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:07:11.753 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:07:11.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:07:11.788 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-19T07:07:11.844 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:07:11.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:07:11.845 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:07:11.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:07:11.910 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2024-09-19T07:07:12.003 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T07:07:12.014 INFO:tasks.cephadm:Teardown begin 2024-09-19T07:07:12.014 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T07:07:12.015 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:07:12.265 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:07:12.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:12 smithi174 ceph-mon[35637]: pgmap v562: 97 pgs: 97 active+clean; 1.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T07:07:12.386 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:12 smithi103 ceph-mon[31158]: pgmap v562: 97 pgs: 97 active+clean; 1.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T07:07:12.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:12 smithi134 ceph-mon[35356]: pgmap v562: 97 pgs: 97 active+clean; 1.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T07:07:12.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.734+0000 7f9f73f51640 1 -- 172.21.15.103:0/1369789594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f6c100b30 msgr2=0x7f9f6c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:12.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.734+0000 7f9f73f51640 1 --2- 172.21.15.103:0/1369789594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f6c100b30 0x7f9f6c100f10 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009900 tx=0x7f9f5c02a470 comp rx=0 tx=0).stop 2024-09-19T07:07:12.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.735+0000 7f9f73f51640 1 -- 172.21.15.103:0/1369789594 shutdown_connections 2024-09-19T07:07:12.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.735+0000 7f9f73f51640 1 --2- 172.21.15.103:0/1369789594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f6c105f40 0x7f9f6c1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:12.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.735+0000 7f9f73f51640 1 --2- 172.21.15.103:0/1369789594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f6c101450 0x7f9f6c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:12.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.735+0000 7f9f73f51640 1 --2- 172.21.15.103:0/1369789594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f6c100b30 0x7f9f6c100f10 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:12.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.735+0000 7f9f73f51640 1 -- 172.21.15.103:0/1369789594 >> 172.21.15.103:0/1369789594 conn(0x7f9f6c0fc7a0 msgr2=0x7f9f6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:07:12.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.735+0000 7f9f73f51640 1 -- 172.21.15.103:0/1369789594 shutdown_connections 2024-09-19T07:07:12.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.735+0000 7f9f73f51640 1 -- 172.21.15.103:0/1369789594 wait complete. 2024-09-19T07:07:12.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.735+0000 7f9f73f51640 1 Processor -- start 2024-09-19T07:07:12.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.735+0000 7f9f73f51640 1 -- start start 2024-09-19T07:07:12.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.735+0000 7f9f73f51640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f6c100b30 0x7f9f6c1a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:12.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.735+0000 7f9f73f51640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f6c101450 0x7f9f6c1a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:12.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.735+0000 7f9f73f51640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f6c105f40 0x7f9f6c19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:12.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.735+0000 7f9f73f51640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c1185e0 con 0x7f9f6c100b30 2024-09-19T07:07:12.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.735+0000 7f9f73f51640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c118460 con 0x7f9f6c101450 2024-09-19T07:07:12.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.735+0000 7f9f73f51640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c118760 con 0x7f9f6c105f40 2024-09-19T07:07:12.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.736+0000 7f9f71ced640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f6c100b30 0x7f9f6c1a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:12.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.736+0000 7f9f71ced640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f6c100b30 0x7f9f6c1a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51678/0 (socket says 172.21.15.103:51678) 2024-09-19T07:07:12.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.736+0000 7f9f71ced640 1 -- 172.21.15.103:0/1531855612 learned_addr learned my addr 172.21.15.103:0/1531855612 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:07:12.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.736+0000 7f9f714ec640 1 --2- 172.21.15.103:0/1531855612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f6c101450 0x7f9f6c1a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:12.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.736+0000 7f9f724ee640 1 --2- 172.21.15.103:0/1531855612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f6c105f40 0x7f9f6c19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:12.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.736+0000 7f9f71ced640 1 -- 172.21.15.103:0/1531855612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f6c105f40 msgr2=0x7f9f6c19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:12.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.736+0000 7f9f71ced640 1 --2- 172.21.15.103:0/1531855612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f6c105f40 0x7f9f6c19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:12.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.736+0000 7f9f71ced640 1 -- 172.21.15.103:0/1531855612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f6c101450 msgr2=0x7f9f6c1a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:12.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.736+0000 7f9f71ced640 1 --2- 172.21.15.103:0/1531855612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f6c101450 0x7f9f6c1a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:12.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.736+0000 7f9f71ced640 1 -- 172.21.15.103:0/1531855612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6c19f080 con 0x7f9f6c100b30 2024-09-19T07:07:12.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.736+0000 7f9f71ced640 1 --2- 172.21.15.103:0/1531855612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f6c100b30 0x7f9f6c1a4790 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c006010 tx=0x7f9f5c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:07:12.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.736+0000 7f9f62ffd640 1 -- 172.21.15.103:0/1531855612 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c03f030 con 0x7f9f6c100b30 2024-09-19T07:07:12.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.736+0000 7f9f62ffd640 1 -- 172.21.15.103:0/1531855612 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f5c003ce0 con 0x7f9f6c100b30 2024-09-19T07:07:12.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.736+0000 7f9f73f51640 1 -- 172.21.15.103:0/1531855612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6c19f310 con 0x7f9f6c100b30 2024-09-19T07:07:12.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.736+0000 7f9f62ffd640 1 -- 172.21.15.103:0/1531855612 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c035070 con 0x7f9f6c100b30 2024-09-19T07:07:12.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.736+0000 7f9f73f51640 1 -- 172.21.15.103:0/1531855612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f6c1ab660 con 0x7f9f6c100b30 2024-09-19T07:07:12.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.737+0000 7f9f73f51640 1 -- 172.21.15.103:0/1531855612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f6c105920 con 0x7f9f6c100b30 2024-09-19T07:07:12.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.739+0000 7f9f62ffd640 1 -- 172.21.15.103:0/1531855612 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f9f5c042030 con 0x7f9f6c100b30 2024-09-19T07:07:12.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.740+0000 7f9f62ffd640 1 --2- 172.21.15.103:0/1531855612 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9f580a3430 0x7f9f580a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:12.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.740+0000 7f9f714ec640 1 --2- 172.21.15.103:0/1531855612 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9f580a3430 0x7f9f580a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:12.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.740+0000 7f9f62ffd640 1 -- 172.21.15.103:0/1531855612 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8600+0+0 (secure 0 0 0) 0x7f9f5c0f0480 con 0x7f9f6c100b30 2024-09-19T07:07:12.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.740+0000 7f9f714ec640 1 --2- 172.21.15.103:0/1531855612 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9f580a3430 0x7f9f580a58f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0fff00 tx=0x7f9f68009060 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:07:12.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.743+0000 7f9f62ffd640 1 -- 172.21.15.103:0/1531855612 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9f5c0bc7c0 con 0x7f9f6c100b30 2024-09-19T07:07:12.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.945+0000 7f9f73f51640 1 -- 172.21.15.103:0/1531855612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f6c19fe80 con 0x7f9f6c100b30 2024-09-19T07:07:12.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.946+0000 7f9f62ffd640 1 -- 172.21.15.103:0/1531855612 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7f9f6c19fe80 con 0x7f9f6c100b30 2024-09-19T07:07:12.948 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:07:12.948 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:07:12.948 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:07:12.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.948+0000 7f9f73f51640 1 -- 172.21.15.103:0/1531855612 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9f580a3430 msgr2=0x7f9f580a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:12.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.948+0000 7f9f73f51640 1 --2- 172.21.15.103:0/1531855612 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9f580a3430 0x7f9f580a58f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0fff00 tx=0x7f9f68009060 comp rx=0 tx=0).stop 2024-09-19T07:07:12.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.948+0000 7f9f73f51640 1 -- 172.21.15.103:0/1531855612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f6c100b30 msgr2=0x7f9f6c1a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:12.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.948+0000 7f9f73f51640 1 --2- 172.21.15.103:0/1531855612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f6c100b30 0x7f9f6c1a4790 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c006010 tx=0x7f9f5c0042f0 comp rx=0 tx=0).stop 2024-09-19T07:07:12.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.949+0000 7f9f73f51640 1 -- 172.21.15.103:0/1531855612 shutdown_connections 2024-09-19T07:07:12.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.949+0000 7f9f73f51640 1 --2- 172.21.15.103:0/1531855612 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f9f580a3430 0x7f9f580a58f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:12.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.949+0000 7f9f73f51640 1 --2- 172.21.15.103:0/1531855612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f6c105f40 0x7f9f6c19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:12.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.949+0000 7f9f73f51640 1 --2- 172.21.15.103:0/1531855612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f6c101450 0x7f9f6c1a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:12.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.949+0000 7f9f73f51640 1 --2- 172.21.15.103:0/1531855612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f6c100b30 0x7f9f6c1a4790 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:12.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.949+0000 7f9f73f51640 1 -- 172.21.15.103:0/1531855612 >> 172.21.15.103:0/1531855612 conn(0x7f9f6c0fc7a0 msgr2=0x7f9f6c1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:07:12.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.949+0000 7f9f73f51640 1 -- 172.21.15.103:0/1531855612 shutdown_connections 2024-09-19T07:07:12.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:12.949+0000 7f9f73f51640 1 -- 172.21.15.103:0/1531855612 wait complete. 2024-09-19T07:07:13.311 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:13 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1531855612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:07:13.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:13 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1531855612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:07:13.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:13 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1531855612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:07:13.642 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:07:13.889 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:07:14.275 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:14 smithi103 ceph-mon[31158]: pgmap v563: 97 pgs: 97 active+clean; 1.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T07:07:14.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:14 smithi174 ceph-mon[35637]: pgmap v563: 97 pgs: 97 active+clean; 1.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T07:07:14.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.353+0000 7fecd33c3640 1 -- 172.21.15.103:0/3585251022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc100e20 msgr2=0x7feccc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:14.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.353+0000 7fecd33c3640 1 --2- 172.21.15.103:0/3585251022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc100e20 0x7feccc101200 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7fecbc009900 tx=0x7fecbc02a480 comp rx=0 tx=0).stop 2024-09-19T07:07:14.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.353+0000 7fecd33c3640 1 -- 172.21.15.103:0/3585251022 shutdown_connections 2024-09-19T07:07:14.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.353+0000 7fecd33c3640 1 --2- 172.21.15.103:0/3585251022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feccc102ef0 0x7feccc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:14.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.353+0000 7fecd33c3640 1 --2- 172.21.15.103:0/3585251022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc102570 0x7feccc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:14.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.353+0000 7fecd33c3640 1 --2- 172.21.15.103:0/3585251022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc100e20 0x7feccc101200 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:14.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.353+0000 7fecd33c3640 1 -- 172.21.15.103:0/3585251022 >> 172.21.15.103:0/3585251022 conn(0x7feccc0fc7d0 msgr2=0x7feccc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:07:14.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.353+0000 7fecd33c3640 1 -- 172.21.15.103:0/3585251022 shutdown_connections 2024-09-19T07:07:14.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.353+0000 7fecd33c3640 1 -- 172.21.15.103:0/3585251022 wait complete. 2024-09-19T07:07:14.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd33c3640 1 Processor -- start 2024-09-19T07:07:14.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd33c3640 1 -- start start 2024-09-19T07:07:14.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd33c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feccc100e20 0x7feccc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:14.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd33c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc102570 0x7feccc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:14.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd33c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc102ef0 0x7feccc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:14.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd33c3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feccc110fa0 con 0x7feccc102ef0 2024-09-19T07:07:14.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd33c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feccc110e20 con 0x7feccc100e20 2024-09-19T07:07:14.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd33c3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feccc111120 con 0x7feccc102570 2024-09-19T07:07:14.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd1960640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc102ef0 0x7feccc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:14.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd1960640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc102ef0 0x7feccc0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51690/0 (socket says 172.21.15.103:51690) 2024-09-19T07:07:14.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd1960640 1 -- 172.21.15.103:0/632009442 learned_addr learned my addr 172.21.15.103:0/632009442 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:07:14.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd115f640 1 --2- 172.21.15.103:0/632009442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feccc100e20 0x7feccc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:14.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd095e640 1 --2- 172.21.15.103:0/632009442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc102570 0x7feccc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:14.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd1960640 1 -- 172.21.15.103:0/632009442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc102570 msgr2=0x7feccc0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:14.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd1960640 1 --2- 172.21.15.103:0/632009442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc102570 0x7feccc0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:14.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd1960640 1 -- 172.21.15.103:0/632009442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feccc100e20 msgr2=0x7feccc0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:14.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd1960640 1 --2- 172.21.15.103:0/632009442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feccc100e20 0x7feccc0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:14.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd1960640 1 -- 172.21.15.103:0/632009442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc1a6a50 con 0x7feccc102ef0 2024-09-19T07:07:14.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.354+0000 7fecd115f640 1 --2- 172.21.15.103:0/632009442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feccc100e20 0x7feccc0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:07:14.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.355+0000 7fecd1960640 1 --2- 172.21.15.103:0/632009442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc102ef0 0x7feccc0fd700 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7fecc400d750 tx=0x7fecc400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:07:14.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.355+0000 7fecca7fc640 1 -- 172.21.15.103:0/632009442 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4017070 con 0x7feccc102ef0 2024-09-19T07:07:14.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.355+0000 7fecd33c3640 1 -- 172.21.15.103:0/632009442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc1a6d40 con 0x7feccc102ef0 2024-09-19T07:07:14.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.355+0000 7fecca7fc640 1 -- 172.21.15.103:0/632009442 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecc4005500 con 0x7feccc102ef0 2024-09-19T07:07:14.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.355+0000 7fecca7fc640 1 -- 172.21.15.103:0/632009442 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4004d90 con 0x7feccc102ef0 2024-09-19T07:07:14.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.355+0000 7fecd33c3640 1 -- 172.21.15.103:0/632009442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc1a7230 con 0x7feccc102ef0 2024-09-19T07:07:14.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.356+0000 7fecd33c3640 1 -- 172.21.15.103:0/632009442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feccc101200 con 0x7feccc102ef0 2024-09-19T07:07:14.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.357+0000 7fecca7fc640 1 -- 172.21.15.103:0/632009442 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fecc4004f30 con 0x7feccc102ef0 2024-09-19T07:07:14.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.358+0000 7fecca7fc640 1 --2- 172.21.15.103:0/632009442 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fecb40a33b0 0x7fecb40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:14.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.358+0000 7fecd115f640 1 --2- 172.21.15.103:0/632009442 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fecb40a33b0 0x7fecb40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:14.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.358+0000 7fecca7fc640 1 -- 172.21.15.103:0/632009442 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8600+0+0 (secure 0 0 0) 0x7fecc40cf840 con 0x7feccc102ef0 2024-09-19T07:07:14.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.358+0000 7fecd115f640 1 --2- 172.21.15.103:0/632009442 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fecb40a33b0 0x7fecb40a5870 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fecbc005ab0 tx=0x7fecbc005a20 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:07:14.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.362+0000 7fecca7fc640 1 -- 172.21.15.103:0/632009442 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fecc409c1d0 con 0x7feccc102ef0 2024-09-19T07:07:14.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:14 smithi134 ceph-mon[35356]: pgmap v563: 97 pgs: 97 active+clean; 1.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T07:07:14.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.567+0000 7fecd33c3640 1 -- 172.21.15.103:0/632009442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feccc10a0a0 con 0x7feccc102ef0 2024-09-19T07:07:14.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.568+0000 7fecca7fc640 1 -- 172.21.15.103:0/632009442 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11994 (secure 0 0 0) 0x7fecc409bad0 con 0x7feccc102ef0 2024-09-19T07:07:14.570 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:07:14.570 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T06:49:14:915668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/4290108970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4290108970},{"type":"v1","addr":"172.21.15.103:6835","nonce":4290108970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/323370484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":323370484},{"type":"v1","addr":"172.21.15.103:6837","nonce":323370484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:47:25.446176+0000","modified":"2024-09-19T06:49:14.707403+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24455,"mds_2":24454,"mds_3":24436,"mds_4":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/3119326077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3119326077},{"type":"v1","addr":"172.21.15.103:6839","nonce":3119326077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/3771437304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":3771437304},{"type":"v1","addr":"172.21.15.103:6841","nonce":3771437304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/2319563070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2319563070},{"type":"v1","addr":"172.21.15.174:6833","nonce":2319563070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/1556743259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1556743259},{"type":"v1","addr":"172.21.15.134:6833","nonce":1556743259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/963732203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":963732203},{"type":"v1","addr":"172.21.15.174:6835","nonce":963732203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6835/3092414367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3092414367},{"type":"v1","addr":"172.21.15.134:6835","nonce":3092414367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1053388911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1053388911},{"type":"v1","addr":"172.21.15.174:6837","nonce":1053388911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/4202229983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":4202229983},{"type":"v1","addr":"172.21.15.174:6839","nonce":4202229983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/1590453330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1590453330},{"type":"v1","addr":"172.21.15.134:6837","nonce":1590453330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/2333076106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":2333076106},{"type":"v1","addr":"172.21.15.134:6839","nonce":2333076106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24431,24436,24454,24455,24482]},"id":1}]} 2024-09-19T07:07:14.570 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:07:14.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.570+0000 7fecd33c3640 1 -- 172.21.15.103:0/632009442 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fecb40a33b0 msgr2=0x7fecb40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:14.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.570+0000 7fecd33c3640 1 --2- 172.21.15.103:0/632009442 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fecb40a33b0 0x7fecb40a5870 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fecbc005ab0 tx=0x7fecbc005a20 comp rx=0 tx=0).stop 2024-09-19T07:07:14.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.570+0000 7fecd33c3640 1 -- 172.21.15.103:0/632009442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc102ef0 msgr2=0x7feccc0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:14.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.570+0000 7fecd33c3640 1 --2- 172.21.15.103:0/632009442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc102ef0 0x7feccc0fd700 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7fecc400d750 tx=0x7fecc400dc10 comp rx=0 tx=0).stop 2024-09-19T07:07:14.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.570+0000 7fecd33c3640 1 -- 172.21.15.103:0/632009442 shutdown_connections 2024-09-19T07:07:14.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.570+0000 7fecd33c3640 1 --2- 172.21.15.103:0/632009442 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fecb40a33b0 0x7fecb40a5870 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:14.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.570+0000 7fecd33c3640 1 --2- 172.21.15.103:0/632009442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc102ef0 0x7feccc0fd700 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:14.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.570+0000 7fecd33c3640 1 --2- 172.21.15.103:0/632009442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc102570 0x7feccc0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:14.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.570+0000 7fecd33c3640 1 --2- 172.21.15.103:0/632009442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feccc100e20 0x7feccc0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:14.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.570+0000 7fecd33c3640 1 -- 172.21.15.103:0/632009442 >> 172.21.15.103:0/632009442 conn(0x7feccc0fc7d0 msgr2=0x7feccc06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:07:14.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.570+0000 7fecd33c3640 1 -- 172.21.15.103:0/632009442 shutdown_connections 2024-09-19T07:07:14.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:14.570+0000 7fecd33c3640 1 -- 172.21.15.103:0/632009442 wait complete. 2024-09-19T07:07:15.281 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:15 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/632009442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:07:15.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:07:15.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:15 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/632009442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:07:15.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:15 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/632009442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:07:15.553 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:07:15.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.997+0000 7f0af21ec640 1 -- 172.21.15.103:0/47111743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec102570 msgr2=0x7f0aec1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:15.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.997+0000 7f0af21ec640 1 --2- 172.21.15.103:0/47111743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec102570 0x7f0aec1029b0 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009900 tx=0x7f0ad402a480 comp rx=0 tx=0).stop 2024-09-19T07:07:15.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.997+0000 7f0af21ec640 1 -- 172.21.15.103:0/47111743 shutdown_connections 2024-09-19T07:07:15.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.997+0000 7f0af21ec640 1 --2- 172.21.15.103:0/47111743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0aec102ef0 0x7f0aec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:15.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.997+0000 7f0af21ec640 1 --2- 172.21.15.103:0/47111743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec102570 0x7f0aec1029b0 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:15.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.997+0000 7f0af21ec640 1 --2- 172.21.15.103:0/47111743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec100e20 0x7f0aec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:15.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.997+0000 7f0af21ec640 1 -- 172.21.15.103:0/47111743 >> 172.21.15.103:0/47111743 conn(0x7f0aec0fc7d0 msgr2=0x7f0aec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:07:15.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.997+0000 7f0af21ec640 1 -- 172.21.15.103:0/47111743 shutdown_connections 2024-09-19T07:07:15.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.997+0000 7f0af21ec640 1 -- 172.21.15.103:0/47111743 wait complete. 2024-09-19T07:07:15.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.997+0000 7f0af21ec640 1 Processor -- start 2024-09-19T07:07:16.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.997+0000 7f0af21ec640 1 -- start start 2024-09-19T07:07:16.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0af21ec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec100e20 0x7f0aec1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:16.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0af21ec640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec102570 0x7f0aec198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:16.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0af21ec640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0aec102ef0 0x7f0aec19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:16.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0af21ec640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec110f80 con 0x7f0aec100e20 2024-09-19T07:07:16.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0af21ec640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec110e00 con 0x7f0aec102ef0 2024-09-19T07:07:16.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0af21ec640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec111100 con 0x7f0aec102570 2024-09-19T07:07:16.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0aeb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec100e20 0x7f0aec1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:16.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0aeb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec100e20 0x7f0aec1984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51700/0 (socket says 172.21.15.103:51700) 2024-09-19T07:07:16.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0aeb7fe640 1 -- 172.21.15.103:0/4280105170 learned_addr learned my addr 172.21.15.103:0/4280105170 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:07:16.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0aeaffd640 1 --2- 172.21.15.103:0/4280105170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec102570 0x7f0aec198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:16.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0aebfff640 1 --2- 172.21.15.103:0/4280105170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0aec102ef0 0x7f0aec19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:16.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0aeb7fe640 1 -- 172.21.15.103:0/4280105170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec102570 msgr2=0x7f0aec198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:16.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0aeb7fe640 1 --2- 172.21.15.103:0/4280105170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec102570 0x7f0aec198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:16.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0aeb7fe640 1 -- 172.21.15.103:0/4280105170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0aec102ef0 msgr2=0x7f0aec19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:16.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0aeb7fe640 1 --2- 172.21.15.103:0/4280105170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0aec102ef0 0x7f0aec19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:16.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0aeb7fe640 1 -- 172.21.15.103:0/4280105170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aec19d4a0 con 0x7f0aec100e20 2024-09-19T07:07:16.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0aebfff640 1 --2- 172.21.15.103:0/4280105170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0aec102ef0 0x7f0aec19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:07:16.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.998+0000 7f0aeb7fe640 1 --2- 172.21.15.103:0/4280105170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec100e20 0x7f0aec1984f0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400ea80 tx=0x7f0ae400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:07:16.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.999+0000 7f0ae8ff9640 1 -- 172.21.15.103:0/4280105170 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae400ce40 con 0x7f0aec100e20 2024-09-19T07:07:16.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.999+0000 7f0af21ec640 1 -- 172.21.15.103:0/4280105170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aec19d790 con 0x7f0aec100e20 2024-09-19T07:07:16.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.999+0000 7f0ae8ff9640 1 -- 172.21.15.103:0/4280105170 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ae4005020 con 0x7f0aec100e20 2024-09-19T07:07:16.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.999+0000 7f0ae8ff9640 1 -- 172.21.15.103:0/4280105170 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4011760 con 0x7f0aec100e20 2024-09-19T07:07:16.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:15.999+0000 7f0af21ec640 1 -- 172.21.15.103:0/4280105170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aec1a5070 con 0x7f0aec100e20 2024-09-19T07:07:16.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.000+0000 7f0af21ec640 1 -- 172.21.15.103:0/4280105170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aec101200 con 0x7f0aec100e20 2024-09-19T07:07:16.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.001+0000 7f0ae8ff9640 1 -- 172.21.15.103:0/4280105170 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f0ae40051d0 con 0x7f0aec100e20 2024-09-19T07:07:16.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.002+0000 7f0ae8ff9640 1 --2- 172.21.15.103:0/4280105170 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0ad80a33b0 0x7f0ad80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:16.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.002+0000 7f0aeaffd640 1 --2- 172.21.15.103:0/4280105170 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0ad80a33b0 0x7f0ad80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:16.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.002+0000 7f0ae8ff9640 1 -- 172.21.15.103:0/4280105170 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8600+0+0 (secure 0 0 0) 0x7f0ae40d0400 con 0x7f0aec100e20 2024-09-19T07:07:16.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.003+0000 7f0aeaffd640 1 --2- 172.21.15.103:0/4280105170 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0ad80a33b0 0x7f0ad80a5870 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009230 tx=0x7f0ad40058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:07:16.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.005+0000 7f0ae8ff9640 1 -- 172.21.15.103:0/4280105170 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0ae4016030 con 0x7f0aec100e20 2024-09-19T07:07:16.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.175+0000 7f0af21ec640 1 -- 172.21.15.103:0/4280105170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0aec199850 con 0x7f0aec100e20 2024-09-19T07:07:16.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.176+0000 7f0ae8ff9640 1 -- 172.21.15.103:0/4280105170 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+23335 (secure 0 0 0) 0x7f0ae409cd90 con 0x7f0aec100e20 2024-09-19T07:07:16.178 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:07:16.178 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":104,"fsid":"d158d934-7651-11ef-bceb-c7b262605968","created":"2024-09-19T06:39:46.840574+0000","modified":"2024-09-19T07:07:10.040691+0000","last_up_change":"2024-09-19T06:46:35.652960+0000","last_in_change":"2024-09-19T06:46:11.630585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:42:52.050881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:47:21.192611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:47:23.643291+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":30,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5d45dab3-e434-4783-b2e3-f658c3ba583b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6803","nonce":3157697740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6805","nonce":3157697740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6809","nonce":3157697740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3157697740},{"type":"v1","addr":"172.21.15.103:6807","nonce":3157697740}]},"public_addr":"172.21.15.103:6803/3157697740","cluster_addr":"172.21.15.103:6805/3157697740","heartbeat_back_addr":"172.21.15.103:6809/3157697740","heartbeat_front_addr":"172.21.15.103:6807/3157697740","state":["exists","up"]},{"osd":1,"uuid":"ee0e92bd-15ab-4867-9a18-20abb9f5fc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6801","nonce":1418301195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6803","nonce":1418301195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6807","nonce":1418301195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1418301195},{"type":"v1","addr":"172.21.15.134:6805","nonce":1418301195}]},"public_addr":"172.21.15.134:6801/1418301195","cluster_addr":"172.21.15.134:6803/1418301195","heartbeat_back_addr":"172.21.15.134:6807/1418301195","heartbeat_front_addr":"172.21.15.134:6805/1418301195","state":["exists","up"]},{"osd":2,"uuid":"0230caed-025b-46e9-a011-237601d16ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6801","nonce":2292242290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6803","nonce":2292242290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6807","nonce":2292242290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2292242290},{"type":"v1","addr":"172.21.15.174:6805","nonce":2292242290}]},"public_addr":"172.21.15.174:6801/2292242290","cluster_addr":"172.21.15.174:6803/2292242290","heartbeat_back_addr":"172.21.15.174:6807/2292242290","heartbeat_front_addr":"172.21.15.174:6805/2292242290","state":["exists","up"]},{"osd":3,"uuid":"c54a8399-bf4b-4d70-b207-a13d85f99590","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6811","nonce":2698986333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6813","nonce":2698986333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6817","nonce":2698986333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2698986333},{"type":"v1","addr":"172.21.15.103:6815","nonce":2698986333}]},"public_addr":"172.21.15.103:6811/2698986333","cluster_addr":"172.21.15.103:6813/2698986333","heartbeat_back_addr":"172.21.15.103:6817/2698986333","heartbeat_front_addr":"172.21.15.103:6815/2698986333","state":["exists","up"]},{"osd":4,"uuid":"7c046be6-464a-4ab5-812b-1781121fc317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6809","nonce":3169891968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6811","nonce":3169891968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6815","nonce":3169891968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3169891968},{"type":"v1","addr":"172.21.15.134:6813","nonce":3169891968}]},"public_addr":"172.21.15.134:6809/3169891968","cluster_addr":"172.21.15.134:6811/3169891968","heartbeat_back_addr":"172.21.15.134:6815/3169891968","heartbeat_front_addr":"172.21.15.134:6813/3169891968","state":["exists","up"]},{"osd":5,"uuid":"8622596b-6a06-459c-882d-50fba9e296ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6809","nonce":763749644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6811","nonce":763749644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6815","nonce":763749644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":763749644},{"type":"v1","addr":"172.21.15.174:6813","nonce":763749644}]},"public_addr":"172.21.15.174:6809/763749644","cluster_addr":"172.21.15.174:6811/763749644","heartbeat_back_addr":"172.21.15.174:6815/763749644","heartbeat_front_addr":"172.21.15.174:6813/763749644","state":["exists","up"]},{"osd":6,"uuid":"64d858d6-a8b3-4f65-a187-af9ab1ccee0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6819","nonce":1396972394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6821","nonce":1396972394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6825","nonce":1396972394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1396972394},{"type":"v1","addr":"172.21.15.103:6823","nonce":1396972394}]},"public_addr":"172.21.15.103:6819/1396972394","cluster_addr":"172.21.15.103:6821/1396972394","heartbeat_back_addr":"172.21.15.103:6825/1396972394","heartbeat_front_addr":"172.21.15.103:6823/1396972394","state":["exists","up"]},{"osd":7,"uuid":"d405d1c6-839c-4c92-8261-4adb3163c90f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6817","nonce":2196366765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6819","nonce":2196366765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6823","nonce":2196366765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2196366765},{"type":"v1","addr":"172.21.15.134:6821","nonce":2196366765}]},"public_addr":"172.21.15.134:6817/2196366765","cluster_addr":"172.21.15.134:6819/2196366765","heartbeat_back_addr":"172.21.15.134:6823/2196366765","heartbeat_front_addr":"172.21.15.134:6821/2196366765","state":["exists","up"]},{"osd":8,"uuid":"4c452cd9-816c-4f77-9368-2cd4bbeee4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6817","nonce":189977465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6819","nonce":189977465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6823","nonce":189977465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":189977465},{"type":"v1","addr":"172.21.15.174:6821","nonce":189977465}]},"public_addr":"172.21.15.174:6817/189977465","cluster_addr":"172.21.15.174:6819/189977465","heartbeat_back_addr":"172.21.15.174:6823/189977465","heartbeat_front_addr":"172.21.15.174:6821/189977465","state":["exists","up"]},{"osd":9,"uuid":"fb3662e0-2d8e-45c4-9119-6d08b35443da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6827","nonce":3161966413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6829","nonce":3161966413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6833","nonce":3161966413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3161966413},{"type":"v1","addr":"172.21.15.103:6831","nonce":3161966413}]},"public_addr":"172.21.15.103:6827/3161966413","cluster_addr":"172.21.15.103:6829/3161966413","heartbeat_back_addr":"172.21.15.103:6833/3161966413","heartbeat_front_addr":"172.21.15.103:6831/3161966413","state":["exists","up"]},{"osd":10,"uuid":"e52c3cbe-a6af-441e-b423-b94f3a2da544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6825","nonce":2532758632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6827","nonce":2532758632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6831","nonce":2532758632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":2532758632},{"type":"v1","addr":"172.21.15.134:6829","nonce":2532758632}]},"public_addr":"172.21.15.134:6825/2532758632","cluster_addr":"172.21.15.134:6827/2532758632","heartbeat_back_addr":"172.21.15.134:6831/2532758632","heartbeat_front_addr":"172.21.15.134:6829/2532758632","state":["exists","up"]},{"osd":11,"uuid":"9040f3f8-3873-4dea-8716-dcda5811c8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6825","nonce":161916442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6827","nonce":161916442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6831","nonce":161916442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":161916442},{"type":"v1","addr":"172.21.15.174:6829","nonce":161916442}]},"public_addr":"172.21.15.174:6825/161916442","cluster_addr":"172.21.15.174:6827/161916442","heartbeat_back_addr":"172.21.15.174:6831/161916442","heartbeat_front_addr":"172.21.15.174:6829/161916442","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:41:59.656854+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:24.797551+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:42:49.685497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:33.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:43:39.324983+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:03.987310+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:28.747971+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:44:53.827780+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:18.554450+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:45:43.705691+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:08.401275+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:46:33.502542+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2169657832":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/1596229922":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1642809745":"2024-09-20T06:40:16.780890+0000","172.21.15.103:6800/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/70266056":"2024-09-20T06:49:13.909393+0000","172.21.15.103:6801/2003588522":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/1496153385":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/471230312":"2024-09-20T06:49:13.909393+0000","172.21.15.103:6801/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/4122140716":"2024-09-20T06:40:39.739638+0000","172.21.15.103:0/2309624637":"2024-09-20T06:40:16.780890+0000","172.21.15.103:0/3727259725":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6801/2590139910":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/2050139492":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/2867049580":"2024-09-20T06:40:39.739638+0000","172.21.15.103:6800/2590139910":"2024-09-20T06:49:13.909393+0000","172.21.15.103:0/1074662744":"2024-09-20T06:49:13.909393+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":30,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":30,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:07:16.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.178+0000 7f0af21ec640 1 -- 172.21.15.103:0/4280105170 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0ad80a33b0 msgr2=0x7f0ad80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:16.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.178+0000 7f0af21ec640 1 --2- 172.21.15.103:0/4280105170 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0ad80a33b0 0x7f0ad80a5870 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009230 tx=0x7f0ad40058d0 comp rx=0 tx=0).stop 2024-09-19T07:07:16.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.178+0000 7f0af21ec640 1 -- 172.21.15.103:0/4280105170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec100e20 msgr2=0x7f0aec1984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:16.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.178+0000 7f0af21ec640 1 --2- 172.21.15.103:0/4280105170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec100e20 0x7f0aec1984f0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400ea80 tx=0x7f0ae400ef40 comp rx=0 tx=0).stop 2024-09-19T07:07:16.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.178+0000 7f0af21ec640 1 -- 172.21.15.103:0/4280105170 shutdown_connections 2024-09-19T07:07:16.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.178+0000 7f0af21ec640 1 --2- 172.21.15.103:0/4280105170 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f0ad80a33b0 0x7f0ad80a5870 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:16.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.178+0000 7f0af21ec640 1 --2- 172.21.15.103:0/4280105170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0aec102ef0 0x7f0aec19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:16.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.178+0000 7f0af21ec640 1 --2- 172.21.15.103:0/4280105170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec102570 0x7f0aec198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:16.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.178+0000 7f0af21ec640 1 --2- 172.21.15.103:0/4280105170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec100e20 0x7f0aec1984f0 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:16.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.178+0000 7f0af21ec640 1 -- 172.21.15.103:0/4280105170 >> 172.21.15.103:0/4280105170 conn(0x7f0aec0fc7d0 msgr2=0x7f0aec069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:07:16.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.179+0000 7f0af21ec640 1 -- 172.21.15.103:0/4280105170 shutdown_connections 2024-09-19T07:07:16.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:16.179+0000 7f0af21ec640 1 -- 172.21.15.103:0/4280105170 wait complete. 2024-09-19T07:07:16.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:16 smithi174 ceph-mon[35637]: pgmap v564: 97 pgs: 97 active+clean; 1.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T07:07:16.338 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:16 smithi134 ceph-mon[35356]: pgmap v564: 97 pgs: 97 active+clean; 1.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T07:07:16.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:16 smithi103 ceph-mon[31158]: pgmap v564: 97 pgs: 97 active+clean; 1.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T07:07:16.992 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T07:07:17.244 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:07:17.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:17 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/4280105170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:07:17.365 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:17 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/4280105170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:07:17.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/4280105170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:07:17.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.696+0000 7ff32a2cb640 1 -- 172.21.15.103:0/3183938516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff324106f10 msgr2=0x7ff32410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:17.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.696+0000 7ff32a2cb640 1 --2- 172.21.15.103:0/3183938516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff324106f10 0x7ff32410ea90 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009ca0 tx=0x7ff31c02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:07:17.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.696+0000 7ff32a2cb640 1 -- 172.21.15.103:0/3183938516 shutdown_connections 2024-09-19T07:07:17.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.696+0000 7ff32a2cb640 1 --2- 172.21.15.103:0/3183938516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff324106f10 0x7ff32410ea90 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:17.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.696+0000 7ff32a2cb640 1 --2- 172.21.15.103:0/3183938516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff324102790 0x7ff3241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:17.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.696+0000 7ff32a2cb640 1 --2- 172.21.15.103:0/3183938516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff324101de0 0x7ff3241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:17.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.696+0000 7ff32a2cb640 1 -- 172.21.15.103:0/3183938516 >> 172.21.15.103:0/3183938516 conn(0x7ff3240fd8c0 msgr2=0x7ff3240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:07:17.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.696+0000 7ff32a2cb640 1 -- 172.21.15.103:0/3183938516 shutdown_connections 2024-09-19T07:07:17.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.696+0000 7ff32a2cb640 1 -- 172.21.15.103:0/3183938516 wait complete. 2024-09-19T07:07:17.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.697+0000 7ff32a2cb640 1 Processor -- start 2024-09-19T07:07:17.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.697+0000 7ff32a2cb640 1 -- start start 2024-09-19T07:07:17.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.697+0000 7ff32a2cb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff324101de0 0x7ff324198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:17.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.697+0000 7ff32a2cb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff324102790 0x7ff324198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:17.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.697+0000 7ff32a2cb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff324106f10 0x7ff32419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:17.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.697+0000 7ff32a2cb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff324112010 con 0x7ff324101de0 2024-09-19T07:07:17.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.697+0000 7ff32a2cb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff324111e90 con 0x7ff324102790 2024-09-19T07:07:17.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.697+0000 7ff32a2cb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff324112190 con 0x7ff324106f10 2024-09-19T07:07:17.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.697+0000 7ff323fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff324101de0 0x7ff324198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:17.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.697+0000 7ff323fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff324101de0 0x7ff324198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40164/0 (socket says 172.21.15.103:40164) 2024-09-19T07:07:17.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.697+0000 7ff323fff640 1 -- 172.21.15.103:0/4122518811 learned_addr learned my addr 172.21.15.103:0/4122518811 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:07:17.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.697+0000 7ff3237fe640 1 --2- 172.21.15.103:0/4122518811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff324102790 0x7ff324198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:17.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.698+0000 7ff328868640 1 --2- 172.21.15.103:0/4122518811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff324106f10 0x7ff32419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:17.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.698+0000 7ff323fff640 1 -- 172.21.15.103:0/4122518811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff324106f10 msgr2=0x7ff32419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:17.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.698+0000 7ff323fff640 1 --2- 172.21.15.103:0/4122518811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff324106f10 0x7ff32419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:17.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.698+0000 7ff323fff640 1 -- 172.21.15.103:0/4122518811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff324102790 msgr2=0x7ff324198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:17.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.698+0000 7ff323fff640 1 --2- 172.21.15.103:0/4122518811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff324102790 0x7ff324198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:17.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.698+0000 7ff323fff640 1 -- 172.21.15.103:0/4122518811 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32419d540 con 0x7ff324101de0 2024-09-19T07:07:17.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.698+0000 7ff323fff640 1 --2- 172.21.15.103:0/4122518811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff324101de0 0x7ff324198590 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7ff31400b4a0 tx=0x7ff31400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:07:17.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.698+0000 7ff3217fa640 1 -- 172.21.15.103:0/4122518811 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314015020 con 0x7ff324101de0 2024-09-19T07:07:17.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.698+0000 7ff3217fa640 1 -- 172.21.15.103:0/4122518811 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff314011d10 con 0x7ff324101de0 2024-09-19T07:07:17.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.698+0000 7ff32a2cb640 1 -- 172.21.15.103:0/4122518811 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32419d830 con 0x7ff324101de0 2024-09-19T07:07:17.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.698+0000 7ff3217fa640 1 -- 172.21.15.103:0/4122518811 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31400f2e0 con 0x7ff324101de0 2024-09-19T07:07:17.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.698+0000 7ff32a2cb640 1 -- 172.21.15.103:0/4122518811 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3241a5110 con 0x7ff324101de0 2024-09-19T07:07:17.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.699+0000 7ff32a2cb640 1 -- 172.21.15.103:0/4122518811 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff324102240 con 0x7ff324101de0 2024-09-19T07:07:17.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.700+0000 7ff3217fa640 1 -- 172.21.15.103:0/4122518811 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7ff314018020 con 0x7ff324101de0 2024-09-19T07:07:17.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.701+0000 7ff3217fa640 1 --2- 172.21.15.103:0/4122518811 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff30c0a33b0 0x7ff30c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:17.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.701+0000 7ff3237fe640 1 --2- 172.21.15.103:0/4122518811 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff30c0a33b0 0x7ff30c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:17.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.701+0000 7ff3217fa640 1 -- 172.21.15.103:0/4122518811 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8600+0+0 (secure 0 0 0) 0x7ff3140cf570 con 0x7ff324101de0 2024-09-19T07:07:17.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.702+0000 7ff3237fe640 1 --2- 172.21.15.103:0/4122518811 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff30c0a33b0 0x7ff30c0a5870 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff3240fdd50 tx=0x7ff318009110 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:07:17.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.705+0000 7ff3217fa640 1 -- 172.21.15.103:0/4122518811 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff31401f020 con 0x7ff324101de0 2024-09-19T07:07:17.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.906+0000 7ff32a2cb640 1 -- 172.21.15.103:0/4122518811 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff32410a150 con 0x7ff324101de0 2024-09-19T07:07:17.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.908+0000 7ff3217fa640 1 -- 172.21.15.103:0/4122518811 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7ff31409c010 con 0x7ff324101de0 2024-09-19T07:07:17.910 INFO:teuthology.orchestra.run.smithi103.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T07:07:17.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.911+0000 7ff32a2cb640 1 -- 172.21.15.103:0/4122518811 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff30c0a33b0 msgr2=0x7ff30c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:17.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.911+0000 7ff32a2cb640 1 --2- 172.21.15.103:0/4122518811 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff30c0a33b0 0x7ff30c0a5870 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff3240fdd50 tx=0x7ff318009110 comp rx=0 tx=0).stop 2024-09-19T07:07:17.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.911+0000 7ff32a2cb640 1 -- 172.21.15.103:0/4122518811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff324101de0 msgr2=0x7ff324198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:17.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.911+0000 7ff32a2cb640 1 --2- 172.21.15.103:0/4122518811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff324101de0 0x7ff324198590 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7ff31400b4a0 tx=0x7ff31400b8a0 comp rx=0 tx=0).stop 2024-09-19T07:07:17.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.912+0000 7ff32a2cb640 1 -- 172.21.15.103:0/4122518811 shutdown_connections 2024-09-19T07:07:17.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.912+0000 7ff32a2cb640 1 --2- 172.21.15.103:0/4122518811 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7ff30c0a33b0 0x7ff30c0a5870 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:17.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.912+0000 7ff32a2cb640 1 --2- 172.21.15.103:0/4122518811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff324106f10 0x7ff32419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:17.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.912+0000 7ff32a2cb640 1 --2- 172.21.15.103:0/4122518811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff324102790 0x7ff324198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:17.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.912+0000 7ff32a2cb640 1 --2- 172.21.15.103:0/4122518811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff324101de0 0x7ff324198590 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:17.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.912+0000 7ff32a2cb640 1 -- 172.21.15.103:0/4122518811 >> 172.21.15.103:0/4122518811 conn(0x7ff3240fd8c0 msgr2=0x7ff3240fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:07:17.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.912+0000 7ff32a2cb640 1 -- 172.21.15.103:0/4122518811 shutdown_connections 2024-09-19T07:07:17.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:17.912+0000 7ff32a2cb640 1 -- 172.21.15.103:0/4122518811 wait complete. 2024-09-19T07:07:18.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:18 smithi174 ceph-mon[35637]: pgmap v565: 97 pgs: 97 active+clean; 1.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 819 B/s wr, 11 op/s 2024-09-19T07:07:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:18 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/4122518811' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T07:07:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:18 smithi174 ceph-mon[35637]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T07:07:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:18 smithi174 ceph-mon[35637]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:07:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:18 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/4122518811' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T07:07:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:18 smithi174 ceph-mon[35637]: osdmap e105: 12 total, 12 up, 12 in 2024-09-19T07:07:18.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:18 smithi174 ceph-mon[35637]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T07:07:18.333 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:07:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-f[65748]: ignoring --setuser ceph since I am not root 2024-09-19T07:07:18.333 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:07:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-f[65748]: ignoring --setgroup ceph since I am not root 2024-09-19T07:07:18.334 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:07:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-f[65748]: starting mds.f at 2024-09-19T07:07:18.334 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:07:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-c[66054]: ignoring --setuser ceph since I am not root 2024-09-19T07:07:18.334 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:07:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-c[66054]: ignoring --setgroup ceph since I am not root 2024-09-19T07:07:18.334 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:07:17 smithi174 ceph-mds[66058]: -- 172.21.15.174:0/62624957 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ee5a0cd860 con 0x55ee5a0ab000 2024-09-19T07:07:18.334 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:07:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-c[66054]: starting mds.c at 2024-09-19T07:07:18.334 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:07:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-i[66358]: ignoring --setuser ceph since I am not root 2024-09-19T07:07:18.334 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:07:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-i[66358]: ignoring --setgroup ceph since I am not root 2024-09-19T07:07:18.334 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:07:17 smithi174 ceph-mds[66362]: -- 172.21.15.174:0/2172121662 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559ed1e7f860 con 0x559ed1e5c800 2024-09-19T07:07:18.335 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:07:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-i[66358]: starting mds.i at 2024-09-19T07:07:18.335 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:07:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-l[65449]: ignoring --setuser ceph since I am not root 2024-09-19T07:07:18.335 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:07:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-l[65449]: ignoring --setgroup ceph since I am not root 2024-09-19T07:07:18.335 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:07:17 smithi174 ceph-mds[65453]: -- 172.21.15.174:0/2700027158 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564f29591860 con 0x564f2956f400 2024-09-19T07:07:18.335 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:07:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-l[65449]: starting mds.l at 2024-09-19T07:07:18.387 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:07:17 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-a[72495]: ignoring --setuser ceph since I am not root 2024-09-19T07:07:18.388 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:07:17 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-a[72495]: ignoring --setgroup ceph since I am not root 2024-09-19T07:07:18.388 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:07:17 smithi103 ceph-mds[72499]: -- 172.21.15.103:0/2083512949 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dc862d3860 con 0x55dc862b0800 2024-09-19T07:07:18.388 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:07:17 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-a[72495]: starting mds.a at 2024-09-19T07:07:18.388 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:07:17 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-g[72112]: ignoring --setuser ceph since I am not root 2024-09-19T07:07:18.388 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:07:17 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-g[72112]: ignoring --setgroup ceph since I am not root 2024-09-19T07:07:18.388 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:07:17 smithi103 ceph-mds[72129]: -- 172.21.15.103:0/4148342290 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2d11e5860 con 0x55c2d11c3000 2024-09-19T07:07:18.388 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:07:17 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-g[72112]: starting mds.g at 2024-09-19T07:07:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:17 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-a[31153]: 2024-09-19T07:07:17.907+0000 7f46c52f5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:07:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:18 smithi103 ceph-mon[31158]: pgmap v565: 97 pgs: 97 active+clean; 1.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 819 B/s wr, 11 op/s 2024-09-19T07:07:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:18 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/4122518811' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T07:07:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:18 smithi103 ceph-mon[31158]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T07:07:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:18 smithi103 ceph-mon[31158]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:07:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:18 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/4122518811' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T07:07:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:18 smithi103 ceph-mon[31158]: osdmap e105: 12 total, 12 up, 12 in 2024-09-19T07:07:18.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:18 smithi103 ceph-mon[31158]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T07:07:18.437 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-h[65311]: ignoring --setuser ceph since I am not root 2024-09-19T07:07:18.437 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-h[65311]: ignoring --setgroup ceph since I am not root 2024-09-19T07:07:18.437 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-mds[65315]: -- 172.21.15.134:0/4029140142 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5612abbd1c20 con 0x5612abbaf000 2024-09-19T07:07:18.437 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-h[65311]: starting mds.h at 2024-09-19T07:07:18.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:18 smithi134 ceph-mon[35356]: pgmap v565: 97 pgs: 97 active+clean; 1.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 819 B/s wr, 11 op/s 2024-09-19T07:07:18.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:18 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/4122518811' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T07:07:18.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:18 smithi134 ceph-mon[35356]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T07:07:18.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:18 smithi134 ceph-mon[35356]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:07:18.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:18 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/4122518811' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T07:07:18.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:18 smithi134 ceph-mon[35356]: osdmap e105: 12 total, 12 up, 12 in 2024-09-19T07:07:18.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:18 smithi134 ceph-mon[35356]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T07:07:18.438 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-b[65626]: ignoring --setuser ceph since I am not root 2024-09-19T07:07:18.438 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-b[65626]: ignoring --setgroup ceph since I am not root 2024-09-19T07:07:18.438 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-mds[65630]: -- 172.21.15.134:0/3810358512 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e63a37860 con 0x558e63a15400 2024-09-19T07:07:18.438 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-b[65626]: starting mds.b at 2024-09-19T07:07:18.439 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-e[65009]: ignoring --setuser ceph since I am not root 2024-09-19T07:07:18.439 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-e[65009]: ignoring --setgroup ceph since I am not root 2024-09-19T07:07:18.439 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-mds[65013]: -- 172.21.15.134:0/4130370641 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b06917b860 con 0x55b069158800 2024-09-19T07:07:18.439 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-e[65009]: starting mds.e at 2024-09-19T07:07:18.439 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-k[64693]: ignoring --setuser ceph since I am not root 2024-09-19T07:07:18.439 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-k[64693]: ignoring --setgroup ceph since I am not root 2024-09-19T07:07:18.439 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-mds[64697]: -- 172.21.15.134:0/3777488880 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a063f01860 con 0x55a063ede800 2024-09-19T07:07:18.440 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:07:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-k[64693]: starting mds.k at 2024-09-19T07:07:18.617 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T07:07:18.864 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:07:19.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.340+0000 7f212b1a2640 1 -- 172.21.15.103:0/3389120684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106a80 msgr2=0x7f2124106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:19.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.340+0000 7f212b1a2640 1 --2- 172.21.15.103:0/3389120684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106a80 0x7f2124106e60 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f2118009930 tx=0x7f211802a420 comp rx=0 tx=0).stop 2024-09-19T07:07:19.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.340+0000 7f212b1a2640 1 -- 172.21.15.103:0/3389120684 shutdown_connections 2024-09-19T07:07:19.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.340+0000 7f212b1a2640 1 --2- 172.21.15.103:0/3389120684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2124101420 0x7f212410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:19.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.340+0000 7f212b1a2640 1 --2- 172.21.15.103:0/3389120684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2124100a80 0x7f2124100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:19.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.340+0000 7f212b1a2640 1 --2- 172.21.15.103:0/3389120684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106a80 0x7f2124106e60 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:19.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.340+0000 7f212b1a2640 1 -- 172.21.15.103:0/3389120684 >> 172.21.15.103:0/3389120684 conn(0x7f21240fc7a0 msgr2=0x7f21240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:07:19.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.340+0000 7f212b1a2640 1 -- 172.21.15.103:0/3389120684 shutdown_connections 2024-09-19T07:07:19.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.340+0000 7f212b1a2640 1 -- 172.21.15.103:0/3389120684 wait complete. 2024-09-19T07:07:19.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.340+0000 7f212b1a2640 1 Processor -- start 2024-09-19T07:07:19.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.340+0000 7f212b1a2640 1 -- start start 2024-09-19T07:07:19.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f212b1a2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2124100a80 0x7f21240fce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:19.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f212b1a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2124101420 0x7f21240fd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:19.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f212b1a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106a80 0x7f21240fd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:19.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f212b1a2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2124110ea0 con 0x7f2124106a80 2024-09-19T07:07:19.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f212b1a2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2124110d20 con 0x7f2124100a80 2024-09-19T07:07:19.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f212b1a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2124111020 con 0x7f2124101420 2024-09-19T07:07:19.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f212973f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106a80 0x7f21240fd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:19.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f212973f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106a80 0x7f21240fd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40230/0 (socket says 172.21.15.103:40230) 2024-09-19T07:07:19.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f212973f640 1 -- 172.21.15.103:0/2415608639 learned_addr learned my addr 172.21.15.103:0/2415608639 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:07:19.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f2128f3e640 1 --2- 172.21.15.103:0/2415608639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2124100a80 0x7f21240fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:19.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f2123fff640 1 --2- 172.21.15.103:0/2415608639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2124101420 0x7f21240fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:19.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f212973f640 1 -- 172.21.15.103:0/2415608639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2124101420 msgr2=0x7f21240fd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:19.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f212973f640 1 --2- 172.21.15.103:0/2415608639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2124101420 0x7f21240fd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:19.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f212973f640 1 -- 172.21.15.103:0/2415608639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2124100a80 msgr2=0x7f21240fce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:19.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f212973f640 1 --2- 172.21.15.103:0/2415608639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2124100a80 0x7f21240fce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:19.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f212973f640 1 -- 172.21.15.103:0/2415608639 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21241a6940 con 0x7f2124106a80 2024-09-19T07:07:19.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.341+0000 7f212973f640 1 --2- 172.21.15.103:0/2415608639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106a80 0x7f21240fd8a0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f211c00cce0 tx=0x7f211c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:07:19.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.342+0000 7f2121ffb640 1 -- 172.21.15.103:0/2415608639 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c00f450 con 0x7f2124106a80 2024-09-19T07:07:19.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.342+0000 7f2121ffb640 1 -- 172.21.15.103:0/2415608639 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f211c00fad0 con 0x7f2124106a80 2024-09-19T07:07:19.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.342+0000 7f212b1a2640 1 -- 172.21.15.103:0/2415608639 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21241a6c30 con 0x7f2124106a80 2024-09-19T07:07:19.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.342+0000 7f2121ffb640 1 -- 172.21.15.103:0/2415608639 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c018b60 con 0x7f2124106a80 2024-09-19T07:07:19.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.342+0000 7f212b1a2640 1 -- 172.21.15.103:0/2415608639 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21241a7170 con 0x7f2124106a80 2024-09-19T07:07:19.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.343+0000 7f212b1a2640 1 -- 172.21.15.103:0/2415608639 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2124100ee0 con 0x7f2124106a80 2024-09-19T07:07:19.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.344+0000 7f2121ffb640 1 -- 172.21.15.103:0/2415608639 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f211c00fc80 con 0x7f2124106a80 2024-09-19T07:07:19.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.344+0000 7f2121ffb640 1 --2- 172.21.15.103:0/2415608639 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21140a3430 0x7f21140a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:19.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.344+0000 7f2128f3e640 1 --2- 172.21.15.103:0/2415608639 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21140a3430 0x7f21140a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:19.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.344+0000 7f2121ffb640 1 -- 172.21.15.103:0/2415608639 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8986+0+0 (secure 0 0 0) 0x7f211c0d0320 con 0x7f2124106a80 2024-09-19T07:07:19.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.345+0000 7f2128f3e640 1 --2- 172.21.15.103:0/2415608639 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21140a3430 0x7f21140a58f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2118009900 tx=0x7f21180058d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:07:19.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.349+0000 7f2121ffb640 1 -- 172.21.15.103:0/2415608639 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f211c014030 con 0x7f2124106a80 2024-09-19T07:07:19.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:19.550+0000 7f212b1a2640 1 -- 172.21.15.103:0/2415608639 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f212410a000 con 0x7f2124106a80 2024-09-19T07:07:20.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:20.076+0000 7f2121ffb640 1 -- 172.21.15.103:0/2415608639 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7f211c09cb30 con 0x7f2124106a80 2024-09-19T07:07:20.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:20.078+0000 7f212b1a2640 1 -- 172.21.15.103:0/2415608639 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21140a3430 msgr2=0x7f21140a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:20.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:20.078+0000 7f212b1a2640 1 --2- 172.21.15.103:0/2415608639 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21140a3430 0x7f21140a58f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2118009900 tx=0x7f21180058d0 comp rx=0 tx=0).stop 2024-09-19T07:07:20.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:20.078+0000 7f212b1a2640 1 -- 172.21.15.103:0/2415608639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106a80 msgr2=0x7f21240fd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:20.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:20.078+0000 7f212b1a2640 1 --2- 172.21.15.103:0/2415608639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106a80 0x7f21240fd8a0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f211c00cce0 tx=0x7f211c00a620 comp rx=0 tx=0).stop 2024-09-19T07:07:20.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:20.078+0000 7f212b1a2640 1 -- 172.21.15.103:0/2415608639 shutdown_connections 2024-09-19T07:07:20.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:20.078+0000 7f212b1a2640 1 --2- 172.21.15.103:0/2415608639 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f21140a3430 0x7f21140a58f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:20.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:20.078+0000 7f212b1a2640 1 --2- 172.21.15.103:0/2415608639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2124106a80 0x7f21240fd8a0 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:20.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:20.078+0000 7f212b1a2640 1 --2- 172.21.15.103:0/2415608639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2124101420 0x7f21240fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:20.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:20.078+0000 7f212b1a2640 1 --2- 172.21.15.103:0/2415608639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2124100a80 0x7f21240fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:20.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:20.078+0000 7f212b1a2640 1 -- 172.21.15.103:0/2415608639 >> 172.21.15.103:0/2415608639 conn(0x7f21240fc7a0 msgr2=0x7f212406aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:07:20.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:20.078+0000 7f212b1a2640 1 -- 172.21.15.103:0/2415608639 shutdown_connections 2024-09-19T07:07:20.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:20.078+0000 7f212b1a2640 1 -- 172.21.15.103:0/2415608639 wait complete. 2024-09-19T07:07:20.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: pgmap v567: 97 pgs: 97 active+clean; 1.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1003 B/s wr, 10 op/s 2024-09-19T07:07:20.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.103:6838/3945854272,v1:172.21.15.103:6839/3945854272] up:boot 2024-09-19T07:07:20.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.134:6832/1955170157,v1:172.21.15.134:6833/1955170157] up:boot 2024-09-19T07:07:20.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.103:6840/2789738933,v1:172.21.15.103:6841/2789738933] up:boot 2024-09-19T07:07:20.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.134:6834/104501337,v1:172.21.15.134:6835/104501337] up:boot 2024-09-19T07:07:20.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.134:6836/2037740069,v1:172.21.15.134:6837/2037740069] up:boot 2024-09-19T07:07:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.174:6832/4270412786,v1:172.21.15.174:6834/4270412786] up:boot 2024-09-19T07:07:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.174:6833/3479840111,v1:172.21.15.174:6835/3479840111] up:boot 2024-09-19T07:07:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.134:6838/2308903079,v1:172.21.15.134:6839/2308903079] up:boot 2024-09-19T07:07:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.174:6836/659682975,v1:172.21.15.174:6838/659682975] up:boot 2024-09-19T07:07:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: mds.? [v2:172.21.15.174:6837/4285314574,v1:172.21.15.174:6839/4285314574] up:boot 2024-09-19T07:07:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T07:07:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T07:07:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T07:07:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:07:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:07:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:07:20.188 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:07:20.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T07:07:20.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:07:20.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:07:20.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T07:07:20.189 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:19 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2415608639' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:07:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: pgmap v567: 97 pgs: 97 active+clean; 1.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1003 B/s wr, 10 op/s 2024-09-19T07:07:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.103:6838/3945854272,v1:172.21.15.103:6839/3945854272] up:boot 2024-09-19T07:07:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.134:6832/1955170157,v1:172.21.15.134:6833/1955170157] up:boot 2024-09-19T07:07:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.103:6840/2789738933,v1:172.21.15.103:6841/2789738933] up:boot 2024-09-19T07:07:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.134:6834/104501337,v1:172.21.15.134:6835/104501337] up:boot 2024-09-19T07:07:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.134:6836/2037740069,v1:172.21.15.134:6837/2037740069] up:boot 2024-09-19T07:07:20.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.174:6832/4270412786,v1:172.21.15.174:6834/4270412786] up:boot 2024-09-19T07:07:20.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.174:6833/3479840111,v1:172.21.15.174:6835/3479840111] up:boot 2024-09-19T07:07:20.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.134:6838/2308903079,v1:172.21.15.134:6839/2308903079] up:boot 2024-09-19T07:07:20.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.174:6836/659682975,v1:172.21.15.174:6838/659682975] up:boot 2024-09-19T07:07:20.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: mds.? [v2:172.21.15.174:6837/4285314574,v1:172.21.15.174:6839/4285314574] up:boot 2024-09-19T07:07:20.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T07:07:20.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T07:07:20.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T07:07:20.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:07:20.334 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:07:20.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:07:20.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:07:20.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T07:07:20.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:07:20.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:07:20.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T07:07:20.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:19 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2415608639' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:07:20.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: pgmap v567: 97 pgs: 97 active+clean; 1.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1003 B/s wr, 10 op/s 2024-09-19T07:07:20.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.103:6838/3945854272,v1:172.21.15.103:6839/3945854272] up:boot 2024-09-19T07:07:20.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.134:6832/1955170157,v1:172.21.15.134:6833/1955170157] up:boot 2024-09-19T07:07:20.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.103:6840/2789738933,v1:172.21.15.103:6841/2789738933] up:boot 2024-09-19T07:07:20.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.134:6834/104501337,v1:172.21.15.134:6835/104501337] up:boot 2024-09-19T07:07:20.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.134:6836/2037740069,v1:172.21.15.134:6837/2037740069] up:boot 2024-09-19T07:07:20.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.174:6832/4270412786,v1:172.21.15.174:6834/4270412786] up:boot 2024-09-19T07:07:20.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.174:6833/3479840111,v1:172.21.15.174:6835/3479840111] up:boot 2024-09-19T07:07:20.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.134:6838/2308903079,v1:172.21.15.134:6839/2308903079] up:boot 2024-09-19T07:07:20.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.174:6836/659682975,v1:172.21.15.174:6838/659682975] up:boot 2024-09-19T07:07:20.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: mds.? [v2:172.21.15.174:6837/4285314574,v1:172.21.15.174:6839/4285314574] up:boot 2024-09-19T07:07:20.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T07:07:20.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T07:07:20.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T07:07:20.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:07:20.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:07:20.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:07:20.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:07:20.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T07:07:20.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:07:20.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:07:20.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: from='mgr.15210 172.21.15.103:0/1046034432' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T07:07:20.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:19 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2415608639' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:07:20.822 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T07:07:21.075 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:07:21.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:20 smithi103 ceph-mon[31158]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T07:07:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:20 smithi103 ceph-mon[31158]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T07:07:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:20 smithi103 ceph-mon[31158]: Cluster is now healthy 2024-09-19T07:07:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:20 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/2415608639' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T07:07:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:20 smithi103 ceph-mon[31158]: fsmap 12 up:standby 2024-09-19T07:07:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:20 smithi134 ceph-mon[35356]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T07:07:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:20 smithi134 ceph-mon[35356]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T07:07:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:20 smithi134 ceph-mon[35356]: Cluster is now healthy 2024-09-19T07:07:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:20 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/2415608639' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T07:07:21.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:20 smithi134 ceph-mon[35356]: fsmap 12 up:standby 2024-09-19T07:07:21.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:20 smithi174 ceph-mon[35637]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T07:07:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:20 smithi174 ceph-mon[35637]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T07:07:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:20 smithi174 ceph-mon[35637]: Cluster is now healthy 2024-09-19T07:07:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:20 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/2415608639' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T07:07:21.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:20 smithi174 ceph-mon[35637]: fsmap 12 up:standby 2024-09-19T07:07:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.492+0000 7f227eaf8640 1 -- 172.21.15.103:0/2791206892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2278100ca0 msgr2=0x7f2278106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.492+0000 7f227eaf8640 1 --2- 172.21.15.103:0/2791206892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2278100ca0 0x7f2278106840 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f2260009930 tx=0x7f226002a420 comp rx=0 tx=0).stop 2024-09-19T07:07:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.492+0000 7f227eaf8640 1 -- 172.21.15.103:0/2791206892 shutdown_connections 2024-09-19T07:07:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.492+0000 7f227eaf8640 1 --2- 172.21.15.103:0/2791206892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2278106f50 0x7f227810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.492+0000 7f227eaf8640 1 --2- 172.21.15.103:0/2791206892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2278100ca0 0x7f2278106840 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.492+0000 7f227eaf8640 1 --2- 172.21.15.103:0/2791206892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22781002f0 0x7f22781006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.492+0000 7f227eaf8640 1 -- 172.21.15.103:0/2791206892 >> 172.21.15.103:0/2791206892 conn(0x7f2278077210 msgr2=0x7f22780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:07:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.492+0000 7f227eaf8640 1 -- 172.21.15.103:0/2791206892 shutdown_connections 2024-09-19T07:07:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.492+0000 7f227eaf8640 1 -- 172.21.15.103:0/2791206892 wait complete. 2024-09-19T07:07:21.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f227eaf8640 1 Processor -- start 2024-09-19T07:07:21.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f227eaf8640 1 -- start start 2024-09-19T07:07:21.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f227eaf8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22781002f0 0x7f22781984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:21.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f227eaf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2278100ca0 0x7f2278198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:21.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f227eaf8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2278106f50 0x7f227819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:21.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f227eaf8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2278111fa0 con 0x7f2278100ca0 2024-09-19T07:07:21.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f227eaf8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2278111e20 con 0x7f2278106f50 2024-09-19T07:07:21.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f227eaf8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2278112120 con 0x7f22781002f0 2024-09-19T07:07:21.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f2277fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2278100ca0 0x7f2278198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:21.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f2277fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2278100ca0 0x7f2278198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40254/0 (socket says 172.21.15.103:40254) 2024-09-19T07:07:21.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f2277fff640 1 -- 172.21.15.103:0/3502543193 learned_addr learned my addr 172.21.15.103:0/3502543193 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:07:21.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f227c894640 1 --2- 172.21.15.103:0/3502543193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22781002f0 0x7f22781984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:21.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f227d095640 1 --2- 172.21.15.103:0/3502543193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2278106f50 0x7f227819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:21.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f2277fff640 1 -- 172.21.15.103:0/3502543193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22781002f0 msgr2=0x7f22781984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:21.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f2277fff640 1 --2- 172.21.15.103:0/3502543193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22781002f0 0x7f22781984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:21.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f2277fff640 1 -- 172.21.15.103:0/3502543193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2278106f50 msgr2=0x7f227819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:21.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f2277fff640 1 --2- 172.21.15.103:0/3502543193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2278106f50 0x7f227819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:21.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f2277fff640 1 -- 172.21.15.103:0/3502543193 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227819d480 con 0x7f2278100ca0 2024-09-19T07:07:21.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.493+0000 7f227d095640 1 --2- 172.21.15.103:0/3502543193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2278106f50 0x7f227819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:07:21.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.494+0000 7f2277fff640 1 --2- 172.21.15.103:0/3502543193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2278100ca0 0x7f2278198a10 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f2260009230 tx=0x7f22600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:07:21.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.494+0000 7f2275ffb640 1 -- 172.21.15.103:0/3502543193 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226003f030 con 0x7f2278100ca0 2024-09-19T07:07:21.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.494+0000 7f2275ffb640 1 -- 172.21.15.103:0/3502543193 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2260003ce0 con 0x7f2278100ca0 2024-09-19T07:07:21.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.494+0000 7f2275ffb640 1 -- 172.21.15.103:0/3502543193 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260035070 con 0x7f2278100ca0 2024-09-19T07:07:21.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.494+0000 7f227eaf8640 1 -- 172.21.15.103:0/3502543193 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f227819d710 con 0x7f2278100ca0 2024-09-19T07:07:21.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.494+0000 7f227eaf8640 1 -- 172.21.15.103:0/3502543193 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22781a5050 con 0x7f2278100ca0 2024-09-19T07:07:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.495+0000 7f227eaf8640 1 -- 172.21.15.103:0/3502543193 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22781033b0 con 0x7f2278100ca0 2024-09-19T07:07:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.496+0000 7f2275ffb640 1 -- 172.21.15.103:0/3502543193 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7f2260042030 con 0x7f2278100ca0 2024-09-19T07:07:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.497+0000 7f2275ffb640 1 --2- 172.21.15.103:0/3502543193 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f22640a33b0 0x7f22640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.497+0000 7f227c894640 1 --2- 172.21.15.103:0/3502543193 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f22640a33b0 0x7f22640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.497+0000 7f2275ffb640 1 -- 172.21.15.103:0/3502543193 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8986+0+0 (secure 0 0 0) 0x7f22600f04b0 con 0x7f2278100ca0 2024-09-19T07:07:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.497+0000 7f227c894640 1 --2- 172.21.15.103:0/3502543193 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f22640a33b0 0x7f22640a5870 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f22700058c0 tx=0x7f227000a2d0 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:07:21.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.501+0000 7f2275ffb640 1 -- 172.21.15.103:0/3502543193 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f22600bccc0 con 0x7f2278100ca0 2024-09-19T07:07:21.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.666+0000 7f227eaf8640 1 -- 172.21.15.103:0/3502543193 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f227810a120 con 0x7f2278100ca0 2024-09-19T07:07:21.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.752+0000 7f2275ffb640 1 -- 172.21.15.103:0/3502543193 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v106) ==== 181+0+0 (secure 0 0 0) 0x7f22600bc5c0 con 0x7f2278100ca0 2024-09-19T07:07:21.754 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_metadata' removed 2024-09-19T07:07:21.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.756+0000 7f227eaf8640 1 -- 172.21.15.103:0/3502543193 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f22640a33b0 msgr2=0x7f22640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:21.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.756+0000 7f227eaf8640 1 --2- 172.21.15.103:0/3502543193 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f22640a33b0 0x7f22640a5870 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f22700058c0 tx=0x7f227000a2d0 comp rx=0 tx=0).stop 2024-09-19T07:07:21.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.756+0000 7f227eaf8640 1 -- 172.21.15.103:0/3502543193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2278100ca0 msgr2=0x7f2278198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:21.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.756+0000 7f227eaf8640 1 --2- 172.21.15.103:0/3502543193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2278100ca0 0x7f2278198a10 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f2260009230 tx=0x7f22600042f0 comp rx=0 tx=0).stop 2024-09-19T07:07:21.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.756+0000 7f227eaf8640 1 -- 172.21.15.103:0/3502543193 shutdown_connections 2024-09-19T07:07:21.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.756+0000 7f227eaf8640 1 --2- 172.21.15.103:0/3502543193 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7f22640a33b0 0x7f22640a5870 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:21.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.756+0000 7f227eaf8640 1 --2- 172.21.15.103:0/3502543193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2278106f50 0x7f227819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:21.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.756+0000 7f227eaf8640 1 --2- 172.21.15.103:0/3502543193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2278100ca0 0x7f2278198a10 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:21.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.756+0000 7f227eaf8640 1 --2- 172.21.15.103:0/3502543193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22781002f0 0x7f22781984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:21.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.756+0000 7f227eaf8640 1 -- 172.21.15.103:0/3502543193 >> 172.21.15.103:0/3502543193 conn(0x7f2278077210 msgr2=0x7f22780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:07:21.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.757+0000 7f227eaf8640 1 -- 172.21.15.103:0/3502543193 shutdown_connections 2024-09-19T07:07:21.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:21.757+0000 7f227eaf8640 1 -- 172.21.15.103:0/3502543193 wait complete. 2024-09-19T07:07:22.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:22 smithi174 ceph-mon[35637]: pgmap v568: 97 pgs: 97 active+clean; 1.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 716 B/s wr, 9 op/s 2024-09-19T07:07:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:22 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3502543193' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:07:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:22 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/3502543193' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:07:22.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:22 smithi174 ceph-mon[35637]: osdmap e106: 12 total, 12 up, 12 in 2024-09-19T07:07:22.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:22 smithi103 ceph-mon[31158]: pgmap v568: 97 pgs: 97 active+clean; 1.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 716 B/s wr, 9 op/s 2024-09-19T07:07:22.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:22 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3502543193' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:07:22.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:22 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/3502543193' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:07:22.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:22 smithi103 ceph-mon[31158]: osdmap e106: 12 total, 12 up, 12 in 2024-09-19T07:07:22.419 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid d158d934-7651-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T07:07:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:22 smithi134 ceph-mon[35356]: pgmap v568: 97 pgs: 97 active+clean; 1.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 716 B/s wr, 9 op/s 2024-09-19T07:07:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:22 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3502543193' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:07:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:22 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/3502543193' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:07:22.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:22 smithi134 ceph-mon[35356]: osdmap e106: 12 total, 12 up, 12 in 2024-09-19T07:07:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:07:23.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.140+0000 7fe527f9e640 1 -- 172.21.15.103:0/376696430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe520102d90 msgr2=0x7fe520103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:23.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.140+0000 7fe527f9e640 1 --2- 172.21.15.103:0/376696430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe520102d90 0x7fe520103170 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fe518009930 tx=0x7fe51802a420 comp rx=0 tx=0).stop 2024-09-19T07:07:23.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.140+0000 7fe527f9e640 1 -- 172.21.15.103:0/376696430 shutdown_connections 2024-09-19T07:07:23.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.140+0000 7fe527f9e640 1 --2- 172.21.15.103:0/376696430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520108290 0x7fe52010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:23.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.140+0000 7fe527f9e640 1 --2- 172.21.15.103:0/376696430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520103740 0x7fe520107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:23.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.140+0000 7fe527f9e640 1 --2- 172.21.15.103:0/376696430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe520102d90 0x7fe520103170 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:23.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.140+0000 7fe527f9e640 1 -- 172.21.15.103:0/376696430 >> 172.21.15.103:0/376696430 conn(0x7fe5200fc7d0 msgr2=0x7fe5200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:07:23.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.140+0000 7fe527f9e640 1 -- 172.21.15.103:0/376696430 shutdown_connections 2024-09-19T07:07:23.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.140+0000 7fe527f9e640 1 -- 172.21.15.103:0/376696430 wait complete. 2024-09-19T07:07:23.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.140+0000 7fe527f9e640 1 Processor -- start 2024-09-19T07:07:23.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.140+0000 7fe527f9e640 1 -- start start 2024-09-19T07:07:23.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe527f9e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe520102d90 0x7fe5201985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:23.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe527f9e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520103740 0x7fe520198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:23.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe527f9e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520108290 0x7fe52019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:23.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe527f9e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe520112090 con 0x7fe520102d90 2024-09-19T07:07:23.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe527f9e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe520111f10 con 0x7fe520108290 2024-09-19T07:07:23.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe527f9e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe520112210 con 0x7fe520103740 2024-09-19T07:07:23.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe525d3a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe520102d90 0x7fe5201985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:23.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe525d3a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe520102d90 0x7fe5201985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40270/0 (socket says 172.21.15.103:40270) 2024-09-19T07:07:23.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe525d3a640 1 -- 172.21.15.103:0/1297645897 learned_addr learned my addr 172.21.15.103:0/1297645897 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:07:23.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe52653b640 1 --2- 172.21.15.103:0/1297645897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520108290 0x7fe52019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:23.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe525539640 1 --2- 172.21.15.103:0/1297645897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520103740 0x7fe520198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:23.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe525d3a640 1 -- 172.21.15.103:0/1297645897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520103740 msgr2=0x7fe520198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:23.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe525d3a640 1 --2- 172.21.15.103:0/1297645897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520103740 0x7fe520198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:23.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe525d3a640 1 -- 172.21.15.103:0/1297645897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520108290 msgr2=0x7fe52019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:23.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe525d3a640 1 --2- 172.21.15.103:0/1297645897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520108290 0x7fe52019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:23.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe525d3a640 1 -- 172.21.15.103:0/1297645897 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52019d550 con 0x7fe520102d90 2024-09-19T07:07:23.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe525539640 1 --2- 172.21.15.103:0/1297645897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520103740 0x7fe520198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:07:23.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe525d3a640 1 --2- 172.21.15.103:0/1297645897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe520102d90 0x7fe5201985a0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7fe518009230 tx=0x7fe518004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:07:23.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe51effd640 1 -- 172.21.15.103:0/1297645897 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51803f030 con 0x7fe520102d90 2024-09-19T07:07:23.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.141+0000 7fe51effd640 1 -- 172.21.15.103:0/1297645897 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe518003ad0 con 0x7fe520102d90 2024-09-19T07:07:23.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.142+0000 7fe527f9e640 1 -- 172.21.15.103:0/1297645897 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52019d7e0 con 0x7fe520102d90 2024-09-19T07:07:23.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.142+0000 7fe51effd640 1 -- 172.21.15.103:0/1297645897 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518035070 con 0x7fe520102d90 2024-09-19T07:07:23.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.142+0000 7fe527f9e640 1 -- 172.21.15.103:0/1297645897 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5201a5120 con 0x7fe520102d90 2024-09-19T07:07:23.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.143+0000 7fe527f9e640 1 -- 172.21.15.103:0/1297645897 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe52006a7f0 con 0x7fe520102d90 2024-09-19T07:07:23.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.144+0000 7fe51effd640 1 -- 172.21.15.103:0/1297645897 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138487+0+0 (secure 0 0 0) 0x7fe518042030 con 0x7fe520102d90 2024-09-19T07:07:23.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.145+0000 7fe51effd640 1 --2- 172.21.15.103:0/1297645897 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe5080a3430 0x7fe5080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:07:23.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.145+0000 7fe525539640 1 --2- 172.21.15.103:0/1297645897 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe5080a3430 0x7fe5080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:07:23.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.145+0000 7fe51effd640 1 -- 172.21.15.103:0/1297645897 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8403+0+0 (secure 0 0 0) 0x7fe5180efe50 con 0x7fe520102d90 2024-09-19T07:07:23.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.145+0000 7fe525539640 1 --2- 172.21.15.103:0/1297645897 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe5080a3430 0x7fe5080a58f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe5200fee40 tx=0x7fe514005e50 comp rx=0 tx=0).ready entity=mgr.15210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:07:23.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.149+0000 7fe51effd640 1 -- 172.21.15.103:0/1297645897 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe5180bc8a0 con 0x7fe520102d90 2024-09-19T07:07:23.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:23.316+0000 7fe527f9e640 1 -- 172.21.15.103:0/1297645897 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe52010a2a0 con 0x7fe520102d90 2024-09-19T07:07:24.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:24.081+0000 7fe51effd640 1 -- 172.21.15.103:0/1297645897 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v107) ==== 169+0+0 (secure 0 0 0) 0x7fe5180bc1a0 con 0x7fe520102d90 2024-09-19T07:07:24.083 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_data' removed 2024-09-19T07:07:24.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:24.084+0000 7fe527f9e640 1 -- 172.21.15.103:0/1297645897 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe5080a3430 msgr2=0x7fe5080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:24.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:24.084+0000 7fe527f9e640 1 --2- 172.21.15.103:0/1297645897 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe5080a3430 0x7fe5080a58f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe5200fee40 tx=0x7fe514005e50 comp rx=0 tx=0).stop 2024-09-19T07:07:24.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:24.084+0000 7fe527f9e640 1 -- 172.21.15.103:0/1297645897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe520102d90 msgr2=0x7fe5201985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:07:24.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:24.084+0000 7fe527f9e640 1 --2- 172.21.15.103:0/1297645897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe520102d90 0x7fe5201985a0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7fe518009230 tx=0x7fe518004270 comp rx=0 tx=0).stop 2024-09-19T07:07:24.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:24.084+0000 7fe527f9e640 1 -- 172.21.15.103:0/1297645897 shutdown_connections 2024-09-19T07:07:24.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:24.084+0000 7fe527f9e640 1 --2- 172.21.15.103:0/1297645897 >> [v2:172.21.15.103:6800/82882809,v1:172.21.15.103:6801/82882809] conn(0x7fe5080a3430 0x7fe5080a58f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:24.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:24.084+0000 7fe527f9e640 1 --2- 172.21.15.103:0/1297645897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe520108290 0x7fe52019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:24.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:24.084+0000 7fe527f9e640 1 --2- 172.21.15.103:0/1297645897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520103740 0x7fe520198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:24.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:24.084+0000 7fe527f9e640 1 --2- 172.21.15.103:0/1297645897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe520102d90 0x7fe5201985a0 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:07:24.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:24.084+0000 7fe527f9e640 1 -- 172.21.15.103:0/1297645897 >> 172.21.15.103:0/1297645897 conn(0x7fe5200fc7d0 msgr2=0x7fe520105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:07:24.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:24.085+0000 7fe527f9e640 1 -- 172.21.15.103:0/1297645897 shutdown_connections 2024-09-19T07:07:24.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:07:24.085+0000 7fe527f9e640 1 -- 172.21.15.103:0/1297645897 wait complete. 2024-09-19T07:07:24.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:24 smithi103 ceph-mon[31158]: pgmap v570: 65 pgs: 65 active+clean; 577 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:07:24.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:24 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1297645897' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:07:24.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:24 smithi174 ceph-mon[35637]: pgmap v570: 65 pgs: 65 active+clean; 577 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:07:24.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:24 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1297645897' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:07:24.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:24 smithi134 ceph-mon[35356]: pgmap v570: 65 pgs: 65 active+clean; 577 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:07:24.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:24 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1297645897' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:07:24.861 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:07:24.889 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:07:24.924 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:07:24.952 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T07:07:24.953 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T07:07:24.969 DEBUG:teuthology.orchestra.run.smithi134:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T07:07:24.987 DEBUG:teuthology.orchestra.run.smithi174:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T07:07:25.007 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T07:07:25.008 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T07:07:25.008 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mon.a 2024-09-19T07:07:25.094 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:25 smithi103 systemd[1]: Stopping Ceph mon.a for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:07:25.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:25 smithi134 ceph-mon[35356]: from='client.? 172.21.15.103:0/1297645897' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:07:25.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:25 smithi134 ceph-mon[35356]: osdmap e107: 12 total, 12 up, 12 in 2024-09-19T07:07:25.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:25 smithi174 ceph-mon[35637]: from='client.? 172.21.15.103:0/1297645897' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:07:25.333 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:25 smithi174 ceph-mon[35637]: osdmap e107: 12 total, 12 up, 12 in 2024-09-19T07:07:25.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:25 smithi103 ceph-mon[31158]: from='client.? 172.21.15.103:0/1297645897' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:07:25.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:25 smithi103 ceph-mon[31158]: osdmap e107: 12 total, 12 up, 12 in 2024-09-19T07:07:25.387 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:25 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-a[31153]: 2024-09-19T07:07:25.337+0000 7f46cab00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:07:25.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:25 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-a[31153]: 2024-09-19T07:07:25.337+0000 7f46cab00640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T07:07:25.668 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:25 smithi103 podman[136777]: 2024-09-19 07:07:25.400370147 +0000 UTC m=+0.190591118 container died 0b2d620a0814f47adb7b84144bc63a8b715f29fc07a1846a6080b944f3fcd578 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-19T07:07:26.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:25 smithi103 podman[136777]: 2024-09-19 07:07:25.858855743 +0000 UTC m=+0.649076752 container cleanup 0b2d620a0814f47adb7b84144bc63a8b715f29fc07a1846a6080b944f3fcd578 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T07:07:26.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:07:25 smithi103 bash[136777]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-a 2024-09-19T07:07:26.282 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T07:07:26.341 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:07:26.341 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T07:07:26.341 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T07:07:26.341 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mon.b 2024-09-19T07:07:26.687 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:26 smithi134 systemd[1]: Stopping Ceph mon.b for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:07:27.070 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:26 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-b[35352]: 2024-09-19T07:07:26.767+0000 7f71b6f1d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:07:27.071 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:26 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-b[35352]: 2024-09-19T07:07:26.767+0000 7f71b6f1d640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T07:07:27.071 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:26 smithi134 podman[71952]: 2024-09-19 07:07:26.840399396 +0000 UTC m=+0.298473161 container died 7a24aa27c53bd62577d0880ca2123fde232675f37a7197a6932385ec7f21bab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-19T07:07:27.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:27 smithi134 podman[71952]: 2024-09-19 07:07:27.202087919 +0000 UTC m=+0.660161674 container cleanup 7a24aa27c53bd62577d0880ca2123fde232675f37a7197a6932385ec7f21bab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-19T07:07:27.952 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:27 smithi134 bash[71952]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-b 2024-09-19T07:07:27.953 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T07:07:27.954 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:27 smithi134 podman[71965]: 2024-09-19 07:07:27.511806616 +0000 UTC m=+0.665824693 container remove 7a24aa27c53bd62577d0880ca2123fde232675f37a7197a6932385ec7f21bab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T07:07:27.954 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:27 smithi134 systemd[1]: ceph-d158d934-7651-11ef-bceb-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-19T07:07:27.954 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:27 smithi134 systemd[1]: Stopped Ceph mon.b for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T07:07:27.954 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:07:27 smithi134 systemd[1]: ceph-d158d934-7651-11ef-bceb-c7b262605968@mon.b.service: Consumed 15.285s CPU time. 2024-09-19T07:07:28.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:07:28.003 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T07:07:28.003 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T07:07:28.003 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mon.c 2024-09-19T07:07:28.292 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:28 smithi174 systemd[1]: Stopping Ceph mon.c for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:07:28.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:28 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-c[35632]: 2024-09-19T07:07:28.290+0000 7febfad64640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:07:28.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:28 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-c[35632]: 2024-09-19T07:07:28.290+0000 7febfad64640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T07:07:28.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:28 smithi174 podman[73343]: 2024-09-19 07:07:28.496342927 +0000 UTC m=+0.298865591 container died 20df604859787554ab824e27d3818b846b735254786065b7a83faf110ddb4891 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-c, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T07:07:29.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:28 smithi174 podman[73343]: 2024-09-19 07:07:28.841294163 +0000 UTC m=+0.643816815 container cleanup 20df604859787554ab824e27d3818b846b735254786065b7a83faf110ddb4891 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T07:07:29.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:07:28 smithi174 bash[73343]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mon-c 2024-09-19T07:07:29.265 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T07:07:29.314 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:07:29.314 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T07:07:29.314 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T07:07:29.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mgr.x 2024-09-19T07:07:29.631 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:07:29 smithi103 systemd[1]: Stopping Ceph mgr.x for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:07:29.887 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:07:29 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x[31388]: 2024-09-19T07:07:29.628+0000 7fb38d1ad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:07:29.887 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:07:29 smithi103 podman[136895]: 2024-09-19 07:07:29.661045161 +0000 UTC m=+0.155873481 container died b4b96839d3583e0872c36852435d25a01a2492f5c94488682f434a871f93afd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-19T07:07:30.387 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:07:30 smithi103 podman[136895]: 2024-09-19 07:07:30.206213185 +0000 UTC m=+0.701041497 container cleanup b4b96839d3583e0872c36852435d25a01a2492f5c94488682f434a871f93afd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T07:07:30.387 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:07:30 smithi103 bash[136895]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x 2024-09-19T07:07:30.669 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:07:30 smithi103 podman[136909]: 2024-09-19 07:07:30.418531497 +0000 UTC m=+0.752844547 container remove b4b96839d3583e0872c36852435d25a01a2492f5c94488682f434a871f93afd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True) 2024-09-19T07:07:30.672 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T07:07:30.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:07:30.708 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T07:07:30.708 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T07:07:30.708 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mgr.y 2024-09-19T07:07:31.062 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:07:30 smithi134 systemd[1]: Stopping Ceph mgr.y for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:07:31.349 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:07:31 smithi134 podman[72071]: 2024-09-19 07:07:31.062209753 +0000 UTC m=+0.156652928 container died 534d031929765e28c32700d636af6887a042dde41f13a1c081dd0f43567b28e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T07:07:31.687 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:07:31 smithi134 podman[72071]: 2024-09-19 07:07:31.447582132 +0000 UTC m=+0.542025305 container cleanup 534d031929765e28c32700d636af6887a042dde41f13a1c081dd0f43567b28e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T07:07:31.687 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:07:31 smithi134 bash[72071]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-y 2024-09-19T07:07:31.838 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T07:07:31.872 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:07:31.872 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T07:07:31.872 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T07:07:31.873 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mgr.z 2024-09-19T07:07:32.200 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:07:31 smithi174 systemd[1]: Stopping Ceph mgr.z for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:07:32.480 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:07:32 smithi174 podman[73465]: 2024-09-19 07:07:32.200291211 +0000 UTC m=+0.134581390 container stop ba463a50c210b5a3eb5502ffae3ad6c9112b6b9af7ce4918f0e75c6dc198168b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T07:07:32.480 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:07:32 smithi174 podman[73465]: 2024-09-19 07:07:32.226058002 +0000 UTC m=+0.160348182 container died ba463a50c210b5a3eb5502ffae3ad6c9112b6b9af7ce4918f0e75c6dc198168b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:07:32.747 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:07:32 smithi174 podman[73465]: 2024-09-19 07:07:32.56235598 +0000 UTC m=+0.496646157 container cleanup ba463a50c210b5a3eb5502ffae3ad6c9112b6b9af7ce4918f0e75c6dc198168b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:07:32.747 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:07:32 smithi174 bash[73465]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z 2024-09-19T07:07:32.983 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T07:07:33.005 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:07:32 smithi174 podman[73477]: 2024-09-19 07:07:32.747018885 +0000 UTC m=+0.540870598 container remove ba463a50c210b5a3eb5502ffae3ad6c9112b6b9af7ce4918f0e75c6dc198168b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-19T07:07:33.005 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:07:32 smithi174 systemd[1]: ceph-d158d934-7651-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-19T07:07:33.005 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:07:32 smithi174 systemd[1]: ceph-d158d934-7651-11ef-bceb-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-19T07:07:33.005 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:07:32 smithi174 systemd[1]: Stopped Ceph mgr.z for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T07:07:33.005 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:07:32 smithi174 systemd[1]: ceph-d158d934-7651-11ef-bceb-c7b262605968@mgr.z.service: Consumed 11.892s CPU time. 2024-09-19T07:07:33.018 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:07:33.018 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T07:07:33.018 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T07:07:33.018 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.0 2024-09-19T07:07:33.311 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:07:33 smithi103 systemd[1]: Stopping Ceph osd.0 for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:07:33.572 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:07:33 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-0[44957]: 2024-09-19T07:07:33.310+0000 7f702397d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:07:33.572 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:07:33 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-0[44957]: 2024-09-19T07:07:33.310+0000 7f702397d640 -1 osd.0 107 *** Got signal Terminated *** 2024-09-19T07:07:33.572 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:07:33 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-0[44957]: 2024-09-19T07:07:33.310+0000 7f702397d640 -1 osd.0 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:07:38.989 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:07:38 smithi103 podman[137016]: 2024-09-19 07:07:38.714488413 +0000 UTC m=+5.504032106 container died 19100663f0409637a005f5bbc87ce088863ff2451bd802167c28c44b578bb453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T07:07:39.387 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:07:39 smithi103 podman[137016]: 2024-09-19 07:07:39.186830479 +0000 UTC m=+5.976374171 container cleanup 19100663f0409637a005f5bbc87ce088863ff2451bd802167c28c44b578bb453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-0, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T07:07:39.388 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:07:39 smithi103 bash[137016]: ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-0 2024-09-19T07:07:39.798 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:07:39 smithi103 podman[137043]: 2024-09-19 07:07:39.463465876 +0000 UTC m=+0.744054477 container remove 19100663f0409637a005f5bbc87ce088863ff2451bd802167c28c44b578bb453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:07:40.134 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:07:39 smithi103 podman[137134]: 2024-09-19 07:07:39.797672262 +0000 UTC m=+0.106460514 container create ed21bb3afd29bf80bf01783e9afdf8bc5a13f04c10005dbe7b0e24728cfffb89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T07:07:40.134 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:07:39 smithi103 podman[137134]: 2024-09-19 07:07:39.707161672 +0000 UTC m=+0.015949886 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:07:40.134 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:07:40 smithi103 podman[137134]: 2024-09-19 07:07:40.041920756 +0000 UTC m=+0.350708968 container init ed21bb3afd29bf80bf01783e9afdf8bc5a13f04c10005dbe7b0e24728cfffb89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:07:40.134 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:07:40 smithi103 podman[137134]: 2024-09-19 07:07:40.045294798 +0000 UTC m=+0.354083011 container start ed21bb3afd29bf80bf01783e9afdf8bc5a13f04c10005dbe7b0e24728cfffb89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T07:07:40.134 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:07:40 smithi103 podman[137134]: 2024-09-19 07:07:40.133554438 +0000 UTC m=+0.442342647 container attach ed21bb3afd29bf80bf01783e9afdf8bc5a13f04c10005dbe7b0e24728cfffb89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-09-19T07:07:40.387 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:07:40 smithi103 podman[137134]: 2024-09-19 07:07:40.2292818 +0000 UTC m=+0.538070004 container died ed21bb3afd29bf80bf01783e9afdf8bc5a13f04c10005dbe7b0e24728cfffb89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T07:07:40.826 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:07:40 smithi103 podman[137158]: 2024-09-19 07:07:40.812571774 +0000 UTC m=+0.577455988 container remove ed21bb3afd29bf80bf01783e9afdf8bc5a13f04c10005dbe7b0e24728cfffb89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-19T07:07:40.831 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T07:07:40.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:07:40.870 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T07:07:40.871 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T07:07:40.871 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.1 2024-09-19T07:07:41.187 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:07:40 smithi134 systemd[1]: Stopping Ceph osd.1 for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:07:41.687 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:07:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-1[39741]: 2024-09-19T07:07:41.202+0000 7f99336ed640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:07:41.687 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:07:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-1[39741]: 2024-09-19T07:07:41.202+0000 7f99336ed640 -1 osd.1 107 *** Got signal Terminated *** 2024-09-19T07:07:41.687 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:07:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-1[39741]: 2024-09-19T07:07:41.202+0000 7f99336ed640 -1 osd.1 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:07:46.936 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:07:46 smithi134 podman[72198]: 2024-09-19 07:07:46.661374431 +0000 UTC m=+5.602610480 container died 719f1614387f73f9950cd109439806cbfeb9604078b3d7042ac19cb16ce29bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-1, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T07:07:47.187 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:07:47 smithi134 podman[72198]: 2024-09-19 07:07:47.035078404 +0000 UTC m=+5.976314452 container cleanup 719f1614387f73f9950cd109439806cbfeb9604078b3d7042ac19cb16ce29bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-19T07:07:47.187 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:07:47 smithi134 bash[72198]: ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-1 2024-09-19T07:07:47.687 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:07:47 smithi134 podman[72213]: 2024-09-19 07:07:47.345133715 +0000 UTC m=+0.679201154 container remove 719f1614387f73f9950cd109439806cbfeb9604078b3d7042ac19cb16ce29bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T07:07:47.982 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:07:47 smithi134 podman[72298]: 2024-09-19 07:07:47.603509316 +0000 UTC m=+0.015913020 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:07:47.982 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:07:47 smithi134 podman[72298]: 2024-09-19 07:07:47.737491627 +0000 UTC m=+0.149895329 container create 012f500a25a23f57baf584e725bfcf86cac50205ae335105d23cb75f2eb02977 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:07:47.982 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:07:47 smithi134 podman[72298]: 2024-09-19 07:07:47.981839863 +0000 UTC m=+0.394243565 container init 012f500a25a23f57baf584e725bfcf86cac50205ae335105d23cb75f2eb02977 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:07:48.437 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:07:47 smithi134 podman[72298]: 2024-09-19 07:07:47.985323263 +0000 UTC m=+0.397726966 container start 012f500a25a23f57baf584e725bfcf86cac50205ae335105d23cb75f2eb02977 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T07:07:48.437 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:07:48 smithi134 podman[72298]: 2024-09-19 07:07:48.081716025 +0000 UTC m=+0.494119731 container attach 012f500a25a23f57baf584e725bfcf86cac50205ae335105d23cb75f2eb02977 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T07:07:48.437 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:07:48 smithi134 podman[72298]: 2024-09-19 07:07:48.173261468 +0000 UTC m=+0.585665171 container died 012f500a25a23f57baf584e725bfcf86cac50205ae335105d23cb75f2eb02977 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T07:07:48.800 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T07:07:48.839 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:07:48.839 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T07:07:48.840 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T07:07:48.840 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.2 2024-09-19T07:07:49.332 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:07:48 smithi174 systemd[1]: Stopping Ceph osd.2 for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:07:49.832 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:07:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-2[40637]: 2024-09-19T07:07:49.366+0000 7fc7c4c21640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:07:49.833 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:07:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-2[40637]: 2024-09-19T07:07:49.366+0000 7fc7c4c21640 -1 osd.2 107 *** Got signal Terminated *** 2024-09-19T07:07:49.833 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:07:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-2[40637]: 2024-09-19T07:07:49.366+0000 7fc7c4c21640 -1 osd.2 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:07:54.999 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:07:54 smithi174 podman[73594]: 2024-09-19 07:07:54.689986449 +0000 UTC m=+5.664098612 container died 68b1ae96be173da1f4eaf65c2cf70d31fe234a724c1b46a62e917925683ba13f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T07:07:55.332 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:07:54 smithi174 podman[73594]: 2024-09-19 07:07:54.998705539 +0000 UTC m=+5.972817694 container cleanup 68b1ae96be173da1f4eaf65c2cf70d31fe234a724c1b46a62e917925683ba13f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T07:07:55.333 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:07:54 smithi174 bash[73594]: ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-2 2024-09-19T07:07:55.333 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:07:55 smithi174 podman[73609]: 2024-09-19 07:07:55.242192541 +0000 UTC m=+0.548166785 container remove 68b1ae96be173da1f4eaf65c2cf70d31fe234a724c1b46a62e917925683ba13f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T07:07:55.832 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:07:55 smithi174 podman[73699]: 2024-09-19 07:07:55.454361497 +0000 UTC m=+0.015956281 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:07:55.833 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:07:55 smithi174 podman[73699]: 2024-09-19 07:07:55.559451273 +0000 UTC m=+0.121046084 container create 7f6d07d4c6390d171a970779bfed1360f7421ca081a008e18df5eba057044f71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-19T07:07:55.833 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:07:55 smithi174 podman[73699]: 2024-09-19 07:07:55.761429437 +0000 UTC m=+0.323024207 container init 7f6d07d4c6390d171a970779bfed1360f7421ca081a008e18df5eba057044f71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:07:55.833 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:07:55 smithi174 podman[73699]: 2024-09-19 07:07:55.764862024 +0000 UTC m=+0.326456797 container start 7f6d07d4c6390d171a970779bfed1360f7421ca081a008e18df5eba057044f71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T07:07:56.298 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:07:55 smithi174 podman[73699]: 2024-09-19 07:07:55.836308513 +0000 UTC m=+0.397903310 container attach 7f6d07d4c6390d171a970779bfed1360f7421ca081a008e18df5eba057044f71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T07:07:56.298 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:07:55 smithi174 podman[73699]: 2024-09-19 07:07:55.964063914 +0000 UTC m=+0.525658680 container died 7f6d07d4c6390d171a970779bfed1360f7421ca081a008e18df5eba057044f71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T07:07:56.550 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:07:56 smithi174 podman[73722]: 2024-09-19 07:07:56.548078902 +0000 UTC m=+0.577617097 container remove 7f6d07d4c6390d171a970779bfed1360f7421ca081a008e18df5eba057044f71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-19T07:07:56.580 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T07:07:56.620 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:07:56.620 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T07:07:56.620 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T07:07:56.620 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.3 2024-09-19T07:07:56.941 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:07:56 smithi103 systemd[1]: Stopping Ceph osd.3 for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:07:57.387 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:07:56 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3[55210]: 2024-09-19T07:07:56.940+0000 7f7145ba1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:07:57.387 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:07:56 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3[55210]: 2024-09-19T07:07:56.940+0000 7f7145ba1640 -1 osd.3 107 *** Got signal Terminated *** 2024-09-19T07:07:57.387 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:07:56 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3[55210]: 2024-09-19T07:07:56.940+0000 7f7145ba1640 -1 osd.3 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:08:00.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:00 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:00.300+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:01.582 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:01 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:01.307+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:01.637 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:01 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:01.265+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:02.566 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:02 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:02.263+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:02.633 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:02 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:02.293+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:02.633 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:08:02 smithi103 podman[137237]: 2024-09-19 07:08:02.34114661 +0000 UTC m=+5.522521685 container died cb1e7247b99f16d7b93f7f85bd7d1ae52b8a2d81af0a8e40d2f9341db090ff92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:08:02.832 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:02 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:02.565+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:35.781675+0000 front 2024-09-19T07:07:35.781764+0000 (oldest deadline 2024-09-19T07:08:01.681658+0000) 2024-09-19T07:08:02.887 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:08:02 smithi103 podman[137237]: 2024-09-19 07:08:02.81548085 +0000 UTC m=+5.996855913 container cleanup cb1e7247b99f16d7b93f7f85bd7d1ae52b8a2d81af0a8e40d2f9341db090ff92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T07:08:02.887 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:08:02 smithi103 bash[137237]: ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3 2024-09-19T07:08:03.387 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:08:03 smithi103 podman[137251]: 2024-09-19 07:08:03.092399193 +0000 UTC m=+0.740558036 container remove cb1e7247b99f16d7b93f7f85bd7d1ae52b8a2d81af0a8e40d2f9341db090ff92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T07:08:03.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:03 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:03.300+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:03.389 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:03 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:03.113+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:03.582 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:03 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:03.285+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:03.679 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:08:03 smithi103 podman[137341]: 2024-09-19 07:08:03.342525986 +0000 UTC m=+0.016230814 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:08:03.679 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:08:03 smithi103 podman[137341]: 2024-09-19 07:08:03.443437143 +0000 UTC m=+0.117141956 container create 3b70b41adbce75374f2e33a3d2044bc8f24b9f10b621cae55e98b6fb4b67b7b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:08:03.679 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:08:03 smithi103 podman[137341]: 2024-09-19 07:08:03.679261213 +0000 UTC m=+0.352966038 container init 3b70b41adbce75374f2e33a3d2044bc8f24b9f10b621cae55e98b6fb4b67b7b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T07:08:04.082 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:03 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:03.604+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:35.781675+0000 front 2024-09-19T07:07:35.781764+0000 (oldest deadline 2024-09-19T07:08:01.681658+0000) 2024-09-19T07:08:04.136 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:08:03 smithi103 podman[137341]: 2024-09-19 07:08:03.682731182 +0000 UTC m=+0.356436006 container start 3b70b41adbce75374f2e33a3d2044bc8f24b9f10b621cae55e98b6fb4b67b7b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:08:04.136 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:08:03 smithi103 podman[137341]: 2024-09-19 07:08:03.787801194 +0000 UTC m=+0.461506021 container attach 3b70b41adbce75374f2e33a3d2044bc8f24b9f10b621cae55e98b6fb4b67b7b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T07:08:04.136 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:08:03 smithi103 podman[137341]: 2024-09-19 07:08:03.876799497 +0000 UTC m=+0.550504316 container died 3b70b41adbce75374f2e33a3d2044bc8f24b9f10b621cae55e98b6fb4b67b7b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-19T07:08:04.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:04 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:04.270+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:04.388 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:04 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:04.135+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:04.438 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:04 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:04.138+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:04.579 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:04 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:04.317+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:04.631 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T07:08:04.673 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:08:04.674 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T07:08:04.674 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T07:08:04.674 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.4 2024-09-19T07:08:04.832 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:04 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:04.579+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:35.781675+0000 front 2024-09-19T07:07:35.781764+0000 (oldest deadline 2024-09-19T07:08:01.681658+0000) 2024-09-19T07:08:05.014 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:08:04 smithi134 systemd[1]: Stopping Ceph osd.4 for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:08:05.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:05 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:05.176+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:05.437 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:08:05 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4[46370]: 2024-09-19T07:08:05.013+0000 7f852496b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:08:05.437 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:08:05 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4[46370]: 2024-09-19T07:08:05.013+0000 7f852496b640 -1 osd.4 107 *** Got signal Terminated *** 2024-09-19T07:08:05.438 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:08:05 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4[46370]: 2024-09-19T07:08:05.013+0000 7f852496b640 -1 osd.4 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:08:05.555 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:05 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:05.288+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:05.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:05 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:05.152+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:05.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:05 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:05.244+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:05.832 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:05 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:05.554+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:35.781675+0000 front 2024-09-19T07:07:35.781764+0000 (oldest deadline 2024-09-19T07:08:01.681658+0000) 2024-09-19T07:08:06.387 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:06 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:06.127+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:06.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:06 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:06.236+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:06.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:06 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:06.169+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:06.581 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:06 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:06.295+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:06.832 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:06 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:06.578+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:35.781675+0000 front 2024-09-19T07:07:35.781764+0000 (oldest deadline 2024-09-19T07:08:01.681658+0000) 2024-09-19T07:08:07.300 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:06 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:06.887+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:07.582 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:07 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:07.312+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:07.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:07 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:07.162+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:07.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:07 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:07.218+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:07.687 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:07 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:07.201+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:07.908 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:07 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:07.606+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:35.781675+0000 front 2024-09-19T07:07:35.781764+0000 (oldest deadline 2024-09-19T07:08:01.681658+0000) 2024-09-19T07:08:07.908 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:07 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:07.606+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:43.883020+0000 front 2024-09-19T07:07:43.883106+0000 (oldest deadline 2024-09-19T07:08:07.383026+0000) 2024-09-19T07:08:08.274 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:07 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:07.907+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:08.578 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:08 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:08.245+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:08.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:08 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:08.273+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:08.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:08 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:08.273+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:08.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:08 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:08.210+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:08.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:08 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:08.210+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:08.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:08 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:08.189+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:08.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:08 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:08.189+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:08.914 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:08 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:08.655+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:35.781675+0000 front 2024-09-19T07:07:35.781764+0000 (oldest deadline 2024-09-19T07:08:01.681658+0000) 2024-09-19T07:08:08.914 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:08 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:08.655+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:43.883020+0000 front 2024-09-19T07:07:43.883106+0000 (oldest deadline 2024-09-19T07:08:07.383026+0000) 2024-09-19T07:08:08.930 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:08 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:08.576+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:09.187 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:08:08 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4[46370]: 2024-09-19T07:08:08.928+0000 7f8520f70640 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.459040+0000 front 2024-09-19T07:07:42.459075+0000 (oldest deadline 2024-09-19T07:08:08.359245+0000) 2024-09-19T07:08:09.261 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:08 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:08.913+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:09.582 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:09 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:09.260+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:09.582 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:09 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:09.260+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:09.626 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:09 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:09.223+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:09.626 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:09 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:09.223+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:09.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:09 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:09.244+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:09.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:09 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:09.244+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:09.637 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:09 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:09.180+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:09.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:09 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:09.180+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:09.928 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:09 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:09.624+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:10.082 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:09 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:09.921+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:10.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:09 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:09.692+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:35.781675+0000 front 2024-09-19T07:07:35.781764+0000 (oldest deadline 2024-09-19T07:08:01.681658+0000) 2024-09-19T07:08:10.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:09 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:09.692+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:43.883020+0000 front 2024-09-19T07:07:43.883106+0000 (oldest deadline 2024-09-19T07:08:07.383026+0000) 2024-09-19T07:08:10.187 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:08:09 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4[46370]: 2024-09-19T07:08:09.926+0000 7f8520f70640 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.459040+0000 front 2024-09-19T07:07:42.459075+0000 (oldest deadline 2024-09-19T07:08:08.359245+0000) 2024-09-19T07:08:10.582 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:10 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:10.277+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:10.582 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:10 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:10.277+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:10.594 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:08:10 smithi134 podman[72413]: 2024-09-19 07:08:10.420098895 +0000 UTC m=+5.549115276 container died 8b4812b261e4d4068b1d0deff739a8fd77f7b90f323954756ed161e2f89b3e8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:08:10.594 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:10 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:10.254+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:10.594 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:10 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:10.254+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:10.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:10 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:10.268+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:10.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:10 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:10.268+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:10.637 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:10 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:10.197+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:10.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:10 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:10.197+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:10.937 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:10 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:10.593+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:10.937 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:08:10 smithi134 podman[72413]: 2024-09-19 07:08:10.796202952 +0000 UTC m=+5.925219398 container cleanup 8b4812b261e4d4068b1d0deff739a8fd77f7b90f323954756ed161e2f89b3e8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T07:08:10.937 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:08:10 smithi134 bash[72413]: ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4 2024-09-19T07:08:11.082 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:10 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:10.906+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:11.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:10 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:10.705+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:35.781675+0000 front 2024-09-19T07:07:35.781764+0000 (oldest deadline 2024-09-19T07:08:01.681658+0000) 2024-09-19T07:08:11.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:10 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:10.705+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:43.883020+0000 front 2024-09-19T07:07:43.883106+0000 (oldest deadline 2024-09-19T07:08:07.383026+0000) 2024-09-19T07:08:11.366 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:08:11 smithi134 podman[72429]: 2024-09-19 07:08:11.055754003 +0000 UTC m=+0.635578594 container remove 8b4812b261e4d4068b1d0deff739a8fd77f7b90f323954756ed161e2f89b3e8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-19T07:08:11.367 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:11 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:11.267+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:11.367 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:11 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:11.267+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:11.582 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:11 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:11.245+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:11.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:11 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:11.245+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:11.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:11 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:11.227+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:11.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:11 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:11.227+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:11.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:11 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:11.196+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:11.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:11 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:11.196+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:11.650 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:11 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:11.568+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:11.650 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:08:11 smithi134 podman[72520]: 2024-09-19 07:08:11.269166378 +0000 UTC m=+0.015801331 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:08:11.650 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:08:11 smithi134 podman[72520]: 2024-09-19 07:08:11.373340794 +0000 UTC m=+0.119975806 container create 5a5094377728da3c980cf574b50cc170f0163aa9b68544b4434908467e0a4c05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-19T07:08:11.650 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:08:11 smithi134 podman[72520]: 2024-09-19 07:08:11.56721347 +0000 UTC m=+0.313848437 container init 5a5094377728da3c980cf574b50cc170f0163aa9b68544b4434908467e0a4c05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T07:08:11.650 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:08:11 smithi134 podman[72520]: 2024-09-19 07:08:11.571316798 +0000 UTC m=+0.317951754 container start 5a5094377728da3c980cf574b50cc170f0163aa9b68544b4434908467e0a4c05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-19T07:08:11.650 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:08:11 smithi134 podman[72520]: 2024-09-19 07:08:11.650297139 +0000 UTC m=+0.396932093 container attach 5a5094377728da3c980cf574b50cc170f0163aa9b68544b4434908467e0a4c05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True) 2024-09-19T07:08:11.937 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:08:11 smithi134 podman[72520]: 2024-09-19 07:08:11.757367841 +0000 UTC m=+0.504002795 container died 5a5094377728da3c980cf574b50cc170f0163aa9b68544b4434908467e0a4c05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T07:08:12.082 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:11 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:11.880+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:12.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:11 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:11.673+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:35.781675+0000 front 2024-09-19T07:07:35.781764+0000 (oldest deadline 2024-09-19T07:08:01.681658+0000) 2024-09-19T07:08:12.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:11 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:11.673+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:43.883020+0000 front 2024-09-19T07:07:43.883106+0000 (oldest deadline 2024-09-19T07:08:07.383026+0000) 2024-09-19T07:08:12.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:12 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:12.250+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:12.371 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:12 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:12.250+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:12.387 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T07:08:12.425 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:08:12.426 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T07:08:12.426 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T07:08:12.426 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.5 2024-09-19T07:08:12.582 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:12 smithi174 systemd[1]: Stopping Ceph osd.5 for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:08:12.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:12.253+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:12.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:12.253+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:12.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:12.235+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:12.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:12.235+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:12.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:12.245+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:12.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:12 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:12.245+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:12.687 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:12 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:12.571+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:13.082 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:12.884+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:13.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:12.689+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:35.781675+0000 front 2024-09-19T07:07:35.781764+0000 (oldest deadline 2024-09-19T07:08:01.681658+0000) 2024-09-19T07:08:13.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:12.689+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:43.883020+0000 front 2024-09-19T07:07:43.883106+0000 (oldest deadline 2024-09-19T07:08:07.383026+0000) 2024-09-19T07:08:13.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:12.717+0000 7fb8259b7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:08:13.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:12.717+0000 7fb8259b7640 -1 osd.5 107 *** Got signal Terminated *** 2024-09-19T07:08:13.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:12 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:12.717+0000 7fb8259b7640 -1 osd.5 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:08:13.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:13 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:13.293+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:13.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:13 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:13.293+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:13.606 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:13 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:13.209+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:13.606 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:13 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:13.209+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:13.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:13 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:13.202+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:13.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:13 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:13.202+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:13.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:13 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:13.217+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:13.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:13 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:13.217+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:13.937 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:13 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:13.604+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:14.082 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:13 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:13.855+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:14.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:13 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:13.685+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:35.781675+0000 front 2024-09-19T07:07:35.781764+0000 (oldest deadline 2024-09-19T07:08:01.681658+0000) 2024-09-19T07:08:14.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:13 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:13.685+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:43.883020+0000 front 2024-09-19T07:07:43.883106+0000 (oldest deadline 2024-09-19T07:08:07.383026+0000) 2024-09-19T07:08:14.585 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:14 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:14.187+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:14.585 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:14 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:14.187+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:14.585 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:14 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:14.307+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:14.585 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:14 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:14.307+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:14.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:14.234+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:14.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:14.234+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:14.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:14.230+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:14.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:14.230+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:14.937 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:14 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:14.583+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:15.082 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:14 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:14.666+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:35.781675+0000 front 2024-09-19T07:07:35.781764+0000 (oldest deadline 2024-09-19T07:08:01.681658+0000) 2024-09-19T07:08:15.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:14 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:14.666+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:43.883020+0000 front 2024-09-19T07:07:43.883106+0000 (oldest deadline 2024-09-19T07:08:07.383026+0000) 2024-09-19T07:08:15.083 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:14 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:14.904+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:15.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:15 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:15.164+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:15.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:15 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:15.164+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:15.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:15 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:15.261+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:15.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:15 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:15.261+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:15.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:15 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:15.261+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:15.637 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:15.210+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:15.637 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:15.210+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:15.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:15.230+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:15.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:15 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:15.230+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:15.937 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:15 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:15.600+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:16.082 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:15 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:15.874+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:16.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:15 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:15.701+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:35.781675+0000 front 2024-09-19T07:07:35.781764+0000 (oldest deadline 2024-09-19T07:08:01.681658+0000) 2024-09-19T07:08:16.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:15 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:15.701+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:43.883020+0000 front 2024-09-19T07:07:43.883106+0000 (oldest deadline 2024-09-19T07:08:07.383026+0000) 2024-09-19T07:08:16.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:16 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:16.166+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:16.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:16 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:16.166+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:16.582 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:16 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:16.234+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:16.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:16 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:16.234+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:16.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:16 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:16.234+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:16.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:16 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:16.217+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:16.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:16 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:16.217+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:16.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:16 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:16.217+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.330556+0000 front 2024-09-19T07:07:54.330463+0000 (oldest deadline 2024-09-19T07:08:15.430404+0000) 2024-09-19T07:08:16.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:16 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:16.231+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:16.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:16 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:16.231+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:16.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:16 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:16.231+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:16.937 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:16 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:16.594+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:17.082 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:16 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:16.852+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:17.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:16 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:16.699+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:35.781675+0000 front 2024-09-19T07:07:35.781764+0000 (oldest deadline 2024-09-19T07:08:01.681658+0000) 2024-09-19T07:08:17.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:16 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:16.699+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:43.883020+0000 front 2024-09-19T07:07:43.883106+0000 (oldest deadline 2024-09-19T07:08:07.383026+0000) 2024-09-19T07:08:17.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:16 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:16.699+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.984315+0000 front 2024-09-19T07:07:51.984339+0000 (oldest deadline 2024-09-19T07:08:16.084461+0000) 2024-09-19T07:08:17.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:17.143+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:17.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:17.143+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:17.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:17.143+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:17.582 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:17.260+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:17.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:17.260+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:17.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:17.260+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:17.637 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:17 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:17.204+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:17.637 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:17 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:17.204+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:17.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:17 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:17.204+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:17.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:17 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:17.227+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:17.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:17 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:17.227+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:17.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:17 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:17.227+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.330556+0000 front 2024-09-19T07:07:54.330463+0000 (oldest deadline 2024-09-19T07:08:15.430404+0000) 2024-09-19T07:08:17.700 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:17.566+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:17.700 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:17 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:17.566+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:53.209289+0000 front 2024-09-19T07:07:53.209258+0000 (oldest deadline 2024-09-19T07:08:16.709355+0000) 2024-09-19T07:08:18.082 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:17.808+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:18.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:17.673+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:35.781675+0000 front 2024-09-19T07:07:35.781764+0000 (oldest deadline 2024-09-19T07:08:01.681658+0000) 2024-09-19T07:08:18.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:17.673+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:43.883020+0000 front 2024-09-19T07:07:43.883106+0000 (oldest deadline 2024-09-19T07:08:07.383026+0000) 2024-09-19T07:08:18.083 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:17 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5[47559]: 2024-09-19T07:08:17.673+0000 7fb821fbc640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.984315+0000 front 2024-09-19T07:07:51.984339+0000 (oldest deadline 2024-09-19T07:08:16.084461+0000) 2024-09-19T07:08:18.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:18 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:18.176+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:18.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:18 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:18.176+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:18.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:18 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:18.176+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:18.574 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:18 smithi174 podman[73815]: 2024-09-19 07:08:18.255890527 +0000 UTC m=+5.629472179 container died 92483c47a45110a9b4fcc3c3ac8ca65d7075164413fcc59e6bff4dd6fadc9e4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T07:08:18.575 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:18 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:18.256+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:18.575 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:18 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:18.256+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:18.575 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:18 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:18.256+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:18.637 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:18.226+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:18.637 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:18.226+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:18.637 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:18.226+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:18.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:18.185+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:18.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:18.185+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:18.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:18 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:18.185+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.330556+0000 front 2024-09-19T07:07:54.330463+0000 (oldest deadline 2024-09-19T07:08:15.430404+0000) 2024-09-19T07:08:18.832 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:18 smithi174 podman[73815]: 2024-09-19 07:08:18.574397391 +0000 UTC m=+5.947979053 container cleanup 92483c47a45110a9b4fcc3c3ac8ca65d7075164413fcc59e6bff4dd6fadc9e4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:08:18.833 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:18 smithi174 bash[73815]: ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5 2024-09-19T07:08:18.937 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:18 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:18.600+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:18.937 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:18 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:18.600+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:53.209289+0000 front 2024-09-19T07:07:53.209258+0000 (oldest deadline 2024-09-19T07:08:16.709355+0000) 2024-09-19T07:08:19.224 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:18 smithi174 podman[73831]: 2024-09-19 07:08:18.875858585 +0000 UTC m=+0.616482604 container remove 92483c47a45110a9b4fcc3c3ac8ca65d7075164413fcc59e6bff4dd6fadc9e4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T07:08:19.225 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:18 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:18.847+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:19.225 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:18 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:18.847+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:19.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:19 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:19.149+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:19.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:19 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:19.149+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:19.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:19 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:19.149+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:19.579 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:19 smithi174 podman[73918]: 2024-09-19 07:08:19.208242257 +0000 UTC m=+0.015770899 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:08:19.579 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:19 smithi174 podman[73918]: 2024-09-19 07:08:19.334902989 +0000 UTC m=+0.142431660 container create bc23112b4ad370a68380352be9f31f6e535aefbdedcad4d82364c7a631943104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-09-19T07:08:19.579 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:19 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:19.223+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:19.579 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:19 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:19.223+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:19.579 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:19 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:19.223+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:19.637 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:19 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:19.225+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:19.637 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:19 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:19.225+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:19.637 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:19 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:19.225+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:19.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:19 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:19.181+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:19.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:19 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:19.181+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:19.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:19 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:19.181+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.330556+0000 front 2024-09-19T07:07:54.330463+0000 (oldest deadline 2024-09-19T07:08:15.430404+0000) 2024-09-19T07:08:19.833 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:19 smithi174 podman[73918]: 2024-09-19 07:08:19.578957617 +0000 UTC m=+0.386486247 container init bc23112b4ad370a68380352be9f31f6e535aefbdedcad4d82364c7a631943104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-19T07:08:19.833 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:19 smithi174 podman[73918]: 2024-09-19 07:08:19.582513072 +0000 UTC m=+0.390041702 container start bc23112b4ad370a68380352be9f31f6e535aefbdedcad4d82364c7a631943104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T07:08:19.833 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:19 smithi174 podman[73918]: 2024-09-19 07:08:19.65392649 +0000 UTC m=+0.461455146 container attach bc23112b4ad370a68380352be9f31f6e535aefbdedcad4d82364c7a631943104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:08:19.833 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:08:19 smithi174 podman[73918]: 2024-09-19 07:08:19.770479412 +0000 UTC m=+0.578008052 container died bc23112b4ad370a68380352be9f31f6e535aefbdedcad4d82364c7a631943104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T07:08:19.834 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:19 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:19.809+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:19.834 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:19 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:19.809+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:19.937 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:19 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:19.644+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:19.937 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:19 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:19.644+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:53.209289+0000 front 2024-09-19T07:07:53.209258+0000 (oldest deadline 2024-09-19T07:08:16.709355+0000) 2024-09-19T07:08:20.357 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:20 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:20.195+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:20.357 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:20 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:20.195+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:20.357 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:20 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:20.195+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:20.382 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T07:08:20.420 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:08:20.421 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T07:08:20.421 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T07:08:20.421 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.6 2024-09-19T07:08:20.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:20 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:20.173+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:20.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:20 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:20.173+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:20.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:20 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:20.173+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:20.475 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:20 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:20.217+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:20.475 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:20 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:20.217+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:20.475 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:20 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:20.217+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.330556+0000 front 2024-09-19T07:07:54.330463+0000 (oldest deadline 2024-09-19T07:08:15.430404+0000) 2024-09-19T07:08:20.475 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:20 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:20.212+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:20.475 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:20 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:20.212+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:20.476 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:20 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:20.212+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:20.887 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:20 smithi103 systemd[1]: Stopping Ceph osd.6 for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:08:20.887 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:20 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:20.712+0000 7ff8b695d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:08:20.887 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:20 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:20.712+0000 7ff8b695d640 -1 osd.6 107 *** Got signal Terminated *** 2024-09-19T07:08:20.888 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:20 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:20.712+0000 7ff8b695d640 -1 osd.6 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:08:20.937 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:20 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:20.671+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:20.937 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:20 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:20.671+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:53.209289+0000 front 2024-09-19T07:07:53.209258+0000 (oldest deadline 2024-09-19T07:08:16.709355+0000) 2024-09-19T07:08:21.151 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:20 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:20.835+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:21.151 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:20 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:20.835+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:21.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:21 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:21.164+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:21.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:21 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:21.164+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:21.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:21 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:21.164+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:21.582 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:21 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:21.151+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:21.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:21 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:21.151+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:21.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:21 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:21.151+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:21.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:21.216+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:21.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:21.216+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:21.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:21.216+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.330556+0000 front 2024-09-19T07:07:54.330463+0000 (oldest deadline 2024-09-19T07:08:15.430404+0000) 2024-09-19T07:08:21.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:21.194+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:21.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:21.194+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:21.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:21 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:21.194+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:22.110 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:21 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:21.835+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:22.110 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:21 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:21.835+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:22.135 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:21 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:21.698+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:22.135 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:21 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:21.698+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:53.209289+0000 front 2024-09-19T07:07:53.209258+0000 (oldest deadline 2024-09-19T07:08:16.709355+0000) 2024-09-19T07:08:22.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:22 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:22.134+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:22.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:22 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:22.134+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:22.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:22 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:22.134+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:22.582 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:22 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:22.110+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:22.582 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:22 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:22.110+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:22.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:22 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:22.110+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:22.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:22 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:22.208+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:22.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:22 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:22.208+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:22.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:22 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:22.208+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.330556+0000 front 2024-09-19T07:07:54.330463+0000 (oldest deadline 2024-09-19T07:08:15.430404+0000) 2024-09-19T07:08:22.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:22 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:22.158+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:22.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:22 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:22.158+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:22.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:22 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:22.158+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:23.112 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:22 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:22.735+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:23.112 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:22 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:22.735+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:53.209289+0000 front 2024-09-19T07:07:53.209258+0000 (oldest deadline 2024-09-19T07:08:16.709355+0000) 2024-09-19T07:08:23.332 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:22 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:22.873+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:23.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:22 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:22.873+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:23.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:23 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:23.073+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:23.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:23 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:23.073+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:23.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:23 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:23.073+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:23.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:23 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:23.111+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:23.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:23 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:23.111+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:23.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:23 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:23.111+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:23.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:23 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:23.177+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:23.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:23 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:23.177+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:23.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:23 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:23.177+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.330556+0000 front 2024-09-19T07:07:54.330463+0000 (oldest deadline 2024-09-19T07:08:15.430404+0000) 2024-09-19T07:08:23.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:23 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:23.165+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:23.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:23 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:23.165+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:23.638 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:23 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:23.165+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:24.082 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:23 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:23.825+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:24.083 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:23 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:23.825+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:24.095 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:23 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:23.711+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:24.096 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:23 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:23.711+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:53.209289+0000 front 2024-09-19T07:07:53.209258+0000 (oldest deadline 2024-09-19T07:08:16.709355+0000) 2024-09-19T07:08:24.096 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:23 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:23.711+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:58.410510+0000 front 2024-09-19T07:07:58.410622+0000 (oldest deadline 2024-09-19T07:08:23.110533+0000) 2024-09-19T07:08:24.358 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:24 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:24.082+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:24.358 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:24 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:24.082+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:24.358 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:24 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:24.082+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:24.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:24 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:24.122+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:24.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:24 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:24.122+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:24.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:24 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:24.122+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:24.388 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:24 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:24.206+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:24.388 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:24 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:24.206+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:24.388 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:24 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:24.206+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.330556+0000 front 2024-09-19T07:07:54.330463+0000 (oldest deadline 2024-09-19T07:08:15.430404+0000) 2024-09-19T07:08:24.388 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:24 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:24.206+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:01.732295+0000 front 2024-09-19T07:08:01.732338+0000 (oldest deadline 2024-09-19T07:08:24.032401+0000) 2024-09-19T07:08:24.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:24 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:24.095+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:24.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:24 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:24.095+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:24.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:24 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:24.095+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:24.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:24 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:24.095+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:25.073 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:24 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:24.735+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:25.074 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:24 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:24.735+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:53.209289+0000 front 2024-09-19T07:07:53.209258+0000 (oldest deadline 2024-09-19T07:08:16.709355+0000) 2024-09-19T07:08:25.074 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:24 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:24.735+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:58.410510+0000 front 2024-09-19T07:07:58.410622+0000 (oldest deadline 2024-09-19T07:08:23.110533+0000) 2024-09-19T07:08:25.318 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:24 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:24.858+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:25.319 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:24 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:24.858+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:25.319 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:24 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:24.858+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:25.319 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:25 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:25.093+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:25.319 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:25 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:25.093+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:25.319 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:25 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:25.093+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:25.319 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:25 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:25.093+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:25.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:25 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:25.073+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:25.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:25 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:25.073+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:25.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:25 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:25.073+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:25.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:25 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:25.073+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:25.637 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:25 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:25.157+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:25.637 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:25 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:25.157+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:25.637 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:25 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:25.157+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:25.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:25 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:25.208+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:38.227089+0000 front 2024-09-19T07:07:38.227125+0000 (oldest deadline 2024-09-19T07:08:02.327257+0000) 2024-09-19T07:08:25.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:25 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:25.208+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:42.828403+0000 front 2024-09-19T07:07:42.828394+0000 (oldest deadline 2024-09-19T07:08:07.528253+0000) 2024-09-19T07:08:25.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:25 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:25.208+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.330556+0000 front 2024-09-19T07:07:54.330463+0000 (oldest deadline 2024-09-19T07:08:15.430404+0000) 2024-09-19T07:08:25.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:25 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6[58826]: 2024-09-19T07:08:25.208+0000 7ff8b2f62640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:01.732295+0000 front 2024-09-19T07:08:01.732338+0000 (oldest deadline 2024-09-19T07:08:24.032401+0000) 2024-09-19T07:08:26.031 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:25 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:25.769+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:26.032 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:25 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:25.769+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:53.209289+0000 front 2024-09-19T07:07:53.209258+0000 (oldest deadline 2024-09-19T07:08:16.709355+0000) 2024-09-19T07:08:26.032 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:25 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:25.769+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:58.410510+0000 front 2024-09-19T07:07:58.410622+0000 (oldest deadline 2024-09-19T07:08:23.110533+0000) 2024-09-19T07:08:26.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:25 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:25.889+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:26.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:25 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:25.889+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:26.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:25 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:25.889+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:26.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:26 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:26.095+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:26.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:26 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:26.095+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:26.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:26 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:26.095+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:26.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:26 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:26.095+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:26.347 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:26 smithi103 podman[137456]: 2024-09-19 07:08:26.085130321 +0000 UTC m=+5.461539549 container died be69a923ec5dbfca8daddd04668b53a7c6744e420b44c79a47992c670fa09a22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T07:08:26.348 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:26 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:26.128+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:26.348 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:26 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:26.128+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:26.348 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:26 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:26.128+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:26.348 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:26 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:26.128+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:26.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:26 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:26.030+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:26.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:26 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:26.031+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:26.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:26 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:26.031+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:26.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:26 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:26.031+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:26.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:26 smithi103 podman[137456]: 2024-09-19 07:08:26.563778834 +0000 UTC m=+5.940188053 container cleanup be69a923ec5dbfca8daddd04668b53a7c6744e420b44c79a47992c670fa09a22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-19T07:08:26.638 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:26 smithi103 bash[137456]: ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6 2024-09-19T07:08:27.061 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:26 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:26.796+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:27.061 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:26 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:26.796+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:53.209289+0000 front 2024-09-19T07:07:53.209258+0000 (oldest deadline 2024-09-19T07:08:16.709355+0000) 2024-09-19T07:08:27.061 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:26 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:26.796+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:58.410510+0000 front 2024-09-19T07:07:58.410622+0000 (oldest deadline 2024-09-19T07:08:23.110533+0000) 2024-09-19T07:08:27.133 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:26 smithi103 podman[137472]: 2024-09-19 07:08:26.823546894 +0000 UTC m=+0.733502741 container remove be69a923ec5dbfca8daddd04668b53a7c6744e420b44c79a47992c670fa09a22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T07:08:27.332 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:26 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:26.933+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:27.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:26 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:26.933+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:27.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:26 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:26.933+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:27.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:27 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:27.075+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:27.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:27 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:27.075+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:27.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:27 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:27.075+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:27.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:27 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:27.075+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:27.385 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:27 smithi103 podman[137564]: 2024-09-19 07:08:27.078615584 +0000 UTC m=+0.016274028 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:08:27.385 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:27 smithi103 podman[137564]: 2024-09-19 07:08:27.182880562 +0000 UTC m=+0.120539005 container create e28e74e292aaca9b4615753af7fbb3df30774cc14d447652457a508854aa01fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T07:08:27.385 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:27 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:27.132+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:27.385 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:27 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:27.132+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:27.385 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:27 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:27.132+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:27.386 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:27 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:27.132+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:27.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:27 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:27.060+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:27.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:27 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:27.060+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:27.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:27 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:27.060+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:27.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:27 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:27.060+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:27.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:27 smithi103 podman[137564]: 2024-09-19 07:08:27.385185119 +0000 UTC m=+0.322843563 container init e28e74e292aaca9b4615753af7fbb3df30774cc14d447652457a508854aa01fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T07:08:27.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:27 smithi103 podman[137564]: 2024-09-19 07:08:27.388570357 +0000 UTC m=+0.326228805 container start e28e74e292aaca9b4615753af7fbb3df30774cc14d447652457a508854aa01fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T07:08:27.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:27 smithi103 podman[137564]: 2024-09-19 07:08:27.476825315 +0000 UTC m=+0.414483769 container attach e28e74e292aaca9b4615753af7fbb3df30774cc14d447652457a508854aa01fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T07:08:27.637 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:08:27 smithi103 podman[137564]: 2024-09-19 07:08:27.588498102 +0000 UTC m=+0.526156549 container died e28e74e292aaca9b4615753af7fbb3df30774cc14d447652457a508854aa01fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T07:08:28.172 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:27 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:27.841+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:28.172 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:27 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:27.841+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:53.209289+0000 front 2024-09-19T07:07:53.209258+0000 (oldest deadline 2024-09-19T07:08:16.709355+0000) 2024-09-19T07:08:28.172 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:27 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:27.841+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:58.410510+0000 front 2024-09-19T07:07:58.410622+0000 (oldest deadline 2024-09-19T07:08:23.110533+0000) 2024-09-19T07:08:28.172 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:28 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:28.061+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:28.172 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:28 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:28.061+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:28.173 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:28 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:28.061+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:28.173 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:28 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:28.061+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:28.228 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T07:08:28.269 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:08:28.269 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T07:08:28.270 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T07:08:28.270 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.7 2024-09-19T07:08:28.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:27 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:27.964+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:28.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:27 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:27.964+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:28.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:27 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:27.964+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:28.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:28 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:28.050+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:28.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:28 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:28.050+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:28.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:28 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:28.050+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:28.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:28 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:28.050+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:28.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:28 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:28.089+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:28.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:28 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:28.089+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:28.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:28 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:28.089+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:28.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:28 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:28.089+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:28.436 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:28 smithi134 systemd[1]: Stopping Ceph osd.7 for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:08:28.893 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:28 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:28.573+0000 7f822d86b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:08:28.893 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:28 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:28.573+0000 7f822d86b640 -1 osd.7 107 *** Got signal Terminated *** 2024-09-19T07:08:28.893 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:28 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:28.573+0000 7f822d86b640 -1 osd.7 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:08:29.187 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:28 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:28.892+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:29.187 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:28 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:28.892+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:53.209289+0000 front 2024-09-19T07:07:53.209258+0000 (oldest deadline 2024-09-19T07:08:16.709355+0000) 2024-09-19T07:08:29.187 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:28 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:28.892+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:58.410510+0000 front 2024-09-19T07:07:58.410622+0000 (oldest deadline 2024-09-19T07:08:23.110533+0000) 2024-09-19T07:08:29.188 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:29 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:29.031+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:29.188 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:29 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:29.031+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:29.188 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:29 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:29.031+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:29.188 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:29 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:29.031+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:29.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:28 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:28.983+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:29.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:28 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:28.983+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:29.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:28 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:28.983+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:29.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:29 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:29.093+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:29.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:29 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:29.093+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:29.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:29 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:29.093+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:29.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:29 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:29.093+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:29.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:29 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:29.085+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:29.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:29 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:29.085+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:29.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:29 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:29.085+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:29.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:29 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:29.085+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:30.187 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:29 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:29.856+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:30.187 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:29 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:29.856+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:53.209289+0000 front 2024-09-19T07:07:53.209258+0000 (oldest deadline 2024-09-19T07:08:16.709355+0000) 2024-09-19T07:08:30.187 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:29 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:29.856+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:58.410510+0000 front 2024-09-19T07:07:58.410622+0000 (oldest deadline 2024-09-19T07:08:23.110533+0000) 2024-09-19T07:08:30.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:30 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:30.078+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:30.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:30 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:30.078+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:30.188 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:30 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:30.078+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:30.188 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:30 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:30.078+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:30.332 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:29 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:29.945+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:30.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:29 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:29.945+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:30.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:29 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:29.945+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:30.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:30 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:30.121+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:30.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:30 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:30.121+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:30.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:30 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:30.121+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:30.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:30 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:30.121+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:30.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:30 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:30.056+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:30.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:30 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:30.056+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:30.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:30 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:30.056+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:30.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:30 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:30.056+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:31.187 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:30 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:30.826+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:31.187 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:30 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:30.826+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:53.209289+0000 front 2024-09-19T07:07:53.209258+0000 (oldest deadline 2024-09-19T07:08:16.709355+0000) 2024-09-19T07:08:31.187 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:30 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:30.826+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:58.410510+0000 front 2024-09-19T07:07:58.410622+0000 (oldest deadline 2024-09-19T07:08:23.110533+0000) 2024-09-19T07:08:31.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:31 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:31.046+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:31.188 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:31 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:31.046+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:31.188 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:31 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:31.046+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:31.188 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:31 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:31.046+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:31.332 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:30 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:30.935+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:31.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:30 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:30.935+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:31.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:30 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:30.935+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:31.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:31 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:31.102+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:31.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:31 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:31.102+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:31.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:31 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:31.102+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:31.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:31 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:31.102+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:31.389 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:31 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:31.046+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:31.389 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:31 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:31.046+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:31.389 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:31 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:31.046+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:31.389 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:31 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:31.046+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:32.187 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:31 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:31.875+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:32.187 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:31 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:31.875+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:53.209289+0000 front 2024-09-19T07:07:53.209258+0000 (oldest deadline 2024-09-19T07:08:16.709355+0000) 2024-09-19T07:08:32.187 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:31 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:31.875+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:58.410510+0000 front 2024-09-19T07:07:58.410622+0000 (oldest deadline 2024-09-19T07:08:23.110533+0000) 2024-09-19T07:08:32.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:32 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:32.046+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:32.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:32 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:32.046+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:32.188 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:32 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:32.046+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:32.188 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:32 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:32.046+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:32.332 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:31 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:31.946+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:32.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:31 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:31.946+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:32.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:31 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:31.946+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:32.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:31 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:31.946+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.185228+0000 front 2024-09-19T07:08:04.185080+0000 (oldest deadline 2024-09-19T07:08:31.785369+0000) 2024-09-19T07:08:32.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:32 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:32.124+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:32.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:32 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:32.124+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:32.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:32 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:32.124+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:32.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:32 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:32.124+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:32.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:32 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:32.057+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:32.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:32 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:32.057+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:32.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:32 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:32.057+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:32.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:32 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:32.057+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:33.187 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:32 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:32.871+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:45.107853+0000 front 2024-09-19T07:07:45.107937+0000 (oldest deadline 2024-09-19T07:08:08.007962+0000) 2024-09-19T07:08:33.187 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:32 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:32.871+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:53.209289+0000 front 2024-09-19T07:07:53.209258+0000 (oldest deadline 2024-09-19T07:08:16.709355+0000) 2024-09-19T07:08:33.187 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:32 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7[53354]: 2024-09-19T07:08:32.871+0000 7f8225e68640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:58.410510+0000 front 2024-09-19T07:07:58.410622+0000 (oldest deadline 2024-09-19T07:08:23.110533+0000) 2024-09-19T07:08:33.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:33 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:33.028+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:33.188 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:33 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:33.028+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:33.188 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:33 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:33.028+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:33.188 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:33 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:33.028+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:33.299 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:33 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:33.043+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:33.300 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:33 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:33.043+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:33.300 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:33 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:33.043+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:33.300 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:33 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:33.043+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:33.332 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:32 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:32.944+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:33.332 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:32 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:32.944+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:33.332 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:32 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:32.944+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:33.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:32 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:32.944+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.185228+0000 front 2024-09-19T07:08:04.185080+0000 (oldest deadline 2024-09-19T07:08:31.785369+0000) 2024-09-19T07:08:33.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:33 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:33.114+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:33.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:33 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:33.114+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:33.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:33 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:33.114+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:33.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:33 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:33.114+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:34.316 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:34 smithi134 podman[72634]: 2024-09-19 07:08:34.046802891 +0000 UTC m=+5.589011472 container died 154f7848ea2ff41b7494ff7da9b81d8669275a4ba3aa72ab556e84733f04bc70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T07:08:34.361 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:34 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:34.031+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:34.361 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:34 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:34.031+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:34.361 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:34 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:34.031+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:34.361 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:34 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:34.031+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:34.362 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:33 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:33.956+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:34.362 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:33 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:33.956+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:34.362 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:33 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:33.956+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:34.362 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:33 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:33.956+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.185228+0000 front 2024-09-19T07:08:04.185080+0000 (oldest deadline 2024-09-19T07:08:31.785369+0000) 2024-09-19T07:08:34.362 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:34 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:34.070+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:34.363 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:34 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:34.070+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:34.363 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:34 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:34.070+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:34.363 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:34 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:34.070+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:34.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:34 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:34.013+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:34.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:34 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:34.013+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:34.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:34 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:34.013+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:34.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:34 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:34.013+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:34.687 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:34 smithi134 podman[72634]: 2024-09-19 07:08:34.397945327 +0000 UTC m=+5.940153917 container cleanup 154f7848ea2ff41b7494ff7da9b81d8669275a4ba3aa72ab556e84733f04bc70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T07:08:34.687 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:34 smithi134 bash[72634]: ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7 2024-09-19T07:08:35.016 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:34 smithi134 podman[72649]: 2024-09-19 07:08:34.690691769 +0000 UTC m=+0.642446999 container remove 154f7848ea2ff41b7494ff7da9b81d8669275a4ba3aa72ab556e84733f04bc70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T07:08:35.082 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:34 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:34.966+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:35.083 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:34 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:34.966+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:35.083 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:34 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:34.966+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:35.083 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:34 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:34.966+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.185228+0000 front 2024-09-19T07:08:04.185080+0000 (oldest deadline 2024-09-19T07:08:31.785369+0000) 2024-09-19T07:08:35.083 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:35 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:35.068+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:35.083 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:35 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:35.068+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:35.083 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:35 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:35.068+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:35.083 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:35 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:35.068+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:35.303 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:35 smithi134 podman[72737]: 2024-09-19 07:08:34.918345996 +0000 UTC m=+0.015767075 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:08:35.303 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:35 smithi134 podman[72737]: 2024-09-19 07:08:35.032997752 +0000 UTC m=+0.130418821 container create 8eaad9a48e221a1010ca7a5ea9da1b0f621c97354d347d6f9976d5d6eec52993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7-deactivate, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T07:08:35.303 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:35 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:35.057+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:35.303 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:35 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:35.057+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:35.303 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:35 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:35.057+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:35.303 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:35 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:35.057+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:35.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:35 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:35.022+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:35.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:35 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:35.022+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:35.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:35 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:35.022+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:35.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:35 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:35.022+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:35.687 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:35 smithi134 podman[72737]: 2024-09-19 07:08:35.302907365 +0000 UTC m=+0.400328443 container init 8eaad9a48e221a1010ca7a5ea9da1b0f621c97354d347d6f9976d5d6eec52993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T07:08:35.687 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:35 smithi134 podman[72737]: 2024-09-19 07:08:35.307127573 +0000 UTC m=+0.404548652 container start 8eaad9a48e221a1010ca7a5ea9da1b0f621c97354d347d6f9976d5d6eec52993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-19T07:08:35.687 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:35 smithi134 podman[72737]: 2024-09-19 07:08:35.394358562 +0000 UTC m=+0.491779640 container attach 8eaad9a48e221a1010ca7a5ea9da1b0f621c97354d347d6f9976d5d6eec52993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T07:08:35.687 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:08:35 smithi134 podman[72737]: 2024-09-19 07:08:35.494948659 +0000 UTC m=+0.592369746 container died 8eaad9a48e221a1010ca7a5ea9da1b0f621c97354d347d6f9976d5d6eec52993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T07:08:36.071 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T07:08:36.111 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:08:36.112 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T07:08:36.112 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T07:08:36.112 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.8 2024-09-19T07:08:36.318 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:36 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:36.059+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:36.318 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:36 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:36.059+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:36.318 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:36 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:36.059+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:36.318 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:36 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:36.059+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:36.318 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:36 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:36.059+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:36.332 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:36 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:36.006+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:36.332 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:36 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:36.006+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:36.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:36 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:36.006+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:36.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:36 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:36.006+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.185228+0000 front 2024-09-19T07:08:04.185080+0000 (oldest deadline 2024-09-19T07:08:31.785369+0000) 2024-09-19T07:08:36.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:36 smithi174 systemd[1]: Stopping Ceph osd.8 for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:08:36.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:36 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:36.037+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:36.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:36 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:36.037+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:36.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:36 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:36.037+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:36.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:36 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:36.037+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:36.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:36 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:36.037+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:36.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:36 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:36.011+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:36.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:36 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:36.011+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:36.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:36 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:36.011+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:36.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:36 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:36.011+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:36.833 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:36 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:36.433+0000 7f2f3163d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:08:36.833 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:36 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:36.433+0000 7f2f3163d640 -1 osd.8 107 *** Got signal Terminated *** 2024-09-19T07:08:36.833 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:36 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:36.433+0000 7f2f3163d640 -1 osd.8 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:08:37.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:37 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:37.030+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:37.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:37 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:37.030+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:37.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:37 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:37.030+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:37.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:37 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:37.030+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.185228+0000 front 2024-09-19T07:08:04.185080+0000 (oldest deadline 2024-09-19T07:08:31.785369+0000) 2024-09-19T07:08:37.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:37 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:37.046+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:37.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:37 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:37.046+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:37.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:37 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:37.046+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:37.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:37 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:37.046+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:37.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:37 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:37.046+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:37.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:37 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:37.060+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:37.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:37 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:37.060+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:37.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:37 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:37.060+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:37.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:37 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:37.060+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:37.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:37 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:37.073+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:37.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:37 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:37.073+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:37.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:37 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:37.073+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:37.438 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:37 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:37.073+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:37.438 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:37 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:37.073+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:38.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:37 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:37.996+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:38.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:37 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:37.996+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:38.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:37 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:37.996+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:38.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:37 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:37.996+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.185228+0000 front 2024-09-19T07:08:04.185080+0000 (oldest deadline 2024-09-19T07:08:31.785369+0000) 2024-09-19T07:08:38.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:38 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:38.051+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:38.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:38 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:38.051+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:38.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:38 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:38.051+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:38.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:38 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:38.051+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:38.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:38 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:38.051+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:38.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:38 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:38.072+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:38.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:38 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:38.072+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:38.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:38 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:38.072+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:38.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:38 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:38.072+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:38.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:38 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:38.088+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:38.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:38 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:38.088+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:38.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:38 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:38.088+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:38.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:38 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:38.088+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:38.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:38 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:38.088+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:39.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:38.999+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:39.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:38.999+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:39.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:38.999+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:39.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:38.999+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.185228+0000 front 2024-09-19T07:08:04.185080+0000 (oldest deadline 2024-09-19T07:08:31.785369+0000) 2024-09-19T07:08:39.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:39.034+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:39.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:39.034+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:39.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:39.034+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:39.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:39.034+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:39.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:39.034+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:39.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:39 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:39.074+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:39.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:39 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:39.074+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:39.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:39 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:39.074+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:39.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:39 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:39.074+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:39.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:39 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:39.080+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:39.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:39 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:39.080+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:39.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:39 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:39.080+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:39.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:39 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:39.080+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:39.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:39 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:39.080+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:40.332 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:40 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:40.035+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:40.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:40 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:40.035+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:40.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:40 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:40.035+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:40.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:40 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:40.035+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:40.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:40 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:40.035+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:40.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:40 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:40.035+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:40.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:39.978+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:40.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:39.978+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:40.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:39.978+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:40.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:39.978+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.185228+0000 front 2024-09-19T07:08:04.185080+0000 (oldest deadline 2024-09-19T07:08:31.785369+0000) 2024-09-19T07:08:40.334 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:39.978+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:14.086117+0000 front 2024-09-19T07:08:14.086145+0000 (oldest deadline 2024-09-19T07:08:39.386325+0000) 2024-09-19T07:08:40.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:40.063+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:40.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:40.063+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:40.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:40.063+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:40.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:40 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:40.063+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:40.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:40 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:40.058+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:40.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:40 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:40.058+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:40.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:40 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:40.058+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:40.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:40 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:40.058+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:40.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:40 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:40.058+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:41.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:41 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:41.000+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:41.382328+0000 front 2024-09-19T07:07:41.382314+0000 (oldest deadline 2024-09-19T07:08:06.682035+0000) 2024-09-19T07:08:41.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:41 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:41.000+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:52.483545+0000 front 2024-09-19T07:07:52.483655+0000 (oldest deadline 2024-09-19T07:08:18.383717+0000) 2024-09-19T07:08:41.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:41 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:41.000+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:07:59.484811+0000 front 2024-09-19T07:07:59.484492+0000 (oldest deadline 2024-09-19T07:08:24.184532+0000) 2024-09-19T07:08:41.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:41 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:41.000+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.185228+0000 front 2024-09-19T07:08:04.185080+0000 (oldest deadline 2024-09-19T07:08:31.785369+0000) 2024-09-19T07:08:41.333 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:41 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8[54101]: 2024-09-19T07:08:41.000+0000 7f2f2dc42640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:14.086117+0000 front 2024-09-19T07:08:14.086145+0000 (oldest deadline 2024-09-19T07:08:39.386325+0000) 2024-09-19T07:08:41.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:40 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:40.993+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:41.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:40 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:40.993+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:41.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:40 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:40.993+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:41.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:40 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:40.993+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:41.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:40 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:40.993+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:41.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:40 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:40.993+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:41.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:41 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:41.058+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:41.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:41 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:41.058+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:41.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:41 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:41.058+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:41.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:41 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:41.058+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:41.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:41.073+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:41.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:41.073+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:41.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:41.073+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:41.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:41.073+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:41.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:41 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:41.073+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:42.143 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:41 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:41.983+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:42.143 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:41 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:41.983+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:42.143 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:41 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:41.983+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:42.143 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:41 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:41.983+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:42.143 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:41 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:41.983+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:42.143 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:41 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:41.983+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:42.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:41 smithi174 podman[74032]: 2024-09-19 07:08:41.865110263 +0000 UTC m=+5.557676868 container died 06c393ec9db5d6fe95a62fd1d52ef01f3979aac2d307eee42e4758bae6d684fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-19T07:08:42.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:42 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:42.088+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:42.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:42 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:42.088+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:42.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:42 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:42.088+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:42.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:42 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:42.088+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:42.438 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:42 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:42.044+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:42.440 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:42 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:42.044+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:42.440 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:42 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:42.044+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:42.440 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:42 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:42.044+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:42.441 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:42 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:42.044+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:42.441 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:42 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:42.044+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.286292+0000 front 2024-09-19T07:08:15.286352+0000 (oldest deadline 2024-09-19T07:08:41.086616+0000) 2024-09-19T07:08:42.493 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:42 smithi174 podman[74032]: 2024-09-19 07:08:42.233895182 +0000 UTC m=+5.926461790 container cleanup 06c393ec9db5d6fe95a62fd1d52ef01f3979aac2d307eee42e4758bae6d684fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T07:08:42.493 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:42 smithi174 bash[74032]: ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8 2024-09-19T07:08:42.832 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:42 smithi174 podman[74049]: 2024-09-19 07:08:42.493126131 +0000 UTC m=+0.624077527 container remove 06c393ec9db5d6fe95a62fd1d52ef01f3979aac2d307eee42e4758bae6d684fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T07:08:43.172 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:43 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:43.004+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:43.172 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:43 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:43.004+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:43.172 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:43 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:43.004+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:43.172 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:43 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:43.004+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:43.172 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:43 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:43.004+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:43.172 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:43 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:43.004+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:43.172 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:42 smithi174 podman[74138]: 2024-09-19 07:08:42.749040513 +0000 UTC m=+0.015470589 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:08:43.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:42 smithi174 podman[74138]: 2024-09-19 07:08:42.852293901 +0000 UTC m=+0.118724006 container create c8d4f7dee0eeb217a5cc88170203cca323aa088acf0fbae2a5947b4dd96338ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T07:08:43.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:43 smithi174 podman[74138]: 2024-09-19 07:08:43.087570113 +0000 UTC m=+0.354000176 container init c8d4f7dee0eeb217a5cc88170203cca323aa088acf0fbae2a5947b4dd96338ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:08:43.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:43 smithi174 podman[74138]: 2024-09-19 07:08:43.090774604 +0000 UTC m=+0.357204674 container start c8d4f7dee0eeb217a5cc88170203cca323aa088acf0fbae2a5947b4dd96338ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T07:08:43.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:43 smithi174 podman[74138]: 2024-09-19 07:08:43.170859784 +0000 UTC m=+0.437289858 container attach c8d4f7dee0eeb217a5cc88170203cca323aa088acf0fbae2a5947b4dd96338ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:08:43.344 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:43 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:43.062+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:43.344 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:43 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:43.062+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:43.344 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:43 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:43.062+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:43.344 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:43 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:43.062+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:43.344 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:43 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:43.062+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:16.960235+0000 front 2024-09-19T07:08:16.960151+0000 (oldest deadline 2024-09-19T07:08:42.259887+0000) 2024-09-19T07:08:43.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:43 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:43.018+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:43.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:43 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:43.018+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:43.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:43 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:43.018+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:43.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:43 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:43.018+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:43.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:43 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:43.018+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:43.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:43 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:43.018+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.286292+0000 front 2024-09-19T07:08:15.286352+0000 (oldest deadline 2024-09-19T07:08:41.086616+0000) 2024-09-19T07:08:43.582 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:08:43 smithi174 podman[74138]: 2024-09-19 07:08:43.275416834 +0000 UTC m=+0.541846900 container died c8d4f7dee0eeb217a5cc88170203cca323aa088acf0fbae2a5947b4dd96338ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:08:43.911 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T07:08:43.948 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:08:43.948 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T07:08:43.948 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T07:08:43.948 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.9 2024-09-19T07:08:44.266 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:43 smithi103 systemd[1]: Stopping Ceph osd.9 for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:08:44.267 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:44 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:44.044+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:44.267 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:44 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:44.044+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:44.267 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:44 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:44.044+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:44.267 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:44 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:44.044+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:44.267 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:44 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:44.044+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:16.960235+0000 front 2024-09-19T07:08:16.960151+0000 (oldest deadline 2024-09-19T07:08:42.259887+0000) 2024-09-19T07:08:44.267 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:44 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:44.265+0000 7f9e4103d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:08:44.267 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:44 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:44.265+0000 7f9e4103d640 -1 osd.9 107 *** Got signal Terminated *** 2024-09-19T07:08:44.267 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:44 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:44.265+0000 7f9e4103d640 -1 osd.9 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:08:44.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:44 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:44.037+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:44.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:44 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:44.037+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:44.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:44 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:44.037+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:44.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:44 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:44.037+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:44.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:44 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:44.037+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:44.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:44 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:44.037+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:44.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:43 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:43.984+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:44.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:43 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:43.984+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:44.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:43 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:43.984+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:44.438 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:43 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:43.984+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:44.438 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:43 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:43.984+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:44.438 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:43 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:43.984+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.286292+0000 front 2024-09-19T07:08:15.286352+0000 (oldest deadline 2024-09-19T07:08:41.086616+0000) 2024-09-19T07:08:45.332 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:45 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:45.072+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:45.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:45 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:45.072+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:45.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:45 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:45.072+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:45.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:45 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:45.072+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:45.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:45 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:45.072+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:45.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:45 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:45.072+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:45.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:45 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:45.069+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:45.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:45 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:45.069+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:45.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:45 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:45.069+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:45.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:45 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:45.069+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:45.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:45 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:45.069+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:16.960235+0000 front 2024-09-19T07:08:16.960151+0000 (oldest deadline 2024-09-19T07:08:42.259887+0000) 2024-09-19T07:08:45.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:45 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:45.023+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:45.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:45 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:45.023+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:45.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:45 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:45.023+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:45.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:45 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:45.023+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:45.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:45 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:45.023+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:45.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:45 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:45.023+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.286292+0000 front 2024-09-19T07:08:15.286352+0000 (oldest deadline 2024-09-19T07:08:41.086616+0000) 2024-09-19T07:08:46.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:46 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:46.033+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:46.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:46 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:46.033+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:46.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:46 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:46.033+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:46.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:46 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:46.033+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:46.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:46 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:46.033+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:46.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:46 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:46.033+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:46.338 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:46 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:46.024+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:46.338 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:46 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:46.024+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:46.338 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:46 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:46.024+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:46.338 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:46 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:46.024+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:46.338 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:46 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:46.024+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:46.338 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:46 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:46.024+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.286292+0000 front 2024-09-19T07:08:15.286352+0000 (oldest deadline 2024-09-19T07:08:41.086616+0000) 2024-09-19T07:08:46.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:46 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:46.093+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:46.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:46 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:46.093+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:46.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:46 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:46.093+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:46.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:46 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:46.093+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:46.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:46 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:46.093+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:16.960235+0000 front 2024-09-19T07:08:16.960151+0000 (oldest deadline 2024-09-19T07:08:42.259887+0000) 2024-09-19T07:08:47.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:46 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:46.990+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:47.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:46 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:46.990+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:47.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:46 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:46.990+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:47.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:46 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:46.990+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:47.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:46 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:46.990+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:47.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:46 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:46.990+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.286292+0000 front 2024-09-19T07:08:15.286352+0000 (oldest deadline 2024-09-19T07:08:41.086616+0000) 2024-09-19T07:08:47.332 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:47 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:47.055+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:47.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:47 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:47.055+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:47.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:47 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:47.055+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:47.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:47 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:47.055+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:47.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:47 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:47.055+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:47.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:47 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:47.055+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:47.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:47 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:47.108+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:47.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:47 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:47.108+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:47.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:47 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:47.108+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:47.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:47 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:47.108+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:47.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:47 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:47.108+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:16.960235+0000 front 2024-09-19T07:08:16.960151+0000 (oldest deadline 2024-09-19T07:08:42.259887+0000) 2024-09-19T07:08:48.332 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:48 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:48.032+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:48.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:48 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:48.032+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:48.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:48 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:48.032+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:48.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:48 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:48.032+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:48.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:48 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:48.032+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:48.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:48 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:48.032+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:48.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:48 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:48.114+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:48.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:48 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:48.114+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:48.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:48 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:48.114+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:48.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:48 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:48.114+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:48.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:48 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:48.114+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:16.960235+0000 front 2024-09-19T07:08:16.960151+0000 (oldest deadline 2024-09-19T07:08:42.259887+0000) 2024-09-19T07:08:48.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:47 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:47.976+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:48.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:47 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:47.976+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:48.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:47 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:47.976+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:48.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:47 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:47.976+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:48.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:47 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:47.976+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:48.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:47 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:47.976+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.286292+0000 front 2024-09-19T07:08:15.286352+0000 (oldest deadline 2024-09-19T07:08:41.086616+0000) 2024-09-19T07:08:49.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:49.026+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:49.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:49.026+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:49.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:49.026+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:49.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:49.026+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:49.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:49.026+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:49.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:49.026+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:49.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:49 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:49.081+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:34.853883+0000 front 2024-09-19T07:07:34.853898+0000 (oldest deadline 2024-09-19T07:08:00.753988+0000) 2024-09-19T07:08:49.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:49 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:49.081+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.855128+0000 front 2024-09-19T07:07:44.855027+0000 (oldest deadline 2024-09-19T07:08:07.755173+0000) 2024-09-19T07:08:49.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:49 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:49.081+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:51.856177+0000 front 2024-09-19T07:07:51.856226+0000 (oldest deadline 2024-09-19T07:08:15.356222+0000) 2024-09-19T07:08:49.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:49 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:49.081+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.657254+0000 front 2024-09-19T07:08:00.657358+0000 (oldest deadline 2024-09-19T07:08:25.357333+0000) 2024-09-19T07:08:49.388 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:49 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9[65661]: 2024-09-19T07:08:49.081+0000 7f9e3d642640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:16.960235+0000 front 2024-09-19T07:08:16.960151+0000 (oldest deadline 2024-09-19T07:08:42.259887+0000) 2024-09-19T07:08:49.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:48 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:48.969+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:49.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:48 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:48.969+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:49.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:48 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:48.969+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:49.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:48 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:48.969+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:49.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:48 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:48.969+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:49.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:48 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:48.969+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.286292+0000 front 2024-09-19T07:08:15.286352+0000 (oldest deadline 2024-09-19T07:08:41.086616+0000) 2024-09-19T07:08:49.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:48 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:48.969+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:08:23.387456+0000 front 2024-09-19T07:08:23.387428+0000 (oldest deadline 2024-09-19T07:08:48.687292+0000) 2024-09-19T07:08:50.025 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:49 smithi103 podman[137680]: 2024-09-19 07:08:49.767833984 +0000 UTC m=+5.622459126 container died 7d9e98cfcc597ec06ad548769adcd9aeca910856bb34547e255ce90891203003 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T07:08:50.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:49 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:49.922+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:50.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:49 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:49.922+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:50.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:49 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:49.922+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:50.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:49 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:49.922+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:50.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:49 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:49.922+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:50.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:49 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:49.922+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.286292+0000 front 2024-09-19T07:08:15.286352+0000 (oldest deadline 2024-09-19T07:08:41.086616+0000) 2024-09-19T07:08:50.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:49 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:49.922+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:08:23.387456+0000 front 2024-09-19T07:08:23.387428+0000 (oldest deadline 2024-09-19T07:08:48.687292+0000) 2024-09-19T07:08:50.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:49.989+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:50.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:49.989+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:50.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:49.989+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:50.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:49.989+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:50.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:49.989+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:50.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:49 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:49.989+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:50.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:50 smithi103 podman[137680]: 2024-09-19 07:08:50.182619668 +0000 UTC m=+6.037244841 container cleanup 7d9e98cfcc597ec06ad548769adcd9aeca910856bb34547e255ce90891203003 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T07:08:50.387 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:50 smithi103 bash[137680]: ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9 2024-09-19T07:08:50.850 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:50 smithi103 podman[137694]: 2024-09-19 07:08:50.411716242 +0000 UTC m=+0.636114925 container remove 7d9e98cfcc597ec06ad548769adcd9aeca910856bb34547e255ce90891203003 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T07:08:51.137 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:50 smithi103 podman[137783]: 2024-09-19 07:08:50.751780238 +0000 UTC m=+0.015466148 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:08:51.138 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:50 smithi103 podman[137783]: 2024-09-19 07:08:50.851374729 +0000 UTC m=+0.115060651 container create a6dd1e5f17d767567930790c07199cef41d3683ddfd72440f1bc85d75bd91b8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-19T07:08:51.138 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:51 smithi103 podman[137783]: 2024-09-19 07:08:51.07032879 +0000 UTC m=+0.334014687 container init a6dd1e5f17d767567930790c07199cef41d3683ddfd72440f1bc85d75bd91b8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T07:08:51.138 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:51 smithi103 podman[137783]: 2024-09-19 07:08:51.073660354 +0000 UTC m=+0.337346234 container start a6dd1e5f17d767567930790c07199cef41d3683ddfd72440f1bc85d75bd91b8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:08:51.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:51 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:51.036+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:51.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:51 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:51.036+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:51.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:51 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:51.036+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:51.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:51 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:51.036+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:51.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:51 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:51.036+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:51.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:51 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:51.036+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:51.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:50 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:50.959+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:51.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:50 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:50.959+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:51.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:50 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:50.959+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:51.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:50 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:50.959+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:51.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:50 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:50.959+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:51.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:50 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:50.959+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.286292+0000 front 2024-09-19T07:08:15.286352+0000 (oldest deadline 2024-09-19T07:08:41.086616+0000) 2024-09-19T07:08:51.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:50 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:50.959+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:08:23.387456+0000 front 2024-09-19T07:08:23.387428+0000 (oldest deadline 2024-09-19T07:08:48.687292+0000) 2024-09-19T07:08:51.556 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:51 smithi103 podman[137783]: 2024-09-19 07:08:51.170079466 +0000 UTC m=+0.433765347 container attach a6dd1e5f17d767567930790c07199cef41d3683ddfd72440f1bc85d75bd91b8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T07:08:51.556 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:08:51 smithi103 podman[137783]: 2024-09-19 07:08:51.258988652 +0000 UTC m=+0.522674539 container died a6dd1e5f17d767567930790c07199cef41d3683ddfd72440f1bc85d75bd91b8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T07:08:51.940 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T07:08:51.980 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:08:51.980 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T07:08:51.981 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T07:08:51.981 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.10 2024-09-19T07:08:52.291 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:51 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:51.997+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:52.292 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:51 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:51.997+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:52.292 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:51 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:51.997+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:52.292 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:51 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:51.997+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:52.292 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:51 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:51.997+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:52.292 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:51 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:51.997+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.286292+0000 front 2024-09-19T07:08:15.286352+0000 (oldest deadline 2024-09-19T07:08:41.086616+0000) 2024-09-19T07:08:52.292 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:51 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:51.997+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:08:23.387456+0000 front 2024-09-19T07:08:23.387428+0000 (oldest deadline 2024-09-19T07:08:48.687292+0000) 2024-09-19T07:08:52.292 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:52 smithi134 systemd[1]: Stopping Ceph osd.10 for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:08:52.292 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:52 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:52.289+0000 7fef4425b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:08:52.292 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:52 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:52.289+0000 7fef4425b640 -1 osd.10 107 *** Got signal Terminated *** 2024-09-19T07:08:52.292 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:52 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:52.289+0000 7fef4425b640 -1 osd.10 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:08:52.332 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:52 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:52.008+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:52.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:52 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:52.008+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:52.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:52 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:52.008+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:52.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:52 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:52.008+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:52.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:52 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:52.008+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:52.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:52 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:52.008+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:53.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:53 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:53.040+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:53.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:53 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:53.040+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:53.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:53 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:53.040+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:53.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:53 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:53.040+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:53.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:53 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:53.040+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:53.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:53 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:53.040+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:53.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:52 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:52.957+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:53.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:52 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:52.957+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:53.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:52 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:52.957+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:53.438 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:52 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:52.957+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:53.438 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:52 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:52.957+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:53.438 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:52 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:52.957+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.286292+0000 front 2024-09-19T07:08:15.286352+0000 (oldest deadline 2024-09-19T07:08:41.086616+0000) 2024-09-19T07:08:53.438 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:52 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:52.957+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:08:23.387456+0000 front 2024-09-19T07:08:23.387428+0000 (oldest deadline 2024-09-19T07:08:48.687292+0000) 2024-09-19T07:08:54.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:53 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:53.921+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:54.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:53 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:53.921+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:54.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:53 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:53.921+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:54.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:53 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:53.921+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:54.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:53 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:53.921+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:54.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:53 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:53.921+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.286292+0000 front 2024-09-19T07:08:15.286352+0000 (oldest deadline 2024-09-19T07:08:41.086616+0000) 2024-09-19T07:08:54.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:53 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:53.921+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:08:23.387456+0000 front 2024-09-19T07:08:23.387428+0000 (oldest deadline 2024-09-19T07:08:48.687292+0000) 2024-09-19T07:08:54.332 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:54 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:54.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:54.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:54 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:54.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:54.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:54 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:54.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:54.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:54 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:54.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:54.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:54 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:54.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:54.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:54 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:54.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:55.198 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:54 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:54.907+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:55.198 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:54 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:54.907+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:55.198 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:54 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:54.907+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:55.198 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:54 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:54.907+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:55.198 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:54 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:54.907+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:55.198 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:54 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:54.907+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.286292+0000 front 2024-09-19T07:08:15.286352+0000 (oldest deadline 2024-09-19T07:08:41.086616+0000) 2024-09-19T07:08:55.198 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:54 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:54.907+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:08:23.387456+0000 front 2024-09-19T07:08:23.387428+0000 (oldest deadline 2024-09-19T07:08:48.687292+0000) 2024-09-19T07:08:55.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:55 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:55.025+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:55.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:55 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:55.025+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:55.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:55 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:55.025+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:55.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:55 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:55.025+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:55.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:55 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:55.025+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:55.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:55 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:55.025+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:55.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:55 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:55.025+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:08:31.789305+0000 front 2024-09-19T07:08:31.789366+0000 (oldest deadline 2024-09-19T07:08:54.689143+0000) 2024-09-19T07:08:56.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:55 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:55.870+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:56.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:55 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:55.870+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:56.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:55 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:55.870+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:56.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:55 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:55.870+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:56.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:55 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:55.870+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:56.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:55 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:55.870+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.286292+0000 front 2024-09-19T07:08:15.286352+0000 (oldest deadline 2024-09-19T07:08:41.086616+0000) 2024-09-19T07:08:56.187 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:55 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:55.870+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:08:23.387456+0000 front 2024-09-19T07:08:23.387428+0000 (oldest deadline 2024-09-19T07:08:48.687292+0000) 2024-09-19T07:08:56.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:56 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:56.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:56.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:56 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:56.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:56.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:56 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:56.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:56.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:56 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:56.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:56.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:56 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:56.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:56.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:56 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:56.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:56.334 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:56 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:56.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:08:31.789305+0000 front 2024-09-19T07:08:31.789366+0000 (oldest deadline 2024-09-19T07:08:54.689143+0000) 2024-09-19T07:08:57.084 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:56 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:56.821+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:37.881305+0000 front 2024-09-19T07:07:37.881172+0000 (oldest deadline 2024-09-19T07:08:03.181251+0000) 2024-09-19T07:08:57.084 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:56 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:56.821+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.882116+0000 front 2024-09-19T07:07:44.882188+0000 (oldest deadline 2024-09-19T07:08:08.382169+0000) 2024-09-19T07:08:57.084 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:56 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:56.821+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:54.183846+0000 front 2024-09-19T07:07:54.183609+0000 (oldest deadline 2024-09-19T07:08:16.483665+0000) 2024-09-19T07:08:57.084 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:56 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:56.821+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.584771+0000 front 2024-09-19T07:08:00.584816+0000 (oldest deadline 2024-09-19T07:08:23.484752+0000) 2024-09-19T07:08:57.085 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:56 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:56.821+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:09.385575+0000 front 2024-09-19T07:08:09.385606+0000 (oldest deadline 2024-09-19T07:08:35.285733+0000) 2024-09-19T07:08:57.085 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:56 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:56.821+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.286292+0000 front 2024-09-19T07:08:15.286352+0000 (oldest deadline 2024-09-19T07:08:41.086616+0000) 2024-09-19T07:08:57.085 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:56 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10[60399]: 2024-09-19T07:08:56.821+0000 7fef40860640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:08:23.387456+0000 front 2024-09-19T07:08:23.387428+0000 (oldest deadline 2024-09-19T07:08:48.687292+0000) 2024-09-19T07:08:57.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:57 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:57.102+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:57.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:57 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:57.102+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:57.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:57 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:57.102+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:57.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:57 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:57.102+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:57.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:57 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:57.102+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:57.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:57 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:57.102+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:57.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:57 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:57.102+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:08:31.789305+0000 front 2024-09-19T07:08:31.789366+0000 (oldest deadline 2024-09-19T07:08:54.689143+0000) 2024-09-19T07:08:57.937 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:57 smithi134 podman[72855]: 2024-09-19 07:08:57.843887133 +0000 UTC m=+5.666496071 container died c739b2754c194cae63f88efcd515e09bc9aca930a9713e3674e2587775341b4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:08:58.434 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:58 smithi134 podman[72855]: 2024-09-19 07:08:58.207655832 +0000 UTC m=+6.030264771 container cleanup c739b2754c194cae63f88efcd515e09bc9aca930a9713e3674e2587775341b4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:08:58.434 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:58 smithi134 bash[72855]: ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10 2024-09-19T07:08:58.585 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:58 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:58.112+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:58.585 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:58 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:58.112+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:58.585 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:58 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:58.112+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:58.585 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:58 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:58.112+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:58.585 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:58 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:58.112+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:58.585 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:58 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:58.112+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:58.585 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:58 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:58.112+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:08:31.789305+0000 front 2024-09-19T07:08:31.789366+0000 (oldest deadline 2024-09-19T07:08:54.689143+0000) 2024-09-19T07:08:58.687 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:58 smithi134 podman[72871]: 2024-09-19 07:08:58.433944879 +0000 UTC m=+0.592348351 container remove c739b2754c194cae63f88efcd515e09bc9aca930a9713e3674e2587775341b4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T07:08:59.137 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:58 smithi134 podman[72957]: 2024-09-19 07:08:58.775879004 +0000 UTC m=+0.016327000 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:08:59.137 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:58 smithi134 podman[72957]: 2024-09-19 07:08:58.893055306 +0000 UTC m=+0.133503335 container create 63cea5bd34c4db72a46a443f5b9240225879e5199a3cc94ed2fdf0b029a8a806 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T07:08:59.332 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:59 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:59.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:08:59.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:59 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:59.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:08:59.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:59 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:59.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:08:59.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:59 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:59.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:08:59.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:59 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:59.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:08:59.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:59 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:59.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:08:59.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:08:59 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:08:59.071+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:08:31.789305+0000 front 2024-09-19T07:08:31.789366+0000 (oldest deadline 2024-09-19T07:08:54.689143+0000) 2024-09-19T07:08:59.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:59 smithi134 podman[72957]: 2024-09-19 07:08:59.137101636 +0000 UTC m=+0.377549623 container init 63cea5bd34c4db72a46a443f5b9240225879e5199a3cc94ed2fdf0b029a8a806 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T07:08:59.437 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:59 smithi134 podman[72957]: 2024-09-19 07:08:59.140444931 +0000 UTC m=+0.380892919 container start 63cea5bd34c4db72a46a443f5b9240225879e5199a3cc94ed2fdf0b029a8a806 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T07:08:59.438 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:59 smithi134 podman[72957]: 2024-09-19 07:08:59.212014488 +0000 UTC m=+0.452462491 container attach 63cea5bd34c4db72a46a443f5b9240225879e5199a3cc94ed2fdf0b029a8a806 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-09-19T07:08:59.438 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:59 smithi134 podman[72957]: 2024-09-19 07:08:59.332279689 +0000 UTC m=+0.572727679 container died 63cea5bd34c4db72a46a443f5b9240225879e5199a3cc94ed2fdf0b029a8a806 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-09-19T07:08:59.923 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T07:08:59.937 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:59 smithi134 podman[72981]: 2024-09-19 07:08:59.90714704 +0000 UTC m=+0.569240335 container remove 63cea5bd34c4db72a46a443f5b9240225879e5199a3cc94ed2fdf0b029a8a806 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:08:59.937 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:59 smithi134 systemd[1]: ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.10.service: Deactivated successfully. 2024-09-19T07:08:59.937 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:59 smithi134 systemd[1]: Stopped Ceph osd.10 for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T07:08:59.937 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:08:59 smithi134 systemd[1]: ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.10.service: Consumed 15.661s CPU time. 2024-09-19T07:08:59.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:08:59.962 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T07:08:59.962 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T07:08:59.963 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.11 2024-09-19T07:09:00.286 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:00 smithi174 systemd[1]: Stopping Ceph osd.11 for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:09:00.286 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:00 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:00.111+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:09:00.286 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:00 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:00.111+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:09:00.286 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:00 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:00.111+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:09:00.286 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:00 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:00.111+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:09:00.286 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:00 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:00.111+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:09:00.286 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:00 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:00.111+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:09:00.286 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:00 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:00.111+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:08:31.789305+0000 front 2024-09-19T07:08:31.789366+0000 (oldest deadline 2024-09-19T07:08:54.689143+0000) 2024-09-19T07:09:00.287 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:00 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:00.284+0000 7f694b884640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:09:00.287 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:00 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:00.284+0000 7f694b884640 -1 osd.11 107 *** Got signal Terminated *** 2024-09-19T07:09:00.287 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:00 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:00.284+0000 7f694b884640 -1 osd.11 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:09:01.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:01 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:01.115+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:09:01.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:01 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:01.115+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:09:01.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:01 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:01.115+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:09:01.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:01 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:01.115+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:09:01.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:01 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:01.115+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:09:01.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:01 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:01.115+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:09:01.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:01 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:01.115+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:08:31.789305+0000 front 2024-09-19T07:08:31.789366+0000 (oldest deadline 2024-09-19T07:08:54.689143+0000) 2024-09-19T07:09:02.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:02 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:02.067+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:09:02.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:02 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:02.067+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:09:02.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:02 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:02.067+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:09:02.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:02 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:02.067+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:09:02.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:02 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:02.067+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:09:02.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:02 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:02.067+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:09:02.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:02 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:02.067+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:08:31.789305+0000 front 2024-09-19T07:08:31.789366+0000 (oldest deadline 2024-09-19T07:08:54.689143+0000) 2024-09-19T07:09:03.332 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:03 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:03.069+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:09:03.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:03 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:03.069+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:09:03.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:03 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:03.069+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:09:03.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:03 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:03.069+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:09:03.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:03 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:03.069+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:09:03.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:03 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:03.069+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:09:03.333 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:03 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:03.070+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:08:31.789305+0000 front 2024-09-19T07:08:31.789366+0000 (oldest deadline 2024-09-19T07:08:54.689143+0000) 2024-09-19T07:09:04.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:04 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:04.108+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:09:04.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:04 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:04.108+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:09:04.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:04 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:04.108+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:09:04.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:04 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:04.108+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:09:04.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:04 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:04.108+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:09:04.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:04 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:04.108+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:09:04.583 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:04 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:04.108+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:08:31.789305+0000 front 2024-09-19T07:08:31.789366+0000 (oldest deadline 2024-09-19T07:08:54.689143+0000) 2024-09-19T07:09:05.582 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:05 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:05.100+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:07:33.480926+0000 front 2024-09-19T07:07:33.480852+0000 (oldest deadline 2024-09-19T07:07:59.380890+0000) 2024-09-19T07:09:05.600 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:05 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:05.100+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:07:44.582489+0000 front 2024-09-19T07:07:44.582408+0000 (oldest deadline 2024-09-19T07:08:08.082363+0000) 2024-09-19T07:09:05.600 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:05 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:05.100+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:07:50.883790+0000 front 2024-09-19T07:07:50.883917+0000 (oldest deadline 2024-09-19T07:08:14.983934+0000) 2024-09-19T07:09:05.601 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:05 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:05.100+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:08:00.685552+0000 front 2024-09-19T07:08:00.685658+0000 (oldest deadline 2024-09-19T07:08:24.785598+0000) 2024-09-19T07:09:05.601 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:05 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:05.100+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:08:04.786075+0000 front 2024-09-19T07:08:04.786094+0000 (oldest deadline 2024-09-19T07:08:35.986590+0000) 2024-09-19T07:09:05.601 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:05 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:05.100+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:08:15.987019+0000 front 2024-09-19T07:08:15.986903+0000 (oldest deadline 2024-09-19T07:08:39.486973+0000) 2024-09-19T07:09:05.601 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:05 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11[60876]: 2024-09-19T07:09:05.100+0000 7f6943e81640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:08:31.789305+0000 front 2024-09-19T07:08:31.789366+0000 (oldest deadline 2024-09-19T07:08:54.689143+0000) 2024-09-19T07:09:05.868 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:05 smithi174 podman[74253]: 2024-09-19 07:09:05.617273362 +0000 UTC m=+5.456694108 container died 7dfe262d614fd14113b7327d4fe724637d3587978f473d6f1ad3ce14bc2b033f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:09:06.143 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:05 smithi174 podman[74253]: 2024-09-19 07:09:05.950917206 +0000 UTC m=+5.790337952 container cleanup 7dfe262d614fd14113b7327d4fe724637d3587978f473d6f1ad3ce14bc2b033f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-19T07:09:06.144 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:05 smithi174 bash[74253]: ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11 2024-09-19T07:09:06.545 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:06 smithi174 podman[74269]: 2024-09-19 07:09:06.143684208 +0000 UTC m=+0.520012190 container remove 7dfe262d614fd14113b7327d4fe724637d3587978f473d6f1ad3ce14bc2b033f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T07:09:06.832 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:06 smithi174 podman[74360]: 2024-09-19 07:09:06.544499915 +0000 UTC m=+0.108020723 container create 1860ded7e2cd9144c6bde5fa1f7669ddc361478c82f9474b4334a35ea2d1b5c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-19T07:09:06.833 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:06 smithi174 podman[74360]: 2024-09-19 07:09:06.452056389 +0000 UTC m=+0.015577162 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:09:06.833 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:06 smithi174 podman[74360]: 2024-09-19 07:09:06.763354067 +0000 UTC m=+0.326874835 container init 1860ded7e2cd9144c6bde5fa1f7669ddc361478c82f9474b4334a35ea2d1b5c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T07:09:06.833 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:06 smithi174 podman[74360]: 2024-09-19 07:09:06.766630011 +0000 UTC m=+0.330150785 container start 1860ded7e2cd9144c6bde5fa1f7669ddc361478c82f9474b4334a35ea2d1b5c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-19T07:09:07.241 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:06 smithi174 podman[74360]: 2024-09-19 07:09:06.854900328 +0000 UTC m=+0.418421106 container attach 1860ded7e2cd9144c6bde5fa1f7669ddc361478c82f9474b4334a35ea2d1b5c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T07:09:07.241 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:09:06 smithi174 podman[74360]: 2024-09-19 07:09:06.953071608 +0000 UTC m=+0.516592376 container died 1860ded7e2cd9144c6bde5fa1f7669ddc361478c82f9474b4334a35ea2d1b5c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:09:07.528 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T07:09:07.566 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:09:07.567 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T07:09:07.567 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T07:09:07.567 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.a 2024-09-19T07:09:07.868 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:09:07 smithi103 systemd[1]: Stopping Ceph mds.a for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:09:08.137 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:09:07 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-a[72495]: 2024-09-19T07:09:07.866+0000 7f956a7b9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:09:08.137 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:09:07 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-a[72495]: 2024-09-19T07:09:07.866+0000 7f956a7b9640 -1 mds.a *** got signal Terminated *** 2024-09-19T07:09:10.387 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:09:09 smithi103 podman[137885]: 2024-09-19 07:09:09.967140584 +0000 UTC m=+2.203405285 container died a036b7678ef625c8e1776c40d4a8686e6a5eb294dc7e54c7eeff8c533a7a0ed0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-19T07:09:10.877 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:09:10 smithi103 podman[137885]: 2024-09-19 07:09:10.650411139 +0000 UTC m=+2.886675840 container cleanup a036b7678ef625c8e1776c40d4a8686e6a5eb294dc7e54c7eeff8c533a7a0ed0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-a, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T07:09:10.877 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:09:10 smithi103 bash[137885]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-a 2024-09-19T07:09:11.028 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T07:09:11.063 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:09:11.063 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T07:09:11.063 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T07:09:11.064 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.d 2024-09-19T07:09:11.637 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:09:11 smithi103 systemd[1]: Stopping Ceph mds.d for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:09:11.637 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:09:11 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-d[71316]: 2024-09-19T07:09:11.395+0000 7ff1906a2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:09:11.637 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:09:11 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-d[71316]: 2024-09-19T07:09:11.395+0000 7ff1906a2640 -1 mds.d *** got signal Terminated *** 2024-09-19T07:09:13.887 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:09:13 smithi103 podman[138004]: 2024-09-19 07:09:13.62846944 +0000 UTC m=+2.342551027 container died 940e4279d328c783d88d3829bc064490103ae269ce2eeae0011632537bc6051d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T07:09:14.247 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:09:14 smithi103 podman[138004]: 2024-09-19 07:09:14.054324546 +0000 UTC m=+2.768406119 container cleanup 940e4279d328c783d88d3829bc064490103ae269ce2eeae0011632537bc6051d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-d, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:09:14.247 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:09:14 smithi103 bash[138004]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-d 2024-09-19T07:09:14.474 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T07:09:14.507 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:09:14.507 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T07:09:14.507 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T07:09:14.507 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.g 2024-09-19T07:09:14.817 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:09:14 smithi103 systemd[1]: Stopping Ceph mds.g for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:09:15.137 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:09:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-g[72112]: 2024-09-19T07:09:14.815+0000 7feb0c240640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:09:15.137 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:09:14 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-g[72112]: 2024-09-19T07:09:14.815+0000 7feb0c240640 -1 mds.g *** got signal Terminated *** 2024-09-19T07:09:18.301 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:09:17 smithi103 podman[138125]: 2024-09-19 07:09:17.977220488 +0000 UTC m=+3.256779609 container died e59a59eb9a948822cad8d1e640eb051f7688106335231a110e511e4cfcf1a156 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-g, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:09:18.637 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:09:18 smithi103 podman[138125]: 2024-09-19 07:09:18.474847139 +0000 UTC m=+3.754406259 container cleanup e59a59eb9a948822cad8d1e640eb051f7688106335231a110e511e4cfcf1a156 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-g, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:09:18.637 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:09:18 smithi103 bash[138125]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-g 2024-09-19T07:09:18.893 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T07:09:18.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:09:18.925 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T07:09:18.925 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T07:09:18.925 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.j 2024-09-19T07:09:19.278 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:09:19 smithi103 systemd[1]: Stopping Ceph mds.j for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:09:19.637 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:09:19 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-j[70918]: 2024-09-19T07:09:19.278+0000 7fc801f00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:09:19.637 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:09:19 smithi103 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-j[70918]: 2024-09-19T07:09:19.278+0000 7fc801f00640 -1 mds.j *** got signal Terminated *** 2024-09-19T07:09:22.387 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:09:22 smithi103 podman[138246]: 2024-09-19 07:09:22.075401428 +0000 UTC m=+2.927575531 container died c9305037c62eaa32e907dec30eb55e69431438974a8494104fb9fe535eb04401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:09:23.022 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:09:22 smithi103 podman[138246]: 2024-09-19 07:09:22.81210465 +0000 UTC m=+3.664278750 container cleanup c9305037c62eaa32e907dec30eb55e69431438974a8494104fb9fe535eb04401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-j, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-19T07:09:23.022 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:09:22 smithi103 bash[138246]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-j 2024-09-19T07:09:23.231 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T07:09:23.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:09:23.264 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T07:09:23.264 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T07:09:23.264 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.b 2024-09-19T07:09:23.558 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:09:23 smithi134 systemd[1]: Stopping Ceph mds.b for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:09:23.814 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:09:23 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-b[65626]: 2024-09-19T07:09:23.557+0000 7fb5156d1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:09:23.814 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:09:23 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-b[65626]: 2024-09-19T07:09:23.557+0000 7fb5156d1640 -1 mds.b *** got signal Terminated *** 2024-09-19T07:09:26.283 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:09:25 smithi134 podman[73061]: 2024-09-19 07:09:25.973676336 +0000 UTC m=+2.509940979 container died a91eef13aaebec802961e8a425ca147c8e21d9849ff4dd0e94e333fdef311f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:09:26.591 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:09:26 smithi134 podman[73061]: 2024-09-19 07:09:26.365337937 +0000 UTC m=+2.901602581 container cleanup a91eef13aaebec802961e8a425ca147c8e21d9849ff4dd0e94e333fdef311f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-b, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:09:26.592 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:09:26 smithi134 bash[73061]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-b 2024-09-19T07:09:26.724 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T07:09:26.761 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:09:26.761 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T07:09:26.761 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T07:09:26.761 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.e 2024-09-19T07:09:26.937 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:09:26 smithi134 systemd[1]: Stopping Ceph mds.e for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:09:27.437 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:09:27 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-e[65009]: 2024-09-19T07:09:27.077+0000 7fa9c0c1b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:09:27.437 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:09:27 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-e[65009]: 2024-09-19T07:09:27.077+0000 7fa9c0c1b640 -1 mds.e *** got signal Terminated *** 2024-09-19T07:09:30.262 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:09:29 smithi134 podman[73194]: 2024-09-19 07:09:29.974608162 +0000 UTC m=+2.997106816 container died 3d782529ff85ae7e8d3ea066b9cfd1a08aba4abe234d4bcb4abd99d627fa71d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:09:30.595 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:09:30 smithi134 podman[73194]: 2024-09-19 07:09:30.352835367 +0000 UTC m=+3.375334022 container cleanup 3d782529ff85ae7e8d3ea066b9cfd1a08aba4abe234d4bcb4abd99d627fa71d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T07:09:30.595 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:09:30 smithi134 bash[73194]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-e 2024-09-19T07:09:30.744 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T07:09:30.777 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:09:30.777 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T07:09:30.778 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T07:09:30.778 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.h 2024-09-19T07:09:31.187 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:09:30 smithi134 systemd[1]: Stopping Ceph mds.h for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:09:31.187 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:09:31 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-h[65311]: 2024-09-19T07:09:31.089+0000 7f9c1a0bf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:09:31.187 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:09:31 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-h[65311]: 2024-09-19T07:09:31.089+0000 7f9c1a0bf640 -1 mds.h *** got signal Terminated *** 2024-09-19T07:09:34.280 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:09:33 smithi134 podman[73314]: 2024-09-19 07:09:33.98651927 +0000 UTC m=+2.987837235 container died 91cf7bec2287c75bb9d7ea0d6ad6381a33a3e1757e35d4a8a95a4f85e908c8cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-h, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:09:34.687 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:09:34 smithi134 podman[73314]: 2024-09-19 07:09:34.314244139 +0000 UTC m=+3.315562111 container cleanup 91cf7bec2287c75bb9d7ea0d6ad6381a33a3e1757e35d4a8a95a4f85e908c8cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T07:09:34.687 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:09:34 smithi134 bash[73314]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-h 2024-09-19T07:09:34.687 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:09:34 smithi134 podman[73325]: 2024-09-19 07:09:34.523828205 +0000 UTC m=+0.530834256 container remove 91cf7bec2287c75bb9d7ea0d6ad6381a33a3e1757e35d4a8a95a4f85e908c8cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T07:09:34.739 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T07:09:34.774 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:09:34.775 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T07:09:34.775 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T07:09:34.775 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.k 2024-09-19T07:09:35.093 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:09:34 smithi134 systemd[1]: Stopping Ceph mds.k for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:09:35.437 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:09:35 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-k[64693]: 2024-09-19T07:09:35.091+0000 7f6b3e38b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:09:35.437 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:09:35 smithi134 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-k[64693]: 2024-09-19T07:09:35.091+0000 7f6b3e38b640 -1 mds.k *** got signal Terminated *** 2024-09-19T07:09:38.252 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:09:37 smithi134 podman[73434]: 2024-09-19 07:09:37.98006934 +0000 UTC m=+2.990780082 container died d2780a11bcfb42f6317855c11fe289b691ea42a0a1f790adc3845bc3fbbe007d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T07:09:38.502 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:09:38 smithi134 podman[73434]: 2024-09-19 07:09:38.318157637 +0000 UTC m=+3.328868379 container cleanup d2780a11bcfb42f6317855c11fe289b691ea42a0a1f790adc3845bc3fbbe007d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-k, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T07:09:38.502 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:09:38 smithi134 bash[73434]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-k 2024-09-19T07:09:38.736 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T07:09:38.757 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:09:38 smithi134 podman[73447]: 2024-09-19 07:09:38.502386849 +0000 UTC m=+0.518571668 container remove d2780a11bcfb42f6317855c11fe289b691ea42a0a1f790adc3845bc3fbbe007d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-k, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:09:38.757 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:09:38 smithi134 systemd[1]: ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.k.service: Deactivated successfully. 2024-09-19T07:09:38.758 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:09:38 smithi134 systemd[1]: Stopped Ceph mds.k for d158d934-7651-11ef-bceb-c7b262605968. 2024-09-19T07:09:38.758 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:09:38 smithi134 systemd[1]: ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.k.service: Consumed 2.682s CPU time. 2024-09-19T07:09:38.769 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:09:38.769 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T07:09:38.769 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T07:09:38.769 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.c 2024-09-19T07:09:39.083 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:09:38 smithi174 systemd[1]: Stopping Ceph mds.c for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:09:39.582 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:09:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-c[66054]: 2024-09-19T07:09:39.123+0000 7ff6e127a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:09:39.582 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:09:39 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-c[66054]: 2024-09-19T07:09:39.123+0000 7ff6e127a640 -1 mds.c *** got signal Terminated *** 2024-09-19T07:09:42.290 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:09:41 smithi174 podman[74463]: 2024-09-19 07:09:41.982460177 +0000 UTC m=+3.025050022 container died e81cc5d6af587aee4aa9dd5330ec3c0d1daea70375b09e09c689b42fb9a74e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-c, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T07:09:42.582 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:09:42 smithi174 podman[74463]: 2024-09-19 07:09:42.390004376 +0000 UTC m=+3.432594222 container cleanup e81cc5d6af587aee4aa9dd5330ec3c0d1daea70375b09e09c689b42fb9a74e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T07:09:42.583 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:09:42 smithi174 bash[74463]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-c 2024-09-19T07:09:42.839 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T07:09:42.876 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:09:42.876 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T07:09:42.877 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T07:09:42.877 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.f 2024-09-19T07:09:43.194 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:09:42 smithi174 systemd[1]: Stopping Ceph mds.f for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:09:43.582 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:09:43 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-f[65748]: 2024-09-19T07:09:43.193+0000 7f7969539640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:09:43.582 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:09:43 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-f[65748]: 2024-09-19T07:09:43.193+0000 7f7969539640 -1 mds.f *** got signal Terminated *** 2024-09-19T07:09:46.244 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:09:45 smithi174 podman[74596]: 2024-09-19 07:09:45.98920842 +0000 UTC m=+2.892681823 container died 93a672d45a6fbb5f0320b4e806b05000da0580be40faf1d5d2765541e139ad42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-f, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T07:09:46.583 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:09:46 smithi174 podman[74596]: 2024-09-19 07:09:46.351713559 +0000 UTC m=+3.255186951 container cleanup 93a672d45a6fbb5f0320b4e806b05000da0580be40faf1d5d2765541e139ad42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T07:09:46.583 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:09:46 smithi174 bash[74596]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-f 2024-09-19T07:09:46.745 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T07:09:46.779 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:09:46.779 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T07:09:46.779 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T07:09:46.779 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.i 2024-09-19T07:09:47.082 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:09:46 smithi174 systemd[1]: Stopping Ceph mds.i for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:09:47.583 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:09:47 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-i[66358]: 2024-09-19T07:09:47.088+0000 7f0d2819f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:09:47.583 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:09:47 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-i[66358]: 2024-09-19T07:09:47.088+0000 7f0d2819f640 -1 mds.i *** got signal Terminated *** 2024-09-19T07:09:50.256 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:09:49 smithi174 podman[74716]: 2024-09-19 07:09:49.987432622 +0000 UTC m=+2.990634039 container died 8fbbb96b41ec47e0fa8dcf6a38651e9d5b582831eaa938c95cafca920b5a04a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T07:09:50.582 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:09:50 smithi174 podman[74716]: 2024-09-19 07:09:50.396342861 +0000 UTC m=+3.399544278 container cleanup 8fbbb96b41ec47e0fa8dcf6a38651e9d5b582831eaa938c95cafca920b5a04a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-i, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T07:09:50.582 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:09:50 smithi174 bash[74716]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-i 2024-09-19T07:09:50.738 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T07:09:50.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:09:50.771 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T07:09:50.771 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T07:09:50.771 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.l 2024-09-19T07:09:51.332 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:09:50 smithi174 systemd[1]: Stopping Ceph mds.l for d158d934-7651-11ef-bceb-c7b262605968... 2024-09-19T07:09:51.333 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:09:51 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-l[65449]: 2024-09-19T07:09:51.083+0000 7f8b4a7e9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:09:51.333 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:09:51 smithi174 ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-l[65449]: 2024-09-19T07:09:51.083+0000 7f8b4a7e9640 -1 mds.l *** got signal Terminated *** 2024-09-19T07:09:54.250 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:09:53 smithi174 podman[74836]: 2024-09-19 07:09:53.985542664 +0000 UTC m=+2.994199525 container died ad715ddcc610065aea0786c64bec8447f2156b2275b819a930f34d4262267101 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:09:54.551 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:09:54 smithi174 podman[74836]: 2024-09-19 07:09:54.349881482 +0000 UTC m=+3.358538353 container cleanup ad715ddcc610065aea0786c64bec8447f2156b2275b819a930f34d4262267101 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T07:09:54.551 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:09:54 smithi174 bash[74836]: ceph-d158d934-7651-11ef-bceb-c7b262605968-mds-l 2024-09-19T07:09:54.774 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d158d934-7651-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T07:09:54.806 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:09:54.807 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T07:09:54.807 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d158d934-7651-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T07:09:55.007 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T07:09:57.402 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d158d934-7651-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T07:09:57.602 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T07:10:00.020 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d158d934-7651-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T07:10:00.223 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T07:10:02.628 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:10:02.662 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:10:02.695 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:10:02.723 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T07:10:02.728 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911754/remote/smithi103/crash 2024-09-19T07:10:02.729 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/crash -- . 2024-09-19T07:10:02.767 INFO:teuthology.orchestra.run.smithi103.stderr:tar: /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T07:10:02.767 INFO:teuthology.orchestra.run.smithi103.stderr:tar: Error is not recoverable: exiting now 2024-09-19T07:10:02.769 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911754/remote/smithi134/crash 2024-09-19T07:10:02.770 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/crash -- . 2024-09-19T07:10:02.847 INFO:teuthology.orchestra.run.smithi134.stderr:tar: /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T07:10:02.847 INFO:teuthology.orchestra.run.smithi134.stderr:tar: Error is not recoverable: exiting now 2024-09-19T07:10:02.849 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911754/remote/smithi174/crash 2024-09-19T07:10:02.850 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/crash -- . 2024-09-19T07:10:02.889 INFO:teuthology.orchestra.run.smithi174.stderr:tar: /var/lib/ceph/d158d934-7651-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T07:10:02.889 INFO:teuthology.orchestra.run.smithi174.stderr:tar: Error is not recoverable: exiting now 2024-09-19T07:10:02.890 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T07:10:02.890 DEBUG:teuthology.orchestra.run.smithi103:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T07:10:02.926 INFO:tasks.cephadm:Compressing logs... 2024-09-19T07:10:02.926 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T07:10:02.968 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T07:10:02.970 DEBUG:teuthology.orchestra.run.smithi174:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T07:10:02.989 INFO:teuthology.orchestra.run.smithi103.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T07:10:02.989 INFO:teuthology.orchestra.run.smithi103.stderr:: No such file or directory 2024-09-19T07:10:02.990 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-19T07:10:02.990 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T07:10:02.991 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T07:10:02.991 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.9.log: /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T07:10:02.991 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T07:10:02.992 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.audit.log: /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T07:10:02.992 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T07:10:02.994 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mon.a.log: 84.7% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T07:10:02.994 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T07:10:02.994 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.log 2024-09-19T07:10:02.994 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.g.log: /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.6.log: 90.4% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T07:10:02.996 INFO:teuthology.orchestra.run.smithi174.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T07:10:02.996 INFO:teuthology.orchestra.run.smithi174.stderr:: No such file or directory 2024-09-19T07:10:02.996 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T07:10:02.996 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T07:10:02.996 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T07:10:02.996 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T07:10:03.000 INFO:teuthology.orchestra.run.smithi134.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T07:10:03.000 INFO:teuthology.orchestra.run.smithi134.stderr:: No such file or directory 2024-09-19T07:10:03.000 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T07:10:03.000 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T07:10:03.000 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.10.log: /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T07:10:03.000 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T07:10:03.000 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T07:10:03.000 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T07:10:03.000 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T07:10:03.003 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T07:10:03.003 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mon.c.log: /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T07:10:03.003 INFO:teuthology.orchestra.run.smithi174.stderr: 82.3% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T07:10:03.003 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T07:10:03.003 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.log 2024-09-19T07:10:03.003 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T07:10:03.003 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.log: 90.5% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T07:10:03.003 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T07:10:03.004 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.l.log: 87.5% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T07:10:03.004 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T07:10:03.004 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T07:10:03.004 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.audit.log: 90.5% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T07:10:03.004 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T07:10:03.005 INFO:teuthology.orchestra.run.smithi103.stderr: 90.8% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T07:10:03.005 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.cephadm.log: 82.3% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T07:10:03.009 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T07:10:03.009 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T07:10:03.010 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.8.log: 90.9% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T07:10:03.010 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.log 2024-09-19T07:10:03.012 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.d.log: 87.5% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T07:10:03.013 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mgr.y.log: 90.9% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T07:10:03.015 INFO:teuthology.orchestra.run.smithi103.stderr: 94.6% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T07:10:03.016 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T07:10:03.016 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T07:10:03.017 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T07:10:03.017 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.log: 87.5% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T07:10:03.020 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T07:10:03.021 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T07:10:03.022 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.j.log: 94.7% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T07:10:03.022 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.h.log: 92.3% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T07:10:03.023 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T07:10:03.026 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T07:10:03.028 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T07:10:03.031 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mgr.z.log: 92.3% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T07:10:03.033 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T07:10:03.038 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T07:10:03.046 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T07:10:03.061 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T07:10:03.062 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T07:10:03.080 INFO:teuthology.orchestra.run.smithi103.stderr: 94.6% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T07:10:03.081 INFO:teuthology.orchestra.run.smithi103.stderr: 94.7% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T07:10:03.088 INFO:teuthology.orchestra.run.smithi174.stderr: 94.6% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T07:10:03.089 INFO:teuthology.orchestra.run.smithi134.stderr: 94.7% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T07:10:03.121 INFO:teuthology.orchestra.run.smithi134.stderr: 92.8% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T07:10:03.132 INFO:teuthology.orchestra.run.smithi174.stderr: 93.1% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T07:10:03.132 INFO:teuthology.orchestra.run.smithi134.stderr: 93.1% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T07:10:03.165 INFO:teuthology.orchestra.run.smithi174.stderr: 92.8% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T07:10:03.393 INFO:teuthology.orchestra.run.smithi174.stderr: 93.7% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T07:10:03.444 INFO:teuthology.orchestra.run.smithi174.stderr: 91.6% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T07:10:03.499 INFO:teuthology.orchestra.run.smithi103.stderr: 90.3% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T07:10:03.603 INFO:teuthology.orchestra.run.smithi134.stderr: 92.1% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T07:10:03.683 INFO:teuthology.orchestra.run.smithi174.stderr: 93.1% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T07:10:03.822 INFO:teuthology.orchestra.run.smithi134.stderr: 92.4% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T07:10:03.847 INFO:teuthology.orchestra.run.smithi103.stderr: 92.6% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T07:10:03.858 INFO:teuthology.orchestra.run.smithi134.stderr: 92.8% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T07:10:03.903 INFO:teuthology.orchestra.run.smithi103.stderr: 92.9% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T07:10:03.930 INFO:teuthology.orchestra.run.smithi174.stderr: 92.8% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T07:10:04.024 INFO:teuthology.orchestra.run.smithi174.stderr: 92.6% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T07:10:04.036 INFO:teuthology.orchestra.run.smithi103.stderr: 93.1% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T07:10:04.127 INFO:teuthology.orchestra.run.smithi134.stderr: 92.7% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T07:10:04.139 INFO:teuthology.orchestra.run.smithi103.stderr: 90.5% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T07:10:04.203 INFO:teuthology.orchestra.run.smithi103.stderr: 93.0% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T07:10:04.205 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-19T07:10:04.205 INFO:teuthology.orchestra.run.smithi103.stderr:real 0m1.225s 2024-09-19T07:10:04.205 INFO:teuthology.orchestra.run.smithi103.stderr:user 0m5.628s 2024-09-19T07:10:04.205 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m0.226s 2024-09-19T07:10:04.456 INFO:teuthology.orchestra.run.smithi174.stderr: 92.9% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T07:10:04.458 INFO:teuthology.orchestra.run.smithi174.stderr: 2024-09-19T07:10:04.458 INFO:teuthology.orchestra.run.smithi174.stderr:real 0m1.473s 2024-09-19T07:10:04.458 INFO:teuthology.orchestra.run.smithi174.stderr:user 0m5.070s 2024-09-19T07:10:04.458 INFO:teuthology.orchestra.run.smithi174.stderr:sys 0m0.187s 2024-09-19T07:10:04.557 INFO:teuthology.orchestra.run.smithi134.stderr: 93.3% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T07:10:05.116 INFO:teuthology.orchestra.run.smithi134.stderr: 93.0% -- replaced with /var/log/ceph/d158d934-7651-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T07:10:05.117 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-19T07:10:05.117 INFO:teuthology.orchestra.run.smithi134.stderr:real 0m2.132s 2024-09-19T07:10:05.118 INFO:teuthology.orchestra.run.smithi134.stderr:user 0m7.109s 2024-09-19T07:10:05.118 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m0.248s 2024-09-19T07:10:05.118 INFO:tasks.cephadm:Archiving logs... 2024-09-19T07:10:05.119 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911754/remote/smithi103/log 2024-09-19T07:10:05.120 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T07:10:05.922 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911754/remote/smithi134/log 2024-09-19T07:10:05.923 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T07:10:06.886 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911754/remote/smithi174/log 2024-09-19T07:10:06.887 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T07:10:07.600 INFO:tasks.cephadm:Removing cluster... 2024-09-19T07:10:07.600 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d158d934-7651-11ef-bceb-c7b262605968 --force 2024-09-19T07:10:07.805 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T07:10:08.132 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d158d934-7651-11ef-bceb-c7b262605968 --force 2024-09-19T07:10:08.333 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T07:10:08.599 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d158d934-7651-11ef-bceb-c7b262605968 --force 2024-09-19T07:10:08.799 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: d158d934-7651-11ef-bceb-c7b262605968 2024-09-19T07:10:09.060 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T07:10:09.060 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T07:10:09.079 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T07:10:09.096 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T07:10:09.112 INFO:tasks.cephadm:Teardown complete 2024-09-19T07:10:09.112 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T07:10:09.124 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T07:10:09.124 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T07:10:09.127 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T07:10:09.138 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T07:10:09.250 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T07:10:09.323 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T07:10:09.323 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:10:09.326 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:10:09.329 DEBUG:teuthology.orchestra.run.smithi174:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:10:09.341 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:10:09.341 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:10:09.343 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:10:09.345 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:10:09.345 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-19T07:10:09.345 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv01.front.sepia.ceph.com 3 6 377 41 +674us[ +669us] +/- 51ms 2024-09-19T07:10:09.345 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -1255us[-1260us] +/- 58ms 2024-09-19T07:10:09.345 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +492us[ +487us] +/- 75ms 2024-09-19T07:10:09.345 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:10:09.347 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:10:09.347 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-09-19T07:10:09.347 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 +661us[ +663us] +/- 51ms 2024-09-19T07:10:09.347 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -1283us[-1281us] +/- 58ms 2024-09-19T07:10:09.347 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 42 +490us[ +490us] +/- 75ms 2024-09-19T07:10:09.347 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:10:09.348 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:10:09.348 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-09-19T07:10:09.348 INFO:teuthology.orchestra.run.smithi174.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 +650us[ +641us] +/- 51ms 2024-09-19T07:10:09.348 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 46 -1302us[-1311us] +/- 58ms 2024-09-19T07:10:09.348 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +450us[ +442us] +/- 75ms 2024-09-19T07:10:09.348 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:10:09.349 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T07:10:09.361 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T07:10:09.362 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T07:10:09.374 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T07:10:09.482 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T07:10:09.549 DEBUG:teuthology.orchestra.run.smithi174:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T07:10:09.617 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:10:09.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:10:09.647 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:10:09.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:10:09.676 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:10:09.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:10:09.704 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T07:10:09.717 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T07:10:09.730 INFO:teuthology.task.internal:Duration was 2233.243881 seconds 2024-09-19T07:10:09.731 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T07:10:09.741 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T07:10:09.741 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T07:10:09.743 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T07:10:09.746 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T07:10:09.781 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:10:09.781 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:10:09.784 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:10:10.230 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T07:10:10.230 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T07:10:10.230 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T07:10:10.252 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-09-19T07:10:10.252 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T07:10:10.298 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi174.front.sepia.ceph.com 2024-09-19T07:10:10.298 DEBUG:teuthology.orchestra.run.smithi174:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T07:10:10.336 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T07:10:10.336 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T07:10:10.338 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T07:10:10.341 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T07:10:10.405 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T07:10:10.406 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T07:10:10.408 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T07:10:10.410 DEBUG:teuthology.orchestra.run.smithi174:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T07:10:12.308 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T07:10:12.322 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T07:10:12.322 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T07:10:12.347 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T07:10:12.378 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T07:10:12.405 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T07:10:12.416 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:10:12.418 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:10:12.421 DEBUG:teuthology.orchestra.run.smithi174:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:10:12.439 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2024-09-19T07:10:12.446 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-09-19T07:10:12.469 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = core 2024-09-19T07:10:12.522 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:10:12.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:10:12.537 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:10:12.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:10:12.692 DEBUG:teuthology.orchestra.run.smithi174:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:10:12.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:10:12.707 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T07:10:12.891 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T07:10:12.892 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911754/remote/smithi103 2024-09-19T07:10:12.892 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T07:10:13.658 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911754/remote/smithi134 2024-09-19T07:10:13.658 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T07:10:13.705 DEBUG:teuthology.misc:Transferring archived files from smithi174:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911754/remote/smithi174 2024-09-19T07:10:13.705 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T07:10:13.754 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T07:10:13.754 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T07:10:13.757 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T07:10:13.759 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T07:10:13.799 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T07:10:13.813 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T07:10:13.813 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T07:10:13.826 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T07:10:13.826 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T07:10:13.829 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T07:10:13.831 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T07:10:13.844 INFO:teuthology.orchestra.run.smithi103.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 07:10 /home/ubuntu/cephtest 2024-09-19T07:10:13.846 INFO:teuthology.orchestra.run.smithi134.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 07:10 /home/ubuntu/cephtest 2024-09-19T07:10:13.855 INFO:teuthology.orchestra.run.smithi174.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 07:10 /home/ubuntu/cephtest 2024-09-19T07:10:13.856 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T07:10:13.869 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T07:10:13.905 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 2233.24388051033 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T07:10:13.905 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T07:10:13.995 INFO:teuthology.run:pass